2024-04-20T11:16:23.509 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-20T11:16:23.509 DEBUG:teuthology.run:Teuthology command: teuthology --name rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi --description fs/volumes/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/misc}} --owner scheduled_rishabh@teuthology --archive /home/teuthworker/archive/rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7664528 --verbose -- /home/teuthworker/archive/rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7664528/orig.config.yaml 2024-04-20T11:16:23.558 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-20T11:16:23.639 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7664528 branch: wip-rishabh-testing-20240416.193735 description: fs/volumes/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/misc}} email: null first_in_suite: false job_id: '7664528' kernel: flavor: default kdb: true sha1: 806ec36b21129f113ce356c007dd59694d340b37 last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-rishabh-testing-20240416.193735 ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug finisher: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL sha1: a8c84f5997950fd7d008204839253291d52e6eda ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: a8c84f5997950fd7d008204839253291d52e6eda selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-testing-20240416.193735 sha1: a8c84f5997950fd7d008204839253291d52e6eda owner: scheduled_rishabh@teuthology priority: 50 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 7954 sha1: a8c84f5997950fd7d008204839253291d52e6eda sleep_before_teardown: 0 subset: 364/512 suite: fs suite_branch: wip-rishabh-testing-20240416.193735 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: a8c84f5997950fd7d008204839253291d52e6eda syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi152.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOVyVBdxmuG8HMwDEY6F4/Lr1b7DL56GYyxRXWbfXazQz/oSpmCDGlA7vu8AiwGwSHiEeadPLzoz3W+lgsRim50= smithi175.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDMTvBfTjexM8GK5wjqYosCBbUVUEKh7Cqxx6F7+ox2/x5WsWbP9nnMtiYAGheE3S14t5e++9VidTHs74dFQ13o= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - ceph-fuse: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_volumes.TestMisc teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-20_09:50:15 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-20T11:16:23.639 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa; will attempt to use it 2024-04-20T11:16:23.640 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks 2024-04-20T11:16:23.640 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-20T11:16:23.641 INFO:teuthology.task.internal:Checking packages... 2024-04-20T11:16:23.664 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'a8c84f5997950fd7d008204839253291d52e6eda' 2024-04-20T11:16:23.664 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-20T11:16:23.664 INFO:teuthology.packaging:ref: None 2024-04-20T11:16:23.664 INFO:teuthology.packaging:tag: None 2024-04-20T11:16:23.664 INFO:teuthology.packaging:branch: wip-rishabh-testing-20240416.193735 2024-04-20T11:16:23.664 INFO:teuthology.packaging:sha1: a8c84f5997950fd7d008204839253291d52e6eda 2024-04-20T11:16:23.664 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-rishabh-testing-20240416.193735 2024-04-20T11:16:23.977 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3133.ga8c84f59 2024-04-20T11:16:23.978 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-20T11:16:23.985 INFO:teuthology.task.internal:no buildpackages task found 2024-04-20T11:16:23.985 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-20T11:16:23.992 INFO:teuthology.task.internal:Saving configuration 2024-04-20T11:16:24.010 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-20T11:16:24.016 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-20T11:16:24.038 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi152.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7664528', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-20 11:11:18.838779', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOVyVBdxmuG8HMwDEY6F4/Lr1b7DL56GYyxRXWbfXazQz/oSpmCDGlA7vu8AiwGwSHiEeadPLzoz3W+lgsRim50='} 2024-04-20T11:16:24.057 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi175.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7664528', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-20 11:11:18.840722', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDMTvBfTjexM8GK5wjqYosCBbUVUEKh7Cqxx6F7+ox2/x5WsWbP9nnMtiYAGheE3S14t5e++9VidTHs74dFQ13o='} 2024-04-20T11:16:24.057 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-20T11:16:24.063 INFO:teuthology.task.internal:roles: ubuntu@smithi152.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-04-20T11:16:24.063 INFO:teuthology.task.internal:roles: ubuntu@smithi175.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-04-20T11:16:24.063 INFO:teuthology.run_tasks:Running task console_log... 2024-04-20T11:16:24.124 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f516553c160>, signals=[15]) 2024-04-20T11:16:24.124 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-20T11:16:24.131 INFO:teuthology.task.internal:Opening connections... 2024-04-20T11:16:24.131 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi152.front.sepia.ceph.com 2024-04-20T11:16:24.133 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi152.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-20T11:16:24.206 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi175.front.sepia.ceph.com 2024-04-20T11:16:24.207 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi175.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-20T11:16:24.281 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-20T11:16:24.288 DEBUG:teuthology.orchestra.run.smithi152:> uname -m 2024-04-20T11:16:24.305 INFO:teuthology.orchestra.run.smithi152.stdout:x86_64 2024-04-20T11:16:24.305 DEBUG:teuthology.orchestra.run.smithi152:> cat /etc/os-release 2024-04-20T11:16:24.362 INFO:teuthology.orchestra.run.smithi152.stdout:NAME="CentOS Stream" 2024-04-20T11:16:24.362 INFO:teuthology.orchestra.run.smithi152.stdout:VERSION="9" 2024-04-20T11:16:24.362 INFO:teuthology.orchestra.run.smithi152.stdout:ID="centos" 2024-04-20T11:16:24.362 INFO:teuthology.orchestra.run.smithi152.stdout:ID_LIKE="rhel fedora" 2024-04-20T11:16:24.362 INFO:teuthology.orchestra.run.smithi152.stdout:VERSION_ID="9" 2024-04-20T11:16:24.362 INFO:teuthology.orchestra.run.smithi152.stdout:PLATFORM_ID="platform:el9" 2024-04-20T11:16:24.362 INFO:teuthology.orchestra.run.smithi152.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-20T11:16:24.362 INFO:teuthology.orchestra.run.smithi152.stdout:ANSI_COLOR="0;31" 2024-04-20T11:16:24.363 INFO:teuthology.orchestra.run.smithi152.stdout:LOGO="fedora-logo-icon" 2024-04-20T11:16:24.363 INFO:teuthology.orchestra.run.smithi152.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-20T11:16:24.363 INFO:teuthology.orchestra.run.smithi152.stdout:HOME_URL="https://centos.org/" 2024-04-20T11:16:24.363 INFO:teuthology.orchestra.run.smithi152.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-20T11:16:24.363 INFO:teuthology.orchestra.run.smithi152.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-20T11:16:24.363 INFO:teuthology.orchestra.run.smithi152.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-20T11:16:24.363 INFO:teuthology.lock.ops:Updating smithi152.front.sepia.ceph.com on lock server 2024-04-20T11:16:24.385 DEBUG:teuthology.orchestra.run.smithi175:> uname -m 2024-04-20T11:16:24.403 INFO:teuthology.orchestra.run.smithi175.stdout:x86_64 2024-04-20T11:16:24.403 DEBUG:teuthology.orchestra.run.smithi175:> cat /etc/os-release 2024-04-20T11:16:24.460 INFO:teuthology.orchestra.run.smithi175.stdout:NAME="CentOS Stream" 2024-04-20T11:16:24.461 INFO:teuthology.orchestra.run.smithi175.stdout:VERSION="9" 2024-04-20T11:16:24.461 INFO:teuthology.orchestra.run.smithi175.stdout:ID="centos" 2024-04-20T11:16:24.461 INFO:teuthology.orchestra.run.smithi175.stdout:ID_LIKE="rhel fedora" 2024-04-20T11:16:24.461 INFO:teuthology.orchestra.run.smithi175.stdout:VERSION_ID="9" 2024-04-20T11:16:24.461 INFO:teuthology.orchestra.run.smithi175.stdout:PLATFORM_ID="platform:el9" 2024-04-20T11:16:24.461 INFO:teuthology.orchestra.run.smithi175.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-20T11:16:24.461 INFO:teuthology.orchestra.run.smithi175.stdout:ANSI_COLOR="0;31" 2024-04-20T11:16:24.461 INFO:teuthology.orchestra.run.smithi175.stdout:LOGO="fedora-logo-icon" 2024-04-20T11:16:24.461 INFO:teuthology.orchestra.run.smithi175.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-20T11:16:24.461 INFO:teuthology.orchestra.run.smithi175.stdout:HOME_URL="https://centos.org/" 2024-04-20T11:16:24.461 INFO:teuthology.orchestra.run.smithi175.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-20T11:16:24.461 INFO:teuthology.orchestra.run.smithi175.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-20T11:16:24.461 INFO:teuthology.orchestra.run.smithi175.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-20T11:16:24.462 INFO:teuthology.lock.ops:Updating smithi175.front.sepia.ceph.com on lock server 2024-04-20T11:16:24.485 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-20T11:16:24.494 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-20T11:16:24.499 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-20T11:16:24.499 DEBUG:teuthology.orchestra.run.smithi152:> test '!' -e /home/ubuntu/cephtest 2024-04-20T11:16:24.502 DEBUG:teuthology.orchestra.run.smithi175:> test '!' -e /home/ubuntu/cephtest 2024-04-20T11:16:24.518 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-20T11:16:24.524 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-20T11:16:24.524 DEBUG:teuthology.orchestra.run.smithi152:> test -z $(ls -A /var/lib/ceph) 2024-04-20T11:16:24.559 DEBUG:teuthology.orchestra.run.smithi175:> test -z $(ls -A /var/lib/ceph) 2024-04-20T11:16:24.754 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-20T11:16:24.812 INFO:teuthology.run_tasks:Running task kernel... 2024-04-20T11:16:24.826 INFO:teuthology.task.kernel:normalize config orig: {'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'} 2024-04-20T11:16:24.826 INFO:teuthology.task.kernel:config {'mon.a': {'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'}, 'mon.b': {'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'}}, timeout 300 2024-04-20T11:16:24.826 DEBUG:teuthology.orchestra.run.smithi152:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-20T11:16:24.827 DEBUG:teuthology.orchestra.run.smithi175:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-20T11:16:24.843 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T11:16:24.843 DEBUG:teuthology.task.kernel:sha1 for mon.b is 806ec36b21129f113ce356c007dd59694d340b37 2024-04-20T11:16:24.844 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-04-20T11:16:24.844 DEBUG:teuthology.orchestra.run.smithi175:> uname -r 2024-04-20T11:16:24.845 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T11:16:24.845 DEBUG:teuthology.task.kernel:sha1 for mon.a is 806ec36b21129f113ce356c007dd59694d340b37 2024-04-20T11:16:24.845 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-04-20T11:16:24.845 DEBUG:teuthology.orchestra.run.smithi152:> uname -r 2024-04-20T11:16:24.900 INFO:teuthology.orchestra.run.smithi175.stdout:5.14.0-437.el9.x86_64 2024-04-20T11:16:24.901 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs 806ec36b21129f113ce356c007dd59694d340b37 2024-04-20T11:16:24.901 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-20T11:16:24.901 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-20T11:16:24.905 INFO:teuthology.orchestra.run.smithi152.stdout:5.14.0-437.el9.x86_64 2024-04-20T11:16:24.905 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs 806ec36b21129f113ce356c007dd59694d340b37 2024-04-20T11:16:24.905 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-20T11:16:24.905 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-20T11:16:25.047 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum upgrade -y linux-firmware 2024-04-20T11:16:25.164 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum upgrade -y linux-firmware 2024-04-20T11:16:27.868 INFO:teuthology.orchestra.run.smithi152.stdout:CentOS Stream 9 - BaseOS 6.9 MB/s | 8.0 MB 00:01 2024-04-20T11:16:28.215 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 9 - BaseOS 5.3 MB/s | 8.0 MB 00:01 2024-04-20T11:16:32.319 INFO:teuthology.orchestra.run.smithi152.stdout:CentOS Stream 9 - AppStream 5.6 MB/s | 19 MB 00:03 2024-04-20T11:16:32.715 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 9 - AppStream 5.5 MB/s | 19 MB 00:03 2024-04-20T11:16:37.449 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 9 - CRB 5.6 MB/s | 6.2 MB 00:01 2024-04-20T11:16:38.444 INFO:teuthology.orchestra.run.smithi152.stdout:CentOS Stream 9 - CRB 2.4 MB/s | 6.2 MB 00:02 2024-04-20T11:16:39.108 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 9 - Extras packages 51 kB/s | 16 kB 00:00 2024-04-20T11:16:40.328 INFO:teuthology.orchestra.run.smithi152.stdout:CentOS Stream 9 - Extras packages 30 kB/s | 16 kB 00:00 2024-04-20T11:16:41.028 INFO:teuthology.orchestra.run.smithi175.stdout:Extra Packages for Enterprise Linux 12 MB/s | 21 MB 00:01 2024-04-20T11:16:42.207 INFO:teuthology.orchestra.run.smithi152.stdout:Extra Packages for Enterprise Linux 12 MB/s | 21 MB 00:01 2024-04-20T11:16:46.134 INFO:teuthology.orchestra.run.smithi175.stdout:lab-extras 17 kB/s | 1.7 kB 00:00 2024-04-20T11:16:47.280 INFO:teuthology.orchestra.run.smithi152.stdout:lab-extras 23 kB/s | 1.7 kB 00:00 2024-04-20T11:16:48.153 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2024-04-20T11:16:48.154 INFO:teuthology.orchestra.run.smithi175.stdout:Nothing to do. 2024-04-20T11:16:48.154 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2024-04-20T11:16:48.357 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on mon.b... 2024-04-20T11:16:48.357 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-20T11:16:48.502 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ 2024-04-20T11:16:48.503 DEBUG:teuthology.orchestra.run.smithi175:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-20T11:16:48.505 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi175.front.sepia.ceph.com to complete... 2024-04-20T11:16:49.312 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2024-04-20T11:16:49.313 INFO:teuthology.orchestra.run.smithi152.stdout:Nothing to do. 2024-04-20T11:16:49.313 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2024-04-20T11:16:49.432 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on mon.a... 2024-04-20T11:16:49.433 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-20T11:16:49.579 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ 2024-04-20T11:16:49.579 DEBUG:teuthology.orchestra.run.smithi152:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-20T11:16:49.582 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi152.front.sepia.ceph.com to complete... 2024-04-20T11:17:04.733 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20 11:17:04 URL:https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm [357367792/357367792] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-20T11:17:04.733 INFO:teuthology.orchestra.run.smithi175.stderr:FINISHED --2024-04-20 11:17:04-- 2024-04-20T11:17:04.733 INFO:teuthology.orchestra.run.smithi175.stderr:Total wall clock time: 16s 2024-04-20T11:17:04.733 INFO:teuthology.orchestra.run.smithi175.stderr:Downloaded: 1 files, 341M in 16s (21.4 MB/s) 2024-04-20T11:17:04.736 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on mon.b... 2024-04-20T11:17:04.736 DEBUG:teuthology.orchestra.run.smithi175:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-20T11:17:06.844 INFO:teuthology.orchestra.run.smithi175.stdout:Verifying... ######################################## 2024-04-20T11:17:06.988 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing... ######################################## 2024-04-20T11:17:07.101 INFO:teuthology.orchestra.run.smithi175.stdout:Updating / installing... 2024-04-20T11:17:15.114 INFO:teuthology.orchestra.run.smithi175.stdout:kernel-6.9.0_rc3_g806ec36b2112-1 ######################################## 2024-04-20T11:17:20.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20 11:17:20 URL:https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm [357367792/357367792] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-20T11:17:20.638 INFO:teuthology.orchestra.run.smithi152.stderr:FINISHED --2024-04-20 11:17:20-- 2024-04-20T11:17:20.638 INFO:teuthology.orchestra.run.smithi152.stderr:Total wall clock time: 31s 2024-04-20T11:17:20.638 INFO:teuthology.orchestra.run.smithi152.stderr:Downloaded: 1 files, 341M in 30s (11.2 MB/s) 2024-04-20T11:17:20.640 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on mon.a... 2024-04-20T11:17:20.640 DEBUG:teuthology.orchestra.run.smithi152:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-20T11:17:22.742 INFO:teuthology.orchestra.run.smithi152.stdout:Verifying... ######################################## 2024-04-20T11:17:23.731 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing... ######################################## 2024-04-20T11:17:24.085 INFO:teuthology.orchestra.run.smithi152.stdout:Updating / installing... 2024-04-20T11:17:32.102 INFO:teuthology.orchestra.run.smithi152.stdout:kernel-6.9.0_rc3_g806ec36b2112-1 ######################################## 2024-04-20T11:17:54.331 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi175.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-20T11:17:54.332 DEBUG:teuthology.orchestra.run.smithi175:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-20T11:17:54.424 INFO:teuthology.orchestra.run.smithi175.stdout:/boot/System.map-6.9.0-rc3-g806ec36b2112 2024-04-20T11:17:54.424 INFO:teuthology.orchestra.run.smithi175.stdout:/boot/config-6.9.0-rc3-g806ec36b2112 2024-04-20T11:17:54.424 INFO:teuthology.orchestra.run.smithi175.stdout:/boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-20T11:17:54.424 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/System.map 2024-04-20T11:17:54.424 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/config 2024-04-20T11:17:54.425 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel 2024-04-20T11:17:54.425 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch 2024-04-20T11:17:54.425 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86 2024-04-20T11:17:54.425 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto 2024-04-20T11:17:54.425 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-20T11:17:54.425 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-20T11:17:54.425 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-20T11:17:54.425 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-20T11:17:54.425 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-20T11:17:54.425 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-20T11:17:54.425 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-20T11:17:54.425 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-20T11:17:54.425 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-20T11:17:54.425 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-20T11:17:54.426 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-20T11:17:54.426 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-20T11:17:54.426 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-20T11:17:54.426 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-20T11:17:54.426 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-20T11:17:54.426 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-20T11:17:54.426 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-20T11:17:54.426 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-20T11:17:54.426 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-20T11:17:54.426 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-20T11:17:54.426 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel 2024-04-20T11:17:54.426 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu 2024-04-20T11:17:54.426 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce 2024-04-20T11:17:54.426 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-20T11:17:54.427 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm 2024-04-20T11:17:54.427 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-20T11:17:54.427 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-20T11:17:54.427 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-04-20T11:17:54.427 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto 2024-04-20T11:17:54.427 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-04-20T11:17:54.427 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-04-20T11:17:54.427 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-04-20T11:17:54.427 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx 2024-04-20T11:17:54.427 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-04-20T11:17:54.427 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-04-20T11:17:54.427 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-20T11:17:54.427 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-04-20T11:17:54.428 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-04-20T11:17:54.428 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-04-20T11:17:54.428 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-04-20T11:17:54.428 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-04-20T11:17:54.428 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-04-20T11:17:54.428 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-04-20T11:17:54.428 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-04-20T11:17:54.428 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-04-20T11:17:54.428 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-04-20T11:17:54.428 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-04-20T11:17:54.428 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-04-20T11:17:54.428 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-04-20T11:17:54.428 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-04-20T11:17:54.429 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-04-20T11:17:54.429 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-04-20T11:17:54.429 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-04-20T11:17:54.429 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-04-20T11:17:54.429 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-04-20T11:17:54.429 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-04-20T11:17:54.429 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-04-20T11:17:54.429 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-04-20T11:17:54.429 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-04-20T11:17:54.429 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-04-20T11:17:54.429 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-04-20T11:17:54.429 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-04-20T11:17:54.429 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-04-20T11:17:54.429 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-04-20T11:17:54.429 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-04-20T11:17:54.430 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-04-20T11:17:54.430 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-04-20T11:17:54.430 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-04-20T11:17:54.430 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-04-20T11:17:54.430 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-04-20T11:17:54.430 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-04-20T11:17:54.430 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-04-20T11:17:54.430 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-04-20T11:17:54.430 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-04-20T11:17:54.430 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-04-20T11:17:54.430 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-04-20T11:17:54.430 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-04-20T11:17:54.430 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-04-20T11:17:54.431 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers 2024-04-20T11:17:54.431 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi 2024-04-20T11:17:54.431 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-20T11:17:54.431 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-04-20T11:17:54.431 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-04-20T11:17:54.431 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-04-20T11:17:54.431 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-04-20T11:17:54.431 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-04-20T11:17:54.431 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-04-20T11:17:54.431 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata 2024-04-20T11:17:54.431 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-04-20T11:17:54.431 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-04-20T11:17:54.431 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ata_generic.ko 2024-04-20T11:17:54.431 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-04-20T11:17:54.432 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-04-20T11:17:54.432 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-04-20T11:17:54.432 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-04-20T11:17:54.432 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-04-20T11:17:54.432 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-04-20T11:17:54.432 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-04-20T11:17:54.432 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-04-20T11:17:54.432 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-04-20T11:17:54.432 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-04-20T11:17:54.432 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-04-20T11:17:54.432 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-04-20T11:17:54.432 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-04-20T11:17:54.432 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-20T11:17:54.432 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-20T11:17:54.432 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-04-20T11:17:54.433 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-04-20T11:17:54.433 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-04-20T11:17:54.433 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-04-20T11:17:54.433 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-04-20T11:17:54.433 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-04-20T11:17:54.433 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-04-20T11:17:54.433 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-04-20T11:17:54.433 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-04-20T11:17:54.433 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-04-20T11:17:54.433 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-04-20T11:17:54.433 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-04-20T11:17:54.433 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-04-20T11:17:54.433 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-20T11:17:54.433 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-20T11:17:54.433 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-04-20T11:17:54.434 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-04-20T11:17:54.434 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-04-20T11:17:54.434 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-04-20T11:17:54.434 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-04-20T11:17:54.434 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sis.ko 2024-04-20T11:17:54.434 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-04-20T11:17:54.434 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-04-20T11:17:54.434 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-04-20T11:17:54.434 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-04-20T11:17:54.434 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-04-20T11:17:54.434 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-04-20T11:17:54.434 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-04-20T11:17:54.434 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-04-20T11:17:54.434 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-04-20T11:17:54.435 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-04-20T11:17:54.435 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-04-20T11:17:54.435 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-04-20T11:17:54.435 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-04-20T11:17:54.435 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-04-20T11:17:54.435 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-04-20T11:17:54.435 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-04-20T11:17:54.435 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-04-20T11:17:54.435 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay 2024-04-20T11:17:54.435 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-04-20T11:17:54.435 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base 2024-04-20T11:17:54.435 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap 2024-04-20T11:17:54.435 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-20T11:17:54.435 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-20T11:17:54.436 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma 2024-04-20T11:17:54.436 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-04-20T11:17:54.436 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block 2024-04-20T11:17:54.436 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe 2024-04-20T11:17:54.436 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-04-20T11:17:54.436 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/brd.ko 2024-04-20T11:17:54.436 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd 2024-04-20T11:17:54.436 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-04-20T11:17:54.436 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-04-20T11:17:54.436 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/loop.ko 2024-04-20T11:17:54.436 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx 2024-04-20T11:17:54.436 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-20T11:17:54.436 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-04-20T11:17:54.436 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk 2024-04-20T11:17:54.437 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-04-20T11:17:54.437 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-04-20T11:17:54.437 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-04-20T11:17:54.437 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/virtio_blk.ko 2024-04-20T11:17:54.437 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback 2024-04-20T11:17:54.437 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-20T11:17:54.437 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkfront.ko 2024-04-20T11:17:54.437 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram 2024-04-20T11:17:54.437 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-04-20T11:17:54.437 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char 2024-04-20T11:17:54.437 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-04-20T11:17:54.437 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random 2024-04-20T11:17:54.437 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-20T11:17:54.437 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-20T11:17:54.438 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-20T11:17:54.438 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-04-20T11:17:54.438 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-20T11:17:54.438 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi 2024-04-20T11:17:54.438 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-20T11:17:54.438 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-20T11:17:54.438 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-20T11:17:54.438 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-20T11:17:54.438 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-20T11:17:54.438 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-20T11:17:54.438 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-04-20T11:17:54.438 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave 2024-04-20T11:17:54.438 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-04-20T11:17:54.438 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-04-20T11:17:54.439 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-04-20T11:17:54.439 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm 2024-04-20T11:17:54.439 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm.ko 2024-04-20T11:17:54.439 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-20T11:17:54.439 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-20T11:17:54.439 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-20T11:17:54.439 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-20T11:17:54.439 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-20T11:17:54.439 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/uv_mmtimer.ko 2024-04-20T11:17:54.439 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/virtio_console.ko 2024-04-20T11:17:54.439 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq 2024-04-20T11:17:54.439 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-20T11:17:54.439 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-20T11:17:54.439 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-20T11:17:54.439 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-20T11:17:54.440 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-20T11:17:54.440 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-20T11:17:54.440 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto 2024-04-20T11:17:54.440 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp 2024-04-20T11:17:54.440 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-20T11:17:54.440 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-04-20T11:17:54.440 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel 2024-04-20T11:17:54.440 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat 2024-04-20T11:17:54.440 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common 2024-04-20T11:17:54.440 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-20T11:17:54.440 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-20T11:17:54.440 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-20T11:17:54.440 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-04-20T11:17:54.441 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-04-20T11:17:54.441 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio 2024-04-20T11:17:54.441 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-20T11:17:54.441 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca 2024-04-20T11:17:54.441 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-04-20T11:17:54.441 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq 2024-04-20T11:17:54.441 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-20T11:17:54.441 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma 2024-04-20T11:17:54.441 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw 2024-04-20T11:17:54.441 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-20T11:17:54.441 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat 2024-04-20T11:17:54.441 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-20T11:17:54.441 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac 2024-04-20T11:17:54.442 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-04-20T11:17:54.442 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-04-20T11:17:54.442 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-04-20T11:17:54.442 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-04-20T11:17:54.442 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-04-20T11:17:54.442 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-04-20T11:17:54.442 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-04-20T11:17:54.442 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-04-20T11:17:54.442 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-04-20T11:17:54.442 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-04-20T11:17:54.442 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-04-20T11:17:54.442 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-04-20T11:17:54.442 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-04-20T11:17:54.442 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware 2024-04-20T11:17:54.443 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/edd.ko 2024-04-20T11:17:54.443 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-20T11:17:54.443 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio 2024-04-20T11:17:54.443 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-04-20T11:17:54.443 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-20T11:17:54.443 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid 2024-04-20T11:17:54.443 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-04-20T11:17:54.443 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-04-20T11:17:54.443 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-04-20T11:17:54.443 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-04-20T11:17:54.443 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-04-20T11:17:54.443 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-04-20T11:17:54.443 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-04-20T11:17:54.444 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-04-20T11:17:54.444 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-04-20T11:17:54.444 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-04-20T11:17:54.444 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-20T11:17:54.444 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-20T11:17:54.444 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-04-20T11:17:54.444 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-04-20T11:17:54.444 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-04-20T11:17:54.444 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-04-20T11:17:54.444 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-04-20T11:17:54.444 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-04-20T11:17:54.444 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-04-20T11:17:54.444 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-04-20T11:17:54.444 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-20T11:17:54.445 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-20T11:17:54.445 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-04-20T11:17:54.445 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-04-20T11:17:54.445 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-04-20T11:17:54.445 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-04-20T11:17:54.445 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-04-20T11:17:54.445 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-04-20T11:17:54.445 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-04-20T11:17:54.445 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-04-20T11:17:54.445 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-04-20T11:17:54.445 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-20T11:17:54.445 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-04-20T11:17:54.445 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-20T11:17:54.445 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-20T11:17:54.446 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-20T11:17:54.446 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-20T11:17:54.446 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-20T11:17:54.446 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-20T11:17:54.446 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-20T11:17:54.446 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-20T11:17:54.446 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-20T11:17:54.446 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-04-20T11:17:54.446 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-04-20T11:17:54.446 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-04-20T11:17:54.446 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-20T11:17:54.446 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-04-20T11:17:54.446 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-04-20T11:17:54.447 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-04-20T11:17:54.447 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-04-20T11:17:54.447 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-04-20T11:17:54.447 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-20T11:17:54.447 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-04-20T11:17:54.447 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-04-20T11:17:54.447 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-04-20T11:17:54.447 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-04-20T11:17:54.447 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-04-20T11:17:54.447 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-04-20T11:17:54.447 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-04-20T11:17:54.447 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-04-20T11:17:54.447 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-04-20T11:17:54.447 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-04-20T11:17:54.447 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-04-20T11:17:54.448 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-04-20T11:17:54.448 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv 2024-04-20T11:17:54.448 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-04-20T11:17:54.448 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-04-20T11:17:54.448 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-04-20T11:17:54.448 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon 2024-04-20T11:17:54.448 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-04-20T11:17:54.448 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-04-20T11:17:54.448 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-20T11:17:54.448 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-04-20T11:17:54.448 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-04-20T11:17:54.448 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-04-20T11:17:54.448 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-04-20T11:17:54.448 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-04-20T11:17:54.448 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-04-20T11:17:54.449 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-04-20T11:17:54.449 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-04-20T11:17:54.449 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-04-20T11:17:54.449 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-04-20T11:17:54.449 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-04-20T11:17:54.449 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-04-20T11:17:54.449 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-04-20T11:17:54.449 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-04-20T11:17:54.449 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-04-20T11:17:54.449 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-04-20T11:17:54.449 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-04-20T11:17:54.449 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-04-20T11:17:54.449 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-04-20T11:17:54.450 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-20T11:17:54.450 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-04-20T11:17:54.450 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-04-20T11:17:54.450 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-20T11:17:54.450 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-04-20T11:17:54.450 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-04-20T11:17:54.451 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-04-20T11:17:54.451 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-04-20T11:17:54.451 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-04-20T11:17:54.451 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-04-20T11:17:54.451 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-04-20T11:17:54.451 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-04-20T11:17:54.451 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-04-20T11:17:54.451 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-04-20T11:17:54.451 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-04-20T11:17:54.451 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-04-20T11:17:54.451 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-04-20T11:17:54.451 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-04-20T11:17:54.451 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-20T11:17:54.452 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-04-20T11:17:54.452 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-04-20T11:17:54.452 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-04-20T11:17:54.452 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-04-20T11:17:54.452 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-04-20T11:17:54.452 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-04-20T11:17:54.452 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-04-20T11:17:54.452 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-04-20T11:17:54.452 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-04-20T11:17:54.452 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-04-20T11:17:54.452 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-04-20T11:17:54.452 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-04-20T11:17:54.452 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-04-20T11:17:54.452 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-04-20T11:17:54.453 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-04-20T11:17:54.453 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-04-20T11:17:54.453 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-04-20T11:17:54.453 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-04-20T11:17:54.453 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-04-20T11:17:54.453 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-04-20T11:17:54.453 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-04-20T11:17:54.453 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-04-20T11:17:54.453 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-04-20T11:17:54.453 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-04-20T11:17:54.453 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-04-20T11:17:54.453 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-04-20T11:17:54.453 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-04-20T11:17:54.453 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-04-20T11:17:54.453 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-04-20T11:17:54.454 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-04-20T11:17:54.454 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-04-20T11:17:54.454 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-04-20T11:17:54.454 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-04-20T11:17:54.454 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-04-20T11:17:54.454 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-04-20T11:17:54.454 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-04-20T11:17:54.454 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-04-20T11:17:54.454 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-04-20T11:17:54.454 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-04-20T11:17:54.454 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-04-20T11:17:54.454 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-04-20T11:17:54.454 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-04-20T11:17:54.454 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-04-20T11:17:54.455 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-04-20T11:17:54.455 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-04-20T11:17:54.455 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-04-20T11:17:54.455 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-04-20T11:17:54.455 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus 2024-04-20T11:17:54.455 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-20T11:17:54.455 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-20T11:17:54.455 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-20T11:17:54.455 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-20T11:17:54.455 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-20T11:17:54.455 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-20T11:17:54.455 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-20T11:17:54.455 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-20T11:17:54.455 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-20T11:17:54.456 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-20T11:17:54.456 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-20T11:17:54.456 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-20T11:17:54.456 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-04-20T11:17:54.456 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-04-20T11:17:54.456 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-04-20T11:17:54.456 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-20T11:17:54.456 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-04-20T11:17:54.456 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-04-20T11:17:54.456 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-04-20T11:17:54.456 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-04-20T11:17:54.456 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-04-20T11:17:54.456 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-04-20T11:17:54.456 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-04-20T11:17:54.457 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-04-20T11:17:54.457 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-04-20T11:17:54.457 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-04-20T11:17:54.457 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-04-20T11:17:54.457 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-04-20T11:17:54.457 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-04-20T11:17:54.457 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-04-20T11:17:54.457 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-04-20T11:17:54.457 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-04-20T11:17:54.457 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-04-20T11:17:54.457 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-04-20T11:17:54.457 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-04-20T11:17:54.457 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-04-20T11:17:54.457 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-04-20T11:17:54.458 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-04-20T11:17:54.458 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-04-20T11:17:54.458 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-04-20T11:17:54.458 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-04-20T11:17:54.458 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c 2024-04-20T11:17:54.458 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos 2024-04-20T11:17:54.458 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-20T11:17:54.458 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-20T11:17:54.458 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses 2024-04-20T11:17:54.458 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-20T11:17:54.458 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-20T11:17:54.458 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-20T11:17:54.458 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-20T11:17:54.458 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-20T11:17:54.459 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-20T11:17:54.459 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-20T11:17:54.459 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-20T11:17:54.459 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-20T11:17:54.459 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-20T11:17:54.459 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-20T11:17:54.459 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-20T11:17:54.459 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-20T11:17:54.459 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-20T11:17:54.459 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-20T11:17:54.459 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-20T11:17:54.460 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-20T11:17:54.460 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-20T11:17:54.460 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-20T11:17:54.460 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-20T11:17:54.460 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-20T11:17:54.460 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-20T11:17:54.460 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-20T11:17:54.460 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-dev.ko 2024-04-20T11:17:54.460 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-04-20T11:17:54.460 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-20T11:17:54.460 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-04-20T11:17:54.460 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-04-20T11:17:54.460 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband 2024-04-20T11:17:54.460 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core 2024-04-20T11:17:54.461 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-20T11:17:54.461 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-04-20T11:17:54.461 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-20T11:17:54.461 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-20T11:17:54.461 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-20T11:17:54.461 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-20T11:17:54.461 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-20T11:17:54.461 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw 2024-04-20T11:17:54.461 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4 2024-04-20T11:17:54.461 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-20T11:17:54.461 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4 2024-04-20T11:17:54.461 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-20T11:17:54.461 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5 2024-04-20T11:17:54.461 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-20T11:17:54.462 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca 2024-04-20T11:17:54.462 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-20T11:17:54.462 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp 2024-04-20T11:17:54.462 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib 2024-04-20T11:17:54.462 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-20T11:17:54.462 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser 2024-04-20T11:17:54.462 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-20T11:17:54.462 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert 2024-04-20T11:17:54.462 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-20T11:17:54.462 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp 2024-04-20T11:17:54.462 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-20T11:17:54.462 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt 2024-04-20T11:17:54.462 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-20T11:17:54.462 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input 2024-04-20T11:17:54.463 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport 2024-04-20T11:17:54.463 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-20T11:17:54.463 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-20T11:17:54.463 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-04-20T11:17:54.463 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-04-20T11:17:54.463 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard 2024-04-20T11:17:54.463 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-20T11:17:54.463 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse 2024-04-20T11:17:54.463 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-04-20T11:17:54.463 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-04-20T11:17:54.463 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-04-20T11:17:54.463 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-20T11:17:54.463 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-04-20T11:17:54.463 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-20T11:17:54.464 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-20T11:17:54.464 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-20T11:17:54.464 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4 2024-04-20T11:17:54.464 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-20T11:17:54.464 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio 2024-04-20T11:17:54.464 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-04-20T11:17:54.464 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-04-20T11:17:54.464 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-20T11:17:54.464 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-04-20T11:17:54.464 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-04-20T11:17:54.464 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds 2024-04-20T11:17:54.464 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-04-20T11:17:54.464 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-04-20T11:17:54.464 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-04-20T11:17:54.464 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-04-20T11:17:54.465 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger 2024-04-20T11:17:54.465 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-20T11:17:54.465 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-20T11:17:54.465 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-20T11:17:54.465 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-20T11:17:54.465 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-20T11:17:54.465 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-20T11:17:54.465 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-20T11:17:54.465 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-20T11:17:54.465 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-20T11:17:54.465 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md 2024-04-20T11:17:54.465 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache 2024-04-20T11:17:54.465 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-04-20T11:17:54.465 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-04-20T11:17:54.465 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-04-20T11:17:54.466 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-04-20T11:17:54.466 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-04-20T11:17:54.466 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-04-20T11:17:54.466 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-04-20T11:17:54.466 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-04-20T11:17:54.466 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-04-20T11:17:54.466 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-04-20T11:17:54.466 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-04-20T11:17:54.466 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-04-20T11:17:54.466 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-04-20T11:17:54.466 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-04-20T11:17:54.466 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-04-20T11:17:54.466 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-04-20T11:17:54.467 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data 2024-04-20T11:17:54.467 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-20T11:17:54.467 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-04-20T11:17:54.467 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-04-20T11:17:54.467 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-04-20T11:17:54.467 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-04-20T11:17:54.467 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message 2024-04-20T11:17:54.467 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion 2024-04-20T11:17:54.467 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-04-20T11:17:54.467 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-04-20T11:17:54.467 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-04-20T11:17:54.467 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-04-20T11:17:54.467 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-04-20T11:17:54.467 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-04-20T11:17:54.468 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-04-20T11:17:54.468 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd 2024-04-20T11:17:54.468 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-04-20T11:17:54.468 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-04-20T11:17:54.468 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mfd-core.ko 2024-04-20T11:17:54.468 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-04-20T11:17:54.468 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-04-20T11:17:54.468 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-04-20T11:17:54.468 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-04-20T11:17:54.468 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc 2024-04-20T11:17:54.468 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl 2024-04-20T11:17:54.468 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-20T11:17:54.468 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-04-20T11:17:54.468 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-04-20T11:17:54.469 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-04-20T11:17:54.469 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710 2024-04-20T11:17:54.469 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-04-20T11:17:54.469 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom 2024-04-20T11:17:54.469 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-04-20T11:17:54.469 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-20T11:17:54.469 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-04-20T11:17:54.469 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-04-20T11:17:54.469 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-04-20T11:17:54.469 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-04-20T11:17:54.469 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-04-20T11:17:54.469 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d 2024-04-20T11:17:54.469 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-20T11:17:54.469 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-20T11:17:54.469 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei 2024-04-20T11:17:54.470 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-04-20T11:17:54.470 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-04-20T11:17:54.470 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-04-20T11:17:54.470 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru 2024-04-20T11:17:54.470 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-20T11:17:54.470 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp 2024-04-20T11:17:54.470 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-20T11:17:54.470 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-20T11:17:54.470 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-20T11:17:54.470 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-04-20T11:17:54.470 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-04-20T11:17:54.470 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-04-20T11:17:54.470 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-04-20T11:17:54.470 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci 2024-04-20T11:17:54.471 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-20T11:17:54.471 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net 2024-04-20T11:17:54.471 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding 2024-04-20T11:17:54.471 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-04-20T11:17:54.471 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa 2024-04-20T11:17:54.471 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53 2024-04-20T11:17:54.471 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-20T11:17:54.471 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-20T11:17:54.471 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-20T11:17:54.471 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx 2024-04-20T11:17:54.471 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-20T11:17:54.471 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-04-20T11:17:54.471 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-04-20T11:17:54.471 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet 2024-04-20T11:17:54.472 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com 2024-04-20T11:17:54.472 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-20T11:17:54.472 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-20T11:17:54.472 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-20T11:17:54.472 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-20T11:17:54.472 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390 2024-04-20T11:17:54.472 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-20T11:17:54.472 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-20T11:17:54.472 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-20T11:17:54.472 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-20T11:17:54.472 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec 2024-04-20T11:17:54.472 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-20T11:17:54.472 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere 2024-04-20T11:17:54.472 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-20T11:17:54.472 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon 2024-04-20T11:17:54.473 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-20T11:17:54.473 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera 2024-04-20T11:17:54.473 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-20T11:17:54.473 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd 2024-04-20T11:17:54.473 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-20T11:17:54.473 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-20T11:17:54.473 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-20T11:17:54.473 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros 2024-04-20T11:17:54.473 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx 2024-04-20T11:17:54.473 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-20T11:17:54.473 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-20T11:17:54.473 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-20T11:17:54.473 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-20T11:17:54.473 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-20T11:17:54.474 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx 2024-04-20T11:17:54.474 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-20T11:17:54.474 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-20T11:17:54.474 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom 2024-04-20T11:17:54.474 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-20T11:17:54.474 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-20T11:17:54.474 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-20T11:17:54.474 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-20T11:17:54.474 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-20T11:17:54.474 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet 2024-04-20T11:17:54.474 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-20T11:17:54.474 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-20T11:17:54.474 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade 2024-04-20T11:17:54.475 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna 2024-04-20T11:17:54.475 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-20T11:17:54.475 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio 2024-04-20T11:17:54.475 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-20T11:17:54.475 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-20T11:17:54.475 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-20T11:17:54.475 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-20T11:17:54.475 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-20T11:17:54.475 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-20T11:17:54.475 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-20T11:17:54.475 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-20T11:17:54.475 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-20T11:17:54.475 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-20T11:17:54.475 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco 2024-04-20T11:17:54.476 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic 2024-04-20T11:17:54.476 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-20T11:17:54.476 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec 2024-04-20T11:17:54.476 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip 2024-04-20T11:17:54.476 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-20T11:17:54.476 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-20T11:17:54.476 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-20T11:17:54.476 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-20T11:17:54.476 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-20T11:17:54.476 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-20T11:17:54.476 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink 2024-04-20T11:17:54.476 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-20T11:17:54.476 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-20T11:17:54.476 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-04-20T11:17:54.476 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex 2024-04-20T11:17:54.477 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet 2024-04-20T11:17:54.477 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-20T11:17:54.477 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-04-20T11:17:54.477 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-04-20T11:17:54.477 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel 2024-04-20T11:17:54.477 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-20T11:17:54.477 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000 2024-04-20T11:17:54.477 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-20T11:17:54.477 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e 2024-04-20T11:17:54.477 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-20T11:17:54.477 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k 2024-04-20T11:17:54.477 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-20T11:17:54.477 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e 2024-04-20T11:17:54.477 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-20T11:17:54.478 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf 2024-04-20T11:17:54.478 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-20T11:17:54.478 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb 2024-04-20T11:17:54.478 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-20T11:17:54.478 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf 2024-04-20T11:17:54.478 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-20T11:17:54.478 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-20T11:17:54.478 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-20T11:17:54.478 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-20T11:17:54.478 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-20T11:17:54.478 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-04-20T11:17:54.478 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell 2024-04-20T11:17:54.478 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-20T11:17:54.478 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-20T11:17:54.479 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-20T11:17:54.479 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox 2024-04-20T11:17:54.479 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-20T11:17:54.479 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-20T11:17:54.479 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-20T11:17:54.479 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-20T11:17:54.479 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-20T11:17:54.479 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-20T11:17:54.479 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel 2024-04-20T11:17:54.479 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-20T11:17:54.479 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom 2024-04-20T11:17:54.479 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-20T11:17:54.479 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-20T11:17:54.479 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi 2024-04-20T11:17:54.479 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-20T11:17:54.479 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-20T11:17:54.480 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion 2024-04-20T11:17:54.480 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-20T11:17:54.480 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia 2024-04-20T11:17:54.480 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-20T11:17:54.480 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines 2024-04-20T11:17:54.480 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-20T11:17:54.480 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-20T11:17:54.480 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic 2024-04-20T11:17:54.480 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-20T11:17:54.480 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-20T11:17:54.480 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-20T11:17:54.480 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-20T11:17:54.480 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-20T11:17:54.480 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc 2024-04-20T11:17:54.481 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-20T11:17:54.481 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek 2024-04-20T11:17:54.481 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-20T11:17:54.481 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-20T11:17:54.481 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-20T11:17:54.481 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-20T11:17:54.481 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker 2024-04-20T11:17:54.481 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-20T11:17:54.481 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc 2024-04-20T11:17:54.481 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-20T11:17:54.481 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan 2024-04-20T11:17:54.481 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-20T11:17:54.481 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis 2024-04-20T11:17:54.481 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-20T11:17:54.481 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-20T11:17:54.482 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc 2024-04-20T11:17:54.482 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-20T11:17:54.482 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-20T11:17:54.482 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-20T11:17:54.482 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-20T11:17:54.482 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro 2024-04-20T11:17:54.482 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-20T11:17:54.482 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-20T11:17:54.482 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-20T11:17:54.482 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun 2024-04-20T11:17:54.482 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-20T11:17:54.482 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-20T11:17:54.482 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-20T11:17:54.482 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-20T11:17:54.483 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti 2024-04-20T11:17:54.483 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-20T11:17:54.483 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti 2024-04-20T11:17:54.483 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-20T11:17:54.483 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via 2024-04-20T11:17:54.483 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-20T11:17:54.483 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-20T11:17:54.483 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet 2024-04-20T11:17:54.483 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-20T11:17:54.483 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-20T11:17:54.483 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom 2024-04-20T11:17:54.483 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-20T11:17:54.483 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-04-20T11:17:54.483 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv 2024-04-20T11:17:54.484 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-20T11:17:54.484 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154 2024-04-20T11:17:54.484 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-20T11:17:54.484 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-04-20T11:17:54.484 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan 2024-04-20T11:17:54.484 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-20T11:17:54.484 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-04-20T11:17:54.484 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-04-20T11:17:54.484 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio 2024-04-20T11:17:54.484 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-04-20T11:17:54.484 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-20T11:17:54.484 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-20T11:17:54.484 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-20T11:17:54.484 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-04-20T11:17:54.485 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/net_failover.ko 2024-04-20T11:17:54.485 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-04-20T11:17:54.485 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-04-20T11:17:54.485 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs 2024-04-20T11:17:54.485 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-20T11:17:54.485 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-20T11:17:54.485 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy 2024-04-20T11:17:54.485 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-04-20T11:17:54.485 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-20T11:17:54.485 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-20T11:17:54.485 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-20T11:17:54.485 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-04-20T11:17:54.485 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-04-20T11:17:54.485 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-04-20T11:17:54.485 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-04-20T11:17:54.486 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/dp83640.ko 2024-04-20T11:17:54.486 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-04-20T11:17:54.486 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-04-20T11:17:54.486 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-04-20T11:17:54.486 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-04-20T11:17:54.486 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-04-20T11:17:54.486 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-04-20T11:17:54.486 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-04-20T11:17:54.486 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-04-20T11:17:54.486 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-04-20T11:17:54.486 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-04-20T11:17:54.486 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-04-20T11:17:54.486 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-04-20T11:17:54.486 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp 2024-04-20T11:17:54.487 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-20T11:17:54.487 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-04-20T11:17:54.487 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-20T11:17:54.487 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-20T11:17:54.487 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-20T11:17:54.487 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-20T11:17:54.487 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-04-20T11:17:54.487 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-04-20T11:17:54.487 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-04-20T11:17:54.487 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip 2024-04-20T11:17:54.487 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slhc.ko 2024-04-20T11:17:54.487 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-04-20T11:17:54.487 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-04-20T11:17:54.487 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-04-20T11:17:54.487 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team 2024-04-20T11:17:54.488 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-04-20T11:17:54.488 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-20T11:17:54.488 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-20T11:17:54.488 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-20T11:17:54.488 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-04-20T11:17:54.488 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-20T11:17:54.488 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tun.ko 2024-04-20T11:17:54.488 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-04-20T11:17:54.488 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/virtio_net.ko 2024-04-20T11:17:54.488 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3 2024-04-20T11:17:54.488 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-20T11:17:54.488 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan 2024-04-20T11:17:54.488 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-04-20T11:17:54.488 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback 2024-04-20T11:17:54.489 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-20T11:17:54.489 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netfront.ko 2024-04-20T11:17:54.489 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb 2024-04-20T11:17:54.489 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-04-20T11:17:54.489 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme 2024-04-20T11:17:54.489 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host 2024-04-20T11:17:54.489 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-04-20T11:17:54.489 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-04-20T11:17:54.489 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport 2024-04-20T11:17:54.489 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-04-20T11:17:54.489 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-04-20T11:17:54.489 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-04-20T11:17:54.489 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-04-20T11:17:54.489 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci 2024-04-20T11:17:54.490 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug 2024-04-20T11:17:54.490 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-20T11:17:54.490 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie 2024-04-20T11:17:54.490 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-20T11:17:54.490 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-04-20T11:17:54.490 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia 2024-04-20T11:17:54.490 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-04-20T11:17:54.490 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-04-20T11:17:54.490 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-20T11:17:54.490 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl 2024-04-20T11:17:54.490 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel 2024-04-20T11:17:54.490 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-20T11:17:54.490 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform 2024-04-20T11:17:54.490 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86 2024-04-20T11:17:54.491 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-20T11:17:54.491 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-04-20T11:17:54.491 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-20T11:17:54.491 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-20T11:17:54.491 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-20T11:17:54.491 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-20T11:17:54.491 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-20T11:17:54.491 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-20T11:17:54.491 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-20T11:17:54.491 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-20T11:17:54.491 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-20T11:17:54.491 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-04-20T11:17:54.491 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel 2024-04-20T11:17:54.491 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-20T11:17:54.492 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-04-20T11:17:54.492 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-20T11:17:54.492 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-20T11:17:54.492 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-20T11:17:54.492 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-20T11:17:54.492 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-20T11:17:54.492 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-20T11:17:54.492 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-20T11:17:54.492 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-20T11:17:54.492 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-20T11:17:54.492 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-04-20T11:17:54.492 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap 2024-04-20T11:17:54.492 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-20T11:17:54.492 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-20T11:17:54.492 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps 2024-04-20T11:17:54.493 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients 2024-04-20T11:17:54.493 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-20T11:17:54.493 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-20T11:17:54.493 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-04-20T11:17:54.493 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-04-20T11:17:54.493 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp 2024-04-20T11:17:54.493 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-04-20T11:17:54.493 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-04-20T11:17:54.493 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras 2024-04-20T11:17:54.493 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd 2024-04-20T11:17:54.493 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl 2024-04-20T11:17:54.493 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-20T11:17:54.493 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-04-20T11:17:54.493 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc 2024-04-20T11:17:54.493 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-20T11:17:54.494 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-20T11:17:54.494 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-20T11:17:54.494 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-20T11:17:54.494 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-20T11:17:54.494 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-20T11:17:54.494 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-20T11:17:54.494 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-20T11:17:54.494 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-20T11:17:54.494 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-20T11:17:54.494 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-04-20T11:17:54.494 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-20T11:17:54.494 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-20T11:17:54.494 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-20T11:17:54.494 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-20T11:17:54.495 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-20T11:17:54.495 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-20T11:17:54.495 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-04-20T11:17:54.495 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-20T11:17:54.495 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-20T11:17:54.495 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-20T11:17:54.495 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-20T11:17:54.495 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-20T11:17:54.495 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-20T11:17:54.495 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-20T11:17:54.495 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-20T11:17:54.495 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-20T11:17:54.495 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-20T11:17:54.495 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-20T11:17:54.496 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-20T11:17:54.496 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-04-20T11:17:54.496 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi 2024-04-20T11:17:54.496 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-04-20T11:17:54.496 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-04-20T11:17:54.496 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-04-20T11:17:54.496 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-04-20T11:17:54.496 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-04-20T11:17:54.496 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid 2024-04-20T11:17:54.496 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-20T11:17:54.496 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-04-20T11:17:54.496 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx 2024-04-20T11:17:54.496 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-20T11:17:54.496 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-20T11:17:54.496 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-04-20T11:17:54.497 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr 2024-04-20T11:17:54.497 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-20T11:17:54.497 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-04-20T11:17:54.497 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi 2024-04-20T11:17:54.497 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-20T11:17:54.497 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa 2024-04-20T11:17:54.497 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-04-20T11:17:54.497 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc 2024-04-20T11:17:54.497 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-20T11:17:54.497 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i 2024-04-20T11:17:54.497 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-20T11:17:54.497 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-04-20T11:17:54.497 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor 2024-04-20T11:17:54.497 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-20T11:17:54.497 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi 2024-04-20T11:17:54.498 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-20T11:17:54.498 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-20T11:17:54.498 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-20T11:17:54.498 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-20T11:17:54.498 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-20T11:17:54.498 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-04-20T11:17:54.498 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler 2024-04-20T11:17:54.498 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-20T11:17:54.498 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-20T11:17:54.498 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-20T11:17:54.498 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-20T11:17:54.498 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-04-20T11:17:54.498 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r 2024-04-20T11:17:54.499 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-20T11:17:54.499 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-04-20T11:17:54.499 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe 2024-04-20T11:17:54.499 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-20T11:17:54.499 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-20T11:17:54.499 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic 2024-04-20T11:17:54.499 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-04-20T11:17:54.499 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-04-20T11:17:54.499 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-04-20T11:17:54.499 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-04-20T11:17:54.499 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-04-20T11:17:54.499 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-04-20T11:17:54.499 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-04-20T11:17:54.499 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci 2024-04-20T11:17:54.499 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-04-20T11:17:54.500 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-20T11:17:54.500 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-20T11:17:54.500 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc 2024-04-20T11:17:54.500 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-04-20T11:17:54.500 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-04-20T11:17:54.500 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-20T11:17:54.500 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas 2024-04-20T11:17:54.500 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-04-20T11:17:54.500 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc 2024-04-20T11:17:54.500 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-20T11:17:54.500 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid 2024-04-20T11:17:54.500 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-04-20T11:17:54.500 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-20T11:17:54.500 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-20T11:17:54.501 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-20T11:17:54.501 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas 2024-04-20T11:17:54.501 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-20T11:17:54.501 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas 2024-04-20T11:17:54.501 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-20T11:17:54.501 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-04-20T11:17:54.501 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001 2024-04-20T11:17:54.501 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-20T11:17:54.501 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-04-20T11:17:54.501 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-04-20T11:17:54.501 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx 2024-04-20T11:17:54.501 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-20T11:17:54.501 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-20T11:17:54.501 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx 2024-04-20T11:17:54.502 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-20T11:17:54.502 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-04-20T11:17:54.502 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-04-20T11:17:54.502 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-20T11:17:54.502 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-20T11:17:54.502 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-20T11:17:54.502 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-20T11:17:54.502 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-20T11:17:54.502 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-04-20T11:17:54.502 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-04-20T11:17:54.502 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-04-20T11:17:54.502 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2 2024-04-20T11:17:54.502 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-20T11:17:54.502 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-04-20T11:17:54.503 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-20T11:17:54.503 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-04-20T11:17:54.503 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb 2024-04-20T11:17:54.503 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-04-20T11:17:54.503 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target 2024-04-20T11:17:54.503 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi 2024-04-20T11:17:54.503 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-20T11:17:54.503 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback 2024-04-20T11:17:54.503 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-20T11:17:54.503 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-04-20T11:17:54.503 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-04-20T11:17:54.503 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-04-20T11:17:54.503 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-04-20T11:17:54.503 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-04-20T11:17:54.503 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc 2024-04-20T11:17:54.504 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-20T11:17:54.504 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal 2024-04-20T11:17:54.504 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel 2024-04-20T11:17:54.504 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal 2024-04-20T11:17:54.504 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-20T11:17:54.504 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-20T11:17:54.504 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-20T11:17:54.504 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-20T11:17:54.504 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-20T11:17:54.504 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-20T11:17:54.504 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-20T11:17:54.504 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-20T11:17:54.504 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-20T11:17:54.504 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-20T11:17:54.505 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-20T11:17:54.505 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-20T11:17:54.505 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-20T11:17:54.505 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-20T11:17:54.505 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-20T11:17:54.505 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-20T11:17:54.505 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-20T11:17:54.505 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-20T11:17:54.505 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty 2024-04-20T11:17:54.505 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless 2024-04-20T11:17:54.505 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-20T11:17:54.505 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-04-20T11:17:54.505 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-04-20T11:17:54.505 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-04-20T11:17:54.505 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial 2024-04-20T11:17:54.506 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250 2024-04-20T11:17:54.506 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-20T11:17:54.506 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-04-20T11:17:54.506 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm 2024-04-20T11:17:54.506 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-20T11:17:54.506 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs 2024-04-20T11:17:54.506 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core 2024-04-20T11:17:54.506 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-20T11:17:54.506 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host 2024-04-20T11:17:54.506 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-20T11:17:54.506 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio 2024-04-20T11:17:54.506 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-04-20T11:17:54.506 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-04-20T11:17:54.506 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-04-20T11:17:54.507 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-04-20T11:17:54.507 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-04-20T11:17:54.507 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb 2024-04-20T11:17:54.507 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class 2024-04-20T11:17:54.507 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-04-20T11:17:54.507 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-20T11:17:54.507 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-04-20T11:17:54.507 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-04-20T11:17:54.507 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host 2024-04-20T11:17:54.507 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-20T11:17:54.507 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image 2024-04-20T11:17:54.507 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-04-20T11:17:54.507 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-04-20T11:17:54.507 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc 2024-04-20T11:17:54.507 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-04-20T11:17:54.507 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-04-20T11:17:54.508 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-04-20T11:17:54.508 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-04-20T11:17:54.508 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-04-20T11:17:54.508 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-04-20T11:17:54.508 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-04-20T11:17:54.508 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-20T11:17:54.508 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-04-20T11:17:54.508 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-04-20T11:17:54.508 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga 2024-04-20T11:17:54.508 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-20T11:17:54.508 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-20T11:17:54.508 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-04-20T11:17:54.508 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-04-20T11:17:54.508 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-20T11:17:54.509 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-04-20T11:17:54.509 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-04-20T11:17:54.509 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy 2024-04-20T11:17:54.509 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-04-20T11:17:54.509 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial 2024-04-20T11:17:54.509 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-04-20T11:17:54.509 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-04-20T11:17:54.509 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-20T11:17:54.509 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-04-20T11:17:54.509 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-04-20T11:17:54.509 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-04-20T11:17:54.509 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-20T11:17:54.509 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-20T11:17:54.509 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-04-20T11:17:54.510 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-20T11:17:54.510 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-20T11:17:54.510 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-20T11:17:54.510 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-04-20T11:17:54.510 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-04-20T11:17:54.510 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-04-20T11:17:54.510 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-04-20T11:17:54.510 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-20T11:17:54.510 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-04-20T11:17:54.510 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-20T11:17:54.510 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-20T11:17:54.510 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-20T11:17:54.510 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-04-20T11:17:54.510 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-04-20T11:17:54.511 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-04-20T11:17:54.511 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-04-20T11:17:54.511 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-04-20T11:17:54.511 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-04-20T11:17:54.511 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-04-20T11:17:54.511 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-04-20T11:17:54.511 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-04-20T11:17:54.511 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-04-20T11:17:54.511 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-04-20T11:17:54.511 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-04-20T11:17:54.511 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-04-20T11:17:54.511 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-04-20T11:17:54.511 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-20T11:17:54.511 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-04-20T11:17:54.511 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-04-20T11:17:54.512 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-20T11:17:54.512 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-20T11:17:54.512 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-04-20T11:17:54.512 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-20T11:17:54.512 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-04-20T11:17:54.512 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-04-20T11:17:54.512 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-20T11:17:54.512 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage 2024-04-20T11:17:54.512 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-04-20T11:17:54.512 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-20T11:17:54.512 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-20T11:17:54.512 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-20T11:17:54.512 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-20T11:17:54.512 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-20T11:17:54.512 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-20T11:17:54.513 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-20T11:17:54.513 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-04-20T11:17:54.513 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-20T11:17:54.513 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-20T11:17:54.513 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-20T11:17:54.513 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-20T11:17:54.513 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-20T11:17:54.513 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-04-20T11:17:54.513 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip 2024-04-20T11:17:54.513 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-20T11:17:54.513 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-20T11:17:54.513 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-20T11:17:54.513 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio 2024-04-20T11:17:54.513 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci 2024-04-20T11:17:54.514 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-20T11:17:54.514 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-20T11:17:54.514 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-04-20T11:17:54.514 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-20T11:17:54.514 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost 2024-04-20T11:17:54.514 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-04-20T11:17:54.514 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-20T11:17:54.514 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-04-20T11:17:54.514 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-04-20T11:17:54.514 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video 2024-04-20T11:17:54.514 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight 2024-04-20T11:17:54.514 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-04-20T11:17:54.514 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-04-20T11:17:54.514 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-20T11:17:54.515 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio 2024-04-20T11:17:54.515 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio.ko 2024-04-20T11:17:54.515 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_balloon.ko 2024-04-20T11:17:54.515 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_mmio.ko 2024-04-20T11:17:54.515 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci.ko 2024-04-20T11:17:54.515 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-20T11:17:54.515 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-20T11:17:54.515 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_ring.ko 2024-04-20T11:17:54.515 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1 2024-04-20T11:17:54.515 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters 2024-04-20T11:17:54.515 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-04-20T11:17:54.515 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-04-20T11:17:54.515 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves 2024-04-20T11:17:54.515 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-20T11:17:54.515 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-20T11:17:54.516 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-20T11:17:54.516 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-20T11:17:54.516 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-20T11:17:54.516 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-20T11:17:54.516 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-20T11:17:54.516 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-20T11:17:54.516 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-20T11:17:54.516 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-20T11:17:54.516 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-20T11:17:54.516 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-04-20T11:17:54.516 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog 2024-04-20T11:17:54.516 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-20T11:17:54.516 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-20T11:17:54.516 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-20T11:17:54.516 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-04-20T11:17:54.517 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-04-20T11:17:54.517 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-20T11:17:54.517 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-20T11:17:54.517 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-04-20T11:17:54.517 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-04-20T11:17:54.517 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-20T11:17:54.517 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-20T11:17:54.517 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-04-20T11:17:54.517 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-04-20T11:17:54.517 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-04-20T11:17:54.517 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-20T11:17:54.517 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-20T11:17:54.517 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-20T11:17:54.517 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-20T11:17:54.517 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-04-20T11:17:54.518 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-20T11:17:54.518 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-04-20T11:17:54.518 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-20T11:17:54.518 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-20T11:17:54.518 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-20T11:17:54.518 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-04-20T11:17:54.518 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-04-20T11:17:54.518 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen 2024-04-20T11:17:54.518 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-20T11:17:54.518 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-04-20T11:17:54.518 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-04-20T11:17:54.518 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-04-20T11:17:54.518 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback 2024-04-20T11:17:54.518 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-20T11:17:54.518 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-04-20T11:17:54.519 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs 2024-04-20T11:17:54.519 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-20T11:17:54.519 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs 2024-04-20T11:17:54.519 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p 2024-04-20T11:17:54.519 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-04-20T11:17:54.519 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs 2024-04-20T11:17:54.519 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-04-20T11:17:54.519 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs 2024-04-20T11:17:54.519 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-04-20T11:17:54.519 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-04-20T11:17:54.519 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs 2024-04-20T11:17:54.519 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-04-20T11:17:54.519 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles 2024-04-20T11:17:54.519 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-04-20T11:17:54.519 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph 2024-04-20T11:17:54.520 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-04-20T11:17:54.520 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda 2024-04-20T11:17:54.520 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-04-20T11:17:54.520 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs 2024-04-20T11:17:54.520 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-04-20T11:17:54.520 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm 2024-04-20T11:17:54.520 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-04-20T11:17:54.520 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs 2024-04-20T11:17:54.520 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-20T11:17:54.520 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs 2024-04-20T11:17:54.520 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-04-20T11:17:54.520 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat 2024-04-20T11:17:54.520 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/fat.ko 2024-04-20T11:17:54.520 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-04-20T11:17:54.520 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/vfat.ko 2024-04-20T11:17:54.521 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse 2024-04-20T11:17:54.521 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-04-20T11:17:54.521 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/fuse.ko 2024-04-20T11:17:54.521 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2 2024-04-20T11:17:54.521 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-04-20T11:17:54.521 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs 2024-04-20T11:17:54.521 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-04-20T11:17:54.521 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus 2024-04-20T11:17:54.521 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-04-20T11:17:54.521 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs 2024-04-20T11:17:54.521 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-04-20T11:17:54.521 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs 2024-04-20T11:17:54.521 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-04-20T11:17:54.521 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd 2024-04-20T11:17:54.522 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-04-20T11:17:54.522 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix 2024-04-20T11:17:54.522 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-04-20T11:17:54.522 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs 2024-04-20T11:17:54.522 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-04-20T11:17:54.522 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs 2024-04-20T11:17:54.522 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout 2024-04-20T11:17:54.522 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-20T11:17:54.522 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout 2024-04-20T11:17:54.522 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-20T11:17:54.522 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout 2024-04-20T11:17:54.522 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-20T11:17:54.522 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-04-20T11:17:54.522 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-04-20T11:17:54.522 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-04-20T11:17:54.523 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common 2024-04-20T11:17:54.523 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-04-20T11:17:54.523 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-04-20T11:17:54.523 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd 2024-04-20T11:17:54.523 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-04-20T11:17:54.523 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2 2024-04-20T11:17:54.523 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-04-20T11:17:54.523 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls 2024-04-20T11:17:54.523 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-04-20T11:17:54.523 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-04-20T11:17:54.523 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-04-20T11:17:54.523 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-04-20T11:17:54.523 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-04-20T11:17:54.523 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-04-20T11:17:54.523 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-04-20T11:17:54.524 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-04-20T11:17:54.524 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-04-20T11:17:54.524 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-04-20T11:17:54.524 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-04-20T11:17:54.524 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-04-20T11:17:54.524 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-04-20T11:17:54.524 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-04-20T11:17:54.524 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-04-20T11:17:54.524 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-04-20T11:17:54.524 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-04-20T11:17:54.524 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-04-20T11:17:54.524 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-04-20T11:17:54.524 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-04-20T11:17:54.524 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-04-20T11:17:54.525 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-04-20T11:17:54.525 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-04-20T11:17:54.525 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-04-20T11:17:54.525 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-04-20T11:17:54.525 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-04-20T11:17:54.525 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-04-20T11:17:54.525 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-04-20T11:17:54.525 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-04-20T11:17:54.525 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-04-20T11:17:54.525 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-04-20T11:17:54.525 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-04-20T11:17:54.525 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-04-20T11:17:54.525 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-04-20T11:17:54.525 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-04-20T11:17:54.525 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-04-20T11:17:54.526 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-04-20T11:17:54.526 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-04-20T11:17:54.526 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-04-20T11:17:54.526 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-04-20T11:17:54.526 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-04-20T11:17:54.526 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-04-20T11:17:54.526 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-04-20T11:17:54.526 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-04-20T11:17:54.526 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-04-20T11:17:54.526 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-04-20T11:17:54.526 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-04-20T11:17:54.526 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-04-20T11:17:54.526 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-20T11:17:54.526 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-04-20T11:17:54.527 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2 2024-04-20T11:17:54.527 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster 2024-04-20T11:17:54.527 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-20T11:17:54.527 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm 2024-04-20T11:17:54.527 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-20T11:17:54.527 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs 2024-04-20T11:17:54.527 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-20T11:17:54.527 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-04-20T11:17:54.527 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-20T11:17:54.527 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-20T11:17:54.527 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-20T11:17:54.527 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs 2024-04-20T11:17:54.527 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-04-20T11:17:54.527 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore 2024-04-20T11:17:54.527 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-04-20T11:17:54.527 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs 2024-04-20T11:17:54.528 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-04-20T11:17:54.528 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs 2024-04-20T11:17:54.528 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-04-20T11:17:54.528 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb 2024-04-20T11:17:54.528 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client 2024-04-20T11:17:54.528 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-04-20T11:17:54.528 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common 2024-04-20T11:17:54.528 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-04-20T11:17:54.528 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-04-20T11:17:54.528 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs 2024-04-20T11:17:54.528 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-04-20T11:17:54.528 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv 2024-04-20T11:17:54.528 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-04-20T11:17:54.528 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf 2024-04-20T11:17:54.528 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-04-20T11:17:54.529 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs 2024-04-20T11:17:54.529 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-04-20T11:17:54.529 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs 2024-04-20T11:17:54.529 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-04-20T11:17:54.529 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel 2024-04-20T11:17:54.529 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu 2024-04-20T11:17:54.529 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu/rcutorture.ko 2024-04-20T11:17:54.529 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-04-20T11:17:54.529 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace 2024-04-20T11:17:54.529 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-20T11:17:54.529 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib 2024-04-20T11:17:54.529 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/asn1_encoder.ko 2024-04-20T11:17:54.529 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-04-20T11:17:54.529 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-04-20T11:17:54.529 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto 2024-04-20T11:17:54.530 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-04-20T11:17:54.530 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-04-20T11:17:54.530 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-04-20T11:17:54.530 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-04-20T11:17:54.530 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4 2024-04-20T11:17:54.530 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-04-20T11:17:54.530 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-04-20T11:17:54.530 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math 2024-04-20T11:17:54.530 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-04-20T11:17:54.530 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6 2024-04-20T11:17:54.530 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-04-20T11:17:54.530 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon 2024-04-20T11:17:54.530 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-20T11:17:54.530 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-04-20T11:17:54.531 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-04-20T11:17:54.531 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-04-20T11:17:54.531 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd 2024-04-20T11:17:54.531 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd/zstd_compress.ko 2024-04-20T11:17:54.531 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm 2024-04-20T11:17:54.531 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-04-20T11:17:54.531 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net 2024-04-20T11:17:54.531 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan 2024-04-20T11:17:54.531 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-04-20T11:17:54.531 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-04-20T11:17:54.531 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-04-20T11:17:54.531 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-04-20T11:17:54.531 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-20T11:17:54.531 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-04-20T11:17:54.531 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-04-20T11:17:54.532 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-04-20T11:17:54.532 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802 2024-04-20T11:17:54.532 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-04-20T11:17:54.532 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-04-20T11:17:54.532 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-04-20T11:17:54.532 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-04-20T11:17:54.532 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-04-20T11:17:54.532 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q 2024-04-20T11:17:54.532 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-04-20T11:17:54.532 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p 2024-04-20T11:17:54.532 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-04-20T11:17:54.532 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-04-20T11:17:54.532 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-04-20T11:17:54.532 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-04-20T11:17:54.533 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv 2024-04-20T11:17:54.533 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-04-20T11:17:54.533 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge 2024-04-20T11:17:54.533 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-04-20T11:17:54.533 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-04-20T11:17:54.533 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter 2024-04-20T11:17:54.533 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-20T11:17:54.533 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-20T11:17:54.533 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-20T11:17:54.533 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-20T11:17:54.533 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-20T11:17:54.533 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-20T11:17:54.533 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-20T11:17:54.533 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-20T11:17:54.533 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-20T11:17:54.534 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-20T11:17:54.534 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-20T11:17:54.534 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-20T11:17:54.534 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-20T11:17:54.534 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-20T11:17:54.534 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-20T11:17:54.534 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-20T11:17:54.534 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-20T11:17:54.534 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-20T11:17:54.534 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-20T11:17:54.534 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-20T11:17:54.534 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-04-20T11:17:54.534 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-20T11:17:54.534 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-20T11:17:54.535 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-20T11:17:54.535 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph 2024-04-20T11:17:54.535 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-04-20T11:17:54.535 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core 2024-04-20T11:17:54.535 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/failover.ko 2024-04-20T11:17:54.535 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-04-20T11:17:54.535 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp 2024-04-20T11:17:54.535 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-04-20T11:17:54.535 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-04-20T11:17:54.535 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-04-20T11:17:54.535 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-04-20T11:17:54.535 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver 2024-04-20T11:17:54.535 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver/dns_resolver.ko 2024-04-20T11:17:54.535 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa 2024-04-20T11:17:54.535 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-04-20T11:17:54.536 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-04-20T11:17:54.536 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-04-20T11:17:54.536 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-04-20T11:17:54.536 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-04-20T11:17:54.536 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154 2024-04-20T11:17:54.536 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan 2024-04-20T11:17:54.536 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-20T11:17:54.536 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-04-20T11:17:54.536 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-20T11:17:54.536 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4 2024-04-20T11:17:54.536 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-04-20T11:17:54.536 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-04-20T11:17:54.536 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-04-20T11:17:54.536 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-04-20T11:17:54.537 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-04-20T11:17:54.537 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-04-20T11:17:54.537 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-04-20T11:17:54.537 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-04-20T11:17:54.537 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-04-20T11:17:54.537 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-04-20T11:17:54.537 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter 2024-04-20T11:17:54.537 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-20T11:17:54.537 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-20T11:17:54.537 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-20T11:17:54.537 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-20T11:17:54.537 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-20T11:17:54.537 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-20T11:17:54.537 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-20T11:17:54.537 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-20T11:17:54.538 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-20T11:17:54.538 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-20T11:17:54.538 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-20T11:17:54.538 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-20T11:17:54.538 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-20T11:17:54.538 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-20T11:17:54.538 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-20T11:17:54.538 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-20T11:17:54.538 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-20T11:17:54.538 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-20T11:17:54.538 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-20T11:17:54.538 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-20T11:17:54.538 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-20T11:17:54.538 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-04-20T11:17:54.539 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-04-20T11:17:54.539 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-04-20T11:17:54.539 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-04-20T11:17:54.539 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-04-20T11:17:54.539 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-04-20T11:17:54.539 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-04-20T11:17:54.539 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-04-20T11:17:54.539 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-04-20T11:17:54.539 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-04-20T11:17:54.539 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-04-20T11:17:54.539 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-04-20T11:17:54.539 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-04-20T11:17:54.539 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-04-20T11:17:54.539 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-04-20T11:17:54.539 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-04-20T11:17:54.540 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-20T11:17:54.540 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6 2024-04-20T11:17:54.540 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-04-20T11:17:54.540 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-04-20T11:17:54.540 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-04-20T11:17:54.540 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-04-20T11:17:54.540 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-20T11:17:54.540 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-04-20T11:17:54.540 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-04-20T11:17:54.540 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter 2024-04-20T11:17:54.540 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-20T11:17:54.540 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-20T11:17:54.540 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-20T11:17:54.540 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-20T11:17:54.540 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-20T11:17:54.541 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-20T11:17:54.541 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-20T11:17:54.541 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-20T11:17:54.541 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-20T11:17:54.541 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-20T11:17:54.541 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-20T11:17:54.541 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-20T11:17:54.541 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-20T11:17:54.541 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-20T11:17:54.541 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-20T11:17:54.541 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-20T11:17:54.541 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-20T11:17:54.541 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-20T11:17:54.541 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-20T11:17:54.541 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-20T11:17:54.542 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-20T11:17:54.542 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-20T11:17:54.542 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-20T11:17:54.542 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-20T11:17:54.542 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-04-20T11:17:54.542 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-04-20T11:17:54.542 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-20T11:17:54.542 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key 2024-04-20T11:17:54.542 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-04-20T11:17:54.542 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp 2024-04-20T11:17:54.542 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-04-20T11:17:54.542 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-20T11:17:54.542 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-04-20T11:17:54.542 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-04-20T11:17:54.543 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-04-20T11:17:54.543 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-04-20T11:17:54.543 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-04-20T11:17:54.543 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc 2024-04-20T11:17:54.543 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-04-20T11:17:54.543 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154 2024-04-20T11:17:54.543 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-04-20T11:17:54.543 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls 2024-04-20T11:17:54.543 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-04-20T11:17:54.543 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter 2024-04-20T11:17:54.543 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset 2024-04-20T11:17:54.543 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-04-20T11:17:54.543 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-20T11:17:54.543 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-20T11:17:54.543 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-20T11:17:54.544 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-20T11:17:54.544 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-20T11:17:54.544 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-20T11:17:54.544 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-20T11:17:54.544 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-20T11:17:54.544 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-20T11:17:54.544 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-20T11:17:54.544 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-20T11:17:54.544 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-20T11:17:54.544 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-20T11:17:54.544 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-20T11:17:54.544 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-20T11:17:54.544 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs 2024-04-20T11:17:54.544 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-20T11:17:54.545 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-20T11:17:54.545 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-20T11:17:54.545 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-20T11:17:54.545 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-20T11:17:54.545 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-20T11:17:54.545 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-20T11:17:54.545 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-20T11:17:54.545 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-20T11:17:54.545 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-20T11:17:54.545 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-20T11:17:54.545 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-20T11:17:54.545 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-20T11:17:54.545 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-20T11:17:54.545 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-04-20T11:17:54.545 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-04-20T11:17:54.546 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-20T11:17:54.546 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-20T11:17:54.546 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-20T11:17:54.546 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-20T11:17:54.546 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-20T11:17:54.546 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-20T11:17:54.546 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-20T11:17:54.546 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-20T11:17:54.546 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-20T11:17:54.546 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-20T11:17:54.546 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-20T11:17:54.546 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-20T11:17:54.546 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-20T11:17:54.547 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-04-20T11:17:54.547 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-20T11:17:54.547 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-04-20T11:17:54.547 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-04-20T11:17:54.547 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-20T11:17:54.547 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-20T11:17:54.547 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-04-20T11:17:54.547 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-04-20T11:17:54.547 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-20T11:17:54.547 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-20T11:17:54.547 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-04-20T11:17:54.547 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-04-20T11:17:54.547 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-20T11:17:54.547 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-04-20T11:17:54.548 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-20T11:17:54.548 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-20T11:17:54.548 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-04-20T11:17:54.548 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-04-20T11:17:54.548 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-04-20T11:17:54.548 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-20T11:17:54.548 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-04-20T11:17:54.548 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-20T11:17:54.548 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-04-20T11:17:54.548 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-04-20T11:17:54.548 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-04-20T11:17:54.548 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-04-20T11:17:54.548 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-04-20T11:17:54.548 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-04-20T11:17:54.548 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-04-20T11:17:54.548 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_quota.ko 2024-04-20T11:17:54.549 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-04-20T11:17:54.549 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-04-20T11:17:54.549 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-04-20T11:17:54.549 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-04-20T11:17:54.549 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tunnel.ko 2024-04-20T11:17:54.549 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-04-20T11:17:54.549 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-20T11:17:54.549 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-20T11:17:54.549 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-20T11:17:54.549 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-04-20T11:17:54.549 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-04-20T11:17:54.549 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-04-20T11:17:54.549 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-04-20T11:17:54.549 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-20T11:17:54.549 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-04-20T11:17:54.550 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-04-20T11:17:54.550 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-20T11:17:54.550 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-04-20T11:17:54.550 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-04-20T11:17:54.550 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-20T11:17:54.550 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-04-20T11:17:54.550 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-20T11:17:54.550 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-04-20T11:17:54.550 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-20T11:17:54.550 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-20T11:17:54.550 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-04-20T11:17:54.550 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-04-20T11:17:54.550 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-04-20T11:17:54.550 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-04-20T11:17:54.550 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-04-20T11:17:54.551 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-04-20T11:17:54.551 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-04-20T11:17:54.551 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-04-20T11:17:54.551 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-04-20T11:17:54.551 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-04-20T11:17:54.551 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-04-20T11:17:54.551 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-04-20T11:17:54.551 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-04-20T11:17:54.551 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-04-20T11:17:54.551 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-04-20T11:17:54.551 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-04-20T11:17:54.551 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-04-20T11:17:54.551 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-04-20T11:17:54.551 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-04-20T11:17:54.551 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-04-20T11:17:54.552 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-04-20T11:17:54.552 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-04-20T11:17:54.552 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-04-20T11:17:54.552 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-04-20T11:17:54.552 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-04-20T11:17:54.552 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-04-20T11:17:54.552 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-04-20T11:17:54.552 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-04-20T11:17:54.552 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-04-20T11:17:54.552 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-04-20T11:17:54.552 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-04-20T11:17:54.552 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-04-20T11:17:54.552 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-04-20T11:17:54.552 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-04-20T11:17:54.553 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-04-20T11:17:54.553 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-04-20T11:17:54.553 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-04-20T11:17:54.553 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-04-20T11:17:54.553 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-04-20T11:17:54.553 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-04-20T11:17:54.553 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-04-20T11:17:54.553 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-04-20T11:17:54.553 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-04-20T11:17:54.553 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-04-20T11:17:54.553 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-04-20T11:17:54.553 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-04-20T11:17:54.553 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-04-20T11:17:54.553 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-04-20T11:17:54.554 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-04-20T11:17:54.554 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-04-20T11:17:54.554 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-04-20T11:17:54.554 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink 2024-04-20T11:17:54.554 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-04-20T11:17:54.554 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh 2024-04-20T11:17:54.554 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-04-20T11:17:54.554 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch 2024-04-20T11:17:54.554 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-04-20T11:17:54.554 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-04-20T11:17:54.554 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-04-20T11:17:54.554 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-04-20T11:17:54.554 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet 2024-04-20T11:17:54.554 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-04-20T11:17:54.555 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds 2024-04-20T11:17:54.555 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-04-20T11:17:54.555 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-04-20T11:17:54.555 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-04-20T11:17:54.555 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched 2024-04-20T11:17:54.555 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-04-20T11:17:54.555 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-04-20T11:17:54.555 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-04-20T11:17:54.555 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-04-20T11:17:54.555 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-04-20T11:17:54.555 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-04-20T11:17:54.555 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-04-20T11:17:54.555 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-04-20T11:17:54.555 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-04-20T11:17:54.556 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-04-20T11:17:54.556 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-04-20T11:17:54.556 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-04-20T11:17:54.556 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-04-20T11:17:54.556 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-04-20T11:17:54.556 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-04-20T11:17:54.556 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-04-20T11:17:54.556 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-04-20T11:17:54.556 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-04-20T11:17:54.556 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-04-20T11:17:54.556 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-04-20T11:17:54.556 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-04-20T11:17:54.556 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-04-20T11:17:54.556 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-04-20T11:17:54.556 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-04-20T11:17:54.557 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-04-20T11:17:54.557 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-04-20T11:17:54.557 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-04-20T11:17:54.557 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-04-20T11:17:54.557 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-04-20T11:17:54.557 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-04-20T11:17:54.557 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-04-20T11:17:54.557 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-04-20T11:17:54.557 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-04-20T11:17:54.557 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-04-20T11:17:54.557 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-04-20T11:17:54.557 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-04-20T11:17:54.557 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-04-20T11:17:54.557 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-04-20T11:17:54.558 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-04-20T11:17:54.558 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-04-20T11:17:54.558 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-04-20T11:17:54.558 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-04-20T11:17:54.558 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-04-20T11:17:54.558 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp 2024-04-20T11:17:54.558 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-04-20T11:17:54.558 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-04-20T11:17:54.558 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc 2024-04-20T11:17:54.558 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss 2024-04-20T11:17:54.558 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-20T11:17:54.558 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-20T11:17:54.558 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-04-20T11:17:54.558 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma 2024-04-20T11:17:54.559 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-20T11:17:54.559 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc 2024-04-20T11:17:54.559 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-04-20T11:17:54.559 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-04-20T11:17:54.559 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix 2024-04-20T11:17:54.559 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-04-20T11:17:54.559 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock 2024-04-20T11:17:54.559 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-20T11:17:54.559 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-20T11:17:54.559 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-04-20T11:17:54.559 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-20T11:17:54.559 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-20T11:17:54.560 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm 2024-04-20T11:17:54.560 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-20T11:17:54.560 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security 2024-04-20T11:17:54.560 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys 2024-04-20T11:17:54.560 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys 2024-04-20T11:17:54.560 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-20T11:17:54.560 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys 2024-04-20T11:17:54.560 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys/trusted.ko 2024-04-20T11:17:54.560 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-04-20T11:17:54.560 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-04-20T11:17:54.560 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-04-20T11:17:54.560 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-04-20T11:17:54.560 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-04-20T11:17:54.560 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-04-20T11:17:54.560 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-04-20T11:17:54.561 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-04-20T11:17:54.561 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-04-20T11:17:54.561 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-04-20T11:17:54.561 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-04-20T11:17:54.561 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-04-20T11:17:54.561 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-04-20T11:17:54.561 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-20T11:17:54.565 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-04-20T11:17:54.565 DEBUG:teuthology.orchestra.run.smithi175:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-20T11:17:54.589 INFO:teuthology.orchestra.run.smithi175.stdout:postinstall scriptlet (using /bin/sh): 2024-04-20T11:17:54.589 INFO:teuthology.orchestra.run.smithi175.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-20T11:17:54.589 INFO:teuthology.orchestra.run.smithi175.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g806ec36b2112 /lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-20T11:17:54.589 INFO:teuthology.orchestra.run.smithi175.stdout:fi 2024-04-20T11:17:54.589 INFO:teuthology.orchestra.run.smithi175.stdout:for file in vmlinuz System.map config; do 2024-04-20T11:17:54.589 INFO:teuthology.orchestra.run.smithi175.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112"; then 2024-04-20T11:17:54.589 INFO:teuthology.orchestra.run.smithi175.stdout: cp "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112" 2024-04-20T11:17:54.589 INFO:teuthology.orchestra.run.smithi175.stdout: fi 2024-04-20T11:17:54.590 INFO:teuthology.orchestra.run.smithi175.stdout:done 2024-04-20T11:17:54.590 INFO:teuthology.orchestra.run.smithi175.stdout:if [ -d "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" ] && \ 2024-04-20T11:17:54.590 INFO:teuthology.orchestra.run.smithi175.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" >/dev/null 2>&1; then 2024-04-20T11:17:54.590 INFO:teuthology.orchestra.run.smithi175.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-20T11:17:54.590 INFO:teuthology.orchestra.run.smithi175.stdout: cp -r "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-20T11:17:54.590 INFO:teuthology.orchestra.run.smithi175.stdout:fi 2024-04-20T11:17:54.590 INFO:teuthology.orchestra.run.smithi175.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep" ]; then 2024-04-20T11:17:54.590 INFO:teuthology.orchestra.run.smithi175.stdout: /usr/sbin/depmod 6.9.0-rc3-g806ec36b2112 2024-04-20T11:17:54.590 INFO:teuthology.orchestra.run.smithi175.stdout:fi 2024-04-20T11:17:54.590 INFO:teuthology.orchestra.run.smithi175.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-20T11:17:54.590 INFO:teuthology.orchestra.run.smithi175.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-20T11:17:54.590 INFO:teuthology.orchestra.run.smithi175.stdout:kernel-install remove 6.9.0-rc3-g806ec36b2112 2024-04-20T11:17:54.591 INFO:teuthology.orchestra.run.smithi175.stdout:fi 2024-04-20T11:17:54.591 DEBUG:teuthology.orchestra.run.smithi175:> sudo rpm -qi grub2-tools 2024-04-20T11:17:54.666 INFO:teuthology.orchestra.run.smithi175.stdout:Name : grub2-tools 2024-04-20T11:17:54.666 INFO:teuthology.orchestra.run.smithi175.stdout:Epoch : 1 2024-04-20T11:17:54.666 INFO:teuthology.orchestra.run.smithi175.stdout:Version : 2.06 2024-04-20T11:17:54.666 INFO:teuthology.orchestra.run.smithi175.stdout:Release : 68.el9 2024-04-20T11:17:54.667 INFO:teuthology.orchestra.run.smithi175.stdout:Architecture: x86_64 2024-04-20T11:17:54.667 INFO:teuthology.orchestra.run.smithi175.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-04-20T11:17:54.667 INFO:teuthology.orchestra.run.smithi175.stdout:Group : Unspecified 2024-04-20T11:17:54.667 INFO:teuthology.orchestra.run.smithi175.stdout:Size : 8264254 2024-04-20T11:17:54.667 INFO:teuthology.orchestra.run.smithi175.stdout:License : GPLv3+ 2024-04-20T11:17:54.667 INFO:teuthology.orchestra.run.smithi175.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-20T11:17:54.667 INFO:teuthology.orchestra.run.smithi175.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-20T11:17:54.667 INFO:teuthology.orchestra.run.smithi175.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-20T11:17:54.667 INFO:teuthology.orchestra.run.smithi175.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-20T11:17:54.667 INFO:teuthology.orchestra.run.smithi175.stdout:Packager : builder@centos.org 2024-04-20T11:17:54.667 INFO:teuthology.orchestra.run.smithi175.stdout:Vendor : CentOS 2024-04-20T11:17:54.667 INFO:teuthology.orchestra.run.smithi175.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-20T11:17:54.667 INFO:teuthology.orchestra.run.smithi175.stdout:Summary : Support tools for GRUB. 2024-04-20T11:17:54.667 INFO:teuthology.orchestra.run.smithi175.stdout:Description : 2024-04-20T11:17:54.668 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-20T11:17:54.668 INFO:teuthology.orchestra.run.smithi175.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-20T11:17:54.668 INFO:teuthology.orchestra.run.smithi175.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-20T11:17:54.668 INFO:teuthology.orchestra.run.smithi175.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-20T11:17:54.668 INFO:teuthology.orchestra.run.smithi175.stdout:hardware devices. 2024-04-20T11:17:54.668 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-20T11:17:54.668 INFO:teuthology.orchestra.run.smithi175.stdout:This subpackage provides tools for support of all platforms. 2024-04-20T11:17:54.669 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-20T11:17:54.669 INFO:teuthology.task.kernel:Updating grub on smithi175 to boot 6.9.0-rc3-g806ec36b2112 2024-04-20T11:17:54.669 DEBUG:teuthology.orchestra.run.smithi175:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-20T11:17:55.257 INFO:teuthology.orchestra.run.smithi175.stderr:Generating grub configuration file ... 2024-04-20T11:17:56.698 INFO:teuthology.orchestra.run.smithi175.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-20T11:17:56.729 INFO:teuthology.orchestra.run.smithi175.stderr:done 2024-04-20T11:17:56.731 DEBUG:teuthology.orchestra.run.smithi175:> mktemp 2024-04-20T11:17:56.763 INFO:teuthology.orchestra.run.smithi175.stdout:/tmp/tmp.FpFI6aDLD5 2024-04-20T11:17:56.764 DEBUG:teuthology.orchestra.run.smithi175:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.FpFI6aDLD5 2024-04-20T11:17:56.831 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod 0666 /tmp/tmp.FpFI6aDLD5 2024-04-20T11:17:56.975 DEBUG:teuthology.orchestra.remote:smithi175:/tmp/tmp.FpFI6aDLD5 is 6KB 2024-04-20T11:17:57.022 DEBUG:teuthology.orchestra.run.smithi175:> rm -fr /tmp/tmp.FpFI6aDLD5 2024-04-20T11:17:57.036 DEBUG:teuthology.orchestra.run.smithi175:> sudo /bin/ls /boot/loader/entries || true 2024-04-20T11:17:57.097 INFO:teuthology.orchestra.run.smithi175.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-04-20T11:17:57.098 INFO:teuthology.orchestra.run.smithi175.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-04-20T11:17:57.098 INFO:teuthology.orchestra.run.smithi175.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-04-20T11:17:57.098 INFO:teuthology.orchestra.run.smithi175.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112.conf 2024-04-20T11:17:57.099 DEBUG:teuthology.orchestra.run.smithi175:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112 2024-04-20T11:17:57.257 DEBUG:teuthology.orchestra.run.smithi175:> sudo shutdown -r now 2024-04-20T11:18:09.392 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi152.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-20T11:18:09.392 DEBUG:teuthology.orchestra.run.smithi152:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-20T11:18:09.495 INFO:teuthology.orchestra.run.smithi152.stdout:/boot/System.map-6.9.0-rc3-g806ec36b2112 2024-04-20T11:18:09.496 INFO:teuthology.orchestra.run.smithi152.stdout:/boot/config-6.9.0-rc3-g806ec36b2112 2024-04-20T11:18:09.496 INFO:teuthology.orchestra.run.smithi152.stdout:/boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-20T11:18:09.496 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/System.map 2024-04-20T11:18:09.496 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/config 2024-04-20T11:18:09.496 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel 2024-04-20T11:18:09.496 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch 2024-04-20T11:18:09.496 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86 2024-04-20T11:18:09.496 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto 2024-04-20T11:18:09.496 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-20T11:18:09.496 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-20T11:18:09.496 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-20T11:18:09.496 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-20T11:18:09.496 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-20T11:18:09.496 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-20T11:18:09.497 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-20T11:18:09.497 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-20T11:18:09.497 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-20T11:18:09.497 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-20T11:18:09.497 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-20T11:18:09.497 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-20T11:18:09.497 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-20T11:18:09.497 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-20T11:18:09.497 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-20T11:18:09.497 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-20T11:18:09.497 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-20T11:18:09.497 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-20T11:18:09.497 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-20T11:18:09.497 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-20T11:18:09.498 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel 2024-04-20T11:18:09.498 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu 2024-04-20T11:18:09.498 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce 2024-04-20T11:18:09.498 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-20T11:18:09.498 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm 2024-04-20T11:18:09.498 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-20T11:18:09.498 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-20T11:18:09.498 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-04-20T11:18:09.498 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto 2024-04-20T11:18:09.498 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-04-20T11:18:09.498 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-04-20T11:18:09.499 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-04-20T11:18:09.499 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx 2024-04-20T11:18:09.499 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-04-20T11:18:09.499 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-04-20T11:18:09.499 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-20T11:18:09.499 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-04-20T11:18:09.499 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-04-20T11:18:09.499 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-04-20T11:18:09.499 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-04-20T11:18:09.499 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-04-20T11:18:09.500 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-04-20T11:18:09.500 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-04-20T11:18:09.500 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-04-20T11:18:09.500 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-04-20T11:18:09.500 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-04-20T11:18:09.500 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-04-20T11:18:09.500 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-04-20T11:18:09.500 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-04-20T11:18:09.500 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-04-20T11:18:09.500 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-04-20T11:18:09.501 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-04-20T11:18:09.501 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-04-20T11:18:09.501 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-04-20T11:18:09.501 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-04-20T11:18:09.501 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-04-20T11:18:09.501 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-04-20T11:18:09.501 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-04-20T11:18:09.501 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-04-20T11:18:09.501 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-04-20T11:18:09.501 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-04-20T11:18:09.501 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-04-20T11:18:09.502 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-04-20T11:18:09.502 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-04-20T11:18:09.502 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-04-20T11:18:09.502 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-04-20T11:18:09.502 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-04-20T11:18:09.502 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-04-20T11:18:09.502 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-04-20T11:18:09.502 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-04-20T11:18:09.502 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-04-20T11:18:09.502 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-04-20T11:18:09.503 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-04-20T11:18:09.503 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-04-20T11:18:09.503 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-04-20T11:18:09.503 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-04-20T11:18:09.503 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-04-20T11:18:09.503 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-04-20T11:18:09.503 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers 2024-04-20T11:18:09.503 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi 2024-04-20T11:18:09.503 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-20T11:18:09.503 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-04-20T11:18:09.503 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-04-20T11:18:09.504 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-04-20T11:18:09.504 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-04-20T11:18:09.504 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-04-20T11:18:09.504 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-04-20T11:18:09.504 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata 2024-04-20T11:18:09.504 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-04-20T11:18:09.504 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-04-20T11:18:09.504 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ata_generic.ko 2024-04-20T11:18:09.504 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-04-20T11:18:09.504 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-04-20T11:18:09.504 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-04-20T11:18:09.505 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-04-20T11:18:09.505 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-04-20T11:18:09.505 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-04-20T11:18:09.505 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-04-20T11:18:09.505 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-04-20T11:18:09.505 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-04-20T11:18:09.505 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-04-20T11:18:09.505 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-04-20T11:18:09.505 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-04-20T11:18:09.505 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-04-20T11:18:09.505 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-20T11:18:09.505 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-20T11:18:09.506 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-04-20T11:18:09.506 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-04-20T11:18:09.506 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-04-20T11:18:09.506 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-04-20T11:18:09.506 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-04-20T11:18:09.506 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-04-20T11:18:09.506 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-04-20T11:18:09.506 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-04-20T11:18:09.506 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-04-20T11:18:09.506 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-04-20T11:18:09.506 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-04-20T11:18:09.506 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-04-20T11:18:09.506 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-04-20T11:18:09.507 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-20T11:18:09.507 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-20T11:18:09.507 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-04-20T11:18:09.507 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-04-20T11:18:09.507 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-04-20T11:18:09.507 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-04-20T11:18:09.507 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-04-20T11:18:09.507 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sis.ko 2024-04-20T11:18:09.507 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-04-20T11:18:09.507 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-04-20T11:18:09.507 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-04-20T11:18:09.507 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-04-20T11:18:09.507 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-04-20T11:18:09.507 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-04-20T11:18:09.508 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-04-20T11:18:09.508 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-04-20T11:18:09.508 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-04-20T11:18:09.508 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-04-20T11:18:09.508 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-04-20T11:18:09.508 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-04-20T11:18:09.508 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-04-20T11:18:09.508 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-04-20T11:18:09.508 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-04-20T11:18:09.508 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-04-20T11:18:09.508 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-04-20T11:18:09.508 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay 2024-04-20T11:18:09.508 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-04-20T11:18:09.509 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base 2024-04-20T11:18:09.509 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap 2024-04-20T11:18:09.509 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-20T11:18:09.509 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-20T11:18:09.509 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma 2024-04-20T11:18:09.509 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-04-20T11:18:09.509 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block 2024-04-20T11:18:09.509 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe 2024-04-20T11:18:09.509 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-04-20T11:18:09.509 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/brd.ko 2024-04-20T11:18:09.509 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd 2024-04-20T11:18:09.509 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-04-20T11:18:09.510 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-04-20T11:18:09.510 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/loop.ko 2024-04-20T11:18:09.510 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx 2024-04-20T11:18:09.510 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-20T11:18:09.510 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-04-20T11:18:09.510 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk 2024-04-20T11:18:09.510 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-04-20T11:18:09.510 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-04-20T11:18:09.510 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-04-20T11:18:09.510 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/virtio_blk.ko 2024-04-20T11:18:09.510 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback 2024-04-20T11:18:09.510 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-20T11:18:09.510 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkfront.ko 2024-04-20T11:18:09.511 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram 2024-04-20T11:18:09.511 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-04-20T11:18:09.511 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char 2024-04-20T11:18:09.511 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-04-20T11:18:09.511 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random 2024-04-20T11:18:09.511 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-20T11:18:09.511 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-20T11:18:09.511 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-20T11:18:09.511 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-04-20T11:18:09.511 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-20T11:18:09.511 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi 2024-04-20T11:18:09.511 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-20T11:18:09.511 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-20T11:18:09.511 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-20T11:18:09.511 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-20T11:18:09.512 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-20T11:18:09.512 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-20T11:18:09.512 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-04-20T11:18:09.512 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave 2024-04-20T11:18:09.512 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-04-20T11:18:09.512 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-04-20T11:18:09.512 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-04-20T11:18:09.512 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm 2024-04-20T11:18:09.512 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm.ko 2024-04-20T11:18:09.512 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-20T11:18:09.512 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-20T11:18:09.512 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-20T11:18:09.512 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-20T11:18:09.513 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-20T11:18:09.513 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/uv_mmtimer.ko 2024-04-20T11:18:09.513 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/virtio_console.ko 2024-04-20T11:18:09.513 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq 2024-04-20T11:18:09.513 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-20T11:18:09.513 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-20T11:18:09.513 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-20T11:18:09.513 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-20T11:18:09.513 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-20T11:18:09.513 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-20T11:18:09.513 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto 2024-04-20T11:18:09.513 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp 2024-04-20T11:18:09.513 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-20T11:18:09.513 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-04-20T11:18:09.514 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel 2024-04-20T11:18:09.514 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat 2024-04-20T11:18:09.514 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common 2024-04-20T11:18:09.514 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-20T11:18:09.514 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-20T11:18:09.514 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-20T11:18:09.514 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-04-20T11:18:09.514 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-04-20T11:18:09.514 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio 2024-04-20T11:18:09.514 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-20T11:18:09.514 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca 2024-04-20T11:18:09.514 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-04-20T11:18:09.515 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq 2024-04-20T11:18:09.515 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-20T11:18:09.515 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma 2024-04-20T11:18:09.515 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw 2024-04-20T11:18:09.515 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-20T11:18:09.515 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat 2024-04-20T11:18:09.515 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-20T11:18:09.515 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac 2024-04-20T11:18:09.515 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-04-20T11:18:09.515 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-04-20T11:18:09.515 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-04-20T11:18:09.515 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-04-20T11:18:09.515 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-04-20T11:18:09.515 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-04-20T11:18:09.516 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-04-20T11:18:09.516 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-04-20T11:18:09.516 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-04-20T11:18:09.516 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-04-20T11:18:09.516 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-04-20T11:18:09.516 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-04-20T11:18:09.516 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-04-20T11:18:09.516 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware 2024-04-20T11:18:09.516 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/edd.ko 2024-04-20T11:18:09.516 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-20T11:18:09.516 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio 2024-04-20T11:18:09.516 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-04-20T11:18:09.516 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-20T11:18:09.517 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid 2024-04-20T11:18:09.517 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-04-20T11:18:09.517 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-04-20T11:18:09.517 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-04-20T11:18:09.517 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-04-20T11:18:09.517 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-04-20T11:18:09.517 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-04-20T11:18:09.517 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-04-20T11:18:09.517 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-04-20T11:18:09.517 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-04-20T11:18:09.517 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-04-20T11:18:09.517 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-20T11:18:09.517 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-20T11:18:09.518 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-04-20T11:18:09.518 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-04-20T11:18:09.518 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-04-20T11:18:09.518 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-04-20T11:18:09.518 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-04-20T11:18:09.518 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-04-20T11:18:09.518 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-04-20T11:18:09.518 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-04-20T11:18:09.518 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-20T11:18:09.518 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-20T11:18:09.518 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-04-20T11:18:09.518 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-04-20T11:18:09.518 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-04-20T11:18:09.519 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-04-20T11:18:09.519 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-04-20T11:18:09.519 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-04-20T11:18:09.519 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-04-20T11:18:09.519 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-04-20T11:18:09.519 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-04-20T11:18:09.519 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-20T11:18:09.519 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-04-20T11:18:09.519 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-20T11:18:09.519 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-20T11:18:09.519 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-20T11:18:09.519 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-20T11:18:09.519 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-20T11:18:09.519 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-20T11:18:09.520 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-20T11:18:09.520 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-20T11:18:09.520 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-20T11:18:09.520 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-04-20T11:18:09.520 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-04-20T11:18:09.520 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-04-20T11:18:09.520 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-20T11:18:09.520 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-04-20T11:18:09.520 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-04-20T11:18:09.520 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-04-20T11:18:09.520 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-04-20T11:18:09.520 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-04-20T11:18:09.520 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-20T11:18:09.521 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-04-20T11:18:09.521 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-04-20T11:18:09.521 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-04-20T11:18:09.521 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-04-20T11:18:09.521 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-04-20T11:18:09.521 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-04-20T11:18:09.521 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-04-20T11:18:09.521 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-04-20T11:18:09.521 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-04-20T11:18:09.521 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-04-20T11:18:09.521 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-04-20T11:18:09.521 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-04-20T11:18:09.522 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv 2024-04-20T11:18:09.522 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-04-20T11:18:09.522 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-04-20T11:18:09.522 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-04-20T11:18:09.522 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon 2024-04-20T11:18:09.522 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-04-20T11:18:09.522 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-04-20T11:18:09.522 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-20T11:18:09.522 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-04-20T11:18:09.522 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-04-20T11:18:09.522 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-04-20T11:18:09.522 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-04-20T11:18:09.522 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-04-20T11:18:09.522 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-04-20T11:18:09.523 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-04-20T11:18:09.523 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-04-20T11:18:09.523 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-04-20T11:18:09.523 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-04-20T11:18:09.523 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-04-20T11:18:09.523 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-04-20T11:18:09.523 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-04-20T11:18:09.523 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-04-20T11:18:09.523 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-04-20T11:18:09.524 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-04-20T11:18:09.524 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-04-20T11:18:09.524 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-04-20T11:18:09.524 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-04-20T11:18:09.524 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-20T11:18:09.524 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-04-20T11:18:09.524 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-04-20T11:18:09.524 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-20T11:18:09.524 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-04-20T11:18:09.525 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-04-20T11:18:09.525 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-04-20T11:18:09.525 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-04-20T11:18:09.525 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-04-20T11:18:09.525 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-04-20T11:18:09.525 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-04-20T11:18:09.525 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-04-20T11:18:09.525 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-04-20T11:18:09.525 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-04-20T11:18:09.525 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-04-20T11:18:09.526 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-04-20T11:18:09.526 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-04-20T11:18:09.526 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-04-20T11:18:09.526 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-20T11:18:09.526 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-04-20T11:18:09.526 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-04-20T11:18:09.526 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-04-20T11:18:09.526 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-04-20T11:18:09.526 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-04-20T11:18:09.526 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-04-20T11:18:09.526 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-04-20T11:18:09.527 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-04-20T11:18:09.527 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-04-20T11:18:09.527 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-04-20T11:18:09.527 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-04-20T11:18:09.527 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-04-20T11:18:09.527 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-04-20T11:18:09.527 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-04-20T11:18:09.527 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-04-20T11:18:09.527 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-04-20T11:18:09.527 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-04-20T11:18:09.527 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-04-20T11:18:09.528 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-04-20T11:18:09.528 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-04-20T11:18:09.528 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-04-20T11:18:09.528 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-04-20T11:18:09.528 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-04-20T11:18:09.528 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-04-20T11:18:09.528 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-04-20T11:18:09.528 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-04-20T11:18:09.528 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-04-20T11:18:09.528 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-04-20T11:18:09.528 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-04-20T11:18:09.529 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-04-20T11:18:09.529 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-04-20T11:18:09.529 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-04-20T11:18:09.529 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-04-20T11:18:09.529 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-04-20T11:18:09.529 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-04-20T11:18:09.529 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-04-20T11:18:09.529 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-04-20T11:18:09.529 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-04-20T11:18:09.529 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-04-20T11:18:09.529 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-04-20T11:18:09.530 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-04-20T11:18:09.530 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-04-20T11:18:09.530 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-04-20T11:18:09.530 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-04-20T11:18:09.530 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-04-20T11:18:09.530 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-04-20T11:18:09.530 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-04-20T11:18:09.530 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus 2024-04-20T11:18:09.530 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-20T11:18:09.530 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-20T11:18:09.531 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-20T11:18:09.531 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-20T11:18:09.531 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-20T11:18:09.531 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-20T11:18:09.531 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-20T11:18:09.531 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-20T11:18:09.531 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-20T11:18:09.531 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-20T11:18:09.531 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-20T11:18:09.531 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-20T11:18:09.531 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-04-20T11:18:09.531 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-04-20T11:18:09.532 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-04-20T11:18:09.532 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-20T11:18:09.532 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-04-20T11:18:09.532 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-04-20T11:18:09.532 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-04-20T11:18:09.532 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-04-20T11:18:09.532 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-04-20T11:18:09.532 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-04-20T11:18:09.532 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-04-20T11:18:09.532 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-04-20T11:18:09.532 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-04-20T11:18:09.533 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-04-20T11:18:09.533 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-04-20T11:18:09.533 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-04-20T11:18:09.533 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-04-20T11:18:09.533 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-04-20T11:18:09.533 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-04-20T11:18:09.533 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-04-20T11:18:09.533 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-04-20T11:18:09.533 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-04-20T11:18:09.533 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-04-20T11:18:09.533 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-04-20T11:18:09.534 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-04-20T11:18:09.534 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-04-20T11:18:09.534 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-04-20T11:18:09.534 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-04-20T11:18:09.534 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-04-20T11:18:09.534 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c 2024-04-20T11:18:09.534 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos 2024-04-20T11:18:09.534 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-20T11:18:09.534 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-20T11:18:09.534 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses 2024-04-20T11:18:09.534 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-20T11:18:09.534 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-20T11:18:09.535 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-20T11:18:09.535 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-20T11:18:09.535 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-20T11:18:09.535 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-20T11:18:09.535 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-20T11:18:09.535 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-20T11:18:09.535 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-20T11:18:09.535 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-20T11:18:09.535 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-20T11:18:09.535 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-20T11:18:09.535 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-20T11:18:09.535 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-20T11:18:09.535 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-20T11:18:09.536 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-20T11:18:09.536 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-20T11:18:09.536 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-20T11:18:09.536 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-20T11:18:09.536 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-20T11:18:09.536 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-20T11:18:09.536 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-20T11:18:09.536 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-20T11:18:09.536 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-dev.ko 2024-04-20T11:18:09.536 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-04-20T11:18:09.536 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-20T11:18:09.536 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-04-20T11:18:09.536 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-04-20T11:18:09.536 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband 2024-04-20T11:18:09.537 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core 2024-04-20T11:18:09.537 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-20T11:18:09.537 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-04-20T11:18:09.537 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-20T11:18:09.537 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-20T11:18:09.537 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-20T11:18:09.537 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-20T11:18:09.537 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-20T11:18:09.537 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw 2024-04-20T11:18:09.537 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4 2024-04-20T11:18:09.537 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-20T11:18:09.537 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4 2024-04-20T11:18:09.538 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-20T11:18:09.538 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5 2024-04-20T11:18:09.538 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-20T11:18:09.538 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca 2024-04-20T11:18:09.538 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-20T11:18:09.538 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp 2024-04-20T11:18:09.538 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib 2024-04-20T11:18:09.538 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-20T11:18:09.538 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser 2024-04-20T11:18:09.538 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-20T11:18:09.539 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert 2024-04-20T11:18:09.539 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-20T11:18:09.539 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp 2024-04-20T11:18:09.539 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-20T11:18:09.539 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt 2024-04-20T11:18:09.539 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-20T11:18:09.539 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input 2024-04-20T11:18:09.539 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport 2024-04-20T11:18:09.539 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-20T11:18:09.539 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-20T11:18:09.539 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-04-20T11:18:09.539 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-04-20T11:18:09.539 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard 2024-04-20T11:18:09.540 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-20T11:18:09.540 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse 2024-04-20T11:18:09.540 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-04-20T11:18:09.540 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-04-20T11:18:09.540 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-04-20T11:18:09.540 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-20T11:18:09.540 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-04-20T11:18:09.540 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-20T11:18:09.540 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-20T11:18:09.540 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-20T11:18:09.540 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4 2024-04-20T11:18:09.540 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-20T11:18:09.540 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio 2024-04-20T11:18:09.540 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-04-20T11:18:09.541 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-04-20T11:18:09.541 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-20T11:18:09.541 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-04-20T11:18:09.541 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-04-20T11:18:09.541 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds 2024-04-20T11:18:09.541 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-04-20T11:18:09.541 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-04-20T11:18:09.541 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-04-20T11:18:09.541 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-04-20T11:18:09.541 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger 2024-04-20T11:18:09.541 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-20T11:18:09.541 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-20T11:18:09.541 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-20T11:18:09.542 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-20T11:18:09.542 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-20T11:18:09.542 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-20T11:18:09.542 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-20T11:18:09.542 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-20T11:18:09.542 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-20T11:18:09.542 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md 2024-04-20T11:18:09.542 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache 2024-04-20T11:18:09.542 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-04-20T11:18:09.542 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-04-20T11:18:09.542 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-04-20T11:18:09.542 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-04-20T11:18:09.542 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-04-20T11:18:09.543 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-04-20T11:18:09.543 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-04-20T11:18:09.543 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-04-20T11:18:09.543 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-04-20T11:18:09.543 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-04-20T11:18:09.543 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-04-20T11:18:09.543 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-04-20T11:18:09.543 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-04-20T11:18:09.543 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-04-20T11:18:09.543 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-04-20T11:18:09.543 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-04-20T11:18:09.543 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data 2024-04-20T11:18:09.543 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-20T11:18:09.543 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-04-20T11:18:09.544 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-04-20T11:18:09.544 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-04-20T11:18:09.544 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-04-20T11:18:09.544 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message 2024-04-20T11:18:09.544 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion 2024-04-20T11:18:09.544 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-04-20T11:18:09.544 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-04-20T11:18:09.544 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-04-20T11:18:09.544 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-04-20T11:18:09.544 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-04-20T11:18:09.544 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-04-20T11:18:09.544 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-04-20T11:18:09.544 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd 2024-04-20T11:18:09.544 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-04-20T11:18:09.545 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-04-20T11:18:09.545 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mfd-core.ko 2024-04-20T11:18:09.545 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-04-20T11:18:09.545 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-04-20T11:18:09.545 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-04-20T11:18:09.545 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-04-20T11:18:09.545 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc 2024-04-20T11:18:09.545 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl 2024-04-20T11:18:09.545 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-20T11:18:09.545 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-04-20T11:18:09.545 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-04-20T11:18:09.545 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-04-20T11:18:09.545 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710 2024-04-20T11:18:09.545 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-04-20T11:18:09.546 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom 2024-04-20T11:18:09.546 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-04-20T11:18:09.546 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-20T11:18:09.546 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-04-20T11:18:09.546 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-04-20T11:18:09.546 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-04-20T11:18:09.546 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-04-20T11:18:09.546 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-04-20T11:18:09.546 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d 2024-04-20T11:18:09.546 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-20T11:18:09.546 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-20T11:18:09.546 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei 2024-04-20T11:18:09.546 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-04-20T11:18:09.546 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-04-20T11:18:09.547 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-04-20T11:18:09.547 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru 2024-04-20T11:18:09.547 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-20T11:18:09.547 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp 2024-04-20T11:18:09.547 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-20T11:18:09.547 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-20T11:18:09.547 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-20T11:18:09.547 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-04-20T11:18:09.547 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-04-20T11:18:09.547 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-04-20T11:18:09.547 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-04-20T11:18:09.547 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci 2024-04-20T11:18:09.547 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-20T11:18:09.547 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net 2024-04-20T11:18:09.548 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding 2024-04-20T11:18:09.548 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-04-20T11:18:09.548 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa 2024-04-20T11:18:09.548 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53 2024-04-20T11:18:09.548 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-20T11:18:09.548 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-20T11:18:09.548 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-20T11:18:09.548 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx 2024-04-20T11:18:09.548 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-20T11:18:09.548 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-04-20T11:18:09.548 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-04-20T11:18:09.548 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet 2024-04-20T11:18:09.548 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com 2024-04-20T11:18:09.548 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-20T11:18:09.549 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-20T11:18:09.549 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-20T11:18:09.549 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-20T11:18:09.549 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390 2024-04-20T11:18:09.549 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-20T11:18:09.549 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-20T11:18:09.549 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-20T11:18:09.549 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-20T11:18:09.549 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec 2024-04-20T11:18:09.549 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-20T11:18:09.549 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere 2024-04-20T11:18:09.549 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-20T11:18:09.549 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon 2024-04-20T11:18:09.549 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-20T11:18:09.549 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera 2024-04-20T11:18:09.550 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-20T11:18:09.550 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd 2024-04-20T11:18:09.550 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-20T11:18:09.550 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-20T11:18:09.550 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-20T11:18:09.550 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros 2024-04-20T11:18:09.550 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx 2024-04-20T11:18:09.550 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-20T11:18:09.550 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-20T11:18:09.550 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-20T11:18:09.550 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-20T11:18:09.550 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-20T11:18:09.550 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx 2024-04-20T11:18:09.550 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-20T11:18:09.551 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-20T11:18:09.551 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom 2024-04-20T11:18:09.551 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-20T11:18:09.551 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-20T11:18:09.551 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-20T11:18:09.551 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-20T11:18:09.551 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-20T11:18:09.551 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet 2024-04-20T11:18:09.551 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-20T11:18:09.551 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-20T11:18:09.551 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade 2024-04-20T11:18:09.551 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna 2024-04-20T11:18:09.551 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-20T11:18:09.551 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio 2024-04-20T11:18:09.551 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-20T11:18:09.552 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-20T11:18:09.552 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-20T11:18:09.552 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-20T11:18:09.552 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-20T11:18:09.552 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-20T11:18:09.552 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-20T11:18:09.552 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-20T11:18:09.552 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-20T11:18:09.552 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-20T11:18:09.552 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco 2024-04-20T11:18:09.552 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic 2024-04-20T11:18:09.552 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-20T11:18:09.552 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec 2024-04-20T11:18:09.552 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip 2024-04-20T11:18:09.552 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-20T11:18:09.553 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-20T11:18:09.553 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-20T11:18:09.553 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-20T11:18:09.553 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-20T11:18:09.553 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-20T11:18:09.553 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink 2024-04-20T11:18:09.553 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-20T11:18:09.553 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-20T11:18:09.553 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-04-20T11:18:09.553 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex 2024-04-20T11:18:09.553 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet 2024-04-20T11:18:09.553 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-20T11:18:09.553 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-04-20T11:18:09.553 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-04-20T11:18:09.553 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel 2024-04-20T11:18:09.554 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-20T11:18:09.554 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000 2024-04-20T11:18:09.554 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-20T11:18:09.554 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e 2024-04-20T11:18:09.554 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-20T11:18:09.554 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k 2024-04-20T11:18:09.554 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-20T11:18:09.554 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e 2024-04-20T11:18:09.554 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-20T11:18:09.554 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf 2024-04-20T11:18:09.554 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-20T11:18:09.554 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb 2024-04-20T11:18:09.554 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-20T11:18:09.554 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf 2024-04-20T11:18:09.555 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-20T11:18:09.555 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-20T11:18:09.555 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-20T11:18:09.558 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-20T11:18:09.558 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-20T11:18:09.558 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-04-20T11:18:09.558 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell 2024-04-20T11:18:09.558 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-20T11:18:09.558 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-20T11:18:09.558 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-20T11:18:09.558 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox 2024-04-20T11:18:09.558 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-20T11:18:09.558 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-20T11:18:09.558 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-20T11:18:09.558 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-20T11:18:09.559 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-20T11:18:09.559 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-20T11:18:09.559 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel 2024-04-20T11:18:09.559 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-20T11:18:09.559 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom 2024-04-20T11:18:09.559 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-20T11:18:09.559 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-20T11:18:09.559 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi 2024-04-20T11:18:09.559 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-20T11:18:09.559 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-20T11:18:09.559 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion 2024-04-20T11:18:09.559 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-20T11:18:09.559 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia 2024-04-20T11:18:09.559 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-20T11:18:09.559 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines 2024-04-20T11:18:09.560 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-20T11:18:09.560 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-20T11:18:09.560 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic 2024-04-20T11:18:09.560 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-20T11:18:09.560 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-20T11:18:09.560 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-20T11:18:09.560 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-20T11:18:09.560 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-20T11:18:09.560 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc 2024-04-20T11:18:09.560 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-20T11:18:09.560 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek 2024-04-20T11:18:09.560 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-20T11:18:09.560 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-20T11:18:09.560 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-20T11:18:09.561 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-20T11:18:09.561 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker 2024-04-20T11:18:09.561 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-20T11:18:09.561 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc 2024-04-20T11:18:09.561 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-20T11:18:09.561 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan 2024-04-20T11:18:09.561 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-20T11:18:09.561 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis 2024-04-20T11:18:09.561 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-20T11:18:09.561 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-20T11:18:09.561 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc 2024-04-20T11:18:09.561 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-20T11:18:09.561 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-20T11:18:09.561 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-20T11:18:09.562 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-20T11:18:09.562 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro 2024-04-20T11:18:09.562 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-20T11:18:09.562 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-20T11:18:09.562 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-20T11:18:09.562 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun 2024-04-20T11:18:09.562 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-20T11:18:09.562 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-20T11:18:09.562 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-20T11:18:09.562 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-20T11:18:09.562 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti 2024-04-20T11:18:09.562 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-20T11:18:09.562 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti 2024-04-20T11:18:09.562 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-20T11:18:09.562 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via 2024-04-20T11:18:09.563 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-20T11:18:09.563 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-20T11:18:09.563 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet 2024-04-20T11:18:09.563 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-20T11:18:09.563 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-20T11:18:09.563 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom 2024-04-20T11:18:09.563 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-20T11:18:09.563 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-04-20T11:18:09.563 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv 2024-04-20T11:18:09.563 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-20T11:18:09.563 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154 2024-04-20T11:18:09.563 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-20T11:18:09.563 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-04-20T11:18:09.563 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan 2024-04-20T11:18:09.563 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-20T11:18:09.564 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-04-20T11:18:09.564 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-04-20T11:18:09.564 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio 2024-04-20T11:18:09.564 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-04-20T11:18:09.564 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-20T11:18:09.564 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-20T11:18:09.564 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-20T11:18:09.564 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-04-20T11:18:09.564 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/net_failover.ko 2024-04-20T11:18:09.564 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-04-20T11:18:09.564 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-04-20T11:18:09.564 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs 2024-04-20T11:18:09.564 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-20T11:18:09.564 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-20T11:18:09.565 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy 2024-04-20T11:18:09.565 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-04-20T11:18:09.565 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-20T11:18:09.565 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-20T11:18:09.565 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-20T11:18:09.565 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-04-20T11:18:09.565 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-04-20T11:18:09.565 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-04-20T11:18:09.565 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-04-20T11:18:09.565 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/dp83640.ko 2024-04-20T11:18:09.565 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-04-20T11:18:09.565 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-04-20T11:18:09.565 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-04-20T11:18:09.566 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-04-20T11:18:09.566 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-04-20T11:18:09.566 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-04-20T11:18:09.566 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-04-20T11:18:09.566 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-04-20T11:18:09.566 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-04-20T11:18:09.566 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-04-20T11:18:09.566 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-04-20T11:18:09.566 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-04-20T11:18:09.566 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp 2024-04-20T11:18:09.566 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-20T11:18:09.567 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-04-20T11:18:09.567 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-20T11:18:09.567 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-20T11:18:09.567 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-20T11:18:09.567 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-20T11:18:09.567 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-04-20T11:18:09.567 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-04-20T11:18:09.567 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-04-20T11:18:09.567 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip 2024-04-20T11:18:09.567 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slhc.ko 2024-04-20T11:18:09.567 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-04-20T11:18:09.567 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-04-20T11:18:09.567 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-04-20T11:18:09.568 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team 2024-04-20T11:18:09.568 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-04-20T11:18:09.568 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-20T11:18:09.568 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-20T11:18:09.568 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-20T11:18:09.568 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-04-20T11:18:09.568 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-20T11:18:09.568 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tun.ko 2024-04-20T11:18:09.568 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-04-20T11:18:09.568 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/virtio_net.ko 2024-04-20T11:18:09.568 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3 2024-04-20T11:18:09.568 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-20T11:18:09.568 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan 2024-04-20T11:18:09.568 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-04-20T11:18:09.569 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback 2024-04-20T11:18:09.569 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-20T11:18:09.569 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netfront.ko 2024-04-20T11:18:09.569 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb 2024-04-20T11:18:09.569 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-04-20T11:18:09.569 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme 2024-04-20T11:18:09.569 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host 2024-04-20T11:18:09.569 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-04-20T11:18:09.569 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-04-20T11:18:09.569 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport 2024-04-20T11:18:09.569 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-04-20T11:18:09.569 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-04-20T11:18:09.569 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-04-20T11:18:09.569 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-04-20T11:18:09.570 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci 2024-04-20T11:18:09.570 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug 2024-04-20T11:18:09.570 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-20T11:18:09.570 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie 2024-04-20T11:18:09.570 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-20T11:18:09.570 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-04-20T11:18:09.570 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia 2024-04-20T11:18:09.570 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-04-20T11:18:09.570 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-04-20T11:18:09.570 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-20T11:18:09.570 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl 2024-04-20T11:18:09.570 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel 2024-04-20T11:18:09.570 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-20T11:18:09.571 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform 2024-04-20T11:18:09.571 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86 2024-04-20T11:18:09.571 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-20T11:18:09.571 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-04-20T11:18:09.571 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-20T11:18:09.571 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-20T11:18:09.571 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-20T11:18:09.571 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-20T11:18:09.571 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-20T11:18:09.571 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-20T11:18:09.571 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-20T11:18:09.571 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-20T11:18:09.571 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-20T11:18:09.571 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-04-20T11:18:09.571 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel 2024-04-20T11:18:09.572 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-20T11:18:09.572 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-04-20T11:18:09.572 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-20T11:18:09.572 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-20T11:18:09.572 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-20T11:18:09.572 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-20T11:18:09.572 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-20T11:18:09.572 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-20T11:18:09.572 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-20T11:18:09.572 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-20T11:18:09.572 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-20T11:18:09.572 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-04-20T11:18:09.572 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap 2024-04-20T11:18:09.573 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-20T11:18:09.573 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-20T11:18:09.573 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps 2024-04-20T11:18:09.573 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients 2024-04-20T11:18:09.573 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-20T11:18:09.573 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-20T11:18:09.573 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-04-20T11:18:09.573 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-04-20T11:18:09.573 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp 2024-04-20T11:18:09.573 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-04-20T11:18:09.573 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-04-20T11:18:09.573 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras 2024-04-20T11:18:09.573 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd 2024-04-20T11:18:09.574 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl 2024-04-20T11:18:09.574 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-20T11:18:09.574 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-04-20T11:18:09.574 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc 2024-04-20T11:18:09.574 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-20T11:18:09.574 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-20T11:18:09.574 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-20T11:18:09.574 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-20T11:18:09.574 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-20T11:18:09.574 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-20T11:18:09.574 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-20T11:18:09.574 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-20T11:18:09.575 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-20T11:18:09.575 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-20T11:18:09.575 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-04-20T11:18:09.575 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-20T11:18:09.575 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-20T11:18:09.575 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-20T11:18:09.575 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-20T11:18:09.575 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-20T11:18:09.575 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-20T11:18:09.575 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-04-20T11:18:09.575 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-20T11:18:09.575 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-20T11:18:09.575 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-20T11:18:09.575 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-20T11:18:09.575 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-20T11:18:09.576 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-20T11:18:09.576 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-20T11:18:09.576 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-20T11:18:09.576 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-20T11:18:09.576 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-20T11:18:09.576 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-20T11:18:09.576 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-20T11:18:09.576 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-04-20T11:18:09.576 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi 2024-04-20T11:18:09.576 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-04-20T11:18:09.576 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-04-20T11:18:09.576 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-04-20T11:18:09.576 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-04-20T11:18:09.576 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-04-20T11:18:09.577 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid 2024-04-20T11:18:09.577 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-20T11:18:09.577 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-04-20T11:18:09.577 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx 2024-04-20T11:18:09.577 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-20T11:18:09.577 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-20T11:18:09.577 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-04-20T11:18:09.577 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr 2024-04-20T11:18:09.577 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-20T11:18:09.577 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-04-20T11:18:09.577 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi 2024-04-20T11:18:09.577 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-20T11:18:09.577 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa 2024-04-20T11:18:09.577 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-04-20T11:18:09.578 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc 2024-04-20T11:18:09.578 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-20T11:18:09.578 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i 2024-04-20T11:18:09.578 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-20T11:18:09.578 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-04-20T11:18:09.578 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor 2024-04-20T11:18:09.578 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-20T11:18:09.578 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi 2024-04-20T11:18:09.578 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-20T11:18:09.578 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-20T11:18:09.578 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-20T11:18:09.578 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-20T11:18:09.578 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-20T11:18:09.579 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-04-20T11:18:09.579 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler 2024-04-20T11:18:09.579 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-20T11:18:09.579 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-20T11:18:09.579 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-20T11:18:09.579 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-20T11:18:09.579 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-04-20T11:18:09.579 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r 2024-04-20T11:18:09.579 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-20T11:18:09.579 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-04-20T11:18:09.579 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe 2024-04-20T11:18:09.579 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-20T11:18:09.579 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-20T11:18:09.579 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic 2024-04-20T11:18:09.580 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-04-20T11:18:09.580 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-04-20T11:18:09.580 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-04-20T11:18:09.580 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-04-20T11:18:09.580 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-04-20T11:18:09.580 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-04-20T11:18:09.580 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-04-20T11:18:09.580 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci 2024-04-20T11:18:09.580 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-04-20T11:18:09.580 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-20T11:18:09.580 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-20T11:18:09.580 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc 2024-04-20T11:18:09.580 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-04-20T11:18:09.581 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-04-20T11:18:09.581 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-20T11:18:09.581 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas 2024-04-20T11:18:09.581 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-04-20T11:18:09.581 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc 2024-04-20T11:18:09.581 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-20T11:18:09.581 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid 2024-04-20T11:18:09.581 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-04-20T11:18:09.581 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-20T11:18:09.581 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-20T11:18:09.581 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-20T11:18:09.581 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas 2024-04-20T11:18:09.581 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-20T11:18:09.582 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas 2024-04-20T11:18:09.582 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-20T11:18:09.582 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-04-20T11:18:09.582 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001 2024-04-20T11:18:09.582 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-20T11:18:09.582 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-04-20T11:18:09.582 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-04-20T11:18:09.582 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx 2024-04-20T11:18:09.582 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-20T11:18:09.582 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-20T11:18:09.582 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx 2024-04-20T11:18:09.582 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-20T11:18:09.582 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-04-20T11:18:09.582 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-04-20T11:18:09.583 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-20T11:18:09.583 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-20T11:18:09.583 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-20T11:18:09.583 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-20T11:18:09.583 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-20T11:18:09.583 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-04-20T11:18:09.583 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-04-20T11:18:09.583 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-04-20T11:18:09.583 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2 2024-04-20T11:18:09.583 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-20T11:18:09.583 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-04-20T11:18:09.583 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-20T11:18:09.583 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-04-20T11:18:09.583 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb 2024-04-20T11:18:09.584 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-04-20T11:18:09.584 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target 2024-04-20T11:18:09.584 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi 2024-04-20T11:18:09.584 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-20T11:18:09.584 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback 2024-04-20T11:18:09.584 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-20T11:18:09.584 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-04-20T11:18:09.584 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-04-20T11:18:09.584 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-04-20T11:18:09.584 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-04-20T11:18:09.584 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-04-20T11:18:09.584 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc 2024-04-20T11:18:09.584 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-20T11:18:09.584 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal 2024-04-20T11:18:09.584 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel 2024-04-20T11:18:09.585 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal 2024-04-20T11:18:09.585 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-20T11:18:09.585 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-20T11:18:09.585 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-20T11:18:09.585 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-20T11:18:09.585 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-20T11:18:09.585 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-20T11:18:09.585 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-20T11:18:09.585 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-20T11:18:09.585 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-20T11:18:09.585 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-20T11:18:09.585 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-20T11:18:09.585 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-20T11:18:09.586 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-20T11:18:09.586 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-20T11:18:09.586 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-20T11:18:09.586 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-20T11:18:09.586 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-20T11:18:09.586 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-20T11:18:09.586 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty 2024-04-20T11:18:09.586 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless 2024-04-20T11:18:09.586 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-20T11:18:09.586 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-04-20T11:18:09.586 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-04-20T11:18:09.586 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-04-20T11:18:09.587 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial 2024-04-20T11:18:09.587 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250 2024-04-20T11:18:09.587 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-20T11:18:09.587 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-04-20T11:18:09.587 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm 2024-04-20T11:18:09.587 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-20T11:18:09.587 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs 2024-04-20T11:18:09.587 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core 2024-04-20T11:18:09.587 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-20T11:18:09.587 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host 2024-04-20T11:18:09.587 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-20T11:18:09.587 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio 2024-04-20T11:18:09.587 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-04-20T11:18:09.587 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-04-20T11:18:09.588 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-04-20T11:18:09.588 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-04-20T11:18:09.588 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-04-20T11:18:09.588 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb 2024-04-20T11:18:09.588 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class 2024-04-20T11:18:09.588 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-04-20T11:18:09.588 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-20T11:18:09.588 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-04-20T11:18:09.588 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-04-20T11:18:09.588 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host 2024-04-20T11:18:09.588 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-20T11:18:09.588 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image 2024-04-20T11:18:09.588 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-04-20T11:18:09.588 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-04-20T11:18:09.588 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc 2024-04-20T11:18:09.589 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-04-20T11:18:09.589 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-04-20T11:18:09.589 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-04-20T11:18:09.589 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-04-20T11:18:09.589 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-04-20T11:18:09.589 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-04-20T11:18:09.589 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-04-20T11:18:09.589 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-20T11:18:09.589 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-04-20T11:18:09.589 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-04-20T11:18:09.589 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga 2024-04-20T11:18:09.589 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-20T11:18:09.589 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-20T11:18:09.589 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-04-20T11:18:09.589 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-04-20T11:18:09.590 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-20T11:18:09.590 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-04-20T11:18:09.590 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-04-20T11:18:09.590 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy 2024-04-20T11:18:09.590 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-04-20T11:18:09.590 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial 2024-04-20T11:18:09.590 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-04-20T11:18:09.590 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-04-20T11:18:09.590 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-20T11:18:09.590 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-04-20T11:18:09.590 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-04-20T11:18:09.590 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-04-20T11:18:09.590 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-20T11:18:09.590 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-20T11:18:09.591 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-04-20T11:18:09.591 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-20T11:18:09.591 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-20T11:18:09.591 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-20T11:18:09.591 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-04-20T11:18:09.591 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-04-20T11:18:09.591 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-04-20T11:18:09.591 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-04-20T11:18:09.591 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-20T11:18:09.591 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-04-20T11:18:09.591 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-20T11:18:09.591 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-20T11:18:09.591 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-20T11:18:09.591 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-04-20T11:18:09.591 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-04-20T11:18:09.591 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-04-20T11:18:09.592 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-04-20T11:18:09.592 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-04-20T11:18:09.592 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-04-20T11:18:09.592 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-04-20T11:18:09.592 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-04-20T11:18:09.592 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-04-20T11:18:09.592 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-04-20T11:18:09.592 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-04-20T11:18:09.592 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-04-20T11:18:09.592 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-04-20T11:18:09.592 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-04-20T11:18:09.592 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-20T11:18:09.592 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-04-20T11:18:09.592 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-04-20T11:18:09.592 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-20T11:18:09.593 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-20T11:18:09.593 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-04-20T11:18:09.593 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-20T11:18:09.593 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-04-20T11:18:09.593 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-04-20T11:18:09.593 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-20T11:18:09.593 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage 2024-04-20T11:18:09.593 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-04-20T11:18:09.593 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-20T11:18:09.593 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-20T11:18:09.593 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-20T11:18:09.593 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-20T11:18:09.593 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-20T11:18:09.593 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-20T11:18:09.594 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-20T11:18:09.594 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-04-20T11:18:09.594 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-20T11:18:09.594 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-20T11:18:09.594 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-20T11:18:09.594 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-20T11:18:09.594 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-20T11:18:09.594 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-04-20T11:18:09.594 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip 2024-04-20T11:18:09.594 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-20T11:18:09.594 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-20T11:18:09.594 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-20T11:18:09.594 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio 2024-04-20T11:18:09.594 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci 2024-04-20T11:18:09.594 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-20T11:18:09.595 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-20T11:18:09.595 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-04-20T11:18:09.595 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-20T11:18:09.595 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost 2024-04-20T11:18:09.595 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-04-20T11:18:09.595 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-20T11:18:09.595 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-04-20T11:18:09.595 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-04-20T11:18:09.595 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video 2024-04-20T11:18:09.595 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight 2024-04-20T11:18:09.595 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-04-20T11:18:09.595 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-04-20T11:18:09.595 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-20T11:18:09.595 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio 2024-04-20T11:18:09.596 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio.ko 2024-04-20T11:18:09.596 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_balloon.ko 2024-04-20T11:18:09.596 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_mmio.ko 2024-04-20T11:18:09.596 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci.ko 2024-04-20T11:18:09.596 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-20T11:18:09.596 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-20T11:18:09.596 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_ring.ko 2024-04-20T11:18:09.596 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1 2024-04-20T11:18:09.596 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters 2024-04-20T11:18:09.596 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-04-20T11:18:09.596 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-04-20T11:18:09.596 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves 2024-04-20T11:18:09.596 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-20T11:18:09.596 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-20T11:18:09.596 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-20T11:18:09.597 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-20T11:18:09.597 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-20T11:18:09.597 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-20T11:18:09.597 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-20T11:18:09.597 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-20T11:18:09.597 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-20T11:18:09.597 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-20T11:18:09.597 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-20T11:18:09.597 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-04-20T11:18:09.597 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog 2024-04-20T11:18:09.597 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-20T11:18:09.597 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-20T11:18:09.597 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-20T11:18:09.597 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-04-20T11:18:09.598 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-04-20T11:18:09.598 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-20T11:18:09.598 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-20T11:18:09.598 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-04-20T11:18:09.598 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-04-20T11:18:09.598 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-20T11:18:09.598 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-20T11:18:09.598 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-04-20T11:18:09.598 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-04-20T11:18:09.598 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-04-20T11:18:09.598 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-20T11:18:09.598 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-20T11:18:09.598 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-20T11:18:09.598 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-20T11:18:09.598 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-04-20T11:18:09.599 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-20T11:18:09.599 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-04-20T11:18:09.599 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-20T11:18:09.599 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-20T11:18:09.599 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-20T11:18:09.599 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-04-20T11:18:09.599 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-04-20T11:18:09.599 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen 2024-04-20T11:18:09.599 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-20T11:18:09.599 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-04-20T11:18:09.599 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-04-20T11:18:09.599 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-04-20T11:18:09.599 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback 2024-04-20T11:18:09.599 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-20T11:18:09.600 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-04-20T11:18:09.600 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs 2024-04-20T11:18:09.600 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-20T11:18:09.600 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs 2024-04-20T11:18:09.600 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p 2024-04-20T11:18:09.600 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-04-20T11:18:09.600 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs 2024-04-20T11:18:09.600 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-04-20T11:18:09.600 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs 2024-04-20T11:18:09.600 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-04-20T11:18:09.600 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-04-20T11:18:09.600 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs 2024-04-20T11:18:09.600 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-04-20T11:18:09.601 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles 2024-04-20T11:18:09.601 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-04-20T11:18:09.601 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph 2024-04-20T11:18:09.601 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-04-20T11:18:09.601 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda 2024-04-20T11:18:09.601 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-04-20T11:18:09.601 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs 2024-04-20T11:18:09.601 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-04-20T11:18:09.601 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm 2024-04-20T11:18:09.601 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-04-20T11:18:09.601 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs 2024-04-20T11:18:09.602 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-20T11:18:09.602 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs 2024-04-20T11:18:09.602 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-04-20T11:18:09.602 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat 2024-04-20T11:18:09.602 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/fat.ko 2024-04-20T11:18:09.602 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-04-20T11:18:09.602 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/vfat.ko 2024-04-20T11:18:09.602 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse 2024-04-20T11:18:09.602 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-04-20T11:18:09.602 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/fuse.ko 2024-04-20T11:18:09.602 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2 2024-04-20T11:18:09.602 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-04-20T11:18:09.602 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs 2024-04-20T11:18:09.602 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-04-20T11:18:09.603 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus 2024-04-20T11:18:09.603 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-04-20T11:18:09.603 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs 2024-04-20T11:18:09.603 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-04-20T11:18:09.603 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs 2024-04-20T11:18:09.603 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-04-20T11:18:09.603 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd 2024-04-20T11:18:09.603 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-04-20T11:18:09.603 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix 2024-04-20T11:18:09.603 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-04-20T11:18:09.603 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs 2024-04-20T11:18:09.603 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-04-20T11:18:09.603 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs 2024-04-20T11:18:09.603 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout 2024-04-20T11:18:09.604 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-20T11:18:09.604 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout 2024-04-20T11:18:09.604 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-20T11:18:09.604 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout 2024-04-20T11:18:09.604 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-20T11:18:09.604 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-04-20T11:18:09.604 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-04-20T11:18:09.604 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-04-20T11:18:09.604 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common 2024-04-20T11:18:09.604 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-04-20T11:18:09.604 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-04-20T11:18:09.604 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd 2024-04-20T11:18:09.604 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-04-20T11:18:09.604 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2 2024-04-20T11:18:09.605 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-04-20T11:18:09.605 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls 2024-04-20T11:18:09.605 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-04-20T11:18:09.605 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-04-20T11:18:09.605 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-04-20T11:18:09.605 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-04-20T11:18:09.605 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-04-20T11:18:09.605 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-04-20T11:18:09.605 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-04-20T11:18:09.605 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-04-20T11:18:09.605 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-04-20T11:18:09.605 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-04-20T11:18:09.605 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-04-20T11:18:09.605 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-04-20T11:18:09.605 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-04-20T11:18:09.606 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-04-20T11:18:09.606 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-04-20T11:18:09.606 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-04-20T11:18:09.606 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-04-20T11:18:09.606 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-04-20T11:18:09.606 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-04-20T11:18:09.606 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-04-20T11:18:09.606 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-04-20T11:18:09.606 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-04-20T11:18:09.606 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-04-20T11:18:09.606 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-04-20T11:18:09.606 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-04-20T11:18:09.606 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-04-20T11:18:09.606 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-04-20T11:18:09.607 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-04-20T11:18:09.607 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-04-20T11:18:09.607 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-04-20T11:18:09.607 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-04-20T11:18:09.607 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-04-20T11:18:09.607 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-04-20T11:18:09.607 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-04-20T11:18:09.607 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-04-20T11:18:09.607 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-04-20T11:18:09.607 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-04-20T11:18:09.607 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-04-20T11:18:09.607 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-04-20T11:18:09.607 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-04-20T11:18:09.607 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-04-20T11:18:09.608 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-04-20T11:18:09.608 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-04-20T11:18:09.608 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-04-20T11:18:09.608 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-04-20T11:18:09.608 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-04-20T11:18:09.608 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-04-20T11:18:09.608 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-04-20T11:18:09.608 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-20T11:18:09.608 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-04-20T11:18:09.608 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2 2024-04-20T11:18:09.608 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster 2024-04-20T11:18:09.608 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-20T11:18:09.608 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm 2024-04-20T11:18:09.608 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-20T11:18:09.608 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs 2024-04-20T11:18:09.609 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-20T11:18:09.609 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-04-20T11:18:09.609 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-20T11:18:09.609 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-20T11:18:09.609 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-20T11:18:09.609 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs 2024-04-20T11:18:09.609 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-04-20T11:18:09.609 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore 2024-04-20T11:18:09.609 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-04-20T11:18:09.609 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs 2024-04-20T11:18:09.609 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-04-20T11:18:09.609 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs 2024-04-20T11:18:09.609 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-04-20T11:18:09.609 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb 2024-04-20T11:18:09.610 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client 2024-04-20T11:18:09.610 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-04-20T11:18:09.610 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common 2024-04-20T11:18:09.610 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-04-20T11:18:09.610 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-04-20T11:18:09.610 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs 2024-04-20T11:18:09.610 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-04-20T11:18:09.610 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv 2024-04-20T11:18:09.610 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-04-20T11:18:09.610 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf 2024-04-20T11:18:09.610 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-04-20T11:18:09.610 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs 2024-04-20T11:18:09.610 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-04-20T11:18:09.610 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs 2024-04-20T11:18:09.610 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-04-20T11:18:09.611 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel 2024-04-20T11:18:09.611 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu 2024-04-20T11:18:09.611 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu/rcutorture.ko 2024-04-20T11:18:09.611 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-04-20T11:18:09.611 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace 2024-04-20T11:18:09.611 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-20T11:18:09.611 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib 2024-04-20T11:18:09.611 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/asn1_encoder.ko 2024-04-20T11:18:09.611 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-04-20T11:18:09.611 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-04-20T11:18:09.611 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto 2024-04-20T11:18:09.611 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-04-20T11:18:09.611 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-04-20T11:18:09.611 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-04-20T11:18:09.612 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-04-20T11:18:09.612 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4 2024-04-20T11:18:09.612 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-04-20T11:18:09.612 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-04-20T11:18:09.612 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math 2024-04-20T11:18:09.612 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-04-20T11:18:09.612 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6 2024-04-20T11:18:09.612 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-04-20T11:18:09.612 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon 2024-04-20T11:18:09.612 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-20T11:18:09.612 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-04-20T11:18:09.612 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-04-20T11:18:09.612 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-04-20T11:18:09.612 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd 2024-04-20T11:18:09.613 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd/zstd_compress.ko 2024-04-20T11:18:09.613 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm 2024-04-20T11:18:09.613 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-04-20T11:18:09.613 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net 2024-04-20T11:18:09.613 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan 2024-04-20T11:18:09.613 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-04-20T11:18:09.613 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-04-20T11:18:09.613 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-04-20T11:18:09.613 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-04-20T11:18:09.613 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-20T11:18:09.613 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-04-20T11:18:09.613 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-04-20T11:18:09.614 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-04-20T11:18:09.614 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802 2024-04-20T11:18:09.614 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-04-20T11:18:09.614 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-04-20T11:18:09.614 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-04-20T11:18:09.614 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-04-20T11:18:09.614 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-04-20T11:18:09.614 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q 2024-04-20T11:18:09.614 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-04-20T11:18:09.614 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p 2024-04-20T11:18:09.614 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-04-20T11:18:09.614 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-04-20T11:18:09.614 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-04-20T11:18:09.614 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-04-20T11:18:09.615 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv 2024-04-20T11:18:09.615 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-04-20T11:18:09.615 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge 2024-04-20T11:18:09.615 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-04-20T11:18:09.615 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-04-20T11:18:09.615 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter 2024-04-20T11:18:09.615 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-20T11:18:09.615 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-20T11:18:09.615 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-20T11:18:09.615 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-20T11:18:09.615 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-20T11:18:09.615 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-20T11:18:09.615 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-20T11:18:09.616 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-20T11:18:09.616 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-20T11:18:09.616 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-20T11:18:09.616 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-20T11:18:09.616 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-20T11:18:09.616 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-20T11:18:09.616 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-20T11:18:09.616 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-20T11:18:09.616 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-20T11:18:09.616 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-20T11:18:09.616 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-20T11:18:09.616 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-20T11:18:09.616 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-20T11:18:09.616 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-04-20T11:18:09.617 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-20T11:18:09.617 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-20T11:18:09.617 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-20T11:18:09.617 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph 2024-04-20T11:18:09.617 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-04-20T11:18:09.617 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core 2024-04-20T11:18:09.617 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/failover.ko 2024-04-20T11:18:09.617 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-04-20T11:18:09.617 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp 2024-04-20T11:18:09.617 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-04-20T11:18:09.617 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-04-20T11:18:09.617 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-04-20T11:18:09.617 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-04-20T11:18:09.617 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver 2024-04-20T11:18:09.618 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver/dns_resolver.ko 2024-04-20T11:18:09.618 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa 2024-04-20T11:18:09.618 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-04-20T11:18:09.618 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-04-20T11:18:09.618 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-04-20T11:18:09.618 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-04-20T11:18:09.618 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-04-20T11:18:09.618 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154 2024-04-20T11:18:09.618 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan 2024-04-20T11:18:09.618 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-20T11:18:09.618 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-04-20T11:18:09.618 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-20T11:18:09.619 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4 2024-04-20T11:18:09.619 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-04-20T11:18:09.619 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-04-20T11:18:09.619 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-04-20T11:18:09.619 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-04-20T11:18:09.619 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-04-20T11:18:09.619 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-04-20T11:18:09.619 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-04-20T11:18:09.619 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-04-20T11:18:09.619 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-04-20T11:18:09.620 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-04-20T11:18:09.620 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter 2024-04-20T11:18:09.620 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-20T11:18:09.620 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-20T11:18:09.620 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-20T11:18:09.620 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-20T11:18:09.620 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-20T11:18:09.620 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-20T11:18:09.620 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-20T11:18:09.620 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-20T11:18:09.620 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-20T11:18:09.620 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-20T11:18:09.621 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-20T11:18:09.621 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-20T11:18:09.621 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-20T11:18:09.621 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-20T11:18:09.621 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-20T11:18:09.621 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-20T11:18:09.621 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-20T11:18:09.621 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-20T11:18:09.621 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-20T11:18:09.621 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-20T11:18:09.621 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-20T11:18:09.621 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-04-20T11:18:09.621 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-04-20T11:18:09.621 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-04-20T11:18:09.622 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-04-20T11:18:09.622 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-04-20T11:18:09.622 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-04-20T11:18:09.622 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-04-20T11:18:09.622 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-04-20T11:18:09.622 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-04-20T11:18:09.622 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-04-20T11:18:09.622 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-04-20T11:18:09.622 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-04-20T11:18:09.622 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-04-20T11:18:09.622 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-04-20T11:18:09.622 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-04-20T11:18:09.622 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-04-20T11:18:09.622 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-20T11:18:09.622 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6 2024-04-20T11:18:09.623 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-04-20T11:18:09.623 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-04-20T11:18:09.623 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-04-20T11:18:09.623 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-04-20T11:18:09.623 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-20T11:18:09.623 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-04-20T11:18:09.623 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-04-20T11:18:09.623 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter 2024-04-20T11:18:09.623 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-20T11:18:09.623 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-20T11:18:09.623 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-20T11:18:09.623 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-20T11:18:09.623 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-20T11:18:09.623 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-20T11:18:09.623 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-20T11:18:09.624 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-20T11:18:09.624 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-20T11:18:09.624 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-20T11:18:09.624 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-20T11:18:09.624 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-20T11:18:09.624 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-20T11:18:09.624 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-20T11:18:09.624 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-20T11:18:09.624 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-20T11:18:09.624 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-20T11:18:09.624 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-20T11:18:09.624 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-20T11:18:09.624 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-20T11:18:09.624 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-20T11:18:09.624 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-20T11:18:09.625 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-20T11:18:09.625 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-20T11:18:09.625 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-04-20T11:18:09.625 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-04-20T11:18:09.625 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-20T11:18:09.625 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key 2024-04-20T11:18:09.625 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-04-20T11:18:09.625 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp 2024-04-20T11:18:09.625 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-04-20T11:18:09.625 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-20T11:18:09.625 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-04-20T11:18:09.625 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-04-20T11:18:09.625 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-04-20T11:18:09.625 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-04-20T11:18:09.625 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-04-20T11:18:09.626 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc 2024-04-20T11:18:09.626 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-04-20T11:18:09.626 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154 2024-04-20T11:18:09.626 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-04-20T11:18:09.626 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls 2024-04-20T11:18:09.626 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-04-20T11:18:09.626 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter 2024-04-20T11:18:09.626 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset 2024-04-20T11:18:09.626 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-04-20T11:18:09.626 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-20T11:18:09.626 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-20T11:18:09.626 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-20T11:18:09.626 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-20T11:18:09.626 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-20T11:18:09.627 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-20T11:18:09.627 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-20T11:18:09.627 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-20T11:18:09.627 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-20T11:18:09.627 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-20T11:18:09.627 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-20T11:18:09.627 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-20T11:18:09.627 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-20T11:18:09.627 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-20T11:18:09.627 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-20T11:18:09.627 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs 2024-04-20T11:18:09.627 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-20T11:18:09.627 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-20T11:18:09.627 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-20T11:18:09.627 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-20T11:18:09.628 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-20T11:18:09.628 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-20T11:18:09.628 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-20T11:18:09.628 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-20T11:18:09.628 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-20T11:18:09.628 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-20T11:18:09.628 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-20T11:18:09.628 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-20T11:18:09.628 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-20T11:18:09.628 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-20T11:18:09.628 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-04-20T11:18:09.628 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-04-20T11:18:09.628 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-20T11:18:09.628 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-20T11:18:09.629 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-20T11:18:09.629 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-20T11:18:09.629 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-20T11:18:09.629 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-20T11:18:09.629 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-20T11:18:09.629 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-20T11:18:09.629 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-20T11:18:09.629 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-20T11:18:09.629 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-20T11:18:09.629 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-20T11:18:09.629 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-20T11:18:09.629 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-04-20T11:18:09.629 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-20T11:18:09.629 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-04-20T11:18:09.629 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-04-20T11:18:09.630 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-20T11:18:09.630 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-20T11:18:09.630 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-04-20T11:18:09.630 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-04-20T11:18:09.630 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-20T11:18:09.630 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-20T11:18:09.630 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-04-20T11:18:09.630 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-04-20T11:18:09.630 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-20T11:18:09.630 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-04-20T11:18:09.630 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-20T11:18:09.630 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-20T11:18:09.630 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-04-20T11:18:09.630 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-04-20T11:18:09.631 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-04-20T11:18:09.631 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-20T11:18:09.631 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-04-20T11:18:09.631 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-20T11:18:09.631 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-04-20T11:18:09.631 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-04-20T11:18:09.631 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-04-20T11:18:09.631 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-04-20T11:18:09.631 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-04-20T11:18:09.631 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-04-20T11:18:09.631 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-04-20T11:18:09.631 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_quota.ko 2024-04-20T11:18:09.631 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-04-20T11:18:09.631 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-04-20T11:18:09.632 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-04-20T11:18:09.632 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-04-20T11:18:09.632 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tunnel.ko 2024-04-20T11:18:09.632 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-04-20T11:18:09.632 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-20T11:18:09.632 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-20T11:18:09.632 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-20T11:18:09.632 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-04-20T11:18:09.632 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-04-20T11:18:09.632 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-04-20T11:18:09.632 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-04-20T11:18:09.632 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-20T11:18:09.632 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-04-20T11:18:09.632 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-04-20T11:18:09.633 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-20T11:18:09.633 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-04-20T11:18:09.633 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-04-20T11:18:09.633 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-20T11:18:09.633 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-04-20T11:18:09.633 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-20T11:18:09.633 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-04-20T11:18:09.633 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-20T11:18:09.633 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-20T11:18:09.633 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-04-20T11:18:09.633 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-04-20T11:18:09.633 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-04-20T11:18:09.633 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-04-20T11:18:09.633 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-04-20T11:18:09.634 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-04-20T11:18:09.634 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-04-20T11:18:09.634 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-04-20T11:18:09.634 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-04-20T11:18:09.634 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-04-20T11:18:09.634 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-04-20T11:18:09.634 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-04-20T11:18:09.634 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-04-20T11:18:09.634 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-04-20T11:18:09.634 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-04-20T11:18:09.634 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-04-20T11:18:09.634 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-04-20T11:18:09.634 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-04-20T11:18:09.634 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-04-20T11:18:09.635 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-04-20T11:18:09.635 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-04-20T11:18:09.635 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-04-20T11:18:09.635 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-04-20T11:18:09.635 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-04-20T11:18:09.635 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-04-20T11:18:09.635 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-04-20T11:18:09.635 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-04-20T11:18:09.635 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-04-20T11:18:09.635 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-04-20T11:18:09.635 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-04-20T11:18:09.635 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-04-20T11:18:09.635 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-04-20T11:18:09.635 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-04-20T11:18:09.635 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-04-20T11:18:09.636 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-04-20T11:18:09.636 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-04-20T11:18:09.636 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-04-20T11:18:09.636 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-04-20T11:18:09.636 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-04-20T11:18:09.636 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-04-20T11:18:09.636 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-04-20T11:18:09.636 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-04-20T11:18:09.636 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-04-20T11:18:09.636 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-04-20T11:18:09.636 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-04-20T11:18:09.636 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-04-20T11:18:09.636 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-04-20T11:18:09.637 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-04-20T11:18:09.637 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-04-20T11:18:09.637 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-04-20T11:18:09.637 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-04-20T11:18:09.637 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink 2024-04-20T11:18:09.637 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-04-20T11:18:09.637 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh 2024-04-20T11:18:09.637 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-04-20T11:18:09.637 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch 2024-04-20T11:18:09.637 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-04-20T11:18:09.637 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-04-20T11:18:09.637 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-04-20T11:18:09.638 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-04-20T11:18:09.638 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet 2024-04-20T11:18:09.638 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-04-20T11:18:09.638 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds 2024-04-20T11:18:09.638 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-04-20T11:18:09.638 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-04-20T11:18:09.638 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-04-20T11:18:09.638 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched 2024-04-20T11:18:09.638 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-04-20T11:18:09.639 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-04-20T11:18:09.639 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-04-20T11:18:09.639 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-04-20T11:18:09.639 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-04-20T11:18:09.639 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-04-20T11:18:09.639 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-04-20T11:18:09.639 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-04-20T11:18:09.639 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-04-20T11:18:09.639 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-04-20T11:18:09.640 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-04-20T11:18:09.640 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-04-20T11:18:09.640 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-04-20T11:18:09.640 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-04-20T11:18:09.640 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-04-20T11:18:09.640 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-04-20T11:18:09.640 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-04-20T11:18:09.640 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-04-20T11:18:09.640 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-04-20T11:18:09.640 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-04-20T11:18:09.640 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-04-20T11:18:09.640 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-04-20T11:18:09.641 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-04-20T11:18:09.641 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-04-20T11:18:09.641 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-04-20T11:18:09.641 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-04-20T11:18:09.641 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-04-20T11:18:09.641 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-04-20T11:18:09.641 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-04-20T11:18:09.641 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-04-20T11:18:09.641 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-04-20T11:18:09.641 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-04-20T11:18:09.641 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-04-20T11:18:09.642 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-04-20T11:18:09.642 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-04-20T11:18:09.642 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-04-20T11:18:09.642 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-04-20T11:18:09.642 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-04-20T11:18:09.642 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-04-20T11:18:09.642 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-04-20T11:18:09.642 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-04-20T11:18:09.642 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-04-20T11:18:09.643 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-04-20T11:18:09.643 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp 2024-04-20T11:18:09.643 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-04-20T11:18:09.643 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-04-20T11:18:09.643 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc 2024-04-20T11:18:09.643 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss 2024-04-20T11:18:09.643 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-20T11:18:09.643 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-20T11:18:09.643 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-04-20T11:18:09.643 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma 2024-04-20T11:18:09.643 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-20T11:18:09.643 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc 2024-04-20T11:18:09.644 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-04-20T11:18:09.644 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-04-20T11:18:09.644 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix 2024-04-20T11:18:09.644 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-04-20T11:18:09.644 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock 2024-04-20T11:18:09.644 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-20T11:18:09.644 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-20T11:18:09.644 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-04-20T11:18:09.644 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-20T11:18:09.644 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-20T11:18:09.644 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm 2024-04-20T11:18:09.645 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-20T11:18:09.645 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security 2024-04-20T11:18:09.645 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys 2024-04-20T11:18:09.645 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys 2024-04-20T11:18:09.645 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-20T11:18:09.645 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys 2024-04-20T11:18:09.645 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys/trusted.ko 2024-04-20T11:18:09.645 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-04-20T11:18:09.645 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-04-20T11:18:09.645 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-04-20T11:18:09.645 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-04-20T11:18:09.645 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-04-20T11:18:09.645 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-04-20T11:18:09.646 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-04-20T11:18:09.646 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-04-20T11:18:09.646 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-04-20T11:18:09.646 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-04-20T11:18:09.646 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-04-20T11:18:09.646 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-04-20T11:18:09.646 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-04-20T11:18:09.646 INFO:teuthology.orchestra.run.smithi152.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-20T11:18:09.650 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-04-20T11:18:09.650 DEBUG:teuthology.orchestra.run.smithi152:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-20T11:18:09.672 INFO:teuthology.orchestra.run.smithi152.stdout:postinstall scriptlet (using /bin/sh): 2024-04-20T11:18:09.672 INFO:teuthology.orchestra.run.smithi152.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-20T11:18:09.672 INFO:teuthology.orchestra.run.smithi152.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g806ec36b2112 /lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-20T11:18:09.672 INFO:teuthology.orchestra.run.smithi152.stdout:fi 2024-04-20T11:18:09.672 INFO:teuthology.orchestra.run.smithi152.stdout:for file in vmlinuz System.map config; do 2024-04-20T11:18:09.672 INFO:teuthology.orchestra.run.smithi152.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112"; then 2024-04-20T11:18:09.672 INFO:teuthology.orchestra.run.smithi152.stdout: cp "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112" 2024-04-20T11:18:09.672 INFO:teuthology.orchestra.run.smithi152.stdout: fi 2024-04-20T11:18:09.672 INFO:teuthology.orchestra.run.smithi152.stdout:done 2024-04-20T11:18:09.672 INFO:teuthology.orchestra.run.smithi152.stdout:if [ -d "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" ] && \ 2024-04-20T11:18:09.673 INFO:teuthology.orchestra.run.smithi152.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" >/dev/null 2>&1; then 2024-04-20T11:18:09.673 INFO:teuthology.orchestra.run.smithi152.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-20T11:18:09.673 INFO:teuthology.orchestra.run.smithi152.stdout: cp -r "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-20T11:18:09.673 INFO:teuthology.orchestra.run.smithi152.stdout:fi 2024-04-20T11:18:09.673 INFO:teuthology.orchestra.run.smithi152.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep" ]; then 2024-04-20T11:18:09.673 INFO:teuthology.orchestra.run.smithi152.stdout: /usr/sbin/depmod 6.9.0-rc3-g806ec36b2112 2024-04-20T11:18:09.673 INFO:teuthology.orchestra.run.smithi152.stdout:fi 2024-04-20T11:18:09.673 INFO:teuthology.orchestra.run.smithi152.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-20T11:18:09.673 INFO:teuthology.orchestra.run.smithi152.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-20T11:18:09.673 INFO:teuthology.orchestra.run.smithi152.stdout:kernel-install remove 6.9.0-rc3-g806ec36b2112 2024-04-20T11:18:09.673 INFO:teuthology.orchestra.run.smithi152.stdout:fi 2024-04-20T11:18:09.674 DEBUG:teuthology.orchestra.run.smithi152:> sudo rpm -qi grub2-tools 2024-04-20T11:18:09.747 INFO:teuthology.orchestra.run.smithi152.stdout:Name : grub2-tools 2024-04-20T11:18:09.747 INFO:teuthology.orchestra.run.smithi152.stdout:Epoch : 1 2024-04-20T11:18:09.747 INFO:teuthology.orchestra.run.smithi152.stdout:Version : 2.06 2024-04-20T11:18:09.747 INFO:teuthology.orchestra.run.smithi152.stdout:Release : 68.el9 2024-04-20T11:18:09.747 INFO:teuthology.orchestra.run.smithi152.stdout:Architecture: x86_64 2024-04-20T11:18:09.747 INFO:teuthology.orchestra.run.smithi152.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-04-20T11:18:09.747 INFO:teuthology.orchestra.run.smithi152.stdout:Group : Unspecified 2024-04-20T11:18:09.748 INFO:teuthology.orchestra.run.smithi152.stdout:Size : 8264254 2024-04-20T11:18:09.748 INFO:teuthology.orchestra.run.smithi152.stdout:License : GPLv3+ 2024-04-20T11:18:09.748 INFO:teuthology.orchestra.run.smithi152.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-20T11:18:09.748 INFO:teuthology.orchestra.run.smithi152.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-20T11:18:09.748 INFO:teuthology.orchestra.run.smithi152.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-20T11:18:09.748 INFO:teuthology.orchestra.run.smithi152.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-20T11:18:09.748 INFO:teuthology.orchestra.run.smithi152.stdout:Packager : builder@centos.org 2024-04-20T11:18:09.748 INFO:teuthology.orchestra.run.smithi152.stdout:Vendor : CentOS 2024-04-20T11:18:09.748 INFO:teuthology.orchestra.run.smithi152.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-20T11:18:09.748 INFO:teuthology.orchestra.run.smithi152.stdout:Summary : Support tools for GRUB. 2024-04-20T11:18:09.748 INFO:teuthology.orchestra.run.smithi152.stdout:Description : 2024-04-20T11:18:09.748 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:18:09.748 INFO:teuthology.orchestra.run.smithi152.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-20T11:18:09.748 INFO:teuthology.orchestra.run.smithi152.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-20T11:18:09.749 INFO:teuthology.orchestra.run.smithi152.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-20T11:18:09.749 INFO:teuthology.orchestra.run.smithi152.stdout:hardware devices. 2024-04-20T11:18:09.749 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:18:09.749 INFO:teuthology.orchestra.run.smithi152.stdout:This subpackage provides tools for support of all platforms. 2024-04-20T11:18:09.750 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-20T11:18:09.750 INFO:teuthology.task.kernel:Updating grub on smithi152 to boot 6.9.0-rc3-g806ec36b2112 2024-04-20T11:18:09.750 DEBUG:teuthology.orchestra.run.smithi152:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-20T11:18:10.322 INFO:teuthology.orchestra.run.smithi152.stderr:Generating grub configuration file ... 2024-04-20T11:18:11.745 INFO:teuthology.orchestra.run.smithi152.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-20T11:18:11.771 INFO:teuthology.orchestra.run.smithi152.stderr:done 2024-04-20T11:18:11.773 DEBUG:teuthology.orchestra.run.smithi152:> mktemp 2024-04-20T11:18:11.794 INFO:teuthology.orchestra.run.smithi152.stdout:/tmp/tmp.f9nNLKPRkp 2024-04-20T11:18:11.794 DEBUG:teuthology.orchestra.run.smithi152:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.f9nNLKPRkp 2024-04-20T11:18:11.868 DEBUG:teuthology.orchestra.run.smithi152:> sudo chmod 0666 /tmp/tmp.f9nNLKPRkp 2024-04-20T11:18:12.011 DEBUG:teuthology.orchestra.remote:smithi152:/tmp/tmp.f9nNLKPRkp is 6KB 2024-04-20T11:18:12.059 DEBUG:teuthology.orchestra.run.smithi152:> rm -fr /tmp/tmp.f9nNLKPRkp 2024-04-20T11:18:12.073 DEBUG:teuthology.orchestra.run.smithi152:> sudo /bin/ls /boot/loader/entries || true 2024-04-20T11:18:12.136 INFO:teuthology.orchestra.run.smithi152.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-04-20T11:18:12.136 INFO:teuthology.orchestra.run.smithi152.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-04-20T11:18:12.136 INFO:teuthology.orchestra.run.smithi152.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-04-20T11:18:12.136 INFO:teuthology.orchestra.run.smithi152.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112.conf 2024-04-20T11:18:12.137 DEBUG:teuthology.orchestra.run.smithi152:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112 2024-04-20T11:18:12.284 DEBUG:teuthology.orchestra.run.smithi152:> sudo shutdown -r now 2024-04-20T11:18:27.274 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-04-20T11:18:27.274 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi175.front.sepia.ceph.com' 2024-04-20T11:18:27.275 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi175.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-20T11:18:42.302 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-04-20T11:18:42.302 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi152.front.sepia.ceph.com' 2024-04-20T11:18:42.303 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi152.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-20T11:18:45.794 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.175 2024-04-20T11:18:54.802 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi175.front.sepia.ceph.com' 2024-04-20T11:18:54.803 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi175.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-20T11:19:00.638 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.152 2024-04-20T11:19:09.646 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi152.front.sepia.ceph.com' 2024-04-20T11:19:09.647 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi152.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-20T11:19:12.705 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.152 2024-04-20T11:19:24.714 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi152.front.sepia.ceph.com' 2024-04-20T11:19:24.715 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi152.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-20T11:19:54.826 DEBUG:teuthology.orchestra.remote:timed out 2024-04-20T11:20:06.838 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi175.front.sepia.ceph.com' 2024-04-20T11:20:06.839 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi175.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-20T11:20:07.193 DEBUG:teuthology.orchestra.run.smithi175:> true 2024-04-20T11:20:07.809 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi175.front.sepia.ceph.com' 2024-04-20T11:20:07.809 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "6.9.0-rc3-g806ec36b2112"... 2024-04-20T11:20:07.809 DEBUG:teuthology.orchestra.run.smithi175:> uname -r 2024-04-20T11:20:07.861 INFO:teuthology.orchestra.run.smithi175.stdout:6.9.0-rc3-g806ec36b2112 2024-04-20T11:20:07.861 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-04-20T11:20:07.861 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-20T11:20:07.861 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-20T11:20:08.862 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-04-20T11:20:08.862 DEBUG:teuthology.orchestra.run.smithi175:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-20T11:20:08.993 INFO:teuthology.orchestra.run.smithi175.stdout:ttyS1 2024-04-20T11:20:09.069 DEBUG:teuthology.parallel:result is None 2024-04-20T11:20:24.716 DEBUG:teuthology.orchestra.remote:timed out 2024-04-20T11:20:39.717 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi152.front.sepia.ceph.com' 2024-04-20T11:20:39.718 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi152.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-20T11:20:40.056 DEBUG:teuthology.orchestra.run.smithi152:> true 2024-04-20T11:20:40.690 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi152.front.sepia.ceph.com' 2024-04-20T11:20:40.691 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "6.9.0-rc3-g806ec36b2112"... 2024-04-20T11:20:40.691 DEBUG:teuthology.orchestra.run.smithi152:> uname -r 2024-04-20T11:20:40.743 INFO:teuthology.orchestra.run.smithi152.stdout:6.9.0-rc3-g806ec36b2112 2024-04-20T11:20:40.743 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-04-20T11:20:40.743 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-20T11:20:40.743 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-20T11:20:41.744 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-04-20T11:20:41.744 DEBUG:teuthology.orchestra.run.smithi152:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-20T11:20:41.871 INFO:teuthology.orchestra.run.smithi152.stdout:ttyS1 2024-04-20T11:20:41.954 DEBUG:teuthology.parallel:result is None 2024-04-20T11:20:41.954 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-20T11:20:41.961 INFO:teuthology.task.internal:Creating test directory... 2024-04-20T11:20:41.961 DEBUG:teuthology.orchestra.run.smithi152:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-20T11:20:41.968 DEBUG:teuthology.orchestra.run.smithi175:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-20T11:20:42.026 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-20T11:20:42.039 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-20T11:20:42.048 INFO:teuthology.task.internal:Creating archive directory... 2024-04-20T11:20:42.049 DEBUG:teuthology.orchestra.run.smithi152:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-20T11:20:42.056 DEBUG:teuthology.orchestra.run.smithi175:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-20T11:20:42.126 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-20T11:20:42.133 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-20T11:20:42.133 DEBUG:teuthology.orchestra.run.smithi152:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-20T11:20:42.140 DEBUG:teuthology.orchestra.run.smithi175:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-20T11:20:42.234 INFO:teuthology.orchestra.run.smithi152.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-20T11:20:42.235 INFO:teuthology.orchestra.run.smithi175.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-20T11:20:42.266 INFO:teuthology.orchestra.run.smithi175.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-20T11:20:42.266 INFO:teuthology.orchestra.run.smithi152.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-20T11:20:42.271 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-20T11:20:42.278 INFO:teuthology.task.internal:Configuring sudo... 2024-04-20T11:20:42.278 DEBUG:teuthology.orchestra.run.smithi152:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-20T11:20:42.283 DEBUG:teuthology.orchestra.run.smithi175:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-20T11:20:42.368 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-20T11:20:42.378 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-20T11:20:42.378 DEBUG:teuthology.orchestra.run.smithi152:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-20T11:20:42.386 DEBUG:teuthology.orchestra.run.smithi175:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-20T11:20:42.442 DEBUG:teuthology.orchestra.run.smithi152:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-20T11:20:42.519 DEBUG:teuthology.orchestra.run.smithi152:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-20T11:20:42.643 DEBUG:teuthology.orchestra.run.smithi152:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-20T11:20:42.729 DEBUG:teuthology.orchestra.run.smithi152:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-20T11:20:42.817 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-20T11:20:42.817 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-20T11:20:42.904 DEBUG:teuthology.orchestra.run.smithi175:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-20T11:20:42.989 DEBUG:teuthology.orchestra.run.smithi175:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-20T11:20:43.101 DEBUG:teuthology.orchestra.run.smithi175:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-20T11:20:43.179 DEBUG:teuthology.orchestra.run.smithi175:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-20T11:20:43.265 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T11:20:43.265 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-20T11:20:43.347 DEBUG:teuthology.orchestra.run.smithi152:> sudo service rsyslog restart 2024-04-20T11:20:43.355 DEBUG:teuthology.orchestra.run.smithi175:> sudo service rsyslog restart 2024-04-20T11:20:43.459 INFO:teuthology.orchestra.run.smithi152.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-20T11:20:43.501 INFO:teuthology.orchestra.run.smithi175.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-20T11:20:43.986 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-20T11:20:44.000 INFO:teuthology.task.internal:Starting timer... 2024-04-20T11:20:44.001 INFO:teuthology.run_tasks:Running task pcp... 2024-04-20T11:20:44.012 INFO:teuthology.run_tasks:Running task selinux... 2024-04-20T11:20:44.020 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-04-20T11:20:44.020 DEBUG:teuthology.orchestra.run.smithi152:> sudo service auditd rotate 2024-04-20T11:20:44.143 INFO:teuthology.orchestra.run.smithi152.stdout:Rotating logs: 2024-04-20T11:20:44.148 DEBUG:teuthology.orchestra.run.smithi175:> sudo service auditd rotate 2024-04-20T11:20:44.301 INFO:teuthology.orchestra.run.smithi175.stdout:Rotating logs: 2024-04-20T11:20:44.307 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-20T11:20:44.308 DEBUG:teuthology.orchestra.run.smithi152:> /usr/sbin/getenforce 2024-04-20T11:20:44.385 INFO:teuthology.orchestra.run.smithi152.stdout:Permissive 2024-04-20T11:20:44.385 DEBUG:teuthology.orchestra.run.smithi175:> /usr/sbin/getenforce 2024-04-20T11:20:44.439 INFO:teuthology.orchestra.run.smithi175.stdout:Permissive 2024-04-20T11:20:44.440 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi152.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi175.front.sepia.ceph.com': 'permissive'} 2024-04-20T11:20:44.440 DEBUG:teuthology.orchestra.run.smithi152:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-20T11:20:44.520 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T11:20:44.521 DEBUG:teuthology.orchestra.run.smithi175:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-20T11:20:44.592 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T11:20:44.592 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-20T11:20:44.593 DEBUG:teuthology.orchestra.run.smithi152:> sudo /usr/sbin/setenforce permissive 2024-04-20T11:20:44.659 DEBUG:teuthology.orchestra.run.smithi175:> sudo /usr/sbin/setenforce permissive 2024-04-20T11:20:44.732 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-20T11:20:44.743 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-20T11:20:44.760 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-20T11:20:44.761 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi152.front.sepia.ceph.com,smithi175.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-20T11:23:06.861 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi152.front.sepia.ceph.com'), Remote(name='ubuntu@smithi175.front.sepia.ceph.com')] 2024-04-20T11:23:06.862 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi152.front.sepia.ceph.com' 2024-04-20T11:23:06.863 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi152.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-20T11:23:06.965 DEBUG:teuthology.orchestra.run.smithi152:> true 2024-04-20T11:23:07.113 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi152.front.sepia.ceph.com' 2024-04-20T11:23:07.113 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi175.front.sepia.ceph.com' 2024-04-20T11:23:07.113 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi175.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-20T11:23:07.216 DEBUG:teuthology.orchestra.run.smithi175:> true 2024-04-20T11:23:07.362 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi175.front.sepia.ceph.com' 2024-04-20T11:23:07.362 INFO:teuthology.run_tasks:Running task clock... 2024-04-20T11:23:07.372 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-20T11:23:07.373 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-20T11:23:07.373 DEBUG:teuthology.orchestra.run.smithi152:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-20T11:23:07.380 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-20T11:23:07.380 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-20T11:23:07.466 INFO:teuthology.orchestra.run.smithi152.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-20T11:23:07.476 INFO:teuthology.orchestra.run.smithi175.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-20T11:23:07.507 INFO:teuthology.orchestra.run.smithi152.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-20T11:23:07.518 INFO:teuthology.orchestra.run.smithi175.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-20T11:23:07.579 INFO:teuthology.orchestra.run.smithi152.stderr:sudo: ntpd: command not found 2024-04-20T11:23:07.591 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: ntpd: command not found 2024-04-20T11:23:07.611 INFO:teuthology.orchestra.run.smithi152.stdout:506 Cannot talk to daemon 2024-04-20T11:23:07.624 INFO:teuthology.orchestra.run.smithi175.stdout:506 Cannot talk to daemon 2024-04-20T11:23:07.653 INFO:teuthology.orchestra.run.smithi152.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-20T11:23:07.670 INFO:teuthology.orchestra.run.smithi175.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-20T11:23:07.696 INFO:teuthology.orchestra.run.smithi152.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-20T11:23:07.712 INFO:teuthology.orchestra.run.smithi175.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-20T11:23:07.813 INFO:teuthology.orchestra.run.smithi152.stderr:bash: line 1: ntpq: command not found 2024-04-20T11:23:07.822 INFO:teuthology.orchestra.run.smithi152.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-20T11:23:07.822 INFO:teuthology.orchestra.run.smithi152.stdout:=============================================================================== 2024-04-20T11:23:07.822 INFO:teuthology.orchestra.run.smithi152.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T11:23:07.822 INFO:teuthology.orchestra.run.smithi152.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T11:23:07.822 INFO:teuthology.orchestra.run.smithi152.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T11:23:07.822 INFO:teuthology.orchestra.run.smithi152.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T11:23:07.850 INFO:teuthology.orchestra.run.smithi175.stderr:bash: line 1: ntpq: command not found 2024-04-20T11:23:07.859 INFO:teuthology.orchestra.run.smithi175.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-20T11:23:07.860 INFO:teuthology.orchestra.run.smithi175.stdout:=============================================================================== 2024-04-20T11:23:07.860 INFO:teuthology.orchestra.run.smithi175.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T11:23:07.860 INFO:teuthology.orchestra.run.smithi175.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T11:23:07.860 INFO:teuthology.orchestra.run.smithi175.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T11:23:07.860 INFO:teuthology.orchestra.run.smithi175.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T11:23:07.861 INFO:teuthology.run_tasks:Running task install... 2024-04-20T11:23:07.869 DEBUG:teuthology.task.install:project ceph 2024-04-20T11:23:07.869 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda'}} 2024-04-20T11:23:07.869 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda'} 2024-04-20T11:23:07.870 INFO:teuthology.task.install:Using flavor: default 2024-04-20T11:23:07.875 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-20T11:23:07.875 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-20T11:23:07.876 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda', 'tag': None, 'wait_for_package': False} 2024-04-20T11:23:07.876 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a8c84f5997950fd7d008204839253291d52e6eda 2024-04-20T11:23:07.878 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda', 'tag': None, 'wait_for_package': False} 2024-04-20T11:23:07.878 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a8c84f5997950fd7d008204839253291d52e6eda 2024-04-20T11:23:08.104 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/ 2024-04-20T11:23:08.105 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3133.ga8c84f59 2024-04-20T11:23:08.108 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/ 2024-04-20T11:23:08.108 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3133.ga8c84f59 2024-04-20T11:23:08.284 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-20T11:23:08.284 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-20T11:23:08.284 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-20T11:23:08.294 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-20T11:23:08.295 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T11:23:08.295 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-20T11:23:08.379 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-20T11:23:08.379 DEBUG:teuthology.orchestra.run.smithi152:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a8c84f5997950fd7d008204839253291d52e6eda/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-20T11:23:08.386 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-20T11:23:08.386 DEBUG:teuthology.orchestra.run.smithi175:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a8c84f5997950fd7d008204839253291d52e6eda/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-20T11:23:08.506 DEBUG:teuthology.orchestra.run.smithi152:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-20T11:23:08.515 DEBUG:teuthology.orchestra.run.smithi175:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-20T11:23:08.609 DEBUG:teuthology.orchestra.run.smithi152:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-20T11:23:08.627 DEBUG:teuthology.orchestra.run.smithi175:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-20T11:23:08.682 INFO:teuthology.orchestra.run.smithi152.stdout:check_obsoletes = 1 2024-04-20T11:23:08.686 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum clean all 2024-04-20T11:23:08.699 INFO:teuthology.orchestra.run.smithi175.stdout:check_obsoletes = 1 2024-04-20T11:23:08.703 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum clean all 2024-04-20T11:23:09.019 INFO:teuthology.orchestra.run.smithi152.stdout:48 files removed 2024-04-20T11:23:09.022 INFO:teuthology.orchestra.run.smithi175.stdout:48 files removed 2024-04-20T11:23:09.061 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-20T11:23:09.062 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-20T11:23:09.978 INFO:teuthology.orchestra.run.smithi175.stdout:ceph packages for x86_64 172 kB/s | 84 kB 00:00 2024-04-20T11:23:10.123 INFO:teuthology.orchestra.run.smithi152.stdout:ceph packages for x86_64 135 kB/s | 84 kB 00:00 2024-04-20T11:23:10.618 INFO:teuthology.orchestra.run.smithi175.stdout:ceph noarch packages 21 kB/s | 12 kB 00:00 2024-04-20T11:23:10.782 INFO:teuthology.orchestra.run.smithi152.stdout:ceph noarch packages 21 kB/s | 12 kB 00:00 2024-04-20T11:23:11.358 INFO:teuthology.orchestra.run.smithi152.stdout:ceph source packages 3.9 kB/s | 1.9 kB 00:00 2024-04-20T11:23:11.363 INFO:teuthology.orchestra.run.smithi175.stdout:ceph source packages 2.9 kB/s | 1.9 kB 00:00 2024-04-20T11:23:13.131 INFO:teuthology.orchestra.run.smithi152.stdout:CentOS Stream 9 - BaseOS 4.8 MB/s | 8.0 MB 00:01 2024-04-20T11:23:15.282 INFO:teuthology.orchestra.run.smithi152.stdout:CentOS Stream 9 - AppStream 20 MB/s | 19 MB 00:00 2024-04-20T11:23:20.549 INFO:teuthology.orchestra.run.smithi152.stdout:CentOS Stream 9 - CRB 4.4 MB/s | 6.2 MB 00:01 2024-04-20T11:23:22.151 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 9 - BaseOS 770 kB/s | 8.0 MB 00:10 2024-04-20T11:23:22.754 INFO:teuthology.orchestra.run.smithi152.stdout:CentOS Stream 9 - Extras packages 20 kB/s | 16 kB 00:00 2024-04-20T11:23:25.477 INFO:teuthology.orchestra.run.smithi152.stdout:Extra Packages for Enterprise Linux 8.2 MB/s | 21 MB 00:02 2024-04-20T11:23:25.777 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 9 - AppStream 7.9 MB/s | 19 MB 00:02 2024-04-20T11:23:31.269 INFO:teuthology.orchestra.run.smithi152.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-04-20T11:23:31.602 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 9 - CRB 3.3 MB/s | 6.2 MB 00:01 2024-04-20T11:23:33.663 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 9 - Extras packages 24 kB/s | 16 kB 00:00 2024-04-20T11:23:33.756 INFO:teuthology.orchestra.run.smithi152.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-20T11:23:33.757 INFO:teuthology.orchestra.run.smithi152.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-20T11:23:33.777 INFO:teuthology.orchestra.run.smithi152.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-20T11:23:33.779 INFO:teuthology.orchestra.run.smithi152.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-20T11:23:33.780 INFO:teuthology.orchestra.run.smithi152.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-20T11:23:33.781 INFO:teuthology.orchestra.run.smithi152.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-20T11:23:33.784 INFO:teuthology.orchestra.run.smithi152.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-20T11:23:33.787 INFO:teuthology.orchestra.run.smithi152.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-20T11:23:33.788 INFO:teuthology.orchestra.run.smithi152.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-20T11:23:33.858 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2024-04-20T11:23:33.875 INFO:teuthology.orchestra.run.smithi152.stdout:======================================================================================= 2024-04-20T11:23:33.875 INFO:teuthology.orchestra.run.smithi152.stdout: Package Arch Version Repository Size 2024-04-20T11:23:33.875 INFO:teuthology.orchestra.run.smithi152.stdout:======================================================================================= 2024-04-20T11:23:33.875 INFO:teuthology.orchestra.run.smithi152.stdout:Installing: 2024-04-20T11:23:33.876 INFO:teuthology.orchestra.run.smithi152.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-20T11:23:33.876 INFO:teuthology.orchestra.run.smithi152.stdout: ceph x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 6.6 k 2024-04-20T11:23:33.876 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-base x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 5.4 M 2024-04-20T11:23:33.876 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-fuse x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 893 k 2024-04-20T11:23:33.876 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 145 k 2024-04-20T11:23:33.876 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 1.5 M 2024-04-20T11:23:33.876 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 142 k 2024-04-20T11:23:33.876 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 3.5 M 2024-04-20T11:23:33.876 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 7.4 M 2024-04-20T11:23:33.876 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-rook noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 49 k 2024-04-20T11:23:33.876 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-radosgw x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 11 M 2024-04-20T11:23:33.876 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-test x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 49 M 2024-04-20T11:23:33.877 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-volume noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 273 k 2024-04-20T11:23:33.877 INFO:teuthology.orchestra.run.smithi152.stdout: cephadm noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 763 k 2024-04-20T11:23:33.877 INFO:teuthology.orchestra.run.smithi152.stdout: cephfs-mirror x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 225 k 2024-04-20T11:23:33.877 INFO:teuthology.orchestra.run.smithi152.stdout: cephfs-shell x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 27 k 2024-04-20T11:23:33.877 INFO:teuthology.orchestra.run.smithi152.stdout: cephfs-top noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 18 k 2024-04-20T11:23:33.877 INFO:teuthology.orchestra.run.smithi152.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-20T11:23:33.877 INFO:teuthology.orchestra.run.smithi152.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-20T11:23:33.877 INFO:teuthology.orchestra.run.smithi152.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-20T11:23:33.877 INFO:teuthology.orchestra.run.smithi152.stdout: libcephfs-devel x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 32 k 2024-04-20T11:23:33.877 INFO:teuthology.orchestra.run.smithi152.stdout: libcephfs2 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 748 k 2024-04-20T11:23:33.877 INFO:teuthology.orchestra.run.smithi152.stdout: librados-devel x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 126 k 2024-04-20T11:23:33.877 INFO:teuthology.orchestra.run.smithi152.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-20T11:23:33.877 INFO:teuthology.orchestra.run.smithi152.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-20T11:23:33.877 INFO:teuthology.orchestra.run.smithi152.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-20T11:23:33.878 INFO:teuthology.orchestra.run.smithi152.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-20T11:23:33.878 INFO:teuthology.orchestra.run.smithi152.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-20T11:23:33.878 INFO:teuthology.orchestra.run.smithi152.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-20T11:23:33.878 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cephfs x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 162 k 2024-04-20T11:23:33.878 INFO:teuthology.orchestra.run.smithi152.stdout: python3-rados x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 321 k 2024-04-20T11:23:33.878 INFO:teuthology.orchestra.run.smithi152.stdout: python3-rbd x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 299 k 2024-04-20T11:23:33.878 INFO:teuthology.orchestra.run.smithi152.stdout: python3-rgw x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 99 k 2024-04-20T11:23:33.878 INFO:teuthology.orchestra.run.smithi152.stdout: rbd-fuse x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 85 k 2024-04-20T11:23:33.878 INFO:teuthology.orchestra.run.smithi152.stdout: rbd-mirror x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 3.1 M 2024-04-20T11:23:33.878 INFO:teuthology.orchestra.run.smithi152.stdout: rbd-nbd x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 170 k 2024-04-20T11:23:33.879 INFO:teuthology.orchestra.run.smithi152.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-20T11:23:33.879 INFO:teuthology.orchestra.run.smithi152.stdout:Upgrading: 2024-04-20T11:23:33.879 INFO:teuthology.orchestra.run.smithi152.stdout: librados2 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 3.4 M 2024-04-20T11:23:33.879 INFO:teuthology.orchestra.run.smithi152.stdout: librbd1 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 3.1 M 2024-04-20T11:23:33.879 INFO:teuthology.orchestra.run.smithi152.stdout:Installing dependencies: 2024-04-20T11:23:33.879 INFO:teuthology.orchestra.run.smithi152.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-20T11:23:33.879 INFO:teuthology.orchestra.run.smithi152.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-20T11:23:33.879 INFO:teuthology.orchestra.run.smithi152.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-20T11:23:33.879 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-common x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 21 M 2024-04-20T11:23:33.879 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 34 k 2024-04-20T11:23:33.879 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mds x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 2.4 M 2024-04-20T11:23:33.879 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 249 k 2024-04-20T11:23:33.880 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mon x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 4.7 M 2024-04-20T11:23:33.880 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-osd x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 17 M 2024-04-20T11:23:33.880 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 16 k 2024-04-20T11:23:33.880 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-selinux x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 25 k 2024-04-20T11:23:33.880 INFO:teuthology.orchestra.run.smithi152.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-20T11:23:33.880 INFO:teuthology.orchestra.run.smithi152.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-20T11:23:33.880 INFO:teuthology.orchestra.run.smithi152.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-20T11:23:33.880 INFO:teuthology.orchestra.run.smithi152.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-20T11:23:33.880 INFO:teuthology.orchestra.run.smithi152.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-20T11:23:33.880 INFO:teuthology.orchestra.run.smithi152.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-20T11:23:33.880 INFO:teuthology.orchestra.run.smithi152.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-20T11:23:33.880 INFO:teuthology.orchestra.run.smithi152.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-20T11:23:33.881 INFO:teuthology.orchestra.run.smithi152.stdout: libcephsqlite x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 163 k 2024-04-20T11:23:33.881 INFO:teuthology.orchestra.run.smithi152.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-20T11:23:33.881 INFO:teuthology.orchestra.run.smithi152.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-20T11:23:33.881 INFO:teuthology.orchestra.run.smithi152.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-20T11:23:33.881 INFO:teuthology.orchestra.run.smithi152.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-20T11:23:33.881 INFO:teuthology.orchestra.run.smithi152.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-20T11:23:33.881 INFO:teuthology.orchestra.run.smithi152.stdout: libradosstriper1 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 501 k 2024-04-20T11:23:33.881 INFO:teuthology.orchestra.run.smithi152.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-20T11:23:33.881 INFO:teuthology.orchestra.run.smithi152.stdout: librgw2 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 5.4 M 2024-04-20T11:23:33.881 INFO:teuthology.orchestra.run.smithi152.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-20T11:23:33.881 INFO:teuthology.orchestra.run.smithi152.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-20T11:23:33.881 INFO:teuthology.orchestra.run.smithi152.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-20T11:23:33.881 INFO:teuthology.orchestra.run.smithi152.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-20T11:23:33.881 INFO:teuthology.orchestra.run.smithi152.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-20T11:23:33.882 INFO:teuthology.orchestra.run.smithi152.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-20T11:23:33.882 INFO:teuthology.orchestra.run.smithi152.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-20T11:23:33.882 INFO:teuthology.orchestra.run.smithi152.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-20T11:23:33.882 INFO:teuthology.orchestra.run.smithi152.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-20T11:23:33.882 INFO:teuthology.orchestra.run.smithi152.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-20T11:23:33.882 INFO:teuthology.orchestra.run.smithi152.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-20T11:23:33.882 INFO:teuthology.orchestra.run.smithi152.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-20T11:23:33.882 INFO:teuthology.orchestra.run.smithi152.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-20T11:23:33.882 INFO:teuthology.orchestra.run.smithi152.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-20T11:23:33.882 INFO:teuthology.orchestra.run.smithi152.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-20T11:23:33.882 INFO:teuthology.orchestra.run.smithi152.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-20T11:23:33.882 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-20T11:23:33.882 INFO:teuthology.orchestra.run.smithi152.stdout: python3-ceph-argparse x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 45 k 2024-04-20T11:23:33.883 INFO:teuthology.orchestra.run.smithi152.stdout: python3-ceph-common x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 135 k 2024-04-20T11:23:33.883 INFO:teuthology.orchestra.run.smithi152.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-20T11:23:33.883 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-20T11:23:33.883 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-20T11:23:33.883 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-20T11:23:33.883 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-20T11:23:33.883 INFO:teuthology.orchestra.run.smithi152.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-20T11:23:33.883 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-20T11:23:33.883 INFO:teuthology.orchestra.run.smithi152.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-20T11:23:33.883 INFO:teuthology.orchestra.run.smithi152.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-20T11:23:33.883 INFO:teuthology.orchestra.run.smithi152.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-20T11:23:33.883 INFO:teuthology.orchestra.run.smithi152.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-20T11:23:33.883 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-20T11:23:33.884 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-20T11:23:33.884 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-20T11:23:33.884 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-20T11:23:33.884 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-20T11:23:33.884 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-20T11:23:33.884 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-20T11:23:33.884 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-20T11:23:33.884 INFO:teuthology.orchestra.run.smithi152.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-20T11:23:33.884 INFO:teuthology.orchestra.run.smithi152.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-20T11:23:33.884 INFO:teuthology.orchestra.run.smithi152.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-20T11:23:33.884 INFO:teuthology.orchestra.run.smithi152.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-20T11:23:33.884 INFO:teuthology.orchestra.run.smithi152.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-20T11:23:33.884 INFO:teuthology.orchestra.run.smithi152.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-20T11:23:33.885 INFO:teuthology.orchestra.run.smithi152.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-20T11:23:33.885 INFO:teuthology.orchestra.run.smithi152.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-20T11:23:33.885 INFO:teuthology.orchestra.run.smithi152.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-20T11:23:33.885 INFO:teuthology.orchestra.run.smithi152.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-20T11:23:33.885 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-20T11:23:33.885 INFO:teuthology.orchestra.run.smithi152.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-20T11:23:33.885 INFO:teuthology.orchestra.run.smithi152.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-20T11:23:33.885 INFO:teuthology.orchestra.run.smithi152.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-20T11:23:33.885 INFO:teuthology.orchestra.run.smithi152.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-20T11:23:33.885 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-20T11:23:33.885 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-20T11:23:33.885 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-20T11:23:33.885 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-20T11:23:33.885 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-20T11:23:33.885 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-20T11:23:33.886 INFO:teuthology.orchestra.run.smithi152.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-20T11:23:33.886 INFO:teuthology.orchestra.run.smithi152.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-20T11:23:33.886 INFO:teuthology.orchestra.run.smithi152.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-20T11:23:33.886 INFO:teuthology.orchestra.run.smithi152.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-20T11:23:33.886 INFO:teuthology.orchestra.run.smithi152.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-20T11:23:33.886 INFO:teuthology.orchestra.run.smithi152.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-20T11:23:33.886 INFO:teuthology.orchestra.run.smithi152.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-20T11:23:33.886 INFO:teuthology.orchestra.run.smithi152.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-20T11:23:33.886 INFO:teuthology.orchestra.run.smithi152.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-20T11:23:33.886 INFO:teuthology.orchestra.run.smithi152.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-20T11:23:33.886 INFO:teuthology.orchestra.run.smithi152.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-20T11:23:33.886 INFO:teuthology.orchestra.run.smithi152.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-20T11:23:33.886 INFO:teuthology.orchestra.run.smithi152.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-20T11:23:33.887 INFO:teuthology.orchestra.run.smithi152.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-20T11:23:33.887 INFO:teuthology.orchestra.run.smithi152.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-20T11:23:33.887 INFO:teuthology.orchestra.run.smithi152.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-20T11:23:33.887 INFO:teuthology.orchestra.run.smithi152.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-20T11:23:33.887 INFO:teuthology.orchestra.run.smithi152.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-20T11:23:33.887 INFO:teuthology.orchestra.run.smithi152.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-20T11:23:33.887 INFO:teuthology.orchestra.run.smithi152.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-20T11:23:33.887 INFO:teuthology.orchestra.run.smithi152.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-20T11:23:33.887 INFO:teuthology.orchestra.run.smithi152.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-20T11:23:33.887 INFO:teuthology.orchestra.run.smithi152.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-20T11:23:33.887 INFO:teuthology.orchestra.run.smithi152.stdout:Installing weak dependencies: 2024-04-20T11:23:33.887 INFO:teuthology.orchestra.run.smithi152.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-20T11:23:33.887 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:23:33.887 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction Summary 2024-04-20T11:23:33.887 INFO:teuthology.orchestra.run.smithi152.stdout:======================================================================================= 2024-04-20T11:23:33.888 INFO:teuthology.orchestra.run.smithi152.stdout:Install 148 Packages 2024-04-20T11:23:33.888 INFO:teuthology.orchestra.run.smithi152.stdout:Upgrade 2 Packages 2024-04-20T11:23:33.888 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:23:33.888 INFO:teuthology.orchestra.run.smithi152.stdout:Total download size: 218 M 2024-04-20T11:23:33.888 INFO:teuthology.orchestra.run.smithi152.stdout:Downloading Packages: 2024-04-20T11:23:35.298 INFO:teuthology.orchestra.run.smithi152.stdout:(1/150): ceph-19.0.0-3133.ga8c84f59.el9.x86_64. 25 kB/s | 6.6 kB 00:00 2024-04-20T11:23:35.507 INFO:teuthology.orchestra.run.smithi152.stdout:(2/150): ceph-fuse-19.0.0-3133.ga8c84f59.el9.x8 4.2 MB/s | 893 kB 00:00 2024-04-20T11:23:35.566 INFO:teuthology.orchestra.run.smithi152.stdout:(3/150): ceph-immutable-object-cache-19.0.0-313 2.5 MB/s | 145 kB 00:00 2024-04-20T11:23:35.866 INFO:teuthology.orchestra.run.smithi152.stdout:(4/150): ceph-base-19.0.0-3133.ga8c84f59.el9.x8 6.5 MB/s | 5.4 MB 00:00 2024-04-20T11:23:35.950 INFO:teuthology.orchestra.run.smithi152.stdout:(5/150): ceph-mds-19.0.0-3133.ga8c84f59.el9.x86 6.2 MB/s | 2.4 MB 00:00 2024-04-20T11:23:36.033 INFO:teuthology.orchestra.run.smithi152.stdout:(6/150): ceph-mgr-19.0.0-3133.ga8c84f59.el9.x86 9.0 MB/s | 1.5 MB 00:00 2024-04-20T11:23:36.533 INFO:teuthology.orchestra.run.smithi152.stdout:(7/150): ceph-mon-19.0.0-3133.ga8c84f59.el9.x86 8.1 MB/s | 4.7 MB 00:00 2024-04-20T11:23:36.712 INFO:teuthology.orchestra.run.smithi175.stdout:Extra Packages for Enterprise Linux 7.3 MB/s | 21 MB 00:02 2024-04-20T11:23:37.550 INFO:teuthology.orchestra.run.smithi152.stdout:(8/150): ceph-radosgw-19.0.0-3133.ga8c84f59.el9 10 MB/s | 11 MB 00:01 2024-04-20T11:23:37.651 INFO:teuthology.orchestra.run.smithi152.stdout:(9/150): ceph-selinux-19.0.0-3133.ga8c84f59.el9 254 kB/s | 25 kB 00:00 2024-04-20T11:23:37.884 INFO:teuthology.orchestra.run.smithi152.stdout:(10/150): ceph-osd-19.0.0-3133.ga8c84f59.el9.x8 9.1 MB/s | 17 MB 00:01 2024-04-20T11:23:38.418 INFO:teuthology.orchestra.run.smithi152.stdout:(11/150): cephfs-mirror-19.0.0-3133.ga8c84f59.e 422 kB/s | 225 kB 00:00 2024-04-20T11:23:38.510 INFO:teuthology.orchestra.run.smithi152.stdout:(12/150): cephfs-shell-19.0.0-3133.ga8c84f59.el 298 kB/s | 27 kB 00:00 2024-04-20T11:23:38.785 INFO:teuthology.orchestra.run.smithi152.stdout:(13/150): ceph-common-19.0.0-3133.ga8c84f59.el9 5.6 MB/s | 21 MB 00:03 2024-04-20T11:23:38.810 INFO:teuthology.orchestra.run.smithi152.stdout:(14/150): libcephfs-devel-19.0.0-3133.ga8c84f59 106 kB/s | 32 kB 00:00 2024-04-20T11:23:38.952 INFO:teuthology.orchestra.run.smithi152.stdout:(15/150): libcephsqlite-19.0.0-3133.ga8c84f59.e 1.1 MB/s | 163 kB 00:00 2024-04-20T11:23:39.002 INFO:teuthology.orchestra.run.smithi152.stdout:(16/150): libcephfs2-19.0.0-3133.ga8c84f59.el9. 3.4 MB/s | 748 kB 00:00 2024-04-20T11:23:39.044 INFO:teuthology.orchestra.run.smithi152.stdout:(17/150): librados-devel-19.0.0-3133.ga8c84f59. 1.4 MB/s | 126 kB 00:00 2024-04-20T11:23:39.203 INFO:teuthology.orchestra.run.smithi152.stdout:(18/150): libradosstriper1-19.0.0-3133.ga8c84f5 2.4 MB/s | 501 kB 00:00 2024-04-20T11:23:39.312 INFO:teuthology.orchestra.run.smithi152.stdout:(19/150): python3-ceph-argparse-19.0.0-3133.ga8 420 kB/s | 45 kB 00:00 2024-04-20T11:23:39.429 INFO:teuthology.orchestra.run.smithi152.stdout:(20/150): python3-ceph-common-19.0.0-3133.ga8c8 1.1 MB/s | 135 kB 00:00 2024-04-20T11:23:39.537 INFO:teuthology.orchestra.run.smithi152.stdout:(21/150): python3-cephfs-19.0.0-3133.ga8c84f59. 1.5 MB/s | 162 kB 00:00 2024-04-20T11:23:39.662 INFO:teuthology.orchestra.run.smithi152.stdout:(22/150): python3-rados-19.0.0-3133.ga8c84f59.e 2.5 MB/s | 321 kB 00:00 2024-04-20T11:23:40.571 INFO:teuthology.orchestra.run.smithi152.stdout:(23/150): python3-rbd-19.0.0-3133.ga8c84f59.el9 329 kB/s | 299 kB 00:00 2024-04-20T11:23:40.746 INFO:teuthology.orchestra.run.smithi152.stdout:(24/150): librgw2-19.0.0-3133.ga8c84f59.el9.x86 3.2 MB/s | 5.4 MB 00:01 2024-04-20T11:23:40.780 INFO:teuthology.orchestra.run.smithi152.stdout:(25/150): python3-rgw-19.0.0-3133.ga8c84f59.el9 480 kB/s | 99 kB 00:00 2024-04-20T11:23:40.853 INFO:teuthology.orchestra.run.smithi152.stdout:(26/150): rbd-fuse-19.0.0-3133.ga8c84f59.el9.x8 809 kB/s | 85 kB 00:00 2024-04-20T11:23:40.964 INFO:teuthology.orchestra.run.smithi152.stdout:(27/150): rbd-nbd-19.0.0-3133.ga8c84f59.el9.x86 1.5 MB/s | 170 kB 00:00 2024-04-20T11:23:41.064 INFO:teuthology.orchestra.run.smithi152.stdout:(28/150): ceph-grafana-dashboards-19.0.0-3133.g 345 kB/s | 34 kB 00:00 2024-04-20T11:23:41.173 INFO:teuthology.orchestra.run.smithi152.stdout:(29/150): ceph-mgr-cephadm-19.0.0-3133.ga8c84f5 1.3 MB/s | 142 kB 00:00 2024-04-20T11:23:41.857 INFO:teuthology.orchestra.run.smithi152.stdout:(30/150): rbd-mirror-19.0.0-3133.ga8c84f59.el9. 2.9 MB/s | 3.1 MB 00:01 2024-04-20T11:23:42.572 INFO:teuthology.orchestra.run.smithi175.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-04-20T11:23:42.632 INFO:teuthology.orchestra.run.smithi152.stdout:(31/150): ceph-mgr-dashboard-19.0.0-3133.ga8c84 2.4 MB/s | 3.5 MB 00:01 2024-04-20T11:23:42.799 INFO:teuthology.orchestra.run.smithi152.stdout:(32/150): ceph-mgr-modules-core-19.0.0-3133.ga8 1.5 MB/s | 249 kB 00:00 2024-04-20T11:23:42.891 INFO:teuthology.orchestra.run.smithi152.stdout:(33/150): ceph-mgr-rook-19.0.0-3133.ga8c84f59.e 543 kB/s | 49 kB 00:00 2024-04-20T11:23:42.983 INFO:teuthology.orchestra.run.smithi152.stdout:(34/150): ceph-prometheus-alerts-19.0.0-3133.ga 181 kB/s | 16 kB 00:00 2024-04-20T11:23:43.166 INFO:teuthology.orchestra.run.smithi152.stdout:(35/150): ceph-volume-19.0.0-3133.ga8c84f59.el9 1.5 MB/s | 273 kB 00:00 2024-04-20T11:23:43.425 INFO:teuthology.orchestra.run.smithi152.stdout:(36/150): cephadm-19.0.0-3133.ga8c84f59.el9.noa 2.9 MB/s | 763 kB 00:00 2024-04-20T11:23:43.517 INFO:teuthology.orchestra.run.smithi152.stdout:(37/150): cephfs-top-19.0.0-3133.ga8c84f59.el9. 201 kB/s | 18 kB 00:00 2024-04-20T11:23:43.625 INFO:teuthology.orchestra.run.smithi152.stdout:(38/150): fuse-2.9.9-15.el9.x86_64.rpm 751 kB/s | 80 kB 00:00 2024-04-20T11:23:43.768 INFO:teuthology.orchestra.run.smithi152.stdout:(39/150): libgfortran-11.4.1-3.el9.x86_64.rpm 5.6 MB/s | 803 kB 00:00 2024-04-20T11:23:43.818 INFO:teuthology.orchestra.run.smithi152.stdout:(40/150): libquadmath-11.4.1-3.el9.x86_64.rpm 3.9 MB/s | 195 kB 00:00 2024-04-20T11:23:43.876 INFO:teuthology.orchestra.run.smithi152.stdout:(41/150): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.3 MB/s | 253 kB 00:00 2024-04-20T11:23:43.977 INFO:teuthology.orchestra.run.smithi152.stdout:(42/150): python3-cryptography-36.0.1-4.el9.x86 12 MB/s | 1.2 MB 00:00 2024-04-20T11:23:44.027 INFO:teuthology.orchestra.run.smithi152.stdout:(43/150): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-04-20T11:23:44.078 INFO:teuthology.orchestra.run.smithi152.stdout:(44/150): python3-pycparser-2.20-6.el9.noarch.r 2.7 MB/s | 135 kB 00:00 2024-04-20T11:23:44.128 INFO:teuthology.orchestra.run.smithi152.stdout:(45/150): python3-requests-2.25.1-8.el9.noarch. 2.5 MB/s | 125 kB 00:00 2024-04-20T11:23:44.178 INFO:teuthology.orchestra.run.smithi152.stdout:(46/150): python3-urllib3-1.26.5-5.el9.noarch.r 4.3 MB/s | 215 kB 00:00 2024-04-20T11:23:44.437 INFO:teuthology.orchestra.run.smithi152.stdout:(47/150): ceph-mgr-diskprediction-local-19.0.0- 2.9 MB/s | 7.4 MB 00:02 2024-04-20T11:23:45.040 INFO:teuthology.orchestra.run.smithi175.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-20T11:23:45.042 INFO:teuthology.orchestra.run.smithi175.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-20T11:23:45.062 INFO:teuthology.orchestra.run.smithi175.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-20T11:23:45.063 INFO:teuthology.orchestra.run.smithi175.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-20T11:23:45.064 INFO:teuthology.orchestra.run.smithi175.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-20T11:23:45.065 INFO:teuthology.orchestra.run.smithi175.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-20T11:23:45.068 INFO:teuthology.orchestra.run.smithi175.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-20T11:23:45.071 INFO:teuthology.orchestra.run.smithi175.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-20T11:23:45.072 INFO:teuthology.orchestra.run.smithi175.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-20T11:23:45.142 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2024-04-20T11:23:45.162 INFO:teuthology.orchestra.run.smithi175.stdout:======================================================================================= 2024-04-20T11:23:45.162 INFO:teuthology.orchestra.run.smithi175.stdout: Package Arch Version Repository Size 2024-04-20T11:23:45.162 INFO:teuthology.orchestra.run.smithi175.stdout:======================================================================================= 2024-04-20T11:23:45.163 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2024-04-20T11:23:45.163 INFO:teuthology.orchestra.run.smithi175.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-20T11:23:45.163 INFO:teuthology.orchestra.run.smithi175.stdout: ceph x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 6.6 k 2024-04-20T11:23:45.163 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-base x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 5.4 M 2024-04-20T11:23:45.163 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-fuse x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 893 k 2024-04-20T11:23:45.163 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 145 k 2024-04-20T11:23:45.163 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 1.5 M 2024-04-20T11:23:45.163 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 142 k 2024-04-20T11:23:45.163 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 3.5 M 2024-04-20T11:23:45.163 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 7.4 M 2024-04-20T11:23:45.163 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-rook noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 49 k 2024-04-20T11:23:45.163 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-radosgw x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 11 M 2024-04-20T11:23:45.163 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-test x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 49 M 2024-04-20T11:23:45.164 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-volume noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 273 k 2024-04-20T11:23:45.164 INFO:teuthology.orchestra.run.smithi175.stdout: cephadm noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 763 k 2024-04-20T11:23:45.164 INFO:teuthology.orchestra.run.smithi175.stdout: cephfs-mirror x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 225 k 2024-04-20T11:23:45.164 INFO:teuthology.orchestra.run.smithi175.stdout: cephfs-shell x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 27 k 2024-04-20T11:23:45.164 INFO:teuthology.orchestra.run.smithi175.stdout: cephfs-top noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 18 k 2024-04-20T11:23:45.164 INFO:teuthology.orchestra.run.smithi175.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-20T11:23:45.164 INFO:teuthology.orchestra.run.smithi175.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-20T11:23:45.164 INFO:teuthology.orchestra.run.smithi175.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-20T11:23:45.164 INFO:teuthology.orchestra.run.smithi175.stdout: libcephfs-devel x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 32 k 2024-04-20T11:23:45.164 INFO:teuthology.orchestra.run.smithi175.stdout: libcephfs2 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 748 k 2024-04-20T11:23:45.164 INFO:teuthology.orchestra.run.smithi175.stdout: librados-devel x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 126 k 2024-04-20T11:23:45.164 INFO:teuthology.orchestra.run.smithi175.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-20T11:23:45.164 INFO:teuthology.orchestra.run.smithi175.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-20T11:23:45.164 INFO:teuthology.orchestra.run.smithi175.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-20T11:23:45.165 INFO:teuthology.orchestra.run.smithi175.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-20T11:23:45.165 INFO:teuthology.orchestra.run.smithi175.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-20T11:23:45.165 INFO:teuthology.orchestra.run.smithi175.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-20T11:23:45.165 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cephfs x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 162 k 2024-04-20T11:23:45.165 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rados x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 321 k 2024-04-20T11:23:45.165 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rbd x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 299 k 2024-04-20T11:23:45.165 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rgw x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 99 k 2024-04-20T11:23:45.165 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-fuse x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 85 k 2024-04-20T11:23:45.165 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-mirror x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 3.1 M 2024-04-20T11:23:45.165 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-nbd x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 170 k 2024-04-20T11:23:45.165 INFO:teuthology.orchestra.run.smithi175.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-20T11:23:45.165 INFO:teuthology.orchestra.run.smithi175.stdout:Upgrading: 2024-04-20T11:23:45.165 INFO:teuthology.orchestra.run.smithi175.stdout: librados2 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 3.4 M 2024-04-20T11:23:45.166 INFO:teuthology.orchestra.run.smithi175.stdout: librbd1 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 3.1 M 2024-04-20T11:23:45.166 INFO:teuthology.orchestra.run.smithi175.stdout:Installing dependencies: 2024-04-20T11:23:45.166 INFO:teuthology.orchestra.run.smithi175.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-20T11:23:45.166 INFO:teuthology.orchestra.run.smithi175.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-20T11:23:45.166 INFO:teuthology.orchestra.run.smithi175.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-20T11:23:45.166 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-common x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 21 M 2024-04-20T11:23:45.166 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 34 k 2024-04-20T11:23:45.166 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mds x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 2.4 M 2024-04-20T11:23:45.166 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 249 k 2024-04-20T11:23:45.166 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mon x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 4.7 M 2024-04-20T11:23:45.166 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-osd x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 17 M 2024-04-20T11:23:45.166 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 16 k 2024-04-20T11:23:45.166 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-selinux x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 25 k 2024-04-20T11:23:45.167 INFO:teuthology.orchestra.run.smithi175.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-20T11:23:45.167 INFO:teuthology.orchestra.run.smithi175.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-20T11:23:45.167 INFO:teuthology.orchestra.run.smithi175.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-20T11:23:45.167 INFO:teuthology.orchestra.run.smithi175.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-20T11:23:45.167 INFO:teuthology.orchestra.run.smithi175.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-20T11:23:45.167 INFO:teuthology.orchestra.run.smithi175.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-20T11:23:45.167 INFO:teuthology.orchestra.run.smithi175.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-20T11:23:45.167 INFO:teuthology.orchestra.run.smithi175.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-20T11:23:45.167 INFO:teuthology.orchestra.run.smithi175.stdout: libcephsqlite x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 163 k 2024-04-20T11:23:45.167 INFO:teuthology.orchestra.run.smithi175.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-20T11:23:45.167 INFO:teuthology.orchestra.run.smithi175.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-20T11:23:45.167 INFO:teuthology.orchestra.run.smithi175.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-20T11:23:45.167 INFO:teuthology.orchestra.run.smithi175.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-20T11:23:45.167 INFO:teuthology.orchestra.run.smithi175.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-20T11:23:45.168 INFO:teuthology.orchestra.run.smithi175.stdout: libradosstriper1 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 501 k 2024-04-20T11:23:45.168 INFO:teuthology.orchestra.run.smithi175.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-20T11:23:45.168 INFO:teuthology.orchestra.run.smithi175.stdout: librgw2 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 5.4 M 2024-04-20T11:23:45.168 INFO:teuthology.orchestra.run.smithi175.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-20T11:23:45.168 INFO:teuthology.orchestra.run.smithi175.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-20T11:23:45.168 INFO:teuthology.orchestra.run.smithi175.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-20T11:23:45.168 INFO:teuthology.orchestra.run.smithi175.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-20T11:23:45.168 INFO:teuthology.orchestra.run.smithi175.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-20T11:23:45.168 INFO:teuthology.orchestra.run.smithi175.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-20T11:23:45.168 INFO:teuthology.orchestra.run.smithi175.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-20T11:23:45.168 INFO:teuthology.orchestra.run.smithi175.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-20T11:23:45.168 INFO:teuthology.orchestra.run.smithi175.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-20T11:23:45.168 INFO:teuthology.orchestra.run.smithi175.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-20T11:23:45.168 INFO:teuthology.orchestra.run.smithi175.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-20T11:23:45.169 INFO:teuthology.orchestra.run.smithi175.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-20T11:23:45.169 INFO:teuthology.orchestra.run.smithi175.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-20T11:23:45.169 INFO:teuthology.orchestra.run.smithi175.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-20T11:23:45.169 INFO:teuthology.orchestra.run.smithi175.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-20T11:23:45.169 INFO:teuthology.orchestra.run.smithi175.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-20T11:23:45.169 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-20T11:23:45.169 INFO:teuthology.orchestra.run.smithi175.stdout: python3-ceph-argparse x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 45 k 2024-04-20T11:23:45.169 INFO:teuthology.orchestra.run.smithi175.stdout: python3-ceph-common x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 135 k 2024-04-20T11:23:45.169 INFO:teuthology.orchestra.run.smithi175.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-20T11:23:45.169 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-20T11:23:45.169 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-20T11:23:45.169 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-20T11:23:45.169 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-20T11:23:45.169 INFO:teuthology.orchestra.run.smithi175.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-20T11:23:45.170 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-20T11:23:45.170 INFO:teuthology.orchestra.run.smithi175.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-20T11:23:45.170 INFO:teuthology.orchestra.run.smithi175.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-20T11:23:45.170 INFO:teuthology.orchestra.run.smithi175.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-20T11:23:45.170 INFO:teuthology.orchestra.run.smithi175.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-20T11:23:45.170 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-20T11:23:45.170 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-20T11:23:45.170 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-20T11:23:45.170 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-20T11:23:45.170 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-20T11:23:45.170 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-20T11:23:45.170 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-20T11:23:45.170 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-20T11:23:45.170 INFO:teuthology.orchestra.run.smithi175.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-20T11:23:45.170 INFO:teuthology.orchestra.run.smithi175.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-20T11:23:45.171 INFO:teuthology.orchestra.run.smithi175.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-20T11:23:45.171 INFO:teuthology.orchestra.run.smithi175.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-20T11:23:45.171 INFO:teuthology.orchestra.run.smithi175.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-20T11:23:45.171 INFO:teuthology.orchestra.run.smithi175.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-20T11:23:45.171 INFO:teuthology.orchestra.run.smithi175.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-20T11:23:45.171 INFO:teuthology.orchestra.run.smithi175.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-20T11:23:45.171 INFO:teuthology.orchestra.run.smithi175.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-20T11:23:45.171 INFO:teuthology.orchestra.run.smithi175.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-20T11:23:45.171 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-20T11:23:45.171 INFO:teuthology.orchestra.run.smithi175.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-20T11:23:45.171 INFO:teuthology.orchestra.run.smithi175.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-20T11:23:45.171 INFO:teuthology.orchestra.run.smithi175.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-20T11:23:45.171 INFO:teuthology.orchestra.run.smithi175.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-20T11:23:45.171 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-20T11:23:45.171 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-20T11:23:45.172 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-20T11:23:45.172 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-20T11:23:45.172 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-20T11:23:45.172 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-20T11:23:45.172 INFO:teuthology.orchestra.run.smithi175.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-20T11:23:45.172 INFO:teuthology.orchestra.run.smithi175.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-20T11:23:45.172 INFO:teuthology.orchestra.run.smithi175.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-20T11:23:45.172 INFO:teuthology.orchestra.run.smithi175.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-20T11:23:45.172 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-20T11:23:45.172 INFO:teuthology.orchestra.run.smithi175.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-20T11:23:45.172 INFO:teuthology.orchestra.run.smithi175.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-20T11:23:45.172 INFO:teuthology.orchestra.run.smithi175.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-20T11:23:45.172 INFO:teuthology.orchestra.run.smithi175.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-20T11:23:45.172 INFO:teuthology.orchestra.run.smithi175.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-20T11:23:45.173 INFO:teuthology.orchestra.run.smithi175.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-20T11:23:45.173 INFO:teuthology.orchestra.run.smithi175.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-20T11:23:45.173 INFO:teuthology.orchestra.run.smithi175.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-20T11:23:45.173 INFO:teuthology.orchestra.run.smithi175.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-20T11:23:45.173 INFO:teuthology.orchestra.run.smithi175.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-20T11:23:45.173 INFO:teuthology.orchestra.run.smithi175.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-20T11:23:45.173 INFO:teuthology.orchestra.run.smithi175.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-20T11:23:45.173 INFO:teuthology.orchestra.run.smithi175.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-20T11:23:45.173 INFO:teuthology.orchestra.run.smithi175.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-20T11:23:45.173 INFO:teuthology.orchestra.run.smithi175.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-20T11:23:45.173 INFO:teuthology.orchestra.run.smithi175.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-20T11:23:45.173 INFO:teuthology.orchestra.run.smithi175.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-20T11:23:45.173 INFO:teuthology.orchestra.run.smithi175.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-20T11:23:45.173 INFO:teuthology.orchestra.run.smithi175.stdout:Installing weak dependencies: 2024-04-20T11:23:45.173 INFO:teuthology.orchestra.run.smithi175.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-20T11:23:45.174 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-20T11:23:45.174 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2024-04-20T11:23:45.174 INFO:teuthology.orchestra.run.smithi175.stdout:======================================================================================= 2024-04-20T11:23:45.174 INFO:teuthology.orchestra.run.smithi175.stdout:Install 148 Packages 2024-04-20T11:23:45.174 INFO:teuthology.orchestra.run.smithi175.stdout:Upgrade 2 Packages 2024-04-20T11:23:45.174 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-20T11:23:45.174 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 218 M 2024-04-20T11:23:45.174 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2024-04-20T11:23:45.837 INFO:teuthology.orchestra.run.smithi152.stdout:(48/150): ceph-test-19.0.0-3133.ga8c84f59.el9.x 6.0 MB/s | 49 MB 00:08 2024-04-20T11:23:46.179 INFO:teuthology.orchestra.run.smithi152.stdout:(49/150): automake-1.16.2-8.el9.noarch.rpm 344 kB/s | 689 kB 00:01 2024-04-20T11:23:46.338 INFO:teuthology.orchestra.run.smithi152.stdout:(50/150): bison-3.7.4-5.el9.x86_64.rpm 497 kB/s | 943 kB 00:01 2024-04-20T11:23:46.624 INFO:teuthology.orchestra.run.smithi175.stdout:(1/150): ceph-19.0.0-3133.ga8c84f59.el9.x86_64. 13 kB/s | 6.6 kB 00:00 2024-04-20T11:23:46.630 INFO:teuthology.orchestra.run.smithi152.stdout:(51/150): boost-program-options-1.75.0-8.el9.x8 135 kB/s | 107 kB 00:00 2024-04-20T11:23:46.805 INFO:teuthology.orchestra.run.smithi152.stdout:(52/150): elfutils-libelf-devel-0.190-2.el9.x86 34 kB/s | 21 kB 00:00 2024-04-20T11:23:47.280 INFO:teuthology.orchestra.run.smithi152.stdout:(53/150): flexiblas-3.0.4-8.el9.x86_64.rpm 50 kB/s | 33 kB 00:00 2024-04-20T11:23:47.431 INFO:teuthology.orchestra.run.smithi152.stdout:(54/150): flex-2.6.4-9.el9.x86_64.rpm 287 kB/s | 313 kB 00:01 2024-04-20T11:23:47.956 INFO:teuthology.orchestra.run.smithi152.stdout:(55/150): flexiblas-openblas-openmp-3.0.4-8.el9 27 kB/s | 18 kB 00:00 2024-04-20T11:23:48.006 INFO:teuthology.orchestra.run.smithi152.stdout:(56/150): libaio-devel-0.3.111-13.el9.x86_64.rp 19 kB/s | 11 kB 00:00 2024-04-20T11:23:48.698 INFO:teuthology.orchestra.run.smithi152.stdout:(57/150): librabbitmq-0.11.0-7.el9.x86_64.rpm 65 kB/s | 45 kB 00:00 2024-04-20T11:23:48.815 INFO:teuthology.orchestra.run.smithi175.stdout:(2/150): ceph-base-19.0.0-3133.ga8c84f59.el9.x8 2.0 MB/s | 5.4 MB 00:02 2024-04-20T11:23:48.831 INFO:teuthology.orchestra.run.smithi152.stdout:(58/150): libpmemobj-1.12.1-1.el9.x86_64.rpm 183 kB/s | 160 kB 00:00 2024-04-20T11:23:48.923 INFO:teuthology.orchestra.run.smithi175.stdout:(3/150): ceph-immutable-object-cache-19.0.0-313 1.3 MB/s | 145 kB 00:00 2024-04-20T11:23:49.624 INFO:teuthology.orchestra.run.smithi175.stdout:(4/150): ceph-mds-19.0.0-3133.ga8c84f59.el9.x86 3.4 MB/s | 2.4 MB 00:00 2024-04-20T11:23:50.091 INFO:teuthology.orchestra.run.smithi175.stdout:(5/150): ceph-mgr-19.0.0-3133.ga8c84f59.el9.x86 3.2 MB/s | 1.5 MB 00:00 2024-04-20T11:23:50.199 INFO:teuthology.orchestra.run.smithi152.stdout:(59/150): librdkafka-1.6.1-102.el9.x86_64.rpm 442 kB/s | 662 kB 00:01 2024-04-20T11:23:50.266 INFO:teuthology.orchestra.run.smithi152.stdout:(60/150): libtool-2.4.6-45.el9.x86_64.rpm 406 kB/s | 582 kB 00:01 2024-04-20T11:23:50.849 INFO:teuthology.orchestra.run.smithi152.stdout:(61/150): libuuid-devel-2.37.4-18.el9.x86_64.rp 39 kB/s | 25 kB 00:00 2024-04-20T11:23:50.958 INFO:teuthology.orchestra.run.smithi152.stdout:(62/150): libzstd-devel-1.5.1-2.el9.x86_64.rpm 69 kB/s | 47 kB 00:00 2024-04-20T11:23:51.241 INFO:teuthology.orchestra.run.smithi175.stdout:(6/150): ceph-mon-19.0.0-3133.ga8c84f59.el9.x86 4.1 MB/s | 4.7 MB 00:01 2024-04-20T11:23:51.391 INFO:teuthology.orchestra.run.smithi175.stdout:(7/150): ceph-fuse-19.0.0-3133.ga8c84f59.el9.x8 187 kB/s | 893 kB 00:04 2024-04-20T11:23:51.508 INFO:teuthology.orchestra.run.smithi152.stdout:(63/150): flexiblas-netlib-3.0.4-8.el9.x86_64.r 651 kB/s | 3.0 MB 00:04 2024-04-20T11:23:51.917 INFO:teuthology.orchestra.run.smithi152.stdout:(64/150): lttng-ust-2.12.0-6.el9.x86_64.rpm 274 kB/s | 292 kB 00:01 2024-04-20T11:23:51.975 INFO:teuthology.orchestra.run.smithi152.stdout:(65/150): lua-5.4.4-4.el9.x86_64.rpm 185 kB/s | 188 kB 00:01 2024-04-20T11:23:52.309 INFO:teuthology.orchestra.run.smithi152.stdout:(66/150): openblas-0.3.21-2.el9.x86_64.rpm 44 kB/s | 35 kB 00:00 2024-04-20T11:23:54.984 INFO:teuthology.orchestra.run.smithi152.stdout:(67/150): postgresql-13.14-1.el9.x86_64.rpm 608 kB/s | 1.6 MB 00:02 2024-04-20T11:23:56.049 INFO:teuthology.orchestra.run.smithi175.stdout:(8/150): ceph-common-19.0.0-3133.ga8c84f59.el9. 2.1 MB/s | 21 MB 00:09 2024-04-20T11:23:56.166 INFO:teuthology.orchestra.run.smithi175.stdout:(9/150): ceph-selinux-19.0.0-3133.ga8c84f59.el9 218 kB/s | 25 kB 00:00 2024-04-20T11:23:56.883 INFO:teuthology.orchestra.run.smithi175.stdout:(10/150): ceph-osd-19.0.0-3133.ga8c84f59.el9.x8 3.0 MB/s | 17 MB 00:05 2024-04-20T11:23:56.909 INFO:teuthology.orchestra.run.smithi152.stdout:(68/150): postgresql-contrib-13.14-1.el9.x86_64 474 kB/s | 911 kB 00:01 2024-04-20T11:23:57.033 INFO:teuthology.orchestra.run.smithi175.stdout:(11/150): cephfs-mirror-19.0.0-3133.ga8c84f59.e 1.5 MB/s | 225 kB 00:00 2024-04-20T11:23:57.100 INFO:teuthology.orchestra.run.smithi175.stdout:(12/150): cephfs-shell-19.0.0-3133.ga8c84f59.el 411 kB/s | 27 kB 00:00 2024-04-20T11:23:57.183 INFO:teuthology.orchestra.run.smithi175.stdout:(13/150): libcephfs-devel-19.0.0-3133.ga8c84f59 383 kB/s | 32 kB 00:00 2024-04-20T11:23:57.541 INFO:teuthology.orchestra.run.smithi175.stdout:(14/150): libcephfs2-19.0.0-3133.ga8c84f59.el9. 2.0 MB/s | 748 kB 00:00 2024-04-20T11:23:57.633 INFO:teuthology.orchestra.run.smithi175.stdout:(15/150): libcephsqlite-19.0.0-3133.ga8c84f59.e 1.7 MB/s | 163 kB 00:00 2024-04-20T11:23:57.742 INFO:teuthology.orchestra.run.smithi175.stdout:(16/150): librados-devel-19.0.0-3133.ga8c84f59. 1.1 MB/s | 126 kB 00:00 2024-04-20T11:23:57.801 INFO:teuthology.orchestra.run.smithi152.stdout:(69/150): postgresql-private-libs-13.14-1.el9.x 150 kB/s | 133 kB 00:00 2024-04-20T11:23:58.001 INFO:teuthology.orchestra.run.smithi175.stdout:(17/150): libradosstriper1-19.0.0-3133.ga8c84f5 1.9 MB/s | 501 kB 00:00 2024-04-20T11:23:58.168 INFO:teuthology.orchestra.run.smithi152.stdout:(70/150): openssl-devel-3.0.7-27.el9.x86_64.rpm 683 kB/s | 4.1 MB 00:06 2024-04-20T11:23:59.051 INFO:teuthology.orchestra.run.smithi152.stdout:(71/150): openblas-openmp-0.3.21-2.el9.x86_64.r 680 kB/s | 4.7 MB 00:07 2024-04-20T11:23:59.885 INFO:teuthology.orchestra.run.smithi152.stdout:(72/150): python3-attrs-20.3.0-7.el9.noarch.rpm 105 kB/s | 87 kB 00:00 2024-04-20T11:24:00.109 INFO:teuthology.orchestra.run.smithi175.stdout:(18/150): librgw2-19.0.0-3133.ga8c84f59.el9.x86 2.5 MB/s | 5.4 MB 00:02 2024-04-20T11:24:00.209 INFO:teuthology.orchestra.run.smithi175.stdout:(19/150): python3-ceph-argparse-19.0.0-3133.ga8 456 kB/s | 45 kB 00:00 2024-04-20T11:24:00.340 INFO:teuthology.orchestra.run.smithi175.stdout:(20/150): python3-ceph-common-19.0.0-3133.ga8c8 1.0 MB/s | 135 kB 00:00 2024-04-20T11:24:00.457 INFO:teuthology.orchestra.run.smithi152.stdout:(73/150): protobuf-3.14.0-13.el9.x86_64.rpm 466 kB/s | 1.0 MB 00:02 2024-04-20T11:24:00.977 INFO:teuthology.orchestra.run.smithi175.stdout:(21/150): python3-cephfs-19.0.0-3133.ga8c84f59. 255 kB/s | 162 kB 00:00 2024-04-20T11:24:01.218 INFO:teuthology.orchestra.run.smithi175.stdout:(22/150): python3-rados-19.0.0-3133.ga8c84f59.e 1.3 MB/s | 321 kB 00:00 2024-04-20T11:24:01.443 INFO:teuthology.orchestra.run.smithi175.stdout:(23/150): python3-rbd-19.0.0-3133.ga8c84f59.el9 1.3 MB/s | 299 kB 00:00 2024-04-20T11:24:01.518 INFO:teuthology.orchestra.run.smithi152.stdout:(74/150): python3-devel-3.9.18-3.el9.x86_64.rpm 230 kB/s | 244 kB 00:01 2024-04-20T11:24:01.544 INFO:teuthology.orchestra.run.smithi175.stdout:(24/150): python3-rgw-19.0.0-3133.ga8c84f59.el9 1.0 MB/s | 99 kB 00:00 2024-04-20T11:24:01.644 INFO:teuthology.orchestra.run.smithi175.stdout:(25/150): rbd-fuse-19.0.0-3133.ga8c84f59.el9.x8 857 kB/s | 85 kB 00:00 2024-04-20T11:24:02.560 INFO:teuthology.orchestra.run.smithi152.stdout:(75/150): python3-jinja2-2.11.3-5.el9.noarch.rp 238 kB/s | 248 kB 00:01 2024-04-20T11:24:02.937 INFO:teuthology.orchestra.run.smithi175.stdout:(26/150): rbd-mirror-19.0.0-3133.ga8c84f59.el9. 2.4 MB/s | 3.1 MB 00:01 2024-04-20T11:24:03.036 INFO:teuthology.orchestra.run.smithi175.stdout:(27/150): rbd-nbd-19.0.0-3133.ga8c84f59.el9.x86 1.7 MB/s | 170 kB 00:00 2024-04-20T11:24:03.120 INFO:teuthology.orchestra.run.smithi175.stdout:(28/150): ceph-grafana-dashboards-19.0.0-3133.g 413 kB/s | 34 kB 00:00 2024-04-20T11:24:03.204 INFO:teuthology.orchestra.run.smithi175.stdout:(29/150): ceph-mgr-cephadm-19.0.0-3133.ga8c84f5 1.7 MB/s | 142 kB 00:00 2024-04-20T11:24:03.325 INFO:teuthology.orchestra.run.smithi152.stdout:(76/150): python3-jsonpatch-1.21-16.el9.noarch. 34 kB/s | 26 kB 00:00 2024-04-20T11:24:03.970 INFO:teuthology.orchestra.run.smithi152.stdout:(77/150): python3-jsonpointer-2.0-4.el9.noarch. 30 kB/s | 19 kB 00:00 2024-04-20T11:24:04.662 INFO:teuthology.orchestra.run.smithi175.stdout:(30/150): ceph-mgr-dashboard-19.0.0-3133.ga8c84 2.4 MB/s | 3.5 MB 00:01 2024-04-20T11:24:04.961 INFO:teuthology.orchestra.run.smithi152.stdout:(78/150): python3-mako-1.1.4-6.el9.noarch.rpm 174 kB/s | 172 kB 00:00 2024-04-20T11:24:05.745 INFO:teuthology.orchestra.run.smithi152.stdout:(79/150): python3-markupsafe-1.1.1-12.el9.x86_6 44 kB/s | 35 kB 00:00 2024-04-20T11:24:06.587 INFO:teuthology.orchestra.run.smithi175.stdout:(31/150): ceph-radosgw-19.0.0-3133.ga8c84f59.el 714 kB/s | 11 MB 00:15 2024-04-20T11:24:06.887 INFO:teuthology.orchestra.run.smithi175.stdout:(32/150): ceph-mgr-modules-core-19.0.0-3133.ga8 832 kB/s | 249 kB 00:00 2024-04-20T11:24:06.916 INFO:teuthology.orchestra.run.smithi152.stdout:(80/150): postgresql-server-13.14-1.el9.x86_64. 649 kB/s | 5.8 MB 00:09 2024-04-20T11:24:07.004 INFO:teuthology.orchestra.run.smithi175.stdout:(33/150): ceph-mgr-rook-19.0.0-3133.ga8c84f59.e 426 kB/s | 49 kB 00:00 2024-04-20T11:24:07.150 INFO:teuthology.orchestra.run.smithi175.stdout:(34/150): ceph-prometheus-alerts-19.0.0-3133.ga 113 kB/s | 16 kB 00:00 2024-04-20T11:24:07.288 INFO:teuthology.orchestra.run.smithi175.stdout:(35/150): ceph-mgr-diskprediction-local-19.0.0- 2.8 MB/s | 7.4 MB 00:02 2024-04-20T11:24:07.472 INFO:teuthology.orchestra.run.smithi175.stdout:(36/150): ceph-volume-19.0.0-3133.ga8c84f59.el9 853 kB/s | 273 kB 00:00 2024-04-20T11:24:07.588 INFO:teuthology.orchestra.run.smithi175.stdout:(37/150): cephadm-19.0.0-3133.ga8c84f59.el9.noa 2.5 MB/s | 763 kB 00:00 2024-04-20T11:24:07.622 INFO:teuthology.orchestra.run.smithi175.stdout:(38/150): cephfs-top-19.0.0-3133.ga8c84f59.el9. 122 kB/s | 18 kB 00:00 2024-04-20T11:24:08.062 INFO:teuthology.orchestra.run.smithi152.stdout:(81/150): python3-numpy-f2py-1.20.1-5.el9.x86_6 219 kB/s | 250 kB 00:01 2024-04-20T11:24:08.123 INFO:teuthology.orchestra.run.smithi175.stdout:(39/150): fuse-2.9.9-15.el9.x86_64.rpm 151 kB/s | 80 kB 00:00 2024-04-20T11:24:08.458 INFO:teuthology.orchestra.run.smithi175.stdout:(40/150): libquadmath-11.4.1-3.el9.x86_64.rpm 584 kB/s | 195 kB 00:00 2024-04-20T11:24:08.523 INFO:teuthology.orchestra.run.smithi175.stdout:(41/150): libgfortran-11.4.1-3.el9.x86_64.rpm 892 kB/s | 803 kB 00:00 2024-04-20T11:24:08.682 INFO:teuthology.orchestra.run.smithi175.stdout:(42/150): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.1 MB/s | 253 kB 00:00 2024-04-20T11:24:08.807 INFO:teuthology.orchestra.run.smithi175.stdout:(43/150): python3-ply-3.11-14.el9.noarch.rpm 857 kB/s | 106 kB 00:00 2024-04-20T11:24:08.873 INFO:teuthology.orchestra.run.smithi175.stdout:(44/150): python3-cryptography-36.0.1-4.el9.x86 3.5 MB/s | 1.2 MB 00:00 2024-04-20T11:24:08.941 INFO:teuthology.orchestra.run.smithi175.stdout:(45/150): python3-pycparser-2.20-6.el9.noarch.r 1.0 MB/s | 135 kB 00:00 2024-04-20T11:24:08.991 INFO:teuthology.orchestra.run.smithi175.stdout:(46/150): python3-requests-2.25.1-8.el9.noarch. 1.0 MB/s | 125 kB 00:00 2024-04-20T11:24:09.058 INFO:teuthology.orchestra.run.smithi175.stdout:(47/150): python3-urllib3-1.26.5-5.el9.noarch.r 1.8 MB/s | 215 kB 00:00 2024-04-20T11:24:09.125 INFO:teuthology.orchestra.run.smithi175.stdout:(48/150): automake-1.16.2-8.el9.noarch.rpm 5.1 MB/s | 689 kB 00:00 2024-04-20T11:24:09.129 INFO:teuthology.orchestra.run.smithi152.stdout:(82/150): python3-oauthlib-3.1.1-5.el9.noarch.r 209 kB/s | 222 kB 00:01 2024-04-20T11:24:09.251 INFO:teuthology.orchestra.run.smithi175.stdout:(49/150): boost-program-options-1.75.0-8.el9.x8 861 kB/s | 107 kB 00:00 2024-04-20T11:24:09.375 INFO:teuthology.orchestra.run.smithi175.stdout:(50/150): elfutils-libelf-devel-0.190-2.el9.x86 173 kB/s | 21 kB 00:00 2024-04-20T11:24:09.409 INFO:teuthology.orchestra.run.smithi175.stdout:(51/150): bison-3.7.4-5.el9.x86_64.rpm 2.6 MB/s | 943 kB 00:00 2024-04-20T11:24:09.568 INFO:teuthology.orchestra.run.smithi175.stdout:(52/150): flexiblas-3.0.4-8.el9.x86_64.rpm 207 kB/s | 33 kB 00:00 2024-04-20T11:24:09.705 INFO:teuthology.orchestra.run.smithi152.stdout:(83/150): python3-babel-2.9.1-2.el9.noarch.rpm 622 kB/s | 6.0 MB 00:09 2024-04-20T11:24:09.860 INFO:teuthology.orchestra.run.smithi175.stdout:(53/150): flexiblas-netlib-3.0.4-8.el9.x86_64.r 10 MB/s | 3.0 MB 00:00 2024-04-20T11:24:10.077 INFO:teuthology.orchestra.run.smithi175.stdout:(54/150): flexiblas-openblas-openmp-3.0.4-8.el9 84 kB/s | 18 kB 00:00 2024-04-20T11:24:10.146 INFO:teuthology.orchestra.run.smithi152.stdout:(84/150): python3-packaging-20.9-5.el9.noarch.r 76 kB/s | 77 kB 00:01 2024-04-20T11:24:10.219 INFO:teuthology.orchestra.run.smithi175.stdout:(55/150): libaio-devel-0.3.111-13.el9.x86_64.rp 79 kB/s | 11 kB 00:00 2024-04-20T11:24:10.505 INFO:teuthology.orchestra.run.smithi152.stdout:(85/150): python3-prettytable-0.7.2-27.el9.noar 53 kB/s | 42 kB 00:00 2024-04-20T11:24:10.519 INFO:teuthology.orchestra.run.smithi175.stdout:(56/150): libpmemobj-1.12.1-1.el9.x86_64.rpm 536 kB/s | 160 kB 00:00 2024-04-20T11:24:10.653 INFO:teuthology.orchestra.run.smithi175.stdout:(57/150): librabbitmq-0.11.0-7.el9.x86_64.rpm 341 kB/s | 45 kB 00:00 2024-04-20T11:24:10.878 INFO:teuthology.orchestra.run.smithi175.stdout:(58/150): flex-2.6.4-9.el9.x86_64.rpm 209 kB/s | 313 kB 00:01 2024-04-20T11:24:11.011 INFO:teuthology.orchestra.run.smithi175.stdout:(59/150): libtool-2.4.6-45.el9.x86_64.rpm 4.3 MB/s | 582 kB 00:00 2024-04-20T11:24:11.145 INFO:teuthology.orchestra.run.smithi175.stdout:(60/150): libuuid-devel-2.37.4-18.el9.x86_64.rp 192 kB/s | 25 kB 00:00 2024-04-20T11:24:11.254 INFO:teuthology.orchestra.run.smithi175.stdout:(61/150): libzstd-devel-1.5.1-2.el9.x86_64.rpm 441 kB/s | 47 kB 00:00 2024-04-20T11:24:11.313 INFO:teuthology.orchestra.run.smithi152.stdout:(86/150): python3-protobuf-3.14.0-13.el9.noarch 232 kB/s | 269 kB 00:01 2024-04-20T11:24:11.379 INFO:teuthology.orchestra.run.smithi175.stdout:(62/150): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.3 MB/s | 292 kB 00:00 2024-04-20T11:24:11.504 INFO:teuthology.orchestra.run.smithi175.stdout:(63/150): lua-5.4.4-4.el9.x86_64.rpm 1.5 MB/s | 188 kB 00:00 2024-04-20T11:24:11.555 INFO:teuthology.orchestra.run.smithi152.stdout:(87/150): python3-pyasn1-0.4.8-6.el9.noarch.rpm 151 kB/s | 159 kB 00:01 2024-04-20T11:24:11.613 INFO:teuthology.orchestra.run.smithi175.stdout:(64/150): openblas-0.3.21-2.el9.x86_64.rpm 327 kB/s | 35 kB 00:00 2024-04-20T11:24:11.729 INFO:teuthology.orchestra.run.smithi175.stdout:(65/150): librdkafka-1.6.1-102.el9.x86_64.rpm 616 kB/s | 662 kB 00:01 2024-04-20T11:24:12.297 INFO:teuthology.orchestra.run.smithi152.stdout:(88/150): python3-pytz-2021.1-5.el9.noarch.rpm 69 kB/s | 51 kB 00:00 2024-04-20T11:24:12.481 INFO:teuthology.orchestra.run.smithi152.stdout:(89/150): python3-pyasn1-modules-0.4.8-6.el9.no 239 kB/s | 279 kB 00:01 2024-04-20T11:24:12.796 INFO:teuthology.orchestra.run.smithi175.stdout:(66/150): openblas-openmp-0.3.21-2.el9.x86_64.r 4.0 MB/s | 4.7 MB 00:01 2024-04-20T11:24:13.156 INFO:teuthology.orchestra.run.smithi152.stdout:(90/150): python3-requests-oauthlib-1.3.0-12.el 63 kB/s | 54 kB 00:00 2024-04-20T11:24:13.288 INFO:teuthology.orchestra.run.smithi175.stdout:(67/150): postgresql-13.14-1.el9.x86_64.rpm 3.2 MB/s | 1.6 MB 00:00 2024-04-20T11:24:13.615 INFO:teuthology.orchestra.run.smithi175.stdout:(68/150): postgresql-contrib-13.14-1.el9.x86_64 2.7 MB/s | 911 kB 00:00 2024-04-20T11:24:13.731 INFO:teuthology.orchestra.run.smithi175.stdout:(69/150): postgresql-private-libs-13.14-1.el9.x 1.1 MB/s | 133 kB 00:00 2024-04-20T11:24:13.848 INFO:teuthology.orchestra.run.smithi152.stdout:(91/150): python3-toml-0.10.2-6.el9.noarch.rpm 60 kB/s | 42 kB 00:00 2024-04-20T11:24:14.040 INFO:teuthology.orchestra.run.smithi152.stdout:(92/150): python3-numpy-1.20.1-5.el9.x86_64.rpm 629 kB/s | 5.1 MB 00:08 2024-04-20T11:24:14.665 INFO:teuthology.orchestra.run.smithi152.stdout:(93/150): python3-wcwidth-0.2.5-8.el9.noarch.rp 54 kB/s | 44 kB 00:00 2024-04-20T11:24:15.115 INFO:teuthology.orchestra.run.smithi152.stdout:(94/150): qatlib-24.02.0-1.el9.x86_64.rpm 206 kB/s | 221 kB 00:01 2024-04-20T11:24:15.349 INFO:teuthology.orchestra.run.smithi152.stdout:(95/150): qatlib-service-24.02.0-1.el9.x86_64.r 53 kB/s | 36 kB 00:00 2024-04-20T11:24:15.506 INFO:teuthology.orchestra.run.smithi175.stdout:(70/150): ceph-test-19.0.0-3133.ga8c84f59.el9.x 2.5 MB/s | 49 MB 00:19 2024-04-20T11:24:15.639 INFO:teuthology.orchestra.run.smithi175.stdout:(71/150): postgresql-server-13.14-1.el9.x86_64. 3.0 MB/s | 5.8 MB 00:01 2024-04-20T11:24:15.764 INFO:teuthology.orchestra.run.smithi175.stdout:(72/150): python3-attrs-20.3.0-7.el9.noarch.rpm 705 kB/s | 87 kB 00:00 2024-04-20T11:24:15.824 INFO:teuthology.orchestra.run.smithi152.stdout:(96/150): qatzip-libs-1.2.0-1.el9.x86_64.rpm 67 kB/s | 47 kB 00:00 2024-04-20T11:24:15.848 INFO:teuthology.orchestra.run.smithi175.stdout:(73/150): openssl-devel-3.0.7-27.el9.x86_64.rpm 1.0 MB/s | 4.1 MB 00:04 2024-04-20T11:24:16.207 INFO:teuthology.orchestra.run.smithi175.stdout:(74/150): python3-devel-3.9.18-3.el9.x86_64.rpm 683 kB/s | 244 kB 00:00 2024-04-20T11:24:16.440 INFO:teuthology.orchestra.run.smithi175.stdout:(75/150): protobuf-3.14.0-13.el9.x86_64.rpm 1.1 MB/s | 1.0 MB 00:00 2024-04-20T11:24:16.474 INFO:teuthology.orchestra.run.smithi152.stdout:(97/150): socat-1.7.4.1-5.el9.x86_64.rpm 289 kB/s | 305 kB 00:01 2024-04-20T11:24:16.557 INFO:teuthology.orchestra.run.smithi175.stdout:(76/150): python3-jsonpatch-1.21-16.el9.noarch. 220 kB/s | 26 kB 00:00 2024-04-20T11:24:16.583 INFO:teuthology.orchestra.run.smithi152.stdout:(98/150): uuid-1.6.2-55.el9.x86_64.rpm 76 kB/s | 58 kB 00:00 2024-04-20T11:24:16.591 INFO:teuthology.orchestra.run.smithi175.stdout:(77/150): python3-jinja2-2.11.3-5.el9.noarch.rp 649 kB/s | 248 kB 00:00 2024-04-20T11:24:16.666 INFO:teuthology.orchestra.run.smithi175.stdout:(78/150): python3-jsonpointer-2.0-4.el9.noarch. 177 kB/s | 19 kB 00:00 2024-04-20T11:24:16.774 INFO:teuthology.orchestra.run.smithi175.stdout:(79/150): python3-markupsafe-1.1.1-12.el9.x86_6 323 kB/s | 35 kB 00:00 2024-04-20T11:24:16.900 INFO:teuthology.orchestra.run.smithi175.stdout:(80/150): python3-mako-1.1.4-6.el9.noarch.rpm 559 kB/s | 172 kB 00:00 2024-04-20T11:24:17.317 INFO:teuthology.orchestra.run.smithi175.stdout:(81/150): python3-numpy-f2py-1.20.1-5.el9.x86_6 602 kB/s | 250 kB 00:00 2024-04-20T11:24:17.350 INFO:teuthology.orchestra.run.smithi152.stdout:(99/150): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 107 kB/s | 94 kB 00:00 2024-04-20T11:24:17.476 INFO:teuthology.orchestra.run.smithi152.stdout:(100/150): xmlstarlet-1.6.1-20.el9.x86_64.rpm 71 kB/s | 64 kB 00:00 2024-04-20T11:24:17.500 INFO:teuthology.orchestra.run.smithi175.stdout:(82/150): python3-babel-2.9.1-2.el9.noarch.rpm 3.4 MB/s | 6.0 MB 00:01 2024-04-20T11:24:17.601 INFO:teuthology.orchestra.run.smithi152.stdout:(101/150): lua-devel-5.4.4-4.el9.x86_64.rpm 180 kB/s | 22 kB 00:00 2024-04-20T11:24:17.609 INFO:teuthology.orchestra.run.smithi175.stdout:(83/150): python3-packaging-20.9-5.el9.noarch.r 718 kB/s | 77 kB 00:00 2024-04-20T11:24:17.726 INFO:teuthology.orchestra.run.smithi175.stdout:(84/150): python3-oauthlib-3.1.1-5.el9.noarch.r 544 kB/s | 222 kB 00:00 2024-04-20T11:24:17.751 INFO:teuthology.orchestra.run.smithi175.stdout:(85/150): python3-prettytable-0.7.2-27.el9.noar 299 kB/s | 42 kB 00:00 2024-04-20T11:24:17.860 INFO:teuthology.orchestra.run.smithi175.stdout:(86/150): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.4 MB/s | 159 kB 00:00 2024-04-20T11:24:17.977 INFO:teuthology.orchestra.run.smithi175.stdout:(87/150): python3-pyasn1-modules-0.4.8-6.el9.no 2.3 MB/s | 279 kB 00:00 2024-04-20T11:24:17.984 INFO:teuthology.orchestra.run.smithi152.stdout:(102/150): protobuf-compiler-3.14.0-13.el9.x86_ 2.7 MB/s | 863 kB 00:00 2024-04-20T11:24:18.085 INFO:teuthology.orchestra.run.smithi175.stdout:(88/150): python3-pytz-2021.1-5.el9.noarch.rpm 475 kB/s | 51 kB 00:00 2024-04-20T11:24:18.110 INFO:teuthology.orchestra.run.smithi152.stdout:(103/150): zlib-devel-1.2.11-41.el9.x86_64.rpm 59 kB/s | 45 kB 00:00 2024-04-20T11:24:18.169 INFO:teuthology.orchestra.run.smithi175.stdout:(89/150): python3-protobuf-3.14.0-13.el9.noarch 609 kB/s | 269 kB 00:00 2024-04-20T11:24:18.194 INFO:teuthology.orchestra.run.smithi175.stdout:(90/150): python3-requests-oauthlib-1.3.0-12.el 497 kB/s | 54 kB 00:00 2024-04-20T11:24:18.302 INFO:teuthology.orchestra.run.smithi175.stdout:(91/150): python3-toml-0.10.2-6.el9.noarch.rpm 386 kB/s | 42 kB 00:00 2024-04-20T11:24:18.310 INFO:teuthology.orchestra.run.smithi152.stdout:(104/150): abseil-cpp-20211102.0-3.el9.x86_64.r 1.7 MB/s | 548 kB 00:00 2024-04-20T11:24:18.343 INFO:teuthology.orchestra.run.smithi152.stdout:(105/150): gperftools-libs-2.9.1-2.el9.x86_64.r 1.3 MB/s | 309 kB 00:00 2024-04-20T11:24:18.369 INFO:teuthology.orchestra.run.smithi152.stdout:(106/150): grpc-data-1.46.7-10.el9.noarch.rpm 353 kB/s | 19 kB 00:00 2024-04-20T11:24:18.435 INFO:teuthology.orchestra.run.smithi175.stdout:(92/150): python3-wcwidth-0.2.5-8.el9.noarch.rp 335 kB/s | 44 kB 00:00 2024-04-20T11:24:18.552 INFO:teuthology.orchestra.run.smithi152.stdout:(107/150): libarrow-9.0.0-10.el9.x86_64.rpm 21 MB/s | 4.4 MB 00:00 2024-04-20T11:24:18.578 INFO:teuthology.orchestra.run.smithi152.stdout:(108/150): libarrow-doc-9.0.0-10.el9.noarch.rpm 131 kB/s | 26 kB 00:00 2024-04-20T11:24:18.591 INFO:teuthology.orchestra.run.smithi175.stdout:(93/150): python3-numpy-1.20.1-5.el9.x86_64.rpm 2.8 MB/s | 5.1 MB 00:01 2024-04-20T11:24:18.628 INFO:teuthology.orchestra.run.smithi152.stdout:(109/150): liboath-2.6.7-2.el9.x86_64.rpm 997 kB/s | 49 kB 00:00 2024-04-20T11:24:18.628 INFO:teuthology.orchestra.run.smithi175.stdout:(94/150): qatlib-24.02.0-1.el9.x86_64.rpm 1.1 MB/s | 221 kB 00:00 2024-04-20T11:24:18.645 INFO:teuthology.orchestra.run.smithi152.stdout:(110/150): libunwind-1.6.2-1.el9.x86_64.rpm 1.0 MB/s | 67 kB 00:00 2024-04-20T11:24:18.703 INFO:teuthology.orchestra.run.smithi152.stdout:(111/150): luarocks-3.9.2-1.el9.noarch.rpm 2.6 MB/s | 151 kB 00:00 2024-04-20T11:24:18.704 INFO:teuthology.orchestra.run.smithi175.stdout:(95/150): qatlib-service-24.02.0-1.el9.x86_64.r 322 kB/s | 36 kB 00:00 2024-04-20T11:24:18.737 INFO:teuthology.orchestra.run.smithi175.stdout:(96/150): qatzip-libs-1.2.0-1.el9.x86_64.rpm 440 kB/s | 47 kB 00:00 2024-04-20T11:24:18.745 INFO:teuthology.orchestra.run.smithi152.stdout:(112/150): parquet-libs-9.0.0-10.el9.x86_64.rpm 8.3 MB/s | 838 kB 00:00 2024-04-20T11:24:18.812 INFO:teuthology.orchestra.run.smithi152.stdout:(113/150): python3-asyncssh-2.13.2-5.el9.noarch 8.2 MB/s | 548 kB 00:00 2024-04-20T11:24:18.862 INFO:teuthology.orchestra.run.smithi175.stdout:(97/150): uuid-1.6.2-55.el9.x86_64.rpm 466 kB/s | 58 kB 00:00 2024-04-20T11:24:18.864 INFO:teuthology.orchestra.run.smithi152.stdout:(114/150): python3-bcrypt-3.2.2-1.el9.x86_64.rp 372 kB/s | 43 kB 00:00 2024-04-20T11:24:18.913 INFO:teuthology.orchestra.run.smithi152.stdout:(115/150): python3-cachetools-4.2.4-1.el9.noarc 672 kB/s | 32 kB 00:00 2024-04-20T11:24:18.938 INFO:teuthology.orchestra.run.smithi152.stdout:(116/150): python3-certifi-2021.10.8-2.el9.noar 205 kB/s | 15 kB 00:00 2024-04-20T11:24:18.972 INFO:teuthology.orchestra.run.smithi152.stdout:(117/150): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-04-20T11:24:18.979 INFO:teuthology.orchestra.run.smithi175.stdout:(98/150): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 807 kB/s | 94 kB 00:00 2024-04-20T11:24:19.005 INFO:teuthology.orchestra.run.smithi152.stdout:(118/150): python3-cherrypy-18.6.1-2.el9.noarch 5.3 MB/s | 358 kB 00:00 2024-04-20T11:24:19.056 INFO:teuthology.orchestra.run.smithi152.stdout:(119/150): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.7 MB/s | 303 kB 00:00 2024-04-20T11:24:19.081 INFO:teuthology.orchestra.run.smithi152.stdout:(120/150): python3-colorama-0.4.4-12.el9.noarch 493 kB/s | 35 kB 00:00 2024-04-20T11:24:19.088 INFO:teuthology.orchestra.run.smithi175.stdout:(99/150): xmlstarlet-1.6.1-20.el9.x86_64.rpm 591 kB/s | 64 kB 00:00 2024-04-20T11:24:19.115 INFO:teuthology.orchestra.run.smithi152.stdout:(121/150): python3-google-auth-2.28.2-1.el9.noa 3.5 MB/s | 203 kB 00:00 2024-04-20T11:24:19.188 INFO:teuthology.orchestra.run.smithi175.stdout:(100/150): socat-1.7.4.1-5.el9.x86_64.rpm 630 kB/s | 305 kB 00:00 2024-04-20T11:24:19.213 INFO:teuthology.orchestra.run.smithi175.stdout:(101/150): zlib-devel-1.2.11-41.el9.x86_64.rpm 360 kB/s | 45 kB 00:00 2024-04-20T11:24:19.215 INFO:teuthology.orchestra.run.smithi152.stdout:(122/150): python3-grpcio-1.46.7-10.el9.x86_64. 15 MB/s | 2.0 MB 00:00 2024-04-20T11:24:19.240 INFO:teuthology.orchestra.run.smithi152.stdout:(123/150): python3-grpcio-tools-1.46.7-10.el9.x 1.3 MB/s | 144 kB 00:00 2024-04-20T11:24:19.266 INFO:teuthology.orchestra.run.smithi152.stdout:(124/150): python3-jaraco-8.2.1-3.el9.noarch.rp 223 kB/s | 11 kB 00:00 2024-04-20T11:24:19.291 INFO:teuthology.orchestra.run.smithi152.stdout:(125/150): python3-jaraco-classes-3.2.1-5.el9.n 357 kB/s | 18 kB 00:00 2024-04-20T11:24:19.316 INFO:teuthology.orchestra.run.smithi152.stdout:(126/150): python3-jaraco-collections-3.0.0-8.e 465 kB/s | 23 kB 00:00 2024-04-20T11:24:19.322 INFO:teuthology.orchestra.run.smithi175.stdout:(102/150): lua-devel-5.4.4-4.el9.x86_64.rpm 168 kB/s | 22 kB 00:00 2024-04-20T11:24:19.350 INFO:teuthology.orchestra.run.smithi152.stdout:(127/150): python3-jaraco-functools-3.5.0-2.el9 336 kB/s | 19 kB 00:00 2024-04-20T11:24:19.375 INFO:teuthology.orchestra.run.smithi152.stdout:(128/150): python3-jaraco-text-3.2.0-6.el9.noar 338 kB/s | 20 kB 00:00 2024-04-20T11:24:19.459 INFO:teuthology.orchestra.run.smithi152.stdout:(129/150): python3-kubernetes-26.1.0-2.el9.noar 9.5 MB/s | 1.0 MB 00:00 2024-04-20T11:24:19.489 INFO:teuthology.orchestra.run.smithi175.stdout:(103/150): protobuf-compiler-3.14.0-13.el9.x86_ 3.1 MB/s | 863 kB 00:00 2024-04-20T11:24:19.493 INFO:teuthology.orchestra.run.smithi152.stdout:(130/150): python3-logutils-0.3.5-21.el9.noarch 397 kB/s | 46 kB 00:00 2024-04-20T11:24:19.518 INFO:teuthology.orchestra.run.smithi152.stdout:(131/150): python3-more-itertools-8.12.0-2.el9. 1.3 MB/s | 79 kB 00:00 2024-04-20T11:24:19.543 INFO:teuthology.orchestra.run.smithi152.stdout:(132/150): python3-natsort-7.1.1-5.el9.noarch.r 1.2 MB/s | 58 kB 00:00 2024-04-20T11:24:19.577 INFO:teuthology.orchestra.run.smithi152.stdout:(133/150): python3-pecan-1.4.2-3.el9.noarch.rpm 4.6 MB/s | 272 kB 00:00 2024-04-20T11:24:19.603 INFO:teuthology.orchestra.run.smithi152.stdout:(134/150): python3-portend-3.1.0-2.el9.noarch.r 282 kB/s | 16 kB 00:00 2024-04-20T11:24:19.628 INFO:teuthology.orchestra.run.smithi152.stdout:(135/150): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.8 MB/s | 90 kB 00:00 2024-04-20T11:24:19.653 INFO:teuthology.orchestra.run.smithi152.stdout:(136/150): python3-pyperclip-1.8.2-2.el9.noarch 585 kB/s | 28 kB 00:00 2024-04-20T11:24:19.678 INFO:teuthology.orchestra.run.smithi152.stdout:(137/150): python3-repoze-lru-0.7-16.el9.noarch 621 kB/s | 31 kB 00:00 2024-04-20T11:24:19.712 INFO:teuthology.orchestra.run.smithi152.stdout:(138/150): python3-routes-2.5.1-5.el9.noarch.rp 3.2 MB/s | 188 kB 00:00 2024-04-20T11:24:19.737 INFO:teuthology.orchestra.run.smithi152.stdout:(139/150): python3-rsa-4.9-2.el9.noarch.rpm 1.0 MB/s | 59 kB 00:00 2024-04-20T11:24:19.763 INFO:teuthology.orchestra.run.smithi152.stdout:(140/150): python3-tempora-5.0.0-2.el9.noarch.r 725 kB/s | 36 kB 00:00 2024-04-20T11:24:19.788 INFO:teuthology.orchestra.run.smithi152.stdout:(141/150): python3-typing-extensions-4.4.0-2.el 1.0 MB/s | 51 kB 00:00 2024-04-20T11:24:19.821 INFO:teuthology.orchestra.run.smithi152.stdout:(142/150): python3-webob-1.8.7-6.el9.noarch.rpm 3.9 MB/s | 230 kB 00:00 2024-04-20T11:24:19.847 INFO:teuthology.orchestra.run.smithi152.stdout:(143/150): python3-websocket-client-1.2.3-2.el9 1.5 MB/s | 90 kB 00:00 2024-04-20T11:24:19.897 INFO:teuthology.orchestra.run.smithi152.stdout:(144/150): python3-werkzeug-2.0.3-3.el9.1.noarc 5.6 MB/s | 427 kB 00:00 2024-04-20T11:24:19.897 INFO:teuthology.orchestra.run.smithi175.stdout:(104/150): abseil-cpp-20211102.0-3.el9.x86_64.r 954 kB/s | 548 kB 00:00 2024-04-20T11:24:19.931 INFO:teuthology.orchestra.run.smithi152.stdout:(145/150): python3-zc-lockfile-2.0-10.el9.noarc 241 kB/s | 20 kB 00:00 2024-04-20T11:24:19.966 INFO:teuthology.orchestra.run.smithi152.stdout:(146/150): re2-20211101-3.el9.x86_64.rpm 2.8 MB/s | 192 kB 00:00 2024-04-20T11:24:19.998 INFO:teuthology.orchestra.run.smithi175.stdout:(105/150): grpc-data-1.46.7-10.el9.noarch.rpm 196 kB/s | 19 kB 00:00 2024-04-20T11:24:20.031 INFO:teuthology.orchestra.run.smithi175.stdout:(106/150): gperftools-libs-2.9.1-2.el9.x86_64.r 570 kB/s | 309 kB 00:00 2024-04-20T11:24:20.040 INFO:teuthology.orchestra.run.smithi152.stdout:(147/150): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-04-20T11:24:20.148 INFO:teuthology.orchestra.run.smithi175.stdout:(107/150): libarrow-doc-9.0.0-10.el9.noarch.rpm 224 kB/s | 26 kB 00:00 2024-04-20T11:24:20.240 INFO:teuthology.orchestra.run.smithi175.stdout:(108/150): liboath-2.6.7-2.el9.x86_64.rpm 532 kB/s | 49 kB 00:00 2024-04-20T11:24:20.382 INFO:teuthology.orchestra.run.smithi175.stdout:(109/150): libarrow-9.0.0-10.el9.x86_64.rpm 12 MB/s | 4.4 MB 00:00 2024-04-20T11:24:20.408 INFO:teuthology.orchestra.run.smithi175.stdout:(110/150): libunwind-1.6.2-1.el9.x86_64.rpm 405 kB/s | 67 kB 00:00 2024-04-20T11:24:20.457 INFO:teuthology.orchestra.run.smithi152.stdout:(148/150): librados2-19.0.0-3133.ga8c84f59.el9. 7.1 MB/s | 3.4 MB 00:00 2024-04-20T11:24:20.466 INFO:teuthology.orchestra.run.smithi175.stdout:(111/150): luarocks-3.9.2-1.el9.noarch.rpm 1.8 MB/s | 151 kB 00:00 2024-04-20T11:24:20.584 INFO:teuthology.orchestra.run.smithi175.stdout:(112/150): python3-asyncssh-2.13.2-5.el9.noarch 4.6 MB/s | 548 kB 00:00 2024-04-20T11:24:20.590 INFO:teuthology.orchestra.run.smithi152.stdout:(149/150): librbd1-19.0.0-3133.ga8c84f59.el9.x8 5.7 MB/s | 3.1 MB 00:00 2024-04-20T11:24:20.634 INFO:teuthology.orchestra.run.smithi175.stdout:(113/150): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.6 MB/s | 838 kB 00:00 2024-04-20T11:24:20.659 INFO:teuthology.orchestra.run.smithi175.stdout:(114/150): python3-bcrypt-3.2.2-1.el9.x86_64.rp 585 kB/s | 43 kB 00:00 2024-04-20T11:24:20.709 INFO:teuthology.orchestra.run.smithi175.stdout:(115/150): python3-cachetools-4.2.4-1.el9.noarc 432 kB/s | 32 kB 00:00 2024-04-20T11:24:20.734 INFO:teuthology.orchestra.run.smithi175.stdout:(116/150): python3-certifi-2021.10.8-2.el9.noar 201 kB/s | 15 kB 00:00 2024-04-20T11:24:20.793 INFO:teuthology.orchestra.run.smithi175.stdout:(117/150): python3-cheroot-8.6.0-4.el9.noarch.r 2.0 MB/s | 172 kB 00:00 2024-04-20T11:24:20.826 INFO:teuthology.orchestra.run.smithi175.stdout:(118/150): python3-cherrypy-18.6.1-2.el9.noarch 3.8 MB/s | 358 kB 00:00 2024-04-20T11:24:20.885 INFO:teuthology.orchestra.run.smithi175.stdout:(119/150): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.3 MB/s | 303 kB 00:00 2024-04-20T11:24:20.910 INFO:teuthology.orchestra.run.smithi175.stdout:(120/150): python3-colorama-0.4.4-12.el9.noarch 425 kB/s | 35 kB 00:00 2024-04-20T11:24:21.002 INFO:teuthology.orchestra.run.smithi175.stdout:(121/150): python3-google-auth-2.28.2-1.el9.noa 1.7 MB/s | 203 kB 00:00 2024-04-20T11:24:21.102 INFO:teuthology.orchestra.run.smithi175.stdout:(122/150): python3-grpcio-1.46.7-10.el9.x86_64. 11 MB/s | 2.0 MB 00:00 2024-04-20T11:24:21.136 INFO:teuthology.orchestra.run.smithi175.stdout:(123/150): python3-grpcio-tools-1.46.7-10.el9.x 1.1 MB/s | 144 kB 00:00 2024-04-20T11:24:21.170 INFO:teuthology.orchestra.run.smithi175.stdout:(124/150): python3-jaraco-8.2.1-3.el9.noarch.rp 162 kB/s | 11 kB 00:00 2024-04-20T11:24:21.303 INFO:teuthology.orchestra.run.smithi175.stdout:(125/150): python3-jaraco-classes-3.2.1-5.el9.n 106 kB/s | 18 kB 00:00 2024-04-20T11:24:21.328 INFO:teuthology.orchestra.run.smithi175.stdout:(126/150): python3-jaraco-collections-3.0.0-8.e 146 kB/s | 23 kB 00:00 2024-04-20T11:24:21.370 INFO:teuthology.orchestra.run.smithi175.stdout:(127/150): python3-jaraco-functools-3.5.0-2.el9 293 kB/s | 19 kB 00:00 2024-04-20T11:24:21.422 INFO:teuthology.orchestra.run.smithi175.stdout:(128/150): python3-jaraco-text-3.2.0-6.el9.noar 213 kB/s | 20 kB 00:00 2024-04-20T11:24:21.496 INFO:teuthology.orchestra.run.smithi175.stdout:(129/150): python3-kubernetes-26.1.0-2.el9.noar 8.2 MB/s | 1.0 MB 00:00 2024-04-20T11:24:21.513 INFO:teuthology.orchestra.run.smithi175.stdout:(130/150): python3-logutils-0.3.5-21.el9.noarch 511 kB/s | 46 kB 00:00 2024-04-20T11:24:21.572 INFO:teuthology.orchestra.run.smithi175.stdout:(131/150): python3-more-itertools-8.12.0-2.el9. 1.0 MB/s | 79 kB 00:00 2024-04-20T11:24:21.588 INFO:teuthology.orchestra.run.smithi175.stdout:(132/150): python3-natsort-7.1.1-5.el9.noarch.r 773 kB/s | 58 kB 00:00 2024-04-20T11:24:21.664 INFO:teuthology.orchestra.run.smithi175.stdout:(133/150): python3-pecan-1.4.2-3.el9.noarch.rpm 2.9 MB/s | 272 kB 00:00 2024-04-20T11:24:21.689 INFO:teuthology.orchestra.run.smithi175.stdout:(134/150): python3-portend-3.1.0-2.el9.noarch.r 164 kB/s | 16 kB 00:00 2024-04-20T11:24:21.747 INFO:teuthology.orchestra.run.smithi175.stdout:(135/150): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.1 MB/s | 90 kB 00:00 2024-04-20T11:24:21.773 INFO:teuthology.orchestra.run.smithi175.stdout:(136/150): python3-pyperclip-1.8.2-2.el9.noarch 340 kB/s | 28 kB 00:00 2024-04-20T11:24:21.823 INFO:teuthology.orchestra.run.smithi175.stdout:(137/150): python3-repoze-lru-0.7-16.el9.noarch 413 kB/s | 31 kB 00:00 2024-04-20T11:24:21.873 INFO:teuthology.orchestra.run.smithi175.stdout:(138/150): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2024-04-20T11:24:21.898 INFO:teuthology.orchestra.run.smithi175.stdout:(139/150): python3-rsa-4.9-2.el9.noarch.rpm 791 kB/s | 59 kB 00:00 2024-04-20T11:24:21.949 INFO:teuthology.orchestra.run.smithi175.stdout:(140/150): python3-tempora-5.0.0-2.el9.noarch.r 480 kB/s | 36 kB 00:00 2024-04-20T11:24:21.974 INFO:teuthology.orchestra.run.smithi175.stdout:(141/150): python3-typing-extensions-4.4.0-2.el 684 kB/s | 51 kB 00:00 2024-04-20T11:24:22.041 INFO:teuthology.orchestra.run.smithi175.stdout:(142/150): python3-webob-1.8.7-6.el9.noarch.rpm 2.5 MB/s | 230 kB 00:00 2024-04-20T11:24:22.075 INFO:teuthology.orchestra.run.smithi175.stdout:(143/150): python3-websocket-client-1.2.3-2.el9 897 kB/s | 90 kB 00:00 2024-04-20T11:24:22.167 INFO:teuthology.orchestra.run.smithi175.stdout:(144/150): python3-zc-lockfile-2.0-10.el9.noarc 219 kB/s | 20 kB 00:00 2024-04-20T11:24:22.200 INFO:teuthology.orchestra.run.smithi175.stdout:(145/150): python3-werkzeug-2.0.3-3.el9.1.noarc 2.6 MB/s | 427 kB 00:00 2024-04-20T11:24:22.276 INFO:teuthology.orchestra.run.smithi175.stdout:(146/150): re2-20211101-3.el9.x86_64.rpm 1.7 MB/s | 192 kB 00:00 2024-04-20T11:24:22.401 INFO:teuthology.orchestra.run.smithi175.stdout:(147/150): thrift-0.15.0-2.el9.x86_64.rpm 7.9 MB/s | 1.6 MB 00:00 2024-04-20T11:24:23.526 INFO:teuthology.orchestra.run.smithi175.stdout:(148/150): librbd1-19.0.0-3133.ga8c84f59.el9.x8 2.8 MB/s | 3.1 MB 00:01 2024-04-20T11:24:24.668 INFO:teuthology.orchestra.run.smithi175.stdout:(149/150): librados2-19.0.0-3133.ga8c84f59.el9. 1.4 MB/s | 3.4 MB 00:02 2024-04-20T11:24:35.564 INFO:teuthology.orchestra.run.smithi152.stdout:(150/150): python3-scipy-1.6.2-8.el9.x86_64.rpm 721 kB/s | 16 MB 00:23 2024-04-20T11:24:35.575 INFO:teuthology.orchestra.run.smithi152.stdout:-------------------------------------------------------------------------------- 2024-04-20T11:24:35.576 INFO:teuthology.orchestra.run.smithi152.stdout:Total 3.5 MB/s | 218 MB 01:01 2024-04-20T11:24:37.057 INFO:teuthology.orchestra.run.smithi175.stdout:(150/150): python3-scipy-1.6.2-8.el9.x86_64.rpm 881 kB/s | 16 MB 00:18 2024-04-20T11:24:37.068 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2024-04-20T11:24:37.069 INFO:teuthology.orchestra.run.smithi175.stdout:Total 4.2 MB/s | 218 MB 00:51 2024-04-20T11:24:38.319 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction check 2024-04-20T11:24:38.551 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction check succeeded. 2024-04-20T11:24:38.551 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction test 2024-04-20T11:24:39.726 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2024-04-20T11:24:39.946 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2024-04-20T11:24:39.946 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2024-04-20T11:24:41.240 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction test succeeded. 2024-04-20T11:24:41.241 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction 2024-04-20T11:24:42.506 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2024-04-20T11:24:42.506 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2024-04-20T11:24:47.436 INFO:teuthology.orchestra.run.smithi152.stdout: Preparing : 1/1 2024-04-20T11:24:47.523 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/152 2024-04-20T11:24:47.560 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/152 2024-04-20T11:24:47.700 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/152 2024-04-20T11:24:48.114 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/152 2024-04-20T11:24:48.209 INFO:teuthology.orchestra.run.smithi152.stdout: Upgrading : librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 5/152 2024-04-20T11:24:48.379 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 5/152 2024-04-20T11:24:48.410 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 6/152 2024-04-20T11:24:48.537 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 6/152 2024-04-20T11:24:48.592 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-rados-2:19.0.0-3133.ga8c84f59.el9.x86_64 7/152 2024-04-20T11:24:48.625 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/152 2024-04-20T11:24:48.661 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/152 2024-04-20T11:24:48.728 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2024-04-20T11:24:48.850 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/152 2024-04-20T11:24:48.895 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/152 2024-04-20T11:24:49.029 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/152 2024-04-20T11:24:49.078 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 10/152 2024-04-20T11:24:49.146 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : postgresql-13.14-1.el9.x86_64 11/152 2024-04-20T11:24:49.174 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 12/152 2024-04-20T11:24:49.343 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 12/152 2024-04-20T11:24:49.401 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86 13/152 2024-04-20T11:24:49.481 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/152 2024-04-20T11:24:49.497 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86 13/152 2024-04-20T11:24:49.526 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-04-20T11:24:49.596 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-04-20T11:24:49.610 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 5/152 2024-04-20T11:24:49.791 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 5/152 2024-04-20T11:24:49.827 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 6/152 2024-04-20T11:24:49.955 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 6/152 2024-04-20T11:24:50.055 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 15/152 2024-04-20T11:24:50.077 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-rados-2:19.0.0-3133.ga8c84f59.el9.x86_64 7/152 2024-04-20T11:24:50.120 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 15/152 2024-04-20T11:24:50.120 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/152 2024-04-20T11:24:50.156 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/152 2024-04-20T11:24:50.321 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : re2-1:20211101-3.el9.x86_64 16/152 2024-04-20T11:24:50.452 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 17/152 2024-04-20T11:24:50.496 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 18/152 2024-04-20T11:24:50.564 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 10/152 2024-04-20T11:24:50.570 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : liboath-2.6.7-2.el9.x86_64 19/152 2024-04-20T11:24:50.632 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : postgresql-13.14-1.el9.x86_64 11/152 2024-04-20T11:24:50.666 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 12/152 2024-04-20T11:24:50.688 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 20/152 2024-04-20T11:24:50.743 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 21/152 2024-04-20T11:24:50.797 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 22/152 2024-04-20T11:24:50.821 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 12/152 2024-04-20T11:24:50.847 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 23/152 2024-04-20T11:24:50.850 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86 13/152 2024-04-20T11:24:50.886 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : lua-5.4.4-4.el9.x86_64 24/152 2024-04-20T11:24:50.943 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86 13/152 2024-04-20T11:24:50.967 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-04-20T11:24:51.003 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 25/152 2024-04-20T11:24:51.048 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-04-20T11:24:51.077 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 26/152 2024-04-20T11:24:51.148 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 27/152 2024-04-20T11:24:51.219 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 28/152 2024-04-20T11:24:51.337 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 29/152 2024-04-20T11:24:51.385 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-ceph-common-2:19.0.0-3133.ga8c84f59.el9. 30/152 2024-04-20T11:24:51.431 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-ceph-argparse-2:19.0.0-3133.ga8c84f59.el 31/152 2024-04-20T11:24:51.467 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-cephfs-2:19.0.0-3133.ga8c84f59.el9.x86_6 32/152 2024-04-20T11:24:51.491 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 15/152 2024-04-20T11:24:51.555 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 15/152 2024-04-20T11:24:51.586 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 33/152 2024-04-20T11:24:51.662 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : luarocks-3.9.2-1.el9.noarch 34/152 2024-04-20T11:24:51.750 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : re2-1:20211101-3.el9.x86_64 16/152 2024-04-20T11:24:51.793 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 35/152 2024-04-20T11:24:51.882 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 17/152 2024-04-20T11:24:51.918 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 18/152 2024-04-20T11:24:51.965 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : liboath-2.6.7-2.el9.x86_64 19/152 2024-04-20T11:24:51.980 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 36/152 2024-04-20T11:24:52.050 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 37/152 2024-04-20T11:24:52.058 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 20/152 2024-04-20T11:24:52.106 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 21/152 2024-04-20T11:24:52.193 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 22/152 2024-04-20T11:24:52.243 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 23/152 2024-04-20T11:24:52.283 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : lua-5.4.4-4.el9.x86_64 24/152 2024-04-20T11:24:52.401 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 25/152 2024-04-20T11:24:52.488 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 26/152 2024-04-20T11:24:52.526 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 27/152 2024-04-20T11:24:52.584 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 28/152 2024-04-20T11:24:52.698 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 29/152 2024-04-20T11:24:52.739 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-ceph-common-2:19.0.0-3133.ga8c84f59.el9. 30/152 2024-04-20T11:24:52.776 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-rsa-4.9-2.el9.noarch 38/152 2024-04-20T11:24:52.789 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-ceph-argparse-2:19.0.0-3133.ga8c84f59.el 31/152 2024-04-20T11:24:52.828 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cephfs-2:19.0.0-3133.ga8c84f59.el9.x86_6 32/152 2024-04-20T11:24:52.881 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 39/152 2024-04-20T11:24:52.923 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 40/152 2024-04-20T11:24:52.959 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 33/152 2024-04-20T11:24:52.968 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 41/152 2024-04-20T11:24:53.010 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : librados-devel-2:19.0.0-3133.ga8c84f59.el9.x86_6 42/152 2024-04-20T11:24:53.044 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : luarocks-3.9.2-1.el9.noarch 34/152 2024-04-20T11:24:53.052 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 43/152 2024-04-20T11:24:53.137 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 44/152 2024-04-20T11:24:53.157 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 35/152 2024-04-20T11:24:53.233 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 45/152 2024-04-20T11:24:53.282 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 46/152 2024-04-20T11:24:53.321 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 47/152 2024-04-20T11:24:53.349 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 36/152 2024-04-20T11:24:53.392 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 48/152 2024-04-20T11:24:53.427 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 37/152 2024-04-20T11:24:53.456 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 49/152 2024-04-20T11:24:53.519 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 50/152 2024-04-20T11:24:53.585 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 51/152 2024-04-20T11:24:53.797 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 52/152 2024-04-20T11:24:53.854 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 53/152 2024-04-20T11:24:53.891 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 54/152 2024-04-20T11:24:53.949 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 55/152 2024-04-20T11:24:54.124 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 56/152 2024-04-20T11:24:54.146 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-rsa-4.9-2.el9.noarch 38/152 2024-04-20T11:24:54.165 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 57/152 2024-04-20T11:24:54.242 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 39/152 2024-04-20T11:24:54.247 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 58/152 2024-04-20T11:24:54.282 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 59/152 2024-04-20T11:24:54.300 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 40/152 2024-04-20T11:24:54.315 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 60/152 2024-04-20T11:24:54.347 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 41/152 2024-04-20T11:24:54.389 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : librados-devel-2:19.0.0-3133.ga8c84f59.el9.x86_6 42/152 2024-04-20T11:24:54.432 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 43/152 2024-04-20T11:24:54.501 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 61/152 2024-04-20T11:24:54.528 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 44/152 2024-04-20T11:24:54.635 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 45/152 2024-04-20T11:24:54.677 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 46/152 2024-04-20T11:24:54.689 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 62/152 2024-04-20T11:24:54.745 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 63/152 2024-04-20T11:24:54.746 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 47/152 2024-04-20T11:24:54.793 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 64/152 2024-04-20T11:24:54.821 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 48/152 2024-04-20T11:24:54.855 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 65/152 2024-04-20T11:24:54.866 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 49/152 2024-04-20T11:24:54.929 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 50/152 2024-04-20T11:24:54.948 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : uuid-1.6.2-55.el9.x86_64 66/152 2024-04-20T11:24:54.994 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 51/152 2024-04-20T11:24:55.015 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 67/152 2024-04-20T11:24:55.069 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 68/152 2024-04-20T11:24:55.118 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 69/152 2024-04-20T11:24:55.195 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 52/152 2024-04-20T11:24:55.198 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 70/152 2024-04-20T11:24:55.249 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 71/152 2024-04-20T11:24:55.255 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 53/152 2024-04-20T11:24:55.293 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 54/152 2024-04-20T11:24:55.324 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 72/152 2024-04-20T11:24:55.351 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 55/152 2024-04-20T11:24:55.370 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 73/152 2024-04-20T11:24:55.491 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 56/152 2024-04-20T11:24:55.515 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 74/152 2024-04-20T11:24:55.552 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 57/152 2024-04-20T11:24:55.577 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 75/152 2024-04-20T11:24:55.620 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 76/152 2024-04-20T11:24:55.620 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 58/152 2024-04-20T11:24:55.657 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 59/152 2024-04-20T11:24:55.690 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 60/152 2024-04-20T11:24:55.692 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 77/152 2024-04-20T11:24:55.876 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-packaging-20.9-5.el9.noarch 78/152 2024-04-20T11:24:55.887 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 61/152 2024-04-20T11:24:55.954 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 79/152 2024-04-20T11:24:56.068 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 62/152 2024-04-20T11:24:56.112 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 63/152 2024-04-20T11:24:56.151 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 64/152 2024-04-20T11:24:56.201 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 65/152 2024-04-20T11:24:56.281 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : uuid-1.6.2-55.el9.x86_64 66/152 2024-04-20T11:24:56.347 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 67/152 2024-04-20T11:24:56.399 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 68/152 2024-04-20T11:24:56.457 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 69/152 2024-04-20T11:24:56.504 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 70/152 2024-04-20T11:24:56.566 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 71/152 2024-04-20T11:24:56.625 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 72/152 2024-04-20T11:24:56.670 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 73/152 2024-04-20T11:24:56.810 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 74/152 2024-04-20T11:24:56.892 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 75/152 2024-04-20T11:24:56.936 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 76/152 2024-04-20T11:24:56.955 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 80/152 2024-04-20T11:24:57.006 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 77/152 2024-04-20T11:24:57.008 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 81/152 2024-04-20T11:24:57.042 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 82/152 2024-04-20T11:24:57.155 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-packaging-20.9-5.el9.noarch 78/152 2024-04-20T11:24:57.176 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 83/152 2024-04-20T11:24:57.224 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 79/152 2024-04-20T11:24:57.246 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 84/152 2024-04-20T11:24:57.351 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 85/152 2024-04-20T11:24:57.393 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 86/152 2024-04-20T11:24:57.626 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : openblas-0.3.21-2.el9.x86_64 87/152 2024-04-20T11:24:57.657 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 88/152 2024-04-20T11:24:57.756 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 89/152 2024-04-20T11:24:58.201 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 80/152 2024-04-20T11:24:58.248 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 81/152 2024-04-20T11:24:58.282 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 82/152 2024-04-20T11:24:58.401 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 83/152 2024-04-20T11:24:58.468 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 84/152 2024-04-20T11:24:58.564 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 85/152 2024-04-20T11:24:58.608 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 86/152 2024-04-20T11:24:58.638 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 90/152 2024-04-20T11:24:58.783 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 91/152 2024-04-20T11:24:58.835 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : openblas-0.3.21-2.el9.x86_64 87/152 2024-04-20T11:24:58.863 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 88/152 2024-04-20T11:24:58.967 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 89/152 2024-04-20T11:24:59.822 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 90/152 2024-04-20T11:25:00.103 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 91/152 2024-04-20T11:25:00.434 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 92/152 2024-04-20T11:25:00.532 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 93/152 2024-04-20T11:25:00.571 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 94/152 2024-04-20T11:25:00.616 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 95/152 2024-04-20T11:25:00.993 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 96/152 2024-04-20T11:25:01.037 INFO:teuthology.orchestra.run.smithi152.stdout: Upgrading : librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 97/152 2024-04-20T11:25:01.159 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 97/152 2024-04-20T11:25:01.194 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-rbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 98/152 2024-04-20T11:25:01.251 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 99/152 2024-04-20T11:25:01.800 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 92/152 2024-04-20T11:25:01.802 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 100/152 2024-04-20T11:25:01.840 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 101/152 2024-04-20T11:25:01.902 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 93/152 2024-04-20T11:25:01.940 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 101/152 2024-04-20T11:25:01.971 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 94/152 2024-04-20T11:25:01.975 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-rgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 102/152 2024-04-20T11:25:02.025 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 95/152 2024-04-20T11:25:02.410 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 96/152 2024-04-20T11:25:02.457 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 97/152 2024-04-20T11:25:02.578 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 97/152 2024-04-20T11:25:02.603 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-rbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 98/152 2024-04-20T11:25:02.664 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 99/152 2024-04-20T11:25:03.203 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 100/152 2024-04-20T11:25:03.250 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 101/152 2024-04-20T11:25:03.343 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 101/152 2024-04-20T11:25:03.369 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-rgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 102/152 2024-04-20T11:25:04.715 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 103/152 2024-04-20T11:25:04.756 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 103/152 2024-04-20T11:25:05.394 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 103/152 2024-04-20T11:25:05.443 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 104/152 2024-04-20T11:25:05.629 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 104/152 2024-04-20T11:25:05.629 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-20T11:25:05.629 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-20T11:25:05.630 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:25:05.668 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 105/152 2024-04-20T11:25:06.088 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 103/152 2024-04-20T11:25:06.132 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 103/152 2024-04-20T11:25:06.769 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 103/152 2024-04-20T11:25:06.809 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 104/152 2024-04-20T11:25:06.984 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 104/152 2024-04-20T11:25:06.984 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-20T11:25:06.984 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-20T11:25:06.984 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-20T11:25:07.027 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 105/152 2024-04-20T11:25:18.524 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 105/152 2024-04-20T11:25:18.526 INFO:teuthology.orchestra.run.smithi152.stdout:skipping the directory /sys 2024-04-20T11:25:18.526 INFO:teuthology.orchestra.run.smithi152.stdout:skipping the directory /proc 2024-04-20T11:25:18.526 INFO:teuthology.orchestra.run.smithi152.stdout:skipping the directory /mnt 2024-04-20T11:25:18.526 INFO:teuthology.orchestra.run.smithi152.stdout:skipping the directory /var/tmp 2024-04-20T11:25:18.526 INFO:teuthology.orchestra.run.smithi152.stdout:skipping the directory /home 2024-04-20T11:25:18.526 INFO:teuthology.orchestra.run.smithi152.stdout:skipping the directory /root 2024-04-20T11:25:18.526 INFO:teuthology.orchestra.run.smithi152.stdout:skipping the directory /tmp 2024-04-20T11:25:18.526 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:25:18.764 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 106/152 2024-04-20T11:25:18.812 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 106/152 2024-04-20T11:25:18.812 INFO:teuthology.orchestra.run.smithi152.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T11:25:18.812 INFO:teuthology.orchestra.run.smithi152.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-20T11:25:18.812 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:25:20.154 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 105/152 2024-04-20T11:25:20.154 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /sys 2024-04-20T11:25:20.154 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /proc 2024-04-20T11:25:20.154 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /mnt 2024-04-20T11:25:20.154 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /var/tmp 2024-04-20T11:25:20.154 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /home 2024-04-20T11:25:20.154 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /root 2024-04-20T11:25:20.154 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /tmp 2024-04-20T11:25:20.155 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-20T11:25:20.392 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 106/152 2024-04-20T11:25:20.437 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 106/152 2024-04-20T11:25:20.437 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T11:25:20.437 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-20T11:25:20.437 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-20T11:25:20.489 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 107/152 2024-04-20T11:25:20.583 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 107/152 2024-04-20T11:25:20.583 INFO:teuthology.orchestra.run.smithi152.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T11:25:20.583 INFO:teuthology.orchestra.run.smithi152.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-20T11:25:20.583 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-20T11:25:20.583 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-20T11:25:20.583 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:25:20.839 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 108/152 2024-04-20T11:25:20.918 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 108/152 2024-04-20T11:25:20.918 INFO:teuthology.orchestra.run.smithi152.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T11:25:20.918 INFO:teuthology.orchestra.run.smithi152.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-20T11:25:20.918 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-20T11:25:20.918 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-20T11:25:20.918 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:25:21.407 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 109/152 2024-04-20T11:25:21.481 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 109/152 2024-04-20T11:25:21.481 INFO:teuthology.orchestra.run.smithi152.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T11:25:21.481 INFO:teuthology.orchestra.run.smithi152.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-20T11:25:21.481 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-20T11:25:21.481 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-20T11:25:21.482 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:25:21.616 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : automake-1.16.2-8.el9.noarch 110/152 2024-04-20T11:25:21.677 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-ply-3.11-14.el9.noarch 111/152 2024-04-20T11:25:21.754 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 112/152 2024-04-20T11:25:21.987 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 113/152 2024-04-20T11:25:22.054 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 114/152 2024-04-20T11:25:22.056 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 107/152 2024-04-20T11:25:22.134 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 107/152 2024-04-20T11:25:22.135 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T11:25:22.135 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-20T11:25:22.135 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-20T11:25:22.135 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-20T11:25:22.135 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-20T11:25:22.140 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 115/152 2024-04-20T11:25:22.250 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 116/152 2024-04-20T11:25:22.382 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 108/152 2024-04-20T11:25:22.419 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 117/152 2024-04-20T11:25:22.452 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 108/152 2024-04-20T11:25:22.452 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T11:25:22.452 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-20T11:25:22.453 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-20T11:25:22.453 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-20T11:25:22.453 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-20T11:25:22.496 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 118/152 2024-04-20T11:25:22.541 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 119/152 2024-04-20T11:25:22.577 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : fuse-2.9.9-15.el9.x86_64 120/152 2024-04-20T11:25:22.941 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 109/152 2024-04-20T11:25:23.030 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 109/152 2024-04-20T11:25:23.030 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T11:25:23.030 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-20T11:25:23.030 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-20T11:25:23.030 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-20T11:25:23.030 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-20T11:25:23.174 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : automake-1.16.2-8.el9.noarch 110/152 2024-04-20T11:25:23.205 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 121/152 2024-04-20T11:25:23.231 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 121/152 2024-04-20T11:25:23.240 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-ply-3.11-14.el9.noarch 111/152 2024-04-20T11:25:23.283 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3133.ga8c84f59.e 122/152 2024-04-20T11:25:23.323 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 112/152 2024-04-20T11:25:23.382 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3133.ga8c84f59. 123/152 2024-04-20T11:25:23.417 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noa 124/152 2024-04-20T11:25:23.558 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 113/152 2024-04-20T11:25:23.621 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 114/152 2024-04-20T11:25:23.714 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 115/152 2024-04-20T11:25:23.822 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 116/152 2024-04-20T11:25:23.986 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 117/152 2024-04-20T11:25:24.044 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 118/152 2024-04-20T11:25:24.086 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 119/152 2024-04-20T11:25:24.126 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : fuse-2.9.9-15.el9.x86_64 120/152 2024-04-20T11:25:24.701 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noa 124/152 2024-04-20T11:25:24.744 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.n 125/152 2024-04-20T11:25:24.744 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 121/152 2024-04-20T11:25:24.779 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 121/152 2024-04-20T11:25:24.858 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3133.ga8c84f59.e 122/152 2024-04-20T11:25:24.971 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3133.ga8c84f59. 123/152 2024-04-20T11:25:25.014 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noa 124/152 2024-04-20T11:25:25.847 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.n 125/152 2024-04-20T11:25:25.877 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c 126/152 2024-04-20T11:25:26.022 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c 126/152 2024-04-20T11:25:26.213 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3133.ga8c84f59.el 127/152 2024-04-20T11:25:26.246 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 128/152 2024-04-20T11:25:26.318 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 128/152 2024-04-20T11:25:26.318 INFO:teuthology.orchestra.run.smithi152.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T11:25:26.318 INFO:teuthology.orchestra.run.smithi152.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-20T11:25:26.318 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-20T11:25:26.318 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-20T11:25:26.318 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:25:26.331 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noa 124/152 2024-04-20T11:25:26.375 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.n 125/152 2024-04-20T11:25:26.380 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 129/152 2024-04-20T11:25:26.420 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 129/152 2024-04-20T11:25:26.563 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-2:19.0.0-3133.ga8c84f59.el9.x86_64 130/152 2024-04-20T11:25:26.645 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 131/152 2024-04-20T11:25:27.477 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.n 125/152 2024-04-20T11:25:27.516 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c 126/152 2024-04-20T11:25:27.661 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c 126/152 2024-04-20T11:25:27.711 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libtool-2.4.6-45.el9.x86_64 132/152 2024-04-20T11:25:27.741 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 133/152 2024-04-20T11:25:27.806 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 133/152 2024-04-20T11:25:27.806 INFO:teuthology.orchestra.run.smithi152.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T11:25:27.806 INFO:teuthology.orchestra.run.smithi152.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-20T11:25:27.806 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-20T11:25:27.806 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-20T11:25:27.806 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:25:27.846 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3133.ga8c84f59.el 127/152 2024-04-20T11:25:27.859 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3133.ga8c84 134/152 2024-04-20T11:25:27.876 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 128/152 2024-04-20T11:25:27.926 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3133.ga8c84 134/152 2024-04-20T11:25:27.927 INFO:teuthology.orchestra.run.smithi152.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T11:25:27.927 INFO:teuthology.orchestra.run.smithi152.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-20T11:25:27.927 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:25:27.974 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 128/152 2024-04-20T11:25:27.975 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T11:25:27.975 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-20T11:25:27.975 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-20T11:25:27.975 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-20T11:25:27.975 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-20T11:25:27.984 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 135/152 2024-04-20T11:25:28.035 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 129/152 2024-04-20T11:25:28.075 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 129/152 2024-04-20T11:25:28.077 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 135/152 2024-04-20T11:25:28.078 INFO:teuthology.orchestra.run.smithi152.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T11:25:28.078 INFO:teuthology.orchestra.run.smithi152.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-20T11:25:28.078 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-20T11:25:28.078 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-20T11:25:28.078 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:25:28.220 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-2:19.0.0-3133.ga8c84f59.el9.x86_64 130/152 2024-04-20T11:25:28.298 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 131/152 2024-04-20T11:25:28.468 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 136/152 2024-04-20T11:25:28.549 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 136/152 2024-04-20T11:25:28.550 INFO:teuthology.orchestra.run.smithi152.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T11:25:28.550 INFO:teuthology.orchestra.run.smithi152.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-20T11:25:28.550 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-20T11:25:28.550 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-20T11:25:28.550 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:25:29.372 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libtool-2.4.6-45.el9.x86_64 132/152 2024-04-20T11:25:29.413 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 133/152 2024-04-20T11:25:29.480 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 133/152 2024-04-20T11:25:29.480 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T11:25:29.480 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-20T11:25:29.480 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-20T11:25:29.480 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-20T11:25:29.481 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-20T11:25:29.538 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3133.ga8c84 134/152 2024-04-20T11:25:29.619 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3133.ga8c84 134/152 2024-04-20T11:25:29.619 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T11:25:29.619 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-20T11:25:29.619 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-20T11:25:29.697 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 135/152 2024-04-20T11:25:29.774 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 135/152 2024-04-20T11:25:29.774 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T11:25:29.774 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-20T11:25:29.774 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-20T11:25:29.774 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-20T11:25:29.774 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-20T11:25:30.157 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 136/152 2024-04-20T11:25:30.226 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 136/152 2024-04-20T11:25:30.226 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T11:25:30.226 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-20T11:25:30.226 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-20T11:25:30.226 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-20T11:25:30.226 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-20T11:25:33.633 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-test-2:19.0.0-3133.ga8c84f59.el9.x86_64 137/152 2024-04-20T11:25:33.678 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : rbd-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 138/152 2024-04-20T11:25:33.765 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : rbd-nbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 139/152 2024-04-20T11:25:33.813 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 140/152 2024-04-20T11:25:33.861 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 141/152 2024-04-20T11:25:34.193 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : cephfs-shell-2:19.0.0-3133.ga8c84f59.el9.x86_64 142/152 2024-04-20T11:25:34.256 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 143/152 2024-04-20T11:25:34.283 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libcephfs-devel-2:19.0.0-3133.ga8c84f59.el9.x86_ 144/152 2024-04-20T11:25:35.223 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-20T11:25:35.274 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-20T11:25:35.310 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-test-2:19.0.0-3133.ga8c84f59.el9.x86_64 137/152 2024-04-20T11:25:35.330 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-20T11:25:35.370 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : rbd-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 138/152 2024-04-20T11:25:35.474 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : rbd-nbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 139/152 2024-04-20T11:25:35.528 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 140/152 2024-04-20T11:25:35.576 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 141/152 2024-04-20T11:25:35.891 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : cephfs-shell-2:19.0.0-3133.ga8c84f59.el9.x86_64 142/152 2024-04-20T11:25:35.951 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 143/152 2024-04-20T11:25:35.979 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libcephfs-devel-2:19.0.0-3133.ga8c84f59.el9.x86_ 144/152 2024-04-20T11:25:36.631 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : cephfs-top-2:19.0.0-3133.ga8c84f59.el9.noarch 146/152 2024-04-20T11:25:36.739 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 147/152 2024-04-20T11:25:36.823 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 148/152 2024-04-20T11:25:36.907 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-20T11:25:36.952 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-20T11:25:37.007 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-20T11:25:37.054 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : flex-2.6.4-9.el9.x86_64 149/152 2024-04-20T11:25:37.097 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : bison-3.7.4-5.el9.x86_64 150/152 2024-04-20T11:25:37.098 INFO:teuthology.orchestra.run.smithi152.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-04-20T11:25:37.222 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-04-20T11:25:37.223 INFO:teuthology.orchestra.run.smithi152.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 152/152 2024-04-20T11:25:37.758 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 152/152 2024-04-20T11:25:37.758 INFO:teuthology.orchestra.run.smithi152.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-20T11:25:37.758 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:25:38.274 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : cephfs-top-2:19.0.0-3133.ga8c84f59.el9.noarch 146/152 2024-04-20T11:25:38.423 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 147/152 2024-04-20T11:25:38.526 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 148/152 2024-04-20T11:25:38.705 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : flex-2.6.4-9.el9.x86_64 149/152 2024-04-20T11:25:38.747 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : bison-3.7.4-5.el9.x86_64 150/152 2024-04-20T11:25:38.747 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-04-20T11:25:38.897 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-04-20T11:25:38.897 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 152/152 2024-04-20T11:25:39.383 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 152/152 2024-04-20T11:25:39.383 INFO:teuthology.orchestra.run.smithi175.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-20T11:25:39.383 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-20T11:25:43.658 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-2:19.0.0-3133.ga8c84f59.el9.x86_64 1/152 2024-04-20T11:25:43.658 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 2/152 2024-04-20T11:25:43.658 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 3/152 2024-04-20T11:25:43.658 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 4/152 2024-04-20T11:25:43.658 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3133.ga8c84 5/152 2024-04-20T11:25:43.659 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 6/152 2024-04-20T11:25:43.659 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 7/152 2024-04-20T11:25:43.659 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 8/152 2024-04-20T11:25:43.659 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 9/152 2024-04-20T11:25:43.659 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 10/152 2024-04-20T11:25:43.659 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 11/152 2024-04-20T11:25:43.659 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-test-2:19.0.0-3133.ga8c84f59.el9.x86_64 12/152 2024-04-20T11:25:43.659 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 13/152 2024-04-20T11:25:43.659 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : cephfs-shell-2:19.0.0-3133.ga8c84f59.el9.x86_64 14/152 2024-04-20T11:25:43.659 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libcephfs-devel-2:19.0.0-3133.ga8c84f59.el9.x86_ 15/152 2024-04-20T11:25:43.659 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 16/152 2024-04-20T11:25:43.659 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 17/152 2024-04-20T11:25:43.659 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : librados-devel-2:19.0.0-3133.ga8c84f59.el9.x86_6 18/152 2024-04-20T11:25:43.659 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86 19/152 2024-04-20T11:25:43.660 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 20/152 2024-04-20T11:25:43.660 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3133.ga8c84f59.el 21/152 2024-04-20T11:25:43.660 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-ceph-common-2:19.0.0-3133.ga8c84f59.el9. 22/152 2024-04-20T11:25:43.660 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-cephfs-2:19.0.0-3133.ga8c84f59.el9.x86_6 23/152 2024-04-20T11:25:43.660 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-rados-2:19.0.0-3133.ga8c84f59.el9.x86_64 24/152 2024-04-20T11:25:43.660 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-rbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 25/152 2024-04-20T11:25:43.660 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-rgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 26/152 2024-04-20T11:25:43.660 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : rbd-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 27/152 2024-04-20T11:25:43.660 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 28/152 2024-04-20T11:25:43.660 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : rbd-nbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 29/152 2024-04-20T11:25:43.660 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3133.ga8c84f59. 30/152 2024-04-20T11:25:43.673 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noa 31/152 2024-04-20T11:25:43.673 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.n 32/152 2024-04-20T11:25:43.673 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c 33/152 2024-04-20T11:25:43.673 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3133.ga8c84f59.el 34/152 2024-04-20T11:25:43.673 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 35/152 2024-04-20T11:25:43.673 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3133.ga8c84f59.e 36/152 2024-04-20T11:25:43.674 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 37/152 2024-04-20T11:25:43.674 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 38/152 2024-04-20T11:25:43.674 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : cephfs-top-2:19.0.0-3133.ga8c84f59.el9.noarch 39/152 2024-04-20T11:25:43.674 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/152 2024-04-20T11:25:43.674 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/152 2024-04-20T11:25:43.674 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/152 2024-04-20T11:25:43.674 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/152 2024-04-20T11:25:43.674 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/152 2024-04-20T11:25:43.674 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/152 2024-04-20T11:25:43.674 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/152 2024-04-20T11:25:43.674 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/152 2024-04-20T11:25:43.675 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/152 2024-04-20T11:25:43.675 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/152 2024-04-20T11:25:43.675 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/152 2024-04-20T11:25:43.675 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/152 2024-04-20T11:25:43.675 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/152 2024-04-20T11:25:43.675 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/152 2024-04-20T11:25:43.675 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/152 2024-04-20T11:25:43.675 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/152 2024-04-20T11:25:43.675 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/152 2024-04-20T11:25:43.675 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/152 2024-04-20T11:25:43.675 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/152 2024-04-20T11:25:43.675 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/152 2024-04-20T11:25:43.676 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/152 2024-04-20T11:25:43.676 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/152 2024-04-20T11:25:43.676 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/152 2024-04-20T11:25:43.676 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/152 2024-04-20T11:25:43.676 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/152 2024-04-20T11:25:43.676 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/152 2024-04-20T11:25:43.676 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/152 2024-04-20T11:25:43.676 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/152 2024-04-20T11:25:43.676 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 68/152 2024-04-20T11:25:43.676 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/152 2024-04-20T11:25:43.676 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/152 2024-04-20T11:25:43.676 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/152 2024-04-20T11:25:43.676 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/152 2024-04-20T11:25:43.676 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/152 2024-04-20T11:25:43.677 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/152 2024-04-20T11:25:43.677 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/152 2024-04-20T11:25:43.677 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/152 2024-04-20T11:25:43.677 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/152 2024-04-20T11:25:43.677 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/152 2024-04-20T11:25:43.677 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/152 2024-04-20T11:25:43.677 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/152 2024-04-20T11:25:43.677 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/152 2024-04-20T11:25:43.677 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/152 2024-04-20T11:25:43.677 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/152 2024-04-20T11:25:43.677 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/152 2024-04-20T11:25:43.677 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/152 2024-04-20T11:25:43.677 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/152 2024-04-20T11:25:43.677 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/152 2024-04-20T11:25:43.678 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/152 2024-04-20T11:25:43.678 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/152 2024-04-20T11:25:43.678 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/152 2024-04-20T11:25:43.678 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/152 2024-04-20T11:25:43.678 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/152 2024-04-20T11:25:43.678 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/152 2024-04-20T11:25:43.678 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/152 2024-04-20T11:25:43.678 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/152 2024-04-20T11:25:43.678 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/152 2024-04-20T11:25:43.678 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/152 2024-04-20T11:25:43.678 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/152 2024-04-20T11:25:43.678 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/152 2024-04-20T11:25:43.678 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/152 2024-04-20T11:25:43.679 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/152 2024-04-20T11:25:43.679 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/152 2024-04-20T11:25:43.679 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/152 2024-04-20T11:25:43.679 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/152 2024-04-20T11:25:43.679 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/152 2024-04-20T11:25:43.679 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/152 2024-04-20T11:25:43.679 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/152 2024-04-20T11:25:43.679 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/152 2024-04-20T11:25:43.679 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/152 2024-04-20T11:25:43.679 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/152 2024-04-20T11:25:43.679 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/152 2024-04-20T11:25:43.679 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/152 2024-04-20T11:25:43.679 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/152 2024-04-20T11:25:43.679 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/152 2024-04-20T11:25:43.679 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/152 2024-04-20T11:25:43.680 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/152 2024-04-20T11:25:43.680 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/152 2024-04-20T11:25:43.680 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/152 2024-04-20T11:25:43.680 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/152 2024-04-20T11:25:43.680 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/152 2024-04-20T11:25:43.680 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/152 2024-04-20T11:25:43.680 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 122/152 2024-04-20T11:25:43.680 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/152 2024-04-20T11:25:43.680 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/152 2024-04-20T11:25:43.681 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/152 2024-04-20T11:25:43.681 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/152 2024-04-20T11:25:43.681 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/152 2024-04-20T11:25:43.681 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/152 2024-04-20T11:25:43.681 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/152 2024-04-20T11:25:43.681 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/152 2024-04-20T11:25:43.682 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/152 2024-04-20T11:25:43.682 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/152 2024-04-20T11:25:43.682 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/152 2024-04-20T11:25:43.682 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/152 2024-04-20T11:25:43.682 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/152 2024-04-20T11:25:43.682 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/152 2024-04-20T11:25:43.682 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/152 2024-04-20T11:25:43.683 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/152 2024-04-20T11:25:43.683 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/152 2024-04-20T11:25:43.683 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/152 2024-04-20T11:25:43.683 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/152 2024-04-20T11:25:43.683 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/152 2024-04-20T11:25:43.683 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/152 2024-04-20T11:25:43.683 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/152 2024-04-20T11:25:43.683 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/152 2024-04-20T11:25:43.683 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/152 2024-04-20T11:25:43.683 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/152 2024-04-20T11:25:43.683 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/152 2024-04-20T11:25:43.683 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 149/152 2024-04-20T11:25:43.683 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/152 2024-04-20T11:25:43.683 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 151/152 2024-04-20T11:25:44.892 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-2:19.0.0-3133.ga8c84f59.el9.x86_64 1/152 2024-04-20T11:25:44.893 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 2/152 2024-04-20T11:25:44.893 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 3/152 2024-04-20T11:25:44.893 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 4/152 2024-04-20T11:25:44.893 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3133.ga8c84 5/152 2024-04-20T11:25:44.893 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 6/152 2024-04-20T11:25:44.893 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 7/152 2024-04-20T11:25:44.893 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 8/152 2024-04-20T11:25:44.893 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 9/152 2024-04-20T11:25:44.893 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 10/152 2024-04-20T11:25:44.893 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 11/152 2024-04-20T11:25:44.893 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-test-2:19.0.0-3133.ga8c84f59.el9.x86_64 12/152 2024-04-20T11:25:44.893 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 13/152 2024-04-20T11:25:44.893 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : cephfs-shell-2:19.0.0-3133.ga8c84f59.el9.x86_64 14/152 2024-04-20T11:25:44.894 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libcephfs-devel-2:19.0.0-3133.ga8c84f59.el9.x86_ 15/152 2024-04-20T11:25:44.894 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 16/152 2024-04-20T11:25:44.894 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 17/152 2024-04-20T11:25:44.894 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librados-devel-2:19.0.0-3133.ga8c84f59.el9.x86_6 18/152 2024-04-20T11:25:44.894 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86 19/152 2024-04-20T11:25:44.894 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 20/152 2024-04-20T11:25:44.894 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3133.ga8c84f59.el 21/152 2024-04-20T11:25:44.894 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-ceph-common-2:19.0.0-3133.ga8c84f59.el9. 22/152 2024-04-20T11:25:44.894 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cephfs-2:19.0.0-3133.ga8c84f59.el9.x86_6 23/152 2024-04-20T11:25:44.894 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-rados-2:19.0.0-3133.ga8c84f59.el9.x86_64 24/152 2024-04-20T11:25:44.894 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-rbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 25/152 2024-04-20T11:25:44.894 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-rgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 26/152 2024-04-20T11:25:44.894 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : rbd-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 27/152 2024-04-20T11:25:44.894 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 28/152 2024-04-20T11:25:44.906 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : rbd-nbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 29/152 2024-04-20T11:25:44.906 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3133.ga8c84f59. 30/152 2024-04-20T11:25:44.906 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noa 31/152 2024-04-20T11:25:44.906 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.n 32/152 2024-04-20T11:25:44.906 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c 33/152 2024-04-20T11:25:44.906 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3133.ga8c84f59.el 34/152 2024-04-20T11:25:44.906 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 35/152 2024-04-20T11:25:44.906 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3133.ga8c84f59.e 36/152 2024-04-20T11:25:44.906 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 37/152 2024-04-20T11:25:44.907 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 38/152 2024-04-20T11:25:44.907 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : cephfs-top-2:19.0.0-3133.ga8c84f59.el9.noarch 39/152 2024-04-20T11:25:44.907 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/152 2024-04-20T11:25:44.907 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/152 2024-04-20T11:25:44.907 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/152 2024-04-20T11:25:44.907 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/152 2024-04-20T11:25:44.907 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/152 2024-04-20T11:25:44.907 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/152 2024-04-20T11:25:44.907 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/152 2024-04-20T11:25:44.907 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/152 2024-04-20T11:25:44.907 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/152 2024-04-20T11:25:44.907 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/152 2024-04-20T11:25:44.907 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/152 2024-04-20T11:25:44.907 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/152 2024-04-20T11:25:44.908 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/152 2024-04-20T11:25:44.908 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/152 2024-04-20T11:25:44.908 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/152 2024-04-20T11:25:44.908 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/152 2024-04-20T11:25:44.908 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/152 2024-04-20T11:25:44.908 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/152 2024-04-20T11:25:44.908 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/152 2024-04-20T11:25:44.908 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/152 2024-04-20T11:25:44.908 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/152 2024-04-20T11:25:44.908 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/152 2024-04-20T11:25:44.908 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/152 2024-04-20T11:25:44.908 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/152 2024-04-20T11:25:44.908 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/152 2024-04-20T11:25:44.908 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/152 2024-04-20T11:25:44.909 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/152 2024-04-20T11:25:44.909 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/152 2024-04-20T11:25:44.909 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 68/152 2024-04-20T11:25:44.909 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/152 2024-04-20T11:25:44.909 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/152 2024-04-20T11:25:44.909 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/152 2024-04-20T11:25:44.909 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/152 2024-04-20T11:25:44.909 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/152 2024-04-20T11:25:44.909 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/152 2024-04-20T11:25:44.909 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/152 2024-04-20T11:25:44.909 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/152 2024-04-20T11:25:44.909 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/152 2024-04-20T11:25:44.910 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/152 2024-04-20T11:25:44.910 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/152 2024-04-20T11:25:44.910 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/152 2024-04-20T11:25:44.910 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/152 2024-04-20T11:25:44.910 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/152 2024-04-20T11:25:44.910 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/152 2024-04-20T11:25:44.910 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/152 2024-04-20T11:25:44.910 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/152 2024-04-20T11:25:44.910 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/152 2024-04-20T11:25:44.910 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/152 2024-04-20T11:25:44.910 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/152 2024-04-20T11:25:44.910 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/152 2024-04-20T11:25:44.910 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/152 2024-04-20T11:25:44.911 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/152 2024-04-20T11:25:44.911 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/152 2024-04-20T11:25:44.911 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/152 2024-04-20T11:25:44.911 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/152 2024-04-20T11:25:44.911 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/152 2024-04-20T11:25:44.911 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/152 2024-04-20T11:25:44.911 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/152 2024-04-20T11:25:44.911 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/152 2024-04-20T11:25:44.911 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/152 2024-04-20T11:25:44.911 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/152 2024-04-20T11:25:44.911 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/152 2024-04-20T11:25:44.911 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/152 2024-04-20T11:25:44.911 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/152 2024-04-20T11:25:44.911 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/152 2024-04-20T11:25:44.912 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/152 2024-04-20T11:25:44.912 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/152 2024-04-20T11:25:44.912 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/152 2024-04-20T11:25:44.912 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/152 2024-04-20T11:25:44.912 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/152 2024-04-20T11:25:44.912 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/152 2024-04-20T11:25:44.912 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/152 2024-04-20T11:25:44.912 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/152 2024-04-20T11:25:44.912 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/152 2024-04-20T11:25:44.912 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/152 2024-04-20T11:25:44.912 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/152 2024-04-20T11:25:44.912 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/152 2024-04-20T11:25:44.912 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/152 2024-04-20T11:25:44.913 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/152 2024-04-20T11:25:44.913 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/152 2024-04-20T11:25:44.913 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/152 2024-04-20T11:25:44.913 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/152 2024-04-20T11:25:44.913 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 122/152 2024-04-20T11:25:44.913 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/152 2024-04-20T11:25:44.913 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/152 2024-04-20T11:25:44.913 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/152 2024-04-20T11:25:44.913 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/152 2024-04-20T11:25:44.913 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/152 2024-04-20T11:25:44.913 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/152 2024-04-20T11:25:44.913 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/152 2024-04-20T11:25:44.913 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/152 2024-04-20T11:25:44.914 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/152 2024-04-20T11:25:44.914 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/152 2024-04-20T11:25:44.914 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/152 2024-04-20T11:25:44.914 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/152 2024-04-20T11:25:44.914 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/152 2024-04-20T11:25:44.914 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/152 2024-04-20T11:25:44.914 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/152 2024-04-20T11:25:44.914 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/152 2024-04-20T11:25:44.914 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/152 2024-04-20T11:25:44.914 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/152 2024-04-20T11:25:44.914 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/152 2024-04-20T11:25:44.914 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/152 2024-04-20T11:25:44.915 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/152 2024-04-20T11:25:44.915 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/152 2024-04-20T11:25:44.915 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/152 2024-04-20T11:25:44.915 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/152 2024-04-20T11:25:44.915 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/152 2024-04-20T11:25:44.915 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/152 2024-04-20T11:25:44.915 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 149/152 2024-04-20T11:25:44.915 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/152 2024-04-20T11:25:44.915 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 151/152 2024-04-20T11:25:46.131 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/152 2024-04-20T11:25:46.132 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:25:46.132 INFO:teuthology.orchestra.run.smithi152.stdout:Upgraded: 2024-04-20T11:25:46.132 INFO:teuthology.orchestra.run.smithi152.stdout: librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:25:46.132 INFO:teuthology.orchestra.run.smithi152.stdout: librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:25:46.132 INFO:teuthology.orchestra.run.smithi152.stdout:Installed: 2024-04-20T11:25:46.132 INFO:teuthology.orchestra.run.smithi152.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-20T11:25:46.132 INFO:teuthology.orchestra.run.smithi152.stdout: automake-1.16.2-8.el9.noarch 2024-04-20T11:25:46.132 INFO:teuthology.orchestra.run.smithi152.stdout: bison-3.7.4-5.el9.x86_64 2024-04-20T11:25:46.132 INFO:teuthology.orchestra.run.smithi152.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-20T11:25:46.132 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:25:46.132 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:25:46.132 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:25:46.132 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:25:46.133 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-grafana-dashboards-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T11:25:46.133 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-immutable-object-cache-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:25:46.133 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:25:46.133 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:25:46.133 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T11:25:46.133 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T11:25:46.133 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T11:25:46.133 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-modules-core-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T11:25:46.133 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T11:25:46.133 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:25:46.133 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:25:46.133 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-prometheus-alerts-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T11:25:46.133 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:25:46.133 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:25:46.134 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-test-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:25:46.134 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T11:25:46.134 INFO:teuthology.orchestra.run.smithi152.stdout: cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T11:25:46.134 INFO:teuthology.orchestra.run.smithi152.stdout: cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:25:46.134 INFO:teuthology.orchestra.run.smithi152.stdout: cephfs-shell-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:25:46.134 INFO:teuthology.orchestra.run.smithi152.stdout: cephfs-top-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T11:25:46.134 INFO:teuthology.orchestra.run.smithi152.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-20T11:25:46.134 INFO:teuthology.orchestra.run.smithi152.stdout: flex-2.6.4-9.el9.x86_64 2024-04-20T11:25:46.134 INFO:teuthology.orchestra.run.smithi152.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-20T11:25:46.134 INFO:teuthology.orchestra.run.smithi152.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-20T11:25:46.134 INFO:teuthology.orchestra.run.smithi152.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-20T11:25:46.134 INFO:teuthology.orchestra.run.smithi152.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-20T11:25:46.135 INFO:teuthology.orchestra.run.smithi152.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-20T11:25:46.135 INFO:teuthology.orchestra.run.smithi152.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-20T11:25:46.135 INFO:teuthology.orchestra.run.smithi152.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-20T11:25:46.135 INFO:teuthology.orchestra.run.smithi152.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-20T11:25:46.135 INFO:teuthology.orchestra.run.smithi152.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-20T11:25:46.135 INFO:teuthology.orchestra.run.smithi152.stdout: libcephfs-devel-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:25:46.135 INFO:teuthology.orchestra.run.smithi152.stdout: libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:25:46.135 INFO:teuthology.orchestra.run.smithi152.stdout: libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:25:46.135 INFO:teuthology.orchestra.run.smithi152.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-20T11:25:46.135 INFO:teuthology.orchestra.run.smithi152.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-20T11:25:46.135 INFO:teuthology.orchestra.run.smithi152.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-20T11:25:46.135 INFO:teuthology.orchestra.run.smithi152.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-20T11:25:46.135 INFO:teuthology.orchestra.run.smithi152.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-20T11:25:46.136 INFO:teuthology.orchestra.run.smithi152.stdout: librados-devel-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:25:46.136 INFO:teuthology.orchestra.run.smithi152.stdout: libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:25:46.136 INFO:teuthology.orchestra.run.smithi152.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-20T11:25:46.136 INFO:teuthology.orchestra.run.smithi152.stdout: librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:25:46.136 INFO:teuthology.orchestra.run.smithi152.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-20T11:25:46.136 INFO:teuthology.orchestra.run.smithi152.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-20T11:25:46.136 INFO:teuthology.orchestra.run.smithi152.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-20T11:25:46.136 INFO:teuthology.orchestra.run.smithi152.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-20T11:25:46.136 INFO:teuthology.orchestra.run.smithi152.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-20T11:25:46.136 INFO:teuthology.orchestra.run.smithi152.stdout: lua-5.4.4-4.el9.x86_64 2024-04-20T11:25:46.136 INFO:teuthology.orchestra.run.smithi152.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-20T11:25:46.136 INFO:teuthology.orchestra.run.smithi152.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-20T11:25:46.136 INFO:teuthology.orchestra.run.smithi152.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-20T11:25:46.136 INFO:teuthology.orchestra.run.smithi152.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-20T11:25:46.136 INFO:teuthology.orchestra.run.smithi152.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-20T11:25:46.137 INFO:teuthology.orchestra.run.smithi152.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-20T11:25:46.137 INFO:teuthology.orchestra.run.smithi152.stdout: postgresql-13.14-1.el9.x86_64 2024-04-20T11:25:46.137 INFO:teuthology.orchestra.run.smithi152.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-20T11:25:46.137 INFO:teuthology.orchestra.run.smithi152.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-20T11:25:46.137 INFO:teuthology.orchestra.run.smithi152.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-20T11:25:46.137 INFO:teuthology.orchestra.run.smithi152.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-20T11:25:46.137 INFO:teuthology.orchestra.run.smithi152.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-20T11:25:46.137 INFO:teuthology.orchestra.run.smithi152.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-20T11:25:46.137 INFO:teuthology.orchestra.run.smithi152.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-20T11:25:46.137 INFO:teuthology.orchestra.run.smithi152.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-20T11:25:46.137 INFO:teuthology.orchestra.run.smithi152.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-20T11:25:46.137 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-20T11:25:46.137 INFO:teuthology.orchestra.run.smithi152.stdout: python3-ceph-argparse-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:25:46.137 INFO:teuthology.orchestra.run.smithi152.stdout: python3-ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:25:46.138 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cephfs-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:25:46.138 INFO:teuthology.orchestra.run.smithi152.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-20T11:25:46.138 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-20T11:25:46.138 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-20T11:25:46.138 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-20T11:25:46.138 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-20T11:25:46.138 INFO:teuthology.orchestra.run.smithi152.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-20T11:25:46.138 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-20T11:25:46.138 INFO:teuthology.orchestra.run.smithi152.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-20T11:25:46.138 INFO:teuthology.orchestra.run.smithi152.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-20T11:25:46.138 INFO:teuthology.orchestra.run.smithi152.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-20T11:25:46.138 INFO:teuthology.orchestra.run.smithi152.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-20T11:25:46.138 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-20T11:25:46.139 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-20T11:25:46.139 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-20T11:25:46.139 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-20T11:25:46.139 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-20T11:25:46.139 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-20T11:25:46.139 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-20T11:25:46.139 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-20T11:25:46.139 INFO:teuthology.orchestra.run.smithi152.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-20T11:25:46.139 INFO:teuthology.orchestra.run.smithi152.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-20T11:25:46.139 INFO:teuthology.orchestra.run.smithi152.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-20T11:25:46.139 INFO:teuthology.orchestra.run.smithi152.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-20T11:25:46.139 INFO:teuthology.orchestra.run.smithi152.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-20T11:25:46.139 INFO:teuthology.orchestra.run.smithi152.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-20T11:25:46.139 INFO:teuthology.orchestra.run.smithi152.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-20T11:25:46.140 INFO:teuthology.orchestra.run.smithi152.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-20T11:25:46.140 INFO:teuthology.orchestra.run.smithi152.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-20T11:25:46.140 INFO:teuthology.orchestra.run.smithi152.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-20T11:25:46.140 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-20T11:25:46.140 INFO:teuthology.orchestra.run.smithi152.stdout: python3-ply-3.11-14.el9.noarch 2024-04-20T11:25:46.140 INFO:teuthology.orchestra.run.smithi152.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-20T11:25:46.140 INFO:teuthology.orchestra.run.smithi152.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-20T11:25:46.140 INFO:teuthology.orchestra.run.smithi152.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-20T11:25:46.140 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-20T11:25:46.140 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-20T11:25:46.140 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-20T11:25:46.140 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-20T11:25:46.140 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-20T11:25:46.140 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-20T11:25:46.140 INFO:teuthology.orchestra.run.smithi152.stdout: python3-rados-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:25:46.141 INFO:teuthology.orchestra.run.smithi152.stdout: python3-rbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:25:46.141 INFO:teuthology.orchestra.run.smithi152.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-20T11:25:46.141 INFO:teuthology.orchestra.run.smithi152.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-20T11:25:46.141 INFO:teuthology.orchestra.run.smithi152.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-20T11:25:46.141 INFO:teuthology.orchestra.run.smithi152.stdout: python3-rgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:25:46.141 INFO:teuthology.orchestra.run.smithi152.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-20T11:25:46.141 INFO:teuthology.orchestra.run.smithi152.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-20T11:25:46.141 INFO:teuthology.orchestra.run.smithi152.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-20T11:25:46.141 INFO:teuthology.orchestra.run.smithi152.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-20T11:25:46.141 INFO:teuthology.orchestra.run.smithi152.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-20T11:25:46.141 INFO:teuthology.orchestra.run.smithi152.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-20T11:25:46.141 INFO:teuthology.orchestra.run.smithi152.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-20T11:25:46.141 INFO:teuthology.orchestra.run.smithi152.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-20T11:25:46.141 INFO:teuthology.orchestra.run.smithi152.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-20T11:25:46.141 INFO:teuthology.orchestra.run.smithi152.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-20T11:25:46.142 INFO:teuthology.orchestra.run.smithi152.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-20T11:25:46.142 INFO:teuthology.orchestra.run.smithi152.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-20T11:25:46.142 INFO:teuthology.orchestra.run.smithi152.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-20T11:25:46.142 INFO:teuthology.orchestra.run.smithi152.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-20T11:25:46.142 INFO:teuthology.orchestra.run.smithi152.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-20T11:25:46.142 INFO:teuthology.orchestra.run.smithi152.stdout: rbd-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:25:46.142 INFO:teuthology.orchestra.run.smithi152.stdout: rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:25:46.142 INFO:teuthology.orchestra.run.smithi152.stdout: rbd-nbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:25:46.142 INFO:teuthology.orchestra.run.smithi152.stdout: re2-1:20211101-3.el9.x86_64 2024-04-20T11:25:46.142 INFO:teuthology.orchestra.run.smithi152.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-20T11:25:46.142 INFO:teuthology.orchestra.run.smithi152.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-20T11:25:46.142 INFO:teuthology.orchestra.run.smithi152.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-20T11:25:46.142 INFO:teuthology.orchestra.run.smithi152.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-20T11:25:46.143 INFO:teuthology.orchestra.run.smithi152.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-20T11:25:46.143 INFO:teuthology.orchestra.run.smithi152.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-20T11:25:46.143 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:25:46.143 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2024-04-20T11:25:46.708 DEBUG:teuthology.parallel:result is None 2024-04-20T11:25:47.356 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/152 2024-04-20T11:25:47.356 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-20T11:25:47.356 INFO:teuthology.orchestra.run.smithi175.stdout:Upgraded: 2024-04-20T11:25:47.356 INFO:teuthology.orchestra.run.smithi175.stdout: librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:25:47.356 INFO:teuthology.orchestra.run.smithi175.stdout: librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:25:47.356 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2024-04-20T11:25:47.356 INFO:teuthology.orchestra.run.smithi175.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-20T11:25:47.356 INFO:teuthology.orchestra.run.smithi175.stdout: automake-1.16.2-8.el9.noarch 2024-04-20T11:25:47.356 INFO:teuthology.orchestra.run.smithi175.stdout: bison-3.7.4-5.el9.x86_64 2024-04-20T11:25:47.356 INFO:teuthology.orchestra.run.smithi175.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-20T11:25:47.356 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:25:47.356 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:25:47.357 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:25:47.357 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:25:47.357 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-grafana-dashboards-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T11:25:47.357 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-immutable-object-cache-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:25:47.357 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:25:47.357 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:25:47.357 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T11:25:47.357 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T11:25:47.357 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T11:25:47.357 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-modules-core-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T11:25:47.357 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T11:25:47.357 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:25:47.357 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:25:47.357 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-prometheus-alerts-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T11:25:47.358 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:25:47.358 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:25:47.358 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-test-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:25:47.358 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T11:25:47.358 INFO:teuthology.orchestra.run.smithi175.stdout: cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T11:25:47.358 INFO:teuthology.orchestra.run.smithi175.stdout: cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:25:47.358 INFO:teuthology.orchestra.run.smithi175.stdout: cephfs-shell-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:25:47.358 INFO:teuthology.orchestra.run.smithi175.stdout: cephfs-top-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T11:25:47.358 INFO:teuthology.orchestra.run.smithi175.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-20T11:25:47.358 INFO:teuthology.orchestra.run.smithi175.stdout: flex-2.6.4-9.el9.x86_64 2024-04-20T11:25:47.358 INFO:teuthology.orchestra.run.smithi175.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-20T11:25:47.358 INFO:teuthology.orchestra.run.smithi175.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-20T11:25:47.358 INFO:teuthology.orchestra.run.smithi175.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-20T11:25:47.359 INFO:teuthology.orchestra.run.smithi175.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-20T11:25:47.359 INFO:teuthology.orchestra.run.smithi175.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-20T11:25:47.359 INFO:teuthology.orchestra.run.smithi175.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-20T11:25:47.359 INFO:teuthology.orchestra.run.smithi175.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-20T11:25:47.359 INFO:teuthology.orchestra.run.smithi175.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-20T11:25:47.359 INFO:teuthology.orchestra.run.smithi175.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-20T11:25:47.359 INFO:teuthology.orchestra.run.smithi175.stdout: libcephfs-devel-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:25:47.359 INFO:teuthology.orchestra.run.smithi175.stdout: libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:25:47.359 INFO:teuthology.orchestra.run.smithi175.stdout: libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:25:47.359 INFO:teuthology.orchestra.run.smithi175.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-20T11:25:47.359 INFO:teuthology.orchestra.run.smithi175.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-20T11:25:47.359 INFO:teuthology.orchestra.run.smithi175.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-20T11:25:47.359 INFO:teuthology.orchestra.run.smithi175.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-20T11:25:47.359 INFO:teuthology.orchestra.run.smithi175.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-20T11:25:47.360 INFO:teuthology.orchestra.run.smithi175.stdout: librados-devel-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:25:47.360 INFO:teuthology.orchestra.run.smithi175.stdout: libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:25:47.360 INFO:teuthology.orchestra.run.smithi175.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-20T11:25:47.360 INFO:teuthology.orchestra.run.smithi175.stdout: librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:25:47.360 INFO:teuthology.orchestra.run.smithi175.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-20T11:25:47.360 INFO:teuthology.orchestra.run.smithi175.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-20T11:25:47.360 INFO:teuthology.orchestra.run.smithi175.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-20T11:25:47.360 INFO:teuthology.orchestra.run.smithi175.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-20T11:25:47.360 INFO:teuthology.orchestra.run.smithi175.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-20T11:25:47.360 INFO:teuthology.orchestra.run.smithi175.stdout: lua-5.4.4-4.el9.x86_64 2024-04-20T11:25:47.360 INFO:teuthology.orchestra.run.smithi175.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-20T11:25:47.360 INFO:teuthology.orchestra.run.smithi175.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-20T11:25:47.360 INFO:teuthology.orchestra.run.smithi175.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-20T11:25:47.360 INFO:teuthology.orchestra.run.smithi175.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-20T11:25:47.361 INFO:teuthology.orchestra.run.smithi175.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-20T11:25:47.361 INFO:teuthology.orchestra.run.smithi175.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-20T11:25:47.361 INFO:teuthology.orchestra.run.smithi175.stdout: postgresql-13.14-1.el9.x86_64 2024-04-20T11:25:47.361 INFO:teuthology.orchestra.run.smithi175.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-20T11:25:47.361 INFO:teuthology.orchestra.run.smithi175.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-20T11:25:47.361 INFO:teuthology.orchestra.run.smithi175.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-20T11:25:47.361 INFO:teuthology.orchestra.run.smithi175.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-20T11:25:47.361 INFO:teuthology.orchestra.run.smithi175.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-20T11:25:47.361 INFO:teuthology.orchestra.run.smithi175.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-20T11:25:47.361 INFO:teuthology.orchestra.run.smithi175.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-20T11:25:47.361 INFO:teuthology.orchestra.run.smithi175.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-20T11:25:47.361 INFO:teuthology.orchestra.run.smithi175.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-20T11:25:47.361 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-20T11:25:47.361 INFO:teuthology.orchestra.run.smithi175.stdout: python3-ceph-argparse-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:25:47.362 INFO:teuthology.orchestra.run.smithi175.stdout: python3-ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:25:47.362 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cephfs-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:25:47.362 INFO:teuthology.orchestra.run.smithi175.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-20T11:25:47.362 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-20T11:25:47.362 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-20T11:25:47.362 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-20T11:25:47.362 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-20T11:25:47.362 INFO:teuthology.orchestra.run.smithi175.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-20T11:25:47.362 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-20T11:25:47.362 INFO:teuthology.orchestra.run.smithi175.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-20T11:25:47.362 INFO:teuthology.orchestra.run.smithi175.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-20T11:25:47.362 INFO:teuthology.orchestra.run.smithi175.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-20T11:25:47.362 INFO:teuthology.orchestra.run.smithi175.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-20T11:25:47.362 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-20T11:25:47.363 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-20T11:25:47.363 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-20T11:25:47.363 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-20T11:25:47.363 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-20T11:25:47.363 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-20T11:25:47.363 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-20T11:25:47.363 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-20T11:25:47.363 INFO:teuthology.orchestra.run.smithi175.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-20T11:25:47.363 INFO:teuthology.orchestra.run.smithi175.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-20T11:25:47.363 INFO:teuthology.orchestra.run.smithi175.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-20T11:25:47.363 INFO:teuthology.orchestra.run.smithi175.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-20T11:25:47.363 INFO:teuthology.orchestra.run.smithi175.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-20T11:25:47.363 INFO:teuthology.orchestra.run.smithi175.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-20T11:25:47.364 INFO:teuthology.orchestra.run.smithi175.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-20T11:25:47.364 INFO:teuthology.orchestra.run.smithi175.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-20T11:25:47.364 INFO:teuthology.orchestra.run.smithi175.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-20T11:25:47.364 INFO:teuthology.orchestra.run.smithi175.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-20T11:25:47.364 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-20T11:25:47.364 INFO:teuthology.orchestra.run.smithi175.stdout: python3-ply-3.11-14.el9.noarch 2024-04-20T11:25:47.364 INFO:teuthology.orchestra.run.smithi175.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-20T11:25:47.364 INFO:teuthology.orchestra.run.smithi175.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-20T11:25:47.364 INFO:teuthology.orchestra.run.smithi175.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-20T11:25:47.364 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-20T11:25:47.364 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-20T11:25:47.364 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-20T11:25:47.364 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-20T11:25:47.364 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-20T11:25:47.364 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-20T11:25:47.365 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rados-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:25:47.365 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:25:47.365 INFO:teuthology.orchestra.run.smithi175.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-20T11:25:47.365 INFO:teuthology.orchestra.run.smithi175.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-20T11:25:47.365 INFO:teuthology.orchestra.run.smithi175.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-20T11:25:47.365 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:25:47.365 INFO:teuthology.orchestra.run.smithi175.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-20T11:25:47.365 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-20T11:25:47.365 INFO:teuthology.orchestra.run.smithi175.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-20T11:25:47.365 INFO:teuthology.orchestra.run.smithi175.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-20T11:25:47.365 INFO:teuthology.orchestra.run.smithi175.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-20T11:25:47.365 INFO:teuthology.orchestra.run.smithi175.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-20T11:25:47.365 INFO:teuthology.orchestra.run.smithi175.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-20T11:25:47.365 INFO:teuthology.orchestra.run.smithi175.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-20T11:25:47.366 INFO:teuthology.orchestra.run.smithi175.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-20T11:25:47.366 INFO:teuthology.orchestra.run.smithi175.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-20T11:25:47.366 INFO:teuthology.orchestra.run.smithi175.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-20T11:25:47.366 INFO:teuthology.orchestra.run.smithi175.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-20T11:25:47.366 INFO:teuthology.orchestra.run.smithi175.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-20T11:25:47.366 INFO:teuthology.orchestra.run.smithi175.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-20T11:25:47.366 INFO:teuthology.orchestra.run.smithi175.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-20T11:25:47.366 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:25:47.366 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:25:47.366 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-nbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:25:47.366 INFO:teuthology.orchestra.run.smithi175.stdout: re2-1:20211101-3.el9.x86_64 2024-04-20T11:25:47.366 INFO:teuthology.orchestra.run.smithi175.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-20T11:25:47.366 INFO:teuthology.orchestra.run.smithi175.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-20T11:25:47.366 INFO:teuthology.orchestra.run.smithi175.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-20T11:25:47.367 INFO:teuthology.orchestra.run.smithi175.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-20T11:25:47.367 INFO:teuthology.orchestra.run.smithi175.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-20T11:25:47.367 INFO:teuthology.orchestra.run.smithi175.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-20T11:25:47.367 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-20T11:25:47.367 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2024-04-20T11:25:47.927 DEBUG:teuthology.parallel:result is None 2024-04-20T11:25:47.927 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a8c84f5997950fd7d008204839253291d52e6eda 2024-04-20T11:25:48.198 DEBUG:teuthology.orchestra.run.smithi152:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-20T11:25:48.250 INFO:teuthology.orchestra.run.smithi152.stdout:19.0.0-3133.ga8c84f59.el9 2024-04-20T11:25:48.251 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3133.ga8c84f59.el9 2024-04-20T11:25:48.251 INFO:teuthology.task.install:The correct ceph version 19.0.0-3133.ga8c84f59 is installed. 2024-04-20T11:25:48.253 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a8c84f5997950fd7d008204839253291d52e6eda 2024-04-20T11:25:48.492 DEBUG:teuthology.orchestra.run.smithi175:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-20T11:25:48.544 INFO:teuthology.orchestra.run.smithi175.stdout:19.0.0-3133.ga8c84f59.el9 2024-04-20T11:25:48.545 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3133.ga8c84f59.el9 2024-04-20T11:25:48.545 INFO:teuthology.task.install:The correct ceph version 19.0.0-3133.ga8c84f59 is installed. 2024-04-20T11:25:48.547 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-20T11:25:48.547 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-20T11:25:48.547 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-20T11:25:48.619 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T11:25:48.620 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-20T11:25:48.695 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-20T11:25:48.695 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-20T11:25:48.695 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/usr/bin/daemon-helper 2024-04-20T11:25:48.756 DEBUG:teuthology.orchestra.run.smithi152:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-20T11:25:48.857 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T11:25:48.857 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/usr/bin/daemon-helper 2024-04-20T11:25:48.932 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-20T11:25:49.039 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-20T11:25:49.039 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-20T11:25:49.039 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-20T11:25:49.105 DEBUG:teuthology.orchestra.run.smithi152:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-20T11:25:49.209 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T11:25:49.209 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-20T11:25:49.278 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-20T11:25:49.384 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-20T11:25:49.385 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-20T11:25:49.385 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/usr/bin/stdin-killer 2024-04-20T11:25:49.447 DEBUG:teuthology.orchestra.run.smithi152:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-20T11:25:49.547 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T11:25:49.547 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/usr/bin/stdin-killer 2024-04-20T11:25:49.613 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-20T11:25:49.722 INFO:teuthology.run_tasks:Running task ceph... 2024-04-20T11:25:49.808 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-20T11:25:49.809 DEBUG:teuthology.orchestra.run.smithi152:> sudo chmod 777 /var/log/ceph 2024-04-20T11:25:49.814 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod 777 /var/log/ceph 2024-04-20T11:25:49.891 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-20T11:25:49.891 DEBUG:teuthology.orchestra.run.smithi152:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-20T11:25:49.896 DEBUG:teuthology.orchestra.run.smithi175:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-20T11:25:49.960 INFO:tasks.ceph:Creating extra log directories... 2024-04-20T11:25:49.961 DEBUG:teuthology.orchestra.run.smithi152:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-20T11:25:49.965 DEBUG:teuthology.orchestra.run.smithi175:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-20T11:25:50.037 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-04-20T11:25:50.037 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-04-20T11:25:50.038 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-04-20T11:25:50.038 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-20T11:25:50.038 DEBUG:teuthology.orchestra.run.smithi152:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-20T11:25:50.077 DEBUG:teuthology.orchestra.run.smithi152:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-20T11:25:50.266 DEBUG:teuthology.orchestra.run.smithi152:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-20T11:25:50.329 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T11:25:50.329 DEBUG:teuthology.orchestra.run.smithi175:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-20T11:25:50.375 DEBUG:teuthology.orchestra.run.smithi175:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-20T11:25:50.569 DEBUG:teuthology.orchestra.run.smithi175:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-20T11:25:50.645 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-20T11:25:50.645 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-20T11:25:50.645 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7664528', 'branch': 'wip-rishabh-testing-20240416.193735', 'description': 'fs/volumes/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/misc}}', 'email': None, 'first_in_suite': False, 'job_id': '7664528', 'kernel': {'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-rishabh-testing-20240416.193735'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-rishabh-testing-20240416.193735', 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda'}}, 'owner': 'scheduled_rishabh@teuthology', 'priority': 50, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 7954, 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda', 'sleep_before_teardown': 0, 'subset': '364/512', 'suite': 'fs', 'suite_branch': 'wip-rishabh-testing-20240416.193735', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': 'a8c84f5997950fd7d008204839253291d52e6eda', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi152.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOVyVBdxmuG8HMwDEY6F4/Lr1b7DL56GYyxRXWbfXazQz/oSpmCDGlA7vu8AiwGwSHiEeadPLzoz3W+lgsRim50=', 'smithi175.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDMTvBfTjexM8GK5wjqYosCBbUVUEKh7Cqxx6F7+ox2/x5WsWbP9nnMtiYAGheE3S14t5e++9VidTHs74dFQ13o='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'ceph-fuse': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_volumes.TestMisc']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17', 'timestamp': '2024-04-20_09:50:15', 'tube': 'smithi', 'user': 'rishabh', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-20T11:25:50.646 DEBUG:teuthology.orchestra.run.smithi152:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-20T11:25:50.650 DEBUG:teuthology.orchestra.run.smithi175:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-20T11:25:50.692 DEBUG:teuthology.orchestra.run.smithi152:> sudo install -d -m0777 -- /var/run/ceph 2024-04-20T11:25:50.732 DEBUG:teuthology.orchestra.run.smithi175:> sudo install -d -m0777 -- /var/run/ceph 2024-04-20T11:25:50.799 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-20T11:25:50.799 DEBUG:teuthology.orchestra.run.smithi152:> dd if=/scratch_devs of=/dev/stdout 2024-04-20T11:25:50.836 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-20T11:25:50.836 DEBUG:teuthology.orchestra.run.smithi152:> stat /dev/vg_nvme/lv_1 2024-04-20T11:25:50.915 INFO:teuthology.orchestra.run.smithi152.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-20T11:25:50.915 INFO:teuthology.orchestra.run.smithi152.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T11:25:50.916 INFO:teuthology.orchestra.run.smithi152.stdout:Device: 6h/6d Inode: 631 Links: 1 2024-04-20T11:25:50.916 INFO:teuthology.orchestra.run.smithi152.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T11:25:50.916 INFO:teuthology.orchestra.run.smithi152.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T11:25:50.916 INFO:teuthology.orchestra.run.smithi152.stdout:Access: 2024-04-20 11:25:38.248415091 +0000 2024-04-20T11:25:50.916 INFO:teuthology.orchestra.run.smithi152.stdout:Modify: 2024-04-20 11:22:37.849220992 +0000 2024-04-20T11:25:50.916 INFO:teuthology.orchestra.run.smithi152.stdout:Change: 2024-04-20 11:22:37.849220992 +0000 2024-04-20T11:25:50.916 INFO:teuthology.orchestra.run.smithi152.stdout: Birth: 2024-04-20 11:22:37.849220992 +0000 2024-04-20T11:25:50.916 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-20T11:25:51.022 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records in 2024-04-20T11:25:51.022 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records out 2024-04-20T11:25:51.022 INFO:teuthology.orchestra.run.smithi152.stderr:512 bytes copied, 0.000332504 s, 1.5 MB/s 2024-04-20T11:25:51.025 DEBUG:teuthology.orchestra.run.smithi152:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-20T11:25:51.066 DEBUG:teuthology.orchestra.run.smithi152:> stat /dev/vg_nvme/lv_2 2024-04-20T11:25:51.150 INFO:teuthology.orchestra.run.smithi152.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-20T11:25:51.150 INFO:teuthology.orchestra.run.smithi152.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T11:25:51.150 INFO:teuthology.orchestra.run.smithi152.stdout:Device: 6h/6d Inode: 652 Links: 1 2024-04-20T11:25:51.150 INFO:teuthology.orchestra.run.smithi152.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T11:25:51.150 INFO:teuthology.orchestra.run.smithi152.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T11:25:51.150 INFO:teuthology.orchestra.run.smithi152.stdout:Access: 2024-04-20 11:25:38.250415068 +0000 2024-04-20T11:25:51.150 INFO:teuthology.orchestra.run.smithi152.stdout:Modify: 2024-04-20 11:22:38.432214315 +0000 2024-04-20T11:25:51.150 INFO:teuthology.orchestra.run.smithi152.stdout:Change: 2024-04-20 11:22:38.432214315 +0000 2024-04-20T11:25:51.150 INFO:teuthology.orchestra.run.smithi152.stdout: Birth: 2024-04-20 11:22:38.432214315 +0000 2024-04-20T11:25:51.151 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-20T11:25:51.248 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records in 2024-04-20T11:25:51.248 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records out 2024-04-20T11:25:51.248 INFO:teuthology.orchestra.run.smithi152.stderr:512 bytes copied, 0.000271683 s, 1.9 MB/s 2024-04-20T11:25:51.252 DEBUG:teuthology.orchestra.run.smithi152:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-20T11:25:51.292 DEBUG:teuthology.orchestra.run.smithi152:> stat /dev/vg_nvme/lv_3 2024-04-20T11:25:51.371 INFO:teuthology.orchestra.run.smithi152.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-20T11:25:51.371 INFO:teuthology.orchestra.run.smithi152.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T11:25:51.371 INFO:teuthology.orchestra.run.smithi152.stdout:Device: 6h/6d Inode: 672 Links: 1 2024-04-20T11:25:51.371 INFO:teuthology.orchestra.run.smithi152.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T11:25:51.371 INFO:teuthology.orchestra.run.smithi152.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T11:25:51.371 INFO:teuthology.orchestra.run.smithi152.stdout:Access: 2024-04-20 11:25:38.250415068 +0000 2024-04-20T11:25:51.371 INFO:teuthology.orchestra.run.smithi152.stdout:Modify: 2024-04-20 11:22:39.046207283 +0000 2024-04-20T11:25:51.371 INFO:teuthology.orchestra.run.smithi152.stdout:Change: 2024-04-20 11:22:39.046207283 +0000 2024-04-20T11:25:51.371 INFO:teuthology.orchestra.run.smithi152.stdout: Birth: 2024-04-20 11:22:39.046207283 +0000 2024-04-20T11:25:51.372 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-20T11:25:51.466 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records in 2024-04-20T11:25:51.466 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records out 2024-04-20T11:25:51.466 INFO:teuthology.orchestra.run.smithi152.stderr:512 bytes copied, 0.000265778 s, 1.9 MB/s 2024-04-20T11:25:51.469 DEBUG:teuthology.orchestra.run.smithi152:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-20T11:25:51.511 DEBUG:teuthology.orchestra.run.smithi152:> stat /dev/vg_nvme/lv_4 2024-04-20T11:25:51.594 INFO:teuthology.orchestra.run.smithi152.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-20T11:25:51.594 INFO:teuthology.orchestra.run.smithi152.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T11:25:51.594 INFO:teuthology.orchestra.run.smithi152.stdout:Device: 6h/6d Inode: 689 Links: 1 2024-04-20T11:25:51.594 INFO:teuthology.orchestra.run.smithi152.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T11:25:51.595 INFO:teuthology.orchestra.run.smithi152.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T11:25:51.595 INFO:teuthology.orchestra.run.smithi152.stdout:Access: 2024-04-20 11:25:38.251415056 +0000 2024-04-20T11:25:51.595 INFO:teuthology.orchestra.run.smithi152.stdout:Modify: 2024-04-20 11:22:39.645200422 +0000 2024-04-20T11:25:51.595 INFO:teuthology.orchestra.run.smithi152.stdout:Change: 2024-04-20 11:22:39.645200422 +0000 2024-04-20T11:25:51.595 INFO:teuthology.orchestra.run.smithi152.stdout: Birth: 2024-04-20 11:22:39.645200422 +0000 2024-04-20T11:25:51.595 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-20T11:25:51.690 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records in 2024-04-20T11:25:51.691 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records out 2024-04-20T11:25:51.691 INFO:teuthology.orchestra.run.smithi152.stderr:512 bytes copied, 0.000260102 s, 2.0 MB/s 2024-04-20T11:25:51.693 DEBUG:teuthology.orchestra.run.smithi152:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-20T11:25:51.734 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-20T11:25:51.735 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T11:25:51.735 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/scratch_devs of=/dev/stdout 2024-04-20T11:25:51.779 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-20T11:25:51.779 DEBUG:teuthology.orchestra.run.smithi175:> stat /dev/vg_nvme/lv_1 2024-04-20T11:25:51.863 INFO:teuthology.orchestra.run.smithi175.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-20T11:25:51.863 INFO:teuthology.orchestra.run.smithi175.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T11:25:51.863 INFO:teuthology.orchestra.run.smithi175.stdout:Device: 6h/6d Inode: 632 Links: 1 2024-04-20T11:25:51.863 INFO:teuthology.orchestra.run.smithi175.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T11:25:51.863 INFO:teuthology.orchestra.run.smithi175.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T11:25:51.863 INFO:teuthology.orchestra.run.smithi175.stdout:Access: 2024-04-20 11:25:39.935747251 +0000 2024-04-20T11:25:51.863 INFO:teuthology.orchestra.run.smithi175.stdout:Modify: 2024-04-20 11:22:39.692378406 +0000 2024-04-20T11:25:51.863 INFO:teuthology.orchestra.run.smithi175.stdout:Change: 2024-04-20 11:22:39.692378406 +0000 2024-04-20T11:25:51.863 INFO:teuthology.orchestra.run.smithi175.stdout: Birth: 2024-04-20 11:22:39.692378406 +0000 2024-04-20T11:25:51.864 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-20T11:25:51.960 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records in 2024-04-20T11:25:51.960 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records out 2024-04-20T11:25:51.960 INFO:teuthology.orchestra.run.smithi175.stderr:512 bytes copied, 0.000413989 s, 1.2 MB/s 2024-04-20T11:25:51.963 DEBUG:teuthology.orchestra.run.smithi175:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-20T11:25:52.004 DEBUG:teuthology.orchestra.run.smithi175:> stat /dev/vg_nvme/lv_2 2024-04-20T11:25:52.086 INFO:teuthology.orchestra.run.smithi175.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-20T11:25:52.086 INFO:teuthology.orchestra.run.smithi175.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T11:25:52.086 INFO:teuthology.orchestra.run.smithi175.stdout:Device: 6h/6d Inode: 651 Links: 1 2024-04-20T11:25:52.086 INFO:teuthology.orchestra.run.smithi175.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T11:25:52.086 INFO:teuthology.orchestra.run.smithi175.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T11:25:52.087 INFO:teuthology.orchestra.run.smithi175.stdout:Access: 2024-04-20 11:25:39.937747219 +0000 2024-04-20T11:25:52.087 INFO:teuthology.orchestra.run.smithi175.stdout:Modify: 2024-04-20 11:22:40.300368940 +0000 2024-04-20T11:25:52.087 INFO:teuthology.orchestra.run.smithi175.stdout:Change: 2024-04-20 11:22:40.300368940 +0000 2024-04-20T11:25:52.087 INFO:teuthology.orchestra.run.smithi175.stdout: Birth: 2024-04-20 11:22:40.300368940 +0000 2024-04-20T11:25:52.087 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-20T11:25:52.201 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records in 2024-04-20T11:25:52.201 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records out 2024-04-20T11:25:52.201 INFO:teuthology.orchestra.run.smithi175.stderr:512 bytes copied, 0.000270949 s, 1.9 MB/s 2024-04-20T11:25:52.203 DEBUG:teuthology.orchestra.run.smithi175:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-20T11:25:52.245 DEBUG:teuthology.orchestra.run.smithi175:> stat /dev/vg_nvme/lv_3 2024-04-20T11:25:52.328 INFO:teuthology.orchestra.run.smithi175.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-20T11:25:52.328 INFO:teuthology.orchestra.run.smithi175.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T11:25:52.328 INFO:teuthology.orchestra.run.smithi175.stdout:Device: 6h/6d Inode: 669 Links: 1 2024-04-20T11:25:52.328 INFO:teuthology.orchestra.run.smithi175.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T11:25:52.328 INFO:teuthology.orchestra.run.smithi175.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T11:25:52.329 INFO:teuthology.orchestra.run.smithi175.stdout:Access: 2024-04-20 11:25:39.937747219 +0000 2024-04-20T11:25:52.329 INFO:teuthology.orchestra.run.smithi175.stdout:Modify: 2024-04-20 11:22:40.892359723 +0000 2024-04-20T11:25:52.329 INFO:teuthology.orchestra.run.smithi175.stdout:Change: 2024-04-20 11:22:40.892359723 +0000 2024-04-20T11:25:52.329 INFO:teuthology.orchestra.run.smithi175.stdout: Birth: 2024-04-20 11:22:40.892359723 +0000 2024-04-20T11:25:52.329 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-20T11:25:52.431 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records in 2024-04-20T11:25:52.431 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records out 2024-04-20T11:25:52.431 INFO:teuthology.orchestra.run.smithi175.stderr:512 bytes copied, 0.000324146 s, 1.6 MB/s 2024-04-20T11:25:52.434 DEBUG:teuthology.orchestra.run.smithi175:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-20T11:25:52.477 DEBUG:teuthology.orchestra.run.smithi175:> stat /dev/vg_nvme/lv_4 2024-04-20T11:25:52.556 INFO:teuthology.orchestra.run.smithi175.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-20T11:25:52.556 INFO:teuthology.orchestra.run.smithi175.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T11:25:52.556 INFO:teuthology.orchestra.run.smithi175.stdout:Device: 6h/6d Inode: 688 Links: 1 2024-04-20T11:25:52.556 INFO:teuthology.orchestra.run.smithi175.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T11:25:52.556 INFO:teuthology.orchestra.run.smithi175.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T11:25:52.556 INFO:teuthology.orchestra.run.smithi175.stdout:Access: 2024-04-20 11:25:39.938747204 +0000 2024-04-20T11:25:52.556 INFO:teuthology.orchestra.run.smithi175.stdout:Modify: 2024-04-20 11:22:41.513350054 +0000 2024-04-20T11:25:52.556 INFO:teuthology.orchestra.run.smithi175.stdout:Change: 2024-04-20 11:22:41.513350054 +0000 2024-04-20T11:25:52.556 INFO:teuthology.orchestra.run.smithi175.stdout: Birth: 2024-04-20 11:22:41.513350054 +0000 2024-04-20T11:25:52.557 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-20T11:25:52.656 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records in 2024-04-20T11:25:52.656 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records out 2024-04-20T11:25:52.656 INFO:teuthology.orchestra.run.smithi175.stderr:512 bytes copied, 0.000280345 s, 1.8 MB/s 2024-04-20T11:25:52.659 DEBUG:teuthology.orchestra.run.smithi175:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-20T11:25:52.703 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-20T11:25:52.703 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi152.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi175.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-20T11:25:52.703 INFO:tasks.ceph:Generating config... 2024-04-20T11:25:52.704 INFO:tasks.ceph:[client] client mount timeout = 600 2024-04-20T11:25:52.705 INFO:tasks.ceph:[client] debug client = 20 2024-04-20T11:25:52.705 INFO:tasks.ceph:[client] debug ms = 1 2024-04-20T11:25:52.705 INFO:tasks.ceph:[client] pid file = 2024-04-20T11:25:52.705 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-04-20T11:25:52.705 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-04-20T11:25:52.705 INFO:tasks.ceph:[mds] debug mds = 20 2024-04-20T11:25:52.705 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-04-20T11:25:52.705 INFO:tasks.ceph:[mds] debug ms = 1 2024-04-20T11:25:52.705 INFO:tasks.ceph:[mds] mds debug frag = True 2024-04-20T11:25:52.705 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-04-20T11:25:52.705 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-04-20T11:25:52.705 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-04-20T11:25:52.705 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-04-20T11:25:52.705 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-04-20T11:25:52.706 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-04-20T11:25:52.706 INFO:tasks.ceph:[mgr] debug client = 20 2024-04-20T11:25:52.706 INFO:tasks.ceph:[mgr] debug finisher = 20 2024-04-20T11:25:52.706 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-20T11:25:52.706 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-20T11:25:52.706 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-20T11:25:52.706 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-20T11:25:52.706 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-20T11:25:52.706 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-04-20T11:25:52.706 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-04-20T11:25:52.706 INFO:tasks.ceph:[osd] bdev async discard = True 2024-04-20T11:25:52.706 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-04-20T11:25:52.706 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-04-20T11:25:52.706 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-20T11:25:52.706 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-20T11:25:52.707 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-04-20T11:25:52.707 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-04-20T11:25:52.707 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-20T11:25:52.707 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-20T11:25:52.707 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-04-20T11:25:52.707 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-20T11:25:52.707 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-20T11:25:52.707 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-20T11:25:52.707 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-20T11:25:52.707 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-20T11:25:52.707 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-04-20T11:25:52.707 INFO:tasks.ceph:Setting up mon.a... 2024-04-20T11:25:52.708 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-20T11:25:52.808 INFO:teuthology.orchestra.run.smithi152.stdout:creating /etc/ceph/ceph.keyring 2024-04-20T11:25:52.814 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-20T11:25:52.903 DEBUG:teuthology.orchestra.run.smithi152:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-20T11:25:52.963 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.152'), ('mon.b', '172.21.15.175'), ('mon.c', '[v2:172.21.15.175:3301,v1:172.21.15.175:6790]')] 2024-04-20T11:25:52.964 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.152,172.21.15.175,[v2:172.21.15.175:3301,v1:172.21.15.175:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20, 'debug finisher': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-04-20T11:25:52.965 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-20T11:25:52.965 DEBUG:teuthology.orchestra.run.smithi152:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-20T11:25:53.013 DEBUG:teuthology.orchestra.run.smithi152:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.152 --add b 172.21.15.175 --addv c '[v2:172.21.15.175:3301,v1:172.21.15.175:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-04-20T11:25:53.160 INFO:teuthology.orchestra.run.smithi152.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-20T11:25:53.160 INFO:teuthology.orchestra.run.smithi152.stdout:monmaptool: generated fsid c8a72131-0d4f-42c7-892f-3f3af4d2bd3d 2024-04-20T11:25:53.161 INFO:teuthology.orchestra.run.smithi152.stdout:setting min_mon_release = quincy 2024-04-20T11:25:53.161 INFO:teuthology.orchestra.run.smithi152.stdout:epoch 0 2024-04-20T11:25:53.161 INFO:teuthology.orchestra.run.smithi152.stdout:fsid c8a72131-0d4f-42c7-892f-3f3af4d2bd3d 2024-04-20T11:25:53.161 INFO:teuthology.orchestra.run.smithi152.stdout:last_changed 2024-04-20T11:25:53.161797+0000 2024-04-20T11:25:53.161 INFO:teuthology.orchestra.run.smithi152.stdout:created 2024-04-20T11:25:53.161797+0000 2024-04-20T11:25:53.161 INFO:teuthology.orchestra.run.smithi152.stdout:min_mon_release 17 (quincy) 2024-04-20T11:25:53.161 INFO:teuthology.orchestra.run.smithi152.stdout:election_strategy: 1 2024-04-20T11:25:53.161 INFO:teuthology.orchestra.run.smithi152.stdout:0: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.a 2024-04-20T11:25:53.161 INFO:teuthology.orchestra.run.smithi152.stdout:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2024-04-20T11:25:53.161 INFO:teuthology.orchestra.run.smithi152.stdout:2: [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] mon.c 2024-04-20T11:25:53.161 INFO:teuthology.orchestra.run.smithi152.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-04-20T11:25:53.164 DEBUG:teuthology.orchestra.run.smithi152:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-20T11:25:53.209 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID c8a72131-0d4f-42c7-892f-3f3af4d2bd3d... 2024-04-20T11:25:53.210 DEBUG:teuthology.orchestra.run.smithi152:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-20T11:25:53.255 DEBUG:teuthology.orchestra.run.smithi175:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-20T11:25:53.367 INFO:teuthology.orchestra.run.smithi175.stdout:[global] 2024-04-20T11:25:53.367 INFO:teuthology.orchestra.run.smithi175.stdout: chdir = "" 2024-04-20T11:25:53.367 INFO:teuthology.orchestra.run.smithi175.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-20T11:25:53.367 INFO:teuthology.orchestra.run.smithi175.stdout: auth supported = cephx 2024-04-20T11:25:53.367 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-20T11:25:53.367 INFO:teuthology.orchestra.run.smithi175.stdout: filestore xattr use omap = true 2024-04-20T11:25:53.367 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-20T11:25:53.367 INFO:teuthology.orchestra.run.smithi175.stdout: mon clock drift allowed = 1.000 2024-04-20T11:25:53.367 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-20T11:25:53.367 INFO:teuthology.orchestra.run.smithi175.stdout: osd crush chooseleaf type = 0 2024-04-20T11:25:53.367 INFO:teuthology.orchestra.run.smithi175.stdout: auth debug = true 2024-04-20T11:25:53.368 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-20T11:25:53.368 INFO:teuthology.orchestra.run.smithi175.stdout: ms die on old message = true 2024-04-20T11:25:53.368 INFO:teuthology.orchestra.run.smithi175.stdout: ms die on bug = true 2024-04-20T11:25:53.368 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-20T11:25:53.368 INFO:teuthology.orchestra.run.smithi175.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-20T11:25:53.368 INFO:teuthology.orchestra.run.smithi175.stdout: mon pg warn max object skew = 0 2024-04-20T11:25:53.368 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-20T11:25:53.368 INFO:teuthology.orchestra.run.smithi175.stdout: # disable pg_autoscaler by default for new pools 2024-04-20T11:25:53.368 INFO:teuthology.orchestra.run.smithi175.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-20T11:25:53.368 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-20T11:25:53.368 INFO:teuthology.orchestra.run.smithi175.stdout: osd pool default size = 2 2024-04-20T11:25:53.368 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-20T11:25:53.368 INFO:teuthology.orchestra.run.smithi175.stdout: mon osd allow primary affinity = true 2024-04-20T11:25:53.368 INFO:teuthology.orchestra.run.smithi175.stdout: mon osd allow pg remap = true 2024-04-20T11:25:53.368 INFO:teuthology.orchestra.run.smithi175.stdout: mon warn on legacy crush tunables = false 2024-04-20T11:25:53.369 INFO:teuthology.orchestra.run.smithi175.stdout: mon warn on crush straw calc version zero = false 2024-04-20T11:25:53.369 INFO:teuthology.orchestra.run.smithi175.stdout: mon warn on no sortbitwise = false 2024-04-20T11:25:53.369 INFO:teuthology.orchestra.run.smithi175.stdout: mon warn on osd down out interval zero = false 2024-04-20T11:25:53.369 INFO:teuthology.orchestra.run.smithi175.stdout: mon warn on too few osds = false 2024-04-20T11:25:53.369 INFO:teuthology.orchestra.run.smithi175.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-20T11:25:53.369 INFO:teuthology.orchestra.run.smithi175.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-20T11:25:53.369 INFO:teuthology.orchestra.run.smithi175.stdout: mon_allow_pool_size_one = true 2024-04-20T11:25:53.369 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-20T11:25:53.369 INFO:teuthology.orchestra.run.smithi175.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-20T11:25:53.369 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-20T11:25:53.369 INFO:teuthology.orchestra.run.smithi175.stdout: osd default data pool replay window = 5 2024-04-20T11:25:53.369 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-20T11:25:53.369 INFO:teuthology.orchestra.run.smithi175.stdout: mon allow pool delete = true 2024-04-20T11:25:53.369 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-20T11:25:53.370 INFO:teuthology.orchestra.run.smithi175.stdout: mon cluster log file level = debug 2024-04-20T11:25:53.370 INFO:teuthology.orchestra.run.smithi175.stdout: debug asserts on shutdown = true 2024-04-20T11:25:53.370 INFO:teuthology.orchestra.run.smithi175.stdout: mon health detail to clog = false 2024-04-20T11:25:53.370 INFO:teuthology.orchestra.run.smithi175.stdout: mon host = "172.21.15.152,172.21.15.175,[v2:172.21.15.175:3301,v1:172.21.15.175:6790]" 2024-04-20T11:25:53.370 INFO:teuthology.orchestra.run.smithi175.stdout: fsid = c8a72131-0d4f-42c7-892f-3f3af4d2bd3d 2024-04-20T11:25:53.370 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-20T11:25:53.370 INFO:teuthology.orchestra.run.smithi175.stdout:[osd] 2024-04-20T11:25:53.370 INFO:teuthology.orchestra.run.smithi175.stdout: osd journal size = 100 2024-04-20T11:25:53.370 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-20T11:25:53.370 INFO:teuthology.orchestra.run.smithi175.stdout: osd scrub load threshold = 5.0 2024-04-20T11:25:53.370 INFO:teuthology.orchestra.run.smithi175.stdout: osd scrub max interval = 600 2024-04-20T11:25:53.370 INFO:teuthology.orchestra.run.smithi175.stdout: osd mclock profile = high_recovery_ops 2024-04-20T11:25:53.370 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-20T11:25:53.370 INFO:teuthology.orchestra.run.smithi175.stdout: osd recover clone overlap = true 2024-04-20T11:25:53.370 INFO:teuthology.orchestra.run.smithi175.stdout: osd recovery max chunk = 1048576 2024-04-20T11:25:53.371 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-20T11:25:53.371 INFO:teuthology.orchestra.run.smithi175.stdout: osd debug shutdown = true 2024-04-20T11:25:53.371 INFO:teuthology.orchestra.run.smithi175.stdout: osd debug op order = true 2024-04-20T11:25:53.371 INFO:teuthology.orchestra.run.smithi175.stdout: osd debug verify stray on activate = true 2024-04-20T11:25:53.371 INFO:teuthology.orchestra.run.smithi175.stdout: osd debug trim objects = true 2024-04-20T11:25:53.371 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-20T11:25:53.371 INFO:teuthology.orchestra.run.smithi175.stdout: osd open classes on start = true 2024-04-20T11:25:53.371 INFO:teuthology.orchestra.run.smithi175.stdout: osd debug pg log writeout = true 2024-04-20T11:25:53.371 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-20T11:25:53.371 INFO:teuthology.orchestra.run.smithi175.stdout: osd deep scrub update digest min age = 30 2024-04-20T11:25:53.372 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-20T11:25:53.372 INFO:teuthology.orchestra.run.smithi175.stdout: osd map max advance = 10 2024-04-20T11:25:53.372 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-20T11:25:53.372 INFO:teuthology.orchestra.run.smithi175.stdout: journal zero on create = true 2024-04-20T11:25:53.372 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-20T11:25:53.372 INFO:teuthology.orchestra.run.smithi175.stdout: filestore ondisk finisher threads = 3 2024-04-20T11:25:53.372 INFO:teuthology.orchestra.run.smithi175.stdout: filestore apply finisher threads = 3 2024-04-20T11:25:53.372 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-20T11:25:53.372 INFO:teuthology.orchestra.run.smithi175.stdout: bdev debug aio = true 2024-04-20T11:25:53.372 INFO:teuthology.orchestra.run.smithi175.stdout: osd debug misdirected ops = true 2024-04-20T11:25:53.372 INFO:teuthology.orchestra.run.smithi175.stdout: bdev async discard = True 2024-04-20T11:25:53.372 INFO:teuthology.orchestra.run.smithi175.stdout: bdev enable discard = True 2024-04-20T11:25:53.372 INFO:teuthology.orchestra.run.smithi175.stdout: bluestore allocator = bitmap 2024-04-20T11:25:53.372 INFO:teuthology.orchestra.run.smithi175.stdout: bluestore block size = 96636764160 2024-04-20T11:25:53.372 INFO:teuthology.orchestra.run.smithi175.stdout: bluestore fsck on mount = True 2024-04-20T11:25:53.373 INFO:teuthology.orchestra.run.smithi175.stdout: debug bluefs = 1/20 2024-04-20T11:25:53.373 INFO:teuthology.orchestra.run.smithi175.stdout: debug bluestore = 1/20 2024-04-20T11:25:53.373 INFO:teuthology.orchestra.run.smithi175.stdout: debug ms = 1 2024-04-20T11:25:53.373 INFO:teuthology.orchestra.run.smithi175.stdout: debug osd = 20 2024-04-20T11:25:53.373 INFO:teuthology.orchestra.run.smithi175.stdout: debug rocksdb = 4/10 2024-04-20T11:25:53.373 INFO:teuthology.orchestra.run.smithi175.stdout: mon osd backfillfull_ratio = 0.85 2024-04-20T11:25:53.373 INFO:teuthology.orchestra.run.smithi175.stdout: mon osd full ratio = 0.9 2024-04-20T11:25:53.373 INFO:teuthology.orchestra.run.smithi175.stdout: mon osd nearfull ratio = 0.8 2024-04-20T11:25:53.373 INFO:teuthology.orchestra.run.smithi175.stdout: osd failsafe full ratio = 0.95 2024-04-20T11:25:53.373 INFO:teuthology.orchestra.run.smithi175.stdout: osd objectstore = bluestore 2024-04-20T11:25:53.373 INFO:teuthology.orchestra.run.smithi175.stdout: osd op complaint time = 180 2024-04-20T11:25:53.373 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-20T11:25:53.373 INFO:teuthology.orchestra.run.smithi175.stdout:[mgr] 2024-04-20T11:25:53.373 INFO:teuthology.orchestra.run.smithi175.stdout: debug ms = 1 2024-04-20T11:25:53.374 INFO:teuthology.orchestra.run.smithi175.stdout: debug mgr = 20 2024-04-20T11:25:53.374 INFO:teuthology.orchestra.run.smithi175.stdout: debug mon = 20 2024-04-20T11:25:53.374 INFO:teuthology.orchestra.run.smithi175.stdout: debug auth = 20 2024-04-20T11:25:53.374 INFO:teuthology.orchestra.run.smithi175.stdout: mon reweight min pgs per osd = 4 2024-04-20T11:25:53.374 INFO:teuthology.orchestra.run.smithi175.stdout: mon reweight min bytes per osd = 10 2024-04-20T11:25:53.374 INFO:teuthology.orchestra.run.smithi175.stdout: mgr/telemetry/nag = false 2024-04-20T11:25:53.374 INFO:teuthology.orchestra.run.smithi175.stdout: debug client = 20 2024-04-20T11:25:53.374 INFO:teuthology.orchestra.run.smithi175.stdout: debug finisher = 20 2024-04-20T11:25:53.374 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-20T11:25:53.374 INFO:teuthology.orchestra.run.smithi175.stdout:[mon] 2024-04-20T11:25:53.374 INFO:teuthology.orchestra.run.smithi175.stdout: debug ms = 1 2024-04-20T11:25:53.375 INFO:teuthology.orchestra.run.smithi175.stdout: debug mon = 20 2024-04-20T11:25:53.375 INFO:teuthology.orchestra.run.smithi175.stdout: debug paxos = 20 2024-04-20T11:25:53.375 INFO:teuthology.orchestra.run.smithi175.stdout: debug auth = 20 2024-04-20T11:25:53.375 INFO:teuthology.orchestra.run.smithi175.stdout: mon data avail warn = 5 2024-04-20T11:25:53.375 INFO:teuthology.orchestra.run.smithi175.stdout: mon mgr mkfs grace = 240 2024-04-20T11:25:53.375 INFO:teuthology.orchestra.run.smithi175.stdout: mon reweight min pgs per osd = 4 2024-04-20T11:25:53.375 INFO:teuthology.orchestra.run.smithi175.stdout: mon osd reporter subtree level = osd 2024-04-20T11:25:53.375 INFO:teuthology.orchestra.run.smithi175.stdout: mon osd prime pg temp = true 2024-04-20T11:25:53.375 INFO:teuthology.orchestra.run.smithi175.stdout: mon reweight min bytes per osd = 10 2024-04-20T11:25:53.375 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-20T11:25:53.375 INFO:teuthology.orchestra.run.smithi175.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-20T11:25:53.375 INFO:teuthology.orchestra.run.smithi175.stdout: auth mon ticket ttl = 660 # 11m 2024-04-20T11:25:53.375 INFO:teuthology.orchestra.run.smithi175.stdout: auth service ticket ttl = 240 # 4m 2024-04-20T11:25:53.375 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-20T11:25:53.375 INFO:teuthology.orchestra.run.smithi175.stdout: # don't complain about insecure global_id in the test suite 2024-04-20T11:25:53.376 INFO:teuthology.orchestra.run.smithi175.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-20T11:25:53.376 INFO:teuthology.orchestra.run.smithi175.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-20T11:25:53.376 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-20T11:25:53.376 INFO:teuthology.orchestra.run.smithi175.stdout: # 1m isn't quite enough 2024-04-20T11:25:53.376 INFO:teuthology.orchestra.run.smithi175.stdout: mon_down_mkfs_grace = 2m 2024-04-20T11:25:53.376 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-20T11:25:53.376 INFO:teuthology.orchestra.run.smithi175.stdout: mon_warn_on_filestore_osds = false 2024-04-20T11:25:53.376 INFO:teuthology.orchestra.run.smithi175.stdout: mon down mkfs grace = 300 2024-04-20T11:25:53.376 INFO:teuthology.orchestra.run.smithi175.stdout: mon op complaint time = 120 2024-04-20T11:25:53.376 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-20T11:25:53.376 INFO:teuthology.orchestra.run.smithi175.stdout:[client] 2024-04-20T11:25:53.376 INFO:teuthology.orchestra.run.smithi175.stdout: rgw cache enabled = true 2024-04-20T11:25:53.376 INFO:teuthology.orchestra.run.smithi175.stdout: rgw enable ops log = true 2024-04-20T11:25:53.376 INFO:teuthology.orchestra.run.smithi175.stdout: rgw enable usage log = true 2024-04-20T11:25:53.376 INFO:teuthology.orchestra.run.smithi175.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-20T11:25:53.377 INFO:teuthology.orchestra.run.smithi175.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-20T11:25:53.377 INFO:teuthology.orchestra.run.smithi175.stdout: client mount timeout = 600 2024-04-20T11:25:53.377 INFO:teuthology.orchestra.run.smithi175.stdout: debug client = 20 2024-04-20T11:25:53.377 INFO:teuthology.orchestra.run.smithi175.stdout: debug ms = 1 2024-04-20T11:25:53.377 INFO:teuthology.orchestra.run.smithi175.stdout: pid file = "" 2024-04-20T11:25:53.377 INFO:teuthology.orchestra.run.smithi175.stdout: rados mon op timeout = 900 2024-04-20T11:25:53.377 INFO:teuthology.orchestra.run.smithi175.stdout: rados osd op timeout = 900 2024-04-20T11:25:53.377 INFO:teuthology.orchestra.run.smithi175.stdout:[mon.a] 2024-04-20T11:25:53.377 INFO:teuthology.orchestra.run.smithi175.stdout:[mon.b] 2024-04-20T11:25:53.377 INFO:teuthology.orchestra.run.smithi175.stdout:[mon.c] 2024-04-20T11:25:53.377 INFO:teuthology.orchestra.run.smithi175.stdout:[mds.a] 2024-04-20T11:25:53.377 INFO:teuthology.orchestra.run.smithi175.stdout:[mds.c] 2024-04-20T11:25:53.377 INFO:teuthology.orchestra.run.smithi175.stdout:[mds.b] 2024-04-20T11:25:53.377 INFO:teuthology.orchestra.run.smithi175.stdout:[mds.d] 2024-04-20T11:25:53.378 INFO:teuthology.orchestra.run.smithi175.stdout:[mds] 2024-04-20T11:25:53.378 INFO:teuthology.orchestra.run.smithi175.stdout: debug mds = 20 2024-04-20T11:25:53.378 INFO:teuthology.orchestra.run.smithi175.stdout: debug mds balancer = 20 2024-04-20T11:25:53.378 INFO:teuthology.orchestra.run.smithi175.stdout: debug ms = 1 2024-04-20T11:25:53.378 INFO:teuthology.orchestra.run.smithi175.stdout: mds debug frag = True 2024-04-20T11:25:53.378 INFO:teuthology.orchestra.run.smithi175.stdout: mds debug scatterstat = True 2024-04-20T11:25:53.378 INFO:teuthology.orchestra.run.smithi175.stdout: mds op complaint time = 180 2024-04-20T11:25:53.378 INFO:teuthology.orchestra.run.smithi175.stdout: mds verify scatter = True 2024-04-20T11:25:53.378 INFO:teuthology.orchestra.run.smithi175.stdout: osd op complaint time = 180 2024-04-20T11:25:53.378 INFO:teuthology.orchestra.run.smithi175.stdout: rados mon op timeout = 900 2024-04-20T11:25:53.378 INFO:teuthology.orchestra.run.smithi175.stdout: rados osd op timeout = 900 2024-04-20T11:25:53.379 INFO:teuthology.orchestra.run.smithi152.stdout:[global] 2024-04-20T11:25:53.379 INFO:teuthology.orchestra.run.smithi152.stdout: chdir = "" 2024-04-20T11:25:53.379 INFO:teuthology.orchestra.run.smithi152.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-20T11:25:53.379 INFO:teuthology.orchestra.run.smithi152.stdout: auth supported = cephx 2024-04-20T11:25:53.380 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:25:53.380 INFO:teuthology.orchestra.run.smithi152.stdout: filestore xattr use omap = true 2024-04-20T11:25:53.380 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:25:53.380 INFO:teuthology.orchestra.run.smithi152.stdout: mon clock drift allowed = 1.000 2024-04-20T11:25:53.380 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:25:53.380 INFO:teuthology.orchestra.run.smithi152.stdout: osd crush chooseleaf type = 0 2024-04-20T11:25:53.380 INFO:teuthology.orchestra.run.smithi152.stdout: auth debug = true 2024-04-20T11:25:53.380 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:25:53.380 INFO:teuthology.orchestra.run.smithi152.stdout: ms die on old message = true 2024-04-20T11:25:53.380 INFO:teuthology.orchestra.run.smithi152.stdout: ms die on bug = true 2024-04-20T11:25:53.380 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:25:53.380 INFO:teuthology.orchestra.run.smithi152.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-20T11:25:53.380 INFO:teuthology.orchestra.run.smithi152.stdout: mon pg warn max object skew = 0 2024-04-20T11:25:53.380 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:25:53.380 INFO:teuthology.orchestra.run.smithi152.stdout: # disable pg_autoscaler by default for new pools 2024-04-20T11:25:53.381 INFO:teuthology.orchestra.run.smithi152.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-20T11:25:53.381 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:25:53.381 INFO:teuthology.orchestra.run.smithi152.stdout: osd pool default size = 2 2024-04-20T11:25:53.381 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:25:53.381 INFO:teuthology.orchestra.run.smithi152.stdout: mon osd allow primary affinity = true 2024-04-20T11:25:53.381 INFO:teuthology.orchestra.run.smithi152.stdout: mon osd allow pg remap = true 2024-04-20T11:25:53.381 INFO:teuthology.orchestra.run.smithi152.stdout: mon warn on legacy crush tunables = false 2024-04-20T11:25:53.381 INFO:teuthology.orchestra.run.smithi152.stdout: mon warn on crush straw calc version zero = false 2024-04-20T11:25:53.381 INFO:teuthology.orchestra.run.smithi152.stdout: mon warn on no sortbitwise = false 2024-04-20T11:25:53.381 INFO:teuthology.orchestra.run.smithi152.stdout: mon warn on osd down out interval zero = false 2024-04-20T11:25:53.381 INFO:teuthology.orchestra.run.smithi152.stdout: mon warn on too few osds = false 2024-04-20T11:25:53.381 INFO:teuthology.orchestra.run.smithi152.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-20T11:25:53.381 INFO:teuthology.orchestra.run.smithi152.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-20T11:25:53.382 INFO:teuthology.orchestra.run.smithi152.stdout: mon_allow_pool_size_one = true 2024-04-20T11:25:53.382 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:25:53.382 INFO:teuthology.orchestra.run.smithi152.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-20T11:25:53.382 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:25:53.382 INFO:teuthology.orchestra.run.smithi152.stdout: osd default data pool replay window = 5 2024-04-20T11:25:53.382 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:25:53.382 INFO:teuthology.orchestra.run.smithi152.stdout: mon allow pool delete = true 2024-04-20T11:25:53.382 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:25:53.382 INFO:teuthology.orchestra.run.smithi152.stdout: mon cluster log file level = debug 2024-04-20T11:25:53.382 INFO:teuthology.orchestra.run.smithi152.stdout: debug asserts on shutdown = true 2024-04-20T11:25:53.382 INFO:teuthology.orchestra.run.smithi152.stdout: mon health detail to clog = false 2024-04-20T11:25:53.382 INFO:teuthology.orchestra.run.smithi152.stdout: mon host = "172.21.15.152,172.21.15.175,[v2:172.21.15.175:3301,v1:172.21.15.175:6790]" 2024-04-20T11:25:53.382 INFO:teuthology.orchestra.run.smithi152.stdout: fsid = c8a72131-0d4f-42c7-892f-3f3af4d2bd3d 2024-04-20T11:25:53.382 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:25:53.383 INFO:teuthology.orchestra.run.smithi152.stdout:[osd] 2024-04-20T11:25:53.383 INFO:teuthology.orchestra.run.smithi152.stdout: osd journal size = 100 2024-04-20T11:25:53.383 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:25:53.383 INFO:teuthology.orchestra.run.smithi152.stdout: osd scrub load threshold = 5.0 2024-04-20T11:25:53.383 INFO:teuthology.orchestra.run.smithi152.stdout: osd scrub max interval = 600 2024-04-20T11:25:53.383 INFO:teuthology.orchestra.run.smithi152.stdout: osd mclock profile = high_recovery_ops 2024-04-20T11:25:53.383 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:25:53.383 INFO:teuthology.orchestra.run.smithi152.stdout: osd recover clone overlap = true 2024-04-20T11:25:53.383 INFO:teuthology.orchestra.run.smithi152.stdout: osd recovery max chunk = 1048576 2024-04-20T11:25:53.383 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:25:53.383 INFO:teuthology.orchestra.run.smithi152.stdout: osd debug shutdown = true 2024-04-20T11:25:53.383 INFO:teuthology.orchestra.run.smithi152.stdout: osd debug op order = true 2024-04-20T11:25:53.383 INFO:teuthology.orchestra.run.smithi152.stdout: osd debug verify stray on activate = true 2024-04-20T11:25:53.383 INFO:teuthology.orchestra.run.smithi152.stdout: osd debug trim objects = true 2024-04-20T11:25:53.383 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:25:53.384 INFO:teuthology.orchestra.run.smithi152.stdout: osd open classes on start = true 2024-04-20T11:25:53.384 INFO:teuthology.orchestra.run.smithi152.stdout: osd debug pg log writeout = true 2024-04-20T11:25:53.384 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:25:53.384 INFO:teuthology.orchestra.run.smithi152.stdout: osd deep scrub update digest min age = 30 2024-04-20T11:25:53.384 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:25:53.384 INFO:teuthology.orchestra.run.smithi152.stdout: osd map max advance = 10 2024-04-20T11:25:53.384 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:25:53.384 INFO:teuthology.orchestra.run.smithi152.stdout: journal zero on create = true 2024-04-20T11:25:53.384 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:25:53.384 INFO:teuthology.orchestra.run.smithi152.stdout: filestore ondisk finisher threads = 3 2024-04-20T11:25:53.384 INFO:teuthology.orchestra.run.smithi152.stdout: filestore apply finisher threads = 3 2024-04-20T11:25:53.384 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:25:53.384 INFO:teuthology.orchestra.run.smithi152.stdout: bdev debug aio = true 2024-04-20T11:25:53.384 INFO:teuthology.orchestra.run.smithi152.stdout: osd debug misdirected ops = true 2024-04-20T11:25:53.384 INFO:teuthology.orchestra.run.smithi152.stdout: bdev async discard = True 2024-04-20T11:25:53.385 INFO:teuthology.orchestra.run.smithi152.stdout: bdev enable discard = True 2024-04-20T11:25:53.385 INFO:teuthology.orchestra.run.smithi152.stdout: bluestore allocator = bitmap 2024-04-20T11:25:53.385 INFO:teuthology.orchestra.run.smithi152.stdout: bluestore block size = 96636764160 2024-04-20T11:25:53.385 INFO:teuthology.orchestra.run.smithi152.stdout: bluestore fsck on mount = True 2024-04-20T11:25:53.385 INFO:teuthology.orchestra.run.smithi152.stdout: debug bluefs = 1/20 2024-04-20T11:25:53.385 INFO:teuthology.orchestra.run.smithi152.stdout: debug bluestore = 1/20 2024-04-20T11:25:53.385 INFO:teuthology.orchestra.run.smithi152.stdout: debug ms = 1 2024-04-20T11:25:53.385 INFO:teuthology.orchestra.run.smithi152.stdout: debug osd = 20 2024-04-20T11:25:53.385 INFO:teuthology.orchestra.run.smithi152.stdout: debug rocksdb = 4/10 2024-04-20T11:25:53.385 INFO:teuthology.orchestra.run.smithi152.stdout: mon osd backfillfull_ratio = 0.85 2024-04-20T11:25:53.385 INFO:teuthology.orchestra.run.smithi152.stdout: mon osd full ratio = 0.9 2024-04-20T11:25:53.385 INFO:teuthology.orchestra.run.smithi152.stdout: mon osd nearfull ratio = 0.8 2024-04-20T11:25:53.385 INFO:teuthology.orchestra.run.smithi152.stdout: osd failsafe full ratio = 0.95 2024-04-20T11:25:53.385 INFO:teuthology.orchestra.run.smithi152.stdout: osd objectstore = bluestore 2024-04-20T11:25:53.386 INFO:teuthology.orchestra.run.smithi152.stdout: osd op complaint time = 180 2024-04-20T11:25:53.386 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:25:53.386 INFO:teuthology.orchestra.run.smithi152.stdout:[mgr] 2024-04-20T11:25:53.386 INFO:teuthology.orchestra.run.smithi152.stdout: debug ms = 1 2024-04-20T11:25:53.386 INFO:teuthology.orchestra.run.smithi152.stdout: debug mgr = 20 2024-04-20T11:25:53.386 INFO:teuthology.orchestra.run.smithi152.stdout: debug mon = 20 2024-04-20T11:25:53.386 INFO:teuthology.orchestra.run.smithi152.stdout: debug auth = 20 2024-04-20T11:25:53.386 INFO:teuthology.orchestra.run.smithi152.stdout: mon reweight min pgs per osd = 4 2024-04-20T11:25:53.386 INFO:teuthology.orchestra.run.smithi152.stdout: mon reweight min bytes per osd = 10 2024-04-20T11:25:53.386 INFO:teuthology.orchestra.run.smithi152.stdout: mgr/telemetry/nag = false 2024-04-20T11:25:53.386 INFO:teuthology.orchestra.run.smithi152.stdout: debug client = 20 2024-04-20T11:25:53.386 INFO:teuthology.orchestra.run.smithi152.stdout: debug finisher = 20 2024-04-20T11:25:53.386 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:25:53.386 INFO:teuthology.orchestra.run.smithi152.stdout:[mon] 2024-04-20T11:25:53.386 INFO:teuthology.orchestra.run.smithi152.stdout: debug ms = 1 2024-04-20T11:25:53.387 INFO:teuthology.orchestra.run.smithi152.stdout: debug mon = 20 2024-04-20T11:25:53.387 INFO:teuthology.orchestra.run.smithi152.stdout: debug paxos = 20 2024-04-20T11:25:53.387 INFO:teuthology.orchestra.run.smithi152.stdout: debug auth = 20 2024-04-20T11:25:53.387 INFO:teuthology.orchestra.run.smithi152.stdout: mon data avail warn = 5 2024-04-20T11:25:53.387 INFO:teuthology.orchestra.run.smithi152.stdout: mon mgr mkfs grace = 240 2024-04-20T11:25:53.387 INFO:teuthology.orchestra.run.smithi152.stdout: mon reweight min pgs per osd = 4 2024-04-20T11:25:53.387 INFO:teuthology.orchestra.run.smithi152.stdout: mon osd reporter subtree level = osd 2024-04-20T11:25:53.387 INFO:teuthology.orchestra.run.smithi152.stdout: mon osd prime pg temp = true 2024-04-20T11:25:53.387 INFO:teuthology.orchestra.run.smithi152.stdout: mon reweight min bytes per osd = 10 2024-04-20T11:25:53.387 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:25:53.387 INFO:teuthology.orchestra.run.smithi152.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-20T11:25:53.387 INFO:teuthology.orchestra.run.smithi152.stdout: auth mon ticket ttl = 660 # 11m 2024-04-20T11:25:53.387 INFO:teuthology.orchestra.run.smithi152.stdout: auth service ticket ttl = 240 # 4m 2024-04-20T11:25:53.387 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:25:53.388 INFO:teuthology.orchestra.run.smithi152.stdout: # don't complain about insecure global_id in the test suite 2024-04-20T11:25:53.388 INFO:teuthology.orchestra.run.smithi152.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-20T11:25:53.388 INFO:teuthology.orchestra.run.smithi152.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-20T11:25:53.388 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:25:53.388 INFO:teuthology.orchestra.run.smithi152.stdout: # 1m isn't quite enough 2024-04-20T11:25:53.388 INFO:teuthology.orchestra.run.smithi152.stdout: mon_down_mkfs_grace = 2m 2024-04-20T11:25:53.388 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:25:53.388 INFO:teuthology.orchestra.run.smithi152.stdout: mon_warn_on_filestore_osds = false 2024-04-20T11:25:53.388 INFO:teuthology.orchestra.run.smithi152.stdout: mon down mkfs grace = 300 2024-04-20T11:25:53.388 INFO:teuthology.orchestra.run.smithi152.stdout: mon op complaint time = 120 2024-04-20T11:25:53.388 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:25:53.388 INFO:teuthology.orchestra.run.smithi152.stdout:[client] 2024-04-20T11:25:53.388 INFO:teuthology.orchestra.run.smithi152.stdout: rgw cache enabled = true 2024-04-20T11:25:53.388 INFO:teuthology.orchestra.run.smithi152.stdout: rgw enable ops log = true 2024-04-20T11:25:53.388 INFO:teuthology.orchestra.run.smithi152.stdout: rgw enable usage log = true 2024-04-20T11:25:53.389 INFO:teuthology.orchestra.run.smithi152.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-20T11:25:53.389 INFO:teuthology.orchestra.run.smithi152.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-20T11:25:53.389 INFO:teuthology.orchestra.run.smithi152.stdout: client mount timeout = 600 2024-04-20T11:25:53.389 INFO:teuthology.orchestra.run.smithi152.stdout: debug client = 20 2024-04-20T11:25:53.389 INFO:teuthology.orchestra.run.smithi152.stdout: debug ms = 1 2024-04-20T11:25:53.389 INFO:teuthology.orchestra.run.smithi152.stdout: pid file = "" 2024-04-20T11:25:53.389 INFO:teuthology.orchestra.run.smithi152.stdout: rados mon op timeout = 900 2024-04-20T11:25:53.389 INFO:teuthology.orchestra.run.smithi152.stdout: rados osd op timeout = 900 2024-04-20T11:25:53.389 INFO:teuthology.orchestra.run.smithi152.stdout:[mon.a] 2024-04-20T11:25:53.389 INFO:teuthology.orchestra.run.smithi152.stdout:[mon.b] 2024-04-20T11:25:53.389 INFO:teuthology.orchestra.run.smithi152.stdout:[mon.c] 2024-04-20T11:25:53.389 INFO:teuthology.orchestra.run.smithi152.stdout:[mds.a] 2024-04-20T11:25:53.389 INFO:teuthology.orchestra.run.smithi152.stdout:[mds.c] 2024-04-20T11:25:53.389 INFO:teuthology.orchestra.run.smithi152.stdout:[mds.b] 2024-04-20T11:25:53.389 INFO:teuthology.orchestra.run.smithi152.stdout:[mds.d] 2024-04-20T11:25:53.390 INFO:teuthology.orchestra.run.smithi152.stdout:[mds] 2024-04-20T11:25:53.390 INFO:teuthology.orchestra.run.smithi152.stdout: debug mds = 20 2024-04-20T11:25:53.390 INFO:teuthology.orchestra.run.smithi152.stdout: debug mds balancer = 20 2024-04-20T11:25:53.390 INFO:teuthology.orchestra.run.smithi152.stdout: debug ms = 1 2024-04-20T11:25:53.390 INFO:teuthology.orchestra.run.smithi152.stdout: mds debug frag = True 2024-04-20T11:25:53.390 INFO:teuthology.orchestra.run.smithi152.stdout: mds debug scatterstat = True 2024-04-20T11:25:53.390 INFO:teuthology.orchestra.run.smithi152.stdout: mds op complaint time = 180 2024-04-20T11:25:53.390 INFO:teuthology.orchestra.run.smithi152.stdout: mds verify scatter = True 2024-04-20T11:25:53.390 INFO:teuthology.orchestra.run.smithi152.stdout: osd op complaint time = 180 2024-04-20T11:25:53.390 INFO:teuthology.orchestra.run.smithi152.stdout: rados mon op timeout = 900 2024-04-20T11:25:53.390 INFO:teuthology.orchestra.run.smithi152.stdout: rados osd op timeout = 900 2024-04-20T11:25:53.412 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-20T11:25:53.413 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-20T11:25:53.525 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-20T11:25:53.525 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-20T11:25:53.525 DEBUG:teuthology.orchestra.run.smithi152:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-20T11:25:53.571 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-20T11:25:53.571 DEBUG:teuthology.orchestra.run.smithi152:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-20T11:25:53.664 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi152.front.sepia.ceph.com 2024-04-20T11:25:53.665 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-20T11:25:53.665 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-20T11:25:53.665 DEBUG:teuthology.orchestra.run.smithi152:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-20T11:25:53.783 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-20T11:25:53.783 DEBUG:teuthology.orchestra.run.smithi152:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-20T11:25:53.877 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi175.front.sepia.ceph.com 2024-04-20T11:25:53.877 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T11:25:53.877 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-20T11:25:53.877 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-20T11:25:53.983 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T11:25:53.983 DEBUG:teuthology.orchestra.run.smithi175:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-20T11:25:54.076 INFO:tasks.ceph:Setting up mon nodes... 2024-04-20T11:25:54.077 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-20T11:25:54.077 DEBUG:teuthology.orchestra.run.smithi152:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-04-20T11:25:54.229 INFO:teuthology.orchestra.run.smithi152.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-04-20T11:25:54.237 DEBUG:teuthology.orchestra.run.smithi175:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-20T11:25:54.395 INFO:teuthology.orchestra.run.smithi175.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-20T11:25:54.405 INFO:tasks.ceph:Setting up mds nodes... 2024-04-20T11:25:54.405 DEBUG:teuthology.orchestra.run.smithi152:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-04-20T11:25:54.540 INFO:teuthology.orchestra.run.smithi152.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-04-20T11:25:54.546 DEBUG:teuthology.orchestra.run.smithi152:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-04-20T11:25:54.614 DEBUG:teuthology.orchestra.run.smithi152:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-04-20T11:25:54.763 INFO:teuthology.orchestra.run.smithi152.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-04-20T11:25:54.769 DEBUG:teuthology.orchestra.run.smithi152:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-04-20T11:25:54.841 DEBUG:teuthology.orchestra.run.smithi175:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-04-20T11:25:54.983 INFO:teuthology.orchestra.run.smithi175.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-04-20T11:25:54.989 DEBUG:teuthology.orchestra.run.smithi175:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-04-20T11:25:55.067 DEBUG:teuthology.orchestra.run.smithi175:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-04-20T11:25:55.189 INFO:teuthology.orchestra.run.smithi175.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-04-20T11:25:55.195 DEBUG:teuthology.orchestra.run.smithi175:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-04-20T11:25:55.260 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-20T11:25:55.260 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-20T11:25:55.373 INFO:teuthology.orchestra.run.smithi152.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-20T11:25:55.405 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-04-20T11:25:55.500 INFO:teuthology.orchestra.run.smithi152.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-04-20T11:25:55.542 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-04-20T11:25:55.659 INFO:teuthology.orchestra.run.smithi175.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-04-20T11:25:55.697 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-04-20T11:25:55.809 INFO:teuthology.orchestra.run.smithi175.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-04-20T11:25:55.846 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-20T11:25:55.846 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi152.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi175.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-20T11:25:55.846 DEBUG:teuthology.orchestra.run.smithi152:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-20T11:25:55.934 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-20T11:25:55.935 INFO:tasks.ceph:role: osd.0 2024-04-20T11:25:55.935 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi152.front.sepia.ceph.com 2024-04-20T11:25:55.935 DEBUG:teuthology.orchestra.run.smithi152:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-20T11:25:56.024 INFO:teuthology.orchestra.run.smithi152.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-20T11:25:56.024 INFO:teuthology.orchestra.run.smithi152.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-20T11:25:56.024 INFO:teuthology.orchestra.run.smithi152.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-20T11:25:56.024 INFO:teuthology.orchestra.run.smithi152.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-20T11:25:56.024 INFO:teuthology.orchestra.run.smithi152.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-20T11:25:56.024 INFO:teuthology.orchestra.run.smithi152.stdout: = sunit=0 swidth=0 blks 2024-04-20T11:25:56.024 INFO:teuthology.orchestra.run.smithi152.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-20T11:25:56.024 INFO:teuthology.orchestra.run.smithi152.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-20T11:25:56.025 INFO:teuthology.orchestra.run.smithi152.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-20T11:25:56.025 INFO:teuthology.orchestra.run.smithi152.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-20T11:25:56.440 INFO:teuthology.orchestra.run.smithi152.stdout:Discarding blocks...Done. 2024-04-20T11:25:56.445 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi152.front.sepia.ceph.com -o noatime 2024-04-20T11:25:56.446 DEBUG:teuthology.orchestra.run.smithi152:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-20T11:25:56.606 DEBUG:teuthology.orchestra.run.smithi152:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-20T11:25:56.705 DEBUG:teuthology.orchestra.run.smithi152:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-20T11:25:56.789 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-20T11:25:56.789 INFO:tasks.ceph:role: osd.1 2024-04-20T11:25:56.789 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi152.front.sepia.ceph.com 2024-04-20T11:25:56.789 DEBUG:teuthology.orchestra.run.smithi152:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-20T11:25:56.877 INFO:teuthology.orchestra.run.smithi152.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-20T11:25:56.877 INFO:teuthology.orchestra.run.smithi152.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-20T11:25:56.877 INFO:teuthology.orchestra.run.smithi152.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-20T11:25:56.878 INFO:teuthology.orchestra.run.smithi152.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-20T11:25:56.878 INFO:teuthology.orchestra.run.smithi152.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-20T11:25:56.878 INFO:teuthology.orchestra.run.smithi152.stdout: = sunit=0 swidth=0 blks 2024-04-20T11:25:56.878 INFO:teuthology.orchestra.run.smithi152.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-20T11:25:56.878 INFO:teuthology.orchestra.run.smithi152.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-20T11:25:56.878 INFO:teuthology.orchestra.run.smithi152.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-20T11:25:56.878 INFO:teuthology.orchestra.run.smithi152.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-20T11:25:57.234 INFO:teuthology.orchestra.run.smithi152.stdout:Discarding blocks...Done. 2024-04-20T11:25:57.239 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi152.front.sepia.ceph.com -o noatime 2024-04-20T11:25:57.239 DEBUG:teuthology.orchestra.run.smithi152:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-04-20T11:25:57.369 DEBUG:teuthology.orchestra.run.smithi152:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-20T11:25:57.460 DEBUG:teuthology.orchestra.run.smithi152:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-20T11:25:57.540 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-20T11:25:57.540 INFO:tasks.ceph:role: osd.2 2024-04-20T11:25:57.540 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi152.front.sepia.ceph.com 2024-04-20T11:25:57.541 DEBUG:teuthology.orchestra.run.smithi152:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-20T11:25:57.629 INFO:teuthology.orchestra.run.smithi152.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-20T11:25:57.629 INFO:teuthology.orchestra.run.smithi152.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-20T11:25:57.629 INFO:teuthology.orchestra.run.smithi152.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-20T11:25:57.629 INFO:teuthology.orchestra.run.smithi152.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-20T11:25:57.629 INFO:teuthology.orchestra.run.smithi152.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-20T11:25:57.630 INFO:teuthology.orchestra.run.smithi152.stdout: = sunit=0 swidth=0 blks 2024-04-20T11:25:57.630 INFO:teuthology.orchestra.run.smithi152.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-20T11:25:57.630 INFO:teuthology.orchestra.run.smithi152.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-20T11:25:57.630 INFO:teuthology.orchestra.run.smithi152.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-20T11:25:57.630 INFO:teuthology.orchestra.run.smithi152.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-20T11:25:58.017 INFO:teuthology.orchestra.run.smithi152.stdout:Discarding blocks...Done. 2024-04-20T11:25:58.023 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi152.front.sepia.ceph.com -o noatime 2024-04-20T11:25:58.023 DEBUG:teuthology.orchestra.run.smithi152:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-04-20T11:25:58.166 DEBUG:teuthology.orchestra.run.smithi152:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-20T11:25:58.252 DEBUG:teuthology.orchestra.run.smithi152:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-20T11:25:58.330 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-20T11:25:58.331 INFO:tasks.ceph:role: osd.3 2024-04-20T11:25:58.331 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi152.front.sepia.ceph.com 2024-04-20T11:25:58.331 DEBUG:teuthology.orchestra.run.smithi152:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-20T11:25:58.417 INFO:teuthology.orchestra.run.smithi152.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-20T11:25:58.417 INFO:teuthology.orchestra.run.smithi152.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-20T11:25:58.417 INFO:teuthology.orchestra.run.smithi152.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-20T11:25:58.417 INFO:teuthology.orchestra.run.smithi152.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-20T11:25:58.417 INFO:teuthology.orchestra.run.smithi152.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-20T11:25:58.417 INFO:teuthology.orchestra.run.smithi152.stdout: = sunit=0 swidth=0 blks 2024-04-20T11:25:58.417 INFO:teuthology.orchestra.run.smithi152.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-20T11:25:58.417 INFO:teuthology.orchestra.run.smithi152.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-20T11:25:58.418 INFO:teuthology.orchestra.run.smithi152.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-20T11:25:58.418 INFO:teuthology.orchestra.run.smithi152.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-20T11:25:58.833 INFO:teuthology.orchestra.run.smithi152.stdout:Discarding blocks...Done. 2024-04-20T11:25:58.840 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi152.front.sepia.ceph.com -o noatime 2024-04-20T11:25:58.840 DEBUG:teuthology.orchestra.run.smithi152:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-04-20T11:25:58.978 DEBUG:teuthology.orchestra.run.smithi152:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-20T11:25:59.064 DEBUG:teuthology.orchestra.run.smithi152:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-20T11:25:59.207 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:25:59.208+0000 7febac5dc640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-20T11:25:59.208 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:25:59.208+0000 7febac5dc640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-20T11:25:59.208 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:25:59.208+0000 7febac5dc640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-20T11:25:59.208 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:25:59.208+0000 7febac5dc640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-20T11:25:59.208 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:25:59.208+0000 7febac5dc640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-20T11:25:59.212 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:25:59.212+0000 7febac5dc640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-04-20T11:26:02.426 DEBUG:teuthology.orchestra.run.smithi152:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-20T11:26:02.514 DEBUG:teuthology.orchestra.run.smithi152:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-20T11:26:02.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:02.645+0000 7f554b599640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-20T11:26:02.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:02.646+0000 7f554b599640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-20T11:26:02.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:02.646+0000 7f554b599640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-20T11:26:02.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:02.646+0000 7f554b599640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-20T11:26:02.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:02.646+0000 7f554b599640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-20T11:26:02.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:02.649+0000 7f554b599640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-04-20T11:26:05.627 DEBUG:teuthology.orchestra.run.smithi152:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-20T11:26:05.711 DEBUG:teuthology.orchestra.run.smithi152:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-20T11:26:05.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:05.841+0000 7fc3cd933640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-20T11:26:05.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:05.842+0000 7fc3cd933640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-20T11:26:05.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:05.842+0000 7fc3cd933640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-20T11:26:05.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:05.842+0000 7fc3cd933640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-20T11:26:05.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:05.842+0000 7fc3cd933640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-20T11:26:05.844 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:05.844+0000 7fc3cd933640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-04-20T11:26:08.831 DEBUG:teuthology.orchestra.run.smithi152:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-20T11:26:08.916 DEBUG:teuthology.orchestra.run.smithi152:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-20T11:26:09.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:09.042+0000 7f49a8405640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-04-20T11:26:09.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:09.043+0000 7f49a8405640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-04-20T11:26:09.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:09.043+0000 7f49a8405640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-20T11:26:09.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:09.043+0000 7f49a8405640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-20T11:26:09.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:09.043+0000 7f49a8405640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-20T11:26:09.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:09.046+0000 7f49a8405640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-04-20T11:26:11.990 DEBUG:teuthology.orchestra.run.smithi152:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-20T11:26:12.072 DEBUG:teuthology.orchestra.run.smithi175:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-04-20T11:26:12.155 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-20T11:26:12.155 INFO:tasks.ceph:role: osd.4 2024-04-20T11:26:12.155 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi175.front.sepia.ceph.com 2024-04-20T11:26:12.155 DEBUG:teuthology.orchestra.run.smithi175:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-20T11:26:12.242 INFO:teuthology.orchestra.run.smithi175.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-20T11:26:12.243 INFO:teuthology.orchestra.run.smithi175.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-20T11:26:12.243 INFO:teuthology.orchestra.run.smithi175.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-20T11:26:12.243 INFO:teuthology.orchestra.run.smithi175.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-20T11:26:12.243 INFO:teuthology.orchestra.run.smithi175.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-20T11:26:12.243 INFO:teuthology.orchestra.run.smithi175.stdout: = sunit=0 swidth=0 blks 2024-04-20T11:26:12.243 INFO:teuthology.orchestra.run.smithi175.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-20T11:26:12.243 INFO:teuthology.orchestra.run.smithi175.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-20T11:26:12.243 INFO:teuthology.orchestra.run.smithi175.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-20T11:26:12.243 INFO:teuthology.orchestra.run.smithi175.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-20T11:26:12.658 INFO:teuthology.orchestra.run.smithi175.stdout:Discarding blocks...Done. 2024-04-20T11:26:12.664 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi175.front.sepia.ceph.com -o noatime 2024-04-20T11:26:12.664 DEBUG:teuthology.orchestra.run.smithi175:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-04-20T11:26:12.820 DEBUG:teuthology.orchestra.run.smithi175:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-04-20T11:26:12.915 DEBUG:teuthology.orchestra.run.smithi175:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-04-20T11:26:12.995 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-20T11:26:12.995 INFO:tasks.ceph:role: osd.5 2024-04-20T11:26:12.995 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi175.front.sepia.ceph.com 2024-04-20T11:26:12.995 DEBUG:teuthology.orchestra.run.smithi175:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-20T11:26:13.081 INFO:teuthology.orchestra.run.smithi175.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-20T11:26:13.081 INFO:teuthology.orchestra.run.smithi175.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-20T11:26:13.081 INFO:teuthology.orchestra.run.smithi175.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-20T11:26:13.081 INFO:teuthology.orchestra.run.smithi175.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-20T11:26:13.081 INFO:teuthology.orchestra.run.smithi175.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-20T11:26:13.081 INFO:teuthology.orchestra.run.smithi175.stdout: = sunit=0 swidth=0 blks 2024-04-20T11:26:13.081 INFO:teuthology.orchestra.run.smithi175.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-20T11:26:13.081 INFO:teuthology.orchestra.run.smithi175.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-20T11:26:13.081 INFO:teuthology.orchestra.run.smithi175.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-20T11:26:13.082 INFO:teuthology.orchestra.run.smithi175.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-20T11:26:13.438 INFO:teuthology.orchestra.run.smithi175.stdout:Discarding blocks...Done. 2024-04-20T11:26:13.444 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi175.front.sepia.ceph.com -o noatime 2024-04-20T11:26:13.444 DEBUG:teuthology.orchestra.run.smithi175:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-04-20T11:26:13.565 DEBUG:teuthology.orchestra.run.smithi175:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-04-20T11:26:13.640 DEBUG:teuthology.orchestra.run.smithi175:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-04-20T11:26:13.719 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-20T11:26:13.719 INFO:tasks.ceph:role: osd.6 2024-04-20T11:26:13.719 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi175.front.sepia.ceph.com 2024-04-20T11:26:13.719 DEBUG:teuthology.orchestra.run.smithi175:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-20T11:26:13.809 INFO:teuthology.orchestra.run.smithi175.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-20T11:26:13.809 INFO:teuthology.orchestra.run.smithi175.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-20T11:26:13.809 INFO:teuthology.orchestra.run.smithi175.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-20T11:26:13.810 INFO:teuthology.orchestra.run.smithi175.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-20T11:26:13.810 INFO:teuthology.orchestra.run.smithi175.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-20T11:26:13.810 INFO:teuthology.orchestra.run.smithi175.stdout: = sunit=0 swidth=0 blks 2024-04-20T11:26:13.810 INFO:teuthology.orchestra.run.smithi175.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-20T11:26:13.810 INFO:teuthology.orchestra.run.smithi175.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-20T11:26:13.810 INFO:teuthology.orchestra.run.smithi175.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-20T11:26:13.810 INFO:teuthology.orchestra.run.smithi175.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-20T11:26:14.202 INFO:teuthology.orchestra.run.smithi175.stdout:Discarding blocks...Done. 2024-04-20T11:26:14.210 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi175.front.sepia.ceph.com -o noatime 2024-04-20T11:26:14.210 DEBUG:teuthology.orchestra.run.smithi175:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-04-20T11:26:14.362 DEBUG:teuthology.orchestra.run.smithi175:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-04-20T11:26:14.460 DEBUG:teuthology.orchestra.run.smithi175:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-04-20T11:26:14.545 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-20T11:26:14.545 INFO:tasks.ceph:role: osd.7 2024-04-20T11:26:14.545 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi175.front.sepia.ceph.com 2024-04-20T11:26:14.545 DEBUG:teuthology.orchestra.run.smithi175:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-20T11:26:14.634 INFO:teuthology.orchestra.run.smithi175.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-20T11:26:14.634 INFO:teuthology.orchestra.run.smithi175.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-20T11:26:14.635 INFO:teuthology.orchestra.run.smithi175.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-20T11:26:14.635 INFO:teuthology.orchestra.run.smithi175.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-20T11:26:14.635 INFO:teuthology.orchestra.run.smithi175.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-20T11:26:14.635 INFO:teuthology.orchestra.run.smithi175.stdout: = sunit=0 swidth=0 blks 2024-04-20T11:26:14.635 INFO:teuthology.orchestra.run.smithi175.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-20T11:26:14.635 INFO:teuthology.orchestra.run.smithi175.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-20T11:26:14.635 INFO:teuthology.orchestra.run.smithi175.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-20T11:26:14.635 INFO:teuthology.orchestra.run.smithi175.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-20T11:26:15.040 INFO:teuthology.orchestra.run.smithi175.stdout:Discarding blocks...Done. 2024-04-20T11:26:15.047 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi175.front.sepia.ceph.com -o noatime 2024-04-20T11:26:15.047 DEBUG:teuthology.orchestra.run.smithi175:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-04-20T11:26:15.171 DEBUG:teuthology.orchestra.run.smithi175:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-04-20T11:26:15.248 DEBUG:teuthology.orchestra.run.smithi175:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-20T11:26:15.386 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:15.386+0000 7efd57c9c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-04-20T11:26:15.387 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:15.387+0000 7efd57c9c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-04-20T11:26:15.387 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:15.387+0000 7efd57c9c640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-20T11:26:15.387 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:15.387+0000 7efd57c9c640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-20T11:26:15.387 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:15.387+0000 7efd57c9c640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-20T11:26:15.390 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:15.390+0000 7efd57c9c640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-04-20T11:26:18.615 DEBUG:teuthology.orchestra.run.smithi175:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-20T11:26:18.700 DEBUG:teuthology.orchestra.run.smithi175:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-20T11:26:18.825 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:18.826+0000 7ff6cff87640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-04-20T11:26:18.826 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:18.827+0000 7ff6cff87640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-04-20T11:26:18.826 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:18.827+0000 7ff6cff87640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-20T11:26:18.827 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:18.827+0000 7ff6cff87640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-20T11:26:18.827 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:18.827+0000 7ff6cff87640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-20T11:26:18.828 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:18.830+0000 7ff6cff87640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-04-20T11:26:20.647 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:26:20.655 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:26:22.051 DEBUG:teuthology.orchestra.run.smithi175:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-20T11:26:22.136 DEBUG:teuthology.orchestra.run.smithi175:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-20T11:26:22.265 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:22.265+0000 7fec9a01f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-04-20T11:26:22.265 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:22.266+0000 7fec9a01f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-04-20T11:26:22.265 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:22.266+0000 7fec9a01f640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-20T11:26:22.265 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:22.266+0000 7fec9a01f640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-20T11:26:22.265 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:22.266+0000 7fec9a01f640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-20T11:26:22.267 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:22.268+0000 7fec9a01f640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-04-20T11:26:25.270 DEBUG:teuthology.orchestra.run.smithi175:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-04-20T11:26:25.355 DEBUG:teuthology.orchestra.run.smithi175:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-20T11:26:25.484 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:25.485+0000 7f041b007640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-04-20T11:26:25.485 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:25.486+0000 7f041b007640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-04-20T11:26:25.485 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:25.486+0000 7f041b007640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-20T11:26:25.485 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:25.486+0000 7f041b007640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-20T11:26:25.485 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:25.486+0000 7f041b007640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-20T11:26:25.487 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:25.488+0000 7f041b007640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-04-20T11:26:28.716 DEBUG:teuthology.orchestra.run.smithi175:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-04-20T11:26:28.797 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-20T11:26:28.797 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-20T11:26:28.797 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-04-20T11:26:28.889 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-20T11:26:28.889 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-04-20T11:26:28.975 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-20T11:26:28.975 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-04-20T11:26:29.058 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-20T11:26:29.058 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-20T11:26:29.145 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-20T11:26:29.145 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-20T11:26:29.230 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-20T11:26:29.230 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-20T11:26:29.312 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-20T11:26:29.312 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-20T11:26:29.395 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T11:26:29.396 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-20T11:26:29.484 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T11:26:29.484 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-04-20T11:26:29.567 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T11:26:29.568 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-04-20T11:26:29.654 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T11:26:29.654 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-04-20T11:26:29.738 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T11:26:29.738 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-04-20T11:26:29.819 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T11:26:29.819 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-04-20T11:26:29.904 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T11:26:29.904 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-04-20T11:26:29.985 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-20T11:26:29.985 DEBUG:teuthology.orchestra.run.smithi152:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-20T11:26:30.041 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-20T11:26:30.041 DEBUG:teuthology.orchestra.run.smithi152:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-04-20T11:26:30.094 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T11:26:30.094 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-04-20T11:26:30.153 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T11:26:30.154 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-04-20T11:26:30.207 INFO:tasks.ceph:Adding keys to all mons... 2024-04-20T11:26:30.207 DEBUG:teuthology.orchestra.run.smithi152:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-20T11:26:30.215 DEBUG:teuthology.orchestra.run.smithi175:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-20T11:26:30.293 INFO:teuthology.orchestra.run.smithi152.stdout:[mgr.y] 2024-04-20T11:26:30.294 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQBCpiNmP1+5DRAAs6bVAEOr0i6FdPmSEyo/vQ== 2024-04-20T11:26:30.294 INFO:teuthology.orchestra.run.smithi152.stdout:[mds.a] 2024-04-20T11:26:30.294 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQBCpiNmyFRJIBAAj2aIMXdb65nqRgKFDBLLzA== 2024-04-20T11:26:30.294 INFO:teuthology.orchestra.run.smithi152.stdout:[mds.c] 2024-04-20T11:26:30.294 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQBCpiNm5ViKLRAAg7VBQG4hFTLYUpJMpBAaVA== 2024-04-20T11:26:30.294 INFO:teuthology.orchestra.run.smithi152.stdout:[osd.0] 2024-04-20T11:26:30.294 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQBHpiNmpuhyDBAAsQXX9DNvWlo0UYh53lb+FA== 2024-04-20T11:26:30.294 INFO:teuthology.orchestra.run.smithi152.stdout:[osd.1] 2024-04-20T11:26:30.294 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQBKpiNmxBiLJhAAHon1adC8G2jbYoDcWdE9jA== 2024-04-20T11:26:30.294 INFO:teuthology.orchestra.run.smithi152.stdout:[osd.2] 2024-04-20T11:26:30.294 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQBNpiNmdn0zMhAAqwIO2kKahdSzTVh8/p3FCg== 2024-04-20T11:26:30.295 INFO:teuthology.orchestra.run.smithi152.stdout:[osd.3] 2024-04-20T11:26:30.295 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQBRpiNmTPCYAhAAHFNHX/xKn21jEFHDkHMjEw== 2024-04-20T11:26:30.295 INFO:teuthology.orchestra.run.smithi152.stdout:[mgr.x] 2024-04-20T11:26:30.295 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQBCpiNmuhalFxAA9dTl/zSbUovCCZAanAipww== 2024-04-20T11:26:30.295 INFO:teuthology.orchestra.run.smithi152.stdout:[mds.b] 2024-04-20T11:26:30.295 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQBCpiNmVPytOhAAzQkvNOJaNy+MeKETWhYLyg== 2024-04-20T11:26:30.295 INFO:teuthology.orchestra.run.smithi152.stdout:[mds.d] 2024-04-20T11:26:30.295 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQBDpiNmLWhaCxAA2e4qd/7+TKenJlWICIsvGg== 2024-04-20T11:26:30.295 INFO:teuthology.orchestra.run.smithi152.stdout:[osd.4] 2024-04-20T11:26:30.295 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQBXpiNmM8cVFxAAJl9Hu+MvlkEiMglutI89eg== 2024-04-20T11:26:30.295 INFO:teuthology.orchestra.run.smithi152.stdout:[osd.5] 2024-04-20T11:26:30.295 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQBapiNmnGxJMRAA90P0T+VsTA8LCTcexTVr7w== 2024-04-20T11:26:30.295 INFO:teuthology.orchestra.run.smithi152.stdout:[osd.6] 2024-04-20T11:26:30.295 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQBepiNmyLXWDxAAhtHD5h478mLovE5IejAFng== 2024-04-20T11:26:30.295 INFO:teuthology.orchestra.run.smithi152.stdout:[osd.7] 2024-04-20T11:26:30.296 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQBhpiNmB3jwHBAAqco9om6r2N/GqAV6rVKiaA== 2024-04-20T11:26:30.296 INFO:teuthology.orchestra.run.smithi152.stdout:[client.0] 2024-04-20T11:26:30.296 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQBDpiNmtsBWFhAArq9A7r33CcwgMmOU2pAGPw== 2024-04-20T11:26:30.296 INFO:teuthology.orchestra.run.smithi152.stdout:[client.2] 2024-04-20T11:26:30.296 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQBDpiNm5QLtHRAAa2kHTDVdgSh/B4XGZaKa5Q== 2024-04-20T11:26:30.296 INFO:teuthology.orchestra.run.smithi152.stdout:[client.1] 2024-04-20T11:26:30.296 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQBDpiNm6aNnJxAAB431GSGytZ0IKWMh4DMalw== 2024-04-20T11:26:30.296 INFO:teuthology.orchestra.run.smithi152.stdout:[client.3] 2024-04-20T11:26:30.296 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQBDpiNmw95SMBAAdodzTuPp4WdKSEf+jI8Uuw== 2024-04-20T11:26:30.296 INFO:teuthology.orchestra.run.smithi175.stdout:[mgr.y] 2024-04-20T11:26:30.297 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQBCpiNmP1+5DRAAs6bVAEOr0i6FdPmSEyo/vQ== 2024-04-20T11:26:30.297 INFO:teuthology.orchestra.run.smithi175.stdout:[mds.a] 2024-04-20T11:26:30.297 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQBCpiNmyFRJIBAAj2aIMXdb65nqRgKFDBLLzA== 2024-04-20T11:26:30.297 INFO:teuthology.orchestra.run.smithi175.stdout:[mds.c] 2024-04-20T11:26:30.297 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQBCpiNm5ViKLRAAg7VBQG4hFTLYUpJMpBAaVA== 2024-04-20T11:26:30.297 INFO:teuthology.orchestra.run.smithi175.stdout:[osd.0] 2024-04-20T11:26:30.297 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQBHpiNmpuhyDBAAsQXX9DNvWlo0UYh53lb+FA== 2024-04-20T11:26:30.297 INFO:teuthology.orchestra.run.smithi175.stdout:[osd.1] 2024-04-20T11:26:30.297 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQBKpiNmxBiLJhAAHon1adC8G2jbYoDcWdE9jA== 2024-04-20T11:26:30.297 INFO:teuthology.orchestra.run.smithi175.stdout:[osd.2] 2024-04-20T11:26:30.297 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQBNpiNmdn0zMhAAqwIO2kKahdSzTVh8/p3FCg== 2024-04-20T11:26:30.297 INFO:teuthology.orchestra.run.smithi175.stdout:[osd.3] 2024-04-20T11:26:30.297 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQBRpiNmTPCYAhAAHFNHX/xKn21jEFHDkHMjEw== 2024-04-20T11:26:30.297 INFO:teuthology.orchestra.run.smithi175.stdout:[mgr.x] 2024-04-20T11:26:30.298 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQBCpiNmuhalFxAA9dTl/zSbUovCCZAanAipww== 2024-04-20T11:26:30.298 INFO:teuthology.orchestra.run.smithi175.stdout:[mds.b] 2024-04-20T11:26:30.298 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQBCpiNmVPytOhAAzQkvNOJaNy+MeKETWhYLyg== 2024-04-20T11:26:30.298 INFO:teuthology.orchestra.run.smithi175.stdout:[mds.d] 2024-04-20T11:26:30.298 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQBDpiNmLWhaCxAA2e4qd/7+TKenJlWICIsvGg== 2024-04-20T11:26:30.298 INFO:teuthology.orchestra.run.smithi175.stdout:[osd.4] 2024-04-20T11:26:30.298 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQBXpiNmM8cVFxAAJl9Hu+MvlkEiMglutI89eg== 2024-04-20T11:26:30.298 INFO:teuthology.orchestra.run.smithi175.stdout:[osd.5] 2024-04-20T11:26:30.298 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQBapiNmnGxJMRAA90P0T+VsTA8LCTcexTVr7w== 2024-04-20T11:26:30.298 INFO:teuthology.orchestra.run.smithi175.stdout:[osd.6] 2024-04-20T11:26:30.298 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQBepiNmyLXWDxAAhtHD5h478mLovE5IejAFng== 2024-04-20T11:26:30.298 INFO:teuthology.orchestra.run.smithi175.stdout:[osd.7] 2024-04-20T11:26:30.298 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQBhpiNmB3jwHBAAqco9om6r2N/GqAV6rVKiaA== 2024-04-20T11:26:30.298 INFO:teuthology.orchestra.run.smithi175.stdout:[client.0] 2024-04-20T11:26:30.299 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQBDpiNmtsBWFhAArq9A7r33CcwgMmOU2pAGPw== 2024-04-20T11:26:30.299 INFO:teuthology.orchestra.run.smithi175.stdout:[client.2] 2024-04-20T11:26:30.299 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQBDpiNm5QLtHRAAa2kHTDVdgSh/B4XGZaKa5Q== 2024-04-20T11:26:30.299 INFO:teuthology.orchestra.run.smithi175.stdout:[client.1] 2024-04-20T11:26:30.299 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQBDpiNm6aNnJxAAB431GSGytZ0IKWMh4DMalw== 2024-04-20T11:26:30.299 INFO:teuthology.orchestra.run.smithi175.stdout:[client.3] 2024-04-20T11:26:30.299 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQBDpiNmw95SMBAAdodzTuPp4WdKSEf+jI8Uuw== 2024-04-20T11:26:30.300 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-20T11:26:30.347 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-20T11:26:30.488 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-20T11:26:30.495 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-20T11:26:30.626 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-20T11:26:30.631 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-20T11:26:30.757 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T11:26:30.761 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T11:26:30.890 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T11:26:30.895 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T11:26:31.023 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T11:26:31.028 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T11:26:31.166 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T11:26:31.171 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T11:26:31.300 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-20T11:26:31.306 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-20T11:26:31.436 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-20T11:26:31.440 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-20T11:26:31.571 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-20T11:26:31.578 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-20T11:26:31.708 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T11:26:31.713 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T11:26:31.844 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T11:26:31.851 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T11:26:31.985 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T11:26:31.992 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T11:26:32.124 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T11:26:32.131 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T11:26:32.259 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-20T11:26:32.264 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-20T11:26:32.397 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-20T11:26:32.403 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-20T11:26:32.530 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-20T11:26:32.536 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-20T11:26:32.664 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-20T11:26:32.671 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-20T11:26:32.804 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-20T11:26:32.805 DEBUG:teuthology.orchestra.run.smithi152:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-20T11:26:32.889 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-20T11:26:33.029 DEBUG:teuthology.orchestra.run.smithi152:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-20T11:26:33.108 DEBUG:teuthology.orchestra.run.smithi175:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-04-20T11:26:33.197 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-20T11:26:33.345 DEBUG:teuthology.orchestra.run.smithi175:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-04-20T11:26:33.415 DEBUG:teuthology.orchestra.run.smithi175:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-04-20T11:26:33.498 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-20T11:26:33.639 DEBUG:teuthology.orchestra.run.smithi175:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-04-20T11:26:33.704 DEBUG:teuthology.orchestra.run.smithi152:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-20T11:26:33.712 DEBUG:teuthology.orchestra.run.smithi175:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-20T11:26:33.761 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-20T11:26:33.762 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-20T11:26:33.762 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-20T11:26:33.769 INFO:tasks.ceph.mon.a:Started 2024-04-20T11:26:33.769 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-20T11:26:33.769 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-20T11:26:33.773 INFO:tasks.ceph.mon.b:Started 2024-04-20T11:26:33.773 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-20T11:26:33.774 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-20T11:26:33.778 INFO:tasks.ceph.mon.c:Started 2024-04-20T11:26:33.778 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-20T11:26:33.778 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-04-20T11:26:33.778 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-04-20T11:26:33.785 INFO:tasks.ceph.mgr.y:Started 2024-04-20T11:26:33.785 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-20T11:26:33.785 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-20T11:26:33.789 INFO:tasks.ceph.mgr.x:Started 2024-04-20T11:26:33.789 INFO:tasks.ceph:Setting crush tunables to default 2024-04-20T11:26:33.789 DEBUG:teuthology.orchestra.run.smithi152:> sudo ceph --cluster ceph osd crush tunables default 2024-04-20T11:26:33.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:33.959+0000 7fafaa800640 1 Processor -- start 2024-04-20T11:26:33.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:33.959+0000 7fafaa800640 1 -- start start 2024-04-20T11:26:33.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:33.960+0000 7fafaa800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fafa40b2880 0x7fafa40b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:33.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:33.960+0000 7fafaa800640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fafa40cd1b0 0x7fafa40cd600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:33.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:33.960+0000 7fafaa800640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fafa41463e0 con 0x7fafa4146000 2024-04-20T11:26:33.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:33.960+0000 7fafaa800640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fafa40b4750 con 0x7fafa40cd1b0 2024-04-20T11:26:33.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:33.960+0000 7fafaa800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fafa40b3e20 con 0x7fafa40b2880 2024-04-20T11:26:33.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:33.960+0000 7faf9be00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fafa40b2880 0x7fafa40b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:33.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:33.961+0000 7faf9be00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fafa40b2880 0x7fafa40b4be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.152:47530/0 (socket says 172.21.15.152:47530) 2024-04-20T11:26:33.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:33.961+0000 7faf9be00640 1 -- 172.21.15.152:0/273909328 learned_addr learned my addr 172.21.15.152:0/273909328 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:26:33.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:33.961+0000 7faf9be00640 1 -- 172.21.15.152:0/273909328 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fafa40b2880 msgr2=0x7fafa40b4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-20T11:26:33.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:33.961+0000 7faf9be00640 1 -- 172.21.15.152:0/273909328 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fafa40b2880 msgr2=0x7fafa40b4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T11:26:33.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:33.961+0000 7faf9be00640 1 --2- 172.21.15.152:0/273909328 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fafa40b2880 0x7fafa40b4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T11:26:33.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:33.961+0000 7faf9be00640 1 --2- 172.21.15.152:0/273909328 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fafa40b2880 0x7fafa40b4be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T11:26:33.963 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:33.961+0000 7fafa3e00640 1 --2- 172.21.15.152:0/273909328 >> v2:172.21.15.175:3300/0 conn(0x7fafa40cd1b0 0x7fafa40cd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:33.963 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:33.961+0000 7fafa3e00640 1 -- 172.21.15.152:0/273909328 >> v2:172.21.15.175:3300/0 conn(0x7fafa40cd1b0 msgr2=0x7fafa40cd600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-20T11:26:33.963 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:33.961+0000 7fafa3e00640 1 -- 172.21.15.152:0/273909328 >> v2:172.21.15.175:3300/0 conn(0x7fafa40cd1b0 msgr2=0x7fafa40cd600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T11:26:33.963 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:33.961+0000 7fafa3e00640 1 --2- 172.21.15.152:0/273909328 >> v2:172.21.15.175:3300/0 conn(0x7fafa40cd1b0 0x7fafa40cd600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T11:26:33.963 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:33.961+0000 7fafa3e00640 1 --2- 172.21.15.152:0/273909328 >> v2:172.21.15.175:3300/0 conn(0x7fafa40cd1b0 0x7fafa40cd600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T11:26:33.979 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:33.977+0000 7fafa2a00640 1 -- 172.21.15.152:0/273909328 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1491424579 0 0) 0x7fafa41463e0 con 0x7fafa4146000 2024-04-20T11:26:33.979 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:33.977+0000 7fafa2a00640 1 -- 172.21.15.152:0/273909328 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faf88003520 con 0x7fafa4146000 2024-04-20T11:26:33.979 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:33.978+0000 7fafa2a00640 1 -- 172.21.15.152:0/273909328 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3374882927 0 0) 0x7faf88003520 con 0x7fafa4146000 2024-04-20T11:26:33.980 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:33.978+0000 7fafa2a00640 1 -- 172.21.15.152:0/273909328 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fafa40b2880 msgr2=0x7fafa40b4be0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T11:26:33.980 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:33.978+0000 7fafa2a00640 1 --2- 172.21.15.152:0/273909328 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fafa40b2880 0x7fafa40b4be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:33.980 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:33.978+0000 7fafa2a00640 1 -- 172.21.15.152:0/273909328 >> v2:172.21.15.175:3300/0 conn(0x7fafa40cd1b0 msgr2=0x7fafa40cd600 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T11:26:33.980 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:33.978+0000 7fafa2a00640 1 --2- 172.21.15.152:0/273909328 >> v2:172.21.15.175:3300/0 conn(0x7fafa40cd1b0 0x7fafa40cd600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:33.980 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:33.978+0000 7fafa2a00640 1 -- 172.21.15.152:0/273909328 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafa40b4100 con 0x7fafa4146000 2024-04-20T11:26:33.980 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:33.978+0000 7fafa2a00640 1 -- 172.21.15.152:0/273909328 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7faf94001670 con 0x7fafa4146000 2024-04-20T11:26:33.980 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:33.978+0000 7fafa2a00640 1 -- 172.21.15.152:0/273909328 >> v1:172.21.15.152:6789/0 conn(0x7fafa4146000 legacy=0x7fafa40b2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:33.980 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:33.978+0000 7fafa2a00640 1 --2- 172.21.15.152:0/273909328 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fafa40b2880 0x7faf880039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:33.980 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:33.978+0000 7fafa2a00640 1 --2- 172.21.15.152:0/273909328 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faf88003fe0 0x7faf88004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:33.980 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:33.978+0000 7fafa2a00640 1 --2- 172.21.15.152:0/273909328 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faf88004d40 0x7faf880051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:33.980 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:33.978+0000 7faf9be00640 1 --2- 172.21.15.152:0/273909328 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faf88003fe0 0x7faf88004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:33.980 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:33.978+0000 7fafa2a00640 1 -- 172.21.15.152:0/273909328 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fafa40b4750 con 0x7faf88003fe0 2024-04-20T11:26:33.980 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:33.978+0000 7fafa2a00640 1 -- 172.21.15.152:0/273909328 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fafa40b3e20 con 0x7faf88004d40 2024-04-20T11:26:33.980 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:33.978+0000 7fafa2a00640 1 -- 172.21.15.152:0/273909328 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7faf8800bb80 con 0x7fafa40b2880 2024-04-20T11:26:33.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:33.979+0000 7fafa3400640 1 --2- 172.21.15.152:0/273909328 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fafa40b2880 0x7faf880039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:33.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:33.979+0000 7fafa3e00640 1 --2- 172.21.15.152:0/273909328 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faf88004d40 0x7faf880051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:33.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:33.979+0000 7faf9be00640 1 -- 172.21.15.152:0/273909328 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fafa40b2880 msgr2=0x7faf880039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:33.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:33.979+0000 7faf9be00640 1 --2- 172.21.15.152:0/273909328 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fafa40b2880 0x7faf880039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:33.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:33.979+0000 7faf9be00640 1 -- 172.21.15.152:0/273909328 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faf88004d40 msgr2=0x7faf880051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:33.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:33.979+0000 7faf9be00640 1 --2- 172.21.15.152:0/273909328 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faf88004d40 0x7faf880051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:33.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:33.979+0000 7faf9be00640 1 -- 172.21.15.152:0/273909328 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf88003520 con 0x7faf88003fe0 2024-04-20T11:26:33.982 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:33.980+0000 7faf9be00640 1 --2- 172.21.15.152:0/273909328 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faf88003fe0 0x7faf88004800 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7faf8c009890 tx=0x7faf88005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9fbb68cf12f241 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:33.982 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:33.980+0000 7fafa2a00640 1 -- 172.21.15.152:0/273909328 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf8c034070 con 0x7faf88003fe0 2024-04-20T11:26:33.982 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:33.980+0000 7fafa2a00640 1 -- 172.21.15.152:0/273909328 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf8c030660 con 0x7faf88003fe0 2024-04-20T11:26:33.982 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:33.980+0000 7fafa2a00640 1 -- 172.21.15.152:0/273909328 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf8c00f040 con 0x7faf88003fe0 2024-04-20T11:26:33.982 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:33.981+0000 7fafaa800640 1 -- 172.21.15.152:0/273909328 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faf88003fe0 msgr2=0x7faf88004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:33.982 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:33.981+0000 7fafaa800640 1 --2- 172.21.15.152:0/273909328 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faf88003fe0 0x7faf88004800 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7faf8c009890 tx=0x7faf88005830 comp rx=0 tx=0).stop 2024-04-20T11:26:33.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:33.981+0000 7fafa3e00640 1 -- 172.21.15.152:0/273909328 reap_dead start 2024-04-20T11:26:33.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:33.981+0000 7fafaa800640 1 -- 172.21.15.152:0/273909328 shutdown_connections 2024-04-20T11:26:33.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:33.981+0000 7fafaa800640 1 -- 172.21.15.152:0/273909328 >> 172.21.15.152:0/273909328 conn(0x7fafa406b980 msgr2=0x7fafa406bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:33.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:33.981+0000 7fafaa800640 1 -- 172.21.15.152:0/273909328 shutdown_connections 2024-04-20T11:26:33.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:33.981+0000 7fafaa800640 1 -- 172.21.15.152:0/273909328 wait complete. 2024-04-20T11:26:33.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:33.983+0000 7fafaa800640 1 Processor -- start 2024-04-20T11:26:33.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:33.983+0000 7fafaa800640 1 -- start start 2024-04-20T11:26:33.986 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:33.984+0000 7fafaa800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faf88003fe0 0x7fafa41412d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:33.986 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:33.984+0000 7fafaa800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fafa40ae9f0 0x7fafa40aee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:33.986 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:33.984+0000 7fafaa800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fafa413efd0 0x7fafa413f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:33.986 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:33.984+0000 7fafaa800640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fafa40cdcc0 con 0x7faf88003fe0 2024-04-20T11:26:33.986 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:33.984+0000 7fafaa800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fafa40cdb40 con 0x7fafa40ae9f0 2024-04-20T11:26:33.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:33.984+0000 7fafaa800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fafa40b4be0 con 0x7fafa413efd0 2024-04-20T11:26:33.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:33.984+0000 7fafa3400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faf88003fe0 0x7fafa41412d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:33.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:33.984+0000 7fafa3400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faf88003fe0 0x7fafa41412d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:34896/0 (socket says 172.21.15.152:34896) 2024-04-20T11:26:33.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:33.984+0000 7fafa3400640 1 -- 172.21.15.152:0/553274315 learned_addr learned my addr 172.21.15.152:0/553274315 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:26:33.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:33.985+0000 7fafa3e00640 1 --2- 172.21.15.152:0/553274315 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fafa413efd0 0x7fafa413f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:33.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:33.985+0000 7faf9be00640 1 --2- 172.21.15.152:0/553274315 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fafa40ae9f0 0x7fafa40aee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:33.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:33.985+0000 7fafa3400640 1 -- 172.21.15.152:0/553274315 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fafa413efd0 msgr2=0x7fafa413f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:33.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:33.985+0000 7fafa3400640 1 --2- 172.21.15.152:0/553274315 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fafa413efd0 0x7fafa413f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:33.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:33.985+0000 7fafa3400640 1 -- 172.21.15.152:0/553274315 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fafa40ae9f0 msgr2=0x7fafa40aee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:33.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:33.985+0000 7fafa3400640 1 --2- 172.21.15.152:0/553274315 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fafa40ae9f0 0x7fafa40aee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:33.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:33.985+0000 7fafa3400640 1 -- 172.21.15.152:0/553274315 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafa40ad950 con 0x7faf88003fe0 2024-04-20T11:26:33.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:33.985+0000 7fafa3400640 1 --2- 172.21.15.152:0/553274315 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faf88003fe0 0x7fafa41412d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7faf9400e890 tx=0x7faf9400ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:33.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:33.985+0000 7faf9b400640 1 -- 172.21.15.152:0/553274315 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf940052e0 con 0x7faf88003fe0 2024-04-20T11:26:33.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:33.985+0000 7faf9b400640 1 -- 172.21.15.152:0/553274315 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf94018070 con 0x7faf88003fe0 2024-04-20T11:26:33.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:33.986+0000 7fafaa800640 1 -- 172.21.15.152:0/553274315 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafa40adc30 con 0x7faf88003fe0 2024-04-20T11:26:33.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:33.986+0000 7faf9b400640 1 -- 172.21.15.152:0/553274315 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf9400c900 con 0x7faf88003fe0 2024-04-20T11:26:33.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:33.986+0000 7fafaa800640 1 -- 172.21.15.152:0/553274315 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafa40ae1f0 con 0x7faf88003fe0 2024-04-20T11:26:33.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:33.986+0000 7faf9b400640 1 -- 172.21.15.152:0/553274315 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7faf9400caa0 con 0x7faf88003fe0 2024-04-20T11:26:33.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:33.986+0000 7faf9b400640 1 -- 172.21.15.152:0/553274315 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7faf9401d460 con 0x7faf88003fe0 2024-04-20T11:26:33.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:33.987+0000 7fafaa800640 1 -- 172.21.15.152:0/553274315 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafa40689f0 con 0x7faf88003fe0 2024-04-20T11:26:33.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:33.990+0000 7faf9b400640 1 -- 172.21.15.152:0/553274315 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7faf94003d70 con 0x7faf88003fe0 2024-04-20T11:26:34.061 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:34.061+0000 7fafaa800640 1 -- 172.21.15.152:0/553274315 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fafa415cca0 con 0x7faf88003fe0 2024-04-20T11:26:34.064 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:34.064+0000 7faf9b400640 1 -- 172.21.15.152:0/553274315 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7faf940035f0 con 0x7faf88003fe0 2024-04-20T11:26:34.064 INFO:teuthology.orchestra.run.smithi152.stderr:adjusted tunables profile to default 2024-04-20T11:26:34.066 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:34.065+0000 7fafaa800640 1 -- 172.21.15.152:0/553274315 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faf88003fe0 msgr2=0x7fafa41412d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:34.066 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:34.065+0000 7fafaa800640 1 --2- 172.21.15.152:0/553274315 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faf88003fe0 0x7fafa41412d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7faf9400e890 tx=0x7faf9400ed50 comp rx=0 tx=0).stop 2024-04-20T11:26:34.066 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:34.066+0000 7fafaa800640 1 -- 172.21.15.152:0/553274315 shutdown_connections 2024-04-20T11:26:34.067 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:34.066+0000 7fafaa800640 1 --2- 172.21.15.152:0/553274315 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fafa413efd0 0x7fafa413f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:34.067 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:34.066+0000 7fafaa800640 1 --2- 172.21.15.152:0/553274315 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fafa40ae9f0 0x7fafa40aee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:34.067 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:34.066+0000 7fafaa800640 1 --2- 172.21.15.152:0/553274315 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faf88003fe0 0x7fafa41412d0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:34.067 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:34.066+0000 7fafaa800640 1 -- 172.21.15.152:0/553274315 >> 172.21.15.152:0/553274315 conn(0x7fafa406b980 msgr2=0x7fafa40ce3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:34.067 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:34.066+0000 7fafaa800640 1 -- 172.21.15.152:0/553274315 shutdown_connections 2024-04-20T11:26:34.067 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:34.066+0000 7fafaa800640 1 -- 172.21.15.152:0/553274315 wait complete. 2024-04-20T11:26:34.085 INFO:tasks.ceph:check_enable_crimson: False 2024-04-20T11:26:34.085 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-20T11:26:34.085 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-20T11:26:34.085 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-20T11:26:34.178 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-20T11:26:34.178 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-20T11:26:34.290 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-20T11:26:34.291 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-20T11:26:34.295 INFO:tasks.ceph.mgr.y.smithi152.stderr:2024-04-20T11:26:34.294+0000 7fd1f000e040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-20T11:26:34.309 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:26:34.309+0000 7f8ae6493040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-20T11:26:34.361 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-20T11:26:34.361 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-20T11:26:34.476 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T11:26:34.476 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-04-20T11:26:34.499 INFO:tasks.ceph.mgr.y.smithi152.stderr:2024-04-20T11:26:34.498+0000 7fd1f000e040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-20T11:26:34.510 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:26:34.510+0000 7f8ae6493040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-20T11:26:34.549 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T11:26:34.549 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-04-20T11:26:34.565 INFO:tasks.ceph.mgr.y.smithi152.stderr:2024-04-20T11:26:34.564+0000 7fd1f000e040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-20T11:26:34.578 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:26:34.578+0000 7f8ae6493040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-20T11:26:34.657 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T11:26:34.657 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-04-20T11:26:34.768 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T11:26:34.768 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-04-20T11:26:34.872 INFO:tasks.ceph.mgr.y.smithi152.stderr:2024-04-20T11:26:34.872+0000 7fd1f000e040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-20T11:26:34.885 DEBUG:teuthology.orchestra.run.smithi175:> sudo ceph --cluster ceph osd new c34f4e30-d57a-4adb-aeed-19ea76f925d1 0 2024-04-20T11:26:34.912 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:26:34.912+0000 7f8ae6493040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-20T11:26:34.925 INFO:tasks.ceph.mgr.y.smithi152.stderr:2024-04-20T11:26:34.925+0000 7fd1f000e040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-20T11:26:34.972 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:26:34.972+0000 7f8ae6493040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-20T11:26:35.068 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.068+0000 7f952fa00640 1 Processor -- start 2024-04-20T11:26:35.069 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.068+0000 7f952fa00640 1 -- start start 2024-04-20T11:26:35.070 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.070+0000 7f952fa00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f95280b42d0 0x7f952806c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:35.070 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.070+0000 7f952fa00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9528162ee0 con 0x7f952815e750 2024-04-20T11:26:35.070 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.070+0000 7f952fa00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95281631a0 con 0x7f9528070190 2024-04-20T11:26:35.070 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.070+0000 7f952fa00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f952806b980 con 0x7f95280b42d0 2024-04-20T11:26:35.071 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.070+0000 7f9527e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f95280b42d0 0x7f952806c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:35.071 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.070+0000 7f952da00640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7f9528070190 0x7f952806c910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.175:38616/0 (socket says 172.21.15.175:38616) 2024-04-20T11:26:35.071 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.071+0000 7f952da00640 1 -- 172.21.15.175:0/651233346 learned_addr learned my addr 172.21.15.175:0/651233346 (peer_addr_for_me v1:172.21.15.175:0/0) 2024-04-20T11:26:35.071 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.071+0000 7f9527400640 1 -- 172.21.15.175:0/651233346 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1731780587 0 0) 0x7f95281631a0 con 0x7f9528070190 2024-04-20T11:26:35.071 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.071+0000 7f9527400640 1 -- 172.21.15.175:0/651233346 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9510003520 con 0x7f9528070190 2024-04-20T11:26:35.071 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.071+0000 7f9527e00640 1 -- 172.21.15.175:0/651233346 >> v1:172.21.15.175:6789/0 conn(0x7f9528070190 legacy=0x7f952806c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:35.072 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.071+0000 7f9527e00640 1 -- 172.21.15.175:0/651233346 >> v1:172.21.15.152:6789/0 conn(0x7f952815e750 legacy=0x7f95280b3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:35.072 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.071+0000 7f9527e00640 1 -- 172.21.15.175:0/651233346 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f952806bbd0 con 0x7f95280b42d0 2024-04-20T11:26:35.072 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.072+0000 7f9527e00640 1 --2- 172.21.15.175:0/651233346 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f95280b42d0 0x7f952806c3d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f95140097d0 tx=0x7f951402b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b185697e158ebbb4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:35.072 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.072+0000 7f9527400640 1 -- 172.21.15.175:0/651233346 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9514035020 con 0x7f95280b42d0 2024-04-20T11:26:35.072 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.072+0000 7f9527400640 1 -- 172.21.15.175:0/651233346 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9514031890 con 0x7f95280b42d0 2024-04-20T11:26:35.072 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.072+0000 7f9527400640 1 -- 172.21.15.175:0/651233346 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9514031c60 con 0x7f95280b42d0 2024-04-20T11:26:35.073 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.073+0000 7f952fa00640 1 -- 172.21.15.175:0/651233346 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f95280b42d0 msgr2=0x7f952806c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:35.073 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.073+0000 7f952fa00640 1 --2- 172.21.15.175:0/651233346 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f95280b42d0 0x7f952806c3d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f95140097d0 tx=0x7f951402b930 comp rx=0 tx=0).stop 2024-04-20T11:26:35.073 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.073+0000 7f952fa00640 1 -- 172.21.15.175:0/651233346 shutdown_connections 2024-04-20T11:26:35.074 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.073+0000 7f952fa00640 1 --2- 172.21.15.175:0/651233346 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f95280b42d0 0x7f952806c3d0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:35.074 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.073+0000 7f952fa00640 1 -- 172.21.15.175:0/651233346 >> 172.21.15.175:0/651233346 conn(0x7f95280d0c10 msgr2=0x7f95280d1020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:35.074 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.074+0000 7f952fa00640 1 -- 172.21.15.175:0/651233346 shutdown_connections 2024-04-20T11:26:35.074 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.074+0000 7f952fa00640 1 -- 172.21.15.175:0/651233346 wait complete. 2024-04-20T11:26:35.076 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.076+0000 7f952fa00640 1 Processor -- start 2024-04-20T11:26:35.076 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.076+0000 7f952fa00640 1 -- start start 2024-04-20T11:26:35.078 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.077+0000 7f952fa00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9528070190 0x7f95281512d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:35.078 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.077+0000 7f952fa00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f95280b42d0 0x7f95281592f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:35.078 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.077+0000 7f952fa00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f952815e750 0x7f952814f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:35.078 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.077+0000 7f952fa00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f952815eb30 con 0x7f952815e750 2024-04-20T11:26:35.078 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.077+0000 7f952fa00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f952806bf40 con 0x7f95280b42d0 2024-04-20T11:26:35.078 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.077+0000 7f952fa00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f9528151bd0 con 0x7f9528070190 2024-04-20T11:26:35.078 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.077+0000 7f952d000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9528070190 0x7f95281512d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:35.078 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.077+0000 7f9527e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f95280b42d0 0x7f95281592f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:35.078 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.077+0000 7f952d000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9528070190 0x7f95281512d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.175:49698/0 (socket says 172.21.15.175:49698) 2024-04-20T11:26:35.078 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.077+0000 7f952d000640 1 -- 172.21.15.175:0/1492173915 learned_addr learned my addr 172.21.15.175:0/1492173915 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-04-20T11:26:35.078 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.077+0000 7f952da00640 1 --2- 172.21.15.175:0/1492173915 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f952815e750 0x7f952814f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:35.079 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.078+0000 7f952d000640 1 -- 172.21.15.175:0/1492173915 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f95280b42d0 msgr2=0x7f95281592f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:35.079 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.078+0000 7f952d000640 1 --2- 172.21.15.175:0/1492173915 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f95280b42d0 0x7f95281592f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:35.079 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.078+0000 7f952d000640 1 -- 172.21.15.175:0/1492173915 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f952815e750 msgr2=0x7f952814f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:35.079 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.078+0000 7f952d000640 1 --2- 172.21.15.175:0/1492173915 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f952815e750 0x7f952814f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:35.079 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.078+0000 7f952d000640 1 -- 172.21.15.175:0/1492173915 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9528151eb0 con 0x7f9528070190 2024-04-20T11:26:35.079 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.078+0000 7f952da00640 1 --2- 172.21.15.175:0/1492173915 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f952815e750 0x7f952814f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:26:35.079 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.078+0000 7f952d000640 1 --2- 172.21.15.175:0/1492173915 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9528070190 0x7f95281512d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f951c00c430 tx=0x7f951c00c8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:35.080 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.078+0000 7f9524c00640 1 -- 172.21.15.175:0/1492173915 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f951c017070 con 0x7f9528070190 2024-04-20T11:26:35.080 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.078+0000 7f9524c00640 1 -- 172.21.15.175:0/1492173915 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f951c0052d0 con 0x7f9528070190 2024-04-20T11:26:35.080 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.078+0000 7f9524c00640 1 -- 172.21.15.175:0/1492173915 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f951c003830 con 0x7f9528070190 2024-04-20T11:26:35.080 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.078+0000 7f952fa00640 1 -- 172.21.15.175:0/1492173915 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9528152190 con 0x7f9528070190 2024-04-20T11:26:35.080 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.078+0000 7f952fa00640 1 -- 172.21.15.175:0/1492173915 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f9528152750 con 0x7f9528070190 2024-04-20T11:26:35.080 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.079+0000 7f9524c00640 1 -- 172.21.15.175:0/1492173915 <== mon.2 v2:172.21.15.175:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f951c01a070 con 0x7f9528070190 2024-04-20T11:26:35.080 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.079+0000 7f9524c00640 1 -- 172.21.15.175:0/1492173915 <== mon.2 v2:172.21.15.175:3301/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7f951c01c440 con 0x7f9528070190 2024-04-20T11:26:35.080 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.080+0000 7f952fa00640 1 -- 172.21.15.175:0/1492173915 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9528151930 con 0x7f9528070190 2024-04-20T11:26:35.082 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.082+0000 7f9524c00640 1 -- 172.21.15.175:0/1492173915 <== mon.2 v2:172.21.15.175:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7f951c021020 con 0x7f9528070190 2024-04-20T11:26:35.161 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.161+0000 7f952fa00640 1 -- 172.21.15.175:0/1492173915 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "osd new", "uuid": "c34f4e30-d57a-4adb-aeed-19ea76f925d1", "id": 0} v 0) -- 0x7f9528069100 con 0x7f9528070190 2024-04-20T11:26:35.168 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.168+0000 7f9524c00640 1 -- 172.21.15.175:0/1492173915 <== mon.2 v2:172.21.15.175:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c34f4e30-d57a-4adb-aeed-19ea76f925d1", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7f951c011590 con 0x7f9528070190 2024-04-20T11:26:35.168 INFO:teuthology.orchestra.run.smithi175.stdout:0 2024-04-20T11:26:35.170 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.170+0000 7f952fa00640 1 -- 172.21.15.175:0/1492173915 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9528070190 msgr2=0x7f95281512d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:35.171 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.170+0000 7f952fa00640 1 --2- 172.21.15.175:0/1492173915 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9528070190 0x7f95281512d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f951c00c430 tx=0x7f951c00c8f0 comp rx=0 tx=0).stop 2024-04-20T11:26:35.171 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.170+0000 7f952fa00640 1 -- 172.21.15.175:0/1492173915 shutdown_connections 2024-04-20T11:26:35.171 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.170+0000 7f952fa00640 1 --2- 172.21.15.175:0/1492173915 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f952815e750 0x7f952814f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:35.171 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.170+0000 7f952fa00640 1 --2- 172.21.15.175:0/1492173915 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f95280b42d0 0x7f95281592f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:35.171 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.170+0000 7f952fa00640 1 --2- 172.21.15.175:0/1492173915 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9528070190 0x7f95281512d0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:35.171 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.170+0000 7f952fa00640 1 -- 172.21.15.175:0/1492173915 >> 172.21.15.175:0/1492173915 conn(0x7f95280d0c10 msgr2=0x7f952815f410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:35.171 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.170+0000 7f952fa00640 1 -- 172.21.15.175:0/1492173915 shutdown_connections 2024-04-20T11:26:35.171 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.170+0000 7f952fa00640 1 -- 172.21.15.175:0/1492173915 wait complete. 2024-04-20T11:26:35.186 DEBUG:teuthology.orchestra.run.smithi175:> sudo ceph --cluster ceph osd new b518a138-bfc3-43bd-8f4d-e0ab01bf7316 1 2024-04-20T11:26:35.411 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.411+0000 7f2826400640 1 Processor -- start 2024-04-20T11:26:35.411 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.411+0000 7f2826400640 1 -- start start 2024-04-20T11:26:35.413 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.412+0000 7f2826400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f28200d46a0 0x7f28200d4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:35.413 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.412+0000 7f2826400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f28200d0d90 0x7f28200d1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:35.413 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.412+0000 7f2826400640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28200d19a0 con 0x7f282015e9e0 2024-04-20T11:26:35.413 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.412+0000 7f2826400640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f282006da30 con 0x7f28200d46a0 2024-04-20T11:26:35.413 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.413+0000 7f2826400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f28200d1760 con 0x7f28200d0d90 2024-04-20T11:26:35.413 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.413+0000 7f281f400640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7f282015e9e0 0x7f282006d320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.175:38628/0 (socket says 172.21.15.175:38628) 2024-04-20T11:26:35.413 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.413+0000 7f281fe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f28200d0d90 0x7f28200d1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:35.413 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.413+0000 7f281f400640 1 -- 172.21.15.175:0/3165287197 learned_addr learned my addr 172.21.15.175:0/3165287197 (peer_addr_for_me v1:172.21.15.175:0/0) 2024-04-20T11:26:35.414 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.413+0000 7f281fe00640 1 -- 172.21.15.175:0/3165287197 >> v2:172.21.15.152:3300/0 conn(0x7f28200d46a0 msgr2=0x7f28200d4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:35.414 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.413+0000 7f281fe00640 1 --2- 172.21.15.175:0/3165287197 >> v2:172.21.15.152:3300/0 conn(0x7f28200d46a0 0x7f28200d4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:35.414 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.413+0000 7f281fe00640 1 -- 172.21.15.175:0/3165287197 >> v1:172.21.15.175:6789/0 conn(0x7f282015e9e0 legacy=0x7f282006d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:35.414 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.413+0000 7f281fe00640 1 -- 172.21.15.175:0/3165287197 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28200d1c80 con 0x7f28200d0d90 2024-04-20T11:26:35.414 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.413+0000 7f281e000640 1 -- 172.21.15.175:0/3165287197 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2000007823 0 0) 0x7f28200d19a0 con 0x7f282015e9e0 2024-04-20T11:26:35.415 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.414+0000 7f281fe00640 1 --2- 172.21.15.175:0/3165287197 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f28200d0d90 0x7f28200d1220 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2818009d90 tx=0x7f281802bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3cd87a375d1e7c41 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:35.415 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.415+0000 7f281e000640 1 -- 172.21.15.175:0/3165287197 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2818039070 con 0x7f28200d0d90 2024-04-20T11:26:35.415 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.415+0000 7f281e000640 1 -- 172.21.15.175:0/3165287197 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f281802cc50 con 0x7f28200d0d90 2024-04-20T11:26:35.415 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.415+0000 7f281e000640 1 -- 172.21.15.175:0/3165287197 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2818034440 con 0x7f28200d0d90 2024-04-20T11:26:35.416 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.415+0000 7f2826400640 1 -- 172.21.15.175:0/3165287197 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f28200d0d90 msgr2=0x7f28200d1220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:35.416 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.415+0000 7f2826400640 1 --2- 172.21.15.175:0/3165287197 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f28200d0d90 0x7f28200d1220 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2818009d90 tx=0x7f281802bea0 comp rx=0 tx=0).stop 2024-04-20T11:26:35.416 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.416+0000 7f2826400640 1 -- 172.21.15.175:0/3165287197 shutdown_connections 2024-04-20T11:26:35.416 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.416+0000 7f2826400640 1 --2- 172.21.15.175:0/3165287197 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f28200d0d90 0x7f28200d1220 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:35.416 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.416+0000 7f2826400640 1 --2- 172.21.15.175:0/3165287197 >> v2:172.21.15.152:3300/0 conn(0x7f28200d46a0 0x7f28200d4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:35.416 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.416+0000 7f2826400640 1 -- 172.21.15.175:0/3165287197 >> 172.21.15.175:0/3165287197 conn(0x7f282009dab0 msgr2=0x7f282008bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:35.416 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.416+0000 7f2826400640 1 -- 172.21.15.175:0/3165287197 shutdown_connections 2024-04-20T11:26:35.416 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.416+0000 7f2826400640 1 -- 172.21.15.175:0/3165287197 wait complete. 2024-04-20T11:26:35.418 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.418+0000 7f2826400640 1 Processor -- start 2024-04-20T11:26:35.418 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.419+0000 7f2826400640 1 -- start start 2024-04-20T11:26:35.420 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.420+0000 7f2826400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28200d0d90 0x7f282006de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:35.421 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.420+0000 7f2826400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f28200d46a0 0x7f28200cc770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:35.421 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.420+0000 7f2826400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f282015e9e0 0x7f28200cd560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:35.421 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.420+0000 7f2826400640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f28200cf5a0 con 0x7f282015e9e0 2024-04-20T11:26:35.421 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.420+0000 7f2826400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f28200d5040 con 0x7f28200d0d90 2024-04-20T11:26:35.421 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.420+0000 7f2826400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f28200cf880 con 0x7f28200d46a0 2024-04-20T11:26:35.421 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.420+0000 7f281f400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28200d0d90 0x7f282006de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:35.421 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.420+0000 7f281ea00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f28200d46a0 0x7f28200cc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:35.421 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.420+0000 7f281f400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28200d0d90 0x7f282006de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:58786/0 (socket says 172.21.15.175:58786) 2024-04-20T11:26:35.421 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.420+0000 7f281f400640 1 -- 172.21.15.175:0/2757704089 learned_addr learned my addr 172.21.15.175:0/2757704089 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-04-20T11:26:35.421 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.421+0000 7f281fe00640 1 --2- 172.21.15.175:0/2757704089 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f282015e9e0 0x7f28200cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:35.421 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.421+0000 7f281ea00640 1 -- 172.21.15.175:0/2757704089 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28200d0d90 msgr2=0x7f282006de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:35.421 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.421+0000 7f281ea00640 1 --2- 172.21.15.175:0/2757704089 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28200d0d90 0x7f282006de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:35.421 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.421+0000 7f281ea00640 1 -- 172.21.15.175:0/2757704089 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f282015e9e0 msgr2=0x7f28200cd560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:35.421 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.421+0000 7f281ea00640 1 --2- 172.21.15.175:0/2757704089 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f282015e9e0 0x7f28200cd560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:35.422 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.421+0000 7f281ea00640 1 -- 172.21.15.175:0/2757704089 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28200cb9b0 con 0x7f28200d46a0 2024-04-20T11:26:35.422 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.421+0000 7f281f400640 1 --2- 172.21.15.175:0/2757704089 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28200d0d90 0x7f282006de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:26:35.423 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.421+0000 7f281ea00640 1 --2- 172.21.15.175:0/2757704089 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f28200d46a0 0x7f28200cc770 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f280c00b3f0 tx=0x7f280c00b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:35.423 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.421+0000 7f2817400640 1 -- 172.21.15.175:0/2757704089 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f280c015020 con 0x7f28200d46a0 2024-04-20T11:26:35.423 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.421+0000 7f2817400640 1 -- 172.21.15.175:0/2757704089 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f280c011830 con 0x7f28200d46a0 2024-04-20T11:26:35.423 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.421+0000 7f2817400640 1 -- 172.21.15.175:0/2757704089 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f280c011c20 con 0x7f28200d46a0 2024-04-20T11:26:35.423 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.421+0000 7f2826400640 1 -- 172.21.15.175:0/2757704089 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28200cbc90 con 0x7f28200d46a0 2024-04-20T11:26:35.423 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.422+0000 7f2826400640 1 -- 172.21.15.175:0/2757704089 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f2820150ca0 con 0x7f28200d46a0 2024-04-20T11:26:35.423 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.422+0000 7f2817400640 1 -- 172.21.15.175:0/2757704089 <== mon.2 v2:172.21.15.175:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f280c018020 con 0x7f28200d46a0 2024-04-20T11:26:35.423 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.422+0000 7f2817400640 1 -- 172.21.15.175:0/2757704089 <== mon.2 v2:172.21.15.175:3301/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7f280c0197a0 con 0x7f28200d46a0 2024-04-20T11:26:35.423 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.423+0000 7f2826400640 1 -- 172.21.15.175:0/2757704089 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28200689f0 con 0x7f28200d46a0 2024-04-20T11:26:35.425 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.425+0000 7f2817400640 1 -- 172.21.15.175:0/2757704089 <== mon.2 v2:172.21.15.175:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7f280c01f020 con 0x7f28200d46a0 2024-04-20T11:26:35.504 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.504+0000 7f2826400640 1 -- 172.21.15.175:0/2757704089 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "osd new", "uuid": "b518a138-bfc3-43bd-8f4d-e0ab01bf7316", "id": 1} v 0) -- 0x7f282006cdb0 con 0x7f28200d46a0 2024-04-20T11:26:35.511 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.511+0000 7f2817400640 1 -- 172.21.15.175:0/2757704089 <== mon.2 v2:172.21.15.175:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b518a138-bfc3-43bd-8f4d-e0ab01bf7316", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7f280c011dc0 con 0x7f28200d46a0 2024-04-20T11:26:35.511 INFO:teuthology.orchestra.run.smithi175.stdout:1 2024-04-20T11:26:35.512 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.512+0000 7f2826400640 1 -- 172.21.15.175:0/2757704089 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f28200d46a0 msgr2=0x7f28200cc770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:35.512 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.512+0000 7f2826400640 1 --2- 172.21.15.175:0/2757704089 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f28200d46a0 0x7f28200cc770 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f280c00b3f0 tx=0x7f280c00b8b0 comp rx=0 tx=0).stop 2024-04-20T11:26:35.512 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.512+0000 7f2826400640 1 -- 172.21.15.175:0/2757704089 shutdown_connections 2024-04-20T11:26:35.512 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.512+0000 7f2826400640 1 --2- 172.21.15.175:0/2757704089 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f282015e9e0 0x7f28200cd560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:35.513 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.512+0000 7f2826400640 1 --2- 172.21.15.175:0/2757704089 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f28200d46a0 0x7f28200cc770 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:35.513 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.512+0000 7f2826400640 1 --2- 172.21.15.175:0/2757704089 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28200d0d90 0x7f282006de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:35.513 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.512+0000 7f2826400640 1 -- 172.21.15.175:0/2757704089 >> 172.21.15.175:0/2757704089 conn(0x7f282009dab0 msgr2=0x7f282008bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:35.513 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.512+0000 7f2826400640 1 -- 172.21.15.175:0/2757704089 shutdown_connections 2024-04-20T11:26:35.513 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.512+0000 7f2826400640 1 -- 172.21.15.175:0/2757704089 wait complete. 2024-04-20T11:26:35.528 DEBUG:teuthology.orchestra.run.smithi175:> sudo ceph --cluster ceph osd new f3d523ee-a814-4537-92cc-87a84bb60958 2 2024-04-20T11:26:35.699 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.698+0000 7fea10000640 1 Processor -- start 2024-04-20T11:26:35.699 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.699+0000 7fea10000640 1 -- start start 2024-04-20T11:26:35.701 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.700+0000 7fea10000640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fea0815e9e0 0x7fea0806d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:35.701 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.700+0000 7fea10000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fea080d46a0 0x7fea080d4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:35.701 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.700+0000 7fea10000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fea080d0d90 0x7fea080d1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:35.701 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.700+0000 7fea10000640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fea080d1760 con 0x7fea080d0d90 2024-04-20T11:26:35.701 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.700+0000 7fea10000640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fea080d5040 con 0x7fea0815e9e0 2024-04-20T11:26:35.701 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.700+0000 7fea10000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fea080d19a0 con 0x7fea080d46a0 2024-04-20T11:26:35.701 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.700+0000 7fea0cc00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fea080d46a0 0x7fea080d4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:35.701 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.700+0000 7fea0d600640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fea0815e9e0 0x7fea0806d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:35.701 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.700+0000 7fea0cc00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fea080d46a0 0x7fea080d4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.175:49730/0 (socket says 172.21.15.175:49730) 2024-04-20T11:26:35.701 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.700+0000 7fea0d600640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fea0815e9e0 0x7fea0806d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:58802/0 (socket says 172.21.15.175:58802) 2024-04-20T11:26:35.701 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.700+0000 7fea0cc00640 1 -- 172.21.15.175:0/813838580 learned_addr learned my addr 172.21.15.175:0/813838580 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-04-20T11:26:35.702 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.701+0000 7fea0e000640 1 --2- 172.21.15.175:0/813838580 >> v2:172.21.15.152:3300/0 conn(0x7fea080d0d90 0x7fea080d1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:35.702 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.701+0000 7fea0d600640 1 -- 172.21.15.175:0/813838580 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fea080d46a0 msgr2=0x7fea080d4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:35.702 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.701+0000 7fea0d600640 1 --2- 172.21.15.175:0/813838580 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fea080d46a0 0x7fea080d4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:35.702 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.701+0000 7fea0d600640 1 -- 172.21.15.175:0/813838580 >> v2:172.21.15.152:3300/0 conn(0x7fea080d0d90 msgr2=0x7fea080d1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:35.702 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.701+0000 7fea0d600640 1 --2- 172.21.15.175:0/813838580 >> v2:172.21.15.152:3300/0 conn(0x7fea080d0d90 0x7fea080d1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:35.702 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.701+0000 7fea0d600640 1 -- 172.21.15.175:0/813838580 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea080d1c80 con 0x7fea0815e9e0 2024-04-20T11:26:35.702 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.701+0000 7fea0cc00640 1 --2- 172.21.15.175:0/813838580 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fea080d46a0 0x7fea080d4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:26:35.703 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.702+0000 7fea0d600640 1 --2- 172.21.15.175:0/813838580 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fea0815e9e0 0x7fea0806d320 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fea0809e0e0 tx=0x7fe9fc02b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=388cda7e6c86a15f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:35.704 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.702+0000 7fea07e00640 1 -- 172.21.15.175:0/813838580 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9fc0315a0 con 0x7fea0815e9e0 2024-04-20T11:26:35.704 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.702+0000 7fea07e00640 1 -- 172.21.15.175:0/813838580 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9fc037070 con 0x7fea0815e9e0 2024-04-20T11:26:35.704 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.702+0000 7fea07e00640 1 -- 172.21.15.175:0/813838580 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9fc031960 con 0x7fea0815e9e0 2024-04-20T11:26:35.704 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.703+0000 7fea10000640 1 -- 172.21.15.175:0/813838580 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fea0815e9e0 msgr2=0x7fea0806d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:35.704 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.703+0000 7fea10000640 1 --2- 172.21.15.175:0/813838580 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fea0815e9e0 0x7fea0806d320 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fea0809e0e0 tx=0x7fe9fc02b840 comp rx=0 tx=0).stop 2024-04-20T11:26:35.704 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.703+0000 7fea10000640 1 -- 172.21.15.175:0/813838580 shutdown_connections 2024-04-20T11:26:35.704 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.703+0000 7fea10000640 1 --2- 172.21.15.175:0/813838580 >> v2:172.21.15.152:3300/0 conn(0x7fea080d0d90 0x7fea080d1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:35.704 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.703+0000 7fea10000640 1 --2- 172.21.15.175:0/813838580 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fea080d46a0 0x7fea080d4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:35.704 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.703+0000 7fea10000640 1 --2- 172.21.15.175:0/813838580 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fea0815e9e0 0x7fea0806d320 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:35.704 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.703+0000 7fea10000640 1 -- 172.21.15.175:0/813838580 >> 172.21.15.175:0/813838580 conn(0x7fea0809dab0 msgr2=0x7fea0808bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:35.704 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.703+0000 7fea10000640 1 -- 172.21.15.175:0/813838580 shutdown_connections 2024-04-20T11:26:35.704 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.703+0000 7fea10000640 1 -- 172.21.15.175:0/813838580 wait complete. 2024-04-20T11:26:35.705 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.705+0000 7fea10000640 1 Processor -- start 2024-04-20T11:26:35.705 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.705+0000 7fea10000640 1 -- start start 2024-04-20T11:26:35.708 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.706+0000 7fea10000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fea080d0d90 0x7fea080b6c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:35.708 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.706+0000 7fea10000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fea080d46a0 0x7fea080b7360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:35.708 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.706+0000 7fea10000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fea0815e9e0 0x7fea080ba030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:35.708 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.706+0000 7fea10000640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fea080cf880 con 0x7fea080d0d90 2024-04-20T11:26:35.708 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.706+0000 7fea10000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fea080cf5a0 con 0x7fea0815e9e0 2024-04-20T11:26:35.708 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.706+0000 7fea10000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fea0806d860 con 0x7fea080d46a0 2024-04-20T11:26:35.708 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.706+0000 7fea0cc00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fea080d46a0 0x7fea080b7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:35.708 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.706+0000 7fea0e000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fea0815e9e0 0x7fea080ba030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:35.708 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.706+0000 7fea0cc00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fea080d46a0 0x7fea080b7360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.175:49732/0 (socket says 172.21.15.175:49732) 2024-04-20T11:26:35.708 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.706+0000 7fea0cc00640 1 -- 172.21.15.175:0/1883716435 learned_addr learned my addr 172.21.15.175:0/1883716435 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-04-20T11:26:35.708 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.706+0000 7fea0d600640 1 --2- 172.21.15.175:0/1883716435 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fea080d0d90 0x7fea080b6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:35.708 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.707+0000 7fea0cc00640 1 -- 172.21.15.175:0/1883716435 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fea0815e9e0 msgr2=0x7fea080ba030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:35.709 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.707+0000 7fea0cc00640 1 --2- 172.21.15.175:0/1883716435 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fea0815e9e0 0x7fea080ba030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:35.709 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.707+0000 7fea0cc00640 1 -- 172.21.15.175:0/1883716435 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fea080d0d90 msgr2=0x7fea080b6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:35.709 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.707+0000 7fea0cc00640 1 --2- 172.21.15.175:0/1883716435 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fea080d0d90 0x7fea080b6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:35.709 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.707+0000 7fea0cc00640 1 -- 172.21.15.175:0/1883716435 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea080b6830 con 0x7fea080d46a0 2024-04-20T11:26:35.709 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.707+0000 7fea0d600640 1 --2- 172.21.15.175:0/1883716435 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fea080d0d90 0x7fea080b6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:26:35.709 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.707+0000 7fea0e000640 1 --2- 172.21.15.175:0/1883716435 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fea0815e9e0 0x7fea080ba030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:26:35.709 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.707+0000 7fea0cc00640 1 --2- 172.21.15.175:0/1883716435 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fea080d46a0 0x7fea080b7360 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe9f400e9d0 tx=0x7fe9f400ee90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:35.709 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.707+0000 7fea05600640 1 -- 172.21.15.175:0/1883716435 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f4017070 con 0x7fea080d46a0 2024-04-20T11:26:35.709 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.707+0000 7fea05600640 1 -- 172.21.15.175:0/1883716435 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9f400ca60 con 0x7fea080d46a0 2024-04-20T11:26:35.709 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.707+0000 7fea05600640 1 -- 172.21.15.175:0/1883716435 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f400ce10 con 0x7fea080d46a0 2024-04-20T11:26:35.709 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.707+0000 7fea10000640 1 -- 172.21.15.175:0/1883716435 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea080b6270 con 0x7fea080d46a0 2024-04-20T11:26:35.709 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.707+0000 7fea10000640 1 -- 172.21.15.175:0/1883716435 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fea080c83a0 con 0x7fea080d46a0 2024-04-20T11:26:35.709 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.708+0000 7fea05600640 1 -- 172.21.15.175:0/1883716435 <== mon.2 v2:172.21.15.175:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe9f401a070 con 0x7fea080d46a0 2024-04-20T11:26:35.709 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.708+0000 7fea05600640 1 -- 172.21.15.175:0/1883716435 <== mon.2 v2:172.21.15.175:3301/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7fe9f4006020 con 0x7fea080d46a0 2024-04-20T11:26:35.710 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.709+0000 7fea10000640 1 -- 172.21.15.175:0/1883716435 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea080689f0 con 0x7fea080d46a0 2024-04-20T11:26:35.711 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.711+0000 7fea05600640 1 -- 172.21.15.175:0/1883716435 <== mon.2 v2:172.21.15.175:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7fe9f4006520 con 0x7fea080d46a0 2024-04-20T11:26:35.791 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.791+0000 7fea10000640 1 -- 172.21.15.175:0/1883716435 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "osd new", "uuid": "f3d523ee-a814-4537-92cc-87a84bb60958", "id": 2} v 0) -- 0x7fea0806cdb0 con 0x7fea080d46a0 2024-04-20T11:26:35.798 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.798+0000 7fea05600640 1 -- 172.21.15.175:0/1883716435 <== mon.2 v2:172.21.15.175:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f3d523ee-a814-4537-92cc-87a84bb60958", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7fe9f40067a0 con 0x7fea080d46a0 2024-04-20T11:26:35.798 INFO:teuthology.orchestra.run.smithi175.stdout:2 2024-04-20T11:26:35.799 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.799+0000 7fea10000640 1 -- 172.21.15.175:0/1883716435 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fea080d46a0 msgr2=0x7fea080b7360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:35.799 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.799+0000 7fea10000640 1 --2- 172.21.15.175:0/1883716435 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fea080d46a0 0x7fea080b7360 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe9f400e9d0 tx=0x7fe9f400ee90 comp rx=0 tx=0).stop 2024-04-20T11:26:35.800 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.799+0000 7fea10000640 1 -- 172.21.15.175:0/1883716435 shutdown_connections 2024-04-20T11:26:35.800 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.799+0000 7fea10000640 1 --2- 172.21.15.175:0/1883716435 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fea0815e9e0 0x7fea080ba030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:35.800 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.799+0000 7fea10000640 1 --2- 172.21.15.175:0/1883716435 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fea080d46a0 0x7fea080b7360 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:35.800 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.799+0000 7fea10000640 1 --2- 172.21.15.175:0/1883716435 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fea080d0d90 0x7fea080b6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:35.800 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.799+0000 7fea10000640 1 -- 172.21.15.175:0/1883716435 >> 172.21.15.175:0/1883716435 conn(0x7fea0809dab0 msgr2=0x7fea0815dc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:35.800 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.799+0000 7fea10000640 1 -- 172.21.15.175:0/1883716435 shutdown_connections 2024-04-20T11:26:35.800 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.799+0000 7fea10000640 1 -- 172.21.15.175:0/1883716435 wait complete. 2024-04-20T11:26:35.816 DEBUG:teuthology.orchestra.run.smithi175:> sudo ceph --cluster ceph osd new 27143d57-518d-4081-9ac5-c49ee82adf90 3 2024-04-20T11:26:35.910 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:26:35.910+0000 7f8ae6493040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-20T11:26:35.933 INFO:tasks.ceph.mgr.y.smithi152.stderr:2024-04-20T11:26:35.932+0000 7fd1f000e040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-20T11:26:35.988 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.988+0000 7f6654a00640 1 Processor -- start 2024-04-20T11:26:35.988 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.988+0000 7f6654a00640 1 -- start start 2024-04-20T11:26:35.990 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.990+0000 7f6654a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6650163230 0x7f66500d2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:35.990 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.990+0000 7f6654a00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f66500d33d0 0x7f66500cf470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:35.991 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.990+0000 7f6654a00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66500d19c0 con 0x7f665015eb50 2024-04-20T11:26:35.991 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.990+0000 7f6654a00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f6650162750 con 0x7f66500d33d0 2024-04-20T11:26:35.991 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.990+0000 7f6654a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6650162a30 con 0x7f6650163230 2024-04-20T11:26:35.991 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.990+0000 7f664e800640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7f665015eb50 0x7f66500d0f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.175:38640/0 (socket says 172.21.15.175:38640) 2024-04-20T11:26:35.991 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.990+0000 7f664e800640 1 -- 172.21.15.175:0/1797259185 learned_addr learned my addr 172.21.15.175:0/1797259185 (peer_addr_for_me v1:172.21.15.175:0/0) 2024-04-20T11:26:35.991 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.990+0000 7f664f200640 1 --2- 172.21.15.175:0/1797259185 >> v2:172.21.15.175:3300/0 conn(0x7f66500d33d0 0x7f66500cf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:35.991 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.990+0000 7f6645e00640 1 --2- 172.21.15.175:0/1797259185 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6650163230 0x7f66500d2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:35.992 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.991+0000 7f664de00640 1 -- 172.21.15.175:0/1797259185 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1393925394 0 0) 0x7f66500d19c0 con 0x7f665015eb50 2024-04-20T11:26:35.992 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.991+0000 7f664de00640 1 -- 172.21.15.175:0/1797259185 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6634003520 con 0x7f665015eb50 2024-04-20T11:26:35.992 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.991+0000 7f6645e00640 1 -- 172.21.15.175:0/1797259185 >> v2:172.21.15.175:3300/0 conn(0x7f66500d33d0 msgr2=0x7f66500cf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:35.992 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.991+0000 7f6645e00640 1 --2- 172.21.15.175:0/1797259185 >> v2:172.21.15.175:3300/0 conn(0x7f66500d33d0 0x7f66500cf470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:35.992 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.991+0000 7f6645e00640 1 -- 172.21.15.175:0/1797259185 >> v1:172.21.15.175:6789/0 conn(0x7f665015eb50 legacy=0x7f66500d0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:35.992 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.991+0000 7f6645e00640 1 -- 172.21.15.175:0/1797259185 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6650162d10 con 0x7f6650163230 2024-04-20T11:26:35.992 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.991+0000 7f664f200640 1 --2- 172.21.15.175:0/1797259185 >> v2:172.21.15.175:3300/0 conn(0x7f66500d33d0 0x7f66500cf470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:26:35.992 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.992+0000 7f6645e00640 1 --2- 172.21.15.175:0/1797259185 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6650163230 0x7f66500d2e90 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f6638009790 tx=0x7f663802b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26372425c3704dd6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:35.993 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.992+0000 7f664de00640 1 -- 172.21.15.175:0/1797259185 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6638031780 con 0x7f6650163230 2024-04-20T11:26:35.994 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.992+0000 7f664de00640 1 -- 172.21.15.175:0/1797259185 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6638037070 con 0x7f6650163230 2024-04-20T11:26:35.994 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.992+0000 7f664de00640 1 -- 172.21.15.175:0/1797259185 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6638031b40 con 0x7f6650163230 2024-04-20T11:26:35.994 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.993+0000 7f6654a00640 1 -- 172.21.15.175:0/1797259185 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6650163230 msgr2=0x7f66500d2e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:35.994 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.993+0000 7f6654a00640 1 --2- 172.21.15.175:0/1797259185 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6650163230 0x7f66500d2e90 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f6638009790 tx=0x7f663802b820 comp rx=0 tx=0).stop 2024-04-20T11:26:35.994 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.993+0000 7f6654a00640 1 -- 172.21.15.175:0/1797259185 shutdown_connections 2024-04-20T11:26:35.994 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.993+0000 7f6654a00640 1 --2- 172.21.15.175:0/1797259185 >> v2:172.21.15.175:3300/0 conn(0x7f66500d33d0 0x7f66500cf470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:35.994 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.993+0000 7f6654a00640 1 --2- 172.21.15.175:0/1797259185 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6650163230 0x7f66500d2e90 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:35.994 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.993+0000 7f6654a00640 1 -- 172.21.15.175:0/1797259185 >> 172.21.15.175:0/1797259185 conn(0x7f665008bfd0 msgr2=0x7f66500d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:35.994 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.993+0000 7f6654a00640 1 -- 172.21.15.175:0/1797259185 shutdown_connections 2024-04-20T11:26:35.994 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.993+0000 7f6654a00640 1 -- 172.21.15.175:0/1797259185 wait complete. 2024-04-20T11:26:35.995 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.995+0000 7f6654a00640 1 Processor -- start 2024-04-20T11:26:35.995 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.995+0000 7f6654a00640 1 -- start start 2024-04-20T11:26:35.998 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.996+0000 7f6654a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f66500d33d0 0x7f665014c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:35.998 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.996+0000 7f6654a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f665015eb50 0x7f66500b6a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:35.998 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.996+0000 7f6654a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6650163230 0x7f665014f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:35.998 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.996+0000 7f6654a00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f66500d1610 con 0x7f66500d33d0 2024-04-20T11:26:35.998 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.996+0000 7f6654a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f66500cf9b0 con 0x7f6650163230 2024-04-20T11:26:35.999 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.996+0000 7f6654a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f665014f440 con 0x7f665015eb50 2024-04-20T11:26:35.999 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.996+0000 7f6645e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f665015eb50 0x7f66500b6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:35.999 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.996+0000 7f664f200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6650163230 0x7f665014f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:35.999 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.996+0000 7f6645e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f665015eb50 0x7f66500b6a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.175:49758/0 (socket says 172.21.15.175:49758) 2024-04-20T11:26:35.999 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.996+0000 7f6645e00640 1 -- 172.21.15.175:0/607935152 learned_addr learned my addr 172.21.15.175:0/607935152 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-04-20T11:26:35.999 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.997+0000 7f664e800640 1 --2- 172.21.15.175:0/607935152 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f66500d33d0 0x7f665014c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:35.999 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.997+0000 7f6645e00640 1 -- 172.21.15.175:0/607935152 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6650163230 msgr2=0x7f665014f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:35.999 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.997+0000 7f6645e00640 1 --2- 172.21.15.175:0/607935152 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6650163230 0x7f665014f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:35.999 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.997+0000 7f6645e00640 1 -- 172.21.15.175:0/607935152 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f66500d33d0 msgr2=0x7f665014c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:35.999 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.997+0000 7f6645e00640 1 --2- 172.21.15.175:0/607935152 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f66500d33d0 0x7f665014c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:35.999 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.997+0000 7f6645e00640 1 -- 172.21.15.175:0/607935152 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66500b5780 con 0x7f665015eb50 2024-04-20T11:26:35.999 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.997+0000 7f664f200640 1 --2- 172.21.15.175:0/607935152 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6650163230 0x7f665014f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:26:35.999 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.997+0000 7f6645e00640 1 --2- 172.21.15.175:0/607935152 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f665015eb50 0x7f66500b6a80 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f663802be30 tx=0x7f66380046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:35.999 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.998+0000 7f6647400640 1 -- 172.21.15.175:0/607935152 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6638049070 con 0x7f665015eb50 2024-04-20T11:26:36.000 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.998+0000 7f6647400640 1 -- 172.21.15.175:0/607935152 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6638031ec0 con 0x7f665015eb50 2024-04-20T11:26:36.000 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.998+0000 7f6647400640 1 -- 172.21.15.175:0/607935152 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6638044630 con 0x7f665015eb50 2024-04-20T11:26:36.000 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.998+0000 7f6654a00640 1 -- 172.21.15.175:0/607935152 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66500b5a60 con 0x7f665015eb50 2024-04-20T11:26:36.000 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.998+0000 7f6654a00640 1 -- 172.21.15.175:0/607935152 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f665014fe10 con 0x7f665015eb50 2024-04-20T11:26:36.000 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.998+0000 7f6647400640 1 -- 172.21.15.175:0/607935152 <== mon.2 v2:172.21.15.175:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6638005990 con 0x7f665015eb50 2024-04-20T11:26:36.000 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.998+0000 7f6647400640 1 -- 172.21.15.175:0/607935152 <== mon.2 v2:172.21.15.175:3301/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7f6638005c80 con 0x7f665015eb50 2024-04-20T11:26:36.000 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:35.999+0000 7f6654a00640 1 -- 172.21.15.175:0/607935152 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6650075e60 con 0x7f665015eb50 2024-04-20T11:26:36.001 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.001+0000 7f6647400640 1 -- 172.21.15.175:0/607935152 <== mon.2 v2:172.21.15.175:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7f663802f050 con 0x7f665015eb50 2024-04-20T11:26:36.027 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:26:36.027+0000 7f8ae6493040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-20T11:26:36.038 INFO:tasks.ceph.mgr.y.smithi152.stderr:2024-04-20T11:26:36.038+0000 7fd1f000e040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-20T11:26:36.080 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.080+0000 7f6654a00640 1 -- 172.21.15.175:0/607935152 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "osd new", "uuid": "27143d57-518d-4081-9ac5-c49ee82adf90", "id": 3} v 0) -- 0x7f66500ce560 con 0x7f665015eb50 2024-04-20T11:26:36.082 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:26:36.082+0000 7f8ae6493040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-20T11:26:36.086 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.086+0000 7f6647400640 1 -- 172.21.15.175:0/607935152 <== mon.2 v2:172.21.15.175:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "27143d57-518d-4081-9ac5-c49ee82adf90", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7f66380447d0 con 0x7f665015eb50 2024-04-20T11:26:36.086 INFO:teuthology.orchestra.run.smithi175.stdout:3 2024-04-20T11:26:36.087 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.087+0000 7f6654a00640 1 -- 172.21.15.175:0/607935152 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f665015eb50 msgr2=0x7f66500b6a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:36.087 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.087+0000 7f6654a00640 1 --2- 172.21.15.175:0/607935152 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f665015eb50 0x7f66500b6a80 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f663802be30 tx=0x7f66380046c0 comp rx=0 tx=0).stop 2024-04-20T11:26:36.088 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.087+0000 7f6654a00640 1 -- 172.21.15.175:0/607935152 shutdown_connections 2024-04-20T11:26:36.088 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.087+0000 7f6654a00640 1 --2- 172.21.15.175:0/607935152 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6650163230 0x7f665014f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:36.088 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.087+0000 7f6654a00640 1 --2- 172.21.15.175:0/607935152 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f665015eb50 0x7f66500b6a80 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:36.088 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.087+0000 7f6654a00640 1 --2- 172.21.15.175:0/607935152 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f66500d33d0 0x7f665014c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:36.088 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.087+0000 7f6654a00640 1 -- 172.21.15.175:0/607935152 >> 172.21.15.175:0/607935152 conn(0x7f665008bfd0 msgr2=0x7f66500d2480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:36.088 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.087+0000 7f6654a00640 1 -- 172.21.15.175:0/607935152 shutdown_connections 2024-04-20T11:26:36.088 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.087+0000 7f6654a00640 1 -- 172.21.15.175:0/607935152 wait complete. 2024-04-20T11:26:36.089 INFO:tasks.ceph.mgr.y.smithi152.stderr:2024-04-20T11:26:36.089+0000 7fd1f000e040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-20T11:26:36.104 DEBUG:teuthology.orchestra.run.smithi175:> sudo ceph --cluster ceph osd new d431d09f-6287-457c-bb3f-7f521756de61 4 2024-04-20T11:26:36.277 INFO:tasks.ceph.mgr.y.smithi152.stderr:2024-04-20T11:26:36.277+0000 7fd1f000e040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-20T11:26:36.287 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.287+0000 7fa7d0a00640 1 Processor -- start 2024-04-20T11:26:36.287 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.287+0000 7fa7d0a00640 1 -- start start 2024-04-20T11:26:36.289 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.289+0000 7fa7d0a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fa7cc0d3340 0x7fa7cc1698b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:36.289 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.289+0000 7fa7d0a00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7cc163360 con 0x7fa7cc06e630 2024-04-20T11:26:36.289 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.289+0000 7fa7d0a00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7cc0cf510 con 0x7fa7cc0d0570 2024-04-20T11:26:36.289 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.289+0000 7fa7d0a00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fa7cc0cf7b0 con 0x7fa7cc0d3340 2024-04-20T11:26:36.289 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.289+0000 7fa7c1400640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7fa7cc0d0570 0x7fa7cc162c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.175:38644/0 (socket says 172.21.15.175:38644) 2024-04-20T11:26:36.289 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.289+0000 7fa7c1400640 1 -- 172.21.15.175:0/3800260063 learned_addr learned my addr 172.21.15.175:0/3800260063 (peer_addr_for_me v1:172.21.15.175:0/0) 2024-04-20T11:26:36.293 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.289+0000 7fa7ca800640 1 --2- 172.21.15.175:0/3800260063 >> v2:172.21.15.152:3300/0 conn(0x7fa7cc0d3340 0x7fa7cc1698b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:36.293 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.289+0000 7fa7c9400640 1 -- 172.21.15.175:0/3800260063 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2727296306 0 0) 0x7fa7cc0cf510 con 0x7fa7cc0d0570 2024-04-20T11:26:36.293 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.289+0000 7fa7c9400640 1 -- 172.21.15.175:0/3800260063 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa7b0003520 con 0x7fa7cc0d0570 2024-04-20T11:26:36.293 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.290+0000 7fa7c9400640 1 -- 172.21.15.175:0/3800260063 <== mon.1 v1:172.21.15.175:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1216083024 0 0) 0x7fa7b0003520 con 0x7fa7cc0d0570 2024-04-20T11:26:36.293 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.290+0000 7fa7c9400640 1 -- 172.21.15.175:0/3800260063 >> v2:172.21.15.152:3300/0 conn(0x7fa7cc0d3340 msgr2=0x7fa7cc1698b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:36.293 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.290+0000 7fa7c9400640 1 --2- 172.21.15.175:0/3800260063 >> v2:172.21.15.152:3300/0 conn(0x7fa7cc0d3340 0x7fa7cc1698b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:36.293 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.290+0000 7fa7c9400640 1 -- 172.21.15.175:0/3800260063 >> v1:172.21.15.152:6789/0 conn(0x7fa7cc06e630 legacy=0x7fa7cc162540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:36.293 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.290+0000 7fa7c9400640 1 -- 172.21.15.175:0/3800260063 --> v1:172.21.15.175:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7cc0cfa90 con 0x7fa7cc0d0570 2024-04-20T11:26:36.293 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.290+0000 7fa7c9400640 1 -- 172.21.15.175:0/3800260063 <== mon.1 v1:172.21.15.175:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7fa7b4001670 con 0x7fa7cc0d0570 2024-04-20T11:26:36.293 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.290+0000 7fa7ca800640 1 --2- 172.21.15.175:0/3800260063 >> v2:172.21.15.152:3300/0 conn(0x7fa7cc0d3340 0x7fa7cc1698b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:26:36.293 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.290+0000 7fa7c9400640 1 -- 172.21.15.175:0/3800260063 >> v1:172.21.15.175:6789/0 conn(0x7fa7cc0d0570 legacy=0x7fa7cc162c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:36.294 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.290+0000 7fa7c9400640 1 --2- 172.21.15.175:0/3800260063 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7b00039b0 0x7fa7b0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:36.294 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.290+0000 7fa7c9400640 1 --2- 172.21.15.175:0/3800260063 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa7b0004800 0x7fa7b0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:36.294 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.290+0000 7fa7c9400640 1 --2- 172.21.15.175:0/3800260063 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa7b00051c0 0x7fa7b000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:36.294 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.290+0000 7fa7c9400640 1 -- 172.21.15.175:0/3800260063 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fa7cc0cf7b0 con 0x7fa7b0004800 2024-04-20T11:26:36.294 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.290+0000 7fa7c9400640 1 -- 172.21.15.175:0/3800260063 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b000c110 con 0x7fa7b00039b0 2024-04-20T11:26:36.294 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.290+0000 7fa7c9400640 1 -- 172.21.15.175:0/3800260063 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa7b000c310 con 0x7fa7b00051c0 2024-04-20T11:26:36.294 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.290+0000 7fa7c9e00640 1 --2- 172.21.15.175:0/3800260063 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7b00039b0 0x7fa7b0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:36.294 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.290+0000 7fa7ca800640 1 --2- 172.21.15.175:0/3800260063 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa7b00051c0 0x7fa7b000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:36.294 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.291+0000 7fa7c1400640 1 --2- 172.21.15.175:0/3800260063 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa7b0004800 0x7fa7b0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:36.294 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.291+0000 7fa7c9e00640 1 -- 172.21.15.175:0/3800260063 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa7b00051c0 msgr2=0x7fa7b000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:36.294 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.291+0000 7fa7c9e00640 1 --2- 172.21.15.175:0/3800260063 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa7b00051c0 0x7fa7b000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:36.294 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.291+0000 7fa7c9e00640 1 -- 172.21.15.175:0/3800260063 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa7b0004800 msgr2=0x7fa7b0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:36.294 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.291+0000 7fa7c9e00640 1 --2- 172.21.15.175:0/3800260063 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa7b0004800 0x7fa7b0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:36.294 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.291+0000 7fa7c9e00640 1 -- 172.21.15.175:0/3800260063 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7b0003520 con 0x7fa7b00039b0 2024-04-20T11:26:36.294 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.291+0000 7fa7c1400640 1 --2- 172.21.15.175:0/3800260063 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa7b0004800 0x7fa7b0004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:26:36.295 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.291+0000 7fa7ca800640 1 --2- 172.21.15.175:0/3800260063 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa7b00051c0 0x7fa7b000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:26:36.295 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.291+0000 7fa7ca800640 1 -- 172.21.15.175:0/3800260063 reap_dead start 2024-04-20T11:26:36.295 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.292+0000 7fa7c9e00640 1 --2- 172.21.15.175:0/3800260063 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7b00039b0 0x7fa7b0003dd0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc001750 tx=0x7fa7b0005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5e31bf16af603c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:36.295 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.292+0000 7fa7c9400640 1 -- 172.21.15.175:0/3800260063 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7bc001370 con 0x7fa7b00039b0 2024-04-20T11:26:36.295 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.292+0000 7fa7c9400640 1 -- 172.21.15.175:0/3800260063 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7bc0052d0 con 0x7fa7b00039b0 2024-04-20T11:26:36.295 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.292+0000 7fa7c9400640 1 -- 172.21.15.175:0/3800260063 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7bc003830 con 0x7fa7b00039b0 2024-04-20T11:26:36.295 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.292+0000 7fa7d0a00640 1 -- 172.21.15.175:0/3800260063 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7b00039b0 msgr2=0x7fa7b0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:36.295 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.292+0000 7fa7d0a00640 1 --2- 172.21.15.175:0/3800260063 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7b00039b0 0x7fa7b0003dd0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc001750 tx=0x7fa7b0005830 comp rx=0 tx=0).stop 2024-04-20T11:26:36.295 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.293+0000 7fa7d0a00640 1 -- 172.21.15.175:0/3800260063 shutdown_connections 2024-04-20T11:26:36.295 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.293+0000 7fa7d0a00640 1 --2- 172.21.15.175:0/3800260063 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7b00039b0 0x7fa7b0003dd0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:36.295 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.293+0000 7fa7d0a00640 1 -- 172.21.15.175:0/3800260063 >> 172.21.15.175:0/3800260063 conn(0x7fa7cc08bfd0 msgr2=0x7fa7cc0d4760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:36.295 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.293+0000 7fa7d0a00640 1 -- 172.21.15.175:0/3800260063 shutdown_connections 2024-04-20T11:26:36.295 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.293+0000 7fa7d0a00640 1 -- 172.21.15.175:0/3800260063 wait complete. 2024-04-20T11:26:36.298 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.294+0000 7fa7d0a00640 1 Processor -- start 2024-04-20T11:26:36.298 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.295+0000 7fa7d0a00640 1 -- start start 2024-04-20T11:26:36.298 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.295+0000 7fa7d0a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa7b00039b0 0x7fa7cc0c8180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:36.298 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.295+0000 7fa7d0a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa7cc1f60a0 0x7fa7cc0caa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:36.298 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.295+0000 7fa7d0a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7cc1f7cf0 0x7fa7cc0b61a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:36.298 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.295+0000 7fa7d0a00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fa7cc0d5370 con 0x7fa7cc1f60a0 2024-04-20T11:26:36.298 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.295+0000 7fa7d0a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa7cc0d0050 con 0x7fa7cc1f7cf0 2024-04-20T11:26:36.298 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.295+0000 7fa7d0a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa7cc06ea10 con 0x7fa7b00039b0 2024-04-20T11:26:36.298 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.296+0000 7fa7c9e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa7b00039b0 0x7fa7cc0c8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:36.298 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.296+0000 7fa7c9e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa7b00039b0 0x7fa7cc0c8180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.175:49772/0 (socket says 172.21.15.175:49772) 2024-04-20T11:26:36.299 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.296+0000 7fa7c9e00640 1 -- 172.21.15.175:0/2707479545 learned_addr learned my addr 172.21.15.175:0/2707479545 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-04-20T11:26:36.299 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.296+0000 7fa7ca800640 1 --2- 172.21.15.175:0/2707479545 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7cc1f7cf0 0x7fa7cc0b61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:36.299 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.296+0000 7fa7c1400640 1 --2- 172.21.15.175:0/2707479545 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa7cc1f60a0 0x7fa7cc0caa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:36.299 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.296+0000 7fa7ca800640 1 -- 172.21.15.175:0/2707479545 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa7b00039b0 msgr2=0x7fa7cc0c8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:36.299 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.296+0000 7fa7ca800640 1 --2- 172.21.15.175:0/2707479545 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa7b00039b0 0x7fa7cc0c8180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:36.299 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.296+0000 7fa7ca800640 1 -- 172.21.15.175:0/2707479545 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa7cc1f60a0 msgr2=0x7fa7cc0caa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:36.299 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.296+0000 7fa7ca800640 1 --2- 172.21.15.175:0/2707479545 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa7cc1f60a0 0x7fa7cc0caa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:36.299 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.296+0000 7fa7ca800640 1 -- 172.21.15.175:0/2707479545 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7cc0caf40 con 0x7fa7cc1f7cf0 2024-04-20T11:26:36.299 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.296+0000 7fa7c9e00640 1 --2- 172.21.15.175:0/2707479545 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa7b00039b0 0x7fa7cc0c8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:26:36.299 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.297+0000 7fa7ca800640 1 --2- 172.21.15.175:0/2707479545 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7cc1f7cf0 0x7fa7cc0b61a0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc0c8150 tx=0x7fa7c4004840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:36.299 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.297+0000 7fa7c2a00640 1 -- 172.21.15.175:0/2707479545 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7c4007cd0 con 0x7fa7cc1f7cf0 2024-04-20T11:26:36.299 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.297+0000 7fa7c2a00640 1 -- 172.21.15.175:0/2707479545 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7c4019070 con 0x7fa7cc1f7cf0 2024-04-20T11:26:36.299 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.297+0000 7fa7d0a00640 1 -- 172.21.15.175:0/2707479545 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7cc0c8760 con 0x7fa7cc1f7cf0 2024-04-20T11:26:36.299 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.297+0000 7fa7c2a00640 1 -- 172.21.15.175:0/2707479545 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7c4014650 con 0x7fa7cc1f7cf0 2024-04-20T11:26:36.300 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.297+0000 7fa7d0a00640 1 -- 172.21.15.175:0/2707479545 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7cc0c8d20 con 0x7fa7cc1f7cf0 2024-04-20T11:26:36.300 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.297+0000 7fa7c2a00640 1 -- 172.21.15.175:0/2707479545 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa7c40147f0 con 0x7fa7cc1f7cf0 2024-04-20T11:26:36.300 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.297+0000 7fa7c2a00640 1 -- 172.21.15.175:0/2707479545 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7fa7c401e460 con 0x7fa7cc1f7cf0 2024-04-20T11:26:36.300 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.298+0000 7fa7d0a00640 1 -- 172.21.15.175:0/2707479545 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7cc0d3f50 con 0x7fa7cc1f7cf0 2024-04-20T11:26:36.300 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:26:36.300+0000 7f8ae6493040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-20T11:26:36.301 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.301+0000 7fa7c2a00640 1 -- 172.21.15.175:0/2707479545 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7fa7cc0d3f50 con 0x7fa7cc1f7cf0 2024-04-20T11:26:36.346 INFO:tasks.ceph.mgr.y.smithi152.stderr:2024-04-20T11:26:36.346+0000 7fd1f000e040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-20T11:26:36.378 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.379+0000 7fa7d0a00640 1 -- 172.21.15.175:0/2707479545 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd new", "uuid": "d431d09f-6287-457c-bb3f-7f521756de61", "id": 4} v 0) -- 0x7fa7cc069100 con 0x7fa7cc1f7cf0 2024-04-20T11:26:36.381 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:26:36.381+0000 7f8ae6493040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-20T11:26:36.385 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.385+0000 7fa7c2a00640 1 -- 172.21.15.175:0/2707479545 <== mon.1 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d431d09f-6287-457c-bb3f-7f521756de61", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7fa7c4014ae0 con 0x7fa7cc1f7cf0 2024-04-20T11:26:36.386 INFO:teuthology.orchestra.run.smithi175.stdout:4 2024-04-20T11:26:36.387 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.387+0000 7fa7d0a00640 1 -- 172.21.15.175:0/2707479545 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7cc1f7cf0 msgr2=0x7fa7cc0b61a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:36.387 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.387+0000 7fa7d0a00640 1 --2- 172.21.15.175:0/2707479545 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7cc1f7cf0 0x7fa7cc0b61a0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc0c8150 tx=0x7fa7c4004840 comp rx=0 tx=0).stop 2024-04-20T11:26:36.387 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.387+0000 7fa7d0a00640 1 -- 172.21.15.175:0/2707479545 shutdown_connections 2024-04-20T11:26:36.387 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.387+0000 7fa7d0a00640 1 --2- 172.21.15.175:0/2707479545 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7cc1f7cf0 0x7fa7cc0b61a0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:36.387 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.387+0000 7fa7d0a00640 1 --2- 172.21.15.175:0/2707479545 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa7cc1f60a0 0x7fa7cc0caa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:36.387 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.387+0000 7fa7d0a00640 1 --2- 172.21.15.175:0/2707479545 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa7b00039b0 0x7fa7cc0c8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:36.387 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.387+0000 7fa7d0a00640 1 -- 172.21.15.175:0/2707479545 >> 172.21.15.175:0/2707479545 conn(0x7fa7cc08bfd0 msgr2=0x7fa7cc0b33b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:36.387 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.387+0000 7fa7d0a00640 1 -- 172.21.15.175:0/2707479545 shutdown_connections 2024-04-20T11:26:36.388 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.387+0000 7fa7d0a00640 1 -- 172.21.15.175:0/2707479545 wait complete. 2024-04-20T11:26:36.404 DEBUG:teuthology.orchestra.run.smithi175:> sudo ceph --cluster ceph osd new 38e483ba-3b23-4492-8c5c-b3af28240a11 5 2024-04-20T11:26:36.408 INFO:tasks.ceph.mgr.y.smithi152.stderr:2024-04-20T11:26:36.408+0000 7fd1f000e040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-20T11:26:36.449 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:26:36.450+0000 7f8ae6493040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-20T11:26:36.538 INFO:tasks.ceph.mgr.y.smithi152.stderr:2024-04-20T11:26:36.537+0000 7fd1f000e040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-20T11:26:36.583 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:26:36.583+0000 7f8ae6493040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-20T11:26:36.590 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.590+0000 7f2573e00640 1 Processor -- start 2024-04-20T11:26:36.590 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.590+0000 7f2573e00640 1 -- start start 2024-04-20T11:26:36.592 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.591+0000 7f2573e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f257415e780 0x7f25740b4710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:36.592 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.591+0000 7f2573e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f257406fa60 0x7f257406c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:36.592 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.592+0000 7f2573e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f257406ced0 0x7f257406e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:36.592 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.592+0000 7f2573e00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f257415eb60 con 0x7f257406ced0 2024-04-20T11:26:36.592 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.592+0000 7f2573e00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f25740aece0 con 0x7f257406fa60 2024-04-20T11:26:36.592 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.592+0000 7f2573e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f25740ae9b0 con 0x7f257415e780 2024-04-20T11:26:36.592 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.592+0000 7f2572a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f257415e780 0x7f25740b4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:36.592 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.592+0000 7f256be00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f257406fa60 0x7f257406c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:36.592 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.592+0000 7f2572a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f257415e780 0x7f25740b4710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.175:49784/0 (socket says 172.21.15.175:49784) 2024-04-20T11:26:36.592 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.592+0000 7f2572a00640 1 -- 172.21.15.175:0/1840090155 learned_addr learned my addr 172.21.15.175:0/1840090155 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-04-20T11:26:36.593 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.592+0000 7f2573400640 1 --2- 172.21.15.175:0/1840090155 >> v2:172.21.15.152:3300/0 conn(0x7f257406ced0 0x7f257406e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:36.593 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.592+0000 7f2572a00640 1 -- 172.21.15.175:0/1840090155 >> v2:172.21.15.175:3300/0 conn(0x7f257406fa60 msgr2=0x7f257406c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:36.593 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.592+0000 7f2572a00640 1 --2- 172.21.15.175:0/1840090155 >> v2:172.21.15.175:3300/0 conn(0x7f257406fa60 0x7f257406c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:36.593 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.592+0000 7f2572a00640 1 -- 172.21.15.175:0/1840090155 >> v2:172.21.15.152:3300/0 conn(0x7f257406ced0 msgr2=0x7f257406e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:36.593 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.592+0000 7f2572a00640 1 --2- 172.21.15.175:0/1840090155 >> v2:172.21.15.152:3300/0 conn(0x7f257406ced0 0x7f257406e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:36.593 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.592+0000 7f2572a00640 1 -- 172.21.15.175:0/1840090155 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25740702b0 con 0x7f257415e780 2024-04-20T11:26:36.594 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.593+0000 7f2572a00640 1 --2- 172.21.15.175:0/1840090155 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f257415e780 0x7f25740b4710 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f25740735f0 tx=0x7f256402b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8bd767d3a69f800 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:36.595 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.594+0000 7f2572000640 1 -- 172.21.15.175:0/1840090155 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2564035020 con 0x7f257415e780 2024-04-20T11:26:36.595 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.594+0000 7f2572000640 1 -- 172.21.15.175:0/1840090155 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2564031870 con 0x7f257415e780 2024-04-20T11:26:36.595 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.594+0000 7f2572000640 1 -- 172.21.15.175:0/1840090155 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2564031c40 con 0x7f257415e780 2024-04-20T11:26:36.595 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.594+0000 7f2573e00640 1 -- 172.21.15.175:0/1840090155 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f257415e780 msgr2=0x7f25740b4710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:36.595 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.594+0000 7f2573e00640 1 --2- 172.21.15.175:0/1840090155 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f257415e780 0x7f25740b4710 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f25740735f0 tx=0x7f256402b910 comp rx=0 tx=0).stop 2024-04-20T11:26:36.595 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.595+0000 7f2573e00640 1 -- 172.21.15.175:0/1840090155 shutdown_connections 2024-04-20T11:26:36.595 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.595+0000 7f2573e00640 1 --2- 172.21.15.175:0/1840090155 >> v2:172.21.15.152:3300/0 conn(0x7f257406ced0 0x7f257406e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:36.595 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.595+0000 7f2573e00640 1 --2- 172.21.15.175:0/1840090155 >> v2:172.21.15.175:3300/0 conn(0x7f257406fa60 0x7f257406c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:36.595 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.595+0000 7f2573e00640 1 --2- 172.21.15.175:0/1840090155 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f257415e780 0x7f25740b4710 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:36.595 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.595+0000 7f2573e00640 1 -- 172.21.15.175:0/1840090155 >> 172.21.15.175:0/1840090155 conn(0x7f25740724c0 msgr2=0x7f25740728d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:36.595 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.595+0000 7f2573e00640 1 -- 172.21.15.175:0/1840090155 shutdown_connections 2024-04-20T11:26:36.595 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.595+0000 7f2573e00640 1 -- 172.21.15.175:0/1840090155 wait complete. 2024-04-20T11:26:36.596 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.596+0000 7f2573e00640 1 Processor -- start 2024-04-20T11:26:36.596 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.596+0000 7f2573e00640 1 -- start start 2024-04-20T11:26:36.598 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.597+0000 7f2573e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f257406ced0 0x7f257414a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:36.598 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.597+0000 7f2573e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f257406fa60 0x7f2574155fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:36.598 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.597+0000 7f2573e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f257415e780 0x7f25741564e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:36.598 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.597+0000 7f2573e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f257406fec0 con 0x7f257406ced0 2024-04-20T11:26:36.598 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.597+0000 7f2573e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2574070870 con 0x7f257415e780 2024-04-20T11:26:36.598 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.597+0000 7f2573e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2574070040 con 0x7f257406fa60 2024-04-20T11:26:36.598 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.598+0000 7f256be00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f257406fa60 0x7f2574155fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:36.598 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.598+0000 7f256be00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f257406fa60 0x7f2574155fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.175:49786/0 (socket says 172.21.15.175:49786) 2024-04-20T11:26:36.599 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.598+0000 7f256be00640 1 -- 172.21.15.175:0/364788037 learned_addr learned my addr 172.21.15.175:0/364788037 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-04-20T11:26:36.599 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.598+0000 7f2573400640 1 --2- 172.21.15.175:0/364788037 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f257415e780 0x7f25741564e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:36.599 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.598+0000 7f2572a00640 1 --2- 172.21.15.175:0/364788037 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f257406ced0 0x7f257414a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:36.599 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.598+0000 7f256be00640 1 -- 172.21.15.175:0/364788037 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f257415e780 msgr2=0x7f25741564e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:36.599 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.598+0000 7f256be00640 1 --2- 172.21.15.175:0/364788037 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f257415e780 0x7f25741564e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:36.599 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.598+0000 7f256be00640 1 -- 172.21.15.175:0/364788037 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f257406ced0 msgr2=0x7f257414a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:36.599 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.598+0000 7f256be00640 1 --2- 172.21.15.175:0/364788037 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f257406ced0 0x7f257414a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:36.599 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.598+0000 7f256be00640 1 -- 172.21.15.175:0/364788037 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25741f7c50 con 0x7f257406fa60 2024-04-20T11:26:36.600 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.598+0000 7f256be00640 1 --2- 172.21.15.175:0/364788037 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f257406fa60 0x7f2574155fa0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f255c00e9b0 tx=0x7f255c00ee70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:36.600 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.599+0000 7f256b400640 1 -- 172.21.15.175:0/364788037 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f255c017070 con 0x7f257406fa60 2024-04-20T11:26:36.600 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.599+0000 7f256b400640 1 -- 172.21.15.175:0/364788037 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f255c00c9a0 con 0x7f257406fa60 2024-04-20T11:26:36.600 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.599+0000 7f2573e00640 1 -- 172.21.15.175:0/364788037 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25741f7f30 con 0x7f257406fa60 2024-04-20T11:26:36.600 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.599+0000 7f256b400640 1 -- 172.21.15.175:0/364788037 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f255c00cd90 con 0x7f257406fa60 2024-04-20T11:26:36.600 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.599+0000 7f2573e00640 1 -- 172.21.15.175:0/364788037 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f25741f84f0 con 0x7f257406fa60 2024-04-20T11:26:36.600 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.599+0000 7f256b400640 1 -- 172.21.15.175:0/364788037 <== mon.2 v2:172.21.15.175:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f255c01a070 con 0x7f257406fa60 2024-04-20T11:26:36.600 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.599+0000 7f256b400640 1 -- 172.21.15.175:0/364788037 <== mon.2 v2:172.21.15.175:3301/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7f255c0139f0 con 0x7f257406fa60 2024-04-20T11:26:36.601 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.600+0000 7f2573e00640 1 -- 172.21.15.175:0/364788037 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2574075e60 con 0x7f257406fa60 2024-04-20T11:26:36.602 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.602+0000 7f256b400640 1 -- 172.21.15.175:0/364788037 <== mon.2 v2:172.21.15.175:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7f255c013c90 con 0x7f257406fa60 2024-04-20T11:26:36.681 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.681+0000 7f2573e00640 1 -- 172.21.15.175:0/364788037 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "osd new", "uuid": "38e483ba-3b23-4492-8c5c-b3af28240a11", "id": 5} v 0) -- 0x7f25740694e0 con 0x7f257406fa60 2024-04-20T11:26:36.688 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.688+0000 7f256b400640 1 -- 172.21.15.175:0/364788037 <== mon.2 v2:172.21.15.175:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "38e483ba-3b23-4492-8c5c-b3af28240a11", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7f255c013470 con 0x7f257406fa60 2024-04-20T11:26:36.688 INFO:teuthology.orchestra.run.smithi175.stdout:5 2024-04-20T11:26:36.689 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.689+0000 7f2573e00640 1 -- 172.21.15.175:0/364788037 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f257406fa60 msgr2=0x7f2574155fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:36.689 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.689+0000 7f2573e00640 1 --2- 172.21.15.175:0/364788037 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f257406fa60 0x7f2574155fa0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f255c00e9b0 tx=0x7f255c00ee70 comp rx=0 tx=0).stop 2024-04-20T11:26:36.689 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.689+0000 7f2573e00640 1 -- 172.21.15.175:0/364788037 shutdown_connections 2024-04-20T11:26:36.689 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.689+0000 7f2573e00640 1 --2- 172.21.15.175:0/364788037 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f257415e780 0x7f25741564e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:36.689 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.689+0000 7f2573e00640 1 --2- 172.21.15.175:0/364788037 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f257406fa60 0x7f2574155fa0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:36.689 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.689+0000 7f2573e00640 1 --2- 172.21.15.175:0/364788037 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f257406ced0 0x7f257414a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:36.690 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.689+0000 7f2573e00640 1 -- 172.21.15.175:0/364788037 >> 172.21.15.175:0/364788037 conn(0x7f25740724c0 msgr2=0x7f257406c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:36.690 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.689+0000 7f2573e00640 1 -- 172.21.15.175:0/364788037 shutdown_connections 2024-04-20T11:26:36.690 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.689+0000 7f2573e00640 1 -- 172.21.15.175:0/364788037 wait complete. 2024-04-20T11:26:36.706 DEBUG:teuthology.orchestra.run.smithi175:> sudo ceph --cluster ceph osd new 5271d8ec-ba0e-4387-8f4b-113ef144f40e 6 2024-04-20T11:26:36.926 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.926+0000 7fc7ff200640 1 Processor -- start 2024-04-20T11:26:36.926 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.926+0000 7fc7ff200640 1 -- start start 2024-04-20T11:26:36.928 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.927+0000 7fc7ff200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fc80015e880 0x7fc8000b5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:36.928 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.927+0000 7fc7ff200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc8000d5190 0x7fc8000b6530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:36.928 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.927+0000 7fc7ff200640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8000d29a0 con 0x7fc8000d4c80 2024-04-20T11:26:36.928 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.927+0000 7fc7ff200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fc8000d32d0 con 0x7fc80015e880 2024-04-20T11:26:36.928 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.927+0000 7fc7ff200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc8000d2c80 con 0x7fc8000d5190 2024-04-20T11:26:36.928 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.928+0000 7fc7fde00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fc80015e880 0x7fc8000b5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:36.928 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.928+0000 7fc7fd400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc8000d5190 0x7fc8000b6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:36.929 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.928+0000 7fc7fd400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc8000d5190 0x7fc8000b6530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.175:49792/0 (socket says 172.21.15.175:49792) 2024-04-20T11:26:36.929 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.928+0000 7fc7fd400640 1 -- 172.21.15.175:0/133347831 learned_addr learned my addr 172.21.15.175:0/133347831 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-04-20T11:26:36.929 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.928+0000 7fc7fd400640 1 -- 172.21.15.175:0/133347831 >> v2:172.21.15.175:3300/0 conn(0x7fc80015e880 msgr2=0x7fc8000b5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:36.929 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.928+0000 7fc7fd400640 1 --2- 172.21.15.175:0/133347831 >> v2:172.21.15.175:3300/0 conn(0x7fc80015e880 0x7fc8000b5ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:36.929 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.928+0000 7fc7fd400640 1 -- 172.21.15.175:0/133347831 >> v1:172.21.15.152:6789/0 conn(0x7fc8000d4c80 legacy=0x7fc800162290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:36.929 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.928+0000 7fc7fd400640 1 -- 172.21.15.175:0/133347831 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8000d2f60 con 0x7fc8000d5190 2024-04-20T11:26:36.929 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.928+0000 7fc7fde00640 1 --2- 172.21.15.175:0/133347831 >> v2:172.21.15.175:3300/0 conn(0x7fc80015e880 0x7fc8000b5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:26:36.929 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.929+0000 7fc7fd400640 1 --2- 172.21.15.175:0/133347831 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc8000d5190 0x7fc8000b6530 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec009850 tx=0x7fc7ec02b7c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d7caa86914e28fd server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:36.930 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.930+0000 7fc7fca00640 1 -- 172.21.15.175:0/133347831 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7ec038070 con 0x7fc8000d5190 2024-04-20T11:26:36.930 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.930+0000 7fc7fca00640 1 -- 172.21.15.175:0/133347831 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7ec004030 con 0x7fc8000d5190 2024-04-20T11:26:36.930 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.930+0000 7fc7fca00640 1 -- 172.21.15.175:0/133347831 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7ec004460 con 0x7fc8000d5190 2024-04-20T11:26:36.931 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.930+0000 7fc7ff200640 1 -- 172.21.15.175:0/133347831 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc8000d5190 msgr2=0x7fc8000b6530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:36.931 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.930+0000 7fc7ff200640 1 --2- 172.21.15.175:0/133347831 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc8000d5190 0x7fc8000b6530 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec009850 tx=0x7fc7ec02b7c0 comp rx=0 tx=0).stop 2024-04-20T11:26:36.931 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.930+0000 7fc7ff200640 1 -- 172.21.15.175:0/133347831 shutdown_connections 2024-04-20T11:26:36.931 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.930+0000 7fc7ff200640 1 --2- 172.21.15.175:0/133347831 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc8000d5190 0x7fc8000b6530 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:36.931 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.930+0000 7fc7ff200640 1 --2- 172.21.15.175:0/133347831 >> v2:172.21.15.175:3300/0 conn(0x7fc80015e880 0x7fc8000b5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:36.931 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.930+0000 7fc7ff200640 1 -- 172.21.15.175:0/133347831 >> 172.21.15.175:0/133347831 conn(0x7fc80008bfd0 msgr2=0x7fc8000d4570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:36.931 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.930+0000 7fc7ff200640 1 -- 172.21.15.175:0/133347831 shutdown_connections 2024-04-20T11:26:36.931 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.930+0000 7fc7ff200640 1 -- 172.21.15.175:0/133347831 wait complete. 2024-04-20T11:26:36.932 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.932+0000 7fc7ff200640 1 Processor -- start 2024-04-20T11:26:36.932 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.932+0000 7fc7ff200640 1 -- start start 2024-04-20T11:26:36.934 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.933+0000 7fc7ff200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc8000d4c80 0x7fc800150bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:36.934 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.933+0000 7fc7ff200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc8000d5190 0x7fc8000cc0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:36.934 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.933+0000 7fc7ff200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc80015e880 0x7fc80015f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:36.934 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.933+0000 7fc7ff200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fc8000cf1f0 con 0x7fc8000d4c80 2024-04-20T11:26:36.934 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.933+0000 7fc7ff200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc8000cf070 con 0x7fc80015e880 2024-04-20T11:26:36.934 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.933+0000 7fc7ff200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc8000d3760 con 0x7fc8000d5190 2024-04-20T11:26:36.934 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.933+0000 7fc7fd400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc8000d5190 0x7fc8000cc0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:36.934 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.933+0000 7fc7fd400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc8000d5190 0x7fc8000cc0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.175:49800/0 (socket says 172.21.15.175:49800) 2024-04-20T11:26:36.934 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.933+0000 7fc7fd400640 1 -- 172.21.15.175:0/13563392 learned_addr learned my addr 172.21.15.175:0/13563392 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-04-20T11:26:36.934 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.933+0000 7fc7fe800640 1 --2- 172.21.15.175:0/13563392 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc80015e880 0x7fc80015f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:36.934 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.934+0000 7fc7fde00640 1 --2- 172.21.15.175:0/13563392 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc8000d4c80 0x7fc800150bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:36.934 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.934+0000 7fc7fd400640 1 -- 172.21.15.175:0/13563392 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc80015e880 msgr2=0x7fc80015f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:36.935 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.934+0000 7fc7fd400640 1 --2- 172.21.15.175:0/13563392 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc80015e880 0x7fc80015f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:36.935 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.934+0000 7fc7fd400640 1 -- 172.21.15.175:0/13563392 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc8000d4c80 msgr2=0x7fc800150bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:36.935 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.934+0000 7fc7fd400640 1 --2- 172.21.15.175:0/13563392 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc8000d4c80 0x7fc800150bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:36.935 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.934+0000 7fc7fd400640 1 -- 172.21.15.175:0/13563392 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8000ccbb0 con 0x7fc8000d5190 2024-04-20T11:26:36.936 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.934+0000 7fc7fd400640 1 --2- 172.21.15.175:0/13563392 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc8000d5190 0x7fc8000cc0f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec036040 tx=0x7fc7ec0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:36.936 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.934+0000 7fc7f6000640 1 -- 172.21.15.175:0/13563392 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7ec038050 con 0x7fc8000d5190 2024-04-20T11:26:36.936 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.934+0000 7fc7f6000640 1 -- 172.21.15.175:0/13563392 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7ec039040 con 0x7fc8000d5190 2024-04-20T11:26:36.936 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.934+0000 7fc7ff200640 1 -- 172.21.15.175:0/13563392 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc800160190 con 0x7fc8000d5190 2024-04-20T11:26:36.936 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.934+0000 7fc7f6000640 1 -- 172.21.15.175:0/13563392 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7ec003ac0 con 0x7fc8000d5190 2024-04-20T11:26:36.936 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.935+0000 7fc7ff200640 1 -- 172.21.15.175:0/13563392 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc800160750 con 0x7fc8000d5190 2024-04-20T11:26:36.936 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.935+0000 7fc7f6000640 1 -- 172.21.15.175:0/13563392 <== mon.2 v2:172.21.15.175:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc7ec046020 con 0x7fc8000d5190 2024-04-20T11:26:36.936 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.935+0000 7fc7f6000640 1 -- 172.21.15.175:0/13563392 <== mon.2 v2:172.21.15.175:3301/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7fc7ec006630 con 0x7fc8000d5190 2024-04-20T11:26:36.936 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.936+0000 7fc7ff200640 1 -- 172.21.15.175:0/13563392 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc800075e60 con 0x7fc8000d5190 2024-04-20T11:26:36.938 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:36.938+0000 7fc7f6000640 1 -- 172.21.15.175:0/13563392 <== mon.2 v2:172.21.15.175:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7fc7ec003c60 con 0x7fc8000d5190 2024-04-20T11:26:36.998 INFO:tasks.ceph.mgr.y.smithi152.stderr:2024-04-20T11:26:36.997+0000 7fd1f000e040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-20T11:26:37.017 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:37.017+0000 7fc7ff200640 1 -- 172.21.15.175:0/13563392 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "osd new", "uuid": "5271d8ec-ba0e-4387-8f4b-113ef144f40e", "id": 6} v 0) -- 0x7fc800075070 con 0x7fc8000d5190 2024-04-20T11:26:37.027 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:37.027+0000 7fc7f6000640 1 -- 172.21.15.175:0/13563392 <== mon.2 v2:172.21.15.175:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "5271d8ec-ba0e-4387-8f4b-113ef144f40e", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7fc7ec006040 con 0x7fc8000d5190 2024-04-20T11:26:37.027 INFO:teuthology.orchestra.run.smithi175.stdout:6 2024-04-20T11:26:37.029 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:37.028+0000 7fc7ff200640 1 -- 172.21.15.175:0/13563392 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc8000d5190 msgr2=0x7fc8000cc0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:37.029 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:37.028+0000 7fc7ff200640 1 --2- 172.21.15.175:0/13563392 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc8000d5190 0x7fc8000cc0f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec036040 tx=0x7fc7ec0046c0 comp rx=0 tx=0).stop 2024-04-20T11:26:37.029 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:37.029+0000 7fc7ff200640 1 -- 172.21.15.175:0/13563392 shutdown_connections 2024-04-20T11:26:37.029 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:37.029+0000 7fc7ff200640 1 --2- 172.21.15.175:0/13563392 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc80015e880 0x7fc80015f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:37.029 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:37.029+0000 7fc7ff200640 1 --2- 172.21.15.175:0/13563392 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc8000d5190 0x7fc8000cc0f0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:37.029 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:37.029+0000 7fc7ff200640 1 --2- 172.21.15.175:0/13563392 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc8000d4c80 0x7fc800150bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:37.029 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:37.029+0000 7fc7ff200640 1 -- 172.21.15.175:0/13563392 >> 172.21.15.175:0/13563392 conn(0x7fc80008bfd0 msgr2=0x7fc8001532c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:37.029 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:37.029+0000 7fc7ff200640 1 -- 172.21.15.175:0/13563392 shutdown_connections 2024-04-20T11:26:37.029 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:37.029+0000 7fc7ff200640 1 -- 172.21.15.175:0/13563392 wait complete. 2024-04-20T11:26:37.046 DEBUG:teuthology.orchestra.run.smithi175:> sudo ceph --cluster ceph osd new f2e6b6f3-b14b-43fa-a2f3-c253a43accfa 7 2024-04-20T11:26:37.065 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:26:37.065+0000 7f8ae6493040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-20T11:26:37.227 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:37.227+0000 7f8593200640 1 Processor -- start 2024-04-20T11:26:37.227 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:37.227+0000 7f8593200640 1 -- start start 2024-04-20T11:26:37.229 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:37.228+0000 7f8593200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f859415e880 0x7f85940b5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:37.229 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:37.228+0000 7f8593200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f85940d5190 0x7f85940b6530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:37.229 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:37.228+0000 7f8593200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85940d29a0 con 0x7f85940d4c80 2024-04-20T11:26:37.229 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:37.228+0000 7f8593200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f85940d32d0 con 0x7f859415e880 2024-04-20T11:26:37.229 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:37.228+0000 7f8593200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f85940d2c80 con 0x7f85940d5190 2024-04-20T11:26:37.230 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:37.229+0000 7f8591400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f85940d5190 0x7f85940b6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:37.230 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:37.229+0000 7f8591400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f85940d5190 0x7f85940b6530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.175:49812/0 (socket says 172.21.15.175:49812) 2024-04-20T11:26:37.230 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:37.229+0000 7f8592800640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7f85940d4c80 0x7f8594162290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.175:38654/0 (socket says 172.21.15.175:38654) 2024-04-20T11:26:37.230 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:37.229+0000 7f8591400640 1 -- 172.21.15.175:0/200934991 learned_addr learned my addr 172.21.15.175:0/200934991 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-04-20T11:26:37.230 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:37.229+0000 7f8591e00640 1 --2- 172.21.15.175:0/200934991 >> v2:172.21.15.152:3300/0 conn(0x7f859415e880 0x7f85940b5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:37.230 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:37.229+0000 7f8591400640 1 -- 172.21.15.175:0/200934991 >> v2:172.21.15.152:3300/0 conn(0x7f859415e880 msgr2=0x7f85940b5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:37.230 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:37.229+0000 7f8591400640 1 --2- 172.21.15.175:0/200934991 >> v2:172.21.15.152:3300/0 conn(0x7f859415e880 0x7f85940b5ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:37.230 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:37.229+0000 7f8590a00640 1 -- 172.21.15.175:0/200934991 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2591535783 0 0) 0x7f85940d29a0 con 0x7f85940d4c80 2024-04-20T11:26:37.230 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:37.229+0000 7f8591400640 1 -- 172.21.15.175:0/200934991 >> v1:172.21.15.175:6789/0 conn(0x7f85940d4c80 legacy=0x7f8594162290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:37.230 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:37.229+0000 7f8591400640 1 -- 172.21.15.175:0/200934991 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85940d2f60 con 0x7f85940d5190 2024-04-20T11:26:37.231 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:37.231+0000 7f8591400640 1 --2- 172.21.15.175:0/200934991 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f85940d5190 0x7f85940b6530 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8580009850 tx=0x7f858002b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f010e9d58c8fa099 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:37.231 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:37.231+0000 7f8590a00640 1 -- 172.21.15.175:0/200934991 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8580037070 con 0x7f85940d5190 2024-04-20T11:26:37.231 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:37.231+0000 7f8590a00640 1 -- 172.21.15.175:0/200934991 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8580004030 con 0x7f85940d5190 2024-04-20T11:26:37.231 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:37.231+0000 7f8590a00640 1 -- 172.21.15.175:0/200934991 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8580004420 con 0x7f85940d5190 2024-04-20T11:26:37.232 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:37.232+0000 7f8593200640 1 -- 172.21.15.175:0/200934991 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f85940d5190 msgr2=0x7f85940b6530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:37.232 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:37.232+0000 7f8593200640 1 --2- 172.21.15.175:0/200934991 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f85940d5190 0x7f85940b6530 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8580009850 tx=0x7f858002b840 comp rx=0 tx=0).stop 2024-04-20T11:26:37.232 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:37.232+0000 7f8593200640 1 -- 172.21.15.175:0/200934991 shutdown_connections 2024-04-20T11:26:37.232 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:37.232+0000 7f8593200640 1 --2- 172.21.15.175:0/200934991 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f85940d5190 0x7f85940b6530 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:37.232 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:37.232+0000 7f8593200640 1 --2- 172.21.15.175:0/200934991 >> v2:172.21.15.152:3300/0 conn(0x7f859415e880 0x7f85940b5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:37.233 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:37.232+0000 7f8593200640 1 -- 172.21.15.175:0/200934991 >> 172.21.15.175:0/200934991 conn(0x7f859408bfd0 msgr2=0x7f85940d4570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:37.233 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:37.232+0000 7f8593200640 1 -- 172.21.15.175:0/200934991 shutdown_connections 2024-04-20T11:26:37.233 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:37.232+0000 7f8593200640 1 -- 172.21.15.175:0/200934991 wait complete. 2024-04-20T11:26:37.233 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:37.234+0000 7f8593200640 1 Processor -- start 2024-04-20T11:26:37.234 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:37.234+0000 7f8593200640 1 -- start start 2024-04-20T11:26:37.236 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:37.235+0000 7f8593200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f85940d4c80 0x7f85940b9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:37.236 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:37.235+0000 7f8593200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85940d5190 0x7f85940b5c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:37.236 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:37.235+0000 7f8593200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f859415e880 0x7f85940cbba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:37.236 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:37.235+0000 7f8593200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f85940cf1f0 con 0x7f85940d4c80 2024-04-20T11:26:37.236 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:37.235+0000 7f8593200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f85940cf070 con 0x7f85940d5190 2024-04-20T11:26:37.237 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:37.235+0000 7f8593200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f85940d3760 con 0x7f859415e880 2024-04-20T11:26:37.237 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:37.235+0000 7f8592800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f859415e880 0x7f85940cbba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:37.237 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:37.235+0000 7f8591400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85940d5190 0x7f85940b5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:37.237 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:37.235+0000 7f8591400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85940d5190 0x7f85940b5c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:58884/0 (socket says 172.21.15.175:58884) 2024-04-20T11:26:37.237 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:37.235+0000 7f8591400640 1 -- 172.21.15.175:0/2321814595 learned_addr learned my addr 172.21.15.175:0/2321814595 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-04-20T11:26:37.237 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:37.235+0000 7f8591e00640 1 --2- 172.21.15.175:0/2321814595 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f85940d4c80 0x7f85940b9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:37.237 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:37.235+0000 7f8591400640 1 -- 172.21.15.175:0/2321814595 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f859415e880 msgr2=0x7f85940cbba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:37.237 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:37.235+0000 7f8591400640 1 --2- 172.21.15.175:0/2321814595 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f859415e880 0x7f85940cbba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:37.237 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:37.235+0000 7f8591400640 1 -- 172.21.15.175:0/2321814595 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f85940d4c80 msgr2=0x7f85940b9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:37.237 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:37.235+0000 7f8591400640 1 --2- 172.21.15.175:0/2321814595 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f85940d4c80 0x7f85940b9aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:37.237 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:37.235+0000 7f8591400640 1 -- 172.21.15.175:0/2321814595 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85940b4bb0 con 0x7f85940d5190 2024-04-20T11:26:37.237 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:37.235+0000 7f8592800640 1 --2- 172.21.15.175:0/2321814595 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f859415e880 0x7f85940cbba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:26:37.237 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:37.236+0000 7f8591400640 1 --2- 172.21.15.175:0/2321814595 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85940d5190 0x7f85940b5c50 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f8580035040 tx=0x7f85800046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:37.238 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:37.236+0000 7f858a000640 1 -- 172.21.15.175:0/2321814595 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8580037050 con 0x7f85940d5190 2024-04-20T11:26:37.238 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:37.236+0000 7f858a000640 1 -- 172.21.15.175:0/2321814595 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8580043070 con 0x7f85940d5190 2024-04-20T11:26:37.238 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:37.236+0000 7f858a000640 1 -- 172.21.15.175:0/2321814595 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8580003e20 con 0x7f85940d5190 2024-04-20T11:26:37.238 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:37.236+0000 7f8593200640 1 -- 172.21.15.175:0/2321814595 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85940b4e90 con 0x7f85940d5190 2024-04-20T11:26:37.238 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:37.236+0000 7f8593200640 1 -- 172.21.15.175:0/2321814595 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85940b5450 con 0x7f85940d5190 2024-04-20T11:26:37.238 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:37.236+0000 7f858a000640 1 -- 172.21.15.175:0/2321814595 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8580006040 con 0x7f85940d5190 2024-04-20T11:26:37.238 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:37.236+0000 7f858a000640 1 -- 172.21.15.175:0/2321814595 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7f858002f020 con 0x7f85940d5190 2024-04-20T11:26:37.238 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:37.237+0000 7f8593200640 1 -- 172.21.15.175:0/2321814595 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8594075e60 con 0x7f85940d5190 2024-04-20T11:26:37.239 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:37.240+0000 7f858a000640 1 -- 172.21.15.175:0/2321814595 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7f8580006330 con 0x7f85940d5190 2024-04-20T11:26:37.291 INFO:tasks.ceph.mgr.y.smithi152.stderr:2024-04-20T11:26:37.291+0000 7fd1f000e040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-20T11:26:37.318 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:37.318+0000 7f8593200640 1 -- 172.21.15.175:0/2321814595 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd new", "uuid": "f2e6b6f3-b14b-43fa-a2f3-c253a43accfa", "id": 7} v 0) -- 0x7f859415c320 con 0x7f85940d5190 2024-04-20T11:26:37.329 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:37.329+0000 7f858a000640 1 -- 172.21.15.175:0/2321814595 <== mon.1 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f2e6b6f3-b14b-43fa-a2f3-c253a43accfa", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7f8580006510 con 0x7f85940d5190 2024-04-20T11:26:37.330 INFO:teuthology.orchestra.run.smithi175.stdout:7 2024-04-20T11:26:37.331 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:37.330+0000 7f8593200640 1 -- 172.21.15.175:0/2321814595 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85940d5190 msgr2=0x7f85940b5c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:37.331 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:37.330+0000 7f8593200640 1 --2- 172.21.15.175:0/2321814595 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85940d5190 0x7f85940b5c50 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f8580035040 tx=0x7f85800046c0 comp rx=0 tx=0).stop 2024-04-20T11:26:37.331 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:37.331+0000 7f8593200640 1 -- 172.21.15.175:0/2321814595 shutdown_connections 2024-04-20T11:26:37.331 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:37.331+0000 7f8593200640 1 --2- 172.21.15.175:0/2321814595 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f859415e880 0x7f85940cbba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:37.331 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:37.331+0000 7f8593200640 1 --2- 172.21.15.175:0/2321814595 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85940d5190 0x7f85940b5c50 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:37.331 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:37.331+0000 7f8593200640 1 --2- 172.21.15.175:0/2321814595 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f85940d4c80 0x7f85940b9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:37.331 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:37.331+0000 7f8593200640 1 -- 172.21.15.175:0/2321814595 >> 172.21.15.175:0/2321814595 conn(0x7f859408bfd0 msgr2=0x7f859406ca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:37.331 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:37.331+0000 7f8593200640 1 -- 172.21.15.175:0/2321814595 shutdown_connections 2024-04-20T11:26:37.332 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:26:37.331+0000 7f8593200640 1 -- 172.21.15.175:0/2321814595 wait complete. 2024-04-20T11:26:37.348 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-20T11:26:37.348 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-04-20T11:26:37.355 INFO:tasks.ceph.osd.0:Started 2024-04-20T11:26:37.356 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-20T11:26:37.356 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-20T11:26:37.356 INFO:tasks.ceph.mgr.y.smithi152.stderr:2024-04-20T11:26:37.352+0000 7fd1f000e040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-20T11:26:37.364 INFO:tasks.ceph.osd.1:Started 2024-04-20T11:26:37.364 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-20T11:26:37.364 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-20T11:26:37.368 INFO:tasks.ceph.osd.2:Started 2024-04-20T11:26:37.368 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-20T11:26:37.369 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-04-20T11:26:37.374 INFO:tasks.ceph.osd.3:Started 2024-04-20T11:26:37.374 INFO:tasks.ceph.osd.4:Restarting daemon 2024-04-20T11:26:37.375 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-04-20T11:26:37.375 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:26:37.375+0000 7f8ae6493040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-20T11:26:37.379 INFO:tasks.ceph.osd.4:Started 2024-04-20T11:26:37.379 INFO:tasks.ceph.osd.5:Restarting daemon 2024-04-20T11:26:37.379 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-04-20T11:26:37.383 INFO:tasks.ceph.osd.5:Started 2024-04-20T11:26:37.383 INFO:tasks.ceph.osd.6:Restarting daemon 2024-04-20T11:26:37.383 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-04-20T11:26:37.388 INFO:tasks.ceph.osd.6:Started 2024-04-20T11:26:37.388 INFO:tasks.ceph.osd.7:Restarting daemon 2024-04-20T11:26:37.388 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-04-20T11:26:37.393 INFO:tasks.ceph.osd.7:Started 2024-04-20T11:26:37.393 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:26:37.459 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:26:37.459+0000 7f8ae6493040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-20T11:26:37.542 INFO:tasks.ceph.mgr.y.smithi152.stderr:2024-04-20T11:26:37.541+0000 7fd1f000e040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-20T11:26:37.623 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:26:37.623+0000 7f8ae6493040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-20T11:26:37.627 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:37.627+0000 7f3554c00640 1 Processor -- start 2024-04-20T11:26:37.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:37.627+0000 7f3554c00640 1 -- start start 2024-04-20T11:26:37.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:37.628+0000 7f3554c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f355015eb80 0x7f35500b79a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:37.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:37.628+0000 7f3554c00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f35500d1810 0x7f35500d1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:37.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:37.628+0000 7f3554c00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35500b8330 con 0x7f35500ba520 2024-04-20T11:26:37.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:37.628+0000 7f3554c00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f35500b8610 con 0x7f35500d1810 2024-04-20T11:26:37.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:37.628+0000 7f3554c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f35500b88f0 con 0x7f355015eb80 2024-04-20T11:26:37.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:37.629+0000 7f354e800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f35500d1810 0x7f35500d1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:37.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:37.629+0000 7f354e800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f35500d1810 0x7f35500d1ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:37910/0 (socket says 172.21.15.152:37910) 2024-04-20T11:26:37.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:37.629+0000 7f354e800640 1 -- 172.21.15.152:0/2995988244 learned_addr learned my addr 172.21.15.152:0/2995988244 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:26:37.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:37.629+0000 7f354de00640 1 --2- 172.21.15.152:0/2995988244 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f355015eb80 0x7f35500b79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:37.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:37.629+0000 7f354e800640 1 -- 172.21.15.152:0/2995988244 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f355015eb80 msgr2=0x7f35500b79a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:37.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:37.629+0000 7f354e800640 1 --2- 172.21.15.152:0/2995988244 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f355015eb80 0x7f35500b79a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:37.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:37.629+0000 7f354e800640 1 -- 172.21.15.152:0/2995988244 >> v1:172.21.15.175:6789/0 conn(0x7f35500ba520 legacy=0x7f35500d28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:37.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:37.629+0000 7f354e800640 1 -- 172.21.15.152:0/2995988244 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35500b8bd0 con 0x7f35500d1810 2024-04-20T11:26:37.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:37.630+0000 7f354e800640 1 --2- 172.21.15.152:0/2995988244 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f35500d1810 0x7f35500d1ca0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3548009e50 tx=0x7f354802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92a6bdf6376a6eba server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:37.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:37.630+0000 7f354d400640 1 -- 172.21.15.152:0/2995988244 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f354802ca70 con 0x7f35500d1810 2024-04-20T11:26:37.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:37.631+0000 7f354d400640 1 -- 172.21.15.152:0/2995988244 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f354802cc10 con 0x7f35500d1810 2024-04-20T11:26:37.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:37.631+0000 7f354d400640 1 -- 172.21.15.152:0/2995988244 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35480343f0 con 0x7f35500d1810 2024-04-20T11:26:37.632 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:37.631+0000 7f3554c00640 1 -- 172.21.15.152:0/2995988244 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f35500d1810 msgr2=0x7f35500d1ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:37.632 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:37.631+0000 7f3554c00640 1 --2- 172.21.15.152:0/2995988244 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f35500d1810 0x7f35500d1ca0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3548009e50 tx=0x7f354802bed0 comp rx=0 tx=0).stop 2024-04-20T11:26:37.632 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:37.631+0000 7f3554c00640 1 -- 172.21.15.152:0/2995988244 shutdown_connections 2024-04-20T11:26:37.632 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:37.631+0000 7f3554c00640 1 --2- 172.21.15.152:0/2995988244 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f35500d1810 0x7f35500d1ca0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:37.632 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:37.631+0000 7f3554c00640 1 --2- 172.21.15.152:0/2995988244 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f355015eb80 0x7f35500b79a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:37.632 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:37.631+0000 7f3554c00640 1 -- 172.21.15.152:0/2995988244 >> 172.21.15.152:0/2995988244 conn(0x7f3550067ae0 msgr2=0x7f3550067ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:37.632 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:37.631+0000 7f3554c00640 1 -- 172.21.15.152:0/2995988244 shutdown_connections 2024-04-20T11:26:37.632 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:37.631+0000 7f3554c00640 1 -- 172.21.15.152:0/2995988244 wait complete. 2024-04-20T11:26:37.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:37.633+0000 7f3554c00640 1 Processor -- start 2024-04-20T11:26:37.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:37.633+0000 7f3554c00640 1 -- start start 2024-04-20T11:26:37.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:37.634+0000 7f3554c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f35500ba520 0x7f355014ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:37.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:37.634+0000 7f3554c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f35500d1810 0x7f35500cb940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:37.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:37.634+0000 7f3554c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f355015eb80 0x7f35500cd540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:37.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:37.634+0000 7f3554c00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f35500b8060 con 0x7f35500ba520 2024-04-20T11:26:37.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:37.634+0000 7f3554c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f35500b7ee0 con 0x7f355015eb80 2024-04-20T11:26:37.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:37.634+0000 7f3554c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f35500b93d0 con 0x7f35500d1810 2024-04-20T11:26:37.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:37.634+0000 7f354de00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f35500ba520 0x7f355014ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:37.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:37.634+0000 7f354de00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f35500ba520 0x7f355014ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:37912/0 (socket says 172.21.15.152:37912) 2024-04-20T11:26:37.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:37.634+0000 7f354de00640 1 -- 172.21.15.152:0/179760288 learned_addr learned my addr 172.21.15.152:0/179760288 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:26:37.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:37.634+0000 7f3547e00640 1 --2- 172.21.15.152:0/179760288 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f35500d1810 0x7f35500cb940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:37.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:37.634+0000 7f354e800640 1 --2- 172.21.15.152:0/179760288 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f355015eb80 0x7f35500cd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:37.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:37.635+0000 7f354de00640 1 -- 172.21.15.152:0/179760288 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f35500d1810 msgr2=0x7f35500cb940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:37.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:37.635+0000 7f354de00640 1 --2- 172.21.15.152:0/179760288 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f35500d1810 0x7f35500cb940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:37.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:37.635+0000 7f354de00640 1 -- 172.21.15.152:0/179760288 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f355015eb80 msgr2=0x7f35500cd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:37.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:37.635+0000 7f354de00640 1 --2- 172.21.15.152:0/179760288 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f355015eb80 0x7f35500cd540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:37.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:37.635+0000 7f354de00640 1 -- 172.21.15.152:0/179760288 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35500ce450 con 0x7f35500ba520 2024-04-20T11:26:37.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:37.635+0000 7f354de00640 1 --2- 172.21.15.152:0/179760288 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f35500ba520 0x7f355014ad70 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f354000e870 tx=0x7f354000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:37.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:37.635+0000 7f3546000640 1 -- 172.21.15.152:0/179760288 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3540017070 con 0x7f35500ba520 2024-04-20T11:26:37.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:37.635+0000 7f3546000640 1 -- 172.21.15.152:0/179760288 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f354000c940 con 0x7f35500ba520 2024-04-20T11:26:37.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:37.635+0000 7f3554c00640 1 -- 172.21.15.152:0/179760288 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35500ce730 con 0x7f35500ba520 2024-04-20T11:26:37.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:37.635+0000 7f3546000640 1 -- 172.21.15.152:0/179760288 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f354000cd30 con 0x7f35500ba520 2024-04-20T11:26:37.638 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:37.635+0000 7f3554c00640 1 -- 172.21.15.152:0/179760288 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f355015f6c0 con 0x7f35500ba520 2024-04-20T11:26:37.638 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:37.636+0000 7f3546000640 1 -- 172.21.15.152:0/179760288 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f354001a070 con 0x7f35500ba520 2024-04-20T11:26:37.638 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:37.636+0000 7f3546000640 1 -- 172.21.15.152:0/179760288 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f3540013b40 con 0x7f35500ba520 2024-04-20T11:26:37.638 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:37.636+0000 7f3554c00640 1 -- 172.21.15.152:0/179760288 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35500689f0 con 0x7f35500ba520 2024-04-20T11:26:37.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:37.639+0000 7f3546000640 1 -- 172.21.15.152:0/179760288 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7f3540013e20 con 0x7f35500ba520 2024-04-20T11:26:37.706 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:37.706+0000 7f3554c00640 1 -- 172.21.15.152:0/179760288 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f355015f890 con 0x7f35500ba520 2024-04-20T11:26:37.707 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:37.707+0000 7f3546000640 1 -- 172.21.15.152:0/179760288 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f3540006520 con 0x7f35500ba520 2024-04-20T11:26:37.707 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:26:37.707 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":10,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:26:37.321214+0000","last_up_change":"0.000000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:26:37.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:37.708+0000 7f3554c00640 1 -- 172.21.15.152:0/179760288 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f35500ba520 msgr2=0x7f355014ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:37.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:37.708+0000 7f3554c00640 1 --2- 172.21.15.152:0/179760288 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f35500ba520 0x7f355014ad70 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f354000e870 tx=0x7f354000ec70 comp rx=0 tx=0).stop 2024-04-20T11:26:37.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:37.708+0000 7f3554c00640 1 -- 172.21.15.152:0/179760288 shutdown_connections 2024-04-20T11:26:37.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:37.708+0000 7f3554c00640 1 --2- 172.21.15.152:0/179760288 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f355015eb80 0x7f35500cd540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:37.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:37.708+0000 7f3554c00640 1 --2- 172.21.15.152:0/179760288 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f35500d1810 0x7f35500cb940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:37.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:37.708+0000 7f3554c00640 1 --2- 172.21.15.152:0/179760288 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f35500ba520 0x7f355014ad70 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:37.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:37.708+0000 7f3554c00640 1 -- 172.21.15.152:0/179760288 >> 172.21.15.152:0/179760288 conn(0x7f3550067ae0 msgr2=0x7f3550067ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:37.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:37.708+0000 7f3554c00640 1 -- 172.21.15.152:0/179760288 shutdown_connections 2024-04-20T11:26:37.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:37.708+0000 7f3554c00640 1 -- 172.21.15.152:0/179760288 wait complete. 2024-04-20T11:26:37.726 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-20T11:26:37.726 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-20T11:26:37.899 INFO:tasks.ceph.mgr.y.smithi152.stderr:2024-04-20T11:26:37.899+0000 7fd1f000e040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-20T11:26:37.953 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:26:37.953+0000 7f8ae6493040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-20T11:26:37.955 INFO:tasks.ceph.mgr.y.smithi152.stderr:2024-04-20T11:26:37.954+0000 7fd1f000e040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-20T11:26:38.010 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:26:38.010+0000 7f8ae6493040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-20T11:26:38.058 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-04-20T11:26:38.058+0000 7fc347e9c640 -1 Falling back to public interface 2024-04-20T11:26:38.059 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-04-20T11:26:38.058+0000 7efea4212640 -1 Falling back to public interface 2024-04-20T11:26:38.059 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-04-20T11:26:38.059+0000 7fc346a00640 -1 PosixStack listen unable to listen on v1:0.0.0.0:6801/0: (98) Address already in use 2024-04-20T11:26:38.065 INFO:tasks.ceph.osd.2.smithi152.stderr:2024-04-20T11:26:38.065+0000 7f526340c640 -1 Falling back to public interface 2024-04-20T11:26:38.071 INFO:tasks.ceph.osd.1.smithi152.stderr:2024-04-20T11:26:38.071+0000 7f1c4e421640 -1 Falling back to public interface 2024-04-20T11:26:38.089 INFO:tasks.ceph.osd.3.smithi152.stderr:2024-04-20T11:26:38.089+0000 7f32b201e640 -1 Falling back to public interface 2024-04-20T11:26:38.094 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-04-20T11:26:38.094+0000 7efc4e706640 -1 Falling back to public interface 2024-04-20T11:26:38.096 INFO:tasks.ceph.osd.0.smithi152.stderr:2024-04-20T11:26:38.096+0000 7f6b7332f640 -1 Falling back to public interface 2024-04-20T11:26:38.099 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-04-20T11:26:38.099+0000 7f80aa3a7640 -1 Falling back to public interface 2024-04-20T11:26:38.100 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-04-20T11:26:38.100+0000 7efc4d200640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6820/0: (98) Address already in use 2024-04-20T11:26:38.141 INFO:tasks.ceph.mgr.y.smithi152.stderr:2024-04-20T11:26:38.141+0000 7fd1f000e040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-20T11:26:38.195 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:26:38.195+0000 7f8ae6493040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-20T11:26:38.307 INFO:tasks.ceph.mgr.y.smithi152.stderr:2024-04-20T11:26:38.306+0000 7fd1f000e040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-20T11:26:38.360 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:26:38.360+0000 7f8ae6493040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-20T11:26:38.366 INFO:tasks.ceph.mgr.y.smithi152.stderr:2024-04-20T11:26:38.366+0000 7fd1f000e040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-20T11:26:38.420 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:26:38.420+0000 7f8ae6493040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-20T11:26:38.449 INFO:tasks.ceph.mgr.y.smithi152.stderr:2024-04-20T11:26:38.449+0000 7fd1f000e040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-20T11:26:38.501 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:26:38.501+0000 7f8ae6493040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-20T11:26:38.501 INFO:tasks.ceph.mgr.y.smithi152.stderr:2024-04-20T11:26:38.501+0000 7fd1f000e040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-20T11:26:38.532 DEBUG:teuthology.orchestra.run.smithi152:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-20T11:26:38.553 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:26:38.553+0000 7f8ae6493040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-20T11:26:38.688 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:38.688+0000 7f52fb200640 1 Processor -- start 2024-04-20T11:26:38.688 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:38.688+0000 7f52fb200640 1 -- start start 2024-04-20T11:26:38.690 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:38.689+0000 7f52fb200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f52fc0cee70 0x7f52fc0cf570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:38.690 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:38.689+0000 7f52fb200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f52fc0cfab0 0x7f52fc15e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:38.690 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:38.689+0000 7f52fb200640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52fc0b7400 con 0x7f52fc0b9de0 2024-04-20T11:26:38.690 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:38.689+0000 7f52fb200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f52fc0cff20 con 0x7f52fc0cfab0 2024-04-20T11:26:38.690 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:38.689+0000 7f52fb200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f52fc0b76e0 con 0x7f52fc0cee70 2024-04-20T11:26:38.690 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:38.689+0000 7f52fa800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f52fc0cfab0 0x7f52fc15e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:38.690 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:38.689+0000 7f52fa800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f52fc0cfab0 0x7f52fc15e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:37924/0 (socket says 172.21.15.152:37924) 2024-04-20T11:26:38.690 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:38.689+0000 7f52fa800640 1 -- 172.21.15.152:0/363632042 learned_addr learned my addr 172.21.15.152:0/363632042 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:26:38.690 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:38.689+0000 7f52f9e00640 1 --1- 172.21.15.152:0/363632042 >> v1:172.21.15.152:6789/0 conn(0x7f52fc0b9de0 0x7f52fc0d32d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:52014/0 (socket says 172.21.15.152:52014) 2024-04-20T11:26:38.690 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:38.689+0000 7f52f9400640 1 --2- 172.21.15.152:0/363632042 >> v2:172.21.15.175:3300/0 conn(0x7f52fc0cee70 0x7f52fc0cf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:38.691 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:38.690+0000 7f52fa800640 1 -- 172.21.15.152:0/363632042 >> v2:172.21.15.175:3300/0 conn(0x7f52fc0cee70 msgr2=0x7f52fc0cf570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:38.691 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:38.690+0000 7f52fa800640 1 --2- 172.21.15.152:0/363632042 >> v2:172.21.15.175:3300/0 conn(0x7f52fc0cee70 0x7f52fc0cf570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:38.691 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:38.690+0000 7f52fa800640 1 -- 172.21.15.152:0/363632042 >> v1:172.21.15.152:6789/0 conn(0x7f52fc0b9de0 legacy=0x7f52fc0d32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:38.691 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:38.690+0000 7f52f8a00640 1 -- 172.21.15.152:0/363632042 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2141635579 0 0) 0x7f52fc0b7400 con 0x7f52fc0b9de0 2024-04-20T11:26:38.691 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:38.690+0000 7f52fa800640 1 -- 172.21.15.152:0/363632042 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52fc0b79c0 con 0x7f52fc0cfab0 2024-04-20T11:26:38.692 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:38.691+0000 7f52fa800640 1 --2- 172.21.15.152:0/363632042 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f52fc0cfab0 0x7f52fc15e160 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f52fc06c370 tx=0x7f52f402bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=716f4b2cbdcec9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:38.692 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:38.692+0000 7f52f8a00640 1 -- 172.21.15.152:0/363632042 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f402cb20 con 0x7f52fc0cfab0 2024-04-20T11:26:38.692 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:38.692+0000 7f52f8a00640 1 -- 172.21.15.152:0/363632042 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52f402ccc0 con 0x7f52fc0cfab0 2024-04-20T11:26:38.692 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:38.692+0000 7f52f8a00640 1 -- 172.21.15.152:0/363632042 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f4034610 con 0x7f52fc0cfab0 2024-04-20T11:26:38.693 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:38.692+0000 7f52fb200640 1 -- 172.21.15.152:0/363632042 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f52fc0cfab0 msgr2=0x7f52fc15e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:38.693 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:38.692+0000 7f52fb200640 1 --2- 172.21.15.152:0/363632042 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f52fc0cfab0 0x7f52fc15e160 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f52fc06c370 tx=0x7f52f402bf80 comp rx=0 tx=0).stop 2024-04-20T11:26:38.693 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:38.692+0000 7f52fb200640 1 -- 172.21.15.152:0/363632042 shutdown_connections 2024-04-20T11:26:38.693 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:38.692+0000 7f52fb200640 1 --2- 172.21.15.152:0/363632042 >> v2:172.21.15.175:3300/0 conn(0x7f52fc0cee70 0x7f52fc0cf570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:38.693 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:38.692+0000 7f52fb200640 1 --2- 172.21.15.152:0/363632042 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f52fc0cfab0 0x7f52fc15e160 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:38.693 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:38.692+0000 7f52fb200640 1 -- 172.21.15.152:0/363632042 >> 172.21.15.152:0/363632042 conn(0x7f52fc08bf70 msgr2=0x7f52fc0d4510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:38.693 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:38.693+0000 7f52fb200640 1 -- 172.21.15.152:0/363632042 shutdown_connections 2024-04-20T11:26:38.694 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:38.693+0000 7f52fb200640 1 -- 172.21.15.152:0/363632042 wait complete. 2024-04-20T11:26:38.695 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:38.694+0000 7f52fb200640 1 Processor -- start 2024-04-20T11:26:38.695 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:38.695+0000 7f52fb200640 1 -- start start 2024-04-20T11:26:38.697 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:38.695+0000 7f52fb200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f52fc0b9de0 0x7f52fc150490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:38.697 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:38.695+0000 7f52fb200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f52fc0cee70 0x7f52fc14cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:38.697 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:38.695+0000 7f52fb200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f52fc0cfab0 0x7f52fc1f7d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:38.698 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:38.695+0000 7f52fb200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f52fc0ba1c0 con 0x7f52fc0cee70 2024-04-20T11:26:38.698 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:38.696+0000 7f52fb200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f52fc0d3be0 con 0x7f52fc0b9de0 2024-04-20T11:26:38.698 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:38.696+0000 7f52fb200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f52fc15e6a0 con 0x7f52fc0cfab0 2024-04-20T11:26:38.698 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:38.696+0000 7f52f9400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f52fc0cee70 0x7f52fc14cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:38.698 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:38.696+0000 7f52f9400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f52fc0cee70 0x7f52fc14cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:37930/0 (socket says 172.21.15.152:37930) 2024-04-20T11:26:38.698 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:38.696+0000 7f52f9400640 1 -- 172.21.15.152:0/3077298264 learned_addr learned my addr 172.21.15.152:0/3077298264 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:26:38.698 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:38.696+0000 7f52fa800640 1 --2- 172.21.15.152:0/3077298264 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f52fc0cfab0 0x7f52fc1f7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:38.698 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:38.696+0000 7f52f9e00640 1 --2- 172.21.15.152:0/3077298264 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f52fc0b9de0 0x7f52fc150490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:38.698 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:38.696+0000 7f52f9400640 1 -- 172.21.15.152:0/3077298264 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f52fc0cfab0 msgr2=0x7f52fc1f7d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:38.698 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:38.696+0000 7f52f9400640 1 --2- 172.21.15.152:0/3077298264 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f52fc0cfab0 0x7f52fc1f7d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:38.698 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:38.696+0000 7f52f9400640 1 -- 172.21.15.152:0/3077298264 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f52fc0b9de0 msgr2=0x7f52fc150490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:38.698 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:38.696+0000 7f52f9400640 1 --2- 172.21.15.152:0/3077298264 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f52fc0b9de0 0x7f52fc150490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:38.698 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:38.696+0000 7f52f9400640 1 -- 172.21.15.152:0/3077298264 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52fc1f8b80 con 0x7f52fc0cee70 2024-04-20T11:26:38.698 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:38.696+0000 7f52f9400640 1 --2- 172.21.15.152:0/3077298264 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f52fc0cee70 0x7f52fc14cf80 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f52e800e9b0 tx=0x7f52e800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:38.699 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:38.697+0000 7f52f2000640 1 -- 172.21.15.152:0/3077298264 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52e8017070 con 0x7f52fc0cee70 2024-04-20T11:26:38.699 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:38.697+0000 7f52f2000640 1 -- 172.21.15.152:0/3077298264 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52e800c9a0 con 0x7f52fc0cee70 2024-04-20T11:26:38.699 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:38.697+0000 7f52f2000640 1 -- 172.21.15.152:0/3077298264 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52e800cd90 con 0x7f52fc0cee70 2024-04-20T11:26:38.699 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:38.697+0000 7f52fb200640 1 -- 172.21.15.152:0/3077298264 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52fc1f88a0 con 0x7f52fc0cee70 2024-04-20T11:26:38.699 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:38.697+0000 7f52fb200640 1 -- 172.21.15.152:0/3077298264 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52fc14fc90 con 0x7f52fc0cee70 2024-04-20T11:26:38.699 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:38.697+0000 7f52f2000640 1 -- 172.21.15.152:0/3077298264 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f52e801a070 con 0x7f52fc0cee70 2024-04-20T11:26:38.699 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:38.697+0000 7f52f2000640 1 -- 172.21.15.152:0/3077298264 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f52e8006660 con 0x7f52fc0cee70 2024-04-20T11:26:38.699 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:38.698+0000 7f52fb200640 1 -- 172.21.15.152:0/3077298264 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52fc163180 con 0x7f52fc0cee70 2024-04-20T11:26:38.701 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:38.700+0000 7f52f2000640 1 -- 172.21.15.152:0/3077298264 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7f52fc163180 con 0x7f52fc0cee70 2024-04-20T11:26:38.765 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:38.764+0000 7f52fb200640 1 -- 172.21.15.152:0/3077298264 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f52fc04fcf0 con 0x7f52fc0cee70 2024-04-20T11:26:38.765 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:38.765+0000 7f52f2000640 1 -- 172.21.15.152:0/3077298264 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f52e8005990 con 0x7f52fc0cee70 2024-04-20T11:26:38.766 INFO:teuthology.misc.health.smithi152.stdout: 2024-04-20T11:26:38.766 INFO:teuthology.misc.health.smithi152.stdout:{"epoch":10,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:26:37.321214+0000","last_up_change":"0.000000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:26:38.767 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:38.766+0000 7f52fb200640 1 -- 172.21.15.152:0/3077298264 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f52fc0cee70 msgr2=0x7f52fc14cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:38.767 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:38.766+0000 7f52fb200640 1 --2- 172.21.15.152:0/3077298264 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f52fc0cee70 0x7f52fc14cf80 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f52e800e9b0 tx=0x7f52e800ee70 comp rx=0 tx=0).stop 2024-04-20T11:26:38.767 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:38.766+0000 7f52fb200640 1 -- 172.21.15.152:0/3077298264 shutdown_connections 2024-04-20T11:26:38.767 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:38.766+0000 7f52fb200640 1 --2- 172.21.15.152:0/3077298264 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f52fc0cfab0 0x7f52fc1f7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:38.767 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:38.766+0000 7f52fb200640 1 --2- 172.21.15.152:0/3077298264 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f52fc0cee70 0x7f52fc14cf80 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:38.767 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:38.766+0000 7f52fb200640 1 --2- 172.21.15.152:0/3077298264 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f52fc0b9de0 0x7f52fc150490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:38.767 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:38.766+0000 7f52fb200640 1 -- 172.21.15.152:0/3077298264 >> 172.21.15.152:0/3077298264 conn(0x7f52fc08bf70 msgr2=0x7f52fc0d4510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:38.767 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:38.766+0000 7f52fb200640 1 -- 172.21.15.152:0/3077298264 shutdown_connections 2024-04-20T11:26:38.768 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:38.767+0000 7f52fb200640 1 -- 172.21.15.152:0/3077298264 wait complete. 2024-04-20T11:26:38.780 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-20T11:26:39.056 INFO:tasks.ceph.mgr.y.smithi152.stderr:2024-04-20T11:26:39.055+0000 7fd1f000e040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-20T11:26:39.097 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:26:39.096+0000 7f8ae6493040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-20T11:26:39.188 INFO:tasks.ceph.mgr.y.smithi152.stderr:2024-04-20T11:26:39.187+0000 7fd1f000e040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-20T11:26:39.233 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:26:39.233+0000 7f8ae6493040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-20T11:26:40.232 INFO:tasks.ceph.mgr.y.smithi152.stderr:2024-04-20T11:26:40.232+0000 7fd190600640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-20T11:26:40.233 INFO:tasks.ceph.mgr.y.smithi152.stderr:2024-04-20T11:26:40.232+0000 7fd190600640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-20T11:26:41.448 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-04-20T11:26:41.448+0000 7fc347e9c640 -1 osd.4 0 log_to_monitors true 2024-04-20T11:26:41.666 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-04-20T11:26:41.666+0000 7efea4212640 -1 osd.5 0 log_to_monitors true 2024-04-20T11:26:41.703 INFO:tasks.ceph.osd.1.smithi152.stderr:2024-04-20T11:26:41.702+0000 7f1c4e421640 -1 osd.1 0 log_to_monitors true 2024-04-20T11:26:41.703 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-04-20T11:26:41.703+0000 7efc4e706640 -1 osd.7 0 log_to_monitors true 2024-04-20T11:26:41.723 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-04-20T11:26:41.723+0000 7f80aa3a7640 -1 osd.6 0 log_to_monitors true 2024-04-20T11:26:41.768 INFO:tasks.ceph.osd.3.smithi152.stderr:2024-04-20T11:26:41.768+0000 7f32b201e640 -1 osd.3 0 log_to_monitors true 2024-04-20T11:26:41.770 INFO:tasks.ceph.osd.0.smithi152.stderr:2024-04-20T11:26:41.769+0000 7f6b7332f640 -1 osd.0 0 log_to_monitors true 2024-04-20T11:26:41.893 INFO:tasks.ceph.osd.2.smithi152.stderr:2024-04-20T11:26:41.893+0000 7f526340c640 -1 osd.2 0 log_to_monitors true 2024-04-20T11:26:45.513 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-04-20T11:26:45.513+0000 7fc342e00640 -1 osd.4 0 waiting for initial osdmap 2024-04-20T11:26:45.526 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-04-20T11:26:45.527+0000 7fc338e00640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T11:26:45.548 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-04-20T11:26:45.548+0000 7f80a5c00640 -1 osd.6 0 waiting for initial osdmap 2024-04-20T11:26:45.559 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-04-20T11:26:45.559+0000 7f809b200640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T11:26:45.570 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-04-20T11:26:45.570+0000 7efc4a000640 -1 osd.7 0 waiting for initial osdmap 2024-04-20T11:26:45.583 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-04-20T11:26:45.583+0000 7efc3f600640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T11:26:45.586 DEBUG:teuthology.orchestra.run.smithi152:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-20T11:26:45.615 INFO:tasks.ceph.osd.0.smithi152.stderr:2024-04-20T11:26:45.615+0000 7f6b6e200640 -1 osd.0 0 waiting for initial osdmap 2024-04-20T11:26:45.618 INFO:tasks.ceph.osd.2.smithi152.stderr:2024-04-20T11:26:45.618+0000 7f525dc00640 -1 osd.2 0 waiting for initial osdmap 2024-04-20T11:26:45.620 INFO:tasks.ceph.osd.1.smithi152.stderr:2024-04-20T11:26:45.620+0000 7f1c49800640 -1 osd.1 0 waiting for initial osdmap 2024-04-20T11:26:45.623 INFO:tasks.ceph.osd.0.smithi152.stderr:2024-04-20T11:26:45.623+0000 7f6b64200640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T11:26:45.623 INFO:tasks.ceph.osd.3.smithi152.stderr:2024-04-20T11:26:45.623+0000 7f32ac800640 -1 osd.3 0 waiting for initial osdmap 2024-04-20T11:26:45.627 INFO:tasks.ceph.osd.3.smithi152.stderr:2024-04-20T11:26:45.627+0000 7f32a2800640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T11:26:45.628 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-04-20T11:26:45.628+0000 7efe9ea00640 -1 osd.5 0 waiting for initial osdmap 2024-04-20T11:26:45.629 INFO:tasks.ceph.osd.2.smithi152.stderr:2024-04-20T11:26:45.629+0000 7f5253c00640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T11:26:45.634 INFO:tasks.ceph.osd.1.smithi152.stderr:2024-04-20T11:26:45.634+0000 7f1c3ee00640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T11:26:45.635 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-04-20T11:26:45.635+0000 7efe94a00640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T11:26:45.744 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:45.743+0000 7f9f91c00640 1 Processor -- start 2024-04-20T11:26:45.744 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:45.743+0000 7f9f91c00640 1 -- start start 2024-04-20T11:26:45.745 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:45.744+0000 7f9f91c00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f9f8c0b3fe0 0x7f9f8c0b4490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:45.745 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:45.745+0000 7f9f91c00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f8c06eaf0 con 0x7f9f8c166480 2024-04-20T11:26:45.745 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:45.745+0000 7f9f91c00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f8c06edd0 con 0x7f9f8c158ef0 2024-04-20T11:26:45.745 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:45.745+0000 7f9f91c00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f9f8c06f700 con 0x7f9f8c0b3fe0 2024-04-20T11:26:45.745 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:45.745+0000 7f9f83e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f9f8c0b3fe0 0x7f9f8c0b4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:45.745 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:45.745+0000 7f9f8b400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f9f8c166480 0x7f9f8c168870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:52030/0 (socket says 172.21.15.152:52030) 2024-04-20T11:26:45.746 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:45.745+0000 7f9f8b400640 1 -- 172.21.15.152:0/2916872287 learned_addr learned my addr 172.21.15.152:0/2916872287 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:26:45.746 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:45.745+0000 7f9f83e00640 1 -- 172.21.15.152:0/2916872287 >> v1:172.21.15.175:6789/0 conn(0x7f9f8c158ef0 legacy=0x7f9f8c15f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:45.746 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:45.745+0000 7f9f83e00640 1 -- 172.21.15.152:0/2916872287 >> v1:172.21.15.152:6789/0 conn(0x7f9f8c166480 legacy=0x7f9f8c168870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:45.746 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:45.745+0000 7f9f83e00640 1 -- 172.21.15.152:0/2916872287 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f8c06f0b0 con 0x7f9f8c0b3fe0 2024-04-20T11:26:45.746 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:45.745+0000 7f9f8a000640 1 -- 172.21.15.152:0/2916872287 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 233555038 0 0) 0x7f9f8c06eaf0 con 0x7f9f8c166480 2024-04-20T11:26:45.747 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:45.746+0000 7f9f83e00640 1 --2- 172.21.15.152:0/2916872287 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9f8c0b3fe0 0x7f9f8c0b4490 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f9f740097a0 tx=0x7f9f7402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed5901c5a7a6592f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:45.747 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:45.747+0000 7f9f8a000640 1 -- 172.21.15.152:0/2916872287 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f74035020 con 0x7f9f8c0b3fe0 2024-04-20T11:26:45.747 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:45.747+0000 7f9f8a000640 1 -- 172.21.15.152:0/2916872287 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f74031830 con 0x7f9f8c0b3fe0 2024-04-20T11:26:45.747 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:45.747+0000 7f9f8a000640 1 -- 172.21.15.152:0/2916872287 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f74031c00 con 0x7f9f8c0b3fe0 2024-04-20T11:26:45.748 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:45.747+0000 7f9f91c00640 1 -- 172.21.15.152:0/2916872287 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9f8c0b3fe0 msgr2=0x7f9f8c0b4490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:45.748 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:45.747+0000 7f9f91c00640 1 --2- 172.21.15.152:0/2916872287 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9f8c0b3fe0 0x7f9f8c0b4490 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f9f740097a0 tx=0x7f9f7402b8d0 comp rx=0 tx=0).stop 2024-04-20T11:26:45.748 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:45.747+0000 7f9f91c00640 1 -- 172.21.15.152:0/2916872287 shutdown_connections 2024-04-20T11:26:45.748 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:45.747+0000 7f9f91c00640 1 --2- 172.21.15.152:0/2916872287 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9f8c0b3fe0 0x7f9f8c0b4490 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:45.748 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:45.747+0000 7f9f91c00640 1 -- 172.21.15.152:0/2916872287 >> 172.21.15.152:0/2916872287 conn(0x7f9f8c0d0c40 msgr2=0x7f9f8c0d1020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:45.748 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:45.747+0000 7f9f91c00640 1 -- 172.21.15.152:0/2916872287 shutdown_connections 2024-04-20T11:26:45.749 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:45.747+0000 7f9f91c00640 1 -- 172.21.15.152:0/2916872287 wait complete. 2024-04-20T11:26:45.749 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:45.749+0000 7f9f91c00640 1 Processor -- start 2024-04-20T11:26:45.749 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:45.749+0000 7f9f91c00640 1 -- start start 2024-04-20T11:26:45.752 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:45.750+0000 7f9f91c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9f8c0b3fe0 0x7f9f8c0cdf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:45.752 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:45.750+0000 7f9f91c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9f8c158ef0 0x7f9f8c06bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:45.752 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:45.750+0000 7f9f91c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9f8c166480 0x7f9f8c14a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:45.752 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:45.750+0000 7f9f91c00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f9f8c06ca40 con 0x7f9f8c166480 2024-04-20T11:26:45.752 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:45.750+0000 7f9f91c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9f8c1592d0 con 0x7f9f8c0b3fe0 2024-04-20T11:26:45.752 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:45.750+0000 7f9f91c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f9f8c06e7c0 con 0x7f9f8c158ef0 2024-04-20T11:26:45.752 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:45.750+0000 7f9f8b400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9f8c166480 0x7f9f8c14a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:45.752 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:45.750+0000 7f9f8b400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9f8c166480 0x7f9f8c14a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:38068/0 (socket says 172.21.15.152:38068) 2024-04-20T11:26:45.752 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:45.750+0000 7f9f8b400640 1 -- 172.21.15.152:0/4050838342 learned_addr learned my addr 172.21.15.152:0/4050838342 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:26:45.752 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:45.751+0000 7f9f83e00640 1 --2- 172.21.15.152:0/4050838342 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9f8c158ef0 0x7f9f8c06bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:45.752 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:45.751+0000 7f9f8aa00640 1 --2- 172.21.15.152:0/4050838342 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9f8c0b3fe0 0x7f9f8c0cdf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:45.752 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:45.751+0000 7f9f8b400640 1 -- 172.21.15.152:0/4050838342 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9f8c158ef0 msgr2=0x7f9f8c06bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:45.752 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:45.751+0000 7f9f8b400640 1 --2- 172.21.15.152:0/4050838342 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9f8c158ef0 0x7f9f8c06bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:45.753 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:45.751+0000 7f9f8b400640 1 -- 172.21.15.152:0/4050838342 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9f8c0b3fe0 msgr2=0x7f9f8c0cdf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:45.753 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:45.751+0000 7f9f8b400640 1 --2- 172.21.15.152:0/4050838342 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9f8c0b3fe0 0x7f9f8c0cdf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:45.753 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:45.751+0000 7f9f8b400640 1 -- 172.21.15.152:0/4050838342 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f8c0cfd60 con 0x7f9f8c166480 2024-04-20T11:26:45.753 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:45.751+0000 7f9f8aa00640 1 --2- 172.21.15.152:0/4050838342 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9f8c0b3fe0 0x7f9f8c0cdf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:26:45.753 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:45.751+0000 7f9f8b400640 1 --2- 172.21.15.152:0/4050838342 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9f8c166480 0x7f9f8c14a510 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f9f8400c7d0 tx=0x7f9f8400cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:45.753 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:45.751+0000 7f9f82a00640 1 -- 172.21.15.152:0/4050838342 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f84018070 con 0x7f9f8c166480 2024-04-20T11:26:45.753 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:45.751+0000 7f9f82a00640 1 -- 172.21.15.152:0/4050838342 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f8400a5e0 con 0x7f9f8c166480 2024-04-20T11:26:45.753 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:45.751+0000 7f9f82a00640 1 -- 172.21.15.152:0/4050838342 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f8400aa10 con 0x7f9f8c166480 2024-04-20T11:26:45.753 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:45.751+0000 7f9f91c00640 1 -- 172.21.15.152:0/4050838342 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f8c0ae320 con 0x7f9f8c166480 2024-04-20T11:26:45.753 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:45.751+0000 7f9f91c00640 1 -- 172.21.15.152:0/4050838342 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f8c0ae8e0 con 0x7f9f8c166480 2024-04-20T11:26:45.754 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:45.753+0000 7f9f91c00640 1 -- 172.21.15.152:0/4050838342 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f8c069320 con 0x7f9f8c166480 2024-04-20T11:26:45.754 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:45.754+0000 7f9f82a00640 1 -- 172.21.15.152:0/4050838342 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 4) ==== 90984+0+0 (secure 0 0 0) 0x7f9f8400abb0 con 0x7f9f8c166480 2024-04-20T11:26:45.755 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:45.754+0000 7f9f82a00640 1 --2- 172.21.15.152:0/4050838342 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f9f6806c860 0x7f9f6806ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:45.755 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:45.754+0000 7f9f82a00640 1 -- 172.21.15.152:0/4050838342 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2231+0+0 (secure 0 0 0) 0x7f9f8408a370 con 0x7f9f8c166480 2024-04-20T11:26:45.755 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:45.755+0000 7f9f8aa00640 1 --2- 172.21.15.152:0/4050838342 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f9f6806c860 0x7f9f6806ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:45.755 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:45.755+0000 7f9f8aa00640 1 --2- 172.21.15.152:0/4050838342 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f9f6806c860 0x7f9f6806ed20 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c00a830 tx=0x7f9f7c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:45.760 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:45.759+0000 7f9f82a00640 1 -- 172.21.15.152:0/4050838342 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f9f84059580 con 0x7f9f8c166480 2024-04-20T11:26:45.948 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:45.948+0000 7f9f91c00640 1 -- 172.21.15.152:0/4050838342 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9f8c15f3d0 con 0x7f9f8c166480 2024-04-20T11:26:45.949 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:45.949+0000 7f9f82a00640 1 -- 172.21.15.152:0/4050838342 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) ==== 74+0+7058 (secure 0 0 0) 0x7f9f84058e80 con 0x7f9f8c166480 2024-04-20T11:26:45.950 INFO:teuthology.misc.health.smithi152.stdout: 2024-04-20T11:26:45.950 INFO:teuthology.misc.health.smithi152.stdout:{"epoch":12,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:26:43.239360+0000","last_up_change":"0.000000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:26:45.952 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:45.951+0000 7f9f91c00640 1 -- 172.21.15.152:0/4050838342 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f9f6806c860 msgr2=0x7f9f6806ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:45.952 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:45.951+0000 7f9f91c00640 1 --2- 172.21.15.152:0/4050838342 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f9f6806c860 0x7f9f6806ed20 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c00a830 tx=0x7f9f7c008040 comp rx=0 tx=0).stop 2024-04-20T11:26:45.952 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:45.951+0000 7f9f91c00640 1 -- 172.21.15.152:0/4050838342 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9f8c166480 msgr2=0x7f9f8c14a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:45.952 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:45.951+0000 7f9f91c00640 1 --2- 172.21.15.152:0/4050838342 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9f8c166480 0x7f9f8c14a510 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f9f8400c7d0 tx=0x7f9f8400cc90 comp rx=0 tx=0).stop 2024-04-20T11:26:45.953 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:45.952+0000 7f9f91c00640 1 -- 172.21.15.152:0/4050838342 shutdown_connections 2024-04-20T11:26:45.953 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:45.952+0000 7f9f91c00640 1 --2- 172.21.15.152:0/4050838342 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f9f6806c860 0x7f9f6806ed20 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:45.953 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:45.952+0000 7f9f91c00640 1 --2- 172.21.15.152:0/4050838342 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9f8c166480 0x7f9f8c14a510 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:45.953 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:45.952+0000 7f9f91c00640 1 --2- 172.21.15.152:0/4050838342 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9f8c158ef0 0x7f9f8c06bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:45.953 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:45.952+0000 7f9f91c00640 1 --2- 172.21.15.152:0/4050838342 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9f8c0b3fe0 0x7f9f8c0cdf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:45.953 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:45.952+0000 7f9f91c00640 1 -- 172.21.15.152:0/4050838342 >> 172.21.15.152:0/4050838342 conn(0x7f9f8c0d0c40 msgr2=0x7f9f8c158730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:45.953 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:45.952+0000 7f9f91c00640 1 -- 172.21.15.152:0/4050838342 shutdown_connections 2024-04-20T11:26:45.953 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:45.952+0000 7f9f91c00640 1 -- 172.21.15.152:0/4050838342 wait complete. 2024-04-20T11:26:45.966 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-20T11:26:50.808 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:26:50.815 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:26:52.773 DEBUG:teuthology.orchestra.run.smithi152:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-20T11:26:52.958 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:52.957+0000 7fc423200640 1 Processor -- start 2024-04-20T11:26:52.958 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:52.957+0000 7fc423200640 1 -- start start 2024-04-20T11:26:52.959 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:52.959+0000 7fc423200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fc4240cd1d0 0x7fc4240b4210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:52.959 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:52.959+0000 7fc423200640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4240d0360 con 0x7fc4240ca2d0 2024-04-20T11:26:52.959 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:52.959+0000 7fc423200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4240d0640 con 0x7fc42415c6f0 2024-04-20T11:26:52.959 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:52.959+0000 7fc423200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fc4240d0920 con 0x7fc4240cd1d0 2024-04-20T11:26:52.959 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:52.959+0000 7fc41be00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fc4240cd1d0 0x7fc4240b4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:52.959 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:52.959+0000 7fc422800640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fc4240ca2d0 0x7fc4240b4750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:53814/0 (socket says 172.21.15.152:53814) 2024-04-20T11:26:52.959 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:52.959+0000 7fc422800640 1 -- 172.21.15.152:0/2836663006 learned_addr learned my addr 172.21.15.152:0/2836663006 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:26:52.960 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:52.959+0000 7fc41be00640 1 -- 172.21.15.152:0/2836663006 >> v1:172.21.15.175:6789/0 conn(0x7fc42415c6f0 legacy=0x7fc4240ccac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:52.960 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:52.959+0000 7fc41be00640 1 -- 172.21.15.152:0/2836663006 >> v1:172.21.15.152:6789/0 conn(0x7fc4240ca2d0 legacy=0x7fc4240b4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:52.960 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:52.960+0000 7fc41be00640 1 -- 172.21.15.152:0/2836663006 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4240d0c00 con 0x7fc4240cd1d0 2024-04-20T11:26:52.960 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:52.960+0000 7fc421400640 1 -- 172.21.15.152:0/2836663006 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 739272832 0 0) 0x7fc4240d0360 con 0x7fc4240ca2d0 2024-04-20T11:26:52.960 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:52.961+0000 7fc41be00640 1 --2- 172.21.15.152:0/2836663006 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc4240cd1d0 0x7fc4240b4210 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc40c0097a0 tx=0x7fc40c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab4888aea0eb8ad3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:52.960 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:52.961+0000 7fc421400640 1 -- 172.21.15.152:0/2836663006 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc40c035020 con 0x7fc4240cd1d0 2024-04-20T11:26:52.961 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:52.961+0000 7fc421400640 1 -- 172.21.15.152:0/2836663006 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc40c031830 con 0x7fc4240cd1d0 2024-04-20T11:26:52.961 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:52.961+0000 7fc421400640 1 -- 172.21.15.152:0/2836663006 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc40c031c00 con 0x7fc4240cd1d0 2024-04-20T11:26:52.961 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:52.961+0000 7fc423200640 1 -- 172.21.15.152:0/2836663006 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc4240cd1d0 msgr2=0x7fc4240b4210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:52.961 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:52.961+0000 7fc423200640 1 --2- 172.21.15.152:0/2836663006 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc4240cd1d0 0x7fc4240b4210 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc40c0097a0 tx=0x7fc40c02b8d0 comp rx=0 tx=0).stop 2024-04-20T11:26:52.962 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:52.962+0000 7fc423200640 1 -- 172.21.15.152:0/2836663006 shutdown_connections 2024-04-20T11:26:52.962 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:52.962+0000 7fc423200640 1 --2- 172.21.15.152:0/2836663006 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc4240cd1d0 0x7fc4240b4210 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:52.962 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:52.962+0000 7fc423200640 1 -- 172.21.15.152:0/2836663006 >> 172.21.15.152:0/2836663006 conn(0x7fc42406b920 msgr2=0x7fc42406bd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:52.962 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:52.962+0000 7fc423200640 1 -- 172.21.15.152:0/2836663006 shutdown_connections 2024-04-20T11:26:52.962 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:52.962+0000 7fc423200640 1 -- 172.21.15.152:0/2836663006 wait complete. 2024-04-20T11:26:52.963 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:52.963+0000 7fc423200640 1 Processor -- start 2024-04-20T11:26:52.963 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:52.964+0000 7fc423200640 1 -- start start 2024-04-20T11:26:52.965 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:52.964+0000 7fc423200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc4240ca2d0 0x7fc424142700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:52.965 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:52.964+0000 7fc423200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc4240cd1d0 0x7fc4240c9480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:52.965 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:52.964+0000 7fc423200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc42415c6f0 0x7fc4240c99c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:52.965 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:52.964+0000 7fc423200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fc4240d00f0 con 0x7fc4240ca2d0 2024-04-20T11:26:52.965 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:52.964+0000 7fc423200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc4240cff70 con 0x7fc42415c6f0 2024-04-20T11:26:52.965 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:52.964+0000 7fc423200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc42406e330 con 0x7fc4240cd1d0 2024-04-20T11:26:52.965 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:52.965+0000 7fc421e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc4240ca2d0 0x7fc424142700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:52.965 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:52.965+0000 7fc421e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc4240ca2d0 0x7fc424142700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:45154/0 (socket says 172.21.15.152:45154) 2024-04-20T11:26:52.965 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:52.965+0000 7fc421e00640 1 -- 172.21.15.152:0/731315907 learned_addr learned my addr 172.21.15.152:0/731315907 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:26:52.965 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:52.965+0000 7fc421e00640 1 -- 172.21.15.152:0/731315907 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc4240cd1d0 msgr2=0x7fc4240c9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:52.965 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:52.965+0000 7fc421e00640 1 --2- 172.21.15.152:0/731315907 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc4240cd1d0 0x7fc4240c9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:52.965 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:52.965+0000 7fc421e00640 1 -- 172.21.15.152:0/731315907 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc42415c6f0 msgr2=0x7fc4240c99c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:52.965 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:52.965+0000 7fc421e00640 1 --2- 172.21.15.152:0/731315907 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc42415c6f0 0x7fc4240c99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:52.966 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:52.965+0000 7fc421e00640 1 -- 172.21.15.152:0/731315907 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4241431a0 con 0x7fc4240ca2d0 2024-04-20T11:26:52.966 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:52.965+0000 7fc421e00640 1 --2- 172.21.15.152:0/731315907 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc4240ca2d0 0x7fc424142700 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc41400b3f0 tx=0x7fc41400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:52.966 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:52.966+0000 7fc41a000640 1 -- 172.21.15.152:0/731315907 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc414015020 con 0x7fc4240ca2d0 2024-04-20T11:26:52.967 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:52.966+0000 7fc41a000640 1 -- 172.21.15.152:0/731315907 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc414011830 con 0x7fc4240ca2d0 2024-04-20T11:26:52.967 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:52.966+0000 7fc41a000640 1 -- 172.21.15.152:0/731315907 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc414011c20 con 0x7fc4240ca2d0 2024-04-20T11:26:52.967 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:52.966+0000 7fc423200640 1 -- 172.21.15.152:0/731315907 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc424143480 con 0x7fc4240ca2d0 2024-04-20T11:26:52.967 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:52.966+0000 7fc423200640 1 -- 172.21.15.152:0/731315907 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc424159f10 con 0x7fc4240ca2d0 2024-04-20T11:26:52.967 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:52.967+0000 7fc423200640 1 -- 172.21.15.152:0/731315907 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc424140670 con 0x7fc4240ca2d0 2024-04-20T11:26:52.967 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:52.968+0000 7fc41a000640 1 -- 172.21.15.152:0/731315907 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc414018020 con 0x7fc4240ca2d0 2024-04-20T11:26:52.968 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:52.969+0000 7fc41a000640 1 --2- 172.21.15.152:0/731315907 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc40006c920 0x7fc40006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:52.968 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:52.969+0000 7fc41a000640 1 -- 172.21.15.152:0/731315907 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fc41408b3b0 con 0x7fc4240ca2d0 2024-04-20T11:26:52.968 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:52.969+0000 7fc41be00640 1 --2- 172.21.15.152:0/731315907 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc40006c920 0x7fc40006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:52.969 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:52.970+0000 7fc41be00640 1 --2- 172.21.15.152:0/731315907 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc40006c920 0x7fc40006ede0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc40c036000 tx=0x7fc40c030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:52.974 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:52.975+0000 7fc41a000640 1 -- 172.21.15.152:0/731315907 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fc41401f030 con 0x7fc4240ca2d0 2024-04-20T11:26:53.174 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:53.174+0000 7fc423200640 1 -- 172.21.15.152:0/731315907 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc424143650 con 0x7fc4240ca2d0 2024-04-20T11:26:53.177 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:53.177+0000 7fc41a000640 1 -- 172.21.15.152:0/731315907 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12944 (secure 0 0 0) 0x7fc424143650 con 0x7fc4240ca2d0 2024-04-20T11:26:53.177 INFO:teuthology.misc.health.smithi152.stdout: 2024-04-20T11:26:53.177 INFO:teuthology.misc.health.smithi152.stdout:{"epoch":16,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:26:49.573250+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:26:53.179 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:53.180+0000 7fc423200640 1 -- 172.21.15.152:0/731315907 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc40006c920 msgr2=0x7fc40006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:53.180 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:53.180+0000 7fc423200640 1 --2- 172.21.15.152:0/731315907 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc40006c920 0x7fc40006ede0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc40c036000 tx=0x7fc40c030000 comp rx=0 tx=0).stop 2024-04-20T11:26:53.180 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:53.180+0000 7fc423200640 1 -- 172.21.15.152:0/731315907 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc4240ca2d0 msgr2=0x7fc424142700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:53.180 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:53.180+0000 7fc423200640 1 --2- 172.21.15.152:0/731315907 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc4240ca2d0 0x7fc424142700 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc41400b3f0 tx=0x7fc41400b8b0 comp rx=0 tx=0).stop 2024-04-20T11:26:53.180 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:53.180+0000 7fc423200640 1 -- 172.21.15.152:0/731315907 shutdown_connections 2024-04-20T11:26:53.180 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:53.180+0000 7fc423200640 1 --2- 172.21.15.152:0/731315907 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc40006c920 0x7fc40006ede0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:53.180 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:53.180+0000 7fc423200640 1 --2- 172.21.15.152:0/731315907 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc42415c6f0 0x7fc4240c99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:53.180 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:53.180+0000 7fc423200640 1 --2- 172.21.15.152:0/731315907 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc4240cd1d0 0x7fc4240c9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:53.180 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:53.180+0000 7fc423200640 1 --2- 172.21.15.152:0/731315907 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc4240ca2d0 0x7fc424142700 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:53.180 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:53.180+0000 7fc423200640 1 -- 172.21.15.152:0/731315907 >> 172.21.15.152:0/731315907 conn(0x7fc42406b920 msgr2=0x7fc424160680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:53.180 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:53.180+0000 7fc423200640 1 -- 172.21.15.152:0/731315907 shutdown_connections 2024-04-20T11:26:53.180 INFO:teuthology.misc.health.smithi152.stderr:2024-04-20T11:26:53.180+0000 7fc423200640 1 -- 172.21.15.152:0/731315907 wait complete. 2024-04-20T11:26:53.193 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-04-20T11:26:53.193 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-20T11:26:53.193 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-20T11:26:53.193 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-20T11:26:53.201 INFO:tasks.ceph.mds.a:Started 2024-04-20T11:26:53.202 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-20T11:26:53.202 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-20T11:26:53.208 INFO:tasks.ceph.mds.c:Started 2024-04-20T11:26:53.209 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-20T11:26:53.209 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-20T11:26:53.217 INFO:tasks.ceph.mds.b:Started 2024-04-20T11:26:53.217 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-20T11:26:53.217 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-20T11:26:53.224 INFO:tasks.ceph.mds.d:Started 2024-04-20T11:26:53.224 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-04-20T11:26:53.225 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:26:53.343 INFO:tasks.ceph.mds.a.smithi152.stdout:starting mds.a at 2024-04-20T11:26:53.353 INFO:tasks.ceph.mds.c.smithi152.stdout:starting mds.c at 2024-04-20T11:26:53.373 INFO:tasks.ceph.mds.b.smithi175.stdout:starting mds.b at 2024-04-20T11:26:53.376 INFO:tasks.ceph.mds.d.smithi175.stdout:starting mds.d at 2024-04-20T11:26:53.399 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.399+0000 7f7c3de00640 1 Processor -- start 2024-04-20T11:26:53.399 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.399+0000 7f7c3de00640 1 -- start start 2024-04-20T11:26:53.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.400+0000 7f7c3de00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f7c3815fcc0 0x7f7c380d4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:53.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.400+0000 7f7c3de00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7c380b65c0 0x7f7c380cf240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:53.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.400+0000 7f7c3de00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c380d3350 con 0x7f7c381603d0 2024-04-20T11:26:53.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.400+0000 7f7c3de00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f7c380d3630 con 0x7f7c3815fcc0 2024-04-20T11:26:53.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.400+0000 7f7c3de00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7c380d3910 con 0x7f7c380b65c0 2024-04-20T11:26:53.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.401+0000 7f7c3ca00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f7c3815fcc0 0x7f7c380d4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:53.401 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.401+0000 7f7c2fe00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f7c381603d0 0x7f7c380b5eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:53846/0 (socket says 172.21.15.152:53846) 2024-04-20T11:26:53.401 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.401+0000 7f7c2fe00640 1 -- 172.21.15.152:0/92790389 learned_addr learned my addr 172.21.15.152:0/92790389 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:26:53.401 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.401+0000 7f7c3d400640 1 --2- 172.21.15.152:0/92790389 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7c380b65c0 0x7f7c380cf240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:53.402 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.401+0000 7f7c3ca00640 1 -- 172.21.15.152:0/92790389 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7c380b65c0 msgr2=0x7f7c380cf240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:53.402 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.401+0000 7f7c3ca00640 1 --2- 172.21.15.152:0/92790389 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7c380b65c0 0x7f7c380cf240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:53.402 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.401+0000 7f7c3ca00640 1 -- 172.21.15.152:0/92790389 >> v1:172.21.15.152:6789/0 conn(0x7f7c381603d0 legacy=0x7f7c380b5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:53.402 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.401+0000 7f7c3ca00640 1 -- 172.21.15.152:0/92790389 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c380ced20 con 0x7f7c3815fcc0 2024-04-20T11:26:53.402 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.401+0000 7f7c37e00640 1 -- 172.21.15.152:0/92790389 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3465984084 0 0) 0x7f7c380d3350 con 0x7f7c381603d0 2024-04-20T11:26:53.402 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.402+0000 7f7c3ca00640 1 --2- 172.21.15.152:0/92790389 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7c3815fcc0 0x7f7c380d4e70 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7c280097a0 tx=0x7f7c2802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=212beea711d31699 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:53.402 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.402+0000 7f7c37e00640 1 -- 172.21.15.152:0/92790389 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c28035020 con 0x7f7c3815fcc0 2024-04-20T11:26:53.402 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.402+0000 7f7c37e00640 1 -- 172.21.15.152:0/92790389 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c28031810 con 0x7f7c3815fcc0 2024-04-20T11:26:53.403 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.402+0000 7f7c37e00640 1 -- 172.21.15.152:0/92790389 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c28031be0 con 0x7f7c3815fcc0 2024-04-20T11:26:53.403 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.403+0000 7f7c3de00640 1 -- 172.21.15.152:0/92790389 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7c3815fcc0 msgr2=0x7f7c380d4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:53.404 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.403+0000 7f7c3de00640 1 --2- 172.21.15.152:0/92790389 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7c3815fcc0 0x7f7c380d4e70 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7c280097a0 tx=0x7f7c2802b8b0 comp rx=0 tx=0).stop 2024-04-20T11:26:53.404 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.403+0000 7f7c3de00640 1 -- 172.21.15.152:0/92790389 shutdown_connections 2024-04-20T11:26:53.404 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.403+0000 7f7c3de00640 1 --2- 172.21.15.152:0/92790389 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7c380b65c0 0x7f7c380cf240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:53.404 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.403+0000 7f7c3de00640 1 --2- 172.21.15.152:0/92790389 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7c3815fcc0 0x7f7c380d4e70 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:53.404 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.403+0000 7f7c3de00640 1 -- 172.21.15.152:0/92790389 >> 172.21.15.152:0/92790389 conn(0x7f7c3808bfd0 msgr2=0x7f7c380d4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:53.404 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.403+0000 7f7c3de00640 1 -- 172.21.15.152:0/92790389 shutdown_connections 2024-04-20T11:26:53.404 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.403+0000 7f7c3de00640 1 -- 172.21.15.152:0/92790389 wait complete. 2024-04-20T11:26:53.405 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.405+0000 7f7c3de00640 1 Processor -- start 2024-04-20T11:26:53.405 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.405+0000 7f7c3de00640 1 -- start start 2024-04-20T11:26:53.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.406+0000 7f7c3de00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7c380b65c0 0x7f7c380d28f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:53.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.406+0000 7f7c3de00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7c3815fcc0 0x7f7c3814d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:53.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.406+0000 7f7c3de00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c381603d0 0x7f7c3814dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:53.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.406+0000 7f7c3de00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f7c380b5600 con 0x7f7c380b65c0 2024-04-20T11:26:53.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.406+0000 7f7c3de00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7c380b5480 con 0x7f7c381603d0 2024-04-20T11:26:53.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.406+0000 7f7c3de00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7c380cf780 con 0x7f7c3815fcc0 2024-04-20T11:26:53.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.406+0000 7f7c3ca00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7c380b65c0 0x7f7c380d28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:53.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.406+0000 7f7c3ca00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7c380b65c0 0x7f7c380d28f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:45206/0 (socket says 172.21.15.152:45206) 2024-04-20T11:26:53.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.406+0000 7f7c3ca00640 1 -- 172.21.15.152:0/2718149138 learned_addr learned my addr 172.21.15.152:0/2718149138 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:26:53.408 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.406+0000 7f7c3d400640 1 --2- 172.21.15.152:0/2718149138 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c381603d0 0x7f7c3814dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:53.408 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.406+0000 7f7c2fe00640 1 --2- 172.21.15.152:0/2718149138 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7c3815fcc0 0x7f7c3814d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:53.408 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.406+0000 7f7c3ca00640 1 -- 172.21.15.152:0/2718149138 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7c3815fcc0 msgr2=0x7f7c3814d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:53.408 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.406+0000 7f7c3ca00640 1 --2- 172.21.15.152:0/2718149138 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7c3815fcc0 0x7f7c3814d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:53.408 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.406+0000 7f7c3ca00640 1 -- 172.21.15.152:0/2718149138 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c381603d0 msgr2=0x7f7c3814dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:53.408 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.406+0000 7f7c3ca00640 1 --2- 172.21.15.152:0/2718149138 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c381603d0 0x7f7c3814dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:53.408 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.406+0000 7f7c3ca00640 1 -- 172.21.15.152:0/2718149138 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c3814f090 con 0x7f7c380b65c0 2024-04-20T11:26:53.408 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.406+0000 7f7c2fe00640 1 --2- 172.21.15.152:0/2718149138 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7c3815fcc0 0x7f7c3814d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:26:53.408 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.407+0000 7f7c3ca00640 1 --2- 172.21.15.152:0/2718149138 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7c380b65c0 0x7f7c380d28f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7c280098c0 tx=0x7f7c280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:53.408 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.407+0000 7f7c35600640 1 -- 172.21.15.152:0/2718149138 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c28006e10 con 0x7f7c380b65c0 2024-04-20T11:26:53.408 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.407+0000 7f7c35600640 1 -- 172.21.15.152:0/2718149138 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c2803bd70 con 0x7f7c380b65c0 2024-04-20T11:26:53.409 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.407+0000 7f7c35600640 1 -- 172.21.15.152:0/2718149138 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c28045610 con 0x7f7c380b65c0 2024-04-20T11:26:53.409 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.407+0000 7f7c3de00640 1 -- 172.21.15.152:0/2718149138 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c3814f370 con 0x7f7c380b65c0 2024-04-20T11:26:53.409 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.408+0000 7f7c3de00640 1 -- 172.21.15.152:0/2718149138 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c3814f930 con 0x7f7c380b65c0 2024-04-20T11:26:53.409 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.409+0000 7f7c3de00640 1 -- 172.21.15.152:0/2718149138 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c38075e60 con 0x7f7c380b65c0 2024-04-20T11:26:53.409 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.410+0000 7f7c35600640 1 -- 172.21.15.152:0/2718149138 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7c2804c060 con 0x7f7c380b65c0 2024-04-20T11:26:53.410 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.411+0000 7f7c35600640 1 --2- 172.21.15.152:0/2718149138 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f7c1406c8f0 0x7f7c1406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:53.410 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.411+0000 7f7c35600640 1 -- 172.21.15.152:0/2718149138 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f7c280ad710 con 0x7f7c380b65c0 2024-04-20T11:26:53.410 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.411+0000 7f7c2fe00640 1 --2- 172.21.15.152:0/2718149138 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f7c1406c8f0 0x7f7c1406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:53.411 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.412+0000 7f7c2fe00640 1 --2- 172.21.15.152:0/2718149138 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f7c1406c8f0 0x7f7c1406edb0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7c20001000 tx=0x7f7c20009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:53.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.415+0000 7f7c35600640 1 -- 172.21.15.152:0/2718149138 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f7c28040070 con 0x7f7c380b65c0 2024-04-20T11:26:53.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.615+0000 7f7c3de00640 1 -- 172.21.15.152:0/2718149138 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7c380ba4d0 con 0x7f7c380b65c0 2024-04-20T11:26:53.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.617+0000 7f7c35600640 1 -- 172.21.15.152:0/2718149138 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12944 (secure 0 0 0) 0x7f7c2807bdf0 con 0x7f7c380b65c0 2024-04-20T11:26:53.617 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:26:53.617 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":16,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:26:49.573250+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:26:53.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.620+0000 7f7c3de00640 1 -- 172.21.15.152:0/2718149138 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f7c1406c8f0 msgr2=0x7f7c1406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:53.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.620+0000 7f7c3de00640 1 --2- 172.21.15.152:0/2718149138 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f7c1406c8f0 0x7f7c1406edb0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7c20001000 tx=0x7f7c20009040 comp rx=0 tx=0).stop 2024-04-20T11:26:53.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.620+0000 7f7c3de00640 1 -- 172.21.15.152:0/2718149138 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7c380b65c0 msgr2=0x7f7c380d28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:53.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.620+0000 7f7c3de00640 1 --2- 172.21.15.152:0/2718149138 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7c380b65c0 0x7f7c380d28f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7c280098c0 tx=0x7f7c280046c0 comp rx=0 tx=0).stop 2024-04-20T11:26:53.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.620+0000 7f7c3de00640 1 -- 172.21.15.152:0/2718149138 shutdown_connections 2024-04-20T11:26:53.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.620+0000 7f7c3de00640 1 --2- 172.21.15.152:0/2718149138 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f7c1406c8f0 0x7f7c1406edb0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:53.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.620+0000 7f7c3de00640 1 --2- 172.21.15.152:0/2718149138 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c381603d0 0x7f7c3814dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:53.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.620+0000 7f7c3de00640 1 --2- 172.21.15.152:0/2718149138 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7c3815fcc0 0x7f7c3814d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:53.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.620+0000 7f7c3de00640 1 --2- 172.21.15.152:0/2718149138 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7c380b65c0 0x7f7c380d28f0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:53.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.620+0000 7f7c3de00640 1 -- 172.21.15.152:0/2718149138 >> 172.21.15.152:0/2718149138 conn(0x7f7c3808bfd0 msgr2=0x7f7c38161ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:53.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.620+0000 7f7c3de00640 1 -- 172.21.15.152:0/2718149138 shutdown_connections 2024-04-20T11:26:53.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.621+0000 7f7c3de00640 1 -- 172.21.15.152:0/2718149138 wait complete. 2024-04-20T11:26:53.637 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T11:26:47.246571+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T11:26:53.637 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T11:26:53.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.830+0000 7f6766600640 1 Processor -- start 2024-04-20T11:26:53.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.830+0000 7f6766600640 1 -- start start 2024-04-20T11:26:53.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.831+0000 7f6766600640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f6760152450 0x7f676015c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:53.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.831+0000 7f6766600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f67600b3520 0x7f67600ae470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:53.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.831+0000 7f6766600640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6760152830 con 0x7f67600b30c0 2024-04-20T11:26:53.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.831+0000 7f6766600640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f67600b39c0 con 0x7f6760152450 2024-04-20T11:26:53.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.831+0000 7f6766600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f67600b3ca0 con 0x7f67600b3520 2024-04-20T11:26:53.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.832+0000 7f6765200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f6760152450 0x7f676015c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:53.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.832+0000 7f6765c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f67600b3520 0x7f67600ae470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:53.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.832+0000 7f6765200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f6760152450 0x7f676015c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.152:54604/0 (socket says 172.21.15.152:54604) 2024-04-20T11:26:53.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.832+0000 7f6765c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f67600b3520 0x7f67600ae470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.152:53592/0 (socket says 172.21.15.152:53592) 2024-04-20T11:26:53.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.832+0000 7f6765200640 1 -- 172.21.15.152:0/3380490002 learned_addr learned my addr 172.21.15.152:0/3380490002 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:26:53.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.833+0000 7f6765200640 1 -- 172.21.15.152:0/3380490002 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f67600b3520 msgr2=0x7f67600ae470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:53.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.833+0000 7f6765200640 1 --2- 172.21.15.152:0/3380490002 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f67600b3520 0x7f67600ae470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:53.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.833+0000 7f6765200640 1 -- 172.21.15.152:0/3380490002 >> v1:172.21.15.175:6789/0 conn(0x7f67600b30c0 legacy=0x7f676006bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:53.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.833+0000 7f6765200640 1 -- 172.21.15.152:0/3380490002 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67600b3f80 con 0x7f6760152450 2024-04-20T11:26:53.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.834+0000 7f6765200640 1 --2- 172.21.15.152:0/3380490002 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6760152450 0x7f676015c590 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f6754009870 tx=0x7f675402b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c35f172d225c12e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:53.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.835+0000 7f675f400640 1 -- 172.21.15.152:0/3380490002 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6754034070 con 0x7f6760152450 2024-04-20T11:26:53.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.835+0000 7f675f400640 1 -- 172.21.15.152:0/3380490002 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6754030840 con 0x7f6760152450 2024-04-20T11:26:53.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.835+0000 7f675f400640 1 -- 172.21.15.152:0/3380490002 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6754030c10 con 0x7f6760152450 2024-04-20T11:26:53.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.835+0000 7f6766600640 1 -- 172.21.15.152:0/3380490002 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6760152450 msgr2=0x7f676015c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:53.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.835+0000 7f6766600640 1 --2- 172.21.15.152:0/3380490002 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6760152450 0x7f676015c590 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f6754009870 tx=0x7f675402b980 comp rx=0 tx=0).stop 2024-04-20T11:26:53.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.835+0000 7f6766600640 1 -- 172.21.15.152:0/3380490002 shutdown_connections 2024-04-20T11:26:53.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.835+0000 7f6766600640 1 --2- 172.21.15.152:0/3380490002 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f67600b3520 0x7f67600ae470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:53.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.835+0000 7f6766600640 1 --2- 172.21.15.152:0/3380490002 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6760152450 0x7f676015c590 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:53.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.835+0000 7f6766600640 1 -- 172.21.15.152:0/3380490002 >> 172.21.15.152:0/3380490002 conn(0x7f67600b21e0 msgr2=0x7f67600b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:53.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.836+0000 7f6766600640 1 -- 172.21.15.152:0/3380490002 shutdown_connections 2024-04-20T11:26:53.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.836+0000 7f6766600640 1 -- 172.21.15.152:0/3380490002 wait complete. 2024-04-20T11:26:53.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.838+0000 7f6766600640 1 Processor -- start 2024-04-20T11:26:53.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.838+0000 7f6766600640 1 -- start start 2024-04-20T11:26:53.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.839+0000 7f6766600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67600b30c0 0x7f676014d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:53.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.839+0000 7f6766600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f67600b3520 0x7f67600cd820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:53.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.839+0000 7f6766600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f67600cf010 0x7f6760149b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:53.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.839+0000 7f6766600640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f676006c4b0 con 0x7f67600cf010 2024-04-20T11:26:53.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.839+0000 7f6766600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f67600adf00 con 0x7f67600b30c0 2024-04-20T11:26:53.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.839+0000 7f6766600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f676006c630 con 0x7f67600b3520 2024-04-20T11:26:53.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.839+0000 7f6765c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f67600cf010 0x7f6760149b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:53.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.839+0000 7f6765c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f67600cf010 0x7f6760149b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:45214/0 (socket says 172.21.15.152:45214) 2024-04-20T11:26:53.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.839+0000 7f6765c00640 1 -- 172.21.15.152:0/1823345184 learned_addr learned my addr 172.21.15.152:0/1823345184 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:26:53.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.840+0000 7f6765200640 1 --2- 172.21.15.152:0/1823345184 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67600b30c0 0x7f676014d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:53.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.840+0000 7f675fe00640 1 --2- 172.21.15.152:0/1823345184 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f67600b3520 0x7f67600cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:53.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.840+0000 7f6765c00640 1 -- 172.21.15.152:0/1823345184 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f67600b3520 msgr2=0x7f67600cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:53.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.840+0000 7f6765c00640 1 --2- 172.21.15.152:0/1823345184 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f67600b3520 0x7f67600cd820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:53.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.840+0000 7f6765c00640 1 -- 172.21.15.152:0/1823345184 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67600b30c0 msgr2=0x7f676014d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:53.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.840+0000 7f6765c00640 1 --2- 172.21.15.152:0/1823345184 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67600b30c0 0x7f676014d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:53.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.840+0000 7f6765c00640 1 -- 172.21.15.152:0/1823345184 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67600cdf70 con 0x7f67600cf010 2024-04-20T11:26:53.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.840+0000 7f675fe00640 1 --2- 172.21.15.152:0/1823345184 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f67600b3520 0x7f67600cd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:26:53.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.840+0000 7f6765c00640 1 --2- 172.21.15.152:0/1823345184 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f67600cf010 0x7f6760149b40 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f675800ef90 tx=0x7f675800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:53.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.840+0000 7f675cc00640 1 -- 172.21.15.152:0/1823345184 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6758018020 con 0x7f67600cf010 2024-04-20T11:26:53.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.840+0000 7f675cc00640 1 -- 172.21.15.152:0/1823345184 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6758014480 con 0x7f67600cf010 2024-04-20T11:26:53.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.840+0000 7f6766600640 1 -- 172.21.15.152:0/1823345184 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67600ce250 con 0x7f67600cf010 2024-04-20T11:26:53.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.841+0000 7f675cc00640 1 -- 172.21.15.152:0/1823345184 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6758009e60 con 0x7f67600cf010 2024-04-20T11:26:53.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.841+0000 7f6766600640 1 -- 172.21.15.152:0/1823345184 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67600ce810 con 0x7f67600cf010 2024-04-20T11:26:53.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.842+0000 7f6766600640 1 -- 172.21.15.152:0/1823345184 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6760075e60 con 0x7f67600cf010 2024-04-20T11:26:53.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.843+0000 7f675cc00640 1 -- 172.21.15.152:0/1823345184 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6758005d10 con 0x7f67600cf010 2024-04-20T11:26:53.843 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.843+0000 7f675cc00640 1 --2- 172.21.15.152:0/1823345184 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f674406c8f0 0x7f674406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:53.843 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.844+0000 7f675cc00640 1 -- 172.21.15.152:0/1823345184 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f675808c4a0 con 0x7f67600cf010 2024-04-20T11:26:53.843 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.844+0000 7f6765200640 1 --2- 172.21.15.152:0/1823345184 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f674406c8f0 0x7f674406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:53.844 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.844+0000 7f6765200640 1 --2- 172.21.15.152:0/1823345184 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f674406c8f0 0x7f674406edb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6754036000 tx=0x7f675403b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:53.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:53.848+0000 7f675cc00640 1 -- 172.21.15.152:0/1823345184 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f6758022030 con 0x7f67600cf010 2024-04-20T11:26:54.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.043+0000 7f6766600640 1 -- 172.21.15.152:0/1823345184 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f676006f770 con 0x7f67600cf010 2024-04-20T11:26:54.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.044+0000 7f675cc00640 1 -- 172.21.15.152:0/1823345184 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) ==== 93+0+10 (secure 0 0 0) 0x7f6758010040 con 0x7f67600cf010 2024-04-20T11:26:54.044 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 1 2024-04-20T11:26:54.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.047+0000 7f6766600640 1 -- 172.21.15.152:0/1823345184 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f674406c8f0 msgr2=0x7f674406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:54.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.047+0000 7f6766600640 1 --2- 172.21.15.152:0/1823345184 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f674406c8f0 0x7f674406edb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6754036000 tx=0x7f675403b040 comp rx=0 tx=0).stop 2024-04-20T11:26:54.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.047+0000 7f6766600640 1 -- 172.21.15.152:0/1823345184 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f67600cf010 msgr2=0x7f6760149b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:54.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.047+0000 7f6766600640 1 --2- 172.21.15.152:0/1823345184 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f67600cf010 0x7f6760149b40 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f675800ef90 tx=0x7f675800c490 comp rx=0 tx=0).stop 2024-04-20T11:26:54.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.047+0000 7f6766600640 1 -- 172.21.15.152:0/1823345184 shutdown_connections 2024-04-20T11:26:54.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.047+0000 7f6766600640 1 --2- 172.21.15.152:0/1823345184 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f674406c8f0 0x7f674406edb0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:54.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.047+0000 7f6766600640 1 --2- 172.21.15.152:0/1823345184 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f67600cf010 0x7f6760149b40 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:54.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.047+0000 7f6766600640 1 --2- 172.21.15.152:0/1823345184 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f67600b3520 0x7f67600cd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:54.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.047+0000 7f6766600640 1 --2- 172.21.15.152:0/1823345184 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67600b30c0 0x7f676014d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:54.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.047+0000 7f6766600640 1 -- 172.21.15.152:0/1823345184 >> 172.21.15.152:0/1823345184 conn(0x7f67600b21e0 msgr2=0x7f67600ae470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:54.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.047+0000 7f6766600640 1 -- 172.21.15.152:0/1823345184 shutdown_connections 2024-04-20T11:26:54.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.047+0000 7f6766600640 1 -- 172.21.15.152:0/1823345184 wait complete. 2024-04-20T11:26:54.065 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:26:54.262 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.262+0000 7faa16000640 1 Processor -- start 2024-04-20T11:26:54.262 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.262+0000 7faa16000640 1 -- start start 2024-04-20T11:26:54.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.264+0000 7faa16000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7faa1013e530 0x7faa1006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:54.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.264+0000 7faa16000640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7faa1006f340 0x7faa10140dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:54.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.264+0000 7faa16000640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa100cf0a0 con 0x7faa10141310 2024-04-20T11:26:54.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.264+0000 7faa16000640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7faa10141b00 con 0x7faa1013e530 2024-04-20T11:26:54.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.264+0000 7faa16000640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7faa100cfc50 con 0x7faa1006f340 2024-04-20T11:26:54.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.264+0000 7faa14c00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7faa1013e530 0x7faa1006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:54.265 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.264+0000 7faa14c00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7faa1013e530 0x7faa1006ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:45226/0 (socket says 172.21.15.152:45226) 2024-04-20T11:26:54.265 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.264+0000 7faa14c00640 1 -- 172.21.15.152:0/4164360894 learned_addr learned my addr 172.21.15.152:0/4164360894 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:26:54.265 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.265+0000 7faa07e00640 1 --2- 172.21.15.152:0/4164360894 >> v2:172.21.15.175:3300/0 conn(0x7faa1006f340 0x7faa10140dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:54.265 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.265+0000 7faa14c00640 1 -- 172.21.15.152:0/4164360894 >> v2:172.21.15.175:3300/0 conn(0x7faa1006f340 msgr2=0x7faa10140dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:54.265 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.265+0000 7faa14c00640 1 --2- 172.21.15.152:0/4164360894 >> v2:172.21.15.175:3300/0 conn(0x7faa1006f340 0x7faa10140dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:54.265 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.265+0000 7faa14c00640 1 -- 172.21.15.152:0/4164360894 >> v1:172.21.15.175:6789/0 conn(0x7faa10141310 legacy=0x7faa1015efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:54.265 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.265+0000 7faa14c00640 1 -- 172.21.15.152:0/4164360894 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa100d00e0 con 0x7faa1013e530 2024-04-20T11:26:54.266 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.266+0000 7faa14c00640 1 --2- 172.21.15.152:0/4164360894 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faa1013e530 0x7faa1006ee00 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7faa1006feb0 tx=0x7faa0002e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7deb8d3fef8a2aa server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:54.266 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.266+0000 7faa0fe00640 1 -- 172.21.15.152:0/4164360894 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa0003b070 con 0x7faa1013e530 2024-04-20T11:26:54.266 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.266+0000 7faa0fe00640 1 -- 172.21.15.152:0/4164360894 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa00036440 con 0x7faa1013e530 2024-04-20T11:26:54.266 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.266+0000 7faa0fe00640 1 -- 172.21.15.152:0/4164360894 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa00036810 con 0x7faa1013e530 2024-04-20T11:26:54.266 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.267+0000 7faa16000640 1 -- 172.21.15.152:0/4164360894 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faa1013e530 msgr2=0x7faa1006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:54.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.267+0000 7faa16000640 1 --2- 172.21.15.152:0/4164360894 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faa1013e530 0x7faa1006ee00 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7faa1006feb0 tx=0x7faa0002e0e0 comp rx=0 tx=0).stop 2024-04-20T11:26:54.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.267+0000 7faa16000640 1 -- 172.21.15.152:0/4164360894 shutdown_connections 2024-04-20T11:26:54.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.267+0000 7faa16000640 1 --2- 172.21.15.152:0/4164360894 >> v2:172.21.15.175:3300/0 conn(0x7faa1006f340 0x7faa10140dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:54.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.267+0000 7faa16000640 1 --2- 172.21.15.152:0/4164360894 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faa1013e530 0x7faa1006ee00 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:54.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.267+0000 7faa16000640 1 -- 172.21.15.152:0/4164360894 >> 172.21.15.152:0/4164360894 conn(0x7faa10067ae0 msgr2=0x7faa1006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:54.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.267+0000 7faa16000640 1 -- 172.21.15.152:0/4164360894 shutdown_connections 2024-04-20T11:26:54.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.267+0000 7faa16000640 1 -- 172.21.15.152:0/4164360894 wait complete. 2024-04-20T11:26:54.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.269+0000 7faa16000640 1 Processor -- start 2024-04-20T11:26:54.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.269+0000 7faa16000640 1 -- start start 2024-04-20T11:26:54.270 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.270+0000 7faa16000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faa1006f340 0x7faa10158820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:54.271 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.270+0000 7faa16000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faa1013e530 0x7faa10156010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:54.271 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.270+0000 7faa16000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faa10141310 0x7faa101556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:54.271 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.270+0000 7faa16000640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7faa100cf600 con 0x7faa10141310 2024-04-20T11:26:54.271 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.270+0000 7faa16000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7faa10072ce0 con 0x7faa1013e530 2024-04-20T11:26:54.271 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.270+0000 7faa16000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7faa100cf8e0 con 0x7faa1006f340 2024-04-20T11:26:54.271 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.270+0000 7faa15600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faa10141310 0x7faa101556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:54.271 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.270+0000 7faa15600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faa10141310 0x7faa101556b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:45240/0 (socket says 172.21.15.152:45240) 2024-04-20T11:26:54.271 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.270+0000 7faa15600640 1 -- 172.21.15.152:0/3727189951 learned_addr learned my addr 172.21.15.152:0/3727189951 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:26:54.271 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.270+0000 7faa14c00640 1 --2- 172.21.15.152:0/3727189951 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faa1006f340 0x7faa10158820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:54.271 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.270+0000 7faa07e00640 1 --2- 172.21.15.152:0/3727189951 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faa1013e530 0x7faa10156010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:54.271 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.270+0000 7faa15600640 1 -- 172.21.15.152:0/3727189951 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faa1006f340 msgr2=0x7faa10158820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:54.271 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.270+0000 7faa15600640 1 --2- 172.21.15.152:0/3727189951 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faa1006f340 0x7faa10158820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:54.271 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.271+0000 7faa15600640 1 -- 172.21.15.152:0/3727189951 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faa1013e530 msgr2=0x7faa10156010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:54.271 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.271+0000 7faa15600640 1 --2- 172.21.15.152:0/3727189951 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faa1013e530 0x7faa10156010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:54.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.271+0000 7faa15600640 1 -- 172.21.15.152:0/3727189951 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa10156550 con 0x7faa10141310 2024-04-20T11:26:54.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.271+0000 7faa07e00640 1 --2- 172.21.15.152:0/3727189951 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faa1013e530 0x7faa10156010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:26:54.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.271+0000 7faa14c00640 1 --2- 172.21.15.152:0/3727189951 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faa1006f340 0x7faa10158820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:26:54.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.271+0000 7faa15600640 1 --2- 172.21.15.152:0/3727189951 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faa10141310 0x7faa101556b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7faa0800b9c0 tx=0x7faa0800be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:54.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.271+0000 7faa0d600640 1 -- 172.21.15.152:0/3727189951 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa08019070 con 0x7faa10141310 2024-04-20T11:26:54.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.271+0000 7faa0d600640 1 -- 172.21.15.152:0/3727189951 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa0800cba0 con 0x7faa10141310 2024-04-20T11:26:54.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.271+0000 7faa0d600640 1 -- 172.21.15.152:0/3727189951 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa08014440 con 0x7faa10141310 2024-04-20T11:26:54.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.271+0000 7faa16000640 1 -- 172.21.15.152:0/3727189951 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa10158e20 con 0x7faa10141310 2024-04-20T11:26:54.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.271+0000 7faa16000640 1 -- 172.21.15.152:0/3727189951 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa10156c00 con 0x7faa10141310 2024-04-20T11:26:54.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.272+0000 7faa16000640 1 -- 172.21.15.152:0/3727189951 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa10075e60 con 0x7faa10141310 2024-04-20T11:26:54.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.273+0000 7faa0d600640 1 -- 172.21.15.152:0/3727189951 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faa08007370 con 0x7faa10141310 2024-04-20T11:26:54.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.275+0000 7faa0d600640 1 --2- 172.21.15.152:0/3727189951 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fa9ec06c920 0x7fa9ec06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:54.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.275+0000 7faa0d600640 1 -- 172.21.15.152:0/3727189951 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7faa0808b430 con 0x7faa10141310 2024-04-20T11:26:54.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.275+0000 7faa14c00640 1 --2- 172.21.15.152:0/3727189951 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fa9ec06c920 0x7fa9ec06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:54.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.276+0000 7faa14c00640 1 --2- 172.21.15.152:0/3727189951 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fa9ec06c920 0x7fa9ec06ede0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7faa00038000 tx=0x7faa00008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:54.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.280+0000 7faa0d600640 1 -- 172.21.15.152:0/3727189951 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7faa08059b10 con 0x7faa10141310 2024-04-20T11:26:54.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.519+0000 7faa16000640 1 -- 172.21.15.152:0/3727189951 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faa100b3490 con 0x7faa10141310 2024-04-20T11:26:54.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.520+0000 7faa0d600640 1 -- 172.21.15.152:0/3727189951 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3867 (secure 0 0 0) 0x7faa08059410 con 0x7faa10141310 2024-04-20T11:26:54.519 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:26:54.520 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/853233806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":853233806},{"type":"v1","addr":"172.21.15.175:6833","nonce":853233806}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4253,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/100300237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":100300237},{"type":"v1","addr":"172.21.15.175:6835","nonce":100300237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/2924034058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":2924034058},{"type":"v1","addr":"172.21.15.152:6835","nonce":2924034058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4269,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/927639144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":927639144},{"type":"v1","addr":"172.21.15.152:6837","nonce":927639144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-20T11:26:54.520 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 3 2024-04-20T11:26:54.522 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.522+0000 7faa16000640 1 -- 172.21.15.152:0/3727189951 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fa9ec06c920 msgr2=0x7fa9ec06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:54.522 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.522+0000 7faa16000640 1 --2- 172.21.15.152:0/3727189951 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fa9ec06c920 0x7fa9ec06ede0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7faa00038000 tx=0x7faa00008040 comp rx=0 tx=0).stop 2024-04-20T11:26:54.522 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.522+0000 7faa16000640 1 -- 172.21.15.152:0/3727189951 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faa10141310 msgr2=0x7faa101556b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:54.522 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.522+0000 7faa16000640 1 --2- 172.21.15.152:0/3727189951 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faa10141310 0x7faa101556b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7faa0800b9c0 tx=0x7faa0800be80 comp rx=0 tx=0).stop 2024-04-20T11:26:54.522 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.522+0000 7faa16000640 1 -- 172.21.15.152:0/3727189951 shutdown_connections 2024-04-20T11:26:54.523 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.522+0000 7faa16000640 1 --2- 172.21.15.152:0/3727189951 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fa9ec06c920 0x7fa9ec06ede0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:54.523 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.522+0000 7faa16000640 1 --2- 172.21.15.152:0/3727189951 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faa10141310 0x7faa101556b0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:54.523 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.522+0000 7faa16000640 1 --2- 172.21.15.152:0/3727189951 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faa1013e530 0x7faa10156010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:54.523 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.522+0000 7faa16000640 1 --2- 172.21.15.152:0/3727189951 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faa1006f340 0x7faa10158820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:54.523 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.522+0000 7faa16000640 1 -- 172.21.15.152:0/3727189951 >> 172.21.15.152:0/3727189951 conn(0x7faa10067ae0 msgr2=0x7faa100cd590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:54.523 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.522+0000 7faa16000640 1 -- 172.21.15.152:0/3727189951 shutdown_connections 2024-04-20T11:26:54.523 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.523+0000 7faa16000640 1 -- 172.21.15.152:0/3727189951 wait complete. 2024-04-20T11:26:54.540 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:26:54.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.730+0000 7fb1f5000640 1 Processor -- start 2024-04-20T11:26:54.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.730+0000 7fb1f5000640 1 -- start start 2024-04-20T11:26:54.731 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.731+0000 7fb1f5000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb1f006f510 0x7fb1f0140dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:54.731 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.731+0000 7fb1f5000640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1f00cf0a0 con 0x7fb1f013e530 2024-04-20T11:26:54.731 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.731+0000 7fb1f5000640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1f00d00e0 con 0x7fb1f0141310 2024-04-20T11:26:54.731 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.732+0000 7fb1f5000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb1f0141b00 con 0x7fb1f006f510 2024-04-20T11:26:54.731 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.732+0000 7fb1ef400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fb1f013e530 0x7fb1f006ee00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:53854/0 (socket says 172.21.15.152:53854) 2024-04-20T11:26:54.731 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.732+0000 7fb1ef400640 1 -- 172.21.15.152:0/3042129422 learned_addr learned my addr 172.21.15.152:0/3042129422 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:26:54.733 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.732+0000 7fb1eea00640 1 -- 172.21.15.152:0/3042129422 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3403549113 0 0) 0x7fb1e0001110 con 0x7fb1f013e530 2024-04-20T11:26:54.733 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.732+0000 7fb1eea00640 1 -- 172.21.15.152:0/3042129422 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb1d0003520 con 0x7fb1f013e530 2024-04-20T11:26:54.733 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.732+0000 7fb1e6a00640 1 --2- 172.21.15.152:0/3042129422 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb1f006f510 0x7fb1f0140dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:54.733 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.732+0000 7fb1eea00640 1 -- 172.21.15.152:0/3042129422 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3824833807 0 0) 0x7fb1d0003520 con 0x7fb1f013e530 2024-04-20T11:26:54.733 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.732+0000 7fb1eea00640 1 -- 172.21.15.152:0/3042129422 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb1f006f510 msgr2=0x7fb1f0140dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:54.733 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.732+0000 7fb1eea00640 1 --2- 172.21.15.152:0/3042129422 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb1f006f510 0x7fb1f0140dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:54.733 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.732+0000 7fb1eea00640 1 -- 172.21.15.152:0/3042129422 >> v1:172.21.15.175:6789/0 conn(0x7fb1f0141310 legacy=0x7fb1f0071cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:54.733 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.732+0000 7fb1eea00640 1 -- 172.21.15.152:0/3042129422 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1f00cf320 con 0x7fb1f013e530 2024-04-20T11:26:54.734 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.733+0000 7fb1eea00640 1 -- 172.21.15.152:0/3042129422 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7fb1e0001840 con 0x7fb1f013e530 2024-04-20T11:26:54.734 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.733+0000 7fb1eea00640 1 -- 172.21.15.152:0/3042129422 >> v1:172.21.15.152:6789/0 conn(0x7fb1f013e530 legacy=0x7fb1f006ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:54.734 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.733+0000 7fb1eea00640 1 --2- 172.21.15.152:0/3042129422 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb1d00039e0 0x7fb1d0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:54.734 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.733+0000 7fb1eea00640 1 --2- 172.21.15.152:0/3042129422 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1d0004800 0x7fb1d0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:54.734 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.733+0000 7fb1eea00640 1 --2- 172.21.15.152:0/3042129422 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb1f006f510 0x7fb1d0005270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:54.734 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.733+0000 7fb1eea00640 1 -- 172.21.15.152:0/3042129422 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fb1d000dd60 con 0x7fb1d00039e0 2024-04-20T11:26:54.734 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.733+0000 7fb1eea00640 1 -- 172.21.15.152:0/3042129422 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb1d000dee0 con 0x7fb1d0004800 2024-04-20T11:26:54.734 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.733+0000 7fb1eea00640 1 -- 172.21.15.152:0/3042129422 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb1d000e0b0 con 0x7fb1f006f510 2024-04-20T11:26:54.734 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.733+0000 7fb1ef400640 1 --2- 172.21.15.152:0/3042129422 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb1d00039e0 0x7fb1d0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:54.734 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.733+0000 7fb1e6a00640 1 --2- 172.21.15.152:0/3042129422 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1d0004800 0x7fb1d0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:54.734 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.733+0000 7fb1ef400640 1 -- 172.21.15.152:0/3042129422 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb1f006f510 msgr2=0x7fb1d0005270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:54.734 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.733+0000 7fb1ef400640 1 --2- 172.21.15.152:0/3042129422 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb1f006f510 0x7fb1d0005270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:54.734 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.733+0000 7fb1ef400640 1 -- 172.21.15.152:0/3042129422 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1d0004800 msgr2=0x7fb1d0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:54.734 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.733+0000 7fb1ef400640 1 --2- 172.21.15.152:0/3042129422 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1d0004800 0x7fb1d0004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:54.735 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.733+0000 7fb1ef400640 1 -- 172.21.15.152:0/3042129422 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1d0003520 con 0x7fb1d00039e0 2024-04-20T11:26:54.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.734+0000 7fb1ef400640 1 --2- 172.21.15.152:0/3042129422 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb1d00039e0 0x7fb1d0003de0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb1e000c320 tx=0x7fb1d0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49640968e7bcc53b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:54.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.734+0000 7fb1eea00640 1 -- 172.21.15.152:0/3042129422 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e0007e10 con 0x7fb1d00039e0 2024-04-20T11:26:54.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.734+0000 7fb1eea00640 1 -- 172.21.15.152:0/3042129422 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1e0030070 con 0x7fb1d00039e0 2024-04-20T11:26:54.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.734+0000 7fb1eea00640 1 -- 172.21.15.152:0/3042129422 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e002ce50 con 0x7fb1d00039e0 2024-04-20T11:26:54.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.735+0000 7fb1f5000640 1 -- 172.21.15.152:0/3042129422 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb1d00039e0 msgr2=0x7fb1d0003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:54.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.735+0000 7fb1f5000640 1 --2- 172.21.15.152:0/3042129422 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb1d00039e0 0x7fb1d0003de0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb1e000c320 tx=0x7fb1d0005830 comp rx=0 tx=0).stop 2024-04-20T11:26:54.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.735+0000 7fb1efe00640 1 -- 172.21.15.152:0/3042129422 reap_dead start 2024-04-20T11:26:54.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.735+0000 7fb1f5000640 1 -- 172.21.15.152:0/3042129422 shutdown_connections 2024-04-20T11:26:54.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.735+0000 7fb1f5000640 1 -- 172.21.15.152:0/3042129422 >> 172.21.15.152:0/3042129422 conn(0x7fb1f0067ae0 msgr2=0x7fb1f006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:54.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.735+0000 7fb1f5000640 1 -- 172.21.15.152:0/3042129422 shutdown_connections 2024-04-20T11:26:54.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.735+0000 7fb1f5000640 1 -- 172.21.15.152:0/3042129422 wait complete. 2024-04-20T11:26:54.738 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.737+0000 7fb1f5000640 1 Processor -- start 2024-04-20T11:26:54.738 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.737+0000 7fb1f5000640 1 -- start start 2024-04-20T11:26:54.738 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.738+0000 7fb1f5000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb1d00039e0 0x7fb1f01e3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:54.739 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.738+0000 7fb1f5000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb1f013d850 0x7fb1f015ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:54.739 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.738+0000 7fb1f5000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1f01e5d00 0x7fb1f013f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:54.739 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.738+0000 7fb1f5000640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f00cf8e0 con 0x7fb1f013d850 2024-04-20T11:26:54.739 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.738+0000 7fb1f5000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f00cfc50 con 0x7fb1f01e5d00 2024-04-20T11:26:54.739 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.738+0000 7fb1f5000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb1f01417c0 con 0x7fb1d00039e0 2024-04-20T11:26:54.739 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.738+0000 7fb1e6a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb1f013d850 0x7fb1f015ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:54.739 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.738+0000 7fb1e6a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb1f013d850 0x7fb1f015ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:45246/0 (socket says 172.21.15.152:45246) 2024-04-20T11:26:54.739 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.738+0000 7fb1e6a00640 1 -- 172.21.15.152:0/2466985226 learned_addr learned my addr 172.21.15.152:0/2466985226 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:26:54.739 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.738+0000 7fb1ef400640 1 --2- 172.21.15.152:0/2466985226 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb1d00039e0 0x7fb1f01e3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:54.739 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.738+0000 7fb1efe00640 1 --2- 172.21.15.152:0/2466985226 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1f01e5d00 0x7fb1f013f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:54.739 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.738+0000 7fb1e6a00640 1 -- 172.21.15.152:0/2466985226 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb1d00039e0 msgr2=0x7fb1f01e3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:54.739 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.738+0000 7fb1e6a00640 1 --2- 172.21.15.152:0/2466985226 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb1d00039e0 0x7fb1f01e3ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:54.739 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.738+0000 7fb1e6a00640 1 -- 172.21.15.152:0/2466985226 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1f01e5d00 msgr2=0x7fb1f013f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:54.740 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.738+0000 7fb1e6a00640 1 --2- 172.21.15.152:0/2466985226 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1f01e5d00 0x7fb1f013f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:54.740 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.738+0000 7fb1e6a00640 1 -- 172.21.15.152:0/2466985226 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1f01e4c60 con 0x7fb1f013d850 2024-04-20T11:26:54.740 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.738+0000 7fb1ef400640 1 --2- 172.21.15.152:0/2466985226 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb1d00039e0 0x7fb1f01e3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:26:54.740 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.739+0000 7fb1e6a00640 1 --2- 172.21.15.152:0/2466985226 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb1f013d850 0x7fb1f015ad10 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb1d800e7e0 tx=0x7fb1d800eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:54.740 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.739+0000 7fb1e7e00640 1 -- 172.21.15.152:0/2466985226 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1d8017070 con 0x7fb1f013d850 2024-04-20T11:26:54.740 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.739+0000 7fb1e7e00640 1 -- 172.21.15.152:0/2466985226 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1d800c670 con 0x7fb1f013d850 2024-04-20T11:26:54.740 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.739+0000 7fb1f5000640 1 -- 172.21.15.152:0/2466985226 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1f01e4f40 con 0x7fb1f013d850 2024-04-20T11:26:54.740 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.739+0000 7fb1e7e00640 1 -- 172.21.15.152:0/2466985226 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1d800f040 con 0x7fb1f013d850 2024-04-20T11:26:54.740 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.739+0000 7fb1f5000640 1 -- 172.21.15.152:0/2466985226 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1f01e5500 con 0x7fb1f013d850 2024-04-20T11:26:54.740 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.740+0000 7fb1f5000640 1 -- 172.21.15.152:0/2466985226 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1f0075e60 con 0x7fb1f013d850 2024-04-20T11:26:54.741 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.741+0000 7fb1e7e00640 1 -- 172.21.15.152:0/2466985226 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb1d801a070 con 0x7fb1f013d850 2024-04-20T11:26:54.741 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.742+0000 7fb1e7e00640 1 --2- 172.21.15.152:0/2466985226 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fb1cc06c8f0 0x7fb1cc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:54.741 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.742+0000 7fb1e7e00640 1 -- 172.21.15.152:0/2466985226 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fb1d808b230 con 0x7fb1f013d850 2024-04-20T11:26:54.741 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.742+0000 7fb1ef400640 1 --2- 172.21.15.152:0/2466985226 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fb1cc06c8f0 0x7fb1cc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:54.742 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.743+0000 7fb1ef400640 1 --2- 172.21.15.152:0/2466985226 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fb1cc06c8f0 0x7fb1cc06edb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb1e0034000 tx=0x7fb1e0007000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:54.745 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.746+0000 7fb1e7e00640 1 -- 172.21.15.152:0/2466985226 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fb1d8059a40 con 0x7fb1f013d850 2024-04-20T11:26:54.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.948+0000 7fb1f5000640 1 -- 172.21.15.152:0/2466985226 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb1f00b4610 con 0x7fb1f013d850 2024-04-20T11:26:54.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.950+0000 7fb1e7e00640 1 -- 172.21.15.152:0/2466985226 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12944 (secure 0 0 0) 0x7fb1d8059340 con 0x7fb1f013d850 2024-04-20T11:26:54.950 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:26:54.950 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":16,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:26:49.573250+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:26:54.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.952+0000 7fb1f5000640 1 -- 172.21.15.152:0/2466985226 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fb1cc06c8f0 msgr2=0x7fb1cc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:54.953 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.952+0000 7fb1f5000640 1 --2- 172.21.15.152:0/2466985226 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fb1cc06c8f0 0x7fb1cc06edb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb1e0034000 tx=0x7fb1e0007000 comp rx=0 tx=0).stop 2024-04-20T11:26:54.953 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.952+0000 7fb1f5000640 1 -- 172.21.15.152:0/2466985226 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb1f013d850 msgr2=0x7fb1f015ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:54.953 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.953+0000 7fb1f5000640 1 --2- 172.21.15.152:0/2466985226 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb1f013d850 0x7fb1f015ad10 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb1d800e7e0 tx=0x7fb1d800eca0 comp rx=0 tx=0).stop 2024-04-20T11:26:54.953 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.953+0000 7fb1f5000640 1 -- 172.21.15.152:0/2466985226 shutdown_connections 2024-04-20T11:26:54.953 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.953+0000 7fb1f5000640 1 --2- 172.21.15.152:0/2466985226 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fb1cc06c8f0 0x7fb1cc06edb0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:54.953 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.953+0000 7fb1f5000640 1 --2- 172.21.15.152:0/2466985226 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1f01e5d00 0x7fb1f013f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:54.953 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.953+0000 7fb1f5000640 1 --2- 172.21.15.152:0/2466985226 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb1f013d850 0x7fb1f015ad10 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:54.953 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.953+0000 7fb1f5000640 1 --2- 172.21.15.152:0/2466985226 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb1d00039e0 0x7fb1f01e3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:54.953 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.953+0000 7fb1f5000640 1 -- 172.21.15.152:0/2466985226 >> 172.21.15.152:0/2466985226 conn(0x7fb1f0067ae0 msgr2=0x7fb1f015d170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:54.953 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.953+0000 7fb1f5000640 1 -- 172.21.15.152:0/2466985226 shutdown_connections 2024-04-20T11:26:54.953 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:54.953+0000 7fb1f5000640 1 -- 172.21.15.152:0/2466985226 wait complete. 2024-04-20T11:26:54.970 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T11:26:47.246571+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T11:26:54.970 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T11:26:55.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.164+0000 7fdfffe00640 1 Processor -- start 2024-04-20T11:26:55.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.165+0000 7fdfffe00640 1 -- start start 2024-04-20T11:26:55.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.166+0000 7fdfffe00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fe00015e640 0x7fe00006dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:55.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.166+0000 7fdfffe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe00015ea20 0x7fe00006bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:55.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.166+0000 7fdfffe00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0000b5070 con 0x7fe00006f410 2024-04-20T11:26:55.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.166+0000 7fdfffe00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fe00006e1a0 con 0x7fe00015e640 2024-04-20T11:26:55.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.166+0000 7fdfffe00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe00006e480 con 0x7fe00015ea20 2024-04-20T11:26:55.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.167+0000 7fdffea00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fe00015e640 0x7fe00006dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:55.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.167+0000 7fdff7e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe00015ea20 0x7fe00006bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:55.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.167+0000 7fdff7e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe00015ea20 0x7fe00006bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.152:53644/0 (socket says 172.21.15.152:53644) 2024-04-20T11:26:55.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.167+0000 7fdff7e00640 1 -- 172.21.15.152:0/30030858 learned_addr learned my addr 172.21.15.152:0/30030858 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:26:55.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.167+0000 7fdffea00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fe00015e640 0x7fe00006dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.152:54672/0 (socket says 172.21.15.152:54672) 2024-04-20T11:26:55.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.167+0000 7fdffea00640 1 -- 172.21.15.152:0/30030858 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe00015ea20 msgr2=0x7fe00006bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:55.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.167+0000 7fdffea00640 1 --2- 172.21.15.152:0/30030858 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe00015ea20 0x7fe00006bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:55.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.167+0000 7fdffea00640 1 -- 172.21.15.152:0/30030858 >> v1:172.21.15.175:6789/0 conn(0x7fe00006f410 legacy=0x7fe0000b3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:55.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.167+0000 7fdffea00640 1 -- 172.21.15.152:0/30030858 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe00006e760 con 0x7fe00015e640 2024-04-20T11:26:55.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.167+0000 7fdff7e00640 1 --2- 172.21.15.152:0/30030858 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe00015ea20 0x7fe00006bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:26:55.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.169+0000 7fdffea00640 1 --2- 172.21.15.152:0/30030858 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe00015e640 0x7fe00006dbc0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fdff00097a0 tx=0x7fdff002b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac74d668a38031cb server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:55.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.169+0000 7fdffe000640 1 -- 172.21.15.152:0/30030858 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdff0035020 con 0x7fe00015e640 2024-04-20T11:26:55.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.169+0000 7fdffe000640 1 -- 172.21.15.152:0/30030858 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdff0031810 con 0x7fe00015e640 2024-04-20T11:26:55.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.169+0000 7fdffe000640 1 -- 172.21.15.152:0/30030858 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdff0031be0 con 0x7fe00015e640 2024-04-20T11:26:55.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.170+0000 7fdfffe00640 1 -- 172.21.15.152:0/30030858 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe00015e640 msgr2=0x7fe00006dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:55.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.170+0000 7fdfffe00640 1 --2- 172.21.15.152:0/30030858 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe00015e640 0x7fe00006dbc0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fdff00097a0 tx=0x7fdff002b8b0 comp rx=0 tx=0).stop 2024-04-20T11:26:55.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.170+0000 7fdfffe00640 1 -- 172.21.15.152:0/30030858 shutdown_connections 2024-04-20T11:26:55.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.170+0000 7fdfffe00640 1 --2- 172.21.15.152:0/30030858 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe00015ea20 0x7fe00006bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:55.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.170+0000 7fdfffe00640 1 --2- 172.21.15.152:0/30030858 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe00015e640 0x7fe00006dbc0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:55.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.170+0000 7fdfffe00640 1 -- 172.21.15.152:0/30030858 >> 172.21.15.152:0/30030858 conn(0x7fe0000b21e0 msgr2=0x7fe0000b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:55.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.170+0000 7fdfffe00640 1 -- 172.21.15.152:0/30030858 shutdown_connections 2024-04-20T11:26:55.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.170+0000 7fdfffe00640 1 -- 172.21.15.152:0/30030858 wait complete. 2024-04-20T11:26:55.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.172+0000 7fdfffe00640 1 Processor -- start 2024-04-20T11:26:55.172 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.172+0000 7fdfffe00640 1 -- start start 2024-04-20T11:26:55.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.173+0000 7fdfffe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe00006f410 0x7fe00014b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:55.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.173+0000 7fdfffe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe00015e640 0x7fe000151d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:55.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.174+0000 7fdfffe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe000161980 0x7fe0001545b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:55.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.174+0000 7fdfffe00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fe0000ae5d0 con 0x7fe00006f410 2024-04-20T11:26:55.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.174+0000 7fdfffe00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe0000ad0a0 con 0x7fe000161980 2024-04-20T11:26:55.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.174+0000 7fdfffe00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe0000ae750 con 0x7fe00015e640 2024-04-20T11:26:55.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.174+0000 7fdffea00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe00006f410 0x7fe00014b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:55.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.174+0000 7fdffea00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe00006f410 0x7fe00014b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:45258/0 (socket says 172.21.15.152:45258) 2024-04-20T11:26:55.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.174+0000 7fdffea00640 1 -- 172.21.15.152:0/2865678196 learned_addr learned my addr 172.21.15.152:0/2865678196 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:26:55.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.174+0000 7fdff7e00640 1 --2- 172.21.15.152:0/2865678196 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe00015e640 0x7fe000151d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:55.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.174+0000 7fdfff400640 1 --2- 172.21.15.152:0/2865678196 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe000161980 0x7fe0001545b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:55.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.174+0000 7fdffea00640 1 -- 172.21.15.152:0/2865678196 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe00015e640 msgr2=0x7fe000151d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:55.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.174+0000 7fdffea00640 1 --2- 172.21.15.152:0/2865678196 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe00015e640 0x7fe000151d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:55.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.174+0000 7fdffea00640 1 -- 172.21.15.152:0/2865678196 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe000161980 msgr2=0x7fe0001545b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:55.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.174+0000 7fdffea00640 1 --2- 172.21.15.152:0/2865678196 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe000161980 0x7fe0001545b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:55.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.174+0000 7fdffea00640 1 -- 172.21.15.152:0/2865678196 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe000150a20 con 0x7fe00006f410 2024-04-20T11:26:55.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.174+0000 7fdfff400640 1 --2- 172.21.15.152:0/2865678196 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe000161980 0x7fe0001545b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:26:55.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.174+0000 7fdff7e00640 1 --2- 172.21.15.152:0/2865678196 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe00015e640 0x7fe000151d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:26:55.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.175+0000 7fdffea00640 1 --2- 172.21.15.152:0/2865678196 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe00006f410 0x7fe00014b150 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fdff002bec0 tx=0x7fdff00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:55.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.175+0000 7fdff7400640 1 -- 172.21.15.152:0/2865678196 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdff0041070 con 0x7fe00006f410 2024-04-20T11:26:55.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.175+0000 7fdff7400640 1 -- 172.21.15.152:0/2865678196 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdff003ad00 con 0x7fe00006f410 2024-04-20T11:26:55.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.175+0000 7fdff7400640 1 -- 172.21.15.152:0/2865678196 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdff0035040 con 0x7fe00006f410 2024-04-20T11:26:55.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.175+0000 7fdfffe00640 1 -- 172.21.15.152:0/2865678196 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe000150cc0 con 0x7fe00006f410 2024-04-20T11:26:55.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.175+0000 7fdfffe00640 1 -- 172.21.15.152:0/2865678196 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe000150f60 con 0x7fe00006f410 2024-04-20T11:26:55.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.176+0000 7fdfffe00640 1 -- 172.21.15.152:0/2865678196 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe000075e60 con 0x7fe00006f410 2024-04-20T11:26:55.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.177+0000 7fdff7400640 1 -- 172.21.15.152:0/2865678196 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdff004c060 con 0x7fe00006f410 2024-04-20T11:26:55.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.179+0000 7fdff7400640 1 --2- 172.21.15.152:0/2865678196 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fdfdc06c920 0x7fdfdc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:55.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.179+0000 7fdff7400640 1 -- 172.21.15.152:0/2865678196 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fdff00ad4d0 con 0x7fe00006f410 2024-04-20T11:26:55.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.179+0000 7fdff7e00640 1 --2- 172.21.15.152:0/2865678196 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fdfdc06c920 0x7fdfdc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:55.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.180+0000 7fdff7e00640 1 --2- 172.21.15.152:0/2865678196 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fdfdc06c920 0x7fdfdc06ede0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe000161530 tx=0x7fdfe8008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:55.182 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.183+0000 7fdff7400640 1 -- 172.21.15.152:0/2865678196 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fdff004b2c0 con 0x7fe00006f410 2024-04-20T11:26:55.369 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.369+0000 7fdfffe00640 1 -- 172.21.15.152:0/2865678196 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe0000b4840 con 0x7fe00006f410 2024-04-20T11:26:55.369 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.370+0000 7fdff7400640 1 -- 172.21.15.152:0/2865678196 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) ==== 93+0+10 (secure 0 0 0) 0x7fdff007bb20 con 0x7fe00006f410 2024-04-20T11:26:55.369 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 1 2024-04-20T11:26:55.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.372+0000 7fdfffe00640 1 -- 172.21.15.152:0/2865678196 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fdfdc06c920 msgr2=0x7fdfdc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:55.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.372+0000 7fdfffe00640 1 --2- 172.21.15.152:0/2865678196 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fdfdc06c920 0x7fdfdc06ede0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe000161530 tx=0x7fdfe8008040 comp rx=0 tx=0).stop 2024-04-20T11:26:55.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.372+0000 7fdfffe00640 1 -- 172.21.15.152:0/2865678196 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe00006f410 msgr2=0x7fe00014b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:55.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.372+0000 7fdfffe00640 1 --2- 172.21.15.152:0/2865678196 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe00006f410 0x7fe00014b150 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fdff002bec0 tx=0x7fdff00046c0 comp rx=0 tx=0).stop 2024-04-20T11:26:55.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.372+0000 7fdfffe00640 1 -- 172.21.15.152:0/2865678196 shutdown_connections 2024-04-20T11:26:55.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.372+0000 7fdfffe00640 1 --2- 172.21.15.152:0/2865678196 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fdfdc06c920 0x7fdfdc06ede0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:55.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.372+0000 7fdfffe00640 1 --2- 172.21.15.152:0/2865678196 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe000161980 0x7fe0001545b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:55.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.372+0000 7fdfffe00640 1 --2- 172.21.15.152:0/2865678196 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe00015e640 0x7fe000151d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:55.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.372+0000 7fdfffe00640 1 --2- 172.21.15.152:0/2865678196 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe00006f410 0x7fe00014b150 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:55.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.372+0000 7fdfffe00640 1 -- 172.21.15.152:0/2865678196 >> 172.21.15.152:0/2865678196 conn(0x7fe0000b21e0 msgr2=0x7fe00006bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:55.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.373+0000 7fdfffe00640 1 -- 172.21.15.152:0/2865678196 shutdown_connections 2024-04-20T11:26:55.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.373+0000 7fdfffe00640 1 -- 172.21.15.152:0/2865678196 wait complete. 2024-04-20T11:26:55.390 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-20T11:26:55.586 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.586+0000 7f21f7200640 1 Processor -- start 2024-04-20T11:26:55.586 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.586+0000 7f21f7200640 1 -- start start 2024-04-20T11:26:55.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.588+0000 7f21f7200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f21f0145f60 0x7f21f006ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:55.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.588+0000 7f21f7200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f21f00cd0d0 0x7f21f00cd580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:55.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.588+0000 7f21f7200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f21f0071fe0 0x7f21f0072470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:55.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.588+0000 7f21f7200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f21f006e9f0 con 0x7f21f00cd0d0 2024-04-20T11:26:55.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.588+0000 7f21f7200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f21f0146340 con 0x7f21f0145f60 2024-04-20T11:26:55.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.588+0000 7f21f7200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f21f006f3c0 con 0x7f21f0071fe0 2024-04-20T11:26:55.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.588+0000 7f21ef400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f21f00cd0d0 0x7f21f00cd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:55.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.588+0000 7f21ef400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f21f00cd0d0 0x7f21f00cd580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:45270/0 (socket says 172.21.15.152:45270) 2024-04-20T11:26:55.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.588+0000 7f21ef400640 1 -- 172.21.15.152:0/1100077598 learned_addr learned my addr 172.21.15.152:0/1100077598 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:26:55.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.589+0000 7f21efe00640 1 --2- 172.21.15.152:0/1100077598 >> v2:172.21.15.175:3300/0 conn(0x7f21f0145f60 0x7f21f006ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:55.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.589+0000 7f21f5200640 1 --2- 172.21.15.152:0/1100077598 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f21f0071fe0 0x7f21f0072470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:55.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.589+0000 7f21ef400640 1 -- 172.21.15.152:0/1100077598 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f21f0071fe0 msgr2=0x7f21f0072470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:55.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.589+0000 7f21ef400640 1 --2- 172.21.15.152:0/1100077598 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f21f0071fe0 0x7f21f0072470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:55.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.589+0000 7f21ef400640 1 -- 172.21.15.152:0/1100077598 >> v2:172.21.15.175:3300/0 conn(0x7f21f0145f60 msgr2=0x7f21f006ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:55.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.589+0000 7f21ef400640 1 --2- 172.21.15.152:0/1100077598 >> v2:172.21.15.175:3300/0 conn(0x7f21f0145f60 0x7f21f006ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:55.590 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.589+0000 7f21ef400640 1 -- 172.21.15.152:0/1100077598 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21f0072d50 con 0x7f21f00cd0d0 2024-04-20T11:26:55.590 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.589+0000 7f21efe00640 1 --2- 172.21.15.152:0/1100077598 >> v2:172.21.15.175:3300/0 conn(0x7f21f0145f60 0x7f21f006ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:26:55.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.590+0000 7f21ef400640 1 --2- 172.21.15.152:0/1100077598 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f21f00cd0d0 0x7f21f00cd580 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f21f0073230 tx=0x7f21dc02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d6dd284947b046a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:55.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.590+0000 7f21eea00640 1 -- 172.21.15.152:0/1100077598 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21dc035020 con 0x7f21f00cd0d0 2024-04-20T11:26:55.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.590+0000 7f21eea00640 1 -- 172.21.15.152:0/1100077598 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21dc031870 con 0x7f21f00cd0d0 2024-04-20T11:26:55.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.590+0000 7f21eea00640 1 -- 172.21.15.152:0/1100077598 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21dc031c40 con 0x7f21f00cd0d0 2024-04-20T11:26:55.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.591+0000 7f21f7200640 1 -- 172.21.15.152:0/1100077598 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f21f00cd0d0 msgr2=0x7f21f00cd580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:55.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.591+0000 7f21f7200640 1 --2- 172.21.15.152:0/1100077598 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f21f00cd0d0 0x7f21f00cd580 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f21f0073230 tx=0x7f21dc02b910 comp rx=0 tx=0).stop 2024-04-20T11:26:55.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.591+0000 7f21f7200640 1 -- 172.21.15.152:0/1100077598 shutdown_connections 2024-04-20T11:26:55.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.591+0000 7f21f7200640 1 --2- 172.21.15.152:0/1100077598 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f21f0071fe0 0x7f21f0072470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:55.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.591+0000 7f21f7200640 1 --2- 172.21.15.152:0/1100077598 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f21f00cd0d0 0x7f21f00cd580 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:55.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.591+0000 7f21f7200640 1 --2- 172.21.15.152:0/1100077598 >> v2:172.21.15.175:3300/0 conn(0x7f21f0145f60 0x7f21f006ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:55.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.591+0000 7f21f7200640 1 -- 172.21.15.152:0/1100077598 >> 172.21.15.152:0/1100077598 conn(0x7f21f0067cf0 msgr2=0x7f21f006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:55.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.591+0000 7f21f7200640 1 -- 172.21.15.152:0/1100077598 shutdown_connections 2024-04-20T11:26:55.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.591+0000 7f21f7200640 1 -- 172.21.15.152:0/1100077598 wait complete. 2024-04-20T11:26:55.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.593+0000 7f21f7200640 1 Processor -- start 2024-04-20T11:26:55.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.593+0000 7f21f7200640 1 -- start start 2024-04-20T11:26:55.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.594+0000 7f21f7200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f21f0071fe0 0x7f21f013df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:55.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.594+0000 7f21f7200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f21f00cd0d0 0x7f21f0154350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:55.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.594+0000 7f21f7200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f21f0145f60 0x7f21f00d0060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:55.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.594+0000 7f21f7200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f21f006e680 con 0x7f21f0071fe0 2024-04-20T11:26:55.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.594+0000 7f21f7200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f21f006e3a0 con 0x7f21f00cd0d0 2024-04-20T11:26:55.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.594+0000 7f21f7200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f21f00729b0 con 0x7f21f0145f60 2024-04-20T11:26:55.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.594+0000 7f21efe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f21f0071fe0 0x7f21f013df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:55.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.594+0000 7f21efe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f21f0071fe0 0x7f21f013df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:45272/0 (socket says 172.21.15.152:45272) 2024-04-20T11:26:55.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.594+0000 7f21efe00640 1 -- 172.21.15.152:0/4079146844 learned_addr learned my addr 172.21.15.152:0/4079146844 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:26:55.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.594+0000 7f21ef400640 1 --2- 172.21.15.152:0/4079146844 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f21f00cd0d0 0x7f21f0154350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:55.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.594+0000 7f21f5200640 1 --2- 172.21.15.152:0/4079146844 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f21f0145f60 0x7f21f00d0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:55.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.595+0000 7f21efe00640 1 -- 172.21.15.152:0/4079146844 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f21f0145f60 msgr2=0x7f21f00d0060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:55.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.595+0000 7f21efe00640 1 --2- 172.21.15.152:0/4079146844 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f21f0145f60 0x7f21f00d0060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:55.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.595+0000 7f21efe00640 1 -- 172.21.15.152:0/4079146844 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f21f00cd0d0 msgr2=0x7f21f0154350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:55.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.595+0000 7f21efe00640 1 --2- 172.21.15.152:0/4079146844 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f21f00cd0d0 0x7f21f0154350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:55.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.595+0000 7f21efe00640 1 -- 172.21.15.152:0/4079146844 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21f0154890 con 0x7f21f0071fe0 2024-04-20T11:26:55.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.595+0000 7f21efe00640 1 --2- 172.21.15.152:0/4079146844 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f21f0071fe0 0x7f21f013df50 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f21e000e7e0 tx=0x7f21e000eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:55.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.595+0000 7f21e7e00640 1 -- 172.21.15.152:0/4079146844 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e000f040 con 0x7f21f0071fe0 2024-04-20T11:26:55.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.595+0000 7f21e7e00640 1 -- 172.21.15.152:0/4079146844 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21e0004030 con 0x7f21f0071fe0 2024-04-20T11:26:55.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.595+0000 7f21f7200640 1 -- 172.21.15.152:0/4079146844 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21f01f4af0 con 0x7f21f0071fe0 2024-04-20T11:26:55.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.595+0000 7f21e7e00640 1 -- 172.21.15.152:0/4079146844 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e0004400 con 0x7f21f0071fe0 2024-04-20T11:26:55.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.595+0000 7f21f7200640 1 -- 172.21.15.152:0/4079146844 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21f01f50b0 con 0x7f21f0071fe0 2024-04-20T11:26:55.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.596+0000 7f21f7200640 1 -- 172.21.15.152:0/4079146844 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21f0072470 con 0x7f21f0071fe0 2024-04-20T11:26:55.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.597+0000 7f21e7e00640 1 -- 172.21.15.152:0/4079146844 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f21e000c6d0 con 0x7f21f0071fe0 2024-04-20T11:26:55.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.598+0000 7f21e7e00640 1 --2- 172.21.15.152:0/4079146844 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f21d006c8f0 0x7f21d006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:55.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.598+0000 7f21e7e00640 1 -- 172.21.15.152:0/4079146844 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f21e008b5f0 con 0x7f21f0071fe0 2024-04-20T11:26:55.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.599+0000 7f21ef400640 1 --2- 172.21.15.152:0/4079146844 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f21d006c8f0 0x7f21d006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:55.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.599+0000 7f21ef400640 1 --2- 172.21.15.152:0/4079146844 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f21d006c8f0 0x7f21d006edb0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f21dc036000 tx=0x7f21dc03c040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:55.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.603+0000 7f21e7e00640 1 -- 172.21.15.152:0/4079146844 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f21f0072470 con 0x7f21f0071fe0 2024-04-20T11:26:55.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.807+0000 7f21f7200640 1 -- 172.21.15.152:0/4079146844 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f21f0072770 con 0x7f21f0071fe0 2024-04-20T11:26:55.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.808+0000 7f21e7e00640 1 -- 172.21.15.152:0/4079146844 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) ==== 84+0+65 (secure 0 0 0) 0x7f21f0072770 con 0x7f21f0071fe0 2024-04-20T11:26:55.807 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:26:55.807 INFO:teuthology.orchestra.run.smithi152.stdout:[ 2024-04-20T11:26:55.808 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:26:55.808 INFO:teuthology.orchestra.run.smithi152.stdout: "poolnum": 1, 2024-04-20T11:26:55.808 INFO:teuthology.orchestra.run.smithi152.stdout: "poolname": ".mgr" 2024-04-20T11:26:55.808 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:26:55.808 INFO:teuthology.orchestra.run.smithi152.stdout:] 2024-04-20T11:26:55.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.810+0000 7f21f7200640 1 -- 172.21.15.152:0/4079146844 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f21d006c8f0 msgr2=0x7f21d006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:55.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.810+0000 7f21f7200640 1 --2- 172.21.15.152:0/4079146844 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f21d006c8f0 0x7f21d006edb0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f21dc036000 tx=0x7f21dc03c040 comp rx=0 tx=0).stop 2024-04-20T11:26:55.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.810+0000 7f21f7200640 1 -- 172.21.15.152:0/4079146844 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f21f0071fe0 msgr2=0x7f21f013df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:55.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.810+0000 7f21f7200640 1 --2- 172.21.15.152:0/4079146844 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f21f0071fe0 0x7f21f013df50 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f21e000e7e0 tx=0x7f21e000eca0 comp rx=0 tx=0).stop 2024-04-20T11:26:55.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.811+0000 7f21f7200640 1 -- 172.21.15.152:0/4079146844 shutdown_connections 2024-04-20T11:26:55.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.811+0000 7f21f7200640 1 --2- 172.21.15.152:0/4079146844 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f21d006c8f0 0x7f21d006edb0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:55.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.811+0000 7f21f7200640 1 --2- 172.21.15.152:0/4079146844 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f21f0145f60 0x7f21f00d0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:55.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.811+0000 7f21f7200640 1 --2- 172.21.15.152:0/4079146844 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f21f00cd0d0 0x7f21f0154350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:55.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.811+0000 7f21f7200640 1 --2- 172.21.15.152:0/4079146844 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f21f0071fe0 0x7f21f013df50 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:55.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.811+0000 7f21f7200640 1 -- 172.21.15.152:0/4079146844 >> 172.21.15.152:0/4079146844 conn(0x7f21f0067cf0 msgr2=0x7f21f006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:55.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.811+0000 7f21f7200640 1 -- 172.21.15.152:0/4079146844 shutdown_connections 2024-04-20T11:26:55.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:55.811+0000 7f21f7200640 1 -- 172.21.15.152:0/4079146844 wait complete. 2024-04-20T11:26:55.827 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-20T11:26:55.827 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-20T11:26:56.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.032+0000 7fa406200640 1 Processor -- start 2024-04-20T11:26:56.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.032+0000 7fa406200640 1 -- start start 2024-04-20T11:26:56.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.033+0000 7fa406200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fa400144fa0 0x7fa4000b3470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:56.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.033+0000 7fa406200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fa400145620 0x7fa40006e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:56.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.033+0000 7fa406200640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4000b3a50 con 0x7fa400071e20 2024-04-20T11:26:56.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.033+0000 7fa406200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fa40006ee20 con 0x7fa400145620 2024-04-20T11:26:56.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.033+0000 7fa406200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fa400145a80 con 0x7fa400144fa0 2024-04-20T11:26:56.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.034+0000 7fa3ffe00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fa400071e20 0x7fa4000af100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:40578/0 (socket says 172.21.15.152:40578) 2024-04-20T11:26:56.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.034+0000 7fa3ffe00640 1 -- 172.21.15.152:0/2408745144 learned_addr learned my addr 172.21.15.152:0/2408745144 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:26:56.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.034+0000 7fa3f7e00640 1 --2- 172.21.15.152:0/2408745144 >> v2:172.21.15.152:3300/0 conn(0x7fa400145620 0x7fa40006e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:56.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.034+0000 7fa3ff400640 1 --2- 172.21.15.152:0/2408745144 >> v2:172.21.15.175:3300/0 conn(0x7fa400144fa0 0x7fa4000b3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:56.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.034+0000 7fa3fea00640 1 -- 172.21.15.152:0/2408745144 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 840082072 0 0) 0x7fa4000b3a50 con 0x7fa400071e20 2024-04-20T11:26:56.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.034+0000 7fa3fea00640 1 -- 172.21.15.152:0/2408745144 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa3e4003520 con 0x7fa400071e20 2024-04-20T11:26:56.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.034+0000 7fa3f7e00640 1 -- 172.21.15.152:0/2408745144 >> v2:172.21.15.175:3300/0 conn(0x7fa400144fa0 msgr2=0x7fa4000b3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:56.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.034+0000 7fa3f7e00640 1 --2- 172.21.15.152:0/2408745144 >> v2:172.21.15.175:3300/0 conn(0x7fa400144fa0 0x7fa4000b3470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:56.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.034+0000 7fa3f7e00640 1 -- 172.21.15.152:0/2408745144 >> v1:172.21.15.152:6789/0 conn(0x7fa400071e20 legacy=0x7fa4000af100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:56.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.034+0000 7fa3f7e00640 1 -- 172.21.15.152:0/2408745144 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4000b3d30 con 0x7fa400145620 2024-04-20T11:26:56.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.035+0000 7fa3f7e00640 1 --2- 172.21.15.152:0/2408745144 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa400145620 0x7fa40006e8e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa400073700 tx=0x7fa3e802b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32375ce7d88c49df server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:56.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.036+0000 7fa3fea00640 1 -- 172.21.15.152:0/2408745144 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e8038070 con 0x7fa400145620 2024-04-20T11:26:56.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.036+0000 7fa3fea00640 1 -- 172.21.15.152:0/2408745144 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3e8031700 con 0x7fa400145620 2024-04-20T11:26:56.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.036+0000 7fa3fea00640 1 -- 172.21.15.152:0/2408745144 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e8031ad0 con 0x7fa400145620 2024-04-20T11:26:56.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.036+0000 7fa406200640 1 -- 172.21.15.152:0/2408745144 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa400145620 msgr2=0x7fa40006e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:56.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.036+0000 7fa406200640 1 --2- 172.21.15.152:0/2408745144 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa400145620 0x7fa40006e8e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa400073700 tx=0x7fa3e802b7a0 comp rx=0 tx=0).stop 2024-04-20T11:26:56.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.036+0000 7fa406200640 1 -- 172.21.15.152:0/2408745144 shutdown_connections 2024-04-20T11:26:56.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.036+0000 7fa406200640 1 --2- 172.21.15.152:0/2408745144 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa400145620 0x7fa40006e8e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:56.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.036+0000 7fa406200640 1 --2- 172.21.15.152:0/2408745144 >> v2:172.21.15.175:3300/0 conn(0x7fa400144fa0 0x7fa4000b3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:56.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.036+0000 7fa406200640 1 -- 172.21.15.152:0/2408745144 >> 172.21.15.152:0/2408745144 conn(0x7fa40006b980 msgr2=0x7fa40006bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:56.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.036+0000 7fa406200640 1 -- 172.21.15.152:0/2408745144 shutdown_connections 2024-04-20T11:26:56.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.036+0000 7fa406200640 1 -- 172.21.15.152:0/2408745144 wait complete. 2024-04-20T11:26:56.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.038+0000 7fa406200640 1 Processor -- start 2024-04-20T11:26:56.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.038+0000 7fa406200640 1 -- start start 2024-04-20T11:26:56.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.039+0000 7fa406200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa400071e20 0x7fa40015e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:56.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.039+0000 7fa406200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa400144fa0 0x7fa4001f5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:56.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.039+0000 7fa406200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa400145620 0x7fa400142ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:56.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.039+0000 7fa406200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fa4000aeb90 con 0x7fa400071e20 2024-04-20T11:26:56.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.039+0000 7fa406200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa4000b42f0 con 0x7fa400144fa0 2024-04-20T11:26:56.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.039+0000 7fa406200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa400072250 con 0x7fa400145620 2024-04-20T11:26:56.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.039+0000 7fa3ff400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa400071e20 0x7fa40015e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:56.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.039+0000 7fa3ff400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa400071e20 0x7fa40015e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:42456/0 (socket says 172.21.15.152:42456) 2024-04-20T11:26:56.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.039+0000 7fa3ff400640 1 -- 172.21.15.152:0/2575611353 learned_addr learned my addr 172.21.15.152:0/2575611353 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:26:56.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.040+0000 7fa3ff400640 1 -- 172.21.15.152:0/2575611353 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa400145620 msgr2=0x7fa400142ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:56.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.040+0000 7fa3ff400640 1 --2- 172.21.15.152:0/2575611353 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa400145620 0x7fa400142ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:56.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.040+0000 7fa3ff400640 1 -- 172.21.15.152:0/2575611353 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa400144fa0 msgr2=0x7fa4001f5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:56.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.040+0000 7fa3ff400640 1 --2- 172.21.15.152:0/2575611353 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa400144fa0 0x7fa4001f5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:56.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.040+0000 7fa3ff400640 1 -- 172.21.15.152:0/2575611353 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa400139c40 con 0x7fa400071e20 2024-04-20T11:26:56.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.040+0000 7fa3ff400640 1 --2- 172.21.15.152:0/2575611353 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa400071e20 0x7fa40015e270 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa3f000d800 tx=0x7fa3f000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:56.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.040+0000 7fa3f7400640 1 -- 172.21.15.152:0/2575611353 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3f000b6c0 con 0x7fa400071e20 2024-04-20T11:26:56.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.040+0000 7fa3f7400640 1 -- 172.21.15.152:0/2575611353 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3f0019070 con 0x7fa400071e20 2024-04-20T11:26:56.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.040+0000 7fa3f7400640 1 -- 172.21.15.152:0/2575611353 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3f000ba80 con 0x7fa400071e20 2024-04-20T11:26:56.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.040+0000 7fa406200640 1 -- 172.21.15.152:0/2575611353 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa400139eb0 con 0x7fa400071e20 2024-04-20T11:26:56.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.040+0000 7fa406200640 1 -- 172.21.15.152:0/2575611353 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa40013a470 con 0x7fa400071e20 2024-04-20T11:26:56.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.041+0000 7fa406200640 1 -- 172.21.15.152:0/2575611353 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa400068a60 con 0x7fa400071e20 2024-04-20T11:26:56.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.042+0000 7fa3f7400640 1 -- 172.21.15.152:0/2575611353 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa3f000bc20 con 0x7fa400071e20 2024-04-20T11:26:56.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.044+0000 7fa3f7400640 1 --2- 172.21.15.152:0/2575611353 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fa3dc06c8f0 0x7fa3dc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:56.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.044+0000 7fa3f7400640 1 -- 172.21.15.152:0/2575611353 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fa3f008bd90 con 0x7fa400071e20 2024-04-20T11:26:56.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.044+0000 7fa3f7e00640 1 --2- 172.21.15.152:0/2575611353 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fa3dc06c8f0 0x7fa3dc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:56.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.044+0000 7fa3f7e00640 1 --2- 172.21.15.152:0/2575611353 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fa3dc06c8f0 0x7fa3dc06edb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa3e8035000 tx=0x7fa3e8030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:56.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.048+0000 7fa3f7400640 1 -- 172.21.15.152:0/2575611353 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fa3f001c360 con 0x7fa400071e20 2024-04-20T11:26:56.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.245+0000 7fa406200640 1 -- 172.21.15.152:0/2575611353 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fa4000cfd00 con 0x7fa400071e20 2024-04-20T11:26:56.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.637+0000 7fa3f7400640 1 -- 172.21.15.152:0/2575611353 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) ==== 138+0+0 (secure 0 0 0) 0x7fa3f005a470 con 0x7fa400071e20 2024-04-20T11:26:56.637 INFO:teuthology.orchestra.run.smithi152.stderr:pool 'cephfs_metadata' created 2024-04-20T11:26:56.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.640+0000 7fa406200640 1 -- 172.21.15.152:0/2575611353 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fa3dc06c8f0 msgr2=0x7fa3dc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:56.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.640+0000 7fa406200640 1 --2- 172.21.15.152:0/2575611353 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fa3dc06c8f0 0x7fa3dc06edb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa3e8035000 tx=0x7fa3e8030000 comp rx=0 tx=0).stop 2024-04-20T11:26:56.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.640+0000 7fa406200640 1 -- 172.21.15.152:0/2575611353 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa400071e20 msgr2=0x7fa40015e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:56.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.640+0000 7fa406200640 1 --2- 172.21.15.152:0/2575611353 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa400071e20 0x7fa40015e270 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa3f000d800 tx=0x7fa3f000dcc0 comp rx=0 tx=0).stop 2024-04-20T11:26:56.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.640+0000 7fa406200640 1 -- 172.21.15.152:0/2575611353 shutdown_connections 2024-04-20T11:26:56.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.641+0000 7fa406200640 1 --2- 172.21.15.152:0/2575611353 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fa3dc06c8f0 0x7fa3dc06edb0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:56.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.641+0000 7fa406200640 1 --2- 172.21.15.152:0/2575611353 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa400145620 0x7fa400142ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:56.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.641+0000 7fa406200640 1 --2- 172.21.15.152:0/2575611353 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa400144fa0 0x7fa4001f5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:56.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.641+0000 7fa406200640 1 --2- 172.21.15.152:0/2575611353 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa400071e20 0x7fa40015e270 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:56.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.641+0000 7fa406200640 1 -- 172.21.15.152:0/2575611353 >> 172.21.15.152:0/2575611353 conn(0x7fa40006b980 msgr2=0x7fa40006e5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:56.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.641+0000 7fa406200640 1 -- 172.21.15.152:0/2575611353 shutdown_connections 2024-04-20T11:26:56.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.641+0000 7fa406200640 1 -- 172.21.15.152:0/2575611353 wait complete. 2024-04-20T11:26:56.667 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-20T11:26:56.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.859+0000 7f5eb7a00640 1 Processor -- start 2024-04-20T11:26:56.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.860+0000 7f5eb7a00640 1 -- start start 2024-04-20T11:26:56.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.861+0000 7f5eb7a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f5eb015e9a0 0x7f5eb00d29a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:56.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.861+0000 7f5eb7a00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f5eb00d4340 0x7f5eb00d1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:56.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.861+0000 7f5eb7a00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5eb00b8960 con 0x7f5eb015ed80 2024-04-20T11:26:56.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.861+0000 7f5eb7a00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f5eb00d32a0 con 0x7f5eb015e9a0 2024-04-20T11:26:56.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.861+0000 7f5eb7a00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f5eb00d3580 con 0x7f5eb00d4340 2024-04-20T11:26:56.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.862+0000 7f5eb5000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f5eb015e9a0 0x7f5eb00d29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:56.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.862+0000 7f5eb5000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f5eb015e9a0 0x7f5eb00d29a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:42472/0 (socket says 172.21.15.152:42472) 2024-04-20T11:26:56.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.862+0000 7f5eb5000640 1 -- 172.21.15.152:0/3525774811 learned_addr learned my addr 172.21.15.152:0/3525774811 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:26:56.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.862+0000 7f5eafe00640 1 --2- 172.21.15.152:0/3525774811 >> v2:172.21.15.175:3300/0 conn(0x7f5eb00d4340 0x7f5eb00d1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:56.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.862+0000 7f5eb5000640 1 -- 172.21.15.152:0/3525774811 >> v2:172.21.15.175:3300/0 conn(0x7f5eb00d4340 msgr2=0x7f5eb00d1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:56.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.862+0000 7f5eb5000640 1 --2- 172.21.15.152:0/3525774811 >> v2:172.21.15.175:3300/0 conn(0x7f5eb00d4340 0x7f5eb00d1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:56.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.862+0000 7f5eb5000640 1 -- 172.21.15.152:0/3525774811 >> v1:172.21.15.175:6789/0 conn(0x7f5eb015ed80 legacy=0x7f5eb00b62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:56.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.862+0000 7f5eb5000640 1 -- 172.21.15.152:0/3525774811 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5eb00d3860 con 0x7f5eb015e9a0 2024-04-20T11:26:56.863 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.863+0000 7f5eb5000640 1 --2- 172.21.15.152:0/3525774811 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5eb015e9a0 0x7f5eb00d29a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5ea0009870 tx=0x7f5ea002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39d0be5a87c01443 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:56.863 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.864+0000 7f5eaf400640 1 -- 172.21.15.152:0/3525774811 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea0034070 con 0x7f5eb015e9a0 2024-04-20T11:26:56.863 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.864+0000 7f5eaf400640 1 -- 172.21.15.152:0/3525774811 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ea0030860 con 0x7f5eb015e9a0 2024-04-20T11:26:56.863 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.864+0000 7f5eaf400640 1 -- 172.21.15.152:0/3525774811 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea0030c30 con 0x7f5eb015e9a0 2024-04-20T11:26:56.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.864+0000 7f5eb7a00640 1 -- 172.21.15.152:0/3525774811 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5eb015e9a0 msgr2=0x7f5eb00d29a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:56.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.864+0000 7f5eb7a00640 1 --2- 172.21.15.152:0/3525774811 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5eb015e9a0 0x7f5eb00d29a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5ea0009870 tx=0x7f5ea002b9a0 comp rx=0 tx=0).stop 2024-04-20T11:26:56.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.864+0000 7f5eb7a00640 1 -- 172.21.15.152:0/3525774811 shutdown_connections 2024-04-20T11:26:56.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.864+0000 7f5eb7a00640 1 --2- 172.21.15.152:0/3525774811 >> v2:172.21.15.175:3300/0 conn(0x7f5eb00d4340 0x7f5eb00d1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:56.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.864+0000 7f5eb7a00640 1 --2- 172.21.15.152:0/3525774811 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5eb015e9a0 0x7f5eb00d29a0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:56.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.864+0000 7f5eb7a00640 1 -- 172.21.15.152:0/3525774811 >> 172.21.15.152:0/3525774811 conn(0x7f5eb006cc00 msgr2=0x7f5eb006d010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:56.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.865+0000 7f5eb7a00640 1 -- 172.21.15.152:0/3525774811 shutdown_connections 2024-04-20T11:26:56.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.865+0000 7f5eb7a00640 1 -- 172.21.15.152:0/3525774811 wait complete. 2024-04-20T11:26:56.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.866+0000 7f5eb7a00640 1 Processor -- start 2024-04-20T11:26:56.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.866+0000 7f5eb7a00640 1 -- start start 2024-04-20T11:26:56.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.867+0000 7f5eb7a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5eb015e9a0 0x7f5eb0150d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:56.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.867+0000 7f5eb7a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5eb015ed80 0x7f5eb01585c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:56.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.867+0000 7f5eb7a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5eb00c95a0 0x7f5eb0158b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:56.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.867+0000 7f5eb7a00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f5eb00d3060 con 0x7f5eb015ed80 2024-04-20T11:26:56.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.867+0000 7f5eb7a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5eb00d2ee0 con 0x7f5eb015e9a0 2024-04-20T11:26:56.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.867+0000 7f5eb7a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5eb00ba1b0 con 0x7f5eb00c95a0 2024-04-20T11:26:56.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.868+0000 7f5eafe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5eb015ed80 0x7f5eb01585c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:56.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.868+0000 7f5eafe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5eb015ed80 0x7f5eb01585c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:42480/0 (socket says 172.21.15.152:42480) 2024-04-20T11:26:56.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.868+0000 7f5eafe00640 1 -- 172.21.15.152:0/1687860764 learned_addr learned my addr 172.21.15.152:0/1687860764 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:26:56.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.868+0000 7f5eb5000640 1 --2- 172.21.15.152:0/1687860764 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5eb015e9a0 0x7f5eb0150d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:56.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.868+0000 7f5eb5a00640 1 --2- 172.21.15.152:0/1687860764 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5eb00c95a0 0x7f5eb0158b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:56.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.868+0000 7f5eafe00640 1 -- 172.21.15.152:0/1687860764 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5eb00c95a0 msgr2=0x7f5eb0158b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:56.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.868+0000 7f5eafe00640 1 --2- 172.21.15.152:0/1687860764 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5eb00c95a0 0x7f5eb0158b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:56.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.868+0000 7f5eafe00640 1 -- 172.21.15.152:0/1687860764 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5eb015e9a0 msgr2=0x7f5eb0150d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:56.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.868+0000 7f5eafe00640 1 --2- 172.21.15.152:0/1687860764 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5eb015e9a0 0x7f5eb0150d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:56.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.868+0000 7f5eafe00640 1 -- 172.21.15.152:0/1687860764 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5eb01f4860 con 0x7f5eb015ed80 2024-04-20T11:26:56.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.868+0000 7f5eafe00640 1 --2- 172.21.15.152:0/1687860764 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5eb015ed80 0x7f5eb01585c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5e9800efc0 tx=0x7f5e9800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:56.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.869+0000 7f5eacc00640 1 -- 172.21.15.152:0/1687860764 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e98018020 con 0x7f5eb015ed80 2024-04-20T11:26:56.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.869+0000 7f5eacc00640 1 -- 172.21.15.152:0/1687860764 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e98014480 con 0x7f5eb015ed80 2024-04-20T11:26:56.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.869+0000 7f5eacc00640 1 -- 172.21.15.152:0/1687860764 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e98014870 con 0x7f5eb015ed80 2024-04-20T11:26:56.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.869+0000 7f5eb7a00640 1 -- 172.21.15.152:0/1687860764 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5eb01f4ae0 con 0x7f5eb015ed80 2024-04-20T11:26:56.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.869+0000 7f5eb7a00640 1 -- 172.21.15.152:0/1687860764 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5eb01f50a0 con 0x7f5eb015ed80 2024-04-20T11:26:56.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.870+0000 7f5eb7a00640 1 -- 172.21.15.152:0/1687860764 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5eb00689f0 con 0x7f5eb015ed80 2024-04-20T11:26:56.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.871+0000 7f5eacc00640 1 -- 172.21.15.152:0/1687860764 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5e98005d10 con 0x7f5eb015ed80 2024-04-20T11:26:56.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.872+0000 7f5eacc00640 1 --2- 172.21.15.152:0/1687860764 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f5e8c06c920 0x7f5e8c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:56.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.872+0000 7f5eacc00640 1 -- 172.21.15.152:0/1687860764 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 5227+0+0 (secure 0 0 0) 0x7f5e9808c8e0 con 0x7f5eb015ed80 2024-04-20T11:26:56.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.872+0000 7f5eb5000640 1 --2- 172.21.15.152:0/1687860764 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f5e8c06c920 0x7f5e8c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:56.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.873+0000 7f5eb5000640 1 --2- 172.21.15.152:0/1687860764 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f5e8c06c920 0x7f5e8c06ede0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5ea0036000 tx=0x7f5ea003b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:56.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:56.877+0000 7f5eacc00640 1 -- 172.21.15.152:0/1687860764 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f5e98022030 con 0x7f5eb015ed80 2024-04-20T11:26:57.063 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:57.063+0000 7f5eb7a00640 1 -- 172.21.15.152:0/1687860764 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f5eb00cb210 con 0x7f5eb015ed80 2024-04-20T11:26:57.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:57.650+0000 7f5eacc00640 1 -- 172.21.15.152:0/1687860764 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v18) ==== 170+0+0 (secure 0 0 0) 0x7f5eb01f4860 con 0x7f5eb015ed80 2024-04-20T11:26:57.650 INFO:teuthology.orchestra.run.smithi152.stderr:pool 'cephfs_data' created 2024-04-20T11:26:57.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:57.654+0000 7f5eb7a00640 1 -- 172.21.15.152:0/1687860764 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f5e8c06c920 msgr2=0x7f5e8c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:57.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:57.654+0000 7f5eb7a00640 1 --2- 172.21.15.152:0/1687860764 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f5e8c06c920 0x7f5e8c06ede0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5ea0036000 tx=0x7f5ea003b040 comp rx=0 tx=0).stop 2024-04-20T11:26:57.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:57.654+0000 7f5eb7a00640 1 -- 172.21.15.152:0/1687860764 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5eb015ed80 msgr2=0x7f5eb01585c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:57.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:57.654+0000 7f5eb7a00640 1 --2- 172.21.15.152:0/1687860764 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5eb015ed80 0x7f5eb01585c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5e9800efc0 tx=0x7f5e9800c490 comp rx=0 tx=0).stop 2024-04-20T11:26:57.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:57.655+0000 7f5eb7a00640 1 -- 172.21.15.152:0/1687860764 shutdown_connections 2024-04-20T11:26:57.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:57.655+0000 7f5eb7a00640 1 --2- 172.21.15.152:0/1687860764 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f5e8c06c920 0x7f5e8c06ede0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:57.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:57.655+0000 7f5eb7a00640 1 --2- 172.21.15.152:0/1687860764 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5eb00c95a0 0x7f5eb0158b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:57.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:57.655+0000 7f5eb7a00640 1 --2- 172.21.15.152:0/1687860764 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5eb015ed80 0x7f5eb01585c0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:57.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:57.655+0000 7f5eb7a00640 1 --2- 172.21.15.152:0/1687860764 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5eb015e9a0 0x7f5eb0150d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:57.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:57.655+0000 7f5eb7a00640 1 -- 172.21.15.152:0/1687860764 >> 172.21.15.152:0/1687860764 conn(0x7f5eb006cc00 msgr2=0x7f5eb00b7950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:57.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:57.655+0000 7f5eb7a00640 1 -- 172.21.15.152:0/1687860764 shutdown_connections 2024-04-20T11:26:57.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:57.656+0000 7f5eb7a00640 1 -- 172.21.15.152:0/1687860764 wait complete. 2024-04-20T11:26:57.697 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-20T11:26:57.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:57.869+0000 7fc597c00640 1 Processor -- start 2024-04-20T11:26:57.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:57.870+0000 7fc597c00640 1 -- start start 2024-04-20T11:26:57.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:57.871+0000 7fc597c00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fc5900cfc40 0x7fc59015fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:57.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:57.871+0000 7fc597c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc5900b2880 0x7fc5900b3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:57.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:57.871+0000 7fc597c00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc59015f1c0 con 0x7fc59015b510 2024-04-20T11:26:57.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:57.871+0000 7fc597c00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fc59006f420 con 0x7fc5900cfc40 2024-04-20T11:26:57.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:57.871+0000 7fc597c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc5900b4430 con 0x7fc5900b2880 2024-04-20T11:26:57.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:57.872+0000 7fc58fe00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fc5900cfc40 0x7fc59015fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:57.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:57.872+0000 7fc58fe00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fc5900cfc40 0x7fc59015fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.152:49852/0 (socket says 172.21.15.152:49852) 2024-04-20T11:26:57.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:57.872+0000 7fc595200640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7fc59015b510 0x7fc5900b2170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.152:49204/0 (socket says 172.21.15.152:49204) 2024-04-20T11:26:57.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:57.872+0000 7fc58fe00640 1 -- 172.21.15.152:0/3523495037 learned_addr learned my addr 172.21.15.152:0/3523495037 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:26:57.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:57.872+0000 7fc595c00640 1 --2- 172.21.15.152:0/3523495037 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc5900b2880 0x7fc5900b3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:57.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:57.873+0000 7fc58fe00640 1 -- 172.21.15.152:0/3523495037 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc5900b2880 msgr2=0x7fc5900b3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:57.873 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:57.873+0000 7fc58fe00640 1 --2- 172.21.15.152:0/3523495037 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc5900b2880 0x7fc5900b3ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:57.873 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:57.873+0000 7fc58fe00640 1 -- 172.21.15.152:0/3523495037 >> v1:172.21.15.175:6789/0 conn(0x7fc59015b510 legacy=0x7fc5900b2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:57.873 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:57.873+0000 7fc58fe00640 1 -- 172.21.15.152:0/3523495037 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc59006f670 con 0x7fc5900cfc40 2024-04-20T11:26:57.873 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:57.873+0000 7fc595c00640 1 --2- 172.21.15.152:0/3523495037 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc5900b2880 0x7fc5900b3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:26:57.873 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:57.874+0000 7fc58fe00640 1 --2- 172.21.15.152:0/3523495037 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5900cfc40 0x7fc59015fc30 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc57c00bb10 tx=0x7fc57c02e150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1477ac62a7ce929e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:57.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:57.875+0000 7fc58f400640 1 -- 172.21.15.152:0/3523495037 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc57c03b070 con 0x7fc5900cfc40 2024-04-20T11:26:57.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:57.875+0000 7fc58f400640 1 -- 172.21.15.152:0/3523495037 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc57c036440 con 0x7fc5900cfc40 2024-04-20T11:26:57.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:57.875+0000 7fc58f400640 1 -- 172.21.15.152:0/3523495037 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc57c036810 con 0x7fc5900cfc40 2024-04-20T11:26:57.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:57.876+0000 7fc597c00640 1 -- 172.21.15.152:0/3523495037 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5900cfc40 msgr2=0x7fc59015fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:57.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:57.876+0000 7fc597c00640 1 --2- 172.21.15.152:0/3523495037 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5900cfc40 0x7fc59015fc30 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc57c00bb10 tx=0x7fc57c02e150 comp rx=0 tx=0).stop 2024-04-20T11:26:57.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:57.876+0000 7fc597c00640 1 -- 172.21.15.152:0/3523495037 shutdown_connections 2024-04-20T11:26:57.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:57.876+0000 7fc597c00640 1 --2- 172.21.15.152:0/3523495037 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc5900b2880 0x7fc5900b3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:57.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:57.876+0000 7fc597c00640 1 --2- 172.21.15.152:0/3523495037 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5900cfc40 0x7fc59015fc30 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:57.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:57.876+0000 7fc597c00640 1 -- 172.21.15.152:0/3523495037 >> 172.21.15.152:0/3523495037 conn(0x7fc59006b980 msgr2=0x7fc59006bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:57.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:57.876+0000 7fc597c00640 1 -- 172.21.15.152:0/3523495037 shutdown_connections 2024-04-20T11:26:57.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:57.876+0000 7fc597c00640 1 -- 172.21.15.152:0/3523495037 wait complete. 2024-04-20T11:26:57.877 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:57.878+0000 7fc597c00640 1 Processor -- start 2024-04-20T11:26:57.877 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:57.878+0000 7fc597c00640 1 -- start start 2024-04-20T11:26:57.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:57.879+0000 7fc597c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc5900b2880 0x7fc590154050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:57.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:57.879+0000 7fc597c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc5900cfc40 0x7fc5901e4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:57.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:57.879+0000 7fc597c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc59015b510 0x7fc5901e5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:57.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:57.879+0000 7fc597c00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fc59015b8f0 con 0x7fc5900b2880 2024-04-20T11:26:57.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:57.879+0000 7fc597c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc59015f710 con 0x7fc59015b510 2024-04-20T11:26:57.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:57.879+0000 7fc597c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc59006f950 con 0x7fc5900cfc40 2024-04-20T11:26:57.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:57.879+0000 7fc595200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc5900b2880 0x7fc590154050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:57.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:57.879+0000 7fc595200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc5900b2880 0x7fc590154050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:42484/0 (socket says 172.21.15.152:42484) 2024-04-20T11:26:57.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:57.879+0000 7fc595200640 1 -- 172.21.15.152:0/2949197242 learned_addr learned my addr 172.21.15.152:0/2949197242 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:26:57.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:57.880+0000 7fc595c00640 1 --2- 172.21.15.152:0/2949197242 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc59015b510 0x7fc5901e5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:57.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:57.880+0000 7fc58fe00640 1 --2- 172.21.15.152:0/2949197242 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc5900cfc40 0x7fc5901e4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:57.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:57.880+0000 7fc595200640 1 -- 172.21.15.152:0/2949197242 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc5900cfc40 msgr2=0x7fc5901e4f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:57.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:57.880+0000 7fc595200640 1 --2- 172.21.15.152:0/2949197242 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc5900cfc40 0x7fc5901e4f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:57.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:57.880+0000 7fc595200640 1 -- 172.21.15.152:0/2949197242 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc59015b510 msgr2=0x7fc5901e5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:57.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:57.880+0000 7fc595200640 1 --2- 172.21.15.152:0/2949197242 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc59015b510 0x7fc5901e5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:57.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:57.880+0000 7fc595200640 1 -- 172.21.15.152:0/2949197242 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc59015d030 con 0x7fc5900b2880 2024-04-20T11:26:57.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:57.880+0000 7fc58fe00640 1 --2- 172.21.15.152:0/2949197242 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc5900cfc40 0x7fc5901e4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:26:57.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:57.880+0000 7fc595c00640 1 --2- 172.21.15.152:0/2949197242 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc59015b510 0x7fc5901e5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:26:57.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:57.880+0000 7fc595200640 1 --2- 172.21.15.152:0/2949197242 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc5900b2880 0x7fc590154050 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc58000c3c0 tx=0x7fc58000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:57.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:57.880+0000 7fc58cc00640 1 -- 172.21.15.152:0/2949197242 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5800074e0 con 0x7fc5900b2880 2024-04-20T11:26:57.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:57.881+0000 7fc58cc00640 1 -- 172.21.15.152:0/2949197242 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc580007680 con 0x7fc5900b2880 2024-04-20T11:26:57.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:57.881+0000 7fc58cc00640 1 -- 172.21.15.152:0/2949197242 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc580004a00 con 0x7fc5900b2880 2024-04-20T11:26:57.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:57.881+0000 7fc597c00640 1 -- 172.21.15.152:0/2949197242 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc59015d310 con 0x7fc5900b2880 2024-04-20T11:26:57.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:57.881+0000 7fc597c00640 1 -- 172.21.15.152:0/2949197242 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc59015d8d0 con 0x7fc5900b2880 2024-04-20T11:26:57.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:57.882+0000 7fc597c00640 1 -- 172.21.15.152:0/2949197242 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5900689f0 con 0x7fc5900b2880 2024-04-20T11:26:57.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:57.883+0000 7fc58cc00640 1 -- 172.21.15.152:0/2949197242 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc58001a020 con 0x7fc5900b2880 2024-04-20T11:26:57.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:57.884+0000 7fc58cc00640 1 --2- 172.21.15.152:0/2949197242 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc57006c8f0 0x7fc57006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:57.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:57.884+0000 7fc58cc00640 1 -- 172.21.15.152:0/2949197242 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5609+0+0 (secure 0 0 0) 0x7fc58008b2a0 con 0x7fc5900b2880 2024-04-20T11:26:57.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:57.884+0000 7fc58fe00640 1 --2- 172.21.15.152:0/2949197242 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc57006c8f0 0x7fc57006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:57.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:57.885+0000 7fc58fe00640 1 --2- 172.21.15.152:0/2949197242 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc57006c8f0 0x7fc57006edb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc57c038000 tx=0x7fc57c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:57.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:57.889+0000 7fc58cc00640 1 -- 172.21.15.152:0/2949197242 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fc580059720 con 0x7fc5900b2880 2024-04-20T11:26:58.130 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:58.131+0000 7fc597c00640 1 -- 172.21.15.152:0/2949197242 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fc59006ecf0 con 0x7fc5900b2880 2024-04-20T11:26:58.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:58.655+0000 7fc58cc00640 1 -- 172.21.15.152:0/2949197242 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7fc580059020 con 0x7fc5900b2880 2024-04-20T11:26:58.655 INFO:teuthology.orchestra.run.smithi152.stderr:new fs with metadata pool 2 and data pool 3 2024-04-20T11:26:58.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:58.659+0000 7fc597c00640 1 -- 172.21.15.152:0/2949197242 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc57006c8f0 msgr2=0x7fc57006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:58.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:58.659+0000 7fc597c00640 1 --2- 172.21.15.152:0/2949197242 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc57006c8f0 0x7fc57006edb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc57c038000 tx=0x7fc57c008040 comp rx=0 tx=0).stop 2024-04-20T11:26:58.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:58.659+0000 7fc597c00640 1 -- 172.21.15.152:0/2949197242 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc5900b2880 msgr2=0x7fc590154050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:58.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:58.659+0000 7fc597c00640 1 --2- 172.21.15.152:0/2949197242 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc5900b2880 0x7fc590154050 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc58000c3c0 tx=0x7fc58000c880 comp rx=0 tx=0).stop 2024-04-20T11:26:58.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:58.659+0000 7fc597c00640 1 -- 172.21.15.152:0/2949197242 shutdown_connections 2024-04-20T11:26:58.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:58.659+0000 7fc597c00640 1 --2- 172.21.15.152:0/2949197242 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc57006c8f0 0x7fc57006edb0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:58.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:58.659+0000 7fc597c00640 1 --2- 172.21.15.152:0/2949197242 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc59015b510 0x7fc5901e5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:58.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:58.659+0000 7fc597c00640 1 --2- 172.21.15.152:0/2949197242 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc5900cfc40 0x7fc5901e4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:58.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:58.659+0000 7fc597c00640 1 --2- 172.21.15.152:0/2949197242 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc5900b2880 0x7fc590154050 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:58.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:58.659+0000 7fc597c00640 1 -- 172.21.15.152:0/2949197242 >> 172.21.15.152:0/2949197242 conn(0x7fc59006b980 msgr2=0x7fc59006dc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:58.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:58.661+0000 7fc597c00640 1 -- 172.21.15.152:0/2949197242 shutdown_connections 2024-04-20T11:26:58.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:58.662+0000 7fc597c00640 1 -- 172.21.15.152:0/2949197242 wait complete. 2024-04-20T11:26:58.712 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:26:58.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:58.875+0000 7f6426400640 1 Processor -- start 2024-04-20T11:26:58.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:58.875+0000 7f6426400640 1 -- start start 2024-04-20T11:26:58.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:58.877+0000 7f6426400640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f64200cd0d0 0x7f64200cd580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:58.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:58.877+0000 7f6426400640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6420146340 con 0x7f6420071fe0 2024-04-20T11:26:58.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:58.877+0000 7f6426400640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6420072d50 con 0x7f6420145f60 2024-04-20T11:26:58.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:58.877+0000 7f6426400640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f642006e9f0 con 0x7f64200cd0d0 2024-04-20T11:26:58.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:58.877+0000 7f641fe00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f6420071fe0 0x7f6420072470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:40580/0 (socket says 172.21.15.152:40580) 2024-04-20T11:26:58.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:58.877+0000 7f641fe00640 1 -- 172.21.15.152:0/1891502853 learned_addr learned my addr 172.21.15.152:0/1891502853 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:26:58.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:58.877+0000 7f6416a00640 1 --2- 172.21.15.152:0/1891502853 >> v2:172.21.15.175:3300/0 conn(0x7f64200cd0d0 0x7f64200cd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:58.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:58.878+0000 7f641ea00640 1 -- 172.21.15.152:0/1891502853 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 693651837 0 0) 0x7f6420146340 con 0x7f6420071fe0 2024-04-20T11:26:58.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:58.878+0000 7f641ea00640 1 -- 172.21.15.152:0/1891502853 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6404003520 con 0x7f6420071fe0 2024-04-20T11:26:58.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:58.878+0000 7f641ea00640 1 -- 172.21.15.152:0/1891502853 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2066102709 0 0) 0x7f6404003520 con 0x7f6420071fe0 2024-04-20T11:26:58.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:58.878+0000 7f641ea00640 1 -- 172.21.15.152:0/1891502853 >> v2:172.21.15.175:3300/0 conn(0x7f64200cd0d0 msgr2=0x7f64200cd580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:58.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:58.878+0000 7f641ea00640 1 --2- 172.21.15.152:0/1891502853 >> v2:172.21.15.175:3300/0 conn(0x7f64200cd0d0 0x7f64200cd580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:58.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:58.878+0000 7f641ea00640 1 -- 172.21.15.152:0/1891502853 >> v1:172.21.15.175:6789/0 conn(0x7f6420145f60 legacy=0x7f642006ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:58.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:58.878+0000 7f641ea00640 1 -- 172.21.15.152:0/1891502853 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f642006e0c0 con 0x7f6420071fe0 2024-04-20T11:26:58.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:58.878+0000 7f641ea00640 1 -- 172.21.15.152:0/1891502853 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f64180019f0 con 0x7f6420071fe0 2024-04-20T11:26:58.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:58.878+0000 7f641ea00640 1 -- 172.21.15.152:0/1891502853 >> v1:172.21.15.152:6789/0 conn(0x7f6420071fe0 legacy=0x7f6420072470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:58.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:58.878+0000 7f641ea00640 1 --2- 172.21.15.152:0/1891502853 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6404003a20 0x7f6404003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:58.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:58.878+0000 7f641ea00640 1 --2- 172.21.15.152:0/1891502853 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6404004800 0x7f6404004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:58.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:58.878+0000 7f641ea00640 1 --2- 172.21.15.152:0/1891502853 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f64040051c0 0x7f640400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:58.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:58.878+0000 7f641ea00640 1 -- 172.21.15.152:0/1891502853 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f642006e9f0 con 0x7f64040051c0 2024-04-20T11:26:58.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:58.878+0000 7f641ea00640 1 -- 172.21.15.152:0/1891502853 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f640400c0f0 con 0x7f6404004800 2024-04-20T11:26:58.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:58.878+0000 7f641ea00640 1 -- 172.21.15.152:0/1891502853 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f640400c2f0 con 0x7f6404003a20 2024-04-20T11:26:58.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:58.878+0000 7f641fe00640 1 --2- 172.21.15.152:0/1891502853 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f64040051c0 0x7f640400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:58.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:58.879+0000 7f641f400640 1 --2- 172.21.15.152:0/1891502853 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6404003a20 0x7f6404003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:58.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:58.879+0000 7f6416a00640 1 --2- 172.21.15.152:0/1891502853 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6404004800 0x7f6404004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:58.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:58.879+0000 7f641fe00640 1 -- 172.21.15.152:0/1891502853 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6404003a20 msgr2=0x7f6404003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:58.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:58.879+0000 7f641fe00640 1 --2- 172.21.15.152:0/1891502853 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6404003a20 0x7f6404003e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:58.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:58.879+0000 7f641fe00640 1 -- 172.21.15.152:0/1891502853 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6404004800 msgr2=0x7f6404004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:58.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:58.879+0000 7f641fe00640 1 --2- 172.21.15.152:0/1891502853 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6404004800 0x7f6404004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:58.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:58.879+0000 7f641fe00640 1 -- 172.21.15.152:0/1891502853 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6404003520 con 0x7f64040051c0 2024-04-20T11:26:58.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:58.879+0000 7f6416a00640 1 --2- 172.21.15.152:0/1891502853 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6404004800 0x7f6404004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:26:58.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:58.880+0000 7f641fe00640 1 -- 172.21.15.152:0/1891502853 reap_dead start 2024-04-20T11:26:58.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:58.880+0000 7f641fe00640 1 --2- 172.21.15.152:0/1891502853 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f64040051c0 0x7f640400b9c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f641800a840 tx=0x7f6404005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b550695f8f2d1b47 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:58.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:58.880+0000 7f641ea00640 1 -- 172.21.15.152:0/1891502853 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f641800a460 con 0x7f64040051c0 2024-04-20T11:26:58.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:58.880+0000 7f641ea00640 1 -- 172.21.15.152:0/1891502853 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6418034650 con 0x7f64040051c0 2024-04-20T11:26:58.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:58.880+0000 7f641ea00640 1 -- 172.21.15.152:0/1891502853 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6418034a40 con 0x7f64040051c0 2024-04-20T11:26:58.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:58.880+0000 7f6426400640 1 -- 172.21.15.152:0/1891502853 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f64040051c0 msgr2=0x7f640400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:58.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:58.880+0000 7f6426400640 1 --2- 172.21.15.152:0/1891502853 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f64040051c0 0x7f640400b9c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f641800a840 tx=0x7f6404005830 comp rx=0 tx=0).stop 2024-04-20T11:26:58.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:58.881+0000 7f6426400640 1 -- 172.21.15.152:0/1891502853 shutdown_connections 2024-04-20T11:26:58.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:58.881+0000 7f6426400640 1 --2- 172.21.15.152:0/1891502853 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f64040051c0 0x7f640400b9c0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:58.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:58.881+0000 7f6426400640 1 -- 172.21.15.152:0/1891502853 >> 172.21.15.152:0/1891502853 conn(0x7f6420067cf0 msgr2=0x7f642006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:58.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:58.881+0000 7f6426400640 1 -- 172.21.15.152:0/1891502853 shutdown_connections 2024-04-20T11:26:58.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:58.881+0000 7f6426400640 1 -- 172.21.15.152:0/1891502853 wait complete. 2024-04-20T11:26:58.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:58.882+0000 7f6426400640 1 Processor -- start 2024-04-20T11:26:58.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:58.882+0000 7f6426400640 1 -- start start 2024-04-20T11:26:58.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:58.883+0000 7f6426400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f64040051c0 0x7f64200a94a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:58.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:58.883+0000 7f6426400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6420160350 0x7f642015f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:58.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:58.883+0000 7f6426400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6420134a50 0x7f64201f24d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:58.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:58.883+0000 7f6426400640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f64200cd580 con 0x7f64040051c0 2024-04-20T11:26:58.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:58.883+0000 7f6426400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f642006e680 con 0x7f6420134a50 2024-04-20T11:26:58.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:58.883+0000 7f6426400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f64200cd700 con 0x7f6420160350 2024-04-20T11:26:58.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:58.884+0000 7f641f400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f64040051c0 0x7f64200a94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:58.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:58.884+0000 7f641f400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f64040051c0 0x7f64200a94a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:42512/0 (socket says 172.21.15.152:42512) 2024-04-20T11:26:58.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:58.884+0000 7f641f400640 1 -- 172.21.15.152:0/729735867 learned_addr learned my addr 172.21.15.152:0/729735867 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:26:58.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:58.884+0000 7f6416a00640 1 --2- 172.21.15.152:0/729735867 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6420160350 0x7f642015f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:58.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:58.884+0000 7f641fe00640 1 --2- 172.21.15.152:0/729735867 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6420134a50 0x7f64201f24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:58.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:58.884+0000 7f641f400640 1 -- 172.21.15.152:0/729735867 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6420160350 msgr2=0x7f642015f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:58.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:58.884+0000 7f641f400640 1 --2- 172.21.15.152:0/729735867 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6420160350 0x7f642015f510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:58.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:58.884+0000 7f641f400640 1 -- 172.21.15.152:0/729735867 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6420134a50 msgr2=0x7f64201f24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:58.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:58.884+0000 7f641f400640 1 --2- 172.21.15.152:0/729735867 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6420134a50 0x7f64201f24d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:58.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:58.884+0000 7f641f400640 1 -- 172.21.15.152:0/729735867 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f642015e750 con 0x7f64040051c0 2024-04-20T11:26:58.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:58.884+0000 7f641f400640 1 --2- 172.21.15.152:0/729735867 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f64040051c0 0x7f64200a94a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f641000d4c0 tx=0x7f641000d980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:58.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:58.885+0000 7f6417e00640 1 -- 172.21.15.152:0/729735867 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6410005480 con 0x7f64040051c0 2024-04-20T11:26:58.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:58.885+0000 7f6417e00640 1 -- 172.21.15.152:0/729735867 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6410017070 con 0x7f64040051c0 2024-04-20T11:26:58.888 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:58.885+0000 7f6417e00640 1 -- 172.21.15.152:0/729735867 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6410003830 con 0x7f64040051c0 2024-04-20T11:26:58.888 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:58.885+0000 7f6426400640 1 -- 172.21.15.152:0/729735867 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f642015ea30 con 0x7f64040051c0 2024-04-20T11:26:58.888 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:58.885+0000 7f6426400640 1 -- 172.21.15.152:0/729735867 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f642015eff0 con 0x7f64040051c0 2024-04-20T11:26:58.888 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:58.886+0000 7f6426400640 1 -- 172.21.15.152:0/729735867 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64200689f0 con 0x7f64040051c0 2024-04-20T11:26:58.888 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:58.887+0000 7f6417e00640 1 -- 172.21.15.152:0/729735867 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6410003a90 con 0x7f64040051c0 2024-04-20T11:26:58.888 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:58.888+0000 7f6417e00640 1 --2- 172.21.15.152:0/729735867 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f63fc06c8f0 0x7f63fc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:58.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:58.888+0000 7f6417e00640 1 -- 172.21.15.152:0/729735867 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7f641008b970 con 0x7f64040051c0 2024-04-20T11:26:58.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:58.888+0000 7f6416a00640 1 --2- 172.21.15.152:0/729735867 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f63fc06c8f0 0x7f63fc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:58.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:58.889+0000 7f6416a00640 1 --2- 172.21.15.152:0/729735867 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f63fc06c8f0 0x7f63fc06edb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f64080099c0 tx=0x7f6408008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:58.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:58.893+0000 7f6417e00640 1 -- 172.21.15.152:0/729735867 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f6410059d90 con 0x7f64040051c0 2024-04-20T11:26:59.082 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.083+0000 7f6426400640 1 -- 172.21.15.152:0/729735867 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f64200afd00 con 0x7f64040051c0 2024-04-20T11:26:59.084 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.085+0000 7f6417e00640 1 -- 172.21.15.152:0/729735867 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+16678 (secure 0 0 0) 0x7f6410059690 con 0x7f64040051c0 2024-04-20T11:26:59.085 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:26:59.085 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":19,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:26:58.132335+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:26:56.261424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T11:26:57.079286+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:26:59.087 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.087+0000 7f6426400640 1 -- 172.21.15.152:0/729735867 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f63fc06c8f0 msgr2=0x7f63fc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:59.087 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.087+0000 7f6426400640 1 --2- 172.21.15.152:0/729735867 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f63fc06c8f0 0x7f63fc06edb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f64080099c0 tx=0x7f6408008040 comp rx=0 tx=0).stop 2024-04-20T11:26:59.087 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.087+0000 7f6426400640 1 -- 172.21.15.152:0/729735867 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f64040051c0 msgr2=0x7f64200a94a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:59.087 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.087+0000 7f6426400640 1 --2- 172.21.15.152:0/729735867 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f64040051c0 0x7f64200a94a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f641000d4c0 tx=0x7f641000d980 comp rx=0 tx=0).stop 2024-04-20T11:26:59.087 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.087+0000 7f6426400640 1 -- 172.21.15.152:0/729735867 shutdown_connections 2024-04-20T11:26:59.088 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.087+0000 7f6426400640 1 --2- 172.21.15.152:0/729735867 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f63fc06c8f0 0x7f63fc06edb0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:59.088 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.087+0000 7f6426400640 1 --2- 172.21.15.152:0/729735867 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6420134a50 0x7f64201f24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:59.088 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.087+0000 7f6426400640 1 --2- 172.21.15.152:0/729735867 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6420160350 0x7f642015f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:59.088 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.087+0000 7f6426400640 1 --2- 172.21.15.152:0/729735867 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f64040051c0 0x7f64200a94a0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:59.088 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.087+0000 7f6426400640 1 -- 172.21.15.152:0/729735867 >> 172.21.15.152:0/729735867 conn(0x7f6420067cf0 msgr2=0x7f64200d0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:59.088 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.088+0000 7f6426400640 1 -- 172.21.15.152:0/729735867 shutdown_connections 2024-04-20T11:26:59.088 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.088+0000 7f6426400640 1 -- 172.21.15.152:0/729735867 wait complete. 2024-04-20T11:26:59.105 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:26:59.292 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.292+0000 7fb92ea00640 1 Processor -- start 2024-04-20T11:26:59.292 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.292+0000 7fb92ea00640 1 -- start start 2024-04-20T11:26:59.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.294+0000 7fb92ea00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fb928145f60 0x7fb92806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:59.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.294+0000 7fb92ea00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fb9280cd0d0 0x7fb9280cd580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:59.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.294+0000 7fb92ea00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb928071fe0 0x7fb928072470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:59.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.294+0000 7fb92ea00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fb92806e9f0 con 0x7fb9280cd0d0 2024-04-20T11:26:59.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.294+0000 7fb92ea00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fb928146340 con 0x7fb928145f60 2024-04-20T11:26:59.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.294+0000 7fb92ea00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb92806f3c0 con 0x7fb928071fe0 2024-04-20T11:26:59.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.294+0000 7fb927400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fb9280cd0d0 0x7fb9280cd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:59.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.294+0000 7fb927400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fb9280cd0d0 0x7fb9280cd580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:42524/0 (socket says 172.21.15.152:42524) 2024-04-20T11:26:59.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.294+0000 7fb927400640 1 -- 172.21.15.152:0/2637703245 learned_addr learned my addr 172.21.15.152:0/2637703245 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:26:59.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.294+0000 7fb92ca00640 1 --2- 172.21.15.152:0/2637703245 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb928071fe0 0x7fb928072470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:59.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.294+0000 7fb927e00640 1 --2- 172.21.15.152:0/2637703245 >> v2:172.21.15.175:3300/0 conn(0x7fb928145f60 0x7fb92806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:59.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.294+0000 7fb927400640 1 -- 172.21.15.152:0/2637703245 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb928071fe0 msgr2=0x7fb928072470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:59.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.294+0000 7fb927400640 1 --2- 172.21.15.152:0/2637703245 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb928071fe0 0x7fb928072470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:59.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.294+0000 7fb927400640 1 -- 172.21.15.152:0/2637703245 >> v2:172.21.15.175:3300/0 conn(0x7fb928145f60 msgr2=0x7fb92806ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:59.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.294+0000 7fb927400640 1 --2- 172.21.15.152:0/2637703245 >> v2:172.21.15.175:3300/0 conn(0x7fb928145f60 0x7fb92806ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:59.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.294+0000 7fb927400640 1 -- 172.21.15.152:0/2637703245 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb928072d50 con 0x7fb9280cd0d0 2024-04-20T11:26:59.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.296+0000 7fb927400640 1 --2- 172.21.15.152:0/2637703245 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb9280cd0d0 0x7fb9280cd580 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb928073230 tx=0x7fb91402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a42bee676fb49602 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:59.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.296+0000 7fb926a00640 1 -- 172.21.15.152:0/2637703245 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb914035020 con 0x7fb9280cd0d0 2024-04-20T11:26:59.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.296+0000 7fb926a00640 1 -- 172.21.15.152:0/2637703245 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb914031870 con 0x7fb9280cd0d0 2024-04-20T11:26:59.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.296+0000 7fb926a00640 1 -- 172.21.15.152:0/2637703245 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb914031c40 con 0x7fb9280cd0d0 2024-04-20T11:26:59.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.297+0000 7fb92ea00640 1 -- 172.21.15.152:0/2637703245 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb9280cd0d0 msgr2=0x7fb9280cd580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:59.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.297+0000 7fb92ea00640 1 --2- 172.21.15.152:0/2637703245 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb9280cd0d0 0x7fb9280cd580 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb928073230 tx=0x7fb91402b910 comp rx=0 tx=0).stop 2024-04-20T11:26:59.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.297+0000 7fb92ea00640 1 -- 172.21.15.152:0/2637703245 shutdown_connections 2024-04-20T11:26:59.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.297+0000 7fb92ea00640 1 --2- 172.21.15.152:0/2637703245 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb928071fe0 0x7fb928072470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:59.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.297+0000 7fb92ea00640 1 --2- 172.21.15.152:0/2637703245 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb9280cd0d0 0x7fb9280cd580 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:59.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.297+0000 7fb92ea00640 1 --2- 172.21.15.152:0/2637703245 >> v2:172.21.15.175:3300/0 conn(0x7fb928145f60 0x7fb92806ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:59.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.297+0000 7fb92ea00640 1 -- 172.21.15.152:0/2637703245 >> 172.21.15.152:0/2637703245 conn(0x7fb928067cf0 msgr2=0x7fb92806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:59.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.297+0000 7fb92ea00640 1 -- 172.21.15.152:0/2637703245 shutdown_connections 2024-04-20T11:26:59.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.297+0000 7fb92ea00640 1 -- 172.21.15.152:0/2637703245 wait complete. 2024-04-20T11:26:59.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.299+0000 7fb92ea00640 1 Processor -- start 2024-04-20T11:26:59.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.299+0000 7fb92ea00640 1 -- start start 2024-04-20T11:26:59.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.300+0000 7fb92ea00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb928071fe0 0x7fb92815ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:59.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.300+0000 7fb92ea00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9280cd0d0 0x7fb928140890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:59.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.300+0000 7fb92ea00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb928145f60 0x7fb928159910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:59.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.300+0000 7fb92ea00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fb92806e680 con 0x7fb928145f60 2024-04-20T11:26:59.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.300+0000 7fb92ea00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb92806e3a0 con 0x7fb9280cd0d0 2024-04-20T11:26:59.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.300+0000 7fb92ea00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb9280729b0 con 0x7fb928071fe0 2024-04-20T11:26:59.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.300+0000 7fb92ca00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb928145f60 0x7fb928159910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:59.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.300+0000 7fb92ca00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb928145f60 0x7fb928159910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:42528/0 (socket says 172.21.15.152:42528) 2024-04-20T11:26:59.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.300+0000 7fb92ca00640 1 -- 172.21.15.152:0/3730300323 learned_addr learned my addr 172.21.15.152:0/3730300323 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:26:59.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.300+0000 7fb927e00640 1 --2- 172.21.15.152:0/3730300323 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb928071fe0 0x7fb92815ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:59.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.300+0000 7fb927400640 1 --2- 172.21.15.152:0/3730300323 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9280cd0d0 0x7fb928140890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:59.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.300+0000 7fb92ca00640 1 -- 172.21.15.152:0/3730300323 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb928071fe0 msgr2=0x7fb92815ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:59.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.300+0000 7fb92ca00640 1 --2- 172.21.15.152:0/3730300323 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb928071fe0 0x7fb92815ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:59.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.300+0000 7fb92ca00640 1 -- 172.21.15.152:0/3730300323 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9280cd0d0 msgr2=0x7fb928140890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:59.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.300+0000 7fb92ca00640 1 --2- 172.21.15.152:0/3730300323 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9280cd0d0 0x7fb928140890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:59.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.300+0000 7fb92ca00640 1 -- 172.21.15.152:0/3730300323 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb928141200 con 0x7fb928145f60 2024-04-20T11:26:59.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.301+0000 7fb92ca00640 1 --2- 172.21.15.152:0/3730300323 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb928145f60 0x7fb928159910 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb920007d20 tx=0x7fb92000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:59.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.301+0000 7fb91fe00640 1 -- 172.21.15.152:0/3730300323 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb920014420 con 0x7fb928145f60 2024-04-20T11:26:59.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.301+0000 7fb91fe00640 1 -- 172.21.15.152:0/3730300323 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb920019070 con 0x7fb928145f60 2024-04-20T11:26:59.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.301+0000 7fb91fe00640 1 -- 172.21.15.152:0/3730300323 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb920014820 con 0x7fb928145f60 2024-04-20T11:26:59.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.301+0000 7fb92ea00640 1 -- 172.21.15.152:0/3730300323 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9281414e0 con 0x7fb928145f60 2024-04-20T11:26:59.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.301+0000 7fb92ea00640 1 -- 172.21.15.152:0/3730300323 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb928141aa0 con 0x7fb928145f60 2024-04-20T11:26:59.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.302+0000 7fb92ea00640 1 -- 172.21.15.152:0/3730300323 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9280689f0 con 0x7fb928145f60 2024-04-20T11:26:59.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.303+0000 7fb91fe00640 1 -- 172.21.15.152:0/3730300323 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb9200149c0 con 0x7fb928145f60 2024-04-20T11:26:59.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.304+0000 7fb91fe00640 1 --2- 172.21.15.152:0/3730300323 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fb90806c8f0 0x7fb90806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:59.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.304+0000 7fb91fe00640 1 -- 172.21.15.152:0/3730300323 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7fb92008c3f0 con 0x7fb928145f60 2024-04-20T11:26:59.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.304+0000 7fb927e00640 1 --2- 172.21.15.152:0/3730300323 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fb90806c8f0 0x7fb90806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:59.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.305+0000 7fb927e00640 1 --2- 172.21.15.152:0/3730300323 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fb90806c8f0 0x7fb90806edb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb91800a970 tx=0x7fb918008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:59.308 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.309+0000 7fb91fe00640 1 -- 172.21.15.152:0/3730300323 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fb92005a810 con 0x7fb928145f60 2024-04-20T11:26:59.372 INFO:tasks.ceph.mds.c.smithi152.stderr:2024-04-20T11:26:59.372+0000 7feaaae00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-20T11:26:59.509 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.510+0000 7fb92ea00640 1 -- 172.21.15.152:0/3730300323 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb92806bb90 con 0x7fb928145f60 2024-04-20T11:26:59.512 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.513+0000 7fb91fe00640 1 -- 172.21.15.152:0/3730300323 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+16678 (secure 0 0 0) 0x7fb92005a110 con 0x7fb928145f60 2024-04-20T11:26:59.513 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:26:59.513 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":19,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:26:58.132335+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:26:56.261424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T11:26:57.079286+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:26:59.514 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.515+0000 7fb92ea00640 1 -- 172.21.15.152:0/3730300323 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fb90806c8f0 msgr2=0x7fb90806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:59.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.515+0000 7fb92ea00640 1 --2- 172.21.15.152:0/3730300323 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fb90806c8f0 0x7fb90806edb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb91800a970 tx=0x7fb918008040 comp rx=0 tx=0).stop 2024-04-20T11:26:59.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.515+0000 7fb92ea00640 1 -- 172.21.15.152:0/3730300323 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb928145f60 msgr2=0x7fb928159910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:59.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.515+0000 7fb92ea00640 1 --2- 172.21.15.152:0/3730300323 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb928145f60 0x7fb928159910 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb920007d20 tx=0x7fb92000e4b0 comp rx=0 tx=0).stop 2024-04-20T11:26:59.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.515+0000 7fb92ea00640 1 -- 172.21.15.152:0/3730300323 shutdown_connections 2024-04-20T11:26:59.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.515+0000 7fb92ea00640 1 --2- 172.21.15.152:0/3730300323 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fb90806c8f0 0x7fb90806edb0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:59.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.515+0000 7fb92ea00640 1 --2- 172.21.15.152:0/3730300323 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb928145f60 0x7fb928159910 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:59.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.515+0000 7fb92ea00640 1 --2- 172.21.15.152:0/3730300323 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9280cd0d0 0x7fb928140890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:59.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.515+0000 7fb92ea00640 1 --2- 172.21.15.152:0/3730300323 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb928071fe0 0x7fb92815ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:59.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.515+0000 7fb92ea00640 1 -- 172.21.15.152:0/3730300323 >> 172.21.15.152:0/3730300323 conn(0x7fb928067cf0 msgr2=0x7fb9281e5bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:59.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.516+0000 7fb92ea00640 1 -- 172.21.15.152:0/3730300323 shutdown_connections 2024-04-20T11:26:59.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.516+0000 7fb92ea00640 1 -- 172.21.15.152:0/3730300323 wait complete. 2024-04-20T11:26:59.535 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-20T11:26:59.732 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.733+0000 7fe7ef200640 1 Processor -- start 2024-04-20T11:26:59.733 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.733+0000 7fe7ef200640 1 -- start start 2024-04-20T11:26:59.735 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.734+0000 7fe7ef200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fe7f006ee00 0x7fe7f006f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:59.735 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.734+0000 7fe7ef200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fe7f015fbf0 0x7fe7f015c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:59.735 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.734+0000 7fe7ef200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7f00cf110 0x7fe7f00b3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:59.735 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.734+0000 7fe7ef200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fe7f00d00d0 con 0x7fe7f015fbf0 2024-04-20T11:26:59.735 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.734+0000 7fe7ef200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fe7f006f720 con 0x7fe7f006ee00 2024-04-20T11:26:59.735 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.734+0000 7fe7ef200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe7f00cfb10 con 0x7fe7f00cf110 2024-04-20T11:26:59.735 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.735+0000 7fe7ed400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fe7f015fbf0 0x7fe7f015c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:59.735 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.735+0000 7fe7ed400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fe7f015fbf0 0x7fe7f015c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:42544/0 (socket says 172.21.15.152:42544) 2024-04-20T11:26:59.735 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.735+0000 7fe7ed400640 1 -- 172.21.15.152:0/3097340623 learned_addr learned my addr 172.21.15.152:0/3097340623 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:26:59.735 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.735+0000 7fe7ee800640 1 --2- 172.21.15.152:0/3097340623 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7f00cf110 0x7fe7f00b3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:59.735 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.735+0000 7fe7ed400640 1 -- 172.21.15.152:0/3097340623 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7f00cf110 msgr2=0x7fe7f00b3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:59.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.735+0000 7fe7ed400640 1 --2- 172.21.15.152:0/3097340623 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7f00cf110 0x7fe7f00b3ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:59.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.735+0000 7fe7ed400640 1 -- 172.21.15.152:0/3097340623 >> v2:172.21.15.175:3300/0 conn(0x7fe7f006ee00 msgr2=0x7fe7f006f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:59.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.735+0000 7fe7ed400640 1 --2- 172.21.15.152:0/3097340623 >> v2:172.21.15.175:3300/0 conn(0x7fe7f006ee00 0x7fe7f006f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:59.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.735+0000 7fe7ed400640 1 -- 172.21.15.152:0/3097340623 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7f00cfdf0 con 0x7fe7f015fbf0 2024-04-20T11:26:59.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.735+0000 7fe7ee800640 1 --2- 172.21.15.152:0/3097340623 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7f00cf110 0x7fe7f00b3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:26:59.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.736+0000 7fe7ed400640 1 --2- 172.21.15.152:0/3097340623 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe7f015fbf0 0x7fe7f015c390 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe7f006ffc0 tx=0x7fe7e802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da8c37be6d12093b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:59.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.736+0000 7fe7eca00640 1 -- 172.21.15.152:0/3097340623 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e8035020 con 0x7fe7f015fbf0 2024-04-20T11:26:59.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.736+0000 7fe7eca00640 1 -- 172.21.15.152:0/3097340623 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7e8031870 con 0x7fe7f015fbf0 2024-04-20T11:26:59.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.736+0000 7fe7eca00640 1 -- 172.21.15.152:0/3097340623 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e8031c40 con 0x7fe7f015fbf0 2024-04-20T11:26:59.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.737+0000 7fe7ef200640 1 -- 172.21.15.152:0/3097340623 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe7f015fbf0 msgr2=0x7fe7f015c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:59.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.737+0000 7fe7ef200640 1 --2- 172.21.15.152:0/3097340623 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe7f015fbf0 0x7fe7f015c390 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe7f006ffc0 tx=0x7fe7e802b910 comp rx=0 tx=0).stop 2024-04-20T11:26:59.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.737+0000 7fe7ef200640 1 -- 172.21.15.152:0/3097340623 shutdown_connections 2024-04-20T11:26:59.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.737+0000 7fe7ef200640 1 --2- 172.21.15.152:0/3097340623 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7f00cf110 0x7fe7f00b3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:59.738 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.737+0000 7fe7ef200640 1 --2- 172.21.15.152:0/3097340623 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe7f015fbf0 0x7fe7f015c390 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:59.738 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.737+0000 7fe7ef200640 1 --2- 172.21.15.152:0/3097340623 >> v2:172.21.15.175:3300/0 conn(0x7fe7f006ee00 0x7fe7f006f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:59.738 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.737+0000 7fe7ef200640 1 -- 172.21.15.152:0/3097340623 >> 172.21.15.152:0/3097340623 conn(0x7fe7f0067ae0 msgr2=0x7fe7f006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:59.738 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.738+0000 7fe7ef200640 1 -- 172.21.15.152:0/3097340623 shutdown_connections 2024-04-20T11:26:59.738 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.738+0000 7fe7ef200640 1 -- 172.21.15.152:0/3097340623 wait complete. 2024-04-20T11:26:59.738 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.739+0000 7fe7ef200640 1 Processor -- start 2024-04-20T11:26:59.739 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.739+0000 7fe7ef200640 1 -- start start 2024-04-20T11:26:59.741 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.740+0000 7fe7ef200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe7f006ee00 0x7fe7f015a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:59.741 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.740+0000 7fe7ef200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7f00cf110 0x7fe7f0140b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:59.741 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.740+0000 7fe7ef200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7f015fbf0 0x7fe7f013f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:59.741 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.740+0000 7fe7ef200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f015c8d0 con 0x7fe7f006ee00 2024-04-20T11:26:59.741 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.740+0000 7fe7ef200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f015be70 con 0x7fe7f00cf110 2024-04-20T11:26:59.741 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.740+0000 7fe7ef200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe7f0144970 con 0x7fe7f015fbf0 2024-04-20T11:26:59.741 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.740+0000 7fe7ede00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe7f006ee00 0x7fe7f015a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:59.741 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.740+0000 7fe7ede00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe7f006ee00 0x7fe7f015a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:42560/0 (socket says 172.21.15.152:42560) 2024-04-20T11:26:59.742 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.740+0000 7fe7ede00640 1 -- 172.21.15.152:0/2839866902 learned_addr learned my addr 172.21.15.152:0/2839866902 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:26:59.742 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.741+0000 7fe7ed400640 1 --2- 172.21.15.152:0/2839866902 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7f00cf110 0x7fe7f0140b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:59.742 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.741+0000 7fe7ede00640 1 -- 172.21.15.152:0/2839866902 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7f015fbf0 msgr2=0x7fe7f013f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:59.742 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.741+0000 7fe7ee800640 1 --2- 172.21.15.152:0/2839866902 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7f015fbf0 0x7fe7f013f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:59.742 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.741+0000 7fe7ede00640 1 --2- 172.21.15.152:0/2839866902 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7f015fbf0 0x7fe7f013f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:59.742 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.741+0000 7fe7ede00640 1 -- 172.21.15.152:0/2839866902 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7f00cf110 msgr2=0x7fe7f0140b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:59.742 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.741+0000 7fe7ede00640 1 --2- 172.21.15.152:0/2839866902 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7f00cf110 0x7fe7f0140b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:59.742 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.741+0000 7fe7ede00640 1 -- 172.21.15.152:0/2839866902 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7f0141610 con 0x7fe7f006ee00 2024-04-20T11:26:59.742 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.741+0000 7fe7ede00640 1 --2- 172.21.15.152:0/2839866902 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe7f006ee00 0x7fe7f015a540 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe7e000b4a0 tx=0x7fe7e000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:59.742 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.741+0000 7fe7e6000640 1 -- 172.21.15.152:0/2839866902 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e0014070 con 0x7fe7f006ee00 2024-04-20T11:26:59.742 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.741+0000 7fe7e6000640 1 -- 172.21.15.152:0/2839866902 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7e0010610 con 0x7fe7f006ee00 2024-04-20T11:26:59.742 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.741+0000 7fe7e6000640 1 -- 172.21.15.152:0/2839866902 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e0010a00 con 0x7fe7f006ee00 2024-04-20T11:26:59.742 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.741+0000 7fe7ef200640 1 -- 172.21.15.152:0/2839866902 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7f01418f0 con 0x7fe7f006ee00 2024-04-20T11:26:59.742 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.742+0000 7fe7ef200640 1 -- 172.21.15.152:0/2839866902 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7f0141eb0 con 0x7fe7f006ee00 2024-04-20T11:26:59.743 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.743+0000 7fe7ef200640 1 -- 172.21.15.152:0/2839866902 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7f0075e60 con 0x7fe7f006ee00 2024-04-20T11:26:59.743 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.744+0000 7fe7e6000640 1 -- 172.21.15.152:0/2839866902 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe7e0017070 con 0x7fe7f006ee00 2024-04-20T11:26:59.744 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.744+0000 7fe7e6000640 1 --2- 172.21.15.152:0/2839866902 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe7cc06c8f0 0x7fe7cc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:59.744 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.745+0000 7fe7e6000640 1 -- 172.21.15.152:0/2839866902 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fe7e008b5f0 con 0x7fe7f006ee00 2024-04-20T11:26:59.744 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.745+0000 7fe7ed400640 1 --2- 172.21.15.152:0/2839866902 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe7cc06c8f0 0x7fe7cc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:59.744 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.745+0000 7fe7ed400640 1 --2- 172.21.15.152:0/2839866902 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe7cc06c8f0 0x7fe7cc06edb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe7e8036000 tx=0x7fe7e803c040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:59.748 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.749+0000 7fe7e6000640 1 -- 172.21.15.152:0/2839866902 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe7e0059a20 con 0x7fe7f006ee00 2024-04-20T11:26:59.974 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:26:59.975+0000 7fe7ef200640 1 -- 172.21.15.152:0/2839866902 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fe7f00b4140 con 0x7fe7f006ee00 2024-04-20T11:27:00.948 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:00.948+0000 7fe7e6000640 1 -- 172.21.15.152:0/2839866902 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v6) ==== 118+0+0 (secure 0 0 0) 0x7fe7e0059320 con 0x7fe7f006ee00 2024-04-20T11:27:00.951 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:00.951+0000 7fe7ef200640 1 -- 172.21.15.152:0/2839866902 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe7cc06c8f0 msgr2=0x7fe7cc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:00.951 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:00.951+0000 7fe7ef200640 1 --2- 172.21.15.152:0/2839866902 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe7cc06c8f0 0x7fe7cc06edb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe7e8036000 tx=0x7fe7e803c040 comp rx=0 tx=0).stop 2024-04-20T11:27:00.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:00.951+0000 7fe7ef200640 1 -- 172.21.15.152:0/2839866902 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe7f006ee00 msgr2=0x7fe7f015a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:00.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:00.951+0000 7fe7ef200640 1 --2- 172.21.15.152:0/2839866902 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe7f006ee00 0x7fe7f015a540 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe7e000b4a0 tx=0x7fe7e000b960 comp rx=0 tx=0).stop 2024-04-20T11:27:00.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:00.952+0000 7fe7ef200640 1 -- 172.21.15.152:0/2839866902 shutdown_connections 2024-04-20T11:27:00.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:00.952+0000 7fe7ef200640 1 --2- 172.21.15.152:0/2839866902 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe7cc06c8f0 0x7fe7cc06edb0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:00.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:00.952+0000 7fe7ef200640 1 --2- 172.21.15.152:0/2839866902 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7f015fbf0 0x7fe7f013f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:00.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:00.952+0000 7fe7ef200640 1 --2- 172.21.15.152:0/2839866902 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7f00cf110 0x7fe7f0140b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:00.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:00.952+0000 7fe7ef200640 1 --2- 172.21.15.152:0/2839866902 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe7f006ee00 0x7fe7f015a540 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:00.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:00.952+0000 7fe7ef200640 1 -- 172.21.15.152:0/2839866902 >> 172.21.15.152:0/2839866902 conn(0x7fe7f0067ae0 msgr2=0x7fe7f00b0000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:00.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:00.952+0000 7fe7ef200640 1 -- 172.21.15.152:0/2839866902 shutdown_connections 2024-04-20T11:27:00.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:00.952+0000 7fe7ef200640 1 -- 172.21.15.152:0/2839866902 wait complete. 2024-04-20T11:27:00.974 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-20T11:27:00.974 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-20T11:27:01.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:01.180+0000 7f7678800640 1 Processor -- start 2024-04-20T11:27:01.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:01.181+0000 7f7678800640 1 -- start start 2024-04-20T11:27:01.182 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:01.182+0000 7f7678800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f767015e9e0 0x7f767006d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:01.182 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:01.182+0000 7f7678800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f76700d46a0 0x7f76700d4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:01.182 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:01.182+0000 7f7678800640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76700d19a0 con 0x7f76700d0d90 2024-04-20T11:27:01.182 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:01.182+0000 7f7678800640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f76700d5040 con 0x7f76700d46a0 2024-04-20T11:27:01.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:01.182+0000 7f7678800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f76700d1c80 con 0x7f767015e9e0 2024-04-20T11:27:01.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:01.182+0000 7f7675400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f76700d46a0 0x7f76700d4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:01.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:01.182+0000 7f7675400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f76700d46a0 0x7f76700d4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:42562/0 (socket says 172.21.15.152:42562) 2024-04-20T11:27:01.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:01.182+0000 7f7675400640 1 -- 172.21.15.152:0/2576372853 learned_addr learned my addr 172.21.15.152:0/2576372853 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:01.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:01.182+0000 7f7675e00640 1 --2- 172.21.15.152:0/2576372853 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f767015e9e0 0x7f767006d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:01.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:01.183+0000 7f7675400640 1 -- 172.21.15.152:0/2576372853 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f767015e9e0 msgr2=0x7f767006d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:01.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:01.183+0000 7f7675400640 1 --2- 172.21.15.152:0/2576372853 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f767015e9e0 0x7f767006d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:01.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:01.183+0000 7f7675400640 1 -- 172.21.15.152:0/2576372853 >> v1:172.21.15.175:6789/0 conn(0x7f76700d0d90 legacy=0x7f76700cfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:01.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:01.183+0000 7f7675400640 1 -- 172.21.15.152:0/2576372853 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76700d1f60 con 0x7f76700d46a0 2024-04-20T11:27:01.184 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:01.184+0000 7f7675400640 1 --2- 172.21.15.152:0/2576372853 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f76700d46a0 0x7f76700d4b00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7660009870 tx=0x7f766002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ddfa208b290e8e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:01.184 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:01.184+0000 7f7674a00640 1 -- 172.21.15.152:0/2576372853 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7660035020 con 0x7f76700d46a0 2024-04-20T11:27:01.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:01.184+0000 7f7674a00640 1 -- 172.21.15.152:0/2576372853 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7660031640 con 0x7f76700d46a0 2024-04-20T11:27:01.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:01.184+0000 7f7674a00640 1 -- 172.21.15.152:0/2576372853 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f766002d040 con 0x7f76700d46a0 2024-04-20T11:27:01.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:01.185+0000 7f7678800640 1 -- 172.21.15.152:0/2576372853 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f76700d46a0 msgr2=0x7f76700d4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:01.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:01.185+0000 7f7678800640 1 --2- 172.21.15.152:0/2576372853 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f76700d46a0 0x7f76700d4b00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7660009870 tx=0x7f766002b8f0 comp rx=0 tx=0).stop 2024-04-20T11:27:01.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:01.185+0000 7f7678800640 1 -- 172.21.15.152:0/2576372853 shutdown_connections 2024-04-20T11:27:01.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:01.185+0000 7f7678800640 1 --2- 172.21.15.152:0/2576372853 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f76700d46a0 0x7f76700d4b00 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:01.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:01.185+0000 7f7678800640 1 --2- 172.21.15.152:0/2576372853 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f767015e9e0 0x7f767006d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:01.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:01.185+0000 7f7678800640 1 -- 172.21.15.152:0/2576372853 >> 172.21.15.152:0/2576372853 conn(0x7f767009dab0 msgr2=0x7f767008bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:01.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:01.185+0000 7f7678800640 1 -- 172.21.15.152:0/2576372853 shutdown_connections 2024-04-20T11:27:01.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:01.185+0000 7f7678800640 1 -- 172.21.15.152:0/2576372853 wait complete. 2024-04-20T11:27:01.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:01.187+0000 7f7678800640 1 Processor -- start 2024-04-20T11:27:01.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:01.187+0000 7f7678800640 1 -- start start 2024-04-20T11:27:01.188 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:01.188+0000 7f7678800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f76700d0d90 0x7f767015f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:01.188 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:01.188+0000 7f7678800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f76700d46a0 0x7f76700cac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:01.188 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:01.188+0000 7f7678800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f767015e9e0 0x7f76700b6610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:01.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:01.188+0000 7f7678800640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f767006d860 con 0x7f767015e9e0 2024-04-20T11:27:01.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:01.188+0000 7f7678800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f76700cf880 con 0x7f76700d46a0 2024-04-20T11:27:01.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:01.188+0000 7f7678800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f767006d9e0 con 0x7f76700d0d90 2024-04-20T11:27:01.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:01.188+0000 7f7676800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f767015e9e0 0x7f76700b6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:01.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:01.188+0000 7f7676800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f767015e9e0 0x7f76700b6610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:42574/0 (socket says 172.21.15.152:42574) 2024-04-20T11:27:01.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:01.188+0000 7f7676800640 1 -- 172.21.15.152:0/2099761052 learned_addr learned my addr 172.21.15.152:0/2099761052 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:01.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:01.189+0000 7f7675400640 1 --2- 172.21.15.152:0/2099761052 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f76700d46a0 0x7f76700cac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:01.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:01.189+0000 7f7675e00640 1 --2- 172.21.15.152:0/2099761052 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f76700d0d90 0x7f767015f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:01.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:01.189+0000 7f7676800640 1 -- 172.21.15.152:0/2099761052 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f76700d0d90 msgr2=0x7f767015f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:01.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:01.189+0000 7f7676800640 1 --2- 172.21.15.152:0/2099761052 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f76700d0d90 0x7f767015f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:01.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:01.189+0000 7f7676800640 1 -- 172.21.15.152:0/2099761052 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f76700d46a0 msgr2=0x7f76700cac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:01.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:01.189+0000 7f7676800640 1 --2- 172.21.15.152:0/2099761052 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f76700d46a0 0x7f76700cac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:01.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:01.189+0000 7f7676800640 1 -- 172.21.15.152:0/2099761052 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76700b6b50 con 0x7f767015e9e0 2024-04-20T11:27:01.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:01.189+0000 7f7675400640 1 --2- 172.21.15.152:0/2099761052 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f76700d46a0 0x7f76700cac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:27:01.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:01.189+0000 7f7676800640 1 --2- 172.21.15.152:0/2099761052 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f767015e9e0 0x7f76700b6610 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f766c00c440 tx=0x7f766c00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:01.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:01.189+0000 7f7666000640 1 -- 172.21.15.152:0/2099761052 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f766c011790 con 0x7f767015e9e0 2024-04-20T11:27:01.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:01.189+0000 7f7666000640 1 -- 172.21.15.152:0/2099761052 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f766c017070 con 0x7f767015e9e0 2024-04-20T11:27:01.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:01.189+0000 7f7666000640 1 -- 172.21.15.152:0/2099761052 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f766c011b70 con 0x7f767015e9e0 2024-04-20T11:27:01.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:01.190+0000 7f7678800640 1 -- 172.21.15.152:0/2099761052 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76700b5570 con 0x7f767015e9e0 2024-04-20T11:27:01.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:01.190+0000 7f7678800640 1 -- 172.21.15.152:0/2099761052 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76700b5b30 con 0x7f767015e9e0 2024-04-20T11:27:01.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:01.191+0000 7f7678800640 1 -- 172.21.15.152:0/2099761052 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76700689f0 con 0x7f767015e9e0 2024-04-20T11:27:01.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:01.192+0000 7f7666000640 1 -- 172.21.15.152:0/2099761052 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f766c011d10 con 0x7f767015e9e0 2024-04-20T11:27:01.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:01.193+0000 7f7666000640 1 --2- 172.21.15.152:0/2099761052 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f765406c8f0 0x7f765406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:01.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:01.193+0000 7f7666000640 1 -- 172.21.15.152:0/2099761052 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f766c08b930 con 0x7f767015e9e0 2024-04-20T11:27:01.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:01.193+0000 7f7675e00640 1 --2- 172.21.15.152:0/2099761052 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f765406c8f0 0x7f765406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:01.193 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:01.194+0000 7f7675e00640 1 --2- 172.21.15.152:0/2099761052 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f765406c8f0 0x7f765406edb0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f76680099c0 tx=0x7f7668003990 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:01.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:01.197+0000 7f7666000640 1 -- 172.21.15.152:0/2099761052 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f766c059d60 con 0x7f767015e9e0 2024-04-20T11:27:01.442 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:01.442+0000 7f7678800640 1 -- 172.21.15.152:0/2099761052 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f767006d5d0 con 0x7f767015e9e0 2024-04-20T11:27:01.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:01.959+0000 7f7666000640 1 -- 172.21.15.152:0/2099761052 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v7) ==== 122+0+0 (secure 0 0 0) 0x7f766c059660 con 0x7f767015e9e0 2024-04-20T11:27:01.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:01.963+0000 7f7678800640 1 -- 172.21.15.152:0/2099761052 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f765406c8f0 msgr2=0x7f765406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:01.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:01.963+0000 7f7678800640 1 --2- 172.21.15.152:0/2099761052 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f765406c8f0 0x7f765406edb0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f76680099c0 tx=0x7f7668003990 comp rx=0 tx=0).stop 2024-04-20T11:27:01.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:01.963+0000 7f7678800640 1 -- 172.21.15.152:0/2099761052 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f767015e9e0 msgr2=0x7f76700b6610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:01.963 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:01.963+0000 7f7678800640 1 --2- 172.21.15.152:0/2099761052 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f767015e9e0 0x7f76700b6610 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f766c00c440 tx=0x7f766c00c900 comp rx=0 tx=0).stop 2024-04-20T11:27:01.963 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:01.963+0000 7f7678800640 1 -- 172.21.15.152:0/2099761052 shutdown_connections 2024-04-20T11:27:01.963 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:01.963+0000 7f7678800640 1 --2- 172.21.15.152:0/2099761052 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f765406c8f0 0x7f765406edb0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:01.963 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:01.963+0000 7f7678800640 1 --2- 172.21.15.152:0/2099761052 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f767015e9e0 0x7f76700b6610 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:01.964 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:01.963+0000 7f7678800640 1 --2- 172.21.15.152:0/2099761052 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f76700d46a0 0x7f76700cac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:01.964 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:01.963+0000 7f7678800640 1 --2- 172.21.15.152:0/2099761052 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f76700d0d90 0x7f767015f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:01.964 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:01.963+0000 7f7678800640 1 -- 172.21.15.152:0/2099761052 >> 172.21.15.152:0/2099761052 conn(0x7f767009dab0 msgr2=0x7f767008bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:01.964 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:01.964+0000 7f7678800640 1 -- 172.21.15.152:0/2099761052 shutdown_connections 2024-04-20T11:27:01.964 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:01.964+0000 7f7678800640 1 -- 172.21.15.152:0/2099761052 wait complete. 2024-04-20T11:27:01.984 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-20T11:27:01.985 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:27:02.182 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.183+0000 7fe0fe600640 1 Processor -- start 2024-04-20T11:27:02.182 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.183+0000 7fe0fe600640 1 -- start start 2024-04-20T11:27:02.184 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.184+0000 7fe0fe600640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fe0f815d1e0 0x7fe0f8160b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:02.184 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.184+0000 7fe0fe600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe0f815d5c0 0x7fe0f80cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:02.184 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.184+0000 7fe0fe600640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fe0f8071cd0 0x7fe0f8072160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:02.184 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.184+0000 7fe0fe600640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fe0f80726a0 con 0x7fe0f8071cd0 2024-04-20T11:27:02.184 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.184+0000 7fe0fe600640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fe0f80ce8c0 con 0x7fe0f815d1e0 2024-04-20T11:27:02.184 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.184+0000 7fe0fe600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe0f81600b0 con 0x7fe0f815d5c0 2024-04-20T11:27:02.184 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.185+0000 7fe0fdc00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fe0f8071cd0 0x7fe0f8072160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:02.184 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.185+0000 7fe0fdc00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fe0f8071cd0 0x7fe0f8072160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:42588/0 (socket says 172.21.15.152:42588) 2024-04-20T11:27:02.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.185+0000 7fe0fdc00640 1 -- 172.21.15.152:0/1696526818 learned_addr learned my addr 172.21.15.152:0/1696526818 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:02.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.185+0000 7fe0fd200640 1 --2- 172.21.15.152:0/1696526818 >> v2:172.21.15.175:3300/0 conn(0x7fe0f815d1e0 0x7fe0f8160b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:02.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.185+0000 7fe0f7e00640 1 --2- 172.21.15.152:0/1696526818 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe0f815d5c0 0x7fe0f80cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:02.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.185+0000 7fe0fdc00640 1 -- 172.21.15.152:0/1696526818 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe0f815d5c0 msgr2=0x7fe0f80cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:02.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.185+0000 7fe0fdc00640 1 --2- 172.21.15.152:0/1696526818 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe0f815d5c0 0x7fe0f80cdfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:02.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.185+0000 7fe0fdc00640 1 -- 172.21.15.152:0/1696526818 >> v2:172.21.15.175:3300/0 conn(0x7fe0f815d1e0 msgr2=0x7fe0f8160b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:02.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.185+0000 7fe0fdc00640 1 --2- 172.21.15.152:0/1696526818 >> v2:172.21.15.175:3300/0 conn(0x7fe0f815d1e0 0x7fe0f8160b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:02.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.185+0000 7fe0fdc00640 1 -- 172.21.15.152:0/1696526818 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0f8160390 con 0x7fe0f8071cd0 2024-04-20T11:27:02.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.185+0000 7fe0fd200640 1 --2- 172.21.15.152:0/1696526818 >> v2:172.21.15.175:3300/0 conn(0x7fe0f815d1e0 0x7fe0f8160b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:27:02.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.185+0000 7fe0f7e00640 1 --2- 172.21.15.152:0/1696526818 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe0f815d5c0 0x7fe0f80cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:27:02.187 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.186+0000 7fe0fdc00640 1 --2- 172.21.15.152:0/1696526818 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe0f8071cd0 0x7fe0f8072160 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe0f806c9b0 tx=0x7fe0f002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=479b3817a0938d57 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:02.187 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.186+0000 7fe0f7400640 1 -- 172.21.15.152:0/1696526818 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0f002ca80 con 0x7fe0f8071cd0 2024-04-20T11:27:02.187 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.187+0000 7fe0f7400640 1 -- 172.21.15.152:0/1696526818 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0f002cc20 con 0x7fe0f8071cd0 2024-04-20T11:27:02.187 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.187+0000 7fe0f7400640 1 -- 172.21.15.152:0/1696526818 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0f00343f0 con 0x7fe0f8071cd0 2024-04-20T11:27:02.187 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.187+0000 7fe0fe600640 1 -- 172.21.15.152:0/1696526818 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe0f8071cd0 msgr2=0x7fe0f8072160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:02.188 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.187+0000 7fe0fe600640 1 --2- 172.21.15.152:0/1696526818 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe0f8071cd0 0x7fe0f8072160 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe0f806c9b0 tx=0x7fe0f002bee0 comp rx=0 tx=0).stop 2024-04-20T11:27:02.188 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.187+0000 7fe0fe600640 1 -- 172.21.15.152:0/1696526818 shutdown_connections 2024-04-20T11:27:02.188 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.187+0000 7fe0fe600640 1 --2- 172.21.15.152:0/1696526818 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe0f8071cd0 0x7fe0f8072160 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:02.188 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.187+0000 7fe0fe600640 1 --2- 172.21.15.152:0/1696526818 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe0f815d5c0 0x7fe0f80cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:02.188 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.187+0000 7fe0fe600640 1 --2- 172.21.15.152:0/1696526818 >> v2:172.21.15.175:3300/0 conn(0x7fe0f815d1e0 0x7fe0f8160b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:02.188 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.187+0000 7fe0fe600640 1 -- 172.21.15.152:0/1696526818 >> 172.21.15.152:0/1696526818 conn(0x7fe0f8067ae0 msgr2=0x7fe0f806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:02.188 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.187+0000 7fe0fe600640 1 -- 172.21.15.152:0/1696526818 shutdown_connections 2024-04-20T11:27:02.188 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.187+0000 7fe0fe600640 1 -- 172.21.15.152:0/1696526818 wait complete. 2024-04-20T11:27:02.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.189+0000 7fe0fe600640 1 Processor -- start 2024-04-20T11:27:02.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.189+0000 7fe0fe600640 1 -- start start 2024-04-20T11:27:02.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.190+0000 7fe0fe600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0f8071cd0 0x7fe0f81546b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:02.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.190+0000 7fe0fe600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe0f815d1e0 0x7fe0f8150b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:02.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.190+0000 7fe0fe600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe0f815d5c0 0x7fe0f8152b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:02.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.190+0000 7fe0fe600640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fe0f80ce520 con 0x7fe0f815d1e0 2024-04-20T11:27:02.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.190+0000 7fe0fe600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe0f80ceba0 con 0x7fe0f8071cd0 2024-04-20T11:27:02.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.190+0000 7fe0fe600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe0f80ce6a0 con 0x7fe0f815d5c0 2024-04-20T11:27:02.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.190+0000 7fe0f7e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe0f815d1e0 0x7fe0f8150b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:02.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.190+0000 7fe0f7e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe0f815d1e0 0x7fe0f8150b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:42592/0 (socket says 172.21.15.152:42592) 2024-04-20T11:27:02.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.190+0000 7fe0f7e00640 1 -- 172.21.15.152:0/797470799 learned_addr learned my addr 172.21.15.152:0/797470799 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:02.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.191+0000 7fe0fd200640 1 --2- 172.21.15.152:0/797470799 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0f8071cd0 0x7fe0f81546b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:02.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.191+0000 7fe0fdc00640 1 --2- 172.21.15.152:0/797470799 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe0f815d5c0 0x7fe0f8152b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:02.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.191+0000 7fe0f7e00640 1 -- 172.21.15.152:0/797470799 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe0f815d5c0 msgr2=0x7fe0f8152b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:02.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.191+0000 7fe0f7e00640 1 --2- 172.21.15.152:0/797470799 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe0f815d5c0 0x7fe0f8152b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:02.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.191+0000 7fe0f7e00640 1 -- 172.21.15.152:0/797470799 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0f8071cd0 msgr2=0x7fe0f81546b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:02.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.191+0000 7fe0f7e00640 1 --2- 172.21.15.152:0/797470799 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0f8071cd0 0x7fe0f81546b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:02.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.191+0000 7fe0f7e00640 1 -- 172.21.15.152:0/797470799 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0f80afab0 con 0x7fe0f815d1e0 2024-04-20T11:27:02.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.191+0000 7fe0fd200640 1 --2- 172.21.15.152:0/797470799 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0f8071cd0 0x7fe0f81546b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:27:02.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.191+0000 7fe0f7e00640 1 --2- 172.21.15.152:0/797470799 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe0f815d1e0 0x7fe0f8150b30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe0e400c7f0 tx=0x7fe0e400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:02.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.191+0000 7fe0f4c00640 1 -- 172.21.15.152:0/797470799 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e4015070 con 0x7fe0f815d1e0 2024-04-20T11:27:02.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.191+0000 7fe0f4c00640 1 -- 172.21.15.152:0/797470799 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0e4007a30 con 0x7fe0f815d1e0 2024-04-20T11:27:02.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.191+0000 7fe0fe600640 1 -- 172.21.15.152:0/797470799 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0f8152050 con 0x7fe0f815d1e0 2024-04-20T11:27:02.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.191+0000 7fe0f4c00640 1 -- 172.21.15.152:0/797470799 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e4007e00 con 0x7fe0f815d1e0 2024-04-20T11:27:02.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.191+0000 7fe0fe600640 1 -- 172.21.15.152:0/797470799 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0f8152610 con 0x7fe0f815d1e0 2024-04-20T11:27:02.193 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.192+0000 7fe0fe600640 1 -- 172.21.15.152:0/797470799 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0f8075e60 con 0x7fe0f815d1e0 2024-04-20T11:27:02.193 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.193+0000 7fe0f4c00640 1 -- 172.21.15.152:0/797470799 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe0e4018070 con 0x7fe0f815d1e0 2024-04-20T11:27:02.194 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.194+0000 7fe0f4c00640 1 --2- 172.21.15.152:0/797470799 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe0d806c8f0 0x7fe0d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:02.194 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.194+0000 7fe0f4c00640 1 -- 172.21.15.152:0/797470799 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fe0e408bb40 con 0x7fe0f815d1e0 2024-04-20T11:27:02.194 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.194+0000 7fe0fd200640 1 --2- 172.21.15.152:0/797470799 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe0d806c8f0 0x7fe0d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:02.194 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.195+0000 7fe0fd200640 1 --2- 172.21.15.152:0/797470799 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe0d806c8f0 0x7fe0d806edb0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec0088b0 tx=0x7fe0ec008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:02.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.199+0000 7fe0f4c00640 1 -- 172.21.15.152:0/797470799 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe0e4059f60 con 0x7fe0f815d1e0 2024-04-20T11:27:02.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.425+0000 7fe0fe600640 1 -- 172.21.15.152:0/797470799 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe0f80cdcf0 con 0x7fe0f815d1e0 2024-04-20T11:27:02.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.427+0000 7fe0f4c00640 1 -- 172.21.15.152:0/797470799 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5196 (secure 0 0 0) 0x7fe0e4059860 con 0x7fe0f815d1e0 2024-04-20T11:27:02.427 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:27:02.427 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/853233806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":853233806},{"type":"v1","addr":"172.21.15.175:6833","nonce":853233806}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4253,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/100300237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":100300237},{"type":"v1","addr":"172.21.15.175:6835","nonce":100300237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/2924034058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":2924034058},{"type":"v1","addr":"172.21.15.152:6835","nonce":2924034058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T11:26:58.132293+0000","modified":"2024-04-20T11:27:01.444285+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4269":{"gid":4269,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6837/927639144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":927639144},{"type":"v1","addr":"172.21.15.152:6837","nonce":927639144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4269,"qdb_cluster":[4269]},"id":1}]} 2024-04-20T11:27:02.427 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 7 2024-04-20T11:27:02.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.429+0000 7fe0fe600640 1 -- 172.21.15.152:0/797470799 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe0d806c8f0 msgr2=0x7fe0d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:02.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.429+0000 7fe0fe600640 1 --2- 172.21.15.152:0/797470799 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe0d806c8f0 0x7fe0d806edb0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec0088b0 tx=0x7fe0ec008040 comp rx=0 tx=0).stop 2024-04-20T11:27:02.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.429+0000 7fe0fe600640 1 -- 172.21.15.152:0/797470799 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe0f815d1e0 msgr2=0x7fe0f8150b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:02.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.429+0000 7fe0fe600640 1 --2- 172.21.15.152:0/797470799 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe0f815d1e0 0x7fe0f8150b30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe0e400c7f0 tx=0x7fe0e400cbf0 comp rx=0 tx=0).stop 2024-04-20T11:27:02.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.429+0000 7fe0fe600640 1 -- 172.21.15.152:0/797470799 shutdown_connections 2024-04-20T11:27:02.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.429+0000 7fe0fe600640 1 --2- 172.21.15.152:0/797470799 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe0d806c8f0 0x7fe0d806edb0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:02.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.429+0000 7fe0fe600640 1 --2- 172.21.15.152:0/797470799 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe0f815d5c0 0x7fe0f8152b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:02.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.429+0000 7fe0fe600640 1 --2- 172.21.15.152:0/797470799 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe0f815d1e0 0x7fe0f8150b30 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:02.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.429+0000 7fe0fe600640 1 --2- 172.21.15.152:0/797470799 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0f8071cd0 0x7fe0f81546b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:02.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.429+0000 7fe0fe600640 1 -- 172.21.15.152:0/797470799 >> 172.21.15.152:0/797470799 conn(0x7fe0f8067ae0 msgr2=0x7fe0f815b2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:02.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.429+0000 7fe0fe600640 1 -- 172.21.15.152:0/797470799 shutdown_connections 2024-04-20T11:27:02.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.429+0000 7fe0fe600640 1 -- 172.21.15.152:0/797470799 wait complete. 2024-04-20T11:27:02.448 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:27:02.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.648+0000 7fa9e6e00640 1 Processor -- start 2024-04-20T11:27:02.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.648+0000 7fa9e6e00640 1 -- start start 2024-04-20T11:27:02.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.650+0000 7fa9e6e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fa9e015fcc0 0x7fa9e00d4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:02.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.650+0000 7fa9e6e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fa9e01603d0 0x7fa9e00b5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:02.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.650+0000 7fa9e6e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa9e00b5480 0x7fa9e00b63f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:02.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.650+0000 7fa9e6e00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fa9e00d3350 con 0x7fa9e015fcc0 2024-04-20T11:27:02.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.650+0000 7fa9e6e00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fa9e00d3630 con 0x7fa9e01603d0 2024-04-20T11:27:02.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.650+0000 7fa9e6e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa9e00d3910 con 0x7fa9e00b5480 2024-04-20T11:27:02.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.650+0000 7fa9e5a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fa9e015fcc0 0x7fa9e00d4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:02.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.650+0000 7fa9e5a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fa9e015fcc0 0x7fa9e00d4e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:42598/0 (socket says 172.21.15.152:42598) 2024-04-20T11:27:02.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.650+0000 7fa9e5a00640 1 -- 172.21.15.152:0/113438207 learned_addr learned my addr 172.21.15.152:0/113438207 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:02.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.650+0000 7fa9e5a00640 1 -- 172.21.15.152:0/113438207 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa9e00b5480 msgr2=0x7fa9e00b63f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:02.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.650+0000 7fa9e5a00640 1 --2- 172.21.15.152:0/113438207 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa9e00b5480 0x7fa9e00b63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:02.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.650+0000 7fa9e5000640 1 --2- 172.21.15.152:0/113438207 >> v2:172.21.15.175:3300/0 conn(0x7fa9e01603d0 0x7fa9e00b5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:02.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.650+0000 7fa9e5a00640 1 -- 172.21.15.152:0/113438207 >> v2:172.21.15.175:3300/0 conn(0x7fa9e01603d0 msgr2=0x7fa9e00b5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:02.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.650+0000 7fa9e5a00640 1 --2- 172.21.15.152:0/113438207 >> v2:172.21.15.175:3300/0 conn(0x7fa9e01603d0 0x7fa9e00b5eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:02.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.651+0000 7fa9e5a00640 1 -- 172.21.15.152:0/113438207 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9e00ced20 con 0x7fa9e015fcc0 2024-04-20T11:27:02.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.652+0000 7fa9e5a00640 1 --2- 172.21.15.152:0/113438207 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa9e015fcc0 0x7fa9e00d4e70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa9e008ca40 tx=0x7fa9d402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b7bbb1862824e7c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:02.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.652+0000 7fa9dfe00640 1 -- 172.21.15.152:0/113438207 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d4035020 con 0x7fa9e015fcc0 2024-04-20T11:27:02.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.652+0000 7fa9dfe00640 1 -- 172.21.15.152:0/113438207 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9d4031870 con 0x7fa9e015fcc0 2024-04-20T11:27:02.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.652+0000 7fa9dfe00640 1 -- 172.21.15.152:0/113438207 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d4031c40 con 0x7fa9e015fcc0 2024-04-20T11:27:02.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.653+0000 7fa9e6e00640 1 -- 172.21.15.152:0/113438207 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa9e015fcc0 msgr2=0x7fa9e00d4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:02.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.653+0000 7fa9e6e00640 1 --2- 172.21.15.152:0/113438207 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa9e015fcc0 0x7fa9e00d4e70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa9e008ca40 tx=0x7fa9d402b910 comp rx=0 tx=0).stop 2024-04-20T11:27:02.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.653+0000 7fa9e6e00640 1 -- 172.21.15.152:0/113438207 shutdown_connections 2024-04-20T11:27:02.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.653+0000 7fa9e6e00640 1 --2- 172.21.15.152:0/113438207 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa9e00b5480 0x7fa9e00b63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:02.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.653+0000 7fa9e6e00640 1 --2- 172.21.15.152:0/113438207 >> v2:172.21.15.175:3300/0 conn(0x7fa9e01603d0 0x7fa9e00b5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:02.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.653+0000 7fa9e6e00640 1 --2- 172.21.15.152:0/113438207 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa9e015fcc0 0x7fa9e00d4e70 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:02.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.653+0000 7fa9e6e00640 1 -- 172.21.15.152:0/113438207 >> 172.21.15.152:0/113438207 conn(0x7fa9e008bfd0 msgr2=0x7fa9e00d4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:02.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.653+0000 7fa9e6e00640 1 -- 172.21.15.152:0/113438207 shutdown_connections 2024-04-20T11:27:02.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.653+0000 7fa9e6e00640 1 -- 172.21.15.152:0/113438207 wait complete. 2024-04-20T11:27:02.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.655+0000 7fa9e6e00640 1 Processor -- start 2024-04-20T11:27:02.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.655+0000 7fa9e6e00640 1 -- start start 2024-04-20T11:27:02.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.656+0000 7fa9e6e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa9e00b5480 0x7fa9e00b6f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:02.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.656+0000 7fa9e6e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9e015fcc0 0x7fa9e00cb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:02.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.656+0000 7fa9e6e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa9e01603d0 0x7fa9e0160e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:02.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.656+0000 7fa9e6e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e0162650 con 0x7fa9e00b5480 2024-04-20T11:27:02.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.656+0000 7fa9e6e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e01624d0 con 0x7fa9e015fcc0 2024-04-20T11:27:02.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.656+0000 7fa9e6e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa9e01627d0 con 0x7fa9e01603d0 2024-04-20T11:27:02.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.656+0000 7fa9e5a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa9e00b5480 0x7fa9e00b6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:02.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.656+0000 7fa9e5a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa9e00b5480 0x7fa9e00b6f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:42610/0 (socket says 172.21.15.152:42610) 2024-04-20T11:27:02.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.656+0000 7fa9e5a00640 1 -- 172.21.15.152:0/3040471651 learned_addr learned my addr 172.21.15.152:0/3040471651 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:02.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.657+0000 7fa9e5000640 1 --2- 172.21.15.152:0/3040471651 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9e015fcc0 0x7fa9e00cb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:02.664 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.657+0000 7fa9e5a00640 1 -- 172.21.15.152:0/3040471651 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa9e01603d0 msgr2=0x7fa9e0160e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:02.664 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.657+0000 7fa9e5a00640 1 --2- 172.21.15.152:0/3040471651 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa9e01603d0 0x7fa9e0160e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:02.664 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.657+0000 7fa9e5a00640 1 -- 172.21.15.152:0/3040471651 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9e015fcc0 msgr2=0x7fa9e00cb7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:02.664 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.657+0000 7fa9e5a00640 1 --2- 172.21.15.152:0/3040471651 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9e015fcc0 0x7fa9e00cb7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:02.664 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.657+0000 7fa9e5a00640 1 -- 172.21.15.152:0/3040471651 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9e014b4c0 con 0x7fa9e00b5480 2024-04-20T11:27:02.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.657+0000 7fa9e5a00640 1 --2- 172.21.15.152:0/3040471651 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa9e00b5480 0x7fa9e00b6f00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa9d4000c00 tx=0x7fa9d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:02.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.658+0000 7fa9dd600640 1 -- 172.21.15.152:0/3040471651 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d4040070 con 0x7fa9e00b5480 2024-04-20T11:27:02.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.658+0000 7fa9dd600640 1 -- 172.21.15.152:0/3040471651 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9d403a420 con 0x7fa9e00b5480 2024-04-20T11:27:02.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.658+0000 7fa9e6e00640 1 -- 172.21.15.152:0/3040471651 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9e014b7a0 con 0x7fa9e00b5480 2024-04-20T11:27:02.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.658+0000 7fa9dd600640 1 -- 172.21.15.152:0/3040471651 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d4035040 con 0x7fa9e00b5480 2024-04-20T11:27:02.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.658+0000 7fa9e6e00640 1 -- 172.21.15.152:0/3040471651 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9e014bd60 con 0x7fa9e00b5480 2024-04-20T11:27:02.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.659+0000 7fa9e6e00640 1 -- 172.21.15.152:0/3040471651 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9e0075e60 con 0x7fa9e00b5480 2024-04-20T11:27:02.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.660+0000 7fa9dd600640 1 -- 172.21.15.152:0/3040471651 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa9d404b060 con 0x7fa9e00b5480 2024-04-20T11:27:02.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.661+0000 7fa9dd600640 1 --2- 172.21.15.152:0/3040471651 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fa9c006c920 0x7fa9c006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:02.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.661+0000 7fa9dd600640 1 -- 172.21.15.152:0/3040471651 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa9d40ada00 con 0x7fa9e00b5480 2024-04-20T11:27:02.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.661+0000 7fa9e5000640 1 --2- 172.21.15.152:0/3040471651 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fa9c006c920 0x7fa9c006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:02.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.662+0000 7fa9e5000640 1 --2- 172.21.15.152:0/3040471651 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fa9c006c920 0x7fa9c006ede0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa9e0161a60 tx=0x7fa9cc008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:02.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.665+0000 7fa9dd600640 1 -- 172.21.15.152:0/3040471651 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fa9d407be20 con 0x7fa9e00b5480 2024-04-20T11:27:02.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.868+0000 7fa9e6e00640 1 -- 172.21.15.152:0/3040471651 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa9e006cb90 con 0x7fa9e00b5480 2024-04-20T11:27:02.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.871+0000 7fa9dd600640 1 -- 172.21.15.152:0/3040471651 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16665 (secure 0 0 0) 0x7fa9d407b720 con 0x7fa9e00b5480 2024-04-20T11:27:02.872 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:27:02.872 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":20,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:26:59.675971+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:26:56.261424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T11:26:57.079286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:27:02.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.874+0000 7fa9e6e00640 1 -- 172.21.15.152:0/3040471651 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fa9c006c920 msgr2=0x7fa9c006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:02.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.874+0000 7fa9e6e00640 1 --2- 172.21.15.152:0/3040471651 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fa9c006c920 0x7fa9c006ede0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa9e0161a60 tx=0x7fa9cc008040 comp rx=0 tx=0).stop 2024-04-20T11:27:02.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.874+0000 7fa9e6e00640 1 -- 172.21.15.152:0/3040471651 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa9e00b5480 msgr2=0x7fa9e00b6f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:02.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.874+0000 7fa9e6e00640 1 --2- 172.21.15.152:0/3040471651 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa9e00b5480 0x7fa9e00b6f00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa9d4000c00 tx=0x7fa9d40046c0 comp rx=0 tx=0).stop 2024-04-20T11:27:02.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.874+0000 7fa9e6e00640 1 -- 172.21.15.152:0/3040471651 shutdown_connections 2024-04-20T11:27:02.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.874+0000 7fa9e6e00640 1 --2- 172.21.15.152:0/3040471651 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fa9c006c920 0x7fa9c006ede0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:02.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.874+0000 7fa9e6e00640 1 --2- 172.21.15.152:0/3040471651 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa9e01603d0 0x7fa9e0160e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:02.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.874+0000 7fa9e6e00640 1 --2- 172.21.15.152:0/3040471651 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9e015fcc0 0x7fa9e00cb7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:02.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.874+0000 7fa9e6e00640 1 --2- 172.21.15.152:0/3040471651 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa9e00b5480 0x7fa9e00b6f00 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:02.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.874+0000 7fa9e6e00640 1 -- 172.21.15.152:0/3040471651 >> 172.21.15.152:0/3040471651 conn(0x7fa9e008bfd0 msgr2=0x7fa9e0162d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:02.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.875+0000 7fa9e6e00640 1 -- 172.21.15.152:0/3040471651 shutdown_connections 2024-04-20T11:27:02.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:02.875+0000 7fa9e6e00640 1 -- 172.21.15.152:0/3040471651 wait complete. 2024-04-20T11:27:02.893 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-20T11:27:02.894 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T11:27:03.087 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.087+0000 7fb10b000640 1 Processor -- start 2024-04-20T11:27:03.087 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.088+0000 7fb10b000640 1 -- start start 2024-04-20T11:27:03.089 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.089+0000 7fb10b000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fb10415fcc0 0x7fb1040d4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:03.089 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.089+0000 7fb10b000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb1041603d0 0x7fb1040b5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:03.089 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.089+0000 7fb10b000640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1040d3350 con 0x7fb1040b5480 2024-04-20T11:27:03.089 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.089+0000 7fb10b000640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fb1040d3630 con 0x7fb10415fcc0 2024-04-20T11:27:03.089 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.089+0000 7fb10b000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb1040d3910 con 0x7fb1041603d0 2024-04-20T11:27:03.089 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.089+0000 7fb109c00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fb10415fcc0 0x7fb1040d4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:03.089 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.089+0000 7fb109c00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fb10415fcc0 0x7fb1040d4e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:42616/0 (socket says 172.21.15.152:42616) 2024-04-20T11:27:03.089 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.089+0000 7fb109c00640 1 -- 172.21.15.152:0/3611001722 learned_addr learned my addr 172.21.15.152:0/3611001722 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:03.090 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.090+0000 7fb109c00640 1 -- 172.21.15.152:0/3611001722 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb1041603d0 msgr2=0x7fb1040b5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:03.090 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.090+0000 7fb109200640 1 --2- 172.21.15.152:0/3611001722 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb1041603d0 0x7fb1040b5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:03.090 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.090+0000 7fb109c00640 1 --2- 172.21.15.152:0/3611001722 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb1041603d0 0x7fb1040b5eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:03.090 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.090+0000 7fb109c00640 1 -- 172.21.15.152:0/3611001722 >> v1:172.21.15.175:6789/0 conn(0x7fb1040b5480 legacy=0x7fb1041624d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:03.090 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.090+0000 7fb109c00640 1 -- 172.21.15.152:0/3611001722 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1040ced20 con 0x7fb10415fcc0 2024-04-20T11:27:03.090 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.091+0000 7fb109c00640 1 --2- 172.21.15.152:0/3611001722 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb10415fcc0 0x7fb1040d4e70 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb0f8009870 tx=0x7fb0f802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1d8f14a0c32781 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:03.091 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.091+0000 7fb103e00640 1 -- 172.21.15.152:0/3611001722 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f8035020 con 0x7fb10415fcc0 2024-04-20T11:27:03.091 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.091+0000 7fb103e00640 1 -- 172.21.15.152:0/3611001722 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0f8031640 con 0x7fb10415fcc0 2024-04-20T11:27:03.091 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.091+0000 7fb103e00640 1 -- 172.21.15.152:0/3611001722 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f802d040 con 0x7fb10415fcc0 2024-04-20T11:27:03.091 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.092+0000 7fb10b000640 1 -- 172.21.15.152:0/3611001722 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb10415fcc0 msgr2=0x7fb1040d4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:03.092 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.092+0000 7fb10b000640 1 --2- 172.21.15.152:0/3611001722 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb10415fcc0 0x7fb1040d4e70 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb0f8009870 tx=0x7fb0f802b8f0 comp rx=0 tx=0).stop 2024-04-20T11:27:03.092 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.092+0000 7fb10b000640 1 -- 172.21.15.152:0/3611001722 shutdown_connections 2024-04-20T11:27:03.092 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.092+0000 7fb10b000640 1 --2- 172.21.15.152:0/3611001722 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb1041603d0 0x7fb1040b5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:03.092 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.092+0000 7fb10b000640 1 --2- 172.21.15.152:0/3611001722 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb10415fcc0 0x7fb1040d4e70 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:03.092 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.092+0000 7fb10b000640 1 -- 172.21.15.152:0/3611001722 >> 172.21.15.152:0/3611001722 conn(0x7fb10408bfd0 msgr2=0x7fb1040d4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:03.092 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.092+0000 7fb10b000640 1 -- 172.21.15.152:0/3611001722 shutdown_connections 2024-04-20T11:27:03.092 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.092+0000 7fb10b000640 1 -- 172.21.15.152:0/3611001722 wait complete. 2024-04-20T11:27:03.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.093+0000 7fb10b000640 1 Processor -- start 2024-04-20T11:27:03.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.094+0000 7fb10b000640 1 -- start start 2024-04-20T11:27:03.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.094+0000 7fb10b000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1040b5480 0x7fb1040ca690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:03.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.094+0000 7fb10b000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb10415fcc0 0x7fb104149f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:03.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.094+0000 7fb10b000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb1041603d0 0x7fb104147480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:03.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.095+0000 7fb10b000640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fb1040b6570 con 0x7fb10415fcc0 2024-04-20T11:27:03.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.095+0000 7fb10b000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb1040b63f0 con 0x7fb1040b5480 2024-04-20T11:27:03.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.095+0000 7fb10b000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb1040b66f0 con 0x7fb1041603d0 2024-04-20T11:27:03.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.095+0000 7fb109200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb10415fcc0 0x7fb104149f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:03.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.095+0000 7fb109200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb10415fcc0 0x7fb104149f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:42632/0 (socket says 172.21.15.152:42632) 2024-04-20T11:27:03.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.095+0000 7fb109200640 1 -- 172.21.15.152:0/2238916174 learned_addr learned my addr 172.21.15.152:0/2238916174 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:03.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.095+0000 7fb109c00640 1 --2- 172.21.15.152:0/2238916174 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1040b5480 0x7fb1040ca690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:03.096 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.095+0000 7fb10a600640 1 --2- 172.21.15.152:0/2238916174 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb1041603d0 0x7fb104147480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:03.096 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.095+0000 7fb109200640 1 -- 172.21.15.152:0/2238916174 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb1041603d0 msgr2=0x7fb104147480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:03.096 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.095+0000 7fb109200640 1 --2- 172.21.15.152:0/2238916174 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb1041603d0 0x7fb104147480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:03.096 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.095+0000 7fb109200640 1 -- 172.21.15.152:0/2238916174 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1040b5480 msgr2=0x7fb1040ca690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:03.096 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.095+0000 7fb109200640 1 --2- 172.21.15.152:0/2238916174 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1040b5480 0x7fb1040ca690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:03.096 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.095+0000 7fb109200640 1 -- 172.21.15.152:0/2238916174 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb10414a460 con 0x7fb10415fcc0 2024-04-20T11:27:03.096 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.095+0000 7fb109c00640 1 --2- 172.21.15.152:0/2238916174 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1040b5480 0x7fb1040ca690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:27:03.096 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.096+0000 7fb109200640 1 --2- 172.21.15.152:0/2238916174 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb10415fcc0 0x7fb104149f20 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb0f000c820 tx=0x7fb0f000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:03.097 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.096+0000 7fb101600640 1 -- 172.21.15.152:0/2238916174 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f0015070 con 0x7fb10415fcc0 2024-04-20T11:27:03.097 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.096+0000 7fb101600640 1 -- 172.21.15.152:0/2238916174 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0f0004220 con 0x7fb10415fcc0 2024-04-20T11:27:03.097 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.096+0000 7fb10b000640 1 -- 172.21.15.152:0/2238916174 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1041479c0 con 0x7fb10415fcc0 2024-04-20T11:27:03.097 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.096+0000 7fb101600640 1 -- 172.21.15.152:0/2238916174 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f000ce10 con 0x7fb10415fcc0 2024-04-20T11:27:03.097 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.096+0000 7fb10b000640 1 -- 172.21.15.152:0/2238916174 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb104159c50 con 0x7fb10415fcc0 2024-04-20T11:27:03.098 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.097+0000 7fb10b000640 1 -- 172.21.15.152:0/2238916174 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb104075e60 con 0x7fb10415fcc0 2024-04-20T11:27:03.098 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.098+0000 7fb101600640 1 -- 172.21.15.152:0/2238916174 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb0f0007830 con 0x7fb10415fcc0 2024-04-20T11:27:03.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.099+0000 7fb101600640 1 --2- 172.21.15.152:0/2238916174 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fb0e406c8f0 0x7fb0e406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:03.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.100+0000 7fb101600640 1 -- 172.21.15.152:0/2238916174 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb0f008bdc0 con 0x7fb10415fcc0 2024-04-20T11:27:03.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.100+0000 7fb109c00640 1 --2- 172.21.15.152:0/2238916174 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fb0e406c8f0 0x7fb0e406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:03.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.100+0000 7fb109c00640 1 --2- 172.21.15.152:0/2238916174 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fb0e406c8f0 0x7fb0e406edb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb0f8036000 tx=0x7fb0f803c040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:03.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.105+0000 7fb101600640 1 -- 172.21.15.152:0/2238916174 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fb10414a460 con 0x7fb10415fcc0 2024-04-20T11:27:03.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.300+0000 7fb10b000640 1 -- 172.21.15.152:0/2238916174 --> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb1040b6ce0 con 0x7fb0e406c8f0 2024-04-20T11:27:03.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.322+0000 7fb101600640 1 -- 172.21.15.152:0/2238916174 <== mgr.4113 v2:172.21.15.152:6832/2285110240 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251948 (secure 0 0 0) 0x7fb1040b6ce0 con 0x7fb0e406c8f0 2024-04-20T11:27:03.326 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:27:03.327 INFO:teuthology.orchestra.run.smithi152.stderr:dumped all 2024-04-20T11:27:03.329 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.329+0000 7fb10b000640 1 -- 172.21.15.152:0/2238916174 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fb0e406c8f0 msgr2=0x7fb0e406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:03.329 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.329+0000 7fb10b000640 1 --2- 172.21.15.152:0/2238916174 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fb0e406c8f0 0x7fb0e406edb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb0f8036000 tx=0x7fb0f803c040 comp rx=0 tx=0).stop 2024-04-20T11:27:03.329 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.329+0000 7fb10b000640 1 -- 172.21.15.152:0/2238916174 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb10415fcc0 msgr2=0x7fb104149f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:03.329 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.329+0000 7fb10b000640 1 --2- 172.21.15.152:0/2238916174 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb10415fcc0 0x7fb104149f20 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb0f000c820 tx=0x7fb0f000cce0 comp rx=0 tx=0).stop 2024-04-20T11:27:03.329 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.329+0000 7fb10b000640 1 -- 172.21.15.152:0/2238916174 shutdown_connections 2024-04-20T11:27:03.329 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.329+0000 7fb10b000640 1 --2- 172.21.15.152:0/2238916174 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fb0e406c8f0 0x7fb0e406edb0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:03.329 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.329+0000 7fb10b000640 1 --2- 172.21.15.152:0/2238916174 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb1041603d0 0x7fb104147480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:03.329 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.329+0000 7fb10b000640 1 --2- 172.21.15.152:0/2238916174 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb10415fcc0 0x7fb104149f20 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:03.329 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.329+0000 7fb10b000640 1 --2- 172.21.15.152:0/2238916174 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1040b5480 0x7fb1040ca690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:03.329 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.329+0000 7fb10b000640 1 -- 172.21.15.152:0/2238916174 >> 172.21.15.152:0/2238916174 conn(0x7fb10408bfd0 msgr2=0x7fb1040d0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:03.329 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.329+0000 7fb10b000640 1 -- 172.21.15.152:0/2238916174 shutdown_connections 2024-04-20T11:27:03.329 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.329+0000 7fb10b000640 1 -- 172.21.15.152:0/2238916174 wait complete. 2024-04-20T11:27:03.347 INFO:teuthology.orchestra.run.smithi152.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-04-20T11:27:03.213056+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219000,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755720,"statfs":{"total":773094113280,"available":772869857280,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12746,"internal_metadata":221236790},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001225"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.185059+0000","last_change":"2024-04-20T11:26:58.706148+0000","last_active":"2024-04-20T11:27:00.185059+0000","last_peered":"2024-04-20T11:27:00.185059+0000","last_clean":"2024-04-20T11:27:00.185059+0000","last_became_active":"2024-04-20T11:26:58.705669+0000","last_became_peered":"2024-04-20T11:26:58.705669+0000","last_unstale":"2024-04-20T11:27:00.185059+0000","last_undegraded":"2024-04-20T11:27:00.185059+0000","last_fullsized":"2024-04-20T11:27:00.185059+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:25:02.202901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.187149+0000","last_change":"2024-04-20T11:26:58.688184+0000","last_active":"2024-04-20T11:27:00.187149+0000","last_peered":"2024-04-20T11:27:00.187149+0000","last_clean":"2024-04-20T11:27:00.187149+0000","last_became_active":"2024-04-20T11:26:58.685678+0000","last_became_peered":"2024-04-20T11:26:58.685678+0000","last_unstale":"2024-04-20T11:27:00.187149+0000","last_undegraded":"2024-04-20T11:27:00.187149+0000","last_fullsized":"2024-04-20T11:27:00.187149+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:30:01.981489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.692840+0000","last_change":"2024-04-20T11:26:58.675003+0000","last_active":"2024-04-20T11:26:59.692840+0000","last_peered":"2024-04-20T11:26:59.692840+0000","last_clean":"2024-04-20T11:26:59.692840+0000","last_became_active":"2024-04-20T11:26:58.674697+0000","last_became_peered":"2024-04-20T11:26:58.674697+0000","last_unstale":"2024-04-20T11:26:59.692840+0000","last_undegraded":"2024-04-20T11:26:59.692840+0000","last_fullsized":"2024-04-20T11:26:59.692840+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:48:49.870718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.697801+0000","last_change":"2024-04-20T11:26:58.678578+0000","last_active":"2024-04-20T11:26:59.697801+0000","last_peered":"2024-04-20T11:26:59.697801+0000","last_clean":"2024-04-20T11:26:59.697801+0000","last_became_active":"2024-04-20T11:26:58.677959+0000","last_became_peered":"2024-04-20T11:26:58.677959+0000","last_unstale":"2024-04-20T11:26:59.697801+0000","last_undegraded":"2024-04-20T11:26:59.697801+0000","last_fullsized":"2024-04-20T11:26:59.697801+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:27:35.003583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.185015+0000","last_change":"2024-04-20T11:26:58.703032+0000","last_active":"2024-04-20T11:27:00.185015+0000","last_peered":"2024-04-20T11:27:00.185015+0000","last_clean":"2024-04-20T11:27:00.185015+0000","last_became_active":"2024-04-20T11:26:58.702687+0000","last_became_peered":"2024-04-20T11:26:58.702687+0000","last_unstale":"2024-04-20T11:27:00.185015+0000","last_undegraded":"2024-04-20T11:27:00.185015+0000","last_fullsized":"2024-04-20T11:27:00.185015+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:35:38.308224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.698653+0000","last_change":"2024-04-20T11:26:58.692001+0000","last_active":"2024-04-20T11:26:59.698653+0000","last_peered":"2024-04-20T11:26:59.698653+0000","last_clean":"2024-04-20T11:26:59.698653+0000","last_became_active":"2024-04-20T11:26:58.689784+0000","last_became_peered":"2024-04-20T11:26:58.689784+0000","last_unstale":"2024-04-20T11:26:59.698653+0000","last_undegraded":"2024-04-20T11:26:59.698653+0000","last_fullsized":"2024-04-20T11:26:59.698653+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:35:38.308224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.184927+0000","last_change":"2024-04-20T11:26:58.700657+0000","last_active":"2024-04-20T11:27:00.184927+0000","last_peered":"2024-04-20T11:27:00.184927+0000","last_clean":"2024-04-20T11:27:00.184927+0000","last_became_active":"2024-04-20T11:26:58.700233+0000","last_became_peered":"2024-04-20T11:26:58.700233+0000","last_unstale":"2024-04-20T11:27:00.184927+0000","last_undegraded":"2024-04-20T11:27:00.184927+0000","last_fullsized":"2024-04-20T11:27:00.184927+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:40:22.418833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.183717+0000","last_change":"2024-04-20T11:26:58.691319+0000","last_active":"2024-04-20T11:27:00.183717+0000","last_peered":"2024-04-20T11:27:00.183717+0000","last_clean":"2024-04-20T11:27:00.183717+0000","last_became_active":"2024-04-20T11:26:58.690968+0000","last_became_peered":"2024-04-20T11:26:58.690968+0000","last_unstale":"2024-04-20T11:27:00.183717+0000","last_undegraded":"2024-04-20T11:27:00.183717+0000","last_fullsized":"2024-04-20T11:27:00.183717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:09:06.943774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.185058+0000","last_change":"2024-04-20T11:26:58.708815+0000","last_active":"2024-04-20T11:27:00.185058+0000","last_peered":"2024-04-20T11:27:00.185058+0000","last_clean":"2024-04-20T11:27:00.185058+0000","last_became_active":"2024-04-20T11:26:58.708527+0000","last_became_peered":"2024-04-20T11:26:58.708527+0000","last_unstale":"2024-04-20T11:27:00.185058+0000","last_undegraded":"2024-04-20T11:27:00.185058+0000","last_fullsized":"2024-04-20T11:27:00.185058+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:37:41.100182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.739853+0000","last_change":"2024-04-20T11:26:58.702277+0000","last_active":"2024-04-20T11:26:59.739853+0000","last_peered":"2024-04-20T11:26:59.739853+0000","last_clean":"2024-04-20T11:26:59.739853+0000","last_became_active":"2024-04-20T11:26:58.701978+0000","last_became_peered":"2024-04-20T11:26:58.701978+0000","last_unstale":"2024-04-20T11:26:59.739853+0000","last_undegraded":"2024-04-20T11:26:59.739853+0000","last_fullsized":"2024-04-20T11:26:59.739853+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:53:57.559834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.739719+0000","last_change":"2024-04-20T11:26:58.702193+0000","last_active":"2024-04-20T11:26:59.739719+0000","last_peered":"2024-04-20T11:26:59.739719+0000","last_clean":"2024-04-20T11:26:59.739719+0000","last_became_active":"2024-04-20T11:26:58.701865+0000","last_became_peered":"2024-04-20T11:26:58.701865+0000","last_unstale":"2024-04-20T11:26:59.739719+0000","last_undegraded":"2024-04-20T11:26:59.739719+0000","last_fullsized":"2024-04-20T11:26:59.739719+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:01:34.618021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.698023+0000","last_change":"2024-04-20T11:26:58.701101+0000","last_active":"2024-04-20T11:26:59.698023+0000","last_peered":"2024-04-20T11:26:59.698023+0000","last_clean":"2024-04-20T11:26:59.698023+0000","last_became_active":"2024-04-20T11:26:58.699917+0000","last_became_peered":"2024-04-20T11:26:58.699917+0000","last_unstale":"2024-04-20T11:26:59.698023+0000","last_undegraded":"2024-04-20T11:26:59.698023+0000","last_fullsized":"2024-04-20T11:26:59.698023+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:30:35.842066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.184787+0000","last_change":"2024-04-20T11:26:58.695384+0000","last_active":"2024-04-20T11:27:00.184787+0000","last_peered":"2024-04-20T11:27:00.184787+0000","last_clean":"2024-04-20T11:27:00.184787+0000","last_became_active":"2024-04-20T11:26:58.695086+0000","last_became_peered":"2024-04-20T11:26:58.695086+0000","last_unstale":"2024-04-20T11:27:00.184787+0000","last_undegraded":"2024-04-20T11:27:00.184787+0000","last_fullsized":"2024-04-20T11:27:00.184787+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:47:21.129095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.693173+0000","last_change":"2024-04-20T11:26:58.677865+0000","last_active":"2024-04-20T11:26:59.693173+0000","last_peered":"2024-04-20T11:26:59.693173+0000","last_clean":"2024-04-20T11:26:59.693173+0000","last_became_active":"2024-04-20T11:26:58.677541+0000","last_became_peered":"2024-04-20T11:26:58.677541+0000","last_unstale":"2024-04-20T11:26:59.693173+0000","last_undegraded":"2024-04-20T11:26:59.693173+0000","last_fullsized":"2024-04-20T11:26:59.693173+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:47:21.129095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.186987+0000","last_change":"2024-04-20T11:26:58.693039+0000","last_active":"2024-04-20T11:27:00.186987+0000","last_peered":"2024-04-20T11:27:00.186987+0000","last_clean":"2024-04-20T11:27:00.186987+0000","last_became_active":"2024-04-20T11:26:58.692020+0000","last_became_peered":"2024-04-20T11:26:58.692020+0000","last_unstale":"2024-04-20T11:27:00.186987+0000","last_undegraded":"2024-04-20T11:27:00.186987+0000","last_fullsized":"2024-04-20T11:27:00.186987+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:35:40.422299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.693509+0000","last_change":"2024-04-20T11:26:58.686574+0000","last_active":"2024-04-20T11:26:59.693509+0000","last_peered":"2024-04-20T11:26:59.693509+0000","last_clean":"2024-04-20T11:26:59.693509+0000","last_became_active":"2024-04-20T11:26:58.686297+0000","last_became_peered":"2024-04-20T11:26:58.686297+0000","last_unstale":"2024-04-20T11:26:59.693509+0000","last_undegraded":"2024-04-20T11:26:59.693509+0000","last_fullsized":"2024-04-20T11:26:59.693509+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:01:34.618021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.695098+0000","last_change":"2024-04-20T11:26:58.701917+0000","last_active":"2024-04-20T11:26:59.695098+0000","last_peered":"2024-04-20T11:26:59.695098+0000","last_clean":"2024-04-20T11:26:59.695098+0000","last_became_active":"2024-04-20T11:26:58.701625+0000","last_became_peered":"2024-04-20T11:26:58.701625+0000","last_unstale":"2024-04-20T11:26:59.695098+0000","last_undegraded":"2024-04-20T11:26:59.695098+0000","last_fullsized":"2024-04-20T11:26:59.695098+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:35:40.422299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.185755+0000","last_change":"2024-04-20T11:26:58.697089+0000","last_active":"2024-04-20T11:27:00.185755+0000","last_peered":"2024-04-20T11:27:00.185755+0000","last_clean":"2024-04-20T11:27:00.185755+0000","last_became_active":"2024-04-20T11:26:58.696794+0000","last_became_peered":"2024-04-20T11:26:58.696794+0000","last_unstale":"2024-04-20T11:27:00.185755+0000","last_undegraded":"2024-04-20T11:27:00.185755+0000","last_fullsized":"2024-04-20T11:27:00.185755+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:30:35.842066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.184901+0000","last_change":"2024-04-20T11:26:58.707870+0000","last_active":"2024-04-20T11:27:00.184901+0000","last_peered":"2024-04-20T11:27:00.184901+0000","last_clean":"2024-04-20T11:27:00.184901+0000","last_became_active":"2024-04-20T11:26:58.702746+0000","last_became_peered":"2024-04-20T11:26:58.702746+0000","last_unstale":"2024-04-20T11:27:00.184901+0000","last_undegraded":"2024-04-20T11:27:00.184901+0000","last_fullsized":"2024-04-20T11:27:00.184901+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:28:57.233788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.739521+0000","last_change":"2024-04-20T11:26:58.707789+0000","last_active":"2024-04-20T11:26:59.739521+0000","last_peered":"2024-04-20T11:26:59.739521+0000","last_clean":"2024-04-20T11:26:59.739521+0000","last_became_active":"2024-04-20T11:26:58.707272+0000","last_became_peered":"2024-04-20T11:26:58.707272+0000","last_unstale":"2024-04-20T11:26:59.739521+0000","last_undegraded":"2024-04-20T11:26:59.739521+0000","last_fullsized":"2024-04-20T11:26:59.739521+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:01:06.520823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.692973+0000","last_change":"2024-04-20T11:26:58.698142+0000","last_active":"2024-04-20T11:26:59.692973+0000","last_peered":"2024-04-20T11:26:59.692973+0000","last_clean":"2024-04-20T11:26:59.692973+0000","last_became_active":"2024-04-20T11:26:58.697869+0000","last_became_peered":"2024-04-20T11:26:58.697869+0000","last_unstale":"2024-04-20T11:26:59.692973+0000","last_undegraded":"2024-04-20T11:26:59.692973+0000","last_fullsized":"2024-04-20T11:26:59.692973+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:43:20.032648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.696504+0000","last_change":"2024-04-20T11:26:58.692191+0000","last_active":"2024-04-20T11:26:59.696504+0000","last_peered":"2024-04-20T11:26:59.696504+0000","last_clean":"2024-04-20T11:26:59.696504+0000","last_became_active":"2024-04-20T11:26:58.690512+0000","last_became_peered":"2024-04-20T11:26:58.690512+0000","last_unstale":"2024-04-20T11:26:59.696504+0000","last_undegraded":"2024-04-20T11:26:59.696504+0000","last_fullsized":"2024-04-20T11:26:59.696504+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:01:48.311110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.185102+0000","last_change":"2024-04-20T11:26:58.693404+0000","last_active":"2024-04-20T11:27:00.185102+0000","last_peered":"2024-04-20T11:27:00.185102+0000","last_clean":"2024-04-20T11:27:00.185102+0000","last_became_active":"2024-04-20T11:26:58.693146+0000","last_became_peered":"2024-04-20T11:26:58.693146+0000","last_unstale":"2024-04-20T11:27:00.185102+0000","last_undegraded":"2024-04-20T11:27:00.185102+0000","last_fullsized":"2024-04-20T11:27:00.185102+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:01:34.618021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.693752+0000","last_change":"2024-04-20T11:26:58.682815+0000","last_active":"2024-04-20T11:26:59.693752+0000","last_peered":"2024-04-20T11:26:59.693752+0000","last_clean":"2024-04-20T11:26:59.693752+0000","last_became_active":"2024-04-20T11:26:58.679077+0000","last_became_peered":"2024-04-20T11:26:58.679077+0000","last_unstale":"2024-04-20T11:26:59.693752+0000","last_undegraded":"2024-04-20T11:26:59.693752+0000","last_fullsized":"2024-04-20T11:26:59.693752+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:53:57.559834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.186319+0000","last_change":"2024-04-20T11:26:58.706958+0000","last_active":"2024-04-20T11:27:00.186319+0000","last_peered":"2024-04-20T11:27:00.186319+0000","last_clean":"2024-04-20T11:27:00.186319+0000","last_became_active":"2024-04-20T11:26:58.706682+0000","last_became_peered":"2024-04-20T11:26:58.706682+0000","last_unstale":"2024-04-20T11:27:00.186319+0000","last_undegraded":"2024-04-20T11:27:00.186319+0000","last_fullsized":"2024-04-20T11:27:00.186319+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:40:45.000805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.739905+0000","last_change":"2024-04-20T11:26:58.693542+0000","last_active":"2024-04-20T11:26:59.739905+0000","last_peered":"2024-04-20T11:26:59.739905+0000","last_clean":"2024-04-20T11:26:59.739905+0000","last_became_active":"2024-04-20T11:26:58.693091+0000","last_became_peered":"2024-04-20T11:26:58.693091+0000","last_unstale":"2024-04-20T11:26:59.739905+0000","last_undegraded":"2024-04-20T11:26:59.739905+0000","last_fullsized":"2024-04-20T11:26:59.739905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:35:40.422299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.186334+0000","last_change":"2024-04-20T11:26:58.683272+0000","last_active":"2024-04-20T11:27:00.186334+0000","last_peered":"2024-04-20T11:27:00.186334+0000","last_clean":"2024-04-20T11:27:00.186334+0000","last_became_active":"2024-04-20T11:26:58.683012+0000","last_became_peered":"2024-04-20T11:26:58.683012+0000","last_unstale":"2024-04-20T11:27:00.186334+0000","last_undegraded":"2024-04-20T11:27:00.186334+0000","last_fullsized":"2024-04-20T11:27:00.186334+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:53:57.559834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.185138+0000","last_change":"2024-04-20T11:26:58.703756+0000","last_active":"2024-04-20T11:27:00.185138+0000","last_peered":"2024-04-20T11:27:00.185138+0000","last_clean":"2024-04-20T11:27:00.185138+0000","last_became_active":"2024-04-20T11:26:58.703323+0000","last_became_peered":"2024-04-20T11:26:58.703323+0000","last_unstale":"2024-04-20T11:27:00.185138+0000","last_undegraded":"2024-04-20T11:27:00.185138+0000","last_fullsized":"2024-04-20T11:27:00.185138+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:53:57.559834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.692748+0000","last_change":"2024-04-20T11:26:58.681550+0000","last_active":"2024-04-20T11:26:59.692748+0000","last_peered":"2024-04-20T11:26:59.692748+0000","last_clean":"2024-04-20T11:26:59.692748+0000","last_became_active":"2024-04-20T11:26:58.681270+0000","last_became_peered":"2024-04-20T11:26:58.681270+0000","last_unstale":"2024-04-20T11:26:59.692748+0000","last_undegraded":"2024-04-20T11:26:59.692748+0000","last_fullsized":"2024-04-20T11:26:59.692748+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:39:04.516141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.696801+0000","last_change":"2024-04-20T11:26:58.682629+0000","last_active":"2024-04-20T11:26:59.696801+0000","last_peered":"2024-04-20T11:26:59.696801+0000","last_clean":"2024-04-20T11:26:59.696801+0000","last_became_active":"2024-04-20T11:26:58.680875+0000","last_became_peered":"2024-04-20T11:26:58.680875+0000","last_unstale":"2024-04-20T11:26:59.696801+0000","last_undegraded":"2024-04-20T11:26:59.696801+0000","last_fullsized":"2024-04-20T11:26:59.696801+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:12:41.134688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.183098+0000","last_change":"2024-04-20T11:26:58.702006+0000","last_active":"2024-04-20T11:27:00.183098+0000","last_peered":"2024-04-20T11:27:00.183098+0000","last_clean":"2024-04-20T11:27:00.183098+0000","last_became_active":"2024-04-20T11:26:58.701697+0000","last_became_peered":"2024-04-20T11:26:58.701697+0000","last_unstale":"2024-04-20T11:27:00.183098+0000","last_undegraded":"2024-04-20T11:27:00.183098+0000","last_fullsized":"2024-04-20T11:27:00.183098+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:56:23.896778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.738970+0000","last_change":"2024-04-20T11:26:58.695448+0000","last_active":"2024-04-20T11:26:59.738970+0000","last_peered":"2024-04-20T11:26:59.738970+0000","last_clean":"2024-04-20T11:26:59.738970+0000","last_became_active":"2024-04-20T11:26:58.694946+0000","last_became_peered":"2024-04-20T11:26:58.694946+0000","last_unstale":"2024-04-20T11:26:59.738970+0000","last_undegraded":"2024-04-20T11:26:59.738970+0000","last_fullsized":"2024-04-20T11:26:59.738970+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:02:15.732106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.696114+0000","last_change":"2024-04-20T11:26:58.697053+0000","last_active":"2024-04-20T11:26:59.696114+0000","last_peered":"2024-04-20T11:26:59.696114+0000","last_clean":"2024-04-20T11:26:59.696114+0000","last_became_active":"2024-04-20T11:26:58.696713+0000","last_became_peered":"2024-04-20T11:26:58.696713+0000","last_unstale":"2024-04-20T11:26:59.696114+0000","last_undegraded":"2024-04-20T11:26:59.696114+0000","last_fullsized":"2024-04-20T11:26:59.696114+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:22:15.505173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.185319+0000","last_change":"2024-04-20T11:26:57.680979+0000","last_active":"2024-04-20T11:27:00.185319+0000","last_peered":"2024-04-20T11:27:00.185319+0000","last_clean":"2024-04-20T11:27:00.185319+0000","last_became_active":"2024-04-20T11:26:57.679239+0000","last_became_peered":"2024-04-20T11:26:57.679239+0000","last_unstale":"2024-04-20T11:27:00.185319+0000","last_undegraded":"2024-04-20T11:27:00.185319+0000","last_fullsized":"2024-04-20T11:27:00.185319+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:03:42.590011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.692914+0000","last_change":"2024-04-20T11:26:58.693305+0000","last_active":"2024-04-20T11:26:59.692914+0000","last_peered":"2024-04-20T11:26:59.692914+0000","last_clean":"2024-04-20T11:26:59.692914+0000","last_became_active":"2024-04-20T11:26:58.692561+0000","last_became_peered":"2024-04-20T11:26:58.692561+0000","last_unstale":"2024-04-20T11:26:59.692914+0000","last_undegraded":"2024-04-20T11:26:59.692914+0000","last_fullsized":"2024-04-20T11:26:59.692914+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:09:06.943774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.738687+0000","last_change":"2024-04-20T11:26:57.696531+0000","last_active":"2024-04-20T11:26:59.738687+0000","last_peered":"2024-04-20T11:26:59.738687+0000","last_clean":"2024-04-20T11:26:59.738687+0000","last_became_active":"2024-04-20T11:26:57.693505+0000","last_became_peered":"2024-04-20T11:26:57.693505+0000","last_unstale":"2024-04-20T11:26:59.738687+0000","last_undegraded":"2024-04-20T11:26:59.738687+0000","last_fullsized":"2024-04-20T11:26:59.738687+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:22:14.490133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.185207+0000","last_change":"2024-04-20T11:26:57.700780+0000","last_active":"2024-04-20T11:27:00.185207+0000","last_peered":"2024-04-20T11:27:00.185207+0000","last_clean":"2024-04-20T11:27:00.185207+0000","last_became_active":"2024-04-20T11:26:57.700475+0000","last_became_peered":"2024-04-20T11:26:57.700475+0000","last_unstale":"2024-04-20T11:27:00.185207+0000","last_undegraded":"2024-04-20T11:27:00.185207+0000","last_fullsized":"2024-04-20T11:27:00.185207+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:35:40.489733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.183864+0000","last_change":"2024-04-20T11:26:58.695758+0000","last_active":"2024-04-20T11:27:00.183864+0000","last_peered":"2024-04-20T11:27:00.183864+0000","last_clean":"2024-04-20T11:27:00.183864+0000","last_became_active":"2024-04-20T11:26:58.695496+0000","last_became_peered":"2024-04-20T11:26:58.695496+0000","last_unstale":"2024-04-20T11:27:00.183864+0000","last_undegraded":"2024-04-20T11:27:00.183864+0000","last_fullsized":"2024-04-20T11:27:00.183864+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:43:20.032648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.693649+0000","last_change":"2024-04-20T11:26:57.695300+0000","last_active":"2024-04-20T11:26:59.693649+0000","last_peered":"2024-04-20T11:26:59.693649+0000","last_clean":"2024-04-20T11:26:59.693649+0000","last_became_active":"2024-04-20T11:26:57.694343+0000","last_became_peered":"2024-04-20T11:26:57.694343+0000","last_unstale":"2024-04-20T11:26:59.693649+0000","last_undegraded":"2024-04-20T11:26:59.693649+0000","last_fullsized":"2024-04-20T11:26:59.693649+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:03:42.590011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.183738+0000","last_change":"2024-04-20T11:26:58.695911+0000","last_active":"2024-04-20T11:27:00.183738+0000","last_peered":"2024-04-20T11:27:00.183738+0000","last_clean":"2024-04-20T11:27:00.183738+0000","last_became_active":"2024-04-20T11:26:58.695607+0000","last_became_peered":"2024-04-20T11:26:58.695607+0000","last_unstale":"2024-04-20T11:27:00.183738+0000","last_undegraded":"2024-04-20T11:27:00.183738+0000","last_fullsized":"2024-04-20T11:27:00.183738+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:47:21.129095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.739639+0000","last_change":"2024-04-20T11:26:57.692397+0000","last_active":"2024-04-20T11:26:59.739639+0000","last_peered":"2024-04-20T11:26:59.739639+0000","last_clean":"2024-04-20T11:26:59.739639+0000","last_became_active":"2024-04-20T11:26:57.692046+0000","last_became_peered":"2024-04-20T11:26:57.692046+0000","last_unstale":"2024-04-20T11:26:59.739639+0000","last_undegraded":"2024-04-20T11:26:59.739639+0000","last_fullsized":"2024-04-20T11:26:59.739639+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:11:19.111199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.695485+0000","last_change":"2024-04-20T11:26:58.691351+0000","last_active":"2024-04-20T11:26:59.695485+0000","last_peered":"2024-04-20T11:26:59.695485+0000","last_clean":"2024-04-20T11:26:59.695485+0000","last_became_active":"2024-04-20T11:26:58.691059+0000","last_became_peered":"2024-04-20T11:26:58.691059+0000","last_unstale":"2024-04-20T11:26:59.695485+0000","last_undegraded":"2024-04-20T11:26:59.695485+0000","last_fullsized":"2024-04-20T11:26:59.695485+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:40:45.000805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.694315+0000","last_change":"2024-04-20T11:26:57.699532+0000","last_active":"2024-04-20T11:26:59.694315+0000","last_peered":"2024-04-20T11:26:59.694315+0000","last_clean":"2024-04-20T11:26:59.694315+0000","last_became_active":"2024-04-20T11:26:57.699281+0000","last_became_peered":"2024-04-20T11:26:57.699281+0000","last_unstale":"2024-04-20T11:26:59.694315+0000","last_undegraded":"2024-04-20T11:26:59.694315+0000","last_fullsized":"2024-04-20T11:26:59.694315+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:22:14.490133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.693433+0000","last_change":"2024-04-20T11:26:58.692727+0000","last_active":"2024-04-20T11:26:59.693433+0000","last_peered":"2024-04-20T11:26:59.693433+0000","last_clean":"2024-04-20T11:26:59.693433+0000","last_became_active":"2024-04-20T11:26:58.692415+0000","last_became_peered":"2024-04-20T11:26:58.692415+0000","last_unstale":"2024-04-20T11:26:59.693433+0000","last_undegraded":"2024-04-20T11:26:59.693433+0000","last_fullsized":"2024-04-20T11:26:59.693433+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:01:34.618021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.186405+0000","last_change":"2024-04-20T11:26:57.681417+0000","last_active":"2024-04-20T11:27:00.186405+0000","last_peered":"2024-04-20T11:27:00.186405+0000","last_clean":"2024-04-20T11:27:00.186405+0000","last_became_active":"2024-04-20T11:26:57.680971+0000","last_became_peered":"2024-04-20T11:26:57.680971+0000","last_unstale":"2024-04-20T11:27:00.186405+0000","last_undegraded":"2024-04-20T11:27:00.186405+0000","last_fullsized":"2024-04-20T11:27:00.186405+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:09:13.468036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.698358+0000","last_change":"2024-04-20T11:26:58.710488+0000","last_active":"2024-04-20T11:26:59.698358+0000","last_peered":"2024-04-20T11:26:59.698358+0000","last_clean":"2024-04-20T11:26:59.698358+0000","last_became_active":"2024-04-20T11:26:58.710179+0000","last_became_peered":"2024-04-20T11:26:58.710179+0000","last_unstale":"2024-04-20T11:26:59.698358+0000","last_undegraded":"2024-04-20T11:26:59.698358+0000","last_fullsized":"2024-04-20T11:26:59.698358+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:09:14.483077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.739503+0000","last_change":"2024-04-20T11:26:57.700540+0000","last_active":"2024-04-20T11:26:59.739503+0000","last_peered":"2024-04-20T11:26:59.739503+0000","last_clean":"2024-04-20T11:26:59.739503+0000","last_became_active":"2024-04-20T11:26:57.697881+0000","last_became_peered":"2024-04-20T11:26:57.697881+0000","last_unstale":"2024-04-20T11:26:59.739503+0000","last_undegraded":"2024-04-20T11:26:59.739503+0000","last_fullsized":"2024-04-20T11:26:59.739503+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:28:39.881689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.184257+0000","last_change":"2024-04-20T11:26:58.710228+0000","last_active":"2024-04-20T11:27:00.184257+0000","last_peered":"2024-04-20T11:27:00.184257+0000","last_clean":"2024-04-20T11:27:00.184257+0000","last_became_active":"2024-04-20T11:26:58.709942+0000","last_became_peered":"2024-04-20T11:26:58.709942+0000","last_unstale":"2024-04-20T11:27:00.184257+0000","last_undegraded":"2024-04-20T11:27:00.184257+0000","last_fullsized":"2024-04-20T11:27:00.184257+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:01:48.311110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.692978+0000","last_change":"2024-04-20T11:26:57.691897+0000","last_active":"2024-04-20T11:26:59.692978+0000","last_peered":"2024-04-20T11:26:59.692978+0000","last_clean":"2024-04-20T11:26:59.692978+0000","last_became_active":"2024-04-20T11:26:57.690649+0000","last_became_peered":"2024-04-20T11:26:57.690649+0000","last_unstale":"2024-04-20T11:26:59.692978+0000","last_undegraded":"2024-04-20T11:26:59.692978+0000","last_fullsized":"2024-04-20T11:26:59.692978+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:11:36.407427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.693551+0000","last_change":"2024-04-20T11:26:58.696119+0000","last_active":"2024-04-20T11:26:59.693551+0000","last_peered":"2024-04-20T11:26:59.693551+0000","last_clean":"2024-04-20T11:26:59.693551+0000","last_became_active":"2024-04-20T11:26:58.695849+0000","last_became_peered":"2024-04-20T11:26:58.695849+0000","last_unstale":"2024-04-20T11:26:59.693551+0000","last_undegraded":"2024-04-20T11:26:59.693551+0000","last_fullsized":"2024-04-20T11:26:59.693551+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:53:57.559834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.696904+0000","last_change":"2024-04-20T11:26:57.693789+0000","last_active":"2024-04-20T11:26:59.696904+0000","last_peered":"2024-04-20T11:26:59.696904+0000","last_clean":"2024-04-20T11:26:59.696904+0000","last_became_active":"2024-04-20T11:26:57.688385+0000","last_became_peered":"2024-04-20T11:26:57.688385+0000","last_unstale":"2024-04-20T11:26:59.696904+0000","last_undegraded":"2024-04-20T11:26:59.696904+0000","last_fullsized":"2024-04-20T11:26:59.696904+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:59:13.454290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.185068+0000","last_change":"2024-04-20T11:26:58.713408+0000","last_active":"2024-04-20T11:27:00.185068+0000","last_peered":"2024-04-20T11:27:00.185068+0000","last_clean":"2024-04-20T11:27:00.185068+0000","last_became_active":"2024-04-20T11:26:58.713119+0000","last_became_peered":"2024-04-20T11:26:58.713119+0000","last_unstale":"2024-04-20T11:27:00.185068+0000","last_undegraded":"2024-04-20T11:27:00.185068+0000","last_fullsized":"2024-04-20T11:27:00.185068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:12:24.274916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.738625+0000","last_change":"2024-04-20T11:26:57.695426+0000","last_active":"2024-04-20T11:26:59.738625+0000","last_peered":"2024-04-20T11:26:59.738625+0000","last_clean":"2024-04-20T11:26:59.738625+0000","last_became_active":"2024-04-20T11:26:57.695023+0000","last_became_peered":"2024-04-20T11:26:57.695023+0000","last_unstale":"2024-04-20T11:26:59.738625+0000","last_undegraded":"2024-04-20T11:26:59.738625+0000","last_fullsized":"2024-04-20T11:26:59.738625+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:03:42.590011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.739012+0000","last_change":"2024-04-20T11:26:58.695228+0000","last_active":"2024-04-20T11:26:59.739012+0000","last_peered":"2024-04-20T11:26:59.739012+0000","last_clean":"2024-04-20T11:26:59.739012+0000","last_became_active":"2024-04-20T11:26:58.694540+0000","last_became_peered":"2024-04-20T11:26:58.694540+0000","last_unstale":"2024-04-20T11:26:59.739012+0000","last_undegraded":"2024-04-20T11:26:59.739012+0000","last_fullsized":"2024-04-20T11:26:59.739012+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:47:21.129095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.182683+0000","last_change":"2024-04-20T11:26:57.679351+0000","last_active":"2024-04-20T11:27:00.182683+0000","last_peered":"2024-04-20T11:27:00.182683+0000","last_clean":"2024-04-20T11:27:00.182683+0000","last_became_active":"2024-04-20T11:26:57.678821+0000","last_became_peered":"2024-04-20T11:26:57.678821+0000","last_unstale":"2024-04-20T11:27:00.182683+0000","last_undegraded":"2024-04-20T11:27:00.182683+0000","last_fullsized":"2024-04-20T11:27:00.182683+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:28:39.881689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.184115+0000","last_change":"2024-04-20T11:26:58.713370+0000","last_active":"2024-04-20T11:27:00.184115+0000","last_peered":"2024-04-20T11:27:00.184115+0000","last_clean":"2024-04-20T11:27:00.184115+0000","last_became_active":"2024-04-20T11:26:58.713068+0000","last_became_peered":"2024-04-20T11:26:58.713068+0000","last_unstale":"2024-04-20T11:27:00.184115+0000","last_undegraded":"2024-04-20T11:27:00.184115+0000","last_fullsized":"2024-04-20T11:27:00.184115+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:55:47.019482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.695270+0000","last_change":"2024-04-20T11:26:57.687641+0000","last_active":"2024-04-20T11:26:59.695270+0000","last_peered":"2024-04-20T11:26:59.695270+0000","last_clean":"2024-04-20T11:26:59.695270+0000","last_became_active":"2024-04-20T11:26:57.687237+0000","last_became_peered":"2024-04-20T11:26:57.687237+0000","last_unstale":"2024-04-20T11:26:59.695270+0000","last_undegraded":"2024-04-20T11:26:59.695270+0000","last_fullsized":"2024-04-20T11:26:59.695270+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:43:12.722459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.184814+0000","last_change":"2024-04-20T11:26:58.711133+0000","last_active":"2024-04-20T11:27:00.184814+0000","last_peered":"2024-04-20T11:27:00.184814+0000","last_clean":"2024-04-20T11:27:00.184814+0000","last_became_active":"2024-04-20T11:26:58.710818+0000","last_became_peered":"2024-04-20T11:26:58.710818+0000","last_unstale":"2024-04-20T11:27:00.184814+0000","last_undegraded":"2024-04-20T11:27:00.184814+0000","last_fullsized":"2024-04-20T11:27:00.184814+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:27:35.003583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.185902+0000","last_change":"2024-04-20T11:26:57.675885+0000","last_active":"2024-04-20T11:27:00.185902+0000","last_peered":"2024-04-20T11:27:00.185902+0000","last_clean":"2024-04-20T11:27:00.185902+0000","last_became_active":"2024-04-20T11:26:57.675625+0000","last_became_peered":"2024-04-20T11:26:57.675625+0000","last_unstale":"2024-04-20T11:27:00.185902+0000","last_undegraded":"2024-04-20T11:27:00.185902+0000","last_fullsized":"2024-04-20T11:27:00.185902+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:03:42.590011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.185312+0000","last_change":"2024-04-20T11:26:58.713427+0000","last_active":"2024-04-20T11:27:00.185312+0000","last_peered":"2024-04-20T11:27:00.185312+0000","last_clean":"2024-04-20T11:27:00.185312+0000","last_became_active":"2024-04-20T11:26:58.713112+0000","last_became_peered":"2024-04-20T11:26:58.713112+0000","last_unstale":"2024-04-20T11:27:00.185312+0000","last_undegraded":"2024-04-20T11:27:00.185312+0000","last_fullsized":"2024-04-20T11:27:00.185312+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:26:45.461271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.186539+0000","last_change":"2024-04-20T11:26:57.693246+0000","last_active":"2024-04-20T11:27:00.186539+0000","last_peered":"2024-04-20T11:27:00.186539+0000","last_clean":"2024-04-20T11:27:00.186539+0000","last_became_active":"2024-04-20T11:26:57.692699+0000","last_became_peered":"2024-04-20T11:26:57.692699+0000","last_unstale":"2024-04-20T11:27:00.186539+0000","last_undegraded":"2024-04-20T11:27:00.186539+0000","last_fullsized":"2024-04-20T11:27:00.186539+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:22:14.490133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.698394+0000","last_change":"2024-04-20T11:26:58.676238+0000","last_active":"2024-04-20T11:26:59.698394+0000","last_peered":"2024-04-20T11:26:59.698394+0000","last_clean":"2024-04-20T11:26:59.698394+0000","last_became_active":"2024-04-20T11:26:58.675427+0000","last_became_peered":"2024-04-20T11:26:58.675427+0000","last_unstale":"2024-04-20T11:26:59.698394+0000","last_undegraded":"2024-04-20T11:26:59.698394+0000","last_fullsized":"2024-04-20T11:26:59.698394+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:28:57.233788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.185685+0000","last_change":"2024-04-20T11:26:57.681076+0000","last_active":"2024-04-20T11:27:00.185685+0000","last_peered":"2024-04-20T11:27:00.185685+0000","last_clean":"2024-04-20T11:27:00.185685+0000","last_became_active":"2024-04-20T11:26:57.680806+0000","last_became_peered":"2024-04-20T11:26:57.680806+0000","last_unstale":"2024-04-20T11:27:00.185685+0000","last_undegraded":"2024-04-20T11:27:00.185685+0000","last_fullsized":"2024-04-20T11:27:00.185685+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:11:19.111199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.184412+0000","last_change":"2024-04-20T11:26:58.708666+0000","last_active":"2024-04-20T11:27:00.184412+0000","last_peered":"2024-04-20T11:27:00.184412+0000","last_clean":"2024-04-20T11:27:00.184412+0000","last_became_active":"2024-04-20T11:26:58.708362+0000","last_became_peered":"2024-04-20T11:26:58.708362+0000","last_unstale":"2024-04-20T11:27:00.184412+0000","last_undegraded":"2024-04-20T11:27:00.184412+0000","last_fullsized":"2024-04-20T11:27:00.184412+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:12:41.134688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.184117+0000","last_change":"2024-04-20T11:26:48.661897+0000","last_active":"2024-04-20T11:27:00.184117+0000","last_peered":"2024-04-20T11:27:00.184117+0000","last_clean":"2024-04-20T11:27:00.184117+0000","last_became_active":"2024-04-20T11:26:48.656535+0000","last_became_peered":"2024-04-20T11:26:48.656535+0000","last_unstale":"2024-04-20T11:27:00.184117+0000","last_undegraded":"2024-04-20T11:27:00.184117+0000","last_fullsized":"2024-04-20T11:27:00.184117+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:47.543004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:47.543004+0000","last_clean_scrub_stamp":"2024-04-20T11:26:47.543004+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:58:35.527105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.694066+0000","last_change":"2024-04-20T11:26:57.680902+0000","last_active":"2024-04-20T11:26:59.694066+0000","last_peered":"2024-04-20T11:26:59.694066+0000","last_clean":"2024-04-20T11:26:59.694066+0000","last_became_active":"2024-04-20T11:26:57.680506+0000","last_became_peered":"2024-04-20T11:26:57.680506+0000","last_unstale":"2024-04-20T11:26:59.694066+0000","last_undegraded":"2024-04-20T11:26:59.694066+0000","last_fullsized":"2024-04-20T11:26:59.694066+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:28:39.881689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.185222+0000","last_change":"2024-04-20T11:26:58.697905+0000","last_active":"2024-04-20T11:27:00.185222+0000","last_peered":"2024-04-20T11:27:00.185222+0000","last_clean":"2024-04-20T11:27:00.185222+0000","last_became_active":"2024-04-20T11:26:58.697612+0000","last_became_peered":"2024-04-20T11:26:58.697612+0000","last_unstale":"2024-04-20T11:27:00.185222+0000","last_undegraded":"2024-04-20T11:27:00.185222+0000","last_fullsized":"2024-04-20T11:27:00.185222+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:30:01.981489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.183754+0000","last_change":"2024-04-20T11:26:57.697322+0000","last_active":"2024-04-20T11:27:00.183754+0000","last_peered":"2024-04-20T11:27:00.183754+0000","last_clean":"2024-04-20T11:27:00.183754+0000","last_became_active":"2024-04-20T11:26:57.696775+0000","last_became_peered":"2024-04-20T11:26:57.696775+0000","last_unstale":"2024-04-20T11:27:00.183754+0000","last_undegraded":"2024-04-20T11:27:00.183754+0000","last_fullsized":"2024-04-20T11:27:00.183754+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:30:34.827026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.692465+0000","last_change":"2024-04-20T11:26:58.690752+0000","last_active":"2024-04-20T11:26:59.692465+0000","last_peered":"2024-04-20T11:26:59.692465+0000","last_clean":"2024-04-20T11:26:59.692465+0000","last_became_active":"2024-04-20T11:26:58.690451+0000","last_became_peered":"2024-04-20T11:26:58.690451+0000","last_unstale":"2024-04-20T11:26:59.692465+0000","last_undegraded":"2024-04-20T11:26:59.692465+0000","last_fullsized":"2024-04-20T11:26:59.692465+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:33:15.328953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.184689+0000","last_change":"2024-04-20T11:26:57.680139+0000","last_active":"2024-04-20T11:27:00.184689+0000","last_peered":"2024-04-20T11:27:00.184689+0000","last_clean":"2024-04-20T11:27:00.184689+0000","last_became_active":"2024-04-20T11:26:57.679820+0000","last_became_peered":"2024-04-20T11:26:57.679820+0000","last_unstale":"2024-04-20T11:27:00.184689+0000","last_undegraded":"2024-04-20T11:27:00.184689+0000","last_fullsized":"2024-04-20T11:27:00.184689+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:35:40.489733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.184019+0000","last_change":"2024-04-20T11:26:58.677911+0000","last_active":"2024-04-20T11:27:00.184019+0000","last_peered":"2024-04-20T11:27:00.184019+0000","last_clean":"2024-04-20T11:27:00.184019+0000","last_became_active":"2024-04-20T11:26:58.677645+0000","last_became_peered":"2024-04-20T11:26:58.677645+0000","last_unstale":"2024-04-20T11:27:00.184019+0000","last_undegraded":"2024-04-20T11:27:00.184019+0000","last_fullsized":"2024-04-20T11:27:00.184019+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:01:06.520823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.697092+0000","last_change":"2024-04-20T11:26:57.676331+0000","last_active":"2024-04-20T11:26:59.697092+0000","last_peered":"2024-04-20T11:26:59.697092+0000","last_clean":"2024-04-20T11:26:59.697092+0000","last_became_active":"2024-04-20T11:26:57.675643+0000","last_became_peered":"2024-04-20T11:26:57.675643+0000","last_unstale":"2024-04-20T11:26:59.697092+0000","last_undegraded":"2024-04-20T11:26:59.697092+0000","last_fullsized":"2024-04-20T11:26:59.697092+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:11:19.111199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.697040+0000","last_change":"2024-04-20T11:26:58.701514+0000","last_active":"2024-04-20T11:26:59.697040+0000","last_peered":"2024-04-20T11:26:59.697040+0000","last_clean":"2024-04-20T11:26:59.697040+0000","last_became_active":"2024-04-20T11:26:58.692435+0000","last_became_peered":"2024-04-20T11:26:58.692435+0000","last_unstale":"2024-04-20T11:26:59.697040+0000","last_undegraded":"2024-04-20T11:26:59.697040+0000","last_fullsized":"2024-04-20T11:26:59.697040+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:56:23.896778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.185970+0000","last_change":"2024-04-20T11:26:57.681965+0000","last_active":"2024-04-20T11:27:00.185970+0000","last_peered":"2024-04-20T11:27:00.185970+0000","last_clean":"2024-04-20T11:27:00.185970+0000","last_became_active":"2024-04-20T11:26:57.681130+0000","last_became_peered":"2024-04-20T11:26:57.681130+0000","last_unstale":"2024-04-20T11:27:00.185970+0000","last_undegraded":"2024-04-20T11:27:00.185970+0000","last_fullsized":"2024-04-20T11:27:00.185970+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:59:13.454290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.695098+0000","last_change":"2024-04-20T11:26:58.687674+0000","last_active":"2024-04-20T11:26:59.695098+0000","last_peered":"2024-04-20T11:26:59.695098+0000","last_clean":"2024-04-20T11:26:59.695098+0000","last_became_active":"2024-04-20T11:26:58.687373+0000","last_became_peered":"2024-04-20T11:26:58.687373+0000","last_unstale":"2024-04-20T11:26:59.695098+0000","last_undegraded":"2024-04-20T11:26:59.695098+0000","last_fullsized":"2024-04-20T11:26:59.695098+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:30:01.981489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.184744+0000","last_change":"2024-04-20T11:26:57.700654+0000","last_active":"2024-04-20T11:27:00.184744+0000","last_peered":"2024-04-20T11:27:00.184744+0000","last_clean":"2024-04-20T11:27:00.184744+0000","last_became_active":"2024-04-20T11:26:57.698785+0000","last_became_peered":"2024-04-20T11:26:57.698785+0000","last_unstale":"2024-04-20T11:27:00.184744+0000","last_undegraded":"2024-04-20T11:27:00.184744+0000","last_fullsized":"2024-04-20T11:27:00.184744+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:39:03.501101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.185902+0000","last_change":"2024-04-20T11:26:58.680824+0000","last_active":"2024-04-20T11:27:00.185902+0000","last_peered":"2024-04-20T11:27:00.185902+0000","last_clean":"2024-04-20T11:27:00.185902+0000","last_became_active":"2024-04-20T11:26:58.680518+0000","last_became_peered":"2024-04-20T11:26:58.680518+0000","last_unstale":"2024-04-20T11:27:00.185902+0000","last_undegraded":"2024-04-20T11:27:00.185902+0000","last_fullsized":"2024-04-20T11:27:00.185902+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:01:34.618021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.187052+0000","last_change":"2024-04-20T11:26:57.695120+0000","last_active":"2024-04-20T11:27:00.187052+0000","last_peered":"2024-04-20T11:27:00.187052+0000","last_clean":"2024-04-20T11:27:00.187052+0000","last_became_active":"2024-04-20T11:26:57.694850+0000","last_became_peered":"2024-04-20T11:26:57.694850+0000","last_unstale":"2024-04-20T11:27:00.187052+0000","last_undegraded":"2024-04-20T11:27:00.187052+0000","last_fullsized":"2024-04-20T11:27:00.187052+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:56:22.881738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.186628+0000","last_change":"2024-04-20T11:26:58.704801+0000","last_active":"2024-04-20T11:27:00.186628+0000","last_peered":"2024-04-20T11:27:00.186628+0000","last_clean":"2024-04-20T11:27:00.186628+0000","last_became_active":"2024-04-20T11:26:58.703638+0000","last_became_peered":"2024-04-20T11:26:58.703638+0000","last_unstale":"2024-04-20T11:27:00.186628+0000","last_undegraded":"2024-04-20T11:27:00.186628+0000","last_fullsized":"2024-04-20T11:27:00.186628+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:55:47.019482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.739746+0000","last_change":"2024-04-20T11:26:57.702231+0000","last_active":"2024-04-20T11:26:59.739746+0000","last_peered":"2024-04-20T11:26:59.739746+0000","last_clean":"2024-04-20T11:26:59.739746+0000","last_became_active":"2024-04-20T11:26:57.701649+0000","last_became_peered":"2024-04-20T11:26:57.701649+0000","last_unstale":"2024-04-20T11:26:59.739746+0000","last_undegraded":"2024-04-20T11:26:59.739746+0000","last_fullsized":"2024-04-20T11:26:59.739746+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:59:13.454290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.693158+0000","last_change":"2024-04-20T11:26:58.685253+0000","last_active":"2024-04-20T11:26:59.693158+0000","last_peered":"2024-04-20T11:26:59.693158+0000","last_clean":"2024-04-20T11:26:59.693158+0000","last_became_active":"2024-04-20T11:26:58.684993+0000","last_became_peered":"2024-04-20T11:26:58.684993+0000","last_unstale":"2024-04-20T11:26:59.693158+0000","last_undegraded":"2024-04-20T11:26:59.693158+0000","last_fullsized":"2024-04-20T11:26:59.693158+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:02:15.732106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.184446+0000","last_change":"2024-04-20T11:26:57.697065+0000","last_active":"2024-04-20T11:27:00.184446+0000","last_peered":"2024-04-20T11:27:00.184446+0000","last_clean":"2024-04-20T11:27:00.184446+0000","last_became_active":"2024-04-20T11:26:57.696461+0000","last_became_peered":"2024-04-20T11:26:57.696461+0000","last_unstale":"2024-04-20T11:27:00.184446+0000","last_undegraded":"2024-04-20T11:27:00.184446+0000","last_fullsized":"2024-04-20T11:27:00.184446+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:56:22.881738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.695053+0000","last_change":"2024-04-20T11:26:58.685057+0000","last_active":"2024-04-20T11:26:59.695053+0000","last_peered":"2024-04-20T11:26:59.695053+0000","last_clean":"2024-04-20T11:26:59.695053+0000","last_became_active":"2024-04-20T11:26:58.684493+0000","last_became_peered":"2024-04-20T11:26:58.684493+0000","last_unstale":"2024-04-20T11:26:59.695053+0000","last_undegraded":"2024-04-20T11:26:59.695053+0000","last_fullsized":"2024-04-20T11:26:59.695053+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:59:14.469331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.185352+0000","last_change":"2024-04-20T11:26:57.690472+0000","last_active":"2024-04-20T11:27:00.185352+0000","last_peered":"2024-04-20T11:27:00.185352+0000","last_clean":"2024-04-20T11:27:00.185352+0000","last_became_active":"2024-04-20T11:26:57.689786+0000","last_became_peered":"2024-04-20T11:26:57.689786+0000","last_unstale":"2024-04-20T11:27:00.185352+0000","last_undegraded":"2024-04-20T11:27:00.185352+0000","last_fullsized":"2024-04-20T11:27:00.185352+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:30:34.827026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.184283+0000","last_change":"2024-04-20T11:26:58.677827+0000","last_active":"2024-04-20T11:27:00.184283+0000","last_peered":"2024-04-20T11:27:00.184283+0000","last_clean":"2024-04-20T11:27:00.184283+0000","last_became_active":"2024-04-20T11:26:58.677521+0000","last_became_peered":"2024-04-20T11:26:58.677521+0000","last_unstale":"2024-04-20T11:27:00.184283+0000","last_undegraded":"2024-04-20T11:27:00.184283+0000","last_fullsized":"2024-04-20T11:27:00.184283+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:02:15.732106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.185683+0000","last_change":"2024-04-20T11:26:57.673530+0000","last_active":"2024-04-20T11:27:00.185683+0000","last_peered":"2024-04-20T11:27:00.185683+0000","last_clean":"2024-04-20T11:27:00.185683+0000","last_became_active":"2024-04-20T11:26:57.671513+0000","last_became_peered":"2024-04-20T11:26:57.671513+0000","last_unstale":"2024-04-20T11:27:00.185683+0000","last_undegraded":"2024-04-20T11:27:00.185683+0000","last_fullsized":"2024-04-20T11:27:00.185683+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:11:19.111199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.185170+0000","last_change":"2024-04-20T11:26:58.702869+0000","last_active":"2024-04-20T11:27:00.185170+0000","last_peered":"2024-04-20T11:27:00.185170+0000","last_clean":"2024-04-20T11:27:00.185170+0000","last_became_active":"2024-04-20T11:26:58.698423+0000","last_became_peered":"2024-04-20T11:26:58.698423+0000","last_unstale":"2024-04-20T11:27:00.185170+0000","last_undegraded":"2024-04-20T11:27:00.185170+0000","last_fullsized":"2024-04-20T11:27:00.185170+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:35:40.422299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.185236+0000","last_change":"2024-04-20T11:26:57.699410+0000","last_active":"2024-04-20T11:27:00.185236+0000","last_peered":"2024-04-20T11:27:00.185236+0000","last_clean":"2024-04-20T11:27:00.185236+0000","last_became_active":"2024-04-20T11:26:57.697630+0000","last_became_peered":"2024-04-20T11:26:57.697630+0000","last_unstale":"2024-04-20T11:27:00.185236+0000","last_undegraded":"2024-04-20T11:27:00.185236+0000","last_fullsized":"2024-04-20T11:27:00.185236+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:18:30.073624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.738917+0000","last_change":"2024-04-20T11:26:58.711494+0000","last_active":"2024-04-20T11:26:59.738917+0000","last_peered":"2024-04-20T11:26:59.738917+0000","last_clean":"2024-04-20T11:26:59.738917+0000","last_became_active":"2024-04-20T11:26:58.711116+0000","last_became_peered":"2024-04-20T11:26:58.711116+0000","last_unstale":"2024-04-20T11:26:59.738917+0000","last_undegraded":"2024-04-20T11:26:59.738917+0000","last_fullsized":"2024-04-20T11:26:59.738917+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:43:20.032648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.695487+0000","last_change":"2024-04-20T11:26:58.701576+0000","last_active":"2024-04-20T11:26:59.695487+0000","last_peered":"2024-04-20T11:26:59.695487+0000","last_clean":"2024-04-20T11:26:59.695487+0000","last_became_active":"2024-04-20T11:26:58.701222+0000","last_became_peered":"2024-04-20T11:26:58.701222+0000","last_unstale":"2024-04-20T11:26:59.695487+0000","last_undegraded":"2024-04-20T11:26:59.695487+0000","last_fullsized":"2024-04-20T11:26:59.695487+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:55:47.019482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.184217+0000","last_change":"2024-04-20T11:26:57.701154+0000","last_active":"2024-04-20T11:27:00.184217+0000","last_peered":"2024-04-20T11:27:00.184217+0000","last_clean":"2024-04-20T11:27:00.184217+0000","last_became_active":"2024-04-20T11:26:57.700716+0000","last_became_peered":"2024-04-20T11:26:57.700716+0000","last_unstale":"2024-04-20T11:27:00.184217+0000","last_undegraded":"2024-04-20T11:27:00.184217+0000","last_fullsized":"2024-04-20T11:27:00.184217+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:09:13.468036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.698090+0000","last_change":"2024-04-20T11:26:58.706275+0000","last_active":"2024-04-20T11:26:59.698090+0000","last_peered":"2024-04-20T11:26:59.698090+0000","last_clean":"2024-04-20T11:26:59.698090+0000","last_became_active":"2024-04-20T11:26:58.705922+0000","last_became_peered":"2024-04-20T11:26:58.705922+0000","last_unstale":"2024-04-20T11:26:59.698090+0000","last_undegraded":"2024-04-20T11:26:59.698090+0000","last_fullsized":"2024-04-20T11:26:59.698090+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:40:22.418833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.184419+0000","last_change":"2024-04-20T11:26:57.701304+0000","last_active":"2024-04-20T11:27:00.184419+0000","last_peered":"2024-04-20T11:27:00.184419+0000","last_clean":"2024-04-20T11:27:00.184419+0000","last_became_active":"2024-04-20T11:26:57.700620+0000","last_became_peered":"2024-04-20T11:26:57.700620+0000","last_unstale":"2024-04-20T11:27:00.184419+0000","last_undegraded":"2024-04-20T11:27:00.184419+0000","last_fullsized":"2024-04-20T11:27:00.184419+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:33:14.313913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.693201+0000","last_change":"2024-04-20T11:26:58.688125+0000","last_active":"2024-04-20T11:26:59.693201+0000","last_peered":"2024-04-20T11:26:59.693201+0000","last_clean":"2024-04-20T11:26:59.693201+0000","last_became_active":"2024-04-20T11:26:58.687917+0000","last_became_peered":"2024-04-20T11:26:58.687917+0000","last_unstale":"2024-04-20T11:26:59.693201+0000","last_undegraded":"2024-04-20T11:26:59.693201+0000","last_fullsized":"2024-04-20T11:26:59.693201+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:01:06.520823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.185078+0000","last_change":"2024-04-20T11:26:57.671052+0000","last_active":"2024-04-20T11:27:00.185078+0000","last_peered":"2024-04-20T11:27:00.185078+0000","last_clean":"2024-04-20T11:27:00.185078+0000","last_became_active":"2024-04-20T11:26:57.670620+0000","last_became_peered":"2024-04-20T11:26:57.670620+0000","last_unstale":"2024-04-20T11:27:00.185078+0000","last_undegraded":"2024-04-20T11:27:00.185078+0000","last_fullsized":"2024-04-20T11:27:00.185078+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:11:36.407427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.184939+0000","last_change":"2024-04-20T11:26:58.701127+0000","last_active":"2024-04-20T11:27:00.184939+0000","last_peered":"2024-04-20T11:27:00.184939+0000","last_clean":"2024-04-20T11:27:00.184939+0000","last_became_active":"2024-04-20T11:26:58.700948+0000","last_became_peered":"2024-04-20T11:26:58.700948+0000","last_unstale":"2024-04-20T11:27:00.184939+0000","last_undegraded":"2024-04-20T11:27:00.184939+0000","last_fullsized":"2024-04-20T11:27:00.184939+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:01:06.520823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.693275+0000","last_change":"2024-04-20T11:26:57.692256+0000","last_active":"2024-04-20T11:26:59.693275+0000","last_peered":"2024-04-20T11:26:59.693275+0000","last_clean":"2024-04-20T11:26:59.693275+0000","last_became_active":"2024-04-20T11:26:57.691929+0000","last_became_peered":"2024-04-20T11:26:57.691929+0000","last_unstale":"2024-04-20T11:26:59.693275+0000","last_undegraded":"2024-04-20T11:26:59.693275+0000","last_fullsized":"2024-04-20T11:26:59.693275+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:34:59.927980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T11:27:03.352 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T11:27:03.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.592+0000 7f0b15800640 1 Processor -- start 2024-04-20T11:27:03.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.592+0000 7f0b15800640 1 -- start start 2024-04-20T11:27:03.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.593+0000 7f0b15800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f0b1006eb40 0x7f0b1006f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:03.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.593+0000 7f0b15800640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f0b1006f880 0x7f0b1015cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:03.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.593+0000 7f0b15800640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b1006fca0 con 0x7f0b1015b5e0 2024-04-20T11:27:03.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.593+0000 7f0b15800640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f0b1006de10 con 0x7f0b1006eb40 2024-04-20T11:27:03.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.593+0000 7f0b15800640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f0b100b5770 con 0x7f0b1006f880 2024-04-20T11:27:03.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.593+0000 7f0b0ea00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f0b1015b5e0 0x7f0b100b4c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:40584/0 (socket says 172.21.15.152:40584) 2024-04-20T11:27:03.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.593+0000 7f0b0ea00640 1 -- 172.21.15.152:0/872652257 learned_addr learned my addr 172.21.15.152:0/872652257 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:27:03.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.594+0000 7f0b0e000640 1 --2- 172.21.15.152:0/872652257 >> v2:172.21.15.152:3300/0 conn(0x7f0b1006eb40 0x7f0b1006f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:03.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.594+0000 7f0b0f400640 1 --2- 172.21.15.152:0/872652257 >> v2:172.21.15.175:3300/0 conn(0x7f0b1006f880 0x7f0b1015cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:03.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.594+0000 7f0b0d600640 1 -- 172.21.15.152:0/872652257 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 768459031 0 0) 0x7f0b1006fca0 con 0x7f0b1015b5e0 2024-04-20T11:27:03.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.594+0000 7f0b0d600640 1 -- 172.21.15.152:0/872652257 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0af4003520 con 0x7f0b1015b5e0 2024-04-20T11:27:03.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.594+0000 7f0b0e000640 1 -- 172.21.15.152:0/872652257 >> v2:172.21.15.175:3300/0 conn(0x7f0b1006f880 msgr2=0x7f0b1015cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:03.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.594+0000 7f0b0e000640 1 --2- 172.21.15.152:0/872652257 >> v2:172.21.15.175:3300/0 conn(0x7f0b1006f880 0x7f0b1015cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:03.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.594+0000 7f0b0e000640 1 -- 172.21.15.152:0/872652257 >> v1:172.21.15.152:6789/0 conn(0x7f0b1015b5e0 legacy=0x7f0b100b4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:03.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.594+0000 7f0b0e000640 1 -- 172.21.15.152:0/872652257 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b1006e060 con 0x7f0b1006eb40 2024-04-20T11:27:03.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.595+0000 7f0b0e000640 1 --2- 172.21.15.152:0/872652257 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0b1006eb40 0x7f0b1006f340 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0b100d0e90 tx=0x7f0afc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf00c7ac3c6a208a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:03.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.595+0000 7f0b0d600640 1 -- 172.21.15.152:0/872652257 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0afc035020 con 0x7f0b1006eb40 2024-04-20T11:27:03.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.595+0000 7f0b0d600640 1 -- 172.21.15.152:0/872652257 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0afc031830 con 0x7f0b1006eb40 2024-04-20T11:27:03.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.595+0000 7f0b0d600640 1 -- 172.21.15.152:0/872652257 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0afc031c00 con 0x7f0b1006eb40 2024-04-20T11:27:03.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.596+0000 7f0b15800640 1 -- 172.21.15.152:0/872652257 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0b1006eb40 msgr2=0x7f0b1006f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:03.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.596+0000 7f0b15800640 1 --2- 172.21.15.152:0/872652257 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0b1006eb40 0x7f0b1006f340 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0b100d0e90 tx=0x7f0afc02b8d0 comp rx=0 tx=0).stop 2024-04-20T11:27:03.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.596+0000 7f0b15800640 1 -- 172.21.15.152:0/872652257 shutdown_connections 2024-04-20T11:27:03.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.596+0000 7f0b15800640 1 --2- 172.21.15.152:0/872652257 >> v2:172.21.15.175:3300/0 conn(0x7f0b1006f880 0x7f0b1015cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:03.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.596+0000 7f0b15800640 1 --2- 172.21.15.152:0/872652257 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0b1006eb40 0x7f0b1006f340 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:03.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.596+0000 7f0b15800640 1 -- 172.21.15.152:0/872652257 >> 172.21.15.152:0/872652257 conn(0x7f0b10067cf0 msgr2=0x7f0b1006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:03.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.596+0000 7f0b15800640 1 -- 172.21.15.152:0/872652257 shutdown_connections 2024-04-20T11:27:03.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.596+0000 7f0b15800640 1 -- 172.21.15.152:0/872652257 wait complete. 2024-04-20T11:27:03.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.598+0000 7f0b15800640 1 Processor -- start 2024-04-20T11:27:03.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.598+0000 7f0b15800640 1 -- start start 2024-04-20T11:27:03.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.599+0000 7f0b15800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b1006eb40 0x7f0b1015cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:03.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.599+0000 7f0b15800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b1006f880 0x7f0b100b1020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:03.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.599+0000 7f0b15800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0b1015b5e0 0x7f0b100b1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:03.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.599+0000 7f0b15800640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f0b1015c9d0 con 0x7f0b1015b5e0 2024-04-20T11:27:03.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.599+0000 7f0b15800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0b1006e620 con 0x7f0b1006eb40 2024-04-20T11:27:03.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.599+0000 7f0b15800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0b100b42d0 con 0x7f0b1006f880 2024-04-20T11:27:03.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.599+0000 7f0b0f400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0b1015b5e0 0x7f0b100b1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:03.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.599+0000 7f0b0f400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0b1015b5e0 0x7f0b100b1560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:42656/0 (socket says 172.21.15.152:42656) 2024-04-20T11:27:03.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.599+0000 7f0b0f400640 1 -- 172.21.15.152:0/3324072710 learned_addr learned my addr 172.21.15.152:0/3324072710 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:03.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.599+0000 7f0b0ea00640 1 --2- 172.21.15.152:0/3324072710 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b1006eb40 0x7f0b1015cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:03.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.599+0000 7f0b0e000640 1 --2- 172.21.15.152:0/3324072710 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b1006f880 0x7f0b100b1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:03.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.599+0000 7f0b0f400640 1 -- 172.21.15.152:0/3324072710 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b1006f880 msgr2=0x7f0b100b1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:03.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.599+0000 7f0b0f400640 1 --2- 172.21.15.152:0/3324072710 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b1006f880 0x7f0b100b1020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:03.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.600+0000 7f0b0f400640 1 -- 172.21.15.152:0/3324072710 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b1006eb40 msgr2=0x7f0b1015cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:03.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.600+0000 7f0b0f400640 1 --2- 172.21.15.152:0/3324072710 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b1006eb40 0x7f0b1015cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:03.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.600+0000 7f0b0f400640 1 -- 172.21.15.152:0/3324072710 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b10145380 con 0x7f0b1015b5e0 2024-04-20T11:27:03.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.600+0000 7f0b0e000640 1 --2- 172.21.15.152:0/3324072710 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b1006f880 0x7f0b100b1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:27:03.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.600+0000 7f0b0f400640 1 --2- 172.21.15.152:0/3324072710 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0b1015b5e0 0x7f0b100b1560 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0b08007e40 tx=0x7f0b0800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:03.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.600+0000 7f0b06a00640 1 -- 172.21.15.152:0/3324072710 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b08014620 con 0x7f0b1015b5e0 2024-04-20T11:27:03.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.600+0000 7f0b06a00640 1 -- 172.21.15.152:0/3324072710 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b08019070 con 0x7f0b1015b5e0 2024-04-20T11:27:03.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.600+0000 7f0b06a00640 1 -- 172.21.15.152:0/3324072710 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b08014a20 con 0x7f0b1015b5e0 2024-04-20T11:27:03.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.600+0000 7f0b15800640 1 -- 172.21.15.152:0/3324072710 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b100b0260 con 0x7f0b1015b5e0 2024-04-20T11:27:03.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.600+0000 7f0b15800640 1 -- 172.21.15.152:0/3324072710 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b100b0820 con 0x7f0b1015b5e0 2024-04-20T11:27:03.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.601+0000 7f0b15800640 1 -- 172.21.15.152:0/3324072710 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b100689f0 con 0x7f0b1015b5e0 2024-04-20T11:27:03.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.602+0000 7f0b06a00640 1 -- 172.21.15.152:0/3324072710 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0b08014bc0 con 0x7f0b1015b5e0 2024-04-20T11:27:03.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.603+0000 7f0b06a00640 1 --2- 172.21.15.152:0/3324072710 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f0af006c8f0 0x7f0af006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:03.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.603+0000 7f0b06a00640 1 -- 172.21.15.152:0/3324072710 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f0b0808c4c0 con 0x7f0b1015b5e0 2024-04-20T11:27:03.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.603+0000 7f0b0ea00640 1 --2- 172.21.15.152:0/3324072710 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f0af006c8f0 0x7f0af006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:03.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.604+0000 7f0b0ea00640 1 --2- 172.21.15.152:0/3324072710 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f0af006c8f0 0x7f0af006edb0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0b00005880 tx=0x7f0b00009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:03.607 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.607+0000 7f0b06a00640 1 -- 172.21.15.152:0/3324072710 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f0b0805a8e0 con 0x7f0b1015b5e0 2024-04-20T11:27:03.805 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.805+0000 7f0b15800640 1 -- 172.21.15.152:0/3324072710 --> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0b1014f0c0 con 0x7f0af006c8f0 2024-04-20T11:27:03.812 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.812+0000 7f0b06a00640 1 -- 172.21.15.152:0/3324072710 <== mgr.4113 v2:172.21.15.152:6832/2285110240 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251948 (secure 0 0 0) 0x7f0b1014f0c0 con 0x7f0af006c8f0 2024-04-20T11:27:03.815 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:27:03.817 INFO:teuthology.orchestra.run.smithi152.stderr:dumped all 2024-04-20T11:27:03.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.818+0000 7f0b15800640 1 -- 172.21.15.152:0/3324072710 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f0af006c8f0 msgr2=0x7f0af006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:03.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.818+0000 7f0b15800640 1 --2- 172.21.15.152:0/3324072710 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f0af006c8f0 0x7f0af006edb0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0b00005880 tx=0x7f0b00009040 comp rx=0 tx=0).stop 2024-04-20T11:27:03.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.818+0000 7f0b15800640 1 -- 172.21.15.152:0/3324072710 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0b1015b5e0 msgr2=0x7f0b100b1560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:03.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.818+0000 7f0b15800640 1 --2- 172.21.15.152:0/3324072710 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0b1015b5e0 0x7f0b100b1560 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0b08007e40 tx=0x7f0b0800e4b0 comp rx=0 tx=0).stop 2024-04-20T11:27:03.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.819+0000 7f0b15800640 1 -- 172.21.15.152:0/3324072710 shutdown_connections 2024-04-20T11:27:03.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.819+0000 7f0b15800640 1 --2- 172.21.15.152:0/3324072710 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f0af006c8f0 0x7f0af006edb0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:03.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.819+0000 7f0b15800640 1 --2- 172.21.15.152:0/3324072710 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0b1015b5e0 0x7f0b100b1560 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:03.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.819+0000 7f0b15800640 1 --2- 172.21.15.152:0/3324072710 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b1006f880 0x7f0b100b1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:03.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.819+0000 7f0b15800640 1 --2- 172.21.15.152:0/3324072710 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b1006eb40 0x7f0b1015cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:03.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.819+0000 7f0b15800640 1 -- 172.21.15.152:0/3324072710 >> 172.21.15.152:0/3324072710 conn(0x7f0b10067cf0 msgr2=0x7f0b10159be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:03.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.819+0000 7f0b15800640 1 -- 172.21.15.152:0/3324072710 shutdown_connections 2024-04-20T11:27:03.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:03.819+0000 7f0b15800640 1 -- 172.21.15.152:0/3324072710 wait complete. 2024-04-20T11:27:03.839 INFO:teuthology.orchestra.run.smithi152.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-04-20T11:27:03.213056+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219000,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755720,"statfs":{"total":773094113280,"available":772869857280,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12746,"internal_metadata":221236790},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001225"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.185059+0000","last_change":"2024-04-20T11:26:58.706148+0000","last_active":"2024-04-20T11:27:00.185059+0000","last_peered":"2024-04-20T11:27:00.185059+0000","last_clean":"2024-04-20T11:27:00.185059+0000","last_became_active":"2024-04-20T11:26:58.705669+0000","last_became_peered":"2024-04-20T11:26:58.705669+0000","last_unstale":"2024-04-20T11:27:00.185059+0000","last_undegraded":"2024-04-20T11:27:00.185059+0000","last_fullsized":"2024-04-20T11:27:00.185059+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:25:02.202901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.187149+0000","last_change":"2024-04-20T11:26:58.688184+0000","last_active":"2024-04-20T11:27:00.187149+0000","last_peered":"2024-04-20T11:27:00.187149+0000","last_clean":"2024-04-20T11:27:00.187149+0000","last_became_active":"2024-04-20T11:26:58.685678+0000","last_became_peered":"2024-04-20T11:26:58.685678+0000","last_unstale":"2024-04-20T11:27:00.187149+0000","last_undegraded":"2024-04-20T11:27:00.187149+0000","last_fullsized":"2024-04-20T11:27:00.187149+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:30:01.981489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.692840+0000","last_change":"2024-04-20T11:26:58.675003+0000","last_active":"2024-04-20T11:26:59.692840+0000","last_peered":"2024-04-20T11:26:59.692840+0000","last_clean":"2024-04-20T11:26:59.692840+0000","last_became_active":"2024-04-20T11:26:58.674697+0000","last_became_peered":"2024-04-20T11:26:58.674697+0000","last_unstale":"2024-04-20T11:26:59.692840+0000","last_undegraded":"2024-04-20T11:26:59.692840+0000","last_fullsized":"2024-04-20T11:26:59.692840+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:48:49.870718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.697801+0000","last_change":"2024-04-20T11:26:58.678578+0000","last_active":"2024-04-20T11:26:59.697801+0000","last_peered":"2024-04-20T11:26:59.697801+0000","last_clean":"2024-04-20T11:26:59.697801+0000","last_became_active":"2024-04-20T11:26:58.677959+0000","last_became_peered":"2024-04-20T11:26:58.677959+0000","last_unstale":"2024-04-20T11:26:59.697801+0000","last_undegraded":"2024-04-20T11:26:59.697801+0000","last_fullsized":"2024-04-20T11:26:59.697801+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:27:35.003583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.185015+0000","last_change":"2024-04-20T11:26:58.703032+0000","last_active":"2024-04-20T11:27:00.185015+0000","last_peered":"2024-04-20T11:27:00.185015+0000","last_clean":"2024-04-20T11:27:00.185015+0000","last_became_active":"2024-04-20T11:26:58.702687+0000","last_became_peered":"2024-04-20T11:26:58.702687+0000","last_unstale":"2024-04-20T11:27:00.185015+0000","last_undegraded":"2024-04-20T11:27:00.185015+0000","last_fullsized":"2024-04-20T11:27:00.185015+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:35:38.308224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.698653+0000","last_change":"2024-04-20T11:26:58.692001+0000","last_active":"2024-04-20T11:26:59.698653+0000","last_peered":"2024-04-20T11:26:59.698653+0000","last_clean":"2024-04-20T11:26:59.698653+0000","last_became_active":"2024-04-20T11:26:58.689784+0000","last_became_peered":"2024-04-20T11:26:58.689784+0000","last_unstale":"2024-04-20T11:26:59.698653+0000","last_undegraded":"2024-04-20T11:26:59.698653+0000","last_fullsized":"2024-04-20T11:26:59.698653+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:35:38.308224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.184927+0000","last_change":"2024-04-20T11:26:58.700657+0000","last_active":"2024-04-20T11:27:00.184927+0000","last_peered":"2024-04-20T11:27:00.184927+0000","last_clean":"2024-04-20T11:27:00.184927+0000","last_became_active":"2024-04-20T11:26:58.700233+0000","last_became_peered":"2024-04-20T11:26:58.700233+0000","last_unstale":"2024-04-20T11:27:00.184927+0000","last_undegraded":"2024-04-20T11:27:00.184927+0000","last_fullsized":"2024-04-20T11:27:00.184927+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:40:22.418833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.183717+0000","last_change":"2024-04-20T11:26:58.691319+0000","last_active":"2024-04-20T11:27:00.183717+0000","last_peered":"2024-04-20T11:27:00.183717+0000","last_clean":"2024-04-20T11:27:00.183717+0000","last_became_active":"2024-04-20T11:26:58.690968+0000","last_became_peered":"2024-04-20T11:26:58.690968+0000","last_unstale":"2024-04-20T11:27:00.183717+0000","last_undegraded":"2024-04-20T11:27:00.183717+0000","last_fullsized":"2024-04-20T11:27:00.183717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:09:06.943774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.185058+0000","last_change":"2024-04-20T11:26:58.708815+0000","last_active":"2024-04-20T11:27:00.185058+0000","last_peered":"2024-04-20T11:27:00.185058+0000","last_clean":"2024-04-20T11:27:00.185058+0000","last_became_active":"2024-04-20T11:26:58.708527+0000","last_became_peered":"2024-04-20T11:26:58.708527+0000","last_unstale":"2024-04-20T11:27:00.185058+0000","last_undegraded":"2024-04-20T11:27:00.185058+0000","last_fullsized":"2024-04-20T11:27:00.185058+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:37:41.100182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.739853+0000","last_change":"2024-04-20T11:26:58.702277+0000","last_active":"2024-04-20T11:26:59.739853+0000","last_peered":"2024-04-20T11:26:59.739853+0000","last_clean":"2024-04-20T11:26:59.739853+0000","last_became_active":"2024-04-20T11:26:58.701978+0000","last_became_peered":"2024-04-20T11:26:58.701978+0000","last_unstale":"2024-04-20T11:26:59.739853+0000","last_undegraded":"2024-04-20T11:26:59.739853+0000","last_fullsized":"2024-04-20T11:26:59.739853+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:53:57.559834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.739719+0000","last_change":"2024-04-20T11:26:58.702193+0000","last_active":"2024-04-20T11:26:59.739719+0000","last_peered":"2024-04-20T11:26:59.739719+0000","last_clean":"2024-04-20T11:26:59.739719+0000","last_became_active":"2024-04-20T11:26:58.701865+0000","last_became_peered":"2024-04-20T11:26:58.701865+0000","last_unstale":"2024-04-20T11:26:59.739719+0000","last_undegraded":"2024-04-20T11:26:59.739719+0000","last_fullsized":"2024-04-20T11:26:59.739719+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:01:34.618021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.698023+0000","last_change":"2024-04-20T11:26:58.701101+0000","last_active":"2024-04-20T11:26:59.698023+0000","last_peered":"2024-04-20T11:26:59.698023+0000","last_clean":"2024-04-20T11:26:59.698023+0000","last_became_active":"2024-04-20T11:26:58.699917+0000","last_became_peered":"2024-04-20T11:26:58.699917+0000","last_unstale":"2024-04-20T11:26:59.698023+0000","last_undegraded":"2024-04-20T11:26:59.698023+0000","last_fullsized":"2024-04-20T11:26:59.698023+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:30:35.842066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.184787+0000","last_change":"2024-04-20T11:26:58.695384+0000","last_active":"2024-04-20T11:27:00.184787+0000","last_peered":"2024-04-20T11:27:00.184787+0000","last_clean":"2024-04-20T11:27:00.184787+0000","last_became_active":"2024-04-20T11:26:58.695086+0000","last_became_peered":"2024-04-20T11:26:58.695086+0000","last_unstale":"2024-04-20T11:27:00.184787+0000","last_undegraded":"2024-04-20T11:27:00.184787+0000","last_fullsized":"2024-04-20T11:27:00.184787+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:47:21.129095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.693173+0000","last_change":"2024-04-20T11:26:58.677865+0000","last_active":"2024-04-20T11:26:59.693173+0000","last_peered":"2024-04-20T11:26:59.693173+0000","last_clean":"2024-04-20T11:26:59.693173+0000","last_became_active":"2024-04-20T11:26:58.677541+0000","last_became_peered":"2024-04-20T11:26:58.677541+0000","last_unstale":"2024-04-20T11:26:59.693173+0000","last_undegraded":"2024-04-20T11:26:59.693173+0000","last_fullsized":"2024-04-20T11:26:59.693173+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:47:21.129095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.186987+0000","last_change":"2024-04-20T11:26:58.693039+0000","last_active":"2024-04-20T11:27:00.186987+0000","last_peered":"2024-04-20T11:27:00.186987+0000","last_clean":"2024-04-20T11:27:00.186987+0000","last_became_active":"2024-04-20T11:26:58.692020+0000","last_became_peered":"2024-04-20T11:26:58.692020+0000","last_unstale":"2024-04-20T11:27:00.186987+0000","last_undegraded":"2024-04-20T11:27:00.186987+0000","last_fullsized":"2024-04-20T11:27:00.186987+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:35:40.422299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.693509+0000","last_change":"2024-04-20T11:26:58.686574+0000","last_active":"2024-04-20T11:26:59.693509+0000","last_peered":"2024-04-20T11:26:59.693509+0000","last_clean":"2024-04-20T11:26:59.693509+0000","last_became_active":"2024-04-20T11:26:58.686297+0000","last_became_peered":"2024-04-20T11:26:58.686297+0000","last_unstale":"2024-04-20T11:26:59.693509+0000","last_undegraded":"2024-04-20T11:26:59.693509+0000","last_fullsized":"2024-04-20T11:26:59.693509+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:01:34.618021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.695098+0000","last_change":"2024-04-20T11:26:58.701917+0000","last_active":"2024-04-20T11:26:59.695098+0000","last_peered":"2024-04-20T11:26:59.695098+0000","last_clean":"2024-04-20T11:26:59.695098+0000","last_became_active":"2024-04-20T11:26:58.701625+0000","last_became_peered":"2024-04-20T11:26:58.701625+0000","last_unstale":"2024-04-20T11:26:59.695098+0000","last_undegraded":"2024-04-20T11:26:59.695098+0000","last_fullsized":"2024-04-20T11:26:59.695098+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:35:40.422299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.185755+0000","last_change":"2024-04-20T11:26:58.697089+0000","last_active":"2024-04-20T11:27:00.185755+0000","last_peered":"2024-04-20T11:27:00.185755+0000","last_clean":"2024-04-20T11:27:00.185755+0000","last_became_active":"2024-04-20T11:26:58.696794+0000","last_became_peered":"2024-04-20T11:26:58.696794+0000","last_unstale":"2024-04-20T11:27:00.185755+0000","last_undegraded":"2024-04-20T11:27:00.185755+0000","last_fullsized":"2024-04-20T11:27:00.185755+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:30:35.842066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.184901+0000","last_change":"2024-04-20T11:26:58.707870+0000","last_active":"2024-04-20T11:27:00.184901+0000","last_peered":"2024-04-20T11:27:00.184901+0000","last_clean":"2024-04-20T11:27:00.184901+0000","last_became_active":"2024-04-20T11:26:58.702746+0000","last_became_peered":"2024-04-20T11:26:58.702746+0000","last_unstale":"2024-04-20T11:27:00.184901+0000","last_undegraded":"2024-04-20T11:27:00.184901+0000","last_fullsized":"2024-04-20T11:27:00.184901+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:28:57.233788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.739521+0000","last_change":"2024-04-20T11:26:58.707789+0000","last_active":"2024-04-20T11:26:59.739521+0000","last_peered":"2024-04-20T11:26:59.739521+0000","last_clean":"2024-04-20T11:26:59.739521+0000","last_became_active":"2024-04-20T11:26:58.707272+0000","last_became_peered":"2024-04-20T11:26:58.707272+0000","last_unstale":"2024-04-20T11:26:59.739521+0000","last_undegraded":"2024-04-20T11:26:59.739521+0000","last_fullsized":"2024-04-20T11:26:59.739521+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:01:06.520823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.692973+0000","last_change":"2024-04-20T11:26:58.698142+0000","last_active":"2024-04-20T11:26:59.692973+0000","last_peered":"2024-04-20T11:26:59.692973+0000","last_clean":"2024-04-20T11:26:59.692973+0000","last_became_active":"2024-04-20T11:26:58.697869+0000","last_became_peered":"2024-04-20T11:26:58.697869+0000","last_unstale":"2024-04-20T11:26:59.692973+0000","last_undegraded":"2024-04-20T11:26:59.692973+0000","last_fullsized":"2024-04-20T11:26:59.692973+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:43:20.032648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.696504+0000","last_change":"2024-04-20T11:26:58.692191+0000","last_active":"2024-04-20T11:26:59.696504+0000","last_peered":"2024-04-20T11:26:59.696504+0000","last_clean":"2024-04-20T11:26:59.696504+0000","last_became_active":"2024-04-20T11:26:58.690512+0000","last_became_peered":"2024-04-20T11:26:58.690512+0000","last_unstale":"2024-04-20T11:26:59.696504+0000","last_undegraded":"2024-04-20T11:26:59.696504+0000","last_fullsized":"2024-04-20T11:26:59.696504+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:01:48.311110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.185102+0000","last_change":"2024-04-20T11:26:58.693404+0000","last_active":"2024-04-20T11:27:00.185102+0000","last_peered":"2024-04-20T11:27:00.185102+0000","last_clean":"2024-04-20T11:27:00.185102+0000","last_became_active":"2024-04-20T11:26:58.693146+0000","last_became_peered":"2024-04-20T11:26:58.693146+0000","last_unstale":"2024-04-20T11:27:00.185102+0000","last_undegraded":"2024-04-20T11:27:00.185102+0000","last_fullsized":"2024-04-20T11:27:00.185102+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:01:34.618021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.693752+0000","last_change":"2024-04-20T11:26:58.682815+0000","last_active":"2024-04-20T11:26:59.693752+0000","last_peered":"2024-04-20T11:26:59.693752+0000","last_clean":"2024-04-20T11:26:59.693752+0000","last_became_active":"2024-04-20T11:26:58.679077+0000","last_became_peered":"2024-04-20T11:26:58.679077+0000","last_unstale":"2024-04-20T11:26:59.693752+0000","last_undegraded":"2024-04-20T11:26:59.693752+0000","last_fullsized":"2024-04-20T11:26:59.693752+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:53:57.559834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.186319+0000","last_change":"2024-04-20T11:26:58.706958+0000","last_active":"2024-04-20T11:27:00.186319+0000","last_peered":"2024-04-20T11:27:00.186319+0000","last_clean":"2024-04-20T11:27:00.186319+0000","last_became_active":"2024-04-20T11:26:58.706682+0000","last_became_peered":"2024-04-20T11:26:58.706682+0000","last_unstale":"2024-04-20T11:27:00.186319+0000","last_undegraded":"2024-04-20T11:27:00.186319+0000","last_fullsized":"2024-04-20T11:27:00.186319+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:40:45.000805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.739905+0000","last_change":"2024-04-20T11:26:58.693542+0000","last_active":"2024-04-20T11:26:59.739905+0000","last_peered":"2024-04-20T11:26:59.739905+0000","last_clean":"2024-04-20T11:26:59.739905+0000","last_became_active":"2024-04-20T11:26:58.693091+0000","last_became_peered":"2024-04-20T11:26:58.693091+0000","last_unstale":"2024-04-20T11:26:59.739905+0000","last_undegraded":"2024-04-20T11:26:59.739905+0000","last_fullsized":"2024-04-20T11:26:59.739905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:35:40.422299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.186334+0000","last_change":"2024-04-20T11:26:58.683272+0000","last_active":"2024-04-20T11:27:00.186334+0000","last_peered":"2024-04-20T11:27:00.186334+0000","last_clean":"2024-04-20T11:27:00.186334+0000","last_became_active":"2024-04-20T11:26:58.683012+0000","last_became_peered":"2024-04-20T11:26:58.683012+0000","last_unstale":"2024-04-20T11:27:00.186334+0000","last_undegraded":"2024-04-20T11:27:00.186334+0000","last_fullsized":"2024-04-20T11:27:00.186334+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:53:57.559834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.185138+0000","last_change":"2024-04-20T11:26:58.703756+0000","last_active":"2024-04-20T11:27:00.185138+0000","last_peered":"2024-04-20T11:27:00.185138+0000","last_clean":"2024-04-20T11:27:00.185138+0000","last_became_active":"2024-04-20T11:26:58.703323+0000","last_became_peered":"2024-04-20T11:26:58.703323+0000","last_unstale":"2024-04-20T11:27:00.185138+0000","last_undegraded":"2024-04-20T11:27:00.185138+0000","last_fullsized":"2024-04-20T11:27:00.185138+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:53:57.559834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.692748+0000","last_change":"2024-04-20T11:26:58.681550+0000","last_active":"2024-04-20T11:26:59.692748+0000","last_peered":"2024-04-20T11:26:59.692748+0000","last_clean":"2024-04-20T11:26:59.692748+0000","last_became_active":"2024-04-20T11:26:58.681270+0000","last_became_peered":"2024-04-20T11:26:58.681270+0000","last_unstale":"2024-04-20T11:26:59.692748+0000","last_undegraded":"2024-04-20T11:26:59.692748+0000","last_fullsized":"2024-04-20T11:26:59.692748+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:39:04.516141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.696801+0000","last_change":"2024-04-20T11:26:58.682629+0000","last_active":"2024-04-20T11:26:59.696801+0000","last_peered":"2024-04-20T11:26:59.696801+0000","last_clean":"2024-04-20T11:26:59.696801+0000","last_became_active":"2024-04-20T11:26:58.680875+0000","last_became_peered":"2024-04-20T11:26:58.680875+0000","last_unstale":"2024-04-20T11:26:59.696801+0000","last_undegraded":"2024-04-20T11:26:59.696801+0000","last_fullsized":"2024-04-20T11:26:59.696801+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:12:41.134688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.183098+0000","last_change":"2024-04-20T11:26:58.702006+0000","last_active":"2024-04-20T11:27:00.183098+0000","last_peered":"2024-04-20T11:27:00.183098+0000","last_clean":"2024-04-20T11:27:00.183098+0000","last_became_active":"2024-04-20T11:26:58.701697+0000","last_became_peered":"2024-04-20T11:26:58.701697+0000","last_unstale":"2024-04-20T11:27:00.183098+0000","last_undegraded":"2024-04-20T11:27:00.183098+0000","last_fullsized":"2024-04-20T11:27:00.183098+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:56:23.896778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.738970+0000","last_change":"2024-04-20T11:26:58.695448+0000","last_active":"2024-04-20T11:26:59.738970+0000","last_peered":"2024-04-20T11:26:59.738970+0000","last_clean":"2024-04-20T11:26:59.738970+0000","last_became_active":"2024-04-20T11:26:58.694946+0000","last_became_peered":"2024-04-20T11:26:58.694946+0000","last_unstale":"2024-04-20T11:26:59.738970+0000","last_undegraded":"2024-04-20T11:26:59.738970+0000","last_fullsized":"2024-04-20T11:26:59.738970+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:02:15.732106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.696114+0000","last_change":"2024-04-20T11:26:58.697053+0000","last_active":"2024-04-20T11:26:59.696114+0000","last_peered":"2024-04-20T11:26:59.696114+0000","last_clean":"2024-04-20T11:26:59.696114+0000","last_became_active":"2024-04-20T11:26:58.696713+0000","last_became_peered":"2024-04-20T11:26:58.696713+0000","last_unstale":"2024-04-20T11:26:59.696114+0000","last_undegraded":"2024-04-20T11:26:59.696114+0000","last_fullsized":"2024-04-20T11:26:59.696114+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:22:15.505173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.185319+0000","last_change":"2024-04-20T11:26:57.680979+0000","last_active":"2024-04-20T11:27:00.185319+0000","last_peered":"2024-04-20T11:27:00.185319+0000","last_clean":"2024-04-20T11:27:00.185319+0000","last_became_active":"2024-04-20T11:26:57.679239+0000","last_became_peered":"2024-04-20T11:26:57.679239+0000","last_unstale":"2024-04-20T11:27:00.185319+0000","last_undegraded":"2024-04-20T11:27:00.185319+0000","last_fullsized":"2024-04-20T11:27:00.185319+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:03:42.590011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.692914+0000","last_change":"2024-04-20T11:26:58.693305+0000","last_active":"2024-04-20T11:26:59.692914+0000","last_peered":"2024-04-20T11:26:59.692914+0000","last_clean":"2024-04-20T11:26:59.692914+0000","last_became_active":"2024-04-20T11:26:58.692561+0000","last_became_peered":"2024-04-20T11:26:58.692561+0000","last_unstale":"2024-04-20T11:26:59.692914+0000","last_undegraded":"2024-04-20T11:26:59.692914+0000","last_fullsized":"2024-04-20T11:26:59.692914+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:09:06.943774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.738687+0000","last_change":"2024-04-20T11:26:57.696531+0000","last_active":"2024-04-20T11:26:59.738687+0000","last_peered":"2024-04-20T11:26:59.738687+0000","last_clean":"2024-04-20T11:26:59.738687+0000","last_became_active":"2024-04-20T11:26:57.693505+0000","last_became_peered":"2024-04-20T11:26:57.693505+0000","last_unstale":"2024-04-20T11:26:59.738687+0000","last_undegraded":"2024-04-20T11:26:59.738687+0000","last_fullsized":"2024-04-20T11:26:59.738687+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:22:14.490133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.185207+0000","last_change":"2024-04-20T11:26:57.700780+0000","last_active":"2024-04-20T11:27:00.185207+0000","last_peered":"2024-04-20T11:27:00.185207+0000","last_clean":"2024-04-20T11:27:00.185207+0000","last_became_active":"2024-04-20T11:26:57.700475+0000","last_became_peered":"2024-04-20T11:26:57.700475+0000","last_unstale":"2024-04-20T11:27:00.185207+0000","last_undegraded":"2024-04-20T11:27:00.185207+0000","last_fullsized":"2024-04-20T11:27:00.185207+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:35:40.489733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.183864+0000","last_change":"2024-04-20T11:26:58.695758+0000","last_active":"2024-04-20T11:27:00.183864+0000","last_peered":"2024-04-20T11:27:00.183864+0000","last_clean":"2024-04-20T11:27:00.183864+0000","last_became_active":"2024-04-20T11:26:58.695496+0000","last_became_peered":"2024-04-20T11:26:58.695496+0000","last_unstale":"2024-04-20T11:27:00.183864+0000","last_undegraded":"2024-04-20T11:27:00.183864+0000","last_fullsized":"2024-04-20T11:27:00.183864+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:43:20.032648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.693649+0000","last_change":"2024-04-20T11:26:57.695300+0000","last_active":"2024-04-20T11:26:59.693649+0000","last_peered":"2024-04-20T11:26:59.693649+0000","last_clean":"2024-04-20T11:26:59.693649+0000","last_became_active":"2024-04-20T11:26:57.694343+0000","last_became_peered":"2024-04-20T11:26:57.694343+0000","last_unstale":"2024-04-20T11:26:59.693649+0000","last_undegraded":"2024-04-20T11:26:59.693649+0000","last_fullsized":"2024-04-20T11:26:59.693649+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:03:42.590011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.183738+0000","last_change":"2024-04-20T11:26:58.695911+0000","last_active":"2024-04-20T11:27:00.183738+0000","last_peered":"2024-04-20T11:27:00.183738+0000","last_clean":"2024-04-20T11:27:00.183738+0000","last_became_active":"2024-04-20T11:26:58.695607+0000","last_became_peered":"2024-04-20T11:26:58.695607+0000","last_unstale":"2024-04-20T11:27:00.183738+0000","last_undegraded":"2024-04-20T11:27:00.183738+0000","last_fullsized":"2024-04-20T11:27:00.183738+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:47:21.129095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.739639+0000","last_change":"2024-04-20T11:26:57.692397+0000","last_active":"2024-04-20T11:26:59.739639+0000","last_peered":"2024-04-20T11:26:59.739639+0000","last_clean":"2024-04-20T11:26:59.739639+0000","last_became_active":"2024-04-20T11:26:57.692046+0000","last_became_peered":"2024-04-20T11:26:57.692046+0000","last_unstale":"2024-04-20T11:26:59.739639+0000","last_undegraded":"2024-04-20T11:26:59.739639+0000","last_fullsized":"2024-04-20T11:26:59.739639+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:11:19.111199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.695485+0000","last_change":"2024-04-20T11:26:58.691351+0000","last_active":"2024-04-20T11:26:59.695485+0000","last_peered":"2024-04-20T11:26:59.695485+0000","last_clean":"2024-04-20T11:26:59.695485+0000","last_became_active":"2024-04-20T11:26:58.691059+0000","last_became_peered":"2024-04-20T11:26:58.691059+0000","last_unstale":"2024-04-20T11:26:59.695485+0000","last_undegraded":"2024-04-20T11:26:59.695485+0000","last_fullsized":"2024-04-20T11:26:59.695485+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:40:45.000805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.694315+0000","last_change":"2024-04-20T11:26:57.699532+0000","last_active":"2024-04-20T11:26:59.694315+0000","last_peered":"2024-04-20T11:26:59.694315+0000","last_clean":"2024-04-20T11:26:59.694315+0000","last_became_active":"2024-04-20T11:26:57.699281+0000","last_became_peered":"2024-04-20T11:26:57.699281+0000","last_unstale":"2024-04-20T11:26:59.694315+0000","last_undegraded":"2024-04-20T11:26:59.694315+0000","last_fullsized":"2024-04-20T11:26:59.694315+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:22:14.490133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.693433+0000","last_change":"2024-04-20T11:26:58.692727+0000","last_active":"2024-04-20T11:26:59.693433+0000","last_peered":"2024-04-20T11:26:59.693433+0000","last_clean":"2024-04-20T11:26:59.693433+0000","last_became_active":"2024-04-20T11:26:58.692415+0000","last_became_peered":"2024-04-20T11:26:58.692415+0000","last_unstale":"2024-04-20T11:26:59.693433+0000","last_undegraded":"2024-04-20T11:26:59.693433+0000","last_fullsized":"2024-04-20T11:26:59.693433+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:01:34.618021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.186405+0000","last_change":"2024-04-20T11:26:57.681417+0000","last_active":"2024-04-20T11:27:00.186405+0000","last_peered":"2024-04-20T11:27:00.186405+0000","last_clean":"2024-04-20T11:27:00.186405+0000","last_became_active":"2024-04-20T11:26:57.680971+0000","last_became_peered":"2024-04-20T11:26:57.680971+0000","last_unstale":"2024-04-20T11:27:00.186405+0000","last_undegraded":"2024-04-20T11:27:00.186405+0000","last_fullsized":"2024-04-20T11:27:00.186405+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:09:13.468036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.698358+0000","last_change":"2024-04-20T11:26:58.710488+0000","last_active":"2024-04-20T11:26:59.698358+0000","last_peered":"2024-04-20T11:26:59.698358+0000","last_clean":"2024-04-20T11:26:59.698358+0000","last_became_active":"2024-04-20T11:26:58.710179+0000","last_became_peered":"2024-04-20T11:26:58.710179+0000","last_unstale":"2024-04-20T11:26:59.698358+0000","last_undegraded":"2024-04-20T11:26:59.698358+0000","last_fullsized":"2024-04-20T11:26:59.698358+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:09:14.483077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.739503+0000","last_change":"2024-04-20T11:26:57.700540+0000","last_active":"2024-04-20T11:26:59.739503+0000","last_peered":"2024-04-20T11:26:59.739503+0000","last_clean":"2024-04-20T11:26:59.739503+0000","last_became_active":"2024-04-20T11:26:57.697881+0000","last_became_peered":"2024-04-20T11:26:57.697881+0000","last_unstale":"2024-04-20T11:26:59.739503+0000","last_undegraded":"2024-04-20T11:26:59.739503+0000","last_fullsized":"2024-04-20T11:26:59.739503+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:28:39.881689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.184257+0000","last_change":"2024-04-20T11:26:58.710228+0000","last_active":"2024-04-20T11:27:00.184257+0000","last_peered":"2024-04-20T11:27:00.184257+0000","last_clean":"2024-04-20T11:27:00.184257+0000","last_became_active":"2024-04-20T11:26:58.709942+0000","last_became_peered":"2024-04-20T11:26:58.709942+0000","last_unstale":"2024-04-20T11:27:00.184257+0000","last_undegraded":"2024-04-20T11:27:00.184257+0000","last_fullsized":"2024-04-20T11:27:00.184257+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:01:48.311110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.692978+0000","last_change":"2024-04-20T11:26:57.691897+0000","last_active":"2024-04-20T11:26:59.692978+0000","last_peered":"2024-04-20T11:26:59.692978+0000","last_clean":"2024-04-20T11:26:59.692978+0000","last_became_active":"2024-04-20T11:26:57.690649+0000","last_became_peered":"2024-04-20T11:26:57.690649+0000","last_unstale":"2024-04-20T11:26:59.692978+0000","last_undegraded":"2024-04-20T11:26:59.692978+0000","last_fullsized":"2024-04-20T11:26:59.692978+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:11:36.407427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.693551+0000","last_change":"2024-04-20T11:26:58.696119+0000","last_active":"2024-04-20T11:26:59.693551+0000","last_peered":"2024-04-20T11:26:59.693551+0000","last_clean":"2024-04-20T11:26:59.693551+0000","last_became_active":"2024-04-20T11:26:58.695849+0000","last_became_peered":"2024-04-20T11:26:58.695849+0000","last_unstale":"2024-04-20T11:26:59.693551+0000","last_undegraded":"2024-04-20T11:26:59.693551+0000","last_fullsized":"2024-04-20T11:26:59.693551+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:53:57.559834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.696904+0000","last_change":"2024-04-20T11:26:57.693789+0000","last_active":"2024-04-20T11:26:59.696904+0000","last_peered":"2024-04-20T11:26:59.696904+0000","last_clean":"2024-04-20T11:26:59.696904+0000","last_became_active":"2024-04-20T11:26:57.688385+0000","last_became_peered":"2024-04-20T11:26:57.688385+0000","last_unstale":"2024-04-20T11:26:59.696904+0000","last_undegraded":"2024-04-20T11:26:59.696904+0000","last_fullsized":"2024-04-20T11:26:59.696904+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:59:13.454290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.185068+0000","last_change":"2024-04-20T11:26:58.713408+0000","last_active":"2024-04-20T11:27:00.185068+0000","last_peered":"2024-04-20T11:27:00.185068+0000","last_clean":"2024-04-20T11:27:00.185068+0000","last_became_active":"2024-04-20T11:26:58.713119+0000","last_became_peered":"2024-04-20T11:26:58.713119+0000","last_unstale":"2024-04-20T11:27:00.185068+0000","last_undegraded":"2024-04-20T11:27:00.185068+0000","last_fullsized":"2024-04-20T11:27:00.185068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:12:24.274916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.738625+0000","last_change":"2024-04-20T11:26:57.695426+0000","last_active":"2024-04-20T11:26:59.738625+0000","last_peered":"2024-04-20T11:26:59.738625+0000","last_clean":"2024-04-20T11:26:59.738625+0000","last_became_active":"2024-04-20T11:26:57.695023+0000","last_became_peered":"2024-04-20T11:26:57.695023+0000","last_unstale":"2024-04-20T11:26:59.738625+0000","last_undegraded":"2024-04-20T11:26:59.738625+0000","last_fullsized":"2024-04-20T11:26:59.738625+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:03:42.590011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.739012+0000","last_change":"2024-04-20T11:26:58.695228+0000","last_active":"2024-04-20T11:26:59.739012+0000","last_peered":"2024-04-20T11:26:59.739012+0000","last_clean":"2024-04-20T11:26:59.739012+0000","last_became_active":"2024-04-20T11:26:58.694540+0000","last_became_peered":"2024-04-20T11:26:58.694540+0000","last_unstale":"2024-04-20T11:26:59.739012+0000","last_undegraded":"2024-04-20T11:26:59.739012+0000","last_fullsized":"2024-04-20T11:26:59.739012+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:47:21.129095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.182683+0000","last_change":"2024-04-20T11:26:57.679351+0000","last_active":"2024-04-20T11:27:00.182683+0000","last_peered":"2024-04-20T11:27:00.182683+0000","last_clean":"2024-04-20T11:27:00.182683+0000","last_became_active":"2024-04-20T11:26:57.678821+0000","last_became_peered":"2024-04-20T11:26:57.678821+0000","last_unstale":"2024-04-20T11:27:00.182683+0000","last_undegraded":"2024-04-20T11:27:00.182683+0000","last_fullsized":"2024-04-20T11:27:00.182683+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:28:39.881689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.184115+0000","last_change":"2024-04-20T11:26:58.713370+0000","last_active":"2024-04-20T11:27:00.184115+0000","last_peered":"2024-04-20T11:27:00.184115+0000","last_clean":"2024-04-20T11:27:00.184115+0000","last_became_active":"2024-04-20T11:26:58.713068+0000","last_became_peered":"2024-04-20T11:26:58.713068+0000","last_unstale":"2024-04-20T11:27:00.184115+0000","last_undegraded":"2024-04-20T11:27:00.184115+0000","last_fullsized":"2024-04-20T11:27:00.184115+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:55:47.019482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.695270+0000","last_change":"2024-04-20T11:26:57.687641+0000","last_active":"2024-04-20T11:26:59.695270+0000","last_peered":"2024-04-20T11:26:59.695270+0000","last_clean":"2024-04-20T11:26:59.695270+0000","last_became_active":"2024-04-20T11:26:57.687237+0000","last_became_peered":"2024-04-20T11:26:57.687237+0000","last_unstale":"2024-04-20T11:26:59.695270+0000","last_undegraded":"2024-04-20T11:26:59.695270+0000","last_fullsized":"2024-04-20T11:26:59.695270+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:43:12.722459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.184814+0000","last_change":"2024-04-20T11:26:58.711133+0000","last_active":"2024-04-20T11:27:00.184814+0000","last_peered":"2024-04-20T11:27:00.184814+0000","last_clean":"2024-04-20T11:27:00.184814+0000","last_became_active":"2024-04-20T11:26:58.710818+0000","last_became_peered":"2024-04-20T11:26:58.710818+0000","last_unstale":"2024-04-20T11:27:00.184814+0000","last_undegraded":"2024-04-20T11:27:00.184814+0000","last_fullsized":"2024-04-20T11:27:00.184814+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:27:35.003583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.185902+0000","last_change":"2024-04-20T11:26:57.675885+0000","last_active":"2024-04-20T11:27:00.185902+0000","last_peered":"2024-04-20T11:27:00.185902+0000","last_clean":"2024-04-20T11:27:00.185902+0000","last_became_active":"2024-04-20T11:26:57.675625+0000","last_became_peered":"2024-04-20T11:26:57.675625+0000","last_unstale":"2024-04-20T11:27:00.185902+0000","last_undegraded":"2024-04-20T11:27:00.185902+0000","last_fullsized":"2024-04-20T11:27:00.185902+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:03:42.590011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.185312+0000","last_change":"2024-04-20T11:26:58.713427+0000","last_active":"2024-04-20T11:27:00.185312+0000","last_peered":"2024-04-20T11:27:00.185312+0000","last_clean":"2024-04-20T11:27:00.185312+0000","last_became_active":"2024-04-20T11:26:58.713112+0000","last_became_peered":"2024-04-20T11:26:58.713112+0000","last_unstale":"2024-04-20T11:27:00.185312+0000","last_undegraded":"2024-04-20T11:27:00.185312+0000","last_fullsized":"2024-04-20T11:27:00.185312+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:26:45.461271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.186539+0000","last_change":"2024-04-20T11:26:57.693246+0000","last_active":"2024-04-20T11:27:00.186539+0000","last_peered":"2024-04-20T11:27:00.186539+0000","last_clean":"2024-04-20T11:27:00.186539+0000","last_became_active":"2024-04-20T11:26:57.692699+0000","last_became_peered":"2024-04-20T11:26:57.692699+0000","last_unstale":"2024-04-20T11:27:00.186539+0000","last_undegraded":"2024-04-20T11:27:00.186539+0000","last_fullsized":"2024-04-20T11:27:00.186539+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:22:14.490133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.698394+0000","last_change":"2024-04-20T11:26:58.676238+0000","last_active":"2024-04-20T11:26:59.698394+0000","last_peered":"2024-04-20T11:26:59.698394+0000","last_clean":"2024-04-20T11:26:59.698394+0000","last_became_active":"2024-04-20T11:26:58.675427+0000","last_became_peered":"2024-04-20T11:26:58.675427+0000","last_unstale":"2024-04-20T11:26:59.698394+0000","last_undegraded":"2024-04-20T11:26:59.698394+0000","last_fullsized":"2024-04-20T11:26:59.698394+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:28:57.233788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.185685+0000","last_change":"2024-04-20T11:26:57.681076+0000","last_active":"2024-04-20T11:27:00.185685+0000","last_peered":"2024-04-20T11:27:00.185685+0000","last_clean":"2024-04-20T11:27:00.185685+0000","last_became_active":"2024-04-20T11:26:57.680806+0000","last_became_peered":"2024-04-20T11:26:57.680806+0000","last_unstale":"2024-04-20T11:27:00.185685+0000","last_undegraded":"2024-04-20T11:27:00.185685+0000","last_fullsized":"2024-04-20T11:27:00.185685+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:11:19.111199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.184412+0000","last_change":"2024-04-20T11:26:58.708666+0000","last_active":"2024-04-20T11:27:00.184412+0000","last_peered":"2024-04-20T11:27:00.184412+0000","last_clean":"2024-04-20T11:27:00.184412+0000","last_became_active":"2024-04-20T11:26:58.708362+0000","last_became_peered":"2024-04-20T11:26:58.708362+0000","last_unstale":"2024-04-20T11:27:00.184412+0000","last_undegraded":"2024-04-20T11:27:00.184412+0000","last_fullsized":"2024-04-20T11:27:00.184412+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:12:41.134688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.184117+0000","last_change":"2024-04-20T11:26:48.661897+0000","last_active":"2024-04-20T11:27:00.184117+0000","last_peered":"2024-04-20T11:27:00.184117+0000","last_clean":"2024-04-20T11:27:00.184117+0000","last_became_active":"2024-04-20T11:26:48.656535+0000","last_became_peered":"2024-04-20T11:26:48.656535+0000","last_unstale":"2024-04-20T11:27:00.184117+0000","last_undegraded":"2024-04-20T11:27:00.184117+0000","last_fullsized":"2024-04-20T11:27:00.184117+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:47.543004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:47.543004+0000","last_clean_scrub_stamp":"2024-04-20T11:26:47.543004+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:58:35.527105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.694066+0000","last_change":"2024-04-20T11:26:57.680902+0000","last_active":"2024-04-20T11:26:59.694066+0000","last_peered":"2024-04-20T11:26:59.694066+0000","last_clean":"2024-04-20T11:26:59.694066+0000","last_became_active":"2024-04-20T11:26:57.680506+0000","last_became_peered":"2024-04-20T11:26:57.680506+0000","last_unstale":"2024-04-20T11:26:59.694066+0000","last_undegraded":"2024-04-20T11:26:59.694066+0000","last_fullsized":"2024-04-20T11:26:59.694066+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:28:39.881689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.185222+0000","last_change":"2024-04-20T11:26:58.697905+0000","last_active":"2024-04-20T11:27:00.185222+0000","last_peered":"2024-04-20T11:27:00.185222+0000","last_clean":"2024-04-20T11:27:00.185222+0000","last_became_active":"2024-04-20T11:26:58.697612+0000","last_became_peered":"2024-04-20T11:26:58.697612+0000","last_unstale":"2024-04-20T11:27:00.185222+0000","last_undegraded":"2024-04-20T11:27:00.185222+0000","last_fullsized":"2024-04-20T11:27:00.185222+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:30:01.981489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.183754+0000","last_change":"2024-04-20T11:26:57.697322+0000","last_active":"2024-04-20T11:27:00.183754+0000","last_peered":"2024-04-20T11:27:00.183754+0000","last_clean":"2024-04-20T11:27:00.183754+0000","last_became_active":"2024-04-20T11:26:57.696775+0000","last_became_peered":"2024-04-20T11:26:57.696775+0000","last_unstale":"2024-04-20T11:27:00.183754+0000","last_undegraded":"2024-04-20T11:27:00.183754+0000","last_fullsized":"2024-04-20T11:27:00.183754+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:30:34.827026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.692465+0000","last_change":"2024-04-20T11:26:58.690752+0000","last_active":"2024-04-20T11:26:59.692465+0000","last_peered":"2024-04-20T11:26:59.692465+0000","last_clean":"2024-04-20T11:26:59.692465+0000","last_became_active":"2024-04-20T11:26:58.690451+0000","last_became_peered":"2024-04-20T11:26:58.690451+0000","last_unstale":"2024-04-20T11:26:59.692465+0000","last_undegraded":"2024-04-20T11:26:59.692465+0000","last_fullsized":"2024-04-20T11:26:59.692465+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:33:15.328953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.184689+0000","last_change":"2024-04-20T11:26:57.680139+0000","last_active":"2024-04-20T11:27:00.184689+0000","last_peered":"2024-04-20T11:27:00.184689+0000","last_clean":"2024-04-20T11:27:00.184689+0000","last_became_active":"2024-04-20T11:26:57.679820+0000","last_became_peered":"2024-04-20T11:26:57.679820+0000","last_unstale":"2024-04-20T11:27:00.184689+0000","last_undegraded":"2024-04-20T11:27:00.184689+0000","last_fullsized":"2024-04-20T11:27:00.184689+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:35:40.489733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.184019+0000","last_change":"2024-04-20T11:26:58.677911+0000","last_active":"2024-04-20T11:27:00.184019+0000","last_peered":"2024-04-20T11:27:00.184019+0000","last_clean":"2024-04-20T11:27:00.184019+0000","last_became_active":"2024-04-20T11:26:58.677645+0000","last_became_peered":"2024-04-20T11:26:58.677645+0000","last_unstale":"2024-04-20T11:27:00.184019+0000","last_undegraded":"2024-04-20T11:27:00.184019+0000","last_fullsized":"2024-04-20T11:27:00.184019+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:01:06.520823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.697092+0000","last_change":"2024-04-20T11:26:57.676331+0000","last_active":"2024-04-20T11:26:59.697092+0000","last_peered":"2024-04-20T11:26:59.697092+0000","last_clean":"2024-04-20T11:26:59.697092+0000","last_became_active":"2024-04-20T11:26:57.675643+0000","last_became_peered":"2024-04-20T11:26:57.675643+0000","last_unstale":"2024-04-20T11:26:59.697092+0000","last_undegraded":"2024-04-20T11:26:59.697092+0000","last_fullsized":"2024-04-20T11:26:59.697092+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:11:19.111199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.697040+0000","last_change":"2024-04-20T11:26:58.701514+0000","last_active":"2024-04-20T11:26:59.697040+0000","last_peered":"2024-04-20T11:26:59.697040+0000","last_clean":"2024-04-20T11:26:59.697040+0000","last_became_active":"2024-04-20T11:26:58.692435+0000","last_became_peered":"2024-04-20T11:26:58.692435+0000","last_unstale":"2024-04-20T11:26:59.697040+0000","last_undegraded":"2024-04-20T11:26:59.697040+0000","last_fullsized":"2024-04-20T11:26:59.697040+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:56:23.896778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.185970+0000","last_change":"2024-04-20T11:26:57.681965+0000","last_active":"2024-04-20T11:27:00.185970+0000","last_peered":"2024-04-20T11:27:00.185970+0000","last_clean":"2024-04-20T11:27:00.185970+0000","last_became_active":"2024-04-20T11:26:57.681130+0000","last_became_peered":"2024-04-20T11:26:57.681130+0000","last_unstale":"2024-04-20T11:27:00.185970+0000","last_undegraded":"2024-04-20T11:27:00.185970+0000","last_fullsized":"2024-04-20T11:27:00.185970+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:59:13.454290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.695098+0000","last_change":"2024-04-20T11:26:58.687674+0000","last_active":"2024-04-20T11:26:59.695098+0000","last_peered":"2024-04-20T11:26:59.695098+0000","last_clean":"2024-04-20T11:26:59.695098+0000","last_became_active":"2024-04-20T11:26:58.687373+0000","last_became_peered":"2024-04-20T11:26:58.687373+0000","last_unstale":"2024-04-20T11:26:59.695098+0000","last_undegraded":"2024-04-20T11:26:59.695098+0000","last_fullsized":"2024-04-20T11:26:59.695098+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:30:01.981489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.184744+0000","last_change":"2024-04-20T11:26:57.700654+0000","last_active":"2024-04-20T11:27:00.184744+0000","last_peered":"2024-04-20T11:27:00.184744+0000","last_clean":"2024-04-20T11:27:00.184744+0000","last_became_active":"2024-04-20T11:26:57.698785+0000","last_became_peered":"2024-04-20T11:26:57.698785+0000","last_unstale":"2024-04-20T11:27:00.184744+0000","last_undegraded":"2024-04-20T11:27:00.184744+0000","last_fullsized":"2024-04-20T11:27:00.184744+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:39:03.501101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.185902+0000","last_change":"2024-04-20T11:26:58.680824+0000","last_active":"2024-04-20T11:27:00.185902+0000","last_peered":"2024-04-20T11:27:00.185902+0000","last_clean":"2024-04-20T11:27:00.185902+0000","last_became_active":"2024-04-20T11:26:58.680518+0000","last_became_peered":"2024-04-20T11:26:58.680518+0000","last_unstale":"2024-04-20T11:27:00.185902+0000","last_undegraded":"2024-04-20T11:27:00.185902+0000","last_fullsized":"2024-04-20T11:27:00.185902+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:01:34.618021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.187052+0000","last_change":"2024-04-20T11:26:57.695120+0000","last_active":"2024-04-20T11:27:00.187052+0000","last_peered":"2024-04-20T11:27:00.187052+0000","last_clean":"2024-04-20T11:27:00.187052+0000","last_became_active":"2024-04-20T11:26:57.694850+0000","last_became_peered":"2024-04-20T11:26:57.694850+0000","last_unstale":"2024-04-20T11:27:00.187052+0000","last_undegraded":"2024-04-20T11:27:00.187052+0000","last_fullsized":"2024-04-20T11:27:00.187052+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:56:22.881738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.186628+0000","last_change":"2024-04-20T11:26:58.704801+0000","last_active":"2024-04-20T11:27:00.186628+0000","last_peered":"2024-04-20T11:27:00.186628+0000","last_clean":"2024-04-20T11:27:00.186628+0000","last_became_active":"2024-04-20T11:26:58.703638+0000","last_became_peered":"2024-04-20T11:26:58.703638+0000","last_unstale":"2024-04-20T11:27:00.186628+0000","last_undegraded":"2024-04-20T11:27:00.186628+0000","last_fullsized":"2024-04-20T11:27:00.186628+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:55:47.019482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.739746+0000","last_change":"2024-04-20T11:26:57.702231+0000","last_active":"2024-04-20T11:26:59.739746+0000","last_peered":"2024-04-20T11:26:59.739746+0000","last_clean":"2024-04-20T11:26:59.739746+0000","last_became_active":"2024-04-20T11:26:57.701649+0000","last_became_peered":"2024-04-20T11:26:57.701649+0000","last_unstale":"2024-04-20T11:26:59.739746+0000","last_undegraded":"2024-04-20T11:26:59.739746+0000","last_fullsized":"2024-04-20T11:26:59.739746+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:59:13.454290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.693158+0000","last_change":"2024-04-20T11:26:58.685253+0000","last_active":"2024-04-20T11:26:59.693158+0000","last_peered":"2024-04-20T11:26:59.693158+0000","last_clean":"2024-04-20T11:26:59.693158+0000","last_became_active":"2024-04-20T11:26:58.684993+0000","last_became_peered":"2024-04-20T11:26:58.684993+0000","last_unstale":"2024-04-20T11:26:59.693158+0000","last_undegraded":"2024-04-20T11:26:59.693158+0000","last_fullsized":"2024-04-20T11:26:59.693158+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:02:15.732106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.184446+0000","last_change":"2024-04-20T11:26:57.697065+0000","last_active":"2024-04-20T11:27:00.184446+0000","last_peered":"2024-04-20T11:27:00.184446+0000","last_clean":"2024-04-20T11:27:00.184446+0000","last_became_active":"2024-04-20T11:26:57.696461+0000","last_became_peered":"2024-04-20T11:26:57.696461+0000","last_unstale":"2024-04-20T11:27:00.184446+0000","last_undegraded":"2024-04-20T11:27:00.184446+0000","last_fullsized":"2024-04-20T11:27:00.184446+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:56:22.881738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.695053+0000","last_change":"2024-04-20T11:26:58.685057+0000","last_active":"2024-04-20T11:26:59.695053+0000","last_peered":"2024-04-20T11:26:59.695053+0000","last_clean":"2024-04-20T11:26:59.695053+0000","last_became_active":"2024-04-20T11:26:58.684493+0000","last_became_peered":"2024-04-20T11:26:58.684493+0000","last_unstale":"2024-04-20T11:26:59.695053+0000","last_undegraded":"2024-04-20T11:26:59.695053+0000","last_fullsized":"2024-04-20T11:26:59.695053+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:59:14.469331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.185352+0000","last_change":"2024-04-20T11:26:57.690472+0000","last_active":"2024-04-20T11:27:00.185352+0000","last_peered":"2024-04-20T11:27:00.185352+0000","last_clean":"2024-04-20T11:27:00.185352+0000","last_became_active":"2024-04-20T11:26:57.689786+0000","last_became_peered":"2024-04-20T11:26:57.689786+0000","last_unstale":"2024-04-20T11:27:00.185352+0000","last_undegraded":"2024-04-20T11:27:00.185352+0000","last_fullsized":"2024-04-20T11:27:00.185352+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:30:34.827026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.184283+0000","last_change":"2024-04-20T11:26:58.677827+0000","last_active":"2024-04-20T11:27:00.184283+0000","last_peered":"2024-04-20T11:27:00.184283+0000","last_clean":"2024-04-20T11:27:00.184283+0000","last_became_active":"2024-04-20T11:26:58.677521+0000","last_became_peered":"2024-04-20T11:26:58.677521+0000","last_unstale":"2024-04-20T11:27:00.184283+0000","last_undegraded":"2024-04-20T11:27:00.184283+0000","last_fullsized":"2024-04-20T11:27:00.184283+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:02:15.732106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.185683+0000","last_change":"2024-04-20T11:26:57.673530+0000","last_active":"2024-04-20T11:27:00.185683+0000","last_peered":"2024-04-20T11:27:00.185683+0000","last_clean":"2024-04-20T11:27:00.185683+0000","last_became_active":"2024-04-20T11:26:57.671513+0000","last_became_peered":"2024-04-20T11:26:57.671513+0000","last_unstale":"2024-04-20T11:27:00.185683+0000","last_undegraded":"2024-04-20T11:27:00.185683+0000","last_fullsized":"2024-04-20T11:27:00.185683+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:11:19.111199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.185170+0000","last_change":"2024-04-20T11:26:58.702869+0000","last_active":"2024-04-20T11:27:00.185170+0000","last_peered":"2024-04-20T11:27:00.185170+0000","last_clean":"2024-04-20T11:27:00.185170+0000","last_became_active":"2024-04-20T11:26:58.698423+0000","last_became_peered":"2024-04-20T11:26:58.698423+0000","last_unstale":"2024-04-20T11:27:00.185170+0000","last_undegraded":"2024-04-20T11:27:00.185170+0000","last_fullsized":"2024-04-20T11:27:00.185170+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:35:40.422299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.185236+0000","last_change":"2024-04-20T11:26:57.699410+0000","last_active":"2024-04-20T11:27:00.185236+0000","last_peered":"2024-04-20T11:27:00.185236+0000","last_clean":"2024-04-20T11:27:00.185236+0000","last_became_active":"2024-04-20T11:26:57.697630+0000","last_became_peered":"2024-04-20T11:26:57.697630+0000","last_unstale":"2024-04-20T11:27:00.185236+0000","last_undegraded":"2024-04-20T11:27:00.185236+0000","last_fullsized":"2024-04-20T11:27:00.185236+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:18:30.073624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.738917+0000","last_change":"2024-04-20T11:26:58.711494+0000","last_active":"2024-04-20T11:26:59.738917+0000","last_peered":"2024-04-20T11:26:59.738917+0000","last_clean":"2024-04-20T11:26:59.738917+0000","last_became_active":"2024-04-20T11:26:58.711116+0000","last_became_peered":"2024-04-20T11:26:58.711116+0000","last_unstale":"2024-04-20T11:26:59.738917+0000","last_undegraded":"2024-04-20T11:26:59.738917+0000","last_fullsized":"2024-04-20T11:26:59.738917+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:43:20.032648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.695487+0000","last_change":"2024-04-20T11:26:58.701576+0000","last_active":"2024-04-20T11:26:59.695487+0000","last_peered":"2024-04-20T11:26:59.695487+0000","last_clean":"2024-04-20T11:26:59.695487+0000","last_became_active":"2024-04-20T11:26:58.701222+0000","last_became_peered":"2024-04-20T11:26:58.701222+0000","last_unstale":"2024-04-20T11:26:59.695487+0000","last_undegraded":"2024-04-20T11:26:59.695487+0000","last_fullsized":"2024-04-20T11:26:59.695487+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:55:47.019482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.184217+0000","last_change":"2024-04-20T11:26:57.701154+0000","last_active":"2024-04-20T11:27:00.184217+0000","last_peered":"2024-04-20T11:27:00.184217+0000","last_clean":"2024-04-20T11:27:00.184217+0000","last_became_active":"2024-04-20T11:26:57.700716+0000","last_became_peered":"2024-04-20T11:26:57.700716+0000","last_unstale":"2024-04-20T11:27:00.184217+0000","last_undegraded":"2024-04-20T11:27:00.184217+0000","last_fullsized":"2024-04-20T11:27:00.184217+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:09:13.468036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.698090+0000","last_change":"2024-04-20T11:26:58.706275+0000","last_active":"2024-04-20T11:26:59.698090+0000","last_peered":"2024-04-20T11:26:59.698090+0000","last_clean":"2024-04-20T11:26:59.698090+0000","last_became_active":"2024-04-20T11:26:58.705922+0000","last_became_peered":"2024-04-20T11:26:58.705922+0000","last_unstale":"2024-04-20T11:26:59.698090+0000","last_undegraded":"2024-04-20T11:26:59.698090+0000","last_fullsized":"2024-04-20T11:26:59.698090+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:40:22.418833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.184419+0000","last_change":"2024-04-20T11:26:57.701304+0000","last_active":"2024-04-20T11:27:00.184419+0000","last_peered":"2024-04-20T11:27:00.184419+0000","last_clean":"2024-04-20T11:27:00.184419+0000","last_became_active":"2024-04-20T11:26:57.700620+0000","last_became_peered":"2024-04-20T11:26:57.700620+0000","last_unstale":"2024-04-20T11:27:00.184419+0000","last_undegraded":"2024-04-20T11:27:00.184419+0000","last_fullsized":"2024-04-20T11:27:00.184419+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:33:14.313913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.693201+0000","last_change":"2024-04-20T11:26:58.688125+0000","last_active":"2024-04-20T11:26:59.693201+0000","last_peered":"2024-04-20T11:26:59.693201+0000","last_clean":"2024-04-20T11:26:59.693201+0000","last_became_active":"2024-04-20T11:26:58.687917+0000","last_became_peered":"2024-04-20T11:26:58.687917+0000","last_unstale":"2024-04-20T11:26:59.693201+0000","last_undegraded":"2024-04-20T11:26:59.693201+0000","last_fullsized":"2024-04-20T11:26:59.693201+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:01:06.520823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.185078+0000","last_change":"2024-04-20T11:26:57.671052+0000","last_active":"2024-04-20T11:27:00.185078+0000","last_peered":"2024-04-20T11:27:00.185078+0000","last_clean":"2024-04-20T11:27:00.185078+0000","last_became_active":"2024-04-20T11:26:57.670620+0000","last_became_peered":"2024-04-20T11:26:57.670620+0000","last_unstale":"2024-04-20T11:27:00.185078+0000","last_undegraded":"2024-04-20T11:27:00.185078+0000","last_fullsized":"2024-04-20T11:27:00.185078+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:11:36.407427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.184939+0000","last_change":"2024-04-20T11:26:58.701127+0000","last_active":"2024-04-20T11:27:00.184939+0000","last_peered":"2024-04-20T11:27:00.184939+0000","last_clean":"2024-04-20T11:27:00.184939+0000","last_became_active":"2024-04-20T11:26:58.700948+0000","last_became_peered":"2024-04-20T11:26:58.700948+0000","last_unstale":"2024-04-20T11:27:00.184939+0000","last_undegraded":"2024-04-20T11:27:00.184939+0000","last_fullsized":"2024-04-20T11:27:00.184939+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:01:06.520823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.693275+0000","last_change":"2024-04-20T11:26:57.692256+0000","last_active":"2024-04-20T11:26:59.693275+0000","last_peered":"2024-04-20T11:26:59.693275+0000","last_clean":"2024-04-20T11:26:59.693275+0000","last_became_active":"2024-04-20T11:26:57.691929+0000","last_became_peered":"2024-04-20T11:26:57.691929+0000","last_unstale":"2024-04-20T11:26:59.693275+0000","last_undegraded":"2024-04-20T11:26:59.693275+0000","last_fullsized":"2024-04-20T11:26:59.693275+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:34:59.927980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T11:27:03.843 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-20T11:27:03.843 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-20T11:27:03.843 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-20T11:27:03.844 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-20T11:27:03.844 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-20T11:27:04.078 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.078+0000 7fc805a00640 1 Processor -- start 2024-04-20T11:27:04.078 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.079+0000 7fc805a00640 1 -- start start 2024-04-20T11:27:04.080 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.080+0000 7fc805a00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fc8001603d0 0x7fc8000b5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:04.080 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.080+0000 7fc805a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fc8000b5480 0x7fc8000b63f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:04.080 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.080+0000 7fc805a00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8000d3350 con 0x7fc80015fcc0 2024-04-20T11:27:04.080 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.080+0000 7fc805a00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fc8000d3630 con 0x7fc8000b5480 2024-04-20T11:27:04.080 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.080+0000 7fc805a00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fc8000d3910 con 0x7fc8001603d0 2024-04-20T11:27:04.080 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.080+0000 7fc805000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fc8000b5480 0x7fc8000b63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:04.080 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.080+0000 7fc805000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fc8000b5480 0x7fc8000b63f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:42660/0 (socket says 172.21.15.152:42660) 2024-04-20T11:27:04.080 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.080+0000 7fc805000640 1 -- 172.21.15.152:0/2241339127 learned_addr learned my addr 172.21.15.152:0/2241339127 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:04.080 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.081+0000 7fc7ff400640 1 --2- 172.21.15.152:0/2241339127 >> v2:172.21.15.175:3300/0 conn(0x7fc8001603d0 0x7fc8000b5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:04.081 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.081+0000 7fc805000640 1 -- 172.21.15.152:0/2241339127 >> v2:172.21.15.175:3300/0 conn(0x7fc8001603d0 msgr2=0x7fc8000b5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:04.081 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.081+0000 7fc805000640 1 --2- 172.21.15.152:0/2241339127 >> v2:172.21.15.175:3300/0 conn(0x7fc8001603d0 0x7fc8000b5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:04.081 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.081+0000 7fc805000640 1 -- 172.21.15.152:0/2241339127 >> v1:172.21.15.175:6789/0 conn(0x7fc80015fcc0 legacy=0x7fc8000d4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:04.081 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.081+0000 7fc805000640 1 -- 172.21.15.152:0/2241339127 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8000ced20 con 0x7fc8000b5480 2024-04-20T11:27:04.082 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.082+0000 7fc805000640 1 --2- 172.21.15.152:0/2241339127 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc8000b5480 0x7fc8000b63f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc80008ca40 tx=0x7fc7f802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b65c8dc79350da6a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:04.082 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.082+0000 7fc7fea00640 1 -- 172.21.15.152:0/2241339127 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f8035020 con 0x7fc8000b5480 2024-04-20T11:27:04.082 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.082+0000 7fc7fea00640 1 -- 172.21.15.152:0/2241339127 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7f8031870 con 0x7fc8000b5480 2024-04-20T11:27:04.082 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.082+0000 7fc7fea00640 1 -- 172.21.15.152:0/2241339127 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f8031c40 con 0x7fc8000b5480 2024-04-20T11:27:04.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.083+0000 7fc805a00640 1 -- 172.21.15.152:0/2241339127 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc8000b5480 msgr2=0x7fc8000b63f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:04.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.083+0000 7fc805a00640 1 --2- 172.21.15.152:0/2241339127 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc8000b5480 0x7fc8000b63f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc80008ca40 tx=0x7fc7f802b910 comp rx=0 tx=0).stop 2024-04-20T11:27:04.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.083+0000 7fc805a00640 1 -- 172.21.15.152:0/2241339127 shutdown_connections 2024-04-20T11:27:04.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.083+0000 7fc805a00640 1 --2- 172.21.15.152:0/2241339127 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc8000b5480 0x7fc8000b63f0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:04.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.083+0000 7fc805a00640 1 --2- 172.21.15.152:0/2241339127 >> v2:172.21.15.175:3300/0 conn(0x7fc8001603d0 0x7fc8000b5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:04.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.083+0000 7fc805a00640 1 -- 172.21.15.152:0/2241339127 >> 172.21.15.152:0/2241339127 conn(0x7fc80008bfd0 msgr2=0x7fc8000d4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:04.084 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.083+0000 7fc805a00640 1 -- 172.21.15.152:0/2241339127 shutdown_connections 2024-04-20T11:27:04.084 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.083+0000 7fc805a00640 1 -- 172.21.15.152:0/2241339127 wait complete. 2024-04-20T11:27:04.084 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.085+0000 7fc805a00640 1 Processor -- start 2024-04-20T11:27:04.084 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.085+0000 7fc805a00640 1 -- start start 2024-04-20T11:27:04.086 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.086+0000 7fc805a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8000b5480 0x7fc80014c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:04.086 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.086+0000 7fc805a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc80015fcc0 0x7fc8000ca700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:04.087 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.086+0000 7fc805a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc8001603d0 0x7fc80015e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:04.087 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.086+0000 7fc805a00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fc8000cf610 con 0x7fc8001603d0 2024-04-20T11:27:04.087 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.086+0000 7fc805a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc8000cf490 con 0x7fc8000b5480 2024-04-20T11:27:04.087 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.086+0000 7fc805a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc8000cf790 con 0x7fc80015fcc0 2024-04-20T11:27:04.087 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.086+0000 7fc805000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc8001603d0 0x7fc80015e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:04.087 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.087+0000 7fc805000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc8001603d0 0x7fc80015e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:42662/0 (socket says 172.21.15.152:42662) 2024-04-20T11:27:04.087 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.087+0000 7fc805000640 1 -- 172.21.15.152:0/2015098869 learned_addr learned my addr 172.21.15.152:0/2015098869 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:04.087 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.087+0000 7fc7ffe00640 1 --2- 172.21.15.152:0/2015098869 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8000b5480 0x7fc80014c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:04.087 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.087+0000 7fc7ff400640 1 --2- 172.21.15.152:0/2015098869 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc80015fcc0 0x7fc8000ca700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:04.087 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.087+0000 7fc805000640 1 -- 172.21.15.152:0/2015098869 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc80015fcc0 msgr2=0x7fc8000ca700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:04.087 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.087+0000 7fc805000640 1 --2- 172.21.15.152:0/2015098869 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc80015fcc0 0x7fc8000ca700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:04.087 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.087+0000 7fc805000640 1 -- 172.21.15.152:0/2015098869 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8000b5480 msgr2=0x7fc80014c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:04.087 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.087+0000 7fc805000640 1 --2- 172.21.15.152:0/2015098869 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8000b5480 0x7fc80014c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:04.088 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.087+0000 7fc805000640 1 -- 172.21.15.152:0/2015098869 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8000c9440 con 0x7fc8001603d0 2024-04-20T11:27:04.088 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.087+0000 7fc7ffe00640 1 --2- 172.21.15.152:0/2015098869 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8000b5480 0x7fc80014c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:27:04.088 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.087+0000 7fc805000640 1 --2- 172.21.15.152:0/2015098869 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc8001603d0 0x7fc80015e430 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc7f802b950 tx=0x7fc7f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:04.088 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.088+0000 7fc7f7e00640 1 -- 172.21.15.152:0/2015098869 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f8040070 con 0x7fc8001603d0 2024-04-20T11:27:04.088 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.088+0000 7fc7f7e00640 1 -- 172.21.15.152:0/2015098869 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7f8030d00 con 0x7fc8001603d0 2024-04-20T11:27:04.089 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.088+0000 7fc7f7e00640 1 -- 172.21.15.152:0/2015098869 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f8035040 con 0x7fc8001603d0 2024-04-20T11:27:04.089 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.088+0000 7fc805a00640 1 -- 172.21.15.152:0/2015098869 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8000c9720 con 0x7fc8001603d0 2024-04-20T11:27:04.089 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.088+0000 7fc805a00640 1 -- 172.21.15.152:0/2015098869 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8000c9ce0 con 0x7fc8001603d0 2024-04-20T11:27:04.089 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.089+0000 7fc805a00640 1 -- 172.21.15.152:0/2015098869 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc800075e60 con 0x7fc8001603d0 2024-04-20T11:27:04.089 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.090+0000 7fc7f7e00640 1 -- 172.21.15.152:0/2015098869 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc7f8038020 con 0x7fc8001603d0 2024-04-20T11:27:04.090 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.091+0000 7fc7f7e00640 1 --2- 172.21.15.152:0/2015098869 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc7dc06c920 0x7fc7dc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:04.090 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.091+0000 7fc7f7e00640 1 -- 172.21.15.152:0/2015098869 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc7f80ad890 con 0x7fc8001603d0 2024-04-20T11:27:04.090 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.091+0000 7fc7ffe00640 1 --2- 172.21.15.152:0/2015098869 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc7dc06c920 0x7fc7dc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:04.091 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.092+0000 7fc7ffe00640 1 --2- 172.21.15.152:0/2015098869 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc7dc06c920 0x7fc7dc06ede0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc7e8003eb0 tx=0x7fc7e8009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:04.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.096+0000 7fc7f7e00640 1 -- 172.21.15.152:0/2015098869 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fc7f807bcb0 con 0x7fc8001603d0 2024-04-20T11:27:04.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.340+0000 7fc805a00640 1 -- 172.21.15.152:0/2015098869 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fc8000ba4d0 con 0x7fc8001603d0 2024-04-20T11:27:04.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.349+0000 7fc7f7e00640 1 -- 172.21.15.152:0/2015098869 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+174143 (secure 0 0 0) 0x7fc7f807b5b0 con 0x7fc8001603d0 2024-04-20T11:27:04.352 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:27:04.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.353+0000 7fc805a00640 1 -- 172.21.15.152:0/2015098869 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc7dc06c920 msgr2=0x7fc7dc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:04.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.353+0000 7fc805a00640 1 --2- 172.21.15.152:0/2015098869 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc7dc06c920 0x7fc7dc06ede0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc7e8003eb0 tx=0x7fc7e8009040 comp rx=0 tx=0).stop 2024-04-20T11:27:04.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.353+0000 7fc805a00640 1 -- 172.21.15.152:0/2015098869 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc8001603d0 msgr2=0x7fc80015e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:04.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.353+0000 7fc805a00640 1 --2- 172.21.15.152:0/2015098869 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc8001603d0 0x7fc80015e430 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc7f802b950 tx=0x7fc7f80046c0 comp rx=0 tx=0).stop 2024-04-20T11:27:04.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.354+0000 7fc805a00640 1 -- 172.21.15.152:0/2015098869 shutdown_connections 2024-04-20T11:27:04.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.354+0000 7fc805a00640 1 --2- 172.21.15.152:0/2015098869 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc7dc06c920 0x7fc7dc06ede0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:04.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.354+0000 7fc805a00640 1 --2- 172.21.15.152:0/2015098869 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc8001603d0 0x7fc80015e430 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:04.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.354+0000 7fc805a00640 1 --2- 172.21.15.152:0/2015098869 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc80015fcc0 0x7fc8000ca700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:04.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.354+0000 7fc805a00640 1 --2- 172.21.15.152:0/2015098869 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8000b5480 0x7fc80014c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:04.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.354+0000 7fc805a00640 1 -- 172.21.15.152:0/2015098869 >> 172.21.15.152:0/2015098869 conn(0x7fc80008bfd0 msgr2=0x7fc8000d4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:04.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.354+0000 7fc805a00640 1 -- 172.21.15.152:0/2015098869 shutdown_connections 2024-04-20T11:27:04.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.354+0000 7fc805a00640 1 -- 172.21.15.152:0/2015098869 wait complete. 2024-04-20T11:27:04.369 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":5,"flags":0,"active_gid":4113,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":2285110240},{"type":"v1","addr":"172.21.15.152:6833","nonce":2285110240}]},"active_addr":"172.21.15.152:6833/2285110240","active_change":"2024-04-20T11:26:39.191554+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4109,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.152:0","nonce":2510881178}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.152:0","nonce":2828322453}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.152:0","nonce":698943652}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.152:0","nonce":2184739958}]}]} 2024-04-20T11:27:04.373 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-20T11:27:04.373 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-20T11:27:04.373 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:27:04.559 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.560+0000 7f9df5600640 1 Processor -- start 2024-04-20T11:27:04.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.560+0000 7f9df5600640 1 -- start start 2024-04-20T11:27:04.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.561+0000 7f9df5600640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f9df0145f60 0x7f9df006ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:04.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.561+0000 7f9df5600640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9df0146340 con 0x7f9df00cd0d0 2024-04-20T11:27:04.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.562+0000 7f9df5600640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9df006f3c0 con 0x7f9df00726f0 2024-04-20T11:27:04.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.562+0000 7f9df5600640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f9df006e9f0 con 0x7f9df0145f60 2024-04-20T11:27:04.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.562+0000 7f9deea00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f9df0145f60 0x7f9df006ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:04.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.562+0000 7f9dee000640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f9df00cd0d0 0x7f9df0071fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:40600/0 (socket says 172.21.15.152:40600) 2024-04-20T11:27:04.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.562+0000 7f9dee000640 1 -- 172.21.15.152:0/1382626429 learned_addr learned my addr 172.21.15.152:0/1382626429 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:27:04.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.562+0000 7f9deea00640 1 -- 172.21.15.152:0/1382626429 >> v1:172.21.15.175:6789/0 conn(0x7f9df00726f0 legacy=0x7f9df00cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:04.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.562+0000 7f9deea00640 1 -- 172.21.15.152:0/1382626429 >> v1:172.21.15.152:6789/0 conn(0x7f9df00cd0d0 legacy=0x7f9df0071fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:04.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.562+0000 7f9deea00640 1 -- 172.21.15.152:0/1382626429 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9df0072d50 con 0x7f9df0145f60 2024-04-20T11:27:04.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.563+0000 7f9deea00640 1 --2- 172.21.15.152:0/1382626429 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9df0145f60 0x7f9df006ee80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9de00097a0 tx=0x7f9de002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4bee7b1efc2dcb server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:04.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.564+0000 7f9ded600640 1 -- 172.21.15.152:0/1382626429 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9de0035020 con 0x7f9df0145f60 2024-04-20T11:27:04.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.564+0000 7f9ded600640 1 -- 172.21.15.152:0/1382626429 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9de0031830 con 0x7f9df0145f60 2024-04-20T11:27:04.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.564+0000 7f9ded600640 1 -- 172.21.15.152:0/1382626429 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9de0031c00 con 0x7f9df0145f60 2024-04-20T11:27:04.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.564+0000 7f9df5600640 1 -- 172.21.15.152:0/1382626429 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9df0145f60 msgr2=0x7f9df006ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:04.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.564+0000 7f9df5600640 1 --2- 172.21.15.152:0/1382626429 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9df0145f60 0x7f9df006ee80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9de00097a0 tx=0x7f9de002b8d0 comp rx=0 tx=0).stop 2024-04-20T11:27:04.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.564+0000 7f9df5600640 1 -- 172.21.15.152:0/1382626429 shutdown_connections 2024-04-20T11:27:04.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.564+0000 7f9df5600640 1 --2- 172.21.15.152:0/1382626429 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9df0145f60 0x7f9df006ee80 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:04.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.564+0000 7f9df5600640 1 -- 172.21.15.152:0/1382626429 >> 172.21.15.152:0/1382626429 conn(0x7f9df0067cf0 msgr2=0x7f9df006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:04.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.564+0000 7f9df5600640 1 -- 172.21.15.152:0/1382626429 shutdown_connections 2024-04-20T11:27:04.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.564+0000 7f9df5600640 1 -- 172.21.15.152:0/1382626429 wait complete. 2024-04-20T11:27:04.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.566+0000 7f9df5600640 1 Processor -- start 2024-04-20T11:27:04.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.566+0000 7f9df5600640 1 -- start start 2024-04-20T11:27:04.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.567+0000 7f9df5600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9df00726f0 0x7f9df00c76d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:04.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.567+0000 7f9df5600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9df00cd0d0 0x7f9df00b3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:04.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.567+0000 7f9df5600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9df0145f60 0x7f9df00b4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:04.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.567+0000 7f9df5600640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f9df006e680 con 0x7f9df00726f0 2024-04-20T11:27:04.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.567+0000 7f9df5600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9df006e3a0 con 0x7f9df0145f60 2024-04-20T11:27:04.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.567+0000 7f9df5600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f9df00cd580 con 0x7f9df00cd0d0 2024-04-20T11:27:04.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.567+0000 7f9deea00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9df00726f0 0x7f9df00c76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:04.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.567+0000 7f9deea00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9df00726f0 0x7f9df00c76d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:42684/0 (socket says 172.21.15.152:42684) 2024-04-20T11:27:04.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.567+0000 7f9deea00640 1 -- 172.21.15.152:0/3825610539 learned_addr learned my addr 172.21.15.152:0/3825610539 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:04.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.568+0000 7f9deea00640 1 -- 172.21.15.152:0/3825610539 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9df00cd0d0 msgr2=0x7f9df00b3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:04.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.568+0000 7f9def400640 1 --2- 172.21.15.152:0/3825610539 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9df0145f60 0x7f9df00b4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:04.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.568+0000 7f9dee000640 1 --2- 172.21.15.152:0/3825610539 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9df00cd0d0 0x7f9df00b3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:04.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.568+0000 7f9deea00640 1 --2- 172.21.15.152:0/3825610539 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9df00cd0d0 0x7f9df00b3570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:04.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.568+0000 7f9deea00640 1 -- 172.21.15.152:0/3825610539 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9df0145f60 msgr2=0x7f9df00b4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:04.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.568+0000 7f9deea00640 1 --2- 172.21.15.152:0/3825610539 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9df0145f60 0x7f9df00b4fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:04.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.568+0000 7f9deea00640 1 -- 172.21.15.152:0/3825610539 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9df00b3ce0 con 0x7f9df00726f0 2024-04-20T11:27:04.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.568+0000 7f9def400640 1 --2- 172.21.15.152:0/3825610539 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9df0145f60 0x7f9df00b4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:27:04.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.568+0000 7f9deea00640 1 --2- 172.21.15.152:0/3825610539 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9df00726f0 0x7f9df00c76d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9de0000c00 tx=0x7f9de00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:04.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.568+0000 7f9de6a00640 1 -- 172.21.15.152:0/3825610539 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9de0040070 con 0x7f9df00726f0 2024-04-20T11:27:04.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.568+0000 7f9de6a00640 1 -- 172.21.15.152:0/3825610539 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9de0039a00 con 0x7f9df00726f0 2024-04-20T11:27:04.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.568+0000 7f9de6a00640 1 -- 172.21.15.152:0/3825610539 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9de0035040 con 0x7f9df00726f0 2024-04-20T11:27:04.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.568+0000 7f9df5600640 1 -- 172.21.15.152:0/3825610539 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9df00b3fc0 con 0x7f9df00726f0 2024-04-20T11:27:04.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.568+0000 7f9df5600640 1 -- 172.21.15.152:0/3825610539 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9df00c5580 con 0x7f9df00726f0 2024-04-20T11:27:04.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.569+0000 7f9df5600640 1 -- 172.21.15.152:0/3825610539 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9df00c71a0 con 0x7f9df00726f0 2024-04-20T11:27:04.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.570+0000 7f9de6a00640 1 -- 172.21.15.152:0/3825610539 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9de004b060 con 0x7f9df00726f0 2024-04-20T11:27:04.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.571+0000 7f9de6a00640 1 --2- 172.21.15.152:0/3825610539 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f9dd006c920 0x7f9dd006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:04.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.571+0000 7f9de6a00640 1 -- 172.21.15.152:0/3825610539 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f9de00ad530 con 0x7f9df00726f0 2024-04-20T11:27:04.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.572+0000 7f9dee000640 1 --2- 172.21.15.152:0/3825610539 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f9dd006c920 0x7f9dd006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:04.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.572+0000 7f9dee000640 1 --2- 172.21.15.152:0/3825610539 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f9dd006c920 0x7f9dd006ede0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9df00c6c20 tx=0x7f9ddc009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:04.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.576+0000 7f9de6a00640 1 -- 172.21.15.152:0/3825610539 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f9df00c71a0 con 0x7f9df00726f0 2024-04-20T11:27:04.776 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.776+0000 7f9df5600640 1 -- 172.21.15.152:0/3825610539 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9df00689f0 con 0x7f9df00726f0 2024-04-20T11:27:04.778 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.778+0000 7f9de6a00640 1 -- 172.21.15.152:0/3825610539 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16665 (secure 0 0 0) 0x7f9de007b960 con 0x7f9df00726f0 2024-04-20T11:27:04.778 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:27:04.778 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":20,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:26:59.675971+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:26:56.261424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T11:26:57.079286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:27:04.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.781+0000 7f9df5600640 1 -- 172.21.15.152:0/3825610539 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f9dd006c920 msgr2=0x7f9dd006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:04.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.781+0000 7f9df5600640 1 --2- 172.21.15.152:0/3825610539 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f9dd006c920 0x7f9dd006ede0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9df00c6c20 tx=0x7f9ddc009040 comp rx=0 tx=0).stop 2024-04-20T11:27:04.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.781+0000 7f9df5600640 1 -- 172.21.15.152:0/3825610539 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9df00726f0 msgr2=0x7f9df00c76d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:04.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.781+0000 7f9df5600640 1 --2- 172.21.15.152:0/3825610539 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9df00726f0 0x7f9df00c76d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9de0000c00 tx=0x7f9de00046c0 comp rx=0 tx=0).stop 2024-04-20T11:27:04.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.781+0000 7f9df5600640 1 -- 172.21.15.152:0/3825610539 shutdown_connections 2024-04-20T11:27:04.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.781+0000 7f9df5600640 1 --2- 172.21.15.152:0/3825610539 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f9dd006c920 0x7f9dd006ede0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:04.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.781+0000 7f9df5600640 1 --2- 172.21.15.152:0/3825610539 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9df0145f60 0x7f9df00b4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:04.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.781+0000 7f9df5600640 1 --2- 172.21.15.152:0/3825610539 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9df00cd0d0 0x7f9df00b3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:04.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.781+0000 7f9df5600640 1 --2- 172.21.15.152:0/3825610539 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9df00726f0 0x7f9df00c76d0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:04.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.781+0000 7f9df5600640 1 -- 172.21.15.152:0/3825610539 >> 172.21.15.152:0/3825610539 conn(0x7f9df0067cf0 msgr2=0x7f9df015a970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:04.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.781+0000 7f9df5600640 1 -- 172.21.15.152:0/3825610539 shutdown_connections 2024-04-20T11:27:04.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:04.781+0000 7f9df5600640 1 -- 172.21.15.152:0/3825610539 wait complete. 2024-04-20T11:27:04.798 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-20T11:27:04.798 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:27:04.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.000+0000 7fba44000640 1 Processor -- start 2024-04-20T11:27:05.000 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.000+0000 7fba44000640 1 -- start start 2024-04-20T11:27:05.002 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.002+0000 7fba44000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba3c0cc9b0 0x7fba3c0b43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:05.002 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.002+0000 7fba44000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fba3c0ccd90 0x7fba3c0b4930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:05.002 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.002+0000 7fba44000640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba3c15b890 con 0x7fba3c15b320 2024-04-20T11:27:05.002 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.002+0000 7fba44000640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fba3c15bb70 con 0x7fba3c0ccd90 2024-04-20T11:27:05.002 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.002+0000 7fba44000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fba3c15be50 con 0x7fba3c0cc9b0 2024-04-20T11:27:05.002 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.002+0000 7fba42000640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fba3c15b320 0x7fba3c15e910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:40608/0 (socket says 172.21.15.152:40608) 2024-04-20T11:27:05.002 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.002+0000 7fba42000640 1 -- 172.21.15.152:0/4263395446 learned_addr learned my addr 172.21.15.152:0/4263395446 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:27:05.002 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.002+0000 7fba40c00640 1 --2- 172.21.15.152:0/4263395446 >> v2:172.21.15.152:3300/0 conn(0x7fba3c0ccd90 0x7fba3c0b4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:05.002 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.002+0000 7fba41600640 1 --2- 172.21.15.152:0/4263395446 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba3c0cc9b0 0x7fba3c0b43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:05.002 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.002+0000 7fba40c00640 1 -- 172.21.15.152:0/4263395446 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba3c0cc9b0 msgr2=0x7fba3c0b43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:05.003 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.002+0000 7fba3be00640 1 -- 172.21.15.152:0/4263395446 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1893272077 0 0) 0x7fba3c15b890 con 0x7fba3c15b320 2024-04-20T11:27:05.003 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.002+0000 7fba40c00640 1 --2- 172.21.15.152:0/4263395446 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba3c0cc9b0 0x7fba3c0b43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:05.003 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.002+0000 7fba40c00640 1 -- 172.21.15.152:0/4263395446 >> v1:172.21.15.152:6789/0 conn(0x7fba3c15b320 legacy=0x7fba3c15e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:05.003 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.002+0000 7fba40c00640 1 -- 172.21.15.152:0/4263395446 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba3c15c130 con 0x7fba3c0ccd90 2024-04-20T11:27:05.004 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.004+0000 7fba40c00640 1 --2- 172.21.15.152:0/4263395446 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fba3c0ccd90 0x7fba3c0b4930 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fba2c0097a0 tx=0x7fba2c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa86582229f2f496 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:05.004 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.004+0000 7fba3be00640 1 -- 172.21.15.152:0/4263395446 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba2c035020 con 0x7fba3c0ccd90 2024-04-20T11:27:05.004 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.004+0000 7fba3be00640 1 -- 172.21.15.152:0/4263395446 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba2c031830 con 0x7fba3c0ccd90 2024-04-20T11:27:05.004 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.004+0000 7fba3be00640 1 -- 172.21.15.152:0/4263395446 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba2c031c00 con 0x7fba3c0ccd90 2024-04-20T11:27:05.004 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.004+0000 7fba44000640 1 -- 172.21.15.152:0/4263395446 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fba3c0ccd90 msgr2=0x7fba3c0b4930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:05.004 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.004+0000 7fba44000640 1 --2- 172.21.15.152:0/4263395446 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fba3c0ccd90 0x7fba3c0b4930 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fba2c0097a0 tx=0x7fba2c02b8d0 comp rx=0 tx=0).stop 2024-04-20T11:27:05.005 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.004+0000 7fba44000640 1 -- 172.21.15.152:0/4263395446 shutdown_connections 2024-04-20T11:27:05.005 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.004+0000 7fba44000640 1 --2- 172.21.15.152:0/4263395446 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fba3c0ccd90 0x7fba3c0b4930 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:05.005 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.004+0000 7fba44000640 1 --2- 172.21.15.152:0/4263395446 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba3c0cc9b0 0x7fba3c0b43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:05.005 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.004+0000 7fba44000640 1 -- 172.21.15.152:0/4263395446 >> 172.21.15.152:0/4263395446 conn(0x7fba3c067cf0 msgr2=0x7fba3c06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:05.005 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.005+0000 7fba44000640 1 -- 172.21.15.152:0/4263395446 shutdown_connections 2024-04-20T11:27:05.005 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.005+0000 7fba44000640 1 -- 172.21.15.152:0/4263395446 wait complete. 2024-04-20T11:27:05.005 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.006+0000 7fba44000640 1 Processor -- start 2024-04-20T11:27:05.006 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.007+0000 7fba44000640 1 -- start start 2024-04-20T11:27:05.008 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.007+0000 7fba44000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba3c0cc9b0 0x7fba3c159df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:05.009 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.008+0000 7fba44000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba3c15b320 0x7fba3c15a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:05.009 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.008+0000 7fba44000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fba3c0cd910 0x7fba3c0cddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:05.009 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.008+0000 7fba44000640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fba3c06f3f0 con 0x7fba3c0cd910 2024-04-20T11:27:05.009 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.008+0000 7fba44000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fba3c06f270 con 0x7fba3c0cc9b0 2024-04-20T11:27:05.009 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.008+0000 7fba44000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fba3c06f570 con 0x7fba3c15b320 2024-04-20T11:27:05.009 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.008+0000 7fba42000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fba3c0cd910 0x7fba3c0cddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:05.009 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.008+0000 7fba42000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fba3c0cd910 0x7fba3c0cddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:42694/0 (socket says 172.21.15.152:42694) 2024-04-20T11:27:05.009 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.008+0000 7fba42000640 1 -- 172.21.15.152:0/1652562806 learned_addr learned my addr 172.21.15.152:0/1652562806 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:05.009 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.008+0000 7fba41600640 1 --2- 172.21.15.152:0/1652562806 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba3c0cc9b0 0x7fba3c159df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:05.009 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.008+0000 7fba40c00640 1 --2- 172.21.15.152:0/1652562806 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba3c15b320 0x7fba3c15a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:05.009 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.008+0000 7fba42000640 1 -- 172.21.15.152:0/1652562806 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba3c15b320 msgr2=0x7fba3c15a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:05.009 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.008+0000 7fba42000640 1 --2- 172.21.15.152:0/1652562806 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba3c15b320 0x7fba3c15a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:05.009 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.008+0000 7fba42000640 1 -- 172.21.15.152:0/1652562806 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba3c0cc9b0 msgr2=0x7fba3c159df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:05.010 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.008+0000 7fba42000640 1 --2- 172.21.15.152:0/1652562806 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba3c0cc9b0 0x7fba3c159df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:05.010 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.008+0000 7fba42000640 1 -- 172.21.15.152:0/1652562806 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba3c14dda0 con 0x7fba3c0cd910 2024-04-20T11:27:05.010 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.008+0000 7fba40c00640 1 --2- 172.21.15.152:0/1652562806 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba3c15b320 0x7fba3c15a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:27:05.010 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.009+0000 7fba42000640 1 --2- 172.21.15.152:0/1652562806 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fba3c0cd910 0x7fba3c0cddc0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fba3400c7a0 tx=0x7fba3400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:05.010 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.009+0000 7fba39600640 1 -- 172.21.15.152:0/1652562806 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba34018070 con 0x7fba3c0cd910 2024-04-20T11:27:05.010 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.009+0000 7fba39600640 1 -- 172.21.15.152:0/1652562806 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba3400a5e0 con 0x7fba3c0cd910 2024-04-20T11:27:05.010 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.009+0000 7fba44000640 1 -- 172.21.15.152:0/1652562806 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba3c13c090 con 0x7fba3c0cd910 2024-04-20T11:27:05.010 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.009+0000 7fba39600640 1 -- 172.21.15.152:0/1652562806 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba3400aa10 con 0x7fba3c0cd910 2024-04-20T11:27:05.010 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.009+0000 7fba44000640 1 -- 172.21.15.152:0/1652562806 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba3c14e040 con 0x7fba3c0cd910 2024-04-20T11:27:05.014 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.010+0000 7fba44000640 1 -- 172.21.15.152:0/1652562806 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba3c0689f0 con 0x7fba3c0cd910 2024-04-20T11:27:05.015 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.011+0000 7fba39600640 1 -- 172.21.15.152:0/1652562806 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fba3401b070 con 0x7fba3c0cd910 2024-04-20T11:27:05.015 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.013+0000 7fba39600640 1 --2- 172.21.15.152:0/1652562806 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fba2006c920 0x7fba2006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:05.015 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.013+0000 7fba39600640 1 -- 172.21.15.152:0/1652562806 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fba3408b1a0 con 0x7fba3c0cd910 2024-04-20T11:27:05.015 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.014+0000 7fba41600640 1 --2- 172.21.15.152:0/1652562806 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fba2006c920 0x7fba2006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:05.015 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.014+0000 7fba41600640 1 --2- 172.21.15.152:0/1652562806 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fba2006c920 0x7fba2006ede0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fba30006fd0 tx=0x7fba30008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:05.017 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.018+0000 7fba39600640 1 -- 172.21.15.152:0/1652562806 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fba340594a0 con 0x7fba3c0cd910 2024-04-20T11:27:05.220 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.221+0000 7fba44000640 1 -- 172.21.15.152:0/1652562806 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fba3c0b0d00 con 0x7fba3c0cd910 2024-04-20T11:27:05.222 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.222+0000 7fba39600640 1 -- 172.21.15.152:0/1652562806 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16665 (secure 0 0 0) 0x7fba34058da0 con 0x7fba3c0cd910 2024-04-20T11:27:05.222 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:27:05.222 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":20,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:26:59.675971+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:26:56.261424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T11:26:57.079286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:27:05.225 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.225+0000 7fba44000640 1 -- 172.21.15.152:0/1652562806 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fba2006c920 msgr2=0x7fba2006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:05.225 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.225+0000 7fba44000640 1 --2- 172.21.15.152:0/1652562806 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fba2006c920 0x7fba2006ede0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fba30006fd0 tx=0x7fba30008040 comp rx=0 tx=0).stop 2024-04-20T11:27:05.225 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.225+0000 7fba44000640 1 -- 172.21.15.152:0/1652562806 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fba3c0cd910 msgr2=0x7fba3c0cddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:05.225 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.225+0000 7fba44000640 1 --2- 172.21.15.152:0/1652562806 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fba3c0cd910 0x7fba3c0cddc0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fba3400c7a0 tx=0x7fba3400cc60 comp rx=0 tx=0).stop 2024-04-20T11:27:05.225 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.225+0000 7fba44000640 1 -- 172.21.15.152:0/1652562806 shutdown_connections 2024-04-20T11:27:05.225 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.225+0000 7fba44000640 1 --2- 172.21.15.152:0/1652562806 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fba2006c920 0x7fba2006ede0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:05.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.225+0000 7fba44000640 1 --2- 172.21.15.152:0/1652562806 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fba3c0cd910 0x7fba3c0cddc0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:05.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.225+0000 7fba44000640 1 --2- 172.21.15.152:0/1652562806 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba3c15b320 0x7fba3c15a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:05.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.225+0000 7fba44000640 1 --2- 172.21.15.152:0/1652562806 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba3c0cc9b0 0x7fba3c159df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:05.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.225+0000 7fba44000640 1 -- 172.21.15.152:0/1652562806 >> 172.21.15.152:0/1652562806 conn(0x7fba3c067cf0 msgr2=0x7fba3c0b00e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:05.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.226+0000 7fba44000640 1 -- 172.21.15.152:0/1652562806 shutdown_connections 2024-04-20T11:27:05.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.226+0000 7fba44000640 1 -- 172.21.15.152:0/1652562806 wait complete. 2024-04-20T11:27:05.244 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-20T11:27:05.245 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-20T11:27:05.245 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-20T11:27:05.246 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-20T11:27:05.246 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-20T11:27:05.246 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-20T11:27:05.247 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-20T11:27:05.247 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-20T11:27:05.530 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.530+0000 7f94b6e00640 1 Processor -- start 2024-04-20T11:27:05.530 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.530+0000 7f94b6e00640 1 -- start start 2024-04-20T11:27:05.531 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.532+0000 7f94b6e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94a80552f0 0x7f94a80068a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:05.531 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.532+0000 7f94b6e00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94a80387d0 con 0x7f94a80eb050 2024-04-20T11:27:05.531 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.532+0000 7f94b6e00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94a8056d80 con 0x7f94a80ebb40 2024-04-20T11:27:05.531 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.532+0000 7f94b6e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f94a8057e20 con 0x7f94a80552f0 2024-04-20T11:27:05.532 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.532+0000 7f94afe00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f94a80eb050 0x7f94a80eb430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:40620/0 (socket says 172.21.15.152:40620) 2024-04-20T11:27:05.532 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.532+0000 7f94afe00640 1 -- 172.21.15.152:0/372261558 learned_addr learned my addr 172.21.15.152:0/372261558 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:27:05.532 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.533+0000 7f94b4e00640 1 --2- 172.21.15.152:0/372261558 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94a80552f0 0x7f94a80068a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:05.533 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.533+0000 7f94aea00640 1 -- 172.21.15.152:0/372261558 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2688173369 0 0) 0x7f94a4001110 con 0x7f94a80eb050 2024-04-20T11:27:05.533 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.534+0000 7f94b4e00640 1 -- 172.21.15.152:0/372261558 >> v1:172.21.15.175:6789/0 conn(0x7f94a80ebb40 legacy=0x7f94a8006190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:05.533 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.534+0000 7f94b4e00640 1 -- 172.21.15.152:0/372261558 >> v1:172.21.15.152:6789/0 conn(0x7f94a80eb050 legacy=0x7f94a80eb430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:05.534 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.534+0000 7f94b4e00640 1 -- 172.21.15.152:0/372261558 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94a8057060 con 0x7f94a80552f0 2024-04-20T11:27:05.534 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.534+0000 7fe774000640 1 Processor -- start 2024-04-20T11:27:05.534 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.535+0000 7fe774000640 1 -- start start 2024-04-20T11:27:05.535 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.535+0000 7f94b4e00640 1 --2- 172.21.15.152:0/372261558 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94a80552f0 0x7f94a80068a0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f94b006a100 tx=0x7f94b00a7360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=338a5af9328cb0ca server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:05.535 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.536+0000 7f94aea00640 1 -- 172.21.15.152:0/372261558 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94b00b4070 con 0x7f94a80552f0 2024-04-20T11:27:05.535 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.536+0000 7f94aea00640 1 -- 172.21.15.152:0/372261558 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94b00af680 con 0x7f94a80552f0 2024-04-20T11:27:05.535 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.536+0000 7f94aea00640 1 -- 172.21.15.152:0/372261558 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94b00afa70 con 0x7f94a80552f0 2024-04-20T11:27:05.536 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.537+0000 7f94b6e00640 1 -- 172.21.15.152:0/372261558 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94a80552f0 msgr2=0x7f94a80068a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:05.536 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.537+0000 7f94b6e00640 1 --2- 172.21.15.152:0/372261558 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94a80552f0 0x7f94a80068a0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f94b006a100 tx=0x7f94b00a7360 comp rx=0 tx=0).stop 2024-04-20T11:27:05.537 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.538+0000 7fe774000640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fe76c0d0720 0x7fe76c0afbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:05.537 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.538+0000 7f94b6e00640 1 -- 172.21.15.152:0/372261558 shutdown_connections 2024-04-20T11:27:05.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.538+0000 7f94b6e00640 1 --2- 172.21.15.152:0/372261558 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94a80552f0 0x7f94a80068a0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:05.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.538+0000 7f94b6e00640 1 -- 172.21.15.152:0/372261558 >> 172.21.15.152:0/372261558 conn(0x7f94a80559d0 msgr2=0x7f94a8055de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:05.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.538+0000 7f94b6e00640 1 -- 172.21.15.152:0/372261558 shutdown_connections 2024-04-20T11:27:05.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.538+0000 7f94b6e00640 1 -- 172.21.15.152:0/372261558 wait complete. 2024-04-20T11:27:05.539 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.538+0000 7fe774000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe76c0d0b00 0x7fe76c0b1390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:05.539 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.538+0000 7fe774000640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe76c0b02f0 con 0x7fe76c0d0f60 2024-04-20T11:27:05.539 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.538+0000 7fe774000640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fe76c0b05d0 con 0x7fe76c0d0720 2024-04-20T11:27:05.539 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.538+0000 7fe774000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe76c0b08b0 con 0x7fe76c0d0b00 2024-04-20T11:27:05.540 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.540+0000 7fe772000640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fe76c0d0f60 0x7fe76c145310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:40622/0 (socket says 172.21.15.152:40622) 2024-04-20T11:27:05.540 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.540+0000 7fe770c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe76c0d0b00 0x7fe76c0b1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:05.540 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.540+0000 7fe770c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe76c0d0b00 0x7fe76c0b1390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.152:51808/0 (socket says 172.21.15.152:51808) 2024-04-20T11:27:05.540 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.540+0000 7fe770c00640 1 -- 172.21.15.152:0/2037698837 learned_addr learned my addr 172.21.15.152:0/2037698837 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:05.541 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.540+0000 7f94b6e00640 1 Processor -- start 2024-04-20T11:27:05.541 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.540+0000 7fe771600640 1 --2- 172.21.15.152:0/2037698837 >> v2:172.21.15.175:3300/0 conn(0x7fe76c0d0720 0x7fe76c0afbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:05.541 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.541+0000 7f638be00640 1 Processor -- start 2024-04-20T11:27:05.542 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.541+0000 7f94b6e00640 1 -- start start 2024-04-20T11:27:05.542 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.541+0000 7fe770c00640 1 -- 172.21.15.152:0/2037698837 >> v2:172.21.15.175:3300/0 conn(0x7fe76c0d0720 msgr2=0x7fe76c0afbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:05.542 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.541+0000 7fe770c00640 1 --2- 172.21.15.152:0/2037698837 >> v2:172.21.15.175:3300/0 conn(0x7fe76c0d0720 0x7fe76c0afbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:05.542 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.541+0000 7fe770c00640 1 -- 172.21.15.152:0/2037698837 >> v1:172.21.15.152:6789/0 conn(0x7fe76c0d0f60 legacy=0x7fe76c145310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:05.542 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.541+0000 7fe770c00640 1 -- 172.21.15.152:0/2037698837 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe76c0b0b90 con 0x7fe76c0d0b00 2024-04-20T11:27:05.543 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.541+0000 7f638be00640 1 -- start start 2024-04-20T11:27:05.543 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.542+0000 7fb216a00640 1 Processor -- start 2024-04-20T11:27:05.543 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.542+0000 7f638be00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f63840e1220 0x7f638400deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:05.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.542+0000 7f638be00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6384057910 0x7f6384005e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:05.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.542+0000 7f638be00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f63840063d0 0x7f638403b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:05.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.542+0000 7f94b6e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94a80552f0 0x7f94a804f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:05.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.542+0000 7f94b6e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94a80eb050 0x7f94a8186480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:05.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.542+0000 7f94b6e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f94a80ebb40 0x7f94a8052160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:05.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.542+0000 7f638be00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f638400c2d0 con 0x7f63840e1220 2024-04-20T11:27:05.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.542+0000 7f638be00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f63840395e0 con 0x7f63840063d0 2024-04-20T11:27:05.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.542+0000 7f638be00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f638400c570 con 0x7f6384057910 2024-04-20T11:27:05.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.542+0000 7fb216a00640 1 -- start start 2024-04-20T11:27:05.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.543+0000 7f94b6e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f94a8055780 con 0x7f94a80ebb40 2024-04-20T11:27:05.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.543+0000 7f94b6e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f94a8057990 con 0x7f94a80eb050 2024-04-20T11:27:05.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.543+0000 7f94b6e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f94a8057620 con 0x7f94a80552f0 2024-04-20T11:27:05.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.543+0000 7f94b4e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f94a80ebb40 0x7f94a8052160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:05.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.543+0000 7f94b4e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f94a80ebb40 0x7f94a8052160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:42710/0 (socket says 172.21.15.152:42710) 2024-04-20T11:27:05.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.543+0000 7f94b4e00640 1 -- 172.21.15.152:0/4117503173 learned_addr learned my addr 172.21.15.152:0/4117503173 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:05.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.543+0000 7fe770c00640 1 --2- 172.21.15.152:0/2037698837 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe76c0d0b00 0x7fe76c0b1390 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe760009870 tx=0x7fe76002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5cca5e3660e77ff server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:05.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.543+0000 7f638aa00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f63840e1220 0x7f638400deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:05.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.543+0000 7f638aa00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f63840e1220 0x7f638400deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:42716/0 (socket says 172.21.15.152:42716) 2024-04-20T11:27:05.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.543+0000 7f638aa00640 1 -- 172.21.15.152:0/1261492926 learned_addr learned my addr 172.21.15.152:0/1261492926 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:05.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.543+0000 7f638b400640 1 --2- 172.21.15.152:0/1261492926 >> v2:172.21.15.175:3300/0 conn(0x7f63840063d0 0x7f638403b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:05.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.544+0000 7f638a000640 1 --2- 172.21.15.152:0/1261492926 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6384057910 0x7f6384005e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:05.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.544+0000 7f94b4e00640 1 -- 172.21.15.152:0/4117503173 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94a80552f0 msgr2=0x7f94a804f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:05.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.544+0000 7f94b4e00640 1 --2- 172.21.15.152:0/4117503173 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94a80552f0 0x7f94a804f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:05.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.544+0000 7f94b4e00640 1 -- 172.21.15.152:0/4117503173 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94a80eb050 msgr2=0x7f94a8186480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:05.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.544+0000 7f94b4e00640 1 --2- 172.21.15.152:0/4117503173 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94a80eb050 0x7f94a8186480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:05.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.544+0000 7f94b4e00640 1 -- 172.21.15.152:0/4117503173 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94a803e530 con 0x7f94a80ebb40 2024-04-20T11:27:05.548 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.544+0000 7fe76be00640 1 -- 172.21.15.152:0/2037698837 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe760034070 con 0x7fe76c0d0b00 2024-04-20T11:27:05.548 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.544+0000 7fe76be00640 1 -- 172.21.15.152:0/2037698837 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe760030840 con 0x7fe76c0d0b00 2024-04-20T11:27:05.548 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.544+0000 7fe76be00640 1 -- 172.21.15.152:0/2037698837 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe760030c10 con 0x7fe76c0d0b00 2024-04-20T11:27:05.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.544+0000 7fb216a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb20803e840 0x7fb208053bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:05.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.544+0000 7fb216a00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fb208054130 0x7fb2080068d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:05.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.544+0000 7fb216a00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb208058570 con 0x7fb2080e30c0 2024-04-20T11:27:05.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.544+0000 7fb216a00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fb208058850 con 0x7fb208054130 2024-04-20T11:27:05.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.544+0000 7fb216a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb208058b30 con 0x7fb20803e840 2024-04-20T11:27:05.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.544+0000 7f94b4e00640 1 --2- 172.21.15.152:0/4117503173 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f94a80ebb40 0x7f94a8052160 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f94b0076cd0 tx=0x7f94b0063760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:05.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.544+0000 7f638aa00640 1 -- 172.21.15.152:0/1261492926 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6384057910 msgr2=0x7f6384005e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:05.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.544+0000 7f638aa00640 1 --2- 172.21.15.152:0/1261492926 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6384057910 0x7f6384005e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:05.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.545+0000 7f94a3e00640 1 -- 172.21.15.152:0/4117503173 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94b00bc070 con 0x7f94a80ebb40 2024-04-20T11:27:05.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.544+0000 7f638aa00640 1 -- 172.21.15.152:0/1261492926 >> v2:172.21.15.175:3300/0 conn(0x7f63840063d0 msgr2=0x7f638403b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:05.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.544+0000 7f638aa00640 1 --2- 172.21.15.152:0/1261492926 >> v2:172.21.15.175:3300/0 conn(0x7f63840063d0 0x7f638403b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:05.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.544+0000 7f638aa00640 1 -- 172.21.15.152:0/1261492926 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f638400c850 con 0x7f63840e1220 2024-04-20T11:27:05.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.545+0000 7f94a3e00640 1 -- 172.21.15.152:0/4117503173 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94b00678c0 con 0x7f94a80ebb40 2024-04-20T11:27:05.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.545+0000 7f94a3e00640 1 -- 172.21.15.152:0/4117503173 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94b00b4040 con 0x7f94a80ebb40 2024-04-20T11:27:05.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.545+0000 7fb214c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb20803e840 0x7fb208053bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:05.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.545+0000 7f94b6e00640 1 -- 172.21.15.152:0/4117503173 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94a803e120 con 0x7f94a80ebb40 2024-04-20T11:27:05.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.545+0000 7fe774000640 1 -- 172.21.15.152:0/2037698837 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe76c0d0b00 msgr2=0x7fe76c0b1390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:05.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.545+0000 7fe774000640 1 --2- 172.21.15.152:0/2037698837 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe76c0d0b00 0x7fe76c0b1390 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe760009870 tx=0x7fe76002b980 comp rx=0 tx=0).stop 2024-04-20T11:27:05.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.545+0000 7fb215600640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7fb2080e30c0 0x7fb208057d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.152:49240/0 (socket says 172.21.15.152:49240) 2024-04-20T11:27:05.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.545+0000 7fb215600640 1 -- 172.21.15.152:0/4166805706 learned_addr learned my addr 172.21.15.152:0/4166805706 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:27:05.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.545+0000 7fb216000640 1 --2- 172.21.15.152:0/4166805706 >> v2:172.21.15.175:3300/0 conn(0x7fb208054130 0x7fb2080068d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:05.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.545+0000 7f638aa00640 1 --2- 172.21.15.152:0/1261492926 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f63840e1220 0x7f638400deb0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f638c050f60 tx=0x7f638c0ab860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b84b9dc1f7a4f97 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:05.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.546+0000 7fe774000640 1 -- 172.21.15.152:0/2037698837 shutdown_connections 2024-04-20T11:27:05.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.546+0000 7fe774000640 1 --2- 172.21.15.152:0/2037698837 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe76c0d0b00 0x7fe76c0b1390 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:05.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.546+0000 7fe774000640 1 --2- 172.21.15.152:0/2037698837 >> v2:172.21.15.175:3300/0 conn(0x7fe76c0d0720 0x7fe76c0afbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:05.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.546+0000 7fe774000640 1 -- 172.21.15.152:0/2037698837 >> 172.21.15.152:0/2037698837 conn(0x7fe76c0b55e0 msgr2=0x7fe76c0b59f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:05.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.546+0000 7fb214c00640 1 -- 172.21.15.152:0/4166805706 >> v2:172.21.15.175:3300/0 conn(0x7fb208054130 msgr2=0x7fb2080068d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:05.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.546+0000 7fb214c00640 1 --2- 172.21.15.152:0/4166805706 >> v2:172.21.15.175:3300/0 conn(0x7fb208054130 0x7fb2080068d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:05.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.546+0000 7fb214c00640 1 -- 172.21.15.152:0/4166805706 >> v1:172.21.15.175:6789/0 conn(0x7fb2080e30c0 legacy=0x7fb208057d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:05.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.546+0000 7fb214c00640 1 -- 172.21.15.152:0/4166805706 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2080063b0 con 0x7fb20803e840 2024-04-20T11:27:05.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.547+0000 7fb216000640 1 --2- 172.21.15.152:0/4166805706 >> v2:172.21.15.175:3300/0 conn(0x7fb208054130 0x7fb2080068d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:27:05.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.547+0000 7f6389600640 1 -- 172.21.15.152:0/1261492926 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f638c0b8070 con 0x7f63840e1220 2024-04-20T11:27:05.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.547+0000 7f6389600640 1 -- 172.21.15.152:0/1261492926 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f638c068560 con 0x7f63840e1220 2024-04-20T11:27:05.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.547+0000 7f6389600640 1 -- 172.21.15.152:0/1261492926 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f638c0678c0 con 0x7f63840e1220 2024-04-20T11:27:05.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.547+0000 7f94b6e00640 1 -- 172.21.15.152:0/4117503173 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94a804cd70 con 0x7f94a80ebb40 2024-04-20T11:27:05.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.547+0000 7fb214c00640 1 --2- 172.21.15.152:0/4166805706 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb20803e840 0x7fb208053bf0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fb210000b60 tx=0x7fb2100a73e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c706945de5ef7b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:05.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.548+0000 7fb20fe00640 1 -- 172.21.15.152:0/4166805706 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2100b3070 con 0x7fb20803e840 2024-04-20T11:27:05.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.548+0000 7fe774000640 1 -- 172.21.15.152:0/2037698837 shutdown_connections 2024-04-20T11:27:05.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.548+0000 7f638be00640 1 -- 172.21.15.152:0/1261492926 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f63840e1220 msgr2=0x7f638400deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:05.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.548+0000 7f638be00640 1 --2- 172.21.15.152:0/1261492926 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f63840e1220 0x7f638400deb0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f638c050f60 tx=0x7f638c0ab860 comp rx=0 tx=0).stop 2024-04-20T11:27:05.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.548+0000 7fb20fe00640 1 -- 172.21.15.152:0/4166805706 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb210068310 con 0x7fb20803e840 2024-04-20T11:27:05.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.548+0000 7fb20fe00640 1 -- 172.21.15.152:0/4166805706 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2100b3070 con 0x7fb20803e840 2024-04-20T11:27:05.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.548+0000 7f638be00640 1 -- 172.21.15.152:0/1261492926 shutdown_connections 2024-04-20T11:27:05.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.548+0000 7f638be00640 1 --2- 172.21.15.152:0/1261492926 >> v2:172.21.15.175:3300/0 conn(0x7f63840063d0 0x7f638403b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:05.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.548+0000 7f638be00640 1 --2- 172.21.15.152:0/1261492926 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6384057910 0x7f6384005e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:05.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.548+0000 7f638be00640 1 --2- 172.21.15.152:0/1261492926 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f63840e1220 0x7f638400deb0 secure :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f638c050f60 tx=0x7f638c0ab860 comp rx=0 tx=0).stop 2024-04-20T11:27:05.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.548+0000 7f638be00640 1 -- 172.21.15.152:0/1261492926 >> 172.21.15.152:0/1261492926 conn(0x7f6384020070 msgr2=0x7f6384056ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:05.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.549+0000 7f638be00640 1 -- 172.21.15.152:0/1261492926 shutdown_connections 2024-04-20T11:27:05.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.549+0000 7fe6a7800640 1 Processor -- start 2024-04-20T11:27:05.558 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.549+0000 7f638be00640 1 -- 172.21.15.152:0/1261492926 wait complete. 2024-04-20T11:27:05.558 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.549+0000 7f94b6e00640 1 -- 172.21.15.152:0/4117503173 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f94a80eb960 con 0x7f94a80ebb40 2024-04-20T11:27:05.558 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.549+0000 7f94a3e00640 1 -- 172.21.15.152:0/4117503173 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f94b00bf070 con 0x7f94a80ebb40 2024-04-20T11:27:05.559 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.549+0000 7fb216a00640 1 -- 172.21.15.152:0/4166805706 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb20803e840 msgr2=0x7fb208053bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:05.559 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.550+0000 7fb216a00640 1 --2- 172.21.15.152:0/4166805706 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb20803e840 0x7fb208053bf0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fb210000b60 tx=0x7fb2100a73e0 comp rx=0 tx=0).stop 2024-04-20T11:27:05.559 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.550+0000 7fb216a00640 1 -- 172.21.15.152:0/4166805706 shutdown_connections 2024-04-20T11:27:05.559 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.548+0000 7fe774000640 1 -- 172.21.15.152:0/2037698837 wait complete. 2024-04-20T11:27:05.559 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.550+0000 7fe774000640 1 Processor -- start 2024-04-20T11:27:05.559 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.550+0000 7fb216a00640 1 --2- 172.21.15.152:0/4166805706 >> v2:172.21.15.175:3300/0 conn(0x7fb208054130 0x7fb2080068d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:05.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.550+0000 7fb216a00640 1 --2- 172.21.15.152:0/4166805706 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb20803e840 0x7fb208053bf0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:05.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.550+0000 7fb216a00640 1 -- 172.21.15.152:0/4166805706 >> 172.21.15.152:0/4166805706 conn(0x7fb2080559e0 msgr2=0x7fb208055df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:05.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.550+0000 7f94a3e00640 1 --2- 172.21.15.152:0/4117503173 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f949006c940 0x7f949006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:05.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.551+0000 7fe774000640 1 -- start start 2024-04-20T11:27:05.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.551+0000 7fb216a00640 1 -- 172.21.15.152:0/4166805706 shutdown_connections 2024-04-20T11:27:05.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.549+0000 7fe6a7800640 1 -- start start 2024-04-20T11:27:05.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.551+0000 7f94a3e00640 1 -- 172.21.15.152:0/4117503173 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f94b00f3df0 con 0x7f94a80ebb40 2024-04-20T11:27:05.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.551+0000 7fe6a7800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fe69803c090 0x7fe6980e7770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:05.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.551+0000 7fe6a7800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe698053ca0 0x7fe698054130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:05.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.551+0000 7fe6a7800640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe698054670 con 0x7fe6980e3150 2024-04-20T11:27:05.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.551+0000 7fe6a7800640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fe698007b60 con 0x7fe69803c090 2024-04-20T11:27:05.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.551+0000 7fe6a7800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe698058800 con 0x7fe698053ca0 2024-04-20T11:27:05.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.551+0000 7fb216a00640 1 -- 172.21.15.152:0/4166805706 wait complete. 2024-04-20T11:27:05.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.552+0000 7fe6a4e00640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7fe6980e3150 0x7fe698058040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.152:49248/0 (socket says 172.21.15.152:49248) 2024-04-20T11:27:05.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.552+0000 7fe6a4e00640 1 -- 172.21.15.152:0/3302975206 learned_addr learned my addr 172.21.15.152:0/3302975206 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:27:05.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.552+0000 7fe69fe00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fe69803c090 0x7fe6980e7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:05.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.552+0000 7f638be00640 1 Processor -- start 2024-04-20T11:27:05.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.552+0000 7fe6a5800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe698053ca0 0x7fe698054130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:05.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.551+0000 7f94afe00640 1 --2- 172.21.15.152:0/4117503173 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f949006c940 0x7f949006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:05.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.551+0000 7f94a3e00640 1 --2- 172.21.15.152:0/4117503173 >> [v2:172.21.15.152:6824/1121306353,v1:172.21.15.152:6825/1121306353] conn(0x7f9490074bc0 0x7f9490076fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:05.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.553+0000 7f94a3e00640 1 -- 172.21.15.152:0/4117503173 --> [v2:172.21.15.152:6824/1121306353,v1:172.21.15.152:6825/1121306353] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f94b0067e50 con 0x7f9490074bc0 2024-04-20T11:27:05.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.553+0000 7f94a3e00640 1 -- 172.21.15.152:0/4117503173 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f94b012d060 con 0x7f94a80ebb40 2024-04-20T11:27:05.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.553+0000 7fe69f400640 1 -- 172.21.15.152:0/3302975206 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1827151299 0 0) 0x7fe698054670 con 0x7fe6980e3150 2024-04-20T11:27:05.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.553+0000 7fe69f400640 1 -- 172.21.15.152:0/3302975206 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe688003520 con 0x7fe6980e3150 2024-04-20T11:27:05.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.553+0000 7f94af400640 1 --2- 172.21.15.152:0/4117503173 >> [v2:172.21.15.152:6824/1121306353,v1:172.21.15.152:6825/1121306353] conn(0x7f9490074bc0 0x7f9490076fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:05.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.553+0000 7fb216a00640 1 Processor -- start 2024-04-20T11:27:05.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.554+0000 7f94afe00640 1 --2- 172.21.15.152:0/4117503173 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f949006c940 0x7f949006ee00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f94a40013b0 tx=0x7f94a4001300 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:05.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.554+0000 7f94af400640 1 --2- 172.21.15.152:0/4117503173 >> [v2:172.21.15.152:6824/1121306353,v1:172.21.15.152:6825/1121306353] conn(0x7f9490074bc0 0x7f9490076fe0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:05.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.554+0000 7f638be00640 1 -- start start 2024-04-20T11:27:05.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.553+0000 7fb216a00640 1 -- start start 2024-04-20T11:27:05.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.555+0000 7fb216a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb20803e840 0x7fb2080d68b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:05.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.555+0000 7fb216a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb208054130 0x7fb2080e6a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:05.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.555+0000 7fb216a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb2080e30c0 0x7fb2080e3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:05.567 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.554+0000 7fe774000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe76c0d0720 0x7fe76c15ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:05.567 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.554+0000 7fe774000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe76c0d0b00 0x7fe76c156a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:05.567 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.554+0000 7fe774000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe76c0d0f60 0x7fe76c159450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:05.567 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.554+0000 7fe774000640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fe76c145ba0 con 0x7fe76c0d0b00 2024-04-20T11:27:05.567 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.554+0000 7fe774000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe76c145a20 con 0x7fe76c0d0720 2024-04-20T11:27:05.567 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.554+0000 7fe774000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe76c0b18d0 con 0x7fe76c0d0f60 2024-04-20T11:27:05.567 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.555+0000 7fb216a00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fb208006e10 con 0x7fb208054130 2024-04-20T11:27:05.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.553+0000 7fe6a5800640 1 -- 172.21.15.152:0/3302975206 >> v2:172.21.15.152:3300/0 conn(0x7fe69803c090 msgr2=0x7fe6980e7770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:05.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.553+0000 7fe6a5800640 1 --2- 172.21.15.152:0/3302975206 >> v2:172.21.15.152:3300/0 conn(0x7fe69803c090 0x7fe6980e7770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:05.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.553+0000 7fe6a5800640 1 -- 172.21.15.152:0/3302975206 >> v1:172.21.15.175:6789/0 conn(0x7fe6980e3150 legacy=0x7fe698058040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:05.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.553+0000 7fe6a5800640 1 -- 172.21.15.152:0/3302975206 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe698058ae0 con 0x7fe698053ca0 2024-04-20T11:27:05.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.555+0000 7fb216a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb2080e34a0 con 0x7fb2080e30c0 2024-04-20T11:27:05.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.555+0000 7fb216a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb208006f90 con 0x7fb20803e840 2024-04-20T11:27:05.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.556+0000 7fe772000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe76c0d0f60 0x7fe76c159450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:05.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.555+0000 7fb214c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb208054130 0x7fb2080e6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:05.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.556+0000 7fe772000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe76c0d0f60 0x7fe76c159450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.152:51848/0 (socket says 172.21.15.152:51848) 2024-04-20T11:27:05.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.556+0000 7fe772000640 1 -- 172.21.15.152:0/2589952259 learned_addr learned my addr 172.21.15.152:0/2589952259 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:05.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.556+0000 7fe6a5800640 1 --2- 172.21.15.152:0/3302975206 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe698053ca0 0x7fe698054130 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe6a006a100 tx=0x7fe6a00a7380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=996812351968cb38 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:05.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.555+0000 7fb215600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb20803e840 0x7fb2080d68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:05.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.555+0000 7fb214c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb208054130 0x7fb2080e6a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:42744/0 (socket says 172.21.15.152:42744) 2024-04-20T11:27:05.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.555+0000 7fb214c00640 1 -- 172.21.15.152:0/2064569994 learned_addr learned my addr 172.21.15.152:0/2064569994 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:05.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.556+0000 7fb216000640 1 --2- 172.21.15.152:0/2064569994 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb2080e30c0 0x7fb2080e3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:05.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.556+0000 7fb214c00640 1 -- 172.21.15.152:0/2064569994 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb20803e840 msgr2=0x7fb2080d68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:05.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.556+0000 7fb214c00640 1 --2- 172.21.15.152:0/2064569994 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb20803e840 0x7fb2080d68b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:05.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.556+0000 7fb214c00640 1 -- 172.21.15.152:0/2064569994 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb2080e30c0 msgr2=0x7fb2080e3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:05.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.556+0000 7fb214c00640 1 --2- 172.21.15.152:0/2064569994 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb2080e30c0 0x7fb2080e3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:05.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.556+0000 7fb214c00640 1 -- 172.21.15.152:0/2064569994 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2080e73c0 con 0x7fb208054130 2024-04-20T11:27:05.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.556+0000 7f94a3e00640 1 -- 172.21.15.152:0/4117503173 <== osd.0 v2:172.21.15.152:6824/1121306353 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f94b0067e50 con 0x7f9490074bc0 2024-04-20T11:27:05.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.557+0000 7fb214c00640 1 --2- 172.21.15.152:0/2064569994 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb208054130 0x7fb2080e6a20 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb210076cd0 tx=0x7fb210069770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:05.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.557+0000 7fb20d600640 1 -- 172.21.15.152:0/2064569994 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2100bc070 con 0x7fb208054130 2024-04-20T11:27:05.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.557+0000 7fb20d600640 1 -- 172.21.15.152:0/2064569994 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2100b8480 con 0x7fb208054130 2024-04-20T11:27:05.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.557+0000 7fb20d600640 1 -- 172.21.15.152:0/2064569994 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2100bc070 con 0x7fb208054130 2024-04-20T11:27:05.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.557+0000 7fb216a00640 1 -- 172.21.15.152:0/2064569994 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2080d7060 con 0x7fb208054130 2024-04-20T11:27:05.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.556+0000 7fe772000640 1 -- 172.21.15.152:0/2589952259 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe76c0d0720 msgr2=0x7fe76c15ed00 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-20T11:27:05.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.556+0000 7fe772000640 1 --2- 172.21.15.152:0/2589952259 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe76c0d0720 0x7fe76c15ed00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:05.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.557+0000 7fe772000640 1 -- 172.21.15.152:0/2589952259 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe76c0d0b00 msgr2=0x7fe76c156a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:05.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.557+0000 7fe772000640 1 --2- 172.21.15.152:0/2589952259 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe76c0d0b00 0x7fe76c156a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:05.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.557+0000 7fe772000640 1 -- 172.21.15.152:0/2589952259 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe76c1486a0 con 0x7fe76c0d0f60 2024-04-20T11:27:05.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.556+0000 7f638be00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f63840063d0 0x7f6384180410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:05.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.556+0000 7f638be00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6384057910 0x7f63840e3220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:05.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.556+0000 7f638be00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6384049b70 0x7f63840e3760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:05.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.556+0000 7f638be00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f6384039170 con 0x7f6384049b70 2024-04-20T11:27:05.574 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.556+0000 7f638be00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6384007860 con 0x7f6384057910 2024-04-20T11:27:05.574 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.556+0000 7f638be00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f63840392f0 con 0x7f63840063d0 2024-04-20T11:27:05.574 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.556+0000 7f638b400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6384049b70 0x7f63840e3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:05.574 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.556+0000 7f638b400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6384049b70 0x7f63840e3760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:42760/0 (socket says 172.21.15.152:42760) 2024-04-20T11:27:05.574 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.556+0000 7f638b400640 1 -- 172.21.15.152:0/994081729 learned_addr learned my addr 172.21.15.152:0/994081729 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:05.574 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.557+0000 7f638aa00640 1 --2- 172.21.15.152:0/994081729 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f63840063d0 0x7f6384180410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:05.574 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.557+0000 7fe69f400640 1 -- 172.21.15.152:0/3302975206 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6a00b3070 con 0x7fe698053ca0 2024-04-20T11:27:05.574 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.557+0000 7fe69f400640 1 -- 172.21.15.152:0/3302975206 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6a00ac810 con 0x7fe698053ca0 2024-04-20T11:27:05.574 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.557+0000 7fe69f400640 1 -- 172.21.15.152:0/3302975206 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6a00acc80 con 0x7fe698053ca0 2024-04-20T11:27:05.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.557+0000 7fb216a00640 1 -- 172.21.15.152:0/2064569994 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2080e6f60 con 0x7fb208054130 2024-04-20T11:27:05.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.558+0000 7f638a000640 1 --2- 172.21.15.152:0/994081729 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6384057910 0x7f63840e3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:05.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.558+0000 7fe772000640 1 --2- 172.21.15.152:0/2589952259 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe76c0d0f60 0x7fe76c159450 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe76400c730 tx=0x7fe76400cbf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:05.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.558+0000 7fe6a7800640 1 -- 172.21.15.152:0/3302975206 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe698053ca0 msgr2=0x7fe698054130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:05.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.558+0000 7fe6a7800640 1 --2- 172.21.15.152:0/3302975206 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe698053ca0 0x7fe698054130 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe6a006a100 tx=0x7fe6a00a7380 comp rx=0 tx=0).stop 2024-04-20T11:27:05.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.558+0000 7fe6a7800640 1 -- 172.21.15.152:0/3302975206 shutdown_connections 2024-04-20T11:27:05.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.558+0000 7fe6a7800640 1 --2- 172.21.15.152:0/3302975206 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe698053ca0 0x7fe698054130 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:05.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.558+0000 7fe6a7800640 1 --2- 172.21.15.152:0/3302975206 >> v2:172.21.15.152:3300/0 conn(0x7fe69803c090 0x7fe6980e7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:05.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.558+0000 7fe6a7800640 1 -- 172.21.15.152:0/3302975206 >> 172.21.15.152:0/3302975206 conn(0x7fe698058ed0 msgr2=0x7fe6980592b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:05.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.559+0000 7fe769600640 1 -- 172.21.15.152:0/2589952259 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe764018070 con 0x7fe76c0d0f60 2024-04-20T11:27:05.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.559+0000 7fe769600640 1 -- 172.21.15.152:0/2589952259 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe764004d60 con 0x7fe76c0d0f60 2024-04-20T11:27:05.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.559+0000 7fe769600640 1 -- 172.21.15.152:0/2589952259 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe76400a880 con 0x7fe76c0d0f60 2024-04-20T11:27:05.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.559+0000 7fb216a00640 1 -- 172.21.15.152:0/2064569994 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb2080057e0 con 0x7fb208054130 2024-04-20T11:27:05.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.558+0000 7f5293800640 1 Processor -- start 2024-04-20T11:27:05.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.558+0000 7f5293800640 1 -- start start 2024-04-20T11:27:05.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.560+0000 7fe6a7800640 1 -- 172.21.15.152:0/3302975206 shutdown_connections 2024-04-20T11:27:05.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.560+0000 7f5293800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f528c0d0dc0 0x7f528c0d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:05.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.560+0000 7fe6a7800640 1 -- 172.21.15.152:0/3302975206 wait complete. 2024-04-20T11:27:05.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.560+0000 7fe774000640 1 -- 172.21.15.152:0/2589952259 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe76c148980 con 0x7fe76c0d0f60 2024-04-20T11:27:05.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.560+0000 7fe774000640 1 -- 172.21.15.152:0/2589952259 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe76c148f40 con 0x7fe76c0d0f60 2024-04-20T11:27:05.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.560+0000 7f5293800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f528c0d1c20 0x7f528c0b7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:05.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.560+0000 7f5293800640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f528c0d2030 con 0x7f528c162df0 2024-04-20T11:27:05.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.560+0000 7f5293800640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f528c0d49a0 con 0x7f528c0d0dc0 2024-04-20T11:27:05.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.560+0000 7f5293800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f528c0d4c00 con 0x7f528c0d1c20 2024-04-20T11:27:05.580 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.560+0000 7f638b400640 1 -- 172.21.15.152:0/994081729 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f63840063d0 msgr2=0x7f6384180410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:05.580 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.560+0000 7f638b400640 1 --2- 172.21.15.152:0/994081729 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f63840063d0 0x7f6384180410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:05.580 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.560+0000 7f638b400640 1 -- 172.21.15.152:0/994081729 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6384057910 msgr2=0x7f63840e3220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:05.580 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.560+0000 7f638b400640 1 --2- 172.21.15.152:0/994081729 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6384057910 0x7f63840e3220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:05.580 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.560+0000 7f638b400640 1 -- 172.21.15.152:0/994081729 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f638404d2c0 con 0x7f6384049b70 2024-04-20T11:27:05.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.561+0000 7f5290e00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f528c162df0 0x7f528c0d4290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:40636/0 (socket says 172.21.15.152:40636) 2024-04-20T11:27:05.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.561+0000 7f5290e00640 1 -- 172.21.15.152:0/884081323 learned_addr learned my addr 172.21.15.152:0/884081323 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:27:05.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.561+0000 7f5291800640 1 --2- 172.21.15.152:0/884081323 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f528c0d1c20 0x7f528c0b7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:05.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.561+0000 7f528be00640 1 --2- 172.21.15.152:0/884081323 >> v2:172.21.15.152:3300/0 conn(0x7f528c0d0dc0 0x7f528c0d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:05.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.561+0000 7f638b400640 1 --2- 172.21.15.152:0/994081729 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6384049b70 0x7f63840e3760 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f637c00e860 tx=0x7f637c00ed20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:05.582 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.562+0000 7fb20d600640 1 -- 172.21.15.152:0/2064569994 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb2100bf070 con 0x7fb208054130 2024-04-20T11:27:05.582 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.562+0000 7ffa69a00640 1 Processor -- start 2024-04-20T11:27:05.582 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.562+0000 7f5291800640 1 -- 172.21.15.152:0/884081323 >> v2:172.21.15.152:3300/0 conn(0x7f528c0d0dc0 msgr2=0x7f528c0d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:05.582 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.562+0000 7f5291800640 1 --2- 172.21.15.152:0/884081323 >> v2:172.21.15.152:3300/0 conn(0x7f528c0d0dc0 0x7f528c0d16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:05.582 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.562+0000 7f5291800640 1 -- 172.21.15.152:0/884081323 >> v1:172.21.15.152:6789/0 conn(0x7f528c162df0 legacy=0x7f528c0d4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:05.583 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.562+0000 7f5291800640 1 -- 172.21.15.152:0/884081323 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f528c0d4ee0 con 0x7f528c0d1c20 2024-04-20T11:27:05.583 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.562+0000 7fe75ea00640 1 -- 172.21.15.152:0/2589952259 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe76c15f240 con 0x7fe76c0d0f60 2024-04-20T11:27:05.583 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.562+0000 7fe6a7800640 1 Processor -- start 2024-04-20T11:27:05.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.563+0000 7f6383400640 1 -- 172.21.15.152:0/994081729 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f637c017070 con 0x7f6384049b70 2024-04-20T11:27:05.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.563+0000 7f638be00640 1 -- 172.21.15.152:0/994081729 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f638404d530 con 0x7f6384049b70 2024-04-20T11:27:05.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.563+0000 7f638be00640 1 -- 172.21.15.152:0/994081729 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f638404b4b0 con 0x7f6384049b70 2024-04-20T11:27:05.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.564+0000 7f638be00640 1 -- 172.21.15.152:0/994081729 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6384005390 con 0x7f6384049b70 2024-04-20T11:27:05.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.562+0000 7ffa69a00640 1 -- start start 2024-04-20T11:27:05.585 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.563+0000 7fe6a7800640 1 -- start start 2024-04-20T11:27:05.585 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.564+0000 7fe6a7800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe69803c090 0x7fe69817c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:05.585 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.564+0000 7fe6a7800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe698053ca0 0x7fe69804eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:05.585 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.564+0000 7fe6a7800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe6980e3150 0x7fe69803d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:05.585 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.564+0000 7fe6a7800640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fe6980e3530 con 0x7fe698053ca0 2024-04-20T11:27:05.585 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.564+0000 7fe6a7800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe698006630 con 0x7fe69803c090 2024-04-20T11:27:05.585 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.564+0000 7fe6a7800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe698006b50 con 0x7fe6980e3150 2024-04-20T11:27:05.585 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.564+0000 7fb20d600640 1 --2- 172.21.15.152:0/2064569994 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fb1f006caf0 0x7fb1f006efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:05.585 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.564+0000 7fb20d600640 1 -- 172.21.15.152:0/2064569994 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb2101289c0 con 0x7fb208054130 2024-04-20T11:27:05.585 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.564+0000 7fb20d600640 1 --2- 172.21.15.152:0/2064569994 >> [v2:172.21.15.152:6808/1552206819,v1:172.21.15.152:6809/1552206819] conn(0x7fb1f0074cf0 0x7fb1f0077110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:05.586 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.565+0000 7fb20d600640 1 -- 172.21.15.152:0/2064569994 --> [v2:172.21.15.152:6808/1552206819,v1:172.21.15.152:6809/1552206819] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb210063200 con 0x7fb1f0074cf0 2024-04-20T11:27:05.586 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.565+0000 7fb20d600640 1 -- 172.21.15.152:0/2064569994 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fb21012d2c0 con 0x7fb208054130 2024-04-20T11:27:05.586 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.565+0000 7f6383400640 1 -- 172.21.15.152:0/994081729 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f637c00c940 con 0x7f6384049b70 2024-04-20T11:27:05.586 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.565+0000 7f6383400640 1 -- 172.21.15.152:0/994081729 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f637c017070 con 0x7f6384049b70 2024-04-20T11:27:05.586 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.565+0000 7fe6a4e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe69803c090 0x7fe69817c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:05.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.565+0000 7fe6a5800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe6980e3150 0x7fe69803d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:05.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.565+0000 7fb215600640 1 --2- 172.21.15.152:0/2064569994 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fb1f006caf0 0x7fb1f006efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:05.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.565+0000 7fe6a4e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe69803c090 0x7fe69817c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.152:50082/0 (socket says 172.21.15.152:50082) 2024-04-20T11:27:05.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.565+0000 7fe6a4e00640 1 -- 172.21.15.152:0/4039569064 learned_addr learned my addr 172.21.15.152:0/4039569064 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:05.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.565+0000 7fb216000640 1 --2- 172.21.15.152:0/2064569994 >> [v2:172.21.15.152:6808/1552206819,v1:172.21.15.152:6809/1552206819] conn(0x7fb1f0074cf0 0x7fb1f0077110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:05.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.565+0000 7ffa69a00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ffa640af960 0x7ffa640b0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:05.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.566+0000 7ffa69a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa640b06a0 0x7ffa6406efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:05.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.566+0000 7ffa69a00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa640b4b80 con 0x7ffa6415b680 2024-04-20T11:27:05.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.566+0000 7ffa69a00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7ffa640b4e60 con 0x7ffa640af960 2024-04-20T11:27:05.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.566+0000 7ffa69a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ffa640b5140 con 0x7ffa640b06a0 2024-04-20T11:27:05.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.565+0000 7f5291800640 1 --2- 172.21.15.152:0/884081323 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f528c0d1c20 0x7f528c0b7aa0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5284009d90 tx=0x7f528402bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14775a917dd8f934 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:05.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.566+0000 7f528b400640 1 -- 172.21.15.152:0/884081323 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5284039070 con 0x7f528c0d1c20 2024-04-20T11:27:05.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.566+0000 7f528b400640 1 -- 172.21.15.152:0/884081323 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f528402cc50 con 0x7f528c0d1c20 2024-04-20T11:27:05.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.566+0000 7f528b400640 1 -- 172.21.15.152:0/884081323 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5284034440 con 0x7f528c0d1c20 2024-04-20T11:27:05.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.566+0000 7fb215600640 1 --2- 172.21.15.152:0/2064569994 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fb1f006caf0 0x7fb1f006efb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc005880 tx=0x7fb1fc009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:05.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.566+0000 7fe769600640 1 -- 172.21.15.152:0/2589952259 <== mon.2 v2:172.21.15.175:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe76400ac30 con 0x7fe76c0d0f60 2024-04-20T11:27:05.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.565+0000 7ffa62a00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7ffa6415b680 0x7ffa640b43a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:40646/0 (socket says 172.21.15.152:40646) 2024-04-20T11:27:05.590 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.566+0000 7ffa62a00640 1 -- 172.21.15.152:0/3306225649 learned_addr learned my addr 172.21.15.152:0/3306225649 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:27:05.590 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.566+0000 7ffa5be00640 1 --2- 172.21.15.152:0/3306225649 >> v2:172.21.15.175:3300/0 conn(0x7ffa640af960 0x7ffa640b0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:05.590 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.566+0000 7fe69fe00640 1 --2- 172.21.15.152:0/4039569064 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe698053ca0 0x7fe69804eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:05.590 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.566+0000 7fe6a4e00640 1 -- 172.21.15.152:0/4039569064 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe6980e3150 msgr2=0x7fe69803d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:05.590 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.566+0000 7fe6a4e00640 1 --2- 172.21.15.152:0/4039569064 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe6980e3150 0x7fe69803d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:05.590 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.566+0000 7fe6a4e00640 1 -- 172.21.15.152:0/4039569064 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe698053ca0 msgr2=0x7fe69804eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:05.590 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.566+0000 7fe6a4e00640 1 --2- 172.21.15.152:0/4039569064 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe698053ca0 0x7fe69804eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:05.590 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.566+0000 7fe6a4e00640 1 -- 172.21.15.152:0/4039569064 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe69817ca50 con 0x7fe69803c090 2024-04-20T11:27:05.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.567+0000 7f5293800640 1 -- 172.21.15.152:0/884081323 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f528c0d1c20 msgr2=0x7f528c0b7aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:05.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.567+0000 7f5293800640 1 --2- 172.21.15.152:0/884081323 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f528c0d1c20 0x7f528c0b7aa0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5284009d90 tx=0x7f528402bea0 comp rx=0 tx=0).stop 2024-04-20T11:27:05.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.567+0000 7f5293800640 1 -- 172.21.15.152:0/884081323 shutdown_connections 2024-04-20T11:27:05.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.567+0000 7f5293800640 1 --2- 172.21.15.152:0/884081323 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f528c0d1c20 0x7f528c0b7aa0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:05.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.567+0000 7f5293800640 1 --2- 172.21.15.152:0/884081323 >> v2:172.21.15.152:3300/0 conn(0x7f528c0d0dc0 0x7f528c0d16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:05.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.567+0000 7f5293800640 1 -- 172.21.15.152:0/884081323 >> 172.21.15.152:0/884081323 conn(0x7f528c067ae0 msgr2=0x7f528c067ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:05.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.567+0000 7f5293800640 1 -- 172.21.15.152:0/884081323 shutdown_connections 2024-04-20T11:27:05.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.567+0000 7f5293800640 1 -- 172.21.15.152:0/884081323 wait complete. 2024-04-20T11:27:05.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.567+0000 7ffa5be00640 1 -- 172.21.15.152:0/3306225649 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa640b06a0 msgr2=0x7ffa6406efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:05.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.567+0000 7ffa63400640 1 --2- 172.21.15.152:0/3306225649 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa640b06a0 0x7ffa6406efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:05.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.567+0000 7ffa5be00640 1 --2- 172.21.15.152:0/3306225649 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa640b06a0 0x7ffa6406efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:05.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.567+0000 7ffa5be00640 1 -- 172.21.15.152:0/3306225649 >> v1:172.21.15.152:6789/0 conn(0x7ffa6415b680 legacy=0x7ffa640b43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:05.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.568+0000 7fe769600640 1 --2- 172.21.15.152:0/2589952259 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe74c06c8f0 0x7fe74c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:05.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.568+0000 7fa514a00640 1 Processor -- start 2024-04-20T11:27:05.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.568+0000 7fa514a00640 1 -- start start 2024-04-20T11:27:05.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.568+0000 7fe6a4e00640 1 --2- 172.21.15.152:0/4039569064 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe69803c090 0x7fe69817c3c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe69400c410 tx=0x7fe69400c8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:05.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.568+0000 7f6383400640 1 -- 172.21.15.152:0/994081729 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f637c01a070 con 0x7f6384049b70 2024-04-20T11:27:05.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.568+0000 7fe769600640 1 -- 172.21.15.152:0/2589952259 <== mon.2 v2:172.21.15.175:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fe76408b400 con 0x7fe76c0d0f60 2024-04-20T11:27:05.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.568+0000 7fe769600640 1 --2- 172.21.15.152:0/2589952259 >> [v2:172.21.15.152:6800/2519167656,v1:172.21.15.152:6801/2519167656] conn(0x7fe74c074b70 0x7fe74c076f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:05.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.568+0000 7fe769600640 1 -- 172.21.15.152:0/2589952259 --> [v2:172.21.15.152:6800/2519167656,v1:172.21.15.152:6801/2519167656] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe74c077640 con 0x7fe74c074b70 2024-04-20T11:27:05.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.568+0000 7fe769600640 1 -- 172.21.15.152:0/2589952259 <== mon.2 v2:172.21.15.175:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fe76408b7f0 con 0x7fe76c0d0f60 2024-04-20T11:27:05.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.568+0000 7ffa5be00640 1 -- 172.21.15.152:0/3306225649 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa640b5420 con 0x7ffa640af960 2024-04-20T11:27:05.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.569+0000 7fe771600640 1 --2- 172.21.15.152:0/2589952259 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe74c06c8f0 0x7fe74c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:05.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.569+0000 7fe770c00640 1 --2- 172.21.15.152:0/2589952259 >> [v2:172.21.15.152:6800/2519167656,v1:172.21.15.152:6801/2519167656] conn(0x7fe74c074b70 0x7fe74c076f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:05.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.569+0000 7fe69cc00640 1 -- 172.21.15.152:0/4039569064 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe694017070 con 0x7fe69803c090 2024-04-20T11:27:05.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.570+0000 7fe69cc00640 1 -- 172.21.15.152:0/4039569064 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe694005ce0 con 0x7fe69803c090 2024-04-20T11:27:05.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.570+0000 7fe69cc00640 1 -- 172.21.15.152:0/4039569064 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6940072f0 con 0x7fe69803c090 2024-04-20T11:27:05.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.570+0000 7fe770c00640 1 --2- 172.21.15.152:0/2589952259 >> [v2:172.21.15.152:6800/2519167656,v1:172.21.15.152:6801/2519167656] conn(0x7fe74c074b70 0x7fe74c076f90 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:05.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.570+0000 7f5293800640 1 Processor -- start 2024-04-20T11:27:05.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.570+0000 7fa514a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa5100d0810 0x7fa5101693f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:05.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.570+0000 7fa514a00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa51015fd20 con 0x7fa5100b4500 2024-04-20T11:27:05.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.570+0000 7fa514a00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa51015ef60 con 0x7fa51015c560 2024-04-20T11:27:05.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.570+0000 7fa514a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa51015f240 con 0x7fa5100d0810 2024-04-20T11:27:05.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.570+0000 7fa50de00640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7fa51015c560 0x7fa5100b3df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.152:49256/0 (socket says 172.21.15.152:49256) 2024-04-20T11:27:05.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.570+0000 7fa50de00640 1 -- 172.21.15.152:0/1724502136 learned_addr learned my addr 172.21.15.152:0/1724502136 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:27:05.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.570+0000 7f6383400640 1 --2- 172.21.15.152:0/994081729 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f636c06caf0 0x7f636c06efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:05.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.570+0000 7fe6a7800640 1 -- 172.21.15.152:0/4039569064 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe69817cd30 con 0x7fe69803c090 2024-04-20T11:27:05.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.570+0000 7fe6a7800640 1 -- 172.21.15.152:0/4039569064 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe698178fe0 con 0x7fe69803c090 2024-04-20T11:27:05.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.571+0000 7fa50e800640 1 --2- 172.21.15.152:0/1724502136 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa5100d0810 0x7fa5101693f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:05.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.571+0000 7fb216000640 1 --2- 172.21.15.152:0/2064569994 >> [v2:172.21.15.152:6808/1552206819,v1:172.21.15.152:6809/1552206819] conn(0x7fb1f0074cf0 0x7fb1f0077110 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:05.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.571+0000 7fe771600640 1 --2- 172.21.15.152:0/2589952259 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe74c06c8f0 0x7fe74c06edb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe7580099d0 tx=0x7fe758009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:05.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.571+0000 7f6383400640 1 -- 172.21.15.152:0/994081729 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f637c08b9c0 con 0x7f6384049b70 2024-04-20T11:27:05.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.571+0000 7f6383400640 1 --2- 172.21.15.152:0/994081729 >> [v2:172.21.15.152:6816/3995926281,v1:172.21.15.152:6817/3995926281] conn(0x7f636c074d10 0x7f636c077130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:05.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.571+0000 7f6383400640 1 -- 172.21.15.152:0/994081729 --> [v2:172.21.15.152:6816/3995926281,v1:172.21.15.152:6817/3995926281] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f637c00ae90 con 0x7f636c074d10 2024-04-20T11:27:05.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.571+0000 7f6383400640 1 -- 172.21.15.152:0/994081729 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f637c08bdb0 con 0x7f6384049b70 2024-04-20T11:27:05.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.571+0000 7ffa5be00640 1 --2- 172.21.15.152:0/3306225649 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa640af960 0x7ffa640b0160 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c0097a0 tx=0x7ffa4c02b940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99b65e3ac0f1d745 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:05.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.572+0000 7fe769600640 1 -- 172.21.15.152:0/2589952259 <== osd.2 v2:172.21.15.152:6800/2519167656 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe74c077640 con 0x7fe74c074b70 2024-04-20T11:27:05.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.570+0000 7f5293800640 1 -- start start 2024-04-20T11:27:05.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.572+0000 7f5293800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f528c0d0dc0 0x7f528c1631d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:05.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.572+0000 7f5293800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f528c0d1c20 0x7f528c150c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:05.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.572+0000 7f5293800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f528c162df0 0x7f528c0ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:05.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.572+0000 7f5293800640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f528c0b7fe0 con 0x7f528c0d0dc0 2024-04-20T11:27:05.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.572+0000 7f5293800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f528c0d54a0 con 0x7f528c0d1c20 2024-04-20T11:27:05.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.572+0000 7f5293800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f528c0b8160 con 0x7f528c162df0 2024-04-20T11:27:05.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.572+0000 7fb20d600640 1 -- 172.21.15.152:0/2064569994 <== osd.1 v2:172.21.15.152:6808/1552206819 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb210063200 con 0x7fb1f0074cf0 2024-04-20T11:27:05.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.571+0000 7f638aa00640 1 --2- 172.21.15.152:0/994081729 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f636c06caf0 0x7f636c06efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:05.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.571+0000 7f638a000640 1 --2- 172.21.15.152:0/994081729 >> [v2:172.21.15.152:6816/3995926281,v1:172.21.15.152:6817/3995926281] conn(0x7f636c074d10 0x7f636c077130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:05.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.572+0000 7f638aa00640 1 --2- 172.21.15.152:0/994081729 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f636c06caf0 0x7f636c06efb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6384011390 tx=0x7f638c0b0000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:05.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.572+0000 7fa50d400640 1 -- 172.21.15.152:0/1724502136 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 131687399 0 0) 0x7fa51015ef60 con 0x7fa51015c560 2024-04-20T11:27:05.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.572+0000 7fa50d400640 1 -- 172.21.15.152:0/1724502136 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa4f4003520 con 0x7fa51015c560 2024-04-20T11:27:05.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.572+0000 7fa50d400640 1 -- 172.21.15.152:0/1724502136 <== mon.1 v1:172.21.15.175:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2653981071 0 0) 0x7fa4f4003520 con 0x7fa51015c560 2024-04-20T11:27:05.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.573+0000 7fa50d400640 1 -- 172.21.15.152:0/1724502136 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa5100d0810 msgr2=0x7fa5101693f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:05.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.573+0000 7fa50d400640 1 --2- 172.21.15.152:0/1724502136 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa5100d0810 0x7fa5101693f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:05.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.573+0000 7fa50d400640 1 -- 172.21.15.152:0/1724502136 >> v1:172.21.15.152:6789/0 conn(0x7fa5100b4500 legacy=0x7fa5100d0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:05.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.573+0000 7f5290e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f528c0d0dc0 0x7f528c1631d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:05.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.573+0000 7f638a000640 1 --2- 172.21.15.152:0/994081729 >> [v2:172.21.15.152:6816/3995926281,v1:172.21.15.152:6817/3995926281] conn(0x7f636c074d10 0x7f636c077130 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:05.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.573+0000 7fa50d400640 1 -- 172.21.15.152:0/1724502136 --> v1:172.21.15.175:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa51015f520 con 0x7fa51015c560 2024-04-20T11:27:05.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.573+0000 7fa50d400640 1 -- 172.21.15.152:0/1724502136 <== mon.1 v1:172.21.15.175:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7fa5000019f0 con 0x7fa51015c560 2024-04-20T11:27:05.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.573+0000 7f5290e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f528c0d0dc0 0x7f528c1631d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:42778/0 (socket says 172.21.15.152:42778) 2024-04-20T11:27:05.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.573+0000 7fa50d400640 1 -- 172.21.15.152:0/1724502136 >> v1:172.21.15.175:6789/0 conn(0x7fa51015c560 legacy=0x7fa5100b3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:05.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.573+0000 7f5290e00640 1 -- 172.21.15.152:0/2387096322 learned_addr learned my addr 172.21.15.152:0/2387096322 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:05.607 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.573+0000 7fa50d400640 1 --2- 172.21.15.152:0/1724502136 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa4f4003a20 0x7fa4f4003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:05.607 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.573+0000 7fa50d400640 1 --2- 172.21.15.152:0/1724502136 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa5100d0810 0x7fa4f4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:05.607 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.573+0000 7fa50d400640 1 --2- 172.21.15.152:0/1724502136 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa4f4004d40 0x7fa4f40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:05.607 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.573+0000 7fa50d400640 1 -- 172.21.15.152:0/1724502136 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fa51015f240 con 0x7fa4f4004d40 2024-04-20T11:27:05.607 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.573+0000 7fa50d400640 1 -- 172.21.15.152:0/1724502136 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f400bb50 con 0x7fa4f4003a20 2024-04-20T11:27:05.607 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.573+0000 7fa50d400640 1 -- 172.21.15.152:0/1724502136 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa4f400bd50 con 0x7fa5100d0810 2024-04-20T11:27:05.607 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.573+0000 7fe6a7800640 1 -- 172.21.15.152:0/4039569064 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe69804f000 con 0x7fe69803c090 2024-04-20T11:27:05.608 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.573+0000 7ffa62000640 1 -- 172.21.15.152:0/3306225649 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa4c034070 con 0x7ffa640af960 2024-04-20T11:27:05.608 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.573+0000 7ffa62000640 1 -- 172.21.15.152:0/3306225649 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa4c030800 con 0x7ffa640af960 2024-04-20T11:27:05.608 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.573+0000 7ffa62000640 1 -- 172.21.15.152:0/3306225649 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa4c030bd0 con 0x7ffa640af960 2024-04-20T11:27:05.608 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.573+0000 7f5291800640 1 --2- 172.21.15.152:0/2387096322 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f528c162df0 0x7f528c0ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:05.608 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.573+0000 7fa50e800640 1 --2- 172.21.15.152:0/1724502136 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa4f4004d40 0x7fa4f40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:05.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.575+0000 7f6383400640 1 -- 172.21.15.152:0/994081729 <== osd.3 v2:172.21.15.152:6816/3995926281 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f637c00ae90 con 0x7f636c074d10 2024-04-20T11:27:05.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.574+0000 7f5290e00640 1 -- 172.21.15.152:0/2387096322 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f528c162df0 msgr2=0x7f528c0ccc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:05.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.574+0000 7f5290e00640 1 --2- 172.21.15.152:0/2387096322 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f528c162df0 0x7f528c0ccc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:05.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.574+0000 7f5290e00640 1 -- 172.21.15.152:0/2387096322 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f528c0d1c20 msgr2=0x7f528c150c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:05.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.574+0000 7f5290e00640 1 --2- 172.21.15.152:0/2387096322 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f528c0d1c20 0x7f528c150c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:05.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.574+0000 7f5290e00640 1 -- 172.21.15.152:0/2387096322 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f528c1f9cc0 con 0x7f528c0d0dc0 2024-04-20T11:27:05.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.575+0000 7f5290e00640 1 --2- 172.21.15.152:0/2387096322 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f528c0d0dc0 0x7f528c1631d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f528000c390 tx=0x7f528000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:05.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.575+0000 7f5288c00640 1 -- 172.21.15.152:0/2387096322 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5280017070 con 0x7f528c0d0dc0 2024-04-20T11:27:05.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.575+0000 7f5288c00640 1 -- 172.21.15.152:0/2387096322 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5280005d00 con 0x7f528c0d0dc0 2024-04-20T11:27:05.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.576+0000 7f5288c00640 1 -- 172.21.15.152:0/2387096322 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5280007310 con 0x7f528c0d0dc0 2024-04-20T11:27:05.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.576+0000 7f5293800640 1 -- 172.21.15.152:0/2387096322 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f528c1f9fa0 con 0x7f528c0d0dc0 2024-04-20T11:27:05.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.576+0000 7f5293800640 1 -- 172.21.15.152:0/2387096322 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f528c1fa560 con 0x7f528c0d0dc0 2024-04-20T11:27:05.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.575+0000 7ffa69a00640 1 -- 172.21.15.152:0/3306225649 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa640af960 msgr2=0x7ffa640b0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:05.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.575+0000 7ffa69a00640 1 --2- 172.21.15.152:0/3306225649 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa640af960 0x7ffa640b0160 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c0097a0 tx=0x7ffa4c02b940 comp rx=0 tx=0).stop 2024-04-20T11:27:05.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.575+0000 7ffa69a00640 1 -- 172.21.15.152:0/3306225649 shutdown_connections 2024-04-20T11:27:05.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.575+0000 7ffa69a00640 1 --2- 172.21.15.152:0/3306225649 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa640b06a0 0x7ffa6406efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:05.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.575+0000 7ffa69a00640 1 --2- 172.21.15.152:0/3306225649 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa640af960 0x7ffa640b0160 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:05.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.575+0000 7ffa69a00640 1 -- 172.21.15.152:0/3306225649 >> 172.21.15.152:0/3306225649 conn(0x7ffa6409aa40 msgr2=0x7ffa640b68c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:05.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.575+0000 7ffa69a00640 1 -- 172.21.15.152:0/3306225649 shutdown_connections 2024-04-20T11:27:05.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.575+0000 7ffa69a00640 1 -- 172.21.15.152:0/3306225649 wait complete. 2024-04-20T11:27:05.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.574+0000 7fa50e800640 1 -- 172.21.15.152:0/1724502136 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa5100d0810 msgr2=0x7fa4f4004800 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-20T11:27:05.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.574+0000 7fa50e800640 1 --2- 172.21.15.152:0/1724502136 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa5100d0810 0x7fa4f4004800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:05.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.574+0000 7fa50e800640 1 -- 172.21.15.152:0/1724502136 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa4f4003a20 msgr2=0x7fa4f4003e40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T11:27:05.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.574+0000 7fa50e800640 1 --2- 172.21.15.152:0/1724502136 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa4f4003a20 0x7fa4f4003e40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:05.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.574+0000 7fa50e800640 1 -- 172.21.15.152:0/1724502136 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa51015f520 con 0x7fa4f4004d40 2024-04-20T11:27:05.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.577+0000 7fa50e800640 1 --2- 172.21.15.152:0/1724502136 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa4f4004d40 0x7fa4f40051c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa508009ac0 tx=0x7fa4f4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fefef5ea2da47e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:05.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.577+0000 7ffa69a00640 1 Processor -- start 2024-04-20T11:27:05.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.577+0000 7fe69cc00640 1 -- 172.21.15.152:0/4039569064 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe69401a070 con 0x7fe69803c090 2024-04-20T11:27:05.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.578+0000 7fe69cc00640 1 --2- 172.21.15.152:0/4039569064 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe68006c940 0x7fe68006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:05.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.578+0000 7fe69cc00640 1 -- 172.21.15.152:0/4039569064 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fe6940575a0 con 0x7fe69803c090 2024-04-20T11:27:05.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.577+0000 7ffa69a00640 1 -- start start 2024-04-20T11:27:05.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.579+0000 7fe69cc00640 1 --2- 172.21.15.152:0/4039569064 >> [v2:172.21.15.175:6800/1235215467,v1:172.21.15.175:6802/1235215467] conn(0x7fe680075840 0x7fe680077ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:05.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.579+0000 7ffa69a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffa640af960 0x7ffa6414e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:05.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.579+0000 7ffa69a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa640b06a0 0x7ffa640b2850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:05.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.579+0000 7ffa69a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa6415b680 0x7ffa64151f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:05.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.579+0000 7fe69cc00640 1 -- 172.21.15.152:0/4039569064 --> [v2:172.21.15.175:6800/1235215467,v1:172.21.15.175:6802/1235215467] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe6800783b0 con 0x7fe680075840 2024-04-20T11:27:05.614 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.579+0000 7ffa69a00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7ffa6406f510 con 0x7ffa640af960 2024-04-20T11:27:05.614 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.579+0000 7ffa69a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ffa6415ba60 con 0x7ffa640b06a0 2024-04-20T11:27:05.614 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.579+0000 7ffa69a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ffa6406f690 con 0x7ffa6415b680 2024-04-20T11:27:05.614 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.579+0000 7fe69cc00640 1 -- 172.21.15.152:0/4039569064 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fe694055f30 con 0x7fe69803c090 2024-04-20T11:27:05.614 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.579+0000 7fe69fe00640 1 --2- 172.21.15.152:0/4039569064 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe68006c940 0x7fe68006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:05.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.579+0000 7ffa62a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffa640af960 0x7ffa6414e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:05.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.579+0000 7fa50d400640 1 -- 172.21.15.152:0/1724502136 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa508039070 con 0x7fa4f4004d40 2024-04-20T11:27:05.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.580+0000 7fa50d400640 1 -- 172.21.15.152:0/1724502136 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa50802c6d0 con 0x7fa4f4004d40 2024-04-20T11:27:05.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.580+0000 7fa50d400640 1 -- 172.21.15.152:0/1724502136 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa508009690 con 0x7fa4f4004d40 2024-04-20T11:27:05.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.580+0000 7f5293800640 1 -- 172.21.15.152:0/2387096322 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f528c0689f0 con 0x7f528c0d0dc0 2024-04-20T11:27:05.616 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.580+0000 7fa514a00640 1 -- 172.21.15.152:0/1724502136 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa4f4004d40 msgr2=0x7fa4f40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:05.616 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.580+0000 7fa514a00640 1 --2- 172.21.15.152:0/1724502136 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa4f4004d40 0x7fa4f40051c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa508009ac0 tx=0x7fa4f4005830 comp rx=0 tx=0).stop 2024-04-20T11:27:05.616 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.580+0000 7ffa62a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffa640af960 0x7ffa6414e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:42810/0 (socket says 172.21.15.152:42810) 2024-04-20T11:27:05.616 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.580+0000 7ffa62a00640 1 -- 172.21.15.152:0/317650018 learned_addr learned my addr 172.21.15.152:0/317650018 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:05.616 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.580+0000 7ffa62a00640 1 -- 172.21.15.152:0/317650018 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa6415b680 msgr2=0x7ffa64151f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:05.616 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.580+0000 7ffa5be00640 1 --2- 172.21.15.152:0/317650018 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa640b06a0 0x7ffa640b2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:05.616 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.580+0000 7ffa63400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa6415b680 0x7ffa64151f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:05.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.580+0000 7fa50e800640 1 -- 172.21.15.152:0/1724502136 reap_dead start 2024-04-20T11:27:05.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.581+0000 7fa514a00640 1 -- 172.21.15.152:0/1724502136 shutdown_connections 2024-04-20T11:27:05.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.581+0000 7fa514a00640 1 -- 172.21.15.152:0/1724502136 >> 172.21.15.152:0/1724502136 conn(0x7fa510067cf0 msgr2=0x7fa51006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:05.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.581+0000 7ffa62a00640 1 --2- 172.21.15.152:0/317650018 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa6415b680 0x7ffa64151f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:05.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.581+0000 7ffa62a00640 1 -- 172.21.15.152:0/317650018 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa640b06a0 msgr2=0x7ffa640b2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:05.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.581+0000 7ffa62a00640 1 --2- 172.21.15.152:0/317650018 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa640b06a0 0x7ffa640b2850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:05.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.581+0000 7ffa62a00640 1 -- 172.21.15.152:0/317650018 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa640b3ae0 con 0x7ffa640af960 2024-04-20T11:27:05.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.581+0000 7fa514a00640 1 -- 172.21.15.152:0/1724502136 shutdown_connections 2024-04-20T11:27:05.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.581+0000 7ffa5be00640 1 --2- 172.21.15.152:0/317650018 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa640b06a0 0x7ffa640b2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:27:05.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.581+0000 7ffa63400640 1 --2- 172.21.15.152:0/317650018 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa6415b680 0x7ffa64151f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:27:05.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.581+0000 7fa514a00640 1 -- 172.21.15.152:0/1724502136 wait complete. 2024-04-20T11:27:05.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.581+0000 7ffa62a00640 1 --2- 172.21.15.152:0/317650018 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffa640af960 0x7ffa6414e5e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ffa5400c390 tx=0x7ffa5400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:05.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.580+0000 7fe6a5800640 1 --2- 172.21.15.152:0/4039569064 >> [v2:172.21.15.175:6800/1235215467,v1:172.21.15.175:6802/1235215467] conn(0x7fe680075840 0x7fe680077ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:05.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.584+0000 7f5288c00640 1 -- 172.21.15.152:0/2387096322 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f528001a070 con 0x7f528c0d0dc0 2024-04-20T11:27:05.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.584+0000 7fa514a00640 1 Processor -- start 2024-04-20T11:27:05.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.585+0000 7fe69fe00640 1 --2- 172.21.15.152:0/4039569064 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe68006c940 0x7fe68006ee00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe69804e650 tx=0x7fe68c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:05.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.585+0000 7fe6a5800640 1 --2- 172.21.15.152:0/4039569064 >> [v2:172.21.15.175:6800/1235215467,v1:172.21.15.175:6802/1235215467] conn(0x7fe680075840 0x7fe680077ce0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:05.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.585+0000 7f5288c00640 1 --2- 172.21.15.152:0/2387096322 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f526c06c8f0 0x7f526c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:05.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.585+0000 7f5288c00640 1 -- 172.21.15.152:0/2387096322 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f528008b6d0 con 0x7f528c0d0dc0 2024-04-20T11:27:05.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.586+0000 7ffa5aa00640 1 -- 172.21.15.152:0/317650018 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa54017070 con 0x7ffa640af960 2024-04-20T11:27:05.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.586+0000 7ffa5aa00640 1 -- 172.21.15.152:0/317650018 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa54005ce0 con 0x7ffa640af960 2024-04-20T11:27:05.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.586+0000 7ffa5aa00640 1 -- 172.21.15.152:0/317650018 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa540072d0 con 0x7ffa640af960 2024-04-20T11:27:05.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.586+0000 7ffa69a00640 1 -- 172.21.15.152:0/317650018 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa640b17d0 con 0x7ffa640af960 2024-04-20T11:27:05.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.586+0000 7ffa69a00640 1 -- 172.21.15.152:0/317650018 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa640b1d90 con 0x7ffa640af960 2024-04-20T11:27:05.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.586+0000 7f5288c00640 1 --2- 172.21.15.152:0/2387096322 >> [v2:172.21.15.175:6801/3465937300,v1:172.21.15.175:6804/3465937300] conn(0x7f526c074b10 0x7f526c076f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:05.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.586+0000 7f5288c00640 1 -- 172.21.15.152:0/2387096322 --> [v2:172.21.15.175:6801/3465937300,v1:172.21.15.175:6804/3465937300] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5280010e50 con 0x7f526c074b10 2024-04-20T11:27:05.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.586+0000 7f5288c00640 1 -- 172.21.15.152:0/2387096322 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f528008bac0 con 0x7f528c0d0dc0 2024-04-20T11:27:05.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.586+0000 7fa514a00640 1 -- start start 2024-04-20T11:27:05.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.588+0000 7fe69cc00640 1 -- 172.21.15.152:0/4039569064 <== osd.4 v2:172.21.15.175:6800/1235215467 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe6800783b0 con 0x7fe680075840 2024-04-20T11:27:05.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.589+0000 7ffa69a00640 1 -- 172.21.15.152:0/317650018 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ffa64069100 con 0x7ffa640af960 2024-04-20T11:27:05.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.590+0000 7f5291800640 1 --2- 172.21.15.152:0/2387096322 >> [v2:172.21.15.175:6801/3465937300,v1:172.21.15.175:6804/3465937300] conn(0x7f526c074b10 0x7f526c076f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:05.627 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.588+0000 7fa514a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa4f4004d40 0x7fa510160c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:05.627 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.588+0000 7fa514a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa5100cdbf0 0x7fa5101501e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:05.627 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.588+0000 7fa514a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa51015c940 0x7fa51015cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:05.627 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.588+0000 7fa514a00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fa5100aeaa0 con 0x7fa4f4004d40 2024-04-20T11:27:05.627 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.588+0000 7fa514a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa5100d0dc0 con 0x7fa51015c940 2024-04-20T11:27:05.627 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.588+0000 7fa514a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa5100aec20 con 0x7fa5100cdbf0 2024-04-20T11:27:05.627 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.588+0000 7fa505400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa5100cdbf0 0x7fa5101501e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:05.627 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.589+0000 7fa505400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa5100cdbf0 0x7fa5101501e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.152:51922/0 (socket says 172.21.15.152:51922) 2024-04-20T11:27:05.627 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.589+0000 7fa505400640 1 -- 172.21.15.152:0/3263806857 learned_addr learned my addr 172.21.15.152:0/3263806857 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:05.627 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.589+0000 7fa50e800640 1 --2- 172.21.15.152:0/3263806857 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa51015c940 0x7fa51015cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:05.627 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.589+0000 7fa505400640 1 -- 172.21.15.152:0/3263806857 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa51015c940 msgr2=0x7fa51015cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:05.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.589+0000 7fa505400640 1 --2- 172.21.15.152:0/3263806857 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa51015c940 0x7fa51015cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:05.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.589+0000 7fa505400640 1 -- 172.21.15.152:0/3263806857 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa4f4004d40 msgr2=0x7fa510160c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:05.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.589+0000 7fa505400640 1 --2- 172.21.15.152:0/3263806857 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa4f4004d40 0x7fa510160c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:05.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.589+0000 7fa505400640 1 -- 172.21.15.152:0/3263806857 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5100ce360 con 0x7fa5100cdbf0 2024-04-20T11:27:05.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.590+0000 7fa505400640 1 --2- 172.21.15.152:0/3263806857 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa5100cdbf0 0x7fa5101501e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa4f800c460 tx=0x7fa4f800c920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:05.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.590+0000 7fa506a00640 1 -- 172.21.15.152:0/3263806857 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f80052d0 con 0x7fa5100cdbf0 2024-04-20T11:27:05.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.590+0000 7fa506a00640 1 -- 172.21.15.152:0/3263806857 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4f8005470 con 0x7fa5100cdbf0 2024-04-20T11:27:05.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.591+0000 7f5291800640 1 --2- 172.21.15.152:0/2387096322 >> [v2:172.21.15.175:6801/3465937300,v1:172.21.15.175:6804/3465937300] conn(0x7f526c074b10 0x7f526c076f30 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:05.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.590+0000 7fa506a00640 1 -- 172.21.15.152:0/3263806857 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f8012850 con 0x7fa5100cdbf0 2024-04-20T11:27:05.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.592+0000 7f528be00640 1 --2- 172.21.15.152:0/2387096322 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f526c06c8f0 0x7f526c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:05.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.591+0000 7fa514a00640 1 -- 172.21.15.152:0/3263806857 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5100ce640 con 0x7fa5100cdbf0 2024-04-20T11:27:05.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.591+0000 7fa514a00640 1 -- 172.21.15.152:0/3263806857 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa510155be0 con 0x7fa5100cdbf0 2024-04-20T11:27:05.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.592+0000 7fa514a00640 1 -- 172.21.15.152:0/3263806857 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa5100682e0 con 0x7fa5100cdbf0 2024-04-20T11:27:05.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.593+0000 7f528be00640 1 --2- 172.21.15.152:0/2387096322 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f526c06c8f0 0x7f526c06edb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f528c0b64a0 tx=0x7f5278008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:05.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.591+0000 7ffa5aa00640 1 -- 172.21.15.152:0/317650018 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ffa5401a070 con 0x7ffa640af960 2024-04-20T11:27:05.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.593+0000 7ffa5aa00640 1 --2- 172.21.15.152:0/317650018 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ffa4006c8f0 0x7ffa4006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:05.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.593+0000 7ffa5aa00640 1 -- 172.21.15.152:0/317650018 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ffa5408adc0 con 0x7ffa640af960 2024-04-20T11:27:05.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.594+0000 7f94b6e00640 1 -- 172.21.15.152:0/4117503173 --> [v2:172.21.15.152:6824/1121306353,v1:172.21.15.152:6825/1121306353] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f94a80057d0 con 0x7f9490074bc0 2024-04-20T11:27:05.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.594+0000 7ffa5aa00640 1 --2- 172.21.15.152:0/317650018 >> [v2:172.21.15.175:6820/2659150371,v1:172.21.15.175:6822/2659150371] conn(0x7ffa40074a90 0x7ffa40076eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:05.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.594+0000 7ffa5aa00640 1 -- 172.21.15.152:0/317650018 --> [v2:172.21.15.175:6820/2659150371,v1:172.21.15.175:6822/2659150371] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ffa54013e50 con 0x7ffa40074a90 2024-04-20T11:27:05.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.594+0000 7ffa5aa00640 1 -- 172.21.15.152:0/317650018 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7ffa5408d3e0 con 0x7ffa640af960 2024-04-20T11:27:05.632 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.594+0000 7f94a3e00640 1 -- 172.21.15.152:0/4117503173 <== osd.0 v2:172.21.15.152:6824/1121306353 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f94a80057d0 con 0x7f9490074bc0 2024-04-20T11:27:05.632 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.595+0000 7f94b6e00640 1 -- 172.21.15.152:0/4117503173 >> [v2:172.21.15.152:6824/1121306353,v1:172.21.15.152:6825/1121306353] conn(0x7f9490074bc0 msgr2=0x7f9490076fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:05.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.595+0000 7f94b6e00640 1 --2- 172.21.15.152:0/4117503173 >> [v2:172.21.15.152:6824/1121306353,v1:172.21.15.152:6825/1121306353] conn(0x7f9490074bc0 0x7f9490076fe0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:05.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.595+0000 7ffa5be00640 1 --2- 172.21.15.152:0/317650018 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ffa4006c8f0 0x7ffa4006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:05.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.596+0000 7f94b6e00640 1 -- 172.21.15.152:0/4117503173 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f949006c940 msgr2=0x7f949006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:05.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.596+0000 7f94b6e00640 1 --2- 172.21.15.152:0/4117503173 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f949006c940 0x7f949006ee00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f94a40013b0 tx=0x7f94a4001300 comp rx=0 tx=0).stop 2024-04-20T11:27:05.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.596+0000 7f94b6e00640 1 -- 172.21.15.152:0/4117503173 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f94a80ebb40 msgr2=0x7f94a8052160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:05.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.596+0000 7f94b6e00640 1 --2- 172.21.15.152:0/4117503173 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f94a80ebb40 0x7f94a8052160 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f94b0076cd0 tx=0x7f94b0063760 comp rx=0 tx=0).stop 2024-04-20T11:27:05.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.596+0000 7f94b4e00640 1 -- 172.21.15.152:0/4117503173 reap_dead start 2024-04-20T11:27:05.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.596+0000 7ffa63400640 1 --2- 172.21.15.152:0/317650018 >> [v2:172.21.15.175:6820/2659150371,v1:172.21.15.175:6822/2659150371] conn(0x7ffa40074a90 0x7ffa40076eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:05.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.596+0000 7f94b6e00640 1 -- 172.21.15.152:0/4117503173 shutdown_connections 2024-04-20T11:27:05.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.596+0000 7f94b6e00640 1 -- 172.21.15.152:0/4117503173 >> 172.21.15.152:0/4117503173 conn(0x7f94a80559d0 msgr2=0x7f94a80e8020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:05.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.596+0000 7f94b6e00640 1 -- 172.21.15.152:0/4117503173 shutdown_connections 2024-04-20T11:27:05.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.596+0000 7f94b6e00640 1 -- 172.21.15.152:0/4117503173 wait complete. 2024-04-20T11:27:05.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.597+0000 7f5288c00640 1 -- 172.21.15.152:0/2387096322 <== osd.5 v2:172.21.15.175:6801/3465937300 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5280010e50 con 0x7f526c074b10 2024-04-20T11:27:05.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.597+0000 7ffa5be00640 1 --2- 172.21.15.152:0/317650018 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ffa4006c8f0 0x7ffa4006edb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c036000 tx=0x7ffa4c03b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:05.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.597+0000 7fa506a00640 1 -- 172.21.15.152:0/3263806857 <== mon.2 v2:172.21.15.175:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa4f80129f0 con 0x7fa5100cdbf0 2024-04-20T11:27:05.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.598+0000 7fb216a00640 1 -- 172.21.15.152:0/2064569994 --> [v2:172.21.15.152:6808/1552206819,v1:172.21.15.152:6809/1552206819] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb208057e50 con 0x7fb1f0074cf0 2024-04-20T11:27:05.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.598+0000 7ffa63400640 1 --2- 172.21.15.152:0/317650018 >> [v2:172.21.15.175:6820/2659150371,v1:172.21.15.175:6822/2659150371] conn(0x7ffa40074a90 0x7ffa40076eb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:05.638 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.598+0000 7fa506a00640 1 --2- 172.21.15.152:0/3263806857 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fa4ec06c9c0 0x7fa4ec06ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:05.638 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.598+0000 7fa506a00640 1 -- 172.21.15.152:0/3263806857 <== mon.2 v2:172.21.15.175:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa4f808c090 con 0x7fa5100cdbf0 2024-04-20T11:27:05.638 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.599+0000 7fa506a00640 1 --2- 172.21.15.152:0/3263806857 >> [v2:172.21.15.175:6816/3116669518,v1:172.21.15.175:6817/3116669518] conn(0x7fa4ec074c40 0x7fa4ec077060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:05.638 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.599+0000 7fa506a00640 1 -- 172.21.15.152:0/3263806857 --> [v2:172.21.15.175:6816/3116669518,v1:172.21.15.175:6817/3116669518] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa4ec077710 con 0x7fa4ec074c40 2024-04-20T11:27:05.638 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.599+0000 7fa506a00640 1 -- 172.21.15.152:0/3263806857 <== mon.2 v2:172.21.15.175:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fa4f808c480 con 0x7fa5100cdbf0 2024-04-20T11:27:05.638 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.599+0000 7fa50de00640 1 --2- 172.21.15.152:0/3263806857 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fa4ec06c9c0 0x7fa4ec06ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:05.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.599+0000 7fa50de00640 1 --2- 172.21.15.152:0/3263806857 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fa4ec06c9c0 0x7fa4ec06ee80 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa500001fa0 tx=0x7fa500001930 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:05.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.600+0000 7fb20d600640 1 -- 172.21.15.152:0/2064569994 <== osd.1 v2:172.21.15.152:6808/1552206819 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fb208057e50 con 0x7fb1f0074cf0 2024-04-20T11:27:05.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.600+0000 7fa50e800640 1 --2- 172.21.15.152:0/3263806857 >> [v2:172.21.15.175:6816/3116669518,v1:172.21.15.175:6817/3116669518] conn(0x7fa4ec074c40 0x7fa4ec077060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:05.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.600+0000 7ffa5aa00640 1 -- 172.21.15.152:0/317650018 <== osd.6 v2:172.21.15.175:6820/2659150371 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ffa54013e50 con 0x7ffa40074a90 2024-04-20T11:27:05.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.601+0000 7fa50e800640 1 --2- 172.21.15.152:0/3263806857 >> [v2:172.21.15.175:6816/3116669518,v1:172.21.15.175:6817/3116669518] conn(0x7fa4ec074c40 0x7fa4ec077060 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:05.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.602+0000 7fb216a00640 1 -- 172.21.15.152:0/2064569994 >> [v2:172.21.15.152:6808/1552206819,v1:172.21.15.152:6809/1552206819] conn(0x7fb1f0074cf0 msgr2=0x7fb1f0077110 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:05.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.602+0000 7fb216a00640 1 --2- 172.21.15.152:0/2064569994 >> [v2:172.21.15.152:6808/1552206819,v1:172.21.15.152:6809/1552206819] conn(0x7fb1f0074cf0 0x7fb1f0077110 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:05.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.604+0000 7fa506a00640 1 -- 172.21.15.152:0/3263806857 <== osd.7 v2:172.21.15.175:6816/3116669518 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa4ec077710 con 0x7fa4ec074c40 2024-04-20T11:27:05.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.604+0000 7fb216a00640 1 -- 172.21.15.152:0/2064569994 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fb1f006caf0 msgr2=0x7fb1f006efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:05.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.604+0000 7fb216a00640 1 --2- 172.21.15.152:0/2064569994 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fb1f006caf0 0x7fb1f006efb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc005880 tx=0x7fb1fc009040 comp rx=0 tx=0).stop 2024-04-20T11:27:05.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.604+0000 7fb216a00640 1 -- 172.21.15.152:0/2064569994 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb208054130 msgr2=0x7fb2080e6a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:05.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.604+0000 7fb216a00640 1 --2- 172.21.15.152:0/2064569994 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb208054130 0x7fb2080e6a20 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb210076cd0 tx=0x7fb210069770 comp rx=0 tx=0).stop 2024-04-20T11:27:05.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.604+0000 7fb216000640 1 -- 172.21.15.152:0/2064569994 reap_dead start 2024-04-20T11:27:05.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.604+0000 7fb216a00640 1 -- 172.21.15.152:0/2064569994 shutdown_connections 2024-04-20T11:27:05.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.604+0000 7fb216a00640 1 -- 172.21.15.152:0/2064569994 >> 172.21.15.152:0/2064569994 conn(0x7fb2080559e0 msgr2=0x7fb20803ddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:05.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.609+0000 7fb216a00640 1 -- 172.21.15.152:0/2064569994 shutdown_connections 2024-04-20T11:27:05.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.609+0000 7fb216a00640 1 -- 172.21.15.152:0/2064569994 wait complete. 2024-04-20T11:27:05.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.612+0000 7fe75ea00640 1 -- 172.21.15.152:0/2589952259 --> [v2:172.21.15.152:6800/2519167656,v1:172.21.15.152:6801/2519167656] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe76c075e60 con 0x7fe74c074b70 2024-04-20T11:27:05.643 INFO:teuthology.orchestra.run.smithi152.stdout:55834574854 2024-04-20T11:27:05.643 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-20T11:27:05.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.614+0000 7f638be00640 1 -- 172.21.15.152:0/994081729 --> [v2:172.21.15.152:6816/3995926281,v1:172.21.15.152:6817/3995926281] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6384038b30 con 0x7f636c074d10 2024-04-20T11:27:05.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.614+0000 7fe769600640 1 -- 172.21.15.152:0/2589952259 <== osd.2 v2:172.21.15.152:6800/2519167656 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fe76c075e60 con 0x7fe74c074b70 2024-04-20T11:27:05.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.615+0000 7fe75ea00640 1 -- 172.21.15.152:0/2589952259 >> [v2:172.21.15.152:6800/2519167656,v1:172.21.15.152:6801/2519167656] conn(0x7fe74c074b70 msgr2=0x7fe74c076f90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:05.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.615+0000 7fe75ea00640 1 --2- 172.21.15.152:0/2589952259 >> [v2:172.21.15.152:6800/2519167656,v1:172.21.15.152:6801/2519167656] conn(0x7fe74c074b70 0x7fe74c076f90 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:05.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.615+0000 7f6383400640 1 -- 172.21.15.152:0/994081729 <== osd.3 v2:172.21.15.152:6816/3995926281 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f6384038b30 con 0x7f636c074d10 2024-04-20T11:27:05.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.615+0000 7fe75ea00640 1 -- 172.21.15.152:0/2589952259 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe74c06c8f0 msgr2=0x7fe74c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:05.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.615+0000 7fe75ea00640 1 --2- 172.21.15.152:0/2589952259 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe74c06c8f0 0x7fe74c06edb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe7580099d0 tx=0x7fe758009190 comp rx=0 tx=0).stop 2024-04-20T11:27:05.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.615+0000 7fe75ea00640 1 -- 172.21.15.152:0/2589952259 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe76c0d0f60 msgr2=0x7fe76c159450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:05.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.615+0000 7fe75ea00640 1 --2- 172.21.15.152:0/2589952259 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe76c0d0f60 0x7fe76c159450 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe76400c730 tx=0x7fe76400cbf0 comp rx=0 tx=0).stop 2024-04-20T11:27:05.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.616+0000 7fe772000640 1 -- 172.21.15.152:0/2589952259 reap_dead start 2024-04-20T11:27:05.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.616+0000 7fe75ea00640 1 -- 172.21.15.152:0/2589952259 shutdown_connections 2024-04-20T11:27:05.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.616+0000 7fe75ea00640 1 -- 172.21.15.152:0/2589952259 >> 172.21.15.152:0/2589952259 conn(0x7fe76c0b55e0 msgr2=0x7fe76c146670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:05.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.616+0000 7f638be00640 1 -- 172.21.15.152:0/994081729 >> [v2:172.21.15.152:6816/3995926281,v1:172.21.15.152:6817/3995926281] conn(0x7f636c074d10 msgr2=0x7f636c077130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:05.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.616+0000 7f638be00640 1 --2- 172.21.15.152:0/994081729 >> [v2:172.21.15.152:6816/3995926281,v1:172.21.15.152:6817/3995926281] conn(0x7f636c074d10 0x7f636c077130 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:05.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.616+0000 7fe75ea00640 1 -- 172.21.15.152:0/2589952259 shutdown_connections 2024-04-20T11:27:05.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.616+0000 7fe75ea00640 1 -- 172.21.15.152:0/2589952259 wait complete. 2024-04-20T11:27:05.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.616+0000 7f638be00640 1 -- 172.21.15.152:0/994081729 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f636c06caf0 msgr2=0x7f636c06efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:05.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.616+0000 7f638be00640 1 --2- 172.21.15.152:0/994081729 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f636c06caf0 0x7f636c06efb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6384011390 tx=0x7f638c0b0000 comp rx=0 tx=0).stop 2024-04-20T11:27:05.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.616+0000 7f638be00640 1 -- 172.21.15.152:0/994081729 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6384049b70 msgr2=0x7f63840e3760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:05.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.616+0000 7f638be00640 1 --2- 172.21.15.152:0/994081729 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6384049b70 0x7f63840e3760 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f637c00e860 tx=0x7f637c00ed20 comp rx=0 tx=0).stop 2024-04-20T11:27:05.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.617+0000 7f638b400640 1 -- 172.21.15.152:0/994081729 reap_dead start 2024-04-20T11:27:05.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.617+0000 7f638be00640 1 -- 172.21.15.152:0/994081729 shutdown_connections 2024-04-20T11:27:05.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.617+0000 7f638be00640 1 -- 172.21.15.152:0/994081729 >> 172.21.15.152:0/994081729 conn(0x7f6384020070 msgr2=0x7f63840e04c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:05.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.617+0000 7f638be00640 1 -- 172.21.15.152:0/994081729 shutdown_connections 2024-04-20T11:27:05.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.617+0000 7f638be00640 1 -- 172.21.15.152:0/994081729 wait complete. 2024-04-20T11:27:05.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.622+0000 7fe6a7800640 1 -- 172.21.15.152:0/4039569064 --> [v2:172.21.15.175:6800/1235215467,v1:172.21.15.175:6802/1235215467] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe69804f000 con 0x7fe680075840 2024-04-20T11:27:05.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.624+0000 7fe69cc00640 1 -- 172.21.15.152:0/4039569064 <== osd.4 v2:172.21.15.175:6800/1235215467 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fe69804f000 con 0x7fe680075840 2024-04-20T11:27:05.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.624+0000 7fe692000640 1 -- 172.21.15.152:0/4039569064 >> [v2:172.21.15.175:6800/1235215467,v1:172.21.15.175:6802/1235215467] conn(0x7fe680075840 msgr2=0x7fe680077ce0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:05.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.624+0000 7fe692000640 1 --2- 172.21.15.152:0/4039569064 >> [v2:172.21.15.175:6800/1235215467,v1:172.21.15.175:6802/1235215467] conn(0x7fe680075840 0x7fe680077ce0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:05.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.625+0000 7fe692000640 1 -- 172.21.15.152:0/4039569064 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe68006c940 msgr2=0x7fe68006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:05.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.625+0000 7fe692000640 1 --2- 172.21.15.152:0/4039569064 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe68006c940 0x7fe68006ee00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe69804e650 tx=0x7fe68c008040 comp rx=0 tx=0).stop 2024-04-20T11:27:05.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.625+0000 7fe692000640 1 -- 172.21.15.152:0/4039569064 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe69803c090 msgr2=0x7fe69817c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:05.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.625+0000 7fe692000640 1 --2- 172.21.15.152:0/4039569064 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe69803c090 0x7fe69817c3c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe69400c410 tx=0x7fe69400c8d0 comp rx=0 tx=0).stop 2024-04-20T11:27:05.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.625+0000 7fe6a5800640 1 -- 172.21.15.152:0/4039569064 reap_dead start 2024-04-20T11:27:05.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.625+0000 7fe692000640 1 -- 172.21.15.152:0/4039569064 shutdown_connections 2024-04-20T11:27:05.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.625+0000 7fe692000640 1 -- 172.21.15.152:0/4039569064 >> 172.21.15.152:0/4039569064 conn(0x7fe698058ed0 msgr2=0x7fe698006f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:05.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.627+0000 7fe692000640 1 -- 172.21.15.152:0/4039569064 shutdown_connections 2024-04-20T11:27:05.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.627+0000 7fe692000640 1 -- 172.21.15.152:0/4039569064 wait complete. 2024-04-20T11:27:05.652 INFO:teuthology.orchestra.run.smithi152.stdout:55834574853 2024-04-20T11:27:05.652 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-20T11:27:05.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.633+0000 7f5293800640 1 -- 172.21.15.152:0/2387096322 --> [v2:172.21.15.175:6801/3465937300,v1:172.21.15.175:6804/3465937300] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f528c0b7b40 con 0x7f526c074b10 2024-04-20T11:27:05.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.633+0000 7ffa69a00640 1 -- 172.21.15.152:0/317650018 --> [v2:172.21.15.175:6820/2659150371,v1:172.21.15.175:6822/2659150371] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ffa640d2160 con 0x7ffa40074a90 2024-04-20T11:27:05.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.634+0000 7ffa5aa00640 1 -- 172.21.15.152:0/317650018 <== osd.6 v2:172.21.15.175:6820/2659150371 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ffa640d2160 con 0x7ffa40074a90 2024-04-20T11:27:05.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.634+0000 7f5288c00640 1 -- 172.21.15.152:0/2387096322 <== osd.5 v2:172.21.15.175:6801/3465937300 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f528c0b7b40 con 0x7f526c074b10 2024-04-20T11:27:05.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.634+0000 7fa514a00640 1 -- 172.21.15.152:0/3263806857 --> [v2:172.21.15.175:6816/3116669518,v1:172.21.15.175:6817/3116669518] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa5100689f0 con 0x7fa4ec074c40 2024-04-20T11:27:05.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.634+0000 7ffa69a00640 1 -- 172.21.15.152:0/317650018 >> [v2:172.21.15.175:6820/2659150371,v1:172.21.15.175:6822/2659150371] conn(0x7ffa40074a90 msgr2=0x7ffa40076eb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:05.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.634+0000 7ffa69a00640 1 --2- 172.21.15.152:0/317650018 >> [v2:172.21.15.175:6820/2659150371,v1:172.21.15.175:6822/2659150371] conn(0x7ffa40074a90 0x7ffa40076eb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:05.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.635+0000 7ffa69a00640 1 -- 172.21.15.152:0/317650018 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ffa4006c8f0 msgr2=0x7ffa4006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:05.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.635+0000 7ffa69a00640 1 --2- 172.21.15.152:0/317650018 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ffa4006c8f0 0x7ffa4006edb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c036000 tx=0x7ffa4c03b040 comp rx=0 tx=0).stop 2024-04-20T11:27:05.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.635+0000 7f5293800640 1 -- 172.21.15.152:0/2387096322 >> [v2:172.21.15.175:6801/3465937300,v1:172.21.15.175:6804/3465937300] conn(0x7f526c074b10 msgr2=0x7f526c076f30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:05.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.635+0000 7f5293800640 1 --2- 172.21.15.152:0/2387096322 >> [v2:172.21.15.175:6801/3465937300,v1:172.21.15.175:6804/3465937300] conn(0x7f526c074b10 0x7f526c076f30 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:05.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.635+0000 7ffa69a00640 1 -- 172.21.15.152:0/317650018 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffa640af960 msgr2=0x7ffa6414e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:05.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.635+0000 7ffa69a00640 1 --2- 172.21.15.152:0/317650018 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffa640af960 0x7ffa6414e5e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ffa5400c390 tx=0x7ffa5400c850 comp rx=0 tx=0).stop 2024-04-20T11:27:05.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.635+0000 7ffa63400640 1 -- 172.21.15.152:0/317650018 reap_dead start 2024-04-20T11:27:05.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.635+0000 7ffa69a00640 1 -- 172.21.15.152:0/317650018 shutdown_connections 2024-04-20T11:27:05.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.635+0000 7ffa69a00640 1 -- 172.21.15.152:0/317650018 >> 172.21.15.152:0/317650018 conn(0x7ffa6409aa40 msgr2=0x7ffa64159e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:05.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.635+0000 7f5293800640 1 -- 172.21.15.152:0/2387096322 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f526c06c8f0 msgr2=0x7f526c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:05.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.635+0000 7f5293800640 1 --2- 172.21.15.152:0/2387096322 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f526c06c8f0 0x7f526c06edb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f528c0b64a0 tx=0x7f5278008040 comp rx=0 tx=0).stop 2024-04-20T11:27:05.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.635+0000 7f5293800640 1 -- 172.21.15.152:0/2387096322 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f528c0d0dc0 msgr2=0x7f528c1631d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:05.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.635+0000 7f5293800640 1 --2- 172.21.15.152:0/2387096322 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f528c0d0dc0 0x7f528c1631d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f528000c390 tx=0x7f528000c850 comp rx=0 tx=0).stop 2024-04-20T11:27:05.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.635+0000 7ffa69a00640 1 -- 172.21.15.152:0/317650018 shutdown_connections 2024-04-20T11:27:05.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.635+0000 7fa506a00640 1 -- 172.21.15.152:0/3263806857 <== osd.7 v2:172.21.15.175:6816/3116669518 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fa5100689f0 con 0x7fa4ec074c40 2024-04-20T11:27:05.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.635+0000 7f5291800640 1 -- 172.21.15.152:0/2387096322 reap_dead start 2024-04-20T11:27:05.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.635+0000 7f5293800640 1 -- 172.21.15.152:0/2387096322 shutdown_connections 2024-04-20T11:27:05.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.635+0000 7f5293800640 1 -- 172.21.15.152:0/2387096322 >> 172.21.15.152:0/2387096322 conn(0x7f528c067ae0 msgr2=0x7f528c0d4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:05.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.636+0000 7ffa69a00640 1 -- 172.21.15.152:0/317650018 wait complete. 2024-04-20T11:27:05.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.636+0000 7f5293800640 1 -- 172.21.15.152:0/2387096322 shutdown_connections 2024-04-20T11:27:05.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.636+0000 7f5293800640 1 -- 172.21.15.152:0/2387096322 wait complete. 2024-04-20T11:27:05.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.636+0000 7fa514a00640 1 -- 172.21.15.152:0/3263806857 >> [v2:172.21.15.175:6816/3116669518,v1:172.21.15.175:6817/3116669518] conn(0x7fa4ec074c40 msgr2=0x7fa4ec077060 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:05.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.636+0000 7fa514a00640 1 --2- 172.21.15.152:0/3263806857 >> [v2:172.21.15.175:6816/3116669518,v1:172.21.15.175:6817/3116669518] conn(0x7fa4ec074c40 0x7fa4ec077060 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:05.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.637+0000 7fa514a00640 1 -- 172.21.15.152:0/3263806857 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fa4ec06c9c0 msgr2=0x7fa4ec06ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:05.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.637+0000 7fa514a00640 1 --2- 172.21.15.152:0/3263806857 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fa4ec06c9c0 0x7fa4ec06ee80 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa500001fa0 tx=0x7fa500001930 comp rx=0 tx=0).stop 2024-04-20T11:27:05.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.637+0000 7fa514a00640 1 -- 172.21.15.152:0/3263806857 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa5100cdbf0 msgr2=0x7fa5101501e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:05.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.637+0000 7fa514a00640 1 --2- 172.21.15.152:0/3263806857 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa5100cdbf0 0x7fa5101501e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa4f800c460 tx=0x7fa4f800c920 comp rx=0 tx=0).stop 2024-04-20T11:27:05.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.637+0000 7fa50e800640 1 -- 172.21.15.152:0/3263806857 reap_dead start 2024-04-20T11:27:05.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.637+0000 7fa514a00640 1 -- 172.21.15.152:0/3263806857 shutdown_connections 2024-04-20T11:27:05.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.637+0000 7fa514a00640 1 -- 172.21.15.152:0/3263806857 >> 172.21.15.152:0/3263806857 conn(0x7fa510067cf0 msgr2=0x7fa5100cf690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:05.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.638+0000 7fa514a00640 1 -- 172.21.15.152:0/3263806857 shutdown_connections 2024-04-20T11:27:05.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.638+0000 7fa514a00640 1 -- 172.21.15.152:0/3263806857 wait complete. 2024-04-20T11:27:05.661 INFO:teuthology.orchestra.run.smithi152.stdout:55834574855 2024-04-20T11:27:05.661 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-20T11:27:05.662 INFO:teuthology.orchestra.run.smithi152.stdout:55834574854 2024-04-20T11:27:05.663 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-20T11:27:05.664 INFO:teuthology.orchestra.run.smithi152.stdout:55834574853 2024-04-20T11:27:05.664 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-20T11:27:05.665 INFO:teuthology.orchestra.run.smithi152.stdout:55834574856 2024-04-20T11:27:05.665 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-20T11:27:05.666 INFO:teuthology.orchestra.run.smithi152.stdout:55834574853 2024-04-20T11:27:05.667 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-20T11:27:05.667 INFO:teuthology.orchestra.run.smithi152.stdout:55834574855 2024-04-20T11:27:05.668 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-20T11:27:05.978 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.978+0000 7fc143e00640 1 Processor -- start 2024-04-20T11:27:05.978 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.979+0000 7fc143e00640 1 -- start start 2024-04-20T11:27:05.980 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.981+0000 7fc143e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc1340e7640 0x7fc134055400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:05.980 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.981+0000 7fc143e00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc134057f70 con 0x7fc134053c00 2024-04-20T11:27:05.980 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.981+0000 7fc143e00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc134056ed0 con 0x7fc1340e7a20 2024-04-20T11:27:05.980 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.981+0000 7fc143e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc1340e7e80 con 0x7fc1340e7640 2024-04-20T11:27:05.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.982+0000 7f1cbe800640 1 Processor -- start 2024-04-20T11:27:05.982 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.982+0000 7fc141e00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fc134053c00 0x7fc134006230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:48440/0 (socket says 172.21.15.152:48440) 2024-04-20T11:27:05.982 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.982+0000 7fc141e00640 1 -- 172.21.15.152:0/1743206834 learned_addr learned my addr 172.21.15.152:0/1743206834 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:27:05.984 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.985+0000 7fc141400640 1 --2- 172.21.15.152:0/1743206834 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc1340e7640 0x7fc134055400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:05.985 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.983+0000 7f1cbe800640 1 -- start start 2024-04-20T11:27:05.985 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.985+0000 7f1cbe800640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f1cb00e7a40 0x7f1cb0049b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:05.985 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.985+0000 7f1cbe800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f1cb004a090 0x7f1cb0054890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:05.985 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.985+0000 7f1cbe800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1cb0054dd0 0x7f1cb003c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:05.985 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.985+0000 7f1cbe800640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f1cb00e7e20 con 0x7f1cb004a090 2024-04-20T11:27:05.985 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.985+0000 7f1cbe800640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f1cb003b590 con 0x7f1cb00e7a40 2024-04-20T11:27:05.985 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.985+0000 7f1cbe800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1cb003b870 con 0x7f1cb0054dd0 2024-04-20T11:27:05.986 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.985+0000 7fc141400640 1 -- 172.21.15.152:0/1743206834 >> v1:172.21.15.175:6789/0 conn(0x7fc1340e7a20 legacy=0x7fc1340534f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:05.986 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.985+0000 7fc141400640 1 -- 172.21.15.152:0/1743206834 >> v1:172.21.15.152:6789/0 conn(0x7fc134053c00 legacy=0x7fc134006230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:05.986 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.985+0000 7fc141400640 1 -- 172.21.15.152:0/1743206834 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1340571b0 con 0x7fc1340e7640 2024-04-20T11:27:05.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.986+0000 7f1cb6a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f1cb004a090 0x7f1cb0054890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:05.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.986+0000 7f1cb6a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f1cb004a090 0x7f1cb0054890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:47954/0 (socket says 172.21.15.152:47954) 2024-04-20T11:27:05.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.986+0000 7f1cb6a00640 1 -- 172.21.15.152:0/768531408 learned_addr learned my addr 172.21.15.152:0/768531408 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:05.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.987+0000 7f1cb6a00640 1 -- 172.21.15.152:0/768531408 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1cb0054dd0 msgr2=0x7f1cb003c350 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-20T11:27:05.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.987+0000 7f1cb7e00640 1 --2- 172.21.15.152:0/768531408 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1cb0054dd0 0x7f1cb003c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:05.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.987+0000 7f1cb6a00640 1 --2- 172.21.15.152:0/768531408 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1cb0054dd0 0x7f1cb003c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:05.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.987+0000 7f1cb6a00640 1 -- 172.21.15.152:0/768531408 >> v2:172.21.15.175:3300/0 conn(0x7f1cb00e7a40 msgr2=0x7f1cb0049b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:05.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.987+0000 7f1cb7400640 1 --2- 172.21.15.152:0/768531408 >> v2:172.21.15.175:3300/0 conn(0x7f1cb00e7a40 0x7f1cb0049b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:05.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.987+0000 7fc141400640 1 --2- 172.21.15.152:0/1743206834 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc1340e7640 0x7fc134055400 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc1300097d0 tx=0x7fc13002b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c756c3620512fd4a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:05.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.988+0000 7f1cb6a00640 1 --2- 172.21.15.152:0/768531408 >> v2:172.21.15.175:3300/0 conn(0x7f1cb00e7a40 0x7f1cb0049b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:05.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.988+0000 7f1cb6a00640 1 -- 172.21.15.152:0/768531408 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cb003bb50 con 0x7f1cb004a090 2024-04-20T11:27:05.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.988+0000 7f1cb7400640 1 --2- 172.21.15.152:0/768531408 >> v2:172.21.15.175:3300/0 conn(0x7f1cb00e7a40 0x7f1cb0049b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:27:05.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.988+0000 7fc13be00640 1 -- 172.21.15.152:0/1743206834 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc130034070 con 0x7fc1340e7640 2024-04-20T11:27:05.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.988+0000 7fc13be00640 1 -- 172.21.15.152:0/1743206834 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc130030610 con 0x7fc1340e7640 2024-04-20T11:27:05.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.988+0000 7fc13be00640 1 -- 172.21.15.152:0/1743206834 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1300309e0 con 0x7fc1340e7640 2024-04-20T11:27:05.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.990+0000 7fc143e00640 1 -- 172.21.15.152:0/1743206834 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc1340e7640 msgr2=0x7fc134055400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:05.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.990+0000 7fc143e00640 1 --2- 172.21.15.152:0/1743206834 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc1340e7640 0x7fc134055400 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc1300097d0 tx=0x7fc13002b960 comp rx=0 tx=0).stop 2024-04-20T11:27:05.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.990+0000 7f1cb6a00640 1 --2- 172.21.15.152:0/768531408 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1cb004a090 0x7f1cb0054890 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1cb8076cd0 tx=0x7f1cb80a7420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=968c59d8568e280d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:05.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.991+0000 7f1cb6000640 1 -- 172.21.15.152:0/768531408 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb80b3070 con 0x7f1cb004a090 2024-04-20T11:27:05.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.991+0000 7f1cb6000640 1 -- 172.21.15.152:0/768531408 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cb80ac810 con 0x7f1cb004a090 2024-04-20T11:27:05.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.991+0000 7f1cb6000640 1 -- 172.21.15.152:0/768531408 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb80acc80 con 0x7f1cb004a090 2024-04-20T11:27:05.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.991+0000 7fc143e00640 1 -- 172.21.15.152:0/1743206834 shutdown_connections 2024-04-20T11:27:05.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.991+0000 7fc143e00640 1 --2- 172.21.15.152:0/1743206834 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc1340e7640 0x7fc134055400 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:05.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.991+0000 7fc143e00640 1 -- 172.21.15.152:0/1743206834 >> 172.21.15.152:0/1743206834 conn(0x7fc13403aa20 msgr2=0x7fc13403ae30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:05.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.992+0000 7fc143e00640 1 -- 172.21.15.152:0/1743206834 shutdown_connections 2024-04-20T11:27:05.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.992+0000 7fc143e00640 1 -- 172.21.15.152:0/1743206834 wait complete. 2024-04-20T11:27:05.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.992+0000 7f1cbe800640 1 -- 172.21.15.152:0/768531408 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1cb004a090 msgr2=0x7f1cb0054890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:05.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.992+0000 7f1cbe800640 1 --2- 172.21.15.152:0/768531408 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1cb004a090 0x7f1cb0054890 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1cb8076cd0 tx=0x7f1cb80a7420 comp rx=0 tx=0).stop 2024-04-20T11:27:05.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.992+0000 7ff6a1800640 1 Processor -- start 2024-04-20T11:27:05.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.992+0000 7ff6a1800640 1 -- start start 2024-04-20T11:27:05.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.993+0000 7f1cbe800640 1 -- 172.21.15.152:0/768531408 shutdown_connections 2024-04-20T11:27:05.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.993+0000 7f1cbe800640 1 --2- 172.21.15.152:0/768531408 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1cb0054dd0 0x7f1cb003c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:05.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.993+0000 7f1cbe800640 1 --2- 172.21.15.152:0/768531408 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1cb004a090 0x7f1cb0054890 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:05.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.993+0000 7f1cbe800640 1 --2- 172.21.15.152:0/768531408 >> v2:172.21.15.175:3300/0 conn(0x7f1cb00e7a40 0x7f1cb0049b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:05.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.993+0000 7f1cbe800640 1 -- 172.21.15.152:0/768531408 >> 172.21.15.152:0/768531408 conn(0x7f1cb0056cc0 msgr2=0x7f1cb00570d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:05.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.994+0000 7f1cbe800640 1 -- 172.21.15.152:0/768531408 shutdown_connections 2024-04-20T11:27:05.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.994+0000 7f1cbe800640 1 -- 172.21.15.152:0/768531408 wait complete. 2024-04-20T11:27:05.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.994+0000 7fc143e00640 1 Processor -- start 2024-04-20T11:27:05.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.995+0000 7fc143e00640 1 -- start start 2024-04-20T11:27:05.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.996+0000 7f1cbe800640 1 Processor -- start 2024-04-20T11:27:05.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.995+0000 7ff6a1800640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ff69c0d4ec0 0x7ff69c0d0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:05.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.996+0000 7ff6a1800640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff69c0cf3e0 con 0x7ff69c166270 2024-04-20T11:27:05.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.996+0000 7ff6a1800640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff69c0cf6c0 con 0x7ff69c166650 2024-04-20T11:27:05.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.996+0000 7ff6a1800640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7ff69c0cf1b0 con 0x7ff69c0d4ec0 2024-04-20T11:27:05.997 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.996+0000 7f1cbe800640 1 -- start start 2024-04-20T11:27:05.997 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.996+0000 7ff6a0e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ff69c0d4ec0 0x7ff69c0d0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:05.997 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.996+0000 7ff6a0e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ff69c0d4ec0 0x7ff69c0d0f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.152:43300/0 (socket says 172.21.15.152:43300) 2024-04-20T11:27:05.997 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.996+0000 7ff6a0e00640 1 -- 172.21.15.152:0/4010308426 learned_addr learned my addr 172.21.15.152:0/4010308426 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:05.997 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.996+0000 7fc143e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc134053c00 0x7fc1340e4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:05.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.996+0000 7fc143e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc1340e7640 0x7fc13404f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:05.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.996+0000 7fc143e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc1340e7a20 0x7fc13404c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:05.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.998+0000 7fc143e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fc134058ce0 con 0x7fc1340e7640 2024-04-20T11:27:05.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.998+0000 7fc143e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc134056c80 con 0x7fc134053c00 2024-04-20T11:27:05.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.998+0000 7fc143e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc134057770 con 0x7fc1340e7a20 2024-04-20T11:27:05.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.998+0000 7fc140a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc1340e7640 0x7fc13404f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:05.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.998+0000 7fc141400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc134053c00 0x7fc1340e4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:05.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.998+0000 7f1cbe800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1cb004a090 0x7f1cb00e1420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:05.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.998+0000 7f1cbe800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1cb0054dd0 0x7f1cb0047c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:05.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.998+0000 7f1cbe800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1cb00e7a40 0x7f1cb004b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:05.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.998+0000 7f1cbe800640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb0039ca0 con 0x7f1cb00e7a40 2024-04-20T11:27:05.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.998+0000 7f1cbe800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb004a5a0 con 0x7f1cb0054dd0 2024-04-20T11:27:05.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.998+0000 7f1cbe800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1cb0039e20 con 0x7f1cb004a090 2024-04-20T11:27:05.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.998+0000 7f1cb7e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1cb00e7a40 0x7f1cb004b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:05.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.998+0000 7f1cb7e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1cb00e7a40 0x7f1cb004b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:47966/0 (socket says 172.21.15.152:47966) 2024-04-20T11:27:05.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.998+0000 7f1cb7e00640 1 -- 172.21.15.152:0/1867463124 learned_addr learned my addr 172.21.15.152:0/1867463124 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:05.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.998+0000 7f1cb7400640 1 --2- 172.21.15.152:0/1867463124 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1cb004a090 0x7f1cb00e1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:05.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.999+0000 7f1cb6a00640 1 --2- 172.21.15.152:0/1867463124 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1cb0054dd0 0x7f1cb0047c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:06.000 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.997+0000 7ff6a0e00640 1 -- 172.21.15.152:0/4010308426 >> v1:172.21.15.175:6789/0 conn(0x7ff69c166650 legacy=0x7ff69c0d01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:06.000 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.999+0000 7fc140a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc1340e7640 0x7fc13404f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:47964/0 (socket says 172.21.15.152:47964) 2024-04-20T11:27:06.000 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.999+0000 7fc140a00640 1 -- 172.21.15.152:0/2411782992 learned_addr learned my addr 172.21.15.152:0/2411782992 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:06.000 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.999+0000 7f1cb7e00640 1 -- 172.21.15.152:0/1867463124 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1cb004a090 msgr2=0x7f1cb00e1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:06.001 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.999+0000 7f1cb7e00640 1 --2- 172.21.15.152:0/1867463124 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1cb004a090 0x7f1cb00e1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:06.001 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.999+0000 7f1cb7e00640 1 -- 172.21.15.152:0/1867463124 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1cb0054dd0 msgr2=0x7f1cb0047c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:06.001 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.999+0000 7f1cb7e00640 1 --2- 172.21.15.152:0/1867463124 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1cb0054dd0 0x7f1cb0047c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:06.001 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.999+0000 7f1cb7e00640 1 -- 172.21.15.152:0/1867463124 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cb0009e00 con 0x7f1cb00e7a40 2024-04-20T11:27:06.001 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.997+0000 7ff6a0e00640 1 -- 172.21.15.152:0/4010308426 >> v1:172.21.15.152:6789/0 conn(0x7ff69c166270 legacy=0x7ff69c0d3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:06.001 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.999+0000 7fc141e00640 1 --2- 172.21.15.152:0/2411782992 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc1340e7a20 0x7fc13404c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:06.002 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:05.997+0000 7ff6a0e00640 1 -- 172.21.15.152:0/4010308426 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff69c0cf9a0 con 0x7ff69c0d4ec0 2024-04-20T11:27:06.002 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.000+0000 7f1cb7e00640 1 --2- 172.21.15.152:0/1867463124 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1cb00e7a40 0x7f1cb004b3c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1cac00c540 tx=0x7f1cac00ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:06.002 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.000+0000 7fc140a00640 1 -- 172.21.15.152:0/2411782992 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc1340e7a20 msgr2=0x7fc13404c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:06.002 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.000+0000 7fc140a00640 1 --2- 172.21.15.152:0/2411782992 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc1340e7a20 0x7fc13404c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:06.002 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.000+0000 7fc140a00640 1 -- 172.21.15.152:0/2411782992 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc134053c00 msgr2=0x7fc1340e4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:06.003 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.000+0000 7fc140a00640 1 --2- 172.21.15.152:0/2411782992 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc134053c00 0x7fc1340e4600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:06.003 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.000+0000 7fc140a00640 1 -- 172.21.15.152:0/2411782992 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1340e4b40 con 0x7fc1340e7640 2024-04-20T11:27:06.003 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.001+0000 7fc140a00640 1 --2- 172.21.15.152:0/2411782992 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc1340e7640 0x7fc13404f4a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc12800c480 tx=0x7fc12800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:06.003 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.001+0000 7fc139600640 1 -- 172.21.15.152:0/2411782992 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc128017070 con 0x7fc1340e7640 2024-04-20T11:27:06.003 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.000+0000 7f1cab400640 1 -- 172.21.15.152:0/1867463124 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cac015070 con 0x7f1cb00e7a40 2024-04-20T11:27:06.003 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.000+0000 7f1cab400640 1 -- 172.21.15.152:0/1867463124 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cac004220 con 0x7f1cb00e7a40 2024-04-20T11:27:06.003 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.000+0000 7f1cab400640 1 -- 172.21.15.152:0/1867463124 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cac003610 con 0x7f1cb00e7a40 2024-04-20T11:27:06.003 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.000+0000 7ff6a0e00640 1 --2- 172.21.15.152:0/4010308426 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff69c0d4ec0 0x7ff69c0d0f00 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff694009e50 tx=0x7ff69402bdc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a361ade4366098b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:06.004 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.002+0000 7f1cbe800640 1 -- 172.21.15.152:0/1867463124 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cb004a8e0 con 0x7f1cb00e7a40 2024-04-20T11:27:06.004 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.002+0000 7fc263800640 1 Processor -- start 2024-04-20T11:27:06.004 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.002+0000 7ff69aa00640 1 -- 172.21.15.152:0/4010308426 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff69402c960 con 0x7ff69c0d4ec0 2024-04-20T11:27:06.005 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.002+0000 7f1cbe800640 1 -- 172.21.15.152:0/1867463124 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cb0009920 con 0x7f1cb00e7a40 2024-04-20T11:27:06.005 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.002+0000 7ff69aa00640 1 -- 172.21.15.152:0/4010308426 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff69402cb00 con 0x7ff69c0d4ec0 2024-04-20T11:27:06.005 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.002+0000 7fc143e00640 1 -- 172.21.15.152:0/2411782992 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1340e3560 con 0x7fc1340e7640 2024-04-20T11:27:06.005 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.002+0000 7fc143e00640 1 -- 172.21.15.152:0/2411782992 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1340e3b20 con 0x7fc1340e7640 2024-04-20T11:27:06.005 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.002+0000 7fc139600640 1 -- 172.21.15.152:0/2411782992 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1280052d0 con 0x7fc1340e7640 2024-04-20T11:27:06.005 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.002+0000 7fc139600640 1 -- 172.21.15.152:0/2411782992 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc128017070 con 0x7fc1340e7640 2024-04-20T11:27:06.006 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.003+0000 7f6e15000640 1 Processor -- start 2024-04-20T11:27:06.006 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.003+0000 7f6e15000640 1 -- start start 2024-04-20T11:27:06.007 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.003+0000 7ff69aa00640 1 -- 172.21.15.152:0/4010308426 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6940343f0 con 0x7ff69c0d4ec0 2024-04-20T11:27:06.007 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.004+0000 7ff6a1800640 1 -- 172.21.15.152:0/4010308426 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff69c0d4ec0 msgr2=0x7ff69c0d0f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:06.007 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.004+0000 7ff6a1800640 1 --2- 172.21.15.152:0/4010308426 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff69c0d4ec0 0x7ff69c0d0f00 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff694009e50 tx=0x7ff69402bdc0 comp rx=0 tx=0).stop 2024-04-20T11:27:06.007 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.004+0000 7ff6a1800640 1 -- 172.21.15.152:0/4010308426 shutdown_connections 2024-04-20T11:27:06.007 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.004+0000 7ff6a1800640 1 --2- 172.21.15.152:0/4010308426 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff69c0d4ec0 0x7ff69c0d0f00 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:06.007 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.004+0000 7ff6a1800640 1 -- 172.21.15.152:0/4010308426 >> 172.21.15.152:0/4010308426 conn(0x7ff69c08bfd0 msgr2=0x7ff69c0d45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:06.008 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.004+0000 7f1cbe800640 1 -- 172.21.15.152:0/1867463124 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cb00057d0 con 0x7f1cb00e7a40 2024-04-20T11:27:06.008 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.005+0000 7f6e15000640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f6e1015ec80 0x7f6e100ba180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:06.008 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.005+0000 7f6e15000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f6e100cfa00 0x7f6e100b80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:06.009 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.005+0000 7f6e15000640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e100b8cf0 con 0x7f6e100cf5a0 2024-04-20T11:27:06.009 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.005+0000 7f6e15000640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f6e100b9d90 con 0x7f6e100cfa00 2024-04-20T11:27:06.009 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.005+0000 7f6e15000640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f6e100b9900 con 0x7f6e1015ec80 2024-04-20T11:27:06.009 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.006+0000 7f6e0de00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f6e1015ec80 0x7f6e100ba180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:06.009 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.006+0000 7f6e0de00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f6e1015ec80 0x7f6e100ba180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.152:43334/0 (socket says 172.21.15.152:43334) 2024-04-20T11:27:06.009 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.006+0000 7f6e0de00640 1 -- 172.21.15.152:0/2232076055 learned_addr learned my addr 172.21.15.152:0/2232076055 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:06.009 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.006+0000 7f6e0de00640 1 -- 172.21.15.152:0/2232076055 >> v2:172.21.15.152:3300/0 conn(0x7f6e100cfa00 msgr2=0x7f6e100b80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:06.009 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.006+0000 7f6e0de00640 1 --2- 172.21.15.152:0/2232076055 >> v2:172.21.15.152:3300/0 conn(0x7f6e100cfa00 0x7f6e100b80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:06.009 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.006+0000 7f6e0de00640 1 -- 172.21.15.152:0/2232076055 >> v1:172.21.15.152:6789/0 conn(0x7f6e100cf5a0 legacy=0x7f6e100b7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:06.009 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.006+0000 7f6e0de00640 1 -- 172.21.15.152:0/2232076055 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e100b8fd0 con 0x7f6e1015ec80 2024-04-20T11:27:06.010 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.004+0000 7fc263800640 1 -- start start 2024-04-20T11:27:06.010 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.006+0000 7fc263800640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fc25c162b40 0x7fc25c0b3470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:06.010 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.006+0000 7fc263800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fc25c0cfff0 0x7fc25c0d0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:06.010 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.006+0000 7fc263800640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc25c0b3a50 con 0x7fc25c06f810 2024-04-20T11:27:06.010 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.006+0000 7fc263800640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fc25c0d09a0 con 0x7fc25c0cfff0 2024-04-20T11:27:06.010 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.006+0000 7fc263800640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fc25c06fc70 con 0x7fc25c162b40 2024-04-20T11:27:06.010 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.007+0000 7ff6a1800640 1 -- 172.21.15.152:0/4010308426 shutdown_connections 2024-04-20T11:27:06.011 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.007+0000 7fc143e00640 1 -- 172.21.15.152:0/2411782992 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc118005290 con 0x7fc1340e7640 2024-04-20T11:27:06.011 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.007+0000 7fc260e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fc25c162b40 0x7fc25c0b3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:06.011 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.007+0000 7fc260e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fc25c162b40 0x7fc25c0b3470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.152:43340/0 (socket says 172.21.15.152:43340) 2024-04-20T11:27:06.012 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.007+0000 7f1527200640 1 Processor -- start 2024-04-20T11:27:06.012 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.007+0000 7fc260e00640 1 -- 172.21.15.152:0/2285621282 learned_addr learned my addr 172.21.15.152:0/2285621282 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:06.012 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.007+0000 7fc261800640 1 --2- 172.21.15.152:0/2285621282 >> v2:172.21.15.152:3300/0 conn(0x7fc25c0cfff0 0x7fc25c0d0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:06.013 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.008+0000 7f1527200640 1 -- start start 2024-04-20T11:27:06.013 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.008+0000 7fc261800640 1 -- 172.21.15.152:0/2285621282 >> v2:172.21.15.175:3300/0 conn(0x7fc25c162b40 msgr2=0x7fc25c0b3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:06.013 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.008+0000 7fc261800640 1 --2- 172.21.15.152:0/2285621282 >> v2:172.21.15.175:3300/0 conn(0x7fc25c162b40 0x7fc25c0b3470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:06.013 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.008+0000 7fc261800640 1 -- 172.21.15.152:0/2285621282 >> v1:172.21.15.152:6789/0 conn(0x7fc25c06f810 legacy=0x7fc25c06ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:06.013 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.008+0000 7f6e0de00640 1 --2- 172.21.15.152:0/2232076055 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e1015ec80 0x7f6e100ba180 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6e100d5890 tx=0x7f6e0802bee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ff66ea577dc676e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:06.014 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.009+0000 7f1527200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f152015ec80 0x7f15200ba180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:06.014 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.009+0000 7f1527200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f15200cf5a0 0x7f15200cfa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:06.014 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.009+0000 7f1527200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15200b8cf0 con 0x7f15200b7990 2024-04-20T11:27:06.014 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.009+0000 7f1527200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f15200b9d90 con 0x7f152015ec80 2024-04-20T11:27:06.015 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.009+0000 7f1527200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f15200cff40 con 0x7f15200cf5a0 2024-04-20T11:27:06.015 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.009+0000 7f6e0ca00640 1 -- 172.21.15.152:0/2232076055 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e0802ca80 con 0x7f6e1015ec80 2024-04-20T11:27:06.015 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.010+0000 7f6e0ca00640 1 -- 172.21.15.152:0/2232076055 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e0802cc20 con 0x7f6e1015ec80 2024-04-20T11:27:06.015 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.010+0000 7f6e0ca00640 1 -- 172.21.15.152:0/2232076055 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e080343f0 con 0x7f6e1015ec80 2024-04-20T11:27:06.016 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.010+0000 7ff6a1800640 1 -- 172.21.15.152:0/4010308426 wait complete. 2024-04-20T11:27:06.016 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.009+0000 7fc261800640 1 -- 172.21.15.152:0/2285621282 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc25c06e300 con 0x7fc25c0cfff0 2024-04-20T11:27:06.016 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.010+0000 7f1525200640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7f15200b7990 0x7f15200b7e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.152:49472/0 (socket says 172.21.15.152:49472) 2024-04-20T11:27:06.017 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.010+0000 7f1525200640 1 -- 172.21.15.152:0/3270428710 learned_addr learned my addr 172.21.15.152:0/3270428710 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:27:06.017 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.010+0000 7f151fe00640 1 --2- 172.21.15.152:0/3270428710 >> v2:172.21.15.152:3300/0 conn(0x7f152015ec80 0x7f15200ba180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:06.017 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.009+0000 7fc139600640 1 -- 172.21.15.152:0/2411782992 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc12801a070 con 0x7fc1340e7640 2024-04-20T11:27:06.017 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.011+0000 7fc139600640 1 --2- 172.21.15.152:0/2411782992 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc11c06cb20 0x7fc11c06efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:06.017 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.011+0000 7fc139600640 1 -- 172.21.15.152:0/2411782992 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc12808a910 con 0x7fc1340e7640 2024-04-20T11:27:06.018 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.011+0000 7f151f400640 1 --2- 172.21.15.152:0/3270428710 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f15200cf5a0 0x7f15200cfa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:06.018 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.011+0000 7f151fe00640 1 -- 172.21.15.152:0/3270428710 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f15200cf5a0 msgr2=0x7f15200cfa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:06.018 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.012+0000 7fc141400640 1 --2- 172.21.15.152:0/2411782992 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc11c06cb20 0x7fc11c06efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:06.019 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.010+0000 7f1cab400640 1 -- 172.21.15.152:0/1867463124 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1cac0037b0 con 0x7f1cb00e7a40 2024-04-20T11:27:06.019 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.012+0000 7f1cab400640 1 --2- 172.21.15.152:0/1867463124 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f1c9806c920 0x7f1c9806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:06.019 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.012+0000 7f1cab400640 1 -- 172.21.15.152:0/1867463124 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f1cac08bb80 con 0x7f1cb00e7a40 2024-04-20T11:27:06.019 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.011+0000 7fc261800640 1 --2- 172.21.15.152:0/2285621282 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc25c0cfff0 0x7fc25c0d0460 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc25c0700a0 tx=0x7fc25402be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58f8863c648924d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:06.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.011+0000 7f6e15000640 1 -- 172.21.15.152:0/2232076055 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e1015ec80 msgr2=0x7f6e100ba180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:06.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.011+0000 7f6e15000640 1 --2- 172.21.15.152:0/2232076055 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e1015ec80 0x7f6e100ba180 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6e100d5890 tx=0x7f6e0802bee0 comp rx=0 tx=0).stop 2024-04-20T11:27:06.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.011+0000 7f6e15000640 1 -- 172.21.15.152:0/2232076055 shutdown_connections 2024-04-20T11:27:06.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.011+0000 7f6e15000640 1 --2- 172.21.15.152:0/2232076055 >> v2:172.21.15.152:3300/0 conn(0x7f6e100cfa00 0x7f6e100b80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:06.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.011+0000 7f6e15000640 1 --2- 172.21.15.152:0/2232076055 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e1015ec80 0x7f6e100ba180 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:06.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.011+0000 7f6e15000640 1 -- 172.21.15.152:0/2232076055 >> 172.21.15.152:0/2232076055 conn(0x7f6e10067ae0 msgr2=0x7f6e10067ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:06.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.011+0000 7f6e15000640 1 -- 172.21.15.152:0/2232076055 shutdown_connections 2024-04-20T11:27:06.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.011+0000 7f6e15000640 1 -- 172.21.15.152:0/2232076055 wait complete. 2024-04-20T11:27:06.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.013+0000 7fc141400640 1 --2- 172.21.15.152:0/2411782992 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc11c06cb20 0x7fc11c06efe0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc130036000 tx=0x7fc13003b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:06.021 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.013+0000 7f1cb7400640 1 --2- 172.21.15.152:0/1867463124 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f1c9806c920 0x7f1c9806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:06.021 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.011+0000 7f151fe00640 1 --2- 172.21.15.152:0/3270428710 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f15200cf5a0 0x7f15200cfa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:06.021 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.011+0000 7f151fe00640 1 -- 172.21.15.152:0/3270428710 >> v1:172.21.15.175:6789/0 conn(0x7f15200b7990 legacy=0x7f15200b7e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:06.021 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.014+0000 7ff6a1800640 1 Processor -- start 2024-04-20T11:27:06.022 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.011+0000 7f151fe00640 1 -- 172.21.15.152:0/3270428710 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15200b8fd0 con 0x7f152015ec80 2024-04-20T11:27:06.022 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.011+0000 7f151f400640 1 --2- 172.21.15.152:0/3270428710 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f15200cf5a0 0x7f15200cfa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:27:06.022 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.014+0000 7f6e15000640 1 Processor -- start 2024-04-20T11:27:06.022 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.014+0000 7ff6a1800640 1 -- start start 2024-04-20T11:27:06.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.014+0000 7f151fe00640 1 --2- 172.21.15.152:0/3270428710 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f152015ec80 0x7f15200ba180 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f150c0097a0 tx=0x7f150c02b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=998ea6a7e2868d9e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:06.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.014+0000 7f6e15000640 1 -- start start 2024-04-20T11:27:06.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.015+0000 7f1cb7400640 1 --2- 172.21.15.152:0/1867463124 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f1c9806c920 0x7f1c9806ede0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1ca400aae0 tx=0x7f1ca4008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:06.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.015+0000 7fa60f400640 1 Processor -- start 2024-04-20T11:27:06.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.015+0000 7f0c42c00640 1 Processor -- start 2024-04-20T11:27:06.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.015+0000 7f0c42c00640 1 -- start start 2024-04-20T11:27:06.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.015+0000 7fa60f400640 1 -- start start 2024-04-20T11:27:06.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.015+0000 7ff6a1800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff69c166270 0x7ff69c153470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:06.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.015+0000 7ff6a1800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff69c166650 0x7ff69c0b71c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:06.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.016+0000 7ff6a1800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff69c0caae0 0x7ff69c1f8140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:06.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.016+0000 7ff6a1800640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7ff69c0d1440 con 0x7ff69c0caae0 2024-04-20T11:27:06.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.016+0000 7ff6a1800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff69c0d52d0 con 0x7ff69c166270 2024-04-20T11:27:06.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.016+0000 7ff6a1800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff69c0d15c0 con 0x7ff69c166650 2024-04-20T11:27:06.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.015+0000 7f6e15000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e100cf5a0 0x7f6e10157c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:06.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.015+0000 7f6e15000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6e100cfa00 0x7f6e100cb930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:06.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.015+0000 7f6e15000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e1015ec80 0x7f6e100d3e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:06.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.015+0000 7f6e15000640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f6e100cfe90 con 0x7f6e100cfa00 2024-04-20T11:27:06.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.015+0000 7f6e15000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6e100b9590 con 0x7f6e100cf5a0 2024-04-20T11:27:06.026 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.015+0000 7f6e15000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6e1015f060 con 0x7f6e1015ec80 2024-04-20T11:27:06.026 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.016+0000 7ff6a0e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff69c0caae0 0x7ff69c1f8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:06.026 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.016+0000 7ff6a0e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff69c0caae0 0x7ff69c1f8140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:47988/0 (socket says 172.21.15.152:47988) 2024-04-20T11:27:06.026 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.016+0000 7ff6a0e00640 1 -- 172.21.15.152:0/2281423501 learned_addr learned my addr 172.21.15.152:0/2281423501 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:06.026 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.016+0000 7f6e0d400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6e100cfa00 0x7f6e100cb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:06.026 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.016+0000 7f6e0d400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6e100cfa00 0x7f6e100cb930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:47996/0 (socket says 172.21.15.152:47996) 2024-04-20T11:27:06.026 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.016+0000 7f6e0d400640 1 -- 172.21.15.152:0/361768913 learned_addr learned my addr 172.21.15.152:0/361768913 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:06.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.016+0000 7ff69b400640 1 --2- 172.21.15.152:0/2281423501 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff69c166650 0x7ff69c0b71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:06.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.016+0000 7f6e0de00640 1 --2- 172.21.15.152:0/361768913 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e100cf5a0 0x7f6e10157c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:06.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.016+0000 7ff69be00640 1 --2- 172.21.15.152:0/2281423501 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff69c166270 0x7ff69c153470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:06.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.016+0000 7f6e0e800640 1 --2- 172.21.15.152:0/361768913 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e1015ec80 0x7f6e100d3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:06.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.017+0000 7f6e0d400640 1 -- 172.21.15.152:0/361768913 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e1015ec80 msgr2=0x7f6e100d3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:06.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.017+0000 7f6e0d400640 1 --2- 172.21.15.152:0/361768913 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e1015ec80 0x7f6e100d3e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:06.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.017+0000 7f6e0d400640 1 -- 172.21.15.152:0/361768913 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e100cf5a0 msgr2=0x7f6e10157c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:06.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.017+0000 7f6e0d400640 1 --2- 172.21.15.152:0/361768913 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e100cf5a0 0x7f6e10157c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:06.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.017+0000 7f6e0d400640 1 -- 172.21.15.152:0/361768913 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e100cc030 con 0x7f6e100cfa00 2024-04-20T11:27:06.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.017+0000 7f0c42c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0c3c15a860 0x7f0c3c0b3780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:06.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.017+0000 7ff6a0e00640 1 -- 172.21.15.152:0/2281423501 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff69c166650 msgr2=0x7ff69c0b71c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:06.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.017+0000 7ff6a0e00640 1 --2- 172.21.15.152:0/2281423501 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff69c166650 0x7ff69c0b71c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:06.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.017+0000 7ff6a0e00640 1 -- 172.21.15.152:0/2281423501 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff69c166270 msgr2=0x7ff69c153470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:06.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.017+0000 7ff6a0e00640 1 --2- 172.21.15.152:0/2281423501 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff69c166270 0x7ff69c153470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:06.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.017+0000 7ff6a0e00640 1 -- 172.21.15.152:0/2281423501 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff69c0c9aa0 con 0x7ff69c0caae0 2024-04-20T11:27:06.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.017+0000 7f0c42c00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f0c3c071db0 0x7f0c3c072260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:06.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.017+0000 7f0c42c00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c3c0b4e00 con 0x7f0c3c06fea0 2024-04-20T11:27:06.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.017+0000 7f0c42c00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f0c3c0b3d60 con 0x7f0c3c071db0 2024-04-20T11:27:06.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.017+0000 7f0c42c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0c3c0b4040 con 0x7f0c3c15a860 2024-04-20T11:27:06.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.017+0000 7f0c3b400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f0c3c06fea0 0x7f0c3c15ac40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:48476/0 (socket says 172.21.15.152:48476) 2024-04-20T11:27:06.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.017+0000 7f0c3b400640 1 -- 172.21.15.152:0/3207842677 learned_addr learned my addr 172.21.15.152:0/3207842677 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:27:06.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.017+0000 7fa60f400640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fa60806fa50 0x7fa6080b3470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:06.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.017+0000 7fa60f400640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6080b4130 con 0x7fa60815fd60 2024-04-20T11:27:06.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.017+0000 7fa60f400640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa60815f280 con 0x7fa60815b2b0 2024-04-20T11:27:06.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.017+0000 7fa60f400640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fa60815f8d0 con 0x7fa60806fa50 2024-04-20T11:27:06.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.018+0000 7f0c40c00640 1 --2- 172.21.15.152:0/3207842677 >> v2:172.21.15.152:3300/0 conn(0x7f0c3c071db0 0x7f0c3c072260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:06.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.018+0000 7f0c3be00640 1 --2- 172.21.15.152:0/3207842677 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0c3c15a860 0x7f0c3c0b3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:06.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.018+0000 7f1cab400640 1 -- 172.21.15.152:0/1867463124 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f1cac059ad0 con 0x7f1cb00e7a40 2024-04-20T11:27:06.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.018+0000 7ff6a0e00640 1 --2- 172.21.15.152:0/2281423501 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff69c0caae0 0x7ff69c1f8140 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff694009e50 tx=0x7ff694004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:06.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.018+0000 7f6e0d400640 1 --2- 172.21.15.152:0/361768913 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6e100cfa00 0x7f6e100cb930 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc00c410 tx=0x7f6dfc00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:06.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.018+0000 7fa60d400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fa60815fd60 0x7fa6080b4790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:48488/0 (socket says 172.21.15.152:48488) 2024-04-20T11:27:06.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.018+0000 7fa60d400640 1 -- 172.21.15.152:0/2509715726 learned_addr learned my addr 172.21.15.152:0/2509715726 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:27:06.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.018+0000 7f0c40c00640 1 -- 172.21.15.152:0/3207842677 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0c3c15a860 msgr2=0x7f0c3c0b3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:06.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.018+0000 7f0c40c00640 1 --2- 172.21.15.152:0/3207842677 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0c3c15a860 0x7f0c3c0b3780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:06.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.018+0000 7f0c40c00640 1 -- 172.21.15.152:0/3207842677 >> v1:172.21.15.152:6789/0 conn(0x7f0c3c06fea0 legacy=0x7f0c3c15ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:06.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.018+0000 7f0c40c00640 1 -- 172.21.15.152:0/3207842677 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c3c0b4320 con 0x7f0c3c071db0 2024-04-20T11:27:06.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.018+0000 7f0c3be00640 1 --2- 172.21.15.152:0/3207842677 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0c3c15a860 0x7f0c3c0b3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:27:06.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.018+0000 7fa607e00640 1 --2- 172.21.15.152:0/2509715726 >> v2:172.21.15.175:3300/0 conn(0x7fa60806fa50 0x7fa6080b3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:06.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.019+0000 7fa607400640 1 -- 172.21.15.152:0/2509715726 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 327489586 0 0) 0x7fa60815f280 con 0x7fa60815b2b0 2024-04-20T11:27:06.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.019+0000 7fa607400640 1 -- 172.21.15.152:0/2509715726 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa5f0003520 con 0x7fa60815b2b0 2024-04-20T11:27:06.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.019+0000 7fa607e00640 1 -- 172.21.15.152:0/2509715726 >> v1:172.21.15.175:6789/0 conn(0x7fa60815b2b0 legacy=0x7fa60806f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:06.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.019+0000 7fa607e00640 1 -- 172.21.15.152:0/2509715726 >> v1:172.21.15.152:6789/0 conn(0x7fa60815fd60 legacy=0x7fa6080b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:06.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.019+0000 7fa607e00640 1 -- 172.21.15.152:0/2509715726 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa60815f560 con 0x7fa60806fa50 2024-04-20T11:27:06.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.020+0000 7f0c40c00640 1 --2- 172.21.15.152:0/3207842677 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0c3c071db0 0x7f0c3c072260 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f0c3400bfa0 tx=0x7f0c3402e0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=744fa0975e33b1d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:06.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.021+0000 7fa607e00640 1 --2- 172.21.15.152:0/2509715726 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa60806fa50 0x7fa6080b3470 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa5f40097a0 tx=0x7fa5f402b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=77c19bc9b0b1297a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:06.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.021+0000 7fa607400640 1 -- 172.21.15.152:0/2509715726 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f4034070 con 0x7fa60806fa50 2024-04-20T11:27:06.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.021+0000 7fa607400640 1 -- 172.21.15.152:0/2509715726 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5f4030610 con 0x7fa60806fa50 2024-04-20T11:27:06.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.021+0000 7fa607400640 1 -- 172.21.15.152:0/2509715726 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f40309e0 con 0x7fa60806fa50 2024-04-20T11:27:06.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.022+0000 7fa60f400640 1 -- 172.21.15.152:0/2509715726 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa60806fa50 msgr2=0x7fa6080b3470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:06.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.022+0000 7fa60f400640 1 --2- 172.21.15.152:0/2509715726 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa60806fa50 0x7fa6080b3470 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa5f40097a0 tx=0x7fa5f402b960 comp rx=0 tx=0).stop 2024-04-20T11:27:06.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.022+0000 7fa60f400640 1 -- 172.21.15.152:0/2509715726 shutdown_connections 2024-04-20T11:27:06.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.022+0000 7fa60f400640 1 --2- 172.21.15.152:0/2509715726 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa60806fa50 0x7fa6080b3470 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:06.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.022+0000 7fa60f400640 1 -- 172.21.15.152:0/2509715726 >> 172.21.15.152:0/2509715726 conn(0x7fa608067cf0 msgr2=0x7fa60806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:06.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.022+0000 7fa60f400640 1 -- 172.21.15.152:0/2509715726 shutdown_connections 2024-04-20T11:27:06.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.022+0000 7fa60f400640 1 -- 172.21.15.152:0/2509715726 wait complete. 2024-04-20T11:27:06.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.024+0000 7f0c3aa00640 1 -- 172.21.15.152:0/3207842677 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c3402ea40 con 0x7f0c3c071db0 2024-04-20T11:27:06.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.024+0000 7f6e06000640 1 -- 172.21.15.152:0/361768913 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dfc017070 con 0x7f6e100cfa00 2024-04-20T11:27:06.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.024+0000 7ff693e00640 1 -- 172.21.15.152:0/2281423501 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff694043020 con 0x7ff69c0caae0 2024-04-20T11:27:06.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.024+0000 7ff693e00640 1 -- 172.21.15.152:0/2281423501 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff69403e450 con 0x7ff69c0caae0 2024-04-20T11:27:06.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.024+0000 7ff693e00640 1 -- 172.21.15.152:0/2281423501 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff69403e860 con 0x7ff69c0caae0 2024-04-20T11:27:06.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.024+0000 7f0c3aa00640 1 -- 172.21.15.152:0/3207842677 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c3402ebe0 con 0x7f0c3c071db0 2024-04-20T11:27:06.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.024+0000 7f0c3aa00640 1 -- 172.21.15.152:0/3207842677 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c34009540 con 0x7f0c3c071db0 2024-04-20T11:27:06.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.024+0000 7ff6a1800640 1 -- 172.21.15.152:0/2281423501 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff69c0b7700 con 0x7ff69c0caae0 2024-04-20T11:27:06.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.024+0000 7f151ea00640 1 -- 172.21.15.152:0/3270428710 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f150c031780 con 0x7f152015ec80 2024-04-20T11:27:06.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.024+0000 7f151ea00640 1 -- 172.21.15.152:0/3270428710 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f150c037070 con 0x7f152015ec80 2024-04-20T11:27:06.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.024+0000 7f151ea00640 1 -- 172.21.15.152:0/3270428710 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f150c031b40 con 0x7f152015ec80 2024-04-20T11:27:06.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.024+0000 7f6e06000640 1 -- 172.21.15.152:0/361768913 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6dfc0035f0 con 0x7f6e100cfa00 2024-04-20T11:27:06.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.024+0000 7f6e15000640 1 -- 172.21.15.152:0/361768913 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e100cc310 con 0x7f6e100cfa00 2024-04-20T11:27:06.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.024+0000 7f6e06000640 1 -- 172.21.15.152:0/361768913 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dfc003a20 con 0x7f6e100cfa00 2024-04-20T11:27:06.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.025+0000 7fc139600640 1 -- 172.21.15.152:0/2411782992 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fc128058660 con 0x7fc1340e7640 2024-04-20T11:27:06.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.024+0000 7fc25b400640 1 -- 172.21.15.152:0/2285621282 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc254039070 con 0x7fc25c0cfff0 2024-04-20T11:27:06.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.025+0000 7fc25b400640 1 -- 172.21.15.152:0/2285621282 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc25402cbe0 con 0x7fc25c0cfff0 2024-04-20T11:27:06.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.025+0000 7fc25b400640 1 -- 172.21.15.152:0/2285621282 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc254034440 con 0x7fc25c0cfff0 2024-04-20T11:27:06.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.025+0000 7fa60f400640 1 Processor -- start 2024-04-20T11:27:06.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.024+0000 7ff6a1800640 1 -- 172.21.15.152:0/2281423501 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff69c0ca000 con 0x7ff69c0caae0 2024-04-20T11:27:06.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.025+0000 7f6e15000640 1 -- 172.21.15.152:0/361768913 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e10156740 con 0x7f6e100cfa00 2024-04-20T11:27:06.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.025+0000 7fa60f400640 1 -- start start 2024-04-20T11:27:06.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.026+0000 7fc263800640 1 -- 172.21.15.152:0/2285621282 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc25c0cfff0 msgr2=0x7fc25c0d0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:06.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.026+0000 7fc263800640 1 --2- 172.21.15.152:0/2285621282 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc25c0cfff0 0x7fc25c0d0460 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc25c0700a0 tx=0x7fc25402be30 comp rx=0 tx=0).stop 2024-04-20T11:27:06.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.025+0000 7f0c42c00640 1 -- 172.21.15.152:0/3207842677 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0c3c071db0 msgr2=0x7f0c3c072260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:06.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.025+0000 7f0c42c00640 1 --2- 172.21.15.152:0/3207842677 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0c3c071db0 0x7f0c3c072260 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f0c3400bfa0 tx=0x7f0c3402e0a0 comp rx=0 tx=0).stop 2024-04-20T11:27:06.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.026+0000 7fa60f400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa60806fa50 0x7fa6080d0510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:06.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.026+0000 7fa60f400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa60815b2b0 0x7fa6081f8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:06.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.026+0000 7fa60f400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa60815fd60 0x7fa6081f8d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:06.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.026+0000 7fa60f400640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fa6080b39b0 con 0x7fa60806fa50 2024-04-20T11:27:06.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.026+0000 7fa60f400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa60815b8e0 con 0x7fa60815fd60 2024-04-20T11:27:06.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.026+0000 7fa60f400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa6080b3b30 con 0x7fa60815b2b0 2024-04-20T11:27:06.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.026+0000 7f1527200640 1 -- 172.21.15.152:0/3270428710 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f152015ec80 msgr2=0x7f15200ba180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:06.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.026+0000 7f1527200640 1 --2- 172.21.15.152:0/3270428710 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f152015ec80 0x7f15200ba180 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f150c0097a0 tx=0x7f150c02b820 comp rx=0 tx=0).stop 2024-04-20T11:27:06.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.027+0000 7ff6a1800640 1 -- 172.21.15.152:0/2281423501 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff674005290 con 0x7ff69c0caae0 2024-04-20T11:27:06.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.027+0000 7fc263800640 1 -- 172.21.15.152:0/2285621282 shutdown_connections 2024-04-20T11:27:06.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.027+0000 7fc263800640 1 --2- 172.21.15.152:0/2285621282 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc25c0cfff0 0x7fc25c0d0460 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:06.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.027+0000 7fc263800640 1 --2- 172.21.15.152:0/2285621282 >> v2:172.21.15.175:3300/0 conn(0x7fc25c162b40 0x7fc25c0b3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:06.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.027+0000 7fc263800640 1 -- 172.21.15.152:0/2285621282 >> 172.21.15.152:0/2285621282 conn(0x7fc25c067cf0 msgr2=0x7fc25c06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:06.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.027+0000 7f0c42c00640 1 -- 172.21.15.152:0/3207842677 shutdown_connections 2024-04-20T11:27:06.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.027+0000 7f0c42c00640 1 --2- 172.21.15.152:0/3207842677 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0c3c071db0 0x7f0c3c072260 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:06.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.027+0000 7f0c42c00640 1 --2- 172.21.15.152:0/3207842677 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0c3c15a860 0x7f0c3c0b3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:06.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.027+0000 7f0c42c00640 1 -- 172.21.15.152:0/3207842677 >> 172.21.15.152:0/3207842677 conn(0x7f0c3c067cf0 msgr2=0x7f0c3c06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:06.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.027+0000 7f1527200640 1 -- 172.21.15.152:0/3270428710 shutdown_connections 2024-04-20T11:27:06.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.027+0000 7f1527200640 1 --2- 172.21.15.152:0/3270428710 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f15200cf5a0 0x7f15200cfa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:06.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.027+0000 7f1527200640 1 --2- 172.21.15.152:0/3270428710 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f152015ec80 0x7f15200ba180 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:06.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.027+0000 7f1527200640 1 -- 172.21.15.152:0/3270428710 >> 172.21.15.152:0/3270428710 conn(0x7f1520067ae0 msgr2=0x7f1520067ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:06.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.027+0000 7fa60ca00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa60806fa50 0x7fa6080d0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:06.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.028+0000 7f0c42c00640 1 -- 172.21.15.152:0/3207842677 shutdown_connections 2024-04-20T11:27:06.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.028+0000 7f1527200640 1 -- 172.21.15.152:0/3270428710 shutdown_connections 2024-04-20T11:27:06.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.027+0000 7f6e15000640 1 -- 172.21.15.152:0/361768913 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e100689f0 con 0x7f6e100cfa00 2024-04-20T11:27:06.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.028+0000 7f1527200640 1 -- 172.21.15.152:0/3270428710 wait complete. 2024-04-20T11:27:06.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.027+0000 7fa60ca00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa60806fa50 0x7fa6080d0510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:48012/0 (socket says 172.21.15.152:48012) 2024-04-20T11:27:06.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.027+0000 7fa60ca00640 1 -- 172.21.15.152:0/3211252062 learned_addr learned my addr 172.21.15.152:0/3211252062 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:06.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.028+0000 7fa60d400640 1 --2- 172.21.15.152:0/3211252062 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa60815fd60 0x7fa6081f8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:06.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.027+0000 7fa607e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa60815b2b0 0x7fa6081f8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:06.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.028+0000 7fa60ca00640 1 -- 172.21.15.152:0/3211252062 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa60815b2b0 msgr2=0x7fa6081f8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:06.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.028+0000 7fa60ca00640 1 --2- 172.21.15.152:0/3211252062 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa60815b2b0 0x7fa6081f8810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:06.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.028+0000 7fa60ca00640 1 -- 172.21.15.152:0/3211252062 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa60815fd60 msgr2=0x7fa6081f8d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:06.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.028+0000 7fa60ca00640 1 --2- 172.21.15.152:0/3211252062 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa60815fd60 0x7fa6081f8d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:06.054 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.028+0000 7fa60ca00640 1 -- 172.21.15.152:0/3211252062 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa60815d720 con 0x7fa60806fa50 2024-04-20T11:27:06.054 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.029+0000 7fa60ca00640 1 --2- 172.21.15.152:0/3211252062 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa60806fa50 0x7fa6080d0510 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc00c530 tx=0x7fa5fc00c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:06.054 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.028+0000 7f0c42c00640 1 -- 172.21.15.152:0/3207842677 wait complete. 2024-04-20T11:27:06.054 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.028+0000 7fc263800640 1 -- 172.21.15.152:0/2285621282 shutdown_connections 2024-04-20T11:27:06.054 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.028+0000 7fc263800640 1 -- 172.21.15.152:0/2285621282 wait complete. 2024-04-20T11:27:06.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.031+0000 7f0c42c00640 1 Processor -- start 2024-04-20T11:27:06.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.031+0000 7f0c42c00640 1 -- start start 2024-04-20T11:27:06.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.031+0000 7f1527200640 1 Processor -- start 2024-04-20T11:27:06.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.029+0000 7ff693e00640 1 -- 172.21.15.152:0/2281423501 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff694005d10 con 0x7ff69c0caae0 2024-04-20T11:27:06.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.031+0000 7ff693e00640 1 --2- 172.21.15.152:0/2281423501 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff67806c940 0x7ff67806ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:06.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.031+0000 7ff693e00640 1 -- 172.21.15.152:0/2281423501 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff6940ae540 con 0x7ff69c0caae0 2024-04-20T11:27:06.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.031+0000 7f1527200640 1 -- start start 2024-04-20T11:27:06.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.029+0000 7f6e06000640 1 -- 172.21.15.152:0/361768913 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6dfc01a070 con 0x7f6e100cfa00 2024-04-20T11:27:06.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.031+0000 7f6e06000640 1 --2- 172.21.15.152:0/361768913 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f6df006c940 0x7f6df006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:06.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.033+0000 7fc263800640 1 Processor -- start 2024-04-20T11:27:06.057 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.033+0000 7ff69be00640 1 --2- 172.21.15.152:0/2281423501 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff67806c940 0x7ff67806ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:06.057 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.033+0000 7fc263800640 1 -- start start 2024-04-20T11:27:06.057 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.033+0000 7f1527200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f15200b7990 0x7f1520151cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:06.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.033+0000 7f0c42c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0c3c06fea0 0x7f0c3c158330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:06.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.033+0000 7f1527200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f15200cf5a0 0x7f15200ca990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:06.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.034+0000 7f1527200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f152015ec80 0x7f15200cd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:06.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.033+0000 7f0c42c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0c3c071db0 0x7f0c3c145a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:06.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.034+0000 7f1527200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f15200b9590 con 0x7f15200b7990 2024-04-20T11:27:06.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.034+0000 7f1527200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f15200b9900 con 0x7f152015ec80 2024-04-20T11:27:06.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.034+0000 7f1527200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f152015f060 con 0x7f15200cf5a0 2024-04-20T11:27:06.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.034+0000 7f0c42c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c3c15a860 0x7f0c3c145f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:06.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.032+0000 7f6e0de00640 1 --2- 172.21.15.152:0/361768913 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f6df006c940 0x7f6df006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:06.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.034+0000 7f6e0de00640 1 --2- 172.21.15.152:0/361768913 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f6df006c940 0x7f6df006ee00 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6e08035000 tx=0x7f6e08030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:06.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.034+0000 7f151f400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f15200cf5a0 0x7f15200ca990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:06.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.034+0000 7f151fe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f15200b7990 0x7f1520151cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:06.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.035+0000 7f1525200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f152015ec80 0x7f15200cd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:06.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.035+0000 7f151fe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f15200b7990 0x7f1520151cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:48028/0 (socket says 172.21.15.152:48028) 2024-04-20T11:27:06.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.035+0000 7f151fe00640 1 -- 172.21.15.152:0/820000764 learned_addr learned my addr 172.21.15.152:0/820000764 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:06.061 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.035+0000 7fc263800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc25c06f810 0x7fc25c14b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:06.061 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.035+0000 7f0c3be00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0c3c06fea0 0x7f0c3c158330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:06.061 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.035+0000 7f0c3be00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0c3c06fea0 0x7f0c3c158330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:48030/0 (socket says 172.21.15.152:48030) 2024-04-20T11:27:06.061 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.035+0000 7f151f400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f15200cf5a0 0x7f15200ca990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.152:34086/0 (socket says 172.21.15.152:34086) 2024-04-20T11:27:06.062 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.035+0000 7ff69be00640 1 --2- 172.21.15.152:0/2281423501 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff67806c940 0x7ff67806ee00 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff684004f60 tx=0x7ff684007e30 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:06.062 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.035+0000 7fc263800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc25c0cfff0 0x7fc25c144f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:06.062 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.035+0000 7fc263800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc25c162b40 0x7fc25c1616d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:06.062 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.035+0000 7fc263800640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fc25c06e950 con 0x7fc25c0cfff0 2024-04-20T11:27:06.062 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.035+0000 7f0c42c00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f0c3c0afdd0 con 0x7f0c3c06fea0 2024-04-20T11:27:06.062 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.035+0000 7f0c42c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0c3c0b48e0 con 0x7f0c3c15a860 2024-04-20T11:27:06.063 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.035+0000 7f0c42c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0c3c1f6d90 con 0x7f0c3c071db0 2024-04-20T11:27:06.063 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.035+0000 7f0c3be00640 1 -- 172.21.15.152:0/464555806 learned_addr learned my addr 172.21.15.152:0/464555806 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:06.063 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.035+0000 7f0c3b400640 1 --2- 172.21.15.152:0/464555806 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0c3c071db0 0x7f0c3c145a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:06.063 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.035+0000 7fc263800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc25c06e0b0 con 0x7fc25c162b40 2024-04-20T11:27:06.063 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.036+0000 7fc260e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc25c06f810 0x7fc25c14b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:06.063 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.036+0000 7fc260e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc25c06f810 0x7fc25c14b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.152:34106/0 (socket says 172.21.15.152:34106) 2024-04-20T11:27:06.063 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.036+0000 7fc260e00640 1 -- 172.21.15.152:0/4010114470 learned_addr learned my addr 172.21.15.152:0/4010114470 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:06.063 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.036+0000 7fc263800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc25c162f20 con 0x7fc25c06f810 2024-04-20T11:27:06.064 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.036+0000 7f151f400640 1 -- 172.21.15.152:0/820000764 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f152015ec80 msgr2=0x7f15200cd4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:06.064 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.036+0000 7f151f400640 1 --2- 172.21.15.152:0/820000764 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f152015ec80 0x7f15200cd4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:06.064 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.036+0000 7f151f400640 1 -- 172.21.15.152:0/820000764 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f15200b7990 msgr2=0x7f1520151cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:06.064 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.036+0000 7f151f400640 1 --2- 172.21.15.152:0/820000764 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f15200b7990 0x7f1520151cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:06.064 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.036+0000 7f151f400640 1 -- 172.21.15.152:0/820000764 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15200c8f90 con 0x7f15200cf5a0 2024-04-20T11:27:06.064 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.036+0000 7fc25be00640 1 --2- 172.21.15.152:0/4010114470 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc25c0cfff0 0x7fc25c144f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:06.065 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.036+0000 7f0c40c00640 1 --2- 172.21.15.152:0/464555806 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c3c15a860 0x7f0c3c145f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:06.065 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.036+0000 7fa604c00640 1 -- 172.21.15.152:0/3211252062 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5fc0052d0 con 0x7fa60806fa50 2024-04-20T11:27:06.065 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.037+0000 7fa604c00640 1 -- 172.21.15.152:0/3211252062 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5fc005490 con 0x7fa60806fa50 2024-04-20T11:27:06.065 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.037+0000 7f0c3be00640 1 -- 172.21.15.152:0/464555806 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0c3c071db0 msgr2=0x7f0c3c145a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:06.066 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.037+0000 7f0c3be00640 1 --2- 172.21.15.152:0/464555806 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0c3c071db0 0x7f0c3c145a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:06.066 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.036+0000 7fc261800640 1 --2- 172.21.15.152:0/4010114470 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc25c162b40 0x7fc25c1616d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:06.066 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.037+0000 7f0c3be00640 1 -- 172.21.15.152:0/464555806 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c3c15a860 msgr2=0x7f0c3c145f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:06.066 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.037+0000 7f0c3be00640 1 --2- 172.21.15.152:0/464555806 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c3c15a860 0x7f0c3c145f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:06.067 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.037+0000 7f0c3be00640 1 -- 172.21.15.152:0/464555806 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c3c0cee60 con 0x7f0c3c06fea0 2024-04-20T11:27:06.067 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.037+0000 7f151f400640 1 --2- 172.21.15.152:0/820000764 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f15200cf5a0 0x7f15200ca990 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1510006fd0 tx=0x7f151000ec70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:06.067 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.037+0000 7fa60f400640 1 -- 172.21.15.152:0/3211252062 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa60815ddb0 con 0x7fa60806fa50 2024-04-20T11:27:06.067 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.037+0000 7fa60f400640 1 -- 172.21.15.152:0/3211252062 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa60815e370 con 0x7fa60806fa50 2024-04-20T11:27:06.067 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.037+0000 7fa604c00640 1 -- 172.21.15.152:0/3211252062 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5fc012870 con 0x7fa60806fa50 2024-04-20T11:27:06.067 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.037+0000 7f1517e00640 1 -- 172.21.15.152:0/820000764 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f151000f040 con 0x7f15200cf5a0 2024-04-20T11:27:06.068 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.037+0000 7f1527200640 1 -- 172.21.15.152:0/820000764 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15200c9270 con 0x7f15200cf5a0 2024-04-20T11:27:06.068 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.037+0000 7ff693e00640 1 -- 172.21.15.152:0/2281423501 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ff69c0b7700 con 0x7ff69c0caae0 2024-04-20T11:27:06.068 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.037+0000 7f1527200640 1 -- 172.21.15.152:0/820000764 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f15200c9830 con 0x7f15200cf5a0 2024-04-20T11:27:06.068 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.038+0000 7f1517e00640 1 -- 172.21.15.152:0/820000764 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1510004030 con 0x7f15200cf5a0 2024-04-20T11:27:06.068 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.038+0000 7f1517e00640 1 -- 172.21.15.152:0/820000764 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15100043e0 con 0x7f15200cf5a0 2024-04-20T11:27:06.069 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.038+0000 7f6e06000640 1 -- 172.21.15.152:0/361768913 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f6dfc056b20 con 0x7f6e100cfa00 2024-04-20T11:27:06.069 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.038+0000 7f0c3be00640 1 --2- 172.21.15.152:0/464555806 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0c3c06fea0 0x7f0c3c158330 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c00d7d0 tx=0x7f0c2c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:06.070 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.038+0000 7fc260e00640 1 -- 172.21.15.152:0/4010114470 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc25c162b40 msgr2=0x7fc25c1616d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:06.070 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.038+0000 7fc260e00640 1 --2- 172.21.15.152:0/4010114470 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc25c162b40 0x7fc25c1616d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:06.070 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.038+0000 7fc260e00640 1 -- 172.21.15.152:0/4010114470 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc25c0cfff0 msgr2=0x7fc25c144f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:06.070 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.038+0000 7fc260e00640 1 --2- 172.21.15.152:0/4010114470 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc25c0cfff0 0x7fc25c144f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:06.070 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.039+0000 7fc260e00640 1 -- 172.21.15.152:0/4010114470 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc25c14a690 con 0x7fc25c06f810 2024-04-20T11:27:06.070 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.039+0000 7fc261800640 1 --2- 172.21.15.152:0/4010114470 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc25c162b40 0x7fc25c1616d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:27:06.070 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.039+0000 7fc25be00640 1 --2- 172.21.15.152:0/4010114470 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc25c0cfff0 0x7fc25c144f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:27:06.071 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.039+0000 7fa60f400640 1 -- 172.21.15.152:0/3211252062 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5e4005290 con 0x7fa60806fa50 2024-04-20T11:27:06.071 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.039+0000 7fc260e00640 1 --2- 172.21.15.152:0/4010114470 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc25c06f810 0x7fc25c14b450 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc25000e8b0 tx=0x7fc25000ed70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:06.071 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.040+0000 7fc258c00640 1 -- 172.21.15.152:0/4010114470 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc250017070 con 0x7fc25c06f810 2024-04-20T11:27:06.071 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.040+0000 7fc263800640 1 -- 172.21.15.152:0/4010114470 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc25c14a970 con 0x7fc25c06f810 2024-04-20T11:27:06.071 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.040+0000 7f1527200640 1 -- 172.21.15.152:0/820000764 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14f8005290 con 0x7f15200cf5a0 2024-04-20T11:27:06.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.041+0000 7f1517e00640 1 -- 172.21.15.152:0/820000764 <== mon.2 v2:172.21.15.175:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f151000c6d0 con 0x7f15200cf5a0 2024-04-20T11:27:06.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.040+0000 7fc258c00640 1 -- 172.21.15.152:0/4010114470 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc25000c940 con 0x7fc25c06f810 2024-04-20T11:27:06.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.040+0000 7fc258c00640 1 -- 172.21.15.152:0/4010114470 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc250017070 con 0x7fc25c06f810 2024-04-20T11:27:06.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.040+0000 7fc263800640 1 -- 172.21.15.152:0/4010114470 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc25c14af30 con 0x7fc25c06f810 2024-04-20T11:27:06.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.041+0000 7fc263800640 1 -- 172.21.15.152:0/4010114470 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc25c0689f0 con 0x7fc25c06f810 2024-04-20T11:27:06.073 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.043+0000 7f1517e00640 1 --2- 172.21.15.152:0/820000764 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f150006c8f0 0x7f150006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:06.073 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.043+0000 7f1517e00640 1 -- 172.21.15.152:0/820000764 <== mon.2 v2:172.21.15.175:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f151008b680 con 0x7f15200cf5a0 2024-04-20T11:27:06.073 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.043+0000 7f151fe00640 1 --2- 172.21.15.152:0/820000764 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f150006c8f0 0x7f150006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:06.073 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.044+0000 7f151fe00640 1 --2- 172.21.15.152:0/820000764 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f150006c8f0 0x7f150006edb0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f150c02b860 tx=0x7f150c03d040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:06.074 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.044+0000 7fc258c00640 1 -- 172.21.15.152:0/4010114470 <== mon.2 v2:172.21.15.175:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc25001a070 con 0x7fc25c06f810 2024-04-20T11:27:06.074 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.045+0000 7fc258c00640 1 --2- 172.21.15.152:0/4010114470 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc23c06cb20 0x7fc23c06efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:06.074 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.045+0000 7fc258c00640 1 -- 172.21.15.152:0/4010114470 <== mon.2 v2:172.21.15.175:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc25008bcc0 con 0x7fc25c06f810 2024-04-20T11:27:06.074 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.045+0000 7fc25be00640 1 --2- 172.21.15.152:0/4010114470 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc23c06cb20 0x7fc23c06efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:06.075 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.046+0000 7fc25be00640 1 --2- 172.21.15.152:0/4010114470 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc23c06cb20 0x7fc23c06efe0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc248007fd0 tx=0x7fc248009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:06.075 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.047+0000 7f6e06000640 1 -- 172.21.15.152:0/361768913 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f6dfc0010c0 con 0x7f6e100cfa00 2024-04-20T11:27:06.076 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.050+0000 7f0c33e00640 1 -- 172.21.15.152:0/464555806 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c2c004030 con 0x7f0c3c06fea0 2024-04-20T11:27:06.076 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.050+0000 7f0c42c00640 1 -- 172.21.15.152:0/464555806 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c3c0cf140 con 0x7f0c3c06fea0 2024-04-20T11:27:06.076 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.050+0000 7f0c42c00640 1 -- 172.21.15.152:0/464555806 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c3c1f72c0 con 0x7f0c3c06fea0 2024-04-20T11:27:06.076 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.051+0000 7f0c33e00640 1 -- 172.21.15.152:0/464555806 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c2c0041d0 con 0x7f0c3c06fea0 2024-04-20T11:27:06.076 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.052+0000 7f0c42c00640 1 -- 172.21.15.152:0/464555806 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c3c0ccdd0 con 0x7f0c3c06fea0 2024-04-20T11:27:06.076 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.052+0000 7fa604c00640 1 -- 172.21.15.152:0/3211252062 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa5fc0035f0 con 0x7fa60806fa50 2024-04-20T11:27:06.077 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.052+0000 7f0c33e00640 1 -- 172.21.15.152:0/464555806 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c2c004030 con 0x7f0c3c06fea0 2024-04-20T11:27:06.077 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.053+0000 7fa604c00640 1 --2- 172.21.15.152:0/3211252062 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fa5e806c9c0 0x7fa5e806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:06.078 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.054+0000 7fa604c00640 1 -- 172.21.15.152:0/3211252062 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa5fc08c200 con 0x7fa60806fa50 2024-04-20T11:27:06.078 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.054+0000 7fa607e00640 1 --2- 172.21.15.152:0/3211252062 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fa5e806c9c0 0x7fa5e806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:06.078 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.055+0000 7fa607e00640 1 --2- 172.21.15.152:0/3211252062 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fa5e806c9c0 0x7fa5e806ee80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa5f4036000 tx=0x7fa5f403b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:06.078 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.057+0000 7fa604c00640 1 -- 172.21.15.152:0/3211252062 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fa5fc05a6a0 con 0x7fa60806fa50 2024-04-20T11:27:06.078 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.061+0000 7f1517e00640 1 -- 172.21.15.152:0/820000764 <== mon.2 v2:172.21.15.175:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f1510017020 con 0x7f15200cf5a0 2024-04-20T11:27:06.079 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.060+0000 7f0c33e00640 1 -- 172.21.15.152:0/464555806 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0c2c004380 con 0x7f0c3c06fea0 2024-04-20T11:27:06.079 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.062+0000 7f0c33e00640 1 --2- 172.21.15.152:0/464555806 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f0c1c06cb20 0x7f0c1c06efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:06.079 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.062+0000 7f0c33e00640 1 -- 172.21.15.152:0/464555806 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f0c2c08c430 con 0x7f0c3c06fea0 2024-04-20T11:27:06.079 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.062+0000 7f0c3b400640 1 --2- 172.21.15.152:0/464555806 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f0c1c06cb20 0x7f0c1c06efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:06.079 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.064+0000 7f0c3b400640 1 --2- 172.21.15.152:0/464555806 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f0c1c06cb20 0x7f0c1c06efe0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c1fc380 tx=0x7f0c28007e80 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:06.079 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.067+0000 7f0c33e00640 1 -- 172.21.15.152:0/464555806 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f0c3c0ccdd0 con 0x7f0c3c06fea0 2024-04-20T11:27:06.080 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.080+0000 7fc258c00640 1 -- 172.21.15.152:0/4010114470 <== mon.2 v2:172.21.15.175:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fc250059c10 con 0x7fc25c06f810 2024-04-20T11:27:06.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.428+0000 7f1cbe800640 1 -- 172.21.15.152:0/1867463124 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f1cb00578e0 con 0x7f1cb00e7a40 2024-04-20T11:27:06.431 INFO:teuthology.orchestra.run.smithi152.stdout:55834574852 2024-04-20T11:27:06.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.429+0000 7f1cab400640 1 -- 172.21.15.152:0/1867463124 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1cac05ddd0 con 0x7f1cb00e7a40 2024-04-20T11:27:06.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.432+0000 7f1ca8c00640 1 -- 172.21.15.152:0/1867463124 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f1c9806c920 msgr2=0x7f1c9806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:06.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.432+0000 7f1ca8c00640 1 --2- 172.21.15.152:0/1867463124 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f1c9806c920 0x7f1c9806ede0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1ca400aae0 tx=0x7f1ca4008040 comp rx=0 tx=0).stop 2024-04-20T11:27:06.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.432+0000 7f1ca8c00640 1 -- 172.21.15.152:0/1867463124 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1cb00e7a40 msgr2=0x7f1cb004b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:06.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.432+0000 7f1ca8c00640 1 --2- 172.21.15.152:0/1867463124 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1cb00e7a40 0x7f1cb004b3c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1cac00c540 tx=0x7f1cac00ca00 comp rx=0 tx=0).stop 2024-04-20T11:27:06.433 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.433+0000 7f1ca8c00640 1 -- 172.21.15.152:0/1867463124 shutdown_connections 2024-04-20T11:27:06.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.433+0000 7f1ca8c00640 1 --2- 172.21.15.152:0/1867463124 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f1c9806c920 0x7f1c9806ede0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:06.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.433+0000 7f1ca8c00640 1 --2- 172.21.15.152:0/1867463124 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1cb00e7a40 0x7f1cb004b3c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:06.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.433+0000 7f1ca8c00640 1 --2- 172.21.15.152:0/1867463124 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1cb0054dd0 0x7f1cb0047c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:06.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.433+0000 7f1ca8c00640 1 --2- 172.21.15.152:0/1867463124 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1cb004a090 0x7f1cb00e1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:06.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.433+0000 7f1ca8c00640 1 -- 172.21.15.152:0/1867463124 >> 172.21.15.152:0/1867463124 conn(0x7f1cb0056cc0 msgr2=0x7f1cb00552b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:06.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.434+0000 7f1ca8c00640 1 -- 172.21.15.152:0/1867463124 shutdown_connections 2024-04-20T11:27:06.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.435+0000 7f1ca8c00640 1 -- 172.21.15.152:0/1867463124 wait complete. 2024-04-20T11:27:06.456 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.457+0000 7fc143e00640 1 -- 172.21.15.152:0/2411782992 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fc118005580 con 0x7fc1340e7640 2024-04-20T11:27:06.457 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.457+0000 7fc139600640 1 -- 172.21.15.152:0/2411782992 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc12805c960 con 0x7fc1340e7640 2024-04-20T11:27:06.457 INFO:teuthology.orchestra.run.smithi152.stdout:55834574853 2024-04-20T11:27:06.460 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.1 2024-04-20T11:27:06.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.460+0000 7ff6a1800640 1 -- 172.21.15.152:0/2281423501 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7ff674005580 con 0x7ff69c0caae0 2024-04-20T11:27:06.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.461+0000 7ff693e00640 1 -- 172.21.15.152:0/2281423501 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff69c0c9aa0 con 0x7ff69c0caae0 2024-04-20T11:27:06.461 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.461+0000 7fc143e00640 1 -- 172.21.15.152:0/2411782992 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc11c06cb20 msgr2=0x7fc11c06efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:06.461 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.461+0000 7fc143e00640 1 --2- 172.21.15.152:0/2411782992 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc11c06cb20 0x7fc11c06efe0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc130036000 tx=0x7fc13003b040 comp rx=0 tx=0).stop 2024-04-20T11:27:06.461 INFO:teuthology.orchestra.run.smithi152.stdout:55834574854 2024-04-20T11:27:06.463 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.464+0000 7fc143e00640 1 -- 172.21.15.152:0/2411782992 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc1340e7640 msgr2=0x7fc13404f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:06.463 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.464+0000 7fc143e00640 1 --2- 172.21.15.152:0/2411782992 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc1340e7640 0x7fc13404f4a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc12800c480 tx=0x7fc12800c880 comp rx=0 tx=0).stop 2024-04-20T11:27:06.464 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.464+0000 7fc143e00640 1 -- 172.21.15.152:0/2411782992 shutdown_connections 2024-04-20T11:27:06.464 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.464+0000 7fc143e00640 1 --2- 172.21.15.152:0/2411782992 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc11c06cb20 0x7fc11c06efe0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:06.464 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.464+0000 7fc143e00640 1 --2- 172.21.15.152:0/2411782992 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc1340e7a20 0x7fc13404c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:06.464 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.464+0000 7fc143e00640 1 --2- 172.21.15.152:0/2411782992 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc1340e7640 0x7fc13404f4a0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:06.464 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.464+0000 7fc143e00640 1 --2- 172.21.15.152:0/2411782992 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc134053c00 0x7fc1340e4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:06.464 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.464+0000 7fc143e00640 1 -- 172.21.15.152:0/2411782992 >> 172.21.15.152:0/2411782992 conn(0x7fc13403aa20 msgr2=0x7fc134006d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:06.464 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.465+0000 7fc143e00640 1 -- 172.21.15.152:0/2411782992 shutdown_connections 2024-04-20T11:27:06.465 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.465+0000 7ff6a1800640 1 -- 172.21.15.152:0/2281423501 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff67806c940 msgr2=0x7ff67806ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:06.465 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.465+0000 7ff6a1800640 1 --2- 172.21.15.152:0/2281423501 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff67806c940 0x7ff67806ee00 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff684004f60 tx=0x7ff684007e30 comp rx=0 tx=0).stop 2024-04-20T11:27:06.465 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.465+0000 7fc143e00640 1 -- 172.21.15.152:0/2411782992 wait complete. 2024-04-20T11:27:06.466 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.465+0000 7ff6a1800640 1 -- 172.21.15.152:0/2281423501 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff69c0caae0 msgr2=0x7ff69c1f8140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:06.466 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.465+0000 7ff6a1800640 1 --2- 172.21.15.152:0/2281423501 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff69c0caae0 0x7ff69c1f8140 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff694009e50 tx=0x7ff694004a40 comp rx=0 tx=0).stop 2024-04-20T11:27:06.466 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.465+0000 7ff6a1800640 1 -- 172.21.15.152:0/2281423501 shutdown_connections 2024-04-20T11:27:06.466 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.465+0000 7ff6a1800640 1 --2- 172.21.15.152:0/2281423501 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff67806c940 0x7ff67806ee00 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:06.466 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.465+0000 7ff6a1800640 1 --2- 172.21.15.152:0/2281423501 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff69c0caae0 0x7ff69c1f8140 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:06.466 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.465+0000 7ff6a1800640 1 --2- 172.21.15.152:0/2281423501 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff69c166650 0x7ff69c0b71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:06.466 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.465+0000 7ff6a1800640 1 --2- 172.21.15.152:0/2281423501 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff69c166270 0x7ff69c153470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:06.466 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.465+0000 7ff6a1800640 1 -- 172.21.15.152:0/2281423501 >> 172.21.15.152:0/2281423501 conn(0x7ff69c08bfd0 msgr2=0x7ff69c165a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:06.466 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.466+0000 7ff6a1800640 1 -- 172.21.15.152:0/2281423501 shutdown_connections 2024-04-20T11:27:06.466 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.466+0000 7ff6a1800640 1 -- 172.21.15.152:0/2281423501 wait complete. 2024-04-20T11:27:06.485 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.486+0000 7f6e15000640 1 -- 172.21.15.152:0/361768913 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f6e10069290 con 0x7f6e100cfa00 2024-04-20T11:27:06.485 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.486+0000 7fc263800640 1 -- 172.21.15.152:0/4010114470 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fc25c14b100 con 0x7fc25c06f810 2024-04-20T11:27:06.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.486+0000 7f6e06000640 1 -- 172.21.15.152:0/361768913 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f6dfc058a80 con 0x7f6e100cfa00 2024-04-20T11:27:06.486 INFO:teuthology.orchestra.run.smithi152.stdout:55834574852 2024-04-20T11:27:06.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.487+0000 7fc258c00640 1 -- 172.21.15.152:0/4010114470 <== mon.2 v2:172.21.15.175:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc25005df10 con 0x7fc25c06f810 2024-04-20T11:27:06.487 INFO:teuthology.orchestra.run.smithi152.stdout:55834574853 2024-04-20T11:27:06.489 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.490+0000 7f6e15000640 1 -- 172.21.15.152:0/361768913 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f6df006c940 msgr2=0x7f6df006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:06.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.490+0000 7f6e15000640 1 --2- 172.21.15.152:0/361768913 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f6df006c940 0x7f6df006ee00 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6e08035000 tx=0x7f6e08030000 comp rx=0 tx=0).stop 2024-04-20T11:27:06.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.490+0000 7f6e15000640 1 -- 172.21.15.152:0/361768913 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6e100cfa00 msgr2=0x7f6e100cb930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:06.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.490+0000 7f6e15000640 1 --2- 172.21.15.152:0/361768913 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6e100cfa00 0x7f6e100cb930 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc00c410 tx=0x7f6dfc00c8d0 comp rx=0 tx=0).stop 2024-04-20T11:27:06.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.490+0000 7f1527200640 1 -- 172.21.15.152:0/820000764 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f14f8005850 con 0x7f15200cf5a0 2024-04-20T11:27:06.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.491+0000 7f6e15000640 1 -- 172.21.15.152:0/361768913 shutdown_connections 2024-04-20T11:27:06.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.491+0000 7fc263800640 1 -- 172.21.15.152:0/4010114470 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc23c06cb20 msgr2=0x7fc23c06efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:06.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.491+0000 7fc263800640 1 --2- 172.21.15.152:0/4010114470 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc23c06cb20 0x7fc23c06efe0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc248007fd0 tx=0x7fc248009040 comp rx=0 tx=0).stop 2024-04-20T11:27:06.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.491+0000 7f6e15000640 1 --2- 172.21.15.152:0/361768913 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f6df006c940 0x7f6df006ee00 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:06.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.491+0000 7f6e15000640 1 --2- 172.21.15.152:0/361768913 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e1015ec80 0x7f6e100d3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:06.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.491+0000 7f6e15000640 1 --2- 172.21.15.152:0/361768913 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6e100cfa00 0x7f6e100cb930 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:06.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.491+0000 7f6e15000640 1 --2- 172.21.15.152:0/361768913 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e100cf5a0 0x7f6e10157c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:06.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.491+0000 7f6e15000640 1 -- 172.21.15.152:0/361768913 >> 172.21.15.152:0/361768913 conn(0x7f6e10067ae0 msgr2=0x7f6e1015b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:06.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.491+0000 7fc263800640 1 -- 172.21.15.152:0/4010114470 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc25c06f810 msgr2=0x7fc25c14b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:06.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.491+0000 7fc263800640 1 --2- 172.21.15.152:0/4010114470 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc25c06f810 0x7fc25c14b450 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc25000e8b0 tx=0x7fc25000ed70 comp rx=0 tx=0).stop 2024-04-20T11:27:06.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.491+0000 7fc263800640 1 -- 172.21.15.152:0/4010114470 shutdown_connections 2024-04-20T11:27:06.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.491+0000 7fc263800640 1 --2- 172.21.15.152:0/4010114470 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc23c06cb20 0x7fc23c06efe0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:06.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.491+0000 7fc263800640 1 --2- 172.21.15.152:0/4010114470 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc25c162b40 0x7fc25c1616d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:06.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.491+0000 7fc263800640 1 --2- 172.21.15.152:0/4010114470 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc25c0cfff0 0x7fc25c144f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:06.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.491+0000 7fc263800640 1 --2- 172.21.15.152:0/4010114470 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc25c06f810 0x7fc25c14b450 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:06.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.491+0000 7fc263800640 1 -- 172.21.15.152:0/4010114470 >> 172.21.15.152:0/4010114470 conn(0x7fc25c067cf0 msgr2=0x7fc25c0b08a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:06.493 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.0 2024-04-20T11:27:06.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.491+0000 7fc263800640 1 -- 172.21.15.152:0/4010114470 shutdown_connections 2024-04-20T11:27:06.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.491+0000 7f1517e00640 1 -- 172.21.15.152:0/820000764 <== mon.2 v2:172.21.15.175:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f15100594d0 con 0x7f15200cf5a0 2024-04-20T11:27:06.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.491+0000 7fc263800640 1 -- 172.21.15.152:0/4010114470 wait complete. 2024-04-20T11:27:06.494 INFO:teuthology.orchestra.run.smithi152.stdout:55834574852 2024-04-20T11:27:06.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.492+0000 7f6e15000640 1 -- 172.21.15.152:0/361768913 shutdown_connections 2024-04-20T11:27:06.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.492+0000 7f6e15000640 1 -- 172.21.15.152:0/361768913 wait complete. 2024-04-20T11:27:06.495 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.2 2024-04-20T11:27:06.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.493+0000 7f0c42c00640 1 -- 172.21.15.152:0/464555806 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f0c3c0689f0 con 0x7f0c3c06fea0 2024-04-20T11:27:06.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.493+0000 7f0c33e00640 1 -- 172.21.15.152:0/464555806 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f0c2c05a8d0 con 0x7f0c3c06fea0 2024-04-20T11:27:06.496 INFO:teuthology.orchestra.run.smithi152.stdout:55834574855 2024-04-20T11:27:06.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.494+0000 7f1527200640 1 -- 172.21.15.152:0/820000764 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f150006c8f0 msgr2=0x7f150006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:06.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.494+0000 7f1527200640 1 --2- 172.21.15.152:0/820000764 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f150006c8f0 0x7f150006edb0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f150c02b860 tx=0x7f150c03d040 comp rx=0 tx=0).stop 2024-04-20T11:27:06.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.494+0000 7f1527200640 1 -- 172.21.15.152:0/820000764 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f15200cf5a0 msgr2=0x7f15200ca990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:06.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.494+0000 7f1527200640 1 --2- 172.21.15.152:0/820000764 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f15200cf5a0 0x7f15200ca990 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1510006fd0 tx=0x7f151000ec70 comp rx=0 tx=0).stop 2024-04-20T11:27:06.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.494+0000 7f1527200640 1 -- 172.21.15.152:0/820000764 shutdown_connections 2024-04-20T11:27:06.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.494+0000 7f1527200640 1 --2- 172.21.15.152:0/820000764 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f150006c8f0 0x7f150006edb0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:06.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.494+0000 7f1527200640 1 --2- 172.21.15.152:0/820000764 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f152015ec80 0x7f15200cd4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:06.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.494+0000 7f1527200640 1 --2- 172.21.15.152:0/820000764 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f15200cf5a0 0x7f15200ca990 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:06.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.494+0000 7f1527200640 1 --2- 172.21.15.152:0/820000764 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f15200b7990 0x7f1520151cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:06.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.494+0000 7f1527200640 1 -- 172.21.15.152:0/820000764 >> 172.21.15.152:0/820000764 conn(0x7f1520067ae0 msgr2=0x7f15200b7de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:06.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.497+0000 7f1527200640 1 -- 172.21.15.152:0/820000764 shutdown_connections 2024-04-20T11:27:06.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.497+0000 7f1527200640 1 -- 172.21.15.152:0/820000764 wait complete. 2024-04-20T11:27:06.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.497+0000 7f0c42c00640 1 -- 172.21.15.152:0/464555806 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f0c1c06cb20 msgr2=0x7f0c1c06efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:06.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.497+0000 7f0c42c00640 1 --2- 172.21.15.152:0/464555806 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f0c1c06cb20 0x7f0c1c06efe0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c1fc380 tx=0x7f0c28007e80 comp rx=0 tx=0).stop 2024-04-20T11:27:06.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.497+0000 7f0c42c00640 1 -- 172.21.15.152:0/464555806 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0c3c06fea0 msgr2=0x7f0c3c158330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:06.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.497+0000 7f0c42c00640 1 --2- 172.21.15.152:0/464555806 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0c3c06fea0 0x7f0c3c158330 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c00d7d0 tx=0x7f0c2c00dc90 comp rx=0 tx=0).stop 2024-04-20T11:27:06.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.497+0000 7f0c42c00640 1 -- 172.21.15.152:0/464555806 shutdown_connections 2024-04-20T11:27:06.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.497+0000 7f0c42c00640 1 --2- 172.21.15.152:0/464555806 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f0c1c06cb20 0x7f0c1c06efe0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:06.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.497+0000 7f0c42c00640 1 --2- 172.21.15.152:0/464555806 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c3c15a860 0x7f0c3c145f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:06.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.497+0000 7f0c42c00640 1 --2- 172.21.15.152:0/464555806 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0c3c071db0 0x7f0c3c145a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:06.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.497+0000 7f0c42c00640 1 --2- 172.21.15.152:0/464555806 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0c3c06fea0 0x7f0c3c158330 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:06.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.497+0000 7f0c42c00640 1 -- 172.21.15.152:0/464555806 >> 172.21.15.152:0/464555806 conn(0x7f0c3c067cf0 msgr2=0x7f0c3c0b12a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:06.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.498+0000 7f0c42c00640 1 -- 172.21.15.152:0/464555806 shutdown_connections 2024-04-20T11:27:06.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.498+0000 7fa60f400640 1 -- 172.21.15.152:0/3211252062 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fa5e4005850 con 0x7fa60806fa50 2024-04-20T11:27:06.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.498+0000 7f0c42c00640 1 -- 172.21.15.152:0/464555806 wait complete. 2024-04-20T11:27:06.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.498+0000 7fa604c00640 1 -- 172.21.15.152:0/3211252062 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa5fc059fa0 con 0x7fa60806fa50 2024-04-20T11:27:06.500 INFO:teuthology.orchestra.run.smithi152.stdout:55834574854 2024-04-20T11:27:06.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.501+0000 7fa60f400640 1 -- 172.21.15.152:0/3211252062 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fa5e806c9c0 msgr2=0x7fa5e806ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:06.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.501+0000 7fa60f400640 1 --2- 172.21.15.152:0/3211252062 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fa5e806c9c0 0x7fa5e806ee80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa5f4036000 tx=0x7fa5f403b040 comp rx=0 tx=0).stop 2024-04-20T11:27:06.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.501+0000 7fa60f400640 1 -- 172.21.15.152:0/3211252062 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa60806fa50 msgr2=0x7fa6080d0510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:06.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.501+0000 7fa60f400640 1 --2- 172.21.15.152:0/3211252062 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa60806fa50 0x7fa6080d0510 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc00c530 tx=0x7fa5fc00c9f0 comp rx=0 tx=0).stop 2024-04-20T11:27:06.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.502+0000 7fa60f400640 1 -- 172.21.15.152:0/3211252062 shutdown_connections 2024-04-20T11:27:06.502 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.502+0000 7fa60f400640 1 --2- 172.21.15.152:0/3211252062 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fa5e806c9c0 0x7fa5e806ee80 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:06.502 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.502+0000 7fa60f400640 1 --2- 172.21.15.152:0/3211252062 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa60815fd60 0x7fa6081f8d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:06.502 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.502+0000 7fa60f400640 1 --2- 172.21.15.152:0/3211252062 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa60815b2b0 0x7fa6081f8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:06.502 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.502+0000 7fa60f400640 1 --2- 172.21.15.152:0/3211252062 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa60806fa50 0x7fa6080d0510 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:06.502 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.502+0000 7fa60f400640 1 -- 172.21.15.152:0/3211252062 >> 172.21.15.152:0/3211252062 conn(0x7fa608067cf0 msgr2=0x7fa60806e720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:06.502 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.502+0000 7fa60f400640 1 -- 172.21.15.152:0/3211252062 shutdown_connections 2024-04-20T11:27:06.502 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:06.502+0000 7fa60f400640 1 -- 172.21.15.152:0/3211252062 wait complete. 2024-04-20T11:27:06.517 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.4 2024-04-20T11:27:06.518 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.3 2024-04-20T11:27:06.518 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.6 2024-04-20T11:27:06.523 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.5 2024-04-20T11:27:06.524 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.7 2024-04-20T11:27:07.461 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-20T11:27:07.495 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-20T11:27:07.496 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-20T11:27:07.517 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-20T11:27:07.518 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-20T11:27:07.519 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-20T11:27:07.524 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-20T11:27:07.524 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-20T11:27:07.713 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.713+0000 7ffa1e600640 1 Processor -- start 2024-04-20T11:27:07.713 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.713+0000 7ffa1e600640 1 -- start start 2024-04-20T11:27:07.715 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.716+0000 7ffa1e600640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7ffa1815ec80 0x7ffa180ba180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:07.718 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.717+0000 7ffa1e600640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa180b8cf0 con 0x7ffa180cfa00 2024-04-20T11:27:07.718 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.717+0000 7ffa16a00640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7ffa180cf5a0 0x7ffa180b7990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.152:49486/0 (socket says 172.21.15.152:49486) 2024-04-20T11:27:07.718 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.717+0000 7ffa16a00640 1 -- 172.21.15.152:0/721461364 learned_addr learned my addr 172.21.15.152:0/721461364 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:27:07.718 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.718+0000 7ffa1e600640 1 -- 172.21.15.152:0/721461364 --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa180b8fd0 con 0x7ffa180cf5a0 2024-04-20T11:27:07.718 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.718+0000 7ffa1e600640 1 -- 172.21.15.152:0/721461364 --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7ffa180b9d90 con 0x7ffa1815ec80 2024-04-20T11:27:07.719 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.719+0000 7ffa17400640 1 --2- 172.21.15.152:0/721461364 >> v2:172.21.15.152:3300/0 conn(0x7ffa1815ec80 0x7ffa180ba180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:07.719 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.720+0000 7ffa17400640 1 -- 172.21.15.152:0/721461364 >> v1:172.21.15.175:6789/0 conn(0x7ffa180cf5a0 legacy=0x7ffa180b7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:07.719 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.720+0000 7ffa16000640 1 -- 172.21.15.152:0/721461364 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3523778840 0 0) 0x7ffa180b8fd0 con 0x7ffa180cf5a0 2024-04-20T11:27:07.721 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.720+0000 7ffa17400640 1 -- 172.21.15.152:0/721461364 >> v1:172.21.15.152:6789/0 conn(0x7ffa180cfa00 legacy=0x7ffa180b80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:07.721 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.720+0000 7ffa17400640 1 -- 172.21.15.152:0/721461364 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa180b92b0 con 0x7ffa1815ec80 2024-04-20T11:27:07.722 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.722+0000 7ffa17400640 1 --2- 172.21.15.152:0/721461364 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffa1815ec80 0x7ffa180ba180 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ffa040097a0 tx=0x7ffa0402b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b8f2e995c3cb4da server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:07.722 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.723+0000 7ffa16000640 1 -- 172.21.15.152:0/721461364 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa04037070 con 0x7ffa1815ec80 2024-04-20T11:27:07.722 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.723+0000 7ffa16000640 1 -- 172.21.15.152:0/721461364 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa04030710 con 0x7ffa1815ec80 2024-04-20T11:27:07.723 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.723+0000 7ffa16000640 1 -- 172.21.15.152:0/721461364 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa04030ae0 con 0x7ffa1815ec80 2024-04-20T11:27:07.726 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.725+0000 7ffa1e600640 1 -- 172.21.15.152:0/721461364 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffa1815ec80 msgr2=0x7ffa180ba180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:07.726 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.725+0000 7ffa1e600640 1 --2- 172.21.15.152:0/721461364 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffa1815ec80 0x7ffa180ba180 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ffa040097a0 tx=0x7ffa0402b790 comp rx=0 tx=0).stop 2024-04-20T11:27:07.726 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.726+0000 7ffa1e600640 1 -- 172.21.15.152:0/721461364 shutdown_connections 2024-04-20T11:27:07.726 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.726+0000 7ffa1e600640 1 --2- 172.21.15.152:0/721461364 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffa1815ec80 0x7ffa180ba180 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:07.726 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.726+0000 7ffa1e600640 1 -- 172.21.15.152:0/721461364 >> 172.21.15.152:0/721461364 conn(0x7ffa18067ae0 msgr2=0x7ffa18067ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:07.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.727+0000 7ffa1e600640 1 -- 172.21.15.152:0/721461364 shutdown_connections 2024-04-20T11:27:07.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.727+0000 7ffa1e600640 1 -- 172.21.15.152:0/721461364 wait complete. 2024-04-20T11:27:07.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.729+0000 7ffa1e600640 1 Processor -- start 2024-04-20T11:27:07.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.730+0000 7ffa1e600640 1 -- start start 2024-04-20T11:27:07.731 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.732+0000 7ffa1e600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa180cf5a0 0x7ffa180cd6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:07.731 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.732+0000 7ffa1e600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffa180cfa00 0x7ffa182044e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:07.732 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.733+0000 7ffa17400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa180cf5a0 0x7ffa180cd6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:07.732 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.733+0000 7ffa17400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa180cf5a0 0x7ffa180cd6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.152:43406/0 (socket says 172.21.15.152:43406) 2024-04-20T11:27:07.733 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.733+0000 7ffa17400640 1 -- 172.21.15.152:0/1376285398 learned_addr learned my addr 172.21.15.152:0/1376285398 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:07.733 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.733+0000 7ffa1e600640 1 --2- 172.21.15.152:0/1376285398 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa1815ec80 0x7ffa1815a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:07.733 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.734+0000 7ffa1e600640 1 -- 172.21.15.152:0/1376285398 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7ffa180cfe90 con 0x7ffa180cfa00 2024-04-20T11:27:07.733 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.734+0000 7ffa1e600640 1 -- 172.21.15.152:0/1376285398 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ffa180b9900 con 0x7ffa180cf5a0 2024-04-20T11:27:07.733 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.734+0000 7ffa17e00640 1 --2- 172.21.15.152:0/1376285398 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa1815ec80 0x7ffa1815a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:07.734 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.734+0000 7ffa16a00640 1 --2- 172.21.15.152:0/1376285398 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffa180cfa00 0x7ffa182044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:07.734 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.734+0000 7ffa1e600640 1 -- 172.21.15.152:0/1376285398 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ffa1815f060 con 0x7ffa1815ec80 2024-04-20T11:27:07.735 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.736+0000 7ffa16a00640 1 -- 172.21.15.152:0/1376285398 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa1815ec80 msgr2=0x7ffa1815a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:07.735 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.736+0000 7ffa16a00640 1 --2- 172.21.15.152:0/1376285398 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa1815ec80 0x7ffa1815a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:07.735 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.736+0000 7ffa16a00640 1 -- 172.21.15.152:0/1376285398 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa180cf5a0 msgr2=0x7ffa180cd6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:07.735 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.736+0000 7ffa16a00640 1 --2- 172.21.15.152:0/1376285398 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa180cf5a0 0x7ffa180cd6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:07.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.736+0000 7ffa16a00640 1 -- 172.21.15.152:0/1376285398 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa1815ab40 con 0x7ffa180cfa00 2024-04-20T11:27:07.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.736+0000 7ffa17e00640 1 --2- 172.21.15.152:0/1376285398 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa1815ec80 0x7ffa1815a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:27:07.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.737+0000 7ffa16a00640 1 --2- 172.21.15.152:0/1376285398 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffa180cfa00 0x7ffa182044e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ffa10000c00 tx=0x7ffa1000cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:07.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.738+0000 7ffa0f400640 1 -- 172.21.15.152:0/1376285398 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa10018070 con 0x7ffa180cfa00 2024-04-20T11:27:07.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.738+0000 7ffa0f400640 1 -- 172.21.15.152:0/1376285398 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa100070d0 con 0x7ffa180cfa00 2024-04-20T11:27:07.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.738+0000 7ffa0f400640 1 -- 172.21.15.152:0/1376285398 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa100074e0 con 0x7ffa180cfa00 2024-04-20T11:27:07.739 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.739+0000 7ffa1e600640 1 -- 172.21.15.152:0/1376285398 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa1815ae20 con 0x7ffa180cfa00 2024-04-20T11:27:07.739 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.739+0000 7ffa1e600640 1 -- 172.21.15.152:0/1376285398 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa1815b3e0 con 0x7ffa180cfa00 2024-04-20T11:27:07.743 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.743+0000 7ffa0f400640 1 -- 172.21.15.152:0/1376285398 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ffa1001b070 con 0x7ffa180cfa00 2024-04-20T11:27:07.745 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.744+0000 7ffa1e600640 1 -- 172.21.15.152:0/1376285398 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa180689f0 con 0x7ffa180cfa00 2024-04-20T11:27:07.745 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.745+0000 7ffa0f400640 1 --2- 172.21.15.152:0/1376285398 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff9f806c9c0 0x7ff9f806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:07.745 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.745+0000 7ffa0f400640 1 -- 172.21.15.152:0/1376285398 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ffa1008ae60 con 0x7ffa180cfa00 2024-04-20T11:27:07.749 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.750+0000 7ffa17400640 1 --2- 172.21.15.152:0/1376285398 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff9f806c9c0 0x7ff9f806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:07.750 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.751+0000 7ffa17400640 1 --2- 172.21.15.152:0/1376285398 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff9f806c9c0 0x7ff9f806ee80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ffa040097a0 tx=0x7ffa0402f000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:07.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.757+0000 7ffa0f400640 1 -- 172.21.15.152:0/1376285398 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ffa10058d70 con 0x7ffa180cfa00 2024-04-20T11:27:07.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.758+0000 7fd255e00640 1 Processor -- start 2024-04-20T11:27:07.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.757+0000 7f13bfe00640 1 Processor -- start 2024-04-20T11:27:07.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.758+0000 7f13bfe00640 1 -- start start 2024-04-20T11:27:07.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.759+0000 7f13bfe00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f13c01663f0 0x7f13c00b26e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:07.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.760+0000 7f13bfe00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f13c01674e0 0x7f13c00cf210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:07.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.760+0000 7f13bea00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f13c01663f0 0x7f13c00b26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:07.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.760+0000 7f13bea00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f13c01663f0 0x7f13c00b26e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:48080/0 (socket says 172.21.15.152:48080) 2024-04-20T11:27:07.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.760+0000 7f13bea00640 1 -- 172.21.15.152:0/2186369882 learned_addr learned my addr 172.21.15.152:0/2186369882 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:07.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.761+0000 7f13bfe00640 1 -- 172.21.15.152:0/2186369882 --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13c00730f0 con 0x7f13c00cf750 2024-04-20T11:27:07.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.761+0000 7f13bfe00640 1 -- 172.21.15.152:0/2186369882 --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f13c0073d00 con 0x7f13c01663f0 2024-04-20T11:27:07.762 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.761+0000 7f13bfe00640 1 -- 172.21.15.152:0/2186369882 --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f13c00733d0 con 0x7f13c01674e0 2024-04-20T11:27:07.762 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.761+0000 7f13be000640 1 --2- 172.21.15.152:0/2186369882 >> v2:172.21.15.175:3300/0 conn(0x7f13c01674e0 0x7f13c00cf210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:07.762 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.762+0000 7f13be000640 1 -- 172.21.15.152:0/2186369882 >> v2:172.21.15.152:3300/0 conn(0x7f13c01663f0 msgr2=0x7f13c00b26e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:07.762 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.762+0000 7f13be000640 1 --2- 172.21.15.152:0/2186369882 >> v2:172.21.15.152:3300/0 conn(0x7f13c01663f0 0x7f13c00b26e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:07.762 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.762+0000 7f13be000640 1 -- 172.21.15.152:0/2186369882 >> v1:172.21.15.175:6789/0 conn(0x7f13c00cf750 legacy=0x7f13c00b0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:07.762 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.762+0000 7f13be000640 1 -- 172.21.15.152:0/2186369882 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13c00736b0 con 0x7f13c01674e0 2024-04-20T11:27:07.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.762+0000 7fd255e00640 1 -- start start 2024-04-20T11:27:07.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.764+0000 7fd255e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd2480ead00 0x7fd24804fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:07.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.764+0000 7fd255e00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd248053720 con 0x7fd248006ea0 2024-04-20T11:27:07.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.764+0000 7fd255e00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd248053a00 con 0x7fd24804ffd0 2024-04-20T11:27:07.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.764+0000 7fd255e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd248054330 con 0x7fd2480ead00 2024-04-20T11:27:07.764 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.764+0000 7f13be000640 1 --2- 172.21.15.152:0/2186369882 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f13c01674e0 0x7f13c00cf210 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f13c00d12b0 tx=0x7f13b002b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=79129dae8613a460 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:07.764 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.764+0000 7f13bd600640 1 -- 172.21.15.152:0/2186369882 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13b0035020 con 0x7f13c01674e0 2024-04-20T11:27:07.764 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.765+0000 7fd254a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd2480ead00 0x7fd24804fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:07.765 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.765+0000 7fd254a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd2480ead00 0x7fd24804fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.152:34124/0 (socket says 172.21.15.152:34124) 2024-04-20T11:27:07.765 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.765+0000 7fd254a00640 1 -- 172.21.15.152:0/2084172371 learned_addr learned my addr 172.21.15.152:0/2084172371 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:07.765 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.765+0000 7fd255400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fd248006ea0 0x7fd2480382e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:48506/0 (socket says 172.21.15.152:48506) 2024-04-20T11:27:07.765 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.766+0000 7fd254a00640 1 -- 172.21.15.152:0/2084172371 >> v1:172.21.15.175:6789/0 conn(0x7fd24804ffd0 legacy=0x7fd248037bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:07.765 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.766+0000 7fd254a00640 1 -- 172.21.15.152:0/2084172371 >> v1:172.21.15.152:6789/0 conn(0x7fd248006ea0 legacy=0x7fd2480382e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:07.765 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.766+0000 7fd254a00640 1 -- 172.21.15.152:0/2084172371 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd248053ce0 con 0x7fd2480ead00 2024-04-20T11:27:07.766 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.765+0000 7f13bd600640 1 -- 172.21.15.152:0/2186369882 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13b0031830 con 0x7f13c01674e0 2024-04-20T11:27:07.766 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.765+0000 7f13bd600640 1 -- 172.21.15.152:0/2186369882 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13b0031c00 con 0x7f13c01674e0 2024-04-20T11:27:07.766 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.766+0000 7f13bfe00640 1 -- 172.21.15.152:0/2186369882 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f13c01674e0 msgr2=0x7f13c00cf210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:07.766 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.766+0000 7f13bfe00640 1 --2- 172.21.15.152:0/2186369882 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f13c01674e0 0x7f13c00cf210 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f13c00d12b0 tx=0x7f13b002b8d0 comp rx=0 tx=0).stop 2024-04-20T11:27:07.766 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.766+0000 7f13bfe00640 1 -- 172.21.15.152:0/2186369882 shutdown_connections 2024-04-20T11:27:07.767 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.766+0000 7f13bfe00640 1 --2- 172.21.15.152:0/2186369882 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f13c01674e0 0x7f13c00cf210 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:07.767 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.767+0000 7f13bfe00640 1 --2- 172.21.15.152:0/2186369882 >> v2:172.21.15.152:3300/0 conn(0x7f13c01663f0 0x7f13c00b26e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:07.767 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.767+0000 7f13bfe00640 1 -- 172.21.15.152:0/2186369882 >> 172.21.15.152:0/2186369882 conn(0x7f13c00b5a10 msgr2=0x7f13c00b1fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:07.767 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.767+0000 7f13bfe00640 1 -- 172.21.15.152:0/2186369882 shutdown_connections 2024-04-20T11:27:07.767 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.767+0000 7f13bfe00640 1 -- 172.21.15.152:0/2186369882 wait complete. 2024-04-20T11:27:07.767 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.768+0000 7fd254a00640 1 --2- 172.21.15.152:0/2084172371 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd2480ead00 0x7fd24804fa90 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd2440098a0 tx=0x7fd24402ba10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4c3714ec37dfb0a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:07.767 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.768+0000 7fd24f400640 1 -- 172.21.15.152:0/2084172371 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd24402f830 con 0x7fd2480ead00 2024-04-20T11:27:07.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.768+0000 7fd24f400640 1 -- 172.21.15.152:0/2084172371 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd24402f9d0 con 0x7fd2480ead00 2024-04-20T11:27:07.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.768+0000 7fd24f400640 1 -- 172.21.15.152:0/2084172371 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd24402fda0 con 0x7fd2480ead00 2024-04-20T11:27:07.769 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.770+0000 7f13bfe00640 1 Processor -- start 2024-04-20T11:27:07.770 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.771+0000 7f13bfe00640 1 -- start start 2024-04-20T11:27:07.771 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.772+0000 7fd255e00640 1 -- 172.21.15.152:0/2084172371 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd2480ead00 msgr2=0x7fd24804fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:07.771 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.772+0000 7fd255e00640 1 --2- 172.21.15.152:0/2084172371 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd2480ead00 0x7fd24804fa90 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd2440098a0 tx=0x7fd24402ba10 comp rx=0 tx=0).stop 2024-04-20T11:27:07.772 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.772+0000 7fd255e00640 1 -- 172.21.15.152:0/2084172371 shutdown_connections 2024-04-20T11:27:07.772 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.772+0000 7fd255e00640 1 --2- 172.21.15.152:0/2084172371 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd2480ead00 0x7fd24804fa90 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:07.772 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.772+0000 7fd255e00640 1 -- 172.21.15.152:0/2084172371 >> 172.21.15.152:0/2084172371 conn(0x7fd248009fb0 msgr2=0x7fd24800a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:07.772 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.773+0000 7f13bfe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f13c00cf750 0x7f13c0161980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:07.772 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.773+0000 7f13bfe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f13c01663f0 0x7f13c00b4a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:07.773 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.773+0000 7f13bfe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f13c01674e0 0x7f13c009b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:07.773 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.773+0000 7fd255e00640 1 -- 172.21.15.152:0/2084172371 shutdown_connections 2024-04-20T11:27:07.773 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.773+0000 7fd255e00640 1 -- 172.21.15.152:0/2084172371 wait complete. 2024-04-20T11:27:07.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.774+0000 7f13bfe00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f13c01667d0 con 0x7f13c00cf750 2024-04-20T11:27:07.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.774+0000 7f13bfe00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f13c0167950 con 0x7f13c01663f0 2024-04-20T11:27:07.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.774+0000 7f13bfe00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f13c00cfba0 con 0x7f13c01674e0 2024-04-20T11:27:07.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.774+0000 7f13be000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f13c01663f0 0x7f13c00b4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:07.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.774+0000 7f13bf400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f13c01674e0 0x7f13c009b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:07.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.774+0000 7f13bf400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f13c01674e0 0x7f13c009b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.152:34140/0 (socket says 172.21.15.152:34140) 2024-04-20T11:27:07.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.774+0000 7f13bf400640 1 -- 172.21.15.152:0/3323557946 learned_addr learned my addr 172.21.15.152:0/3323557946 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:07.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.775+0000 7f13bf400640 1 -- 172.21.15.152:0/3323557946 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f13c01663f0 msgr2=0x7f13c00b4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:07.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.775+0000 7f13bf400640 1 --2- 172.21.15.152:0/3323557946 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f13c01663f0 0x7f13c00b4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:07.776 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.775+0000 7f13bf400640 1 -- 172.21.15.152:0/3323557946 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f13c00cf750 msgr2=0x7f13c0161980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:07.776 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.775+0000 7f13bf400640 1 --2- 172.21.15.152:0/3323557946 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f13c00cf750 0x7f13c0161980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:07.776 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.775+0000 7f13bf400640 1 -- 172.21.15.152:0/3323557946 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13c009a3c0 con 0x7f13c01674e0 2024-04-20T11:27:07.776 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.777+0000 7fd255e00640 1 Processor -- start 2024-04-20T11:27:07.776 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.777+0000 7fd255e00640 1 -- start start 2024-04-20T11:27:07.779 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.779+0000 7fd255e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd248006ea0 0x7fd24804e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:07.779 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.779+0000 7fd255e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd24804ffd0 0x7fd248034d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:07.779 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.779+0000 7fd255e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2480ead00 0x7fd2480d7b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:07.779 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.779+0000 7fd255e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fd248054940 con 0x7fd248006ea0 2024-04-20T11:27:07.779 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.779+0000 7fd255e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd2480547c0 con 0x7fd2480ead00 2024-04-20T11:27:07.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.779+0000 7fd255e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd24804f550 con 0x7fd24804ffd0 2024-04-20T11:27:07.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.779+0000 7fd24fe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd24804ffd0 0x7fd248034d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:07.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.780+0000 7fd24fe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd24804ffd0 0x7fd248034d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.152:34142/0 (socket says 172.21.15.152:34142) 2024-04-20T11:27:07.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.780+0000 7fd24fe00640 1 -- 172.21.15.152:0/2981202700 learned_addr learned my addr 172.21.15.152:0/2981202700 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:07.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.780+0000 7fd255400640 1 --2- 172.21.15.152:0/2981202700 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2480ead00 0x7fd2480d7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:07.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.780+0000 7fd24fe00640 1 -- 172.21.15.152:0/2981202700 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2480ead00 msgr2=0x7fd2480d7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:07.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.781+0000 7fd254a00640 1 --2- 172.21.15.152:0/2981202700 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd248006ea0 0x7fd24804e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:07.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.775+0000 7f13be000640 1 --2- 172.21.15.152:0/3323557946 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f13c01663f0 0x7f13c00b4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:27:07.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.776+0000 7f13bf400640 1 --2- 172.21.15.152:0/3323557946 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f13c01674e0 0x7f13c009b700 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f13b800c730 tx=0x7f13b800cb30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:07.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.776+0000 7f13b6a00640 1 -- 172.21.15.152:0/3323557946 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13b8016020 con 0x7f13c01674e0 2024-04-20T11:27:07.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.776+0000 7f13b6a00640 1 -- 172.21.15.152:0/3323557946 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13b8006d50 con 0x7f13c01674e0 2024-04-20T11:27:07.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.776+0000 7f13b6a00640 1 -- 172.21.15.152:0/3323557946 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13b8003b70 con 0x7f13c01674e0 2024-04-20T11:27:07.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.776+0000 7f13bfe00640 1 -- 172.21.15.152:0/3323557946 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13c009a660 con 0x7f13c01674e0 2024-04-20T11:27:07.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.776+0000 7f13bfe00640 1 -- 172.21.15.152:0/3323557946 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f13c009af00 con 0x7f13c01674e0 2024-04-20T11:27:07.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.781+0000 7fd24fe00640 1 --2- 172.21.15.152:0/2981202700 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2480ead00 0x7fd2480d7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:07.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.781+0000 7fd24fe00640 1 -- 172.21.15.152:0/2981202700 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd248006ea0 msgr2=0x7fd24804e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:07.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.781+0000 7fd24fe00640 1 --2- 172.21.15.152:0/2981202700 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd248006ea0 0x7fd24804e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:07.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.781+0000 7fd24fe00640 1 -- 172.21.15.152:0/2981202700 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2480352c0 con 0x7fd24804ffd0 2024-04-20T11:27:07.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.781+0000 7fd254a00640 1 --2- 172.21.15.152:0/2981202700 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd248006ea0 0x7fd24804e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:27:07.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.781+0000 7fd255400640 1 --2- 172.21.15.152:0/2981202700 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2480ead00 0x7fd2480d7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:27:07.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.782+0000 7fd24fe00640 1 --2- 172.21.15.152:0/2981202700 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd24804ffd0 0x7fd248034d80 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd23c00c450 tx=0x7fd23c00c850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:07.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.781+0000 7f13b6a00640 1 -- 172.21.15.152:0/3323557946 <== mon.2 v2:172.21.15.175:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f13b8003d10 con 0x7f13c01674e0 2024-04-20T11:27:07.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.782+0000 7f13bfe00640 1 -- 172.21.15.152:0/3323557946 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1398005290 con 0x7f13c01674e0 2024-04-20T11:27:07.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.783+0000 7fd24cc00640 1 -- 172.21.15.152:0/2981202700 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd23c017070 con 0x7fd24804ffd0 2024-04-20T11:27:07.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.783+0000 7fd24cc00640 1 -- 172.21.15.152:0/2981202700 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd23c0052d0 con 0x7fd24804ffd0 2024-04-20T11:27:07.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.783+0000 7fd24cc00640 1 -- 172.21.15.152:0/2981202700 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd23c003830 con 0x7fd24804ffd0 2024-04-20T11:27:07.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.783+0000 7fd255e00640 1 -- 172.21.15.152:0/2981202700 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd24804d1f0 con 0x7fd24804ffd0 2024-04-20T11:27:07.784 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.783+0000 7f13b6a00640 1 --2- 172.21.15.152:0/3323557946 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f13a006c9f0 0x7f13a006eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:07.784 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.783+0000 7f13b6a00640 1 -- 172.21.15.152:0/3323557946 <== mon.2 v2:172.21.15.175:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f13b808b660 con 0x7f13c01674e0 2024-04-20T11:27:07.784 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.784+0000 7fd255e00640 1 -- 172.21.15.152:0/2981202700 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd24804d7b0 con 0x7fd24804ffd0 2024-04-20T11:27:07.784 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.784+0000 7f13bea00640 1 --2- 172.21.15.152:0/3323557946 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f13a006c9f0 0x7f13a006eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:07.785 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.785+0000 7f13bea00640 1 --2- 172.21.15.152:0/3323557946 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f13a006c9f0 0x7f13a006eeb0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f13ac006f40 tx=0x7f13ac006e90 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:07.785 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.786+0000 7fd242000640 1 -- 172.21.15.152:0/2981202700 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd224005290 con 0x7fd24804ffd0 2024-04-20T11:27:07.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.787+0000 7fd24cc00640 1 -- 172.21.15.152:0/2981202700 <== mon.2 v2:172.21.15.175:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd23c01a070 con 0x7fd24804ffd0 2024-04-20T11:27:07.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.789+0000 7fd24cc00640 1 --2- 172.21.15.152:0/2981202700 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fd23006c940 0x7fd23006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:07.789 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.789+0000 7fd254a00640 1 --2- 172.21.15.152:0/2981202700 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fd23006c940 0x7fd23006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:07.790 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.791+0000 7fd254a00640 1 --2- 172.21.15.152:0/2981202700 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fd23006c940 0x7fd23006ee00 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd244036000 tx=0x7fd244039040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:07.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.800+0000 7fd24cc00640 1 -- 172.21.15.152:0/2981202700 <== mon.2 v2:172.21.15.175:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd23c056cb0 con 0x7fd24804ffd0 2024-04-20T11:27:07.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.800+0000 7f13b6a00640 1 -- 172.21.15.152:0/3323557946 <== mon.2 v2:172.21.15.175:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f13b8020020 con 0x7f13c01674e0 2024-04-20T11:27:07.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.815+0000 7fd24cc00640 1 -- 172.21.15.152:0/2981202700 <== mon.2 v2:172.21.15.175:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fd23c059310 con 0x7fd24804ffd0 2024-04-20T11:27:07.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.833+0000 7fe86fa00640 1 Processor -- start 2024-04-20T11:27:07.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.833+0000 7fe86fa00640 1 -- start start 2024-04-20T11:27:07.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.835+0000 7fe86fa00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fe86815e830 0x7fe86806de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:07.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.835+0000 7fe86fa00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe86806f060 0x7fe86806f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:07.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.835+0000 7fe86fa00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8680afef0 con 0x7fe8680b3e80 2024-04-20T11:27:07.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.835+0000 7fe86fa00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fe86806e5c0 con 0x7fe86815e830 2024-04-20T11:27:07.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.835+0000 7fe86fa00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe86815ec10 con 0x7fe86806f060 2024-04-20T11:27:07.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.836+0000 7fe86d000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fe86815e830 0x7fe86806de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:07.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.836+0000 7fe86d000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fe86815e830 0x7fe86806de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:48106/0 (socket says 172.21.15.152:48106) 2024-04-20T11:27:07.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.836+0000 7fe86d000640 1 -- 172.21.15.152:0/3089408823 learned_addr learned my addr 172.21.15.152:0/3089408823 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:07.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.836+0000 7fe867e00640 1 --2- 172.21.15.152:0/3089408823 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe86806f060 0x7fe86806f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:07.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.837+0000 7fe867e00640 1 -- 172.21.15.152:0/3089408823 >> v2:172.21.15.152:3300/0 conn(0x7fe86815e830 msgr2=0x7fe86806de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:07.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.837+0000 7fe867e00640 1 --2- 172.21.15.152:0/3089408823 >> v2:172.21.15.152:3300/0 conn(0x7fe86815e830 0x7fe86806de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:07.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.837+0000 7fe867e00640 1 -- 172.21.15.152:0/3089408823 >> v1:172.21.15.175:6789/0 conn(0x7fe8680b3e80 legacy=0x7fe868071db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:07.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.837+0000 7fe867e00640 1 -- 172.21.15.152:0/3089408823 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8680afc10 con 0x7fe86806f060 2024-04-20T11:27:07.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.839+0000 7f003d800640 1 Processor -- start 2024-04-20T11:27:07.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.839+0000 7fe86d000640 1 --2- 172.21.15.152:0/3089408823 >> v2:172.21.15.152:3300/0 conn(0x7fe86815e830 0x7fe86806de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:27:07.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.839+0000 7fe867e00640 1 --2- 172.21.15.152:0/3089408823 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe86806f060 0x7fe86806f810 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe860009d90 tx=0x7fe86002be50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ba660a9218f0b83 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:07.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.839+0000 7f4f9c400640 1 Processor -- start 2024-04-20T11:27:07.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.839+0000 7f003d800640 1 -- start start 2024-04-20T11:27:07.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.840+0000 7fe867400640 1 -- 172.21.15.152:0/3089408823 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe860039070 con 0x7fe86806f060 2024-04-20T11:27:07.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.840+0000 7fe867400640 1 -- 172.21.15.152:0/3089408823 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe86002cc00 con 0x7fe86806f060 2024-04-20T11:27:07.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.840+0000 7fe867400640 1 -- 172.21.15.152:0/3089408823 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe860034440 con 0x7fe86806f060 2024-04-20T11:27:07.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.841+0000 7fe86fa00640 1 -- 172.21.15.152:0/3089408823 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe86806f060 msgr2=0x7fe86806f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:07.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.841+0000 7fe86fa00640 1 --2- 172.21.15.152:0/3089408823 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe86806f060 0x7fe86806f810 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe860009d90 tx=0x7fe86002be50 comp rx=0 tx=0).stop 2024-04-20T11:27:07.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.840+0000 7f4f9c400640 1 -- start start 2024-04-20T11:27:07.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.842+0000 7f003d800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f0038162f10 0x7f003806b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:07.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.842+0000 7fe86fa00640 1 -- 172.21.15.152:0/3089408823 shutdown_connections 2024-04-20T11:27:07.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.842+0000 7fe86fa00640 1 --2- 172.21.15.152:0/3089408823 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe86806f060 0x7fe86806f810 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:07.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.842+0000 7fe86fa00640 1 --2- 172.21.15.152:0/3089408823 >> v2:172.21.15.152:3300/0 conn(0x7fe86815e830 0x7fe86806de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:07.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.842+0000 7fe86fa00640 1 -- 172.21.15.152:0/3089408823 >> 172.21.15.152:0/3089408823 conn(0x7fe8680678a0 msgr2=0x7fe86806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:07.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.842+0000 7f003d800640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f00380b3990 0x7f003806bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:07.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.842+0000 7f4f9c400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f4f9406df30 0x7f4f9415a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:07.843 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.842+0000 7f003d800640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00381632f0 con 0x7f00380d0ae0 2024-04-20T11:27:07.843 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.842+0000 7f003d800640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f00380b1a00 con 0x7f0038162f10 2024-04-20T11:27:07.843 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.843+0000 7f003d800640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f00380c4430 con 0x7f00380b3990 2024-04-20T11:27:07.843 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.843+0000 7f0037400640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f00380b3990 0x7f003806bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:07.843 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.843+0000 7f0037400640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f00380b3990 0x7f003806bec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.152:43448/0 (socket says 172.21.15.152:43448) 2024-04-20T11:27:07.843 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.843+0000 7fe86fa00640 1 -- 172.21.15.152:0/3089408823 shutdown_connections 2024-04-20T11:27:07.844 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.842+0000 7f4f9c400640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f9415b380 con 0x7f4f94067d50 2024-04-20T11:27:07.844 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.842+0000 7f4f9c400640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f9415b660 con 0x7f4f940b46b0 2024-04-20T11:27:07.844 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.842+0000 7f4f9c400640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f4f9415b940 con 0x7f4f9406df30 2024-04-20T11:27:07.844 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.843+0000 7fe86fa00640 1 -- 172.21.15.152:0/3089408823 wait complete. 2024-04-20T11:27:07.844 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.843+0000 7f4f99a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f4f9406df30 0x7f4f9415a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:07.844 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.843+0000 7f4f99000640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f4f94067d50 0x7f4f940b3fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:48508/0 (socket says 172.21.15.152:48508) 2024-04-20T11:27:07.845 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.843+0000 7f4f99000640 1 -- 172.21.15.152:0/3551846468 learned_addr learned my addr 172.21.15.152:0/3551846468 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:27:07.845 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.843+0000 7f0037400640 1 -- 172.21.15.152:0/3372224599 learned_addr learned my addr 172.21.15.152:0/3372224599 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:07.845 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.843+0000 7f003ce00640 1 --1- 172.21.15.152:0/3372224599 >> v1:172.21.15.152:6789/0 conn(0x7f00380d0ae0 0x7f00380ae150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:48522/0 (socket says 172.21.15.152:48522) 2024-04-20T11:27:07.845 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.844+0000 7f4f99a00640 1 -- 172.21.15.152:0/3551846468 >> v1:172.21.15.175:6789/0 conn(0x7f4f940b46b0 legacy=0x7f4f941654b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:07.845 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.844+0000 7f4f99a00640 1 -- 172.21.15.152:0/3551846468 >> v1:172.21.15.152:6789/0 conn(0x7f4f94067d50 legacy=0x7f4f940b3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:07.846 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.844+0000 7f4f99a00640 1 -- 172.21.15.152:0/3551846468 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f9415bc20 con 0x7f4f9406df30 2024-04-20T11:27:07.846 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.843+0000 7f0037e00640 1 --2- 172.21.15.152:0/3372224599 >> v2:172.21.15.152:3300/0 conn(0x7f0038162f10 0x7f003806b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:07.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.844+0000 7f0036a00640 1 -- 172.21.15.152:0/3372224599 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 917814957 0 0) 0x7f00381632f0 con 0x7f00380d0ae0 2024-04-20T11:27:07.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.844+0000 7f0036a00640 1 -- 172.21.15.152:0/3372224599 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f001c003520 con 0x7f00380d0ae0 2024-04-20T11:27:07.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.845+0000 7f0036a00640 1 -- 172.21.15.152:0/3372224599 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2432362433 0 0) 0x7f001c003520 con 0x7f00380d0ae0 2024-04-20T11:27:07.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.845+0000 7f0036a00640 1 -- 172.21.15.152:0/3372224599 >> v2:172.21.15.175:3300/0 conn(0x7f00380b3990 msgr2=0x7f003806bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:07.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.845+0000 7f0036a00640 1 --2- 172.21.15.152:0/3372224599 >> v2:172.21.15.175:3300/0 conn(0x7f00380b3990 0x7f003806bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:07.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.845+0000 7f0036a00640 1 -- 172.21.15.152:0/3372224599 >> v2:172.21.15.152:3300/0 conn(0x7f0038162f10 msgr2=0x7f003806b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:07.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.845+0000 7f0036a00640 1 --2- 172.21.15.152:0/3372224599 >> v2:172.21.15.152:3300/0 conn(0x7f0038162f10 0x7f003806b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:07.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.845+0000 7f0036a00640 1 -- 172.21.15.152:0/3372224599 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00380c4710 con 0x7f00380d0ae0 2024-04-20T11:27:07.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.845+0000 7f0036a00640 1 -- 172.21.15.152:0/3372224599 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f0024001670 con 0x7f00380d0ae0 2024-04-20T11:27:07.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.845+0000 7f0036a00640 1 -- 172.21.15.152:0/3372224599 >> v1:172.21.15.152:6789/0 conn(0x7f00380d0ae0 legacy=0x7f00380ae150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:07.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.846+0000 7fe86fa00640 1 Processor -- start 2024-04-20T11:27:07.848 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.846+0000 7f0036a00640 1 --2- 172.21.15.152:0/3372224599 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f001c003a00 0x7f001c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:07.848 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.846+0000 7f0036a00640 1 --2- 172.21.15.152:0/3372224599 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f001c004800 0x7f001c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:07.848 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.846+0000 7f0036a00640 1 --2- 172.21.15.152:0/3372224599 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f001c0051c0 0x7f001c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:07.848 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.846+0000 7f0036a00640 1 -- 172.21.15.152:0/3372224599 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f00380b1a00 con 0x7f001c003a00 2024-04-20T11:27:07.848 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.846+0000 7f0036a00640 1 -- 172.21.15.152:0/3372224599 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f00380c4430 con 0x7f001c004800 2024-04-20T11:27:07.848 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.846+0000 7f0036a00640 1 -- 172.21.15.152:0/3372224599 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f001c00bfb0 con 0x7f001c0051c0 2024-04-20T11:27:07.848 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.846+0000 7fe86fa00640 1 -- start start 2024-04-20T11:27:07.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.847+0000 7f0037e00640 1 --2- 172.21.15.152:0/3372224599 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f001c003a00 0x7f001c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:07.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.847+0000 7f003ce00640 1 --2- 172.21.15.152:0/3372224599 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f001c0051c0 0x7f001c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:07.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.847+0000 7f0037e00640 1 -- 172.21.15.152:0/3372224599 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f001c0051c0 msgr2=0x7f001c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:07.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.847+0000 7f0037e00640 1 --2- 172.21.15.152:0/3372224599 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f001c0051c0 0x7f001c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:07.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.847+0000 7f0037e00640 1 -- 172.21.15.152:0/3372224599 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f001c004800 msgr2=0x7f001c004c80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-20T11:27:07.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.847+0000 7f0037e00640 1 --2- 172.21.15.152:0/3372224599 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f001c004800 0x7f001c004c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:07.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.848+0000 7f0037e00640 1 -- 172.21.15.152:0/3372224599 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f001c003520 con 0x7f001c003a00 2024-04-20T11:27:07.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.848+0000 7f003ce00640 1 -- 172.21.15.152:0/3372224599 reap_dead start 2024-04-20T11:27:07.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.848+0000 7fe86fa00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe86806f060 0x7fe8681ed630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:07.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.848+0000 7fe86fa00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe8680b3e80 0x7fe8680cf740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:07.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.848+0000 7fe86fa00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe86815e830 0x7fe8680b14b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:07.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.849+0000 7fe86fa00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fe8680b42f0 con 0x7fe8680b3e80 2024-04-20T11:27:07.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.849+0000 7fe86fa00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe86806eb40 con 0x7fe86806f060 2024-04-20T11:27:07.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.849+0000 7fe86fa00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe8680b4470 con 0x7fe86815e830 2024-04-20T11:27:07.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.849+0000 7f4f99a00640 1 --2- 172.21.15.152:0/3551846468 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4f9406df30 0x7f4f9415a8d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4f88010b90 tx=0x7f4f88030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2143a1823932daff server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:07.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.849+0000 7fe867e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe8680b3e80 0x7fe8680cf740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:07.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.849+0000 7fe867e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe8680b3e80 0x7fe8680cf740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:48150/0 (socket says 172.21.15.152:48150) 2024-04-20T11:27:07.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.849+0000 7fe867e00640 1 -- 172.21.15.152:0/3926195852 learned_addr learned my addr 172.21.15.152:0/3926195852 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:07.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.849+0000 7fe86da00640 1 --2- 172.21.15.152:0/3926195852 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe86815e830 0x7fe8680b14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:07.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.850+0000 7fe86d000640 1 --2- 172.21.15.152:0/3926195852 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe86806f060 0x7fe8681ed630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:07.852 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.850+0000 7f0037e00640 1 --2- 172.21.15.152:0/3372224599 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f001c003a00 0x7f001c003e20 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0028009d20 tx=0x7f001c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=124284bcd1276a04 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:07.852 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.850+0000 7fe86da00640 1 -- 172.21.15.152:0/3926195852 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe86806f060 msgr2=0x7fe8681ed630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:07.852 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.850+0000 7fe86da00640 1 --2- 172.21.15.152:0/3926195852 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe86806f060 0x7fe8681ed630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:07.852 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.850+0000 7fe86da00640 1 -- 172.21.15.152:0/3926195852 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe8680b3e80 msgr2=0x7fe8680cf740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:07.852 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.850+0000 7f0036a00640 1 -- 172.21.15.152:0/3372224599 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0028039070 con 0x7f001c003a00 2024-04-20T11:27:07.852 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.851+0000 7f0036a00640 1 -- 172.21.15.152:0/3372224599 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f002802c910 con 0x7f001c003a00 2024-04-20T11:27:07.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.851+0000 7f0036a00640 1 -- 172.21.15.152:0/3372224599 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0028009690 con 0x7f001c003a00 2024-04-20T11:27:07.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.851+0000 7fe86da00640 1 --2- 172.21.15.152:0/3926195852 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe8680b3e80 0x7fe8680cf740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:07.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.851+0000 7fe86da00640 1 -- 172.21.15.152:0/3926195852 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8681edb70 con 0x7fe86815e830 2024-04-20T11:27:07.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.851+0000 7fe867e00640 1 --2- 172.21.15.152:0/3926195852 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe8680b3e80 0x7fe8680cf740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:27:07.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.852+0000 7f003d800640 1 -- 172.21.15.152:0/3372224599 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f001c003a00 msgr2=0x7f001c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:07.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.852+0000 7f003d800640 1 --2- 172.21.15.152:0/3372224599 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f001c003a00 0x7f001c003e20 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0028009d20 tx=0x7f001c005830 comp rx=0 tx=0).stop 2024-04-20T11:27:07.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.852+0000 7f003d800640 1 -- 172.21.15.152:0/3372224599 shutdown_connections 2024-04-20T11:27:07.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.852+0000 7f003d800640 1 --2- 172.21.15.152:0/3372224599 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f001c003a00 0x7f001c003e20 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:07.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.852+0000 7f003d800640 1 -- 172.21.15.152:0/3372224599 >> 172.21.15.152:0/3372224599 conn(0x7f00380b2e70 msgr2=0x7f00380b3280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:07.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.852+0000 7f4f93e00640 1 -- 172.21.15.152:0/3551846468 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f8803d070 con 0x7f4f9406df30 2024-04-20T11:27:07.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.852+0000 7fe86da00640 1 --2- 172.21.15.152:0/3926195852 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe86815e830 0x7fe8680b14b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe85400c3c0 tx=0x7fe85400c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:07.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.852+0000 7f003d800640 1 -- 172.21.15.152:0/3372224599 shutdown_connections 2024-04-20T11:27:07.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.852+0000 7f003d800640 1 -- 172.21.15.152:0/3372224599 wait complete. 2024-04-20T11:27:07.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.853+0000 7fe864c00640 1 -- 172.21.15.152:0/3926195852 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8540052d0 con 0x7fe86815e830 2024-04-20T11:27:07.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.854+0000 7fe864c00640 1 -- 172.21.15.152:0/3926195852 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe854005470 con 0x7fe86815e830 2024-04-20T11:27:07.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.854+0000 7fe864c00640 1 -- 172.21.15.152:0/3926195852 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe854007260 con 0x7fe86815e830 2024-04-20T11:27:07.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.854+0000 7fe86fa00640 1 -- 172.21.15.152:0/3926195852 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8681ec590 con 0x7fe86815e830 2024-04-20T11:27:07.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.854+0000 7fe86fa00640 1 -- 172.21.15.152:0/3926195852 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8681ecb50 con 0x7fe86815e830 2024-04-20T11:27:07.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.852+0000 7f4f93e00640 1 -- 172.21.15.152:0/3551846468 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f88030db0 con 0x7f4f9406df30 2024-04-20T11:27:07.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.852+0000 7f4f93e00640 1 -- 172.21.15.152:0/3551846468 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f88038660 con 0x7f4f9406df30 2024-04-20T11:27:07.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.853+0000 7f4f9c400640 1 -- 172.21.15.152:0/3551846468 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4f9406df30 msgr2=0x7f4f9415a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:07.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.853+0000 7f4f9c400640 1 --2- 172.21.15.152:0/3551846468 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4f9406df30 0x7f4f9415a8d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4f88010b90 tx=0x7f4f88030000 comp rx=0 tx=0).stop 2024-04-20T11:27:07.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.854+0000 7f4f9c400640 1 -- 172.21.15.152:0/3551846468 shutdown_connections 2024-04-20T11:27:07.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.854+0000 7f4f9c400640 1 --2- 172.21.15.152:0/3551846468 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4f9406df30 0x7f4f9415a8d0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:07.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.854+0000 7f4f9c400640 1 -- 172.21.15.152:0/3551846468 >> 172.21.15.152:0/3551846468 conn(0x7f4f940678a0 msgr2=0x7f4f9406b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:07.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.855+0000 7f003d800640 1 Processor -- start 2024-04-20T11:27:07.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.855+0000 7f003d800640 1 -- start start 2024-04-20T11:27:07.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.856+0000 7fe86fa00640 1 -- 172.21.15.152:0/3926195852 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8680689f0 con 0x7fe86815e830 2024-04-20T11:27:07.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.857+0000 7f003d800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f001c003a00 0x7f00381f7f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:07.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.857+0000 7f003d800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f00380c7640 0x7f00380cdba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:07.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.857+0000 7f003d800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f00380d1140 0x7f00380d15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:07.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.857+0000 7f003d800640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f00380b22f0 con 0x7f00380c7640 2024-04-20T11:27:07.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.857+0000 7f003d800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f00380b2170 con 0x7f001c003a00 2024-04-20T11:27:07.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.857+0000 7f003d800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f00380b0420 con 0x7f00380d1140 2024-04-20T11:27:07.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.858+0000 7f0037400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f00380c7640 0x7f00380cdba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:07.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.858+0000 7f0037e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f001c003a00 0x7f00381f7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:07.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.858+0000 7f0037400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f00380c7640 0x7f00380cdba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:48166/0 (socket says 172.21.15.152:48166) 2024-04-20T11:27:07.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.858+0000 7f0037400640 1 -- 172.21.15.152:0/2282640299 learned_addr learned my addr 172.21.15.152:0/2282640299 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:07.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.858+0000 7f003ce00640 1 --2- 172.21.15.152:0/2282640299 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f00380d1140 0x7f00380d15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:07.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.855+0000 7f4f9c400640 1 -- 172.21.15.152:0/3551846468 shutdown_connections 2024-04-20T11:27:07.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.855+0000 7f4f9c400640 1 -- 172.21.15.152:0/3551846468 wait complete. 2024-04-20T11:27:07.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.859+0000 7f0037e00640 1 -- 172.21.15.152:0/2282640299 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f00380d1140 msgr2=0x7f00380d15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:07.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.859+0000 7f0037e00640 1 --2- 172.21.15.152:0/2282640299 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f00380d1140 0x7f00380d15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:07.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.859+0000 7f0037e00640 1 -- 172.21.15.152:0/2282640299 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f00380c7640 msgr2=0x7f00380cdba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:07.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.859+0000 7f0037e00640 1 --2- 172.21.15.152:0/2282640299 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f00380c7640 0x7f00380cdba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:07.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.858+0000 7fe864c00640 1 -- 172.21.15.152:0/3926195852 <== mon.2 v2:172.21.15.175:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe854007400 con 0x7fe86815e830 2024-04-20T11:27:07.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.859+0000 7f0037e00640 1 -- 172.21.15.152:0/2282640299 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00380ce4a0 con 0x7f001c003a00 2024-04-20T11:27:07.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.859+0000 7f003ce00640 1 --2- 172.21.15.152:0/2282640299 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f00380d1140 0x7f00380d15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:27:07.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.859+0000 7f0037400640 1 --2- 172.21.15.152:0/2282640299 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f00380c7640 0x7f00380cdba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:27:07.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.860+0000 7f4f9c400640 1 Processor -- start 2024-04-20T11:27:07.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.861+0000 7f4f9c400640 1 -- start start 2024-04-20T11:27:07.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.861+0000 7fe864c00640 1 --2- 172.21.15.152:0/3926195852 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe84806c970 0x7fe84806ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:07.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.861+0000 7fe864c00640 1 -- 172.21.15.152:0/3926195852 <== mon.2 v2:172.21.15.175:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fe85408acc0 con 0x7fe86815e830 2024-04-20T11:27:07.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.861+0000 7f0037e00640 1 --2- 172.21.15.152:0/2282640299 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f001c003a00 0x7f00381f7f30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0028033000 tx=0x7f0028004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:07.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.862+0000 7fe86d000640 1 --2- 172.21.15.152:0/3926195852 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe84806c970 0x7fe84806ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:07.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.862+0000 7f002fe00640 1 -- 172.21.15.152:0/2282640299 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0028044020 con 0x7f001c003a00 2024-04-20T11:27:07.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.862+0000 7f002fe00640 1 -- 172.21.15.152:0/2282640299 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0028004440 con 0x7f001c003a00 2024-04-20T11:27:07.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.862+0000 7f002fe00640 1 -- 172.21.15.152:0/2282640299 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0028039040 con 0x7f001c003a00 2024-04-20T11:27:07.863 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.863+0000 7f4f9c400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4f94067d50 0x7f4f9414d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:07.863 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.863+0000 7f4f9c400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4f9406df30 0x7f4f940cdfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:07.863 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.863+0000 7f003d800640 1 -- 172.21.15.152:0/2282640299 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00380c4f30 con 0x7f001c003a00 2024-04-20T11:27:07.863 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.863+0000 7f4f9c400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4f940b46b0 0x7f4f9415e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:07.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.863+0000 7fe86d000640 1 --2- 172.21.15.152:0/3926195852 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe84806c970 0x7fe84806ee30 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe858009630 tx=0x7fe858009240 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:07.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.863+0000 7f003d800640 1 -- 172.21.15.152:0/2282640299 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00381f8470 con 0x7f001c003a00 2024-04-20T11:27:07.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.863+0000 7f4f99a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4f94067d50 0x7f4f9414d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:07.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.863+0000 7f4f99a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4f94067d50 0x7f4f9414d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.152:34192/0 (socket says 172.21.15.152:34192) 2024-04-20T11:27:07.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.864+0000 7f4f99000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4f9406df30 0x7f4f940cdfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:07.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.864+0000 7f4f99000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4f9406df30 0x7f4f940cdfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.152:43490/0 (socket says 172.21.15.152:43490) 2024-04-20T11:27:07.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.864+0000 7f4f9a400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4f940b46b0 0x7f4f9415e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:07.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.864+0000 7f4f9a400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4f940b46b0 0x7f4f9415e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:48168/0 (socket says 172.21.15.152:48168) 2024-04-20T11:27:07.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.864+0000 7f4f99a00640 1 -- 172.21.15.152:0/4175255632 learned_addr learned my addr 172.21.15.152:0/4175255632 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:07.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.864+0000 7f4f9c400640 1 -- 172.21.15.152:0/4175255632 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f4f9415af90 con 0x7f4f940b46b0 2024-04-20T11:27:07.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.864+0000 7f4f9c400640 1 -- 172.21.15.152:0/4175255632 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4f9415ae10 con 0x7f4f9406df30 2024-04-20T11:27:07.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.864+0000 7f4f9c400640 1 -- 172.21.15.152:0/4175255632 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4f9415b110 con 0x7f4f94067d50 2024-04-20T11:27:07.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.865+0000 7f4f9a400640 1 -- 172.21.15.152:0/4175255632 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4f94067d50 msgr2=0x7f4f9414d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:07.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.865+0000 7f4f9a400640 1 --2- 172.21.15.152:0/4175255632 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4f94067d50 0x7f4f9414d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:07.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.865+0000 7f4f9a400640 1 -- 172.21.15.152:0/4175255632 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4f9406df30 msgr2=0x7f4f940cdfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:07.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.865+0000 7f4f9a400640 1 --2- 172.21.15.152:0/4175255632 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4f9406df30 0x7f4f940cdfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:07.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.865+0000 7f4f9a400640 1 -- 172.21.15.152:0/4175255632 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f9414d780 con 0x7f4f940b46b0 2024-04-20T11:27:07.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.866+0000 7f4f9a400640 1 --2- 172.21.15.152:0/4175255632 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4f940b46b0 0x7f4f9415e5b0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c00c730 tx=0x7f4f8c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:07.867 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.865+0000 7f003d800640 1 -- 172.21.15.152:0/2282640299 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0038075e60 con 0x7f001c003a00 2024-04-20T11:27:07.867 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.867+0000 7f4f91600640 1 -- 172.21.15.152:0/4175255632 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f8c016020 con 0x7f4f940b46b0 2024-04-20T11:27:07.867 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.867+0000 7f4f91600640 1 -- 172.21.15.152:0/4175255632 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f8c006d50 con 0x7f4f940b46b0 2024-04-20T11:27:07.867 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.867+0000 7f4f91600640 1 -- 172.21.15.152:0/4175255632 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f8c003bb0 con 0x7f4f940b46b0 2024-04-20T11:27:07.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.867+0000 7f4f9c400640 1 -- 172.21.15.152:0/4175255632 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f9414da60 con 0x7f4f940b46b0 2024-04-20T11:27:07.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.867+0000 7f4f9c400640 1 -- 172.21.15.152:0/4175255632 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f940ce980 con 0x7f4f940b46b0 2024-04-20T11:27:07.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.868+0000 7f002fe00640 1 -- 172.21.15.152:0/2282640299 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0028047020 con 0x7f001c003a00 2024-04-20T11:27:07.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.869+0000 7f002fe00640 1 --2- 172.21.15.152:0/2282640299 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f001806c8f0 0x7f001806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:07.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.869+0000 7f4f9c400640 1 -- 172.21.15.152:0/4175255632 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f940ceb80 con 0x7f4f940b46b0 2024-04-20T11:27:07.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.869+0000 7f002fe00640 1 -- 172.21.15.152:0/2282640299 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f0028079090 con 0x7f001c003a00 2024-04-20T11:27:07.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.870+0000 7f0037400640 1 --2- 172.21.15.152:0/2282640299 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f001806c8f0 0x7f001806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:07.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.871+0000 7f4f91600640 1 -- 172.21.15.152:0/4175255632 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4f8c0050d0 con 0x7f4f940b46b0 2024-04-20T11:27:07.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.871+0000 7f0037400640 1 --2- 172.21.15.152:0/2282640299 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f001806c8f0 0x7f001806edb0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f00380cce20 tx=0x7f003000b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:07.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.872+0000 7f4f91600640 1 --2- 172.21.15.152:0/4175255632 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f4f7406c940 0x7f4f7406ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:07.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.872+0000 7f4f91600640 1 -- 172.21.15.152:0/4175255632 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f4f8c08b620 con 0x7f4f940b46b0 2024-04-20T11:27:07.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.873+0000 7f4f99a00640 1 --2- 172.21.15.152:0/4175255632 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f4f7406c940 0x7f4f7406ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:07.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.874+0000 7f4f99a00640 1 --2- 172.21.15.152:0/4175255632 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f4f7406c940 0x7f4f7406ee00 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4f88010950 tx=0x7f4f88034000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:07.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.877+0000 7f4f91600640 1 -- 172.21.15.152:0/4175255632 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f4f8c020020 con 0x7f4f940b46b0 2024-04-20T11:27:07.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.877+0000 7fe864c00640 1 -- 172.21.15.152:0/3926195852 <== mon.2 v2:172.21.15.175:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe854058c10 con 0x7fe86815e830 2024-04-20T11:27:07.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.884+0000 7f002fe00640 1 -- 172.21.15.152:0/2282640299 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f002807aff0 con 0x7f001c003a00 2024-04-20T11:27:07.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.884+0000 7fd88da00640 1 Processor -- start 2024-04-20T11:27:07.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.884+0000 7fd88da00640 1 -- start start 2024-04-20T11:27:07.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.886+0000 7fd88da00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fd888161d30 0x7fd8880d0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:07.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.886+0000 7fd88da00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fd8880d4ec0 0x7fd8880d15e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:07.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.886+0000 7fd88da00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8880d1b20 con 0x7fd8880d2b80 2024-04-20T11:27:07.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.886+0000 7fd88da00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fd8880d0540 con 0x7fd8880d4ec0 2024-04-20T11:27:07.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.886+0000 7fd88da00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fd8880cfb70 con 0x7fd888161d30 2024-04-20T11:27:07.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.887+0000 7fd887400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fd8880d4ec0 0x7fd8880d15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:07.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.887+0000 7fd887400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fd8880d4ec0 0x7fd8880d15e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:48184/0 (socket says 172.21.15.152:48184) 2024-04-20T11:27:07.888 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.887+0000 7fd887400640 1 -- 172.21.15.152:0/2151533928 learned_addr learned my addr 172.21.15.152:0/2151533928 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:07.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.888+0000 7fd886a00640 1 -- 172.21.15.152:0/2151533928 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1260738024 0 0) 0x7fd8880d1b20 con 0x7fd8880d2b80 2024-04-20T11:27:07.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.888+0000 7fd886a00640 1 -- 172.21.15.152:0/2151533928 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd86c003520 con 0x7fd8880d2b80 2024-04-20T11:27:07.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.887+0000 7fd887e00640 1 --2- 172.21.15.152:0/2151533928 >> v2:172.21.15.175:3300/0 conn(0x7fd888161d30 0x7fd8880d0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:07.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.889+0000 7fd886a00640 1 -- 172.21.15.152:0/2151533928 <== mon.1 v1:172.21.15.175:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 986641151 0 0) 0x7fd86c003520 con 0x7fd8880d2b80 2024-04-20T11:27:07.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.889+0000 7fd886a00640 1 -- 172.21.15.152:0/2151533928 >> v2:172.21.15.175:3300/0 conn(0x7fd888161d30 msgr2=0x7fd8880d0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:07.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.889+0000 7fd886a00640 1 --2- 172.21.15.152:0/2151533928 >> v2:172.21.15.175:3300/0 conn(0x7fd888161d30 0x7fd8880d0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:07.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.889+0000 7fd886a00640 1 -- 172.21.15.152:0/2151533928 >> v2:172.21.15.152:3300/0 conn(0x7fd8880d4ec0 msgr2=0x7fd8880d15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:07.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.889+0000 7fd886a00640 1 --2- 172.21.15.152:0/2151533928 >> v2:172.21.15.152:3300/0 conn(0x7fd8880d4ec0 0x7fd8880d15e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:07.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.889+0000 7fd886a00640 1 -- 172.21.15.152:0/2151533928 --> v1:172.21.15.175:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8880d26a0 con 0x7fd8880d2b80 2024-04-20T11:27:07.890 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.890+0000 7fd886a00640 1 -- 172.21.15.152:0/2151533928 <== mon.1 v1:172.21.15.175:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7fd8800019d0 con 0x7fd8880d2b80 2024-04-20T11:27:07.890 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.890+0000 7fd886a00640 1 -- 172.21.15.152:0/2151533928 >> v1:172.21.15.175:6789/0 conn(0x7fd8880d2b80 legacy=0x7fd88806c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:07.890 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.891+0000 7fd886a00640 1 --2- 172.21.15.152:0/2151533928 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd86c0039b0 0x7fd86c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:07.890 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.891+0000 7fd886a00640 1 --2- 172.21.15.152:0/2151533928 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd86c004800 0x7fd86c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:07.890 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.891+0000 7fd886a00640 1 --2- 172.21.15.152:0/2151533928 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd86c0051c0 0x7fd86c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:07.891 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.891+0000 7fd886a00640 1 -- 172.21.15.152:0/2151533928 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fd8880d0540 con 0x7fd86c0051c0 2024-04-20T11:27:07.891 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.891+0000 7fd886a00640 1 -- 172.21.15.152:0/2151533928 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd8880cfb70 con 0x7fd86c004800 2024-04-20T11:27:07.891 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.891+0000 7fd886a00640 1 -- 172.21.15.152:0/2151533928 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd86c00c0f0 con 0x7fd86c0039b0 2024-04-20T11:27:07.891 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.891+0000 7fd88d000640 1 --2- 172.21.15.152:0/2151533928 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd86c0051c0 0x7fd86c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:07.891 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.891+0000 7fd887400640 1 --2- 172.21.15.152:0/2151533928 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd86c004800 0x7fd86c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:07.891 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.892+0000 7fd887e00640 1 --2- 172.21.15.152:0/2151533928 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd86c0039b0 0x7fd86c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:07.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.892+0000 7fd88d000640 1 -- 172.21.15.152:0/2151533928 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd86c0039b0 msgr2=0x7fd86c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:07.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.892+0000 7fd88d000640 1 --2- 172.21.15.152:0/2151533928 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd86c0039b0 0x7fd86c003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:07.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.892+0000 7fd88d000640 1 -- 172.21.15.152:0/2151533928 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd86c004800 msgr2=0x7fd86c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:07.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.892+0000 7fd88d000640 1 --2- 172.21.15.152:0/2151533928 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd86c004800 0x7fd86c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:07.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.892+0000 7fd88d000640 1 -- 172.21.15.152:0/2151533928 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd86c003520 con 0x7fd86c0051c0 2024-04-20T11:27:07.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.894+0000 7fd88d000640 1 -- 172.21.15.152:0/2151533928 reap_dead start 2024-04-20T11:27:07.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.897+0000 7fd88d000640 1 --2- 172.21.15.152:0/2151533928 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd86c0051c0 0x7fd86c00b9c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd880004360 tx=0x7fd86c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15728181cb5f4d3b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:07.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.898+0000 7fd886a00640 1 -- 172.21.15.152:0/2151533928 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd880003f40 con 0x7fd86c0051c0 2024-04-20T11:27:07.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.898+0000 7fd886a00640 1 -- 172.21.15.152:0/2151533928 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd880004be0 con 0x7fd86c0051c0 2024-04-20T11:27:07.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.898+0000 7fd886a00640 1 -- 172.21.15.152:0/2151533928 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd880006070 con 0x7fd86c0051c0 2024-04-20T11:27:07.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.899+0000 7fd88da00640 1 -- 172.21.15.152:0/2151533928 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd86c0051c0 msgr2=0x7fd86c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:07.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.899+0000 7fd88da00640 1 --2- 172.21.15.152:0/2151533928 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd86c0051c0 0x7fd86c00b9c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd880004360 tx=0x7fd86c005830 comp rx=0 tx=0).stop 2024-04-20T11:27:07.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.899+0000 7fd88da00640 1 -- 172.21.15.152:0/2151533928 shutdown_connections 2024-04-20T11:27:07.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.899+0000 7fd88da00640 1 --2- 172.21.15.152:0/2151533928 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd86c0051c0 0x7fd86c00b9c0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:07.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.899+0000 7fd88da00640 1 -- 172.21.15.152:0/2151533928 >> 172.21.15.152:0/2151533928 conn(0x7fd88808bfd0 msgr2=0x7fd8880d45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:07.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.899+0000 7f6e93600640 1 Processor -- start 2024-04-20T11:27:07.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.899+0000 7f6e93600640 1 -- start start 2024-04-20T11:27:07.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.901+0000 7f6e93600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e8c0cf4e0 0x7f6e8c0ce880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:07.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.901+0000 7f6e93600640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f6e8c0cd0f0 0x7f6e8c0cf8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:07.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.901+0000 7f6e93600640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f6e8c0cfe00 0x7f6e8c15f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:07.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.901+0000 7f6e93600640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f6e8c0cedc0 con 0x7f6e8c0cfe00 2024-04-20T11:27:07.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.902+0000 7f6e93600640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f6e8c0ccc60 con 0x7f6e8c0cd0f0 2024-04-20T11:27:07.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.902+0000 7fd88da00640 1 -- 172.21.15.152:0/2151533928 shutdown_connections 2024-04-20T11:27:07.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.902+0000 7f6e93600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6e8c0cc330 con 0x7f6e8c0cf4e0 2024-04-20T11:27:07.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.902+0000 7f6e8be00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f6e8c0cd0f0 0x7f6e8c0cf8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:07.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.902+0000 7f6e8be00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f6e8c0cd0f0 0x7f6e8c0cf8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.152:43528/0 (socket says 172.21.15.152:43528) 2024-04-20T11:27:07.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.902+0000 7f6e8be00640 1 -- 172.21.15.152:0/356576623 learned_addr learned my addr 172.21.15.152:0/356576623 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:07.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.903+0000 7f6e91600640 1 --2- 172.21.15.152:0/356576623 >> v2:172.21.15.152:3300/0 conn(0x7f6e8c0cfe00 0x7f6e8c15f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:07.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.903+0000 7f6e90c00640 1 --2- 172.21.15.152:0/356576623 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e8c0cf4e0 0x7f6e8c0ce880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:07.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.903+0000 7f6e8be00640 1 -- 172.21.15.152:0/356576623 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e8c0cf4e0 msgr2=0x7f6e8c0ce880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:07.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.903+0000 7f6e8be00640 1 --2- 172.21.15.152:0/356576623 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e8c0cf4e0 0x7f6e8c0ce880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:07.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.903+0000 7f6e8be00640 1 -- 172.21.15.152:0/356576623 >> v2:172.21.15.152:3300/0 conn(0x7f6e8c0cfe00 msgr2=0x7f6e8c15f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:07.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.903+0000 7f6e8be00640 1 --2- 172.21.15.152:0/356576623 >> v2:172.21.15.152:3300/0 conn(0x7f6e8c0cfe00 0x7f6e8c15f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:07.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.903+0000 7f6e8be00640 1 -- 172.21.15.152:0/356576623 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e8c0cc610 con 0x7f6e8c0cd0f0 2024-04-20T11:27:07.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.903+0000 7f6e91600640 1 --2- 172.21.15.152:0/356576623 >> v2:172.21.15.152:3300/0 conn(0x7f6e8c0cfe00 0x7f6e8c15f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:27:07.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.902+0000 7fd88da00640 1 -- 172.21.15.152:0/2151533928 wait complete. 2024-04-20T11:27:07.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.905+0000 7fd88da00640 1 Processor -- start 2024-04-20T11:27:07.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.905+0000 7fd88da00640 1 -- start start 2024-04-20T11:27:07.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.905+0000 7f6e8be00640 1 --2- 172.21.15.152:0/356576623 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e8c0cd0f0 0x7f6e8c0cf8c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6e8c0718c0 tx=0x7f6e7802b940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47f1d4dc8bd5ada2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:07.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.906+0000 7f6e8b400640 1 -- 172.21.15.152:0/356576623 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e78034070 con 0x7f6e8c0cd0f0 2024-04-20T11:27:07.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.906+0000 7f6e8b400640 1 -- 172.21.15.152:0/356576623 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e78030800 con 0x7f6e8c0cd0f0 2024-04-20T11:27:07.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.906+0000 7f6e8b400640 1 -- 172.21.15.152:0/356576623 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e78030c10 con 0x7f6e8c0cd0f0 2024-04-20T11:27:07.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.907+0000 7f6e93600640 1 -- 172.21.15.152:0/356576623 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e8c0cd0f0 msgr2=0x7f6e8c0cf8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:07.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.907+0000 7f6e93600640 1 --2- 172.21.15.152:0/356576623 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e8c0cd0f0 0x7f6e8c0cf8c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6e8c0718c0 tx=0x7f6e7802b940 comp rx=0 tx=0).stop 2024-04-20T11:27:07.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.907+0000 7fd88da00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd86c0051c0 0x7fd8882019d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:07.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.908+0000 7fd88da00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd88815b250 0x7fd8882006d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:07.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.908+0000 7fd88da00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd88815d8d0 0x7fd88815dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:07.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.908+0000 7fd887400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd88815b250 0x7fd8882006d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:07.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.909+0000 7fd887400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd88815b250 0x7fd8882006d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.152:34220/0 (socket says 172.21.15.152:34220) 2024-04-20T11:27:07.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.909+0000 7fd887400640 1 -- 172.21.15.152:0/1268622032 learned_addr learned my addr 172.21.15.152:0/1268622032 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:07.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.909+0000 7f6e93600640 1 -- 172.21.15.152:0/356576623 shutdown_connections 2024-04-20T11:27:07.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.909+0000 7f6e93600640 1 --2- 172.21.15.152:0/356576623 >> v2:172.21.15.152:3300/0 conn(0x7f6e8c0cfe00 0x7f6e8c15f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:07.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.909+0000 7f6e93600640 1 --2- 172.21.15.152:0/356576623 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e8c0cd0f0 0x7f6e8c0cf8c0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:07.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.909+0000 7f6e93600640 1 --2- 172.21.15.152:0/356576623 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e8c0cf4e0 0x7f6e8c0ce880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:07.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.909+0000 7f6e93600640 1 -- 172.21.15.152:0/356576623 >> 172.21.15.152:0/356576623 conn(0x7f6e8c0b3030 msgr2=0x7f6e8c0b3440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:07.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.909+0000 7fd88da00640 1 -- 172.21.15.152:0/1268622032 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fd8880cf800 con 0x7fd88815d8d0 2024-04-20T11:27:07.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.909+0000 7f6e93600640 1 -- 172.21.15.152:0/356576623 shutdown_connections 2024-04-20T11:27:07.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.909+0000 7f6e93600640 1 -- 172.21.15.152:0/356576623 wait complete. 2024-04-20T11:27:07.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.909+0000 7fd887e00640 1 --2- 172.21.15.152:0/1268622032 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd86c0051c0 0x7fd8882019d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:07.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.909+0000 7fd88d000640 1 --2- 172.21.15.152:0/1268622032 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd88815d8d0 0x7fd88815dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:07.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.910+0000 7fd88da00640 1 -- 172.21.15.152:0/1268622032 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd8880cf520 con 0x7fd86c0051c0 2024-04-20T11:27:07.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.910+0000 7fd88da00640 1 -- 172.21.15.152:0/1268622032 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd8880cf1b0 con 0x7fd88815b250 2024-04-20T11:27:07.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.911+0000 7fd887e00640 1 -- 172.21.15.152:0/1268622032 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd88815b250 msgr2=0x7fd8882006d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:07.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.911+0000 7fd887e00640 1 --2- 172.21.15.152:0/1268622032 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd88815b250 0x7fd8882006d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:07.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.911+0000 7fd887e00640 1 -- 172.21.15.152:0/1268622032 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd88815d8d0 msgr2=0x7fd88815dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:07.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.911+0000 7fd887e00640 1 --2- 172.21.15.152:0/1268622032 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd88815d8d0 0x7fd88815dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:07.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.911+0000 7fd887e00640 1 -- 172.21.15.152:0/1268622032 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd888201f10 con 0x7fd86c0051c0 2024-04-20T11:27:07.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.912+0000 7f6e93600640 1 Processor -- start 2024-04-20T11:27:07.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.912+0000 7fd88d000640 1 --2- 172.21.15.152:0/1268622032 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd88815d8d0 0x7fd88815dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:27:07.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.912+0000 7fd887e00640 1 --2- 172.21.15.152:0/1268622032 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd86c0051c0 0x7fd8882019d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd8880515f0 tx=0x7fd87400e900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:07.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.912+0000 7fd887400640 1 --2- 172.21.15.152:0/1268622032 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd88815b250 0x7fd8882006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:27:07.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.912+0000 7f6e93600640 1 -- start start 2024-04-20T11:27:07.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.913+0000 7fd87fe00640 1 -- 172.21.15.152:0/1268622032 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd87400f040 con 0x7fd86c0051c0 2024-04-20T11:27:07.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.913+0000 7fd87fe00640 1 -- 172.21.15.152:0/1268622032 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd874004230 con 0x7fd86c0051c0 2024-04-20T11:27:07.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.913+0000 7fd87fe00640 1 -- 172.21.15.152:0/1268622032 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd874003610 con 0x7fd86c0051c0 2024-04-20T11:27:07.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.914+0000 7fd88da00640 1 -- 172.21.15.152:0/1268622032 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd88815b9a0 con 0x7fd86c0051c0 2024-04-20T11:27:07.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.914+0000 7fd88da00640 1 -- 172.21.15.152:0/1268622032 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd88815bf60 con 0x7fd86c0051c0 2024-04-20T11:27:07.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.915+0000 7f6e93600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6e8c0cd0f0 0x7f6e8c0cafd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:07.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.915+0000 7f6e93600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e8c0cf4e0 0x7f6e8c0c6b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:07.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.916+0000 7f6e93600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e8c0cfe00 0x7f6e8c0c7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:07.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.916+0000 7f6e93600640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f6e8c0cbf70 con 0x7f6e8c0cd0f0 2024-04-20T11:27:07.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.916+0000 7f6e93600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6e8c070c50 con 0x7f6e8c0cfe00 2024-04-20T11:27:07.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.916+0000 7f6e93600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6e8c0cc0f0 con 0x7f6e8c0cf4e0 2024-04-20T11:27:07.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.916+0000 7f6e90c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6e8c0cd0f0 0x7f6e8c0cafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:07.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.916+0000 7f6e90c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6e8c0cd0f0 0x7f6e8c0cafd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:48206/0 (socket says 172.21.15.152:48206) 2024-04-20T11:27:07.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.916+0000 7f6e90c00640 1 -- 172.21.15.152:0/72581053 learned_addr learned my addr 172.21.15.152:0/72581053 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:07.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.916+0000 7fd88da00640 1 -- 172.21.15.152:0/1268622032 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd888075e60 con 0x7fd86c0051c0 2024-04-20T11:27:07.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.916+0000 7f6e8be00640 1 --2- 172.21.15.152:0/72581053 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e8c0cf4e0 0x7f6e8c0c6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:07.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.916+0000 7f6e91600640 1 --2- 172.21.15.152:0/72581053 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e8c0cfe00 0x7f6e8c0c7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:07.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.917+0000 7f6e90c00640 1 -- 172.21.15.152:0/72581053 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e8c0cf4e0 msgr2=0x7f6e8c0c6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:07.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.917+0000 7f6e90c00640 1 --2- 172.21.15.152:0/72581053 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e8c0cf4e0 0x7f6e8c0c6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:07.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.917+0000 7f6e90c00640 1 -- 172.21.15.152:0/72581053 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e8c0cfe00 msgr2=0x7f6e8c0c7040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:07.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.917+0000 7f6e90c00640 1 --2- 172.21.15.152:0/72581053 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e8c0cfe00 0x7f6e8c0c7040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:07.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.917+0000 7f6e90c00640 1 -- 172.21.15.152:0/72581053 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e8c0c9220 con 0x7f6e8c0cd0f0 2024-04-20T11:27:07.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.917+0000 7f6e90c00640 1 --2- 172.21.15.152:0/72581053 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6e8c0cd0f0 0x7f6e8c0cafd0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6e8000c7f0 tx=0x7f6e8000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:07.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.918+0000 7f6e88c00640 1 -- 172.21.15.152:0/72581053 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e80015070 con 0x7f6e8c0cd0f0 2024-04-20T11:27:07.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.918+0000 7f6e88c00640 1 -- 172.21.15.152:0/72581053 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e80004220 con 0x7f6e8c0cd0f0 2024-04-20T11:27:07.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.918+0000 7f6e88c00640 1 -- 172.21.15.152:0/72581053 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e80007620 con 0x7f6e8c0cd0f0 2024-04-20T11:27:07.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.919+0000 7fd87fe00640 1 -- 172.21.15.152:0/1268622032 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd8740037b0 con 0x7fd86c0051c0 2024-04-20T11:27:07.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.920+0000 7fd87fe00640 1 --2- 172.21.15.152:0/1268622032 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fd86806c920 0x7fd86806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:07.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.920+0000 7fd87fe00640 1 -- 172.21.15.152:0/1268622032 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd87408bcc0 con 0x7fd86c0051c0 2024-04-20T11:27:07.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.920+0000 7f6e93600640 1 -- 172.21.15.152:0/72581053 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e8c0c9500 con 0x7f6e8c0cd0f0 2024-04-20T11:27:07.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.920+0000 7f6e93600640 1 -- 172.21.15.152:0/72581053 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e8c0c9ac0 con 0x7f6e8c0cd0f0 2024-04-20T11:27:07.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.921+0000 7fd887400640 1 --2- 172.21.15.152:0/1268622032 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fd86806c920 0x7fd86806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:07.922 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.923+0000 7fd887400640 1 --2- 172.21.15.152:0/1268622032 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fd86806c920 0x7fd86806ede0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd878006f60 tx=0x7fd878008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:07.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.922+0000 7f6e93600640 1 -- 172.21.15.152:0/72581053 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e8c0c9e00 con 0x7f6e8c0cd0f0 2024-04-20T11:27:07.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.925+0000 7f6e88c00640 1 -- 172.21.15.152:0/72581053 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6e800077c0 con 0x7f6e8c0cd0f0 2024-04-20T11:27:07.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.927+0000 7f6e88c00640 1 --2- 172.21.15.152:0/72581053 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f6e6c06c9c0 0x7f6e6c06ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:07.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.927+0000 7f6e88c00640 1 -- 172.21.15.152:0/72581053 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f6e8008bea0 con 0x7f6e8c0cd0f0 2024-04-20T11:27:07.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.928+0000 7f6e8be00640 1 --2- 172.21.15.152:0/72581053 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f6e6c06c9c0 0x7f6e6c06ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:07.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.929+0000 7f6e8be00640 1 --2- 172.21.15.152:0/72581053 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f6e6c06c9c0 0x7f6e6c06ee80 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f6e78036000 tx=0x7f6e78008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:07.931 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.931+0000 7f6e88c00640 1 -- 172.21.15.152:0/72581053 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f6e8c0c9e00 con 0x7f6e8c0cd0f0 2024-04-20T11:27:07.931 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:07.932+0000 7fd87fe00640 1 -- 172.21.15.152:0/1268622032 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fd888201f10 con 0x7fd86c0051c0 2024-04-20T11:27:08.195 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.196+0000 7ffa1e600640 1 -- 172.21.15.152:0/1376285398 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7ffa18072e90 con 0x7ffa180cfa00 2024-04-20T11:27:08.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.197+0000 7ffa0f400640 1 -- 172.21.15.152:0/1376285398 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ffa1005d070 con 0x7ffa180cfa00 2024-04-20T11:27:08.197 INFO:teuthology.orchestra.run.smithi152.stdout:55834574853 2024-04-20T11:27:08.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.200+0000 7ffa1e600640 1 -- 172.21.15.152:0/1376285398 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff9f806c9c0 msgr2=0x7ff9f806ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:08.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.200+0000 7ffa1e600640 1 --2- 172.21.15.152:0/1376285398 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff9f806c9c0 0x7ff9f806ee80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ffa040097a0 tx=0x7ffa0402f000 comp rx=0 tx=0).stop 2024-04-20T11:27:08.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.201+0000 7ffa1e600640 1 -- 172.21.15.152:0/1376285398 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffa180cfa00 msgr2=0x7ffa182044e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:08.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.201+0000 7ffa1e600640 1 --2- 172.21.15.152:0/1376285398 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffa180cfa00 0x7ffa182044e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ffa10000c00 tx=0x7ffa1000cbe0 comp rx=0 tx=0).stop 2024-04-20T11:27:08.201 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.202+0000 7ffa1e600640 1 -- 172.21.15.152:0/1376285398 shutdown_connections 2024-04-20T11:27:08.201 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.202+0000 7ffa1e600640 1 --2- 172.21.15.152:0/1376285398 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff9f806c9c0 0x7ff9f806ee80 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:08.201 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.202+0000 7ffa1e600640 1 --2- 172.21.15.152:0/1376285398 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa1815ec80 0x7ffa1815a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:08.201 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.202+0000 7ffa1e600640 1 --2- 172.21.15.152:0/1376285398 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffa180cfa00 0x7ffa182044e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:08.201 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.202+0000 7ffa1e600640 1 --2- 172.21.15.152:0/1376285398 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa180cf5a0 0x7ffa180cd6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:08.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.202+0000 7ffa1e600640 1 -- 172.21.15.152:0/1376285398 >> 172.21.15.152:0/1376285398 conn(0x7ffa18067ae0 msgr2=0x7ffa180ceb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:08.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.202+0000 7ffa1e600640 1 -- 172.21.15.152:0/1376285398 shutdown_connections 2024-04-20T11:27:08.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.203+0000 7ffa1e600640 1 -- 172.21.15.152:0/1376285398 wait complete. 2024-04-20T11:27:08.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.229+0000 7f13bfe00640 1 -- 172.21.15.152:0/3323557946 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f1398005580 con 0x7f13c01674e0 2024-04-20T11:27:08.230 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.1 2024-04-20T11:27:08.230 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.230+0000 7f13b6a00640 1 -- 172.21.15.152:0/3323557946 <== mon.2 v2:172.21.15.175:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f13b8059b00 con 0x7f13c01674e0 2024-04-20T11:27:08.230 INFO:teuthology.orchestra.run.smithi152.stdout:55834574855 2024-04-20T11:27:08.230 DEBUG:teuthology.parallel:result is None 2024-04-20T11:27:08.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.233+0000 7f13bfe00640 1 -- 172.21.15.152:0/3323557946 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f13a006c9f0 msgr2=0x7f13a006eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:08.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.233+0000 7f13bfe00640 1 --2- 172.21.15.152:0/3323557946 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f13a006c9f0 0x7f13a006eeb0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f13ac006f40 tx=0x7f13ac006e90 comp rx=0 tx=0).stop 2024-04-20T11:27:08.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.233+0000 7f13bfe00640 1 -- 172.21.15.152:0/3323557946 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f13c01674e0 msgr2=0x7f13c009b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:08.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.234+0000 7f13bfe00640 1 --2- 172.21.15.152:0/3323557946 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f13c01674e0 0x7f13c009b700 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f13b800c730 tx=0x7f13b800cb30 comp rx=0 tx=0).stop 2024-04-20T11:27:08.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.234+0000 7f13bfe00640 1 -- 172.21.15.152:0/3323557946 shutdown_connections 2024-04-20T11:27:08.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.234+0000 7f13bfe00640 1 --2- 172.21.15.152:0/3323557946 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f13a006c9f0 0x7f13a006eeb0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:08.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.234+0000 7f13bfe00640 1 --2- 172.21.15.152:0/3323557946 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f13c01674e0 0x7f13c009b700 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:08.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.234+0000 7f13bfe00640 1 --2- 172.21.15.152:0/3323557946 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f13c01663f0 0x7f13c00b4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:08.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.234+0000 7f13bfe00640 1 --2- 172.21.15.152:0/3323557946 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f13c00cf750 0x7f13c0161980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:08.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.234+0000 7f13bfe00640 1 -- 172.21.15.152:0/3323557946 >> 172.21.15.152:0/3323557946 conn(0x7f13c00b5a10 msgr2=0x7f13c0099810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:08.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.235+0000 7f13bfe00640 1 -- 172.21.15.152:0/3323557946 shutdown_connections 2024-04-20T11:27:08.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.235+0000 7f13bfe00640 1 -- 172.21.15.152:0/3323557946 wait complete. 2024-04-20T11:27:08.253 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.254+0000 7fd242000640 1 -- 172.21.15.152:0/2981202700 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fd224005580 con 0x7fd24804ffd0 2024-04-20T11:27:08.254 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.255+0000 7fd24cc00640 1 -- 172.21.15.152:0/2981202700 <== mon.2 v2:172.21.15.175:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd23c058c10 con 0x7fd24804ffd0 2024-04-20T11:27:08.254 INFO:teuthology.orchestra.run.smithi152.stdout:55834574854 2024-04-20T11:27:08.257 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.258+0000 7fd242000640 1 -- 172.21.15.152:0/2981202700 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fd23006c940 msgr2=0x7fd23006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:08.257 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.258+0000 7fd242000640 1 --2- 172.21.15.152:0/2981202700 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fd23006c940 0x7fd23006ee00 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd244036000 tx=0x7fd244039040 comp rx=0 tx=0).stop 2024-04-20T11:27:08.257 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.258+0000 7fd242000640 1 -- 172.21.15.152:0/2981202700 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd24804ffd0 msgr2=0x7fd248034d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:08.257 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.258+0000 7fd242000640 1 --2- 172.21.15.152:0/2981202700 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd24804ffd0 0x7fd248034d80 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd23c00c450 tx=0x7fd23c00c850 comp rx=0 tx=0).stop 2024-04-20T11:27:08.258 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.258+0000 7fd242000640 1 -- 172.21.15.152:0/2981202700 shutdown_connections 2024-04-20T11:27:08.258 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.258+0000 7fd242000640 1 --2- 172.21.15.152:0/2981202700 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fd23006c940 0x7fd23006ee00 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:08.258 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.258+0000 7fd242000640 1 --2- 172.21.15.152:0/2981202700 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2480ead00 0x7fd2480d7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:08.258 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.258+0000 7fd242000640 1 --2- 172.21.15.152:0/2981202700 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd24804ffd0 0x7fd248034d80 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:08.258 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.258+0000 7fd242000640 1 --2- 172.21.15.152:0/2981202700 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd248006ea0 0x7fd24804e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:08.258 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.258+0000 7fd242000640 1 -- 172.21.15.152:0/2981202700 >> 172.21.15.152:0/2981202700 conn(0x7fd248009fb0 msgr2=0x7fd2480495d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:08.258 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.259+0000 7fd242000640 1 -- 172.21.15.152:0/2981202700 shutdown_connections 2024-04-20T11:27:08.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.259+0000 7fd242000640 1 -- 172.21.15.152:0/2981202700 wait complete. 2024-04-20T11:27:08.262 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.2 2024-04-20T11:27:08.262 DEBUG:teuthology.parallel:result is None 2024-04-20T11:27:08.262 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.263+0000 7fe86fa00640 1 -- 172.21.15.152:0/3926195852 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fe86806cd90 con 0x7fe86815e830 2024-04-20T11:27:08.263 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.264+0000 7fe864c00640 1 -- 172.21.15.152:0/3926195852 <== mon.2 v2:172.21.15.175:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe85405cf10 con 0x7fe86815e830 2024-04-20T11:27:08.263 INFO:teuthology.orchestra.run.smithi152.stdout:55834574853 2024-04-20T11:27:08.266 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.266+0000 7fe86fa00640 1 -- 172.21.15.152:0/3926195852 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe84806c970 msgr2=0x7fe84806ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:08.266 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.266+0000 7fe86fa00640 1 --2- 172.21.15.152:0/3926195852 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe84806c970 0x7fe84806ee30 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe858009630 tx=0x7fe858009240 comp rx=0 tx=0).stop 2024-04-20T11:27:08.266 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.266+0000 7fe86fa00640 1 -- 172.21.15.152:0/3926195852 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe86815e830 msgr2=0x7fe8680b14b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:08.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.266+0000 7fe86fa00640 1 --2- 172.21.15.152:0/3926195852 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe86815e830 0x7fe8680b14b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe85400c3c0 tx=0x7fe85400c880 comp rx=0 tx=0).stop 2024-04-20T11:27:08.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.266+0000 7fe86fa00640 1 -- 172.21.15.152:0/3926195852 shutdown_connections 2024-04-20T11:27:08.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.266+0000 7fe86fa00640 1 --2- 172.21.15.152:0/3926195852 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe84806c970 0x7fe84806ee30 secure :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe858009630 tx=0x7fe858009240 comp rx=0 tx=0).stop 2024-04-20T11:27:08.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.266+0000 7fe86fa00640 1 --2- 172.21.15.152:0/3926195852 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe86815e830 0x7fe8680b14b0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:08.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.266+0000 7fe86fa00640 1 --2- 172.21.15.152:0/3926195852 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe8680b3e80 0x7fe8680cf740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:08.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.266+0000 7fe86fa00640 1 --2- 172.21.15.152:0/3926195852 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe86806f060 0x7fe8681ed630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:08.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.266+0000 7fe86fa00640 1 -- 172.21.15.152:0/3926195852 >> 172.21.15.152:0/3926195852 conn(0x7fe8680678a0 msgr2=0x7fe8680b2150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:08.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.267+0000 7fe86fa00640 1 -- 172.21.15.152:0/3926195852 shutdown_connections 2024-04-20T11:27:08.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.267+0000 7fe86fa00640 1 -- 172.21.15.152:0/3926195852 wait complete. 2024-04-20T11:27:08.284 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.0 2024-04-20T11:27:08.284 DEBUG:teuthology.parallel:result is None 2024-04-20T11:27:08.290 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.6 2024-04-20T11:27:08.290 DEBUG:teuthology.parallel:result is None 2024-04-20T11:27:08.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.302+0000 7f4f9c400640 1 -- 172.21.15.152:0/4175255632 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f4f940689f0 con 0x7f4f940b46b0 2024-04-20T11:27:08.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.303+0000 7f4f91600640 1 -- 172.21.15.152:0/4175255632 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f4f8c059a40 con 0x7f4f940b46b0 2024-04-20T11:27:08.303 INFO:teuthology.orchestra.run.smithi152.stdout:55834574854 2024-04-20T11:27:08.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.306+0000 7f4f9c400640 1 -- 172.21.15.152:0/4175255632 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f4f7406c940 msgr2=0x7f4f7406ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:08.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.306+0000 7f4f9c400640 1 --2- 172.21.15.152:0/4175255632 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f4f7406c940 0x7f4f7406ee00 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4f88010950 tx=0x7f4f88034000 comp rx=0 tx=0).stop 2024-04-20T11:27:08.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.306+0000 7f4f9c400640 1 -- 172.21.15.152:0/4175255632 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4f940b46b0 msgr2=0x7f4f9415e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:08.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.306+0000 7f4f9c400640 1 --2- 172.21.15.152:0/4175255632 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4f940b46b0 0x7f4f9415e5b0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c00c730 tx=0x7f4f8c00cbf0 comp rx=0 tx=0).stop 2024-04-20T11:27:08.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.306+0000 7f4f9c400640 1 -- 172.21.15.152:0/4175255632 shutdown_connections 2024-04-20T11:27:08.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.306+0000 7f4f9c400640 1 --2- 172.21.15.152:0/4175255632 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f4f7406c940 0x7f4f7406ee00 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:08.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.306+0000 7f4f9c400640 1 --2- 172.21.15.152:0/4175255632 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4f940b46b0 0x7f4f9415e5b0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:08.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.306+0000 7f4f9c400640 1 --2- 172.21.15.152:0/4175255632 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4f9406df30 0x7f4f940cdfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:08.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.306+0000 7f4f9c400640 1 --2- 172.21.15.152:0/4175255632 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4f94067d50 0x7f4f9414d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:08.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.306+0000 7f4f9c400640 1 -- 172.21.15.152:0/4175255632 >> 172.21.15.152:0/4175255632 conn(0x7f4f940678a0 msgr2=0x7f4f94158e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:08.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.306+0000 7f4f9c400640 1 -- 172.21.15.152:0/4175255632 shutdown_connections 2024-04-20T11:27:08.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.306+0000 7f4f9c400640 1 -- 172.21.15.152:0/4175255632 wait complete. 2024-04-20T11:27:08.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.312+0000 7fd88da00640 1 -- 172.21.15.152:0/1268622032 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fd8880d1d70 con 0x7fd86c0051c0 2024-04-20T11:27:08.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.313+0000 7fd87fe00640 1 -- 172.21.15.152:0/1268622032 <== mon.1 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd874059b10 con 0x7fd86c0051c0 2024-04-20T11:27:08.313 INFO:teuthology.orchestra.run.smithi152.stdout:55834574855 2024-04-20T11:27:08.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.316+0000 7fd88da00640 1 -- 172.21.15.152:0/1268622032 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fd86806c920 msgr2=0x7fd86806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:08.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.316+0000 7fd88da00640 1 --2- 172.21.15.152:0/1268622032 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fd86806c920 0x7fd86806ede0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd878006f60 tx=0x7fd878008040 comp rx=0 tx=0).stop 2024-04-20T11:27:08.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.316+0000 7fd88da00640 1 -- 172.21.15.152:0/1268622032 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd86c0051c0 msgr2=0x7fd8882019d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:08.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.316+0000 7fd88da00640 1 --2- 172.21.15.152:0/1268622032 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd86c0051c0 0x7fd8882019d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd8880515f0 tx=0x7fd87400e900 comp rx=0 tx=0).stop 2024-04-20T11:27:08.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.316+0000 7fd88da00640 1 -- 172.21.15.152:0/1268622032 shutdown_connections 2024-04-20T11:27:08.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.316+0000 7fd88da00640 1 --2- 172.21.15.152:0/1268622032 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fd86806c920 0x7fd86806ede0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:08.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.316+0000 7fd88da00640 1 --2- 172.21.15.152:0/1268622032 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd88815d8d0 0x7fd88815dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:08.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.316+0000 7fd88da00640 1 --2- 172.21.15.152:0/1268622032 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd88815b250 0x7fd8882006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:08.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.316+0000 7fd88da00640 1 --2- 172.21.15.152:0/1268622032 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd86c0051c0 0x7fd8882019d0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:08.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.316+0000 7fd88da00640 1 -- 172.21.15.152:0/1268622032 >> 172.21.15.152:0/1268622032 conn(0x7fd88808bfd0 msgr2=0x7fd8880b31e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:08.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.316+0000 7fd88da00640 1 -- 172.21.15.152:0/1268622032 shutdown_connections 2024-04-20T11:27:08.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.316+0000 7fd88da00640 1 -- 172.21.15.152:0/1268622032 wait complete. 2024-04-20T11:27:08.322 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.3 2024-04-20T11:27:08.322 DEBUG:teuthology.parallel:result is None 2024-04-20T11:27:08.326 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.327+0000 7f003d800640 1 -- 172.21.15.152:0/2282640299 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f00380d05f0 con 0x7f001c003a00 2024-04-20T11:27:08.327 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.327+0000 7f002fe00640 1 -- 172.21.15.152:0/2282640299 <== mon.1 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f002807f2f0 con 0x7f001c003a00 2024-04-20T11:27:08.327 INFO:teuthology.orchestra.run.smithi152.stdout:55834574853 2024-04-20T11:27:08.330 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.331+0000 7f003d800640 1 -- 172.21.15.152:0/2282640299 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f001806c8f0 msgr2=0x7f001806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:08.331 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.331+0000 7f003d800640 1 --2- 172.21.15.152:0/2282640299 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f001806c8f0 0x7f001806edb0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f00380cce20 tx=0x7f003000b040 comp rx=0 tx=0).stop 2024-04-20T11:27:08.331 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.331+0000 7f003d800640 1 -- 172.21.15.152:0/2282640299 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f001c003a00 msgr2=0x7f00381f7f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:08.331 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.331+0000 7f003d800640 1 --2- 172.21.15.152:0/2282640299 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f001c003a00 0x7f00381f7f30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0028033000 tx=0x7f0028004730 comp rx=0 tx=0).stop 2024-04-20T11:27:08.331 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.331+0000 7f003d800640 1 -- 172.21.15.152:0/2282640299 shutdown_connections 2024-04-20T11:27:08.331 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.331+0000 7f003d800640 1 --2- 172.21.15.152:0/2282640299 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f001806c8f0 0x7f001806edb0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:08.331 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.331+0000 7f003d800640 1 --2- 172.21.15.152:0/2282640299 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f00380d1140 0x7f00380d15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:08.331 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.331+0000 7f003d800640 1 --2- 172.21.15.152:0/2282640299 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f00380c7640 0x7f00380cdba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:08.331 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.331+0000 7f003d800640 1 --2- 172.21.15.152:0/2282640299 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f001c003a00 0x7f00381f7f30 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:08.331 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.331+0000 7f003d800640 1 -- 172.21.15.152:0/2282640299 >> 172.21.15.152:0/2282640299 conn(0x7f00380b2e70 msgr2=0x7f003815ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:08.331 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.331+0000 7f003d800640 1 -- 172.21.15.152:0/2282640299 shutdown_connections 2024-04-20T11:27:08.331 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.331+0000 7f003d800640 1 -- 172.21.15.152:0/2282640299 wait complete. 2024-04-20T11:27:08.332 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.5 2024-04-20T11:27:08.332 DEBUG:teuthology.parallel:result is None 2024-04-20T11:27:08.354 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.4 2024-04-20T11:27:08.354 DEBUG:teuthology.parallel:result is None 2024-04-20T11:27:08.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.362+0000 7f6e93600640 1 -- 172.21.15.152:0/72581053 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f6e8c069100 con 0x7f6e8c0cd0f0 2024-04-20T11:27:08.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.362+0000 7f6e88c00640 1 -- 172.21.15.152:0/72581053 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f6e80059bd0 con 0x7f6e8c0cd0f0 2024-04-20T11:27:08.362 INFO:teuthology.orchestra.run.smithi152.stdout:55834574856 2024-04-20T11:27:08.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.364+0000 7f6e93600640 1 -- 172.21.15.152:0/72581053 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f6e6c06c9c0 msgr2=0x7f6e6c06ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:08.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.364+0000 7f6e93600640 1 --2- 172.21.15.152:0/72581053 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f6e6c06c9c0 0x7f6e6c06ee80 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f6e78036000 tx=0x7f6e78008030 comp rx=0 tx=0).stop 2024-04-20T11:27:08.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.364+0000 7f6e93600640 1 -- 172.21.15.152:0/72581053 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6e8c0cd0f0 msgr2=0x7f6e8c0cafd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:08.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.364+0000 7f6e93600640 1 --2- 172.21.15.152:0/72581053 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6e8c0cd0f0 0x7f6e8c0cafd0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6e8000c7f0 tx=0x7f6e8000ccb0 comp rx=0 tx=0).stop 2024-04-20T11:27:08.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.365+0000 7f6e93600640 1 -- 172.21.15.152:0/72581053 shutdown_connections 2024-04-20T11:27:08.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.365+0000 7f6e93600640 1 --2- 172.21.15.152:0/72581053 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f6e6c06c9c0 0x7f6e6c06ee80 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:08.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.365+0000 7f6e93600640 1 --2- 172.21.15.152:0/72581053 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e8c0cfe00 0x7f6e8c0c7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:08.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.365+0000 7f6e93600640 1 --2- 172.21.15.152:0/72581053 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e8c0cf4e0 0x7f6e8c0c6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:08.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.365+0000 7f6e93600640 1 --2- 172.21.15.152:0/72581053 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6e8c0cd0f0 0x7f6e8c0cafd0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:08.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.365+0000 7f6e93600640 1 -- 172.21.15.152:0/72581053 >> 172.21.15.152:0/72581053 conn(0x7f6e8c0b3030 msgr2=0x7f6e8c0b0e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:08.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.365+0000 7f6e93600640 1 -- 172.21.15.152:0/72581053 shutdown_connections 2024-04-20T11:27:08.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.365+0000 7f6e93600640 1 -- 172.21.15.152:0/72581053 wait complete. 2024-04-20T11:27:08.380 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.7 2024-04-20T11:27:08.380 DEBUG:teuthology.parallel:result is None 2024-04-20T11:27:08.380 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-20T11:27:08.380 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T11:27:08.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.595+0000 7fc3b6000640 1 Processor -- start 2024-04-20T11:27:08.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.596+0000 7fc3b6000640 1 -- start start 2024-04-20T11:27:08.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.597+0000 7fc3b6000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc3b015eae0 0x7fc3b00d2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:08.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.597+0000 7fc3b6000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fc3b00ba210 0x7fc3b00d1850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:08.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.597+0000 7fc3b6000640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3b00d3190 con 0x7fc3b00d1d90 2024-04-20T11:27:08.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.597+0000 7fc3b6000640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fc3b015eec0 con 0x7fc3b00ba210 2024-04-20T11:27:08.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.597+0000 7fc3b6000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc3b00d3410 con 0x7fc3b015eae0 2024-04-20T11:27:08.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.597+0000 7fc3aea00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fc3b00ba210 0x7fc3b00d1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:08.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.597+0000 7fc3aea00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fc3b00ba210 0x7fc3b00d1850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:48216/0 (socket says 172.21.15.152:48216) 2024-04-20T11:27:08.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.597+0000 7fc3aea00640 1 -- 172.21.15.152:0/1573741741 learned_addr learned my addr 172.21.15.152:0/1573741741 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:08.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.598+0000 7fc3aea00640 1 -- 172.21.15.152:0/1573741741 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc3b015eae0 msgr2=0x7fc3b00d2c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:08.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.598+0000 7fc3aea00640 1 --2- 172.21.15.152:0/1573741741 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc3b015eae0 0x7fc3b00d2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:08.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.598+0000 7fc3aea00640 1 -- 172.21.15.152:0/1573741741 >> v1:172.21.15.175:6789/0 conn(0x7fc3b00d1d90 legacy=0x7fc3b00d4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:08.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.598+0000 7fc3aea00640 1 -- 172.21.15.152:0/1573741741 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3b00d36f0 con 0x7fc3b00ba210 2024-04-20T11:27:08.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.599+0000 7fc3aea00640 1 --2- 172.21.15.152:0/1573741741 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc3b00ba210 0x7fc3b00d1850 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc398009e50 tx=0x7fc39802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=509ab9f9f9e6098b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:08.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.599+0000 7fc3ae000640 1 -- 172.21.15.152:0/1573741741 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc39802ca80 con 0x7fc3b00ba210 2024-04-20T11:27:08.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.599+0000 7fc3ae000640 1 -- 172.21.15.152:0/1573741741 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc39802cc20 con 0x7fc3b00ba210 2024-04-20T11:27:08.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.599+0000 7fc3ae000640 1 -- 172.21.15.152:0/1573741741 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3980343f0 con 0x7fc3b00ba210 2024-04-20T11:27:08.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.600+0000 7fc3b6000640 1 -- 172.21.15.152:0/1573741741 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc3b00ba210 msgr2=0x7fc3b00d1850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:08.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.600+0000 7fc3b6000640 1 --2- 172.21.15.152:0/1573741741 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc3b00ba210 0x7fc3b00d1850 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc398009e50 tx=0x7fc39802bee0 comp rx=0 tx=0).stop 2024-04-20T11:27:08.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.600+0000 7fc3b6000640 1 -- 172.21.15.152:0/1573741741 shutdown_connections 2024-04-20T11:27:08.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.600+0000 7fc3b6000640 1 --2- 172.21.15.152:0/1573741741 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc3b00ba210 0x7fc3b00d1850 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:08.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.600+0000 7fc3b6000640 1 --2- 172.21.15.152:0/1573741741 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc3b015eae0 0x7fc3b00d2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:08.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.600+0000 7fc3b6000640 1 -- 172.21.15.152:0/1573741741 >> 172.21.15.152:0/1573741741 conn(0x7fc3b006d260 msgr2=0x7fc3b006d670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:08.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.600+0000 7fc3b6000640 1 -- 172.21.15.152:0/1573741741 shutdown_connections 2024-04-20T11:27:08.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.600+0000 7fc3b6000640 1 -- 172.21.15.152:0/1573741741 wait complete. 2024-04-20T11:27:08.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.601+0000 7fc3b6000640 1 Processor -- start 2024-04-20T11:27:08.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.601+0000 7fc3b6000640 1 -- start start 2024-04-20T11:27:08.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.602+0000 7fc3b6000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc3b00ba210 0x7fc3b015f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:08.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.602+0000 7fc3b6000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc3b00d1d90 0x7fc3b015f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:08.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.602+0000 7fc3b6000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc3b015eae0 0x7fc3b015fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:08.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.602+0000 7fc3b6000640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fc3b00d3f90 con 0x7fc3b015eae0 2024-04-20T11:27:08.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.602+0000 7fc3b6000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc3b00d3cb0 con 0x7fc3b00ba210 2024-04-20T11:27:08.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.602+0000 7fc3b6000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc3b00d44b0 con 0x7fc3b00d1d90 2024-04-20T11:27:08.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.603+0000 7fc3afe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc3b015eae0 0x7fc3b015fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:08.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.603+0000 7fc3afe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc3b015eae0 0x7fc3b015fad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:48222/0 (socket says 172.21.15.152:48222) 2024-04-20T11:27:08.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.603+0000 7fc3afe00640 1 -- 172.21.15.152:0/1511880701 learned_addr learned my addr 172.21.15.152:0/1511880701 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:08.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.603+0000 7fc3aea00640 1 --2- 172.21.15.152:0/1511880701 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc3b00d1d90 0x7fc3b015f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:08.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.603+0000 7fc3af400640 1 --2- 172.21.15.152:0/1511880701 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc3b00ba210 0x7fc3b015f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:08.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.603+0000 7fc3afe00640 1 -- 172.21.15.152:0/1511880701 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc3b00d1d90 msgr2=0x7fc3b015f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:08.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.603+0000 7fc3afe00640 1 --2- 172.21.15.152:0/1511880701 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc3b00d1d90 0x7fc3b015f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:08.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.603+0000 7fc3afe00640 1 -- 172.21.15.152:0/1511880701 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc3b00ba210 msgr2=0x7fc3b015f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:08.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.603+0000 7fc3afe00640 1 --2- 172.21.15.152:0/1511880701 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc3b00ba210 0x7fc3b015f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:08.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.603+0000 7fc3afe00640 1 -- 172.21.15.152:0/1511880701 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3b01565e0 con 0x7fc3b015eae0 2024-04-20T11:27:08.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.603+0000 7fc3afe00640 1 --2- 172.21.15.152:0/1511880701 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc3b015eae0 0x7fc3b015fad0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc3a800c3c0 tx=0x7fc3a800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:08.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.604+0000 7fc3a7400640 1 -- 172.21.15.152:0/1511880701 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3a8017070 con 0x7fc3b015eae0 2024-04-20T11:27:08.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.604+0000 7fc3a7400640 1 -- 172.21.15.152:0/1511880701 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3a80035f0 con 0x7fc3b015eae0 2024-04-20T11:27:08.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.604+0000 7fc3b6000640 1 -- 172.21.15.152:0/1511880701 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3b0159360 con 0x7fc3b015eae0 2024-04-20T11:27:08.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.604+0000 7fc3a7400640 1 -- 172.21.15.152:0/1511880701 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3a80039e0 con 0x7fc3b015eae0 2024-04-20T11:27:08.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.604+0000 7fc3b6000640 1 -- 172.21.15.152:0/1511880701 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3b00b3f00 con 0x7fc3b015eae0 2024-04-20T11:27:08.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.605+0000 7fc3b6000640 1 -- 172.21.15.152:0/1511880701 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3b00689f0 con 0x7fc3b015eae0 2024-04-20T11:27:08.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.606+0000 7fc3a7400640 1 -- 172.21.15.152:0/1511880701 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc3a801a070 con 0x7fc3b015eae0 2024-04-20T11:27:08.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.607+0000 7fc3a7400640 1 --2- 172.21.15.152:0/1511880701 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc38c06c8f0 0x7fc38c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:08.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.607+0000 7fc3a7400640 1 -- 172.21.15.152:0/1511880701 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc3a808af20 con 0x7fc3b015eae0 2024-04-20T11:27:08.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.607+0000 7fc3af400640 1 --2- 172.21.15.152:0/1511880701 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc38c06c8f0 0x7fc38c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:08.607 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.608+0000 7fc3af400640 1 --2- 172.21.15.152:0/1511880701 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc38c06c8f0 0x7fc38c06edb0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc3a000a770 tx=0x7fc3a0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:08.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.611+0000 7fc3a7400640 1 -- 172.21.15.152:0/1511880701 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fc3b01565e0 con 0x7fc3b015eae0 2024-04-20T11:27:08.791 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.791+0000 7fc3b6000640 1 -- 172.21.15.152:0/1511880701 --> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc3b015ca60 con 0x7fc38c06c8f0 2024-04-20T11:27:08.809 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.810+0000 7fc3a7400640 1 -- 172.21.15.152:0/1511880701 <== mgr.4113 v2:172.21.15.152:6832/2285110240 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251836 (secure 0 0 0) 0x7fc3b015ca60 con 0x7fc38c06c8f0 2024-04-20T11:27:08.813 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:27:08.814 INFO:teuthology.orchestra.run.smithi152.stderr:dumped all 2024-04-20T11:27:08.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.815+0000 7fc3b6000640 1 -- 172.21.15.152:0/1511880701 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc38c06c8f0 msgr2=0x7fc38c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:08.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.815+0000 7fc3b6000640 1 --2- 172.21.15.152:0/1511880701 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc38c06c8f0 0x7fc38c06edb0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc3a000a770 tx=0x7fc3a0008040 comp rx=0 tx=0).stop 2024-04-20T11:27:08.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.815+0000 7fc3b6000640 1 -- 172.21.15.152:0/1511880701 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc3b015eae0 msgr2=0x7fc3b015fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:08.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.815+0000 7fc3b6000640 1 --2- 172.21.15.152:0/1511880701 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc3b015eae0 0x7fc3b015fad0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc3a800c3c0 tx=0x7fc3a800c880 comp rx=0 tx=0).stop 2024-04-20T11:27:08.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.815+0000 7fc3b6000640 1 -- 172.21.15.152:0/1511880701 shutdown_connections 2024-04-20T11:27:08.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.816+0000 7fc3b6000640 1 --2- 172.21.15.152:0/1511880701 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc38c06c8f0 0x7fc38c06edb0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:08.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.816+0000 7fc3b6000640 1 --2- 172.21.15.152:0/1511880701 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc3b015eae0 0x7fc3b015fad0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:08.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.816+0000 7fc3b6000640 1 --2- 172.21.15.152:0/1511880701 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc3b00d1d90 0x7fc3b015f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:08.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.816+0000 7fc3b6000640 1 --2- 172.21.15.152:0/1511880701 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc3b00ba210 0x7fc3b015f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:08.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.816+0000 7fc3b6000640 1 -- 172.21.15.152:0/1511880701 >> 172.21.15.152:0/1511880701 conn(0x7fc3b006d260 msgr2=0x7fc3b00b79e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:08.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.816+0000 7fc3b6000640 1 -- 172.21.15.152:0/1511880701 shutdown_connections 2024-04-20T11:27:08.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:08.816+0000 7fc3b6000640 1 -- 172.21.15.152:0/1511880701 wait complete. 2024-04-20T11:27:08.832 INFO:teuthology.orchestra.run.smithi152.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-04-20T11:27:07.214486+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219000,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755720,"statfs":{"total":773094113280,"available":772869857280,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12746,"internal_metadata":221236790},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.555998"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.185059+0000","last_change":"2024-04-20T11:26:58.706148+0000","last_active":"2024-04-20T11:27:00.185059+0000","last_peered":"2024-04-20T11:27:00.185059+0000","last_clean":"2024-04-20T11:27:00.185059+0000","last_became_active":"2024-04-20T11:26:58.705669+0000","last_became_peered":"2024-04-20T11:26:58.705669+0000","last_unstale":"2024-04-20T11:27:00.185059+0000","last_undegraded":"2024-04-20T11:27:00.185059+0000","last_fullsized":"2024-04-20T11:27:00.185059+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:25:02.202901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.187149+0000","last_change":"2024-04-20T11:26:58.688184+0000","last_active":"2024-04-20T11:27:00.187149+0000","last_peered":"2024-04-20T11:27:00.187149+0000","last_clean":"2024-04-20T11:27:00.187149+0000","last_became_active":"2024-04-20T11:26:58.685678+0000","last_became_peered":"2024-04-20T11:26:58.685678+0000","last_unstale":"2024-04-20T11:27:00.187149+0000","last_undegraded":"2024-04-20T11:27:00.187149+0000","last_fullsized":"2024-04-20T11:27:00.187149+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:30:01.981489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.692840+0000","last_change":"2024-04-20T11:26:58.675003+0000","last_active":"2024-04-20T11:26:59.692840+0000","last_peered":"2024-04-20T11:26:59.692840+0000","last_clean":"2024-04-20T11:26:59.692840+0000","last_became_active":"2024-04-20T11:26:58.674697+0000","last_became_peered":"2024-04-20T11:26:58.674697+0000","last_unstale":"2024-04-20T11:26:59.692840+0000","last_undegraded":"2024-04-20T11:26:59.692840+0000","last_fullsized":"2024-04-20T11:26:59.692840+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:48:49.870718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.697801+0000","last_change":"2024-04-20T11:26:58.678578+0000","last_active":"2024-04-20T11:26:59.697801+0000","last_peered":"2024-04-20T11:26:59.697801+0000","last_clean":"2024-04-20T11:26:59.697801+0000","last_became_active":"2024-04-20T11:26:58.677959+0000","last_became_peered":"2024-04-20T11:26:58.677959+0000","last_unstale":"2024-04-20T11:26:59.697801+0000","last_undegraded":"2024-04-20T11:26:59.697801+0000","last_fullsized":"2024-04-20T11:26:59.697801+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:27:35.003583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.185015+0000","last_change":"2024-04-20T11:26:58.703032+0000","last_active":"2024-04-20T11:27:00.185015+0000","last_peered":"2024-04-20T11:27:00.185015+0000","last_clean":"2024-04-20T11:27:00.185015+0000","last_became_active":"2024-04-20T11:26:58.702687+0000","last_became_peered":"2024-04-20T11:26:58.702687+0000","last_unstale":"2024-04-20T11:27:00.185015+0000","last_undegraded":"2024-04-20T11:27:00.185015+0000","last_fullsized":"2024-04-20T11:27:00.185015+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:35:38.308224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.698653+0000","last_change":"2024-04-20T11:26:58.692001+0000","last_active":"2024-04-20T11:26:59.698653+0000","last_peered":"2024-04-20T11:26:59.698653+0000","last_clean":"2024-04-20T11:26:59.698653+0000","last_became_active":"2024-04-20T11:26:58.689784+0000","last_became_peered":"2024-04-20T11:26:58.689784+0000","last_unstale":"2024-04-20T11:26:59.698653+0000","last_undegraded":"2024-04-20T11:26:59.698653+0000","last_fullsized":"2024-04-20T11:26:59.698653+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:35:38.308224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.184927+0000","last_change":"2024-04-20T11:26:58.700657+0000","last_active":"2024-04-20T11:27:00.184927+0000","last_peered":"2024-04-20T11:27:00.184927+0000","last_clean":"2024-04-20T11:27:00.184927+0000","last_became_active":"2024-04-20T11:26:58.700233+0000","last_became_peered":"2024-04-20T11:26:58.700233+0000","last_unstale":"2024-04-20T11:27:00.184927+0000","last_undegraded":"2024-04-20T11:27:00.184927+0000","last_fullsized":"2024-04-20T11:27:00.184927+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:40:22.418833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.183717+0000","last_change":"2024-04-20T11:26:58.691319+0000","last_active":"2024-04-20T11:27:00.183717+0000","last_peered":"2024-04-20T11:27:00.183717+0000","last_clean":"2024-04-20T11:27:00.183717+0000","last_became_active":"2024-04-20T11:26:58.690968+0000","last_became_peered":"2024-04-20T11:26:58.690968+0000","last_unstale":"2024-04-20T11:27:00.183717+0000","last_undegraded":"2024-04-20T11:27:00.183717+0000","last_fullsized":"2024-04-20T11:27:00.183717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:09:06.943774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.185058+0000","last_change":"2024-04-20T11:26:58.708815+0000","last_active":"2024-04-20T11:27:00.185058+0000","last_peered":"2024-04-20T11:27:00.185058+0000","last_clean":"2024-04-20T11:27:00.185058+0000","last_became_active":"2024-04-20T11:26:58.708527+0000","last_became_peered":"2024-04-20T11:26:58.708527+0000","last_unstale":"2024-04-20T11:27:00.185058+0000","last_undegraded":"2024-04-20T11:27:00.185058+0000","last_fullsized":"2024-04-20T11:27:00.185058+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:37:41.100182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.739853+0000","last_change":"2024-04-20T11:26:58.702277+0000","last_active":"2024-04-20T11:26:59.739853+0000","last_peered":"2024-04-20T11:26:59.739853+0000","last_clean":"2024-04-20T11:26:59.739853+0000","last_became_active":"2024-04-20T11:26:58.701978+0000","last_became_peered":"2024-04-20T11:26:58.701978+0000","last_unstale":"2024-04-20T11:26:59.739853+0000","last_undegraded":"2024-04-20T11:26:59.739853+0000","last_fullsized":"2024-04-20T11:26:59.739853+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:53:57.559834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.739719+0000","last_change":"2024-04-20T11:26:58.702193+0000","last_active":"2024-04-20T11:26:59.739719+0000","last_peered":"2024-04-20T11:26:59.739719+0000","last_clean":"2024-04-20T11:26:59.739719+0000","last_became_active":"2024-04-20T11:26:58.701865+0000","last_became_peered":"2024-04-20T11:26:58.701865+0000","last_unstale":"2024-04-20T11:26:59.739719+0000","last_undegraded":"2024-04-20T11:26:59.739719+0000","last_fullsized":"2024-04-20T11:26:59.739719+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:01:34.618021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.698023+0000","last_change":"2024-04-20T11:26:58.701101+0000","last_active":"2024-04-20T11:26:59.698023+0000","last_peered":"2024-04-20T11:26:59.698023+0000","last_clean":"2024-04-20T11:26:59.698023+0000","last_became_active":"2024-04-20T11:26:58.699917+0000","last_became_peered":"2024-04-20T11:26:58.699917+0000","last_unstale":"2024-04-20T11:26:59.698023+0000","last_undegraded":"2024-04-20T11:26:59.698023+0000","last_fullsized":"2024-04-20T11:26:59.698023+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:30:35.842066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.184787+0000","last_change":"2024-04-20T11:26:58.695384+0000","last_active":"2024-04-20T11:27:00.184787+0000","last_peered":"2024-04-20T11:27:00.184787+0000","last_clean":"2024-04-20T11:27:00.184787+0000","last_became_active":"2024-04-20T11:26:58.695086+0000","last_became_peered":"2024-04-20T11:26:58.695086+0000","last_unstale":"2024-04-20T11:27:00.184787+0000","last_undegraded":"2024-04-20T11:27:00.184787+0000","last_fullsized":"2024-04-20T11:27:00.184787+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:47:21.129095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.693173+0000","last_change":"2024-04-20T11:26:58.677865+0000","last_active":"2024-04-20T11:26:59.693173+0000","last_peered":"2024-04-20T11:26:59.693173+0000","last_clean":"2024-04-20T11:26:59.693173+0000","last_became_active":"2024-04-20T11:26:58.677541+0000","last_became_peered":"2024-04-20T11:26:58.677541+0000","last_unstale":"2024-04-20T11:26:59.693173+0000","last_undegraded":"2024-04-20T11:26:59.693173+0000","last_fullsized":"2024-04-20T11:26:59.693173+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:47:21.129095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.186987+0000","last_change":"2024-04-20T11:26:58.693039+0000","last_active":"2024-04-20T11:27:00.186987+0000","last_peered":"2024-04-20T11:27:00.186987+0000","last_clean":"2024-04-20T11:27:00.186987+0000","last_became_active":"2024-04-20T11:26:58.692020+0000","last_became_peered":"2024-04-20T11:26:58.692020+0000","last_unstale":"2024-04-20T11:27:00.186987+0000","last_undegraded":"2024-04-20T11:27:00.186987+0000","last_fullsized":"2024-04-20T11:27:00.186987+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:35:40.422299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.693509+0000","last_change":"2024-04-20T11:26:58.686574+0000","last_active":"2024-04-20T11:26:59.693509+0000","last_peered":"2024-04-20T11:26:59.693509+0000","last_clean":"2024-04-20T11:26:59.693509+0000","last_became_active":"2024-04-20T11:26:58.686297+0000","last_became_peered":"2024-04-20T11:26:58.686297+0000","last_unstale":"2024-04-20T11:26:59.693509+0000","last_undegraded":"2024-04-20T11:26:59.693509+0000","last_fullsized":"2024-04-20T11:26:59.693509+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:01:34.618021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.695098+0000","last_change":"2024-04-20T11:26:58.701917+0000","last_active":"2024-04-20T11:26:59.695098+0000","last_peered":"2024-04-20T11:26:59.695098+0000","last_clean":"2024-04-20T11:26:59.695098+0000","last_became_active":"2024-04-20T11:26:58.701625+0000","last_became_peered":"2024-04-20T11:26:58.701625+0000","last_unstale":"2024-04-20T11:26:59.695098+0000","last_undegraded":"2024-04-20T11:26:59.695098+0000","last_fullsized":"2024-04-20T11:26:59.695098+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:35:40.422299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.185755+0000","last_change":"2024-04-20T11:26:58.697089+0000","last_active":"2024-04-20T11:27:00.185755+0000","last_peered":"2024-04-20T11:27:00.185755+0000","last_clean":"2024-04-20T11:27:00.185755+0000","last_became_active":"2024-04-20T11:26:58.696794+0000","last_became_peered":"2024-04-20T11:26:58.696794+0000","last_unstale":"2024-04-20T11:27:00.185755+0000","last_undegraded":"2024-04-20T11:27:00.185755+0000","last_fullsized":"2024-04-20T11:27:00.185755+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:30:35.842066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.184901+0000","last_change":"2024-04-20T11:26:58.707870+0000","last_active":"2024-04-20T11:27:00.184901+0000","last_peered":"2024-04-20T11:27:00.184901+0000","last_clean":"2024-04-20T11:27:00.184901+0000","last_became_active":"2024-04-20T11:26:58.702746+0000","last_became_peered":"2024-04-20T11:26:58.702746+0000","last_unstale":"2024-04-20T11:27:00.184901+0000","last_undegraded":"2024-04-20T11:27:00.184901+0000","last_fullsized":"2024-04-20T11:27:00.184901+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:28:57.233788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.739521+0000","last_change":"2024-04-20T11:26:58.707789+0000","last_active":"2024-04-20T11:26:59.739521+0000","last_peered":"2024-04-20T11:26:59.739521+0000","last_clean":"2024-04-20T11:26:59.739521+0000","last_became_active":"2024-04-20T11:26:58.707272+0000","last_became_peered":"2024-04-20T11:26:58.707272+0000","last_unstale":"2024-04-20T11:26:59.739521+0000","last_undegraded":"2024-04-20T11:26:59.739521+0000","last_fullsized":"2024-04-20T11:26:59.739521+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:01:06.520823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.692973+0000","last_change":"2024-04-20T11:26:58.698142+0000","last_active":"2024-04-20T11:26:59.692973+0000","last_peered":"2024-04-20T11:26:59.692973+0000","last_clean":"2024-04-20T11:26:59.692973+0000","last_became_active":"2024-04-20T11:26:58.697869+0000","last_became_peered":"2024-04-20T11:26:58.697869+0000","last_unstale":"2024-04-20T11:26:59.692973+0000","last_undegraded":"2024-04-20T11:26:59.692973+0000","last_fullsized":"2024-04-20T11:26:59.692973+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:43:20.032648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.696504+0000","last_change":"2024-04-20T11:26:58.692191+0000","last_active":"2024-04-20T11:26:59.696504+0000","last_peered":"2024-04-20T11:26:59.696504+0000","last_clean":"2024-04-20T11:26:59.696504+0000","last_became_active":"2024-04-20T11:26:58.690512+0000","last_became_peered":"2024-04-20T11:26:58.690512+0000","last_unstale":"2024-04-20T11:26:59.696504+0000","last_undegraded":"2024-04-20T11:26:59.696504+0000","last_fullsized":"2024-04-20T11:26:59.696504+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:01:48.311110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.185102+0000","last_change":"2024-04-20T11:26:58.693404+0000","last_active":"2024-04-20T11:27:00.185102+0000","last_peered":"2024-04-20T11:27:00.185102+0000","last_clean":"2024-04-20T11:27:00.185102+0000","last_became_active":"2024-04-20T11:26:58.693146+0000","last_became_peered":"2024-04-20T11:26:58.693146+0000","last_unstale":"2024-04-20T11:27:00.185102+0000","last_undegraded":"2024-04-20T11:27:00.185102+0000","last_fullsized":"2024-04-20T11:27:00.185102+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:01:34.618021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.693752+0000","last_change":"2024-04-20T11:26:58.682815+0000","last_active":"2024-04-20T11:26:59.693752+0000","last_peered":"2024-04-20T11:26:59.693752+0000","last_clean":"2024-04-20T11:26:59.693752+0000","last_became_active":"2024-04-20T11:26:58.679077+0000","last_became_peered":"2024-04-20T11:26:58.679077+0000","last_unstale":"2024-04-20T11:26:59.693752+0000","last_undegraded":"2024-04-20T11:26:59.693752+0000","last_fullsized":"2024-04-20T11:26:59.693752+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:53:57.559834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.186319+0000","last_change":"2024-04-20T11:26:58.706958+0000","last_active":"2024-04-20T11:27:00.186319+0000","last_peered":"2024-04-20T11:27:00.186319+0000","last_clean":"2024-04-20T11:27:00.186319+0000","last_became_active":"2024-04-20T11:26:58.706682+0000","last_became_peered":"2024-04-20T11:26:58.706682+0000","last_unstale":"2024-04-20T11:27:00.186319+0000","last_undegraded":"2024-04-20T11:27:00.186319+0000","last_fullsized":"2024-04-20T11:27:00.186319+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:40:45.000805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.739905+0000","last_change":"2024-04-20T11:26:58.693542+0000","last_active":"2024-04-20T11:26:59.739905+0000","last_peered":"2024-04-20T11:26:59.739905+0000","last_clean":"2024-04-20T11:26:59.739905+0000","last_became_active":"2024-04-20T11:26:58.693091+0000","last_became_peered":"2024-04-20T11:26:58.693091+0000","last_unstale":"2024-04-20T11:26:59.739905+0000","last_undegraded":"2024-04-20T11:26:59.739905+0000","last_fullsized":"2024-04-20T11:26:59.739905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:35:40.422299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.186334+0000","last_change":"2024-04-20T11:26:58.683272+0000","last_active":"2024-04-20T11:27:00.186334+0000","last_peered":"2024-04-20T11:27:00.186334+0000","last_clean":"2024-04-20T11:27:00.186334+0000","last_became_active":"2024-04-20T11:26:58.683012+0000","last_became_peered":"2024-04-20T11:26:58.683012+0000","last_unstale":"2024-04-20T11:27:00.186334+0000","last_undegraded":"2024-04-20T11:27:00.186334+0000","last_fullsized":"2024-04-20T11:27:00.186334+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:53:57.559834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.185138+0000","last_change":"2024-04-20T11:26:58.703756+0000","last_active":"2024-04-20T11:27:00.185138+0000","last_peered":"2024-04-20T11:27:00.185138+0000","last_clean":"2024-04-20T11:27:00.185138+0000","last_became_active":"2024-04-20T11:26:58.703323+0000","last_became_peered":"2024-04-20T11:26:58.703323+0000","last_unstale":"2024-04-20T11:27:00.185138+0000","last_undegraded":"2024-04-20T11:27:00.185138+0000","last_fullsized":"2024-04-20T11:27:00.185138+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:53:57.559834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.692748+0000","last_change":"2024-04-20T11:26:58.681550+0000","last_active":"2024-04-20T11:26:59.692748+0000","last_peered":"2024-04-20T11:26:59.692748+0000","last_clean":"2024-04-20T11:26:59.692748+0000","last_became_active":"2024-04-20T11:26:58.681270+0000","last_became_peered":"2024-04-20T11:26:58.681270+0000","last_unstale":"2024-04-20T11:26:59.692748+0000","last_undegraded":"2024-04-20T11:26:59.692748+0000","last_fullsized":"2024-04-20T11:26:59.692748+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:39:04.516141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.696801+0000","last_change":"2024-04-20T11:26:58.682629+0000","last_active":"2024-04-20T11:26:59.696801+0000","last_peered":"2024-04-20T11:26:59.696801+0000","last_clean":"2024-04-20T11:26:59.696801+0000","last_became_active":"2024-04-20T11:26:58.680875+0000","last_became_peered":"2024-04-20T11:26:58.680875+0000","last_unstale":"2024-04-20T11:26:59.696801+0000","last_undegraded":"2024-04-20T11:26:59.696801+0000","last_fullsized":"2024-04-20T11:26:59.696801+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:12:41.134688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.183098+0000","last_change":"2024-04-20T11:26:58.702006+0000","last_active":"2024-04-20T11:27:00.183098+0000","last_peered":"2024-04-20T11:27:00.183098+0000","last_clean":"2024-04-20T11:27:00.183098+0000","last_became_active":"2024-04-20T11:26:58.701697+0000","last_became_peered":"2024-04-20T11:26:58.701697+0000","last_unstale":"2024-04-20T11:27:00.183098+0000","last_undegraded":"2024-04-20T11:27:00.183098+0000","last_fullsized":"2024-04-20T11:27:00.183098+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:56:23.896778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.738970+0000","last_change":"2024-04-20T11:26:58.695448+0000","last_active":"2024-04-20T11:26:59.738970+0000","last_peered":"2024-04-20T11:26:59.738970+0000","last_clean":"2024-04-20T11:26:59.738970+0000","last_became_active":"2024-04-20T11:26:58.694946+0000","last_became_peered":"2024-04-20T11:26:58.694946+0000","last_unstale":"2024-04-20T11:26:59.738970+0000","last_undegraded":"2024-04-20T11:26:59.738970+0000","last_fullsized":"2024-04-20T11:26:59.738970+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:02:15.732106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.696114+0000","last_change":"2024-04-20T11:26:58.697053+0000","last_active":"2024-04-20T11:26:59.696114+0000","last_peered":"2024-04-20T11:26:59.696114+0000","last_clean":"2024-04-20T11:26:59.696114+0000","last_became_active":"2024-04-20T11:26:58.696713+0000","last_became_peered":"2024-04-20T11:26:58.696713+0000","last_unstale":"2024-04-20T11:26:59.696114+0000","last_undegraded":"2024-04-20T11:26:59.696114+0000","last_fullsized":"2024-04-20T11:26:59.696114+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:22:15.505173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.185319+0000","last_change":"2024-04-20T11:26:57.680979+0000","last_active":"2024-04-20T11:27:00.185319+0000","last_peered":"2024-04-20T11:27:00.185319+0000","last_clean":"2024-04-20T11:27:00.185319+0000","last_became_active":"2024-04-20T11:26:57.679239+0000","last_became_peered":"2024-04-20T11:26:57.679239+0000","last_unstale":"2024-04-20T11:27:00.185319+0000","last_undegraded":"2024-04-20T11:27:00.185319+0000","last_fullsized":"2024-04-20T11:27:00.185319+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:03:42.590011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.692914+0000","last_change":"2024-04-20T11:26:58.693305+0000","last_active":"2024-04-20T11:26:59.692914+0000","last_peered":"2024-04-20T11:26:59.692914+0000","last_clean":"2024-04-20T11:26:59.692914+0000","last_became_active":"2024-04-20T11:26:58.692561+0000","last_became_peered":"2024-04-20T11:26:58.692561+0000","last_unstale":"2024-04-20T11:26:59.692914+0000","last_undegraded":"2024-04-20T11:26:59.692914+0000","last_fullsized":"2024-04-20T11:26:59.692914+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:09:06.943774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.738687+0000","last_change":"2024-04-20T11:26:57.696531+0000","last_active":"2024-04-20T11:26:59.738687+0000","last_peered":"2024-04-20T11:26:59.738687+0000","last_clean":"2024-04-20T11:26:59.738687+0000","last_became_active":"2024-04-20T11:26:57.693505+0000","last_became_peered":"2024-04-20T11:26:57.693505+0000","last_unstale":"2024-04-20T11:26:59.738687+0000","last_undegraded":"2024-04-20T11:26:59.738687+0000","last_fullsized":"2024-04-20T11:26:59.738687+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:22:14.490133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.185207+0000","last_change":"2024-04-20T11:26:57.700780+0000","last_active":"2024-04-20T11:27:00.185207+0000","last_peered":"2024-04-20T11:27:00.185207+0000","last_clean":"2024-04-20T11:27:00.185207+0000","last_became_active":"2024-04-20T11:26:57.700475+0000","last_became_peered":"2024-04-20T11:26:57.700475+0000","last_unstale":"2024-04-20T11:27:00.185207+0000","last_undegraded":"2024-04-20T11:27:00.185207+0000","last_fullsized":"2024-04-20T11:27:00.185207+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:35:40.489733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.183864+0000","last_change":"2024-04-20T11:26:58.695758+0000","last_active":"2024-04-20T11:27:00.183864+0000","last_peered":"2024-04-20T11:27:00.183864+0000","last_clean":"2024-04-20T11:27:00.183864+0000","last_became_active":"2024-04-20T11:26:58.695496+0000","last_became_peered":"2024-04-20T11:26:58.695496+0000","last_unstale":"2024-04-20T11:27:00.183864+0000","last_undegraded":"2024-04-20T11:27:00.183864+0000","last_fullsized":"2024-04-20T11:27:00.183864+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:43:20.032648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.693649+0000","last_change":"2024-04-20T11:26:57.695300+0000","last_active":"2024-04-20T11:26:59.693649+0000","last_peered":"2024-04-20T11:26:59.693649+0000","last_clean":"2024-04-20T11:26:59.693649+0000","last_became_active":"2024-04-20T11:26:57.694343+0000","last_became_peered":"2024-04-20T11:26:57.694343+0000","last_unstale":"2024-04-20T11:26:59.693649+0000","last_undegraded":"2024-04-20T11:26:59.693649+0000","last_fullsized":"2024-04-20T11:26:59.693649+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:03:42.590011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.183738+0000","last_change":"2024-04-20T11:26:58.695911+0000","last_active":"2024-04-20T11:27:00.183738+0000","last_peered":"2024-04-20T11:27:00.183738+0000","last_clean":"2024-04-20T11:27:00.183738+0000","last_became_active":"2024-04-20T11:26:58.695607+0000","last_became_peered":"2024-04-20T11:26:58.695607+0000","last_unstale":"2024-04-20T11:27:00.183738+0000","last_undegraded":"2024-04-20T11:27:00.183738+0000","last_fullsized":"2024-04-20T11:27:00.183738+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:47:21.129095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.739639+0000","last_change":"2024-04-20T11:26:57.692397+0000","last_active":"2024-04-20T11:26:59.739639+0000","last_peered":"2024-04-20T11:26:59.739639+0000","last_clean":"2024-04-20T11:26:59.739639+0000","last_became_active":"2024-04-20T11:26:57.692046+0000","last_became_peered":"2024-04-20T11:26:57.692046+0000","last_unstale":"2024-04-20T11:26:59.739639+0000","last_undegraded":"2024-04-20T11:26:59.739639+0000","last_fullsized":"2024-04-20T11:26:59.739639+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:11:19.111199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.695485+0000","last_change":"2024-04-20T11:26:58.691351+0000","last_active":"2024-04-20T11:26:59.695485+0000","last_peered":"2024-04-20T11:26:59.695485+0000","last_clean":"2024-04-20T11:26:59.695485+0000","last_became_active":"2024-04-20T11:26:58.691059+0000","last_became_peered":"2024-04-20T11:26:58.691059+0000","last_unstale":"2024-04-20T11:26:59.695485+0000","last_undegraded":"2024-04-20T11:26:59.695485+0000","last_fullsized":"2024-04-20T11:26:59.695485+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:40:45.000805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.694315+0000","last_change":"2024-04-20T11:26:57.699532+0000","last_active":"2024-04-20T11:26:59.694315+0000","last_peered":"2024-04-20T11:26:59.694315+0000","last_clean":"2024-04-20T11:26:59.694315+0000","last_became_active":"2024-04-20T11:26:57.699281+0000","last_became_peered":"2024-04-20T11:26:57.699281+0000","last_unstale":"2024-04-20T11:26:59.694315+0000","last_undegraded":"2024-04-20T11:26:59.694315+0000","last_fullsized":"2024-04-20T11:26:59.694315+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:22:14.490133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.693433+0000","last_change":"2024-04-20T11:26:58.692727+0000","last_active":"2024-04-20T11:26:59.693433+0000","last_peered":"2024-04-20T11:26:59.693433+0000","last_clean":"2024-04-20T11:26:59.693433+0000","last_became_active":"2024-04-20T11:26:58.692415+0000","last_became_peered":"2024-04-20T11:26:58.692415+0000","last_unstale":"2024-04-20T11:26:59.693433+0000","last_undegraded":"2024-04-20T11:26:59.693433+0000","last_fullsized":"2024-04-20T11:26:59.693433+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:01:34.618021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.186405+0000","last_change":"2024-04-20T11:26:57.681417+0000","last_active":"2024-04-20T11:27:00.186405+0000","last_peered":"2024-04-20T11:27:00.186405+0000","last_clean":"2024-04-20T11:27:00.186405+0000","last_became_active":"2024-04-20T11:26:57.680971+0000","last_became_peered":"2024-04-20T11:26:57.680971+0000","last_unstale":"2024-04-20T11:27:00.186405+0000","last_undegraded":"2024-04-20T11:27:00.186405+0000","last_fullsized":"2024-04-20T11:27:00.186405+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:09:13.468036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.698358+0000","last_change":"2024-04-20T11:26:58.710488+0000","last_active":"2024-04-20T11:26:59.698358+0000","last_peered":"2024-04-20T11:26:59.698358+0000","last_clean":"2024-04-20T11:26:59.698358+0000","last_became_active":"2024-04-20T11:26:58.710179+0000","last_became_peered":"2024-04-20T11:26:58.710179+0000","last_unstale":"2024-04-20T11:26:59.698358+0000","last_undegraded":"2024-04-20T11:26:59.698358+0000","last_fullsized":"2024-04-20T11:26:59.698358+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:09:14.483077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.739503+0000","last_change":"2024-04-20T11:26:57.700540+0000","last_active":"2024-04-20T11:26:59.739503+0000","last_peered":"2024-04-20T11:26:59.739503+0000","last_clean":"2024-04-20T11:26:59.739503+0000","last_became_active":"2024-04-20T11:26:57.697881+0000","last_became_peered":"2024-04-20T11:26:57.697881+0000","last_unstale":"2024-04-20T11:26:59.739503+0000","last_undegraded":"2024-04-20T11:26:59.739503+0000","last_fullsized":"2024-04-20T11:26:59.739503+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:28:39.881689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.184257+0000","last_change":"2024-04-20T11:26:58.710228+0000","last_active":"2024-04-20T11:27:00.184257+0000","last_peered":"2024-04-20T11:27:00.184257+0000","last_clean":"2024-04-20T11:27:00.184257+0000","last_became_active":"2024-04-20T11:26:58.709942+0000","last_became_peered":"2024-04-20T11:26:58.709942+0000","last_unstale":"2024-04-20T11:27:00.184257+0000","last_undegraded":"2024-04-20T11:27:00.184257+0000","last_fullsized":"2024-04-20T11:27:00.184257+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:01:48.311110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.692978+0000","last_change":"2024-04-20T11:26:57.691897+0000","last_active":"2024-04-20T11:26:59.692978+0000","last_peered":"2024-04-20T11:26:59.692978+0000","last_clean":"2024-04-20T11:26:59.692978+0000","last_became_active":"2024-04-20T11:26:57.690649+0000","last_became_peered":"2024-04-20T11:26:57.690649+0000","last_unstale":"2024-04-20T11:26:59.692978+0000","last_undegraded":"2024-04-20T11:26:59.692978+0000","last_fullsized":"2024-04-20T11:26:59.692978+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:11:36.407427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.693551+0000","last_change":"2024-04-20T11:26:58.696119+0000","last_active":"2024-04-20T11:26:59.693551+0000","last_peered":"2024-04-20T11:26:59.693551+0000","last_clean":"2024-04-20T11:26:59.693551+0000","last_became_active":"2024-04-20T11:26:58.695849+0000","last_became_peered":"2024-04-20T11:26:58.695849+0000","last_unstale":"2024-04-20T11:26:59.693551+0000","last_undegraded":"2024-04-20T11:26:59.693551+0000","last_fullsized":"2024-04-20T11:26:59.693551+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:53:57.559834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.696904+0000","last_change":"2024-04-20T11:26:57.693789+0000","last_active":"2024-04-20T11:26:59.696904+0000","last_peered":"2024-04-20T11:26:59.696904+0000","last_clean":"2024-04-20T11:26:59.696904+0000","last_became_active":"2024-04-20T11:26:57.688385+0000","last_became_peered":"2024-04-20T11:26:57.688385+0000","last_unstale":"2024-04-20T11:26:59.696904+0000","last_undegraded":"2024-04-20T11:26:59.696904+0000","last_fullsized":"2024-04-20T11:26:59.696904+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:59:13.454290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.185068+0000","last_change":"2024-04-20T11:26:58.713408+0000","last_active":"2024-04-20T11:27:00.185068+0000","last_peered":"2024-04-20T11:27:00.185068+0000","last_clean":"2024-04-20T11:27:00.185068+0000","last_became_active":"2024-04-20T11:26:58.713119+0000","last_became_peered":"2024-04-20T11:26:58.713119+0000","last_unstale":"2024-04-20T11:27:00.185068+0000","last_undegraded":"2024-04-20T11:27:00.185068+0000","last_fullsized":"2024-04-20T11:27:00.185068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:12:24.274916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.738625+0000","last_change":"2024-04-20T11:26:57.695426+0000","last_active":"2024-04-20T11:26:59.738625+0000","last_peered":"2024-04-20T11:26:59.738625+0000","last_clean":"2024-04-20T11:26:59.738625+0000","last_became_active":"2024-04-20T11:26:57.695023+0000","last_became_peered":"2024-04-20T11:26:57.695023+0000","last_unstale":"2024-04-20T11:26:59.738625+0000","last_undegraded":"2024-04-20T11:26:59.738625+0000","last_fullsized":"2024-04-20T11:26:59.738625+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:03:42.590011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.739012+0000","last_change":"2024-04-20T11:26:58.695228+0000","last_active":"2024-04-20T11:26:59.739012+0000","last_peered":"2024-04-20T11:26:59.739012+0000","last_clean":"2024-04-20T11:26:59.739012+0000","last_became_active":"2024-04-20T11:26:58.694540+0000","last_became_peered":"2024-04-20T11:26:58.694540+0000","last_unstale":"2024-04-20T11:26:59.739012+0000","last_undegraded":"2024-04-20T11:26:59.739012+0000","last_fullsized":"2024-04-20T11:26:59.739012+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:47:21.129095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.182683+0000","last_change":"2024-04-20T11:26:57.679351+0000","last_active":"2024-04-20T11:27:00.182683+0000","last_peered":"2024-04-20T11:27:00.182683+0000","last_clean":"2024-04-20T11:27:00.182683+0000","last_became_active":"2024-04-20T11:26:57.678821+0000","last_became_peered":"2024-04-20T11:26:57.678821+0000","last_unstale":"2024-04-20T11:27:00.182683+0000","last_undegraded":"2024-04-20T11:27:00.182683+0000","last_fullsized":"2024-04-20T11:27:00.182683+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:28:39.881689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.184115+0000","last_change":"2024-04-20T11:26:58.713370+0000","last_active":"2024-04-20T11:27:00.184115+0000","last_peered":"2024-04-20T11:27:00.184115+0000","last_clean":"2024-04-20T11:27:00.184115+0000","last_became_active":"2024-04-20T11:26:58.713068+0000","last_became_peered":"2024-04-20T11:26:58.713068+0000","last_unstale":"2024-04-20T11:27:00.184115+0000","last_undegraded":"2024-04-20T11:27:00.184115+0000","last_fullsized":"2024-04-20T11:27:00.184115+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:55:47.019482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.695270+0000","last_change":"2024-04-20T11:26:57.687641+0000","last_active":"2024-04-20T11:26:59.695270+0000","last_peered":"2024-04-20T11:26:59.695270+0000","last_clean":"2024-04-20T11:26:59.695270+0000","last_became_active":"2024-04-20T11:26:57.687237+0000","last_became_peered":"2024-04-20T11:26:57.687237+0000","last_unstale":"2024-04-20T11:26:59.695270+0000","last_undegraded":"2024-04-20T11:26:59.695270+0000","last_fullsized":"2024-04-20T11:26:59.695270+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:43:12.722459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.184814+0000","last_change":"2024-04-20T11:26:58.711133+0000","last_active":"2024-04-20T11:27:00.184814+0000","last_peered":"2024-04-20T11:27:00.184814+0000","last_clean":"2024-04-20T11:27:00.184814+0000","last_became_active":"2024-04-20T11:26:58.710818+0000","last_became_peered":"2024-04-20T11:26:58.710818+0000","last_unstale":"2024-04-20T11:27:00.184814+0000","last_undegraded":"2024-04-20T11:27:00.184814+0000","last_fullsized":"2024-04-20T11:27:00.184814+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:27:35.003583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.185902+0000","last_change":"2024-04-20T11:26:57.675885+0000","last_active":"2024-04-20T11:27:00.185902+0000","last_peered":"2024-04-20T11:27:00.185902+0000","last_clean":"2024-04-20T11:27:00.185902+0000","last_became_active":"2024-04-20T11:26:57.675625+0000","last_became_peered":"2024-04-20T11:26:57.675625+0000","last_unstale":"2024-04-20T11:27:00.185902+0000","last_undegraded":"2024-04-20T11:27:00.185902+0000","last_fullsized":"2024-04-20T11:27:00.185902+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:03:42.590011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.185312+0000","last_change":"2024-04-20T11:26:58.713427+0000","last_active":"2024-04-20T11:27:00.185312+0000","last_peered":"2024-04-20T11:27:00.185312+0000","last_clean":"2024-04-20T11:27:00.185312+0000","last_became_active":"2024-04-20T11:26:58.713112+0000","last_became_peered":"2024-04-20T11:26:58.713112+0000","last_unstale":"2024-04-20T11:27:00.185312+0000","last_undegraded":"2024-04-20T11:27:00.185312+0000","last_fullsized":"2024-04-20T11:27:00.185312+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:26:45.461271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.186539+0000","last_change":"2024-04-20T11:26:57.693246+0000","last_active":"2024-04-20T11:27:00.186539+0000","last_peered":"2024-04-20T11:27:00.186539+0000","last_clean":"2024-04-20T11:27:00.186539+0000","last_became_active":"2024-04-20T11:26:57.692699+0000","last_became_peered":"2024-04-20T11:26:57.692699+0000","last_unstale":"2024-04-20T11:27:00.186539+0000","last_undegraded":"2024-04-20T11:27:00.186539+0000","last_fullsized":"2024-04-20T11:27:00.186539+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:22:14.490133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.698394+0000","last_change":"2024-04-20T11:26:58.676238+0000","last_active":"2024-04-20T11:26:59.698394+0000","last_peered":"2024-04-20T11:26:59.698394+0000","last_clean":"2024-04-20T11:26:59.698394+0000","last_became_active":"2024-04-20T11:26:58.675427+0000","last_became_peered":"2024-04-20T11:26:58.675427+0000","last_unstale":"2024-04-20T11:26:59.698394+0000","last_undegraded":"2024-04-20T11:26:59.698394+0000","last_fullsized":"2024-04-20T11:26:59.698394+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:28:57.233788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.185685+0000","last_change":"2024-04-20T11:26:57.681076+0000","last_active":"2024-04-20T11:27:00.185685+0000","last_peered":"2024-04-20T11:27:00.185685+0000","last_clean":"2024-04-20T11:27:00.185685+0000","last_became_active":"2024-04-20T11:26:57.680806+0000","last_became_peered":"2024-04-20T11:26:57.680806+0000","last_unstale":"2024-04-20T11:27:00.185685+0000","last_undegraded":"2024-04-20T11:27:00.185685+0000","last_fullsized":"2024-04-20T11:27:00.185685+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:11:19.111199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.184412+0000","last_change":"2024-04-20T11:26:58.708666+0000","last_active":"2024-04-20T11:27:00.184412+0000","last_peered":"2024-04-20T11:27:00.184412+0000","last_clean":"2024-04-20T11:27:00.184412+0000","last_became_active":"2024-04-20T11:26:58.708362+0000","last_became_peered":"2024-04-20T11:26:58.708362+0000","last_unstale":"2024-04-20T11:27:00.184412+0000","last_undegraded":"2024-04-20T11:27:00.184412+0000","last_fullsized":"2024-04-20T11:27:00.184412+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:12:41.134688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.184117+0000","last_change":"2024-04-20T11:26:48.661897+0000","last_active":"2024-04-20T11:27:00.184117+0000","last_peered":"2024-04-20T11:27:00.184117+0000","last_clean":"2024-04-20T11:27:00.184117+0000","last_became_active":"2024-04-20T11:26:48.656535+0000","last_became_peered":"2024-04-20T11:26:48.656535+0000","last_unstale":"2024-04-20T11:27:00.184117+0000","last_undegraded":"2024-04-20T11:27:00.184117+0000","last_fullsized":"2024-04-20T11:27:00.184117+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:47.543004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:47.543004+0000","last_clean_scrub_stamp":"2024-04-20T11:26:47.543004+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:58:35.527105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.694066+0000","last_change":"2024-04-20T11:26:57.680902+0000","last_active":"2024-04-20T11:26:59.694066+0000","last_peered":"2024-04-20T11:26:59.694066+0000","last_clean":"2024-04-20T11:26:59.694066+0000","last_became_active":"2024-04-20T11:26:57.680506+0000","last_became_peered":"2024-04-20T11:26:57.680506+0000","last_unstale":"2024-04-20T11:26:59.694066+0000","last_undegraded":"2024-04-20T11:26:59.694066+0000","last_fullsized":"2024-04-20T11:26:59.694066+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:28:39.881689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.185222+0000","last_change":"2024-04-20T11:26:58.697905+0000","last_active":"2024-04-20T11:27:00.185222+0000","last_peered":"2024-04-20T11:27:00.185222+0000","last_clean":"2024-04-20T11:27:00.185222+0000","last_became_active":"2024-04-20T11:26:58.697612+0000","last_became_peered":"2024-04-20T11:26:58.697612+0000","last_unstale":"2024-04-20T11:27:00.185222+0000","last_undegraded":"2024-04-20T11:27:00.185222+0000","last_fullsized":"2024-04-20T11:27:00.185222+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:30:01.981489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.183754+0000","last_change":"2024-04-20T11:26:57.697322+0000","last_active":"2024-04-20T11:27:00.183754+0000","last_peered":"2024-04-20T11:27:00.183754+0000","last_clean":"2024-04-20T11:27:00.183754+0000","last_became_active":"2024-04-20T11:26:57.696775+0000","last_became_peered":"2024-04-20T11:26:57.696775+0000","last_unstale":"2024-04-20T11:27:00.183754+0000","last_undegraded":"2024-04-20T11:27:00.183754+0000","last_fullsized":"2024-04-20T11:27:00.183754+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:30:34.827026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.692465+0000","last_change":"2024-04-20T11:26:58.690752+0000","last_active":"2024-04-20T11:26:59.692465+0000","last_peered":"2024-04-20T11:26:59.692465+0000","last_clean":"2024-04-20T11:26:59.692465+0000","last_became_active":"2024-04-20T11:26:58.690451+0000","last_became_peered":"2024-04-20T11:26:58.690451+0000","last_unstale":"2024-04-20T11:26:59.692465+0000","last_undegraded":"2024-04-20T11:26:59.692465+0000","last_fullsized":"2024-04-20T11:26:59.692465+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:33:15.328953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.184689+0000","last_change":"2024-04-20T11:26:57.680139+0000","last_active":"2024-04-20T11:27:00.184689+0000","last_peered":"2024-04-20T11:27:00.184689+0000","last_clean":"2024-04-20T11:27:00.184689+0000","last_became_active":"2024-04-20T11:26:57.679820+0000","last_became_peered":"2024-04-20T11:26:57.679820+0000","last_unstale":"2024-04-20T11:27:00.184689+0000","last_undegraded":"2024-04-20T11:27:00.184689+0000","last_fullsized":"2024-04-20T11:27:00.184689+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:35:40.489733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.184019+0000","last_change":"2024-04-20T11:26:58.677911+0000","last_active":"2024-04-20T11:27:00.184019+0000","last_peered":"2024-04-20T11:27:00.184019+0000","last_clean":"2024-04-20T11:27:00.184019+0000","last_became_active":"2024-04-20T11:26:58.677645+0000","last_became_peered":"2024-04-20T11:26:58.677645+0000","last_unstale":"2024-04-20T11:27:00.184019+0000","last_undegraded":"2024-04-20T11:27:00.184019+0000","last_fullsized":"2024-04-20T11:27:00.184019+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:01:06.520823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.697092+0000","last_change":"2024-04-20T11:26:57.676331+0000","last_active":"2024-04-20T11:26:59.697092+0000","last_peered":"2024-04-20T11:26:59.697092+0000","last_clean":"2024-04-20T11:26:59.697092+0000","last_became_active":"2024-04-20T11:26:57.675643+0000","last_became_peered":"2024-04-20T11:26:57.675643+0000","last_unstale":"2024-04-20T11:26:59.697092+0000","last_undegraded":"2024-04-20T11:26:59.697092+0000","last_fullsized":"2024-04-20T11:26:59.697092+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:11:19.111199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.697040+0000","last_change":"2024-04-20T11:26:58.701514+0000","last_active":"2024-04-20T11:26:59.697040+0000","last_peered":"2024-04-20T11:26:59.697040+0000","last_clean":"2024-04-20T11:26:59.697040+0000","last_became_active":"2024-04-20T11:26:58.692435+0000","last_became_peered":"2024-04-20T11:26:58.692435+0000","last_unstale":"2024-04-20T11:26:59.697040+0000","last_undegraded":"2024-04-20T11:26:59.697040+0000","last_fullsized":"2024-04-20T11:26:59.697040+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:56:23.896778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.185970+0000","last_change":"2024-04-20T11:26:57.681965+0000","last_active":"2024-04-20T11:27:00.185970+0000","last_peered":"2024-04-20T11:27:00.185970+0000","last_clean":"2024-04-20T11:27:00.185970+0000","last_became_active":"2024-04-20T11:26:57.681130+0000","last_became_peered":"2024-04-20T11:26:57.681130+0000","last_unstale":"2024-04-20T11:27:00.185970+0000","last_undegraded":"2024-04-20T11:27:00.185970+0000","last_fullsized":"2024-04-20T11:27:00.185970+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:59:13.454290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.695098+0000","last_change":"2024-04-20T11:26:58.687674+0000","last_active":"2024-04-20T11:26:59.695098+0000","last_peered":"2024-04-20T11:26:59.695098+0000","last_clean":"2024-04-20T11:26:59.695098+0000","last_became_active":"2024-04-20T11:26:58.687373+0000","last_became_peered":"2024-04-20T11:26:58.687373+0000","last_unstale":"2024-04-20T11:26:59.695098+0000","last_undegraded":"2024-04-20T11:26:59.695098+0000","last_fullsized":"2024-04-20T11:26:59.695098+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:30:01.981489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.184744+0000","last_change":"2024-04-20T11:26:57.700654+0000","last_active":"2024-04-20T11:27:00.184744+0000","last_peered":"2024-04-20T11:27:00.184744+0000","last_clean":"2024-04-20T11:27:00.184744+0000","last_became_active":"2024-04-20T11:26:57.698785+0000","last_became_peered":"2024-04-20T11:26:57.698785+0000","last_unstale":"2024-04-20T11:27:00.184744+0000","last_undegraded":"2024-04-20T11:27:00.184744+0000","last_fullsized":"2024-04-20T11:27:00.184744+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:39:03.501101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.185902+0000","last_change":"2024-04-20T11:26:58.680824+0000","last_active":"2024-04-20T11:27:00.185902+0000","last_peered":"2024-04-20T11:27:00.185902+0000","last_clean":"2024-04-20T11:27:00.185902+0000","last_became_active":"2024-04-20T11:26:58.680518+0000","last_became_peered":"2024-04-20T11:26:58.680518+0000","last_unstale":"2024-04-20T11:27:00.185902+0000","last_undegraded":"2024-04-20T11:27:00.185902+0000","last_fullsized":"2024-04-20T11:27:00.185902+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:01:34.618021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.187052+0000","last_change":"2024-04-20T11:26:57.695120+0000","last_active":"2024-04-20T11:27:00.187052+0000","last_peered":"2024-04-20T11:27:00.187052+0000","last_clean":"2024-04-20T11:27:00.187052+0000","last_became_active":"2024-04-20T11:26:57.694850+0000","last_became_peered":"2024-04-20T11:26:57.694850+0000","last_unstale":"2024-04-20T11:27:00.187052+0000","last_undegraded":"2024-04-20T11:27:00.187052+0000","last_fullsized":"2024-04-20T11:27:00.187052+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:56:22.881738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.186628+0000","last_change":"2024-04-20T11:26:58.704801+0000","last_active":"2024-04-20T11:27:00.186628+0000","last_peered":"2024-04-20T11:27:00.186628+0000","last_clean":"2024-04-20T11:27:00.186628+0000","last_became_active":"2024-04-20T11:26:58.703638+0000","last_became_peered":"2024-04-20T11:26:58.703638+0000","last_unstale":"2024-04-20T11:27:00.186628+0000","last_undegraded":"2024-04-20T11:27:00.186628+0000","last_fullsized":"2024-04-20T11:27:00.186628+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:55:47.019482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.739746+0000","last_change":"2024-04-20T11:26:57.702231+0000","last_active":"2024-04-20T11:26:59.739746+0000","last_peered":"2024-04-20T11:26:59.739746+0000","last_clean":"2024-04-20T11:26:59.739746+0000","last_became_active":"2024-04-20T11:26:57.701649+0000","last_became_peered":"2024-04-20T11:26:57.701649+0000","last_unstale":"2024-04-20T11:26:59.739746+0000","last_undegraded":"2024-04-20T11:26:59.739746+0000","last_fullsized":"2024-04-20T11:26:59.739746+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:59:13.454290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.693158+0000","last_change":"2024-04-20T11:26:58.685253+0000","last_active":"2024-04-20T11:26:59.693158+0000","last_peered":"2024-04-20T11:26:59.693158+0000","last_clean":"2024-04-20T11:26:59.693158+0000","last_became_active":"2024-04-20T11:26:58.684993+0000","last_became_peered":"2024-04-20T11:26:58.684993+0000","last_unstale":"2024-04-20T11:26:59.693158+0000","last_undegraded":"2024-04-20T11:26:59.693158+0000","last_fullsized":"2024-04-20T11:26:59.693158+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:02:15.732106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.184446+0000","last_change":"2024-04-20T11:26:57.697065+0000","last_active":"2024-04-20T11:27:00.184446+0000","last_peered":"2024-04-20T11:27:00.184446+0000","last_clean":"2024-04-20T11:27:00.184446+0000","last_became_active":"2024-04-20T11:26:57.696461+0000","last_became_peered":"2024-04-20T11:26:57.696461+0000","last_unstale":"2024-04-20T11:27:00.184446+0000","last_undegraded":"2024-04-20T11:27:00.184446+0000","last_fullsized":"2024-04-20T11:27:00.184446+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:56:22.881738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.695053+0000","last_change":"2024-04-20T11:26:58.685057+0000","last_active":"2024-04-20T11:26:59.695053+0000","last_peered":"2024-04-20T11:26:59.695053+0000","last_clean":"2024-04-20T11:26:59.695053+0000","last_became_active":"2024-04-20T11:26:58.684493+0000","last_became_peered":"2024-04-20T11:26:58.684493+0000","last_unstale":"2024-04-20T11:26:59.695053+0000","last_undegraded":"2024-04-20T11:26:59.695053+0000","last_fullsized":"2024-04-20T11:26:59.695053+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:59:14.469331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.185352+0000","last_change":"2024-04-20T11:26:57.690472+0000","last_active":"2024-04-20T11:27:00.185352+0000","last_peered":"2024-04-20T11:27:00.185352+0000","last_clean":"2024-04-20T11:27:00.185352+0000","last_became_active":"2024-04-20T11:26:57.689786+0000","last_became_peered":"2024-04-20T11:26:57.689786+0000","last_unstale":"2024-04-20T11:27:00.185352+0000","last_undegraded":"2024-04-20T11:27:00.185352+0000","last_fullsized":"2024-04-20T11:27:00.185352+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:30:34.827026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.184283+0000","last_change":"2024-04-20T11:26:58.677827+0000","last_active":"2024-04-20T11:27:00.184283+0000","last_peered":"2024-04-20T11:27:00.184283+0000","last_clean":"2024-04-20T11:27:00.184283+0000","last_became_active":"2024-04-20T11:26:58.677521+0000","last_became_peered":"2024-04-20T11:26:58.677521+0000","last_unstale":"2024-04-20T11:27:00.184283+0000","last_undegraded":"2024-04-20T11:27:00.184283+0000","last_fullsized":"2024-04-20T11:27:00.184283+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:02:15.732106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.185683+0000","last_change":"2024-04-20T11:26:57.673530+0000","last_active":"2024-04-20T11:27:00.185683+0000","last_peered":"2024-04-20T11:27:00.185683+0000","last_clean":"2024-04-20T11:27:00.185683+0000","last_became_active":"2024-04-20T11:26:57.671513+0000","last_became_peered":"2024-04-20T11:26:57.671513+0000","last_unstale":"2024-04-20T11:27:00.185683+0000","last_undegraded":"2024-04-20T11:27:00.185683+0000","last_fullsized":"2024-04-20T11:27:00.185683+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:11:19.111199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.185170+0000","last_change":"2024-04-20T11:26:58.702869+0000","last_active":"2024-04-20T11:27:00.185170+0000","last_peered":"2024-04-20T11:27:00.185170+0000","last_clean":"2024-04-20T11:27:00.185170+0000","last_became_active":"2024-04-20T11:26:58.698423+0000","last_became_peered":"2024-04-20T11:26:58.698423+0000","last_unstale":"2024-04-20T11:27:00.185170+0000","last_undegraded":"2024-04-20T11:27:00.185170+0000","last_fullsized":"2024-04-20T11:27:00.185170+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:35:40.422299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.185236+0000","last_change":"2024-04-20T11:26:57.699410+0000","last_active":"2024-04-20T11:27:00.185236+0000","last_peered":"2024-04-20T11:27:00.185236+0000","last_clean":"2024-04-20T11:27:00.185236+0000","last_became_active":"2024-04-20T11:26:57.697630+0000","last_became_peered":"2024-04-20T11:26:57.697630+0000","last_unstale":"2024-04-20T11:27:00.185236+0000","last_undegraded":"2024-04-20T11:27:00.185236+0000","last_fullsized":"2024-04-20T11:27:00.185236+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:18:30.073624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.738917+0000","last_change":"2024-04-20T11:26:58.711494+0000","last_active":"2024-04-20T11:26:59.738917+0000","last_peered":"2024-04-20T11:26:59.738917+0000","last_clean":"2024-04-20T11:26:59.738917+0000","last_became_active":"2024-04-20T11:26:58.711116+0000","last_became_peered":"2024-04-20T11:26:58.711116+0000","last_unstale":"2024-04-20T11:26:59.738917+0000","last_undegraded":"2024-04-20T11:26:59.738917+0000","last_fullsized":"2024-04-20T11:26:59.738917+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:43:20.032648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.695487+0000","last_change":"2024-04-20T11:26:58.701576+0000","last_active":"2024-04-20T11:26:59.695487+0000","last_peered":"2024-04-20T11:26:59.695487+0000","last_clean":"2024-04-20T11:26:59.695487+0000","last_became_active":"2024-04-20T11:26:58.701222+0000","last_became_peered":"2024-04-20T11:26:58.701222+0000","last_unstale":"2024-04-20T11:26:59.695487+0000","last_undegraded":"2024-04-20T11:26:59.695487+0000","last_fullsized":"2024-04-20T11:26:59.695487+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:55:47.019482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.184217+0000","last_change":"2024-04-20T11:26:57.701154+0000","last_active":"2024-04-20T11:27:00.184217+0000","last_peered":"2024-04-20T11:27:00.184217+0000","last_clean":"2024-04-20T11:27:00.184217+0000","last_became_active":"2024-04-20T11:26:57.700716+0000","last_became_peered":"2024-04-20T11:26:57.700716+0000","last_unstale":"2024-04-20T11:27:00.184217+0000","last_undegraded":"2024-04-20T11:27:00.184217+0000","last_fullsized":"2024-04-20T11:27:00.184217+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:09:13.468036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.698090+0000","last_change":"2024-04-20T11:26:58.706275+0000","last_active":"2024-04-20T11:26:59.698090+0000","last_peered":"2024-04-20T11:26:59.698090+0000","last_clean":"2024-04-20T11:26:59.698090+0000","last_became_active":"2024-04-20T11:26:58.705922+0000","last_became_peered":"2024-04-20T11:26:58.705922+0000","last_unstale":"2024-04-20T11:26:59.698090+0000","last_undegraded":"2024-04-20T11:26:59.698090+0000","last_fullsized":"2024-04-20T11:26:59.698090+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:40:22.418833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.184419+0000","last_change":"2024-04-20T11:26:57.701304+0000","last_active":"2024-04-20T11:27:00.184419+0000","last_peered":"2024-04-20T11:27:00.184419+0000","last_clean":"2024-04-20T11:27:00.184419+0000","last_became_active":"2024-04-20T11:26:57.700620+0000","last_became_peered":"2024-04-20T11:26:57.700620+0000","last_unstale":"2024-04-20T11:27:00.184419+0000","last_undegraded":"2024-04-20T11:27:00.184419+0000","last_fullsized":"2024-04-20T11:27:00.184419+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:33:14.313913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.693201+0000","last_change":"2024-04-20T11:26:58.688125+0000","last_active":"2024-04-20T11:26:59.693201+0000","last_peered":"2024-04-20T11:26:59.693201+0000","last_clean":"2024-04-20T11:26:59.693201+0000","last_became_active":"2024-04-20T11:26:58.687917+0000","last_became_peered":"2024-04-20T11:26:58.687917+0000","last_unstale":"2024-04-20T11:26:59.693201+0000","last_undegraded":"2024-04-20T11:26:59.693201+0000","last_fullsized":"2024-04-20T11:26:59.693201+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:01:06.520823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.185078+0000","last_change":"2024-04-20T11:26:57.671052+0000","last_active":"2024-04-20T11:27:00.185078+0000","last_peered":"2024-04-20T11:27:00.185078+0000","last_clean":"2024-04-20T11:27:00.185078+0000","last_became_active":"2024-04-20T11:26:57.670620+0000","last_became_peered":"2024-04-20T11:26:57.670620+0000","last_unstale":"2024-04-20T11:27:00.185078+0000","last_undegraded":"2024-04-20T11:27:00.185078+0000","last_fullsized":"2024-04-20T11:27:00.185078+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:11:36.407427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.184939+0000","last_change":"2024-04-20T11:26:58.701127+0000","last_active":"2024-04-20T11:27:00.184939+0000","last_peered":"2024-04-20T11:27:00.184939+0000","last_clean":"2024-04-20T11:27:00.184939+0000","last_became_active":"2024-04-20T11:26:58.700948+0000","last_became_peered":"2024-04-20T11:26:58.700948+0000","last_unstale":"2024-04-20T11:27:00.184939+0000","last_undegraded":"2024-04-20T11:27:00.184939+0000","last_fullsized":"2024-04-20T11:27:00.184939+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:01:06.520823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.693275+0000","last_change":"2024-04-20T11:26:57.692256+0000","last_active":"2024-04-20T11:26:59.693275+0000","last_peered":"2024-04-20T11:26:59.693275+0000","last_clean":"2024-04-20T11:26:59.693275+0000","last_became_active":"2024-04-20T11:26:57.691929+0000","last_became_peered":"2024-04-20T11:26:57.691929+0000","last_unstale":"2024-04-20T11:26:59.693275+0000","last_undegraded":"2024-04-20T11:26:59.693275+0000","last_fullsized":"2024-04-20T11:26:59.693275+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:34:59.927980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T11:27:08.836 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T11:27:09.061 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.062+0000 7f693be00640 1 Processor -- start 2024-04-20T11:27:09.062 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.062+0000 7f693be00640 1 -- start start 2024-04-20T11:27:09.063 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.063+0000 7f693be00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f693c13e530 0x7f693c06ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:09.063 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.064+0000 7f693be00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f693c06f340 0x7f693c140dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:09.063 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.064+0000 7f693be00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f693c141310 0x7f693c15efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:09.063 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.064+0000 7f693be00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f693c141b00 con 0x7f693c06f340 2024-04-20T11:27:09.064 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.064+0000 7f693be00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f693c0cfc50 con 0x7f693c13e530 2024-04-20T11:27:09.064 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.064+0000 7f693be00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f693c0cf0a0 con 0x7f693c141310 2024-04-20T11:27:09.064 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.064+0000 7f6932000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f693c06f340 0x7f693c140dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:09.064 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.064+0000 7f6932000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f693c06f340 0x7f693c140dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:48228/0 (socket says 172.21.15.152:48228) 2024-04-20T11:27:09.064 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.064+0000 7f6932000640 1 -- 172.21.15.152:0/2915860260 learned_addr learned my addr 172.21.15.152:0/2915860260 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:09.064 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.064+0000 7f6932000640 1 -- 172.21.15.152:0/2915860260 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f693c141310 msgr2=0x7f693c15efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:09.065 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.064+0000 7f6932000640 1 --2- 172.21.15.152:0/2915860260 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f693c141310 0x7f693c15efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:09.065 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.064+0000 7f6932000640 1 -- 172.21.15.152:0/2915860260 >> v2:172.21.15.175:3300/0 conn(0x7f693c13e530 msgr2=0x7f693c06ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:09.065 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.064+0000 7f6932000640 1 --2- 172.21.15.152:0/2915860260 >> v2:172.21.15.175:3300/0 conn(0x7f693c13e530 0x7f693c06ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:09.065 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.064+0000 7f6932000640 1 -- 172.21.15.152:0/2915860260 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f693c0d00e0 con 0x7f693c06f340 2024-04-20T11:27:09.065 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.065+0000 7f6932000640 1 --2- 172.21.15.152:0/2915860260 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f693c06f340 0x7f693c140dd0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f693c070110 tx=0x7f692402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5eb248a11ab50aeb server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:09.065 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.066+0000 7f693a000640 1 -- 172.21.15.152:0/2915860260 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6924035020 con 0x7f693c06f340 2024-04-20T11:27:09.066 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.066+0000 7f693a000640 1 -- 172.21.15.152:0/2915860260 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6924031870 con 0x7f693c06f340 2024-04-20T11:27:09.066 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.066+0000 7f693a000640 1 -- 172.21.15.152:0/2915860260 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6924031c40 con 0x7f693c06f340 2024-04-20T11:27:09.066 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.066+0000 7f693be00640 1 -- 172.21.15.152:0/2915860260 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f693c06f340 msgr2=0x7f693c140dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:09.066 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.066+0000 7f693be00640 1 --2- 172.21.15.152:0/2915860260 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f693c06f340 0x7f693c140dd0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f693c070110 tx=0x7f692402b910 comp rx=0 tx=0).stop 2024-04-20T11:27:09.066 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.066+0000 7f693be00640 1 -- 172.21.15.152:0/2915860260 shutdown_connections 2024-04-20T11:27:09.066 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.066+0000 7f693be00640 1 --2- 172.21.15.152:0/2915860260 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f693c141310 0x7f693c15efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:09.067 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.066+0000 7f693be00640 1 --2- 172.21.15.152:0/2915860260 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f693c06f340 0x7f693c140dd0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:09.067 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.066+0000 7f693be00640 1 --2- 172.21.15.152:0/2915860260 >> v2:172.21.15.175:3300/0 conn(0x7f693c13e530 0x7f693c06ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:09.067 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.066+0000 7f693be00640 1 -- 172.21.15.152:0/2915860260 >> 172.21.15.152:0/2915860260 conn(0x7f693c067ae0 msgr2=0x7f693c06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:09.067 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.067+0000 7f693be00640 1 -- 172.21.15.152:0/2915860260 shutdown_connections 2024-04-20T11:27:09.067 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.067+0000 7f693be00640 1 -- 172.21.15.152:0/2915860260 wait complete. 2024-04-20T11:27:09.068 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.068+0000 7f693be00640 1 Processor -- start 2024-04-20T11:27:09.068 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.069+0000 7f693be00640 1 -- start start 2024-04-20T11:27:09.070 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.070+0000 7f693be00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f693c06f340 0x7f693c158af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:09.070 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.070+0000 7f693be00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f693c13e530 0x7f693c1e7f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:09.070 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.070+0000 7f693be00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f693c141310 0x7f693c1e84a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:09.070 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.070+0000 7f693be00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f693c0cf8e0 con 0x7f693c06f340 2024-04-20T11:27:09.070 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.070+0000 7f693be00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f693c0cf600 con 0x7f693c141310 2024-04-20T11:27:09.070 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.070+0000 7f693be00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f693c1417a0 con 0x7f693c13e530 2024-04-20T11:27:09.070 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.070+0000 7f693aa00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f693c06f340 0x7f693c158af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:09.070 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.070+0000 7f693aa00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f693c06f340 0x7f693c158af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:48238/0 (socket says 172.21.15.152:48238) 2024-04-20T11:27:09.070 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.070+0000 7f693aa00640 1 -- 172.21.15.152:0/2026700801 learned_addr learned my addr 172.21.15.152:0/2026700801 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:09.070 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.070+0000 7f693aa00640 1 -- 172.21.15.152:0/2026700801 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f693c13e530 msgr2=0x7f693c1e7f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:09.070 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.070+0000 7f693aa00640 1 --2- 172.21.15.152:0/2026700801 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f693c13e530 0x7f693c1e7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:09.070 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.070+0000 7f693aa00640 1 -- 172.21.15.152:0/2026700801 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f693c141310 msgr2=0x7f693c1e84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:09.071 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.070+0000 7f693aa00640 1 --2- 172.21.15.152:0/2026700801 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f693c141310 0x7f693c1e84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:09.071 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.070+0000 7f693aa00640 1 -- 172.21.15.152:0/2026700801 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f693c12f5c0 con 0x7f693c06f340 2024-04-20T11:27:09.071 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.071+0000 7f693aa00640 1 --2- 172.21.15.152:0/2026700801 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f693c06f340 0x7f693c158af0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f692c00b4d0 tx=0x7f692c00b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:09.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.071+0000 7f6933400640 1 -- 172.21.15.152:0/2026700801 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f692c014070 con 0x7f693c06f340 2024-04-20T11:27:09.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.071+0000 7f6933400640 1 -- 172.21.15.152:0/2026700801 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f692c010870 con 0x7f693c06f340 2024-04-20T11:27:09.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.071+0000 7f6933400640 1 -- 172.21.15.152:0/2026700801 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f692c010c60 con 0x7f693c06f340 2024-04-20T11:27:09.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.071+0000 7f693be00640 1 -- 172.21.15.152:0/2026700801 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f693c12c720 con 0x7f693c06f340 2024-04-20T11:27:09.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.071+0000 7f693be00640 1 -- 172.21.15.152:0/2026700801 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f693c12f2e0 con 0x7f693c06f340 2024-04-20T11:27:09.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.072+0000 7f693be00640 1 -- 172.21.15.152:0/2026700801 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f693c075e60 con 0x7f693c06f340 2024-04-20T11:27:09.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.073+0000 7f6933400640 1 -- 172.21.15.152:0/2026700801 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f692c017070 con 0x7f693c06f340 2024-04-20T11:27:09.073 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.074+0000 7f6933400640 1 --2- 172.21.15.152:0/2026700801 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f691806c920 0x7f691806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:09.073 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.074+0000 7f6933400640 1 -- 172.21.15.152:0/2026700801 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f692c08b850 con 0x7f693c06f340 2024-04-20T11:27:09.074 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.074+0000 7f6932000640 1 --2- 172.21.15.152:0/2026700801 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f691806c920 0x7f691806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:09.074 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.075+0000 7f6932000640 1 --2- 172.21.15.152:0/2026700801 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f691806c920 0x7f691806ede0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6924036000 tx=0x7f6924030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:09.078 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.079+0000 7f6933400640 1 -- 172.21.15.152:0/2026700801 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f692c059d10 con 0x7f693c06f340 2024-04-20T11:27:09.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.261+0000 7f693be00640 1 -- 172.21.15.152:0/2026700801 --> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f693c12cda0 con 0x7f691806c920 2024-04-20T11:27:09.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.269+0000 7f6933400640 1 -- 172.21.15.152:0/2026700801 <== mgr.4113 v2:172.21.15.152:6832/2285110240 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251837 (secure 0 0 0) 0x7f693c12cda0 con 0x7f691806c920 2024-04-20T11:27:09.271 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:27:09.273 INFO:teuthology.orchestra.run.smithi152.stderr:dumped all 2024-04-20T11:27:09.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.275+0000 7f693be00640 1 -- 172.21.15.152:0/2026700801 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f691806c920 msgr2=0x7f691806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:09.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.275+0000 7f693be00640 1 --2- 172.21.15.152:0/2026700801 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f691806c920 0x7f691806ede0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6924036000 tx=0x7f6924030000 comp rx=0 tx=0).stop 2024-04-20T11:27:09.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.275+0000 7f693be00640 1 -- 172.21.15.152:0/2026700801 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f693c06f340 msgr2=0x7f693c158af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:09.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.275+0000 7f693be00640 1 --2- 172.21.15.152:0/2026700801 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f693c06f340 0x7f693c158af0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f692c00b4d0 tx=0x7f692c00b990 comp rx=0 tx=0).stop 2024-04-20T11:27:09.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.275+0000 7f693be00640 1 -- 172.21.15.152:0/2026700801 shutdown_connections 2024-04-20T11:27:09.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.275+0000 7f693be00640 1 --2- 172.21.15.152:0/2026700801 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f691806c920 0x7f691806ede0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:09.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.275+0000 7f693be00640 1 --2- 172.21.15.152:0/2026700801 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f693c141310 0x7f693c1e84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:09.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.275+0000 7f693be00640 1 --2- 172.21.15.152:0/2026700801 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f693c13e530 0x7f693c1e7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:09.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.275+0000 7f693be00640 1 --2- 172.21.15.152:0/2026700801 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f693c06f340 0x7f693c158af0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:09.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.275+0000 7f693be00640 1 -- 172.21.15.152:0/2026700801 >> 172.21.15.152:0/2026700801 conn(0x7f693c067ae0 msgr2=0x7f693c15d320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:09.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.275+0000 7f693be00640 1 -- 172.21.15.152:0/2026700801 shutdown_connections 2024-04-20T11:27:09.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.275+0000 7f693be00640 1 -- 172.21.15.152:0/2026700801 wait complete. 2024-04-20T11:27:09.292 INFO:teuthology.orchestra.run.smithi152.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-04-20T11:27:09.215453+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219000,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755720,"statfs":{"total":773094113280,"available":772869857280,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12746,"internal_metadata":221236790},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.003101"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.185059+0000","last_change":"2024-04-20T11:26:58.706148+0000","last_active":"2024-04-20T11:27:00.185059+0000","last_peered":"2024-04-20T11:27:00.185059+0000","last_clean":"2024-04-20T11:27:00.185059+0000","last_became_active":"2024-04-20T11:26:58.705669+0000","last_became_peered":"2024-04-20T11:26:58.705669+0000","last_unstale":"2024-04-20T11:27:00.185059+0000","last_undegraded":"2024-04-20T11:27:00.185059+0000","last_fullsized":"2024-04-20T11:27:00.185059+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:25:02.202901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.187149+0000","last_change":"2024-04-20T11:26:58.688184+0000","last_active":"2024-04-20T11:27:00.187149+0000","last_peered":"2024-04-20T11:27:00.187149+0000","last_clean":"2024-04-20T11:27:00.187149+0000","last_became_active":"2024-04-20T11:26:58.685678+0000","last_became_peered":"2024-04-20T11:26:58.685678+0000","last_unstale":"2024-04-20T11:27:00.187149+0000","last_undegraded":"2024-04-20T11:27:00.187149+0000","last_fullsized":"2024-04-20T11:27:00.187149+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:30:01.981489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.692840+0000","last_change":"2024-04-20T11:26:58.675003+0000","last_active":"2024-04-20T11:26:59.692840+0000","last_peered":"2024-04-20T11:26:59.692840+0000","last_clean":"2024-04-20T11:26:59.692840+0000","last_became_active":"2024-04-20T11:26:58.674697+0000","last_became_peered":"2024-04-20T11:26:58.674697+0000","last_unstale":"2024-04-20T11:26:59.692840+0000","last_undegraded":"2024-04-20T11:26:59.692840+0000","last_fullsized":"2024-04-20T11:26:59.692840+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:48:49.870718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.697801+0000","last_change":"2024-04-20T11:26:58.678578+0000","last_active":"2024-04-20T11:26:59.697801+0000","last_peered":"2024-04-20T11:26:59.697801+0000","last_clean":"2024-04-20T11:26:59.697801+0000","last_became_active":"2024-04-20T11:26:58.677959+0000","last_became_peered":"2024-04-20T11:26:58.677959+0000","last_unstale":"2024-04-20T11:26:59.697801+0000","last_undegraded":"2024-04-20T11:26:59.697801+0000","last_fullsized":"2024-04-20T11:26:59.697801+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:27:35.003583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.185015+0000","last_change":"2024-04-20T11:26:58.703032+0000","last_active":"2024-04-20T11:27:00.185015+0000","last_peered":"2024-04-20T11:27:00.185015+0000","last_clean":"2024-04-20T11:27:00.185015+0000","last_became_active":"2024-04-20T11:26:58.702687+0000","last_became_peered":"2024-04-20T11:26:58.702687+0000","last_unstale":"2024-04-20T11:27:00.185015+0000","last_undegraded":"2024-04-20T11:27:00.185015+0000","last_fullsized":"2024-04-20T11:27:00.185015+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:35:38.308224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.698653+0000","last_change":"2024-04-20T11:26:58.692001+0000","last_active":"2024-04-20T11:26:59.698653+0000","last_peered":"2024-04-20T11:26:59.698653+0000","last_clean":"2024-04-20T11:26:59.698653+0000","last_became_active":"2024-04-20T11:26:58.689784+0000","last_became_peered":"2024-04-20T11:26:58.689784+0000","last_unstale":"2024-04-20T11:26:59.698653+0000","last_undegraded":"2024-04-20T11:26:59.698653+0000","last_fullsized":"2024-04-20T11:26:59.698653+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:35:38.308224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.184927+0000","last_change":"2024-04-20T11:26:58.700657+0000","last_active":"2024-04-20T11:27:00.184927+0000","last_peered":"2024-04-20T11:27:00.184927+0000","last_clean":"2024-04-20T11:27:00.184927+0000","last_became_active":"2024-04-20T11:26:58.700233+0000","last_became_peered":"2024-04-20T11:26:58.700233+0000","last_unstale":"2024-04-20T11:27:00.184927+0000","last_undegraded":"2024-04-20T11:27:00.184927+0000","last_fullsized":"2024-04-20T11:27:00.184927+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:40:22.418833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.183717+0000","last_change":"2024-04-20T11:26:58.691319+0000","last_active":"2024-04-20T11:27:00.183717+0000","last_peered":"2024-04-20T11:27:00.183717+0000","last_clean":"2024-04-20T11:27:00.183717+0000","last_became_active":"2024-04-20T11:26:58.690968+0000","last_became_peered":"2024-04-20T11:26:58.690968+0000","last_unstale":"2024-04-20T11:27:00.183717+0000","last_undegraded":"2024-04-20T11:27:00.183717+0000","last_fullsized":"2024-04-20T11:27:00.183717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:09:06.943774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.185058+0000","last_change":"2024-04-20T11:26:58.708815+0000","last_active":"2024-04-20T11:27:00.185058+0000","last_peered":"2024-04-20T11:27:00.185058+0000","last_clean":"2024-04-20T11:27:00.185058+0000","last_became_active":"2024-04-20T11:26:58.708527+0000","last_became_peered":"2024-04-20T11:26:58.708527+0000","last_unstale":"2024-04-20T11:27:00.185058+0000","last_undegraded":"2024-04-20T11:27:00.185058+0000","last_fullsized":"2024-04-20T11:27:00.185058+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:37:41.100182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.739853+0000","last_change":"2024-04-20T11:26:58.702277+0000","last_active":"2024-04-20T11:26:59.739853+0000","last_peered":"2024-04-20T11:26:59.739853+0000","last_clean":"2024-04-20T11:26:59.739853+0000","last_became_active":"2024-04-20T11:26:58.701978+0000","last_became_peered":"2024-04-20T11:26:58.701978+0000","last_unstale":"2024-04-20T11:26:59.739853+0000","last_undegraded":"2024-04-20T11:26:59.739853+0000","last_fullsized":"2024-04-20T11:26:59.739853+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:53:57.559834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.739719+0000","last_change":"2024-04-20T11:26:58.702193+0000","last_active":"2024-04-20T11:26:59.739719+0000","last_peered":"2024-04-20T11:26:59.739719+0000","last_clean":"2024-04-20T11:26:59.739719+0000","last_became_active":"2024-04-20T11:26:58.701865+0000","last_became_peered":"2024-04-20T11:26:58.701865+0000","last_unstale":"2024-04-20T11:26:59.739719+0000","last_undegraded":"2024-04-20T11:26:59.739719+0000","last_fullsized":"2024-04-20T11:26:59.739719+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:01:34.618021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.698023+0000","last_change":"2024-04-20T11:26:58.701101+0000","last_active":"2024-04-20T11:26:59.698023+0000","last_peered":"2024-04-20T11:26:59.698023+0000","last_clean":"2024-04-20T11:26:59.698023+0000","last_became_active":"2024-04-20T11:26:58.699917+0000","last_became_peered":"2024-04-20T11:26:58.699917+0000","last_unstale":"2024-04-20T11:26:59.698023+0000","last_undegraded":"2024-04-20T11:26:59.698023+0000","last_fullsized":"2024-04-20T11:26:59.698023+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:30:35.842066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.184787+0000","last_change":"2024-04-20T11:26:58.695384+0000","last_active":"2024-04-20T11:27:00.184787+0000","last_peered":"2024-04-20T11:27:00.184787+0000","last_clean":"2024-04-20T11:27:00.184787+0000","last_became_active":"2024-04-20T11:26:58.695086+0000","last_became_peered":"2024-04-20T11:26:58.695086+0000","last_unstale":"2024-04-20T11:27:00.184787+0000","last_undegraded":"2024-04-20T11:27:00.184787+0000","last_fullsized":"2024-04-20T11:27:00.184787+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:47:21.129095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.693173+0000","last_change":"2024-04-20T11:26:58.677865+0000","last_active":"2024-04-20T11:26:59.693173+0000","last_peered":"2024-04-20T11:26:59.693173+0000","last_clean":"2024-04-20T11:26:59.693173+0000","last_became_active":"2024-04-20T11:26:58.677541+0000","last_became_peered":"2024-04-20T11:26:58.677541+0000","last_unstale":"2024-04-20T11:26:59.693173+0000","last_undegraded":"2024-04-20T11:26:59.693173+0000","last_fullsized":"2024-04-20T11:26:59.693173+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:47:21.129095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.186987+0000","last_change":"2024-04-20T11:26:58.693039+0000","last_active":"2024-04-20T11:27:00.186987+0000","last_peered":"2024-04-20T11:27:00.186987+0000","last_clean":"2024-04-20T11:27:00.186987+0000","last_became_active":"2024-04-20T11:26:58.692020+0000","last_became_peered":"2024-04-20T11:26:58.692020+0000","last_unstale":"2024-04-20T11:27:00.186987+0000","last_undegraded":"2024-04-20T11:27:00.186987+0000","last_fullsized":"2024-04-20T11:27:00.186987+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:35:40.422299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.693509+0000","last_change":"2024-04-20T11:26:58.686574+0000","last_active":"2024-04-20T11:26:59.693509+0000","last_peered":"2024-04-20T11:26:59.693509+0000","last_clean":"2024-04-20T11:26:59.693509+0000","last_became_active":"2024-04-20T11:26:58.686297+0000","last_became_peered":"2024-04-20T11:26:58.686297+0000","last_unstale":"2024-04-20T11:26:59.693509+0000","last_undegraded":"2024-04-20T11:26:59.693509+0000","last_fullsized":"2024-04-20T11:26:59.693509+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:01:34.618021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.695098+0000","last_change":"2024-04-20T11:26:58.701917+0000","last_active":"2024-04-20T11:26:59.695098+0000","last_peered":"2024-04-20T11:26:59.695098+0000","last_clean":"2024-04-20T11:26:59.695098+0000","last_became_active":"2024-04-20T11:26:58.701625+0000","last_became_peered":"2024-04-20T11:26:58.701625+0000","last_unstale":"2024-04-20T11:26:59.695098+0000","last_undegraded":"2024-04-20T11:26:59.695098+0000","last_fullsized":"2024-04-20T11:26:59.695098+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:35:40.422299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.185755+0000","last_change":"2024-04-20T11:26:58.697089+0000","last_active":"2024-04-20T11:27:00.185755+0000","last_peered":"2024-04-20T11:27:00.185755+0000","last_clean":"2024-04-20T11:27:00.185755+0000","last_became_active":"2024-04-20T11:26:58.696794+0000","last_became_peered":"2024-04-20T11:26:58.696794+0000","last_unstale":"2024-04-20T11:27:00.185755+0000","last_undegraded":"2024-04-20T11:27:00.185755+0000","last_fullsized":"2024-04-20T11:27:00.185755+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:30:35.842066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.184901+0000","last_change":"2024-04-20T11:26:58.707870+0000","last_active":"2024-04-20T11:27:00.184901+0000","last_peered":"2024-04-20T11:27:00.184901+0000","last_clean":"2024-04-20T11:27:00.184901+0000","last_became_active":"2024-04-20T11:26:58.702746+0000","last_became_peered":"2024-04-20T11:26:58.702746+0000","last_unstale":"2024-04-20T11:27:00.184901+0000","last_undegraded":"2024-04-20T11:27:00.184901+0000","last_fullsized":"2024-04-20T11:27:00.184901+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:28:57.233788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.739521+0000","last_change":"2024-04-20T11:26:58.707789+0000","last_active":"2024-04-20T11:26:59.739521+0000","last_peered":"2024-04-20T11:26:59.739521+0000","last_clean":"2024-04-20T11:26:59.739521+0000","last_became_active":"2024-04-20T11:26:58.707272+0000","last_became_peered":"2024-04-20T11:26:58.707272+0000","last_unstale":"2024-04-20T11:26:59.739521+0000","last_undegraded":"2024-04-20T11:26:59.739521+0000","last_fullsized":"2024-04-20T11:26:59.739521+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:01:06.520823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.692973+0000","last_change":"2024-04-20T11:26:58.698142+0000","last_active":"2024-04-20T11:26:59.692973+0000","last_peered":"2024-04-20T11:26:59.692973+0000","last_clean":"2024-04-20T11:26:59.692973+0000","last_became_active":"2024-04-20T11:26:58.697869+0000","last_became_peered":"2024-04-20T11:26:58.697869+0000","last_unstale":"2024-04-20T11:26:59.692973+0000","last_undegraded":"2024-04-20T11:26:59.692973+0000","last_fullsized":"2024-04-20T11:26:59.692973+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:43:20.032648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.696504+0000","last_change":"2024-04-20T11:26:58.692191+0000","last_active":"2024-04-20T11:26:59.696504+0000","last_peered":"2024-04-20T11:26:59.696504+0000","last_clean":"2024-04-20T11:26:59.696504+0000","last_became_active":"2024-04-20T11:26:58.690512+0000","last_became_peered":"2024-04-20T11:26:58.690512+0000","last_unstale":"2024-04-20T11:26:59.696504+0000","last_undegraded":"2024-04-20T11:26:59.696504+0000","last_fullsized":"2024-04-20T11:26:59.696504+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:01:48.311110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.185102+0000","last_change":"2024-04-20T11:26:58.693404+0000","last_active":"2024-04-20T11:27:00.185102+0000","last_peered":"2024-04-20T11:27:00.185102+0000","last_clean":"2024-04-20T11:27:00.185102+0000","last_became_active":"2024-04-20T11:26:58.693146+0000","last_became_peered":"2024-04-20T11:26:58.693146+0000","last_unstale":"2024-04-20T11:27:00.185102+0000","last_undegraded":"2024-04-20T11:27:00.185102+0000","last_fullsized":"2024-04-20T11:27:00.185102+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:01:34.618021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.693752+0000","last_change":"2024-04-20T11:26:58.682815+0000","last_active":"2024-04-20T11:26:59.693752+0000","last_peered":"2024-04-20T11:26:59.693752+0000","last_clean":"2024-04-20T11:26:59.693752+0000","last_became_active":"2024-04-20T11:26:58.679077+0000","last_became_peered":"2024-04-20T11:26:58.679077+0000","last_unstale":"2024-04-20T11:26:59.693752+0000","last_undegraded":"2024-04-20T11:26:59.693752+0000","last_fullsized":"2024-04-20T11:26:59.693752+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:53:57.559834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.186319+0000","last_change":"2024-04-20T11:26:58.706958+0000","last_active":"2024-04-20T11:27:00.186319+0000","last_peered":"2024-04-20T11:27:00.186319+0000","last_clean":"2024-04-20T11:27:00.186319+0000","last_became_active":"2024-04-20T11:26:58.706682+0000","last_became_peered":"2024-04-20T11:26:58.706682+0000","last_unstale":"2024-04-20T11:27:00.186319+0000","last_undegraded":"2024-04-20T11:27:00.186319+0000","last_fullsized":"2024-04-20T11:27:00.186319+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:40:45.000805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.739905+0000","last_change":"2024-04-20T11:26:58.693542+0000","last_active":"2024-04-20T11:26:59.739905+0000","last_peered":"2024-04-20T11:26:59.739905+0000","last_clean":"2024-04-20T11:26:59.739905+0000","last_became_active":"2024-04-20T11:26:58.693091+0000","last_became_peered":"2024-04-20T11:26:58.693091+0000","last_unstale":"2024-04-20T11:26:59.739905+0000","last_undegraded":"2024-04-20T11:26:59.739905+0000","last_fullsized":"2024-04-20T11:26:59.739905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:35:40.422299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.186334+0000","last_change":"2024-04-20T11:26:58.683272+0000","last_active":"2024-04-20T11:27:00.186334+0000","last_peered":"2024-04-20T11:27:00.186334+0000","last_clean":"2024-04-20T11:27:00.186334+0000","last_became_active":"2024-04-20T11:26:58.683012+0000","last_became_peered":"2024-04-20T11:26:58.683012+0000","last_unstale":"2024-04-20T11:27:00.186334+0000","last_undegraded":"2024-04-20T11:27:00.186334+0000","last_fullsized":"2024-04-20T11:27:00.186334+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:53:57.559834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.185138+0000","last_change":"2024-04-20T11:26:58.703756+0000","last_active":"2024-04-20T11:27:00.185138+0000","last_peered":"2024-04-20T11:27:00.185138+0000","last_clean":"2024-04-20T11:27:00.185138+0000","last_became_active":"2024-04-20T11:26:58.703323+0000","last_became_peered":"2024-04-20T11:26:58.703323+0000","last_unstale":"2024-04-20T11:27:00.185138+0000","last_undegraded":"2024-04-20T11:27:00.185138+0000","last_fullsized":"2024-04-20T11:27:00.185138+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:53:57.559834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.692748+0000","last_change":"2024-04-20T11:26:58.681550+0000","last_active":"2024-04-20T11:26:59.692748+0000","last_peered":"2024-04-20T11:26:59.692748+0000","last_clean":"2024-04-20T11:26:59.692748+0000","last_became_active":"2024-04-20T11:26:58.681270+0000","last_became_peered":"2024-04-20T11:26:58.681270+0000","last_unstale":"2024-04-20T11:26:59.692748+0000","last_undegraded":"2024-04-20T11:26:59.692748+0000","last_fullsized":"2024-04-20T11:26:59.692748+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:39:04.516141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.696801+0000","last_change":"2024-04-20T11:26:58.682629+0000","last_active":"2024-04-20T11:26:59.696801+0000","last_peered":"2024-04-20T11:26:59.696801+0000","last_clean":"2024-04-20T11:26:59.696801+0000","last_became_active":"2024-04-20T11:26:58.680875+0000","last_became_peered":"2024-04-20T11:26:58.680875+0000","last_unstale":"2024-04-20T11:26:59.696801+0000","last_undegraded":"2024-04-20T11:26:59.696801+0000","last_fullsized":"2024-04-20T11:26:59.696801+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:12:41.134688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.183098+0000","last_change":"2024-04-20T11:26:58.702006+0000","last_active":"2024-04-20T11:27:00.183098+0000","last_peered":"2024-04-20T11:27:00.183098+0000","last_clean":"2024-04-20T11:27:00.183098+0000","last_became_active":"2024-04-20T11:26:58.701697+0000","last_became_peered":"2024-04-20T11:26:58.701697+0000","last_unstale":"2024-04-20T11:27:00.183098+0000","last_undegraded":"2024-04-20T11:27:00.183098+0000","last_fullsized":"2024-04-20T11:27:00.183098+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:56:23.896778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.738970+0000","last_change":"2024-04-20T11:26:58.695448+0000","last_active":"2024-04-20T11:26:59.738970+0000","last_peered":"2024-04-20T11:26:59.738970+0000","last_clean":"2024-04-20T11:26:59.738970+0000","last_became_active":"2024-04-20T11:26:58.694946+0000","last_became_peered":"2024-04-20T11:26:58.694946+0000","last_unstale":"2024-04-20T11:26:59.738970+0000","last_undegraded":"2024-04-20T11:26:59.738970+0000","last_fullsized":"2024-04-20T11:26:59.738970+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:02:15.732106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.696114+0000","last_change":"2024-04-20T11:26:58.697053+0000","last_active":"2024-04-20T11:26:59.696114+0000","last_peered":"2024-04-20T11:26:59.696114+0000","last_clean":"2024-04-20T11:26:59.696114+0000","last_became_active":"2024-04-20T11:26:58.696713+0000","last_became_peered":"2024-04-20T11:26:58.696713+0000","last_unstale":"2024-04-20T11:26:59.696114+0000","last_undegraded":"2024-04-20T11:26:59.696114+0000","last_fullsized":"2024-04-20T11:26:59.696114+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:22:15.505173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.185319+0000","last_change":"2024-04-20T11:26:57.680979+0000","last_active":"2024-04-20T11:27:00.185319+0000","last_peered":"2024-04-20T11:27:00.185319+0000","last_clean":"2024-04-20T11:27:00.185319+0000","last_became_active":"2024-04-20T11:26:57.679239+0000","last_became_peered":"2024-04-20T11:26:57.679239+0000","last_unstale":"2024-04-20T11:27:00.185319+0000","last_undegraded":"2024-04-20T11:27:00.185319+0000","last_fullsized":"2024-04-20T11:27:00.185319+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:03:42.590011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.692914+0000","last_change":"2024-04-20T11:26:58.693305+0000","last_active":"2024-04-20T11:26:59.692914+0000","last_peered":"2024-04-20T11:26:59.692914+0000","last_clean":"2024-04-20T11:26:59.692914+0000","last_became_active":"2024-04-20T11:26:58.692561+0000","last_became_peered":"2024-04-20T11:26:58.692561+0000","last_unstale":"2024-04-20T11:26:59.692914+0000","last_undegraded":"2024-04-20T11:26:59.692914+0000","last_fullsized":"2024-04-20T11:26:59.692914+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:09:06.943774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.738687+0000","last_change":"2024-04-20T11:26:57.696531+0000","last_active":"2024-04-20T11:26:59.738687+0000","last_peered":"2024-04-20T11:26:59.738687+0000","last_clean":"2024-04-20T11:26:59.738687+0000","last_became_active":"2024-04-20T11:26:57.693505+0000","last_became_peered":"2024-04-20T11:26:57.693505+0000","last_unstale":"2024-04-20T11:26:59.738687+0000","last_undegraded":"2024-04-20T11:26:59.738687+0000","last_fullsized":"2024-04-20T11:26:59.738687+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:22:14.490133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.185207+0000","last_change":"2024-04-20T11:26:57.700780+0000","last_active":"2024-04-20T11:27:00.185207+0000","last_peered":"2024-04-20T11:27:00.185207+0000","last_clean":"2024-04-20T11:27:00.185207+0000","last_became_active":"2024-04-20T11:26:57.700475+0000","last_became_peered":"2024-04-20T11:26:57.700475+0000","last_unstale":"2024-04-20T11:27:00.185207+0000","last_undegraded":"2024-04-20T11:27:00.185207+0000","last_fullsized":"2024-04-20T11:27:00.185207+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:35:40.489733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.183864+0000","last_change":"2024-04-20T11:26:58.695758+0000","last_active":"2024-04-20T11:27:00.183864+0000","last_peered":"2024-04-20T11:27:00.183864+0000","last_clean":"2024-04-20T11:27:00.183864+0000","last_became_active":"2024-04-20T11:26:58.695496+0000","last_became_peered":"2024-04-20T11:26:58.695496+0000","last_unstale":"2024-04-20T11:27:00.183864+0000","last_undegraded":"2024-04-20T11:27:00.183864+0000","last_fullsized":"2024-04-20T11:27:00.183864+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:43:20.032648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.693649+0000","last_change":"2024-04-20T11:26:57.695300+0000","last_active":"2024-04-20T11:26:59.693649+0000","last_peered":"2024-04-20T11:26:59.693649+0000","last_clean":"2024-04-20T11:26:59.693649+0000","last_became_active":"2024-04-20T11:26:57.694343+0000","last_became_peered":"2024-04-20T11:26:57.694343+0000","last_unstale":"2024-04-20T11:26:59.693649+0000","last_undegraded":"2024-04-20T11:26:59.693649+0000","last_fullsized":"2024-04-20T11:26:59.693649+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:03:42.590011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.183738+0000","last_change":"2024-04-20T11:26:58.695911+0000","last_active":"2024-04-20T11:27:00.183738+0000","last_peered":"2024-04-20T11:27:00.183738+0000","last_clean":"2024-04-20T11:27:00.183738+0000","last_became_active":"2024-04-20T11:26:58.695607+0000","last_became_peered":"2024-04-20T11:26:58.695607+0000","last_unstale":"2024-04-20T11:27:00.183738+0000","last_undegraded":"2024-04-20T11:27:00.183738+0000","last_fullsized":"2024-04-20T11:27:00.183738+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:47:21.129095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.739639+0000","last_change":"2024-04-20T11:26:57.692397+0000","last_active":"2024-04-20T11:26:59.739639+0000","last_peered":"2024-04-20T11:26:59.739639+0000","last_clean":"2024-04-20T11:26:59.739639+0000","last_became_active":"2024-04-20T11:26:57.692046+0000","last_became_peered":"2024-04-20T11:26:57.692046+0000","last_unstale":"2024-04-20T11:26:59.739639+0000","last_undegraded":"2024-04-20T11:26:59.739639+0000","last_fullsized":"2024-04-20T11:26:59.739639+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:11:19.111199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.695485+0000","last_change":"2024-04-20T11:26:58.691351+0000","last_active":"2024-04-20T11:26:59.695485+0000","last_peered":"2024-04-20T11:26:59.695485+0000","last_clean":"2024-04-20T11:26:59.695485+0000","last_became_active":"2024-04-20T11:26:58.691059+0000","last_became_peered":"2024-04-20T11:26:58.691059+0000","last_unstale":"2024-04-20T11:26:59.695485+0000","last_undegraded":"2024-04-20T11:26:59.695485+0000","last_fullsized":"2024-04-20T11:26:59.695485+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:40:45.000805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.694315+0000","last_change":"2024-04-20T11:26:57.699532+0000","last_active":"2024-04-20T11:26:59.694315+0000","last_peered":"2024-04-20T11:26:59.694315+0000","last_clean":"2024-04-20T11:26:59.694315+0000","last_became_active":"2024-04-20T11:26:57.699281+0000","last_became_peered":"2024-04-20T11:26:57.699281+0000","last_unstale":"2024-04-20T11:26:59.694315+0000","last_undegraded":"2024-04-20T11:26:59.694315+0000","last_fullsized":"2024-04-20T11:26:59.694315+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:22:14.490133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.693433+0000","last_change":"2024-04-20T11:26:58.692727+0000","last_active":"2024-04-20T11:26:59.693433+0000","last_peered":"2024-04-20T11:26:59.693433+0000","last_clean":"2024-04-20T11:26:59.693433+0000","last_became_active":"2024-04-20T11:26:58.692415+0000","last_became_peered":"2024-04-20T11:26:58.692415+0000","last_unstale":"2024-04-20T11:26:59.693433+0000","last_undegraded":"2024-04-20T11:26:59.693433+0000","last_fullsized":"2024-04-20T11:26:59.693433+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:01:34.618021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.186405+0000","last_change":"2024-04-20T11:26:57.681417+0000","last_active":"2024-04-20T11:27:00.186405+0000","last_peered":"2024-04-20T11:27:00.186405+0000","last_clean":"2024-04-20T11:27:00.186405+0000","last_became_active":"2024-04-20T11:26:57.680971+0000","last_became_peered":"2024-04-20T11:26:57.680971+0000","last_unstale":"2024-04-20T11:27:00.186405+0000","last_undegraded":"2024-04-20T11:27:00.186405+0000","last_fullsized":"2024-04-20T11:27:00.186405+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:09:13.468036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.698358+0000","last_change":"2024-04-20T11:26:58.710488+0000","last_active":"2024-04-20T11:26:59.698358+0000","last_peered":"2024-04-20T11:26:59.698358+0000","last_clean":"2024-04-20T11:26:59.698358+0000","last_became_active":"2024-04-20T11:26:58.710179+0000","last_became_peered":"2024-04-20T11:26:58.710179+0000","last_unstale":"2024-04-20T11:26:59.698358+0000","last_undegraded":"2024-04-20T11:26:59.698358+0000","last_fullsized":"2024-04-20T11:26:59.698358+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:09:14.483077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.739503+0000","last_change":"2024-04-20T11:26:57.700540+0000","last_active":"2024-04-20T11:26:59.739503+0000","last_peered":"2024-04-20T11:26:59.739503+0000","last_clean":"2024-04-20T11:26:59.739503+0000","last_became_active":"2024-04-20T11:26:57.697881+0000","last_became_peered":"2024-04-20T11:26:57.697881+0000","last_unstale":"2024-04-20T11:26:59.739503+0000","last_undegraded":"2024-04-20T11:26:59.739503+0000","last_fullsized":"2024-04-20T11:26:59.739503+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:28:39.881689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.184257+0000","last_change":"2024-04-20T11:26:58.710228+0000","last_active":"2024-04-20T11:27:00.184257+0000","last_peered":"2024-04-20T11:27:00.184257+0000","last_clean":"2024-04-20T11:27:00.184257+0000","last_became_active":"2024-04-20T11:26:58.709942+0000","last_became_peered":"2024-04-20T11:26:58.709942+0000","last_unstale":"2024-04-20T11:27:00.184257+0000","last_undegraded":"2024-04-20T11:27:00.184257+0000","last_fullsized":"2024-04-20T11:27:00.184257+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:01:48.311110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.692978+0000","last_change":"2024-04-20T11:26:57.691897+0000","last_active":"2024-04-20T11:26:59.692978+0000","last_peered":"2024-04-20T11:26:59.692978+0000","last_clean":"2024-04-20T11:26:59.692978+0000","last_became_active":"2024-04-20T11:26:57.690649+0000","last_became_peered":"2024-04-20T11:26:57.690649+0000","last_unstale":"2024-04-20T11:26:59.692978+0000","last_undegraded":"2024-04-20T11:26:59.692978+0000","last_fullsized":"2024-04-20T11:26:59.692978+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:11:36.407427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.693551+0000","last_change":"2024-04-20T11:26:58.696119+0000","last_active":"2024-04-20T11:26:59.693551+0000","last_peered":"2024-04-20T11:26:59.693551+0000","last_clean":"2024-04-20T11:26:59.693551+0000","last_became_active":"2024-04-20T11:26:58.695849+0000","last_became_peered":"2024-04-20T11:26:58.695849+0000","last_unstale":"2024-04-20T11:26:59.693551+0000","last_undegraded":"2024-04-20T11:26:59.693551+0000","last_fullsized":"2024-04-20T11:26:59.693551+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:53:57.559834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.696904+0000","last_change":"2024-04-20T11:26:57.693789+0000","last_active":"2024-04-20T11:26:59.696904+0000","last_peered":"2024-04-20T11:26:59.696904+0000","last_clean":"2024-04-20T11:26:59.696904+0000","last_became_active":"2024-04-20T11:26:57.688385+0000","last_became_peered":"2024-04-20T11:26:57.688385+0000","last_unstale":"2024-04-20T11:26:59.696904+0000","last_undegraded":"2024-04-20T11:26:59.696904+0000","last_fullsized":"2024-04-20T11:26:59.696904+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:59:13.454290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.185068+0000","last_change":"2024-04-20T11:26:58.713408+0000","last_active":"2024-04-20T11:27:00.185068+0000","last_peered":"2024-04-20T11:27:00.185068+0000","last_clean":"2024-04-20T11:27:00.185068+0000","last_became_active":"2024-04-20T11:26:58.713119+0000","last_became_peered":"2024-04-20T11:26:58.713119+0000","last_unstale":"2024-04-20T11:27:00.185068+0000","last_undegraded":"2024-04-20T11:27:00.185068+0000","last_fullsized":"2024-04-20T11:27:00.185068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:12:24.274916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.738625+0000","last_change":"2024-04-20T11:26:57.695426+0000","last_active":"2024-04-20T11:26:59.738625+0000","last_peered":"2024-04-20T11:26:59.738625+0000","last_clean":"2024-04-20T11:26:59.738625+0000","last_became_active":"2024-04-20T11:26:57.695023+0000","last_became_peered":"2024-04-20T11:26:57.695023+0000","last_unstale":"2024-04-20T11:26:59.738625+0000","last_undegraded":"2024-04-20T11:26:59.738625+0000","last_fullsized":"2024-04-20T11:26:59.738625+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:03:42.590011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.739012+0000","last_change":"2024-04-20T11:26:58.695228+0000","last_active":"2024-04-20T11:26:59.739012+0000","last_peered":"2024-04-20T11:26:59.739012+0000","last_clean":"2024-04-20T11:26:59.739012+0000","last_became_active":"2024-04-20T11:26:58.694540+0000","last_became_peered":"2024-04-20T11:26:58.694540+0000","last_unstale":"2024-04-20T11:26:59.739012+0000","last_undegraded":"2024-04-20T11:26:59.739012+0000","last_fullsized":"2024-04-20T11:26:59.739012+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:47:21.129095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.182683+0000","last_change":"2024-04-20T11:26:57.679351+0000","last_active":"2024-04-20T11:27:00.182683+0000","last_peered":"2024-04-20T11:27:00.182683+0000","last_clean":"2024-04-20T11:27:00.182683+0000","last_became_active":"2024-04-20T11:26:57.678821+0000","last_became_peered":"2024-04-20T11:26:57.678821+0000","last_unstale":"2024-04-20T11:27:00.182683+0000","last_undegraded":"2024-04-20T11:27:00.182683+0000","last_fullsized":"2024-04-20T11:27:00.182683+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:28:39.881689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.184115+0000","last_change":"2024-04-20T11:26:58.713370+0000","last_active":"2024-04-20T11:27:00.184115+0000","last_peered":"2024-04-20T11:27:00.184115+0000","last_clean":"2024-04-20T11:27:00.184115+0000","last_became_active":"2024-04-20T11:26:58.713068+0000","last_became_peered":"2024-04-20T11:26:58.713068+0000","last_unstale":"2024-04-20T11:27:00.184115+0000","last_undegraded":"2024-04-20T11:27:00.184115+0000","last_fullsized":"2024-04-20T11:27:00.184115+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:55:47.019482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.695270+0000","last_change":"2024-04-20T11:26:57.687641+0000","last_active":"2024-04-20T11:26:59.695270+0000","last_peered":"2024-04-20T11:26:59.695270+0000","last_clean":"2024-04-20T11:26:59.695270+0000","last_became_active":"2024-04-20T11:26:57.687237+0000","last_became_peered":"2024-04-20T11:26:57.687237+0000","last_unstale":"2024-04-20T11:26:59.695270+0000","last_undegraded":"2024-04-20T11:26:59.695270+0000","last_fullsized":"2024-04-20T11:26:59.695270+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:43:12.722459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.184814+0000","last_change":"2024-04-20T11:26:58.711133+0000","last_active":"2024-04-20T11:27:00.184814+0000","last_peered":"2024-04-20T11:27:00.184814+0000","last_clean":"2024-04-20T11:27:00.184814+0000","last_became_active":"2024-04-20T11:26:58.710818+0000","last_became_peered":"2024-04-20T11:26:58.710818+0000","last_unstale":"2024-04-20T11:27:00.184814+0000","last_undegraded":"2024-04-20T11:27:00.184814+0000","last_fullsized":"2024-04-20T11:27:00.184814+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:27:35.003583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.185902+0000","last_change":"2024-04-20T11:26:57.675885+0000","last_active":"2024-04-20T11:27:00.185902+0000","last_peered":"2024-04-20T11:27:00.185902+0000","last_clean":"2024-04-20T11:27:00.185902+0000","last_became_active":"2024-04-20T11:26:57.675625+0000","last_became_peered":"2024-04-20T11:26:57.675625+0000","last_unstale":"2024-04-20T11:27:00.185902+0000","last_undegraded":"2024-04-20T11:27:00.185902+0000","last_fullsized":"2024-04-20T11:27:00.185902+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:03:42.590011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.185312+0000","last_change":"2024-04-20T11:26:58.713427+0000","last_active":"2024-04-20T11:27:00.185312+0000","last_peered":"2024-04-20T11:27:00.185312+0000","last_clean":"2024-04-20T11:27:00.185312+0000","last_became_active":"2024-04-20T11:26:58.713112+0000","last_became_peered":"2024-04-20T11:26:58.713112+0000","last_unstale":"2024-04-20T11:27:00.185312+0000","last_undegraded":"2024-04-20T11:27:00.185312+0000","last_fullsized":"2024-04-20T11:27:00.185312+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:26:45.461271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.186539+0000","last_change":"2024-04-20T11:26:57.693246+0000","last_active":"2024-04-20T11:27:00.186539+0000","last_peered":"2024-04-20T11:27:00.186539+0000","last_clean":"2024-04-20T11:27:00.186539+0000","last_became_active":"2024-04-20T11:26:57.692699+0000","last_became_peered":"2024-04-20T11:26:57.692699+0000","last_unstale":"2024-04-20T11:27:00.186539+0000","last_undegraded":"2024-04-20T11:27:00.186539+0000","last_fullsized":"2024-04-20T11:27:00.186539+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:22:14.490133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.698394+0000","last_change":"2024-04-20T11:26:58.676238+0000","last_active":"2024-04-20T11:26:59.698394+0000","last_peered":"2024-04-20T11:26:59.698394+0000","last_clean":"2024-04-20T11:26:59.698394+0000","last_became_active":"2024-04-20T11:26:58.675427+0000","last_became_peered":"2024-04-20T11:26:58.675427+0000","last_unstale":"2024-04-20T11:26:59.698394+0000","last_undegraded":"2024-04-20T11:26:59.698394+0000","last_fullsized":"2024-04-20T11:26:59.698394+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:28:57.233788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.185685+0000","last_change":"2024-04-20T11:26:57.681076+0000","last_active":"2024-04-20T11:27:00.185685+0000","last_peered":"2024-04-20T11:27:00.185685+0000","last_clean":"2024-04-20T11:27:00.185685+0000","last_became_active":"2024-04-20T11:26:57.680806+0000","last_became_peered":"2024-04-20T11:26:57.680806+0000","last_unstale":"2024-04-20T11:27:00.185685+0000","last_undegraded":"2024-04-20T11:27:00.185685+0000","last_fullsized":"2024-04-20T11:27:00.185685+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:11:19.111199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.184412+0000","last_change":"2024-04-20T11:26:58.708666+0000","last_active":"2024-04-20T11:27:00.184412+0000","last_peered":"2024-04-20T11:27:00.184412+0000","last_clean":"2024-04-20T11:27:00.184412+0000","last_became_active":"2024-04-20T11:26:58.708362+0000","last_became_peered":"2024-04-20T11:26:58.708362+0000","last_unstale":"2024-04-20T11:27:00.184412+0000","last_undegraded":"2024-04-20T11:27:00.184412+0000","last_fullsized":"2024-04-20T11:27:00.184412+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:12:41.134688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.184117+0000","last_change":"2024-04-20T11:26:48.661897+0000","last_active":"2024-04-20T11:27:00.184117+0000","last_peered":"2024-04-20T11:27:00.184117+0000","last_clean":"2024-04-20T11:27:00.184117+0000","last_became_active":"2024-04-20T11:26:48.656535+0000","last_became_peered":"2024-04-20T11:26:48.656535+0000","last_unstale":"2024-04-20T11:27:00.184117+0000","last_undegraded":"2024-04-20T11:27:00.184117+0000","last_fullsized":"2024-04-20T11:27:00.184117+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:47.543004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:47.543004+0000","last_clean_scrub_stamp":"2024-04-20T11:26:47.543004+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:58:35.527105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.694066+0000","last_change":"2024-04-20T11:26:57.680902+0000","last_active":"2024-04-20T11:26:59.694066+0000","last_peered":"2024-04-20T11:26:59.694066+0000","last_clean":"2024-04-20T11:26:59.694066+0000","last_became_active":"2024-04-20T11:26:57.680506+0000","last_became_peered":"2024-04-20T11:26:57.680506+0000","last_unstale":"2024-04-20T11:26:59.694066+0000","last_undegraded":"2024-04-20T11:26:59.694066+0000","last_fullsized":"2024-04-20T11:26:59.694066+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:28:39.881689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.185222+0000","last_change":"2024-04-20T11:26:58.697905+0000","last_active":"2024-04-20T11:27:00.185222+0000","last_peered":"2024-04-20T11:27:00.185222+0000","last_clean":"2024-04-20T11:27:00.185222+0000","last_became_active":"2024-04-20T11:26:58.697612+0000","last_became_peered":"2024-04-20T11:26:58.697612+0000","last_unstale":"2024-04-20T11:27:00.185222+0000","last_undegraded":"2024-04-20T11:27:00.185222+0000","last_fullsized":"2024-04-20T11:27:00.185222+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:30:01.981489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.183754+0000","last_change":"2024-04-20T11:26:57.697322+0000","last_active":"2024-04-20T11:27:00.183754+0000","last_peered":"2024-04-20T11:27:00.183754+0000","last_clean":"2024-04-20T11:27:00.183754+0000","last_became_active":"2024-04-20T11:26:57.696775+0000","last_became_peered":"2024-04-20T11:26:57.696775+0000","last_unstale":"2024-04-20T11:27:00.183754+0000","last_undegraded":"2024-04-20T11:27:00.183754+0000","last_fullsized":"2024-04-20T11:27:00.183754+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:30:34.827026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.692465+0000","last_change":"2024-04-20T11:26:58.690752+0000","last_active":"2024-04-20T11:26:59.692465+0000","last_peered":"2024-04-20T11:26:59.692465+0000","last_clean":"2024-04-20T11:26:59.692465+0000","last_became_active":"2024-04-20T11:26:58.690451+0000","last_became_peered":"2024-04-20T11:26:58.690451+0000","last_unstale":"2024-04-20T11:26:59.692465+0000","last_undegraded":"2024-04-20T11:26:59.692465+0000","last_fullsized":"2024-04-20T11:26:59.692465+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:33:15.328953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.184689+0000","last_change":"2024-04-20T11:26:57.680139+0000","last_active":"2024-04-20T11:27:00.184689+0000","last_peered":"2024-04-20T11:27:00.184689+0000","last_clean":"2024-04-20T11:27:00.184689+0000","last_became_active":"2024-04-20T11:26:57.679820+0000","last_became_peered":"2024-04-20T11:26:57.679820+0000","last_unstale":"2024-04-20T11:27:00.184689+0000","last_undegraded":"2024-04-20T11:27:00.184689+0000","last_fullsized":"2024-04-20T11:27:00.184689+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:35:40.489733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.184019+0000","last_change":"2024-04-20T11:26:58.677911+0000","last_active":"2024-04-20T11:27:00.184019+0000","last_peered":"2024-04-20T11:27:00.184019+0000","last_clean":"2024-04-20T11:27:00.184019+0000","last_became_active":"2024-04-20T11:26:58.677645+0000","last_became_peered":"2024-04-20T11:26:58.677645+0000","last_unstale":"2024-04-20T11:27:00.184019+0000","last_undegraded":"2024-04-20T11:27:00.184019+0000","last_fullsized":"2024-04-20T11:27:00.184019+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:01:06.520823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.697092+0000","last_change":"2024-04-20T11:26:57.676331+0000","last_active":"2024-04-20T11:26:59.697092+0000","last_peered":"2024-04-20T11:26:59.697092+0000","last_clean":"2024-04-20T11:26:59.697092+0000","last_became_active":"2024-04-20T11:26:57.675643+0000","last_became_peered":"2024-04-20T11:26:57.675643+0000","last_unstale":"2024-04-20T11:26:59.697092+0000","last_undegraded":"2024-04-20T11:26:59.697092+0000","last_fullsized":"2024-04-20T11:26:59.697092+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:11:19.111199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.697040+0000","last_change":"2024-04-20T11:26:58.701514+0000","last_active":"2024-04-20T11:26:59.697040+0000","last_peered":"2024-04-20T11:26:59.697040+0000","last_clean":"2024-04-20T11:26:59.697040+0000","last_became_active":"2024-04-20T11:26:58.692435+0000","last_became_peered":"2024-04-20T11:26:58.692435+0000","last_unstale":"2024-04-20T11:26:59.697040+0000","last_undegraded":"2024-04-20T11:26:59.697040+0000","last_fullsized":"2024-04-20T11:26:59.697040+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:56:23.896778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.185970+0000","last_change":"2024-04-20T11:26:57.681965+0000","last_active":"2024-04-20T11:27:00.185970+0000","last_peered":"2024-04-20T11:27:00.185970+0000","last_clean":"2024-04-20T11:27:00.185970+0000","last_became_active":"2024-04-20T11:26:57.681130+0000","last_became_peered":"2024-04-20T11:26:57.681130+0000","last_unstale":"2024-04-20T11:27:00.185970+0000","last_undegraded":"2024-04-20T11:27:00.185970+0000","last_fullsized":"2024-04-20T11:27:00.185970+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:59:13.454290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.695098+0000","last_change":"2024-04-20T11:26:58.687674+0000","last_active":"2024-04-20T11:26:59.695098+0000","last_peered":"2024-04-20T11:26:59.695098+0000","last_clean":"2024-04-20T11:26:59.695098+0000","last_became_active":"2024-04-20T11:26:58.687373+0000","last_became_peered":"2024-04-20T11:26:58.687373+0000","last_unstale":"2024-04-20T11:26:59.695098+0000","last_undegraded":"2024-04-20T11:26:59.695098+0000","last_fullsized":"2024-04-20T11:26:59.695098+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:30:01.981489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.184744+0000","last_change":"2024-04-20T11:26:57.700654+0000","last_active":"2024-04-20T11:27:00.184744+0000","last_peered":"2024-04-20T11:27:00.184744+0000","last_clean":"2024-04-20T11:27:00.184744+0000","last_became_active":"2024-04-20T11:26:57.698785+0000","last_became_peered":"2024-04-20T11:26:57.698785+0000","last_unstale":"2024-04-20T11:27:00.184744+0000","last_undegraded":"2024-04-20T11:27:00.184744+0000","last_fullsized":"2024-04-20T11:27:00.184744+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:39:03.501101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.185902+0000","last_change":"2024-04-20T11:26:58.680824+0000","last_active":"2024-04-20T11:27:00.185902+0000","last_peered":"2024-04-20T11:27:00.185902+0000","last_clean":"2024-04-20T11:27:00.185902+0000","last_became_active":"2024-04-20T11:26:58.680518+0000","last_became_peered":"2024-04-20T11:26:58.680518+0000","last_unstale":"2024-04-20T11:27:00.185902+0000","last_undegraded":"2024-04-20T11:27:00.185902+0000","last_fullsized":"2024-04-20T11:27:00.185902+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:01:34.618021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.187052+0000","last_change":"2024-04-20T11:26:57.695120+0000","last_active":"2024-04-20T11:27:00.187052+0000","last_peered":"2024-04-20T11:27:00.187052+0000","last_clean":"2024-04-20T11:27:00.187052+0000","last_became_active":"2024-04-20T11:26:57.694850+0000","last_became_peered":"2024-04-20T11:26:57.694850+0000","last_unstale":"2024-04-20T11:27:00.187052+0000","last_undegraded":"2024-04-20T11:27:00.187052+0000","last_fullsized":"2024-04-20T11:27:00.187052+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:56:22.881738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.186628+0000","last_change":"2024-04-20T11:26:58.704801+0000","last_active":"2024-04-20T11:27:00.186628+0000","last_peered":"2024-04-20T11:27:00.186628+0000","last_clean":"2024-04-20T11:27:00.186628+0000","last_became_active":"2024-04-20T11:26:58.703638+0000","last_became_peered":"2024-04-20T11:26:58.703638+0000","last_unstale":"2024-04-20T11:27:00.186628+0000","last_undegraded":"2024-04-20T11:27:00.186628+0000","last_fullsized":"2024-04-20T11:27:00.186628+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:55:47.019482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.739746+0000","last_change":"2024-04-20T11:26:57.702231+0000","last_active":"2024-04-20T11:26:59.739746+0000","last_peered":"2024-04-20T11:26:59.739746+0000","last_clean":"2024-04-20T11:26:59.739746+0000","last_became_active":"2024-04-20T11:26:57.701649+0000","last_became_peered":"2024-04-20T11:26:57.701649+0000","last_unstale":"2024-04-20T11:26:59.739746+0000","last_undegraded":"2024-04-20T11:26:59.739746+0000","last_fullsized":"2024-04-20T11:26:59.739746+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:59:13.454290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.693158+0000","last_change":"2024-04-20T11:26:58.685253+0000","last_active":"2024-04-20T11:26:59.693158+0000","last_peered":"2024-04-20T11:26:59.693158+0000","last_clean":"2024-04-20T11:26:59.693158+0000","last_became_active":"2024-04-20T11:26:58.684993+0000","last_became_peered":"2024-04-20T11:26:58.684993+0000","last_unstale":"2024-04-20T11:26:59.693158+0000","last_undegraded":"2024-04-20T11:26:59.693158+0000","last_fullsized":"2024-04-20T11:26:59.693158+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:02:15.732106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.184446+0000","last_change":"2024-04-20T11:26:57.697065+0000","last_active":"2024-04-20T11:27:00.184446+0000","last_peered":"2024-04-20T11:27:00.184446+0000","last_clean":"2024-04-20T11:27:00.184446+0000","last_became_active":"2024-04-20T11:26:57.696461+0000","last_became_peered":"2024-04-20T11:26:57.696461+0000","last_unstale":"2024-04-20T11:27:00.184446+0000","last_undegraded":"2024-04-20T11:27:00.184446+0000","last_fullsized":"2024-04-20T11:27:00.184446+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:56:22.881738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.695053+0000","last_change":"2024-04-20T11:26:58.685057+0000","last_active":"2024-04-20T11:26:59.695053+0000","last_peered":"2024-04-20T11:26:59.695053+0000","last_clean":"2024-04-20T11:26:59.695053+0000","last_became_active":"2024-04-20T11:26:58.684493+0000","last_became_peered":"2024-04-20T11:26:58.684493+0000","last_unstale":"2024-04-20T11:26:59.695053+0000","last_undegraded":"2024-04-20T11:26:59.695053+0000","last_fullsized":"2024-04-20T11:26:59.695053+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:59:14.469331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.185352+0000","last_change":"2024-04-20T11:26:57.690472+0000","last_active":"2024-04-20T11:27:00.185352+0000","last_peered":"2024-04-20T11:27:00.185352+0000","last_clean":"2024-04-20T11:27:00.185352+0000","last_became_active":"2024-04-20T11:26:57.689786+0000","last_became_peered":"2024-04-20T11:26:57.689786+0000","last_unstale":"2024-04-20T11:27:00.185352+0000","last_undegraded":"2024-04-20T11:27:00.185352+0000","last_fullsized":"2024-04-20T11:27:00.185352+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:30:34.827026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.184283+0000","last_change":"2024-04-20T11:26:58.677827+0000","last_active":"2024-04-20T11:27:00.184283+0000","last_peered":"2024-04-20T11:27:00.184283+0000","last_clean":"2024-04-20T11:27:00.184283+0000","last_became_active":"2024-04-20T11:26:58.677521+0000","last_became_peered":"2024-04-20T11:26:58.677521+0000","last_unstale":"2024-04-20T11:27:00.184283+0000","last_undegraded":"2024-04-20T11:27:00.184283+0000","last_fullsized":"2024-04-20T11:27:00.184283+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:02:15.732106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.185683+0000","last_change":"2024-04-20T11:26:57.673530+0000","last_active":"2024-04-20T11:27:00.185683+0000","last_peered":"2024-04-20T11:27:00.185683+0000","last_clean":"2024-04-20T11:27:00.185683+0000","last_became_active":"2024-04-20T11:26:57.671513+0000","last_became_peered":"2024-04-20T11:26:57.671513+0000","last_unstale":"2024-04-20T11:27:00.185683+0000","last_undegraded":"2024-04-20T11:27:00.185683+0000","last_fullsized":"2024-04-20T11:27:00.185683+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:11:19.111199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.185170+0000","last_change":"2024-04-20T11:26:58.702869+0000","last_active":"2024-04-20T11:27:00.185170+0000","last_peered":"2024-04-20T11:27:00.185170+0000","last_clean":"2024-04-20T11:27:00.185170+0000","last_became_active":"2024-04-20T11:26:58.698423+0000","last_became_peered":"2024-04-20T11:26:58.698423+0000","last_unstale":"2024-04-20T11:27:00.185170+0000","last_undegraded":"2024-04-20T11:27:00.185170+0000","last_fullsized":"2024-04-20T11:27:00.185170+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:35:40.422299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.185236+0000","last_change":"2024-04-20T11:26:57.699410+0000","last_active":"2024-04-20T11:27:00.185236+0000","last_peered":"2024-04-20T11:27:00.185236+0000","last_clean":"2024-04-20T11:27:00.185236+0000","last_became_active":"2024-04-20T11:26:57.697630+0000","last_became_peered":"2024-04-20T11:26:57.697630+0000","last_unstale":"2024-04-20T11:27:00.185236+0000","last_undegraded":"2024-04-20T11:27:00.185236+0000","last_fullsized":"2024-04-20T11:27:00.185236+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:18:30.073624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.738917+0000","last_change":"2024-04-20T11:26:58.711494+0000","last_active":"2024-04-20T11:26:59.738917+0000","last_peered":"2024-04-20T11:26:59.738917+0000","last_clean":"2024-04-20T11:26:59.738917+0000","last_became_active":"2024-04-20T11:26:58.711116+0000","last_became_peered":"2024-04-20T11:26:58.711116+0000","last_unstale":"2024-04-20T11:26:59.738917+0000","last_undegraded":"2024-04-20T11:26:59.738917+0000","last_fullsized":"2024-04-20T11:26:59.738917+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:43:20.032648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.695487+0000","last_change":"2024-04-20T11:26:58.701576+0000","last_active":"2024-04-20T11:26:59.695487+0000","last_peered":"2024-04-20T11:26:59.695487+0000","last_clean":"2024-04-20T11:26:59.695487+0000","last_became_active":"2024-04-20T11:26:58.701222+0000","last_became_peered":"2024-04-20T11:26:58.701222+0000","last_unstale":"2024-04-20T11:26:59.695487+0000","last_undegraded":"2024-04-20T11:26:59.695487+0000","last_fullsized":"2024-04-20T11:26:59.695487+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:55:47.019482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.184217+0000","last_change":"2024-04-20T11:26:57.701154+0000","last_active":"2024-04-20T11:27:00.184217+0000","last_peered":"2024-04-20T11:27:00.184217+0000","last_clean":"2024-04-20T11:27:00.184217+0000","last_became_active":"2024-04-20T11:26:57.700716+0000","last_became_peered":"2024-04-20T11:26:57.700716+0000","last_unstale":"2024-04-20T11:27:00.184217+0000","last_undegraded":"2024-04-20T11:27:00.184217+0000","last_fullsized":"2024-04-20T11:27:00.184217+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:09:13.468036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.698090+0000","last_change":"2024-04-20T11:26:58.706275+0000","last_active":"2024-04-20T11:26:59.698090+0000","last_peered":"2024-04-20T11:26:59.698090+0000","last_clean":"2024-04-20T11:26:59.698090+0000","last_became_active":"2024-04-20T11:26:58.705922+0000","last_became_peered":"2024-04-20T11:26:58.705922+0000","last_unstale":"2024-04-20T11:26:59.698090+0000","last_undegraded":"2024-04-20T11:26:59.698090+0000","last_fullsized":"2024-04-20T11:26:59.698090+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:40:22.418833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.184419+0000","last_change":"2024-04-20T11:26:57.701304+0000","last_active":"2024-04-20T11:27:00.184419+0000","last_peered":"2024-04-20T11:27:00.184419+0000","last_clean":"2024-04-20T11:27:00.184419+0000","last_became_active":"2024-04-20T11:26:57.700620+0000","last_became_peered":"2024-04-20T11:26:57.700620+0000","last_unstale":"2024-04-20T11:27:00.184419+0000","last_undegraded":"2024-04-20T11:27:00.184419+0000","last_fullsized":"2024-04-20T11:27:00.184419+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:33:14.313913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.693201+0000","last_change":"2024-04-20T11:26:58.688125+0000","last_active":"2024-04-20T11:26:59.693201+0000","last_peered":"2024-04-20T11:26:59.693201+0000","last_clean":"2024-04-20T11:26:59.693201+0000","last_became_active":"2024-04-20T11:26:58.687917+0000","last_became_peered":"2024-04-20T11:26:58.687917+0000","last_unstale":"2024-04-20T11:26:59.693201+0000","last_undegraded":"2024-04-20T11:26:59.693201+0000","last_fullsized":"2024-04-20T11:26:59.693201+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:01:06.520823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.185078+0000","last_change":"2024-04-20T11:26:57.671052+0000","last_active":"2024-04-20T11:27:00.185078+0000","last_peered":"2024-04-20T11:27:00.185078+0000","last_clean":"2024-04-20T11:27:00.185078+0000","last_became_active":"2024-04-20T11:26:57.670620+0000","last_became_peered":"2024-04-20T11:26:57.670620+0000","last_unstale":"2024-04-20T11:27:00.185078+0000","last_undegraded":"2024-04-20T11:27:00.185078+0000","last_fullsized":"2024-04-20T11:27:00.185078+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:11:36.407427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:27:00.184939+0000","last_change":"2024-04-20T11:26:58.701127+0000","last_active":"2024-04-20T11:27:00.184939+0000","last_peered":"2024-04-20T11:27:00.184939+0000","last_clean":"2024-04-20T11:27:00.184939+0000","last_became_active":"2024-04-20T11:26:58.700948+0000","last_became_peered":"2024-04-20T11:26:58.700948+0000","last_unstale":"2024-04-20T11:27:00.184939+0000","last_undegraded":"2024-04-20T11:27:00.184939+0000","last_fullsized":"2024-04-20T11:27:00.184939+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:57.638380+0000","last_clean_scrub_stamp":"2024-04-20T11:26:57.638380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:01:06.520823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T11:26:59.693275+0000","last_change":"2024-04-20T11:26:57.692256+0000","last_active":"2024-04-20T11:26:59.693275+0000","last_peered":"2024-04-20T11:26:59.693275+0000","last_clean":"2024-04-20T11:26:59.693275+0000","last_became_active":"2024-04-20T11:26:57.691929+0000","last_became_peered":"2024-04-20T11:26:57.691929+0000","last_unstale":"2024-04-20T11:26:59.693275+0000","last_undegraded":"2024-04-20T11:26:59.693275+0000","last_fullsized":"2024-04-20T11:26:59.693275+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:56.623339+0000","last_clean_scrub_stamp":"2024-04-20T11:26:56.623339+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:34:59.927980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574857,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T11:27:09.296 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-20T11:27:09.296 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-20T11:27:09.296 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-20T11:27:09.296 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-20T11:27:09.530 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.529+0000 7f0c34a00640 1 Processor -- start 2024-04-20T11:27:09.530 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.530+0000 7f0c34a00640 1 -- start start 2024-04-20T11:27:09.531 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.531+0000 7f0c34a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f0c3006c400 0x7f0c300d05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:09.531 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.531+0000 7f0c34a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0c300cfda0 0x7f0c300ae2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:09.531 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.531+0000 7f0c34a00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c300d0b00 con 0x7f0c300703e0 2024-04-20T11:27:09.531 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.531+0000 7f0c34a00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f0c300d10b0 con 0x7f0c3006c400 2024-04-20T11:27:09.532 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.531+0000 7f0c34a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0c300d0dd0 con 0x7f0c300cfda0 2024-04-20T11:27:09.532 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.531+0000 7f0c2ea00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f0c3006c400 0x7f0c300d05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:09.532 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.531+0000 7f0c2ea00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f0c3006c400 0x7f0c300d05c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:48246/0 (socket says 172.21.15.152:48246) 2024-04-20T11:27:09.532 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.531+0000 7f0c2ea00640 1 -- 172.21.15.152:0/1868826616 learned_addr learned my addr 172.21.15.152:0/1868826616 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:09.532 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.532+0000 7f0c2fe00640 1 --2- 172.21.15.152:0/1868826616 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0c300cfda0 0x7f0c300ae2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:09.532 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.532+0000 7f0c2ea00640 1 -- 172.21.15.152:0/1868826616 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0c300cfda0 msgr2=0x7f0c300ae2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:09.532 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.532+0000 7f0c2ea00640 1 --2- 172.21.15.152:0/1868826616 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0c300cfda0 0x7f0c300ae2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:09.532 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.532+0000 7f0c2ea00640 1 -- 172.21.15.152:0/1868826616 >> v1:172.21.15.175:6789/0 conn(0x7f0c300703e0 legacy=0x7f0c300707c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:09.532 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.532+0000 7f0c2ea00640 1 -- 172.21.15.152:0/1868826616 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c300cf2c0 con 0x7f0c3006c400 2024-04-20T11:27:09.533 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.533+0000 7f0c2ea00640 1 --2- 172.21.15.152:0/1868826616 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0c3006c400 0x7f0c300d05c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c009870 tx=0x7f0c1c02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71de9e3cd554b1ae server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:09.535 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.533+0000 7f0c2e000640 1 -- 172.21.15.152:0/1868826616 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c1c0317c0 con 0x7f0c3006c400 2024-04-20T11:27:09.535 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.533+0000 7f0c2e000640 1 -- 172.21.15.152:0/1868826616 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c1c037070 con 0x7f0c3006c400 2024-04-20T11:27:09.535 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.533+0000 7f0c2e000640 1 -- 172.21.15.152:0/1868826616 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c1c031b80 con 0x7f0c3006c400 2024-04-20T11:27:09.535 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.534+0000 7f0c34a00640 1 -- 172.21.15.152:0/1868826616 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0c3006c400 msgr2=0x7f0c300d05c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:09.535 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.534+0000 7f0c34a00640 1 --2- 172.21.15.152:0/1868826616 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0c3006c400 0x7f0c300d05c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c009870 tx=0x7f0c1c02b860 comp rx=0 tx=0).stop 2024-04-20T11:27:09.535 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.534+0000 7f0c34a00640 1 -- 172.21.15.152:0/1868826616 shutdown_connections 2024-04-20T11:27:09.535 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.534+0000 7f0c34a00640 1 --2- 172.21.15.152:0/1868826616 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0c300cfda0 0x7f0c300ae2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:09.535 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.534+0000 7f0c34a00640 1 --2- 172.21.15.152:0/1868826616 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0c3006c400 0x7f0c300d05c0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:09.535 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.534+0000 7f0c34a00640 1 -- 172.21.15.152:0/1868826616 >> 172.21.15.152:0/1868826616 conn(0x7f0c3009aa40 msgr2=0x7f0c300b2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:09.535 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.534+0000 7f0c34a00640 1 -- 172.21.15.152:0/1868826616 shutdown_connections 2024-04-20T11:27:09.536 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.534+0000 7f0c34a00640 1 -- 172.21.15.152:0/1868826616 wait complete. 2024-04-20T11:27:09.536 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.535+0000 7f0c34a00640 1 Processor -- start 2024-04-20T11:27:09.536 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.536+0000 7f0c34a00640 1 -- start start 2024-04-20T11:27:09.537 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.536+0000 7f0c34a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0c3006c400 0x7f0c30139f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:09.537 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.536+0000 7f0c34a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0c300703e0 0x7f0c301f4490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:09.537 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.536+0000 7f0c34a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c300cfda0 0x7f0c3013c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:09.537 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.536+0000 7f0c34a00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f0c300d0230 con 0x7f0c300703e0 2024-04-20T11:27:09.537 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.536+0000 7f0c34a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0c300cf880 con 0x7f0c300cfda0 2024-04-20T11:27:09.537 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.536+0000 7f0c34a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0c30136b30 con 0x7f0c3006c400 2024-04-20T11:27:09.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.537+0000 7f0c2ea00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0c300703e0 0x7f0c301f4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:09.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.537+0000 7f0c2ea00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0c300703e0 0x7f0c301f4490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:48254/0 (socket says 172.21.15.152:48254) 2024-04-20T11:27:09.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.537+0000 7f0c2ea00640 1 -- 172.21.15.152:0/1665806740 learned_addr learned my addr 172.21.15.152:0/1665806740 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:09.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.537+0000 7f0c2f400640 1 --2- 172.21.15.152:0/1665806740 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0c3006c400 0x7f0c30139f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:09.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.537+0000 7f0c2ea00640 1 -- 172.21.15.152:0/1665806740 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0c3006c400 msgr2=0x7f0c30139f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:09.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.537+0000 7f0c2ea00640 1 --2- 172.21.15.152:0/1665806740 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0c3006c400 0x7f0c30139f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:09.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.537+0000 7f0c2ea00640 1 -- 172.21.15.152:0/1665806740 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c300cfda0 msgr2=0x7f0c3013c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:09.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.537+0000 7f0c2ea00640 1 --2- 172.21.15.152:0/1665806740 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c300cfda0 0x7f0c3013c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:09.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.537+0000 7f0c2ea00640 1 -- 172.21.15.152:0/1665806740 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c30136da0 con 0x7f0c300703e0 2024-04-20T11:27:09.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.538+0000 7f0c2ea00640 1 --2- 172.21.15.152:0/1665806740 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0c300703e0 0x7f0c301f4490 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c000c00 tx=0x7f0c1c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:09.539 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.538+0000 7f0c27400640 1 -- 172.21.15.152:0/1665806740 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c1c032650 con 0x7f0c300703e0 2024-04-20T11:27:09.539 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.538+0000 7f0c27400640 1 -- 172.21.15.152:0/1665806740 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c1c032810 con 0x7f0c300703e0 2024-04-20T11:27:09.539 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.538+0000 7f0c34a00640 1 -- 172.21.15.152:0/1665806740 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c30137080 con 0x7f0c300703e0 2024-04-20T11:27:09.539 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.538+0000 7f0c27400640 1 -- 172.21.15.152:0/1665806740 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c1c032c00 con 0x7f0c300703e0 2024-04-20T11:27:09.539 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.538+0000 7f0c34a00640 1 -- 172.21.15.152:0/1665806740 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c30137640 con 0x7f0c300703e0 2024-04-20T11:27:09.539 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.539+0000 7f0c34a00640 1 -- 172.21.15.152:0/1665806740 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c30075e60 con 0x7f0c300703e0 2024-04-20T11:27:09.540 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.540+0000 7f0c27400640 1 -- 172.21.15.152:0/1665806740 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0c1c04c060 con 0x7f0c300703e0 2024-04-20T11:27:09.541 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.541+0000 7f0c27400640 1 --2- 172.21.15.152:0/1665806740 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f0c1006c8f0 0x7f0c1006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:09.541 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.541+0000 7f0c27400640 1 -- 172.21.15.152:0/1665806740 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f0c1c0ad3d0 con 0x7f0c300703e0 2024-04-20T11:27:09.541 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.541+0000 7f0c2f400640 1 --2- 172.21.15.152:0/1665806740 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f0c1006c8f0 0x7f0c1006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:09.541 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.542+0000 7f0c2f400640 1 --2- 172.21.15.152:0/1665806740 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f0c1006c8f0 0x7f0c1006edb0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f0c20006fd0 tx=0x7f0c20008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:09.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.546+0000 7f0c27400640 1 -- 172.21.15.152:0/1665806740 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f0c1c040070 con 0x7f0c300703e0 2024-04-20T11:27:09.789 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.790+0000 7f0c34a00640 1 -- 172.21.15.152:0/1665806740 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f0c30067a30 con 0x7f0c300703e0 2024-04-20T11:27:09.790 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.790+0000 7f0c27400640 1 -- 172.21.15.152:0/1665806740 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f0c1c07b7e0 con 0x7f0c300703e0 2024-04-20T11:27:09.790 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:27:09.790 INFO:teuthology.orchestra.run.smithi152.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-20T11:27:09.792 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.793+0000 7f0c34a00640 1 -- 172.21.15.152:0/1665806740 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f0c1006c8f0 msgr2=0x7f0c1006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:09.793 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.793+0000 7f0c34a00640 1 --2- 172.21.15.152:0/1665806740 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f0c1006c8f0 0x7f0c1006edb0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f0c20006fd0 tx=0x7f0c20008040 comp rx=0 tx=0).stop 2024-04-20T11:27:09.793 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.793+0000 7f0c34a00640 1 -- 172.21.15.152:0/1665806740 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0c300703e0 msgr2=0x7f0c301f4490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:09.793 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.793+0000 7f0c34a00640 1 --2- 172.21.15.152:0/1665806740 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0c300703e0 0x7f0c301f4490 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c000c00 tx=0x7f0c1c0046c0 comp rx=0 tx=0).stop 2024-04-20T11:27:09.793 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.793+0000 7f0c34a00640 1 -- 172.21.15.152:0/1665806740 shutdown_connections 2024-04-20T11:27:09.793 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.793+0000 7f0c34a00640 1 --2- 172.21.15.152:0/1665806740 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f0c1006c8f0 0x7f0c1006edb0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:09.793 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.793+0000 7f0c34a00640 1 --2- 172.21.15.152:0/1665806740 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c300cfda0 0x7f0c3013c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:09.793 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.793+0000 7f0c34a00640 1 --2- 172.21.15.152:0/1665806740 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0c300703e0 0x7f0c301f4490 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:09.793 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.793+0000 7f0c34a00640 1 --2- 172.21.15.152:0/1665806740 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0c3006c400 0x7f0c30139f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:09.793 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.793+0000 7f0c34a00640 1 -- 172.21.15.152:0/1665806740 >> 172.21.15.152:0/1665806740 conn(0x7f0c3009aa40 msgr2=0x7f0c3015dfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:09.793 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.793+0000 7f0c34a00640 1 -- 172.21.15.152:0/1665806740 shutdown_connections 2024-04-20T11:27:09.793 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:09.793+0000 7f0c34a00640 1 -- 172.21.15.152:0/1665806740 wait complete. 2024-04-20T11:27:09.810 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-20T11:27:09.810 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:27:10.000 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.000+0000 7fc8a3000640 1 Processor -- start 2024-04-20T11:27:10.000 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.000+0000 7fc8a3000640 1 -- start start 2024-04-20T11:27:10.002 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.002+0000 7fc8a3000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fc89c15e7f0 0x7fc89c0d3210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:10.002 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.002+0000 7fc8a3000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc89c0cfdc0 0x7fc89c0d3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:10.002 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.002+0000 7fc8a3000640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc89c0ced40 con 0x7fc89c0b5ff0 2024-04-20T11:27:10.002 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.002+0000 7fc8a3000640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fc89c0b6b90 con 0x7fc89c15e7f0 2024-04-20T11:27:10.002 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.002+0000 7fc8a3000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc89c0cf000 con 0x7fc89c0cfdc0 2024-04-20T11:27:10.002 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.002+0000 7fc8a2600640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fc89c0b5ff0 0x7fc89c0b6480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:48532/0 (socket says 172.21.15.152:48532) 2024-04-20T11:27:10.002 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.002+0000 7fc8a2600640 1 -- 172.21.15.152:0/908257948 learned_addr learned my addr 172.21.15.152:0/908257948 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:27:10.002 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.002+0000 7fc8a1c00640 1 --2- 172.21.15.152:0/908257948 >> v2:172.21.15.152:3300/0 conn(0x7fc89c15e7f0 0x7fc89c0d3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:10.003 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.003+0000 7fc8a1200640 1 --2- 172.21.15.152:0/908257948 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc89c0cfdc0 0x7fc89c0d3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:10.003 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.003+0000 7fc89be00640 1 -- 172.21.15.152:0/908257948 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1566732178 0 0) 0x7fc89c0ced40 con 0x7fc89c0b5ff0 2024-04-20T11:27:10.003 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.003+0000 7fc89be00640 1 -- 172.21.15.152:0/908257948 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc884003520 con 0x7fc89c0b5ff0 2024-04-20T11:27:10.003 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.003+0000 7fc8a1c00640 1 -- 172.21.15.152:0/908257948 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc89c0cfdc0 msgr2=0x7fc89c0d3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:10.003 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.003+0000 7fc8a1c00640 1 --2- 172.21.15.152:0/908257948 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc89c0cfdc0 0x7fc89c0d3750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:10.003 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.003+0000 7fc8a1c00640 1 -- 172.21.15.152:0/908257948 >> v1:172.21.15.152:6789/0 conn(0x7fc89c0b5ff0 legacy=0x7fc89c0b6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:10.003 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.003+0000 7fc8a1c00640 1 -- 172.21.15.152:0/908257948 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc89c0cf2e0 con 0x7fc89c15e7f0 2024-04-20T11:27:10.004 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.004+0000 7fc8a1c00640 1 --2- 172.21.15.152:0/908257948 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc89c15e7f0 0x7fc89c0d3210 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc88c0097a0 tx=0x7fc88c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84d2117de3ae9a8f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:10.005 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.005+0000 7fc89be00640 1 -- 172.21.15.152:0/908257948 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc88c035020 con 0x7fc89c15e7f0 2024-04-20T11:27:10.005 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.005+0000 7fc89be00640 1 -- 172.21.15.152:0/908257948 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc88c031830 con 0x7fc89c15e7f0 2024-04-20T11:27:10.005 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.005+0000 7fc89be00640 1 -- 172.21.15.152:0/908257948 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc88c031be0 con 0x7fc89c15e7f0 2024-04-20T11:27:10.006 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.006+0000 7fc8a3000640 1 -- 172.21.15.152:0/908257948 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc89c15e7f0 msgr2=0x7fc89c0d3210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:10.006 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.006+0000 7fc8a3000640 1 --2- 172.21.15.152:0/908257948 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc89c15e7f0 0x7fc89c0d3210 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc88c0097a0 tx=0x7fc88c02b8d0 comp rx=0 tx=0).stop 2024-04-20T11:27:10.006 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.006+0000 7fc8a3000640 1 -- 172.21.15.152:0/908257948 shutdown_connections 2024-04-20T11:27:10.006 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.006+0000 7fc8a3000640 1 --2- 172.21.15.152:0/908257948 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc89c0cfdc0 0x7fc89c0d3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:10.006 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.006+0000 7fc8a3000640 1 --2- 172.21.15.152:0/908257948 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc89c15e7f0 0x7fc89c0d3210 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:10.006 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.006+0000 7fc8a3000640 1 -- 172.21.15.152:0/908257948 >> 172.21.15.152:0/908257948 conn(0x7fc89c08bfd0 msgr2=0x7fc89c0d4570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:10.006 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.006+0000 7fc8a3000640 1 -- 172.21.15.152:0/908257948 shutdown_connections 2024-04-20T11:27:10.006 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.006+0000 7fc8a3000640 1 -- 172.21.15.152:0/908257948 wait complete. 2024-04-20T11:27:10.007 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.008+0000 7fc8a3000640 1 Processor -- start 2024-04-20T11:27:10.007 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.008+0000 7fc8a3000640 1 -- start start 2024-04-20T11:27:10.009 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.009+0000 7fc8a3000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc89c0b5ff0 0x7fc89c1f5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:10.009 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.009+0000 7fc8a3000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc89c0cfdc0 0x7fc89c1f71c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:10.009 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.009+0000 7fc8a3000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc89c15e7f0 0x7fc89c0ccf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:10.009 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.009+0000 7fc8a3000640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fc89c15ebd0 con 0x7fc89c15e7f0 2024-04-20T11:27:10.009 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.009+0000 7fc8a3000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc89c0cf8a0 con 0x7fc89c0cfdc0 2024-04-20T11:27:10.009 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.009+0000 7fc8a3000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc89c15ed50 con 0x7fc89c0b5ff0 2024-04-20T11:27:10.010 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.009+0000 7fc8a2600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc89c15e7f0 0x7fc89c0ccf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:10.010 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.009+0000 7fc8a2600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc89c15e7f0 0x7fc89c0ccf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:48270/0 (socket says 172.21.15.152:48270) 2024-04-20T11:27:10.010 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.009+0000 7fc8a2600640 1 -- 172.21.15.152:0/4243076983 learned_addr learned my addr 172.21.15.152:0/4243076983 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:10.010 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.009+0000 7fc8a1c00640 1 --2- 172.21.15.152:0/4243076983 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc89c0b5ff0 0x7fc89c1f5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:10.010 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.010+0000 7fc8a1200640 1 --2- 172.21.15.152:0/4243076983 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc89c0cfdc0 0x7fc89c1f71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:10.010 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.010+0000 7fc8a2600640 1 -- 172.21.15.152:0/4243076983 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc89c0b5ff0 msgr2=0x7fc89c1f5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:10.010 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.010+0000 7fc8a2600640 1 --2- 172.21.15.152:0/4243076983 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc89c0b5ff0 0x7fc89c1f5490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:10.010 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.010+0000 7fc8a2600640 1 -- 172.21.15.152:0/4243076983 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc89c0cfdc0 msgr2=0x7fc89c1f71c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:10.010 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.010+0000 7fc8a2600640 1 --2- 172.21.15.152:0/4243076983 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc89c0cfdc0 0x7fc89c1f71c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:10.010 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.010+0000 7fc8a2600640 1 -- 172.21.15.152:0/4243076983 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc89c0cc3e0 con 0x7fc89c15e7f0 2024-04-20T11:27:10.010 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.010+0000 7fc8a2600640 1 --2- 172.21.15.152:0/4243076983 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc89c15e7f0 0x7fc89c0ccf60 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc89400cb40 tx=0x7fc89400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:10.011 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.010+0000 7fc899600640 1 -- 172.21.15.152:0/4243076983 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc894017070 con 0x7fc89c15e7f0 2024-04-20T11:27:10.011 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.010+0000 7fc899600640 1 -- 172.21.15.152:0/4243076983 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc89400aca0 con 0x7fc89c15e7f0 2024-04-20T11:27:10.011 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.010+0000 7fc899600640 1 -- 172.21.15.152:0/4243076983 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc894012620 con 0x7fc89c15e7f0 2024-04-20T11:27:10.011 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.010+0000 7fc8a3000640 1 -- 172.21.15.152:0/4243076983 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc89c0b72c0 con 0x7fc89c15e7f0 2024-04-20T11:27:10.011 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.010+0000 7fc8a3000640 1 -- 172.21.15.152:0/4243076983 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc89c0cd4a0 con 0x7fc89c15e7f0 2024-04-20T11:27:10.011 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.012+0000 7fc8a3000640 1 -- 172.21.15.152:0/4243076983 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc89c075e60 con 0x7fc89c15e7f0 2024-04-20T11:27:10.012 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.012+0000 7fc899600640 1 -- 172.21.15.152:0/4243076983 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc89401a070 con 0x7fc89c15e7f0 2024-04-20T11:27:10.013 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.014+0000 7fc899600640 1 --2- 172.21.15.152:0/4243076983 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc88006c920 0x7fc88006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:10.013 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.014+0000 7fc899600640 1 -- 172.21.15.152:0/4243076983 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc89408b900 con 0x7fc89c15e7f0 2024-04-20T11:27:10.013 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.014+0000 7fc8a1c00640 1 --2- 172.21.15.152:0/4243076983 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc88006c920 0x7fc88006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:10.014 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.015+0000 7fc8a1c00640 1 --2- 172.21.15.152:0/4243076983 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc88006c920 0x7fc88006ede0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc88c036000 tx=0x7fc88c03c040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:10.017 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.018+0000 7fc899600640 1 -- 172.21.15.152:0/4243076983 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fc894059dc0 con 0x7fc89c15e7f0 2024-04-20T11:27:10.204 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.204+0000 7fc8a3000640 1 -- 172.21.15.152:0/4243076983 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc89c0b6d40 con 0x7fc89c15e7f0 2024-04-20T11:27:10.207 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.207+0000 7fc899600640 1 -- 172.21.15.152:0/4243076983 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16665 (secure 0 0 0) 0x7fc8940596c0 con 0x7fc89c15e7f0 2024-04-20T11:27:10.207 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:27:10.208 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":20,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:26:59.675971+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:26:56.261424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T11:26:57.079286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:27:10.210 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.210+0000 7fc8a3000640 1 -- 172.21.15.152:0/4243076983 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc88006c920 msgr2=0x7fc88006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:10.210 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.210+0000 7fc8a3000640 1 --2- 172.21.15.152:0/4243076983 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc88006c920 0x7fc88006ede0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc88c036000 tx=0x7fc88c03c040 comp rx=0 tx=0).stop 2024-04-20T11:27:10.210 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.210+0000 7fc8a3000640 1 -- 172.21.15.152:0/4243076983 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc89c15e7f0 msgr2=0x7fc89c0ccf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:10.210 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.210+0000 7fc8a3000640 1 --2- 172.21.15.152:0/4243076983 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc89c15e7f0 0x7fc89c0ccf60 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc89400cb40 tx=0x7fc89400a060 comp rx=0 tx=0).stop 2024-04-20T11:27:10.210 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.210+0000 7fc8a3000640 1 -- 172.21.15.152:0/4243076983 shutdown_connections 2024-04-20T11:27:10.210 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.210+0000 7fc8a3000640 1 --2- 172.21.15.152:0/4243076983 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc88006c920 0x7fc88006ede0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:10.210 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.210+0000 7fc8a3000640 1 --2- 172.21.15.152:0/4243076983 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc89c15e7f0 0x7fc89c0ccf60 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:10.211 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.210+0000 7fc8a3000640 1 --2- 172.21.15.152:0/4243076983 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc89c0cfdc0 0x7fc89c1f71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:10.211 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.210+0000 7fc8a3000640 1 --2- 172.21.15.152:0/4243076983 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc89c0b5ff0 0x7fc89c1f5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:10.211 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.210+0000 7fc8a3000640 1 -- 172.21.15.152:0/4243076983 >> 172.21.15.152:0/4243076983 conn(0x7fc89c08bfd0 msgr2=0x7fc89c0d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:10.211 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.210+0000 7fc8a3000640 1 -- 172.21.15.152:0/4243076983 shutdown_connections 2024-04-20T11:27:10.211 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.211+0000 7fc8a3000640 1 -- 172.21.15.152:0/4243076983 wait complete. 2024-04-20T11:27:10.228 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T11:26:47.246571+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T11:26:56.261424+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T11:26:57.079286+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T11:27:10.228 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T11:27:10.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.429+0000 7f9d39e00640 1 Processor -- start 2024-04-20T11:27:10.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.429+0000 7f9d39e00640 1 -- start start 2024-04-20T11:27:10.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.431+0000 7f9d39e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d340afba0 0x7f9d34162440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:10.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.431+0000 7f9d39e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f9d340b3fa0 0x7f9d340ae110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:10.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.431+0000 7f9d39e00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d340adc30 con 0x7f9d3415e5d0 2024-04-20T11:27:10.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.431+0000 7f9d39e00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f9d34162a70 con 0x7f9d340b3fa0 2024-04-20T11:27:10.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.431+0000 7f9d39e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f9d34162d50 con 0x7f9d340afba0 2024-04-20T11:27:10.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.431+0000 7f9d33400640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f9d340b3fa0 0x7f9d340ae110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:10.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.431+0000 7f9d32000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d340afba0 0x7f9d34162440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:10.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.432+0000 7f9d32000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d340afba0 0x7f9d34162440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.152:34286/0 (socket says 172.21.15.152:34286) 2024-04-20T11:27:10.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.432+0000 7f9d32000640 1 -- 172.21.15.152:0/1716581442 learned_addr learned my addr 172.21.15.152:0/1716581442 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:10.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.432+0000 7f9d33400640 1 -- 172.21.15.152:0/1716581442 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d340afba0 msgr2=0x7f9d34162440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:10.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.432+0000 7f9d33400640 1 --2- 172.21.15.152:0/1716581442 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d340afba0 0x7f9d34162440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:10.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.432+0000 7f9d33400640 1 -- 172.21.15.152:0/1716581442 >> v1:172.21.15.175:6789/0 conn(0x7f9d3415e5d0 legacy=0x7f9d340af490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:10.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.432+0000 7f9d33400640 1 -- 172.21.15.152:0/1716581442 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d34163030 con 0x7f9d340b3fa0 2024-04-20T11:27:10.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.432+0000 7f9d31600640 1 -- 172.21.15.152:0/1716581442 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 880251074 0 0) 0x7f9d340adc30 con 0x7f9d3415e5d0 2024-04-20T11:27:10.433 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.434+0000 7f9d33400640 1 --2- 172.21.15.152:0/1716581442 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d340b3fa0 0x7f9d340ae110 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c009d90 tx=0x7f9d2c02bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=916b98e9185bcf0f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:10.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.434+0000 7f9d31600640 1 -- 172.21.15.152:0/1716581442 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d2c039070 con 0x7f9d340b3fa0 2024-04-20T11:27:10.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.434+0000 7f9d31600640 1 -- 172.21.15.152:0/1716581442 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d2c02cc50 con 0x7f9d340b3fa0 2024-04-20T11:27:10.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.434+0000 7f9d31600640 1 -- 172.21.15.152:0/1716581442 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d2c034440 con 0x7f9d340b3fa0 2024-04-20T11:27:10.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.435+0000 7f9d39e00640 1 -- 172.21.15.152:0/1716581442 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d340b3fa0 msgr2=0x7f9d340ae110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:10.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.435+0000 7f9d39e00640 1 --2- 172.21.15.152:0/1716581442 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d340b3fa0 0x7f9d340ae110 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c009d90 tx=0x7f9d2c02bea0 comp rx=0 tx=0).stop 2024-04-20T11:27:10.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.435+0000 7f9d39e00640 1 -- 172.21.15.152:0/1716581442 shutdown_connections 2024-04-20T11:27:10.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.435+0000 7f9d39e00640 1 --2- 172.21.15.152:0/1716581442 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d340b3fa0 0x7f9d340ae110 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:10.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.435+0000 7f9d39e00640 1 --2- 172.21.15.152:0/1716581442 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d340afba0 0x7f9d34162440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:10.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.435+0000 7f9d39e00640 1 -- 172.21.15.152:0/1716581442 >> 172.21.15.152:0/1716581442 conn(0x7f9d3406b980 msgr2=0x7f9d3406bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:10.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.435+0000 7f9d39e00640 1 -- 172.21.15.152:0/1716581442 shutdown_connections 2024-04-20T11:27:10.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.435+0000 7f9d39e00640 1 -- 172.21.15.152:0/1716581442 wait complete. 2024-04-20T11:27:10.436 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.437+0000 7f9d39e00640 1 Processor -- start 2024-04-20T11:27:10.436 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.437+0000 7f9d39e00640 1 -- start start 2024-04-20T11:27:10.439 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.438+0000 7f9d39e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d340afba0 0x7f9d34149f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:10.440 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.438+0000 7f9d39e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d340b3fa0 0x7f9d3415f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:10.440 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.438+0000 7f9d39e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d3415e5d0 0x7f9d340b1400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:10.440 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.438+0000 7f9d39e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f9d3406e0a0 con 0x7f9d340afba0 2024-04-20T11:27:10.440 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.438+0000 7f9d39e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9d34163310 con 0x7f9d3415e5d0 2024-04-20T11:27:10.440 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.438+0000 7f9d39e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f9d3406e220 con 0x7f9d340b3fa0 2024-04-20T11:27:10.440 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.438+0000 7f9d32a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d340afba0 0x7f9d34149f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:10.440 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.438+0000 7f9d32a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d340afba0 0x7f9d34149f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:48280/0 (socket says 172.21.15.152:48280) 2024-04-20T11:27:10.440 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.438+0000 7f9d32a00640 1 -- 172.21.15.152:0/2100132145 learned_addr learned my addr 172.21.15.152:0/2100132145 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:10.440 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.439+0000 7f9d33400640 1 --2- 172.21.15.152:0/2100132145 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d3415e5d0 0x7f9d340b1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:10.440 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.439+0000 7f9d32000640 1 --2- 172.21.15.152:0/2100132145 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d340b3fa0 0x7f9d3415f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:10.440 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.439+0000 7f9d32a00640 1 -- 172.21.15.152:0/2100132145 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d340b3fa0 msgr2=0x7f9d3415f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:10.440 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.439+0000 7f9d32a00640 1 --2- 172.21.15.152:0/2100132145 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d340b3fa0 0x7f9d3415f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:10.440 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.439+0000 7f9d32a00640 1 -- 172.21.15.152:0/2100132145 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d3415e5d0 msgr2=0x7f9d340b1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:10.441 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.439+0000 7f9d32a00640 1 --2- 172.21.15.152:0/2100132145 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d3415e5d0 0x7f9d340b1400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:10.441 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.439+0000 7f9d32a00640 1 -- 172.21.15.152:0/2100132145 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d3415fcc0 con 0x7f9d340afba0 2024-04-20T11:27:10.441 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.439+0000 7f9d32a00640 1 --2- 172.21.15.152:0/2100132145 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d340afba0 0x7f9d34149f20 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9d2400c440 tx=0x7f9d2400c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:10.441 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.439+0000 7f9d2b400640 1 -- 172.21.15.152:0/2100132145 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d24017070 con 0x7f9d340afba0 2024-04-20T11:27:10.441 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.439+0000 7f9d2b400640 1 -- 172.21.15.152:0/2100132145 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d240035f0 con 0x7f9d340afba0 2024-04-20T11:27:10.441 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.439+0000 7f9d2b400640 1 -- 172.21.15.152:0/2100132145 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d240039e0 con 0x7f9d340afba0 2024-04-20T11:27:10.441 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.439+0000 7f9d39e00640 1 -- 172.21.15.152:0/2100132145 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d3415ffa0 con 0x7f9d340afba0 2024-04-20T11:27:10.441 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.440+0000 7f9d39e00640 1 -- 172.21.15.152:0/2100132145 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d34160560 con 0x7f9d340afba0 2024-04-20T11:27:10.442 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.441+0000 7f9d39e00640 1 -- 172.21.15.152:0/2100132145 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d340689f0 con 0x7f9d340afba0 2024-04-20T11:27:10.442 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.441+0000 7f9d2b400640 1 -- 172.21.15.152:0/2100132145 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9d2401a070 con 0x7f9d340afba0 2024-04-20T11:27:10.442 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.442+0000 7f9d2b400640 1 --2- 172.21.15.152:0/2100132145 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f9d1406c8f0 0x7f9d1406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:10.442 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.442+0000 7f9d2b400640 1 -- 172.21.15.152:0/2100132145 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f9d2408af10 con 0x7f9d340afba0 2024-04-20T11:27:10.442 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.443+0000 7f9d32000640 1 --2- 172.21.15.152:0/2100132145 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f9d1406c8f0 0x7f9d1406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:10.442 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.443+0000 7f9d32000640 1 --2- 172.21.15.152:0/2100132145 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f9d1406c8f0 0x7f9d1406edb0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f9d340b21b0 tx=0x7f9d20008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:10.446 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.447+0000 7f9d2b400640 1 -- 172.21.15.152:0/2100132145 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f9d24059340 con 0x7f9d340afba0 2024-04-20T11:27:10.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.631+0000 7f9d39e00640 1 -- 172.21.15.152:0/2100132145 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9d341625e0 con 0x7f9d340afba0 2024-04-20T11:27:10.632 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.632+0000 7f9d2b400640 1 -- 172.21.15.152:0/2100132145 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f9d24058c40 con 0x7f9d340afba0 2024-04-20T11:27:10.632 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 1 2024-04-20T11:27:10.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.634+0000 7f9d39e00640 1 -- 172.21.15.152:0/2100132145 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f9d1406c8f0 msgr2=0x7f9d1406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:10.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.634+0000 7f9d39e00640 1 --2- 172.21.15.152:0/2100132145 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f9d1406c8f0 0x7f9d1406edb0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f9d340b21b0 tx=0x7f9d20008040 comp rx=0 tx=0).stop 2024-04-20T11:27:10.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.634+0000 7f9d39e00640 1 -- 172.21.15.152:0/2100132145 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d340afba0 msgr2=0x7f9d34149f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:10.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.634+0000 7f9d39e00640 1 --2- 172.21.15.152:0/2100132145 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d340afba0 0x7f9d34149f20 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9d2400c440 tx=0x7f9d2400c900 comp rx=0 tx=0).stop 2024-04-20T11:27:10.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.635+0000 7f9d39e00640 1 -- 172.21.15.152:0/2100132145 shutdown_connections 2024-04-20T11:27:10.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.635+0000 7f9d39e00640 1 --2- 172.21.15.152:0/2100132145 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f9d1406c8f0 0x7f9d1406edb0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:10.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.635+0000 7f9d39e00640 1 --2- 172.21.15.152:0/2100132145 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d3415e5d0 0x7f9d340b1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:10.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.635+0000 7f9d39e00640 1 --2- 172.21.15.152:0/2100132145 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d340b3fa0 0x7f9d3415f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:10.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.635+0000 7f9d39e00640 1 --2- 172.21.15.152:0/2100132145 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d340afba0 0x7f9d34149f20 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:10.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.635+0000 7f9d39e00640 1 -- 172.21.15.152:0/2100132145 >> 172.21.15.152:0/2100132145 conn(0x7f9d3406b980 msgr2=0x7f9d340ae3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:10.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.635+0000 7f9d39e00640 1 -- 172.21.15.152:0/2100132145 shutdown_connections 2024-04-20T11:27:10.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.635+0000 7f9d39e00640 1 -- 172.21.15.152:0/2100132145 wait complete. 2024-04-20T11:27:10.651 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T11:27:10.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.851+0000 7f3702000640 1 Processor -- start 2024-04-20T11:27:10.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.851+0000 7f3702000640 1 -- start start 2024-04-20T11:27:10.852 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.853+0000 7f3702000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f36fc06ee00 0x7f36fc06f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:10.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.853+0000 7f3702000640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f36fc15fbf0 0x7f36fc15c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:10.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.853+0000 7f3702000640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36fc0d00d0 con 0x7f36fc0cf110 2024-04-20T11:27:10.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.853+0000 7f3702000640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f36fc06f720 con 0x7f36fc15fbf0 2024-04-20T11:27:10.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.853+0000 7f3702000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f36fc0cfb10 con 0x7f36fc06ee00 2024-04-20T11:27:10.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.854+0000 7f3700c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f36fc06ee00 0x7f36fc06f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:10.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.854+0000 7f36f3e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f36fc15fbf0 0x7f36fc15c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:10.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.854+0000 7f3700c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f36fc06ee00 0x7f36fc06f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.152:34298/0 (socket says 172.21.15.152:34298) 2024-04-20T11:27:10.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.854+0000 7f3700c00640 1 -- 172.21.15.152:0/652231521 learned_addr learned my addr 172.21.15.152:0/652231521 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:10.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.854+0000 7f36f3e00640 1 -- 172.21.15.152:0/652231521 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f36fc06ee00 msgr2=0x7f36fc06f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:10.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.854+0000 7f36f3e00640 1 --2- 172.21.15.152:0/652231521 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f36fc06ee00 0x7f36fc06f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:10.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.854+0000 7f36f3e00640 1 -- 172.21.15.152:0/652231521 >> v1:172.21.15.175:6789/0 conn(0x7f36fc0cf110 legacy=0x7f36fc0b3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:10.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.854+0000 7f36f3e00640 1 -- 172.21.15.152:0/652231521 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36fc0cfdf0 con 0x7f36fc15fbf0 2024-04-20T11:27:10.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.856+0000 7f36f3e00640 1 --2- 172.21.15.152:0/652231521 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36fc15fbf0 0x7f36fc15c390 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f36e40097a0 tx=0x7f36e402b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b67a5b005bf25803 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:10.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.856+0000 7f36fbe00640 1 -- 172.21.15.152:0/652231521 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e4035020 con 0x7f36fc15fbf0 2024-04-20T11:27:10.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.856+0000 7f36fbe00640 1 -- 172.21.15.152:0/652231521 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36e4031830 con 0x7f36fc15fbf0 2024-04-20T11:27:10.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.856+0000 7f36fbe00640 1 -- 172.21.15.152:0/652231521 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e4031c00 con 0x7f36fc15fbf0 2024-04-20T11:27:10.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.857+0000 7f3702000640 1 -- 172.21.15.152:0/652231521 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36fc15fbf0 msgr2=0x7f36fc15c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:10.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.857+0000 7f3702000640 1 --2- 172.21.15.152:0/652231521 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36fc15fbf0 0x7f36fc15c390 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f36e40097a0 tx=0x7f36e402b8d0 comp rx=0 tx=0).stop 2024-04-20T11:27:10.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.857+0000 7f3702000640 1 -- 172.21.15.152:0/652231521 shutdown_connections 2024-04-20T11:27:10.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.857+0000 7f3702000640 1 --2- 172.21.15.152:0/652231521 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36fc15fbf0 0x7f36fc15c390 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:10.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.857+0000 7f3702000640 1 --2- 172.21.15.152:0/652231521 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f36fc06ee00 0x7f36fc06f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:10.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.857+0000 7f3702000640 1 -- 172.21.15.152:0/652231521 >> 172.21.15.152:0/652231521 conn(0x7f36fc067ae0 msgr2=0x7f36fc06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:10.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.857+0000 7f3702000640 1 -- 172.21.15.152:0/652231521 shutdown_connections 2024-04-20T11:27:10.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.857+0000 7f3702000640 1 -- 172.21.15.152:0/652231521 wait complete. 2024-04-20T11:27:10.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.858+0000 7f3702000640 1 Processor -- start 2024-04-20T11:27:10.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.859+0000 7f3702000640 1 -- start start 2024-04-20T11:27:10.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.859+0000 7f3702000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f36fc06ee00 0x7f36fc0aee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:10.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.859+0000 7f3702000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36fc0cf110 0x7f36fc168d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:10.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.860+0000 7f3702000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f36fc15fbf0 0x7f36fc143b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:10.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.860+0000 7f3702000640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f36fc15c8d0 con 0x7f36fc15fbf0 2024-04-20T11:27:10.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.860+0000 7f3702000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f36fc15be70 con 0x7f36fc0cf110 2024-04-20T11:27:10.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.860+0000 7f3702000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f36fc144970 con 0x7f36fc06ee00 2024-04-20T11:27:10.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.860+0000 7f3701600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f36fc15fbf0 0x7f36fc143b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:10.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.860+0000 7f3701600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f36fc15fbf0 0x7f36fc143b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:48296/0 (socket says 172.21.15.152:48296) 2024-04-20T11:27:10.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.860+0000 7f3701600640 1 -- 172.21.15.152:0/80164817 learned_addr learned my addr 172.21.15.152:0/80164817 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:10.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.860+0000 7f36f3e00640 1 --2- 172.21.15.152:0/80164817 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36fc0cf110 0x7f36fc168d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:10.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.860+0000 7f3700c00640 1 --2- 172.21.15.152:0/80164817 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f36fc06ee00 0x7f36fc0aee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:10.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.860+0000 7f3701600640 1 -- 172.21.15.152:0/80164817 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f36fc06ee00 msgr2=0x7f36fc0aee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:10.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.860+0000 7f3701600640 1 --2- 172.21.15.152:0/80164817 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f36fc06ee00 0x7f36fc0aee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:10.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.860+0000 7f3701600640 1 -- 172.21.15.152:0/80164817 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36fc0cf110 msgr2=0x7f36fc168d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:10.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.860+0000 7f3701600640 1 --2- 172.21.15.152:0/80164817 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36fc0cf110 0x7f36fc168d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:10.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.860+0000 7f3701600640 1 -- 172.21.15.152:0/80164817 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36fc0adde0 con 0x7f36fc15fbf0 2024-04-20T11:27:10.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.860+0000 7f3700c00640 1 --2- 172.21.15.152:0/80164817 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f36fc06ee00 0x7f36fc0aee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:27:10.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.860+0000 7f36f3e00640 1 --2- 172.21.15.152:0/80164817 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36fc0cf110 0x7f36fc168d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:27:10.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.861+0000 7f3701600640 1 --2- 172.21.15.152:0/80164817 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f36fc15fbf0 0x7f36fc143b50 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f36f400c7a0 tx=0x7f36f400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:10.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.861+0000 7f36f9600640 1 -- 172.21.15.152:0/80164817 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f4003b70 con 0x7f36fc15fbf0 2024-04-20T11:27:10.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.861+0000 7f36f9600640 1 -- 172.21.15.152:0/80164817 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36f4003d30 con 0x7f36fc15fbf0 2024-04-20T11:27:10.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.861+0000 7f3702000640 1 -- 172.21.15.152:0/80164817 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36fc0ae0c0 con 0x7f36fc15fbf0 2024-04-20T11:27:10.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.861+0000 7f3702000640 1 -- 172.21.15.152:0/80164817 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36fc0ae680 con 0x7f36fc15fbf0 2024-04-20T11:27:10.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.861+0000 7f36f9600640 1 -- 172.21.15.152:0/80164817 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f400aa50 con 0x7f36fc15fbf0 2024-04-20T11:27:10.863 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.862+0000 7f3702000640 1 -- 172.21.15.152:0/80164817 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36fc075e60 con 0x7f36fc15fbf0 2024-04-20T11:27:10.863 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.863+0000 7f36f9600640 1 -- 172.21.15.152:0/80164817 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f36f400abf0 con 0x7f36fc15fbf0 2024-04-20T11:27:10.863 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.864+0000 7f36f9600640 1 --2- 172.21.15.152:0/80164817 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f36d806c8f0 0x7f36d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:10.863 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.864+0000 7f36f9600640 1 -- 172.21.15.152:0/80164817 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f36f408bbb0 con 0x7f36fc15fbf0 2024-04-20T11:27:10.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.864+0000 7f3700c00640 1 --2- 172.21.15.152:0/80164817 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f36d806c8f0 0x7f36d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:10.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.865+0000 7f3700c00640 1 --2- 172.21.15.152:0/80164817 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f36d806c8f0 0x7f36d806edb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f36ec009a20 tx=0x7f36ec008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:10.867 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:10.868+0000 7f36f9600640 1 -- 172.21.15.152:0/80164817 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f36f4059ec0 con 0x7f36fc15fbf0 2024-04-20T11:27:11.065 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.065+0000 7f3702000640 1 -- 172.21.15.152:0/80164817 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f36fc06f490 con 0x7f36fc15fbf0 2024-04-20T11:27:11.066 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.066+0000 7f36f9600640 1 -- 172.21.15.152:0/80164817 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f36f40597c0 con 0x7f36fc15fbf0 2024-04-20T11:27:11.066 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 32 2024-04-20T11:27:11.068 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.068+0000 7f3702000640 1 -- 172.21.15.152:0/80164817 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f36d806c8f0 msgr2=0x7f36d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:11.068 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.068+0000 7f3702000640 1 --2- 172.21.15.152:0/80164817 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f36d806c8f0 0x7f36d806edb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f36ec009a20 tx=0x7f36ec008040 comp rx=0 tx=0).stop 2024-04-20T11:27:11.068 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.068+0000 7f3702000640 1 -- 172.21.15.152:0/80164817 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f36fc15fbf0 msgr2=0x7f36fc143b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:11.068 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.069+0000 7f3702000640 1 --2- 172.21.15.152:0/80164817 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f36fc15fbf0 0x7f36fc143b50 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f36f400c7a0 tx=0x7f36f400cc60 comp rx=0 tx=0).stop 2024-04-20T11:27:11.069 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.069+0000 7f3702000640 1 -- 172.21.15.152:0/80164817 shutdown_connections 2024-04-20T11:27:11.069 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.069+0000 7f3702000640 1 --2- 172.21.15.152:0/80164817 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f36d806c8f0 0x7f36d806edb0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:11.069 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.069+0000 7f3702000640 1 --2- 172.21.15.152:0/80164817 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f36fc15fbf0 0x7f36fc143b50 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:11.069 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.069+0000 7f3702000640 1 --2- 172.21.15.152:0/80164817 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36fc0cf110 0x7f36fc168d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:11.069 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.069+0000 7f3702000640 1 --2- 172.21.15.152:0/80164817 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f36fc06ee00 0x7f36fc0aee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:11.069 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.069+0000 7f3702000640 1 -- 172.21.15.152:0/80164817 >> 172.21.15.152:0/80164817 conn(0x7f36fc067ae0 msgr2=0x7f36fc13f120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:11.069 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.069+0000 7f3702000640 1 -- 172.21.15.152:0/80164817 shutdown_connections 2024-04-20T11:27:11.069 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.069+0000 7f3702000640 1 -- 172.21.15.152:0/80164817 wait complete. 2024-04-20T11:27:11.087 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T11:27:11.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.282+0000 7f7926c00640 1 Processor -- start 2024-04-20T11:27:11.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.282+0000 7f7926c00640 1 -- start start 2024-04-20T11:27:11.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.283+0000 7f7926c00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f7920156c00 0x7f7920160a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:11.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.283+0000 7f7926c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f792015ca40 0x7f792015cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:11.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.283+0000 7f7926c00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f792006bda0 0x7f792006c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:11.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.284+0000 7f7926c00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f7920160fd0 con 0x7f792006bda0 2024-04-20T11:27:11.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.284+0000 7f7926c00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f792015d5c0 con 0x7f7920156c00 2024-04-20T11:27:11.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.284+0000 7f7926c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f792015d8a0 con 0x7f792015ca40 2024-04-20T11:27:11.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.284+0000 7f7926200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f792006bda0 0x7f792006c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:11.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.284+0000 7f7926200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f792006bda0 0x7f792006c230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:48312/0 (socket says 172.21.15.152:48312) 2024-04-20T11:27:11.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.284+0000 7f7926200640 1 -- 172.21.15.152:0/4091929269 learned_addr learned my addr 172.21.15.152:0/4091929269 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:11.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.284+0000 7f7926200640 1 -- 172.21.15.152:0/4091929269 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f792015ca40 msgr2=0x7f792015cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:11.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.284+0000 7f7926200640 1 --2- 172.21.15.152:0/4091929269 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f792015ca40 0x7f792015cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:11.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.284+0000 7f7926200640 1 -- 172.21.15.152:0/4091929269 >> v2:172.21.15.175:3300/0 conn(0x7f7920156c00 msgr2=0x7f7920160a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:11.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.284+0000 7f7926200640 1 --2- 172.21.15.152:0/4091929269 >> v2:172.21.15.175:3300/0 conn(0x7f7920156c00 0x7f7920160a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:11.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.284+0000 7f7926200640 1 -- 172.21.15.152:0/4091929269 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7920160290 con 0x7f792006bda0 2024-04-20T11:27:11.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.285+0000 7f7926200640 1 --2- 172.21.15.152:0/4091929269 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f792006bda0 0x7f792006c230 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f79200d1250 tx=0x7f791802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=146f7d61f563baf1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:11.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.285+0000 7f791fe00640 1 -- 172.21.15.152:0/4091929269 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f791802ca80 con 0x7f792006bda0 2024-04-20T11:27:11.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.286+0000 7f791fe00640 1 -- 172.21.15.152:0/4091929269 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f791802cc20 con 0x7f792006bda0 2024-04-20T11:27:11.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.286+0000 7f791fe00640 1 -- 172.21.15.152:0/4091929269 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79180343f0 con 0x7f792006bda0 2024-04-20T11:27:11.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.286+0000 7f7926c00640 1 -- 172.21.15.152:0/4091929269 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f792006bda0 msgr2=0x7f792006c230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:11.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.286+0000 7f7926c00640 1 --2- 172.21.15.152:0/4091929269 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f792006bda0 0x7f792006c230 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f79200d1250 tx=0x7f791802bee0 comp rx=0 tx=0).stop 2024-04-20T11:27:11.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.286+0000 7f7926c00640 1 -- 172.21.15.152:0/4091929269 shutdown_connections 2024-04-20T11:27:11.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.286+0000 7f7926c00640 1 --2- 172.21.15.152:0/4091929269 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f792006bda0 0x7f792006c230 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:11.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.286+0000 7f7926c00640 1 --2- 172.21.15.152:0/4091929269 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f792015ca40 0x7f792015cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:11.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.286+0000 7f7926c00640 1 --2- 172.21.15.152:0/4091929269 >> v2:172.21.15.175:3300/0 conn(0x7f7920156c00 0x7f7920160a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:11.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.286+0000 7f7926c00640 1 -- 172.21.15.152:0/4091929269 >> 172.21.15.152:0/4091929269 conn(0x7f79200b21e0 msgr2=0x7f79200b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:11.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.286+0000 7f7926c00640 1 -- 172.21.15.152:0/4091929269 shutdown_connections 2024-04-20T11:27:11.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.286+0000 7f7926c00640 1 -- 172.21.15.152:0/4091929269 wait complete. 2024-04-20T11:27:11.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.288+0000 7f7926c00640 1 Processor -- start 2024-04-20T11:27:11.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.288+0000 7f7926c00640 1 -- start start 2024-04-20T11:27:11.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.289+0000 7f7926c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f792006bda0 0x7f792014df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:11.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.289+0000 7f7926c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7920156c00 0x7f7920072840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:11.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.289+0000 7f7926c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f792015ca40 0x7f79201e7520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:11.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.289+0000 7f7926c00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f79200b3240 con 0x7f792015ca40 2024-04-20T11:27:11.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.289+0000 7f7926c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f79200b30c0 con 0x7f792006bda0 2024-04-20T11:27:11.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.289+0000 7f7926c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f79200b33c0 con 0x7f7920156c00 2024-04-20T11:27:11.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.289+0000 7f7926200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f792015ca40 0x7f79201e7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:11.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.289+0000 7f7926200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f792015ca40 0x7f79201e7520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:48318/0 (socket says 172.21.15.152:48318) 2024-04-20T11:27:11.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.289+0000 7f7926200640 1 -- 172.21.15.152:0/2937654465 learned_addr learned my addr 172.21.15.152:0/2937654465 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:11.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.289+0000 7f7924e00640 1 --2- 172.21.15.152:0/2937654465 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7920156c00 0x7f7920072840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:11.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.289+0000 7f7925800640 1 --2- 172.21.15.152:0/2937654465 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f792006bda0 0x7f792014df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:11.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.289+0000 7f7926200640 1 -- 172.21.15.152:0/2937654465 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7920156c00 msgr2=0x7f7920072840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:11.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.289+0000 7f7926200640 1 --2- 172.21.15.152:0/2937654465 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7920156c00 0x7f7920072840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:11.291 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.289+0000 7f7926200640 1 -- 172.21.15.152:0/2937654465 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f792006bda0 msgr2=0x7f792014df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:11.291 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.290+0000 7f7926200640 1 --2- 172.21.15.152:0/2937654465 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f792006bda0 0x7f792014df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:11.291 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.290+0000 7f7926200640 1 -- 172.21.15.152:0/2937654465 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7920151970 con 0x7f792015ca40 2024-04-20T11:27:11.291 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.290+0000 7f7925800640 1 --2- 172.21.15.152:0/2937654465 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f792006bda0 0x7f792014df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:27:11.291 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.290+0000 7f7926200640 1 --2- 172.21.15.152:0/2937654465 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f792015ca40 0x7f79201e7520 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7918009f40 tx=0x7f7918004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:11.291 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.290+0000 7f791d600640 1 -- 172.21.15.152:0/2937654465 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7918043020 con 0x7f792015ca40 2024-04-20T11:27:11.291 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.290+0000 7f791d600640 1 -- 172.21.15.152:0/2937654465 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f791803e450 con 0x7f792015ca40 2024-04-20T11:27:11.291 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.290+0000 7f791d600640 1 -- 172.21.15.152:0/2937654465 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f791803e820 con 0x7f792015ca40 2024-04-20T11:27:11.291 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.290+0000 7f7926c00640 1 -- 172.21.15.152:0/2937654465 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7920072d80 con 0x7f792015ca40 2024-04-20T11:27:11.291 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.290+0000 7f7926c00640 1 -- 172.21.15.152:0/2937654465 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79201e6760 con 0x7f792015ca40 2024-04-20T11:27:11.292 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.292+0000 7f7926c00640 1 -- 172.21.15.152:0/2937654465 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7920075e60 con 0x7f792015ca40 2024-04-20T11:27:11.292 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.292+0000 7f791d600640 1 -- 172.21.15.152:0/2937654465 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7918046020 con 0x7f792015ca40 2024-04-20T11:27:11.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.294+0000 7f791d600640 1 --2- 172.21.15.152:0/2937654465 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f790006c8f0 0x7f790006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:11.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.294+0000 7f791d600640 1 -- 172.21.15.152:0/2937654465 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f79180ae4b0 con 0x7f792015ca40 2024-04-20T11:27:11.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.294+0000 7f7925800640 1 --2- 172.21.15.152:0/2937654465 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f790006c8f0 0x7f790006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:11.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.295+0000 7f7925800640 1 --2- 172.21.15.152:0/2937654465 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f790006c8f0 0x7f790006edb0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7914006fd0 tx=0x7f7914008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:11.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.298+0000 7f791d600640 1 -- 172.21.15.152:0/2937654465 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f791804d030 con 0x7f792015ca40 2024-04-20T11:27:11.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.497+0000 7f7926c00640 1 -- 172.21.15.152:0/2937654465 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f792006f390 con 0x7f792015ca40 2024-04-20T11:27:11.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.498+0000 7f791d600640 1 -- 172.21.15.152:0/2937654465 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f791807c8d0 con 0x7f792015ca40 2024-04-20T11:27:11.497 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 64 2024-04-20T11:27:11.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.500+0000 7f7926c00640 1 -- 172.21.15.152:0/2937654465 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f790006c8f0 msgr2=0x7f790006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:11.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.500+0000 7f7926c00640 1 --2- 172.21.15.152:0/2937654465 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f790006c8f0 0x7f790006edb0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7914006fd0 tx=0x7f7914008040 comp rx=0 tx=0).stop 2024-04-20T11:27:11.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.500+0000 7f7926c00640 1 -- 172.21.15.152:0/2937654465 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f792015ca40 msgr2=0x7f79201e7520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:11.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.500+0000 7f7926c00640 1 --2- 172.21.15.152:0/2937654465 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f792015ca40 0x7f79201e7520 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7918009f40 tx=0x7f7918004a40 comp rx=0 tx=0).stop 2024-04-20T11:27:11.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.501+0000 7f7926c00640 1 -- 172.21.15.152:0/2937654465 shutdown_connections 2024-04-20T11:27:11.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.501+0000 7f7926c00640 1 --2- 172.21.15.152:0/2937654465 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f790006c8f0 0x7f790006edb0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:11.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.501+0000 7f7926c00640 1 --2- 172.21.15.152:0/2937654465 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f792015ca40 0x7f79201e7520 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:11.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.501+0000 7f7926c00640 1 --2- 172.21.15.152:0/2937654465 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7920156c00 0x7f7920072840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:11.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.501+0000 7f7926c00640 1 --2- 172.21.15.152:0/2937654465 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f792006bda0 0x7f792014df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:11.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.501+0000 7f7926c00640 1 -- 172.21.15.152:0/2937654465 >> 172.21.15.152:0/2937654465 conn(0x7f79200b21e0 msgr2=0x7f7920154950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:11.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.501+0000 7f7926c00640 1 -- 172.21.15.152:0/2937654465 shutdown_connections 2024-04-20T11:27:11.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.501+0000 7f7926c00640 1 -- 172.21.15.152:0/2937654465 wait complete. 2024-04-20T11:27:11.517 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:27:11.716 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.716+0000 7fdf36e00640 1 Processor -- start 2024-04-20T11:27:11.716 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.717+0000 7fdf36e00640 1 -- start start 2024-04-20T11:27:11.718 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.718+0000 7fdf36e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdf3015fcc0 0x7fdf300d4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:11.718 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.718+0000 7fdf36e00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf300d3350 con 0x7fdf300b65c0 2024-04-20T11:27:11.718 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.718+0000 7fdf36e00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf300d3630 con 0x7fdf301603d0 2024-04-20T11:27:11.718 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.718+0000 7fdf36e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fdf300d3910 con 0x7fdf3015fcc0 2024-04-20T11:27:11.718 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.718+0000 7fdf36400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fdf300b65c0 0x7fdf300cf240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:48536/0 (socket says 172.21.15.152:48536) 2024-04-20T11:27:11.718 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.718+0000 7fdf36400640 1 -- 172.21.15.152:0/423482311 learned_addr learned my addr 172.21.15.152:0/423482311 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:27:11.720 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.719+0000 7fdf35a00640 1 --2- 172.21.15.152:0/423482311 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdf3015fcc0 0x7fdf300d4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:11.720 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.719+0000 7fdf2fe00640 1 -- 172.21.15.152:0/423482311 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2693949514 0 0) 0x7fdf300d3350 con 0x7fdf300b65c0 2024-04-20T11:27:11.720 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.719+0000 7fdf2fe00640 1 -- 172.21.15.152:0/423482311 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdf14003520 con 0x7fdf300b65c0 2024-04-20T11:27:11.720 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.719+0000 7fdf2fe00640 1 -- 172.21.15.152:0/423482311 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 318472718 0 0) 0x7fdf14003520 con 0x7fdf300b65c0 2024-04-20T11:27:11.720 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.719+0000 7fdf2fe00640 1 -- 172.21.15.152:0/423482311 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdf3015fcc0 msgr2=0x7fdf300d4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:11.720 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.719+0000 7fdf2fe00640 1 --2- 172.21.15.152:0/423482311 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdf3015fcc0 0x7fdf300d4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:11.720 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.719+0000 7fdf2fe00640 1 -- 172.21.15.152:0/423482311 >> v1:172.21.15.175:6789/0 conn(0x7fdf301603d0 legacy=0x7fdf300b5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:11.720 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.719+0000 7fdf2fe00640 1 -- 172.21.15.152:0/423482311 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf300ced20 con 0x7fdf300b65c0 2024-04-20T11:27:11.720 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.719+0000 7fdf2fe00640 1 -- 172.21.15.152:0/423482311 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7fdf28001670 con 0x7fdf300b65c0 2024-04-20T11:27:11.721 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.719+0000 7fdf2fe00640 1 -- 172.21.15.152:0/423482311 >> v1:172.21.15.152:6789/0 conn(0x7fdf300b65c0 legacy=0x7fdf300cf240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:11.721 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.719+0000 7fdf2fe00640 1 --2- 172.21.15.152:0/423482311 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf140039e0 0x7fdf14003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:11.721 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.719+0000 7fdf2fe00640 1 --2- 172.21.15.152:0/423482311 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdf14004800 0x7fdf14004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:11.721 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.719+0000 7fdf2fe00640 1 --2- 172.21.15.152:0/423482311 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdf3015fcc0 0x7fdf140051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:11.721 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.719+0000 7fdf2fe00640 1 -- 172.21.15.152:0/423482311 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fdf300d3910 con 0x7fdf14004800 2024-04-20T11:27:11.721 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.719+0000 7fdf2fe00640 1 -- 172.21.15.152:0/423482311 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdf1400ba70 con 0x7fdf140039e0 2024-04-20T11:27:11.721 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.719+0000 7fdf2fe00640 1 -- 172.21.15.152:0/423482311 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fdf1400bc70 con 0x7fdf3015fcc0 2024-04-20T11:27:11.721 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.720+0000 7fdf35000640 1 --2- 172.21.15.152:0/423482311 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdf14004800 0x7fdf14004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:11.721 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.720+0000 7fdf35a00640 1 --2- 172.21.15.152:0/423482311 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf140039e0 0x7fdf14003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:11.721 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.720+0000 7fdf36400640 1 --2- 172.21.15.152:0/423482311 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdf3015fcc0 0x7fdf140051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:11.721 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.720+0000 7fdf35000640 1 -- 172.21.15.152:0/423482311 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdf3015fcc0 msgr2=0x7fdf140051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:11.721 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.720+0000 7fdf35000640 1 --2- 172.21.15.152:0/423482311 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdf3015fcc0 0x7fdf140051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:11.721 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.720+0000 7fdf35000640 1 -- 172.21.15.152:0/423482311 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf140039e0 msgr2=0x7fdf14003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:11.722 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.720+0000 7fdf35000640 1 --2- 172.21.15.152:0/423482311 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf140039e0 0x7fdf14003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:11.722 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.720+0000 7fdf35000640 1 -- 172.21.15.152:0/423482311 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf14003520 con 0x7fdf14004800 2024-04-20T11:27:11.723 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.721+0000 7fdf35000640 1 --2- 172.21.15.152:0/423482311 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdf14004800 0x7fdf14004c80 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c007fd0 tx=0x7fdf14005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4870019841328ed6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:11.723 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.721+0000 7fdf2fe00640 1 -- 172.21.15.152:0/423482311 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf1c0013c0 con 0x7fdf14004800 2024-04-20T11:27:11.723 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.721+0000 7fdf2fe00640 1 -- 172.21.15.152:0/423482311 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf1c0052d0 con 0x7fdf14004800 2024-04-20T11:27:11.723 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.721+0000 7fdf2fe00640 1 -- 172.21.15.152:0/423482311 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf1c003640 con 0x7fdf14004800 2024-04-20T11:27:11.723 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.722+0000 7fdf36e00640 1 -- 172.21.15.152:0/423482311 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdf14004800 msgr2=0x7fdf14004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:11.723 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.722+0000 7fdf36e00640 1 --2- 172.21.15.152:0/423482311 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdf14004800 0x7fdf14004c80 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c007fd0 tx=0x7fdf14005830 comp rx=0 tx=0).stop 2024-04-20T11:27:11.723 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.722+0000 7fdf36400640 1 -- 172.21.15.152:0/423482311 reap_dead start 2024-04-20T11:27:11.723 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.722+0000 7fdf36e00640 1 -- 172.21.15.152:0/423482311 shutdown_connections 2024-04-20T11:27:11.723 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.722+0000 7fdf36e00640 1 -- 172.21.15.152:0/423482311 >> 172.21.15.152:0/423482311 conn(0x7fdf3008bfd0 msgr2=0x7fdf300d4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:11.723 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.722+0000 7fdf36e00640 1 -- 172.21.15.152:0/423482311 shutdown_connections 2024-04-20T11:27:11.723 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.722+0000 7fdf36e00640 1 -- 172.21.15.152:0/423482311 wait complete. 2024-04-20T11:27:11.724 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.724+0000 7fdf36e00640 1 Processor -- start 2024-04-20T11:27:11.724 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.724+0000 7fdf36e00640 1 -- start start 2024-04-20T11:27:11.726 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.725+0000 7fdf36e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdf14004800 0x7fdf3015b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:11.726 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.725+0000 7fdf36e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdf301f5650 0x7fdf300b83f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:11.726 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.725+0000 7fdf36e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf300b9110 0x7fdf300b9590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:11.726 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.725+0000 7fdf36e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fdf300d4e70 con 0x7fdf301f5650 2024-04-20T11:27:11.726 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.725+0000 7fdf36e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdf300b56d0 con 0x7fdf300b9110 2024-04-20T11:27:11.726 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.725+0000 7fdf36e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fdf300d4ff0 con 0x7fdf14004800 2024-04-20T11:27:11.726 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.725+0000 7fdf35000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdf301f5650 0x7fdf300b83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:11.726 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.725+0000 7fdf35000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdf301f5650 0x7fdf300b83f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:48332/0 (socket says 172.21.15.152:48332) 2024-04-20T11:27:11.726 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.725+0000 7fdf35000640 1 -- 172.21.15.152:0/1006263075 learned_addr learned my addr 172.21.15.152:0/1006263075 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:11.726 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.726+0000 7fdf35a00640 1 --2- 172.21.15.152:0/1006263075 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdf14004800 0x7fdf3015b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:11.727 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.726+0000 7fdf35000640 1 -- 172.21.15.152:0/1006263075 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdf14004800 msgr2=0x7fdf3015b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:11.727 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.726+0000 7fdf36400640 1 --2- 172.21.15.152:0/1006263075 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf300b9110 0x7fdf300b9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:11.727 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.726+0000 7fdf35000640 1 --2- 172.21.15.152:0/1006263075 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdf14004800 0x7fdf3015b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:11.727 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.726+0000 7fdf35000640 1 -- 172.21.15.152:0/1006263075 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf300b9110 msgr2=0x7fdf300b9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:11.727 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.726+0000 7fdf35000640 1 --2- 172.21.15.152:0/1006263075 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf300b9110 0x7fdf300b9590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:11.727 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.726+0000 7fdf35000640 1 -- 172.21.15.152:0/1006263075 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf300b7350 con 0x7fdf301f5650 2024-04-20T11:27:11.727 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.726+0000 7fdf35a00640 1 --2- 172.21.15.152:0/1006263075 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdf14004800 0x7fdf3015b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:27:11.727 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.726+0000 7fdf35000640 1 --2- 172.21.15.152:0/1006263075 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdf301f5650 0x7fdf300b83f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c02bc50 tx=0x7fdf1c001170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:11.727 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.726+0000 7fdf2d600640 1 -- 172.21.15.152:0/1006263075 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf1c041020 con 0x7fdf301f5650 2024-04-20T11:27:11.727 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.726+0000 7fdf2d600640 1 -- 172.21.15.152:0/1006263075 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf1c0050d0 con 0x7fdf301f5650 2024-04-20T11:27:11.727 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.726+0000 7fdf2d600640 1 -- 172.21.15.152:0/1006263075 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf1c007490 con 0x7fdf301f5650 2024-04-20T11:27:11.727 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.726+0000 7fdf36e00640 1 -- 172.21.15.152:0/1006263075 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf300b7630 con 0x7fdf301f5650 2024-04-20T11:27:11.728 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.727+0000 7fdf36e00640 1 -- 172.21.15.152:0/1006263075 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf300b7bf0 con 0x7fdf301f5650 2024-04-20T11:27:11.728 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.728+0000 7fdf36e00640 1 -- 172.21.15.152:0/1006263075 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf30075e60 con 0x7fdf301f5650 2024-04-20T11:27:11.728 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.729+0000 7fdf2d600640 1 -- 172.21.15.152:0/1006263075 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdf1c044020 con 0x7fdf301f5650 2024-04-20T11:27:11.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.730+0000 7fdf2d600640 1 --2- 172.21.15.152:0/1006263075 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fdf1006c920 0x7fdf1006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:11.730 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.730+0000 7fdf2d600640 1 -- 172.21.15.152:0/1006263075 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fdf1c0ad630 con 0x7fdf301f5650 2024-04-20T11:27:11.730 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.730+0000 7fdf35a00640 1 --2- 172.21.15.152:0/1006263075 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fdf1006c920 0x7fdf1006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:11.730 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.731+0000 7fdf35a00640 1 --2- 172.21.15.152:0/1006263075 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fdf1006c920 0x7fdf1006ede0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fdf20007440 tx=0x7fdf2000b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:11.734 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.735+0000 7fdf2d600640 1 -- 172.21.15.152:0/1006263075 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fdf1c04b030 con 0x7fdf301f5650 2024-04-20T11:27:11.967 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.968+0000 7fdf36e00640 1 -- 172.21.15.152:0/1006263075 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdf300ba4d0 con 0x7fdf301f5650 2024-04-20T11:27:11.968 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.969+0000 7fdf2d600640 1 -- 172.21.15.152:0/1006263075 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5196 (secure 0 0 0) 0x7fdf1c03b330 con 0x7fdf301f5650 2024-04-20T11:27:11.969 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:27:11.969 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/853233806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":853233806},{"type":"v1","addr":"172.21.15.175:6833","nonce":853233806}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4253,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/100300237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":100300237},{"type":"v1","addr":"172.21.15.175:6835","nonce":100300237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/2924034058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":2924034058},{"type":"v1","addr":"172.21.15.152:6835","nonce":2924034058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T11:26:58.132293+0000","modified":"2024-04-20T11:27:01.444285+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4269":{"gid":4269,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6837/927639144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":927639144},{"type":"v1","addr":"172.21.15.152:6837","nonce":927639144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4269,"qdb_cluster":[4269]},"id":1}]} 2024-04-20T11:27:11.969 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 7 2024-04-20T11:27:11.971 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.971+0000 7fdf36e00640 1 -- 172.21.15.152:0/1006263075 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fdf1006c920 msgr2=0x7fdf1006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:11.971 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.971+0000 7fdf36e00640 1 --2- 172.21.15.152:0/1006263075 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fdf1006c920 0x7fdf1006ede0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fdf20007440 tx=0x7fdf2000b040 comp rx=0 tx=0).stop 2024-04-20T11:27:11.971 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.971+0000 7fdf36e00640 1 -- 172.21.15.152:0/1006263075 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdf301f5650 msgr2=0x7fdf300b83f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:11.971 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.971+0000 7fdf36e00640 1 --2- 172.21.15.152:0/1006263075 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdf301f5650 0x7fdf300b83f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c02bc50 tx=0x7fdf1c001170 comp rx=0 tx=0).stop 2024-04-20T11:27:11.971 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.972+0000 7fdf36e00640 1 -- 172.21.15.152:0/1006263075 shutdown_connections 2024-04-20T11:27:11.972 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.972+0000 7fdf36e00640 1 --2- 172.21.15.152:0/1006263075 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fdf1006c920 0x7fdf1006ede0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:11.972 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.972+0000 7fdf36e00640 1 --2- 172.21.15.152:0/1006263075 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf300b9110 0x7fdf300b9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:11.972 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.972+0000 7fdf36e00640 1 --2- 172.21.15.152:0/1006263075 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdf301f5650 0x7fdf300b83f0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:11.972 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.972+0000 7fdf36e00640 1 --2- 172.21.15.152:0/1006263075 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdf14004800 0x7fdf3015b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:11.972 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.972+0000 7fdf36e00640 1 -- 172.21.15.152:0/1006263075 >> 172.21.15.152:0/1006263075 conn(0x7fdf3008bfd0 msgr2=0x7fdf300d0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:11.972 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.972+0000 7fdf36e00640 1 -- 172.21.15.152:0/1006263075 shutdown_connections 2024-04-20T11:27:11.972 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:11.972+0000 7fdf36e00640 1 -- 172.21.15.152:0/1006263075 wait complete. 2024-04-20T11:27:11.990 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:27:12.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.186+0000 7f5bdd400640 1 Processor -- start 2024-04-20T11:27:12.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.186+0000 7f5bdd400640 1 -- start start 2024-04-20T11:27:12.187 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.187+0000 7f5bdd400640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f5bd815d1e0 0x7f5bd8160b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:12.187 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.187+0000 7f5bdd400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5bd8071cd0 0x7f5bd8072160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:12.187 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.188+0000 7f5bdd400640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5bd80ce8c0 con 0x7f5bd815d5c0 2024-04-20T11:27:12.187 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.188+0000 7f5bdd400640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f5bd80726a0 con 0x7f5bd815d1e0 2024-04-20T11:27:12.188 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.188+0000 7f5bdd400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5bd81600b0 con 0x7f5bd8071cd0 2024-04-20T11:27:12.188 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.188+0000 7f5bd7400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f5bd815d5c0 0x7f5bd80cdfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:48542/0 (socket says 172.21.15.152:48542) 2024-04-20T11:27:12.188 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.188+0000 7f5bd7400640 1 -- 172.21.15.152:0/2537489241 learned_addr learned my addr 172.21.15.152:0/2537489241 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:27:12.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.188+0000 7f5bd7e00640 1 --2- 172.21.15.152:0/2537489241 >> v2:172.21.15.175:3300/0 conn(0x7f5bd815d1e0 0x7f5bd8160b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:12.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.188+0000 7f5bdca00640 1 --2- 172.21.15.152:0/2537489241 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5bd8071cd0 0x7f5bd8072160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:12.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.188+0000 7f5bd6a00640 1 -- 172.21.15.152:0/2537489241 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3222263302 0 0) 0x7f5bd80ce8c0 con 0x7f5bd815d5c0 2024-04-20T11:27:12.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.188+0000 7f5bd6a00640 1 -- 172.21.15.152:0/2537489241 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5bbc003520 con 0x7f5bd815d5c0 2024-04-20T11:27:12.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.189+0000 7f5bd6a00640 1 -- 172.21.15.152:0/2537489241 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3496918414 0 0) 0x7f5bbc003520 con 0x7f5bd815d5c0 2024-04-20T11:27:12.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.189+0000 7f5bd6a00640 1 -- 172.21.15.152:0/2537489241 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5bd8071cd0 msgr2=0x7f5bd8072160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:12.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.189+0000 7f5bd6a00640 1 --2- 172.21.15.152:0/2537489241 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5bd8071cd0 0x7f5bd8072160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:12.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.189+0000 7f5bd6a00640 1 -- 172.21.15.152:0/2537489241 >> v2:172.21.15.175:3300/0 conn(0x7f5bd815d1e0 msgr2=0x7f5bd8160b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:12.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.189+0000 7f5bd6a00640 1 --2- 172.21.15.152:0/2537489241 >> v2:172.21.15.175:3300/0 conn(0x7f5bd815d1e0 0x7f5bd8160b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:12.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.189+0000 7f5bd6a00640 1 -- 172.21.15.152:0/2537489241 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bd8160390 con 0x7f5bd815d5c0 2024-04-20T11:27:12.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.189+0000 7f5bd6a00640 1 -- 172.21.15.152:0/2537489241 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f5bc4001670 con 0x7f5bd815d5c0 2024-04-20T11:27:12.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.189+0000 7f5bd6a00640 1 -- 172.21.15.152:0/2537489241 >> v1:172.21.15.152:6789/0 conn(0x7f5bd815d5c0 legacy=0x7f5bd80cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:12.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.189+0000 7f5bd6a00640 1 --2- 172.21.15.152:0/2537489241 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5bd8071cd0 0x7f5bbc0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:12.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.189+0000 7f5bd6a00640 1 --2- 172.21.15.152:0/2537489241 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5bbc003f80 0x7f5bbc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:12.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.189+0000 7f5bd6a00640 1 --2- 172.21.15.152:0/2537489241 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5bbc004d40 0x7f5bbc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:12.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.189+0000 7f5bd6a00640 1 -- 172.21.15.152:0/2537489241 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f5bd80726a0 con 0x7f5bbc003f80 2024-04-20T11:27:12.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.189+0000 7f5bd6a00640 1 -- 172.21.15.152:0/2537489241 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5bd81600b0 con 0x7f5bbc004d40 2024-04-20T11:27:12.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.189+0000 7f5bd6a00640 1 -- 172.21.15.152:0/2537489241 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5bbc00b9c0 con 0x7f5bd8071cd0 2024-04-20T11:27:12.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.189+0000 7f5bd7400640 1 --2- 172.21.15.152:0/2537489241 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5bbc003f80 0x7f5bbc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:12.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.189+0000 7f5bd7e00640 1 --2- 172.21.15.152:0/2537489241 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5bd8071cd0 0x7f5bbc0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:12.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.189+0000 7f5bdca00640 1 --2- 172.21.15.152:0/2537489241 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5bbc004d40 0x7f5bbc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:12.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.190+0000 7f5bd7400640 1 -- 172.21.15.152:0/2537489241 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5bd8071cd0 msgr2=0x7f5bbc0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:12.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.190+0000 7f5bd7400640 1 --2- 172.21.15.152:0/2537489241 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5bd8071cd0 0x7f5bbc0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:12.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.190+0000 7f5bd7400640 1 -- 172.21.15.152:0/2537489241 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5bbc004d40 msgr2=0x7f5bbc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:12.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.190+0000 7f5bd7400640 1 --2- 172.21.15.152:0/2537489241 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5bbc004d40 0x7f5bbc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:12.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.190+0000 7f5bd7400640 1 -- 172.21.15.152:0/2537489241 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bbc003520 con 0x7f5bbc003f80 2024-04-20T11:27:12.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.190+0000 7f5bdca00640 1 --2- 172.21.15.152:0/2537489241 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5bbc004d40 0x7f5bbc0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:27:12.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.191+0000 7f5bd7400640 1 --2- 172.21.15.152:0/2537489241 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5bbc003f80 0x7f5bbc004800 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5bc400c110 tx=0x7f5bbc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b5ba7aa4f4d21b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:12.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.191+0000 7f5bd6a00640 1 -- 172.21.15.152:0/2537489241 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bc4007c40 con 0x7f5bbc003f80 2024-04-20T11:27:12.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.191+0000 7f5bd6a00640 1 -- 172.21.15.152:0/2537489241 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bc4030070 con 0x7f5bbc003f80 2024-04-20T11:27:12.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.191+0000 7f5bd6a00640 1 -- 172.21.15.152:0/2537489241 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bc4005d30 con 0x7f5bbc003f80 2024-04-20T11:27:12.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.191+0000 7f5bdd400640 1 -- 172.21.15.152:0/2537489241 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5bbc003f80 msgr2=0x7f5bbc004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:12.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.191+0000 7f5bdd400640 1 --2- 172.21.15.152:0/2537489241 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5bbc003f80 0x7f5bbc004800 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5bc400c110 tx=0x7f5bbc005830 comp rx=0 tx=0).stop 2024-04-20T11:27:12.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.191+0000 7f5bdca00640 1 -- 172.21.15.152:0/2537489241 reap_dead start 2024-04-20T11:27:12.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.192+0000 7f5bdd400640 1 -- 172.21.15.152:0/2537489241 shutdown_connections 2024-04-20T11:27:12.193 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.192+0000 7f5bdd400640 1 -- 172.21.15.152:0/2537489241 >> 172.21.15.152:0/2537489241 conn(0x7f5bd8067ae0 msgr2=0x7f5bd806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:12.193 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.192+0000 7f5bdd400640 1 -- 172.21.15.152:0/2537489241 shutdown_connections 2024-04-20T11:27:12.193 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.192+0000 7f5bdd400640 1 -- 172.21.15.152:0/2537489241 wait complete. 2024-04-20T11:27:12.193 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.193+0000 7f5bdd400640 1 Processor -- start 2024-04-20T11:27:12.193 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.194+0000 7f5bdd400640 1 -- start start 2024-04-20T11:27:12.195 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.194+0000 7f5bdd400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5bbc003f80 0x7f5bd81e6610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:12.195 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.194+0000 7f5bdd400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5bd8155eb0 0x7f5bd8155010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:12.195 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.195+0000 7f5bdd400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5bd8157050 0x7f5bd80b2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:12.195 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.195+0000 7f5bdd400640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f5bd8072160 con 0x7f5bd8157050 2024-04-20T11:27:12.195 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.195+0000 7f5bdd400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5bd80ceba0 con 0x7f5bd8155eb0 2024-04-20T11:27:12.195 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.195+0000 7f5bdd400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5bd80722e0 con 0x7f5bbc003f80 2024-04-20T11:27:12.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.195+0000 7f5bdca00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5bd8157050 0x7f5bd80b2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:12.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.195+0000 7f5bdca00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5bd8157050 0x7f5bd80b2070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:48356/0 (socket says 172.21.15.152:48356) 2024-04-20T11:27:12.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.195+0000 7f5bdca00640 1 -- 172.21.15.152:0/2626368346 learned_addr learned my addr 172.21.15.152:0/2626368346 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:12.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.195+0000 7f5bd7e00640 1 --2- 172.21.15.152:0/2626368346 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5bbc003f80 0x7f5bd81e6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:12.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.195+0000 7f5bd7400640 1 --2- 172.21.15.152:0/2626368346 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5bd8155eb0 0x7f5bd8155010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:12.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.195+0000 7f5bdca00640 1 -- 172.21.15.152:0/2626368346 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5bbc003f80 msgr2=0x7f5bd81e6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:12.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.195+0000 7f5bdca00640 1 --2- 172.21.15.152:0/2626368346 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5bbc003f80 0x7f5bd81e6610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:12.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.195+0000 7f5bdca00640 1 -- 172.21.15.152:0/2626368346 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5bd8155eb0 msgr2=0x7f5bd8155010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:12.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.195+0000 7f5bdca00640 1 --2- 172.21.15.152:0/2626368346 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5bd8155eb0 0x7f5bd8155010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:12.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.195+0000 7f5bdca00640 1 -- 172.21.15.152:0/2626368346 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bd81e6b50 con 0x7f5bd8157050 2024-04-20T11:27:12.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.196+0000 7f5bdca00640 1 --2- 172.21.15.152:0/2626368346 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5bd8157050 0x7f5bd80b2070 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f5bd806fe80 tx=0x7f5bd000f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:12.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.196+0000 7f5bcfe00640 1 -- 172.21.15.152:0/2626368346 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd001c070 con 0x7f5bd8157050 2024-04-20T11:27:12.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.196+0000 7f5bcfe00640 1 -- 172.21.15.152:0/2626368346 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bd0017400 con 0x7f5bd8157050 2024-04-20T11:27:12.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.196+0000 7f5bcfe00640 1 -- 172.21.15.152:0/2626368346 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd00177f0 con 0x7f5bd8157050 2024-04-20T11:27:12.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.196+0000 7f5bdd400640 1 -- 172.21.15.152:0/2626368346 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bd8157550 con 0x7f5bd8157050 2024-04-20T11:27:12.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.196+0000 7f5bdd400640 1 -- 172.21.15.152:0/2626368346 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bd8157b10 con 0x7f5bd8157050 2024-04-20T11:27:12.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.197+0000 7f5bdd400640 1 -- 172.21.15.152:0/2626368346 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bd8075e60 con 0x7f5bd8157050 2024-04-20T11:27:12.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.198+0000 7f5bcfe00640 1 -- 172.21.15.152:0/2626368346 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5bd0005d10 con 0x7f5bd8157050 2024-04-20T11:27:12.199 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.199+0000 7f5bcfe00640 1 --2- 172.21.15.152:0/2626368346 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f5bb806c8f0 0x7f5bb806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:12.199 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.199+0000 7f5bcfe00640 1 -- 172.21.15.152:0/2626368346 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f5bd008bbe0 con 0x7f5bd8157050 2024-04-20T11:27:12.199 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.200+0000 7f5bd7e00640 1 --2- 172.21.15.152:0/2626368346 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f5bb806c8f0 0x7f5bb806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:12.199 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.200+0000 7f5bd7e00640 1 --2- 172.21.15.152:0/2626368346 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f5bb806c8f0 0x7f5bb806edb0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5bc80099d0 tx=0x7f5bc8008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:12.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.204+0000 7f5bcfe00640 1 -- 172.21.15.152:0/2626368346 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f5bd0012070 con 0x7f5bd8157050 2024-04-20T11:27:12.409 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.410+0000 7f5bdd400640 1 -- 172.21.15.152:0/2626368346 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5bd80b1750 con 0x7f5bd8157050 2024-04-20T11:27:12.412 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.413+0000 7f5bcfe00640 1 -- 172.21.15.152:0/2626368346 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16665 (secure 0 0 0) 0x7f5bd0059f70 con 0x7f5bd8157050 2024-04-20T11:27:12.412 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:27:12.412 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":20,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:26:59.675971+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:26:56.261424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T11:26:57.079286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:27:12.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.415+0000 7f5bdd400640 1 -- 172.21.15.152:0/2626368346 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f5bb806c8f0 msgr2=0x7f5bb806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:12.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.415+0000 7f5bdd400640 1 --2- 172.21.15.152:0/2626368346 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f5bb806c8f0 0x7f5bb806edb0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5bc80099d0 tx=0x7f5bc8008040 comp rx=0 tx=0).stop 2024-04-20T11:27:12.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.415+0000 7f5bdd400640 1 -- 172.21.15.152:0/2626368346 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5bd8157050 msgr2=0x7f5bd80b2070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:12.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.415+0000 7f5bdd400640 1 --2- 172.21.15.152:0/2626368346 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5bd8157050 0x7f5bd80b2070 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f5bd806fe80 tx=0x7f5bd000f4a0 comp rx=0 tx=0).stop 2024-04-20T11:27:12.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.415+0000 7f5bdd400640 1 -- 172.21.15.152:0/2626368346 shutdown_connections 2024-04-20T11:27:12.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.415+0000 7f5bdd400640 1 --2- 172.21.15.152:0/2626368346 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f5bb806c8f0 0x7f5bb806edb0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:12.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.415+0000 7f5bdd400640 1 --2- 172.21.15.152:0/2626368346 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5bd8157050 0x7f5bd80b2070 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:12.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.415+0000 7f5bdd400640 1 --2- 172.21.15.152:0/2626368346 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5bd8155eb0 0x7f5bd8155010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:12.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.415+0000 7f5bdd400640 1 --2- 172.21.15.152:0/2626368346 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5bbc003f80 0x7f5bd81e6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:12.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.415+0000 7f5bdd400640 1 -- 172.21.15.152:0/2626368346 >> 172.21.15.152:0/2626368346 conn(0x7f5bd8067ae0 msgr2=0x7f5bd806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:12.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.416+0000 7f5bdd400640 1 -- 172.21.15.152:0/2626368346 shutdown_connections 2024-04-20T11:27:12.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.416+0000 7f5bdd400640 1 -- 172.21.15.152:0/2626368346 wait complete. 2024-04-20T11:27:12.434 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T11:26:47.246571+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T11:26:56.261424+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T11:26:57.079286+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T11:27:12.434 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T11:27:12.638 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.638+0000 7ff5fa400640 1 Processor -- start 2024-04-20T11:27:12.638 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.638+0000 7ff5fa400640 1 -- start start 2024-04-20T11:27:12.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.640+0000 7ff5fa400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7ff5f415e5d0 0x7ff5f4162440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:12.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.640+0000 7ff5fa400640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5f40adc30 con 0x7ff5f40afba0 2024-04-20T11:27:12.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.640+0000 7ff5fa400640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5f4162a70 con 0x7ff5f40b3fa0 2024-04-20T11:27:12.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.640+0000 7ff5fa400640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7ff5f4162d50 con 0x7ff5f415e5d0 2024-04-20T11:27:12.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.640+0000 7ff5f3e00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7ff5f40afba0 0x7ff5f40ae110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:48552/0 (socket says 172.21.15.152:48552) 2024-04-20T11:27:12.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.640+0000 7ff5f3e00640 1 -- 172.21.15.152:0/1991244027 learned_addr learned my addr 172.21.15.152:0/1991244027 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:27:12.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.640+0000 7ff5f3400640 1 --2- 172.21.15.152:0/1991244027 >> v2:172.21.15.152:3300/0 conn(0x7ff5f415e5d0 0x7ff5f4162440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:12.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.641+0000 7ff5f2a00640 1 -- 172.21.15.152:0/1991244027 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 592154847 0 0) 0x7ff5f40adc30 con 0x7ff5f40afba0 2024-04-20T11:27:12.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.641+0000 7ff5f2a00640 1 -- 172.21.15.152:0/1991244027 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff5d8003520 con 0x7ff5f40afba0 2024-04-20T11:27:12.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.641+0000 7ff5f3400640 1 -- 172.21.15.152:0/1991244027 >> v1:172.21.15.175:6789/0 conn(0x7ff5f40b3fa0 legacy=0x7ff5f40af490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:12.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.641+0000 7ff5f3400640 1 -- 172.21.15.152:0/1991244027 >> v1:172.21.15.152:6789/0 conn(0x7ff5f40afba0 legacy=0x7ff5f40ae110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:12.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.641+0000 7ff5f3400640 1 -- 172.21.15.152:0/1991244027 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5f4163030 con 0x7ff5f415e5d0 2024-04-20T11:27:12.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.642+0000 7ff5f3400640 1 --2- 172.21.15.152:0/1991244027 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff5f415e5d0 0x7ff5f4162440 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff5e4009d90 tx=0x7ff5e402bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=626a7151eac32340 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:12.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.643+0000 7ff5f2a00640 1 -- 172.21.15.152:0/1991244027 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e4039070 con 0x7ff5f415e5d0 2024-04-20T11:27:12.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.643+0000 7ff5f2a00640 1 -- 172.21.15.152:0/1991244027 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5e402cd00 con 0x7ff5f415e5d0 2024-04-20T11:27:12.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.643+0000 7ff5f2a00640 1 -- 172.21.15.152:0/1991244027 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e4034660 con 0x7ff5f415e5d0 2024-04-20T11:27:12.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.643+0000 7ff5fa400640 1 -- 172.21.15.152:0/1991244027 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff5f415e5d0 msgr2=0x7ff5f4162440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:12.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.643+0000 7ff5fa400640 1 --2- 172.21.15.152:0/1991244027 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff5f415e5d0 0x7ff5f4162440 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff5e4009d90 tx=0x7ff5e402bf50 comp rx=0 tx=0).stop 2024-04-20T11:27:12.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.643+0000 7ff5fa400640 1 -- 172.21.15.152:0/1991244027 shutdown_connections 2024-04-20T11:27:12.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.643+0000 7ff5fa400640 1 --2- 172.21.15.152:0/1991244027 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff5f415e5d0 0x7ff5f4162440 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:12.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.643+0000 7ff5fa400640 1 -- 172.21.15.152:0/1991244027 >> 172.21.15.152:0/1991244027 conn(0x7ff5f406b980 msgr2=0x7ff5f406bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:12.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.643+0000 7ff5fa400640 1 -- 172.21.15.152:0/1991244027 shutdown_connections 2024-04-20T11:27:12.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.643+0000 7ff5fa400640 1 -- 172.21.15.152:0/1991244027 wait complete. 2024-04-20T11:27:12.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.645+0000 7ff5fa400640 1 Processor -- start 2024-04-20T11:27:12.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.645+0000 7ff5fa400640 1 -- start start 2024-04-20T11:27:12.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.646+0000 7ff5fa400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff5f40afba0 0x7ff5f4144c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:12.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.646+0000 7ff5fa400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff5f40b3fa0 0x7ff5f4153210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:12.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.646+0000 7ff5fa400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5f415e5d0 0x7ff5f4153750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:12.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.646+0000 7ff5fa400640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f406e0a0 con 0x7ff5f40afba0 2024-04-20T11:27:12.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.646+0000 7ff5fa400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f4163310 con 0x7ff5f415e5d0 2024-04-20T11:27:12.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.646+0000 7ff5fa400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff5f406e220 con 0x7ff5f40b3fa0 2024-04-20T11:27:12.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.646+0000 7ff5f3400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff5f40afba0 0x7ff5f4144c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:12.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.646+0000 7ff5f3400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff5f40afba0 0x7ff5f4144c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:48378/0 (socket says 172.21.15.152:48378) 2024-04-20T11:27:12.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.646+0000 7ff5f3400640 1 -- 172.21.15.152:0/2697648799 learned_addr learned my addr 172.21.15.152:0/2697648799 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:12.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.647+0000 7ff5f3e00640 1 --2- 172.21.15.152:0/2697648799 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5f415e5d0 0x7ff5f4153750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:12.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.647+0000 7ff5f3400640 1 -- 172.21.15.152:0/2697648799 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff5f40b3fa0 msgr2=0x7ff5f4153210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:12.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.647+0000 7ff5eaa00640 1 --2- 172.21.15.152:0/2697648799 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff5f40b3fa0 0x7ff5f4153210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:12.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.647+0000 7ff5f3400640 1 --2- 172.21.15.152:0/2697648799 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff5f40b3fa0 0x7ff5f4153210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:12.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.647+0000 7ff5f3400640 1 -- 172.21.15.152:0/2697648799 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5f415e5d0 msgr2=0x7ff5f4153750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:12.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.647+0000 7ff5f3400640 1 --2- 172.21.15.152:0/2697648799 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5f415e5d0 0x7ff5f4153750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:12.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.647+0000 7ff5f3400640 1 -- 172.21.15.152:0/2697648799 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5f4150ac0 con 0x7ff5f40afba0 2024-04-20T11:27:12.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.647+0000 7ff5f3e00640 1 --2- 172.21.15.152:0/2697648799 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5f415e5d0 0x7ff5f4153750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:27:12.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.647+0000 7ff5f3400640 1 --2- 172.21.15.152:0/2697648799 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff5f40afba0 0x7ff5f4144c30 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff5e4009d90 tx=0x7ff5e4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:12.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.647+0000 7ff5ebe00640 1 -- 172.21.15.152:0/2697648799 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e4039050 con 0x7ff5f40afba0 2024-04-20T11:27:12.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.647+0000 7ff5ebe00640 1 -- 172.21.15.152:0/2697648799 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5e4045070 con 0x7ff5f40afba0 2024-04-20T11:27:12.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.647+0000 7ff5ebe00640 1 -- 172.21.15.152:0/2697648799 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e403e850 con 0x7ff5f40afba0 2024-04-20T11:27:12.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.647+0000 7ff5fa400640 1 -- 172.21.15.152:0/2697648799 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5f41611d0 con 0x7ff5f40afba0 2024-04-20T11:27:12.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.648+0000 7ff5fa400640 1 -- 172.21.15.152:0/2697648799 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5f4151060 con 0x7ff5f40afba0 2024-04-20T11:27:12.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.649+0000 7ff5fa400640 1 -- 172.21.15.152:0/2697648799 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5f40689f0 con 0x7ff5f40afba0 2024-04-20T11:27:12.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.650+0000 7ff5ebe00640 1 -- 172.21.15.152:0/2697648799 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff5e403e9f0 con 0x7ff5f40afba0 2024-04-20T11:27:12.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.650+0000 7ff5ebe00640 1 --2- 172.21.15.152:0/2697648799 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff5d006c8f0 0x7ff5d006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:12.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.651+0000 7ff5ebe00640 1 -- 172.21.15.152:0/2697648799 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff5e40ae980 con 0x7ff5f40afba0 2024-04-20T11:27:12.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.651+0000 7ff5eaa00640 1 --2- 172.21.15.152:0/2697648799 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff5d006c8f0 0x7ff5d006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:12.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.651+0000 7ff5eaa00640 1 --2- 172.21.15.152:0/2697648799 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff5d006c8f0 0x7ff5d006edb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff5f4161720 tx=0x7ff5dc009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:12.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.655+0000 7ff5ebe00640 1 -- 172.21.15.152:0/2697648799 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ff5e407cf60 con 0x7ff5f40afba0 2024-04-20T11:27:12.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.852+0000 7ff5fa400640 1 -- 172.21.15.152:0/2697648799 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff5f40ae190 con 0x7ff5f40afba0 2024-04-20T11:27:12.852 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.852+0000 7ff5ebe00640 1 -- 172.21.15.152:0/2697648799 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7ff5e407c860 con 0x7ff5f40afba0 2024-04-20T11:27:12.852 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 1 2024-04-20T11:27:12.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.855+0000 7ff5fa400640 1 -- 172.21.15.152:0/2697648799 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff5d006c8f0 msgr2=0x7ff5d006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:12.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.855+0000 7ff5fa400640 1 --2- 172.21.15.152:0/2697648799 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff5d006c8f0 0x7ff5d006edb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff5f4161720 tx=0x7ff5dc009040 comp rx=0 tx=0).stop 2024-04-20T11:27:12.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.855+0000 7ff5fa400640 1 -- 172.21.15.152:0/2697648799 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff5f40afba0 msgr2=0x7ff5f4144c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:12.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.855+0000 7ff5fa400640 1 --2- 172.21.15.152:0/2697648799 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff5f40afba0 0x7ff5f4144c30 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff5e4009d90 tx=0x7ff5e4004a40 comp rx=0 tx=0).stop 2024-04-20T11:27:12.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.855+0000 7ff5fa400640 1 -- 172.21.15.152:0/2697648799 shutdown_connections 2024-04-20T11:27:12.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.855+0000 7ff5fa400640 1 --2- 172.21.15.152:0/2697648799 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff5d006c8f0 0x7ff5d006edb0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:12.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.855+0000 7ff5fa400640 1 --2- 172.21.15.152:0/2697648799 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5f415e5d0 0x7ff5f4153750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:12.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.855+0000 7ff5fa400640 1 --2- 172.21.15.152:0/2697648799 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff5f40b3fa0 0x7ff5f4153210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:12.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.855+0000 7ff5fa400640 1 --2- 172.21.15.152:0/2697648799 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff5f40afba0 0x7ff5f4144c30 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:12.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.855+0000 7ff5fa400640 1 -- 172.21.15.152:0/2697648799 >> 172.21.15.152:0/2697648799 conn(0x7ff5f406b980 msgr2=0x7ff5f406e8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:12.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.856+0000 7ff5fa400640 1 -- 172.21.15.152:0/2697648799 shutdown_connections 2024-04-20T11:27:12.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:12.856+0000 7ff5fa400640 1 -- 172.21.15.152:0/2697648799 wait complete. 2024-04-20T11:27:12.874 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T11:27:13.066 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.067+0000 7fb377a00640 1 Processor -- start 2024-04-20T11:27:13.067 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.067+0000 7fb377a00640 1 -- start start 2024-04-20T11:27:13.068 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.069+0000 7fb377a00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fb3700cd8c0 0x7fb3700b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:13.068 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.069+0000 7fb377a00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3701463e0 con 0x7fb370146000 2024-04-20T11:27:13.069 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.069+0000 7fb377a00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3700b3e20 con 0x7fb3700b2880 2024-04-20T11:27:13.069 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.069+0000 7fb377a00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fb3700b4750 con 0x7fb3700cd8c0 2024-04-20T11:27:13.069 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.069+0000 7fb375000640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fb370146000 0x7fb3700b2170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:48566/0 (socket says 172.21.15.152:48566) 2024-04-20T11:27:13.069 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.069+0000 7fb375000640 1 -- 172.21.15.152:0/2406660111 learned_addr learned my addr 172.21.15.152:0/2406660111 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:27:13.069 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.069+0000 7fb375a00640 1 --2- 172.21.15.152:0/2406660111 >> v2:172.21.15.175:3300/0 conn(0x7fb3700cd8c0 0x7fb3700b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:13.071 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.069+0000 7fb36f400640 1 -- 172.21.15.152:0/2406660111 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 873909006 0 0) 0x7fb3701463e0 con 0x7fb370146000 2024-04-20T11:27:13.071 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.070+0000 7fb36f400640 1 -- 172.21.15.152:0/2406660111 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb358003520 con 0x7fb370146000 2024-04-20T11:27:13.071 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.070+0000 7fb36f400640 1 -- 172.21.15.152:0/2406660111 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3294657711 0 0) 0x7fb358003520 con 0x7fb370146000 2024-04-20T11:27:13.071 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.070+0000 7fb36f400640 1 -- 172.21.15.152:0/2406660111 >> v2:172.21.15.175:3300/0 conn(0x7fb3700cd8c0 msgr2=0x7fb3700b4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:13.071 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.070+0000 7fb36f400640 1 --2- 172.21.15.152:0/2406660111 >> v2:172.21.15.175:3300/0 conn(0x7fb3700cd8c0 0x7fb3700b4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:13.071 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.070+0000 7fb36f400640 1 -- 172.21.15.152:0/2406660111 >> v1:172.21.15.175:6789/0 conn(0x7fb3700b2880 legacy=0x7fb3700cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:13.071 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.070+0000 7fb36f400640 1 -- 172.21.15.152:0/2406660111 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3700b4100 con 0x7fb370146000 2024-04-20T11:27:13.071 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.070+0000 7fb36f400640 1 -- 172.21.15.152:0/2406660111 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7fb364001670 con 0x7fb370146000 2024-04-20T11:27:13.071 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.070+0000 7fb36f400640 1 -- 172.21.15.152:0/2406660111 >> v1:172.21.15.152:6789/0 conn(0x7fb370146000 legacy=0x7fb3700b2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:13.071 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.070+0000 7fb36f400640 1 --2- 172.21.15.152:0/2406660111 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb3580039b0 0x7fb358003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:13.071 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.070+0000 7fb36f400640 1 --2- 172.21.15.152:0/2406660111 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb358004800 0x7fb358004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:13.071 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.070+0000 7fb36f400640 1 --2- 172.21.15.152:0/2406660111 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb3580051c0 0x7fb35800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:13.071 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.070+0000 7fb36f400640 1 -- 172.21.15.152:0/2406660111 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fb3700b4750 con 0x7fb3580051c0 2024-04-20T11:27:13.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.070+0000 7fb36f400640 1 -- 172.21.15.152:0/2406660111 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb35800c160 con 0x7fb358004800 2024-04-20T11:27:13.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.070+0000 7fb36f400640 1 -- 172.21.15.152:0/2406660111 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb35800c360 con 0x7fb3580039b0 2024-04-20T11:27:13.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.070+0000 7fb375a00640 1 --2- 172.21.15.152:0/2406660111 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb3580051c0 0x7fb35800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:13.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.071+0000 7fb375000640 1 --2- 172.21.15.152:0/2406660111 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb3580039b0 0x7fb358003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:13.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.071+0000 7fb36fe00640 1 --2- 172.21.15.152:0/2406660111 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb358004800 0x7fb358004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:13.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.071+0000 7fb375a00640 1 -- 172.21.15.152:0/2406660111 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb3580039b0 msgr2=0x7fb358003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:13.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.071+0000 7fb375a00640 1 --2- 172.21.15.152:0/2406660111 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb3580039b0 0x7fb358003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:13.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.071+0000 7fb375a00640 1 -- 172.21.15.152:0/2406660111 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb358004800 msgr2=0x7fb358004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:13.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.071+0000 7fb375a00640 1 --2- 172.21.15.152:0/2406660111 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb358004800 0x7fb358004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:13.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.071+0000 7fb375a00640 1 -- 172.21.15.152:0/2406660111 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb358003520 con 0x7fb3580051c0 2024-04-20T11:27:13.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.071+0000 7fb375000640 1 --2- 172.21.15.152:0/2406660111 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb3580039b0 0x7fb358003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:27:13.073 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.072+0000 7fb375a00640 1 -- 172.21.15.152:0/2406660111 reap_dead start 2024-04-20T11:27:13.073 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.072+0000 7fb375a00640 1 --2- 172.21.15.152:0/2406660111 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb3580051c0 0x7fb35800b9c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb368009fc0 tx=0x7fb358005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad288de74e593a3a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:13.073 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.072+0000 7fb36f400640 1 -- 172.21.15.152:0/2406660111 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb368009c10 con 0x7fb3580051c0 2024-04-20T11:27:13.074 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.072+0000 7fb36f400640 1 -- 172.21.15.152:0/2406660111 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb36802cca0 con 0x7fb3580051c0 2024-04-20T11:27:13.074 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.072+0000 7fb36f400640 1 -- 172.21.15.152:0/2406660111 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb368003bd0 con 0x7fb3580051c0 2024-04-20T11:27:13.074 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.073+0000 7fb377a00640 1 -- 172.21.15.152:0/2406660111 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb3580051c0 msgr2=0x7fb35800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:13.074 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.073+0000 7fb377a00640 1 --2- 172.21.15.152:0/2406660111 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb3580051c0 0x7fb35800b9c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb368009fc0 tx=0x7fb358005830 comp rx=0 tx=0).stop 2024-04-20T11:27:13.074 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.073+0000 7fb377a00640 1 -- 172.21.15.152:0/2406660111 shutdown_connections 2024-04-20T11:27:13.074 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.073+0000 7fb377a00640 1 --2- 172.21.15.152:0/2406660111 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb3580051c0 0x7fb35800b9c0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:13.074 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.073+0000 7fb377a00640 1 -- 172.21.15.152:0/2406660111 >> 172.21.15.152:0/2406660111 conn(0x7fb37006b980 msgr2=0x7fb37006bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:13.074 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.073+0000 7fb377a00640 1 -- 172.21.15.152:0/2406660111 shutdown_connections 2024-04-20T11:27:13.074 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.073+0000 7fb377a00640 1 -- 172.21.15.152:0/2406660111 wait complete. 2024-04-20T11:27:13.074 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.075+0000 7fb377a00640 1 Processor -- start 2024-04-20T11:27:13.075 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.075+0000 7fb377a00640 1 -- start start 2024-04-20T11:27:13.075 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.076+0000 7fb377a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb3580051c0 0x7fb37015bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:13.076 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.076+0000 7fb377a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb37013bbc0 0x7fb37013c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:13.076 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.076+0000 7fb377a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb37013c580 0x7fb37013d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:13.076 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.076+0000 7fb377a00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fb3700b4d60 con 0x7fb3580051c0 2024-04-20T11:27:13.076 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.076+0000 7fb377a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb3700b4be0 con 0x7fb37013bbc0 2024-04-20T11:27:13.076 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.076+0000 7fb377a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb3700b4ee0 con 0x7fb37013c580 2024-04-20T11:27:13.076 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.076+0000 7fb375000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb3580051c0 0x7fb37015bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:13.076 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.076+0000 7fb375000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb3580051c0 0x7fb37015bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:48396/0 (socket says 172.21.15.152:48396) 2024-04-20T11:27:13.076 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.076+0000 7fb375000640 1 -- 172.21.15.152:0/2736819498 learned_addr learned my addr 172.21.15.152:0/2736819498 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:13.077 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.077+0000 7fb36fe00640 1 --2- 172.21.15.152:0/2736819498 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb37013bbc0 0x7fb37013c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:13.077 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.077+0000 7fb375000640 1 -- 172.21.15.152:0/2736819498 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb37013c580 msgr2=0x7fb37013d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:13.077 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.077+0000 7fb375000640 1 --2- 172.21.15.152:0/2736819498 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb37013c580 0x7fb37013d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:13.077 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.077+0000 7fb375000640 1 -- 172.21.15.152:0/2736819498 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb37013bbc0 msgr2=0x7fb37013c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:13.077 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.077+0000 7fb375000640 1 --2- 172.21.15.152:0/2736819498 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb37013bbc0 0x7fb37013c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:13.077 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.077+0000 7fb375000640 1 -- 172.21.15.152:0/2736819498 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb37013ca00 con 0x7fb3580051c0 2024-04-20T11:27:13.078 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.077+0000 7fb375000640 1 --2- 172.21.15.152:0/2736819498 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb3580051c0 0x7fb37015bda0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb3640017e0 tx=0x7fb36400ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:13.078 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.077+0000 7fb36cc00640 1 -- 172.21.15.152:0/2736819498 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb36400ee60 con 0x7fb3580051c0 2024-04-20T11:27:13.078 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.077+0000 7fb36cc00640 1 -- 172.21.15.152:0/2736819498 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb364018070 con 0x7fb3580051c0 2024-04-20T11:27:13.078 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.077+0000 7fb36cc00640 1 -- 172.21.15.152:0/2736819498 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb36400cad0 con 0x7fb3580051c0 2024-04-20T11:27:13.078 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.077+0000 7fb377a00640 1 -- 172.21.15.152:0/2736819498 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3700cbb00 con 0x7fb3580051c0 2024-04-20T11:27:13.078 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.077+0000 7fb377a00640 1 -- 172.21.15.152:0/2736819498 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3700cc0c0 con 0x7fb3580051c0 2024-04-20T11:27:13.079 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.078+0000 7fb377a00640 1 -- 172.21.15.152:0/2736819498 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3700689f0 con 0x7fb3580051c0 2024-04-20T11:27:13.079 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.079+0000 7fb36cc00640 1 -- 172.21.15.152:0/2736819498 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb36400cc70 con 0x7fb3580051c0 2024-04-20T11:27:13.080 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.080+0000 7fb36cc00640 1 --2- 172.21.15.152:0/2736819498 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fb35006c8f0 0x7fb35006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:13.080 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.080+0000 7fb36cc00640 1 -- 172.21.15.152:0/2736819498 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb36408b9a0 con 0x7fb3580051c0 2024-04-20T11:27:13.080 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.080+0000 7fb36fe00640 1 --2- 172.21.15.152:0/2736819498 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fb35006c8f0 0x7fb35006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:13.080 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.081+0000 7fb36fe00640 1 --2- 172.21.15.152:0/2736819498 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fb35006c8f0 0x7fb35006edb0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb35c007fd0 tx=0x7fb35c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:13.084 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.085+0000 7fb36cc00640 1 -- 172.21.15.152:0/2736819498 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fb364059dd0 con 0x7fb3580051c0 2024-04-20T11:27:13.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.286+0000 7fb377a00640 1 -- 172.21.15.152:0/2736819498 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb370143ea0 con 0x7fb3580051c0 2024-04-20T11:27:13.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.287+0000 7fb36cc00640 1 -- 172.21.15.152:0/2736819498 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7fb3640596d0 con 0x7fb3580051c0 2024-04-20T11:27:13.287 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 32 2024-04-20T11:27:13.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.290+0000 7fb377a00640 1 -- 172.21.15.152:0/2736819498 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fb35006c8f0 msgr2=0x7fb35006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:13.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.290+0000 7fb377a00640 1 --2- 172.21.15.152:0/2736819498 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fb35006c8f0 0x7fb35006edb0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb35c007fd0 tx=0x7fb35c009040 comp rx=0 tx=0).stop 2024-04-20T11:27:13.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.290+0000 7fb377a00640 1 -- 172.21.15.152:0/2736819498 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb3580051c0 msgr2=0x7fb37015bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:13.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.290+0000 7fb377a00640 1 --2- 172.21.15.152:0/2736819498 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb3580051c0 0x7fb37015bda0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb3640017e0 tx=0x7fb36400ec10 comp rx=0 tx=0).stop 2024-04-20T11:27:13.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.290+0000 7fb377a00640 1 -- 172.21.15.152:0/2736819498 shutdown_connections 2024-04-20T11:27:13.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.290+0000 7fb377a00640 1 --2- 172.21.15.152:0/2736819498 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fb35006c8f0 0x7fb35006edb0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:13.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.290+0000 7fb377a00640 1 --2- 172.21.15.152:0/2736819498 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb37013c580 0x7fb37013d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:13.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.290+0000 7fb377a00640 1 --2- 172.21.15.152:0/2736819498 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb37013bbc0 0x7fb37013c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:13.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.290+0000 7fb377a00640 1 --2- 172.21.15.152:0/2736819498 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb3580051c0 0x7fb37015bda0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:13.291 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.290+0000 7fb377a00640 1 -- 172.21.15.152:0/2736819498 >> 172.21.15.152:0/2736819498 conn(0x7fb37006b980 msgr2=0x7fb37006bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:13.291 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.290+0000 7fb377a00640 1 -- 172.21.15.152:0/2736819498 shutdown_connections 2024-04-20T11:27:13.291 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.290+0000 7fb377a00640 1 -- 172.21.15.152:0/2736819498 wait complete. 2024-04-20T11:27:13.308 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T11:27:13.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.515+0000 7f0615c00640 1 Processor -- start 2024-04-20T11:27:13.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.516+0000 7f0615c00640 1 -- start start 2024-04-20T11:27:13.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.517+0000 7f0615c00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f06100ceeb0 0x7f06100d4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:13.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.517+0000 7f0615c00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06100d2b70 con 0x7f0610160230 2024-04-20T11:27:13.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.517+0000 7f0615c00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06100d2e50 con 0x7f06100cf5b0 2024-04-20T11:27:13.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.517+0000 7f0615c00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f06100d5420 con 0x7f06100ceeb0 2024-04-20T11:27:13.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.517+0000 7f060f400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f06100ceeb0 0x7f06100d4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:13.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.518+0000 7f060f400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f06100ceeb0 0x7f06100d4ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:48400/0 (socket says 172.21.15.152:48400) 2024-04-20T11:27:13.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.518+0000 7f060f400640 1 -- 172.21.15.152:0/3784749302 learned_addr learned my addr 172.21.15.152:0/3784749302 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:13.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.518+0000 7f060f400640 1 -- 172.21.15.152:0/3784749302 >> v1:172.21.15.175:6789/0 conn(0x7f06100cf5b0 legacy=0x7f06100cf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:13.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.518+0000 7f060f400640 1 -- 172.21.15.152:0/3784749302 >> v1:172.21.15.152:6789/0 conn(0x7f0610160230 legacy=0x7f06101606e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:13.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.518+0000 7f060f400640 1 -- 172.21.15.152:0/3784749302 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06100d3130 con 0x7f06100ceeb0 2024-04-20T11:27:13.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.519+0000 7f060f400640 1 --2- 172.21.15.152:0/3784749302 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f06100ceeb0 0x7f06100d4ee0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f05fc009860 tx=0x7f05fc02baa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d4effbf45b706b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:13.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.519+0000 7f060ea00640 1 -- 172.21.15.152:0/3784749302 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05fc02d860 con 0x7f06100ceeb0 2024-04-20T11:27:13.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.520+0000 7f060ea00640 1 -- 172.21.15.152:0/3784749302 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05fc02da00 con 0x7f06100ceeb0 2024-04-20T11:27:13.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.520+0000 7f060ea00640 1 -- 172.21.15.152:0/3784749302 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05fc02ddd0 con 0x7f06100ceeb0 2024-04-20T11:27:13.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.520+0000 7f0615c00640 1 -- 172.21.15.152:0/3784749302 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f06100ceeb0 msgr2=0x7f06100d4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:13.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.520+0000 7f0615c00640 1 --2- 172.21.15.152:0/3784749302 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f06100ceeb0 0x7f06100d4ee0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f05fc009860 tx=0x7f05fc02baa0 comp rx=0 tx=0).stop 2024-04-20T11:27:13.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.520+0000 7f0615c00640 1 -- 172.21.15.152:0/3784749302 shutdown_connections 2024-04-20T11:27:13.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.520+0000 7f0615c00640 1 --2- 172.21.15.152:0/3784749302 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f06100ceeb0 0x7f06100d4ee0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:13.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.520+0000 7f0615c00640 1 -- 172.21.15.152:0/3784749302 >> 172.21.15.152:0/3784749302 conn(0x7f061008bfd0 msgr2=0x7f06100d4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:13.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.520+0000 7f0615c00640 1 -- 172.21.15.152:0/3784749302 shutdown_connections 2024-04-20T11:27:13.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.520+0000 7f0615c00640 1 -- 172.21.15.152:0/3784749302 wait complete. 2024-04-20T11:27:13.522 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.523+0000 7f0615c00640 1 Processor -- start 2024-04-20T11:27:13.522 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.523+0000 7f0615c00640 1 -- start start 2024-04-20T11:27:13.525 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.524+0000 7f0615c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f06100ceeb0 0x7f06101f4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:13.525 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.524+0000 7f0615c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f06100cf5b0 0x7f061009ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:13.526 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.524+0000 7f0615c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0610160230 0x7f06100cd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:13.526 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.524+0000 7f0615c00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f06100d3c10 con 0x7f0610160230 2024-04-20T11:27:13.526 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.524+0000 7f0615c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f06100d36f0 con 0x7f06100cf5b0 2024-04-20T11:27:13.526 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.524+0000 7f0615c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f06100d2790 con 0x7f06100ceeb0 2024-04-20T11:27:13.526 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.524+0000 7f0615200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0610160230 0x7f06100cd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:13.526 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.524+0000 7f0615200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0610160230 0x7f06100cd7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:48414/0 (socket says 172.21.15.152:48414) 2024-04-20T11:27:13.526 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.524+0000 7f0615200640 1 -- 172.21.15.152:0/1594970093 learned_addr learned my addr 172.21.15.152:0/1594970093 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:13.526 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.525+0000 7f060f400640 1 --2- 172.21.15.152:0/1594970093 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f06100cf5b0 0x7f061009ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:13.526 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.525+0000 7f060fe00640 1 --2- 172.21.15.152:0/1594970093 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f06100ceeb0 0x7f06101f4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:13.526 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.525+0000 7f0615200640 1 -- 172.21.15.152:0/1594970093 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f06100ceeb0 msgr2=0x7f06101f4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:13.526 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.525+0000 7f0615200640 1 --2- 172.21.15.152:0/1594970093 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f06100ceeb0 0x7f06101f4fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:13.527 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.525+0000 7f0615200640 1 -- 172.21.15.152:0/1594970093 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f06100cf5b0 msgr2=0x7f061009ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:13.527 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.525+0000 7f0615200640 1 --2- 172.21.15.152:0/1594970093 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f06100cf5b0 0x7f061009ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:13.527 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.525+0000 7f0615200640 1 -- 172.21.15.152:0/1594970093 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f061009f960 con 0x7f0610160230 2024-04-20T11:27:13.527 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.525+0000 7f060fe00640 1 --2- 172.21.15.152:0/1594970093 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f06100ceeb0 0x7f06101f4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:27:13.527 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.525+0000 7f0615200640 1 --2- 172.21.15.152:0/1594970093 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0610160230 0x7f06100cd7f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f060800c710 tx=0x7f060800cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:13.527 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.525+0000 7f0607e00640 1 -- 172.21.15.152:0/1594970093 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0608016020 con 0x7f0610160230 2024-04-20T11:27:13.527 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.526+0000 7f0607e00640 1 -- 172.21.15.152:0/1594970093 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0608006d50 con 0x7f0610160230 2024-04-20T11:27:13.527 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.526+0000 7f0615c00640 1 -- 172.21.15.152:0/1594970093 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06100b84d0 con 0x7f0610160230 2024-04-20T11:27:13.527 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.526+0000 7f0607e00640 1 -- 172.21.15.152:0/1594970093 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0608003bb0 con 0x7f0610160230 2024-04-20T11:27:13.527 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.526+0000 7f0615c00640 1 -- 172.21.15.152:0/1594970093 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06100b8a90 con 0x7f0610160230 2024-04-20T11:27:13.528 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.527+0000 7f0615c00640 1 -- 172.21.15.152:0/1594970093 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f061006e500 con 0x7f0610160230 2024-04-20T11:27:13.528 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.528+0000 7f0607e00640 1 -- 172.21.15.152:0/1594970093 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f060800a850 con 0x7f0610160230 2024-04-20T11:27:13.528 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.529+0000 7f0607e00640 1 --2- 172.21.15.152:0/1594970093 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f05f006c8f0 0x7f05f006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:13.529 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.529+0000 7f0607e00640 1 -- 172.21.15.152:0/1594970093 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f060808b830 con 0x7f0610160230 2024-04-20T11:27:13.529 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.530+0000 7f060fe00640 1 --2- 172.21.15.152:0/1594970093 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f05f006c8f0 0x7f05f006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:13.529 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.530+0000 7f060fe00640 1 --2- 172.21.15.152:0/1594970093 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f05f006c8f0 0x7f05f006edb0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f0600000f60 tx=0x7f0600009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:13.533 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.534+0000 7f0607e00640 1 -- 172.21.15.152:0/1594970093 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f061006e500 con 0x7f0610160230 2024-04-20T11:27:13.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.737+0000 7f0615c00640 1 -- 172.21.15.152:0/1594970093 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f06100693d0 con 0x7f0610160230 2024-04-20T11:27:13.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.738+0000 7f0607e00640 1 -- 172.21.15.152:0/1594970093 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f06100693d0 con 0x7f0610160230 2024-04-20T11:27:13.737 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 64 2024-04-20T11:27:13.740 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.740+0000 7f0615c00640 1 -- 172.21.15.152:0/1594970093 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f05f006c8f0 msgr2=0x7f05f006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:13.741 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.740+0000 7f0615c00640 1 --2- 172.21.15.152:0/1594970093 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f05f006c8f0 0x7f05f006edb0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f0600000f60 tx=0x7f0600009040 comp rx=0 tx=0).stop 2024-04-20T11:27:13.741 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.741+0000 7f0615c00640 1 -- 172.21.15.152:0/1594970093 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0610160230 msgr2=0x7f06100cd7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:13.741 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.741+0000 7f0615c00640 1 --2- 172.21.15.152:0/1594970093 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0610160230 0x7f06100cd7f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f060800c710 tx=0x7f060800cbd0 comp rx=0 tx=0).stop 2024-04-20T11:27:13.741 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.741+0000 7f0615c00640 1 -- 172.21.15.152:0/1594970093 shutdown_connections 2024-04-20T11:27:13.741 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.741+0000 7f0615c00640 1 --2- 172.21.15.152:0/1594970093 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f05f006c8f0 0x7f05f006edb0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:13.741 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.741+0000 7f0615c00640 1 --2- 172.21.15.152:0/1594970093 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0610160230 0x7f06100cd7f0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:13.741 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.741+0000 7f0615c00640 1 --2- 172.21.15.152:0/1594970093 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f06100cf5b0 0x7f061009ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:13.741 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.741+0000 7f0615c00640 1 --2- 172.21.15.152:0/1594970093 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f06100ceeb0 0x7f06101f4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:13.741 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.741+0000 7f0615c00640 1 -- 172.21.15.152:0/1594970093 >> 172.21.15.152:0/1594970093 conn(0x7f061008bfd0 msgr2=0x7f061006da30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:13.741 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.741+0000 7f0615c00640 1 -- 172.21.15.152:0/1594970093 shutdown_connections 2024-04-20T11:27:13.741 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.741+0000 7f0615c00640 1 -- 172.21.15.152:0/1594970093 wait complete. 2024-04-20T11:27:13.760 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:27:13.957 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.958+0000 7f7262000640 1 Processor -- start 2024-04-20T11:27:13.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.958+0000 7f7262000640 1 -- start start 2024-04-20T11:27:13.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.959+0000 7f7262000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f725c0cd0d0 0x7f725c0cd580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:13.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.959+0000 7f7262000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f725c071fe0 0x7f725c072470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:13.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.960+0000 7f7262000640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f725c146340 con 0x7f725c145f60 2024-04-20T11:27:13.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.960+0000 7f7262000640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f725c06e9f0 con 0x7f725c0cd0d0 2024-04-20T11:27:13.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.960+0000 7f7262000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f725c072d50 con 0x7f725c071fe0 2024-04-20T11:27:13.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.960+0000 7f725b400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f725c145f60 0x7f725c06ee80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:48572/0 (socket says 172.21.15.152:48572) 2024-04-20T11:27:13.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.960+0000 7f725b400640 1 -- 172.21.15.152:0/287485814 learned_addr learned my addr 172.21.15.152:0/287485814 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:27:13.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.960+0000 7f725aa00640 1 --2- 172.21.15.152:0/287485814 >> v2:172.21.15.152:3300/0 conn(0x7f725c0cd0d0 0x7f725c0cd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:13.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.960+0000 7f725be00640 1 --2- 172.21.15.152:0/287485814 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f725c071fe0 0x7f725c072470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:13.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.960+0000 7f725a000640 1 -- 172.21.15.152:0/287485814 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3907213796 0 0) 0x7f725c146340 con 0x7f725c145f60 2024-04-20T11:27:13.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.961+0000 7f725a000640 1 -- 172.21.15.152:0/287485814 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7240003520 con 0x7f725c145f60 2024-04-20T11:27:13.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.961+0000 7f725aa00640 1 -- 172.21.15.152:0/287485814 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f725c071fe0 msgr2=0x7f725c072470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:13.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.961+0000 7f725aa00640 1 --2- 172.21.15.152:0/287485814 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f725c071fe0 0x7f725c072470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:13.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.961+0000 7f725aa00640 1 -- 172.21.15.152:0/287485814 >> v1:172.21.15.152:6789/0 conn(0x7f725c145f60 legacy=0x7f725c06ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:13.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.961+0000 7f725aa00640 1 -- 172.21.15.152:0/287485814 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f725c06e0c0 con 0x7f725c0cd0d0 2024-04-20T11:27:13.963 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.962+0000 7f725aa00640 1 --2- 172.21.15.152:0/287485814 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f725c0cd0d0 0x7f725c0cd580 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f72480097a0 tx=0x7f724802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4f75af983115326 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:13.963 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.962+0000 7f725a000640 1 -- 172.21.15.152:0/287485814 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7248035020 con 0x7f725c0cd0d0 2024-04-20T11:27:13.963 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.962+0000 7f725a000640 1 -- 172.21.15.152:0/287485814 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7248031830 con 0x7f725c0cd0d0 2024-04-20T11:27:13.963 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.962+0000 7f725a000640 1 -- 172.21.15.152:0/287485814 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7248031be0 con 0x7f725c0cd0d0 2024-04-20T11:27:13.964 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.963+0000 7f7262000640 1 -- 172.21.15.152:0/287485814 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f725c0cd0d0 msgr2=0x7f725c0cd580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:13.964 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.963+0000 7f7262000640 1 --2- 172.21.15.152:0/287485814 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f725c0cd0d0 0x7f725c0cd580 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f72480097a0 tx=0x7f724802b8d0 comp rx=0 tx=0).stop 2024-04-20T11:27:13.964 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.963+0000 7f7262000640 1 -- 172.21.15.152:0/287485814 shutdown_connections 2024-04-20T11:27:13.964 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.963+0000 7f7262000640 1 --2- 172.21.15.152:0/287485814 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f725c071fe0 0x7f725c072470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:13.964 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.963+0000 7f7262000640 1 --2- 172.21.15.152:0/287485814 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f725c0cd0d0 0x7f725c0cd580 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:13.964 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.963+0000 7f7262000640 1 -- 172.21.15.152:0/287485814 >> 172.21.15.152:0/287485814 conn(0x7f725c067cf0 msgr2=0x7f725c06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:13.964 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.963+0000 7f7262000640 1 -- 172.21.15.152:0/287485814 shutdown_connections 2024-04-20T11:27:13.964 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.963+0000 7f7262000640 1 -- 172.21.15.152:0/287485814 wait complete. 2024-04-20T11:27:13.966 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.964+0000 7f7262000640 1 Processor -- start 2024-04-20T11:27:13.966 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.965+0000 7f7262000640 1 -- start start 2024-04-20T11:27:13.966 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.965+0000 7f7262000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f725c071fe0 0x7f725c0af490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:13.967 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.966+0000 7f7262000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f725c0cd0d0 0x7f725c0ad440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:13.967 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.966+0000 7f7262000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f725c145f60 0x7f725c0ad980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:13.967 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.966+0000 7f7262000640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f725c0729b0 con 0x7f725c071fe0 2024-04-20T11:27:13.967 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.966+0000 7f7262000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f725c06e680 con 0x7f725c0cd0d0 2024-04-20T11:27:13.967 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.966+0000 7f7262000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f725c072b30 con 0x7f725c145f60 2024-04-20T11:27:13.967 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.966+0000 7f725b400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f725c071fe0 0x7f725c0af490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:13.968 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.966+0000 7f725b400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f725c071fe0 0x7f725c0af490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:48434/0 (socket says 172.21.15.152:48434) 2024-04-20T11:27:13.968 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.966+0000 7f725b400640 1 -- 172.21.15.152:0/2917068933 learned_addr learned my addr 172.21.15.152:0/2917068933 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:13.968 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.966+0000 7f725b400640 1 -- 172.21.15.152:0/2917068933 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f725c145f60 msgr2=0x7f725c0ad980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:13.968 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.966+0000 7f725b400640 1 --2- 172.21.15.152:0/2917068933 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f725c145f60 0x7f725c0ad980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:13.968 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.966+0000 7f725b400640 1 -- 172.21.15.152:0/2917068933 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f725c0cd0d0 msgr2=0x7f725c0ad440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:13.968 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.966+0000 7f725b400640 1 --2- 172.21.15.152:0/2917068933 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f725c0cd0d0 0x7f725c0ad440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:13.968 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.966+0000 7f725b400640 1 -- 172.21.15.152:0/2917068933 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f725c0afd30 con 0x7f725c071fe0 2024-04-20T11:27:13.968 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.967+0000 7f725b400640 1 --2- 172.21.15.152:0/2917068933 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f725c071fe0 0x7f725c0af490 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f724c0010e0 tx=0x7f724c00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:13.968 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.967+0000 7f7253400640 1 -- 172.21.15.152:0/2917068933 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f724c017070 con 0x7f725c071fe0 2024-04-20T11:27:13.968 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.967+0000 7f7253400640 1 -- 172.21.15.152:0/2917068933 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f724c005ce0 con 0x7f725c071fe0 2024-04-20T11:27:13.968 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.967+0000 7f7253400640 1 -- 172.21.15.152:0/2917068933 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f724c007310 con 0x7f725c071fe0 2024-04-20T11:27:13.968 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.967+0000 7f7262000640 1 -- 172.21.15.152:0/2917068933 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f725c133180 con 0x7f725c071fe0 2024-04-20T11:27:13.969 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.967+0000 7f7262000640 1 -- 172.21.15.152:0/2917068933 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f725c133740 con 0x7f725c071fe0 2024-04-20T11:27:13.969 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.968+0000 7f7262000640 1 -- 172.21.15.152:0/2917068933 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f725c0689f0 con 0x7f725c071fe0 2024-04-20T11:27:13.969 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.969+0000 7f7253400640 1 -- 172.21.15.152:0/2917068933 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f724c01a070 con 0x7f725c071fe0 2024-04-20T11:27:13.970 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.970+0000 7f7253400640 1 --2- 172.21.15.152:0/2917068933 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f723c06c8f0 0x7f723c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:13.970 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.970+0000 7f7253400640 1 -- 172.21.15.152:0/2917068933 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f724c08ae50 con 0x7f725c071fe0 2024-04-20T11:27:13.970 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.970+0000 7f725aa00640 1 --2- 172.21.15.152:0/2917068933 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f723c06c8f0 0x7f723c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:13.970 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.971+0000 7f725aa00640 1 --2- 172.21.15.152:0/2917068933 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f723c06c8f0 0x7f723c06edb0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7248036000 tx=0x7f7248030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:13.974 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:13.975+0000 7f7253400640 1 -- 172.21.15.152:0/2917068933 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f724c059310 con 0x7f725c071fe0 2024-04-20T11:27:14.214 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.215+0000 7f7262000640 1 -- 172.21.15.152:0/2917068933 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f725c06cd90 con 0x7f725c071fe0 2024-04-20T11:27:14.216 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.216+0000 7f7253400640 1 -- 172.21.15.152:0/2917068933 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5196 (secure 0 0 0) 0x7f724c058c10 con 0x7f725c071fe0 2024-04-20T11:27:14.216 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:27:14.216 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/853233806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":853233806},{"type":"v1","addr":"172.21.15.175:6833","nonce":853233806}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4253,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/100300237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":100300237},{"type":"v1","addr":"172.21.15.175:6835","nonce":100300237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/2924034058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":2924034058},{"type":"v1","addr":"172.21.15.152:6835","nonce":2924034058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T11:26:58.132293+0000","modified":"2024-04-20T11:27:01.444285+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4269":{"gid":4269,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6837/927639144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":927639144},{"type":"v1","addr":"172.21.15.152:6837","nonce":927639144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4269,"qdb_cluster":[4269]},"id":1}]} 2024-04-20T11:27:14.216 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 7 2024-04-20T11:27:14.219 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.219+0000 7f7262000640 1 -- 172.21.15.152:0/2917068933 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f723c06c8f0 msgr2=0x7f723c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:14.219 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.219+0000 7f7262000640 1 --2- 172.21.15.152:0/2917068933 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f723c06c8f0 0x7f723c06edb0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7248036000 tx=0x7f7248030000 comp rx=0 tx=0).stop 2024-04-20T11:27:14.219 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.219+0000 7f7262000640 1 -- 172.21.15.152:0/2917068933 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f725c071fe0 msgr2=0x7f725c0af490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:14.219 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.219+0000 7f7262000640 1 --2- 172.21.15.152:0/2917068933 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f725c071fe0 0x7f725c0af490 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f724c0010e0 tx=0x7f724c00c890 comp rx=0 tx=0).stop 2024-04-20T11:27:14.219 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.219+0000 7f7262000640 1 -- 172.21.15.152:0/2917068933 shutdown_connections 2024-04-20T11:27:14.219 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.219+0000 7f7262000640 1 --2- 172.21.15.152:0/2917068933 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f723c06c8f0 0x7f723c06edb0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:14.219 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.219+0000 7f7262000640 1 --2- 172.21.15.152:0/2917068933 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f725c145f60 0x7f725c0ad980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:14.219 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.219+0000 7f7262000640 1 --2- 172.21.15.152:0/2917068933 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f725c0cd0d0 0x7f725c0ad440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:14.219 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.219+0000 7f7262000640 1 --2- 172.21.15.152:0/2917068933 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f725c071fe0 0x7f725c0af490 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:14.219 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.219+0000 7f7262000640 1 -- 172.21.15.152:0/2917068933 >> 172.21.15.152:0/2917068933 conn(0x7f725c067cf0 msgr2=0x7f725c15be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:14.219 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.219+0000 7f7262000640 1 -- 172.21.15.152:0/2917068933 shutdown_connections 2024-04-20T11:27:14.219 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.219+0000 7f7262000640 1 -- 172.21.15.152:0/2917068933 wait complete. 2024-04-20T11:27:14.238 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:27:14.444 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.444+0000 7f9b8be00640 1 Processor -- start 2024-04-20T11:27:14.444 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.444+0000 7f9b8be00640 1 -- start start 2024-04-20T11:27:14.446 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.446+0000 7f9b8be00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f9b8c147410 0x7f9b8c0afa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:14.446 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.446+0000 7f9b8be00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9b8c06e720 0x7f9b8c072130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:14.446 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.446+0000 7f9b8be00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b8c0aec90 con 0x7f9b8c071cd0 2024-04-20T11:27:14.446 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.446+0000 7f9b8be00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f9b8c072670 con 0x7f9b8c147410 2024-04-20T11:27:14.446 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.446+0000 7f9b8be00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f9b8c0aef70 con 0x7f9b8c06e720 2024-04-20T11:27:14.446 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.446+0000 7f9b8aa00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f9b8c147410 0x7f9b8c0afa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:14.446 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.446+0000 7f9b8aa00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f9b8c147410 0x7f9b8c0afa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:48444/0 (socket says 172.21.15.152:48444) 2024-04-20T11:27:14.446 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.446+0000 7f9b8aa00640 1 -- 172.21.15.152:0/8091272 learned_addr learned my addr 172.21.15.152:0/8091272 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:14.447 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.447+0000 7f9b8b400640 1 --2- 172.21.15.152:0/8091272 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9b8c06e720 0x7f9b8c072130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:14.447 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.447+0000 7f9b8aa00640 1 -- 172.21.15.152:0/8091272 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9b8c06e720 msgr2=0x7f9b8c072130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:14.447 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.447+0000 7f9b8aa00640 1 --2- 172.21.15.152:0/8091272 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9b8c06e720 0x7f9b8c072130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:14.447 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.447+0000 7f9b8aa00640 1 -- 172.21.15.152:0/8091272 >> v1:172.21.15.175:6789/0 conn(0x7f9b8c071cd0 legacy=0x7f9b8c06e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:14.447 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.447+0000 7f9b8aa00640 1 -- 172.21.15.152:0/8091272 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b8c0af250 con 0x7f9b8c147410 2024-04-20T11:27:14.447 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.447+0000 7f9b8b400640 1 --2- 172.21.15.152:0/8091272 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9b8c06e720 0x7f9b8c072130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:27:14.449 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.448+0000 7f9b8aa00640 1 --2- 172.21.15.152:0/8091272 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9b8c147410 0x7f9b8c0afa50 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c009870 tx=0x7f9b7c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b3e644aad5d7768 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:14.449 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.448+0000 7f9b8a000640 1 -- 172.21.15.152:0/8091272 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b7c035020 con 0x7f9b8c147410 2024-04-20T11:27:14.449 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.448+0000 7f9b8a000640 1 -- 172.21.15.152:0/8091272 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b7c031640 con 0x7f9b8c147410 2024-04-20T11:27:14.449 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.448+0000 7f9b8a000640 1 -- 172.21.15.152:0/8091272 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b7c02d040 con 0x7f9b8c147410 2024-04-20T11:27:14.449 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.449+0000 7f9b8be00640 1 -- 172.21.15.152:0/8091272 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9b8c147410 msgr2=0x7f9b8c0afa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:14.449 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.449+0000 7f9b8be00640 1 --2- 172.21.15.152:0/8091272 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9b8c147410 0x7f9b8c0afa50 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c009870 tx=0x7f9b7c02b8f0 comp rx=0 tx=0).stop 2024-04-20T11:27:14.449 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.449+0000 7f9b8be00640 1 -- 172.21.15.152:0/8091272 shutdown_connections 2024-04-20T11:27:14.449 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.449+0000 7f9b8be00640 1 --2- 172.21.15.152:0/8091272 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9b8c06e720 0x7f9b8c072130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:14.449 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.449+0000 7f9b8be00640 1 --2- 172.21.15.152:0/8091272 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9b8c147410 0x7f9b8c0afa50 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:14.449 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.449+0000 7f9b8be00640 1 -- 172.21.15.152:0/8091272 >> 172.21.15.152:0/8091272 conn(0x7f9b8c067ae0 msgr2=0x7f9b8c06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:14.449 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.449+0000 7f9b8be00640 1 -- 172.21.15.152:0/8091272 shutdown_connections 2024-04-20T11:27:14.449 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.449+0000 7f9b8be00640 1 -- 172.21.15.152:0/8091272 wait complete. 2024-04-20T11:27:14.450 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.451+0000 7f9b8be00640 1 Processor -- start 2024-04-20T11:27:14.450 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.451+0000 7f9b8be00640 1 -- start start 2024-04-20T11:27:14.452 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.452+0000 7f9b8be00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9b8c06e720 0x7f9b8c13e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:14.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.452+0000 7f9b8be00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9b8c071cd0 0x7f9b8c13e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:14.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.452+0000 7f9b8be00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9b8c147410 0x7f9b8c13bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:14.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.452+0000 7f9b8be00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f9b8c0b3390 con 0x7f9b8c06e720 2024-04-20T11:27:14.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.452+0000 7f9b8be00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9b8c0ae980 con 0x7f9b8c147410 2024-04-20T11:27:14.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.452+0000 7f9b8be00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f9b8c0b3510 con 0x7f9b8c071cd0 2024-04-20T11:27:14.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.452+0000 7f9b8aa00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9b8c06e720 0x7f9b8c13e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:14.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.452+0000 7f9b8aa00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9b8c06e720 0x7f9b8c13e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:48456/0 (socket says 172.21.15.152:48456) 2024-04-20T11:27:14.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.452+0000 7f9b8aa00640 1 -- 172.21.15.152:0/3549566527 learned_addr learned my addr 172.21.15.152:0/3549566527 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:14.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.452+0000 7f9b8aa00640 1 -- 172.21.15.152:0/3549566527 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9b8c071cd0 msgr2=0x7f9b8c13e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:14.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.452+0000 7f9b8aa00640 1 --2- 172.21.15.152:0/3549566527 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9b8c071cd0 0x7f9b8c13e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:14.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.452+0000 7f9b8aa00640 1 -- 172.21.15.152:0/3549566527 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9b8c147410 msgr2=0x7f9b8c13bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:14.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.452+0000 7f9b8aa00640 1 --2- 172.21.15.152:0/3549566527 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9b8c147410 0x7f9b8c13bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:14.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.453+0000 7f9b8aa00640 1 -- 172.21.15.152:0/3549566527 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b8c15f970 con 0x7f9b8c06e720 2024-04-20T11:27:14.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.453+0000 7f9b8aa00640 1 --2- 172.21.15.152:0/3549566527 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9b8c06e720 0x7f9b8c13e4a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c000c00 tx=0x7f9b7c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:14.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.453+0000 7f9b82a00640 1 -- 172.21.15.152:0/3549566527 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b7c040070 con 0x7f9b8c06e720 2024-04-20T11:27:14.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.453+0000 7f9b82a00640 1 -- 172.21.15.152:0/3549566527 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b7c039bf0 con 0x7f9b8c06e720 2024-04-20T11:27:14.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.453+0000 7f9b8be00640 1 -- 172.21.15.152:0/3549566527 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b8c15fc50 con 0x7f9b8c06e720 2024-04-20T11:27:14.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.453+0000 7f9b82a00640 1 -- 172.21.15.152:0/3549566527 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b7c035040 con 0x7f9b8c06e720 2024-04-20T11:27:14.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.453+0000 7f9b8be00640 1 -- 172.21.15.152:0/3549566527 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b8c160210 con 0x7f9b8c06e720 2024-04-20T11:27:14.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.455+0000 7f9b8be00640 1 -- 172.21.15.152:0/3549566527 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b8c075e60 con 0x7f9b8c06e720 2024-04-20T11:27:14.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.455+0000 7f9b82a00640 1 -- 172.21.15.152:0/3549566527 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9b7c038020 con 0x7f9b8c06e720 2024-04-20T11:27:14.456 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.456+0000 7f9b82a00640 1 --2- 172.21.15.152:0/3549566527 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f9b6806c8f0 0x7f9b6806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:14.456 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.456+0000 7f9b82a00640 1 -- 172.21.15.152:0/3549566527 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f9b7c0ad680 con 0x7f9b8c06e720 2024-04-20T11:27:14.456 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.456+0000 7f9b83e00640 1 --2- 172.21.15.152:0/3549566527 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f9b6806c8f0 0x7f9b6806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:14.456 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.457+0000 7f9b83e00640 1 --2- 172.21.15.152:0/3549566527 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f9b6806c8f0 0x7f9b6806edb0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c13b630 tx=0x7f9b740079b0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:14.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.461+0000 7f9b82a00640 1 -- 172.21.15.152:0/3549566527 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f9b7c07bb40 con 0x7f9b8c06e720 2024-04-20T11:27:14.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.660+0000 7f9b8be00640 1 -- 172.21.15.152:0/3549566527 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9b8c0b44b0 con 0x7f9b8c06e720 2024-04-20T11:27:14.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.663+0000 7f9b82a00640 1 -- 172.21.15.152:0/3549566527 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16665 (secure 0 0 0) 0x7f9b7c07b440 con 0x7f9b8c06e720 2024-04-20T11:27:14.664 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:27:14.664 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":20,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:26:59.675971+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:26:56.261424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T11:26:57.079286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:27:14.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.666+0000 7f9b8be00640 1 -- 172.21.15.152:0/3549566527 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f9b6806c8f0 msgr2=0x7f9b6806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:14.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.666+0000 7f9b8be00640 1 --2- 172.21.15.152:0/3549566527 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f9b6806c8f0 0x7f9b6806edb0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c13b630 tx=0x7f9b740079b0 comp rx=0 tx=0).stop 2024-04-20T11:27:14.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.666+0000 7f9b8be00640 1 -- 172.21.15.152:0/3549566527 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9b8c06e720 msgr2=0x7f9b8c13e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:14.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.666+0000 7f9b8be00640 1 --2- 172.21.15.152:0/3549566527 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9b8c06e720 0x7f9b8c13e4a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c000c00 tx=0x7f9b7c0046c0 comp rx=0 tx=0).stop 2024-04-20T11:27:14.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.667+0000 7f9b8be00640 1 -- 172.21.15.152:0/3549566527 shutdown_connections 2024-04-20T11:27:14.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.667+0000 7f9b8be00640 1 --2- 172.21.15.152:0/3549566527 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f9b6806c8f0 0x7f9b6806edb0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:14.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.667+0000 7f9b8be00640 1 --2- 172.21.15.152:0/3549566527 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9b8c147410 0x7f9b8c13bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:14.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.667+0000 7f9b8be00640 1 --2- 172.21.15.152:0/3549566527 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9b8c071cd0 0x7f9b8c13e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:14.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.667+0000 7f9b8be00640 1 --2- 172.21.15.152:0/3549566527 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9b8c06e720 0x7f9b8c13e4a0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:14.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.667+0000 7f9b8be00640 1 -- 172.21.15.152:0/3549566527 >> 172.21.15.152:0/3549566527 conn(0x7f9b8c067ae0 msgr2=0x7f9b8c06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:14.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.667+0000 7f9b8be00640 1 -- 172.21.15.152:0/3549566527 shutdown_connections 2024-04-20T11:27:14.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.667+0000 7f9b8be00640 1 -- 172.21.15.152:0/3549566527 wait complete. 2024-04-20T11:27:14.687 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:27:14.890 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.890+0000 7f5eb2800640 1 Processor -- start 2024-04-20T11:27:14.890 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.890+0000 7f5eb2800640 1 -- start start 2024-04-20T11:27:14.891 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.892+0000 7f5eb2800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f5eac0726f0 0x7f5eac0cd580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:14.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.892+0000 7f5eb2800640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5eac146340 con 0x7f5eac145f60 2024-04-20T11:27:14.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.892+0000 7f5eb2800640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5eac072d50 con 0x7f5eac0cd0d0 2024-04-20T11:27:14.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.892+0000 7f5eb2800640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f5eac06e9f0 con 0x7f5eac0726f0 2024-04-20T11:27:14.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.892+0000 7f5eab400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f5eac145f60 0x7f5eac06ee80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:48580/0 (socket says 172.21.15.152:48580) 2024-04-20T11:27:14.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.892+0000 7f5eab400640 1 -- 172.21.15.152:0/2187054593 learned_addr learned my addr 172.21.15.152:0/2187054593 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:27:14.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.892+0000 7f5eabe00640 1 --2- 172.21.15.152:0/2187054593 >> v2:172.21.15.152:3300/0 conn(0x7f5eac0726f0 0x7f5eac0cd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:14.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.893+0000 7f5eaaa00640 1 -- 172.21.15.152:0/2187054593 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2998692141 0 0) 0x7f5eac146340 con 0x7f5eac145f60 2024-04-20T11:27:14.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.893+0000 7f5eaaa00640 1 -- 172.21.15.152:0/2187054593 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5e90003520 con 0x7f5eac145f60 2024-04-20T11:27:14.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.893+0000 7f5eabe00640 1 -- 172.21.15.152:0/2187054593 >> v1:172.21.15.175:6789/0 conn(0x7f5eac0cd0d0 legacy=0x7f5eac071fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:14.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.893+0000 7f5eabe00640 1 -- 172.21.15.152:0/2187054593 >> v1:172.21.15.152:6789/0 conn(0x7f5eac145f60 legacy=0x7f5eac06ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:14.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.893+0000 7f5eabe00640 1 -- 172.21.15.152:0/2187054593 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5eac06e0c0 con 0x7f5eac0726f0 2024-04-20T11:27:14.894 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.894+0000 7f5eabe00640 1 --2- 172.21.15.152:0/2187054593 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5eac0726f0 0x7f5eac0cd580 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5ea4009d90 tx=0x7f5ea402bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=699a9611217bc87b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:14.894 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.894+0000 7f5eaaa00640 1 -- 172.21.15.152:0/2187054593 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea4039070 con 0x7f5eac0726f0 2024-04-20T11:27:14.894 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.894+0000 7f5eaaa00640 1 -- 172.21.15.152:0/2187054593 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ea402cd00 con 0x7f5eac0726f0 2024-04-20T11:27:14.894 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.894+0000 7f5eaaa00640 1 -- 172.21.15.152:0/2187054593 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea4034640 con 0x7f5eac0726f0 2024-04-20T11:27:14.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.895+0000 7f5eb2800640 1 -- 172.21.15.152:0/2187054593 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5eac0726f0 msgr2=0x7f5eac0cd580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:14.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.895+0000 7f5eb2800640 1 --2- 172.21.15.152:0/2187054593 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5eac0726f0 0x7f5eac0cd580 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5ea4009d90 tx=0x7f5ea402bf50 comp rx=0 tx=0).stop 2024-04-20T11:27:14.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.895+0000 7f5eb2800640 1 -- 172.21.15.152:0/2187054593 shutdown_connections 2024-04-20T11:27:14.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.895+0000 7f5eb2800640 1 --2- 172.21.15.152:0/2187054593 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5eac0726f0 0x7f5eac0cd580 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:14.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.895+0000 7f5eb2800640 1 -- 172.21.15.152:0/2187054593 >> 172.21.15.152:0/2187054593 conn(0x7f5eac067cf0 msgr2=0x7f5eac06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:14.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.895+0000 7f5eb2800640 1 -- 172.21.15.152:0/2187054593 shutdown_connections 2024-04-20T11:27:14.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.895+0000 7f5eb2800640 1 -- 172.21.15.152:0/2187054593 wait complete. 2024-04-20T11:27:14.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.897+0000 7f5eb2800640 1 Processor -- start 2024-04-20T11:27:14.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.897+0000 7f5eb2800640 1 -- start start 2024-04-20T11:27:14.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.898+0000 7f5eb2800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5eac0726f0 0x7f5eac0b4830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:14.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.898+0000 7f5eb2800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5eac0cd0d0 0x7f5eac0c8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:14.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.898+0000 7f5eb2800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5eac145f60 0x7f5eac0c9200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:14.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.898+0000 7f5eb2800640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f5eac0cf290 con 0x7f5eac0726f0 2024-04-20T11:27:14.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.898+0000 7f5eb2800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5eac06e680 con 0x7f5eac145f60 2024-04-20T11:27:14.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.898+0000 7f5eb2800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5eac0cf410 con 0x7f5eac0cd0d0 2024-04-20T11:27:14.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.898+0000 7f5eab400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5eac0726f0 0x7f5eac0b4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:14.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.898+0000 7f5eab400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5eac0726f0 0x7f5eac0b4830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:48486/0 (socket says 172.21.15.152:48486) 2024-04-20T11:27:14.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.898+0000 7f5eab400640 1 -- 172.21.15.152:0/193775441 learned_addr learned my addr 172.21.15.152:0/193775441 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:14.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.898+0000 7f5eabe00640 1 --2- 172.21.15.152:0/193775441 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5eac145f60 0x7f5eac0c9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:14.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.898+0000 7f5ea3e00640 1 --2- 172.21.15.152:0/193775441 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5eac0cd0d0 0x7f5eac0c8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:14.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.898+0000 7f5eab400640 1 -- 172.21.15.152:0/193775441 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5eac0cd0d0 msgr2=0x7f5eac0c8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:14.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.899+0000 7f5eab400640 1 --2- 172.21.15.152:0/193775441 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5eac0cd0d0 0x7f5eac0c8cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:14.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.899+0000 7f5eab400640 1 -- 172.21.15.152:0/193775441 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5eac145f60 msgr2=0x7f5eac0c9200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:14.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.899+0000 7f5eab400640 1 --2- 172.21.15.152:0/193775441 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5eac145f60 0x7f5eac0c9200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:14.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.899+0000 7f5eab400640 1 -- 172.21.15.152:0/193775441 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5eac13c530 con 0x7f5eac0726f0 2024-04-20T11:27:14.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.899+0000 7f5ea3e00640 1 --2- 172.21.15.152:0/193775441 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5eac0cd0d0 0x7f5eac0c8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:27:14.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.899+0000 7f5eab400640 1 --2- 172.21.15.152:0/193775441 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5eac0726f0 0x7f5eac0b4830 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c00c410 tx=0x7f5e9c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:14.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.899+0000 7f5ea3400640 1 -- 172.21.15.152:0/193775441 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e9c017070 con 0x7f5eac0726f0 2024-04-20T11:27:14.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.899+0000 7f5ea3400640 1 -- 172.21.15.152:0/193775441 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e9c005ce0 con 0x7f5eac0726f0 2024-04-20T11:27:14.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.899+0000 7f5eb2800640 1 -- 172.21.15.152:0/193775441 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5eac0caa80 con 0x7f5eac0726f0 2024-04-20T11:27:14.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.899+0000 7f5ea3400640 1 -- 172.21.15.152:0/193775441 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e9c0072f0 con 0x7f5eac0726f0 2024-04-20T11:27:14.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.899+0000 7f5eb2800640 1 -- 172.21.15.152:0/193775441 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5eac139650 con 0x7f5eac0726f0 2024-04-20T11:27:14.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.900+0000 7f5eb2800640 1 -- 172.21.15.152:0/193775441 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5eac0689f0 con 0x7f5eac0726f0 2024-04-20T11:27:14.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.901+0000 7f5ea3400640 1 -- 172.21.15.152:0/193775441 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5e9c01a070 con 0x7f5eac0726f0 2024-04-20T11:27:14.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.902+0000 7f5ea3400640 1 --2- 172.21.15.152:0/193775441 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f5e8806c8f0 0x7f5e8806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:14.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.902+0000 7f5ea3400640 1 -- 172.21.15.152:0/193775441 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f5e9c08af90 con 0x7f5eac0726f0 2024-04-20T11:27:14.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.903+0000 7f5ea3e00640 1 --2- 172.21.15.152:0/193775441 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f5e8806c8f0 0x7f5e8806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:14.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.903+0000 7f5ea3e00640 1 --2- 172.21.15.152:0/193775441 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f5e8806c8f0 0x7f5e8806edb0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5eac0cff30 tx=0x7f5e94009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:14.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:14.907+0000 7f5ea3400640 1 -- 172.21.15.152:0/193775441 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f5eac0caa80 con 0x7f5eac0726f0 2024-04-20T11:27:15.146 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.146+0000 7f5eb2800640 1 -- 172.21.15.152:0/193775441 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5eac0cd900 con 0x7f5eac0726f0 2024-04-20T11:27:15.146 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.147+0000 7f5ea3400640 1 -- 172.21.15.152:0/193775441 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5196 (secure 0 0 0) 0x7f5e9c059480 con 0x7f5eac0726f0 2024-04-20T11:27:15.147 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:27:15.147 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/853233806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":853233806},{"type":"v1","addr":"172.21.15.175:6833","nonce":853233806}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4253,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/100300237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":100300237},{"type":"v1","addr":"172.21.15.175:6835","nonce":100300237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/2924034058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":2924034058},{"type":"v1","addr":"172.21.15.152:6835","nonce":2924034058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T11:26:58.132293+0000","modified":"2024-04-20T11:27:01.444285+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4269":{"gid":4269,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6837/927639144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":927639144},{"type":"v1","addr":"172.21.15.152:6837","nonce":927639144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4269,"qdb_cluster":[4269]},"id":1}]} 2024-04-20T11:27:15.147 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 7 2024-04-20T11:27:15.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.149+0000 7f5eb2800640 1 -- 172.21.15.152:0/193775441 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f5e8806c8f0 msgr2=0x7f5e8806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:15.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.149+0000 7f5eb2800640 1 --2- 172.21.15.152:0/193775441 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f5e8806c8f0 0x7f5e8806edb0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5eac0cff30 tx=0x7f5e94009040 comp rx=0 tx=0).stop 2024-04-20T11:27:15.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.149+0000 7f5eb2800640 1 -- 172.21.15.152:0/193775441 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5eac0726f0 msgr2=0x7f5eac0b4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:15.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.149+0000 7f5eb2800640 1 --2- 172.21.15.152:0/193775441 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5eac0726f0 0x7f5eac0b4830 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c00c410 tx=0x7f5e9c00c8d0 comp rx=0 tx=0).stop 2024-04-20T11:27:15.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.150+0000 7f5eb2800640 1 -- 172.21.15.152:0/193775441 shutdown_connections 2024-04-20T11:27:15.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.150+0000 7f5eb2800640 1 --2- 172.21.15.152:0/193775441 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f5e8806c8f0 0x7f5e8806edb0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:15.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.150+0000 7f5eb2800640 1 --2- 172.21.15.152:0/193775441 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5eac145f60 0x7f5eac0c9200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:15.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.150+0000 7f5eb2800640 1 --2- 172.21.15.152:0/193775441 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5eac0cd0d0 0x7f5eac0c8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:15.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.150+0000 7f5eb2800640 1 --2- 172.21.15.152:0/193775441 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5eac0726f0 0x7f5eac0b4830 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:15.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.150+0000 7f5eb2800640 1 -- 172.21.15.152:0/193775441 >> 172.21.15.152:0/193775441 conn(0x7f5eac067cf0 msgr2=0x7f5eac0b1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:15.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.150+0000 7f5eb2800640 1 -- 172.21.15.152:0/193775441 shutdown_connections 2024-04-20T11:27:15.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.150+0000 7f5eb2800640 1 -- 172.21.15.152:0/193775441 wait complete. 2024-04-20T11:27:15.572 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-20T11:26:58.132293+0000', 'modified': '2024-04-20T11:27:01.444285+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'snap_rstat': True, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4269}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4269': {'gid': 4269, 'name': 'c', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.152:6837/927639144', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6836', 'nonce': 927639144}, {'type': 'v1', 'addr': '172.21.15.152:6837', 'nonce': 927639144}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4269, 'qdb_cluster': [4269]} 2024-04-20T11:27:15.572 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-20T11:27:15.572 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-20T11:27:15.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.783+0000 7fe382200640 1 Processor -- start 2024-04-20T11:27:15.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.783+0000 7fe382200640 1 -- start start 2024-04-20T11:27:15.785 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.785+0000 7fe382200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe37c0d4ec0 0x7fe37c0d0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:15.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.785+0000 7fe382200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fe37c0d2b80 0x7fe37c0d3010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:15.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.785+0000 7fe382200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe37c0d26a0 con 0x7fe37c161d30 2024-04-20T11:27:15.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.785+0000 7fe382200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fe37c0d0540 con 0x7fe37c0d2b80 2024-04-20T11:27:15.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.785+0000 7fe382200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe37c0cfb70 con 0x7fe37c0d4ec0 2024-04-20T11:27:15.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.785+0000 7fe381800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fe37c0d2b80 0x7fe37c0d3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:15.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.785+0000 7fe381800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fe37c0d2b80 0x7fe37c0d3010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:48500/0 (socket says 172.21.15.152:48500) 2024-04-20T11:27:15.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.785+0000 7fe381800640 1 -- 172.21.15.152:0/679434992 learned_addr learned my addr 172.21.15.152:0/679434992 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:15.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.786+0000 7fe37be00640 1 --2- 172.21.15.152:0/679434992 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe37c0d4ec0 0x7fe37c0d0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:15.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.786+0000 7fe381800640 1 -- 172.21.15.152:0/679434992 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe37c0d4ec0 msgr2=0x7fe37c0d0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:15.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.786+0000 7fe381800640 1 --2- 172.21.15.152:0/679434992 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe37c0d4ec0 0x7fe37c0d0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:15.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.786+0000 7fe381800640 1 -- 172.21.15.152:0/679434992 >> v1:172.21.15.175:6789/0 conn(0x7fe37c161d30 legacy=0x7fe37c0d15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:15.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.786+0000 7fe381800640 1 -- 172.21.15.152:0/679434992 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe37c0d53c0 con 0x7fe37c0d2b80 2024-04-20T11:27:15.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.787+0000 7fe381800640 1 --2- 172.21.15.152:0/679434992 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe37c0d2b80 0x7fe37c0d3010 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe374009e50 tx=0x7fe37402bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a310a06a24e75118 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:15.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.787+0000 7fe37b400640 1 -- 172.21.15.152:0/679434992 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe37402cb00 con 0x7fe37c0d2b80 2024-04-20T11:27:15.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.787+0000 7fe37b400640 1 -- 172.21.15.152:0/679434992 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe37402cca0 con 0x7fe37c0d2b80 2024-04-20T11:27:15.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.787+0000 7fe37b400640 1 -- 172.21.15.152:0/679434992 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe374034610 con 0x7fe37c0d2b80 2024-04-20T11:27:15.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.788+0000 7fe382200640 1 -- 172.21.15.152:0/679434992 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe37c0d2b80 msgr2=0x7fe37c0d3010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:15.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.788+0000 7fe382200640 1 --2- 172.21.15.152:0/679434992 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe37c0d2b80 0x7fe37c0d3010 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe374009e50 tx=0x7fe37402bf60 comp rx=0 tx=0).stop 2024-04-20T11:27:15.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.788+0000 7fe382200640 1 -- 172.21.15.152:0/679434992 shutdown_connections 2024-04-20T11:27:15.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.788+0000 7fe382200640 1 --2- 172.21.15.152:0/679434992 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe37c0d2b80 0x7fe37c0d3010 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:15.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.788+0000 7fe382200640 1 --2- 172.21.15.152:0/679434992 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe37c0d4ec0 0x7fe37c0d0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:15.789 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.788+0000 7fe382200640 1 -- 172.21.15.152:0/679434992 >> 172.21.15.152:0/679434992 conn(0x7fe37c08bfd0 msgr2=0x7fe37c0d45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:15.789 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.788+0000 7fe382200640 1 -- 172.21.15.152:0/679434992 shutdown_connections 2024-04-20T11:27:15.789 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.788+0000 7fe382200640 1 -- 172.21.15.152:0/679434992 wait complete. 2024-04-20T11:27:15.789 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.790+0000 7fe382200640 1 Processor -- start 2024-04-20T11:27:15.789 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.790+0000 7fe382200640 1 -- start start 2024-04-20T11:27:15.792 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.791+0000 7fe382200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe37c0d2b80 0x7fe37c09faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:15.792 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.791+0000 7fe382200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe37c0d4ec0 0x7fe37c0ce3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:15.792 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.791+0000 7fe382200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe37c161d30 0x7fe37c0ccfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:15.792 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.791+0000 7fe382200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fe37c0cf1b0 con 0x7fe37c0d4ec0 2024-04-20T11:27:15.792 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.791+0000 7fe382200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe37c0cf800 con 0x7fe37c0d2b80 2024-04-20T11:27:15.792 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.791+0000 7fe382200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe37c162110 con 0x7fe37c161d30 2024-04-20T11:27:15.792 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.791+0000 7fe37be00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe37c0d4ec0 0x7fe37c0ce3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:15.792 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.791+0000 7fe37be00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe37c0d4ec0 0x7fe37c0ce3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:48516/0 (socket says 172.21.15.152:48516) 2024-04-20T11:27:15.793 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.791+0000 7fe37be00640 1 -- 172.21.15.152:0/3713173787 learned_addr learned my addr 172.21.15.152:0/3713173787 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:15.793 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.792+0000 7fe381800640 1 --2- 172.21.15.152:0/3713173787 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe37c161d30 0x7fe37c0ccfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:15.793 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.792+0000 7fe380e00640 1 --2- 172.21.15.152:0/3713173787 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe37c0d2b80 0x7fe37c09faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:15.793 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.792+0000 7fe37be00640 1 -- 172.21.15.152:0/3713173787 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe37c161d30 msgr2=0x7fe37c0ccfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:15.793 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.792+0000 7fe37be00640 1 --2- 172.21.15.152:0/3713173787 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe37c161d30 0x7fe37c0ccfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:15.793 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.792+0000 7fe37be00640 1 -- 172.21.15.152:0/3713173787 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe37c0d2b80 msgr2=0x7fe37c09faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:15.793 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.792+0000 7fe37be00640 1 --2- 172.21.15.152:0/3713173787 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe37c0d2b80 0x7fe37c09faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:15.793 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.792+0000 7fe37be00640 1 -- 172.21.15.152:0/3713173787 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe37c0cbf30 con 0x7fe37c0d4ec0 2024-04-20T11:27:15.793 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.792+0000 7fe37be00640 1 --2- 172.21.15.152:0/3713173787 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe37c0d4ec0 0x7fe37c0ce3d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe36800d800 tx=0x7fe36800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:15.793 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.792+0000 7fe378c00640 1 -- 172.21.15.152:0/3713173787 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe36800b6c0 con 0x7fe37c0d4ec0 2024-04-20T11:27:15.793 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.792+0000 7fe378c00640 1 -- 172.21.15.152:0/3713173787 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe368019070 con 0x7fe37c0d4ec0 2024-04-20T11:27:15.793 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.792+0000 7fe378c00640 1 -- 172.21.15.152:0/3713173787 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe36800baa0 con 0x7fe37c0d4ec0 2024-04-20T11:27:15.793 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.792+0000 7fe382200640 1 -- 172.21.15.152:0/3713173787 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe37c0cc210 con 0x7fe37c0d4ec0 2024-04-20T11:27:15.793 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.792+0000 7fe382200640 1 -- 172.21.15.152:0/3713173787 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe37c0cc7d0 con 0x7fe37c0d4ec0 2024-04-20T11:27:15.794 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.795+0000 7fe378c00640 1 -- 172.21.15.152:0/3713173787 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe36800bc40 con 0x7fe37c0d4ec0 2024-04-20T11:27:15.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.796+0000 7fe378c00640 1 --2- 172.21.15.152:0/3713173787 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe35c06c8f0 0x7fe35c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:15.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.796+0000 7fe378c00640 1 -- 172.21.15.152:0/3713173787 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fe36808c080 con 0x7fe37c0d4ec0 2024-04-20T11:27:15.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.797+0000 7fe380e00640 1 --2- 172.21.15.152:0/3713173787 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe35c06c8f0 0x7fe35c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:15.797 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.797+0000 7fe386593740 1 Processor -- start 2024-04-20T11:27:15.797 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.797+0000 7fe386593740 1 -- start start 2024-04-20T11:27:15.797 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.797+0000 7fe380e00640 1 --2- 172.21.15.152:0/3713173787 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe35c06c8f0 0x7fe35c06edb0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe36c0015a0 tx=0x7fe36c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:15.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.798+0000 7fe386593740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55af14a85670 0x55af14a3cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:15.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.798+0000 7fe386593740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55af14a3b6f0 0x55af14a3d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:15.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.798+0000 7fe386593740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55af14a37b90 0x55af14a67980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:15.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.798+0000 7fe386593740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55af149a39d0 con 0x55af14a37b90 2024-04-20T11:27:15.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.798+0000 7fe386593740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55af1494a270 con 0x55af14a85670 2024-04-20T11:27:15.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.798+0000 7fe386593740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x55af1499f990 con 0x55af14a3b6f0 2024-04-20T11:27:15.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.798+0000 7fe381800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55af14a37b90 0x55af14a67980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:15.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.798+0000 7fe381800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55af14a37b90 0x55af14a67980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:57950/0 (socket says 172.21.15.152:57950) 2024-04-20T11:27:15.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.798+0000 7fe381800640 1 -- 172.21.15.152:0/4250016573 learned_addr learned my addr 172.21.15.152:0/4250016573 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:15.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.799+0000 7fe380e00640 1 --2- 172.21.15.152:0/4250016573 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55af14a85670 0x55af14a3cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:15.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.799+0000 7fe381800640 1 -- 172.21.15.152:0/4250016573 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55af14a3b6f0 msgr2=0x55af14a3d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:15.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.799+0000 7fe37be00640 1 --2- 172.21.15.152:0/4250016573 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55af14a3b6f0 0x55af14a3d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:15.800 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.799+0000 7fe381800640 1 --2- 172.21.15.152:0/4250016573 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55af14a3b6f0 0x55af14a3d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:15.800 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.799+0000 7fe381800640 1 -- 172.21.15.152:0/4250016573 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55af14a85670 msgr2=0x55af14a3cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:15.800 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.799+0000 7fe381800640 1 --2- 172.21.15.152:0/4250016573 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55af14a85670 0x55af14a3cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:15.800 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.799+0000 7fe381800640 1 -- 172.21.15.152:0/4250016573 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af146b62d0 con 0x55af14a37b90 2024-04-20T11:27:15.800 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.799+0000 7fe381800640 1 --2- 172.21.15.152:0/4250016573 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55af14a37b90 0x55af14a67980 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe374009f70 tx=0x7fe37402e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=453bff66370a5319 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:15.800 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.799+0000 7fe371600640 1 -- 172.21.15.152:0/4250016573 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe374045070 con 0x55af14a37b90 2024-04-20T11:27:15.800 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.799+0000 7fe371600640 1 -- 172.21.15.152:0/4250016573 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe37402ce50 con 0x55af14a37b90 2024-04-20T11:27:15.800 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.799+0000 7fe371600640 1 -- 172.21.15.152:0/4250016573 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe374009510 con 0x55af14a37b90 2024-04-20T11:27:15.801 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.800+0000 7fe386593740 1 -- 172.21.15.152:0/4250016573 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55af14a37b90 msgr2=0x55af14a67980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:15.801 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.800+0000 7fe386593740 1 --2- 172.21.15.152:0/4250016573 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55af14a37b90 0x55af14a67980 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe374009f70 tx=0x7fe37402e000 comp rx=0 tx=0).stop 2024-04-20T11:27:15.801 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.800+0000 7fe386593740 1 -- 172.21.15.152:0/4250016573 shutdown_connections 2024-04-20T11:27:15.801 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.800+0000 7fe386593740 1 --2- 172.21.15.152:0/4250016573 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55af14a37b90 0x55af14a67980 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:15.801 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.800+0000 7fe386593740 1 --2- 172.21.15.152:0/4250016573 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55af14a3b6f0 0x55af14a3d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:15.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.800+0000 7fe386593740 1 --2- 172.21.15.152:0/4250016573 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55af14a85670 0x55af14a3cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:15.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.800+0000 7fe386593740 1 -- 172.21.15.152:0/4250016573 >> 172.21.15.152:0/4250016573 conn(0x55af14a39e00 msgr2=0x55af14a82980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:15.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.800+0000 7fe386593740 1 -- 172.21.15.152:0/4250016573 shutdown_connections 2024-04-20T11:27:15.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.800+0000 7fe386593740 1 -- 172.21.15.152:0/4250016573 wait complete. 2024-04-20T11:27:15.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.801+0000 7fe386593740 1 -- start start 2024-04-20T11:27:15.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.801+0000 7fe386593740 1 Processor -- start 2024-04-20T11:27:15.805 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.802+0000 7fe386593740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55af14a3b6f0 0x55af14a5d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:15.805 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.802+0000 7fe386593740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55af14a85670 0x55af14a89fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:15.805 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.802+0000 7fe386593740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55af14a887d0 0x55af14a36b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:15.805 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.802+0000 7fe386593740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55af14a3bc00 con 0x55af14a3b6f0 2024-04-20T11:27:15.805 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.802+0000 7fe386593740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55af1495d5e0 con 0x55af14a85670 2024-04-20T11:27:15.805 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.802+0000 7fe386593740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x55af14a3bee0 con 0x55af14a887d0 2024-04-20T11:27:15.805 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.803+0000 7fe386593740 10 client.0 _ll_register_callbacks cb 0x55af146b5ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:27:15.805 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.803+0000 7fe380e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55af14a3b6f0 0x55af14a5d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:15.805 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.803+0000 7fe380e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55af14a3b6f0 0x55af14a5d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:57954/0 (socket says 172.21.15.152:57954) 2024-04-20T11:27:15.805 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.803+0000 7fe380e00640 1 -- 172.21.15.152:0/3525773287 learned_addr learned my addr 172.21.15.152:0/3525773287 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:15.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.803+0000 7fe381800640 1 --2- 172.21.15.152:0/3525773287 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55af14a887d0 0x55af14a36b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:15.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.803+0000 7fe37be00640 1 --2- 172.21.15.152:0/3525773287 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55af14a85670 0x55af14a89fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:15.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.803+0000 7fe380e00640 1 -- 172.21.15.152:0/3525773287 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55af14a887d0 msgr2=0x55af14a36b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:15.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.803+0000 7fe380e00640 1 --2- 172.21.15.152:0/3525773287 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55af14a887d0 0x55af14a36b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:15.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.803+0000 7fe380e00640 1 -- 172.21.15.152:0/3525773287 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55af14a85670 msgr2=0x55af14a89fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:15.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.803+0000 7fe380e00640 1 --2- 172.21.15.152:0/3525773287 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55af14a85670 0x55af14a89fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:15.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.803+0000 7fe380e00640 1 -- 172.21.15.152:0/3525773287 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af147806b0 con 0x55af14a3b6f0 2024-04-20T11:27:15.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.803+0000 7fe37be00640 1 --2- 172.21.15.152:0/3525773287 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55af14a85670 0x55af14a89fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:27:15.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.803+0000 7fe380e00640 1 --2- 172.21.15.152:0/3525773287 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55af14a3b6f0 0x55af14a5d890 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe36c00dce0 tx=0x7fe36c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21a2dc908ff25912 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:15.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.803+0000 7fe366000640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-04-20T11:27:15.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.804+0000 7fe366000640 1 -- 172.21.15.152:0/3525773287 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe36c020420 con 0x55af14a3b6f0 2024-04-20T11:27:15.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.804+0000 7fe366000640 1 -- 172.21.15.152:0/3525773287 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fe36c0205c0 con 0x55af14a3b6f0 2024-04-20T11:27:15.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.804+0000 7fe386593740 1 -- 172.21.15.152:0/3525773287 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe37402ce50 con 0x55af14a3b6f0 2024-04-20T11:27:15.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.804+0000 7fe366000640 1 -- 172.21.15.152:0/3525773287 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7fe36c02d050 con 0x55af14a3b6f0 2024-04-20T11:27:15.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.804+0000 7fe386593740 10 client.4635 fetch_fsmap learned FSMap version 7 2024-04-20T11:27:15.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.804+0000 7fe386593740 1 -- 172.21.15.152:0/3525773287 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x55af14a87fd0 con 0x55af14a3b6f0 2024-04-20T11:27:15.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.804+0000 7fe366000640 1 -- 172.21.15.152:0/3525773287 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe37402ce50 con 0x55af14a3b6f0 2024-04-20T11:27:15.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.804+0000 7fe366000640 1 -- 172.21.15.152:0/3525773287 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 7) ==== 3495+0+0 (secure 0 0 0) 0x7fe36c030020 con 0x55af14a3b6f0 2024-04-20T11:27:15.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.804+0000 7fe386593740 10 client.4635 fetch_fsmap finished waiting for FSMap version 7 2024-04-20T11:27:15.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.804+0000 7fe386593740 10 client.4635 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-20T11:27:15.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.804+0000 7fe386593740 20 client.4635 populate_metadata read hostname 'smithi152' 2024-04-20T11:27:15.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.805+0000 7fe386593740 1 --2- 172.21.15.152:0/3525773287 >> [v2:172.21.15.152:6836/927639144,v1:172.21.15.152:6837/927639144] conn(0x55af14a66670 0x55af14a64d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:15.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.805+0000 7fe386593740 4 client.4635 mds_command: new command op to 4269 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:27:15.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.805+0000 7fe386593740 1 -- 172.21.15.152:0/3525773287 --> [v2:172.21.15.152:6836/927639144,v1:172.21.15.152:6837/927639144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55af149b13f0 con 0x55af14a66670 2024-04-20T11:27:15.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.805+0000 7fe37be00640 1 --2- 172.21.15.152:0/3525773287 >> [v2:172.21.15.152:6836/927639144,v1:172.21.15.152:6837/927639144] conn(0x55af14a66670 0x55af14a64d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:15.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.805+0000 7fe37be00640 1 --2- 172.21.15.152:0/3525773287 >> [v2:172.21.15.152:6836/927639144,v1:172.21.15.152:6837/927639144] conn(0x55af14a66670 0x55af14a64d90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a34d32ba35488c76 server_cookie=d0a4e71ab1f0c3aa in_seq=0 out_seq=0 2024-04-20T11:27:15.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.805+0000 7fe366000640 10 client.4635 ms_handle_connect on v2:172.21.15.152:6836/927639144 2024-04-20T11:27:15.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.806+0000 7fe366000640 1 -- 172.21.15.152:0/3525773287 <== mds.0 v2:172.21.15.152:6836/927639144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55af149b13f0 con 0x55af14a66670 2024-04-20T11:27:15.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.806+0000 7fe366000640 10 client.4635 handle_command_reply: tid=0 2024-04-20T11:27:15.809 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.806+0000 7fe386593740 1 client.4635 shutdown 2024-04-20T11:27:15.809 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.807+0000 7fe386593740 1 -- 172.21.15.152:0/3525773287 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55af14a3b6f0 msgr2=0x55af14a5d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:15.809 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.807+0000 7fe386593740 1 --2- 172.21.15.152:0/3525773287 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55af14a3b6f0 0x55af14a5d890 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe36c00dce0 tx=0x7fe36c000bc0 comp rx=0 tx=0).stop 2024-04-20T11:27:15.809 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.807+0000 7fe386593740 1 -- 172.21.15.152:0/3525773287 shutdown_connections 2024-04-20T11:27:15.809 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.807+0000 7fe386593740 1 --2- 172.21.15.152:0/3525773287 >> [v2:172.21.15.152:6836/927639144,v1:172.21.15.152:6837/927639144] conn(0x55af14a66670 0x55af14a64d90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:15.809 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.807+0000 7fe386593740 1 --2- 172.21.15.152:0/3525773287 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55af14a887d0 0x55af14a36b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:15.809 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.807+0000 7fe386593740 1 --2- 172.21.15.152:0/3525773287 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55af14a85670 0x55af14a89fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:15.809 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.807+0000 7fe386593740 1 --2- 172.21.15.152:0/3525773287 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55af14a3b6f0 0x55af14a5d890 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:15.809 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.807+0000 7fe366000640 0 client.4635 ms_handle_reset on v2:172.21.15.152:6836/927639144 2024-04-20T11:27:15.809 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.807+0000 7fe386593740 1 -- 172.21.15.152:0/3525773287 >> 172.21.15.152:0/3525773287 conn(0x55af14a37b90 msgr2=0x55af14a3ca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:15.809 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.807+0000 7fe386593740 1 -- 172.21.15.152:0/3525773287 shutdown_connections 2024-04-20T11:27:15.809 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.807+0000 7fe386593740 1 -- 172.21.15.152:0/3525773287 wait complete. 2024-04-20T11:27:15.809 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.807+0000 7fe386593740 20 client.4635 trim_cache size 0 max 16384 2024-04-20T11:27:15.826 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.827+0000 7fe386593740 1 Processor -- start 2024-04-20T11:27:15.826 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.827+0000 7fe386593740 1 -- start start 2024-04-20T11:27:15.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.827+0000 7fe386593740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55af14a3b6f0 0x55af14a5cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:15.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.827+0000 7fe386593740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55af14a85670 0x55af14a5fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:15.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.827+0000 7fe386593740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55af14a887d0 0x55af14a603b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:15.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.828+0000 7fe386593740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fe37c1ffbe0 con 0x55af14a887d0 2024-04-20T11:27:15.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.828+0000 7fe386593740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe37c0b8e60 con 0x55af14a3b6f0 2024-04-20T11:27:15.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.828+0000 7fe386593740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe37c1ff590 con 0x55af14a85670 2024-04-20T11:27:15.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.828+0000 7fe381800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55af14a887d0 0x55af14a603b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:15.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.828+0000 7fe381800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55af14a887d0 0x55af14a603b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:57966/0 (socket says 172.21.15.152:57966) 2024-04-20T11:27:15.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.828+0000 7fe381800640 1 -- 172.21.15.152:0/903212732 learned_addr learned my addr 172.21.15.152:0/903212732 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:15.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.828+0000 7fe380e00640 1 --2- 172.21.15.152:0/903212732 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55af14a3b6f0 0x55af14a5cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:15.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.828+0000 7fe381800640 1 -- 172.21.15.152:0/903212732 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55af14a85670 msgr2=0x55af14a5fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:15.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.828+0000 7fe37be00640 1 --2- 172.21.15.152:0/903212732 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55af14a85670 0x55af14a5fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:15.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.828+0000 7fe381800640 1 --2- 172.21.15.152:0/903212732 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55af14a85670 0x55af14a5fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:15.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.828+0000 7fe381800640 1 -- 172.21.15.152:0/903212732 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55af14a3b6f0 msgr2=0x55af14a5cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:15.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.828+0000 7fe381800640 1 --2- 172.21.15.152:0/903212732 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55af14a3b6f0 0x55af14a5cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:15.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.828+0000 7fe381800640 1 -- 172.21.15.152:0/903212732 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af14a36ed0 con 0x55af14a887d0 2024-04-20T11:27:15.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.828+0000 7fe380e00640 1 --2- 172.21.15.152:0/903212732 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55af14a3b6f0 0x55af14a5cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:27:15.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.829+0000 7fe381800640 1 --2- 172.21.15.152:0/903212732 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55af14a887d0 0x55af14a603b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe374009e50 tx=0x7fe374009000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a48a4383d086847a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:15.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.829+0000 7fe370c00640 1 -- 172.21.15.152:0/903212732 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe374045040 con 0x55af14a887d0 2024-04-20T11:27:15.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.829+0000 7fe370c00640 1 -- 172.21.15.152:0/903212732 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe374003bb0 con 0x55af14a887d0 2024-04-20T11:27:15.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.829+0000 7fe370c00640 1 -- 172.21.15.152:0/903212732 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe37402c900 con 0x55af14a887d0 2024-04-20T11:27:15.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.829+0000 7fe386593740 1 -- 172.21.15.152:0/903212732 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55af14a887d0 msgr2=0x55af14a603b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:15.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.829+0000 7fe386593740 1 --2- 172.21.15.152:0/903212732 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55af14a887d0 0x55af14a603b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe374009e50 tx=0x7fe374009000 comp rx=0 tx=0).stop 2024-04-20T11:27:15.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.830+0000 7fe386593740 1 -- 172.21.15.152:0/903212732 shutdown_connections 2024-04-20T11:27:15.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.830+0000 7fe386593740 1 --2- 172.21.15.152:0/903212732 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55af14a887d0 0x55af14a603b0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:15.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.830+0000 7fe386593740 1 --2- 172.21.15.152:0/903212732 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55af14a85670 0x55af14a5fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:15.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.830+0000 7fe386593740 1 --2- 172.21.15.152:0/903212732 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55af14a3b6f0 0x55af14a5cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:15.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.830+0000 7fe386593740 1 -- 172.21.15.152:0/903212732 >> 172.21.15.152:0/903212732 conn(0x55af14a37b90 msgr2=0x55af14a6b340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:15.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.830+0000 7fe386593740 1 -- 172.21.15.152:0/903212732 shutdown_connections 2024-04-20T11:27:15.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.830+0000 7fe386593740 1 -- 172.21.15.152:0/903212732 wait complete. 2024-04-20T11:27:15.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.830+0000 7fe386593740 1 -- start start 2024-04-20T11:27:15.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.831+0000 7fe386593740 1 Processor -- start 2024-04-20T11:27:15.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.831+0000 7fe386593740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55af14a3b6f0 0x55af14afcf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:15.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.832+0000 7fe386593740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55af14a85670 0x55af14aa5b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:15.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.832+0000 7fe386593740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55af14a887d0 0x55af14a5b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:15.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.832+0000 7fe386593740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55af14a664f0 con 0x55af14a887d0 2024-04-20T11:27:15.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.832+0000 7fe386593740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55af14a66f20 con 0x55af14a3b6f0 2024-04-20T11:27:15.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.832+0000 7fe386593740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x55af14a66ae0 con 0x55af14a85670 2024-04-20T11:27:15.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.832+0000 7fe386593740 10 client.0 _ll_register_callbacks cb 0x7fe36c00d4a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:27:15.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.832+0000 7fe381800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55af14a887d0 0x55af14a5b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:15.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.832+0000 7fe381800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55af14a887d0 0x55af14a5b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:57970/0 (socket says 172.21.15.152:57970) 2024-04-20T11:27:15.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.832+0000 7fe381800640 1 -- 172.21.15.152:0/3426287800 learned_addr learned my addr 172.21.15.152:0/3426287800 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:15.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.832+0000 7fe380e00640 1 --2- 172.21.15.152:0/3426287800 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55af14a3b6f0 0x55af14afcf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:15.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.832+0000 7fe381800640 1 -- 172.21.15.152:0/3426287800 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55af14a85670 msgr2=0x55af14aa5b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:15.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.832+0000 7fe37be00640 1 --2- 172.21.15.152:0/3426287800 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55af14a85670 0x55af14aa5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:15.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.832+0000 7fe381800640 1 --2- 172.21.15.152:0/3426287800 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55af14a85670 0x55af14aa5b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:15.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.832+0000 7fe381800640 1 -- 172.21.15.152:0/3426287800 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55af14a3b6f0 msgr2=0x55af14afcf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:15.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.832+0000 7fe381800640 1 --2- 172.21.15.152:0/3426287800 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55af14a3b6f0 0x55af14afcf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:15.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.832+0000 7fe381800640 1 -- 172.21.15.152:0/3426287800 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af14a95920 con 0x55af14a887d0 2024-04-20T11:27:15.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.832+0000 7fe380e00640 1 --2- 172.21.15.152:0/3426287800 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55af14a3b6f0 0x55af14afcf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:27:15.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.833+0000 7fe381800640 1 --2- 172.21.15.152:0/3426287800 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55af14a887d0 0x55af14a5b4b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe37402c570 tx=0x7fe374004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fbdcf368de7cf95 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:15.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.833+0000 7fe366000640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-04-20T11:27:15.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.833+0000 7fe366000640 1 -- 172.21.15.152:0/3426287800 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe374045070 con 0x55af14a887d0 2024-04-20T11:27:15.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.833+0000 7fe386593740 1 -- 172.21.15.152:0/3426287800 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe374003bb0 con 0x55af14a887d0 2024-04-20T11:27:15.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.833+0000 7fe366000640 1 -- 172.21.15.152:0/3426287800 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fe374004030 con 0x55af14a887d0 2024-04-20T11:27:15.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.833+0000 7fe366000640 1 -- 172.21.15.152:0/3426287800 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7fe374056050 con 0x55af14a887d0 2024-04-20T11:27:15.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.833+0000 7fe386593740 10 client.4641 fetch_fsmap learned FSMap version 7 2024-04-20T11:27:15.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.833+0000 7fe386593740 1 -- 172.21.15.152:0/3426287800 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x55af14a3cb30 con 0x55af14a887d0 2024-04-20T11:27:15.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.833+0000 7fe366000640 1 -- 172.21.15.152:0/3426287800 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe374003bb0 con 0x55af14a887d0 2024-04-20T11:27:15.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.834+0000 7fe366000640 1 -- 172.21.15.152:0/3426287800 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 7) ==== 3495+0+0 (secure 0 0 0) 0x7fe374059020 con 0x55af14a887d0 2024-04-20T11:27:15.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.834+0000 7fe386593740 10 client.4641 fetch_fsmap finished waiting for FSMap version 7 2024-04-20T11:27:15.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.834+0000 7fe386593740 10 client.4641 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-20T11:27:15.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.834+0000 7fe386593740 20 client.4641 populate_metadata read hostname 'smithi152' 2024-04-20T11:27:15.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.834+0000 7fe386593740 1 --2- 172.21.15.152:0/3426287800 >> [v2:172.21.15.152:6836/927639144,v1:172.21.15.152:6837/927639144] conn(0x55af14a5fd30 0x55af14a60110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:15.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.834+0000 7fe386593740 4 client.4641 mds_command: new command op to 4269 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-20T11:27:15.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.834+0000 7fe386593740 1 -- 172.21.15.152:0/3426287800 --> [v2:172.21.15.152:6836/927639144,v1:172.21.15.152:6837/927639144] -- command(tid 0: {"prefix": "status"}) -- 0x55af149ab260 con 0x55af14a5fd30 2024-04-20T11:27:15.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.834+0000 7fe380e00640 1 --2- 172.21.15.152:0/3426287800 >> [v2:172.21.15.152:6836/927639144,v1:172.21.15.152:6837/927639144] conn(0x55af14a5fd30 0x55af14a60110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:15.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.834+0000 7fe380e00640 1 --2- 172.21.15.152:0/3426287800 >> [v2:172.21.15.152:6836/927639144,v1:172.21.15.152:6837/927639144] conn(0x55af14a5fd30 0x55af14a60110 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44aba303ae09368c server_cookie=4c3ca68b92c2db9f in_seq=0 out_seq=0 2024-04-20T11:27:15.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.834+0000 7fe366000640 10 client.4641 ms_handle_connect on v2:172.21.15.152:6836/927639144 2024-04-20T11:27:15.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.835+0000 7fe366000640 1 -- 172.21.15.152:0/3426287800 <== mds.0 v2:172.21.15.152:6836/927639144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x55af149ab260 con 0x55af14a5fd30 2024-04-20T11:27:15.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.835+0000 7fe366000640 10 client.4641 handle_command_reply: tid=0 2024-04-20T11:27:15.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.835+0000 7fe386593740 1 client.4641 shutdown 2024-04-20T11:27:15.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.835+0000 7fe386593740 1 -- 172.21.15.152:0/3426287800 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55af14a887d0 msgr2=0x55af14a5b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:15.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.835+0000 7fe386593740 1 --2- 172.21.15.152:0/3426287800 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55af14a887d0 0x55af14a5b4b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe37402c570 tx=0x7fe374004ab0 comp rx=0 tx=0).stop 2024-04-20T11:27:15.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.835+0000 7fe386593740 1 -- 172.21.15.152:0/3426287800 shutdown_connections 2024-04-20T11:27:15.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.835+0000 7fe386593740 1 --2- 172.21.15.152:0/3426287800 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55af14a887d0 0x55af14a5b4b0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:15.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.835+0000 7fe386593740 1 --2- 172.21.15.152:0/3426287800 >> [v2:172.21.15.152:6836/927639144,v1:172.21.15.152:6837/927639144] conn(0x55af14a5fd30 0x55af14a60110 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:15.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.835+0000 7fe386593740 1 --2- 172.21.15.152:0/3426287800 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55af14a85670 0x55af14aa5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:15.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.835+0000 7fe386593740 1 --2- 172.21.15.152:0/3426287800 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55af14a3b6f0 0x55af14afcf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:15.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.835+0000 7fe366000640 0 client.4641 ms_handle_reset on v2:172.21.15.152:6836/927639144 2024-04-20T11:27:15.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.835+0000 7fe386593740 1 -- 172.21.15.152:0/3426287800 >> 172.21.15.152:0/3426287800 conn(0x55af14a37b90 msgr2=0x55af14a3d7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:15.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.835+0000 7fe386593740 1 -- 172.21.15.152:0/3426287800 shutdown_connections 2024-04-20T11:27:15.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.835+0000 7fe386593740 1 -- 172.21.15.152:0/3426287800 wait complete. 2024-04-20T11:27:15.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.835+0000 7fe386593740 20 client.4641 trim_cache size 0 max 16384 2024-04-20T11:27:15.839 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-04-20T11:27:15.839 INFO:teuthology.orchestra.run.smithi152.stdout: "cluster_fsid": "c8a72131-0d4f-42c7-892f-3f3af4d2bd3d", 2024-04-20T11:27:15.839 INFO:teuthology.orchestra.run.smithi152.stdout: "whoami": 0, 2024-04-20T11:27:15.839 INFO:teuthology.orchestra.run.smithi152.stdout: "id": 4269, 2024-04-20T11:27:15.839 INFO:teuthology.orchestra.run.smithi152.stdout: "want_state": "up:active", 2024-04-20T11:27:15.839 INFO:teuthology.orchestra.run.smithi152.stdout: "state": "up:active", 2024-04-20T11:27:15.839 INFO:teuthology.orchestra.run.smithi152.stdout: "fs_name": "cephfs", 2024-04-20T11:27:15.840 INFO:teuthology.orchestra.run.smithi152.stdout: "rank_uptime": 17.178558322000001, 2024-04-20T11:27:15.840 INFO:teuthology.orchestra.run.smithi152.stdout: "mdsmap_epoch": 7, 2024-04-20T11:27:15.840 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_epoch": 20, 2024-04-20T11:27:15.840 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_epoch_barrier": 20, 2024-04-20T11:27:15.840 INFO:teuthology.orchestra.run.smithi152.stdout: "uptime": 22.476220799 2024-04-20T11:27:15.840 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-04-20T11:27:15.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.836+0000 7fe382200640 1 -- 172.21.15.152:0/3713173787 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe35c06c8f0 msgr2=0x7fe35c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:15.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.836+0000 7fe382200640 1 --2- 172.21.15.152:0/3713173787 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe35c06c8f0 0x7fe35c06edb0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe36c0015a0 tx=0x7fe36c009040 comp rx=0 tx=0).stop 2024-04-20T11:27:15.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.836+0000 7fe382200640 1 -- 172.21.15.152:0/3713173787 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe37c0d4ec0 msgr2=0x7fe37c0ce3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:15.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.836+0000 7fe382200640 1 --2- 172.21.15.152:0/3713173787 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe37c0d4ec0 0x7fe37c0ce3d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe36800d800 tx=0x7fe36800dcc0 comp rx=0 tx=0).stop 2024-04-20T11:27:15.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.836+0000 7fe382200640 1 -- 172.21.15.152:0/3713173787 shutdown_connections 2024-04-20T11:27:15.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.836+0000 7fe382200640 1 --2- 172.21.15.152:0/3713173787 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe35c06c8f0 0x7fe35c06edb0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:15.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.836+0000 7fe382200640 1 --2- 172.21.15.152:0/3713173787 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe37c161d30 0x7fe37c0ccfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:15.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.836+0000 7fe382200640 1 --2- 172.21.15.152:0/3713173787 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe37c0d4ec0 0x7fe37c0ce3d0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:15.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.836+0000 7fe382200640 1 --2- 172.21.15.152:0/3713173787 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe37c0d2b80 0x7fe37c09faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:15.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.836+0000 7fe382200640 1 -- 172.21.15.152:0/3713173787 >> 172.21.15.152:0/3713173787 conn(0x7fe37c08bfd0 msgr2=0x7fe37c0b4d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:15.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.836+0000 7fe382200640 1 -- 172.21.15.152:0/3713173787 shutdown_connections 2024-04-20T11:27:15.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:15.836+0000 7fe382200640 1 -- 172.21.15.152:0/3713173787 wait complete. 2024-04-20T11:27:15.852 INFO:teuthology.run_tasks:Running task sequential... 2024-04-20T11:27:15.862 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-20T11:27:15.862 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-20T11:27:15.862 INFO:teuthology.task.print:Enabling mgr modules 2024-04-20T11:27:15.862 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-04-20T11:27:15.883 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-04-20T11:27:15.884 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-04-20T11:27:15.884 INFO:tasks.ceph_fuse:client.0 config is {} 2024-04-20T11:27:15.884 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-20T11:27:15.884 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-20T11:27:15.884 INFO:tasks.ceph_fuse:client.2 config is {} 2024-04-20T11:27:15.884 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-20T11:27:15.884 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-20T11:27:15.884 INFO:tasks.ceph_fuse:client.1 config is {} 2024-04-20T11:27:15.884 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-20T11:27:15.884 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-20T11:27:15.885 INFO:tasks.ceph_fuse:client.3 config is {} 2024-04-20T11:27:15.885 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-20T11:27:15.885 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-20T11:27:15.885 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:27:15.885 DEBUG:teuthology.orchestra.run.smithi152:> ip netns list 2024-04-20T11:27:15.937 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:27:15.937 DEBUG:teuthology.orchestra.run.smithi152:> sudo ip link delete ceph-brx 2024-04-20T11:27:16.023 INFO:teuthology.orchestra.run.smithi152.stderr:Cannot find device "ceph-brx" 2024-04-20T11:27:16.027 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T11:27:16.027 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:27:16.028 DEBUG:teuthology.orchestra.run.smithi175:> ip netns list 2024-04-20T11:27:16.086 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:27:16.087 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip link delete ceph-brx 2024-04-20T11:27:16.167 INFO:teuthology.orchestra.run.smithi175.stderr:Cannot find device "ceph-brx" 2024-04-20T11:27:16.170 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T11:27:16.171 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-04-20T11:27:16.171 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-20T11:27:16.377 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:16.377+0000 7f8794c00640 1 Processor -- start 2024-04-20T11:27:16.377 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:16.377+0000 7f8794c00640 1 -- start start 2024-04-20T11:27:16.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:16.379+0000 7f8794c00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f879013e530 0x7f879006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:16.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:16.379+0000 7f8794c00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87900cf0a0 con 0x7f87901414e0 2024-04-20T11:27:16.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:16.379+0000 7f8794c00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87900d00e0 con 0x7f879006f340 2024-04-20T11:27:16.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:16.379+0000 7f8794c00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f8790141b00 con 0x7f879013e530 2024-04-20T11:27:16.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:16.379+0000 7f878fe00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f87901414e0 0x7f8790071cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:40572/0 (socket says 172.21.15.152:40572) 2024-04-20T11:27:16.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:16.379+0000 7f878fe00640 1 -- 172.21.15.152:0/2841528585 learned_addr learned my addr 172.21.15.152:0/2841528585 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:27:16.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:16.380+0000 7f878f400640 1 --2- 172.21.15.152:0/2841528585 >> v2:172.21.15.175:3300/0 conn(0x7f879013e530 0x7f879006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:16.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:16.380+0000 7f878ea00640 1 -- 172.21.15.152:0/2841528585 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 71270224 0 0) 0x7f8788001490 con 0x7f87901414e0 2024-04-20T11:27:16.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:16.380+0000 7f878ea00640 1 -- 172.21.15.152:0/2841528585 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8774003520 con 0x7f87901414e0 2024-04-20T11:27:16.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:16.380+0000 7f878ea00640 1 -- 172.21.15.152:0/2841528585 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1955794072 0 0) 0x7f8774003520 con 0x7f87901414e0 2024-04-20T11:27:16.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:16.380+0000 7f878ea00640 1 -- 172.21.15.152:0/2841528585 >> v2:172.21.15.175:3300/0 conn(0x7f879013e530 msgr2=0x7f879006ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:16.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:16.380+0000 7f878ea00640 1 --2- 172.21.15.152:0/2841528585 >> v2:172.21.15.175:3300/0 conn(0x7f879013e530 0x7f879006ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:16.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:16.380+0000 7f878ea00640 1 -- 172.21.15.152:0/2841528585 >> v1:172.21.15.175:6789/0 conn(0x7f879006f340 legacy=0x7f8790140dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:16.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:16.380+0000 7f878ea00640 1 -- 172.21.15.152:0/2841528585 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87900cf320 con 0x7f87901414e0 2024-04-20T11:27:16.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:16.380+0000 7f878ea00640 1 -- 172.21.15.152:0/2841528585 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f8788001bc0 con 0x7f87901414e0 2024-04-20T11:27:16.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:16.380+0000 7f878ea00640 1 -- 172.21.15.152:0/2841528585 >> v1:172.21.15.152:6789/0 conn(0x7f87901414e0 legacy=0x7f8790071cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:16.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:16.380+0000 7f878ea00640 1 --2- 172.21.15.152:0/2841528585 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f87740039b0 0x7f8774003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:16.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:16.380+0000 7f878ea00640 1 --2- 172.21.15.152:0/2841528585 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8774004800 0x7f8774004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:16.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:16.381+0000 7f878ea00640 1 --2- 172.21.15.152:0/2841528585 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87740051c0 0x7f877400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:16.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:16.381+0000 7f878ea00640 1 -- 172.21.15.152:0/2841528585 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f877400c030 con 0x7f87740051c0 2024-04-20T11:27:16.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:16.381+0000 7f878ea00640 1 -- 172.21.15.152:0/2841528585 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f877400e3f0 con 0x7f8774004800 2024-04-20T11:27:16.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:16.381+0000 7f878ea00640 1 -- 172.21.15.152:0/2841528585 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f877400e5f0 con 0x7f87740039b0 2024-04-20T11:27:16.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:16.381+0000 7f878fe00640 1 --2- 172.21.15.152:0/2841528585 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87740051c0 0x7f877400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:16.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:16.381+0000 7f8786a00640 1 --2- 172.21.15.152:0/2841528585 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8774004800 0x7f8774004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:16.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:16.381+0000 7f878f400640 1 --2- 172.21.15.152:0/2841528585 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f87740039b0 0x7f8774003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:16.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:16.381+0000 7f878fe00640 1 -- 172.21.15.152:0/2841528585 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f87740039b0 msgr2=0x7f8774003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:16.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:16.381+0000 7f878fe00640 1 --2- 172.21.15.152:0/2841528585 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f87740039b0 0x7f8774003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:16.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:16.381+0000 7f878fe00640 1 -- 172.21.15.152:0/2841528585 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8774004800 msgr2=0x7f8774004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:16.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:16.381+0000 7f878fe00640 1 --2- 172.21.15.152:0/2841528585 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8774004800 0x7f8774004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:16.383 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:16.381+0000 7f878fe00640 1 -- 172.21.15.152:0/2841528585 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8774003520 con 0x7f87740051c0 2024-04-20T11:27:16.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:16.382+0000 7f878fe00640 1 -- 172.21.15.152:0/2841528585 reap_dead start 2024-04-20T11:27:16.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:16.382+0000 7f878fe00640 1 --2- 172.21.15.152:0/2841528585 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87740051c0 0x7f877400b9c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f878800aa20 tx=0x7f8774005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55fd03fe542fc1de server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:16.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:16.383+0000 7f878ea00640 1 -- 172.21.15.152:0/2841528585 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8788038070 con 0x7f87740051c0 2024-04-20T11:27:16.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:16.383+0000 7f878ea00640 1 -- 172.21.15.152:0/2841528585 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8788033620 con 0x7f87740051c0 2024-04-20T11:27:16.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:16.383+0000 7f878ea00640 1 -- 172.21.15.152:0/2841528585 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f878800a640 con 0x7f87740051c0 2024-04-20T11:27:16.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:16.383+0000 7f8794c00640 1 -- 172.21.15.152:0/2841528585 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87740051c0 msgr2=0x7f877400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:16.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:16.383+0000 7f8794c00640 1 --2- 172.21.15.152:0/2841528585 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87740051c0 0x7f877400b9c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f878800aa20 tx=0x7f8774005830 comp rx=0 tx=0).stop 2024-04-20T11:27:16.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:16.383+0000 7f8794c00640 1 -- 172.21.15.152:0/2841528585 shutdown_connections 2024-04-20T11:27:16.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:16.383+0000 7f8794c00640 1 --2- 172.21.15.152:0/2841528585 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87740051c0 0x7f877400b9c0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:16.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:16.383+0000 7f8794c00640 1 -- 172.21.15.152:0/2841528585 >> 172.21.15.152:0/2841528585 conn(0x7f8790067ae0 msgr2=0x7f879006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:16.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:16.383+0000 7f8794c00640 1 -- 172.21.15.152:0/2841528585 shutdown_connections 2024-04-20T11:27:16.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:16.383+0000 7f8794c00640 1 -- 172.21.15.152:0/2841528585 wait complete. 2024-04-20T11:27:16.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:16.385+0000 7f8794c00640 1 Processor -- start 2024-04-20T11:27:16.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:16.385+0000 7f8794c00640 1 -- start start 2024-04-20T11:27:16.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:16.386+0000 7f8794c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87740051c0 0x7f879015db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:16.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:16.386+0000 7f8794c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87900b09e0 0x7f87901597e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:16.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:16.386+0000 7f8794c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f879013d770 0x7f879013dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:16.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:16.386+0000 7f8794c00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f87900cf8e0 con 0x7f87900b09e0 2024-04-20T11:27:16.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:16.386+0000 7f8794c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f87900cfc50 con 0x7f87740051c0 2024-04-20T11:27:16.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:16.386+0000 7f8794c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8790140dd0 con 0x7f879013d770 2024-04-20T11:27:16.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:16.386+0000 7f8786a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87900b09e0 0x7f87901597e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:16.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:16.387+0000 7f8786a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87900b09e0 0x7f87901597e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:57998/0 (socket says 172.21.15.152:57998) 2024-04-20T11:27:16.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:16.387+0000 7f8786a00640 1 -- 172.21.15.152:0/572630188 learned_addr learned my addr 172.21.15.152:0/572630188 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:16.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:16.387+0000 7f878fe00640 1 --2- 172.21.15.152:0/572630188 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f879013d770 0x7f879013dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:16.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:16.387+0000 7f878f400640 1 --2- 172.21.15.152:0/572630188 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87740051c0 0x7f879015db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:16.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:16.387+0000 7f8786a00640 1 -- 172.21.15.152:0/572630188 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f879013d770 msgr2=0x7f879013dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:16.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:16.387+0000 7f8786a00640 1 --2- 172.21.15.152:0/572630188 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f879013d770 0x7f879013dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:16.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:16.387+0000 7f8786a00640 1 -- 172.21.15.152:0/572630188 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87740051c0 msgr2=0x7f879015db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:16.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:16.387+0000 7f8786a00640 1 --2- 172.21.15.152:0/572630188 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87740051c0 0x7f879015db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:16.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:16.387+0000 7f8786a00640 1 -- 172.21.15.152:0/572630188 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87900b1150 con 0x7f87900b09e0 2024-04-20T11:27:16.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:16.387+0000 7f8786a00640 1 --2- 172.21.15.152:0/572630188 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87900b09e0 0x7f87901597e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f877800e480 tx=0x7f877800e940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:16.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:16.387+0000 7f8787e00640 1 -- 172.21.15.152:0/572630188 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87780052e0 con 0x7f87900b09e0 2024-04-20T11:27:16.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:16.388+0000 7f8787e00640 1 -- 172.21.15.152:0/572630188 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8778018070 con 0x7f87900b09e0 2024-04-20T11:27:16.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:16.388+0000 7f8787e00640 1 -- 172.21.15.152:0/572630188 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8778003830 con 0x7f87900b09e0 2024-04-20T11:27:16.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:16.388+0000 7f8794c00640 1 -- 172.21.15.152:0/572630188 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87900b1430 con 0x7f87900b09e0 2024-04-20T11:27:16.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:16.388+0000 7f8794c00640 1 -- 172.21.15.152:0/572630188 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87900b19f0 con 0x7f87900b09e0 2024-04-20T11:27:16.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:16.389+0000 7f8794c00640 1 -- 172.21.15.152:0/572630188 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8790075e60 con 0x7f87900b09e0 2024-04-20T11:27:16.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:16.390+0000 7f8787e00640 1 -- 172.21.15.152:0/572630188 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f87780039d0 con 0x7f87900b09e0 2024-04-20T11:27:16.391 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:16.391+0000 7f8787e00640 1 --2- 172.21.15.152:0/572630188 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f876c06c8f0 0x7f876c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:16.391 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:16.391+0000 7f8787e00640 1 -- 172.21.15.152:0/572630188 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f877808b8a0 con 0x7f87900b09e0 2024-04-20T11:27:16.391 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:16.392+0000 7f878f400640 1 --2- 172.21.15.152:0/572630188 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f876c06c8f0 0x7f876c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:16.391 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:16.392+0000 7f878f400640 1 --2- 172.21.15.152:0/572630188 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f876c06c8f0 0x7f876c06edb0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8780009a10 tx=0x7f8780008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:16.395 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:16.396+0000 7f8787e00640 1 -- 172.21.15.152:0/572630188 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f8778059cc0 con 0x7f87900b09e0 2024-04-20T11:27:16.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:16.636+0000 7f8794c00640 1 -- 172.21.15.152:0/572630188 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f87900b3490 con 0x7f87900b09e0 2024-04-20T11:27:16.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:16.637+0000 7f8787e00640 1 -- 172.21.15.152:0/572630188 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+73 (secure 0 0 0) 0x7f87780595c0 con 0x7f87900b09e0 2024-04-20T11:27:16.636 INFO:teuthology.orchestra.run.smithi152.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-20T11:27:16.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:16.640+0000 7f8794c00640 1 -- 172.21.15.152:0/572630188 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f876c06c8f0 msgr2=0x7f876c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:16.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:16.640+0000 7f8794c00640 1 --2- 172.21.15.152:0/572630188 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f876c06c8f0 0x7f876c06edb0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8780009a10 tx=0x7f8780008040 comp rx=0 tx=0).stop 2024-04-20T11:27:16.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:16.640+0000 7f8794c00640 1 -- 172.21.15.152:0/572630188 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87900b09e0 msgr2=0x7f87901597e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:16.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:16.640+0000 7f8794c00640 1 --2- 172.21.15.152:0/572630188 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87900b09e0 0x7f87901597e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f877800e480 tx=0x7f877800e940 comp rx=0 tx=0).stop 2024-04-20T11:27:16.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:16.640+0000 7f8794c00640 1 -- 172.21.15.152:0/572630188 shutdown_connections 2024-04-20T11:27:16.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:16.640+0000 7f8794c00640 1 --2- 172.21.15.152:0/572630188 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f876c06c8f0 0x7f876c06edb0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:16.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:16.640+0000 7f8794c00640 1 --2- 172.21.15.152:0/572630188 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f879013d770 0x7f879013dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:16.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:16.640+0000 7f8794c00640 1 --2- 172.21.15.152:0/572630188 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87900b09e0 0x7f87901597e0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:16.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:16.640+0000 7f8794c00640 1 --2- 172.21.15.152:0/572630188 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87740051c0 0x7f879015db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:16.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:16.640+0000 7f8794c00640 1 -- 172.21.15.152:0/572630188 >> 172.21.15.152:0/572630188 conn(0x7f8790067ae0 msgr2=0x7f879013eb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:16.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:16.640+0000 7f8794c00640 1 -- 172.21.15.152:0/572630188 shutdown_connections 2024-04-20T11:27:16.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:16.640+0000 7f8794c00640 1 -- 172.21.15.152:0/572630188 wait complete. 2024-04-20T11:27:16.660 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-20T11:27:16.660 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-20T11:27:16.660 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi152.front.sepia.ceph.com 2024-04-20T11:27:16.661 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-20T11:27:16.661 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-20T11:27:16.661 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-20T11:27:16.661 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-20T11:27:16.661 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-20T11:27:16.661 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-20T11:27:16.661 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:27:16.661 DEBUG:teuthology.orchestra.run.smithi152:> ip addr 2024-04-20T11:27:16.713 INFO:teuthology.orchestra.run.smithi152.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-20T11:27:16.713 INFO:teuthology.orchestra.run.smithi152.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-20T11:27:16.713 INFO:teuthology.orchestra.run.smithi152.stdout: inet 127.0.0.1/8 scope host lo 2024-04-20T11:27:16.714 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:27:16.714 INFO:teuthology.orchestra.run.smithi152.stdout: inet6 ::1/128 scope host 2024-04-20T11:27:16.714 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:27:16.714 INFO:teuthology.orchestra.run.smithi152.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T11:27:16.714 INFO:teuthology.orchestra.run.smithi152.stdout: link/ether 0c:c4:7a:da:6c:44 brd ff:ff:ff:ff:ff:ff 2024-04-20T11:27:16.714 INFO:teuthology.orchestra.run.smithi152.stdout: altname enp7s0f0 2024-04-20T11:27:16.714 INFO:teuthology.orchestra.run.smithi152.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T11:27:16.714 INFO:teuthology.orchestra.run.smithi152.stdout: link/ether 0c:c4:7a:da:6c:45 brd ff:ff:ff:ff:ff:ff 2024-04-20T11:27:16.714 INFO:teuthology.orchestra.run.smithi152.stdout: altname enp7s0f1 2024-04-20T11:27:16.714 INFO:teuthology.orchestra.run.smithi152.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T11:27:16.714 INFO:teuthology.orchestra.run.smithi152.stdout: link/ether 0c:c4:7a:88:6d:e2 brd ff:ff:ff:ff:ff:ff 2024-04-20T11:27:16.714 INFO:teuthology.orchestra.run.smithi152.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-20T11:27:16.714 INFO:teuthology.orchestra.run.smithi152.stdout: link/ether 0c:c4:7a:88:6d:e3 brd ff:ff:ff:ff:ff:ff 2024-04-20T11:27:16.715 INFO:teuthology.orchestra.run.smithi152.stdout: inet 172.21.15.152/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-20T11:27:16.715 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft 42769sec preferred_lft 42769sec 2024-04-20T11:27:16.715 INFO:teuthology.orchestra.run.smithi152.stdout: inet6 fe80::ec4:7aff:fe88:6de3/64 scope link 2024-04-20T11:27:16.715 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:27:16.716 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-20T11:27:16.716 DEBUG:teuthology.orchestra.run.smithi152:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T11:27:16.716 DEBUG:teuthology.orchestra.run.smithi152:> set -e 2024-04-20T11:27:16.716 DEBUG:teuthology.orchestra.run.smithi152:> sudo ip link add name ceph-brx type bridge 2024-04-20T11:27:16.716 DEBUG:teuthology.orchestra.run.smithi152:> sudo ip addr flush dev ceph-brx 2024-04-20T11:27:16.716 DEBUG:teuthology.orchestra.run.smithi152:> sudo ip link set ceph-brx up 2024-04-20T11:27:16.716 DEBUG:teuthology.orchestra.run.smithi152:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-20T11:27:16.716 DEBUG:teuthology.orchestra.run.smithi152:> ') 2024-04-20T11:27:16.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T11:27:17.195 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T11:27:17.208 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:27:17.208 DEBUG:teuthology.orchestra.run.smithi152:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-20T11:27:17.274 INFO:teuthology.orchestra.run.smithi152.stdout:1 2024-04-20T11:27:17.278 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:27:17.278 DEBUG:teuthology.orchestra.run.smithi152:> route 2024-04-20T11:27:17.337 INFO:teuthology.orchestra.run.smithi152.stdout:Kernel IP routing table 2024-04-20T11:27:17.337 INFO:teuthology.orchestra.run.smithi152.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-20T11:27:17.337 INFO:teuthology.orchestra.run.smithi152.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-20T11:27:17.337 INFO:teuthology.orchestra.run.smithi152.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-20T11:27:17.337 INFO:teuthology.orchestra.run.smithi152.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-20T11:27:17.338 DEBUG:teuthology.orchestra.run.smithi152:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T11:27:17.338 DEBUG:teuthology.orchestra.run.smithi152:> set -e 2024-04-20T11:27:17.338 DEBUG:teuthology.orchestra.run.smithi152:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-20T11:27:17.338 DEBUG:teuthology.orchestra.run.smithi152:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-20T11:27:17.338 DEBUG:teuthology.orchestra.run.smithi152:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-20T11:27:17.338 DEBUG:teuthology.orchestra.run.smithi152:> ') 2024-04-20T11:27:17.437 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T11:27:18.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T11:27:18.045 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:27:18.045 DEBUG:teuthology.orchestra.run.smithi152:> ip netns list 2024-04-20T11:27:18.098 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:27:18.099 DEBUG:teuthology.orchestra.run.smithi152:> ip netns list-id 2024-04-20T11:27:18.150 DEBUG:teuthology.orchestra.run.smithi152:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T11:27:18.151 DEBUG:teuthology.orchestra.run.smithi152:> set -e 2024-04-20T11:27:18.151 DEBUG:teuthology.orchestra.run.smithi152:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T11:27:18.151 DEBUG:teuthology.orchestra.run.smithi152:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-20T11:27:18.151 DEBUG:teuthology.orchestra.run.smithi152:> ') 2024-04-20T11:27:18.255 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T11:27:18.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T11:27:18.332 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-20T11:27:18.332 DEBUG:teuthology.orchestra.run.smithi152:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T11:27:18.332 DEBUG:teuthology.orchestra.run.smithi152:> set -e 2024-04-20T11:27:18.332 DEBUG:teuthology.orchestra.run.smithi152:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-20T11:27:18.332 DEBUG:teuthology.orchestra.run.smithi152:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-20T11:27:18.332 DEBUG:teuthology.orchestra.run.smithi152:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-20T11:27:18.332 DEBUG:teuthology.orchestra.run.smithi152:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-20T11:27:18.332 DEBUG:teuthology.orchestra.run.smithi152:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-20T11:27:18.333 DEBUG:teuthology.orchestra.run.smithi152:> ') 2024-04-20T11:27:18.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T11:27:18.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T11:27:18.623 DEBUG:teuthology.orchestra.run.smithi152:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T11:27:18.623 DEBUG:teuthology.orchestra.run.smithi152:> set -e 2024-04-20T11:27:18.623 DEBUG:teuthology.orchestra.run.smithi152:> sudo ip link set brx.0 up 2024-04-20T11:27:18.623 DEBUG:teuthology.orchestra.run.smithi152:> sudo ip link set dev brx.0 master ceph-brx 2024-04-20T11:27:18.624 DEBUG:teuthology.orchestra.run.smithi152:> ') 2024-04-20T11:27:18.718 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T11:27:18.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T11:27:18.825 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-20T11:27:18.826 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T11:27:18.826 DEBUG:teuthology.orchestra.run.smithi152:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-20T11:27:18.868 INFO:teuthology.orchestra.run.smithi152.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-20T11:27:18.868 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T11:27:18.868 DEBUG:teuthology.orchestra.run.smithi152:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-20T11:27:18.960 DEBUG:teuthology.orchestra.run.smithi152:> sudo modprobe fuse 2024-04-20T11:27:19.042 DEBUG:teuthology.orchestra.run.smithi152:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T11:27:19.094 INFO:teuthology.orchestra.run.smithi152.stdout:/proc 2024-04-20T11:27:19.094 INFO:teuthology.orchestra.run.smithi152.stdout:/sys 2024-04-20T11:27:19.094 INFO:teuthology.orchestra.run.smithi152.stdout:/dev 2024-04-20T11:27:19.094 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/security 2024-04-20T11:27:19.094 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/shm 2024-04-20T11:27:19.095 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/pts 2024-04-20T11:27:19.095 INFO:teuthology.orchestra.run.smithi152.stdout:/run 2024-04-20T11:27:19.095 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/cgroup 2024-04-20T11:27:19.095 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/pstore 2024-04-20T11:27:19.095 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/bpf 2024-04-20T11:27:19.095 INFO:teuthology.orchestra.run.smithi152.stdout:/ 2024-04-20T11:27:19.095 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/selinux 2024-04-20T11:27:19.095 INFO:teuthology.orchestra.run.smithi152.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T11:27:19.095 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/mqueue 2024-04-20T11:27:19.095 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/hugepages 2024-04-20T11:27:19.095 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/debug 2024-04-20T11:27:19.095 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/tracing 2024-04-20T11:27:19.095 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/fuse/connections 2024-04-20T11:27:19.096 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/config 2024-04-20T11:27:19.096 INFO:teuthology.orchestra.run.smithi152.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T11:27:19.096 INFO:teuthology.orchestra.run.smithi152.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T11:27:19.096 INFO:teuthology.orchestra.run.smithi152.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T11:27:19.096 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T11:27:19.096 INFO:teuthology.orchestra.run.smithi152.stdout:/run/user/1000 2024-04-20T11:27:19.096 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph 2024-04-20T11:27:19.096 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph/osd/ceph-0 2024-04-20T11:27:19.096 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph/osd/ceph-1 2024-04-20T11:27:19.096 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph/osd/ceph-2 2024-04-20T11:27:19.096 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph/osd/ceph-3 2024-04-20T11:27:19.096 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns 2024-04-20T11:27:19.096 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T11:27:19.096 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T11:27:19.097 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:27:19.097 DEBUG:teuthology.orchestra.run.smithi152:> ls /sys/fs/fuse/connections 2024-04-20T11:27:19.143 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-20T11:27:19.143 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-20T11:27:19.190 DEBUG:teuthology.orchestra.run.smithi152:> sudo modprobe fuse 2024-04-20T11:27:19.264 DEBUG:teuthology.orchestra.run.smithi152:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T11:27:19.307 INFO:teuthology.orchestra.run.smithi152.stdout:/proc 2024-04-20T11:27:19.307 INFO:teuthology.orchestra.run.smithi152.stdout:/sys 2024-04-20T11:27:19.307 INFO:teuthology.orchestra.run.smithi152.stdout:/dev 2024-04-20T11:27:19.307 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/security 2024-04-20T11:27:19.308 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/shm 2024-04-20T11:27:19.308 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/pts 2024-04-20T11:27:19.308 INFO:teuthology.orchestra.run.smithi152.stdout:/run 2024-04-20T11:27:19.308 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/cgroup 2024-04-20T11:27:19.308 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/pstore 2024-04-20T11:27:19.308 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/bpf 2024-04-20T11:27:19.308 INFO:teuthology.orchestra.run.smithi152.stdout:/ 2024-04-20T11:27:19.308 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/selinux 2024-04-20T11:27:19.308 INFO:teuthology.orchestra.run.smithi152.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T11:27:19.308 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/mqueue 2024-04-20T11:27:19.308 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/hugepages 2024-04-20T11:27:19.308 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/debug 2024-04-20T11:27:19.308 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/tracing 2024-04-20T11:27:19.308 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/fuse/connections 2024-04-20T11:27:19.309 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/config 2024-04-20T11:27:19.309 INFO:teuthology.orchestra.run.smithi152.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T11:27:19.309 INFO:teuthology.orchestra.run.smithi152.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T11:27:19.309 INFO:teuthology.orchestra.run.smithi152.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T11:27:19.309 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T11:27:19.309 INFO:teuthology.orchestra.run.smithi152.stdout:/run/user/1000 2024-04-20T11:27:19.309 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph 2024-04-20T11:27:19.309 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph/osd/ceph-0 2024-04-20T11:27:19.309 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph/osd/ceph-1 2024-04-20T11:27:19.309 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph/osd/ceph-2 2024-04-20T11:27:19.309 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph/osd/ceph-3 2024-04-20T11:27:19.309 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns 2024-04-20T11:27:19.309 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T11:27:19.309 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T11:27:19.310 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:27:19.311 DEBUG:teuthology.orchestra.run.smithi152:> ls /sys/fs/fuse/connections 2024-04-20T11:27:19.382 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi152.stderr:2024-04-20T11:27:19.382+0000 7fb953e39480 -1 init, newargv = 0x55d8face84a0 newargc=15 2024-04-20T11:27:19.382 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi152.stderr:ceph-fuse[30923]: starting ceph client 2024-04-20T11:27:19.394 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi152.stderr:ceph-fuse[30923]: starting fuse 2024-04-20T11:27:19.396 INFO:teuthology.orchestra.run.smithi152.stdout:42 2024-04-20T11:27:19.396 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-04-20T11:27:19.396 DEBUG:teuthology.orchestra.run.smithi152:> sudo stdin-killer -- python3 -c ' 2024-04-20T11:27:19.396 DEBUG:teuthology.orchestra.run.smithi152:> import glob 2024-04-20T11:27:19.396 DEBUG:teuthology.orchestra.run.smithi152:> import re 2024-04-20T11:27:19.396 DEBUG:teuthology.orchestra.run.smithi152:> import os 2024-04-20T11:27:19.396 DEBUG:teuthology.orchestra.run.smithi152:> import subprocess 2024-04-20T11:27:19.396 DEBUG:teuthology.orchestra.run.smithi152:> 2024-04-20T11:27:19.396 DEBUG:teuthology.orchestra.run.smithi152:> def _find_admin_socket(client_name): 2024-04-20T11:27:19.396 DEBUG:teuthology.orchestra.run.smithi152:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-20T11:27:19.397 DEBUG:teuthology.orchestra.run.smithi152:> files = glob.glob(asok_path) 2024-04-20T11:27:19.397 DEBUG:teuthology.orchestra.run.smithi152:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-20T11:27:19.397 DEBUG:teuthology.orchestra.run.smithi152:> 2024-04-20T11:27:19.397 DEBUG:teuthology.orchestra.run.smithi152:> # Given a non-glob path, it better be there 2024-04-20T11:27:19.397 DEBUG:teuthology.orchestra.run.smithi152:> if "*" not in asok_path: 2024-04-20T11:27:19.397 DEBUG:teuthology.orchestra.run.smithi152:> assert(len(files) == 1) 2024-04-20T11:27:19.397 DEBUG:teuthology.orchestra.run.smithi152:> return files[0] 2024-04-20T11:27:19.397 DEBUG:teuthology.orchestra.run.smithi152:> 2024-04-20T11:27:19.397 DEBUG:teuthology.orchestra.run.smithi152:> for f in files: 2024-04-20T11:27:19.397 DEBUG:teuthology.orchestra.run.smithi152:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-20T11:27:19.397 DEBUG:teuthology.orchestra.run.smithi152:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-20T11:27:19.397 DEBUG:teuthology.orchestra.run.smithi152:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-20T11:27:19.397 DEBUG:teuthology.orchestra.run.smithi152:> contents = proc_f.read() 2024-04-20T11:27:19.397 DEBUG:teuthology.orchestra.run.smithi152:> if mountpoint in contents: 2024-04-20T11:27:19.397 DEBUG:teuthology.orchestra.run.smithi152:> return f 2024-04-20T11:27:19.397 DEBUG:teuthology.orchestra.run.smithi152:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-20T11:27:19.398 DEBUG:teuthology.orchestra.run.smithi152:> 2024-04-20T11:27:19.398 DEBUG:teuthology.orchestra.run.smithi152:> print(_find_admin_socket("client.0")) 2024-04-20T11:27:19.398 DEBUG:teuthology.orchestra.run.smithi152:> ' 2024-04-20T11:27:19.579 INFO:teuthology.orchestra.run.smithi152.stdout:/var/run/ceph/ceph-client.0.30923.asok 2024-04-20T11:27:19.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T11:27:19.596 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.30923.asok 2024-04-20T11:27:19.596 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:27:19.596 DEBUG:teuthology.orchestra.run.smithi152:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.30923.asok status 2024-04-20T11:27:19.825 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-04-20T11:27:19.825 INFO:teuthology.orchestra.run.smithi152.stdout: "metadata": { 2024-04-20T11:27:19.825 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_sha1": "a8c84f5997950fd7d008204839253291d52e6eda", 2024-04-20T11:27:19.825 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", 2024-04-20T11:27:19.825 INFO:teuthology.orchestra.run.smithi152.stdout: "entity_id": "0", 2024-04-20T11:27:19.825 INFO:teuthology.orchestra.run.smithi152.stdout: "hostname": "smithi152", 2024-04-20T11:27:19.825 INFO:teuthology.orchestra.run.smithi152.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-20T11:27:19.825 INFO:teuthology.orchestra.run.smithi152.stdout: "pid": "30923", 2024-04-20T11:27:19.825 INFO:teuthology.orchestra.run.smithi152.stdout: "root": "/" 2024-04-20T11:27:19.825 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:27:19.825 INFO:teuthology.orchestra.run.smithi152.stdout: "dentry_count": 0, 2024-04-20T11:27:19.826 INFO:teuthology.orchestra.run.smithi152.stdout: "dentry_pinned_count": 0, 2024-04-20T11:27:19.826 INFO:teuthology.orchestra.run.smithi152.stdout: "id": 4650, 2024-04-20T11:27:19.826 INFO:teuthology.orchestra.run.smithi152.stdout: "inst": { 2024-04-20T11:27:19.826 INFO:teuthology.orchestra.run.smithi152.stdout: "name": { 2024-04-20T11:27:19.826 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "client", 2024-04-20T11:27:19.826 INFO:teuthology.orchestra.run.smithi152.stdout: "num": 4650 2024-04-20T11:27:19.826 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:27:19.827 INFO:teuthology.orchestra.run.smithi152.stdout: "addr": { 2024-04-20T11:27:19.827 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "v1", 2024-04-20T11:27:19.827 INFO:teuthology.orchestra.run.smithi152.stdout: "addr": "172.21.15.152:0", 2024-04-20T11:27:19.827 INFO:teuthology.orchestra.run.smithi152.stdout: "nonce": 297387260 2024-04-20T11:27:19.827 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:27:19.827 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:27:19.827 INFO:teuthology.orchestra.run.smithi152.stdout: "addr": { 2024-04-20T11:27:19.828 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "v1", 2024-04-20T11:27:19.828 INFO:teuthology.orchestra.run.smithi152.stdout: "addr": "172.21.15.152:0", 2024-04-20T11:27:19.828 INFO:teuthology.orchestra.run.smithi152.stdout: "nonce": 297387260 2024-04-20T11:27:19.828 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:27:19.828 INFO:teuthology.orchestra.run.smithi152.stdout: "inst_str": "client.4650 172.21.15.152:0/297387260", 2024-04-20T11:27:19.828 INFO:teuthology.orchestra.run.smithi152.stdout: "addr_str": "172.21.15.152:0/297387260", 2024-04-20T11:27:19.829 INFO:teuthology.orchestra.run.smithi152.stdout: "inode_count": 1, 2024-04-20T11:27:19.829 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_epoch": 7, 2024-04-20T11:27:19.829 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_epoch": 20, 2024-04-20T11:27:19.829 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_epoch_barrier": 0, 2024-04-20T11:27:19.829 INFO:teuthology.orchestra.run.smithi152.stdout: "blocklisted": false, 2024-04-20T11:27:19.829 INFO:teuthology.orchestra.run.smithi152.stdout: "fs_name": "cephfs" 2024-04-20T11:27:19.829 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-04-20T11:27:19.841 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-20T11:27:20.134 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:20.135+0000 7f4ebfe00640 1 Processor -- start 2024-04-20T11:27:20.135 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:20.135+0000 7f4ebfe00640 1 -- start start 2024-04-20T11:27:20.137 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:20.137+0000 7f4ebfe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4ec0160b90 0x7f4ec015d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:20.137 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:20.137+0000 7f4ebfe00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f4ec0071cd0 0x7f4ec0072160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:20.137 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:20.137+0000 7f4ebfe00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ec00ce8c0 con 0x7f4ec015d1e0 2024-04-20T11:27:20.137 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:20.137+0000 7f4ebfe00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f4ec00726a0 con 0x7f4ec0071cd0 2024-04-20T11:27:20.137 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:20.137+0000 7f4ebfe00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4ec01600b0 con 0x7f4ec0160b90 2024-04-20T11:27:20.137 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:20.137+0000 7f4ebf400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f4ec0071cd0 0x7f4ec0072160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:20.137 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:20.137+0000 7f4ebf400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f4ec0071cd0 0x7f4ec0072160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:58010/0 (socket says 172.21.15.152:58010) 2024-04-20T11:27:20.137 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:20.137+0000 7f4ebf400640 1 -- 172.21.15.152:0/3445481279 learned_addr learned my addr 172.21.15.152:0/3445481279 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:20.137 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:20.137+0000 7f4ebe000640 1 --2- 172.21.15.152:0/3445481279 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4ec0160b90 0x7f4ec015d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:20.138 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:20.138+0000 7f4ebf400640 1 -- 172.21.15.152:0/3445481279 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4ec0160b90 msgr2=0x7f4ec015d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:20.138 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:20.138+0000 7f4ebf400640 1 --2- 172.21.15.152:0/3445481279 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4ec0160b90 0x7f4ec015d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:20.138 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:20.138+0000 7f4ebf400640 1 -- 172.21.15.152:0/3445481279 >> v1:172.21.15.175:6789/0 conn(0x7f4ec015d1e0 legacy=0x7f4ec00cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:20.138 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:20.138+0000 7f4ebf400640 1 -- 172.21.15.152:0/3445481279 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ec0160390 con 0x7f4ec0071cd0 2024-04-20T11:27:20.139 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:20.139+0000 7f4ebf400640 1 --2- 172.21.15.152:0/3445481279 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ec0071cd0 0x7f4ec0072160 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4eb8009e50 tx=0x7f4eb802bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0aece5a66226606 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:20.139 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:20.139+0000 7f4ebd600640 1 -- 172.21.15.152:0/3445481279 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb802caf0 con 0x7f4ec0071cd0 2024-04-20T11:27:20.139 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:20.139+0000 7f4ebd600640 1 -- 172.21.15.152:0/3445481279 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4eb802cc90 con 0x7f4ec0071cd0 2024-04-20T11:27:20.139 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:20.139+0000 7f4ebd600640 1 -- 172.21.15.152:0/3445481279 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb8034610 con 0x7f4ec0071cd0 2024-04-20T11:27:20.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:20.140+0000 7f4ebfe00640 1 -- 172.21.15.152:0/3445481279 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ec0071cd0 msgr2=0x7f4ec0072160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:20.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:20.140+0000 7f4ebfe00640 1 --2- 172.21.15.152:0/3445481279 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ec0071cd0 0x7f4ec0072160 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4eb8009e50 tx=0x7f4eb802bf50 comp rx=0 tx=0).stop 2024-04-20T11:27:20.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:20.140+0000 7f4ebfe00640 1 -- 172.21.15.152:0/3445481279 shutdown_connections 2024-04-20T11:27:20.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:20.140+0000 7f4ebfe00640 1 --2- 172.21.15.152:0/3445481279 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ec0071cd0 0x7f4ec0072160 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:20.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:20.140+0000 7f4ebfe00640 1 --2- 172.21.15.152:0/3445481279 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4ec0160b90 0x7f4ec015d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:20.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:20.140+0000 7f4ebfe00640 1 -- 172.21.15.152:0/3445481279 >> 172.21.15.152:0/3445481279 conn(0x7f4ec0067ae0 msgr2=0x7f4ec006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:20.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:20.140+0000 7f4ebfe00640 1 -- 172.21.15.152:0/3445481279 shutdown_connections 2024-04-20T11:27:20.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:20.140+0000 7f4ebfe00640 1 -- 172.21.15.152:0/3445481279 wait complete. 2024-04-20T11:27:20.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:20.142+0000 7f4ebfe00640 1 Processor -- start 2024-04-20T11:27:20.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:20.142+0000 7f4ebfe00640 1 -- start start 2024-04-20T11:27:20.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:20.143+0000 7f4ebfe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ec0071cd0 0x7f4ec00aeb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:20.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:20.143+0000 7f4ebfe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4ec015d1e0 0x7f4ec015edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:20.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:20.143+0000 7f4ebfe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4ec0160b90 0x7f4ec00ad8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:20.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:20.143+0000 7f4ebfe00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec00ccd70 con 0x7f4ec0071cd0 2024-04-20T11:27:20.145 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:20.143+0000 7f4ebfe00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec00ceba0 con 0x7f4ec0160b90 2024-04-20T11:27:20.145 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:20.143+0000 7f4ebfe00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4ec00ccef0 con 0x7f4ec015d1e0 2024-04-20T11:27:20.145 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:20.143+0000 7f4ebea00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ec0071cd0 0x7f4ec00aeb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:20.145 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:20.143+0000 7f4ebea00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ec0071cd0 0x7f4ec00aeb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:58016/0 (socket says 172.21.15.152:58016) 2024-04-20T11:27:20.145 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:20.144+0000 7f4ebea00640 1 -- 172.21.15.152:0/3439560870 learned_addr learned my addr 172.21.15.152:0/3439560870 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:20.145 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:20.144+0000 7f4ebe000640 1 --2- 172.21.15.152:0/3439560870 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4ec015d1e0 0x7f4ec015edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:20.145 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:20.144+0000 7f4ebf400640 1 --2- 172.21.15.152:0/3439560870 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4ec0160b90 0x7f4ec00ad8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:20.145 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:20.144+0000 7f4ebea00640 1 -- 172.21.15.152:0/3439560870 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4ec015d1e0 msgr2=0x7f4ec015edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:20.145 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:20.144+0000 7f4ebea00640 1 --2- 172.21.15.152:0/3439560870 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4ec015d1e0 0x7f4ec015edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:20.145 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:20.144+0000 7f4ebea00640 1 -- 172.21.15.152:0/3439560870 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4ec0160b90 msgr2=0x7f4ec00ad8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:20.145 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:20.144+0000 7f4ebea00640 1 --2- 172.21.15.152:0/3439560870 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4ec0160b90 0x7f4ec00ad8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:20.145 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:20.144+0000 7f4ebea00640 1 -- 172.21.15.152:0/3439560870 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ec01e5e30 con 0x7f4ec0071cd0 2024-04-20T11:27:20.145 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:20.144+0000 7f4ebf400640 1 --2- 172.21.15.152:0/3439560870 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4ec0160b90 0x7f4ec00ad8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:27:20.145 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:20.144+0000 7f4ebea00640 1 --2- 172.21.15.152:0/3439560870 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ec0071cd0 0x7f4ec00aeb40 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4eb000c430 tx=0x7f4eb000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:20.146 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:20.144+0000 7f4eb6a00640 1 -- 172.21.15.152:0/3439560870 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb00052d0 con 0x7f4ec0071cd0 2024-04-20T11:27:20.146 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:20.145+0000 7f4eb6a00640 1 -- 172.21.15.152:0/3439560870 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4eb0005490 con 0x7f4ec0071cd0 2024-04-20T11:27:20.146 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:20.145+0000 7f4eb6a00640 1 -- 172.21.15.152:0/3439560870 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb0007260 con 0x7f4ec0071cd0 2024-04-20T11:27:20.146 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:20.145+0000 7f4ebfe00640 1 -- 172.21.15.152:0/3439560870 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ec01e6110 con 0x7f4ec0071cd0 2024-04-20T11:27:20.146 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:20.145+0000 7f4ebfe00640 1 -- 172.21.15.152:0/3439560870 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ec01e66d0 con 0x7f4ec0071cd0 2024-04-20T11:27:20.146 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:20.146+0000 7f4ebfe00640 1 -- 172.21.15.152:0/3439560870 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ec0075e60 con 0x7f4ec0071cd0 2024-04-20T11:27:20.146 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:20.147+0000 7f4eb6a00640 1 -- 172.21.15.152:0/3439560870 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4eb0007400 con 0x7f4ec0071cd0 2024-04-20T11:27:20.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:20.148+0000 7f4eb6a00640 1 --2- 172.21.15.152:0/3439560870 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f4ea006c8f0 0x7f4ea006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:20.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:20.149+0000 7f4eb6a00640 1 -- 172.21.15.152:0/3439560870 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f4eb008b0d0 con 0x7f4ec0071cd0 2024-04-20T11:27:20.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:20.149+0000 7f4ebe000640 1 --2- 172.21.15.152:0/3439560870 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f4ea006c8f0 0x7f4ea006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:20.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:20.150+0000 7f4ebe000640 1 --2- 172.21.15.152:0/3439560870 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f4ea006c8f0 0x7f4ea006edb0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4ec0159ab0 tx=0x7f4eac008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:20.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:20.154+0000 7f4eb6a00640 1 -- 172.21.15.152:0/3439560870 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f4eb0059500 con 0x7f4ec0071cd0 2024-04-20T11:27:20.397 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:20.398+0000 7f4ebfe00640 1 -- 172.21.15.152:0/3439560870 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f4ec0072c90 con 0x7f4ec0071cd0 2024-04-20T11:27:20.398 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:20.398+0000 7f4eb6a00640 1 -- 172.21.15.152:0/3439560870 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+73 (secure 0 0 0) 0x7f4eb0058e00 con 0x7f4ec0071cd0 2024-04-20T11:27:20.398 INFO:teuthology.orchestra.run.smithi152.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-20T11:27:20.401 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:20.401+0000 7f4ebfe00640 1 -- 172.21.15.152:0/3439560870 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f4ea006c8f0 msgr2=0x7f4ea006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:20.401 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:20.401+0000 7f4ebfe00640 1 --2- 172.21.15.152:0/3439560870 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f4ea006c8f0 0x7f4ea006edb0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4ec0159ab0 tx=0x7f4eac008040 comp rx=0 tx=0).stop 2024-04-20T11:27:20.401 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:20.401+0000 7f4ebfe00640 1 -- 172.21.15.152:0/3439560870 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ec0071cd0 msgr2=0x7f4ec00aeb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:20.401 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:20.401+0000 7f4ebfe00640 1 --2- 172.21.15.152:0/3439560870 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ec0071cd0 0x7f4ec00aeb40 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4eb000c430 tx=0x7f4eb000c8f0 comp rx=0 tx=0).stop 2024-04-20T11:27:20.401 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:20.401+0000 7f4ebfe00640 1 -- 172.21.15.152:0/3439560870 shutdown_connections 2024-04-20T11:27:20.401 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:20.401+0000 7f4ebfe00640 1 --2- 172.21.15.152:0/3439560870 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f4ea006c8f0 0x7f4ea006edb0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:20.402 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:20.401+0000 7f4ebfe00640 1 --2- 172.21.15.152:0/3439560870 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4ec0160b90 0x7f4ec00ad8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:20.402 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:20.401+0000 7f4ebfe00640 1 --2- 172.21.15.152:0/3439560870 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4ec015d1e0 0x7f4ec015edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:20.402 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:20.401+0000 7f4ebfe00640 1 --2- 172.21.15.152:0/3439560870 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ec0071cd0 0x7f4ec00aeb40 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:20.402 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:20.401+0000 7f4ebfe00640 1 -- 172.21.15.152:0/3439560870 >> 172.21.15.152:0/3439560870 conn(0x7f4ec0067ae0 msgr2=0x7f4ec00b2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:20.402 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:20.402+0000 7f4ebfe00640 1 -- 172.21.15.152:0/3439560870 shutdown_connections 2024-04-20T11:27:20.402 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:20.402+0000 7f4ebfe00640 1 -- 172.21.15.152:0/3439560870 wait complete. 2024-04-20T11:27:20.420 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-20T11:27:20.420 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-20T11:27:20.420 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi152.front.sepia.ceph.com 2024-04-20T11:27:20.420 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-04-20T11:27:20.420 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-20T11:27:20.420 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-20T11:27:20.421 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-20T11:27:20.421 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-20T11:27:20.421 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-04-20T11:27:20.421 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:27:20.421 DEBUG:teuthology.orchestra.run.smithi152:> ip addr 2024-04-20T11:27:20.468 INFO:teuthology.orchestra.run.smithi152.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-20T11:27:20.468 INFO:teuthology.orchestra.run.smithi152.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-20T11:27:20.468 INFO:teuthology.orchestra.run.smithi152.stdout: inet 127.0.0.1/8 scope host lo 2024-04-20T11:27:20.468 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:27:20.468 INFO:teuthology.orchestra.run.smithi152.stdout: inet6 ::1/128 scope host 2024-04-20T11:27:20.468 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:27:20.468 INFO:teuthology.orchestra.run.smithi152.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T11:27:20.468 INFO:teuthology.orchestra.run.smithi152.stdout: link/ether 0c:c4:7a:da:6c:44 brd ff:ff:ff:ff:ff:ff 2024-04-20T11:27:20.468 INFO:teuthology.orchestra.run.smithi152.stdout: altname enp7s0f0 2024-04-20T11:27:20.468 INFO:teuthology.orchestra.run.smithi152.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T11:27:20.468 INFO:teuthology.orchestra.run.smithi152.stdout: link/ether 0c:c4:7a:da:6c:45 brd ff:ff:ff:ff:ff:ff 2024-04-20T11:27:20.469 INFO:teuthology.orchestra.run.smithi152.stdout: altname enp7s0f1 2024-04-20T11:27:20.469 INFO:teuthology.orchestra.run.smithi152.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T11:27:20.469 INFO:teuthology.orchestra.run.smithi152.stdout: link/ether 0c:c4:7a:88:6d:e2 brd ff:ff:ff:ff:ff:ff 2024-04-20T11:27:20.469 INFO:teuthology.orchestra.run.smithi152.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-20T11:27:20.469 INFO:teuthology.orchestra.run.smithi152.stdout: link/ether 0c:c4:7a:88:6d:e3 brd ff:ff:ff:ff:ff:ff 2024-04-20T11:27:20.469 INFO:teuthology.orchestra.run.smithi152.stdout: inet 172.21.15.152/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-20T11:27:20.469 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft 42766sec preferred_lft 42766sec 2024-04-20T11:27:20.469 INFO:teuthology.orchestra.run.smithi152.stdout: inet6 fe80::ec4:7aff:fe88:6de3/64 scope link 2024-04-20T11:27:20.469 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:27:20.469 INFO:teuthology.orchestra.run.smithi152.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-20T11:27:20.469 INFO:teuthology.orchestra.run.smithi152.stdout: link/ether 16:e5:08:78:20:47 brd ff:ff:ff:ff:ff:ff 2024-04-20T11:27:20.469 INFO:teuthology.orchestra.run.smithi152.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-20T11:27:20.469 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:27:20.470 INFO:teuthology.orchestra.run.smithi152.stdout: inet6 fe80::a019:d8ff:fe70:1185/64 scope link 2024-04-20T11:27:20.470 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:27:20.470 INFO:teuthology.orchestra.run.smithi152.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-20T11:27:20.470 INFO:teuthology.orchestra.run.smithi152.stdout: link/ether 16:e5:08:78:20:47 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T11:27:20.470 INFO:teuthology.orchestra.run.smithi152.stdout: inet6 fe80::14e5:8ff:fe78:2047/64 scope link 2024-04-20T11:27:20.470 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:27:20.471 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:27:20.471 DEBUG:teuthology.orchestra.run.smithi152:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-20T11:27:20.595 INFO:teuthology.orchestra.run.smithi152.stdout:1 2024-04-20T11:27:20.599 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:27:20.599 DEBUG:teuthology.orchestra.run.smithi152:> route 2024-04-20T11:27:20.653 INFO:teuthology.orchestra.run.smithi152.stdout:Kernel IP routing table 2024-04-20T11:27:20.654 INFO:teuthology.orchestra.run.smithi152.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-20T11:27:20.654 INFO:teuthology.orchestra.run.smithi152.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-20T11:27:20.654 INFO:teuthology.orchestra.run.smithi152.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-20T11:27:20.654 INFO:teuthology.orchestra.run.smithi152.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-20T11:27:20.654 DEBUG:teuthology.orchestra.run.smithi152:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T11:27:20.654 DEBUG:teuthology.orchestra.run.smithi152:> set -e 2024-04-20T11:27:20.654 DEBUG:teuthology.orchestra.run.smithi152:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-20T11:27:20.654 DEBUG:teuthology.orchestra.run.smithi152:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-20T11:27:20.654 DEBUG:teuthology.orchestra.run.smithi152:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-20T11:27:20.655 DEBUG:teuthology.orchestra.run.smithi152:> ') 2024-04-20T11:27:20.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:20 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T11:27:20.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T11:27:20.863 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:27:20.863 DEBUG:teuthology.orchestra.run.smithi152:> ip netns list 2024-04-20T11:27:20.912 INFO:teuthology.orchestra.run.smithi152.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-20T11:27:20.912 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:27:20.912 DEBUG:teuthology.orchestra.run.smithi152:> ip netns list-id 2024-04-20T11:27:20.962 INFO:teuthology.orchestra.run.smithi152.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-04-20T11:27:20.963 DEBUG:teuthology.orchestra.run.smithi152:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T11:27:20.963 DEBUG:teuthology.orchestra.run.smithi152:> set -e 2024-04-20T11:27:20.963 DEBUG:teuthology.orchestra.run.smithi152:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T11:27:20.963 DEBUG:teuthology.orchestra.run.smithi152:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-04-20T11:27:20.963 DEBUG:teuthology.orchestra.run.smithi152:> ') 2024-04-20T11:27:21.050 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:27:21.058 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:27:21.067 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:21 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T11:27:21.133 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T11:27:21.143 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:27:21.143 DEBUG:teuthology.orchestra.run.smithi152:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-20T11:27:21.283 INFO:teuthology.orchestra.run.smithi152.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-20T11:27:21.283 INFO:teuthology.orchestra.run.smithi152.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-20T11:27:21.283 INFO:teuthology.orchestra.run.smithi152.stdout: inet 127.0.0.1/8 scope host lo 2024-04-20T11:27:21.283 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:27:21.283 INFO:teuthology.orchestra.run.smithi152.stdout: inet6 ::1/128 scope host 2024-04-20T11:27:21.283 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:27:21.283 INFO:teuthology.orchestra.run.smithi152.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-20T11:27:21.283 INFO:teuthology.orchestra.run.smithi152.stdout: link/ether 6e:62:68:98:4b:25 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-20T11:27:21.283 INFO:teuthology.orchestra.run.smithi152.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-20T11:27:21.283 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:27:21.283 INFO:teuthology.orchestra.run.smithi152.stdout: inet6 fe80::6c62:68ff:fe98:4b25/64 scope link 2024-04-20T11:27:21.283 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:27:21.286 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:27:21.286 DEBUG:teuthology.orchestra.run.smithi152:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-20T11:27:21.355 INFO:teuthology.orchestra.run.smithi152.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-20T11:27:21.355 INFO:teuthology.orchestra.run.smithi152.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-20T11:27:21.355 INFO:teuthology.orchestra.run.smithi152.stdout: inet 127.0.0.1/8 scope host lo 2024-04-20T11:27:21.355 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:27:21.355 INFO:teuthology.orchestra.run.smithi152.stdout: inet6 ::1/128 scope host 2024-04-20T11:27:21.355 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:27:21.355 INFO:teuthology.orchestra.run.smithi152.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-20T11:27:21.355 INFO:teuthology.orchestra.run.smithi152.stdout: link/ether 6e:62:68:98:4b:25 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-20T11:27:21.355 INFO:teuthology.orchestra.run.smithi152.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-20T11:27:21.355 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:27:21.355 INFO:teuthology.orchestra.run.smithi152.stdout: inet6 fe80::6c62:68ff:fe98:4b25/64 scope link 2024-04-20T11:27:21.356 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:27:21.358 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-04-20T11:27:21.359 DEBUG:teuthology.orchestra.run.smithi152:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T11:27:21.359 DEBUG:teuthology.orchestra.run.smithi152:> set -e 2024-04-20T11:27:21.359 DEBUG:teuthology.orchestra.run.smithi152:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-04-20T11:27:21.359 DEBUG:teuthology.orchestra.run.smithi152:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-20T11:27:21.359 DEBUG:teuthology.orchestra.run.smithi152:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-04-20T11:27:21.359 DEBUG:teuthology.orchestra.run.smithi152:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-04-20T11:27:21.359 DEBUG:teuthology.orchestra.run.smithi152:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-04-20T11:27:21.359 DEBUG:teuthology.orchestra.run.smithi152:> ') 2024-04-20T11:27:21.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:21 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T11:27:21.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T11:27:21.619 DEBUG:teuthology.orchestra.run.smithi152:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T11:27:21.619 DEBUG:teuthology.orchestra.run.smithi152:> set -e 2024-04-20T11:27:21.619 DEBUG:teuthology.orchestra.run.smithi152:> sudo ip link set brx.1 up 2024-04-20T11:27:21.619 DEBUG:teuthology.orchestra.run.smithi152:> sudo ip link set dev brx.1 master ceph-brx 2024-04-20T11:27:21.619 DEBUG:teuthology.orchestra.run.smithi152:> ') 2024-04-20T11:27:21.714 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:21 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T11:27:21.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T11:27:21.816 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2024-04-20T11:27:21.816 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T11:27:21.816 DEBUG:teuthology.orchestra.run.smithi152:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-04-20T11:27:21.855 INFO:teuthology.orchestra.run.smithi152.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-04-20T11:27:21.856 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T11:27:21.856 DEBUG:teuthology.orchestra.run.smithi152:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-04-20T11:27:21.946 DEBUG:teuthology.orchestra.run.smithi152:> sudo modprobe fuse 2024-04-20T11:27:22.064 DEBUG:teuthology.orchestra.run.smithi152:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T11:27:22.106 INFO:teuthology.orchestra.run.smithi152.stdout:/proc 2024-04-20T11:27:22.106 INFO:teuthology.orchestra.run.smithi152.stdout:/sys 2024-04-20T11:27:22.106 INFO:teuthology.orchestra.run.smithi152.stdout:/dev 2024-04-20T11:27:22.107 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/security 2024-04-20T11:27:22.107 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/shm 2024-04-20T11:27:22.107 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/pts 2024-04-20T11:27:22.107 INFO:teuthology.orchestra.run.smithi152.stdout:/run 2024-04-20T11:27:22.107 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/cgroup 2024-04-20T11:27:22.107 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/pstore 2024-04-20T11:27:22.107 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/bpf 2024-04-20T11:27:22.107 INFO:teuthology.orchestra.run.smithi152.stdout:/ 2024-04-20T11:27:22.107 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/selinux 2024-04-20T11:27:22.107 INFO:teuthology.orchestra.run.smithi152.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T11:27:22.107 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/mqueue 2024-04-20T11:27:22.107 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/hugepages 2024-04-20T11:27:22.107 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/debug 2024-04-20T11:27:22.107 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/tracing 2024-04-20T11:27:22.108 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/fuse/connections 2024-04-20T11:27:22.108 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/config 2024-04-20T11:27:22.108 INFO:teuthology.orchestra.run.smithi152.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T11:27:22.108 INFO:teuthology.orchestra.run.smithi152.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T11:27:22.108 INFO:teuthology.orchestra.run.smithi152.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T11:27:22.108 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T11:27:22.108 INFO:teuthology.orchestra.run.smithi152.stdout:/run/user/1000 2024-04-20T11:27:22.108 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph 2024-04-20T11:27:22.108 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph/osd/ceph-0 2024-04-20T11:27:22.108 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph/osd/ceph-1 2024-04-20T11:27:22.108 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph/osd/ceph-2 2024-04-20T11:27:22.108 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph/osd/ceph-3 2024-04-20T11:27:22.108 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns 2024-04-20T11:27:22.108 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T11:27:22.109 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T11:27:22.109 INFO:teuthology.orchestra.run.smithi152.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-20T11:27:22.109 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T11:27:22.109 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T11:27:22.110 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:27:22.110 DEBUG:teuthology.orchestra.run.smithi152:> ls /sys/fs/fuse/connections 2024-04-20T11:27:22.196 INFO:teuthology.orchestra.run.smithi152.stdout:42 2024-04-20T11:27:22.196 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [42] 2024-04-20T11:27:22.197 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2024-04-20T11:27:22.243 DEBUG:teuthology.orchestra.run.smithi152:> sudo modprobe fuse 2024-04-20T11:27:22.307 DEBUG:teuthology.orchestra.run.smithi152:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T11:27:22.349 INFO:teuthology.orchestra.run.smithi152.stdout:/proc 2024-04-20T11:27:22.349 INFO:teuthology.orchestra.run.smithi152.stdout:/sys 2024-04-20T11:27:22.349 INFO:teuthology.orchestra.run.smithi152.stdout:/dev 2024-04-20T11:27:22.349 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/security 2024-04-20T11:27:22.349 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/shm 2024-04-20T11:27:22.349 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/pts 2024-04-20T11:27:22.349 INFO:teuthology.orchestra.run.smithi152.stdout:/run 2024-04-20T11:27:22.350 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/cgroup 2024-04-20T11:27:22.350 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/pstore 2024-04-20T11:27:22.350 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/bpf 2024-04-20T11:27:22.350 INFO:teuthology.orchestra.run.smithi152.stdout:/ 2024-04-20T11:27:22.350 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/selinux 2024-04-20T11:27:22.350 INFO:teuthology.orchestra.run.smithi152.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T11:27:22.350 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/mqueue 2024-04-20T11:27:22.350 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/hugepages 2024-04-20T11:27:22.350 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/debug 2024-04-20T11:27:22.350 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/tracing 2024-04-20T11:27:22.350 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/fuse/connections 2024-04-20T11:27:22.350 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/config 2024-04-20T11:27:22.350 INFO:teuthology.orchestra.run.smithi152.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T11:27:22.350 INFO:teuthology.orchestra.run.smithi152.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T11:27:22.351 INFO:teuthology.orchestra.run.smithi152.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T11:27:22.351 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T11:27:22.351 INFO:teuthology.orchestra.run.smithi152.stdout:/run/user/1000 2024-04-20T11:27:22.351 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph 2024-04-20T11:27:22.351 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph/osd/ceph-0 2024-04-20T11:27:22.351 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph/osd/ceph-1 2024-04-20T11:27:22.351 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph/osd/ceph-2 2024-04-20T11:27:22.351 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph/osd/ceph-3 2024-04-20T11:27:22.351 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns 2024-04-20T11:27:22.351 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T11:27:22.351 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T11:27:22.351 INFO:teuthology.orchestra.run.smithi152.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-20T11:27:22.351 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T11:27:22.352 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T11:27:22.352 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:27:22.352 DEBUG:teuthology.orchestra.run.smithi152:> ls /sys/fs/fuse/connections 2024-04-20T11:27:22.407 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi152.stderr:2024-04-20T11:27:22.408+0000 7f92aee4a480 -1 init, newargv = 0x563bc718d8d0 newargc=15 2024-04-20T11:27:22.407 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi152.stderr:ceph-fuse[31450]: starting ceph client 2024-04-20T11:27:22.418 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi152.stderr:ceph-fuse[31450]: starting fuse 2024-04-20T11:27:22.434 INFO:teuthology.orchestra.run.smithi152.stdout:42 2024-04-20T11:27:22.434 INFO:teuthology.orchestra.run.smithi152.stdout:43 2024-04-20T11:27:22.434 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42, 43] 2024-04-20T11:27:22.434 DEBUG:teuthology.orchestra.run.smithi152:> sudo stdin-killer -- python3 -c ' 2024-04-20T11:27:22.435 DEBUG:teuthology.orchestra.run.smithi152:> import glob 2024-04-20T11:27:22.435 DEBUG:teuthology.orchestra.run.smithi152:> import re 2024-04-20T11:27:22.435 DEBUG:teuthology.orchestra.run.smithi152:> import os 2024-04-20T11:27:22.435 DEBUG:teuthology.orchestra.run.smithi152:> import subprocess 2024-04-20T11:27:22.435 DEBUG:teuthology.orchestra.run.smithi152:> 2024-04-20T11:27:22.435 DEBUG:teuthology.orchestra.run.smithi152:> def _find_admin_socket(client_name): 2024-04-20T11:27:22.435 DEBUG:teuthology.orchestra.run.smithi152:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2024-04-20T11:27:22.435 DEBUG:teuthology.orchestra.run.smithi152:> files = glob.glob(asok_path) 2024-04-20T11:27:22.435 DEBUG:teuthology.orchestra.run.smithi152:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2024-04-20T11:27:22.435 DEBUG:teuthology.orchestra.run.smithi152:> 2024-04-20T11:27:22.435 DEBUG:teuthology.orchestra.run.smithi152:> # Given a non-glob path, it better be there 2024-04-20T11:27:22.435 DEBUG:teuthology.orchestra.run.smithi152:> if "*" not in asok_path: 2024-04-20T11:27:22.435 DEBUG:teuthology.orchestra.run.smithi152:> assert(len(files) == 1) 2024-04-20T11:27:22.435 DEBUG:teuthology.orchestra.run.smithi152:> return files[0] 2024-04-20T11:27:22.435 DEBUG:teuthology.orchestra.run.smithi152:> 2024-04-20T11:27:22.435 DEBUG:teuthology.orchestra.run.smithi152:> for f in files: 2024-04-20T11:27:22.435 DEBUG:teuthology.orchestra.run.smithi152:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-20T11:27:22.436 DEBUG:teuthology.orchestra.run.smithi152:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-20T11:27:22.436 DEBUG:teuthology.orchestra.run.smithi152:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-20T11:27:22.436 DEBUG:teuthology.orchestra.run.smithi152:> contents = proc_f.read() 2024-04-20T11:27:22.436 DEBUG:teuthology.orchestra.run.smithi152:> if mountpoint in contents: 2024-04-20T11:27:22.436 DEBUG:teuthology.orchestra.run.smithi152:> return f 2024-04-20T11:27:22.436 DEBUG:teuthology.orchestra.run.smithi152:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-20T11:27:22.436 DEBUG:teuthology.orchestra.run.smithi152:> 2024-04-20T11:27:22.436 DEBUG:teuthology.orchestra.run.smithi152:> print(_find_admin_socket("client.2")) 2024-04-20T11:27:22.436 DEBUG:teuthology.orchestra.run.smithi152:> ' 2024-04-20T11:27:22.617 INFO:teuthology.orchestra.run.smithi152.stdout:/var/run/ceph/ceph-client.2.31450.asok 2024-04-20T11:27:22.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T11:27:22.633 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.31450.asok 2024-04-20T11:27:22.633 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:27:22.633 DEBUG:teuthology.orchestra.run.smithi152:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.31450.asok status 2024-04-20T11:27:22.839 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-04-20T11:27:22.839 INFO:teuthology.orchestra.run.smithi152.stdout: "metadata": { 2024-04-20T11:27:22.839 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_sha1": "a8c84f5997950fd7d008204839253291d52e6eda", 2024-04-20T11:27:22.839 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", 2024-04-20T11:27:22.839 INFO:teuthology.orchestra.run.smithi152.stdout: "entity_id": "2", 2024-04-20T11:27:22.839 INFO:teuthology.orchestra.run.smithi152.stdout: "hostname": "smithi152", 2024-04-20T11:27:22.839 INFO:teuthology.orchestra.run.smithi152.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2024-04-20T11:27:22.839 INFO:teuthology.orchestra.run.smithi152.stdout: "pid": "31450", 2024-04-20T11:27:22.839 INFO:teuthology.orchestra.run.smithi152.stdout: "root": "/" 2024-04-20T11:27:22.839 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:27:22.839 INFO:teuthology.orchestra.run.smithi152.stdout: "dentry_count": 0, 2024-04-20T11:27:22.840 INFO:teuthology.orchestra.run.smithi152.stdout: "dentry_pinned_count": 0, 2024-04-20T11:27:22.840 INFO:teuthology.orchestra.run.smithi152.stdout: "id": 4662, 2024-04-20T11:27:22.840 INFO:teuthology.orchestra.run.smithi152.stdout: "inst": { 2024-04-20T11:27:22.840 INFO:teuthology.orchestra.run.smithi152.stdout: "name": { 2024-04-20T11:27:22.840 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "client", 2024-04-20T11:27:22.840 INFO:teuthology.orchestra.run.smithi152.stdout: "num": 4662 2024-04-20T11:27:22.840 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:27:22.840 INFO:teuthology.orchestra.run.smithi152.stdout: "addr": { 2024-04-20T11:27:22.840 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "v1", 2024-04-20T11:27:22.840 INFO:teuthology.orchestra.run.smithi152.stdout: "addr": "192.168.0.2:0", 2024-04-20T11:27:22.840 INFO:teuthology.orchestra.run.smithi152.stdout: "nonce": 3133920578 2024-04-20T11:27:22.840 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:27:22.840 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:27:22.840 INFO:teuthology.orchestra.run.smithi152.stdout: "addr": { 2024-04-20T11:27:22.841 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "v1", 2024-04-20T11:27:22.841 INFO:teuthology.orchestra.run.smithi152.stdout: "addr": "192.168.0.2:0", 2024-04-20T11:27:22.841 INFO:teuthology.orchestra.run.smithi152.stdout: "nonce": 3133920578 2024-04-20T11:27:22.841 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:27:22.841 INFO:teuthology.orchestra.run.smithi152.stdout: "inst_str": "client.4662 192.168.0.2:0/3133920578", 2024-04-20T11:27:22.841 INFO:teuthology.orchestra.run.smithi152.stdout: "addr_str": "192.168.0.2:0/3133920578", 2024-04-20T11:27:22.841 INFO:teuthology.orchestra.run.smithi152.stdout: "inode_count": 1, 2024-04-20T11:27:22.841 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_epoch": 7, 2024-04-20T11:27:22.841 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_epoch": 20, 2024-04-20T11:27:22.841 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_epoch_barrier": 0, 2024-04-20T11:27:22.841 INFO:teuthology.orchestra.run.smithi152.stdout: "blocklisted": false, 2024-04-20T11:27:22.841 INFO:teuthology.orchestra.run.smithi152.stdout: "fs_name": "cephfs" 2024-04-20T11:27:22.841 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-04-20T11:27:22.853 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-20T11:27:23.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:23.040+0000 7f4f85400640 1 Processor -- start 2024-04-20T11:27:23.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:23.040+0000 7f4f85400640 1 -- start start 2024-04-20T11:27:23.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:23.042+0000 7f4f85400640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f4f80147410 0x7f4f800afa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:23.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:23.042+0000 7f4f85400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4f8006e720 0x7f4f80072130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:23.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:23.042+0000 7f4f85400640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f800aec90 con 0x7f4f80071cd0 2024-04-20T11:27:23.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:23.042+0000 7f4f85400640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f4f80072670 con 0x7f4f80147410 2024-04-20T11:27:23.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:23.042+0000 7f4f85400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4f800aef70 con 0x7f4f8006e720 2024-04-20T11:27:23.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:23.042+0000 7f4f77400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f4f80071cd0 0x7f4f8006e010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:40588/0 (socket says 172.21.15.152:40588) 2024-04-20T11:27:23.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:23.042+0000 7f4f77400640 1 -- 172.21.15.152:0/2890846626 learned_addr learned my addr 172.21.15.152:0/2890846626 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:27:23.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:23.042+0000 7f4f7f400640 1 -- 172.21.15.152:0/2890846626 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 386762135 0 0) 0x7f4f800aec90 con 0x7f4f80071cd0 2024-04-20T11:27:23.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:23.042+0000 7f4f84a00640 1 --2- 172.21.15.152:0/2890846626 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4f8006e720 0x7f4f80072130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:23.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:23.042+0000 7f4f7fe00640 1 --2- 172.21.15.152:0/2890846626 >> v2:172.21.15.175:3300/0 conn(0x7f4f80147410 0x7f4f800afa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:23.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:23.042+0000 7f4f7f400640 1 -- 172.21.15.152:0/2890846626 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4f64003520 con 0x7f4f80071cd0 2024-04-20T11:27:23.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:23.043+0000 7f4f7f400640 1 -- 172.21.15.152:0/2890846626 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2735063738 0 0) 0x7f4f64003520 con 0x7f4f80071cd0 2024-04-20T11:27:23.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:23.043+0000 7f4f7f400640 1 -- 172.21.15.152:0/2890846626 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4f8006e720 msgr2=0x7f4f80072130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:23.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:23.043+0000 7f4f7f400640 1 --2- 172.21.15.152:0/2890846626 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4f8006e720 0x7f4f80072130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:23.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:23.043+0000 7f4f7f400640 1 -- 172.21.15.152:0/2890846626 >> v2:172.21.15.175:3300/0 conn(0x7f4f80147410 msgr2=0x7f4f800afa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:23.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:23.043+0000 7f4f7f400640 1 --2- 172.21.15.152:0/2890846626 >> v2:172.21.15.175:3300/0 conn(0x7f4f80147410 0x7f4f800afa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:23.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:23.043+0000 7f4f7f400640 1 -- 172.21.15.152:0/2890846626 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f800af250 con 0x7f4f80071cd0 2024-04-20T11:27:23.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:23.043+0000 7f4f7f400640 1 -- 172.21.15.152:0/2890846626 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f4f68001670 con 0x7f4f80071cd0 2024-04-20T11:27:23.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:23.043+0000 7f4f7f400640 1 -- 172.21.15.152:0/2890846626 >> v1:172.21.15.152:6789/0 conn(0x7f4f80071cd0 legacy=0x7f4f8006e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:23.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:23.043+0000 7f4f7f400640 1 --2- 172.21.15.152:0/2890846626 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4f640039b0 0x7f4f64003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:23.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:23.043+0000 7f4f7f400640 1 --2- 172.21.15.152:0/2890846626 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4f8006e720 0x7f4f64004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:23.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:23.043+0000 7f4f7f400640 1 --2- 172.21.15.152:0/2890846626 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4f64004d40 0x7f4f640051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:23.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:23.043+0000 7f4f7f400640 1 -- 172.21.15.152:0/2890846626 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f4f800aef70 con 0x7f4f64004d40 2024-04-20T11:27:23.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:23.043+0000 7f4f7f400640 1 -- 172.21.15.152:0/2890846626 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4f6400baa0 con 0x7f4f640039b0 2024-04-20T11:27:23.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:23.043+0000 7f4f7f400640 1 -- 172.21.15.152:0/2890846626 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4f6400bca0 con 0x7f4f8006e720 2024-04-20T11:27:23.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:23.043+0000 7f4f84a00640 1 --2- 172.21.15.152:0/2890846626 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4f64004d40 0x7f4f640051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:23.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:23.043+0000 7f4f77400640 1 --2- 172.21.15.152:0/2890846626 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4f8006e720 0x7f4f64004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:23.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:23.044+0000 7f4f7fe00640 1 --2- 172.21.15.152:0/2890846626 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4f640039b0 0x7f4f64003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:23.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:23.044+0000 7f4f84a00640 1 -- 172.21.15.152:0/2890846626 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4f8006e720 msgr2=0x7f4f64004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:23.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:23.044+0000 7f4f84a00640 1 --2- 172.21.15.152:0/2890846626 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4f8006e720 0x7f4f64004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:23.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:23.044+0000 7f4f84a00640 1 -- 172.21.15.152:0/2890846626 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4f640039b0 msgr2=0x7f4f64003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:23.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:23.044+0000 7f4f84a00640 1 --2- 172.21.15.152:0/2890846626 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4f640039b0 0x7f4f64003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:23.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:23.044+0000 7f4f84a00640 1 -- 172.21.15.152:0/2890846626 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f64003520 con 0x7f4f64004d40 2024-04-20T11:27:23.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:23.044+0000 7f4f77400640 1 --2- 172.21.15.152:0/2890846626 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4f8006e720 0x7f4f64004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:27:23.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:23.044+0000 7f4f7fe00640 1 --2- 172.21.15.152:0/2890846626 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4f640039b0 0x7f4f64003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:27:23.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:23.045+0000 7f4f84a00640 1 --2- 172.21.15.152:0/2890846626 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4f64004d40 0x7f4f640051c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4f78007360 tx=0x7f4f64005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1460565f304390c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:23.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:23.045+0000 7f4f7f400640 1 -- 172.21.15.152:0/2890846626 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f78039070 con 0x7f4f64004d40 2024-04-20T11:27:23.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:23.045+0000 7f4f7f400640 1 -- 172.21.15.152:0/2890846626 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f7802ccd0 con 0x7f4f64004d40 2024-04-20T11:27:23.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:23.045+0000 7f4f7f400640 1 -- 172.21.15.152:0/2890846626 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f78009be0 con 0x7f4f64004d40 2024-04-20T11:27:23.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:23.045+0000 7f4f85400640 1 -- 172.21.15.152:0/2890846626 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4f64004d40 msgr2=0x7f4f640051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:23.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:23.045+0000 7f4f85400640 1 --2- 172.21.15.152:0/2890846626 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4f64004d40 0x7f4f640051c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4f78007360 tx=0x7f4f64005830 comp rx=0 tx=0).stop 2024-04-20T11:27:23.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:23.045+0000 7f4f84a00640 1 -- 172.21.15.152:0/2890846626 reap_dead start 2024-04-20T11:27:23.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:23.046+0000 7f4f85400640 1 -- 172.21.15.152:0/2890846626 shutdown_connections 2024-04-20T11:27:23.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:23.046+0000 7f4f85400640 1 -- 172.21.15.152:0/2890846626 >> 172.21.15.152:0/2890846626 conn(0x7f4f80067ae0 msgr2=0x7f4f8006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:23.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:23.046+0000 7f4f85400640 1 -- 172.21.15.152:0/2890846626 shutdown_connections 2024-04-20T11:27:23.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:23.046+0000 7f4f85400640 1 -- 172.21.15.152:0/2890846626 wait complete. 2024-04-20T11:27:23.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:23.047+0000 7f4f85400640 1 Processor -- start 2024-04-20T11:27:23.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:23.048+0000 7f4f85400640 1 -- start start 2024-04-20T11:27:23.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:23.048+0000 7f4f85400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4f64004d40 0x7f4f801f6b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:23.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:23.048+0000 7f4f85400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4f80143a40 0x7f4f80143ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:23.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:23.048+0000 7f4f85400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4f8015fcc0 0x7f4f80160170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:23.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:23.048+0000 7f4f85400640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f4f80072130 con 0x7f4f80143a40 2024-04-20T11:27:23.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:23.048+0000 7f4f85400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4f800ae980 con 0x7f4f8015fcc0 2024-04-20T11:27:23.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:23.048+0000 7f4f85400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4f800722b0 con 0x7f4f64004d40 2024-04-20T11:27:23.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:23.049+0000 7f4f77400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4f80143a40 0x7f4f80143ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:23.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:23.049+0000 7f4f77400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4f80143a40 0x7f4f80143ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:58030/0 (socket says 172.21.15.152:58030) 2024-04-20T11:27:23.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:23.049+0000 7f4f77400640 1 -- 172.21.15.152:0/3745091385 learned_addr learned my addr 172.21.15.152:0/3745091385 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:23.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:23.049+0000 7f4f7fe00640 1 --2- 172.21.15.152:0/3745091385 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4f64004d40 0x7f4f801f6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:23.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:23.049+0000 7f4f84a00640 1 --2- 172.21.15.152:0/3745091385 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4f8015fcc0 0x7f4f80160170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:23.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:23.049+0000 7f4f77400640 1 -- 172.21.15.152:0/3745091385 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4f64004d40 msgr2=0x7f4f801f6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:23.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:23.049+0000 7f4f77400640 1 --2- 172.21.15.152:0/3745091385 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4f64004d40 0x7f4f801f6b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:23.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:23.049+0000 7f4f77400640 1 -- 172.21.15.152:0/3745091385 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4f8015fcc0 msgr2=0x7f4f80160170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:23.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:23.049+0000 7f4f77400640 1 --2- 172.21.15.152:0/3745091385 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4f8015fcc0 0x7f4f80160170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:23.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:23.049+0000 7f4f77400640 1 -- 172.21.15.152:0/3745091385 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f80144500 con 0x7f4f80143a40 2024-04-20T11:27:23.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:23.049+0000 7f4f7fe00640 1 --2- 172.21.15.152:0/3745091385 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4f64004d40 0x7f4f801f6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:27:23.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:23.049+0000 7f4f77400640 1 --2- 172.21.15.152:0/3745091385 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4f80143a40 0x7f4f80143ec0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4f6800d8b0 tx=0x7f4f6800dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:23.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:23.050+0000 7f4f7cc00640 1 -- 172.21.15.152:0/3745091385 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f68019070 con 0x7f4f80143a40 2024-04-20T11:27:23.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:23.050+0000 7f4f7cc00640 1 -- 172.21.15.152:0/3745091385 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f680074e0 con 0x7f4f80143a40 2024-04-20T11:27:23.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:23.050+0000 7f4f7cc00640 1 -- 172.21.15.152:0/3745091385 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f680049e0 con 0x7f4f80143a40 2024-04-20T11:27:23.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:23.050+0000 7f4f85400640 1 -- 172.21.15.152:0/3745091385 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f801447e0 con 0x7f4f80143a40 2024-04-20T11:27:23.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:23.050+0000 7f4f85400640 1 -- 172.21.15.152:0/3745091385 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f80144da0 con 0x7f4f80143a40 2024-04-20T11:27:23.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:23.051+0000 7f4f85400640 1 -- 172.21.15.152:0/3745091385 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f80075e60 con 0x7f4f80143a40 2024-04-20T11:27:23.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:23.052+0000 7f4f7cc00640 1 -- 172.21.15.152:0/3745091385 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4f6801c070 con 0x7f4f80143a40 2024-04-20T11:27:23.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:23.053+0000 7f4f7cc00640 1 --2- 172.21.15.152:0/3745091385 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f4f5c06c920 0x7f4f5c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:23.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:23.053+0000 7f4f7cc00640 1 -- 172.21.15.152:0/3745091385 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f4f6808b770 con 0x7f4f80143a40 2024-04-20T11:27:23.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:23.053+0000 7f4f7fe00640 1 --2- 172.21.15.152:0/3745091385 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f4f5c06c920 0x7f4f5c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:23.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:23.054+0000 7f4f7fe00640 1 --2- 172.21.15.152:0/3745091385 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f4f5c06c920 0x7f4f5c06ede0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4f70006fd0 tx=0x7f4f70008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:23.057 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:23.057+0000 7f4f7cc00640 1 -- 172.21.15.152:0/3745091385 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f4f68059b00 con 0x7f4f80143a40 2024-04-20T11:27:23.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:23.283+0000 7f4f85400640 1 -- 172.21.15.152:0/3745091385 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f4f801612d0 con 0x7f4f80143a40 2024-04-20T11:27:23.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:23.284+0000 7f4f7cc00640 1 -- 172.21.15.152:0/3745091385 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+73 (secure 0 0 0) 0x7f4f68059400 con 0x7f4f80143a40 2024-04-20T11:27:23.284 INFO:teuthology.orchestra.run.smithi152.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-20T11:27:23.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:23.287+0000 7f4f85400640 1 -- 172.21.15.152:0/3745091385 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f4f5c06c920 msgr2=0x7f4f5c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:23.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:23.287+0000 7f4f85400640 1 --2- 172.21.15.152:0/3745091385 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f4f5c06c920 0x7f4f5c06ede0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4f70006fd0 tx=0x7f4f70008040 comp rx=0 tx=0).stop 2024-04-20T11:27:23.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:23.287+0000 7f4f85400640 1 -- 172.21.15.152:0/3745091385 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4f80143a40 msgr2=0x7f4f80143ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:23.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:23.287+0000 7f4f85400640 1 --2- 172.21.15.152:0/3745091385 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4f80143a40 0x7f4f80143ec0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4f6800d8b0 tx=0x7f4f6800dd70 comp rx=0 tx=0).stop 2024-04-20T11:27:23.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:23.287+0000 7f4f85400640 1 -- 172.21.15.152:0/3745091385 shutdown_connections 2024-04-20T11:27:23.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:23.287+0000 7f4f85400640 1 --2- 172.21.15.152:0/3745091385 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f4f5c06c920 0x7f4f5c06ede0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:23.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:23.287+0000 7f4f85400640 1 --2- 172.21.15.152:0/3745091385 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4f8015fcc0 0x7f4f80160170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:23.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:23.287+0000 7f4f85400640 1 --2- 172.21.15.152:0/3745091385 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4f80143a40 0x7f4f80143ec0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:23.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:23.287+0000 7f4f85400640 1 --2- 172.21.15.152:0/3745091385 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4f64004d40 0x7f4f801f6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:23.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:23.287+0000 7f4f85400640 1 -- 172.21.15.152:0/3745091385 >> 172.21.15.152:0/3745091385 conn(0x7f4f80067ae0 msgr2=0x7f4f800b3a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:23.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:23.287+0000 7f4f85400640 1 -- 172.21.15.152:0/3745091385 shutdown_connections 2024-04-20T11:27:23.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:23.287+0000 7f4f85400640 1 -- 172.21.15.152:0/3745091385 wait complete. 2024-04-20T11:27:23.303 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-20T11:27:23.303 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-20T11:27:23.303 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi175.front.sepia.ceph.com 2024-04-20T11:27:23.303 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-20T11:27:23.303 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-20T11:27:23.303 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-20T11:27:23.303 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-20T11:27:23.303 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-20T11:27:23.303 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-20T11:27:23.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:27:23.304 DEBUG:teuthology.orchestra.run.smithi175:> ip addr 2024-04-20T11:27:23.361 INFO:teuthology.orchestra.run.smithi175.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-20T11:27:23.361 INFO:teuthology.orchestra.run.smithi175.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-20T11:27:23.361 INFO:teuthology.orchestra.run.smithi175.stdout: inet 127.0.0.1/8 scope host lo 2024-04-20T11:27:23.361 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:27:23.361 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 ::1/128 scope host 2024-04-20T11:27:23.362 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:27:23.362 INFO:teuthology.orchestra.run.smithi175.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T11:27:23.362 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:d9:39:ee brd ff:ff:ff:ff:ff:ff 2024-04-20T11:27:23.362 INFO:teuthology.orchestra.run.smithi175.stdout: altname enp7s0f0 2024-04-20T11:27:23.362 INFO:teuthology.orchestra.run.smithi175.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T11:27:23.362 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:d9:39:ef brd ff:ff:ff:ff:ff:ff 2024-04-20T11:27:23.362 INFO:teuthology.orchestra.run.smithi175.stdout: altname enp7s0f1 2024-04-20T11:27:23.362 INFO:teuthology.orchestra.run.smithi175.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T11:27:23.362 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:88:71:f0 brd ff:ff:ff:ff:ff:ff 2024-04-20T11:27:23.362 INFO:teuthology.orchestra.run.smithi175.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-20T11:27:23.362 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:88:71:f1 brd ff:ff:ff:ff:ff:ff 2024-04-20T11:27:23.362 INFO:teuthology.orchestra.run.smithi175.stdout: inet 172.21.15.175/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-20T11:27:23.362 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft 42738sec preferred_lft 42738sec 2024-04-20T11:27:23.363 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::ec4:7aff:fe88:71f1/64 scope link 2024-04-20T11:27:23.363 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:27:23.363 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-20T11:27:23.364 DEBUG:teuthology.orchestra.run.smithi175:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T11:27:23.364 DEBUG:teuthology.orchestra.run.smithi175:> set -e 2024-04-20T11:27:23.364 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip link add name ceph-brx type bridge 2024-04-20T11:27:23.364 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip addr flush dev ceph-brx 2024-04-20T11:27:23.364 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip link set ceph-brx up 2024-04-20T11:27:23.364 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-20T11:27:23.364 DEBUG:teuthology.orchestra.run.smithi175:> ') 2024-04-20T11:27:23.467 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:27:23 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T11:27:23.824 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:27:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T11:27:23.833 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:27:23.833 DEBUG:teuthology.orchestra.run.smithi175:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-20T11:27:23.895 INFO:teuthology.orchestra.run.smithi175.stdout:1 2024-04-20T11:27:23.898 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:27:23.899 DEBUG:teuthology.orchestra.run.smithi175:> route 2024-04-20T11:27:23.951 INFO:teuthology.orchestra.run.smithi175.stdout:Kernel IP routing table 2024-04-20T11:27:23.951 INFO:teuthology.orchestra.run.smithi175.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-20T11:27:23.951 INFO:teuthology.orchestra.run.smithi175.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-20T11:27:23.951 INFO:teuthology.orchestra.run.smithi175.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-20T11:27:23.951 INFO:teuthology.orchestra.run.smithi175.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-20T11:27:23.951 DEBUG:teuthology.orchestra.run.smithi175:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T11:27:23.952 DEBUG:teuthology.orchestra.run.smithi175:> set -e 2024-04-20T11:27:23.952 DEBUG:teuthology.orchestra.run.smithi175:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-20T11:27:23.952 DEBUG:teuthology.orchestra.run.smithi175:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-20T11:27:23.952 DEBUG:teuthology.orchestra.run.smithi175:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-20T11:27:23.952 DEBUG:teuthology.orchestra.run.smithi175:> ') 2024-04-20T11:27:24.039 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:27:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T11:27:24.739 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:27:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T11:27:24.748 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:27:24.748 DEBUG:teuthology.orchestra.run.smithi175:> ip netns list 2024-04-20T11:27:24.799 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:27:24.799 DEBUG:teuthology.orchestra.run.smithi175:> ip netns list-id 2024-04-20T11:27:24.847 DEBUG:teuthology.orchestra.run.smithi175:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T11:27:24.847 DEBUG:teuthology.orchestra.run.smithi175:> set -e 2024-04-20T11:27:24.847 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-20T11:27:24.847 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-04-20T11:27:24.847 DEBUG:teuthology.orchestra.run.smithi175:> ') 2024-04-20T11:27:24.949 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:27:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T11:27:25.021 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:27:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T11:27:25.031 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-04-20T11:27:25.031 DEBUG:teuthology.orchestra.run.smithi175:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T11:27:25.031 DEBUG:teuthology.orchestra.run.smithi175:> set -e 2024-04-20T11:27:25.031 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-04-20T11:27:25.031 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-20T11:27:25.031 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-04-20T11:27:25.032 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-04-20T11:27:25.032 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-04-20T11:27:25.032 DEBUG:teuthology.orchestra.run.smithi175:> ') 2024-04-20T11:27:25.136 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:27:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T11:27:25.333 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:27:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T11:27:25.342 DEBUG:teuthology.orchestra.run.smithi175:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T11:27:25.342 DEBUG:teuthology.orchestra.run.smithi175:> set -e 2024-04-20T11:27:25.343 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip link set brx.0 up 2024-04-20T11:27:25.343 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip link set dev brx.0 master ceph-brx 2024-04-20T11:27:25.343 DEBUG:teuthology.orchestra.run.smithi175:> ') 2024-04-20T11:27:25.440 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:27:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T11:27:25.537 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:27:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T11:27:25.547 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-04-20T11:27:25.547 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T11:27:25.547 DEBUG:teuthology.orchestra.run.smithi175:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-20T11:27:25.588 INFO:teuthology.orchestra.run.smithi175.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-20T11:27:25.588 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T11:27:25.588 DEBUG:teuthology.orchestra.run.smithi175:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-20T11:27:25.681 DEBUG:teuthology.orchestra.run.smithi175:> sudo modprobe fuse 2024-04-20T11:27:25.758 DEBUG:teuthology.orchestra.run.smithi175:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T11:27:25.809 INFO:teuthology.orchestra.run.smithi175.stdout:/proc 2024-04-20T11:27:25.809 INFO:teuthology.orchestra.run.smithi175.stdout:/sys 2024-04-20T11:27:25.810 INFO:teuthology.orchestra.run.smithi175.stdout:/dev 2024-04-20T11:27:25.810 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/security 2024-04-20T11:27:25.810 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/shm 2024-04-20T11:27:25.810 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/pts 2024-04-20T11:27:25.810 INFO:teuthology.orchestra.run.smithi175.stdout:/run 2024-04-20T11:27:25.810 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/cgroup 2024-04-20T11:27:25.810 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/pstore 2024-04-20T11:27:25.810 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/bpf 2024-04-20T11:27:25.810 INFO:teuthology.orchestra.run.smithi175.stdout:/ 2024-04-20T11:27:25.810 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/selinux 2024-04-20T11:27:25.810 INFO:teuthology.orchestra.run.smithi175.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T11:27:25.810 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/mqueue 2024-04-20T11:27:25.811 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/debug 2024-04-20T11:27:25.811 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/hugepages 2024-04-20T11:27:25.811 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/tracing 2024-04-20T11:27:25.811 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/fuse/connections 2024-04-20T11:27:25.811 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/config 2024-04-20T11:27:25.811 INFO:teuthology.orchestra.run.smithi175.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T11:27:25.811 INFO:teuthology.orchestra.run.smithi175.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T11:27:25.811 INFO:teuthology.orchestra.run.smithi175.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T11:27:25.811 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T11:27:25.811 INFO:teuthology.orchestra.run.smithi175.stdout:/run/user/1000 2024-04-20T11:27:25.811 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph 2024-04-20T11:27:25.811 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-4 2024-04-20T11:27:25.811 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-5 2024-04-20T11:27:25.811 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-6 2024-04-20T11:27:25.811 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-7 2024-04-20T11:27:25.811 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns 2024-04-20T11:27:25.812 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-20T11:27:25.812 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-20T11:27:25.812 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:27:25.813 DEBUG:teuthology.orchestra.run.smithi175:> ls /sys/fs/fuse/connections 2024-04-20T11:27:25.860 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-20T11:27:25.860 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-04-20T11:27:25.865 DEBUG:teuthology.orchestra.run.smithi175:> sudo modprobe fuse 2024-04-20T11:27:25.934 DEBUG:teuthology.orchestra.run.smithi175:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T11:27:25.976 INFO:teuthology.orchestra.run.smithi175.stdout:/proc 2024-04-20T11:27:25.976 INFO:teuthology.orchestra.run.smithi175.stdout:/sys 2024-04-20T11:27:25.976 INFO:teuthology.orchestra.run.smithi175.stdout:/dev 2024-04-20T11:27:25.976 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/security 2024-04-20T11:27:25.976 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/shm 2024-04-20T11:27:25.976 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/pts 2024-04-20T11:27:25.976 INFO:teuthology.orchestra.run.smithi175.stdout:/run 2024-04-20T11:27:25.976 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/cgroup 2024-04-20T11:27:25.976 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/pstore 2024-04-20T11:27:25.977 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/bpf 2024-04-20T11:27:25.977 INFO:teuthology.orchestra.run.smithi175.stdout:/ 2024-04-20T11:27:25.977 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/selinux 2024-04-20T11:27:25.977 INFO:teuthology.orchestra.run.smithi175.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T11:27:25.977 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/mqueue 2024-04-20T11:27:25.977 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/debug 2024-04-20T11:27:25.977 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/hugepages 2024-04-20T11:27:25.977 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/tracing 2024-04-20T11:27:25.977 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/fuse/connections 2024-04-20T11:27:25.977 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/config 2024-04-20T11:27:25.977 INFO:teuthology.orchestra.run.smithi175.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T11:27:25.977 INFO:teuthology.orchestra.run.smithi175.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T11:27:25.977 INFO:teuthology.orchestra.run.smithi175.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T11:27:25.977 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T11:27:25.977 INFO:teuthology.orchestra.run.smithi175.stdout:/run/user/1000 2024-04-20T11:27:25.978 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph 2024-04-20T11:27:25.978 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-4 2024-04-20T11:27:25.978 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-5 2024-04-20T11:27:25.978 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-6 2024-04-20T11:27:25.978 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-7 2024-04-20T11:27:25.978 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns 2024-04-20T11:27:25.978 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-20T11:27:25.978 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-20T11:27:25.979 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:27:25.979 DEBUG:teuthology.orchestra.run.smithi175:> ls /sys/fs/fuse/connections 2024-04-20T11:27:26.052 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi175.stderr:2024-04-20T11:27:26.053+0000 7f3093044480 -1 init, newargv = 0x55cb0b1662c0 newargc=15 2024-04-20T11:27:26.053 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi175.stderr:ceph-fuse[28538]: starting ceph client 2024-04-20T11:27:26.066 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi175.stderr:ceph-fuse[28538]: starting fuse 2024-04-20T11:27:27.163 DEBUG:teuthology.orchestra.run.smithi175:> sudo modprobe fuse 2024-04-20T11:27:27.255 DEBUG:teuthology.orchestra.run.smithi175:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T11:27:27.307 INFO:teuthology.orchestra.run.smithi175.stdout:/proc 2024-04-20T11:27:27.308 INFO:teuthology.orchestra.run.smithi175.stdout:/sys 2024-04-20T11:27:27.308 INFO:teuthology.orchestra.run.smithi175.stdout:/dev 2024-04-20T11:27:27.308 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/security 2024-04-20T11:27:27.308 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/shm 2024-04-20T11:27:27.308 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/pts 2024-04-20T11:27:27.308 INFO:teuthology.orchestra.run.smithi175.stdout:/run 2024-04-20T11:27:27.308 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/cgroup 2024-04-20T11:27:27.308 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/pstore 2024-04-20T11:27:27.308 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/bpf 2024-04-20T11:27:27.308 INFO:teuthology.orchestra.run.smithi175.stdout:/ 2024-04-20T11:27:27.308 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/selinux 2024-04-20T11:27:27.308 INFO:teuthology.orchestra.run.smithi175.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T11:27:27.308 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/mqueue 2024-04-20T11:27:27.308 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/debug 2024-04-20T11:27:27.309 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/hugepages 2024-04-20T11:27:27.309 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/tracing 2024-04-20T11:27:27.309 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/fuse/connections 2024-04-20T11:27:27.309 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/config 2024-04-20T11:27:27.309 INFO:teuthology.orchestra.run.smithi175.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T11:27:27.309 INFO:teuthology.orchestra.run.smithi175.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T11:27:27.309 INFO:teuthology.orchestra.run.smithi175.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T11:27:27.309 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T11:27:27.309 INFO:teuthology.orchestra.run.smithi175.stdout:/run/user/1000 2024-04-20T11:27:27.309 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph 2024-04-20T11:27:27.309 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-4 2024-04-20T11:27:27.309 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-5 2024-04-20T11:27:27.309 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-6 2024-04-20T11:27:27.309 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-7 2024-04-20T11:27:27.310 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns 2024-04-20T11:27:27.310 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-20T11:27:27.310 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-20T11:27:27.310 INFO:teuthology.orchestra.run.smithi175.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-20T11:27:27.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:27:27.311 DEBUG:teuthology.orchestra.run.smithi175:> ls /sys/fs/fuse/connections 2024-04-20T11:27:27.355 INFO:teuthology.orchestra.run.smithi175.stdout:42 2024-04-20T11:27:27.356 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-04-20T11:27:27.356 DEBUG:teuthology.orchestra.run.smithi175:> sudo stdin-killer -- python3 -c ' 2024-04-20T11:27:27.356 DEBUG:teuthology.orchestra.run.smithi175:> import glob 2024-04-20T11:27:27.356 DEBUG:teuthology.orchestra.run.smithi175:> import re 2024-04-20T11:27:27.356 DEBUG:teuthology.orchestra.run.smithi175:> import os 2024-04-20T11:27:27.356 DEBUG:teuthology.orchestra.run.smithi175:> import subprocess 2024-04-20T11:27:27.356 DEBUG:teuthology.orchestra.run.smithi175:> 2024-04-20T11:27:27.356 DEBUG:teuthology.orchestra.run.smithi175:> def _find_admin_socket(client_name): 2024-04-20T11:27:27.356 DEBUG:teuthology.orchestra.run.smithi175:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-04-20T11:27:27.356 DEBUG:teuthology.orchestra.run.smithi175:> files = glob.glob(asok_path) 2024-04-20T11:27:27.356 DEBUG:teuthology.orchestra.run.smithi175:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-04-20T11:27:27.357 DEBUG:teuthology.orchestra.run.smithi175:> 2024-04-20T11:27:27.357 DEBUG:teuthology.orchestra.run.smithi175:> # Given a non-glob path, it better be there 2024-04-20T11:27:27.357 DEBUG:teuthology.orchestra.run.smithi175:> if "*" not in asok_path: 2024-04-20T11:27:27.357 DEBUG:teuthology.orchestra.run.smithi175:> assert(len(files) == 1) 2024-04-20T11:27:27.357 DEBUG:teuthology.orchestra.run.smithi175:> return files[0] 2024-04-20T11:27:27.357 DEBUG:teuthology.orchestra.run.smithi175:> 2024-04-20T11:27:27.357 DEBUG:teuthology.orchestra.run.smithi175:> for f in files: 2024-04-20T11:27:27.357 DEBUG:teuthology.orchestra.run.smithi175:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-20T11:27:27.357 DEBUG:teuthology.orchestra.run.smithi175:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-20T11:27:27.357 DEBUG:teuthology.orchestra.run.smithi175:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-20T11:27:27.357 DEBUG:teuthology.orchestra.run.smithi175:> contents = proc_f.read() 2024-04-20T11:27:27.357 DEBUG:teuthology.orchestra.run.smithi175:> if mountpoint in contents: 2024-04-20T11:27:27.357 DEBUG:teuthology.orchestra.run.smithi175:> return f 2024-04-20T11:27:27.357 DEBUG:teuthology.orchestra.run.smithi175:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-20T11:27:27.357 DEBUG:teuthology.orchestra.run.smithi175:> 2024-04-20T11:27:27.357 DEBUG:teuthology.orchestra.run.smithi175:> print(_find_admin_socket("client.1")) 2024-04-20T11:27:27.357 DEBUG:teuthology.orchestra.run.smithi175:> ' 2024-04-20T11:27:27.566 INFO:teuthology.orchestra.run.smithi175.stdout:/var/run/ceph/ceph-client.1.28538.asok 2024-04-20T11:27:27.570 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:27:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T11:27:27.583 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.28538.asok 2024-04-20T11:27:27.583 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:27:27.583 DEBUG:teuthology.orchestra.run.smithi175:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.28538.asok status 2024-04-20T11:27:27.811 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-04-20T11:27:27.811 INFO:teuthology.orchestra.run.smithi175.stdout: "metadata": { 2024-04-20T11:27:27.811 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_sha1": "a8c84f5997950fd7d008204839253291d52e6eda", 2024-04-20T11:27:27.811 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", 2024-04-20T11:27:27.811 INFO:teuthology.orchestra.run.smithi175.stdout: "entity_id": "1", 2024-04-20T11:27:27.811 INFO:teuthology.orchestra.run.smithi175.stdout: "hostname": "smithi175", 2024-04-20T11:27:27.811 INFO:teuthology.orchestra.run.smithi175.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-04-20T11:27:27.811 INFO:teuthology.orchestra.run.smithi175.stdout: "pid": "28538", 2024-04-20T11:27:27.811 INFO:teuthology.orchestra.run.smithi175.stdout: "root": "/" 2024-04-20T11:27:27.811 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-20T11:27:27.811 INFO:teuthology.orchestra.run.smithi175.stdout: "dentry_count": 0, 2024-04-20T11:27:27.811 INFO:teuthology.orchestra.run.smithi175.stdout: "dentry_pinned_count": 0, 2024-04-20T11:27:27.812 INFO:teuthology.orchestra.run.smithi175.stdout: "id": 4466, 2024-04-20T11:27:27.812 INFO:teuthology.orchestra.run.smithi175.stdout: "inst": { 2024-04-20T11:27:27.812 INFO:teuthology.orchestra.run.smithi175.stdout: "name": { 2024-04-20T11:27:27.812 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "client", 2024-04-20T11:27:27.812 INFO:teuthology.orchestra.run.smithi175.stdout: "num": 4466 2024-04-20T11:27:27.812 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-20T11:27:27.812 INFO:teuthology.orchestra.run.smithi175.stdout: "addr": { 2024-04-20T11:27:27.812 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "v1", 2024-04-20T11:27:27.812 INFO:teuthology.orchestra.run.smithi175.stdout: "addr": "192.168.0.1:0", 2024-04-20T11:27:27.812 INFO:teuthology.orchestra.run.smithi175.stdout: "nonce": 492379544 2024-04-20T11:27:27.812 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-04-20T11:27:27.812 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-20T11:27:27.812 INFO:teuthology.orchestra.run.smithi175.stdout: "addr": { 2024-04-20T11:27:27.812 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "v1", 2024-04-20T11:27:27.812 INFO:teuthology.orchestra.run.smithi175.stdout: "addr": "192.168.0.1:0", 2024-04-20T11:27:27.813 INFO:teuthology.orchestra.run.smithi175.stdout: "nonce": 492379544 2024-04-20T11:27:27.813 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-20T11:27:27.813 INFO:teuthology.orchestra.run.smithi175.stdout: "inst_str": "client.4466 192.168.0.1:0/492379544", 2024-04-20T11:27:27.813 INFO:teuthology.orchestra.run.smithi175.stdout: "addr_str": "192.168.0.1:0/492379544", 2024-04-20T11:27:27.813 INFO:teuthology.orchestra.run.smithi175.stdout: "inode_count": 1, 2024-04-20T11:27:27.813 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_epoch": 7, 2024-04-20T11:27:27.813 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_epoch": 20, 2024-04-20T11:27:27.813 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_epoch_barrier": 0, 2024-04-20T11:27:27.813 INFO:teuthology.orchestra.run.smithi175.stdout: "blocklisted": false, 2024-04-20T11:27:27.813 INFO:teuthology.orchestra.run.smithi175.stdout: "fs_name": "cephfs" 2024-04-20T11:27:27.813 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-04-20T11:27:27.825 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-20T11:27:28.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:28.034+0000 7fc79be00640 1 Processor -- start 2024-04-20T11:27:28.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:28.034+0000 7fc79be00640 1 -- start start 2024-04-20T11:27:28.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:28.035+0000 7fc79be00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fc79c15d1e0 0x7fc79c160b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:28.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:28.035+0000 7fc79be00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc79c15d5c0 0x7fc79c0cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:28.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:28.035+0000 7fc79be00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fc79c071cd0 0x7fc79c072160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:28.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:28.035+0000 7fc79be00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fc79c0726a0 con 0x7fc79c071cd0 2024-04-20T11:27:28.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:28.035+0000 7fc79be00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fc79c0ce8c0 con 0x7fc79c15d1e0 2024-04-20T11:27:28.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:28.035+0000 7fc79be00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc79c1600b0 con 0x7fc79c15d5c0 2024-04-20T11:27:28.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:28.036+0000 7fc79b400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fc79c071cd0 0x7fc79c072160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:28.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:28.036+0000 7fc79b400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fc79c071cd0 0x7fc79c072160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:45056/0 (socket says 172.21.15.152:45056) 2024-04-20T11:27:28.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:28.036+0000 7fc79b400640 1 -- 172.21.15.152:0/3553362450 learned_addr learned my addr 172.21.15.152:0/3553362450 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:28.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:28.036+0000 7fc792000640 1 --2- 172.21.15.152:0/3553362450 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc79c15d5c0 0x7fc79c0cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:28.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:28.036+0000 7fc79aa00640 1 --2- 172.21.15.152:0/3553362450 >> v2:172.21.15.175:3300/0 conn(0x7fc79c15d1e0 0x7fc79c160b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:28.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:28.036+0000 7fc79b400640 1 -- 172.21.15.152:0/3553362450 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc79c15d5c0 msgr2=0x7fc79c0cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:28.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:28.036+0000 7fc79b400640 1 --2- 172.21.15.152:0/3553362450 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc79c15d5c0 0x7fc79c0cdfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:28.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:28.036+0000 7fc79b400640 1 -- 172.21.15.152:0/3553362450 >> v2:172.21.15.175:3300/0 conn(0x7fc79c15d1e0 msgr2=0x7fc79c160b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:28.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:28.036+0000 7fc79b400640 1 --2- 172.21.15.152:0/3553362450 >> v2:172.21.15.175:3300/0 conn(0x7fc79c15d1e0 0x7fc79c160b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:28.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:28.036+0000 7fc79b400640 1 -- 172.21.15.152:0/3553362450 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc79c160390 con 0x7fc79c071cd0 2024-04-20T11:27:28.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:28.037+0000 7fc79b400640 1 --2- 172.21.15.152:0/3553362450 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc79c071cd0 0x7fc79c072160 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc79c06c9b0 tx=0x7fc79402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae726863f8801909 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:28.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:28.038+0000 7fc79a000640 1 -- 172.21.15.152:0/3553362450 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc79402ca70 con 0x7fc79c071cd0 2024-04-20T11:27:28.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:28.038+0000 7fc79a000640 1 -- 172.21.15.152:0/3553362450 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc79402cc10 con 0x7fc79c071cd0 2024-04-20T11:27:28.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:28.038+0000 7fc79a000640 1 -- 172.21.15.152:0/3553362450 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7940343f0 con 0x7fc79c071cd0 2024-04-20T11:27:28.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:28.038+0000 7fc79be00640 1 -- 172.21.15.152:0/3553362450 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc79c071cd0 msgr2=0x7fc79c072160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:28.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:28.038+0000 7fc79be00640 1 --2- 172.21.15.152:0/3553362450 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc79c071cd0 0x7fc79c072160 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc79c06c9b0 tx=0x7fc79402bed0 comp rx=0 tx=0).stop 2024-04-20T11:27:28.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:28.038+0000 7fc79be00640 1 -- 172.21.15.152:0/3553362450 shutdown_connections 2024-04-20T11:27:28.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:28.038+0000 7fc79be00640 1 --2- 172.21.15.152:0/3553362450 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc79c071cd0 0x7fc79c072160 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:28.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:28.038+0000 7fc79be00640 1 --2- 172.21.15.152:0/3553362450 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc79c15d5c0 0x7fc79c0cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:28.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:28.038+0000 7fc79be00640 1 --2- 172.21.15.152:0/3553362450 >> v2:172.21.15.175:3300/0 conn(0x7fc79c15d1e0 0x7fc79c160b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:28.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:28.038+0000 7fc79be00640 1 -- 172.21.15.152:0/3553362450 >> 172.21.15.152:0/3553362450 conn(0x7fc79c067ae0 msgr2=0x7fc79c06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:28.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:28.038+0000 7fc79be00640 1 -- 172.21.15.152:0/3553362450 shutdown_connections 2024-04-20T11:27:28.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:28.039+0000 7fc79be00640 1 -- 172.21.15.152:0/3553362450 wait complete. 2024-04-20T11:27:28.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:28.040+0000 7fc79be00640 1 Processor -- start 2024-04-20T11:27:28.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:28.040+0000 7fc79be00640 1 -- start start 2024-04-20T11:27:28.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:28.041+0000 7fc79be00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc79c071cd0 0x7fc79c1e8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:28.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:28.041+0000 7fc79be00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc79c15d1e0 0x7fc79c14bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:28.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:28.041+0000 7fc79be00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc79c15d5c0 0x7fc79c0b41d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:28.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:28.041+0000 7fc79be00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fc79c0ce520 con 0x7fc79c071cd0 2024-04-20T11:27:28.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:28.041+0000 7fc79be00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc79c0ceba0 con 0x7fc79c15d5c0 2024-04-20T11:27:28.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:28.041+0000 7fc79be00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc79c0ce6a0 con 0x7fc79c15d1e0 2024-04-20T11:27:28.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:28.042+0000 7fc79aa00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc79c071cd0 0x7fc79c1e8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:28.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:28.042+0000 7fc79aa00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc79c071cd0 0x7fc79c1e8990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:45066/0 (socket says 172.21.15.152:45066) 2024-04-20T11:27:28.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:28.042+0000 7fc79aa00640 1 -- 172.21.15.152:0/3476229262 learned_addr learned my addr 172.21.15.152:0/3476229262 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:28.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:28.042+0000 7fc79b400640 1 --2- 172.21.15.152:0/3476229262 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc79c15d5c0 0x7fc79c0b41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:28.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:28.042+0000 7fc79aa00640 1 -- 172.21.15.152:0/3476229262 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc79c15d1e0 msgr2=0x7fc79c14bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:28.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:28.042+0000 7fc79aa00640 1 --2- 172.21.15.152:0/3476229262 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc79c15d1e0 0x7fc79c14bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:28.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:28.042+0000 7fc79aa00640 1 -- 172.21.15.152:0/3476229262 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc79c15d5c0 msgr2=0x7fc79c0b41d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:28.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:28.042+0000 7fc79aa00640 1 --2- 172.21.15.152:0/3476229262 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc79c15d5c0 0x7fc79c0b41d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:28.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:28.042+0000 7fc79aa00640 1 -- 172.21.15.152:0/3476229262 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc79c1e78f0 con 0x7fc79c071cd0 2024-04-20T11:27:28.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:28.042+0000 7fc79aa00640 1 --2- 172.21.15.152:0/3476229262 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc79c071cd0 0x7fc79c1e8990 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc78c00d800 tx=0x7fc78c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:28.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:28.043+0000 7fc793400640 1 -- 172.21.15.152:0/3476229262 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc78c018020 con 0x7fc79c071cd0 2024-04-20T11:27:28.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:28.043+0000 7fc793400640 1 -- 172.21.15.152:0/3476229262 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc78c0096a0 con 0x7fc79c071cd0 2024-04-20T11:27:28.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:28.043+0000 7fc793400640 1 -- 172.21.15.152:0/3476229262 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc78c009a70 con 0x7fc79c071cd0 2024-04-20T11:27:28.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:28.043+0000 7fc79be00640 1 -- 172.21.15.152:0/3476229262 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc79c1e7bd0 con 0x7fc79c071cd0 2024-04-20T11:27:28.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:28.043+0000 7fc79be00640 1 -- 172.21.15.152:0/3476229262 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc79c1e8190 con 0x7fc79c071cd0 2024-04-20T11:27:28.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:28.044+0000 7fc79be00640 1 -- 172.21.15.152:0/3476229262 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc79c0cfa10 con 0x7fc79c071cd0 2024-04-20T11:27:28.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:28.045+0000 7fc793400640 1 -- 172.21.15.152:0/3476229262 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc78c01b020 con 0x7fc79c071cd0 2024-04-20T11:27:28.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:28.046+0000 7fc793400640 1 --2- 172.21.15.152:0/3476229262 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc77806c8f0 0x7fc77806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:28.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:28.046+0000 7fc793400640 1 -- 172.21.15.152:0/3476229262 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc78c08c140 con 0x7fc79c071cd0 2024-04-20T11:27:28.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:28.046+0000 7fc792000640 1 --2- 172.21.15.152:0/3476229262 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc77806c8f0 0x7fc77806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:28.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:28.047+0000 7fc792000640 1 --2- 172.21.15.152:0/3476229262 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc77806c8f0 0x7fc77806edb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc784006fd0 tx=0x7fc784008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:28.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:28.050+0000 7fc793400640 1 -- 172.21.15.152:0/3476229262 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fc79c0cfa10 con 0x7fc79c071cd0 2024-04-20T11:27:28.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:28.293+0000 7fc79be00640 1 -- 172.21.15.152:0/3476229262 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fc79c075e60 con 0x7fc79c071cd0 2024-04-20T11:27:28.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:28.294+0000 7fc793400640 1 -- 172.21.15.152:0/3476229262 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+73 (secure 0 0 0) 0x7fc78c022020 con 0x7fc79c071cd0 2024-04-20T11:27:28.293 INFO:teuthology.orchestra.run.smithi152.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-20T11:27:28.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:28.296+0000 7fc79be00640 1 -- 172.21.15.152:0/3476229262 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc77806c8f0 msgr2=0x7fc77806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:28.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:28.296+0000 7fc79be00640 1 --2- 172.21.15.152:0/3476229262 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc77806c8f0 0x7fc77806edb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc784006fd0 tx=0x7fc784008040 comp rx=0 tx=0).stop 2024-04-20T11:27:28.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:28.296+0000 7fc79be00640 1 -- 172.21.15.152:0/3476229262 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc79c071cd0 msgr2=0x7fc79c1e8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:28.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:28.296+0000 7fc79be00640 1 --2- 172.21.15.152:0/3476229262 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc79c071cd0 0x7fc79c1e8990 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc78c00d800 tx=0x7fc78c00dcc0 comp rx=0 tx=0).stop 2024-04-20T11:27:28.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:28.296+0000 7fc79be00640 1 -- 172.21.15.152:0/3476229262 shutdown_connections 2024-04-20T11:27:28.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:28.297+0000 7fc79be00640 1 --2- 172.21.15.152:0/3476229262 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc77806c8f0 0x7fc77806edb0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:28.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:28.297+0000 7fc79be00640 1 --2- 172.21.15.152:0/3476229262 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc79c15d5c0 0x7fc79c0b41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:28.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:28.297+0000 7fc79be00640 1 --2- 172.21.15.152:0/3476229262 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc79c15d1e0 0x7fc79c14bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:28.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:28.297+0000 7fc79be00640 1 --2- 172.21.15.152:0/3476229262 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc79c071cd0 0x7fc79c1e8990 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:28.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:28.297+0000 7fc79be00640 1 -- 172.21.15.152:0/3476229262 >> 172.21.15.152:0/3476229262 conn(0x7fc79c067ae0 msgr2=0x7fc79c06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:28.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:28.297+0000 7fc79be00640 1 -- 172.21.15.152:0/3476229262 shutdown_connections 2024-04-20T11:27:28.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:28.297+0000 7fc79be00640 1 -- 172.21.15.152:0/3476229262 wait complete. 2024-04-20T11:27:28.315 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-20T11:27:28.315 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-20T11:27:28.315 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi175.front.sepia.ceph.com 2024-04-20T11:27:28.316 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-04-20T11:27:28.316 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-20T11:27:28.316 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-20T11:27:28.316 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-20T11:27:28.316 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-20T11:27:28.316 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-04-20T11:27:28.316 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:27:28.316 DEBUG:teuthology.orchestra.run.smithi175:> ip addr 2024-04-20T11:27:28.373 INFO:teuthology.orchestra.run.smithi175.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-20T11:27:28.373 INFO:teuthology.orchestra.run.smithi175.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-20T11:27:28.373 INFO:teuthology.orchestra.run.smithi175.stdout: inet 127.0.0.1/8 scope host lo 2024-04-20T11:27:28.373 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:27:28.373 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 ::1/128 scope host 2024-04-20T11:27:28.373 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:27:28.373 INFO:teuthology.orchestra.run.smithi175.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T11:27:28.373 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:d9:39:ee brd ff:ff:ff:ff:ff:ff 2024-04-20T11:27:28.373 INFO:teuthology.orchestra.run.smithi175.stdout: altname enp7s0f0 2024-04-20T11:27:28.373 INFO:teuthology.orchestra.run.smithi175.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T11:27:28.374 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:d9:39:ef brd ff:ff:ff:ff:ff:ff 2024-04-20T11:27:28.374 INFO:teuthology.orchestra.run.smithi175.stdout: altname enp7s0f1 2024-04-20T11:27:28.374 INFO:teuthology.orchestra.run.smithi175.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T11:27:28.374 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:88:71:f0 brd ff:ff:ff:ff:ff:ff 2024-04-20T11:27:28.374 INFO:teuthology.orchestra.run.smithi175.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-20T11:27:28.374 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:88:71:f1 brd ff:ff:ff:ff:ff:ff 2024-04-20T11:27:28.374 INFO:teuthology.orchestra.run.smithi175.stdout: inet 172.21.15.175/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-20T11:27:28.374 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft 42733sec preferred_lft 42733sec 2024-04-20T11:27:28.374 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::ec4:7aff:fe88:71f1/64 scope link 2024-04-20T11:27:28.374 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:27:28.374 INFO:teuthology.orchestra.run.smithi175.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-20T11:27:28.374 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 02:d6:c6:31:26:70 brd ff:ff:ff:ff:ff:ff 2024-04-20T11:27:28.374 INFO:teuthology.orchestra.run.smithi175.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-20T11:27:28.374 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:27:28.374 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::5091:d5ff:fee7:54b3/64 scope link 2024-04-20T11:27:28.375 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:27:28.375 INFO:teuthology.orchestra.run.smithi175.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-20T11:27:28.375 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 02:d6:c6:31:26:70 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-20T11:27:28.375 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::d6:c6ff:fe31:2670/64 scope link 2024-04-20T11:27:28.375 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:27:28.376 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:27:28.376 DEBUG:teuthology.orchestra.run.smithi175:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-20T11:27:28.455 INFO:teuthology.orchestra.run.smithi175.stdout:1 2024-04-20T11:27:28.460 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:27:28.460 DEBUG:teuthology.orchestra.run.smithi175:> route 2024-04-20T11:27:28.510 INFO:teuthology.orchestra.run.smithi175.stdout:Kernel IP routing table 2024-04-20T11:27:28.511 INFO:teuthology.orchestra.run.smithi175.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-20T11:27:28.511 INFO:teuthology.orchestra.run.smithi175.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-20T11:27:28.511 INFO:teuthology.orchestra.run.smithi175.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-20T11:27:28.511 INFO:teuthology.orchestra.run.smithi175.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-20T11:27:28.513 DEBUG:teuthology.orchestra.run.smithi175:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T11:27:28.513 DEBUG:teuthology.orchestra.run.smithi175:> set -e 2024-04-20T11:27:28.513 DEBUG:teuthology.orchestra.run.smithi175:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-20T11:27:28.513 DEBUG:teuthology.orchestra.run.smithi175:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-20T11:27:28.514 DEBUG:teuthology.orchestra.run.smithi175:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-20T11:27:28.514 DEBUG:teuthology.orchestra.run.smithi175:> ') 2024-04-20T11:27:28.619 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:27:28 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T11:27:28.713 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:27:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T11:27:28.723 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:27:28.723 DEBUG:teuthology.orchestra.run.smithi175:> ip netns list 2024-04-20T11:27:28.771 INFO:teuthology.orchestra.run.smithi175.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-20T11:27:28.772 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:27:28.772 DEBUG:teuthology.orchestra.run.smithi175:> ip netns list-id 2024-04-20T11:27:28.824 INFO:teuthology.orchestra.run.smithi175.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-04-20T11:27:28.825 DEBUG:teuthology.orchestra.run.smithi175:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T11:27:28.825 DEBUG:teuthology.orchestra.run.smithi175:> set -e 2024-04-20T11:27:28.825 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-20T11:27:28.825 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-04-20T11:27:28.825 DEBUG:teuthology.orchestra.run.smithi175:> ') 2024-04-20T11:27:28.928 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:27:28 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T11:27:28.999 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:27:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T11:27:29.008 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:27:29.009 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-20T11:27:29.081 INFO:teuthology.orchestra.run.smithi175.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-20T11:27:29.081 INFO:teuthology.orchestra.run.smithi175.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-20T11:27:29.081 INFO:teuthology.orchestra.run.smithi175.stdout: inet 127.0.0.1/8 scope host lo 2024-04-20T11:27:29.081 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:27:29.081 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 ::1/128 scope host 2024-04-20T11:27:29.081 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:27:29.081 INFO:teuthology.orchestra.run.smithi175.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-20T11:27:29.082 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 3e:1f:54:03:e2:c3 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-20T11:27:29.082 INFO:teuthology.orchestra.run.smithi175.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-20T11:27:29.082 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:27:29.082 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::3c1f:54ff:fe03:e2c3/64 scope link 2024-04-20T11:27:29.082 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:27:29.084 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:27:29.084 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-20T11:27:29.166 INFO:teuthology.orchestra.run.smithi175.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-20T11:27:29.167 INFO:teuthology.orchestra.run.smithi175.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-20T11:27:29.167 INFO:teuthology.orchestra.run.smithi175.stdout: inet 127.0.0.1/8 scope host lo 2024-04-20T11:27:29.167 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:27:29.167 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 ::1/128 scope host 2024-04-20T11:27:29.167 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:27:29.167 INFO:teuthology.orchestra.run.smithi175.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-20T11:27:29.167 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 3e:1f:54:03:e2:c3 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-20T11:27:29.167 INFO:teuthology.orchestra.run.smithi175.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-20T11:27:29.167 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:27:29.167 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::3c1f:54ff:fe03:e2c3/64 scope link 2024-04-20T11:27:29.168 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:27:29.171 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-04-20T11:27:29.171 DEBUG:teuthology.orchestra.run.smithi175:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T11:27:29.171 DEBUG:teuthology.orchestra.run.smithi175:> set -e 2024-04-20T11:27:29.171 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-04-20T11:27:29.171 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-20T11:27:29.171 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-04-20T11:27:29.171 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-04-20T11:27:29.171 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-04-20T11:27:29.172 DEBUG:teuthology.orchestra.run.smithi175:> ') 2024-04-20T11:27:29.280 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:27:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T11:27:29.424 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:27:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T11:27:29.433 DEBUG:teuthology.orchestra.run.smithi175:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T11:27:29.433 DEBUG:teuthology.orchestra.run.smithi175:> set -e 2024-04-20T11:27:29.433 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip link set brx.1 up 2024-04-20T11:27:29.433 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip link set dev brx.1 master ceph-brx 2024-04-20T11:27:29.433 DEBUG:teuthology.orchestra.run.smithi175:> ') 2024-04-20T11:27:29.527 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:27:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T11:27:29.623 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:27:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T11:27:29.634 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2024-04-20T11:27:29.634 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T11:27:29.634 DEBUG:teuthology.orchestra.run.smithi175:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-04-20T11:27:29.680 INFO:teuthology.orchestra.run.smithi175.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-04-20T11:27:29.680 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T11:27:29.680 DEBUG:teuthology.orchestra.run.smithi175:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-04-20T11:27:29.773 DEBUG:teuthology.orchestra.run.smithi175:> sudo modprobe fuse 2024-04-20T11:27:29.851 DEBUG:teuthology.orchestra.run.smithi175:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T11:27:29.902 INFO:teuthology.orchestra.run.smithi175.stdout:/proc 2024-04-20T11:27:29.902 INFO:teuthology.orchestra.run.smithi175.stdout:/sys 2024-04-20T11:27:29.902 INFO:teuthology.orchestra.run.smithi175.stdout:/dev 2024-04-20T11:27:29.902 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/security 2024-04-20T11:27:29.902 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/shm 2024-04-20T11:27:29.903 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/pts 2024-04-20T11:27:29.903 INFO:teuthology.orchestra.run.smithi175.stdout:/run 2024-04-20T11:27:29.903 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/cgroup 2024-04-20T11:27:29.903 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/pstore 2024-04-20T11:27:29.903 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/bpf 2024-04-20T11:27:29.903 INFO:teuthology.orchestra.run.smithi175.stdout:/ 2024-04-20T11:27:29.903 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/selinux 2024-04-20T11:27:29.903 INFO:teuthology.orchestra.run.smithi175.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T11:27:29.903 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/mqueue 2024-04-20T11:27:29.903 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/debug 2024-04-20T11:27:29.903 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/hugepages 2024-04-20T11:27:29.903 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/tracing 2024-04-20T11:27:29.903 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/fuse/connections 2024-04-20T11:27:29.903 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/config 2024-04-20T11:27:29.903 INFO:teuthology.orchestra.run.smithi175.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T11:27:29.904 INFO:teuthology.orchestra.run.smithi175.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T11:27:29.904 INFO:teuthology.orchestra.run.smithi175.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T11:27:29.904 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T11:27:29.904 INFO:teuthology.orchestra.run.smithi175.stdout:/run/user/1000 2024-04-20T11:27:29.904 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph 2024-04-20T11:27:29.904 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-4 2024-04-20T11:27:29.904 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-5 2024-04-20T11:27:29.904 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-6 2024-04-20T11:27:29.904 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-7 2024-04-20T11:27:29.904 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns 2024-04-20T11:27:29.904 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-20T11:27:29.904 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-20T11:27:29.904 INFO:teuthology.orchestra.run.smithi175.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-20T11:27:29.904 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-20T11:27:29.904 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-20T11:27:29.905 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:27:29.905 DEBUG:teuthology.orchestra.run.smithi175:> ls /sys/fs/fuse/connections 2024-04-20T11:27:29.952 INFO:teuthology.orchestra.run.smithi175.stdout:42 2024-04-20T11:27:29.953 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [42] 2024-04-20T11:27:29.953 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2024-04-20T11:27:30.000 DEBUG:teuthology.orchestra.run.smithi175:> sudo modprobe fuse 2024-04-20T11:27:30.070 DEBUG:teuthology.orchestra.run.smithi175:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T11:27:30.111 INFO:teuthology.orchestra.run.smithi175.stdout:/proc 2024-04-20T11:27:30.111 INFO:teuthology.orchestra.run.smithi175.stdout:/sys 2024-04-20T11:27:30.111 INFO:teuthology.orchestra.run.smithi175.stdout:/dev 2024-04-20T11:27:30.111 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/security 2024-04-20T11:27:30.111 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/shm 2024-04-20T11:27:30.112 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/pts 2024-04-20T11:27:30.112 INFO:teuthology.orchestra.run.smithi175.stdout:/run 2024-04-20T11:27:30.112 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/cgroup 2024-04-20T11:27:30.112 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/pstore 2024-04-20T11:27:30.112 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/bpf 2024-04-20T11:27:30.112 INFO:teuthology.orchestra.run.smithi175.stdout:/ 2024-04-20T11:27:30.112 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/selinux 2024-04-20T11:27:30.112 INFO:teuthology.orchestra.run.smithi175.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T11:27:30.112 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/mqueue 2024-04-20T11:27:30.112 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/debug 2024-04-20T11:27:30.112 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/hugepages 2024-04-20T11:27:30.112 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/tracing 2024-04-20T11:27:30.112 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/fuse/connections 2024-04-20T11:27:30.112 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/config 2024-04-20T11:27:30.113 INFO:teuthology.orchestra.run.smithi175.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T11:27:30.113 INFO:teuthology.orchestra.run.smithi175.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T11:27:30.113 INFO:teuthology.orchestra.run.smithi175.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T11:27:30.113 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T11:27:30.113 INFO:teuthology.orchestra.run.smithi175.stdout:/run/user/1000 2024-04-20T11:27:30.113 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph 2024-04-20T11:27:30.113 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-4 2024-04-20T11:27:30.113 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-5 2024-04-20T11:27:30.113 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-6 2024-04-20T11:27:30.113 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-7 2024-04-20T11:27:30.113 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns 2024-04-20T11:27:30.113 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-20T11:27:30.113 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-20T11:27:30.113 INFO:teuthology.orchestra.run.smithi175.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-20T11:27:30.114 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-20T11:27:30.114 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-20T11:27:30.114 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:27:30.115 DEBUG:teuthology.orchestra.run.smithi175:> ls /sys/fs/fuse/connections 2024-04-20T11:27:30.173 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi175.stderr:2024-04-20T11:27:30.174+0000 7f6970c0c480 -1 init, newargv = 0x555875520e50 newargc=15 2024-04-20T11:27:30.173 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi175.stderr:ceph-fuse[29062]: starting ceph client 2024-04-20T11:27:30.184 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi175.stderr:ceph-fuse[29062]: starting fuse 2024-04-20T11:27:30.207 INFO:teuthology.orchestra.run.smithi175.stdout:42 2024-04-20T11:27:30.207 INFO:teuthology.orchestra.run.smithi175.stdout:43 2024-04-20T11:27:30.208 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42, 43] 2024-04-20T11:27:30.208 DEBUG:teuthology.orchestra.run.smithi175:> sudo stdin-killer -- python3 -c ' 2024-04-20T11:27:30.208 DEBUG:teuthology.orchestra.run.smithi175:> import glob 2024-04-20T11:27:30.208 DEBUG:teuthology.orchestra.run.smithi175:> import re 2024-04-20T11:27:30.208 DEBUG:teuthology.orchestra.run.smithi175:> import os 2024-04-20T11:27:30.208 DEBUG:teuthology.orchestra.run.smithi175:> import subprocess 2024-04-20T11:27:30.208 DEBUG:teuthology.orchestra.run.smithi175:> 2024-04-20T11:27:30.208 DEBUG:teuthology.orchestra.run.smithi175:> def _find_admin_socket(client_name): 2024-04-20T11:27:30.208 DEBUG:teuthology.orchestra.run.smithi175:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2024-04-20T11:27:30.208 DEBUG:teuthology.orchestra.run.smithi175:> files = glob.glob(asok_path) 2024-04-20T11:27:30.208 DEBUG:teuthology.orchestra.run.smithi175:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2024-04-20T11:27:30.209 DEBUG:teuthology.orchestra.run.smithi175:> 2024-04-20T11:27:30.209 DEBUG:teuthology.orchestra.run.smithi175:> # Given a non-glob path, it better be there 2024-04-20T11:27:30.209 DEBUG:teuthology.orchestra.run.smithi175:> if "*" not in asok_path: 2024-04-20T11:27:30.209 DEBUG:teuthology.orchestra.run.smithi175:> assert(len(files) == 1) 2024-04-20T11:27:30.209 DEBUG:teuthology.orchestra.run.smithi175:> return files[0] 2024-04-20T11:27:30.209 DEBUG:teuthology.orchestra.run.smithi175:> 2024-04-20T11:27:30.209 DEBUG:teuthology.orchestra.run.smithi175:> for f in files: 2024-04-20T11:27:30.209 DEBUG:teuthology.orchestra.run.smithi175:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-20T11:27:30.209 DEBUG:teuthology.orchestra.run.smithi175:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-20T11:27:30.209 DEBUG:teuthology.orchestra.run.smithi175:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-20T11:27:30.209 DEBUG:teuthology.orchestra.run.smithi175:> contents = proc_f.read() 2024-04-20T11:27:30.210 DEBUG:teuthology.orchestra.run.smithi175:> if mountpoint in contents: 2024-04-20T11:27:30.210 DEBUG:teuthology.orchestra.run.smithi175:> return f 2024-04-20T11:27:30.210 DEBUG:teuthology.orchestra.run.smithi175:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-20T11:27:30.210 DEBUG:teuthology.orchestra.run.smithi175:> 2024-04-20T11:27:30.210 DEBUG:teuthology.orchestra.run.smithi175:> print(_find_admin_socket("client.3")) 2024-04-20T11:27:30.210 DEBUG:teuthology.orchestra.run.smithi175:> ' 2024-04-20T11:27:30.409 INFO:teuthology.orchestra.run.smithi175.stdout:/var/run/ceph/ceph-client.3.29062.asok 2024-04-20T11:27:30.413 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:27:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T11:27:30.425 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.29062.asok 2024-04-20T11:27:30.425 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:27:30.425 DEBUG:teuthology.orchestra.run.smithi175:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.29062.asok status 2024-04-20T11:27:30.650 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-04-20T11:27:30.650 INFO:teuthology.orchestra.run.smithi175.stdout: "metadata": { 2024-04-20T11:27:30.650 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_sha1": "a8c84f5997950fd7d008204839253291d52e6eda", 2024-04-20T11:27:30.650 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", 2024-04-20T11:27:30.650 INFO:teuthology.orchestra.run.smithi175.stdout: "entity_id": "3", 2024-04-20T11:27:30.650 INFO:teuthology.orchestra.run.smithi175.stdout: "hostname": "smithi175", 2024-04-20T11:27:30.650 INFO:teuthology.orchestra.run.smithi175.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2024-04-20T11:27:30.650 INFO:teuthology.orchestra.run.smithi175.stdout: "pid": "29062", 2024-04-20T11:27:30.650 INFO:teuthology.orchestra.run.smithi175.stdout: "root": "/" 2024-04-20T11:27:30.650 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-20T11:27:30.650 INFO:teuthology.orchestra.run.smithi175.stdout: "dentry_count": 0, 2024-04-20T11:27:30.650 INFO:teuthology.orchestra.run.smithi175.stdout: "dentry_pinned_count": 0, 2024-04-20T11:27:30.651 INFO:teuthology.orchestra.run.smithi175.stdout: "id": 4472, 2024-04-20T11:27:30.651 INFO:teuthology.orchestra.run.smithi175.stdout: "inst": { 2024-04-20T11:27:30.651 INFO:teuthology.orchestra.run.smithi175.stdout: "name": { 2024-04-20T11:27:30.651 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "client", 2024-04-20T11:27:30.651 INFO:teuthology.orchestra.run.smithi175.stdout: "num": 4472 2024-04-20T11:27:30.651 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-20T11:27:30.651 INFO:teuthology.orchestra.run.smithi175.stdout: "addr": { 2024-04-20T11:27:30.651 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "v1", 2024-04-20T11:27:30.651 INFO:teuthology.orchestra.run.smithi175.stdout: "addr": "192.168.0.2:0", 2024-04-20T11:27:30.651 INFO:teuthology.orchestra.run.smithi175.stdout: "nonce": 3944666185 2024-04-20T11:27:30.651 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-04-20T11:27:30.651 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-20T11:27:30.652 INFO:teuthology.orchestra.run.smithi175.stdout: "addr": { 2024-04-20T11:27:30.652 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "v1", 2024-04-20T11:27:30.652 INFO:teuthology.orchestra.run.smithi175.stdout: "addr": "192.168.0.2:0", 2024-04-20T11:27:30.652 INFO:teuthology.orchestra.run.smithi175.stdout: "nonce": 3944666185 2024-04-20T11:27:30.652 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-20T11:27:30.652 INFO:teuthology.orchestra.run.smithi175.stdout: "inst_str": "client.4472 192.168.0.2:0/3944666185", 2024-04-20T11:27:30.652 INFO:teuthology.orchestra.run.smithi175.stdout: "addr_str": "192.168.0.2:0/3944666185", 2024-04-20T11:27:30.652 INFO:teuthology.orchestra.run.smithi175.stdout: "inode_count": 1, 2024-04-20T11:27:30.652 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_epoch": 7, 2024-04-20T11:27:30.652 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_epoch": 20, 2024-04-20T11:27:30.652 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_epoch_barrier": 0, 2024-04-20T11:27:30.652 INFO:teuthology.orchestra.run.smithi175.stdout: "blocklisted": false, 2024-04-20T11:27:30.652 INFO:teuthology.orchestra.run.smithi175.stdout: "fs_name": "cephfs" 2024-04-20T11:27:30.652 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-04-20T11:27:30.665 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:27:30.665 DEBUG:teuthology.orchestra.run.smithi152:> stat --file-system '--printf=%T 2024-04-20T11:27:30.665 DEBUG:teuthology.orchestra.run.smithi152:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-20T11:27:30.725 INFO:teuthology.orchestra.run.smithi152.stdout:fuseblk 2024-04-20T11:27:30.725 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-20T11:27:30.725 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:27:30.725 DEBUG:teuthology.orchestra.run.smithi152:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-20T11:27:30.807 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:27:30.807 DEBUG:teuthology.orchestra.run.smithi152:> stat --file-system '--printf=%T 2024-04-20T11:27:30.807 DEBUG:teuthology.orchestra.run.smithi152:> ' -- /home/ubuntu/cephtest/mnt.2 2024-04-20T11:27:30.862 INFO:teuthology.orchestra.run.smithi152.stdout:fuseblk 2024-04-20T11:27:30.863 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2024-04-20T11:27:30.863 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:27:30.863 DEBUG:teuthology.orchestra.run.smithi152:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-04-20T11:27:30.951 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:27:30.951 DEBUG:teuthology.orchestra.run.smithi175:> stat --file-system '--printf=%T 2024-04-20T11:27:30.952 DEBUG:teuthology.orchestra.run.smithi175:> ' -- /home/ubuntu/cephtest/mnt.1 2024-04-20T11:27:31.010 INFO:teuthology.orchestra.run.smithi175.stdout:fuseblk 2024-04-20T11:27:31.010 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-04-20T11:27:31.011 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:27:31.011 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-20T11:27:31.096 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:27:31.096 DEBUG:teuthology.orchestra.run.smithi175:> stat --file-system '--printf=%T 2024-04-20T11:27:31.096 DEBUG:teuthology.orchestra.run.smithi175:> ' -- /home/ubuntu/cephtest/mnt.3 2024-04-20T11:27:31.144 INFO:teuthology.orchestra.run.smithi175.stdout:fuseblk 2024-04-20T11:27:31.144 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2024-04-20T11:27:31.144 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:27:31.145 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-04-20T11:27:31.219 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-04-20T11:27:31.234 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:27:31.445 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.445+0000 7effc5400640 1 Processor -- start 2024-04-20T11:27:31.445 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.445+0000 7effc5400640 1 -- start start 2024-04-20T11:27:31.447 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.447+0000 7effc5400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7effc015e890 0x7effc00d27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:31.447 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.447+0000 7effc5400640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7effc00d2d20 0x7effc00b7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:31.447 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.447+0000 7effc5400640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7effc015ec70 con 0x7effc00b7cc0 2024-04-20T11:27:31.447 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.447+0000 7effc5400640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7effc00b8e20 con 0x7effc015e890 2024-04-20T11:27:31.447 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.447+0000 7effc5400640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7effc00b8210 con 0x7effc00d2d20 2024-04-20T11:27:31.447 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.447+0000 7effbe800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7effc015e890 0x7effc00d27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:31.447 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.447+0000 7effbe800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7effc015e890 0x7effc00d27e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:45074/0 (socket says 172.21.15.152:45074) 2024-04-20T11:27:31.447 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.447+0000 7effbe800640 1 -- 172.21.15.152:0/2648944936 learned_addr learned my addr 172.21.15.152:0/2648944936 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:31.448 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.448+0000 7effb5e00640 1 --2- 172.21.15.152:0/2648944936 >> v2:172.21.15.175:3300/0 conn(0x7effc00d2d20 0x7effc00b7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:31.448 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.448+0000 7effbe800640 1 -- 172.21.15.152:0/2648944936 >> v2:172.21.15.175:3300/0 conn(0x7effc00d2d20 msgr2=0x7effc00b7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:31.448 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.448+0000 7effbe800640 1 --2- 172.21.15.152:0/2648944936 >> v2:172.21.15.175:3300/0 conn(0x7effc00d2d20 0x7effc00b7780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:31.448 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.448+0000 7effbe800640 1 -- 172.21.15.152:0/2648944936 >> v1:172.21.15.175:6789/0 conn(0x7effc00b7cc0 legacy=0x7effc00d1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:31.448 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.448+0000 7effbe800640 1 -- 172.21.15.152:0/2648944936 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effc00b84f0 con 0x7effc015e890 2024-04-20T11:27:31.449 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.449+0000 7effbe800640 1 --2- 172.21.15.152:0/2648944936 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7effc015e890 0x7effc00d27e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7effc00d2520 tx=0x7effb002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70685f58e6f7d646 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:31.449 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.449+0000 7effbde00640 1 -- 172.21.15.152:0/2648944936 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effb0034070 con 0x7effc015e890 2024-04-20T11:27:31.449 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.449+0000 7effbde00640 1 -- 172.21.15.152:0/2648944936 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effb0030860 con 0x7effc015e890 2024-04-20T11:27:31.449 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.449+0000 7effbde00640 1 -- 172.21.15.152:0/2648944936 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effb0030c30 con 0x7effc015e890 2024-04-20T11:27:31.449 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.450+0000 7effc5400640 1 -- 172.21.15.152:0/2648944936 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7effc015e890 msgr2=0x7effc00d27e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:31.450 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.450+0000 7effc5400640 1 --2- 172.21.15.152:0/2648944936 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7effc015e890 0x7effc00d27e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7effc00d2520 tx=0x7effb002b9a0 comp rx=0 tx=0).stop 2024-04-20T11:27:31.450 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.450+0000 7effc5400640 1 -- 172.21.15.152:0/2648944936 shutdown_connections 2024-04-20T11:27:31.450 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.450+0000 7effc5400640 1 --2- 172.21.15.152:0/2648944936 >> v2:172.21.15.175:3300/0 conn(0x7effc00d2d20 0x7effc00b7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:31.450 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.450+0000 7effc5400640 1 --2- 172.21.15.152:0/2648944936 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7effc015e890 0x7effc00d27e0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:31.450 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.450+0000 7effc5400640 1 -- 172.21.15.152:0/2648944936 >> 172.21.15.152:0/2648944936 conn(0x7effc0067ae0 msgr2=0x7effc0067ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:31.450 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.450+0000 7effc5400640 1 -- 172.21.15.152:0/2648944936 shutdown_connections 2024-04-20T11:27:31.450 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.450+0000 7effc5400640 1 -- 172.21.15.152:0/2648944936 wait complete. 2024-04-20T11:27:31.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.452+0000 7effc5400640 1 Processor -- start 2024-04-20T11:27:31.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.452+0000 7effc5400640 1 -- start start 2024-04-20T11:27:31.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.453+0000 7effc5400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7effc00b7cc0 0x7effc00c8f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:31.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.453+0000 7effc5400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7effc015e890 0x7effc01f7a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:31.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.453+0000 7effc5400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7effc00c9480 0x7effc01f7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:31.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.453+0000 7effc5400640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7effc00d1ea0 con 0x7effc00c9480 2024-04-20T11:27:31.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.453+0000 7effc5400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7effc00b8ab0 con 0x7effc015e890 2024-04-20T11:27:31.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.453+0000 7effc5400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7effc00d2020 con 0x7effc00b7cc0 2024-04-20T11:27:31.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.453+0000 7effbf200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7effc00c9480 0x7effc01f7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:31.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.453+0000 7effbf200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7effc00c9480 0x7effc01f7fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:45078/0 (socket says 172.21.15.152:45078) 2024-04-20T11:27:31.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.453+0000 7effbf200640 1 -- 172.21.15.152:0/2330877912 learned_addr learned my addr 172.21.15.152:0/2330877912 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:31.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.454+0000 7effb5e00640 1 --2- 172.21.15.152:0/2330877912 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7effc015e890 0x7effc01f7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:31.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.454+0000 7effbe800640 1 --2- 172.21.15.152:0/2330877912 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7effc00b7cc0 0x7effc00c8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:31.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.454+0000 7effbf200640 1 -- 172.21.15.152:0/2330877912 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7effc00b7cc0 msgr2=0x7effc00c8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:31.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.454+0000 7effbf200640 1 --2- 172.21.15.152:0/2330877912 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7effc00b7cc0 0x7effc00c8f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:31.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.454+0000 7effbf200640 1 -- 172.21.15.152:0/2330877912 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7effc015e890 msgr2=0x7effc01f7a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:31.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.454+0000 7effbf200640 1 --2- 172.21.15.152:0/2330877912 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7effc015e890 0x7effc01f7a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:31.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.454+0000 7effbf200640 1 -- 172.21.15.152:0/2330877912 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effc01f1e60 con 0x7effc00c9480 2024-04-20T11:27:31.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.454+0000 7effbf200640 1 --2- 172.21.15.152:0/2330877912 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7effc00c9480 0x7effc01f7fd0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7effb800c7b0 tx=0x7effb800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:31.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.454+0000 7effb7400640 1 -- 172.21.15.152:0/2330877912 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effb8017070 con 0x7effc00c9480 2024-04-20T11:27:31.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.454+0000 7effb7400640 1 -- 172.21.15.152:0/2330877912 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effb8003da0 con 0x7effc00c9480 2024-04-20T11:27:31.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.454+0000 7effb7400640 1 -- 172.21.15.152:0/2330877912 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effb80052b0 con 0x7effc00c9480 2024-04-20T11:27:31.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.454+0000 7effc5400640 1 -- 172.21.15.152:0/2330877912 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effc015c4f0 con 0x7effc00c9480 2024-04-20T11:27:31.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.455+0000 7effc5400640 1 -- 172.21.15.152:0/2330877912 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effc015c770 con 0x7effc00c9480 2024-04-20T11:27:31.456 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.456+0000 7effc5400640 1 -- 172.21.15.152:0/2330877912 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effc00689f0 con 0x7effc00c9480 2024-04-20T11:27:31.456 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.457+0000 7effb7400640 1 -- 172.21.15.152:0/2330877912 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7effb8005450 con 0x7effc00c9480 2024-04-20T11:27:31.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.458+0000 7effb7400640 1 --2- 172.21.15.152:0/2330877912 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7eff9c06c8f0 0x7eff9c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:31.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.458+0000 7effb7400640 1 -- 172.21.15.152:0/2330877912 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7effb808b150 con 0x7effc00c9480 2024-04-20T11:27:31.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.459+0000 7effbe800640 1 --2- 172.21.15.152:0/2330877912 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7eff9c06c8f0 0x7eff9c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:31.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.459+0000 7effbe800640 1 --2- 172.21.15.152:0/2330877912 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7eff9c06c8f0 0x7eff9c06edb0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7effb0036000 tx=0x7effb003b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:31.462 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.463+0000 7effb7400640 1 -- 172.21.15.152:0/2330877912 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7effc015c4f0 con 0x7effc00c9480 2024-04-20T11:27:31.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.663+0000 7effc5400640 1 -- 172.21.15.152:0/2330877912 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7effc00cf520 con 0x7effc00c9480 2024-04-20T11:27:31.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.666+0000 7effb7400640 1 -- 172.21.15.152:0/2330877912 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16665 (secure 0 0 0) 0x7effb8059570 con 0x7effc00c9480 2024-04-20T11:27:31.666 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:27:31.666 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":20,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:26:59.675971+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:26:56.261424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T11:26:57.079286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:27:31.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.668+0000 7effc5400640 1 -- 172.21.15.152:0/2330877912 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7eff9c06c8f0 msgr2=0x7eff9c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:31.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.668+0000 7effc5400640 1 --2- 172.21.15.152:0/2330877912 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7eff9c06c8f0 0x7eff9c06edb0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7effb0036000 tx=0x7effb003b040 comp rx=0 tx=0).stop 2024-04-20T11:27:31.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.668+0000 7effc5400640 1 -- 172.21.15.152:0/2330877912 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7effc00c9480 msgr2=0x7effc01f7fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:31.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.668+0000 7effc5400640 1 --2- 172.21.15.152:0/2330877912 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7effc00c9480 0x7effc01f7fd0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7effb800c7b0 tx=0x7effb800cc70 comp rx=0 tx=0).stop 2024-04-20T11:27:31.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.669+0000 7effc5400640 1 -- 172.21.15.152:0/2330877912 shutdown_connections 2024-04-20T11:27:31.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.669+0000 7effc5400640 1 --2- 172.21.15.152:0/2330877912 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7eff9c06c8f0 0x7eff9c06edb0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:31.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.669+0000 7effc5400640 1 --2- 172.21.15.152:0/2330877912 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7effc00c9480 0x7effc01f7fd0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:31.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.669+0000 7effc5400640 1 --2- 172.21.15.152:0/2330877912 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7effc015e890 0x7effc01f7a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:31.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.669+0000 7effc5400640 1 --2- 172.21.15.152:0/2330877912 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7effc00b7cc0 0x7effc00c8f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:31.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.669+0000 7effc5400640 1 -- 172.21.15.152:0/2330877912 >> 172.21.15.152:0/2330877912 conn(0x7effc0067ae0 msgr2=0x7effc00b55b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:31.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.669+0000 7effc5400640 1 -- 172.21.15.152:0/2330877912 shutdown_connections 2024-04-20T11:27:31.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.669+0000 7effc5400640 1 -- 172.21.15.152:0/2330877912 wait complete. 2024-04-20T11:27:31.688 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T11:26:47.246571+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T11:26:56.261424+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T11:26:57.079286+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T11:27:31.689 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T11:27:31.891 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.891+0000 7f468c600640 1 Processor -- start 2024-04-20T11:27:31.891 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.891+0000 7f468c600640 1 -- start start 2024-04-20T11:27:31.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.893+0000 7f468c600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f46840724c0 0x7f46840b3c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:31.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.893+0000 7f468c600640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f468415b6a0 con 0x7f468415c050 2024-04-20T11:27:31.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.893+0000 7f468c600640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46840b4800 con 0x7f46840b4160 2024-04-20T11:27:31.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.893+0000 7f468c600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f46840b45d0 con 0x7f46840724c0 2024-04-20T11:27:31.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.893+0000 7f4689c00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f468415c050 0x7f4684071db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:44698/0 (socket says 172.21.15.152:44698) 2024-04-20T11:27:31.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.893+0000 7f4689c00640 1 -- 172.21.15.152:0/3807869343 learned_addr learned my addr 172.21.15.152:0/3807869343 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:27:31.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.894+0000 7f4683e00640 1 -- 172.21.15.152:0/3807869343 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2252002940 0 0) 0x7f4678001110 con 0x7f468415c050 2024-04-20T11:27:31.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.894+0000 7f4689200640 1 --2- 172.21.15.152:0/3807869343 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f46840724c0 0x7f46840b3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:31.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.894+0000 7f4683e00640 1 -- 172.21.15.152:0/3807869343 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f466c005730 con 0x7f468415c050 2024-04-20T11:27:31.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.894+0000 7f4683e00640 1 -- 172.21.15.152:0/3807869343 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2577473470 0 0) 0x7f466c005730 con 0x7f468415c050 2024-04-20T11:27:31.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.894+0000 7f4683e00640 1 -- 172.21.15.152:0/3807869343 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f46840724c0 msgr2=0x7f46840b3c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:31.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.894+0000 7f4683e00640 1 --2- 172.21.15.152:0/3807869343 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f46840724c0 0x7f46840b3c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:31.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.894+0000 7f4683e00640 1 -- 172.21.15.152:0/3807869343 >> v1:172.21.15.175:6789/0 conn(0x7f46840b4160 legacy=0x7f468406e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:31.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.894+0000 7f4683e00640 1 -- 172.21.15.152:0/3807869343 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46840b4ae0 con 0x7f468415c050 2024-04-20T11:27:31.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.894+0000 7f4683e00640 1 -- 172.21.15.152:0/3807869343 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f4678001840 con 0x7f468415c050 2024-04-20T11:27:31.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.894+0000 7f4683e00640 1 -- 172.21.15.152:0/3807869343 >> v1:172.21.15.152:6789/0 conn(0x7f468415c050 legacy=0x7f4684071db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:31.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.894+0000 7f4683e00640 1 --2- 172.21.15.152:0/3807869343 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f466c005bf0 0x7f466c006010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:31.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.894+0000 7f4683e00640 1 --2- 172.21.15.152:0/3807869343 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f46840724c0 0x7f466c006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:31.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.894+0000 7f4683e00640 1 --2- 172.21.15.152:0/3807869343 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f466c006f50 0x7f466c0073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:31.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.894+0000 7f4683e00640 1 -- 172.21.15.152:0/3807869343 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f466c00de20 con 0x7f466c006f50 2024-04-20T11:27:31.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.894+0000 7f4683e00640 1 -- 172.21.15.152:0/3807869343 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f466c00dfd0 con 0x7f466c005bf0 2024-04-20T11:27:31.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.894+0000 7f4683e00640 1 -- 172.21.15.152:0/3807869343 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f466c00e1d0 con 0x7f46840724c0 2024-04-20T11:27:31.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.895+0000 7f468a600640 1 --2- 172.21.15.152:0/3807869343 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f466c006f50 0x7f466c0073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:31.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.895+0000 7f4689c00640 1 --2- 172.21.15.152:0/3807869343 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f466c005bf0 0x7f466c006010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:31.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.895+0000 7f4689200640 1 --2- 172.21.15.152:0/3807869343 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f46840724c0 0x7f466c006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:31.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.895+0000 7f468a600640 1 -- 172.21.15.152:0/3807869343 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f46840724c0 msgr2=0x7f466c006a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:31.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.895+0000 7f468a600640 1 --2- 172.21.15.152:0/3807869343 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f46840724c0 0x7f466c006a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:31.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.895+0000 7f468a600640 1 -- 172.21.15.152:0/3807869343 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f466c005bf0 msgr2=0x7f466c006010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:31.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.895+0000 7f468a600640 1 --2- 172.21.15.152:0/3807869343 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f466c005bf0 0x7f466c006010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:31.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.895+0000 7f468a600640 1 -- 172.21.15.152:0/3807869343 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f466c005730 con 0x7f466c006f50 2024-04-20T11:27:31.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.895+0000 7f4689200640 1 --2- 172.21.15.152:0/3807869343 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f46840724c0 0x7f466c006a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:27:31.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.896+0000 7f468a600640 1 --2- 172.21.15.152:0/3807869343 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f466c006f50 0x7f466c0073d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f467c001e50 tx=0x7f466c007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41136398bb931cee server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:31.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.896+0000 7f4683e00640 1 -- 172.21.15.152:0/3807869343 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f467c001a70 con 0x7f466c006f50 2024-04-20T11:27:31.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.897+0000 7f4683e00640 1 -- 172.21.15.152:0/3807869343 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f467c003b70 con 0x7f466c006f50 2024-04-20T11:27:31.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.897+0000 7f4683e00640 1 -- 172.21.15.152:0/3807869343 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f467c0050d0 con 0x7f466c006f50 2024-04-20T11:27:31.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.897+0000 7f468c600640 1 -- 172.21.15.152:0/3807869343 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f466c006f50 msgr2=0x7f466c0073d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:31.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.897+0000 7f468c600640 1 --2- 172.21.15.152:0/3807869343 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f466c006f50 0x7f466c0073d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f467c001e50 tx=0x7f466c007a40 comp rx=0 tx=0).stop 2024-04-20T11:27:31.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.897+0000 7f468a600640 1 -- 172.21.15.152:0/3807869343 reap_dead start 2024-04-20T11:27:31.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.897+0000 7f468c600640 1 -- 172.21.15.152:0/3807869343 shutdown_connections 2024-04-20T11:27:31.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.897+0000 7f468c600640 1 -- 172.21.15.152:0/3807869343 >> 172.21.15.152:0/3807869343 conn(0x7f468406b980 msgr2=0x7f468406bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:31.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.897+0000 7f468c600640 1 -- 172.21.15.152:0/3807869343 shutdown_connections 2024-04-20T11:27:31.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.897+0000 7f468c600640 1 -- 172.21.15.152:0/3807869343 wait complete. 2024-04-20T11:27:31.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.899+0000 7f468c600640 1 Processor -- start 2024-04-20T11:27:31.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.899+0000 7f468c600640 1 -- start start 2024-04-20T11:27:31.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.900+0000 7f468c600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f466c006f50 0x7f4684150790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:31.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.900+0000 7f468c600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f468414e220 0x7f46841f89c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:31.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.900+0000 7f468c600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f468415a040 0x7f468414cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:31.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.900+0000 7f468c600640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f468406dc50 con 0x7f468414e220 2024-04-20T11:27:31.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.900+0000 7f468c600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4684155810 con 0x7f468415a040 2024-04-20T11:27:31.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.900+0000 7f468c600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f468406dea0 con 0x7f466c006f50 2024-04-20T11:27:31.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.900+0000 7f4689200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f468414e220 0x7f46841f89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:31.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.900+0000 7f4689200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f468414e220 0x7f46841f89c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:45106/0 (socket says 172.21.15.152:45106) 2024-04-20T11:27:31.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.900+0000 7f4689200640 1 -- 172.21.15.152:0/1560878374 learned_addr learned my addr 172.21.15.152:0/1560878374 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:31.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.901+0000 7f468a600640 1 --2- 172.21.15.152:0/1560878374 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f468415a040 0x7f468414cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:31.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.901+0000 7f4689c00640 1 --2- 172.21.15.152:0/1560878374 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f466c006f50 0x7f4684150790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:31.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.901+0000 7f4689200640 1 -- 172.21.15.152:0/1560878374 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f466c006f50 msgr2=0x7f4684150790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:31.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.901+0000 7f4689200640 1 --2- 172.21.15.152:0/1560878374 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f466c006f50 0x7f4684150790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:31.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.901+0000 7f4689200640 1 -- 172.21.15.152:0/1560878374 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f468415a040 msgr2=0x7f468414cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:31.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.901+0000 7f4689200640 1 --2- 172.21.15.152:0/1560878374 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f468415a040 0x7f468414cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:31.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.901+0000 7f4689200640 1 -- 172.21.15.152:0/1560878374 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f468414da70 con 0x7f468414e220 2024-04-20T11:27:31.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.901+0000 7f468a600640 1 --2- 172.21.15.152:0/1560878374 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f468415a040 0x7f468414cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:27:31.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.901+0000 7f4689c00640 1 --2- 172.21.15.152:0/1560878374 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f466c006f50 0x7f4684150790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:27:31.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.901+0000 7f4689200640 1 --2- 172.21.15.152:0/1560878374 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f468414e220 0x7f46841f89c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f467000e810 tx=0x7f467000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:31.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.901+0000 7f4681600640 1 -- 172.21.15.152:0/1560878374 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4670017070 con 0x7f468414e220 2024-04-20T11:27:31.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.902+0000 7f4681600640 1 -- 172.21.15.152:0/1560878374 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f467000c6d0 con 0x7f468414e220 2024-04-20T11:27:31.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.902+0000 7f4681600640 1 -- 172.21.15.152:0/1560878374 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f467000f040 con 0x7f468414e220 2024-04-20T11:27:31.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.902+0000 7f468c600640 1 -- 172.21.15.152:0/1560878374 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f468414dd50 con 0x7f468414e220 2024-04-20T11:27:31.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.902+0000 7f468c600640 1 -- 172.21.15.152:0/1560878374 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46841f84a0 con 0x7f468414e220 2024-04-20T11:27:31.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.903+0000 7f468c600640 1 -- 172.21.15.152:0/1560878374 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46840689f0 con 0x7f468414e220 2024-04-20T11:27:31.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.904+0000 7f4681600640 1 -- 172.21.15.152:0/1560878374 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f467001a070 con 0x7f468414e220 2024-04-20T11:27:31.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.905+0000 7f4681600640 1 --2- 172.21.15.152:0/1560878374 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f466406c920 0x7f466406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:31.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.905+0000 7f4681600640 1 -- 172.21.15.152:0/1560878374 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f467008b7c0 con 0x7f468414e220 2024-04-20T11:27:31.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.905+0000 7f4689c00640 1 --2- 172.21.15.152:0/1560878374 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f466406c920 0x7f466406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:31.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.906+0000 7f4689c00640 1 --2- 172.21.15.152:0/1560878374 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f466406c920 0x7f466406ede0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f46780058b0 tx=0x7f4678000ed0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:31.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:31.909+0000 7f4681600640 1 -- 172.21.15.152:0/1560878374 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f4670059b50 con 0x7f468414e220 2024-04-20T11:27:32.107 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.108+0000 7f468c600640 1 -- 172.21.15.152:0/1560878374 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4684072a90 con 0x7f468414e220 2024-04-20T11:27:32.108 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.108+0000 7f4681600640 1 -- 172.21.15.152:0/1560878374 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f4670059450 con 0x7f468414e220 2024-04-20T11:27:32.108 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 1 2024-04-20T11:27:32.112 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.111+0000 7f468c600640 1 -- 172.21.15.152:0/1560878374 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f466406c920 msgr2=0x7f466406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:32.112 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.111+0000 7f468c600640 1 --2- 172.21.15.152:0/1560878374 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f466406c920 0x7f466406ede0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f46780058b0 tx=0x7f4678000ed0 comp rx=0 tx=0).stop 2024-04-20T11:27:32.112 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.111+0000 7f468c600640 1 -- 172.21.15.152:0/1560878374 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f468414e220 msgr2=0x7f46841f89c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:32.112 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.111+0000 7f468c600640 1 --2- 172.21.15.152:0/1560878374 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f468414e220 0x7f46841f89c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f467000e810 tx=0x7f467000ecd0 comp rx=0 tx=0).stop 2024-04-20T11:27:32.112 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.111+0000 7f468c600640 1 -- 172.21.15.152:0/1560878374 shutdown_connections 2024-04-20T11:27:32.112 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.111+0000 7f468c600640 1 --2- 172.21.15.152:0/1560878374 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f466406c920 0x7f466406ede0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:32.112 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.111+0000 7f468c600640 1 --2- 172.21.15.152:0/1560878374 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f468415a040 0x7f468414cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:32.113 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.111+0000 7f468c600640 1 --2- 172.21.15.152:0/1560878374 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f468414e220 0x7f46841f89c0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:32.113 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.111+0000 7f468c600640 1 --2- 172.21.15.152:0/1560878374 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f466c006f50 0x7f4684150790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:32.113 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.111+0000 7f468c600640 1 -- 172.21.15.152:0/1560878374 >> 172.21.15.152:0/1560878374 conn(0x7f468406b980 msgr2=0x7f4684153a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:32.113 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.111+0000 7f468c600640 1 -- 172.21.15.152:0/1560878374 shutdown_connections 2024-04-20T11:27:32.113 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.111+0000 7f468c600640 1 -- 172.21.15.152:0/1560878374 wait complete. 2024-04-20T11:27:32.130 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T11:27:32.331 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.332+0000 7f5d48c00640 1 Processor -- start 2024-04-20T11:27:32.332 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.332+0000 7f5d48c00640 1 -- start start 2024-04-20T11:27:32.333 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.334+0000 7f5d48c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5d4406ee00 0x7f5d4406f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:32.333 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.334+0000 7f5d48c00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f5d4415fbf0 0x7f5d4415c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:32.333 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.334+0000 7f5d48c00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d440d00d0 con 0x7f5d440cf110 2024-04-20T11:27:32.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.334+0000 7f5d48c00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f5d4406f720 con 0x7f5d4415fbf0 2024-04-20T11:27:32.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.334+0000 7f5d48c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5d440cfb10 con 0x7f5d4406ee00 2024-04-20T11:27:32.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.334+0000 7f5d3aa00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f5d4415fbf0 0x7f5d4415c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:32.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.334+0000 7f5d43400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5d4406ee00 0x7f5d4406f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:32.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.335+0000 7f5d43400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5d4406ee00 0x7f5d4406f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.152:52684/0 (socket says 172.21.15.152:52684) 2024-04-20T11:27:32.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.335+0000 7f5d43400640 1 -- 172.21.15.152:0/3919970946 learned_addr learned my addr 172.21.15.152:0/3919970946 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:32.335 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.335+0000 7f5d3aa00640 1 -- 172.21.15.152:0/3919970946 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5d4406ee00 msgr2=0x7f5d4406f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:32.335 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.335+0000 7f5d3aa00640 1 --2- 172.21.15.152:0/3919970946 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5d4406ee00 0x7f5d4406f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:32.335 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.335+0000 7f5d3aa00640 1 -- 172.21.15.152:0/3919970946 >> v1:172.21.15.175:6789/0 conn(0x7f5d440cf110 legacy=0x7f5d440b3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:32.335 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.335+0000 7f5d3aa00640 1 -- 172.21.15.152:0/3919970946 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d440cfdf0 con 0x7f5d4415fbf0 2024-04-20T11:27:32.336 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.337+0000 7f5d3aa00640 1 --2- 172.21.15.152:0/3919970946 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d4415fbf0 0x7f5d4415c390 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c0097a0 tx=0x7f5d2c02b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e095d44696f32862 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:32.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.337+0000 7f5d42a00640 1 -- 172.21.15.152:0/3919970946 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d2c035020 con 0x7f5d4415fbf0 2024-04-20T11:27:32.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.337+0000 7f5d42a00640 1 -- 172.21.15.152:0/3919970946 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d2c031830 con 0x7f5d4415fbf0 2024-04-20T11:27:32.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.337+0000 7f5d42a00640 1 -- 172.21.15.152:0/3919970946 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d2c031c00 con 0x7f5d4415fbf0 2024-04-20T11:27:32.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.338+0000 7f5d48c00640 1 -- 172.21.15.152:0/3919970946 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d4415fbf0 msgr2=0x7f5d4415c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:32.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.338+0000 7f5d48c00640 1 --2- 172.21.15.152:0/3919970946 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d4415fbf0 0x7f5d4415c390 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c0097a0 tx=0x7f5d2c02b8d0 comp rx=0 tx=0).stop 2024-04-20T11:27:32.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.338+0000 7f5d48c00640 1 -- 172.21.15.152:0/3919970946 shutdown_connections 2024-04-20T11:27:32.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.338+0000 7f5d48c00640 1 --2- 172.21.15.152:0/3919970946 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d4415fbf0 0x7f5d4415c390 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:32.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.338+0000 7f5d48c00640 1 --2- 172.21.15.152:0/3919970946 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5d4406ee00 0x7f5d4406f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:32.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.338+0000 7f5d48c00640 1 -- 172.21.15.152:0/3919970946 >> 172.21.15.152:0/3919970946 conn(0x7f5d44067ae0 msgr2=0x7f5d4406b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:32.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.338+0000 7f5d48c00640 1 -- 172.21.15.152:0/3919970946 shutdown_connections 2024-04-20T11:27:32.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.338+0000 7f5d48c00640 1 -- 172.21.15.152:0/3919970946 wait complete. 2024-04-20T11:27:32.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.340+0000 7f5d48c00640 1 Processor -- start 2024-04-20T11:27:32.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.340+0000 7f5d48c00640 1 -- start start 2024-04-20T11:27:32.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.341+0000 7f5d48c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5d4406ee00 0x7f5d4413d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:32.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.341+0000 7f5d48c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d440cf110 0x7f5d441e5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:32.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.341+0000 7f5d48c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5d4415fbf0 0x7f5d441426c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:32.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.341+0000 7f5d48c00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f5d4415c8d0 con 0x7f5d4406ee00 2024-04-20T11:27:32.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.341+0000 7f5d48c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5d4415be70 con 0x7f5d440cf110 2024-04-20T11:27:32.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.341+0000 7f5d48c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5d44144970 con 0x7f5d4415fbf0 2024-04-20T11:27:32.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.341+0000 7f5d43400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5d4406ee00 0x7f5d4413d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:32.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.341+0000 7f5d43400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5d4406ee00 0x7f5d4413d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:45122/0 (socket says 172.21.15.152:45122) 2024-04-20T11:27:32.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.341+0000 7f5d43400640 1 -- 172.21.15.152:0/3489828234 learned_addr learned my addr 172.21.15.152:0/3489828234 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:32.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.342+0000 7f5d3aa00640 1 --2- 172.21.15.152:0/3489828234 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d440cf110 0x7f5d441e5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:32.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.342+0000 7f5d43e00640 1 --2- 172.21.15.152:0/3489828234 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5d4415fbf0 0x7f5d441426c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:32.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.342+0000 7f5d43400640 1 -- 172.21.15.152:0/3489828234 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5d4415fbf0 msgr2=0x7f5d441426c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:32.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.342+0000 7f5d43400640 1 --2- 172.21.15.152:0/3489828234 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5d4415fbf0 0x7f5d441426c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:32.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.342+0000 7f5d43400640 1 -- 172.21.15.152:0/3489828234 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d440cf110 msgr2=0x7f5d441e5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:32.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.342+0000 7f5d43400640 1 --2- 172.21.15.152:0/3489828234 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d440cf110 0x7f5d441e5ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:32.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.342+0000 7f5d43400640 1 -- 172.21.15.152:0/3489828234 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d441e4e00 con 0x7f5d4406ee00 2024-04-20T11:27:32.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.342+0000 7f5d43400640 1 --2- 172.21.15.152:0/3489828234 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5d4406ee00 0x7f5d4413d1e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5d340093b0 tx=0x7f5d3400e490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:32.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.342+0000 7f5d3be00640 1 -- 172.21.15.152:0/3489828234 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d3401a070 con 0x7f5d4406ee00 2024-04-20T11:27:32.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.342+0000 7f5d3be00640 1 -- 172.21.15.152:0/3489828234 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d3400b2d0 con 0x7f5d4406ee00 2024-04-20T11:27:32.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.342+0000 7f5d3be00640 1 -- 172.21.15.152:0/3489828234 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d3400b480 con 0x7f5d4406ee00 2024-04-20T11:27:32.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.342+0000 7f5d48c00640 1 -- 172.21.15.152:0/3489828234 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d440b2e10 con 0x7f5d4406ee00 2024-04-20T11:27:32.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.343+0000 7f5d48c00640 1 -- 172.21.15.152:0/3489828234 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d441e53a0 con 0x7f5d4406ee00 2024-04-20T11:27:32.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.344+0000 7f5d48c00640 1 -- 172.21.15.152:0/3489828234 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d441444b0 con 0x7f5d4406ee00 2024-04-20T11:27:32.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.345+0000 7f5d3be00640 1 -- 172.21.15.152:0/3489828234 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5d34005d10 con 0x7f5d4406ee00 2024-04-20T11:27:32.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.345+0000 7f5d3be00640 1 --2- 172.21.15.152:0/3489828234 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f5d2006c920 0x7f5d2006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:32.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.346+0000 7f5d3be00640 1 -- 172.21.15.152:0/3489828234 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f5d3408be20 con 0x7f5d4406ee00 2024-04-20T11:27:32.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.346+0000 7f5d3aa00640 1 --2- 172.21.15.152:0/3489828234 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f5d2006c920 0x7f5d2006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:32.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.346+0000 7f5d3aa00640 1 --2- 172.21.15.152:0/3489828234 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f5d2006c920 0x7f5d2006ede0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c036000 tx=0x7f5d2c03c040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:32.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.350+0000 7f5d3be00640 1 -- 172.21.15.152:0/3489828234 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f5d34059b60 con 0x7f5d4406ee00 2024-04-20T11:27:32.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.547+0000 7f5d48c00640 1 -- 172.21.15.152:0/3489828234 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f5d44075e60 con 0x7f5d4406ee00 2024-04-20T11:27:32.548 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.548+0000 7f5d3be00640 1 -- 172.21.15.152:0/3489828234 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f5d3405de60 con 0x7f5d4406ee00 2024-04-20T11:27:32.548 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 32 2024-04-20T11:27:32.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.551+0000 7f5d48c00640 1 -- 172.21.15.152:0/3489828234 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f5d2006c920 msgr2=0x7f5d2006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:32.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.551+0000 7f5d48c00640 1 --2- 172.21.15.152:0/3489828234 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f5d2006c920 0x7f5d2006ede0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c036000 tx=0x7f5d2c03c040 comp rx=0 tx=0).stop 2024-04-20T11:27:32.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.551+0000 7f5d48c00640 1 -- 172.21.15.152:0/3489828234 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5d4406ee00 msgr2=0x7f5d4413d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:32.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.551+0000 7f5d48c00640 1 --2- 172.21.15.152:0/3489828234 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5d4406ee00 0x7f5d4413d1e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5d340093b0 tx=0x7f5d3400e490 comp rx=0 tx=0).stop 2024-04-20T11:27:32.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.551+0000 7f5d48c00640 1 -- 172.21.15.152:0/3489828234 shutdown_connections 2024-04-20T11:27:32.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.551+0000 7f5d48c00640 1 --2- 172.21.15.152:0/3489828234 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f5d2006c920 0x7f5d2006ede0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:32.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.551+0000 7f5d48c00640 1 --2- 172.21.15.152:0/3489828234 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5d4415fbf0 0x7f5d441426c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:32.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.551+0000 7f5d48c00640 1 --2- 172.21.15.152:0/3489828234 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d440cf110 0x7f5d441e5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:32.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.551+0000 7f5d48c00640 1 --2- 172.21.15.152:0/3489828234 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5d4406ee00 0x7f5d4413d1e0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:32.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.551+0000 7f5d48c00640 1 -- 172.21.15.152:0/3489828234 >> 172.21.15.152:0/3489828234 conn(0x7f5d44067ae0 msgr2=0x7f5d440aef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:32.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.551+0000 7f5d48c00640 1 -- 172.21.15.152:0/3489828234 shutdown_connections 2024-04-20T11:27:32.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.551+0000 7f5d48c00640 1 -- 172.21.15.152:0/3489828234 wait complete. 2024-04-20T11:27:32.570 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T11:27:32.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.776+0000 7fa23b200640 1 Processor -- start 2024-04-20T11:27:32.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.776+0000 7fa23b200640 1 -- start start 2024-04-20T11:27:32.777 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.777+0000 7fa23b200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa2340d14a0 0x7fa2340d4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:32.777 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.777+0000 7fa23b200640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2340d19a0 con 0x7fa23415e9e0 2024-04-20T11:27:32.777 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.777+0000 7fa23b200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2340d1c80 con 0x7fa2340d46a0 2024-04-20T11:27:32.777 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.777+0000 7fa23b200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa23406da30 con 0x7fa2340d14a0 2024-04-20T11:27:32.779 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.778+0000 7fa233e00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fa23415e9e0 0x7fa23406d320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:44710/0 (socket says 172.21.15.152:44710) 2024-04-20T11:27:32.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.778+0000 7fa233e00640 1 -- 172.21.15.152:0/3751455623 learned_addr learned my addr 172.21.15.152:0/3751455623 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:27:32.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.778+0000 7fa239200640 1 --2- 172.21.15.152:0/3751455623 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa2340d14a0 0x7fa2340d4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:32.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.778+0000 7fa233400640 1 -- 172.21.15.152:0/3751455623 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2555792280 0 0) 0x7fa2340d19a0 con 0x7fa23415e9e0 2024-04-20T11:27:32.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.778+0000 7fa233400640 1 -- 172.21.15.152:0/3751455623 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa218003520 con 0x7fa23415e9e0 2024-04-20T11:27:32.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.779+0000 7fa233400640 1 -- 172.21.15.152:0/3751455623 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3069229066 0 0) 0x7fa218003520 con 0x7fa23415e9e0 2024-04-20T11:27:32.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.779+0000 7fa233400640 1 -- 172.21.15.152:0/3751455623 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa2340d14a0 msgr2=0x7fa2340d4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:32.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.779+0000 7fa233400640 1 --2- 172.21.15.152:0/3751455623 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa2340d14a0 0x7fa2340d4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:32.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.779+0000 7fa233400640 1 -- 172.21.15.152:0/3751455623 >> v1:172.21.15.175:6789/0 conn(0x7fa2340d46a0 legacy=0x7fa2340d0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:32.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.779+0000 7fa233400640 1 -- 172.21.15.152:0/3751455623 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2340d1f60 con 0x7fa23415e9e0 2024-04-20T11:27:32.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.779+0000 7fa233400640 1 -- 172.21.15.152:0/3751455623 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7fa224001670 con 0x7fa23415e9e0 2024-04-20T11:27:32.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.779+0000 7fa233400640 1 -- 172.21.15.152:0/3751455623 >> v1:172.21.15.152:6789/0 conn(0x7fa23415e9e0 legacy=0x7fa23406d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:32.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.779+0000 7fa233400640 1 --2- 172.21.15.152:0/3751455623 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2180039e0 0x7fa218003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:32.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.779+0000 7fa233400640 1 --2- 172.21.15.152:0/3751455623 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa218004800 0x7fa218004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:32.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.779+0000 7fa233400640 1 --2- 172.21.15.152:0/3751455623 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa2340d14a0 0x7fa2180051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:32.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.779+0000 7fa233400640 1 -- 172.21.15.152:0/3751455623 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fa23406da30 con 0x7fa218004800 2024-04-20T11:27:32.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.779+0000 7fa233400640 1 -- 172.21.15.152:0/3751455623 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa21800baa0 con 0x7fa2180039e0 2024-04-20T11:27:32.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.779+0000 7fa233400640 1 -- 172.21.15.152:0/3751455623 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa21800bca0 con 0x7fa2340d14a0 2024-04-20T11:27:32.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.779+0000 7fa22be00640 1 --2- 172.21.15.152:0/3751455623 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa218004800 0x7fa218004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:32.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.779+0000 7fa233e00640 1 --2- 172.21.15.152:0/3751455623 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2180039e0 0x7fa218003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:32.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.780+0000 7fa239200640 1 --2- 172.21.15.152:0/3751455623 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa2340d14a0 0x7fa2180051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:32.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.780+0000 7fa22be00640 1 -- 172.21.15.152:0/3751455623 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa2340d14a0 msgr2=0x7fa2180051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:32.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.780+0000 7fa22be00640 1 --2- 172.21.15.152:0/3751455623 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa2340d14a0 0x7fa2180051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:32.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.780+0000 7fa22be00640 1 -- 172.21.15.152:0/3751455623 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2180039e0 msgr2=0x7fa218003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:32.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.780+0000 7fa22be00640 1 --2- 172.21.15.152:0/3751455623 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2180039e0 0x7fa218003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:32.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.780+0000 7fa22be00640 1 -- 172.21.15.152:0/3751455623 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa218003520 con 0x7fa218004800 2024-04-20T11:27:32.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.781+0000 7fa22be00640 1 --2- 172.21.15.152:0/3751455623 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa218004800 0x7fa218004c80 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa21c001720 tx=0x7fa218005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7b5a26eeca27a4e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:32.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.781+0000 7fa233400640 1 -- 172.21.15.152:0/3751455623 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa21c001370 con 0x7fa218004800 2024-04-20T11:27:32.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.781+0000 7fa233400640 1 -- 172.21.15.152:0/3751455623 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa21c0052d0 con 0x7fa218004800 2024-04-20T11:27:32.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.781+0000 7fa233400640 1 -- 172.21.15.152:0/3751455623 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa21c003830 con 0x7fa218004800 2024-04-20T11:27:32.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.782+0000 7fa23b200640 1 -- 172.21.15.152:0/3751455623 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa218004800 msgr2=0x7fa218004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:32.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.782+0000 7fa23b200640 1 --2- 172.21.15.152:0/3751455623 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa218004800 0x7fa218004c80 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa21c001720 tx=0x7fa218005830 comp rx=0 tx=0).stop 2024-04-20T11:27:32.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.782+0000 7fa239200640 1 -- 172.21.15.152:0/3751455623 reap_dead start 2024-04-20T11:27:32.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.782+0000 7fa23b200640 1 -- 172.21.15.152:0/3751455623 shutdown_connections 2024-04-20T11:27:32.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.782+0000 7fa23b200640 1 -- 172.21.15.152:0/3751455623 >> 172.21.15.152:0/3751455623 conn(0x7fa23409dab0 msgr2=0x7fa23408bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:32.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.782+0000 7fa23b200640 1 -- 172.21.15.152:0/3751455623 shutdown_connections 2024-04-20T11:27:32.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.782+0000 7fa23b200640 1 -- 172.21.15.152:0/3751455623 wait complete. 2024-04-20T11:27:32.784 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.784+0000 7fa23b200640 1 Processor -- start 2024-04-20T11:27:32.784 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.784+0000 7fa23b200640 1 -- start start 2024-04-20T11:27:32.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.785+0000 7fa23b200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa218004800 0x7fa23415aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:32.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.785+0000 7fa23b200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2340b7360 0x7fa2340cabf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:32.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.785+0000 7fa23b200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa2340cb130 0x7fa2340ce0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:32.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.785+0000 7fa23b200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fa23415edc0 con 0x7fa2340cb130 2024-04-20T11:27:32.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.785+0000 7fa23b200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa2340cf880 con 0x7fa2340b7360 2024-04-20T11:27:32.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.785+0000 7fa23b200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa23415ef40 con 0x7fa218004800 2024-04-20T11:27:32.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.785+0000 7fa239200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa2340cb130 0x7fa2340ce0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:32.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.786+0000 7fa239200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa2340cb130 0x7fa2340ce0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:45142/0 (socket says 172.21.15.152:45142) 2024-04-20T11:27:32.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.786+0000 7fa239200640 1 -- 172.21.15.152:0/1201012004 learned_addr learned my addr 172.21.15.152:0/1201012004 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:32.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.786+0000 7fa22be00640 1 --2- 172.21.15.152:0/1201012004 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2340b7360 0x7fa2340cabf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:32.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.786+0000 7fa233e00640 1 --2- 172.21.15.152:0/1201012004 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa218004800 0x7fa23415aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:32.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.786+0000 7fa239200640 1 -- 172.21.15.152:0/1201012004 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa218004800 msgr2=0x7fa23415aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:32.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.786+0000 7fa239200640 1 --2- 172.21.15.152:0/1201012004 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa218004800 0x7fa23415aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:32.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.786+0000 7fa239200640 1 -- 172.21.15.152:0/1201012004 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2340b7360 msgr2=0x7fa2340cabf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:32.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.786+0000 7fa239200640 1 --2- 172.21.15.152:0/1201012004 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2340b7360 0x7fa2340cabf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:32.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.786+0000 7fa239200640 1 -- 172.21.15.152:0/1201012004 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2340cc0c0 con 0x7fa2340cb130 2024-04-20T11:27:32.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.786+0000 7fa22be00640 1 --2- 172.21.15.152:0/1201012004 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2340b7360 0x7fa2340cabf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:27:32.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.786+0000 7fa239200640 1 --2- 172.21.15.152:0/1201012004 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa2340cb130 0x7fa2340ce0f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa2340d18d0 tx=0x7fa22c0115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:32.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.787+0000 7fa230c00640 1 -- 172.21.15.152:0/1201012004 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa22c011ea0 con 0x7fa2340cb130 2024-04-20T11:27:32.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.787+0000 7fa230c00640 1 -- 172.21.15.152:0/1201012004 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa22c01b070 con 0x7fa2340cb130 2024-04-20T11:27:32.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.787+0000 7fa23b200640 1 -- 172.21.15.152:0/1201012004 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2340cc3a0 con 0x7fa2340cb130 2024-04-20T11:27:32.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.787+0000 7fa230c00640 1 -- 172.21.15.152:0/1201012004 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa22c003990 con 0x7fa2340cb130 2024-04-20T11:27:32.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.787+0000 7fa23b200640 1 -- 172.21.15.152:0/1201012004 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2340cc960 con 0x7fa2340cb130 2024-04-20T11:27:32.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.788+0000 7fa23b200640 1 -- 172.21.15.152:0/1201012004 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2340b2a30 con 0x7fa2340cb130 2024-04-20T11:27:32.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.789+0000 7fa230c00640 1 -- 172.21.15.152:0/1201012004 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa22c003b30 con 0x7fa2340cb130 2024-04-20T11:27:32.789 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.790+0000 7fa230c00640 1 --2- 172.21.15.152:0/1201012004 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fa21006c920 0x7fa21006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:32.789 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.790+0000 7fa230c00640 1 -- 172.21.15.152:0/1201012004 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa22c08c440 con 0x7fa2340cb130 2024-04-20T11:27:32.790 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.790+0000 7fa233e00640 1 --2- 172.21.15.152:0/1201012004 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fa21006c920 0x7fa21006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:32.790 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.791+0000 7fa233e00640 1 --2- 172.21.15.152:0/1201012004 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fa21006c920 0x7fa21006ede0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa224001050 tx=0x7fa224009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:32.794 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.795+0000 7fa230c00640 1 -- 172.21.15.152:0/1201012004 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fa2340b2a30 con 0x7fa2340cb130 2024-04-20T11:27:32.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.995+0000 7fa23b200640 1 -- 172.21.15.152:0/1201012004 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa2340b5280 con 0x7fa2340cb130 2024-04-20T11:27:32.995 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.996+0000 7fa230c00640 1 -- 172.21.15.152:0/1201012004 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7fa2340b5280 con 0x7fa2340cb130 2024-04-20T11:27:32.995 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 64 2024-04-20T11:27:32.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.998+0000 7fa23b200640 1 -- 172.21.15.152:0/1201012004 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fa21006c920 msgr2=0x7fa21006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:32.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.998+0000 7fa23b200640 1 --2- 172.21.15.152:0/1201012004 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fa21006c920 0x7fa21006ede0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa224001050 tx=0x7fa224009040 comp rx=0 tx=0).stop 2024-04-20T11:27:32.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.998+0000 7fa23b200640 1 -- 172.21.15.152:0/1201012004 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa2340cb130 msgr2=0x7fa2340ce0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:32.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.998+0000 7fa23b200640 1 --2- 172.21.15.152:0/1201012004 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa2340cb130 0x7fa2340ce0f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa2340d18d0 tx=0x7fa22c0115a0 comp rx=0 tx=0).stop 2024-04-20T11:27:32.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.999+0000 7fa23b200640 1 -- 172.21.15.152:0/1201012004 shutdown_connections 2024-04-20T11:27:32.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.999+0000 7fa23b200640 1 --2- 172.21.15.152:0/1201012004 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fa21006c920 0x7fa21006ede0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:32.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.999+0000 7fa23b200640 1 --2- 172.21.15.152:0/1201012004 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa2340cb130 0x7fa2340ce0f0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:32.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.999+0000 7fa23b200640 1 --2- 172.21.15.152:0/1201012004 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2340b7360 0x7fa2340cabf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:32.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.999+0000 7fa23b200640 1 --2- 172.21.15.152:0/1201012004 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa218004800 0x7fa23415aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:32.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.999+0000 7fa23b200640 1 -- 172.21.15.152:0/1201012004 >> 172.21.15.152:0/1201012004 conn(0x7fa23409dab0 msgr2=0x7fa2340b4b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:32.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.999+0000 7fa23b200640 1 -- 172.21.15.152:0/1201012004 shutdown_connections 2024-04-20T11:27:32.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:32.999+0000 7fa23b200640 1 -- 172.21.15.152:0/1201012004 wait complete. 2024-04-20T11:27:33.017 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:27:33.207 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.208+0000 7fb4c8a00640 1 Processor -- start 2024-04-20T11:27:33.207 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.208+0000 7fb4c8a00640 1 -- start start 2024-04-20T11:27:33.209 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.209+0000 7fb4c8a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fb4c415d1e0 0x7fb4c4160b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:33.209 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.209+0000 7fb4c8a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb4c4071cd0 0x7fb4c4072160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:33.209 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.209+0000 7fb4c8a00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4c40ce8c0 con 0x7fb4c415d5c0 2024-04-20T11:27:33.209 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.209+0000 7fb4c8a00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fb4c40726a0 con 0x7fb4c415d1e0 2024-04-20T11:27:33.209 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.209+0000 7fb4c8a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb4c41600b0 con 0x7fb4c4071cd0 2024-04-20T11:27:33.209 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.210+0000 7fb4c3400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fb4c415d1e0 0x7fb4c4160b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:33.210 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.210+0000 7fb4c3400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fb4c415d1e0 0x7fb4c4160b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:45144/0 (socket says 172.21.15.152:45144) 2024-04-20T11:27:33.210 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.210+0000 7fb4c3400640 1 -- 172.21.15.152:0/733333179 learned_addr learned my addr 172.21.15.152:0/733333179 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:33.210 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.210+0000 7fb4c3e00640 1 --2- 172.21.15.152:0/733333179 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb4c4071cd0 0x7fb4c4072160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:33.210 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.210+0000 7fb4c2000640 1 -- 172.21.15.152:0/733333179 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1443359138 0 0) 0x7fb4c40ce8c0 con 0x7fb4c415d5c0 2024-04-20T11:27:33.211 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.211+0000 7fb4c2000640 1 -- 172.21.15.152:0/733333179 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb4a8003520 con 0x7fb4c415d5c0 2024-04-20T11:27:33.211 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.211+0000 7fb4c3400640 1 -- 172.21.15.152:0/733333179 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb4c4071cd0 msgr2=0x7fb4c4072160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:33.211 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.211+0000 7fb4c3400640 1 --2- 172.21.15.152:0/733333179 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb4c4071cd0 0x7fb4c4072160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:33.211 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.211+0000 7fb4c3400640 1 -- 172.21.15.152:0/733333179 >> v1:172.21.15.152:6789/0 conn(0x7fb4c415d5c0 legacy=0x7fb4c40cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:33.211 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.211+0000 7fb4c3400640 1 -- 172.21.15.152:0/733333179 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4c4160390 con 0x7fb4c415d1e0 2024-04-20T11:27:33.213 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.212+0000 7fb4c3400640 1 --2- 172.21.15.152:0/733333179 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb4c415d1e0 0x7fb4c4160b90 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb4b40098a0 tx=0x7fb4b402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c1333dfd11af9fa server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:33.213 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.212+0000 7fb4c2000640 1 -- 172.21.15.152:0/733333179 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b4034070 con 0x7fb4c415d1e0 2024-04-20T11:27:33.213 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.212+0000 7fb4c2000640 1 -- 172.21.15.152:0/733333179 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4b4030840 con 0x7fb4c415d1e0 2024-04-20T11:27:33.213 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.212+0000 7fb4c2000640 1 -- 172.21.15.152:0/733333179 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b4030bf0 con 0x7fb4c415d1e0 2024-04-20T11:27:33.215 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.213+0000 7fb4c8a00640 1 -- 172.21.15.152:0/733333179 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb4c415d1e0 msgr2=0x7fb4c4160b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:33.215 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.213+0000 7fb4c8a00640 1 --2- 172.21.15.152:0/733333179 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb4c415d1e0 0x7fb4c4160b90 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb4b40098a0 tx=0x7fb4b402b980 comp rx=0 tx=0).stop 2024-04-20T11:27:33.215 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.213+0000 7fb4c8a00640 1 -- 172.21.15.152:0/733333179 shutdown_connections 2024-04-20T11:27:33.215 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.213+0000 7fb4c8a00640 1 --2- 172.21.15.152:0/733333179 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb4c4071cd0 0x7fb4c4072160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:33.215 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.213+0000 7fb4c8a00640 1 --2- 172.21.15.152:0/733333179 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb4c415d1e0 0x7fb4c4160b90 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:33.215 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.213+0000 7fb4c8a00640 1 -- 172.21.15.152:0/733333179 >> 172.21.15.152:0/733333179 conn(0x7fb4c4067ae0 msgr2=0x7fb4c406b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:33.215 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.213+0000 7fb4c8a00640 1 -- 172.21.15.152:0/733333179 shutdown_connections 2024-04-20T11:27:33.215 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.213+0000 7fb4c8a00640 1 -- 172.21.15.152:0/733333179 wait complete. 2024-04-20T11:27:33.215 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.215+0000 7fb4c8a00640 1 Processor -- start 2024-04-20T11:27:33.215 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.215+0000 7fb4c8a00640 1 -- start start 2024-04-20T11:27:33.217 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.216+0000 7fb4c8a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb4c4071cd0 0x7fb4c41f1f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:33.217 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.216+0000 7fb4c8a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb4c415d1e0 0x7fb4c4155800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:33.217 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.216+0000 7fb4c8a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4c415d5c0 0x7fb4c4155d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:33.217 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.216+0000 7fb4c8a00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c40ccd70 con 0x7fb4c4071cd0 2024-04-20T11:27:33.217 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.216+0000 7fb4c8a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c40ceba0 con 0x7fb4c415d5c0 2024-04-20T11:27:33.217 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.216+0000 7fb4c8a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb4c40ccef0 con 0x7fb4c415d1e0 2024-04-20T11:27:33.218 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.216+0000 7fb4c3400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb4c4071cd0 0x7fb4c41f1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:33.218 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.216+0000 7fb4c3400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb4c4071cd0 0x7fb4c41f1f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:45156/0 (socket says 172.21.15.152:45156) 2024-04-20T11:27:33.218 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.216+0000 7fb4c3400640 1 -- 172.21.15.152:0/3048938163 learned_addr learned my addr 172.21.15.152:0/3048938163 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:33.218 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.217+0000 7fb4c3e00640 1 --2- 172.21.15.152:0/3048938163 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4c415d5c0 0x7fb4c4155d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:33.218 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.217+0000 7fb4c2a00640 1 --2- 172.21.15.152:0/3048938163 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb4c415d1e0 0x7fb4c4155800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:33.218 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.217+0000 7fb4c3400640 1 -- 172.21.15.152:0/3048938163 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb4c415d1e0 msgr2=0x7fb4c4155800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:33.218 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.217+0000 7fb4c3400640 1 --2- 172.21.15.152:0/3048938163 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb4c415d1e0 0x7fb4c4155800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:33.218 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.217+0000 7fb4c3400640 1 -- 172.21.15.152:0/3048938163 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4c415d5c0 msgr2=0x7fb4c4155d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:33.218 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.217+0000 7fb4c3400640 1 --2- 172.21.15.152:0/3048938163 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4c415d5c0 0x7fb4c4155d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:33.218 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.217+0000 7fb4c3400640 1 -- 172.21.15.152:0/3048938163 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4c4159150 con 0x7fb4c4071cd0 2024-04-20T11:27:33.218 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.217+0000 7fb4c3400640 1 --2- 172.21.15.152:0/3048938163 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb4c4071cd0 0x7fb4c41f1f40 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb4b402b9c0 tx=0x7fb4b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:33.218 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.217+0000 7fb4bb400640 1 -- 172.21.15.152:0/3048938163 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b4047070 con 0x7fb4c4071cd0 2024-04-20T11:27:33.218 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.217+0000 7fb4bb400640 1 -- 172.21.15.152:0/3048938163 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4b4030640 con 0x7fb4c4071cd0 2024-04-20T11:27:33.219 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.217+0000 7fb4bb400640 1 -- 172.21.15.152:0/3048938163 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b4034040 con 0x7fb4c4071cd0 2024-04-20T11:27:33.219 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.218+0000 7fb4c8a00640 1 -- 172.21.15.152:0/3048938163 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4c4159430 con 0x7fb4c4071cd0 2024-04-20T11:27:33.219 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.218+0000 7fb4c8a00640 1 -- 172.21.15.152:0/3048938163 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4c41599f0 con 0x7fb4c4071cd0 2024-04-20T11:27:33.219 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.219+0000 7fb4c8a00640 1 -- 172.21.15.152:0/3048938163 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4c4075e60 con 0x7fb4c4071cd0 2024-04-20T11:27:33.219 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.220+0000 7fb4bb400640 1 -- 172.21.15.152:0/3048938163 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb4b4037070 con 0x7fb4c4071cd0 2024-04-20T11:27:33.221 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.221+0000 7fb4bb400640 1 --2- 172.21.15.152:0/3048938163 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fb4a006c8f0 0x7fb4a006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:33.221 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.221+0000 7fb4bb400640 1 -- 172.21.15.152:0/3048938163 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb4b40ad3e0 con 0x7fb4c4071cd0 2024-04-20T11:27:33.221 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.222+0000 7fb4c2a00640 1 --2- 172.21.15.152:0/3048938163 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fb4a006c8f0 0x7fb4a006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:33.221 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.222+0000 7fb4c2a00640 1 --2- 172.21.15.152:0/3048938163 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fb4a006c8f0 0x7fb4a006edb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb4c40b1820 tx=0x7fb4ac009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:33.225 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.226+0000 7fb4bb400640 1 -- 172.21.15.152:0/3048938163 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fb4b407b790 con 0x7fb4c4071cd0 2024-04-20T11:27:33.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.428+0000 7fb4c8a00640 1 -- 172.21.15.152:0/3048938163 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb4c415c800 con 0x7fb4c4071cd0 2024-04-20T11:27:33.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.431+0000 7fb4bb400640 1 -- 172.21.15.152:0/3048938163 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16665 (secure 0 0 0) 0x7fb4b407b090 con 0x7fb4c4071cd0 2024-04-20T11:27:33.431 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:27:33.432 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":20,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:26:59.675971+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:26:56.261424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T11:26:57.079286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:27:33.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.434+0000 7fb4c8a00640 1 -- 172.21.15.152:0/3048938163 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fb4a006c8f0 msgr2=0x7fb4a006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:33.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.434+0000 7fb4c8a00640 1 --2- 172.21.15.152:0/3048938163 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fb4a006c8f0 0x7fb4a006edb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb4c40b1820 tx=0x7fb4ac009040 comp rx=0 tx=0).stop 2024-04-20T11:27:33.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.434+0000 7fb4c8a00640 1 -- 172.21.15.152:0/3048938163 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb4c4071cd0 msgr2=0x7fb4c41f1f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:33.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.434+0000 7fb4c8a00640 1 --2- 172.21.15.152:0/3048938163 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb4c4071cd0 0x7fb4c41f1f40 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb4b402b9c0 tx=0x7fb4b40046c0 comp rx=0 tx=0).stop 2024-04-20T11:27:33.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.434+0000 7fb4c8a00640 1 -- 172.21.15.152:0/3048938163 shutdown_connections 2024-04-20T11:27:33.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.434+0000 7fb4c8a00640 1 --2- 172.21.15.152:0/3048938163 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fb4a006c8f0 0x7fb4a006edb0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:33.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.434+0000 7fb4c8a00640 1 --2- 172.21.15.152:0/3048938163 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4c415d5c0 0x7fb4c4155d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:33.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.434+0000 7fb4c8a00640 1 --2- 172.21.15.152:0/3048938163 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb4c415d1e0 0x7fb4c4155800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:33.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.434+0000 7fb4c8a00640 1 --2- 172.21.15.152:0/3048938163 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb4c4071cd0 0x7fb4c41f1f40 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:33.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.434+0000 7fb4c8a00640 1 -- 172.21.15.152:0/3048938163 >> 172.21.15.152:0/3048938163 conn(0x7fb4c4067ae0 msgr2=0x7fb4c40b0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:33.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.435+0000 7fb4c8a00640 1 -- 172.21.15.152:0/3048938163 shutdown_connections 2024-04-20T11:27:33.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.435+0000 7fb4c8a00640 1 -- 172.21.15.152:0/3048938163 wait complete. 2024-04-20T11:27:33.451 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T11:26:47.246571+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T11:26:56.261424+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T11:26:57.079286+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T11:27:33.452 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T11:27:33.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.650+0000 7f053a200640 1 Processor -- start 2024-04-20T11:27:33.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.650+0000 7f053a200640 1 -- start start 2024-04-20T11:27:33.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.651+0000 7f053a200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f053406c4b0 0x7f0534166330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:33.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.651+0000 7f053a200640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0534152830 con 0x7f0534152450 2024-04-20T11:27:33.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.651+0000 7f053a200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05340b39c0 con 0x7f053415c590 2024-04-20T11:27:33.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.651+0000 7f053a200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f05340b3ca0 con 0x7f053406c4b0 2024-04-20T11:27:33.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.652+0000 7f0539800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f053406c4b0 0x7f0534166330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:33.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.652+0000 7f0538e00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f0534152450 0x7f05340b30c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:44724/0 (socket says 172.21.15.152:44724) 2024-04-20T11:27:33.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.652+0000 7f0538e00640 1 -- 172.21.15.152:0/660539615 learned_addr learned my addr 172.21.15.152:0/660539615 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:27:33.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.652+0000 7f0539800640 1 -- 172.21.15.152:0/660539615 >> v1:172.21.15.175:6789/0 conn(0x7f053415c590 legacy=0x7f053406bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:33.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.652+0000 7f0539800640 1 -- 172.21.15.152:0/660539615 >> v1:172.21.15.152:6789/0 conn(0x7f0534152450 legacy=0x7f05340b30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:33.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.652+0000 7f0533e00640 1 -- 172.21.15.152:0/660539615 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2636811470 0 0) 0x7f0534152830 con 0x7f0534152450 2024-04-20T11:27:33.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.652+0000 7f0539800640 1 -- 172.21.15.152:0/660539615 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05340b3f80 con 0x7f053406c4b0 2024-04-20T11:27:33.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.653+0000 7f0539800640 1 --2- 172.21.15.152:0/660539615 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f053406c4b0 0x7f0534166330 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f052c009d90 tx=0x7f052c02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d89f9533c77e2bb9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:33.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.654+0000 7f0533e00640 1 -- 172.21.15.152:0/660539615 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f052c039070 con 0x7f053406c4b0 2024-04-20T11:27:33.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.654+0000 7f0533e00640 1 -- 172.21.15.152:0/660539615 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f052c02cc60 con 0x7f053406c4b0 2024-04-20T11:27:33.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.654+0000 7f0533e00640 1 -- 172.21.15.152:0/660539615 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f052c034660 con 0x7f053406c4b0 2024-04-20T11:27:33.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.654+0000 7f053a200640 1 -- 172.21.15.152:0/660539615 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f053406c4b0 msgr2=0x7f0534166330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:33.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.654+0000 7f053a200640 1 --2- 172.21.15.152:0/660539615 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f053406c4b0 0x7f0534166330 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f052c009d90 tx=0x7f052c02beb0 comp rx=0 tx=0).stop 2024-04-20T11:27:33.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.654+0000 7f053a200640 1 -- 172.21.15.152:0/660539615 shutdown_connections 2024-04-20T11:27:33.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.654+0000 7f053a200640 1 --2- 172.21.15.152:0/660539615 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f053406c4b0 0x7f0534166330 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:33.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.654+0000 7f053a200640 1 -- 172.21.15.152:0/660539615 >> 172.21.15.152:0/660539615 conn(0x7f05340b21e0 msgr2=0x7f05340b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:33.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.654+0000 7f053a200640 1 -- 172.21.15.152:0/660539615 shutdown_connections 2024-04-20T11:27:33.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.655+0000 7f053a200640 1 -- 172.21.15.152:0/660539615 wait complete. 2024-04-20T11:27:33.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.656+0000 7f053a200640 1 Processor -- start 2024-04-20T11:27:33.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.656+0000 7f053a200640 1 -- start start 2024-04-20T11:27:33.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.657+0000 7f053a200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f053406c4b0 0x7f053406ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:33.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.657+0000 7f053a200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0534152450 0x7f05340ce210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:33.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.657+0000 7f053a200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f053415c590 0x7f05340cc340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:33.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.657+0000 7f053a200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f05340af480 con 0x7f0534152450 2024-04-20T11:27:33.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.657+0000 7f053a200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f05340adf00 con 0x7f053406c4b0 2024-04-20T11:27:33.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.657+0000 7f053a200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f05340af600 con 0x7f053415c590 2024-04-20T11:27:33.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.657+0000 7f052be00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0534152450 0x7f05340ce210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:33.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.657+0000 7f052be00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0534152450 0x7f05340ce210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:45178/0 (socket says 172.21.15.152:45178) 2024-04-20T11:27:33.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.657+0000 7f052be00640 1 -- 172.21.15.152:0/3906945736 learned_addr learned my addr 172.21.15.152:0/3906945736 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:33.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.658+0000 7f0538e00640 1 --2- 172.21.15.152:0/3906945736 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f053406c4b0 0x7f053406ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:33.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.658+0000 7f0539800640 1 --2- 172.21.15.152:0/3906945736 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f053415c590 0x7f05340cc340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:33.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.658+0000 7f052be00640 1 -- 172.21.15.152:0/3906945736 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f053415c590 msgr2=0x7f05340cc340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:33.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.658+0000 7f052be00640 1 --2- 172.21.15.152:0/3906945736 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f053415c590 0x7f05340cc340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:33.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.658+0000 7f052be00640 1 -- 172.21.15.152:0/3906945736 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f053406c4b0 msgr2=0x7f053406ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:33.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.658+0000 7f052be00640 1 --2- 172.21.15.152:0/3906945736 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f053406c4b0 0x7f053406ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:33.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.658+0000 7f052be00640 1 -- 172.21.15.152:0/3906945736 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05340ce750 con 0x7f0534152450 2024-04-20T11:27:33.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.658+0000 7f0538e00640 1 --2- 172.21.15.152:0/3906945736 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f053406c4b0 0x7f053406ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:27:33.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.658+0000 7f052be00640 1 --2- 172.21.15.152:0/3906945736 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0534152450 0x7f05340ce210 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f051c00b3f0 tx=0x7f051c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:33.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.658+0000 7f0531600640 1 -- 172.21.15.152:0/3906945736 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f051c015020 con 0x7f0534152450 2024-04-20T11:27:33.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.658+0000 7f0531600640 1 -- 172.21.15.152:0/3906945736 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f051c011830 con 0x7f0534152450 2024-04-20T11:27:33.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.658+0000 7f0531600640 1 -- 172.21.15.152:0/3906945736 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f051c011c20 con 0x7f0534152450 2024-04-20T11:27:33.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.658+0000 7f053a200640 1 -- 172.21.15.152:0/3906945736 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05340ce9f0 con 0x7f0534152450 2024-04-20T11:27:33.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.659+0000 7f053a200640 1 -- 172.21.15.152:0/3906945736 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05340cefb0 con 0x7f0534152450 2024-04-20T11:27:33.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.660+0000 7f053a200640 1 -- 172.21.15.152:0/3906945736 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0534075e60 con 0x7f0534152450 2024-04-20T11:27:33.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.661+0000 7f0531600640 1 -- 172.21.15.152:0/3906945736 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f051c018020 con 0x7f0534152450 2024-04-20T11:27:33.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.661+0000 7f0531600640 1 --2- 172.21.15.152:0/3906945736 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f051006c8f0 0x7f051006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:33.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.662+0000 7f0531600640 1 -- 172.21.15.152:0/3906945736 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f051c08b7d0 con 0x7f0534152450 2024-04-20T11:27:33.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.662+0000 7f0538e00640 1 --2- 172.21.15.152:0/3906945736 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f051006c8f0 0x7f051006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:33.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.662+0000 7f0538e00640 1 --2- 172.21.15.152:0/3906945736 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f051006c8f0 0x7f051006edb0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0524000f60 tx=0x7f0524009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:33.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.666+0000 7f0531600640 1 -- 172.21.15.152:0/3906945736 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f051c01f030 con 0x7f0534152450 2024-04-20T11:27:33.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.865+0000 7f053a200640 1 -- 172.21.15.152:0/3906945736 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f053406f770 con 0x7f0534152450 2024-04-20T11:27:33.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.866+0000 7f0531600640 1 -- 172.21.15.152:0/3906945736 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f051c059bf0 con 0x7f0534152450 2024-04-20T11:27:33.865 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 1 2024-04-20T11:27:33.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.868+0000 7f053a200640 1 -- 172.21.15.152:0/3906945736 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f051006c8f0 msgr2=0x7f051006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:33.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.868+0000 7f053a200640 1 --2- 172.21.15.152:0/3906945736 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f051006c8f0 0x7f051006edb0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0524000f60 tx=0x7f0524009040 comp rx=0 tx=0).stop 2024-04-20T11:27:33.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.868+0000 7f053a200640 1 -- 172.21.15.152:0/3906945736 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0534152450 msgr2=0x7f05340ce210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:33.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.868+0000 7f053a200640 1 --2- 172.21.15.152:0/3906945736 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0534152450 0x7f05340ce210 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f051c00b3f0 tx=0x7f051c00b8b0 comp rx=0 tx=0).stop 2024-04-20T11:27:33.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.869+0000 7f053a200640 1 -- 172.21.15.152:0/3906945736 shutdown_connections 2024-04-20T11:27:33.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.869+0000 7f053a200640 1 --2- 172.21.15.152:0/3906945736 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f051006c8f0 0x7f051006edb0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:33.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.869+0000 7f053a200640 1 --2- 172.21.15.152:0/3906945736 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f053415c590 0x7f05340cc340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:33.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.869+0000 7f053a200640 1 --2- 172.21.15.152:0/3906945736 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0534152450 0x7f05340ce210 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:33.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.869+0000 7f053a200640 1 --2- 172.21.15.152:0/3906945736 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f053406c4b0 0x7f053406ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:33.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.869+0000 7f053a200640 1 -- 172.21.15.152:0/3906945736 >> 172.21.15.152:0/3906945736 conn(0x7f05340b21e0 msgr2=0x7f0534072fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:33.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.869+0000 7f053a200640 1 -- 172.21.15.152:0/3906945736 shutdown_connections 2024-04-20T11:27:33.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:33.869+0000 7f053a200640 1 -- 172.21.15.152:0/3906945736 wait complete. 2024-04-20T11:27:33.888 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T11:27:34.092 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.093+0000 7fbc7c600640 1 Processor -- start 2024-04-20T11:27:34.092 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.093+0000 7fbc7c600640 1 -- start start 2024-04-20T11:27:34.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.094+0000 7fbc7c600640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fbc740d0570 0x7fbc74162c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:34.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.094+0000 7fbc7c600640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fbc740d2330 0x7fbc740d2790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:34.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.094+0000 7fbc7c600640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc740cf510 con 0x7fbc7406e630 2024-04-20T11:27:34.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.094+0000 7fbc7c600640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fbc740cf7b0 con 0x7fbc740d0570 2024-04-20T11:27:34.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.094+0000 7fbc7c600640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fbc740cfa90 con 0x7fbc740d2330 2024-04-20T11:27:34.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.095+0000 7fbc79200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fbc740d0570 0x7fbc74162c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:34.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.095+0000 7fbc79200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fbc740d0570 0x7fbc74162c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:45184/0 (socket says 172.21.15.152:45184) 2024-04-20T11:27:34.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.095+0000 7fbc79200640 1 -- 172.21.15.152:0/372048817 learned_addr learned my addr 172.21.15.152:0/372048817 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:34.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.095+0000 7fbc7a600640 1 --2- 172.21.15.152:0/372048817 >> v2:172.21.15.175:3300/0 conn(0x7fbc740d2330 0x7fbc740d2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:34.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.095+0000 7fbc79200640 1 -- 172.21.15.152:0/372048817 >> v2:172.21.15.175:3300/0 conn(0x7fbc740d2330 msgr2=0x7fbc740d2790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:34.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.095+0000 7fbc79200640 1 --2- 172.21.15.152:0/372048817 >> v2:172.21.15.175:3300/0 conn(0x7fbc740d2330 0x7fbc740d2790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:34.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.095+0000 7fbc79200640 1 -- 172.21.15.152:0/372048817 >> v1:172.21.15.175:6789/0 conn(0x7fbc7406e630 legacy=0x7fbc74162540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:34.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.095+0000 7fbc79200640 1 -- 172.21.15.152:0/372048817 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc740cfd70 con 0x7fbc740d0570 2024-04-20T11:27:34.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.096+0000 7fbc79200640 1 --2- 172.21.15.152:0/372048817 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc740d0570 0x7fbc74162c50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fbc740d46c0 tx=0x7fbc6402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ff60bb3451cf0e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:34.096 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.096+0000 7fbc73e00640 1 -- 172.21.15.152:0/372048817 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc6402ca50 con 0x7fbc740d0570 2024-04-20T11:27:34.096 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.097+0000 7fbc73e00640 1 -- 172.21.15.152:0/372048817 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc6402cbf0 con 0x7fbc740d0570 2024-04-20T11:27:34.096 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.097+0000 7fbc73e00640 1 -- 172.21.15.152:0/372048817 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc640343f0 con 0x7fbc740d0570 2024-04-20T11:27:34.097 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.097+0000 7fbc7c600640 1 -- 172.21.15.152:0/372048817 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc740d0570 msgr2=0x7fbc74162c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:34.097 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.097+0000 7fbc7c600640 1 --2- 172.21.15.152:0/372048817 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc740d0570 0x7fbc74162c50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fbc740d46c0 tx=0x7fbc6402beb0 comp rx=0 tx=0).stop 2024-04-20T11:27:34.097 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.097+0000 7fbc7c600640 1 -- 172.21.15.152:0/372048817 shutdown_connections 2024-04-20T11:27:34.097 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.097+0000 7fbc7c600640 1 --2- 172.21.15.152:0/372048817 >> v2:172.21.15.175:3300/0 conn(0x7fbc740d2330 0x7fbc740d2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:34.097 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.097+0000 7fbc7c600640 1 --2- 172.21.15.152:0/372048817 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc740d0570 0x7fbc74162c50 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:34.097 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.097+0000 7fbc7c600640 1 -- 172.21.15.152:0/372048817 >> 172.21.15.152:0/372048817 conn(0x7fbc7408bfd0 msgr2=0x7fbc740d4760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:34.097 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.097+0000 7fbc7c600640 1 -- 172.21.15.152:0/372048817 shutdown_connections 2024-04-20T11:27:34.097 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.097+0000 7fbc7c600640 1 -- 172.21.15.152:0/372048817 wait complete. 2024-04-20T11:27:34.098 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.099+0000 7fbc7c600640 1 Processor -- start 2024-04-20T11:27:34.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.099+0000 7fbc7c600640 1 -- start start 2024-04-20T11:27:34.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.100+0000 7fbc7c600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbc7406e630 0x7fbc7415d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:34.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.100+0000 7fbc7c600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc740d2330 0x7fbc740b5d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:34.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.100+0000 7fbc7c600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbc740b6260 0x7fbc741ea100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:34.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.100+0000 7fbc7c600640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fbc7406ea10 con 0x7fbc740d2330 2024-04-20T11:27:34.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.100+0000 7fbc7c600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbc740d5370 con 0x7fbc740b6260 2024-04-20T11:27:34.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.100+0000 7fbc7c600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fbc74163190 con 0x7fbc7406e630 2024-04-20T11:27:34.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.100+0000 7fbc79200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc740d2330 0x7fbc740b5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:34.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.100+0000 7fbc79200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc740d2330 0x7fbc740b5d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:45200/0 (socket says 172.21.15.152:45200) 2024-04-20T11:27:34.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.100+0000 7fbc79200640 1 -- 172.21.15.152:0/1996960960 learned_addr learned my addr 172.21.15.152:0/1996960960 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:34.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.101+0000 7fbc7a600640 1 --2- 172.21.15.152:0/1996960960 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbc740b6260 0x7fbc741ea100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:34.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.101+0000 7fbc79c00640 1 --2- 172.21.15.152:0/1996960960 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbc7406e630 0x7fbc7415d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:34.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.101+0000 7fbc79200640 1 -- 172.21.15.152:0/1996960960 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbc7406e630 msgr2=0x7fbc7415d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:34.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.101+0000 7fbc79200640 1 --2- 172.21.15.152:0/1996960960 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbc7406e630 0x7fbc7415d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:34.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.101+0000 7fbc79200640 1 -- 172.21.15.152:0/1996960960 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbc740b6260 msgr2=0x7fbc741ea100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:34.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.101+0000 7fbc79200640 1 --2- 172.21.15.152:0/1996960960 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbc740b6260 0x7fbc741ea100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:34.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.101+0000 7fbc79200640 1 -- 172.21.15.152:0/1996960960 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc741e9480 con 0x7fbc740d2330 2024-04-20T11:27:34.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.101+0000 7fbc79c00640 1 --2- 172.21.15.152:0/1996960960 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbc7406e630 0x7fbc7415d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:27:34.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.101+0000 7fbc7a600640 1 --2- 172.21.15.152:0/1996960960 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbc740b6260 0x7fbc741ea100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:27:34.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.101+0000 7fbc79200640 1 --2- 172.21.15.152:0/1996960960 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc740d2330 0x7fbc740b5d20 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fbc64000f80 tx=0x7fbc64004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:34.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.102+0000 7fbc71600640 1 -- 172.21.15.152:0/1996960960 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc64043020 con 0x7fbc740d2330 2024-04-20T11:27:34.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.102+0000 7fbc71600640 1 -- 172.21.15.152:0/1996960960 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc6403e450 con 0x7fbc740d2330 2024-04-20T11:27:34.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.102+0000 7fbc71600640 1 -- 172.21.15.152:0/1996960960 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc64009510 con 0x7fbc740d2330 2024-04-20T11:27:34.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.102+0000 7fbc7c600640 1 -- 172.21.15.152:0/1996960960 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc741e83e0 con 0x7fbc740d2330 2024-04-20T11:27:34.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.102+0000 7fbc7c600640 1 -- 172.21.15.152:0/1996960960 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc741e89a0 con 0x7fbc740d2330 2024-04-20T11:27:34.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.103+0000 7fbc7c600640 1 -- 172.21.15.152:0/1996960960 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc740b6670 con 0x7fbc740d2330 2024-04-20T11:27:34.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.104+0000 7fbc71600640 1 -- 172.21.15.152:0/1996960960 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbc64046020 con 0x7fbc740d2330 2024-04-20T11:27:34.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.105+0000 7fbc71600640 1 --2- 172.21.15.152:0/1996960960 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fbc5806c8f0 0x7fbc5806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:34.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.105+0000 7fbc71600640 1 -- 172.21.15.152:0/1996960960 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fbc640ae370 con 0x7fbc740d2330 2024-04-20T11:27:34.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.105+0000 7fbc79c00640 1 --2- 172.21.15.152:0/1996960960 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fbc5806c8f0 0x7fbc5806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:34.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.106+0000 7fbc79c00640 1 --2- 172.21.15.152:0/1996960960 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fbc5806c8f0 0x7fbc5806edb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fbc740d1790 tx=0x7fbc68009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:34.110 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.111+0000 7fbc71600640 1 -- 172.21.15.152:0/1996960960 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fbc740b6670 con 0x7fbc740d2330 2024-04-20T11:27:34.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.311+0000 7fbc7c600640 1 -- 172.21.15.152:0/1996960960 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fbc740d1e50 con 0x7fbc740d2330 2024-04-20T11:27:34.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.312+0000 7fbc71600640 1 -- 172.21.15.152:0/1996960960 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7fbc6404d030 con 0x7fbc740d2330 2024-04-20T11:27:34.312 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 32 2024-04-20T11:27:34.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.315+0000 7fbc7c600640 1 -- 172.21.15.152:0/1996960960 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fbc5806c8f0 msgr2=0x7fbc5806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:34.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.315+0000 7fbc7c600640 1 --2- 172.21.15.152:0/1996960960 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fbc5806c8f0 0x7fbc5806edb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fbc740d1790 tx=0x7fbc68009040 comp rx=0 tx=0).stop 2024-04-20T11:27:34.315 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.315+0000 7fbc7c600640 1 -- 172.21.15.152:0/1996960960 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc740d2330 msgr2=0x7fbc740b5d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:34.315 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.315+0000 7fbc7c600640 1 --2- 172.21.15.152:0/1996960960 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc740d2330 0x7fbc740b5d20 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fbc64000f80 tx=0x7fbc64004a40 comp rx=0 tx=0).stop 2024-04-20T11:27:34.315 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.315+0000 7fbc7c600640 1 -- 172.21.15.152:0/1996960960 shutdown_connections 2024-04-20T11:27:34.315 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.315+0000 7fbc7c600640 1 --2- 172.21.15.152:0/1996960960 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fbc5806c8f0 0x7fbc5806edb0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:34.315 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.315+0000 7fbc7c600640 1 --2- 172.21.15.152:0/1996960960 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbc740b6260 0x7fbc741ea100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:34.315 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.315+0000 7fbc7c600640 1 --2- 172.21.15.152:0/1996960960 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc740d2330 0x7fbc740b5d20 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:34.315 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.315+0000 7fbc7c600640 1 --2- 172.21.15.152:0/1996960960 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbc7406e630 0x7fbc7415d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:34.315 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.315+0000 7fbc7c600640 1 -- 172.21.15.152:0/1996960960 >> 172.21.15.152:0/1996960960 conn(0x7fbc7408bfd0 msgr2=0x7fbc740b4b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:34.315 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.315+0000 7fbc7c600640 1 -- 172.21.15.152:0/1996960960 shutdown_connections 2024-04-20T11:27:34.315 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.315+0000 7fbc7c600640 1 -- 172.21.15.152:0/1996960960 wait complete. 2024-04-20T11:27:34.334 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T11:27:34.534 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.534+0000 7fef29400640 1 Processor -- start 2024-04-20T11:27:34.534 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.534+0000 7fef29400640 1 -- start start 2024-04-20T11:27:34.535 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.536+0000 7fef29400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fef24152450 0x7fef2415c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:34.536 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.536+0000 7fef29400640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fef240b3520 0x7fef240ae470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:34.536 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.536+0000 7fef29400640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef24152830 con 0x7fef240b30c0 2024-04-20T11:27:34.536 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.536+0000 7fef29400640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fef240b39c0 con 0x7fef24152450 2024-04-20T11:27:34.536 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.536+0000 7fef29400640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fef240b3ca0 con 0x7fef240b3520 2024-04-20T11:27:34.536 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.536+0000 7fef23e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fef24152450 0x7fef2415c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:34.536 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.536+0000 7fef23e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fef24152450 0x7fef2415c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:45204/0 (socket says 172.21.15.152:45204) 2024-04-20T11:27:34.536 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.536+0000 7fef23e00640 1 -- 172.21.15.152:0/2438042788 learned_addr learned my addr 172.21.15.152:0/2438042788 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:34.537 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.536+0000 7fef28a00640 1 --2- 172.21.15.152:0/2438042788 >> v2:172.21.15.175:3300/0 conn(0x7fef240b3520 0x7fef240ae470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:34.537 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.537+0000 7fef23e00640 1 -- 172.21.15.152:0/2438042788 >> v2:172.21.15.175:3300/0 conn(0x7fef240b3520 msgr2=0x7fef240ae470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:34.537 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.537+0000 7fef23e00640 1 --2- 172.21.15.152:0/2438042788 >> v2:172.21.15.175:3300/0 conn(0x7fef240b3520 0x7fef240ae470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:34.537 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.537+0000 7fef23e00640 1 -- 172.21.15.152:0/2438042788 >> v1:172.21.15.175:6789/0 conn(0x7fef240b30c0 legacy=0x7fef2406bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:34.537 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.537+0000 7fef23e00640 1 -- 172.21.15.152:0/2438042788 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef240b3f80 con 0x7fef24152450 2024-04-20T11:27:34.537 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.537+0000 7fef28a00640 1 --2- 172.21.15.152:0/2438042788 >> v2:172.21.15.175:3300/0 conn(0x7fef240b3520 0x7fef240ae470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:27:34.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.538+0000 7fef23e00640 1 --2- 172.21.15.152:0/2438042788 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef24152450 0x7fef2415c590 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fef240afce0 tx=0x7fef1402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=457c14979468c82a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:34.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.538+0000 7fef23400640 1 -- 172.21.15.152:0/2438042788 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef14035020 con 0x7fef24152450 2024-04-20T11:27:34.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.538+0000 7fef23400640 1 -- 172.21.15.152:0/2438042788 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef14031870 con 0x7fef24152450 2024-04-20T11:27:34.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.538+0000 7fef23400640 1 -- 172.21.15.152:0/2438042788 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef14031c40 con 0x7fef24152450 2024-04-20T11:27:34.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.539+0000 7fef29400640 1 -- 172.21.15.152:0/2438042788 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef24152450 msgr2=0x7fef2415c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:34.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.539+0000 7fef29400640 1 --2- 172.21.15.152:0/2438042788 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef24152450 0x7fef2415c590 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fef240afce0 tx=0x7fef1402b910 comp rx=0 tx=0).stop 2024-04-20T11:27:34.539 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.539+0000 7fef29400640 1 -- 172.21.15.152:0/2438042788 shutdown_connections 2024-04-20T11:27:34.539 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.539+0000 7fef29400640 1 --2- 172.21.15.152:0/2438042788 >> v2:172.21.15.175:3300/0 conn(0x7fef240b3520 0x7fef240ae470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:34.539 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.539+0000 7fef29400640 1 --2- 172.21.15.152:0/2438042788 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef24152450 0x7fef2415c590 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:34.539 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.539+0000 7fef29400640 1 -- 172.21.15.152:0/2438042788 >> 172.21.15.152:0/2438042788 conn(0x7fef240b21e0 msgr2=0x7fef240b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:34.539 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.539+0000 7fef29400640 1 -- 172.21.15.152:0/2438042788 shutdown_connections 2024-04-20T11:27:34.539 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.539+0000 7fef29400640 1 -- 172.21.15.152:0/2438042788 wait complete. 2024-04-20T11:27:34.540 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.541+0000 7fef29400640 1 Processor -- start 2024-04-20T11:27:34.540 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.541+0000 7fef29400640 1 -- start start 2024-04-20T11:27:34.543 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.542+0000 7fef29400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef240b30c0 0x7fef2414e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:34.543 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.542+0000 7fef29400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef240b3520 0x7fef24146430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:34.543 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.542+0000 7fef29400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef24152450 0x7fef24153680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:34.543 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.542+0000 7fef29400640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fef2406c4b0 con 0x7fef24152450 2024-04-20T11:27:34.543 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.542+0000 7fef29400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fef240adf00 con 0x7fef240b3520 2024-04-20T11:27:34.543 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.542+0000 7fef29400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fef2406c630 con 0x7fef240b30c0 2024-04-20T11:27:34.543 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.542+0000 7fef28a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef24152450 0x7fef24153680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:34.543 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.542+0000 7fef28a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef24152450 0x7fef24153680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:45210/0 (socket says 172.21.15.152:45210) 2024-04-20T11:27:34.543 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.542+0000 7fef28a00640 1 -- 172.21.15.152:0/2234152965 learned_addr learned my addr 172.21.15.152:0/2234152965 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:34.543 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.543+0000 7fef23e00640 1 --2- 172.21.15.152:0/2234152965 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef240b30c0 0x7fef2414e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:34.543 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.543+0000 7fef1be00640 1 --2- 172.21.15.152:0/2234152965 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef240b3520 0x7fef24146430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:34.543 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.543+0000 7fef28a00640 1 -- 172.21.15.152:0/2234152965 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef240b30c0 msgr2=0x7fef2414e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:34.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.543+0000 7fef28a00640 1 --2- 172.21.15.152:0/2234152965 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef240b30c0 0x7fef2414e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:34.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.543+0000 7fef28a00640 1 -- 172.21.15.152:0/2234152965 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef240b3520 msgr2=0x7fef24146430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:34.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.543+0000 7fef28a00640 1 --2- 172.21.15.152:0/2234152965 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef240b3520 0x7fef24146430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:34.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.543+0000 7fef28a00640 1 -- 172.21.15.152:0/2234152965 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef2414de70 con 0x7fef24152450 2024-04-20T11:27:34.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.543+0000 7fef23e00640 1 --2- 172.21.15.152:0/2234152965 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef240b30c0 0x7fef2414e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:27:34.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.543+0000 7fef1be00640 1 --2- 172.21.15.152:0/2234152965 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef240b3520 0x7fef24146430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:27:34.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.543+0000 7fef28a00640 1 --2- 172.21.15.152:0/2234152965 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef24152450 0x7fef24153680 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fef1c00cef0 tx=0x7fef1c00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:34.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.543+0000 7fef20c00640 1 -- 172.21.15.152:0/2234152965 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef1c01a070 con 0x7fef24152450 2024-04-20T11:27:34.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.543+0000 7fef20c00640 1 -- 172.21.15.152:0/2234152965 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef1c00d860 con 0x7fef24152450 2024-04-20T11:27:34.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.543+0000 7fef20c00640 1 -- 172.21.15.152:0/2234152965 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef1c00dc50 con 0x7fef24152450 2024-04-20T11:27:34.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.543+0000 7fef29400640 1 -- 172.21.15.152:0/2234152965 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef24146970 con 0x7fef24152450 2024-04-20T11:27:34.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.544+0000 7fef29400640 1 -- 172.21.15.152:0/2234152965 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef2414f270 con 0x7fef24152450 2024-04-20T11:27:34.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.545+0000 7fef29400640 1 -- 172.21.15.152:0/2234152965 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef24075e60 con 0x7fef24152450 2024-04-20T11:27:34.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.546+0000 7fef20c00640 1 -- 172.21.15.152:0/2234152965 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fef1c01d070 con 0x7fef24152450 2024-04-20T11:27:34.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.547+0000 7fef20c00640 1 --2- 172.21.15.152:0/2234152965 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fef0006c8f0 0x7fef0006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:34.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.547+0000 7fef20c00640 1 -- 172.21.15.152:0/2234152965 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fef1c08c430 con 0x7fef24152450 2024-04-20T11:27:34.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.547+0000 7fef23e00640 1 --2- 172.21.15.152:0/2234152965 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fef0006c8f0 0x7fef0006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:34.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.548+0000 7fef23e00640 1 --2- 172.21.15.152:0/2234152965 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fef0006c8f0 0x7fef0006edb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fef14036000 tx=0x7fef14008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:34.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.551+0000 7fef20c00640 1 -- 172.21.15.152:0/2234152965 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fef1c05a910 con 0x7fef24152450 2024-04-20T11:27:34.747 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.747+0000 7fef29400640 1 -- 172.21.15.152:0/2234152965 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fef2406f770 con 0x7fef24152450 2024-04-20T11:27:34.747 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.748+0000 7fef20c00640 1 -- 172.21.15.152:0/2234152965 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7fef1c05a210 con 0x7fef24152450 2024-04-20T11:27:34.747 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 64 2024-04-20T11:27:34.750 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.750+0000 7fef29400640 1 -- 172.21.15.152:0/2234152965 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fef0006c8f0 msgr2=0x7fef0006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:34.750 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.750+0000 7fef29400640 1 --2- 172.21.15.152:0/2234152965 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fef0006c8f0 0x7fef0006edb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fef14036000 tx=0x7fef14008030 comp rx=0 tx=0).stop 2024-04-20T11:27:34.750 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.750+0000 7fef29400640 1 -- 172.21.15.152:0/2234152965 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef24152450 msgr2=0x7fef24153680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:34.751 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.750+0000 7fef29400640 1 --2- 172.21.15.152:0/2234152965 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef24152450 0x7fef24153680 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fef1c00cef0 tx=0x7fef1c00a710 comp rx=0 tx=0).stop 2024-04-20T11:27:34.751 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.751+0000 7fef29400640 1 -- 172.21.15.152:0/2234152965 shutdown_connections 2024-04-20T11:27:34.751 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.751+0000 7fef29400640 1 --2- 172.21.15.152:0/2234152965 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fef0006c8f0 0x7fef0006edb0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:34.751 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.751+0000 7fef29400640 1 --2- 172.21.15.152:0/2234152965 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef24152450 0x7fef24153680 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:34.751 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.751+0000 7fef29400640 1 --2- 172.21.15.152:0/2234152965 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef240b3520 0x7fef24146430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:34.751 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.751+0000 7fef29400640 1 --2- 172.21.15.152:0/2234152965 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef240b30c0 0x7fef2414e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:34.751 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.751+0000 7fef29400640 1 -- 172.21.15.152:0/2234152965 >> 172.21.15.152:0/2234152965 conn(0x7fef240b21e0 msgr2=0x7fef2406bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:34.751 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.751+0000 7fef29400640 1 -- 172.21.15.152:0/2234152965 shutdown_connections 2024-04-20T11:27:34.751 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.751+0000 7fef29400640 1 -- 172.21.15.152:0/2234152965 wait complete. 2024-04-20T11:27:34.767 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:27:34.963 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.964+0000 7f7c3d800640 1 Processor -- start 2024-04-20T11:27:34.963 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.964+0000 7f7c3d800640 1 -- start start 2024-04-20T11:27:34.965 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.965+0000 7f7c3d800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f7c380cce30 0x7f7c380b1800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:34.965 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.965+0000 7f7c3d800640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c380b0d90 con 0x7f7c380b2b20 2024-04-20T11:27:34.965 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.965+0000 7f7c3d800640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c380b1d40 con 0x7f7c380cdda0 2024-04-20T11:27:34.965 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.965+0000 7f7c3d800640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f7c380b1370 con 0x7f7c380cce30 2024-04-20T11:27:34.965 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.965+0000 7f7c36a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f7c380cce30 0x7f7c380b1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:34.965 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.966+0000 7f7c36a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f7c380cce30 0x7f7c380b1800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:45224/0 (socket says 172.21.15.152:45224) 2024-04-20T11:27:34.965 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.966+0000 7f7c37400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f7c380b2b20 0x7f7c380cba60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:44732/0 (socket says 172.21.15.152:44732) 2024-04-20T11:27:34.965 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.966+0000 7f7c36a00640 1 -- 172.21.15.152:0/756010363 learned_addr learned my addr 172.21.15.152:0/756010363 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:34.966 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.966+0000 7f7c36a00640 1 -- 172.21.15.152:0/756010363 >> v1:172.21.15.175:6789/0 conn(0x7f7c380cdda0 legacy=0x7f7c380b2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:34.966 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.966+0000 7f7c36a00640 1 -- 172.21.15.152:0/756010363 >> v1:172.21.15.152:6789/0 conn(0x7f7c380b2b20 legacy=0x7f7c380cba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:34.966 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.966+0000 7f7c36a00640 1 -- 172.21.15.152:0/756010363 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c380cd970 con 0x7f7c380cce30 2024-04-20T11:27:34.966 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.966+0000 7f7c36000640 1 -- 172.21.15.152:0/756010363 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1706572680 0 0) 0x7f7c380b0d90 con 0x7f7c380b2b20 2024-04-20T11:27:34.967 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.967+0000 7f7c36a00640 1 --2- 172.21.15.152:0/756010363 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7c380cce30 0x7f7c380b1800 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7c380b14f0 tx=0x7f7c2802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35c42dc9ee2454d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:34.967 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.967+0000 7f7c36000640 1 -- 172.21.15.152:0/756010363 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c28034070 con 0x7f7c380cce30 2024-04-20T11:27:34.967 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.967+0000 7f7c36000640 1 -- 172.21.15.152:0/756010363 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c28030840 con 0x7f7c380cce30 2024-04-20T11:27:34.967 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.967+0000 7f7c36000640 1 -- 172.21.15.152:0/756010363 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c28030c10 con 0x7f7c380cce30 2024-04-20T11:27:34.968 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.968+0000 7f7c3d800640 1 -- 172.21.15.152:0/756010363 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7c380cce30 msgr2=0x7f7c380b1800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:34.968 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.968+0000 7f7c3d800640 1 --2- 172.21.15.152:0/756010363 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7c380cce30 0x7f7c380b1800 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7c380b14f0 tx=0x7f7c2802b980 comp rx=0 tx=0).stop 2024-04-20T11:27:34.968 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.968+0000 7f7c3d800640 1 -- 172.21.15.152:0/756010363 shutdown_connections 2024-04-20T11:27:34.968 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.968+0000 7f7c3d800640 1 --2- 172.21.15.152:0/756010363 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7c380cce30 0x7f7c380b1800 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:34.968 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.968+0000 7f7c3d800640 1 -- 172.21.15.152:0/756010363 >> 172.21.15.152:0/756010363 conn(0x7f7c380b2fc0 msgr2=0x7f7c380b33a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:34.968 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.968+0000 7f7c3d800640 1 -- 172.21.15.152:0/756010363 shutdown_connections 2024-04-20T11:27:34.968 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.968+0000 7f7c3d800640 1 -- 172.21.15.152:0/756010363 wait complete. 2024-04-20T11:27:34.969 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.970+0000 7f7c3d800640 1 Processor -- start 2024-04-20T11:27:34.969 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.970+0000 7f7c3d800640 1 -- start start 2024-04-20T11:27:34.971 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.971+0000 7f7c3d800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c380b2b20 0x7f7c381f8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:34.971 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.971+0000 7f7c3d800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7c380cce30 0x7f7c380c8760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:34.971 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.971+0000 7f7c3d800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7c380cdda0 0x7f7c38155860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:34.971 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.971+0000 7f7c3d800640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f7c380cd210 con 0x7f7c380cdda0 2024-04-20T11:27:34.971 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.971+0000 7f7c3d800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7c380b1000 con 0x7f7c380b2b20 2024-04-20T11:27:34.971 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.971+0000 7f7c3d800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7c380cebf0 con 0x7f7c380cce30 2024-04-20T11:27:34.972 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.971+0000 7f7c37400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7c380cdda0 0x7f7c38155860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:34.972 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.971+0000 7f7c37400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7c380cdda0 0x7f7c38155860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:45230/0 (socket says 172.21.15.152:45230) 2024-04-20T11:27:34.972 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.971+0000 7f7c37400640 1 -- 172.21.15.152:0/4205004482 learned_addr learned my addr 172.21.15.152:0/4205004482 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:34.972 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.972+0000 7f7c36a00640 1 --2- 172.21.15.152:0/4205004482 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c380b2b20 0x7f7c381f8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:34.972 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.972+0000 7f7c2e000640 1 --2- 172.21.15.152:0/4205004482 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7c380cce30 0x7f7c380c8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:34.972 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.972+0000 7f7c37400640 1 -- 172.21.15.152:0/4205004482 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7c380cce30 msgr2=0x7f7c380c8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:34.972 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.972+0000 7f7c37400640 1 --2- 172.21.15.152:0/4205004482 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7c380cce30 0x7f7c380c8760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:34.972 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.972+0000 7f7c37400640 1 -- 172.21.15.152:0/4205004482 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c380b2b20 msgr2=0x7f7c381f8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:34.972 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.972+0000 7f7c37400640 1 --2- 172.21.15.152:0/4205004482 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c380b2b20 0x7f7c381f8810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:34.972 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.972+0000 7f7c37400640 1 -- 172.21.15.152:0/4205004482 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c380b6960 con 0x7f7c380cdda0 2024-04-20T11:27:34.972 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.972+0000 7f7c37400640 1 --2- 172.21.15.152:0/4205004482 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7c380cdda0 0x7f7c38155860 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7c3000c7b0 tx=0x7f7c3000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:34.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.972+0000 7f7c2f400640 1 -- 172.21.15.152:0/4205004482 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c30018070 con 0x7f7c380cdda0 2024-04-20T11:27:34.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.972+0000 7f7c2f400640 1 -- 172.21.15.152:0/4205004482 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c3000a640 con 0x7f7c380cdda0 2024-04-20T11:27:34.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.972+0000 7f7c2f400640 1 -- 172.21.15.152:0/4205004482 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c3000ce60 con 0x7f7c380cdda0 2024-04-20T11:27:34.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.972+0000 7f7c3d800640 1 -- 172.21.15.152:0/4205004482 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c380c69c0 con 0x7f7c380cdda0 2024-04-20T11:27:34.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.972+0000 7f7c3d800640 1 -- 172.21.15.152:0/4205004482 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c380c6f80 con 0x7f7c380cdda0 2024-04-20T11:27:34.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.974+0000 7f7c3d800640 1 -- 172.21.15.152:0/4205004482 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c380689f0 con 0x7f7c380cdda0 2024-04-20T11:27:34.974 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.975+0000 7f7c2f400640 1 -- 172.21.15.152:0/4205004482 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7c3001b070 con 0x7f7c380cdda0 2024-04-20T11:27:34.975 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.976+0000 7f7c2f400640 1 --2- 172.21.15.152:0/4205004482 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f7c1406c8f0 0x7f7c1406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:34.975 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.976+0000 7f7c2f400640 1 -- 172.21.15.152:0/4205004482 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f7c3008b100 con 0x7f7c380cdda0 2024-04-20T11:27:34.975 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.976+0000 7f7c36a00640 1 --2- 172.21.15.152:0/4205004482 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f7c1406c8f0 0x7f7c1406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:34.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.977+0000 7f7c36a00640 1 --2- 172.21.15.152:0/4205004482 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f7c1406c8f0 0x7f7c1406edb0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7c28036000 tx=0x7f7c28008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:34.979 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:34.980+0000 7f7c2f400640 1 -- 172.21.15.152:0/4205004482 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f7c30059490 con 0x7f7c380cdda0 2024-04-20T11:27:35.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.169+0000 7f7c3d800640 1 -- 172.21.15.152:0/4205004482 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7c380709d0 con 0x7f7c380cdda0 2024-04-20T11:27:35.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.171+0000 7f7c2f400640 1 -- 172.21.15.152:0/4205004482 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16665 (secure 0 0 0) 0x7f7c30058d90 con 0x7f7c380cdda0 2024-04-20T11:27:35.171 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:27:35.171 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":20,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:26:59.675971+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:26:56.261424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T11:26:57.079286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:27:35.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.173+0000 7f7c3d800640 1 -- 172.21.15.152:0/4205004482 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f7c1406c8f0 msgr2=0x7f7c1406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:35.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.173+0000 7f7c3d800640 1 --2- 172.21.15.152:0/4205004482 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f7c1406c8f0 0x7f7c1406edb0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7c28036000 tx=0x7f7c28008030 comp rx=0 tx=0).stop 2024-04-20T11:27:35.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.173+0000 7f7c3d800640 1 -- 172.21.15.152:0/4205004482 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7c380cdda0 msgr2=0x7f7c38155860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:35.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.173+0000 7f7c3d800640 1 --2- 172.21.15.152:0/4205004482 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7c380cdda0 0x7f7c38155860 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7c3000c7b0 tx=0x7f7c3000cc70 comp rx=0 tx=0).stop 2024-04-20T11:27:35.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.174+0000 7f7c3d800640 1 -- 172.21.15.152:0/4205004482 shutdown_connections 2024-04-20T11:27:35.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.174+0000 7f7c3d800640 1 --2- 172.21.15.152:0/4205004482 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f7c1406c8f0 0x7f7c1406edb0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:35.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.174+0000 7f7c3d800640 1 --2- 172.21.15.152:0/4205004482 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7c380cdda0 0x7f7c38155860 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:35.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.174+0000 7f7c3d800640 1 --2- 172.21.15.152:0/4205004482 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7c380cce30 0x7f7c380c8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:35.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.174+0000 7f7c3d800640 1 --2- 172.21.15.152:0/4205004482 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c380b2b20 0x7f7c381f8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:35.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.174+0000 7f7c3d800640 1 -- 172.21.15.152:0/4205004482 >> 172.21.15.152:0/4205004482 conn(0x7f7c380b2fc0 msgr2=0x7f7c380b2410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:35.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.174+0000 7f7c3d800640 1 -- 172.21.15.152:0/4205004482 shutdown_connections 2024-04-20T11:27:35.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.174+0000 7f7c3d800640 1 -- 172.21.15.152:0/4205004482 wait complete. 2024-04-20T11:27:35.190 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T11:26:47.246571+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T11:26:56.261424+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T11:26:57.079286+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T11:27:35.191 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T11:27:35.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.382+0000 7fb5aee00640 1 Processor -- start 2024-04-20T11:27:35.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.382+0000 7fb5aee00640 1 -- start start 2024-04-20T11:27:35.384 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.384+0000 7fb5aee00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb5a815b510 0x7fb5a815fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:35.384 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.384+0000 7fb5aee00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fb5a80cfc40 0x7fb5a80b3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:35.384 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.384+0000 7fb5aee00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5a815f1c0 con 0x7fb5a80b2170 2024-04-20T11:27:35.384 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.384+0000 7fb5aee00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fb5a806f420 con 0x7fb5a80cfc40 2024-04-20T11:27:35.384 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.384+0000 7fb5aee00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb5a80b4430 con 0x7fb5a815b510 2024-04-20T11:27:35.384 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.384+0000 7fb5a7400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fb5a80cfc40 0x7fb5a80b3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:35.384 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.384+0000 7fb5a7400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fb5a80cfc40 0x7fb5a80b3ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:45238/0 (socket says 172.21.15.152:45238) 2024-04-20T11:27:35.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.384+0000 7fb5a7400640 1 -- 172.21.15.152:0/2488843617 learned_addr learned my addr 172.21.15.152:0/2488843617 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:35.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.385+0000 7fb5a7e00640 1 --2- 172.21.15.152:0/2488843617 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb5a815b510 0x7fb5a815fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:35.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.385+0000 7fb5a7400640 1 -- 172.21.15.152:0/2488843617 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb5a815b510 msgr2=0x7fb5a815fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:35.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.385+0000 7fb5a7400640 1 --2- 172.21.15.152:0/2488843617 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb5a815b510 0x7fb5a815fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:35.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.385+0000 7fb5a7400640 1 -- 172.21.15.152:0/2488843617 >> v1:172.21.15.175:6789/0 conn(0x7fb5a80b2170 legacy=0x7fb5a80b4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:35.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.385+0000 7fb5a7400640 1 -- 172.21.15.152:0/2488843617 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5a806f670 con 0x7fb5a80cfc40 2024-04-20T11:27:35.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.386+0000 7fb5a7400640 1 --2- 172.21.15.152:0/2488843617 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb5a80cfc40 0x7fb5a80b3ef0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb59400bb10 tx=0x7fb59402e0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd7194bfe1f60f03 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:35.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.386+0000 7fb5a6a00640 1 -- 172.21.15.152:0/2488843617 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb59403b070 con 0x7fb5a80cfc40 2024-04-20T11:27:35.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.386+0000 7fb5a6a00640 1 -- 172.21.15.152:0/2488843617 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb594036440 con 0x7fb5a80cfc40 2024-04-20T11:27:35.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.386+0000 7fb5a6a00640 1 -- 172.21.15.152:0/2488843617 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb594036810 con 0x7fb5a80cfc40 2024-04-20T11:27:35.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.387+0000 7fb5aee00640 1 -- 172.21.15.152:0/2488843617 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb5a80cfc40 msgr2=0x7fb5a80b3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:35.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.387+0000 7fb5aee00640 1 --2- 172.21.15.152:0/2488843617 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb5a80cfc40 0x7fb5a80b3ef0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb59400bb10 tx=0x7fb59402e0c0 comp rx=0 tx=0).stop 2024-04-20T11:27:35.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.387+0000 7fb5aee00640 1 -- 172.21.15.152:0/2488843617 shutdown_connections 2024-04-20T11:27:35.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.387+0000 7fb5aee00640 1 --2- 172.21.15.152:0/2488843617 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb5a80cfc40 0x7fb5a80b3ef0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:35.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.387+0000 7fb5aee00640 1 --2- 172.21.15.152:0/2488843617 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb5a815b510 0x7fb5a815fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:35.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.387+0000 7fb5aee00640 1 -- 172.21.15.152:0/2488843617 >> 172.21.15.152:0/2488843617 conn(0x7fb5a806b980 msgr2=0x7fb5a806bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:35.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.387+0000 7fb5aee00640 1 -- 172.21.15.152:0/2488843617 shutdown_connections 2024-04-20T11:27:35.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.387+0000 7fb5aee00640 1 -- 172.21.15.152:0/2488843617 wait complete. 2024-04-20T11:27:35.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.388+0000 7fb5aee00640 1 Processor -- start 2024-04-20T11:27:35.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.389+0000 7fb5aee00640 1 -- start start 2024-04-20T11:27:35.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.390+0000 7fb5aee00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb5a80b2170 0x7fb5a8148320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:35.391 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.390+0000 7fb5aee00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb5a80cfc40 0x7fb5a8148860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:35.391 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.390+0000 7fb5aee00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5a815b510 0x7fb5a8145580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:35.391 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.390+0000 7fb5aee00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fb5a815b8f0 con 0x7fb5a80cfc40 2024-04-20T11:27:35.391 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.390+0000 7fb5aee00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb5a815f710 con 0x7fb5a815b510 2024-04-20T11:27:35.391 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.390+0000 7fb5aee00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb5a806f950 con 0x7fb5a80b2170 2024-04-20T11:27:35.391 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.390+0000 7fb5a7400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb5a80cfc40 0x7fb5a8148860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:35.391 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.390+0000 7fb5a7400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb5a80cfc40 0x7fb5a8148860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:45254/0 (socket says 172.21.15.152:45254) 2024-04-20T11:27:35.391 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.390+0000 7fb5a7400640 1 -- 172.21.15.152:0/1776182921 learned_addr learned my addr 172.21.15.152:0/1776182921 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:35.391 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.390+0000 7fb5a7400640 1 -- 172.21.15.152:0/1776182921 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb5a80b2170 msgr2=0x7fb5a8148320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:35.391 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.390+0000 7fb5a7400640 1 --2- 172.21.15.152:0/1776182921 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb5a80b2170 0x7fb5a8148320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:35.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.390+0000 7fb5a7400640 1 -- 172.21.15.152:0/1776182921 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5a815b510 msgr2=0x7fb5a8145580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:35.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.390+0000 7fb5a7400640 1 --2- 172.21.15.152:0/1776182921 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5a815b510 0x7fb5a8145580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:35.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.390+0000 7fb5a7400640 1 -- 172.21.15.152:0/1776182921 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5a80ae6e0 con 0x7fb5a80cfc40 2024-04-20T11:27:35.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.391+0000 7fb5a7400640 1 --2- 172.21.15.152:0/1776182921 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb5a80cfc40 0x7fb5a8148860 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb594000c00 tx=0x7fb5940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:35.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.391+0000 7fb59fe00640 1 -- 172.21.15.152:0/1776182921 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb59403b030 con 0x7fb5a80cfc40 2024-04-20T11:27:35.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.391+0000 7fb59fe00640 1 -- 172.21.15.152:0/1776182921 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb59403d070 con 0x7fb5a80cfc40 2024-04-20T11:27:35.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.391+0000 7fb59fe00640 1 -- 172.21.15.152:0/1776182921 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb594034ba0 con 0x7fb5a80cfc40 2024-04-20T11:27:35.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.391+0000 7fb5aee00640 1 -- 172.21.15.152:0/1776182921 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5a80ae9c0 con 0x7fb5a80cfc40 2024-04-20T11:27:35.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.391+0000 7fb5aee00640 1 -- 172.21.15.152:0/1776182921 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5a80aef80 con 0x7fb5a80cfc40 2024-04-20T11:27:35.393 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.392+0000 7fb5aee00640 1 -- 172.21.15.152:0/1776182921 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5a80af1c0 con 0x7fb5a80cfc40 2024-04-20T11:27:35.393 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.393+0000 7fb59fe00640 1 -- 172.21.15.152:0/1776182921 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb594048060 con 0x7fb5a80cfc40 2024-04-20T11:27:35.394 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.394+0000 7fb59fe00640 1 --2- 172.21.15.152:0/1776182921 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fb58806c8f0 0x7fb58806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:35.394 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.394+0000 7fb59fe00640 1 -- 172.21.15.152:0/1776182921 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb5940ad910 con 0x7fb5a80cfc40 2024-04-20T11:27:35.394 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.394+0000 7fb5a7e00640 1 --2- 172.21.15.152:0/1776182921 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fb58806c8f0 0x7fb58806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:35.394 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.395+0000 7fb5a7e00640 1 --2- 172.21.15.152:0/1776182921 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fb58806c8f0 0x7fb58806edb0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb59800a810 tx=0x7fb598008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:35.397 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.398+0000 7fb59fe00640 1 -- 172.21.15.152:0/1776182921 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fb5a80af1c0 con 0x7fb5a80cfc40 2024-04-20T11:27:35.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.584+0000 7fb5aee00640 1 -- 172.21.15.152:0/1776182921 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb5a80689f0 con 0x7fb5a80cfc40 2024-04-20T11:27:35.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.585+0000 7fb59fe00640 1 -- 172.21.15.152:0/1776182921 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7fb59407bd30 con 0x7fb5a80cfc40 2024-04-20T11:27:35.585 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 1 2024-04-20T11:27:35.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.587+0000 7fb5aee00640 1 -- 172.21.15.152:0/1776182921 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fb58806c8f0 msgr2=0x7fb58806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:35.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.587+0000 7fb5aee00640 1 --2- 172.21.15.152:0/1776182921 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fb58806c8f0 0x7fb58806edb0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb59800a810 tx=0x7fb598008040 comp rx=0 tx=0).stop 2024-04-20T11:27:35.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.587+0000 7fb5aee00640 1 -- 172.21.15.152:0/1776182921 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb5a80cfc40 msgr2=0x7fb5a8148860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:35.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.587+0000 7fb5aee00640 1 --2- 172.21.15.152:0/1776182921 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb5a80cfc40 0x7fb5a8148860 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb594000c00 tx=0x7fb5940046c0 comp rx=0 tx=0).stop 2024-04-20T11:27:35.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.587+0000 7fb5aee00640 1 -- 172.21.15.152:0/1776182921 shutdown_connections 2024-04-20T11:27:35.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.587+0000 7fb5aee00640 1 --2- 172.21.15.152:0/1776182921 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fb58806c8f0 0x7fb58806edb0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:35.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.587+0000 7fb5aee00640 1 --2- 172.21.15.152:0/1776182921 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5a815b510 0x7fb5a8145580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:35.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.587+0000 7fb5aee00640 1 --2- 172.21.15.152:0/1776182921 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb5a80cfc40 0x7fb5a8148860 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:35.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.587+0000 7fb5aee00640 1 --2- 172.21.15.152:0/1776182921 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb5a80b2170 0x7fb5a8148320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:35.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.587+0000 7fb5aee00640 1 -- 172.21.15.152:0/1776182921 >> 172.21.15.152:0/1776182921 conn(0x7fb5a806b980 msgr2=0x7fb5a80b14f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:35.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.588+0000 7fb5aee00640 1 -- 172.21.15.152:0/1776182921 shutdown_connections 2024-04-20T11:27:35.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.588+0000 7fb5aee00640 1 -- 172.21.15.152:0/1776182921 wait complete. 2024-04-20T11:27:35.604 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T11:27:35.801 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.801+0000 7fc711200640 1 Processor -- start 2024-04-20T11:27:35.801 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.801+0000 7fc711200640 1 -- start start 2024-04-20T11:27:35.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.802+0000 7fc711200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fc70c15eb50 0x7fc70c0d0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:35.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.802+0000 7fc711200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fc70c163230 0x7fc70c0d2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:35.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.802+0000 7fc711200640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc70c0d19c0 con 0x7fc70c0d1440 2024-04-20T11:27:35.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.802+0000 7fc711200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fc70c162750 con 0x7fc70c15eb50 2024-04-20T11:27:35.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.802+0000 7fc711200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fc70c162a30 con 0x7fc70c163230 2024-04-20T11:27:35.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.803+0000 7fc70b400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fc70c15eb50 0x7fc70c0d0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:35.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.803+0000 7fc70be00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fc70c0d1440 0x7fc70c15ef30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:39936/0 (socket says 172.21.15.152:39936) 2024-04-20T11:27:35.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.803+0000 7fc70be00640 1 -- 172.21.15.152:0/3908272799 learned_addr learned my addr 172.21.15.152:0/3908272799 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:27:35.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.803+0000 7fc70aa00640 1 --2- 172.21.15.152:0/3908272799 >> v2:172.21.15.175:3300/0 conn(0x7fc70c163230 0x7fc70c0d2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:35.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.803+0000 7fc70a000640 1 -- 172.21.15.152:0/3908272799 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1161808626 0 0) 0x7fc70c0d19c0 con 0x7fc70c0d1440 2024-04-20T11:27:35.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.803+0000 7fc70b400640 1 -- 172.21.15.152:0/3908272799 >> v2:172.21.15.175:3300/0 conn(0x7fc70c163230 msgr2=0x7fc70c0d2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:35.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.803+0000 7fc70b400640 1 --2- 172.21.15.152:0/3908272799 >> v2:172.21.15.175:3300/0 conn(0x7fc70c163230 0x7fc70c0d2e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:35.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.803+0000 7fc70b400640 1 -- 172.21.15.152:0/3908272799 >> v1:172.21.15.152:6789/0 conn(0x7fc70c0d1440 legacy=0x7fc70c15ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:35.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.803+0000 7fc70b400640 1 -- 172.21.15.152:0/3908272799 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc70c162d10 con 0x7fc70c15eb50 2024-04-20T11:27:35.805 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.805+0000 7fc70b400640 1 --2- 172.21.15.152:0/3908272799 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc70c15eb50 0x7fc70c0d0f00 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc70c08df30 tx=0x7fc6fc02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13b2d58734f66587 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:35.805 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.805+0000 7fc70a000640 1 -- 172.21.15.152:0/3908272799 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6fc0315a0 con 0x7fc70c15eb50 2024-04-20T11:27:35.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.805+0000 7fc70a000640 1 -- 172.21.15.152:0/3908272799 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6fc037070 con 0x7fc70c15eb50 2024-04-20T11:27:35.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.805+0000 7fc70a000640 1 -- 172.21.15.152:0/3908272799 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6fc031960 con 0x7fc70c15eb50 2024-04-20T11:27:35.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.805+0000 7fc711200640 1 -- 172.21.15.152:0/3908272799 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc70c15eb50 msgr2=0x7fc70c0d0f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:35.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.806+0000 7fc711200640 1 --2- 172.21.15.152:0/3908272799 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc70c15eb50 0x7fc70c0d0f00 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc70c08df30 tx=0x7fc6fc02b840 comp rx=0 tx=0).stop 2024-04-20T11:27:35.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.806+0000 7fc711200640 1 -- 172.21.15.152:0/3908272799 shutdown_connections 2024-04-20T11:27:35.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.806+0000 7fc711200640 1 --2- 172.21.15.152:0/3908272799 >> v2:172.21.15.175:3300/0 conn(0x7fc70c163230 0x7fc70c0d2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:35.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.806+0000 7fc711200640 1 --2- 172.21.15.152:0/3908272799 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc70c15eb50 0x7fc70c0d0f00 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:35.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.806+0000 7fc711200640 1 -- 172.21.15.152:0/3908272799 >> 172.21.15.152:0/3908272799 conn(0x7fc70c08bfd0 msgr2=0x7fc70c0d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:35.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.806+0000 7fc711200640 1 -- 172.21.15.152:0/3908272799 shutdown_connections 2024-04-20T11:27:35.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.806+0000 7fc711200640 1 -- 172.21.15.152:0/3908272799 wait complete. 2024-04-20T11:27:35.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.807+0000 7fc711200640 1 Processor -- start 2024-04-20T11:27:35.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.808+0000 7fc711200640 1 -- start start 2024-04-20T11:27:35.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.808+0000 7fc711200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc70c0d1440 0x7fc70c156810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:35.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.809+0000 7fc711200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc70c15eb50 0x7fc70c0b3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:35.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.809+0000 7fc711200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc70c163230 0x7fc70c158750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:35.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.809+0000 7fc711200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fc70c0d3550 con 0x7fc70c0d1440 2024-04-20T11:27:35.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.809+0000 7fc711200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc70c0d33d0 con 0x7fc70c15eb50 2024-04-20T11:27:35.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.809+0000 7fc711200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc70c0d36d0 con 0x7fc70c163230 2024-04-20T11:27:35.809 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.809+0000 7fc70b400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc70c0d1440 0x7fc70c156810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:35.809 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.809+0000 7fc70b400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc70c0d1440 0x7fc70c156810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35132/0 (socket says 172.21.15.152:35132) 2024-04-20T11:27:35.809 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.809+0000 7fc70b400640 1 -- 172.21.15.152:0/912238559 learned_addr learned my addr 172.21.15.152:0/912238559 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:35.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.809+0000 7fc70b400640 1 -- 172.21.15.152:0/912238559 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc70c163230 msgr2=0x7fc70c158750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:35.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.809+0000 7fc70aa00640 1 --2- 172.21.15.152:0/912238559 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc70c15eb50 0x7fc70c0b3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:35.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.809+0000 7fc70b400640 1 --2- 172.21.15.152:0/912238559 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc70c163230 0x7fc70c158750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:35.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.809+0000 7fc70b400640 1 -- 172.21.15.152:0/912238559 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc70c15eb50 msgr2=0x7fc70c0b3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:35.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.809+0000 7fc70b400640 1 --2- 172.21.15.152:0/912238559 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc70c15eb50 0x7fc70c0b3fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:35.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.809+0000 7fc70b400640 1 -- 172.21.15.152:0/912238559 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc70c0b4a10 con 0x7fc70c0d1440 2024-04-20T11:27:35.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.810+0000 7fc70b400640 1 --2- 172.21.15.152:0/912238559 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc70c0d1440 0x7fc70c156810 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc000c00 tx=0x7fc6fc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:35.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.810+0000 7fc703400640 1 -- 172.21.15.152:0/912238559 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6fc004230 con 0x7fc70c0d1440 2024-04-20T11:27:35.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.810+0000 7fc703400640 1 -- 172.21.15.152:0/912238559 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6fc0043d0 con 0x7fc70c0d1440 2024-04-20T11:27:35.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.810+0000 7fc711200640 1 -- 172.21.15.152:0/912238559 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc70c0b4cf0 con 0x7fc70c0d1440 2024-04-20T11:27:35.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.810+0000 7fc703400640 1 -- 172.21.15.152:0/912238559 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6fc003850 con 0x7fc70c0d1440 2024-04-20T11:27:35.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.810+0000 7fc711200640 1 -- 172.21.15.152:0/912238559 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc70c156d50 con 0x7fc70c0d1440 2024-04-20T11:27:35.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.811+0000 7fc711200640 1 -- 172.21.15.152:0/912238559 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc70c075e60 con 0x7fc70c0d1440 2024-04-20T11:27:35.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.812+0000 7fc703400640 1 -- 172.21.15.152:0/912238559 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc6fc0039f0 con 0x7fc70c0d1440 2024-04-20T11:27:35.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.814+0000 7fc703400640 1 --2- 172.21.15.152:0/912238559 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc6ec06c8f0 0x7fc6ec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:35.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.814+0000 7fc703400640 1 -- 172.21.15.152:0/912238559 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc6fc02f030 con 0x7fc70c0d1440 2024-04-20T11:27:35.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.814+0000 7fc70aa00640 1 --2- 172.21.15.152:0/912238559 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc6ec06c8f0 0x7fc6ec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:35.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.815+0000 7fc70aa00640 1 --2- 172.21.15.152:0/912238559 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc6ec06c8f0 0x7fc6ec06edb0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc70c159430 tx=0x7fc6f8008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:35.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:35.818+0000 7fc703400640 1 -- 172.21.15.152:0/912238559 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fc6fc041070 con 0x7fc70c0d1440 2024-04-20T11:27:36.015 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.016+0000 7fc711200640 1 -- 172.21.15.152:0/912238559 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc70c08c680 con 0x7fc70c0d1440 2024-04-20T11:27:36.016 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.017+0000 7fc703400640 1 -- 172.21.15.152:0/912238559 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7fc6fc044340 con 0x7fc70c0d1440 2024-04-20T11:27:36.016 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 32 2024-04-20T11:27:36.019 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.019+0000 7fc711200640 1 -- 172.21.15.152:0/912238559 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc6ec06c8f0 msgr2=0x7fc6ec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:36.019 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.019+0000 7fc711200640 1 --2- 172.21.15.152:0/912238559 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc6ec06c8f0 0x7fc6ec06edb0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc70c159430 tx=0x7fc6f8008040 comp rx=0 tx=0).stop 2024-04-20T11:27:36.019 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.019+0000 7fc711200640 1 -- 172.21.15.152:0/912238559 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc70c0d1440 msgr2=0x7fc70c156810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:36.019 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.019+0000 7fc711200640 1 --2- 172.21.15.152:0/912238559 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc70c0d1440 0x7fc70c156810 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc000c00 tx=0x7fc6fc0046c0 comp rx=0 tx=0).stop 2024-04-20T11:27:36.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.019+0000 7fc711200640 1 -- 172.21.15.152:0/912238559 shutdown_connections 2024-04-20T11:27:36.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.019+0000 7fc711200640 1 --2- 172.21.15.152:0/912238559 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc6ec06c8f0 0x7fc6ec06edb0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:36.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.019+0000 7fc711200640 1 --2- 172.21.15.152:0/912238559 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc70c163230 0x7fc70c158750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:36.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.019+0000 7fc711200640 1 --2- 172.21.15.152:0/912238559 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc70c15eb50 0x7fc70c0b3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:36.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.019+0000 7fc711200640 1 --2- 172.21.15.152:0/912238559 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc70c0d1440 0x7fc70c156810 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:36.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.019+0000 7fc711200640 1 -- 172.21.15.152:0/912238559 >> 172.21.15.152:0/912238559 conn(0x7fc70c08bfd0 msgr2=0x7fc70c15ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:36.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.020+0000 7fc711200640 1 -- 172.21.15.152:0/912238559 shutdown_connections 2024-04-20T11:27:36.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.020+0000 7fc711200640 1 -- 172.21.15.152:0/912238559 wait complete. 2024-04-20T11:27:36.036 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T11:27:36.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.235+0000 7ff718e00640 1 Processor -- start 2024-04-20T11:27:36.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.236+0000 7ff718e00640 1 -- start start 2024-04-20T11:27:36.237 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.237+0000 7ff718e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff714162bd0 0x7ff71406f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:36.237 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.237+0000 7ff718e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ff7140b4890 0x7ff71406bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:36.237 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.237+0000 7ff718e00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff71406dc60 con 0x7ff71406c2e0 2024-04-20T11:27:36.237 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.237+0000 7ff718e00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7ff71406c750 con 0x7ff7140b4890 2024-04-20T11:27:36.237 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.237+0000 7ff718e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff71406e870 con 0x7ff714162bd0 2024-04-20T11:27:36.237 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.237+0000 7ff713e00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7ff71406c2e0 0x7ff7140b3690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:39938/0 (socket says 172.21.15.152:39938) 2024-04-20T11:27:36.237 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.237+0000 7ff713e00640 1 -- 172.21.15.152:0/2006670903 learned_addr learned my addr 172.21.15.152:0/2006670903 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:27:36.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.238+0000 7ff70aa00640 1 --2- 172.21.15.152:0/2006670903 >> v2:172.21.15.175:3300/0 conn(0x7ff7140b4890 0x7ff71406bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:36.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.238+0000 7ff713400640 1 --2- 172.21.15.152:0/2006670903 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff714162bd0 0x7ff71406f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:36.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.238+0000 7ff712a00640 1 -- 172.21.15.152:0/2006670903 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4271955736 0 0) 0x7ff71406dc60 con 0x7ff71406c2e0 2024-04-20T11:27:36.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.238+0000 7ff712a00640 1 -- 172.21.15.152:0/2006670903 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff6f8003520 con 0x7ff71406c2e0 2024-04-20T11:27:36.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.238+0000 7ff712a00640 1 -- 172.21.15.152:0/2006670903 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2048457092 0 0) 0x7ff6f8003520 con 0x7ff71406c2e0 2024-04-20T11:27:36.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.238+0000 7ff712a00640 1 -- 172.21.15.152:0/2006670903 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff714162bd0 msgr2=0x7ff71406f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:36.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.238+0000 7ff712a00640 1 --2- 172.21.15.152:0/2006670903 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff714162bd0 0x7ff71406f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:36.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.238+0000 7ff712a00640 1 -- 172.21.15.152:0/2006670903 >> v2:172.21.15.175:3300/0 conn(0x7ff7140b4890 msgr2=0x7ff71406bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:36.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.238+0000 7ff712a00640 1 --2- 172.21.15.152:0/2006670903 >> v2:172.21.15.175:3300/0 conn(0x7ff7140b4890 0x7ff71406bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:36.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.238+0000 7ff712a00640 1 -- 172.21.15.152:0/2006670903 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff71406df40 con 0x7ff71406c2e0 2024-04-20T11:27:36.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.238+0000 7ff712a00640 1 -- 172.21.15.152:0/2006670903 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7ff70c0019f0 con 0x7ff71406c2e0 2024-04-20T11:27:36.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.238+0000 7ff712a00640 1 -- 172.21.15.152:0/2006670903 >> v1:172.21.15.152:6789/0 conn(0x7ff71406c2e0 legacy=0x7ff7140b3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:36.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.238+0000 7ff712a00640 1 --2- 172.21.15.152:0/2006670903 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff6f80039b0 0x7ff6f8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:36.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.238+0000 7ff712a00640 1 --2- 172.21.15.152:0/2006670903 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff714162bd0 0x7ff6f8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:36.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.238+0000 7ff712a00640 1 --2- 172.21.15.152:0/2006670903 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff6f8004d40 0x7ff6f80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:36.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.238+0000 7ff712a00640 1 -- 172.21.15.152:0/2006670903 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7ff71406c750 con 0x7ff6f8004d40 2024-04-20T11:27:36.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.238+0000 7ff712a00640 1 -- 172.21.15.152:0/2006670903 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff71406e870 con 0x7ff6f80039b0 2024-04-20T11:27:36.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.238+0000 7ff712a00640 1 -- 172.21.15.152:0/2006670903 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff6f800b9c0 con 0x7ff714162bd0 2024-04-20T11:27:36.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.239+0000 7ff713e00640 1 --2- 172.21.15.152:0/2006670903 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff6f8004d40 0x7ff6f80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:36.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.239+0000 7ff713400640 1 --2- 172.21.15.152:0/2006670903 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff6f80039b0 0x7ff6f8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:36.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.239+0000 7ff70aa00640 1 --2- 172.21.15.152:0/2006670903 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff714162bd0 0x7ff6f8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:36.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.239+0000 7ff713e00640 1 -- 172.21.15.152:0/2006670903 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff714162bd0 msgr2=0x7ff6f8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:36.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.239+0000 7ff713e00640 1 --2- 172.21.15.152:0/2006670903 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff714162bd0 0x7ff6f8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:36.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.239+0000 7ff713e00640 1 -- 172.21.15.152:0/2006670903 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff6f80039b0 msgr2=0x7ff6f8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:36.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.239+0000 7ff713e00640 1 --2- 172.21.15.152:0/2006670903 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff6f80039b0 0x7ff6f8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:36.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.239+0000 7ff713e00640 1 -- 172.21.15.152:0/2006670903 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6f8003520 con 0x7ff6f8004d40 2024-04-20T11:27:36.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.240+0000 7ff713e00640 1 --2- 172.21.15.152:0/2006670903 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff6f8004d40 0x7ff6f80051c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff70c00a920 tx=0x7ff6f8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa846e378eb0fea4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:36.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.240+0000 7ff712a00640 1 -- 172.21.15.152:0/2006670903 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff70c00a4c0 con 0x7ff6f8004d40 2024-04-20T11:27:36.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.241+0000 7ff712a00640 1 -- 172.21.15.152:0/2006670903 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff70c034650 con 0x7ff6f8004d40 2024-04-20T11:27:36.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.241+0000 7ff712a00640 1 -- 172.21.15.152:0/2006670903 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff70c034a40 con 0x7ff6f8004d40 2024-04-20T11:27:36.243 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.241+0000 7ff718e00640 1 -- 172.21.15.152:0/2006670903 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff6f8004d40 msgr2=0x7ff6f80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:36.243 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.241+0000 7ff718e00640 1 --2- 172.21.15.152:0/2006670903 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff6f8004d40 0x7ff6f80051c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff70c00a920 tx=0x7ff6f8005830 comp rx=0 tx=0).stop 2024-04-20T11:27:36.243 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.241+0000 7ff713e00640 1 -- 172.21.15.152:0/2006670903 reap_dead start 2024-04-20T11:27:36.243 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.241+0000 7ff718e00640 1 -- 172.21.15.152:0/2006670903 shutdown_connections 2024-04-20T11:27:36.243 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.241+0000 7ff718e00640 1 -- 172.21.15.152:0/2006670903 >> 172.21.15.152:0/2006670903 conn(0x7ff7140b21e0 msgr2=0x7ff7140b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:36.243 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.241+0000 7ff718e00640 1 -- 172.21.15.152:0/2006670903 shutdown_connections 2024-04-20T11:27:36.243 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.241+0000 7ff718e00640 1 -- 172.21.15.152:0/2006670903 wait complete. 2024-04-20T11:27:36.243 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.243+0000 7ff718e00640 1 Processor -- start 2024-04-20T11:27:36.243 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.243+0000 7ff718e00640 1 -- start start 2024-04-20T11:27:36.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.244+0000 7ff718e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff6f8004d40 0x7ff71414b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:36.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.244+0000 7ff718e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff714157440 0x7ff71415a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:36.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.244+0000 7ff718e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff7141578a0 0x7ff71415a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:36.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.244+0000 7ff718e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7ff714162fb0 con 0x7ff714157440 2024-04-20T11:27:36.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.244+0000 7ff718e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff71406e500 con 0x7ff6f8004d40 2024-04-20T11:27:36.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.244+0000 7ff718e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff7140ad0a0 con 0x7ff7141578a0 2024-04-20T11:27:36.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.244+0000 7ff70aa00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff714157440 0x7ff71415a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:36.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.244+0000 7ff70aa00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff714157440 0x7ff71415a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35164/0 (socket says 172.21.15.152:35164) 2024-04-20T11:27:36.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.244+0000 7ff70aa00640 1 -- 172.21.15.152:0/3531880566 learned_addr learned my addr 172.21.15.152:0/3531880566 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:36.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.245+0000 7ff713400640 1 --2- 172.21.15.152:0/3531880566 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff6f8004d40 0x7ff71414b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:36.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.245+0000 7ff713e00640 1 --2- 172.21.15.152:0/3531880566 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff7141578a0 0x7ff71415a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:36.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.245+0000 7ff70aa00640 1 -- 172.21.15.152:0/3531880566 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff7141578a0 msgr2=0x7ff71415a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:36.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.245+0000 7ff70aa00640 1 --2- 172.21.15.152:0/3531880566 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff7141578a0 0x7ff71415a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:36.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.245+0000 7ff70aa00640 1 -- 172.21.15.152:0/3531880566 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff6f8004d40 msgr2=0x7ff71414b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:36.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.245+0000 7ff70aa00640 1 --2- 172.21.15.152:0/3531880566 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff6f8004d40 0x7ff71414b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:36.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.245+0000 7ff70aa00640 1 -- 172.21.15.152:0/3531880566 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff714155f90 con 0x7ff714157440 2024-04-20T11:27:36.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.245+0000 7ff713400640 1 --2- 172.21.15.152:0/3531880566 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff6f8004d40 0x7ff71414b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:27:36.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.245+0000 7ff70aa00640 1 --2- 172.21.15.152:0/3531880566 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff714157440 0x7ff71415a3a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc011700 tx=0x7ff6fc011bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:36.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.245+0000 7ff70be00640 1 -- 172.21.15.152:0/3531880566 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6fc00f960 con 0x7ff714157440 2024-04-20T11:27:36.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.245+0000 7ff70be00640 1 -- 172.21.15.152:0/3531880566 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6fc00fb00 con 0x7ff714157440 2024-04-20T11:27:36.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.246+0000 7ff70be00640 1 -- 172.21.15.152:0/3531880566 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6fc015460 con 0x7ff714157440 2024-04-20T11:27:36.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.246+0000 7ff718e00640 1 -- 172.21.15.152:0/3531880566 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff714156270 con 0x7ff714157440 2024-04-20T11:27:36.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.246+0000 7ff718e00640 1 -- 172.21.15.152:0/3531880566 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff714156830 con 0x7ff714157440 2024-04-20T11:27:36.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.247+0000 7ff718e00640 1 -- 172.21.15.152:0/3531880566 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff714075e60 con 0x7ff714157440 2024-04-20T11:27:36.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.248+0000 7ff70be00640 1 -- 172.21.15.152:0/3531880566 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff6fc015600 con 0x7ff714157440 2024-04-20T11:27:36.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.248+0000 7ff70be00640 1 --2- 172.21.15.152:0/3531880566 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff6f006c8f0 0x7ff6f006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:36.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.249+0000 7ff70be00640 1 -- 172.21.15.152:0/3531880566 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff6fc08c670 con 0x7ff714157440 2024-04-20T11:27:36.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.249+0000 7ff713400640 1 --2- 172.21.15.152:0/3531880566 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff6f006c8f0 0x7ff6f006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:36.249 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.249+0000 7ff713400640 1 --2- 172.21.15.152:0/3531880566 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff6f006c8f0 0x7ff6f006edb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff704009a50 tx=0x7ff704008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:36.252 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.253+0000 7ff70be00640 1 -- 172.21.15.152:0/3531880566 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ff6fc019030 con 0x7ff714157440 2024-04-20T11:27:36.437 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.437+0000 7ff718e00640 1 -- 172.21.15.152:0/3531880566 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff714166520 con 0x7ff714157440 2024-04-20T11:27:36.438 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.438+0000 7ff70be00640 1 -- 172.21.15.152:0/3531880566 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7ff6fc05a970 con 0x7ff714157440 2024-04-20T11:27:36.438 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 64 2024-04-20T11:27:36.440 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.440+0000 7ff718e00640 1 -- 172.21.15.152:0/3531880566 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff6f006c8f0 msgr2=0x7ff6f006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:36.440 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.440+0000 7ff718e00640 1 --2- 172.21.15.152:0/3531880566 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff6f006c8f0 0x7ff6f006edb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff704009a50 tx=0x7ff704008040 comp rx=0 tx=0).stop 2024-04-20T11:27:36.441 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.441+0000 7ff718e00640 1 -- 172.21.15.152:0/3531880566 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff714157440 msgr2=0x7ff71415a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:36.441 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.441+0000 7ff718e00640 1 --2- 172.21.15.152:0/3531880566 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff714157440 0x7ff71415a3a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc011700 tx=0x7ff6fc011bc0 comp rx=0 tx=0).stop 2024-04-20T11:27:36.441 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.441+0000 7ff718e00640 1 -- 172.21.15.152:0/3531880566 shutdown_connections 2024-04-20T11:27:36.441 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.441+0000 7ff718e00640 1 --2- 172.21.15.152:0/3531880566 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff6f006c8f0 0x7ff6f006edb0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:36.441 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.441+0000 7ff718e00640 1 --2- 172.21.15.152:0/3531880566 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff7141578a0 0x7ff71415a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:36.441 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.441+0000 7ff718e00640 1 --2- 172.21.15.152:0/3531880566 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff714157440 0x7ff71415a3a0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:36.441 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.441+0000 7ff718e00640 1 --2- 172.21.15.152:0/3531880566 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff6f8004d40 0x7ff71414b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:36.441 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.441+0000 7ff718e00640 1 -- 172.21.15.152:0/3531880566 >> 172.21.15.152:0/3531880566 conn(0x7ff7140b21e0 msgr2=0x7ff7140b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:36.441 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.441+0000 7ff718e00640 1 -- 172.21.15.152:0/3531880566 shutdown_connections 2024-04-20T11:27:36.441 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.441+0000 7ff718e00640 1 -- 172.21.15.152:0/3531880566 wait complete. 2024-04-20T11:27:36.459 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:27:36.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.649+0000 7f7533200640 1 Processor -- start 2024-04-20T11:27:36.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.649+0000 7f7533200640 1 -- start start 2024-04-20T11:27:36.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.650+0000 7f7533200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f7534147410 0x7f75340afa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:36.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.651+0000 7f7533200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f7534071cd0 0x7f7534072130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:36.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.651+0000 7f7533200640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75340aec90 con 0x7f753406e010 2024-04-20T11:27:36.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.651+0000 7f7533200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f7534072670 con 0x7f7534147410 2024-04-20T11:27:36.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.651+0000 7f7533200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f75340aef70 con 0x7f7534071cd0 2024-04-20T11:27:36.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.651+0000 7f7531e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f7534147410 0x7f75340afa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:36.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.651+0000 7f7532800640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f753406e010 0x7f753406e4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:39946/0 (socket says 172.21.15.152:39946) 2024-04-20T11:27:36.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.651+0000 7f7532800640 1 -- 172.21.15.152:0/242221417 learned_addr learned my addr 172.21.15.152:0/242221417 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:27:36.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.651+0000 7f752be00640 1 --2- 172.21.15.152:0/242221417 >> v2:172.21.15.175:3300/0 conn(0x7f7534071cd0 0x7f7534072130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:36.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.652+0000 7f7531400640 1 -- 172.21.15.152:0/242221417 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3009244041 0 0) 0x7f75340aec90 con 0x7f753406e010 2024-04-20T11:27:36.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.652+0000 7f7531400640 1 -- 172.21.15.152:0/242221417 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7518003520 con 0x7f753406e010 2024-04-20T11:27:36.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.652+0000 7f7531e00640 1 -- 172.21.15.152:0/242221417 >> v2:172.21.15.175:3300/0 conn(0x7f7534071cd0 msgr2=0x7f7534072130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:36.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.652+0000 7f7531e00640 1 --2- 172.21.15.152:0/242221417 >> v2:172.21.15.175:3300/0 conn(0x7f7534071cd0 0x7f7534072130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:36.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.652+0000 7f7531e00640 1 -- 172.21.15.152:0/242221417 >> v1:172.21.15.152:6789/0 conn(0x7f753406e010 legacy=0x7f753406e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:36.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.652+0000 7f7531e00640 1 -- 172.21.15.152:0/242221417 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75340af250 con 0x7f7534147410 2024-04-20T11:27:36.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.654+0000 7f7531e00640 1 --2- 172.21.15.152:0/242221417 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7534147410 0x7f75340afa50 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7534073170 tx=0x7f752402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c9d5aac0ae401c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:36.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.654+0000 7f7531400640 1 -- 172.21.15.152:0/242221417 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7524035020 con 0x7f7534147410 2024-04-20T11:27:36.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.654+0000 7f7531400640 1 -- 172.21.15.152:0/242221417 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7524031830 con 0x7f7534147410 2024-04-20T11:27:36.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.654+0000 7f7531400640 1 -- 172.21.15.152:0/242221417 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7524031be0 con 0x7f7534147410 2024-04-20T11:27:36.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.655+0000 7f7533200640 1 -- 172.21.15.152:0/242221417 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7534147410 msgr2=0x7f75340afa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:36.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.655+0000 7f7533200640 1 --2- 172.21.15.152:0/242221417 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7534147410 0x7f75340afa50 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7534073170 tx=0x7f752402b8d0 comp rx=0 tx=0).stop 2024-04-20T11:27:36.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.655+0000 7f7533200640 1 -- 172.21.15.152:0/242221417 shutdown_connections 2024-04-20T11:27:36.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.655+0000 7f7533200640 1 --2- 172.21.15.152:0/242221417 >> v2:172.21.15.175:3300/0 conn(0x7f7534071cd0 0x7f7534072130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:36.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.655+0000 7f7533200640 1 --2- 172.21.15.152:0/242221417 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7534147410 0x7f75340afa50 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:36.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.655+0000 7f7533200640 1 -- 172.21.15.152:0/242221417 >> 172.21.15.152:0/242221417 conn(0x7f7534067ae0 msgr2=0x7f753406b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:36.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.655+0000 7f7533200640 1 -- 172.21.15.152:0/242221417 shutdown_connections 2024-04-20T11:27:36.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.655+0000 7f7533200640 1 -- 172.21.15.152:0/242221417 wait complete. 2024-04-20T11:27:36.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.657+0000 7f7533200640 1 Processor -- start 2024-04-20T11:27:36.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.657+0000 7f7533200640 1 -- start start 2024-04-20T11:27:36.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.658+0000 7f7533200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f753406e010 0x7f75341f4980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:36.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.658+0000 7f7533200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7534071cd0 0x7f753413fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:36.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.658+0000 7f7533200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7534147410 0x7f75341f33c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:36.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.658+0000 7f7533200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f75340b3390 con 0x7f7534071cd0 2024-04-20T11:27:36.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.658+0000 7f7533200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f75340ae980 con 0x7f753406e010 2024-04-20T11:27:36.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.658+0000 7f7533200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f75340b3510 con 0x7f7534147410 2024-04-20T11:27:36.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.658+0000 7f752be00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7534071cd0 0x7f753413fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:36.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.658+0000 7f752be00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7534071cd0 0x7f753413fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35186/0 (socket says 172.21.15.152:35186) 2024-04-20T11:27:36.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.658+0000 7f752be00640 1 -- 172.21.15.152:0/25669548 learned_addr learned my addr 172.21.15.152:0/25669548 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:36.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.659+0000 7f752be00640 1 -- 172.21.15.152:0/25669548 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7534147410 msgr2=0x7f75341f33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:36.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.659+0000 7f752be00640 1 --2- 172.21.15.152:0/25669548 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7534147410 0x7f75341f33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:36.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.659+0000 7f752be00640 1 -- 172.21.15.152:0/25669548 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f753406e010 msgr2=0x7f75341f4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:36.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.659+0000 7f752be00640 1 --2- 172.21.15.152:0/25669548 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f753406e010 0x7f75341f4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:36.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.659+0000 7f752be00640 1 -- 172.21.15.152:0/25669548 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75341f4ec0 con 0x7f7534071cd0 2024-04-20T11:27:36.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.659+0000 7f752be00640 1 --2- 172.21.15.152:0/25669548 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7534071cd0 0x7f753413fe50 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f751c00e9b0 tx=0x7f751c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:36.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.659+0000 7f752a000640 1 -- 172.21.15.152:0/25669548 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f751c017070 con 0x7f7534071cd0 2024-04-20T11:27:36.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.659+0000 7f752a000640 1 -- 172.21.15.152:0/25669548 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f751c00c9a0 con 0x7f7534071cd0 2024-04-20T11:27:36.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.659+0000 7f7533200640 1 -- 172.21.15.152:0/25669548 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f753415ee80 con 0x7f7534071cd0 2024-04-20T11:27:36.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.659+0000 7f752a000640 1 -- 172.21.15.152:0/25669548 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f751c00cd90 con 0x7f7534071cd0 2024-04-20T11:27:36.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.659+0000 7f7533200640 1 -- 172.21.15.152:0/25669548 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f753415f440 con 0x7f7534071cd0 2024-04-20T11:27:36.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.660+0000 7f7533200640 1 -- 172.21.15.152:0/25669548 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7534075e60 con 0x7f7534071cd0 2024-04-20T11:27:36.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.661+0000 7f752a000640 1 -- 172.21.15.152:0/25669548 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f751c01a070 con 0x7f7534071cd0 2024-04-20T11:27:36.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.662+0000 7f752a000640 1 --2- 172.21.15.152:0/25669548 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f751006c920 0x7f751006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:36.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.662+0000 7f752a000640 1 -- 172.21.15.152:0/25669548 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f751c08bad0 con 0x7f7534071cd0 2024-04-20T11:27:36.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.663+0000 7f7531e00640 1 --2- 172.21.15.152:0/25669548 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f751006c920 0x7f751006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:36.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.663+0000 7f7531e00640 1 --2- 172.21.15.152:0/25669548 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f751006c920 0x7f751006ede0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7524036000 tx=0x7f7524030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:36.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.667+0000 7f752a000640 1 -- 172.21.15.152:0/25669548 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f75341f4ec0 con 0x7f7534071cd0 2024-04-20T11:27:36.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.866+0000 7f7533200640 1 -- 172.21.15.152:0/25669548 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f753413e370 con 0x7f7534071cd0 2024-04-20T11:27:36.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.868+0000 7f752a000640 1 -- 172.21.15.152:0/25669548 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16665 (secure 0 0 0) 0x7f751c059ef0 con 0x7f7534071cd0 2024-04-20T11:27:36.868 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:27:36.869 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":20,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:26:59.675971+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:26:56.261424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T11:26:57.079286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:27:36.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.871+0000 7f7533200640 1 -- 172.21.15.152:0/25669548 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f751006c920 msgr2=0x7f751006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:36.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.871+0000 7f7533200640 1 --2- 172.21.15.152:0/25669548 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f751006c920 0x7f751006ede0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7524036000 tx=0x7f7524030000 comp rx=0 tx=0).stop 2024-04-20T11:27:36.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.871+0000 7f7533200640 1 -- 172.21.15.152:0/25669548 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7534071cd0 msgr2=0x7f753413fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:36.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.871+0000 7f7533200640 1 --2- 172.21.15.152:0/25669548 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7534071cd0 0x7f753413fe50 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f751c00e9b0 tx=0x7f751c00ee70 comp rx=0 tx=0).stop 2024-04-20T11:27:36.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.871+0000 7f7533200640 1 -- 172.21.15.152:0/25669548 shutdown_connections 2024-04-20T11:27:36.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.871+0000 7f7533200640 1 --2- 172.21.15.152:0/25669548 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f751006c920 0x7f751006ede0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:36.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.871+0000 7f7533200640 1 --2- 172.21.15.152:0/25669548 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7534147410 0x7f75341f33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:36.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.871+0000 7f7533200640 1 --2- 172.21.15.152:0/25669548 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7534071cd0 0x7f753413fe50 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:36.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.871+0000 7f7533200640 1 --2- 172.21.15.152:0/25669548 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f753406e010 0x7f75341f4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:36.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.871+0000 7f7533200640 1 -- 172.21.15.152:0/25669548 >> 172.21.15.152:0/25669548 conn(0x7f7534067ae0 msgr2=0x7f7534074a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:36.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.872+0000 7f7533200640 1 -- 172.21.15.152:0/25669548 shutdown_connections 2024-04-20T11:27:36.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:36.872+0000 7f7533200640 1 -- 172.21.15.152:0/25669548 wait complete. 2024-04-20T11:27:36.890 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T11:26:47.246571+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T11:26:56.261424+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T11:26:57.079286+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T11:27:36.890 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T11:27:37.091 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.091+0000 7f0015200640 1 Processor -- start 2024-04-20T11:27:37.091 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.092+0000 7f0015200640 1 -- start start 2024-04-20T11:27:37.092 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.093+0000 7f0015200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0010156c00 0x7f0010160a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:37.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.093+0000 7f0015200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f001015ca40 0x7f001015cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:37.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.093+0000 7f0015200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f001006bda0 0x7f001006c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:37.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.093+0000 7f0015200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f0010160fd0 con 0x7f001006bda0 2024-04-20T11:27:37.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.093+0000 7f0015200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f001015d5c0 con 0x7f001015ca40 2024-04-20T11:27:37.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.093+0000 7f0015200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f001015d8a0 con 0x7f0010156c00 2024-04-20T11:27:37.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.093+0000 7f000fe00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f001006bda0 0x7f001006c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:37.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.093+0000 7f000fe00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f001006bda0 0x7f001006c230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35200/0 (socket says 172.21.15.152:35200) 2024-04-20T11:27:37.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.093+0000 7f000fe00640 1 -- 172.21.15.152:0/2467101943 learned_addr learned my addr 172.21.15.152:0/2467101943 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:37.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.094+0000 7f0006a00640 1 --2- 172.21.15.152:0/2467101943 >> v2:172.21.15.175:3300/0 conn(0x7f001015ca40 0x7f001015cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:37.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.094+0000 7f000f400640 1 --2- 172.21.15.152:0/2467101943 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0010156c00 0x7f0010160a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:37.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.094+0000 7f000fe00640 1 -- 172.21.15.152:0/2467101943 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0010156c00 msgr2=0x7f0010160a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:37.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.094+0000 7f000fe00640 1 --2- 172.21.15.152:0/2467101943 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0010156c00 0x7f0010160a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:37.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.094+0000 7f000fe00640 1 -- 172.21.15.152:0/2467101943 >> v2:172.21.15.175:3300/0 conn(0x7f001015ca40 msgr2=0x7f001015cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:37.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.094+0000 7f000fe00640 1 --2- 172.21.15.152:0/2467101943 >> v2:172.21.15.175:3300/0 conn(0x7f001015ca40 0x7f001015cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:37.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.094+0000 7f000fe00640 1 -- 172.21.15.152:0/2467101943 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0010160290 con 0x7f001006bda0 2024-04-20T11:27:37.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.095+0000 7f000fe00640 1 --2- 172.21.15.152:0/2467101943 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f001006bda0 0x7f001006c230 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f00100b2d30 tx=0x7f000802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6be6057fe54fcdec server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:37.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.095+0000 7f000ea00640 1 -- 172.21.15.152:0/2467101943 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0008035020 con 0x7f001006bda0 2024-04-20T11:27:37.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.095+0000 7f000ea00640 1 -- 172.21.15.152:0/2467101943 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0008031870 con 0x7f001006bda0 2024-04-20T11:27:37.096 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.095+0000 7f000ea00640 1 -- 172.21.15.152:0/2467101943 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0008031c40 con 0x7f001006bda0 2024-04-20T11:27:37.096 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.096+0000 7f0015200640 1 -- 172.21.15.152:0/2467101943 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f001006bda0 msgr2=0x7f001006c230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:37.096 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.096+0000 7f0015200640 1 --2- 172.21.15.152:0/2467101943 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f001006bda0 0x7f001006c230 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f00100b2d30 tx=0x7f000802b910 comp rx=0 tx=0).stop 2024-04-20T11:27:37.096 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.096+0000 7f0015200640 1 -- 172.21.15.152:0/2467101943 shutdown_connections 2024-04-20T11:27:37.096 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.096+0000 7f0015200640 1 --2- 172.21.15.152:0/2467101943 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f001006bda0 0x7f001006c230 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:37.096 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.096+0000 7f0015200640 1 --2- 172.21.15.152:0/2467101943 >> v2:172.21.15.175:3300/0 conn(0x7f001015ca40 0x7f001015cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:37.096 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.096+0000 7f0015200640 1 --2- 172.21.15.152:0/2467101943 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0010156c00 0x7f0010160a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:37.096 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.096+0000 7f0015200640 1 -- 172.21.15.152:0/2467101943 >> 172.21.15.152:0/2467101943 conn(0x7f00100b21e0 msgr2=0x7f00100b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:37.096 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.096+0000 7f0015200640 1 -- 172.21.15.152:0/2467101943 shutdown_connections 2024-04-20T11:27:37.096 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.096+0000 7f0015200640 1 -- 172.21.15.152:0/2467101943 wait complete. 2024-04-20T11:27:37.097 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.098+0000 7f0015200640 1 Processor -- start 2024-04-20T11:27:37.097 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.098+0000 7f0015200640 1 -- start start 2024-04-20T11:27:37.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.099+0000 7f0015200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f001006bda0 0x7f00101f61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:37.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.099+0000 7f0015200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0010156c00 0x7f00101585c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:37.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.099+0000 7f0015200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f001015ca40 0x7f00101f20f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:37.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.099+0000 7f0015200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f00100b3240 con 0x7f001006bda0 2024-04-20T11:27:37.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.099+0000 7f0015200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f00100b30c0 con 0x7f001015ca40 2024-04-20T11:27:37.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.099+0000 7f0015200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f00100b33c0 con 0x7f0010156c00 2024-04-20T11:27:37.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.099+0000 7f000f400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f001006bda0 0x7f00101f61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:37.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.099+0000 7f000f400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f001006bda0 0x7f00101f61d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35216/0 (socket says 172.21.15.152:35216) 2024-04-20T11:27:37.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.099+0000 7f000f400640 1 -- 172.21.15.152:0/3186146169 learned_addr learned my addr 172.21.15.152:0/3186146169 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:37.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.099+0000 7f0006a00640 1 --2- 172.21.15.152:0/3186146169 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0010156c00 0x7f00101585c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:37.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.099+0000 7f000f400640 1 -- 172.21.15.152:0/3186146169 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0010156c00 msgr2=0x7f00101585c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:37.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.099+0000 7f000fe00640 1 --2- 172.21.15.152:0/3186146169 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f001015ca40 0x7f00101f20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:37.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.100+0000 7f000f400640 1 --2- 172.21.15.152:0/3186146169 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0010156c00 0x7f00101585c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:37.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.100+0000 7f000f400640 1 -- 172.21.15.152:0/3186146169 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f001015ca40 msgr2=0x7f00101f20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:37.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.100+0000 7f000f400640 1 --2- 172.21.15.152:0/3186146169 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f001015ca40 0x7f00101f20f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:37.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.100+0000 7f000f400640 1 -- 172.21.15.152:0/3186146169 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00101f5710 con 0x7f001006bda0 2024-04-20T11:27:37.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.100+0000 7f0006a00640 1 --2- 172.21.15.152:0/3186146169 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0010156c00 0x7f00101585c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:27:37.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.100+0000 7f000f400640 1 --2- 172.21.15.152:0/3186146169 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f001006bda0 0x7f00101f61d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0000007e60 tx=0x7f000000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:37.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.100+0000 7f0007e00640 1 -- 172.21.15.152:0/3186146169 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0000014620 con 0x7f001006bda0 2024-04-20T11:27:37.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.100+0000 7f0007e00640 1 -- 172.21.15.152:0/3186146169 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0000019070 con 0x7f001006bda0 2024-04-20T11:27:37.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.100+0000 7f0015200640 1 -- 172.21.15.152:0/3186146169 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00101f2c50 con 0x7f001006bda0 2024-04-20T11:27:37.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.100+0000 7f0007e00640 1 -- 172.21.15.152:0/3186146169 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0000014a00 con 0x7f001006bda0 2024-04-20T11:27:37.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.100+0000 7f0015200640 1 -- 172.21.15.152:0/3186146169 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00101f59d0 con 0x7f001006bda0 2024-04-20T11:27:37.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.102+0000 7f0015200640 1 -- 172.21.15.152:0/3186146169 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0010075e60 con 0x7f001006bda0 2024-04-20T11:27:37.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.102+0000 7f0007e00640 1 -- 172.21.15.152:0/3186146169 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0000014ba0 con 0x7f001006bda0 2024-04-20T11:27:37.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.103+0000 7f0007e00640 1 --2- 172.21.15.152:0/3186146169 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7effec06c920 0x7effec06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:37.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.104+0000 7f0007e00640 1 -- 172.21.15.152:0/3186146169 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f000008c3e0 con 0x7f001006bda0 2024-04-20T11:27:37.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.104+0000 7f0006a00640 1 --2- 172.21.15.152:0/3186146169 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7effec06c920 0x7effec06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:37.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.104+0000 7f0006a00640 1 --2- 172.21.15.152:0/3186146169 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7effec06c920 0x7effec06ede0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7efff8006fd0 tx=0x7efff8008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:37.107 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.108+0000 7f0007e00640 1 -- 172.21.15.152:0/3186146169 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f000005a800 con 0x7f001006bda0 2024-04-20T11:27:37.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.310+0000 7f0015200640 1 -- 172.21.15.152:0/3186146169 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f001015fc80 con 0x7f001006bda0 2024-04-20T11:27:37.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.310+0000 7f0007e00640 1 -- 172.21.15.152:0/3186146169 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f000005a100 con 0x7f001006bda0 2024-04-20T11:27:37.310 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 1 2024-04-20T11:27:37.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.313+0000 7f0015200640 1 -- 172.21.15.152:0/3186146169 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7effec06c920 msgr2=0x7effec06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:37.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.313+0000 7f0015200640 1 --2- 172.21.15.152:0/3186146169 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7effec06c920 0x7effec06ede0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7efff8006fd0 tx=0x7efff8008040 comp rx=0 tx=0).stop 2024-04-20T11:27:37.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.313+0000 7f0015200640 1 -- 172.21.15.152:0/3186146169 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f001006bda0 msgr2=0x7f00101f61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:37.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.313+0000 7f0015200640 1 --2- 172.21.15.152:0/3186146169 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f001006bda0 0x7f00101f61d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0000007e60 tx=0x7f000000e4b0 comp rx=0 tx=0).stop 2024-04-20T11:27:37.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.313+0000 7f0015200640 1 -- 172.21.15.152:0/3186146169 shutdown_connections 2024-04-20T11:27:37.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.314+0000 7f0015200640 1 --2- 172.21.15.152:0/3186146169 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7effec06c920 0x7effec06ede0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:37.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.314+0000 7f0015200640 1 --2- 172.21.15.152:0/3186146169 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f001015ca40 0x7f00101f20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:37.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.314+0000 7f0015200640 1 --2- 172.21.15.152:0/3186146169 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0010156c00 0x7f00101585c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:37.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.314+0000 7f0015200640 1 --2- 172.21.15.152:0/3186146169 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f001006bda0 0x7f00101f61d0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:37.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.314+0000 7f0015200640 1 -- 172.21.15.152:0/3186146169 >> 172.21.15.152:0/3186146169 conn(0x7f00100b21e0 msgr2=0x7f001015bcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:37.315 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.314+0000 7f0015200640 1 -- 172.21.15.152:0/3186146169 shutdown_connections 2024-04-20T11:27:37.315 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.314+0000 7f0015200640 1 -- 172.21.15.152:0/3186146169 wait complete. 2024-04-20T11:27:37.332 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T11:27:37.534 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.534+0000 7fe84f200640 1 Processor -- start 2024-04-20T11:27:37.534 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.535+0000 7fe84f200640 1 -- start start 2024-04-20T11:27:37.536 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.536+0000 7fe84f200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe85015eb50 0x7fe8500d0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:37.536 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.536+0000 7fe84f200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fe850163230 0x7fe8500d2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:37.536 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.536+0000 7fe84f200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fe8500d1440 0x7fe8500d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:37.537 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.536+0000 7fe84f200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fe8500d19c0 con 0x7fe850163230 2024-04-20T11:27:37.537 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.536+0000 7fe84f200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fe850162750 con 0x7fe8500d1440 2024-04-20T11:27:37.537 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.536+0000 7fe84f200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe850162a30 con 0x7fe85015eb50 2024-04-20T11:27:37.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.537+0000 7fe84d400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fe850163230 0x7fe8500d2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:37.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.537+0000 7fe84d400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fe850163230 0x7fe8500d2e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35230/0 (socket says 172.21.15.152:35230) 2024-04-20T11:27:37.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.537+0000 7fe84d400640 1 -- 172.21.15.152:0/1567679163 learned_addr learned my addr 172.21.15.152:0/1567679163 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:37.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.537+0000 7fe84e800640 1 --2- 172.21.15.152:0/1567679163 >> v2:172.21.15.175:3300/0 conn(0x7fe8500d1440 0x7fe8500d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:37.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.537+0000 7fe84de00640 1 --2- 172.21.15.152:0/1567679163 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe85015eb50 0x7fe8500d0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:37.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.537+0000 7fe84d400640 1 -- 172.21.15.152:0/1567679163 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe85015eb50 msgr2=0x7fe8500d0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:37.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.537+0000 7fe84d400640 1 --2- 172.21.15.152:0/1567679163 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe85015eb50 0x7fe8500d0f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:37.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.537+0000 7fe84d400640 1 -- 172.21.15.152:0/1567679163 >> v2:172.21.15.175:3300/0 conn(0x7fe8500d1440 msgr2=0x7fe8500d33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:37.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.537+0000 7fe84d400640 1 --2- 172.21.15.152:0/1567679163 >> v2:172.21.15.175:3300/0 conn(0x7fe8500d1440 0x7fe8500d33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:37.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.537+0000 7fe84d400640 1 -- 172.21.15.152:0/1567679163 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe850162d10 con 0x7fe850163230 2024-04-20T11:27:37.539 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.537+0000 7fe84e800640 1 --2- 172.21.15.152:0/1567679163 >> v2:172.21.15.175:3300/0 conn(0x7fe8500d1440 0x7fe8500d33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:27:37.539 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.538+0000 7fe84d400640 1 --2- 172.21.15.152:0/1567679163 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe850163230 0x7fe8500d2e90 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe8500d1ff0 tx=0x7fe83802be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f4dd1ff3ef68f3f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:37.539 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.539+0000 7fe84ca00640 1 -- 172.21.15.152:0/1567679163 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe83802ca30 con 0x7fe850163230 2024-04-20T11:27:37.539 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.539+0000 7fe84ca00640 1 -- 172.21.15.152:0/1567679163 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe83802cbd0 con 0x7fe850163230 2024-04-20T11:27:37.539 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.539+0000 7fe84ca00640 1 -- 172.21.15.152:0/1567679163 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8380343f0 con 0x7fe850163230 2024-04-20T11:27:37.540 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.540+0000 7fe84f200640 1 -- 172.21.15.152:0/1567679163 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe850163230 msgr2=0x7fe8500d2e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:37.540 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.540+0000 7fe84f200640 1 --2- 172.21.15.152:0/1567679163 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe850163230 0x7fe8500d2e90 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe8500d1ff0 tx=0x7fe83802be90 comp rx=0 tx=0).stop 2024-04-20T11:27:37.540 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.540+0000 7fe84f200640 1 -- 172.21.15.152:0/1567679163 shutdown_connections 2024-04-20T11:27:37.540 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.540+0000 7fe84f200640 1 --2- 172.21.15.152:0/1567679163 >> v2:172.21.15.175:3300/0 conn(0x7fe8500d1440 0x7fe8500d33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:37.540 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.540+0000 7fe84f200640 1 --2- 172.21.15.152:0/1567679163 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe850163230 0x7fe8500d2e90 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:37.541 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.540+0000 7fe84f200640 1 --2- 172.21.15.152:0/1567679163 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe85015eb50 0x7fe8500d0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:37.541 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.540+0000 7fe84f200640 1 -- 172.21.15.152:0/1567679163 >> 172.21.15.152:0/1567679163 conn(0x7fe85008bfd0 msgr2=0x7fe8500d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:37.541 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.540+0000 7fe84f200640 1 -- 172.21.15.152:0/1567679163 shutdown_connections 2024-04-20T11:27:37.541 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.540+0000 7fe84f200640 1 -- 172.21.15.152:0/1567679163 wait complete. 2024-04-20T11:27:37.541 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.541+0000 7fe84f200640 1 Processor -- start 2024-04-20T11:27:37.541 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.542+0000 7fe84f200640 1 -- start start 2024-04-20T11:27:37.543 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.542+0000 7fe84f200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8500d1440 0x7fe8500b9c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:37.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.543+0000 7fe84f200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe85015eb50 0x7fe8500b5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:37.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.543+0000 7fe84f200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe850163230 0x7fe8500cb650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:37.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.543+0000 7fe84f200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fe85015f0b0 con 0x7fe85015eb50 2024-04-20T11:27:37.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.543+0000 7fe84f200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe85015ef30 con 0x7fe850163230 2024-04-20T11:27:37.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.543+0000 7fe84f200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe85015f230 con 0x7fe8500d1440 2024-04-20T11:27:37.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.543+0000 7fe84d400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe85015eb50 0x7fe8500b5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:37.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.543+0000 7fe84d400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe85015eb50 0x7fe8500b5fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35244/0 (socket says 172.21.15.152:35244) 2024-04-20T11:27:37.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.543+0000 7fe84d400640 1 -- 172.21.15.152:0/1027743815 learned_addr learned my addr 172.21.15.152:0/1027743815 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:37.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.543+0000 7fe84d400640 1 -- 172.21.15.152:0/1027743815 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8500d1440 msgr2=0x7fe8500b9c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:37.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.543+0000 7fe84d400640 1 --2- 172.21.15.152:0/1027743815 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8500d1440 0x7fe8500b9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:37.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.543+0000 7fe84d400640 1 -- 172.21.15.152:0/1027743815 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe850163230 msgr2=0x7fe8500cb650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:37.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.543+0000 7fe84d400640 1 --2- 172.21.15.152:0/1027743815 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe850163230 0x7fe8500cb650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:37.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.543+0000 7fe84d400640 1 -- 172.21.15.152:0/1027743815 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8500ca340 con 0x7fe85015eb50 2024-04-20T11:27:37.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.544+0000 7fe84d400640 1 --2- 172.21.15.152:0/1027743815 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe85015eb50 0x7fe8500b5fa0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe838000f80 tx=0x7fe838004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:37.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.544+0000 7fe846000640 1 -- 172.21.15.152:0/1027743815 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe838043020 con 0x7fe85015eb50 2024-04-20T11:27:37.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.544+0000 7fe846000640 1 -- 172.21.15.152:0/1027743815 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe83803e450 con 0x7fe85015eb50 2024-04-20T11:27:37.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.544+0000 7fe846000640 1 -- 172.21.15.152:0/1027743815 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe83803e840 con 0x7fe85015eb50 2024-04-20T11:27:37.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.544+0000 7fe84f200640 1 -- 172.21.15.152:0/1027743815 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8500ca620 con 0x7fe85015eb50 2024-04-20T11:27:37.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.544+0000 7fe84f200640 1 -- 172.21.15.152:0/1027743815 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8500ba1d0 con 0x7fe85015eb50 2024-04-20T11:27:37.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.545+0000 7fe84f200640 1 -- 172.21.15.152:0/1027743815 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe850160b20 con 0x7fe85015eb50 2024-04-20T11:27:37.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.546+0000 7fe846000640 1 -- 172.21.15.152:0/1027743815 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe838046020 con 0x7fe85015eb50 2024-04-20T11:27:37.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.548+0000 7fe846000640 1 --2- 172.21.15.152:0/1027743815 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe82c06c920 0x7fe82c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:37.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.548+0000 7fe846000640 1 -- 172.21.15.152:0/1027743815 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fe8380ae4d0 con 0x7fe85015eb50 2024-04-20T11:27:37.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.548+0000 7fe84de00640 1 --2- 172.21.15.152:0/1027743815 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe82c06c920 0x7fe82c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:37.548 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.549+0000 7fe84de00640 1 --2- 172.21.15.152:0/1027743815 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe82c06c920 0x7fe82c06ede0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe8400098f0 tx=0x7fe840008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:37.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.553+0000 7fe846000640 1 -- 172.21.15.152:0/1027743815 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe850160b20 con 0x7fe85015eb50 2024-04-20T11:27:37.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.756+0000 7fe84f200640 1 -- 172.21.15.152:0/1027743815 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe8500ca7f0 con 0x7fe85015eb50 2024-04-20T11:27:37.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.757+0000 7fe846000640 1 -- 172.21.15.152:0/1027743815 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7fe83804d030 con 0x7fe85015eb50 2024-04-20T11:27:37.757 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 32 2024-04-20T11:27:37.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.760+0000 7fe84f200640 1 -- 172.21.15.152:0/1027743815 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe82c06c920 msgr2=0x7fe82c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:37.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.760+0000 7fe84f200640 1 --2- 172.21.15.152:0/1027743815 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe82c06c920 0x7fe82c06ede0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe8400098f0 tx=0x7fe840008040 comp rx=0 tx=0).stop 2024-04-20T11:27:37.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.760+0000 7fe84f200640 1 -- 172.21.15.152:0/1027743815 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe85015eb50 msgr2=0x7fe8500b5fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:37.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.760+0000 7fe84f200640 1 --2- 172.21.15.152:0/1027743815 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe85015eb50 0x7fe8500b5fa0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe838000f80 tx=0x7fe838004a40 comp rx=0 tx=0).stop 2024-04-20T11:27:37.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.760+0000 7fe84f200640 1 -- 172.21.15.152:0/1027743815 shutdown_connections 2024-04-20T11:27:37.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.760+0000 7fe84f200640 1 --2- 172.21.15.152:0/1027743815 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe82c06c920 0x7fe82c06ede0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:37.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.760+0000 7fe84f200640 1 --2- 172.21.15.152:0/1027743815 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe850163230 0x7fe8500cb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:37.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.760+0000 7fe84f200640 1 --2- 172.21.15.152:0/1027743815 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe85015eb50 0x7fe8500b5fa0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:37.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.760+0000 7fe84f200640 1 --2- 172.21.15.152:0/1027743815 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8500d1440 0x7fe8500b9c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:37.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.760+0000 7fe84f200640 1 -- 172.21.15.152:0/1027743815 >> 172.21.15.152:0/1027743815 conn(0x7fe85008bfd0 msgr2=0x7fe8500b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:37.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.760+0000 7fe84f200640 1 -- 172.21.15.152:0/1027743815 shutdown_connections 2024-04-20T11:27:37.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.760+0000 7fe84f200640 1 -- 172.21.15.152:0/1027743815 wait complete. 2024-04-20T11:27:37.780 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T11:27:37.984 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.984+0000 7f41c0c00640 1 Processor -- start 2024-04-20T11:27:37.984 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.985+0000 7f41c0c00640 1 -- start start 2024-04-20T11:27:37.986 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.986+0000 7f41c0c00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f41bc0b2880 0x7f41bc15fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:37.986 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.986+0000 7f41c0c00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41bc15f1c0 con 0x7f41bc15b510 2024-04-20T11:27:37.986 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.986+0000 7f41c0c00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41bc06f670 con 0x7f41bc0cfc40 2024-04-20T11:27:37.986 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.986+0000 7f41c0c00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f41bc06f420 con 0x7f41bc0b2880 2024-04-20T11:27:37.986 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.986+0000 7f41b9e00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f41bc15b510 0x7f41bc0b2170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:39948/0 (socket says 172.21.15.152:39948) 2024-04-20T11:27:37.986 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.986+0000 7f41b9e00640 1 -- 172.21.15.152:0/1096775602 learned_addr learned my addr 172.21.15.152:0/1096775602 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:27:37.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.987+0000 7f41ba800640 1 --2- 172.21.15.152:0/1096775602 >> v2:172.21.15.175:3300/0 conn(0x7f41bc0b2880 0x7f41bc15fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:37.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.987+0000 7f41b9400640 1 -- 172.21.15.152:0/1096775602 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2170802477 0 0) 0x7f41bc15f1c0 con 0x7f41bc15b510 2024-04-20T11:27:37.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.987+0000 7f41b9400640 1 -- 172.21.15.152:0/1096775602 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f41a0005730 con 0x7f41bc15b510 2024-04-20T11:27:37.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.987+0000 7f41b9400640 1 -- 172.21.15.152:0/1096775602 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4107397797 0 0) 0x7f41a0005730 con 0x7f41bc15b510 2024-04-20T11:27:37.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.987+0000 7f41b9400640 1 -- 172.21.15.152:0/1096775602 >> v2:172.21.15.175:3300/0 conn(0x7f41bc0b2880 msgr2=0x7f41bc15fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:37.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.987+0000 7f41b9400640 1 --2- 172.21.15.152:0/1096775602 >> v2:172.21.15.175:3300/0 conn(0x7f41bc0b2880 0x7f41bc15fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:37.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.987+0000 7f41b9400640 1 -- 172.21.15.152:0/1096775602 >> v1:172.21.15.175:6789/0 conn(0x7f41bc0cfc40 legacy=0x7f41bc0b4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:37.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.987+0000 7f41b9400640 1 -- 172.21.15.152:0/1096775602 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41bc15f430 con 0x7f41bc15b510 2024-04-20T11:27:37.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.987+0000 7f41b9400640 1 -- 172.21.15.152:0/1096775602 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f41ac0019f0 con 0x7f41bc15b510 2024-04-20T11:27:37.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.987+0000 7f41b9400640 1 -- 172.21.15.152:0/1096775602 >> v1:172.21.15.152:6789/0 conn(0x7f41bc15b510 legacy=0x7f41bc0b2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:37.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.987+0000 7f41b9400640 1 --2- 172.21.15.152:0/1096775602 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41a0005bc0 0x7f41a0005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:37.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.987+0000 7f41b9400640 1 --2- 172.21.15.152:0/1096775602 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f41a0006a10 0x7f41a0006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:37.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.987+0000 7f41b9400640 1 --2- 172.21.15.152:0/1096775602 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f41a00073d0 0x7f41a000dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:37.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.987+0000 7f41b9400640 1 -- 172.21.15.152:0/1096775602 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f41bc06f420 con 0x7f41a00073d0 2024-04-20T11:27:37.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.987+0000 7f41b9400640 1 -- 172.21.15.152:0/1096775602 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f41a000e370 con 0x7f41a0005bc0 2024-04-20T11:27:37.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.987+0000 7f41b9400640 1 -- 172.21.15.152:0/1096775602 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f41a000e570 con 0x7f41a0006a10 2024-04-20T11:27:37.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.988+0000 7f41ba800640 1 --2- 172.21.15.152:0/1096775602 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f41a00073d0 0x7f41a000dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:37.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.988+0000 7f41b9e00640 1 --2- 172.21.15.152:0/1096775602 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41a0005bc0 0x7f41a0005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:37.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.988+0000 7f41ba800640 1 -- 172.21.15.152:0/1096775602 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f41a0006a10 msgr2=0x7f41a0006e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:37.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.988+0000 7f41ba800640 1 --2- 172.21.15.152:0/1096775602 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f41a0006a10 0x7f41a0006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:37.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.988+0000 7f41ba800640 1 -- 172.21.15.152:0/1096775602 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41a0005bc0 msgr2=0x7f41a0005fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:37.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.988+0000 7f41ba800640 1 --2- 172.21.15.152:0/1096775602 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41a0005bc0 0x7f41a0005fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:37.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.988+0000 7f41ba800640 1 -- 172.21.15.152:0/1096775602 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41a0005730 con 0x7f41a00073d0 2024-04-20T11:27:37.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.989+0000 7f41ba800640 1 -- 172.21.15.152:0/1096775602 reap_dead start 2024-04-20T11:27:37.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.990+0000 7f41ba800640 1 --2- 172.21.15.152:0/1096775602 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f41a00073d0 0x7f41a000dbd0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f41b4009ac0 tx=0x7f41a0007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a267eb609f7f1f2c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:37.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.990+0000 7f41b9400640 1 -- 172.21.15.152:0/1096775602 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41b4009690 con 0x7f41a00073d0 2024-04-20T11:27:37.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.990+0000 7f41b9400640 1 -- 172.21.15.152:0/1096775602 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41b4004020 con 0x7f41a00073d0 2024-04-20T11:27:37.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.990+0000 7f41b9400640 1 -- 172.21.15.152:0/1096775602 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41b4004410 con 0x7f41a00073d0 2024-04-20T11:27:37.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.990+0000 7f41c0c00640 1 -- 172.21.15.152:0/1096775602 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f41a00073d0 msgr2=0x7f41a000dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:37.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.990+0000 7f41c0c00640 1 --2- 172.21.15.152:0/1096775602 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f41a00073d0 0x7f41a000dbd0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f41b4009ac0 tx=0x7f41a0007a40 comp rx=0 tx=0).stop 2024-04-20T11:27:37.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.991+0000 7f41c0c00640 1 -- 172.21.15.152:0/1096775602 shutdown_connections 2024-04-20T11:27:37.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.991+0000 7f41c0c00640 1 --2- 172.21.15.152:0/1096775602 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f41a00073d0 0x7f41a000dbd0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:37.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.991+0000 7f41c0c00640 1 -- 172.21.15.152:0/1096775602 >> 172.21.15.152:0/1096775602 conn(0x7f41bc06b980 msgr2=0x7f41bc06bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:37.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.991+0000 7f41c0c00640 1 -- 172.21.15.152:0/1096775602 shutdown_connections 2024-04-20T11:27:37.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.991+0000 7f41c0c00640 1 -- 172.21.15.152:0/1096775602 wait complete. 2024-04-20T11:27:37.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.992+0000 7f41c0c00640 1 Processor -- start 2024-04-20T11:27:37.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.993+0000 7f41c0c00640 1 -- start start 2024-04-20T11:27:37.995 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.994+0000 7f41c0c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f41a00073d0 0x7f41bc146df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:37.995 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.994+0000 7f41c0c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f41bc1f0460 0x7f41bc1f08e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:37.995 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.994+0000 7f41c0c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41bc0b1530 0x7f41bc0b19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:37.995 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.994+0000 7f41c0c00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f41bc06f950 con 0x7f41a00073d0 2024-04-20T11:27:37.995 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.994+0000 7f41c0c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f41bc15b8f0 con 0x7f41bc0b1530 2024-04-20T11:27:37.995 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.994+0000 7f41c0c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f41bc15fc30 con 0x7f41bc1f0460 2024-04-20T11:27:37.995 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.994+0000 7f41b9e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f41a00073d0 0x7f41bc146df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:37.995 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.994+0000 7f41b9e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f41a00073d0 0x7f41bc146df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35274/0 (socket says 172.21.15.152:35274) 2024-04-20T11:27:37.995 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.994+0000 7f41b9e00640 1 -- 172.21.15.152:0/1317805245 learned_addr learned my addr 172.21.15.152:0/1317805245 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:37.995 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.994+0000 7f41b9e00640 1 -- 172.21.15.152:0/1317805245 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f41bc1f0460 msgr2=0x7f41bc1f08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:37.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.994+0000 7f41ba800640 1 --2- 172.21.15.152:0/1317805245 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41bc0b1530 0x7f41bc0b19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:37.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.994+0000 7f41b9e00640 1 --2- 172.21.15.152:0/1317805245 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f41bc1f0460 0x7f41bc1f08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:37.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.994+0000 7f41b9e00640 1 -- 172.21.15.152:0/1317805245 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41bc0b1530 msgr2=0x7f41bc0b19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:37.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.994+0000 7f41b9e00640 1 --2- 172.21.15.152:0/1317805245 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41bc0b1530 0x7f41bc0b19b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:37.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.994+0000 7f41b9e00640 1 -- 172.21.15.152:0/1317805245 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41bc147330 con 0x7f41a00073d0 2024-04-20T11:27:37.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.995+0000 7f41b9e00640 1 --2- 172.21.15.152:0/1317805245 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f41a00073d0 0x7f41bc146df0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f41ac007a90 tx=0x7f41ac00e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:37.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.995+0000 7f41b2a00640 1 -- 172.21.15.152:0/1317805245 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41ac00ee50 con 0x7f41a00073d0 2024-04-20T11:27:37.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.995+0000 7f41b2a00640 1 -- 172.21.15.152:0/1317805245 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41ac018070 con 0x7f41a00073d0 2024-04-20T11:27:37.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.995+0000 7f41b2a00640 1 -- 172.21.15.152:0/1317805245 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41ac013660 con 0x7f41a00073d0 2024-04-20T11:27:37.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.995+0000 7f41c0c00640 1 -- 172.21.15.152:0/1317805245 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41bc0b01b0 con 0x7f41a00073d0 2024-04-20T11:27:37.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.995+0000 7f41c0c00640 1 -- 172.21.15.152:0/1317805245 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41bc0b0770 con 0x7f41a00073d0 2024-04-20T11:27:37.997 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.996+0000 7f41c0c00640 1 -- 172.21.15.152:0/1317805245 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41bc0689f0 con 0x7f41a00073d0 2024-04-20T11:27:37.997 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.997+0000 7f41b2a00640 1 -- 172.21.15.152:0/1317805245 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f41ac013800 con 0x7f41a00073d0 2024-04-20T11:27:37.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.999+0000 7f41b2a00640 1 --2- 172.21.15.152:0/1317805245 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f419806c8f0 0x7f419806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:37.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.999+0000 7f41b2a00640 1 -- 172.21.15.152:0/1317805245 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f41ac08c2f0 con 0x7f41a00073d0 2024-04-20T11:27:37.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:37.999+0000 7f41b1400640 1 --2- 172.21.15.152:0/1317805245 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f419806c8f0 0x7f419806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:37.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.000+0000 7f41b1400640 1 --2- 172.21.15.152:0/1317805245 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f419806c8f0 0x7f419806edb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f41a4001600 tx=0x7f41a4007990 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:38.002 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.003+0000 7f41b2a00640 1 -- 172.21.15.152:0/1317805245 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f41ac05a680 con 0x7f41a00073d0 2024-04-20T11:27:38.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.203+0000 7f41c0c00640 1 -- 172.21.15.152:0/1317805245 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f41bc06ecf0 con 0x7f41a00073d0 2024-04-20T11:27:38.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.204+0000 7f41b2a00640 1 -- 172.21.15.152:0/1317805245 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f41ac059f80 con 0x7f41a00073d0 2024-04-20T11:27:38.203 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 64 2024-04-20T11:27:38.206 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.206+0000 7f41c0c00640 1 -- 172.21.15.152:0/1317805245 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f419806c8f0 msgr2=0x7f419806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:38.207 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.206+0000 7f41c0c00640 1 --2- 172.21.15.152:0/1317805245 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f419806c8f0 0x7f419806edb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f41a4001600 tx=0x7f41a4007990 comp rx=0 tx=0).stop 2024-04-20T11:27:38.207 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.206+0000 7f41c0c00640 1 -- 172.21.15.152:0/1317805245 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f41a00073d0 msgr2=0x7f41bc146df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:38.207 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.206+0000 7f41c0c00640 1 --2- 172.21.15.152:0/1317805245 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f41a00073d0 0x7f41bc146df0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f41ac007a90 tx=0x7f41ac00e530 comp rx=0 tx=0).stop 2024-04-20T11:27:38.207 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.207+0000 7f41c0c00640 1 -- 172.21.15.152:0/1317805245 shutdown_connections 2024-04-20T11:27:38.207 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.207+0000 7f41c0c00640 1 --2- 172.21.15.152:0/1317805245 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f419806c8f0 0x7f419806edb0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:38.207 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.207+0000 7f41c0c00640 1 --2- 172.21.15.152:0/1317805245 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41bc0b1530 0x7f41bc0b19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:38.207 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.207+0000 7f41c0c00640 1 --2- 172.21.15.152:0/1317805245 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f41bc1f0460 0x7f41bc1f08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:38.207 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.207+0000 7f41c0c00640 1 --2- 172.21.15.152:0/1317805245 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f41a00073d0 0x7f41bc146df0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:38.207 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.207+0000 7f41c0c00640 1 -- 172.21.15.152:0/1317805245 >> 172.21.15.152:0/1317805245 conn(0x7f41bc06b980 msgr2=0x7f41bc0ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:38.207 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.207+0000 7f41c0c00640 1 -- 172.21.15.152:0/1317805245 shutdown_connections 2024-04-20T11:27:38.207 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.207+0000 7f41c0c00640 1 -- 172.21.15.152:0/1317805245 wait complete. 2024-04-20T11:27:38.291 INFO:tasks.cephfs_test_runner:Starting test: test_binary_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc) Validate binary .meta file on legacy subvol root doesn't break the system 2024-04-20T11:27:38.291 INFO:tasks.cephfs_test_runner:test_binary_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc) 2024-04-20T11:27:38.292 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:27:38.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.493+0000 7f1b87200640 1 Processor -- start 2024-04-20T11:27:38.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.493+0000 7f1b87200640 1 -- start start 2024-04-20T11:27:38.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.494+0000 7f1b87200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f1b8013e530 0x7f1b8006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:38.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.494+0000 7f1b87200640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b800cf0a0 con 0x7f1b801414e0 2024-04-20T11:27:38.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.494+0000 7f1b87200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b800d00e0 con 0x7f1b8006f340 2024-04-20T11:27:38.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.494+0000 7f1b87200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f1b80141b00 con 0x7f1b8013e530 2024-04-20T11:27:38.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.495+0000 7f1b86800640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f1b801414e0 0x7f1b80071cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:39960/0 (socket says 172.21.15.152:39960) 2024-04-20T11:27:38.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.495+0000 7f1b86800640 1 -- 172.21.15.152:0/4046168026 learned_addr learned my addr 172.21.15.152:0/4046168026 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:27:38.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.495+0000 7f1b85e00640 1 --2- 172.21.15.152:0/4046168026 >> v2:172.21.15.175:3300/0 conn(0x7f1b8013e530 0x7f1b8006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:38.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.496+0000 7f1b84a00640 1 -- 172.21.15.152:0/4046168026 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2238956362 0 0) 0x7f1b7c001490 con 0x7f1b801414e0 2024-04-20T11:27:38.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.496+0000 7f1b84a00640 1 -- 172.21.15.152:0/4046168026 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1b68003520 con 0x7f1b801414e0 2024-04-20T11:27:38.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.496+0000 7f1b85e00640 1 -- 172.21.15.152:0/4046168026 >> v1:172.21.15.175:6789/0 conn(0x7f1b8006f340 legacy=0x7f1b80140dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:38.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.496+0000 7f1b85e00640 1 -- 172.21.15.152:0/4046168026 >> v1:172.21.15.152:6789/0 conn(0x7f1b801414e0 legacy=0x7f1b80071cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:38.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.496+0000 7f1b85e00640 1 -- 172.21.15.152:0/4046168026 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b800cf320 con 0x7f1b8013e530 2024-04-20T11:27:38.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.497+0000 7f1b85e00640 1 --2- 172.21.15.152:0/4046168026 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b8013e530 0x7f1b8006ee00 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1b740097a0 tx=0x7f1b7402b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d8c8f1b57ddc698 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:38.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.498+0000 7f1b84a00640 1 -- 172.21.15.152:0/4046168026 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b74034070 con 0x7f1b8013e530 2024-04-20T11:27:38.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.498+0000 7f1b84a00640 1 -- 172.21.15.152:0/4046168026 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b74030610 con 0x7f1b8013e530 2024-04-20T11:27:38.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.498+0000 7f1b84a00640 1 -- 172.21.15.152:0/4046168026 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b740309e0 con 0x7f1b8013e530 2024-04-20T11:27:38.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.498+0000 7f1b87200640 1 -- 172.21.15.152:0/4046168026 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b8013e530 msgr2=0x7f1b8006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:38.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.498+0000 7f1b87200640 1 --2- 172.21.15.152:0/4046168026 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b8013e530 0x7f1b8006ee00 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1b740097a0 tx=0x7f1b7402b960 comp rx=0 tx=0).stop 2024-04-20T11:27:38.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.498+0000 7f1b87200640 1 -- 172.21.15.152:0/4046168026 shutdown_connections 2024-04-20T11:27:38.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.498+0000 7f1b87200640 1 --2- 172.21.15.152:0/4046168026 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b8013e530 0x7f1b8006ee00 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:38.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.498+0000 7f1b87200640 1 -- 172.21.15.152:0/4046168026 >> 172.21.15.152:0/4046168026 conn(0x7f1b80067ae0 msgr2=0x7f1b8006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:38.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.498+0000 7f1b87200640 1 -- 172.21.15.152:0/4046168026 shutdown_connections 2024-04-20T11:27:38.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.498+0000 7f1b87200640 1 -- 172.21.15.152:0/4046168026 wait complete. 2024-04-20T11:27:38.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.500+0000 7f1b87200640 1 Processor -- start 2024-04-20T11:27:38.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.501+0000 7f1b87200640 1 -- start start 2024-04-20T11:27:38.502 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.502+0000 7f1b87200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1b8006f340 0x7f1b80154b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:38.502 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.502+0000 7f1b87200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1b8013e530 0x7f1b80156d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:38.502 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.502+0000 7f1b87200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b801414e0 0x7f1b8015c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:38.502 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.502+0000 7f1b87200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f1b800cf8e0 con 0x7f1b8013e530 2024-04-20T11:27:38.502 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.502+0000 7f1b87200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1b800cfc50 con 0x7f1b801414e0 2024-04-20T11:27:38.502 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.502+0000 7f1b87200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1b800723e0 con 0x7f1b8006f340 2024-04-20T11:27:38.502 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.502+0000 7f1b85400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1b8013e530 0x7f1b80156d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:38.502 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.502+0000 7f1b85400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1b8013e530 0x7f1b80156d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35286/0 (socket says 172.21.15.152:35286) 2024-04-20T11:27:38.502 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.502+0000 7f1b85400640 1 -- 172.21.15.152:0/2847480688 learned_addr learned my addr 172.21.15.152:0/2847480688 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:38.503 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.503+0000 7f1b85e00640 1 --2- 172.21.15.152:0/2847480688 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1b8006f340 0x7f1b80154b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:38.503 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.503+0000 7f1b85400640 1 -- 172.21.15.152:0/2847480688 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1b8006f340 msgr2=0x7f1b80154b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:38.503 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.503+0000 7f1b85400640 1 --2- 172.21.15.152:0/2847480688 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1b8006f340 0x7f1b80154b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:38.503 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.503+0000 7f1b85400640 1 -- 172.21.15.152:0/2847480688 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b801414e0 msgr2=0x7f1b8015c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:38.503 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.503+0000 7f1b85400640 1 --2- 172.21.15.152:0/2847480688 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b801414e0 0x7f1b8015c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:38.503 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.503+0000 7f1b85400640 1 -- 172.21.15.152:0/2847480688 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b800b1180 con 0x7f1b8013e530 2024-04-20T11:27:38.503 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.503+0000 7f1b85400640 1 --2- 172.21.15.152:0/2847480688 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1b8013e530 0x7f1b80156d20 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f1b7000c400 tx=0x7f1b7000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:38.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.503+0000 7f1b7a000640 1 -- 172.21.15.152:0/2847480688 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b700052d0 con 0x7f1b8013e530 2024-04-20T11:27:38.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.503+0000 7f1b7a000640 1 -- 172.21.15.152:0/2847480688 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b70005490 con 0x7f1b8013e530 2024-04-20T11:27:38.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.503+0000 7f1b7a000640 1 -- 172.21.15.152:0/2847480688 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b70007260 con 0x7f1b8013e530 2024-04-20T11:27:38.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.503+0000 7f1b87200640 1 -- 172.21.15.152:0/2847480688 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b800b1460 con 0x7f1b8013e530 2024-04-20T11:27:38.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.503+0000 7f1b87200640 1 -- 172.21.15.152:0/2847480688 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b800b1a20 con 0x7f1b8013e530 2024-04-20T11:27:38.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.504+0000 7f1b87200640 1 -- 172.21.15.152:0/2847480688 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b800b1c20 con 0x7f1b8013e530 2024-04-20T11:27:38.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.506+0000 7f1b7a000640 1 -- 172.21.15.152:0/2847480688 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1b70007400 con 0x7f1b8013e530 2024-04-20T11:27:38.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.506+0000 7f1b7a000640 1 --2- 172.21.15.152:0/2847480688 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f1b6406c8f0 0x7f1b6406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:38.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.507+0000 7f1b7a000640 1 -- 172.21.15.152:0/2847480688 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f1b7008b080 con 0x7f1b8013e530 2024-04-20T11:27:38.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.507+0000 7f1b85e00640 1 --2- 172.21.15.152:0/2847480688 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f1b6406c8f0 0x7f1b6406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:38.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.507+0000 7f1b85e00640 1 --2- 172.21.15.152:0/2847480688 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f1b6406c8f0 0x7f1b6406edb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1b74036000 tx=0x7f1b7403b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:38.510 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.511+0000 7f1b7a000640 1 -- 172.21.15.152:0/2847480688 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f1b800b1c20 con 0x7f1b8013e530 2024-04-20T11:27:38.715 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.715+0000 7f1b87200640 1 -- 172.21.15.152:0/2847480688 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1b80075e60 con 0x7f1b8013e530 2024-04-20T11:27:38.718 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.718+0000 7f1b7a000640 1 -- 172.21.15.152:0/2847480688 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16665 (secure 0 0 0) 0x7f1b700594b0 con 0x7f1b8013e530 2024-04-20T11:27:38.719 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:27:38.719 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":20,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:26:59.675971+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:26:56.261424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T11:26:57.079286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:27:38.720 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.721+0000 7f1b87200640 1 -- 172.21.15.152:0/2847480688 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f1b6406c8f0 msgr2=0x7f1b6406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:38.721 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.721+0000 7f1b87200640 1 --2- 172.21.15.152:0/2847480688 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f1b6406c8f0 0x7f1b6406edb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1b74036000 tx=0x7f1b7403b040 comp rx=0 tx=0).stop 2024-04-20T11:27:38.721 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.721+0000 7f1b87200640 1 -- 172.21.15.152:0/2847480688 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1b8013e530 msgr2=0x7f1b80156d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:38.721 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.721+0000 7f1b87200640 1 --2- 172.21.15.152:0/2847480688 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1b8013e530 0x7f1b80156d20 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f1b7000c400 tx=0x7f1b7000c8c0 comp rx=0 tx=0).stop 2024-04-20T11:27:38.721 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.721+0000 7f1b87200640 1 -- 172.21.15.152:0/2847480688 shutdown_connections 2024-04-20T11:27:38.721 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.721+0000 7f1b87200640 1 --2- 172.21.15.152:0/2847480688 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f1b6406c8f0 0x7f1b6406edb0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:38.721 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.721+0000 7f1b87200640 1 --2- 172.21.15.152:0/2847480688 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b801414e0 0x7f1b8015c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:38.721 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.721+0000 7f1b87200640 1 --2- 172.21.15.152:0/2847480688 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1b8013e530 0x7f1b80156d20 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:38.721 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.721+0000 7f1b87200640 1 --2- 172.21.15.152:0/2847480688 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1b8006f340 0x7f1b80154b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:38.721 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.721+0000 7f1b87200640 1 -- 172.21.15.152:0/2847480688 >> 172.21.15.152:0/2847480688 conn(0x7f1b80067ae0 msgr2=0x7f1b80159570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:38.721 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.721+0000 7f1b87200640 1 -- 172.21.15.152:0/2847480688 shutdown_connections 2024-04-20T11:27:38.721 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.721+0000 7f1b87200640 1 -- 172.21.15.152:0/2847480688 wait complete. 2024-04-20T11:27:38.738 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T11:26:47.246571+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T11:26:56.261424+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T11:26:57.079286+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T11:27:38.738 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T11:27:38.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.937+0000 7f3d9ea00640 1 Processor -- start 2024-04-20T11:27:38.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.938+0000 7f3d9ea00640 1 -- start start 2024-04-20T11:27:38.939 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.939+0000 7f3d9ea00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3d980b2880 0x7f3d980b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:38.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.939+0000 7f3d9ea00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f3d980cd1b0 0x7f3d980cd600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:38.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.939+0000 7f3d9ea00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d981463e0 con 0x7f3d98146000 2024-04-20T11:27:38.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.939+0000 7f3d9ea00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f3d980b4750 con 0x7f3d980cd1b0 2024-04-20T11:27:38.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.939+0000 7f3d9ea00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f3d980b3e20 con 0x7f3d980b2880 2024-04-20T11:27:38.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.939+0000 7f3d97e00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f3d98146000 0x7f3d980b2170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:39966/0 (socket says 172.21.15.152:39966) 2024-04-20T11:27:38.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.939+0000 7f3d97e00640 1 -- 172.21.15.152:0/817698796 learned_addr learned my addr 172.21.15.152:0/817698796 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:27:38.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.939+0000 7f3d9ca00640 1 --2- 172.21.15.152:0/817698796 >> v2:172.21.15.152:3300/0 conn(0x7f3d980cd1b0 0x7f3d980cd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:38.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.940+0000 7f3d8f400640 1 --2- 172.21.15.152:0/817698796 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3d980b2880 0x7f3d980b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:38.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.940+0000 7f3d97400640 1 -- 172.21.15.152:0/817698796 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3996302459 0 0) 0x7f3d981463e0 con 0x7f3d98146000 2024-04-20T11:27:38.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.940+0000 7f3d97400640 1 -- 172.21.15.152:0/817698796 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3d7c003520 con 0x7f3d98146000 2024-04-20T11:27:38.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.940+0000 7f3d9ca00640 1 -- 172.21.15.152:0/817698796 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3d980b2880 msgr2=0x7f3d980b4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:38.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.940+0000 7f3d9ca00640 1 --2- 172.21.15.152:0/817698796 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3d980b2880 0x7f3d980b4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:38.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.940+0000 7f3d9ca00640 1 -- 172.21.15.152:0/817698796 >> v1:172.21.15.152:6789/0 conn(0x7f3d98146000 legacy=0x7f3d980b2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:38.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.940+0000 7f3d9ca00640 1 -- 172.21.15.152:0/817698796 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d980b4100 con 0x7f3d980cd1b0 2024-04-20T11:27:38.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.941+0000 7f3d9ca00640 1 --2- 172.21.15.152:0/817698796 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3d980cd1b0 0x7f3d980cd600 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f3d90009d90 tx=0x7f3d9002be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c151ecfcce171b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:38.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.941+0000 7f3d97400640 1 -- 172.21.15.152:0/817698796 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d90039070 con 0x7f3d980cd1b0 2024-04-20T11:27:38.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.941+0000 7f3d97400640 1 -- 172.21.15.152:0/817698796 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d9002cbc0 con 0x7f3d980cd1b0 2024-04-20T11:27:38.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.941+0000 7f3d97400640 1 -- 172.21.15.152:0/817698796 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d90034440 con 0x7f3d980cd1b0 2024-04-20T11:27:38.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.942+0000 7f3d9ea00640 1 -- 172.21.15.152:0/817698796 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3d980cd1b0 msgr2=0x7f3d980cd600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:38.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.942+0000 7f3d9ea00640 1 --2- 172.21.15.152:0/817698796 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3d980cd1b0 0x7f3d980cd600 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f3d90009d90 tx=0x7f3d9002be10 comp rx=0 tx=0).stop 2024-04-20T11:27:38.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.942+0000 7f3d9ea00640 1 -- 172.21.15.152:0/817698796 shutdown_connections 2024-04-20T11:27:38.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.942+0000 7f3d9ea00640 1 --2- 172.21.15.152:0/817698796 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3d980b2880 0x7f3d980b4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:38.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.942+0000 7f3d9ea00640 1 --2- 172.21.15.152:0/817698796 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3d980cd1b0 0x7f3d980cd600 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:38.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.942+0000 7f3d9ea00640 1 -- 172.21.15.152:0/817698796 >> 172.21.15.152:0/817698796 conn(0x7f3d9806b980 msgr2=0x7f3d9806bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:38.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.942+0000 7f3d9ea00640 1 -- 172.21.15.152:0/817698796 shutdown_connections 2024-04-20T11:27:38.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.942+0000 7f3d9ea00640 1 -- 172.21.15.152:0/817698796 wait complete. 2024-04-20T11:27:38.944 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.944+0000 7f3d9ea00640 1 Processor -- start 2024-04-20T11:27:38.944 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.944+0000 7f3d9ea00640 1 -- start start 2024-04-20T11:27:38.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.945+0000 7f3d9ea00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3d980b2880 0x7f3d9815be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:38.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.945+0000 7f3d9ea00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3d980cd1b0 0x7f3d98140270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:38.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.945+0000 7f3d9ea00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3d98146000 0x7f3d9815e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:38.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.945+0000 7f3d9ea00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f3d980cdcc0 con 0x7f3d980b2880 2024-04-20T11:27:38.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.945+0000 7f3d9ea00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3d980cdb40 con 0x7f3d98146000 2024-04-20T11:27:38.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.945+0000 7f3d9ea00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f3d9815ccf0 con 0x7f3d980cd1b0 2024-04-20T11:27:38.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.945+0000 7f3d97e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3d980b2880 0x7f3d9815be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:38.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.945+0000 7f3d97e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3d980b2880 0x7f3d9815be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35308/0 (socket says 172.21.15.152:35308) 2024-04-20T11:27:38.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.945+0000 7f3d97e00640 1 -- 172.21.15.152:0/1404889914 learned_addr learned my addr 172.21.15.152:0/1404889914 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:38.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.946+0000 7f3d9ca00640 1 --2- 172.21.15.152:0/1404889914 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3d98146000 0x7f3d9815e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:38.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.946+0000 7f3d8f400640 1 --2- 172.21.15.152:0/1404889914 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3d980cd1b0 0x7f3d98140270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:38.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.946+0000 7f3d97e00640 1 -- 172.21.15.152:0/1404889914 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3d980cd1b0 msgr2=0x7f3d98140270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:38.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.946+0000 7f3d97e00640 1 --2- 172.21.15.152:0/1404889914 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3d980cd1b0 0x7f3d98140270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:38.948 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.946+0000 7f3d97e00640 1 -- 172.21.15.152:0/1404889914 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3d98146000 msgr2=0x7f3d9815e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:38.948 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.946+0000 7f3d97e00640 1 --2- 172.21.15.152:0/1404889914 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3d98146000 0x7f3d9815e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:38.948 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.946+0000 7f3d97e00640 1 -- 172.21.15.152:0/1404889914 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d9813f1d0 con 0x7f3d980b2880 2024-04-20T11:27:38.948 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.946+0000 7f3d97e00640 1 --2- 172.21.15.152:0/1404889914 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3d980b2880 0x7f3d9815be10 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3d8800c3c0 tx=0x7f3d8800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:38.948 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.946+0000 7f3d94c00640 1 -- 172.21.15.152:0/1404889914 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d88017070 con 0x7f3d980b2880 2024-04-20T11:27:38.948 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.946+0000 7f3d94c00640 1 -- 172.21.15.152:0/1404889914 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d88005ce0 con 0x7f3d980b2880 2024-04-20T11:27:38.948 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.946+0000 7f3d94c00640 1 -- 172.21.15.152:0/1404889914 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d880072f0 con 0x7f3d980b2880 2024-04-20T11:27:38.948 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.946+0000 7f3d9ea00640 1 -- 172.21.15.152:0/1404889914 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d9813f4b0 con 0x7f3d980b2880 2024-04-20T11:27:38.948 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.947+0000 7f3d9ea00640 1 -- 172.21.15.152:0/1404889914 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d9813fa70 con 0x7f3d980b2880 2024-04-20T11:27:38.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.948+0000 7f3d9ea00640 1 -- 172.21.15.152:0/1404889914 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d980689f0 con 0x7f3d980b2880 2024-04-20T11:27:38.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.949+0000 7f3d94c00640 1 -- 172.21.15.152:0/1404889914 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3d8801a070 con 0x7f3d980b2880 2024-04-20T11:27:38.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.949+0000 7f3d94c00640 1 --2- 172.21.15.152:0/1404889914 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f3d7406c8f0 0x7f3d7406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:38.950 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.950+0000 7f3d94c00640 1 -- 172.21.15.152:0/1404889914 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f3d8808afe0 con 0x7f3d980b2880 2024-04-20T11:27:38.950 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.950+0000 7f3d8f400640 1 --2- 172.21.15.152:0/1404889914 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f3d7406c8f0 0x7f3d7406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:38.950 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.950+0000 7f3d8f400640 1 --2- 172.21.15.152:0/1404889914 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f3d7406c8f0 0x7f3d7406edb0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f3d980b0d70 tx=0x7f3d80008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:38.954 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:38.954+0000 7f3d94c00640 1 -- 172.21.15.152:0/1404889914 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f3d88059370 con 0x7f3d980b2880 2024-04-20T11:27:39.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.158+0000 7f3d9ea00640 1 -- 172.21.15.152:0/1404889914 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3d980b2270 con 0x7f3d980b2880 2024-04-20T11:27:39.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.159+0000 7f3d94c00640 1 -- 172.21.15.152:0/1404889914 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f3d8805f020 con 0x7f3d980b2880 2024-04-20T11:27:39.158 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 1 2024-04-20T11:27:39.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.161+0000 7f3d9ea00640 1 -- 172.21.15.152:0/1404889914 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f3d7406c8f0 msgr2=0x7f3d7406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:39.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.161+0000 7f3d9ea00640 1 --2- 172.21.15.152:0/1404889914 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f3d7406c8f0 0x7f3d7406edb0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f3d980b0d70 tx=0x7f3d80008040 comp rx=0 tx=0).stop 2024-04-20T11:27:39.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.161+0000 7f3d9ea00640 1 -- 172.21.15.152:0/1404889914 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3d980b2880 msgr2=0x7f3d9815be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:39.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.161+0000 7f3d9ea00640 1 --2- 172.21.15.152:0/1404889914 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3d980b2880 0x7f3d9815be10 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3d8800c3c0 tx=0x7f3d8800c880 comp rx=0 tx=0).stop 2024-04-20T11:27:39.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.161+0000 7f3d9ea00640 1 -- 172.21.15.152:0/1404889914 shutdown_connections 2024-04-20T11:27:39.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.161+0000 7f3d9ea00640 1 --2- 172.21.15.152:0/1404889914 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f3d7406c8f0 0x7f3d7406edb0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:39.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.161+0000 7f3d9ea00640 1 --2- 172.21.15.152:0/1404889914 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3d98146000 0x7f3d9815e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:39.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.161+0000 7f3d9ea00640 1 --2- 172.21.15.152:0/1404889914 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3d980cd1b0 0x7f3d98140270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:39.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.161+0000 7f3d9ea00640 1 --2- 172.21.15.152:0/1404889914 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3d980b2880 0x7f3d9815be10 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:39.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.161+0000 7f3d9ea00640 1 -- 172.21.15.152:0/1404889914 >> 172.21.15.152:0/1404889914 conn(0x7f3d9806b980 msgr2=0x7f3d980b1810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:39.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.162+0000 7f3d9ea00640 1 -- 172.21.15.152:0/1404889914 shutdown_connections 2024-04-20T11:27:39.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.162+0000 7f3d9ea00640 1 -- 172.21.15.152:0/1404889914 wait complete. 2024-04-20T11:27:39.181 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T11:27:39.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.381+0000 7fef98000640 1 Processor -- start 2024-04-20T11:27:39.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.381+0000 7fef98000640 1 -- start start 2024-04-20T11:27:39.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.382+0000 7fef98000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fef9009b150 0x7fef900cd170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:39.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.382+0000 7fef98000640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef9009b9d0 con 0x7fef9015fdd0 2024-04-20T11:27:39.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.382+0000 7fef98000640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef9009bcb0 con 0x7fef900b34a0 2024-04-20T11:27:39.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.382+0000 7fef98000640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fef900b3ce0 con 0x7fef9009b150 2024-04-20T11:27:39.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.383+0000 7fef95600640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fef9015fdd0 0x7fef9009aa40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:39974/0 (socket says 172.21.15.152:39974) 2024-04-20T11:27:39.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.383+0000 7fef95600640 1 -- 172.21.15.152:0/1819808501 learned_addr learned my addr 172.21.15.152:0/1819808501 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:27:39.383 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.383+0000 7fef94c00640 1 --2- 172.21.15.152:0/1819808501 >> v2:172.21.15.152:3300/0 conn(0x7fef9009b150 0x7fef900cd170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:39.383 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.383+0000 7fef8fe00640 1 -- 172.21.15.152:0/1819808501 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1838579155 0 0) 0x7fef9009b9d0 con 0x7fef9015fdd0 2024-04-20T11:27:39.384 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.383+0000 7fef8fe00640 1 -- 172.21.15.152:0/1819808501 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fef74003520 con 0x7fef9015fdd0 2024-04-20T11:27:39.384 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.383+0000 7fef94c00640 1 -- 172.21.15.152:0/1819808501 >> v1:172.21.15.175:6789/0 conn(0x7fef900b34a0 legacy=0x7fef900cd6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:39.384 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.383+0000 7fef94c00640 1 -- 172.21.15.152:0/1819808501 >> v1:172.21.15.152:6789/0 conn(0x7fef9015fdd0 legacy=0x7fef9009aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:39.384 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.383+0000 7fef94c00640 1 -- 172.21.15.152:0/1819808501 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef900d01d0 con 0x7fef9009b150 2024-04-20T11:27:39.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.385+0000 7fef94c00640 1 --2- 172.21.15.152:0/1819808501 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef9009b150 0x7fef900cd170 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fef7c009d90 tx=0x7fef7c02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53acc15516dfbdff server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:39.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.385+0000 7fef8fe00640 1 -- 172.21.15.152:0/1819808501 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef7c039070 con 0x7fef9009b150 2024-04-20T11:27:39.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.385+0000 7fef8fe00640 1 -- 172.21.15.152:0/1819808501 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef7c02cc60 con 0x7fef9009b150 2024-04-20T11:27:39.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.385+0000 7fef8fe00640 1 -- 172.21.15.152:0/1819808501 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef7c034660 con 0x7fef9009b150 2024-04-20T11:27:39.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.385+0000 7fef98000640 1 -- 172.21.15.152:0/1819808501 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef9009b150 msgr2=0x7fef900cd170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:39.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.385+0000 7fef98000640 1 --2- 172.21.15.152:0/1819808501 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef9009b150 0x7fef900cd170 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fef7c009d90 tx=0x7fef7c02beb0 comp rx=0 tx=0).stop 2024-04-20T11:27:39.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.386+0000 7fef98000640 1 -- 172.21.15.152:0/1819808501 shutdown_connections 2024-04-20T11:27:39.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.386+0000 7fef98000640 1 --2- 172.21.15.152:0/1819808501 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef9009b150 0x7fef900cd170 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:39.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.386+0000 7fef98000640 1 -- 172.21.15.152:0/1819808501 >> 172.21.15.152:0/1819808501 conn(0x7fef900b4050 msgr2=0x7fef900b4460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:39.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.386+0000 7fef98000640 1 -- 172.21.15.152:0/1819808501 shutdown_connections 2024-04-20T11:27:39.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.386+0000 7fef98000640 1 -- 172.21.15.152:0/1819808501 wait complete. 2024-04-20T11:27:39.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.387+0000 7fef98000640 1 Processor -- start 2024-04-20T11:27:39.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.388+0000 7fef98000640 1 -- start start 2024-04-20T11:27:39.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.389+0000 7fef98000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef9009b150 0x7fef900ca700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:39.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.389+0000 7fef98000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef900b34a0 0x7fef901f4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:39.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.389+0000 7fef98000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef9015fdd0 0x7fef9006dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:39.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.389+0000 7fef98000640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fef900c70c0 con 0x7fef9015fdd0 2024-04-20T11:27:39.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.389+0000 7fef98000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fef9009b780 con 0x7fef900b34a0 2024-04-20T11:27:39.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.389+0000 7fef98000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fef900c7240 con 0x7fef9009b150 2024-04-20T11:27:39.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.389+0000 7fef96000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef9015fdd0 0x7fef9006dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:39.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.389+0000 7fef96000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef9015fdd0 0x7fef9006dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35316/0 (socket says 172.21.15.152:35316) 2024-04-20T11:27:39.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.389+0000 7fef96000640 1 -- 172.21.15.152:0/3221971437 learned_addr learned my addr 172.21.15.152:0/3221971437 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:39.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.389+0000 7fef95600640 1 --2- 172.21.15.152:0/3221971437 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef9009b150 0x7fef900ca700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:39.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.389+0000 7fef94c00640 1 --2- 172.21.15.152:0/3221971437 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef900b34a0 0x7fef901f4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:39.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.389+0000 7fef96000640 1 -- 172.21.15.152:0/3221971437 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef9009b150 msgr2=0x7fef900ca700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:39.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.389+0000 7fef96000640 1 --2- 172.21.15.152:0/3221971437 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef9009b150 0x7fef900ca700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:39.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.389+0000 7fef96000640 1 -- 172.21.15.152:0/3221971437 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef900b34a0 msgr2=0x7fef901f4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:39.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.389+0000 7fef96000640 1 --2- 172.21.15.152:0/3221971437 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef900b34a0 0x7fef901f4f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:39.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.389+0000 7fef96000640 1 -- 172.21.15.152:0/3221971437 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef901f5480 con 0x7fef9015fdd0 2024-04-20T11:27:39.391 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.390+0000 7fef96000640 1 --2- 172.21.15.152:0/3221971437 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef9015fdd0 0x7fef9006dd50 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fef8800c410 tx=0x7fef8800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:39.391 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.390+0000 7fef8d600640 1 -- 172.21.15.152:0/3221971437 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef88011790 con 0x7fef9015fdd0 2024-04-20T11:27:39.391 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.390+0000 7fef8d600640 1 -- 172.21.15.152:0/3221971437 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef88017070 con 0x7fef9015fdd0 2024-04-20T11:27:39.391 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.390+0000 7fef8d600640 1 -- 172.21.15.152:0/3221971437 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef88011bb0 con 0x7fef9015fdd0 2024-04-20T11:27:39.391 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.390+0000 7fef98000640 1 -- 172.21.15.152:0/3221971437 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef900b7020 con 0x7fef9015fdd0 2024-04-20T11:27:39.391 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.390+0000 7fef98000640 1 -- 172.21.15.152:0/3221971437 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef900b75e0 con 0x7fef9015fdd0 2024-04-20T11:27:39.391 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.392+0000 7fef98000640 1 -- 172.21.15.152:0/3221971437 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef90069100 con 0x7fef9015fdd0 2024-04-20T11:27:39.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.393+0000 7fef8d600640 1 -- 172.21.15.152:0/3221971437 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fef8801c460 con 0x7fef9015fdd0 2024-04-20T11:27:39.393 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.393+0000 7fef8d600640 1 --2- 172.21.15.152:0/3221971437 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fef7006c8f0 0x7fef7006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:39.393 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.394+0000 7fef8d600640 1 -- 172.21.15.152:0/3221971437 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fef8808b730 con 0x7fef9015fdd0 2024-04-20T11:27:39.393 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.394+0000 7fef95600640 1 --2- 172.21.15.152:0/3221971437 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fef7006c8f0 0x7fef7006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:39.394 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.394+0000 7fef95600640 1 --2- 172.21.15.152:0/3221971437 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fef7006c8f0 0x7fef7006edb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fef80005880 tx=0x7fef80009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:39.397 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.398+0000 7fef8d600640 1 -- 172.21.15.152:0/3221971437 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fef88021020 con 0x7fef9015fdd0 2024-04-20T11:27:39.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.601+0000 7fef98000640 1 -- 172.21.15.152:0/3221971437 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fef900cea50 con 0x7fef9015fdd0 2024-04-20T11:27:39.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.602+0000 7fef8d600640 1 -- 172.21.15.152:0/3221971437 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7fef901f5480 con 0x7fef9015fdd0 2024-04-20T11:27:39.602 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 32 2024-04-20T11:27:39.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.606+0000 7fef98000640 1 -- 172.21.15.152:0/3221971437 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fef7006c8f0 msgr2=0x7fef7006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:39.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.606+0000 7fef98000640 1 --2- 172.21.15.152:0/3221971437 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fef7006c8f0 0x7fef7006edb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fef80005880 tx=0x7fef80009040 comp rx=0 tx=0).stop 2024-04-20T11:27:39.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.606+0000 7fef98000640 1 -- 172.21.15.152:0/3221971437 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef9015fdd0 msgr2=0x7fef9006dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:39.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.606+0000 7fef98000640 1 --2- 172.21.15.152:0/3221971437 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef9015fdd0 0x7fef9006dd50 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fef8800c410 tx=0x7fef8800c8d0 comp rx=0 tx=0).stop 2024-04-20T11:27:39.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.606+0000 7fef98000640 1 -- 172.21.15.152:0/3221971437 shutdown_connections 2024-04-20T11:27:39.607 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.606+0000 7fef98000640 1 --2- 172.21.15.152:0/3221971437 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fef7006c8f0 0x7fef7006edb0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:39.607 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.606+0000 7fef98000640 1 --2- 172.21.15.152:0/3221971437 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef9015fdd0 0x7fef9006dd50 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:39.607 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.606+0000 7fef98000640 1 --2- 172.21.15.152:0/3221971437 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef900b34a0 0x7fef901f4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:39.607 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.606+0000 7fef98000640 1 --2- 172.21.15.152:0/3221971437 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef9009b150 0x7fef900ca700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:39.607 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.606+0000 7fef98000640 1 -- 172.21.15.152:0/3221971437 >> 172.21.15.152:0/3221971437 conn(0x7fef900b4050 msgr2=0x7fef900cefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:39.607 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.607+0000 7fef98000640 1 -- 172.21.15.152:0/3221971437 shutdown_connections 2024-04-20T11:27:39.607 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.607+0000 7fef98000640 1 -- 172.21.15.152:0/3221971437 wait complete. 2024-04-20T11:27:39.624 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T11:27:39.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.823+0000 7f8a3f200640 1 Processor -- start 2024-04-20T11:27:39.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.823+0000 7f8a3f200640 1 -- start start 2024-04-20T11:27:39.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.824+0000 7f8a3f200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f8a400b2a40 0x7f8a400cce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:39.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.824+0000 7f8a3f200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f8a4009ba50 0x7f8a400cd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:39.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.824+0000 7f8a3f200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a400d0690 con 0x7f8a400b3560 2024-04-20T11:27:39.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.824+0000 7f8a3f200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f8a400d0160 con 0x7f8a400b2a40 2024-04-20T11:27:39.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.824+0000 7f8a3f200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f8a400d0d70 con 0x7f8a4009ba50 2024-04-20T11:27:39.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.825+0000 7f8a3de00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f8a400b2a40 0x7f8a400cce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:39.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.825+0000 7f8a3de00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f8a400b2a40 0x7f8a400cce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35330/0 (socket says 172.21.15.152:35330) 2024-04-20T11:27:39.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.825+0000 7f8a3de00640 1 -- 172.21.15.152:0/711007176 learned_addr learned my addr 172.21.15.152:0/711007176 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:39.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.825+0000 7f8a3e800640 1 --2- 172.21.15.152:0/711007176 >> v2:172.21.15.175:3300/0 conn(0x7f8a4009ba50 0x7f8a400cd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:39.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.825+0000 7f8a3de00640 1 -- 172.21.15.152:0/711007176 >> v2:172.21.15.175:3300/0 conn(0x7f8a4009ba50 msgr2=0x7f8a400cd3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:39.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.825+0000 7f8a3de00640 1 --2- 172.21.15.152:0/711007176 >> v2:172.21.15.175:3300/0 conn(0x7f8a4009ba50 0x7f8a400cd3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:39.826 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.825+0000 7f8a3de00640 1 -- 172.21.15.152:0/711007176 >> v1:172.21.15.175:6789/0 conn(0x7f8a400b3560 legacy=0x7f8a400c5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:39.826 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.825+0000 7f8a3de00640 1 -- 172.21.15.152:0/711007176 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a400d03b0 con 0x7f8a400b2a40 2024-04-20T11:27:39.826 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.825+0000 7f8a3e800640 1 --2- 172.21.15.152:0/711007176 >> v2:172.21.15.175:3300/0 conn(0x7f8a4009ba50 0x7f8a400cd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:27:39.826 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.827+0000 7f8a3de00640 1 --2- 172.21.15.152:0/711007176 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8a400b2a40 0x7f8a400cce90 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f8a40073040 tx=0x7f8a3002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=858536844369a2a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:39.826 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.827+0000 7f8a3ca00640 1 -- 172.21.15.152:0/711007176 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a3002e070 con 0x7f8a400b2a40 2024-04-20T11:27:39.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.827+0000 7f8a3ca00640 1 -- 172.21.15.152:0/711007176 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a30037070 con 0x7f8a400b2a40 2024-04-20T11:27:39.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.827+0000 7f8a3ca00640 1 -- 172.21.15.152:0/711007176 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a30031a00 con 0x7f8a400b2a40 2024-04-20T11:27:39.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.827+0000 7f8a3f200640 1 -- 172.21.15.152:0/711007176 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8a400b2a40 msgr2=0x7f8a400cce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:39.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.828+0000 7f8a3f200640 1 --2- 172.21.15.152:0/711007176 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8a400b2a40 0x7f8a400cce90 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f8a40073040 tx=0x7f8a3002b880 comp rx=0 tx=0).stop 2024-04-20T11:27:39.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.828+0000 7f8a3f200640 1 -- 172.21.15.152:0/711007176 shutdown_connections 2024-04-20T11:27:39.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.828+0000 7f8a3f200640 1 --2- 172.21.15.152:0/711007176 >> v2:172.21.15.175:3300/0 conn(0x7f8a4009ba50 0x7f8a400cd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:39.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.828+0000 7f8a3f200640 1 --2- 172.21.15.152:0/711007176 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8a400b2a40 0x7f8a400cce90 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:39.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.828+0000 7f8a3f200640 1 -- 172.21.15.152:0/711007176 >> 172.21.15.152:0/711007176 conn(0x7f8a400b3e70 msgr2=0x7f8a400b4280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:39.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.828+0000 7f8a3f200640 1 -- 172.21.15.152:0/711007176 shutdown_connections 2024-04-20T11:27:39.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.828+0000 7f8a3f200640 1 -- 172.21.15.152:0/711007176 wait complete. 2024-04-20T11:27:39.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.829+0000 7f8a3f200640 1 Processor -- start 2024-04-20T11:27:39.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.830+0000 7f8a3f200640 1 -- start start 2024-04-20T11:27:39.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.830+0000 7f8a3f200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8a400b2a40 0x7f8a400b0860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:39.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.830+0000 7f8a3f200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8a400b3560 0x7f8a40152920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:39.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.830+0000 7f8a3f200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8a400afac0 0x7f8a400c4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:39.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.830+0000 7f8a3f200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f8a400d1200 con 0x7f8a400afac0 2024-04-20T11:27:39.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.830+0000 7f8a3f200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8a400cd910 con 0x7f8a400b2a40 2024-04-20T11:27:39.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.830+0000 7f8a3f200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8a400b2f80 con 0x7f8a400b3560 2024-04-20T11:27:39.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.831+0000 7f8a3e800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8a400afac0 0x7f8a400c4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:39.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.831+0000 7f8a3e800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8a400afac0 0x7f8a400c4e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35338/0 (socket says 172.21.15.152:35338) 2024-04-20T11:27:39.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.831+0000 7f8a3e800640 1 -- 172.21.15.152:0/1559043467 learned_addr learned my addr 172.21.15.152:0/1559043467 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:39.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.831+0000 7f8a3d400640 1 --2- 172.21.15.152:0/1559043467 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8a400b3560 0x7f8a40152920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:39.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.831+0000 7f8a3de00640 1 --2- 172.21.15.152:0/1559043467 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8a400b2a40 0x7f8a400b0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:39.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.831+0000 7f8a3e800640 1 -- 172.21.15.152:0/1559043467 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8a400b3560 msgr2=0x7f8a40152920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:39.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.831+0000 7f8a3e800640 1 --2- 172.21.15.152:0/1559043467 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8a400b3560 0x7f8a40152920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:39.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.831+0000 7f8a3e800640 1 -- 172.21.15.152:0/1559043467 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8a400b2a40 msgr2=0x7f8a400b0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:39.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.831+0000 7f8a3e800640 1 --2- 172.21.15.152:0/1559043467 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8a400b2a40 0x7f8a400b0860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:39.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.831+0000 7f8a3e800640 1 -- 172.21.15.152:0/1559043467 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a400b0da0 con 0x7f8a400afac0 2024-04-20T11:27:39.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.832+0000 7f8a3e800640 1 --2- 172.21.15.152:0/1559043467 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8a400afac0 0x7f8a400c4e60 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f8a3800cf20 tx=0x7f8a3800a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:39.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.832+0000 7f8a36000640 1 -- 172.21.15.152:0/1559043467 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a3801a070 con 0x7f8a400afac0 2024-04-20T11:27:39.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.832+0000 7f8a36000640 1 -- 172.21.15.152:0/1559043467 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a3800d840 con 0x7f8a400afac0 2024-04-20T11:27:39.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.832+0000 7f8a3f200640 1 -- 172.21.15.152:0/1559043467 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a40152e60 con 0x7f8a400afac0 2024-04-20T11:27:39.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.832+0000 7f8a36000640 1 -- 172.21.15.152:0/1559043467 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a3800dc30 con 0x7f8a400afac0 2024-04-20T11:27:39.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.832+0000 7f8a3f200640 1 -- 172.21.15.152:0/1559043467 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a400b0310 con 0x7f8a400afac0 2024-04-20T11:27:39.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.833+0000 7f8a3f200640 1 -- 172.21.15.152:0/1559043467 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a40075e60 con 0x7f8a400afac0 2024-04-20T11:27:39.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.834+0000 7f8a36000640 1 -- 172.21.15.152:0/1559043467 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8a3801d070 con 0x7f8a400afac0 2024-04-20T11:27:39.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.835+0000 7f8a36000640 1 --2- 172.21.15.152:0/1559043467 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f8a2006c8f0 0x7f8a2006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:39.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.835+0000 7f8a36000640 1 -- 172.21.15.152:0/1559043467 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f8a3808c490 con 0x7f8a400afac0 2024-04-20T11:27:39.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.835+0000 7f8a3de00640 1 --2- 172.21.15.152:0/1559043467 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f8a2006c8f0 0x7f8a2006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:39.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.836+0000 7f8a3de00640 1 --2- 172.21.15.152:0/1559043467 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f8a2006c8f0 0x7f8a2006edb0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8a30036000 tx=0x7f8a3003d040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:39.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:39.840+0000 7f8a36000640 1 -- 172.21.15.152:0/1559043467 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f8a3805a820 con 0x7f8a400afac0 2024-04-20T11:27:40.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:40.038+0000 7f8a3f200640 1 -- 172.21.15.152:0/1559043467 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f8a400ce490 con 0x7f8a400afac0 2024-04-20T11:27:40.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:40.039+0000 7f8a36000640 1 -- 172.21.15.152:0/1559043467 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f8a3805a120 con 0x7f8a400afac0 2024-04-20T11:27:40.038 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 64 2024-04-20T11:27:40.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:40.041+0000 7f8a3f200640 1 -- 172.21.15.152:0/1559043467 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f8a2006c8f0 msgr2=0x7f8a2006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:40.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:40.041+0000 7f8a3f200640 1 --2- 172.21.15.152:0/1559043467 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f8a2006c8f0 0x7f8a2006edb0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8a30036000 tx=0x7f8a3003d040 comp rx=0 tx=0).stop 2024-04-20T11:27:40.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:40.041+0000 7f8a3f200640 1 -- 172.21.15.152:0/1559043467 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8a400afac0 msgr2=0x7f8a400c4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:40.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:40.041+0000 7f8a3f200640 1 --2- 172.21.15.152:0/1559043467 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8a400afac0 0x7f8a400c4e60 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f8a3800cf20 tx=0x7f8a3800a7d0 comp rx=0 tx=0).stop 2024-04-20T11:27:40.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:40.041+0000 7f8a3f200640 1 -- 172.21.15.152:0/1559043467 shutdown_connections 2024-04-20T11:27:40.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:40.041+0000 7f8a3f200640 1 --2- 172.21.15.152:0/1559043467 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f8a2006c8f0 0x7f8a2006edb0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:40.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:40.041+0000 7f8a3f200640 1 --2- 172.21.15.152:0/1559043467 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8a400afac0 0x7f8a400c4e60 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:40.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:40.041+0000 7f8a3f200640 1 --2- 172.21.15.152:0/1559043467 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8a400b3560 0x7f8a40152920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:40.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:40.041+0000 7f8a3f200640 1 --2- 172.21.15.152:0/1559043467 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8a400b2a40 0x7f8a400b0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:40.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:40.041+0000 7f8a3f200640 1 -- 172.21.15.152:0/1559043467 >> 172.21.15.152:0/1559043467 conn(0x7f8a400b3e70 msgr2=0x7f8a4015c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:40.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:40.041+0000 7f8a3f200640 1 -- 172.21.15.152:0/1559043467 shutdown_connections 2024-04-20T11:27:40.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:40.041+0000 7f8a3f200640 1 -- 172.21.15.152:0/1559043467 wait complete. 2024-04-20T11:27:40.059 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade' 2024-04-20T11:27:40.252 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:40.251+0000 7f9132c00640 1 Processor -- start 2024-04-20T11:27:40.252 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:40.251+0000 7f9132c00640 1 -- start start 2024-04-20T11:27:40.254 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:40.253+0000 7f9132c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f912c13e530 0x7f912c06ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:40.255 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:40.253+0000 7f9132c00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f912c06f340 0x7f912c140dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:40.255 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:40.253+0000 7f9132c00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f912c141310 0x7f912c15efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:40.255 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:40.253+0000 7f9132c00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f912c141b00 con 0x7f912c141310 2024-04-20T11:27:40.255 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:40.253+0000 7f9132c00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f912c0cfc50 con 0x7f912c06f340 2024-04-20T11:27:40.255 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:40.253+0000 7f9132c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f912c0cf0a0 con 0x7f912c13e530 2024-04-20T11:27:40.255 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:40.253+0000 7f9132200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f912c141310 0x7f912c15efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:40.255 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:40.253+0000 7f9132200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f912c141310 0x7f912c15efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35344/0 (socket says 172.21.15.152:35344) 2024-04-20T11:27:40.255 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:40.253+0000 7f9132200640 1 -- 172.21.15.152:0/3052258440 learned_addr learned my addr 172.21.15.152:0/3052258440 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:40.255 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:40.254+0000 7f9131800640 1 --2- 172.21.15.152:0/3052258440 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f912c13e530 0x7f912c06ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:40.255 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:40.254+0000 7f9130e00640 1 --2- 172.21.15.152:0/3052258440 >> v2:172.21.15.175:3300/0 conn(0x7f912c06f340 0x7f912c140dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:40.255 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:40.254+0000 7f9132200640 1 -- 172.21.15.152:0/3052258440 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f912c13e530 msgr2=0x7f912c06ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:40.255 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:40.254+0000 7f9132200640 1 --2- 172.21.15.152:0/3052258440 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f912c13e530 0x7f912c06ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:40.255 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:40.254+0000 7f9132200640 1 -- 172.21.15.152:0/3052258440 >> v2:172.21.15.175:3300/0 conn(0x7f912c06f340 msgr2=0x7f912c140dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:40.256 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:40.254+0000 7f9132200640 1 --2- 172.21.15.152:0/3052258440 >> v2:172.21.15.175:3300/0 conn(0x7f912c06f340 0x7f912c140dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:40.256 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:40.254+0000 7f9132200640 1 -- 172.21.15.152:0/3052258440 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f912c0d00e0 con 0x7f912c141310 2024-04-20T11:27:40.256 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:40.254+0000 7f9130e00640 1 --2- 172.21.15.152:0/3052258440 >> v2:172.21.15.175:3300/0 conn(0x7f912c06f340 0x7f912c140dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:27:40.257 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:40.255+0000 7f9132200640 1 --2- 172.21.15.152:0/3052258440 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f912c141310 0x7f912c15efd0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f912c06c9d0 tx=0x7f912402e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a68cae5422080c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:40.257 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:40.256+0000 7f912be00640 1 -- 172.21.15.152:0/3052258440 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f912402ea80 con 0x7f912c141310 2024-04-20T11:27:40.257 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:40.256+0000 7f912be00640 1 -- 172.21.15.152:0/3052258440 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f912402ec20 con 0x7f912c141310 2024-04-20T11:27:40.257 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:40.256+0000 7f912be00640 1 -- 172.21.15.152:0/3052258440 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9124009600 con 0x7f912c141310 2024-04-20T11:27:40.257 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:40.256+0000 7f9132c00640 1 -- 172.21.15.152:0/3052258440 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f912c141310 msgr2=0x7f912c15efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:40.258 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:40.256+0000 7f9132c00640 1 --2- 172.21.15.152:0/3052258440 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f912c141310 0x7f912c15efd0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f912c06c9d0 tx=0x7f912402e0e0 comp rx=0 tx=0).stop 2024-04-20T11:27:40.258 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:40.256+0000 7f9132c00640 1 -- 172.21.15.152:0/3052258440 shutdown_connections 2024-04-20T11:27:40.258 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:40.256+0000 7f9132c00640 1 --2- 172.21.15.152:0/3052258440 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f912c141310 0x7f912c15efd0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:40.258 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:40.256+0000 7f9132c00640 1 --2- 172.21.15.152:0/3052258440 >> v2:172.21.15.175:3300/0 conn(0x7f912c06f340 0x7f912c140dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:40.258 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:40.256+0000 7f9132c00640 1 --2- 172.21.15.152:0/3052258440 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f912c13e530 0x7f912c06ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:40.258 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:40.256+0000 7f9132c00640 1 -- 172.21.15.152:0/3052258440 >> 172.21.15.152:0/3052258440 conn(0x7f912c067ae0 msgr2=0x7f912c06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:40.258 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:40.257+0000 7f9132c00640 1 -- 172.21.15.152:0/3052258440 shutdown_connections 2024-04-20T11:27:40.258 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:40.257+0000 7f9132c00640 1 -- 172.21.15.152:0/3052258440 wait complete. 2024-04-20T11:27:40.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:40.258+0000 7f9132c00640 1 Processor -- start 2024-04-20T11:27:40.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:40.259+0000 7f9132c00640 1 -- start start 2024-04-20T11:27:40.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:40.259+0000 7f9132c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f912c06f340 0x7f912c13d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:40.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:40.259+0000 7f9132c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f912c13e530 0x7f912c148e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:40.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:40.259+0000 7f9132c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f912c141310 0x7f912c149340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:40.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:40.260+0000 7f9132c00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f912c0cf8e0 con 0x7f912c141310 2024-04-20T11:27:40.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:40.260+0000 7f9132c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f912c0cf600 con 0x7f912c13e530 2024-04-20T11:27:40.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:40.260+0000 7f9132c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f912c1417a0 con 0x7f912c06f340 2024-04-20T11:27:40.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:40.260+0000 7f9132200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f912c141310 0x7f912c149340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:40.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:40.260+0000 7f9132200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f912c141310 0x7f912c149340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35348/0 (socket says 172.21.15.152:35348) 2024-04-20T11:27:40.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:40.260+0000 7f9132200640 1 -- 172.21.15.152:0/4056648891 learned_addr learned my addr 172.21.15.152:0/4056648891 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:40.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:40.260+0000 7f9131800640 1 --2- 172.21.15.152:0/4056648891 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f912c06f340 0x7f912c13d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:40.262 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:40.260+0000 7f9132200640 1 -- 172.21.15.152:0/4056648891 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f912c06f340 msgr2=0x7f912c13d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:40.262 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:40.260+0000 7f9132200640 1 --2- 172.21.15.152:0/4056648891 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f912c06f340 0x7f912c13d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:40.262 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:40.260+0000 7f9132200640 1 -- 172.21.15.152:0/4056648891 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f912c13e530 msgr2=0x7f912c148e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:40.262 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:40.260+0000 7f9132200640 1 --2- 172.21.15.152:0/4056648891 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f912c13e530 0x7f912c148e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:40.262 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:40.260+0000 7f9132200640 1 -- 172.21.15.152:0/4056648891 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f912c0b0d60 con 0x7f912c141310 2024-04-20T11:27:40.262 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:40.260+0000 7f9131800640 1 --2- 172.21.15.152:0/4056648891 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f912c06f340 0x7f912c13d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:27:40.262 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:40.261+0000 7f9132200640 1 --2- 172.21.15.152:0/4056648891 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f912c141310 0x7f912c149340 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f912402e6f0 tx=0x7f9124004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:40.263 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:40.261+0000 7f9129600640 1 -- 172.21.15.152:0/4056648891 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9124044020 con 0x7f912c141310 2024-04-20T11:27:40.263 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:40.261+0000 7f9129600640 1 -- 172.21.15.152:0/4056648891 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f912403f850 con 0x7f912c141310 2024-04-20T11:27:40.263 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:40.261+0000 7f9129600640 1 -- 172.21.15.152:0/4056648891 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f912402ea80 con 0x7f912c141310 2024-04-20T11:27:40.263 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:40.261+0000 7f9132c00640 1 -- 172.21.15.152:0/4056648891 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f912c0b1040 con 0x7f912c141310 2024-04-20T11:27:40.263 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:40.261+0000 7f9132c00640 1 -- 172.21.15.152:0/4056648891 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f912c0b1600 con 0x7f912c141310 2024-04-20T11:27:40.263 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:40.262+0000 7f9132c00640 1 -- 172.21.15.152:0/4056648891 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f912c075e60 con 0x7f912c141310 2024-04-20T11:27:40.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:40.263+0000 7f9129600640 1 -- 172.21.15.152:0/4056648891 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9124047020 con 0x7f912c141310 2024-04-20T11:27:40.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:40.264+0000 7f9129600640 1 --2- 172.21.15.152:0/4056648891 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f910c06c8f0 0x7f910c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:40.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:40.264+0000 7f9129600640 1 -- 172.21.15.152:0/4056648891 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5825+0+0 (secure 0 0 0) 0x7f91240aebe0 con 0x7f912c141310 2024-04-20T11:27:40.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:40.264+0000 7f9131800640 1 --2- 172.21.15.152:0/4056648891 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f910c06c8f0 0x7f910c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:40.265 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:40.265+0000 7f9131800640 1 --2- 172.21.15.152:0/4056648891 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f910c06c8f0 0x7f910c06edb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9120006fd0 tx=0x7f9120008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:40.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:40.268+0000 7f9129600640 1 -- 172.21.15.152:0/4056648891 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f912404e030 con 0x7f912c141310 2024-04-20T11:27:40.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:40.546+0000 7f9132c00640 1 -- 172.21.15.152:0/4056648891 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade"]} v 0) -- 0x7f912c0b3490 con 0x7f912c141310 2024-04-20T11:27:41.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:41.153+0000 7f9129600640 1 -- 172.21.15.152:0/4056648891 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade"]}]=0 v59) ==== 161+0+0 (secure 0 0 0) 0x7f912407cfc0 con 0x7f912c141310 2024-04-20T11:27:41.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:41.157+0000 7f9132c00640 1 -- 172.21.15.152:0/4056648891 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f910c06c8f0 msgr2=0x7f910c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:41.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:41.157+0000 7f9132c00640 1 --2- 172.21.15.152:0/4056648891 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f910c06c8f0 0x7f910c06edb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9120006fd0 tx=0x7f9120008040 comp rx=0 tx=0).stop 2024-04-20T11:27:41.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:41.157+0000 7f9132c00640 1 -- 172.21.15.152:0/4056648891 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f912c141310 msgr2=0x7f912c149340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:41.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:41.157+0000 7f9132c00640 1 --2- 172.21.15.152:0/4056648891 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f912c141310 0x7f912c149340 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f912402e6f0 tx=0x7f9124004a40 comp rx=0 tx=0).stop 2024-04-20T11:27:41.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:41.157+0000 7f9132c00640 1 -- 172.21.15.152:0/4056648891 shutdown_connections 2024-04-20T11:27:41.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:41.157+0000 7f9132c00640 1 --2- 172.21.15.152:0/4056648891 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f910c06c8f0 0x7f910c06edb0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:41.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:41.157+0000 7f9132c00640 1 --2- 172.21.15.152:0/4056648891 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f912c141310 0x7f912c149340 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:41.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:41.157+0000 7f9132c00640 1 --2- 172.21.15.152:0/4056648891 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f912c13e530 0x7f912c148e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:41.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:41.157+0000 7f9132c00640 1 --2- 172.21.15.152:0/4056648891 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f912c06f340 0x7f912c13d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:41.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:41.157+0000 7f9132c00640 1 -- 172.21.15.152:0/4056648891 >> 172.21.15.152:0/4056648891 conn(0x7f912c067ae0 msgr2=0x7f912c0af000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:41.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:41.157+0000 7f9132c00640 1 -- 172.21.15.152:0/4056648891 shutdown_connections 2024-04-20T11:27:41.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:41.158+0000 7f9132c00640 1 -- 172.21.15.152:0/4056648891 wait complete. 2024-04-20T11:27:41.185 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-20T11:27:41.378 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:41.377+0000 7fe3f7000640 1 Processor -- start 2024-04-20T11:27:41.378 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:41.377+0000 7fe3f7000640 1 -- start start 2024-04-20T11:27:41.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:41.379+0000 7fe3f7000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe3f006ee00 0x7fe3f006f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:41.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:41.379+0000 7fe3f7000640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3f00d00d0 con 0x7fe3f00cf110 2024-04-20T11:27:41.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:41.379+0000 7fe3f7000640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3f006f720 con 0x7fe3f015fbf0 2024-04-20T11:27:41.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:41.379+0000 7fe3f7000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe3f00cfb10 con 0x7fe3f006ee00 2024-04-20T11:27:41.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:41.379+0000 7fe3f6600640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fe3f00cf110 0x7fe3f00b3ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:39976/0 (socket says 172.21.15.152:39976) 2024-04-20T11:27:41.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:41.379+0000 7fe3f6600640 1 -- 172.21.15.152:0/3206309281 learned_addr learned my addr 172.21.15.152:0/3206309281 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:27:41.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:41.380+0000 7fe3f5c00640 1 --2- 172.21.15.152:0/3206309281 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe3f006ee00 0x7fe3f006f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:41.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:41.380+0000 7fe3efe00640 1 -- 172.21.15.152:0/3206309281 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3451824747 0 0) 0x7fe3f00d00d0 con 0x7fe3f00cf110 2024-04-20T11:27:41.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:41.380+0000 7fe3efe00640 1 -- 172.21.15.152:0/3206309281 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe3d8003520 con 0x7fe3f00cf110 2024-04-20T11:27:41.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:41.380+0000 7fe3efe00640 1 -- 172.21.15.152:0/3206309281 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 22186908 0 0) 0x7fe3d8003520 con 0x7fe3f00cf110 2024-04-20T11:27:41.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:41.380+0000 7fe3efe00640 1 -- 172.21.15.152:0/3206309281 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe3f006ee00 msgr2=0x7fe3f006f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:41.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:41.380+0000 7fe3efe00640 1 --2- 172.21.15.152:0/3206309281 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe3f006ee00 0x7fe3f006f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:41.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:41.380+0000 7fe3efe00640 1 -- 172.21.15.152:0/3206309281 >> v1:172.21.15.175:6789/0 conn(0x7fe3f015fbf0 legacy=0x7fe3f015c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:41.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:41.380+0000 7fe3efe00640 1 -- 172.21.15.152:0/3206309281 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3f00cfdf0 con 0x7fe3f00cf110 2024-04-20T11:27:41.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:41.380+0000 7fe3efe00640 1 -- 172.21.15.152:0/3206309281 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7fe3e80019f0 con 0x7fe3f00cf110 2024-04-20T11:27:41.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:41.380+0000 7fe3efe00640 1 -- 172.21.15.152:0/3206309281 >> v1:172.21.15.152:6789/0 conn(0x7fe3f00cf110 legacy=0x7fe3f00b3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:41.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:41.380+0000 7fe3efe00640 1 --2- 172.21.15.152:0/3206309281 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe3d80039e0 0x7fe3d8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:41.383 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:41.380+0000 7fe3efe00640 1 --2- 172.21.15.152:0/3206309281 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe3d8004800 0x7fe3d8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:41.383 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:41.380+0000 7fe3efe00640 1 --2- 172.21.15.152:0/3206309281 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe3f006ee00 0x7fe3d80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:41.383 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:41.380+0000 7fe3efe00640 1 -- 172.21.15.152:0/3206309281 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fe3f00cfb10 con 0x7fe3d8004800 2024-04-20T11:27:41.383 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:41.380+0000 7fe3efe00640 1 -- 172.21.15.152:0/3206309281 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d800ba70 con 0x7fe3d80039e0 2024-04-20T11:27:41.383 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:41.380+0000 7fe3efe00640 1 -- 172.21.15.152:0/3206309281 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe3d800bc70 con 0x7fe3f006ee00 2024-04-20T11:27:41.383 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:41.381+0000 7fe3f5200640 1 --2- 172.21.15.152:0/3206309281 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe3d8004800 0x7fe3d8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:41.383 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:41.381+0000 7fe3f5c00640 1 --2- 172.21.15.152:0/3206309281 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe3d80039e0 0x7fe3d8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:41.383 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:41.381+0000 7fe3f5200640 1 -- 172.21.15.152:0/3206309281 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe3f006ee00 msgr2=0x7fe3d80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:41.383 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:41.381+0000 7fe3f5200640 1 --2- 172.21.15.152:0/3206309281 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe3f006ee00 0x7fe3d80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:41.383 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:41.381+0000 7fe3f5200640 1 -- 172.21.15.152:0/3206309281 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe3d80039e0 msgr2=0x7fe3d8003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:41.383 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:41.381+0000 7fe3f5200640 1 --2- 172.21.15.152:0/3206309281 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe3d80039e0 0x7fe3d8003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:41.383 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:41.381+0000 7fe3f5200640 1 -- 172.21.15.152:0/3206309281 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3d8003520 con 0x7fe3d8004800 2024-04-20T11:27:41.383 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:41.381+0000 7fe3f5c00640 1 --2- 172.21.15.152:0/3206309281 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe3d80039e0 0x7fe3d8003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:27:41.384 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:41.382+0000 7fe3f5200640 1 --2- 172.21.15.152:0/3206309281 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe3d8004800 0x7fe3d8004c80 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc001720 tx=0x7fe3d8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34b10d38d4f2b6d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:41.384 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:41.382+0000 7fe3efe00640 1 -- 172.21.15.152:0/3206309281 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3dc001370 con 0x7fe3d8004800 2024-04-20T11:27:41.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:41.382+0000 7fe3efe00640 1 -- 172.21.15.152:0/3206309281 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3dc0052d0 con 0x7fe3d8004800 2024-04-20T11:27:41.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:41.382+0000 7fe3efe00640 1 -- 172.21.15.152:0/3206309281 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3dc003830 con 0x7fe3d8004800 2024-04-20T11:27:41.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:41.383+0000 7fe3f7000640 1 -- 172.21.15.152:0/3206309281 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe3d8004800 msgr2=0x7fe3d8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:41.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:41.383+0000 7fe3f7000640 1 --2- 172.21.15.152:0/3206309281 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe3d8004800 0x7fe3d8004c80 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc001720 tx=0x7fe3d8005830 comp rx=0 tx=0).stop 2024-04-20T11:27:41.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:41.383+0000 7fe3f6600640 1 -- 172.21.15.152:0/3206309281 reap_dead start 2024-04-20T11:27:41.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:41.383+0000 7fe3f7000640 1 -- 172.21.15.152:0/3206309281 shutdown_connections 2024-04-20T11:27:41.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:41.383+0000 7fe3f7000640 1 -- 172.21.15.152:0/3206309281 >> 172.21.15.152:0/3206309281 conn(0x7fe3f0067ae0 msgr2=0x7fe3f006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:41.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:41.383+0000 7fe3f7000640 1 -- 172.21.15.152:0/3206309281 shutdown_connections 2024-04-20T11:27:41.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:41.383+0000 7fe3f7000640 1 -- 172.21.15.152:0/3206309281 wait complete. 2024-04-20T11:27:41.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:41.385+0000 7fe3f7000640 1 Processor -- start 2024-04-20T11:27:41.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:41.385+0000 7fe3f7000640 1 -- start start 2024-04-20T11:27:41.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:41.386+0000 7fe3f7000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe3d8004800 0x7fe3f00aef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:41.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:41.386+0000 7fe3f7000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe3f01e7820 0x7fe3f0141930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:41.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:41.386+0000 7fe3f7000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe3f00ac4f0 0x7fe3f00ac970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:41.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:41.386+0000 7fe3f7000640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fe3f0144970 con 0x7fe3f01e7820 2024-04-20T11:27:41.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:41.386+0000 7fe3f7000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe3f015be70 con 0x7fe3f00ac4f0 2024-04-20T11:27:41.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:41.386+0000 7fe3f7000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe3f0144af0 con 0x7fe3d8004800 2024-04-20T11:27:41.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:41.386+0000 7fe3f5200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe3f01e7820 0x7fe3f0141930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:41.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:41.386+0000 7fe3f5200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe3f01e7820 0x7fe3f0141930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35360/0 (socket says 172.21.15.152:35360) 2024-04-20T11:27:41.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:41.386+0000 7fe3f5200640 1 -- 172.21.15.152:0/2009744123 learned_addr learned my addr 172.21.15.152:0/2009744123 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:41.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:41.387+0000 7fe3f5c00640 1 --2- 172.21.15.152:0/2009744123 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe3d8004800 0x7fe3f00aef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:41.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:41.387+0000 7fe3f6600640 1 --2- 172.21.15.152:0/2009744123 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe3f00ac4f0 0x7fe3f00ac970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:41.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:41.387+0000 7fe3f5200640 1 -- 172.21.15.152:0/2009744123 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe3d8004800 msgr2=0x7fe3f00aef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:41.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:41.387+0000 7fe3f5200640 1 --2- 172.21.15.152:0/2009744123 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe3d8004800 0x7fe3f00aef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:41.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:41.387+0000 7fe3f5200640 1 -- 172.21.15.152:0/2009744123 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe3f00ac4f0 msgr2=0x7fe3f00ac970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:41.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:41.387+0000 7fe3f5200640 1 --2- 172.21.15.152:0/2009744123 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe3f00ac4f0 0x7fe3f00ac970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:41.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:41.387+0000 7fe3f5200640 1 -- 172.21.15.152:0/2009744123 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3f0141e70 con 0x7fe3f01e7820 2024-04-20T11:27:41.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:41.387+0000 7fe3f5c00640 1 --2- 172.21.15.152:0/2009744123 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe3d8004800 0x7fe3f00aef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:27:41.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:41.387+0000 7fe3f5200640 1 --2- 172.21.15.152:0/2009744123 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe3f01e7820 0x7fe3f0141930 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc008fd0 tx=0x7fe3dc001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:41.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:41.388+0000 7fe3ed600640 1 -- 172.21.15.152:0/2009744123 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3dc041070 con 0x7fe3f01e7820 2024-04-20T11:27:41.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:41.388+0000 7fe3ed600640 1 -- 172.21.15.152:0/2009744123 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3dc005480 con 0x7fe3f01e7820 2024-04-20T11:27:41.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:41.388+0000 7fe3f7000640 1 -- 172.21.15.152:0/2009744123 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3f00aceb0 con 0x7fe3f01e7820 2024-04-20T11:27:41.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:41.388+0000 7fe3ed600640 1 -- 172.21.15.152:0/2009744123 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3dc007a30 con 0x7fe3f01e7820 2024-04-20T11:27:41.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:41.388+0000 7fe3f7000640 1 -- 172.21.15.152:0/2009744123 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3f01e7f70 con 0x7fe3f01e7820 2024-04-20T11:27:41.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:41.389+0000 7fe3f7000640 1 -- 172.21.15.152:0/2009744123 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3f0075e60 con 0x7fe3f01e7820 2024-04-20T11:27:41.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:41.390+0000 7fe3ed600640 1 -- 172.21.15.152:0/2009744123 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe3dc044070 con 0x7fe3f01e7820 2024-04-20T11:27:41.391 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:41.391+0000 7fe3ed600640 1 --2- 172.21.15.152:0/2009744123 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe3d006c920 0x7fe3d006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:41.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:41.391+0000 7fe3ed600640 1 -- 172.21.15.152:0/2009744123 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fe3dc0ad730 con 0x7fe3f01e7820 2024-04-20T11:27:41.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:41.392+0000 7fe3f5c00640 1 --2- 172.21.15.152:0/2009744123 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe3d006c920 0x7fe3d006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:41.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:41.392+0000 7fe3f5c00640 1 --2- 172.21.15.152:0/2009744123 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe3d006c920 0x7fe3d006ede0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe3e4009b40 tx=0x7fe3e4008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:41.396 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:41.396+0000 7fe3ed600640 1 -- 172.21.15.152:0/2009744123 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe3dc07bae0 con 0x7fe3f01e7820 2024-04-20T11:27:41.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:41.596+0000 7fe3f7000640 1 -- 172.21.15.152:0/2009744123 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fe3f00b3980 con 0x7fe3f01e7820 2024-04-20T11:27:41.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:41.603+0000 7fe3ed600640 1 -- 172.21.15.152:0/2009744123 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7fe3dc07b3e0 con 0x7fe3f01e7820 2024-04-20T11:27:41.607 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:41.606+0000 7fe3f7000640 1 -- 172.21.15.152:0/2009744123 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe3d006c920 msgr2=0x7fe3d006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:41.607 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:41.606+0000 7fe3f7000640 1 --2- 172.21.15.152:0/2009744123 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe3d006c920 0x7fe3d006ede0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe3e4009b40 tx=0x7fe3e4008040 comp rx=0 tx=0).stop 2024-04-20T11:27:41.607 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:41.606+0000 7fe3f7000640 1 -- 172.21.15.152:0/2009744123 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe3f01e7820 msgr2=0x7fe3f0141930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:41.607 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:41.606+0000 7fe3f7000640 1 --2- 172.21.15.152:0/2009744123 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe3f01e7820 0x7fe3f0141930 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc008fd0 tx=0x7fe3dc001120 comp rx=0 tx=0).stop 2024-04-20T11:27:41.607 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:41.606+0000 7fe3f7000640 1 -- 172.21.15.152:0/2009744123 shutdown_connections 2024-04-20T11:27:41.607 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:41.606+0000 7fe3f7000640 1 --2- 172.21.15.152:0/2009744123 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe3d006c920 0x7fe3d006ede0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:41.607 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:41.606+0000 7fe3f7000640 1 --2- 172.21.15.152:0/2009744123 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe3f00ac4f0 0x7fe3f00ac970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:41.608 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:41.606+0000 7fe3f7000640 1 --2- 172.21.15.152:0/2009744123 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe3f01e7820 0x7fe3f0141930 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:41.608 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:41.606+0000 7fe3f7000640 1 --2- 172.21.15.152:0/2009744123 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe3d8004800 0x7fe3f00aef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:41.608 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:41.606+0000 7fe3f7000640 1 -- 172.21.15.152:0/2009744123 >> 172.21.15.152:0/2009744123 conn(0x7fe3f0067ae0 msgr2=0x7fe3f00cd590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:41.608 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:41.607+0000 7fe3f7000640 1 -- 172.21.15.152:0/2009744123 shutdown_connections 2024-04-20T11:27:41.608 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:41.607+0000 7fe3f7000640 1 -- 172.21.15.152:0/2009744123 wait complete. 2024-04-20T11:27:41.626 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-20T11:27:41.627 DEBUG:teuthology.orchestra.run.smithi152:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-20T11:27:41.731 DEBUG:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-20T11:27:41.846 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-20T11:27:41.847 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-20T11:27:41.847 DEBUG:teuthology.orchestra.run.smithi152:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:27:41.903 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-20T11:27:41.903 DEBUG:teuthology.orchestra.run.smithi152:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:27:41.955 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:27:42.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.150+0000 7f0a21e00640 1 Processor -- start 2024-04-20T11:27:42.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.150+0000 7f0a21e00640 1 -- start start 2024-04-20T11:27:42.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.152+0000 7f0a21e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f0a1c15c560 0x7f0a1c0b3df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:42.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.152+0000 7f0a21e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0a1c0b4330 0x7f0a1c0d0100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:42.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.152+0000 7f0a21e00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a1c15fd20 con 0x7f0a1c0d0640 2024-04-20T11:27:42.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.152+0000 7f0a21e00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f0a1c15ef60 con 0x7f0a1c15c560 2024-04-20T11:27:42.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.152+0000 7f0a21e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0a1c15f240 con 0x7f0a1c0b4330 2024-04-20T11:27:42.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.152+0000 7f0a1aa00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f0a1c15c560 0x7f0a1c0b3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:42.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.152+0000 7f0a1aa00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f0a1c15c560 0x7f0a1c0b3df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35372/0 (socket says 172.21.15.152:35372) 2024-04-20T11:27:42.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.152+0000 7f0a1aa00640 1 -- 172.21.15.152:0/1138010216 learned_addr learned my addr 172.21.15.152:0/1138010216 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:42.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.153+0000 7f0a1a000640 1 --2- 172.21.15.152:0/1138010216 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0a1c0b4330 0x7f0a1c0d0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:42.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.153+0000 7f0a1aa00640 1 -- 172.21.15.152:0/1138010216 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0a1c0b4330 msgr2=0x7f0a1c0d0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:42.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.153+0000 7f0a1aa00640 1 --2- 172.21.15.152:0/1138010216 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0a1c0b4330 0x7f0a1c0d0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:42.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.153+0000 7f0a1aa00640 1 -- 172.21.15.152:0/1138010216 >> v1:172.21.15.152:6789/0 conn(0x7f0a1c0d0640 legacy=0x7f0a1c071db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:42.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.153+0000 7f0a1aa00640 1 -- 172.21.15.152:0/1138010216 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a1c15f520 con 0x7f0a1c15c560 2024-04-20T11:27:42.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.155+0000 7f0a1aa00640 1 --2- 172.21.15.152:0/1138010216 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0a1c15c560 0x7f0a1c0b3df0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c009870 tx=0x7f0a0c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a90fa083ddaaacc3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:42.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.158+0000 7f0a19600640 1 -- 172.21.15.152:0/1138010216 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a0c035020 con 0x7f0a1c15c560 2024-04-20T11:27:42.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.158+0000 7f0a19600640 1 -- 172.21.15.152:0/1138010216 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a0c031640 con 0x7f0a1c15c560 2024-04-20T11:27:42.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.158+0000 7f0a19600640 1 -- 172.21.15.152:0/1138010216 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a0c02d040 con 0x7f0a1c15c560 2024-04-20T11:27:42.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.159+0000 7f0a21e00640 1 -- 172.21.15.152:0/1138010216 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0a1c15c560 msgr2=0x7f0a1c0b3df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:42.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.159+0000 7f0a21e00640 1 --2- 172.21.15.152:0/1138010216 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0a1c15c560 0x7f0a1c0b3df0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c009870 tx=0x7f0a0c02b8f0 comp rx=0 tx=0).stop 2024-04-20T11:27:42.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.159+0000 7f0a21e00640 1 -- 172.21.15.152:0/1138010216 shutdown_connections 2024-04-20T11:27:42.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.159+0000 7f0a21e00640 1 --2- 172.21.15.152:0/1138010216 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0a1c0b4330 0x7f0a1c0d0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:42.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.159+0000 7f0a21e00640 1 --2- 172.21.15.152:0/1138010216 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0a1c15c560 0x7f0a1c0b3df0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:42.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.159+0000 7f0a21e00640 1 -- 172.21.15.152:0/1138010216 >> 172.21.15.152:0/1138010216 conn(0x7f0a1c067cf0 msgr2=0x7f0a1c06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:42.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.159+0000 7f0a21e00640 1 -- 172.21.15.152:0/1138010216 shutdown_connections 2024-04-20T11:27:42.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.159+0000 7f0a21e00640 1 -- 172.21.15.152:0/1138010216 wait complete. 2024-04-20T11:27:42.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.161+0000 7f0a21e00640 1 Processor -- start 2024-04-20T11:27:42.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.161+0000 7f0a21e00640 1 -- start start 2024-04-20T11:27:42.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.162+0000 7f0a21e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0a1c0b4330 0x7f0a1c152a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:42.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.162+0000 7f0a21e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0a1c0d0640 0x7f0a1c1f8c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:42.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.162+0000 7f0a21e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0a1c15c560 0x7f0a1c15d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:42.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.162+0000 7f0a21e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f0a1c0d0ab0 con 0x7f0a1c0b4330 2024-04-20T11:27:42.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.162+0000 7f0a21e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0a1c0d0dc0 con 0x7f0a1c0d0640 2024-04-20T11:27:42.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.162+0000 7f0a21e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0a1c0aeaa0 con 0x7f0a1c15c560 2024-04-20T11:27:42.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.162+0000 7f0a1aa00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0a1c0b4330 0x7f0a1c152a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:42.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.162+0000 7f0a1aa00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0a1c0b4330 0x7f0a1c152a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35380/0 (socket says 172.21.15.152:35380) 2024-04-20T11:27:42.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.162+0000 7f0a1aa00640 1 -- 172.21.15.152:0/1846001655 learned_addr learned my addr 172.21.15.152:0/1846001655 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:42.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.162+0000 7f0a1aa00640 1 -- 172.21.15.152:0/1846001655 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0a1c15c560 msgr2=0x7f0a1c15d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:42.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.162+0000 7f0a1aa00640 1 --2- 172.21.15.152:0/1846001655 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0a1c15c560 0x7f0a1c15d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:42.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.162+0000 7f0a1aa00640 1 -- 172.21.15.152:0/1846001655 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0a1c0d0640 msgr2=0x7f0a1c1f8c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:42.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.162+0000 7f0a1aa00640 1 --2- 172.21.15.152:0/1846001655 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0a1c0d0640 0x7f0a1c1f8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:42.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.162+0000 7f0a1aa00640 1 -- 172.21.15.152:0/1846001655 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a1c15d940 con 0x7f0a1c0b4330 2024-04-20T11:27:42.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.163+0000 7f0a1aa00640 1 --2- 172.21.15.152:0/1846001655 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0a1c0b4330 0x7f0a1c152a20 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c02b930 tx=0x7f0a0c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:42.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.163+0000 7f0a13400640 1 -- 172.21.15.152:0/1846001655 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a0c040070 con 0x7f0a1c0b4330 2024-04-20T11:27:42.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.163+0000 7f0a13400640 1 -- 172.21.15.152:0/1846001655 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a0c03e070 con 0x7f0a1c0b4330 2024-04-20T11:27:42.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.163+0000 7f0a13400640 1 -- 172.21.15.152:0/1846001655 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a0c035040 con 0x7f0a1c0b4330 2024-04-20T11:27:42.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.163+0000 7f0a21e00640 1 -- 172.21.15.152:0/1846001655 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a1c15dbb0 con 0x7f0a1c0b4330 2024-04-20T11:27:42.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.163+0000 7f0a21e00640 1 -- 172.21.15.152:0/1846001655 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a1c160440 con 0x7f0a1c0b4330 2024-04-20T11:27:42.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.165+0000 7f0a21e00640 1 -- 172.21.15.152:0/1846001655 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a1c15a510 con 0x7f0a1c0b4330 2024-04-20T11:27:42.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.165+0000 7f0a13400640 1 -- 172.21.15.152:0/1846001655 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0a0c038020 con 0x7f0a1c0b4330 2024-04-20T11:27:42.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.166+0000 7f0a13400640 1 --2- 172.21.15.152:0/1846001655 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f09fc06c8f0 0x7f09fc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:42.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.167+0000 7f0a13400640 1 -- 172.21.15.152:0/1846001655 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f0a0c0ad600 con 0x7f0a1c0b4330 2024-04-20T11:27:42.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.167+0000 7f0a1a000640 1 --2- 172.21.15.152:0/1846001655 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f09fc06c8f0 0x7f09fc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:42.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.167+0000 7f0a1a000640 1 --2- 172.21.15.152:0/1846001655 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f09fc06c8f0 0x7f09fc06edb0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0a080070b0 tx=0x7f0a08007040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:42.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.171+0000 7f0a13400640 1 -- 172.21.15.152:0/1846001655 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f0a1c15a510 con 0x7f0a1c0b4330 2024-04-20T11:27:42.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.375+0000 7f0a21e00640 1 -- 172.21.15.152:0/1846001655 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0a1c0689f0 con 0x7f0a1c0b4330 2024-04-20T11:27:42.378 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.378+0000 7f0a13400640 1 -- 172.21.15.152:0/1846001655 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16914 (secure 0 0 0) 0x7f0a0c07b990 con 0x7f0a1c0b4330 2024-04-20T11:27:42.379 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:27:42.379 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":22,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:27:41.163136+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:26:56.261424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T11:26:57.079286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":2}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.26","mappings":[{"from":7,"to":1}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:27:42.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.381+0000 7f0a21e00640 1 -- 172.21.15.152:0/1846001655 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f09fc06c8f0 msgr2=0x7f09fc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:42.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.381+0000 7f0a21e00640 1 --2- 172.21.15.152:0/1846001655 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f09fc06c8f0 0x7f09fc06edb0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0a080070b0 tx=0x7f0a08007040 comp rx=0 tx=0).stop 2024-04-20T11:27:42.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.381+0000 7f0a21e00640 1 -- 172.21.15.152:0/1846001655 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0a1c0b4330 msgr2=0x7f0a1c152a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:42.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.381+0000 7f0a21e00640 1 --2- 172.21.15.152:0/1846001655 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0a1c0b4330 0x7f0a1c152a20 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c02b930 tx=0x7f0a0c0046c0 comp rx=0 tx=0).stop 2024-04-20T11:27:42.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.381+0000 7f0a21e00640 1 -- 172.21.15.152:0/1846001655 shutdown_connections 2024-04-20T11:27:42.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.381+0000 7f0a21e00640 1 --2- 172.21.15.152:0/1846001655 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f09fc06c8f0 0x7f09fc06edb0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:42.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.381+0000 7f0a21e00640 1 --2- 172.21.15.152:0/1846001655 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0a1c15c560 0x7f0a1c15d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:42.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.381+0000 7f0a21e00640 1 --2- 172.21.15.152:0/1846001655 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0a1c0d0640 0x7f0a1c1f8c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:42.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.381+0000 7f0a21e00640 1 --2- 172.21.15.152:0/1846001655 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0a1c0b4330 0x7f0a1c152a20 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:42.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.381+0000 7f0a21e00640 1 -- 172.21.15.152:0/1846001655 >> 172.21.15.152:0/1846001655 conn(0x7f0a1c067cf0 msgr2=0x7f0a1c1599c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:42.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.381+0000 7f0a21e00640 1 -- 172.21.15.152:0/1846001655 shutdown_connections 2024-04-20T11:27:42.383 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.381+0000 7f0a21e00640 1 -- 172.21.15.152:0/1846001655 wait complete. 2024-04-20T11:27:42.402 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T11:26:47.246571+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T11:26:56.261424+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T11:26:57.079286+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T11:27:42.402 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T11:27:42.608 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.607+0000 7f6e84400640 1 Processor -- start 2024-04-20T11:27:42.608 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.608+0000 7f6e84400640 1 -- start start 2024-04-20T11:27:42.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.609+0000 7f6e84400640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f6e7c15e9e0 0x7f6e7c06d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:42.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.609+0000 7f6e84400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f6e7c0d46a0 0x7f6e7c0d4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:42.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.609+0000 7f6e84400640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e7c0d19a0 con 0x7f6e7c0d0d90 2024-04-20T11:27:42.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.609+0000 7f6e84400640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f6e7c0d5040 con 0x7f6e7c0d46a0 2024-04-20T11:27:42.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.609+0000 7f6e84400640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f6e7c0d1c80 con 0x7f6e7c15e9e0 2024-04-20T11:27:42.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.610+0000 7f6e81000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f6e7c0d46a0 0x7f6e7c0d4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:42.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.610+0000 7f6e81000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f6e7c0d46a0 0x7f6e7c0d4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35390/0 (socket says 172.21.15.152:35390) 2024-04-20T11:27:42.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.610+0000 7f6e81000640 1 -- 172.21.15.152:0/2127438899 learned_addr learned my addr 172.21.15.152:0/2127438899 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:42.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.610+0000 7f6e81a00640 1 --2- 172.21.15.152:0/2127438899 >> v2:172.21.15.175:3300/0 conn(0x7f6e7c15e9e0 0x7f6e7c06d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:42.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.611+0000 7f6e81000640 1 -- 172.21.15.152:0/2127438899 >> v2:172.21.15.175:3300/0 conn(0x7f6e7c15e9e0 msgr2=0x7f6e7c06d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:42.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.611+0000 7f6e81000640 1 --2- 172.21.15.152:0/2127438899 >> v2:172.21.15.175:3300/0 conn(0x7f6e7c15e9e0 0x7f6e7c06d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:42.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.611+0000 7f6e81000640 1 -- 172.21.15.152:0/2127438899 >> v1:172.21.15.175:6789/0 conn(0x7f6e7c0d0d90 legacy=0x7f6e7c0cfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:42.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.611+0000 7f6e81000640 1 -- 172.21.15.152:0/2127438899 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e7c0d1f60 con 0x7f6e7c0d46a0 2024-04-20T11:27:42.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.612+0000 7f6e81000640 1 --2- 172.21.15.152:0/2127438899 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6e7c0d46a0 0x7f6e7c0d4b00 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6e7c09e0e0 tx=0x7f6e6802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac26bfc09755aceb server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:42.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.612+0000 7f6e7be00640 1 -- 172.21.15.152:0/2127438899 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e68034070 con 0x7f6e7c0d46a0 2024-04-20T11:27:42.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.612+0000 7f6e7be00640 1 -- 172.21.15.152:0/2127438899 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e68030860 con 0x7f6e7c0d46a0 2024-04-20T11:27:42.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.612+0000 7f6e7be00640 1 -- 172.21.15.152:0/2127438899 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e68030c30 con 0x7f6e7c0d46a0 2024-04-20T11:27:42.614 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.613+0000 7f6e84400640 1 -- 172.21.15.152:0/2127438899 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6e7c0d46a0 msgr2=0x7f6e7c0d4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:42.614 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.613+0000 7f6e84400640 1 --2- 172.21.15.152:0/2127438899 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6e7c0d46a0 0x7f6e7c0d4b00 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6e7c09e0e0 tx=0x7f6e6802b9a0 comp rx=0 tx=0).stop 2024-04-20T11:27:42.614 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.613+0000 7f6e84400640 1 -- 172.21.15.152:0/2127438899 shutdown_connections 2024-04-20T11:27:42.614 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.613+0000 7f6e84400640 1 --2- 172.21.15.152:0/2127438899 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6e7c0d46a0 0x7f6e7c0d4b00 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:42.614 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.613+0000 7f6e84400640 1 --2- 172.21.15.152:0/2127438899 >> v2:172.21.15.175:3300/0 conn(0x7f6e7c15e9e0 0x7f6e7c06d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:42.614 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.613+0000 7f6e84400640 1 -- 172.21.15.152:0/2127438899 >> 172.21.15.152:0/2127438899 conn(0x7f6e7c09dab0 msgr2=0x7f6e7c08bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:42.614 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.613+0000 7f6e84400640 1 -- 172.21.15.152:0/2127438899 shutdown_connections 2024-04-20T11:27:42.614 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.613+0000 7f6e84400640 1 -- 172.21.15.152:0/2127438899 wait complete. 2024-04-20T11:27:42.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.615+0000 7f6e84400640 1 Processor -- start 2024-04-20T11:27:42.616 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.615+0000 7f6e84400640 1 -- start start 2024-04-20T11:27:42.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.616+0000 7f6e84400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e7c0d0d90 0x7f6e7c15cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:42.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.616+0000 7f6e84400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e7c0d46a0 0x7f6e7c144760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:42.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.616+0000 7f6e84400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6e7c15e9e0 0x7f6e7c0b7800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:42.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.616+0000 7f6e84400640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f6e7c06d860 con 0x7f6e7c15e9e0 2024-04-20T11:27:42.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.616+0000 7f6e84400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6e7c0cf880 con 0x7f6e7c0d0d90 2024-04-20T11:27:42.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.616+0000 7f6e84400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6e7c06d9e0 con 0x7f6e7c0d46a0 2024-04-20T11:27:42.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.616+0000 7f6e82400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6e7c15e9e0 0x7f6e7c0b7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:42.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.617+0000 7f6e82400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6e7c15e9e0 0x7f6e7c0b7800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35404/0 (socket says 172.21.15.152:35404) 2024-04-20T11:27:42.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.617+0000 7f6e82400640 1 -- 172.21.15.152:0/2995872668 learned_addr learned my addr 172.21.15.152:0/2995872668 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:42.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.617+0000 7f6e81000640 1 --2- 172.21.15.152:0/2995872668 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e7c0d46a0 0x7f6e7c144760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:42.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.617+0000 7f6e81a00640 1 --2- 172.21.15.152:0/2995872668 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e7c0d0d90 0x7f6e7c15cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:42.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.617+0000 7f6e82400640 1 -- 172.21.15.152:0/2995872668 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e7c0d46a0 msgr2=0x7f6e7c144760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:42.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.617+0000 7f6e82400640 1 --2- 172.21.15.152:0/2995872668 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e7c0d46a0 0x7f6e7c144760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:42.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.617+0000 7f6e82400640 1 -- 172.21.15.152:0/2995872668 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e7c0d0d90 msgr2=0x7f6e7c15cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:42.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.617+0000 7f6e82400640 1 --2- 172.21.15.152:0/2995872668 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e7c0d0d90 0x7f6e7c15cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:42.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.617+0000 7f6e82400640 1 -- 172.21.15.152:0/2995872668 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e7c0b9160 con 0x7f6e7c15e9e0 2024-04-20T11:27:42.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.617+0000 7f6e82400640 1 --2- 172.21.15.152:0/2995872668 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6e7c15e9e0 0x7f6e7c0b7800 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f6e7400c7d0 tx=0x7f6e7400cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:42.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.618+0000 7f6e79600640 1 -- 172.21.15.152:0/2995872668 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e74003b70 con 0x7f6e7c15e9e0 2024-04-20T11:27:42.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.618+0000 7f6e79600640 1 -- 172.21.15.152:0/2995872668 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e74003d30 con 0x7f6e7c15e9e0 2024-04-20T11:27:42.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.618+0000 7f6e79600640 1 -- 172.21.15.152:0/2995872668 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e7400aa90 con 0x7f6e7c15e9e0 2024-04-20T11:27:42.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.618+0000 7f6e84400640 1 -- 172.21.15.152:0/2995872668 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e7c144eb0 con 0x7f6e7c15e9e0 2024-04-20T11:27:42.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.618+0000 7f6e84400640 1 -- 172.21.15.152:0/2995872668 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e7c145470 con 0x7f6e7c15e9e0 2024-04-20T11:27:42.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.619+0000 7f6e84400640 1 -- 172.21.15.152:0/2995872668 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e7c1456b0 con 0x7f6e7c15e9e0 2024-04-20T11:27:42.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.620+0000 7f6e79600640 1 -- 172.21.15.152:0/2995872668 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6e7400ac30 con 0x7f6e7c15e9e0 2024-04-20T11:27:42.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.621+0000 7f6e79600640 1 --2- 172.21.15.152:0/2995872668 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f6e5c06c8f0 0x7f6e5c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:42.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.621+0000 7f6e79600640 1 -- 172.21.15.152:0/2995872668 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f6e7408bbd0 con 0x7f6e7c15e9e0 2024-04-20T11:27:42.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.621+0000 7f6e81a00640 1 --2- 172.21.15.152:0/2995872668 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f6e5c06c8f0 0x7f6e5c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:42.623 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.622+0000 7f6e81a00640 1 --2- 172.21.15.152:0/2995872668 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f6e5c06c8f0 0x7f6e5c06edb0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6e70009a30 tx=0x7f6e70008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:42.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.626+0000 7f6e79600640 1 -- 172.21.15.152:0/2995872668 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f6e7c1456b0 con 0x7f6e7c15e9e0 2024-04-20T11:27:42.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.826+0000 7f6e84400640 1 -- 172.21.15.152:0/2995872668 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6e7c0689f0 con 0x7f6e7c15e9e0 2024-04-20T11:27:42.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.828+0000 7f6e79600640 1 -- 172.21.15.152:0/2995872668 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) ==== 93+0+10 (secure 0 0 0) 0x7f6e74059ee0 con 0x7f6e7c15e9e0 2024-04-20T11:27:42.828 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 1 2024-04-20T11:27:42.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.831+0000 7f6e84400640 1 -- 172.21.15.152:0/2995872668 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f6e5c06c8f0 msgr2=0x7f6e5c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:42.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.831+0000 7f6e84400640 1 --2- 172.21.15.152:0/2995872668 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f6e5c06c8f0 0x7f6e5c06edb0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6e70009a30 tx=0x7f6e70008040 comp rx=0 tx=0).stop 2024-04-20T11:27:42.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.831+0000 7f6e84400640 1 -- 172.21.15.152:0/2995872668 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6e7c15e9e0 msgr2=0x7f6e7c0b7800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:42.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.831+0000 7f6e84400640 1 --2- 172.21.15.152:0/2995872668 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6e7c15e9e0 0x7f6e7c0b7800 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f6e7400c7d0 tx=0x7f6e7400cc90 comp rx=0 tx=0).stop 2024-04-20T11:27:42.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.831+0000 7f6e84400640 1 -- 172.21.15.152:0/2995872668 shutdown_connections 2024-04-20T11:27:42.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.831+0000 7f6e84400640 1 --2- 172.21.15.152:0/2995872668 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f6e5c06c8f0 0x7f6e5c06edb0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:42.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.831+0000 7f6e84400640 1 --2- 172.21.15.152:0/2995872668 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6e7c15e9e0 0x7f6e7c0b7800 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:42.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.831+0000 7f6e84400640 1 --2- 172.21.15.152:0/2995872668 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e7c0d46a0 0x7f6e7c144760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:42.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.831+0000 7f6e84400640 1 --2- 172.21.15.152:0/2995872668 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e7c0d0d90 0x7f6e7c15cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:42.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.831+0000 7f6e84400640 1 -- 172.21.15.152:0/2995872668 >> 172.21.15.152:0/2995872668 conn(0x7f6e7c09dab0 msgr2=0x7f6e7c0b6b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:42.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.831+0000 7f6e84400640 1 -- 172.21.15.152:0/2995872668 shutdown_connections 2024-04-20T11:27:42.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:42.831+0000 7f6e84400640 1 -- 172.21.15.152:0/2995872668 wait complete. 2024-04-20T11:27:42.850 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T11:27:43.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.057+0000 7f228fe00640 1 Processor -- start 2024-04-20T11:27:43.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.058+0000 7f228fe00640 1 -- start start 2024-04-20T11:27:43.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.059+0000 7f228fe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f22901480b0 0x7f229015ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:43.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.059+0000 7f228fe00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f229015f7e0 con 0x7f229015f1f0 2024-04-20T11:27:43.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.059+0000 7f228fe00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f229015fac0 con 0x7f2290147cd0 2024-04-20T11:27:43.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.059+0000 7f228fe00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f229006ee10 con 0x7f22901480b0 2024-04-20T11:27:43.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.060+0000 7f228f400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f229015f1f0 0x7f229006f1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:39998/0 (socket says 172.21.15.152:39998) 2024-04-20T11:27:43.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.060+0000 7f228f400640 1 -- 172.21.15.152:0/4136396283 learned_addr learned my addr 172.21.15.152:0/4136396283 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:27:43.062 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.060+0000 7f228e000640 1 --2- 172.21.15.152:0/4136396283 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f22901480b0 0x7f229015ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:43.062 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.060+0000 7f228d600640 1 -- 172.21.15.152:0/4136396283 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2233926164 0 0) 0x7f229015f7e0 con 0x7f229015f1f0 2024-04-20T11:27:43.063 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.060+0000 7f228d600640 1 -- 172.21.15.152:0/4136396283 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2274003520 con 0x7f229015f1f0 2024-04-20T11:27:43.063 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.060+0000 7f228d600640 1 -- 172.21.15.152:0/4136396283 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1733367464 0 0) 0x7f2274003520 con 0x7f229015f1f0 2024-04-20T11:27:43.063 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.061+0000 7f228d600640 1 -- 172.21.15.152:0/4136396283 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f22901480b0 msgr2=0x7f229015ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:43.063 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.061+0000 7f228d600640 1 --2- 172.21.15.152:0/4136396283 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f22901480b0 0x7f229015ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:43.063 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.061+0000 7f228d600640 1 -- 172.21.15.152:0/4136396283 >> v1:172.21.15.175:6789/0 conn(0x7f2290147cd0 legacy=0x7f229006e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:43.063 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.061+0000 7f228d600640 1 -- 172.21.15.152:0/4136396283 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f229006df00 con 0x7f229015f1f0 2024-04-20T11:27:43.063 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.061+0000 7f228d600640 1 -- 172.21.15.152:0/4136396283 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f22880019f0 con 0x7f229015f1f0 2024-04-20T11:27:43.063 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.061+0000 7f228d600640 1 -- 172.21.15.152:0/4136396283 >> v1:172.21.15.152:6789/0 conn(0x7f229015f1f0 legacy=0x7f229006f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:43.063 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.061+0000 7f228d600640 1 --2- 172.21.15.152:0/4136396283 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f22740039e0 0x7f2274003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:43.063 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.061+0000 7f228d600640 1 --2- 172.21.15.152:0/4136396283 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f22901480b0 0x7f2274004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:43.063 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.061+0000 7f228d600640 1 --2- 172.21.15.152:0/4136396283 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2274004d40 0x7f22740051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:43.063 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.061+0000 7f228d600640 1 -- 172.21.15.152:0/4136396283 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f229006ee10 con 0x7f22740039e0 2024-04-20T11:27:43.063 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.061+0000 7f228d600640 1 -- 172.21.15.152:0/4136396283 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f227400ba70 con 0x7f2274004d40 2024-04-20T11:27:43.064 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.061+0000 7f228d600640 1 -- 172.21.15.152:0/4136396283 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f227400bc70 con 0x7f22901480b0 2024-04-20T11:27:43.064 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.061+0000 7f228ea00640 1 --2- 172.21.15.152:0/4136396283 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f22740039e0 0x7f2274003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:43.064 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.061+0000 7f228f400640 1 --2- 172.21.15.152:0/4136396283 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2274004d40 0x7f22740051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:43.064 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.061+0000 7f228e000640 1 --2- 172.21.15.152:0/4136396283 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f22901480b0 0x7f2274004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:43.064 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.061+0000 7f228ea00640 1 -- 172.21.15.152:0/4136396283 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f22901480b0 msgr2=0x7f2274004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:43.064 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.061+0000 7f228ea00640 1 --2- 172.21.15.152:0/4136396283 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f22901480b0 0x7f2274004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:43.064 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.061+0000 7f228ea00640 1 -- 172.21.15.152:0/4136396283 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2274004d40 msgr2=0x7f22740051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:43.064 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.061+0000 7f228ea00640 1 --2- 172.21.15.152:0/4136396283 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2274004d40 0x7f22740051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:43.064 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.061+0000 7f228ea00640 1 -- 172.21.15.152:0/4136396283 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2274003520 con 0x7f22740039e0 2024-04-20T11:27:43.064 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.062+0000 7f228f400640 1 --2- 172.21.15.152:0/4136396283 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2274004d40 0x7f22740051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:27:43.065 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.063+0000 7f228ea00640 1 --2- 172.21.15.152:0/4136396283 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f22740039e0 0x7f2274003e00 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f2280001720 tx=0x7f2274005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55f9c7042b0c3397 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:43.065 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.063+0000 7f228d600640 1 -- 172.21.15.152:0/4136396283 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2280001370 con 0x7f22740039e0 2024-04-20T11:27:43.065 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.063+0000 7f228d600640 1 -- 172.21.15.152:0/4136396283 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22800052d0 con 0x7f22740039e0 2024-04-20T11:27:43.065 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.063+0000 7f228d600640 1 -- 172.21.15.152:0/4136396283 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2280003830 con 0x7f22740039e0 2024-04-20T11:27:43.065 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.063+0000 7f228fe00640 1 -- 172.21.15.152:0/4136396283 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f22740039e0 msgr2=0x7f2274003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:43.066 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.063+0000 7f228fe00640 1 --2- 172.21.15.152:0/4136396283 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f22740039e0 0x7f2274003e00 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f2280001720 tx=0x7f2274005830 comp rx=0 tx=0).stop 2024-04-20T11:27:43.066 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.063+0000 7f228f400640 1 -- 172.21.15.152:0/4136396283 reap_dead start 2024-04-20T11:27:43.066 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.064+0000 7f228fe00640 1 -- 172.21.15.152:0/4136396283 shutdown_connections 2024-04-20T11:27:43.066 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.064+0000 7f228fe00640 1 -- 172.21.15.152:0/4136396283 >> 172.21.15.152:0/4136396283 conn(0x7f2290067ae0 msgr2=0x7f229006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:43.066 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.064+0000 7f228fe00640 1 -- 172.21.15.152:0/4136396283 shutdown_connections 2024-04-20T11:27:43.066 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.064+0000 7f228fe00640 1 -- 172.21.15.152:0/4136396283 wait complete. 2024-04-20T11:27:43.067 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.065+0000 7f228fe00640 1 Processor -- start 2024-04-20T11:27:43.067 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.066+0000 7f228fe00640 1 -- start start 2024-04-20T11:27:43.067 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.066+0000 7f228fe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f22740039e0 0x7f22900b2210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:43.067 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.067+0000 7f228fe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f229013cb90 0x7f22900aea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:43.067 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.067+0000 7f228fe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22900b04e0 0x7f22900b0960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:43.068 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.067+0000 7f228fe00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f229006f8d0 con 0x7f22740039e0 2024-04-20T11:27:43.068 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.067+0000 7f228fe00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2290071ed0 con 0x7f22900b04e0 2024-04-20T11:27:43.068 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.067+0000 7f228fe00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f229006fa50 con 0x7f229013cb90 2024-04-20T11:27:43.068 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.067+0000 7f228ea00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f22740039e0 0x7f22900b2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:43.068 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.067+0000 7f228ea00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f22740039e0 0x7f22900b2210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35412/0 (socket says 172.21.15.152:35412) 2024-04-20T11:27:43.068 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.067+0000 7f228ea00640 1 -- 172.21.15.152:0/113532498 learned_addr learned my addr 172.21.15.152:0/113532498 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:43.069 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.067+0000 7f228ea00640 1 -- 172.21.15.152:0/113532498 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f229013cb90 msgr2=0x7f22900aea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:43.069 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.067+0000 7f228ea00640 1 --2- 172.21.15.152:0/113532498 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f229013cb90 0x7f22900aea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:43.069 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.067+0000 7f228ea00640 1 -- 172.21.15.152:0/113532498 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22900b04e0 msgr2=0x7f22900b0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:43.069 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.067+0000 7f228ea00640 1 --2- 172.21.15.152:0/113532498 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22900b04e0 0x7f22900b0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:43.069 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.067+0000 7f228ea00640 1 -- 172.21.15.152:0/113532498 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22900b2830 con 0x7f22740039e0 2024-04-20T11:27:43.069 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.068+0000 7f228ea00640 1 --2- 172.21.15.152:0/113532498 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f22740039e0 0x7f22900b2210 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f2280005040 tx=0x7f2280001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:43.069 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.068+0000 7f2286a00640 1 -- 172.21.15.152:0/113532498 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2280041070 con 0x7f22740039e0 2024-04-20T11:27:43.069 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.068+0000 7f2286a00640 1 -- 172.21.15.152:0/113532498 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2280005480 con 0x7f22740039e0 2024-04-20T11:27:43.069 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.068+0000 7f2286a00640 1 -- 172.21.15.152:0/113532498 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2280001370 con 0x7f22740039e0 2024-04-20T11:27:43.069 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.068+0000 7f228fe00640 1 -- 172.21.15.152:0/113532498 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22900b11b0 con 0x7f22740039e0 2024-04-20T11:27:43.069 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.068+0000 7f228fe00640 1 -- 172.21.15.152:0/113532498 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22900b1770 con 0x7f22740039e0 2024-04-20T11:27:43.070 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.069+0000 7f228fe00640 1 -- 172.21.15.152:0/113532498 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2290075e60 con 0x7f22740039e0 2024-04-20T11:27:43.071 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.070+0000 7f2286a00640 1 -- 172.21.15.152:0/113532498 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2280044070 con 0x7f22740039e0 2024-04-20T11:27:43.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.071+0000 7f2286a00640 1 --2- 172.21.15.152:0/113532498 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f226c06c920 0x7f226c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:43.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.071+0000 7f2286a00640 1 -- 172.21.15.152:0/113532498 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f22800ad6e0 con 0x7f22740039e0 2024-04-20T11:27:43.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.072+0000 7f228e000640 1 --2- 172.21.15.152:0/113532498 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f226c06c920 0x7f226c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:43.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.072+0000 7f228e000640 1 --2- 172.21.15.152:0/113532498 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f226c06c920 0x7f226c06ede0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2290146320 tx=0x7f2278008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:43.076 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.076+0000 7f2286a00640 1 -- 172.21.15.152:0/113532498 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f228007b9f0 con 0x7f22740039e0 2024-04-20T11:27:43.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.278+0000 7f228fe00640 1 -- 172.21.15.152:0/113532498 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f22900cd470 con 0x7f22740039e0 2024-04-20T11:27:43.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.279+0000 7f2286a00640 1 -- 172.21.15.152:0/113532498 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) ==== 104+0+11 (secure 0 0 0) 0x7f228007b2f0 con 0x7f22740039e0 2024-04-20T11:27:43.280 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 32 2024-04-20T11:27:43.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.282+0000 7f228fe00640 1 -- 172.21.15.152:0/113532498 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f226c06c920 msgr2=0x7f226c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:43.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.282+0000 7f228fe00640 1 --2- 172.21.15.152:0/113532498 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f226c06c920 0x7f226c06ede0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2290146320 tx=0x7f2278008040 comp rx=0 tx=0).stop 2024-04-20T11:27:43.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.282+0000 7f228fe00640 1 -- 172.21.15.152:0/113532498 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f22740039e0 msgr2=0x7f22900b2210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:43.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.282+0000 7f228fe00640 1 --2- 172.21.15.152:0/113532498 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f22740039e0 0x7f22900b2210 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f2280005040 tx=0x7f2280001120 comp rx=0 tx=0).stop 2024-04-20T11:27:43.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.282+0000 7f228fe00640 1 -- 172.21.15.152:0/113532498 shutdown_connections 2024-04-20T11:27:43.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.282+0000 7f228fe00640 1 --2- 172.21.15.152:0/113532498 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f226c06c920 0x7f226c06ede0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:43.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.282+0000 7f228fe00640 1 --2- 172.21.15.152:0/113532498 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22900b04e0 0x7f22900b0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:43.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.282+0000 7f228fe00640 1 --2- 172.21.15.152:0/113532498 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f229013cb90 0x7f22900aea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:43.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.282+0000 7f228fe00640 1 --2- 172.21.15.152:0/113532498 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f22740039e0 0x7f22900b2210 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:43.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.282+0000 7f228fe00640 1 -- 172.21.15.152:0/113532498 >> 172.21.15.152:0/113532498 conn(0x7f2290067ae0 msgr2=0x7f22900cd7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:43.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.282+0000 7f228fe00640 1 -- 172.21.15.152:0/113532498 shutdown_connections 2024-04-20T11:27:43.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.282+0000 7f228fe00640 1 -- 172.21.15.152:0/113532498 wait complete. 2024-04-20T11:27:43.301 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T11:27:43.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.504+0000 7fed37e00640 1 Processor -- start 2024-04-20T11:27:43.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.504+0000 7fed37e00640 1 -- start start 2024-04-20T11:27:43.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.505+0000 7fed37e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fed38156c00 0x7fed38160a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:43.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.505+0000 7fed37e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fed3815ca40 0x7fed3815cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:43.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.505+0000 7fed37e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fed3806bda0 0x7fed3806c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:43.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.505+0000 7fed37e00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fed38160fd0 con 0x7fed3806bda0 2024-04-20T11:27:43.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.505+0000 7fed37e00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fed3815d5c0 con 0x7fed38156c00 2024-04-20T11:27:43.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.505+0000 7fed37e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fed3815d8a0 con 0x7fed3815ca40 2024-04-20T11:27:43.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.506+0000 7fed37400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fed3806bda0 0x7fed3806c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:43.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.506+0000 7fed37400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fed3806bda0 0x7fed3806c230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35418/0 (socket says 172.21.15.152:35418) 2024-04-20T11:27:43.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.506+0000 7fed37400640 1 -- 172.21.15.152:0/345015969 learned_addr learned my addr 172.21.15.152:0/345015969 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:43.508 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.506+0000 7fed36a00640 1 --2- 172.21.15.152:0/345015969 >> v2:172.21.15.175:3300/0 conn(0x7fed38156c00 0x7fed38160a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:43.508 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.506+0000 7fed36000640 1 --2- 172.21.15.152:0/345015969 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fed3815ca40 0x7fed3815cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:43.508 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.506+0000 7fed37400640 1 -- 172.21.15.152:0/345015969 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fed3815ca40 msgr2=0x7fed3815cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:43.508 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.506+0000 7fed37400640 1 --2- 172.21.15.152:0/345015969 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fed3815ca40 0x7fed3815cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:43.508 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.506+0000 7fed37400640 1 -- 172.21.15.152:0/345015969 >> v2:172.21.15.175:3300/0 conn(0x7fed38156c00 msgr2=0x7fed38160a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:43.508 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.506+0000 7fed37400640 1 --2- 172.21.15.152:0/345015969 >> v2:172.21.15.175:3300/0 conn(0x7fed38156c00 0x7fed38160a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:43.508 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.506+0000 7fed37400640 1 -- 172.21.15.152:0/345015969 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed38160290 con 0x7fed3806bda0 2024-04-20T11:27:43.509 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.508+0000 7fed37400640 1 --2- 172.21.15.152:0/345015969 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fed3806bda0 0x7fed3806c230 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fed380d1250 tx=0x7fed3002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49f7f68f44dd5bf server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:43.509 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.508+0000 7fed35600640 1 -- 172.21.15.152:0/345015969 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed3002ca70 con 0x7fed3806bda0 2024-04-20T11:27:43.509 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.508+0000 7fed35600640 1 -- 172.21.15.152:0/345015969 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed3002cc10 con 0x7fed3806bda0 2024-04-20T11:27:43.509 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.508+0000 7fed35600640 1 -- 172.21.15.152:0/345015969 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed300343f0 con 0x7fed3806bda0 2024-04-20T11:27:43.509 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.509+0000 7fed37e00640 1 -- 172.21.15.152:0/345015969 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fed3806bda0 msgr2=0x7fed3806c230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:43.510 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.509+0000 7fed37e00640 1 --2- 172.21.15.152:0/345015969 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fed3806bda0 0x7fed3806c230 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fed380d1250 tx=0x7fed3002bed0 comp rx=0 tx=0).stop 2024-04-20T11:27:43.510 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.509+0000 7fed37e00640 1 -- 172.21.15.152:0/345015969 shutdown_connections 2024-04-20T11:27:43.510 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.509+0000 7fed37e00640 1 --2- 172.21.15.152:0/345015969 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fed3806bda0 0x7fed3806c230 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:43.510 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.509+0000 7fed37e00640 1 --2- 172.21.15.152:0/345015969 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fed3815ca40 0x7fed3815cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:43.510 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.509+0000 7fed37e00640 1 --2- 172.21.15.152:0/345015969 >> v2:172.21.15.175:3300/0 conn(0x7fed38156c00 0x7fed38160a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:43.510 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.509+0000 7fed37e00640 1 -- 172.21.15.152:0/345015969 >> 172.21.15.152:0/345015969 conn(0x7fed380b21e0 msgr2=0x7fed380b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:43.510 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.509+0000 7fed37e00640 1 -- 172.21.15.152:0/345015969 shutdown_connections 2024-04-20T11:27:43.510 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.509+0000 7fed37e00640 1 -- 172.21.15.152:0/345015969 wait complete. 2024-04-20T11:27:43.511 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.511+0000 7fed37e00640 1 Processor -- start 2024-04-20T11:27:43.511 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.511+0000 7fed37e00640 1 -- start start 2024-04-20T11:27:43.512 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.512+0000 7fed37e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fed3806bda0 0x7fed3814d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:43.512 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.512+0000 7fed37e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fed38156c00 0x7fed3809dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:43.512 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.512+0000 7fed37e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed3815ca40 0x7fed3815b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:43.513 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.512+0000 7fed37e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fed380b3240 con 0x7fed3806bda0 2024-04-20T11:27:43.513 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.512+0000 7fed37e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fed380b30c0 con 0x7fed3815ca40 2024-04-20T11:27:43.513 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.512+0000 7fed37e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fed380b33c0 con 0x7fed38156c00 2024-04-20T11:27:43.513 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.512+0000 7fed36a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fed3806bda0 0x7fed3814d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:43.513 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.512+0000 7fed36a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fed3806bda0 0x7fed3814d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35422/0 (socket says 172.21.15.152:35422) 2024-04-20T11:27:43.513 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.512+0000 7fed36a00640 1 -- 172.21.15.152:0/4096251949 learned_addr learned my addr 172.21.15.152:0/4096251949 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:43.514 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.512+0000 7fed37400640 1 --2- 172.21.15.152:0/4096251949 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed3815ca40 0x7fed3815b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:43.514 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.512+0000 7fed36000640 1 --2- 172.21.15.152:0/4096251949 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fed38156c00 0x7fed3809dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:43.514 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.512+0000 7fed36a00640 1 -- 172.21.15.152:0/4096251949 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fed38156c00 msgr2=0x7fed3809dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:43.514 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.512+0000 7fed36a00640 1 --2- 172.21.15.152:0/4096251949 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fed38156c00 0x7fed3809dbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:43.514 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.512+0000 7fed36a00640 1 -- 172.21.15.152:0/4096251949 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed3815ca40 msgr2=0x7fed3815b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:43.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.512+0000 7fed36a00640 1 --2- 172.21.15.152:0/4096251949 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed3815ca40 0x7fed3815b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:43.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.512+0000 7fed36a00640 1 -- 172.21.15.152:0/4096251949 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed38071d50 con 0x7fed3806bda0 2024-04-20T11:27:43.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.513+0000 7fed36a00640 1 --2- 172.21.15.152:0/4096251949 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fed3806bda0 0x7fed3814d1d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fed2800d800 tx=0x7fed2800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:43.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.513+0000 7fed2ea00640 1 -- 172.21.15.152:0/4096251949 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed28018020 con 0x7fed3806bda0 2024-04-20T11:27:43.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.513+0000 7fed2ea00640 1 -- 172.21.15.152:0/4096251949 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed280096a0 con 0x7fed3806bda0 2024-04-20T11:27:43.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.513+0000 7fed2ea00640 1 -- 172.21.15.152:0/4096251949 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed28009a70 con 0x7fed3806bda0 2024-04-20T11:27:43.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.513+0000 7fed37e00640 1 -- 172.21.15.152:0/4096251949 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed38072030 con 0x7fed3806bda0 2024-04-20T11:27:43.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.513+0000 7fed37e00640 1 -- 172.21.15.152:0/4096251949 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed380725f0 con 0x7fed3806bda0 2024-04-20T11:27:43.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.515+0000 7fed37e00640 1 -- 172.21.15.152:0/4096251949 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed38075e60 con 0x7fed3806bda0 2024-04-20T11:27:43.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.515+0000 7fed2ea00640 1 -- 172.21.15.152:0/4096251949 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fed2801b020 con 0x7fed3806bda0 2024-04-20T11:27:43.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.516+0000 7fed2ea00640 1 --2- 172.21.15.152:0/4096251949 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fed1806c920 0x7fed1806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:43.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.516+0000 7fed2ea00640 1 -- 172.21.15.152:0/4096251949 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fed2808bd80 con 0x7fed3806bda0 2024-04-20T11:27:43.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.517+0000 7fed36000640 1 --2- 172.21.15.152:0/4096251949 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fed1806c920 0x7fed1806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:43.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.517+0000 7fed36000640 1 --2- 172.21.15.152:0/4096251949 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fed1806c920 0x7fed1806ede0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fed38155af0 tx=0x7fed24008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:43.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.521+0000 7fed2ea00640 1 -- 172.21.15.152:0/4096251949 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fed28022030 con 0x7fed3806bda0 2024-04-20T11:27:43.720 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.720+0000 7fed37e00640 1 -- 172.21.15.152:0/4096251949 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fed3815fa60 con 0x7fed3806bda0 2024-04-20T11:27:43.721 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.721+0000 7fed2ea00640 1 -- 172.21.15.152:0/4096251949 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) ==== 100+0+11 (secure 0 0 0) 0x7fed2805a1c0 con 0x7fed3806bda0 2024-04-20T11:27:43.721 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 64 2024-04-20T11:27:43.725 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.724+0000 7fed37e00640 1 -- 172.21.15.152:0/4096251949 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fed1806c920 msgr2=0x7fed1806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:43.725 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.724+0000 7fed37e00640 1 --2- 172.21.15.152:0/4096251949 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fed1806c920 0x7fed1806ede0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fed38155af0 tx=0x7fed24008040 comp rx=0 tx=0).stop 2024-04-20T11:27:43.725 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.724+0000 7fed37e00640 1 -- 172.21.15.152:0/4096251949 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fed3806bda0 msgr2=0x7fed3814d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:43.725 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.724+0000 7fed37e00640 1 --2- 172.21.15.152:0/4096251949 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fed3806bda0 0x7fed3814d1d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fed2800d800 tx=0x7fed2800dcc0 comp rx=0 tx=0).stop 2024-04-20T11:27:43.725 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.724+0000 7fed37e00640 1 -- 172.21.15.152:0/4096251949 shutdown_connections 2024-04-20T11:27:43.725 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.724+0000 7fed37e00640 1 --2- 172.21.15.152:0/4096251949 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fed1806c920 0x7fed1806ede0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:43.725 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.724+0000 7fed37e00640 1 --2- 172.21.15.152:0/4096251949 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed3815ca40 0x7fed3815b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:43.725 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.724+0000 7fed37e00640 1 --2- 172.21.15.152:0/4096251949 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fed38156c00 0x7fed3809dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:43.725 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.724+0000 7fed37e00640 1 --2- 172.21.15.152:0/4096251949 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fed3806bda0 0x7fed3814d1d0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:43.725 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.724+0000 7fed37e00640 1 -- 172.21.15.152:0/4096251949 >> 172.21.15.152:0/4096251949 conn(0x7fed380b21e0 msgr2=0x7fed380b37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:43.725 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.724+0000 7fed37e00640 1 -- 172.21.15.152:0/4096251949 shutdown_connections 2024-04-20T11:27:43.725 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.724+0000 7fed37e00640 1 -- 172.21.15.152:0/4096251949 wait complete. 2024-04-20T11:27:43.744 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-20T11:27:43.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.946+0000 7feafea00640 1 Processor -- start 2024-04-20T11:27:43.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.946+0000 7feafea00640 1 -- start start 2024-04-20T11:27:43.948 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.948+0000 7feafea00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feaf80d0700 0x7feaf806f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:43.948 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.948+0000 7feafea00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feaf80b3a50 con 0x7feaf80b3470 2024-04-20T11:27:43.948 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.948+0000 7feafea00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feaf806e300 con 0x7feaf8162b40 2024-04-20T11:27:43.948 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.948+0000 7feafea00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7feaf806e0b0 con 0x7feaf80d0700 2024-04-20T11:27:43.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.948+0000 7feaf7400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7feaf80b3470 0x7feaf80cfff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:40008/0 (socket says 172.21.15.152:40008) 2024-04-20T11:27:43.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.948+0000 7feaf7400640 1 -- 172.21.15.152:0/57653511 learned_addr learned my addr 172.21.15.152:0/57653511 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:27:43.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.948+0000 7feafca00640 1 --2- 172.21.15.152:0/57653511 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feaf80d0700 0x7feaf806f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:43.950 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.949+0000 7feaf6a00640 1 -- 172.21.15.152:0/57653511 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2225253291 0 0) 0x7feaf80b3a50 con 0x7feaf80b3470 2024-04-20T11:27:43.950 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.949+0000 7feaf6a00640 1 -- 172.21.15.152:0/57653511 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feadc003520 con 0x7feaf80b3470 2024-04-20T11:27:43.950 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.949+0000 7feaf6a00640 1 -- 172.21.15.152:0/57653511 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 357124465 0 0) 0x7feaf806e300 con 0x7feaf8162b40 2024-04-20T11:27:43.950 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.949+0000 7feafca00640 1 -- 172.21.15.152:0/57653511 >> v1:172.21.15.175:6789/0 conn(0x7feaf8162b40 legacy=0x7feaf806ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:43.950 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.949+0000 7feafca00640 1 -- 172.21.15.152:0/57653511 >> v1:172.21.15.152:6789/0 conn(0x7feaf80b3470 legacy=0x7feaf80cfff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:43.950 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.949+0000 7feafca00640 1 -- 172.21.15.152:0/57653511 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaf806e5e0 con 0x7feaf80d0700 2024-04-20T11:27:43.951 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.951+0000 7feafca00640 1 --2- 172.21.15.152:0/57653511 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feaf80d0700 0x7feaf806f810 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7feaf0009dc0 tx=0x7feaf002be90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=797a573f38047bb5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:43.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.952+0000 7feaf6a00640 1 -- 172.21.15.152:0/57653511 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf0039070 con 0x7feaf80d0700 2024-04-20T11:27:43.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.952+0000 7feaf6a00640 1 -- 172.21.15.152:0/57653511 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feaf002cc40 con 0x7feaf80d0700 2024-04-20T11:27:43.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.952+0000 7feaf6a00640 1 -- 172.21.15.152:0/57653511 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf0034440 con 0x7feaf80d0700 2024-04-20T11:27:43.953 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.953+0000 7feafea00640 1 -- 172.21.15.152:0/57653511 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feaf80d0700 msgr2=0x7feaf806f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:43.953 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.953+0000 7feafea00640 1 --2- 172.21.15.152:0/57653511 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feaf80d0700 0x7feaf806f810 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7feaf0009dc0 tx=0x7feaf002be90 comp rx=0 tx=0).stop 2024-04-20T11:27:43.954 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.953+0000 7feafea00640 1 -- 172.21.15.152:0/57653511 shutdown_connections 2024-04-20T11:27:43.954 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.953+0000 7feafea00640 1 --2- 172.21.15.152:0/57653511 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feaf80d0700 0x7feaf806f810 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:43.954 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.953+0000 7feafea00640 1 -- 172.21.15.152:0/57653511 >> 172.21.15.152:0/57653511 conn(0x7feaf8067cf0 msgr2=0x7feaf806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:43.954 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.953+0000 7feafea00640 1 -- 172.21.15.152:0/57653511 shutdown_connections 2024-04-20T11:27:43.954 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.953+0000 7feafea00640 1 -- 172.21.15.152:0/57653511 wait complete. 2024-04-20T11:27:43.956 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.955+0000 7feafea00640 1 Processor -- start 2024-04-20T11:27:43.956 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.955+0000 7feafea00640 1 -- start start 2024-04-20T11:27:43.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.956+0000 7feafea00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feaf80b3470 0x7feaf81f9920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:43.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.957+0000 7feafea00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feaf80d0700 0x7feaf81fb550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:43.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.957+0000 7feafea00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feaf8162b40 0x7feaf80a9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:43.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.957+0000 7feafea00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7feaf80cd0a0 con 0x7feaf80b3470 2024-04-20T11:27:43.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.957+0000 7feafea00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7feaf806e950 con 0x7feaf80d0700 2024-04-20T11:27:43.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.957+0000 7feafea00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7feaf80cd220 con 0x7feaf8162b40 2024-04-20T11:27:43.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.957+0000 7feaf7e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feaf80b3470 0x7feaf81f9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:43.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.957+0000 7feaf7e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feaf80b3470 0x7feaf81f9920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35428/0 (socket says 172.21.15.152:35428) 2024-04-20T11:27:43.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.957+0000 7feaf7e00640 1 -- 172.21.15.152:0/4245152727 learned_addr learned my addr 172.21.15.152:0/4245152727 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:43.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.957+0000 7feaf7400640 1 --2- 172.21.15.152:0/4245152727 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feaf80d0700 0x7feaf81fb550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:43.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.957+0000 7feafca00640 1 --2- 172.21.15.152:0/4245152727 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feaf8162b40 0x7feaf80a9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:43.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.957+0000 7feaf7e00640 1 -- 172.21.15.152:0/4245152727 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feaf8162b40 msgr2=0x7feaf80a9dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:43.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.957+0000 7feaf7e00640 1 --2- 172.21.15.152:0/4245152727 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feaf8162b40 0x7feaf80a9dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:43.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.957+0000 7feaf7e00640 1 -- 172.21.15.152:0/4245152727 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feaf80d0700 msgr2=0x7feaf81fb550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:43.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.957+0000 7feaf7e00640 1 --2- 172.21.15.152:0/4245152727 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feaf80d0700 0x7feaf81fb550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:43.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.957+0000 7feaf7e00640 1 -- 172.21.15.152:0/4245152727 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaf8164260 con 0x7feaf80b3470 2024-04-20T11:27:43.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.957+0000 7feaf7400640 1 --2- 172.21.15.152:0/4245152727 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feaf80d0700 0x7feaf81fb550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:27:43.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.958+0000 7feaf7e00640 1 --2- 172.21.15.152:0/4245152727 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feaf80b3470 0x7feaf81f9920 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7feae800c410 tx=0x7feae800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:43.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.958+0000 7feaefe00640 1 -- 172.21.15.152:0/4245152727 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feae8017070 con 0x7feaf80b3470 2024-04-20T11:27:43.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.958+0000 7feaefe00640 1 -- 172.21.15.152:0/4245152727 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feae80035f0 con 0x7feaf80b3470 2024-04-20T11:27:43.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.958+0000 7feafea00640 1 -- 172.21.15.152:0/4245152727 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feaf8164540 con 0x7feaf80b3470 2024-04-20T11:27:43.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.958+0000 7feaefe00640 1 -- 172.21.15.152:0/4245152727 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feae80039e0 con 0x7feaf80b3470 2024-04-20T11:27:43.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.958+0000 7feafea00640 1 -- 172.21.15.152:0/4245152727 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feaf81f9e60 con 0x7feaf80b3470 2024-04-20T11:27:43.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.959+0000 7feafea00640 1 -- 172.21.15.152:0/4245152727 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feaf80689f0 con 0x7feaf80b3470 2024-04-20T11:27:43.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.960+0000 7feaefe00640 1 -- 172.21.15.152:0/4245152727 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7feae801a070 con 0x7feaf80b3470 2024-04-20T11:27:43.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.961+0000 7feaefe00640 1 --2- 172.21.15.152:0/4245152727 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fead406c8f0 0x7fead406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:43.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.961+0000 7feaefe00640 1 -- 172.21.15.152:0/4245152727 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7feae808b090 con 0x7feaf80b3470 2024-04-20T11:27:43.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.962+0000 7feaf7400640 1 --2- 172.21.15.152:0/4245152727 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fead406c8f0 0x7fead406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:43.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.962+0000 7feaf7400640 1 --2- 172.21.15.152:0/4245152727 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fead406c8f0 0x7fead406edb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7feaf80b0b10 tx=0x7feae0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:43.966 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:43.966+0000 7feaefe00640 1 -- 172.21.15.152:0/4245152727 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7feae80593b0 con 0x7feaf80b3470 2024-04-20T11:27:44.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.178+0000 7feafea00640 1 -- 172.21.15.152:0/4245152727 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7feaf80b0b40 con 0x7feaf80b3470 2024-04-20T11:27:44.182 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.181+0000 7feaefe00640 1 -- 172.21.15.152:0/4245152727 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) ==== 80+0+0 (secure 0 0 0) 0x7feae8058cb0 con 0x7feaf80b3470 2024-04-20T11:27:44.182 INFO:teuthology.orchestra.run.smithi152.stderr:listed 0 entries 2024-04-20T11:27:44.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.184+0000 7feafea00640 1 -- 172.21.15.152:0/4245152727 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fead406c8f0 msgr2=0x7fead406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:44.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.184+0000 7feafea00640 1 --2- 172.21.15.152:0/4245152727 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fead406c8f0 0x7fead406edb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7feaf80b0b10 tx=0x7feae0009040 comp rx=0 tx=0).stop 2024-04-20T11:27:44.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.184+0000 7feafea00640 1 -- 172.21.15.152:0/4245152727 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feaf80b3470 msgr2=0x7feaf81f9920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:44.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.184+0000 7feafea00640 1 --2- 172.21.15.152:0/4245152727 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feaf80b3470 0x7feaf81f9920 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7feae800c410 tx=0x7feae800c8d0 comp rx=0 tx=0).stop 2024-04-20T11:27:44.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.184+0000 7feafea00640 1 -- 172.21.15.152:0/4245152727 shutdown_connections 2024-04-20T11:27:44.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.184+0000 7feafea00640 1 --2- 172.21.15.152:0/4245152727 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fead406c8f0 0x7fead406edb0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:44.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.184+0000 7feafea00640 1 --2- 172.21.15.152:0/4245152727 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feaf8162b40 0x7feaf80a9dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:44.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.184+0000 7feafea00640 1 --2- 172.21.15.152:0/4245152727 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feaf80d0700 0x7feaf81fb550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:44.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.184+0000 7feafea00640 1 --2- 172.21.15.152:0/4245152727 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feaf80b3470 0x7feaf81f9920 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:44.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.184+0000 7feafea00640 1 -- 172.21.15.152:0/4245152727 >> 172.21.15.152:0/4245152727 conn(0x7feaf8067cf0 msgr2=0x7feaf80af710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:44.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.185+0000 7feafea00640 1 -- 172.21.15.152:0/4245152727 shutdown_connections 2024-04-20T11:27:44.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.185+0000 7feafea00640 1 -- 172.21.15.152:0/4245152727 wait complete. 2024-04-20T11:27:44.203 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-20T11:27:44.203 DEBUG:teuthology.orchestra.run.smithi152:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:27:44.252 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:27:44.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.497+0000 7fb28aa00640 1 Processor -- start 2024-04-20T11:27:44.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.497+0000 7fb28aa00640 1 -- start start 2024-04-20T11:27:44.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.499+0000 7fb28aa00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fb2840cd0d0 0x7fb2840cd580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:44.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.499+0000 7fb28aa00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fb284071fe0 0x7fb284072470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:44.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.499+0000 7fb28aa00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb284146340 con 0x7fb284145f60 2024-04-20T11:27:44.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.499+0000 7fb28aa00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fb28406e9f0 con 0x7fb2840cd0d0 2024-04-20T11:27:44.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.499+0000 7fb28aa00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fb284072d50 con 0x7fb284071fe0 2024-04-20T11:27:44.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.499+0000 7fb283e00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fb284145f60 0x7fb28406ee80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:40020/0 (socket says 172.21.15.152:40020) 2024-04-20T11:27:44.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.499+0000 7fb283e00640 1 -- 172.21.15.152:0/1433654855 learned_addr learned my addr 172.21.15.152:0/1433654855 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:27:44.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.500+0000 7fb27b400640 1 --2- 172.21.15.152:0/1433654855 >> v2:172.21.15.152:3300/0 conn(0x7fb2840cd0d0 0x7fb2840cd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:44.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.500+0000 7fb288a00640 1 --2- 172.21.15.152:0/1433654855 >> v2:172.21.15.175:3300/0 conn(0x7fb284071fe0 0x7fb284072470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:44.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.500+0000 7fb283400640 1 -- 172.21.15.152:0/1433654855 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 649098119 0 0) 0x7fb284146340 con 0x7fb284145f60 2024-04-20T11:27:44.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.500+0000 7fb283400640 1 -- 172.21.15.152:0/1433654855 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb268003520 con 0x7fb284145f60 2024-04-20T11:27:44.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.500+0000 7fb27b400640 1 -- 172.21.15.152:0/1433654855 >> v2:172.21.15.175:3300/0 conn(0x7fb284071fe0 msgr2=0x7fb284072470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:44.502 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.500+0000 7fb27b400640 1 --2- 172.21.15.152:0/1433654855 >> v2:172.21.15.175:3300/0 conn(0x7fb284071fe0 0x7fb284072470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:44.502 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.500+0000 7fb27b400640 1 -- 172.21.15.152:0/1433654855 >> v1:172.21.15.152:6789/0 conn(0x7fb284145f60 legacy=0x7fb28406ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:44.502 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.500+0000 7fb27b400640 1 -- 172.21.15.152:0/1433654855 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb28406e0c0 con 0x7fb2840cd0d0 2024-04-20T11:27:44.503 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.501+0000 7fb27b400640 1 --2- 172.21.15.152:0/1433654855 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb2840cd0d0 0x7fb2840cd580 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fb28406eb70 tx=0x7fb26c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcf1ac4b8909e372 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:44.503 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.501+0000 7fb283400640 1 -- 172.21.15.152:0/1433654855 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb26c035020 con 0x7fb2840cd0d0 2024-04-20T11:27:44.503 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.502+0000 7fb283400640 1 -- 172.21.15.152:0/1433654855 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb26c031830 con 0x7fb2840cd0d0 2024-04-20T11:27:44.503 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.502+0000 7fb283400640 1 -- 172.21.15.152:0/1433654855 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb26c031c00 con 0x7fb2840cd0d0 2024-04-20T11:27:44.503 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.502+0000 7fb28aa00640 1 -- 172.21.15.152:0/1433654855 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb2840cd0d0 msgr2=0x7fb2840cd580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:44.503 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.502+0000 7fb28aa00640 1 --2- 172.21.15.152:0/1433654855 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb2840cd0d0 0x7fb2840cd580 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fb28406eb70 tx=0x7fb26c02b8d0 comp rx=0 tx=0).stop 2024-04-20T11:27:44.503 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.502+0000 7fb28aa00640 1 -- 172.21.15.152:0/1433654855 shutdown_connections 2024-04-20T11:27:44.503 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.502+0000 7fb28aa00640 1 --2- 172.21.15.152:0/1433654855 >> v2:172.21.15.175:3300/0 conn(0x7fb284071fe0 0x7fb284072470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:44.503 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.502+0000 7fb28aa00640 1 --2- 172.21.15.152:0/1433654855 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb2840cd0d0 0x7fb2840cd580 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:44.503 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.502+0000 7fb28aa00640 1 -- 172.21.15.152:0/1433654855 >> 172.21.15.152:0/1433654855 conn(0x7fb284067cf0 msgr2=0x7fb28406b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:44.503 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.502+0000 7fb28aa00640 1 -- 172.21.15.152:0/1433654855 shutdown_connections 2024-04-20T11:27:44.503 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.502+0000 7fb28aa00640 1 -- 172.21.15.152:0/1433654855 wait complete. 2024-04-20T11:27:44.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.504+0000 7fb28aa00640 1 Processor -- start 2024-04-20T11:27:44.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.504+0000 7fb28aa00640 1 -- start start 2024-04-20T11:27:44.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.505+0000 7fb28aa00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb284071fe0 0x7fb284159690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:44.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.505+0000 7fb28aa00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb2840cd0d0 0x7fb284137810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:44.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.505+0000 7fb28aa00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb284145f60 0x7fb284137d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:44.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.505+0000 7fb28aa00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fb2840729b0 con 0x7fb284145f60 2024-04-20T11:27:44.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.505+0000 7fb28aa00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb28406e680 con 0x7fb2840cd0d0 2024-04-20T11:27:44.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.505+0000 7fb28aa00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb284072b30 con 0x7fb284071fe0 2024-04-20T11:27:44.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.505+0000 7fb288a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb284145f60 0x7fb284137d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:44.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.505+0000 7fb288a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb284145f60 0x7fb284137d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35438/0 (socket says 172.21.15.152:35438) 2024-04-20T11:27:44.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.505+0000 7fb288a00640 1 -- 172.21.15.152:0/2780280109 learned_addr learned my addr 172.21.15.152:0/2780280109 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:44.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.506+0000 7fb27b400640 1 --2- 172.21.15.152:0/2780280109 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb2840cd0d0 0x7fb284137810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:44.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.506+0000 7fb283e00640 1 --2- 172.21.15.152:0/2780280109 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb284071fe0 0x7fb284159690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:44.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.506+0000 7fb288a00640 1 -- 172.21.15.152:0/2780280109 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb284071fe0 msgr2=0x7fb284159690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:44.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.506+0000 7fb288a00640 1 --2- 172.21.15.152:0/2780280109 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb284071fe0 0x7fb284159690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:44.508 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.506+0000 7fb288a00640 1 -- 172.21.15.152:0/2780280109 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb2840cd0d0 msgr2=0x7fb284137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:44.508 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.506+0000 7fb288a00640 1 --2- 172.21.15.152:0/2780280109 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb2840cd0d0 0x7fb284137810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:44.508 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.506+0000 7fb288a00640 1 -- 172.21.15.152:0/2780280109 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2840ab570 con 0x7fb284145f60 2024-04-20T11:27:44.508 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.506+0000 7fb288a00640 1 --2- 172.21.15.152:0/2780280109 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb284145f60 0x7fb284137d50 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb27c00d830 tx=0x7fb27c00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:44.508 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.506+0000 7fb280c00640 1 -- 172.21.15.152:0/2780280109 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb27c00b6c0 con 0x7fb284145f60 2024-04-20T11:27:44.508 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.506+0000 7fb280c00640 1 -- 172.21.15.152:0/2780280109 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb27c019070 con 0x7fb284145f60 2024-04-20T11:27:44.508 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.506+0000 7fb280c00640 1 -- 172.21.15.152:0/2780280109 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb27c00bac0 con 0x7fb284145f60 2024-04-20T11:27:44.508 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.506+0000 7fb28aa00640 1 -- 172.21.15.152:0/2780280109 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb284159bd0 con 0x7fb284145f60 2024-04-20T11:27:44.508 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.507+0000 7fb28aa00640 1 -- 172.21.15.152:0/2780280109 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb284135b70 con 0x7fb284145f60 2024-04-20T11:27:44.508 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.508+0000 7fb28aa00640 1 -- 172.21.15.152:0/2780280109 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2840689f0 con 0x7fb284145f60 2024-04-20T11:27:44.509 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.509+0000 7fb280c00640 1 -- 172.21.15.152:0/2780280109 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb27c00bc60 con 0x7fb284145f60 2024-04-20T11:27:44.510 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.510+0000 7fb280c00640 1 --2- 172.21.15.152:0/2780280109 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fb26006c8f0 0x7fb26006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:44.510 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.510+0000 7fb280c00640 1 -- 172.21.15.152:0/2780280109 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fb27c08c030 con 0x7fb284145f60 2024-04-20T11:27:44.511 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.510+0000 7fb283e00640 1 --2- 172.21.15.152:0/2780280109 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fb26006c8f0 0x7fb26006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:44.511 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.511+0000 7fb283e00640 1 --2- 172.21.15.152:0/2780280109 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fb26006c8f0 0x7fb26006edb0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb2740012e0 tx=0x7fb274009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:44.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.515+0000 7fb280c00640 1 -- 172.21.15.152:0/2780280109 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fb27c05a3e0 con 0x7fb284145f60 2024-04-20T11:27:44.715 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.714+0000 7fb28aa00640 1 -- 172.21.15.152:0/2780280109 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb28406f720 con 0x7fb284145f60 2024-04-20T11:27:44.716 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.716+0000 7fb280c00640 1 -- 172.21.15.152:0/2780280109 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16914 (secure 0 0 0) 0x7fb27c059ce0 con 0x7fb284145f60 2024-04-20T11:27:44.717 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:27:44.717 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":23,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:27:42.601620+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:26:56.261424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T11:26:57.079286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":2}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.26","mappings":[{"from":7,"to":1}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:27:44.720 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.718+0000 7fb28aa00640 1 -- 172.21.15.152:0/2780280109 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fb26006c8f0 msgr2=0x7fb26006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:44.720 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.718+0000 7fb28aa00640 1 --2- 172.21.15.152:0/2780280109 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fb26006c8f0 0x7fb26006edb0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb2740012e0 tx=0x7fb274009040 comp rx=0 tx=0).stop 2024-04-20T11:27:44.720 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.718+0000 7fb28aa00640 1 -- 172.21.15.152:0/2780280109 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb284145f60 msgr2=0x7fb284137d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:44.720 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.718+0000 7fb28aa00640 1 --2- 172.21.15.152:0/2780280109 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb284145f60 0x7fb284137d50 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb27c00d830 tx=0x7fb27c00dcf0 comp rx=0 tx=0).stop 2024-04-20T11:27:44.720 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.719+0000 7fb28aa00640 1 -- 172.21.15.152:0/2780280109 shutdown_connections 2024-04-20T11:27:44.720 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.719+0000 7fb28aa00640 1 --2- 172.21.15.152:0/2780280109 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fb26006c8f0 0x7fb26006edb0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:44.720 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.719+0000 7fb28aa00640 1 --2- 172.21.15.152:0/2780280109 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb284145f60 0x7fb284137d50 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:44.720 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.719+0000 7fb28aa00640 1 --2- 172.21.15.152:0/2780280109 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb2840cd0d0 0x7fb284137810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:44.720 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.719+0000 7fb28aa00640 1 --2- 172.21.15.152:0/2780280109 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb284071fe0 0x7fb284159690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:44.721 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.719+0000 7fb28aa00640 1 -- 172.21.15.152:0/2780280109 >> 172.21.15.152:0/2780280109 conn(0x7fb284067cf0 msgr2=0x7fb28415a450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:44.721 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.719+0000 7fb28aa00640 1 -- 172.21.15.152:0/2780280109 shutdown_connections 2024-04-20T11:27:44.721 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.719+0000 7fb28aa00640 1 -- 172.21.15.152:0/2780280109 wait complete. 2024-04-20T11:27:44.738 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T11:26:47.246571+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T11:26:56.261424+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T11:26:57.079286+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T11:27:44.738 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T11:27:44.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.934+0000 7fa952a00640 1 Processor -- start 2024-04-20T11:27:44.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.934+0000 7fa952a00640 1 -- start start 2024-04-20T11:27:44.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.935+0000 7fa952a00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fa94c0b2880 0x7fa94c0b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:44.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.935+0000 7fa952a00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa94c1463e0 con 0x7fa94c0cd1b0 2024-04-20T11:27:44.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.936+0000 7fa952a00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa94c0b3e20 con 0x7fa94c146000 2024-04-20T11:27:44.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.936+0000 7fa952a00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fa94c0b4750 con 0x7fa94c0b2880 2024-04-20T11:27:44.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.936+0000 7fa950a00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fa94c0cd1b0 0x7fa94c0cd5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:40022/0 (socket says 172.21.15.152:40022) 2024-04-20T11:27:44.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.936+0000 7fa950a00640 1 -- 172.21.15.152:0/2979345857 learned_addr learned my addr 172.21.15.152:0/2979345857 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:27:44.938 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.936+0000 7fa94b400640 1 -- 172.21.15.152:0/2979345857 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3006334990 0 0) 0x7fa94c1463e0 con 0x7fa94c0cd1b0 2024-04-20T11:27:44.939 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.936+0000 7fa943e00640 1 --2- 172.21.15.152:0/2979345857 >> v2:172.21.15.175:3300/0 conn(0x7fa94c0b2880 0x7fa94c0b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:44.939 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.936+0000 7fa94b400640 1 -- 172.21.15.152:0/2979345857 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa930003520 con 0x7fa94c0cd1b0 2024-04-20T11:27:44.939 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.937+0000 7fa94b400640 1 -- 172.21.15.152:0/2979345857 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 566605393 0 0) 0x7fa930003520 con 0x7fa94c0cd1b0 2024-04-20T11:27:44.939 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.937+0000 7fa94b400640 1 -- 172.21.15.152:0/2979345857 >> v2:172.21.15.175:3300/0 conn(0x7fa94c0b2880 msgr2=0x7fa94c0b4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:44.939 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.937+0000 7fa94b400640 1 --2- 172.21.15.152:0/2979345857 >> v2:172.21.15.175:3300/0 conn(0x7fa94c0b2880 0x7fa94c0b4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:44.939 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.937+0000 7fa94b400640 1 -- 172.21.15.152:0/2979345857 >> v1:172.21.15.175:6789/0 conn(0x7fa94c146000 legacy=0x7fa94c0b2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:44.939 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.937+0000 7fa94b400640 1 -- 172.21.15.152:0/2979345857 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa94c0b4100 con 0x7fa94c0cd1b0 2024-04-20T11:27:44.939 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.937+0000 7fa94b400640 1 -- 172.21.15.152:0/2979345857 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7fa9440019f0 con 0x7fa94c0cd1b0 2024-04-20T11:27:44.939 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.937+0000 7fa94b400640 1 -- 172.21.15.152:0/2979345857 >> v1:172.21.15.152:6789/0 conn(0x7fa94c0cd1b0 legacy=0x7fa94c0cd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:44.939 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.937+0000 7fa94b400640 1 --2- 172.21.15.152:0/2979345857 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa930003a00 0x7fa930003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:44.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.937+0000 7fa94b400640 1 --2- 172.21.15.152:0/2979345857 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa930004800 0x7fa930004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:44.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.937+0000 7fa94b400640 1 --2- 172.21.15.152:0/2979345857 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa9300052e0 0x7fa93000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:44.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.937+0000 7fa94b400640 1 -- 172.21.15.152:0/2979345857 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fa94c0b4750 con 0x7fa9300052e0 2024-04-20T11:27:44.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.937+0000 7fa94b400640 1 -- 172.21.15.152:0/2979345857 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa93000c200 con 0x7fa930004800 2024-04-20T11:27:44.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.937+0000 7fa94b400640 1 -- 172.21.15.152:0/2979345857 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa93000c400 con 0x7fa930003a00 2024-04-20T11:27:44.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.937+0000 7fa950a00640 1 --2- 172.21.15.152:0/2979345857 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa9300052e0 0x7fa93000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:44.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.938+0000 7fa943e00640 1 --2- 172.21.15.152:0/2979345857 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa930004800 0x7fa930004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:44.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.938+0000 7fa94be00640 1 --2- 172.21.15.152:0/2979345857 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa930003a00 0x7fa930003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:44.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.938+0000 7fa950a00640 1 -- 172.21.15.152:0/2979345857 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa930003a00 msgr2=0x7fa930003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:44.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.938+0000 7fa950a00640 1 --2- 172.21.15.152:0/2979345857 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa930003a00 0x7fa930003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:44.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.938+0000 7fa950a00640 1 -- 172.21.15.152:0/2979345857 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa930004800 msgr2=0x7fa930004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:44.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.938+0000 7fa950a00640 1 --2- 172.21.15.152:0/2979345857 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa930004800 0x7fa930004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:44.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.938+0000 7fa950a00640 1 -- 172.21.15.152:0/2979345857 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa930003520 con 0x7fa9300052e0 2024-04-20T11:27:44.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.939+0000 7fa950a00640 1 -- 172.21.15.152:0/2979345857 reap_dead start 2024-04-20T11:27:44.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.939+0000 7fa950a00640 1 --2- 172.21.15.152:0/2979345857 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa9300052e0 0x7fa93000b9c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fa94400a790 tx=0x7fa930005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91c8762b386c8a82 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:44.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.939+0000 7fa94b400640 1 -- 172.21.15.152:0/2979345857 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa94400a3b0 con 0x7fa9300052e0 2024-04-20T11:27:44.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.939+0000 7fa94b400640 1 -- 172.21.15.152:0/2979345857 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa944034670 con 0x7fa9300052e0 2024-04-20T11:27:44.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.939+0000 7fa94b400640 1 -- 172.21.15.152:0/2979345857 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa944034a60 con 0x7fa9300052e0 2024-04-20T11:27:44.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.940+0000 7fa952a00640 1 -- 172.21.15.152:0/2979345857 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa9300052e0 msgr2=0x7fa93000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:44.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.940+0000 7fa952a00640 1 --2- 172.21.15.152:0/2979345857 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa9300052e0 0x7fa93000b9c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fa94400a790 tx=0x7fa930005830 comp rx=0 tx=0).stop 2024-04-20T11:27:44.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.940+0000 7fa952a00640 1 -- 172.21.15.152:0/2979345857 shutdown_connections 2024-04-20T11:27:44.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.940+0000 7fa952a00640 1 --2- 172.21.15.152:0/2979345857 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa9300052e0 0x7fa93000b9c0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:44.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.940+0000 7fa952a00640 1 -- 172.21.15.152:0/2979345857 >> 172.21.15.152:0/2979345857 conn(0x7fa94c06b980 msgr2=0x7fa94c06bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:44.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.940+0000 7fa952a00640 1 -- 172.21.15.152:0/2979345857 shutdown_connections 2024-04-20T11:27:44.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.940+0000 7fa952a00640 1 -- 172.21.15.152:0/2979345857 wait complete. 2024-04-20T11:27:44.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.942+0000 7fa952a00640 1 Processor -- start 2024-04-20T11:27:44.944 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.942+0000 7fa952a00640 1 -- start start 2024-04-20T11:27:44.945 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.943+0000 7fa952a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa9300052e0 0x7fa94c06e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:44.945 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.944+0000 7fa952a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa94c06dcc0 0x7fa94c0b0e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:44.945 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.944+0000 7fa952a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa94c141af0 0x7fa94c0b13d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:44.945 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.944+0000 7fa952a00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fa94c0b4d60 con 0x7fa9300052e0 2024-04-20T11:27:44.945 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.944+0000 7fa952a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa94c0b4be0 con 0x7fa94c141af0 2024-04-20T11:27:44.945 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.944+0000 7fa952a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa94c0b4ee0 con 0x7fa94c06dcc0 2024-04-20T11:27:44.945 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.944+0000 7fa94be00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa9300052e0 0x7fa94c06e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:44.945 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.944+0000 7fa94be00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa9300052e0 0x7fa94c06e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35456/0 (socket says 172.21.15.152:35456) 2024-04-20T11:27:44.945 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.944+0000 7fa94be00640 1 -- 172.21.15.152:0/2023655676 learned_addr learned my addr 172.21.15.152:0/2023655676 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:44.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.944+0000 7fa94be00640 1 -- 172.21.15.152:0/2023655676 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa94c06dcc0 msgr2=0x7fa94c0b0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:44.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.944+0000 7fa94be00640 1 --2- 172.21.15.152:0/2023655676 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa94c06dcc0 0x7fa94c0b0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:44.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.944+0000 7fa94be00640 1 -- 172.21.15.152:0/2023655676 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa94c141af0 msgr2=0x7fa94c0b13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:44.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.944+0000 7fa94be00640 1 --2- 172.21.15.152:0/2023655676 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa94c141af0 0x7fa94c0b13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:44.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.944+0000 7fa94be00640 1 -- 172.21.15.152:0/2023655676 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa94c1f81e0 con 0x7fa9300052e0 2024-04-20T11:27:44.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.945+0000 7fa94be00640 1 --2- 172.21.15.152:0/2023655676 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa9300052e0 0x7fa94c06e360 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa93c00e4d0 tx=0x7fa93c00e990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:44.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.945+0000 7fa948c00640 1 -- 172.21.15.152:0/2023655676 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa93c0052e0 con 0x7fa9300052e0 2024-04-20T11:27:44.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.945+0000 7fa948c00640 1 -- 172.21.15.152:0/2023655676 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa93c018070 con 0x7fa9300052e0 2024-04-20T11:27:44.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.945+0000 7fa948c00640 1 -- 172.21.15.152:0/2023655676 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa93c003810 con 0x7fa9300052e0 2024-04-20T11:27:44.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.945+0000 7fa952a00640 1 -- 172.21.15.152:0/2023655676 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa94c1f84c0 con 0x7fa9300052e0 2024-04-20T11:27:44.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.945+0000 7fa952a00640 1 -- 172.21.15.152:0/2023655676 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa94c1f8a80 con 0x7fa9300052e0 2024-04-20T11:27:44.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.946+0000 7fa952a00640 1 -- 172.21.15.152:0/2023655676 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa94c0689f0 con 0x7fa9300052e0 2024-04-20T11:27:44.948 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.947+0000 7fa948c00640 1 -- 172.21.15.152:0/2023655676 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa93c0039b0 con 0x7fa9300052e0 2024-04-20T11:27:44.948 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.948+0000 7fa948c00640 1 --2- 172.21.15.152:0/2023655676 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fa92806c920 0x7fa92806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:44.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.948+0000 7fa948c00640 1 -- 172.21.15.152:0/2023655676 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fa93c08b890 con 0x7fa9300052e0 2024-04-20T11:27:44.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.948+0000 7fa943e00640 1 --2- 172.21.15.152:0/2023655676 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fa92806c920 0x7fa92806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:44.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.949+0000 7fa943e00640 1 --2- 172.21.15.152:0/2023655676 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fa92806c920 0x7fa92806ede0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa94c13cf60 tx=0x7fa93400d280 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:44.953 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:44.952+0000 7fa948c00640 1 -- 172.21.15.152:0/2023655676 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fa93c059c30 con 0x7fa9300052e0 2024-04-20T11:27:45.138 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.138+0000 7fa952a00640 1 -- 172.21.15.152:0/2023655676 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa94c0ad8b0 con 0x7fa9300052e0 2024-04-20T11:27:45.139 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.139+0000 7fa948c00640 1 -- 172.21.15.152:0/2023655676 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) ==== 93+0+10 (secure 0 0 0) 0x7fa93c059530 con 0x7fa9300052e0 2024-04-20T11:27:45.139 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 1 2024-04-20T11:27:45.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.141+0000 7fa952a00640 1 -- 172.21.15.152:0/2023655676 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fa92806c920 msgr2=0x7fa92806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:45.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.141+0000 7fa952a00640 1 --2- 172.21.15.152:0/2023655676 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fa92806c920 0x7fa92806ede0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa94c13cf60 tx=0x7fa93400d280 comp rx=0 tx=0).stop 2024-04-20T11:27:45.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.141+0000 7fa952a00640 1 -- 172.21.15.152:0/2023655676 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa9300052e0 msgr2=0x7fa94c06e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:45.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.141+0000 7fa952a00640 1 --2- 172.21.15.152:0/2023655676 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa9300052e0 0x7fa94c06e360 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa93c00e4d0 tx=0x7fa93c00e990 comp rx=0 tx=0).stop 2024-04-20T11:27:45.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.141+0000 7fa952a00640 1 -- 172.21.15.152:0/2023655676 shutdown_connections 2024-04-20T11:27:45.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.141+0000 7fa952a00640 1 --2- 172.21.15.152:0/2023655676 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fa92806c920 0x7fa92806ede0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:45.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.141+0000 7fa952a00640 1 --2- 172.21.15.152:0/2023655676 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa94c141af0 0x7fa94c0b13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:45.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.141+0000 7fa952a00640 1 --2- 172.21.15.152:0/2023655676 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa94c06dcc0 0x7fa94c0b0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:45.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.141+0000 7fa952a00640 1 --2- 172.21.15.152:0/2023655676 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa9300052e0 0x7fa94c06e360 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:45.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.141+0000 7fa952a00640 1 -- 172.21.15.152:0/2023655676 >> 172.21.15.152:0/2023655676 conn(0x7fa94c06b980 msgr2=0x7fa94c0d03f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:45.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.141+0000 7fa952a00640 1 -- 172.21.15.152:0/2023655676 shutdown_connections 2024-04-20T11:27:45.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.141+0000 7fa952a00640 1 -- 172.21.15.152:0/2023655676 wait complete. 2024-04-20T11:27:45.160 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T11:27:45.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.363+0000 7f5018a00640 1 Processor -- start 2024-04-20T11:27:45.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.363+0000 7f5018a00640 1 -- start start 2024-04-20T11:27:45.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.364+0000 7f5018a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f501415e9c0 0x7f5014162530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:45.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.364+0000 7f5018a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f50140ba560 0x7f50140d1270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:45.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.364+0000 7f5018a00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f50140d17b0 0x7f50140b62e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:45.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.364+0000 7f5018a00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f5014163460 con 0x7f50140ba560 2024-04-20T11:27:45.366 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.364+0000 7f5018a00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f50140b7920 con 0x7f50140d17b0 2024-04-20T11:27:45.366 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.364+0000 7f5018a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5014162ea0 con 0x7f501415e9c0 2024-04-20T11:27:45.366 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.365+0000 7f5009400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f50140ba560 0x7f50140d1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:45.366 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.365+0000 7f5009400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f50140ba560 0x7f50140d1270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35458/0 (socket says 172.21.15.152:35458) 2024-04-20T11:27:45.366 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.365+0000 7f5009400640 1 -- 172.21.15.152:0/1319013115 learned_addr learned my addr 172.21.15.152:0/1319013115 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:45.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.365+0000 7f5012800640 1 --2- 172.21.15.152:0/1319013115 >> v2:172.21.15.175:3300/0 conn(0x7f50140d17b0 0x7f50140b62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:45.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.365+0000 7f5011e00640 1 --2- 172.21.15.152:0/1319013115 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f501415e9c0 0x7f5014162530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:45.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.365+0000 7f5009400640 1 -- 172.21.15.152:0/1319013115 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f501415e9c0 msgr2=0x7f5014162530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:45.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.365+0000 7f5009400640 1 --2- 172.21.15.152:0/1319013115 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f501415e9c0 0x7f5014162530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:45.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.365+0000 7f5009400640 1 -- 172.21.15.152:0/1319013115 >> v2:172.21.15.175:3300/0 conn(0x7f50140d17b0 msgr2=0x7f50140b62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:45.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.365+0000 7f5009400640 1 --2- 172.21.15.152:0/1319013115 >> v2:172.21.15.175:3300/0 conn(0x7f50140d17b0 0x7f50140b62e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:45.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.365+0000 7f5009400640 1 -- 172.21.15.152:0/1319013115 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5014163180 con 0x7f50140ba560 2024-04-20T11:27:45.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.365+0000 7f5012800640 1 --2- 172.21.15.152:0/1319013115 >> v2:172.21.15.175:3300/0 conn(0x7f50140d17b0 0x7f50140b62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:27:45.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.367+0000 7f5009400640 1 --2- 172.21.15.152:0/1319013115 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f50140ba560 0x7f50140d1270 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f50140cf080 tx=0x7f4ffc02b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3063cc518e6778fe server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:45.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.367+0000 7f5011400640 1 -- 172.21.15.152:0/1319013115 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ffc0317d0 con 0x7f50140ba560 2024-04-20T11:27:45.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.367+0000 7f5011400640 1 -- 172.21.15.152:0/1319013115 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ffc037070 con 0x7f50140ba560 2024-04-20T11:27:45.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.367+0000 7f5011400640 1 -- 172.21.15.152:0/1319013115 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ffc031b90 con 0x7f50140ba560 2024-04-20T11:27:45.369 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.367+0000 7f5018a00640 1 -- 172.21.15.152:0/1319013115 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f50140ba560 msgr2=0x7f50140d1270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:45.369 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.367+0000 7f5018a00640 1 --2- 172.21.15.152:0/1319013115 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f50140ba560 0x7f50140d1270 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f50140cf080 tx=0x7f4ffc02b870 comp rx=0 tx=0).stop 2024-04-20T11:27:45.369 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.368+0000 7f5018a00640 1 -- 172.21.15.152:0/1319013115 shutdown_connections 2024-04-20T11:27:45.369 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.368+0000 7f5018a00640 1 --2- 172.21.15.152:0/1319013115 >> v2:172.21.15.175:3300/0 conn(0x7f50140d17b0 0x7f50140b62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:45.369 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.368+0000 7f5018a00640 1 --2- 172.21.15.152:0/1319013115 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f50140ba560 0x7f50140d1270 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:45.369 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.368+0000 7f5018a00640 1 --2- 172.21.15.152:0/1319013115 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f501415e9c0 0x7f5014162530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:45.369 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.368+0000 7f5018a00640 1 -- 172.21.15.152:0/1319013115 >> 172.21.15.152:0/1319013115 conn(0x7f501406cc40 msgr2=0x7f501406d050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:45.369 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.368+0000 7f5018a00640 1 -- 172.21.15.152:0/1319013115 shutdown_connections 2024-04-20T11:27:45.369 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.368+0000 7f5018a00640 1 -- 172.21.15.152:0/1319013115 wait complete. 2024-04-20T11:27:45.370 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.370+0000 7f5018a00640 1 Processor -- start 2024-04-20T11:27:45.370 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.370+0000 7f5018a00640 1 -- start start 2024-04-20T11:27:45.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.371+0000 7f5018a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f50140ba560 0x7f501415f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:45.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.371+0000 7f5018a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f50140d17b0 0x7f501415ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:45.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.371+0000 7f5018a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f501415e9c0 0x7f501415cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:45.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.371+0000 7f5018a00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f501415eda0 con 0x7f50140ba560 2024-04-20T11:27:45.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.371+0000 7f5018a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f50140d2f70 con 0x7f501415e9c0 2024-04-20T11:27:45.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.371+0000 7f5018a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5014162a70 con 0x7f50140d17b0 2024-04-20T11:27:45.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.371+0000 7f5011e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f50140ba560 0x7f501415f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:45.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.371+0000 7f5011e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f50140ba560 0x7f501415f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35462/0 (socket says 172.21.15.152:35462) 2024-04-20T11:27:45.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.371+0000 7f5011e00640 1 -- 172.21.15.152:0/2636763695 learned_addr learned my addr 172.21.15.152:0/2636763695 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:45.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.371+0000 7f5009400640 1 --2- 172.21.15.152:0/2636763695 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f50140d17b0 0x7f501415ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:45.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.371+0000 7f5011e00640 1 -- 172.21.15.152:0/2636763695 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f50140d17b0 msgr2=0x7f501415ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:45.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.371+0000 7f5012800640 1 --2- 172.21.15.152:0/2636763695 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f501415e9c0 0x7f501415cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:45.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.371+0000 7f5011e00640 1 --2- 172.21.15.152:0/2636763695 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f50140d17b0 0x7f501415ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:45.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.371+0000 7f5011e00640 1 -- 172.21.15.152:0/2636763695 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f501415e9c0 msgr2=0x7f501415cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:45.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.372+0000 7f5011e00640 1 --2- 172.21.15.152:0/2636763695 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f501415e9c0 0x7f501415cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:45.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.372+0000 7f5011e00640 1 -- 172.21.15.152:0/2636763695 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f501415d810 con 0x7f50140ba560 2024-04-20T11:27:45.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.372+0000 7f5009400640 1 --2- 172.21.15.152:0/2636763695 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f50140d17b0 0x7f501415ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:27:45.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.372+0000 7f5011e00640 1 --2- 172.21.15.152:0/2636763695 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f50140ba560 0x7f501415f440 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f500400d5f0 tx=0x7f500400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:45.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.372+0000 7f500aa00640 1 -- 172.21.15.152:0/2636763695 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50040125f0 con 0x7f50140ba560 2024-04-20T11:27:45.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.372+0000 7f500aa00640 1 -- 172.21.15.152:0/2636763695 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5004019070 con 0x7f50140ba560 2024-04-20T11:27:45.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.372+0000 7f5018a00640 1 -- 172.21.15.152:0/2636763695 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5014153c30 con 0x7f50140ba560 2024-04-20T11:27:45.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.372+0000 7f500aa00640 1 -- 172.21.15.152:0/2636763695 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50040129d0 con 0x7f50140ba560 2024-04-20T11:27:45.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.372+0000 7f5018a00640 1 -- 172.21.15.152:0/2636763695 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50141541f0 con 0x7f50140ba560 2024-04-20T11:27:45.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.373+0000 7f5018a00640 1 -- 172.21.15.152:0/2636763695 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5014153a20 con 0x7f50140ba560 2024-04-20T11:27:45.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.374+0000 7f500aa00640 1 -- 172.21.15.152:0/2636763695 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5004012b70 con 0x7f50140ba560 2024-04-20T11:27:45.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.375+0000 7f500aa00640 1 --2- 172.21.15.152:0/2636763695 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f4ff006c8f0 0x7f4ff006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:45.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.375+0000 7f500aa00640 1 -- 172.21.15.152:0/2636763695 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f500408c0c0 con 0x7f50140ba560 2024-04-20T11:27:45.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.375+0000 7f5009400640 1 --2- 172.21.15.152:0/2636763695 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f4ff006c8f0 0x7f4ff006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:45.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.376+0000 7f5009400640 1 --2- 172.21.15.152:0/2636763695 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f4ff006c8f0 0x7f4ff006edb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc036000 tx=0x7f4ffc008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:45.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.380+0000 7f500aa00640 1 -- 172.21.15.152:0/2636763695 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f500405a3d0 con 0x7f50140ba560 2024-04-20T11:27:45.580 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.579+0000 7f5018a00640 1 -- 172.21.15.152:0/2636763695 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f50140689f0 con 0x7f50140ba560 2024-04-20T11:27:45.580 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.580+0000 7f500aa00640 1 -- 172.21.15.152:0/2636763695 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) ==== 104+0+11 (secure 0 0 0) 0x7f5004059cd0 con 0x7f50140ba560 2024-04-20T11:27:45.581 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 32 2024-04-20T11:27:45.583 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.582+0000 7f5018a00640 1 -- 172.21.15.152:0/2636763695 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f4ff006c8f0 msgr2=0x7f4ff006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:45.583 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.582+0000 7f5018a00640 1 --2- 172.21.15.152:0/2636763695 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f4ff006c8f0 0x7f4ff006edb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc036000 tx=0x7f4ffc008030 comp rx=0 tx=0).stop 2024-04-20T11:27:45.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.582+0000 7f5018a00640 1 -- 172.21.15.152:0/2636763695 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f50140ba560 msgr2=0x7f501415f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:45.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.582+0000 7f5018a00640 1 --2- 172.21.15.152:0/2636763695 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f50140ba560 0x7f501415f440 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f500400d5f0 tx=0x7f500400dab0 comp rx=0 tx=0).stop 2024-04-20T11:27:45.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.582+0000 7f5018a00640 1 -- 172.21.15.152:0/2636763695 shutdown_connections 2024-04-20T11:27:45.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.583+0000 7f5018a00640 1 --2- 172.21.15.152:0/2636763695 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f4ff006c8f0 0x7f4ff006edb0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:45.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.583+0000 7f5018a00640 1 --2- 172.21.15.152:0/2636763695 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f501415e9c0 0x7f501415cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:45.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.583+0000 7f5018a00640 1 --2- 172.21.15.152:0/2636763695 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f50140d17b0 0x7f501415ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:45.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.583+0000 7f5018a00640 1 --2- 172.21.15.152:0/2636763695 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f50140ba560 0x7f501415f440 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:45.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.583+0000 7f5018a00640 1 -- 172.21.15.152:0/2636763695 >> 172.21.15.152:0/2636763695 conn(0x7f501406cc40 msgr2=0x7f5014159990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:45.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.583+0000 7f5018a00640 1 -- 172.21.15.152:0/2636763695 shutdown_connections 2024-04-20T11:27:45.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.583+0000 7f5018a00640 1 -- 172.21.15.152:0/2636763695 wait complete. 2024-04-20T11:27:45.603 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T11:27:45.800 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.799+0000 7f8631600640 1 Processor -- start 2024-04-20T11:27:45.800 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.799+0000 7f8631600640 1 -- start start 2024-04-20T11:27:45.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.801+0000 7f8631600640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f862c15e5d0 0x7f862c162440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:45.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.801+0000 7f8631600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f862c0afba0 0x7f862c0ae110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:45.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.801+0000 7f8631600640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f862c0adc30 con 0x7f862c0b3fa0 2024-04-20T11:27:45.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.801+0000 7f8631600640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f862c162a70 con 0x7f862c15e5d0 2024-04-20T11:27:45.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.801+0000 7f8631600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f862c162d50 con 0x7f862c0afba0 2024-04-20T11:27:45.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.801+0000 7f862aa00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f862c15e5d0 0x7f862c162440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:45.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.802+0000 7f862a000640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f862c0b3fa0 0x7f862c0af490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:44838/0 (socket says 172.21.15.152:44838) 2024-04-20T11:27:45.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.802+0000 7f862a000640 1 -- 172.21.15.152:0/3014194616 learned_addr learned my addr 172.21.15.152:0/3014194616 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:27:45.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.802+0000 7f862b400640 1 --2- 172.21.15.152:0/3014194616 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f862c0afba0 0x7f862c0ae110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:45.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.802+0000 7f862aa00640 1 -- 172.21.15.152:0/3014194616 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f862c0afba0 msgr2=0x7f862c0ae110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:45.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.802+0000 7f862aa00640 1 --2- 172.21.15.152:0/3014194616 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f862c0afba0 0x7f862c0ae110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:45.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.802+0000 7f862aa00640 1 -- 172.21.15.152:0/3014194616 >> v1:172.21.15.152:6789/0 conn(0x7f862c0b3fa0 legacy=0x7f862c0af490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:45.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.802+0000 7f862aa00640 1 -- 172.21.15.152:0/3014194616 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f862c163030 con 0x7f862c15e5d0 2024-04-20T11:27:45.805 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.803+0000 7f862aa00640 1 --2- 172.21.15.152:0/3014194616 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f862c15e5d0 0x7f862c162440 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f861c0097a0 tx=0x7f861c02b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ee61b652e893d30 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:45.805 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.803+0000 7f8629600640 1 -- 172.21.15.152:0/3014194616 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f861c035020 con 0x7f862c15e5d0 2024-04-20T11:27:45.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.804+0000 7f8629600640 1 -- 172.21.15.152:0/3014194616 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f861c031810 con 0x7f862c15e5d0 2024-04-20T11:27:45.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.804+0000 7f8629600640 1 -- 172.21.15.152:0/3014194616 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f861c031be0 con 0x7f862c15e5d0 2024-04-20T11:27:45.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.804+0000 7f8631600640 1 -- 172.21.15.152:0/3014194616 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f862c15e5d0 msgr2=0x7f862c162440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:45.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.804+0000 7f8631600640 1 --2- 172.21.15.152:0/3014194616 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f862c15e5d0 0x7f862c162440 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f861c0097a0 tx=0x7f861c02b8b0 comp rx=0 tx=0).stop 2024-04-20T11:27:45.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.804+0000 7f8631600640 1 -- 172.21.15.152:0/3014194616 shutdown_connections 2024-04-20T11:27:45.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.804+0000 7f8631600640 1 --2- 172.21.15.152:0/3014194616 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f862c0afba0 0x7f862c0ae110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:45.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.804+0000 7f8631600640 1 --2- 172.21.15.152:0/3014194616 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f862c15e5d0 0x7f862c162440 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:45.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.804+0000 7f8631600640 1 -- 172.21.15.152:0/3014194616 >> 172.21.15.152:0/3014194616 conn(0x7f862c06b980 msgr2=0x7f862c06bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:45.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.804+0000 7f8631600640 1 -- 172.21.15.152:0/3014194616 shutdown_connections 2024-04-20T11:27:45.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.804+0000 7f8631600640 1 -- 172.21.15.152:0/3014194616 wait complete. 2024-04-20T11:27:45.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.806+0000 7f8631600640 1 Processor -- start 2024-04-20T11:27:45.809 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.806+0000 7f8631600640 1 -- start start 2024-04-20T11:27:45.809 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.807+0000 7f8631600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f862c0afba0 0x7f862c150e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:45.809 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.807+0000 7f8631600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f862c0b3fa0 0x7f862c152510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:45.809 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.807+0000 7f8631600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f862c15e5d0 0x7f862c14ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:45.809 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.807+0000 7f8631600640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f862c06e0a0 con 0x7f862c15e5d0 2024-04-20T11:27:45.809 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.807+0000 7f8631600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f862c163310 con 0x7f862c0b3fa0 2024-04-20T11:27:45.809 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.807+0000 7f8631600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f862c06e220 con 0x7f862c0afba0 2024-04-20T11:27:45.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.807+0000 7f862b400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f862c15e5d0 0x7f862c14ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:45.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.807+0000 7f862b400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f862c15e5d0 0x7f862c14ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43186/0 (socket says 172.21.15.152:43186) 2024-04-20T11:27:45.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.807+0000 7f862b400640 1 -- 172.21.15.152:0/1534159079 learned_addr learned my addr 172.21.15.152:0/1534159079 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:45.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.808+0000 7f862aa00640 1 --2- 172.21.15.152:0/1534159079 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f862c0afba0 0x7f862c150e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:45.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.808+0000 7f862a000640 1 --2- 172.21.15.152:0/1534159079 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f862c0b3fa0 0x7f862c152510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:45.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.808+0000 7f862b400640 1 -- 172.21.15.152:0/1534159079 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f862c0afba0 msgr2=0x7f862c150e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:45.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.808+0000 7f862b400640 1 --2- 172.21.15.152:0/1534159079 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f862c0afba0 0x7f862c150e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:45.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.808+0000 7f862b400640 1 -- 172.21.15.152:0/1534159079 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f862c0b3fa0 msgr2=0x7f862c152510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:45.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.808+0000 7f862b400640 1 --2- 172.21.15.152:0/1534159079 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f862c0b3fa0 0x7f862c152510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:45.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.808+0000 7f862b400640 1 -- 172.21.15.152:0/1534159079 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f862c152a50 con 0x7f862c15e5d0 2024-04-20T11:27:45.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.808+0000 7f862a000640 1 --2- 172.21.15.152:0/1534159079 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f862c0b3fa0 0x7f862c152510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:27:45.812 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.808+0000 7f862b400640 1 --2- 172.21.15.152:0/1534159079 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f862c15e5d0 0x7f862c14ed40 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f862400d800 tx=0x7f862400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:45.812 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.808+0000 7f8622a00640 1 -- 172.21.15.152:0/1534159079 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f862400b6c0 con 0x7f862c15e5d0 2024-04-20T11:27:45.812 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.809+0000 7f8622a00640 1 -- 172.21.15.152:0/1534159079 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8624019070 con 0x7f862c15e5d0 2024-04-20T11:27:45.812 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.809+0000 7f8622a00640 1 -- 172.21.15.152:0/1534159079 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f862400bac0 con 0x7f862c15e5d0 2024-04-20T11:27:45.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.809+0000 7f8631600640 1 -- 172.21.15.152:0/1534159079 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f862c14f280 con 0x7f862c15e5d0 2024-04-20T11:27:45.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.809+0000 7f8631600640 1 -- 172.21.15.152:0/1534159079 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f862c150080 con 0x7f862c15e5d0 2024-04-20T11:27:45.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.810+0000 7f8631600640 1 -- 172.21.15.152:0/1534159079 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f862c0689f0 con 0x7f862c15e5d0 2024-04-20T11:27:45.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.811+0000 7f8622a00640 1 -- 172.21.15.152:0/1534159079 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f862400bc60 con 0x7f862c15e5d0 2024-04-20T11:27:45.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.812+0000 7f8622a00640 1 --2- 172.21.15.152:0/1534159079 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f860806c8f0 0x7f860806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:45.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.812+0000 7f8622a00640 1 -- 172.21.15.152:0/1534159079 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f862408bff0 con 0x7f862c15e5d0 2024-04-20T11:27:45.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.813+0000 7f862aa00640 1 --2- 172.21.15.152:0/1534159079 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f860806c8f0 0x7f860806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:45.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.813+0000 7f862aa00640 1 --2- 172.21.15.152:0/1534159079 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f860806c8f0 0x7f860806edb0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f861c036000 tx=0x7f861c03c040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:45.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:45.817+0000 7f8622a00640 1 -- 172.21.15.152:0/1534159079 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f862c152a50 con 0x7f862c15e5d0 2024-04-20T11:27:46.013 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.012+0000 7f8631600640 1 -- 172.21.15.152:0/1534159079 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f862c0b16a0 con 0x7f862c15e5d0 2024-04-20T11:27:46.014 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.013+0000 7f8622a00640 1 -- 172.21.15.152:0/1534159079 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) ==== 100+0+11 (secure 0 0 0) 0x7f862405a3f0 con 0x7f862c15e5d0 2024-04-20T11:27:46.014 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 64 2024-04-20T11:27:46.017 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.016+0000 7f8631600640 1 -- 172.21.15.152:0/1534159079 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f860806c8f0 msgr2=0x7f860806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:46.017 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.016+0000 7f8631600640 1 --2- 172.21.15.152:0/1534159079 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f860806c8f0 0x7f860806edb0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f861c036000 tx=0x7f861c03c040 comp rx=0 tx=0).stop 2024-04-20T11:27:46.017 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.016+0000 7f8631600640 1 -- 172.21.15.152:0/1534159079 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f862c15e5d0 msgr2=0x7f862c14ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:46.018 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.016+0000 7f8631600640 1 --2- 172.21.15.152:0/1534159079 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f862c15e5d0 0x7f862c14ed40 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f862400d800 tx=0x7f862400dcc0 comp rx=0 tx=0).stop 2024-04-20T11:27:46.018 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.016+0000 7f8631600640 1 -- 172.21.15.152:0/1534159079 shutdown_connections 2024-04-20T11:27:46.018 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.016+0000 7f8631600640 1 --2- 172.21.15.152:0/1534159079 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f860806c8f0 0x7f860806edb0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:46.018 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.016+0000 7f8631600640 1 --2- 172.21.15.152:0/1534159079 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f862c15e5d0 0x7f862c14ed40 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:46.018 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.016+0000 7f8631600640 1 --2- 172.21.15.152:0/1534159079 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f862c0b3fa0 0x7f862c152510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:46.018 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.016+0000 7f8631600640 1 --2- 172.21.15.152:0/1534159079 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f862c0afba0 0x7f862c150e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:46.018 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.016+0000 7f8631600640 1 -- 172.21.15.152:0/1534159079 >> 172.21.15.152:0/1534159079 conn(0x7f862c06b980 msgr2=0x7f862c0b2180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:46.018 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.016+0000 7f8631600640 1 -- 172.21.15.152:0/1534159079 shutdown_connections 2024-04-20T11:27:46.019 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.016+0000 7f8631600640 1 -- 172.21.15.152:0/1534159079 wait complete. 2024-04-20T11:27:46.036 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-20T11:27:46.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.235+0000 7f51ff200640 1 Processor -- start 2024-04-20T11:27:46.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.235+0000 7f51ff200640 1 -- start start 2024-04-20T11:27:46.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.237+0000 7f51ff200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5200159040 0x7f52000cf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:46.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.237+0000 7f51ff200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f520006f6a0 0x7f52000ce140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:46.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.237+0000 7f51ff200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5200159420 con 0x7f5200162530 2024-04-20T11:27:46.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.237+0000 7f51ff200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f52000ce680 con 0x7f520006f6a0 2024-04-20T11:27:46.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.237+0000 7f51ff200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f520006fb00 con 0x7f5200159040 2024-04-20T11:27:46.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.237+0000 7f51fd400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f520006f6a0 0x7f52000ce140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:46.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.237+0000 7f51fd400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f520006f6a0 0x7f52000ce140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43190/0 (socket says 172.21.15.152:43190) 2024-04-20T11:27:46.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.237+0000 7f51fd400640 1 -- 172.21.15.152:0/3953109551 learned_addr learned my addr 172.21.15.152:0/3953109551 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:46.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.238+0000 7f51fde00640 1 --2- 172.21.15.152:0/3953109551 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5200159040 0x7f52000cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:46.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.238+0000 7f51fd400640 1 -- 172.21.15.152:0/3953109551 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5200159040 msgr2=0x7f52000cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:46.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.238+0000 7f51fd400640 1 --2- 172.21.15.152:0/3953109551 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5200159040 0x7f52000cf950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:46.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.238+0000 7f51fd400640 1 -- 172.21.15.152:0/3953109551 >> v1:172.21.15.175:6789/0 conn(0x7f5200162530 legacy=0x7f52000b32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:46.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.238+0000 7f51fd400640 1 -- 172.21.15.152:0/3953109551 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52000ce8b0 con 0x7f520006f6a0 2024-04-20T11:27:46.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.239+0000 7f51fd400640 1 --2- 172.21.15.152:0/3953109551 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f520006f6a0 0x7f52000ce140 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f51e8009870 tx=0x7f51e802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c72461592e79e52 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:46.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.239+0000 7f51fca00640 1 -- 172.21.15.152:0/3953109551 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51e8035020 con 0x7f520006f6a0 2024-04-20T11:27:46.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.239+0000 7f51fca00640 1 -- 172.21.15.152:0/3953109551 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51e8031640 con 0x7f520006f6a0 2024-04-20T11:27:46.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.239+0000 7f51fca00640 1 -- 172.21.15.152:0/3953109551 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51e802d040 con 0x7f520006f6a0 2024-04-20T11:27:46.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.240+0000 7f51ff200640 1 -- 172.21.15.152:0/3953109551 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f520006f6a0 msgr2=0x7f52000ce140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:46.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.240+0000 7f51ff200640 1 --2- 172.21.15.152:0/3953109551 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f520006f6a0 0x7f52000ce140 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f51e8009870 tx=0x7f51e802b8f0 comp rx=0 tx=0).stop 2024-04-20T11:27:46.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.240+0000 7f51ff200640 1 -- 172.21.15.152:0/3953109551 shutdown_connections 2024-04-20T11:27:46.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.240+0000 7f51ff200640 1 --2- 172.21.15.152:0/3953109551 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f520006f6a0 0x7f52000ce140 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:46.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.240+0000 7f51ff200640 1 --2- 172.21.15.152:0/3953109551 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5200159040 0x7f52000cf950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:46.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.240+0000 7f51ff200640 1 -- 172.21.15.152:0/3953109551 >> 172.21.15.152:0/3953109551 conn(0x7f52000d0260 msgr2=0x7f52000d0670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:46.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.241+0000 7f51ff200640 1 -- 172.21.15.152:0/3953109551 shutdown_connections 2024-04-20T11:27:46.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.241+0000 7f51ff200640 1 -- 172.21.15.152:0/3953109551 wait complete. 2024-04-20T11:27:46.243 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.243+0000 7f51ff200640 1 Processor -- start 2024-04-20T11:27:46.243 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.243+0000 7f51ff200640 1 -- start start 2024-04-20T11:27:46.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.244+0000 7f51ff200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f520006f6a0 0x7f5200146820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:46.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.244+0000 7f51ff200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5200159040 0x7f5200072f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:46.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.244+0000 7f51ff200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5200162530 0x7f5200073490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:46.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.244+0000 7f51ff200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f52000cf150 con 0x7f5200159040 2024-04-20T11:27:46.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.244+0000 7f51ff200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f52000cee70 con 0x7f5200162530 2024-04-20T11:27:46.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.244+0000 7f51ff200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f52000cf430 con 0x7f520006f6a0 2024-04-20T11:27:46.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.245+0000 7f51fd400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5200159040 0x7f5200072f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:46.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.245+0000 7f51fd400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5200159040 0x7f5200072f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43194/0 (socket says 172.21.15.152:43194) 2024-04-20T11:27:46.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.245+0000 7f51fd400640 1 -- 172.21.15.152:0/4103359267 learned_addr learned my addr 172.21.15.152:0/4103359267 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:46.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.245+0000 7f51fde00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f520006f6a0 0x7f5200146820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:46.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.245+0000 7f51fe800640 1 --2- 172.21.15.152:0/4103359267 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5200162530 0x7f5200073490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:46.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.245+0000 7f51fd400640 1 -- 172.21.15.152:0/4103359267 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f520006f6a0 msgr2=0x7f5200146820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:46.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.245+0000 7f51fd400640 1 --2- 172.21.15.152:0/4103359267 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f520006f6a0 0x7f5200146820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:46.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.245+0000 7f51fd400640 1 -- 172.21.15.152:0/4103359267 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5200162530 msgr2=0x7f5200073490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:46.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.245+0000 7f51fd400640 1 --2- 172.21.15.152:0/4103359267 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5200162530 0x7f5200073490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:46.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.245+0000 7f51fd400640 1 -- 172.21.15.152:0/4103359267 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52001569b0 con 0x7f5200159040 2024-04-20T11:27:46.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.246+0000 7f51fd400640 1 --2- 172.21.15.152:0/4103359267 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5200159040 0x7f5200072f50 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f51e802b930 tx=0x7f51e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:46.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.246+0000 7f51f6000640 1 -- 172.21.15.152:0/4103359267 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51e8040070 con 0x7f5200159040 2024-04-20T11:27:46.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.246+0000 7f51f6000640 1 -- 172.21.15.152:0/4103359267 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51e80399f0 con 0x7f5200159040 2024-04-20T11:27:46.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.246+0000 7f51ff200640 1 -- 172.21.15.152:0/4103359267 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5200155f80 con 0x7f5200159040 2024-04-20T11:27:46.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.246+0000 7f51f6000640 1 -- 172.21.15.152:0/4103359267 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51e8035040 con 0x7f5200159040 2024-04-20T11:27:46.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.246+0000 7f51ff200640 1 -- 172.21.15.152:0/4103359267 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5200156260 con 0x7f5200159040 2024-04-20T11:27:46.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.247+0000 7f51ff200640 1 -- 172.21.15.152:0/4103359267 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5200075e60 con 0x7f5200159040 2024-04-20T11:27:46.249 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.248+0000 7f51f6000640 1 -- 172.21.15.152:0/4103359267 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f51e804b060 con 0x7f5200159040 2024-04-20T11:27:46.250 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.250+0000 7f51f6000640 1 --2- 172.21.15.152:0/4103359267 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f51dc06c8f0 0x7f51dc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:46.250 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.250+0000 7f51f6000640 1 -- 172.21.15.152:0/4103359267 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f51e80ad3b0 con 0x7f5200159040 2024-04-20T11:27:46.250 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.250+0000 7f51fde00640 1 --2- 172.21.15.152:0/4103359267 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f51dc06c8f0 0x7f51dc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:46.251 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.251+0000 7f51fde00640 1 --2- 172.21.15.152:0/4103359267 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f51dc06c8f0 0x7f51dc06edb0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f51f00079a0 tx=0x7f51f0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:46.255 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.254+0000 7f51f6000640 1 -- 172.21.15.152:0/4103359267 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f5200159420 con 0x7f5200159040 2024-04-20T11:27:46.468 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.467+0000 7f51ff200640 1 -- 172.21.15.152:0/4103359267 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f52000aeb90 con 0x7f5200159040 2024-04-20T11:27:46.468 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.468+0000 7f51f6000640 1 -- 172.21.15.152:0/4103359267 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) ==== 80+0+0 (secure 0 0 0) 0x7f51e807b100 con 0x7f5200159040 2024-04-20T11:27:46.469 INFO:teuthology.orchestra.run.smithi152.stderr:listed 0 entries 2024-04-20T11:27:46.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.471+0000 7f51ff200640 1 -- 172.21.15.152:0/4103359267 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f51dc06c8f0 msgr2=0x7f51dc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:46.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.471+0000 7f51ff200640 1 --2- 172.21.15.152:0/4103359267 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f51dc06c8f0 0x7f51dc06edb0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f51f00079a0 tx=0x7f51f0008040 comp rx=0 tx=0).stop 2024-04-20T11:27:46.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.471+0000 7f51ff200640 1 -- 172.21.15.152:0/4103359267 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5200159040 msgr2=0x7f5200072f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:46.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.471+0000 7f51ff200640 1 --2- 172.21.15.152:0/4103359267 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5200159040 0x7f5200072f50 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f51e802b930 tx=0x7f51e80046c0 comp rx=0 tx=0).stop 2024-04-20T11:27:46.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.471+0000 7f51ff200640 1 -- 172.21.15.152:0/4103359267 shutdown_connections 2024-04-20T11:27:46.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.471+0000 7f51ff200640 1 --2- 172.21.15.152:0/4103359267 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f51dc06c8f0 0x7f51dc06edb0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:46.473 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.471+0000 7f51ff200640 1 --2- 172.21.15.152:0/4103359267 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5200162530 0x7f5200073490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:46.473 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.471+0000 7f51ff200640 1 --2- 172.21.15.152:0/4103359267 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5200159040 0x7f5200072f50 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:46.473 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.471+0000 7f51ff200640 1 --2- 172.21.15.152:0/4103359267 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f520006f6a0 0x7f5200146820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:46.473 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.471+0000 7f51ff200640 1 -- 172.21.15.152:0/4103359267 >> 172.21.15.152:0/4103359267 conn(0x7f52000d0260 msgr2=0x7f52000d0640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:46.473 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.471+0000 7f51ff200640 1 -- 172.21.15.152:0/4103359267 shutdown_connections 2024-04-20T11:27:46.473 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.472+0000 7f51ff200640 1 -- 172.21.15.152:0/4103359267 wait complete. 2024-04-20T11:27:46.491 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi152.front.sepia.ceph.com... 2024-04-20T11:27:46.491 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:27:46.492 DEBUG:teuthology.orchestra.run.smithi152:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-20T11:27:46.573 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T11:27:46.574 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi152.front.sepia.ceph.com 2024-04-20T11:27:46.574 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:27:46.574 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-20T11:27:46.636 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T11:27:46.636 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:27:46.694 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T11:27:46.695 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:27:46.744 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:27:46.955 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.955+0000 7fcade000640 1 Processor -- start 2024-04-20T11:27:46.956 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.955+0000 7fcade000640 1 -- start start 2024-04-20T11:27:46.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.957+0000 7fcade000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fcad8160b90 0x7fcad815d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:46.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.957+0000 7fcade000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcad8071cd0 0x7fcad8072160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:46.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.957+0000 7fcade000640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcad80ce8c0 con 0x7fcad815d1e0 2024-04-20T11:27:46.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.957+0000 7fcade000640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fcad80726a0 con 0x7fcad8160b90 2024-04-20T11:27:46.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.957+0000 7fcade000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fcad81600b0 con 0x7fcad8071cd0 2024-04-20T11:27:46.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.957+0000 7fcadcc00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fcad815d1e0 0x7fcad80cdfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:44846/0 (socket says 172.21.15.152:44846) 2024-04-20T11:27:46.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.957+0000 7fcadcc00640 1 -- 172.21.15.152:0/687570584 learned_addr learned my addr 172.21.15.152:0/687570584 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:27:46.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.957+0000 7fcacfe00640 1 --2- 172.21.15.152:0/687570584 >> v2:172.21.15.152:3300/0 conn(0x7fcad8160b90 0x7fcad815d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:46.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.958+0000 7fcadd600640 1 --2- 172.21.15.152:0/687570584 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcad8071cd0 0x7fcad8072160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:46.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.958+0000 7fcad7e00640 1 -- 172.21.15.152:0/687570584 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3317029062 0 0) 0x7fcad80ce8c0 con 0x7fcad815d1e0 2024-04-20T11:27:46.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.958+0000 7fcad7e00640 1 -- 172.21.15.152:0/687570584 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcabc003520 con 0x7fcad815d1e0 2024-04-20T11:27:46.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.958+0000 7fcacfe00640 1 -- 172.21.15.152:0/687570584 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcad8071cd0 msgr2=0x7fcad8072160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:46.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.958+0000 7fcacfe00640 1 --2- 172.21.15.152:0/687570584 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcad8071cd0 0x7fcad8072160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:46.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.958+0000 7fcacfe00640 1 -- 172.21.15.152:0/687570584 >> v1:172.21.15.152:6789/0 conn(0x7fcad815d1e0 legacy=0x7fcad80cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:46.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.958+0000 7fcacfe00640 1 -- 172.21.15.152:0/687570584 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcad8160390 con 0x7fcad8160b90 2024-04-20T11:27:46.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.959+0000 7fcacfe00640 1 --2- 172.21.15.152:0/687570584 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcad8160b90 0x7fcad815d5c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fcac00097d0 tx=0x7fcac002b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=996faae07ebaaf53 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:46.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.959+0000 7fcad7e00640 1 -- 172.21.15.152:0/687570584 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac0035020 con 0x7fcad8160b90 2024-04-20T11:27:46.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.959+0000 7fcad7e00640 1 -- 172.21.15.152:0/687570584 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcac0031800 con 0x7fcad8160b90 2024-04-20T11:27:46.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.959+0000 7fcad7e00640 1 -- 172.21.15.152:0/687570584 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac0031bd0 con 0x7fcad8160b90 2024-04-20T11:27:46.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.960+0000 7fcade000640 1 -- 172.21.15.152:0/687570584 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcad8160b90 msgr2=0x7fcad815d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:46.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.960+0000 7fcade000640 1 --2- 172.21.15.152:0/687570584 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcad8160b90 0x7fcad815d5c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fcac00097d0 tx=0x7fcac002b8a0 comp rx=0 tx=0).stop 2024-04-20T11:27:46.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.960+0000 7fcade000640 1 -- 172.21.15.152:0/687570584 shutdown_connections 2024-04-20T11:27:46.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.960+0000 7fcade000640 1 --2- 172.21.15.152:0/687570584 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcad8071cd0 0x7fcad8072160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:46.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.960+0000 7fcade000640 1 --2- 172.21.15.152:0/687570584 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcad8160b90 0x7fcad815d5c0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:46.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.960+0000 7fcade000640 1 -- 172.21.15.152:0/687570584 >> 172.21.15.152:0/687570584 conn(0x7fcad8067ae0 msgr2=0x7fcad806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:46.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.960+0000 7fcade000640 1 -- 172.21.15.152:0/687570584 shutdown_connections 2024-04-20T11:27:46.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.960+0000 7fcade000640 1 -- 172.21.15.152:0/687570584 wait complete. 2024-04-20T11:27:46.963 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.962+0000 7fcade000640 1 Processor -- start 2024-04-20T11:27:46.963 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.962+0000 7fcade000640 1 -- start start 2024-04-20T11:27:46.963 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.963+0000 7fcade000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcad8071cd0 0x7fcad814ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:46.964 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.963+0000 7fcade000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcad815d1e0 0x7fcad81536a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:46.964 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.963+0000 7fcade000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcad8160b90 0x7fcad8155fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:46.964 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.963+0000 7fcade000640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fcad80ccd70 con 0x7fcad8160b90 2024-04-20T11:27:46.964 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.963+0000 7fcade000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcad80ceba0 con 0x7fcad8071cd0 2024-04-20T11:27:46.964 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.963+0000 7fcade000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fcad80ccef0 con 0x7fcad815d1e0 2024-04-20T11:27:46.964 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.963+0000 7fcadd600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcad8160b90 0x7fcad8155fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:46.964 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.963+0000 7fcadd600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcad8160b90 0x7fcad8155fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43214/0 (socket says 172.21.15.152:43214) 2024-04-20T11:27:46.964 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.963+0000 7fcadd600640 1 -- 172.21.15.152:0/212802616 learned_addr learned my addr 172.21.15.152:0/212802616 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:46.966 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.964+0000 7fcacfe00640 1 --2- 172.21.15.152:0/212802616 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcad815d1e0 0x7fcad81536a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:46.966 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.964+0000 7fcadcc00640 1 --2- 172.21.15.152:0/212802616 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcad8071cd0 0x7fcad814ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:46.966 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.964+0000 7fcadd600640 1 -- 172.21.15.152:0/212802616 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcad815d1e0 msgr2=0x7fcad81536a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:46.966 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.964+0000 7fcadd600640 1 --2- 172.21.15.152:0/212802616 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcad815d1e0 0x7fcad81536a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:46.967 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.964+0000 7fcadd600640 1 -- 172.21.15.152:0/212802616 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcad8071cd0 msgr2=0x7fcad814ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:46.967 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.964+0000 7fcadd600640 1 --2- 172.21.15.152:0/212802616 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcad8071cd0 0x7fcad814ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:46.967 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.964+0000 7fcadd600640 1 -- 172.21.15.152:0/212802616 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcad8153be0 con 0x7fcad8160b90 2024-04-20T11:27:46.967 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.964+0000 7fcadcc00640 1 --2- 172.21.15.152:0/212802616 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcad8071cd0 0x7fcad814ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:27:46.967 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.964+0000 7fcadd600640 1 --2- 172.21.15.152:0/212802616 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcad8160b90 0x7fcad8155fc0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fcad000d830 tx=0x7fcad000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:46.967 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.964+0000 7fcad5600640 1 -- 172.21.15.152:0/212802616 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad000b6c0 con 0x7fcad8160b90 2024-04-20T11:27:46.967 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.964+0000 7fcad5600640 1 -- 172.21.15.152:0/212802616 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcad0019070 con 0x7fcad8160b90 2024-04-20T11:27:46.967 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.964+0000 7fcad5600640 1 -- 172.21.15.152:0/212802616 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad000bac0 con 0x7fcad8160b90 2024-04-20T11:27:46.967 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.964+0000 7fcade000640 1 -- 172.21.15.152:0/212802616 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcad81f56d0 con 0x7fcad8160b90 2024-04-20T11:27:46.967 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.965+0000 7fcade000640 1 -- 172.21.15.152:0/212802616 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcad81f5c90 con 0x7fcad8160b90 2024-04-20T11:27:46.967 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.966+0000 7fcade000640 1 -- 172.21.15.152:0/212802616 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcad8075e60 con 0x7fcad8160b90 2024-04-20T11:27:46.968 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.967+0000 7fcad5600640 1 -- 172.21.15.152:0/212802616 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcad001e460 con 0x7fcad8160b90 2024-04-20T11:27:46.968 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.968+0000 7fcad5600640 1 --2- 172.21.15.152:0/212802616 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fcab406c8f0 0x7fcab406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:46.968 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.968+0000 7fcad5600640 1 -- 172.21.15.152:0/212802616 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fcad008c350 con 0x7fcad8160b90 2024-04-20T11:27:46.968 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.968+0000 7fcadcc00640 1 --2- 172.21.15.152:0/212802616 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fcab406c8f0 0x7fcab406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:46.969 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.969+0000 7fcadcc00640 1 --2- 172.21.15.152:0/212802616 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fcab406c8f0 0x7fcab406edb0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fcac8005c00 tx=0x7fcac8009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:46.972 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:46.972+0000 7fcad5600640 1 -- 172.21.15.152:0/212802616 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fcad005a660 con 0x7fcad8160b90 2024-04-20T11:27:47.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.177+0000 7fcade000640 1 -- 172.21.15.152:0/212802616 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcad8072c90 con 0x7fcad8160b90 2024-04-20T11:27:47.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.179+0000 7fcad5600640 1 -- 172.21.15.152:0/212802616 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16914 (secure 0 0 0) 0x7fcad0059f60 con 0x7fcad8160b90 2024-04-20T11:27:47.180 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:27:47.180 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":23,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:27:42.601620+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:26:56.261424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T11:26:57.079286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":2}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.26","mappings":[{"from":7,"to":1}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:27:47.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.182+0000 7fcade000640 1 -- 172.21.15.152:0/212802616 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fcab406c8f0 msgr2=0x7fcab406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:47.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.182+0000 7fcade000640 1 --2- 172.21.15.152:0/212802616 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fcab406c8f0 0x7fcab406edb0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fcac8005c00 tx=0x7fcac8009040 comp rx=0 tx=0).stop 2024-04-20T11:27:47.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.182+0000 7fcade000640 1 -- 172.21.15.152:0/212802616 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcad8160b90 msgr2=0x7fcad8155fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:47.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.182+0000 7fcade000640 1 --2- 172.21.15.152:0/212802616 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcad8160b90 0x7fcad8155fc0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fcad000d830 tx=0x7fcad000dcf0 comp rx=0 tx=0).stop 2024-04-20T11:27:47.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.182+0000 7fcade000640 1 -- 172.21.15.152:0/212802616 shutdown_connections 2024-04-20T11:27:47.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.182+0000 7fcade000640 1 --2- 172.21.15.152:0/212802616 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fcab406c8f0 0x7fcab406edb0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:47.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.182+0000 7fcade000640 1 --2- 172.21.15.152:0/212802616 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcad8160b90 0x7fcad8155fc0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:47.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.182+0000 7fcade000640 1 --2- 172.21.15.152:0/212802616 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcad815d1e0 0x7fcad81536a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:47.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.182+0000 7fcade000640 1 --2- 172.21.15.152:0/212802616 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcad8071cd0 0x7fcad814ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:47.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.182+0000 7fcade000640 1 -- 172.21.15.152:0/212802616 >> 172.21.15.152:0/212802616 conn(0x7fcad8067ae0 msgr2=0x7fcad80b1360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:47.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.182+0000 7fcade000640 1 -- 172.21.15.152:0/212802616 shutdown_connections 2024-04-20T11:27:47.184 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.182+0000 7fcade000640 1 -- 172.21.15.152:0/212802616 wait complete. 2024-04-20T11:27:47.203 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T11:26:47.246571+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T11:26:56.261424+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T11:26:57.079286+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T11:27:47.204 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T11:27:47.397 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.396+0000 7effcf200640 1 Processor -- start 2024-04-20T11:27:47.397 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.396+0000 7effcf200640 1 -- start start 2024-04-20T11:27:47.399 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.397+0000 7effcf200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7effd0156c00 0x7effd0160a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:47.399 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.397+0000 7effcf200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7effd015ca40 0x7effd015cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:47.399 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.398+0000 7effcf200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7effd006bda0 0x7effd006c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:47.399 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.398+0000 7effcf200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7effd0160fd0 con 0x7effd0156c00 2024-04-20T11:27:47.399 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.398+0000 7effcf200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7effd015d5c0 con 0x7effd006bda0 2024-04-20T11:27:47.399 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.398+0000 7effcf200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7effd015d8a0 con 0x7effd015ca40 2024-04-20T11:27:47.399 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.398+0000 7effcde00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7effd0156c00 0x7effd0160a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:47.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.398+0000 7effcde00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7effd0156c00 0x7effd0160a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43222/0 (socket says 172.21.15.152:43222) 2024-04-20T11:27:47.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.398+0000 7effcde00640 1 -- 172.21.15.152:0/3170983125 learned_addr learned my addr 172.21.15.152:0/3170983125 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:47.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.398+0000 7effce800640 1 --2- 172.21.15.152:0/3170983125 >> v2:172.21.15.175:3300/0 conn(0x7effd006bda0 0x7effd006c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:47.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.398+0000 7effcd400640 1 --2- 172.21.15.152:0/3170983125 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7effd015ca40 0x7effd015cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:47.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.398+0000 7effcde00640 1 -- 172.21.15.152:0/3170983125 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7effd015ca40 msgr2=0x7effd015cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:47.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.398+0000 7effcde00640 1 --2- 172.21.15.152:0/3170983125 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7effd015ca40 0x7effd015cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:47.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.398+0000 7effcde00640 1 -- 172.21.15.152:0/3170983125 >> v2:172.21.15.175:3300/0 conn(0x7effd006bda0 msgr2=0x7effd006c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:47.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.398+0000 7effcde00640 1 --2- 172.21.15.152:0/3170983125 >> v2:172.21.15.175:3300/0 conn(0x7effd006bda0 0x7effd006c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:47.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.398+0000 7effcde00640 1 -- 172.21.15.152:0/3170983125 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effd0160290 con 0x7effd0156c00 2024-04-20T11:27:47.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.398+0000 7effce800640 1 --2- 172.21.15.152:0/3170983125 >> v2:172.21.15.175:3300/0 conn(0x7effd006bda0 0x7effd006c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:27:47.401 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.400+0000 7effcde00640 1 --2- 172.21.15.152:0/3170983125 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7effd0156c00 0x7effd0160a90 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7effd006f310 tx=0x7effc002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32cd5cc4c5d6575b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:47.401 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.400+0000 7effcca00640 1 -- 172.21.15.152:0/3170983125 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc0035020 con 0x7effd0156c00 2024-04-20T11:27:47.402 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.400+0000 7effcca00640 1 -- 172.21.15.152:0/3170983125 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effc0031870 con 0x7effd0156c00 2024-04-20T11:27:47.402 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.400+0000 7effcca00640 1 -- 172.21.15.152:0/3170983125 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc0031c40 con 0x7effd0156c00 2024-04-20T11:27:47.402 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.401+0000 7effcf200640 1 -- 172.21.15.152:0/3170983125 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7effd0156c00 msgr2=0x7effd0160a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:47.402 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.401+0000 7effcf200640 1 --2- 172.21.15.152:0/3170983125 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7effd0156c00 0x7effd0160a90 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7effd006f310 tx=0x7effc002b910 comp rx=0 tx=0).stop 2024-04-20T11:27:47.402 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.401+0000 7effcf200640 1 -- 172.21.15.152:0/3170983125 shutdown_connections 2024-04-20T11:27:47.402 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.401+0000 7effcf200640 1 --2- 172.21.15.152:0/3170983125 >> v2:172.21.15.175:3300/0 conn(0x7effd006bda0 0x7effd006c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:47.402 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.401+0000 7effcf200640 1 --2- 172.21.15.152:0/3170983125 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7effd015ca40 0x7effd015cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:47.402 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.401+0000 7effcf200640 1 --2- 172.21.15.152:0/3170983125 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7effd0156c00 0x7effd0160a90 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:47.402 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.401+0000 7effcf200640 1 -- 172.21.15.152:0/3170983125 >> 172.21.15.152:0/3170983125 conn(0x7effd00b21e0 msgr2=0x7effd00b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:47.402 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.401+0000 7effcf200640 1 -- 172.21.15.152:0/3170983125 shutdown_connections 2024-04-20T11:27:47.402 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.401+0000 7effcf200640 1 -- 172.21.15.152:0/3170983125 wait complete. 2024-04-20T11:27:47.403 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.403+0000 7effcf200640 1 Processor -- start 2024-04-20T11:27:47.403 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.403+0000 7effcf200640 1 -- start start 2024-04-20T11:27:47.405 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.404+0000 7effcf200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7effd006bda0 0x7effd0099a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:47.405 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.404+0000 7effcf200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7effd0156c00 0x7effd0143310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:47.405 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.404+0000 7effcf200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7effd015ca40 0x7effd009a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:47.405 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.404+0000 7effcf200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7effd00b3240 con 0x7effd006bda0 2024-04-20T11:27:47.405 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.404+0000 7effcf200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7effd00b30c0 con 0x7effd0156c00 2024-04-20T11:27:47.405 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.404+0000 7effcf200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7effd00b33c0 con 0x7effd015ca40 2024-04-20T11:27:47.405 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.404+0000 7effcde00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7effd006bda0 0x7effd0099a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:47.405 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.404+0000 7effcde00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7effd006bda0 0x7effd0099a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43228/0 (socket says 172.21.15.152:43228) 2024-04-20T11:27:47.405 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.404+0000 7effcde00640 1 -- 172.21.15.152:0/1612659649 learned_addr learned my addr 172.21.15.152:0/1612659649 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:47.405 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.404+0000 7effcd400640 1 --2- 172.21.15.152:0/1612659649 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7effd0156c00 0x7effd0143310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:47.405 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.404+0000 7effcde00640 1 -- 172.21.15.152:0/1612659649 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7effd015ca40 msgr2=0x7effd009a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:47.406 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.404+0000 7effcde00640 1 --2- 172.21.15.152:0/1612659649 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7effd015ca40 0x7effd009a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:47.406 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.404+0000 7effcde00640 1 -- 172.21.15.152:0/1612659649 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7effd0156c00 msgr2=0x7effd0143310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:47.406 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.404+0000 7effcde00640 1 --2- 172.21.15.152:0/1612659649 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7effd0156c00 0x7effd0143310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:47.406 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.404+0000 7effcde00640 1 -- 172.21.15.152:0/1612659649 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effd009c860 con 0x7effd006bda0 2024-04-20T11:27:47.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.405+0000 7effcde00640 1 --2- 172.21.15.152:0/1612659649 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7effd006bda0 0x7effd0099a90 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7effc0000c00 tx=0x7effc00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:47.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.405+0000 7effc6000640 1 -- 172.21.15.152:0/1612659649 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc0035040 con 0x7effd006bda0 2024-04-20T11:27:47.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.405+0000 7effc6000640 1 -- 172.21.15.152:0/1612659649 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effc0030d00 con 0x7effd006bda0 2024-04-20T11:27:47.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.405+0000 7effc6000640 1 -- 172.21.15.152:0/1612659649 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc00456a0 con 0x7effd006bda0 2024-04-20T11:27:47.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.405+0000 7effcf200640 1 -- 172.21.15.152:0/1612659649 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effd009cb40 con 0x7effd006bda0 2024-04-20T11:27:47.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.405+0000 7effcf200640 1 -- 172.21.15.152:0/1612659649 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effd0143850 con 0x7effd006bda0 2024-04-20T11:27:47.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.406+0000 7effcf200640 1 -- 172.21.15.152:0/1612659649 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effd0075e60 con 0x7effd006bda0 2024-04-20T11:27:47.408 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.407+0000 7effc6000640 1 -- 172.21.15.152:0/1612659649 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7effc0038020 con 0x7effd006bda0 2024-04-20T11:27:47.409 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.408+0000 7effc6000640 1 --2- 172.21.15.152:0/1612659649 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7effb006c8f0 0x7effb006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:47.409 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.408+0000 7effc6000640 1 -- 172.21.15.152:0/1612659649 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7effc00adf20 con 0x7effd006bda0 2024-04-20T11:27:47.409 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.409+0000 7effcd400640 1 --2- 172.21.15.152:0/1612659649 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7effb006c8f0 0x7effb006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:47.410 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.409+0000 7effcd400640 1 --2- 172.21.15.152:0/1612659649 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7effb006c8f0 0x7effb006edb0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7effbc006fd0 tx=0x7effbc008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:47.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.413+0000 7effc6000640 1 -- 172.21.15.152:0/1612659649 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7effc007c2c0 con 0x7effd006bda0 2024-04-20T11:27:47.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.614+0000 7effcf200640 1 -- 172.21.15.152:0/1612659649 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7effd006f390 con 0x7effd006bda0 2024-04-20T11:27:47.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.615+0000 7effc6000640 1 -- 172.21.15.152:0/1612659649 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) ==== 93+0+10 (secure 0 0 0) 0x7effc007bbc0 con 0x7effd006bda0 2024-04-20T11:27:47.616 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 1 2024-04-20T11:27:47.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.617+0000 7effcf200640 1 -- 172.21.15.152:0/1612659649 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7effb006c8f0 msgr2=0x7effb006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:47.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.617+0000 7effcf200640 1 --2- 172.21.15.152:0/1612659649 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7effb006c8f0 0x7effb006edb0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7effbc006fd0 tx=0x7effbc008040 comp rx=0 tx=0).stop 2024-04-20T11:27:47.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.617+0000 7effcf200640 1 -- 172.21.15.152:0/1612659649 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7effd006bda0 msgr2=0x7effd0099a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:47.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.617+0000 7effcf200640 1 --2- 172.21.15.152:0/1612659649 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7effd006bda0 0x7effd0099a90 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7effc0000c00 tx=0x7effc00046c0 comp rx=0 tx=0).stop 2024-04-20T11:27:47.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.618+0000 7effcf200640 1 -- 172.21.15.152:0/1612659649 shutdown_connections 2024-04-20T11:27:47.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.618+0000 7effcf200640 1 --2- 172.21.15.152:0/1612659649 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7effb006c8f0 0x7effb006edb0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:47.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.618+0000 7effcf200640 1 --2- 172.21.15.152:0/1612659649 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7effd015ca40 0x7effd009a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:47.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.618+0000 7effcf200640 1 --2- 172.21.15.152:0/1612659649 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7effd0156c00 0x7effd0143310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:47.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.618+0000 7effcf200640 1 --2- 172.21.15.152:0/1612659649 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7effd006bda0 0x7effd0099a90 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:47.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.618+0000 7effcf200640 1 -- 172.21.15.152:0/1612659649 >> 172.21.15.152:0/1612659649 conn(0x7effd00b21e0 msgr2=0x7effd0155f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:47.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.618+0000 7effcf200640 1 -- 172.21.15.152:0/1612659649 shutdown_connections 2024-04-20T11:27:47.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.618+0000 7effcf200640 1 -- 172.21.15.152:0/1612659649 wait complete. 2024-04-20T11:27:47.637 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T11:27:47.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.841+0000 7f442f000640 1 Processor -- start 2024-04-20T11:27:47.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.841+0000 7f442f000640 1 -- start start 2024-04-20T11:27:47.843 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.843+0000 7f442f000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4428160220 0x7f44281606b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:47.843 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.843+0000 7f442f000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f4428141450 0x7f44281418e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:47.843 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.843+0000 7f442f000640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f442806fb40 con 0x7f4428071cd0 2024-04-20T11:27:47.843 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.843+0000 7f442f000640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f442813f8e0 con 0x7f4428141450 2024-04-20T11:27:47.844 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.843+0000 7f442f000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4428160f90 con 0x7f4428160220 2024-04-20T11:27:47.844 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.843+0000 7f442e600640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f4428141450 0x7f44281418e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:47.845 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.843+0000 7f442dc00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f4428071cd0 0x7f44280720b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:44856/0 (socket says 172.21.15.152:44856) 2024-04-20T11:27:47.845 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.843+0000 7f442dc00640 1 -- 172.21.15.152:0/1261815214 learned_addr learned my addr 172.21.15.152:0/1261815214 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:27:47.845 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.844+0000 7f442d200640 1 --2- 172.21.15.152:0/1261815214 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4428160220 0x7f44281606b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:47.845 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.844+0000 7f442e600640 1 -- 172.21.15.152:0/1261815214 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4428160220 msgr2=0x7f44281606b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:47.845 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.844+0000 7f442e600640 1 --2- 172.21.15.152:0/1261815214 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4428160220 0x7f44281606b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:47.845 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.844+0000 7f442e600640 1 -- 172.21.15.152:0/1261815214 >> v1:172.21.15.152:6789/0 conn(0x7f4428071cd0 legacy=0x7f44280720b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:47.845 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.844+0000 7f442e600640 1 -- 172.21.15.152:0/1261815214 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f442806dd90 con 0x7f4428141450 2024-04-20T11:27:47.846 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.845+0000 7f442e600640 1 --2- 172.21.15.152:0/1261815214 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4428141450 0x7f44281418e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f4420009dc0 tx=0x7f442002bdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3417ede5cda82f38 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:47.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.845+0000 7f4427e00640 1 -- 172.21.15.152:0/1261815214 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4420039070 con 0x7f4428141450 2024-04-20T11:27:47.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.845+0000 7f4427e00640 1 -- 172.21.15.152:0/1261815214 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f442002cba0 con 0x7f4428141450 2024-04-20T11:27:47.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.845+0000 7f4427e00640 1 -- 172.21.15.152:0/1261815214 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4420034440 con 0x7f4428141450 2024-04-20T11:27:47.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.846+0000 7f442f000640 1 -- 172.21.15.152:0/1261815214 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4428141450 msgr2=0x7f44281418e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:47.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.846+0000 7f442f000640 1 --2- 172.21.15.152:0/1261815214 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4428141450 0x7f44281418e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f4420009dc0 tx=0x7f442002bdf0 comp rx=0 tx=0).stop 2024-04-20T11:27:47.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.846+0000 7f442f000640 1 -- 172.21.15.152:0/1261815214 shutdown_connections 2024-04-20T11:27:47.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.846+0000 7f442f000640 1 --2- 172.21.15.152:0/1261815214 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4428160220 0x7f44281606b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:47.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.846+0000 7f442f000640 1 --2- 172.21.15.152:0/1261815214 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4428141450 0x7f44281418e0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:47.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.846+0000 7f442f000640 1 -- 172.21.15.152:0/1261815214 >> 172.21.15.152:0/1261815214 conn(0x7f4428067ae0 msgr2=0x7f442806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:47.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.846+0000 7f442f000640 1 -- 172.21.15.152:0/1261815214 shutdown_connections 2024-04-20T11:27:47.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.846+0000 7f442f000640 1 -- 172.21.15.152:0/1261815214 wait complete. 2024-04-20T11:27:47.848 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.848+0000 7f442f000640 1 Processor -- start 2024-04-20T11:27:47.848 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.848+0000 7f442f000640 1 -- start start 2024-04-20T11:27:47.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.849+0000 7f442f000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4428071cd0 0x7f442813e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:47.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.849+0000 7f442f000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4428141450 0x7f442812e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:47.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.849+0000 7f442f000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4428160220 0x7f4428159510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:47.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.849+0000 7f442f000640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f442806e630 con 0x7f4428160220 2024-04-20T11:27:47.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.849+0000 7f442f000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f442806e350 con 0x7f4428071cd0 2024-04-20T11:27:47.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.849+0000 7f442f000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4428160bf0 con 0x7f4428141450 2024-04-20T11:27:47.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.849+0000 7f442e600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4428160220 0x7f4428159510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:47.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.849+0000 7f442e600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4428160220 0x7f4428159510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43246/0 (socket says 172.21.15.152:43246) 2024-04-20T11:27:47.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.849+0000 7f442e600640 1 -- 172.21.15.152:0/4275607677 learned_addr learned my addr 172.21.15.152:0/4275607677 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:47.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.850+0000 7f442dc00640 1 --2- 172.21.15.152:0/4275607677 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4428071cd0 0x7f442813e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:47.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.850+0000 7f442d200640 1 --2- 172.21.15.152:0/4275607677 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4428141450 0x7f442812e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:47.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.850+0000 7f442e600640 1 -- 172.21.15.152:0/4275607677 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4428141450 msgr2=0x7f442812e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:47.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.850+0000 7f442e600640 1 --2- 172.21.15.152:0/4275607677 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4428141450 0x7f442812e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:47.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.850+0000 7f442e600640 1 -- 172.21.15.152:0/4275607677 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4428071cd0 msgr2=0x7f442813e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:47.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.850+0000 7f442e600640 1 --2- 172.21.15.152:0/4275607677 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4428071cd0 0x7f442813e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:47.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.850+0000 7f442e600640 1 -- 172.21.15.152:0/4275607677 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4428139800 con 0x7f4428160220 2024-04-20T11:27:47.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.850+0000 7f442dc00640 1 --2- 172.21.15.152:0/4275607677 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4428071cd0 0x7f442813e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:27:47.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.850+0000 7f442e600640 1 --2- 172.21.15.152:0/4275607677 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4428160220 0x7f4428159510 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f4420037040 tx=0x7f4420004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:47.852 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.850+0000 7f4425600640 1 -- 172.21.15.152:0/4275607677 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4420039050 con 0x7f4428160220 2024-04-20T11:27:47.852 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.850+0000 7f4425600640 1 -- 172.21.15.152:0/4275607677 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4420045070 con 0x7f4428160220 2024-04-20T11:27:47.852 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.850+0000 7f442f000640 1 -- 172.21.15.152:0/4275607677 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4428139ae0 con 0x7f4428160220 2024-04-20T11:27:47.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.850+0000 7f4425600640 1 -- 172.21.15.152:0/4275607677 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f442003e850 con 0x7f4428160220 2024-04-20T11:27:47.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.850+0000 7f442f000640 1 -- 172.21.15.152:0/4275607677 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f442813a0a0 con 0x7f4428160220 2024-04-20T11:27:47.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.852+0000 7f442f000640 1 -- 172.21.15.152:0/4275607677 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4428075e60 con 0x7f4428160220 2024-04-20T11:27:47.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.853+0000 7f4425600640 1 -- 172.21.15.152:0/4275607677 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f442003e9f0 con 0x7f4428160220 2024-04-20T11:27:47.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.854+0000 7f4425600640 1 --2- 172.21.15.152:0/4275607677 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f440806c8f0 0x7f440806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:47.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.854+0000 7f4425600640 1 -- 172.21.15.152:0/4275607677 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f44200ae9a0 con 0x7f4428160220 2024-04-20T11:27:47.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.855+0000 7f442dc00640 1 --2- 172.21.15.152:0/4275607677 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f440806c8f0 0x7f440806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:47.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.855+0000 7f442dc00640 1 --2- 172.21.15.152:0/4275607677 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f440806c8f0 0x7f440806edb0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f441c000f60 tx=0x7f441c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:47.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:47.859+0000 7f4425600640 1 -- 172.21.15.152:0/4275607677 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f442007ce70 con 0x7f4428160220 2024-04-20T11:27:48.054 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.054+0000 7f442f000640 1 -- 172.21.15.152:0/4275607677 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f44280b08e0 con 0x7f4428160220 2024-04-20T11:27:48.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.055+0000 7f4425600640 1 -- 172.21.15.152:0/4275607677 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) ==== 104+0+11 (secure 0 0 0) 0x7f442007c770 con 0x7f4428160220 2024-04-20T11:27:48.055 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 32 2024-04-20T11:27:48.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.057+0000 7f442f000640 1 -- 172.21.15.152:0/4275607677 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f440806c8f0 msgr2=0x7f440806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:48.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.057+0000 7f442f000640 1 --2- 172.21.15.152:0/4275607677 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f440806c8f0 0x7f440806edb0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f441c000f60 tx=0x7f441c009040 comp rx=0 tx=0).stop 2024-04-20T11:27:48.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.057+0000 7f442f000640 1 -- 172.21.15.152:0/4275607677 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4428160220 msgr2=0x7f4428159510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:48.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.057+0000 7f442f000640 1 --2- 172.21.15.152:0/4275607677 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4428160220 0x7f4428159510 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f4420037040 tx=0x7f4420004a40 comp rx=0 tx=0).stop 2024-04-20T11:27:48.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.057+0000 7f442f000640 1 -- 172.21.15.152:0/4275607677 shutdown_connections 2024-04-20T11:27:48.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.057+0000 7f442f000640 1 --2- 172.21.15.152:0/4275607677 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f440806c8f0 0x7f440806edb0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:48.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.057+0000 7f442f000640 1 --2- 172.21.15.152:0/4275607677 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4428160220 0x7f4428159510 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:48.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.057+0000 7f442f000640 1 --2- 172.21.15.152:0/4275607677 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4428141450 0x7f442812e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:48.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.057+0000 7f442f000640 1 --2- 172.21.15.152:0/4275607677 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4428071cd0 0x7f442813e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:48.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.057+0000 7f442f000640 1 -- 172.21.15.152:0/4275607677 >> 172.21.15.152:0/4275607677 conn(0x7f4428067ae0 msgr2=0x7f44280b26e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:48.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.058+0000 7f442f000640 1 -- 172.21.15.152:0/4275607677 shutdown_connections 2024-04-20T11:27:48.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.058+0000 7f442f000640 1 -- 172.21.15.152:0/4275607677 wait complete. 2024-04-20T11:27:48.077 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T11:27:48.277 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.276+0000 7fdc57200640 1 Processor -- start 2024-04-20T11:27:48.277 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.277+0000 7fdc57200640 1 -- start start 2024-04-20T11:27:48.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.278+0000 7fdc57200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fdc5815c590 0x7fdc5806bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:48.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.278+0000 7fdc57200640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc58152830 con 0x7fdc5806c2e0 2024-04-20T11:27:48.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.278+0000 7fdc57200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc580b39c0 con 0x7fdc58152450 2024-04-20T11:27:48.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.278+0000 7fdc57200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fdc5806c750 con 0x7fdc5815c590 2024-04-20T11:27:48.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.278+0000 7fdc4fe00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fdc5815c590 0x7fdc5806bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:48.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.278+0000 7fdc56800640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fdc5806c2e0 0x7fdc580ae470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:44860/0 (socket says 172.21.15.152:44860) 2024-04-20T11:27:48.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.278+0000 7fdc56800640 1 -- 172.21.15.152:0/3190178487 learned_addr learned my addr 172.21.15.152:0/3190178487 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:27:48.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.279+0000 7fdc55400640 1 -- 172.21.15.152:0/3190178487 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1704003145 0 0) 0x7fdc58152830 con 0x7fdc5806c2e0 2024-04-20T11:27:48.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.279+0000 7fdc55400640 1 -- 172.21.15.152:0/3190178487 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdc3c005730 con 0x7fdc5806c2e0 2024-04-20T11:27:48.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.279+0000 7fdc4fe00640 1 -- 172.21.15.152:0/3190178487 >> v1:172.21.15.175:6789/0 conn(0x7fdc58152450 legacy=0x7fdc580b30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:48.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.279+0000 7fdc4fe00640 1 -- 172.21.15.152:0/3190178487 >> v1:172.21.15.152:6789/0 conn(0x7fdc5806c2e0 legacy=0x7fdc580ae470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:48.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.279+0000 7fdc4fe00640 1 -- 172.21.15.152:0/3190178487 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc580b3ca0 con 0x7fdc5815c590 2024-04-20T11:27:48.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.280+0000 7fdc4fe00640 1 --2- 172.21.15.152:0/3190178487 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdc5815c590 0x7fdc5806bda0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fdc400097a0 tx=0x7fdc4002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f48cc048b735341 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:48.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.280+0000 7fdc55400640 1 -- 172.21.15.152:0/3190178487 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc40035020 con 0x7fdc5815c590 2024-04-20T11:27:48.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.280+0000 7fdc55400640 1 -- 172.21.15.152:0/3190178487 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc40031830 con 0x7fdc5815c590 2024-04-20T11:27:48.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.280+0000 7fdc55400640 1 -- 172.21.15.152:0/3190178487 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc40031be0 con 0x7fdc5815c590 2024-04-20T11:27:48.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.281+0000 7fdc57200640 1 -- 172.21.15.152:0/3190178487 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdc5815c590 msgr2=0x7fdc5806bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:48.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.281+0000 7fdc57200640 1 --2- 172.21.15.152:0/3190178487 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdc5815c590 0x7fdc5806bda0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fdc400097a0 tx=0x7fdc4002b8d0 comp rx=0 tx=0).stop 2024-04-20T11:27:48.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.281+0000 7fdc57200640 1 -- 172.21.15.152:0/3190178487 shutdown_connections 2024-04-20T11:27:48.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.281+0000 7fdc57200640 1 --2- 172.21.15.152:0/3190178487 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdc5815c590 0x7fdc5806bda0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:48.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.281+0000 7fdc57200640 1 -- 172.21.15.152:0/3190178487 >> 172.21.15.152:0/3190178487 conn(0x7fdc580b21e0 msgr2=0x7fdc580b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:48.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.281+0000 7fdc57200640 1 -- 172.21.15.152:0/3190178487 shutdown_connections 2024-04-20T11:27:48.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.281+0000 7fdc57200640 1 -- 172.21.15.152:0/3190178487 wait complete. 2024-04-20T11:27:48.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.283+0000 7fdc57200640 1 Processor -- start 2024-04-20T11:27:48.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.283+0000 7fdc57200640 1 -- start start 2024-04-20T11:27:48.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.284+0000 7fdc57200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdc5806c2e0 0x7fdc581f7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:48.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.284+0000 7fdc57200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdc58152450 0x7fdc5814aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:48.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.284+0000 7fdc57200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc5815c590 0x7fdc5814f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:48.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.284+0000 7fdc57200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fdc580adf00 con 0x7fdc5806c2e0 2024-04-20T11:27:48.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.284+0000 7fdc57200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdc580b4260 con 0x7fdc5815c590 2024-04-20T11:27:48.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.284+0000 7fdc57200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fdc580aeb80 con 0x7fdc58152450 2024-04-20T11:27:48.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.284+0000 7fdc55e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdc5806c2e0 0x7fdc581f7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:48.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.284+0000 7fdc55e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdc5806c2e0 0x7fdc581f7150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43272/0 (socket says 172.21.15.152:43272) 2024-04-20T11:27:48.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.284+0000 7fdc55e00640 1 -- 172.21.15.152:0/2711160868 learned_addr learned my addr 172.21.15.152:0/2711160868 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:48.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.284+0000 7fdc55e00640 1 -- 172.21.15.152:0/2711160868 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdc58152450 msgr2=0x7fdc5814aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:48.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.284+0000 7fdc55e00640 1 --2- 172.21.15.152:0/2711160868 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdc58152450 0x7fdc5814aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:48.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.284+0000 7fdc55e00640 1 -- 172.21.15.152:0/2711160868 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc5815c590 msgr2=0x7fdc5814f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:48.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.284+0000 7fdc55e00640 1 --2- 172.21.15.152:0/2711160868 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc5815c590 0x7fdc5814f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:48.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.284+0000 7fdc55e00640 1 -- 172.21.15.152:0/2711160868 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc5814bf90 con 0x7fdc5806c2e0 2024-04-20T11:27:48.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.285+0000 7fdc55e00640 1 --2- 172.21.15.152:0/2711160868 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdc5806c2e0 0x7fdc581f7150 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fdc4800c430 tx=0x7fdc4800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:48.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.285+0000 7fdc4e000640 1 -- 172.21.15.152:0/2711160868 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc48011790 con 0x7fdc5806c2e0 2024-04-20T11:27:48.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.285+0000 7fdc4e000640 1 -- 172.21.15.152:0/2711160868 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc48017070 con 0x7fdc5806c2e0 2024-04-20T11:27:48.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.285+0000 7fdc4e000640 1 -- 172.21.15.152:0/2711160868 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc48011b70 con 0x7fdc5806c2e0 2024-04-20T11:27:48.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.285+0000 7fdc57200640 1 -- 172.21.15.152:0/2711160868 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc5814b430 con 0x7fdc5806c2e0 2024-04-20T11:27:48.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.285+0000 7fdc57200640 1 -- 172.21.15.152:0/2711160868 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc58152be0 con 0x7fdc5806c2e0 2024-04-20T11:27:48.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.286+0000 7fdc57200640 1 -- 172.21.15.152:0/2711160868 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc58075e60 con 0x7fdc5806c2e0 2024-04-20T11:27:48.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.287+0000 7fdc4e000640 1 -- 172.21.15.152:0/2711160868 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdc48011d10 con 0x7fdc5806c2e0 2024-04-20T11:27:48.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.288+0000 7fdc4e000640 1 --2- 172.21.15.152:0/2711160868 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fdc3406c8f0 0x7fdc3406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:48.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.288+0000 7fdc4e000640 1 -- 172.21.15.152:0/2711160868 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fdc4808bae0 con 0x7fdc5806c2e0 2024-04-20T11:27:48.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.288+0000 7fdc4fe00640 1 --2- 172.21.15.152:0/2711160868 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fdc3406c8f0 0x7fdc3406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:48.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.289+0000 7fdc4fe00640 1 --2- 172.21.15.152:0/2711160868 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fdc3406c8f0 0x7fdc3406edb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fdc40036000 tx=0x7fdc40030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:48.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.292+0000 7fdc4e000640 1 -- 172.21.15.152:0/2711160868 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fdc48059e80 con 0x7fdc5806c2e0 2024-04-20T11:27:48.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.476+0000 7fdc57200640 1 -- 172.21.15.152:0/2711160868 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fdc5806f770 con 0x7fdc5806c2e0 2024-04-20T11:27:48.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.477+0000 7fdc4e000640 1 -- 172.21.15.152:0/2711160868 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) ==== 100+0+11 (secure 0 0 0) 0x7fdc48059780 con 0x7fdc5806c2e0 2024-04-20T11:27:48.478 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 64 2024-04-20T11:27:48.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.480+0000 7fdc57200640 1 -- 172.21.15.152:0/2711160868 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fdc3406c8f0 msgr2=0x7fdc3406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:48.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.480+0000 7fdc57200640 1 --2- 172.21.15.152:0/2711160868 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fdc3406c8f0 0x7fdc3406edb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fdc40036000 tx=0x7fdc40030000 comp rx=0 tx=0).stop 2024-04-20T11:27:48.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.480+0000 7fdc57200640 1 -- 172.21.15.152:0/2711160868 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdc5806c2e0 msgr2=0x7fdc581f7150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:48.482 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.480+0000 7fdc57200640 1 --2- 172.21.15.152:0/2711160868 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdc5806c2e0 0x7fdc581f7150 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fdc4800c430 tx=0x7fdc4800c8f0 comp rx=0 tx=0).stop 2024-04-20T11:27:48.482 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.480+0000 7fdc57200640 1 -- 172.21.15.152:0/2711160868 shutdown_connections 2024-04-20T11:27:48.482 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.480+0000 7fdc57200640 1 --2- 172.21.15.152:0/2711160868 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fdc3406c8f0 0x7fdc3406edb0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:48.482 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.480+0000 7fdc57200640 1 --2- 172.21.15.152:0/2711160868 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc5815c590 0x7fdc5814f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:48.482 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.480+0000 7fdc57200640 1 --2- 172.21.15.152:0/2711160868 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdc58152450 0x7fdc5814aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:48.482 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.480+0000 7fdc57200640 1 --2- 172.21.15.152:0/2711160868 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdc5806c2e0 0x7fdc581f7150 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:48.482 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.480+0000 7fdc57200640 1 -- 172.21.15.152:0/2711160868 >> 172.21.15.152:0/2711160868 conn(0x7fdc580b21e0 msgr2=0x7fdc580aef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:48.482 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.480+0000 7fdc57200640 1 -- 172.21.15.152:0/2711160868 shutdown_connections 2024-04-20T11:27:48.482 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.480+0000 7fdc57200640 1 -- 172.21.15.152:0/2711160868 wait complete. 2024-04-20T11:27:48.499 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-20T11:27:48.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.690+0000 7f4ce2c00640 1 Processor -- start 2024-04-20T11:27:48.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.690+0000 7f4ce2c00640 1 -- start start 2024-04-20T11:27:48.692 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.691+0000 7f4ce2c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4cdc0d0e60 0x7f4cdc06d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:48.692 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.691+0000 7f4ce2c00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f4cdc06df20 0x7f4cdc15d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:48.692 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.691+0000 7f4ce2c00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4cdc0cfef0 con 0x7f4cdc159320 2024-04-20T11:27:48.692 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.691+0000 7f4ce2c00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f4cdc06e390 con 0x7f4cdc06df20 2024-04-20T11:27:48.692 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.691+0000 7f4ce2c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4cdc15c3d0 con 0x7f4cdc0d0e60 2024-04-20T11:27:48.692 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.692+0000 7f4ce2200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f4cdc06df20 0x7f4cdc15d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:48.692 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.692+0000 7f4ce2200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f4cdc06df20 0x7f4cdc15d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43282/0 (socket says 172.21.15.152:43282) 2024-04-20T11:27:48.693 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.692+0000 7f4ce2200640 1 -- 172.21.15.152:0/2350969396 learned_addr learned my addr 172.21.15.152:0/2350969396 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:48.693 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.692+0000 7f4ce0e00640 1 --2- 172.21.15.152:0/2350969396 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4cdc0d0e60 0x7f4cdc06d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:48.693 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.692+0000 7f4ce2200640 1 -- 172.21.15.152:0/2350969396 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4cdc0d0e60 msgr2=0x7f4cdc06d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:48.693 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.692+0000 7f4ce2200640 1 --2- 172.21.15.152:0/2350969396 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4cdc0d0e60 0x7f4cdc06d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:48.693 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.692+0000 7f4ce2200640 1 -- 172.21.15.152:0/2350969396 >> v1:172.21.15.175:6789/0 conn(0x7f4cdc159320 legacy=0x7f4cdc15bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:48.693 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.692+0000 7f4ce2200640 1 -- 172.21.15.152:0/2350969396 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cdc15c6b0 con 0x7f4cdc06df20 2024-04-20T11:27:48.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.693+0000 7f4ce2200640 1 --2- 172.21.15.152:0/2350969396 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4cdc06df20 0x7f4cdc15d450 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f4cd4009e50 tx=0x7f4cd402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba8e4252b912718b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:48.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.693+0000 7f4cdbe00640 1 -- 172.21.15.152:0/2350969396 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd402c9e0 con 0x7f4cdc06df20 2024-04-20T11:27:48.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.693+0000 7f4cdbe00640 1 -- 172.21.15.152:0/2350969396 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4cd402cb80 con 0x7f4cdc06df20 2024-04-20T11:27:48.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.693+0000 7f4cdbe00640 1 -- 172.21.15.152:0/2350969396 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd40343f0 con 0x7f4cdc06df20 2024-04-20T11:27:48.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.694+0000 7f4ce2c00640 1 -- 172.21.15.152:0/2350969396 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4cdc06df20 msgr2=0x7f4cdc15d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:48.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.694+0000 7f4ce2c00640 1 --2- 172.21.15.152:0/2350969396 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4cdc06df20 0x7f4cdc15d450 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f4cd4009e50 tx=0x7f4cd402be40 comp rx=0 tx=0).stop 2024-04-20T11:27:48.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.694+0000 7f4ce2c00640 1 -- 172.21.15.152:0/2350969396 shutdown_connections 2024-04-20T11:27:48.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.694+0000 7f4ce2c00640 1 --2- 172.21.15.152:0/2350969396 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4cdc06df20 0x7f4cdc15d450 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:48.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.694+0000 7f4ce2c00640 1 --2- 172.21.15.152:0/2350969396 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4cdc0d0e60 0x7f4cdc06d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:48.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.694+0000 7f4ce2c00640 1 -- 172.21.15.152:0/2350969396 >> 172.21.15.152:0/2350969396 conn(0x7f4cdc0d0260 msgr2=0x7f4cdc0d0670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:48.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.694+0000 7f4ce2c00640 1 -- 172.21.15.152:0/2350969396 shutdown_connections 2024-04-20T11:27:48.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.694+0000 7f4ce2c00640 1 -- 172.21.15.152:0/2350969396 wait complete. 2024-04-20T11:27:48.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.696+0000 7f4ce2c00640 1 Processor -- start 2024-04-20T11:27:48.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.696+0000 7f4ce2c00640 1 -- start start 2024-04-20T11:27:48.699 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.697+0000 7f4ce2c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4cdc06df20 0x7f4cdc1f5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:48.699 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.697+0000 7f4ce2c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4cdc0d0e60 0x7f4cdc09a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:48.699 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.697+0000 7f4ce2c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4cdc159320 0x7f4cdc0ae300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:48.699 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.697+0000 7f4ce2c00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f4cdc15d990 con 0x7f4cdc06df20 2024-04-20T11:27:48.699 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.697+0000 7f4ce2c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4cdc15cf30 con 0x7f4cdc159320 2024-04-20T11:27:48.699 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.697+0000 7f4ce2c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4cdc15db10 con 0x7f4cdc0d0e60 2024-04-20T11:27:48.699 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.697+0000 7f4ce1800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4cdc06df20 0x7f4cdc1f5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:48.699 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.697+0000 7f4ce1800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4cdc06df20 0x7f4cdc1f5320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43288/0 (socket says 172.21.15.152:43288) 2024-04-20T11:27:48.699 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.697+0000 7f4ce1800640 1 -- 172.21.15.152:0/4059603280 learned_addr learned my addr 172.21.15.152:0/4059603280 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:48.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.698+0000 7f4ce2200640 1 --2- 172.21.15.152:0/4059603280 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4cdc159320 0x7f4cdc0ae300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:48.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.698+0000 7f4ce0e00640 1 --2- 172.21.15.152:0/4059603280 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4cdc0d0e60 0x7f4cdc09a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:48.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.698+0000 7f4ce1800640 1 -- 172.21.15.152:0/4059603280 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4cdc0d0e60 msgr2=0x7f4cdc09a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:48.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.698+0000 7f4ce1800640 1 --2- 172.21.15.152:0/4059603280 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4cdc0d0e60 0x7f4cdc09a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:48.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.698+0000 7f4ce1800640 1 -- 172.21.15.152:0/4059603280 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4cdc159320 msgr2=0x7f4cdc0ae300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:48.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.698+0000 7f4ce1800640 1 --2- 172.21.15.152:0/4059603280 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4cdc159320 0x7f4cdc0ae300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:48.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.698+0000 7f4ce1800640 1 -- 172.21.15.152:0/4059603280 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cdc09d2a0 con 0x7f4cdc06df20 2024-04-20T11:27:48.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.698+0000 7f4ce1800640 1 --2- 172.21.15.152:0/4059603280 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4cdc06df20 0x7f4cdc1f5320 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f4cd000b3e0 tx=0x7f4cd000b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:48.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.698+0000 7f4cd9600640 1 -- 172.21.15.152:0/4059603280 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd0015020 con 0x7f4cdc06df20 2024-04-20T11:27:48.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.698+0000 7f4cd9600640 1 -- 172.21.15.152:0/4059603280 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4cd0011850 con 0x7f4cdc06df20 2024-04-20T11:27:48.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.698+0000 7f4ce2c00640 1 -- 172.21.15.152:0/4059603280 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cdc09d580 con 0x7f4cdc06df20 2024-04-20T11:27:48.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.698+0000 7f4cd9600640 1 -- 172.21.15.152:0/4059603280 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd0011c40 con 0x7f4cdc06df20 2024-04-20T11:27:48.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.699+0000 7f4ce2c00640 1 -- 172.21.15.152:0/4059603280 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cdc09db40 con 0x7f4cdc06df20 2024-04-20T11:27:48.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.700+0000 7f4ce2c00640 1 -- 172.21.15.152:0/4059603280 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cdc075e60 con 0x7f4cdc06df20 2024-04-20T11:27:48.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.701+0000 7f4cd9600640 1 -- 172.21.15.152:0/4059603280 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4cd0018020 con 0x7f4cdc06df20 2024-04-20T11:27:48.703 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.702+0000 7f4cd9600640 1 --2- 172.21.15.152:0/4059603280 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f4cbc06c8f0 0x7f4cbc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:48.703 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.702+0000 7f4cd9600640 1 -- 172.21.15.152:0/4059603280 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f4cd008bba0 con 0x7f4cdc06df20 2024-04-20T11:27:48.703 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.702+0000 7f4ce0e00640 1 --2- 172.21.15.152:0/4059603280 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f4cbc06c8f0 0x7f4cbc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:48.703 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.703+0000 7f4ce0e00640 1 --2- 172.21.15.152:0/4059603280 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f4cbc06c8f0 0x7f4cbc06edb0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4cc8006fd0 tx=0x7f4cc8008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:48.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.707+0000 7f4cd9600640 1 -- 172.21.15.152:0/4059603280 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f4cd001f020 con 0x7f4cdc06df20 2024-04-20T11:27:48.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.920+0000 7f4ce2c00640 1 -- 172.21.15.152:0/4059603280 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f4cdc09a900 con 0x7f4cdc06df20 2024-04-20T11:27:48.922 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.921+0000 7f4cd9600640 1 -- 172.21.15.152:0/4059603280 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) ==== 80+0+0 (secure 0 0 0) 0x7f4cd00172c0 con 0x7f4cdc06df20 2024-04-20T11:27:48.922 INFO:teuthology.orchestra.run.smithi152.stderr:listed 0 entries 2024-04-20T11:27:48.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.925+0000 7f4ce2c00640 1 -- 172.21.15.152:0/4059603280 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f4cbc06c8f0 msgr2=0x7f4cbc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:48.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.925+0000 7f4ce2c00640 1 --2- 172.21.15.152:0/4059603280 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f4cbc06c8f0 0x7f4cbc06edb0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4cc8006fd0 tx=0x7f4cc8008040 comp rx=0 tx=0).stop 2024-04-20T11:27:48.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.925+0000 7f4ce2c00640 1 -- 172.21.15.152:0/4059603280 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4cdc06df20 msgr2=0x7f4cdc1f5320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:48.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.925+0000 7f4ce2c00640 1 --2- 172.21.15.152:0/4059603280 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4cdc06df20 0x7f4cdc1f5320 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f4cd000b3e0 tx=0x7f4cd000b8a0 comp rx=0 tx=0).stop 2024-04-20T11:27:48.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.925+0000 7f4ce2c00640 1 -- 172.21.15.152:0/4059603280 shutdown_connections 2024-04-20T11:27:48.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.925+0000 7f4ce2c00640 1 --2- 172.21.15.152:0/4059603280 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f4cbc06c8f0 0x7f4cbc06edb0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:48.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.925+0000 7f4ce2c00640 1 --2- 172.21.15.152:0/4059603280 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4cdc159320 0x7f4cdc0ae300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:48.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.925+0000 7f4ce2c00640 1 --2- 172.21.15.152:0/4059603280 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4cdc0d0e60 0x7f4cdc09a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:48.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.925+0000 7f4ce2c00640 1 --2- 172.21.15.152:0/4059603280 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4cdc06df20 0x7f4cdc1f5320 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:48.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.925+0000 7f4ce2c00640 1 -- 172.21.15.152:0/4059603280 >> 172.21.15.152:0/4059603280 conn(0x7f4cdc0d0260 msgr2=0x7f4cdc06edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:48.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.926+0000 7f4ce2c00640 1 -- 172.21.15.152:0/4059603280 shutdown_connections 2024-04-20T11:27:48.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:48.926+0000 7f4ce2c00640 1 -- 172.21.15.152:0/4059603280 wait complete. 2024-04-20T11:27:48.946 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T11:27:48.946 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:27:49.005 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:27:49.212 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.212+0000 7fe981800640 1 Processor -- start 2024-04-20T11:27:49.213 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.212+0000 7fe981800640 1 -- start start 2024-04-20T11:27:49.214 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.214+0000 7fe981800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe97c070b90 0x7fe97c06b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:49.214 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.214+0000 7fe981800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fe97c06bec0 0x7fe97c160530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:49.215 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.214+0000 7fe981800640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe97c070ff0 con 0x7fe97c0d2110 2024-04-20T11:27:49.215 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.214+0000 7fe981800640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fe97c070320 con 0x7fe97c06bec0 2024-04-20T11:27:49.215 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.214+0000 7fe981800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe97c0b66c0 con 0x7fe97c070b90 2024-04-20T11:27:49.216 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.214+0000 7fe97b400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fe97c06bec0 0x7fe97c160530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:49.216 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.215+0000 7fe97b400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fe97c06bec0 0x7fe97c160530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43302/0 (socket says 172.21.15.152:43302) 2024-04-20T11:27:49.216 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.215+0000 7fe97b400640 1 -- 172.21.15.152:0/4144558150 learned_addr learned my addr 172.21.15.152:0/4144558150 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:49.216 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.215+0000 7fe97a000640 1 --2- 172.21.15.152:0/4144558150 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe97c070b90 0x7fe97c06b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:49.216 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.215+0000 7fe97b400640 1 -- 172.21.15.152:0/4144558150 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe97c070b90 msgr2=0x7fe97c06b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:49.216 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.215+0000 7fe97b400640 1 --2- 172.21.15.152:0/4144558150 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe97c070b90 0x7fe97c06b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:49.216 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.215+0000 7fe97b400640 1 -- 172.21.15.152:0/4144558150 >> v1:172.21.15.175:6789/0 conn(0x7fe97c0d2110 legacy=0x7fe97c0b5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:49.216 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.215+0000 7fe97b400640 1 -- 172.21.15.152:0/4144558150 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe97c06fa80 con 0x7fe97c06bec0 2024-04-20T11:27:49.217 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.217+0000 7fe97b400640 1 --2- 172.21.15.152:0/4144558150 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe97c06bec0 0x7fe97c160530 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe974009e50 tx=0x7fe97402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb673a6ffec20863 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:49.218 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.217+0000 7fe979600640 1 -- 172.21.15.152:0/4144558150 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe97402ca70 con 0x7fe97c06bec0 2024-04-20T11:27:49.218 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.217+0000 7fe979600640 1 -- 172.21.15.152:0/4144558150 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe97402cc10 con 0x7fe97c06bec0 2024-04-20T11:27:49.218 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.217+0000 7fe979600640 1 -- 172.21.15.152:0/4144558150 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9740343f0 con 0x7fe97c06bec0 2024-04-20T11:27:49.218 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.218+0000 7fe981800640 1 -- 172.21.15.152:0/4144558150 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe97c06bec0 msgr2=0x7fe97c160530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:49.219 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.218+0000 7fe981800640 1 --2- 172.21.15.152:0/4144558150 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe97c06bec0 0x7fe97c160530 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe974009e50 tx=0x7fe97402bed0 comp rx=0 tx=0).stop 2024-04-20T11:27:49.219 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.218+0000 7fe981800640 1 -- 172.21.15.152:0/4144558150 shutdown_connections 2024-04-20T11:27:49.219 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.218+0000 7fe981800640 1 --2- 172.21.15.152:0/4144558150 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe97c06bec0 0x7fe97c160530 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:49.219 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.218+0000 7fe981800640 1 --2- 172.21.15.152:0/4144558150 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe97c070b90 0x7fe97c06b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:49.219 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.218+0000 7fe981800640 1 -- 172.21.15.152:0/4144558150 >> 172.21.15.152:0/4144558150 conn(0x7fe97c09aa40 msgr2=0x7fe97c0b68c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:49.219 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.218+0000 7fe981800640 1 -- 172.21.15.152:0/4144558150 shutdown_connections 2024-04-20T11:27:49.219 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.218+0000 7fe981800640 1 -- 172.21.15.152:0/4144558150 wait complete. 2024-04-20T11:27:49.220 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.220+0000 7fe981800640 1 Processor -- start 2024-04-20T11:27:49.220 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.220+0000 7fe981800640 1 -- start start 2024-04-20T11:27:49.223 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.221+0000 7fe981800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe97c06bec0 0x7fe97c146ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:49.223 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.221+0000 7fe981800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe97c070b90 0x7fe97c1575d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:49.223 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.221+0000 7fe981800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe97c0d2110 0x7fe97c157b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:49.223 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.221+0000 7fe981800640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fe97c160a70 con 0x7fe97c06bec0 2024-04-20T11:27:49.223 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.221+0000 7fe981800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe97c070040 con 0x7fe97c070b90 2024-04-20T11:27:49.223 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.221+0000 7fe981800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe97c160bf0 con 0x7fe97c0d2110 2024-04-20T11:27:49.223 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.221+0000 7fe97aa00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe97c06bec0 0x7fe97c146ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:49.223 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.221+0000 7fe97aa00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe97c06bec0 0x7fe97c146ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43310/0 (socket says 172.21.15.152:43310) 2024-04-20T11:27:49.224 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.221+0000 7fe97aa00640 1 -- 172.21.15.152:0/2715234441 learned_addr learned my addr 172.21.15.152:0/2715234441 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:49.224 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.222+0000 7fe97a000640 1 --2- 172.21.15.152:0/2715234441 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe97c070b90 0x7fe97c1575d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:49.224 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.222+0000 7fe97b400640 1 --2- 172.21.15.152:0/2715234441 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe97c0d2110 0x7fe97c157b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:49.224 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.222+0000 7fe97aa00640 1 -- 172.21.15.152:0/2715234441 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe97c0d2110 msgr2=0x7fe97c157b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:49.224 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.222+0000 7fe97aa00640 1 --2- 172.21.15.152:0/2715234441 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe97c0d2110 0x7fe97c157b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:49.224 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.222+0000 7fe97aa00640 1 -- 172.21.15.152:0/2715234441 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe97c070b90 msgr2=0x7fe97c1575d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:49.224 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.222+0000 7fe97aa00640 1 --2- 172.21.15.152:0/2715234441 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe97c070b90 0x7fe97c1575d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:49.224 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.222+0000 7fe97aa00640 1 -- 172.21.15.152:0/2715234441 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe97c157030 con 0x7fe97c06bec0 2024-04-20T11:27:49.224 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.222+0000 7fe97aa00640 1 --2- 172.21.15.152:0/2715234441 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe97c06bec0 0x7fe97c146ed0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fe96c00c400 tx=0x7fe96c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:49.224 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.222+0000 7fe972a00640 1 -- 172.21.15.152:0/2715234441 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe96c017070 con 0x7fe97c06bec0 2024-04-20T11:27:49.224 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.222+0000 7fe972a00640 1 -- 172.21.15.152:0/2715234441 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe96c0052d0 con 0x7fe97c06bec0 2024-04-20T11:27:49.224 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.222+0000 7fe972a00640 1 -- 172.21.15.152:0/2715234441 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe96c003830 con 0x7fe97c06bec0 2024-04-20T11:27:49.224 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.222+0000 7fe981800640 1 -- 172.21.15.152:0/2715234441 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe97c147920 con 0x7fe97c06bec0 2024-04-20T11:27:49.224 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.223+0000 7fe981800640 1 -- 172.21.15.152:0/2715234441 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe97c147ee0 con 0x7fe97c06bec0 2024-04-20T11:27:49.225 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.224+0000 7fe981800640 1 -- 172.21.15.152:0/2715234441 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe97c069100 con 0x7fe97c06bec0 2024-04-20T11:27:49.225 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.225+0000 7fe972a00640 1 -- 172.21.15.152:0/2715234441 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe96c01a070 con 0x7fe97c06bec0 2024-04-20T11:27:49.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.226+0000 7fe972a00640 1 --2- 172.21.15.152:0/2715234441 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe95c06c8f0 0x7fe95c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:49.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.226+0000 7fe972a00640 1 -- 172.21.15.152:0/2715234441 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fe96c08af80 con 0x7fe97c06bec0 2024-04-20T11:27:49.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.226+0000 7fe97a000640 1 --2- 172.21.15.152:0/2715234441 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe95c06c8f0 0x7fe95c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:49.227 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.227+0000 7fe97a000640 1 --2- 172.21.15.152:0/2715234441 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe95c06c8f0 0x7fe95c06edb0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe97c1f7cf0 tx=0x7fe968008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:49.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.231+0000 7fe972a00640 1 -- 172.21.15.152:0/2715234441 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe96c059290 con 0x7fe97c06bec0 2024-04-20T11:27:49.433 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.432+0000 7fe981800640 1 -- 172.21.15.152:0/2715234441 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe97c0b2480 con 0x7fe97c06bec0 2024-04-20T11:27:49.436 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.435+0000 7fe972a00640 1 -- 172.21.15.152:0/2715234441 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16914 (secure 0 0 0) 0x7fe96c058b90 con 0x7fe97c06bec0 2024-04-20T11:27:49.436 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:27:49.436 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":23,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:27:42.601620+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:26:56.261424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T11:26:57.079286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":2}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.26","mappings":[{"from":7,"to":1}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:27:49.438 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.438+0000 7fe981800640 1 -- 172.21.15.152:0/2715234441 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe95c06c8f0 msgr2=0x7fe95c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:49.439 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.438+0000 7fe981800640 1 --2- 172.21.15.152:0/2715234441 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe95c06c8f0 0x7fe95c06edb0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe97c1f7cf0 tx=0x7fe968008040 comp rx=0 tx=0).stop 2024-04-20T11:27:49.439 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.438+0000 7fe981800640 1 -- 172.21.15.152:0/2715234441 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe97c06bec0 msgr2=0x7fe97c146ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:49.439 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.438+0000 7fe981800640 1 --2- 172.21.15.152:0/2715234441 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe97c06bec0 0x7fe97c146ed0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fe96c00c400 tx=0x7fe96c00c8c0 comp rx=0 tx=0).stop 2024-04-20T11:27:49.439 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.438+0000 7fe981800640 1 -- 172.21.15.152:0/2715234441 shutdown_connections 2024-04-20T11:27:49.439 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.438+0000 7fe981800640 1 --2- 172.21.15.152:0/2715234441 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe95c06c8f0 0x7fe95c06edb0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:49.439 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.438+0000 7fe981800640 1 --2- 172.21.15.152:0/2715234441 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe97c0d2110 0x7fe97c157b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:49.439 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.438+0000 7fe981800640 1 --2- 172.21.15.152:0/2715234441 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe97c070b90 0x7fe97c1575d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:49.439 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.438+0000 7fe981800640 1 --2- 172.21.15.152:0/2715234441 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe97c06bec0 0x7fe97c146ed0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:49.439 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.438+0000 7fe981800640 1 -- 172.21.15.152:0/2715234441 >> 172.21.15.152:0/2715234441 conn(0x7fe97c09aa40 msgr2=0x7fe97c0b1010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:49.439 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.438+0000 7fe981800640 1 -- 172.21.15.152:0/2715234441 shutdown_connections 2024-04-20T11:27:49.439 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.438+0000 7fe981800640 1 -- 172.21.15.152:0/2715234441 wait complete. 2024-04-20T11:27:49.458 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T11:26:47.246571+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T11:26:56.261424+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T11:26:57.079286+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T11:27:49.458 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T11:27:49.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.662+0000 7fd73de00640 1 Processor -- start 2024-04-20T11:27:49.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.662+0000 7fd73de00640 1 -- start start 2024-04-20T11:27:49.664 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.663+0000 7fd73de00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd738156c00 0x7fd738160a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:49.664 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.663+0000 7fd73de00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fd73815ca40 0x7fd73815cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:49.664 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.663+0000 7fd73de00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd738160fd0 con 0x7fd73806bda0 2024-04-20T11:27:49.664 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.664+0000 7fd73de00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fd73815d5c0 con 0x7fd73815ca40 2024-04-20T11:27:49.664 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.664+0000 7fd73de00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd73815d8a0 con 0x7fd738156c00 2024-04-20T11:27:49.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.664+0000 7fd73d400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fd73806bda0 0x7fd7380b30c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:44870/0 (socket says 172.21.15.152:44870) 2024-04-20T11:27:49.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.664+0000 7fd73d400640 1 -- 172.21.15.152:0/568198089 learned_addr learned my addr 172.21.15.152:0/568198089 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:27:49.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.664+0000 7fd72fe00640 1 --2- 172.21.15.152:0/568198089 >> v2:172.21.15.175:3300/0 conn(0x7fd73815ca40 0x7fd73815cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:49.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.664+0000 7fd73ca00640 1 --2- 172.21.15.152:0/568198089 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd738156c00 0x7fd738160a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:49.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.664+0000 7fd737e00640 1 -- 172.21.15.152:0/568198089 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1874340508 0 0) 0x7fd738160fd0 con 0x7fd73806bda0 2024-04-20T11:27:49.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.664+0000 7fd737e00640 1 -- 172.21.15.152:0/568198089 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd71c003520 con 0x7fd73806bda0 2024-04-20T11:27:49.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.665+0000 7fd737e00640 1 -- 172.21.15.152:0/568198089 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3287289986 0 0) 0x7fd71c003520 con 0x7fd73806bda0 2024-04-20T11:27:49.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.665+0000 7fd737e00640 1 -- 172.21.15.152:0/568198089 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd738156c00 msgr2=0x7fd738160a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:49.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.665+0000 7fd737e00640 1 --2- 172.21.15.152:0/568198089 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd738156c00 0x7fd738160a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:49.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.665+0000 7fd737e00640 1 -- 172.21.15.152:0/568198089 >> v2:172.21.15.175:3300/0 conn(0x7fd73815ca40 msgr2=0x7fd73815cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:49.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.665+0000 7fd737e00640 1 --2- 172.21.15.152:0/568198089 >> v2:172.21.15.175:3300/0 conn(0x7fd73815ca40 0x7fd73815cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:49.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.665+0000 7fd737e00640 1 -- 172.21.15.152:0/568198089 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd738160290 con 0x7fd73806bda0 2024-04-20T11:27:49.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.665+0000 7fd737e00640 1 -- 172.21.15.152:0/568198089 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7fd7300019f0 con 0x7fd73806bda0 2024-04-20T11:27:49.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.665+0000 7fd737e00640 1 -- 172.21.15.152:0/568198089 >> v1:172.21.15.152:6789/0 conn(0x7fd73806bda0 legacy=0x7fd7380b30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:49.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.665+0000 7fd737e00640 1 --2- 172.21.15.152:0/568198089 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd71c0039b0 0x7fd71c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:49.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.665+0000 7fd737e00640 1 --2- 172.21.15.152:0/568198089 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd738156c00 0x7fd71c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:49.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.665+0000 7fd737e00640 1 --2- 172.21.15.152:0/568198089 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd71c004d40 0x7fd71c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:49.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.665+0000 7fd737e00640 1 -- 172.21.15.152:0/568198089 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fd73815d5c0 con 0x7fd71c0039b0 2024-04-20T11:27:49.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.665+0000 7fd737e00640 1 -- 172.21.15.152:0/568198089 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd73815d8a0 con 0x7fd71c004d40 2024-04-20T11:27:49.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.665+0000 7fd737e00640 1 -- 172.21.15.152:0/568198089 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd71c00b9c0 con 0x7fd738156c00 2024-04-20T11:27:49.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.665+0000 7fd73ca00640 1 --2- 172.21.15.152:0/568198089 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd71c0039b0 0x7fd71c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:49.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.666+0000 7fd73ca00640 1 -- 172.21.15.152:0/568198089 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd738156c00 msgr2=0x7fd71c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:49.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.666+0000 7fd73ca00640 1 --2- 172.21.15.152:0/568198089 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd738156c00 0x7fd71c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:49.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.666+0000 7fd73ca00640 1 -- 172.21.15.152:0/568198089 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd71c004d40 msgr2=0x7fd71c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:49.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.666+0000 7fd73ca00640 1 --2- 172.21.15.152:0/568198089 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd71c004d40 0x7fd71c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:49.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.666+0000 7fd73ca00640 1 -- 172.21.15.152:0/568198089 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd71c003520 con 0x7fd71c0039b0 2024-04-20T11:27:49.670 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.667+0000 7fd73ca00640 1 --2- 172.21.15.152:0/568198089 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd71c0039b0 0x7fd71c003dd0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fd728009ac0 tx=0x7fd71c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce42ba44c49e495c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:49.670 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.667+0000 7fd737e00640 1 -- 172.21.15.152:0/568198089 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd728039070 con 0x7fd71c0039b0 2024-04-20T11:27:49.670 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.667+0000 7fd737e00640 1 -- 172.21.15.152:0/568198089 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd72802c710 con 0x7fd71c0039b0 2024-04-20T11:27:49.670 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.667+0000 7fd737e00640 1 -- 172.21.15.152:0/568198089 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd728009690 con 0x7fd71c0039b0 2024-04-20T11:27:49.670 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.668+0000 7fd73de00640 1 -- 172.21.15.152:0/568198089 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd71c0039b0 msgr2=0x7fd71c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:49.670 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.668+0000 7fd73de00640 1 --2- 172.21.15.152:0/568198089 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd71c0039b0 0x7fd71c003dd0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fd728009ac0 tx=0x7fd71c005830 comp rx=0 tx=0).stop 2024-04-20T11:27:49.670 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.668+0000 7fd73d400640 1 -- 172.21.15.152:0/568198089 reap_dead start 2024-04-20T11:27:49.670 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.668+0000 7fd73de00640 1 -- 172.21.15.152:0/568198089 shutdown_connections 2024-04-20T11:27:49.670 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.668+0000 7fd73de00640 1 -- 172.21.15.152:0/568198089 >> 172.21.15.152:0/568198089 conn(0x7fd7380b21e0 msgr2=0x7fd7380b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:49.670 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.668+0000 7fd73de00640 1 -- 172.21.15.152:0/568198089 shutdown_connections 2024-04-20T11:27:49.670 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.668+0000 7fd73de00640 1 -- 172.21.15.152:0/568198089 wait complete. 2024-04-20T11:27:49.672 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.670+0000 7fd73de00640 1 Processor -- start 2024-04-20T11:27:49.672 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.670+0000 7fd73de00640 1 -- start start 2024-04-20T11:27:49.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.671+0000 7fd73de00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd71c0039b0 0x7fd73809ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:49.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.671+0000 7fd73de00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd7381f5590 0x7fd7380b0170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:49.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.671+0000 7fd73de00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd7380b06b0 0x7fd738156fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:49.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.671+0000 7fd73de00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fd738160c10 con 0x7fd7381f5590 2024-04-20T11:27:49.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.671+0000 7fd73de00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd738160a90 con 0x7fd7380b06b0 2024-04-20T11:27:49.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.671+0000 7fd73de00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd738160d90 con 0x7fd71c0039b0 2024-04-20T11:27:49.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.671+0000 7fd72fe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd7381f5590 0x7fd7380b0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:49.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.671+0000 7fd72fe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd7381f5590 0x7fd7380b0170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43330/0 (socket says 172.21.15.152:43330) 2024-04-20T11:27:49.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.671+0000 7fd72fe00640 1 -- 172.21.15.152:0/2811054530 learned_addr learned my addr 172.21.15.152:0/2811054530 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:49.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.671+0000 7fd73ca00640 1 --2- 172.21.15.152:0/2811054530 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd71c0039b0 0x7fd73809ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:49.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.672+0000 7fd73d400640 1 --2- 172.21.15.152:0/2811054530 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd7380b06b0 0x7fd738156fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:49.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.672+0000 7fd72fe00640 1 -- 172.21.15.152:0/2811054530 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd71c0039b0 msgr2=0x7fd73809ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:49.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.672+0000 7fd72fe00640 1 --2- 172.21.15.152:0/2811054530 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd71c0039b0 0x7fd73809ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:49.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.672+0000 7fd72fe00640 1 -- 172.21.15.152:0/2811054530 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd7380b06b0 msgr2=0x7fd738156fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:49.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.672+0000 7fd72fe00640 1 --2- 172.21.15.152:0/2811054530 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd7380b06b0 0x7fd738156fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:49.674 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.672+0000 7fd72fe00640 1 -- 172.21.15.152:0/2811054530 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7381f5a10 con 0x7fd7381f5590 2024-04-20T11:27:49.674 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.672+0000 7fd73d400640 1 --2- 172.21.15.152:0/2811054530 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd7380b06b0 0x7fd738156fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:27:49.674 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.672+0000 7fd72fe00640 1 --2- 172.21.15.152:0/2811054530 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd7381f5590 0x7fd7380b0170 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fd72000e890 tx=0x7fd72000ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:49.675 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.672+0000 7fd735600640 1 -- 172.21.15.152:0/2811054530 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd720017070 con 0x7fd7381f5590 2024-04-20T11:27:49.675 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.672+0000 7fd735600640 1 -- 172.21.15.152:0/2811054530 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd72000c8e0 con 0x7fd7381f5590 2024-04-20T11:27:49.675 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.672+0000 7fd735600640 1 -- 172.21.15.152:0/2811054530 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd72000ccb0 con 0x7fd7381f5590 2024-04-20T11:27:49.675 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.672+0000 7fd73de00640 1 -- 172.21.15.152:0/2811054530 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd738157c80 con 0x7fd7381f5590 2024-04-20T11:27:49.675 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.673+0000 7fd73de00640 1 -- 172.21.15.152:0/2811054530 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd73809d5d0 con 0x7fd7381f5590 2024-04-20T11:27:49.675 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.674+0000 7fd73de00640 1 -- 172.21.15.152:0/2811054530 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7380b4a00 con 0x7fd7381f5590 2024-04-20T11:27:49.675 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.675+0000 7fd735600640 1 -- 172.21.15.152:0/2811054530 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd72001a070 con 0x7fd7381f5590 2024-04-20T11:27:49.676 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.676+0000 7fd735600640 1 --2- 172.21.15.152:0/2811054530 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fd71406c8f0 0x7fd71406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:49.676 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.676+0000 7fd735600640 1 -- 172.21.15.152:0/2811054530 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fd72008b940 con 0x7fd7381f5590 2024-04-20T11:27:49.676 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.676+0000 7fd73ca00640 1 --2- 172.21.15.152:0/2811054530 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fd71406c8f0 0x7fd71406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:49.677 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.677+0000 7fd73ca00640 1 --2- 172.21.15.152:0/2811054530 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fd71406c8f0 0x7fd71406edb0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd728036000 tx=0x7fd728030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:49.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.680+0000 7fd735600640 1 -- 172.21.15.152:0/2811054530 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fd7380b4a00 con 0x7fd7381f5590 2024-04-20T11:27:49.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.882+0000 7fd73de00640 1 -- 172.21.15.152:0/2811054530 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd738075e60 con 0x7fd7381f5590 2024-04-20T11:27:49.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.883+0000 7fd735600640 1 -- 172.21.15.152:0/2811054530 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) ==== 93+0+10 (secure 0 0 0) 0x7fd720059d80 con 0x7fd7381f5590 2024-04-20T11:27:49.884 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 1 2024-04-20T11:27:49.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.886+0000 7fd73de00640 1 -- 172.21.15.152:0/2811054530 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fd71406c8f0 msgr2=0x7fd71406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:49.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.886+0000 7fd73de00640 1 --2- 172.21.15.152:0/2811054530 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fd71406c8f0 0x7fd71406edb0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd728036000 tx=0x7fd728030000 comp rx=0 tx=0).stop 2024-04-20T11:27:49.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.886+0000 7fd73de00640 1 -- 172.21.15.152:0/2811054530 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd7381f5590 msgr2=0x7fd7380b0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:49.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.886+0000 7fd73de00640 1 --2- 172.21.15.152:0/2811054530 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd7381f5590 0x7fd7380b0170 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fd72000e890 tx=0x7fd72000ed50 comp rx=0 tx=0).stop 2024-04-20T11:27:49.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.886+0000 7fd73de00640 1 -- 172.21.15.152:0/2811054530 shutdown_connections 2024-04-20T11:27:49.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.886+0000 7fd73de00640 1 --2- 172.21.15.152:0/2811054530 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fd71406c8f0 0x7fd71406edb0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:49.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.886+0000 7fd73de00640 1 --2- 172.21.15.152:0/2811054530 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd7380b06b0 0x7fd738156fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:49.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.886+0000 7fd73de00640 1 --2- 172.21.15.152:0/2811054530 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd7381f5590 0x7fd7380b0170 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:49.888 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.886+0000 7fd73de00640 1 --2- 172.21.15.152:0/2811054530 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd71c0039b0 0x7fd73809ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:49.888 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.886+0000 7fd73de00640 1 -- 172.21.15.152:0/2811054530 >> 172.21.15.152:0/2811054530 conn(0x7fd7380b21e0 msgr2=0x7fd7380b42f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:49.888 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.886+0000 7fd73de00640 1 -- 172.21.15.152:0/2811054530 shutdown_connections 2024-04-20T11:27:49.888 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:49.886+0000 7fd73de00640 1 -- 172.21.15.152:0/2811054530 wait complete. 2024-04-20T11:27:49.906 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T11:27:50.112 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.111+0000 7fe1cda00640 1 Processor -- start 2024-04-20T11:27:50.112 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.111+0000 7fe1cda00640 1 -- start start 2024-04-20T11:27:50.113 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.113+0000 7fe1cda00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fe1c8071cd0 0x7fe1c80720b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:50.113 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.113+0000 7fe1cda00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe1c8160220 0x7fe1c81606b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:50.113 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.113+0000 7fe1cda00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1c80725f0 con 0x7fe1c806eb30 2024-04-20T11:27:50.114 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.113+0000 7fe1cda00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fe1c813f8e0 con 0x7fe1c8071cd0 2024-04-20T11:27:50.114 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.113+0000 7fe1cda00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe1c8160f90 con 0x7fe1c8160220 2024-04-20T11:27:50.114 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.113+0000 7fe1cd000640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fe1c806eb30 0x7fe1c806efc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:44884/0 (socket says 172.21.15.152:44884) 2024-04-20T11:27:50.114 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.113+0000 7fe1cd000640 1 -- 172.21.15.152:0/1326462057 learned_addr learned my addr 172.21.15.152:0/1326462057 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:27:50.116 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.113+0000 7fe1c7400640 1 --2- 172.21.15.152:0/1326462057 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe1c8160220 0x7fe1c81606b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:50.116 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.113+0000 7fe1c7e00640 1 --2- 172.21.15.152:0/1326462057 >> v2:172.21.15.175:3300/0 conn(0x7fe1c8071cd0 0x7fe1c80720b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:50.116 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.114+0000 7fe1c6a00640 1 -- 172.21.15.152:0/1326462057 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3582984283 0 0) 0x7fe1c80725f0 con 0x7fe1c806eb30 2024-04-20T11:27:50.116 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.114+0000 7fe1c6a00640 1 -- 172.21.15.152:0/1326462057 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe1ac003520 con 0x7fe1c806eb30 2024-04-20T11:27:50.116 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.114+0000 7fe1c6a00640 1 -- 172.21.15.152:0/1326462057 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 638614766 0 0) 0x7fe1ac003520 con 0x7fe1c806eb30 2024-04-20T11:27:50.116 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.114+0000 7fe1c6a00640 1 -- 172.21.15.152:0/1326462057 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe1c8160220 msgr2=0x7fe1c81606b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:50.116 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.114+0000 7fe1c6a00640 1 --2- 172.21.15.152:0/1326462057 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe1c8160220 0x7fe1c81606b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:50.116 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.114+0000 7fe1c6a00640 1 -- 172.21.15.152:0/1326462057 >> v2:172.21.15.175:3300/0 conn(0x7fe1c8071cd0 msgr2=0x7fe1c80720b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:50.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.114+0000 7fe1c6a00640 1 --2- 172.21.15.152:0/1326462057 >> v2:172.21.15.175:3300/0 conn(0x7fe1c8071cd0 0x7fe1c80720b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:50.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.114+0000 7fe1c6a00640 1 -- 172.21.15.152:0/1326462057 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1c806dd90 con 0x7fe1c806eb30 2024-04-20T11:27:50.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.114+0000 7fe1c6a00640 1 -- 172.21.15.152:0/1326462057 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7fe1c00019f0 con 0x7fe1c806eb30 2024-04-20T11:27:50.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.114+0000 7fe1c6a00640 1 -- 172.21.15.152:0/1326462057 >> v1:172.21.15.152:6789/0 conn(0x7fe1c806eb30 legacy=0x7fe1c806efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:50.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.114+0000 7fe1c6a00640 1 --2- 172.21.15.152:0/1326462057 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe1ac0039b0 0x7fe1ac003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:50.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.114+0000 7fe1c6a00640 1 --2- 172.21.15.152:0/1326462057 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe1c8160220 0x7fe1ac004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:50.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.114+0000 7fe1c6a00640 1 --2- 172.21.15.152:0/1326462057 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1ac004d40 0x7fe1ac0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:50.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.114+0000 7fe1c6a00640 1 -- 172.21.15.152:0/1326462057 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fe1c813f8e0 con 0x7fe1ac0039b0 2024-04-20T11:27:50.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.114+0000 7fe1c6a00640 1 -- 172.21.15.152:0/1326462057 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe1c8160f90 con 0x7fe1ac004d40 2024-04-20T11:27:50.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.114+0000 7fe1c6a00640 1 -- 172.21.15.152:0/1326462057 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe1ac00b9f0 con 0x7fe1c8160220 2024-04-20T11:27:50.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.114+0000 7fe1c7e00640 1 --2- 172.21.15.152:0/1326462057 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe1ac0039b0 0x7fe1ac003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:50.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.115+0000 7fe1c7400640 1 --2- 172.21.15.152:0/1326462057 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe1c8160220 0x7fe1ac004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:50.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.115+0000 7fe1cd000640 1 --2- 172.21.15.152:0/1326462057 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1ac004d40 0x7fe1ac0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:50.118 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.115+0000 7fe1c7e00640 1 -- 172.21.15.152:0/1326462057 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe1c8160220 msgr2=0x7fe1ac004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:50.118 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.115+0000 7fe1c7e00640 1 --2- 172.21.15.152:0/1326462057 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe1c8160220 0x7fe1ac004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:50.118 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.115+0000 7fe1c7e00640 1 -- 172.21.15.152:0/1326462057 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1ac004d40 msgr2=0x7fe1ac0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:50.118 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.115+0000 7fe1c7e00640 1 --2- 172.21.15.152:0/1326462057 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1ac004d40 0x7fe1ac0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:50.118 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.115+0000 7fe1c7e00640 1 -- 172.21.15.152:0/1326462057 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1ac003520 con 0x7fe1ac0039b0 2024-04-20T11:27:50.118 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.115+0000 7fe1cd000640 1 --2- 172.21.15.152:0/1326462057 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1ac004d40 0x7fe1ac0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:27:50.118 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.115+0000 7fe1c7400640 1 --2- 172.21.15.152:0/1326462057 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe1c8160220 0x7fe1ac004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:27:50.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.116+0000 7fe1c7e00640 1 --2- 172.21.15.152:0/1326462057 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe1ac0039b0 0x7fe1ac003dd0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe1b8009ac0 tx=0x7fe1ac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e23e214d310dc1c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:50.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.116+0000 7fe1c6a00640 1 -- 172.21.15.152:0/1326462057 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b8039070 con 0x7fe1ac0039b0 2024-04-20T11:27:50.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.116+0000 7fe1c6a00640 1 -- 172.21.15.152:0/1326462057 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1b802c710 con 0x7fe1ac0039b0 2024-04-20T11:27:50.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.116+0000 7fe1c6a00640 1 -- 172.21.15.152:0/1326462057 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b8009690 con 0x7fe1ac0039b0 2024-04-20T11:27:50.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.117+0000 7fe1cda00640 1 -- 172.21.15.152:0/1326462057 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe1ac0039b0 msgr2=0x7fe1ac003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:50.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.117+0000 7fe1cda00640 1 --2- 172.21.15.152:0/1326462057 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe1ac0039b0 0x7fe1ac003dd0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe1b8009ac0 tx=0x7fe1ac005830 comp rx=0 tx=0).stop 2024-04-20T11:27:50.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.117+0000 7fe1cd000640 1 -- 172.21.15.152:0/1326462057 reap_dead start 2024-04-20T11:27:50.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.117+0000 7fe1cda00640 1 -- 172.21.15.152:0/1326462057 shutdown_connections 2024-04-20T11:27:50.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.117+0000 7fe1cda00640 1 -- 172.21.15.152:0/1326462057 >> 172.21.15.152:0/1326462057 conn(0x7fe1c8067ae0 msgr2=0x7fe1c806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:50.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.117+0000 7fe1cda00640 1 -- 172.21.15.152:0/1326462057 shutdown_connections 2024-04-20T11:27:50.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.117+0000 7fe1cda00640 1 -- 172.21.15.152:0/1326462057 wait complete. 2024-04-20T11:27:50.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.119+0000 7fe1cda00640 1 Processor -- start 2024-04-20T11:27:50.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.119+0000 7fe1cda00640 1 -- start start 2024-04-20T11:27:50.122 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.120+0000 7fe1cda00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe1ac0039b0 0x7fe1c8156a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:50.122 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.120+0000 7fe1cda00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1c8155480 0x7fe1c813e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:50.122 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.120+0000 7fe1cda00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe1c815dcc0 0x7fe1c815e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:50.122 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.120+0000 7fe1cda00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fe1c806e630 con 0x7fe1ac0039b0 2024-04-20T11:27:50.122 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.120+0000 7fe1cda00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe1c806e350 con 0x7fe1c8155480 2024-04-20T11:27:50.122 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.120+0000 7fe1cda00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe1c813fb90 con 0x7fe1c815dcc0 2024-04-20T11:27:50.122 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.121+0000 7fe1c7e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe1ac0039b0 0x7fe1c8156a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:50.122 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.121+0000 7fe1c7e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe1ac0039b0 0x7fe1c8156a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43344/0 (socket says 172.21.15.152:43344) 2024-04-20T11:27:50.122 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.121+0000 7fe1c7e00640 1 -- 172.21.15.152:0/1955461428 learned_addr learned my addr 172.21.15.152:0/1955461428 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:50.123 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.121+0000 7fe1c7400640 1 --2- 172.21.15.152:0/1955461428 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1c8155480 0x7fe1c813e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:50.123 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.121+0000 7fe1cd000640 1 --2- 172.21.15.152:0/1955461428 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe1c815dcc0 0x7fe1c815e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:50.123 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.121+0000 7fe1c7e00640 1 -- 172.21.15.152:0/1955461428 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe1c815dcc0 msgr2=0x7fe1c815e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:50.123 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.121+0000 7fe1c7e00640 1 --2- 172.21.15.152:0/1955461428 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe1c815dcc0 0x7fe1c815e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:50.123 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.121+0000 7fe1c7e00640 1 -- 172.21.15.152:0/1955461428 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1c8155480 msgr2=0x7fe1c813e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:50.123 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.121+0000 7fe1c7e00640 1 --2- 172.21.15.152:0/1955461428 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1c8155480 0x7fe1c813e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:50.123 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.121+0000 7fe1c7e00640 1 -- 172.21.15.152:0/1955461428 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1c81559d0 con 0x7fe1ac0039b0 2024-04-20T11:27:50.123 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.122+0000 7fe1c7e00640 1 --2- 172.21.15.152:0/1955461428 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe1ac0039b0 0x7fe1c8156a70 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fe1b8009be0 tx=0x7fe1b8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:50.124 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.122+0000 7fe1bfe00640 1 -- 172.21.15.152:0/1955461428 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b8042020 con 0x7fe1ac0039b0 2024-04-20T11:27:50.124 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.122+0000 7fe1bfe00640 1 -- 172.21.15.152:0/1955461428 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1b803a040 con 0x7fe1ac0039b0 2024-04-20T11:27:50.124 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.122+0000 7fe1bfe00640 1 -- 172.21.15.152:0/1955461428 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b8039070 con 0x7fe1ac0039b0 2024-04-20T11:27:50.124 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.122+0000 7fe1cda00640 1 -- 172.21.15.152:0/1955461428 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1c8155cb0 con 0x7fe1ac0039b0 2024-04-20T11:27:50.124 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.122+0000 7fe1cda00640 1 -- 172.21.15.152:0/1955461428 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1c8156270 con 0x7fe1ac0039b0 2024-04-20T11:27:50.124 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.123+0000 7fe1cda00640 1 -- 172.21.15.152:0/1955461428 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1c8075e60 con 0x7fe1ac0039b0 2024-04-20T11:27:50.125 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.124+0000 7fe1bfe00640 1 -- 172.21.15.152:0/1955461428 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe1b8045020 con 0x7fe1ac0039b0 2024-04-20T11:27:50.125 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.125+0000 7fe1bfe00640 1 --2- 172.21.15.152:0/1955461428 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe1a806c8f0 0x7fe1a806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:50.126 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.125+0000 7fe1bfe00640 1 -- 172.21.15.152:0/1955461428 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fe1b80ad100 con 0x7fe1ac0039b0 2024-04-20T11:27:50.126 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.125+0000 7fe1c7400640 1 --2- 172.21.15.152:0/1955461428 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe1a806c8f0 0x7fe1a806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:50.126 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.126+0000 7fe1c7400640 1 --2- 172.21.15.152:0/1955461428 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe1a806c8f0 0x7fe1a806edb0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe1c815c7c0 tx=0x7fe1b4008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:50.130 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.130+0000 7fe1bfe00640 1 -- 172.21.15.152:0/1955461428 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe1b804c070 con 0x7fe1ac0039b0 2024-04-20T11:27:50.332 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.332+0000 7fe1cda00640 1 -- 172.21.15.152:0/1955461428 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe1c8072d30 con 0x7fe1ac0039b0 2024-04-20T11:27:50.332 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.332+0000 7fe1bfe00640 1 -- 172.21.15.152:0/1955461428 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) ==== 104+0+11 (secure 0 0 0) 0x7fe1b807b410 con 0x7fe1ac0039b0 2024-04-20T11:27:50.333 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 32 2024-04-20T11:27:50.336 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.335+0000 7fe1cda00640 1 -- 172.21.15.152:0/1955461428 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe1a806c8f0 msgr2=0x7fe1a806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:50.336 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.335+0000 7fe1cda00640 1 --2- 172.21.15.152:0/1955461428 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe1a806c8f0 0x7fe1a806edb0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe1c815c7c0 tx=0x7fe1b4008040 comp rx=0 tx=0).stop 2024-04-20T11:27:50.336 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.335+0000 7fe1cda00640 1 -- 172.21.15.152:0/1955461428 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe1ac0039b0 msgr2=0x7fe1c8156a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:50.336 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.335+0000 7fe1cda00640 1 --2- 172.21.15.152:0/1955461428 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe1ac0039b0 0x7fe1c8156a70 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fe1b8009be0 tx=0x7fe1b8004730 comp rx=0 tx=0).stop 2024-04-20T11:27:50.336 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.335+0000 7fe1cda00640 1 -- 172.21.15.152:0/1955461428 shutdown_connections 2024-04-20T11:27:50.336 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.335+0000 7fe1cda00640 1 --2- 172.21.15.152:0/1955461428 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe1a806c8f0 0x7fe1a806edb0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:50.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.335+0000 7fe1cda00640 1 --2- 172.21.15.152:0/1955461428 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe1c815dcc0 0x7fe1c815e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:50.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.335+0000 7fe1cda00640 1 --2- 172.21.15.152:0/1955461428 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1c8155480 0x7fe1c813e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:50.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.335+0000 7fe1cda00640 1 --2- 172.21.15.152:0/1955461428 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe1ac0039b0 0x7fe1c8156a70 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:50.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.335+0000 7fe1cda00640 1 -- 172.21.15.152:0/1955461428 >> 172.21.15.152:0/1955461428 conn(0x7fe1c8067ae0 msgr2=0x7fe1c81606e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:50.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.336+0000 7fe1cda00640 1 -- 172.21.15.152:0/1955461428 shutdown_connections 2024-04-20T11:27:50.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.336+0000 7fe1cda00640 1 -- 172.21.15.152:0/1955461428 wait complete. 2024-04-20T11:27:50.355 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T11:27:50.540 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.539+0000 7f3bbbe00640 1 Processor -- start 2024-04-20T11:27:50.540 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.539+0000 7f3bbbe00640 1 -- start start 2024-04-20T11:27:50.542 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.541+0000 7f3bbbe00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f3bb40b2880 0x7f3bb40b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:50.542 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.541+0000 7f3bbbe00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f3bb40cd1b0 0x7f3bb40cd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:50.542 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.541+0000 7f3bbbe00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3bb41463e0 con 0x7f3bb4146000 2024-04-20T11:27:50.542 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.541+0000 7f3bbbe00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f3bb40b4750 con 0x7f3bb40b2880 2024-04-20T11:27:50.542 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.541+0000 7f3bbbe00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f3bb40b3e20 con 0x7f3bb40cd1b0 2024-04-20T11:27:50.542 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.541+0000 7f3bb8a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f3bb40b2880 0x7f3bb40b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:50.542 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.541+0000 7f3bb8a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f3bb40b2880 0x7f3bb40b4be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43360/0 (socket says 172.21.15.152:43360) 2024-04-20T11:27:50.542 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.541+0000 7f3bb8a00640 1 -- 172.21.15.152:0/3273318657 learned_addr learned my addr 172.21.15.152:0/3273318657 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:50.543 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.542+0000 7f3bb9e00640 1 --2- 172.21.15.152:0/3273318657 >> v2:172.21.15.175:3300/0 conn(0x7f3bb40cd1b0 0x7f3bb40cd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:50.543 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.542+0000 7f3bb8a00640 1 -- 172.21.15.152:0/3273318657 >> v2:172.21.15.175:3300/0 conn(0x7f3bb40cd1b0 msgr2=0x7f3bb40cd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:50.543 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.542+0000 7f3bb8a00640 1 --2- 172.21.15.152:0/3273318657 >> v2:172.21.15.175:3300/0 conn(0x7f3bb40cd1b0 0x7f3bb40cd5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:50.543 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.542+0000 7f3bb8a00640 1 -- 172.21.15.152:0/3273318657 >> v1:172.21.15.175:6789/0 conn(0x7f3bb4146000 legacy=0x7f3bb40b2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:50.543 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.542+0000 7f3bb8a00640 1 -- 172.21.15.152:0/3273318657 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bb40b4100 con 0x7f3bb40b2880 2024-04-20T11:27:50.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.543+0000 7f3bb8a00640 1 --2- 172.21.15.152:0/3273318657 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3bb40b2880 0x7f3bb40b4be0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f3bb40b48d0 tx=0x7f3ba002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=828a5ab08868d9f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:50.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.543+0000 7f3bb3e00640 1 -- 172.21.15.152:0/3273318657 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba002e070 con 0x7f3bb40b2880 2024-04-20T11:27:50.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.543+0000 7f3bb3e00640 1 -- 172.21.15.152:0/3273318657 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ba0037070 con 0x7f3bb40b2880 2024-04-20T11:27:50.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.543+0000 7f3bb3e00640 1 -- 172.21.15.152:0/3273318657 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba0031a00 con 0x7f3bb40b2880 2024-04-20T11:27:50.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.544+0000 7f3bbbe00640 1 -- 172.21.15.152:0/3273318657 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3bb40b2880 msgr2=0x7f3bb40b4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:50.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.544+0000 7f3bbbe00640 1 --2- 172.21.15.152:0/3273318657 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3bb40b2880 0x7f3bb40b4be0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f3bb40b48d0 tx=0x7f3ba002b880 comp rx=0 tx=0).stop 2024-04-20T11:27:50.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.544+0000 7f3bbbe00640 1 -- 172.21.15.152:0/3273318657 shutdown_connections 2024-04-20T11:27:50.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.544+0000 7f3bbbe00640 1 --2- 172.21.15.152:0/3273318657 >> v2:172.21.15.175:3300/0 conn(0x7f3bb40cd1b0 0x7f3bb40cd5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:50.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.544+0000 7f3bbbe00640 1 --2- 172.21.15.152:0/3273318657 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3bb40b2880 0x7f3bb40b4be0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:50.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.544+0000 7f3bbbe00640 1 -- 172.21.15.152:0/3273318657 >> 172.21.15.152:0/3273318657 conn(0x7f3bb406b980 msgr2=0x7f3bb406bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:50.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.544+0000 7f3bbbe00640 1 -- 172.21.15.152:0/3273318657 shutdown_connections 2024-04-20T11:27:50.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.544+0000 7f3bbbe00640 1 -- 172.21.15.152:0/3273318657 wait complete. 2024-04-20T11:27:50.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.546+0000 7f3bbbe00640 1 Processor -- start 2024-04-20T11:27:50.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.546+0000 7f3bbbe00640 1 -- start start 2024-04-20T11:27:50.548 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.547+0000 7f3bbbe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3bb40b2880 0x7f3bb415c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:50.548 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.547+0000 7f3bbbe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3bb40cd1b0 0x7f3bb41e7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:50.548 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.547+0000 7f3bbbe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3bb4146000 0x7f3bb41e9f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:50.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.547+0000 7f3bbbe00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb40cdb30 con 0x7f3bb4146000 2024-04-20T11:27:50.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.547+0000 7f3bbbe00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb415ccf0 con 0x7f3bb40b2880 2024-04-20T11:27:50.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.547+0000 7f3bbbe00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f3bb41ea470 con 0x7f3bb40cd1b0 2024-04-20T11:27:50.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.547+0000 7f3bb9e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3bb4146000 0x7f3bb41e9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:50.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.547+0000 7f3bb9e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3bb4146000 0x7f3bb41e9f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43366/0 (socket says 172.21.15.152:43366) 2024-04-20T11:27:50.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.547+0000 7f3bb9e00640 1 -- 172.21.15.152:0/2582740983 learned_addr learned my addr 172.21.15.152:0/2582740983 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:50.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.548+0000 7f3bb9400640 1 --2- 172.21.15.152:0/2582740983 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3bb40b2880 0x7f3bb415c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:50.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.548+0000 7f3bb9e00640 1 -- 172.21.15.152:0/2582740983 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3bb40cd1b0 msgr2=0x7f3bb41e7950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:50.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.548+0000 7f3bb9e00640 1 --2- 172.21.15.152:0/2582740983 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3bb40cd1b0 0x7f3bb41e7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:50.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.548+0000 7f3bb9e00640 1 -- 172.21.15.152:0/2582740983 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3bb40b2880 msgr2=0x7f3bb415c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:50.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.548+0000 7f3bb9e00640 1 --2- 172.21.15.152:0/2582740983 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3bb40b2880 0x7f3bb415c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:50.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.548+0000 7f3bb9e00640 1 -- 172.21.15.152:0/2582740983 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bb41e80a0 con 0x7f3bb4146000 2024-04-20T11:27:50.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.548+0000 7f3bb9400640 1 --2- 172.21.15.152:0/2582740983 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3bb40b2880 0x7f3bb415c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:27:50.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.548+0000 7f3bb9e00640 1 --2- 172.21.15.152:0/2582740983 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3bb4146000 0x7f3bb41e9f30 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f3bac007e40 tx=0x7f3bac00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:50.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.548+0000 7f3bb1600640 1 -- 172.21.15.152:0/2582740983 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bac014620 con 0x7f3bb4146000 2024-04-20T11:27:50.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.548+0000 7f3bb1600640 1 -- 172.21.15.152:0/2582740983 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3bac019070 con 0x7f3bb4146000 2024-04-20T11:27:50.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.548+0000 7f3bb1600640 1 -- 172.21.15.152:0/2582740983 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bac014a20 con 0x7f3bb4146000 2024-04-20T11:27:50.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.548+0000 7f3bbbe00640 1 -- 172.21.15.152:0/2582740983 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bb41e8380 con 0x7f3bb4146000 2024-04-20T11:27:50.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.548+0000 7f3bbbe00640 1 -- 172.21.15.152:0/2582740983 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bb41e8940 con 0x7f3bb4146000 2024-04-20T11:27:50.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.550+0000 7f3bbbe00640 1 -- 172.21.15.152:0/2582740983 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bb40689f0 con 0x7f3bb4146000 2024-04-20T11:27:50.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.551+0000 7f3bb1600640 1 -- 172.21.15.152:0/2582740983 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3bac014bc0 con 0x7f3bb4146000 2024-04-20T11:27:50.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.552+0000 7f3bb1600640 1 --2- 172.21.15.152:0/2582740983 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f3b9406c8f0 0x7f3b9406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:50.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.552+0000 7f3bb1600640 1 -- 172.21.15.152:0/2582740983 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f3bac08c520 con 0x7f3bb4146000 2024-04-20T11:27:50.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.552+0000 7f3bb9400640 1 --2- 172.21.15.152:0/2582740983 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f3b9406c8f0 0x7f3b9406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:50.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.553+0000 7f3bb9400640 1 --2- 172.21.15.152:0/2582740983 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f3b9406c8f0 0x7f3b9406edb0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3ba800a830 tx=0x7f3ba8008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:50.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.556+0000 7f3bb1600640 1 -- 172.21.15.152:0/2582740983 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f3bac05a8c0 con 0x7f3bb4146000 2024-04-20T11:27:50.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.754+0000 7f3bbbe00640 1 -- 172.21.15.152:0/2582740983 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f3bb40ccba0 con 0x7f3bb4146000 2024-04-20T11:27:50.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.755+0000 7f3bb1600640 1 -- 172.21.15.152:0/2582740983 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) ==== 100+0+11 (secure 0 0 0) 0x7f3bac05a1c0 con 0x7f3bb4146000 2024-04-20T11:27:50.755 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 64 2024-04-20T11:27:50.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.757+0000 7f3bbbe00640 1 -- 172.21.15.152:0/2582740983 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f3b9406c8f0 msgr2=0x7f3b9406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:50.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.757+0000 7f3bbbe00640 1 --2- 172.21.15.152:0/2582740983 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f3b9406c8f0 0x7f3b9406edb0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3ba800a830 tx=0x7f3ba8008040 comp rx=0 tx=0).stop 2024-04-20T11:27:50.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.757+0000 7f3bbbe00640 1 -- 172.21.15.152:0/2582740983 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3bb4146000 msgr2=0x7f3bb41e9f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:50.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.757+0000 7f3bbbe00640 1 --2- 172.21.15.152:0/2582740983 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3bb4146000 0x7f3bb41e9f30 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f3bac007e40 tx=0x7f3bac00e4b0 comp rx=0 tx=0).stop 2024-04-20T11:27:50.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.757+0000 7f3bbbe00640 1 -- 172.21.15.152:0/2582740983 shutdown_connections 2024-04-20T11:27:50.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.757+0000 7f3bbbe00640 1 --2- 172.21.15.152:0/2582740983 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f3b9406c8f0 0x7f3b9406edb0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:50.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.757+0000 7f3bbbe00640 1 --2- 172.21.15.152:0/2582740983 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3bb4146000 0x7f3bb41e9f30 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:50.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.757+0000 7f3bbbe00640 1 --2- 172.21.15.152:0/2582740983 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3bb40cd1b0 0x7f3bb41e7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:50.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.757+0000 7f3bbbe00640 1 --2- 172.21.15.152:0/2582740983 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3bb40b2880 0x7f3bb415c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:50.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.757+0000 7f3bbbe00640 1 -- 172.21.15.152:0/2582740983 >> 172.21.15.152:0/2582740983 conn(0x7f3bb406b980 msgr2=0x7f3bb406e350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:50.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.758+0000 7f3bbbe00640 1 -- 172.21.15.152:0/2582740983 shutdown_connections 2024-04-20T11:27:50.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.758+0000 7f3bbbe00640 1 -- 172.21.15.152:0/2582740983 wait complete. 2024-04-20T11:27:50.777 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-20T11:27:50.964 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.963+0000 7fc61da00640 1 Processor -- start 2024-04-20T11:27:50.964 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.963+0000 7fc61da00640 1 -- start start 2024-04-20T11:27:50.966 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.965+0000 7fc61da00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc618159040 0x7fc6180cf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:50.966 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.965+0000 7fc61da00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fc618162c40 0x7fc6180ce140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:50.966 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.965+0000 7fc61da00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc618159420 con 0x7fc61806f6a0 2024-04-20T11:27:50.966 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.965+0000 7fc61da00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fc6180ce680 con 0x7fc618162c40 2024-04-20T11:27:50.966 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.965+0000 7fc61da00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc61806fb00 con 0x7fc618159040 2024-04-20T11:27:50.966 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.965+0000 7fc61d000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fc618162c40 0x7fc6180ce140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:50.966 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.965+0000 7fc61d000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fc618162c40 0x7fc6180ce140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43374/0 (socket says 172.21.15.152:43374) 2024-04-20T11:27:50.966 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.965+0000 7fc61d000640 1 -- 172.21.15.152:0/3797327151 learned_addr learned my addr 172.21.15.152:0/3797327151 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:50.967 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.966+0000 7fc617e00640 1 --2- 172.21.15.152:0/3797327151 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc618159040 0x7fc6180cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:50.967 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.966+0000 7fc61d000640 1 -- 172.21.15.152:0/3797327151 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc618159040 msgr2=0x7fc6180cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:50.967 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.966+0000 7fc61d000640 1 --2- 172.21.15.152:0/3797327151 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc618159040 0x7fc6180cf950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:50.967 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.966+0000 7fc61d000640 1 -- 172.21.15.152:0/3797327151 >> v1:172.21.15.175:6789/0 conn(0x7fc61806f6a0 legacy=0x7fc618162530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:50.967 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.966+0000 7fc61d000640 1 -- 172.21.15.152:0/3797327151 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6180ce8b0 con 0x7fc618162c40 2024-04-20T11:27:50.968 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.967+0000 7fc61d000640 1 --2- 172.21.15.152:0/3797327151 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc618162c40 0x7fc6180ce140 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fc610009870 tx=0x7fc61002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d50aa98209f746f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:50.968 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.968+0000 7fc617400640 1 -- 172.21.15.152:0/3797327151 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc610034070 con 0x7fc618162c40 2024-04-20T11:27:50.968 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.968+0000 7fc617400640 1 -- 172.21.15.152:0/3797327151 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc610030840 con 0x7fc618162c40 2024-04-20T11:27:50.969 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.968+0000 7fc617400640 1 -- 172.21.15.152:0/3797327151 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc610030c10 con 0x7fc618162c40 2024-04-20T11:27:50.969 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.968+0000 7fc61da00640 1 -- 172.21.15.152:0/3797327151 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc618162c40 msgr2=0x7fc6180ce140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:50.969 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.968+0000 7fc61da00640 1 --2- 172.21.15.152:0/3797327151 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc618162c40 0x7fc6180ce140 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fc610009870 tx=0x7fc61002b980 comp rx=0 tx=0).stop 2024-04-20T11:27:50.970 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.969+0000 7fc61da00640 1 -- 172.21.15.152:0/3797327151 shutdown_connections 2024-04-20T11:27:50.970 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.969+0000 7fc61da00640 1 --2- 172.21.15.152:0/3797327151 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc618162c40 0x7fc6180ce140 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:50.970 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.969+0000 7fc61da00640 1 --2- 172.21.15.152:0/3797327151 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc618159040 0x7fc6180cf950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:50.970 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.969+0000 7fc61da00640 1 -- 172.21.15.152:0/3797327151 >> 172.21.15.152:0/3797327151 conn(0x7fc6180d0260 msgr2=0x7fc6180d0670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:50.970 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.969+0000 7fc61da00640 1 -- 172.21.15.152:0/3797327151 shutdown_connections 2024-04-20T11:27:50.970 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.969+0000 7fc61da00640 1 -- 172.21.15.152:0/3797327151 wait complete. 2024-04-20T11:27:50.971 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.970+0000 7fc61da00640 1 Processor -- start 2024-04-20T11:27:50.971 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.971+0000 7fc61da00640 1 -- start start 2024-04-20T11:27:50.972 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.971+0000 7fc61da00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc61806f6a0 0x7fc6180726b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:50.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.971+0000 7fc61da00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc618159040 0x7fc618150c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:50.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.972+0000 7fc61da00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc618162c40 0x7fc6181e8d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:50.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.972+0000 7fc61da00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fc6180cf150 con 0x7fc618159040 2024-04-20T11:27:50.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.972+0000 7fc61da00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc6180cee70 con 0x7fc61806f6a0 2024-04-20T11:27:50.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.972+0000 7fc61da00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc6180cf430 con 0x7fc618162c40 2024-04-20T11:27:50.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.972+0000 7fc60f400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc618159040 0x7fc618150c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:50.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.972+0000 7fc60f400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc618159040 0x7fc618150c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43388/0 (socket says 172.21.15.152:43388) 2024-04-20T11:27:50.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.972+0000 7fc60f400640 1 -- 172.21.15.152:0/786966895 learned_addr learned my addr 172.21.15.152:0/786966895 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:50.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.972+0000 7fc617e00640 1 --2- 172.21.15.152:0/786966895 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc61806f6a0 0x7fc6180726b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:50.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.972+0000 7fc60f400640 1 -- 172.21.15.152:0/786966895 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc618162c40 msgr2=0x7fc6181e8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:50.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.972+0000 7fc60f400640 1 --2- 172.21.15.152:0/786966895 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc618162c40 0x7fc6181e8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:50.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.972+0000 7fc60f400640 1 -- 172.21.15.152:0/786966895 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc61806f6a0 msgr2=0x7fc6180726b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:50.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.972+0000 7fc60f400640 1 --2- 172.21.15.152:0/786966895 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc61806f6a0 0x7fc6180726b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:50.974 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.972+0000 7fc60f400640 1 -- 172.21.15.152:0/786966895 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc618161ff0 con 0x7fc618159040 2024-04-20T11:27:50.974 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.973+0000 7fc60f400640 1 --2- 172.21.15.152:0/786966895 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc618159040 0x7fc618150c80 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fc60000c3c0 tx=0x7fc60000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:50.975 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.973+0000 7fc614c00640 1 -- 172.21.15.152:0/786966895 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6000052d0 con 0x7fc618159040 2024-04-20T11:27:50.975 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.973+0000 7fc614c00640 1 -- 172.21.15.152:0/786966895 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc600005490 con 0x7fc618159040 2024-04-20T11:27:50.975 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.973+0000 7fc61da00640 1 -- 172.21.15.152:0/786966895 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6181622d0 con 0x7fc618159040 2024-04-20T11:27:50.975 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.973+0000 7fc614c00640 1 -- 172.21.15.152:0/786966895 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc600007260 con 0x7fc618159040 2024-04-20T11:27:50.975 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.973+0000 7fc61da00640 1 -- 172.21.15.152:0/786966895 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc618161ca0 con 0x7fc618159040 2024-04-20T11:27:50.975 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.974+0000 7fc61da00640 1 -- 172.21.15.152:0/786966895 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc618075e60 con 0x7fc618159040 2024-04-20T11:27:50.975 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.975+0000 7fc614c00640 1 -- 172.21.15.152:0/786966895 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc600007400 con 0x7fc618159040 2024-04-20T11:27:50.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.976+0000 7fc614c00640 1 --2- 172.21.15.152:0/786966895 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc5f406c8f0 0x7fc5f406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:50.977 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.976+0000 7fc614c00640 1 -- 172.21.15.152:0/786966895 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fc60008b070 con 0x7fc618159040 2024-04-20T11:27:50.977 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.977+0000 7fc617e00640 1 --2- 172.21.15.152:0/786966895 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc5f406c8f0 0x7fc5f406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:50.978 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.977+0000 7fc617e00640 1 --2- 172.21.15.152:0/786966895 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc5f406c8f0 0x7fc5f406edb0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc608009e90 tx=0x7fc60800b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:50.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:50.981+0000 7fc614c00640 1 -- 172.21.15.152:0/786966895 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fc618159420 con 0x7fc618159040 2024-04-20T11:27:51.180 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:27:51.187 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:27:51.194 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:51.193+0000 7fc61da00640 1 -- 172.21.15.152:0/786966895 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc6180cd990 con 0x7fc618159040 2024-04-20T11:27:51.194 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:51.194+0000 7fc614c00640 1 -- 172.21.15.152:0/786966895 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) ==== 80+0+0 (secure 0 0 0) 0x7fc600059420 con 0x7fc618159040 2024-04-20T11:27:51.195 INFO:teuthology.orchestra.run.smithi152.stderr:listed 0 entries 2024-04-20T11:27:51.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:51.197+0000 7fc61da00640 1 -- 172.21.15.152:0/786966895 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc5f406c8f0 msgr2=0x7fc5f406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:51.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:51.197+0000 7fc61da00640 1 --2- 172.21.15.152:0/786966895 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc5f406c8f0 0x7fc5f406edb0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc608009e90 tx=0x7fc60800b040 comp rx=0 tx=0).stop 2024-04-20T11:27:51.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:51.197+0000 7fc61da00640 1 -- 172.21.15.152:0/786966895 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc618159040 msgr2=0x7fc618150c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:51.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:51.197+0000 7fc61da00640 1 --2- 172.21.15.152:0/786966895 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc618159040 0x7fc618150c80 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fc60000c3c0 tx=0x7fc60000c880 comp rx=0 tx=0).stop 2024-04-20T11:27:51.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:51.197+0000 7fc61da00640 1 -- 172.21.15.152:0/786966895 shutdown_connections 2024-04-20T11:27:51.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:51.197+0000 7fc61da00640 1 --2- 172.21.15.152:0/786966895 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc5f406c8f0 0x7fc5f406edb0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:51.197+0000 7fc61da00640 1 --2- 172.21.15.152:0/786966895 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc618162c40 0x7fc6181e8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:51.197+0000 7fc61da00640 1 --2- 172.21.15.152:0/786966895 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc618159040 0x7fc618150c80 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:51.197+0000 7fc61da00640 1 --2- 172.21.15.152:0/786966895 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc61806f6a0 0x7fc6180726b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:51.197+0000 7fc61da00640 1 -- 172.21.15.152:0/786966895 >> 172.21.15.152:0/786966895 conn(0x7fc6180d0260 msgr2=0x7fc6180b42c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:51.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:51.197+0000 7fc61da00640 1 -- 172.21.15.152:0/786966895 shutdown_connections 2024-04-20T11:27:51.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:51.197+0000 7fc61da00640 1 -- 172.21.15.152:0/786966895 wait complete. 2024-04-20T11:27:51.216 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi175.front.sepia.ceph.com... 2024-04-20T11:27:51.216 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:27:51.216 DEBUG:teuthology.orchestra.run.smithi175:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-04-20T11:27:51.307 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T11:27:51.307 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi175.front.sepia.ceph.com 2024-04-20T11:27:51.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:27:51.307 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-20T11:27:51.364 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-20T11:27:51.365 DEBUG:teuthology.orchestra.run.smithi152:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:27:51.421 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-20T11:27:51.421 DEBUG:teuthology.orchestra.run.smithi152:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:27:51.473 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:27:51.664 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:51.663+0000 7fbf73800640 1 Processor -- start 2024-04-20T11:27:51.664 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:51.663+0000 7fbf73800640 1 -- start start 2024-04-20T11:27:51.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:51.665+0000 7fbf73800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbf6c15c560 0x7fbf6c0b3df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:51.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:51.665+0000 7fbf73800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fbf6c0d0810 0x7fbf6c0ada90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:51.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:51.665+0000 7fbf73800640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf6c15fd20 con 0x7fbf6c0b4330 2024-04-20T11:27:51.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:51.665+0000 7fbf73800640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fbf6c15ef60 con 0x7fbf6c0d0810 2024-04-20T11:27:51.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:51.665+0000 7fbf73800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fbf6c15f240 con 0x7fbf6c15c560 2024-04-20T11:27:51.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:51.665+0000 7fbf6be00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fbf6c0b4330 0x7fbf6c0d0100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:44888/0 (socket says 172.21.15.152:44888) 2024-04-20T11:27:51.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:51.665+0000 7fbf6be00640 1 -- 172.21.15.152:0/3020312069 learned_addr learned my addr 172.21.15.152:0/3020312069 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:27:51.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:51.665+0000 7fbf71800640 1 --2- 172.21.15.152:0/3020312069 >> v2:172.21.15.152:3300/0 conn(0x7fbf6c0d0810 0x7fbf6c0ada90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:51.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:51.666+0000 7fbf70e00640 1 --2- 172.21.15.152:0/3020312069 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbf6c15c560 0x7fbf6c0b3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:51.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:51.666+0000 7fbf6b400640 1 -- 172.21.15.152:0/3020312069 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3955111222 0 0) 0x7fbf6c15fd20 con 0x7fbf6c0b4330 2024-04-20T11:27:51.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:51.666+0000 7fbf6b400640 1 -- 172.21.15.152:0/3020312069 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbf54003520 con 0x7fbf6c0b4330 2024-04-20T11:27:51.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:51.666+0000 7fbf71800640 1 -- 172.21.15.152:0/3020312069 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbf6c15c560 msgr2=0x7fbf6c0b3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:51.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:51.666+0000 7fbf71800640 1 --2- 172.21.15.152:0/3020312069 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbf6c15c560 0x7fbf6c0b3df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:51.666+0000 7fbf71800640 1 -- 172.21.15.152:0/3020312069 >> v1:172.21.15.152:6789/0 conn(0x7fbf6c0b4330 legacy=0x7fbf6c0d0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:51.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:51.666+0000 7fbf71800640 1 -- 172.21.15.152:0/3020312069 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf6c15f520 con 0x7fbf6c0d0810 2024-04-20T11:27:51.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:51.666+0000 7fbf70e00640 1 --2- 172.21.15.152:0/3020312069 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbf6c15c560 0x7fbf6c0b3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:27:51.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:51.667+0000 7fbf71800640 1 --2- 172.21.15.152:0/3020312069 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbf6c0d0810 0x7fbf6c0ada90 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fbf64009d90 tx=0x7fbf6402be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=689d44b4b79fed3a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:51.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:51.668+0000 7fbf6b400640 1 -- 172.21.15.152:0/3020312069 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf64039070 con 0x7fbf6c0d0810 2024-04-20T11:27:51.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:51.668+0000 7fbf6b400640 1 -- 172.21.15.152:0/3020312069 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf6402cbc0 con 0x7fbf6c0d0810 2024-04-20T11:27:51.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:51.668+0000 7fbf6b400640 1 -- 172.21.15.152:0/3020312069 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf64034440 con 0x7fbf6c0d0810 2024-04-20T11:27:51.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:51.668+0000 7fbf73800640 1 -- 172.21.15.152:0/3020312069 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbf6c0d0810 msgr2=0x7fbf6c0ada90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:51.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:51.668+0000 7fbf73800640 1 --2- 172.21.15.152:0/3020312069 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbf6c0d0810 0x7fbf6c0ada90 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fbf64009d90 tx=0x7fbf6402be10 comp rx=0 tx=0).stop 2024-04-20T11:27:51.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:51.668+0000 7fbf73800640 1 -- 172.21.15.152:0/3020312069 shutdown_connections 2024-04-20T11:27:51.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:51.668+0000 7fbf73800640 1 --2- 172.21.15.152:0/3020312069 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbf6c0d0810 0x7fbf6c0ada90 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:51.668+0000 7fbf73800640 1 --2- 172.21.15.152:0/3020312069 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbf6c15c560 0x7fbf6c0b3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:51.668+0000 7fbf73800640 1 -- 172.21.15.152:0/3020312069 >> 172.21.15.152:0/3020312069 conn(0x7fbf6c067cf0 msgr2=0x7fbf6c06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:51.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:51.668+0000 7fbf73800640 1 -- 172.21.15.152:0/3020312069 shutdown_connections 2024-04-20T11:27:51.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:51.668+0000 7fbf73800640 1 -- 172.21.15.152:0/3020312069 wait complete. 2024-04-20T11:27:51.670 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:51.670+0000 7fbf73800640 1 Processor -- start 2024-04-20T11:27:51.671 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:51.670+0000 7fbf73800640 1 -- start start 2024-04-20T11:27:51.672 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:51.671+0000 7fbf73800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbf6c0b4330 0x7fbf6c157440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:51.672 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:51.671+0000 7fbf73800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbf6c0d0810 0x7fbf6c0b0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:51.672 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:51.671+0000 7fbf73800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbf6c15c560 0x7fbf6c0b1170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:51.672 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:51.671+0000 7fbf73800640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fbf6c0adfd0 con 0x7fbf6c0d0810 2024-04-20T11:27:51.672 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:51.671+0000 7fbf73800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbf6c0d0dc0 con 0x7fbf6c0b4330 2024-04-20T11:27:51.672 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:51.671+0000 7fbf73800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fbf6c0ae150 con 0x7fbf6c15c560 2024-04-20T11:27:51.672 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:51.672+0000 7fbf6be00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbf6c0d0810 0x7fbf6c0b0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:51.672 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:51.672+0000 7fbf6be00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbf6c0d0810 0x7fbf6c0b0c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43404/0 (socket says 172.21.15.152:43404) 2024-04-20T11:27:51.672 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:51.672+0000 7fbf6be00640 1 -- 172.21.15.152:0/3168528182 learned_addr learned my addr 172.21.15.152:0/3168528182 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:51.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:51.672+0000 7fbf71800640 1 --2- 172.21.15.152:0/3168528182 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbf6c15c560 0x7fbf6c0b1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:51.674 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:51.672+0000 7fbf70e00640 1 --2- 172.21.15.152:0/3168528182 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbf6c0b4330 0x7fbf6c157440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:51.674 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:51.672+0000 7fbf6be00640 1 -- 172.21.15.152:0/3168528182 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbf6c15c560 msgr2=0x7fbf6c0b1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:51.674 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:51.672+0000 7fbf6be00640 1 --2- 172.21.15.152:0/3168528182 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbf6c15c560 0x7fbf6c0b1170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.674 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:51.672+0000 7fbf6be00640 1 -- 172.21.15.152:0/3168528182 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbf6c0b4330 msgr2=0x7fbf6c157440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:51.674 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:51.672+0000 7fbf6be00640 1 --2- 172.21.15.152:0/3168528182 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbf6c0b4330 0x7fbf6c157440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.674 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:51.672+0000 7fbf6be00640 1 -- 172.21.15.152:0/3168528182 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf6c158020 con 0x7fbf6c0d0810 2024-04-20T11:27:51.674 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:51.672+0000 7fbf6be00640 1 --2- 172.21.15.152:0/3168528182 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbf6c0d0810 0x7fbf6c0b0c30 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fbf5800c430 tx=0x7fbf5800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:51.674 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:51.673+0000 7fbf68c00640 1 -- 172.21.15.152:0/3168528182 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf58017070 con 0x7fbf6c0d0810 2024-04-20T11:27:51.674 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:51.673+0000 7fbf68c00640 1 -- 172.21.15.152:0/3168528182 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf58005ce0 con 0x7fbf6c0d0810 2024-04-20T11:27:51.674 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:51.673+0000 7fbf68c00640 1 -- 172.21.15.152:0/3168528182 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf580072f0 con 0x7fbf6c0d0810 2024-04-20T11:27:51.674 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:51.673+0000 7fbf73800640 1 -- 172.21.15.152:0/3168528182 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf6c158300 con 0x7fbf6c0d0810 2024-04-20T11:27:51.674 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:51.673+0000 7fbf73800640 1 -- 172.21.15.152:0/3168528182 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf6c1588c0 con 0x7fbf6c0d0810 2024-04-20T11:27:51.675 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:51.674+0000 7fbf73800640 1 -- 172.21.15.152:0/3168528182 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf6c0689f0 con 0x7fbf6c0d0810 2024-04-20T11:27:51.675 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:51.675+0000 7fbf68c00640 1 -- 172.21.15.152:0/3168528182 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbf5801a070 con 0x7fbf6c0d0810 2024-04-20T11:27:51.676 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:51.676+0000 7fbf68c00640 1 --2- 172.21.15.152:0/3168528182 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fbf4c06c8f0 0x7fbf4c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:51.676 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:51.676+0000 7fbf68c00640 1 -- 172.21.15.152:0/3168528182 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fbf5808b130 con 0x7fbf6c0d0810 2024-04-20T11:27:51.676 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:51.676+0000 7fbf70e00640 1 --2- 172.21.15.152:0/3168528182 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fbf4c06c8f0 0x7fbf4c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:51.677 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:51.677+0000 7fbf70e00640 1 --2- 172.21.15.152:0/3168528182 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fbf4c06c8f0 0x7fbf4c06edb0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fbf600096d0 tx=0x7fbf60008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:51.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:51.680+0000 7fbf68c00640 1 -- 172.21.15.152:0/3168528182 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fbf58059440 con 0x7fbf6c0d0810 2024-04-20T11:27:51.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:51.879+0000 7fbf73800640 1 -- 172.21.15.152:0/3168528182 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbf6c06cd90 con 0x7fbf6c0d0810 2024-04-20T11:27:51.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:51.882+0000 7fbf68c00640 1 -- 172.21.15.152:0/3168528182 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16914 (secure 0 0 0) 0x7fbf58058d40 con 0x7fbf6c0d0810 2024-04-20T11:27:51.883 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:27:51.883 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":23,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:27:42.601620+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:26:56.261424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T11:26:57.079286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":2}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.26","mappings":[{"from":7,"to":1}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:27:51.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:51.885+0000 7fbf73800640 1 -- 172.21.15.152:0/3168528182 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fbf4c06c8f0 msgr2=0x7fbf4c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:51.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:51.885+0000 7fbf73800640 1 --2- 172.21.15.152:0/3168528182 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fbf4c06c8f0 0x7fbf4c06edb0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fbf600096d0 tx=0x7fbf60008040 comp rx=0 tx=0).stop 2024-04-20T11:27:51.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:51.885+0000 7fbf73800640 1 -- 172.21.15.152:0/3168528182 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbf6c0d0810 msgr2=0x7fbf6c0b0c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:51.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:51.885+0000 7fbf73800640 1 --2- 172.21.15.152:0/3168528182 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbf6c0d0810 0x7fbf6c0b0c30 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fbf5800c430 tx=0x7fbf5800c8f0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:51.885+0000 7fbf73800640 1 -- 172.21.15.152:0/3168528182 shutdown_connections 2024-04-20T11:27:51.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:51.885+0000 7fbf73800640 1 --2- 172.21.15.152:0/3168528182 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fbf4c06c8f0 0x7fbf4c06edb0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:51.885+0000 7fbf73800640 1 --2- 172.21.15.152:0/3168528182 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbf6c15c560 0x7fbf6c0b1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:51.885+0000 7fbf73800640 1 --2- 172.21.15.152:0/3168528182 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbf6c0d0810 0x7fbf6c0b0c30 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:51.885+0000 7fbf73800640 1 --2- 172.21.15.152:0/3168528182 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbf6c0b4330 0x7fbf6c157440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:51.885+0000 7fbf73800640 1 -- 172.21.15.152:0/3168528182 >> 172.21.15.152:0/3168528182 conn(0x7fbf6c067cf0 msgr2=0x7fbf6c0b2150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:51.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:51.885+0000 7fbf73800640 1 -- 172.21.15.152:0/3168528182 shutdown_connections 2024-04-20T11:27:51.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:51.885+0000 7fbf73800640 1 -- 172.21.15.152:0/3168528182 wait complete. 2024-04-20T11:27:51.905 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T11:26:47.246571+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T11:26:56.261424+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T11:26:57.079286+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T11:27:51.905 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T11:27:52.108 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.107+0000 7f99efe00640 1 Processor -- start 2024-04-20T11:27:52.108 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.108+0000 7f99efe00640 1 -- start start 2024-04-20T11:27:52.110 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.109+0000 7f99efe00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f99f0152450 0x7f99f015c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:52.110 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.109+0000 7f99efe00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f99f00b3520 0x7f99f00ae470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:52.110 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.109+0000 7f99efe00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99f0152830 con 0x7f99f00b30c0 2024-04-20T11:27:52.110 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.109+0000 7f99efe00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f99f00b39c0 con 0x7f99f00b3520 2024-04-20T11:27:52.110 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.109+0000 7f99efe00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f99f00b3ca0 con 0x7f99f0152450 2024-04-20T11:27:52.111 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.110+0000 7f99ef400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f99f00b3520 0x7f99f00ae470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:52.111 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.110+0000 7f99ef400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f99f00b3520 0x7f99f00ae470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43412/0 (socket says 172.21.15.152:43412) 2024-04-20T11:27:52.111 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.110+0000 7f99ef400640 1 -- 172.21.15.152:0/4324774 learned_addr learned my addr 172.21.15.152:0/4324774 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:52.111 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.110+0000 7f99eea00640 1 --2- 172.21.15.152:0/4324774 >> v2:172.21.15.175:3300/0 conn(0x7f99f0152450 0x7f99f015c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:52.111 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.110+0000 7f99ef400640 1 -- 172.21.15.152:0/4324774 >> v2:172.21.15.175:3300/0 conn(0x7f99f0152450 msgr2=0x7f99f015c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:52.111 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.110+0000 7f99ef400640 1 --2- 172.21.15.152:0/4324774 >> v2:172.21.15.175:3300/0 conn(0x7f99f0152450 0x7f99f015c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:52.112 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.110+0000 7f99ef400640 1 -- 172.21.15.152:0/4324774 >> v1:172.21.15.175:6789/0 conn(0x7f99f00b30c0 legacy=0x7f99f006bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:52.112 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.110+0000 7f99ef400640 1 -- 172.21.15.152:0/4324774 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99f00b3f80 con 0x7f99f00b3520 2024-04-20T11:27:52.112 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.111+0000 7f99ef400640 1 --2- 172.21.15.152:0/4324774 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f99f00b3520 0x7f99f00ae470 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f99f00d1250 tx=0x7f99e802bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c1203da752fa3c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:52.112 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.112+0000 7f99ee000640 1 -- 172.21.15.152:0/4324774 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99e802cb20 con 0x7f99f00b3520 2024-04-20T11:27:52.112 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.112+0000 7f99ee000640 1 -- 172.21.15.152:0/4324774 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99e802ccc0 con 0x7f99f00b3520 2024-04-20T11:27:52.112 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.112+0000 7f99ee000640 1 -- 172.21.15.152:0/4324774 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99e8034610 con 0x7f99f00b3520 2024-04-20T11:27:52.113 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.112+0000 7f99efe00640 1 -- 172.21.15.152:0/4324774 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f99f00b3520 msgr2=0x7f99f00ae470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:52.113 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.112+0000 7f99efe00640 1 --2- 172.21.15.152:0/4324774 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f99f00b3520 0x7f99f00ae470 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f99f00d1250 tx=0x7f99e802bf80 comp rx=0 tx=0).stop 2024-04-20T11:27:52.113 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.113+0000 7f99efe00640 1 -- 172.21.15.152:0/4324774 shutdown_connections 2024-04-20T11:27:52.113 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.113+0000 7f99efe00640 1 --2- 172.21.15.152:0/4324774 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f99f00b3520 0x7f99f00ae470 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:52.114 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.113+0000 7f99efe00640 1 --2- 172.21.15.152:0/4324774 >> v2:172.21.15.175:3300/0 conn(0x7f99f0152450 0x7f99f015c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:52.114 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.113+0000 7f99efe00640 1 -- 172.21.15.152:0/4324774 >> 172.21.15.152:0/4324774 conn(0x7f99f00b21e0 msgr2=0x7f99f00b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:52.114 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.113+0000 7f99efe00640 1 -- 172.21.15.152:0/4324774 shutdown_connections 2024-04-20T11:27:52.114 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.113+0000 7f99efe00640 1 -- 172.21.15.152:0/4324774 wait complete. 2024-04-20T11:27:52.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.114+0000 7f99efe00640 1 Processor -- start 2024-04-20T11:27:52.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.114+0000 7f99efe00640 1 -- start start 2024-04-20T11:27:52.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.115+0000 7f99efe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f99f00b30c0 0x7f99f01494a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:52.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.115+0000 7f99efe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99f00b3520 0x7f99f0152a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:52.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.115+0000 7f99efe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f99f0152450 0x7f99f01400a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:52.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.115+0000 7f99efe00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f99f006c4b0 con 0x7f99f0152450 2024-04-20T11:27:52.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.115+0000 7f99efe00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f99f00adf00 con 0x7f99f00b3520 2024-04-20T11:27:52.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.115+0000 7f99efe00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f99f006c630 con 0x7f99f00b30c0 2024-04-20T11:27:52.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.116+0000 7f99ef400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f99f0152450 0x7f99f01400a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:52.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.116+0000 7f99ef400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f99f0152450 0x7f99f01400a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43416/0 (socket says 172.21.15.152:43416) 2024-04-20T11:27:52.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.116+0000 7f99ef400640 1 -- 172.21.15.152:0/969036554 learned_addr learned my addr 172.21.15.152:0/969036554 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:52.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.116+0000 7f99eea00640 1 --2- 172.21.15.152:0/969036554 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f99f00b30c0 0x7f99f01494a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:52.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.116+0000 7f99e7e00640 1 --2- 172.21.15.152:0/969036554 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99f00b3520 0x7f99f0152a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:52.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.116+0000 7f99ef400640 1 -- 172.21.15.152:0/969036554 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f99f00b30c0 msgr2=0x7f99f01494a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:52.118 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.116+0000 7f99ef400640 1 --2- 172.21.15.152:0/969036554 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f99f00b30c0 0x7f99f01494a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:52.118 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.116+0000 7f99ef400640 1 -- 172.21.15.152:0/969036554 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99f00b3520 msgr2=0x7f99f0152a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:52.118 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.116+0000 7f99ef400640 1 --2- 172.21.15.152:0/969036554 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99f00b3520 0x7f99f0152a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:52.118 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.116+0000 7f99ef400640 1 -- 172.21.15.152:0/969036554 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99f0152f60 con 0x7f99f0152450 2024-04-20T11:27:52.118 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.116+0000 7f99ef400640 1 --2- 172.21.15.152:0/969036554 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f99f0152450 0x7f99f01400a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f99e8009f40 tx=0x7f99e8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:52.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.117+0000 7f99e6a00640 1 -- 172.21.15.152:0/969036554 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99e8043020 con 0x7f99f0152450 2024-04-20T11:27:52.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.117+0000 7f99e6a00640 1 -- 172.21.15.152:0/969036554 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99e803e450 con 0x7f99f0152450 2024-04-20T11:27:52.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.117+0000 7f99e6a00640 1 -- 172.21.15.152:0/969036554 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99e8009510 con 0x7f99f0152450 2024-04-20T11:27:52.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.117+0000 7f99efe00640 1 -- 172.21.15.152:0/969036554 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99f0153200 con 0x7f99f0152450 2024-04-20T11:27:52.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.117+0000 7f99efe00640 1 -- 172.21.15.152:0/969036554 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99f01537c0 con 0x7f99f0152450 2024-04-20T11:27:52.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.118+0000 7f99efe00640 1 -- 172.21.15.152:0/969036554 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99f0075e60 con 0x7f99f0152450 2024-04-20T11:27:52.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.119+0000 7f99e6a00640 1 -- 172.21.15.152:0/969036554 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f99e8046020 con 0x7f99f0152450 2024-04-20T11:27:52.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.120+0000 7f99e6a00640 1 --2- 172.21.15.152:0/969036554 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f99cc06c8f0 0x7f99cc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:52.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.120+0000 7f99e6a00640 1 -- 172.21.15.152:0/969036554 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f99e80ae3e0 con 0x7f99f0152450 2024-04-20T11:27:52.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.120+0000 7f99eea00640 1 --2- 172.21.15.152:0/969036554 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f99cc06c8f0 0x7f99cc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:52.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.121+0000 7f99eea00640 1 --2- 172.21.15.152:0/969036554 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f99cc06c8f0 0x7f99cc06edb0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f99e0006fd0 tx=0x7f99e0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:52.124 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.124+0000 7f99e6a00640 1 -- 172.21.15.152:0/969036554 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f99e804d030 con 0x7f99f0152450 2024-04-20T11:27:52.329 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.328+0000 7f99efe00640 1 -- 172.21.15.152:0/969036554 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f99f006f770 con 0x7f99f0152450 2024-04-20T11:27:52.330 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.329+0000 7f99e6a00640 1 -- 172.21.15.152:0/969036554 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) ==== 93+0+10 (secure 0 0 0) 0x7f99e807c780 con 0x7f99f0152450 2024-04-20T11:27:52.330 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 1 2024-04-20T11:27:52.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.333+0000 7f99efe00640 1 -- 172.21.15.152:0/969036554 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f99cc06c8f0 msgr2=0x7f99cc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:52.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.333+0000 7f99efe00640 1 --2- 172.21.15.152:0/969036554 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f99cc06c8f0 0x7f99cc06edb0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f99e0006fd0 tx=0x7f99e0008040 comp rx=0 tx=0).stop 2024-04-20T11:27:52.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.333+0000 7f99efe00640 1 -- 172.21.15.152:0/969036554 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f99f0152450 msgr2=0x7f99f01400a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:52.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.333+0000 7f99efe00640 1 --2- 172.21.15.152:0/969036554 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f99f0152450 0x7f99f01400a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f99e8009f40 tx=0x7f99e8004a40 comp rx=0 tx=0).stop 2024-04-20T11:27:52.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.333+0000 7f99efe00640 1 -- 172.21.15.152:0/969036554 shutdown_connections 2024-04-20T11:27:52.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.333+0000 7f99efe00640 1 --2- 172.21.15.152:0/969036554 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f99cc06c8f0 0x7f99cc06edb0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:52.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.333+0000 7f99efe00640 1 --2- 172.21.15.152:0/969036554 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f99f0152450 0x7f99f01400a0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:52.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.333+0000 7f99efe00640 1 --2- 172.21.15.152:0/969036554 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99f00b3520 0x7f99f0152a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:52.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.333+0000 7f99efe00640 1 --2- 172.21.15.152:0/969036554 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f99f00b30c0 0x7f99f01494a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:52.335 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.333+0000 7f99efe00640 1 -- 172.21.15.152:0/969036554 >> 172.21.15.152:0/969036554 conn(0x7f99f00b21e0 msgr2=0x7f99f0071bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:52.335 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.333+0000 7f99efe00640 1 -- 172.21.15.152:0/969036554 shutdown_connections 2024-04-20T11:27:52.335 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.333+0000 7f99efe00640 1 -- 172.21.15.152:0/969036554 wait complete. 2024-04-20T11:27:52.353 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T11:27:52.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.559+0000 7f1787e00640 1 Processor -- start 2024-04-20T11:27:52.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.560+0000 7f1787e00640 1 -- start start 2024-04-20T11:27:52.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.561+0000 7f1787e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f178815eb50 0x7f17880d0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:52.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.561+0000 7f1787e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1788163230 0x7f17880d2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:52.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.561+0000 7f1787e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f17880d1440 0x7f17880d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:52.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.561+0000 7f1787e00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f17880d19c0 con 0x7f17880d1440 2024-04-20T11:27:52.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.561+0000 7f1787e00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f1788162750 con 0x7f178815eb50 2024-04-20T11:27:52.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.561+0000 7f1787e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1788162a30 con 0x7f1788163230 2024-04-20T11:27:52.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.561+0000 7f1787400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f17880d1440 0x7f17880d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:52.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.562+0000 7f1787400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f17880d1440 0x7f17880d33d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43420/0 (socket says 172.21.15.152:43420) 2024-04-20T11:27:52.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.562+0000 7f1787400640 1 -- 172.21.15.152:0/2429471319 learned_addr learned my addr 172.21.15.152:0/2429471319 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:52.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.562+0000 7f1786000640 1 --2- 172.21.15.152:0/2429471319 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1788163230 0x7f17880d2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:52.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.562+0000 7f1787400640 1 -- 172.21.15.152:0/2429471319 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1788163230 msgr2=0x7f17880d2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:52.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.562+0000 7f1787400640 1 --2- 172.21.15.152:0/2429471319 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1788163230 0x7f17880d2e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:52.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.562+0000 7f1787400640 1 -- 172.21.15.152:0/2429471319 >> v2:172.21.15.175:3300/0 conn(0x7f178815eb50 msgr2=0x7f17880d0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:52.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.562+0000 7f1786a00640 1 --2- 172.21.15.152:0/2429471319 >> v2:172.21.15.175:3300/0 conn(0x7f178815eb50 0x7f17880d0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:52.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.562+0000 7f1787400640 1 --2- 172.21.15.152:0/2429471319 >> v2:172.21.15.175:3300/0 conn(0x7f178815eb50 0x7f17880d0f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:52.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.562+0000 7f1787400640 1 -- 172.21.15.152:0/2429471319 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1788162d10 con 0x7f17880d1440 2024-04-20T11:27:52.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.562+0000 7f1786000640 1 --2- 172.21.15.152:0/2429471319 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1788163230 0x7f17880d2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:27:52.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.562+0000 7f1786a00640 1 --2- 172.21.15.152:0/2429471319 >> v2:172.21.15.175:3300/0 conn(0x7f178815eb50 0x7f17880d0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:27:52.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.563+0000 7f1787400640 1 --2- 172.21.15.152:0/2429471319 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f17880d1440 0x7f17880d33d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f17880b7080 tx=0x7f178002b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83aab302e8f458d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:52.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.564+0000 7f1785600640 1 -- 172.21.15.152:0/2429471319 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17800317f0 con 0x7f17880d1440 2024-04-20T11:27:52.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.564+0000 7f1785600640 1 -- 172.21.15.152:0/2429471319 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1780037070 con 0x7f17880d1440 2024-04-20T11:27:52.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.564+0000 7f1785600640 1 -- 172.21.15.152:0/2429471319 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1780031bb0 con 0x7f17880d1440 2024-04-20T11:27:52.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.564+0000 7f1787e00640 1 -- 172.21.15.152:0/2429471319 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f17880d1440 msgr2=0x7f17880d33d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:52.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.564+0000 7f1787e00640 1 --2- 172.21.15.152:0/2429471319 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f17880d1440 0x7f17880d33d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f17880b7080 tx=0x7f178002b890 comp rx=0 tx=0).stop 2024-04-20T11:27:52.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.564+0000 7f1787e00640 1 -- 172.21.15.152:0/2429471319 shutdown_connections 2024-04-20T11:27:52.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.564+0000 7f1787e00640 1 --2- 172.21.15.152:0/2429471319 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f17880d1440 0x7f17880d33d0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:52.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.564+0000 7f1787e00640 1 --2- 172.21.15.152:0/2429471319 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1788163230 0x7f17880d2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:52.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.564+0000 7f1787e00640 1 --2- 172.21.15.152:0/2429471319 >> v2:172.21.15.175:3300/0 conn(0x7f178815eb50 0x7f17880d0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:52.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.564+0000 7f1787e00640 1 -- 172.21.15.152:0/2429471319 >> 172.21.15.152:0/2429471319 conn(0x7f178808bfd0 msgr2=0x7f17880d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:52.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.564+0000 7f1787e00640 1 -- 172.21.15.152:0/2429471319 shutdown_connections 2024-04-20T11:27:52.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.565+0000 7f1787e00640 1 -- 172.21.15.152:0/2429471319 wait complete. 2024-04-20T11:27:52.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.566+0000 7f1787e00640 1 Processor -- start 2024-04-20T11:27:52.567 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.566+0000 7f1787e00640 1 -- start start 2024-04-20T11:27:52.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.567+0000 7f1787e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f17880d1440 0x7f17880c9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:52.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.567+0000 7f1787e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f178815eb50 0x7f17880b69e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:52.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.567+0000 7f1787e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1788163230 0x7f1788161dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:52.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.567+0000 7f1787e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f178815f0b0 con 0x7f178815eb50 2024-04-20T11:27:52.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.567+0000 7f1787e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f178815ef30 con 0x7f17880d1440 2024-04-20T11:27:52.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.567+0000 7f1787e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f178815f230 con 0x7f1788163230 2024-04-20T11:27:52.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.568+0000 7f1786000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f178815eb50 0x7f17880b69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:52.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.568+0000 7f1786000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f178815eb50 0x7f17880b69e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43426/0 (socket says 172.21.15.152:43426) 2024-04-20T11:27:52.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.568+0000 7f1786000640 1 -- 172.21.15.152:0/382494410 learned_addr learned my addr 172.21.15.152:0/382494410 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:52.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.568+0000 7f1787400640 1 --2- 172.21.15.152:0/382494410 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1788163230 0x7f1788161dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:52.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.568+0000 7f1786a00640 1 --2- 172.21.15.152:0/382494410 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f17880d1440 0x7f17880c9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:52.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.568+0000 7f1786000640 1 -- 172.21.15.152:0/382494410 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1788163230 msgr2=0x7f1788161dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:52.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.568+0000 7f1786000640 1 --2- 172.21.15.152:0/382494410 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1788163230 0x7f1788161dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:52.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.568+0000 7f1786000640 1 -- 172.21.15.152:0/382494410 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f17880d1440 msgr2=0x7f17880c9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:52.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.568+0000 7f1786000640 1 --2- 172.21.15.152:0/382494410 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f17880d1440 0x7f17880c9ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:52.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.568+0000 7f1786000640 1 -- 172.21.15.152:0/382494410 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17880ca770 con 0x7f178815eb50 2024-04-20T11:27:52.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.568+0000 7f1786a00640 1 --2- 172.21.15.152:0/382494410 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f17880d1440 0x7f17880c9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:27:52.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.568+0000 7f1786000640 1 --2- 172.21.15.152:0/382494410 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f178815eb50 0x7f17880b69e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f177000cec0 tx=0x7f177000a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:52.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.569+0000 7f177ea00640 1 -- 172.21.15.152:0/382494410 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f177000e860 con 0x7f178815eb50 2024-04-20T11:27:52.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.569+0000 7f177ea00640 1 -- 172.21.15.152:0/382494410 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f177000ea00 con 0x7f178815eb50 2024-04-20T11:27:52.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.569+0000 7f177ea00640 1 -- 172.21.15.152:0/382494410 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f177000edd0 con 0x7f178815eb50 2024-04-20T11:27:52.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.569+0000 7f1787e00640 1 -- 172.21.15.152:0/382494410 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17880cdf80 con 0x7f178815eb50 2024-04-20T11:27:52.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.569+0000 7f1787e00640 1 -- 172.21.15.152:0/382494410 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17880ce540 con 0x7f178815eb50 2024-04-20T11:27:52.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.570+0000 7f1787e00640 1 -- 172.21.15.152:0/382494410 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1788075e60 con 0x7f178815eb50 2024-04-20T11:27:52.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.571+0000 7f177ea00640 1 -- 172.21.15.152:0/382494410 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1770016420 con 0x7f178815eb50 2024-04-20T11:27:52.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.572+0000 7f177ea00640 1 --2- 172.21.15.152:0/382494410 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f176406c8f0 0x7f176406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:52.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.572+0000 7f177ea00640 1 -- 172.21.15.152:0/382494410 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f177008c9e0 con 0x7f178815eb50 2024-04-20T11:27:52.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.572+0000 7f1786a00640 1 --2- 172.21.15.152:0/382494410 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f176406c8f0 0x7f176406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:52.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.573+0000 7f1786a00640 1 --2- 172.21.15.152:0/382494410 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f176406c8f0 0x7f176406edb0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f17780088b0 tx=0x7f1778008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:52.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.576+0000 7f177ea00640 1 -- 172.21.15.152:0/382494410 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f17880ca770 con 0x7f178815eb50 2024-04-20T11:27:52.778 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.778+0000 7f1787e00640 1 -- 172.21.15.152:0/382494410 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f178806e910 con 0x7f178815eb50 2024-04-20T11:27:52.779 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.779+0000 7f177ea00640 1 -- 172.21.15.152:0/382494410 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) ==== 104+0+11 (secure 0 0 0) 0x7f177005acf0 con 0x7f178815eb50 2024-04-20T11:27:52.779 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 32 2024-04-20T11:27:52.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.781+0000 7f1787e00640 1 -- 172.21.15.152:0/382494410 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f176406c8f0 msgr2=0x7f176406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:52.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.781+0000 7f1787e00640 1 --2- 172.21.15.152:0/382494410 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f176406c8f0 0x7f176406edb0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f17780088b0 tx=0x7f1778008040 comp rx=0 tx=0).stop 2024-04-20T11:27:52.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.781+0000 7f1787e00640 1 -- 172.21.15.152:0/382494410 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f178815eb50 msgr2=0x7f17880b69e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:52.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.781+0000 7f1787e00640 1 --2- 172.21.15.152:0/382494410 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f178815eb50 0x7f17880b69e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f177000cec0 tx=0x7f177000a6c0 comp rx=0 tx=0).stop 2024-04-20T11:27:52.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.781+0000 7f1787e00640 1 -- 172.21.15.152:0/382494410 shutdown_connections 2024-04-20T11:27:52.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.781+0000 7f1787e00640 1 --2- 172.21.15.152:0/382494410 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f176406c8f0 0x7f176406edb0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:52.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.781+0000 7f1787e00640 1 --2- 172.21.15.152:0/382494410 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1788163230 0x7f1788161dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:52.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.781+0000 7f1787e00640 1 --2- 172.21.15.152:0/382494410 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f178815eb50 0x7f17880b69e0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:52.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.781+0000 7f1787e00640 1 --2- 172.21.15.152:0/382494410 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f17880d1440 0x7f17880c9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:52.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.781+0000 7f1787e00640 1 -- 172.21.15.152:0/382494410 >> 172.21.15.152:0/382494410 conn(0x7f178808bfd0 msgr2=0x7f17880b2100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:52.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.782+0000 7f1787e00640 1 -- 172.21.15.152:0/382494410 shutdown_connections 2024-04-20T11:27:52.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:52.782+0000 7f1787e00640 1 -- 172.21.15.152:0/382494410 wait complete. 2024-04-20T11:27:52.802 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T11:27:53.005 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.004+0000 7f4323e00640 1 Processor -- start 2024-04-20T11:27:53.005 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.005+0000 7f4323e00640 1 -- start start 2024-04-20T11:27:53.007 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.006+0000 7f4323e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f43240cf5b0 0x7f43240d4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:53.007 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.006+0000 7f4323e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f43240ceeb0 0x7f43240cf990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:53.007 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.006+0000 7f4323e00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43240cfed0 con 0x7f4324162430 2024-04-20T11:27:53.007 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.006+0000 7f4323e00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f43240d5420 con 0x7f43240cf5b0 2024-04-20T11:27:53.007 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.006+0000 7f4323e00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f43240d2b70 con 0x7f43240ceeb0 2024-04-20T11:27:53.007 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.006+0000 7f4322a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f43240cf5b0 0x7f43240d4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:53.007 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.006+0000 7f4322a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f43240cf5b0 0x7f43240d4ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43436/0 (socket says 172.21.15.152:43436) 2024-04-20T11:27:53.007 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.006+0000 7f4322a00640 1 -- 172.21.15.152:0/1997297612 learned_addr learned my addr 172.21.15.152:0/1997297612 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:53.008 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.007+0000 7f4322000640 1 --2- 172.21.15.152:0/1997297612 >> v2:172.21.15.175:3300/0 conn(0x7f43240ceeb0 0x7f43240cf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:53.008 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.007+0000 7f4322a00640 1 -- 172.21.15.152:0/1997297612 >> v2:172.21.15.175:3300/0 conn(0x7f43240ceeb0 msgr2=0x7f43240cf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:53.008 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.007+0000 7f4322a00640 1 --2- 172.21.15.152:0/1997297612 >> v2:172.21.15.175:3300/0 conn(0x7f43240ceeb0 0x7f43240cf990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:53.008 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.007+0000 7f4322a00640 1 -- 172.21.15.152:0/1997297612 >> v1:172.21.15.175:6789/0 conn(0x7f4324162430 legacy=0x7f43241628c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:53.008 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.007+0000 7f4322a00640 1 -- 172.21.15.152:0/1997297612 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43240d2e50 con 0x7f43240cf5b0 2024-04-20T11:27:53.009 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.008+0000 7f4322a00640 1 --2- 172.21.15.152:0/1997297612 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f43240cf5b0 0x7f43240d4ee0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f432408cb50 tx=0x7f431402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84f0865b8249fcdb server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:53.009 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.009+0000 7f4321600640 1 -- 172.21.15.152:0/1997297612 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4314034070 con 0x7f43240cf5b0 2024-04-20T11:27:53.009 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.009+0000 7f4321600640 1 -- 172.21.15.152:0/1997297612 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4314030860 con 0x7f43240cf5b0 2024-04-20T11:27:53.009 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.009+0000 7f4321600640 1 -- 172.21.15.152:0/1997297612 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4314030c30 con 0x7f43240cf5b0 2024-04-20T11:27:53.010 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.009+0000 7f4323e00640 1 -- 172.21.15.152:0/1997297612 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f43240cf5b0 msgr2=0x7f43240d4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:53.010 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.009+0000 7f4323e00640 1 --2- 172.21.15.152:0/1997297612 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f43240cf5b0 0x7f43240d4ee0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f432408cb50 tx=0x7f431402b9a0 comp rx=0 tx=0).stop 2024-04-20T11:27:53.010 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.009+0000 7f4323e00640 1 -- 172.21.15.152:0/1997297612 shutdown_connections 2024-04-20T11:27:53.010 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.009+0000 7f4323e00640 1 --2- 172.21.15.152:0/1997297612 >> v2:172.21.15.175:3300/0 conn(0x7f43240ceeb0 0x7f43240cf990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:53.010 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.009+0000 7f4323e00640 1 --2- 172.21.15.152:0/1997297612 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f43240cf5b0 0x7f43240d4ee0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:53.010 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.009+0000 7f4323e00640 1 -- 172.21.15.152:0/1997297612 >> 172.21.15.152:0/1997297612 conn(0x7f432408bfd0 msgr2=0x7f43240d4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:53.010 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.010+0000 7f4323e00640 1 -- 172.21.15.152:0/1997297612 shutdown_connections 2024-04-20T11:27:53.010 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.010+0000 7f4323e00640 1 -- 172.21.15.152:0/1997297612 wait complete. 2024-04-20T11:27:53.012 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.011+0000 7f4323e00640 1 Processor -- start 2024-04-20T11:27:53.012 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.011+0000 7f4323e00640 1 -- start start 2024-04-20T11:27:53.013 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.012+0000 7f4323e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f43240ceeb0 0x7f4324151ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:53.013 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.012+0000 7f4323e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f43240cf5b0 0x7f43241e97a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:53.013 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.012+0000 7f4323e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4324162430 0x7f43241f5b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:53.013 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.012+0000 7f4323e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f43240d36f0 con 0x7f43240ceeb0 2024-04-20T11:27:53.013 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.012+0000 7f4323e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f43240d3410 con 0x7f4324162430 2024-04-20T11:27:53.013 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.012+0000 7f4323e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4324161240 con 0x7f43240cf5b0 2024-04-20T11:27:53.013 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.013+0000 7f4322a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f43240ceeb0 0x7f4324151ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:53.014 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.013+0000 7f4322a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f43240ceeb0 0x7f4324151ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43448/0 (socket says 172.21.15.152:43448) 2024-04-20T11:27:53.014 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.013+0000 7f4322a00640 1 -- 172.21.15.152:0/4229744295 learned_addr learned my addr 172.21.15.152:0/4229744295 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:53.015 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.013+0000 7f4322a00640 1 -- 172.21.15.152:0/4229744295 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f43240cf5b0 msgr2=0x7f43241e97a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:53.015 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.013+0000 7f4322a00640 1 --2- 172.21.15.152:0/4229744295 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f43240cf5b0 0x7f43241e97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:53.015 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.013+0000 7f4322a00640 1 -- 172.21.15.152:0/4229744295 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4324162430 msgr2=0x7f43241f5b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:53.015 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.013+0000 7f4323400640 1 --2- 172.21.15.152:0/4229744295 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4324162430 0x7f43241f5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:53.015 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.013+0000 7f4322a00640 1 --2- 172.21.15.152:0/4229744295 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4324162430 0x7f43241f5b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:53.016 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.013+0000 7f4322a00640 1 -- 172.21.15.152:0/4229744295 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43241e8700 con 0x7f43240ceeb0 2024-04-20T11:27:53.016 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.013+0000 7f4322a00640 1 --2- 172.21.15.152:0/4229744295 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f43240ceeb0 0x7f4324151ac0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f4314000c00 tx=0x7f4314004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:53.016 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.014+0000 7f431aa00640 1 -- 172.21.15.152:0/4229744295 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4314048070 con 0x7f43240ceeb0 2024-04-20T11:27:53.016 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.014+0000 7f431aa00640 1 -- 172.21.15.152:0/4229744295 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4314030660 con 0x7f43240ceeb0 2024-04-20T11:27:53.016 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.014+0000 7f431aa00640 1 -- 172.21.15.152:0/4229744295 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4314034040 con 0x7f43240ceeb0 2024-04-20T11:27:53.016 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.014+0000 7f4323e00640 1 -- 172.21.15.152:0/4229744295 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43241e89e0 con 0x7f43240ceeb0 2024-04-20T11:27:53.016 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.014+0000 7f4323e00640 1 -- 172.21.15.152:0/4229744295 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43241e8fa0 con 0x7f43240ceeb0 2024-04-20T11:27:53.016 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.015+0000 7f4323e00640 1 -- 172.21.15.152:0/4229744295 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f432415fe50 con 0x7f43240ceeb0 2024-04-20T11:27:53.017 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.016+0000 7f431aa00640 1 -- 172.21.15.152:0/4229744295 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f431404b070 con 0x7f43240ceeb0 2024-04-20T11:27:53.017 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.017+0000 7f431aa00640 1 --2- 172.21.15.152:0/4229744295 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f430006c8f0 0x7f430006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:53.017 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.017+0000 7f431aa00640 1 -- 172.21.15.152:0/4229744295 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f43140addf0 con 0x7f43240ceeb0 2024-04-20T11:27:53.017 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.017+0000 7f4322000640 1 --2- 172.21.15.152:0/4229744295 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f430006c8f0 0x7f430006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:53.018 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.018+0000 7f4322000640 1 --2- 172.21.15.152:0/4229744295 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f430006c8f0 0x7f430006edb0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f430c0070b0 tx=0x7f430c007040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:53.022 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.021+0000 7f431aa00640 1 -- 172.21.15.152:0/4229744295 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f432415fe50 con 0x7f43240ceeb0 2024-04-20T11:27:53.222 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.221+0000 7f4323e00640 1 -- 172.21.15.152:0/4229744295 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f43240cf390 con 0x7f43240ceeb0 2024-04-20T11:27:53.222 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.222+0000 7f431aa00640 1 -- 172.21.15.152:0/4229744295 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) ==== 100+0+11 (secure 0 0 0) 0x7f431407c100 con 0x7f43240ceeb0 2024-04-20T11:27:53.222 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 64 2024-04-20T11:27:53.225 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.224+0000 7f4323e00640 1 -- 172.21.15.152:0/4229744295 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f430006c8f0 msgr2=0x7f430006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:53.225 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.224+0000 7f4323e00640 1 --2- 172.21.15.152:0/4229744295 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f430006c8f0 0x7f430006edb0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f430c0070b0 tx=0x7f430c007040 comp rx=0 tx=0).stop 2024-04-20T11:27:53.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.224+0000 7f4323e00640 1 -- 172.21.15.152:0/4229744295 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f43240ceeb0 msgr2=0x7f4324151ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:53.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.224+0000 7f4323e00640 1 --2- 172.21.15.152:0/4229744295 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f43240ceeb0 0x7f4324151ac0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f4314000c00 tx=0x7f4314004730 comp rx=0 tx=0).stop 2024-04-20T11:27:53.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.225+0000 7f4323e00640 1 -- 172.21.15.152:0/4229744295 shutdown_connections 2024-04-20T11:27:53.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.225+0000 7f4323e00640 1 --2- 172.21.15.152:0/4229744295 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f430006c8f0 0x7f430006edb0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:53.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.225+0000 7f4323e00640 1 --2- 172.21.15.152:0/4229744295 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4324162430 0x7f43241f5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:53.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.225+0000 7f4323e00640 1 --2- 172.21.15.152:0/4229744295 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f43240cf5b0 0x7f43241e97a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:53.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.225+0000 7f4323e00640 1 --2- 172.21.15.152:0/4229744295 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f43240ceeb0 0x7f4324151ac0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:53.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.225+0000 7f4323e00640 1 -- 172.21.15.152:0/4229744295 >> 172.21.15.152:0/4229744295 conn(0x7f432408bfd0 msgr2=0x7f43240cda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:53.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.225+0000 7f4323e00640 1 -- 172.21.15.152:0/4229744295 shutdown_connections 2024-04-20T11:27:53.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.225+0000 7f4323e00640 1 -- 172.21.15.152:0/4229744295 wait complete. 2024-04-20T11:27:53.244 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-20T11:27:53.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.450+0000 7f697be00640 1 Processor -- start 2024-04-20T11:27:53.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.451+0000 7f697be00640 1 -- start start 2024-04-20T11:27:53.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.452+0000 7f697be00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f697c072b70 0x7f697c073b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:53.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.452+0000 7f697be00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f697c072f70 con 0x7f697c0b4870 2024-04-20T11:27:53.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.452+0000 7f697be00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f697c0ce390 con 0x7f697c15c7f0 2024-04-20T11:27:53.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.452+0000 7f697be00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f697c0ce140 con 0x7f697c072b70 2024-04-20T11:27:53.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.452+0000 7f6972000640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f697c0b4870 0x7f697c0cf430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:44896/0 (socket says 172.21.15.152:44896) 2024-04-20T11:27:53.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.452+0000 7f6972000640 1 -- 172.21.15.152:0/694321127 learned_addr learned my addr 172.21.15.152:0/694321127 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:27:53.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.452+0000 7f697b400640 1 --2- 172.21.15.152:0/694321127 >> v2:172.21.15.152:3300/0 conn(0x7f697c072b70 0x7f697c073b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:53.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.453+0000 7f697a000640 1 -- 172.21.15.152:0/694321127 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3323579104 0 0) 0x7f6964001110 con 0x7f697c0b4870 2024-04-20T11:27:53.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.453+0000 7f697a000640 1 -- 172.21.15.152:0/694321127 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6960003520 con 0x7f697c0b4870 2024-04-20T11:27:53.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.453+0000 7f697b400640 1 -- 172.21.15.152:0/694321127 >> v1:172.21.15.175:6789/0 conn(0x7f697c15c7f0 legacy=0x7f697c0b4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:53.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.453+0000 7f697b400640 1 -- 172.21.15.152:0/694321127 >> v1:172.21.15.152:6789/0 conn(0x7f697c0b4870 legacy=0x7f697c0cf430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:53.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.453+0000 7f697b400640 1 -- 172.21.15.152:0/694321127 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f697c0ce670 con 0x7f697c072b70 2024-04-20T11:27:53.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.454+0000 7f697b400640 1 --2- 172.21.15.152:0/694321127 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f697c072b70 0x7f697c073b60 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f697c0ce2c0 tx=0x7f697402bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6383c47d31cbadb server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:53.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.454+0000 7f697a000640 1 -- 172.21.15.152:0/694321127 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6974039070 con 0x7f697c072b70 2024-04-20T11:27:53.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.455+0000 7f697a000640 1 -- 172.21.15.152:0/694321127 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f697402cc50 con 0x7f697c072b70 2024-04-20T11:27:53.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.455+0000 7f697a000640 1 -- 172.21.15.152:0/694321127 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6974034440 con 0x7f697c072b70 2024-04-20T11:27:53.456 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.455+0000 7f697be00640 1 -- 172.21.15.152:0/694321127 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f697c072b70 msgr2=0x7f697c073b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:53.456 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.455+0000 7f697be00640 1 --2- 172.21.15.152:0/694321127 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f697c072b70 0x7f697c073b60 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f697c0ce2c0 tx=0x7f697402bea0 comp rx=0 tx=0).stop 2024-04-20T11:27:53.456 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.455+0000 7f697be00640 1 -- 172.21.15.152:0/694321127 shutdown_connections 2024-04-20T11:27:53.456 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.455+0000 7f697be00640 1 --2- 172.21.15.152:0/694321127 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f697c072b70 0x7f697c073b60 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:53.456 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.455+0000 7f697be00640 1 -- 172.21.15.152:0/694321127 >> 172.21.15.152:0/694321127 conn(0x7f697c0d0260 msgr2=0x7f697c0d0670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:53.456 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.455+0000 7f697be00640 1 -- 172.21.15.152:0/694321127 shutdown_connections 2024-04-20T11:27:53.456 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.455+0000 7f697be00640 1 -- 172.21.15.152:0/694321127 wait complete. 2024-04-20T11:27:53.457 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.457+0000 7f697be00640 1 Processor -- start 2024-04-20T11:27:53.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.457+0000 7f697be00640 1 -- start start 2024-04-20T11:27:53.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.458+0000 7f697be00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f697c072b70 0x7f697c1e5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:53.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.458+0000 7f697be00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f697c0b4870 0x7f697c06f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:53.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.458+0000 7f697be00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f697c1f7130 0x7f697c1f75c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:53.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.458+0000 7f697be00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f697c0cef10 con 0x7f697c1f7130 2024-04-20T11:27:53.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.458+0000 7f697be00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f697c0cec30 con 0x7f697c0b4870 2024-04-20T11:27:53.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.458+0000 7f697be00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f697c0ad2b0 con 0x7f697c072b70 2024-04-20T11:27:53.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.459+0000 7f697b400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f697c1f7130 0x7f697c1f75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:53.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.459+0000 7f697b400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f697c1f7130 0x7f697c1f75c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43454/0 (socket says 172.21.15.152:43454) 2024-04-20T11:27:53.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.459+0000 7f697b400640 1 -- 172.21.15.152:0/1533521080 learned_addr learned my addr 172.21.15.152:0/1533521080 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:53.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.459+0000 7f697aa00640 1 --2- 172.21.15.152:0/1533521080 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f697c072b70 0x7f697c1e5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:53.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.459+0000 7f697b400640 1 -- 172.21.15.152:0/1533521080 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f697c072b70 msgr2=0x7f697c1e5fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:53.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.459+0000 7f6972000640 1 --2- 172.21.15.152:0/1533521080 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f697c0b4870 0x7f697c06f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:53.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.459+0000 7f697b400640 1 --2- 172.21.15.152:0/1533521080 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f697c072b70 0x7f697c1e5fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:53.461 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.459+0000 7f697b400640 1 -- 172.21.15.152:0/1533521080 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f697c0b4870 msgr2=0x7f697c06f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:53.461 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.459+0000 7f697b400640 1 --2- 172.21.15.152:0/1533521080 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f697c0b4870 0x7f697c06f300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:53.461 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.459+0000 7f697b400640 1 -- 172.21.15.152:0/1533521080 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f697c1e6b90 con 0x7f697c1f7130 2024-04-20T11:27:53.461 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.459+0000 7f697b400640 1 --2- 172.21.15.152:0/1533521080 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f697c1f7130 0x7f697c1f75c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f6974009e80 tx=0x7f6974004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:53.462 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.460+0000 7f6973400640 1 -- 172.21.15.152:0/1533521080 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6974044020 con 0x7f697c1f7130 2024-04-20T11:27:53.462 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.460+0000 7f6973400640 1 -- 172.21.15.152:0/1533521080 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f697403a040 con 0x7f697c1f7130 2024-04-20T11:27:53.462 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.460+0000 7f697be00640 1 -- 172.21.15.152:0/1533521080 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f697c1e6e70 con 0x7f697c1f7130 2024-04-20T11:27:53.462 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.460+0000 7f6973400640 1 -- 172.21.15.152:0/1533521080 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6974039050 con 0x7f697c1f7130 2024-04-20T11:27:53.462 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.460+0000 7f697be00640 1 -- 172.21.15.152:0/1533521080 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f697c1e7430 con 0x7f697c1f7130 2024-04-20T11:27:53.462 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.461+0000 7f697be00640 1 -- 172.21.15.152:0/1533521080 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f697c075e60 con 0x7f697c1f7130 2024-04-20T11:27:53.462 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.462+0000 7f6973400640 1 -- 172.21.15.152:0/1533521080 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6974003990 con 0x7f697c1f7130 2024-04-20T11:27:53.464 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.463+0000 7f6973400640 1 --2- 172.21.15.152:0/1533521080 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f695806c8f0 0x7f695806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:53.464 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.463+0000 7f6973400640 1 -- 172.21.15.152:0/1533521080 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f69740ae1e0 con 0x7f697c1f7130 2024-04-20T11:27:53.464 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.463+0000 7f697aa00640 1 --2- 172.21.15.152:0/1533521080 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f695806c8f0 0x7f695806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:53.464 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.464+0000 7f697aa00640 1 --2- 172.21.15.152:0/1533521080 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f695806c8f0 0x7f695806edb0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f696c001500 tx=0x7f696c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:53.468 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.468+0000 7f6973400640 1 -- 172.21.15.152:0/1533521080 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f697404e020 con 0x7f697c1f7130 2024-04-20T11:27:53.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.686+0000 7f697be00640 1 -- 172.21.15.152:0/1533521080 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f697c0b1500 con 0x7f697c1f7130 2024-04-20T11:27:53.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.686+0000 7f6973400640 1 -- 172.21.15.152:0/1533521080 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) ==== 80+0+0 (secure 0 0 0) 0x7f697403d360 con 0x7f697c1f7130 2024-04-20T11:27:53.687 INFO:teuthology.orchestra.run.smithi152.stderr:listed 0 entries 2024-04-20T11:27:53.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.689+0000 7f697be00640 1 -- 172.21.15.152:0/1533521080 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f695806c8f0 msgr2=0x7f695806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:53.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.689+0000 7f697be00640 1 --2- 172.21.15.152:0/1533521080 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f695806c8f0 0x7f695806edb0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f696c001500 tx=0x7f696c009040 comp rx=0 tx=0).stop 2024-04-20T11:27:53.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.689+0000 7f697be00640 1 -- 172.21.15.152:0/1533521080 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f697c1f7130 msgr2=0x7f697c1f75c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:53.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.689+0000 7f697be00640 1 --2- 172.21.15.152:0/1533521080 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f697c1f7130 0x7f697c1f75c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f6974009e80 tx=0x7f6974004a40 comp rx=0 tx=0).stop 2024-04-20T11:27:53.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.689+0000 7f697be00640 1 -- 172.21.15.152:0/1533521080 shutdown_connections 2024-04-20T11:27:53.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.689+0000 7f697be00640 1 --2- 172.21.15.152:0/1533521080 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f695806c8f0 0x7f695806edb0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:53.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.689+0000 7f697be00640 1 --2- 172.21.15.152:0/1533521080 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f697c1f7130 0x7f697c1f75c0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:53.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.689+0000 7f697be00640 1 --2- 172.21.15.152:0/1533521080 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f697c0b4870 0x7f697c06f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:53.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.689+0000 7f697be00640 1 --2- 172.21.15.152:0/1533521080 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f697c072b70 0x7f697c1e5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:53.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.689+0000 7f697be00640 1 -- 172.21.15.152:0/1533521080 >> 172.21.15.152:0/1533521080 conn(0x7f697c0d0260 msgr2=0x7f697c15b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:53.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.689+0000 7f697be00640 1 -- 172.21.15.152:0/1533521080 shutdown_connections 2024-04-20T11:27:53.691 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:53.689+0000 7f697be00640 1 -- 172.21.15.152:0/1533521080 wait complete. 2024-04-20T11:27:53.710 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-20T11:27:53.710 DEBUG:teuthology.orchestra.run.smithi152:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:27:53.758 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:27:54.003 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.002+0000 7f9440a00640 1 Processor -- start 2024-04-20T11:27:54.003 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.002+0000 7f9440a00640 1 -- start start 2024-04-20T11:27:54.004 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.004+0000 7f9440a00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f943c0add80 0x7f943c0ae210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:54.004 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.004+0000 7f9440a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f943c06e520 0x7f943c06e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:54.004 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.004+0000 7f9440a00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f943c0b55a0 con 0x7f943c15e890 2024-04-20T11:27:54.005 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.004+0000 7f9440a00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f943c0b5880 con 0x7f943c06e520 2024-04-20T11:27:54.005 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.004+0000 7f9440a00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f943c0b5b60 con 0x7f943c0add80 2024-04-20T11:27:54.005 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.004+0000 7f943be00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f943c06e520 0x7f943c06e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:54.006 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.004+0000 7f943b400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f943c15e890 0x7f943c162cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:44900/0 (socket says 172.21.15.152:44900) 2024-04-20T11:27:54.006 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.004+0000 7f943b400640 1 -- 172.21.15.152:0/1190898787 learned_addr learned my addr 172.21.15.152:0/1190898787 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:27:54.006 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.005+0000 7f943aa00640 1 --2- 172.21.15.152:0/1190898787 >> v2:172.21.15.175:3300/0 conn(0x7f943c0add80 0x7f943c0ae210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:54.006 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.005+0000 7f943be00640 1 -- 172.21.15.152:0/1190898787 >> v2:172.21.15.175:3300/0 conn(0x7f943c0add80 msgr2=0x7f943c0ae210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:54.006 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.005+0000 7f943be00640 1 --2- 172.21.15.152:0/1190898787 >> v2:172.21.15.175:3300/0 conn(0x7f943c0add80 0x7f943c0ae210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:54.006 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.005+0000 7f943be00640 1 -- 172.21.15.152:0/1190898787 >> v1:172.21.15.152:6789/0 conn(0x7f943c15e890 legacy=0x7f943c162cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:54.006 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.005+0000 7f943be00640 1 -- 172.21.15.152:0/1190898787 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f943c0b5e40 con 0x7f943c06e520 2024-04-20T11:27:54.006 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.005+0000 7f943a000640 1 -- 172.21.15.152:0/1190898787 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 536260957 0 0) 0x7f943c0b55a0 con 0x7f943c15e890 2024-04-20T11:27:54.007 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.006+0000 7f943be00640 1 --2- 172.21.15.152:0/1190898787 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f943c06e520 0x7f943c06e930 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f943c067c30 tx=0x7f943402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d47cba4d3f80509 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:54.007 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.006+0000 7f943a000640 1 -- 172.21.15.152:0/1190898787 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9434039070 con 0x7f943c06e520 2024-04-20T11:27:54.007 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.006+0000 7f943a000640 1 -- 172.21.15.152:0/1190898787 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f943402cbf0 con 0x7f943c06e520 2024-04-20T11:27:54.007 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.006+0000 7f943a000640 1 -- 172.21.15.152:0/1190898787 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9434034440 con 0x7f943c06e520 2024-04-20T11:27:54.008 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.007+0000 7f9440a00640 1 -- 172.21.15.152:0/1190898787 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f943c06e520 msgr2=0x7f943c06e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:54.008 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.007+0000 7f9440a00640 1 --2- 172.21.15.152:0/1190898787 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f943c06e520 0x7f943c06e930 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f943c067c30 tx=0x7f943402be40 comp rx=0 tx=0).stop 2024-04-20T11:27:54.008 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.007+0000 7f9440a00640 1 -- 172.21.15.152:0/1190898787 shutdown_connections 2024-04-20T11:27:54.008 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.007+0000 7f9440a00640 1 --2- 172.21.15.152:0/1190898787 >> v2:172.21.15.175:3300/0 conn(0x7f943c0add80 0x7f943c0ae210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:54.008 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.007+0000 7f9440a00640 1 --2- 172.21.15.152:0/1190898787 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f943c06e520 0x7f943c06e930 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:54.008 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.007+0000 7f9440a00640 1 -- 172.21.15.152:0/1190898787 >> 172.21.15.152:0/1190898787 conn(0x7f943c073340 msgr2=0x7f943c06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:54.008 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.007+0000 7f9440a00640 1 -- 172.21.15.152:0/1190898787 shutdown_connections 2024-04-20T11:27:54.008 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.007+0000 7f9440a00640 1 -- 172.21.15.152:0/1190898787 wait complete. 2024-04-20T11:27:54.009 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.009+0000 7f9440a00640 1 Processor -- start 2024-04-20T11:27:54.009 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.009+0000 7f9440a00640 1 -- start start 2024-04-20T11:27:54.011 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.010+0000 7f9440a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f943c06e520 0x7f943c1500b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:54.011 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.010+0000 7f9440a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f943c0add80 0x7f943c153910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:54.011 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.010+0000 7f9440a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f943c15e890 0x7f943c15a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:54.011 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.010+0000 7f9440a00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f943c0b67c0 con 0x7f943c06e520 2024-04-20T11:27:54.011 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.010+0000 7f9440a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f943c0b6640 con 0x7f943c0add80 2024-04-20T11:27:54.011 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.010+0000 7f9440a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f943c0aea00 con 0x7f943c15e890 2024-04-20T11:27:54.011 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.010+0000 7f943b400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f943c06e520 0x7f943c1500b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:54.013 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.011+0000 7f943b400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f943c06e520 0x7f943c1500b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43476/0 (socket says 172.21.15.152:43476) 2024-04-20T11:27:54.013 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.011+0000 7f943b400640 1 -- 172.21.15.152:0/3697878515 learned_addr learned my addr 172.21.15.152:0/3697878515 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:54.013 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.011+0000 7f943aa00640 1 --2- 172.21.15.152:0/3697878515 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f943c0add80 0x7f943c153910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:54.013 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.011+0000 7f943be00640 1 --2- 172.21.15.152:0/3697878515 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f943c15e890 0x7f943c15a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:54.013 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.011+0000 7f943b400640 1 -- 172.21.15.152:0/3697878515 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f943c15e890 msgr2=0x7f943c15a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:54.013 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.011+0000 7f943b400640 1 --2- 172.21.15.152:0/3697878515 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f943c15e890 0x7f943c15a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:54.013 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.011+0000 7f943b400640 1 -- 172.21.15.152:0/3697878515 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f943c0add80 msgr2=0x7f943c153910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:54.013 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.011+0000 7f943b400640 1 --2- 172.21.15.152:0/3697878515 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f943c0add80 0x7f943c153910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:54.013 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.011+0000 7f943b400640 1 -- 172.21.15.152:0/3697878515 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f943c152fb0 con 0x7f943c06e520 2024-04-20T11:27:54.013 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.011+0000 7f943b400640 1 --2- 172.21.15.152:0/3697878515 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f943c06e520 0x7f943c1500b0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f942c00c500 tx=0x7f942c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:54.013 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.012+0000 7f9433400640 1 -- 172.21.15.152:0/3697878515 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f942c018070 con 0x7f943c06e520 2024-04-20T11:27:54.014 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.012+0000 7f9433400640 1 -- 172.21.15.152:0/3697878515 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f942c007580 con 0x7f943c06e520 2024-04-20T11:27:54.014 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.012+0000 7f9433400640 1 -- 172.21.15.152:0/3697878515 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f942c004a20 con 0x7f943c06e520 2024-04-20T11:27:54.014 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.012+0000 7f9440a00640 1 -- 172.21.15.152:0/3697878515 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f943c153290 con 0x7f943c06e520 2024-04-20T11:27:54.014 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.012+0000 7f9440a00640 1 -- 172.21.15.152:0/3697878515 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f943c150840 con 0x7f943c06e520 2024-04-20T11:27:54.014 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.013+0000 7f9440a00640 1 -- 172.21.15.152:0/3697878515 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f943c075e60 con 0x7f943c06e520 2024-04-20T11:27:54.015 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.014+0000 7f9433400640 1 -- 172.21.15.152:0/3697878515 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f942c004bc0 con 0x7f943c06e520 2024-04-20T11:27:54.016 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.015+0000 7f9433400640 1 --2- 172.21.15.152:0/3697878515 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f941c06c8f0 0x7f941c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:54.016 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.015+0000 7f9433400640 1 -- 172.21.15.152:0/3697878515 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f942c08aec0 con 0x7f943c06e520 2024-04-20T11:27:54.016 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.015+0000 7f943aa00640 1 --2- 172.21.15.152:0/3697878515 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f941c06c8f0 0x7f941c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:54.016 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.016+0000 7f943aa00640 1 --2- 172.21.15.152:0/3697878515 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f941c06c8f0 0x7f941c06edb0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9428006fd0 tx=0x7f9428008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:54.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.020+0000 7f9433400640 1 -- 172.21.15.152:0/3697878515 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f942c059260 con 0x7f943c06e520 2024-04-20T11:27:54.221 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.220+0000 7f9440a00640 1 -- 172.21.15.152:0/3697878515 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f943c0adb60 con 0x7f943c06e520 2024-04-20T11:27:54.223 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.222+0000 7f9433400640 1 -- 172.21.15.152:0/3697878515 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16914 (secure 0 0 0) 0x7f942c058b60 con 0x7f943c06e520 2024-04-20T11:27:54.223 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:27:54.224 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":23,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:27:42.601620+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:26:56.261424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T11:26:57.079286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":2}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.26","mappings":[{"from":7,"to":1}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:27:54.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.225+0000 7f9440a00640 1 -- 172.21.15.152:0/3697878515 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f941c06c8f0 msgr2=0x7f941c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:54.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.225+0000 7f9440a00640 1 --2- 172.21.15.152:0/3697878515 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f941c06c8f0 0x7f941c06edb0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9428006fd0 tx=0x7f9428008040 comp rx=0 tx=0).stop 2024-04-20T11:27:54.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.225+0000 7f9440a00640 1 -- 172.21.15.152:0/3697878515 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f943c06e520 msgr2=0x7f943c1500b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:54.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.225+0000 7f9440a00640 1 --2- 172.21.15.152:0/3697878515 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f943c06e520 0x7f943c1500b0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f942c00c500 tx=0x7f942c00c9c0 comp rx=0 tx=0).stop 2024-04-20T11:27:54.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.225+0000 7f9440a00640 1 -- 172.21.15.152:0/3697878515 shutdown_connections 2024-04-20T11:27:54.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.225+0000 7f9440a00640 1 --2- 172.21.15.152:0/3697878515 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f941c06c8f0 0x7f941c06edb0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:54.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.225+0000 7f9440a00640 1 --2- 172.21.15.152:0/3697878515 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f943c15e890 0x7f943c15a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:54.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.225+0000 7f9440a00640 1 --2- 172.21.15.152:0/3697878515 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f943c0add80 0x7f943c153910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:54.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.225+0000 7f9440a00640 1 --2- 172.21.15.152:0/3697878515 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f943c06e520 0x7f943c1500b0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:54.227 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.225+0000 7f9440a00640 1 -- 172.21.15.152:0/3697878515 >> 172.21.15.152:0/3697878515 conn(0x7f943c073340 msgr2=0x7f943c06cbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:54.227 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.225+0000 7f9440a00640 1 -- 172.21.15.152:0/3697878515 shutdown_connections 2024-04-20T11:27:54.227 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.226+0000 7f9440a00640 1 -- 172.21.15.152:0/3697878515 wait complete. 2024-04-20T11:27:54.246 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T11:26:47.246571+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T11:26:56.261424+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T11:26:57.079286+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T11:27:54.247 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T11:27:54.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.434+0000 7f07b0000640 1 Processor -- start 2024-04-20T11:27:54.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.434+0000 7f07b0000640 1 -- start start 2024-04-20T11:27:54.436 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.436+0000 7f07b0000640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f07a8146000 0x7f07a80b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:54.436 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.436+0000 7f07b0000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f07a80cd8c0 0x7f07a80b2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:54.437 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.436+0000 7f07b0000640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07a81463e0 con 0x7f07a80b2170 2024-04-20T11:27:54.437 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.436+0000 7f07b0000640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f07a80b4750 con 0x7f07a8146000 2024-04-20T11:27:54.437 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.436+0000 7f07b0000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f07a80b3e20 con 0x7f07a80cd8c0 2024-04-20T11:27:54.437 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.436+0000 7f07acc00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f07a80b2170 0x7f07a80cd1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:44914/0 (socket says 172.21.15.152:44914) 2024-04-20T11:27:54.437 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.436+0000 7f07acc00640 1 -- 172.21.15.152:0/4030398614 learned_addr learned my addr 172.21.15.152:0/4030398614 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:27:54.439 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.436+0000 7f07ae000640 1 --2- 172.21.15.152:0/4030398614 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f07a80cd8c0 0x7f07a80b2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:54.439 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.436+0000 7f07ad600640 1 --2- 172.21.15.152:0/4030398614 >> v2:172.21.15.175:3300/0 conn(0x7f07a8146000 0x7f07a80b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:54.439 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.437+0000 7f07a7e00640 1 -- 172.21.15.152:0/4030398614 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1538542374 0 0) 0x7f07a81463e0 con 0x7f07a80b2170 2024-04-20T11:27:54.440 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.437+0000 7f07a7e00640 1 -- 172.21.15.152:0/4030398614 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0790003520 con 0x7f07a80b2170 2024-04-20T11:27:54.440 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.437+0000 7f07a7e00640 1 -- 172.21.15.152:0/4030398614 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2139019087 0 0) 0x7f0790003520 con 0x7f07a80b2170 2024-04-20T11:27:54.440 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.437+0000 7f07a7e00640 1 -- 172.21.15.152:0/4030398614 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f07a80cd8c0 msgr2=0x7f07a80b2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:54.440 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.437+0000 7f07a7e00640 1 --2- 172.21.15.152:0/4030398614 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f07a80cd8c0 0x7f07a80b2620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:54.440 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.437+0000 7f07a7e00640 1 -- 172.21.15.152:0/4030398614 >> v2:172.21.15.175:3300/0 conn(0x7f07a8146000 msgr2=0x7f07a80b4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:54.440 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.437+0000 7f07a7e00640 1 --2- 172.21.15.152:0/4030398614 >> v2:172.21.15.175:3300/0 conn(0x7f07a8146000 0x7f07a80b4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:54.440 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.437+0000 7f07a7e00640 1 -- 172.21.15.152:0/4030398614 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07a80b4100 con 0x7f07a80b2170 2024-04-20T11:27:54.440 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.437+0000 7f07a7e00640 1 -- 172.21.15.152:0/4030398614 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f0794001690 con 0x7f07a80b2170 2024-04-20T11:27:54.440 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.437+0000 7f07a7e00640 1 -- 172.21.15.152:0/4030398614 >> v1:172.21.15.152:6789/0 conn(0x7f07a80b2170 legacy=0x7f07a80cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:54.440 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.437+0000 7f07a7e00640 1 --2- 172.21.15.152:0/4030398614 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f07900039b0 0x7f0790003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:54.440 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.437+0000 7f07a7e00640 1 --2- 172.21.15.152:0/4030398614 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0790004800 0x7f0790004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:54.440 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.437+0000 7f07a7e00640 1 --2- 172.21.15.152:0/4030398614 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f07a80cd8c0 0x7f07900051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:54.440 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.437+0000 7f07a7e00640 1 -- 172.21.15.152:0/4030398614 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f07a80b4750 con 0x7f0790004800 2024-04-20T11:27:54.440 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.437+0000 7f07a7e00640 1 -- 172.21.15.152:0/4030398614 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f07a80b3e20 con 0x7f07900039b0 2024-04-20T11:27:54.441 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.437+0000 7f07a7e00640 1 -- 172.21.15.152:0/4030398614 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f079000fe20 con 0x7f07a80cd8c0 2024-04-20T11:27:54.441 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.437+0000 7f07acc00640 1 --2- 172.21.15.152:0/4030398614 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0790004800 0x7f0790004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:54.441 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.438+0000 7f07ad600640 1 --2- 172.21.15.152:0/4030398614 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f07900039b0 0x7f0790003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:54.441 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.438+0000 7f07acc00640 1 -- 172.21.15.152:0/4030398614 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f07a80cd8c0 msgr2=0x7f07900051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:54.441 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.438+0000 7f07acc00640 1 --2- 172.21.15.152:0/4030398614 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f07a80cd8c0 0x7f07900051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:54.441 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.438+0000 7f07acc00640 1 -- 172.21.15.152:0/4030398614 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f07900039b0 msgr2=0x7f0790003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:54.441 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.438+0000 7f07acc00640 1 --2- 172.21.15.152:0/4030398614 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f07900039b0 0x7f0790003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:54.441 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.438+0000 7f07acc00640 1 -- 172.21.15.152:0/4030398614 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0790003520 con 0x7f0790004800 2024-04-20T11:27:54.441 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.439+0000 7f07acc00640 1 --2- 172.21.15.152:0/4030398614 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0790004800 0x7f0790004c80 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f079400c160 tx=0x7f0790005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=994c3c27d2d4445a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:54.441 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.439+0000 7f07a7e00640 1 -- 172.21.15.152:0/4030398614 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0794007ca0 con 0x7f0790004800 2024-04-20T11:27:54.441 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.439+0000 7f07a7e00640 1 -- 172.21.15.152:0/4030398614 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0794030070 con 0x7f0790004800 2024-04-20T11:27:54.441 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.439+0000 7f07a7e00640 1 -- 172.21.15.152:0/4030398614 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0794005d10 con 0x7f0790004800 2024-04-20T11:27:54.442 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.440+0000 7f07b0000640 1 -- 172.21.15.152:0/4030398614 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0790004800 msgr2=0x7f0790004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:54.443 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.440+0000 7f07b0000640 1 --2- 172.21.15.152:0/4030398614 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0790004800 0x7f0790004c80 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f079400c160 tx=0x7f0790005830 comp rx=0 tx=0).stop 2024-04-20T11:27:54.443 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.440+0000 7f07ae000640 1 -- 172.21.15.152:0/4030398614 reap_dead start 2024-04-20T11:27:54.443 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.440+0000 7f07b0000640 1 -- 172.21.15.152:0/4030398614 shutdown_connections 2024-04-20T11:27:54.443 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.440+0000 7f07b0000640 1 -- 172.21.15.152:0/4030398614 >> 172.21.15.152:0/4030398614 conn(0x7f07a806b980 msgr2=0x7f07a806bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:54.443 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.440+0000 7f07b0000640 1 -- 172.21.15.152:0/4030398614 shutdown_connections 2024-04-20T11:27:54.443 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.440+0000 7f07b0000640 1 -- 172.21.15.152:0/4030398614 wait complete. 2024-04-20T11:27:54.443 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.442+0000 7f07b0000640 1 Processor -- start 2024-04-20T11:27:54.443 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.442+0000 7f07b0000640 1 -- start start 2024-04-20T11:27:54.445 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.443+0000 7f07b0000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0790004800 0x7f07a814dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:54.445 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.443+0000 7f07b0000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f07a806f2f0 0x7f07a806f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:54.445 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.443+0000 7f07b0000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f07a813d020 0x7f07a813d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:54.445 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.443+0000 7f07b0000640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f07a80b27a0 con 0x7f07a806f2f0 2024-04-20T11:27:54.445 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.443+0000 7f07b0000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f07a80b2620 con 0x7f07a813d020 2024-04-20T11:27:54.445 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.443+0000 7f07b0000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f07a80b2920 con 0x7f0790004800 2024-04-20T11:27:54.445 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.443+0000 7f07acc00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f07a806f2f0 0x7f07a806f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:54.445 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.443+0000 7f07acc00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f07a806f2f0 0x7f07a806f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43492/0 (socket says 172.21.15.152:43492) 2024-04-20T11:27:54.445 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.443+0000 7f07acc00640 1 -- 172.21.15.152:0/2183235186 learned_addr learned my addr 172.21.15.152:0/2183235186 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:54.445 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.443+0000 7f07ad600640 1 --2- 172.21.15.152:0/2183235186 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0790004800 0x7f07a814dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:54.446 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.443+0000 7f07acc00640 1 -- 172.21.15.152:0/2183235186 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0790004800 msgr2=0x7f07a814dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:54.446 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.443+0000 7f07acc00640 1 --2- 172.21.15.152:0/2183235186 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0790004800 0x7f07a814dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:54.446 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.443+0000 7f07acc00640 1 -- 172.21.15.152:0/2183235186 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f07a813d020 msgr2=0x7f07a813d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:54.446 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.443+0000 7f07ae000640 1 --2- 172.21.15.152:0/2183235186 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f07a813d020 0x7f07a813d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:54.446 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.443+0000 7f07acc00640 1 --2- 172.21.15.152:0/2183235186 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f07a813d020 0x7f07a813d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:54.446 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.443+0000 7f07acc00640 1 -- 172.21.15.152:0/2183235186 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07a8140260 con 0x7f07a806f2f0 2024-04-20T11:27:54.446 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.444+0000 7f07acc00640 1 --2- 172.21.15.152:0/2183235186 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f07a806f2f0 0x7f07a806f770 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f079402c480 tx=0x7f0794001990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:54.446 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.444+0000 7f07a5600640 1 -- 172.21.15.152:0/2183235186 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0794045070 con 0x7f07a806f2f0 2024-04-20T11:27:54.446 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.444+0000 7f07a5600640 1 -- 172.21.15.152:0/2183235186 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0794004bf0 con 0x7f07a806f2f0 2024-04-20T11:27:54.447 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.444+0000 7f07a5600640 1 -- 172.21.15.152:0/2183235186 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0794007ca0 con 0x7f07a806f2f0 2024-04-20T11:27:54.447 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.444+0000 7f07b0000640 1 -- 172.21.15.152:0/2183235186 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07a813f4a0 con 0x7f07a806f2f0 2024-04-20T11:27:54.447 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.444+0000 7f07b0000640 1 -- 172.21.15.152:0/2183235186 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07a813fa60 con 0x7f07a806f2f0 2024-04-20T11:27:54.447 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.446+0000 7f07b0000640 1 -- 172.21.15.152:0/2183235186 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07a80689f0 con 0x7f07a806f2f0 2024-04-20T11:27:54.447 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.447+0000 7f07a5600640 1 -- 172.21.15.152:0/2183235186 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0794007070 con 0x7f07a806f2f0 2024-04-20T11:27:54.448 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.448+0000 7f07a5600640 1 --2- 172.21.15.152:0/2183235186 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f078806c8f0 0x7f078806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:54.448 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.448+0000 7f07a5600640 1 -- 172.21.15.152:0/2183235186 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f07940acb60 con 0x7f07a806f2f0 2024-04-20T11:27:54.449 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.448+0000 7f07ad600640 1 --2- 172.21.15.152:0/2183235186 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f078806c8f0 0x7f078806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:54.449 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.449+0000 7f07ad600640 1 --2- 172.21.15.152:0/2183235186 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f078806c8f0 0x7f078806edb0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f079c009910 tx=0x7f079c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:54.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.453+0000 7f07a5600640 1 -- 172.21.15.152:0/2183235186 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f079407aff0 con 0x7f07a806f2f0 2024-04-20T11:27:54.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.657+0000 7f07b0000640 1 -- 172.21.15.152:0/2183235186 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f07a8143700 con 0x7f07a806f2f0 2024-04-20T11:27:54.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.659+0000 7f07a5600640 1 -- 172.21.15.152:0/2183235186 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) ==== 93+0+10 (secure 0 0 0) 0x7f079402de20 con 0x7f07a806f2f0 2024-04-20T11:27:54.659 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 1 2024-04-20T11:27:54.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.662+0000 7f07b0000640 1 -- 172.21.15.152:0/2183235186 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f078806c8f0 msgr2=0x7f078806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:54.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.662+0000 7f07b0000640 1 --2- 172.21.15.152:0/2183235186 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f078806c8f0 0x7f078806edb0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f079c009910 tx=0x7f079c008040 comp rx=0 tx=0).stop 2024-04-20T11:27:54.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.662+0000 7f07b0000640 1 -- 172.21.15.152:0/2183235186 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f07a806f2f0 msgr2=0x7f07a806f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:54.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.662+0000 7f07b0000640 1 --2- 172.21.15.152:0/2183235186 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f07a806f2f0 0x7f07a806f770 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f079402c480 tx=0x7f0794001990 comp rx=0 tx=0).stop 2024-04-20T11:27:54.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.662+0000 7f07b0000640 1 -- 172.21.15.152:0/2183235186 shutdown_connections 2024-04-20T11:27:54.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.662+0000 7f07b0000640 1 --2- 172.21.15.152:0/2183235186 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f078806c8f0 0x7f078806edb0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:54.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.662+0000 7f07b0000640 1 --2- 172.21.15.152:0/2183235186 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f07a813d020 0x7f07a813d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:54.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.662+0000 7f07b0000640 1 --2- 172.21.15.152:0/2183235186 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f07a806f2f0 0x7f07a806f770 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:54.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.662+0000 7f07b0000640 1 --2- 172.21.15.152:0/2183235186 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0790004800 0x7f07a814dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:54.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.662+0000 7f07b0000640 1 -- 172.21.15.152:0/2183235186 >> 172.21.15.152:0/2183235186 conn(0x7f07a806b980 msgr2=0x7f07a806bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:54.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.662+0000 7f07b0000640 1 -- 172.21.15.152:0/2183235186 shutdown_connections 2024-04-20T11:27:54.664 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.662+0000 7f07b0000640 1 -- 172.21.15.152:0/2183235186 wait complete. 2024-04-20T11:27:54.683 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T11:27:54.891 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.891+0000 7f345f200640 1 Processor -- start 2024-04-20T11:27:54.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.891+0000 7f345f200640 1 -- start start 2024-04-20T11:27:54.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.893+0000 7f345f200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3458071cd0 0x7f34580720b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:54.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.893+0000 7f345f200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f3458141450 0x7f34581418e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:54.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.893+0000 7f345f200640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f345806fb40 con 0x7f3458160220 2024-04-20T11:27:54.894 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.893+0000 7f345f200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f345813f8e0 con 0x7f3458141450 2024-04-20T11:27:54.894 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.893+0000 7f345f200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f34580725f0 con 0x7f3458071cd0 2024-04-20T11:27:54.894 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.893+0000 7f345d400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f3458160220 0x7f3458160680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:44916/0 (socket says 172.21.15.152:44916) 2024-04-20T11:27:54.894 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.893+0000 7f345d400640 1 -- 172.21.15.152:0/2773518227 learned_addr learned my addr 172.21.15.152:0/2773518227 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:27:54.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.893+0000 7f345de00640 1 --2- 172.21.15.152:0/2773518227 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3458071cd0 0x7f34580720b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:54.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.893+0000 7f345e800640 1 --2- 172.21.15.152:0/2773518227 >> v2:172.21.15.175:3300/0 conn(0x7f3458141450 0x7f34581418e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:54.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.894+0000 7f345ca00640 1 -- 172.21.15.152:0/2773518227 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 311968719 0 0) 0x7f345806fb40 con 0x7f3458160220 2024-04-20T11:27:54.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.894+0000 7f345ca00640 1 -- 172.21.15.152:0/2773518227 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3440003520 con 0x7f3458160220 2024-04-20T11:27:54.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.894+0000 7f345ca00640 1 -- 172.21.15.152:0/2773518227 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 72199008 0 0) 0x7f3440003520 con 0x7f3458160220 2024-04-20T11:27:54.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.894+0000 7f345ca00640 1 -- 172.21.15.152:0/2773518227 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3458071cd0 msgr2=0x7f34580720b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:54.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.894+0000 7f345ca00640 1 --2- 172.21.15.152:0/2773518227 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3458071cd0 0x7f34580720b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:54.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.894+0000 7f345ca00640 1 -- 172.21.15.152:0/2773518227 >> v2:172.21.15.175:3300/0 conn(0x7f3458141450 msgr2=0x7f34581418e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:54.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.894+0000 7f345ca00640 1 --2- 172.21.15.152:0/2773518227 >> v2:172.21.15.175:3300/0 conn(0x7f3458141450 0x7f34581418e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:54.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.894+0000 7f345ca00640 1 -- 172.21.15.152:0/2773518227 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3458160f90 con 0x7f3458160220 2024-04-20T11:27:54.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.894+0000 7f345ca00640 1 -- 172.21.15.152:0/2773518227 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f3448001670 con 0x7f3458160220 2024-04-20T11:27:54.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.894+0000 7f345ca00640 1 -- 172.21.15.152:0/2773518227 >> v1:172.21.15.152:6789/0 conn(0x7f3458160220 legacy=0x7f3458160680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:54.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.894+0000 7f345ca00640 1 --2- 172.21.15.152:0/2773518227 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f34400039b0 0x7f3440003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:54.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.894+0000 7f345ca00640 1 --2- 172.21.15.152:0/2773518227 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3440004800 0x7f3440004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:54.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.894+0000 7f345ca00640 1 --2- 172.21.15.152:0/2773518227 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3458071cd0 0x7f34400051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:54.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.894+0000 7f345ca00640 1 -- 172.21.15.152:0/2773518227 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f345813f8e0 con 0x7f3440004800 2024-04-20T11:27:54.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.894+0000 7f345ca00640 1 -- 172.21.15.152:0/2773518227 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f34580725f0 con 0x7f34400039b0 2024-04-20T11:27:54.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.894+0000 7f345ca00640 1 -- 172.21.15.152:0/2773518227 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f344000b9c0 con 0x7f3458071cd0 2024-04-20T11:27:54.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.895+0000 7f345d400640 1 --2- 172.21.15.152:0/2773518227 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3440004800 0x7f3440004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:54.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.895+0000 7f345de00640 1 --2- 172.21.15.152:0/2773518227 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f34400039b0 0x7f3440003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:54.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.895+0000 7f345e800640 1 --2- 172.21.15.152:0/2773518227 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3458071cd0 0x7f34400051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:54.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.895+0000 7f345d400640 1 -- 172.21.15.152:0/2773518227 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3458071cd0 msgr2=0x7f34400051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:54.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.895+0000 7f345d400640 1 --2- 172.21.15.152:0/2773518227 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3458071cd0 0x7f34400051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:54.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.895+0000 7f345d400640 1 -- 172.21.15.152:0/2773518227 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f34400039b0 msgr2=0x7f3440003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:54.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.895+0000 7f345d400640 1 --2- 172.21.15.152:0/2773518227 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f34400039b0 0x7f3440003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:54.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.895+0000 7f345d400640 1 -- 172.21.15.152:0/2773518227 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3440003520 con 0x7f3440004800 2024-04-20T11:27:54.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.896+0000 7f345d400640 1 --2- 172.21.15.152:0/2773518227 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3440004800 0x7f3440004c80 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f344800c210 tx=0x7f3440005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6fcf15794a0f6ce server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:54.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.896+0000 7f345ca00640 1 -- 172.21.15.152:0/2773518227 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3448007d50 con 0x7f3440004800 2024-04-20T11:27:54.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.897+0000 7f345ca00640 1 -- 172.21.15.152:0/2773518227 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3448030070 con 0x7f3440004800 2024-04-20T11:27:54.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.897+0000 7f345ca00640 1 -- 172.21.15.152:0/2773518227 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f344802cd20 con 0x7f3440004800 2024-04-20T11:27:54.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.897+0000 7f345f200640 1 -- 172.21.15.152:0/2773518227 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3440004800 msgr2=0x7f3440004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:54.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.897+0000 7f345f200640 1 --2- 172.21.15.152:0/2773518227 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3440004800 0x7f3440004c80 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f344800c210 tx=0x7f3440005830 comp rx=0 tx=0).stop 2024-04-20T11:27:54.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.897+0000 7f345e800640 1 -- 172.21.15.152:0/2773518227 reap_dead start 2024-04-20T11:27:54.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.897+0000 7f345f200640 1 -- 172.21.15.152:0/2773518227 shutdown_connections 2024-04-20T11:27:54.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.897+0000 7f345f200640 1 -- 172.21.15.152:0/2773518227 >> 172.21.15.152:0/2773518227 conn(0x7f3458067ae0 msgr2=0x7f345806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:54.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.897+0000 7f345f200640 1 -- 172.21.15.152:0/2773518227 shutdown_connections 2024-04-20T11:27:54.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.897+0000 7f345f200640 1 -- 172.21.15.152:0/2773518227 wait complete. 2024-04-20T11:27:54.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.899+0000 7f345f200640 1 Processor -- start 2024-04-20T11:27:54.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.899+0000 7f345f200640 1 -- start start 2024-04-20T11:27:54.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.900+0000 7f345f200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3440004800 0x7f345813bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:54.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.900+0000 7f345f200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3458157090 0x7f3458152670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:54.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.900+0000 7f345f200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f34581f4110 0x7f34580b0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:54.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.900+0000 7f345f200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f345806e350 con 0x7f3440004800 2024-04-20T11:27:54.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.900+0000 7f345f200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f345806e070 con 0x7f3458157090 2024-04-20T11:27:54.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.900+0000 7f345f200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f345806e630 con 0x7f34581f4110 2024-04-20T11:27:54.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.900+0000 7f345de00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3440004800 0x7f345813bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:54.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.900+0000 7f345de00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3440004800 0x7f345813bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43502/0 (socket says 172.21.15.152:43502) 2024-04-20T11:27:54.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.900+0000 7f345de00640 1 -- 172.21.15.152:0/1033561832 learned_addr learned my addr 172.21.15.152:0/1033561832 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:54.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.901+0000 7f345d400640 1 --2- 172.21.15.152:0/1033561832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3458157090 0x7f3458152670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:54.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.901+0000 7f345e800640 1 --2- 172.21.15.152:0/1033561832 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f34581f4110 0x7f34580b0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:54.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.901+0000 7f345de00640 1 -- 172.21.15.152:0/1033561832 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f34581f4110 msgr2=0x7f34580b0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:54.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.901+0000 7f345de00640 1 --2- 172.21.15.152:0/1033561832 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f34581f4110 0x7f34580b0190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:54.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.901+0000 7f345de00640 1 -- 172.21.15.152:0/1033561832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3458157090 msgr2=0x7f3458152670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:54.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.901+0000 7f345de00640 1 --2- 172.21.15.152:0/1033561832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3458157090 0x7f3458152670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:54.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.901+0000 7f345de00640 1 -- 172.21.15.152:0/1033561832 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3458152bb0 con 0x7f3440004800 2024-04-20T11:27:54.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.901+0000 7f345d400640 1 --2- 172.21.15.152:0/1033561832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3458157090 0x7f3458152670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:27:54.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.901+0000 7f345de00640 1 --2- 172.21.15.152:0/1033561832 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3440004800 0x7f345813bf60 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f344c011540 tx=0x7f344c011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:54.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.901+0000 7f3452000640 1 -- 172.21.15.152:0/1033561832 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f344c015930 con 0x7f3440004800 2024-04-20T11:27:54.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.901+0000 7f3452000640 1 -- 172.21.15.152:0/1033561832 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f344c015ad0 con 0x7f3440004800 2024-04-20T11:27:54.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.901+0000 7f3452000640 1 -- 172.21.15.152:0/1033561832 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f344c01d400 con 0x7f3440004800 2024-04-20T11:27:54.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.901+0000 7f345f200640 1 -- 172.21.15.152:0/1033561832 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34581f3950 con 0x7f3440004800 2024-04-20T11:27:54.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.902+0000 7f345f200640 1 -- 172.21.15.152:0/1033561832 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f345813c5f0 con 0x7f3440004800 2024-04-20T11:27:54.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.903+0000 7f345f200640 1 -- 172.21.15.152:0/1033561832 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3458075e60 con 0x7f3440004800 2024-04-20T11:27:54.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.904+0000 7f3452000640 1 -- 172.21.15.152:0/1033561832 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f344c01d5a0 con 0x7f3440004800 2024-04-20T11:27:54.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.904+0000 7f3452000640 1 --2- 172.21.15.152:0/1033561832 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f343c06c860 0x7f343c06ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:54.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.905+0000 7f3452000640 1 -- 172.21.15.152:0/1033561832 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f344c08c2f0 con 0x7f3440004800 2024-04-20T11:27:54.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.905+0000 7f345d400640 1 --2- 172.21.15.152:0/1033561832 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f343c06c860 0x7f343c06ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:54.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.906+0000 7f345d400640 1 --2- 172.21.15.152:0/1033561832 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f343c06c860 0x7f343c06ed20 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f3448034000 tx=0x7f3448007000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:54.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:54.909+0000 7f3452000640 1 -- 172.21.15.152:0/1033561832 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f344c019030 con 0x7f3440004800 2024-04-20T11:27:55.109 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.109+0000 7f345f200640 1 -- 172.21.15.152:0/1033561832 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f3458072d30 con 0x7f3440004800 2024-04-20T11:27:55.110 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.109+0000 7f3452000640 1 -- 172.21.15.152:0/1033561832 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) ==== 104+0+11 (secure 0 0 0) 0x7f344c05a690 con 0x7f3440004800 2024-04-20T11:27:55.110 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 32 2024-04-20T11:27:55.113 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.112+0000 7f345f200640 1 -- 172.21.15.152:0/1033561832 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f343c06c860 msgr2=0x7f343c06ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:55.113 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.112+0000 7f345f200640 1 --2- 172.21.15.152:0/1033561832 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f343c06c860 0x7f343c06ed20 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f3448034000 tx=0x7f3448007000 comp rx=0 tx=0).stop 2024-04-20T11:27:55.113 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.112+0000 7f345f200640 1 -- 172.21.15.152:0/1033561832 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3440004800 msgr2=0x7f345813bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:55.113 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.112+0000 7f345f200640 1 --2- 172.21.15.152:0/1033561832 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3440004800 0x7f345813bf60 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f344c011540 tx=0x7f344c011a00 comp rx=0 tx=0).stop 2024-04-20T11:27:55.113 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.112+0000 7f345f200640 1 -- 172.21.15.152:0/1033561832 shutdown_connections 2024-04-20T11:27:55.113 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.112+0000 7f345f200640 1 --2- 172.21.15.152:0/1033561832 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f343c06c860 0x7f343c06ed20 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:55.113 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.112+0000 7f345f200640 1 --2- 172.21.15.152:0/1033561832 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f34581f4110 0x7f34580b0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:55.113 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.112+0000 7f345f200640 1 --2- 172.21.15.152:0/1033561832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3458157090 0x7f3458152670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:55.114 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.112+0000 7f345f200640 1 --2- 172.21.15.152:0/1033561832 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3440004800 0x7f345813bf60 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:55.114 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.112+0000 7f345f200640 1 -- 172.21.15.152:0/1033561832 >> 172.21.15.152:0/1033561832 conn(0x7f3458067ae0 msgr2=0x7f345813fb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:55.114 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.113+0000 7f345f200640 1 -- 172.21.15.152:0/1033561832 shutdown_connections 2024-04-20T11:27:55.114 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.113+0000 7f345f200640 1 -- 172.21.15.152:0/1033561832 wait complete. 2024-04-20T11:27:55.135 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T11:27:55.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.338+0000 7ff9fca00640 1 Processor -- start 2024-04-20T11:27:55.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.338+0000 7ff9fca00640 1 -- start start 2024-04-20T11:27:55.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.339+0000 7ff9fca00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7ff9f815b510 0x7ff9f815fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:55.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.340+0000 7ff9fca00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ff9f80cfc40 0x7ff9f80b3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:55.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.340+0000 7ff9fca00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff9f80b2170 0x7ff9f80b2600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:55.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.340+0000 7ff9fca00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7ff9f806f420 con 0x7ff9f815b510 2024-04-20T11:27:55.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.340+0000 7ff9fca00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7ff9f80b4430 con 0x7ff9f80cfc40 2024-04-20T11:27:55.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.340+0000 7ff9fca00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff9f815f1c0 con 0x7ff9f80b2170 2024-04-20T11:27:55.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.340+0000 7ff9f5e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7ff9f815b510 0x7ff9f815fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:55.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.340+0000 7ff9f5e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7ff9f815b510 0x7ff9f815fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43504/0 (socket says 172.21.15.152:43504) 2024-04-20T11:27:55.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.340+0000 7ff9f5e00640 1 -- 172.21.15.152:0/1748265016 learned_addr learned my addr 172.21.15.152:0/1748265016 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:55.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.340+0000 7ff9f6800640 1 --2- 172.21.15.152:0/1748265016 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff9f80b2170 0x7ff9f80b2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:55.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.340+0000 7ff9f5e00640 1 -- 172.21.15.152:0/1748265016 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff9f80b2170 msgr2=0x7ff9f80b2600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:55.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.340+0000 7ff9f5e00640 1 --2- 172.21.15.152:0/1748265016 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff9f80b2170 0x7ff9f80b2600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:55.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.340+0000 7ff9efe00640 1 --2- 172.21.15.152:0/1748265016 >> v2:172.21.15.175:3300/0 conn(0x7ff9f80cfc40 0x7ff9f80b3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:55.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.340+0000 7ff9f5e00640 1 -- 172.21.15.152:0/1748265016 >> v2:172.21.15.175:3300/0 conn(0x7ff9f80cfc40 msgr2=0x7ff9f80b3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:55.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.341+0000 7ff9f5e00640 1 --2- 172.21.15.152:0/1748265016 >> v2:172.21.15.175:3300/0 conn(0x7ff9f80cfc40 0x7ff9f80b3ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:55.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.341+0000 7ff9f5e00640 1 -- 172.21.15.152:0/1748265016 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9f806f670 con 0x7ff9f815b510 2024-04-20T11:27:55.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.341+0000 7ff9efe00640 1 --2- 172.21.15.152:0/1748265016 >> v2:172.21.15.175:3300/0 conn(0x7ff9f80cfc40 0x7ff9f80b3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:27:55.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.342+0000 7ff9f5e00640 1 --2- 172.21.15.152:0/1748265016 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff9f815b510 0x7ff9f815fc30 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff9f8072fa0 tx=0x7ff9e802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65fe8166b7ca8bfe server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:55.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.342+0000 7ff9f5400640 1 -- 172.21.15.152:0/1748265016 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e8035020 con 0x7ff9f815b510 2024-04-20T11:27:55.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.342+0000 7ff9f5400640 1 -- 172.21.15.152:0/1748265016 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9e8031640 con 0x7ff9f815b510 2024-04-20T11:27:55.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.342+0000 7ff9f5400640 1 -- 172.21.15.152:0/1748265016 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e802d040 con 0x7ff9f815b510 2024-04-20T11:27:55.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.343+0000 7ff9fca00640 1 -- 172.21.15.152:0/1748265016 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff9f815b510 msgr2=0x7ff9f815fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:55.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.343+0000 7ff9fca00640 1 --2- 172.21.15.152:0/1748265016 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff9f815b510 0x7ff9f815fc30 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff9f8072fa0 tx=0x7ff9e802b8f0 comp rx=0 tx=0).stop 2024-04-20T11:27:55.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.343+0000 7ff9fca00640 1 -- 172.21.15.152:0/1748265016 shutdown_connections 2024-04-20T11:27:55.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.343+0000 7ff9fca00640 1 --2- 172.21.15.152:0/1748265016 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff9f80b2170 0x7ff9f80b2600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:55.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.343+0000 7ff9fca00640 1 --2- 172.21.15.152:0/1748265016 >> v2:172.21.15.175:3300/0 conn(0x7ff9f80cfc40 0x7ff9f80b3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:55.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.343+0000 7ff9fca00640 1 --2- 172.21.15.152:0/1748265016 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff9f815b510 0x7ff9f815fc30 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:55.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.343+0000 7ff9fca00640 1 -- 172.21.15.152:0/1748265016 >> 172.21.15.152:0/1748265016 conn(0x7ff9f806b980 msgr2=0x7ff9f806bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:55.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.343+0000 7ff9fca00640 1 -- 172.21.15.152:0/1748265016 shutdown_connections 2024-04-20T11:27:55.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.343+0000 7ff9fca00640 1 -- 172.21.15.152:0/1748265016 wait complete. 2024-04-20T11:27:55.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.345+0000 7ff9fca00640 1 Processor -- start 2024-04-20T11:27:55.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.345+0000 7ff9fca00640 1 -- start start 2024-04-20T11:27:55.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.346+0000 7ff9fca00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff9f80b2170 0x7ff9f814dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:55.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.346+0000 7ff9fca00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff9f80cfc40 0x7ff9f81593d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:55.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.346+0000 7ff9fca00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff9f815b510 0x7ff9f814d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:55.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.346+0000 7ff9fca00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f815b8f0 con 0x7ff9f815b510 2024-04-20T11:27:55.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.346+0000 7ff9fca00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f815f710 con 0x7ff9f80b2170 2024-04-20T11:27:55.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.346+0000 7ff9fca00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff9f806f950 con 0x7ff9f80cfc40 2024-04-20T11:27:55.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.346+0000 7ff9f6800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff9f815b510 0x7ff9f814d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:55.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.346+0000 7ff9f6800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff9f815b510 0x7ff9f814d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43506/0 (socket says 172.21.15.152:43506) 2024-04-20T11:27:55.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.346+0000 7ff9f6800640 1 -- 172.21.15.152:0/2401982820 learned_addr learned my addr 172.21.15.152:0/2401982820 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:55.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.347+0000 7ff9efe00640 1 --2- 172.21.15.152:0/2401982820 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff9f80cfc40 0x7ff9f81593d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:55.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.347+0000 7ff9f5e00640 1 --2- 172.21.15.152:0/2401982820 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff9f80b2170 0x7ff9f814dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:55.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.347+0000 7ff9f6800640 1 -- 172.21.15.152:0/2401982820 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff9f80cfc40 msgr2=0x7ff9f81593d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:55.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.347+0000 7ff9f6800640 1 --2- 172.21.15.152:0/2401982820 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff9f80cfc40 0x7ff9f81593d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:55.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.347+0000 7ff9f6800640 1 -- 172.21.15.152:0/2401982820 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff9f80b2170 msgr2=0x7ff9f814dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:55.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.347+0000 7ff9f6800640 1 --2- 172.21.15.152:0/2401982820 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff9f80b2170 0x7ff9f814dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:55.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.347+0000 7ff9f6800640 1 -- 172.21.15.152:0/2401982820 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9f814dae0 con 0x7ff9f815b510 2024-04-20T11:27:55.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.347+0000 7ff9f6800640 1 --2- 172.21.15.152:0/2401982820 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff9f815b510 0x7ff9f814d5a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ff9f000cd20 tx=0x7ff9f000a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:55.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.347+0000 7ff9ee000640 1 -- 172.21.15.152:0/2401982820 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f0003b90 con 0x7ff9f815b510 2024-04-20T11:27:55.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.347+0000 7ff9ee000640 1 -- 172.21.15.152:0/2401982820 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9f0003d30 con 0x7ff9f815b510 2024-04-20T11:27:55.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.347+0000 7ff9ee000640 1 -- 172.21.15.152:0/2401982820 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f0004240 con 0x7ff9f815b510 2024-04-20T11:27:55.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.347+0000 7ff9fca00640 1 -- 172.21.15.152:0/2401982820 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9f8157880 con 0x7ff9f815b510 2024-04-20T11:27:55.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.347+0000 7ff9fca00640 1 -- 172.21.15.152:0/2401982820 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9f8157e20 con 0x7ff9f815b510 2024-04-20T11:27:55.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.348+0000 7ff9fca00640 1 -- 172.21.15.152:0/2401982820 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9f80689f0 con 0x7ff9f815b510 2024-04-20T11:27:55.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.350+0000 7ff9ee000640 1 -- 172.21.15.152:0/2401982820 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff9f00043e0 con 0x7ff9f815b510 2024-04-20T11:27:55.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.350+0000 7ff9ee000640 1 --2- 172.21.15.152:0/2401982820 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff9d406c8f0 0x7ff9d406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:55.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.351+0000 7ff9ee000640 1 -- 172.21.15.152:0/2401982820 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7ff9f008c6c0 con 0x7ff9f815b510 2024-04-20T11:27:55.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.351+0000 7ff9f5e00640 1 --2- 172.21.15.152:0/2401982820 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff9d406c8f0 0x7ff9d406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:55.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.351+0000 7ff9f5e00640 1 --2- 172.21.15.152:0/2401982820 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff9d406c8f0 0x7ff9d406edb0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff9e8036000 tx=0x7ff9e8008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:55.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.355+0000 7ff9ee000640 1 -- 172.21.15.152:0/2401982820 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ff9f005a9d0 con 0x7ff9f815b510 2024-04-20T11:27:55.558 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.558+0000 7ff9fca00640 1 -- 172.21.15.152:0/2401982820 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff9f80ccc80 con 0x7ff9f815b510 2024-04-20T11:27:55.559 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.559+0000 7ff9ee000640 1 -- 172.21.15.152:0/2401982820 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) ==== 100+0+11 (secure 0 0 0) 0x7ff9f005a2d0 con 0x7ff9f815b510 2024-04-20T11:27:55.559 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 64 2024-04-20T11:27:55.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.561+0000 7ff9fca00640 1 -- 172.21.15.152:0/2401982820 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff9d406c8f0 msgr2=0x7ff9d406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:55.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.561+0000 7ff9fca00640 1 --2- 172.21.15.152:0/2401982820 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff9d406c8f0 0x7ff9d406edb0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff9e8036000 tx=0x7ff9e8008030 comp rx=0 tx=0).stop 2024-04-20T11:27:55.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.561+0000 7ff9fca00640 1 -- 172.21.15.152:0/2401982820 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff9f815b510 msgr2=0x7ff9f814d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:55.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.561+0000 7ff9fca00640 1 --2- 172.21.15.152:0/2401982820 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff9f815b510 0x7ff9f814d5a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ff9f000cd20 tx=0x7ff9f000a660 comp rx=0 tx=0).stop 2024-04-20T11:27:55.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.561+0000 7ff9fca00640 1 -- 172.21.15.152:0/2401982820 shutdown_connections 2024-04-20T11:27:55.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.561+0000 7ff9fca00640 1 --2- 172.21.15.152:0/2401982820 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff9d406c8f0 0x7ff9d406edb0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:55.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.561+0000 7ff9fca00640 1 --2- 172.21.15.152:0/2401982820 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff9f815b510 0x7ff9f814d5a0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:55.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.561+0000 7ff9fca00640 1 --2- 172.21.15.152:0/2401982820 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff9f80cfc40 0x7ff9f81593d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:55.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.561+0000 7ff9fca00640 1 --2- 172.21.15.152:0/2401982820 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff9f80b2170 0x7ff9f814dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:55.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.561+0000 7ff9fca00640 1 -- 172.21.15.152:0/2401982820 >> 172.21.15.152:0/2401982820 conn(0x7ff9f806b980 msgr2=0x7ff9f806dc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:55.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.561+0000 7ff9fca00640 1 -- 172.21.15.152:0/2401982820 shutdown_connections 2024-04-20T11:27:55.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.561+0000 7ff9fca00640 1 -- 172.21.15.152:0/2401982820 wait complete. 2024-04-20T11:27:55.581 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-20T11:27:55.784 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.783+0000 7ff788600640 1 Processor -- start 2024-04-20T11:27:55.784 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.784+0000 7ff788600640 1 -- start start 2024-04-20T11:27:55.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.785+0000 7ff788600640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ff78006e7e0 0x7ff780162580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:55.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.785+0000 7ff788600640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7ff7800d0730 0x7ff780162ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:55.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.785+0000 7ff788600640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff78009bd90 con 0x7ff78006e400 2024-04-20T11:27:55.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.785+0000 7ff788600640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7ff7800d1870 con 0x7ff7800d0730 2024-04-20T11:27:55.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.785+0000 7ff788600640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7ff7800d0c60 con 0x7ff78006e7e0 2024-04-20T11:27:55.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.785+0000 7ff785c00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7ff78006e400 0x7ff78009b680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:44928/0 (socket says 172.21.15.152:44928) 2024-04-20T11:27:55.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.785+0000 7ff785c00640 1 -- 172.21.15.152:0/2614227638 learned_addr learned my addr 172.21.15.152:0/2614227638 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:27:55.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.785+0000 7ff786600640 1 --2- 172.21.15.152:0/2614227638 >> v2:172.21.15.152:3300/0 conn(0x7ff7800d0730 0x7ff780162ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:55.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.786+0000 7ff785200640 1 --2- 172.21.15.152:0/2614227638 >> v2:172.21.15.175:3300/0 conn(0x7ff78006e7e0 0x7ff780162580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:55.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.786+0000 7ff77fe00640 1 -- 172.21.15.152:0/2614227638 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2708951572 0 0) 0x7ff78009bd90 con 0x7ff78006e400 2024-04-20T11:27:55.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.786+0000 7ff77fe00640 1 -- 172.21.15.152:0/2614227638 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff764003520 con 0x7ff78006e400 2024-04-20T11:27:55.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.786+0000 7ff786600640 1 -- 172.21.15.152:0/2614227638 >> v2:172.21.15.175:3300/0 conn(0x7ff78006e7e0 msgr2=0x7ff780162580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:55.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.786+0000 7ff786600640 1 --2- 172.21.15.152:0/2614227638 >> v2:172.21.15.175:3300/0 conn(0x7ff78006e7e0 0x7ff780162580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:55.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.786+0000 7ff786600640 1 -- 172.21.15.152:0/2614227638 >> v1:172.21.15.152:6789/0 conn(0x7ff78006e400 legacy=0x7ff78009b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:55.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.786+0000 7ff786600640 1 -- 172.21.15.152:0/2614227638 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7800d0f40 con 0x7ff7800d0730 2024-04-20T11:27:55.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.788+0000 7ff786600640 1 --2- 172.21.15.152:0/2614227638 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff7800d0730 0x7ff780162ac0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ff780071d60 tx=0x7ff77802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4972d30afd327e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:55.789 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.788+0000 7ff77fe00640 1 -- 172.21.15.152:0/2614227638 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff778035020 con 0x7ff7800d0730 2024-04-20T11:27:55.789 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.788+0000 7ff77fe00640 1 -- 172.21.15.152:0/2614227638 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff778031830 con 0x7ff7800d0730 2024-04-20T11:27:55.789 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.788+0000 7ff77fe00640 1 -- 172.21.15.152:0/2614227638 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff778031c00 con 0x7ff7800d0730 2024-04-20T11:27:55.790 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.789+0000 7ff788600640 1 -- 172.21.15.152:0/2614227638 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff7800d0730 msgr2=0x7ff780162ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:55.790 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.789+0000 7ff788600640 1 --2- 172.21.15.152:0/2614227638 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff7800d0730 0x7ff780162ac0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ff780071d60 tx=0x7ff77802b8d0 comp rx=0 tx=0).stop 2024-04-20T11:27:55.790 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.789+0000 7ff788600640 1 -- 172.21.15.152:0/2614227638 shutdown_connections 2024-04-20T11:27:55.790 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.789+0000 7ff788600640 1 --2- 172.21.15.152:0/2614227638 >> v2:172.21.15.175:3300/0 conn(0x7ff78006e7e0 0x7ff780162580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:55.790 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.789+0000 7ff788600640 1 --2- 172.21.15.152:0/2614227638 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff7800d0730 0x7ff780162ac0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:55.790 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.789+0000 7ff788600640 1 -- 172.21.15.152:0/2614227638 >> 172.21.15.152:0/2614227638 conn(0x7ff7800b4090 msgr2=0x7ff7800b44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:55.790 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.789+0000 7ff788600640 1 -- 172.21.15.152:0/2614227638 shutdown_connections 2024-04-20T11:27:55.790 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.789+0000 7ff788600640 1 -- 172.21.15.152:0/2614227638 wait complete. 2024-04-20T11:27:55.791 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.791+0000 7ff788600640 1 Processor -- start 2024-04-20T11:27:55.791 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.791+0000 7ff788600640 1 -- start start 2024-04-20T11:27:55.792 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.792+0000 7ff788600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff78006e400 0x7ff780155fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:55.792 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.792+0000 7ff788600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff78006e7e0 0x7ff7800c8980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:55.793 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.792+0000 7ff788600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff7800d0730 0x7ff7800c6e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:55.793 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.792+0000 7ff788600640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7ff78015e210 con 0x7ff7800d0730 2024-04-20T11:27:55.793 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.792+0000 7ff788600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff7800d1500 con 0x7ff78006e7e0 2024-04-20T11:27:55.793 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.792+0000 7ff788600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff78015e390 con 0x7ff78006e400 2024-04-20T11:27:55.793 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.792+0000 7ff786600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff7800d0730 0x7ff7800c6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:55.793 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.792+0000 7ff786600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff7800d0730 0x7ff7800c6e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43520/0 (socket says 172.21.15.152:43520) 2024-04-20T11:27:55.793 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.792+0000 7ff786600640 1 -- 172.21.15.152:0/313741927 learned_addr learned my addr 172.21.15.152:0/313741927 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:55.794 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.792+0000 7ff786600640 1 -- 172.21.15.152:0/313741927 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff78006e400 msgr2=0x7ff780155fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:55.794 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.792+0000 7ff786600640 1 --2- 172.21.15.152:0/313741927 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff78006e400 0x7ff780155fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:55.794 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.792+0000 7ff786600640 1 -- 172.21.15.152:0/313741927 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff78006e7e0 msgr2=0x7ff7800c8980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:55.794 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.792+0000 7ff785200640 1 --2- 172.21.15.152:0/313741927 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff78006e7e0 0x7ff7800c8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:55.795 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.792+0000 7ff786600640 1 --2- 172.21.15.152:0/313741927 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff78006e7e0 0x7ff7800c8980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:55.795 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.792+0000 7ff786600640 1 -- 172.21.15.152:0/313741927 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7800c78e0 con 0x7ff7800d0730 2024-04-20T11:27:55.795 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.793+0000 7ff786600640 1 --2- 172.21.15.152:0/313741927 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff7800d0730 0x7ff7800c6e20 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ff778034040 tx=0x7ff7780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:55.795 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.793+0000 7ff77d600640 1 -- 172.21.15.152:0/313741927 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff778040070 con 0x7ff7800d0730 2024-04-20T11:27:55.795 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.793+0000 7ff77d600640 1 -- 172.21.15.152:0/313741927 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff77803a850 con 0x7ff7800d0730 2024-04-20T11:27:55.795 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.793+0000 7ff77d600640 1 -- 172.21.15.152:0/313741927 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff778035040 con 0x7ff7800d0730 2024-04-20T11:27:55.795 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.793+0000 7ff788600640 1 -- 172.21.15.152:0/313741927 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7800c7bc0 con 0x7ff7800d0730 2024-04-20T11:27:55.795 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.793+0000 7ff788600640 1 -- 172.21.15.152:0/313741927 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7800c8180 con 0x7ff7800d0730 2024-04-20T11:27:55.795 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.794+0000 7ff788600640 1 -- 172.21.15.152:0/313741927 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7800689f0 con 0x7ff7800d0730 2024-04-20T11:27:55.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.795+0000 7ff77d600640 1 -- 172.21.15.152:0/313741927 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff778038020 con 0x7ff7800d0730 2024-04-20T11:27:55.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.796+0000 7ff77d600640 1 --2- 172.21.15.152:0/313741927 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff76006c8f0 0x7ff76006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:55.797 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.796+0000 7ff77d600640 1 -- 172.21.15.152:0/313741927 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7ff7780ad4e0 con 0x7ff7800d0730 2024-04-20T11:27:55.797 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.797+0000 7ff785c00640 1 --2- 172.21.15.152:0/313741927 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff76006c8f0 0x7ff76006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:55.798 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.797+0000 7ff785c00640 1 --2- 172.21.15.152:0/313741927 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff76006c8f0 0x7ff76006edb0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff770000f60 tx=0x7ff7700078f0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:55.801 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:55.801+0000 7ff77d600640 1 -- 172.21.15.152:0/313741927 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ff778031800 con 0x7ff7800d0730 2024-04-20T11:27:56.015 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.014+0000 7ff788600640 1 -- 172.21.15.152:0/313741927 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ff7800b8520 con 0x7ff7800d0730 2024-04-20T11:27:56.016 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.015+0000 7ff77d600640 1 -- 172.21.15.152:0/313741927 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) ==== 80+0+0 (secure 0 0 0) 0x7ff77807b880 con 0x7ff7800d0730 2024-04-20T11:27:56.016 INFO:teuthology.orchestra.run.smithi152.stderr:listed 0 entries 2024-04-20T11:27:56.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.018+0000 7ff788600640 1 -- 172.21.15.152:0/313741927 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff76006c8f0 msgr2=0x7ff76006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:56.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.018+0000 7ff788600640 1 --2- 172.21.15.152:0/313741927 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff76006c8f0 0x7ff76006edb0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff770000f60 tx=0x7ff7700078f0 comp rx=0 tx=0).stop 2024-04-20T11:27:56.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.018+0000 7ff788600640 1 -- 172.21.15.152:0/313741927 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff7800d0730 msgr2=0x7ff7800c6e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:56.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.018+0000 7ff788600640 1 --2- 172.21.15.152:0/313741927 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff7800d0730 0x7ff7800c6e20 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ff778034040 tx=0x7ff7780046c0 comp rx=0 tx=0).stop 2024-04-20T11:27:56.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.018+0000 7ff788600640 1 -- 172.21.15.152:0/313741927 shutdown_connections 2024-04-20T11:27:56.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.018+0000 7ff788600640 1 --2- 172.21.15.152:0/313741927 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff76006c8f0 0x7ff76006edb0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:56.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.018+0000 7ff788600640 1 --2- 172.21.15.152:0/313741927 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff7800d0730 0x7ff7800c6e20 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:56.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.018+0000 7ff788600640 1 --2- 172.21.15.152:0/313741927 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff78006e7e0 0x7ff7800c8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:56.021 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.018+0000 7ff788600640 1 --2- 172.21.15.152:0/313741927 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff78006e400 0x7ff780155fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:56.021 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.018+0000 7ff788600640 1 -- 172.21.15.152:0/313741927 >> 172.21.15.152:0/313741927 conn(0x7ff7800b4090 msgr2=0x7ff7800b4470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:56.021 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.018+0000 7ff788600640 1 -- 172.21.15.152:0/313741927 shutdown_connections 2024-04-20T11:27:56.021 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.018+0000 7ff788600640 1 -- 172.21.15.152:0/313741927 wait complete. 2024-04-20T11:27:56.041 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi152.front.sepia.ceph.com... 2024-04-20T11:27:56.041 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:27:56.042 DEBUG:teuthology.orchestra.run.smithi152:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2024-04-20T11:27:56.129 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T11:27:56.130 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi152.front.sepia.ceph.com 2024-04-20T11:27:56.130 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:27:56.130 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-20T11:27:56.181 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T11:27:56.181 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:27:56.237 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T11:27:56.238 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:27:56.289 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:27:56.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.499+0000 7f230b000640 1 Processor -- start 2024-04-20T11:27:56.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.499+0000 7f230b000640 1 -- start start 2024-04-20T11:27:56.502 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.501+0000 7f230b000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2304160b90 0x7f230415d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:56.502 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.501+0000 7f230b000640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f2304071cd0 0x7f2304072160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:56.502 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.501+0000 7f230b000640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23040ce8c0 con 0x7f230415d1e0 2024-04-20T11:27:56.502 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.501+0000 7f230b000640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f23040726a0 con 0x7f2304071cd0 2024-04-20T11:27:56.502 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.501+0000 7f230b000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f23041600b0 con 0x7f2304160b90 2024-04-20T11:27:56.503 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.501+0000 7f2309200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2304160b90 0x7f230415d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:56.503 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.501+0000 7f230a600640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f2304071cd0 0x7f2304072160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:56.503 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.502+0000 7f230a600640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f2304071cd0 0x7f2304072160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.152:44492/0 (socket says 172.21.15.152:44492) 2024-04-20T11:27:56.503 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.502+0000 7f2309c00640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7f230415d1e0 0x7f23040cdfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.152:38326/0 (socket says 172.21.15.152:38326) 2024-04-20T11:27:56.503 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.502+0000 7f230a600640 1 -- 172.21.15.152:0/1799986166 learned_addr learned my addr 172.21.15.152:0/1799986166 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:56.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.502+0000 7f2309200640 1 -- 172.21.15.152:0/1799986166 >> v2:172.21.15.175:3300/0 conn(0x7f2304071cd0 msgr2=0x7f2304072160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:56.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.502+0000 7f2309200640 1 --2- 172.21.15.152:0/1799986166 >> v2:172.21.15.175:3300/0 conn(0x7f2304071cd0 0x7f2304072160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:56.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.502+0000 7f2309200640 1 -- 172.21.15.152:0/1799986166 >> v1:172.21.15.175:6789/0 conn(0x7f230415d1e0 legacy=0x7f23040cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:56.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.502+0000 7f2309200640 1 -- 172.21.15.152:0/1799986166 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2304160390 con 0x7f2304160b90 2024-04-20T11:27:56.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.502+0000 7f230a600640 1 --2- 172.21.15.152:0/1799986166 >> v2:172.21.15.175:3300/0 conn(0x7f2304071cd0 0x7f2304072160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:27:56.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.504+0000 7f2309200640 1 --2- 172.21.15.152:0/1799986166 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2304160b90 0x7f230415d5c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f22f00097a0 tx=0x7f22f002b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ccc241471791960b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:56.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.504+0000 7f2303e00640 1 -- 172.21.15.152:0/1799986166 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f0035020 con 0x7f2304160b90 2024-04-20T11:27:56.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.505+0000 7f2303e00640 1 -- 172.21.15.152:0/1799986166 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22f0031810 con 0x7f2304160b90 2024-04-20T11:27:56.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.505+0000 7f2303e00640 1 -- 172.21.15.152:0/1799986166 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f0031be0 con 0x7f2304160b90 2024-04-20T11:27:56.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.505+0000 7f230b000640 1 -- 172.21.15.152:0/1799986166 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2304160b90 msgr2=0x7f230415d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:56.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.505+0000 7f230b000640 1 --2- 172.21.15.152:0/1799986166 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2304160b90 0x7f230415d5c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f22f00097a0 tx=0x7f22f002b8b0 comp rx=0 tx=0).stop 2024-04-20T11:27:56.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.505+0000 7f230b000640 1 -- 172.21.15.152:0/1799986166 shutdown_connections 2024-04-20T11:27:56.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.505+0000 7f230b000640 1 --2- 172.21.15.152:0/1799986166 >> v2:172.21.15.175:3300/0 conn(0x7f2304071cd0 0x7f2304072160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:56.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.505+0000 7f230b000640 1 --2- 172.21.15.152:0/1799986166 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2304160b90 0x7f230415d5c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:56.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.505+0000 7f230b000640 1 -- 172.21.15.152:0/1799986166 >> 172.21.15.152:0/1799986166 conn(0x7f2304067ae0 msgr2=0x7f230406b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:56.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.505+0000 7f230b000640 1 -- 172.21.15.152:0/1799986166 shutdown_connections 2024-04-20T11:27:56.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.506+0000 7f230b000640 1 -- 172.21.15.152:0/1799986166 wait complete. 2024-04-20T11:27:56.508 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.508+0000 7f230b000640 1 Processor -- start 2024-04-20T11:27:56.508 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.508+0000 7f230b000640 1 -- start start 2024-04-20T11:27:56.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.509+0000 7f230b000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2304071cd0 0x7f2304147ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:56.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.509+0000 7f230b000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f230415d1e0 0x7f2304146770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:56.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.509+0000 7f230b000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2304160b90 0x7f2304144ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:56.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.509+0000 7f230b000640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f23040ccd70 con 0x7f230415d1e0 2024-04-20T11:27:56.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.509+0000 7f230b000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f23040ceba0 con 0x7f2304071cd0 2024-04-20T11:27:56.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.509+0000 7f230b000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f23040ccef0 con 0x7f2304160b90 2024-04-20T11:27:56.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.509+0000 7f2309200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f230415d1e0 0x7f2304146770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:56.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.509+0000 7f2309200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f230415d1e0 0x7f2304146770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:32830/0 (socket says 172.21.15.152:32830) 2024-04-20T11:27:56.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.509+0000 7f2309200640 1 -- 172.21.15.152:0/3522338324 learned_addr learned my addr 172.21.15.152:0/3522338324 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:56.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.510+0000 7f230a600640 1 --2- 172.21.15.152:0/3522338324 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2304160b90 0x7f2304144ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:56.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.510+0000 7f2309c00640 1 --2- 172.21.15.152:0/3522338324 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2304071cd0 0x7f2304147ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:56.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.510+0000 7f2309200640 1 -- 172.21.15.152:0/3522338324 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2304160b90 msgr2=0x7f2304144ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:56.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.510+0000 7f2309200640 1 --2- 172.21.15.152:0/3522338324 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2304160b90 0x7f2304144ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:56.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.510+0000 7f2309200640 1 -- 172.21.15.152:0/3522338324 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2304071cd0 msgr2=0x7f2304147ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:56.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.510+0000 7f2309200640 1 --2- 172.21.15.152:0/3522338324 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2304071cd0 0x7f2304147ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:56.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.510+0000 7f2309200640 1 -- 172.21.15.152:0/3522338324 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2304146cb0 con 0x7f230415d1e0 2024-04-20T11:27:56.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.510+0000 7f2309200640 1 --2- 172.21.15.152:0/3522338324 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f230415d1e0 0x7f2304146770 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f22f00098c0 tx=0x7f22f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:56.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.511+0000 7f2301600640 1 -- 172.21.15.152:0/3522338324 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f0048070 con 0x7f230415d1e0 2024-04-20T11:27:56.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.511+0000 7f2301600640 1 -- 172.21.15.152:0/3522338324 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22f003bd70 con 0x7f230415d1e0 2024-04-20T11:27:56.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.511+0000 7f2301600640 1 -- 172.21.15.152:0/3522338324 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f0043630 con 0x7f230415d1e0 2024-04-20T11:27:56.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.511+0000 7f230b000640 1 -- 172.21.15.152:0/3522338324 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2304145420 con 0x7f230415d1e0 2024-04-20T11:27:56.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.511+0000 7f230b000640 1 -- 172.21.15.152:0/3522338324 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23041459b0 con 0x7f230415d1e0 2024-04-20T11:27:56.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.512+0000 7f230b000640 1 -- 172.21.15.152:0/3522338324 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f230406c360 con 0x7f230415d1e0 2024-04-20T11:27:56.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.513+0000 7f2301600640 1 -- 172.21.15.152:0/3522338324 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f22f0038020 con 0x7f230415d1e0 2024-04-20T11:27:56.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.514+0000 7f2301600640 1 --2- 172.21.15.152:0/3522338324 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f22e406c8f0 0x7f22e406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:56.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.514+0000 7f2301600640 1 -- 172.21.15.152:0/3522338324 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f22f00ad650 con 0x7f230415d1e0 2024-04-20T11:27:56.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.514+0000 7f2309c00640 1 --2- 172.21.15.152:0/3522338324 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f22e406c8f0 0x7f22e406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:56.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.515+0000 7f2309c00640 1 --2- 172.21.15.152:0/3522338324 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f22e406c8f0 0x7f22e406edb0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f22f8000f60 tx=0x7f22f8007cc0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:56.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.518+0000 7f2301600640 1 -- 172.21.15.152:0/3522338324 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f230406c360 con 0x7f230415d1e0 2024-04-20T11:27:56.725 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.723+0000 7f230b000640 1 -- 172.21.15.152:0/3522338324 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f230415a290 con 0x7f230415d1e0 2024-04-20T11:27:56.726 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.726+0000 7f2301600640 1 -- 172.21.15.152:0/3522338324 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16914 (secure 0 0 0) 0x7f22f0040070 con 0x7f230415d1e0 2024-04-20T11:27:56.727 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:27:56.727 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":23,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:27:42.601620+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:26:56.261424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T11:26:57.079286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":2}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.26","mappings":[{"from":7,"to":1}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:27:56.730 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.729+0000 7f230b000640 1 -- 172.21.15.152:0/3522338324 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f22e406c8f0 msgr2=0x7f22e406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:56.730 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.729+0000 7f230b000640 1 --2- 172.21.15.152:0/3522338324 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f22e406c8f0 0x7f22e406edb0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f22f8000f60 tx=0x7f22f8007cc0 comp rx=0 tx=0).stop 2024-04-20T11:27:56.730 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.729+0000 7f230b000640 1 -- 172.21.15.152:0/3522338324 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f230415d1e0 msgr2=0x7f2304146770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:56.730 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.729+0000 7f230b000640 1 --2- 172.21.15.152:0/3522338324 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f230415d1e0 0x7f2304146770 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f22f00098c0 tx=0x7f22f00046c0 comp rx=0 tx=0).stop 2024-04-20T11:27:56.730 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.729+0000 7f230b000640 1 -- 172.21.15.152:0/3522338324 shutdown_connections 2024-04-20T11:27:56.730 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.729+0000 7f230b000640 1 --2- 172.21.15.152:0/3522338324 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f22e406c8f0 0x7f22e406edb0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:56.730 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.729+0000 7f230b000640 1 --2- 172.21.15.152:0/3522338324 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2304160b90 0x7f2304144ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:56.730 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.729+0000 7f230b000640 1 --2- 172.21.15.152:0/3522338324 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f230415d1e0 0x7f2304146770 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:56.730 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.729+0000 7f230b000640 1 --2- 172.21.15.152:0/3522338324 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2304071cd0 0x7f2304147ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:56.730 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.729+0000 7f230b000640 1 -- 172.21.15.152:0/3522338324 >> 172.21.15.152:0/3522338324 conn(0x7f2304067ae0 msgr2=0x7f230415a5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:56.730 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.729+0000 7f230b000640 1 -- 172.21.15.152:0/3522338324 shutdown_connections 2024-04-20T11:27:56.730 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.729+0000 7f230b000640 1 -- 172.21.15.152:0/3522338324 wait complete. 2024-04-20T11:27:56.749 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T11:26:47.246571+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T11:26:56.261424+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T11:26:57.079286+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T11:27:56.749 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T11:27:56.956 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.956+0000 7f720da00640 1 Processor -- start 2024-04-20T11:27:56.956 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.956+0000 7f720da00640 1 -- start start 2024-04-20T11:27:56.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.957+0000 7f720da00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7208152450 0x7f720815c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:56.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.957+0000 7f720da00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7208152830 con 0x7f72080b3520 2024-04-20T11:27:56.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.957+0000 7f720da00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72080b39c0 con 0x7f72080b30c0 2024-04-20T11:27:56.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.957+0000 7f720da00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f72080b3ca0 con 0x7f7208152450 2024-04-20T11:27:56.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.957+0000 7f720d000640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f72080b3520 0x7f72080ae470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:36062/0 (socket says 172.21.15.152:36062) 2024-04-20T11:27:56.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.957+0000 7f720d000640 1 -- 172.21.15.152:0/2393761500 learned_addr learned my addr 172.21.15.152:0/2393761500 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:27:56.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.958+0000 7f7207e00640 1 --2- 172.21.15.152:0/2393761500 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7208152450 0x7f720815c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:56.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.958+0000 7f7207400640 1 -- 172.21.15.152:0/2393761500 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2467212133 0 0) 0x7f7208152830 con 0x7f72080b3520 2024-04-20T11:27:56.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.958+0000 7f7207400640 1 -- 172.21.15.152:0/2393761500 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f71ec003520 con 0x7f72080b3520 2024-04-20T11:27:56.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.958+0000 7f7207400640 1 -- 172.21.15.152:0/2393761500 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2016190451 0 0) 0x7f71ec003520 con 0x7f72080b3520 2024-04-20T11:27:56.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.958+0000 7f7207400640 1 -- 172.21.15.152:0/2393761500 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7208152450 msgr2=0x7f720815c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:56.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.958+0000 7f7207400640 1 --2- 172.21.15.152:0/2393761500 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7208152450 0x7f720815c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:56.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.958+0000 7f7207400640 1 -- 172.21.15.152:0/2393761500 >> v1:172.21.15.175:6789/0 conn(0x7f72080b30c0 legacy=0x7f720806bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:56.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.958+0000 7f7207400640 1 -- 172.21.15.152:0/2393761500 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72080b3f80 con 0x7f72080b3520 2024-04-20T11:27:56.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.958+0000 7f7207400640 1 -- 172.21.15.152:0/2393761500 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f72000019f0 con 0x7f72080b3520 2024-04-20T11:27:56.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.958+0000 7f7207400640 1 -- 172.21.15.152:0/2393761500 >> v1:172.21.15.152:6789/0 conn(0x7f72080b3520 legacy=0x7f72080ae470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:56.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.958+0000 7f7207400640 1 --2- 172.21.15.152:0/2393761500 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71ec0039e0 0x7f71ec003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:56.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.958+0000 7f7207400640 1 --2- 172.21.15.152:0/2393761500 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f71ec004800 0x7f71ec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:56.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.958+0000 7f7207400640 1 --2- 172.21.15.152:0/2393761500 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7208152450 0x7f71ec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:56.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.959+0000 7f7207400640 1 -- 172.21.15.152:0/2393761500 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f72080b3ca0 con 0x7f71ec004800 2024-04-20T11:27:56.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.959+0000 7f7207400640 1 -- 172.21.15.152:0/2393761500 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f71ec00ba70 con 0x7f71ec0039e0 2024-04-20T11:27:56.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.959+0000 7f7207400640 1 -- 172.21.15.152:0/2393761500 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f71ec00bc70 con 0x7f7208152450 2024-04-20T11:27:56.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.959+0000 7f71ff400640 1 --2- 172.21.15.152:0/2393761500 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f71ec004800 0x7f71ec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:56.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.959+0000 7f71ff400640 1 -- 172.21.15.152:0/2393761500 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7208152450 msgr2=0x7f71ec0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:56.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.959+0000 7f71ff400640 1 --2- 172.21.15.152:0/2393761500 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7208152450 0x7f71ec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:56.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.959+0000 7f71ff400640 1 -- 172.21.15.152:0/2393761500 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71ec0039e0 msgr2=0x7f71ec003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:56.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.959+0000 7f7207e00640 1 --2- 172.21.15.152:0/2393761500 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71ec0039e0 0x7f71ec003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:56.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.959+0000 7f71ff400640 1 --2- 172.21.15.152:0/2393761500 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71ec0039e0 0x7f71ec003e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:56.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.959+0000 7f71ff400640 1 -- 172.21.15.152:0/2393761500 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71ec003520 con 0x7f71ec004800 2024-04-20T11:27:56.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.960+0000 7f71ff400640 1 --2- 172.21.15.152:0/2393761500 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f71ec004800 0x7f71ec004c80 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f71f0001720 tx=0x7f71ec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ad317874610876e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:56.963 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.961+0000 7f7207400640 1 -- 172.21.15.152:0/2393761500 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f0001370 con 0x7f71ec004800 2024-04-20T11:27:56.963 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.961+0000 7f7207400640 1 -- 172.21.15.152:0/2393761500 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71f00052d0 con 0x7f71ec004800 2024-04-20T11:27:56.963 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.961+0000 7f7207400640 1 -- 172.21.15.152:0/2393761500 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f0003830 con 0x7f71ec004800 2024-04-20T11:27:56.963 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.961+0000 7f720da00640 1 -- 172.21.15.152:0/2393761500 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f71ec004800 msgr2=0x7f71ec004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:56.963 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.961+0000 7f720da00640 1 --2- 172.21.15.152:0/2393761500 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f71ec004800 0x7f71ec004c80 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f71f0001720 tx=0x7f71ec005830 comp rx=0 tx=0).stop 2024-04-20T11:27:56.963 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.961+0000 7f720d000640 1 -- 172.21.15.152:0/2393761500 reap_dead start 2024-04-20T11:27:56.963 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.961+0000 7f720da00640 1 -- 172.21.15.152:0/2393761500 shutdown_connections 2024-04-20T11:27:56.963 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.961+0000 7f720da00640 1 -- 172.21.15.152:0/2393761500 >> 172.21.15.152:0/2393761500 conn(0x7f72080b21e0 msgr2=0x7f72080b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:56.963 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.961+0000 7f720da00640 1 -- 172.21.15.152:0/2393761500 shutdown_connections 2024-04-20T11:27:56.963 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.961+0000 7f720da00640 1 -- 172.21.15.152:0/2393761500 wait complete. 2024-04-20T11:27:56.964 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.963+0000 7f720da00640 1 Processor -- start 2024-04-20T11:27:56.964 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.963+0000 7f720da00640 1 -- start start 2024-04-20T11:27:56.965 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.964+0000 7f720da00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f71ec004800 0x7f72081f8270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:56.965 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.964+0000 7f720da00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f72081f4490 0x7f72080c9440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:56.965 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.964+0000 7f720da00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72081ee040 0x7f72081ee4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:56.966 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.964+0000 7f720da00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f720806c4b0 con 0x7f71ec004800 2024-04-20T11:27:56.966 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.964+0000 7f720da00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f72080adf00 con 0x7f72081ee040 2024-04-20T11:27:56.966 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.964+0000 7f720da00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f720806c630 con 0x7f72081f4490 2024-04-20T11:27:56.966 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.964+0000 7f7207e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f71ec004800 0x7f72081f8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:56.966 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.964+0000 7f7207e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f71ec004800 0x7f72081f8270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:32854/0 (socket says 172.21.15.152:32854) 2024-04-20T11:27:56.966 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.964+0000 7f7207e00640 1 -- 172.21.15.152:0/4196946235 learned_addr learned my addr 172.21.15.152:0/4196946235 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:56.966 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.965+0000 7f720d000640 1 --2- 172.21.15.152:0/4196946235 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72081ee040 0x7f72081ee4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:56.966 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.965+0000 7f71ff400640 1 --2- 172.21.15.152:0/4196946235 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f72081f4490 0x7f72080c9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:56.966 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.965+0000 7f7207e00640 1 -- 172.21.15.152:0/4196946235 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f72081f4490 msgr2=0x7f72080c9440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:56.966 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.965+0000 7f7207e00640 1 --2- 172.21.15.152:0/4196946235 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f72081f4490 0x7f72080c9440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:56.966 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.965+0000 7f7207e00640 1 -- 172.21.15.152:0/4196946235 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72081ee040 msgr2=0x7f72081ee4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:56.966 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.965+0000 7f7207e00640 1 --2- 172.21.15.152:0/4196946235 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72081ee040 0x7f72081ee4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:56.966 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.965+0000 7f7207e00640 1 -- 172.21.15.152:0/4196946235 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72081f4970 con 0x7f71ec004800 2024-04-20T11:27:56.967 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.965+0000 7f71ff400640 1 --2- 172.21.15.152:0/4196946235 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f72081f4490 0x7f72080c9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:27:56.967 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.965+0000 7f720d000640 1 --2- 172.21.15.152:0/4196946235 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72081ee040 0x7f72081ee4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:27:56.967 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.965+0000 7f7207e00640 1 --2- 172.21.15.152:0/4196946235 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f71ec004800 0x7f72081f8270 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f71f800fd50 tx=0x7f71f800d720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:56.967 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.965+0000 7f7204c00640 1 -- 172.21.15.152:0/4196946235 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f800de30 con 0x7f71ec004800 2024-04-20T11:27:56.968 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.965+0000 7f7204c00640 1 -- 172.21.15.152:0/4196946235 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71f8011660 con 0x7f71ec004800 2024-04-20T11:27:56.968 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.965+0000 7f7204c00640 1 -- 172.21.15.152:0/4196946235 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f8011a30 con 0x7f71ec004800 2024-04-20T11:27:56.968 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.966+0000 7f720da00640 1 -- 172.21.15.152:0/4196946235 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72081f4c50 con 0x7f71ec004800 2024-04-20T11:27:56.968 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.966+0000 7f720da00640 1 -- 172.21.15.152:0/4196946235 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72081f5210 con 0x7f71ec004800 2024-04-20T11:27:56.968 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.967+0000 7f720da00640 1 -- 172.21.15.152:0/4196946235 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7208075e60 con 0x7f71ec004800 2024-04-20T11:27:56.968 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.968+0000 7f7204c00640 1 -- 172.21.15.152:0/4196946235 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f71f8011bd0 con 0x7f71ec004800 2024-04-20T11:27:56.969 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.968+0000 7f7204c00640 1 --2- 172.21.15.152:0/4196946235 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f71e406c8f0 0x7f71e406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:56.969 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.969+0000 7f7204c00640 1 -- 172.21.15.152:0/4196946235 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f71f808ca90 con 0x7f71ec004800 2024-04-20T11:27:56.969 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.969+0000 7f71ff400640 1 --2- 172.21.15.152:0/4196946235 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f71e406c8f0 0x7f71e406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:56.970 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.970+0000 7f71ff400640 1 --2- 172.21.15.152:0/4196946235 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f71e406c8f0 0x7f71e406edb0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f71f0033000 tx=0x7f71f0009030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:56.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:56.973+0000 7f7204c00640 1 -- 172.21.15.152:0/4196946235 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f71f805ada0 con 0x7f71ec004800 2024-04-20T11:27:57.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.174+0000 7f720da00640 1 -- 172.21.15.152:0/4196946235 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f72080734d0 con 0x7f71ec004800 2024-04-20T11:27:57.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.175+0000 7f7204c00640 1 -- 172.21.15.152:0/4196946235 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) ==== 93+0+10 (secure 0 0 0) 0x7f71f805a6a0 con 0x7f71ec004800 2024-04-20T11:27:57.175 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 1 2024-04-20T11:27:57.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.177+0000 7f720da00640 1 -- 172.21.15.152:0/4196946235 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f71e406c8f0 msgr2=0x7f71e406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:57.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.177+0000 7f720da00640 1 --2- 172.21.15.152:0/4196946235 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f71e406c8f0 0x7f71e406edb0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f71f0033000 tx=0x7f71f0009030 comp rx=0 tx=0).stop 2024-04-20T11:27:57.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.177+0000 7f720da00640 1 -- 172.21.15.152:0/4196946235 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f71ec004800 msgr2=0x7f72081f8270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:57.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.177+0000 7f720da00640 1 --2- 172.21.15.152:0/4196946235 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f71ec004800 0x7f72081f8270 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f71f800fd50 tx=0x7f71f800d720 comp rx=0 tx=0).stop 2024-04-20T11:27:57.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.178+0000 7f720da00640 1 -- 172.21.15.152:0/4196946235 shutdown_connections 2024-04-20T11:27:57.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.178+0000 7f720da00640 1 --2- 172.21.15.152:0/4196946235 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f71e406c8f0 0x7f71e406edb0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:57.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.178+0000 7f720da00640 1 --2- 172.21.15.152:0/4196946235 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72081ee040 0x7f72081ee4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:57.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.178+0000 7f720da00640 1 --2- 172.21.15.152:0/4196946235 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f72081f4490 0x7f72080c9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:57.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.178+0000 7f720da00640 1 --2- 172.21.15.152:0/4196946235 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f71ec004800 0x7f72081f8270 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:57.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.178+0000 7f720da00640 1 -- 172.21.15.152:0/4196946235 >> 172.21.15.152:0/4196946235 conn(0x7f72080b21e0 msgr2=0x7f72080b05c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:57.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.178+0000 7f720da00640 1 -- 172.21.15.152:0/4196946235 shutdown_connections 2024-04-20T11:27:57.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.178+0000 7f720da00640 1 -- 172.21.15.152:0/4196946235 wait complete. 2024-04-20T11:27:57.198 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T11:27:57.401 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.400+0000 7f84fa200640 1 Processor -- start 2024-04-20T11:27:57.401 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.400+0000 7f84fa200640 1 -- start start 2024-04-20T11:27:57.403 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.401+0000 7f84fa200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f84f415e9e0 0x7f84f406d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:57.403 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.401+0000 7f84fa200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f84f40d14a0 0x7f84f40d4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:57.403 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.402+0000 7f84fa200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84f40d19a0 con 0x7f84f40d46a0 2024-04-20T11:27:57.403 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.402+0000 7f84fa200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f84f40d5040 con 0x7f84f415e9e0 2024-04-20T11:27:57.403 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.402+0000 7f84fa200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f84f40d1c80 con 0x7f84f40d14a0 2024-04-20T11:27:57.403 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.402+0000 7f84f3400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f84f415e9e0 0x7f84f406d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:57.403 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.402+0000 7f84f3400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f84f415e9e0 0x7f84f406d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:32864/0 (socket says 172.21.15.152:32864) 2024-04-20T11:27:57.403 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.402+0000 7f84f3400640 1 -- 172.21.15.152:0/900834605 learned_addr learned my addr 172.21.15.152:0/900834605 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:57.404 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.402+0000 7f84f3400640 1 -- 172.21.15.152:0/900834605 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f84f40d14a0 msgr2=0x7f84f40d4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:57.404 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.402+0000 7f84f3400640 1 --2- 172.21.15.152:0/900834605 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f84f40d14a0 0x7f84f40d4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:57.404 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.402+0000 7f84f3400640 1 -- 172.21.15.152:0/900834605 >> v1:172.21.15.175:6789/0 conn(0x7f84f40d46a0 legacy=0x7f84f40d0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:57.404 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.402+0000 7f84f3400640 1 -- 172.21.15.152:0/900834605 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84f40d1f60 con 0x7f84f415e9e0 2024-04-20T11:27:57.404 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.403+0000 7f84f3400640 1 --2- 172.21.15.152:0/900834605 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f84f415e9e0 0x7f84f406d320 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f84e4009870 tx=0x7f84e402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=845e49f8031d9d02 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:57.404 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.404+0000 7f84f2000640 1 -- 172.21.15.152:0/900834605 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e4035020 con 0x7f84f415e9e0 2024-04-20T11:27:57.404 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.404+0000 7f84f2000640 1 -- 172.21.15.152:0/900834605 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84e4031640 con 0x7f84f415e9e0 2024-04-20T11:27:57.405 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.404+0000 7f84f2000640 1 -- 172.21.15.152:0/900834605 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e402d040 con 0x7f84f415e9e0 2024-04-20T11:27:57.405 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.404+0000 7f84fa200640 1 -- 172.21.15.152:0/900834605 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f84f415e9e0 msgr2=0x7f84f406d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:57.405 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.404+0000 7f84fa200640 1 --2- 172.21.15.152:0/900834605 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f84f415e9e0 0x7f84f406d320 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f84e4009870 tx=0x7f84e402b8f0 comp rx=0 tx=0).stop 2024-04-20T11:27:57.405 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.405+0000 7f84fa200640 1 -- 172.21.15.152:0/900834605 shutdown_connections 2024-04-20T11:27:57.406 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.405+0000 7f84fa200640 1 --2- 172.21.15.152:0/900834605 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f84f40d14a0 0x7f84f40d4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:57.406 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.405+0000 7f84fa200640 1 --2- 172.21.15.152:0/900834605 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f84f415e9e0 0x7f84f406d320 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:57.406 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.405+0000 7f84fa200640 1 -- 172.21.15.152:0/900834605 >> 172.21.15.152:0/900834605 conn(0x7f84f409dab0 msgr2=0x7f84f408bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:57.406 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.405+0000 7f84fa200640 1 -- 172.21.15.152:0/900834605 shutdown_connections 2024-04-20T11:27:57.406 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.405+0000 7f84fa200640 1 -- 172.21.15.152:0/900834605 wait complete. 2024-04-20T11:27:57.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.406+0000 7f84fa200640 1 Processor -- start 2024-04-20T11:27:57.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.407+0000 7f84fa200640 1 -- start start 2024-04-20T11:27:57.409 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.408+0000 7f84fa200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f84f40d14a0 0x7f84f41522d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:57.409 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.408+0000 7f84fa200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f84f40d46a0 0x7f84f41613d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:57.409 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.408+0000 7f84fa200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f84f415e9e0 0x7f84f40b9530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:57.409 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.408+0000 7f84fa200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f84f406d860 con 0x7f84f40d46a0 2024-04-20T11:27:57.409 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.408+0000 7f84fa200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f84f40cf880 con 0x7f84f415e9e0 2024-04-20T11:27:57.409 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.408+0000 7f84fa200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f84f406d9e0 con 0x7f84f40d14a0 2024-04-20T11:27:57.409 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.408+0000 7f84f2a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f84f40d46a0 0x7f84f41613d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:57.409 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.408+0000 7f84f2a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f84f40d46a0 0x7f84f41613d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:32876/0 (socket says 172.21.15.152:32876) 2024-04-20T11:27:57.410 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.408+0000 7f84f2a00640 1 -- 172.21.15.152:0/2480370652 learned_addr learned my addr 172.21.15.152:0/2480370652 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:57.410 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.408+0000 7f84f3e00640 1 --2- 172.21.15.152:0/2480370652 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f84f415e9e0 0x7f84f40b9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:57.410 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.408+0000 7f84f3400640 1 --2- 172.21.15.152:0/2480370652 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f84f40d14a0 0x7f84f41522d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:57.410 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.408+0000 7f84f2a00640 1 -- 172.21.15.152:0/2480370652 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f84f40d14a0 msgr2=0x7f84f41522d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:57.410 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.408+0000 7f84f2a00640 1 --2- 172.21.15.152:0/2480370652 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f84f40d14a0 0x7f84f41522d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:57.410 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.408+0000 7f84f2a00640 1 -- 172.21.15.152:0/2480370652 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f84f415e9e0 msgr2=0x7f84f40b9530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:57.410 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.408+0000 7f84f2a00640 1 --2- 172.21.15.152:0/2480370652 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f84f415e9e0 0x7f84f40b9530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:57.410 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.409+0000 7f84f2a00640 1 -- 172.21.15.152:0/2480370652 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84f4161d20 con 0x7f84f40d46a0 2024-04-20T11:27:57.410 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.409+0000 7f84f2a00640 1 --2- 172.21.15.152:0/2480370652 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f84f40d46a0 0x7f84f41613d0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f84e000c410 tx=0x7f84e000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:57.411 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.409+0000 7f84eb400640 1 -- 172.21.15.152:0/2480370652 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e0017070 con 0x7f84f40d46a0 2024-04-20T11:27:57.411 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.409+0000 7f84eb400640 1 -- 172.21.15.152:0/2480370652 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84e00035f0 con 0x7f84f40d46a0 2024-04-20T11:27:57.411 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.409+0000 7f84eb400640 1 -- 172.21.15.152:0/2480370652 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e00039e0 con 0x7f84f40d46a0 2024-04-20T11:27:57.411 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.409+0000 7f84fa200640 1 -- 172.21.15.152:0/2480370652 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84f40b5810 con 0x7f84f40d46a0 2024-04-20T11:27:57.411 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.409+0000 7f84fa200640 1 -- 172.21.15.152:0/2480370652 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84f4152810 con 0x7f84f40d46a0 2024-04-20T11:27:57.411 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.410+0000 7f84fa200640 1 -- 172.21.15.152:0/2480370652 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84f40689f0 con 0x7f84f40d46a0 2024-04-20T11:27:57.412 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.411+0000 7f84eb400640 1 -- 172.21.15.152:0/2480370652 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f84e001a070 con 0x7f84f40d46a0 2024-04-20T11:27:57.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.412+0000 7f84eb400640 1 --2- 172.21.15.152:0/2480370652 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f84d406c8f0 0x7f84d406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:57.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.413+0000 7f84eb400640 1 -- 172.21.15.152:0/2480370652 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f84e008b110 con 0x7f84f40d46a0 2024-04-20T11:27:57.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.413+0000 7f84f3400640 1 --2- 172.21.15.152:0/2480370652 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f84d406c8f0 0x7f84d406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:57.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.413+0000 7f84f3400640 1 --2- 172.21.15.152:0/2480370652 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f84d406c8f0 0x7f84d406edb0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f84e4036000 tx=0x7f84e403c040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:57.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.417+0000 7f84eb400640 1 -- 172.21.15.152:0/2480370652 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f84e0059420 con 0x7f84f40d46a0 2024-04-20T11:27:57.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.617+0000 7f84fa200640 1 -- 172.21.15.152:0/2480370652 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f84f406cdb0 con 0x7f84f40d46a0 2024-04-20T11:27:57.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.618+0000 7f84eb400640 1 -- 172.21.15.152:0/2480370652 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) ==== 104+0+11 (secure 0 0 0) 0x7f84e0058d20 con 0x7f84f40d46a0 2024-04-20T11:27:57.618 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 32 2024-04-20T11:27:57.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.620+0000 7f84fa200640 1 -- 172.21.15.152:0/2480370652 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f84d406c8f0 msgr2=0x7f84d406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:57.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.620+0000 7f84fa200640 1 --2- 172.21.15.152:0/2480370652 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f84d406c8f0 0x7f84d406edb0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f84e4036000 tx=0x7f84e403c040 comp rx=0 tx=0).stop 2024-04-20T11:27:57.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.620+0000 7f84fa200640 1 -- 172.21.15.152:0/2480370652 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f84f40d46a0 msgr2=0x7f84f41613d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:57.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.620+0000 7f84fa200640 1 --2- 172.21.15.152:0/2480370652 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f84f40d46a0 0x7f84f41613d0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f84e000c410 tx=0x7f84e000c8d0 comp rx=0 tx=0).stop 2024-04-20T11:27:57.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.621+0000 7f84fa200640 1 -- 172.21.15.152:0/2480370652 shutdown_connections 2024-04-20T11:27:57.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.621+0000 7f84fa200640 1 --2- 172.21.15.152:0/2480370652 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f84d406c8f0 0x7f84d406edb0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:57.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.621+0000 7f84fa200640 1 --2- 172.21.15.152:0/2480370652 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f84f415e9e0 0x7f84f40b9530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:57.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.621+0000 7f84fa200640 1 --2- 172.21.15.152:0/2480370652 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f84f40d46a0 0x7f84f41613d0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:57.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.621+0000 7f84fa200640 1 --2- 172.21.15.152:0/2480370652 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f84f40d14a0 0x7f84f41522d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:57.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.621+0000 7f84fa200640 1 -- 172.21.15.152:0/2480370652 >> 172.21.15.152:0/2480370652 conn(0x7f84f409dab0 msgr2=0x7f84f40d2d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:57.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.621+0000 7f84fa200640 1 -- 172.21.15.152:0/2480370652 shutdown_connections 2024-04-20T11:27:57.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.621+0000 7f84fa200640 1 -- 172.21.15.152:0/2480370652 wait complete. 2024-04-20T11:27:57.641 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T11:27:57.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.847+0000 7f7661400640 1 Processor -- start 2024-04-20T11:27:57.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.847+0000 7f7661400640 1 -- start start 2024-04-20T11:27:57.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.848+0000 7f7661400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f765c0ad5f0 0x7f765c15ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:57.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.848+0000 7f7661400640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f765c0ad9d0 con 0x7f765c06c200 2024-04-20T11:27:57.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.849+0000 7f7661400640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f765c0b4f40 con 0x7f765c06bda0 2024-04-20T11:27:57.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.849+0000 7f7661400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f765c15b190 con 0x7f765c0ad5f0 2024-04-20T11:27:57.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.849+0000 7f7660a00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f765c06c200 0x7f765c0725c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:36066/0 (socket says 172.21.15.152:36066) 2024-04-20T11:27:57.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.849+0000 7f7660a00640 1 -- 172.21.15.152:0/2362489006 learned_addr learned my addr 172.21.15.152:0/2362489006 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:27:57.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.849+0000 7f765be00640 1 --2- 172.21.15.152:0/2362489006 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f765c0ad5f0 0x7f765c15ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:57.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.850+0000 7f765aa00640 1 -- 172.21.15.152:0/2362489006 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3927485176 0 0) 0x7f765c0ad9d0 con 0x7f765c06c200 2024-04-20T11:27:57.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.850+0000 7f765aa00640 1 -- 172.21.15.152:0/2362489006 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7640003520 con 0x7f765c06c200 2024-04-20T11:27:57.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.850+0000 7f765aa00640 1 -- 172.21.15.152:0/2362489006 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1743316035 0 0) 0x7f765c0b4f40 con 0x7f765c06bda0 2024-04-20T11:27:57.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.850+0000 7f765aa00640 1 -- 172.21.15.152:0/2362489006 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f765c0ad9d0 con 0x7f765c06bda0 2024-04-20T11:27:57.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.850+0000 7f765be00640 1 -- 172.21.15.152:0/2362489006 >> v1:172.21.15.175:6789/0 conn(0x7f765c06bda0 legacy=0x7f765c0b4480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:57.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.850+0000 7f765be00640 1 -- 172.21.15.152:0/2362489006 >> v1:172.21.15.152:6789/0 conn(0x7f765c06c200 legacy=0x7f765c0725c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:57.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.850+0000 7f765aa00640 1 -- 172.21.15.152:0/2362489006 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2198596899 0 0) 0x7f7640003520 con 0x7f765c06c200 2024-04-20T11:27:57.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.850+0000 7f765be00640 1 -- 172.21.15.152:0/2362489006 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f765c06dd40 con 0x7f765c0ad5f0 2024-04-20T11:27:57.852 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.851+0000 7f765be00640 1 --2- 172.21.15.152:0/2362489006 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f765c0ad5f0 0x7f765c15ac50 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f764c009dc0 tx=0x7f764c02bde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78c48b29fc087c43 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:57.852 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.852+0000 7f765aa00640 1 -- 172.21.15.152:0/2362489006 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f764c039070 con 0x7f765c0ad5f0 2024-04-20T11:27:57.852 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.852+0000 7f765aa00640 1 -- 172.21.15.152:0/2362489006 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f764c02c910 con 0x7f765c0ad5f0 2024-04-20T11:27:57.852 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.852+0000 7f765aa00640 1 -- 172.21.15.152:0/2362489006 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7654001670 con 0x7f765c0ad5f0 2024-04-20T11:27:57.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.852+0000 7f7661400640 1 -- 172.21.15.152:0/2362489006 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f765c0ad5f0 msgr2=0x7f765c15ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:57.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.852+0000 7f7661400640 1 --2- 172.21.15.152:0/2362489006 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f765c0ad5f0 0x7f765c15ac50 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f764c009dc0 tx=0x7f764c02bde0 comp rx=0 tx=0).stop 2024-04-20T11:27:57.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.853+0000 7f7661400640 1 -- 172.21.15.152:0/2362489006 shutdown_connections 2024-04-20T11:27:57.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.853+0000 7f7661400640 1 --2- 172.21.15.152:0/2362489006 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f765c0ad5f0 0x7f765c15ac50 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:57.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.853+0000 7f7661400640 1 -- 172.21.15.152:0/2362489006 >> 172.21.15.152:0/2362489006 conn(0x7f765c0b21e0 msgr2=0x7f765c0b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:57.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.853+0000 7f7661400640 1 -- 172.21.15.152:0/2362489006 shutdown_connections 2024-04-20T11:27:57.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.853+0000 7f7661400640 1 -- 172.21.15.152:0/2362489006 wait complete. 2024-04-20T11:27:57.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.854+0000 7f7661400640 1 Processor -- start 2024-04-20T11:27:57.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.855+0000 7f7661400640 1 -- start start 2024-04-20T11:27:57.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.855+0000 7f7661400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f765c06bda0 0x7f765c072b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:57.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.855+0000 7f7661400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f765c06c200 0x7f765c06f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:57.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.855+0000 7f7661400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f765c0ad5f0 0x7f765c09bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:57.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.855+0000 7f7661400640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f765c071dd0 con 0x7f765c06c200 2024-04-20T11:27:57.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.856+0000 7f7661400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f765c06e300 con 0x7f765c06bda0 2024-04-20T11:27:57.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.856+0000 7f7661400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f765c0b4b90 con 0x7f765c0ad5f0 2024-04-20T11:27:57.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.856+0000 7f765b400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f765c06c200 0x7f765c06f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:57.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.856+0000 7f765b400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f765c06c200 0x7f765c06f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:32880/0 (socket says 172.21.15.152:32880) 2024-04-20T11:27:57.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.856+0000 7f765b400640 1 -- 172.21.15.152:0/2157792789 learned_addr learned my addr 172.21.15.152:0/2157792789 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:57.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.856+0000 7f765be00640 1 --2- 172.21.15.152:0/2157792789 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f765c06bda0 0x7f765c072b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:57.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.856+0000 7f7660a00640 1 --2- 172.21.15.152:0/2157792789 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f765c0ad5f0 0x7f765c09bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:57.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.856+0000 7f765b400640 1 -- 172.21.15.152:0/2157792789 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f765c0ad5f0 msgr2=0x7f765c09bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:57.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.856+0000 7f765b400640 1 --2- 172.21.15.152:0/2157792789 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f765c0ad5f0 0x7f765c09bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:57.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.856+0000 7f765b400640 1 -- 172.21.15.152:0/2157792789 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f765c06bda0 msgr2=0x7f765c072b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:57.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.856+0000 7f765b400640 1 --2- 172.21.15.152:0/2157792789 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f765c06bda0 0x7f765c072b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:57.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.856+0000 7f765b400640 1 -- 172.21.15.152:0/2157792789 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f765c152bc0 con 0x7f765c06c200 2024-04-20T11:27:57.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.856+0000 7f7660a00640 1 --2- 172.21.15.152:0/2157792789 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f765c0ad5f0 0x7f765c09bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:27:57.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.857+0000 7f765b400640 1 --2- 172.21.15.152:0/2157792789 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f765c06c200 0x7f765c06f5d0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f764400c530 tx=0x7f764400c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:57.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.857+0000 7f7653e00640 1 -- 172.21.15.152:0/2157792789 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76440052d0 con 0x7f765c06c200 2024-04-20T11:27:57.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.857+0000 7f7653e00640 1 -- 172.21.15.152:0/2157792789 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7644005490 con 0x7f765c06c200 2024-04-20T11:27:57.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.857+0000 7f7661400640 1 -- 172.21.15.152:0/2157792789 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f765c152e60 con 0x7f765c06c200 2024-04-20T11:27:57.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.857+0000 7f7653e00640 1 -- 172.21.15.152:0/2157792789 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7644012870 con 0x7f765c06c200 2024-04-20T11:27:57.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.857+0000 7f7661400640 1 -- 172.21.15.152:0/2157792789 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f765c14d0e0 con 0x7f765c06c200 2024-04-20T11:27:57.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.858+0000 7f7661400640 1 -- 172.21.15.152:0/2157792789 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f765c153030 con 0x7f765c06c200 2024-04-20T11:27:57.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.859+0000 7f7653e00640 1 -- 172.21.15.152:0/2157792789 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7644012a10 con 0x7f765c06c200 2024-04-20T11:27:57.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.860+0000 7f7653e00640 1 --2- 172.21.15.152:0/2157792789 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f763806c8f0 0x7f763806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:57.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.860+0000 7f7653e00640 1 -- 172.21.15.152:0/2157792789 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f764408b8d0 con 0x7f765c06c200 2024-04-20T11:27:57.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.861+0000 7f765be00640 1 --2- 172.21.15.152:0/2157792789 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f763806c8f0 0x7f763806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:57.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.861+0000 7f765be00640 1 --2- 172.21.15.152:0/2157792789 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f763806c8f0 0x7f763806edb0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f764c036000 tx=0x7f764c030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:57.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:57.865+0000 7f7653e00640 1 -- 172.21.15.152:0/2157792789 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f765c153030 con 0x7f765c06c200 2024-04-20T11:27:58.066 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.066+0000 7f7661400640 1 -- 172.21.15.152:0/2157792789 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f765c075e60 con 0x7f765c06c200 2024-04-20T11:27:58.067 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.067+0000 7f7653e00640 1 -- 172.21.15.152:0/2157792789 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) ==== 100+0+11 (secure 0 0 0) 0x7f7644059bf0 con 0x7f765c06c200 2024-04-20T11:27:58.067 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 64 2024-04-20T11:27:58.070 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.069+0000 7f7661400640 1 -- 172.21.15.152:0/2157792789 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f763806c8f0 msgr2=0x7f763806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:58.070 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.069+0000 7f7661400640 1 --2- 172.21.15.152:0/2157792789 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f763806c8f0 0x7f763806edb0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f764c036000 tx=0x7f764c030000 comp rx=0 tx=0).stop 2024-04-20T11:27:58.070 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.069+0000 7f7661400640 1 -- 172.21.15.152:0/2157792789 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f765c06c200 msgr2=0x7f765c06f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:58.070 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.069+0000 7f7661400640 1 --2- 172.21.15.152:0/2157792789 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f765c06c200 0x7f765c06f5d0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f764400c530 tx=0x7f764400c9f0 comp rx=0 tx=0).stop 2024-04-20T11:27:58.070 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.069+0000 7f7661400640 1 -- 172.21.15.152:0/2157792789 shutdown_connections 2024-04-20T11:27:58.070 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.069+0000 7f7661400640 1 --2- 172.21.15.152:0/2157792789 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f763806c8f0 0x7f763806edb0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:58.070 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.069+0000 7f7661400640 1 --2- 172.21.15.152:0/2157792789 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f765c0ad5f0 0x7f765c09bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:58.071 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.069+0000 7f7661400640 1 --2- 172.21.15.152:0/2157792789 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f765c06c200 0x7f765c06f5d0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:58.071 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.069+0000 7f7661400640 1 --2- 172.21.15.152:0/2157792789 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f765c06bda0 0x7f765c072b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:58.071 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.069+0000 7f7661400640 1 -- 172.21.15.152:0/2157792789 >> 172.21.15.152:0/2157792789 conn(0x7f765c0b21e0 msgr2=0x7f765c09db10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:58.071 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.069+0000 7f7661400640 1 -- 172.21.15.152:0/2157792789 shutdown_connections 2024-04-20T11:27:58.071 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.070+0000 7f7661400640 1 -- 172.21.15.152:0/2157792789 wait complete. 2024-04-20T11:27:58.090 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-20T11:27:58.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.290+0000 7f87c4a00640 1 Processor -- start 2024-04-20T11:27:58.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.290+0000 7f87c4a00640 1 -- start start 2024-04-20T11:27:58.292 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.291+0000 7f87c4a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f87c00cd4a0 0x7f87c015cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:58.292 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.291+0000 7f87c4a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f87c0070860 0x7f87c0070cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:58.292 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.291+0000 7f87c4a00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87c015d020 con 0x7f87c009b930 2024-04-20T11:27:58.292 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.291+0000 7f87c4a00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f87c00703d0 con 0x7f87c0070860 2024-04-20T11:27:58.292 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.291+0000 7f87c4a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f87c00700a0 con 0x7f87c00cd4a0 2024-04-20T11:27:58.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.292+0000 7f87bfe00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f87c0070860 0x7f87c0070cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:58.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.292+0000 7f87bea00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f87c009b930 0x7f87c00cd880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:36080/0 (socket says 172.21.15.152:36080) 2024-04-20T11:27:58.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.292+0000 7f87bea00640 1 -- 172.21.15.152:0/2383957242 learned_addr learned my addr 172.21.15.152:0/2383957242 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:27:58.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.292+0000 7f87bf400640 1 --2- 172.21.15.152:0/2383957242 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f87c00cd4a0 0x7f87c015cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:58.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.292+0000 7f87bfe00640 1 -- 172.21.15.152:0/2383957242 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f87c00cd4a0 msgr2=0x7f87c015cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:58.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.292+0000 7f87bfe00640 1 --2- 172.21.15.152:0/2383957242 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f87c00cd4a0 0x7f87c015cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:58.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.292+0000 7f87bfe00640 1 -- 172.21.15.152:0/2383957242 >> v1:172.21.15.152:6789/0 conn(0x7f87c009b930 legacy=0x7f87c00cd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:58.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.292+0000 7f87bfe00640 1 -- 172.21.15.152:0/2383957242 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87c015d2a0 con 0x7f87c0070860 2024-04-20T11:27:58.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.292+0000 7f87be000640 1 -- 172.21.15.152:0/2383957242 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1761507114 0 0) 0x7f87a8001110 con 0x7f87c009b930 2024-04-20T11:27:58.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.294+0000 7f87bfe00640 1 --2- 172.21.15.152:0/2383957242 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87c0070860 0x7f87c0070cf0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f87b8009d90 tx=0x7f87b802bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3678e8339deefab server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:58.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.294+0000 7f87be000640 1 -- 172.21.15.152:0/2383957242 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87b8039070 con 0x7f87c0070860 2024-04-20T11:27:58.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.294+0000 7f87be000640 1 -- 172.21.15.152:0/2383957242 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87b802cc50 con 0x7f87c0070860 2024-04-20T11:27:58.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.294+0000 7f87be000640 1 -- 172.21.15.152:0/2383957242 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87b8034440 con 0x7f87c0070860 2024-04-20T11:27:58.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.295+0000 7f87c4a00640 1 -- 172.21.15.152:0/2383957242 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87c0070860 msgr2=0x7f87c0070cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:58.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.295+0000 7f87c4a00640 1 --2- 172.21.15.152:0/2383957242 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87c0070860 0x7f87c0070cf0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f87b8009d90 tx=0x7f87b802bea0 comp rx=0 tx=0).stop 2024-04-20T11:27:58.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.295+0000 7f87c4a00640 1 -- 172.21.15.152:0/2383957242 shutdown_connections 2024-04-20T11:27:58.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.295+0000 7f87c4a00640 1 --2- 172.21.15.152:0/2383957242 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87c0070860 0x7f87c0070cf0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:58.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.295+0000 7f87c4a00640 1 --2- 172.21.15.152:0/2383957242 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f87c00cd4a0 0x7f87c015cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:58.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.295+0000 7f87c4a00640 1 -- 172.21.15.152:0/2383957242 >> 172.21.15.152:0/2383957242 conn(0x7f87c00b3f30 msgr2=0x7f87c00b4340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:58.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.295+0000 7f87c4a00640 1 -- 172.21.15.152:0/2383957242 shutdown_connections 2024-04-20T11:27:58.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.295+0000 7f87c4a00640 1 -- 172.21.15.152:0/2383957242 wait complete. 2024-04-20T11:27:58.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.297+0000 7f87c4a00640 1 Processor -- start 2024-04-20T11:27:58.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.297+0000 7f87c4a00640 1 -- start start 2024-04-20T11:27:58.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.298+0000 7f87c4a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87c0070860 0x7f87c00c4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:58.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.298+0000 7f87c4a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87c009b930 0x7f87c00c8140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:58.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.298+0000 7f87c4a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f87c00cd4a0 0x7f87c00c8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:58.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.298+0000 7f87c4a00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f87c0071230 con 0x7f87c0070860 2024-04-20T11:27:58.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.298+0000 7f87c4a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f87c015d860 con 0x7f87c009b930 2024-04-20T11:27:58.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.298+0000 7f87c4a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f87c00713b0 con 0x7f87c00cd4a0 2024-04-20T11:27:58.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.298+0000 7f87bf400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87c0070860 0x7f87c00c4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:58.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.298+0000 7f87bf400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87c0070860 0x7f87c00c4fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:32898/0 (socket says 172.21.15.152:32898) 2024-04-20T11:27:58.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.298+0000 7f87bf400640 1 -- 172.21.15.152:0/3546259028 learned_addr learned my addr 172.21.15.152:0/3546259028 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:58.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.298+0000 7f87bea00640 1 --2- 172.21.15.152:0/3546259028 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87c009b930 0x7f87c00c8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:58.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.298+0000 7f87bfe00640 1 --2- 172.21.15.152:0/3546259028 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f87c00cd4a0 0x7f87c00c8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:58.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.298+0000 7f87bf400640 1 -- 172.21.15.152:0/3546259028 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f87c00cd4a0 msgr2=0x7f87c00c8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:58.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.298+0000 7f87bf400640 1 --2- 172.21.15.152:0/3546259028 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f87c00cd4a0 0x7f87c00c8680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:58.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.298+0000 7f87bf400640 1 -- 172.21.15.152:0/3546259028 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87c009b930 msgr2=0x7f87c00c8140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:58.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.298+0000 7f87bf400640 1 --2- 172.21.15.152:0/3546259028 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87c009b930 0x7f87c00c8140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:58.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.298+0000 7f87bf400640 1 -- 172.21.15.152:0/3546259028 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87c00c54e0 con 0x7f87c0070860 2024-04-20T11:27:58.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.298+0000 7f87bea00640 1 --2- 172.21.15.152:0/3546259028 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87c009b930 0x7f87c00c8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:27:58.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.299+0000 7f87bf400640 1 --2- 172.21.15.152:0/3546259028 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87c0070860 0x7f87c00c4fa0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f87b000d5f0 tx=0x7f87b000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:58.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.299+0000 7f87b7400640 1 -- 172.21.15.152:0/3546259028 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87b00125f0 con 0x7f87c0070860 2024-04-20T11:27:58.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.299+0000 7f87b7400640 1 -- 172.21.15.152:0/3546259028 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87b0019070 con 0x7f87c0070860 2024-04-20T11:27:58.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.299+0000 7f87b7400640 1 -- 172.21.15.152:0/3546259028 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87b00129d0 con 0x7f87c0070860 2024-04-20T11:27:58.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.299+0000 7f87c4a00640 1 -- 172.21.15.152:0/3546259028 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87c006cb30 con 0x7f87c0070860 2024-04-20T11:27:58.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.299+0000 7f87c4a00640 1 -- 172.21.15.152:0/3546259028 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87c006d0f0 con 0x7f87c0070860 2024-04-20T11:27:58.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.300+0000 7f87c4a00640 1 -- 172.21.15.152:0/3546259028 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87c0075e60 con 0x7f87c0070860 2024-04-20T11:27:58.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.301+0000 7f87b7400640 1 -- 172.21.15.152:0/3546259028 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f87b0012b70 con 0x7f87c0070860 2024-04-20T11:27:58.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.302+0000 7f87b7400640 1 --2- 172.21.15.152:0/3546259028 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f879c06c8f0 0x7f879c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:58.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.302+0000 7f87b7400640 1 -- 172.21.15.152:0/3546259028 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f87b008bfe0 con 0x7f87c0070860 2024-04-20T11:27:58.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.303+0000 7f87bea00640 1 --2- 172.21.15.152:0/3546259028 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f879c06c8f0 0x7f879c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:58.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.303+0000 7f87bea00640 1 --2- 172.21.15.152:0/3546259028 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f879c06c8f0 0x7f879c06edb0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f87a80013b0 tx=0x7f87a8009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:58.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.307+0000 7f87b7400640 1 -- 172.21.15.152:0/3546259028 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f87b005a380 con 0x7f87c0070860 2024-04-20T11:27:58.523 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.523+0000 7f87c4a00640 1 -- 172.21.15.152:0/3546259028 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f87c006d7f0 con 0x7f87c0070860 2024-04-20T11:27:58.524 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.523+0000 7f87b7400640 1 -- 172.21.15.152:0/3546259028 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) ==== 80+0+0 (secure 0 0 0) 0x7f87b0059c80 con 0x7f87c0070860 2024-04-20T11:27:58.524 INFO:teuthology.orchestra.run.smithi152.stderr:listed 0 entries 2024-04-20T11:27:58.527 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.526+0000 7f87c4a00640 1 -- 172.21.15.152:0/3546259028 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f879c06c8f0 msgr2=0x7f879c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:58.527 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.526+0000 7f87c4a00640 1 --2- 172.21.15.152:0/3546259028 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f879c06c8f0 0x7f879c06edb0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f87a80013b0 tx=0x7f87a8009040 comp rx=0 tx=0).stop 2024-04-20T11:27:58.527 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.526+0000 7f87c4a00640 1 -- 172.21.15.152:0/3546259028 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87c0070860 msgr2=0x7f87c00c4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:58.527 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.526+0000 7f87c4a00640 1 --2- 172.21.15.152:0/3546259028 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87c0070860 0x7f87c00c4fa0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f87b000d5f0 tx=0x7f87b000dab0 comp rx=0 tx=0).stop 2024-04-20T11:27:58.527 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.526+0000 7f87c4a00640 1 -- 172.21.15.152:0/3546259028 shutdown_connections 2024-04-20T11:27:58.527 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.526+0000 7f87c4a00640 1 --2- 172.21.15.152:0/3546259028 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f879c06c8f0 0x7f879c06edb0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:58.527 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.526+0000 7f87c4a00640 1 --2- 172.21.15.152:0/3546259028 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f87c00cd4a0 0x7f87c00c8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:58.527 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.526+0000 7f87c4a00640 1 --2- 172.21.15.152:0/3546259028 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87c009b930 0x7f87c00c8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:58.527 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.526+0000 7f87c4a00640 1 --2- 172.21.15.152:0/3546259028 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87c0070860 0x7f87c00c4fa0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:58.527 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.526+0000 7f87c4a00640 1 -- 172.21.15.152:0/3546259028 >> 172.21.15.152:0/3546259028 conn(0x7f87c00b3f30 msgr2=0x7f87c00cd880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:58.528 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.526+0000 7f87c4a00640 1 -- 172.21.15.152:0/3546259028 shutdown_connections 2024-04-20T11:27:58.528 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.527+0000 7f87c4a00640 1 -- 172.21.15.152:0/3546259028 wait complete. 2024-04-20T11:27:58.546 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T11:27:58.546 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:27:58.597 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:27:58.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.801+0000 7fbfad400640 1 Processor -- start 2024-04-20T11:27:58.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.802+0000 7fbfad400640 1 -- start start 2024-04-20T11:27:58.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.803+0000 7fbfad400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fbfa81603d0 0x7fbfa80b5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:58.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.803+0000 7fbfad400640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbfa80d3350 con 0x7fbfa815fcc0 2024-04-20T11:27:58.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.803+0000 7fbfad400640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbfa80d3630 con 0x7fbfa80b5480 2024-04-20T11:27:58.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.803+0000 7fbfad400640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fbfa80d3910 con 0x7fbfa81603d0 2024-04-20T11:27:58.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.803+0000 7fbfa7400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fbfa81603d0 0x7fbfa80b5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:58.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.803+0000 7fbfa7e00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fbfa815fcc0 0x7fbfa80d4e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:36082/0 (socket says 172.21.15.152:36082) 2024-04-20T11:27:58.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.803+0000 7fbfa7e00640 1 -- 172.21.15.152:0/2455409506 learned_addr learned my addr 172.21.15.152:0/2455409506 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:27:58.805 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.804+0000 7fbfa7400640 1 -- 172.21.15.152:0/2455409506 >> v1:172.21.15.175:6789/0 conn(0x7fbfa80b5480 legacy=0x7fbfa80cf240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:58.805 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.804+0000 7fbfa7400640 1 -- 172.21.15.152:0/2455409506 >> v1:172.21.15.152:6789/0 conn(0x7fbfa815fcc0 legacy=0x7fbfa80d4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:58.805 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.804+0000 7fbfa7400640 1 -- 172.21.15.152:0/2455409506 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfa80ced20 con 0x7fbfa81603d0 2024-04-20T11:27:58.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.805+0000 7fbfa7400640 1 --2- 172.21.15.152:0/2455409506 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbfa81603d0 0x7fbfa80b5eb0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fbf94009790 tx=0x7fbf9402b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf04a89db9a4badd server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:58.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.805+0000 7fbfa6a00640 1 -- 172.21.15.152:0/2455409506 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf94034070 con 0x7fbfa81603d0 2024-04-20T11:27:58.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.805+0000 7fbfa6a00640 1 -- 172.21.15.152:0/2455409506 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf94030800 con 0x7fbfa81603d0 2024-04-20T11:27:58.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.805+0000 7fbfa6a00640 1 -- 172.21.15.152:0/2455409506 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf94030bd0 con 0x7fbfa81603d0 2024-04-20T11:27:58.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.806+0000 7fbfad400640 1 -- 172.21.15.152:0/2455409506 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbfa81603d0 msgr2=0x7fbfa80b5eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:58.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.806+0000 7fbfad400640 1 --2- 172.21.15.152:0/2455409506 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbfa81603d0 0x7fbfa80b5eb0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fbf94009790 tx=0x7fbf9402b940 comp rx=0 tx=0).stop 2024-04-20T11:27:58.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.806+0000 7fbfad400640 1 -- 172.21.15.152:0/2455409506 shutdown_connections 2024-04-20T11:27:58.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.806+0000 7fbfad400640 1 --2- 172.21.15.152:0/2455409506 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbfa81603d0 0x7fbfa80b5eb0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:58.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.806+0000 7fbfad400640 1 -- 172.21.15.152:0/2455409506 >> 172.21.15.152:0/2455409506 conn(0x7fbfa808bfd0 msgr2=0x7fbfa80d4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:58.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.806+0000 7fbfad400640 1 -- 172.21.15.152:0/2455409506 shutdown_connections 2024-04-20T11:27:58.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.806+0000 7fbfad400640 1 -- 172.21.15.152:0/2455409506 wait complete. 2024-04-20T11:27:58.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.808+0000 7fbfad400640 1 Processor -- start 2024-04-20T11:27:58.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.808+0000 7fbfad400640 1 -- start start 2024-04-20T11:27:58.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.809+0000 7fbfad400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbfa80b5480 0x7fbfa80cd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:58.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.809+0000 7fbfad400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbfa815fcc0 0x7fbfa815bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:58.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.809+0000 7fbfad400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbfa81603d0 0x7fbfa80c94f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:58.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.809+0000 7fbfad400640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fbfa80b6800 con 0x7fbfa815fcc0 2024-04-20T11:27:58.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.809+0000 7fbfad400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbfa80b6680 con 0x7fbfa81603d0 2024-04-20T11:27:58.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.809+0000 7fbfad400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fbfa80cf950 con 0x7fbfa80b5480 2024-04-20T11:27:58.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.809+0000 7fbfa7400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbfa815fcc0 0x7fbfa815bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:58.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.809+0000 7fbfa7400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbfa815fcc0 0x7fbfa815bad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:32904/0 (socket says 172.21.15.152:32904) 2024-04-20T11:27:58.812 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.809+0000 7fbfa7400640 1 -- 172.21.15.152:0/2986434066 learned_addr learned my addr 172.21.15.152:0/2986434066 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:58.812 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.809+0000 7fbfaca00640 1 --2- 172.21.15.152:0/2986434066 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbfa81603d0 0x7fbfa80c94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:58.812 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.810+0000 7fbfa7e00640 1 --2- 172.21.15.152:0/2986434066 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbfa80b5480 0x7fbfa80cd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:58.812 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.810+0000 7fbfa7400640 1 -- 172.21.15.152:0/2986434066 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbfa80b5480 msgr2=0x7fbfa80cd3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:58.812 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.810+0000 7fbfa7400640 1 --2- 172.21.15.152:0/2986434066 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbfa80b5480 0x7fbfa80cd3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:58.812 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.810+0000 7fbfa7400640 1 -- 172.21.15.152:0/2986434066 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbfa81603d0 msgr2=0x7fbfa80c94f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:58.812 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.810+0000 7fbfa7400640 1 --2- 172.21.15.152:0/2986434066 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbfa81603d0 0x7fbfa80c94f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:58.812 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.810+0000 7fbfa7400640 1 -- 172.21.15.152:0/2986434066 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfa80cd8f0 con 0x7fbfa815fcc0 2024-04-20T11:27:58.812 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.810+0000 7fbfa7400640 1 --2- 172.21.15.152:0/2986434066 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbfa815fcc0 0x7fbfa815bad0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fbf940098b0 tx=0x7fbf940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:58.812 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.810+0000 7fbf9fe00640 1 -- 172.21.15.152:0/2986434066 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf94047070 con 0x7fbfa815fcc0 2024-04-20T11:27:58.812 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.810+0000 7fbf9fe00640 1 -- 172.21.15.152:0/2986434066 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf94039d10 con 0x7fbfa815fcc0 2024-04-20T11:27:58.812 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.810+0000 7fbfad400640 1 -- 172.21.15.152:0/2986434066 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfa80cc030 con 0x7fbfa815fcc0 2024-04-20T11:27:58.812 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.811+0000 7fbf9fe00640 1 -- 172.21.15.152:0/2986434066 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf94034040 con 0x7fbfa815fcc0 2024-04-20T11:27:58.812 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.811+0000 7fbfad400640 1 -- 172.21.15.152:0/2986434066 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfa80cc5f0 con 0x7fbfa815fcc0 2024-04-20T11:27:58.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.812+0000 7fbfad400640 1 -- 172.21.15.152:0/2986434066 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfa8075e60 con 0x7fbfa815fcc0 2024-04-20T11:27:58.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.813+0000 7fbf9fe00640 1 -- 172.21.15.152:0/2986434066 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbf9404a070 con 0x7fbfa815fcc0 2024-04-20T11:27:58.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.814+0000 7fbf9fe00640 1 --2- 172.21.15.152:0/2986434066 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fbf8806c8f0 0x7fbf8806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:58.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.814+0000 7fbf9fe00640 1 -- 172.21.15.152:0/2986434066 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fbf940ad890 con 0x7fbfa815fcc0 2024-04-20T11:27:58.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.814+0000 7fbfa7e00640 1 --2- 172.21.15.152:0/2986434066 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fbf8806c8f0 0x7fbf8806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:58.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.815+0000 7fbfa7e00640 1 --2- 172.21.15.152:0/2986434066 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fbf8806c8f0 0x7fbf8806edb0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fbf98005c00 tx=0x7fbf98009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:58.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:58.819+0000 7fbf9fe00640 1 -- 172.21.15.152:0/2986434066 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fbfa80cd8f0 con 0x7fbfa815fcc0 2024-04-20T11:27:59.018 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.017+0000 7fbfad400640 1 -- 172.21.15.152:0/2986434066 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbfa80b6ce0 con 0x7fbfa815fcc0 2024-04-20T11:27:59.021 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.020+0000 7fbf9fe00640 1 -- 172.21.15.152:0/2986434066 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16914 (secure 0 0 0) 0x7fbf9407bba0 con 0x7fbfa815fcc0 2024-04-20T11:27:59.021 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:27:59.021 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":23,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:27:42.601620+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:26:56.261424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T11:26:57.079286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":2}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.26","mappings":[{"from":7,"to":1}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:27:59.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.023+0000 7fbfad400640 1 -- 172.21.15.152:0/2986434066 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fbf8806c8f0 msgr2=0x7fbf8806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:59.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.023+0000 7fbfad400640 1 --2- 172.21.15.152:0/2986434066 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fbf8806c8f0 0x7fbf8806edb0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fbf98005c00 tx=0x7fbf98009040 comp rx=0 tx=0).stop 2024-04-20T11:27:59.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.023+0000 7fbfad400640 1 -- 172.21.15.152:0/2986434066 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbfa815fcc0 msgr2=0x7fbfa815bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:59.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.023+0000 7fbfad400640 1 --2- 172.21.15.152:0/2986434066 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbfa815fcc0 0x7fbfa815bad0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fbf940098b0 tx=0x7fbf940046c0 comp rx=0 tx=0).stop 2024-04-20T11:27:59.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.023+0000 7fbfad400640 1 -- 172.21.15.152:0/2986434066 shutdown_connections 2024-04-20T11:27:59.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.023+0000 7fbfad400640 1 --2- 172.21.15.152:0/2986434066 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fbf8806c8f0 0x7fbf8806edb0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:59.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.023+0000 7fbfad400640 1 --2- 172.21.15.152:0/2986434066 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbfa81603d0 0x7fbfa80c94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:59.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.023+0000 7fbfad400640 1 --2- 172.21.15.152:0/2986434066 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbfa815fcc0 0x7fbfa815bad0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:59.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.023+0000 7fbfad400640 1 --2- 172.21.15.152:0/2986434066 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbfa80b5480 0x7fbfa80cd3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:59.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.023+0000 7fbfad400640 1 -- 172.21.15.152:0/2986434066 >> 172.21.15.152:0/2986434066 conn(0x7fbfa808bfd0 msgr2=0x7fbfa80d44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:59.026 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.024+0000 7fbfad400640 1 -- 172.21.15.152:0/2986434066 shutdown_connections 2024-04-20T11:27:59.026 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.024+0000 7fbfad400640 1 -- 172.21.15.152:0/2986434066 wait complete. 2024-04-20T11:27:59.043 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T11:26:47.246571+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T11:26:56.261424+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T11:26:57.079286+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T11:27:59.043 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T11:27:59.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.246+0000 7f8b21200640 1 Processor -- start 2024-04-20T11:27:59.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.246+0000 7f8b21200640 1 -- start start 2024-04-20T11:27:59.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.248+0000 7f8b21200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8b1c0cebe0 0x7f8b1c1628c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:59.249 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.248+0000 7f8b21200640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b1c0cde20 con 0x7f8b1c1624e0 2024-04-20T11:27:59.249 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.248+0000 7f8b21200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b1c0ce100 con 0x7f8b1c162e00 2024-04-20T11:27:59.249 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.248+0000 7f8b21200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8b1c0ce3e0 con 0x7f8b1c0cebe0 2024-04-20T11:27:59.249 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.248+0000 7f8b19e00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f8b1c1624e0 0x7f8b1c0b2630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:36090/0 (socket says 172.21.15.152:36090) 2024-04-20T11:27:59.249 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.248+0000 7f8b19e00640 1 -- 172.21.15.152:0/2230964552 learned_addr learned my addr 172.21.15.152:0/2230964552 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:27:59.251 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.249+0000 7f8b11400640 1 --2- 172.21.15.152:0/2230964552 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8b1c0cebe0 0x7f8b1c1628c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:59.251 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.249+0000 7f8b19400640 1 -- 172.21.15.152:0/2230964552 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 838911833 0 0) 0x7f8b1c0cde20 con 0x7f8b1c1624e0 2024-04-20T11:27:59.251 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.249+0000 7f8b19400640 1 -- 172.21.15.152:0/2230964552 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8b00003520 con 0x7f8b1c1624e0 2024-04-20T11:27:59.251 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.249+0000 7f8b19400640 1 -- 172.21.15.152:0/2230964552 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 499563395 0 0) 0x7f8b00003520 con 0x7f8b1c1624e0 2024-04-20T11:27:59.251 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.249+0000 7f8b19400640 1 -- 172.21.15.152:0/2230964552 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8b1c0cebe0 msgr2=0x7f8b1c1628c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:59.251 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.249+0000 7f8b19400640 1 --2- 172.21.15.152:0/2230964552 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8b1c0cebe0 0x7f8b1c1628c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:59.251 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.249+0000 7f8b19400640 1 -- 172.21.15.152:0/2230964552 >> v1:172.21.15.175:6789/0 conn(0x7f8b1c162e00 legacy=0x7f8b1c0cf0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:59.251 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.249+0000 7f8b19400640 1 -- 172.21.15.152:0/2230964552 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b1c0ce6c0 con 0x7f8b1c1624e0 2024-04-20T11:27:59.251 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.249+0000 7f8b19400640 1 -- 172.21.15.152:0/2230964552 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f8b0c001670 con 0x7f8b1c1624e0 2024-04-20T11:27:59.251 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.249+0000 7f8b19400640 1 -- 172.21.15.152:0/2230964552 >> v1:172.21.15.152:6789/0 conn(0x7f8b1c1624e0 legacy=0x7f8b1c0b2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:59.252 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.249+0000 7f8b19400640 1 --2- 172.21.15.152:0/2230964552 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8b1c0cebe0 0x7f8b000039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:59.252 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.249+0000 7f8b19400640 1 --2- 172.21.15.152:0/2230964552 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8b00003f60 0x7f8b00004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:59.252 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.249+0000 7f8b19400640 1 --2- 172.21.15.152:0/2230964552 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b00004d40 0x7f8b000051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:59.252 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.249+0000 7f8b19400640 1 -- 172.21.15.152:0/2230964552 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f8b1c0ce3e0 con 0x7f8b00003f60 2024-04-20T11:27:59.252 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.249+0000 7f8b19400640 1 -- 172.21.15.152:0/2230964552 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8b0000baa0 con 0x7f8b00004d40 2024-04-20T11:27:59.252 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.249+0000 7f8b19400640 1 -- 172.21.15.152:0/2230964552 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8b0000bca0 con 0x7f8b1c0cebe0 2024-04-20T11:27:59.252 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.250+0000 7f8b11400640 1 --2- 172.21.15.152:0/2230964552 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8b00003f60 0x7f8b00004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:59.252 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.250+0000 7f8b1a800640 1 --2- 172.21.15.152:0/2230964552 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b00004d40 0x7f8b000051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:59.252 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.250+0000 7f8b19e00640 1 --2- 172.21.15.152:0/2230964552 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8b1c0cebe0 0x7f8b000039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:59.252 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.250+0000 7f8b11400640 1 -- 172.21.15.152:0/2230964552 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8b1c0cebe0 msgr2=0x7f8b000039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:59.252 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.250+0000 7f8b11400640 1 --2- 172.21.15.152:0/2230964552 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8b1c0cebe0 0x7f8b000039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:59.252 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.250+0000 7f8b11400640 1 -- 172.21.15.152:0/2230964552 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b00004d40 msgr2=0x7f8b000051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:59.252 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.250+0000 7f8b11400640 1 --2- 172.21.15.152:0/2230964552 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b00004d40 0x7f8b000051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:59.252 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.250+0000 7f8b11400640 1 -- 172.21.15.152:0/2230964552 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b00003520 con 0x7f8b00003f60 2024-04-20T11:27:59.253 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.251+0000 7f8b11400640 1 --2- 172.21.15.152:0/2230964552 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8b00003f60 0x7f8b00004800 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f8b04009ac0 tx=0x7f8b00005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cd281290cf3fee7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:59.254 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.252+0000 7f8b19400640 1 -- 172.21.15.152:0/2230964552 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b04039070 con 0x7f8b00003f60 2024-04-20T11:27:59.254 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.252+0000 7f8b19400640 1 -- 172.21.15.152:0/2230964552 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b0402c6b0 con 0x7f8b00003f60 2024-04-20T11:27:59.254 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.252+0000 7f8b19400640 1 -- 172.21.15.152:0/2230964552 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b04009690 con 0x7f8b00003f60 2024-04-20T11:27:59.254 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.252+0000 7f8b21200640 1 -- 172.21.15.152:0/2230964552 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8b00003f60 msgr2=0x7f8b00004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:59.254 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.252+0000 7f8b21200640 1 --2- 172.21.15.152:0/2230964552 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8b00003f60 0x7f8b00004800 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f8b04009ac0 tx=0x7f8b00005830 comp rx=0 tx=0).stop 2024-04-20T11:27:59.254 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.252+0000 7f8b1a800640 1 -- 172.21.15.152:0/2230964552 reap_dead start 2024-04-20T11:27:59.254 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.252+0000 7f8b21200640 1 -- 172.21.15.152:0/2230964552 shutdown_connections 2024-04-20T11:27:59.254 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.252+0000 7f8b21200640 1 -- 172.21.15.152:0/2230964552 >> 172.21.15.152:0/2230964552 conn(0x7f8b1c09aa40 msgr2=0x7f8b1c0b3130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:59.254 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.252+0000 7f8b21200640 1 -- 172.21.15.152:0/2230964552 shutdown_connections 2024-04-20T11:27:59.254 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.252+0000 7f8b21200640 1 -- 172.21.15.152:0/2230964552 wait complete. 2024-04-20T11:27:59.255 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.254+0000 7f8b21200640 1 Processor -- start 2024-04-20T11:27:59.255 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.254+0000 7f8b21200640 1 -- start start 2024-04-20T11:27:59.256 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.255+0000 7f8b21200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8b00003f60 0x7f8b1c14f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:59.256 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.255+0000 7f8b21200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b1c14fd70 0x7f8b1c15d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:59.256 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.255+0000 7f8b21200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8b1c1610c0 0x7f8b1c06b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:59.256 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.255+0000 7f8b21200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f8b1c15ec20 con 0x7f8b1c1610c0 2024-04-20T11:27:59.256 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.255+0000 7f8b21200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8b1c0b2e60 con 0x7f8b1c14fd70 2024-04-20T11:27:59.256 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.255+0000 7f8b21200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8b1c15eda0 con 0x7f8b00003f60 2024-04-20T11:27:59.258 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.256+0000 7f8b1a800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8b1c1610c0 0x7f8b1c06b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:59.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.256+0000 7f8b1a800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8b1c1610c0 0x7f8b1c06b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:32918/0 (socket says 172.21.15.152:32918) 2024-04-20T11:27:59.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.256+0000 7f8b1a800640 1 -- 172.21.15.152:0/1162182609 learned_addr learned my addr 172.21.15.152:0/1162182609 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:59.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.256+0000 7f8b11400640 1 --2- 172.21.15.152:0/1162182609 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b1c14fd70 0x7f8b1c15d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:59.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.256+0000 7f8b19e00640 1 --2- 172.21.15.152:0/1162182609 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8b00003f60 0x7f8b1c14f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:59.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.256+0000 7f8b1a800640 1 -- 172.21.15.152:0/1162182609 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8b00003f60 msgr2=0x7f8b1c14f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:59.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.256+0000 7f8b1a800640 1 --2- 172.21.15.152:0/1162182609 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8b00003f60 0x7f8b1c14f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:59.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.256+0000 7f8b1a800640 1 -- 172.21.15.152:0/1162182609 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b1c14fd70 msgr2=0x7f8b1c15d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:59.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.256+0000 7f8b1a800640 1 --2- 172.21.15.152:0/1162182609 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b1c14fd70 0x7f8b1c15d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:59.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.256+0000 7f8b1a800640 1 -- 172.21.15.152:0/1162182609 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b1c0caf90 con 0x7f8b1c1610c0 2024-04-20T11:27:59.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.256+0000 7f8b1a800640 1 --2- 172.21.15.152:0/1162182609 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8b1c1610c0 0x7f8b1c06b980 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c072de0 tx=0x7f8b1400ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:59.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.257+0000 7f8b12a00640 1 -- 172.21.15.152:0/1162182609 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b14003970 con 0x7f8b1c1610c0 2024-04-20T11:27:59.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.257+0000 7f8b12a00640 1 -- 172.21.15.152:0/1162182609 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b14018070 con 0x7f8b1c1610c0 2024-04-20T11:27:59.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.257+0000 7f8b12a00640 1 -- 172.21.15.152:0/1162182609 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b14003d50 con 0x7f8b1c1610c0 2024-04-20T11:27:59.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.257+0000 7f8b21200640 1 -- 172.21.15.152:0/1162182609 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b1c0cb270 con 0x7f8b1c1610c0 2024-04-20T11:27:59.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.257+0000 7f8b21200640 1 -- 172.21.15.152:0/1162182609 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b1c1608c0 con 0x7f8b1c1610c0 2024-04-20T11:27:59.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.258+0000 7f8b21200640 1 -- 172.21.15.152:0/1162182609 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b1c069100 con 0x7f8b1c1610c0 2024-04-20T11:27:59.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.259+0000 7f8b12a00640 1 -- 172.21.15.152:0/1162182609 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8b140050d0 con 0x7f8b1c1610c0 2024-04-20T11:27:59.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.260+0000 7f8b12a00640 1 --2- 172.21.15.152:0/1162182609 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f8af806c8f0 0x7f8af806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:59.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.261+0000 7f8b12a00640 1 -- 172.21.15.152:0/1162182609 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f8b1408b970 con 0x7f8b1c1610c0 2024-04-20T11:27:59.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.261+0000 7f8b19e00640 1 --2- 172.21.15.152:0/1162182609 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f8af806c8f0 0x7f8af806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:59.262 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.261+0000 7f8b19e00640 1 --2- 172.21.15.152:0/1162182609 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f8af806c8f0 0x7f8af806edb0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c001cd0 tx=0x7f8b0c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:59.266 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.266+0000 7f8b12a00640 1 -- 172.21.15.152:0/1162182609 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f8b14059c80 con 0x7f8b1c1610c0 2024-04-20T11:27:59.465 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.464+0000 7f8b21200640 1 -- 172.21.15.152:0/1162182609 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8b1c09af20 con 0x7f8b1c1610c0 2024-04-20T11:27:59.466 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.465+0000 7f8b12a00640 1 -- 172.21.15.152:0/1162182609 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) ==== 93+0+10 (secure 0 0 0) 0x7f8b14059580 con 0x7f8b1c1610c0 2024-04-20T11:27:59.466 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 1 2024-04-20T11:27:59.469 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.469+0000 7f8b21200640 1 -- 172.21.15.152:0/1162182609 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f8af806c8f0 msgr2=0x7f8af806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:59.469 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.469+0000 7f8b21200640 1 --2- 172.21.15.152:0/1162182609 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f8af806c8f0 0x7f8af806edb0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c001cd0 tx=0x7f8b0c009040 comp rx=0 tx=0).stop 2024-04-20T11:27:59.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.469+0000 7f8b21200640 1 -- 172.21.15.152:0/1162182609 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8b1c1610c0 msgr2=0x7f8b1c06b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:59.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.469+0000 7f8b21200640 1 --2- 172.21.15.152:0/1162182609 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8b1c1610c0 0x7f8b1c06b980 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c072de0 tx=0x7f8b1400ec10 comp rx=0 tx=0).stop 2024-04-20T11:27:59.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.469+0000 7f8b21200640 1 -- 172.21.15.152:0/1162182609 shutdown_connections 2024-04-20T11:27:59.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.469+0000 7f8b21200640 1 --2- 172.21.15.152:0/1162182609 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f8af806c8f0 0x7f8af806edb0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:59.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.469+0000 7f8b21200640 1 --2- 172.21.15.152:0/1162182609 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8b1c1610c0 0x7f8b1c06b980 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:59.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.469+0000 7f8b21200640 1 --2- 172.21.15.152:0/1162182609 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b1c14fd70 0x7f8b1c15d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:59.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.469+0000 7f8b21200640 1 --2- 172.21.15.152:0/1162182609 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8b00003f60 0x7f8b1c14f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:59.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.469+0000 7f8b21200640 1 -- 172.21.15.152:0/1162182609 >> 172.21.15.152:0/1162182609 conn(0x7f8b1c09aa40 msgr2=0x7f8b1c159f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:59.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.469+0000 7f8b21200640 1 -- 172.21.15.152:0/1162182609 shutdown_connections 2024-04-20T11:27:59.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.469+0000 7f8b21200640 1 -- 172.21.15.152:0/1162182609 wait complete. 2024-04-20T11:27:59.488 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T11:27:59.678 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.677+0000 7fa230a00640 1 Processor -- start 2024-04-20T11:27:59.678 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.678+0000 7fa230a00640 1 -- start start 2024-04-20T11:27:59.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.679+0000 7fa230a00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fa22c0cda70 0x7fa22c0d0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:59.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.679+0000 7fa230a00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa22c09bcf0 con 0x7fa22c15c820 2024-04-20T11:27:59.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.679+0000 7fa230a00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa22c0d0fc0 con 0x7fa22c0ccf80 2024-04-20T11:27:59.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.679+0000 7fa230a00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fa22c0b2430 con 0x7fa22c0cda70 2024-04-20T11:27:59.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.679+0000 7fa229e00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fa22c15c820 0x7fa22c0b28c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:36094/0 (socket says 172.21.15.152:36094) 2024-04-20T11:27:59.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.679+0000 7fa229e00640 1 -- 172.21.15.152:0/536367315 learned_addr learned my addr 172.21.15.152:0/536367315 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:27:59.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.680+0000 7fa22b200640 1 --2- 172.21.15.152:0/536367315 >> v2:172.21.15.175:3300/0 conn(0x7fa22c0cda70 0x7fa22c0d0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:59.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.680+0000 7fa229400640 1 -- 172.21.15.152:0/536367315 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4243847454 0 0) 0x7fa22c09bcf0 con 0x7fa22c15c820 2024-04-20T11:27:59.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.680+0000 7fa229400640 1 -- 172.21.15.152:0/536367315 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa20c003520 con 0x7fa22c15c820 2024-04-20T11:27:59.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.680+0000 7fa229400640 1 -- 172.21.15.152:0/536367315 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4289689285 0 0) 0x7fa20c003520 con 0x7fa22c15c820 2024-04-20T11:27:59.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.680+0000 7fa229400640 1 -- 172.21.15.152:0/536367315 >> v2:172.21.15.175:3300/0 conn(0x7fa22c0cda70 msgr2=0x7fa22c0d0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:59.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.680+0000 7fa229400640 1 --2- 172.21.15.152:0/536367315 >> v2:172.21.15.175:3300/0 conn(0x7fa22c0cda70 0x7fa22c0d0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:59.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.680+0000 7fa229400640 1 -- 172.21.15.152:0/536367315 >> v1:172.21.15.175:6789/0 conn(0x7fa22c0ccf80 legacy=0x7fa22c0cd360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:59.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.680+0000 7fa229400640 1 -- 172.21.15.152:0/536367315 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa22c0b1de0 con 0x7fa22c15c820 2024-04-20T11:27:59.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.680+0000 7fa229400640 1 -- 172.21.15.152:0/536367315 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7fa2140019f0 con 0x7fa22c15c820 2024-04-20T11:27:59.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.680+0000 7fa229400640 1 -- 172.21.15.152:0/536367315 >> v1:172.21.15.152:6789/0 conn(0x7fa22c15c820 legacy=0x7fa22c0b28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:59.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.681+0000 7fa229400640 1 --2- 172.21.15.152:0/536367315 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa20c0039b0 0x7fa20c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:59.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.681+0000 7fa229400640 1 --2- 172.21.15.152:0/536367315 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa20c004800 0x7fa20c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:59.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.681+0000 7fa229400640 1 --2- 172.21.15.152:0/536367315 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa20c0052e0 0x7fa20c00fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:59.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.681+0000 7fa229400640 1 -- 172.21.15.152:0/536367315 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fa22c0b2430 con 0x7fa20c004800 2024-04-20T11:27:59.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.681+0000 7fa229400640 1 -- 172.21.15.152:0/536367315 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa20c0105c0 con 0x7fa20c0052e0 2024-04-20T11:27:59.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.681+0000 7fa229400640 1 -- 172.21.15.152:0/536367315 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa20c0107c0 con 0x7fa20c0039b0 2024-04-20T11:27:59.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.681+0000 7fa229e00640 1 --2- 172.21.15.152:0/536367315 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa20c004800 0x7fa20c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:59.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.681+0000 7fa229e00640 1 -- 172.21.15.152:0/536367315 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa20c0039b0 msgr2=0x7fa20c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:59.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.681+0000 7fa229e00640 1 --2- 172.21.15.152:0/536367315 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa20c0039b0 0x7fa20c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:59.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.681+0000 7fa229e00640 1 -- 172.21.15.152:0/536367315 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa20c0052e0 msgr2=0x7fa20c00fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:59.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.681+0000 7fa229e00640 1 --2- 172.21.15.152:0/536367315 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa20c0052e0 0x7fa20c00fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:59.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.681+0000 7fa229e00640 1 -- 172.21.15.152:0/536367315 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa20c003520 con 0x7fa20c004800 2024-04-20T11:27:59.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.681+0000 7fa22b200640 1 -- 172.21.15.152:0/536367315 reap_dead start 2024-04-20T11:27:59.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.682+0000 7fa229e00640 1 --2- 172.21.15.152:0/536367315 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa20c004800 0x7fa20c004c80 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fa214002230 tx=0x7fa20c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee193413b8f2ce80 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:59.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.683+0000 7fa229400640 1 -- 172.21.15.152:0/536367315 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa21400a4a0 con 0x7fa20c004800 2024-04-20T11:27:59.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.683+0000 7fa229400640 1 -- 172.21.15.152:0/536367315 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa214034650 con 0x7fa20c004800 2024-04-20T11:27:59.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.683+0000 7fa229400640 1 -- 172.21.15.152:0/536367315 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa21402ce60 con 0x7fa20c004800 2024-04-20T11:27:59.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.683+0000 7fa230a00640 1 -- 172.21.15.152:0/536367315 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa20c004800 msgr2=0x7fa20c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:59.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.683+0000 7fa230a00640 1 --2- 172.21.15.152:0/536367315 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa20c004800 0x7fa20c004c80 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fa214002230 tx=0x7fa20c005830 comp rx=0 tx=0).stop 2024-04-20T11:27:59.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.683+0000 7fa230a00640 1 -- 172.21.15.152:0/536367315 shutdown_connections 2024-04-20T11:27:59.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.683+0000 7fa230a00640 1 --2- 172.21.15.152:0/536367315 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa20c004800 0x7fa20c004c80 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:59.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.683+0000 7fa230a00640 1 -- 172.21.15.152:0/536367315 >> 172.21.15.152:0/536367315 conn(0x7fa22c0b3e60 msgr2=0x7fa22c0b4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:59.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.683+0000 7fa230a00640 1 -- 172.21.15.152:0/536367315 shutdown_connections 2024-04-20T11:27:59.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.683+0000 7fa230a00640 1 -- 172.21.15.152:0/536367315 wait complete. 2024-04-20T11:27:59.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.685+0000 7fa230a00640 1 Processor -- start 2024-04-20T11:27:59.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.685+0000 7fa230a00640 1 -- start start 2024-04-20T11:27:59.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.686+0000 7fa230a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa20c004800 0x7fa22c0cbd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:59.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.686+0000 7fa230a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa22c15a210 0x7fa22c0b70f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:59.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.686+0000 7fa230a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa22c0b7630 0x7fa22c1fbfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:59.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.686+0000 7fa230a00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fa22c0d0950 con 0x7fa22c0b7630 2024-04-20T11:27:59.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.686+0000 7fa230a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa22c0d12a0 con 0x7fa20c004800 2024-04-20T11:27:59.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.686+0000 7fa230a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa22c0d0ad0 con 0x7fa22c15a210 2024-04-20T11:27:59.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.686+0000 7fa22b200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa22c0b7630 0x7fa22c1fbfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:59.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.687+0000 7fa22b200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa22c0b7630 0x7fa22c1fbfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:32940/0 (socket says 172.21.15.152:32940) 2024-04-20T11:27:59.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.687+0000 7fa22b200640 1 -- 172.21.15.152:0/3384912310 learned_addr learned my addr 172.21.15.152:0/3384912310 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:27:59.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.687+0000 7fa22a800640 1 --2- 172.21.15.152:0/3384912310 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa20c004800 0x7fa22c0cbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:59.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.687+0000 7fa229e00640 1 --2- 172.21.15.152:0/3384912310 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa22c15a210 0x7fa22c0b70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:59.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.687+0000 7fa22b200640 1 -- 172.21.15.152:0/3384912310 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa22c15a210 msgr2=0x7fa22c0b70f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:59.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.687+0000 7fa22b200640 1 --2- 172.21.15.152:0/3384912310 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa22c15a210 0x7fa22c0b70f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:59.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.687+0000 7fa22b200640 1 -- 172.21.15.152:0/3384912310 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa20c004800 msgr2=0x7fa22c0cbd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:59.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.687+0000 7fa22b200640 1 --2- 172.21.15.152:0/3384912310 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa20c004800 0x7fa22c0cbd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:59.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.687+0000 7fa22b200640 1 -- 172.21.15.152:0/3384912310 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa22c0cc290 con 0x7fa22c0b7630 2024-04-20T11:27:59.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.687+0000 7fa229e00640 1 --2- 172.21.15.152:0/3384912310 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa22c15a210 0x7fa22c0b70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:27:59.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.687+0000 7fa22b200640 1 --2- 172.21.15.152:0/3384912310 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa22c0b7630 0x7fa22c1fbfb0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fa22c0cc950 tx=0x7fa22400ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:59.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.687+0000 7fa222a00640 1 -- 172.21.15.152:0/3384912310 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa224017020 con 0x7fa22c0b7630 2024-04-20T11:27:59.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.687+0000 7fa222a00640 1 -- 172.21.15.152:0/3384912310 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa224004030 con 0x7fa22c0b7630 2024-04-20T11:27:59.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.687+0000 7fa230a00640 1 -- 172.21.15.152:0/3384912310 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa22c0b61a0 con 0x7fa22c0b7630 2024-04-20T11:27:59.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.688+0000 7fa222a00640 1 -- 172.21.15.152:0/3384912310 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa224004420 con 0x7fa22c0b7630 2024-04-20T11:27:59.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.688+0000 7fa230a00640 1 -- 172.21.15.152:0/3384912310 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa22c0b6760 con 0x7fa22c0b7630 2024-04-20T11:27:59.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.689+0000 7fa230a00640 1 -- 172.21.15.152:0/3384912310 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa22c075e60 con 0x7fa22c0b7630 2024-04-20T11:27:59.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.690+0000 7fa222a00640 1 -- 172.21.15.152:0/3384912310 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa22401a020 con 0x7fa22c0b7630 2024-04-20T11:27:59.691 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.691+0000 7fa222a00640 1 --2- 172.21.15.152:0/3384912310 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fa20806c8f0 0x7fa20806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:59.691 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.691+0000 7fa222a00640 1 -- 172.21.15.152:0/3384912310 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fa22408b720 con 0x7fa22c0b7630 2024-04-20T11:27:59.692 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.691+0000 7fa22a800640 1 --2- 172.21.15.152:0/3384912310 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fa20806c8f0 0x7fa20806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:59.692 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.692+0000 7fa22a800640 1 --2- 172.21.15.152:0/3384912310 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fa20806c8f0 0x7fa20806edb0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa22c0c93f0 tx=0x7fa21c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:59.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.695+0000 7fa222a00640 1 -- 172.21.15.152:0/3384912310 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fa224021070 con 0x7fa22c0b7630 2024-04-20T11:27:59.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.895+0000 7fa230a00640 1 -- 172.21.15.152:0/3384912310 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa22c0b5240 con 0x7fa22c0b7630 2024-04-20T11:27:59.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.896+0000 7fa222a00640 1 -- 172.21.15.152:0/3384912310 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) ==== 104+0+11 (secure 0 0 0) 0x7fa22c0cc290 con 0x7fa22c0b7630 2024-04-20T11:27:59.896 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 32 2024-04-20T11:27:59.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.898+0000 7fa230a00640 1 -- 172.21.15.152:0/3384912310 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fa20806c8f0 msgr2=0x7fa20806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:59.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.898+0000 7fa230a00640 1 --2- 172.21.15.152:0/3384912310 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fa20806c8f0 0x7fa20806edb0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa22c0c93f0 tx=0x7fa21c009040 comp rx=0 tx=0).stop 2024-04-20T11:27:59.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.898+0000 7fa230a00640 1 -- 172.21.15.152:0/3384912310 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa22c0b7630 msgr2=0x7fa22c1fbfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:59.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.898+0000 7fa230a00640 1 --2- 172.21.15.152:0/3384912310 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa22c0b7630 0x7fa22c1fbfb0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fa22c0cc950 tx=0x7fa22400ec00 comp rx=0 tx=0).stop 2024-04-20T11:27:59.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.898+0000 7fa230a00640 1 -- 172.21.15.152:0/3384912310 shutdown_connections 2024-04-20T11:27:59.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.898+0000 7fa230a00640 1 --2- 172.21.15.152:0/3384912310 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fa20806c8f0 0x7fa20806edb0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:59.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.898+0000 7fa230a00640 1 --2- 172.21.15.152:0/3384912310 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa22c0b7630 0x7fa22c1fbfb0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:59.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.898+0000 7fa230a00640 1 --2- 172.21.15.152:0/3384912310 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa22c15a210 0x7fa22c0b70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:59.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.898+0000 7fa230a00640 1 --2- 172.21.15.152:0/3384912310 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa20c004800 0x7fa22c0cbd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:59.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.898+0000 7fa230a00640 1 -- 172.21.15.152:0/3384912310 >> 172.21.15.152:0/3384912310 conn(0x7fa22c0b3e60 msgr2=0x7fa22c0d0090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:59.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.899+0000 7fa230a00640 1 -- 172.21.15.152:0/3384912310 shutdown_connections 2024-04-20T11:27:59.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:27:59.899+0000 7fa230a00640 1 -- 172.21.15.152:0/3384912310 wait complete. 2024-04-20T11:27:59.919 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T11:28:00.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.116+0000 7f904d800640 1 Processor -- start 2024-04-20T11:28:00.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.117+0000 7f904d800640 1 -- start start 2024-04-20T11:28:00.118 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.118+0000 7f904d800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9048167640 0x7f9048169a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:00.118 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.118+0000 7f904d800640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90480ced60 con 0x7f90481622c0 2024-04-20T11:28:00.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.118+0000 7f904d800640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90480cf040 con 0x7f90480cfe00 2024-04-20T11:28:00.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.118+0000 7f904d800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f9048163170 con 0x7f9048167640 2024-04-20T11:28:00.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.118+0000 7f9047e00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f90481622c0 0x7f90481626a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:36098/0 (socket says 172.21.15.152:36098) 2024-04-20T11:28:00.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.118+0000 7f9047e00640 1 -- 172.21.15.152:0/3515850347 learned_addr learned my addr 172.21.15.152:0/3515850347 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:28:00.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.118+0000 7f904ce00640 1 --2- 172.21.15.152:0/3515850347 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9048167640 0x7f9048169a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:00.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.118+0000 7f9046a00640 1 -- 172.21.15.152:0/3515850347 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 784203062 0 0) 0x7f90480ced60 con 0x7f90481622c0 2024-04-20T11:28:00.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.118+0000 7f9046a00640 1 -- 172.21.15.152:0/3515850347 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f902c003520 con 0x7f90481622c0 2024-04-20T11:28:00.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.119+0000 7f9046a00640 1 -- 172.21.15.152:0/3515850347 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1845820146 0 0) 0x7f902c003520 con 0x7f90481622c0 2024-04-20T11:28:00.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.119+0000 7f9046a00640 1 -- 172.21.15.152:0/3515850347 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9048167640 msgr2=0x7f9048169a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:00.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.119+0000 7f9046a00640 1 --2- 172.21.15.152:0/3515850347 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9048167640 0x7f9048169a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:00.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.119+0000 7f9046a00640 1 -- 172.21.15.152:0/3515850347 >> v1:172.21.15.175:6789/0 conn(0x7f90480cfe00 legacy=0x7f90480d31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:00.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.119+0000 7f9046a00640 1 -- 172.21.15.152:0/3515850347 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90480cf320 con 0x7f90481622c0 2024-04-20T11:28:00.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.119+0000 7f9046a00640 1 -- 172.21.15.152:0/3515850347 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f90380019f0 con 0x7f90481622c0 2024-04-20T11:28:00.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.119+0000 7f9046a00640 1 -- 172.21.15.152:0/3515850347 >> v1:172.21.15.152:6789/0 conn(0x7f90481622c0 legacy=0x7f90481626a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:00.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.119+0000 7f9046a00640 1 --2- 172.21.15.152:0/3515850347 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9048167640 0x7f902c0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:00.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.119+0000 7f9046a00640 1 --2- 172.21.15.152:0/3515850347 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f902c003fb0 0x7f902c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:00.122 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.119+0000 7f9046a00640 1 --2- 172.21.15.152:0/3515850347 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f902c004d40 0x7f902c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:00.122 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.119+0000 7f9046a00640 1 -- 172.21.15.152:0/3515850347 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f902c00bad0 con 0x7f902c003fb0 2024-04-20T11:28:00.122 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.119+0000 7f9046a00640 1 -- 172.21.15.152:0/3515850347 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f902c00bc80 con 0x7f902c004d40 2024-04-20T11:28:00.122 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.119+0000 7f9046a00640 1 -- 172.21.15.152:0/3515850347 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f902c00be80 con 0x7f9048167640 2024-04-20T11:28:00.122 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.119+0000 7f9047400640 1 --2- 172.21.15.152:0/3515850347 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f902c003fb0 0x7f902c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:00.122 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.120+0000 7f904ce00640 1 --2- 172.21.15.152:0/3515850347 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f902c004d40 0x7f902c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:00.122 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.120+0000 7f9047400640 1 -- 172.21.15.152:0/3515850347 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9048167640 msgr2=0x7f902c0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:00.122 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.120+0000 7f9047400640 1 --2- 172.21.15.152:0/3515850347 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9048167640 0x7f902c0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:00.122 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.120+0000 7f9047400640 1 -- 172.21.15.152:0/3515850347 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f902c004d40 msgr2=0x7f902c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:00.122 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.120+0000 7f9047400640 1 --2- 172.21.15.152:0/3515850347 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f902c004d40 0x7f902c0051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:00.122 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.120+0000 7f9047400640 1 -- 172.21.15.152:0/3515850347 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f902c003520 con 0x7f902c003fb0 2024-04-20T11:28:00.124 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.121+0000 7f9047400640 1 --2- 172.21.15.152:0/3515850347 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f902c003fb0 0x7f902c004800 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f90481610b0 tx=0x7f902c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d2f26b2d0709165 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:00.124 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.121+0000 7f9046a00640 1 -- 172.21.15.152:0/3515850347 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90340013c0 con 0x7f902c003fb0 2024-04-20T11:28:00.124 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.121+0000 7f9046a00640 1 -- 172.21.15.152:0/3515850347 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f903402be50 con 0x7f902c003fb0 2024-04-20T11:28:00.124 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.121+0000 7f9046a00640 1 -- 172.21.15.152:0/3515850347 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9034003640 con 0x7f902c003fb0 2024-04-20T11:28:00.124 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.121+0000 7f904d800640 1 -- 172.21.15.152:0/3515850347 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f902c003fb0 msgr2=0x7f902c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:00.124 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.121+0000 7f904d800640 1 --2- 172.21.15.152:0/3515850347 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f902c003fb0 0x7f902c004800 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f90481610b0 tx=0x7f902c005830 comp rx=0 tx=0).stop 2024-04-20T11:28:00.124 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.122+0000 7f904ce00640 1 -- 172.21.15.152:0/3515850347 reap_dead start 2024-04-20T11:28:00.124 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.122+0000 7f904d800640 1 -- 172.21.15.152:0/3515850347 shutdown_connections 2024-04-20T11:28:00.125 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.122+0000 7f904d800640 1 -- 172.21.15.152:0/3515850347 >> 172.21.15.152:0/3515850347 conn(0x7f904808bfd0 msgr2=0x7f90480d4570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:00.125 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.122+0000 7f904d800640 1 -- 172.21.15.152:0/3515850347 shutdown_connections 2024-04-20T11:28:00.125 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.122+0000 7f904d800640 1 -- 172.21.15.152:0/3515850347 wait complete. 2024-04-20T11:28:00.125 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.123+0000 7f904d800640 1 Processor -- start 2024-04-20T11:28:00.125 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.123+0000 7f904d800640 1 -- start start 2024-04-20T11:28:00.127 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.124+0000 7f904d800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f902c003fb0 0x7f904814ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:00.127 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.124+0000 7f904d800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f90480cca50 0x7f90480cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:00.128 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.124+0000 7f904d800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f904815a020 0x7f904815a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:00.128 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.124+0000 7f904d800640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f904815f9c0 con 0x7f902c003fb0 2024-04-20T11:28:00.128 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.124+0000 7f904d800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f90480cf970 con 0x7f904815a020 2024-04-20T11:28:00.128 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.124+0000 7f904d800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f904815fb40 con 0x7f90480cca50 2024-04-20T11:28:00.128 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.125+0000 7f9047e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f902c003fb0 0x7f904814ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:00.128 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.125+0000 7f9047e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f902c003fb0 0x7f904814ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:32946/0 (socket says 172.21.15.152:32946) 2024-04-20T11:28:00.128 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.125+0000 7f9047e00640 1 -- 172.21.15.152:0/310889530 learned_addr learned my addr 172.21.15.152:0/310889530 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:00.128 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.125+0000 7f9047400640 1 --2- 172.21.15.152:0/310889530 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f90480cca50 0x7f90480cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:00.128 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.125+0000 7f904ce00640 1 --2- 172.21.15.152:0/310889530 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f904815a020 0x7f904815a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:00.128 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.125+0000 7f9047e00640 1 -- 172.21.15.152:0/310889530 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f90480cca50 msgr2=0x7f90480cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:00.128 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.125+0000 7f9047e00640 1 --2- 172.21.15.152:0/310889530 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f90480cca50 0x7f90480cceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:00.128 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.125+0000 7f9047e00640 1 -- 172.21.15.152:0/310889530 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f904815a020 msgr2=0x7f904815a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:00.129 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.125+0000 7f9047e00640 1 --2- 172.21.15.152:0/310889530 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f904815a020 0x7f904815a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:00.129 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.125+0000 7f9047e00640 1 -- 172.21.15.152:0/310889530 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f904814ef60 con 0x7f902c003fb0 2024-04-20T11:28:00.129 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.125+0000 7f9047400640 1 --2- 172.21.15.152:0/310889530 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f90480cca50 0x7f90480cceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:28:00.129 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.125+0000 7f9047e00640 1 --2- 172.21.15.152:0/310889530 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f902c003fb0 0x7f904814ea20 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f903800cb00 tx=0x7f9038001b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:00.129 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.126+0000 7f903fe00640 1 -- 172.21.15.152:0/310889530 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9038018070 con 0x7f902c003fb0 2024-04-20T11:28:00.129 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.126+0000 7f903fe00640 1 -- 172.21.15.152:0/310889530 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f903800a760 con 0x7f902c003fb0 2024-04-20T11:28:00.129 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.126+0000 7f904d800640 1 -- 172.21.15.152:0/310889530 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90480cd520 con 0x7f902c003fb0 2024-04-20T11:28:00.129 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.126+0000 7f903fe00640 1 -- 172.21.15.152:0/310889530 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f903800ab50 con 0x7f902c003fb0 2024-04-20T11:28:00.129 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.126+0000 7f904d800640 1 -- 172.21.15.152:0/310889530 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90480cdae0 con 0x7f902c003fb0 2024-04-20T11:28:00.129 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.127+0000 7f904d800640 1 -- 172.21.15.152:0/310889530 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9048075e60 con 0x7f902c003fb0 2024-04-20T11:28:00.130 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.128+0000 7f903fe00640 1 -- 172.21.15.152:0/310889530 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f903801b070 con 0x7f902c003fb0 2024-04-20T11:28:00.130 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.129+0000 7f903fe00640 1 --2- 172.21.15.152:0/310889530 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f902806c8f0 0x7f902806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:00.130 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.129+0000 7f903fe00640 1 -- 172.21.15.152:0/310889530 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f903808b400 con 0x7f902c003fb0 2024-04-20T11:28:00.130 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.129+0000 7f9047400640 1 --2- 172.21.15.152:0/310889530 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f902806c8f0 0x7f902806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:00.130 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.130+0000 7f9047400640 1 --2- 172.21.15.152:0/310889530 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f902806c8f0 0x7f902806edb0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9034033000 tx=0x7f9034009030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:00.133 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.133+0000 7f903fe00640 1 -- 172.21.15.152:0/310889530 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f904814ef60 con 0x7f902c003fb0 2024-04-20T11:28:00.333 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.333+0000 7f904d800640 1 -- 172.21.15.152:0/310889530 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f90480d40d0 con 0x7f902c003fb0 2024-04-20T11:28:00.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.333+0000 7f903fe00640 1 -- 172.21.15.152:0/310889530 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) ==== 100+0+11 (secure 0 0 0) 0x7f9038059710 con 0x7f902c003fb0 2024-04-20T11:28:00.334 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 64 2024-04-20T11:28:00.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.336+0000 7f904d800640 1 -- 172.21.15.152:0/310889530 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f902806c8f0 msgr2=0x7f902806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:00.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.336+0000 7f904d800640 1 --2- 172.21.15.152:0/310889530 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f902806c8f0 0x7f902806edb0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9034033000 tx=0x7f9034009030 comp rx=0 tx=0).stop 2024-04-20T11:28:00.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.336+0000 7f904d800640 1 -- 172.21.15.152:0/310889530 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f902c003fb0 msgr2=0x7f904814ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:00.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.336+0000 7f904d800640 1 --2- 172.21.15.152:0/310889530 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f902c003fb0 0x7f904814ea20 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f903800cb00 tx=0x7f9038001b50 comp rx=0 tx=0).stop 2024-04-20T11:28:00.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.336+0000 7f904d800640 1 -- 172.21.15.152:0/310889530 shutdown_connections 2024-04-20T11:28:00.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.336+0000 7f904d800640 1 --2- 172.21.15.152:0/310889530 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f902806c8f0 0x7f902806edb0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:00.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.336+0000 7f904d800640 1 --2- 172.21.15.152:0/310889530 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f904815a020 0x7f904815a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:00.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.336+0000 7f904d800640 1 --2- 172.21.15.152:0/310889530 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f90480cca50 0x7f90480cceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:00.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.336+0000 7f904d800640 1 --2- 172.21.15.152:0/310889530 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f902c003fb0 0x7f904814ea20 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:00.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.336+0000 7f904d800640 1 -- 172.21.15.152:0/310889530 >> 172.21.15.152:0/310889530 conn(0x7f904808bfd0 msgr2=0x7f9048161220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:00.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.336+0000 7f904d800640 1 -- 172.21.15.152:0/310889530 shutdown_connections 2024-04-20T11:28:00.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.336+0000 7f904d800640 1 -- 172.21.15.152:0/310889530 wait complete. 2024-04-20T11:28:00.355 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-20T11:28:00.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.550+0000 7f4bee200640 1 Processor -- start 2024-04-20T11:28:00.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.551+0000 7f4bee200640 1 -- start start 2024-04-20T11:28:00.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.552+0000 7f4bee200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4be8161d30 0x7f4be80d0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:00.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.552+0000 7f4bee200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f4be80d2b80 0x7f4be80d3030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:00.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.552+0000 7f4bee200640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4be80d26a0 con 0x7f4be80d4ec0 2024-04-20T11:28:00.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.552+0000 7f4bee200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f4be80d0540 con 0x7f4be80d2b80 2024-04-20T11:28:00.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.552+0000 7f4bee200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4be80cfb70 con 0x7f4be8161d30 2024-04-20T11:28:00.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.553+0000 7f4be7e00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f4be80d4ec0 0x7f4be80d15e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:36100/0 (socket says 172.21.15.152:36100) 2024-04-20T11:28:00.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.553+0000 7f4bed800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f4be80d2b80 0x7f4be80d3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:00.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.553+0000 7f4be7e00640 1 -- 172.21.15.152:0/3260772266 learned_addr learned my addr 172.21.15.152:0/3260772266 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:28:00.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.553+0000 7f4bece00640 1 --2- 172.21.15.152:0/3260772266 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4be8161d30 0x7f4be80d0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:00.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.553+0000 7f4bed800640 1 -- 172.21.15.152:0/3260772266 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4be8161d30 msgr2=0x7f4be80d0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:00.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.553+0000 7f4bed800640 1 --2- 172.21.15.152:0/3260772266 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4be8161d30 0x7f4be80d0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:00.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.553+0000 7f4bed800640 1 -- 172.21.15.152:0/3260772266 >> v1:172.21.15.152:6789/0 conn(0x7f4be80d4ec0 legacy=0x7f4be80d15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:00.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.553+0000 7f4be7400640 1 -- 172.21.15.152:0/3260772266 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2511882065 0 0) 0x7f4be80d26a0 con 0x7f4be80d4ec0 2024-04-20T11:28:00.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.553+0000 7f4bed800640 1 -- 172.21.15.152:0/3260772266 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4be80d53c0 con 0x7f4be80d2b80 2024-04-20T11:28:00.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.554+0000 7f4bed800640 1 --2- 172.21.15.152:0/3260772266 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4be80d2b80 0x7f4be80d3030 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f4be00097a0 tx=0x7f4be002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edb7c788c23e4cee server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:00.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.555+0000 7f4be7400640 1 -- 172.21.15.152:0/3260772266 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4be0035020 con 0x7f4be80d2b80 2024-04-20T11:28:00.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.555+0000 7f4be7400640 1 -- 172.21.15.152:0/3260772266 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4be0031830 con 0x7f4be80d2b80 2024-04-20T11:28:00.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.555+0000 7f4be7400640 1 -- 172.21.15.152:0/3260772266 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4be0031c00 con 0x7f4be80d2b80 2024-04-20T11:28:00.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.555+0000 7f4bee200640 1 -- 172.21.15.152:0/3260772266 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4be80d2b80 msgr2=0x7f4be80d3030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:00.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.555+0000 7f4bee200640 1 --2- 172.21.15.152:0/3260772266 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4be80d2b80 0x7f4be80d3030 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f4be00097a0 tx=0x7f4be002b8d0 comp rx=0 tx=0).stop 2024-04-20T11:28:00.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.555+0000 7f4bee200640 1 -- 172.21.15.152:0/3260772266 shutdown_connections 2024-04-20T11:28:00.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.555+0000 7f4bee200640 1 --2- 172.21.15.152:0/3260772266 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4be80d2b80 0x7f4be80d3030 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:00.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.555+0000 7f4bee200640 1 --2- 172.21.15.152:0/3260772266 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4be8161d30 0x7f4be80d0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:00.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.555+0000 7f4bee200640 1 -- 172.21.15.152:0/3260772266 >> 172.21.15.152:0/3260772266 conn(0x7f4be808bfd0 msgr2=0x7f4be80d45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:00.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.555+0000 7f4bee200640 1 -- 172.21.15.152:0/3260772266 shutdown_connections 2024-04-20T11:28:00.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.555+0000 7f4bee200640 1 -- 172.21.15.152:0/3260772266 wait complete. 2024-04-20T11:28:00.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.557+0000 7f4bee200640 1 Processor -- start 2024-04-20T11:28:00.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.557+0000 7f4bee200640 1 -- start start 2024-04-20T11:28:00.559 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.558+0000 7f4bee200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4be80d2b80 0x7f4be81583d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:00.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.558+0000 7f4bee200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4be80d4ec0 0x7f4be814d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:00.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.558+0000 7f4bee200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4be8161d30 0x7f4be815d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:00.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.558+0000 7f4bee200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f4be80cf1b0 con 0x7f4be80d4ec0 2024-04-20T11:28:00.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.558+0000 7f4bee200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4be80cf800 con 0x7f4be8161d30 2024-04-20T11:28:00.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.558+0000 7f4bee200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4be8162110 con 0x7f4be80d2b80 2024-04-20T11:28:00.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.558+0000 7f4be7e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4be80d4ec0 0x7f4be814d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:00.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.559+0000 7f4be7e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4be80d4ec0 0x7f4be814d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:32972/0 (socket says 172.21.15.152:32972) 2024-04-20T11:28:00.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.559+0000 7f4be7e00640 1 -- 172.21.15.152:0/1263353305 learned_addr learned my addr 172.21.15.152:0/1263353305 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:00.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.559+0000 7f4bece00640 1 --2- 172.21.15.152:0/1263353305 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4be80d2b80 0x7f4be81583d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:00.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.559+0000 7f4bed800640 1 --2- 172.21.15.152:0/1263353305 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4be8161d30 0x7f4be815d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:00.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.559+0000 7f4be7e00640 1 -- 172.21.15.152:0/1263353305 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4be80d2b80 msgr2=0x7f4be81583d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:00.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.559+0000 7f4be7e00640 1 --2- 172.21.15.152:0/1263353305 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4be80d2b80 0x7f4be81583d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:00.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.559+0000 7f4be7e00640 1 -- 172.21.15.152:0/1263353305 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4be8161d30 msgr2=0x7f4be815d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:00.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.559+0000 7f4be7e00640 1 --2- 172.21.15.152:0/1263353305 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4be8161d30 0x7f4be815d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:00.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.559+0000 7f4be7e00640 1 -- 172.21.15.152:0/1263353305 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4be8158a90 con 0x7f4be80d4ec0 2024-04-20T11:28:00.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.559+0000 7f4bece00640 1 --2- 172.21.15.152:0/1263353305 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4be80d2b80 0x7f4be81583d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:28:00.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.559+0000 7f4be7e00640 1 --2- 172.21.15.152:0/1263353305 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4be80d4ec0 0x7f4be814d9d0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f4bd000c760 tx=0x7f4bd000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:00.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.560+0000 7f4be4c00640 1 -- 172.21.15.152:0/1263353305 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd0007270 con 0x7f4be80d4ec0 2024-04-20T11:28:00.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.560+0000 7f4be4c00640 1 -- 172.21.15.152:0/1263353305 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bd0007410 con 0x7f4be80d4ec0 2024-04-20T11:28:00.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.560+0000 7f4bee200640 1 -- 172.21.15.152:0/1263353305 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4be8158d70 con 0x7f4be80d4ec0 2024-04-20T11:28:00.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.560+0000 7f4be4c00640 1 -- 172.21.15.152:0/1263353305 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd0001d00 con 0x7f4be80d4ec0 2024-04-20T11:28:00.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.560+0000 7f4bee200640 1 -- 172.21.15.152:0/1263353305 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4be8159330 con 0x7f4be80d4ec0 2024-04-20T11:28:00.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.561+0000 7f4bee200640 1 -- 172.21.15.152:0/1263353305 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4be8075e60 con 0x7f4be80d4ec0 2024-04-20T11:28:00.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.562+0000 7f4be4c00640 1 -- 172.21.15.152:0/1263353305 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4bd00057a0 con 0x7f4be80d4ec0 2024-04-20T11:28:00.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.563+0000 7f4be4c00640 1 --2- 172.21.15.152:0/1263353305 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f4bc406c8f0 0x7f4bc406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:00.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.563+0000 7f4be4c00640 1 -- 172.21.15.152:0/1263353305 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f4bd008ba30 con 0x7f4be80d4ec0 2024-04-20T11:28:00.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.563+0000 7f4bece00640 1 --2- 172.21.15.152:0/1263353305 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f4bc406c8f0 0x7f4bc406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:00.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.564+0000 7f4bece00640 1 --2- 172.21.15.152:0/1263353305 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f4bc406c8f0 0x7f4bc406edb0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4bd8007330 tx=0x7f4bd800b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:00.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.567+0000 7f4be4c00640 1 -- 172.21.15.152:0/1263353305 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f4bd0059cb0 con 0x7f4be80d4ec0 2024-04-20T11:28:00.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.779+0000 7f4bee200640 1 -- 172.21.15.152:0/1263353305 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f4be815fc60 con 0x7f4be80d4ec0 2024-04-20T11:28:00.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.781+0000 7f4be4c00640 1 -- 172.21.15.152:0/1263353305 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) ==== 80+0+0 (secure 0 0 0) 0x7f4bd00595b0 con 0x7f4be80d4ec0 2024-04-20T11:28:00.781 INFO:teuthology.orchestra.run.smithi152.stderr:listed 0 entries 2024-04-20T11:28:00.784 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.783+0000 7f4bee200640 1 -- 172.21.15.152:0/1263353305 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f4bc406c8f0 msgr2=0x7f4bc406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:00.784 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.783+0000 7f4bee200640 1 --2- 172.21.15.152:0/1263353305 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f4bc406c8f0 0x7f4bc406edb0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4bd8007330 tx=0x7f4bd800b040 comp rx=0 tx=0).stop 2024-04-20T11:28:00.784 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.783+0000 7f4bee200640 1 -- 172.21.15.152:0/1263353305 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4be80d4ec0 msgr2=0x7f4be814d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:00.784 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.783+0000 7f4bee200640 1 --2- 172.21.15.152:0/1263353305 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4be80d4ec0 0x7f4be814d9d0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f4bd000c760 tx=0x7f4bd000cc20 comp rx=0 tx=0).stop 2024-04-20T11:28:00.784 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.783+0000 7f4bee200640 1 -- 172.21.15.152:0/1263353305 shutdown_connections 2024-04-20T11:28:00.784 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.783+0000 7f4bee200640 1 --2- 172.21.15.152:0/1263353305 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f4bc406c8f0 0x7f4bc406edb0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:00.784 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.783+0000 7f4bee200640 1 --2- 172.21.15.152:0/1263353305 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4be8161d30 0x7f4be815d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:00.785 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.783+0000 7f4bee200640 1 --2- 172.21.15.152:0/1263353305 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4be80d4ec0 0x7f4be814d9d0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:00.785 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.783+0000 7f4bee200640 1 --2- 172.21.15.152:0/1263353305 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4be80d2b80 0x7f4be81583d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:00.785 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.783+0000 7f4bee200640 1 -- 172.21.15.152:0/1263353305 >> 172.21.15.152:0/1263353305 conn(0x7f4be808bfd0 msgr2=0x7f4be8160fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:00.785 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.784+0000 7f4bee200640 1 -- 172.21.15.152:0/1263353305 shutdown_connections 2024-04-20T11:28:00.785 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:00.784+0000 7f4bee200640 1 -- 172.21.15.152:0/1263353305 wait complete. 2024-04-20T11:28:00.803 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi175.front.sepia.ceph.com... 2024-04-20T11:28:00.803 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:28:00.803 DEBUG:teuthology.orchestra.run.smithi175:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2024-04-20T11:28:00.894 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T11:28:00.895 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi175.front.sepia.ceph.com 2024-04-20T11:28:00.895 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:28:00.895 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-20T11:28:00.949 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-20T11:28:00.949 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:28:01.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.155+0000 7efc16200640 1 Processor -- start 2024-04-20T11:28:01.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.155+0000 7efc16200640 1 -- start start 2024-04-20T11:28:01.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.156+0000 7efc16200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7efc1015b5e0 0x7efc1006eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:01.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.156+0000 7efc16200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efc100b4c50 0x7efc100b50d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:01.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.156+0000 7efc16200640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc100b5770 con 0x7efc1006f340 2024-04-20T11:28:01.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.156+0000 7efc16200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7efc1006de10 con 0x7efc1015b5e0 2024-04-20T11:28:01.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.156+0000 7efc16200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7efc1006e060 con 0x7efc100b4c50 2024-04-20T11:28:01.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.157+0000 7efc0fe00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7efc1006f340 0x7efc1015cd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:36108/0 (socket says 172.21.15.152:36108) 2024-04-20T11:28:01.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.157+0000 7efc0f400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7efc1015b5e0 0x7efc1006eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:01.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.157+0000 7efc0fe00640 1 -- 172.21.15.152:0/1055786284 learned_addr learned my addr 172.21.15.152:0/1055786284 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:28:01.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.157+0000 7efc06a00640 1 --2- 172.21.15.152:0/1055786284 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efc100b4c50 0x7efc100b50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:01.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.157+0000 7efc0f400640 1 -- 172.21.15.152:0/1055786284 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efc100b4c50 msgr2=0x7efc100b50d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:01.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.157+0000 7efc0f400640 1 --2- 172.21.15.152:0/1055786284 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efc100b4c50 0x7efc100b50d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:01.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.157+0000 7efc0ea00640 1 -- 172.21.15.152:0/1055786284 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 254059920 0 0) 0x7efc100b5770 con 0x7efc1006f340 2024-04-20T11:28:01.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.157+0000 7efc0f400640 1 -- 172.21.15.152:0/1055786284 >> v1:172.21.15.152:6789/0 conn(0x7efc1006f340 legacy=0x7efc1015cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:01.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.157+0000 7efc0f400640 1 -- 172.21.15.152:0/1055786284 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc1006e340 con 0x7efc1015b5e0 2024-04-20T11:28:01.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.158+0000 7efc0f400640 1 --2- 172.21.15.152:0/1055786284 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efc1015b5e0 0x7efc1006eb40 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7efc000097a0 tx=0x7efc0002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1aa19e0666c1fd3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:01.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.159+0000 7efc0ea00640 1 -- 172.21.15.152:0/1055786284 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc00035020 con 0x7efc1015b5e0 2024-04-20T11:28:01.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.159+0000 7efc0ea00640 1 -- 172.21.15.152:0/1055786284 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc00031830 con 0x7efc1015b5e0 2024-04-20T11:28:01.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.159+0000 7efc0ea00640 1 -- 172.21.15.152:0/1055786284 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc00031c00 con 0x7efc1015b5e0 2024-04-20T11:28:01.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.159+0000 7efc16200640 1 -- 172.21.15.152:0/1055786284 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efc1015b5e0 msgr2=0x7efc1006eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:01.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.159+0000 7efc16200640 1 --2- 172.21.15.152:0/1055786284 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efc1015b5e0 0x7efc1006eb40 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7efc000097a0 tx=0x7efc0002b8d0 comp rx=0 tx=0).stop 2024-04-20T11:28:01.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.160+0000 7efc16200640 1 -- 172.21.15.152:0/1055786284 shutdown_connections 2024-04-20T11:28:01.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.160+0000 7efc16200640 1 --2- 172.21.15.152:0/1055786284 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efc100b4c50 0x7efc100b50d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:01.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.160+0000 7efc16200640 1 --2- 172.21.15.152:0/1055786284 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efc1015b5e0 0x7efc1006eb40 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:01.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.160+0000 7efc16200640 1 -- 172.21.15.152:0/1055786284 >> 172.21.15.152:0/1055786284 conn(0x7efc10067cf0 msgr2=0x7efc1006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:01.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.160+0000 7efc16200640 1 -- 172.21.15.152:0/1055786284 shutdown_connections 2024-04-20T11:28:01.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.160+0000 7efc16200640 1 -- 172.21.15.152:0/1055786284 wait complete. 2024-04-20T11:28:01.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.161+0000 7efc16200640 1 Processor -- start 2024-04-20T11:28:01.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.162+0000 7efc16200640 1 -- start start 2024-04-20T11:28:01.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.162+0000 7efc16200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efc1006f340 0x7efc10148390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:01.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.162+0000 7efc16200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efc100b4c50 0x7efc101495e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:01.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.162+0000 7efc16200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efc1015b5e0 0x7efc100cf030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:01.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.163+0000 7efc16200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7efc100b42d0 con 0x7efc100b4c50 2024-04-20T11:28:01.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.163+0000 7efc16200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7efc1015c9d0 con 0x7efc1015b5e0 2024-04-20T11:28:01.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.163+0000 7efc16200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7efc1006f790 con 0x7efc1006f340 2024-04-20T11:28:01.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.163+0000 7efc06a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efc100b4c50 0x7efc101495e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:01.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.163+0000 7efc06a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efc100b4c50 0x7efc101495e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:32990/0 (socket says 172.21.15.152:32990) 2024-04-20T11:28:01.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.163+0000 7efc06a00640 1 -- 172.21.15.152:0/3821889993 learned_addr learned my addr 172.21.15.152:0/3821889993 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:01.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.163+0000 7efc0f400640 1 --2- 172.21.15.152:0/3821889993 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efc1006f340 0x7efc10148390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:01.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.163+0000 7efc0fe00640 1 --2- 172.21.15.152:0/3821889993 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efc1015b5e0 0x7efc100cf030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:01.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.163+0000 7efc06a00640 1 -- 172.21.15.152:0/3821889993 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efc1006f340 msgr2=0x7efc10148390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:01.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.163+0000 7efc06a00640 1 --2- 172.21.15.152:0/3821889993 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efc1006f340 0x7efc10148390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:01.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.163+0000 7efc06a00640 1 -- 172.21.15.152:0/3821889993 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efc1015b5e0 msgr2=0x7efc100cf030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:01.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.163+0000 7efc06a00640 1 --2- 172.21.15.152:0/3821889993 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efc1015b5e0 0x7efc100cf030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:01.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.163+0000 7efc06a00640 1 -- 172.21.15.152:0/3821889993 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc100ce270 con 0x7efc100b4c50 2024-04-20T11:28:01.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.163+0000 7efc0fe00640 1 --2- 172.21.15.152:0/3821889993 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efc1015b5e0 0x7efc100cf030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:28:01.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.164+0000 7efc06a00640 1 --2- 172.21.15.152:0/3821889993 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efc100b4c50 0x7efc101495e0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7efbf800d830 tx=0x7efbf800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:01.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.164+0000 7efc07e00640 1 -- 172.21.15.152:0/3821889993 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf800b6c0 con 0x7efc100b4c50 2024-04-20T11:28:01.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.164+0000 7efc07e00640 1 -- 172.21.15.152:0/3821889993 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efbf8019070 con 0x7efc100b4c50 2024-04-20T11:28:01.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.164+0000 7efc07e00640 1 -- 172.21.15.152:0/3821889993 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf800ba80 con 0x7efc100b4c50 2024-04-20T11:28:01.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.164+0000 7efc16200640 1 -- 172.21.15.152:0/3821889993 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc100ce550 con 0x7efc100b4c50 2024-04-20T11:28:01.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.164+0000 7efc16200640 1 -- 172.21.15.152:0/3821889993 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc101488d0 con 0x7efc100b4c50 2024-04-20T11:28:01.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.165+0000 7efc16200640 1 -- 172.21.15.152:0/3821889993 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc100689f0 con 0x7efc100b4c50 2024-04-20T11:28:01.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.166+0000 7efc07e00640 1 -- 172.21.15.152:0/3821889993 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efbf801e460 con 0x7efc100b4c50 2024-04-20T11:28:01.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.167+0000 7efc07e00640 1 --2- 172.21.15.152:0/3821889993 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7efbec06c920 0x7efbec06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:01.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.167+0000 7efc07e00640 1 -- 172.21.15.152:0/3821889993 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7efbf808c1f0 con 0x7efc100b4c50 2024-04-20T11:28:01.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.167+0000 7efc0f400640 1 --2- 172.21.15.152:0/3821889993 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7efbec06c920 0x7efbec06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:01.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.168+0000 7efc0f400640 1 --2- 172.21.15.152:0/3821889993 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7efbec06c920 0x7efbec06ede0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7efc00036000 tx=0x7efc00008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:01.172 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.172+0000 7efc07e00640 1 -- 172.21.15.152:0/3821889993 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7efbf805a630 con 0x7efc100b4c50 2024-04-20T11:28:01.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.414+0000 7efc16200640 1 -- 172.21.15.152:0/3821889993 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efc10152bb0 con 0x7efc100b4c50 2024-04-20T11:28:01.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.415+0000 7efc07e00640 1 -- 172.21.15.152:0/3821889993 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5196 (secure 0 0 0) 0x7efbf8059f30 con 0x7efc100b4c50 2024-04-20T11:28:01.416 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:28:01.416 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/853233806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":853233806},{"type":"v1","addr":"172.21.15.175:6833","nonce":853233806}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4253,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/100300237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":100300237},{"type":"v1","addr":"172.21.15.175:6835","nonce":100300237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/2924034058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":2924034058},{"type":"v1","addr":"172.21.15.152:6835","nonce":2924034058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T11:26:58.132293+0000","modified":"2024-04-20T11:27:01.444285+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4269":{"gid":4269,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6837/927639144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":927639144},{"type":"v1","addr":"172.21.15.152:6837","nonce":927639144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4269,"qdb_cluster":[4269]},"id":1}]} 2024-04-20T11:28:01.417 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 7 2024-04-20T11:28:01.419 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.418+0000 7efc16200640 1 -- 172.21.15.152:0/3821889993 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7efbec06c920 msgr2=0x7efbec06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:01.419 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.418+0000 7efc16200640 1 --2- 172.21.15.152:0/3821889993 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7efbec06c920 0x7efbec06ede0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7efc00036000 tx=0x7efc00008030 comp rx=0 tx=0).stop 2024-04-20T11:28:01.419 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.418+0000 7efc16200640 1 -- 172.21.15.152:0/3821889993 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efc100b4c50 msgr2=0x7efc101495e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:01.419 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.418+0000 7efc16200640 1 --2- 172.21.15.152:0/3821889993 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efc100b4c50 0x7efc101495e0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7efbf800d830 tx=0x7efbf800dcf0 comp rx=0 tx=0).stop 2024-04-20T11:28:01.419 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.418+0000 7efc16200640 1 -- 172.21.15.152:0/3821889993 shutdown_connections 2024-04-20T11:28:01.419 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.418+0000 7efc16200640 1 --2- 172.21.15.152:0/3821889993 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7efbec06c920 0x7efbec06ede0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:01.419 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.418+0000 7efc16200640 1 --2- 172.21.15.152:0/3821889993 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efc1015b5e0 0x7efc100cf030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:01.419 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.418+0000 7efc16200640 1 --2- 172.21.15.152:0/3821889993 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efc100b4c50 0x7efc101495e0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:01.419 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.418+0000 7efc16200640 1 --2- 172.21.15.152:0/3821889993 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efc1006f340 0x7efc10148390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:01.419 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.418+0000 7efc16200640 1 -- 172.21.15.152:0/3821889993 >> 172.21.15.152:0/3821889993 conn(0x7efc10067cf0 msgr2=0x7efc100b2150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:01.419 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.418+0000 7efc16200640 1 -- 172.21.15.152:0/3821889993 shutdown_connections 2024-04-20T11:28:01.420 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.418+0000 7efc16200640 1 -- 172.21.15.152:0/3821889993 wait complete. 2024-04-20T11:28:01.438 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:28:01.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.641+0000 7f8523e00640 1 Processor -- start 2024-04-20T11:28:01.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.641+0000 7f8523e00640 1 -- start start 2024-04-20T11:28:01.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.643+0000 7f8523e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f851c0d0810 0x7f851c1693f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:01.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.643+0000 7f8523e00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f851c15fd20 con 0x7f851c0b4500 2024-04-20T11:28:01.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.643+0000 7f8523e00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f851c15ef60 con 0x7f851c15c560 2024-04-20T11:28:01.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.643+0000 7f8523e00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f851c15f240 con 0x7f851c0d0810 2024-04-20T11:28:01.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.643+0000 7f8520a00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f851c0b4500 0x7f851c0d0100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:36114/0 (socket says 172.21.15.152:36114) 2024-04-20T11:28:01.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.643+0000 7f8520a00640 1 -- 172.21.15.152:0/584724137 learned_addr learned my addr 172.21.15.152:0/584724137 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:28:01.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.644+0000 7f8521e00640 1 --2- 172.21.15.152:0/584724137 >> v2:172.21.15.175:3300/0 conn(0x7f851c0d0810 0x7f851c1693f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:01.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.644+0000 7f851be00640 1 -- 172.21.15.152:0/584724137 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4190833 0 0) 0x7f851c15fd20 con 0x7f851c0b4500 2024-04-20T11:28:01.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.644+0000 7f851be00640 1 -- 172.21.15.152:0/584724137 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8504003520 con 0x7f851c0b4500 2024-04-20T11:28:01.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.644+0000 7f851be00640 1 -- 172.21.15.152:0/584724137 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1623183459 0 0) 0x7f8504003520 con 0x7f851c0b4500 2024-04-20T11:28:01.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.644+0000 7f851be00640 1 -- 172.21.15.152:0/584724137 >> v2:172.21.15.175:3300/0 conn(0x7f851c0d0810 msgr2=0x7f851c1693f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:01.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.644+0000 7f851be00640 1 --2- 172.21.15.152:0/584724137 >> v2:172.21.15.175:3300/0 conn(0x7f851c0d0810 0x7f851c1693f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:01.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.644+0000 7f851be00640 1 -- 172.21.15.152:0/584724137 >> v1:172.21.15.175:6789/0 conn(0x7f851c15c560 legacy=0x7f851c0b3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:01.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.644+0000 7f851be00640 1 -- 172.21.15.152:0/584724137 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f851c15f520 con 0x7f851c0b4500 2024-04-20T11:28:01.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.644+0000 7f851be00640 1 -- 172.21.15.152:0/584724137 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f8508001670 con 0x7f851c0b4500 2024-04-20T11:28:01.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.644+0000 7f851be00640 1 -- 172.21.15.152:0/584724137 >> v1:172.21.15.152:6789/0 conn(0x7f851c0b4500 legacy=0x7f851c0d0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:01.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.644+0000 7f851be00640 1 --2- 172.21.15.152:0/584724137 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8504003a20 0x7f8504003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:01.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.644+0000 7f851be00640 1 --2- 172.21.15.152:0/584724137 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8504004800 0x7f8504004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:01.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.644+0000 7f851be00640 1 --2- 172.21.15.152:0/584724137 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8504005230 0x7f850400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:01.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.644+0000 7f851be00640 1 -- 172.21.15.152:0/584724137 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f851c15f240 con 0x7f8504003a20 2024-04-20T11:28:01.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.644+0000 7f851be00640 1 -- 172.21.15.152:0/584724137 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f850400c190 con 0x7f8504004800 2024-04-20T11:28:01.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.644+0000 7f851be00640 1 -- 172.21.15.152:0/584724137 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f850400c390 con 0x7f8504005230 2024-04-20T11:28:01.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.645+0000 7f8521400640 1 --2- 172.21.15.152:0/584724137 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8504003a20 0x7f8504003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:01.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.645+0000 7f8520a00640 1 --2- 172.21.15.152:0/584724137 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8504004800 0x7f8504004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:01.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.645+0000 7f8521e00640 1 --2- 172.21.15.152:0/584724137 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8504005230 0x7f850400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:01.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.645+0000 7f8521400640 1 -- 172.21.15.152:0/584724137 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8504005230 msgr2=0x7f850400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:01.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.645+0000 7f8521400640 1 --2- 172.21.15.152:0/584724137 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8504005230 0x7f850400b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:01.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.645+0000 7f8521400640 1 -- 172.21.15.152:0/584724137 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8504004800 msgr2=0x7f8504004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:01.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.645+0000 7f8521400640 1 --2- 172.21.15.152:0/584724137 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8504004800 0x7f8504004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:01.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.645+0000 7f8521400640 1 -- 172.21.15.152:0/584724137 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8504003520 con 0x7f8504003a20 2024-04-20T11:28:01.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.645+0000 7f8520a00640 1 --2- 172.21.15.152:0/584724137 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8504004800 0x7f8504004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:28:01.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.645+0000 7f8521e00640 1 -- 172.21.15.152:0/584724137 reap_dead start 2024-04-20T11:28:01.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.646+0000 7f8521400640 1 --2- 172.21.15.152:0/584724137 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8504003a20 0x7f8504003e40 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f8510001750 tx=0x7f8504005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7d7cc41d1eac25 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:01.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.647+0000 7f851be00640 1 -- 172.21.15.152:0/584724137 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8510001370 con 0x7f8504003a20 2024-04-20T11:28:01.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.647+0000 7f851be00640 1 -- 172.21.15.152:0/584724137 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f851002be50 con 0x7f8504003a20 2024-04-20T11:28:01.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.647+0000 7f851be00640 1 -- 172.21.15.152:0/584724137 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85100035f0 con 0x7f8504003a20 2024-04-20T11:28:01.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.647+0000 7f8523e00640 1 -- 172.21.15.152:0/584724137 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8504003a20 msgr2=0x7f8504003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:01.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.647+0000 7f8523e00640 1 --2- 172.21.15.152:0/584724137 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8504003a20 0x7f8504003e40 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f8510001750 tx=0x7f8504005830 comp rx=0 tx=0).stop 2024-04-20T11:28:01.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.647+0000 7f8523e00640 1 -- 172.21.15.152:0/584724137 shutdown_connections 2024-04-20T11:28:01.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.647+0000 7f8523e00640 1 --2- 172.21.15.152:0/584724137 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8504003a20 0x7f8504003e40 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:01.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.647+0000 7f8523e00640 1 -- 172.21.15.152:0/584724137 >> 172.21.15.152:0/584724137 conn(0x7f851c067cf0 msgr2=0x7f851c06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:01.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.647+0000 7f8523e00640 1 -- 172.21.15.152:0/584724137 shutdown_connections 2024-04-20T11:28:01.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.648+0000 7f8523e00640 1 -- 172.21.15.152:0/584724137 wait complete. 2024-04-20T11:28:01.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.649+0000 7f8523e00640 1 Processor -- start 2024-04-20T11:28:01.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.649+0000 7f8523e00640 1 -- start start 2024-04-20T11:28:01.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.650+0000 7f8523e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8504003a20 0x7f851c15c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:01.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.650+0000 7f8523e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f851c1f44e0 0x7f851c0ce6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:01.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.650+0000 7f8523e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f851c0cec10 0x7f851c156a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:01.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.650+0000 7f8523e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f851c0d0100 con 0x7f851c0cec10 2024-04-20T11:28:01.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.651+0000 7f8523e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f851c0d0dc0 con 0x7f8504003a20 2024-04-20T11:28:01.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.651+0000 7f8523e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f851c0d0280 con 0x7f851c1f44e0 2024-04-20T11:28:01.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.651+0000 7f8521e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f851c0cec10 0x7f851c156a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:01.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.651+0000 7f8521e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f851c0cec10 0x7f851c156a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:32998/0 (socket says 172.21.15.152:32998) 2024-04-20T11:28:01.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.651+0000 7f8521e00640 1 -- 172.21.15.152:0/2734890851 learned_addr learned my addr 172.21.15.152:0/2734890851 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:01.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.651+0000 7f8521400640 1 --2- 172.21.15.152:0/2734890851 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8504003a20 0x7f851c15c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:01.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.651+0000 7f8520a00640 1 --2- 172.21.15.152:0/2734890851 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f851c1f44e0 0x7f851c0ce6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:01.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.651+0000 7f8521e00640 1 -- 172.21.15.152:0/2734890851 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f851c1f44e0 msgr2=0x7f851c0ce6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:01.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.651+0000 7f8521e00640 1 --2- 172.21.15.152:0/2734890851 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f851c1f44e0 0x7f851c0ce6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:01.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.651+0000 7f8521e00640 1 -- 172.21.15.152:0/2734890851 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8504003a20 msgr2=0x7f851c15c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:01.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.651+0000 7f8521e00640 1 --2- 172.21.15.152:0/2734890851 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8504003a20 0x7f851c15c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:01.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.651+0000 7f8521e00640 1 -- 172.21.15.152:0/2734890851 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f851c0ccef0 con 0x7f851c0cec10 2024-04-20T11:28:01.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.652+0000 7f8521e00640 1 --2- 172.21.15.152:0/2734890851 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f851c0cec10 0x7f851c156a30 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f851c0b4a60 tx=0x7f851400f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:01.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.652+0000 7f8519600640 1 -- 172.21.15.152:0/2734890851 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8514016620 con 0x7f851c0cec10 2024-04-20T11:28:01.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.652+0000 7f8519600640 1 -- 172.21.15.152:0/2734890851 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f851401b070 con 0x7f851c0cec10 2024-04-20T11:28:01.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.652+0000 7f8523e00640 1 -- 172.21.15.152:0/2734890851 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f851c15ce80 con 0x7f851c0cec10 2024-04-20T11:28:01.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.652+0000 7f8519600640 1 -- 172.21.15.152:0/2734890851 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8514016a20 con 0x7f851c0cec10 2024-04-20T11:28:01.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.652+0000 7f8523e00640 1 -- 172.21.15.152:0/2734890851 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f851c1f4d80 con 0x7f851c0cec10 2024-04-20T11:28:01.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.653+0000 7f8523e00640 1 -- 172.21.15.152:0/2734890851 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f851c0689f0 con 0x7f851c0cec10 2024-04-20T11:28:01.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.654+0000 7f8519600640 1 -- 172.21.15.152:0/2734890851 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8514005d10 con 0x7f851c0cec10 2024-04-20T11:28:01.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.655+0000 7f8519600640 1 --2- 172.21.15.152:0/2734890851 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f84fc06c8f0 0x7f84fc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:01.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.655+0000 7f8519600640 1 -- 172.21.15.152:0/2734890851 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f851408c600 con 0x7f851c0cec10 2024-04-20T11:28:01.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.655+0000 7f8521400640 1 --2- 172.21.15.152:0/2734890851 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f84fc06c8f0 0x7f84fc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:01.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.656+0000 7f8521400640 1 --2- 172.21.15.152:0/2734890851 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f84fc06c8f0 0x7f84fc06edb0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8510033000 tx=0x7f8510009030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:01.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.660+0000 7f8519600640 1 -- 172.21.15.152:0/2734890851 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f851405a910 con 0x7f851c0cec10 2024-04-20T11:28:01.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.867+0000 7f8523e00640 1 -- 172.21.15.152:0/2734890851 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f851c0cc9d0 con 0x7f851c0cec10 2024-04-20T11:28:01.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.870+0000 7f8519600640 1 -- 172.21.15.152:0/2734890851 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16914 (secure 0 0 0) 0x7f851405a210 con 0x7f851c0cec10 2024-04-20T11:28:01.871 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:28:01.871 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":23,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:27:42.601620+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:26:56.261424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T11:26:57.079286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":2}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.26","mappings":[{"from":7,"to":1}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:28:01.873 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.872+0000 7f8523e00640 1 -- 172.21.15.152:0/2734890851 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f84fc06c8f0 msgr2=0x7f84fc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:01.873 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.872+0000 7f8523e00640 1 --2- 172.21.15.152:0/2734890851 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f84fc06c8f0 0x7f84fc06edb0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8510033000 tx=0x7f8510009030 comp rx=0 tx=0).stop 2024-04-20T11:28:01.873 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.872+0000 7f8523e00640 1 -- 172.21.15.152:0/2734890851 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f851c0cec10 msgr2=0x7f851c156a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:01.873 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.872+0000 7f8523e00640 1 --2- 172.21.15.152:0/2734890851 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f851c0cec10 0x7f851c156a30 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f851c0b4a60 tx=0x7f851400f520 comp rx=0 tx=0).stop 2024-04-20T11:28:01.873 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.872+0000 7f8523e00640 1 -- 172.21.15.152:0/2734890851 shutdown_connections 2024-04-20T11:28:01.873 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.872+0000 7f8523e00640 1 --2- 172.21.15.152:0/2734890851 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f84fc06c8f0 0x7f84fc06edb0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:01.873 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.872+0000 7f8523e00640 1 --2- 172.21.15.152:0/2734890851 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f851c0cec10 0x7f851c156a30 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:01.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.872+0000 7f8523e00640 1 --2- 172.21.15.152:0/2734890851 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f851c1f44e0 0x7f851c0ce6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:01.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.872+0000 7f8523e00640 1 --2- 172.21.15.152:0/2734890851 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8504003a20 0x7f851c15c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:01.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.872+0000 7f8523e00640 1 -- 172.21.15.152:0/2734890851 >> 172.21.15.152:0/2734890851 conn(0x7f851c067cf0 msgr2=0x7f851c159670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:01.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.873+0000 7f8523e00640 1 -- 172.21.15.152:0/2734890851 shutdown_connections 2024-04-20T11:28:01.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:01.873+0000 7f8523e00640 1 -- 172.21.15.152:0/2734890851 wait complete. 2024-04-20T11:28:01.893 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T11:26:47.246571+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T11:26:56.261424+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T11:26:57.079286+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T11:28:01.894 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T11:28:02.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.104+0000 7f451fa00640 1 Processor -- start 2024-04-20T11:28:02.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.104+0000 7f451fa00640 1 -- start start 2024-04-20T11:28:02.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.106+0000 7f451fa00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f45180b2880 0x7f45180b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:02.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.106+0000 7f451fa00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45181463e0 con 0x7f45180cd1b0 2024-04-20T11:28:02.107 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.106+0000 7f451fa00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45180b3e20 con 0x7f4518146000 2024-04-20T11:28:02.107 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.106+0000 7f451fa00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f45180b4750 con 0x7f45180b2880 2024-04-20T11:28:02.109 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.106+0000 7f451da00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f45180cd1b0 0x7f45180cd5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:36124/0 (socket says 172.21.15.152:36124) 2024-04-20T11:28:02.109 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.106+0000 7f451da00640 1 -- 172.21.15.152:0/3120797273 learned_addr learned my addr 172.21.15.152:0/3120797273 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:28:02.109 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.107+0000 7f4517e00640 1 --2- 172.21.15.152:0/3120797273 >> v2:172.21.15.175:3300/0 conn(0x7f45180b2880 0x7f45180b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:02.109 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.107+0000 7f4517400640 1 -- 172.21.15.152:0/3120797273 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3026025153 0 0) 0x7f45181463e0 con 0x7f45180cd1b0 2024-04-20T11:28:02.109 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.107+0000 7f4517400640 1 -- 172.21.15.152:0/3120797273 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f44f8003520 con 0x7f45180cd1b0 2024-04-20T11:28:02.109 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.107+0000 7f4517400640 1 -- 172.21.15.152:0/3120797273 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1934692790 0 0) 0x7f44f8003520 con 0x7f45180cd1b0 2024-04-20T11:28:02.109 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.107+0000 7f4517400640 1 -- 172.21.15.152:0/3120797273 >> v2:172.21.15.175:3300/0 conn(0x7f45180b2880 msgr2=0x7f45180b4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:02.109 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.107+0000 7f4517400640 1 --2- 172.21.15.152:0/3120797273 >> v2:172.21.15.175:3300/0 conn(0x7f45180b2880 0x7f45180b4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:02.109 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.107+0000 7f4517400640 1 -- 172.21.15.152:0/3120797273 >> v1:172.21.15.175:6789/0 conn(0x7f4518146000 legacy=0x7f45180b2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:02.109 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.107+0000 7f4517400640 1 -- 172.21.15.152:0/3120797273 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45180b4100 con 0x7f45180cd1b0 2024-04-20T11:28:02.109 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.107+0000 7f4517400640 1 -- 172.21.15.152:0/3120797273 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f45100019f0 con 0x7f45180cd1b0 2024-04-20T11:28:02.109 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.107+0000 7f4517400640 1 -- 172.21.15.152:0/3120797273 >> v1:172.21.15.152:6789/0 conn(0x7f45180cd1b0 legacy=0x7f45180cd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:02.110 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.107+0000 7f4517400640 1 --2- 172.21.15.152:0/3120797273 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f44f80039b0 0x7f44f8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:02.110 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.108+0000 7f4517400640 1 --2- 172.21.15.152:0/3120797273 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44f8004800 0x7f44f8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:02.110 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.108+0000 7f4517400640 1 --2- 172.21.15.152:0/3120797273 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f44f80051c0 0x7f44f800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:02.110 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.108+0000 7f4517400640 1 -- 172.21.15.152:0/3120797273 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f45180b4750 con 0x7f44f80039b0 2024-04-20T11:28:02.110 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.108+0000 7f4517400640 1 -- 172.21.15.152:0/3120797273 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f44f800c160 con 0x7f44f8004800 2024-04-20T11:28:02.110 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.108+0000 7f4517400640 1 -- 172.21.15.152:0/3120797273 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f44f800c360 con 0x7f44f80051c0 2024-04-20T11:28:02.110 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.108+0000 7f451d000640 1 --2- 172.21.15.152:0/3120797273 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f44f80039b0 0x7f44f8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:02.110 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.108+0000 7f4517e00640 1 --2- 172.21.15.152:0/3120797273 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44f8004800 0x7f44f8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:02.110 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.108+0000 7f451d000640 1 -- 172.21.15.152:0/3120797273 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f44f80051c0 msgr2=0x7f44f800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:02.110 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.108+0000 7f451d000640 1 --2- 172.21.15.152:0/3120797273 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f44f80051c0 0x7f44f800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:02.110 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.108+0000 7f451d000640 1 -- 172.21.15.152:0/3120797273 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44f8004800 msgr2=0x7f44f8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:02.110 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.108+0000 7f451d000640 1 --2- 172.21.15.152:0/3120797273 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44f8004800 0x7f44f8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:02.110 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.108+0000 7f451d000640 1 -- 172.21.15.152:0/3120797273 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44f8003520 con 0x7f44f80039b0 2024-04-20T11:28:02.110 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.108+0000 7f451da00640 1 -- 172.21.15.152:0/3120797273 reap_dead start 2024-04-20T11:28:02.112 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.109+0000 7f451d000640 1 --2- 172.21.15.152:0/3120797273 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f44f80039b0 0x7f44f8003dd0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f4508001750 tx=0x7f44f8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d399e266e97d772 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:02.112 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.110+0000 7f4517400640 1 -- 172.21.15.152:0/3120797273 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4508001370 con 0x7f44f80039b0 2024-04-20T11:28:02.112 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.110+0000 7f4517400640 1 -- 172.21.15.152:0/3120797273 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45080052d0 con 0x7f44f80039b0 2024-04-20T11:28:02.112 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.110+0000 7f4517400640 1 -- 172.21.15.152:0/3120797273 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4508003830 con 0x7f44f80039b0 2024-04-20T11:28:02.112 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.110+0000 7f451fa00640 1 -- 172.21.15.152:0/3120797273 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f44f80039b0 msgr2=0x7f44f8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:02.112 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.110+0000 7f451fa00640 1 --2- 172.21.15.152:0/3120797273 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f44f80039b0 0x7f44f8003dd0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f4508001750 tx=0x7f44f8005830 comp rx=0 tx=0).stop 2024-04-20T11:28:02.112 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.110+0000 7f451fa00640 1 -- 172.21.15.152:0/3120797273 shutdown_connections 2024-04-20T11:28:02.112 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.110+0000 7f451fa00640 1 --2- 172.21.15.152:0/3120797273 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f44f80039b0 0x7f44f8003dd0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:02.112 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.110+0000 7f451fa00640 1 -- 172.21.15.152:0/3120797273 >> 172.21.15.152:0/3120797273 conn(0x7f451806b980 msgr2=0x7f451806bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:02.112 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.110+0000 7f451fa00640 1 -- 172.21.15.152:0/3120797273 shutdown_connections 2024-04-20T11:28:02.112 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.110+0000 7f451fa00640 1 -- 172.21.15.152:0/3120797273 wait complete. 2024-04-20T11:28:02.113 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.112+0000 7f451fa00640 1 Processor -- start 2024-04-20T11:28:02.113 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.112+0000 7f451fa00640 1 -- start start 2024-04-20T11:28:02.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.113+0000 7f451fa00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f44f80039b0 0x7f451806de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:02.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.113+0000 7f451fa00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f451815c460 0x7f45181e84b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:02.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.113+0000 7f451fa00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f451806e7d0 0x7f45181e89f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:02.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.113+0000 7f451fa00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f45180b4d60 con 0x7f451815c460 2024-04-20T11:28:02.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.113+0000 7f451fa00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f45180b4be0 con 0x7f451806e7d0 2024-04-20T11:28:02.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.113+0000 7f451fa00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f45180b4ee0 con 0x7f44f80039b0 2024-04-20T11:28:02.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.114+0000 7f4517e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f451815c460 0x7f45181e84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:02.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.114+0000 7f4517e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f451815c460 0x7f45181e84b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:33020/0 (socket says 172.21.15.152:33020) 2024-04-20T11:28:02.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.114+0000 7f4517e00640 1 -- 172.21.15.152:0/1366800850 learned_addr learned my addr 172.21.15.152:0/1366800850 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:02.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.114+0000 7f451d000640 1 --2- 172.21.15.152:0/1366800850 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f44f80039b0 0x7f451806de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:02.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.114+0000 7f451da00640 1 --2- 172.21.15.152:0/1366800850 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f451806e7d0 0x7f45181e89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:02.116 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.114+0000 7f4517e00640 1 -- 172.21.15.152:0/1366800850 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f44f80039b0 msgr2=0x7f451806de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:02.116 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.114+0000 7f4517e00640 1 --2- 172.21.15.152:0/1366800850 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f44f80039b0 0x7f451806de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:02.116 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.114+0000 7f4517e00640 1 -- 172.21.15.152:0/1366800850 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f451806e7d0 msgr2=0x7f45181e89f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:02.116 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.114+0000 7f4517e00640 1 --2- 172.21.15.152:0/1366800850 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f451806e7d0 0x7f45181e89f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:02.116 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.114+0000 7f4517e00640 1 -- 172.21.15.152:0/1366800850 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45181453a0 con 0x7f451815c460 2024-04-20T11:28:02.116 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.114+0000 7f4517e00640 1 --2- 172.21.15.152:0/1366800850 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f451815c460 0x7f45181e84b0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f45000115e0 tx=0x7f4500011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:02.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.115+0000 7f4514c00640 1 -- 172.21.15.152:0/1366800850 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f450000f8f0 con 0x7f451815c460 2024-04-20T11:28:02.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.115+0000 7f4514c00640 1 -- 172.21.15.152:0/1366800850 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f450000fa90 con 0x7f451815c460 2024-04-20T11:28:02.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.115+0000 7f4514c00640 1 -- 172.21.15.152:0/1366800850 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f450000fe40 con 0x7f451815c460 2024-04-20T11:28:02.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.115+0000 7f451fa00640 1 -- 172.21.15.152:0/1366800850 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f451806ed20 con 0x7f451815c460 2024-04-20T11:28:02.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.115+0000 7f451fa00640 1 -- 172.21.15.152:0/1366800850 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f451806f2e0 con 0x7f451815c460 2024-04-20T11:28:02.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.116+0000 7f451fa00640 1 -- 172.21.15.152:0/1366800850 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45180689f0 con 0x7f451815c460 2024-04-20T11:28:02.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.117+0000 7f4514c00640 1 -- 172.21.15.152:0/1366800850 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4500015460 con 0x7f451815c460 2024-04-20T11:28:02.118 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.118+0000 7f4514c00640 1 --2- 172.21.15.152:0/1366800850 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f44f406c8f0 0x7f44f406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:02.118 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.118+0000 7f4514c00640 1 -- 172.21.15.152:0/1366800850 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f450008c110 con 0x7f451815c460 2024-04-20T11:28:02.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.118+0000 7f451d000640 1 --2- 172.21.15.152:0/1366800850 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f44f406c8f0 0x7f44f406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:02.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.119+0000 7f451d000640 1 --2- 172.21.15.152:0/1366800850 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f44f406c8f0 0x7f44f406edb0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4508033000 tx=0x7f4508009030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:02.123 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.122+0000 7f4514c00640 1 -- 172.21.15.152:0/1366800850 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f45181453a0 con 0x7f451815c460 2024-04-20T11:28:02.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.328+0000 7f451fa00640 1 -- 172.21.15.152:0/1366800850 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f451806e5b0 con 0x7f451815c460 2024-04-20T11:28:02.329 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.328+0000 7f4514c00640 1 -- 172.21.15.152:0/1366800850 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) ==== 93+0+10 (secure 0 0 0) 0x7f4500019030 con 0x7f451815c460 2024-04-20T11:28:02.329 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 1 2024-04-20T11:28:02.332 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.331+0000 7f451fa00640 1 -- 172.21.15.152:0/1366800850 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f44f406c8f0 msgr2=0x7f44f406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:02.332 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.331+0000 7f451fa00640 1 --2- 172.21.15.152:0/1366800850 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f44f406c8f0 0x7f44f406edb0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4508033000 tx=0x7f4508009030 comp rx=0 tx=0).stop 2024-04-20T11:28:02.332 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.331+0000 7f451fa00640 1 -- 172.21.15.152:0/1366800850 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f451815c460 msgr2=0x7f45181e84b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:02.332 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.331+0000 7f451fa00640 1 --2- 172.21.15.152:0/1366800850 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f451815c460 0x7f45181e84b0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f45000115e0 tx=0x7f4500011aa0 comp rx=0 tx=0).stop 2024-04-20T11:28:02.332 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.331+0000 7f451fa00640 1 -- 172.21.15.152:0/1366800850 shutdown_connections 2024-04-20T11:28:02.332 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.331+0000 7f451fa00640 1 --2- 172.21.15.152:0/1366800850 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f44f406c8f0 0x7f44f406edb0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:02.332 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.331+0000 7f451fa00640 1 --2- 172.21.15.152:0/1366800850 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f451806e7d0 0x7f45181e89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:02.332 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.331+0000 7f451fa00640 1 --2- 172.21.15.152:0/1366800850 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f451815c460 0x7f45181e84b0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:02.332 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.331+0000 7f451fa00640 1 --2- 172.21.15.152:0/1366800850 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f44f80039b0 0x7f451806de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:02.332 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.331+0000 7f451fa00640 1 -- 172.21.15.152:0/1366800850 >> 172.21.15.152:0/1366800850 conn(0x7f451806b980 msgr2=0x7f45180d03f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:02.333 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.331+0000 7f451fa00640 1 -- 172.21.15.152:0/1366800850 shutdown_connections 2024-04-20T11:28:02.333 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.331+0000 7f451fa00640 1 -- 172.21.15.152:0/1366800850 wait complete. 2024-04-20T11:28:02.351 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T11:28:02.558 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.557+0000 7f0b77200640 1 Processor -- start 2024-04-20T11:28:02.558 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.557+0000 7f0b77200640 1 -- start start 2024-04-20T11:28:02.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.559+0000 7f0b77200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f0b78163230 0x7f0b780d2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:02.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.559+0000 7f0b77200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f0b780d33d0 0x7f0b780cf470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:02.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.559+0000 7f0b77200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b780d19c0 con 0x7f0b7815eb50 2024-04-20T11:28:02.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.559+0000 7f0b77200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f0b78162750 con 0x7f0b78163230 2024-04-20T11:28:02.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.559+0000 7f0b77200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f0b78162a30 con 0x7f0b780d33d0 2024-04-20T11:28:02.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.559+0000 7f0b6d400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f0b78163230 0x7f0b780d2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:02.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.559+0000 7f0b6d400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f0b78163230 0x7f0b780d2e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:33036/0 (socket says 172.21.15.152:33036) 2024-04-20T11:28:02.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.559+0000 7f0b6d400640 1 -- 172.21.15.152:0/4187490478 learned_addr learned my addr 172.21.15.152:0/4187490478 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:02.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.559+0000 7f0b76800640 1 --2- 172.21.15.152:0/4187490478 >> v2:172.21.15.175:3300/0 conn(0x7f0b780d33d0 0x7f0b780cf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:02.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.560+0000 7f0b6d400640 1 -- 172.21.15.152:0/4187490478 >> v2:172.21.15.175:3300/0 conn(0x7f0b780d33d0 msgr2=0x7f0b780cf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:02.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.560+0000 7f0b6d400640 1 --2- 172.21.15.152:0/4187490478 >> v2:172.21.15.175:3300/0 conn(0x7f0b780d33d0 0x7f0b780cf470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:02.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.560+0000 7f0b6d400640 1 -- 172.21.15.152:0/4187490478 >> v1:172.21.15.175:6789/0 conn(0x7f0b7815eb50 legacy=0x7f0b780d0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:02.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.560+0000 7f0b6d400640 1 -- 172.21.15.152:0/4187490478 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b78162d10 con 0x7f0b78163230 2024-04-20T11:28:02.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.560+0000 7f0b76800640 1 --2- 172.21.15.152:0/4187490478 >> v2:172.21.15.175:3300/0 conn(0x7f0b780d33d0 0x7f0b780cf470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:28:02.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.561+0000 7f0b6d400640 1 --2- 172.21.15.152:0/4187490478 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0b78163230 0x7f0b780d2e90 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f0b780d2e40 tx=0x7f0b6002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfcab6b4cc1ad4f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:02.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.561+0000 7f0b75400640 1 -- 172.21.15.152:0/4187490478 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b60037070 con 0x7f0b78163230 2024-04-20T11:28:02.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.561+0000 7f0b75400640 1 -- 172.21.15.152:0/4187490478 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b60004230 con 0x7f0b78163230 2024-04-20T11:28:02.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.561+0000 7f0b75400640 1 -- 172.21.15.152:0/4187490478 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b60003610 con 0x7f0b78163230 2024-04-20T11:28:02.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.562+0000 7f0b77200640 1 -- 172.21.15.152:0/4187490478 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0b78163230 msgr2=0x7f0b780d2e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:02.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.562+0000 7f0b77200640 1 --2- 172.21.15.152:0/4187490478 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0b78163230 0x7f0b780d2e90 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f0b780d2e40 tx=0x7f0b6002b860 comp rx=0 tx=0).stop 2024-04-20T11:28:02.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.562+0000 7f0b77200640 1 -- 172.21.15.152:0/4187490478 shutdown_connections 2024-04-20T11:28:02.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.562+0000 7f0b77200640 1 --2- 172.21.15.152:0/4187490478 >> v2:172.21.15.175:3300/0 conn(0x7f0b780d33d0 0x7f0b780cf470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:02.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.562+0000 7f0b77200640 1 --2- 172.21.15.152:0/4187490478 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0b78163230 0x7f0b780d2e90 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:02.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.562+0000 7f0b77200640 1 -- 172.21.15.152:0/4187490478 >> 172.21.15.152:0/4187490478 conn(0x7f0b7808bfd0 msgr2=0x7f0b780d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:02.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.562+0000 7f0b77200640 1 -- 172.21.15.152:0/4187490478 shutdown_connections 2024-04-20T11:28:02.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.562+0000 7f0b77200640 1 -- 172.21.15.152:0/4187490478 wait complete. 2024-04-20T11:28:02.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.564+0000 7f0b77200640 1 Processor -- start 2024-04-20T11:28:02.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.564+0000 7f0b77200640 1 -- start start 2024-04-20T11:28:02.567 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.565+0000 7f0b77200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0b780d33d0 0x7f0b7814ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:02.567 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.565+0000 7f0b77200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b7815eb50 0x7f0b7815f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:02.567 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.565+0000 7f0b77200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b78163230 0x7f0b781f4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:02.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.565+0000 7f0b77200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f0b780d1610 con 0x7f0b780d33d0 2024-04-20T11:28:02.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.565+0000 7f0b77200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0b780cf9b0 con 0x7f0b7815eb50 2024-04-20T11:28:02.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.565+0000 7f0b77200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0b780b5d60 con 0x7f0b78163230 2024-04-20T11:28:02.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.565+0000 7f0b75e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0b780d33d0 0x7f0b7814ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:02.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.565+0000 7f0b75e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0b780d33d0 0x7f0b7814ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:33052/0 (socket says 172.21.15.152:33052) 2024-04-20T11:28:02.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.565+0000 7f0b75e00640 1 -- 172.21.15.152:0/1038832041 learned_addr learned my addr 172.21.15.152:0/1038832041 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:02.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.566+0000 7f0b75e00640 1 -- 172.21.15.152:0/1038832041 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b78163230 msgr2=0x7f0b781f4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:02.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.566+0000 7f0b75e00640 1 --2- 172.21.15.152:0/1038832041 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b78163230 0x7f0b781f4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:02.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.566+0000 7f0b75e00640 1 -- 172.21.15.152:0/1038832041 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b7815eb50 msgr2=0x7f0b7815f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:02.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.566+0000 7f0b6d400640 1 --2- 172.21.15.152:0/1038832041 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b7815eb50 0x7f0b7815f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:02.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.566+0000 7f0b75e00640 1 --2- 172.21.15.152:0/1038832041 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b7815eb50 0x7f0b7815f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:02.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.566+0000 7f0b75e00640 1 -- 172.21.15.152:0/1038832041 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b780b5a30 con 0x7f0b780d33d0 2024-04-20T11:28:02.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.566+0000 7f0b75e00640 1 --2- 172.21.15.152:0/1038832041 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0b780d33d0 0x7f0b7814ede0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f0b6800c460 tx=0x7f0b6800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:02.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.566+0000 7f0b6ea00640 1 -- 172.21.15.152:0/1038832041 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b68016020 con 0x7f0b780d33d0 2024-04-20T11:28:02.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.566+0000 7f0b6ea00640 1 -- 172.21.15.152:0/1038832041 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b68012870 con 0x7f0b780d33d0 2024-04-20T11:28:02.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.566+0000 7f0b6ea00640 1 -- 172.21.15.152:0/1038832041 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b68012c60 con 0x7f0b780d33d0 2024-04-20T11:28:02.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.566+0000 7f0b77200640 1 -- 172.21.15.152:0/1038832041 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b780b72d0 con 0x7f0b780d33d0 2024-04-20T11:28:02.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.566+0000 7f0b77200640 1 -- 172.21.15.152:0/1038832041 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b781f4e40 con 0x7f0b780d33d0 2024-04-20T11:28:02.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.568+0000 7f0b77200640 1 -- 172.21.15.152:0/1038832041 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b78075e60 con 0x7f0b780d33d0 2024-04-20T11:28:02.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.568+0000 7f0b6ea00640 1 -- 172.21.15.152:0/1038832041 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0b68019020 con 0x7f0b780d33d0 2024-04-20T11:28:02.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.569+0000 7f0b6ea00640 1 --2- 172.21.15.152:0/1038832041 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f0b5406c8f0 0x7f0b5406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:02.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.569+0000 7f0b6ea00640 1 -- 172.21.15.152:0/1038832041 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f0b6808b6f0 con 0x7f0b780d33d0 2024-04-20T11:28:02.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.570+0000 7f0b6d400640 1 --2- 172.21.15.152:0/1038832041 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f0b5406c8f0 0x7f0b5406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:02.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.571+0000 7f0b6d400640 1 --2- 172.21.15.152:0/1038832041 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f0b5406c8f0 0x7f0b5406edb0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0b60034000 tx=0x7f0b60008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:02.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.576+0000 7f0b6ea00640 1 -- 172.21.15.152:0/1038832041 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f0b68020070 con 0x7f0b780d33d0 2024-04-20T11:28:02.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.781+0000 7f0b77200640 1 -- 172.21.15.152:0/1038832041 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f0b7808c680 con 0x7f0b780d33d0 2024-04-20T11:28:02.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.782+0000 7f0b6ea00640 1 -- 172.21.15.152:0/1038832041 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) ==== 104+0+11 (secure 0 0 0) 0x7f0b68059a00 con 0x7f0b780d33d0 2024-04-20T11:28:02.783 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 32 2024-04-20T11:28:02.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.785+0000 7f0b77200640 1 -- 172.21.15.152:0/1038832041 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f0b5406c8f0 msgr2=0x7f0b5406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:02.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.785+0000 7f0b77200640 1 --2- 172.21.15.152:0/1038832041 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f0b5406c8f0 0x7f0b5406edb0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0b60034000 tx=0x7f0b60008030 comp rx=0 tx=0).stop 2024-04-20T11:28:02.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.785+0000 7f0b77200640 1 -- 172.21.15.152:0/1038832041 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0b780d33d0 msgr2=0x7f0b7814ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:02.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.785+0000 7f0b77200640 1 --2- 172.21.15.152:0/1038832041 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0b780d33d0 0x7f0b7814ede0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f0b6800c460 tx=0x7f0b6800c920 comp rx=0 tx=0).stop 2024-04-20T11:28:02.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.785+0000 7f0b77200640 1 -- 172.21.15.152:0/1038832041 shutdown_connections 2024-04-20T11:28:02.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.785+0000 7f0b77200640 1 --2- 172.21.15.152:0/1038832041 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f0b5406c8f0 0x7f0b5406edb0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:02.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.785+0000 7f0b77200640 1 --2- 172.21.15.152:0/1038832041 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b78163230 0x7f0b781f4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:02.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.785+0000 7f0b77200640 1 --2- 172.21.15.152:0/1038832041 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b7815eb50 0x7f0b7815f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:02.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.785+0000 7f0b77200640 1 --2- 172.21.15.152:0/1038832041 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0b780d33d0 0x7f0b7814ede0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:02.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.785+0000 7f0b77200640 1 -- 172.21.15.152:0/1038832041 >> 172.21.15.152:0/1038832041 conn(0x7f0b7808bfd0 msgr2=0x7f0b7815caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:02.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.785+0000 7f0b77200640 1 -- 172.21.15.152:0/1038832041 shutdown_connections 2024-04-20T11:28:02.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:02.785+0000 7f0b77200640 1 -- 172.21.15.152:0/1038832041 wait complete. 2024-04-20T11:28:02.805 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T11:28:03.011 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.010+0000 7fd55b200640 1 Processor -- start 2024-04-20T11:28:03.011 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.010+0000 7fd55b200640 1 -- start start 2024-04-20T11:28:03.012 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.011+0000 7fd55b200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fd55c15ca40 0x7fd55c15cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:03.012 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.011+0000 7fd55b200640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd55c15d5c0 con 0x7fd55c06bda0 2024-04-20T11:28:03.012 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.011+0000 7fd55b200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd55c15d8a0 con 0x7fd55c156c00 2024-04-20T11:28:03.012 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.011+0000 7fd55b200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fd55c160290 con 0x7fd55c15ca40 2024-04-20T11:28:03.012 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.012+0000 7fd559400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fd55c15ca40 0x7fd55c15cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:03.013 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.012+0000 7fd55a800640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fd55c06bda0 0x7fd55c0b30c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:36126/0 (socket says 172.21.15.152:36126) 2024-04-20T11:28:03.013 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.012+0000 7fd55a800640 1 -- 172.21.15.152:0/2419041063 learned_addr learned my addr 172.21.15.152:0/2419041063 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:28:03.013 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.012+0000 7fd559400640 1 -- 172.21.15.152:0/2419041063 >> v1:172.21.15.175:6789/0 conn(0x7fd55c156c00 legacy=0x7fd55c160a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:03.013 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.012+0000 7fd559400640 1 -- 172.21.15.152:0/2419041063 >> v1:172.21.15.152:6789/0 conn(0x7fd55c06bda0 legacy=0x7fd55c0b30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:03.013 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.012+0000 7fd559400640 1 -- 172.21.15.152:0/2419041063 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd55c0ad2e0 con 0x7fd55c15ca40 2024-04-20T11:28:03.013 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.012+0000 7fd558a00640 1 -- 172.21.15.152:0/2419041063 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 545220901 0 0) 0x7fd55c15d5c0 con 0x7fd55c06bda0 2024-04-20T11:28:03.014 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.014+0000 7fd559400640 1 --2- 172.21.15.152:0/2419041063 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd55c15ca40 0x7fd55c15cec0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fd5480097a0 tx=0x7fd54802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71f344a3ec738089 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:03.014 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.014+0000 7fd558a00640 1 -- 172.21.15.152:0/2419041063 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd548035020 con 0x7fd55c15ca40 2024-04-20T11:28:03.014 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.014+0000 7fd558a00640 1 -- 172.21.15.152:0/2419041063 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd548031830 con 0x7fd55c15ca40 2024-04-20T11:28:03.014 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.014+0000 7fd558a00640 1 -- 172.21.15.152:0/2419041063 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd548031c00 con 0x7fd55c15ca40 2024-04-20T11:28:03.015 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.014+0000 7fd55b200640 1 -- 172.21.15.152:0/2419041063 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd55c15ca40 msgr2=0x7fd55c15cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:03.015 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.014+0000 7fd55b200640 1 --2- 172.21.15.152:0/2419041063 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd55c15ca40 0x7fd55c15cec0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fd5480097a0 tx=0x7fd54802b8d0 comp rx=0 tx=0).stop 2024-04-20T11:28:03.015 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.015+0000 7fd55b200640 1 -- 172.21.15.152:0/2419041063 shutdown_connections 2024-04-20T11:28:03.015 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.015+0000 7fd55b200640 1 --2- 172.21.15.152:0/2419041063 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd55c15ca40 0x7fd55c15cec0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:03.015 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.015+0000 7fd55b200640 1 -- 172.21.15.152:0/2419041063 >> 172.21.15.152:0/2419041063 conn(0x7fd55c0b21e0 msgr2=0x7fd55c0b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:03.015 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.015+0000 7fd55b200640 1 -- 172.21.15.152:0/2419041063 shutdown_connections 2024-04-20T11:28:03.016 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.015+0000 7fd55b200640 1 -- 172.21.15.152:0/2419041063 wait complete. 2024-04-20T11:28:03.017 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.016+0000 7fd55b200640 1 Processor -- start 2024-04-20T11:28:03.017 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.017+0000 7fd55b200640 1 -- start start 2024-04-20T11:28:03.019 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.017+0000 7fd55b200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd55c06bda0 0x7fd55c159ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:03.019 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.017+0000 7fd55b200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd55c156c00 0x7fd55c1f5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:03.019 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.017+0000 7fd55b200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd55c15ca40 0x7fd55c14c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:03.019 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.017+0000 7fd55b200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fd55c0b3b60 con 0x7fd55c15ca40 2024-04-20T11:28:03.019 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.017+0000 7fd55b200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd55c0b39e0 con 0x7fd55c06bda0 2024-04-20T11:28:03.019 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.017+0000 7fd55b200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd55c0b3ce0 con 0x7fd55c156c00 2024-04-20T11:28:03.019 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.018+0000 7fd55a800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd55c15ca40 0x7fd55c14c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:03.019 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.018+0000 7fd55a800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd55c15ca40 0x7fd55c14c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:33058/0 (socket says 172.21.15.152:33058) 2024-04-20T11:28:03.019 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.018+0000 7fd55a800640 1 -- 172.21.15.152:0/2647108949 learned_addr learned my addr 172.21.15.152:0/2647108949 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:03.019 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.018+0000 7fd559e00640 1 --2- 172.21.15.152:0/2647108949 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd55c06bda0 0x7fd55c159ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:03.019 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.018+0000 7fd55a800640 1 -- 172.21.15.152:0/2647108949 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd55c156c00 msgr2=0x7fd55c1f5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:03.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.018+0000 7fd55a800640 1 --2- 172.21.15.152:0/2647108949 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd55c156c00 0x7fd55c1f5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:03.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.018+0000 7fd55a800640 1 -- 172.21.15.152:0/2647108949 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd55c06bda0 msgr2=0x7fd55c159ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:03.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.018+0000 7fd55a800640 1 --2- 172.21.15.152:0/2647108949 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd55c06bda0 0x7fd55c159ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:03.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.018+0000 7fd55a800640 1 -- 172.21.15.152:0/2647108949 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd55c14fbe0 con 0x7fd55c15ca40 2024-04-20T11:28:03.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.018+0000 7fd55a800640 1 --2- 172.21.15.152:0/2647108949 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd55c15ca40 0x7fd55c14c150 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fd55400c7a0 tx=0x7fd55400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:03.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.019+0000 7fd552000640 1 -- 172.21.15.152:0/2647108949 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd554018070 con 0x7fd55c15ca40 2024-04-20T11:28:03.021 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.019+0000 7fd552000640 1 -- 172.21.15.152:0/2647108949 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd55400a640 con 0x7fd55c15ca40 2024-04-20T11:28:03.021 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.019+0000 7fd552000640 1 -- 172.21.15.152:0/2647108949 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd55400aab0 con 0x7fd55c15ca40 2024-04-20T11:28:03.021 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.019+0000 7fd55b200640 1 -- 172.21.15.152:0/2647108949 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd55c14fec0 con 0x7fd55c15ca40 2024-04-20T11:28:03.021 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.019+0000 7fd55b200640 1 -- 172.21.15.152:0/2647108949 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd55c150480 con 0x7fd55c15ca40 2024-04-20T11:28:03.021 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.020+0000 7fd55b200640 1 -- 172.21.15.152:0/2647108949 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd55c074ff0 con 0x7fd55c15ca40 2024-04-20T11:28:03.021 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.021+0000 7fd552000640 1 -- 172.21.15.152:0/2647108949 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd55400ac50 con 0x7fd55c15ca40 2024-04-20T11:28:03.022 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.022+0000 7fd552000640 1 --2- 172.21.15.152:0/2647108949 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fd53c06c8f0 0x7fd53c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:03.022 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.022+0000 7fd552000640 1 -- 172.21.15.152:0/2647108949 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fd55408b370 con 0x7fd55c15ca40 2024-04-20T11:28:03.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.022+0000 7fd559e00640 1 --2- 172.21.15.152:0/2647108949 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fd53c06c8f0 0x7fd53c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:03.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.023+0000 7fd559e00640 1 --2- 172.21.15.152:0/2647108949 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fd53c06c8f0 0x7fd53c06edb0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd54c004ea0 tx=0x7fd54c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:03.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.027+0000 7fd552000640 1 -- 172.21.15.152:0/2647108949 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fd55c074ff0 con 0x7fd55c15ca40 2024-04-20T11:28:03.224 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.224+0000 7fd55b200640 1 -- 172.21.15.152:0/2647108949 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd55c06e7f0 con 0x7fd55c15ca40 2024-04-20T11:28:03.225 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.224+0000 7fd552000640 1 -- 172.21.15.152:0/2647108949 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) ==== 100+0+11 (secure 0 0 0) 0x7fd5540595f0 con 0x7fd55c15ca40 2024-04-20T11:28:03.225 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 64 2024-04-20T11:28:03.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.227+0000 7fd55b200640 1 -- 172.21.15.152:0/2647108949 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fd53c06c8f0 msgr2=0x7fd53c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:03.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.227+0000 7fd55b200640 1 --2- 172.21.15.152:0/2647108949 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fd53c06c8f0 0x7fd53c06edb0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd54c004ea0 tx=0x7fd54c009040 comp rx=0 tx=0).stop 2024-04-20T11:28:03.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.227+0000 7fd55b200640 1 -- 172.21.15.152:0/2647108949 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd55c15ca40 msgr2=0x7fd55c14c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:03.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.227+0000 7fd55b200640 1 --2- 172.21.15.152:0/2647108949 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd55c15ca40 0x7fd55c14c150 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fd55400c7a0 tx=0x7fd55400cc60 comp rx=0 tx=0).stop 2024-04-20T11:28:03.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.227+0000 7fd55b200640 1 -- 172.21.15.152:0/2647108949 shutdown_connections 2024-04-20T11:28:03.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.227+0000 7fd55b200640 1 --2- 172.21.15.152:0/2647108949 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fd53c06c8f0 0x7fd53c06edb0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:03.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.227+0000 7fd55b200640 1 --2- 172.21.15.152:0/2647108949 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd55c15ca40 0x7fd55c14c150 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:03.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.227+0000 7fd55b200640 1 --2- 172.21.15.152:0/2647108949 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd55c156c00 0x7fd55c1f5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:03.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.227+0000 7fd55b200640 1 --2- 172.21.15.152:0/2647108949 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd55c06bda0 0x7fd55c159ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:03.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.227+0000 7fd55b200640 1 -- 172.21.15.152:0/2647108949 >> 172.21.15.152:0/2647108949 conn(0x7fd55c0b21e0 msgr2=0x7fd55c15a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:03.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.227+0000 7fd55b200640 1 -- 172.21.15.152:0/2647108949 shutdown_connections 2024-04-20T11:28:03.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.227+0000 7fd55b200640 1 -- 172.21.15.152:0/2647108949 wait complete. 2024-04-20T11:28:03.246 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:28:03.445 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.444+0000 7fead1a00640 1 Processor -- start 2024-04-20T11:28:03.445 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.444+0000 7fead1a00640 1 -- start start 2024-04-20T11:28:03.447 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.446+0000 7fead1a00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7feacc145f60 0x7feacc06ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:03.447 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.446+0000 7fead1a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feacc0726f0 0x7feacc0cd580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:03.447 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.446+0000 7fead1a00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feacc146340 con 0x7feacc0cd0d0 2024-04-20T11:28:03.447 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.446+0000 7fead1a00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7feacc06e9f0 con 0x7feacc145f60 2024-04-20T11:28:03.447 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.446+0000 7fead1a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7feacc06f3c0 con 0x7feacc0726f0 2024-04-20T11:28:03.448 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.447+0000 7feacb400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feacc0726f0 0x7feacc0cd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:03.448 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.447+0000 7feacaa00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7feacc145f60 0x7feacc06ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:03.448 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.447+0000 7feacb400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feacc0726f0 0x7feacc0cd580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.152:59442/0 (socket says 172.21.15.152:59442) 2024-04-20T11:28:03.448 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.447+0000 7feacb400640 1 -- 172.21.15.152:0/75092288 learned_addr learned my addr 172.21.15.152:0/75092288 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:03.448 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.448+0000 7feacaa00640 1 -- 172.21.15.152:0/75092288 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feacc0726f0 msgr2=0x7feacc0cd580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:03.449 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.448+0000 7feacaa00640 1 --2- 172.21.15.152:0/75092288 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feacc0726f0 0x7feacc0cd580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:03.449 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.448+0000 7feacaa00640 1 -- 172.21.15.152:0/75092288 >> v1:172.21.15.175:6789/0 conn(0x7feacc0cd0d0 legacy=0x7feacc071fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:03.449 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.448+0000 7feacaa00640 1 -- 172.21.15.152:0/75092288 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feacc072d50 con 0x7feacc145f60 2024-04-20T11:28:03.449 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.448+0000 7feacb400640 1 --2- 172.21.15.152:0/75092288 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feacc0726f0 0x7feacc0cd580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:28:03.450 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.450+0000 7feacaa00640 1 --2- 172.21.15.152:0/75092288 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feacc145f60 0x7feacc06ee80 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7feabc0097a0 tx=0x7feabc02b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=921dff999679d04b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:03.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.450+0000 7feaca000640 1 -- 172.21.15.152:0/75092288 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feabc035020 con 0x7feacc145f60 2024-04-20T11:28:03.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.450+0000 7feaca000640 1 -- 172.21.15.152:0/75092288 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feabc031830 con 0x7feacc145f60 2024-04-20T11:28:03.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.450+0000 7feaca000640 1 -- 172.21.15.152:0/75092288 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feabc031c00 con 0x7feacc145f60 2024-04-20T11:28:03.452 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.451+0000 7fead1a00640 1 -- 172.21.15.152:0/75092288 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feacc145f60 msgr2=0x7feacc06ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:03.452 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.451+0000 7fead1a00640 1 --2- 172.21.15.152:0/75092288 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feacc145f60 0x7feacc06ee80 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7feabc0097a0 tx=0x7feabc02b8d0 comp rx=0 tx=0).stop 2024-04-20T11:28:03.452 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.451+0000 7fead1a00640 1 -- 172.21.15.152:0/75092288 shutdown_connections 2024-04-20T11:28:03.452 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.451+0000 7fead1a00640 1 --2- 172.21.15.152:0/75092288 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feacc0726f0 0x7feacc0cd580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:03.452 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.451+0000 7fead1a00640 1 --2- 172.21.15.152:0/75092288 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feacc145f60 0x7feacc06ee80 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:03.452 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.451+0000 7fead1a00640 1 -- 172.21.15.152:0/75092288 >> 172.21.15.152:0/75092288 conn(0x7feacc067cf0 msgr2=0x7feacc06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:03.452 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.451+0000 7fead1a00640 1 -- 172.21.15.152:0/75092288 shutdown_connections 2024-04-20T11:28:03.452 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.451+0000 7fead1a00640 1 -- 172.21.15.152:0/75092288 wait complete. 2024-04-20T11:28:03.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.453+0000 7fead1a00640 1 Processor -- start 2024-04-20T11:28:03.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.454+0000 7fead1a00640 1 -- start start 2024-04-20T11:28:03.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.455+0000 7fead1a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feacc0726f0 0x7feacc15ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:03.456 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.455+0000 7fead1a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feacc0cd0d0 0x7feacc144030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:03.456 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.455+0000 7fead1a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feacc145f60 0x7feacc137b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:03.456 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.455+0000 7fead1a00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7feacc06e680 con 0x7feacc0726f0 2024-04-20T11:28:03.456 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.455+0000 7fead1a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7feacc06e3a0 con 0x7feacc145f60 2024-04-20T11:28:03.456 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.455+0000 7fead1a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7feacc0cf290 con 0x7feacc0cd0d0 2024-04-20T11:28:03.456 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.455+0000 7feacaa00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feacc0726f0 0x7feacc15ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:03.456 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.455+0000 7feacaa00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feacc0726f0 0x7feacc15ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:33068/0 (socket says 172.21.15.152:33068) 2024-04-20T11:28:03.456 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.455+0000 7feacaa00640 1 -- 172.21.15.152:0/1482999845 learned_addr learned my addr 172.21.15.152:0/1482999845 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:03.457 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.456+0000 7feacaa00640 1 -- 172.21.15.152:0/1482999845 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feacc0cd0d0 msgr2=0x7feacc144030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:03.457 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.456+0000 7feacaa00640 1 --2- 172.21.15.152:0/1482999845 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feacc0cd0d0 0x7feacc144030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:03.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.456+0000 7feacaa00640 1 -- 172.21.15.152:0/1482999845 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feacc145f60 msgr2=0x7feacc137b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:03.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.456+0000 7feacb400640 1 --2- 172.21.15.152:0/1482999845 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feacc145f60 0x7feacc137b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:03.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.456+0000 7feacaa00640 1 --2- 172.21.15.152:0/1482999845 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feacc145f60 0x7feacc137b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:03.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.456+0000 7feacaa00640 1 -- 172.21.15.152:0/1482999845 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feacc15d070 con 0x7feacc0726f0 2024-04-20T11:28:03.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.456+0000 7feacb400640 1 --2- 172.21.15.152:0/1482999845 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feacc145f60 0x7feacc137b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:28:03.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.456+0000 7feacaa00640 1 --2- 172.21.15.152:0/1482999845 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feacc0726f0 0x7feacc15ca90 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7feabc034040 tx=0x7feabc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:03.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.456+0000 7feac3400640 1 -- 172.21.15.152:0/1482999845 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feabc040070 con 0x7feacc0726f0 2024-04-20T11:28:03.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.456+0000 7feac3400640 1 -- 172.21.15.152:0/1482999845 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feabc03a910 con 0x7feacc0726f0 2024-04-20T11:28:03.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.456+0000 7feac3400640 1 -- 172.21.15.152:0/1482999845 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feabc035040 con 0x7feacc0726f0 2024-04-20T11:28:03.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.456+0000 7fead1a00640 1 -- 172.21.15.152:0/1482999845 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feacc15d350 con 0x7feacc0726f0 2024-04-20T11:28:03.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.457+0000 7fead1a00640 1 -- 172.21.15.152:0/1482999845 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feacc15d910 con 0x7feacc0726f0 2024-04-20T11:28:03.462 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.458+0000 7fead1a00640 1 -- 172.21.15.152:0/1482999845 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feacc0689f0 con 0x7feacc0726f0 2024-04-20T11:28:03.462 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.459+0000 7feac3400640 1 -- 172.21.15.152:0/1482999845 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7feabc038020 con 0x7feacc0726f0 2024-04-20T11:28:03.463 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.459+0000 7feac3400640 1 --2- 172.21.15.152:0/1482999845 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7feaa806c8f0 0x7feaa806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:03.463 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.460+0000 7feac3400640 1 -- 172.21.15.152:0/1482999845 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7feabc0ad6c0 con 0x7feacc0726f0 2024-04-20T11:28:03.463 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.460+0000 7feac2000640 1 --2- 172.21.15.152:0/1482999845 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7feaa806c8f0 0x7feaa806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:03.463 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.460+0000 7feac2000640 1 --2- 172.21.15.152:0/1482999845 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7feaa806c8f0 0x7feaa806edb0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7feacc0b1d70 tx=0x7feab4007970 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:03.464 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.464+0000 7feac3400640 1 -- 172.21.15.152:0/1482999845 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7feabc031800 con 0x7feacc0726f0 2024-04-20T11:28:03.707 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.707+0000 7fead1a00640 1 -- 172.21.15.152:0/1482999845 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feacc06bb20 con 0x7feacc0726f0 2024-04-20T11:28:03.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.708+0000 7feac3400640 1 -- 172.21.15.152:0/1482999845 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5196 (secure 0 0 0) 0x7feabc07b9d0 con 0x7feacc0726f0 2024-04-20T11:28:03.708 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:28:03.709 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/853233806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":853233806},{"type":"v1","addr":"172.21.15.175:6833","nonce":853233806}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4253,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/100300237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":100300237},{"type":"v1","addr":"172.21.15.175:6835","nonce":100300237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/2924034058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":2924034058},{"type":"v1","addr":"172.21.15.152:6835","nonce":2924034058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T11:26:58.132293+0000","modified":"2024-04-20T11:27:01.444285+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4269":{"gid":4269,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6837/927639144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":927639144},{"type":"v1","addr":"172.21.15.152:6837","nonce":927639144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4269,"qdb_cluster":[4269]},"id":1}]} 2024-04-20T11:28:03.709 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 7 2024-04-20T11:28:03.711 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.710+0000 7fead1a00640 1 -- 172.21.15.152:0/1482999845 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7feaa806c8f0 msgr2=0x7feaa806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:03.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.710+0000 7fead1a00640 1 --2- 172.21.15.152:0/1482999845 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7feaa806c8f0 0x7feaa806edb0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7feacc0b1d70 tx=0x7feab4007970 comp rx=0 tx=0).stop 2024-04-20T11:28:03.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.710+0000 7fead1a00640 1 -- 172.21.15.152:0/1482999845 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feacc0726f0 msgr2=0x7feacc15ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:03.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.711+0000 7fead1a00640 1 --2- 172.21.15.152:0/1482999845 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feacc0726f0 0x7feacc15ca90 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7feabc034040 tx=0x7feabc0046c0 comp rx=0 tx=0).stop 2024-04-20T11:28:03.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.711+0000 7fead1a00640 1 -- 172.21.15.152:0/1482999845 shutdown_connections 2024-04-20T11:28:03.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.711+0000 7fead1a00640 1 --2- 172.21.15.152:0/1482999845 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7feaa806c8f0 0x7feaa806edb0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:03.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.711+0000 7fead1a00640 1 --2- 172.21.15.152:0/1482999845 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feacc145f60 0x7feacc137b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:03.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.711+0000 7fead1a00640 1 --2- 172.21.15.152:0/1482999845 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feacc0cd0d0 0x7feacc144030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:03.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.711+0000 7fead1a00640 1 --2- 172.21.15.152:0/1482999845 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feacc0726f0 0x7feacc15ca90 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:03.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.711+0000 7fead1a00640 1 -- 172.21.15.152:0/1482999845 >> 172.21.15.152:0/1482999845 conn(0x7feacc067cf0 msgr2=0x7feacc15bc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:03.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.711+0000 7fead1a00640 1 -- 172.21.15.152:0/1482999845 shutdown_connections 2024-04-20T11:28:03.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.711+0000 7fead1a00640 1 -- 172.21.15.152:0/1482999845 wait complete. 2024-04-20T11:28:03.734 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:28:03.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.926+0000 7f76e1400640 1 Processor -- start 2024-04-20T11:28:03.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.926+0000 7f76e1400640 1 -- start start 2024-04-20T11:28:03.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.928+0000 7f76e1400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f76dc145f60 0x7f76dc06ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:03.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.928+0000 7f76e1400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f76dc0cd0d0 0x7f76dc0cd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:03.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.928+0000 7f76e1400640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76dc146340 con 0x7f76dc071fe0 2024-04-20T11:28:03.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.928+0000 7f76e1400640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f76dc06e9f0 con 0x7f76dc145f60 2024-04-20T11:28:03.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.928+0000 7f76e1400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f76dc06f3c0 con 0x7f76dc0cd0d0 2024-04-20T11:28:03.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.928+0000 7f76da800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f76dc145f60 0x7f76dc06ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:03.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.928+0000 7f76da800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f76dc145f60 0x7f76dc06ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:33084/0 (socket says 172.21.15.152:33084) 2024-04-20T11:28:03.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.928+0000 7f76da800640 1 -- 172.21.15.152:0/2737113923 learned_addr learned my addr 172.21.15.152:0/2737113923 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:03.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.929+0000 7f76da800640 1 -- 172.21.15.152:0/2737113923 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f76dc0cd0d0 msgr2=0x7f76dc0cd550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:03.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.929+0000 7f76da800640 1 --2- 172.21.15.152:0/2737113923 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f76dc0cd0d0 0x7f76dc0cd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:03.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.929+0000 7f76da800640 1 -- 172.21.15.152:0/2737113923 >> v1:172.21.15.175:6789/0 conn(0x7f76dc071fe0 legacy=0x7f76dc072470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:03.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.929+0000 7f76da800640 1 -- 172.21.15.152:0/2737113923 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76dc072d50 con 0x7f76dc145f60 2024-04-20T11:28:03.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.930+0000 7f76da800640 1 --2- 172.21.15.152:0/2737113923 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f76dc145f60 0x7f76dc06ee80 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f76dc06eb70 tx=0x7f76cc02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f06f9226980fc8a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:03.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.930+0000 7f76d9e00640 1 -- 172.21.15.152:0/2737113923 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76cc035020 con 0x7f76dc145f60 2024-04-20T11:28:03.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.930+0000 7f76d9e00640 1 -- 172.21.15.152:0/2737113923 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76cc031640 con 0x7f76dc145f60 2024-04-20T11:28:03.931 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.930+0000 7f76d9e00640 1 -- 172.21.15.152:0/2737113923 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76cc02d040 con 0x7f76dc145f60 2024-04-20T11:28:03.931 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.931+0000 7f76e1400640 1 -- 172.21.15.152:0/2737113923 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f76dc145f60 msgr2=0x7f76dc06ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:03.931 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.931+0000 7f76e1400640 1 --2- 172.21.15.152:0/2737113923 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f76dc145f60 0x7f76dc06ee80 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f76dc06eb70 tx=0x7f76cc02b8f0 comp rx=0 tx=0).stop 2024-04-20T11:28:03.932 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.931+0000 7f76e1400640 1 -- 172.21.15.152:0/2737113923 shutdown_connections 2024-04-20T11:28:03.932 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.931+0000 7f76e1400640 1 --2- 172.21.15.152:0/2737113923 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f76dc0cd0d0 0x7f76dc0cd550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:03.932 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.931+0000 7f76e1400640 1 --2- 172.21.15.152:0/2737113923 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f76dc145f60 0x7f76dc06ee80 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:03.932 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.931+0000 7f76e1400640 1 -- 172.21.15.152:0/2737113923 >> 172.21.15.152:0/2737113923 conn(0x7f76dc067cf0 msgr2=0x7f76dc06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:03.932 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.931+0000 7f76e1400640 1 -- 172.21.15.152:0/2737113923 shutdown_connections 2024-04-20T11:28:03.932 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.931+0000 7f76e1400640 1 -- 172.21.15.152:0/2737113923 wait complete. 2024-04-20T11:28:03.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.932+0000 7f76e1400640 1 Processor -- start 2024-04-20T11:28:03.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.933+0000 7f76e1400640 1 -- start start 2024-04-20T11:28:03.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.933+0000 7f76e1400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f76dc071fe0 0x7f76dc15df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:03.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.934+0000 7f76e1400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f76dc0cd0d0 0x7f76dc15b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:03.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.934+0000 7f76e1400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f76dc145f60 0x7f76dc0aaf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:03.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.934+0000 7f76e1400640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f76dc06e680 con 0x7f76dc0cd0d0 2024-04-20T11:28:03.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.934+0000 7f76e1400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f76dc06e3a0 con 0x7f76dc145f60 2024-04-20T11:28:03.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.934+0000 7f76e1400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f76dc0cf290 con 0x7f76dc071fe0 2024-04-20T11:28:03.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.934+0000 7f76d3e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f76dc0cd0d0 0x7f76dc15b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:03.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.934+0000 7f76d3e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f76dc0cd0d0 0x7f76dc15b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:33088/0 (socket says 172.21.15.152:33088) 2024-04-20T11:28:03.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.934+0000 7f76d3e00640 1 -- 172.21.15.152:0/2971449749 learned_addr learned my addr 172.21.15.152:0/2971449749 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:03.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.934+0000 7f76d3e00640 1 -- 172.21.15.152:0/2971449749 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f76dc071fe0 msgr2=0x7f76dc15df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:03.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.934+0000 7f76d3e00640 1 --2- 172.21.15.152:0/2971449749 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f76dc071fe0 0x7f76dc15df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:03.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.934+0000 7f76d3e00640 1 -- 172.21.15.152:0/2971449749 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f76dc145f60 msgr2=0x7f76dc0aaf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:03.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.934+0000 7f76d3e00640 1 --2- 172.21.15.152:0/2971449749 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f76dc145f60 0x7f76dc0aaf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:03.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.934+0000 7f76d3e00640 1 -- 172.21.15.152:0/2971449749 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76dc15ce70 con 0x7f76dc0cd0d0 2024-04-20T11:28:03.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.935+0000 7f76d3e00640 1 --2- 172.21.15.152:0/2971449749 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f76dc0cd0d0 0x7f76dc15b1b0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f76c400b4a0 tx=0x7f76c400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:03.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.935+0000 7f76d2a00640 1 -- 172.21.15.152:0/2971449749 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c4014070 con 0x7f76dc0cd0d0 2024-04-20T11:28:03.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.935+0000 7f76d2a00640 1 -- 172.21.15.152:0/2971449749 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76c4010610 con 0x7f76dc0cd0d0 2024-04-20T11:28:03.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.935+0000 7f76d2a00640 1 -- 172.21.15.152:0/2971449749 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c4010a00 con 0x7f76dc0cd0d0 2024-04-20T11:28:03.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.935+0000 7f76e1400640 1 -- 172.21.15.152:0/2971449749 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76dc15d150 con 0x7f76dc0cd0d0 2024-04-20T11:28:03.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.935+0000 7f76e1400640 1 -- 172.21.15.152:0/2971449749 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76dc15d710 con 0x7f76dc0cd0d0 2024-04-20T11:28:03.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.936+0000 7f76e1400640 1 -- 172.21.15.152:0/2971449749 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76dc0689f0 con 0x7f76dc0cd0d0 2024-04-20T11:28:03.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.937+0000 7f76d2a00640 1 -- 172.21.15.152:0/2971449749 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f76c4017070 con 0x7f76dc0cd0d0 2024-04-20T11:28:03.939 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.939+0000 7f76d2a00640 1 --2- 172.21.15.152:0/2971449749 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f76b806c8f0 0x7f76b806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:03.939 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.939+0000 7f76d2a00640 1 -- 172.21.15.152:0/2971449749 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f76c408b5d0 con 0x7f76dc0cd0d0 2024-04-20T11:28:03.939 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.939+0000 7f76da800640 1 --2- 172.21.15.152:0/2971449749 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f76b806c8f0 0x7f76b806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:03.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.940+0000 7f76da800640 1 --2- 172.21.15.152:0/2971449749 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f76b806c8f0 0x7f76b806edb0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f76cc036000 tx=0x7f76cc030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:03.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:03.943+0000 7f76d2a00640 1 -- 172.21.15.152:0/2971449749 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f76c4059a10 con 0x7f76dc0cd0d0 2024-04-20T11:28:04.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.150+0000 7f76e1400640 1 -- 172.21.15.152:0/2971449749 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f76dc06bb20 con 0x7f76dc0cd0d0 2024-04-20T11:28:04.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.153+0000 7f76d2a00640 1 -- 172.21.15.152:0/2971449749 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16914 (secure 0 0 0) 0x7f76c4059310 con 0x7f76dc0cd0d0 2024-04-20T11:28:04.154 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:28:04.154 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":23,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:27:42.601620+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:26:56.261424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T11:26:57.079286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":2}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.26","mappings":[{"from":7,"to":1}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:28:04.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.156+0000 7f76e1400640 1 -- 172.21.15.152:0/2971449749 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f76b806c8f0 msgr2=0x7f76b806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:04.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.156+0000 7f76e1400640 1 --2- 172.21.15.152:0/2971449749 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f76b806c8f0 0x7f76b806edb0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f76cc036000 tx=0x7f76cc030000 comp rx=0 tx=0).stop 2024-04-20T11:28:04.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.156+0000 7f76e1400640 1 -- 172.21.15.152:0/2971449749 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f76dc0cd0d0 msgr2=0x7f76dc15b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:04.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.156+0000 7f76e1400640 1 --2- 172.21.15.152:0/2971449749 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f76dc0cd0d0 0x7f76dc15b1b0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f76c400b4a0 tx=0x7f76c400b960 comp rx=0 tx=0).stop 2024-04-20T11:28:04.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.156+0000 7f76e1400640 1 -- 172.21.15.152:0/2971449749 shutdown_connections 2024-04-20T11:28:04.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.156+0000 7f76e1400640 1 --2- 172.21.15.152:0/2971449749 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f76b806c8f0 0x7f76b806edb0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:04.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.156+0000 7f76e1400640 1 --2- 172.21.15.152:0/2971449749 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f76dc145f60 0x7f76dc0aaf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:04.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.156+0000 7f76e1400640 1 --2- 172.21.15.152:0/2971449749 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f76dc0cd0d0 0x7f76dc15b1b0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:04.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.156+0000 7f76e1400640 1 --2- 172.21.15.152:0/2971449749 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f76dc071fe0 0x7f76dc15df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:04.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.156+0000 7f76e1400640 1 -- 172.21.15.152:0/2971449749 >> 172.21.15.152:0/2971449749 conn(0x7f76dc067cf0 msgr2=0x7f76dc0cd530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:04.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.157+0000 7f76e1400640 1 -- 172.21.15.152:0/2971449749 shutdown_connections 2024-04-20T11:28:04.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.157+0000 7f76e1400640 1 -- 172.21.15.152:0/2971449749 wait complete. 2024-04-20T11:28:04.176 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-20T11:28:04.176 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:28:04.377 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.376+0000 7fc3c1e00640 1 Processor -- start 2024-04-20T11:28:04.377 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.377+0000 7fc3c1e00640 1 -- start start 2024-04-20T11:28:04.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.378+0000 7fc3c1e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fc3bc06eb40 0x7fc3bc06f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:04.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.378+0000 7fc3c1e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc3bc06f880 0x7fc3bc15cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:04.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.378+0000 7fc3c1e00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3bc06fca0 con 0x7fc3bc15b5e0 2024-04-20T11:28:04.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.378+0000 7fc3c1e00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fc3bc06de10 con 0x7fc3bc06eb40 2024-04-20T11:28:04.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.378+0000 7fc3c1e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc3bc0b5770 con 0x7fc3bc06f880 2024-04-20T11:28:04.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.379+0000 7fc3ba000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fc3bc06eb40 0x7fc3bc06f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:04.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.379+0000 7fc3ba000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fc3bc06eb40 0x7fc3bc06f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:33094/0 (socket says 172.21.15.152:33094) 2024-04-20T11:28:04.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.379+0000 7fc3ba000640 1 -- 172.21.15.152:0/835927783 learned_addr learned my addr 172.21.15.152:0/835927783 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:04.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.379+0000 7fc3ba000640 1 -- 172.21.15.152:0/835927783 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc3bc06f880 msgr2=0x7fc3bc15cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:04.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.379+0000 7fc3ba000640 1 --2- 172.21.15.152:0/835927783 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc3bc06f880 0x7fc3bc15cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:04.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.379+0000 7fc3ba000640 1 -- 172.21.15.152:0/835927783 >> v1:172.21.15.175:6789/0 conn(0x7fc3bc15b5e0 legacy=0x7fc3bc0b4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:04.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.379+0000 7fc3ba000640 1 -- 172.21.15.152:0/835927783 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3bc06e060 con 0x7fc3bc06eb40 2024-04-20T11:28:04.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.380+0000 7fc3ba000640 1 --2- 172.21.15.152:0/835927783 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc3bc06eb40 0x7fc3bc06f340 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc06c880 tx=0x7fc3a802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac1b77954e248dcc server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:04.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.380+0000 7fc3b9600640 1 -- 172.21.15.152:0/835927783 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3a8035020 con 0x7fc3bc06eb40 2024-04-20T11:28:04.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.381+0000 7fc3b9600640 1 -- 172.21.15.152:0/835927783 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3a8031640 con 0x7fc3bc06eb40 2024-04-20T11:28:04.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.381+0000 7fc3b9600640 1 -- 172.21.15.152:0/835927783 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3a802d040 con 0x7fc3bc06eb40 2024-04-20T11:28:04.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.381+0000 7fc3c1e00640 1 -- 172.21.15.152:0/835927783 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc3bc06eb40 msgr2=0x7fc3bc06f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:04.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.381+0000 7fc3c1e00640 1 --2- 172.21.15.152:0/835927783 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc3bc06eb40 0x7fc3bc06f340 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc06c880 tx=0x7fc3a802b8f0 comp rx=0 tx=0).stop 2024-04-20T11:28:04.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.381+0000 7fc3c1e00640 1 -- 172.21.15.152:0/835927783 shutdown_connections 2024-04-20T11:28:04.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.381+0000 7fc3c1e00640 1 --2- 172.21.15.152:0/835927783 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc3bc06f880 0x7fc3bc15cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:04.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.381+0000 7fc3c1e00640 1 --2- 172.21.15.152:0/835927783 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc3bc06eb40 0x7fc3bc06f340 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:04.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.381+0000 7fc3c1e00640 1 -- 172.21.15.152:0/835927783 >> 172.21.15.152:0/835927783 conn(0x7fc3bc067cf0 msgr2=0x7fc3bc06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:04.383 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.381+0000 7fc3c1e00640 1 -- 172.21.15.152:0/835927783 shutdown_connections 2024-04-20T11:28:04.383 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.381+0000 7fc3c1e00640 1 -- 172.21.15.152:0/835927783 wait complete. 2024-04-20T11:28:04.383 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.383+0000 7fc3c1e00640 1 Processor -- start 2024-04-20T11:28:04.383 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.383+0000 7fc3c1e00640 1 -- start start 2024-04-20T11:28:04.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.384+0000 7fc3c1e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc3bc06eb40 0x7fc3bc158280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:04.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.384+0000 7fc3c1e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc3bc06f880 0x7fc3bc0cfa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:04.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.384+0000 7fc3c1e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc3bc15b5e0 0x7fc3bc0b49a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:04.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.384+0000 7fc3c1e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fc3bc15c9d0 con 0x7fc3bc06eb40 2024-04-20T11:28:04.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.384+0000 7fc3c1e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc3bc06e620 con 0x7fc3bc06f880 2024-04-20T11:28:04.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.384+0000 7fc3c1e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc3bc0b42d0 con 0x7fc3bc15b5e0 2024-04-20T11:28:04.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.384+0000 7fc3baa00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc3bc06eb40 0x7fc3bc158280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:04.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.385+0000 7fc3baa00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc3bc06eb40 0x7fc3bc158280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:33104/0 (socket says 172.21.15.152:33104) 2024-04-20T11:28:04.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.385+0000 7fc3baa00640 1 -- 172.21.15.152:0/4070809977 learned_addr learned my addr 172.21.15.152:0/4070809977 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:04.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.385+0000 7fc3ba000640 1 --2- 172.21.15.152:0/4070809977 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc3bc06f880 0x7fc3bc0cfa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:04.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.385+0000 7fc3bb400640 1 --2- 172.21.15.152:0/4070809977 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc3bc15b5e0 0x7fc3bc0b49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:04.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.385+0000 7fc3baa00640 1 -- 172.21.15.152:0/4070809977 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc3bc15b5e0 msgr2=0x7fc3bc0b49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:04.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.385+0000 7fc3baa00640 1 --2- 172.21.15.152:0/4070809977 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc3bc15b5e0 0x7fc3bc0b49a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:04.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.385+0000 7fc3baa00640 1 -- 172.21.15.152:0/4070809977 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc3bc06f880 msgr2=0x7fc3bc0cfa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:04.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.385+0000 7fc3baa00640 1 --2- 172.21.15.152:0/4070809977 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc3bc06f880 0x7fc3bc0cfa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:04.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.385+0000 7fc3baa00640 1 -- 172.21.15.152:0/4070809977 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3bc15ded0 con 0x7fc3bc06eb40 2024-04-20T11:28:04.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.385+0000 7fc3baa00640 1 --2- 172.21.15.152:0/4070809977 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc3bc06eb40 0x7fc3bc158280 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fc3ac00b410 tx=0x7fc3ac00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:04.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.386+0000 7fc3b3400640 1 -- 172.21.15.152:0/4070809977 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3ac015020 con 0x7fc3bc06eb40 2024-04-20T11:28:04.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.386+0000 7fc3b3400640 1 -- 172.21.15.152:0/4070809977 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3ac011830 con 0x7fc3bc06eb40 2024-04-20T11:28:04.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.386+0000 7fc3c1e00640 1 -- 172.21.15.152:0/4070809977 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3bc15e1b0 con 0x7fc3bc06eb40 2024-04-20T11:28:04.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.386+0000 7fc3b3400640 1 -- 172.21.15.152:0/4070809977 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3ac011c20 con 0x7fc3bc06eb40 2024-04-20T11:28:04.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.386+0000 7fc3c1e00640 1 -- 172.21.15.152:0/4070809977 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3bc1587c0 con 0x7fc3bc06eb40 2024-04-20T11:28:04.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.387+0000 7fc3c1e00640 1 -- 172.21.15.152:0/4070809977 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3bc0689f0 con 0x7fc3bc06eb40 2024-04-20T11:28:04.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.388+0000 7fc3b3400640 1 -- 172.21.15.152:0/4070809977 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc3ac018020 con 0x7fc3bc06eb40 2024-04-20T11:28:04.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.389+0000 7fc3b3400640 1 --2- 172.21.15.152:0/4070809977 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc39c06c8f0 0x7fc39c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:04.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.389+0000 7fc3b3400640 1 -- 172.21.15.152:0/4070809977 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fc3ac08bb30 con 0x7fc3bc06eb40 2024-04-20T11:28:04.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.390+0000 7fc3ba000640 1 --2- 172.21.15.152:0/4070809977 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc39c06c8f0 0x7fc39c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:04.391 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.390+0000 7fc3ba000640 1 --2- 172.21.15.152:0/4070809977 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc39c06c8f0 0x7fc39c06edb0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc3a8036000 tx=0x7fc3a803c040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:04.394 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.394+0000 7fc3b3400640 1 -- 172.21.15.152:0/4070809977 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fc3ac01f020 con 0x7fc3bc06eb40 2024-04-20T11:28:04.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.638+0000 7fc3c1e00640 1 -- 172.21.15.152:0/4070809977 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc3bc14f000 con 0x7fc3bc06eb40 2024-04-20T11:28:04.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.640+0000 7fc3b3400640 1 -- 172.21.15.152:0/4070809977 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5196 (secure 0 0 0) 0x7fc3ac0172c0 con 0x7fc3bc06eb40 2024-04-20T11:28:04.640 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:28:04.640 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/853233806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":853233806},{"type":"v1","addr":"172.21.15.175:6833","nonce":853233806}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4253,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/100300237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":100300237},{"type":"v1","addr":"172.21.15.175:6835","nonce":100300237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/2924034058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":2924034058},{"type":"v1","addr":"172.21.15.152:6835","nonce":2924034058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T11:26:58.132293+0000","modified":"2024-04-20T11:27:01.444285+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4269":{"gid":4269,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6837/927639144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":927639144},{"type":"v1","addr":"172.21.15.152:6837","nonce":927639144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4269,"qdb_cluster":[4269]},"id":1}]} 2024-04-20T11:28:04.641 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 7 2024-04-20T11:28:04.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.642+0000 7fc3c1e00640 1 -- 172.21.15.152:0/4070809977 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc39c06c8f0 msgr2=0x7fc39c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:04.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.642+0000 7fc3c1e00640 1 --2- 172.21.15.152:0/4070809977 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc39c06c8f0 0x7fc39c06edb0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc3a8036000 tx=0x7fc3a803c040 comp rx=0 tx=0).stop 2024-04-20T11:28:04.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.642+0000 7fc3c1e00640 1 -- 172.21.15.152:0/4070809977 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc3bc06eb40 msgr2=0x7fc3bc158280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:04.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.642+0000 7fc3c1e00640 1 --2- 172.21.15.152:0/4070809977 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc3bc06eb40 0x7fc3bc158280 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fc3ac00b410 tx=0x7fc3ac00b8d0 comp rx=0 tx=0).stop 2024-04-20T11:28:04.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.642+0000 7fc3c1e00640 1 -- 172.21.15.152:0/4070809977 shutdown_connections 2024-04-20T11:28:04.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.642+0000 7fc3c1e00640 1 --2- 172.21.15.152:0/4070809977 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc39c06c8f0 0x7fc39c06edb0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:04.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.642+0000 7fc3c1e00640 1 --2- 172.21.15.152:0/4070809977 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc3bc15b5e0 0x7fc3bc0b49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:04.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.642+0000 7fc3c1e00640 1 --2- 172.21.15.152:0/4070809977 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc3bc06f880 0x7fc3bc0cfa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:04.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.642+0000 7fc3c1e00640 1 --2- 172.21.15.152:0/4070809977 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc3bc06eb40 0x7fc3bc158280 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:04.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.642+0000 7fc3c1e00640 1 -- 172.21.15.152:0/4070809977 >> 172.21.15.152:0/4070809977 conn(0x7fc3bc067cf0 msgr2=0x7fc3bc152000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:04.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.643+0000 7fc3c1e00640 1 -- 172.21.15.152:0/4070809977 shutdown_connections 2024-04-20T11:28:04.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.643+0000 7fc3c1e00640 1 -- 172.21.15.152:0/4070809977 wait complete. 2024-04-20T11:28:04.662 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:28:04.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.858+0000 7f87e1400640 1 Processor -- start 2024-04-20T11:28:04.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.859+0000 7f87e1400640 1 -- start start 2024-04-20T11:28:04.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.860+0000 7f87e1400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f87dc1603d0 0x7f87dc0b5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:04.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.860+0000 7f87e1400640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87dc0d3350 con 0x7f87dc15fcc0 2024-04-20T11:28:04.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.860+0000 7f87e1400640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87dc0d3630 con 0x7f87dc0b5480 2024-04-20T11:28:04.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.860+0000 7f87e1400640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f87dc0d3910 con 0x7f87dc1603d0 2024-04-20T11:28:04.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.860+0000 7f87dbe00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f87dc15fcc0 0x7f87dc0d4e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:36130/0 (socket says 172.21.15.152:36130) 2024-04-20T11:28:04.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.860+0000 7f87d3400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f87dc1603d0 0x7f87dc0b5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:04.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.860+0000 7f87dbe00640 1 -- 172.21.15.152:0/2035452831 learned_addr learned my addr 172.21.15.152:0/2035452831 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:28:04.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.861+0000 7f87d3400640 1 -- 172.21.15.152:0/2035452831 >> v1:172.21.15.175:6789/0 conn(0x7f87dc0b5480 legacy=0x7f87dc0cf240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:04.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.861+0000 7f87d3400640 1 -- 172.21.15.152:0/2035452831 >> v1:172.21.15.152:6789/0 conn(0x7f87dc15fcc0 legacy=0x7f87dc0d4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:04.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.861+0000 7f87d3400640 1 -- 172.21.15.152:0/2035452831 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87dc0ced20 con 0x7f87dc1603d0 2024-04-20T11:28:04.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.861+0000 7f87db400640 1 -- 172.21.15.152:0/2035452831 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 510549131 0 0) 0x7f87dc0d3350 con 0x7f87dc15fcc0 2024-04-20T11:28:04.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.862+0000 7f87d3400640 1 --2- 172.21.15.152:0/2035452831 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87dc1603d0 0x7f87dc0b5eb0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f87c4009790 tx=0x7f87c402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0b15aa3d4a8e1d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:04.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.862+0000 7f87db400640 1 -- 172.21.15.152:0/2035452831 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c4035020 con 0x7f87dc1603d0 2024-04-20T11:28:04.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.862+0000 7f87db400640 1 -- 172.21.15.152:0/2035452831 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87c4031830 con 0x7f87dc1603d0 2024-04-20T11:28:04.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.862+0000 7f87db400640 1 -- 172.21.15.152:0/2035452831 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c4031c00 con 0x7f87dc1603d0 2024-04-20T11:28:04.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.863+0000 7f87e1400640 1 -- 172.21.15.152:0/2035452831 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87dc1603d0 msgr2=0x7f87dc0b5eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:04.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.863+0000 7f87e1400640 1 --2- 172.21.15.152:0/2035452831 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87dc1603d0 0x7f87dc0b5eb0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f87c4009790 tx=0x7f87c402b8d0 comp rx=0 tx=0).stop 2024-04-20T11:28:04.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.863+0000 7f87e1400640 1 -- 172.21.15.152:0/2035452831 shutdown_connections 2024-04-20T11:28:04.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.863+0000 7f87e1400640 1 --2- 172.21.15.152:0/2035452831 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87dc1603d0 0x7f87dc0b5eb0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:04.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.863+0000 7f87e1400640 1 -- 172.21.15.152:0/2035452831 >> 172.21.15.152:0/2035452831 conn(0x7f87dc08bfd0 msgr2=0x7f87dc0d4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:04.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.863+0000 7f87e1400640 1 -- 172.21.15.152:0/2035452831 shutdown_connections 2024-04-20T11:28:04.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.863+0000 7f87e1400640 1 -- 172.21.15.152:0/2035452831 wait complete. 2024-04-20T11:28:04.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.865+0000 7f87e1400640 1 Processor -- start 2024-04-20T11:28:04.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.865+0000 7f87e1400640 1 -- start start 2024-04-20T11:28:04.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.866+0000 7f87e1400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f87dc0b5480 0x7f87dc1566a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:04.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.866+0000 7f87e1400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87dc15fcc0 0x7f87dc156be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:04.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.866+0000 7f87e1400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87dc1603d0 0x7f87dc0c90d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:04.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.866+0000 7f87e1400640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f87dc0b6800 con 0x7f87dc1603d0 2024-04-20T11:28:04.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.866+0000 7f87e1400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f87dc0b6680 con 0x7f87dc15fcc0 2024-04-20T11:28:04.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.866+0000 7f87e1400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f87dc0cf950 con 0x7f87dc0b5480 2024-04-20T11:28:04.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.866+0000 7f87e0a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87dc1603d0 0x7f87dc0c90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:04.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.866+0000 7f87e0a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87dc1603d0 0x7f87dc0c90d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:33120/0 (socket says 172.21.15.152:33120) 2024-04-20T11:28:04.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.866+0000 7f87e0a00640 1 -- 172.21.15.152:0/4061557865 learned_addr learned my addr 172.21.15.152:0/4061557865 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:04.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.867+0000 7f87d3400640 1 --2- 172.21.15.152:0/4061557865 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87dc15fcc0 0x7f87dc156be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:04.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.867+0000 7f87e0a00640 1 -- 172.21.15.152:0/4061557865 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f87dc0b5480 msgr2=0x7f87dc1566a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:04.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.867+0000 7f87e0a00640 1 --2- 172.21.15.152:0/4061557865 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f87dc0b5480 0x7f87dc1566a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:04.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.867+0000 7f87e0a00640 1 -- 172.21.15.152:0/4061557865 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87dc15fcc0 msgr2=0x7f87dc156be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:04.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.867+0000 7f87e0a00640 1 --2- 172.21.15.152:0/4061557865 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87dc15fcc0 0x7f87dc156be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:04.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.867+0000 7f87e0a00640 1 -- 172.21.15.152:0/4061557865 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87dc0b7030 con 0x7f87dc1603d0 2024-04-20T11:28:04.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.867+0000 7f87e0a00640 1 --2- 172.21.15.152:0/4061557865 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87dc1603d0 0x7f87dc0c90d0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f87d400c440 tx=0x7f87d400c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:04.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.867+0000 7f87d8c00640 1 -- 172.21.15.152:0/4061557865 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d4011790 con 0x7f87dc1603d0 2024-04-20T11:28:04.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.867+0000 7f87d8c00640 1 -- 172.21.15.152:0/4061557865 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87d4017070 con 0x7f87dc1603d0 2024-04-20T11:28:04.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.867+0000 7f87d8c00640 1 -- 172.21.15.152:0/4061557865 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d4011b70 con 0x7f87dc1603d0 2024-04-20T11:28:04.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.867+0000 7f87e1400640 1 -- 172.21.15.152:0/4061557865 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87dc0b7310 con 0x7f87dc1603d0 2024-04-20T11:28:04.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.867+0000 7f87e1400640 1 -- 172.21.15.152:0/4061557865 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87dc0b78d0 con 0x7f87dc1603d0 2024-04-20T11:28:04.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.868+0000 7f87e1400640 1 -- 172.21.15.152:0/4061557865 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87dc075e60 con 0x7f87dc1603d0 2024-04-20T11:28:04.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.870+0000 7f87d8c00640 1 -- 172.21.15.152:0/4061557865 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f87d4011d10 con 0x7f87dc1603d0 2024-04-20T11:28:04.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.871+0000 7f87d8c00640 1 --2- 172.21.15.152:0/4061557865 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f87b806c8f0 0x7f87b806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:04.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.871+0000 7f87d8c00640 1 -- 172.21.15.152:0/4061557865 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f87d408b9c0 con 0x7f87dc1603d0 2024-04-20T11:28:04.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.871+0000 7f87dbe00640 1 --2- 172.21.15.152:0/4061557865 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f87b806c8f0 0x7f87b806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:04.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.872+0000 7f87dbe00640 1 --2- 172.21.15.152:0/4061557865 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f87b806c8f0 0x7f87b806edb0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f87cc001560 tx=0x7f87cc007ca0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:04.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:04.875+0000 7f87d8c00640 1 -- 172.21.15.152:0/4061557865 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f87d4059d60 con 0x7f87dc1603d0 2024-04-20T11:28:05.109 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.109+0000 7f87e1400640 1 -- 172.21.15.152:0/4061557865 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f87dc161510 con 0x7f87dc1603d0 2024-04-20T11:28:05.111 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.110+0000 7f87d8c00640 1 -- 172.21.15.152:0/4061557865 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5196 (secure 0 0 0) 0x7f87d4059660 con 0x7f87dc1603d0 2024-04-20T11:28:05.111 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:28:05.111 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/853233806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":853233806},{"type":"v1","addr":"172.21.15.175:6833","nonce":853233806}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4253,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/100300237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":100300237},{"type":"v1","addr":"172.21.15.175:6835","nonce":100300237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/2924034058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":2924034058},{"type":"v1","addr":"172.21.15.152:6835","nonce":2924034058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T11:26:58.132293+0000","modified":"2024-04-20T11:27:01.444285+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4269":{"gid":4269,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6837/927639144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":927639144},{"type":"v1","addr":"172.21.15.152:6837","nonce":927639144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4269,"qdb_cluster":[4269]},"id":1}]} 2024-04-20T11:28:05.111 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 7 2024-04-20T11:28:05.114 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.113+0000 7f87e1400640 1 -- 172.21.15.152:0/4061557865 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f87b806c8f0 msgr2=0x7f87b806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:05.114 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.113+0000 7f87e1400640 1 --2- 172.21.15.152:0/4061557865 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f87b806c8f0 0x7f87b806edb0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f87cc001560 tx=0x7f87cc007ca0 comp rx=0 tx=0).stop 2024-04-20T11:28:05.114 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.113+0000 7f87e1400640 1 -- 172.21.15.152:0/4061557865 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87dc1603d0 msgr2=0x7f87dc0c90d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:05.114 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.113+0000 7f87e1400640 1 --2- 172.21.15.152:0/4061557865 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87dc1603d0 0x7f87dc0c90d0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f87d400c440 tx=0x7f87d400c900 comp rx=0 tx=0).stop 2024-04-20T11:28:05.114 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.113+0000 7f87e1400640 1 -- 172.21.15.152:0/4061557865 shutdown_connections 2024-04-20T11:28:05.114 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.113+0000 7f87e1400640 1 --2- 172.21.15.152:0/4061557865 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f87b806c8f0 0x7f87b806edb0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:05.114 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.113+0000 7f87e1400640 1 --2- 172.21.15.152:0/4061557865 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87dc1603d0 0x7f87dc0c90d0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:05.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.113+0000 7f87e1400640 1 --2- 172.21.15.152:0/4061557865 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87dc15fcc0 0x7f87dc156be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:05.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.113+0000 7f87e1400640 1 --2- 172.21.15.152:0/4061557865 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f87dc0b5480 0x7f87dc1566a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:05.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.113+0000 7f87e1400640 1 -- 172.21.15.152:0/4061557865 >> 172.21.15.152:0/4061557865 conn(0x7f87dc08bfd0 msgr2=0x7f87dc06caf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:05.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.113+0000 7f87e1400640 1 -- 172.21.15.152:0/4061557865 shutdown_connections 2024-04-20T11:28:05.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.114+0000 7f87e1400640 1 -- 172.21.15.152:0/4061557865 wait complete. 2024-04-20T11:28:05.132 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:28:05.324 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.323+0000 7f4098400640 1 Processor -- start 2024-04-20T11:28:05.324 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.323+0000 7f4098400640 1 -- start start 2024-04-20T11:28:05.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.325+0000 7f4098400640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f40900d1e50 0x7f40900b9160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:05.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.325+0000 7f4098400640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40900cfd30 con 0x7f409015e8e0 2024-04-20T11:28:05.326 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.325+0000 7f4098400640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40900b8680 con 0x7f40900cf830 2024-04-20T11:28:05.326 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.325+0000 7f4098400640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f40900b8430 con 0x7f40900d1e50 2024-04-20T11:28:05.326 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.325+0000 7f4095a00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f409015e8e0 0x7f40900d1740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:36140/0 (socket says 172.21.15.152:36140) 2024-04-20T11:28:05.326 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.325+0000 7f4095a00640 1 -- 172.21.15.152:0/1309944106 learned_addr learned my addr 172.21.15.152:0/1309944106 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:28:05.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.326+0000 7f4095000640 1 --2- 172.21.15.152:0/1309944106 >> v2:172.21.15.175:3300/0 conn(0x7f40900d1e50 0x7f40900b9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:05.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.326+0000 7f408fe00640 1 -- 172.21.15.152:0/1309944106 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2179628724 0 0) 0x7f40900cfd30 con 0x7f409015e8e0 2024-04-20T11:28:05.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.326+0000 7f408fe00640 1 -- 172.21.15.152:0/1309944106 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4078003520 con 0x7f409015e8e0 2024-04-20T11:28:05.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.326+0000 7f408fe00640 1 -- 172.21.15.152:0/1309944106 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3894994031 0 0) 0x7f4078003520 con 0x7f409015e8e0 2024-04-20T11:28:05.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.326+0000 7f408fe00640 1 -- 172.21.15.152:0/1309944106 >> v2:172.21.15.175:3300/0 conn(0x7f40900d1e50 msgr2=0x7f40900b9160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:05.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.326+0000 7f408fe00640 1 --2- 172.21.15.152:0/1309944106 >> v2:172.21.15.175:3300/0 conn(0x7f40900d1e50 0x7f40900b9160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:05.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.326+0000 7f408fe00640 1 -- 172.21.15.152:0/1309944106 >> v1:172.21.15.175:6789/0 conn(0x7f40900cf830 legacy=0x7f40900b96a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:05.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.326+0000 7f408fe00640 1 -- 172.21.15.152:0/1309944106 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40900b8960 con 0x7f409015e8e0 2024-04-20T11:28:05.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.326+0000 7f408fe00640 1 -- 172.21.15.152:0/1309944106 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f4084001670 con 0x7f409015e8e0 2024-04-20T11:28:05.329 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.326+0000 7f408fe00640 1 -- 172.21.15.152:0/1309944106 >> v1:172.21.15.152:6789/0 conn(0x7f409015e8e0 legacy=0x7f40900d1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:05.329 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.326+0000 7f408fe00640 1 --2- 172.21.15.152:0/1309944106 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4078003a70 0x7f4078003e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:05.329 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.326+0000 7f408fe00640 1 --2- 172.21.15.152:0/1309944106 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4078004800 0x7f4078004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:05.329 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.326+0000 7f408fe00640 1 --2- 172.21.15.152:0/1309944106 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4078005210 0x7f407800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:05.329 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.326+0000 7f408fe00640 1 -- 172.21.15.152:0/1309944106 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f40900b8430 con 0x7f4078004800 2024-04-20T11:28:05.329 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.326+0000 7f408fe00640 1 -- 172.21.15.152:0/1309944106 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f407800c140 con 0x7f4078005210 2024-04-20T11:28:05.329 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.326+0000 7f408fe00640 1 -- 172.21.15.152:0/1309944106 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f407800c340 con 0x7f4078003a70 2024-04-20T11:28:05.329 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.327+0000 7f4095000640 1 --2- 172.21.15.152:0/1309944106 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4078004800 0x7f4078004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:05.329 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.327+0000 7f4095a00640 1 --2- 172.21.15.152:0/1309944106 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4078003a70 0x7f4078003e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:05.329 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.327+0000 7f4096400640 1 --2- 172.21.15.152:0/1309944106 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4078005210 0x7f407800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:05.329 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.327+0000 7f4095000640 1 -- 172.21.15.152:0/1309944106 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4078003a70 msgr2=0x7f4078003e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:05.329 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.327+0000 7f4095000640 1 --2- 172.21.15.152:0/1309944106 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4078003a70 0x7f4078003e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:05.329 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.327+0000 7f4095000640 1 -- 172.21.15.152:0/1309944106 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4078005210 msgr2=0x7f407800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:05.329 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.327+0000 7f4095000640 1 --2- 172.21.15.152:0/1309944106 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4078005210 0x7f407800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:05.329 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.327+0000 7f4095000640 1 -- 172.21.15.152:0/1309944106 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4078003520 con 0x7f4078004800 2024-04-20T11:28:05.330 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.327+0000 7f4096400640 1 --2- 172.21.15.152:0/1309944106 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4078005210 0x7f407800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:28:05.330 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.327+0000 7f4096400640 1 -- 172.21.15.152:0/1309944106 reap_dead start 2024-04-20T11:28:05.331 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.328+0000 7f4095000640 1 --2- 172.21.15.152:0/1309944106 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4078004800 0x7f4078004c80 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f407c009ac0 tx=0x7f4078005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af90b5a84ccc7d7c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:05.331 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.329+0000 7f408fe00640 1 -- 172.21.15.152:0/1309944106 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f407c039070 con 0x7f4078004800 2024-04-20T11:28:05.331 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.329+0000 7f408fe00640 1 -- 172.21.15.152:0/1309944106 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f407c02c6b0 con 0x7f4078004800 2024-04-20T11:28:05.331 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.329+0000 7f408fe00640 1 -- 172.21.15.152:0/1309944106 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f407c009690 con 0x7f4078004800 2024-04-20T11:28:05.331 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.329+0000 7f4098400640 1 -- 172.21.15.152:0/1309944106 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4078004800 msgr2=0x7f4078004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:05.331 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.329+0000 7f4098400640 1 --2- 172.21.15.152:0/1309944106 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4078004800 0x7f4078004c80 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f407c009ac0 tx=0x7f4078005830 comp rx=0 tx=0).stop 2024-04-20T11:28:05.331 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.329+0000 7f4098400640 1 -- 172.21.15.152:0/1309944106 shutdown_connections 2024-04-20T11:28:05.331 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.329+0000 7f4098400640 1 --2- 172.21.15.152:0/1309944106 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4078004800 0x7f4078004c80 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:05.331 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.329+0000 7f4098400640 1 -- 172.21.15.152:0/1309944106 >> 172.21.15.152:0/1309944106 conn(0x7f4090067ae0 msgr2=0x7f4090067ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:05.331 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.329+0000 7f4098400640 1 -- 172.21.15.152:0/1309944106 shutdown_connections 2024-04-20T11:28:05.331 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.329+0000 7f4098400640 1 -- 172.21.15.152:0/1309944106 wait complete. 2024-04-20T11:28:05.332 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.331+0000 7f4098400640 1 Processor -- start 2024-04-20T11:28:05.332 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.331+0000 7f4098400640 1 -- start start 2024-04-20T11:28:05.333 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.332+0000 7f4098400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4078004800 0x7f40900ce410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:05.333 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.332+0000 7f4098400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f40900b52d0 0x7f4090162970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:05.333 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.332+0000 7f4098400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4090162eb0 0x7f409015eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:05.333 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.332+0000 7f4098400640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f40900b3af0 con 0x7f4078004800 2024-04-20T11:28:05.333 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.332+0000 7f4098400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f409015ecc0 con 0x7f4090162eb0 2024-04-20T11:28:05.333 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.332+0000 7f4098400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f40900b3c70 con 0x7f40900b52d0 2024-04-20T11:28:05.333 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.333+0000 7f4095a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4078004800 0x7f40900ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:05.333 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.333+0000 7f4095a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4078004800 0x7f40900ce410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:33134/0 (socket says 172.21.15.152:33134) 2024-04-20T11:28:05.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.333+0000 7f4095a00640 1 -- 172.21.15.152:0/1584806059 learned_addr learned my addr 172.21.15.152:0/1584806059 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:05.335 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.333+0000 7f4095a00640 1 -- 172.21.15.152:0/1584806059 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f40900b52d0 msgr2=0x7f4090162970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:05.335 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.333+0000 7f4095a00640 1 --2- 172.21.15.152:0/1584806059 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f40900b52d0 0x7f4090162970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:05.335 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.333+0000 7f4096400640 1 --2- 172.21.15.152:0/1584806059 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4090162eb0 0x7f409015eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:05.335 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.333+0000 7f4095a00640 1 -- 172.21.15.152:0/1584806059 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4090162eb0 msgr2=0x7f409015eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:05.335 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.333+0000 7f4095a00640 1 --2- 172.21.15.152:0/1584806059 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4090162eb0 0x7f409015eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:05.335 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.333+0000 7f4095a00640 1 -- 172.21.15.152:0/1584806059 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40900b5730 con 0x7f4078004800 2024-04-20T11:28:05.335 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.333+0000 7f4096400640 1 --2- 172.21.15.152:0/1584806059 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4090162eb0 0x7f409015eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:28:05.335 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.333+0000 7f4095a00640 1 --2- 172.21.15.152:0/1584806059 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4078004800 0x7f40900ce410 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f408400e7b0 tx=0x7f408400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:05.335 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.334+0000 7f408d600640 1 -- 172.21.15.152:0/1584806059 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f408400c8d0 con 0x7f4078004800 2024-04-20T11:28:05.336 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.334+0000 7f408d600640 1 -- 172.21.15.152:0/1584806059 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4084018070 con 0x7f4078004800 2024-04-20T11:28:05.336 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.334+0000 7f408d600640 1 -- 172.21.15.152:0/1584806059 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f408400ccd0 con 0x7f4078004800 2024-04-20T11:28:05.336 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.334+0000 7f4098400640 1 -- 172.21.15.152:0/1584806059 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f409014bd60 con 0x7f4078004800 2024-04-20T11:28:05.336 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.334+0000 7f4098400640 1 -- 172.21.15.152:0/1584806059 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f409014c320 con 0x7f4078004800 2024-04-20T11:28:05.336 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.335+0000 7f4098400640 1 -- 172.21.15.152:0/1584806059 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40900689f0 con 0x7f4078004800 2024-04-20T11:28:05.336 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.336+0000 7f408d600640 1 -- 172.21.15.152:0/1584806059 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4084013420 con 0x7f4078004800 2024-04-20T11:28:05.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.337+0000 7f408d600640 1 --2- 172.21.15.152:0/1584806059 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f407006c970 0x7f407006ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:05.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.337+0000 7f408d600640 1 -- 172.21.15.152:0/1584806059 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f408408b7a0 con 0x7f4078004800 2024-04-20T11:28:05.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.337+0000 7f4095000640 1 --2- 172.21.15.152:0/1584806059 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f407006c970 0x7f407006ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:05.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.338+0000 7f4095000640 1 --2- 172.21.15.152:0/1584806059 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f407006c970 0x7f407006ee30 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f407c036000 tx=0x7f407c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:05.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.341+0000 7f408d600640 1 -- 172.21.15.152:0/1584806059 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f4084059b30 con 0x7f4078004800 2024-04-20T11:28:05.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.544+0000 7f4098400640 1 -- 172.21.15.152:0/1584806059 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4090163290 con 0x7f4078004800 2024-04-20T11:28:05.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.545+0000 7f408d600640 1 -- 172.21.15.152:0/1584806059 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16914 (secure 0 0 0) 0x7f4084059430 con 0x7f4078004800 2024-04-20T11:28:05.546 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:28:05.547 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":23,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:27:42.601620+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:26:56.261424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T11:26:57.079286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":2}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.26","mappings":[{"from":7,"to":1}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:28:05.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.548+0000 7f4098400640 1 -- 172.21.15.152:0/1584806059 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f407006c970 msgr2=0x7f407006ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:05.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.548+0000 7f4098400640 1 --2- 172.21.15.152:0/1584806059 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f407006c970 0x7f407006ee30 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f407c036000 tx=0x7f407c02e000 comp rx=0 tx=0).stop 2024-04-20T11:28:05.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.548+0000 7f4098400640 1 -- 172.21.15.152:0/1584806059 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4078004800 msgr2=0x7f40900ce410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:05.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.548+0000 7f4098400640 1 --2- 172.21.15.152:0/1584806059 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4078004800 0x7f40900ce410 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f408400e7b0 tx=0x7f408400ec70 comp rx=0 tx=0).stop 2024-04-20T11:28:05.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.548+0000 7f4098400640 1 -- 172.21.15.152:0/1584806059 shutdown_connections 2024-04-20T11:28:05.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.548+0000 7f4098400640 1 --2- 172.21.15.152:0/1584806059 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f407006c970 0x7f407006ee30 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:05.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.548+0000 7f4098400640 1 --2- 172.21.15.152:0/1584806059 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4090162eb0 0x7f409015eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:05.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.548+0000 7f4098400640 1 --2- 172.21.15.152:0/1584806059 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f40900b52d0 0x7f4090162970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:05.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.548+0000 7f4098400640 1 --2- 172.21.15.152:0/1584806059 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4078004800 0x7f40900ce410 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:05.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.548+0000 7f4098400640 1 -- 172.21.15.152:0/1584806059 >> 172.21.15.152:0/1584806059 conn(0x7f4090067ae0 msgr2=0x7f4090162190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:05.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.548+0000 7f4098400640 1 -- 172.21.15.152:0/1584806059 shutdown_connections 2024-04-20T11:28:05.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.548+0000 7f4098400640 1 -- 172.21.15.152:0/1584806059 wait complete. 2024-04-20T11:28:05.568 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-20T11:28:05.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.758+0000 7fb00fe00640 1 Processor -- start 2024-04-20T11:28:05.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.759+0000 7fb00fe00640 1 -- start start 2024-04-20T11:28:05.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.760+0000 7fb00fe00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fb010161d30 0x7fb0100d0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:05.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.760+0000 7fb00fe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb0100d4ec0 0x7fb0100d15e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:05.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.760+0000 7fb00fe00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fb0100d2b80 0x7fb0100d3010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:05.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.760+0000 7fb00fe00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fb0100d0540 con 0x7fb010161d30 2024-04-20T11:28:05.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.760+0000 7fb00fe00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fb0100cfb70 con 0x7fb0100d2b80 2024-04-20T11:28:05.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.760+0000 7fb00fe00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb0100d1b20 con 0x7fb0100d4ec0 2024-04-20T11:28:05.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.760+0000 7fb00ea00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fb010161d30 0x7fb0100d0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:05.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.760+0000 7fb00ea00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fb010161d30 0x7fb0100d0000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:33146/0 (socket says 172.21.15.152:33146) 2024-04-20T11:28:05.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.760+0000 7fb00ea00640 1 -- 172.21.15.152:0/4124635132 learned_addr learned my addr 172.21.15.152:0/4124635132 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:05.762 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.761+0000 7fb00f400640 1 --2- 172.21.15.152:0/4124635132 >> v2:172.21.15.175:3300/0 conn(0x7fb0100d2b80 0x7fb0100d3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:05.762 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.761+0000 7fb00e000640 1 --2- 172.21.15.152:0/4124635132 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb0100d4ec0 0x7fb0100d15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:05.762 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.761+0000 7fb00ea00640 1 -- 172.21.15.152:0/4124635132 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb0100d4ec0 msgr2=0x7fb0100d15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:05.762 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.761+0000 7fb00ea00640 1 --2- 172.21.15.152:0/4124635132 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb0100d4ec0 0x7fb0100d15e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:05.762 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.761+0000 7fb00ea00640 1 -- 172.21.15.152:0/4124635132 >> v2:172.21.15.175:3300/0 conn(0x7fb0100d2b80 msgr2=0x7fb0100d3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:05.762 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.761+0000 7fb00ea00640 1 --2- 172.21.15.152:0/4124635132 >> v2:172.21.15.175:3300/0 conn(0x7fb0100d2b80 0x7fb0100d3010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:05.762 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.761+0000 7fb00ea00640 1 -- 172.21.15.152:0/4124635132 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0100d26a0 con 0x7fb010161d30 2024-04-20T11:28:05.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.761+0000 7fb00f400640 1 --2- 172.21.15.152:0/4124635132 >> v2:172.21.15.175:3300/0 conn(0x7fb0100d2b80 0x7fb0100d3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:28:05.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.761+0000 7fb00e000640 1 --2- 172.21.15.152:0/4124635132 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb0100d4ec0 0x7fb0100d15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:28:05.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.762+0000 7fb00ea00640 1 --2- 172.21.15.152:0/4124635132 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb010161d30 0x7fb0100d0000 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fb0100d20d0 tx=0x7fb00002b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62c9be49cab4589 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:05.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.762+0000 7fb00d600640 1 -- 172.21.15.152:0/4124635132 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb000035020 con 0x7fb010161d30 2024-04-20T11:28:05.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.762+0000 7fb00d600640 1 -- 172.21.15.152:0/4124635132 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb000031880 con 0x7fb010161d30 2024-04-20T11:28:05.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.762+0000 7fb00d600640 1 -- 172.21.15.152:0/4124635132 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb000031c50 con 0x7fb010161d30 2024-04-20T11:28:05.764 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.763+0000 7fb00fe00640 1 -- 172.21.15.152:0/4124635132 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb010161d30 msgr2=0x7fb0100d0000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:05.764 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.763+0000 7fb00fe00640 1 --2- 172.21.15.152:0/4124635132 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb010161d30 0x7fb0100d0000 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fb0100d20d0 tx=0x7fb00002b920 comp rx=0 tx=0).stop 2024-04-20T11:28:05.764 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.763+0000 7fb00fe00640 1 -- 172.21.15.152:0/4124635132 shutdown_connections 2024-04-20T11:28:05.764 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.763+0000 7fb00fe00640 1 --2- 172.21.15.152:0/4124635132 >> v2:172.21.15.175:3300/0 conn(0x7fb0100d2b80 0x7fb0100d3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:05.764 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.763+0000 7fb00fe00640 1 --2- 172.21.15.152:0/4124635132 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb0100d4ec0 0x7fb0100d15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:05.765 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.763+0000 7fb00fe00640 1 --2- 172.21.15.152:0/4124635132 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb010161d30 0x7fb0100d0000 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:05.765 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.763+0000 7fb00fe00640 1 -- 172.21.15.152:0/4124635132 >> 172.21.15.152:0/4124635132 conn(0x7fb01008bfd0 msgr2=0x7fb0100d45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:05.765 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.763+0000 7fb00fe00640 1 -- 172.21.15.152:0/4124635132 shutdown_connections 2024-04-20T11:28:05.765 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.763+0000 7fb00fe00640 1 -- 172.21.15.152:0/4124635132 wait complete. 2024-04-20T11:28:05.766 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.766+0000 7fb00fe00640 1 Processor -- start 2024-04-20T11:28:05.766 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.766+0000 7fb00fe00640 1 -- start start 2024-04-20T11:28:05.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.767+0000 7fb00fe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb0100d2b80 0x7fb0100c9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:05.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.767+0000 7fb00fe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb0100d4ec0 0x7fb01015c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:05.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.767+0000 7fb00fe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb010161d30 0x7fb01009ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:05.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.767+0000 7fb00fe00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fb0100cf800 con 0x7fb0100d4ec0 2024-04-20T11:28:05.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.767+0000 7fb00fe00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb0100cf520 con 0x7fb0100d2b80 2024-04-20T11:28:05.769 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.767+0000 7fb00fe00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb0100cf1b0 con 0x7fb010161d30 2024-04-20T11:28:05.769 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.767+0000 7fb00e000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb0100d4ec0 0x7fb01015c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:05.769 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.767+0000 7fb00e000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb0100d4ec0 0x7fb01015c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:33148/0 (socket says 172.21.15.152:33148) 2024-04-20T11:28:05.769 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.767+0000 7fb00e000640 1 -- 172.21.15.152:0/2610504936 learned_addr learned my addr 172.21.15.152:0/2610504936 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:05.769 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.768+0000 7fb00ea00640 1 --2- 172.21.15.152:0/2610504936 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb0100d2b80 0x7fb0100c9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:05.769 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.768+0000 7fb00f400640 1 --2- 172.21.15.152:0/2610504936 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb010161d30 0x7fb01009ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:05.769 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.768+0000 7fb00e000640 1 -- 172.21.15.152:0/2610504936 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb010161d30 msgr2=0x7fb01009ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:05.769 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.768+0000 7fb00e000640 1 --2- 172.21.15.152:0/2610504936 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb010161d30 0x7fb01009ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:05.769 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.768+0000 7fb00e000640 1 -- 172.21.15.152:0/2610504936 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb0100d2b80 msgr2=0x7fb0100c9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:05.769 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.768+0000 7fb00e000640 1 --2- 172.21.15.152:0/2610504936 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb0100d2b80 0x7fb0100c9ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:05.769 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.768+0000 7fb00e000640 1 -- 172.21.15.152:0/2610504936 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb01015b0a0 con 0x7fb0100d4ec0 2024-04-20T11:28:05.770 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.768+0000 7fb00e000640 1 --2- 172.21.15.152:0/2610504936 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb0100d4ec0 0x7fb01015c420 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7faff800a390 tx=0x7faff800a850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:05.770 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.768+0000 7fb006a00640 1 -- 172.21.15.152:0/2610504936 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faff8019070 con 0x7fb0100d4ec0 2024-04-20T11:28:05.770 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.769+0000 7fb006a00640 1 -- 172.21.15.152:0/2610504936 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faff8008580 con 0x7fb0100d4ec0 2024-04-20T11:28:05.770 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.769+0000 7fb006a00640 1 -- 172.21.15.152:0/2610504936 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faff8008950 con 0x7fb0100d4ec0 2024-04-20T11:28:05.770 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.769+0000 7fb00fe00640 1 -- 172.21.15.152:0/2610504936 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb01015b380 con 0x7fb0100d4ec0 2024-04-20T11:28:05.770 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.769+0000 7fb00fe00640 1 -- 172.21.15.152:0/2610504936 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb01015b940 con 0x7fb0100d4ec0 2024-04-20T11:28:05.771 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.770+0000 7fb00fe00640 1 -- 172.21.15.152:0/2610504936 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb010075e60 con 0x7fb0100d4ec0 2024-04-20T11:28:05.771 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.771+0000 7fb006a00640 1 -- 172.21.15.152:0/2610504936 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faff801c070 con 0x7fb0100d4ec0 2024-04-20T11:28:05.772 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.772+0000 7fb006a00640 1 --2- 172.21.15.152:0/2610504936 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fafec06c8f0 0x7fafec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:05.772 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.772+0000 7fb006a00640 1 -- 172.21.15.152:0/2610504936 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7faff808bd80 con 0x7fb0100d4ec0 2024-04-20T11:28:05.772 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.772+0000 7fb00ea00640 1 --2- 172.21.15.152:0/2610504936 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fafec06c8f0 0x7fafec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:05.773 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.773+0000 7fb00ea00640 1 --2- 172.21.15.152:0/2610504936 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fafec06c8f0 0x7fafec06edb0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb000036000 tx=0x7fb00003c040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:05.776 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:05.776+0000 7fb006a00640 1 -- 172.21.15.152:0/2610504936 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7faff8059ae0 con 0x7fb0100d4ec0 2024-04-20T11:28:06.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:06.022+0000 7fb00fe00640 1 -- 172.21.15.152:0/2610504936 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fb01015ff90 con 0x7fb0100d4ec0 2024-04-20T11:28:06.024 INFO:tasks.ceph.mon.a.smithi152.stderr:2024-04-20T11:28:06.024+0000 7f8229800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-20T11:28:06.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:06.028+0000 7fb006a00640 1 -- 172.21.15.152:0/2610504936 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v8) ==== 161+0+0 (secure 0 0 0) 0x7faff805dde0 con 0x7fb0100d4ec0 2024-04-20T11:28:06.028 INFO:teuthology.orchestra.run.smithi152.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-20T11:28:06.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:06.031+0000 7fb00fe00640 1 -- 172.21.15.152:0/2610504936 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fafec06c8f0 msgr2=0x7fafec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:06.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:06.031+0000 7fb00fe00640 1 --2- 172.21.15.152:0/2610504936 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fafec06c8f0 0x7fafec06edb0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb000036000 tx=0x7fb00003c040 comp rx=0 tx=0).stop 2024-04-20T11:28:06.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:06.031+0000 7fb00fe00640 1 -- 172.21.15.152:0/2610504936 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb0100d4ec0 msgr2=0x7fb01015c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:06.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:06.031+0000 7fb00fe00640 1 --2- 172.21.15.152:0/2610504936 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb0100d4ec0 0x7fb01015c420 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7faff800a390 tx=0x7faff800a850 comp rx=0 tx=0).stop 2024-04-20T11:28:06.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:06.031+0000 7fb00fe00640 1 -- 172.21.15.152:0/2610504936 shutdown_connections 2024-04-20T11:28:06.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:06.031+0000 7fb00fe00640 1 --2- 172.21.15.152:0/2610504936 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fafec06c8f0 0x7fafec06edb0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:06.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:06.031+0000 7fb00fe00640 1 --2- 172.21.15.152:0/2610504936 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb010161d30 0x7fb01009ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:06.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:06.031+0000 7fb00fe00640 1 --2- 172.21.15.152:0/2610504936 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb0100d4ec0 0x7fb01015c420 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:06.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:06.031+0000 7fb00fe00640 1 --2- 172.21.15.152:0/2610504936 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb0100d2b80 0x7fb0100c9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:06.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:06.031+0000 7fb00fe00640 1 -- 172.21.15.152:0/2610504936 >> 172.21.15.152:0/2610504936 conn(0x7fb01008bfd0 msgr2=0x7fb0100d0860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:06.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:06.032+0000 7fb00fe00640 1 -- 172.21.15.152:0/2610504936 shutdown_connections 2024-04-20T11:28:06.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:06.032+0000 7fb00fe00640 1 -- 172.21.15.152:0/2610504936 wait complete. 2024-04-20T11:28:06.051 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-20T11:28:06.056 INFO:tasks.ceph.mds.c.smithi152.stderr: -4270> 2024-04-20T11:26:59.372+0000 7feaaae00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-20T11:28:06.116 INFO:tasks.ceph.mds.c.smithi152.stdout:starting mds.c at 2024-04-20T11:28:06.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:06.235+0000 7f6845600640 1 Processor -- start 2024-04-20T11:28:06.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:06.235+0000 7f6845600640 1 -- start start 2024-04-20T11:28:06.237 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:06.237+0000 7f6845600640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f68400b2880 0x7f68400b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:06.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:06.237+0000 7f6845600640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68401463e0 con 0x7f6840146000 2024-04-20T11:28:06.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:06.237+0000 7f6845600640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68400b3e20 con 0x7f68400cd1b0 2024-04-20T11:28:06.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:06.237+0000 7f6845600640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f68400b4750 con 0x7f68400b2880 2024-04-20T11:28:06.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:06.237+0000 7f6836000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f68400b2880 0x7f68400b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:06.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:06.237+0000 7f683ea00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f6840146000 0x7f68400b2170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:41150/0 (socket says 172.21.15.152:41150) 2024-04-20T11:28:06.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:06.237+0000 7f683ea00640 1 -- 172.21.15.152:0/1798852512 learned_addr learned my addr 172.21.15.152:0/1798852512 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:28:06.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:06.238+0000 7f6836000640 1 -- 172.21.15.152:0/1798852512 >> v1:172.21.15.175:6789/0 conn(0x7f68400cd1b0 legacy=0x7f68400cd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:06.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:06.238+0000 7f6836000640 1 -- 172.21.15.152:0/1798852512 >> v1:172.21.15.152:6789/0 conn(0x7f6840146000 legacy=0x7f68400b2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:06.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:06.238+0000 7f6836000640 1 -- 172.21.15.152:0/1798852512 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68400b4100 con 0x7f68400b2880 2024-04-20T11:28:06.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:06.239+0000 7f6836000640 1 --2- 172.21.15.152:0/1798852512 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f68400b2880 0x7f68400b4be0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f68280097a0 tx=0x7f682802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e6c4ea817900b9a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:06.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:06.239+0000 7f683e000640 1 -- 172.21.15.152:0/1798852512 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68280315a0 con 0x7f68400b2880 2024-04-20T11:28:06.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:06.239+0000 7f683e000640 1 -- 172.21.15.152:0/1798852512 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6828037070 con 0x7f68400b2880 2024-04-20T11:28:06.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:06.239+0000 7f683e000640 1 -- 172.21.15.152:0/1798852512 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6828031960 con 0x7f68400b2880 2024-04-20T11:28:06.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:06.240+0000 7f6845600640 1 -- 172.21.15.152:0/1798852512 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f68400b2880 msgr2=0x7f68400b4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:06.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:06.240+0000 7f6845600640 1 --2- 172.21.15.152:0/1798852512 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f68400b2880 0x7f68400b4be0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f68280097a0 tx=0x7f682802b840 comp rx=0 tx=0).stop 2024-04-20T11:28:06.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:06.240+0000 7f6845600640 1 -- 172.21.15.152:0/1798852512 shutdown_connections 2024-04-20T11:28:06.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:06.240+0000 7f6845600640 1 --2- 172.21.15.152:0/1798852512 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f68400b2880 0x7f68400b4be0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:06.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:06.240+0000 7f6845600640 1 -- 172.21.15.152:0/1798852512 >> 172.21.15.152:0/1798852512 conn(0x7f684006b980 msgr2=0x7f684006bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:06.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:06.240+0000 7f6845600640 1 -- 172.21.15.152:0/1798852512 shutdown_connections 2024-04-20T11:28:06.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:06.240+0000 7f6845600640 1 -- 172.21.15.152:0/1798852512 wait complete. 2024-04-20T11:28:06.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:06.242+0000 7f6845600640 1 Processor -- start 2024-04-20T11:28:06.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:06.242+0000 7f6845600640 1 -- start start 2024-04-20T11:28:06.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:06.243+0000 7f6845600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68400b2880 0x7f68400af4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:06.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:06.243+0000 7f6845600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f68400cd1b0 0x7f68400ae410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:06.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:06.243+0000 7f6845600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6840146000 0x7f684013d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:06.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:06.243+0000 7f6845600640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f684015ce70 con 0x7f6840146000 2024-04-20T11:28:06.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:06.243+0000 7f6845600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f684015ccf0 con 0x7f68400b2880 2024-04-20T11:28:06.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:06.243+0000 7f6845600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f684015cff0 con 0x7f68400cd1b0 2024-04-20T11:28:06.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:06.243+0000 7f683f400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6840146000 0x7f684013d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:06.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:06.243+0000 7f683f400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6840146000 0x7f684013d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:45942/0 (socket says 172.21.15.152:45942) 2024-04-20T11:28:06.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:06.243+0000 7f683f400640 1 -- 172.21.15.152:0/1239151844 learned_addr learned my addr 172.21.15.152:0/1239151844 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:06.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:06.244+0000 7f683f400640 1 -- 172.21.15.152:0/1239151844 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f68400cd1b0 msgr2=0x7f68400ae410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:06.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:06.244+0000 7f683f400640 1 --2- 172.21.15.152:0/1239151844 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f68400cd1b0 0x7f68400ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:06.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:06.244+0000 7f683f400640 1 -- 172.21.15.152:0/1239151844 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68400b2880 msgr2=0x7f68400af4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:06.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:06.244+0000 7f683f400640 1 --2- 172.21.15.152:0/1239151844 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68400b2880 0x7f68400af4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:06.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:06.244+0000 7f683f400640 1 -- 172.21.15.152:0/1239151844 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f684006e700 con 0x7f6840146000 2024-04-20T11:28:06.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:06.244+0000 7f683f400640 1 --2- 172.21.15.152:0/1239151844 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6840146000 0x7f684013d910 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f683800c7b0 tx=0x7f683800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:06.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:06.244+0000 7f6837400640 1 -- 172.21.15.152:0/1239151844 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6838004b60 con 0x7f6840146000 2024-04-20T11:28:06.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:06.244+0000 7f6837400640 1 -- 172.21.15.152:0/1239151844 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6838004d00 con 0x7f6840146000 2024-04-20T11:28:06.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:06.244+0000 7f6837400640 1 -- 172.21.15.152:0/1239151844 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f683800aa90 con 0x7f6840146000 2024-04-20T11:28:06.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:06.244+0000 7f6845600640 1 -- 172.21.15.152:0/1239151844 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f684006e9a0 con 0x7f6840146000 2024-04-20T11:28:06.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:06.244+0000 7f6845600640 1 -- 172.21.15.152:0/1239151844 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f684006ef60 con 0x7f6840146000 2024-04-20T11:28:06.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:06.245+0000 7f6845600640 1 -- 172.21.15.152:0/1239151844 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68400b54d0 con 0x7f6840146000 2024-04-20T11:28:06.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:06.246+0000 7f6837400640 1 -- 172.21.15.152:0/1239151844 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f683800ac30 con 0x7f6840146000 2024-04-20T11:28:06.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:06.248+0000 7f6837400640 1 --2- 172.21.15.152:0/1239151844 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f681c06c8f0 0x7f681c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:06.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:06.248+0000 7f6837400640 1 -- 172.21.15.152:0/1239151844 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5911+0+0 (secure 0 0 0) 0x7f683808beb0 con 0x7f6840146000 2024-04-20T11:28:06.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:06.248+0000 7f683ea00640 1 --2- 172.21.15.152:0/1239151844 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f681c06c8f0 0x7f681c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:06.249 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:06.249+0000 7f683ea00640 1 --2- 172.21.15.152:0/1239151844 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f681c06c8f0 0x7f681c06edb0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f6830000f60 tx=0x7f6830007a80 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:06.253 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:06.252+0000 7f6837400640 1 -- 172.21.15.152:0/1239151844 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f68400b54d0 con 0x7f6840146000 2024-04-20T11:28:06.488 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:06.488+0000 7f6845600640 1 -- 172.21.15.152:0/1239151844 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f68400689f0 con 0x7f6840146000 2024-04-20T11:28:07.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:07.031+0000 7f6837400640 1 -- 172.21.15.152:0/1239151844 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v9) ==== 104+0+0 (secure 0 0 0) 0x7f6838020070 con 0x7f6840146000 2024-04-20T11:28:07.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:07.035+0000 7f6845600640 1 -- 172.21.15.152:0/1239151844 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f681c06c8f0 msgr2=0x7f681c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:07.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:07.035+0000 7f6845600640 1 --2- 172.21.15.152:0/1239151844 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f681c06c8f0 0x7f681c06edb0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f6830000f60 tx=0x7f6830007a80 comp rx=0 tx=0).stop 2024-04-20T11:28:07.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:07.035+0000 7f6845600640 1 -- 172.21.15.152:0/1239151844 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6840146000 msgr2=0x7f684013d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:07.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:07.035+0000 7f6845600640 1 --2- 172.21.15.152:0/1239151844 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6840146000 0x7f684013d910 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f683800c7b0 tx=0x7f683800cc70 comp rx=0 tx=0).stop 2024-04-20T11:28:07.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:07.035+0000 7f6845600640 1 -- 172.21.15.152:0/1239151844 shutdown_connections 2024-04-20T11:28:07.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:07.035+0000 7f6845600640 1 --2- 172.21.15.152:0/1239151844 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f681c06c8f0 0x7f681c06edb0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:07.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:07.035+0000 7f6845600640 1 --2- 172.21.15.152:0/1239151844 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6840146000 0x7f684013d910 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:07.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:07.035+0000 7f6845600640 1 --2- 172.21.15.152:0/1239151844 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f68400cd1b0 0x7f68400ae410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:07.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:07.035+0000 7f6845600640 1 --2- 172.21.15.152:0/1239151844 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68400b2880 0x7f68400af4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:07.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:07.035+0000 7f6845600640 1 -- 172.21.15.152:0/1239151844 >> 172.21.15.152:0/1239151844 conn(0x7f684006b980 msgr2=0x7f6840071db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:07.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:07.036+0000 7f6845600640 1 -- 172.21.15.152:0/1239151844 shutdown_connections 2024-04-20T11:28:07.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:07.036+0000 7f6845600640 1 -- 172.21.15.152:0/1239151844 wait complete. 2024-04-20T11:28:07.059 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-20T11:28:07.256 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:07.255+0000 7fb4a2e00640 1 Processor -- start 2024-04-20T11:28:07.256 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:07.255+0000 7fb4a2e00640 1 -- start start 2024-04-20T11:28:07.258 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:07.257+0000 7fb4a2e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb49c0d0cd0 0x7fb49c0cebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:07.258 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:07.257+0000 7fb4a2e00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb49c15b9a0 con 0x7fb49c071f10 2024-04-20T11:28:07.258 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:07.257+0000 7fb4a2e00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb49c072370 con 0x7fb49c15b5c0 2024-04-20T11:28:07.258 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:07.257+0000 7fb4a2e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb49c072610 con 0x7fb49c0d0cd0 2024-04-20T11:28:07.258 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:07.257+0000 7fb4a1000640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fb49c071f10 0x7fb49c0d05c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:41166/0 (socket says 172.21.15.152:41166) 2024-04-20T11:28:07.258 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:07.257+0000 7fb4a1000640 1 -- 172.21.15.152:0/1934819560 learned_addr learned my addr 172.21.15.152:0/1934819560 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:28:07.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:07.258+0000 7fb4a2400640 1 --2- 172.21.15.152:0/1934819560 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb49c0d0cd0 0x7fb49c0cebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:07.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:07.258+0000 7fb49be00640 1 -- 172.21.15.152:0/1934819560 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2719565405 0 0) 0x7fb49c15b9a0 con 0x7fb49c071f10 2024-04-20T11:28:07.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:07.258+0000 7fb49be00640 1 -- 172.21.15.152:0/1934819560 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb484005730 con 0x7fb49c071f10 2024-04-20T11:28:07.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:07.258+0000 7fb49be00640 1 -- 172.21.15.152:0/1934819560 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2657173342 0 0) 0x7fb484005730 con 0x7fb49c071f10 2024-04-20T11:28:07.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:07.258+0000 7fb49be00640 1 -- 172.21.15.152:0/1934819560 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb49c0d0cd0 msgr2=0x7fb49c0cebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:07.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:07.258+0000 7fb49be00640 1 --2- 172.21.15.152:0/1934819560 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb49c0d0cd0 0x7fb49c0cebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:07.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:07.258+0000 7fb49be00640 1 -- 172.21.15.152:0/1934819560 >> v1:172.21.15.175:6789/0 conn(0x7fb49c15b5c0 legacy=0x7fb49c0b3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:07.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:07.258+0000 7fb49be00640 1 -- 172.21.15.152:0/1934819560 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb49c0728f0 con 0x7fb49c071f10 2024-04-20T11:28:07.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:07.258+0000 7fb49be00640 1 -- 172.21.15.152:0/1934819560 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7fb488001670 con 0x7fb49c071f10 2024-04-20T11:28:07.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:07.258+0000 7fb49be00640 1 -- 172.21.15.152:0/1934819560 >> v1:172.21.15.152:6789/0 conn(0x7fb49c071f10 legacy=0x7fb49c0d05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:07.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:07.258+0000 7fb49be00640 1 --2- 172.21.15.152:0/1934819560 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb484005bf0 0x7fb484006010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:07.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:07.258+0000 7fb49be00640 1 --2- 172.21.15.152:0/1934819560 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb49c0d0cd0 0x7fb484006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:07.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:07.258+0000 7fb49be00640 1 --2- 172.21.15.152:0/1934819560 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb484006f50 0x7fb4840073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:07.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:07.259+0000 7fb49be00640 1 -- 172.21.15.152:0/1934819560 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fb49c072610 con 0x7fb484006f50 2024-04-20T11:28:07.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:07.259+0000 7fb49be00640 1 -- 172.21.15.152:0/1934819560 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb48400dd30 con 0x7fb484005bf0 2024-04-20T11:28:07.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:07.259+0000 7fb49be00640 1 -- 172.21.15.152:0/1934819560 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb48400df30 con 0x7fb49c0d0cd0 2024-04-20T11:28:07.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:07.259+0000 7fb4a2400640 1 --2- 172.21.15.152:0/1934819560 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb484006f50 0x7fb4840073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:07.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:07.259+0000 7fb4a1000640 1 --2- 172.21.15.152:0/1934819560 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb49c0d0cd0 0x7fb484006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:07.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:07.259+0000 7fb4a2400640 1 -- 172.21.15.152:0/1934819560 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb49c0d0cd0 msgr2=0x7fb484006a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:07.262 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:07.259+0000 7fb4a2400640 1 --2- 172.21.15.152:0/1934819560 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb49c0d0cd0 0x7fb484006a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:07.262 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:07.259+0000 7fb4a2400640 1 -- 172.21.15.152:0/1934819560 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb484005bf0 msgr2=0x7fb484006010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:07.262 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:07.259+0000 7fb4a2400640 1 --2- 172.21.15.152:0/1934819560 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb484005bf0 0x7fb484006010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:07.262 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:07.259+0000 7fb4a2400640 1 -- 172.21.15.152:0/1934819560 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb484005730 con 0x7fb484006f50 2024-04-20T11:28:07.262 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:07.259+0000 7fb4a1000640 1 --2- 172.21.15.152:0/1934819560 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb49c0d0cd0 0x7fb484006a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:28:07.263 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:07.260+0000 7fb4a2400640 1 --2- 172.21.15.152:0/1934819560 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb484006f50 0x7fb4840073d0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fb49c0ccbf0 tx=0x7fb484007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c30be94979ebb3c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:07.263 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:07.261+0000 7fb49be00640 1 -- 172.21.15.152:0/1934819560 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb494039070 con 0x7fb484006f50 2024-04-20T11:28:07.263 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:07.261+0000 7fb49be00640 1 -- 172.21.15.152:0/1934819560 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb49402cb90 con 0x7fb484006f50 2024-04-20T11:28:07.263 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:07.261+0000 7fb49be00640 1 -- 172.21.15.152:0/1934819560 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb494009c10 con 0x7fb484006f50 2024-04-20T11:28:07.263 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:07.261+0000 7fb4a2e00640 1 -- 172.21.15.152:0/1934819560 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb484006f50 msgr2=0x7fb4840073d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:07.263 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:07.261+0000 7fb4a2e00640 1 --2- 172.21.15.152:0/1934819560 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb484006f50 0x7fb4840073d0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fb49c0ccbf0 tx=0x7fb484007a40 comp rx=0 tx=0).stop 2024-04-20T11:28:07.263 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:07.261+0000 7fb4a2400640 1 -- 172.21.15.152:0/1934819560 reap_dead start 2024-04-20T11:28:07.263 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:07.261+0000 7fb4a2e00640 1 -- 172.21.15.152:0/1934819560 shutdown_connections 2024-04-20T11:28:07.263 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:07.261+0000 7fb4a2e00640 1 -- 172.21.15.152:0/1934819560 >> 172.21.15.152:0/1934819560 conn(0x7fb49c06bda0 msgr2=0x7fb49c06c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:07.263 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:07.262+0000 7fb4a2e00640 1 -- 172.21.15.152:0/1934819560 shutdown_connections 2024-04-20T11:28:07.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:07.262+0000 7fb4a2e00640 1 -- 172.21.15.152:0/1934819560 wait complete. 2024-04-20T11:28:07.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:07.263+0000 7fb4a2e00640 1 Processor -- start 2024-04-20T11:28:07.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:07.263+0000 7fb4a2e00640 1 -- start start 2024-04-20T11:28:07.266 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:07.264+0000 7fb4a2e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb484006f50 0x7fb49c0ca7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:07.266 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:07.264+0000 7fb4a2e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb49c1518d0 0x7fb49c0cbc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:07.266 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:07.264+0000 7fb4a2e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb49c0cc1b0 0x7fb49c0afe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:07.266 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:07.264+0000 7fb4a2e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fb49c0b4410 con 0x7fb49c0cc1b0 2024-04-20T11:28:07.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:07.264+0000 7fb4a2e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb49c0b37e0 con 0x7fb49c1518d0 2024-04-20T11:28:07.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:07.264+0000 7fb4a2e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb49c0cebd0 con 0x7fb484006f50 2024-04-20T11:28:07.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:07.265+0000 7fb4a2400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb49c0cc1b0 0x7fb49c0afe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:07.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:07.265+0000 7fb4a2400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb49c0cc1b0 0x7fb49c0afe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:45950/0 (socket says 172.21.15.152:45950) 2024-04-20T11:28:07.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:07.265+0000 7fb4a2400640 1 -- 172.21.15.152:0/1624005425 learned_addr learned my addr 172.21.15.152:0/1624005425 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:07.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:07.265+0000 7fb4a2400640 1 -- 172.21.15.152:0/1624005425 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb484006f50 msgr2=0x7fb49c0ca7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:07.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:07.265+0000 7fb4a1000640 1 --2- 172.21.15.152:0/1624005425 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb49c1518d0 0x7fb49c0cbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:07.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:07.265+0000 7fb4a1a00640 1 --2- 172.21.15.152:0/1624005425 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb484006f50 0x7fb49c0ca7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:07.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:07.265+0000 7fb4a2400640 1 --2- 172.21.15.152:0/1624005425 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb484006f50 0x7fb49c0ca7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:07.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:07.265+0000 7fb4a2400640 1 -- 172.21.15.152:0/1624005425 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb49c1518d0 msgr2=0x7fb49c0cbc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:07.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:07.265+0000 7fb4a2400640 1 --2- 172.21.15.152:0/1624005425 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb49c1518d0 0x7fb49c0cbc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:07.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:07.265+0000 7fb4a2400640 1 -- 172.21.15.152:0/1624005425 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb49c0cace0 con 0x7fb49c0cc1b0 2024-04-20T11:28:07.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:07.265+0000 7fb4a2400640 1 --2- 172.21.15.152:0/1624005425 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb49c0cc1b0 0x7fb49c0afe50 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fb494037040 tx=0x7fb494009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:07.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:07.266+0000 7fb499600640 1 -- 172.21.15.152:0/1624005425 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb494039030 con 0x7fb49c0cc1b0 2024-04-20T11:28:07.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:07.266+0000 7fb499600640 1 -- 172.21.15.152:0/1624005425 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb49403a040 con 0x7fb49c0cc1b0 2024-04-20T11:28:07.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:07.266+0000 7fb4a2e00640 1 -- 172.21.15.152:0/1624005425 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb49c0c9700 con 0x7fb49c0cc1b0 2024-04-20T11:28:07.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:07.266+0000 7fb499600640 1 -- 172.21.15.152:0/1624005425 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb494009c10 con 0x7fb49c0cc1b0 2024-04-20T11:28:07.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:07.266+0000 7fb4a2e00640 1 -- 172.21.15.152:0/1624005425 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb49c0c9cc0 con 0x7fb49c0cc1b0 2024-04-20T11:28:07.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:07.267+0000 7fb4a2e00640 1 -- 172.21.15.152:0/1624005425 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb49c0689f0 con 0x7fb49c0cc1b0 2024-04-20T11:28:07.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:07.268+0000 7fb499600640 1 -- 172.21.15.152:0/1624005425 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb494045030 con 0x7fb49c0cc1b0 2024-04-20T11:28:07.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:07.269+0000 7fb499600640 1 --2- 172.21.15.152:0/1624005425 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fb47c06c8f0 0x7fb47c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:07.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:07.269+0000 7fb499600640 1 -- 172.21.15.152:0/1624005425 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5911+0+0 (secure 0 0 0) 0x7fb4940ad940 con 0x7fb49c0cc1b0 2024-04-20T11:28:07.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:07.269+0000 7fb4a1a00640 1 --2- 172.21.15.152:0/1624005425 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fb47c06c8f0 0x7fb47c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:07.270 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:07.270+0000 7fb4a1a00640 1 --2- 172.21.15.152:0/1624005425 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fb47c06c8f0 0x7fb47c06edb0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb4900015d0 tx=0x7fb490009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:07.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:07.273+0000 7fb499600640 1 -- 172.21.15.152:0/1624005425 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fb49404c060 con 0x7fb49c0cc1b0 2024-04-20T11:28:07.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:07.476+0000 7fb4a2e00640 1 -- 172.21.15.152:0/1624005425 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fb49c0cfe30 con 0x7fb49c0cc1b0 2024-04-20T11:28:08.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:08.336+0000 7fb499600640 1 -- 172.21.15.152:0/1624005425 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v25) ==== 181+0+0 (secure 0 0 0) 0x7fb49407bc90 con 0x7fb49c0cc1b0 2024-04-20T11:28:08.338 INFO:teuthology.orchestra.run.smithi152.stderr:pool 'cephfs_metadata' removed 2024-04-20T11:28:08.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:08.342+0000 7fb4a2e00640 1 -- 172.21.15.152:0/1624005425 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fb47c06c8f0 msgr2=0x7fb47c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:08.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:08.342+0000 7fb4a2e00640 1 --2- 172.21.15.152:0/1624005425 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fb47c06c8f0 0x7fb47c06edb0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb4900015d0 tx=0x7fb490009040 comp rx=0 tx=0).stop 2024-04-20T11:28:08.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:08.342+0000 7fb4a2e00640 1 -- 172.21.15.152:0/1624005425 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb49c0cc1b0 msgr2=0x7fb49c0afe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:08.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:08.342+0000 7fb4a2e00640 1 --2- 172.21.15.152:0/1624005425 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb49c0cc1b0 0x7fb49c0afe50 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fb494037040 tx=0x7fb494009750 comp rx=0 tx=0).stop 2024-04-20T11:28:08.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:08.342+0000 7fb4a2e00640 1 -- 172.21.15.152:0/1624005425 shutdown_connections 2024-04-20T11:28:08.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:08.342+0000 7fb4a2e00640 1 --2- 172.21.15.152:0/1624005425 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fb47c06c8f0 0x7fb47c06edb0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:08.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:08.342+0000 7fb4a2e00640 1 --2- 172.21.15.152:0/1624005425 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb49c0cc1b0 0x7fb49c0afe50 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:08.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:08.342+0000 7fb4a2e00640 1 --2- 172.21.15.152:0/1624005425 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb49c1518d0 0x7fb49c0cbc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:08.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:08.342+0000 7fb4a2e00640 1 --2- 172.21.15.152:0/1624005425 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb484006f50 0x7fb49c0ca7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:08.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:08.342+0000 7fb4a2e00640 1 -- 172.21.15.152:0/1624005425 >> 172.21.15.152:0/1624005425 conn(0x7fb49c06bda0 msgr2=0x7fb49c0ade00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:08.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:08.343+0000 7fb4a2e00640 1 -- 172.21.15.152:0/1624005425 shutdown_connections 2024-04-20T11:28:08.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:08.343+0000 7fb4a2e00640 1 -- 172.21.15.152:0/1624005425 wait complete. 2024-04-20T11:28:08.366 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-20T11:28:08.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:08.565+0000 7f6dc2400640 1 Processor -- start 2024-04-20T11:28:08.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:08.565+0000 7f6dc2400640 1 -- start start 2024-04-20T11:28:08.567 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:08.566+0000 7f6dc2400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f6dbc0d05f0 0x7f6dbc0d09d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:08.567 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:08.567+0000 7f6dc2400640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f6dbc071f20 0x7f6dbc072370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:08.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:08.567+0000 7f6dc2400640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6dbc0d0f10 con 0x7f6dbc0ae180 2024-04-20T11:28:08.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:08.567+0000 7f6dc2400640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f6dbc0aeef0 con 0x7f6dbc0d05f0 2024-04-20T11:28:08.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:08.567+0000 7f6dc2400640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f6dbc0af1d0 con 0x7f6dbc071f20 2024-04-20T11:28:08.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:08.567+0000 7f6dc1000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f6dbc0d05f0 0x7f6dbc0d09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:08.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:08.567+0000 7f6dc1000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f6dbc0d05f0 0x7f6dbc0d09d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:45960/0 (socket says 172.21.15.152:45960) 2024-04-20T11:28:08.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:08.567+0000 7f6dc1000640 1 -- 172.21.15.152:0/3597168660 learned_addr learned my addr 172.21.15.152:0/3597168660 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:08.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:08.567+0000 7f6dc1a00640 1 --2- 172.21.15.152:0/3597168660 >> v2:172.21.15.175:3300/0 conn(0x7f6dbc071f20 0x7f6dbc072370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:08.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:08.568+0000 7f6dc1000640 1 -- 172.21.15.152:0/3597168660 >> v2:172.21.15.175:3300/0 conn(0x7f6dbc071f20 msgr2=0x7f6dbc072370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:08.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:08.568+0000 7f6dc1000640 1 --2- 172.21.15.152:0/3597168660 >> v2:172.21.15.175:3300/0 conn(0x7f6dbc071f20 0x7f6dbc072370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:08.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:08.568+0000 7f6dc1000640 1 -- 172.21.15.152:0/3597168660 >> v1:172.21.15.175:6789/0 conn(0x7f6dbc0ae180 legacy=0x7f6dbc0ae600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:08.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:08.568+0000 7f6dc1000640 1 -- 172.21.15.152:0/3597168660 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dbc0af4b0 con 0x7f6dbc0d05f0 2024-04-20T11:28:08.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:08.569+0000 7f6dc1000640 1 --2- 172.21.15.152:0/3597168660 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6dbc0d05f0 0x7f6dbc0d09d0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc0d1190 tx=0x7f6dac02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a232345b9350a883 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:08.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:08.569+0000 7f6dbb400640 1 -- 172.21.15.152:0/3597168660 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dac02ca80 con 0x7f6dbc0d05f0 2024-04-20T11:28:08.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:08.569+0000 7f6dbb400640 1 -- 172.21.15.152:0/3597168660 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6dac02cc20 con 0x7f6dbc0d05f0 2024-04-20T11:28:08.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:08.569+0000 7f6dbb400640 1 -- 172.21.15.152:0/3597168660 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dac0343f0 con 0x7f6dbc0d05f0 2024-04-20T11:28:08.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:08.570+0000 7f6dc2400640 1 -- 172.21.15.152:0/3597168660 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6dbc0d05f0 msgr2=0x7f6dbc0d09d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:08.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:08.570+0000 7f6dc2400640 1 --2- 172.21.15.152:0/3597168660 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6dbc0d05f0 0x7f6dbc0d09d0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc0d1190 tx=0x7f6dac02bee0 comp rx=0 tx=0).stop 2024-04-20T11:28:08.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:08.570+0000 7f6dc2400640 1 -- 172.21.15.152:0/3597168660 shutdown_connections 2024-04-20T11:28:08.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:08.570+0000 7f6dc2400640 1 --2- 172.21.15.152:0/3597168660 >> v2:172.21.15.175:3300/0 conn(0x7f6dbc071f20 0x7f6dbc072370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:08.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:08.570+0000 7f6dc2400640 1 --2- 172.21.15.152:0/3597168660 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6dbc0d05f0 0x7f6dbc0d09d0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:08.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:08.570+0000 7f6dc2400640 1 -- 172.21.15.152:0/3597168660 >> 172.21.15.152:0/3597168660 conn(0x7f6dbc0b21e0 msgr2=0x7f6dbc0b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:08.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:08.570+0000 7f6dc2400640 1 -- 172.21.15.152:0/3597168660 shutdown_connections 2024-04-20T11:28:08.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:08.570+0000 7f6dc2400640 1 -- 172.21.15.152:0/3597168660 wait complete. 2024-04-20T11:28:08.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:08.572+0000 7f6dc2400640 1 Processor -- start 2024-04-20T11:28:08.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:08.572+0000 7f6dc2400640 1 -- start start 2024-04-20T11:28:08.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:08.573+0000 7f6dc2400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6dbc071f20 0x7f6dbc1569c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:08.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:08.573+0000 7f6dc2400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6dbc0ae180 0x7f6dbc15a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:08.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:08.573+0000 7f6dc2400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6dbc0d05f0 0x7f6dbc154400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:08.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:08.573+0000 7f6dc2400640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f6dbc155ca0 con 0x7f6dbc0ae180 2024-04-20T11:28:08.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:08.573+0000 7f6dc2400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6dbc0afa70 con 0x7f6dbc071f20 2024-04-20T11:28:08.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:08.573+0000 7f6dc2400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6dbc1570a0 con 0x7f6dbc0d05f0 2024-04-20T11:28:08.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:08.573+0000 7f6dbbe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6dbc0ae180 0x7f6dbc15a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:08.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:08.573+0000 7f6dbbe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6dbc0ae180 0x7f6dbc15a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:45972/0 (socket says 172.21.15.152:45972) 2024-04-20T11:28:08.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:08.573+0000 7f6dbbe00640 1 -- 172.21.15.152:0/1612119034 learned_addr learned my addr 172.21.15.152:0/1612119034 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:08.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:08.573+0000 7f6dc1000640 1 --2- 172.21.15.152:0/1612119034 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6dbc071f20 0x7f6dbc1569c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:08.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:08.573+0000 7f6dbbe00640 1 -- 172.21.15.152:0/1612119034 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6dbc0d05f0 msgr2=0x7f6dbc154400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:08.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:08.573+0000 7f6dc1a00640 1 --2- 172.21.15.152:0/1612119034 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6dbc0d05f0 0x7f6dbc154400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:08.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:08.574+0000 7f6dbbe00640 1 --2- 172.21.15.152:0/1612119034 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6dbc0d05f0 0x7f6dbc154400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:08.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:08.574+0000 7f6dbbe00640 1 -- 172.21.15.152:0/1612119034 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6dbc071f20 msgr2=0x7f6dbc1569c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:08.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:08.574+0000 7f6dbbe00640 1 --2- 172.21.15.152:0/1612119034 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6dbc071f20 0x7f6dbc1569c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:08.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:08.574+0000 7f6dbbe00640 1 -- 172.21.15.152:0/1612119034 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dbc157380 con 0x7f6dbc0ae180 2024-04-20T11:28:08.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:08.574+0000 7f6dc1000640 1 --2- 172.21.15.152:0/1612119034 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6dbc071f20 0x7f6dbc1569c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:28:08.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:08.574+0000 7f6dc1a00640 1 --2- 172.21.15.152:0/1612119034 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6dbc0d05f0 0x7f6dbc154400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:28:08.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:08.574+0000 7f6dbbe00640 1 --2- 172.21.15.152:0/1612119034 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6dbc0ae180 0x7f6dbc15a960 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f6da800c410 tx=0x7f6da800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:08.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:08.574+0000 7f6db8c00640 1 -- 172.21.15.152:0/1612119034 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da8017070 con 0x7f6dbc0ae180 2024-04-20T11:28:08.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:08.574+0000 7f6db8c00640 1 -- 172.21.15.152:0/1612119034 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6da80035f0 con 0x7f6dbc0ae180 2024-04-20T11:28:08.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:08.574+0000 7f6db8c00640 1 -- 172.21.15.152:0/1612119034 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da8003a20 con 0x7f6dbc0ae180 2024-04-20T11:28:08.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:08.574+0000 7f6dc2400640 1 -- 172.21.15.152:0/1612119034 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6dbc157660 con 0x7f6dbc0ae180 2024-04-20T11:28:08.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:08.574+0000 7f6dc2400640 1 -- 172.21.15.152:0/1612119034 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6dbc157c20 con 0x7f6dbc0ae180 2024-04-20T11:28:08.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:08.575+0000 7f6dc2400640 1 -- 172.21.15.152:0/1612119034 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6dbc075e60 con 0x7f6dbc0ae180 2024-04-20T11:28:08.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:08.576+0000 7f6db8c00640 1 -- 172.21.15.152:0/1612119034 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6da8003bc0 con 0x7f6dbc0ae180 2024-04-20T11:28:08.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:08.577+0000 7f6db8c00640 1 --2- 172.21.15.152:0/1612119034 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f6d9c06c8f0 0x7f6d9c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:08.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:08.577+0000 7f6db8c00640 1 -- 172.21.15.152:0/1612119034 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5473+0+0 (secure 0 0 0) 0x7f6da808ad90 con 0x7f6dbc0ae180 2024-04-20T11:28:08.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:08.578+0000 7f6dc1000640 1 --2- 172.21.15.152:0/1612119034 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f6d9c06c8f0 0x7f6d9c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:08.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:08.578+0000 7f6dc1000640 1 --2- 172.21.15.152:0/1612119034 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f6d9c06c8f0 0x7f6d9c06edb0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6dac035000 tx=0x7f6dac030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:08.582 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:08.582+0000 7f6db8c00640 1 -- 172.21.15.152:0/1612119034 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f6da8059290 con 0x7f6dbc0ae180 2024-04-20T11:28:08.795 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:08.794+0000 7f6dc2400640 1 -- 172.21.15.152:0/1612119034 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f6dbc157df0 con 0x7f6dbc0ae180 2024-04-20T11:28:09.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:09.340+0000 7f6db8c00640 1 -- 172.21.15.152:0/1612119034 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v26) ==== 169+0+0 (secure 0 0 0) 0x7f6da8058b90 con 0x7f6dbc0ae180 2024-04-20T11:28:09.341 INFO:teuthology.orchestra.run.smithi152.stderr:pool 'cephfs_data' removed 2024-04-20T11:28:09.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:09.345+0000 7f6dc2400640 1 -- 172.21.15.152:0/1612119034 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f6d9c06c8f0 msgr2=0x7f6d9c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:09.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:09.345+0000 7f6dc2400640 1 --2- 172.21.15.152:0/1612119034 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f6d9c06c8f0 0x7f6d9c06edb0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6dac035000 tx=0x7f6dac030000 comp rx=0 tx=0).stop 2024-04-20T11:28:09.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:09.345+0000 7f6dc2400640 1 -- 172.21.15.152:0/1612119034 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6dbc0ae180 msgr2=0x7f6dbc15a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:09.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:09.345+0000 7f6dc2400640 1 --2- 172.21.15.152:0/1612119034 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6dbc0ae180 0x7f6dbc15a960 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f6da800c410 tx=0x7f6da800c8d0 comp rx=0 tx=0).stop 2024-04-20T11:28:09.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:09.346+0000 7f6dc2400640 1 -- 172.21.15.152:0/1612119034 shutdown_connections 2024-04-20T11:28:09.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:09.346+0000 7f6dc2400640 1 --2- 172.21.15.152:0/1612119034 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f6d9c06c8f0 0x7f6d9c06edb0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:09.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:09.346+0000 7f6dc2400640 1 --2- 172.21.15.152:0/1612119034 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6dbc0d05f0 0x7f6dbc154400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:09.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:09.346+0000 7f6dc2400640 1 --2- 172.21.15.152:0/1612119034 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6dbc0ae180 0x7f6dbc15a960 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:09.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:09.346+0000 7f6dc2400640 1 --2- 172.21.15.152:0/1612119034 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6dbc071f20 0x7f6dbc1569c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:09.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:09.346+0000 7f6dc2400640 1 -- 172.21.15.152:0/1612119034 >> 172.21.15.152:0/1612119034 conn(0x7f6dbc0b21e0 msgr2=0x7f6dbc0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:09.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:09.348+0000 7f6dc2400640 1 -- 172.21.15.152:0/1612119034 shutdown_connections 2024-04-20T11:28:09.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:09.348+0000 7f6dc2400640 1 -- 172.21.15.152:0/1612119034 wait complete. 2024-04-20T11:28:09.378 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-20T11:28:09.378 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-20T11:28:09.379 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-20T11:28:09.379 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T11:28:09.379 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-20T11:28:09.379 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-20T11:28:09.379 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-20T11:28:09.379 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T11:28:09.380 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-20T11:28:09.380 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-20T11:28:09.380 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-20T11:28:09.380 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T11:28:09.380 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-20T11:28:09.380 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-20T11:28:09.380 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-20T11:28:09.380 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T11:28:09.397 INFO:tasks.ceph.mds.c:Stopped 2024-04-20T11:28:09.397 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-20T11:28:09.399 INFO:tasks.ceph.mds.a:Stopped 2024-04-20T11:28:09.399 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-20T11:28:09.412 INFO:tasks.ceph.mds.d:Stopped 2024-04-20T11:28:09.413 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-20T11:28:09.415 INFO:tasks.ceph.mds.b:Stopped 2024-04-20T11:28:09.415 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-20T11:28:09.444 INFO:tasks.ceph.mds.c:Started 2024-04-20T11:28:09.444 DEBUG:teuthology.parallel:result is None 2024-04-20T11:28:09.450 INFO:tasks.ceph.mds.a:Started 2024-04-20T11:28:09.450 DEBUG:teuthology.parallel:result is None 2024-04-20T11:28:09.462 INFO:tasks.ceph.mds.d:Started 2024-04-20T11:28:09.463 DEBUG:teuthology.parallel:result is None 2024-04-20T11:28:09.468 INFO:tasks.ceph.mds.b:Started 2024-04-20T11:28:09.469 DEBUG:teuthology.parallel:result is None 2024-04-20T11:28:09.469 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-20T11:28:09.588 INFO:tasks.ceph.mds.c.smithi152.stdout:starting mds.c at 2024-04-20T11:28:09.589 INFO:tasks.ceph.mds.a.smithi152.stdout:starting mds.a at 2024-04-20T11:28:09.610 INFO:tasks.ceph.mds.b.smithi175.stdout:starting mds.b at 2024-04-20T11:28:09.611 INFO:tasks.ceph.mds.d.smithi175.stdout:starting mds.d at 2024-04-20T11:28:09.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:09.641+0000 7f432be00640 1 Processor -- start 2024-04-20T11:28:09.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:09.641+0000 7f432be00640 1 -- start start 2024-04-20T11:28:09.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:09.643+0000 7f432be00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f432c0d4ec0 0x7f432c0d0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:09.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:09.643+0000 7f432be00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f432c0d2b80 0x7f432c0d3010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:09.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:09.643+0000 7f432be00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f432c0d26a0 con 0x7f432c161d30 2024-04-20T11:28:09.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:09.643+0000 7f432be00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f432c0d0540 con 0x7f432c0d2b80 2024-04-20T11:28:09.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:09.643+0000 7f432be00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f432c0cfb70 con 0x7f432c0d4ec0 2024-04-20T11:28:09.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:09.643+0000 7f432aa00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f432c161d30 0x7f432c0d15e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:41174/0 (socket says 172.21.15.152:41174) 2024-04-20T11:28:09.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:09.643+0000 7f432aa00640 1 -- 172.21.15.152:0/2464812669 learned_addr learned my addr 172.21.15.152:0/2464812669 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:28:09.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:09.643+0000 7f432b400640 1 --2- 172.21.15.152:0/2464812669 >> v2:172.21.15.152:3300/0 conn(0x7f432c0d2b80 0x7f432c0d3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:09.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:09.643+0000 7f4322000640 1 --2- 172.21.15.152:0/2464812669 >> v2:172.21.15.175:3300/0 conn(0x7f432c0d4ec0 0x7f432c0d0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:09.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:09.643+0000 7f432a000640 1 -- 172.21.15.152:0/2464812669 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3914663130 0 0) 0x7f432c0d26a0 con 0x7f432c161d30 2024-04-20T11:28:09.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:09.643+0000 7f432b400640 1 -- 172.21.15.152:0/2464812669 >> v2:172.21.15.175:3300/0 conn(0x7f432c0d4ec0 msgr2=0x7f432c0d0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:09.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:09.643+0000 7f432b400640 1 --2- 172.21.15.152:0/2464812669 >> v2:172.21.15.175:3300/0 conn(0x7f432c0d4ec0 0x7f432c0d0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:09.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:09.643+0000 7f432b400640 1 -- 172.21.15.152:0/2464812669 >> v1:172.21.15.152:6789/0 conn(0x7f432c161d30 legacy=0x7f432c0d15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:09.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:09.643+0000 7f432b400640 1 -- 172.21.15.152:0/2464812669 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f432c0d53c0 con 0x7f432c0d2b80 2024-04-20T11:28:09.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:09.644+0000 7f432b400640 1 --2- 172.21.15.152:0/2464812669 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f432c0d2b80 0x7f432c0d3010 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f432c0a2520 tx=0x7f432402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8b3a9a4f3c1d70e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:09.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:09.645+0000 7f432a000640 1 -- 172.21.15.152:0/2464812669 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4324039070 con 0x7f432c0d2b80 2024-04-20T11:28:09.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:09.645+0000 7f432a000640 1 -- 172.21.15.152:0/2464812669 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f432402cc80 con 0x7f432c0d2b80 2024-04-20T11:28:09.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:09.645+0000 7f432a000640 1 -- 172.21.15.152:0/2464812669 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4324034660 con 0x7f432c0d2b80 2024-04-20T11:28:09.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:09.645+0000 7f432be00640 1 -- 172.21.15.152:0/2464812669 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f432c0d2b80 msgr2=0x7f432c0d3010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:09.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:09.645+0000 7f432be00640 1 --2- 172.21.15.152:0/2464812669 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f432c0d2b80 0x7f432c0d3010 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f432c0a2520 tx=0x7f432402bed0 comp rx=0 tx=0).stop 2024-04-20T11:28:09.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:09.645+0000 7f432be00640 1 -- 172.21.15.152:0/2464812669 shutdown_connections 2024-04-20T11:28:09.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:09.645+0000 7f432be00640 1 --2- 172.21.15.152:0/2464812669 >> v2:172.21.15.175:3300/0 conn(0x7f432c0d4ec0 0x7f432c0d0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:09.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:09.645+0000 7f432be00640 1 --2- 172.21.15.152:0/2464812669 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f432c0d2b80 0x7f432c0d3010 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:09.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:09.645+0000 7f432be00640 1 -- 172.21.15.152:0/2464812669 >> 172.21.15.152:0/2464812669 conn(0x7f432c08bfd0 msgr2=0x7f432c0d45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:09.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:09.646+0000 7f432be00640 1 -- 172.21.15.152:0/2464812669 shutdown_connections 2024-04-20T11:28:09.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:09.646+0000 7f432be00640 1 -- 172.21.15.152:0/2464812669 wait complete. 2024-04-20T11:28:09.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:09.647+0000 7f432be00640 1 Processor -- start 2024-04-20T11:28:09.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:09.647+0000 7f432be00640 1 -- start start 2024-04-20T11:28:09.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:09.648+0000 7f432be00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f432c0d2b80 0x7f432c1f4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:09.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:09.648+0000 7f432be00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f432c0d4ec0 0x7f432c09e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:09.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:09.648+0000 7f432be00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f432c161d30 0x7f432c0a0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:09.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:09.648+0000 7f432be00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f432c0cf1b0 con 0x7f432c0d4ec0 2024-04-20T11:28:09.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:09.648+0000 7f432be00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f432c0cf800 con 0x7f432c161d30 2024-04-20T11:28:09.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:09.648+0000 7f432be00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f432c162110 con 0x7f432c0d2b80 2024-04-20T11:28:09.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:09.648+0000 7f4322000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f432c0d4ec0 0x7f432c09e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:09.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:09.649+0000 7f4322000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f432c0d4ec0 0x7f432c09e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:46000/0 (socket says 172.21.15.152:46000) 2024-04-20T11:28:09.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:09.649+0000 7f4322000640 1 -- 172.21.15.152:0/3529021780 learned_addr learned my addr 172.21.15.152:0/3529021780 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:09.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:09.649+0000 7f432b400640 1 --2- 172.21.15.152:0/3529021780 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f432c161d30 0x7f432c0a0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:09.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:09.649+0000 7f432aa00640 1 --2- 172.21.15.152:0/3529021780 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f432c0d2b80 0x7f432c1f4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:09.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:09.649+0000 7f4322000640 1 -- 172.21.15.152:0/3529021780 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f432c0d2b80 msgr2=0x7f432c1f4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:09.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:09.649+0000 7f4322000640 1 --2- 172.21.15.152:0/3529021780 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f432c0d2b80 0x7f432c1f4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:09.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:09.649+0000 7f4322000640 1 -- 172.21.15.152:0/3529021780 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f432c161d30 msgr2=0x7f432c0a0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:09.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:09.649+0000 7f4322000640 1 --2- 172.21.15.152:0/3529021780 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f432c161d30 0x7f432c0a0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:09.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:09.649+0000 7f4322000640 1 -- 172.21.15.152:0/3529021780 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f432c09ee50 con 0x7f432c0d4ec0 2024-04-20T11:28:09.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:09.649+0000 7f4322000640 1 --2- 172.21.15.152:0/3529021780 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f432c0d4ec0 0x7f432c09e350 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f431400e870 tx=0x7f431400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:09.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:09.649+0000 7f4323400640 1 -- 172.21.15.152:0/3529021780 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4314017070 con 0x7f432c0d4ec0 2024-04-20T11:28:09.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:09.650+0000 7f4323400640 1 -- 172.21.15.152:0/3529021780 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f431400c940 con 0x7f432c0d4ec0 2024-04-20T11:28:09.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:09.650+0000 7f4323400640 1 -- 172.21.15.152:0/3529021780 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f431400ccf0 con 0x7f432c0d4ec0 2024-04-20T11:28:09.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:09.650+0000 7f432be00640 1 -- 172.21.15.152:0/3529021780 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f432c09f130 con 0x7f432c0d4ec0 2024-04-20T11:28:09.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:09.650+0000 7f432be00640 1 -- 172.21.15.152:0/3529021780 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f432c09f6f0 con 0x7f432c0d4ec0 2024-04-20T11:28:09.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:09.651+0000 7f432be00640 1 -- 172.21.15.152:0/3529021780 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f432c075e60 con 0x7f432c0d4ec0 2024-04-20T11:28:09.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:09.652+0000 7f4323400640 1 -- 172.21.15.152:0/3529021780 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f431401a070 con 0x7f432c0d4ec0 2024-04-20T11:28:09.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:09.653+0000 7f4323400640 1 --2- 172.21.15.152:0/3529021780 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f430806c8f0 0x7f430806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:09.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:09.653+0000 7f4323400640 1 -- 172.21.15.152:0/3529021780 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 4943+0+0 (secure 0 0 0) 0x7f431408bb00 con 0x7f432c0d4ec0 2024-04-20T11:28:09.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:09.653+0000 7f432aa00640 1 --2- 172.21.15.152:0/3529021780 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f430806c8f0 0x7f430806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:09.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:09.654+0000 7f432aa00640 1 --2- 172.21.15.152:0/3529021780 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f430806c8f0 0x7f430806edb0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f431c000f60 tx=0x7f431c007d50 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:09.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:09.657+0000 7f4323400640 1 -- 172.21.15.152:0/3529021780 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f4314059c90 con 0x7f432c0d4ec0 2024-04-20T11:28:09.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:09.875+0000 7f432be00640 1 -- 172.21.15.152:0/3529021780 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f432c0d1d70 con 0x7f432c0d4ec0 2024-04-20T11:28:10.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:10.381+0000 7f4323400640 1 -- 172.21.15.152:0/3529021780 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v28) ==== 97+0+0 (secure 0 0 0) 0x7f431408be20 con 0x7f432c0d4ec0 2024-04-20T11:28:10.382 INFO:teuthology.orchestra.run.smithi152.stderr: removed all blocklist entries 2024-04-20T11:28:10.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:10.385+0000 7f432be00640 1 -- 172.21.15.152:0/3529021780 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f430806c8f0 msgr2=0x7f430806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:10.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:10.385+0000 7f432be00640 1 --2- 172.21.15.152:0/3529021780 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f430806c8f0 0x7f430806edb0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f431c000f60 tx=0x7f431c007d50 comp rx=0 tx=0).stop 2024-04-20T11:28:10.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:10.385+0000 7f432be00640 1 -- 172.21.15.152:0/3529021780 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f432c0d4ec0 msgr2=0x7f432c09e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:10.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:10.385+0000 7f432be00640 1 --2- 172.21.15.152:0/3529021780 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f432c0d4ec0 0x7f432c09e350 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f431400e870 tx=0x7f431400ed30 comp rx=0 tx=0).stop 2024-04-20T11:28:10.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:10.385+0000 7f432be00640 1 -- 172.21.15.152:0/3529021780 shutdown_connections 2024-04-20T11:28:10.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:10.385+0000 7f432be00640 1 --2- 172.21.15.152:0/3529021780 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f430806c8f0 0x7f430806edb0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:10.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:10.385+0000 7f432be00640 1 --2- 172.21.15.152:0/3529021780 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f432c161d30 0x7f432c0a0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:10.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:10.385+0000 7f432be00640 1 --2- 172.21.15.152:0/3529021780 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f432c0d4ec0 0x7f432c09e350 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:10.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:10.385+0000 7f432be00640 1 --2- 172.21.15.152:0/3529021780 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f432c0d2b80 0x7f432c1f4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:10.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:10.385+0000 7f432be00640 1 -- 172.21.15.152:0/3529021780 >> 172.21.15.152:0/3529021780 conn(0x7f432c08bfd0 msgr2=0x7f432c160930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:10.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:10.386+0000 7f432be00640 1 -- 172.21.15.152:0/3529021780 shutdown_connections 2024-04-20T11:28:10.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:10.386+0000 7f432be00640 1 -- 172.21.15.152:0/3529021780 wait complete. 2024-04-20T11:28:10.409 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-20T11:28:10.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:10.606+0000 7f7c02c00640 1 Processor -- start 2024-04-20T11:28:10.607 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:10.606+0000 7f7c02c00640 1 -- start start 2024-04-20T11:28:10.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:10.608+0000 7f7c02c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7bfc144d40 0x7f7bfc06ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:10.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:10.608+0000 7f7c02c00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f7bfc15fca0 0x7f7bfc06f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:10.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:10.608+0000 7f7c02c00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f7bfc0cf030 0x7f7bfc0cf460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:10.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:10.608+0000 7f7c02c00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f7bfc0cfe20 con 0x7f7bfc15fca0 2024-04-20T11:28:10.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:10.608+0000 7f7c02c00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f7bfc15f1e0 con 0x7f7bfc0cf030 2024-04-20T11:28:10.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:10.608+0000 7f7c02c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7bfc15f4c0 con 0x7f7bfc144d40 2024-04-20T11:28:10.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:10.608+0000 7f7c00e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f7bfc15fca0 0x7f7bfc06f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:10.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:10.608+0000 7f7c00e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f7bfc15fca0 0x7f7bfc06f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:46014/0 (socket says 172.21.15.152:46014) 2024-04-20T11:28:10.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:10.608+0000 7f7c00e00640 1 -- 172.21.15.152:0/693418431 learned_addr learned my addr 172.21.15.152:0/693418431 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:10.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:10.608+0000 7f7c02200640 1 --2- 172.21.15.152:0/693418431 >> v2:172.21.15.175:3300/0 conn(0x7f7bfc0cf030 0x7f7bfc0cf460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:10.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:10.608+0000 7f7c01800640 1 --2- 172.21.15.152:0/693418431 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7bfc144d40 0x7f7bfc06ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:10.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:10.609+0000 7f7c00e00640 1 -- 172.21.15.152:0/693418431 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7bfc144d40 msgr2=0x7f7bfc06ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:10.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:10.609+0000 7f7c00e00640 1 --2- 172.21.15.152:0/693418431 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7bfc144d40 0x7f7bfc06ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:10.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:10.609+0000 7f7c00e00640 1 -- 172.21.15.152:0/693418431 >> v2:172.21.15.175:3300/0 conn(0x7f7bfc0cf030 msgr2=0x7f7bfc0cf460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:10.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:10.609+0000 7f7c00e00640 1 --2- 172.21.15.152:0/693418431 >> v2:172.21.15.175:3300/0 conn(0x7f7bfc0cf030 0x7f7bfc0cf460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:10.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:10.609+0000 7f7c00e00640 1 -- 172.21.15.152:0/693418431 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bfc145120 con 0x7f7bfc15fca0 2024-04-20T11:28:10.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:10.609+0000 7f7c01800640 1 --2- 172.21.15.152:0/693418431 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7bfc144d40 0x7f7bfc06ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:28:10.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:10.609+0000 7f7c02200640 1 --2- 172.21.15.152:0/693418431 >> v2:172.21.15.175:3300/0 conn(0x7f7bfc0cf030 0x7f7bfc0cf460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:28:10.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:10.610+0000 7f7c00e00640 1 --2- 172.21.15.152:0/693418431 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7bfc15fca0 0x7f7bfc06f340 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc070020 tx=0x7f7be802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bf0395400c8b83 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:10.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:10.610+0000 7f7bfbe00640 1 -- 172.21.15.152:0/693418431 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be802e070 con 0x7f7bfc15fca0 2024-04-20T11:28:10.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:10.610+0000 7f7bfbe00640 1 -- 172.21.15.152:0/693418431 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7be8037070 con 0x7f7bfc15fca0 2024-04-20T11:28:10.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:10.610+0000 7f7bfbe00640 1 -- 172.21.15.152:0/693418431 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be8031a00 con 0x7f7bfc15fca0 2024-04-20T11:28:10.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:10.611+0000 7f7c02c00640 1 -- 172.21.15.152:0/693418431 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7bfc15fca0 msgr2=0x7f7bfc06f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:10.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:10.611+0000 7f7c02c00640 1 --2- 172.21.15.152:0/693418431 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7bfc15fca0 0x7f7bfc06f340 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc070020 tx=0x7f7be802b880 comp rx=0 tx=0).stop 2024-04-20T11:28:10.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:10.611+0000 7f7c02c00640 1 -- 172.21.15.152:0/693418431 shutdown_connections 2024-04-20T11:28:10.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:10.611+0000 7f7c02c00640 1 --2- 172.21.15.152:0/693418431 >> v2:172.21.15.175:3300/0 conn(0x7f7bfc0cf030 0x7f7bfc0cf460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:10.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:10.611+0000 7f7c02c00640 1 --2- 172.21.15.152:0/693418431 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7bfc15fca0 0x7f7bfc06f340 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:10.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:10.611+0000 7f7c02c00640 1 --2- 172.21.15.152:0/693418431 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7bfc144d40 0x7f7bfc06ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:10.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:10.611+0000 7f7c02c00640 1 -- 172.21.15.152:0/693418431 >> 172.21.15.152:0/693418431 conn(0x7f7bfc067ae0 msgr2=0x7f7bfc06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:10.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:10.611+0000 7f7c02c00640 1 -- 172.21.15.152:0/693418431 shutdown_connections 2024-04-20T11:28:10.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:10.611+0000 7f7c02c00640 1 -- 172.21.15.152:0/693418431 wait complete. 2024-04-20T11:28:10.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:10.612+0000 7f7c02c00640 1 Processor -- start 2024-04-20T11:28:10.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:10.613+0000 7f7c02c00640 1 -- start start 2024-04-20T11:28:10.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:10.613+0000 7f7c02c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7bfc0cf030 0x7f7bfc138ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:10.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:10.613+0000 7f7c02c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7bfc144d40 0x7f7bfc0a94a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:10.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:10.613+0000 7f7c02c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7bfc15fca0 0x7f7bfc13a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:10.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:10.614+0000 7f7c02c00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f7bfc071cd0 con 0x7f7bfc0cf030 2024-04-20T11:28:10.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:10.614+0000 7f7c02c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7bfc071e50 con 0x7f7bfc144d40 2024-04-20T11:28:10.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:10.614+0000 7f7c02c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7bfc071fd0 con 0x7f7bfc15fca0 2024-04-20T11:28:10.616 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:10.614+0000 7f7c01800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7bfc0cf030 0x7f7bfc138ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:10.616 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:10.614+0000 7f7c01800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7bfc0cf030 0x7f7bfc138ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:46030/0 (socket says 172.21.15.152:46030) 2024-04-20T11:28:10.616 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:10.614+0000 7f7c01800640 1 -- 172.21.15.152:0/940807195 learned_addr learned my addr 172.21.15.152:0/940807195 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:10.616 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:10.614+0000 7f7c01800640 1 -- 172.21.15.152:0/940807195 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7bfc15fca0 msgr2=0x7f7bfc13a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:10.616 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:10.614+0000 7f7c01800640 1 --2- 172.21.15.152:0/940807195 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7bfc15fca0 0x7f7bfc13a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:10.616 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:10.614+0000 7f7c01800640 1 -- 172.21.15.152:0/940807195 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7bfc144d40 msgr2=0x7f7bfc0a94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:10.616 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:10.614+0000 7f7c01800640 1 --2- 172.21.15.152:0/940807195 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7bfc144d40 0x7f7bfc0a94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:10.616 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:10.614+0000 7f7c01800640 1 -- 172.21.15.152:0/940807195 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bfc14a230 con 0x7f7bfc0cf030 2024-04-20T11:28:10.616 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:10.614+0000 7f7c01800640 1 --2- 172.21.15.152:0/940807195 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7bfc0cf030 0x7f7bfc138ce0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f7bf000c820 tx=0x7f7bf000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:10.616 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:10.615+0000 7f7bf9600640 1 -- 172.21.15.152:0/940807195 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf0015070 con 0x7f7bfc0cf030 2024-04-20T11:28:10.616 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:10.615+0000 7f7bf9600640 1 -- 172.21.15.152:0/940807195 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bf0007a30 con 0x7f7bfc0cf030 2024-04-20T11:28:10.616 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:10.615+0000 7f7bf9600640 1 -- 172.21.15.152:0/940807195 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf0007e00 con 0x7f7bfc0cf030 2024-04-20T11:28:10.616 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:10.615+0000 7f7c02c00640 1 -- 172.21.15.152:0/940807195 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bfc14a510 con 0x7f7bfc0cf030 2024-04-20T11:28:10.616 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:10.615+0000 7f7c02c00640 1 -- 172.21.15.152:0/940807195 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bfc14aad0 con 0x7f7bfc0cf030 2024-04-20T11:28:10.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:10.616+0000 7f7c02c00640 1 -- 172.21.15.152:0/940807195 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bfc075e60 con 0x7f7bfc0cf030 2024-04-20T11:28:10.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:10.617+0000 7f7bf9600640 1 -- 172.21.15.152:0/940807195 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7bf0018070 con 0x7f7bfc0cf030 2024-04-20T11:28:10.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:10.618+0000 7f7bf9600640 1 --2- 172.21.15.152:0/940807195 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f7bdc06c8f0 0x7f7bdc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:10.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:10.618+0000 7f7bf9600640 1 -- 172.21.15.152:0/940807195 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7f7bf008b7a0 con 0x7f7bfc0cf030 2024-04-20T11:28:10.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:10.618+0000 7f7c00e00640 1 --2- 172.21.15.152:0/940807195 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f7bdc06c8f0 0x7f7bdc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:10.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:10.619+0000 7f7c00e00640 1 --2- 172.21.15.152:0/940807195 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f7bdc06c8f0 0x7f7bdc06edb0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f7be8036000 tx=0x7f7be8030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:10.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:10.622+0000 7f7bf9600640 1 -- 172.21.15.152:0/940807195 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f7bf0059f20 con 0x7f7bfc0cf030 2024-04-20T11:28:10.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:10.902+0000 7f7c02c00640 1 -- 172.21.15.152:0/940807195 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f7bfc0cc580 con 0x7f7bfc0cf030 2024-04-20T11:28:10.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:10.903+0000 7f7bf9600640 1 -- 172.21.15.152:0/940807195 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v2) ==== 80+0+6645 (secure 0 0 0) 0x7f7bf0059820 con 0x7f7bfc0cf030 2024-04-20T11:28:10.904 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:28:10.904 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-04-20T11:28:10.904 INFO:teuthology.orchestra.run.smithi152.stdout: "auth_dump": [ 2024-04-20T11:28:10.904 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:28:10.904 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "mds.a", 2024-04-20T11:28:10.904 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBCpiNmyFRJIBAAj2aIMXdb65nqRgKFDBLLzA==", 2024-04-20T11:28:10.904 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:28:10.904 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow", 2024-04-20T11:28:10.904 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow *", 2024-04-20T11:28:10.904 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow *", 2024-04-20T11:28:10.904 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:28:10.905 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:28:10.905 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:28:10.905 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:28:10.905 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "mds.b", 2024-04-20T11:28:10.905 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBCpiNmVPytOhAAzQkvNOJaNy+MeKETWhYLyg==", 2024-04-20T11:28:10.905 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:28:10.905 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow", 2024-04-20T11:28:10.905 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow *", 2024-04-20T11:28:10.905 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow *", 2024-04-20T11:28:10.905 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:28:10.905 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:28:10.905 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:28:10.905 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:28:10.905 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "mds.c", 2024-04-20T11:28:10.905 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBCpiNm5ViKLRAAg7VBQG4hFTLYUpJMpBAaVA==", 2024-04-20T11:28:10.906 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:28:10.906 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow", 2024-04-20T11:28:10.906 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow *", 2024-04-20T11:28:10.906 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow *", 2024-04-20T11:28:10.906 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:28:10.906 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:28:10.906 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:28:10.906 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:28:10.906 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "mds.d", 2024-04-20T11:28:10.906 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBDpiNmLWhaCxAA2e4qd/7+TKenJlWICIsvGg==", 2024-04-20T11:28:10.906 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:28:10.906 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow", 2024-04-20T11:28:10.906 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow *", 2024-04-20T11:28:10.907 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow *", 2024-04-20T11:28:10.907 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:28:10.907 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:28:10.907 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:28:10.907 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:28:10.907 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "osd.0", 2024-04-20T11:28:10.907 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBHpiNmpuhyDBAAsQXX9DNvWlo0UYh53lb+FA==", 2024-04-20T11:28:10.907 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:28:10.907 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow profile osd", 2024-04-20T11:28:10.907 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile osd", 2024-04-20T11:28:10.907 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:28:10.907 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:28:10.907 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:28:10.907 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:28:10.908 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "osd.1", 2024-04-20T11:28:10.908 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBKpiNmxBiLJhAAHon1adC8G2jbYoDcWdE9jA==", 2024-04-20T11:28:10.908 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:28:10.908 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow profile osd", 2024-04-20T11:28:10.908 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile osd", 2024-04-20T11:28:10.908 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:28:10.908 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:28:10.908 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:28:10.908 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:28:10.908 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "osd.2", 2024-04-20T11:28:10.908 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBNpiNmdn0zMhAAqwIO2kKahdSzTVh8/p3FCg==", 2024-04-20T11:28:10.908 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:28:10.908 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow profile osd", 2024-04-20T11:28:10.908 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile osd", 2024-04-20T11:28:10.909 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:28:10.909 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:28:10.909 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:28:10.909 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:28:10.909 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "osd.3", 2024-04-20T11:28:10.909 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBRpiNmTPCYAhAAHFNHX/xKn21jEFHDkHMjEw==", 2024-04-20T11:28:10.909 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:28:10.909 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow profile osd", 2024-04-20T11:28:10.909 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile osd", 2024-04-20T11:28:10.909 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:28:10.909 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:28:10.909 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:28:10.909 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:28:10.909 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "osd.4", 2024-04-20T11:28:10.909 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBXpiNmM8cVFxAAJl9Hu+MvlkEiMglutI89eg==", 2024-04-20T11:28:10.910 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:28:10.910 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow profile osd", 2024-04-20T11:28:10.910 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile osd", 2024-04-20T11:28:10.910 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:28:10.910 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:28:10.910 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:28:10.910 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:28:10.910 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "osd.5", 2024-04-20T11:28:10.910 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBapiNmnGxJMRAA90P0T+VsTA8LCTcexTVr7w==", 2024-04-20T11:28:10.910 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:28:10.910 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow profile osd", 2024-04-20T11:28:10.910 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile osd", 2024-04-20T11:28:10.910 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:28:10.910 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:28:10.911 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:28:10.914 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:28:10.914 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "osd.6", 2024-04-20T11:28:10.914 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBepiNmyLXWDxAAhtHD5h478mLovE5IejAFng==", 2024-04-20T11:28:10.914 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:28:10.914 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow profile osd", 2024-04-20T11:28:10.914 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile osd", 2024-04-20T11:28:10.914 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:28:10.914 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:28:10.914 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:28:10.914 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:28:10.914 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "osd.7", 2024-04-20T11:28:10.914 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBhpiNmB3jwHBAAqco9om6r2N/GqAV6rVKiaA==", 2024-04-20T11:28:10.915 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:28:10.915 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow profile osd", 2024-04-20T11:28:10.915 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile osd", 2024-04-20T11:28:10.915 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:28:10.915 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:28:10.915 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:28:10.915 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:28:10.915 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "client.0", 2024-04-20T11:28:10.915 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBDpiNmtsBWFhAArq9A7r33CcwgMmOU2pAGPw==", 2024-04-20T11:28:10.915 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:28:10.915 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow", 2024-04-20T11:28:10.915 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow r", 2024-04-20T11:28:10.915 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow rw", 2024-04-20T11:28:10.915 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow rwx" 2024-04-20T11:28:10.916 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:28:10.916 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:28:10.916 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:28:10.916 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "client.1", 2024-04-20T11:28:10.916 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBDpiNm6aNnJxAAB431GSGytZ0IKWMh4DMalw==", 2024-04-20T11:28:10.916 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:28:10.916 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow", 2024-04-20T11:28:10.916 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow r", 2024-04-20T11:28:10.916 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow rw", 2024-04-20T11:28:10.916 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow rwx" 2024-04-20T11:28:10.916 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:28:10.916 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:28:10.916 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:28:10.917 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "client.2", 2024-04-20T11:28:10.917 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBDpiNm5QLtHRAAa2kHTDVdgSh/B4XGZaKa5Q==", 2024-04-20T11:28:10.917 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:28:10.917 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow", 2024-04-20T11:28:10.917 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow r", 2024-04-20T11:28:10.917 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow rw", 2024-04-20T11:28:10.917 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow rwx" 2024-04-20T11:28:10.917 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:28:10.917 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:28:10.917 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:28:10.917 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "client.3", 2024-04-20T11:28:10.917 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBDpiNmw95SMBAAdodzTuPp4WdKSEf+jI8Uuw==", 2024-04-20T11:28:10.917 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:28:10.917 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow", 2024-04-20T11:28:10.918 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow r", 2024-04-20T11:28:10.918 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow rw", 2024-04-20T11:28:10.918 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow rwx" 2024-04-20T11:28:10.918 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:28:10.918 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:28:10.918 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:28:10.918 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "client.admin", 2024-04-20T11:28:10.918 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBBpiNm3xz6HhAA82+N+trDW/AmBr3q8l4ECw==", 2024-04-20T11:28:10.918 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:28:10.918 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow *", 2024-04-20T11:28:10.918 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow *", 2024-04-20T11:28:10.918 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow *", 2024-04-20T11:28:10.918 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:28:10.919 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:28:10.919 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:28:10.919 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:28:10.919 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "client.bootstrap-mds", 2024-04-20T11:28:10.919 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBppiNmbCglORAAXfX9zgvQN4eAO0J1wGFHkA==", 2024-04-20T11:28:10.919 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:28:10.919 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile bootstrap-mds" 2024-04-20T11:28:10.919 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:28:10.919 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:28:10.919 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:28:10.919 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "client.bootstrap-mgr", 2024-04-20T11:28:10.919 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBppiNm9zIlORAAd5LXPXEDrSDaGodOcisnRA==", 2024-04-20T11:28:10.919 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:28:10.919 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile bootstrap-mgr" 2024-04-20T11:28:10.919 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:28:10.920 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:28:10.920 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:28:10.920 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "client.bootstrap-osd", 2024-04-20T11:28:10.920 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBppiNmGzwlORAAUg2X8g22T4YDCSCzuilpGA==", 2024-04-20T11:28:10.920 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:28:10.920 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile bootstrap-osd" 2024-04-20T11:28:10.920 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:28:10.920 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:28:10.920 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:28:10.920 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "client.bootstrap-rbd", 2024-04-20T11:28:10.920 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBppiNmNEUlORAADrruOn0kXHnrAjeO6kBA9Q==", 2024-04-20T11:28:10.920 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:28:10.920 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile bootstrap-rbd" 2024-04-20T11:28:10.920 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:28:10.921 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:28:10.921 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:28:10.921 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-04-20T11:28:10.921 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBppiNmeU4lORAAiy/RoaIpWquvrwkHgqsXtQ==", 2024-04-20T11:28:10.921 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:28:10.921 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-04-20T11:28:10.921 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:28:10.921 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:28:10.921 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:28:10.921 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "client.bootstrap-rgw", 2024-04-20T11:28:10.921 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBppiNm+FclORAAjD0wKmsRRTO0dGHKUINkjg==", 2024-04-20T11:28:10.921 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:28:10.921 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile bootstrap-rgw" 2024-04-20T11:28:10.921 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:28:10.921 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:28:10.921 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:28:10.922 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "mgr.x", 2024-04-20T11:28:10.922 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBCpiNmuhalFxAA9dTl/zSbUovCCZAanAipww==", 2024-04-20T11:28:10.922 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:28:10.922 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow *", 2024-04-20T11:28:10.922 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile mgr", 2024-04-20T11:28:10.922 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:28:10.922 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:28:10.922 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:28:10.922 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:28:10.922 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "mgr.y", 2024-04-20T11:28:10.922 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBCpiNmP1+5DRAAs6bVAEOr0i6FdPmSEyo/vQ==", 2024-04-20T11:28:10.922 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:28:10.922 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow *", 2024-04-20T11:28:10.922 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile mgr", 2024-04-20T11:28:10.923 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:28:10.923 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:28:10.923 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:28:10.923 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-04-20T11:28:10.923 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-04-20T11:28:10.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:10.906+0000 7f7c02c00640 1 -- 172.21.15.152:0/940807195 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f7bdc06c8f0 msgr2=0x7f7bdc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:10.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:10.906+0000 7f7c02c00640 1 --2- 172.21.15.152:0/940807195 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f7bdc06c8f0 0x7f7bdc06edb0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f7be8036000 tx=0x7f7be8030000 comp rx=0 tx=0).stop 2024-04-20T11:28:10.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:10.906+0000 7f7c02c00640 1 -- 172.21.15.152:0/940807195 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7bfc0cf030 msgr2=0x7f7bfc138ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:10.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:10.906+0000 7f7c02c00640 1 --2- 172.21.15.152:0/940807195 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7bfc0cf030 0x7f7bfc138ce0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f7bf000c820 tx=0x7f7bf000cc20 comp rx=0 tx=0).stop 2024-04-20T11:28:10.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:10.906+0000 7f7c02c00640 1 -- 172.21.15.152:0/940807195 shutdown_connections 2024-04-20T11:28:10.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:10.906+0000 7f7c02c00640 1 --2- 172.21.15.152:0/940807195 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f7bdc06c8f0 0x7f7bdc06edb0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:10.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:10.906+0000 7f7c02c00640 1 --2- 172.21.15.152:0/940807195 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7bfc15fca0 0x7f7bfc13a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:10.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:10.906+0000 7f7c02c00640 1 --2- 172.21.15.152:0/940807195 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7bfc144d40 0x7f7bfc0a94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:10.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:10.906+0000 7f7c02c00640 1 --2- 172.21.15.152:0/940807195 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7bfc0cf030 0x7f7bfc138ce0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:10.925 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:10.906+0000 7f7c02c00640 1 -- 172.21.15.152:0/940807195 >> 172.21.15.152:0/940807195 conn(0x7f7bfc067ae0 msgr2=0x7f7bfc13c540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:10.925 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:10.906+0000 7f7c02c00640 1 -- 172.21.15.152:0/940807195 shutdown_connections 2024-04-20T11:28:10.925 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:10.906+0000 7f7c02c00640 1 -- 172.21.15.152:0/940807195 wait complete. 2024-04-20T11:28:10.926 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-04-20T11:28:11.128 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.127+0000 7ffb1c400640 1 Processor -- start 2024-04-20T11:28:11.128 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.127+0000 7ffb1c400640 1 -- start start 2024-04-20T11:28:11.130 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.129+0000 7ffb1c400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7ffb140b4500 0x7ffb140d0100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:11.130 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.129+0000 7ffb1c400640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb1415fd20 con 0x7ffb140d0640 2024-04-20T11:28:11.130 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.129+0000 7ffb1c400640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb1415ef60 con 0x7ffb1415c560 2024-04-20T11:28:11.130 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.129+0000 7ffb1c400640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7ffb1415f240 con 0x7ffb140b4500 2024-04-20T11:28:11.130 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.129+0000 7ffb19000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7ffb140b4500 0x7ffb140d0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:11.130 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.129+0000 7ffb19000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7ffb140b4500 0x7ffb140d0100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:46038/0 (socket says 172.21.15.152:46038) 2024-04-20T11:28:11.130 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.129+0000 7ffb19000640 1 -- 172.21.15.152:0/3403533057 learned_addr learned my addr 172.21.15.152:0/3403533057 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:11.130 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.129+0000 7ffb1a400640 1 --1- 172.21.15.152:0/3403533057 >> v1:172.21.15.152:6789/0 conn(0x7ffb140d0640 0x7ffb140ada90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:41180/0 (socket says 172.21.15.152:41180) 2024-04-20T11:28:11.131 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.130+0000 7ffb19000640 1 -- 172.21.15.152:0/3403533057 >> v1:172.21.15.175:6789/0 conn(0x7ffb1415c560 legacy=0x7ffb140b3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:11.131 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.130+0000 7ffb19000640 1 -- 172.21.15.152:0/3403533057 >> v1:172.21.15.152:6789/0 conn(0x7ffb140d0640 legacy=0x7ffb140ada90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:11.131 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.130+0000 7ffb19000640 1 -- 172.21.15.152:0/3403533057 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb1415f520 con 0x7ffb140b4500 2024-04-20T11:28:11.132 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.131+0000 7ffb19000640 1 --2- 172.21.15.152:0/3403533057 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffb140b4500 0x7ffb140d0100 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7ffb00009870 tx=0x7ffb0002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bc62db050ac9fca server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:11.132 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.131+0000 7ffb13e00640 1 -- 172.21.15.152:0/3403533057 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb00035020 con 0x7ffb140b4500 2024-04-20T11:28:11.132 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.131+0000 7ffb13e00640 1 -- 172.21.15.152:0/3403533057 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb00031870 con 0x7ffb140b4500 2024-04-20T11:28:11.132 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.131+0000 7ffb13e00640 1 -- 172.21.15.152:0/3403533057 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb00031c40 con 0x7ffb140b4500 2024-04-20T11:28:11.133 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.132+0000 7ffb1c400640 1 -- 172.21.15.152:0/3403533057 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffb140b4500 msgr2=0x7ffb140d0100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:11.133 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.132+0000 7ffb1c400640 1 --2- 172.21.15.152:0/3403533057 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffb140b4500 0x7ffb140d0100 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7ffb00009870 tx=0x7ffb0002b910 comp rx=0 tx=0).stop 2024-04-20T11:28:11.133 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.132+0000 7ffb1c400640 1 -- 172.21.15.152:0/3403533057 shutdown_connections 2024-04-20T11:28:11.133 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.132+0000 7ffb1c400640 1 --2- 172.21.15.152:0/3403533057 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffb140b4500 0x7ffb140d0100 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:11.133 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.132+0000 7ffb1c400640 1 -- 172.21.15.152:0/3403533057 >> 172.21.15.152:0/3403533057 conn(0x7ffb14067cf0 msgr2=0x7ffb1406b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:11.133 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.132+0000 7ffb1c400640 1 -- 172.21.15.152:0/3403533057 shutdown_connections 2024-04-20T11:28:11.133 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.132+0000 7ffb1c400640 1 -- 172.21.15.152:0/3403533057 wait complete. 2024-04-20T11:28:11.134 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.134+0000 7ffb1c400640 1 Processor -- start 2024-04-20T11:28:11.134 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.134+0000 7ffb1c400640 1 -- start start 2024-04-20T11:28:11.136 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.135+0000 7ffb1c400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffb140b4500 0x7ffb1414e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:11.137 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.135+0000 7ffb1c400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffb140d0640 0x7ffb140cfb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:11.137 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.135+0000 7ffb1c400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffb1415c560 0x7ffb140cce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:11.137 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.135+0000 7ffb1c400640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7ffb140d0ad0 con 0x7ffb1415c560 2024-04-20T11:28:11.137 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.135+0000 7ffb1c400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ffb140d0dc0 con 0x7ffb140b4500 2024-04-20T11:28:11.137 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.135+0000 7ffb1c400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ffb140ae1a0 con 0x7ffb140d0640 2024-04-20T11:28:11.137 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.135+0000 7ffb1a400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffb1415c560 0x7ffb140cce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:11.137 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.135+0000 7ffb1a400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffb1415c560 0x7ffb140cce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:46046/0 (socket says 172.21.15.152:46046) 2024-04-20T11:28:11.137 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.135+0000 7ffb1a400640 1 -- 172.21.15.152:0/1721963121 learned_addr learned my addr 172.21.15.152:0/1721963121 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:11.137 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.136+0000 7ffb19000640 1 --2- 172.21.15.152:0/1721963121 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffb140d0640 0x7ffb140cfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:11.137 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.136+0000 7ffb19a00640 1 --2- 172.21.15.152:0/1721963121 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffb140b4500 0x7ffb1414e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:11.137 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.136+0000 7ffb1a400640 1 -- 172.21.15.152:0/1721963121 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffb140d0640 msgr2=0x7ffb140cfb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:11.137 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.136+0000 7ffb1a400640 1 --2- 172.21.15.152:0/1721963121 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffb140d0640 0x7ffb140cfb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:11.137 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.136+0000 7ffb1a400640 1 -- 172.21.15.152:0/1721963121 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffb140b4500 msgr2=0x7ffb1414e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:11.138 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.136+0000 7ffb1a400640 1 --2- 172.21.15.152:0/1721963121 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffb140b4500 0x7ffb1414e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:11.138 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.136+0000 7ffb1a400640 1 -- 172.21.15.152:0/1721963121 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb140cdef0 con 0x7ffb1415c560 2024-04-20T11:28:11.138 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.136+0000 7ffb1a400640 1 --2- 172.21.15.152:0/1721963121 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffb1415c560 0x7ffb140cce50 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c00c7b0 tx=0x7ffb0c00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:11.138 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.136+0000 7ffb11600640 1 -- 172.21.15.152:0/1721963121 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb0c017070 con 0x7ffb1415c560 2024-04-20T11:28:11.139 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.137+0000 7ffb11600640 1 -- 172.21.15.152:0/1721963121 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb0c00a5e0 con 0x7ffb1415c560 2024-04-20T11:28:11.139 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.137+0000 7ffb11600640 1 -- 172.21.15.152:0/1721963121 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb0c00a990 con 0x7ffb1415c560 2024-04-20T11:28:11.139 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.137+0000 7ffb1c400640 1 -- 172.21.15.152:0/1721963121 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb140ce1d0 con 0x7ffb1415c560 2024-04-20T11:28:11.139 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.137+0000 7ffb1c400640 1 -- 172.21.15.152:0/1721963121 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb14157460 con 0x7ffb1415c560 2024-04-20T11:28:11.139 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.138+0000 7ffb1c400640 1 -- 172.21.15.152:0/1721963121 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb140689f0 con 0x7ffb1415c560 2024-04-20T11:28:11.139 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.139+0000 7ffb11600640 1 -- 172.21.15.152:0/1721963121 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ffb0c01a070 con 0x7ffb1415c560 2024-04-20T11:28:11.140 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.140+0000 7ffb11600640 1 --2- 172.21.15.152:0/1721963121 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ffaf406c8f0 0x7ffaf406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:11.140 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.140+0000 7ffb11600640 1 -- 172.21.15.152:0/1721963121 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7ffb0c08ad00 con 0x7ffb1415c560 2024-04-20T11:28:11.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.140+0000 7ffb19a00640 1 --2- 172.21.15.152:0/1721963121 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ffaf406c8f0 0x7ffaf406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:11.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.141+0000 7ffb19a00640 1 --2- 172.21.15.152:0/1721963121 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ffaf406c8f0 0x7ffaf406edb0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ffb08006fd0 tx=0x7ffb08008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:11.145 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.144+0000 7ffb11600640 1 -- 172.21.15.152:0/1721963121 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ffb0c058e20 con 0x7ffb1415c560 2024-04-20T11:28:11.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.426+0000 7ffb1c400640 1 -- 172.21.15.152:0/1721963121 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7ffb1406deb0 con 0x7ffb1415c560 2024-04-20T11:28:11.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.431+0000 7ffb11600640 1 -- 172.21.15.152:0/1721963121 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v3) ==== 90+0+0 (secure 0 0 0) 0x7ffb0c05d120 con 0x7ffb1415c560 2024-04-20T11:28:11.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.433+0000 7ffb1c400640 1 -- 172.21.15.152:0/1721963121 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ffaf406c8f0 msgr2=0x7ffaf406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:11.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.433+0000 7ffb1c400640 1 --2- 172.21.15.152:0/1721963121 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ffaf406c8f0 0x7ffaf406edb0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ffb08006fd0 tx=0x7ffb08008040 comp rx=0 tx=0).stop 2024-04-20T11:28:11.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.433+0000 7ffb1c400640 1 -- 172.21.15.152:0/1721963121 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffb1415c560 msgr2=0x7ffb140cce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:11.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.433+0000 7ffb1c400640 1 --2- 172.21.15.152:0/1721963121 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffb1415c560 0x7ffb140cce50 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c00c7b0 tx=0x7ffb0c00cc70 comp rx=0 tx=0).stop 2024-04-20T11:28:11.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.434+0000 7ffb1c400640 1 -- 172.21.15.152:0/1721963121 shutdown_connections 2024-04-20T11:28:11.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.434+0000 7ffb1c400640 1 --2- 172.21.15.152:0/1721963121 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ffaf406c8f0 0x7ffaf406edb0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:11.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.434+0000 7ffb1c400640 1 --2- 172.21.15.152:0/1721963121 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffb1415c560 0x7ffb140cce50 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:11.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.434+0000 7ffb1c400640 1 --2- 172.21.15.152:0/1721963121 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffb140d0640 0x7ffb140cfb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:11.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.434+0000 7ffb1c400640 1 --2- 172.21.15.152:0/1721963121 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffb140b4500 0x7ffb1414e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:11.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.434+0000 7ffb1c400640 1 -- 172.21.15.152:0/1721963121 >> 172.21.15.152:0/1721963121 conn(0x7ffb14067cf0 msgr2=0x7ffb14159b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:11.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.434+0000 7ffb1c400640 1 -- 172.21.15.152:0/1721963121 shutdown_connections 2024-04-20T11:28:11.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.434+0000 7ffb1c400640 1 -- 172.21.15.152:0/1721963121 wait complete. 2024-04-20T11:28:11.452 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-04-20T11:28:11.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.646+0000 7f7653800640 1 Processor -- start 2024-04-20T11:28:11.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.647+0000 7f7653800640 1 -- start start 2024-04-20T11:28:11.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.648+0000 7f7653800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f764c0cd0d0 0x7f764c0cd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:11.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.648+0000 7f7653800640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f764c071fe0 0x7f764c072470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:11.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.648+0000 7f7653800640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f764c146340 con 0x7f764c145f60 2024-04-20T11:28:11.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.648+0000 7f7653800640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f764c06e9f0 con 0x7f764c071fe0 2024-04-20T11:28:11.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.648+0000 7f7653800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f764c072d50 con 0x7f764c0cd0d0 2024-04-20T11:28:11.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.649+0000 7f7643e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f764c0cd0d0 0x7f764c0cd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:11.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.649+0000 7f7651800640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f764c071fe0 0x7f764c072470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:11.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.649+0000 7f7650e00640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7f764c145f60 0x7f764c06ee80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.152:46552/0 (socket says 172.21.15.152:46552) 2024-04-20T11:28:11.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.649+0000 7f7650e00640 1 -- 172.21.15.152:0/722151148 learned_addr learned my addr 172.21.15.152:0/722151148 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:28:11.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.650+0000 7f7643e00640 1 -- 172.21.15.152:0/722151148 >> v2:172.21.15.175:3300/0 conn(0x7f764c071fe0 msgr2=0x7f764c072470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:11.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.650+0000 7f7643e00640 1 --2- 172.21.15.152:0/722151148 >> v2:172.21.15.175:3300/0 conn(0x7f764c071fe0 0x7f764c072470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:11.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.650+0000 7f7643e00640 1 -- 172.21.15.152:0/722151148 >> v1:172.21.15.175:6789/0 conn(0x7f764c145f60 legacy=0x7f764c06ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:11.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.650+0000 7f7643e00640 1 -- 172.21.15.152:0/722151148 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f764c06e0c0 con 0x7f764c0cd0d0 2024-04-20T11:28:11.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.650+0000 7f7651800640 1 --2- 172.21.15.152:0/722151148 >> v2:172.21.15.175:3300/0 conn(0x7f764c071fe0 0x7f764c072470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:28:11.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.652+0000 7f7643e00640 1 --2- 172.21.15.152:0/722151148 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f764c0cd0d0 0x7f764c0cd550 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f764c06eb70 tx=0x7f763402b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c00b96937185c70 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:11.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.653+0000 7f764be00640 1 -- 172.21.15.152:0/722151148 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7634035020 con 0x7f764c0cd0d0 2024-04-20T11:28:11.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.653+0000 7f764be00640 1 -- 172.21.15.152:0/722151148 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7634031800 con 0x7f764c0cd0d0 2024-04-20T11:28:11.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.653+0000 7f764be00640 1 -- 172.21.15.152:0/722151148 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7634031bd0 con 0x7f764c0cd0d0 2024-04-20T11:28:11.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.654+0000 7f7653800640 1 -- 172.21.15.152:0/722151148 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f764c0cd0d0 msgr2=0x7f764c0cd550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:11.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.654+0000 7f7653800640 1 --2- 172.21.15.152:0/722151148 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f764c0cd0d0 0x7f764c0cd550 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f764c06eb70 tx=0x7f763402b8a0 comp rx=0 tx=0).stop 2024-04-20T11:28:11.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.654+0000 7f7653800640 1 -- 172.21.15.152:0/722151148 shutdown_connections 2024-04-20T11:28:11.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.654+0000 7f7653800640 1 --2- 172.21.15.152:0/722151148 >> v2:172.21.15.175:3300/0 conn(0x7f764c071fe0 0x7f764c072470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:11.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.654+0000 7f7653800640 1 --2- 172.21.15.152:0/722151148 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f764c0cd0d0 0x7f764c0cd550 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:11.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.654+0000 7f7653800640 1 -- 172.21.15.152:0/722151148 >> 172.21.15.152:0/722151148 conn(0x7f764c067cf0 msgr2=0x7f764c06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:11.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.654+0000 7f7653800640 1 -- 172.21.15.152:0/722151148 shutdown_connections 2024-04-20T11:28:11.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.654+0000 7f7653800640 1 -- 172.21.15.152:0/722151148 wait complete. 2024-04-20T11:28:11.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.656+0000 7f7653800640 1 Processor -- start 2024-04-20T11:28:11.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.656+0000 7f7653800640 1 -- start start 2024-04-20T11:28:11.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.657+0000 7f7653800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f764c071fe0 0x7f764c1f31c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:11.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.657+0000 7f7653800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f764c0cd0d0 0x7f764c15da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:11.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.657+0000 7f7653800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f764c145f60 0x7f764c1f0840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:11.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.657+0000 7f7653800640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f764c0729b0 con 0x7f764c145f60 2024-04-20T11:28:11.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.657+0000 7f7653800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f764c06e680 con 0x7f764c0cd0d0 2024-04-20T11:28:11.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.657+0000 7f7653800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f764c072b30 con 0x7f764c071fe0 2024-04-20T11:28:11.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.658+0000 7f7651800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f764c145f60 0x7f764c1f0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:11.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.658+0000 7f7651800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f764c145f60 0x7f764c1f0840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:46056/0 (socket says 172.21.15.152:46056) 2024-04-20T11:28:11.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.658+0000 7f7651800640 1 -- 172.21.15.152:0/3247900100 learned_addr learned my addr 172.21.15.152:0/3247900100 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:11.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.658+0000 7f7650e00640 1 --2- 172.21.15.152:0/3247900100 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f764c071fe0 0x7f764c1f31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:11.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.658+0000 7f7643e00640 1 --2- 172.21.15.152:0/3247900100 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f764c0cd0d0 0x7f764c15da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:11.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.658+0000 7f7651800640 1 -- 172.21.15.152:0/3247900100 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f764c071fe0 msgr2=0x7f764c1f31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:11.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.658+0000 7f7651800640 1 --2- 172.21.15.152:0/3247900100 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f764c071fe0 0x7f764c1f31c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:11.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.658+0000 7f7651800640 1 -- 172.21.15.152:0/3247900100 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f764c0cd0d0 msgr2=0x7f764c15da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:11.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.658+0000 7f7651800640 1 --2- 172.21.15.152:0/3247900100 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f764c0cd0d0 0x7f764c15da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:11.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.658+0000 7f7651800640 1 -- 172.21.15.152:0/3247900100 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f764c15e4e0 con 0x7f764c145f60 2024-04-20T11:28:11.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.659+0000 7f7651800640 1 --2- 172.21.15.152:0/3247900100 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f764c145f60 0x7f764c1f0840 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f764400efc0 tx=0x7f7644009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:11.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.659+0000 7f7649600640 1 -- 172.21.15.152:0/3247900100 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7644018030 con 0x7f764c145f60 2024-04-20T11:28:11.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.659+0000 7f7649600640 1 -- 172.21.15.152:0/3247900100 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f764400cde0 con 0x7f764c145f60 2024-04-20T11:28:11.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.659+0000 7f7649600640 1 -- 172.21.15.152:0/3247900100 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7644003bb0 con 0x7f764c145f60 2024-04-20T11:28:11.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.659+0000 7f7653800640 1 -- 172.21.15.152:0/3247900100 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f764c15e7c0 con 0x7f764c145f60 2024-04-20T11:28:11.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.659+0000 7f7653800640 1 -- 172.21.15.152:0/3247900100 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f764c15ed80 con 0x7f764c145f60 2024-04-20T11:28:11.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.660+0000 7f7653800640 1 -- 172.21.15.152:0/3247900100 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f764c1f0520 con 0x7f764c145f60 2024-04-20T11:28:11.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.661+0000 7f7649600640 1 -- 172.21.15.152:0/3247900100 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f764401b070 con 0x7f764c145f60 2024-04-20T11:28:11.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.662+0000 7f7649600640 1 --2- 172.21.15.152:0/3247900100 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f762806c8f0 0x7f762806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:11.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.662+0000 7f7649600640 1 -- 172.21.15.152:0/3247900100 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7f764408c340 con 0x7f764c145f60 2024-04-20T11:28:11.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.663+0000 7f7650e00640 1 --2- 172.21.15.152:0/3247900100 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f762806c8f0 0x7f762806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:11.664 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.663+0000 7f7650e00640 1 --2- 172.21.15.152:0/3247900100 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f762806c8f0 0x7f762806edb0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f764c0b4df0 tx=0x7f763c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:11.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.667+0000 7f7649600640 1 -- 172.21.15.152:0/3247900100 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f7644010040 con 0x7f764c145f60 2024-04-20T11:28:11.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.940+0000 7f7653800640 1 -- 172.21.15.152:0/3247900100 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7f764c0689f0 con 0x7f764c145f60 2024-04-20T11:28:11.944 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.944+0000 7f7649600640 1 -- 172.21.15.152:0/3247900100 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v4) ==== 90+0+0 (secure 0 0 0) 0x7f764405aa20 con 0x7f764c145f60 2024-04-20T11:28:11.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.947+0000 7f7653800640 1 -- 172.21.15.152:0/3247900100 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f762806c8f0 msgr2=0x7f762806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:11.948 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.947+0000 7f7653800640 1 --2- 172.21.15.152:0/3247900100 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f762806c8f0 0x7f762806edb0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f764c0b4df0 tx=0x7f763c009040 comp rx=0 tx=0).stop 2024-04-20T11:28:11.948 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.947+0000 7f7653800640 1 -- 172.21.15.152:0/3247900100 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f764c145f60 msgr2=0x7f764c1f0840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:11.948 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.947+0000 7f7653800640 1 --2- 172.21.15.152:0/3247900100 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f764c145f60 0x7f764c1f0840 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f764400efc0 tx=0x7f7644009f90 comp rx=0 tx=0).stop 2024-04-20T11:28:11.948 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.947+0000 7f7653800640 1 -- 172.21.15.152:0/3247900100 shutdown_connections 2024-04-20T11:28:11.948 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.947+0000 7f7653800640 1 --2- 172.21.15.152:0/3247900100 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f762806c8f0 0x7f762806edb0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:11.948 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.947+0000 7f7653800640 1 --2- 172.21.15.152:0/3247900100 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f764c145f60 0x7f764c1f0840 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:11.948 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.947+0000 7f7653800640 1 --2- 172.21.15.152:0/3247900100 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f764c0cd0d0 0x7f764c15da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:11.948 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.947+0000 7f7653800640 1 --2- 172.21.15.152:0/3247900100 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f764c071fe0 0x7f764c1f31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:11.948 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.947+0000 7f7653800640 1 -- 172.21.15.152:0/3247900100 >> 172.21.15.152:0/3247900100 conn(0x7f764c067cf0 msgr2=0x7f764c15b2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:11.948 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.947+0000 7f7653800640 1 -- 172.21.15.152:0/3247900100 shutdown_connections 2024-04-20T11:28:11.948 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:11.947+0000 7f7653800640 1 -- 172.21.15.152:0/3247900100 wait complete. 2024-04-20T11:28:11.966 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-04-20T11:28:12.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.156+0000 7ff3a0e00640 1 Processor -- start 2024-04-20T11:28:12.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.156+0000 7ff3a0e00640 1 -- start start 2024-04-20T11:28:12.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.157+0000 7ff3a0e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7ff39c0d33a0 0x7ff39c0cfcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:12.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.157+0000 7ff3a0e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ff39c0d01f0 0x7ff39c0b3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:12.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.157+0000 7ff3a0e00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff39c0d2320 con 0x7ff39c0d0f60 2024-04-20T11:28:12.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.157+0000 7ff3a0e00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7ff39c0d25e0 con 0x7ff39c0d33a0 2024-04-20T11:28:12.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.157+0000 7ff3a0e00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7ff39c0d28c0 con 0x7ff39c0d01f0 2024-04-20T11:28:12.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.158+0000 7ff39aa00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7ff39c0d33a0 0x7ff39c0cfcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:12.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.158+0000 7ff39aa00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7ff39c0d33a0 0x7ff39c0cfcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:46058/0 (socket says 172.21.15.152:46058) 2024-04-20T11:28:12.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.158+0000 7ff39aa00640 1 -- 172.21.15.152:0/1016734313 learned_addr learned my addr 172.21.15.152:0/1016734313 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:12.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.158+0000 7ff39b400640 1 --1- 172.21.15.152:0/1016734313 >> v1:172.21.15.152:6789/0 conn(0x7ff39c0d0f60 0x7ff39c0d1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:41182/0 (socket says 172.21.15.152:41182) 2024-04-20T11:28:12.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.158+0000 7ff39be00640 1 --2- 172.21.15.152:0/1016734313 >> v2:172.21.15.175:3300/0 conn(0x7ff39c0d01f0 0x7ff39c0b3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:12.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.158+0000 7ff39aa00640 1 -- 172.21.15.152:0/1016734313 >> v2:172.21.15.175:3300/0 conn(0x7ff39c0d01f0 msgr2=0x7ff39c0b3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:12.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.158+0000 7ff39aa00640 1 --2- 172.21.15.152:0/1016734313 >> v2:172.21.15.175:3300/0 conn(0x7ff39c0d01f0 0x7ff39c0b3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:12.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.158+0000 7ff39aa00640 1 -- 172.21.15.152:0/1016734313 >> v1:172.21.15.152:6789/0 conn(0x7ff39c0d0f60 legacy=0x7ff39c0d1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:12.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.158+0000 7ff39aa00640 1 -- 172.21.15.152:0/1016734313 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff39c0d2ba0 con 0x7ff39c0d33a0 2024-04-20T11:28:12.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.158+0000 7ff39be00640 1 --2- 172.21.15.152:0/1016734313 >> v2:172.21.15.175:3300/0 conn(0x7ff39c0d01f0 0x7ff39c0b3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:28:12.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.159+0000 7ff39aa00640 1 --2- 172.21.15.152:0/1016734313 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff39c0d33a0 0x7ff39c0cfcb0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7ff39c08e230 tx=0x7ff38802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f782f4518f0a9b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:12.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.160+0000 7ff39a000640 1 -- 172.21.15.152:0/1016734313 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff388034070 con 0x7ff39c0d33a0 2024-04-20T11:28:12.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.160+0000 7ff39a000640 1 -- 172.21.15.152:0/1016734313 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff388030860 con 0x7ff39c0d33a0 2024-04-20T11:28:12.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.160+0000 7ff39a000640 1 -- 172.21.15.152:0/1016734313 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff388030c30 con 0x7ff39c0d33a0 2024-04-20T11:28:12.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.160+0000 7ff3a0e00640 1 -- 172.21.15.152:0/1016734313 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff39c0d33a0 msgr2=0x7ff39c0cfcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:12.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.160+0000 7ff3a0e00640 1 --2- 172.21.15.152:0/1016734313 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff39c0d33a0 0x7ff39c0cfcb0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7ff39c08e230 tx=0x7ff38802b9a0 comp rx=0 tx=0).stop 2024-04-20T11:28:12.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.160+0000 7ff3a0e00640 1 -- 172.21.15.152:0/1016734313 shutdown_connections 2024-04-20T11:28:12.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.160+0000 7ff3a0e00640 1 --2- 172.21.15.152:0/1016734313 >> v2:172.21.15.175:3300/0 conn(0x7ff39c0d01f0 0x7ff39c0b3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:12.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.160+0000 7ff3a0e00640 1 --2- 172.21.15.152:0/1016734313 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff39c0d33a0 0x7ff39c0cfcb0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:12.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.160+0000 7ff3a0e00640 1 -- 172.21.15.152:0/1016734313 >> 172.21.15.152:0/1016734313 conn(0x7ff39c08bfd0 msgr2=0x7ff39c0d45d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:12.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.160+0000 7ff3a0e00640 1 -- 172.21.15.152:0/1016734313 shutdown_connections 2024-04-20T11:28:12.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.161+0000 7ff3a0e00640 1 -- 172.21.15.152:0/1016734313 wait complete. 2024-04-20T11:28:12.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.162+0000 7ff3a0e00640 1 Processor -- start 2024-04-20T11:28:12.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.162+0000 7ff3a0e00640 1 -- start start 2024-04-20T11:28:12.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.163+0000 7ff3a0e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff39c0d01f0 0x7ff39c1f38a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:12.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.163+0000 7ff3a0e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff39c0d0f60 0x7ff39c1f3de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:12.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.163+0000 7ff3a0e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff39c0d33a0 0x7ff39c1f7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:12.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.163+0000 7ff3a0e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7ff39c161fe0 con 0x7ff39c0d01f0 2024-04-20T11:28:12.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.163+0000 7ff3a0e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff39c161e60 con 0x7ff39c0d33a0 2024-04-20T11:28:12.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.163+0000 7ff3a0e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff39c0b4240 con 0x7ff39c0d0f60 2024-04-20T11:28:12.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.164+0000 7ff39b400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff39c0d01f0 0x7ff39c1f38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:12.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.164+0000 7ff39b400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff39c0d01f0 0x7ff39c1f38a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:46072/0 (socket says 172.21.15.152:46072) 2024-04-20T11:28:12.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.164+0000 7ff39b400640 1 -- 172.21.15.152:0/3494984181 learned_addr learned my addr 172.21.15.152:0/3494984181 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:12.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.164+0000 7ff39be00640 1 --2- 172.21.15.152:0/3494984181 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff39c0d33a0 0x7ff39c1f7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:12.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.164+0000 7ff39b400640 1 -- 172.21.15.152:0/3494984181 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff39c0d0f60 msgr2=0x7ff39c1f3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:12.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.164+0000 7ff39b400640 1 --2- 172.21.15.152:0/3494984181 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff39c0d0f60 0x7ff39c1f3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:12.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.164+0000 7ff39b400640 1 -- 172.21.15.152:0/3494984181 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff39c0d33a0 msgr2=0x7ff39c1f7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:12.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.164+0000 7ff39b400640 1 --2- 172.21.15.152:0/3494984181 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff39c0d33a0 0x7ff39c1f7cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:12.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.164+0000 7ff39b400640 1 -- 172.21.15.152:0/3494984181 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff39c1f4320 con 0x7ff39c0d01f0 2024-04-20T11:28:12.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.164+0000 7ff39be00640 1 --2- 172.21.15.152:0/3494984181 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff39c0d33a0 0x7ff39c1f7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:28:12.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.164+0000 7ff39b400640 1 --2- 172.21.15.152:0/3494984181 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff39c0d01f0 0x7ff39c1f38a0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7ff38c00c430 tx=0x7ff38c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:12.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.165+0000 7ff393400640 1 -- 172.21.15.152:0/3494984181 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff38c0074e0 con 0x7ff39c0d01f0 2024-04-20T11:28:12.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.165+0000 7ff393400640 1 -- 172.21.15.152:0/3494984181 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff38c007680 con 0x7ff39c0d01f0 2024-04-20T11:28:12.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.165+0000 7ff393400640 1 -- 172.21.15.152:0/3494984181 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff38c004a00 con 0x7ff39c0d01f0 2024-04-20T11:28:12.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.165+0000 7ff3a0e00640 1 -- 172.21.15.152:0/3494984181 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff39c0cc830 con 0x7ff39c0d01f0 2024-04-20T11:28:12.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.165+0000 7ff3a0e00640 1 -- 172.21.15.152:0/3494984181 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff39c0ccdf0 con 0x7ff39c0d01f0 2024-04-20T11:28:12.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.166+0000 7ff3a0e00640 1 -- 172.21.15.152:0/3494984181 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff39c075e60 con 0x7ff39c0d01f0 2024-04-20T11:28:12.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.167+0000 7ff393400640 1 -- 172.21.15.152:0/3494984181 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff38c01a020 con 0x7ff39c0d01f0 2024-04-20T11:28:12.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.168+0000 7ff393400640 1 --2- 172.21.15.152:0/3494984181 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff37c06c8f0 0x7ff37c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:12.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.168+0000 7ff393400640 1 -- 172.21.15.152:0/3494984181 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7ff38c08afe0 con 0x7ff39c0d01f0 2024-04-20T11:28:12.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.168+0000 7ff39aa00640 1 --2- 172.21.15.152:0/3494984181 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff37c06c8f0 0x7ff37c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:12.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.169+0000 7ff39aa00640 1 --2- 172.21.15.152:0/3494984181 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff37c06c8f0 0x7ff37c06edb0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff388036000 tx=0x7ff388030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:12.172 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.172+0000 7ff393400640 1 -- 172.21.15.152:0/3494984181 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ff38c0597b0 con 0x7ff39c0d01f0 2024-04-20T11:28:12.439 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.438+0000 7ff3a0e00640 1 -- 172.21.15.152:0/3494984181 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7ff39c0cd030 con 0x7ff39c0d01f0 2024-04-20T11:28:12.442 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.442+0000 7ff393400640 1 -- 172.21.15.152:0/3494984181 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v5) ==== 90+0+0 (secure 0 0 0) 0x7ff38c0590b0 con 0x7ff39c0d01f0 2024-04-20T11:28:12.445 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.444+0000 7ff3a0e00640 1 -- 172.21.15.152:0/3494984181 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff37c06c8f0 msgr2=0x7ff37c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:12.445 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.444+0000 7ff3a0e00640 1 --2- 172.21.15.152:0/3494984181 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff37c06c8f0 0x7ff37c06edb0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff388036000 tx=0x7ff388030000 comp rx=0 tx=0).stop 2024-04-20T11:28:12.445 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.444+0000 7ff3a0e00640 1 -- 172.21.15.152:0/3494984181 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff39c0d01f0 msgr2=0x7ff39c1f38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:12.445 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.444+0000 7ff3a0e00640 1 --2- 172.21.15.152:0/3494984181 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff39c0d01f0 0x7ff39c1f38a0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7ff38c00c430 tx=0x7ff38c00c8f0 comp rx=0 tx=0).stop 2024-04-20T11:28:12.445 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.444+0000 7ff3a0e00640 1 -- 172.21.15.152:0/3494984181 shutdown_connections 2024-04-20T11:28:12.445 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.444+0000 7ff3a0e00640 1 --2- 172.21.15.152:0/3494984181 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff37c06c8f0 0x7ff37c06edb0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:12.445 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.444+0000 7ff3a0e00640 1 --2- 172.21.15.152:0/3494984181 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff39c0d33a0 0x7ff39c1f7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:12.446 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.444+0000 7ff3a0e00640 1 --2- 172.21.15.152:0/3494984181 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff39c0d0f60 0x7ff39c1f3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:12.446 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.444+0000 7ff3a0e00640 1 --2- 172.21.15.152:0/3494984181 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff39c0d01f0 0x7ff39c1f38a0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:12.446 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.444+0000 7ff3a0e00640 1 -- 172.21.15.152:0/3494984181 >> 172.21.15.152:0/3494984181 conn(0x7ff39c08bfd0 msgr2=0x7ff39c0d4180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:12.446 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.445+0000 7ff3a0e00640 1 -- 172.21.15.152:0/3494984181 shutdown_connections 2024-04-20T11:28:12.446 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.445+0000 7ff3a0e00640 1 -- 172.21.15.152:0/3494984181 wait complete. 2024-04-20T11:28:12.463 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-04-20T11:28:12.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.650+0000 7fec9bc00640 1 Processor -- start 2024-04-20T11:28:12.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.650+0000 7fec9bc00640 1 -- start start 2024-04-20T11:28:12.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.652+0000 7fec9bc00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fec9415c560 0x7fec940b3df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:12.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.652+0000 7fec9bc00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec9415fd20 con 0x7fec940d0810 2024-04-20T11:28:12.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.652+0000 7fec9bc00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec9415ef60 con 0x7fec940b4330 2024-04-20T11:28:12.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.652+0000 7fec9bc00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fec9415f240 con 0x7fec9415c560 2024-04-20T11:28:12.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.652+0000 7fec99c00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fec940d0810 0x7fec940ada90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:41186/0 (socket says 172.21.15.152:41186) 2024-04-20T11:28:12.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.652+0000 7fec99c00640 1 -- 172.21.15.152:0/311872114 learned_addr learned my addr 172.21.15.152:0/311872114 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:28:12.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.653+0000 7fec99200640 1 --2- 172.21.15.152:0/311872114 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fec9415c560 0x7fec940b3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:12.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.653+0000 7fec93400640 1 -- 172.21.15.152:0/311872114 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4239789007 0 0) 0x7fec9415fd20 con 0x7fec940d0810 2024-04-20T11:28:12.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.653+0000 7fec93400640 1 -- 172.21.15.152:0/311872114 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fec7c003520 con 0x7fec940d0810 2024-04-20T11:28:12.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.653+0000 7fec93400640 1 -- 172.21.15.152:0/311872114 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4097818473 0 0) 0x7fec7c003520 con 0x7fec940d0810 2024-04-20T11:28:12.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.653+0000 7fec93400640 1 -- 172.21.15.152:0/311872114 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fec9415c560 msgr2=0x7fec940b3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:12.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.653+0000 7fec93400640 1 --2- 172.21.15.152:0/311872114 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fec9415c560 0x7fec940b3df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:12.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.653+0000 7fec93400640 1 -- 172.21.15.152:0/311872114 >> v1:172.21.15.175:6789/0 conn(0x7fec940b4330 legacy=0x7fec940d0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:12.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.653+0000 7fec93400640 1 -- 172.21.15.152:0/311872114 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec9415f520 con 0x7fec940d0810 2024-04-20T11:28:12.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.653+0000 7fec93400640 1 -- 172.21.15.152:0/311872114 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7fec8c0019f0 con 0x7fec940d0810 2024-04-20T11:28:12.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.654+0000 7fec93400640 1 -- 172.21.15.152:0/311872114 >> v1:172.21.15.152:6789/0 conn(0x7fec940d0810 legacy=0x7fec940ada90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:12.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.654+0000 7fec93400640 1 --2- 172.21.15.152:0/311872114 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fec7c003a50 0x7fec7c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:12.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.654+0000 7fec93400640 1 --2- 172.21.15.152:0/311872114 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fec7c004800 0x7fec7c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:12.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.654+0000 7fec93400640 1 --2- 172.21.15.152:0/311872114 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fec9415c560 0x7fec7c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:12.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.654+0000 7fec93400640 1 -- 172.21.15.152:0/311872114 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fec9415f240 con 0x7fec7c003a50 2024-04-20T11:28:12.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.654+0000 7fec93400640 1 -- 172.21.15.152:0/311872114 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fec7c005520 con 0x7fec7c004800 2024-04-20T11:28:12.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.654+0000 7fec93400640 1 -- 172.21.15.152:0/311872114 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fec7c00bf00 con 0x7fec9415c560 2024-04-20T11:28:12.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.654+0000 7fec99200640 1 --2- 172.21.15.152:0/311872114 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fec7c003a50 0x7fec7c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:12.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.654+0000 7fec93e00640 1 --2- 172.21.15.152:0/311872114 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fec7c004800 0x7fec7c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:12.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.654+0000 7fec99200640 1 -- 172.21.15.152:0/311872114 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fec9415c560 msgr2=0x7fec7c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:12.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.654+0000 7fec99200640 1 --2- 172.21.15.152:0/311872114 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fec9415c560 0x7fec7c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:12.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.654+0000 7fec99200640 1 -- 172.21.15.152:0/311872114 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fec7c004800 msgr2=0x7fec7c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:12.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.654+0000 7fec99200640 1 --2- 172.21.15.152:0/311872114 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fec7c004800 0x7fec7c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:12.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.654+0000 7fec99200640 1 -- 172.21.15.152:0/311872114 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec7c003520 con 0x7fec7c003a50 2024-04-20T11:28:12.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.654+0000 7fec93e00640 1 --2- 172.21.15.152:0/311872114 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fec7c004800 0x7fec7c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:28:12.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.656+0000 7fec99200640 1 --2- 172.21.15.152:0/311872114 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fec7c003a50 0x7fec7c003e70 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fec88009ac0 tx=0x7fec7c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7b6312735d4ea16 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:12.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.656+0000 7fec93400640 1 -- 172.21.15.152:0/311872114 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec88039070 con 0x7fec7c003a50 2024-04-20T11:28:12.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.656+0000 7fec93400640 1 -- 172.21.15.152:0/311872114 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec8802c710 con 0x7fec7c003a50 2024-04-20T11:28:12.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.656+0000 7fec93400640 1 -- 172.21.15.152:0/311872114 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec88009690 con 0x7fec7c003a50 2024-04-20T11:28:12.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.657+0000 7fec9bc00640 1 -- 172.21.15.152:0/311872114 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fec7c003a50 msgr2=0x7fec7c003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:12.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.657+0000 7fec9bc00640 1 --2- 172.21.15.152:0/311872114 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fec7c003a50 0x7fec7c003e70 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fec88009ac0 tx=0x7fec7c005830 comp rx=0 tx=0).stop 2024-04-20T11:28:12.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.657+0000 7fec99c00640 1 -- 172.21.15.152:0/311872114 reap_dead start 2024-04-20T11:28:12.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.657+0000 7fec9bc00640 1 -- 172.21.15.152:0/311872114 shutdown_connections 2024-04-20T11:28:12.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.657+0000 7fec9bc00640 1 -- 172.21.15.152:0/311872114 >> 172.21.15.152:0/311872114 conn(0x7fec94067cf0 msgr2=0x7fec9406b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:12.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.657+0000 7fec9bc00640 1 -- 172.21.15.152:0/311872114 shutdown_connections 2024-04-20T11:28:12.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.657+0000 7fec9bc00640 1 -- 172.21.15.152:0/311872114 wait complete. 2024-04-20T11:28:12.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.659+0000 7fec9bc00640 1 Processor -- start 2024-04-20T11:28:12.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.659+0000 7fec9bc00640 1 -- start start 2024-04-20T11:28:12.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.660+0000 7fec9bc00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fec7c003a50 0x7fec94156440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:12.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.660+0000 7fec9bc00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fec94152c80 0x7fec94149bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:12.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.660+0000 7fec9bc00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fec9415d4f0 0x7fec9415d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:12.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.660+0000 7fec9bc00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fec940b3df0 con 0x7fec94152c80 2024-04-20T11:28:12.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.660+0000 7fec9bc00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fec940d0dc0 con 0x7fec9415d4f0 2024-04-20T11:28:12.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.660+0000 7fec9bc00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fec940b3f70 con 0x7fec7c003a50 2024-04-20T11:28:12.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.660+0000 7fec93e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fec94152c80 0x7fec94149bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:12.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.660+0000 7fec93e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fec94152c80 0x7fec94149bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:46094/0 (socket says 172.21.15.152:46094) 2024-04-20T11:28:12.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.660+0000 7fec93e00640 1 -- 172.21.15.152:0/820700044 learned_addr learned my addr 172.21.15.152:0/820700044 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:12.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.660+0000 7fec93e00640 1 -- 172.21.15.152:0/820700044 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fec7c003a50 msgr2=0x7fec94156440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:12.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.660+0000 7fec93e00640 1 --2- 172.21.15.152:0/820700044 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fec7c003a50 0x7fec94156440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:12.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.660+0000 7fec93e00640 1 -- 172.21.15.152:0/820700044 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fec9415d4f0 msgr2=0x7fec9415d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:12.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.660+0000 7fec99c00640 1 --2- 172.21.15.152:0/820700044 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fec9415d4f0 0x7fec9415d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:12.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.660+0000 7fec93e00640 1 --2- 172.21.15.152:0/820700044 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fec9415d4f0 0x7fec9415d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:12.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.660+0000 7fec93e00640 1 -- 172.21.15.152:0/820700044 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec941f7bb0 con 0x7fec94152c80 2024-04-20T11:28:12.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.660+0000 7fec99c00640 1 --2- 172.21.15.152:0/820700044 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fec9415d4f0 0x7fec9415d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:28:12.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.661+0000 7fec93e00640 1 --2- 172.21.15.152:0/820700044 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fec94152c80 0x7fec94149bc0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fec8000d7d0 tx=0x7fec8000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:12.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.661+0000 7fec90c00640 1 -- 172.21.15.152:0/820700044 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec80005070 con 0x7fec94152c80 2024-04-20T11:28:12.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.661+0000 7fec90c00640 1 -- 172.21.15.152:0/820700044 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec80005230 con 0x7fec94152c80 2024-04-20T11:28:12.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.661+0000 7fec9bc00640 1 -- 172.21.15.152:0/820700044 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec941f7e90 con 0x7fec94152c80 2024-04-20T11:28:12.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.661+0000 7fec90c00640 1 -- 172.21.15.152:0/820700044 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec80007030 con 0x7fec94152c80 2024-04-20T11:28:12.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.661+0000 7fec9bc00640 1 -- 172.21.15.152:0/820700044 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec941f8450 con 0x7fec94152c80 2024-04-20T11:28:12.664 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.662+0000 7fec9bc00640 1 -- 172.21.15.152:0/820700044 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec940689f0 con 0x7fec94152c80 2024-04-20T11:28:12.664 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.663+0000 7fec90c00640 1 -- 172.21.15.152:0/820700044 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fec8001a020 con 0x7fec94152c80 2024-04-20T11:28:12.664 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.664+0000 7fec90c00640 1 --2- 172.21.15.152:0/820700044 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fec7406c8f0 0x7fec7406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:12.664 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.664+0000 7fec90c00640 1 -- 172.21.15.152:0/820700044 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7fec8008afe0 con 0x7fec94152c80 2024-04-20T11:28:12.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.664+0000 7fec99200640 1 --2- 172.21.15.152:0/820700044 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fec7406c8f0 0x7fec7406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:12.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.665+0000 7fec99200640 1 --2- 172.21.15.152:0/820700044 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fec7406c8f0 0x7fec7406edb0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fec88036000 tx=0x7fec8802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:12.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.668+0000 7fec90c00640 1 -- 172.21.15.152:0/820700044 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fec800597f0 con 0x7fec94152c80 2024-04-20T11:28:12.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.921+0000 7fec9bc00640 1 -- 172.21.15.152:0/820700044 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7fec940b1b80 con 0x7fec94152c80 2024-04-20T11:28:12.925 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.925+0000 7fec90c00640 1 -- 172.21.15.152:0/820700044 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v6) ==== 90+0+0 (secure 0 0 0) 0x7fec800590f0 con 0x7fec94152c80 2024-04-20T11:28:12.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.927+0000 7fec9bc00640 1 -- 172.21.15.152:0/820700044 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fec7406c8f0 msgr2=0x7fec7406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:12.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.927+0000 7fec9bc00640 1 --2- 172.21.15.152:0/820700044 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fec7406c8f0 0x7fec7406edb0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fec88036000 tx=0x7fec8802e000 comp rx=0 tx=0).stop 2024-04-20T11:28:12.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.927+0000 7fec9bc00640 1 -- 172.21.15.152:0/820700044 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fec94152c80 msgr2=0x7fec94149bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:12.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.927+0000 7fec9bc00640 1 --2- 172.21.15.152:0/820700044 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fec94152c80 0x7fec94149bc0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fec8000d7d0 tx=0x7fec8000dc90 comp rx=0 tx=0).stop 2024-04-20T11:28:12.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.928+0000 7fec9bc00640 1 -- 172.21.15.152:0/820700044 shutdown_connections 2024-04-20T11:28:12.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.928+0000 7fec9bc00640 1 --2- 172.21.15.152:0/820700044 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fec7406c8f0 0x7fec7406edb0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:12.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.928+0000 7fec9bc00640 1 --2- 172.21.15.152:0/820700044 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fec9415d4f0 0x7fec9415d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:12.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.928+0000 7fec9bc00640 1 --2- 172.21.15.152:0/820700044 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fec94152c80 0x7fec94149bc0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:12.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.928+0000 7fec9bc00640 1 --2- 172.21.15.152:0/820700044 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fec7c003a50 0x7fec94156440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:12.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.928+0000 7fec9bc00640 1 -- 172.21.15.152:0/820700044 >> 172.21.15.152:0/820700044 conn(0x7fec94067cf0 msgr2=0x7fec9415ae90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:12.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.928+0000 7fec9bc00640 1 -- 172.21.15.152:0/820700044 shutdown_connections 2024-04-20T11:28:12.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:12.928+0000 7fec9bc00640 1 -- 172.21.15.152:0/820700044 wait complete. 2024-04-20T11:28:12.946 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-04-20T11:28:13.146 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.145+0000 7f1818c00640 1 Processor -- start 2024-04-20T11:28:13.146 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.145+0000 7f1818c00640 1 -- start start 2024-04-20T11:28:13.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.147+0000 7f1818c00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f1814145f00 0x7f18140cd0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:13.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.147+0000 7f1818c00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f18140cd610 0x7f181406e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:13.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.147+0000 7f1818c00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f181406f110 con 0x7f181406e900 2024-04-20T11:28:13.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.147+0000 7f1818c00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f18140cda80 con 0x7f1814145f00 2024-04-20T11:28:13.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.147+0000 7f1818c00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f18140cf9f0 con 0x7f18140cd610 2024-04-20T11:28:13.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.147+0000 7f1812800640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f181406e900 0x7f18141601c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:41196/0 (socket says 172.21.15.152:41196) 2024-04-20T11:28:13.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.147+0000 7f1812800640 1 -- 172.21.15.152:0/3372271332 learned_addr learned my addr 172.21.15.152:0/3372271332 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:28:13.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.147+0000 7f1811e00640 1 --2- 172.21.15.152:0/3372271332 >> v2:172.21.15.152:3300/0 conn(0x7f1814145f00 0x7f18140cd0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:13.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.148+0000 7f1809400640 1 --2- 172.21.15.152:0/3372271332 >> v2:172.21.15.175:3300/0 conn(0x7f18140cd610 0x7f181406e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:13.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.148+0000 7f1811400640 1 -- 172.21.15.152:0/3372271332 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3855199362 0 0) 0x7f181406f110 con 0x7f181406e900 2024-04-20T11:28:13.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.148+0000 7f1811400640 1 -- 172.21.15.152:0/3372271332 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f17f8003520 con 0x7f181406e900 2024-04-20T11:28:13.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.148+0000 7f1811e00640 1 -- 172.21.15.152:0/3372271332 >> v2:172.21.15.175:3300/0 conn(0x7f18140cd610 msgr2=0x7f181406e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:13.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.148+0000 7f1811e00640 1 --2- 172.21.15.152:0/3372271332 >> v2:172.21.15.175:3300/0 conn(0x7f18140cd610 0x7f181406e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:13.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.148+0000 7f1811e00640 1 -- 172.21.15.152:0/3372271332 >> v1:172.21.15.152:6789/0 conn(0x7f181406e900 legacy=0x7f18141601c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:13.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.148+0000 7f1811e00640 1 -- 172.21.15.152:0/3372271332 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f181406f3f0 con 0x7f1814145f00 2024-04-20T11:28:13.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.149+0000 7f1811e00640 1 --2- 172.21.15.152:0/3372271332 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1814145f00 0x7f18140cd0d0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f1814160120 tx=0x7f180402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5f39f3fc2ec9054 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:13.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.149+0000 7f1811400640 1 -- 172.21.15.152:0/3372271332 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1804039070 con 0x7f1814145f00 2024-04-20T11:28:13.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.149+0000 7f1811400640 1 -- 172.21.15.152:0/3372271332 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f180402cc80 con 0x7f1814145f00 2024-04-20T11:28:13.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.149+0000 7f1811400640 1 -- 172.21.15.152:0/3372271332 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1804034660 con 0x7f1814145f00 2024-04-20T11:28:13.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.150+0000 7f1818c00640 1 -- 172.21.15.152:0/3372271332 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1814145f00 msgr2=0x7f18140cd0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:13.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.150+0000 7f1818c00640 1 --2- 172.21.15.152:0/3372271332 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1814145f00 0x7f18140cd0d0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f1814160120 tx=0x7f180402bed0 comp rx=0 tx=0).stop 2024-04-20T11:28:13.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.150+0000 7f1818c00640 1 -- 172.21.15.152:0/3372271332 shutdown_connections 2024-04-20T11:28:13.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.150+0000 7f1818c00640 1 --2- 172.21.15.152:0/3372271332 >> v2:172.21.15.175:3300/0 conn(0x7f18140cd610 0x7f181406e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:13.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.150+0000 7f1818c00640 1 --2- 172.21.15.152:0/3372271332 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1814145f00 0x7f18140cd0d0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:13.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.150+0000 7f1818c00640 1 -- 172.21.15.152:0/3372271332 >> 172.21.15.152:0/3372271332 conn(0x7f1814067cf0 msgr2=0x7f181406b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:13.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.150+0000 7f1818c00640 1 -- 172.21.15.152:0/3372271332 shutdown_connections 2024-04-20T11:28:13.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.150+0000 7f1818c00640 1 -- 172.21.15.152:0/3372271332 wait complete. 2024-04-20T11:28:13.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.152+0000 7f1818c00640 1 Processor -- start 2024-04-20T11:28:13.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.152+0000 7f1818c00640 1 -- start start 2024-04-20T11:28:13.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.153+0000 7f1818c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f181406e900 0x7f18141f2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:13.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.153+0000 7f1818c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f18140cd610 0x7f18141f6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:13.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.153+0000 7f1818c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1814145f00 0x7f181415bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:13.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.153+0000 7f1818c00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f18141462e0 con 0x7f1814145f00 2024-04-20T11:28:13.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.153+0000 7f1818c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f18140cf680 con 0x7f181406e900 2024-04-20T11:28:13.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.153+0000 7f1818c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f181406ed50 con 0x7f18140cd610 2024-04-20T11:28:13.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.153+0000 7f1812800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1814145f00 0x7f181415bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:13.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.153+0000 7f1812800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1814145f00 0x7f181415bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:46114/0 (socket says 172.21.15.152:46114) 2024-04-20T11:28:13.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.153+0000 7f1812800640 1 -- 172.21.15.152:0/644441146 learned_addr learned my addr 172.21.15.152:0/644441146 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:13.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.153+0000 7f1811e00640 1 --2- 172.21.15.152:0/644441146 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f181406e900 0x7f18141f2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:13.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.153+0000 7f1809400640 1 --2- 172.21.15.152:0/644441146 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f18140cd610 0x7f18141f6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:13.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.154+0000 7f1812800640 1 -- 172.21.15.152:0/644441146 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f18140cd610 msgr2=0x7f18141f6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:13.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.154+0000 7f1812800640 1 --2- 172.21.15.152:0/644441146 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f18140cd610 0x7f18141f6160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:13.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.154+0000 7f1812800640 1 -- 172.21.15.152:0/644441146 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f181406e900 msgr2=0x7f18141f2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:13.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.154+0000 7f1812800640 1 --2- 172.21.15.152:0/644441146 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f181406e900 0x7f18141f2c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:13.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.154+0000 7f1812800640 1 -- 172.21.15.152:0/644441146 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f181415c3e0 con 0x7f1814145f00 2024-04-20T11:28:13.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.154+0000 7f1811e00640 1 --2- 172.21.15.152:0/644441146 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f181406e900 0x7f18141f2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:28:13.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.154+0000 7f1809400640 1 --2- 172.21.15.152:0/644441146 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f18140cd610 0x7f18141f6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:28:13.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.154+0000 7f1812800640 1 --2- 172.21.15.152:0/644441146 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1814145f00 0x7f181415bea0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f180c00c3c0 tx=0x7f180c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:13.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.154+0000 7f180aa00640 1 -- 172.21.15.152:0/644441146 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f180c017070 con 0x7f1814145f00 2024-04-20T11:28:13.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.154+0000 7f180aa00640 1 -- 172.21.15.152:0/644441146 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f180c005ce0 con 0x7f1814145f00 2024-04-20T11:28:13.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.154+0000 7f180aa00640 1 -- 172.21.15.152:0/644441146 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f180c007310 con 0x7f1814145f00 2024-04-20T11:28:13.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.154+0000 7f1818c00640 1 -- 172.21.15.152:0/644441146 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f181413a080 con 0x7f1814145f00 2024-04-20T11:28:13.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.154+0000 7f1818c00640 1 -- 172.21.15.152:0/644441146 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f181413a640 con 0x7f1814145f00 2024-04-20T11:28:13.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.155+0000 7f1818c00640 1 -- 172.21.15.152:0/644441146 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f181413a880 con 0x7f1814145f00 2024-04-20T11:28:13.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.156+0000 7f180aa00640 1 -- 172.21.15.152:0/644441146 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f180c01a070 con 0x7f1814145f00 2024-04-20T11:28:13.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.157+0000 7f180aa00640 1 --2- 172.21.15.152:0/644441146 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f17f006c8f0 0x7f17f006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:13.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.158+0000 7f180aa00640 1 -- 172.21.15.152:0/644441146 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7f180c08ac40 con 0x7f1814145f00 2024-04-20T11:28:13.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.158+0000 7f1811e00640 1 --2- 172.21.15.152:0/644441146 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f17f006c8f0 0x7f17f006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:13.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.158+0000 7f1811e00640 1 --2- 172.21.15.152:0/644441146 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f17f006c8f0 0x7f17f006edb0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1804036000 tx=0x7f1804030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:13.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.162+0000 7f180aa00640 1 -- 172.21.15.152:0/644441146 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f181413a880 con 0x7f1814145f00 2024-04-20T11:28:13.420 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.420+0000 7f1818c00640 1 -- 172.21.15.152:0/644441146 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7f18140689f0 con 0x7f1814145f00 2024-04-20T11:28:13.424 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.424+0000 7f180aa00640 1 -- 172.21.15.152:0/644441146 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v7) ==== 97+0+0 (secure 0 0 0) 0x7f181415c3e0 con 0x7f1814145f00 2024-04-20T11:28:13.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.426+0000 7f1818c00640 1 -- 172.21.15.152:0/644441146 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f17f006c8f0 msgr2=0x7f17f006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:13.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.426+0000 7f1818c00640 1 --2- 172.21.15.152:0/644441146 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f17f006c8f0 0x7f17f006edb0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1804036000 tx=0x7f1804030000 comp rx=0 tx=0).stop 2024-04-20T11:28:13.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.426+0000 7f1818c00640 1 -- 172.21.15.152:0/644441146 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1814145f00 msgr2=0x7f181415bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:13.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.426+0000 7f1818c00640 1 --2- 172.21.15.152:0/644441146 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1814145f00 0x7f181415bea0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f180c00c3c0 tx=0x7f180c00c880 comp rx=0 tx=0).stop 2024-04-20T11:28:13.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.426+0000 7f1818c00640 1 -- 172.21.15.152:0/644441146 shutdown_connections 2024-04-20T11:28:13.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.426+0000 7f1818c00640 1 --2- 172.21.15.152:0/644441146 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f17f006c8f0 0x7f17f006edb0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:13.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.426+0000 7f1818c00640 1 --2- 172.21.15.152:0/644441146 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1814145f00 0x7f181415bea0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:13.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.426+0000 7f1818c00640 1 --2- 172.21.15.152:0/644441146 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f18140cd610 0x7f18141f6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:13.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.426+0000 7f1818c00640 1 --2- 172.21.15.152:0/644441146 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f181406e900 0x7f18141f2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:13.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.426+0000 7f1818c00640 1 -- 172.21.15.152:0/644441146 >> 172.21.15.152:0/644441146 conn(0x7f1814067cf0 msgr2=0x7f18140b18e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:13.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.426+0000 7f1818c00640 1 -- 172.21.15.152:0/644441146 shutdown_connections 2024-04-20T11:28:13.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.426+0000 7f1818c00640 1 -- 172.21.15.152:0/644441146 wait complete. 2024-04-20T11:28:13.445 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-04-20T11:28:13.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.634+0000 7fa70f000640 1 Processor -- start 2024-04-20T11:28:13.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.634+0000 7fa70f000640 1 -- start start 2024-04-20T11:28:13.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.635+0000 7fa70f000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa708145f60 0x7fa70806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:13.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.635+0000 7fa70f000640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fa7080cd0d0 0x7fa7080cd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:13.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.635+0000 7fa70f000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fa708071fe0 0x7fa708072470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:13.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.635+0000 7fa70f000640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fa70806e9f0 con 0x7fa708071fe0 2024-04-20T11:28:13.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.635+0000 7fa70f000640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fa708146340 con 0x7fa7080cd0d0 2024-04-20T11:28:13.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.635+0000 7fa70f000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa70806f3c0 con 0x7fa708145f60 2024-04-20T11:28:13.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.636+0000 7fa70d000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fa708071fe0 0x7fa708072470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:13.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.636+0000 7fa70d000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fa708071fe0 0x7fa708072470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:46128/0 (socket says 172.21.15.152:46128) 2024-04-20T11:28:13.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.636+0000 7fa70d000640 1 -- 172.21.15.152:0/2239512853 learned_addr learned my addr 172.21.15.152:0/2239512853 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:13.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.636+0000 7fa707e00640 1 --2- 172.21.15.152:0/2239512853 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa708145f60 0x7fa70806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:13.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.636+0000 7fa6ffe00640 1 --2- 172.21.15.152:0/2239512853 >> v2:172.21.15.175:3300/0 conn(0x7fa7080cd0d0 0x7fa7080cd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:13.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.636+0000 7fa70d000640 1 -- 172.21.15.152:0/2239512853 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa708145f60 msgr2=0x7fa70806ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:13.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.636+0000 7fa70d000640 1 --2- 172.21.15.152:0/2239512853 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa708145f60 0x7fa70806ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:13.638 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.636+0000 7fa70d000640 1 -- 172.21.15.152:0/2239512853 >> v2:172.21.15.175:3300/0 conn(0x7fa7080cd0d0 msgr2=0x7fa7080cd550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:13.638 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.636+0000 7fa70d000640 1 --2- 172.21.15.152:0/2239512853 >> v2:172.21.15.175:3300/0 conn(0x7fa7080cd0d0 0x7fa7080cd550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:13.638 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.636+0000 7fa70d000640 1 -- 172.21.15.152:0/2239512853 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa708072d50 con 0x7fa708071fe0 2024-04-20T11:28:13.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.637+0000 7fa70d000640 1 --2- 172.21.15.152:0/2239512853 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa708071fe0 0x7fa708072470 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fa7080b5890 tx=0x7fa70002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33f67513a3278abc server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:13.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.637+0000 7fa707400640 1 -- 172.21.15.152:0/2239512853 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa70002ca70 con 0x7fa708071fe0 2024-04-20T11:28:13.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.637+0000 7fa707400640 1 -- 172.21.15.152:0/2239512853 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa70002cc10 con 0x7fa708071fe0 2024-04-20T11:28:13.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.637+0000 7fa707400640 1 -- 172.21.15.152:0/2239512853 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7000343f0 con 0x7fa708071fe0 2024-04-20T11:28:13.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.638+0000 7fa70f000640 1 -- 172.21.15.152:0/2239512853 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa708071fe0 msgr2=0x7fa708072470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:13.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.638+0000 7fa70f000640 1 --2- 172.21.15.152:0/2239512853 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa708071fe0 0x7fa708072470 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fa7080b5890 tx=0x7fa70002bed0 comp rx=0 tx=0).stop 2024-04-20T11:28:13.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.638+0000 7fa70f000640 1 -- 172.21.15.152:0/2239512853 shutdown_connections 2024-04-20T11:28:13.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.638+0000 7fa70f000640 1 --2- 172.21.15.152:0/2239512853 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa708071fe0 0x7fa708072470 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:13.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.638+0000 7fa70f000640 1 --2- 172.21.15.152:0/2239512853 >> v2:172.21.15.175:3300/0 conn(0x7fa7080cd0d0 0x7fa7080cd550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:13.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.638+0000 7fa70f000640 1 --2- 172.21.15.152:0/2239512853 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa708145f60 0x7fa70806ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:13.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.638+0000 7fa70f000640 1 -- 172.21.15.152:0/2239512853 >> 172.21.15.152:0/2239512853 conn(0x7fa708067cf0 msgr2=0x7fa70806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:13.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.638+0000 7fa70f000640 1 -- 172.21.15.152:0/2239512853 shutdown_connections 2024-04-20T11:28:13.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.638+0000 7fa70f000640 1 -- 172.21.15.152:0/2239512853 wait complete. 2024-04-20T11:28:13.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.640+0000 7fa70f000640 1 Processor -- start 2024-04-20T11:28:13.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.640+0000 7fa70f000640 1 -- start start 2024-04-20T11:28:13.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.641+0000 7fa70f000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa708071fe0 0x7fa708141440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:13.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.641+0000 7fa70f000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa7080cd0d0 0x7fa70815e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:13.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.641+0000 7fa70f000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa708145f60 0x7fa7080ae900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:13.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.641+0000 7fa70f000640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fa70806e680 con 0x7fa7080cd0d0 2024-04-20T11:28:13.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.641+0000 7fa70f000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa70806e3a0 con 0x7fa708071fe0 2024-04-20T11:28:13.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.641+0000 7fa70f000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa7080729b0 con 0x7fa708145f60 2024-04-20T11:28:13.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.641+0000 7fa6ffe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa7080cd0d0 0x7fa70815e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:13.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.641+0000 7fa6ffe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa7080cd0d0 0x7fa70815e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:46144/0 (socket says 172.21.15.152:46144) 2024-04-20T11:28:13.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.641+0000 7fa6ffe00640 1 -- 172.21.15.152:0/3401972988 learned_addr learned my addr 172.21.15.152:0/3401972988 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:13.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.642+0000 7fa707e00640 1 --2- 172.21.15.152:0/3401972988 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa708071fe0 0x7fa708141440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:13.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.642+0000 7fa70d000640 1 --2- 172.21.15.152:0/3401972988 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa708145f60 0x7fa7080ae900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:13.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.642+0000 7fa6ffe00640 1 -- 172.21.15.152:0/3401972988 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa708145f60 msgr2=0x7fa7080ae900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:13.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.642+0000 7fa6ffe00640 1 --2- 172.21.15.152:0/3401972988 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa708145f60 0x7fa7080ae900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:13.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.642+0000 7fa6ffe00640 1 -- 172.21.15.152:0/3401972988 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa708071fe0 msgr2=0x7fa708141440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:13.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.642+0000 7fa6ffe00640 1 --2- 172.21.15.152:0/3401972988 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa708071fe0 0x7fa708141440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:13.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.642+0000 7fa6ffe00640 1 -- 172.21.15.152:0/3401972988 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa70815cb30 con 0x7fa7080cd0d0 2024-04-20T11:28:13.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.642+0000 7fa707e00640 1 --2- 172.21.15.152:0/3401972988 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa708071fe0 0x7fa708141440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:28:13.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.642+0000 7fa6ffe00640 1 --2- 172.21.15.152:0/3401972988 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa7080cd0d0 0x7fa70815e790 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fa6f000d800 tx=0x7fa6f000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:13.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.642+0000 7fa704c00640 1 -- 172.21.15.152:0/3401972988 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6f0018020 con 0x7fa7080cd0d0 2024-04-20T11:28:13.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.642+0000 7fa704c00640 1 -- 172.21.15.152:0/3401972988 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6f00096a0 con 0x7fa7080cd0d0 2024-04-20T11:28:13.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.642+0000 7fa70f000640 1 -- 172.21.15.152:0/3401972988 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa70815cdd0 con 0x7fa7080cd0d0 2024-04-20T11:28:13.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.642+0000 7fa704c00640 1 -- 172.21.15.152:0/3401972988 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6f0009a70 con 0x7fa7080cd0d0 2024-04-20T11:28:13.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.643+0000 7fa70f000640 1 -- 172.21.15.152:0/3401972988 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa708141980 con 0x7fa7080cd0d0 2024-04-20T11:28:13.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.644+0000 7fa70f000640 1 -- 172.21.15.152:0/3401972988 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7080689f0 con 0x7fa7080cd0d0 2024-04-20T11:28:13.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.644+0000 7fa704c00640 1 -- 172.21.15.152:0/3401972988 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa6f001b020 con 0x7fa7080cd0d0 2024-04-20T11:28:13.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.645+0000 7fa704c00640 1 --2- 172.21.15.152:0/3401972988 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fa6e406c8f0 0x7fa6e406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:13.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.645+0000 7fa704c00640 1 -- 172.21.15.152:0/3401972988 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7fa6f008ba90 con 0x7fa7080cd0d0 2024-04-20T11:28:13.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.646+0000 7fa707e00640 1 --2- 172.21.15.152:0/3401972988 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fa6e406c8f0 0x7fa6e406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:13.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.646+0000 7fa707e00640 1 --2- 172.21.15.152:0/3401972988 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fa6e406c8f0 0x7fa6e406edb0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa6f8006fd0 tx=0x7fa6f8008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:13.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.650+0000 7fa704c00640 1 -- 172.21.15.152:0/3401972988 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fa6f0022030 con 0x7fa7080cd0d0 2024-04-20T11:28:13.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.921+0000 7fa70f000640 1 -- 172.21.15.152:0/3401972988 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7fa70815cfa0 con 0x7fa7080cd0d0 2024-04-20T11:28:13.925 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.925+0000 7fa704c00640 1 -- 172.21.15.152:0/3401972988 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v8) ==== 90+0+0 (secure 0 0 0) 0x7fa6f005a170 con 0x7fa7080cd0d0 2024-04-20T11:28:13.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.927+0000 7fa70f000640 1 -- 172.21.15.152:0/3401972988 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fa6e406c8f0 msgr2=0x7fa6e406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:13.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.927+0000 7fa70f000640 1 --2- 172.21.15.152:0/3401972988 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fa6e406c8f0 0x7fa6e406edb0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa6f8006fd0 tx=0x7fa6f8008040 comp rx=0 tx=0).stop 2024-04-20T11:28:13.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.927+0000 7fa70f000640 1 -- 172.21.15.152:0/3401972988 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa7080cd0d0 msgr2=0x7fa70815e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:13.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.927+0000 7fa70f000640 1 --2- 172.21.15.152:0/3401972988 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa7080cd0d0 0x7fa70815e790 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fa6f000d800 tx=0x7fa6f000dcc0 comp rx=0 tx=0).stop 2024-04-20T11:28:13.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.928+0000 7fa70f000640 1 -- 172.21.15.152:0/3401972988 shutdown_connections 2024-04-20T11:28:13.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.928+0000 7fa70f000640 1 --2- 172.21.15.152:0/3401972988 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fa6e406c8f0 0x7fa6e406edb0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:13.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.928+0000 7fa70f000640 1 --2- 172.21.15.152:0/3401972988 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa708145f60 0x7fa7080ae900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:13.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.928+0000 7fa70f000640 1 --2- 172.21.15.152:0/3401972988 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa7080cd0d0 0x7fa70815e790 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:13.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.928+0000 7fa70f000640 1 --2- 172.21.15.152:0/3401972988 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa708071fe0 0x7fa708141440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:13.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.928+0000 7fa70f000640 1 -- 172.21.15.152:0/3401972988 >> 172.21.15.152:0/3401972988 conn(0x7fa708067cf0 msgr2=0x7fa70806f5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:13.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.928+0000 7fa70f000640 1 -- 172.21.15.152:0/3401972988 shutdown_connections 2024-04-20T11:28:13.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:13.928+0000 7fa70f000640 1 -- 172.21.15.152:0/3401972988 wait complete. 2024-04-20T11:28:13.947 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:28:14.145 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.145+0000 7effb4e00640 1 Processor -- start 2024-04-20T11:28:14.145 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.145+0000 7effb4e00640 1 -- start start 2024-04-20T11:28:14.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.147+0000 7effb4e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7effb015fcc0 0x7effb00d4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:14.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.147+0000 7effb4e00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7effb00d3350 con 0x7effb01603d0 2024-04-20T11:28:14.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.147+0000 7effb4e00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7effb00d3630 con 0x7effb00b65c0 2024-04-20T11:28:14.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.147+0000 7effb4e00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7effb00d3910 con 0x7effb015fcc0 2024-04-20T11:28:14.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.147+0000 7effaf400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7effb015fcc0 0x7effb00d4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:14.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.147+0000 7effaf400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7effb015fcc0 0x7effb00d4e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:46154/0 (socket says 172.21.15.152:46154) 2024-04-20T11:28:14.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.147+0000 7effaf400640 1 -- 172.21.15.152:0/1606182991 learned_addr learned my addr 172.21.15.152:0/1606182991 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:14.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.147+0000 7effa6a00640 1 --1- 172.21.15.152:0/1606182991 >> v1:172.21.15.152:6789/0 conn(0x7effb01603d0 0x7effb00b5eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:41198/0 (socket says 172.21.15.152:41198) 2024-04-20T11:28:14.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.147+0000 7effaf400640 1 -- 172.21.15.152:0/1606182991 >> v1:172.21.15.175:6789/0 conn(0x7effb00b65c0 legacy=0x7effb00cf240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:14.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.147+0000 7effaf400640 1 -- 172.21.15.152:0/1606182991 >> v1:172.21.15.152:6789/0 conn(0x7effb01603d0 legacy=0x7effb00b5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:14.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.147+0000 7effaf400640 1 -- 172.21.15.152:0/1606182991 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effb00ced20 con 0x7effb015fcc0 2024-04-20T11:28:14.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.148+0000 7effaf400640 1 --2- 172.21.15.152:0/1606182991 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7effb015fcc0 0x7effb00d4e70 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7effa0009e50 tx=0x7effa002bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec63ccdf66b35bb8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:14.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.148+0000 7effaea00640 1 -- 172.21.15.152:0/1606182991 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa002cb10 con 0x7effb015fcc0 2024-04-20T11:28:14.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.148+0000 7effaea00640 1 -- 172.21.15.152:0/1606182991 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effa002ccb0 con 0x7effb015fcc0 2024-04-20T11:28:14.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.148+0000 7effaea00640 1 -- 172.21.15.152:0/1606182991 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa002ce60 con 0x7effb015fcc0 2024-04-20T11:28:14.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.149+0000 7effb4e00640 1 -- 172.21.15.152:0/1606182991 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7effb015fcc0 msgr2=0x7effb00d4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:14.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.149+0000 7effb4e00640 1 --2- 172.21.15.152:0/1606182991 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7effb015fcc0 0x7effb00d4e70 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7effa0009e50 tx=0x7effa002bf70 comp rx=0 tx=0).stop 2024-04-20T11:28:14.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.149+0000 7effb4e00640 1 -- 172.21.15.152:0/1606182991 shutdown_connections 2024-04-20T11:28:14.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.149+0000 7effb4e00640 1 --2- 172.21.15.152:0/1606182991 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7effb015fcc0 0x7effb00d4e70 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:14.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.149+0000 7effb4e00640 1 -- 172.21.15.152:0/1606182991 >> 172.21.15.152:0/1606182991 conn(0x7effb008bfd0 msgr2=0x7effb00d4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:14.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.149+0000 7effb4e00640 1 -- 172.21.15.152:0/1606182991 shutdown_connections 2024-04-20T11:28:14.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.149+0000 7effb4e00640 1 -- 172.21.15.152:0/1606182991 wait complete. 2024-04-20T11:28:14.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.152+0000 7effb4e00640 1 Processor -- start 2024-04-20T11:28:14.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.152+0000 7effb4e00640 1 -- start start 2024-04-20T11:28:14.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.152+0000 7effb4e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7effb00b65c0 0x7effb015dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:14.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.152+0000 7effb4e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7effb015fcc0 0x7effb00b1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:14.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.152+0000 7effb4e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7effb01603d0 0x7effb00b90d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:14.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.152+0000 7effb4e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7effb00b5600 con 0x7effb015fcc0 2024-04-20T11:28:14.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.152+0000 7effb4e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7effb00b5480 con 0x7effb00b65c0 2024-04-20T11:28:14.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.152+0000 7effb4e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7effb00cf950 con 0x7effb01603d0 2024-04-20T11:28:14.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.152+0000 7effa6a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7effb015fcc0 0x7effb00b1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:14.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.152+0000 7effa6a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7effb015fcc0 0x7effb00b1ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:46156/0 (socket says 172.21.15.152:46156) 2024-04-20T11:28:14.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.152+0000 7effa6a00640 1 -- 172.21.15.152:0/4096381993 learned_addr learned my addr 172.21.15.152:0/4096381993 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:14.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.153+0000 7effaf400640 1 --2- 172.21.15.152:0/4096381993 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7effb00b65c0 0x7effb015dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:14.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.153+0000 7effa6a00640 1 -- 172.21.15.152:0/4096381993 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7effb01603d0 msgr2=0x7effb00b90d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:14.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.153+0000 7effa6a00640 1 --2- 172.21.15.152:0/4096381993 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7effb01603d0 0x7effb00b90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:14.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.153+0000 7effa6a00640 1 -- 172.21.15.152:0/4096381993 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7effb00b65c0 msgr2=0x7effb015dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:14.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.153+0000 7effa6a00640 1 --2- 172.21.15.152:0/4096381993 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7effb00b65c0 0x7effb015dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:14.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.153+0000 7effa6a00640 1 -- 172.21.15.152:0/4096381993 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effb00b45d0 con 0x7effb015fcc0 2024-04-20T11:28:14.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.153+0000 7effaf400640 1 --2- 172.21.15.152:0/4096381993 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7effb00b65c0 0x7effb015dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:28:14.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.153+0000 7effa6a00640 1 --2- 172.21.15.152:0/4096381993 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7effb015fcc0 0x7effb00b1ad0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7eff9800c3f0 tx=0x7eff9800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:14.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.153+0000 7effa7e00640 1 -- 172.21.15.152:0/4096381993 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff98017070 con 0x7effb015fcc0 2024-04-20T11:28:14.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.153+0000 7effa7e00640 1 -- 172.21.15.152:0/4096381993 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff980074e0 con 0x7effb015fcc0 2024-04-20T11:28:14.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.153+0000 7effa7e00640 1 -- 172.21.15.152:0/4096381993 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff980059c0 con 0x7effb015fcc0 2024-04-20T11:28:14.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.153+0000 7effb4e00640 1 -- 172.21.15.152:0/4096381993 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effb00b48b0 con 0x7effb015fcc0 2024-04-20T11:28:14.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.153+0000 7effb4e00640 1 -- 172.21.15.152:0/4096381993 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effb00b4e70 con 0x7effb015fcc0 2024-04-20T11:28:14.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.155+0000 7effb4e00640 1 -- 172.21.15.152:0/4096381993 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effb0075e60 con 0x7effb015fcc0 2024-04-20T11:28:14.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.155+0000 7effa7e00640 1 -- 172.21.15.152:0/4096381993 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7eff9801a070 con 0x7effb015fcc0 2024-04-20T11:28:14.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.156+0000 7effa7e00640 1 --2- 172.21.15.152:0/4096381993 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7eff8c06c8f0 0x7eff8c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:14.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.156+0000 7effa7e00640 1 -- 172.21.15.152:0/4096381993 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7eff9808abb0 con 0x7effb015fcc0 2024-04-20T11:28:14.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.157+0000 7effaf400640 1 --2- 172.21.15.152:0/4096381993 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7eff8c06c8f0 0x7eff8c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:14.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.157+0000 7effaf400640 1 --2- 172.21.15.152:0/4096381993 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7eff8c06c8f0 0x7eff8c06edb0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7effa0035000 tx=0x7effa0030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:14.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.161+0000 7effa7e00640 1 -- 172.21.15.152:0/4096381993 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7eff98059350 con 0x7effb015fcc0 2024-04-20T11:28:14.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.361+0000 7effb4e00640 1 -- 172.21.15.152:0/4096381993 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7effb00ba4d0 con 0x7effb015fcc0 2024-04-20T11:28:14.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.363+0000 7effa7e00640 1 -- 172.21.15.152:0/4096381993 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) ==== 74+0+12951 (secure 0 0 0) 0x7eff98058c50 con 0x7effb015fcc0 2024-04-20T11:28:14.364 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:28:14.364 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":30,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:28:10.396312+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:28:14.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.366+0000 7effb4e00640 1 -- 172.21.15.152:0/4096381993 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7eff8c06c8f0 msgr2=0x7eff8c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:14.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.366+0000 7effb4e00640 1 --2- 172.21.15.152:0/4096381993 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7eff8c06c8f0 0x7eff8c06edb0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7effa0035000 tx=0x7effa0030000 comp rx=0 tx=0).stop 2024-04-20T11:28:14.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.366+0000 7effb4e00640 1 -- 172.21.15.152:0/4096381993 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7effb015fcc0 msgr2=0x7effb00b1ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:14.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.366+0000 7effb4e00640 1 --2- 172.21.15.152:0/4096381993 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7effb015fcc0 0x7effb00b1ad0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7eff9800c3f0 tx=0x7eff9800c8b0 comp rx=0 tx=0).stop 2024-04-20T11:28:14.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.366+0000 7effb4e00640 1 -- 172.21.15.152:0/4096381993 shutdown_connections 2024-04-20T11:28:14.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.366+0000 7effb4e00640 1 --2- 172.21.15.152:0/4096381993 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7eff8c06c8f0 0x7eff8c06edb0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:14.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.366+0000 7effb4e00640 1 --2- 172.21.15.152:0/4096381993 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7effb01603d0 0x7effb00b90d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:14.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.366+0000 7effb4e00640 1 --2- 172.21.15.152:0/4096381993 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7effb015fcc0 0x7effb00b1ad0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:14.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.366+0000 7effb4e00640 1 --2- 172.21.15.152:0/4096381993 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7effb00b65c0 0x7effb015dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:14.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.366+0000 7effb4e00640 1 -- 172.21.15.152:0/4096381993 >> 172.21.15.152:0/4096381993 conn(0x7effb008bfd0 msgr2=0x7effb0161ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:14.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.366+0000 7effb4e00640 1 -- 172.21.15.152:0/4096381993 shutdown_connections 2024-04-20T11:28:14.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.366+0000 7effb4e00640 1 -- 172.21.15.152:0/4096381993 wait complete. 2024-04-20T11:28:14.386 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T11:26:47.246571+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T11:28:14.386 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T11:28:14.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.587+0000 7f91b3e00640 1 Processor -- start 2024-04-20T11:28:14.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.587+0000 7f91b3e00640 1 -- start start 2024-04-20T11:28:14.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.588+0000 7f91b3e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f91b4152450 0x7f91b415c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:14.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.588+0000 7f91b3e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f91b40b30c0 0x7f91b406bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:14.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.588+0000 7f91b3e00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91b4152830 con 0x7f91b40b3520 2024-04-20T11:28:14.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.588+0000 7f91b3e00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f91b40b39c0 con 0x7f91b4152450 2024-04-20T11:28:14.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.588+0000 7f91b3e00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f91b40b3ca0 con 0x7f91b40b30c0 2024-04-20T11:28:14.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.588+0000 7f91b2a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f91b4152450 0x7f91b415c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:14.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.589+0000 7f91b3400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f91b40b3520 0x7f91b415efa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:41206/0 (socket says 172.21.15.152:41206) 2024-04-20T11:28:14.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.589+0000 7f91b3400640 1 -- 172.21.15.152:0/883728552 learned_addr learned my addr 172.21.15.152:0/883728552 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:28:14.590 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.589+0000 7f91b2000640 1 --2- 172.21.15.152:0/883728552 >> v2:172.21.15.175:3300/0 conn(0x7f91b40b30c0 0x7f91b406bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:14.590 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.589+0000 7f91b2a00640 1 -- 172.21.15.152:0/883728552 >> v2:172.21.15.175:3300/0 conn(0x7f91b40b30c0 msgr2=0x7f91b406bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:14.590 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.589+0000 7f91b2a00640 1 --2- 172.21.15.152:0/883728552 >> v2:172.21.15.175:3300/0 conn(0x7f91b40b30c0 0x7f91b406bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:14.590 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.589+0000 7f91b2a00640 1 -- 172.21.15.152:0/883728552 >> v1:172.21.15.152:6789/0 conn(0x7f91b40b3520 legacy=0x7f91b415efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:14.590 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.589+0000 7f91b2a00640 1 -- 172.21.15.152:0/883728552 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91b40b3f80 con 0x7f91b4152450 2024-04-20T11:28:14.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.590+0000 7f91b2a00640 1 --2- 172.21.15.152:0/883728552 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f91b4152450 0x7f91b415c590 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f91b40afb80 tx=0x7f91a402e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29e6b085a904055e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:14.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.591+0000 7f91b1600640 1 -- 172.21.15.152:0/883728552 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91a403b070 con 0x7f91b4152450 2024-04-20T11:28:14.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.591+0000 7f91b1600640 1 -- 172.21.15.152:0/883728552 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91a4036440 con 0x7f91b4152450 2024-04-20T11:28:14.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.591+0000 7f91b1600640 1 -- 172.21.15.152:0/883728552 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91a4036810 con 0x7f91b4152450 2024-04-20T11:28:14.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.591+0000 7f91b3e00640 1 -- 172.21.15.152:0/883728552 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f91b4152450 msgr2=0x7f91b415c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:14.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.591+0000 7f91b3e00640 1 --2- 172.21.15.152:0/883728552 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f91b4152450 0x7f91b415c590 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f91b40afb80 tx=0x7f91a402e0e0 comp rx=0 tx=0).stop 2024-04-20T11:28:14.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.591+0000 7f91b3e00640 1 -- 172.21.15.152:0/883728552 shutdown_connections 2024-04-20T11:28:14.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.591+0000 7f91b3e00640 1 --2- 172.21.15.152:0/883728552 >> v2:172.21.15.175:3300/0 conn(0x7f91b40b30c0 0x7f91b406bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:14.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.591+0000 7f91b3e00640 1 --2- 172.21.15.152:0/883728552 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f91b4152450 0x7f91b415c590 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:14.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.591+0000 7f91b3e00640 1 -- 172.21.15.152:0/883728552 >> 172.21.15.152:0/883728552 conn(0x7f91b40b21e0 msgr2=0x7f91b40b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:14.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.591+0000 7f91b3e00640 1 -- 172.21.15.152:0/883728552 shutdown_connections 2024-04-20T11:28:14.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.592+0000 7f91b3e00640 1 -- 172.21.15.152:0/883728552 wait complete. 2024-04-20T11:28:14.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.593+0000 7f91b3e00640 1 Processor -- start 2024-04-20T11:28:14.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.593+0000 7f91b3e00640 1 -- start start 2024-04-20T11:28:14.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.594+0000 7f91b3e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f91b40b30c0 0x7f91b4099920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:14.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.594+0000 7f91b3e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f91b40b3520 0x7f91b4071ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:14.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.594+0000 7f91b3e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f91b4152450 0x7f91b4072420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:14.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.594+0000 7f91b3e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f91b40af480 con 0x7f91b40b30c0 2024-04-20T11:28:14.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.594+0000 7f91b3e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f91b40adf00 con 0x7f91b4152450 2024-04-20T11:28:14.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.594+0000 7f91b3e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f91b40af600 con 0x7f91b40b3520 2024-04-20T11:28:14.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.595+0000 7f91b2a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f91b40b30c0 0x7f91b4099920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:14.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.595+0000 7f91b2a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f91b40b30c0 0x7f91b4099920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:46174/0 (socket says 172.21.15.152:46174) 2024-04-20T11:28:14.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.595+0000 7f91b2a00640 1 -- 172.21.15.152:0/2003237908 learned_addr learned my addr 172.21.15.152:0/2003237908 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:14.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.595+0000 7f91b2a00640 1 -- 172.21.15.152:0/2003237908 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f91b40b3520 msgr2=0x7f91b4071ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:14.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.595+0000 7f91b2a00640 1 --2- 172.21.15.152:0/2003237908 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f91b40b3520 0x7f91b4071ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:14.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.595+0000 7f91b3400640 1 --2- 172.21.15.152:0/2003237908 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f91b4152450 0x7f91b4072420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:14.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.595+0000 7f91b2a00640 1 -- 172.21.15.152:0/2003237908 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f91b4152450 msgr2=0x7f91b4072420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:14.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.595+0000 7f91b2a00640 1 --2- 172.21.15.152:0/2003237908 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f91b4152450 0x7f91b4072420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:14.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.595+0000 7f91b2a00640 1 -- 172.21.15.152:0/2003237908 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91b409e140 con 0x7f91b40b30c0 2024-04-20T11:28:14.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.595+0000 7f91b3400640 1 --2- 172.21.15.152:0/2003237908 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f91b4152450 0x7f91b4072420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:28:14.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.596+0000 7f91b2a00640 1 --2- 172.21.15.152:0/2003237908 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f91b40b30c0 0x7f91b4099920 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f91a4039040 tx=0x7f91a4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:14.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.596+0000 7f91aaa00640 1 -- 172.21.15.152:0/2003237908 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91a403b050 con 0x7f91b40b30c0 2024-04-20T11:28:14.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.596+0000 7f91aaa00640 1 -- 172.21.15.152:0/2003237908 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91a4047070 con 0x7f91b40b30c0 2024-04-20T11:28:14.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.596+0000 7f91aaa00640 1 -- 172.21.15.152:0/2003237908 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91a4009450 con 0x7f91b40b30c0 2024-04-20T11:28:14.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.596+0000 7f91b3e00640 1 -- 172.21.15.152:0/2003237908 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91b40ce000 con 0x7f91b40b30c0 2024-04-20T11:28:14.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.596+0000 7f91b3e00640 1 -- 172.21.15.152:0/2003237908 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91b409d380 con 0x7f91b40b30c0 2024-04-20T11:28:14.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.597+0000 7f91b3e00640 1 -- 172.21.15.152:0/2003237908 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91b4075e60 con 0x7f91b40b30c0 2024-04-20T11:28:14.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.598+0000 7f91aaa00640 1 -- 172.21.15.152:0/2003237908 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f91a404c460 con 0x7f91b40b30c0 2024-04-20T11:28:14.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.599+0000 7f91aaa00640 1 --2- 172.21.15.152:0/2003237908 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f919406c920 0x7f919406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:14.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.599+0000 7f91aaa00640 1 -- 172.21.15.152:0/2003237908 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7f91a40b09b0 con 0x7f91b40b30c0 2024-04-20T11:28:14.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.599+0000 7f91b2000640 1 --2- 172.21.15.152:0/2003237908 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f919406c920 0x7f919406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:14.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.600+0000 7f91b2000640 1 --2- 172.21.15.152:0/2003237908 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f919406c920 0x7f919406ede0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f91a0006fd0 tx=0x7f91a0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:14.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.603+0000 7f91aaa00640 1 -- 172.21.15.152:0/2003237908 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f91a4036e20 con 0x7f91b40b30c0 2024-04-20T11:28:14.801 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.801+0000 7f91b3e00640 1 -- 172.21.15.152:0/2003237908 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f91b415fb50 con 0x7f91b40b30c0 2024-04-20T11:28:14.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.802+0000 7f91aaa00640 1 -- 172.21.15.152:0/2003237908 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) ==== 93+0+10 (secure 0 0 0) 0x7f91a407f1b0 con 0x7f91b40b30c0 2024-04-20T11:28:14.803 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 1 2024-04-20T11:28:14.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.804+0000 7f91b3e00640 1 -- 172.21.15.152:0/2003237908 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f919406c920 msgr2=0x7f919406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:14.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.804+0000 7f91b3e00640 1 --2- 172.21.15.152:0/2003237908 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f919406c920 0x7f919406ede0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f91a0006fd0 tx=0x7f91a0008040 comp rx=0 tx=0).stop 2024-04-20T11:28:14.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.804+0000 7f91b3e00640 1 -- 172.21.15.152:0/2003237908 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f91b40b30c0 msgr2=0x7f91b4099920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:14.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.804+0000 7f91b3e00640 1 --2- 172.21.15.152:0/2003237908 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f91b40b30c0 0x7f91b4099920 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f91a4039040 tx=0x7f91a4004a40 comp rx=0 tx=0).stop 2024-04-20T11:28:14.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.805+0000 7f91b3e00640 1 -- 172.21.15.152:0/2003237908 shutdown_connections 2024-04-20T11:28:14.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.805+0000 7f91b3e00640 1 --2- 172.21.15.152:0/2003237908 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f919406c920 0x7f919406ede0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:14.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.805+0000 7f91b3e00640 1 --2- 172.21.15.152:0/2003237908 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f91b4152450 0x7f91b4072420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:14.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.805+0000 7f91b3e00640 1 --2- 172.21.15.152:0/2003237908 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f91b40b3520 0x7f91b4071ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:14.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.805+0000 7f91b3e00640 1 --2- 172.21.15.152:0/2003237908 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f91b40b30c0 0x7f91b4099920 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:14.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.805+0000 7f91b3e00640 1 -- 172.21.15.152:0/2003237908 >> 172.21.15.152:0/2003237908 conn(0x7f91b40b21e0 msgr2=0x7f91b414ffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:14.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.805+0000 7f91b3e00640 1 -- 172.21.15.152:0/2003237908 shutdown_connections 2024-04-20T11:28:14.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:14.805+0000 7f91b3e00640 1 -- 172.21.15.152:0/2003237908 wait complete. 2024-04-20T11:28:14.824 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-20T11:28:15.022 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.022+0000 7f2fe9a00640 1 Processor -- start 2024-04-20T11:28:15.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.022+0000 7f2fe9a00640 1 -- start start 2024-04-20T11:28:15.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.023+0000 7f2fe9a00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f2fe4155830 0x7f2fe406ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:15.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.023+0000 7f2fe9a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f2fe40b7270 0x7f2fe409aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:15.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.023+0000 7f2fe9a00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2fe40b5da0 con 0x7f2fe406cfd0 2024-04-20T11:28:15.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.023+0000 7f2fe9a00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f2fe40b6080 con 0x7f2fe40b7270 2024-04-20T11:28:15.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.023+0000 7f2fe9a00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f2fe40b6360 con 0x7f2fe4155830 2024-04-20T11:28:15.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.024+0000 7f2fe3400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f2fe406cfd0 0x7f2fe40b6b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:41212/0 (socket says 172.21.15.152:41212) 2024-04-20T11:28:15.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.024+0000 7f2fe3400640 1 -- 172.21.15.152:0/1837564998 learned_addr learned my addr 172.21.15.152:0/1837564998 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:28:15.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.024+0000 7f2fe9000640 1 --2- 172.21.15.152:0/1837564998 >> v2:172.21.15.152:3300/0 conn(0x7f2fe40b7270 0x7f2fe409aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:15.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.024+0000 7f2fe3e00640 1 --2- 172.21.15.152:0/1837564998 >> v2:172.21.15.175:3300/0 conn(0x7f2fe4155830 0x7f2fe406ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:15.026 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.024+0000 7f2fe9000640 1 -- 172.21.15.152:0/1837564998 >> v2:172.21.15.175:3300/0 conn(0x7f2fe4155830 msgr2=0x7f2fe406ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:15.026 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.024+0000 7f2fe9000640 1 --2- 172.21.15.152:0/1837564998 >> v2:172.21.15.175:3300/0 conn(0x7f2fe4155830 0x7f2fe406ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:15.026 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.024+0000 7f2fe9000640 1 -- 172.21.15.152:0/1837564998 >> v1:172.21.15.152:6789/0 conn(0x7f2fe406cfd0 legacy=0x7f2fe40b6b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:15.026 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.024+0000 7f2fe9000640 1 -- 172.21.15.152:0/1837564998 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fe40714e0 con 0x7f2fe40b7270 2024-04-20T11:28:15.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.024+0000 7f2fe2a00640 1 -- 172.21.15.152:0/1837564998 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 971934711 0 0) 0x7f2fe40b5da0 con 0x7f2fe406cfd0 2024-04-20T11:28:15.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.026+0000 7f2fe9000640 1 --2- 172.21.15.152:0/1837564998 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2fe40b7270 0x7f2fe409aa40 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f2fe4073410 tx=0x7f2fdc02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88325e4e24d383cb server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:15.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.026+0000 7f2fe2a00640 1 -- 172.21.15.152:0/1837564998 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fdc0315a0 con 0x7f2fe40b7270 2024-04-20T11:28:15.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.026+0000 7f2fe2a00640 1 -- 172.21.15.152:0/1837564998 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fdc037070 con 0x7f2fe40b7270 2024-04-20T11:28:15.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.026+0000 7f2fe2a00640 1 -- 172.21.15.152:0/1837564998 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fdc031960 con 0x7f2fe40b7270 2024-04-20T11:28:15.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.026+0000 7f2fe9a00640 1 -- 172.21.15.152:0/1837564998 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2fe40b7270 msgr2=0x7f2fe409aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:15.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.026+0000 7f2fe9a00640 1 --2- 172.21.15.152:0/1837564998 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2fe40b7270 0x7f2fe409aa40 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f2fe4073410 tx=0x7f2fdc02b840 comp rx=0 tx=0).stop 2024-04-20T11:28:15.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.026+0000 7f2fe9a00640 1 -- 172.21.15.152:0/1837564998 shutdown_connections 2024-04-20T11:28:15.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.027+0000 7f2fe9a00640 1 --2- 172.21.15.152:0/1837564998 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2fe40b7270 0x7f2fe409aa40 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:15.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.027+0000 7f2fe9a00640 1 --2- 172.21.15.152:0/1837564998 >> v2:172.21.15.175:3300/0 conn(0x7f2fe4155830 0x7f2fe406ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:15.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.027+0000 7f2fe9a00640 1 -- 172.21.15.152:0/1837564998 >> 172.21.15.152:0/1837564998 conn(0x7f2fe40b3ed0 msgr2=0x7f2fe40b42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:15.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.027+0000 7f2fe9a00640 1 -- 172.21.15.152:0/1837564998 shutdown_connections 2024-04-20T11:28:15.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.027+0000 7f2fe9a00640 1 -- 172.21.15.152:0/1837564998 wait complete. 2024-04-20T11:28:15.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.028+0000 7f2fe9a00640 1 Processor -- start 2024-04-20T11:28:15.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.028+0000 7f2fe9a00640 1 -- start start 2024-04-20T11:28:15.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.029+0000 7f2fe9a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2fe406cfd0 0x7f2fe41f3970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:15.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.029+0000 7f2fe9a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2fe40b7270 0x7f2fe409cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:15.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.029+0000 7f2fe9a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2fe4155830 0x7f2fe40b0200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:15.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.029+0000 7f2fe9a00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f2fe409af80 con 0x7f2fe4155830 2024-04-20T11:28:15.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.029+0000 7f2fe9a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2fe4155e20 con 0x7f2fe406cfd0 2024-04-20T11:28:15.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.029+0000 7f2fe9a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2fe409b100 con 0x7f2fe40b7270 2024-04-20T11:28:15.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.030+0000 7f2fe9000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2fe4155830 0x7f2fe40b0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:15.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.030+0000 7f2fe9000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2fe4155830 0x7f2fe40b0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:46206/0 (socket says 172.21.15.152:46206) 2024-04-20T11:28:15.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.030+0000 7f2fe9000640 1 -- 172.21.15.152:0/2043237385 learned_addr learned my addr 172.21.15.152:0/2043237385 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:15.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.030+0000 7f2fe3e00640 1 --2- 172.21.15.152:0/2043237385 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2fe406cfd0 0x7f2fe41f3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:15.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.030+0000 7f2fe3400640 1 --2- 172.21.15.152:0/2043237385 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2fe40b7270 0x7f2fe409cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:15.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.030+0000 7f2fe9000640 1 -- 172.21.15.152:0/2043237385 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2fe40b7270 msgr2=0x7f2fe409cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:15.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.030+0000 7f2fe9000640 1 --2- 172.21.15.152:0/2043237385 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2fe40b7270 0x7f2fe409cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:15.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.030+0000 7f2fe9000640 1 -- 172.21.15.152:0/2043237385 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2fe406cfd0 msgr2=0x7f2fe41f3970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:15.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.030+0000 7f2fe9000640 1 --2- 172.21.15.152:0/2043237385 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2fe406cfd0 0x7f2fe41f3970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:15.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.030+0000 7f2fe9000640 1 -- 172.21.15.152:0/2043237385 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fe409f930 con 0x7f2fe4155830 2024-04-20T11:28:15.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.030+0000 7f2fe9000640 1 --2- 172.21.15.152:0/2043237385 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2fe4155830 0x7f2fe40b0200 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc009890 tx=0x7f2fdc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:15.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.031+0000 7f2fdbe00640 1 -- 172.21.15.152:0/2043237385 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fdc032840 con 0x7f2fe4155830 2024-04-20T11:28:15.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.031+0000 7f2fdbe00640 1 -- 172.21.15.152:0/2043237385 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fdc032a00 con 0x7f2fe4155830 2024-04-20T11:28:15.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.031+0000 7f2fdbe00640 1 -- 172.21.15.152:0/2043237385 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fdc032df0 con 0x7f2fe4155830 2024-04-20T11:28:15.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.031+0000 7f2fe9a00640 1 -- 172.21.15.152:0/2043237385 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fe409fc10 con 0x7f2fe4155830 2024-04-20T11:28:15.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.031+0000 7f2fe9a00640 1 -- 172.21.15.152:0/2043237385 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fe40a01d0 con 0x7f2fe4155830 2024-04-20T11:28:15.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.032+0000 7f2fe9a00640 1 -- 172.21.15.152:0/2043237385 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fe4075e60 con 0x7f2fe4155830 2024-04-20T11:28:15.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.033+0000 7f2fdbe00640 1 -- 172.21.15.152:0/2043237385 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2fdc04c020 con 0x7f2fe4155830 2024-04-20T11:28:15.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.034+0000 7f2fdbe00640 1 --2- 172.21.15.152:0/2043237385 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f2fc406c8f0 0x7f2fc406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:15.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.034+0000 7f2fdbe00640 1 -- 172.21.15.152:0/2043237385 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7f2fdc02f020 con 0x7f2fe4155830 2024-04-20T11:28:15.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.034+0000 7f2fe3e00640 1 --2- 172.21.15.152:0/2043237385 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f2fc406c8f0 0x7f2fc406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:15.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.035+0000 7f2fe3e00640 1 --2- 172.21.15.152:0/2043237385 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f2fc406c8f0 0x7f2fc406edb0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2fd4006fd0 tx=0x7f2fd4008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:15.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.039+0000 7f2fdbe00640 1 -- 172.21.15.152:0/2043237385 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f2fdc040070 con 0x7f2fe4155830 2024-04-20T11:28:15.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.243+0000 7f2fe9a00640 1 -- 172.21.15.152:0/2043237385 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f2fe40b6c00 con 0x7f2fe4155830 2024-04-20T11:28:15.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.244+0000 7f2fdbe00640 1 -- 172.21.15.152:0/2043237385 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v30) ==== 84+0+65 (secure 0 0 0) 0x7f2fdc07bbc0 con 0x7f2fe4155830 2024-04-20T11:28:15.245 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:28:15.245 INFO:teuthology.orchestra.run.smithi152.stdout:[ 2024-04-20T11:28:15.245 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:28:15.245 INFO:teuthology.orchestra.run.smithi152.stdout: "poolnum": 1, 2024-04-20T11:28:15.245 INFO:teuthology.orchestra.run.smithi152.stdout: "poolname": ".mgr" 2024-04-20T11:28:15.245 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:28:15.245 INFO:teuthology.orchestra.run.smithi152.stdout:] 2024-04-20T11:28:15.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.247+0000 7f2fe9a00640 1 -- 172.21.15.152:0/2043237385 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f2fc406c8f0 msgr2=0x7f2fc406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:15.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.247+0000 7f2fe9a00640 1 --2- 172.21.15.152:0/2043237385 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f2fc406c8f0 0x7f2fc406edb0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2fd4006fd0 tx=0x7f2fd4008040 comp rx=0 tx=0).stop 2024-04-20T11:28:15.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.247+0000 7f2fe9a00640 1 -- 172.21.15.152:0/2043237385 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2fe4155830 msgr2=0x7f2fe40b0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:15.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.247+0000 7f2fe9a00640 1 --2- 172.21.15.152:0/2043237385 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2fe4155830 0x7f2fe40b0200 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc009890 tx=0x7f2fdc0046c0 comp rx=0 tx=0).stop 2024-04-20T11:28:15.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.247+0000 7f2fe9a00640 1 -- 172.21.15.152:0/2043237385 shutdown_connections 2024-04-20T11:28:15.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.247+0000 7f2fe9a00640 1 --2- 172.21.15.152:0/2043237385 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f2fc406c8f0 0x7f2fc406edb0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:15.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.247+0000 7f2fe9a00640 1 --2- 172.21.15.152:0/2043237385 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2fe4155830 0x7f2fe40b0200 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:15.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.247+0000 7f2fe9a00640 1 --2- 172.21.15.152:0/2043237385 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2fe40b7270 0x7f2fe409cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:15.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.247+0000 7f2fe9a00640 1 --2- 172.21.15.152:0/2043237385 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2fe406cfd0 0x7f2fe41f3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:15.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.247+0000 7f2fe9a00640 1 -- 172.21.15.152:0/2043237385 >> 172.21.15.152:0/2043237385 conn(0x7f2fe40b3ed0 msgr2=0x7f2fe4070130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:15.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.247+0000 7f2fe9a00640 1 -- 172.21.15.152:0/2043237385 shutdown_connections 2024-04-20T11:28:15.249 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.247+0000 7f2fe9a00640 1 -- 172.21.15.152:0/2043237385 wait complete. 2024-04-20T11:28:15.267 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-20T11:28:15.267 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-20T11:28:15.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.473+0000 7f2e17e00640 1 Processor -- start 2024-04-20T11:28:15.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.474+0000 7f2e17e00640 1 -- start start 2024-04-20T11:28:15.476 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.475+0000 7f2e17e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2e1815e6c0 0x7f2e1806dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:15.476 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.475+0000 7f2e17e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f2e1806f550 0x7f2e180d05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:15.476 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.475+0000 7f2e17e00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e1806ec10 con 0x7f2e180d0b30 2024-04-20T11:28:15.476 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.475+0000 7f2e17e00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f2e180b50f0 con 0x7f2e1806f550 2024-04-20T11:28:15.476 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.475+0000 7f2e17e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2e1806e370 con 0x7f2e1815e6c0 2024-04-20T11:28:15.476 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.475+0000 7f2e17400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f2e180d0b30 0x7f2e180b36d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:41220/0 (socket says 172.21.15.152:41220) 2024-04-20T11:28:15.476 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.475+0000 7f2e17400640 1 -- 172.21.15.152:0/198871793 learned_addr learned my addr 172.21.15.152:0/198871793 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:28:15.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.476+0000 7f2e0fe00640 1 --2- 172.21.15.152:0/198871793 >> v2:172.21.15.175:3300/0 conn(0x7f2e1806f550 0x7f2e180d05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:15.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.476+0000 7f2e16a00640 1 --2- 172.21.15.152:0/198871793 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2e1815e6c0 0x7f2e1806dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:15.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.476+0000 7f2e16000640 1 -- 172.21.15.152:0/198871793 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1264469588 0 0) 0x7f2e1806ec10 con 0x7f2e180d0b30 2024-04-20T11:28:15.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.476+0000 7f2e16000640 1 -- 172.21.15.152:0/198871793 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2dfc003520 con 0x7f2e180d0b30 2024-04-20T11:28:15.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.476+0000 7f2e16000640 1 -- 172.21.15.152:0/198871793 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 563898420 0 0) 0x7f2dfc003520 con 0x7f2e180d0b30 2024-04-20T11:28:15.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.476+0000 7f2e16000640 1 -- 172.21.15.152:0/198871793 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2e1815e6c0 msgr2=0x7f2e1806dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:15.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.476+0000 7f2e16000640 1 --2- 172.21.15.152:0/198871793 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2e1815e6c0 0x7f2e1806dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:15.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.476+0000 7f2e16000640 1 -- 172.21.15.152:0/198871793 >> v2:172.21.15.175:3300/0 conn(0x7f2e1806f550 msgr2=0x7f2e180d05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:15.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.476+0000 7f2e16000640 1 --2- 172.21.15.152:0/198871793 >> v2:172.21.15.175:3300/0 conn(0x7f2e1806f550 0x7f2e180d05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:15.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.476+0000 7f2e16000640 1 -- 172.21.15.152:0/198871793 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e1806e650 con 0x7f2e180d0b30 2024-04-20T11:28:15.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.476+0000 7f2e16000640 1 -- 172.21.15.152:0/198871793 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f2e100019f0 con 0x7f2e180d0b30 2024-04-20T11:28:15.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.476+0000 7f2e16000640 1 -- 172.21.15.152:0/198871793 >> v1:172.21.15.152:6789/0 conn(0x7f2e180d0b30 legacy=0x7f2e180b36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:15.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.476+0000 7f2e16000640 1 --2- 172.21.15.152:0/198871793 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2e1815e6c0 0x7f2dfc0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:15.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.476+0000 7f2e16000640 1 --2- 172.21.15.152:0/198871793 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2dfc003f30 0x7f2dfc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:15.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.477+0000 7f2e16000640 1 --2- 172.21.15.152:0/198871793 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2dfc004d40 0x7f2dfc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:15.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.477+0000 7f2e16000640 1 -- 172.21.15.152:0/198871793 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f2e180b50f0 con 0x7f2dfc003f30 2024-04-20T11:28:15.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.477+0000 7f2e16000640 1 -- 172.21.15.152:0/198871793 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2e1806e370 con 0x7f2dfc004d40 2024-04-20T11:28:15.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.477+0000 7f2e16000640 1 -- 172.21.15.152:0/198871793 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2dfc00b9c0 con 0x7f2e1815e6c0 2024-04-20T11:28:15.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.477+0000 7f2e0fe00640 1 --2- 172.21.15.152:0/198871793 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2dfc003f30 0x7f2dfc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:15.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.477+0000 7f2e16a00640 1 --2- 172.21.15.152:0/198871793 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2e1815e6c0 0x7f2dfc0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:15.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.477+0000 7f2e0fe00640 1 -- 172.21.15.152:0/198871793 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2e1815e6c0 msgr2=0x7f2dfc0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:15.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.477+0000 7f2e0fe00640 1 --2- 172.21.15.152:0/198871793 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2e1815e6c0 0x7f2dfc0039b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:15.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.477+0000 7f2e0fe00640 1 -- 172.21.15.152:0/198871793 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2dfc004d40 msgr2=0x7f2dfc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:15.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.477+0000 7f2e0fe00640 1 --2- 172.21.15.152:0/198871793 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2dfc004d40 0x7f2dfc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:15.480 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.477+0000 7f2e0fe00640 1 -- 172.21.15.152:0/198871793 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dfc003520 con 0x7f2dfc003f30 2024-04-20T11:28:15.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.478+0000 7f2e0fe00640 1 --2- 172.21.15.152:0/198871793 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2dfc003f30 0x7f2dfc004800 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f2e00009ac0 tx=0x7f2dfc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa886345b63d575b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:15.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.479+0000 7f2e16000640 1 -- 172.21.15.152:0/198871793 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e00039070 con 0x7f2dfc003f30 2024-04-20T11:28:15.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.479+0000 7f2e16000640 1 -- 172.21.15.152:0/198871793 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e0002c6d0 con 0x7f2dfc003f30 2024-04-20T11:28:15.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.479+0000 7f2e16000640 1 -- 172.21.15.152:0/198871793 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e00009690 con 0x7f2dfc003f30 2024-04-20T11:28:15.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.479+0000 7f2e17e00640 1 -- 172.21.15.152:0/198871793 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2dfc003f30 msgr2=0x7f2dfc004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:15.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.479+0000 7f2e17e00640 1 --2- 172.21.15.152:0/198871793 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2dfc003f30 0x7f2dfc004800 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f2e00009ac0 tx=0x7f2dfc005830 comp rx=0 tx=0).stop 2024-04-20T11:28:15.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.479+0000 7f2e17400640 1 -- 172.21.15.152:0/198871793 reap_dead start 2024-04-20T11:28:15.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.479+0000 7f2e17e00640 1 -- 172.21.15.152:0/198871793 shutdown_connections 2024-04-20T11:28:15.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.479+0000 7f2e17e00640 1 -- 172.21.15.152:0/198871793 >> 172.21.15.152:0/198871793 conn(0x7f2e180b21e0 msgr2=0x7f2e180b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:15.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.479+0000 7f2e17e00640 1 -- 172.21.15.152:0/198871793 shutdown_connections 2024-04-20T11:28:15.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.479+0000 7f2e17e00640 1 -- 172.21.15.152:0/198871793 wait complete. 2024-04-20T11:28:15.482 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.481+0000 7f2e17e00640 1 Processor -- start 2024-04-20T11:28:15.482 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.481+0000 7f2e17e00640 1 -- start start 2024-04-20T11:28:15.484 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.482+0000 7f2e17e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2dfc003f30 0x7f2e181f4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:15.484 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.482+0000 7f2e17e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2e1809dbc0 0x7f2e18146180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:15.484 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.482+0000 7f2e17e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2e18156c90 0x7f2e18157110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:15.484 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.482+0000 7f2e17e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f2e1815eaa0 con 0x7f2e1809dbc0 2024-04-20T11:28:15.484 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.482+0000 7f2e17e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2e180ad140 con 0x7f2e18156c90 2024-04-20T11:28:15.484 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.482+0000 7f2e17e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2e180ae670 con 0x7f2dfc003f30 2024-04-20T11:28:15.484 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.482+0000 7f2e0fe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2e1809dbc0 0x7f2e18146180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:15.484 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.483+0000 7f2e0fe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2e1809dbc0 0x7f2e18146180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:46228/0 (socket says 172.21.15.152:46228) 2024-04-20T11:28:15.484 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.483+0000 7f2e0fe00640 1 -- 172.21.15.152:0/1437129843 learned_addr learned my addr 172.21.15.152:0/1437129843 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:15.484 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.483+0000 7f2e16a00640 1 --2- 172.21.15.152:0/1437129843 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2dfc003f30 0x7f2e181f4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:15.484 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.483+0000 7f2e17400640 1 --2- 172.21.15.152:0/1437129843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2e18156c90 0x7f2e18157110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:15.484 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.483+0000 7f2e0fe00640 1 -- 172.21.15.152:0/1437129843 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2dfc003f30 msgr2=0x7f2e181f4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:15.484 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.483+0000 7f2e0fe00640 1 --2- 172.21.15.152:0/1437129843 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2dfc003f30 0x7f2e181f4ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:15.485 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.483+0000 7f2e0fe00640 1 -- 172.21.15.152:0/1437129843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2e18156c90 msgr2=0x7f2e18157110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:15.485 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.483+0000 7f2e0fe00640 1 --2- 172.21.15.152:0/1437129843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2e18156c90 0x7f2e18157110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:15.485 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.483+0000 7f2e0fe00640 1 -- 172.21.15.152:0/1437129843 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e181f5000 con 0x7f2e1809dbc0 2024-04-20T11:28:15.485 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.483+0000 7f2e17400640 1 --2- 172.21.15.152:0/1437129843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2e18156c90 0x7f2e18157110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:28:15.485 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.483+0000 7f2e0fe00640 1 --2- 172.21.15.152:0/1437129843 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2e1809dbc0 0x7f2e18146180 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f2e00037040 tx=0x7f2e00004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:15.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.484+0000 7f2e0ea00640 1 -- 172.21.15.152:0/1437129843 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e00042020 con 0x7f2e1809dbc0 2024-04-20T11:28:15.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.484+0000 7f2e0ea00640 1 -- 172.21.15.152:0/1437129843 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e0003a040 con 0x7f2e1809dbc0 2024-04-20T11:28:15.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.484+0000 7f2e0ea00640 1 -- 172.21.15.152:0/1437129843 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e00039070 con 0x7f2e1809dbc0 2024-04-20T11:28:15.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.484+0000 7f2e17e00640 1 -- 172.21.15.152:0/1437129843 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e18161500 con 0x7f2e1809dbc0 2024-04-20T11:28:15.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.484+0000 7f2e17e00640 1 -- 172.21.15.152:0/1437129843 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e18161ac0 con 0x7f2e1809dbc0 2024-04-20T11:28:15.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.485+0000 7f2e17e00640 1 -- 172.21.15.152:0/1437129843 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e18075e60 con 0x7f2e1809dbc0 2024-04-20T11:28:15.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.486+0000 7f2e0ea00640 1 -- 172.21.15.152:0/1437129843 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2e00045020 con 0x7f2e1809dbc0 2024-04-20T11:28:15.487 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.487+0000 7f2e0ea00640 1 --2- 172.21.15.152:0/1437129843 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f2df406c920 0x7f2df406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:15.487 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.487+0000 7f2e0ea00640 1 -- 172.21.15.152:0/1437129843 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7f2e000ac740 con 0x7f2e1809dbc0 2024-04-20T11:28:15.487 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.487+0000 7f2e16a00640 1 --2- 172.21.15.152:0/1437129843 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f2df406c920 0x7f2df406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:15.488 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.488+0000 7f2e16a00640 1 --2- 172.21.15.152:0/1437129843 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f2df406c920 0x7f2df406ede0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f2e08009930 tx=0x7f2e08008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:15.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.491+0000 7f2e0ea00640 1 -- 172.21.15.152:0/1437129843 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f2e181f5000 con 0x7f2e1809dbc0 2024-04-20T11:28:15.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:15.690+0000 7f2e17e00640 1 -- 172.21.15.152:0/1437129843 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f2e180b18f0 con 0x7f2e1809dbc0 2024-04-20T11:28:16.422 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:16.422+0000 7f2e0ea00640 1 -- 172.21.15.152:0/1437129843 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v31) ==== 138+0+0 (secure 0 0 0) 0x7f2e0007aeb0 con 0x7f2e1809dbc0 2024-04-20T11:28:16.423 INFO:teuthology.orchestra.run.smithi152.stderr:pool 'cephfs_metadata' created 2024-04-20T11:28:16.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:16.426+0000 7f2e17e00640 1 -- 172.21.15.152:0/1437129843 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f2df406c920 msgr2=0x7f2df406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:16.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:16.426+0000 7f2e17e00640 1 --2- 172.21.15.152:0/1437129843 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f2df406c920 0x7f2df406ede0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f2e08009930 tx=0x7f2e08008040 comp rx=0 tx=0).stop 2024-04-20T11:28:16.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:16.426+0000 7f2e17e00640 1 -- 172.21.15.152:0/1437129843 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2e1809dbc0 msgr2=0x7f2e18146180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:16.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:16.426+0000 7f2e17e00640 1 --2- 172.21.15.152:0/1437129843 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2e1809dbc0 0x7f2e18146180 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f2e00037040 tx=0x7f2e00004730 comp rx=0 tx=0).stop 2024-04-20T11:28:16.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:16.427+0000 7f2e17e00640 1 -- 172.21.15.152:0/1437129843 shutdown_connections 2024-04-20T11:28:16.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:16.427+0000 7f2e17e00640 1 --2- 172.21.15.152:0/1437129843 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f2df406c920 0x7f2df406ede0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:16.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:16.427+0000 7f2e17e00640 1 --2- 172.21.15.152:0/1437129843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2e18156c90 0x7f2e18157110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:16.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:16.427+0000 7f2e17e00640 1 --2- 172.21.15.152:0/1437129843 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2e1809dbc0 0x7f2e18146180 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:16.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:16.427+0000 7f2e17e00640 1 --2- 172.21.15.152:0/1437129843 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2dfc003f30 0x7f2e181f4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:16.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:16.427+0000 7f2e17e00640 1 -- 172.21.15.152:0/1437129843 >> 172.21.15.152:0/1437129843 conn(0x7f2e180b21e0 msgr2=0x7f2e180b4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:16.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:16.427+0000 7f2e17e00640 1 -- 172.21.15.152:0/1437129843 shutdown_connections 2024-04-20T11:28:16.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:16.427+0000 7f2e17e00640 1 -- 172.21.15.152:0/1437129843 wait complete. 2024-04-20T11:28:16.454 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-20T11:28:16.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:16.657+0000 7f3ecfe00640 1 Processor -- start 2024-04-20T11:28:16.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:16.657+0000 7f3ecfe00640 1 -- start start 2024-04-20T11:28:16.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:16.658+0000 7f3ecfe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ed006e0f0 0x7f3ed0166220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:16.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:16.659+0000 7f3ecfe00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ed00cfef0 con 0x7f3ed00d0e60 2024-04-20T11:28:16.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:16.659+0000 7f3ecfe00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ed015c3d0 con 0x7f3ed0159320 2024-04-20T11:28:16.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:16.659+0000 7f3ecfe00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f3ed015c6b0 con 0x7f3ed006e0f0 2024-04-20T11:28:16.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:16.659+0000 7f3ece000640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f3ed00d0e60 0x7f3ed006d9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:57926/0 (socket says 172.21.15.152:57926) 2024-04-20T11:28:16.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:16.659+0000 7f3ece000640 1 -- 172.21.15.152:0/1363919851 learned_addr learned my addr 172.21.15.152:0/1363919851 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:28:16.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:16.659+0000 7f3ecf400640 1 --2- 172.21.15.152:0/1363919851 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ed006e0f0 0x7f3ed0166220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:16.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:16.659+0000 7f3ecd600640 1 -- 172.21.15.152:0/1363919851 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3871071140 0 0) 0x7f3ed00cfef0 con 0x7f3ed00d0e60 2024-04-20T11:28:16.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:16.659+0000 7f3ecd600640 1 -- 172.21.15.152:0/1363919851 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3eb4003520 con 0x7f3ed00d0e60 2024-04-20T11:28:16.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:16.660+0000 7f3ecd600640 1 -- 172.21.15.152:0/1363919851 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 354675272 0 0) 0x7f3eb4003520 con 0x7f3ed00d0e60 2024-04-20T11:28:16.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:16.660+0000 7f3ecd600640 1 -- 172.21.15.152:0/1363919851 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ed006e0f0 msgr2=0x7f3ed0166220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:16.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:16.660+0000 7f3ecd600640 1 --2- 172.21.15.152:0/1363919851 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ed006e0f0 0x7f3ed0166220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:16.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:16.660+0000 7f3ecd600640 1 -- 172.21.15.152:0/1363919851 >> v1:172.21.15.175:6789/0 conn(0x7f3ed0159320 legacy=0x7f3ed015bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:16.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:16.660+0000 7f3ecd600640 1 -- 172.21.15.152:0/1363919851 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ed015c990 con 0x7f3ed00d0e60 2024-04-20T11:28:16.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:16.660+0000 7f3ecd600640 1 -- 172.21.15.152:0/1363919851 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f3ebc001670 con 0x7f3ed00d0e60 2024-04-20T11:28:16.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:16.660+0000 7f3ecd600640 1 -- 172.21.15.152:0/1363919851 >> v1:172.21.15.152:6789/0 conn(0x7f3ed00d0e60 legacy=0x7f3ed006d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:16.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:16.660+0000 7f3ecd600640 1 --2- 172.21.15.152:0/1363919851 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3eb40039e0 0x7f3eb4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:16.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:16.660+0000 7f3ecd600640 1 --2- 172.21.15.152:0/1363919851 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3eb4004800 0x7f3eb4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:16.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:16.660+0000 7f3ecd600640 1 --2- 172.21.15.152:0/1363919851 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ed006e0f0 0x7f3eb40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:16.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:16.660+0000 7f3ecd600640 1 -- 172.21.15.152:0/1363919851 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed015c6b0 con 0x7f3eb40039e0 2024-04-20T11:28:16.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:16.660+0000 7f3ecd600640 1 -- 172.21.15.152:0/1363919851 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb400baa0 con 0x7f3eb4004800 2024-04-20T11:28:16.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:16.660+0000 7f3ecd600640 1 -- 172.21.15.152:0/1363919851 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f3eb400bca0 con 0x7f3ed006e0f0 2024-04-20T11:28:16.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:16.660+0000 7f3ecea00640 1 --2- 172.21.15.152:0/1363919851 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3eb40039e0 0x7f3eb4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:16.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:16.661+0000 7f3ece000640 1 --2- 172.21.15.152:0/1363919851 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3eb4004800 0x7f3eb4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:16.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:16.661+0000 7f3ecea00640 1 -- 172.21.15.152:0/1363919851 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ed006e0f0 msgr2=0x7f3eb40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:16.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:16.661+0000 7f3ecea00640 1 --2- 172.21.15.152:0/1363919851 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ed006e0f0 0x7f3eb40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:16.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:16.661+0000 7f3ecea00640 1 -- 172.21.15.152:0/1363919851 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3eb4004800 msgr2=0x7f3eb4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:16.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:16.661+0000 7f3ecea00640 1 --2- 172.21.15.152:0/1363919851 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3eb4004800 0x7f3eb4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:16.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:16.661+0000 7f3ecea00640 1 -- 172.21.15.152:0/1363919851 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3eb4003520 con 0x7f3eb40039e0 2024-04-20T11:28:16.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:16.662+0000 7f3ecea00640 1 --2- 172.21.15.152:0/1363919851 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3eb40039e0 0x7f3eb4003e00 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f3ec0001330 tx=0x7f3eb4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4ec78c1ae28a2cb server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:16.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:16.662+0000 7f3ecd600640 1 -- 172.21.15.152:0/1363919851 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec0039070 con 0x7f3eb40039e0 2024-04-20T11:28:16.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:16.662+0000 7f3ecd600640 1 -- 172.21.15.152:0/1363919851 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ec0030070 con 0x7f3eb40039e0 2024-04-20T11:28:16.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:16.662+0000 7f3ecd600640 1 -- 172.21.15.152:0/1363919851 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec0001370 con 0x7f3eb40039e0 2024-04-20T11:28:16.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:16.663+0000 7f3ecfe00640 1 -- 172.21.15.152:0/1363919851 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3eb40039e0 msgr2=0x7f3eb4003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:16.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:16.663+0000 7f3ecfe00640 1 --2- 172.21.15.152:0/1363919851 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3eb40039e0 0x7f3eb4003e00 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f3ec0001330 tx=0x7f3eb4005830 comp rx=0 tx=0).stop 2024-04-20T11:28:16.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:16.663+0000 7f3ecf400640 1 -- 172.21.15.152:0/1363919851 reap_dead start 2024-04-20T11:28:16.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:16.663+0000 7f3ecfe00640 1 -- 172.21.15.152:0/1363919851 shutdown_connections 2024-04-20T11:28:16.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:16.663+0000 7f3ecfe00640 1 -- 172.21.15.152:0/1363919851 >> 172.21.15.152:0/1363919851 conn(0x7f3ed00d0260 msgr2=0x7f3ed00d0670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:16.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:16.663+0000 7f3ecfe00640 1 -- 172.21.15.152:0/1363919851 shutdown_connections 2024-04-20T11:28:16.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:16.663+0000 7f3ecfe00640 1 -- 172.21.15.152:0/1363919851 wait complete. 2024-04-20T11:28:16.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:16.665+0000 7f3ecfe00640 1 Processor -- start 2024-04-20T11:28:16.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:16.665+0000 7f3ecfe00640 1 -- start start 2024-04-20T11:28:16.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:16.666+0000 7f3ecfe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3eb40039e0 0x7f3ed01541f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:16.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:16.666+0000 7f3ecfe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3ed0157040 0x7f3ed0157aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:16.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:16.666+0000 7f3ecfe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ed0151030 0x7f3ed01514b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:16.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:16.666+0000 7f3ecfe00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed015e5e0 con 0x7f3ed0157040 2024-04-20T11:28:16.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:16.666+0000 7f3ecfe00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed015e460 con 0x7f3ed0151030 2024-04-20T11:28:16.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:16.666+0000 7f3ecfe00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f3ed015e760 con 0x7f3eb40039e0 2024-04-20T11:28:16.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:16.666+0000 7f3ece000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3ed0157040 0x7f3ed0157aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:16.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:16.666+0000 7f3ece000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3ed0157040 0x7f3ed0157aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:41078/0 (socket says 172.21.15.152:41078) 2024-04-20T11:28:16.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:16.666+0000 7f3ece000640 1 -- 172.21.15.152:0/4191557515 learned_addr learned my addr 172.21.15.152:0/4191557515 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:16.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:16.667+0000 7f3ecea00640 1 --2- 172.21.15.152:0/4191557515 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3eb40039e0 0x7f3ed01541f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:16.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:16.667+0000 7f3ecf400640 1 --2- 172.21.15.152:0/4191557515 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ed0151030 0x7f3ed01514b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:16.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:16.667+0000 7f3ece000640 1 -- 172.21.15.152:0/4191557515 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3eb40039e0 msgr2=0x7f3ed01541f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:16.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:16.667+0000 7f3ece000640 1 --2- 172.21.15.152:0/4191557515 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3eb40039e0 0x7f3ed01541f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:16.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:16.667+0000 7f3ece000640 1 -- 172.21.15.152:0/4191557515 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ed0151030 msgr2=0x7f3ed01514b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:16.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:16.667+0000 7f3ece000640 1 --2- 172.21.15.152:0/4191557515 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ed0151030 0x7f3ed01514b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:16.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:16.667+0000 7f3ece000640 1 -- 172.21.15.152:0/4191557515 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ed01519f0 con 0x7f3ed0157040 2024-04-20T11:28:16.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:16.667+0000 7f3ece000640 1 --2- 172.21.15.152:0/4191557515 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3ed0157040 0x7f3ed0157aa0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc00e7e0 tx=0x7f3ebc00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:16.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:16.667+0000 7f3ec7400640 1 -- 172.21.15.152:0/4191557515 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ebc00c870 con 0x7f3ed0157040 2024-04-20T11:28:16.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:16.667+0000 7f3ec7400640 1 -- 172.21.15.152:0/4191557515 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ebc018070 con 0x7f3ed0157040 2024-04-20T11:28:16.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:16.667+0000 7f3ec7400640 1 -- 172.21.15.152:0/4191557515 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ebc00cc50 con 0x7f3ed0157040 2024-04-20T11:28:16.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:16.667+0000 7f3ecfe00640 1 -- 172.21.15.152:0/4191557515 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ed0154820 con 0x7f3ed0157040 2024-04-20T11:28:16.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:16.668+0000 7f3ecfe00640 1 -- 172.21.15.152:0/4191557515 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ed0154de0 con 0x7f3ed0157040 2024-04-20T11:28:16.670 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:16.669+0000 7f3ecfe00640 1 -- 172.21.15.152:0/4191557515 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ed01553f0 con 0x7f3ed0157040 2024-04-20T11:28:16.670 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:16.671+0000 7f3ec7400640 1 -- 172.21.15.152:0/4191557515 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3ebc01d460 con 0x7f3ed0157040 2024-04-20T11:28:16.671 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:16.671+0000 7f3ec7400640 1 --2- 172.21.15.152:0/4191557515 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f3eb006c8f0 0x7f3eb006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:16.672 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:16.671+0000 7f3ec7400640 1 -- 172.21.15.152:0/4191557515 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5227+0+0 (secure 0 0 0) 0x7f3ebc08c380 con 0x7f3ed0157040 2024-04-20T11:28:16.672 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:16.672+0000 7f3ecea00640 1 --2- 172.21.15.152:0/4191557515 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f3eb006c8f0 0x7f3eb006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:16.672 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:16.673+0000 7f3ecea00640 1 --2- 172.21.15.152:0/4191557515 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f3eb006c8f0 0x7f3eb006edb0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f3ec0036000 tx=0x7f3ec0009030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:16.675 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:16.676+0000 7f3ec7400640 1 -- 172.21.15.152:0/4191557515 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f3ed01519f0 con 0x7f3ed0157040 2024-04-20T11:28:16.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:16.874+0000 7f3ecfe00640 1 -- 172.21.15.152:0/4191557515 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f3ed0075e60 con 0x7f3ed0157040 2024-04-20T11:28:17.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:17.429+0000 7f3ec7400640 1 -- 172.21.15.152:0/4191557515 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v32) ==== 170+0+0 (secure 0 0 0) 0x7f3ebc05a970 con 0x7f3ed0157040 2024-04-20T11:28:17.429 INFO:teuthology.orchestra.run.smithi152.stderr:pool 'cephfs_data' created 2024-04-20T11:28:17.433 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:17.433+0000 7f3ecfe00640 1 -- 172.21.15.152:0/4191557515 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f3eb006c8f0 msgr2=0x7f3eb006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:17.433 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:17.433+0000 7f3ecfe00640 1 --2- 172.21.15.152:0/4191557515 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f3eb006c8f0 0x7f3eb006edb0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f3ec0036000 tx=0x7f3ec0009030 comp rx=0 tx=0).stop 2024-04-20T11:28:17.433 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:17.433+0000 7f3ecfe00640 1 -- 172.21.15.152:0/4191557515 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3ed0157040 msgr2=0x7f3ed0157aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:17.433 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:17.433+0000 7f3ecfe00640 1 --2- 172.21.15.152:0/4191557515 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3ed0157040 0x7f3ed0157aa0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc00e7e0 tx=0x7f3ebc00eca0 comp rx=0 tx=0).stop 2024-04-20T11:28:17.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:17.435+0000 7f3ecfe00640 1 -- 172.21.15.152:0/4191557515 shutdown_connections 2024-04-20T11:28:17.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:17.435+0000 7f3ecfe00640 1 --2- 172.21.15.152:0/4191557515 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f3eb006c8f0 0x7f3eb006edb0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:17.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:17.435+0000 7f3ecfe00640 1 --2- 172.21.15.152:0/4191557515 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ed0151030 0x7f3ed01514b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:17.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:17.435+0000 7f3ecfe00640 1 --2- 172.21.15.152:0/4191557515 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3ed0157040 0x7f3ed0157aa0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:17.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:17.435+0000 7f3ecfe00640 1 --2- 172.21.15.152:0/4191557515 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3eb40039e0 0x7f3ed01541f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:17.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:17.435+0000 7f3ecfe00640 1 -- 172.21.15.152:0/4191557515 >> 172.21.15.152:0/4191557515 conn(0x7f3ed00d0260 msgr2=0x7f3ed0071cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:17.438 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:17.438+0000 7f3ecfe00640 1 -- 172.21.15.152:0/4191557515 shutdown_connections 2024-04-20T11:28:17.439 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:17.438+0000 7f3ecfe00640 1 -- 172.21.15.152:0/4191557515 wait complete. 2024-04-20T11:28:17.471 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-20T11:28:17.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:17.647+0000 7fc9fdc00640 1 Processor -- start 2024-04-20T11:28:17.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:17.647+0000 7fc9fdc00640 1 -- start start 2024-04-20T11:28:17.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:17.649+0000 7fc9fdc00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc9f815eb50 0x7fc9f80d0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:17.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:17.649+0000 7fc9fdc00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fc9f80d35a0 0x7fc9f80cf470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:17.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:17.649+0000 7fc9fdc00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9f80d19c0 con 0x7fc9f8163230 2024-04-20T11:28:17.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:17.649+0000 7fc9fdc00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fc9f8162750 con 0x7fc9f80d35a0 2024-04-20T11:28:17.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:17.649+0000 7fc9fdc00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc9f8162a30 con 0x7fc9f815eb50 2024-04-20T11:28:17.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:17.649+0000 7fc9fd200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fc9f80d35a0 0x7fc9f80cf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:17.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:17.649+0000 7fc9f7400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fc9f8163230 0x7fc9f80d2e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:57942/0 (socket says 172.21.15.152:57942) 2024-04-20T11:28:17.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:17.650+0000 7fc9fd200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fc9f80d35a0 0x7fc9f80cf470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:41084/0 (socket says 172.21.15.152:41084) 2024-04-20T11:28:17.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:17.650+0000 7fc9f7400640 1 -- 172.21.15.152:0/3432940797 learned_addr learned my addr 172.21.15.152:0/3432940797 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:28:17.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:17.650+0000 7fc9f7e00640 1 --2- 172.21.15.152:0/3432940797 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc9f815eb50 0x7fc9f80d0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:17.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:17.650+0000 7fc9fd200640 1 -- 172.21.15.152:0/3432940797 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc9f815eb50 msgr2=0x7fc9f80d0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:17.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:17.650+0000 7fc9fd200640 1 --2- 172.21.15.152:0/3432940797 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc9f815eb50 0x7fc9f80d0f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:17.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:17.650+0000 7fc9fd200640 1 -- 172.21.15.152:0/3432940797 >> v1:172.21.15.152:6789/0 conn(0x7fc9f8163230 legacy=0x7fc9f80d2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:17.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:17.650+0000 7fc9fd200640 1 -- 172.21.15.152:0/3432940797 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9f8162d10 con 0x7fc9f80d35a0 2024-04-20T11:28:17.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:17.650+0000 7fc9f6a00640 1 -- 172.21.15.152:0/3432940797 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3011771042 0 0) 0x7fc9f80d19c0 con 0x7fc9f8163230 2024-04-20T11:28:17.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:17.651+0000 7fc9fd200640 1 --2- 172.21.15.152:0/3432940797 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc9f80d35a0 0x7fc9f80cf470 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fc9f00097a0 tx=0x7fc9f002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0d8290d3b814df0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:17.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:17.651+0000 7fc9f6a00640 1 -- 172.21.15.152:0/3432940797 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9f00315a0 con 0x7fc9f80d35a0 2024-04-20T11:28:17.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:17.652+0000 7fc9f6a00640 1 -- 172.21.15.152:0/3432940797 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9f0037070 con 0x7fc9f80d35a0 2024-04-20T11:28:17.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:17.652+0000 7fc9f6a00640 1 -- 172.21.15.152:0/3432940797 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9f0031960 con 0x7fc9f80d35a0 2024-04-20T11:28:17.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:17.652+0000 7fc9fdc00640 1 -- 172.21.15.152:0/3432940797 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc9f80d35a0 msgr2=0x7fc9f80cf470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:17.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:17.652+0000 7fc9fdc00640 1 --2- 172.21.15.152:0/3432940797 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc9f80d35a0 0x7fc9f80cf470 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fc9f00097a0 tx=0x7fc9f002b840 comp rx=0 tx=0).stop 2024-04-20T11:28:17.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:17.652+0000 7fc9fdc00640 1 -- 172.21.15.152:0/3432940797 shutdown_connections 2024-04-20T11:28:17.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:17.652+0000 7fc9fdc00640 1 --2- 172.21.15.152:0/3432940797 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc9f80d35a0 0x7fc9f80cf470 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:17.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:17.652+0000 7fc9fdc00640 1 --2- 172.21.15.152:0/3432940797 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc9f815eb50 0x7fc9f80d0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:17.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:17.652+0000 7fc9fdc00640 1 -- 172.21.15.152:0/3432940797 >> 172.21.15.152:0/3432940797 conn(0x7fc9f808bfd0 msgr2=0x7fc9f80d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:17.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:17.652+0000 7fc9fdc00640 1 -- 172.21.15.152:0/3432940797 shutdown_connections 2024-04-20T11:28:17.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:17.652+0000 7fc9fdc00640 1 -- 172.21.15.152:0/3432940797 wait complete. 2024-04-20T11:28:17.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:17.654+0000 7fc9fdc00640 1 Processor -- start 2024-04-20T11:28:17.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:17.654+0000 7fc9fdc00640 1 -- start start 2024-04-20T11:28:17.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:17.655+0000 7fc9fdc00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc9f80d35a0 0x7fc9f80b5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:17.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:17.655+0000 7fc9fdc00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc9f815eb50 0x7fc9f80b4b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:17.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:17.655+0000 7fc9fdc00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc9f8163230 0x7fc9f8159620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:17.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:17.655+0000 7fc9fdc00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f80d15c0 con 0x7fc9f815eb50 2024-04-20T11:28:17.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:17.655+0000 7fc9fdc00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f80d1440 con 0x7fc9f8163230 2024-04-20T11:28:17.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:17.655+0000 7fc9fdc00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc9f80d1740 con 0x7fc9f80d35a0 2024-04-20T11:28:17.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:17.656+0000 7fc9f7400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc9f815eb50 0x7fc9f80b4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:17.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:17.656+0000 7fc9f7400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc9f815eb50 0x7fc9f80b4b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:41086/0 (socket says 172.21.15.152:41086) 2024-04-20T11:28:17.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:17.656+0000 7fc9f7400640 1 -- 172.21.15.152:0/3660218175 learned_addr learned my addr 172.21.15.152:0/3660218175 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:17.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:17.656+0000 7fc9fd200640 1 --2- 172.21.15.152:0/3660218175 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc9f8163230 0x7fc9f8159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:17.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:17.656+0000 7fc9f7400640 1 -- 172.21.15.152:0/3660218175 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc9f80d35a0 msgr2=0x7fc9f80b5870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:17.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:17.656+0000 7fc9f7400640 1 --2- 172.21.15.152:0/3660218175 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc9f80d35a0 0x7fc9f80b5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:17.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:17.656+0000 7fc9f7400640 1 -- 172.21.15.152:0/3660218175 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc9f8163230 msgr2=0x7fc9f8159620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:17.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:17.656+0000 7fc9f7400640 1 --2- 172.21.15.152:0/3660218175 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc9f8163230 0x7fc9f8159620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:17.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:17.656+0000 7fc9f7400640 1 -- 172.21.15.152:0/3660218175 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9f80b5f30 con 0x7fc9f815eb50 2024-04-20T11:28:17.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:17.656+0000 7fc9f7400640 1 --2- 172.21.15.152:0/3660218175 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc9f815eb50 0x7fc9f80b4b20 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fc9e000c750 tx=0x7fc9e000cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:17.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:17.657+0000 7fc9efe00640 1 -- 172.21.15.152:0/3660218175 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9e000ce60 con 0x7fc9f815eb50 2024-04-20T11:28:17.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:17.657+0000 7fc9efe00640 1 -- 172.21.15.152:0/3660218175 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9e000a640 con 0x7fc9f815eb50 2024-04-20T11:28:17.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:17.657+0000 7fc9efe00640 1 -- 172.21.15.152:0/3660218175 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9e000aa30 con 0x7fc9f815eb50 2024-04-20T11:28:17.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:17.657+0000 7fc9fdc00640 1 -- 172.21.15.152:0/3660218175 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9f80b7130 con 0x7fc9f815eb50 2024-04-20T11:28:17.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:17.657+0000 7fc9fdc00640 1 -- 172.21.15.152:0/3660218175 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9f80b76f0 con 0x7fc9f815eb50 2024-04-20T11:28:17.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:17.658+0000 7fc9fdc00640 1 -- 172.21.15.152:0/3660218175 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9f8075e60 con 0x7fc9f815eb50 2024-04-20T11:28:17.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:17.659+0000 7fc9efe00640 1 -- 172.21.15.152:0/3660218175 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc9e000abd0 con 0x7fc9f815eb50 2024-04-20T11:28:17.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:17.660+0000 7fc9efe00640 1 --2- 172.21.15.152:0/3660218175 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc9d406c8f0 0x7fc9d406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:17.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:17.660+0000 7fc9efe00640 1 -- 172.21.15.152:0/3660218175 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5609+0+0 (secure 0 0 0) 0x7fc9e008ba60 con 0x7fc9f815eb50 2024-04-20T11:28:17.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:17.661+0000 7fc9f7e00640 1 --2- 172.21.15.152:0/3660218175 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc9d406c8f0 0x7fc9d406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:17.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:17.661+0000 7fc9f7e00640 1 --2- 172.21.15.152:0/3660218175 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc9d406c8f0 0x7fc9d406edb0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc9e80098f0 tx=0x7fc9e8008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:17.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:17.666+0000 7fc9efe00640 1 -- 172.21.15.152:0/3660218175 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fc9e0059ee0 con 0x7fc9f815eb50 2024-04-20T11:28:17.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:17.901+0000 7fc9fdc00640 1 -- 172.21.15.152:0/3660218175 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fc9f808c680 con 0x7fc9f815eb50 2024-04-20T11:28:18.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:18.455+0000 7fc9efe00640 1 -- 172.21.15.152:0/3660218175 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v12) ==== 172+0+0 (secure 0 0 0) 0x7fc9e00597e0 con 0x7fc9f815eb50 2024-04-20T11:28:18.455 INFO:teuthology.orchestra.run.smithi152.stderr:new fs with metadata pool 4 and data pool 5 2024-04-20T11:28:18.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:18.459+0000 7fc9fdc00640 1 -- 172.21.15.152:0/3660218175 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc9d406c8f0 msgr2=0x7fc9d406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:18.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:18.459+0000 7fc9fdc00640 1 --2- 172.21.15.152:0/3660218175 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc9d406c8f0 0x7fc9d406edb0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc9e80098f0 tx=0x7fc9e8008040 comp rx=0 tx=0).stop 2024-04-20T11:28:18.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:18.459+0000 7fc9fdc00640 1 -- 172.21.15.152:0/3660218175 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc9f815eb50 msgr2=0x7fc9f80b4b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:18.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:18.459+0000 7fc9fdc00640 1 --2- 172.21.15.152:0/3660218175 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc9f815eb50 0x7fc9f80b4b20 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fc9e000c750 tx=0x7fc9e000cc10 comp rx=0 tx=0).stop 2024-04-20T11:28:18.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:18.459+0000 7fc9fdc00640 1 -- 172.21.15.152:0/3660218175 shutdown_connections 2024-04-20T11:28:18.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:18.459+0000 7fc9fdc00640 1 --2- 172.21.15.152:0/3660218175 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc9d406c8f0 0x7fc9d406edb0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:18.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:18.459+0000 7fc9fdc00640 1 --2- 172.21.15.152:0/3660218175 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc9f8163230 0x7fc9f8159620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:18.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:18.459+0000 7fc9fdc00640 1 --2- 172.21.15.152:0/3660218175 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc9f815eb50 0x7fc9f80b4b20 secure :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fc9e000c750 tx=0x7fc9e000cc10 comp rx=0 tx=0).stop 2024-04-20T11:28:18.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:18.459+0000 7fc9fdc00640 1 --2- 172.21.15.152:0/3660218175 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc9f80d35a0 0x7fc9f80b5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:18.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:18.459+0000 7fc9fdc00640 1 -- 172.21.15.152:0/3660218175 >> 172.21.15.152:0/3660218175 conn(0x7fc9f808bfd0 msgr2=0x7fc9f80b24d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:18.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:18.460+0000 7fc9fdc00640 1 -- 172.21.15.152:0/3660218175 shutdown_connections 2024-04-20T11:28:18.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:18.460+0000 7fc9fdc00640 1 -- 172.21.15.152:0/3660218175 wait complete. 2024-04-20T11:28:18.503 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:28:18.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:18.666+0000 7fa747600640 1 Processor -- start 2024-04-20T11:28:18.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:18.666+0000 7fa747600640 1 -- start start 2024-04-20T11:28:18.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:18.667+0000 7fa747600640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fa740145f60 0x7fa74006ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:18.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:18.667+0000 7fa747600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa7400726f0 0x7fa7400cd580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:18.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:18.667+0000 7fa747600640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa740146340 con 0x7fa7400cd0d0 2024-04-20T11:28:18.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:18.667+0000 7fa747600640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fa74006e9f0 con 0x7fa740145f60 2024-04-20T11:28:18.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:18.667+0000 7fa747600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa74006f3c0 con 0x7fa7400726f0 2024-04-20T11:28:18.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:18.668+0000 7fa744c00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fa740145f60 0x7fa74006ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:18.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:18.668+0000 7fa744c00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fa740145f60 0x7fa74006ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:41100/0 (socket says 172.21.15.152:41100) 2024-04-20T11:28:18.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:18.668+0000 7fa744c00640 1 -- 172.21.15.152:0/3016328094 learned_addr learned my addr 172.21.15.152:0/3016328094 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:18.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:18.668+0000 7fa745600640 1 --2- 172.21.15.152:0/3016328094 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa7400726f0 0x7fa7400cd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:18.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:18.668+0000 7fa744c00640 1 -- 172.21.15.152:0/3016328094 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa7400726f0 msgr2=0x7fa7400cd580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:18.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:18.668+0000 7fa744c00640 1 --2- 172.21.15.152:0/3016328094 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa7400726f0 0x7fa7400cd580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:18.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:18.668+0000 7fa744c00640 1 -- 172.21.15.152:0/3016328094 >> v1:172.21.15.152:6789/0 conn(0x7fa7400cd0d0 legacy=0x7fa740071fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:18.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:18.668+0000 7fa744c00640 1 -- 172.21.15.152:0/3016328094 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa740072d50 con 0x7fa740145f60 2024-04-20T11:28:18.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:18.669+0000 7fa744c00640 1 --2- 172.21.15.152:0/3016328094 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa740145f60 0x7fa74006ee80 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fa738009e50 tx=0x7fa73802bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f61cae59d186429a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:18.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:18.669+0000 7fa73f400640 1 -- 172.21.15.152:0/3016328094 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa73802cb00 con 0x7fa740145f60 2024-04-20T11:28:18.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:18.669+0000 7fa73f400640 1 -- 172.21.15.152:0/3016328094 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa73802cca0 con 0x7fa740145f60 2024-04-20T11:28:18.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:18.669+0000 7fa73f400640 1 -- 172.21.15.152:0/3016328094 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa738034610 con 0x7fa740145f60 2024-04-20T11:28:18.670 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:18.670+0000 7fa747600640 1 -- 172.21.15.152:0/3016328094 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa740145f60 msgr2=0x7fa74006ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:18.670 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:18.670+0000 7fa747600640 1 --2- 172.21.15.152:0/3016328094 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa740145f60 0x7fa74006ee80 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fa738009e50 tx=0x7fa73802bf60 comp rx=0 tx=0).stop 2024-04-20T11:28:18.670 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:18.670+0000 7fa747600640 1 -- 172.21.15.152:0/3016328094 shutdown_connections 2024-04-20T11:28:18.670 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:18.670+0000 7fa747600640 1 --2- 172.21.15.152:0/3016328094 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa7400726f0 0x7fa7400cd580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:18.670 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:18.670+0000 7fa747600640 1 --2- 172.21.15.152:0/3016328094 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa740145f60 0x7fa74006ee80 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:18.670 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:18.670+0000 7fa747600640 1 -- 172.21.15.152:0/3016328094 >> 172.21.15.152:0/3016328094 conn(0x7fa740067cf0 msgr2=0x7fa74006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:18.670 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:18.670+0000 7fa747600640 1 -- 172.21.15.152:0/3016328094 shutdown_connections 2024-04-20T11:28:18.670 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:18.670+0000 7fa747600640 1 -- 172.21.15.152:0/3016328094 wait complete. 2024-04-20T11:28:18.671 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:18.672+0000 7fa747600640 1 Processor -- start 2024-04-20T11:28:18.671 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:18.672+0000 7fa747600640 1 -- start start 2024-04-20T11:28:18.672 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:18.673+0000 7fa747600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa7400726f0 0x7fa7400aeb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:18.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:18.673+0000 7fa747600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7400cd0d0 0x7fa74015acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:18.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:18.673+0000 7fa747600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa740145f60 0x7fa7400b4960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:18.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:18.673+0000 7fa747600640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fa74006e680 con 0x7fa740145f60 2024-04-20T11:28:18.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:18.673+0000 7fa747600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa74006e3a0 con 0x7fa7400cd0d0 2024-04-20T11:28:18.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:18.673+0000 7fa747600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa7400cf290 con 0x7fa7400726f0 2024-04-20T11:28:18.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:18.673+0000 7fa745600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa740145f60 0x7fa7400b4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:18.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:18.673+0000 7fa745600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa740145f60 0x7fa7400b4960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:41116/0 (socket says 172.21.15.152:41116) 2024-04-20T11:28:18.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:18.673+0000 7fa745600640 1 -- 172.21.15.152:0/98750975 learned_addr learned my addr 172.21.15.152:0/98750975 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:18.674 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:18.674+0000 7fa73fe00640 1 --2- 172.21.15.152:0/98750975 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7400cd0d0 0x7fa74015acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:18.674 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:18.674+0000 7fa744c00640 1 --2- 172.21.15.152:0/98750975 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa7400726f0 0x7fa7400aeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:18.675 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:18.674+0000 7fa745600640 1 -- 172.21.15.152:0/98750975 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa7400726f0 msgr2=0x7fa7400aeb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:18.675 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:18.674+0000 7fa745600640 1 --2- 172.21.15.152:0/98750975 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa7400726f0 0x7fa7400aeb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:18.675 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:18.674+0000 7fa745600640 1 -- 172.21.15.152:0/98750975 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7400cd0d0 msgr2=0x7fa74015acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:18.675 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:18.674+0000 7fa745600640 1 --2- 172.21.15.152:0/98750975 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7400cd0d0 0x7fa74015acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:18.675 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:18.674+0000 7fa745600640 1 -- 172.21.15.152:0/98750975 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7400b38c0 con 0x7fa740145f60 2024-04-20T11:28:18.675 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:18.674+0000 7fa73fe00640 1 --2- 172.21.15.152:0/98750975 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7400cd0d0 0x7fa74015acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:28:18.675 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:18.674+0000 7fa744c00640 1 --2- 172.21.15.152:0/98750975 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa7400726f0 0x7fa7400aeb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:28:18.675 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:18.674+0000 7fa745600640 1 --2- 172.21.15.152:0/98750975 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa740145f60 0x7fa7400b4960 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fa73000c7f0 tx=0x7fa73000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:18.675 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:18.674+0000 7fa73cc00640 1 -- 172.21.15.152:0/98750975 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa730015070 con 0x7fa740145f60 2024-04-20T11:28:18.675 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:18.674+0000 7fa73cc00640 1 -- 172.21.15.152:0/98750975 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa730004220 con 0x7fa740145f60 2024-04-20T11:28:18.675 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:18.674+0000 7fa73cc00640 1 -- 172.21.15.152:0/98750975 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa730007620 con 0x7fa740145f60 2024-04-20T11:28:18.675 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:18.674+0000 7fa747600640 1 -- 172.21.15.152:0/98750975 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7400b3ba0 con 0x7fa740145f60 2024-04-20T11:28:18.675 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:18.674+0000 7fa747600640 1 -- 172.21.15.152:0/98750975 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7400b4160 con 0x7fa740145f60 2024-04-20T11:28:18.676 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:18.675+0000 7fa747600640 1 -- 172.21.15.152:0/98750975 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7400730a0 con 0x7fa740145f60 2024-04-20T11:28:18.676 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:18.676+0000 7fa73cc00640 1 -- 172.21.15.152:0/98750975 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa7300079d0 con 0x7fa740145f60 2024-04-20T11:28:18.677 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:18.677+0000 7fa73cc00640 1 --2- 172.21.15.152:0/98750975 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fa71c06c8f0 0x7fa71c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:18.677 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:18.677+0000 7fa73cc00640 1 -- 172.21.15.152:0/98750975 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fa73008bcb0 con 0x7fa740145f60 2024-04-20T11:28:18.677 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:18.677+0000 7fa744c00640 1 --2- 172.21.15.152:0/98750975 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fa71c06c8f0 0x7fa71c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:18.678 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:18.678+0000 7fa744c00640 1 --2- 172.21.15.152:0/98750975 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fa71c06c8f0 0x7fa71c06edb0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa738035000 tx=0x7fa738030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:18.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:18.683+0000 7fa73cc00640 1 -- 172.21.15.152:0/98750975 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fa7400730a0 con 0x7fa740145f60 2024-04-20T11:28:18.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:18.866+0000 7fa747600640 1 -- 172.21.15.152:0/98750975 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa7400689f0 con 0x7fa740145f60 2024-04-20T11:28:18.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:18.868+0000 7fa73cc00640 1 -- 172.21.15.152:0/98750975 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) ==== 74+0+16742 (secure 0 0 0) 0x7fa73005a040 con 0x7fa740145f60 2024-04-20T11:28:18.868 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:28:18.868 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":33,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:28:17.902771+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:28:15.704670+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-20T11:28:16.891101+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:28:18.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:18.871+0000 7fa747600640 1 -- 172.21.15.152:0/98750975 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fa71c06c8f0 msgr2=0x7fa71c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:18.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:18.871+0000 7fa747600640 1 --2- 172.21.15.152:0/98750975 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fa71c06c8f0 0x7fa71c06edb0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa738035000 tx=0x7fa738030000 comp rx=0 tx=0).stop 2024-04-20T11:28:18.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:18.871+0000 7fa747600640 1 -- 172.21.15.152:0/98750975 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa740145f60 msgr2=0x7fa7400b4960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:18.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:18.871+0000 7fa747600640 1 --2- 172.21.15.152:0/98750975 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa740145f60 0x7fa7400b4960 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fa73000c7f0 tx=0x7fa73000ccb0 comp rx=0 tx=0).stop 2024-04-20T11:28:18.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:18.871+0000 7fa747600640 1 -- 172.21.15.152:0/98750975 shutdown_connections 2024-04-20T11:28:18.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:18.871+0000 7fa747600640 1 --2- 172.21.15.152:0/98750975 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fa71c06c8f0 0x7fa71c06edb0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:18.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:18.871+0000 7fa747600640 1 --2- 172.21.15.152:0/98750975 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa740145f60 0x7fa7400b4960 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:18.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:18.871+0000 7fa747600640 1 --2- 172.21.15.152:0/98750975 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7400cd0d0 0x7fa74015acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:18.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:18.871+0000 7fa747600640 1 --2- 172.21.15.152:0/98750975 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa7400726f0 0x7fa7400aeb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:18.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:18.871+0000 7fa747600640 1 -- 172.21.15.152:0/98750975 >> 172.21.15.152:0/98750975 conn(0x7fa740067cf0 msgr2=0x7fa7400b1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:18.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:18.871+0000 7fa747600640 1 -- 172.21.15.152:0/98750975 shutdown_connections 2024-04-20T11:28:18.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:18.871+0000 7fa747600640 1 -- 172.21.15.152:0/98750975 wait complete. 2024-04-20T11:28:18.888 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:28:19.078 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.078+0000 7f3cef000640 1 Processor -- start 2024-04-20T11:28:19.078 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.078+0000 7f3cef000640 1 -- start start 2024-04-20T11:28:19.079 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.080+0000 7f3cef000640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f3ce80d2710 0x7f3ce815f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:19.079 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.080+0000 7f3cef000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ce80cec90 0x7f3ce80cf120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:19.080 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.080+0000 7f3cef000640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ce80d2b30 con 0x7f3ce80b5fe0 2024-04-20T11:28:19.080 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.080+0000 7f3cef000640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f3ce815fd90 con 0x7f3ce80d2710 2024-04-20T11:28:19.080 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.080+0000 7f3cef000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f3ce80d39f0 con 0x7f3ce80cec90 2024-04-20T11:28:19.080 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.080+0000 7f3cedc00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f3ce80b5fe0 0x7f3ce80b63c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:57964/0 (socket says 172.21.15.152:57964) 2024-04-20T11:28:19.080 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.080+0000 7f3cedc00640 1 -- 172.21.15.152:0/26076501 learned_addr learned my addr 172.21.15.152:0/26076501 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:28:19.082 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.081+0000 7f3cee600640 1 --2- 172.21.15.152:0/26076501 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ce80cec90 0x7f3ce80cf120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:19.082 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.081+0000 7f3ced200640 1 --2- 172.21.15.152:0/26076501 >> v2:172.21.15.175:3300/0 conn(0x7f3ce80d2710 0x7f3ce815f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:19.082 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.081+0000 7f3ce7e00640 1 -- 172.21.15.152:0/26076501 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4139686235 0 0) 0x7f3ce80d2b30 con 0x7f3ce80b5fe0 2024-04-20T11:28:19.082 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.081+0000 7f3ce7e00640 1 -- 172.21.15.152:0/26076501 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3cd0003520 con 0x7f3ce80b5fe0 2024-04-20T11:28:19.082 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.081+0000 7f3ce7e00640 1 -- 172.21.15.152:0/26076501 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 207224300 0 0) 0x7f3cd0003520 con 0x7f3ce80b5fe0 2024-04-20T11:28:19.082 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.081+0000 7f3ce7e00640 1 -- 172.21.15.152:0/26076501 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ce80cec90 msgr2=0x7f3ce80cf120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:19.082 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.081+0000 7f3ce7e00640 1 --2- 172.21.15.152:0/26076501 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ce80cec90 0x7f3ce80cf120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:19.082 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.081+0000 7f3ce7e00640 1 -- 172.21.15.152:0/26076501 >> v2:172.21.15.175:3300/0 conn(0x7f3ce80d2710 msgr2=0x7f3ce815f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:19.082 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.081+0000 7f3ce7e00640 1 --2- 172.21.15.152:0/26076501 >> v2:172.21.15.175:3300/0 conn(0x7f3ce80d2710 0x7f3ce815f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:19.082 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.081+0000 7f3ce7e00640 1 -- 172.21.15.152:0/26076501 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ce80d2de0 con 0x7f3ce80b5fe0 2024-04-20T11:28:19.082 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.081+0000 7f3ce7e00640 1 -- 172.21.15.152:0/26076501 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f3cdc001670 con 0x7f3ce80b5fe0 2024-04-20T11:28:19.082 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.081+0000 7f3ce7e00640 1 -- 172.21.15.152:0/26076501 >> v1:172.21.15.152:6789/0 conn(0x7f3ce80b5fe0 legacy=0x7f3ce80b63c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:19.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.081+0000 7f3ce7e00640 1 --2- 172.21.15.152:0/26076501 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3cd00039b0 0x7f3cd0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:19.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.081+0000 7f3ce7e00640 1 --2- 172.21.15.152:0/26076501 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3cd0004800 0x7f3cd0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:19.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.081+0000 7f3ce7e00640 1 --2- 172.21.15.152:0/26076501 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ce80cec90 0x7f3cd00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:19.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.081+0000 7f3ce7e00640 1 -- 172.21.15.152:0/26076501 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f3ce815fd90 con 0x7f3cd00039b0 2024-04-20T11:28:19.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.081+0000 7f3ce7e00640 1 -- 172.21.15.152:0/26076501 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3ce80d39f0 con 0x7f3cd0004800 2024-04-20T11:28:19.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.081+0000 7f3ce7e00640 1 -- 172.21.15.152:0/26076501 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f3cd000ba10 con 0x7f3ce80cec90 2024-04-20T11:28:19.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.082+0000 7f3cedc00640 1 --2- 172.21.15.152:0/26076501 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3cd00039b0 0x7f3cd0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:19.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.082+0000 7f3cedc00640 1 -- 172.21.15.152:0/26076501 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ce80cec90 msgr2=0x7f3cd00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:19.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.082+0000 7f3cedc00640 1 --2- 172.21.15.152:0/26076501 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ce80cec90 0x7f3cd00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:19.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.082+0000 7f3cedc00640 1 -- 172.21.15.152:0/26076501 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3cd0004800 msgr2=0x7f3cd0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:19.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.082+0000 7f3cedc00640 1 --2- 172.21.15.152:0/26076501 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3cd0004800 0x7f3cd0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:19.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.082+0000 7f3cedc00640 1 -- 172.21.15.152:0/26076501 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cd0003520 con 0x7f3cd00039b0 2024-04-20T11:28:19.084 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.083+0000 7f3cedc00640 1 --2- 172.21.15.152:0/26076501 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3cd00039b0 0x7f3cd0003dd0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc00c1c0 tx=0x7f3cd0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=533e25145cf19169 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:19.084 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.083+0000 7f3ce7e00640 1 -- 172.21.15.152:0/26076501 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cdc007c80 con 0x7f3cd00039b0 2024-04-20T11:28:19.084 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.083+0000 7f3ce7e00640 1 -- 172.21.15.152:0/26076501 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3cdc030070 con 0x7f3cd00039b0 2024-04-20T11:28:19.084 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.083+0000 7f3ce7e00640 1 -- 172.21.15.152:0/26076501 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cdc005d30 con 0x7f3cd00039b0 2024-04-20T11:28:19.085 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.084+0000 7f3cef000640 1 -- 172.21.15.152:0/26076501 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3cd00039b0 msgr2=0x7f3cd0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:19.085 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.084+0000 7f3cef000640 1 --2- 172.21.15.152:0/26076501 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3cd00039b0 0x7f3cd0003dd0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc00c1c0 tx=0x7f3cd0005830 comp rx=0 tx=0).stop 2024-04-20T11:28:19.085 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.084+0000 7f3cee600640 1 -- 172.21.15.152:0/26076501 reap_dead start 2024-04-20T11:28:19.085 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.084+0000 7f3cef000640 1 -- 172.21.15.152:0/26076501 shutdown_connections 2024-04-20T11:28:19.085 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.084+0000 7f3cef000640 1 -- 172.21.15.152:0/26076501 >> 172.21.15.152:0/26076501 conn(0x7f3ce808bfd0 msgr2=0x7f3ce80d4560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:19.085 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.084+0000 7f3cef000640 1 -- 172.21.15.152:0/26076501 shutdown_connections 2024-04-20T11:28:19.085 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.084+0000 7f3cef000640 1 -- 172.21.15.152:0/26076501 wait complete. 2024-04-20T11:28:19.085 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.086+0000 7f3cef000640 1 Processor -- start 2024-04-20T11:28:19.085 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.086+0000 7f3cef000640 1 -- start start 2024-04-20T11:28:19.087 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.087+0000 7f3cef000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3cd00039b0 0x7f3ce809e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:19.087 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.087+0000 7f3cef000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ce815d590 0x7f3ce80b7c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:19.087 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.087+0000 7f3cef000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ce80b81a0 0x7f3ce81e8140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:19.087 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.087+0000 7f3cef000640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f3ce80d3680 con 0x7f3cd00039b0 2024-04-20T11:28:19.087 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.087+0000 7f3cef000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3ce80d33a0 con 0x7f3ce80b81a0 2024-04-20T11:28:19.087 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.087+0000 7f3cef000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f3ce815eaa0 con 0x7f3ce815d590 2024-04-20T11:28:19.087 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.087+0000 7f3cedc00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3cd00039b0 0x7f3ce809e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:19.087 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.087+0000 7f3cedc00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3cd00039b0 0x7f3ce809e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:41136/0 (socket says 172.21.15.152:41136) 2024-04-20T11:28:19.087 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.087+0000 7f3cedc00640 1 -- 172.21.15.152:0/3019774712 learned_addr learned my addr 172.21.15.152:0/3019774712 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:19.088 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.088+0000 7f3ced200640 1 --2- 172.21.15.152:0/3019774712 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ce815d590 0x7f3ce80b7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:19.088 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.088+0000 7f3cedc00640 1 -- 172.21.15.152:0/3019774712 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ce815d590 msgr2=0x7f3ce80b7c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:19.088 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.088+0000 7f3cedc00640 1 --2- 172.21.15.152:0/3019774712 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ce815d590 0x7f3ce80b7c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:19.089 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.088+0000 7f3cedc00640 1 -- 172.21.15.152:0/3019774712 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ce80b81a0 msgr2=0x7f3ce81e8140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:19.089 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.088+0000 7f3cedc00640 1 --2- 172.21.15.152:0/3019774712 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ce80b81a0 0x7f3ce81e8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:19.089 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.088+0000 7f3cedc00640 1 -- 172.21.15.152:0/3019774712 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ce80b8720 con 0x7f3cd00039b0 2024-04-20T11:28:19.089 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.088+0000 7f3cedc00640 1 --2- 172.21.15.152:0/3019774712 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3cd00039b0 0x7f3ce809e250 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc00c160 tx=0x7f3cdc001970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:19.089 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.088+0000 7f3ce5600640 1 -- 172.21.15.152:0/3019774712 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cdc001bc0 con 0x7f3cd00039b0 2024-04-20T11:28:19.089 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.088+0000 7f3ce5600640 1 -- 172.21.15.152:0/3019774712 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3cdc004bf0 con 0x7f3cd00039b0 2024-04-20T11:28:19.089 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.088+0000 7f3cef000640 1 -- 172.21.15.152:0/3019774712 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ce80b8a00 con 0x7f3cd00039b0 2024-04-20T11:28:19.089 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.088+0000 7f3ce5600640 1 -- 172.21.15.152:0/3019774712 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cdc02de60 con 0x7f3cd00039b0 2024-04-20T11:28:19.089 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.089+0000 7f3cef000640 1 -- 172.21.15.152:0/3019774712 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ce80b8fc0 con 0x7f3cd00039b0 2024-04-20T11:28:19.089 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.090+0000 7f3cef000640 1 -- 172.21.15.152:0/3019774712 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ce8075e60 con 0x7f3cd00039b0 2024-04-20T11:28:19.090 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.091+0000 7f3ce5600640 1 -- 172.21.15.152:0/3019774712 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3cdc007070 con 0x7f3cd00039b0 2024-04-20T11:28:19.091 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.092+0000 7f3ce5600640 1 --2- 172.21.15.152:0/3019774712 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f3cc806c8f0 0x7f3cc806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:19.091 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.092+0000 7f3ce5600640 1 -- 172.21.15.152:0/3019774712 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f3cdc0ad310 con 0x7f3cd00039b0 2024-04-20T11:28:19.091 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.092+0000 7f3ced200640 1 --2- 172.21.15.152:0/3019774712 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f3cc806c8f0 0x7f3cc806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:19.092 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.093+0000 7f3ced200640 1 --2- 172.21.15.152:0/3019774712 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f3cc806c8f0 0x7f3cc806edb0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f3cd4009890 tx=0x7f3cd4008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:19.096 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.097+0000 7f3ce5600640 1 -- 172.21.15.152:0/3019774712 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f3cdc07b6a0 con 0x7f3cd00039b0 2024-04-20T11:28:19.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.282+0000 7f3cef000640 1 -- 172.21.15.152:0/3019774712 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3ce80ba4e0 con 0x7f3cd00039b0 2024-04-20T11:28:19.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.284+0000 7f3ce5600640 1 -- 172.21.15.152:0/3019774712 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) ==== 74+0+16742 (secure 0 0 0) 0x7f3cdc07afa0 con 0x7f3cd00039b0 2024-04-20T11:28:19.284 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:28:19.284 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":33,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:28:17.902771+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:28:15.704670+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-20T11:28:16.891101+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:28:19.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.286+0000 7f3cef000640 1 -- 172.21.15.152:0/3019774712 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f3cc806c8f0 msgr2=0x7f3cc806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:19.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.286+0000 7f3cef000640 1 --2- 172.21.15.152:0/3019774712 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f3cc806c8f0 0x7f3cc806edb0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f3cd4009890 tx=0x7f3cd4008040 comp rx=0 tx=0).stop 2024-04-20T11:28:19.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.286+0000 7f3cef000640 1 -- 172.21.15.152:0/3019774712 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3cd00039b0 msgr2=0x7f3ce809e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:19.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.286+0000 7f3cef000640 1 --2- 172.21.15.152:0/3019774712 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3cd00039b0 0x7f3ce809e250 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc00c160 tx=0x7f3cdc001970 comp rx=0 tx=0).stop 2024-04-20T11:28:19.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.287+0000 7f3cef000640 1 -- 172.21.15.152:0/3019774712 shutdown_connections 2024-04-20T11:28:19.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.287+0000 7f3cef000640 1 --2- 172.21.15.152:0/3019774712 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f3cc806c8f0 0x7f3cc806edb0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:19.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.287+0000 7f3cef000640 1 --2- 172.21.15.152:0/3019774712 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ce80b81a0 0x7f3ce81e8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:19.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.287+0000 7f3cef000640 1 --2- 172.21.15.152:0/3019774712 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ce815d590 0x7f3ce80b7c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:19.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.287+0000 7f3cef000640 1 --2- 172.21.15.152:0/3019774712 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3cd00039b0 0x7f3ce809e250 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:19.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.287+0000 7f3cef000640 1 -- 172.21.15.152:0/3019774712 >> 172.21.15.152:0/3019774712 conn(0x7f3ce808bfd0 msgr2=0x7f3ce80d4350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:19.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.287+0000 7f3cef000640 1 -- 172.21.15.152:0/3019774712 shutdown_connections 2024-04-20T11:28:19.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.287+0000 7f3cef000640 1 -- 172.21.15.152:0/3019774712 wait complete. 2024-04-20T11:28:19.305 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-20T11:28:19.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.506+0000 7f761b600640 1 Processor -- start 2024-04-20T11:28:19.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.506+0000 7f761b600640 1 -- start start 2024-04-20T11:28:19.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.508+0000 7f761b600640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f761409acb0 0x7f76140d30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:19.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.508+0000 7f761b600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f761409b090 0x7f761415ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:19.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.508+0000 7f761b600640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76140d0ff0 con 0x7f761409b890 2024-04-20T11:28:19.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.508+0000 7f761b600640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f761406dc00 con 0x7f761409acb0 2024-04-20T11:28:19.508 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.508+0000 7f761b600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f76140d1290 con 0x7f761409b090 2024-04-20T11:28:19.508 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.508+0000 7f7619600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f761409b090 0x7f761415ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:19.508 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.509+0000 7f7618c00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f761409acb0 0x7f76140d30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:19.508 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.509+0000 7f7619600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f761409b090 0x7f761415ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.152:59678/0 (socket says 172.21.15.152:59678) 2024-04-20T11:28:19.508 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.509+0000 7f7619600640 1 -- 172.21.15.152:0/3070104401 learned_addr learned my addr 172.21.15.152:0/3070104401 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:19.509 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.509+0000 7f7619600640 1 -- 172.21.15.152:0/3070104401 >> v2:172.21.15.175:3300/0 conn(0x7f761409acb0 msgr2=0x7f76140d30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:19.509 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.509+0000 7f7619600640 1 --2- 172.21.15.152:0/3070104401 >> v2:172.21.15.175:3300/0 conn(0x7f761409acb0 0x7f76140d30d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:19.509 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.510+0000 7f7619600640 1 -- 172.21.15.152:0/3070104401 >> v1:172.21.15.175:6789/0 conn(0x7f761409b890 legacy=0x7f761415e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:19.509 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.510+0000 7f7619600640 1 -- 172.21.15.152:0/3070104401 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f761406dea0 con 0x7f761409b090 2024-04-20T11:28:19.509 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.510+0000 7f7618c00640 1 --2- 172.21.15.152:0/3070104401 >> v2:172.21.15.175:3300/0 conn(0x7f761409acb0 0x7f76140d30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:28:19.510 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.511+0000 7f7619600640 1 --2- 172.21.15.152:0/3070104401 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f761409b090 0x7f761415ed10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f760c009e50 tx=0x7f760c02bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1083f2404b1ef1e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:19.511 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.511+0000 7f7613400640 1 -- 172.21.15.152:0/3070104401 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f760c02ca70 con 0x7f761409b090 2024-04-20T11:28:19.511 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.511+0000 7f7613400640 1 -- 172.21.15.152:0/3070104401 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f760c02cc10 con 0x7f761409b090 2024-04-20T11:28:19.511 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.511+0000 7f7613400640 1 -- 172.21.15.152:0/3070104401 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f760c0343f0 con 0x7f761409b090 2024-04-20T11:28:19.512 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.512+0000 7f761b600640 1 -- 172.21.15.152:0/3070104401 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f761409b090 msgr2=0x7f761415ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:19.512 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.512+0000 7f761b600640 1 --2- 172.21.15.152:0/3070104401 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f761409b090 0x7f761415ed10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f760c009e50 tx=0x7f760c02bed0 comp rx=0 tx=0).stop 2024-04-20T11:28:19.512 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.512+0000 7f761b600640 1 -- 172.21.15.152:0/3070104401 shutdown_connections 2024-04-20T11:28:19.512 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.512+0000 7f761b600640 1 --2- 172.21.15.152:0/3070104401 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f761409b090 0x7f761415ed10 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:19.512 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.512+0000 7f761b600640 1 --2- 172.21.15.152:0/3070104401 >> v2:172.21.15.175:3300/0 conn(0x7f761409acb0 0x7f76140d30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:19.512 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.512+0000 7f761b600640 1 -- 172.21.15.152:0/3070104401 >> 172.21.15.152:0/3070104401 conn(0x7f76140d25b0 msgr2=0x7f76140d29c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:19.512 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.512+0000 7f761b600640 1 -- 172.21.15.152:0/3070104401 shutdown_connections 2024-04-20T11:28:19.512 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.512+0000 7f761b600640 1 -- 172.21.15.152:0/3070104401 wait complete. 2024-04-20T11:28:19.514 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.514+0000 7f761b600640 1 Processor -- start 2024-04-20T11:28:19.514 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.515+0000 7f761b600640 1 -- start start 2024-04-20T11:28:19.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.516+0000 7f761b600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f761409acb0 0x7f7614148890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:19.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.516+0000 7f761b600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f761409b090 0x7f761415a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:19.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.516+0000 7f761b600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f761409b890 0x7f76141f8200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:19.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.516+0000 7f761b600640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f761406e540 con 0x7f761409acb0 2024-04-20T11:28:19.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.516+0000 7f761b600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f761406e3c0 con 0x7f761409b890 2024-04-20T11:28:19.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.516+0000 7f761b600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f761406e6c0 con 0x7f761409b090 2024-04-20T11:28:19.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.516+0000 7f7618c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f761409acb0 0x7f7614148890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:19.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.516+0000 7f7618c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f761409acb0 0x7f7614148890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:41144/0 (socket says 172.21.15.152:41144) 2024-04-20T11:28:19.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.516+0000 7f7618c00640 1 -- 172.21.15.152:0/1194779500 learned_addr learned my addr 172.21.15.152:0/1194779500 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:19.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.516+0000 7f7618c00640 1 -- 172.21.15.152:0/1194779500 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f761409b090 msgr2=0x7f761415a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:19.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.516+0000 7f7618c00640 1 --2- 172.21.15.152:0/1194779500 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f761409b090 0x7f761415a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:19.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.516+0000 7f7618c00640 1 -- 172.21.15.152:0/1194779500 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f761409b890 msgr2=0x7f76141f8200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:19.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.516+0000 7f7618c00640 1 --2- 172.21.15.152:0/1194779500 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f761409b890 0x7f76141f8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:19.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.516+0000 7f7618c00640 1 -- 172.21.15.152:0/1194779500 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7614149370 con 0x7f761409acb0 2024-04-20T11:28:19.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.517+0000 7f7618c00640 1 --2- 172.21.15.152:0/1194779500 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f761409acb0 0x7f7614148890 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f760400e960 tx=0x7f760400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:19.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.517+0000 7f7610c00640 1 -- 172.21.15.152:0/1194779500 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7604017070 con 0x7f761409acb0 2024-04-20T11:28:19.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.517+0000 7f7610c00640 1 -- 172.21.15.152:0/1194779500 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f760400c980 con 0x7f761409acb0 2024-04-20T11:28:19.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.517+0000 7f761b600640 1 -- 172.21.15.152:0/1194779500 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7614149650 con 0x7f761409acb0 2024-04-20T11:28:19.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.517+0000 7f7610c00640 1 -- 172.21.15.152:0/1194779500 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f760400cd50 con 0x7f761409acb0 2024-04-20T11:28:19.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.517+0000 7f761b600640 1 -- 172.21.15.152:0/1194779500 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7614149c10 con 0x7f761409acb0 2024-04-20T11:28:19.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.518+0000 7f761b600640 1 -- 172.21.15.152:0/1194779500 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76140689f0 con 0x7f761409acb0 2024-04-20T11:28:19.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.519+0000 7f7610c00640 1 -- 172.21.15.152:0/1194779500 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f760401a070 con 0x7f761409acb0 2024-04-20T11:28:19.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.520+0000 7f7610c00640 1 --2- 172.21.15.152:0/1194779500 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f75f406c8f0 0x7f75f406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:19.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.520+0000 7f7610c00640 1 -- 172.21.15.152:0/1194779500 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f760408bd40 con 0x7f761409acb0 2024-04-20T11:28:19.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.520+0000 7f7613e00640 1 --2- 172.21.15.152:0/1194779500 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f75f406c8f0 0x7f75f406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:19.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.521+0000 7f7613e00640 1 --2- 172.21.15.152:0/1194779500 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f75f406c8f0 0x7f75f406edb0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7600000f60 tx=0x7f76000079d0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:19.524 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.525+0000 7f7610c00640 1 -- 172.21.15.152:0/1194779500 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f760405a160 con 0x7f761409acb0 2024-04-20T11:28:19.765 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:19.765+0000 7f761b600640 1 -- 172.21.15.152:0/1194779500 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f761409bde0 con 0x7f761409acb0 2024-04-20T11:28:20.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:20.496+0000 7f7610c00640 1 -- 172.21.15.152:0/1194779500 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v14) ==== 118+0+0 (secure 0 0 0) 0x7f7604059a60 con 0x7f761409acb0 2024-04-20T11:28:20.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:20.500+0000 7f761b600640 1 -- 172.21.15.152:0/1194779500 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f75f406c8f0 msgr2=0x7f75f406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:20.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:20.500+0000 7f761b600640 1 --2- 172.21.15.152:0/1194779500 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f75f406c8f0 0x7f75f406edb0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7600000f60 tx=0x7f76000079d0 comp rx=0 tx=0).stop 2024-04-20T11:28:20.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:20.500+0000 7f761b600640 1 -- 172.21.15.152:0/1194779500 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f761409acb0 msgr2=0x7f7614148890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:20.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:20.500+0000 7f761b600640 1 --2- 172.21.15.152:0/1194779500 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f761409acb0 0x7f7614148890 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f760400e960 tx=0x7f760400ee20 comp rx=0 tx=0).stop 2024-04-20T11:28:20.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:20.500+0000 7f761b600640 1 -- 172.21.15.152:0/1194779500 shutdown_connections 2024-04-20T11:28:20.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:20.500+0000 7f761b600640 1 --2- 172.21.15.152:0/1194779500 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f75f406c8f0 0x7f75f406edb0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:20.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:20.500+0000 7f761b600640 1 --2- 172.21.15.152:0/1194779500 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f761409b890 0x7f76141f8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:20.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:20.500+0000 7f761b600640 1 --2- 172.21.15.152:0/1194779500 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f761409b090 0x7f761415a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:20.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:20.500+0000 7f761b600640 1 --2- 172.21.15.152:0/1194779500 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f761409acb0 0x7f7614148890 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:20.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:20.500+0000 7f761b600640 1 -- 172.21.15.152:0/1194779500 >> 172.21.15.152:0/1194779500 conn(0x7f76140d25b0 msgr2=0x7f76140b64a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:20.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:20.501+0000 7f761b600640 1 -- 172.21.15.152:0/1194779500 shutdown_connections 2024-04-20T11:28:20.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:20.501+0000 7f761b600640 1 -- 172.21.15.152:0/1194779500 wait complete. 2024-04-20T11:28:20.525 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-20T11:28:20.525 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-20T11:28:20.735 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:20.735+0000 7f7cd3200640 1 Processor -- start 2024-04-20T11:28:20.735 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:20.736+0000 7f7cd3200640 1 -- start start 2024-04-20T11:28:20.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:20.737+0000 7f7cd3200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f7cd415eb50 0x7f7cd40d0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:20.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:20.737+0000 7f7cd3200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7cd4163230 0x7f7cd40d2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:20.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:20.737+0000 7f7cd3200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7cd40d19c0 con 0x7f7cd40d1440 2024-04-20T11:28:20.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:20.737+0000 7f7cd3200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f7cd4162750 con 0x7f7cd415eb50 2024-04-20T11:28:20.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:20.737+0000 7f7cd3200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7cd4162a30 con 0x7f7cd4163230 2024-04-20T11:28:20.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:20.737+0000 7f7cd1e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f7cd415eb50 0x7f7cd40d0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:20.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:20.737+0000 7f7cd1e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f7cd415eb50 0x7f7cd40d0f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:41148/0 (socket says 172.21.15.152:41148) 2024-04-20T11:28:20.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:20.737+0000 7f7cd1e00640 1 -- 172.21.15.152:0/688107709 learned_addr learned my addr 172.21.15.152:0/688107709 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:20.738 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:20.738+0000 7f7cd1400640 1 --2- 172.21.15.152:0/688107709 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7cd4163230 0x7f7cd40d2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:20.738 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:20.738+0000 7f7cd1e00640 1 -- 172.21.15.152:0/688107709 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7cd4163230 msgr2=0x7f7cd40d2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:20.738 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:20.738+0000 7f7cd1e00640 1 --2- 172.21.15.152:0/688107709 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7cd4163230 0x7f7cd40d2e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:20.738 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:20.738+0000 7f7cd1e00640 1 -- 172.21.15.152:0/688107709 >> v1:172.21.15.175:6789/0 conn(0x7f7cd40d1440 legacy=0x7f7cd415ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:20.738 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:20.738+0000 7f7cd1e00640 1 -- 172.21.15.152:0/688107709 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cd4162d10 con 0x7f7cd415eb50 2024-04-20T11:28:20.738 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:20.738+0000 7f7cd1400640 1 --2- 172.21.15.152:0/688107709 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7cd4163230 0x7f7cd40d2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:28:20.739 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:20.739+0000 7f7cd1e00640 1 --2- 172.21.15.152:0/688107709 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7cd415eb50 0x7f7cd40d0f00 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f7cc4009870 tx=0x7f7cc402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26d6b209172abcbd server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:20.739 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:20.739+0000 7f7cd0a00640 1 -- 172.21.15.152:0/688107709 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cc40317c0 con 0x7f7cd415eb50 2024-04-20T11:28:20.739 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:20.739+0000 7f7cd0a00640 1 -- 172.21.15.152:0/688107709 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cc4037070 con 0x7f7cd415eb50 2024-04-20T11:28:20.739 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:20.739+0000 7f7cd0a00640 1 -- 172.21.15.152:0/688107709 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cc4031b80 con 0x7f7cd415eb50 2024-04-20T11:28:20.740 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:20.740+0000 7f7cd3200640 1 -- 172.21.15.152:0/688107709 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7cd415eb50 msgr2=0x7f7cd40d0f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:20.740 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:20.740+0000 7f7cd3200640 1 --2- 172.21.15.152:0/688107709 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7cd415eb50 0x7f7cd40d0f00 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f7cc4009870 tx=0x7f7cc402b860 comp rx=0 tx=0).stop 2024-04-20T11:28:20.740 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:20.740+0000 7f7cd3200640 1 -- 172.21.15.152:0/688107709 shutdown_connections 2024-04-20T11:28:20.740 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:20.740+0000 7f7cd3200640 1 --2- 172.21.15.152:0/688107709 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7cd4163230 0x7f7cd40d2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:20.740 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:20.740+0000 7f7cd3200640 1 --2- 172.21.15.152:0/688107709 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7cd415eb50 0x7f7cd40d0f00 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:20.740 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:20.740+0000 7f7cd3200640 1 -- 172.21.15.152:0/688107709 >> 172.21.15.152:0/688107709 conn(0x7f7cd408bfd0 msgr2=0x7f7cd40d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:20.740 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:20.740+0000 7f7cd3200640 1 -- 172.21.15.152:0/688107709 shutdown_connections 2024-04-20T11:28:20.741 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:20.740+0000 7f7cd3200640 1 -- 172.21.15.152:0/688107709 wait complete. 2024-04-20T11:28:20.742 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:20.742+0000 7f7cd3200640 1 Processor -- start 2024-04-20T11:28:20.742 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:20.743+0000 7f7cd3200640 1 -- start start 2024-04-20T11:28:20.745 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:20.744+0000 7f7cd3200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7cd40d1440 0x7f7cd40b6a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:20.745 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:20.744+0000 7f7cd3200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7cd415eb50 0x7f7cd40b7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:20.745 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:20.744+0000 7f7cd3200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7cd4163230 0x7f7cd40b9740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:20.745 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:20.744+0000 7f7cd3200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f7cd40d3550 con 0x7f7cd415eb50 2024-04-20T11:28:20.745 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:20.744+0000 7f7cd3200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7cd40d33d0 con 0x7f7cd4163230 2024-04-20T11:28:20.745 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:20.744+0000 7f7cd3200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7cd40d36d0 con 0x7f7cd40d1440 2024-04-20T11:28:20.745 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:20.744+0000 7f7cd1400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7cd415eb50 0x7f7cd40b7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:20.745 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:20.744+0000 7f7cd1400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7cd415eb50 0x7f7cd40b7ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:41152/0 (socket says 172.21.15.152:41152) 2024-04-20T11:28:20.745 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:20.744+0000 7f7cd1400640 1 -- 172.21.15.152:0/1353342560 learned_addr learned my addr 172.21.15.152:0/1353342560 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:20.745 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:20.744+0000 7f7cd1e00640 1 --2- 172.21.15.152:0/1353342560 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7cd40d1440 0x7f7cd40b6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:20.746 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:20.744+0000 7f7cd2800640 1 --2- 172.21.15.152:0/1353342560 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7cd4163230 0x7f7cd40b9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:20.746 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:20.744+0000 7f7cd1400640 1 -- 172.21.15.152:0/1353342560 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7cd40d1440 msgr2=0x7f7cd40b6a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:20.746 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:20.744+0000 7f7cd1400640 1 --2- 172.21.15.152:0/1353342560 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7cd40d1440 0x7f7cd40b6a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:20.746 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:20.744+0000 7f7cd1400640 1 -- 172.21.15.152:0/1353342560 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7cd4163230 msgr2=0x7f7cd40b9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:20.746 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:20.745+0000 7f7cd1400640 1 --2- 172.21.15.152:0/1353342560 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7cd4163230 0x7f7cd40b9740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:20.746 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:20.745+0000 7f7cd1400640 1 -- 172.21.15.152:0/1353342560 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cd40cd4a0 con 0x7f7cd415eb50 2024-04-20T11:28:20.746 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:20.745+0000 7f7cd1e00640 1 --2- 172.21.15.152:0/1353342560 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7cd40d1440 0x7f7cd40b6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:28:20.746 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:20.745+0000 7f7cd2800640 1 --2- 172.21.15.152:0/1353342560 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7cd4163230 0x7f7cd40b9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:28:20.747 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:20.745+0000 7f7cd1400640 1 --2- 172.21.15.152:0/1353342560 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7cd415eb50 0x7f7cd40b7ad0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc00a390 tx=0x7f7cbc00a850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:20.747 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:20.745+0000 7f7cca000640 1 -- 172.21.15.152:0/1353342560 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cbc019070 con 0x7f7cd415eb50 2024-04-20T11:28:20.747 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:20.745+0000 7f7cca000640 1 -- 172.21.15.152:0/1353342560 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cbc008580 con 0x7f7cd415eb50 2024-04-20T11:28:20.747 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:20.745+0000 7f7cca000640 1 -- 172.21.15.152:0/1353342560 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cbc008950 con 0x7f7cd415eb50 2024-04-20T11:28:20.747 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:20.745+0000 7f7cd3200640 1 -- 172.21.15.152:0/1353342560 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cd40cd780 con 0x7f7cd415eb50 2024-04-20T11:28:20.747 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:20.745+0000 7f7cd3200640 1 -- 172.21.15.152:0/1353342560 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cd40cdd40 con 0x7f7cd415eb50 2024-04-20T11:28:20.747 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:20.747+0000 7f7cd3200640 1 -- 172.21.15.152:0/1353342560 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cd4075e60 con 0x7f7cd415eb50 2024-04-20T11:28:20.747 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:20.748+0000 7f7cca000640 1 -- 172.21.15.152:0/1353342560 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7cbc01c070 con 0x7f7cd415eb50 2024-04-20T11:28:20.748 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:20.748+0000 7f7cca000640 1 --2- 172.21.15.152:0/1353342560 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f7cb006c8f0 0x7f7cb006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:20.748 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:20.749+0000 7f7cca000640 1 -- 172.21.15.152:0/1353342560 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f7cbc08be30 con 0x7f7cd415eb50 2024-04-20T11:28:20.748 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:20.749+0000 7f7cd1e00640 1 --2- 172.21.15.152:0/1353342560 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f7cb006c8f0 0x7f7cb006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:20.749 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:20.750+0000 7f7cd1e00640 1 --2- 172.21.15.152:0/1353342560 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f7cb006c8f0 0x7f7cb006edb0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7cc4036000 tx=0x7f7cc4008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:20.753 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:20.753+0000 7f7cca000640 1 -- 172.21.15.152:0/1353342560 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f7cbc059bf0 con 0x7f7cd415eb50 2024-04-20T11:28:20.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:20.989+0000 7f7cd3200640 1 -- 172.21.15.152:0/1353342560 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f7cd40d3de0 con 0x7f7cd415eb50 2024-04-20T11:28:21.319 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:28:21.327 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:28:21.394 INFO:tasks.ceph.mds.b.smithi175.stderr:2024-04-20T11:28:21.394+0000 7fc820000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-20T11:28:21.514 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:21.514+0000 7f7cca000640 1 -- 172.21.15.152:0/1353342560 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v15) ==== 122+0+0 (secure 0 0 0) 0x7f7cbc05def0 con 0x7f7cd415eb50 2024-04-20T11:28:21.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:21.518+0000 7f7cd3200640 1 -- 172.21.15.152:0/1353342560 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f7cb006c8f0 msgr2=0x7f7cb006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:21.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:21.518+0000 7f7cd3200640 1 --2- 172.21.15.152:0/1353342560 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f7cb006c8f0 0x7f7cb006edb0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7cc4036000 tx=0x7f7cc4008030 comp rx=0 tx=0).stop 2024-04-20T11:28:21.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:21.518+0000 7f7cd3200640 1 -- 172.21.15.152:0/1353342560 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7cd415eb50 msgr2=0x7f7cd40b7ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:21.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:21.518+0000 7f7cd3200640 1 --2- 172.21.15.152:0/1353342560 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7cd415eb50 0x7f7cd40b7ad0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc00a390 tx=0x7f7cbc00a850 comp rx=0 tx=0).stop 2024-04-20T11:28:21.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:21.518+0000 7f7cd3200640 1 -- 172.21.15.152:0/1353342560 shutdown_connections 2024-04-20T11:28:21.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:21.518+0000 7f7cd3200640 1 --2- 172.21.15.152:0/1353342560 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f7cb006c8f0 0x7f7cb006edb0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:21.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:21.518+0000 7f7cd3200640 1 --2- 172.21.15.152:0/1353342560 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7cd4163230 0x7f7cd40b9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:21.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:21.518+0000 7f7cd3200640 1 --2- 172.21.15.152:0/1353342560 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7cd415eb50 0x7f7cd40b7ad0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:21.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:21.518+0000 7f7cd3200640 1 --2- 172.21.15.152:0/1353342560 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7cd40d1440 0x7f7cd40b6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:21.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:21.518+0000 7f7cd3200640 1 -- 172.21.15.152:0/1353342560 >> 172.21.15.152:0/1353342560 conn(0x7f7cd408bfd0 msgr2=0x7f7cd40b2cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:21.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:21.518+0000 7f7cd3200640 1 -- 172.21.15.152:0/1353342560 shutdown_connections 2024-04-20T11:28:21.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:21.519+0000 7f7cd3200640 1 -- 172.21.15.152:0/1353342560 wait complete. 2024-04-20T11:28:21.541 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-20T11:28:21.541 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:28:21.745 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:21.746+0000 7facc2000640 1 Processor -- start 2024-04-20T11:28:21.746 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:21.746+0000 7facc2000640 1 -- start start 2024-04-20T11:28:21.747 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:21.747+0000 7facc2000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7facbc0b3cd0 0x7facbc071cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:21.747 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:21.748+0000 7facc2000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7facbc072210 0x7facbc0726a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:21.747 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:21.748+0000 7facc2000640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7facbc15b340 con 0x7facbc151ab0 2024-04-20T11:28:21.747 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:21.748+0000 7facc2000640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7facbc15c100 con 0x7facbc072210 2024-04-20T11:28:21.747 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:21.748+0000 7facc2000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7facbc15bc70 con 0x7facbc0b3cd0 2024-04-20T11:28:21.748 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:21.748+0000 7facc0c00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7facbc151ab0 0x7facbc0b35c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:57966/0 (socket says 172.21.15.152:57966) 2024-04-20T11:28:21.748 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:21.748+0000 7facc0c00640 1 -- 172.21.15.152:0/2622681756 learned_addr learned my addr 172.21.15.152:0/2622681756 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:28:21.748 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:21.748+0000 7facc1600640 1 --2- 172.21.15.152:0/2622681756 >> v2:172.21.15.152:3300/0 conn(0x7facbc072210 0x7facbc0726a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:21.748 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:21.748+0000 7facbbe00640 1 --2- 172.21.15.152:0/2622681756 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7facbc0b3cd0 0x7facbc071cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:21.748 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:21.748+0000 7facc1600640 1 -- 172.21.15.152:0/2622681756 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7facbc0b3cd0 msgr2=0x7facbc071cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:21.748 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:21.748+0000 7facc1600640 1 --2- 172.21.15.152:0/2622681756 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7facbc0b3cd0 0x7facbc071cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:21.748 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:21.748+0000 7facc1600640 1 -- 172.21.15.152:0/2622681756 >> v1:172.21.15.152:6789/0 conn(0x7facbc151ab0 legacy=0x7facbc0b35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:21.749 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:21.749+0000 7facc1600640 1 -- 172.21.15.152:0/2622681756 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facbc15b620 con 0x7facbc072210 2024-04-20T11:28:21.749 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:21.748+0000 7facbb400640 1 -- 172.21.15.152:0/2622681756 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3375269708 0 0) 0x7facbc15b340 con 0x7facbc151ab0 2024-04-20T11:28:21.749 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:21.750+0000 7facc1600640 1 --2- 172.21.15.152:0/2622681756 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7facbc072210 0x7facbc0726a0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7facb4009d90 tx=0x7facb402bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37f20f4d09cd8ff1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:21.749 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:21.750+0000 7facbb400640 1 -- 172.21.15.152:0/2622681756 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb4039070 con 0x7facbc072210 2024-04-20T11:28:21.749 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:21.750+0000 7facbb400640 1 -- 172.21.15.152:0/2622681756 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facb402cc50 con 0x7facbc072210 2024-04-20T11:28:21.749 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:21.750+0000 7facbb400640 1 -- 172.21.15.152:0/2622681756 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb4034440 con 0x7facbc072210 2024-04-20T11:28:21.750 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:21.750+0000 7facc2000640 1 -- 172.21.15.152:0/2622681756 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7facbc072210 msgr2=0x7facbc0726a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:21.750 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:21.750+0000 7facc2000640 1 --2- 172.21.15.152:0/2622681756 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7facbc072210 0x7facbc0726a0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7facb4009d90 tx=0x7facb402bea0 comp rx=0 tx=0).stop 2024-04-20T11:28:21.751 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:21.751+0000 7facc2000640 1 -- 172.21.15.152:0/2622681756 shutdown_connections 2024-04-20T11:28:21.751 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:21.751+0000 7facc2000640 1 --2- 172.21.15.152:0/2622681756 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7facbc0b3cd0 0x7facbc071cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:21.751 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:21.751+0000 7facc2000640 1 --2- 172.21.15.152:0/2622681756 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7facbc072210 0x7facbc0726a0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:21.751 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:21.751+0000 7facc2000640 1 -- 172.21.15.152:0/2622681756 >> 172.21.15.152:0/2622681756 conn(0x7facbc067ae0 msgr2=0x7facbc06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:21.751 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:21.751+0000 7facc2000640 1 -- 172.21.15.152:0/2622681756 shutdown_connections 2024-04-20T11:28:21.751 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:21.751+0000 7facc2000640 1 -- 172.21.15.152:0/2622681756 wait complete. 2024-04-20T11:28:21.752 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:21.752+0000 7facc2000640 1 Processor -- start 2024-04-20T11:28:21.752 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:21.752+0000 7facc2000640 1 -- start start 2024-04-20T11:28:21.753 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:21.753+0000 7facc2000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7facbc072210 0x7facbc150840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:21.753 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:21.753+0000 7facc2000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7facbc0b3cd0 0x7facbc1e5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:21.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:21.753+0000 7facc2000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7facbc151ab0 0x7facbc13aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:21.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:21.753+0000 7facc2000640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7facbc0ce930 con 0x7facbc151ab0 2024-04-20T11:28:21.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:21.753+0000 7facc2000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7facbc0ce7b0 con 0x7facbc0b3cd0 2024-04-20T11:28:21.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:21.753+0000 7facc2000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7facbc151e90 con 0x7facbc072210 2024-04-20T11:28:21.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:21.754+0000 7facc1600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7facbc151ab0 0x7facbc13aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:21.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:21.754+0000 7facc1600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7facbc151ab0 0x7facbc13aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:41174/0 (socket says 172.21.15.152:41174) 2024-04-20T11:28:21.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:21.754+0000 7facc1600640 1 -- 172.21.15.152:0/4274779998 learned_addr learned my addr 172.21.15.152:0/4274779998 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:21.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:21.754+0000 7facc0c00640 1 --2- 172.21.15.152:0/4274779998 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7facbc072210 0x7facbc150840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:21.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:21.754+0000 7facc1600640 1 -- 172.21.15.152:0/4274779998 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7facbc072210 msgr2=0x7facbc150840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:21.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:21.754+0000 7facc1600640 1 --2- 172.21.15.152:0/4274779998 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7facbc072210 0x7facbc150840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:21.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:21.754+0000 7facc1600640 1 -- 172.21.15.152:0/4274779998 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7facbc0b3cd0 msgr2=0x7facbc1e5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:21.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:21.754+0000 7facc1600640 1 --2- 172.21.15.152:0/4274779998 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7facbc0b3cd0 0x7facbc1e5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:21.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:21.754+0000 7facc1600640 1 -- 172.21.15.152:0/4274779998 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facbc1e6190 con 0x7facbc151ab0 2024-04-20T11:28:21.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:21.754+0000 7facc1600640 1 --2- 172.21.15.152:0/4274779998 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7facbc151ab0 0x7facbc13aae0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7facb4009d90 tx=0x7facb4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:21.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:21.755+0000 7facb8c00640 1 -- 172.21.15.152:0/4274779998 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb4044020 con 0x7facbc151ab0 2024-04-20T11:28:21.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:21.755+0000 7facb8c00640 1 -- 172.21.15.152:0/4274779998 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facb403a040 con 0x7facbc151ab0 2024-04-20T11:28:21.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:21.755+0000 7facb8c00640 1 -- 172.21.15.152:0/4274779998 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb4039050 con 0x7facbc151ab0 2024-04-20T11:28:21.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:21.755+0000 7facc2000640 1 -- 172.21.15.152:0/4274779998 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facbc1e6470 con 0x7facbc151ab0 2024-04-20T11:28:21.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:21.755+0000 7facc2000640 1 -- 172.21.15.152:0/4274779998 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facbc1e6a30 con 0x7facbc151ab0 2024-04-20T11:28:21.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:21.756+0000 7facc2000640 1 -- 172.21.15.152:0/4274779998 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facbc075e60 con 0x7facbc151ab0 2024-04-20T11:28:21.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:21.757+0000 7facb8c00640 1 -- 172.21.15.152:0/4274779998 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7facb4003990 con 0x7facbc151ab0 2024-04-20T11:28:21.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:21.758+0000 7facb8c00640 1 --2- 172.21.15.152:0/4274779998 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fac9c06c920 0x7fac9c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:21.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:21.758+0000 7facb8c00640 1 -- 172.21.15.152:0/4274779998 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7facb40adeb0 con 0x7facbc151ab0 2024-04-20T11:28:21.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:21.758+0000 7facc0c00640 1 --2- 172.21.15.152:0/4274779998 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fac9c06c920 0x7fac9c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:21.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:21.759+0000 7facc0c00640 1 --2- 172.21.15.152:0/4274779998 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fac9c06c920 0x7fac9c06ede0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7facac000f60 tx=0x7facac007c00 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:21.762 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:21.762+0000 7facb8c00640 1 -- 172.21.15.152:0/4274779998 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7facb404e060 con 0x7facbc151ab0 2024-04-20T11:28:22.005 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.005+0000 7facc2000640 1 -- 172.21.15.152:0/4274779998 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7facbc06f4c0 con 0x7facbc151ab0 2024-04-20T11:28:22.006 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.006+0000 7facb8c00640 1 -- 172.21.15.152:0/4274779998 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) ==== 94+0+5211 (secure 0 0 0) 0x7facb4082070 con 0x7facbc151ab0 2024-04-20T11:28:22.006 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:28:22.006 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4643,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6834/1871483322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1871483322},{"type":"v1","addr":"172.21.15.175:6834","nonce":1871483322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4786,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/2579061164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6835","nonce":2579061164},{"type":"v1","addr":"172.21.15.152:6837","nonce":2579061164}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5208,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6836/3724129680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":3724129680},{"type":"v1","addr":"172.21.15.152:6836","nonce":3724129680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T11:28:17.902738+0000","modified":"2024-04-20T11:28:20.990798+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5223},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5223":{"gid":5223,"name":"b","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.175:6835/3367117968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6833","nonce":3367117968},{"type":"v1","addr":"172.21.15.175:6835","nonce":3367117968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5223,"qdb_cluster":[5223]},"id":2}]} 2024-04-20T11:28:22.006 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 15 2024-04-20T11:28:22.008 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.008+0000 7facc2000640 1 -- 172.21.15.152:0/4274779998 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fac9c06c920 msgr2=0x7fac9c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:22.008 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.008+0000 7facc2000640 1 --2- 172.21.15.152:0/4274779998 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fac9c06c920 0x7fac9c06ede0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7facac000f60 tx=0x7facac007c00 comp rx=0 tx=0).stop 2024-04-20T11:28:22.008 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.008+0000 7facc2000640 1 -- 172.21.15.152:0/4274779998 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7facbc151ab0 msgr2=0x7facbc13aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:22.008 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.008+0000 7facc2000640 1 --2- 172.21.15.152:0/4274779998 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7facbc151ab0 0x7facbc13aae0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7facb4009d90 tx=0x7facb4004a40 comp rx=0 tx=0).stop 2024-04-20T11:28:22.009 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.009+0000 7facc2000640 1 -- 172.21.15.152:0/4274779998 shutdown_connections 2024-04-20T11:28:22.009 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.009+0000 7facc2000640 1 --2- 172.21.15.152:0/4274779998 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fac9c06c920 0x7fac9c06ede0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:22.009 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.009+0000 7facc2000640 1 --2- 172.21.15.152:0/4274779998 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7facbc151ab0 0x7facbc13aae0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:22.009 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.009+0000 7facc2000640 1 --2- 172.21.15.152:0/4274779998 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7facbc0b3cd0 0x7facbc1e5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:22.009 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.009+0000 7facc2000640 1 --2- 172.21.15.152:0/4274779998 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7facbc072210 0x7facbc150840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:22.009 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.009+0000 7facc2000640 1 -- 172.21.15.152:0/4274779998 >> 172.21.15.152:0/4274779998 conn(0x7facbc067ae0 msgr2=0x7facbc0b3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:22.009 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.009+0000 7facc2000640 1 -- 172.21.15.152:0/4274779998 shutdown_connections 2024-04-20T11:28:22.009 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.009+0000 7facc2000640 1 -- 172.21.15.152:0/4274779998 wait complete. 2024-04-20T11:28:22.029 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:28:22.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.231+0000 7fb5cca00640 1 Processor -- start 2024-04-20T11:28:22.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.231+0000 7fb5cca00640 1 -- start start 2024-04-20T11:28:22.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.233+0000 7fb5cca00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb5c815d1e0 0x7fb5c8160b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:22.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.233+0000 7fb5cca00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fb5c8071cd0 0x7fb5c8072160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:22.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.233+0000 7fb5cca00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5c80ce8c0 con 0x7fb5c815d5c0 2024-04-20T11:28:22.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.233+0000 7fb5cca00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fb5c80726a0 con 0x7fb5c8071cd0 2024-04-20T11:28:22.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.233+0000 7fb5cca00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb5c81600b0 con 0x7fb5c815d1e0 2024-04-20T11:28:22.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.233+0000 7fb5c6800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb5c815d1e0 0x7fb5c8160b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:22.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.234+0000 7fb5c6800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb5c815d1e0 0x7fb5c8160b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.152:59724/0 (socket says 172.21.15.152:59724) 2024-04-20T11:28:22.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.234+0000 7fb5c6800640 1 -- 172.21.15.152:0/2418463913 learned_addr learned my addr 172.21.15.152:0/2418463913 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:22.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.234+0000 7fb5c7200640 1 --2- 172.21.15.152:0/2418463913 >> v2:172.21.15.175:3300/0 conn(0x7fb5c8071cd0 0x7fb5c8072160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:22.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.234+0000 7fb5c6800640 1 -- 172.21.15.152:0/2418463913 >> v2:172.21.15.175:3300/0 conn(0x7fb5c8071cd0 msgr2=0x7fb5c8072160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:22.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.234+0000 7fb5c6800640 1 --2- 172.21.15.152:0/2418463913 >> v2:172.21.15.175:3300/0 conn(0x7fb5c8071cd0 0x7fb5c8072160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:22.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.234+0000 7fb5c6800640 1 -- 172.21.15.152:0/2418463913 >> v1:172.21.15.175:6789/0 conn(0x7fb5c815d5c0 legacy=0x7fb5c80cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:22.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.234+0000 7fb5c6800640 1 -- 172.21.15.152:0/2418463913 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5c8160390 con 0x7fb5c815d1e0 2024-04-20T11:28:22.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.236+0000 7fb5c6800640 1 --2- 172.21.15.152:0/2418463913 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb5c815d1e0 0x7fb5c8160b90 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb5b8009870 tx=0x7fb5b802b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9330d21087c6949 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:22.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.236+0000 7fb5c5400640 1 -- 172.21.15.152:0/2418463913 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b8034070 con 0x7fb5c815d1e0 2024-04-20T11:28:22.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.236+0000 7fb5c5400640 1 -- 172.21.15.152:0/2418463913 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5b8030840 con 0x7fb5c815d1e0 2024-04-20T11:28:22.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.236+0000 7fb5c5400640 1 -- 172.21.15.152:0/2418463913 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b8030c10 con 0x7fb5c815d1e0 2024-04-20T11:28:22.237 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.237+0000 7fb5cca00640 1 -- 172.21.15.152:0/2418463913 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb5c815d1e0 msgr2=0x7fb5c8160b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:22.237 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.237+0000 7fb5cca00640 1 --2- 172.21.15.152:0/2418463913 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb5c815d1e0 0x7fb5c8160b90 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb5b8009870 tx=0x7fb5b802b980 comp rx=0 tx=0).stop 2024-04-20T11:28:22.237 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.237+0000 7fb5cca00640 1 -- 172.21.15.152:0/2418463913 shutdown_connections 2024-04-20T11:28:22.237 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.237+0000 7fb5cca00640 1 --2- 172.21.15.152:0/2418463913 >> v2:172.21.15.175:3300/0 conn(0x7fb5c8071cd0 0x7fb5c8072160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:22.237 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.237+0000 7fb5cca00640 1 --2- 172.21.15.152:0/2418463913 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb5c815d1e0 0x7fb5c8160b90 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:22.237 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.237+0000 7fb5cca00640 1 -- 172.21.15.152:0/2418463913 >> 172.21.15.152:0/2418463913 conn(0x7fb5c8067ae0 msgr2=0x7fb5c806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:22.237 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.237+0000 7fb5cca00640 1 -- 172.21.15.152:0/2418463913 shutdown_connections 2024-04-20T11:28:22.237 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.237+0000 7fb5cca00640 1 -- 172.21.15.152:0/2418463913 wait complete. 2024-04-20T11:28:22.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.239+0000 7fb5cca00640 1 Processor -- start 2024-04-20T11:28:22.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.239+0000 7fb5cca00640 1 -- start start 2024-04-20T11:28:22.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.240+0000 7fb5cca00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5c8071cd0 0x7fb5c8144590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:22.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.240+0000 7fb5cca00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb5c815d1e0 0x7fb5c80b3b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:22.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.240+0000 7fb5cca00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb5c815d5c0 0x7fb5c81e5e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:22.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.240+0000 7fb5cca00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fb5c80ccd70 con 0x7fb5c815d1e0 2024-04-20T11:28:22.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.240+0000 7fb5cca00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb5c80ceba0 con 0x7fb5c8071cd0 2024-04-20T11:28:22.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.240+0000 7fb5cca00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb5c80ccef0 con 0x7fb5c815d5c0 2024-04-20T11:28:22.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.240+0000 7fb5c5e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb5c815d1e0 0x7fb5c80b3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:22.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.240+0000 7fb5c5e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb5c815d1e0 0x7fb5c80b3b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:41188/0 (socket says 172.21.15.152:41188) 2024-04-20T11:28:22.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.240+0000 7fb5c5e00640 1 -- 172.21.15.152:0/1954615046 learned_addr learned my addr 172.21.15.152:0/1954615046 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:22.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.241+0000 7fb5c7200640 1 --2- 172.21.15.152:0/1954615046 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb5c815d5c0 0x7fb5c81e5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:22.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.241+0000 7fb5c6800640 1 --2- 172.21.15.152:0/1954615046 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5c8071cd0 0x7fb5c8144590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:22.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.241+0000 7fb5c5e00640 1 -- 172.21.15.152:0/1954615046 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb5c815d5c0 msgr2=0x7fb5c81e5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:22.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.241+0000 7fb5c5e00640 1 --2- 172.21.15.152:0/1954615046 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb5c815d5c0 0x7fb5c81e5e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:22.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.241+0000 7fb5c5e00640 1 -- 172.21.15.152:0/1954615046 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5c8071cd0 msgr2=0x7fb5c8144590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:22.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.241+0000 7fb5c5e00640 1 --2- 172.21.15.152:0/1954615046 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5c8071cd0 0x7fb5c8144590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:22.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.241+0000 7fb5c5e00640 1 -- 172.21.15.152:0/1954615046 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5c8144d30 con 0x7fb5c815d1e0 2024-04-20T11:28:22.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.241+0000 7fb5c7200640 1 --2- 172.21.15.152:0/1954615046 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb5c815d5c0 0x7fb5c81e5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:28:22.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.241+0000 7fb5c5e00640 1 --2- 172.21.15.152:0/1954615046 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb5c815d1e0 0x7fb5c80b3b00 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fb5b400c3c0 tx=0x7fb5b400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:22.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.242+0000 7fb5bea00640 1 -- 172.21.15.152:0/1954615046 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b40052d0 con 0x7fb5c815d1e0 2024-04-20T11:28:22.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.242+0000 7fb5bea00640 1 -- 172.21.15.152:0/1954615046 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5b4005490 con 0x7fb5c815d1e0 2024-04-20T11:28:22.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.242+0000 7fb5bea00640 1 -- 172.21.15.152:0/1954615046 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b4007260 con 0x7fb5c815d1e0 2024-04-20T11:28:22.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.242+0000 7fb5cca00640 1 -- 172.21.15.152:0/1954615046 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5c80b43d0 con 0x7fb5c815d1e0 2024-04-20T11:28:22.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.242+0000 7fb5cca00640 1 -- 172.21.15.152:0/1954615046 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5c80b4990 con 0x7fb5c815d1e0 2024-04-20T11:28:22.243 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.243+0000 7fb5cca00640 1 -- 172.21.15.152:0/1954615046 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5c80b0120 con 0x7fb5c815d1e0 2024-04-20T11:28:22.243 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.244+0000 7fb5bea00640 1 -- 172.21.15.152:0/1954615046 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb5b4007400 con 0x7fb5c815d1e0 2024-04-20T11:28:22.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.245+0000 7fb5bea00640 1 --2- 172.21.15.152:0/1954615046 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fb5a806c8f0 0x7fb5a806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:22.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.245+0000 7fb5bea00640 1 -- 172.21.15.152:0/1954615046 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fb5b408b060 con 0x7fb5c815d1e0 2024-04-20T11:28:22.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.245+0000 7fb5c6800640 1 --2- 172.21.15.152:0/1954615046 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fb5a806c8f0 0x7fb5a806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:22.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.246+0000 7fb5c6800640 1 --2- 172.21.15.152:0/1954615046 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fb5a806c8f0 0x7fb5a806edb0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb5b8036000 tx=0x7fb5b803b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:22.249 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.250+0000 7fb5bea00640 1 -- 172.21.15.152:0/1954615046 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fb5c80b0120 con 0x7fb5c815d1e0 2024-04-20T11:28:22.452 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.453+0000 7fb5cca00640 1 -- 172.21.15.152:0/1954615046 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb5c8075e60 con 0x7fb5c815d1e0 2024-04-20T11:28:22.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.456+0000 7fb5bea00640 1 -- 172.21.15.152:0/1954615046 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) ==== 74+0+16729 (secure 0 0 0) 0x7fb5b4059480 con 0x7fb5c815d1e0 2024-04-20T11:28:22.456 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:28:22.456 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":34,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:28:19.478546+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:28:15.704670+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-20T11:28:16.891101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:28:22.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.458+0000 7fb5cca00640 1 -- 172.21.15.152:0/1954615046 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fb5a806c8f0 msgr2=0x7fb5a806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:22.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.458+0000 7fb5cca00640 1 --2- 172.21.15.152:0/1954615046 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fb5a806c8f0 0x7fb5a806edb0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb5b8036000 tx=0x7fb5b803b040 comp rx=0 tx=0).stop 2024-04-20T11:28:22.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.458+0000 7fb5cca00640 1 -- 172.21.15.152:0/1954615046 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb5c815d1e0 msgr2=0x7fb5c80b3b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:22.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.458+0000 7fb5cca00640 1 --2- 172.21.15.152:0/1954615046 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb5c815d1e0 0x7fb5c80b3b00 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fb5b400c3c0 tx=0x7fb5b400c880 comp rx=0 tx=0).stop 2024-04-20T11:28:22.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.458+0000 7fb5cca00640 1 -- 172.21.15.152:0/1954615046 shutdown_connections 2024-04-20T11:28:22.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.458+0000 7fb5cca00640 1 --2- 172.21.15.152:0/1954615046 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fb5a806c8f0 0x7fb5a806edb0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:22.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.458+0000 7fb5cca00640 1 --2- 172.21.15.152:0/1954615046 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb5c815d5c0 0x7fb5c81e5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:22.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.458+0000 7fb5cca00640 1 --2- 172.21.15.152:0/1954615046 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb5c815d1e0 0x7fb5c80b3b00 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:22.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.458+0000 7fb5cca00640 1 --2- 172.21.15.152:0/1954615046 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5c8071cd0 0x7fb5c8144590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:22.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.458+0000 7fb5cca00640 1 -- 172.21.15.152:0/1954615046 >> 172.21.15.152:0/1954615046 conn(0x7fb5c8067ae0 msgr2=0x7fb5c806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:22.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.459+0000 7fb5cca00640 1 -- 172.21.15.152:0/1954615046 shutdown_connections 2024-04-20T11:28:22.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.459+0000 7fb5cca00640 1 -- 172.21.15.152:0/1954615046 wait complete. 2024-04-20T11:28:22.479 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-20T11:28:22.479 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T11:28:22.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.682+0000 7f745be00640 1 Processor -- start 2024-04-20T11:28:22.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.682+0000 7f745be00640 1 -- start start 2024-04-20T11:28:22.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.684+0000 7f745be00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f745c15d1e0 0x7f745c160b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:22.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.684+0000 7f745be00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f745c15d5c0 0x7f745c0cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:22.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.684+0000 7f745be00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f745c0ce8c0 con 0x7f745c071cd0 2024-04-20T11:28:22.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.684+0000 7f745be00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f745c1600b0 con 0x7f745c15d1e0 2024-04-20T11:28:22.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.684+0000 7f745be00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f745c160390 con 0x7f745c15d5c0 2024-04-20T11:28:22.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.684+0000 7f745aa00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f745c15d1e0 0x7f745c160b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:22.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.684+0000 7f745b400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f745c071cd0 0x7f745c072120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:57972/0 (socket says 172.21.15.152:57972) 2024-04-20T11:28:22.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.684+0000 7f745b400640 1 -- 172.21.15.152:0/1148261328 learned_addr learned my addr 172.21.15.152:0/1148261328 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:28:22.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.684+0000 7f745a000640 1 --2- 172.21.15.152:0/1148261328 >> v2:172.21.15.175:3300/0 conn(0x7f745c15d5c0 0x7f745c0cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:22.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.685+0000 7f745aa00640 1 -- 172.21.15.152:0/1148261328 >> v2:172.21.15.175:3300/0 conn(0x7f745c15d5c0 msgr2=0x7f745c0cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:22.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.685+0000 7f745aa00640 1 --2- 172.21.15.152:0/1148261328 >> v2:172.21.15.175:3300/0 conn(0x7f745c15d5c0 0x7f745c0cdfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:22.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.685+0000 7f745aa00640 1 -- 172.21.15.152:0/1148261328 >> v1:172.21.15.152:6789/0 conn(0x7f745c071cd0 legacy=0x7f745c072120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:22.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.685+0000 7f745aa00640 1 -- 172.21.15.152:0/1148261328 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f745c160670 con 0x7f745c15d1e0 2024-04-20T11:28:22.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.685+0000 7f7459600640 1 -- 172.21.15.152:0/1148261328 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1712051710 0 0) 0x7f745c0ce8c0 con 0x7f745c071cd0 2024-04-20T11:28:22.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.686+0000 7f745aa00640 1 --2- 172.21.15.152:0/1148261328 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f745c15d1e0 0x7f745c160b90 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f744c0097a0 tx=0x7f744c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b29bb65dcf579592 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:22.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.686+0000 7f7459600640 1 -- 172.21.15.152:0/1148261328 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f744c035020 con 0x7f745c15d1e0 2024-04-20T11:28:22.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.686+0000 7f7459600640 1 -- 172.21.15.152:0/1148261328 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f744c031830 con 0x7f745c15d1e0 2024-04-20T11:28:22.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.686+0000 7f7459600640 1 -- 172.21.15.152:0/1148261328 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f744c031c00 con 0x7f745c15d1e0 2024-04-20T11:28:22.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.687+0000 7f745be00640 1 -- 172.21.15.152:0/1148261328 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f745c15d1e0 msgr2=0x7f745c160b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:22.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.687+0000 7f745be00640 1 --2- 172.21.15.152:0/1148261328 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f745c15d1e0 0x7f745c160b90 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f744c0097a0 tx=0x7f744c02b8d0 comp rx=0 tx=0).stop 2024-04-20T11:28:22.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.687+0000 7f745be00640 1 -- 172.21.15.152:0/1148261328 shutdown_connections 2024-04-20T11:28:22.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.687+0000 7f745be00640 1 --2- 172.21.15.152:0/1148261328 >> v2:172.21.15.175:3300/0 conn(0x7f745c15d5c0 0x7f745c0cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:22.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.687+0000 7f745be00640 1 --2- 172.21.15.152:0/1148261328 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f745c15d1e0 0x7f745c160b90 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:22.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.687+0000 7f745be00640 1 -- 172.21.15.152:0/1148261328 >> 172.21.15.152:0/1148261328 conn(0x7f745c067ae0 msgr2=0x7f745c06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:22.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.687+0000 7f745be00640 1 -- 172.21.15.152:0/1148261328 shutdown_connections 2024-04-20T11:28:22.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.687+0000 7f745be00640 1 -- 172.21.15.152:0/1148261328 wait complete. 2024-04-20T11:28:22.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.689+0000 7f745be00640 1 Processor -- start 2024-04-20T11:28:22.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.689+0000 7f745be00640 1 -- start start 2024-04-20T11:28:22.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.690+0000 7f745be00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f745c071cd0 0x7f745c141ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:22.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.690+0000 7f745be00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f745c15d1e0 0x7f745c152150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:22.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.690+0000 7f745be00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f745c15d5c0 0x7f745c152690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:22.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.690+0000 7f745be00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f745c0ce6a0 con 0x7f745c071cd0 2024-04-20T11:28:22.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.690+0000 7f745be00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f745c0ce520 con 0x7f745c15d5c0 2024-04-20T11:28:22.691 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.690+0000 7f745be00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f745c0ccd70 con 0x7f745c15d1e0 2024-04-20T11:28:22.691 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.690+0000 7f745aa00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f745c071cd0 0x7f745c141ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:22.691 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.690+0000 7f745aa00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f745c071cd0 0x7f745c141ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:41202/0 (socket says 172.21.15.152:41202) 2024-04-20T11:28:22.691 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.690+0000 7f745aa00640 1 -- 172.21.15.152:0/325308359 learned_addr learned my addr 172.21.15.152:0/325308359 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:22.691 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.690+0000 7f745a000640 1 --2- 172.21.15.152:0/325308359 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f745c15d1e0 0x7f745c152150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:22.691 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.690+0000 7f745b400640 1 --2- 172.21.15.152:0/325308359 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f745c15d5c0 0x7f745c152690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:22.691 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.691+0000 7f745aa00640 1 -- 172.21.15.152:0/325308359 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f745c15d1e0 msgr2=0x7f745c152150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:22.691 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.691+0000 7f745aa00640 1 --2- 172.21.15.152:0/325308359 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f745c15d1e0 0x7f745c152150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:22.691 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.691+0000 7f745aa00640 1 -- 172.21.15.152:0/325308359 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f745c15d5c0 msgr2=0x7f745c152690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:22.691 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.691+0000 7f745aa00640 1 --2- 172.21.15.152:0/325308359 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f745c15d5c0 0x7f745c152690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:22.691 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.691+0000 7f745aa00640 1 -- 172.21.15.152:0/325308359 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f745c1f5480 con 0x7f745c071cd0 2024-04-20T11:28:22.691 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.691+0000 7f745a000640 1 --2- 172.21.15.152:0/325308359 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f745c15d1e0 0x7f745c152150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:28:22.691 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.691+0000 7f745b400640 1 --2- 172.21.15.152:0/325308359 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f745c15d5c0 0x7f745c152690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:28:22.692 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.691+0000 7f745aa00640 1 --2- 172.21.15.152:0/325308359 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f745c071cd0 0x7f745c141ce0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f744c009890 tx=0x7f744c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:22.692 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.691+0000 7f7452a00640 1 -- 172.21.15.152:0/325308359 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f744c041070 con 0x7f745c071cd0 2024-04-20T11:28:22.692 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.691+0000 7f7452a00640 1 -- 172.21.15.152:0/325308359 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f744c03b920 con 0x7f745c071cd0 2024-04-20T11:28:22.692 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.691+0000 7f7452a00640 1 -- 172.21.15.152:0/325308359 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f744c035040 con 0x7f745c071cd0 2024-04-20T11:28:22.693 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.691+0000 7f745be00640 1 -- 172.21.15.152:0/325308359 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f745c0ad950 con 0x7f745c071cd0 2024-04-20T11:28:22.693 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.692+0000 7f745be00640 1 -- 172.21.15.152:0/325308359 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f745c0adf10 con 0x7f745c071cd0 2024-04-20T11:28:22.693 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.693+0000 7f745be00640 1 -- 172.21.15.152:0/325308359 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f745c075e60 con 0x7f745c071cd0 2024-04-20T11:28:22.693 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.694+0000 7f7452a00640 1 -- 172.21.15.152:0/325308359 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f744c04c060 con 0x7f745c071cd0 2024-04-20T11:28:22.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.694+0000 7f7452a00640 1 --2- 172.21.15.152:0/325308359 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f743c06c8f0 0x7f743c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:22.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.695+0000 7f7452a00640 1 -- 172.21.15.152:0/325308359 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f744c0ad560 con 0x7f745c071cd0 2024-04-20T11:28:22.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.695+0000 7f745a000640 1 --2- 172.21.15.152:0/325308359 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f743c06c8f0 0x7f743c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:22.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.696+0000 7f745a000640 1 --2- 172.21.15.152:0/325308359 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f743c06c8f0 0x7f743c06edb0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f7448006fd0 tx=0x7f7448008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:22.699 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.699+0000 7f7452a00640 1 -- 172.21.15.152:0/325308359 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f744c07b900 con 0x7f745c071cd0 2024-04-20T11:28:22.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.898+0000 7f745be00640 1 -- 172.21.15.152:0/325308359 --> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f745c0b1d70 con 0x7f743c06c8f0 2024-04-20T11:28:22.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.917+0000 7f7452a00640 1 -- 172.21.15.152:0/325308359 <== mgr.4113 v2:172.21.15.152:6832/2285110240 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+239854 (secure 0 0 0) 0x7f745c0b1d70 con 0x7f743c06c8f0 2024-04-20T11:28:22.920 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:28:22.921 INFO:teuthology.orchestra.run.smithi152.stderr:dumped all 2024-04-20T11:28:22.922 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.922+0000 7f745be00640 1 -- 172.21.15.152:0/325308359 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f743c06c8f0 msgr2=0x7f743c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:22.922 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.922+0000 7f745be00640 1 --2- 172.21.15.152:0/325308359 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f743c06c8f0 0x7f743c06edb0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f7448006fd0 tx=0x7f7448008040 comp rx=0 tx=0).stop 2024-04-20T11:28:22.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.922+0000 7f745be00640 1 -- 172.21.15.152:0/325308359 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f745c071cd0 msgr2=0x7f745c141ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:22.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.922+0000 7f745be00640 1 --2- 172.21.15.152:0/325308359 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f745c071cd0 0x7f745c141ce0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f744c009890 tx=0x7f744c0046c0 comp rx=0 tx=0).stop 2024-04-20T11:28:22.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.922+0000 7f745be00640 1 -- 172.21.15.152:0/325308359 shutdown_connections 2024-04-20T11:28:22.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.922+0000 7f745be00640 1 --2- 172.21.15.152:0/325308359 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f743c06c8f0 0x7f743c06edb0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:22.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.922+0000 7f745be00640 1 --2- 172.21.15.152:0/325308359 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f745c15d5c0 0x7f745c152690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:22.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.922+0000 7f745be00640 1 --2- 172.21.15.152:0/325308359 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f745c15d1e0 0x7f745c152150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:22.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.922+0000 7f745be00640 1 --2- 172.21.15.152:0/325308359 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f745c071cd0 0x7f745c141ce0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:22.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.922+0000 7f745be00640 1 -- 172.21.15.152:0/325308359 >> 172.21.15.152:0/325308359 conn(0x7f745c067ae0 msgr2=0x7f745c15ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:22.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.923+0000 7f745be00640 1 -- 172.21.15.152:0/325308359 shutdown_connections 2024-04-20T11:28:22.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:22.923+0000 7f745be00640 1 -- 172.21.15.152:0/325308359 wait complete. 2024-04-20T11:28:22.941 INFO:teuthology.orchestra.run.smithi152.stdout:{"pg_ready":true,"pg_map":{"version":74,"stamp":"2024-04-20T11:28:21.236333+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220040,"kb_used_data":3868,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754680,"statfs":{"total":773094113280,"available":772868792320,"internally_reserved":0,"allocated":3960832,"data_stored":2241311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12746,"internal_metadata":221236790},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.808501"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:28:17.423007+0000","last_change":"2024-04-20T11:28:17.423007+0000","last_active":"2024-04-20T11:28:17.423007+0000","last_peered":"2024-04-20T11:28:17.423007+0000","last_clean":"2024-04-20T11:28:17.423007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:28:17.423007+0000","last_undegraded":"2024-04-20T11:28:17.423007+0000","last_fullsized":"2024-04-20T11:28:17.423007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:28:17.423007+0000","last_change":"2024-04-20T11:28:17.423007+0000","last_active":"2024-04-20T11:28:17.423007+0000","last_peered":"2024-04-20T11:28:17.423007+0000","last_clean":"2024-04-20T11:28:17.423007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:28:17.423007+0000","last_undegraded":"2024-04-20T11:28:17.423007+0000","last_fullsized":"2024-04-20T11:28:17.423007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:28:17.423007+0000","last_change":"2024-04-20T11:28:17.423007+0000","last_active":"2024-04-20T11:28:17.423007+0000","last_peered":"2024-04-20T11:28:17.423007+0000","last_clean":"2024-04-20T11:28:17.423007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:28:17.423007+0000","last_undegraded":"2024-04-20T11:28:17.423007+0000","last_fullsized":"2024-04-20T11:28:17.423007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:28:17.423007+0000","last_change":"2024-04-20T11:28:17.423007+0000","last_active":"2024-04-20T11:28:17.423007+0000","last_peered":"2024-04-20T11:28:17.423007+0000","last_clean":"2024-04-20T11:28:17.423007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:28:17.423007+0000","last_undegraded":"2024-04-20T11:28:17.423007+0000","last_fullsized":"2024-04-20T11:28:17.423007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:28:17.423007+0000","last_change":"2024-04-20T11:28:17.423007+0000","last_active":"2024-04-20T11:28:17.423007+0000","last_peered":"2024-04-20T11:28:17.423007+0000","last_clean":"2024-04-20T11:28:17.423007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:28:17.423007+0000","last_undegraded":"2024-04-20T11:28:17.423007+0000","last_fullsized":"2024-04-20T11:28:17.423007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:28:17.423007+0000","last_change":"2024-04-20T11:28:17.423007+0000","last_active":"2024-04-20T11:28:17.423007+0000","last_peered":"2024-04-20T11:28:17.423007+0000","last_clean":"2024-04-20T11:28:17.423007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:28:17.423007+0000","last_undegraded":"2024-04-20T11:28:17.423007+0000","last_fullsized":"2024-04-20T11:28:17.423007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:28:17.423007+0000","last_change":"2024-04-20T11:28:17.423007+0000","last_active":"2024-04-20T11:28:17.423007+0000","last_peered":"2024-04-20T11:28:17.423007+0000","last_clean":"2024-04-20T11:28:17.423007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:28:17.423007+0000","last_undegraded":"2024-04-20T11:28:17.423007+0000","last_fullsized":"2024-04-20T11:28:17.423007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:28:17.423007+0000","last_change":"2024-04-20T11:28:17.423007+0000","last_active":"2024-04-20T11:28:17.423007+0000","last_peered":"2024-04-20T11:28:17.423007+0000","last_clean":"2024-04-20T11:28:17.423007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:28:17.423007+0000","last_undegraded":"2024-04-20T11:28:17.423007+0000","last_fullsized":"2024-04-20T11:28:17.423007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:28:17.423007+0000","last_change":"2024-04-20T11:28:17.423007+0000","last_active":"2024-04-20T11:28:17.423007+0000","last_peered":"2024-04-20T11:28:17.423007+0000","last_clean":"2024-04-20T11:28:17.423007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:28:17.423007+0000","last_undegraded":"2024-04-20T11:28:17.423007+0000","last_fullsized":"2024-04-20T11:28:17.423007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:28:17.423007+0000","last_change":"2024-04-20T11:28:17.423007+0000","last_active":"2024-04-20T11:28:17.423007+0000","last_peered":"2024-04-20T11:28:17.423007+0000","last_clean":"2024-04-20T11:28:17.423007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:28:17.423007+0000","last_undegraded":"2024-04-20T11:28:17.423007+0000","last_fullsized":"2024-04-20T11:28:17.423007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:28:17.423007+0000","last_change":"2024-04-20T11:28:17.423007+0000","last_active":"2024-04-20T11:28:17.423007+0000","last_peered":"2024-04-20T11:28:17.423007+0000","last_clean":"2024-04-20T11:28:17.423007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:28:17.423007+0000","last_undegraded":"2024-04-20T11:28:17.423007+0000","last_fullsized":"2024-04-20T11:28:17.423007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:28:17.423007+0000","last_change":"2024-04-20T11:28:17.423007+0000","last_active":"2024-04-20T11:28:17.423007+0000","last_peered":"2024-04-20T11:28:17.423007+0000","last_clean":"2024-04-20T11:28:17.423007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:28:17.423007+0000","last_undegraded":"2024-04-20T11:28:17.423007+0000","last_fullsized":"2024-04-20T11:28:17.423007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:28:17.423007+0000","last_change":"2024-04-20T11:28:17.423007+0000","last_active":"2024-04-20T11:28:17.423007+0000","last_peered":"2024-04-20T11:28:17.423007+0000","last_clean":"2024-04-20T11:28:17.423007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:28:17.423007+0000","last_undegraded":"2024-04-20T11:28:17.423007+0000","last_fullsized":"2024-04-20T11:28:17.423007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:28:17.423007+0000","last_change":"2024-04-20T11:28:17.423007+0000","last_active":"2024-04-20T11:28:17.423007+0000","last_peered":"2024-04-20T11:28:17.423007+0000","last_clean":"2024-04-20T11:28:17.423007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:28:17.423007+0000","last_undegraded":"2024-04-20T11:28:17.423007+0000","last_fullsized":"2024-04-20T11:28:17.423007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:28:17.423007+0000","last_change":"2024-04-20T11:28:17.423007+0000","last_active":"2024-04-20T11:28:17.423007+0000","last_peered":"2024-04-20T11:28:17.423007+0000","last_clean":"2024-04-20T11:28:17.423007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:28:17.423007+0000","last_undegraded":"2024-04-20T11:28:17.423007+0000","last_fullsized":"2024-04-20T11:28:17.423007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:28:17.423007+0000","last_change":"2024-04-20T11:28:17.423007+0000","last_active":"2024-04-20T11:28:17.423007+0000","last_peered":"2024-04-20T11:28:17.423007+0000","last_clean":"2024-04-20T11:28:17.423007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:28:17.423007+0000","last_undegraded":"2024-04-20T11:28:17.423007+0000","last_fullsized":"2024-04-20T11:28:17.423007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:28:17.423007+0000","last_change":"2024-04-20T11:28:17.423007+0000","last_active":"2024-04-20T11:28:17.423007+0000","last_peered":"2024-04-20T11:28:17.423007+0000","last_clean":"2024-04-20T11:28:17.423007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:28:17.423007+0000","last_undegraded":"2024-04-20T11:28:17.423007+0000","last_fullsized":"2024-04-20T11:28:17.423007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:28:17.423007+0000","last_change":"2024-04-20T11:28:17.423007+0000","last_active":"2024-04-20T11:28:17.423007+0000","last_peered":"2024-04-20T11:28:17.423007+0000","last_clean":"2024-04-20T11:28:17.423007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:28:17.423007+0000","last_undegraded":"2024-04-20T11:28:17.423007+0000","last_fullsized":"2024-04-20T11:28:17.423007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:28:17.423007+0000","last_change":"2024-04-20T11:28:17.423007+0000","last_active":"2024-04-20T11:28:17.423007+0000","last_peered":"2024-04-20T11:28:17.423007+0000","last_clean":"2024-04-20T11:28:17.423007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:28:17.423007+0000","last_undegraded":"2024-04-20T11:28:17.423007+0000","last_fullsized":"2024-04-20T11:28:17.423007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:28:17.423007+0000","last_change":"2024-04-20T11:28:17.423007+0000","last_active":"2024-04-20T11:28:17.423007+0000","last_peered":"2024-04-20T11:28:17.423007+0000","last_clean":"2024-04-20T11:28:17.423007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:28:17.423007+0000","last_undegraded":"2024-04-20T11:28:17.423007+0000","last_fullsized":"2024-04-20T11:28:17.423007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:28:17.423007+0000","last_change":"2024-04-20T11:28:17.423007+0000","last_active":"2024-04-20T11:28:17.423007+0000","last_peered":"2024-04-20T11:28:17.423007+0000","last_clean":"2024-04-20T11:28:17.423007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:28:17.423007+0000","last_undegraded":"2024-04-20T11:28:17.423007+0000","last_fullsized":"2024-04-20T11:28:17.423007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:28:17.423007+0000","last_change":"2024-04-20T11:28:17.423007+0000","last_active":"2024-04-20T11:28:17.423007+0000","last_peered":"2024-04-20T11:28:17.423007+0000","last_clean":"2024-04-20T11:28:17.423007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:28:17.423007+0000","last_undegraded":"2024-04-20T11:28:17.423007+0000","last_fullsized":"2024-04-20T11:28:17.423007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:28:17.423007+0000","last_change":"2024-04-20T11:28:17.423007+0000","last_active":"2024-04-20T11:28:17.423007+0000","last_peered":"2024-04-20T11:28:17.423007+0000","last_clean":"2024-04-20T11:28:17.423007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:28:17.423007+0000","last_undegraded":"2024-04-20T11:28:17.423007+0000","last_fullsized":"2024-04-20T11:28:17.423007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:28:17.423007+0000","last_change":"2024-04-20T11:28:17.423007+0000","last_active":"2024-04-20T11:28:17.423007+0000","last_peered":"2024-04-20T11:28:17.423007+0000","last_clean":"2024-04-20T11:28:17.423007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:28:17.423007+0000","last_undegraded":"2024-04-20T11:28:17.423007+0000","last_fullsized":"2024-04-20T11:28:17.423007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:28:17.423007+0000","last_change":"2024-04-20T11:28:17.423007+0000","last_active":"2024-04-20T11:28:17.423007+0000","last_peered":"2024-04-20T11:28:17.423007+0000","last_clean":"2024-04-20T11:28:17.423007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:28:17.423007+0000","last_undegraded":"2024-04-20T11:28:17.423007+0000","last_fullsized":"2024-04-20T11:28:17.423007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:28:17.423007+0000","last_change":"2024-04-20T11:28:17.423007+0000","last_active":"2024-04-20T11:28:17.423007+0000","last_peered":"2024-04-20T11:28:17.423007+0000","last_clean":"2024-04-20T11:28:17.423007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:28:17.423007+0000","last_undegraded":"2024-04-20T11:28:17.423007+0000","last_fullsized":"2024-04-20T11:28:17.423007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:28:17.423007+0000","last_change":"2024-04-20T11:28:17.423007+0000","last_active":"2024-04-20T11:28:17.423007+0000","last_peered":"2024-04-20T11:28:17.423007+0000","last_clean":"2024-04-20T11:28:17.423007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:28:17.423007+0000","last_undegraded":"2024-04-20T11:28:17.423007+0000","last_fullsized":"2024-04-20T11:28:17.423007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:28:17.423007+0000","last_change":"2024-04-20T11:28:17.423007+0000","last_active":"2024-04-20T11:28:17.423007+0000","last_peered":"2024-04-20T11:28:17.423007+0000","last_clean":"2024-04-20T11:28:17.423007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:28:17.423007+0000","last_undegraded":"2024-04-20T11:28:17.423007+0000","last_fullsized":"2024-04-20T11:28:17.423007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:28:17.423007+0000","last_change":"2024-04-20T11:28:17.423007+0000","last_active":"2024-04-20T11:28:17.423007+0000","last_peered":"2024-04-20T11:28:17.423007+0000","last_clean":"2024-04-20T11:28:17.423007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:28:17.423007+0000","last_undegraded":"2024-04-20T11:28:17.423007+0000","last_fullsized":"2024-04-20T11:28:17.423007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:28:17.423007+0000","last_change":"2024-04-20T11:28:17.423007+0000","last_active":"2024-04-20T11:28:17.423007+0000","last_peered":"2024-04-20T11:28:17.423007+0000","last_clean":"2024-04-20T11:28:17.423007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:28:17.423007+0000","last_undegraded":"2024-04-20T11:28:17.423007+0000","last_fullsized":"2024-04-20T11:28:17.423007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:28:17.423007+0000","last_change":"2024-04-20T11:28:17.423007+0000","last_active":"2024-04-20T11:28:17.423007+0000","last_peered":"2024-04-20T11:28:17.423007+0000","last_clean":"2024-04-20T11:28:17.423007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:28:17.423007+0000","last_undegraded":"2024-04-20T11:28:17.423007+0000","last_fullsized":"2024-04-20T11:28:17.423007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:28:17.423007+0000","last_change":"2024-04-20T11:28:17.423007+0000","last_active":"2024-04-20T11:28:17.423007+0000","last_peered":"2024-04-20T11:28:17.423007+0000","last_clean":"2024-04-20T11:28:17.423007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:28:17.423007+0000","last_undegraded":"2024-04-20T11:28:17.423007+0000","last_fullsized":"2024-04-20T11:28:17.423007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:28:17.423007+0000","last_change":"2024-04-20T11:28:17.423007+0000","last_active":"2024-04-20T11:28:17.423007+0000","last_peered":"2024-04-20T11:28:17.423007+0000","last_clean":"2024-04-20T11:28:17.423007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:28:17.423007+0000","last_undegraded":"2024-04-20T11:28:17.423007+0000","last_fullsized":"2024-04-20T11:28:17.423007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-20T11:28:16.438854+0000","last_change":"2024-04-20T11:28:16.438854+0000","last_active":"2024-04-20T11:28:16.405655+0000","last_peered":"2024-04-20T11:28:16.405655+0000","last_clean":"2024-04-20T11:28:16.405655+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:28:16.438854+0000","last_undegraded":"2024-04-20T11:28:16.438854+0000","last_fullsized":"2024-04-20T11:28:16.438854+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_clean_scrub_stamp":"2024-04-20T11:28:16.405655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:28:17.423007+0000","last_change":"2024-04-20T11:28:17.423007+0000","last_active":"2024-04-20T11:28:17.423007+0000","last_peered":"2024-04-20T11:28:17.423007+0000","last_clean":"2024-04-20T11:28:17.423007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:28:17.423007+0000","last_undegraded":"2024-04-20T11:28:17.423007+0000","last_fullsized":"2024-04-20T11:28:17.423007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-20T11:28:16.446699+0000","last_change":"2024-04-20T11:28:16.446699+0000","last_active":"2024-04-20T11:28:16.405655+0000","last_peered":"2024-04-20T11:28:16.405655+0000","last_clean":"2024-04-20T11:28:16.405655+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:28:16.446699+0000","last_undegraded":"2024-04-20T11:28:16.446699+0000","last_fullsized":"2024-04-20T11:28:16.446699+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_clean_scrub_stamp":"2024-04-20T11:28:16.405655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:28:17.423007+0000","last_change":"2024-04-20T11:28:17.423007+0000","last_active":"2024-04-20T11:28:17.423007+0000","last_peered":"2024-04-20T11:28:17.423007+0000","last_clean":"2024-04-20T11:28:17.423007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:28:17.423007+0000","last_undegraded":"2024-04-20T11:28:17.423007+0000","last_fullsized":"2024-04-20T11:28:17.423007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-20T11:28:16.430313+0000","last_change":"2024-04-20T11:28:16.430313+0000","last_active":"2024-04-20T11:28:16.405655+0000","last_peered":"2024-04-20T11:28:16.405655+0000","last_clean":"2024-04-20T11:28:16.405655+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:28:16.430313+0000","last_undegraded":"2024-04-20T11:28:16.430313+0000","last_fullsized":"2024-04-20T11:28:16.430313+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_clean_scrub_stamp":"2024-04-20T11:28:16.405655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:28:17.423007+0000","last_change":"2024-04-20T11:28:17.423007+0000","last_active":"2024-04-20T11:28:17.423007+0000","last_peered":"2024-04-20T11:28:17.423007+0000","last_clean":"2024-04-20T11:28:17.423007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:28:17.423007+0000","last_undegraded":"2024-04-20T11:28:17.423007+0000","last_fullsized":"2024-04-20T11:28:17.423007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-20T11:28:16.434118+0000","last_change":"2024-04-20T11:28:16.434118+0000","last_active":"2024-04-20T11:28:16.405655+0000","last_peered":"2024-04-20T11:28:16.405655+0000","last_clean":"2024-04-20T11:28:16.405655+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:28:16.434118+0000","last_undegraded":"2024-04-20T11:28:16.434118+0000","last_fullsized":"2024-04-20T11:28:16.434118+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_clean_scrub_stamp":"2024-04-20T11:28:16.405655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:28:17.423007+0000","last_change":"2024-04-20T11:28:17.423007+0000","last_active":"2024-04-20T11:28:17.423007+0000","last_peered":"2024-04-20T11:28:17.423007+0000","last_clean":"2024-04-20T11:28:17.423007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:28:17.423007+0000","last_undegraded":"2024-04-20T11:28:17.423007+0000","last_fullsized":"2024-04-20T11:28:17.423007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-20T11:28:16.446551+0000","last_change":"2024-04-20T11:28:16.446551+0000","last_active":"2024-04-20T11:28:16.405655+0000","last_peered":"2024-04-20T11:28:16.405655+0000","last_clean":"2024-04-20T11:28:16.405655+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:28:16.446551+0000","last_undegraded":"2024-04-20T11:28:16.446551+0000","last_fullsized":"2024-04-20T11:28:16.446551+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_clean_scrub_stamp":"2024-04-20T11:28:16.405655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:28:17.423007+0000","last_change":"2024-04-20T11:28:17.423007+0000","last_active":"2024-04-20T11:28:17.423007+0000","last_peered":"2024-04-20T11:28:17.423007+0000","last_clean":"2024-04-20T11:28:17.423007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:28:17.423007+0000","last_undegraded":"2024-04-20T11:28:17.423007+0000","last_fullsized":"2024-04-20T11:28:17.423007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-20T11:28:16.444388+0000","last_change":"2024-04-20T11:28:16.444388+0000","last_active":"2024-04-20T11:28:16.405655+0000","last_peered":"2024-04-20T11:28:16.405655+0000","last_clean":"2024-04-20T11:28:16.405655+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:28:16.444388+0000","last_undegraded":"2024-04-20T11:28:16.444388+0000","last_fullsized":"2024-04-20T11:28:16.444388+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_clean_scrub_stamp":"2024-04-20T11:28:16.405655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:28:17.423007+0000","last_change":"2024-04-20T11:28:17.423007+0000","last_active":"2024-04-20T11:28:17.423007+0000","last_peered":"2024-04-20T11:28:17.423007+0000","last_clean":"2024-04-20T11:28:17.423007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:28:17.423007+0000","last_undegraded":"2024-04-20T11:28:17.423007+0000","last_fullsized":"2024-04-20T11:28:17.423007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-20T11:28:16.429774+0000","last_change":"2024-04-20T11:28:16.429774+0000","last_active":"2024-04-20T11:28:16.405655+0000","last_peered":"2024-04-20T11:28:16.405655+0000","last_clean":"2024-04-20T11:28:16.405655+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:28:16.429774+0000","last_undegraded":"2024-04-20T11:28:16.429774+0000","last_fullsized":"2024-04-20T11:28:16.429774+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_clean_scrub_stamp":"2024-04-20T11:28:16.405655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:28:17.423007+0000","last_change":"2024-04-20T11:28:17.423007+0000","last_active":"2024-04-20T11:28:17.423007+0000","last_peered":"2024-04-20T11:28:17.423007+0000","last_clean":"2024-04-20T11:28:17.423007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:28:17.423007+0000","last_undegraded":"2024-04-20T11:28:17.423007+0000","last_fullsized":"2024-04-20T11:28:17.423007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-20T11:28:16.444340+0000","last_change":"2024-04-20T11:28:16.444340+0000","last_active":"2024-04-20T11:28:16.405655+0000","last_peered":"2024-04-20T11:28:16.405655+0000","last_clean":"2024-04-20T11:28:16.405655+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:28:16.444340+0000","last_undegraded":"2024-04-20T11:28:16.444340+0000","last_fullsized":"2024-04-20T11:28:16.444340+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_clean_scrub_stamp":"2024-04-20T11:28:16.405655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:28:17.423007+0000","last_change":"2024-04-20T11:28:17.423007+0000","last_active":"2024-04-20T11:28:17.423007+0000","last_peered":"2024-04-20T11:28:17.423007+0000","last_clean":"2024-04-20T11:28:17.423007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:28:17.423007+0000","last_undegraded":"2024-04-20T11:28:17.423007+0000","last_fullsized":"2024-04-20T11:28:17.423007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-20T11:28:16.431031+0000","last_change":"2024-04-20T11:28:16.431031+0000","last_active":"2024-04-20T11:28:16.405655+0000","last_peered":"2024-04-20T11:28:16.405655+0000","last_clean":"2024-04-20T11:28:16.405655+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:28:16.431031+0000","last_undegraded":"2024-04-20T11:28:16.431031+0000","last_fullsized":"2024-04-20T11:28:16.431031+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_clean_scrub_stamp":"2024-04-20T11:28:16.405655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:28:17.423007+0000","last_change":"2024-04-20T11:28:17.423007+0000","last_active":"2024-04-20T11:28:17.423007+0000","last_peered":"2024-04-20T11:28:17.423007+0000","last_clean":"2024-04-20T11:28:17.423007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:28:17.423007+0000","last_undegraded":"2024-04-20T11:28:17.423007+0000","last_fullsized":"2024-04-20T11:28:17.423007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-20T11:28:16.444729+0000","last_change":"2024-04-20T11:28:16.444729+0000","last_active":"2024-04-20T11:28:16.405655+0000","last_peered":"2024-04-20T11:28:16.405655+0000","last_clean":"2024-04-20T11:28:16.405655+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:28:16.444729+0000","last_undegraded":"2024-04-20T11:28:16.444729+0000","last_fullsized":"2024-04-20T11:28:16.444729+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_clean_scrub_stamp":"2024-04-20T11:28:16.405655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:28:17.423007+0000","last_change":"2024-04-20T11:28:17.423007+0000","last_active":"2024-04-20T11:28:17.423007+0000","last_peered":"2024-04-20T11:28:17.423007+0000","last_clean":"2024-04-20T11:28:17.423007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:28:17.423007+0000","last_undegraded":"2024-04-20T11:28:17.423007+0000","last_fullsized":"2024-04-20T11:28:17.423007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-20T11:28:16.432813+0000","last_change":"2024-04-20T11:28:16.432813+0000","last_active":"2024-04-20T11:28:16.405655+0000","last_peered":"2024-04-20T11:28:16.405655+0000","last_clean":"2024-04-20T11:28:16.405655+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:28:16.432813+0000","last_undegraded":"2024-04-20T11:28:16.432813+0000","last_fullsized":"2024-04-20T11:28:16.432813+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_clean_scrub_stamp":"2024-04-20T11:28:16.405655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:28:17.423007+0000","last_change":"2024-04-20T11:28:17.423007+0000","last_active":"2024-04-20T11:28:17.423007+0000","last_peered":"2024-04-20T11:28:17.423007+0000","last_clean":"2024-04-20T11:28:17.423007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:28:17.423007+0000","last_undegraded":"2024-04-20T11:28:17.423007+0000","last_fullsized":"2024-04-20T11:28:17.423007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-20T11:28:16.429775+0000","last_change":"2024-04-20T11:28:16.429775+0000","last_active":"2024-04-20T11:28:16.405655+0000","last_peered":"2024-04-20T11:28:16.405655+0000","last_clean":"2024-04-20T11:28:16.405655+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:28:16.429775+0000","last_undegraded":"2024-04-20T11:28:16.429775+0000","last_fullsized":"2024-04-20T11:28:16.429775+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_clean_scrub_stamp":"2024-04-20T11:28:16.405655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:28:17.423007+0000","last_change":"2024-04-20T11:28:17.423007+0000","last_active":"2024-04-20T11:28:17.423007+0000","last_peered":"2024-04-20T11:28:17.423007+0000","last_clean":"2024-04-20T11:28:17.423007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:28:17.423007+0000","last_undegraded":"2024-04-20T11:28:17.423007+0000","last_fullsized":"2024-04-20T11:28:17.423007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-20T11:28:16.444382+0000","last_change":"2024-04-20T11:28:16.444382+0000","last_active":"2024-04-20T11:28:16.405655+0000","last_peered":"2024-04-20T11:28:16.405655+0000","last_clean":"2024-04-20T11:28:16.405655+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:28:16.444382+0000","last_undegraded":"2024-04-20T11:28:16.444382+0000","last_fullsized":"2024-04-20T11:28:16.444382+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_clean_scrub_stamp":"2024-04-20T11:28:16.405655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:28:17.423007+0000","last_change":"2024-04-20T11:28:17.423007+0000","last_active":"2024-04-20T11:28:17.423007+0000","last_peered":"2024-04-20T11:28:17.423007+0000","last_clean":"2024-04-20T11:28:17.423007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:28:17.423007+0000","last_undegraded":"2024-04-20T11:28:17.423007+0000","last_fullsized":"2024-04-20T11:28:17.423007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-20T11:28:16.433627+0000","last_change":"2024-04-20T11:28:16.433627+0000","last_active":"2024-04-20T11:28:16.405655+0000","last_peered":"2024-04-20T11:28:16.405655+0000","last_clean":"2024-04-20T11:28:16.405655+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:28:16.433627+0000","last_undegraded":"2024-04-20T11:28:16.433627+0000","last_fullsized":"2024-04-20T11:28:16.433627+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_clean_scrub_stamp":"2024-04-20T11:28:16.405655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:28:17.423007+0000","last_change":"2024-04-20T11:28:17.423007+0000","last_active":"2024-04-20T11:28:17.423007+0000","last_peered":"2024-04-20T11:28:17.423007+0000","last_clean":"2024-04-20T11:28:17.423007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:28:17.423007+0000","last_undegraded":"2024-04-20T11:28:17.423007+0000","last_fullsized":"2024-04-20T11:28:17.423007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-20T11:28:16.430441+0000","last_change":"2024-04-20T11:28:16.430441+0000","last_active":"2024-04-20T11:28:16.405655+0000","last_peered":"2024-04-20T11:28:16.405655+0000","last_clean":"2024-04-20T11:28:16.405655+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:28:16.430441+0000","last_undegraded":"2024-04-20T11:28:16.430441+0000","last_fullsized":"2024-04-20T11:28:16.430441+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_clean_scrub_stamp":"2024-04-20T11:28:16.405655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:28:17.423007+0000","last_change":"2024-04-20T11:28:17.423007+0000","last_active":"2024-04-20T11:28:17.423007+0000","last_peered":"2024-04-20T11:28:17.423007+0000","last_clean":"2024-04-20T11:28:17.423007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:28:17.423007+0000","last_undegraded":"2024-04-20T11:28:17.423007+0000","last_fullsized":"2024-04-20T11:28:17.423007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-20T11:28:16.430441+0000","last_change":"2024-04-20T11:28:16.430441+0000","last_active":"2024-04-20T11:28:16.405655+0000","last_peered":"2024-04-20T11:28:16.405655+0000","last_clean":"2024-04-20T11:28:16.405655+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:28:16.430441+0000","last_undegraded":"2024-04-20T11:28:16.430441+0000","last_fullsized":"2024-04-20T11:28:16.430441+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_clean_scrub_stamp":"2024-04-20T11:28:16.405655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:28:17.423007+0000","last_change":"2024-04-20T11:28:17.423007+0000","last_active":"2024-04-20T11:28:17.423007+0000","last_peered":"2024-04-20T11:28:17.423007+0000","last_clean":"2024-04-20T11:28:17.423007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:28:17.423007+0000","last_undegraded":"2024-04-20T11:28:17.423007+0000","last_fullsized":"2024-04-20T11:28:17.423007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-20T11:28:16.431058+0000","last_change":"2024-04-20T11:28:16.431058+0000","last_active":"2024-04-20T11:28:16.405655+0000","last_peered":"2024-04-20T11:28:16.405655+0000","last_clean":"2024-04-20T11:28:16.405655+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:28:16.431058+0000","last_undegraded":"2024-04-20T11:28:16.431058+0000","last_fullsized":"2024-04-20T11:28:16.431058+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_clean_scrub_stamp":"2024-04-20T11:28:16.405655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:28:17.423007+0000","last_change":"2024-04-20T11:28:17.423007+0000","last_active":"2024-04-20T11:28:17.423007+0000","last_peered":"2024-04-20T11:28:17.423007+0000","last_clean":"2024-04-20T11:28:17.423007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:28:17.423007+0000","last_undegraded":"2024-04-20T11:28:17.423007+0000","last_fullsized":"2024-04-20T11:28:17.423007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-20T11:28:16.444496+0000","last_change":"2024-04-20T11:28:16.444496+0000","last_active":"2024-04-20T11:28:16.405655+0000","last_peered":"2024-04-20T11:28:16.405655+0000","last_clean":"2024-04-20T11:28:16.405655+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:28:16.444496+0000","last_undegraded":"2024-04-20T11:28:16.444496+0000","last_fullsized":"2024-04-20T11:28:16.444496+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_clean_scrub_stamp":"2024-04-20T11:28:16.405655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:28:17.423007+0000","last_change":"2024-04-20T11:28:17.423007+0000","last_active":"2024-04-20T11:28:17.423007+0000","last_peered":"2024-04-20T11:28:17.423007+0000","last_clean":"2024-04-20T11:28:17.423007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:28:17.423007+0000","last_undegraded":"2024-04-20T11:28:17.423007+0000","last_fullsized":"2024-04-20T11:28:17.423007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-20T11:28:16.444391+0000","last_change":"2024-04-20T11:28:16.444391+0000","last_active":"2024-04-20T11:28:16.405655+0000","last_peered":"2024-04-20T11:28:16.405655+0000","last_clean":"2024-04-20T11:28:16.405655+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:28:16.444391+0000","last_undegraded":"2024-04-20T11:28:16.444391+0000","last_fullsized":"2024-04-20T11:28:16.444391+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_clean_scrub_stamp":"2024-04-20T11:28:16.405655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:28:17.423007+0000","last_change":"2024-04-20T11:28:17.423007+0000","last_active":"2024-04-20T11:28:17.423007+0000","last_peered":"2024-04-20T11:28:17.423007+0000","last_clean":"2024-04-20T11:28:17.423007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:28:17.423007+0000","last_undegraded":"2024-04-20T11:28:17.423007+0000","last_fullsized":"2024-04-20T11:28:17.423007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-20T11:28:16.444374+0000","last_change":"2024-04-20T11:28:16.444374+0000","last_active":"2024-04-20T11:28:16.405655+0000","last_peered":"2024-04-20T11:28:16.405655+0000","last_clean":"2024-04-20T11:28:16.405655+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:28:16.444374+0000","last_undegraded":"2024-04-20T11:28:16.444374+0000","last_fullsized":"2024-04-20T11:28:16.444374+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_clean_scrub_stamp":"2024-04-20T11:28:16.405655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:28:17.423007+0000","last_change":"2024-04-20T11:28:17.423007+0000","last_active":"2024-04-20T11:28:17.423007+0000","last_peered":"2024-04-20T11:28:17.423007+0000","last_clean":"2024-04-20T11:28:17.423007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:28:17.423007+0000","last_undegraded":"2024-04-20T11:28:17.423007+0000","last_fullsized":"2024-04-20T11:28:17.423007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-20T11:28:16.445864+0000","last_change":"2024-04-20T11:28:16.445864+0000","last_active":"2024-04-20T11:28:16.405655+0000","last_peered":"2024-04-20T11:28:16.405655+0000","last_clean":"2024-04-20T11:28:16.405655+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:28:16.445864+0000","last_undegraded":"2024-04-20T11:28:16.445864+0000","last_fullsized":"2024-04-20T11:28:16.445864+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_clean_scrub_stamp":"2024-04-20T11:28:16.405655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:28:17.423007+0000","last_change":"2024-04-20T11:28:17.423007+0000","last_active":"2024-04-20T11:28:17.423007+0000","last_peered":"2024-04-20T11:28:17.423007+0000","last_clean":"2024-04-20T11:28:17.423007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:28:17.423007+0000","last_undegraded":"2024-04-20T11:28:17.423007+0000","last_fullsized":"2024-04-20T11:28:17.423007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-20T11:28:16.438537+0000","last_change":"2024-04-20T11:28:16.438537+0000","last_active":"2024-04-20T11:28:16.405655+0000","last_peered":"2024-04-20T11:28:16.405655+0000","last_clean":"2024-04-20T11:28:16.405655+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:28:16.438537+0000","last_undegraded":"2024-04-20T11:28:16.438537+0000","last_fullsized":"2024-04-20T11:28:16.438537+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_clean_scrub_stamp":"2024-04-20T11:28:16.405655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:28:17.423007+0000","last_change":"2024-04-20T11:28:17.423007+0000","last_active":"2024-04-20T11:28:17.423007+0000","last_peered":"2024-04-20T11:28:17.423007+0000","last_clean":"2024-04-20T11:28:17.423007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:28:17.423007+0000","last_undegraded":"2024-04-20T11:28:17.423007+0000","last_fullsized":"2024-04-20T11:28:17.423007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-20T11:28:16.445730+0000","last_change":"2024-04-20T11:28:16.445730+0000","last_active":"2024-04-20T11:28:16.405655+0000","last_peered":"2024-04-20T11:28:16.405655+0000","last_clean":"2024-04-20T11:28:16.405655+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:28:16.445730+0000","last_undegraded":"2024-04-20T11:28:16.445730+0000","last_fullsized":"2024-04-20T11:28:16.445730+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_clean_scrub_stamp":"2024-04-20T11:28:16.405655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:28:17.423007+0000","last_change":"2024-04-20T11:28:17.423007+0000","last_active":"2024-04-20T11:28:17.423007+0000","last_peered":"2024-04-20T11:28:17.423007+0000","last_clean":"2024-04-20T11:28:17.423007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:28:17.423007+0000","last_undegraded":"2024-04-20T11:28:17.423007+0000","last_fullsized":"2024-04-20T11:28:17.423007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-20T11:28:16.432813+0000","last_change":"2024-04-20T11:28:16.432813+0000","last_active":"2024-04-20T11:28:16.405655+0000","last_peered":"2024-04-20T11:28:16.405655+0000","last_clean":"2024-04-20T11:28:16.405655+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:28:16.432813+0000","last_undegraded":"2024-04-20T11:28:16.432813+0000","last_fullsized":"2024-04-20T11:28:16.432813+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_clean_scrub_stamp":"2024-04-20T11:28:16.405655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:28:17.423007+0000","last_change":"2024-04-20T11:28:17.423007+0000","last_active":"2024-04-20T11:28:17.423007+0000","last_peered":"2024-04-20T11:28:17.423007+0000","last_clean":"2024-04-20T11:28:17.423007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:28:17.423007+0000","last_undegraded":"2024-04-20T11:28:17.423007+0000","last_fullsized":"2024-04-20T11:28:17.423007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-20T11:28:16.430546+0000","last_change":"2024-04-20T11:28:16.430546+0000","last_active":"2024-04-20T11:28:16.405655+0000","last_peered":"2024-04-20T11:28:16.405655+0000","last_clean":"2024-04-20T11:28:16.405655+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:28:16.430546+0000","last_undegraded":"2024-04-20T11:28:16.430546+0000","last_fullsized":"2024-04-20T11:28:16.430546+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_clean_scrub_stamp":"2024-04-20T11:28:16.405655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:28:17.423007+0000","last_change":"2024-04-20T11:28:17.423007+0000","last_active":"2024-04-20T11:28:17.423007+0000","last_peered":"2024-04-20T11:28:17.423007+0000","last_clean":"2024-04-20T11:28:17.423007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:28:17.423007+0000","last_undegraded":"2024-04-20T11:28:17.423007+0000","last_fullsized":"2024-04-20T11:28:17.423007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-20T11:28:16.446015+0000","last_change":"2024-04-20T11:28:16.446015+0000","last_active":"2024-04-20T11:28:16.405655+0000","last_peered":"2024-04-20T11:28:16.405655+0000","last_clean":"2024-04-20T11:28:16.405655+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:28:16.446015+0000","last_undegraded":"2024-04-20T11:28:16.446015+0000","last_fullsized":"2024-04-20T11:28:16.446015+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_clean_scrub_stamp":"2024-04-20T11:28:16.405655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:28:17.423007+0000","last_change":"2024-04-20T11:28:17.423007+0000","last_active":"2024-04-20T11:28:17.423007+0000","last_peered":"2024-04-20T11:28:17.423007+0000","last_clean":"2024-04-20T11:28:17.423007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:28:17.423007+0000","last_undegraded":"2024-04-20T11:28:17.423007+0000","last_fullsized":"2024-04-20T11:28:17.423007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-20T11:28:16.432432+0000","last_change":"2024-04-20T11:28:16.432432+0000","last_active":"2024-04-20T11:28:16.405655+0000","last_peered":"2024-04-20T11:28:16.405655+0000","last_clean":"2024-04-20T11:28:16.405655+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:28:16.432432+0000","last_undegraded":"2024-04-20T11:28:16.432432+0000","last_fullsized":"2024-04-20T11:28:16.432432+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_clean_scrub_stamp":"2024-04-20T11:28:16.405655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:28:17.423007+0000","last_change":"2024-04-20T11:28:17.423007+0000","last_active":"2024-04-20T11:28:17.423007+0000","last_peered":"2024-04-20T11:28:17.423007+0000","last_clean":"2024-04-20T11:28:17.423007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:28:17.423007+0000","last_undegraded":"2024-04-20T11:28:17.423007+0000","last_fullsized":"2024-04-20T11:28:17.423007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-20T11:28:16.444789+0000","last_change":"2024-04-20T11:28:16.444789+0000","last_active":"2024-04-20T11:28:16.405655+0000","last_peered":"2024-04-20T11:28:16.405655+0000","last_clean":"2024-04-20T11:28:16.405655+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:28:16.444789+0000","last_undegraded":"2024-04-20T11:28:16.444789+0000","last_fullsized":"2024-04-20T11:28:16.444789+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_clean_scrub_stamp":"2024-04-20T11:28:16.405655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:28:17.423007+0000","last_change":"2024-04-20T11:28:17.423007+0000","last_active":"2024-04-20T11:28:17.423007+0000","last_peered":"2024-04-20T11:28:17.423007+0000","last_clean":"2024-04-20T11:28:17.423007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:28:17.423007+0000","last_undegraded":"2024-04-20T11:28:17.423007+0000","last_fullsized":"2024-04-20T11:28:17.423007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-20T11:28:16.432378+0000","last_change":"2024-04-20T11:28:16.432378+0000","last_active":"2024-04-20T11:28:16.405655+0000","last_peered":"2024-04-20T11:28:16.405655+0000","last_clean":"2024-04-20T11:28:16.405655+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:28:16.432378+0000","last_undegraded":"2024-04-20T11:28:16.432378+0000","last_fullsized":"2024-04-20T11:28:16.432378+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_clean_scrub_stamp":"2024-04-20T11:28:16.405655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:28:17.423007+0000","last_change":"2024-04-20T11:28:17.423007+0000","last_active":"2024-04-20T11:28:17.423007+0000","last_peered":"2024-04-20T11:28:17.423007+0000","last_clean":"2024-04-20T11:28:17.423007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:28:17.423007+0000","last_undegraded":"2024-04-20T11:28:17.423007+0000","last_fullsized":"2024-04-20T11:28:17.423007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-20T11:28:16.444360+0000","last_change":"2024-04-20T11:28:16.444360+0000","last_active":"2024-04-20T11:28:16.405655+0000","last_peered":"2024-04-20T11:28:16.405655+0000","last_clean":"2024-04-20T11:28:16.405655+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:28:16.444360+0000","last_undegraded":"2024-04-20T11:28:16.444360+0000","last_fullsized":"2024-04-20T11:28:16.444360+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_clean_scrub_stamp":"2024-04-20T11:28:16.405655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:28:17.423007+0000","last_change":"2024-04-20T11:28:17.423007+0000","last_active":"2024-04-20T11:28:17.423007+0000","last_peered":"2024-04-20T11:28:17.423007+0000","last_clean":"2024-04-20T11:28:17.423007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:28:17.423007+0000","last_undegraded":"2024-04-20T11:28:17.423007+0000","last_fullsized":"2024-04-20T11:28:17.423007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-20T11:28:16.429746+0000","last_change":"2024-04-20T11:28:16.429746+0000","last_active":"2024-04-20T11:28:16.405655+0000","last_peered":"2024-04-20T11:28:16.405655+0000","last_clean":"2024-04-20T11:28:16.405655+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:28:16.429746+0000","last_undegraded":"2024-04-20T11:28:16.429746+0000","last_fullsized":"2024-04-20T11:28:16.429746+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_clean_scrub_stamp":"2024-04-20T11:28:16.405655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:28:17.423007+0000","last_change":"2024-04-20T11:28:17.423007+0000","last_active":"2024-04-20T11:28:17.423007+0000","last_peered":"2024-04-20T11:28:17.423007+0000","last_clean":"2024-04-20T11:28:17.423007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:28:17.423007+0000","last_undegraded":"2024-04-20T11:28:17.423007+0000","last_fullsized":"2024-04-20T11:28:17.423007+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-20T11:28:16.438821+0000","last_change":"2024-04-20T11:28:16.438821+0000","last_active":"2024-04-20T11:28:16.405655+0000","last_peered":"2024-04-20T11:28:16.405655+0000","last_clean":"2024-04-20T11:28:16.405655+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:28:16.438821+0000","last_undegraded":"2024-04-20T11:28:16.438821+0000","last_fullsized":"2024-04-20T11:28:16.438821+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_clean_scrub_stamp":"2024-04-20T11:28:16.405655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":189,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-20T11:28:16.436077+0000","last_change":"2024-04-20T11:26:48.661897+0000","last_active":"2024-04-20T11:28:16.436077+0000","last_peered":"2024-04-20T11:28:16.436077+0000","last_clean":"2024-04-20T11:28:16.436077+0000","last_became_active":"2024-04-20T11:26:48.656535+0000","last_became_peered":"2024-04-20T11:26:48.656535+0000","last_unstale":"2024-04-20T11:28:16.436077+0000","last_undegraded":"2024-04-20T11:28:16.436077+0000","last_fullsized":"2024-04-20T11:28:16.436077+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:47.543004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:47.543004+0000","last_clean_scrub_stamp":"2024-04-20T11:26:47.543004+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:58:35.527105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574871,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27940,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343900,"statfs":{"total":96636764160,"available":96608153600,"internally_reserved":0,"allocated":942080,"data_stored":723667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574869,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27940,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343900,"statfs":{"total":96636764160,"available":96608153600,"internally_reserved":0,"allocated":929792,"data_stored":717850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T11:28:22.945 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T11:28:23.184 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.184+0000 7fd257000640 1 Processor -- start 2024-04-20T11:28:23.184 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.184+0000 7fd257000640 1 -- start start 2024-04-20T11:28:23.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.186+0000 7fd257000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fd2501603d0 0x7fd2500b5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:23.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.186+0000 7fd257000640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fd2500b5480 0x7fd2500b63f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:23.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.186+0000 7fd257000640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2500d3350 con 0x7fd25015fcc0 2024-04-20T11:28:23.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.186+0000 7fd257000640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fd2500d3630 con 0x7fd2501603d0 2024-04-20T11:28:23.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.186+0000 7fd257000640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fd2500d3910 con 0x7fd2500b5480 2024-04-20T11:28:23.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.186+0000 7fd255c00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fd25015fcc0 0x7fd2500d4e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:57988/0 (socket says 172.21.15.152:57988) 2024-04-20T11:28:23.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.186+0000 7fd255c00640 1 -- 172.21.15.152:0/3592231444 learned_addr learned my addr 172.21.15.152:0/3592231444 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:28:23.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.186+0000 7fd255200640 1 --2- 172.21.15.152:0/3592231444 >> v2:172.21.15.152:3300/0 conn(0x7fd2501603d0 0x7fd2500b5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:23.187 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.187+0000 7fd256600640 1 --2- 172.21.15.152:0/3592231444 >> v2:172.21.15.175:3300/0 conn(0x7fd2500b5480 0x7fd2500b63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:23.187 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.187+0000 7fd255200640 1 -- 172.21.15.152:0/3592231444 >> v2:172.21.15.175:3300/0 conn(0x7fd2500b5480 msgr2=0x7fd2500b63f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:23.187 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.187+0000 7fd255200640 1 --2- 172.21.15.152:0/3592231444 >> v2:172.21.15.175:3300/0 conn(0x7fd2500b5480 0x7fd2500b63f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:23.187 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.187+0000 7fd255200640 1 -- 172.21.15.152:0/3592231444 >> v1:172.21.15.152:6789/0 conn(0x7fd25015fcc0 legacy=0x7fd2500d4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:23.187 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.187+0000 7fd255200640 1 -- 172.21.15.152:0/3592231444 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2500ced20 con 0x7fd2501603d0 2024-04-20T11:28:23.187 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.187+0000 7fd24fe00640 1 -- 172.21.15.152:0/3592231444 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3307547654 0 0) 0x7fd2500d3350 con 0x7fd25015fcc0 2024-04-20T11:28:23.188 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.188+0000 7fd255200640 1 --2- 172.21.15.152:0/3592231444 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd2501603d0 0x7fd2500b5eb0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fd25008e7f0 tx=0x7fd23c02bf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=873014d4aea59e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:23.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.188+0000 7fd24fe00640 1 -- 172.21.15.152:0/3592231444 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd23c039070 con 0x7fd2501603d0 2024-04-20T11:28:23.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.188+0000 7fd24fe00640 1 -- 172.21.15.152:0/3592231444 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd23c02ccb0 con 0x7fd2501603d0 2024-04-20T11:28:23.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.188+0000 7fd24fe00640 1 -- 172.21.15.152:0/3592231444 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd23c02ce60 con 0x7fd2501603d0 2024-04-20T11:28:23.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.189+0000 7fd257000640 1 -- 172.21.15.152:0/3592231444 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd2501603d0 msgr2=0x7fd2500b5eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:23.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.189+0000 7fd257000640 1 --2- 172.21.15.152:0/3592231444 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd2501603d0 0x7fd2500b5eb0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fd25008e7f0 tx=0x7fd23c02bf00 comp rx=0 tx=0).stop 2024-04-20T11:28:23.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.189+0000 7fd257000640 1 -- 172.21.15.152:0/3592231444 shutdown_connections 2024-04-20T11:28:23.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.189+0000 7fd257000640 1 --2- 172.21.15.152:0/3592231444 >> v2:172.21.15.175:3300/0 conn(0x7fd2500b5480 0x7fd2500b63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:23.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.189+0000 7fd257000640 1 --2- 172.21.15.152:0/3592231444 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd2501603d0 0x7fd2500b5eb0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:23.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.189+0000 7fd257000640 1 -- 172.21.15.152:0/3592231444 >> 172.21.15.152:0/3592231444 conn(0x7fd25008bfd0 msgr2=0x7fd2500d4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:23.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.189+0000 7fd257000640 1 -- 172.21.15.152:0/3592231444 shutdown_connections 2024-04-20T11:28:23.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.189+0000 7fd257000640 1 -- 172.21.15.152:0/3592231444 wait complete. 2024-04-20T11:28:23.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.191+0000 7fd257000640 1 Processor -- start 2024-04-20T11:28:23.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.191+0000 7fd257000640 1 -- start start 2024-04-20T11:28:23.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.192+0000 7fd257000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd2500b5480 0x7fd2500b9150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:23.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.192+0000 7fd257000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd25015fcc0 0x7fd250155e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:23.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.192+0000 7fd257000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd2501603d0 0x7fd2500b6f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:23.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.192+0000 7fd257000640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fd2500cf610 con 0x7fd2500b5480 2024-04-20T11:28:23.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.192+0000 7fd257000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd2500cf490 con 0x7fd25015fcc0 2024-04-20T11:28:23.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.192+0000 7fd257000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd2500cf790 con 0x7fd2501603d0 2024-04-20T11:28:23.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.192+0000 7fd255c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd2500b5480 0x7fd2500b9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:23.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.192+0000 7fd255c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd2500b5480 0x7fd2500b9150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:41218/0 (socket says 172.21.15.152:41218) 2024-04-20T11:28:23.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.192+0000 7fd255c00640 1 -- 172.21.15.152:0/2906645338 learned_addr learned my addr 172.21.15.152:0/2906645338 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:23.193 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.192+0000 7fd256600640 1 --2- 172.21.15.152:0/2906645338 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd2501603d0 0x7fd2500b6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:23.193 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.192+0000 7fd255200640 1 --2- 172.21.15.152:0/2906645338 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd25015fcc0 0x7fd250155e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:23.193 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.192+0000 7fd255c00640 1 -- 172.21.15.152:0/2906645338 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd2501603d0 msgr2=0x7fd2500b6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:23.193 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.192+0000 7fd255c00640 1 --2- 172.21.15.152:0/2906645338 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd2501603d0 0x7fd2500b6f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:23.193 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.193+0000 7fd255c00640 1 -- 172.21.15.152:0/2906645338 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd25015fcc0 msgr2=0x7fd250155e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:23.193 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.193+0000 7fd255c00640 1 --2- 172.21.15.152:0/2906645338 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd25015fcc0 0x7fd250155e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:23.193 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.193+0000 7fd255c00640 1 -- 172.21.15.152:0/2906645338 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2500b9690 con 0x7fd2500b5480 2024-04-20T11:28:23.194 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.193+0000 7fd256600640 1 --2- 172.21.15.152:0/2906645338 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd2501603d0 0x7fd2500b6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:28:23.194 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.193+0000 7fd255c00640 1 --2- 172.21.15.152:0/2906645338 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd2500b5480 0x7fd2500b9150 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fd24000c500 tx=0x7fd24000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:23.194 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.193+0000 7fd24d600640 1 -- 172.21.15.152:0/2906645338 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd240018070 con 0x7fd2500b5480 2024-04-20T11:28:23.194 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.193+0000 7fd24d600640 1 -- 172.21.15.152:0/2906645338 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd240007580 con 0x7fd2500b5480 2024-04-20T11:28:23.194 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.193+0000 7fd24d600640 1 -- 172.21.15.152:0/2906645338 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd240004a20 con 0x7fd2500b5480 2024-04-20T11:28:23.194 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.193+0000 7fd257000640 1 -- 172.21.15.152:0/2906645338 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2500b7950 con 0x7fd2500b5480 2024-04-20T11:28:23.194 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.193+0000 7fd257000640 1 -- 172.21.15.152:0/2906645338 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2500b8950 con 0x7fd2500b5480 2024-04-20T11:28:23.194 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.195+0000 7fd257000640 1 -- 172.21.15.152:0/2906645338 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2500b8760 con 0x7fd2500b5480 2024-04-20T11:28:23.195 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.195+0000 7fd24d600640 1 -- 172.21.15.152:0/2906645338 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd24001b070 con 0x7fd2500b5480 2024-04-20T11:28:23.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.196+0000 7fd24d600640 1 --2- 172.21.15.152:0/2906645338 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fd23006c8f0 0x7fd23006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:23.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.197+0000 7fd24d600640 1 -- 172.21.15.152:0/2906645338 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fd24008aec0 con 0x7fd2500b5480 2024-04-20T11:28:23.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.197+0000 7fd255200640 1 --2- 172.21.15.152:0/2906645338 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fd23006c8f0 0x7fd23006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:23.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.197+0000 7fd255200640 1 --2- 172.21.15.152:0/2906645338 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fd23006c8f0 0x7fd23006edb0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd23c036000 tx=0x7fd23c030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:23.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.201+0000 7fd24d600640 1 -- 172.21.15.152:0/2906645338 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fd2400592e0 con 0x7fd2500b5480 2024-04-20T11:28:23.397 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.398+0000 7fd257000640 1 -- 172.21.15.152:0/2906645338 --> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd2500b6ce0 con 0x7fd23006c8f0 2024-04-20T11:28:23.406 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.407+0000 7fd24d600640 1 -- 172.21.15.152:0/2906645338 <== mgr.4113 v2:172.21.15.152:6832/2285110240 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251951 (secure 0 0 0) 0x7fd2500b6ce0 con 0x7fd23006c8f0 2024-04-20T11:28:23.410 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:28:23.411 INFO:teuthology.orchestra.run.smithi152.stderr:dumped all 2024-04-20T11:28:23.412 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.412+0000 7fd257000640 1 -- 172.21.15.152:0/2906645338 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fd23006c8f0 msgr2=0x7fd23006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:23.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.412+0000 7fd257000640 1 --2- 172.21.15.152:0/2906645338 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fd23006c8f0 0x7fd23006edb0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd23c036000 tx=0x7fd23c030000 comp rx=0 tx=0).stop 2024-04-20T11:28:23.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.412+0000 7fd257000640 1 -- 172.21.15.152:0/2906645338 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd2500b5480 msgr2=0x7fd2500b9150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:23.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.412+0000 7fd257000640 1 --2- 172.21.15.152:0/2906645338 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd2500b5480 0x7fd2500b9150 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fd24000c500 tx=0x7fd24000c9c0 comp rx=0 tx=0).stop 2024-04-20T11:28:23.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.412+0000 7fd257000640 1 -- 172.21.15.152:0/2906645338 shutdown_connections 2024-04-20T11:28:23.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.412+0000 7fd257000640 1 --2- 172.21.15.152:0/2906645338 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fd23006c8f0 0x7fd23006edb0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:23.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.412+0000 7fd257000640 1 --2- 172.21.15.152:0/2906645338 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd2501603d0 0x7fd2500b6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:23.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.412+0000 7fd257000640 1 --2- 172.21.15.152:0/2906645338 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd25015fcc0 0x7fd250155e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:23.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.412+0000 7fd257000640 1 --2- 172.21.15.152:0/2906645338 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd2500b5480 0x7fd2500b9150 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:23.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.412+0000 7fd257000640 1 -- 172.21.15.152:0/2906645338 >> 172.21.15.152:0/2906645338 conn(0x7fd25008bfd0 msgr2=0x7fd2500d3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:23.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.413+0000 7fd257000640 1 -- 172.21.15.152:0/2906645338 shutdown_connections 2024-04-20T11:28:23.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.413+0000 7fd257000640 1 -- 172.21.15.152:0/2906645338 wait complete. 2024-04-20T11:28:23.432 INFO:teuthology.orchestra.run.smithi152.stdout:{"pg_ready":true,"pg_map":{"version":75,"stamp":"2024-04-20T11:28:23.236852+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":171,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220384,"kb_used_data":4208,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754336,"statfs":{"total":773094113280,"available":772868440064,"internally_reserved":0,"allocated":4308992,"data_stored":2394084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12746,"internal_metadata":221236790},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":41,"apply_latency_ms":41,"commit_latency_ns":41000000,"apply_latency_ns":41000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001572"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-20T11:28:21.600975+0000","last_change":"2024-04-20T11:28:18.478962+0000","last_active":"2024-04-20T11:28:21.600975+0000","last_peered":"2024-04-20T11:28:21.600975+0000","last_clean":"2024-04-20T11:28:21.600975+0000","last_became_active":"2024-04-20T11:28:18.478554+0000","last_became_peered":"2024-04-20T11:28:18.478554+0000","last_unstale":"2024-04-20T11:28:21.600975+0000","last_undegraded":"2024-04-20T11:28:21.600975+0000","last_fullsized":"2024-04-20T11:28:21.600975+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:07:30.183270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-20T11:28:19.490996+0000","last_change":"2024-04-20T11:28:18.487748+0000","last_active":"2024-04-20T11:28:19.490996+0000","last_peered":"2024-04-20T11:28:19.490996+0000","last_clean":"2024-04-20T11:28:19.490996+0000","last_became_active":"2024-04-20T11:28:18.487486+0000","last_became_peered":"2024-04-20T11:28:18.487486+0000","last_unstale":"2024-04-20T11:28:19.490996+0000","last_undegraded":"2024-04-20T11:28:19.490996+0000","last_fullsized":"2024-04-20T11:28:19.490996+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:51:25.124854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-20T11:28:20.902446+0000","last_change":"2024-04-20T11:28:18.475550+0000","last_active":"2024-04-20T11:28:20.902446+0000","last_peered":"2024-04-20T11:28:20.902446+0000","last_clean":"2024-04-20T11:28:20.902446+0000","last_became_active":"2024-04-20T11:28:18.475247+0000","last_became_peered":"2024-04-20T11:28:18.475247+0000","last_unstale":"2024-04-20T11:28:20.902446+0000","last_undegraded":"2024-04-20T11:28:20.902446+0000","last_fullsized":"2024-04-20T11:28:20.902446+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:27:20.206896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-20T11:28:21.596031+0000","last_change":"2024-04-20T11:28:18.494033+0000","last_active":"2024-04-20T11:28:21.596031+0000","last_peered":"2024-04-20T11:28:21.596031+0000","last_clean":"2024-04-20T11:28:21.596031+0000","last_became_active":"2024-04-20T11:28:18.493759+0000","last_became_peered":"2024-04-20T11:28:18.493759+0000","last_unstale":"2024-04-20T11:28:21.596031+0000","last_undegraded":"2024-04-20T11:28:21.596031+0000","last_fullsized":"2024-04-20T11:28:21.596031+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:05:22.570704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-20T11:28:21.597161+0000","last_change":"2024-04-20T11:28:18.495016+0000","last_active":"2024-04-20T11:28:21.597161+0000","last_peered":"2024-04-20T11:28:21.597161+0000","last_clean":"2024-04-20T11:28:21.597161+0000","last_became_active":"2024-04-20T11:28:18.494732+0000","last_became_peered":"2024-04-20T11:28:18.494732+0000","last_unstale":"2024-04-20T11:28:21.597161+0000","last_undegraded":"2024-04-20T11:28:21.597161+0000","last_fullsized":"2024-04-20T11:28:21.597161+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:08:18.111843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-20T11:28:19.490002+0000","last_change":"2024-04-20T11:28:18.482862+0000","last_active":"2024-04-20T11:28:19.490002+0000","last_peered":"2024-04-20T11:28:19.490002+0000","last_clean":"2024-04-20T11:28:19.490002+0000","last_became_active":"2024-04-20T11:28:18.482386+0000","last_became_peered":"2024-04-20T11:28:18.482386+0000","last_unstale":"2024-04-20T11:28:19.490002+0000","last_undegraded":"2024-04-20T11:28:19.490002+0000","last_fullsized":"2024-04-20T11:28:19.490002+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:13:27.234211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-20T11:28:19.489725+0000","last_change":"2024-04-20T11:28:18.490773+0000","last_active":"2024-04-20T11:28:19.489725+0000","last_peered":"2024-04-20T11:28:19.489725+0000","last_clean":"2024-04-20T11:28:19.489725+0000","last_became_active":"2024-04-20T11:28:18.490192+0000","last_became_peered":"2024-04-20T11:28:18.490192+0000","last_unstale":"2024-04-20T11:28:19.489725+0000","last_undegraded":"2024-04-20T11:28:19.489725+0000","last_fullsized":"2024-04-20T11:28:19.489725+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:57:21.575886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-20T11:28:19.489363+0000","last_change":"2024-04-20T11:28:18.488614+0000","last_active":"2024-04-20T11:28:19.489363+0000","last_peered":"2024-04-20T11:28:19.489363+0000","last_clean":"2024-04-20T11:28:19.489363+0000","last_became_active":"2024-04-20T11:28:18.488316+0000","last_became_peered":"2024-04-20T11:28:18.488316+0000","last_unstale":"2024-04-20T11:28:19.489363+0000","last_undegraded":"2024-04-20T11:28:19.489363+0000","last_fullsized":"2024-04-20T11:28:19.489363+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:05:22.570704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-20T11:28:21.599702+0000","last_change":"2024-04-20T11:28:18.490877+0000","last_active":"2024-04-20T11:28:21.599702+0000","last_peered":"2024-04-20T11:28:21.599702+0000","last_clean":"2024-04-20T11:28:21.599702+0000","last_became_active":"2024-04-20T11:28:18.490672+0000","last_became_peered":"2024-04-20T11:28:18.490672+0000","last_unstale":"2024-04-20T11:28:21.599702+0000","last_undegraded":"2024-04-20T11:28:21.599702+0000","last_fullsized":"2024-04-20T11:28:21.599702+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:00:28.260369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-20T11:28:21.596542+0000","last_change":"2024-04-20T11:28:18.496362+0000","last_active":"2024-04-20T11:28:21.596542+0000","last_peered":"2024-04-20T11:28:21.596542+0000","last_clean":"2024-04-20T11:28:21.596542+0000","last_became_active":"2024-04-20T11:28:18.491309+0000","last_became_peered":"2024-04-20T11:28:18.491309+0000","last_unstale":"2024-04-20T11:28:21.596542+0000","last_undegraded":"2024-04-20T11:28:21.596542+0000","last_fullsized":"2024-04-20T11:28:21.596542+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:57:21.575886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-20T11:28:19.491094+0000","last_change":"2024-04-20T11:28:18.490490+0000","last_active":"2024-04-20T11:28:19.491094+0000","last_peered":"2024-04-20T11:28:19.491094+0000","last_clean":"2024-04-20T11:28:19.491094+0000","last_became_active":"2024-04-20T11:28:18.489811+0000","last_became_peered":"2024-04-20T11:28:18.489811+0000","last_unstale":"2024-04-20T11:28:19.491094+0000","last_undegraded":"2024-04-20T11:28:19.491094+0000","last_fullsized":"2024-04-20T11:28:19.491094+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:51:25.124854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-20T11:28:20.901696+0000","last_change":"2024-04-20T11:28:18.477774+0000","last_active":"2024-04-20T11:28:20.901696+0000","last_peered":"2024-04-20T11:28:20.901696+0000","last_clean":"2024-04-20T11:28:20.901696+0000","last_became_active":"2024-04-20T11:28:18.476760+0000","last_became_peered":"2024-04-20T11:28:18.476760+0000","last_unstale":"2024-04-20T11:28:20.901696+0000","last_undegraded":"2024-04-20T11:28:20.901696+0000","last_fullsized":"2024-04-20T11:28:20.901696+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:00:28.260369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-20T11:28:21.597269+0000","last_change":"2024-04-20T11:28:18.490250+0000","last_active":"2024-04-20T11:28:21.597269+0000","last_peered":"2024-04-20T11:28:21.597269+0000","last_clean":"2024-04-20T11:28:21.597269+0000","last_became_active":"2024-04-20T11:28:18.488621+0000","last_became_peered":"2024-04-20T11:28:18.488621+0000","last_unstale":"2024-04-20T11:28:21.597269+0000","last_undegraded":"2024-04-20T11:28:21.597269+0000","last_fullsized":"2024-04-20T11:28:21.597269+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:16:10.386997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-20T11:28:19.491083+0000","last_change":"2024-04-20T11:28:18.476958+0000","last_active":"2024-04-20T11:28:19.491083+0000","last_peered":"2024-04-20T11:28:19.491083+0000","last_clean":"2024-04-20T11:28:19.491083+0000","last_became_active":"2024-04-20T11:28:18.476454+0000","last_became_peered":"2024-04-20T11:28:18.476454+0000","last_unstale":"2024-04-20T11:28:19.491083+0000","last_undegraded":"2024-04-20T11:28:19.491083+0000","last_fullsized":"2024-04-20T11:28:19.491083+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:57:08.115797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-20T11:28:20.901664+0000","last_change":"2024-04-20T11:28:18.472913+0000","last_active":"2024-04-20T11:28:20.901664+0000","last_peered":"2024-04-20T11:28:20.901664+0000","last_clean":"2024-04-20T11:28:20.901664+0000","last_became_active":"2024-04-20T11:28:18.472516+0000","last_became_peered":"2024-04-20T11:28:18.472516+0000","last_unstale":"2024-04-20T11:28:20.901664+0000","last_undegraded":"2024-04-20T11:28:20.901664+0000","last_fullsized":"2024-04-20T11:28:20.901664+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:58:18.415855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-20T11:28:19.490523+0000","last_change":"2024-04-20T11:28:18.499831+0000","last_active":"2024-04-20T11:28:19.490523+0000","last_peered":"2024-04-20T11:28:19.490523+0000","last_clean":"2024-04-20T11:28:19.490523+0000","last_became_active":"2024-04-20T11:28:18.499475+0000","last_became_peered":"2024-04-20T11:28:18.499475+0000","last_unstale":"2024-04-20T11:28:19.490523+0000","last_undegraded":"2024-04-20T11:28:19.490523+0000","last_fullsized":"2024-04-20T11:28:19.490523+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:37:50.794752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-20T11:28:19.490597+0000","last_change":"2024-04-20T11:28:18.496855+0000","last_active":"2024-04-20T11:28:19.490597+0000","last_peered":"2024-04-20T11:28:19.490597+0000","last_clean":"2024-04-20T11:28:19.490597+0000","last_became_active":"2024-04-20T11:28:18.493835+0000","last_became_peered":"2024-04-20T11:28:18.493835+0000","last_unstale":"2024-04-20T11:28:19.490597+0000","last_undegraded":"2024-04-20T11:28:19.490597+0000","last_fullsized":"2024-04-20T11:28:19.490597+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:13:27.234211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-20T11:28:19.491239+0000","last_change":"2024-04-20T11:28:18.486227+0000","last_active":"2024-04-20T11:28:19.491239+0000","last_peered":"2024-04-20T11:28:19.491239+0000","last_clean":"2024-04-20T11:28:19.491239+0000","last_became_active":"2024-04-20T11:28:18.485738+0000","last_became_peered":"2024-04-20T11:28:18.485738+0000","last_unstale":"2024-04-20T11:28:19.491239+0000","last_undegraded":"2024-04-20T11:28:19.491239+0000","last_fullsized":"2024-04-20T11:28:19.491239+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:04:41.109202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-20T11:28:19.490787+0000","last_change":"2024-04-20T11:28:18.478041+0000","last_active":"2024-04-20T11:28:19.490787+0000","last_peered":"2024-04-20T11:28:19.490787+0000","last_clean":"2024-04-20T11:28:19.490787+0000","last_became_active":"2024-04-20T11:28:18.477777+0000","last_became_peered":"2024-04-20T11:28:18.477777+0000","last_unstale":"2024-04-20T11:28:19.490787+0000","last_undegraded":"2024-04-20T11:28:19.490787+0000","last_fullsized":"2024-04-20T11:28:19.490787+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:27:20.206896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-20T11:28:21.597697+0000","last_change":"2024-04-20T11:28:18.484711+0000","last_active":"2024-04-20T11:28:21.597697+0000","last_peered":"2024-04-20T11:28:21.597697+0000","last_clean":"2024-04-20T11:28:21.597697+0000","last_became_active":"2024-04-20T11:28:18.484440+0000","last_became_peered":"2024-04-20T11:28:18.484440+0000","last_unstale":"2024-04-20T11:28:21.597697+0000","last_undegraded":"2024-04-20T11:28:21.597697+0000","last_fullsized":"2024-04-20T11:28:21.597697+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:17:00.895742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-20T11:28:21.598787+0000","last_change":"2024-04-20T11:28:18.494273+0000","last_active":"2024-04-20T11:28:21.598787+0000","last_peered":"2024-04-20T11:28:21.598787+0000","last_clean":"2024-04-20T11:28:21.598787+0000","last_became_active":"2024-04-20T11:28:18.493859+0000","last_became_peered":"2024-04-20T11:28:18.493859+0000","last_unstale":"2024-04-20T11:28:21.598787+0000","last_undegraded":"2024-04-20T11:28:21.598787+0000","last_fullsized":"2024-04-20T11:28:21.598787+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:37:43.418204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-20T11:28:21.599232+0000","last_change":"2024-04-20T11:28:18.491858+0000","last_active":"2024-04-20T11:28:21.599232+0000","last_peered":"2024-04-20T11:28:21.599232+0000","last_clean":"2024-04-20T11:28:21.599232+0000","last_became_active":"2024-04-20T11:28:18.491609+0000","last_became_peered":"2024-04-20T11:28:18.491609+0000","last_unstale":"2024-04-20T11:28:21.599232+0000","last_undegraded":"2024-04-20T11:28:21.599232+0000","last_fullsized":"2024-04-20T11:28:21.599232+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:05:22.570704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-20T11:28:19.489365+0000","last_change":"2024-04-20T11:28:18.483996+0000","last_active":"2024-04-20T11:28:19.489365+0000","last_peered":"2024-04-20T11:28:19.489365+0000","last_clean":"2024-04-20T11:28:19.489365+0000","last_became_active":"2024-04-20T11:28:18.483660+0000","last_became_peered":"2024-04-20T11:28:18.483660+0000","last_unstale":"2024-04-20T11:28:19.489365+0000","last_undegraded":"2024-04-20T11:28:19.489365+0000","last_fullsized":"2024-04-20T11:28:19.489365+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:25:00.360993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-20T11:28:20.901845+0000","last_change":"2024-04-20T11:28:18.483183+0000","last_active":"2024-04-20T11:28:20.901845+0000","last_peered":"2024-04-20T11:28:20.901845+0000","last_clean":"2024-04-20T11:28:20.901845+0000","last_became_active":"2024-04-20T11:28:18.482868+0000","last_became_peered":"2024-04-20T11:28:18.482868+0000","last_unstale":"2024-04-20T11:28:20.901845+0000","last_undegraded":"2024-04-20T11:28:20.901845+0000","last_fullsized":"2024-04-20T11:28:20.901845+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:19:40.087008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-20T11:28:21.596822+0000","last_change":"2024-04-20T11:28:18.480113+0000","last_active":"2024-04-20T11:28:21.596822+0000","last_peered":"2024-04-20T11:28:21.596822+0000","last_clean":"2024-04-20T11:28:21.596822+0000","last_became_active":"2024-04-20T11:28:18.479824+0000","last_became_peered":"2024-04-20T11:28:18.479824+0000","last_unstale":"2024-04-20T11:28:21.596822+0000","last_undegraded":"2024-04-20T11:28:21.596822+0000","last_fullsized":"2024-04-20T11:28:21.596822+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:37:50.794752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-20T11:28:19.491130+0000","last_change":"2024-04-20T11:28:18.483258+0000","last_active":"2024-04-20T11:28:19.491130+0000","last_peered":"2024-04-20T11:28:19.491130+0000","last_clean":"2024-04-20T11:28:19.491130+0000","last_became_active":"2024-04-20T11:28:18.482465+0000","last_became_peered":"2024-04-20T11:28:18.482465+0000","last_unstale":"2024-04-20T11:28:19.491130+0000","last_undegraded":"2024-04-20T11:28:19.491130+0000","last_fullsized":"2024-04-20T11:28:19.491130+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:33:12.013287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-20T11:28:21.596519+0000","last_change":"2024-04-20T11:28:18.502368+0000","last_active":"2024-04-20T11:28:21.596519+0000","last_peered":"2024-04-20T11:28:21.596519+0000","last_clean":"2024-04-20T11:28:21.596519+0000","last_became_active":"2024-04-20T11:28:18.498578+0000","last_became_peered":"2024-04-20T11:28:18.498578+0000","last_unstale":"2024-04-20T11:28:21.596519+0000","last_undegraded":"2024-04-20T11:28:21.596519+0000","last_fullsized":"2024-04-20T11:28:21.596519+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:00:28.260369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-20T11:28:20.902217+0000","last_change":"2024-04-20T11:28:18.488948+0000","last_active":"2024-04-20T11:28:20.902217+0000","last_peered":"2024-04-20T11:28:20.902217+0000","last_clean":"2024-04-20T11:28:20.902217+0000","last_became_active":"2024-04-20T11:28:18.488681+0000","last_became_peered":"2024-04-20T11:28:18.488681+0000","last_unstale":"2024-04-20T11:28:20.902217+0000","last_undegraded":"2024-04-20T11:28:20.902217+0000","last_fullsized":"2024-04-20T11:28:20.902217+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:13:27.234211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-20T11:28:21.599481+0000","last_change":"2024-04-20T11:28:18.492041+0000","last_active":"2024-04-20T11:28:21.599481+0000","last_peered":"2024-04-20T11:28:21.599481+0000","last_clean":"2024-04-20T11:28:21.599481+0000","last_became_active":"2024-04-20T11:28:18.491755+0000","last_became_peered":"2024-04-20T11:28:18.491755+0000","last_unstale":"2024-04-20T11:28:21.599481+0000","last_undegraded":"2024-04-20T11:28:21.599481+0000","last_fullsized":"2024-04-20T11:28:21.599481+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:58:18.415855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-20T11:28:19.489621+0000","last_change":"2024-04-20T11:28:18.490169+0000","last_active":"2024-04-20T11:28:19.489621+0000","last_peered":"2024-04-20T11:28:19.489621+0000","last_clean":"2024-04-20T11:28:19.489621+0000","last_became_active":"2024-04-20T11:28:18.489666+0000","last_became_peered":"2024-04-20T11:28:18.489666+0000","last_unstale":"2024-04-20T11:28:19.489621+0000","last_undegraded":"2024-04-20T11:28:19.489621+0000","last_fullsized":"2024-04-20T11:28:19.489621+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:00:28.260369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-20T11:28:21.598628+0000","last_change":"2024-04-20T11:28:18.476112+0000","last_active":"2024-04-20T11:28:21.598628+0000","last_peered":"2024-04-20T11:28:21.598628+0000","last_clean":"2024-04-20T11:28:21.598628+0000","last_became_active":"2024-04-20T11:28:18.475780+0000","last_became_peered":"2024-04-20T11:28:18.475780+0000","last_unstale":"2024-04-20T11:28:21.598628+0000","last_undegraded":"2024-04-20T11:28:21.598628+0000","last_fullsized":"2024-04-20T11:28:21.598628+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:45:50.276844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-20T11:28:19.489402+0000","last_change":"2024-04-20T11:28:18.500597+0000","last_active":"2024-04-20T11:28:19.489402+0000","last_peered":"2024-04-20T11:28:19.489402+0000","last_clean":"2024-04-20T11:28:19.489402+0000","last_became_active":"2024-04-20T11:28:18.500298+0000","last_became_peered":"2024-04-20T11:28:18.500298+0000","last_unstale":"2024-04-20T11:28:19.489402+0000","last_undegraded":"2024-04-20T11:28:19.489402+0000","last_fullsized":"2024-04-20T11:28:19.489402+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:32:49.367846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-20T11:28:19.490328+0000","last_change":"2024-04-20T11:28:18.488651+0000","last_active":"2024-04-20T11:28:19.490328+0000","last_peered":"2024-04-20T11:28:19.490328+0000","last_clean":"2024-04-20T11:28:19.490328+0000","last_became_active":"2024-04-20T11:28:18.488139+0000","last_became_peered":"2024-04-20T11:28:18.488139+0000","last_unstale":"2024-04-20T11:28:19.490328+0000","last_undegraded":"2024-04-20T11:28:19.490328+0000","last_fullsized":"2024-04-20T11:28:19.490328+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:07:30.183270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"33'4","reported_seq":18,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-20T11:28:21.601077+0000","last_change":"2024-04-20T11:28:17.473786+0000","last_active":"2024-04-20T11:28:21.601077+0000","last_peered":"2024-04-20T11:28:21.601077+0000","last_clean":"2024-04-20T11:28:21.601077+0000","last_became_active":"2024-04-20T11:28:17.473340+0000","last_became_peered":"2024-04-20T11:28:17.473340+0000","last_unstale":"2024-04-20T11:28:21.601077+0000","last_undegraded":"2024-04-20T11:28:21.601077+0000","last_fullsized":"2024-04-20T11:28:21.601077+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_clean_scrub_stamp":"2024-04-20T11:28:16.405655+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:08:17.094490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-20T11:28:21.600957+0000","last_change":"2024-04-20T11:28:18.492905+0000","last_active":"2024-04-20T11:28:21.600957+0000","last_peered":"2024-04-20T11:28:21.600957+0000","last_clean":"2024-04-20T11:28:21.600957+0000","last_became_active":"2024-04-20T11:28:18.492623+0000","last_became_peered":"2024-04-20T11:28:18.492623+0000","last_unstale":"2024-04-20T11:28:21.600957+0000","last_undegraded":"2024-04-20T11:28:21.600957+0000","last_fullsized":"2024-04-20T11:28:21.600957+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:46:27.642047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"33'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-20T11:28:19.490690+0000","last_change":"2024-04-20T11:28:17.471167+0000","last_active":"2024-04-20T11:28:19.490690+0000","last_peered":"2024-04-20T11:28:19.490690+0000","last_clean":"2024-04-20T11:28:19.490690+0000","last_became_active":"2024-04-20T11:28:17.470860+0000","last_became_peered":"2024-04-20T11:28:17.470860+0000","last_unstale":"2024-04-20T11:28:19.490690+0000","last_undegraded":"2024-04-20T11:28:19.490690+0000","last_fullsized":"2024-04-20T11:28:19.490690+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_clean_scrub_stamp":"2024-04-20T11:28:16.405655+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:08:17.094490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-20T11:28:21.600254+0000","last_change":"2024-04-20T11:28:18.492295+0000","last_active":"2024-04-20T11:28:21.600254+0000","last_peered":"2024-04-20T11:28:21.600254+0000","last_clean":"2024-04-20T11:28:21.600254+0000","last_became_active":"2024-04-20T11:28:18.489270+0000","last_became_peered":"2024-04-20T11:28:18.489270+0000","last_unstale":"2024-04-20T11:28:21.600254+0000","last_undegraded":"2024-04-20T11:28:21.600254+0000","last_fullsized":"2024-04-20T11:28:21.600254+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:57:21.575886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-20T11:28:19.490948+0000","last_change":"2024-04-20T11:28:17.458579+0000","last_active":"2024-04-20T11:28:19.490948+0000","last_peered":"2024-04-20T11:28:19.490948+0000","last_clean":"2024-04-20T11:28:19.490948+0000","last_became_active":"2024-04-20T11:28:17.458337+0000","last_became_peered":"2024-04-20T11:28:17.458337+0000","last_unstale":"2024-04-20T11:28:19.490948+0000","last_undegraded":"2024-04-20T11:28:19.490948+0000","last_fullsized":"2024-04-20T11:28:19.490948+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_clean_scrub_stamp":"2024-04-20T11:28:16.405655+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:07:29.165918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-20T11:28:21.596744+0000","last_change":"2024-04-20T11:28:18.503901+0000","last_active":"2024-04-20T11:28:21.596744+0000","last_peered":"2024-04-20T11:28:21.596744+0000","last_clean":"2024-04-20T11:28:21.596744+0000","last_became_active":"2024-04-20T11:28:18.500480+0000","last_became_peered":"2024-04-20T11:28:18.500480+0000","last_unstale":"2024-04-20T11:28:21.596744+0000","last_undegraded":"2024-04-20T11:28:21.596744+0000","last_fullsized":"2024-04-20T11:28:21.596744+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:19:40.087008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-20T11:28:21.597767+0000","last_change":"2024-04-20T11:28:17.459827+0000","last_active":"2024-04-20T11:28:21.597767+0000","last_peered":"2024-04-20T11:28:21.597767+0000","last_clean":"2024-04-20T11:28:21.597767+0000","last_became_active":"2024-04-20T11:28:17.459283+0000","last_became_peered":"2024-04-20T11:28:17.459283+0000","last_unstale":"2024-04-20T11:28:21.597767+0000","last_undegraded":"2024-04-20T11:28:21.597767+0000","last_fullsized":"2024-04-20T11:28:21.597767+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_clean_scrub_stamp":"2024-04-20T11:28:16.405655+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:23:22.095341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-20T11:28:19.490736+0000","last_change":"2024-04-20T11:28:18.500764+0000","last_active":"2024-04-20T11:28:19.490736+0000","last_peered":"2024-04-20T11:28:19.490736+0000","last_clean":"2024-04-20T11:28:19.490736+0000","last_became_active":"2024-04-20T11:28:18.500196+0000","last_became_peered":"2024-04-20T11:28:18.500196+0000","last_unstale":"2024-04-20T11:28:19.490736+0000","last_undegraded":"2024-04-20T11:28:19.490736+0000","last_fullsized":"2024-04-20T11:28:19.490736+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:47:19.448695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"33'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-20T11:28:21.599617+0000","last_change":"2024-04-20T11:28:17.467217+0000","last_active":"2024-04-20T11:28:21.599617+0000","last_peered":"2024-04-20T11:28:21.599617+0000","last_clean":"2024-04-20T11:28:21.599617+0000","last_became_active":"2024-04-20T11:28:17.466894+0000","last_became_peered":"2024-04-20T11:28:17.466894+0000","last_unstale":"2024-04-20T11:28:21.599617+0000","last_undegraded":"2024-04-20T11:28:21.599617+0000","last_fullsized":"2024-04-20T11:28:21.599617+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_clean_scrub_stamp":"2024-04-20T11:28:16.405655+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:16:59.878390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-20T11:28:20.902262+0000","last_change":"2024-04-20T11:28:18.491020+0000","last_active":"2024-04-20T11:28:20.902262+0000","last_peered":"2024-04-20T11:28:20.902262+0000","last_clean":"2024-04-20T11:28:20.902262+0000","last_became_active":"2024-04-20T11:28:18.490700+0000","last_became_peered":"2024-04-20T11:28:18.490700+0000","last_unstale":"2024-04-20T11:28:20.902262+0000","last_undegraded":"2024-04-20T11:28:20.902262+0000","last_fullsized":"2024-04-20T11:28:20.902262+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:47:19.448695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"33'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-20T11:28:21.599388+0000","last_change":"2024-04-20T11:28:17.460876+0000","last_active":"2024-04-20T11:28:21.599388+0000","last_peered":"2024-04-20T11:28:21.599388+0000","last_clean":"2024-04-20T11:28:21.599388+0000","last_became_active":"2024-04-20T11:28:17.460055+0000","last_became_peered":"2024-04-20T11:28:17.460055+0000","last_unstale":"2024-04-20T11:28:21.599388+0000","last_undegraded":"2024-04-20T11:28:21.599388+0000","last_fullsized":"2024-04-20T11:28:21.599388+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_clean_scrub_stamp":"2024-04-20T11:28:16.405655+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:25:28.484907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-20T11:28:21.599030+0000","last_change":"2024-04-20T11:28:18.497819+0000","last_active":"2024-04-20T11:28:21.599030+0000","last_peered":"2024-04-20T11:28:21.599030+0000","last_clean":"2024-04-20T11:28:21.599030+0000","last_became_active":"2024-04-20T11:28:18.497461+0000","last_became_peered":"2024-04-20T11:28:18.497461+0000","last_unstale":"2024-04-20T11:28:21.599030+0000","last_undegraded":"2024-04-20T11:28:21.599030+0000","last_fullsized":"2024-04-20T11:28:21.599030+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:05:34.747812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-20T11:28:19.489621+0000","last_change":"2024-04-20T11:28:17.447612+0000","last_active":"2024-04-20T11:28:19.489621+0000","last_peered":"2024-04-20T11:28:19.489621+0000","last_clean":"2024-04-20T11:28:19.489621+0000","last_became_active":"2024-04-20T11:28:17.447316+0000","last_became_peered":"2024-04-20T11:28:17.447316+0000","last_unstale":"2024-04-20T11:28:19.489621+0000","last_undegraded":"2024-04-20T11:28:19.489621+0000","last_fullsized":"2024-04-20T11:28:19.489621+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_clean_scrub_stamp":"2024-04-20T11:28:16.405655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:24:59.343641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-20T11:28:21.596161+0000","last_change":"2024-04-20T11:28:18.476322+0000","last_active":"2024-04-20T11:28:21.596161+0000","last_peered":"2024-04-20T11:28:21.596161+0000","last_clean":"2024-04-20T11:28:21.596161+0000","last_became_active":"2024-04-20T11:28:18.475759+0000","last_became_peered":"2024-04-20T11:28:18.475759+0000","last_unstale":"2024-04-20T11:28:21.596161+0000","last_undegraded":"2024-04-20T11:28:21.596161+0000","last_fullsized":"2024-04-20T11:28:21.596161+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:32:49.367846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-20T11:28:21.598602+0000","last_change":"2024-04-20T11:28:17.471548+0000","last_active":"2024-04-20T11:28:21.598602+0000","last_peered":"2024-04-20T11:28:21.598602+0000","last_clean":"2024-04-20T11:28:21.598602+0000","last_became_active":"2024-04-20T11:28:17.471103+0000","last_became_peered":"2024-04-20T11:28:17.471103+0000","last_unstale":"2024-04-20T11:28:21.598602+0000","last_undegraded":"2024-04-20T11:28:21.598602+0000","last_fullsized":"2024-04-20T11:28:21.598602+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_clean_scrub_stamp":"2024-04-20T11:28:16.405655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:16:09.369645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-20T11:28:20.902013+0000","last_change":"2024-04-20T11:28:18.484025+0000","last_active":"2024-04-20T11:28:20.902013+0000","last_peered":"2024-04-20T11:28:20.902013+0000","last_clean":"2024-04-20T11:28:20.902013+0000","last_became_active":"2024-04-20T11:28:18.483686+0000","last_became_peered":"2024-04-20T11:28:18.483686+0000","last_unstale":"2024-04-20T11:28:20.902013+0000","last_undegraded":"2024-04-20T11:28:20.902013+0000","last_fullsized":"2024-04-20T11:28:20.902013+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:37:50.794752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-20T11:28:19.491532+0000","last_change":"2024-04-20T11:28:17.469220+0000","last_active":"2024-04-20T11:28:19.491532+0000","last_peered":"2024-04-20T11:28:19.491532+0000","last_clean":"2024-04-20T11:28:19.491532+0000","last_became_active":"2024-04-20T11:28:17.468973+0000","last_became_peered":"2024-04-20T11:28:17.468973+0000","last_unstale":"2024-04-20T11:28:19.491532+0000","last_undegraded":"2024-04-20T11:28:19.491532+0000","last_fullsized":"2024-04-20T11:28:19.491532+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_clean_scrub_stamp":"2024-04-20T11:28:16.405655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:46:26.624695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-20T11:28:19.491489+0000","last_change":"2024-04-20T11:28:18.485294+0000","last_active":"2024-04-20T11:28:19.491489+0000","last_peered":"2024-04-20T11:28:19.491489+0000","last_clean":"2024-04-20T11:28:19.491489+0000","last_became_active":"2024-04-20T11:28:18.485100+0000","last_became_peered":"2024-04-20T11:28:18.485100+0000","last_unstale":"2024-04-20T11:28:19.491489+0000","last_undegraded":"2024-04-20T11:28:19.491489+0000","last_fullsized":"2024-04-20T11:28:19.491489+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:13:12.702123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-20T11:28:19.491125+0000","last_change":"2024-04-20T11:28:17.460210+0000","last_active":"2024-04-20T11:28:19.491125+0000","last_peered":"2024-04-20T11:28:19.491125+0000","last_clean":"2024-04-20T11:28:19.491125+0000","last_became_active":"2024-04-20T11:28:17.459896+0000","last_became_peered":"2024-04-20T11:28:17.459896+0000","last_unstale":"2024-04-20T11:28:19.491125+0000","last_undegraded":"2024-04-20T11:28:19.491125+0000","last_fullsized":"2024-04-20T11:28:19.491125+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_clean_scrub_stamp":"2024-04-20T11:28:16.405655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:25:28.484907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-20T11:28:19.491274+0000","last_change":"2024-04-20T11:28:18.482871+0000","last_active":"2024-04-20T11:28:19.491274+0000","last_peered":"2024-04-20T11:28:19.491274+0000","last_clean":"2024-04-20T11:28:19.491274+0000","last_became_active":"2024-04-20T11:28:18.481015+0000","last_became_peered":"2024-04-20T11:28:18.481015+0000","last_unstale":"2024-04-20T11:28:19.491274+0000","last_undegraded":"2024-04-20T11:28:19.491274+0000","last_fullsized":"2024-04-20T11:28:19.491274+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:27:20.273574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"33'3","reported_seq":17,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-20T11:28:20.901981+0000","last_change":"2024-04-20T11:28:17.469106+0000","last_active":"2024-04-20T11:28:20.901981+0000","last_peered":"2024-04-20T11:28:20.901981+0000","last_clean":"2024-04-20T11:28:20.901981+0000","last_became_active":"2024-04-20T11:28:17.468805+0000","last_became_peered":"2024-04-20T11:28:17.468805+0000","last_unstale":"2024-04-20T11:28:20.901981+0000","last_undegraded":"2024-04-20T11:28:20.901981+0000","last_fullsized":"2024-04-20T11:28:20.901981+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_clean_scrub_stamp":"2024-04-20T11:28:16.405655+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:45:49.259492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-20T11:28:20.901818+0000","last_change":"2024-04-20T11:28:18.476381+0000","last_active":"2024-04-20T11:28:20.901818+0000","last_peered":"2024-04-20T11:28:20.901818+0000","last_clean":"2024-04-20T11:28:20.901818+0000","last_became_active":"2024-04-20T11:28:18.475525+0000","last_became_peered":"2024-04-20T11:28:18.475525+0000","last_unstale":"2024-04-20T11:28:20.901818+0000","last_undegraded":"2024-04-20T11:28:20.901818+0000","last_fullsized":"2024-04-20T11:28:20.901818+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:57:21.575886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"33'6","reported_seq":20,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-20T11:28:19.490855+0000","last_change":"2024-04-20T11:28:17.458801+0000","last_active":"2024-04-20T11:28:19.490855+0000","last_peered":"2024-04-20T11:28:19.490855+0000","last_clean":"2024-04-20T11:28:19.490855+0000","last_became_active":"2024-04-20T11:28:17.458372+0000","last_became_peered":"2024-04-20T11:28:17.458372+0000","last_unstale":"2024-04-20T11:28:19.490855+0000","last_undegraded":"2024-04-20T11:28:19.490855+0000","last_fullsized":"2024-04-20T11:28:19.490855+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_clean_scrub_stamp":"2024-04-20T11:28:16.405655+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:45:49.259492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-20T11:28:21.596016+0000","last_change":"2024-04-20T11:28:18.476283+0000","last_active":"2024-04-20T11:28:21.596016+0000","last_peered":"2024-04-20T11:28:21.596016+0000","last_clean":"2024-04-20T11:28:21.596016+0000","last_became_active":"2024-04-20T11:28:18.475993+0000","last_became_peered":"2024-04-20T11:28:18.475993+0000","last_unstale":"2024-04-20T11:28:21.596016+0000","last_undegraded":"2024-04-20T11:28:21.596016+0000","last_fullsized":"2024-04-20T11:28:21.596016+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:40:34.948408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-20T11:28:21.599743+0000","last_change":"2024-04-20T11:28:17.474268+0000","last_active":"2024-04-20T11:28:21.599743+0000","last_peered":"2024-04-20T11:28:21.599743+0000","last_clean":"2024-04-20T11:28:21.599743+0000","last_became_active":"2024-04-20T11:28:17.473059+0000","last_became_peered":"2024-04-20T11:28:17.473059+0000","last_unstale":"2024-04-20T11:28:21.599743+0000","last_undegraded":"2024-04-20T11:28:21.599743+0000","last_fullsized":"2024-04-20T11:28:21.599743+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_clean_scrub_stamp":"2024-04-20T11:28:16.405655+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:19:13.712754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-20T11:28:19.490507+0000","last_change":"2024-04-20T11:28:18.480363+0000","last_active":"2024-04-20T11:28:19.490507+0000","last_peered":"2024-04-20T11:28:19.490507+0000","last_clean":"2024-04-20T11:28:19.490507+0000","last_became_active":"2024-04-20T11:28:18.480064+0000","last_became_peered":"2024-04-20T11:28:18.480064+0000","last_unstale":"2024-04-20T11:28:19.490507+0000","last_undegraded":"2024-04-20T11:28:19.490507+0000","last_fullsized":"2024-04-20T11:28:19.490507+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:47:19.448695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-20T11:28:21.596079+0000","last_change":"2024-04-20T11:28:17.464185+0000","last_active":"2024-04-20T11:28:21.596079+0000","last_peered":"2024-04-20T11:28:21.596079+0000","last_clean":"2024-04-20T11:28:21.596079+0000","last_became_active":"2024-04-20T11:28:17.463919+0000","last_became_peered":"2024-04-20T11:28:17.463919+0000","last_unstale":"2024-04-20T11:28:21.596079+0000","last_undegraded":"2024-04-20T11:28:21.596079+0000","last_fullsized":"2024-04-20T11:28:21.596079+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_clean_scrub_stamp":"2024-04-20T11:28:16.405655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:16:09.369645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-20T11:28:19.490256+0000","last_change":"2024-04-20T11:28:18.506282+0000","last_active":"2024-04-20T11:28:19.490256+0000","last_peered":"2024-04-20T11:28:19.490256+0000","last_clean":"2024-04-20T11:28:19.490256+0000","last_became_active":"2024-04-20T11:28:18.505942+0000","last_became_peered":"2024-04-20T11:28:18.505942+0000","last_unstale":"2024-04-20T11:28:19.490256+0000","last_undegraded":"2024-04-20T11:28:19.490256+0000","last_fullsized":"2024-04-20T11:28:19.490256+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:19:40.087008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-20T11:28:19.489347+0000","last_change":"2024-04-20T11:28:17.469383+0000","last_active":"2024-04-20T11:28:19.489347+0000","last_peered":"2024-04-20T11:28:19.489347+0000","last_clean":"2024-04-20T11:28:19.489347+0000","last_became_active":"2024-04-20T11:28:17.469064+0000","last_became_peered":"2024-04-20T11:28:17.469064+0000","last_unstale":"2024-04-20T11:28:19.489347+0000","last_undegraded":"2024-04-20T11:28:19.489347+0000","last_fullsized":"2024-04-20T11:28:19.489347+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_clean_scrub_stamp":"2024-04-20T11:28:16.405655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:54:04.385485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-20T11:28:21.598904+0000","last_change":"2024-04-20T11:28:18.494971+0000","last_active":"2024-04-20T11:28:21.598904+0000","last_peered":"2024-04-20T11:28:21.598904+0000","last_clean":"2024-04-20T11:28:21.598904+0000","last_became_active":"2024-04-20T11:28:18.492647+0000","last_became_peered":"2024-04-20T11:28:18.492647+0000","last_unstale":"2024-04-20T11:28:21.598904+0000","last_undegraded":"2024-04-20T11:28:21.598904+0000","last_fullsized":"2024-04-20T11:28:21.598904+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:54:05.402837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"33'8","reported_seq":22,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-20T11:28:19.488971+0000","last_change":"2024-04-20T11:28:17.456725+0000","last_active":"2024-04-20T11:28:19.488971+0000","last_peered":"2024-04-20T11:28:19.488971+0000","last_clean":"2024-04-20T11:28:19.488971+0000","last_became_active":"2024-04-20T11:28:17.456423+0000","last_became_peered":"2024-04-20T11:28:17.456423+0000","last_unstale":"2024-04-20T11:28:19.488971+0000","last_undegraded":"2024-04-20T11:28:19.488971+0000","last_fullsized":"2024-04-20T11:28:19.488971+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_clean_scrub_stamp":"2024-04-20T11:28:16.405655+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:39:15.847353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-20T11:28:21.599065+0000","last_change":"2024-04-20T11:28:18.498665+0000","last_active":"2024-04-20T11:28:21.599065+0000","last_peered":"2024-04-20T11:28:21.599065+0000","last_clean":"2024-04-20T11:28:21.599065+0000","last_became_active":"2024-04-20T11:28:18.498339+0000","last_became_peered":"2024-04-20T11:28:18.498339+0000","last_unstale":"2024-04-20T11:28:21.599065+0000","last_undegraded":"2024-04-20T11:28:21.599065+0000","last_fullsized":"2024-04-20T11:28:21.599065+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:40:34.948408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"33'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-20T11:28:21.596993+0000","last_change":"2024-04-20T11:28:17.459859+0000","last_active":"2024-04-20T11:28:21.596993+0000","last_peered":"2024-04-20T11:28:21.596993+0000","last_clean":"2024-04-20T11:28:21.596993+0000","last_became_active":"2024-04-20T11:28:17.459584+0000","last_became_peered":"2024-04-20T11:28:17.459584+0000","last_unstale":"2024-04-20T11:28:21.596993+0000","last_undegraded":"2024-04-20T11:28:21.596993+0000","last_fullsized":"2024-04-20T11:28:21.596993+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_clean_scrub_stamp":"2024-04-20T11:28:16.405655+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:19:13.712754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-20T11:28:20.902461+0000","last_change":"2024-04-20T11:28:18.493649+0000","last_active":"2024-04-20T11:28:20.902461+0000","last_peered":"2024-04-20T11:28:20.902461+0000","last_clean":"2024-04-20T11:28:20.902461+0000","last_became_active":"2024-04-20T11:28:18.493365+0000","last_became_peered":"2024-04-20T11:28:18.493365+0000","last_unstale":"2024-04-20T11:28:20.902461+0000","last_undegraded":"2024-04-20T11:28:20.902461+0000","last_fullsized":"2024-04-20T11:28:20.902461+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:51:25.124854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-20T11:28:21.598885+0000","last_change":"2024-04-20T11:28:17.470381+0000","last_active":"2024-04-20T11:28:21.598885+0000","last_peered":"2024-04-20T11:28:21.598885+0000","last_clean":"2024-04-20T11:28:21.598885+0000","last_became_active":"2024-04-20T11:28:17.470007+0000","last_became_peered":"2024-04-20T11:28:17.470007+0000","last_unstale":"2024-04-20T11:28:21.598885+0000","last_undegraded":"2024-04-20T11:28:21.598885+0000","last_fullsized":"2024-04-20T11:28:21.598885+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_clean_scrub_stamp":"2024-04-20T11:28:16.405655+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:08:17.094490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-20T11:28:21.596323+0000","last_change":"2024-04-20T11:28:18.498248+0000","last_active":"2024-04-20T11:28:21.596323+0000","last_peered":"2024-04-20T11:28:21.596323+0000","last_clean":"2024-04-20T11:28:21.596323+0000","last_became_active":"2024-04-20T11:28:18.497993+0000","last_became_peered":"2024-04-20T11:28:18.497993+0000","last_unstale":"2024-04-20T11:28:21.596323+0000","last_undegraded":"2024-04-20T11:28:21.596323+0000","last_fullsized":"2024-04-20T11:28:21.596323+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:58:18.415855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"33'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-20T11:28:21.600064+0000","last_change":"2024-04-20T11:28:17.465387+0000","last_active":"2024-04-20T11:28:21.600064+0000","last_peered":"2024-04-20T11:28:21.600064+0000","last_clean":"2024-04-20T11:28:21.600064+0000","last_became_active":"2024-04-20T11:28:17.464941+0000","last_became_peered":"2024-04-20T11:28:17.464941+0000","last_unstale":"2024-04-20T11:28:21.600064+0000","last_undegraded":"2024-04-20T11:28:21.600064+0000","last_fullsized":"2024-04-20T11:28:21.600064+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_clean_scrub_stamp":"2024-04-20T11:28:16.405655+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:41:32.390024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-20T11:28:21.596806+0000","last_change":"2024-04-20T11:28:18.482744+0000","last_active":"2024-04-20T11:28:21.596806+0000","last_peered":"2024-04-20T11:28:21.596806+0000","last_clean":"2024-04-20T11:28:21.596806+0000","last_became_active":"2024-04-20T11:28:18.482426+0000","last_became_peered":"2024-04-20T11:28:18.482426+0000","last_unstale":"2024-04-20T11:28:21.596806+0000","last_undegraded":"2024-04-20T11:28:21.596806+0000","last_fullsized":"2024-04-20T11:28:21.596806+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:41:33.407376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-20T11:28:21.600380+0000","last_change":"2024-04-20T11:28:17.469465+0000","last_active":"2024-04-20T11:28:21.600380+0000","last_peered":"2024-04-20T11:28:21.600380+0000","last_clean":"2024-04-20T11:28:21.600380+0000","last_became_active":"2024-04-20T11:28:17.466875+0000","last_became_peered":"2024-04-20T11:28:17.466875+0000","last_unstale":"2024-04-20T11:28:21.600380+0000","last_undegraded":"2024-04-20T11:28:21.600380+0000","last_fullsized":"2024-04-20T11:28:21.600380+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_clean_scrub_stamp":"2024-04-20T11:28:16.405655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:38:05.072757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-20T11:28:21.596752+0000","last_change":"2024-04-20T11:28:18.498337+0000","last_active":"2024-04-20T11:28:21.596752+0000","last_peered":"2024-04-20T11:28:21.596752+0000","last_clean":"2024-04-20T11:28:21.596752+0000","last_became_active":"2024-04-20T11:28:18.498077+0000","last_became_peered":"2024-04-20T11:28:18.498077+0000","last_unstale":"2024-04-20T11:28:21.596752+0000","last_undegraded":"2024-04-20T11:28:21.596752+0000","last_fullsized":"2024-04-20T11:28:21.596752+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:34:09.229419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-20T11:28:19.490528+0000","last_change":"2024-04-20T11:28:17.462552+0000","last_active":"2024-04-20T11:28:19.490528+0000","last_peered":"2024-04-20T11:28:19.490528+0000","last_clean":"2024-04-20T11:28:19.490528+0000","last_became_active":"2024-04-20T11:28:17.461747+0000","last_became_peered":"2024-04-20T11:28:17.461747+0000","last_unstale":"2024-04-20T11:28:19.490528+0000","last_undegraded":"2024-04-20T11:28:19.490528+0000","last_fullsized":"2024-04-20T11:28:19.490528+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_clean_scrub_stamp":"2024-04-20T11:28:16.405655+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:41:32.390024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-20T11:28:21.597369+0000","last_change":"2024-04-20T11:28:18.494703+0000","last_active":"2024-04-20T11:28:21.597369+0000","last_peered":"2024-04-20T11:28:21.597369+0000","last_clean":"2024-04-20T11:28:21.597369+0000","last_became_active":"2024-04-20T11:28:18.492955+0000","last_became_peered":"2024-04-20T11:28:18.492955+0000","last_unstale":"2024-04-20T11:28:21.597369+0000","last_undegraded":"2024-04-20T11:28:21.597369+0000","last_fullsized":"2024-04-20T11:28:21.597369+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:25:29.502259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-20T11:28:21.599661+0000","last_change":"2024-04-20T11:28:17.466476+0000","last_active":"2024-04-20T11:28:21.599661+0000","last_peered":"2024-04-20T11:28:21.599661+0000","last_clean":"2024-04-20T11:28:21.599661+0000","last_became_active":"2024-04-20T11:28:17.465342+0000","last_became_peered":"2024-04-20T11:28:17.465342+0000","last_unstale":"2024-04-20T11:28:21.599661+0000","last_undegraded":"2024-04-20T11:28:21.599661+0000","last_fullsized":"2024-04-20T11:28:21.599661+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_clean_scrub_stamp":"2024-04-20T11:28:16.405655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:58:28.563452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-20T11:28:21.597011+0000","last_change":"2024-04-20T11:28:18.483123+0000","last_active":"2024-04-20T11:28:21.597011+0000","last_peered":"2024-04-20T11:28:21.597011+0000","last_clean":"2024-04-20T11:28:21.597011+0000","last_became_active":"2024-04-20T11:28:18.482919+0000","last_became_peered":"2024-04-20T11:28:18.482919+0000","last_unstale":"2024-04-20T11:28:21.597011+0000","last_undegraded":"2024-04-20T11:28:21.597011+0000","last_fullsized":"2024-04-20T11:28:21.597011+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:58:29.580804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-20T11:28:21.599251+0000","last_change":"2024-04-20T11:28:17.466324+0000","last_active":"2024-04-20T11:28:21.599251+0000","last_peered":"2024-04-20T11:28:21.599251+0000","last_clean":"2024-04-20T11:28:21.599251+0000","last_became_active":"2024-04-20T11:28:17.465729+0000","last_became_peered":"2024-04-20T11:28:17.465729+0000","last_unstale":"2024-04-20T11:28:21.599251+0000","last_undegraded":"2024-04-20T11:28:21.599251+0000","last_fullsized":"2024-04-20T11:28:21.599251+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_clean_scrub_stamp":"2024-04-20T11:28:16.405655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:34:08.212067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-20T11:28:19.489560+0000","last_change":"2024-04-20T11:28:18.490481+0000","last_active":"2024-04-20T11:28:19.489560+0000","last_peered":"2024-04-20T11:28:19.489560+0000","last_clean":"2024-04-20T11:28:19.489560+0000","last_became_active":"2024-04-20T11:28:18.489774+0000","last_became_peered":"2024-04-20T11:28:18.489774+0000","last_unstale":"2024-04-20T11:28:19.489560+0000","last_undegraded":"2024-04-20T11:28:19.489560+0000","last_fullsized":"2024-04-20T11:28:19.489560+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:58:18.415855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-20T11:28:20.901092+0000","last_change":"2024-04-20T11:28:17.471682+0000","last_active":"2024-04-20T11:28:20.901092+0000","last_peered":"2024-04-20T11:28:20.901092+0000","last_clean":"2024-04-20T11:28:20.901092+0000","last_became_active":"2024-04-20T11:28:17.470812+0000","last_became_peered":"2024-04-20T11:28:17.470812+0000","last_unstale":"2024-04-20T11:28:20.901092+0000","last_undegraded":"2024-04-20T11:28:20.901092+0000","last_fullsized":"2024-04-20T11:28:20.901092+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_clean_scrub_stamp":"2024-04-20T11:28:16.405655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:39:15.847353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-20T11:28:21.597966+0000","last_change":"2024-04-20T11:28:18.489069+0000","last_active":"2024-04-20T11:28:21.597966+0000","last_peered":"2024-04-20T11:28:21.597966+0000","last_clean":"2024-04-20T11:28:21.597966+0000","last_became_active":"2024-04-20T11:28:18.488767+0000","last_became_peered":"2024-04-20T11:28:18.488767+0000","last_unstale":"2024-04-20T11:28:21.597966+0000","last_undegraded":"2024-04-20T11:28:21.597966+0000","last_fullsized":"2024-04-20T11:28:21.597966+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:19:14.730106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-20T11:28:21.597079+0000","last_change":"2024-04-20T11:28:17.473043+0000","last_active":"2024-04-20T11:28:21.597079+0000","last_peered":"2024-04-20T11:28:21.597079+0000","last_clean":"2024-04-20T11:28:21.597079+0000","last_became_active":"2024-04-20T11:28:17.472795+0000","last_became_peered":"2024-04-20T11:28:17.472795+0000","last_unstale":"2024-04-20T11:28:21.597079+0000","last_undegraded":"2024-04-20T11:28:21.597079+0000","last_fullsized":"2024-04-20T11:28:21.597079+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_clean_scrub_stamp":"2024-04-20T11:28:16.405655+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:39:15.847353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-20T11:28:20.902599+0000","last_change":"2024-04-20T11:28:18.492680+0000","last_active":"2024-04-20T11:28:20.902599+0000","last_peered":"2024-04-20T11:28:20.902599+0000","last_clean":"2024-04-20T11:28:20.902599+0000","last_became_active":"2024-04-20T11:28:18.491322+0000","last_became_peered":"2024-04-20T11:28:18.491322+0000","last_unstale":"2024-04-20T11:28:20.902599+0000","last_undegraded":"2024-04-20T11:28:20.902599+0000","last_fullsized":"2024-04-20T11:28:20.902599+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:57:08.115797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"33'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-20T11:28:19.490966+0000","last_change":"2024-04-20T11:28:17.462763+0000","last_active":"2024-04-20T11:28:19.490966+0000","last_peered":"2024-04-20T11:28:19.490966+0000","last_clean":"2024-04-20T11:28:19.490966+0000","last_became_active":"2024-04-20T11:28:17.462444+0000","last_became_peered":"2024-04-20T11:28:17.462444+0000","last_unstale":"2024-04-20T11:28:19.490966+0000","last_undegraded":"2024-04-20T11:28:19.490966+0000","last_fullsized":"2024-04-20T11:28:19.490966+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_clean_scrub_stamp":"2024-04-20T11:28:16.405655+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:16:09.369645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-20T11:28:19.490235+0000","last_change":"2024-04-20T11:28:18.492298+0000","last_active":"2024-04-20T11:28:19.490235+0000","last_peered":"2024-04-20T11:28:19.490235+0000","last_clean":"2024-04-20T11:28:19.490235+0000","last_became_active":"2024-04-20T11:28:18.492117+0000","last_became_peered":"2024-04-20T11:28:18.492117+0000","last_unstale":"2024-04-20T11:28:19.490235+0000","last_undegraded":"2024-04-20T11:28:19.490235+0000","last_fullsized":"2024-04-20T11:28:19.490235+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:46:27.642047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-20T11:28:21.596578+0000","last_change":"2024-04-20T11:28:17.461411+0000","last_active":"2024-04-20T11:28:21.596578+0000","last_peered":"2024-04-20T11:28:21.596578+0000","last_clean":"2024-04-20T11:28:21.596578+0000","last_became_active":"2024-04-20T11:28:17.460833+0000","last_became_peered":"2024-04-20T11:28:17.460833+0000","last_unstale":"2024-04-20T11:28:21.596578+0000","last_undegraded":"2024-04-20T11:28:21.596578+0000","last_fullsized":"2024-04-20T11:28:21.596578+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_clean_scrub_stamp":"2024-04-20T11:28:16.405655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:16:59.878390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-20T11:28:21.599538+0000","last_change":"2024-04-20T11:28:18.491637+0000","last_active":"2024-04-20T11:28:21.599538+0000","last_peered":"2024-04-20T11:28:21.599538+0000","last_clean":"2024-04-20T11:28:21.599538+0000","last_became_active":"2024-04-20T11:28:18.491332+0000","last_became_peered":"2024-04-20T11:28:18.491332+0000","last_unstale":"2024-04-20T11:28:21.599538+0000","last_undegraded":"2024-04-20T11:28:21.599538+0000","last_fullsized":"2024-04-20T11:28:21.599538+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:25:00.360993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-20T11:28:19.490704+0000","last_change":"2024-04-20T11:28:17.474177+0000","last_active":"2024-04-20T11:28:19.490704+0000","last_peered":"2024-04-20T11:28:19.490704+0000","last_clean":"2024-04-20T11:28:19.490704+0000","last_became_active":"2024-04-20T11:28:17.473885+0000","last_became_peered":"2024-04-20T11:28:17.473885+0000","last_unstale":"2024-04-20T11:28:19.490704+0000","last_undegraded":"2024-04-20T11:28:19.490704+0000","last_fullsized":"2024-04-20T11:28:19.490704+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_clean_scrub_stamp":"2024-04-20T11:28:16.405655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:58:28.563452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-20T11:28:19.490491+0000","last_change":"2024-04-20T11:28:18.489302+0000","last_active":"2024-04-20T11:28:19.490491+0000","last_peered":"2024-04-20T11:28:19.490491+0000","last_clean":"2024-04-20T11:28:19.490491+0000","last_became_active":"2024-04-20T11:28:18.489037+0000","last_became_peered":"2024-04-20T11:28:18.489037+0000","last_unstale":"2024-04-20T11:28:19.490491+0000","last_undegraded":"2024-04-20T11:28:19.490491+0000","last_fullsized":"2024-04-20T11:28:19.490491+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:26:59.956402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-20T11:28:21.596869+0000","last_change":"2024-04-20T11:28:17.463952+0000","last_active":"2024-04-20T11:28:21.596869+0000","last_peered":"2024-04-20T11:28:21.596869+0000","last_clean":"2024-04-20T11:28:21.596869+0000","last_became_active":"2024-04-20T11:28:17.463388+0000","last_became_peered":"2024-04-20T11:28:17.463388+0000","last_unstale":"2024-04-20T11:28:21.596869+0000","last_undegraded":"2024-04-20T11:28:21.596869+0000","last_fullsized":"2024-04-20T11:28:21.596869+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_clean_scrub_stamp":"2024-04-20T11:28:16.405655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:25:28.484907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-20T11:28:19.488977+0000","last_change":"2024-04-20T11:28:18.479286+0000","last_active":"2024-04-20T11:28:19.488977+0000","last_peered":"2024-04-20T11:28:19.488977+0000","last_clean":"2024-04-20T11:28:19.488977+0000","last_became_active":"2024-04-20T11:28:18.478988+0000","last_became_peered":"2024-04-20T11:28:18.478988+0000","last_unstale":"2024-04-20T11:28:19.488977+0000","last_undegraded":"2024-04-20T11:28:19.488977+0000","last_fullsized":"2024-04-20T11:28:19.488977+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:37:50.794752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-20T11:28:21.600225+0000","last_change":"2024-04-20T11:28:17.473736+0000","last_active":"2024-04-20T11:28:21.600225+0000","last_peered":"2024-04-20T11:28:21.600225+0000","last_clean":"2024-04-20T11:28:21.600225+0000","last_became_active":"2024-04-20T11:28:17.473455+0000","last_became_peered":"2024-04-20T11:28:17.473455+0000","last_unstale":"2024-04-20T11:28:21.600225+0000","last_undegraded":"2024-04-20T11:28:21.600225+0000","last_fullsized":"2024-04-20T11:28:21.600225+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_clean_scrub_stamp":"2024-04-20T11:28:16.405655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:58:28.563452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-20T11:28:21.599424+0000","last_change":"2024-04-20T11:28:18.489894+0000","last_active":"2024-04-20T11:28:21.599424+0000","last_peered":"2024-04-20T11:28:21.599424+0000","last_clean":"2024-04-20T11:28:21.599424+0000","last_became_active":"2024-04-20T11:28:18.488250+0000","last_became_peered":"2024-04-20T11:28:18.488250+0000","last_unstale":"2024-04-20T11:28:21.599424+0000","last_undegraded":"2024-04-20T11:28:21.599424+0000","last_fullsized":"2024-04-20T11:28:21.599424+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:32:49.367846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-20T11:28:19.489032+0000","last_change":"2024-04-20T11:28:17.468622+0000","last_active":"2024-04-20T11:28:19.489032+0000","last_peered":"2024-04-20T11:28:19.489032+0000","last_clean":"2024-04-20T11:28:19.489032+0000","last_became_active":"2024-04-20T11:28:17.468287+0000","last_became_peered":"2024-04-20T11:28:17.468287+0000","last_unstale":"2024-04-20T11:28:19.489032+0000","last_undegraded":"2024-04-20T11:28:19.489032+0000","last_fullsized":"2024-04-20T11:28:19.489032+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_clean_scrub_stamp":"2024-04-20T11:28:16.405655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:26:58.939050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-20T11:28:19.490832+0000","last_change":"2024-04-20T11:28:18.495719+0000","last_active":"2024-04-20T11:28:19.490832+0000","last_peered":"2024-04-20T11:28:19.490832+0000","last_clean":"2024-04-20T11:28:19.490832+0000","last_became_active":"2024-04-20T11:28:18.495440+0000","last_became_peered":"2024-04-20T11:28:18.495440+0000","last_unstale":"2024-04-20T11:28:19.490832+0000","last_undegraded":"2024-04-20T11:28:19.490832+0000","last_fullsized":"2024-04-20T11:28:19.490832+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:17.423007+0000","last_clean_scrub_stamp":"2024-04-20T11:28:17.423007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:27:20.206896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-20T11:28:21.599928+0000","last_change":"2024-04-20T11:28:17.469380+0000","last_active":"2024-04-20T11:28:21.599928+0000","last_peered":"2024-04-20T11:28:21.599928+0000","last_clean":"2024-04-20T11:28:21.599928+0000","last_became_active":"2024-04-20T11:28:17.468984+0000","last_became_peered":"2024-04-20T11:28:17.468984+0000","last_unstale":"2024-04-20T11:28:21.599928+0000","last_undegraded":"2024-04-20T11:28:21.599928+0000","last_fullsized":"2024-04-20T11:28:21.599928+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:16.405655+0000","last_clean_scrub_stamp":"2024-04-20T11:28:16.405655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:41:32.390024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":195,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-20T11:28:19.490402+0000","last_change":"2024-04-20T11:26:48.661897+0000","last_active":"2024-04-20T11:28:19.490402+0000","last_peered":"2024-04-20T11:28:19.490402+0000","last_clean":"2024-04-20T11:28:19.490402+0000","last_became_active":"2024-04-20T11:26:48.656535+0000","last_became_peered":"2024-04-20T11:26:48.656535+0000","last_unstale":"2024-04-20T11:28:19.490402+0000","last_undegraded":"2024-04-20T11:28:19.490402+0000","last_fullsized":"2024-04-20T11:28:19.490402+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:47.543004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:47.543004+0000","last_clean_scrub_stamp":"2024-04-20T11:26:47.543004+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:58:35.527105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574872,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27984,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343856,"statfs":{"total":96636764160,"available":96608108544,"internally_reserved":0,"allocated":983040,"data_stored":743099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574869,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27404,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344436,"statfs":{"total":96636764160,"available":96608702464,"internally_reserved":0,"allocated":393216,"data_stored":153257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574871,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":393216,"data_stored":153313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574869,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":380928,"data_stored":146445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574870,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27416,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344424,"statfs":{"total":96636764160,"available":96608690176,"internally_reserved":0,"allocated":401408,"data_stored":154275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574871,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":389120,"data_stored":152711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574869,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":389120,"data_stored":147975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574870,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343860,"statfs":{"total":96636764160,"available":96608112640,"internally_reserved":0,"allocated":978944,"data_stored":743009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T11:28:23.435 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-20T11:28:23.435 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-20T11:28:23.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.684+0000 7f049e600640 1 Processor -- start 2024-04-20T11:28:23.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.684+0000 7f049e600640 1 -- start start 2024-04-20T11:28:23.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.685+0000 7f049e600640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f049815c7f0 0x7f0498072b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:23.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.686+0000 7f049e600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f04980b4870 0x7f04980ad2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:23.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.686+0000 7f049e600640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04980ce390 con 0x7f0498073b60 2024-04-20T11:28:23.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.686+0000 7f049e600640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f0498073ff0 con 0x7f049815c7f0 2024-04-20T11:28:23.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.686+0000 7f049e600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f04980ce140 con 0x7f04980b4870 2024-04-20T11:28:23.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.686+0000 7f049dc00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f04980b4870 0x7f04980ad2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:23.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.686+0000 7f049dc00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f04980b4870 0x7f04980ad2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.152:59758/0 (socket says 172.21.15.152:59758) 2024-04-20T11:28:23.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.686+0000 7f049dc00640 1 -- 172.21.15.152:0/1867340845 learned_addr learned my addr 172.21.15.152:0/1867340845 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:23.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.687+0000 7f049d200640 1 --2- 172.21.15.152:0/1867340845 >> v2:172.21.15.175:3300/0 conn(0x7f049815c7f0 0x7f0498072b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:23.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.687+0000 7f049dc00640 1 -- 172.21.15.152:0/1867340845 >> v2:172.21.15.175:3300/0 conn(0x7f049815c7f0 msgr2=0x7f0498072b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:23.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.687+0000 7f049dc00640 1 --2- 172.21.15.152:0/1867340845 >> v2:172.21.15.175:3300/0 conn(0x7f049815c7f0 0x7f0498072b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:23.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.687+0000 7f049dc00640 1 -- 172.21.15.152:0/1867340845 >> v1:172.21.15.175:6789/0 conn(0x7f0498073b60 legacy=0x7f04980b4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:23.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.687+0000 7f049dc00640 1 -- 172.21.15.152:0/1867340845 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04980ce670 con 0x7f04980b4870 2024-04-20T11:28:23.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.687+0000 7f049d200640 1 --2- 172.21.15.152:0/1867340845 >> v2:172.21.15.175:3300/0 conn(0x7f049815c7f0 0x7f0498072b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:28:23.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.689+0000 7f049dc00640 1 --2- 172.21.15.152:0/1867340845 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f04980b4870 0x7f04980ad2b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0490009e50 tx=0x7f049002bee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3a3e49bcb5eae8d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:23.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.689+0000 7f0497400640 1 -- 172.21.15.152:0/1867340845 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f049002ca80 con 0x7f04980b4870 2024-04-20T11:28:23.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.689+0000 7f0497400640 1 -- 172.21.15.152:0/1867340845 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f049002cc20 con 0x7f04980b4870 2024-04-20T11:28:23.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.689+0000 7f0497400640 1 -- 172.21.15.152:0/1867340845 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04900343f0 con 0x7f04980b4870 2024-04-20T11:28:23.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.690+0000 7f049e600640 1 -- 172.21.15.152:0/1867340845 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f04980b4870 msgr2=0x7f04980ad2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:23.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.690+0000 7f049e600640 1 --2- 172.21.15.152:0/1867340845 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f04980b4870 0x7f04980ad2b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0490009e50 tx=0x7f049002bee0 comp rx=0 tx=0).stop 2024-04-20T11:28:23.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.690+0000 7f049e600640 1 -- 172.21.15.152:0/1867340845 shutdown_connections 2024-04-20T11:28:23.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.690+0000 7f049e600640 1 --2- 172.21.15.152:0/1867340845 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f04980b4870 0x7f04980ad2b0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:23.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.690+0000 7f049e600640 1 --2- 172.21.15.152:0/1867340845 >> v2:172.21.15.175:3300/0 conn(0x7f049815c7f0 0x7f0498072b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:23.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.690+0000 7f049e600640 1 -- 172.21.15.152:0/1867340845 >> 172.21.15.152:0/1867340845 conn(0x7f04980d0260 msgr2=0x7f04980d0670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:23.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.690+0000 7f049e600640 1 -- 172.21.15.152:0/1867340845 shutdown_connections 2024-04-20T11:28:23.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.690+0000 7f049e600640 1 -- 172.21.15.152:0/1867340845 wait complete. 2024-04-20T11:28:23.691 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.692+0000 7f049e600640 1 Processor -- start 2024-04-20T11:28:23.691 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.692+0000 7f049e600640 1 -- start start 2024-04-20T11:28:23.693 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.693+0000 7f049e600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0498073b60 0x7f04981e7070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:23.693 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.693+0000 7f049e600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f04980b4870 0x7f04981537f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:23.693 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.693+0000 7f049e600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0498155760 0x7f049809d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:23.693 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.693+0000 7f049e600640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f04980cef10 con 0x7f04980b4870 2024-04-20T11:28:23.693 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.693+0000 7f049e600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f04980cec30 con 0x7f0498155760 2024-04-20T11:28:23.693 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.693+0000 7f049e600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f04980cf430 con 0x7f0498073b60 2024-04-20T11:28:23.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.693+0000 7f0497e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f04980b4870 0x7f04981537f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:23.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.693+0000 7f0497e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f04980b4870 0x7f04981537f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:41228/0 (socket says 172.21.15.152:41228) 2024-04-20T11:28:23.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.693+0000 7f0497e00640 1 -- 172.21.15.152:0/498261765 learned_addr learned my addr 172.21.15.152:0/498261765 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:23.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.694+0000 7f049dc00640 1 --2- 172.21.15.152:0/498261765 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0498155760 0x7f049809d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:23.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.694+0000 7f049d200640 1 --2- 172.21.15.152:0/498261765 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0498073b60 0x7f04981e7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:23.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.694+0000 7f0497e00640 1 -- 172.21.15.152:0/498261765 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0498073b60 msgr2=0x7f04981e7070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:23.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.694+0000 7f0497e00640 1 --2- 172.21.15.152:0/498261765 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0498073b60 0x7f04981e7070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:23.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.694+0000 7f0497e00640 1 -- 172.21.15.152:0/498261765 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0498155760 msgr2=0x7f049809d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:23.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.694+0000 7f0497e00640 1 --2- 172.21.15.152:0/498261765 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0498155760 0x7f049809d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:23.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.694+0000 7f0497e00640 1 -- 172.21.15.152:0/498261765 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0498156270 con 0x7f04980b4870 2024-04-20T11:28:23.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.694+0000 7f0497e00640 1 --2- 172.21.15.152:0/498261765 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f04980b4870 0x7f04981537f0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f048400c430 tx=0x7f048400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:23.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.694+0000 7f0494c00640 1 -- 172.21.15.152:0/498261765 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04840052d0 con 0x7f04980b4870 2024-04-20T11:28:23.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.695+0000 7f0494c00640 1 -- 172.21.15.152:0/498261765 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0484005490 con 0x7f04980b4870 2024-04-20T11:28:23.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.695+0000 7f0494c00640 1 -- 172.21.15.152:0/498261765 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0484007260 con 0x7f04980b4870 2024-04-20T11:28:23.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.695+0000 7f049e600640 1 -- 172.21.15.152:0/498261765 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0498156550 con 0x7f04980b4870 2024-04-20T11:28:23.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.695+0000 7f049e600640 1 -- 172.21.15.152:0/498261765 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0498156b10 con 0x7f04980b4870 2024-04-20T11:28:23.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.696+0000 7f049e600640 1 -- 172.21.15.152:0/498261765 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0498075e60 con 0x7f04980b4870 2024-04-20T11:28:23.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.697+0000 7f0494c00640 1 -- 172.21.15.152:0/498261765 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0484007400 con 0x7f04980b4870 2024-04-20T11:28:23.697 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.698+0000 7f0494c00640 1 --2- 172.21.15.152:0/498261765 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f047806c8f0 0x7f047806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:23.697 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.698+0000 7f0494c00640 1 -- 172.21.15.152:0/498261765 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f048408b150 con 0x7f04980b4870 2024-04-20T11:28:23.697 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.698+0000 7f049d200640 1 --2- 172.21.15.152:0/498261765 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f047806c8f0 0x7f047806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:23.698 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.699+0000 7f049d200640 1 --2- 172.21.15.152:0/498261765 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f047806c8f0 0x7f047806edb0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f048c009a30 tx=0x7f048c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:23.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.703+0000 7f0494c00640 1 -- 172.21.15.152:0/498261765 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f04840594e0 con 0x7f04980b4870 2024-04-20T11:28:23.977 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.978+0000 7f049e600640 1 -- 172.21.15.152:0/498261765 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f0498073560 con 0x7f04980b4870 2024-04-20T11:28:23.982 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.983+0000 7f0494c00640 1 -- 172.21.15.152:0/498261765 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v9) ==== 190+0+147 (secure 0 0 0) 0x7f0484058de0 con 0x7f04980b4870 2024-04-20T11:28:23.983 INFO:teuthology.orchestra.run.smithi152.stdout:[client.0] 2024-04-20T11:28:23.983 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQBDpiNmtsBWFhAArq9A7r33CcwgMmOU2pAGPw== 2024-04-20T11:28:23.983 INFO:teuthology.orchestra.run.smithi152.stdout: caps mds = "allow" 2024-04-20T11:28:23.983 INFO:teuthology.orchestra.run.smithi152.stdout: caps mon = "allow r" 2024-04-20T11:28:23.983 INFO:teuthology.orchestra.run.smithi152.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-20T11:28:23.983 INFO:teuthology.orchestra.run.smithi152.stderr:updated caps for client.0 2024-04-20T11:28:23.985 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.986+0000 7f049e600640 1 -- 172.21.15.152:0/498261765 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f047806c8f0 msgr2=0x7f047806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:23.985 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.986+0000 7f049e600640 1 --2- 172.21.15.152:0/498261765 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f047806c8f0 0x7f047806edb0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f048c009a30 tx=0x7f048c008040 comp rx=0 tx=0).stop 2024-04-20T11:28:23.985 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.986+0000 7f049e600640 1 -- 172.21.15.152:0/498261765 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f04980b4870 msgr2=0x7f04981537f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:23.986 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.986+0000 7f049e600640 1 --2- 172.21.15.152:0/498261765 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f04980b4870 0x7f04981537f0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f048400c430 tx=0x7f048400c8f0 comp rx=0 tx=0).stop 2024-04-20T11:28:23.986 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.986+0000 7f049e600640 1 -- 172.21.15.152:0/498261765 shutdown_connections 2024-04-20T11:28:23.986 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.986+0000 7f049e600640 1 --2- 172.21.15.152:0/498261765 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f047806c8f0 0x7f047806edb0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:23.986 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.986+0000 7f049e600640 1 --2- 172.21.15.152:0/498261765 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0498155760 0x7f049809d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:23.986 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.986+0000 7f049e600640 1 --2- 172.21.15.152:0/498261765 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f04980b4870 0x7f04981537f0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:23.986 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.986+0000 7f049e600640 1 --2- 172.21.15.152:0/498261765 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0498073b60 0x7f04981e7070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:23.986 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.986+0000 7f049e600640 1 -- 172.21.15.152:0/498261765 >> 172.21.15.152:0/498261765 conn(0x7f04980d0260 msgr2=0x7f049815bab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:23.986 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.986+0000 7f049e600640 1 -- 172.21.15.152:0/498261765 shutdown_connections 2024-04-20T11:28:23.986 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:23.986+0000 7f049e600640 1 -- 172.21.15.152:0/498261765 wait complete. 2024-04-20T11:28:24.005 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:28:24.199 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:24.199+0000 7fafa7800640 1 Processor -- start 2024-04-20T11:28:24.199 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:24.199+0000 7fafa7800640 1 -- start start 2024-04-20T11:28:24.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:24.201+0000 7fafa7800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fafa015b5e0 0x7fafa006eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:24.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:24.201+0000 7fafa7800640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fafa00b5770 con 0x7fafa00b4c50 2024-04-20T11:28:24.201 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:24.201+0000 7fafa7800640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fafa006e060 con 0x7fafa006fa50 2024-04-20T11:28:24.201 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:24.201+0000 7fafa7800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fafa006de10 con 0x7fafa015b5e0 2024-04-20T11:28:24.201 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:24.201+0000 7faf97e00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fafa00b4c50 0x7fafa006f340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:57992/0 (socket says 172.21.15.152:57992) 2024-04-20T11:28:24.201 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:24.201+0000 7faf97e00640 1 -- 172.21.15.152:0/401882271 learned_addr learned my addr 172.21.15.152:0/401882271 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:28:24.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:24.202+0000 7fafa4e00640 1 --2- 172.21.15.152:0/401882271 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fafa015b5e0 0x7fafa006eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:24.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:24.202+0000 7faf9fe00640 1 -- 172.21.15.152:0/401882271 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2797027586 0 0) 0x7fafa00b5770 con 0x7fafa00b4c50 2024-04-20T11:28:24.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:24.202+0000 7faf9fe00640 1 -- 172.21.15.152:0/401882271 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faf84003520 con 0x7fafa00b4c50 2024-04-20T11:28:24.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:24.202+0000 7faf9fe00640 1 -- 172.21.15.152:0/401882271 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1507114632 0 0) 0x7faf84003520 con 0x7fafa00b4c50 2024-04-20T11:28:24.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:24.202+0000 7faf9fe00640 1 -- 172.21.15.152:0/401882271 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fafa015b5e0 msgr2=0x7fafa006eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:24.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:24.202+0000 7faf9fe00640 1 --2- 172.21.15.152:0/401882271 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fafa015b5e0 0x7fafa006eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:24.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:24.202+0000 7faf9fe00640 1 -- 172.21.15.152:0/401882271 >> v1:172.21.15.175:6789/0 conn(0x7fafa006fa50 legacy=0x7fafa015cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:24.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:24.202+0000 7faf9fe00640 1 -- 172.21.15.152:0/401882271 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafa006e340 con 0x7fafa00b4c50 2024-04-20T11:28:24.204 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:24.202+0000 7faf9fe00640 1 -- 172.21.15.152:0/401882271 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7faf88001690 con 0x7fafa00b4c50 2024-04-20T11:28:24.204 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:24.202+0000 7faf9fe00640 1 -- 172.21.15.152:0/401882271 >> v1:172.21.15.152:6789/0 conn(0x7fafa00b4c50 legacy=0x7fafa006f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:24.204 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:24.202+0000 7faf9fe00640 1 --2- 172.21.15.152:0/401882271 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faf84003a50 0x7faf84003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:24.204 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:24.202+0000 7faf9fe00640 1 --2- 172.21.15.152:0/401882271 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faf84004800 0x7faf84004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:24.204 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:24.202+0000 7faf9fe00640 1 --2- 172.21.15.152:0/401882271 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fafa015b5e0 0x7faf840051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:24.204 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:24.202+0000 7faf9fe00640 1 -- 172.21.15.152:0/401882271 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fafa006de10 con 0x7faf84004800 2024-04-20T11:28:24.204 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:24.202+0000 7faf9fe00640 1 -- 172.21.15.152:0/401882271 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7faf8400bac0 con 0x7faf84003a50 2024-04-20T11:28:24.204 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:24.202+0000 7faf9fe00640 1 -- 172.21.15.152:0/401882271 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7faf8400bcc0 con 0x7fafa015b5e0 2024-04-20T11:28:24.204 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:24.203+0000 7faf97e00640 1 --2- 172.21.15.152:0/401882271 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faf84004800 0x7faf84004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:24.204 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:24.203+0000 7fafa5800640 1 --2- 172.21.15.152:0/401882271 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fafa015b5e0 0x7faf840051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:24.204 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:24.203+0000 7fafa4e00640 1 --2- 172.21.15.152:0/401882271 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faf84003a50 0x7faf84003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:24.204 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:24.203+0000 7faf97e00640 1 -- 172.21.15.152:0/401882271 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fafa015b5e0 msgr2=0x7faf840051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:24.204 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:24.203+0000 7faf97e00640 1 --2- 172.21.15.152:0/401882271 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fafa015b5e0 0x7faf840051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:24.204 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:24.203+0000 7faf97e00640 1 -- 172.21.15.152:0/401882271 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faf84003a50 msgr2=0x7faf84003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:24.205 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:24.203+0000 7faf97e00640 1 --2- 172.21.15.152:0/401882271 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faf84003a50 0x7faf84003e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:24.205 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:24.203+0000 7faf97e00640 1 -- 172.21.15.152:0/401882271 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf84003520 con 0x7faf84004800 2024-04-20T11:28:24.206 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:24.204+0000 7faf97e00640 1 --2- 172.21.15.152:0/401882271 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faf84004800 0x7faf84004c80 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7faf8800c210 tx=0x7faf84005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c93f82112bedaad5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:24.206 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:24.205+0000 7faf9fe00640 1 -- 172.21.15.152:0/401882271 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf88007d10 con 0x7faf84004800 2024-04-20T11:28:24.206 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:24.205+0000 7faf9fe00640 1 -- 172.21.15.152:0/401882271 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf88030070 con 0x7faf84004800 2024-04-20T11:28:24.206 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:24.205+0000 7faf9fe00640 1 -- 172.21.15.152:0/401882271 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf88005d30 con 0x7faf84004800 2024-04-20T11:28:24.206 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:24.205+0000 7fafa7800640 1 -- 172.21.15.152:0/401882271 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faf84004800 msgr2=0x7faf84004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:24.206 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:24.205+0000 7fafa7800640 1 --2- 172.21.15.152:0/401882271 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faf84004800 0x7faf84004c80 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7faf8800c210 tx=0x7faf84005830 comp rx=0 tx=0).stop 2024-04-20T11:28:24.206 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:24.205+0000 7fafa5800640 1 -- 172.21.15.152:0/401882271 reap_dead start 2024-04-20T11:28:24.206 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:24.205+0000 7fafa7800640 1 -- 172.21.15.152:0/401882271 shutdown_connections 2024-04-20T11:28:24.206 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:24.205+0000 7fafa7800640 1 -- 172.21.15.152:0/401882271 >> 172.21.15.152:0/401882271 conn(0x7fafa0067cf0 msgr2=0x7fafa006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:24.206 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:24.205+0000 7fafa7800640 1 -- 172.21.15.152:0/401882271 shutdown_connections 2024-04-20T11:28:24.206 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:24.205+0000 7fafa7800640 1 -- 172.21.15.152:0/401882271 wait complete. 2024-04-20T11:28:24.207 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:24.207+0000 7fafa7800640 1 Processor -- start 2024-04-20T11:28:24.207 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:24.207+0000 7fafa7800640 1 -- start start 2024-04-20T11:28:24.208 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:24.208+0000 7fafa7800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faf84004800 0x7fafa00d0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:24.208 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:24.208+0000 7fafa7800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fafa0153c80 0x7fafa0155790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:24.209 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:24.208+0000 7fafa7800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fafa013dbd0 0x7fafa0155cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:24.209 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:24.208+0000 7fafa7800640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fafa00b42d0 con 0x7fafa013dbd0 2024-04-20T11:28:24.209 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:24.208+0000 7fafa7800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fafa015c9d0 con 0x7faf84004800 2024-04-20T11:28:24.209 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:24.208+0000 7fafa7800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fafa006eb40 con 0x7fafa0153c80 2024-04-20T11:28:24.209 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:24.209+0000 7fafa5800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fafa013dbd0 0x7fafa0155cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:24.209 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:24.209+0000 7fafa5800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fafa013dbd0 0x7fafa0155cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:41240/0 (socket says 172.21.15.152:41240) 2024-04-20T11:28:24.209 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:24.209+0000 7fafa5800640 1 -- 172.21.15.152:0/708228786 learned_addr learned my addr 172.21.15.152:0/708228786 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:24.209 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:24.209+0000 7faf97e00640 1 --2- 172.21.15.152:0/708228786 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fafa0153c80 0x7fafa0155790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:24.209 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:24.209+0000 7fafa4e00640 1 --2- 172.21.15.152:0/708228786 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faf84004800 0x7fafa00d0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:24.209 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:24.209+0000 7fafa5800640 1 -- 172.21.15.152:0/708228786 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fafa0153c80 msgr2=0x7fafa0155790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:24.209 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:24.209+0000 7fafa5800640 1 --2- 172.21.15.152:0/708228786 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fafa0153c80 0x7fafa0155790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:24.209 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:24.209+0000 7fafa5800640 1 -- 172.21.15.152:0/708228786 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faf84004800 msgr2=0x7fafa00d0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:24.209 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:24.209+0000 7fafa5800640 1 --2- 172.21.15.152:0/708228786 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faf84004800 0x7fafa00d0b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:24.209 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:24.209+0000 7fafa5800640 1 -- 172.21.15.152:0/708228786 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafa0151ef0 con 0x7fafa013dbd0 2024-04-20T11:28:24.210 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:24.209+0000 7fafa5800640 1 --2- 172.21.15.152:0/708228786 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fafa013dbd0 0x7fafa0155cd0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fafa0073180 tx=0x7faf9800ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:24.211 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:24.210+0000 7faf9d600640 1 -- 172.21.15.152:0/708228786 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf98003970 con 0x7fafa013dbd0 2024-04-20T11:28:24.211 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:24.210+0000 7faf9d600640 1 -- 172.21.15.152:0/708228786 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf98018070 con 0x7fafa013dbd0 2024-04-20T11:28:24.211 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:24.210+0000 7fafa7800640 1 -- 172.21.15.152:0/708228786 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafa0152190 con 0x7fafa013dbd0 2024-04-20T11:28:24.211 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:24.210+0000 7faf9d600640 1 -- 172.21.15.152:0/708228786 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf98003d70 con 0x7fafa013dbd0 2024-04-20T11:28:24.211 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:24.210+0000 7fafa7800640 1 -- 172.21.15.152:0/708228786 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafa00d1080 con 0x7fafa013dbd0 2024-04-20T11:28:24.211 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:24.211+0000 7fafa7800640 1 -- 172.21.15.152:0/708228786 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafa00b4a40 con 0x7fafa013dbd0 2024-04-20T11:28:24.211 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:24.212+0000 7faf9d600640 1 -- 172.21.15.152:0/708228786 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faf980050d0 con 0x7fafa013dbd0 2024-04-20T11:28:24.212 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:24.213+0000 7faf9d600640 1 --2- 172.21.15.152:0/708228786 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7faf7c06c8f0 0x7faf7c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:24.213 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:24.213+0000 7faf9d600640 1 -- 172.21.15.152:0/708228786 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7faf9808b8f0 con 0x7fafa013dbd0 2024-04-20T11:28:24.213 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:24.213+0000 7fafa4e00640 1 --2- 172.21.15.152:0/708228786 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7faf7c06c8f0 0x7faf7c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:24.213 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:24.214+0000 7fafa4e00640 1 --2- 172.21.15.152:0/708228786 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7faf7c06c8f0 0x7faf7c06edb0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7faf90009910 tx=0x7faf90008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:24.217 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:24.218+0000 7faf9d600640 1 -- 172.21.15.152:0/708228786 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fafa00b4a40 con 0x7fafa013dbd0 2024-04-20T11:28:24.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:24.456+0000 7fafa7800640 1 -- 172.21.15.152:0/708228786 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fafa01527e0 con 0x7fafa013dbd0 2024-04-20T11:28:24.457 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:24.457+0000 7faf9d600640 1 -- 172.21.15.152:0/708228786 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) ==== 94+0+5211 (secure 0 0 0) 0x7faf98059c80 con 0x7fafa013dbd0 2024-04-20T11:28:24.457 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:28:24.457 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4643,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6834/1871483322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1871483322},{"type":"v1","addr":"172.21.15.175:6834","nonce":1871483322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4786,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/2579061164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6835","nonce":2579061164},{"type":"v1","addr":"172.21.15.152:6837","nonce":2579061164}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5208,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6836/3724129680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":3724129680},{"type":"v1","addr":"172.21.15.152:6836","nonce":3724129680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T11:28:17.902738+0000","modified":"2024-04-20T11:28:20.990798+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5223},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5223":{"gid":5223,"name":"b","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.175:6835/3367117968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6833","nonce":3367117968},{"type":"v1","addr":"172.21.15.175:6835","nonce":3367117968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5223,"qdb_cluster":[5223]},"id":2}]} 2024-04-20T11:28:24.458 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 15 2024-04-20T11:28:24.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:24.460+0000 7fafa7800640 1 -- 172.21.15.152:0/708228786 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7faf7c06c8f0 msgr2=0x7faf7c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:24.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:24.460+0000 7fafa7800640 1 --2- 172.21.15.152:0/708228786 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7faf7c06c8f0 0x7faf7c06edb0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7faf90009910 tx=0x7faf90008040 comp rx=0 tx=0).stop 2024-04-20T11:28:24.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:24.460+0000 7fafa7800640 1 -- 172.21.15.152:0/708228786 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fafa013dbd0 msgr2=0x7fafa0155cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:24.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:24.460+0000 7fafa7800640 1 --2- 172.21.15.152:0/708228786 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fafa013dbd0 0x7fafa0155cd0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fafa0073180 tx=0x7faf9800ec10 comp rx=0 tx=0).stop 2024-04-20T11:28:24.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:24.460+0000 7fafa7800640 1 -- 172.21.15.152:0/708228786 shutdown_connections 2024-04-20T11:28:24.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:24.460+0000 7fafa7800640 1 --2- 172.21.15.152:0/708228786 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7faf7c06c8f0 0x7faf7c06edb0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:24.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:24.460+0000 7fafa7800640 1 --2- 172.21.15.152:0/708228786 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fafa013dbd0 0x7fafa0155cd0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:24.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:24.460+0000 7fafa7800640 1 --2- 172.21.15.152:0/708228786 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fafa0153c80 0x7fafa0155790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:24.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:24.460+0000 7fafa7800640 1 --2- 172.21.15.152:0/708228786 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faf84004800 0x7fafa00d0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:24.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:24.460+0000 7fafa7800640 1 -- 172.21.15.152:0/708228786 >> 172.21.15.152:0/708228786 conn(0x7fafa0067cf0 msgr2=0x7fafa01590c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:24.461 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:24.460+0000 7fafa7800640 1 -- 172.21.15.152:0/708228786 shutdown_connections 2024-04-20T11:28:24.461 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:24.460+0000 7fafa7800640 1 -- 172.21.15.152:0/708228786 wait complete. 2024-04-20T11:28:24.882 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 15, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-20T11:28:17.902738+0000', 'modified': '2024-04-20T11:28:20.990798+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'snap_rstat': True, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5223}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5223': {'gid': 5223, 'name': 'b', 'rank': 0, 'incarnation': 12, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.175:6835/3367117968', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6833', 'nonce': 3367117968}, {'type': 'v1', 'addr': '172.21.15.175:6835', 'nonce': 3367117968}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5223, 'qdb_cluster': [5223]} 2024-04-20T11:28:24.882 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-20T11:28:24.882 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-20T11:28:25.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.095+0000 7f8f9a600640 1 Processor -- start 2024-04-20T11:28:25.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.096+0000 7f8f9a600640 1 -- start start 2024-04-20T11:28:25.097 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.097+0000 7f8f9a600640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f8f940d4ec0 0x7f8f940d0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:25.097 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.097+0000 7f8f9a600640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f8f940d2b80 0x7f8f940d3010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:25.097 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.097+0000 7f8f9a600640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f940d26a0 con 0x7f8f94161d30 2024-04-20T11:28:25.097 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.097+0000 7f8f9a600640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f8f940d0540 con 0x7f8f940d2b80 2024-04-20T11:28:25.097 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.097+0000 7f8f9a600640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f8f940cfb70 con 0x7f8f940d4ec0 2024-04-20T11:28:25.098 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.097+0000 7f8f99200640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f8f94161d30 0x7f8f940d15e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:58004/0 (socket says 172.21.15.152:58004) 2024-04-20T11:28:25.098 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.097+0000 7f8f99200640 1 -- 172.21.15.152:0/1364063054 learned_addr learned my addr 172.21.15.152:0/1364063054 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:28:25.098 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.097+0000 7f8f99c00640 1 --2- 172.21.15.152:0/1364063054 >> v2:172.21.15.152:3300/0 conn(0x7f8f940d2b80 0x7f8f940d3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:25.098 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.098+0000 7f8f93e00640 1 --2- 172.21.15.152:0/1364063054 >> v2:172.21.15.175:3300/0 conn(0x7f8f940d4ec0 0x7f8f940d0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:25.098 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.098+0000 7f8f99c00640 1 -- 172.21.15.152:0/1364063054 >> v2:172.21.15.175:3300/0 conn(0x7f8f940d4ec0 msgr2=0x7f8f940d0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:25.098 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.098+0000 7f8f99c00640 1 --2- 172.21.15.152:0/1364063054 >> v2:172.21.15.175:3300/0 conn(0x7f8f940d4ec0 0x7f8f940d0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.098 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.098+0000 7f8f99c00640 1 -- 172.21.15.152:0/1364063054 >> v1:172.21.15.152:6789/0 conn(0x7f8f94161d30 legacy=0x7f8f940d15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:25.098 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.098+0000 7f8f93400640 1 -- 172.21.15.152:0/1364063054 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 206423143 0 0) 0x7f8f940d26a0 con 0x7f8f94161d30 2024-04-20T11:28:25.098 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.098+0000 7f8f99c00640 1 -- 172.21.15.152:0/1364063054 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f940d53c0 con 0x7f8f940d2b80 2024-04-20T11:28:25.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.099+0000 7f8f99c00640 1 --2- 172.21.15.152:0/1364063054 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8f940d2b80 0x7f8f940d3010 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f8f940a2520 tx=0x7f8f8c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c58d8b52894ea4a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:25.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.099+0000 7f8f93400640 1 -- 172.21.15.152:0/1364063054 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f8c035020 con 0x7f8f940d2b80 2024-04-20T11:28:25.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.099+0000 7f8f93400640 1 -- 172.21.15.152:0/1364063054 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f8c031830 con 0x7f8f940d2b80 2024-04-20T11:28:25.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.099+0000 7f8f93400640 1 -- 172.21.15.152:0/1364063054 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f8c031c00 con 0x7f8f940d2b80 2024-04-20T11:28:25.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.100+0000 7f8f9a600640 1 -- 172.21.15.152:0/1364063054 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8f940d2b80 msgr2=0x7f8f940d3010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:25.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.100+0000 7f8f9a600640 1 --2- 172.21.15.152:0/1364063054 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8f940d2b80 0x7f8f940d3010 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f8f940a2520 tx=0x7f8f8c02b8d0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.100+0000 7f8f9a600640 1 -- 172.21.15.152:0/1364063054 shutdown_connections 2024-04-20T11:28:25.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.100+0000 7f8f9a600640 1 --2- 172.21.15.152:0/1364063054 >> v2:172.21.15.175:3300/0 conn(0x7f8f940d4ec0 0x7f8f940d0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.100+0000 7f8f9a600640 1 --2- 172.21.15.152:0/1364063054 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8f940d2b80 0x7f8f940d3010 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.100+0000 7f8f9a600640 1 -- 172.21.15.152:0/1364063054 >> 172.21.15.152:0/1364063054 conn(0x7f8f9408bfd0 msgr2=0x7f8f940d45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:25.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.100+0000 7f8f9a600640 1 -- 172.21.15.152:0/1364063054 shutdown_connections 2024-04-20T11:28:25.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.100+0000 7f8f9a600640 1 -- 172.21.15.152:0/1364063054 wait complete. 2024-04-20T11:28:25.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.102+0000 7f8f9a600640 1 Processor -- start 2024-04-20T11:28:25.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.102+0000 7f8f9a600640 1 -- start start 2024-04-20T11:28:25.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.103+0000 7f8f9a600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8f940d2b80 0x7f8f9409fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:25.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.103+0000 7f8f9a600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8f940d4ec0 0x7f8f9409ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:25.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.103+0000 7f8f9a600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8f94161d30 0x7f8f9414d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:25.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.103+0000 7f8f9a600640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f8f940cf1b0 con 0x7f8f94161d30 2024-04-20T11:28:25.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.103+0000 7f8f9a600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8f940cf800 con 0x7f8f940d2b80 2024-04-20T11:28:25.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.103+0000 7f8f9a600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8f94162110 con 0x7f8f940d4ec0 2024-04-20T11:28:25.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.103+0000 7f8f99c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8f94161d30 0x7f8f9414d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:25.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.103+0000 7f8f99c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8f94161d30 0x7f8f9414d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:41246/0 (socket says 172.21.15.152:41246) 2024-04-20T11:28:25.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.103+0000 7f8f99c00640 1 -- 172.21.15.152:0/4081937700 learned_addr learned my addr 172.21.15.152:0/4081937700 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:25.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.104+0000 7f8f99200640 1 --2- 172.21.15.152:0/4081937700 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8f940d2b80 0x7f8f9409fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:25.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.104+0000 7f8f93e00640 1 --2- 172.21.15.152:0/4081937700 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8f940d4ec0 0x7f8f9409ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:25.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.104+0000 7f8f99c00640 1 -- 172.21.15.152:0/4081937700 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8f940d4ec0 msgr2=0x7f8f9409ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:25.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.104+0000 7f8f99c00640 1 --2- 172.21.15.152:0/4081937700 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8f940d4ec0 0x7f8f9409ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.104+0000 7f8f99c00640 1 -- 172.21.15.152:0/4081937700 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8f940d2b80 msgr2=0x7f8f9409fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:25.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.104+0000 7f8f99c00640 1 --2- 172.21.15.152:0/4081937700 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8f940d2b80 0x7f8f9409fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.104+0000 7f8f99c00640 1 -- 172.21.15.152:0/4081937700 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f9414e340 con 0x7f8f94161d30 2024-04-20T11:28:25.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.104+0000 7f8f99c00640 1 --2- 172.21.15.152:0/4081937700 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8f94161d30 0x7f8f9414d8a0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c034040 tx=0x7f8f8c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:25.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.104+0000 7f8f90c00640 1 -- 172.21.15.152:0/4081937700 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f8c040070 con 0x7f8f94161d30 2024-04-20T11:28:25.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.104+0000 7f8f90c00640 1 -- 172.21.15.152:0/4081937700 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f8c0398e0 con 0x7f8f94161d30 2024-04-20T11:28:25.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.104+0000 7f8f9a600640 1 -- 172.21.15.152:0/4081937700 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f9414e620 con 0x7f8f94161d30 2024-04-20T11:28:25.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.104+0000 7f8f90c00640 1 -- 172.21.15.152:0/4081937700 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f8c035040 con 0x7f8f94161d30 2024-04-20T11:28:25.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.105+0000 7f8f9a600640 1 -- 172.21.15.152:0/4081937700 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f9414ebe0 con 0x7f8f94161d30 2024-04-20T11:28:25.107 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.107+0000 7f8f90c00640 1 -- 172.21.15.152:0/4081937700 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8f8c038020 con 0x7f8f94161d30 2024-04-20T11:28:25.107 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.108+0000 7f8f90c00640 1 --2- 172.21.15.152:0/4081937700 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f8f7006c8f0 0x7f8f7006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:25.108 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.108+0000 7f8f90c00640 1 -- 172.21.15.152:0/4081937700 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f8f8c0ad520 con 0x7f8f94161d30 2024-04-20T11:28:25.108 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.108+0000 7f8f99200640 1 --2- 172.21.15.152:0/4081937700 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f8f7006c8f0 0x7f8f7006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:25.108 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.109+0000 7f8f9e9ac740 1 Processor -- start 2024-04-20T11:28:25.108 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.109+0000 7f8f9e9ac740 1 -- start start 2024-04-20T11:28:25.108 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.109+0000 7f8f99200640 1 --2- 172.21.15.152:0/4081937700 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f8f7006c8f0 0x7f8f7006edb0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f8f840012e0 tx=0x7f8f84009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:25.109 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.110+0000 7f8f9e9ac740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560895eb9670 0x560895e70ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:25.109 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.110+0000 7f8f9e9ac740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560895e6f6f0 0x560895e711e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:25.110 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.110+0000 7f8f9e9ac740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x560895e6bb90 0x560895e9b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:25.110 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.110+0000 7f8f9e9ac740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x560895dd79d0 con 0x560895e6f6f0 2024-04-20T11:28:25.110 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.110+0000 7f8f9e9ac740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x560895d7e270 con 0x560895eb9670 2024-04-20T11:28:25.110 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.110+0000 7f8f9e9ac740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x560895dd3990 con 0x560895e6bb90 2024-04-20T11:28:25.110 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.110+0000 7f8f93e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560895e6f6f0 0x560895e711e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:25.110 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.110+0000 7f8f93e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560895e6f6f0 0x560895e711e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:41252/0 (socket says 172.21.15.152:41252) 2024-04-20T11:28:25.110 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.110+0000 7f8f93e00640 1 -- 172.21.15.152:0/3605252697 learned_addr learned my addr 172.21.15.152:0/3605252697 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:25.112 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.110+0000 7f8f93e00640 1 -- 172.21.15.152:0/3605252697 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x560895e6bb90 msgr2=0x560895e9b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:25.112 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.110+0000 7f8f99c00640 1 --2- 172.21.15.152:0/3605252697 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x560895e6bb90 0x560895e9b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:25.112 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.110+0000 7f8f93e00640 1 --2- 172.21.15.152:0/3605252697 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x560895e6bb90 0x560895e9b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.113 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.110+0000 7f8f93e00640 1 -- 172.21.15.152:0/3605252697 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560895eb9670 msgr2=0x560895e70ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:25.113 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.110+0000 7f8f93e00640 1 --2- 172.21.15.152:0/3605252697 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560895eb9670 0x560895e70ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.113 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.110+0000 7f8f93e00640 1 -- 172.21.15.152:0/3605252697 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560895aea2d0 con 0x560895e6f6f0 2024-04-20T11:28:25.113 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.111+0000 7f8f93e00640 1 --2- 172.21.15.152:0/3605252697 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560895e6f6f0 0x560895e711e0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c009840 tx=0x7f8f7c009d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3efadb3292d9480 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:25.113 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.111+0000 7f8f89600640 1 -- 172.21.15.152:0/3605252697 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f7c017020 con 0x560895e6f6f0 2024-04-20T11:28:25.113 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.111+0000 7f8f89600640 1 -- 172.21.15.152:0/3605252697 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f7c00e950 con 0x560895e6f6f0 2024-04-20T11:28:25.113 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.111+0000 7f8f89600640 1 -- 172.21.15.152:0/3605252697 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f7c00f040 con 0x560895e6f6f0 2024-04-20T11:28:25.113 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.112+0000 7f8f9e9ac740 1 -- 172.21.15.152:0/3605252697 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560895e6f6f0 msgr2=0x560895e711e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:25.113 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.112+0000 7f8f9e9ac740 1 --2- 172.21.15.152:0/3605252697 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560895e6f6f0 0x560895e711e0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c009840 tx=0x7f8f7c009d00 comp rx=0 tx=0).stop 2024-04-20T11:28:25.113 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.112+0000 7f8f9e9ac740 1 -- 172.21.15.152:0/3605252697 shutdown_connections 2024-04-20T11:28:25.113 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.112+0000 7f8f9e9ac740 1 --2- 172.21.15.152:0/3605252697 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x560895e6bb90 0x560895e9b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.113 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.112+0000 7f8f9e9ac740 1 --2- 172.21.15.152:0/3605252697 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560895e6f6f0 0x560895e711e0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.113 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.112+0000 7f8f9e9ac740 1 --2- 172.21.15.152:0/3605252697 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560895eb9670 0x560895e70ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.113 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.112+0000 7f8f9e9ac740 1 -- 172.21.15.152:0/3605252697 >> 172.21.15.152:0/3605252697 conn(0x560895e6de00 msgr2=0x560895eb6980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:25.114 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.112+0000 7f8f9e9ac740 1 -- 172.21.15.152:0/3605252697 shutdown_connections 2024-04-20T11:28:25.114 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.112+0000 7f8f9e9ac740 1 -- 172.21.15.152:0/3605252697 wait complete. 2024-04-20T11:28:25.114 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.113+0000 7f8f9e9ac740 1 -- start start 2024-04-20T11:28:25.114 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.113+0000 7f8f9e9ac740 1 Processor -- start 2024-04-20T11:28:25.118 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.114+0000 7f8f9e9ac740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x560895e6f6f0 0x560895e83fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:25.118 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.114+0000 7f8f9e9ac740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560895eb9670 0x560895ebb050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:25.118 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.114+0000 7f8f9e9ac740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560895e96c80 0x560895e97100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:25.118 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.114+0000 7f8f9e9ac740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x560895e6fc00 con 0x560895e96c80 2024-04-20T11:28:25.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.114+0000 7f8f9e9ac740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x560895d915e0 con 0x560895eb9670 2024-04-20T11:28:25.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.114+0000 7f8f9e9ac740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x560895e6fee0 con 0x560895e6f6f0 2024-04-20T11:28:25.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.114+0000 7f8f9e9ac740 10 client.0 _ll_register_callbacks cb 0x560895ae9ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:28:25.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.114+0000 7f8f99c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560895e96c80 0x560895e97100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:25.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.114+0000 7f8f99c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560895e96c80 0x560895e97100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:41254/0 (socket says 172.21.15.152:41254) 2024-04-20T11:28:25.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.114+0000 7f8f99c00640 1 -- 172.21.15.152:0/809854035 learned_addr learned my addr 172.21.15.152:0/809854035 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:25.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.114+0000 7f8f99c00640 1 -- 172.21.15.152:0/809854035 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x560895e6f6f0 msgr2=0x560895e83fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:25.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.114+0000 7f8f99c00640 1 --2- 172.21.15.152:0/809854035 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x560895e6f6f0 0x560895e83fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.114+0000 7f8f99c00640 1 -- 172.21.15.152:0/809854035 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560895eb9670 msgr2=0x560895ebb050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:25.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.114+0000 7f8f99c00640 1 --2- 172.21.15.152:0/809854035 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560895eb9670 0x560895ebb050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.114+0000 7f8f99c00640 1 -- 172.21.15.152:0/809854035 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560895bb46b0 con 0x560895e96c80 2024-04-20T11:28:25.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.115+0000 7f8f99c00640 1 --2- 172.21.15.152:0/809854035 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560895e96c80 0x560895e97100 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c0ab920 tx=0x7f8f8c07b1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b81cf14187d50b7a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:25.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.115+0000 7f8f82000640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-04-20T11:28:25.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.115+0000 7f8f82000640 1 -- 172.21.15.152:0/809854035 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f8c077820 con 0x560895e96c80 2024-04-20T11:28:25.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.115+0000 7f8f82000640 1 -- 172.21.15.152:0/809854035 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f8f8c039cd0 con 0x560895e96c80 2024-04-20T11:28:25.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.115+0000 7f8f9e9ac740 1 -- 172.21.15.152:0/809854035 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f7c00e950 con 0x560895e96c80 2024-04-20T11:28:25.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.115+0000 7f8f82000640 1 -- 172.21.15.152:0/809854035 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=15) ==== 24+0+0 (secure 0 0 0) 0x7f8f8c0779c0 con 0x560895e96c80 2024-04-20T11:28:25.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.115+0000 7f8f9e9ac740 10 client.5406 fetch_fsmap learned FSMap version 15 2024-04-20T11:28:25.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.115+0000 7f8f9e9ac740 1 -- 172.21.15.152:0/809854035 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=15,monmap=2+}) -- 0x560895ebc4c0 con 0x560895e96c80 2024-04-20T11:28:25.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.115+0000 7f8f82000640 1 -- 172.21.15.152:0/809854035 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f7c00e950 con 0x560895e96c80 2024-04-20T11:28:25.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.115+0000 7f8f82000640 1 -- 172.21.15.152:0/809854035 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 15) ==== 3495+0+0 (secure 0 0 0) 0x7f8f8c07f170 con 0x560895e96c80 2024-04-20T11:28:25.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.116+0000 7f8f9e9ac740 10 client.5406 fetch_fsmap finished waiting for FSMap version 15 2024-04-20T11:28:25.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.116+0000 7f8f9e9ac740 10 client.5406 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-20T11:28:25.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.116+0000 7f8f9e9ac740 20 client.5406 populate_metadata read hostname 'smithi152' 2024-04-20T11:28:25.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.116+0000 7f8f9e9ac740 1 --2- 172.21.15.152:0/809854035 >> [v2:172.21.15.175:6833/3367117968,v1:172.21.15.175:6835/3367117968] conn(0x560895ebd970 0x560895ebdd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:25.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.116+0000 7f8f9e9ac740 4 client.5406 mds_command: new command op to 5223 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:28:25.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.116+0000 7f8f9e9ac740 1 -- 172.21.15.152:0/809854035 --> [v2:172.21.15.175:6833/3367117968,v1:172.21.15.175:6835/3367117968] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560895de53f0 con 0x560895ebd970 2024-04-20T11:28:25.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.116+0000 7f8f99200640 1 --2- 172.21.15.152:0/809854035 >> [v2:172.21.15.175:6833/3367117968,v1:172.21.15.175:6835/3367117968] conn(0x560895ebd970 0x560895ebdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:25.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.117+0000 7f8f99200640 1 --2- 172.21.15.152:0/809854035 >> [v2:172.21.15.175:6833/3367117968,v1:172.21.15.175:6835/3367117968] conn(0x560895ebd970 0x560895ebdd50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0a06256a0b172de server_cookie=321f07465db1e497 in_seq=0 out_seq=0 2024-04-20T11:28:25.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.117+0000 7f8f82000640 10 client.5406 ms_handle_connect on v2:172.21.15.175:6833/3367117968 2024-04-20T11:28:25.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.119+0000 7f8f82000640 1 -- 172.21.15.152:0/809854035 <== mds.0 v2:172.21.15.175:6833/3367117968 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560895de53f0 con 0x560895ebd970 2024-04-20T11:28:25.122 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.119+0000 7f8f82000640 10 client.5406 handle_command_reply: tid=0 2024-04-20T11:28:25.122 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.120+0000 7f8f9e9ac740 1 client.5406 shutdown 2024-04-20T11:28:25.122 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.120+0000 7f8f9e9ac740 1 -- 172.21.15.152:0/809854035 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560895e96c80 msgr2=0x560895e97100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:25.122 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.120+0000 7f8f9e9ac740 1 --2- 172.21.15.152:0/809854035 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560895e96c80 0x560895e97100 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c0ab920 tx=0x7f8f8c07b1b0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.122 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.120+0000 7f8f9e9ac740 1 -- 172.21.15.152:0/809854035 shutdown_connections 2024-04-20T11:28:25.122 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.120+0000 7f8f9e9ac740 1 --2- 172.21.15.152:0/809854035 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560895e96c80 0x560895e97100 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.122 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.120+0000 7f8f9e9ac740 1 --2- 172.21.15.152:0/809854035 >> [v2:172.21.15.175:6833/3367117968,v1:172.21.15.175:6835/3367117968] conn(0x560895ebd970 0x560895ebdd50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.122 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.120+0000 7f8f9e9ac740 1 --2- 172.21.15.152:0/809854035 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560895eb9670 0x560895ebb050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.122 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.120+0000 7f8f9e9ac740 1 --2- 172.21.15.152:0/809854035 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x560895e6f6f0 0x560895e83fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.122 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.120+0000 7f8f82000640 0 client.5406 ms_handle_reset on v2:172.21.15.175:6833/3367117968 2024-04-20T11:28:25.122 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.120+0000 7f8f9e9ac740 1 -- 172.21.15.152:0/809854035 >> 172.21.15.152:0/809854035 conn(0x560895e6bb90 msgr2=0x560895e70a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:25.122 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.120+0000 7f8f9e9ac740 1 -- 172.21.15.152:0/809854035 shutdown_connections 2024-04-20T11:28:25.123 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.120+0000 7f8f9e9ac740 1 -- 172.21.15.152:0/809854035 wait complete. 2024-04-20T11:28:25.123 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.120+0000 7f8f9e9ac740 20 client.5406 trim_cache size 0 max 16384 2024-04-20T11:28:25.139 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.140+0000 7f8f9e9ac740 1 Processor -- start 2024-04-20T11:28:25.139 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.140+0000 7f8f9e9ac740 1 -- start start 2024-04-20T11:28:25.140 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.140+0000 7f8f9e9ac740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x560895e6f6f0 0x560895e70950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:25.140 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.140+0000 7f8f9e9ac740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560895e96c80 0x560895e717b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:25.140 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.140+0000 7f8f9e9ac740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560895eb9670 0x560895e71cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:25.140 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.140+0000 7f8f9e9ac740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f8f94150ed0 con 0x560895e96c80 2024-04-20T11:28:25.140 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.140+0000 7f8f9e9ac740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8f94150230 con 0x560895eb9670 2024-04-20T11:28:25.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.140+0000 7f8f9e9ac740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8f94150880 con 0x560895e6f6f0 2024-04-20T11:28:25.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.141+0000 7f8f93e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560895e96c80 0x560895e717b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:25.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.141+0000 7f8f93e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560895e96c80 0x560895e717b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:41266/0 (socket says 172.21.15.152:41266) 2024-04-20T11:28:25.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.141+0000 7f8f93e00640 1 -- 172.21.15.152:0/2587025330 learned_addr learned my addr 172.21.15.152:0/2587025330 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:25.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.141+0000 7f8f93e00640 1 -- 172.21.15.152:0/2587025330 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x560895e6f6f0 msgr2=0x560895e70950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:25.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.141+0000 7f8f99200640 1 --2- 172.21.15.152:0/2587025330 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x560895e6f6f0 0x560895e70950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:25.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.141+0000 7f8f93e00640 1 --2- 172.21.15.152:0/2587025330 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x560895e6f6f0 0x560895e70950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.141+0000 7f8f93e00640 1 -- 172.21.15.152:0/2587025330 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560895eb9670 msgr2=0x560895e71cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:25.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.141+0000 7f8f93e00640 1 --2- 172.21.15.152:0/2587025330 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560895eb9670 0x560895e71cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.141+0000 7f8f93e00640 1 -- 172.21.15.152:0/2587025330 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560895e6d560 con 0x560895e96c80 2024-04-20T11:28:25.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.142+0000 7f8f93e00640 1 --2- 172.21.15.152:0/2587025330 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560895e96c80 0x560895e717b0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c016040 tx=0x7f8f7c011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eadd9dbacbc9ca75 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:25.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.142+0000 7f8f88c00640 1 -- 172.21.15.152:0/2587025330 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f7c017040 con 0x560895e96c80 2024-04-20T11:28:25.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.142+0000 7f8f88c00640 1 -- 172.21.15.152:0/2587025330 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f7c005bb0 con 0x560895e96c80 2024-04-20T11:28:25.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.142+0000 7f8f88c00640 1 -- 172.21.15.152:0/2587025330 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f7c00e750 con 0x560895e96c80 2024-04-20T11:28:25.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.142+0000 7f8f9e9ac740 1 -- 172.21.15.152:0/2587025330 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560895e96c80 msgr2=0x560895e717b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:25.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.142+0000 7f8f9e9ac740 1 --2- 172.21.15.152:0/2587025330 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560895e96c80 0x560895e717b0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c016040 tx=0x7f8f7c011000 comp rx=0 tx=0).stop 2024-04-20T11:28:25.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.142+0000 7f8f9e9ac740 1 -- 172.21.15.152:0/2587025330 shutdown_connections 2024-04-20T11:28:25.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.142+0000 7f8f9e9ac740 1 --2- 172.21.15.152:0/2587025330 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560895eb9670 0x560895e71cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.142+0000 7f8f9e9ac740 1 --2- 172.21.15.152:0/2587025330 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560895e96c80 0x560895e717b0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.142+0000 7f8f9e9ac740 1 --2- 172.21.15.152:0/2587025330 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x560895e6f6f0 0x560895e70950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.142+0000 7f8f9e9ac740 1 -- 172.21.15.152:0/2587025330 >> 172.21.15.152:0/2587025330 conn(0x560895e6bb90 msgr2=0x560895e83450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:25.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.142+0000 7f8f9e9ac740 1 -- 172.21.15.152:0/2587025330 shutdown_connections 2024-04-20T11:28:25.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.143+0000 7f8f9e9ac740 1 -- 172.21.15.152:0/2587025330 wait complete. 2024-04-20T11:28:25.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.143+0000 7f8f9e9ac740 1 -- start start 2024-04-20T11:28:25.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.143+0000 7f8f9e9ac740 1 Processor -- start 2024-04-20T11:28:25.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.144+0000 7f8f9e9ac740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560895e6f6f0 0x560895ebe1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:25.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.144+0000 7f8f9e9ac740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x560895e96c80 0x560895f1fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:25.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.144+0000 7f8f9e9ac740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560895eb9670 0x560895e70ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:25.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.144+0000 7f8f9e9ac740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x560895ec84c0 con 0x560895eb9670 2024-04-20T11:28:25.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.144+0000 7f8f9e9ac740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x560895ebc740 con 0x560895e6f6f0 2024-04-20T11:28:25.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.144+0000 7f8f9e9ac740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x560895ec7c20 con 0x560895e96c80 2024-04-20T11:28:25.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.144+0000 7f8f9e9ac740 10 client.0 _ll_register_callbacks cb 0x7f8f8c077450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:28:25.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.144+0000 7f8f99c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560895eb9670 0x560895e70ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:25.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.145+0000 7f8f99c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560895eb9670 0x560895e70ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:41276/0 (socket says 172.21.15.152:41276) 2024-04-20T11:28:25.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.145+0000 7f8f99c00640 1 -- 172.21.15.152:0/908144318 learned_addr learned my addr 172.21.15.152:0/908144318 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:25.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.145+0000 7f8f93e00640 1 --2- 172.21.15.152:0/908144318 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x560895e96c80 0x560895f1fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:25.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.145+0000 7f8f99c00640 1 -- 172.21.15.152:0/908144318 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x560895e96c80 msgr2=0x560895f1fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:25.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.145+0000 7f8f99c00640 1 --2- 172.21.15.152:0/908144318 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x560895e96c80 0x560895f1fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.145+0000 7f8f99c00640 1 -- 172.21.15.152:0/908144318 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560895e6f6f0 msgr2=0x560895ebe1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:25.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.145+0000 7f8f99200640 1 --2- 172.21.15.152:0/908144318 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560895e6f6f0 0x560895ebe1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:25.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.145+0000 7f8f99c00640 1 --2- 172.21.15.152:0/908144318 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560895e6f6f0 0x560895ebe1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.145+0000 7f8f99c00640 1 -- 172.21.15.152:0/908144318 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560895f3a850 con 0x560895eb9670 2024-04-20T11:28:25.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.145+0000 7f8f93e00640 1 --2- 172.21.15.152:0/908144318 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x560895e96c80 0x560895f1fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:28:25.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.145+0000 7f8f99c00640 1 --2- 172.21.15.152:0/908144318 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560895eb9670 0x560895e70ef0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c079460 tx=0x7f8f8c031f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a5b38b6cc646696 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:25.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.145+0000 7f8f82000640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-04-20T11:28:25.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.145+0000 7f8f82000640 1 -- 172.21.15.152:0/908144318 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f8c099060 con 0x560895eb9670 2024-04-20T11:28:25.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.145+0000 7f8f9e9ac740 1 -- 172.21.15.152:0/908144318 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f7c005bb0 con 0x560895eb9670 2024-04-20T11:28:25.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.145+0000 7f8f82000640 1 -- 172.21.15.152:0/908144318 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f8f8c0ad2c0 con 0x560895eb9670 2024-04-20T11:28:25.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.146+0000 7f8f82000640 1 -- 172.21.15.152:0/908144318 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=15) ==== 24+0+0 (secure 0 0 0) 0x7f8f8c09f050 con 0x560895eb9670 2024-04-20T11:28:25.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.146+0000 7f8f9e9ac740 10 client.5412 fetch_fsmap learned FSMap version 15 2024-04-20T11:28:25.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.146+0000 7f8f9e9ac740 1 -- 172.21.15.152:0/908144318 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=15,monmap=2+}) -- 0x560895eac4b0 con 0x560895eb9670 2024-04-20T11:28:25.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.146+0000 7f8f82000640 1 -- 172.21.15.152:0/908144318 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f7c005bb0 con 0x560895eb9670 2024-04-20T11:28:25.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.146+0000 7f8f82000640 1 -- 172.21.15.152:0/908144318 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 15) ==== 3495+0+0 (secure 0 0 0) 0x7f8f8c0a2020 con 0x560895eb9670 2024-04-20T11:28:25.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.146+0000 7f8f9e9ac740 10 client.5412 fetch_fsmap finished waiting for FSMap version 15 2024-04-20T11:28:25.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.146+0000 7f8f9e9ac740 10 client.5412 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-20T11:28:25.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.146+0000 7f8f9e9ac740 20 client.5412 populate_metadata read hostname 'smithi152' 2024-04-20T11:28:25.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.146+0000 7f8f9e9ac740 1 --2- 172.21.15.152:0/908144318 >> [v2:172.21.15.175:6833/3367117968,v1:172.21.15.175:6835/3367117968] conn(0x560895f3de90 0x560895eb7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:25.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.146+0000 7f8f9e9ac740 4 client.5412 mds_command: new command op to 5223 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-20T11:28:25.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.146+0000 7f8f9e9ac740 1 -- 172.21.15.152:0/908144318 --> [v2:172.21.15.175:6833/3367117968,v1:172.21.15.175:6835/3367117968] -- command(tid 0: {"prefix": "status"}) -- 0x560895e70250 con 0x560895f3de90 2024-04-20T11:28:25.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.147+0000 7f8f99200640 1 --2- 172.21.15.152:0/908144318 >> [v2:172.21.15.175:6833/3367117968,v1:172.21.15.175:6835/3367117968] conn(0x560895f3de90 0x560895eb7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:25.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.148+0000 7f8f99200640 1 --2- 172.21.15.152:0/908144318 >> [v2:172.21.15.175:6833/3367117968,v1:172.21.15.175:6835/3367117968] conn(0x560895f3de90 0x560895eb7bf0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e2f6a473d1918d9 server_cookie=3f2ce71abd3e62e8 in_seq=0 out_seq=0 2024-04-20T11:28:25.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.148+0000 7f8f82000640 10 client.5412 ms_handle_connect on v2:172.21.15.175:6833/3367117968 2024-04-20T11:28:25.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.148+0000 7f8f82000640 1 -- 172.21.15.152:0/908144318 <== mds.0 v2:172.21.15.175:6833/3367117968 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x560895e70250 con 0x560895f3de90 2024-04-20T11:28:25.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.148+0000 7f8f82000640 10 client.5412 handle_command_reply: tid=0 2024-04-20T11:28:25.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.148+0000 7f8f9e9ac740 1 client.5412 shutdown 2024-04-20T11:28:25.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.148+0000 7f8f9e9ac740 1 -- 172.21.15.152:0/908144318 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560895eb9670 msgr2=0x560895e70ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:25.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.148+0000 7f8f9e9ac740 1 --2- 172.21.15.152:0/908144318 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560895eb9670 0x560895e70ef0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c079460 tx=0x7f8f8c031f90 comp rx=0 tx=0).stop 2024-04-20T11:28:25.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.149+0000 7f8f9e9ac740 1 -- 172.21.15.152:0/908144318 shutdown_connections 2024-04-20T11:28:25.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.149+0000 7f8f9e9ac740 1 --2- 172.21.15.152:0/908144318 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560895eb9670 0x560895e70ef0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.149+0000 7f8f9e9ac740 1 --2- 172.21.15.152:0/908144318 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x560895e96c80 0x560895f1fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.149+0000 7f8f9e9ac740 1 --2- 172.21.15.152:0/908144318 >> [v2:172.21.15.175:6833/3367117968,v1:172.21.15.175:6835/3367117968] conn(0x560895f3de90 0x560895eb7bf0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.149+0000 7f8f9e9ac740 1 --2- 172.21.15.152:0/908144318 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560895e6f6f0 0x560895ebe1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.149+0000 7f8f82000640 0 client.5412 ms_handle_reset on v2:172.21.15.175:6833/3367117968 2024-04-20T11:28:25.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.149+0000 7f8f9e9ac740 1 -- 172.21.15.152:0/908144318 >> 172.21.15.152:0/908144318 conn(0x560895e6bb90 msgr2=0x560895eb55f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:25.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.149+0000 7f8f9e9ac740 1 -- 172.21.15.152:0/908144318 shutdown_connections 2024-04-20T11:28:25.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.149+0000 7f8f9e9ac740 1 -- 172.21.15.152:0/908144318 wait complete. 2024-04-20T11:28:25.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.149+0000 7f8f9e9ac740 20 client.5412 trim_cache size 0 max 16384 2024-04-20T11:28:25.153 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-04-20T11:28:25.153 INFO:teuthology.orchestra.run.smithi152.stdout: "cluster_fsid": "c8a72131-0d4f-42c7-892f-3f3af4d2bd3d", 2024-04-20T11:28:25.153 INFO:teuthology.orchestra.run.smithi152.stdout: "whoami": 0, 2024-04-20T11:28:25.153 INFO:teuthology.orchestra.run.smithi152.stdout: "id": 5223, 2024-04-20T11:28:25.153 INFO:teuthology.orchestra.run.smithi152.stdout: "want_state": "up:active", 2024-04-20T11:28:25.153 INFO:teuthology.orchestra.run.smithi152.stdout: "state": "up:active", 2024-04-20T11:28:25.153 INFO:teuthology.orchestra.run.smithi152.stdout: "fs_name": "cephfs", 2024-04-20T11:28:25.153 INFO:teuthology.orchestra.run.smithi152.stdout: "rank_uptime": 6.6911523739999996, 2024-04-20T11:28:25.153 INFO:teuthology.orchestra.run.smithi152.stdout: "mdsmap_epoch": 15, 2024-04-20T11:28:25.153 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_epoch": 34, 2024-04-20T11:28:25.154 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_epoch_barrier": 34, 2024-04-20T11:28:25.154 INFO:teuthology.orchestra.run.smithi152.stdout: "uptime": 15.531869739999999 2024-04-20T11:28:25.154 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-04-20T11:28:25.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.149+0000 7f8f9a600640 1 -- 172.21.15.152:0/4081937700 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f8f7006c8f0 msgr2=0x7f8f7006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:25.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.149+0000 7f8f9a600640 1 --2- 172.21.15.152:0/4081937700 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f8f7006c8f0 0x7f8f7006edb0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f8f840012e0 tx=0x7f8f84009040 comp rx=0 tx=0).stop 2024-04-20T11:28:25.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.149+0000 7f8f9a600640 1 -- 172.21.15.152:0/4081937700 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8f94161d30 msgr2=0x7f8f9414d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:25.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.149+0000 7f8f9a600640 1 --2- 172.21.15.152:0/4081937700 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8f94161d30 0x7f8f9414d8a0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c034040 tx=0x7f8f8c0046c0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.150+0000 7f8f9a600640 1 -- 172.21.15.152:0/4081937700 shutdown_connections 2024-04-20T11:28:25.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.150+0000 7f8f9a600640 1 --2- 172.21.15.152:0/4081937700 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f8f7006c8f0 0x7f8f7006edb0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.150+0000 7f8f9a600640 1 --2- 172.21.15.152:0/4081937700 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8f94161d30 0x7f8f9414d8a0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.150+0000 7f8f9a600640 1 --2- 172.21.15.152:0/4081937700 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8f940d4ec0 0x7f8f9409ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.150+0000 7f8f9a600640 1 --2- 172.21.15.152:0/4081937700 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8f940d2b80 0x7f8f9409fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.150+0000 7f8f9a600640 1 -- 172.21.15.152:0/4081937700 >> 172.21.15.152:0/4081937700 conn(0x7f8f9408bfd0 msgr2=0x7f8f940b9dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:25.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.150+0000 7f8f9a600640 1 -- 172.21.15.152:0/4081937700 shutdown_connections 2024-04-20T11:28:25.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.150+0000 7f8f9a600640 1 -- 172.21.15.152:0/4081937700 wait complete. 2024-04-20T11:28:25.165 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-20T11:28:25.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.353+0000 7fbf25e00640 1 Processor -- start 2024-04-20T11:28:25.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.353+0000 7fbf25e00640 1 -- start start 2024-04-20T11:28:25.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.355+0000 7fbf25e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fbf2015e890 0x7fbf20162cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:25.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.355+0000 7fbf25e00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf200b0570 con 0x7fbf200add80 2024-04-20T11:28:25.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.355+0000 7fbf25e00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf200b55a0 con 0x7fbf200ae210 2024-04-20T11:28:25.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.355+0000 7fbf25e00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fbf200b5880 con 0x7fbf2015e890 2024-04-20T11:28:25.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.355+0000 7fbf1fe00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fbf200add80 0x7fbf200af750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:58020/0 (socket says 172.21.15.152:58020) 2024-04-20T11:28:25.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.355+0000 7fbf1fe00640 1 -- 172.21.15.152:0/4099933654 learned_addr learned my addr 172.21.15.152:0/4099933654 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:28:25.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.356+0000 7fbf24a00640 1 --2- 172.21.15.152:0/4099933654 >> v2:172.21.15.175:3300/0 conn(0x7fbf2015e890 0x7fbf20162cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:25.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.356+0000 7fbf1f400640 1 -- 172.21.15.152:0/4099933654 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2404794268 0 0) 0x7fbf200b0570 con 0x7fbf200add80 2024-04-20T11:28:25.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.356+0000 7fbf1f400640 1 -- 172.21.15.152:0/4099933654 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbf08003520 con 0x7fbf200add80 2024-04-20T11:28:25.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.356+0000 7fbf1f400640 1 -- 172.21.15.152:0/4099933654 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3524274954 0 0) 0x7fbf08003520 con 0x7fbf200add80 2024-04-20T11:28:25.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.356+0000 7fbf1f400640 1 -- 172.21.15.152:0/4099933654 >> v2:172.21.15.175:3300/0 conn(0x7fbf2015e890 msgr2=0x7fbf20162cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:25.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.356+0000 7fbf1f400640 1 --2- 172.21.15.152:0/4099933654 >> v2:172.21.15.175:3300/0 conn(0x7fbf2015e890 0x7fbf20162cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.356+0000 7fbf1f400640 1 -- 172.21.15.152:0/4099933654 >> v1:172.21.15.175:6789/0 conn(0x7fbf200ae210 legacy=0x7fbf200afe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:25.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.356+0000 7fbf1f400640 1 -- 172.21.15.152:0/4099933654 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf200b5b60 con 0x7fbf200add80 2024-04-20T11:28:25.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.356+0000 7fbf1f400640 1 -- 172.21.15.152:0/4099933654 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7fbf0c001670 con 0x7fbf200add80 2024-04-20T11:28:25.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.356+0000 7fbf1f400640 1 -- 172.21.15.152:0/4099933654 >> v1:172.21.15.152:6789/0 conn(0x7fbf200add80 legacy=0x7fbf200af750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:25.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.356+0000 7fbf1f400640 1 --2- 172.21.15.152:0/4099933654 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbf08003a00 0x7fbf08003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:25.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.356+0000 7fbf1f400640 1 --2- 172.21.15.152:0/4099933654 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbf08004800 0x7fbf08004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:25.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.356+0000 7fbf1f400640 1 --2- 172.21.15.152:0/4099933654 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbf08005210 0x7fbf0800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:25.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.357+0000 7fbf1f400640 1 -- 172.21.15.152:0/4099933654 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fbf200b5880 con 0x7fbf08005210 2024-04-20T11:28:25.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.357+0000 7fbf1f400640 1 -- 172.21.15.152:0/4099933654 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbf0800c160 con 0x7fbf08004800 2024-04-20T11:28:25.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.357+0000 7fbf1f400640 1 -- 172.21.15.152:0/4099933654 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fbf0800c360 con 0x7fbf08003a00 2024-04-20T11:28:25.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.357+0000 7fbf25400640 1 --2- 172.21.15.152:0/4099933654 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbf08005210 0x7fbf0800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:25.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.357+0000 7fbf24a00640 1 --2- 172.21.15.152:0/4099933654 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbf08003a00 0x7fbf08003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:25.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.357+0000 7fbf25400640 1 -- 172.21.15.152:0/4099933654 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbf08003a00 msgr2=0x7fbf08003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:25.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.357+0000 7fbf25400640 1 --2- 172.21.15.152:0/4099933654 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbf08003a00 0x7fbf08003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.357+0000 7fbf25400640 1 -- 172.21.15.152:0/4099933654 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbf08004800 msgr2=0x7fbf08004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:25.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.357+0000 7fbf25400640 1 --2- 172.21.15.152:0/4099933654 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbf08004800 0x7fbf08004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.357+0000 7fbf25400640 1 -- 172.21.15.152:0/4099933654 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf08003520 con 0x7fbf08005210 2024-04-20T11:28:25.359 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.357+0000 7fbf24a00640 1 --2- 172.21.15.152:0/4099933654 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbf08003a00 0x7fbf08003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:28:25.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.358+0000 7fbf25400640 1 -- 172.21.15.152:0/4099933654 reap_dead start 2024-04-20T11:28:25.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.358+0000 7fbf25400640 1 --2- 172.21.15.152:0/4099933654 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbf08005210 0x7fbf0800b9c0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fbf18001e20 tx=0x7fbf08005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=448619de6e01a2b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:25.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.358+0000 7fbf1f400640 1 -- 172.21.15.152:0/4099933654 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf18001a70 con 0x7fbf08005210 2024-04-20T11:28:25.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.359+0000 7fbf1f400640 1 -- 172.21.15.152:0/4099933654 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf18003b70 con 0x7fbf08005210 2024-04-20T11:28:25.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.359+0000 7fbf1f400640 1 -- 172.21.15.152:0/4099933654 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf180050d0 con 0x7fbf08005210 2024-04-20T11:28:25.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.359+0000 7fbf25e00640 1 -- 172.21.15.152:0/4099933654 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbf08005210 msgr2=0x7fbf0800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:25.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.359+0000 7fbf25e00640 1 --2- 172.21.15.152:0/4099933654 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbf08005210 0x7fbf0800b9c0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fbf18001e20 tx=0x7fbf08005830 comp rx=0 tx=0).stop 2024-04-20T11:28:25.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.359+0000 7fbf25e00640 1 -- 172.21.15.152:0/4099933654 shutdown_connections 2024-04-20T11:28:25.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.359+0000 7fbf25e00640 1 --2- 172.21.15.152:0/4099933654 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbf08005210 0x7fbf0800b9c0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.359+0000 7fbf25e00640 1 -- 172.21.15.152:0/4099933654 >> 172.21.15.152:0/4099933654 conn(0x7fbf20073340 msgr2=0x7fbf2006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:25.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.359+0000 7fbf25e00640 1 -- 172.21.15.152:0/4099933654 shutdown_connections 2024-04-20T11:28:25.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.359+0000 7fbf25e00640 1 -- 172.21.15.152:0/4099933654 wait complete. 2024-04-20T11:28:25.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.361+0000 7fbf25e00640 1 Processor -- start 2024-04-20T11:28:25.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.361+0000 7fbf25e00640 1 -- start start 2024-04-20T11:28:25.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.362+0000 7fbf25e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbf08005210 0x7fbf2014add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:25.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.362+0000 7fbf25e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbf20146e30 0x7fbf201f45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:25.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.362+0000 7fbf25e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbf2015c910 0x7fbf2015cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:25.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.362+0000 7fbf25e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fbf200b5250 con 0x7fbf08005210 2024-04-20T11:28:25.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.362+0000 7fbf25e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbf200ae620 con 0x7fbf2015c910 2024-04-20T11:28:25.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.362+0000 7fbf25e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fbf200b6640 con 0x7fbf20146e30 2024-04-20T11:28:25.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.362+0000 7fbf24a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbf08005210 0x7fbf2014add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:25.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.362+0000 7fbf24a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbf08005210 0x7fbf2014add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:41306/0 (socket says 172.21.15.152:41306) 2024-04-20T11:28:25.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.362+0000 7fbf24a00640 1 -- 172.21.15.152:0/3358851907 learned_addr learned my addr 172.21.15.152:0/3358851907 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:25.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.363+0000 7fbf1fe00640 1 --2- 172.21.15.152:0/3358851907 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbf20146e30 0x7fbf201f45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:25.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.363+0000 7fbf25400640 1 --2- 172.21.15.152:0/3358851907 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbf2015c910 0x7fbf2015cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:25.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.363+0000 7fbf24a00640 1 -- 172.21.15.152:0/3358851907 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbf20146e30 msgr2=0x7fbf201f45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:25.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.363+0000 7fbf24a00640 1 --2- 172.21.15.152:0/3358851907 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbf20146e30 0x7fbf201f45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.363+0000 7fbf24a00640 1 -- 172.21.15.152:0/3358851907 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbf2015c910 msgr2=0x7fbf2015cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:25.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.363+0000 7fbf24a00640 1 --2- 172.21.15.152:0/3358851907 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbf2015c910 0x7fbf2015cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.363+0000 7fbf24a00640 1 -- 172.21.15.152:0/3358851907 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf20147660 con 0x7fbf08005210 2024-04-20T11:28:25.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.363+0000 7fbf24a00640 1 --2- 172.21.15.152:0/3358851907 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbf08005210 0x7fbf2014add0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fbf140108e0 tx=0x7fbf14010da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:25.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.363+0000 7fbf1cc00640 1 -- 172.21.15.152:0/3358851907 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf1400f040 con 0x7fbf08005210 2024-04-20T11:28:25.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.364+0000 7fbf1cc00640 1 -- 172.21.15.152:0/3358851907 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf1400ec20 con 0x7fbf08005210 2024-04-20T11:28:25.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.364+0000 7fbf25e00640 1 -- 172.21.15.152:0/3358851907 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf20147940 con 0x7fbf08005210 2024-04-20T11:28:25.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.364+0000 7fbf1cc00640 1 -- 172.21.15.152:0/3358851907 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf1401c400 con 0x7fbf08005210 2024-04-20T11:28:25.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.364+0000 7fbf25e00640 1 -- 172.21.15.152:0/3358851907 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf20144da0 con 0x7fbf08005210 2024-04-20T11:28:25.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.365+0000 7fbf25e00640 1 -- 172.21.15.152:0/3358851907 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf2015d2d0 con 0x7fbf08005210 2024-04-20T11:28:25.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.366+0000 7fbf1cc00640 1 -- 172.21.15.152:0/3358851907 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbf1401c5a0 con 0x7fbf08005210 2024-04-20T11:28:25.366 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.367+0000 7fbf1cc00640 1 --2- 172.21.15.152:0/3358851907 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fbf0006c8f0 0x7fbf0006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:25.366 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.367+0000 7fbf1cc00640 1 -- 172.21.15.152:0/3358851907 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fbf1408c360 con 0x7fbf08005210 2024-04-20T11:28:25.366 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.367+0000 7fbf1fe00640 1 --2- 172.21.15.152:0/3358851907 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fbf0006c8f0 0x7fbf0006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:25.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.368+0000 7fbf1fe00640 1 --2- 172.21.15.152:0/3358851907 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fbf0006c8f0 0x7fbf0006edb0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c0018a0 tx=0x7fbf0c0016f0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:25.370 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.371+0000 7fbf1cc00640 1 -- 172.21.15.152:0/3358851907 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fbf14018030 con 0x7fbf08005210 2024-04-20T11:28:25.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.597+0000 7fbf25e00640 1 -- 172.21.15.152:0/3358851907 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fbf20075e60 con 0x7fbf08005210 2024-04-20T11:28:25.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.598+0000 7fbf1cc00640 1 -- 172.21.15.152:0/3358851907 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v15) ==== 53+0+73 (secure 0 0 0) 0x7fbf1405a770 con 0x7fbf08005210 2024-04-20T11:28:25.598 INFO:teuthology.orchestra.run.smithi152.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-20T11:28:25.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.601+0000 7fbf25e00640 1 -- 172.21.15.152:0/3358851907 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fbf0006c8f0 msgr2=0x7fbf0006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:25.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.601+0000 7fbf25e00640 1 --2- 172.21.15.152:0/3358851907 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fbf0006c8f0 0x7fbf0006edb0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c0018a0 tx=0x7fbf0c0016f0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.601+0000 7fbf25e00640 1 -- 172.21.15.152:0/3358851907 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbf08005210 msgr2=0x7fbf2014add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:25.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.601+0000 7fbf25e00640 1 --2- 172.21.15.152:0/3358851907 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbf08005210 0x7fbf2014add0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fbf140108e0 tx=0x7fbf14010da0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.601+0000 7fbf25e00640 1 -- 172.21.15.152:0/3358851907 shutdown_connections 2024-04-20T11:28:25.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.601+0000 7fbf25e00640 1 --2- 172.21.15.152:0/3358851907 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fbf0006c8f0 0x7fbf0006edb0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.601+0000 7fbf25e00640 1 --2- 172.21.15.152:0/3358851907 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbf2015c910 0x7fbf2015cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.601+0000 7fbf25e00640 1 --2- 172.21.15.152:0/3358851907 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbf20146e30 0x7fbf201f45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.601+0000 7fbf25e00640 1 --2- 172.21.15.152:0/3358851907 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbf08005210 0x7fbf2014add0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.601+0000 7fbf25e00640 1 -- 172.21.15.152:0/3358851907 >> 172.21.15.152:0/3358851907 conn(0x7fbf20073340 msgr2=0x7fbf2006e500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:25.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.601+0000 7fbf25e00640 1 -- 172.21.15.152:0/3358851907 shutdown_connections 2024-04-20T11:28:25.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25.601+0000 7fbf25e00640 1 -- 172.21.15.152:0/3358851907 wait complete. 2024-04-20T11:28:25.618 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-20T11:28:25.618 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-20T11:28:25.618 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi152.front.sepia.ceph.com 2024-04-20T11:28:25.618 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-20T11:28:25.618 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-20T11:28:25.619 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-20T11:28:25.619 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-20T11:28:25.619 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-20T11:28:25.619 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-20T11:28:25.619 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:28:25.619 DEBUG:teuthology.orchestra.run.smithi152:> ip addr 2024-04-20T11:28:25.669 INFO:teuthology.orchestra.run.smithi152.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-20T11:28:25.669 INFO:teuthology.orchestra.run.smithi152.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-20T11:28:25.669 INFO:teuthology.orchestra.run.smithi152.stdout: inet 127.0.0.1/8 scope host lo 2024-04-20T11:28:25.669 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:28:25.670 INFO:teuthology.orchestra.run.smithi152.stdout: inet6 ::1/128 scope host 2024-04-20T11:28:25.670 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:28:25.670 INFO:teuthology.orchestra.run.smithi152.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T11:28:25.670 INFO:teuthology.orchestra.run.smithi152.stdout: link/ether 0c:c4:7a:da:6c:44 brd ff:ff:ff:ff:ff:ff 2024-04-20T11:28:25.670 INFO:teuthology.orchestra.run.smithi152.stdout: altname enp7s0f0 2024-04-20T11:28:25.670 INFO:teuthology.orchestra.run.smithi152.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T11:28:25.670 INFO:teuthology.orchestra.run.smithi152.stdout: link/ether 0c:c4:7a:da:6c:45 brd ff:ff:ff:ff:ff:ff 2024-04-20T11:28:25.670 INFO:teuthology.orchestra.run.smithi152.stdout: altname enp7s0f1 2024-04-20T11:28:25.670 INFO:teuthology.orchestra.run.smithi152.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T11:28:25.670 INFO:teuthology.orchestra.run.smithi152.stdout: link/ether 0c:c4:7a:88:6d:e2 brd ff:ff:ff:ff:ff:ff 2024-04-20T11:28:25.670 INFO:teuthology.orchestra.run.smithi152.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-20T11:28:25.670 INFO:teuthology.orchestra.run.smithi152.stdout: link/ether 0c:c4:7a:88:6d:e3 brd ff:ff:ff:ff:ff:ff 2024-04-20T11:28:25.670 INFO:teuthology.orchestra.run.smithi152.stdout: inet 172.21.15.152/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-20T11:28:25.671 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft 42700sec preferred_lft 42700sec 2024-04-20T11:28:25.671 INFO:teuthology.orchestra.run.smithi152.stdout: inet6 fe80::ec4:7aff:fe88:6de3/64 scope link 2024-04-20T11:28:25.671 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:28:25.671 INFO:teuthology.orchestra.run.smithi152.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-20T11:28:25.671 INFO:teuthology.orchestra.run.smithi152.stdout: link/ether 06:3e:9a:d3:49:1e brd ff:ff:ff:ff:ff:ff 2024-04-20T11:28:25.671 INFO:teuthology.orchestra.run.smithi152.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-20T11:28:25.671 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:28:25.671 INFO:teuthology.orchestra.run.smithi152.stdout: inet6 fe80::a019:d8ff:fe70:1185/64 scope link 2024-04-20T11:28:25.671 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:28:25.671 INFO:teuthology.orchestra.run.smithi152.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-20T11:28:25.671 INFO:teuthology.orchestra.run.smithi152.stdout: link/ether 16:e5:08:78:20:47 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T11:28:25.671 INFO:teuthology.orchestra.run.smithi152.stdout: inet6 fe80::14e5:8ff:fe78:2047/64 scope link 2024-04-20T11:28:25.671 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:28:25.671 INFO:teuthology.orchestra.run.smithi152.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-20T11:28:25.671 INFO:teuthology.orchestra.run.smithi152.stdout: link/ether 06:3e:9a:d3:49:1e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T11:28:25.672 INFO:teuthology.orchestra.run.smithi152.stdout: inet6 fe80::43e:9aff:fed3:491e/64 scope link 2024-04-20T11:28:25.672 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:28:25.672 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:28:25.672 DEBUG:teuthology.orchestra.run.smithi152:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-20T11:28:25.747 INFO:teuthology.orchestra.run.smithi152.stdout:1 2024-04-20T11:28:25.752 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:28:25.752 DEBUG:teuthology.orchestra.run.smithi152:> route 2024-04-20T11:28:25.805 INFO:teuthology.orchestra.run.smithi152.stdout:Kernel IP routing table 2024-04-20T11:28:25.805 INFO:teuthology.orchestra.run.smithi152.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-20T11:28:25.805 INFO:teuthology.orchestra.run.smithi152.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-20T11:28:25.805 INFO:teuthology.orchestra.run.smithi152.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-20T11:28:25.806 INFO:teuthology.orchestra.run.smithi152.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-20T11:28:25.807 DEBUG:teuthology.orchestra.run.smithi152:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T11:28:25.807 DEBUG:teuthology.orchestra.run.smithi152:> set -e 2024-04-20T11:28:25.807 DEBUG:teuthology.orchestra.run.smithi152:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-20T11:28:25.807 DEBUG:teuthology.orchestra.run.smithi152:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-20T11:28:25.807 DEBUG:teuthology.orchestra.run.smithi152:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-20T11:28:25.807 DEBUG:teuthology.orchestra.run.smithi152:> ') 2024-04-20T11:28:25.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T11:28:26.009 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T11:28:26.020 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:28:26.020 DEBUG:teuthology.orchestra.run.smithi152:> ip netns list 2024-04-20T11:28:26.068 INFO:teuthology.orchestra.run.smithi152.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-20T11:28:26.068 INFO:teuthology.orchestra.run.smithi152.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-20T11:28:26.069 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-20T11:28:26.069 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:28:26.069 DEBUG:teuthology.orchestra.run.smithi152:> sudo ip link set brx.0 up 2024-04-20T11:28:26.153 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-20T11:28:26.153 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T11:28:26.153 DEBUG:teuthology.orchestra.run.smithi152:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-20T11:28:26.204 INFO:teuthology.orchestra.run.smithi152.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-20T11:28:26.204 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T11:28:26.204 DEBUG:teuthology.orchestra.run.smithi152:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-20T11:28:26.253 DEBUG:teuthology.orchestra.run.smithi152:> sudo modprobe fuse 2024-04-20T11:28:26.335 DEBUG:teuthology.orchestra.run.smithi152:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T11:28:26.387 INFO:teuthology.orchestra.run.smithi152.stdout:/proc 2024-04-20T11:28:26.387 INFO:teuthology.orchestra.run.smithi152.stdout:/sys 2024-04-20T11:28:26.387 INFO:teuthology.orchestra.run.smithi152.stdout:/dev 2024-04-20T11:28:26.388 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/security 2024-04-20T11:28:26.388 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/shm 2024-04-20T11:28:26.388 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/pts 2024-04-20T11:28:26.388 INFO:teuthology.orchestra.run.smithi152.stdout:/run 2024-04-20T11:28:26.388 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/cgroup 2024-04-20T11:28:26.388 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/pstore 2024-04-20T11:28:26.388 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/bpf 2024-04-20T11:28:26.388 INFO:teuthology.orchestra.run.smithi152.stdout:/ 2024-04-20T11:28:26.388 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/selinux 2024-04-20T11:28:26.388 INFO:teuthology.orchestra.run.smithi152.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T11:28:26.388 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/mqueue 2024-04-20T11:28:26.388 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/hugepages 2024-04-20T11:28:26.389 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/debug 2024-04-20T11:28:26.389 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/tracing 2024-04-20T11:28:26.389 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/fuse/connections 2024-04-20T11:28:26.389 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/config 2024-04-20T11:28:26.389 INFO:teuthology.orchestra.run.smithi152.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T11:28:26.389 INFO:teuthology.orchestra.run.smithi152.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T11:28:26.389 INFO:teuthology.orchestra.run.smithi152.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T11:28:26.389 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T11:28:26.389 INFO:teuthology.orchestra.run.smithi152.stdout:/run/user/1000 2024-04-20T11:28:26.389 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph 2024-04-20T11:28:26.389 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph/osd/ceph-0 2024-04-20T11:28:26.389 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph/osd/ceph-1 2024-04-20T11:28:26.389 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph/osd/ceph-2 2024-04-20T11:28:26.390 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph/osd/ceph-3 2024-04-20T11:28:26.390 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns 2024-04-20T11:28:26.390 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T11:28:26.390 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T11:28:26.390 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T11:28:26.390 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T11:28:26.391 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:28:26.391 DEBUG:teuthology.orchestra.run.smithi152:> ls /sys/fs/fuse/connections 2024-04-20T11:28:26.440 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-20T11:28:26.440 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-20T11:28:26.447 DEBUG:teuthology.orchestra.run.smithi152:> sudo modprobe fuse 2024-04-20T11:28:26.519 DEBUG:teuthology.orchestra.run.smithi152:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T11:28:26.560 INFO:teuthology.orchestra.run.smithi152.stdout:/proc 2024-04-20T11:28:26.560 INFO:teuthology.orchestra.run.smithi152.stdout:/sys 2024-04-20T11:28:26.560 INFO:teuthology.orchestra.run.smithi152.stdout:/dev 2024-04-20T11:28:26.560 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/security 2024-04-20T11:28:26.560 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/shm 2024-04-20T11:28:26.560 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/pts 2024-04-20T11:28:26.561 INFO:teuthology.orchestra.run.smithi152.stdout:/run 2024-04-20T11:28:26.561 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/cgroup 2024-04-20T11:28:26.561 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/pstore 2024-04-20T11:28:26.561 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/bpf 2024-04-20T11:28:26.561 INFO:teuthology.orchestra.run.smithi152.stdout:/ 2024-04-20T11:28:26.561 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/selinux 2024-04-20T11:28:26.561 INFO:teuthology.orchestra.run.smithi152.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T11:28:26.561 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/mqueue 2024-04-20T11:28:26.561 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/hugepages 2024-04-20T11:28:26.561 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/debug 2024-04-20T11:28:26.561 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/tracing 2024-04-20T11:28:26.561 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/fuse/connections 2024-04-20T11:28:26.561 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/config 2024-04-20T11:28:26.561 INFO:teuthology.orchestra.run.smithi152.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T11:28:26.562 INFO:teuthology.orchestra.run.smithi152.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T11:28:26.562 INFO:teuthology.orchestra.run.smithi152.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T11:28:26.562 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T11:28:26.562 INFO:teuthology.orchestra.run.smithi152.stdout:/run/user/1000 2024-04-20T11:28:26.562 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph 2024-04-20T11:28:26.562 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph/osd/ceph-0 2024-04-20T11:28:26.562 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph/osd/ceph-1 2024-04-20T11:28:26.562 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph/osd/ceph-2 2024-04-20T11:28:26.562 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph/osd/ceph-3 2024-04-20T11:28:26.562 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns 2024-04-20T11:28:26.562 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T11:28:26.562 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T11:28:26.562 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T11:28:26.563 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T11:28:26.563 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:28:26.564 DEBUG:teuthology.orchestra.run.smithi152:> ls /sys/fs/fuse/connections 2024-04-20T11:28:26.621 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi152.stderr:2024-04-20T11:28:26.621+0000 7f4f9c63a480 -1 init, newargv = 0x55d62298d1c0 newargc=15 2024-04-20T11:28:26.621 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi152.stderr:ceph-fuse[36577]: starting ceph client 2024-04-20T11:28:26.630 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi152.stderr:ceph-fuse[36577]: starting fuse 2024-04-20T11:28:26.648 INFO:teuthology.orchestra.run.smithi152.stdout:42 2024-04-20T11:28:26.648 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-04-20T11:28:26.649 DEBUG:teuthology.orchestra.run.smithi152:> sudo stdin-killer -- python3 -c ' 2024-04-20T11:28:26.649 DEBUG:teuthology.orchestra.run.smithi152:> import glob 2024-04-20T11:28:26.649 DEBUG:teuthology.orchestra.run.smithi152:> import re 2024-04-20T11:28:26.649 DEBUG:teuthology.orchestra.run.smithi152:> import os 2024-04-20T11:28:26.649 DEBUG:teuthology.orchestra.run.smithi152:> import subprocess 2024-04-20T11:28:26.649 DEBUG:teuthology.orchestra.run.smithi152:> 2024-04-20T11:28:26.649 DEBUG:teuthology.orchestra.run.smithi152:> def _find_admin_socket(client_name): 2024-04-20T11:28:26.649 DEBUG:teuthology.orchestra.run.smithi152:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-20T11:28:26.649 DEBUG:teuthology.orchestra.run.smithi152:> files = glob.glob(asok_path) 2024-04-20T11:28:26.649 DEBUG:teuthology.orchestra.run.smithi152:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-20T11:28:26.649 DEBUG:teuthology.orchestra.run.smithi152:> 2024-04-20T11:28:26.649 DEBUG:teuthology.orchestra.run.smithi152:> # Given a non-glob path, it better be there 2024-04-20T11:28:26.649 DEBUG:teuthology.orchestra.run.smithi152:> if "*" not in asok_path: 2024-04-20T11:28:26.649 DEBUG:teuthology.orchestra.run.smithi152:> assert(len(files) == 1) 2024-04-20T11:28:26.650 DEBUG:teuthology.orchestra.run.smithi152:> return files[0] 2024-04-20T11:28:26.650 DEBUG:teuthology.orchestra.run.smithi152:> 2024-04-20T11:28:26.650 DEBUG:teuthology.orchestra.run.smithi152:> for f in files: 2024-04-20T11:28:26.650 DEBUG:teuthology.orchestra.run.smithi152:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-20T11:28:26.650 DEBUG:teuthology.orchestra.run.smithi152:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-20T11:28:26.650 DEBUG:teuthology.orchestra.run.smithi152:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-20T11:28:26.650 DEBUG:teuthology.orchestra.run.smithi152:> contents = proc_f.read() 2024-04-20T11:28:26.650 DEBUG:teuthology.orchestra.run.smithi152:> if mountpoint in contents: 2024-04-20T11:28:26.650 DEBUG:teuthology.orchestra.run.smithi152:> return f 2024-04-20T11:28:26.650 DEBUG:teuthology.orchestra.run.smithi152:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-20T11:28:26.650 DEBUG:teuthology.orchestra.run.smithi152:> 2024-04-20T11:28:26.650 DEBUG:teuthology.orchestra.run.smithi152:> print(_find_admin_socket("client.0")) 2024-04-20T11:28:26.650 DEBUG:teuthology.orchestra.run.smithi152:> ' 2024-04-20T11:28:26.844 INFO:teuthology.orchestra.run.smithi152.stdout:/var/run/ceph/ceph-client.0.36577.asok 2024-04-20T11:28:26.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T11:28:26.861 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.36577.asok 2024-04-20T11:28:26.862 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:28:26.862 DEBUG:teuthology.orchestra.run.smithi152:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.36577.asok status 2024-04-20T11:28:27.091 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-04-20T11:28:27.091 INFO:teuthology.orchestra.run.smithi152.stdout: "metadata": { 2024-04-20T11:28:27.091 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_sha1": "a8c84f5997950fd7d008204839253291d52e6eda", 2024-04-20T11:28:27.092 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", 2024-04-20T11:28:27.092 INFO:teuthology.orchestra.run.smithi152.stdout: "entity_id": "0", 2024-04-20T11:28:27.092 INFO:teuthology.orchestra.run.smithi152.stdout: "hostname": "smithi152", 2024-04-20T11:28:27.092 INFO:teuthology.orchestra.run.smithi152.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-20T11:28:27.092 INFO:teuthology.orchestra.run.smithi152.stdout: "pid": "36577", 2024-04-20T11:28:27.092 INFO:teuthology.orchestra.run.smithi152.stdout: "root": "/" 2024-04-20T11:28:27.092 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:28:27.092 INFO:teuthology.orchestra.run.smithi152.stdout: "dentry_count": 0, 2024-04-20T11:28:27.092 INFO:teuthology.orchestra.run.smithi152.stdout: "dentry_pinned_count": 0, 2024-04-20T11:28:27.092 INFO:teuthology.orchestra.run.smithi152.stdout: "id": 5427, 2024-04-20T11:28:27.092 INFO:teuthology.orchestra.run.smithi152.stdout: "inst": { 2024-04-20T11:28:27.092 INFO:teuthology.orchestra.run.smithi152.stdout: "name": { 2024-04-20T11:28:27.092 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "client", 2024-04-20T11:28:27.092 INFO:teuthology.orchestra.run.smithi152.stdout: "num": 5427 2024-04-20T11:28:27.092 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:28:27.093 INFO:teuthology.orchestra.run.smithi152.stdout: "addr": { 2024-04-20T11:28:27.093 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "v1", 2024-04-20T11:28:27.093 INFO:teuthology.orchestra.run.smithi152.stdout: "addr": "192.168.0.1:0", 2024-04-20T11:28:27.093 INFO:teuthology.orchestra.run.smithi152.stdout: "nonce": 1027914885 2024-04-20T11:28:27.093 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:28:27.093 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:28:27.093 INFO:teuthology.orchestra.run.smithi152.stdout: "addr": { 2024-04-20T11:28:27.093 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "v1", 2024-04-20T11:28:27.093 INFO:teuthology.orchestra.run.smithi152.stdout: "addr": "192.168.0.1:0", 2024-04-20T11:28:27.093 INFO:teuthology.orchestra.run.smithi152.stdout: "nonce": 1027914885 2024-04-20T11:28:27.093 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:28:27.093 INFO:teuthology.orchestra.run.smithi152.stdout: "inst_str": "client.5427 192.168.0.1:0/1027914885", 2024-04-20T11:28:27.093 INFO:teuthology.orchestra.run.smithi152.stdout: "addr_str": "192.168.0.1:0/1027914885", 2024-04-20T11:28:27.093 INFO:teuthology.orchestra.run.smithi152.stdout: "inode_count": 1, 2024-04-20T11:28:27.094 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_epoch": 15, 2024-04-20T11:28:27.094 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_epoch": 34, 2024-04-20T11:28:27.094 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_epoch_barrier": 0, 2024-04-20T11:28:27.094 INFO:teuthology.orchestra.run.smithi152.stdout: "blocklisted": false, 2024-04-20T11:28:27.094 INFO:teuthology.orchestra.run.smithi152.stdout: "fs_name": "cephfs" 2024-04-20T11:28:27.094 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-04-20T11:28:27.107 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:28:27.107 DEBUG:teuthology.orchestra.run.smithi152:> stat --file-system '--printf=%T 2024-04-20T11:28:27.107 DEBUG:teuthology.orchestra.run.smithi152:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-20T11:28:27.161 INFO:teuthology.orchestra.run.smithi152.stdout:fuseblk 2024-04-20T11:28:27.161 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-20T11:28:27.161 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:28:27.161 DEBUG:teuthology.orchestra.run.smithi152:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-20T11:28:27.247 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-20T11:28:27.441 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:27.441+0000 7f5ef5a00640 1 Processor -- start 2024-04-20T11:28:27.441 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:27.441+0000 7f5ef5a00640 1 -- start start 2024-04-20T11:28:27.443 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:27.442+0000 7f5ef5a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f5ef0147410 0x7f5ef00afa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:27.443 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:27.442+0000 7f5ef5a00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f5ef0071cd0 0x7f5ef0072130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:27.443 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:27.442+0000 7f5ef5a00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ef00aec90 con 0x7f5ef006e010 2024-04-20T11:28:27.443 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:27.443+0000 7f5ef5a00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f5ef0072670 con 0x7f5ef0147410 2024-04-20T11:28:27.443 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:27.443+0000 7f5ef5a00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f5ef00aef70 con 0x7f5ef0071cd0 2024-04-20T11:28:27.443 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:27.443+0000 7f5eefe00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f5ef0147410 0x7f5ef00afa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:27.443 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:27.443+0000 7f5ef5000640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f5ef006e010 0x7f5ef006e4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:40258/0 (socket says 172.21.15.152:40258) 2024-04-20T11:28:27.443 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:27.443+0000 7f5ef5000640 1 -- 172.21.15.152:0/1649835587 learned_addr learned my addr 172.21.15.152:0/1649835587 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:28:27.443 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:27.443+0000 7f5eefe00640 1 -- 172.21.15.152:0/1649835587 >> v2:172.21.15.175:3300/0 conn(0x7f5ef0071cd0 msgr2=0x7f5ef0072130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:27.443 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:27.443+0000 7f5eefe00640 1 --2- 172.21.15.152:0/1649835587 >> v2:172.21.15.175:3300/0 conn(0x7f5ef0071cd0 0x7f5ef0072130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:27.443 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:27.443+0000 7f5eefe00640 1 -- 172.21.15.152:0/1649835587 >> v1:172.21.15.152:6789/0 conn(0x7f5ef006e010 legacy=0x7f5ef006e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:27.444 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:27.443+0000 7f5eefe00640 1 -- 172.21.15.152:0/1649835587 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ef00af250 con 0x7f5ef0147410 2024-04-20T11:28:27.444 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:27.443+0000 7f5eeea00640 1 -- 172.21.15.152:0/1649835587 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2362900764 0 0) 0x7f5ef00aec90 con 0x7f5ef006e010 2024-04-20T11:28:27.445 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:27.444+0000 7f5eefe00640 1 --2- 172.21.15.152:0/1649835587 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5ef0147410 0x7f5ef00afa50 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f5ef0073170 tx=0x7f5ee002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b726c603488bd092 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:27.445 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:27.445+0000 7f5eeea00640 1 -- 172.21.15.152:0/1649835587 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ee0035020 con 0x7f5ef0147410 2024-04-20T11:28:27.445 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:27.445+0000 7f5eeea00640 1 -- 172.21.15.152:0/1649835587 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ee0031830 con 0x7f5ef0147410 2024-04-20T11:28:27.445 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:27.445+0000 7f5eeea00640 1 -- 172.21.15.152:0/1649835587 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ee0031c00 con 0x7f5ef0147410 2024-04-20T11:28:27.445 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:27.445+0000 7f5ef5a00640 1 -- 172.21.15.152:0/1649835587 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5ef0147410 msgr2=0x7f5ef00afa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:27.445 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:27.445+0000 7f5ef5a00640 1 --2- 172.21.15.152:0/1649835587 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5ef0147410 0x7f5ef00afa50 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f5ef0073170 tx=0x7f5ee002b8d0 comp rx=0 tx=0).stop 2024-04-20T11:28:27.445 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:27.445+0000 7f5ef5a00640 1 -- 172.21.15.152:0/1649835587 shutdown_connections 2024-04-20T11:28:27.445 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:27.445+0000 7f5ef5a00640 1 --2- 172.21.15.152:0/1649835587 >> v2:172.21.15.175:3300/0 conn(0x7f5ef0071cd0 0x7f5ef0072130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:27.445 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:27.445+0000 7f5ef5a00640 1 --2- 172.21.15.152:0/1649835587 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5ef0147410 0x7f5ef00afa50 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:27.445 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:27.445+0000 7f5ef5a00640 1 -- 172.21.15.152:0/1649835587 >> 172.21.15.152:0/1649835587 conn(0x7f5ef0067ae0 msgr2=0x7f5ef006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:27.445 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:27.445+0000 7f5ef5a00640 1 -- 172.21.15.152:0/1649835587 shutdown_connections 2024-04-20T11:28:27.445 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:27.445+0000 7f5ef5a00640 1 -- 172.21.15.152:0/1649835587 wait complete. 2024-04-20T11:28:27.446 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:27.447+0000 7f5ef5a00640 1 Processor -- start 2024-04-20T11:28:27.447 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:27.447+0000 7f5ef5a00640 1 -- start start 2024-04-20T11:28:27.448 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:27.448+0000 7f5ef5a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ef006e010 0x7f5ef0136bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:27.448 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:27.448+0000 7f5ef5a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5ef0071cd0 0x7f5ef00c67e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:27.448 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:27.448+0000 7f5ef5a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ef0147410 0x7f5ef0144eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:27.448 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:27.448+0000 7f5ef5a00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef00b3390 con 0x7f5ef0071cd0 2024-04-20T11:28:27.448 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:27.448+0000 7f5ef5a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef00ae980 con 0x7f5ef0147410 2024-04-20T11:28:27.448 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:27.448+0000 7f5ef5a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5ef00b3510 con 0x7f5ef006e010 2024-04-20T11:28:27.448 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:27.449+0000 7f5eef400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5ef0071cd0 0x7f5ef00c67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:27.448 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:27.449+0000 7f5eef400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5ef0071cd0 0x7f5ef00c67e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49106/0 (socket says 172.21.15.152:49106) 2024-04-20T11:28:27.448 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:27.449+0000 7f5eef400640 1 -- 172.21.15.152:0/1515033103 learned_addr learned my addr 172.21.15.152:0/1515033103 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:27.450 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:27.449+0000 7f5eefe00640 1 --2- 172.21.15.152:0/1515033103 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ef006e010 0x7f5ef0136bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:27.450 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:27.449+0000 7f5eef400640 1 -- 172.21.15.152:0/1515033103 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ef006e010 msgr2=0x7f5ef0136bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:27.450 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:27.449+0000 7f5ef5000640 1 --2- 172.21.15.152:0/1515033103 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ef0147410 0x7f5ef0144eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:27.450 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:27.449+0000 7f5eef400640 1 --2- 172.21.15.152:0/1515033103 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ef006e010 0x7f5ef0136bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:27.450 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:27.449+0000 7f5eef400640 1 -- 172.21.15.152:0/1515033103 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ef0147410 msgr2=0x7f5ef0144eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:27.450 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:27.449+0000 7f5eef400640 1 --2- 172.21.15.152:0/1515033103 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ef0147410 0x7f5ef0144eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:27.450 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:27.449+0000 7f5eef400640 1 -- 172.21.15.152:0/1515033103 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ef00c71c0 con 0x7f5ef0071cd0 2024-04-20T11:28:27.450 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:27.449+0000 7f5eef400640 1 --2- 172.21.15.152:0/1515033103 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5ef0071cd0 0x7f5ef00c67e0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f5edc00b4a0 tx=0x7f5edc00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:27.450 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:27.450+0000 7f5ee7e00640 1 -- 172.21.15.152:0/1515033103 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5edc014070 con 0x7f5ef0071cd0 2024-04-20T11:28:27.450 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:27.450+0000 7f5ee7e00640 1 -- 172.21.15.152:0/1515033103 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5edc010610 con 0x7f5ef0071cd0 2024-04-20T11:28:27.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:27.450+0000 7f5ee7e00640 1 -- 172.21.15.152:0/1515033103 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5edc010a00 con 0x7f5ef0071cd0 2024-04-20T11:28:27.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:27.450+0000 7f5ef5a00640 1 -- 172.21.15.152:0/1515033103 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ef0142f50 con 0x7f5ef0071cd0 2024-04-20T11:28:27.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:27.450+0000 7f5ef5a00640 1 -- 172.21.15.152:0/1515033103 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ef01437f0 con 0x7f5ef0071cd0 2024-04-20T11:28:27.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:27.451+0000 7f5ef5a00640 1 -- 172.21.15.152:0/1515033103 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ef0075e60 con 0x7f5ef0071cd0 2024-04-20T11:28:27.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:27.452+0000 7f5ee7e00640 1 -- 172.21.15.152:0/1515033103 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5edc017070 con 0x7f5ef0071cd0 2024-04-20T11:28:27.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:27.453+0000 7f5ee7e00640 1 --2- 172.21.15.152:0/1515033103 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f5ed006c8f0 0x7f5ed006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:27.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:27.453+0000 7f5ee7e00640 1 -- 172.21.15.152:0/1515033103 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f5edc08b600 con 0x7f5ef0071cd0 2024-04-20T11:28:27.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:27.454+0000 7f5eefe00640 1 --2- 172.21.15.152:0/1515033103 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f5ed006c8f0 0x7f5ed006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:27.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:27.454+0000 7f5eefe00640 1 --2- 172.21.15.152:0/1515033103 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f5ed006c8f0 0x7f5ed006edb0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5ee0036000 tx=0x7f5ee0008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:27.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:27.458+0000 7f5ee7e00640 1 -- 172.21.15.152:0/1515033103 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f5edc059a20 con 0x7f5ef0071cd0 2024-04-20T11:28:27.697 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:27.698+0000 7f5ef5a00640 1 -- 172.21.15.152:0/1515033103 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f5ef01439c0 con 0x7f5ef0071cd0 2024-04-20T11:28:27.698 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:27.699+0000 7f5ee7e00640 1 -- 172.21.15.152:0/1515033103 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v15) ==== 53+0+73 (secure 0 0 0) 0x7f5edc059320 con 0x7f5ef0071cd0 2024-04-20T11:28:27.698 INFO:teuthology.orchestra.run.smithi152.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-20T11:28:27.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:27.701+0000 7f5ef5a00640 1 -- 172.21.15.152:0/1515033103 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f5ed006c8f0 msgr2=0x7f5ed006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:27.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:27.701+0000 7f5ef5a00640 1 --2- 172.21.15.152:0/1515033103 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f5ed006c8f0 0x7f5ed006edb0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5ee0036000 tx=0x7f5ee0008030 comp rx=0 tx=0).stop 2024-04-20T11:28:27.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:27.701+0000 7f5ef5a00640 1 -- 172.21.15.152:0/1515033103 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5ef0071cd0 msgr2=0x7f5ef00c67e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:27.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:27.701+0000 7f5ef5a00640 1 --2- 172.21.15.152:0/1515033103 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5ef0071cd0 0x7f5ef00c67e0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f5edc00b4a0 tx=0x7f5edc00b960 comp rx=0 tx=0).stop 2024-04-20T11:28:27.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:27.702+0000 7f5ef5a00640 1 -- 172.21.15.152:0/1515033103 shutdown_connections 2024-04-20T11:28:27.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:27.702+0000 7f5ef5a00640 1 --2- 172.21.15.152:0/1515033103 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f5ed006c8f0 0x7f5ed006edb0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:27.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:27.702+0000 7f5ef5a00640 1 --2- 172.21.15.152:0/1515033103 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ef0147410 0x7f5ef0144eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:27.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:27.702+0000 7f5ef5a00640 1 --2- 172.21.15.152:0/1515033103 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5ef0071cd0 0x7f5ef00c67e0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:27.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:27.702+0000 7f5ef5a00640 1 --2- 172.21.15.152:0/1515033103 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ef006e010 0x7f5ef0136bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:27.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:27.702+0000 7f5ef5a00640 1 -- 172.21.15.152:0/1515033103 >> 172.21.15.152:0/1515033103 conn(0x7f5ef0067ae0 msgr2=0x7f5ef00b3ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:27.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:27.702+0000 7f5ef5a00640 1 -- 172.21.15.152:0/1515033103 shutdown_connections 2024-04-20T11:28:27.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:27.702+0000 7f5ef5a00640 1 -- 172.21.15.152:0/1515033103 wait complete. 2024-04-20T11:28:27.720 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-20T11:28:27.721 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-20T11:28:27.721 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi175.front.sepia.ceph.com 2024-04-20T11:28:27.721 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-20T11:28:27.721 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-20T11:28:27.721 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-20T11:28:27.721 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-20T11:28:27.721 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-20T11:28:27.721 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-20T11:28:27.721 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:28:27.721 DEBUG:teuthology.orchestra.run.smithi175:> ip addr 2024-04-20T11:28:27.781 INFO:teuthology.orchestra.run.smithi175.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-20T11:28:27.781 INFO:teuthology.orchestra.run.smithi175.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-20T11:28:27.781 INFO:teuthology.orchestra.run.smithi175.stdout: inet 127.0.0.1/8 scope host lo 2024-04-20T11:28:27.781 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:28:27.781 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 ::1/128 scope host 2024-04-20T11:28:27.781 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:28:27.781 INFO:teuthology.orchestra.run.smithi175.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T11:28:27.781 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:d9:39:ee brd ff:ff:ff:ff:ff:ff 2024-04-20T11:28:27.781 INFO:teuthology.orchestra.run.smithi175.stdout: altname enp7s0f0 2024-04-20T11:28:27.782 INFO:teuthology.orchestra.run.smithi175.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T11:28:27.782 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:d9:39:ef brd ff:ff:ff:ff:ff:ff 2024-04-20T11:28:27.782 INFO:teuthology.orchestra.run.smithi175.stdout: altname enp7s0f1 2024-04-20T11:28:27.782 INFO:teuthology.orchestra.run.smithi175.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T11:28:27.782 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:88:71:f0 brd ff:ff:ff:ff:ff:ff 2024-04-20T11:28:27.782 INFO:teuthology.orchestra.run.smithi175.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-20T11:28:27.782 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:88:71:f1 brd ff:ff:ff:ff:ff:ff 2024-04-20T11:28:27.782 INFO:teuthology.orchestra.run.smithi175.stdout: inet 172.21.15.175/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-20T11:28:27.782 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft 42673sec preferred_lft 42673sec 2024-04-20T11:28:27.782 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::ec4:7aff:fe88:71f1/64 scope link 2024-04-20T11:28:27.782 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:28:27.782 INFO:teuthology.orchestra.run.smithi175.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-20T11:28:27.782 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 02:d6:c6:31:26:70 brd ff:ff:ff:ff:ff:ff 2024-04-20T11:28:27.782 INFO:teuthology.orchestra.run.smithi175.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-20T11:28:27.783 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:28:27.783 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::5091:d5ff:fee7:54b3/64 scope link 2024-04-20T11:28:27.783 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:28:27.783 INFO:teuthology.orchestra.run.smithi175.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-20T11:28:27.783 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 02:d6:c6:31:26:70 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-20T11:28:27.783 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::d6:c6ff:fe31:2670/64 scope link 2024-04-20T11:28:27.783 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:28:27.783 INFO:teuthology.orchestra.run.smithi175.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-20T11:28:27.783 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 9e:14:f3:e2:18:1e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-20T11:28:27.783 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::9c14:f3ff:fee2:181e/64 scope link 2024-04-20T11:28:27.783 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:28:27.784 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:28:27.784 DEBUG:teuthology.orchestra.run.smithi175:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-20T11:28:27.862 INFO:teuthology.orchestra.run.smithi175.stdout:1 2024-04-20T11:28:27.866 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:28:27.866 DEBUG:teuthology.orchestra.run.smithi175:> route 2024-04-20T11:28:27.917 INFO:teuthology.orchestra.run.smithi175.stdout:Kernel IP routing table 2024-04-20T11:28:27.917 INFO:teuthology.orchestra.run.smithi175.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-20T11:28:27.917 INFO:teuthology.orchestra.run.smithi175.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-20T11:28:27.917 INFO:teuthology.orchestra.run.smithi175.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-20T11:28:27.917 INFO:teuthology.orchestra.run.smithi175.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-20T11:28:27.918 DEBUG:teuthology.orchestra.run.smithi175:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T11:28:27.918 DEBUG:teuthology.orchestra.run.smithi175:> set -e 2024-04-20T11:28:27.919 DEBUG:teuthology.orchestra.run.smithi175:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-20T11:28:27.919 DEBUG:teuthology.orchestra.run.smithi175:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-20T11:28:27.919 DEBUG:teuthology.orchestra.run.smithi175:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-20T11:28:27.919 DEBUG:teuthology.orchestra.run.smithi175:> ') 2024-04-20T11:28:28.026 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:28:28 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T11:28:28.126 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:28:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T11:28:28.136 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:28:28.137 DEBUG:teuthology.orchestra.run.smithi175:> ip netns list 2024-04-20T11:28:28.186 INFO:teuthology.orchestra.run.smithi175.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-20T11:28:28.186 INFO:teuthology.orchestra.run.smithi175.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-20T11:28:28.187 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-20T11:28:28.188 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:28:28.188 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip link set brx.0 up 2024-04-20T11:28:28.270 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-04-20T11:28:28.270 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T11:28:28.270 DEBUG:teuthology.orchestra.run.smithi175:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-20T11:28:28.321 INFO:teuthology.orchestra.run.smithi175.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-20T11:28:28.322 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T11:28:28.322 DEBUG:teuthology.orchestra.run.smithi175:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-20T11:28:28.372 DEBUG:teuthology.orchestra.run.smithi175:> sudo modprobe fuse 2024-04-20T11:28:28.455 DEBUG:teuthology.orchestra.run.smithi175:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T11:28:28.508 INFO:teuthology.orchestra.run.smithi175.stdout:/proc 2024-04-20T11:28:28.509 INFO:teuthology.orchestra.run.smithi175.stdout:/sys 2024-04-20T11:28:28.509 INFO:teuthology.orchestra.run.smithi175.stdout:/dev 2024-04-20T11:28:28.509 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/security 2024-04-20T11:28:28.509 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/shm 2024-04-20T11:28:28.509 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/pts 2024-04-20T11:28:28.509 INFO:teuthology.orchestra.run.smithi175.stdout:/run 2024-04-20T11:28:28.509 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/cgroup 2024-04-20T11:28:28.509 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/pstore 2024-04-20T11:28:28.509 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/bpf 2024-04-20T11:28:28.509 INFO:teuthology.orchestra.run.smithi175.stdout:/ 2024-04-20T11:28:28.509 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/selinux 2024-04-20T11:28:28.509 INFO:teuthology.orchestra.run.smithi175.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T11:28:28.509 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/mqueue 2024-04-20T11:28:28.509 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/debug 2024-04-20T11:28:28.510 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/hugepages 2024-04-20T11:28:28.510 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/tracing 2024-04-20T11:28:28.510 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/fuse/connections 2024-04-20T11:28:28.510 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/config 2024-04-20T11:28:28.510 INFO:teuthology.orchestra.run.smithi175.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T11:28:28.510 INFO:teuthology.orchestra.run.smithi175.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T11:28:28.510 INFO:teuthology.orchestra.run.smithi175.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T11:28:28.510 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T11:28:28.510 INFO:teuthology.orchestra.run.smithi175.stdout:/run/user/1000 2024-04-20T11:28:28.510 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph 2024-04-20T11:28:28.510 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-4 2024-04-20T11:28:28.510 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-5 2024-04-20T11:28:28.511 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-6 2024-04-20T11:28:28.511 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-7 2024-04-20T11:28:28.511 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns 2024-04-20T11:28:28.511 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-20T11:28:28.511 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-20T11:28:28.511 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-20T11:28:28.511 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-20T11:28:28.512 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:28:28.512 DEBUG:teuthology.orchestra.run.smithi175:> ls /sys/fs/fuse/connections 2024-04-20T11:28:28.561 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-20T11:28:28.561 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-04-20T11:28:28.565 DEBUG:teuthology.orchestra.run.smithi175:> sudo modprobe fuse 2024-04-20T11:28:28.630 DEBUG:teuthology.orchestra.run.smithi175:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T11:28:28.669 INFO:teuthology.orchestra.run.smithi175.stdout:/proc 2024-04-20T11:28:28.669 INFO:teuthology.orchestra.run.smithi175.stdout:/sys 2024-04-20T11:28:28.669 INFO:teuthology.orchestra.run.smithi175.stdout:/dev 2024-04-20T11:28:28.669 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/security 2024-04-20T11:28:28.670 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/shm 2024-04-20T11:28:28.670 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/pts 2024-04-20T11:28:28.670 INFO:teuthology.orchestra.run.smithi175.stdout:/run 2024-04-20T11:28:28.670 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/cgroup 2024-04-20T11:28:28.670 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/pstore 2024-04-20T11:28:28.670 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/bpf 2024-04-20T11:28:28.670 INFO:teuthology.orchestra.run.smithi175.stdout:/ 2024-04-20T11:28:28.670 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/selinux 2024-04-20T11:28:28.670 INFO:teuthology.orchestra.run.smithi175.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T11:28:28.670 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/mqueue 2024-04-20T11:28:28.670 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/debug 2024-04-20T11:28:28.670 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/hugepages 2024-04-20T11:28:28.670 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/tracing 2024-04-20T11:28:28.670 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/fuse/connections 2024-04-20T11:28:28.671 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/config 2024-04-20T11:28:28.671 INFO:teuthology.orchestra.run.smithi175.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T11:28:28.671 INFO:teuthology.orchestra.run.smithi175.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T11:28:28.671 INFO:teuthology.orchestra.run.smithi175.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T11:28:28.671 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T11:28:28.671 INFO:teuthology.orchestra.run.smithi175.stdout:/run/user/1000 2024-04-20T11:28:28.671 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph 2024-04-20T11:28:28.671 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-4 2024-04-20T11:28:28.671 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-5 2024-04-20T11:28:28.671 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-6 2024-04-20T11:28:28.671 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-7 2024-04-20T11:28:28.671 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns 2024-04-20T11:28:28.671 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-20T11:28:28.671 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-20T11:28:28.672 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-20T11:28:28.672 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-20T11:28:28.672 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:28:28.673 DEBUG:teuthology.orchestra.run.smithi175:> ls /sys/fs/fuse/connections 2024-04-20T11:28:28.725 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi175.stderr:2024-04-20T11:28:28.726+0000 7f9dee23e480 -1 init, newargv = 0x559cb18bf960 newargc=15 2024-04-20T11:28:28.726 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi175.stderr:ceph-fuse[29798]: starting ceph client 2024-04-20T11:28:28.736 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi175.stderr:ceph-fuse[29798]: starting fuse 2024-04-20T11:28:28.755 INFO:teuthology.orchestra.run.smithi175.stdout:42 2024-04-20T11:28:28.755 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-04-20T11:28:28.756 DEBUG:teuthology.orchestra.run.smithi175:> sudo stdin-killer -- python3 -c ' 2024-04-20T11:28:28.756 DEBUG:teuthology.orchestra.run.smithi175:> import glob 2024-04-20T11:28:28.756 DEBUG:teuthology.orchestra.run.smithi175:> import re 2024-04-20T11:28:28.756 DEBUG:teuthology.orchestra.run.smithi175:> import os 2024-04-20T11:28:28.756 DEBUG:teuthology.orchestra.run.smithi175:> import subprocess 2024-04-20T11:28:28.756 DEBUG:teuthology.orchestra.run.smithi175:> 2024-04-20T11:28:28.756 DEBUG:teuthology.orchestra.run.smithi175:> def _find_admin_socket(client_name): 2024-04-20T11:28:28.756 DEBUG:teuthology.orchestra.run.smithi175:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-04-20T11:28:28.756 DEBUG:teuthology.orchestra.run.smithi175:> files = glob.glob(asok_path) 2024-04-20T11:28:28.756 DEBUG:teuthology.orchestra.run.smithi175:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-04-20T11:28:28.756 DEBUG:teuthology.orchestra.run.smithi175:> 2024-04-20T11:28:28.756 DEBUG:teuthology.orchestra.run.smithi175:> # Given a non-glob path, it better be there 2024-04-20T11:28:28.756 DEBUG:teuthology.orchestra.run.smithi175:> if "*" not in asok_path: 2024-04-20T11:28:28.756 DEBUG:teuthology.orchestra.run.smithi175:> assert(len(files) == 1) 2024-04-20T11:28:28.757 DEBUG:teuthology.orchestra.run.smithi175:> return files[0] 2024-04-20T11:28:28.757 DEBUG:teuthology.orchestra.run.smithi175:> 2024-04-20T11:28:28.757 DEBUG:teuthology.orchestra.run.smithi175:> for f in files: 2024-04-20T11:28:28.757 DEBUG:teuthology.orchestra.run.smithi175:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-20T11:28:28.757 DEBUG:teuthology.orchestra.run.smithi175:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-20T11:28:28.757 DEBUG:teuthology.orchestra.run.smithi175:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-20T11:28:28.757 DEBUG:teuthology.orchestra.run.smithi175:> contents = proc_f.read() 2024-04-20T11:28:28.757 DEBUG:teuthology.orchestra.run.smithi175:> if mountpoint in contents: 2024-04-20T11:28:28.757 DEBUG:teuthology.orchestra.run.smithi175:> return f 2024-04-20T11:28:28.757 DEBUG:teuthology.orchestra.run.smithi175:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-20T11:28:28.757 DEBUG:teuthology.orchestra.run.smithi175:> 2024-04-20T11:28:28.757 DEBUG:teuthology.orchestra.run.smithi175:> print(_find_admin_socket("client.1")) 2024-04-20T11:28:28.757 DEBUG:teuthology.orchestra.run.smithi175:> ' 2024-04-20T11:28:28.943 INFO:teuthology.orchestra.run.smithi175.stdout:/var/run/ceph/ceph-client.1.29798.asok 2024-04-20T11:28:28.947 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:28:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T11:28:28.960 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.29798.asok 2024-04-20T11:28:28.960 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:28:28.960 DEBUG:teuthology.orchestra.run.smithi175:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.29798.asok status 2024-04-20T11:28:29.187 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-04-20T11:28:29.187 INFO:teuthology.orchestra.run.smithi175.stdout: "metadata": { 2024-04-20T11:28:29.187 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_sha1": "a8c84f5997950fd7d008204839253291d52e6eda", 2024-04-20T11:28:29.187 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", 2024-04-20T11:28:29.187 INFO:teuthology.orchestra.run.smithi175.stdout: "entity_id": "1", 2024-04-20T11:28:29.187 INFO:teuthology.orchestra.run.smithi175.stdout: "hostname": "smithi175", 2024-04-20T11:28:29.187 INFO:teuthology.orchestra.run.smithi175.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-04-20T11:28:29.187 INFO:teuthology.orchestra.run.smithi175.stdout: "pid": "29798", 2024-04-20T11:28:29.188 INFO:teuthology.orchestra.run.smithi175.stdout: "root": "/" 2024-04-20T11:28:29.188 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-20T11:28:29.188 INFO:teuthology.orchestra.run.smithi175.stdout: "dentry_count": 0, 2024-04-20T11:28:29.188 INFO:teuthology.orchestra.run.smithi175.stdout: "dentry_pinned_count": 0, 2024-04-20T11:28:29.188 INFO:teuthology.orchestra.run.smithi175.stdout: "id": 4912, 2024-04-20T11:28:29.188 INFO:teuthology.orchestra.run.smithi175.stdout: "inst": { 2024-04-20T11:28:29.188 INFO:teuthology.orchestra.run.smithi175.stdout: "name": { 2024-04-20T11:28:29.188 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "client", 2024-04-20T11:28:29.188 INFO:teuthology.orchestra.run.smithi175.stdout: "num": 4912 2024-04-20T11:28:29.188 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-20T11:28:29.188 INFO:teuthology.orchestra.run.smithi175.stdout: "addr": { 2024-04-20T11:28:29.188 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "v1", 2024-04-20T11:28:29.188 INFO:teuthology.orchestra.run.smithi175.stdout: "addr": "192.168.0.1:0", 2024-04-20T11:28:29.188 INFO:teuthology.orchestra.run.smithi175.stdout: "nonce": 3088465569 2024-04-20T11:28:29.189 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-04-20T11:28:29.189 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-20T11:28:29.189 INFO:teuthology.orchestra.run.smithi175.stdout: "addr": { 2024-04-20T11:28:29.189 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "v1", 2024-04-20T11:28:29.189 INFO:teuthology.orchestra.run.smithi175.stdout: "addr": "192.168.0.1:0", 2024-04-20T11:28:29.189 INFO:teuthology.orchestra.run.smithi175.stdout: "nonce": 3088465569 2024-04-20T11:28:29.189 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-20T11:28:29.189 INFO:teuthology.orchestra.run.smithi175.stdout: "inst_str": "client.4912 192.168.0.1:0/3088465569", 2024-04-20T11:28:29.189 INFO:teuthology.orchestra.run.smithi175.stdout: "addr_str": "192.168.0.1:0/3088465569", 2024-04-20T11:28:29.189 INFO:teuthology.orchestra.run.smithi175.stdout: "inode_count": 1, 2024-04-20T11:28:29.189 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_epoch": 15, 2024-04-20T11:28:29.189 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_epoch": 34, 2024-04-20T11:28:29.189 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_epoch_barrier": 0, 2024-04-20T11:28:29.190 INFO:teuthology.orchestra.run.smithi175.stdout: "blocklisted": false, 2024-04-20T11:28:29.190 INFO:teuthology.orchestra.run.smithi175.stdout: "fs_name": "cephfs" 2024-04-20T11:28:29.190 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-04-20T11:28:29.201 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:28:29.202 DEBUG:teuthology.orchestra.run.smithi175:> stat --file-system '--printf=%T 2024-04-20T11:28:29.202 DEBUG:teuthology.orchestra.run.smithi175:> ' -- /home/ubuntu/cephtest/mnt.1 2024-04-20T11:28:29.252 INFO:teuthology.orchestra.run.smithi175.stdout:fuseblk 2024-04-20T11:28:29.252 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-04-20T11:28:29.253 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:28:29.253 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-20T11:28:29.335 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-04-20T11:28:29.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:29.551+0000 7f7ccf200640 1 Processor -- start 2024-04-20T11:28:29.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:29.552+0000 7f7ccf200640 1 -- start start 2024-04-20T11:28:29.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:29.553+0000 7f7ccf200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7cd00b4ac0 0x7f7cd00b3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:29.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:29.553+0000 7f7ccf200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f7cd00d05c0 0x7f7cd00d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:29.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:29.553+0000 7f7ccf200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f7cd015cae0 0x7f7cd015cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:29.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:29.553+0000 7f7ccf200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f7cd00d0f60 con 0x7f7cd015cae0 2024-04-20T11:28:29.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:29.553+0000 7f7ccf200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f7cd00b3740 con 0x7f7cd00d05c0 2024-04-20T11:28:29.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:29.553+0000 7f7ccf200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7cd015da60 con 0x7f7cd00b4ac0 2024-04-20T11:28:29.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:29.554+0000 7f7cce800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f7cd015cae0 0x7f7cd015cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:29.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:29.554+0000 7f7cce800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f7cd015cae0 0x7f7cd015cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49120/0 (socket says 172.21.15.152:49120) 2024-04-20T11:28:29.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:29.554+0000 7f7cce800640 1 -- 172.21.15.152:0/1742642101 learned_addr learned my addr 172.21.15.152:0/1742642101 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:29.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:29.554+0000 7f7cc7e00640 1 --2- 172.21.15.152:0/1742642101 >> v2:172.21.15.175:3300/0 conn(0x7f7cd00d05c0 0x7f7cd00d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:29.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:29.554+0000 7f7cce800640 1 -- 172.21.15.152:0/1742642101 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7cd00b4ac0 msgr2=0x7f7cd00b3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:29.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:29.554+0000 7f7ccde00640 1 --2- 172.21.15.152:0/1742642101 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7cd00b4ac0 0x7f7cd00b3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:29.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:29.554+0000 7f7cce800640 1 --2- 172.21.15.152:0/1742642101 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7cd00b4ac0 0x7f7cd00b3bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:29.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:29.554+0000 7f7cce800640 1 -- 172.21.15.152:0/1742642101 >> v2:172.21.15.175:3300/0 conn(0x7f7cd00d05c0 msgr2=0x7f7cd00d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:29.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:29.554+0000 7f7cce800640 1 --2- 172.21.15.152:0/1742642101 >> v2:172.21.15.175:3300/0 conn(0x7f7cd00d05c0 0x7f7cd00d0a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:29.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:29.554+0000 7f7cce800640 1 -- 172.21.15.152:0/1742642101 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cd015d4b0 con 0x7f7cd015cae0 2024-04-20T11:28:29.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:29.555+0000 7f7cce800640 1 --2- 172.21.15.152:0/1742642101 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7cd015cae0 0x7f7cd015cf70 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f7cd006c940 tx=0x7f7cc802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ba4642ddcc93fc6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:29.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:29.556+0000 7f7ccd400640 1 -- 172.21.15.152:0/1742642101 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cc802ca70 con 0x7f7cd015cae0 2024-04-20T11:28:29.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:29.556+0000 7f7ccd400640 1 -- 172.21.15.152:0/1742642101 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cc802cc10 con 0x7f7cd015cae0 2024-04-20T11:28:29.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:29.556+0000 7f7ccd400640 1 -- 172.21.15.152:0/1742642101 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cc80343f0 con 0x7f7cd015cae0 2024-04-20T11:28:29.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:29.556+0000 7f7ccf200640 1 -- 172.21.15.152:0/1742642101 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7cd015cae0 msgr2=0x7f7cd015cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:29.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:29.556+0000 7f7ccf200640 1 --2- 172.21.15.152:0/1742642101 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7cd015cae0 0x7f7cd015cf70 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f7cd006c940 tx=0x7f7cc802bed0 comp rx=0 tx=0).stop 2024-04-20T11:28:29.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:29.556+0000 7f7ccf200640 1 -- 172.21.15.152:0/1742642101 shutdown_connections 2024-04-20T11:28:29.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:29.556+0000 7f7ccf200640 1 --2- 172.21.15.152:0/1742642101 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7cd015cae0 0x7f7cd015cf70 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:29.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:29.556+0000 7f7ccf200640 1 --2- 172.21.15.152:0/1742642101 >> v2:172.21.15.175:3300/0 conn(0x7f7cd00d05c0 0x7f7cd00d0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:29.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:29.556+0000 7f7ccf200640 1 --2- 172.21.15.152:0/1742642101 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7cd00b4ac0 0x7f7cd00b3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:29.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:29.556+0000 7f7ccf200640 1 -- 172.21.15.152:0/1742642101 >> 172.21.15.152:0/1742642101 conn(0x7f7cd006bda0 msgr2=0x7f7cd006c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:29.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:29.556+0000 7f7ccf200640 1 -- 172.21.15.152:0/1742642101 shutdown_connections 2024-04-20T11:28:29.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:29.557+0000 7f7ccf200640 1 -- 172.21.15.152:0/1742642101 wait complete. 2024-04-20T11:28:29.558 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:29.558+0000 7f7ccf200640 1 Processor -- start 2024-04-20T11:28:29.558 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:29.558+0000 7f7ccf200640 1 -- start start 2024-04-20T11:28:29.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:29.559+0000 7f7ccf200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7cd00b4ac0 0x7f7cd0099860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:29.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:29.559+0000 7f7ccf200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7cd00d05c0 0x7f7cd0150ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:29.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:29.559+0000 7f7ccf200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7cd015cae0 0x7f7cd00ccdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:29.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:29.559+0000 7f7ccf200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f7cd00b33d0 con 0x7f7cd00b4ac0 2024-04-20T11:28:29.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:29.559+0000 7f7ccf200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7cd00b30f0 con 0x7f7cd00d05c0 2024-04-20T11:28:29.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:29.559+0000 7f7ccf200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7cd00b4ea0 con 0x7f7cd015cae0 2024-04-20T11:28:29.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:29.560+0000 7f7ccde00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7cd00b4ac0 0x7f7cd0099860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:29.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:29.560+0000 7f7ccde00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7cd00b4ac0 0x7f7cd0099860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49126/0 (socket says 172.21.15.152:49126) 2024-04-20T11:28:29.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:29.560+0000 7f7ccde00640 1 -- 172.21.15.152:0/3619635217 learned_addr learned my addr 172.21.15.152:0/3619635217 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:29.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:29.560+0000 7f7cc7e00640 1 --2- 172.21.15.152:0/3619635217 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7cd00d05c0 0x7f7cd0150ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:29.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:29.560+0000 7f7ccde00640 1 -- 172.21.15.152:0/3619635217 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7cd015cae0 msgr2=0x7f7cd00ccdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:29.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:29.560+0000 7f7ccde00640 1 --2- 172.21.15.152:0/3619635217 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7cd015cae0 0x7f7cd00ccdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:29.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:29.560+0000 7f7ccde00640 1 -- 172.21.15.152:0/3619635217 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7cd00d05c0 msgr2=0x7f7cd0150ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:29.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:29.560+0000 7f7ccde00640 1 --2- 172.21.15.152:0/3619635217 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7cd00d05c0 0x7f7cd0150ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:29.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:29.560+0000 7f7ccde00640 1 -- 172.21.15.152:0/3619635217 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cd01f7d80 con 0x7f7cd00b4ac0 2024-04-20T11:28:29.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:29.560+0000 7f7ccde00640 1 --2- 172.21.15.152:0/3619635217 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7cd00b4ac0 0x7f7cd0099860 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f7cc000c7f0 tx=0x7f7cc000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:29.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:29.561+0000 7f7cc6000640 1 -- 172.21.15.152:0/3619635217 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cc0015070 con 0x7f7cd00b4ac0 2024-04-20T11:28:29.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:29.561+0000 7f7cc6000640 1 -- 172.21.15.152:0/3619635217 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cc0004220 con 0x7f7cd00b4ac0 2024-04-20T11:28:29.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:29.561+0000 7f7cc6000640 1 -- 172.21.15.152:0/3619635217 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cc0007620 con 0x7f7cd00b4ac0 2024-04-20T11:28:29.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:29.561+0000 7f7ccf200640 1 -- 172.21.15.152:0/3619635217 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cd01f8060 con 0x7f7cd00b4ac0 2024-04-20T11:28:29.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:29.561+0000 7f7ccf200640 1 -- 172.21.15.152:0/3619635217 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cd01f8620 con 0x7f7cd00b4ac0 2024-04-20T11:28:29.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:29.562+0000 7f7ccf200640 1 -- 172.21.15.152:0/3619635217 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cd00689f0 con 0x7f7cd00b4ac0 2024-04-20T11:28:29.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:29.563+0000 7f7cc6000640 1 -- 172.21.15.152:0/3619635217 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7cc00079d0 con 0x7f7cd00b4ac0 2024-04-20T11:28:29.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:29.564+0000 7f7cc6000640 1 --2- 172.21.15.152:0/3619635217 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f7cac06c8f0 0x7f7cac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:29.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:29.564+0000 7f7cc6000640 1 -- 172.21.15.152:0/3619635217 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f7cc008bc70 con 0x7f7cd00b4ac0 2024-04-20T11:28:29.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:29.564+0000 7f7cc7e00640 1 --2- 172.21.15.152:0/3619635217 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f7cac06c8f0 0x7f7cac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:29.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:29.565+0000 7f7cc7e00640 1 --2- 172.21.15.152:0/3619635217 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f7cac06c8f0 0x7f7cac06edb0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f7cb8006fd0 tx=0x7f7cb8008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:29.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:29.569+0000 7f7cc6000640 1 -- 172.21.15.152:0/3619635217 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f7cc005a000 con 0x7f7cd00b4ac0 2024-04-20T11:28:29.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:29.806+0000 7f7ccf200640 1 -- 172.21.15.152:0/3619635217 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f7cd00cea40 con 0x7f7cd00b4ac0 2024-04-20T11:28:30.558 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:30.559+0000 7f7cc6000640 1 -- 172.21.15.152:0/3619635217 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v16) ==== 136+0+0 (secure 0 0 0) 0x7f7cc0059900 con 0x7f7cd00b4ac0 2024-04-20T11:28:30.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:30.562+0000 7f7ccf200640 1 -- 172.21.15.152:0/3619635217 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f7cac06c8f0 msgr2=0x7f7cac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:30.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:30.562+0000 7f7ccf200640 1 --2- 172.21.15.152:0/3619635217 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f7cac06c8f0 0x7f7cac06edb0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f7cb8006fd0 tx=0x7f7cb8008040 comp rx=0 tx=0).stop 2024-04-20T11:28:30.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:30.562+0000 7f7ccf200640 1 -- 172.21.15.152:0/3619635217 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7cd00b4ac0 msgr2=0x7f7cd0099860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:30.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:30.562+0000 7f7ccf200640 1 --2- 172.21.15.152:0/3619635217 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7cd00b4ac0 0x7f7cd0099860 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f7cc000c7f0 tx=0x7f7cc000ccb0 comp rx=0 tx=0).stop 2024-04-20T11:28:30.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:30.562+0000 7f7ccf200640 1 -- 172.21.15.152:0/3619635217 shutdown_connections 2024-04-20T11:28:30.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:30.562+0000 7f7ccf200640 1 --2- 172.21.15.152:0/3619635217 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f7cac06c8f0 0x7f7cac06edb0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:30.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:30.562+0000 7f7ccf200640 1 --2- 172.21.15.152:0/3619635217 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7cd015cae0 0x7f7cd00ccdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:30.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:30.562+0000 7f7ccf200640 1 --2- 172.21.15.152:0/3619635217 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7cd00d05c0 0x7f7cd0150ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:30.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:30.562+0000 7f7ccf200640 1 --2- 172.21.15.152:0/3619635217 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7cd00b4ac0 0x7f7cd0099860 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:30.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:30.562+0000 7f7ccf200640 1 -- 172.21.15.152:0/3619635217 >> 172.21.15.152:0/3619635217 conn(0x7f7cd006bda0 msgr2=0x7f7cd015de00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:30.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:30.562+0000 7f7ccf200640 1 -- 172.21.15.152:0/3619635217 shutdown_connections 2024-04-20T11:28:30.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:30.563+0000 7f7ccf200640 1 -- 172.21.15.152:0/3619635217 wait complete. 2024-04-20T11:28:30.584 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-04-20T11:28:30.778 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:30.779+0000 7f1ebde00640 1 Processor -- start 2024-04-20T11:28:30.779 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:30.779+0000 7f1ebde00640 1 -- start start 2024-04-20T11:28:30.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:30.781+0000 7f1ebde00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f1eb815b330 0x7f1eb81588f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:30.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:30.781+0000 7f1ebde00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f1eb80b3d60 0x7f1eb80d03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:30.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:30.781+0000 7f1ebde00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1eb815b7c0 con 0x7f1eb8072750 2024-04-20T11:28:30.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:30.781+0000 7f1ebde00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f1eb815baa0 con 0x7f1eb80b3d60 2024-04-20T11:28:30.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:30.781+0000 7f1ebde00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f1eb815bd80 con 0x7f1eb815b330 2024-04-20T11:28:30.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:30.781+0000 7f1eb6000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f1eb80b3d60 0x7f1eb80d03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:30.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:30.781+0000 7f1eb7400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f1eb8072750 0x7f1eb80b4460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:40268/0 (socket says 172.21.15.152:40268) 2024-04-20T11:28:30.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:30.781+0000 7f1eb7400640 1 -- 172.21.15.152:0/3834015509 learned_addr learned my addr 172.21.15.152:0/3834015509 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:28:30.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:30.781+0000 7f1eb6a00640 1 --2- 172.21.15.152:0/3834015509 >> v2:172.21.15.175:3300/0 conn(0x7f1eb815b330 0x7f1eb81588f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:30.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:30.781+0000 7f1eb6000640 1 -- 172.21.15.152:0/3834015509 >> v2:172.21.15.175:3300/0 conn(0x7f1eb815b330 msgr2=0x7f1eb81588f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:30.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:30.781+0000 7f1eb6000640 1 --2- 172.21.15.152:0/3834015509 >> v2:172.21.15.175:3300/0 conn(0x7f1eb815b330 0x7f1eb81588f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:30.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:30.781+0000 7f1eb6000640 1 -- 172.21.15.152:0/3834015509 >> v1:172.21.15.152:6789/0 conn(0x7f1eb8072750 legacy=0x7f1eb80b4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:30.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:30.782+0000 7f1eb6000640 1 -- 172.21.15.152:0/3834015509 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1eb815c060 con 0x7f1eb80b3d60 2024-04-20T11:28:30.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:30.781+0000 7f1eb5600640 1 -- 172.21.15.152:0/3834015509 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2491485256 0 0) 0x7f1eb815b7c0 con 0x7f1eb8072750 2024-04-20T11:28:30.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:30.783+0000 7f1eb6000640 1 --2- 172.21.15.152:0/3834015509 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1eb80b3d60 0x7f1eb80d03c0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f1eb806c7f0 tx=0x7f1ea402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77af29353aba0cfa server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:30.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:30.783+0000 7f1eb5600640 1 -- 172.21.15.152:0/3834015509 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea4035020 con 0x7f1eb80b3d60 2024-04-20T11:28:30.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:30.783+0000 7f1eb5600640 1 -- 172.21.15.152:0/3834015509 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ea4031830 con 0x7f1eb80b3d60 2024-04-20T11:28:30.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:30.783+0000 7f1eb5600640 1 -- 172.21.15.152:0/3834015509 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea4031c00 con 0x7f1eb80b3d60 2024-04-20T11:28:30.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:30.784+0000 7f1ebde00640 1 -- 172.21.15.152:0/3834015509 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1eb80b3d60 msgr2=0x7f1eb80d03c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:30.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:30.784+0000 7f1ebde00640 1 --2- 172.21.15.152:0/3834015509 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1eb80b3d60 0x7f1eb80d03c0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f1eb806c7f0 tx=0x7f1ea402b8d0 comp rx=0 tx=0).stop 2024-04-20T11:28:30.784 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:30.784+0000 7f1ebde00640 1 -- 172.21.15.152:0/3834015509 shutdown_connections 2024-04-20T11:28:30.784 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:30.784+0000 7f1ebde00640 1 --2- 172.21.15.152:0/3834015509 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1eb80b3d60 0x7f1eb80d03c0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:30.784 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:30.784+0000 7f1ebde00640 1 --2- 172.21.15.152:0/3834015509 >> v2:172.21.15.175:3300/0 conn(0x7f1eb815b330 0x7f1eb81588f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:30.784 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:30.784+0000 7f1ebde00640 1 -- 172.21.15.152:0/3834015509 >> 172.21.15.152:0/3834015509 conn(0x7f1eb8067cf0 msgr2=0x7f1eb806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:30.784 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:30.784+0000 7f1ebde00640 1 -- 172.21.15.152:0/3834015509 shutdown_connections 2024-04-20T11:28:30.784 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:30.784+0000 7f1ebde00640 1 -- 172.21.15.152:0/3834015509 wait complete. 2024-04-20T11:28:30.785 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:30.786+0000 7f1ebde00640 1 Processor -- start 2024-04-20T11:28:30.785 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:30.786+0000 7f1ebde00640 1 -- start start 2024-04-20T11:28:30.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:30.787+0000 7f1ebde00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1eb8072750 0x7f1eb81684d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:30.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:30.787+0000 7f1ebde00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1eb80b3d60 0x7f1eb80aed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:30.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:30.787+0000 7f1ebde00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1eb80cebc0 0x7f1eb80ad8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:30.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:30.787+0000 7f1ebde00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb80b4cf0 con 0x7f1eb80cebc0 2024-04-20T11:28:30.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:30.787+0000 7f1ebde00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb80b4b70 con 0x7f1eb8072750 2024-04-20T11:28:30.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:30.787+0000 7f1ebde00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1eb8165a30 con 0x7f1eb80b3d60 2024-04-20T11:28:30.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:30.787+0000 7f1eb7400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1eb80cebc0 0x7f1eb80ad8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:30.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:30.787+0000 7f1eb7400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1eb80cebc0 0x7f1eb80ad8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49136/0 (socket says 172.21.15.152:49136) 2024-04-20T11:28:30.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:30.787+0000 7f1eb7400640 1 -- 172.21.15.152:0/4152925736 learned_addr learned my addr 172.21.15.152:0/4152925736 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:30.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:30.787+0000 7f1eb7400640 1 -- 172.21.15.152:0/4152925736 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1eb80b3d60 msgr2=0x7f1eb80aed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:30.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:30.787+0000 7f1eb7400640 1 --2- 172.21.15.152:0/4152925736 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1eb80b3d60 0x7f1eb80aed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:30.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:30.787+0000 7f1eb6a00640 1 --2- 172.21.15.152:0/4152925736 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1eb8072750 0x7f1eb81684d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:30.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:30.787+0000 7f1eb7400640 1 -- 172.21.15.152:0/4152925736 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1eb8072750 msgr2=0x7f1eb81684d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:30.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:30.787+0000 7f1eb7400640 1 --2- 172.21.15.152:0/4152925736 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1eb8072750 0x7f1eb81684d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:30.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:30.787+0000 7f1eb7400640 1 -- 172.21.15.152:0/4152925736 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1eb80af240 con 0x7f1eb80cebc0 2024-04-20T11:28:30.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:30.788+0000 7f1eb7400640 1 --2- 172.21.15.152:0/4152925736 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1eb80cebc0 0x7f1eb80ad8b0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f1eb000c7a0 tx=0x7f1eb000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:30.789 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:30.788+0000 7f1eaf400640 1 -- 172.21.15.152:0/4152925736 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eb0018070 con 0x7f1eb80cebc0 2024-04-20T11:28:30.789 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:30.788+0000 7f1eaf400640 1 -- 172.21.15.152:0/4152925736 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1eb000a640 con 0x7f1eb80cebc0 2024-04-20T11:28:30.789 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:30.788+0000 7f1eaf400640 1 -- 172.21.15.152:0/4152925736 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eb000aab0 con 0x7f1eb80cebc0 2024-04-20T11:28:30.789 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:30.788+0000 7f1ebde00640 1 -- 172.21.15.152:0/4152925736 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1eb8167710 con 0x7f1eb80cebc0 2024-04-20T11:28:30.789 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:30.788+0000 7f1ebde00640 1 -- 172.21.15.152:0/4152925736 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1eb8167fb0 con 0x7f1eb80cebc0 2024-04-20T11:28:30.789 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:30.789+0000 7f1ebde00640 1 -- 172.21.15.152:0/4152925736 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1eb81678e0 con 0x7f1eb80cebc0 2024-04-20T11:28:30.789 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:30.790+0000 7f1eaf400640 1 -- 172.21.15.152:0/4152925736 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1eb000ac50 con 0x7f1eb80cebc0 2024-04-20T11:28:30.790 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:30.791+0000 7f1eaf400640 1 --2- 172.21.15.152:0/4152925736 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f1e9806c8f0 0x7f1e9806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:30.790 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:30.791+0000 7f1eaf400640 1 -- 172.21.15.152:0/4152925736 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f1eb008b260 con 0x7f1eb80cebc0 2024-04-20T11:28:30.791 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:30.791+0000 7f1eb6a00640 1 --2- 172.21.15.152:0/4152925736 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f1e9806c8f0 0x7f1e9806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:30.791 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:30.792+0000 7f1eb6a00640 1 --2- 172.21.15.152:0/4152925736 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f1e9806c8f0 0x7f1e9806edb0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1ea800b7c0 tx=0x7f1ea8008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:30.795 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:30.795+0000 7f1eaf400640 1 -- 172.21.15.152:0/4152925736 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f1eb00595f0 con 0x7f1eb80cebc0 2024-04-20T11:28:31.017 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:31.017+0000 7f1ebde00640 1 -- 172.21.15.152:0/4152925736 --> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f1eb8167b10 con 0x7f1e9806c8f0 2024-04-20T11:28:31.019 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:31.019+0000 7f1eaf400640 1 -- 172.21.15.152:0/4152925736 <== mgr.4113 v2:172.21.15.152:6832/2285110240 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f1ea800bda0 con 0x7f1e9806c8f0 2024-04-20T11:28:31.019 INFO:teuthology.orchestra.run.smithi152.stdout:[ 2024-04-20T11:28:31.019 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:28:31.019 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "cephfs" 2024-04-20T11:28:31.019 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:28:31.019 INFO:teuthology.orchestra.run.smithi152.stdout:] 2024-04-20T11:28:31.021 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:31.021+0000 7f1ebde00640 1 -- 172.21.15.152:0/4152925736 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f1e9806c8f0 msgr2=0x7f1e9806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:31.021 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:31.021+0000 7f1ebde00640 1 --2- 172.21.15.152:0/4152925736 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f1e9806c8f0 0x7f1e9806edb0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1ea800b7c0 tx=0x7f1ea8008040 comp rx=0 tx=0).stop 2024-04-20T11:28:31.021 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:31.021+0000 7f1ebde00640 1 -- 172.21.15.152:0/4152925736 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1eb80cebc0 msgr2=0x7f1eb80ad8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:31.022 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:31.021+0000 7f1ebde00640 1 --2- 172.21.15.152:0/4152925736 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1eb80cebc0 0x7f1eb80ad8b0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f1eb000c7a0 tx=0x7f1eb000cc60 comp rx=0 tx=0).stop 2024-04-20T11:28:31.022 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:31.022+0000 7f1ebde00640 1 -- 172.21.15.152:0/4152925736 shutdown_connections 2024-04-20T11:28:31.022 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:31.022+0000 7f1ebde00640 1 --2- 172.21.15.152:0/4152925736 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f1e9806c8f0 0x7f1e9806edb0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:31.022 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:31.022+0000 7f1ebde00640 1 --2- 172.21.15.152:0/4152925736 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1eb80cebc0 0x7f1eb80ad8b0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:31.022 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:31.022+0000 7f1ebde00640 1 --2- 172.21.15.152:0/4152925736 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1eb80b3d60 0x7f1eb80aed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:31.022 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:31.022+0000 7f1ebde00640 1 --2- 172.21.15.152:0/4152925736 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1eb8072750 0x7f1eb81684d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:31.022 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:31.022+0000 7f1ebde00640 1 -- 172.21.15.152:0/4152925736 >> 172.21.15.152:0/4152925736 conn(0x7f1eb8067cf0 msgr2=0x7f1eb8157f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:31.022 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:31.022+0000 7f1ebde00640 1 -- 172.21.15.152:0/4152925736 shutdown_connections 2024-04-20T11:28:31.022 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:31.022+0000 7f1ebde00640 1 -- 172.21.15.152:0/4152925736 wait complete. 2024-04-20T11:28:31.038 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-20T11:28:31.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:31.235+0000 7fb6e2400640 1 Processor -- start 2024-04-20T11:28:31.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:31.235+0000 7fb6e2400640 1 -- start start 2024-04-20T11:28:31.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:31.236+0000 7fb6e2400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fb6dc0b3fa0 0x7fb6dc162440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:31.237 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:31.236+0000 7fb6e2400640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6dc0adc30 con 0x7fb6dc0afba0 2024-04-20T11:28:31.237 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:31.236+0000 7fb6e2400640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6dc162a70 con 0x7fb6dc15e5d0 2024-04-20T11:28:31.237 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:31.236+0000 7fb6e2400640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fb6dc162d50 con 0x7fb6dc0b3fa0 2024-04-20T11:28:31.237 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:31.237+0000 7fb6dbe00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fb6dc0b3fa0 0x7fb6dc162440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:31.237 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:31.237+0000 7fb6d3e00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fb6dc0afba0 0x7fb6dc0ae110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:40278/0 (socket says 172.21.15.152:40278) 2024-04-20T11:28:31.237 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:31.237+0000 7fb6dbe00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fb6dc0b3fa0 0x7fb6dc162440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49150/0 (socket says 172.21.15.152:49150) 2024-04-20T11:28:31.237 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:31.237+0000 7fb6d3e00640 1 -- 172.21.15.152:0/724061029 learned_addr learned my addr 172.21.15.152:0/724061029 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:28:31.237 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:31.237+0000 7fb6dbe00640 1 -- 172.21.15.152:0/724061029 >> v1:172.21.15.175:6789/0 conn(0x7fb6dc15e5d0 legacy=0x7fb6dc0af490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:31.237 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:31.237+0000 7fb6dbe00640 1 -- 172.21.15.152:0/724061029 >> v1:172.21.15.152:6789/0 conn(0x7fb6dc0afba0 legacy=0x7fb6dc0ae110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:31.237 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:31.237+0000 7fb6dbe00640 1 -- 172.21.15.152:0/724061029 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6dc163030 con 0x7fb6dc0b3fa0 2024-04-20T11:28:31.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:31.238+0000 7fb6dbe00640 1 --2- 172.21.15.152:0/724061029 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb6dc0b3fa0 0x7fb6dc162440 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fb6d4009d90 tx=0x7fb6d402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84808b66147fd0f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:31.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:31.238+0000 7fb6daa00640 1 -- 172.21.15.152:0/724061029 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d4039070 con 0x7fb6dc0b3fa0 2024-04-20T11:28:31.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:31.238+0000 7fb6daa00640 1 -- 172.21.15.152:0/724061029 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6d402cc60 con 0x7fb6dc0b3fa0 2024-04-20T11:28:31.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:31.238+0000 7fb6daa00640 1 -- 172.21.15.152:0/724061029 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d4034660 con 0x7fb6dc0b3fa0 2024-04-20T11:28:31.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:31.239+0000 7fb6e2400640 1 -- 172.21.15.152:0/724061029 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb6dc0b3fa0 msgr2=0x7fb6dc162440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:31.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:31.239+0000 7fb6e2400640 1 --2- 172.21.15.152:0/724061029 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb6dc0b3fa0 0x7fb6dc162440 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fb6d4009d90 tx=0x7fb6d402beb0 comp rx=0 tx=0).stop 2024-04-20T11:28:31.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:31.239+0000 7fb6e2400640 1 -- 172.21.15.152:0/724061029 shutdown_connections 2024-04-20T11:28:31.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:31.239+0000 7fb6e2400640 1 --2- 172.21.15.152:0/724061029 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb6dc0b3fa0 0x7fb6dc162440 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:31.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:31.239+0000 7fb6e2400640 1 -- 172.21.15.152:0/724061029 >> 172.21.15.152:0/724061029 conn(0x7fb6dc06b980 msgr2=0x7fb6dc06bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:31.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:31.239+0000 7fb6e2400640 1 -- 172.21.15.152:0/724061029 shutdown_connections 2024-04-20T11:28:31.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:31.239+0000 7fb6e2400640 1 -- 172.21.15.152:0/724061029 wait complete. 2024-04-20T11:28:31.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:31.241+0000 7fb6e2400640 1 Processor -- start 2024-04-20T11:28:31.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:31.241+0000 7fb6e2400640 1 -- start start 2024-04-20T11:28:31.243 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:31.242+0000 7fb6e2400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb6dc0afba0 0x7fb6dc14eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:31.243 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:31.242+0000 7fb6e2400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb6dc0b3fa0 0x7fb6dc1513c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:31.243 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:31.242+0000 7fb6e2400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb6dc15e5d0 0x7fb6dc151900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:31.243 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:31.242+0000 7fb6e2400640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fb6dc06e0a0 con 0x7fb6dc15e5d0 2024-04-20T11:28:31.243 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:31.242+0000 7fb6e2400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb6dc163310 con 0x7fb6dc0b3fa0 2024-04-20T11:28:31.243 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:31.242+0000 7fb6e2400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb6dc06e220 con 0x7fb6dc0afba0 2024-04-20T11:28:31.243 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:31.242+0000 7fb6dbe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb6dc15e5d0 0x7fb6dc151900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:31.243 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:31.242+0000 7fb6dbe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb6dc15e5d0 0x7fb6dc151900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49152/0 (socket says 172.21.15.152:49152) 2024-04-20T11:28:31.243 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:31.242+0000 7fb6dbe00640 1 -- 172.21.15.152:0/657053541 learned_addr learned my addr 172.21.15.152:0/657053541 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:31.243 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:31.243+0000 7fb6d3e00640 1 --2- 172.21.15.152:0/657053541 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb6dc0b3fa0 0x7fb6dc1513c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:31.243 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:31.243+0000 7fb6dbe00640 1 -- 172.21.15.152:0/657053541 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb6dc0afba0 msgr2=0x7fb6dc14eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:31.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:31.243+0000 7fb6dbe00640 1 --2- 172.21.15.152:0/657053541 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb6dc0afba0 0x7fb6dc14eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:31.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:31.243+0000 7fb6dbe00640 1 -- 172.21.15.152:0/657053541 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb6dc0b3fa0 msgr2=0x7fb6dc1513c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:31.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:31.243+0000 7fb6dbe00640 1 --2- 172.21.15.152:0/657053541 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb6dc0b3fa0 0x7fb6dc1513c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:31.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:31.243+0000 7fb6dbe00640 1 -- 172.21.15.152:0/657053541 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6dc0b1890 con 0x7fb6dc15e5d0 2024-04-20T11:28:31.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:31.243+0000 7fb6dbe00640 1 --2- 172.21.15.152:0/657053541 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb6dc15e5d0 0x7fb6dc151900 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fb6d4009d90 tx=0x7fb6d4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:31.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:31.243+0000 7fb6d3400640 1 -- 172.21.15.152:0/657053541 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d4044020 con 0x7fb6dc15e5d0 2024-04-20T11:28:31.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:31.243+0000 7fb6d3400640 1 -- 172.21.15.152:0/657053541 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6d403f470 con 0x7fb6dc15e5d0 2024-04-20T11:28:31.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:31.243+0000 7fb6d3400640 1 -- 172.21.15.152:0/657053541 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d4039040 con 0x7fb6dc15e5d0 2024-04-20T11:28:31.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:31.243+0000 7fb6e2400640 1 -- 172.21.15.152:0/657053541 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6dc0cd230 con 0x7fb6dc15e5d0 2024-04-20T11:28:31.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:31.243+0000 7fb6e2400640 1 -- 172.21.15.152:0/657053541 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6dc151e40 con 0x7fb6dc15e5d0 2024-04-20T11:28:31.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:31.244+0000 7fb6e2400640 1 -- 172.21.15.152:0/657053541 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6dc0689f0 con 0x7fb6dc15e5d0 2024-04-20T11:28:31.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:31.245+0000 7fb6d3400640 1 -- 172.21.15.152:0/657053541 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb6d4047020 con 0x7fb6dc15e5d0 2024-04-20T11:28:31.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:31.246+0000 7fb6d3400640 1 --2- 172.21.15.152:0/657053541 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fb6b806c8f0 0x7fb6b806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:31.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:31.246+0000 7fb6d3400640 1 -- 172.21.15.152:0/657053541 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fb6d40ae790 con 0x7fb6dc15e5d0 2024-04-20T11:28:31.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:31.247+0000 7fb6db400640 1 --2- 172.21.15.152:0/657053541 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fb6b806c8f0 0x7fb6b806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:31.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:31.247+0000 7fb6db400640 1 --2- 172.21.15.152:0/657053541 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fb6b806c8f0 0x7fb6b806edb0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb6cc006fd0 tx=0x7fb6cc006bc0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:31.250 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:31.251+0000 7fb6d3400640 1 -- 172.21.15.152:0/657053541 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fb6d404e030 con 0x7fb6dc15e5d0 2024-04-20T11:28:31.450 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:31.451+0000 7fb6e2400640 1 -- 172.21.15.152:0/657053541 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fb6dc0b3d30 con 0x7fb6dc15e5d0 2024-04-20T11:28:31.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:31.452+0000 7fb6d3400640 1 -- 172.21.15.152:0/657053541 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7fb6d407cbb0 con 0x7fb6dc15e5d0 2024-04-20T11:28:31.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:31.454+0000 7fb6e2400640 1 -- 172.21.15.152:0/657053541 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fb6b806c8f0 msgr2=0x7fb6b806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:31.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:31.454+0000 7fb6e2400640 1 --2- 172.21.15.152:0/657053541 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fb6b806c8f0 0x7fb6b806edb0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb6cc006fd0 tx=0x7fb6cc006bc0 comp rx=0 tx=0).stop 2024-04-20T11:28:31.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:31.454+0000 7fb6e2400640 1 -- 172.21.15.152:0/657053541 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb6dc15e5d0 msgr2=0x7fb6dc151900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:31.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:31.454+0000 7fb6e2400640 1 --2- 172.21.15.152:0/657053541 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb6dc15e5d0 0x7fb6dc151900 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fb6d4009d90 tx=0x7fb6d4004a40 comp rx=0 tx=0).stop 2024-04-20T11:28:31.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:31.455+0000 7fb6e2400640 1 -- 172.21.15.152:0/657053541 shutdown_connections 2024-04-20T11:28:31.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:31.455+0000 7fb6e2400640 1 --2- 172.21.15.152:0/657053541 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fb6b806c8f0 0x7fb6b806edb0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:31.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:31.455+0000 7fb6e2400640 1 --2- 172.21.15.152:0/657053541 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb6dc15e5d0 0x7fb6dc151900 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:31.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:31.455+0000 7fb6e2400640 1 --2- 172.21.15.152:0/657053541 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb6dc0b3fa0 0x7fb6dc1513c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:31.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:31.455+0000 7fb6e2400640 1 --2- 172.21.15.152:0/657053541 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb6dc0afba0 0x7fb6dc14eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:31.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:31.455+0000 7fb6e2400640 1 -- 172.21.15.152:0/657053541 >> 172.21.15.152:0/657053541 conn(0x7fb6dc06b980 msgr2=0x7fb6dc0ae9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:31.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:31.455+0000 7fb6e2400640 1 -- 172.21.15.152:0/657053541 shutdown_connections 2024-04-20T11:28:31.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:31.455+0000 7fb6e2400640 1 -- 172.21.15.152:0/657053541 wait complete. 2024-04-20T11:28:31.472 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:28:31.472 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p ./volumes/subvol_grp73750/subvol85970) 2024-04-20T11:28:31.579 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:28:31.579 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2024-04-20T11:28:31.631 INFO:teuthology.orchestra.run.smithi152.stdout:cephfs_data 2024-04-20T11:28:31.632 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:28:31.632 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data ./volumes/subvol_grp73750/subvol85970) 2024-04-20T11:28:31.746 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-20T11:28:31.746 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/home/ubuntu/cephtest/mnt.0/./volumes/subvol_grp73750/subvol85970/.meta 2024-04-20T11:28:31.841 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol85970 subvol_grp73750 2024-04-20T11:28:32.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.093+0000 7fe808200640 1 Processor -- start 2024-04-20T11:28:32.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.093+0000 7fe808200640 1 -- start start 2024-04-20T11:28:32.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.094+0000 7fe808200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe80015e910 0x7fe8000d3870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:32.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.095+0000 7fe808200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fe8000cfbe0 0x7fe8000b78c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:32.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.095+0000 7fe808200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8000d40e0 con 0x7fe8000d22d0 2024-04-20T11:28:32.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.095+0000 7fe808200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fe8000d43c0 con 0x7fe8000cfbe0 2024-04-20T11:28:32.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.095+0000 7fe808200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe8000d46a0 con 0x7fe80015e910 2024-04-20T11:28:32.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.095+0000 7fe806200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fe8000cfbe0 0x7fe8000b78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:32.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.095+0000 7fe805800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe80015e910 0x7fe8000d3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:32.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.096+0000 7fe805800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe80015e910 0x7fe8000d3870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.152:56368/0 (socket says 172.21.15.152:56368) 2024-04-20T11:28:32.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.096+0000 7fe805800640 1 -- 172.21.15.152:0/3164230677 learned_addr learned my addr 172.21.15.152:0/3164230677 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:32.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.096+0000 7fe806200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fe8000cfbe0 0x7fe8000b78c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.152:47028/0 (socket says 172.21.15.152:47028) 2024-04-20T11:28:32.096 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.096+0000 7fe806200640 1 -- 172.21.15.152:0/3164230677 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe80015e910 msgr2=0x7fe8000d3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:32.096 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.096+0000 7fe806200640 1 --2- 172.21.15.152:0/3164230677 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe80015e910 0x7fe8000d3870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:32.096 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.096+0000 7fe806200640 1 -- 172.21.15.152:0/3164230677 >> v1:172.21.15.175:6789/0 conn(0x7fe8000d22d0 legacy=0x7fe8000cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:32.096 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.097+0000 7fe806200640 1 -- 172.21.15.152:0/3164230677 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8000d4980 con 0x7fe8000cfbe0 2024-04-20T11:28:32.097 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.097+0000 7fe7ffe00640 1 -- 172.21.15.152:0/3164230677 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1707112580 0 0) 0x7fe8000d40e0 con 0x7fe8000d22d0 2024-04-20T11:28:32.097 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.097+0000 7fe805800640 1 --2- 172.21.15.152:0/3164230677 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe80015e910 0x7fe8000d3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:28:32.097 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.098+0000 7fe806200640 1 --2- 172.21.15.152:0/3164230677 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8000cfbe0 0x7fe8000b78c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe7f8009d90 tx=0x7fe7f802be20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0eda3243f2f98ba server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:32.098 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.098+0000 7fe7ffe00640 1 -- 172.21.15.152:0/3164230677 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f8039070 con 0x7fe8000cfbe0 2024-04-20T11:28:32.098 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.099+0000 7fe7ffe00640 1 -- 172.21.15.152:0/3164230677 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7f802cbd0 con 0x7fe8000cfbe0 2024-04-20T11:28:32.098 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.099+0000 7fe7ffe00640 1 -- 172.21.15.152:0/3164230677 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f8034440 con 0x7fe8000cfbe0 2024-04-20T11:28:32.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.099+0000 7fe808200640 1 -- 172.21.15.152:0/3164230677 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8000cfbe0 msgr2=0x7fe8000b78c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:32.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.099+0000 7fe808200640 1 --2- 172.21.15.152:0/3164230677 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8000cfbe0 0x7fe8000b78c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe7f8009d90 tx=0x7fe7f802be20 comp rx=0 tx=0).stop 2024-04-20T11:28:32.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.099+0000 7fe808200640 1 -- 172.21.15.152:0/3164230677 shutdown_connections 2024-04-20T11:28:32.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.099+0000 7fe808200640 1 --2- 172.21.15.152:0/3164230677 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8000cfbe0 0x7fe8000b78c0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:32.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.099+0000 7fe808200640 1 --2- 172.21.15.152:0/3164230677 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe80015e910 0x7fe8000d3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:32.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.099+0000 7fe808200640 1 -- 172.21.15.152:0/3164230677 >> 172.21.15.152:0/3164230677 conn(0x7fe800067ae0 msgr2=0x7fe800067ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:32.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.100+0000 7fe808200640 1 -- 172.21.15.152:0/3164230677 shutdown_connections 2024-04-20T11:28:32.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.100+0000 7fe808200640 1 -- 172.21.15.152:0/3164230677 wait complete. 2024-04-20T11:28:32.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.101+0000 7fe808200640 1 Processor -- start 2024-04-20T11:28:32.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.101+0000 7fe808200640 1 -- start start 2024-04-20T11:28:32.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.102+0000 7fe808200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe8000cfbe0 0x7fe800160880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:32.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.102+0000 7fe808200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8000d22d0 0x7fe800150150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:32.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.102+0000 7fe808200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe80015e910 0x7fe8000b9b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:32.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.102+0000 7fe808200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fe80015ecf0 con 0x7fe8000cfbe0 2024-04-20T11:28:32.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.102+0000 7fe808200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe8000d3db0 con 0x7fe8000d22d0 2024-04-20T11:28:32.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.102+0000 7fe808200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe8000b7e00 con 0x7fe80015e910 2024-04-20T11:28:32.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.103+0000 7fe805800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe8000cfbe0 0x7fe800160880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:32.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.103+0000 7fe805800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe8000cfbe0 0x7fe800160880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49166/0 (socket says 172.21.15.152:49166) 2024-04-20T11:28:32.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.103+0000 7fe805800640 1 -- 172.21.15.152:0/272148994 learned_addr learned my addr 172.21.15.152:0/272148994 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:32.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.103+0000 7fe806200640 1 --2- 172.21.15.152:0/272148994 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe80015e910 0x7fe8000b9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:32.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.103+0000 7fe804e00640 1 --2- 172.21.15.152:0/272148994 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8000d22d0 0x7fe800150150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:32.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.103+0000 7fe805800640 1 -- 172.21.15.152:0/272148994 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe80015e910 msgr2=0x7fe8000b9b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:32.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.103+0000 7fe805800640 1 --2- 172.21.15.152:0/272148994 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe80015e910 0x7fe8000b9b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:32.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.103+0000 7fe805800640 1 -- 172.21.15.152:0/272148994 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8000d22d0 msgr2=0x7fe800150150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:32.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.103+0000 7fe805800640 1 --2- 172.21.15.152:0/272148994 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8000d22d0 0x7fe800150150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:32.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.103+0000 7fe805800640 1 -- 172.21.15.152:0/272148994 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe80014cc60 con 0x7fe8000cfbe0 2024-04-20T11:28:32.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.103+0000 7fe804e00640 1 --2- 172.21.15.152:0/272148994 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8000d22d0 0x7fe800150150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:28:32.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.104+0000 7fe805800640 1 --2- 172.21.15.152:0/272148994 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe8000cfbe0 0x7fe800160880 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fe7f400e9b0 tx=0x7fe7f400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:32.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.104+0000 7fe7fd600640 1 -- 172.21.15.152:0/272148994 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f4017070 con 0x7fe8000cfbe0 2024-04-20T11:28:32.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.104+0000 7fe7fd600640 1 -- 172.21.15.152:0/272148994 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7f400c9a0 con 0x7fe8000cfbe0 2024-04-20T11:28:32.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.104+0000 7fe808200640 1 -- 172.21.15.152:0/272148994 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe80014cf40 con 0x7fe8000cfbe0 2024-04-20T11:28:32.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.104+0000 7fe7fd600640 1 -- 172.21.15.152:0/272148994 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f400cd50 con 0x7fe8000cfbe0 2024-04-20T11:28:32.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.104+0000 7fe808200640 1 -- 172.21.15.152:0/272148994 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe80014d500 con 0x7fe8000cfbe0 2024-04-20T11:28:32.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.105+0000 7fe808200640 1 -- 172.21.15.152:0/272148994 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe800068a60 con 0x7fe8000cfbe0 2024-04-20T11:28:32.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.107+0000 7fe7fd600640 1 -- 172.21.15.152:0/272148994 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe7f401a070 con 0x7fe8000cfbe0 2024-04-20T11:28:32.107 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.108+0000 7fe7fd600640 1 --2- 172.21.15.152:0/272148994 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe7e406c8f0 0x7fe7e406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:32.107 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.108+0000 7fe7fd600640 1 -- 172.21.15.152:0/272148994 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fe7f408be50 con 0x7fe8000cfbe0 2024-04-20T11:28:32.107 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.108+0000 7fe804e00640 1 --2- 172.21.15.152:0/272148994 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe7e406c8f0 0x7fe7e406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:32.108 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.109+0000 7fe804e00640 1 --2- 172.21.15.152:0/272148994 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe7e406c8f0 0x7fe7e406edb0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe80014fdb0 tx=0x7fe7f0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:32.111 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.112+0000 7fe7fd600640 1 -- 172.21.15.152:0/272148994 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe7f405a1e0 con 0x7fe8000cfbe0 2024-04-20T11:28:32.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.343+0000 7fe808200640 1 -- 172.21.15.152:0/272148994 --> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol85970", "group_name": "subvol_grp73750", "target": ["mon-mgr", ""]}) -- 0x7fe800161cb0 con 0x7fe7e406c8f0 2024-04-20T11:28:32.352 INFO:tasks.ceph.mgr.y.smithi152.stderr:2024-04-20T11:28:32.352+0000 7fd165e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:28:32.352 INFO:tasks.ceph.mgr.y.smithi152.stderr:2024-04-20T11:28:32.352+0000 7fd165e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:28:32.352 INFO:tasks.ceph.mgr.y.smithi152.stderr:2024-04-20T11:28:32.352+0000 7fd165e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:28:32.352 INFO:tasks.ceph.mgr.y.smithi152.stderr:2024-04-20T11:28:32.352+0000 7fd165e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:28:32.352 INFO:tasks.ceph.mgr.y.smithi152.stderr:2024-04-20T11:28:32.352+0000 7fd165e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:28:32.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.431+0000 7fe7fd600640 1 -- 172.21.15.152:0/272148994 <== mgr.4113 v2:172.21.15.152:6832/2285110240 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe800161cb0 con 0x7fe7e406c8f0 2024-04-20T11:28:32.431 INFO:teuthology.orchestra.run.smithi152.stdout:/volumes/subvol_grp73750/subvol85970 2024-04-20T11:28:32.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.434+0000 7fe808200640 1 -- 172.21.15.152:0/272148994 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe7e406c8f0 msgr2=0x7fe7e406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:32.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.434+0000 7fe808200640 1 --2- 172.21.15.152:0/272148994 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe7e406c8f0 0x7fe7e406edb0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe80014fdb0 tx=0x7fe7f0009040 comp rx=0 tx=0).stop 2024-04-20T11:28:32.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.434+0000 7fe808200640 1 -- 172.21.15.152:0/272148994 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe8000cfbe0 msgr2=0x7fe800160880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:32.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.434+0000 7fe808200640 1 --2- 172.21.15.152:0/272148994 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe8000cfbe0 0x7fe800160880 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fe7f400e9b0 tx=0x7fe7f400ee70 comp rx=0 tx=0).stop 2024-04-20T11:28:32.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.435+0000 7fe808200640 1 -- 172.21.15.152:0/272148994 shutdown_connections 2024-04-20T11:28:32.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.435+0000 7fe808200640 1 --2- 172.21.15.152:0/272148994 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe7e406c8f0 0x7fe7e406edb0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:32.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.435+0000 7fe808200640 1 --2- 172.21.15.152:0/272148994 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe80015e910 0x7fe8000b9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:32.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.435+0000 7fe808200640 1 --2- 172.21.15.152:0/272148994 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8000d22d0 0x7fe800150150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:32.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.435+0000 7fe808200640 1 --2- 172.21.15.152:0/272148994 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe8000cfbe0 0x7fe800160880 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:32.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.435+0000 7fe808200640 1 -- 172.21.15.152:0/272148994 >> 172.21.15.152:0/272148994 conn(0x7fe800067ae0 msgr2=0x7fe8000cb7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:32.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.435+0000 7fe808200640 1 -- 172.21.15.152:0/272148994 shutdown_connections 2024-04-20T11:28:32.436 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.435+0000 7fe808200640 1 -- 172.21.15.152:0/272148994 wait complete. 2024-04-20T11:28:32.457 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol85970 subvol_grp73750 2024-04-20T11:28:32.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.658+0000 7f2f0b200640 1 Processor -- start 2024-04-20T11:28:32.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.658+0000 7f2f0b200640 1 -- start start 2024-04-20T11:28:32.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.660+0000 7f2f0b200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f2f0c15e870 0x7f2f0c0d3370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:32.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.660+0000 7f2f0b200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f2f0c0d06a0 0x7f2f0c0d1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:32.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.660+0000 7f2f0b200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f0c15ec50 con 0x7f2f0c0d0fa0 2024-04-20T11:28:32.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.660+0000 7f2f0b200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f2f0c0d2ee0 con 0x7f2f0c0d06a0 2024-04-20T11:28:32.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.660+0000 7f2f0b200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f2f0c0d38b0 con 0x7f2f0c15e870 2024-04-20T11:28:32.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.660+0000 7f2f0a800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f2f0c0d06a0 0x7f2f0c0d1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:32.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.660+0000 7f2f0a800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f2f0c0d06a0 0x7f2f0c0d1450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49184/0 (socket says 172.21.15.152:49184) 2024-04-20T11:28:32.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.660+0000 7f2f0a800640 1 -- 172.21.15.152:0/800381712 learned_addr learned my addr 172.21.15.152:0/800381712 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:32.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.660+0000 7f2f09e00640 1 --2- 172.21.15.152:0/800381712 >> v2:172.21.15.175:3300/0 conn(0x7f2f0c15e870 0x7f2f0c0d3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:32.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.661+0000 7f2f0a800640 1 -- 172.21.15.152:0/800381712 >> v2:172.21.15.175:3300/0 conn(0x7f2f0c15e870 msgr2=0x7f2f0c0d3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:32.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.661+0000 7f2f0a800640 1 --2- 172.21.15.152:0/800381712 >> v2:172.21.15.175:3300/0 conn(0x7f2f0c15e870 0x7f2f0c0d3370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:32.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.661+0000 7f2f0a800640 1 -- 172.21.15.152:0/800381712 >> v1:172.21.15.175:6789/0 conn(0x7f2f0c0d0fa0 legacy=0x7f2f0c0cff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:32.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.661+0000 7f2f0a800640 1 -- 172.21.15.152:0/800381712 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f0c0d22d0 con 0x7f2f0c0d06a0 2024-04-20T11:28:32.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.662+0000 7f2f0a800640 1 --2- 172.21.15.152:0/800381712 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2f0c0d06a0 0x7f2f0c0d1450 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c08e460 tx=0x7f2f0402be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=310cd2f03da6c0a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:32.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.662+0000 7f2f09400640 1 -- 172.21.15.152:0/800381712 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f0402c9f0 con 0x7f2f0c0d06a0 2024-04-20T11:28:32.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.662+0000 7f2f09400640 1 -- 172.21.15.152:0/800381712 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f0402cb90 con 0x7f2f0c0d06a0 2024-04-20T11:28:32.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.662+0000 7f2f09400640 1 -- 172.21.15.152:0/800381712 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f040343f0 con 0x7f2f0c0d06a0 2024-04-20T11:28:32.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.663+0000 7f2f0b200640 1 -- 172.21.15.152:0/800381712 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2f0c0d06a0 msgr2=0x7f2f0c0d1450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:32.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.663+0000 7f2f0b200640 1 --2- 172.21.15.152:0/800381712 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2f0c0d06a0 0x7f2f0c0d1450 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c08e460 tx=0x7f2f0402be50 comp rx=0 tx=0).stop 2024-04-20T11:28:32.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.663+0000 7f2f0b200640 1 -- 172.21.15.152:0/800381712 shutdown_connections 2024-04-20T11:28:32.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.663+0000 7f2f0b200640 1 --2- 172.21.15.152:0/800381712 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2f0c0d06a0 0x7f2f0c0d1450 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:32.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.663+0000 7f2f0b200640 1 --2- 172.21.15.152:0/800381712 >> v2:172.21.15.175:3300/0 conn(0x7f2f0c15e870 0x7f2f0c0d3370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:32.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.663+0000 7f2f0b200640 1 -- 172.21.15.152:0/800381712 >> 172.21.15.152:0/800381712 conn(0x7f2f0c08bfd0 msgr2=0x7f2f0c0d45d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:32.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.663+0000 7f2f0b200640 1 -- 172.21.15.152:0/800381712 shutdown_connections 2024-04-20T11:28:32.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.663+0000 7f2f0b200640 1 -- 172.21.15.152:0/800381712 wait complete. 2024-04-20T11:28:32.664 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.665+0000 7f2f0b200640 1 Processor -- start 2024-04-20T11:28:32.664 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.665+0000 7f2f0b200640 1 -- start start 2024-04-20T11:28:32.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.666+0000 7f2f0b200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2f0c0d06a0 0x7f2f0c155a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:32.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.666+0000 7f2f0b200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2f0c0d0fa0 0x7f2f0c0c8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:32.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.666+0000 7f2f0b200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2f0c15e870 0x7f2f0c0b22c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:32.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.666+0000 7f2f0b200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f2f0c0d2b70 con 0x7f2f0c0d0fa0 2024-04-20T11:28:32.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.666+0000 7f2f0b200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2f0c0d2890 con 0x7f2f0c15e870 2024-04-20T11:28:32.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.666+0000 7f2f0b200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2f0c06ca80 con 0x7f2f0c0d06a0 2024-04-20T11:28:32.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.666+0000 7f2f03e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2f0c0d0fa0 0x7f2f0c0c8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:32.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.666+0000 7f2f03e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2f0c0d0fa0 0x7f2f0c0c8ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49196/0 (socket says 172.21.15.152:49196) 2024-04-20T11:28:32.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.666+0000 7f2f03e00640 1 -- 172.21.15.152:0/1806335173 learned_addr learned my addr 172.21.15.152:0/1806335173 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:32.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.666+0000 7f2f0a800640 1 --2- 172.21.15.152:0/1806335173 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2f0c15e870 0x7f2f0c0b22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:32.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.666+0000 7f2f03e00640 1 -- 172.21.15.152:0/1806335173 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2f0c0d06a0 msgr2=0x7f2f0c155a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:32.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.666+0000 7f2f03e00640 1 --2- 172.21.15.152:0/1806335173 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2f0c0d06a0 0x7f2f0c155a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:32.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.666+0000 7f2f03e00640 1 -- 172.21.15.152:0/1806335173 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2f0c15e870 msgr2=0x7f2f0c0b22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:32.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.666+0000 7f2f03e00640 1 --2- 172.21.15.152:0/1806335173 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2f0c15e870 0x7f2f0c0b22c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:32.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.667+0000 7f2f03e00640 1 -- 172.21.15.152:0/1806335173 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f0c155fc0 con 0x7f2f0c0d0fa0 2024-04-20T11:28:32.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.667+0000 7f2f03e00640 1 --2- 172.21.15.152:0/1806335173 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2f0c0d0fa0 0x7f2f0c0c8ed0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f2ef4004640 tx=0x7f2ef400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:32.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.667+0000 7f2f02000640 1 -- 172.21.15.152:0/1806335173 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef4011790 con 0x7f2f0c0d0fa0 2024-04-20T11:28:32.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.667+0000 7f2f02000640 1 -- 172.21.15.152:0/1806335173 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ef4017070 con 0x7f2f0c0d0fa0 2024-04-20T11:28:32.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.667+0000 7f2f02000640 1 -- 172.21.15.152:0/1806335173 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef4011b70 con 0x7f2f0c0d0fa0 2024-04-20T11:28:32.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.667+0000 7f2f0b200640 1 -- 172.21.15.152:0/1806335173 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f0c15f340 con 0x7f2f0c0d0fa0 2024-04-20T11:28:32.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.667+0000 7f2f0b200640 1 -- 172.21.15.152:0/1806335173 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f0c15f900 con 0x7f2f0c0d0fa0 2024-04-20T11:28:32.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.669+0000 7f2f0b200640 1 -- 172.21.15.152:0/1806335173 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f0c075e60 con 0x7f2f0c0d0fa0 2024-04-20T11:28:32.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.670+0000 7f2f02000640 1 -- 172.21.15.152:0/1806335173 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2ef4011d10 con 0x7f2f0c0d0fa0 2024-04-20T11:28:32.670 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.670+0000 7f2f02000640 1 --2- 172.21.15.152:0/1806335173 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f2ee806c8f0 0x7f2ee806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:32.670 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.671+0000 7f2f02000640 1 -- 172.21.15.152:0/1806335173 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f2ef408b6c0 con 0x7f2f0c0d0fa0 2024-04-20T11:28:32.670 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.671+0000 7f2f09e00640 1 --2- 172.21.15.152:0/1806335173 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f2ee806c8f0 0x7f2ee806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:32.671 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.671+0000 7f2f09e00640 1 --2- 172.21.15.152:0/1806335173 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f2ee806c8f0 0x7f2ee806edb0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2efc006fd0 tx=0x7f2efc008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:32.674 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.675+0000 7f2f02000640 1 -- 172.21.15.152:0/1806335173 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f2ef4059af0 con 0x7f2f0c0d0fa0 2024-04-20T11:28:32.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.906+0000 7f2f0b200640 1 -- 172.21.15.152:0/1806335173 --> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol85970", "group_name": "subvol_grp73750", "target": ["mon-mgr", ""]}) -- 0x7f2f0c0cb4f0 con 0x7f2ee806c8f0 2024-04-20T11:28:32.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.916+0000 7f2f02000640 1 -- 172.21.15.152:0/1806335173 <== mgr.4113 v2:172.21.15.152:6832/2285110240 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f2f0c0cb4f0 con 0x7f2ee806c8f0 2024-04-20T11:28:32.916 INFO:teuthology.orchestra.run.smithi152.stdout:/volumes/subvol_grp73750/subvol85970 2024-04-20T11:28:32.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.918+0000 7f2f0b200640 1 -- 172.21.15.152:0/1806335173 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f2ee806c8f0 msgr2=0x7f2ee806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:32.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.918+0000 7f2f0b200640 1 --2- 172.21.15.152:0/1806335173 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f2ee806c8f0 0x7f2ee806edb0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2efc006fd0 tx=0x7f2efc008040 comp rx=0 tx=0).stop 2024-04-20T11:28:32.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.918+0000 7f2f0b200640 1 -- 172.21.15.152:0/1806335173 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2f0c0d0fa0 msgr2=0x7f2f0c0c8ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:32.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.918+0000 7f2f0b200640 1 --2- 172.21.15.152:0/1806335173 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2f0c0d0fa0 0x7f2f0c0c8ed0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f2ef4004640 tx=0x7f2ef400c8e0 comp rx=0 tx=0).stop 2024-04-20T11:28:32.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.919+0000 7f2f0b200640 1 -- 172.21.15.152:0/1806335173 shutdown_connections 2024-04-20T11:28:32.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.919+0000 7f2f0b200640 1 --2- 172.21.15.152:0/1806335173 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f2ee806c8f0 0x7f2ee806edb0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:32.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.919+0000 7f2f0b200640 1 --2- 172.21.15.152:0/1806335173 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2f0c15e870 0x7f2f0c0b22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:32.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.919+0000 7f2f0b200640 1 --2- 172.21.15.152:0/1806335173 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2f0c0d0fa0 0x7f2f0c0c8ed0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:32.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.919+0000 7f2f0b200640 1 --2- 172.21.15.152:0/1806335173 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2f0c0d06a0 0x7f2f0c155a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:32.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.919+0000 7f2f0b200640 1 -- 172.21.15.152:0/1806335173 >> 172.21.15.152:0/1806335173 conn(0x7f2f0c08bfd0 msgr2=0x7f2f0c0b42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:32.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.919+0000 7f2f0b200640 1 -- 172.21.15.152:0/1806335173 shutdown_connections 2024-04-20T11:28:32.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:32.919+0000 7f2f0b200640 1 -- 172.21.15.152:0/1806335173 wait complete. 2024-04-20T11:28:32.938 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:28:32.938 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_legacy/8692080786e8858a4e47e57dd683096b.meta) 2024-04-20T11:28:34.633 INFO:teuthology.orchestra.run.smithi152.stdout:[GLOBAL] 2024-04-20T11:28:34.634 INFO:teuthology.orchestra.run.smithi152.stdout:version = 1 2024-04-20T11:28:34.634 INFO:teuthology.orchestra.run.smithi152.stdout:type = subvolume 2024-04-20T11:28:34.634 INFO:teuthology.orchestra.run.smithi152.stdout:path = /volumes/subvol_grp73750/subvol85970 2024-04-20T11:28:34.634 INFO:teuthology.orchestra.run.smithi152.stdout:state = complete 2024-04-20T11:28:34.634 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:28:34.640 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol85970 subvol_grp73750 2024-04-20T11:28:34.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:34.849+0000 7f0d8b200640 1 Processor -- start 2024-04-20T11:28:34.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:34.849+0000 7f0d8b200640 1 -- start start 2024-04-20T11:28:34.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:34.851+0000 7f0d8b200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f0d840b1a20 0x7f0d840b2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:34.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:34.851+0000 7f0d8b200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f0d8406b980 0x7f0d8406bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:34.852 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:34.851+0000 7f0d8b200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0d8415f7a0 0x7f0d8415fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:34.852 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:34.851+0000 7f0d8b200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f0d8406c320 con 0x7f0d840b1a20 2024-04-20T11:28:34.852 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:34.851+0000 7f0d8b200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f0d840b6510 con 0x7f0d8406b980 2024-04-20T11:28:34.852 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:34.851+0000 7f0d8b200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0d840b5f50 con 0x7f0d8415f7a0 2024-04-20T11:28:34.852 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:34.851+0000 7f0d89e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f0d840b1a20 0x7f0d840b2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:34.852 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:34.851+0000 7f0d89e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f0d840b1a20 0x7f0d840b2630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49210/0 (socket says 172.21.15.152:49210) 2024-04-20T11:28:34.852 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:34.851+0000 7f0d89e00640 1 -- 172.21.15.152:0/3286311938 learned_addr learned my addr 172.21.15.152:0/3286311938 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:34.852 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:34.852+0000 7f0d89400640 1 --2- 172.21.15.152:0/3286311938 >> v2:172.21.15.175:3300/0 conn(0x7f0d8406b980 0x7f0d8406bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:34.852 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:34.852+0000 7f0d8a800640 1 --2- 172.21.15.152:0/3286311938 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0d8415f7a0 0x7f0d8415fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:34.852 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:34.852+0000 7f0d89e00640 1 -- 172.21.15.152:0/3286311938 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0d8415f7a0 msgr2=0x7f0d8415fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:34.852 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:34.852+0000 7f0d89e00640 1 --2- 172.21.15.152:0/3286311938 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0d8415f7a0 0x7f0d8415fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:34.852 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:34.852+0000 7f0d89e00640 1 -- 172.21.15.152:0/3286311938 >> v2:172.21.15.175:3300/0 conn(0x7f0d8406b980 msgr2=0x7f0d8406bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:34.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:34.852+0000 7f0d89e00640 1 --2- 172.21.15.152:0/3286311938 >> v2:172.21.15.175:3300/0 conn(0x7f0d8406b980 0x7f0d8406bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:34.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:34.852+0000 7f0d89e00640 1 -- 172.21.15.152:0/3286311938 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d840b6230 con 0x7f0d840b1a20 2024-04-20T11:28:34.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:34.852+0000 7f0d89400640 1 --2- 172.21.15.152:0/3286311938 >> v2:172.21.15.175:3300/0 conn(0x7f0d8406b980 0x7f0d8406bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:28:34.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:34.853+0000 7f0d89e00640 1 --2- 172.21.15.152:0/3286311938 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0d840b1a20 0x7f0d840b2630 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f0d8406e730 tx=0x7f0d7802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cfd117ae11c302 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:34.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:34.853+0000 7f0d88a00640 1 -- 172.21.15.152:0/3286311938 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d7802e070 con 0x7f0d840b1a20 2024-04-20T11:28:34.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:34.853+0000 7f0d88a00640 1 -- 172.21.15.152:0/3286311938 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d78037070 con 0x7f0d840b1a20 2024-04-20T11:28:34.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:34.853+0000 7f0d88a00640 1 -- 172.21.15.152:0/3286311938 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d78031a00 con 0x7f0d840b1a20 2024-04-20T11:28:34.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:34.854+0000 7f0d8b200640 1 -- 172.21.15.152:0/3286311938 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0d840b1a20 msgr2=0x7f0d840b2630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:34.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:34.854+0000 7f0d8b200640 1 --2- 172.21.15.152:0/3286311938 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0d840b1a20 0x7f0d840b2630 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f0d8406e730 tx=0x7f0d7802b880 comp rx=0 tx=0).stop 2024-04-20T11:28:34.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:34.854+0000 7f0d8b200640 1 -- 172.21.15.152:0/3286311938 shutdown_connections 2024-04-20T11:28:34.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:34.854+0000 7f0d8b200640 1 --2- 172.21.15.152:0/3286311938 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0d8415f7a0 0x7f0d8415fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:34.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:34.854+0000 7f0d8b200640 1 --2- 172.21.15.152:0/3286311938 >> v2:172.21.15.175:3300/0 conn(0x7f0d8406b980 0x7f0d8406bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:34.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:34.854+0000 7f0d8b200640 1 --2- 172.21.15.152:0/3286311938 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0d840b1a20 0x7f0d840b2630 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:34.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:34.854+0000 7f0d8b200640 1 -- 172.21.15.152:0/3286311938 >> 172.21.15.152:0/3286311938 conn(0x7f0d840b6700 msgr2=0x7f0d840b6b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:34.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:34.854+0000 7f0d8b200640 1 -- 172.21.15.152:0/3286311938 shutdown_connections 2024-04-20T11:28:34.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:34.854+0000 7f0d8b200640 1 -- 172.21.15.152:0/3286311938 wait complete. 2024-04-20T11:28:34.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:34.856+0000 7f0d8b200640 1 Processor -- start 2024-04-20T11:28:34.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:34.856+0000 7f0d8b200640 1 -- start start 2024-04-20T11:28:34.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:34.857+0000 7f0d8b200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d8406b980 0x7f0d8415e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:34.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:34.857+0000 7f0d8b200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0d8415f7a0 0x7f0d84140d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:34.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:34.857+0000 7f0d8b200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0d84142860 0x7f0d84142d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:34.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:34.857+0000 7f0d8b200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f0d840b1e00 con 0x7f0d84142860 2024-04-20T11:28:34.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:34.857+0000 7f0d8b200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0d8415f280 con 0x7f0d8406b980 2024-04-20T11:28:34.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:34.857+0000 7f0d8b200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0d840b5c40 con 0x7f0d8415f7a0 2024-04-20T11:28:34.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:34.857+0000 7f0d8a800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0d84142860 0x7f0d84142d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:34.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:34.857+0000 7f0d8a800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0d84142860 0x7f0d84142d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49222/0 (socket says 172.21.15.152:49222) 2024-04-20T11:28:34.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:34.857+0000 7f0d8a800640 1 -- 172.21.15.152:0/3031805028 learned_addr learned my addr 172.21.15.152:0/3031805028 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:34.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:34.858+0000 7f0d89400640 1 --2- 172.21.15.152:0/3031805028 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0d8415f7a0 0x7f0d84140d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:34.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:34.858+0000 7f0d89e00640 1 --2- 172.21.15.152:0/3031805028 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d8406b980 0x7f0d8415e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:34.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:34.858+0000 7f0d8a800640 1 -- 172.21.15.152:0/3031805028 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0d8415f7a0 msgr2=0x7f0d84140d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:34.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:34.858+0000 7f0d8a800640 1 --2- 172.21.15.152:0/3031805028 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0d8415f7a0 0x7f0d84140d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:34.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:34.858+0000 7f0d8a800640 1 -- 172.21.15.152:0/3031805028 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d8406b980 msgr2=0x7f0d8415e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:34.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:34.858+0000 7f0d8a800640 1 --2- 172.21.15.152:0/3031805028 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d8406b980 0x7f0d8415e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:34.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:34.858+0000 7f0d8a800640 1 -- 172.21.15.152:0/3031805028 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d841e7450 con 0x7f0d84142860 2024-04-20T11:28:34.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:34.858+0000 7f0d89e00640 1 --2- 172.21.15.152:0/3031805028 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d8406b980 0x7f0d8415e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:28:34.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:34.858+0000 7f0d8a800640 1 --2- 172.21.15.152:0/3031805028 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0d84142860 0x7f0d84142d10 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f0d8000cda0 tx=0x7f0d8000a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:34.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:34.858+0000 7f0d7e000640 1 -- 172.21.15.152:0/3031805028 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d80003b90 con 0x7f0d84142860 2024-04-20T11:28:34.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:34.858+0000 7f0d7e000640 1 -- 172.21.15.152:0/3031805028 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d80003d30 con 0x7f0d84142860 2024-04-20T11:28:34.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:34.859+0000 7f0d7e000640 1 -- 172.21.15.152:0/3031805028 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d80004240 con 0x7f0d84142860 2024-04-20T11:28:34.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:34.859+0000 7f0d8b200640 1 -- 172.21.15.152:0/3031805028 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d841e6e90 con 0x7f0d84142860 2024-04-20T11:28:34.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:34.859+0000 7f0d8b200640 1 -- 172.21.15.152:0/3031805028 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d841e8180 con 0x7f0d84142860 2024-04-20T11:28:34.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:34.860+0000 7f0d8b200640 1 -- 172.21.15.152:0/3031805028 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d84075e60 con 0x7f0d84142860 2024-04-20T11:28:34.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:34.861+0000 7f0d7e000640 1 -- 172.21.15.152:0/3031805028 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f0d800043e0 con 0x7f0d84142860 2024-04-20T11:28:34.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:34.862+0000 7f0d7e000640 1 --2- 172.21.15.152:0/3031805028 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f0d6806c980 0x7f0d6806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:34.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:34.862+0000 7f0d7e000640 1 -- 172.21.15.152:0/3031805028 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f0d8008c7e0 con 0x7f0d84142860 2024-04-20T11:28:34.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:34.862+0000 7f0d89e00640 1 --2- 172.21.15.152:0/3031805028 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f0d6806c980 0x7f0d6806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:34.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:34.863+0000 7f0d89e00640 1 --2- 172.21.15.152:0/3031805028 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f0d6806c980 0x7f0d6806ee40 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f0d78036000 tx=0x7f0d78008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:34.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:34.866+0000 7f0d7e000640 1 -- 172.21.15.152:0/3031805028 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f0d8005aaa0 con 0x7f0d84142860 2024-04-20T11:28:35.098 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:35.099+0000 7f0d8b200640 1 -- 172.21.15.152:0/3031805028 --> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol85970", "group_name": "subvol_grp73750", "target": ["mon-mgr", ""]}) -- 0x7f0d840d19f0 con 0x7f0d6806c980 2024-04-20T11:28:35.138 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:35.138+0000 7f0d7e000640 1 -- 172.21.15.152:0/3031805028 <== mgr.4113 v2:172.21.15.152:6832/2285110240 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0d840d19f0 con 0x7f0d6806c980 2024-04-20T11:28:35.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:35.141+0000 7f0d8b200640 1 -- 172.21.15.152:0/3031805028 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f0d6806c980 msgr2=0x7f0d6806ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:35.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:35.141+0000 7f0d8b200640 1 --2- 172.21.15.152:0/3031805028 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f0d6806c980 0x7f0d6806ee40 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f0d78036000 tx=0x7f0d78008030 comp rx=0 tx=0).stop 2024-04-20T11:28:35.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:35.141+0000 7f0d8b200640 1 -- 172.21.15.152:0/3031805028 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0d84142860 msgr2=0x7f0d84142d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:35.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:35.141+0000 7f0d8b200640 1 --2- 172.21.15.152:0/3031805028 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0d84142860 0x7f0d84142d10 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f0d8000cda0 tx=0x7f0d8000a660 comp rx=0 tx=0).stop 2024-04-20T11:28:35.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:35.141+0000 7f0d8b200640 1 -- 172.21.15.152:0/3031805028 shutdown_connections 2024-04-20T11:28:35.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:35.141+0000 7f0d8b200640 1 --2- 172.21.15.152:0/3031805028 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f0d6806c980 0x7f0d6806ee40 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:35.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:35.141+0000 7f0d8b200640 1 --2- 172.21.15.152:0/3031805028 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0d84142860 0x7f0d84142d10 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:35.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:35.141+0000 7f0d8b200640 1 --2- 172.21.15.152:0/3031805028 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0d8415f7a0 0x7f0d84140d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:35.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:35.141+0000 7f0d8b200640 1 --2- 172.21.15.152:0/3031805028 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d8406b980 0x7f0d8415e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:35.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:35.141+0000 7f0d8b200640 1 -- 172.21.15.152:0/3031805028 >> 172.21.15.152:0/3031805028 conn(0x7f0d840b6700 msgr2=0x7f0d840b3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:35.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:35.141+0000 7f0d8b200640 1 -- 172.21.15.152:0/3031805028 shutdown_connections 2024-04-20T11:28:35.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:35.141+0000 7f0d8b200640 1 -- 172.21.15.152:0/3031805028 wait complete. 2024-04-20T11:28:35.144 INFO:tasks.ceph.mgr.y.smithi152.stderr:2024-04-20T11:28:35.144+0000 7fd169a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:28:35.144 INFO:tasks.ceph.mgr.y.smithi152.stderr:2024-04-20T11:28:35.144+0000 7fd169a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:28:35.144 INFO:tasks.ceph.mgr.y.smithi152.stderr:2024-04-20T11:28:35.144+0000 7fd169a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:28:35.144 INFO:tasks.ceph.mgr.y.smithi152.stderr:2024-04-20T11:28:35.144+0000 7fd169a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:28:35.144 INFO:tasks.ceph.mgr.y.smithi152.stderr:2024-04-20T11:28:35.144+0000 7fd169a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:28:35.162 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-04-20T11:28:35.170 INFO:tasks.ceph.mgr.y.smithi152.stderr:2024-04-20T11:28:35.171+0000 7fd167c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:28:35.170 INFO:tasks.ceph.mgr.y.smithi152.stderr:2024-04-20T11:28:35.171+0000 7fd167c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:28:35.170 INFO:tasks.ceph.mgr.y.smithi152.stderr:2024-04-20T11:28:35.171+0000 7fd167c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:28:35.170 INFO:tasks.ceph.mgr.y.smithi152.stderr:2024-04-20T11:28:35.171+0000 7fd167c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:28:35.170 INFO:tasks.ceph.mgr.y.smithi152.stderr:2024-04-20T11:28:35.171+0000 7fd167c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:28:35.253 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:35 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-20T11:28:35.262 INFO:teuthology.orchestra.run.smithi152.stdout:2 2024-04-20T11:28:35.263 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T11:28:35.272 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup rm cephfs subvol_grp73750 2024-04-20T11:28:35.512 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:35.512+0000 7f107ca00640 1 Processor -- start 2024-04-20T11:28:35.512 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:35.513+0000 7f107ca00640 1 -- start start 2024-04-20T11:28:35.514 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:35.514+0000 7f107ca00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f10780b2880 0x7f10780b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:35.514 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:35.514+0000 7f107ca00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f10780cd1b0 0x7f10780cd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:35.514 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:35.514+0000 7f107ca00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10781463e0 con 0x7f1078146000 2024-04-20T11:28:35.514 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:35.514+0000 7f107ca00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f10780b4750 con 0x7f10780b2880 2024-04-20T11:28:35.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:35.514+0000 7f107ca00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f10780b3e20 con 0x7f10780cd1b0 2024-04-20T11:28:35.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:35.514+0000 7f1075400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f10780b2880 0x7f10780b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:35.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:35.515+0000 7f1075400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f10780b2880 0x7f10780b4be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49248/0 (socket says 172.21.15.152:49248) 2024-04-20T11:28:35.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:35.515+0000 7f1075400640 1 -- 172.21.15.152:0/4222702587 learned_addr learned my addr 172.21.15.152:0/4222702587 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:35.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:35.515+0000 7f1076800640 1 --2- 172.21.15.152:0/4222702587 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f10780cd1b0 0x7f10780cd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:35.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:35.515+0000 7f1075400640 1 -- 172.21.15.152:0/4222702587 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f10780cd1b0 msgr2=0x7f10780cd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:35.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:35.515+0000 7f1075400640 1 --2- 172.21.15.152:0/4222702587 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f10780cd1b0 0x7f10780cd5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:35.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:35.515+0000 7f1075400640 1 -- 172.21.15.152:0/4222702587 >> v1:172.21.15.175:6789/0 conn(0x7f1078146000 legacy=0x7f10780b2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:35.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:35.515+0000 7f1075400640 1 -- 172.21.15.152:0/4222702587 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10780b4100 con 0x7f10780b2880 2024-04-20T11:28:35.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:35.516+0000 7f1075400640 1 --2- 172.21.15.152:0/4222702587 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f10780b2880 0x7f10780b4be0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f10781601b0 tx=0x7f106002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e52300db25b29ee server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:35.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:35.516+0000 7f1074a00640 1 -- 172.21.15.152:0/4222702587 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10600315a0 con 0x7f10780b2880 2024-04-20T11:28:35.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:35.516+0000 7f1074a00640 1 -- 172.21.15.152:0/4222702587 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1060037070 con 0x7f10780b2880 2024-04-20T11:28:35.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:35.516+0000 7f1074a00640 1 -- 172.21.15.152:0/4222702587 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1060031960 con 0x7f10780b2880 2024-04-20T11:28:35.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:35.517+0000 7f107ca00640 1 -- 172.21.15.152:0/4222702587 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f10780b2880 msgr2=0x7f10780b4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:35.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:35.517+0000 7f107ca00640 1 --2- 172.21.15.152:0/4222702587 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f10780b2880 0x7f10780b4be0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f10781601b0 tx=0x7f106002b840 comp rx=0 tx=0).stop 2024-04-20T11:28:35.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:35.517+0000 7f107ca00640 1 -- 172.21.15.152:0/4222702587 shutdown_connections 2024-04-20T11:28:35.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:35.517+0000 7f107ca00640 1 --2- 172.21.15.152:0/4222702587 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f10780cd1b0 0x7f10780cd5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:35.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:35.517+0000 7f107ca00640 1 --2- 172.21.15.152:0/4222702587 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f10780b2880 0x7f10780b4be0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:35.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:35.517+0000 7f107ca00640 1 -- 172.21.15.152:0/4222702587 >> 172.21.15.152:0/4222702587 conn(0x7f107806b980 msgr2=0x7f107806bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:35.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:35.517+0000 7f107ca00640 1 -- 172.21.15.152:0/4222702587 shutdown_connections 2024-04-20T11:28:35.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:35.517+0000 7f107ca00640 1 -- 172.21.15.152:0/4222702587 wait complete. 2024-04-20T11:28:35.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:35.520+0000 7f107ca00640 1 Processor -- start 2024-04-20T11:28:35.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:35.520+0000 7f107ca00640 1 -- start start 2024-04-20T11:28:35.522 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:35.521+0000 7f107ca00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f10780b2880 0x7f107815c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:35.522 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:35.521+0000 7f107ca00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f10780cd1b0 0x7f1078140570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:35.522 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:35.521+0000 7f107ca00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1078146000 0x7f107813e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:35.522 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:35.521+0000 7f107ca00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f10780cdb30 con 0x7f1078146000 2024-04-20T11:28:35.522 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:35.521+0000 7f107ca00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f107815ccf0 con 0x7f10780cd1b0 2024-04-20T11:28:35.522 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:35.521+0000 7f107ca00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f107806dcf0 con 0x7f10780b2880 2024-04-20T11:28:35.522 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:35.521+0000 7f1076800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1078146000 0x7f107813e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:35.522 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:35.521+0000 7f1076800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1078146000 0x7f107813e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49264/0 (socket says 172.21.15.152:49264) 2024-04-20T11:28:35.522 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:35.521+0000 7f1076800640 1 -- 172.21.15.152:0/3782822598 learned_addr learned my addr 172.21.15.152:0/3782822598 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:35.523 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:35.522+0000 7f1075400640 1 --2- 172.21.15.152:0/3782822598 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f10780cd1b0 0x7f1078140570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:35.523 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:35.522+0000 7f1075e00640 1 --2- 172.21.15.152:0/3782822598 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f10780b2880 0x7f107815c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:35.523 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:35.522+0000 7f1076800640 1 -- 172.21.15.152:0/3782822598 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f10780b2880 msgr2=0x7f107815c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:35.523 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:35.522+0000 7f1076800640 1 --2- 172.21.15.152:0/3782822598 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f10780b2880 0x7f107815c010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:35.523 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:35.522+0000 7f1076800640 1 -- 172.21.15.152:0/3782822598 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f10780cd1b0 msgr2=0x7f1078140570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:35.523 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:35.522+0000 7f1076800640 1 --2- 172.21.15.152:0/3782822598 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f10780cd1b0 0x7f1078140570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:35.523 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:35.522+0000 7f1076800640 1 -- 172.21.15.152:0/3782822598 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f107815c550 con 0x7f1078146000 2024-04-20T11:28:35.523 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:35.522+0000 7f1076800640 1 --2- 172.21.15.152:0/3782822598 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1078146000 0x7f107813e720 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f107000cd70 tx=0x7f107000a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:35.523 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:35.522+0000 7f106e000640 1 -- 172.21.15.152:0/3782822598 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1070003b90 con 0x7f1078146000 2024-04-20T11:28:35.523 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:35.522+0000 7f106e000640 1 -- 172.21.15.152:0/3782822598 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1070003d30 con 0x7f1078146000 2024-04-20T11:28:35.523 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:35.522+0000 7f106e000640 1 -- 172.21.15.152:0/3782822598 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1070004240 con 0x7f1078146000 2024-04-20T11:28:35.523 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:35.522+0000 7f107ca00640 1 -- 172.21.15.152:0/3782822598 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1078140ee0 con 0x7f1078146000 2024-04-20T11:28:35.524 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:35.523+0000 7f107ca00640 1 -- 172.21.15.152:0/3782822598 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f107806e240 con 0x7f1078146000 2024-04-20T11:28:35.524 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:35.524+0000 7f107ca00640 1 -- 172.21.15.152:0/3782822598 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1078141280 con 0x7f1078146000 2024-04-20T11:28:35.524 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:35.525+0000 7f106e000640 1 -- 172.21.15.152:0/3782822598 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f10700043e0 con 0x7f1078146000 2024-04-20T11:28:35.526 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:35.526+0000 7f106e000640 1 --2- 172.21.15.152:0/3782822598 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f105406c980 0x7f105406ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:35.526 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:35.526+0000 7f106e000640 1 -- 172.21.15.152:0/3782822598 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f107008c800 con 0x7f1078146000 2024-04-20T11:28:35.526 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:35.526+0000 7f1075e00640 1 --2- 172.21.15.152:0/3782822598 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f105406c980 0x7f105406ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:35.526 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:35.527+0000 7f1075e00640 1 --2- 172.21.15.152:0/3782822598 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f105406c980 0x7f105406ee40 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f106800a750 tx=0x7f1068008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:35.530 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:35.531+0000 7f106e000640 1 -- 172.21.15.152:0/3782822598 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f107815c550 con 0x7f1078146000 2024-04-20T11:28:35.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:35.612+0000 7f106e000640 1 -- 172.21.15.152:0/3782822598 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f10700046d0 con 0x7f1078146000 2024-04-20T11:28:35.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:35.762+0000 7f107ca00640 1 -- 172.21.15.152:0/3782822598 --> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup rm", "vol_name": "cephfs", "group_name": "subvol_grp73750", "target": ["mon-mgr", ""]}) -- 0x7f107806e480 con 0x7f105406c980 2024-04-20T11:28:35.771 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:35.771+0000 7f106e000640 1 -- 172.21.15.152:0/3782822598 <== mgr.4113 v2:172.21.15.152:6832/2285110240 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f106800a380 con 0x7f105406c980 2024-04-20T11:28:35.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:35.774+0000 7f107ca00640 1 -- 172.21.15.152:0/3782822598 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f105406c980 msgr2=0x7f105406ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:35.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:35.774+0000 7f107ca00640 1 --2- 172.21.15.152:0/3782822598 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f105406c980 0x7f105406ee40 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f106800a750 tx=0x7f1068008040 comp rx=0 tx=0).stop 2024-04-20T11:28:35.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:35.774+0000 7f107ca00640 1 -- 172.21.15.152:0/3782822598 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1078146000 msgr2=0x7f107813e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:35.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:35.774+0000 7f107ca00640 1 --2- 172.21.15.152:0/3782822598 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1078146000 0x7f107813e720 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f107000cd70 tx=0x7f107000a660 comp rx=0 tx=0).stop 2024-04-20T11:28:35.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:35.774+0000 7f107ca00640 1 -- 172.21.15.152:0/3782822598 shutdown_connections 2024-04-20T11:28:35.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:35.774+0000 7f107ca00640 1 --2- 172.21.15.152:0/3782822598 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f105406c980 0x7f105406ee40 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:35.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:35.774+0000 7f107ca00640 1 --2- 172.21.15.152:0/3782822598 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1078146000 0x7f107813e720 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:35.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:35.774+0000 7f107ca00640 1 --2- 172.21.15.152:0/3782822598 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f10780cd1b0 0x7f1078140570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:35.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:35.774+0000 7f107ca00640 1 --2- 172.21.15.152:0/3782822598 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f10780b2880 0x7f107815c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:35.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:35.774+0000 7f107ca00640 1 -- 172.21.15.152:0/3782822598 >> 172.21.15.152:0/3782822598 conn(0x7f107806b980 msgr2=0x7f107806bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:35.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:35.774+0000 7f107ca00640 1 -- 172.21.15.152:0/3782822598 shutdown_connections 2024-04-20T11:28:35.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:35.774+0000 7f107ca00640 1 -- 172.21.15.152:0/3782822598 wait complete. 2024-04-20T11:28:35.793 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-20T11:28:35.794 DEBUG:teuthology.orchestra.run.smithi152:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-20T11:28:35.893 DEBUG:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-20T11:28:35.999 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-20T11:28:35.999 INFO:tasks.cephfs.mount:Terminating background process 2024-04-20T11:28:35.999 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-20T11:28:35.999 DEBUG:teuthology.orchestra.run.smithi152:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:28:36.057 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:28:36.254 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.254+0000 7fa637200640 1 Processor -- start 2024-04-20T11:28:36.254 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.254+0000 7fa637200640 1 -- start start 2024-04-20T11:28:36.256 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.256+0000 7fa637200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fa63813e530 0x7fa63806ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:36.256 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.256+0000 7fa637200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fa63806f340 0x7fa638140dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:36.256 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.256+0000 7fa637200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6380cf0a0 con 0x7fa638141310 2024-04-20T11:28:36.256 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.256+0000 7fa637200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fa638141b00 con 0x7fa63813e530 2024-04-20T11:28:36.256 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.256+0000 7fa637200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fa6380cfc50 con 0x7fa63806f340 2024-04-20T11:28:36.257 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.256+0000 7fa635e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fa63813e530 0x7fa63806ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:36.257 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.256+0000 7fa635e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fa63813e530 0x7fa63806ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:59786/0 (socket says 172.21.15.152:59786) 2024-04-20T11:28:36.257 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.256+0000 7fa635e00640 1 -- 172.21.15.152:0/3445262084 learned_addr learned my addr 172.21.15.152:0/3445262084 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:36.257 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.257+0000 7fa635400640 1 --2- 172.21.15.152:0/3445262084 >> v2:172.21.15.175:3300/0 conn(0x7fa63806f340 0x7fa638140dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:36.257 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.257+0000 7fa635e00640 1 -- 172.21.15.152:0/3445262084 >> v2:172.21.15.175:3300/0 conn(0x7fa63806f340 msgr2=0x7fa638140dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:36.257 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.257+0000 7fa635e00640 1 --2- 172.21.15.152:0/3445262084 >> v2:172.21.15.175:3300/0 conn(0x7fa63806f340 0x7fa638140dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:36.257 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.257+0000 7fa635e00640 1 -- 172.21.15.152:0/3445262084 >> v1:172.21.15.175:6789/0 conn(0x7fa638141310 legacy=0x7fa63815efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:36.257 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.257+0000 7fa635e00640 1 -- 172.21.15.152:0/3445262084 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6380d00e0 con 0x7fa63813e530 2024-04-20T11:28:36.258 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.258+0000 7fa635e00640 1 --2- 172.21.15.152:0/3445262084 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa63813e530 0x7fa63806ee00 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fa63806feb0 tx=0x7fa62802e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=875fd1cb2a0fdc15 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:36.258 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.258+0000 7fa634a00640 1 -- 172.21.15.152:0/3445262084 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa62803b070 con 0x7fa63813e530 2024-04-20T11:28:36.258 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.258+0000 7fa634a00640 1 -- 172.21.15.152:0/3445262084 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa628036440 con 0x7fa63813e530 2024-04-20T11:28:36.258 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.258+0000 7fa634a00640 1 -- 172.21.15.152:0/3445262084 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa628036810 con 0x7fa63813e530 2024-04-20T11:28:36.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.259+0000 7fa637200640 1 -- 172.21.15.152:0/3445262084 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa63813e530 msgr2=0x7fa63806ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:36.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.259+0000 7fa637200640 1 --2- 172.21.15.152:0/3445262084 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa63813e530 0x7fa63806ee00 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fa63806feb0 tx=0x7fa62802e0e0 comp rx=0 tx=0).stop 2024-04-20T11:28:36.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.259+0000 7fa637200640 1 -- 172.21.15.152:0/3445262084 shutdown_connections 2024-04-20T11:28:36.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.259+0000 7fa637200640 1 --2- 172.21.15.152:0/3445262084 >> v2:172.21.15.175:3300/0 conn(0x7fa63806f340 0x7fa638140dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:36.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.259+0000 7fa637200640 1 --2- 172.21.15.152:0/3445262084 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa63813e530 0x7fa63806ee00 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:36.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.259+0000 7fa637200640 1 -- 172.21.15.152:0/3445262084 >> 172.21.15.152:0/3445262084 conn(0x7fa638067ae0 msgr2=0x7fa63806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:36.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.259+0000 7fa637200640 1 -- 172.21.15.152:0/3445262084 shutdown_connections 2024-04-20T11:28:36.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.259+0000 7fa637200640 1 -- 172.21.15.152:0/3445262084 wait complete. 2024-04-20T11:28:36.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.261+0000 7fa637200640 1 Processor -- start 2024-04-20T11:28:36.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.261+0000 7fa637200640 1 -- start start 2024-04-20T11:28:36.262 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.262+0000 7fa637200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa63806f340 0x7fa638155700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:36.262 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.262+0000 7fa637200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa63813e530 0x7fa63813cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:36.262 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.262+0000 7fa637200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa638141310 0x7fa63813d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:36.262 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.262+0000 7fa637200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fa6380cf600 con 0x7fa63813e530 2024-04-20T11:28:36.262 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.262+0000 7fa637200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa638072ce0 con 0x7fa638141310 2024-04-20T11:28:36.262 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.262+0000 7fa637200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa6380cf8e0 con 0x7fa63806f340 2024-04-20T11:28:36.262 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.262+0000 7fa635400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa63813e530 0x7fa63813cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:36.262 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.262+0000 7fa635400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa63813e530 0x7fa63813cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:59794/0 (socket says 172.21.15.152:59794) 2024-04-20T11:28:36.262 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.262+0000 7fa635400640 1 -- 172.21.15.152:0/3683993365 learned_addr learned my addr 172.21.15.152:0/3683993365 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:36.262 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.262+0000 7fa636800640 1 --2- 172.21.15.152:0/3683993365 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa638141310 0x7fa63813d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:36.263 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.263+0000 7fa635400640 1 -- 172.21.15.152:0/3683993365 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa63806f340 msgr2=0x7fa638155700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:36.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.263+0000 7fa635e00640 1 --2- 172.21.15.152:0/3683993365 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa63806f340 0x7fa638155700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:36.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.263+0000 7fa635400640 1 --2- 172.21.15.152:0/3683993365 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa63806f340 0x7fa638155700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:36.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.263+0000 7fa635400640 1 -- 172.21.15.152:0/3683993365 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa638141310 msgr2=0x7fa63813d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:36.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.263+0000 7fa635400640 1 --2- 172.21.15.152:0/3683993365 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa638141310 0x7fa63813d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:36.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.263+0000 7fa635400640 1 -- 172.21.15.152:0/3683993365 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6380b1b30 con 0x7fa63813e530 2024-04-20T11:28:36.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.263+0000 7fa635400640 1 --2- 172.21.15.152:0/3683993365 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa63813e530 0x7fa63813cb50 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fa62000d800 tx=0x7fa62000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:36.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.263+0000 7fa62e000640 1 -- 172.21.15.152:0/3683993365 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa62000b6c0 con 0x7fa63813e530 2024-04-20T11:28:36.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.263+0000 7fa62e000640 1 -- 172.21.15.152:0/3683993365 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa620019070 con 0x7fa63813e530 2024-04-20T11:28:36.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.263+0000 7fa637200640 1 -- 172.21.15.152:0/3683993365 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6380b1e10 con 0x7fa63813e530 2024-04-20T11:28:36.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.263+0000 7fa62e000640 1 -- 172.21.15.152:0/3683993365 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa62000ba80 con 0x7fa63813e530 2024-04-20T11:28:36.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.264+0000 7fa637200640 1 -- 172.21.15.152:0/3683993365 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6380b23d0 con 0x7fa63813e530 2024-04-20T11:28:36.265 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.265+0000 7fa637200640 1 -- 172.21.15.152:0/3683993365 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa638075e60 con 0x7fa63813e530 2024-04-20T11:28:36.265 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.266+0000 7fa62e000640 1 -- 172.21.15.152:0/3683993365 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7fa62000bc20 con 0x7fa63813e530 2024-04-20T11:28:36.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.267+0000 7fa62e000640 1 --2- 172.21.15.152:0/3683993365 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fa61406caa0 0x7fa61406ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:36.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.267+0000 7fa62e000640 1 -- 172.21.15.152:0/3683993365 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fa62008c350 con 0x7fa63813e530 2024-04-20T11:28:36.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.268+0000 7fa635e00640 1 --2- 172.21.15.152:0/3683993365 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fa61406caa0 0x7fa61406ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:36.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.268+0000 7fa635e00640 1 --2- 172.21.15.152:0/3683993365 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fa61406caa0 0x7fa61406ef60 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa628038000 tx=0x7fa628032000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:36.271 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.272+0000 7fa62e000640 1 -- 172.21.15.152:0/3683993365 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fa62001c360 con 0x7fa63813e530 2024-04-20T11:28:36.471 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.472+0000 7fa637200640 1 -- 172.21.15.152:0/3683993365 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa6380cddb0 con 0x7fa63813e530 2024-04-20T11:28:36.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.475+0000 7fa62e000640 1 -- 172.21.15.152:0/3683993365 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) ==== 74+0+16729 (secure 0 0 0) 0x7fa62005a4f0 con 0x7fa63813e530 2024-04-20T11:28:36.475 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:28:36.475 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":34,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:28:19.478546+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:28:15.704670+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-20T11:28:16.891101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:28:36.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.477+0000 7fa637200640 1 -- 172.21.15.152:0/3683993365 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fa61406caa0 msgr2=0x7fa61406ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:36.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.477+0000 7fa637200640 1 --2- 172.21.15.152:0/3683993365 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fa61406caa0 0x7fa61406ef60 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa628038000 tx=0x7fa628032000 comp rx=0 tx=0).stop 2024-04-20T11:28:36.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.477+0000 7fa637200640 1 -- 172.21.15.152:0/3683993365 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa63813e530 msgr2=0x7fa63813cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:36.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.477+0000 7fa637200640 1 --2- 172.21.15.152:0/3683993365 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa63813e530 0x7fa63813cb50 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fa62000d800 tx=0x7fa62000dcc0 comp rx=0 tx=0).stop 2024-04-20T11:28:36.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.478+0000 7fa637200640 1 -- 172.21.15.152:0/3683993365 shutdown_connections 2024-04-20T11:28:36.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.478+0000 7fa637200640 1 --2- 172.21.15.152:0/3683993365 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fa61406caa0 0x7fa61406ef60 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:36.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.478+0000 7fa637200640 1 --2- 172.21.15.152:0/3683993365 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa638141310 0x7fa63813d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:36.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.478+0000 7fa637200640 1 --2- 172.21.15.152:0/3683993365 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa63813e530 0x7fa63813cb50 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:36.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.478+0000 7fa637200640 1 --2- 172.21.15.152:0/3683993365 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa63806f340 0x7fa638155700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:36.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.478+0000 7fa637200640 1 -- 172.21.15.152:0/3683993365 >> 172.21.15.152:0/3683993365 conn(0x7fa638067ae0 msgr2=0x7fa638159570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:36.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.478+0000 7fa637200640 1 -- 172.21.15.152:0/3683993365 shutdown_connections 2024-04-20T11:28:36.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.478+0000 7fa637200640 1 -- 172.21.15.152:0/3683993365 wait complete. 2024-04-20T11:28:36.497 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T11:26:47.246571+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T11:28:15.704670+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T11:28:16.891101+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T11:28:36.497 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T11:28:36.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.702+0000 7ff5aaa00640 1 Processor -- start 2024-04-20T11:28:36.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.702+0000 7ff5aaa00640 1 -- start start 2024-04-20T11:28:36.703 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.704+0000 7ff5aaa00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ff5a415e640 0x7ff5a406dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:36.704 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.704+0000 7ff5aaa00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff5a415ea20 0x7ff5a406bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:36.704 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.704+0000 7ff5aaa00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5a40b5070 con 0x7ff5a406f410 2024-04-20T11:28:36.704 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.704+0000 7ff5aaa00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7ff5a406e1a0 con 0x7ff5a415e640 2024-04-20T11:28:36.704 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.704+0000 7ff5aaa00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff5a406e480 con 0x7ff5a415ea20 2024-04-20T11:28:36.704 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.704+0000 7ff5aa000640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7ff5a406f410 0x7ff5a40b3690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:47094/0 (socket says 172.21.15.152:47094) 2024-04-20T11:28:36.704 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.704+0000 7ff5aa000640 1 -- 172.21.15.152:0/2277566864 learned_addr learned my addr 172.21.15.152:0/2277566864 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:28:36.706 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.705+0000 7ff5a8c00640 1 --2- 172.21.15.152:0/2277566864 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff5a415ea20 0x7ff5a406bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:36.706 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.705+0000 7ff5a9600640 1 --2- 172.21.15.152:0/2277566864 >> v2:172.21.15.175:3300/0 conn(0x7ff5a415e640 0x7ff5a406dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:36.706 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.705+0000 7ff5a3e00640 1 -- 172.21.15.152:0/2277566864 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3618349386 0 0) 0x7ff5a40b5070 con 0x7ff5a406f410 2024-04-20T11:28:36.706 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.705+0000 7ff5a3e00640 1 -- 172.21.15.152:0/2277566864 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff58c003520 con 0x7ff5a406f410 2024-04-20T11:28:36.706 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.705+0000 7ff5a3e00640 1 -- 172.21.15.152:0/2277566864 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 734623411 0 0) 0x7ff58c003520 con 0x7ff5a406f410 2024-04-20T11:28:36.706 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.705+0000 7ff5a3e00640 1 -- 172.21.15.152:0/2277566864 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff5a415ea20 msgr2=0x7ff5a406bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:36.707 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.705+0000 7ff5a3e00640 1 --2- 172.21.15.152:0/2277566864 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff5a415ea20 0x7ff5a406bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:36.707 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.705+0000 7ff5a3e00640 1 -- 172.21.15.152:0/2277566864 >> v2:172.21.15.175:3300/0 conn(0x7ff5a415e640 msgr2=0x7ff5a406dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:36.707 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.705+0000 7ff5a3e00640 1 --2- 172.21.15.152:0/2277566864 >> v2:172.21.15.175:3300/0 conn(0x7ff5a415e640 0x7ff5a406dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:36.707 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.705+0000 7ff5a3e00640 1 -- 172.21.15.152:0/2277566864 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5a406e760 con 0x7ff5a406f410 2024-04-20T11:28:36.707 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.705+0000 7ff5a3e00640 1 -- 172.21.15.152:0/2277566864 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7ff59c0019f0 con 0x7ff5a406f410 2024-04-20T11:28:36.707 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.705+0000 7ff5a3e00640 1 -- 172.21.15.152:0/2277566864 >> v1:172.21.15.152:6789/0 conn(0x7ff5a406f410 legacy=0x7ff5a40b3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:36.707 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.705+0000 7ff5a3e00640 1 --2- 172.21.15.152:0/2277566864 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff58c0039b0 0x7ff58c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:36.707 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.705+0000 7ff5a3e00640 1 --2- 172.21.15.152:0/2277566864 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff58c004800 0x7ff58c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:36.707 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.705+0000 7ff5a3e00640 1 --2- 172.21.15.152:0/2277566864 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff58c0051c0 0x7ff58c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:36.707 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.705+0000 7ff5a3e00640 1 -- 172.21.15.152:0/2277566864 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a406e1a0 con 0x7ff58c004800 2024-04-20T11:28:36.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.705+0000 7ff5a3e00640 1 -- 172.21.15.152:0/2277566864 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a406e480 con 0x7ff58c0039b0 2024-04-20T11:28:36.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.705+0000 7ff5a3e00640 1 -- 172.21.15.152:0/2277566864 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff58c0045a0 con 0x7ff58c0051c0 2024-04-20T11:28:36.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.706+0000 7ff5a8c00640 1 --2- 172.21.15.152:0/2277566864 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff58c004800 0x7ff58c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:36.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.706+0000 7ff5a8c00640 1 -- 172.21.15.152:0/2277566864 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff58c0051c0 msgr2=0x7ff58c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:36.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.706+0000 7ff5a8c00640 1 --2- 172.21.15.152:0/2277566864 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff58c0051c0 0x7ff58c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:36.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.706+0000 7ff5a8c00640 1 -- 172.21.15.152:0/2277566864 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff58c0039b0 msgr2=0x7ff58c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:36.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.706+0000 7ff5a8c00640 1 --2- 172.21.15.152:0/2277566864 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff58c0039b0 0x7ff58c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:36.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.706+0000 7ff5a8c00640 1 -- 172.21.15.152:0/2277566864 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff58c003520 con 0x7ff58c004800 2024-04-20T11:28:36.711 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.707+0000 7ff5a8c00640 1 --2- 172.21.15.152:0/2277566864 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff58c004800 0x7ff58c004c80 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7ff590009af0 tx=0x7ff58c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=618092f07a5d3747 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:36.711 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.707+0000 7ff5a3e00640 1 -- 172.21.15.152:0/2277566864 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff590039070 con 0x7ff58c004800 2024-04-20T11:28:36.711 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.707+0000 7ff5a3e00640 1 -- 172.21.15.152:0/2277566864 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff59002c710 con 0x7ff58c004800 2024-04-20T11:28:36.711 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.707+0000 7ff5a3e00640 1 -- 172.21.15.152:0/2277566864 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff590009690 con 0x7ff58c004800 2024-04-20T11:28:36.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.708+0000 7ff5aaa00640 1 -- 172.21.15.152:0/2277566864 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff58c004800 msgr2=0x7ff58c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:36.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.708+0000 7ff5aaa00640 1 --2- 172.21.15.152:0/2277566864 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff58c004800 0x7ff58c004c80 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7ff590009af0 tx=0x7ff58c005830 comp rx=0 tx=0).stop 2024-04-20T11:28:36.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.708+0000 7ff5aa000640 1 -- 172.21.15.152:0/2277566864 reap_dead start 2024-04-20T11:28:36.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.708+0000 7ff5aaa00640 1 -- 172.21.15.152:0/2277566864 shutdown_connections 2024-04-20T11:28:36.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.708+0000 7ff5aaa00640 1 -- 172.21.15.152:0/2277566864 >> 172.21.15.152:0/2277566864 conn(0x7ff5a40b21e0 msgr2=0x7ff5a40b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:36.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.708+0000 7ff5aaa00640 1 -- 172.21.15.152:0/2277566864 shutdown_connections 2024-04-20T11:28:36.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.708+0000 7ff5aaa00640 1 -- 172.21.15.152:0/2277566864 wait complete. 2024-04-20T11:28:36.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.710+0000 7ff5aaa00640 1 Processor -- start 2024-04-20T11:28:36.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.710+0000 7ff5aaa00640 1 -- start start 2024-04-20T11:28:36.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.711+0000 7ff5aaa00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff58c004800 0x7ff5a41510d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:36.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.711+0000 7ff5aaa00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff5a4161750 0x7ff5a4144fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:36.713 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.711+0000 7ff5aaa00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5a41454e0 0x7ff5a4154fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:36.713 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.711+0000 7ff5aaa00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a40ae5d0 con 0x7ff5a4161750 2024-04-20T11:28:36.713 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.711+0000 7ff5aaa00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a40ad0a0 con 0x7ff5a41454e0 2024-04-20T11:28:36.713 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.711+0000 7ff5aaa00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff5a40ae750 con 0x7ff58c004800 2024-04-20T11:28:36.713 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.711+0000 7ff5a8c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff5a4161750 0x7ff5a4144fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:36.713 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.711+0000 7ff5a8c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff5a4161750 0x7ff5a4144fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:59814/0 (socket says 172.21.15.152:59814) 2024-04-20T11:28:36.713 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.711+0000 7ff5a8c00640 1 -- 172.21.15.152:0/1202203202 learned_addr learned my addr 172.21.15.152:0/1202203202 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:36.713 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.711+0000 7ff5aa000640 1 --2- 172.21.15.152:0/1202203202 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5a41454e0 0x7ff5a4154fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:36.713 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.712+0000 7ff5a9600640 1 --2- 172.21.15.152:0/1202203202 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff58c004800 0x7ff5a41510d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:36.713 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.712+0000 7ff5a8c00640 1 -- 172.21.15.152:0/1202203202 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff58c004800 msgr2=0x7ff5a41510d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:36.713 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.712+0000 7ff5a8c00640 1 --2- 172.21.15.152:0/1202203202 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff58c004800 0x7ff5a41510d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:36.713 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.712+0000 7ff5a8c00640 1 -- 172.21.15.152:0/1202203202 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5a41454e0 msgr2=0x7ff5a4154fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:36.713 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.712+0000 7ff5a8c00640 1 --2- 172.21.15.152:0/1202203202 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5a41454e0 0x7ff5a4154fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:36.714 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.712+0000 7ff5a8c00640 1 -- 172.21.15.152:0/1202203202 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5a415f3e0 con 0x7ff5a4161750 2024-04-20T11:28:36.715 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.712+0000 7ff5a8c00640 1 --2- 172.21.15.152:0/1202203202 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff5a4161750 0x7ff5a4144fa0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7ff590000c00 tx=0x7ff590004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:36.715 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.712+0000 7ff5a1600640 1 -- 172.21.15.152:0/1202203202 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff590039050 con 0x7ff5a4161750 2024-04-20T11:28:36.715 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.713+0000 7ff5a1600640 1 -- 172.21.15.152:0/1202203202 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff590045070 con 0x7ff5a4161750 2024-04-20T11:28:36.715 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.713+0000 7ff5a1600640 1 -- 172.21.15.152:0/1202203202 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff590003a50 con 0x7ff5a4161750 2024-04-20T11:28:36.715 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.713+0000 7ff5aaa00640 1 -- 172.21.15.152:0/1202203202 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5a4145970 con 0x7ff5a4161750 2024-04-20T11:28:36.715 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.713+0000 7ff5aaa00640 1 -- 172.21.15.152:0/1202203202 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5a415f960 con 0x7ff5a4161750 2024-04-20T11:28:36.715 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.714+0000 7ff5aaa00640 1 -- 172.21.15.152:0/1202203202 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5a4075e60 con 0x7ff5a4161750 2024-04-20T11:28:36.716 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.715+0000 7ff5a1600640 1 -- 172.21.15.152:0/1202203202 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7ff590003bf0 con 0x7ff5a4161750 2024-04-20T11:28:36.716 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.716+0000 7ff5a1600640 1 --2- 172.21.15.152:0/1202203202 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff58406caa0 0x7ff58406ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:36.716 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.716+0000 7ff5a1600640 1 -- 172.21.15.152:0/1202203202 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7ff5900ad960 con 0x7ff5a4161750 2024-04-20T11:28:36.716 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.716+0000 7ff5a9600640 1 --2- 172.21.15.152:0/1202203202 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff58406caa0 0x7ff58406ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:36.716 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.717+0000 7ff5a9600640 1 --2- 172.21.15.152:0/1202203202 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff58406caa0 0x7ff58406ef60 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff598009ad0 tx=0x7ff598008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:36.720 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.720+0000 7ff5a1600640 1 -- 172.21.15.152:0/1202203202 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ff59004e020 con 0x7ff5a4161750 2024-04-20T11:28:36.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.923+0000 7ff5aaa00640 1 -- 172.21.15.152:0/1202203202 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff5a40aec70 con 0x7ff5a4161750 2024-04-20T11:28:36.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.924+0000 7ff5a1600640 1 -- 172.21.15.152:0/1202203202 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v34) ==== 93+0+10 (secure 0 0 0) 0x7ff59007bb00 con 0x7ff5a4161750 2024-04-20T11:28:36.924 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 1 2024-04-20T11:28:36.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.927+0000 7ff5aaa00640 1 -- 172.21.15.152:0/1202203202 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff58406caa0 msgr2=0x7ff58406ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:36.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.927+0000 7ff5aaa00640 1 --2- 172.21.15.152:0/1202203202 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff58406caa0 0x7ff58406ef60 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff598009ad0 tx=0x7ff598008040 comp rx=0 tx=0).stop 2024-04-20T11:28:36.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.927+0000 7ff5aaa00640 1 -- 172.21.15.152:0/1202203202 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff5a4161750 msgr2=0x7ff5a4144fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:36.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.927+0000 7ff5aaa00640 1 --2- 172.21.15.152:0/1202203202 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff5a4161750 0x7ff5a4144fa0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7ff590000c00 tx=0x7ff590004730 comp rx=0 tx=0).stop 2024-04-20T11:28:36.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.927+0000 7ff5aaa00640 1 -- 172.21.15.152:0/1202203202 shutdown_connections 2024-04-20T11:28:36.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.927+0000 7ff5aaa00640 1 --2- 172.21.15.152:0/1202203202 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff58406caa0 0x7ff58406ef60 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:36.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.927+0000 7ff5aaa00640 1 --2- 172.21.15.152:0/1202203202 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5a41454e0 0x7ff5a4154fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:36.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.927+0000 7ff5aaa00640 1 --2- 172.21.15.152:0/1202203202 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff5a4161750 0x7ff5a4144fa0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:36.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.927+0000 7ff5aaa00640 1 --2- 172.21.15.152:0/1202203202 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff58c004800 0x7ff5a41510d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:36.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.927+0000 7ff5aaa00640 1 -- 172.21.15.152:0/1202203202 >> 172.21.15.152:0/1202203202 conn(0x7ff5a40b21e0 msgr2=0x7ff5a40ad4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:36.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.928+0000 7ff5aaa00640 1 -- 172.21.15.152:0/1202203202 shutdown_connections 2024-04-20T11:28:36.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:36.928+0000 7ff5aaa00640 1 -- 172.21.15.152:0/1202203202 wait complete. 2024-04-20T11:28:36.947 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T11:28:37.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.148+0000 7fcb3ee00640 1 Processor -- start 2024-04-20T11:28:37.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.148+0000 7fcb3ee00640 1 -- start start 2024-04-20T11:28:37.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.150+0000 7fcb3ee00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fcb38146000 0x7fcb380b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:37.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.150+0000 7fcb3ee00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb381463e0 con 0x7fcb380cd8c0 2024-04-20T11:28:37.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.150+0000 7fcb3ee00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb380b3e20 con 0x7fcb380b2170 2024-04-20T11:28:37.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.150+0000 7fcb3ee00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fcb380cdca0 con 0x7fcb38146000 2024-04-20T11:28:37.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.150+0000 7fcb37e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fcb38146000 0x7fcb380b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:37.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.150+0000 7fcb3ce00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fcb380cd8c0 0x7fcb3815ccf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:47102/0 (socket says 172.21.15.152:47102) 2024-04-20T11:28:37.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.150+0000 7fcb3ce00640 1 -- 172.21.15.152:0/539279092 learned_addr learned my addr 172.21.15.152:0/539279092 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:28:37.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.150+0000 7fcb37e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fcb38146000 0x7fcb380b4be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:59828/0 (socket says 172.21.15.152:59828) 2024-04-20T11:28:37.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.151+0000 7fcb37e00640 1 -- 172.21.15.152:0/539279092 >> v1:172.21.15.175:6789/0 conn(0x7fcb380b2170 legacy=0x7fcb380cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:37.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.151+0000 7fcb37e00640 1 -- 172.21.15.152:0/539279092 >> v1:172.21.15.152:6789/0 conn(0x7fcb380cd8c0 legacy=0x7fcb3815ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:37.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.151+0000 7fcb37e00640 1 -- 172.21.15.152:0/539279092 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb380b4100 con 0x7fcb38146000 2024-04-20T11:28:37.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.151+0000 7fcb36a00640 1 -- 172.21.15.152:0/539279092 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 771334387 0 0) 0x7fcb381463e0 con 0x7fcb380cd8c0 2024-04-20T11:28:37.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.152+0000 7fcb37e00640 1 --2- 172.21.15.152:0/539279092 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcb38146000 0x7fcb380b4be0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fcb28009d90 tx=0x7fcb2802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35621c46a2d4f9ef server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:37.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.153+0000 7fcb36a00640 1 -- 172.21.15.152:0/539279092 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb28039070 con 0x7fcb38146000 2024-04-20T11:28:37.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.153+0000 7fcb36a00640 1 -- 172.21.15.152:0/539279092 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb2802cc60 con 0x7fcb38146000 2024-04-20T11:28:37.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.153+0000 7fcb36a00640 1 -- 172.21.15.152:0/539279092 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb28034660 con 0x7fcb38146000 2024-04-20T11:28:37.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.153+0000 7fcb3ee00640 1 -- 172.21.15.152:0/539279092 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcb38146000 msgr2=0x7fcb380b4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:37.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.153+0000 7fcb3ee00640 1 --2- 172.21.15.152:0/539279092 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcb38146000 0x7fcb380b4be0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fcb28009d90 tx=0x7fcb2802beb0 comp rx=0 tx=0).stop 2024-04-20T11:28:37.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.154+0000 7fcb3ee00640 1 -- 172.21.15.152:0/539279092 shutdown_connections 2024-04-20T11:28:37.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.154+0000 7fcb3ee00640 1 --2- 172.21.15.152:0/539279092 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcb38146000 0x7fcb380b4be0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:37.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.154+0000 7fcb3ee00640 1 -- 172.21.15.152:0/539279092 >> 172.21.15.152:0/539279092 conn(0x7fcb3806b980 msgr2=0x7fcb3806bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:37.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.154+0000 7fcb3ee00640 1 -- 172.21.15.152:0/539279092 shutdown_connections 2024-04-20T11:28:37.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.154+0000 7fcb3ee00640 1 -- 172.21.15.152:0/539279092 wait complete. 2024-04-20T11:28:37.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.155+0000 7fcb3ee00640 1 Processor -- start 2024-04-20T11:28:37.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.156+0000 7fcb3ee00640 1 -- start start 2024-04-20T11:28:37.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.156+0000 7fcb3ee00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcb380b2170 0x7fcb3806e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:37.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.157+0000 7fcb3ee00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcb380cd8c0 0x7fcb3813e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:37.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.157+0000 7fcb3ee00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcb38146000 0x7fcb3806dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:37.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.157+0000 7fcb3ee00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fcb380b2620 con 0x7fcb380b2170 2024-04-20T11:28:37.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.157+0000 7fcb3ee00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcb380b4750 con 0x7fcb380cd8c0 2024-04-20T11:28:37.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.157+0000 7fcb3ee00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fcb380b27a0 con 0x7fcb38146000 2024-04-20T11:28:37.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.157+0000 7fcb37e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcb380b2170 0x7fcb3806e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:37.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.157+0000 7fcb37e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcb380b2170 0x7fcb3806e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:59844/0 (socket says 172.21.15.152:59844) 2024-04-20T11:28:37.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.157+0000 7fcb37e00640 1 -- 172.21.15.152:0/613489605 learned_addr learned my addr 172.21.15.152:0/613489605 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:37.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.157+0000 7fcb37e00640 1 -- 172.21.15.152:0/613489605 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcb38146000 msgr2=0x7fcb3806dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:37.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.157+0000 7fcb37e00640 1 --2- 172.21.15.152:0/613489605 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcb38146000 0x7fcb3806dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:37.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.157+0000 7fcb37e00640 1 -- 172.21.15.152:0/613489605 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcb380cd8c0 msgr2=0x7fcb3813e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:37.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.157+0000 7fcb37e00640 1 --2- 172.21.15.152:0/613489605 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcb380cd8c0 0x7fcb3813e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:37.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.157+0000 7fcb37e00640 1 -- 172.21.15.152:0/613489605 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb3806eac0 con 0x7fcb380b2170 2024-04-20T11:28:37.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.158+0000 7fcb37e00640 1 --2- 172.21.15.152:0/613489605 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcb380b2170 0x7fcb3806e450 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fcb28037040 tx=0x7fcb28004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:37.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.158+0000 7fcb2fe00640 1 -- 172.21.15.152:0/613489605 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb28039050 con 0x7fcb380b2170 2024-04-20T11:28:37.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.158+0000 7fcb2fe00640 1 -- 172.21.15.152:0/613489605 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb28045070 con 0x7fcb380b2170 2024-04-20T11:28:37.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.158+0000 7fcb3ee00640 1 -- 172.21.15.152:0/613489605 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb3806eda0 con 0x7fcb380b2170 2024-04-20T11:28:37.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.158+0000 7fcb2fe00640 1 -- 172.21.15.152:0/613489605 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb2803ea50 con 0x7fcb380b2170 2024-04-20T11:28:37.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.158+0000 7fcb3ee00640 1 -- 172.21.15.152:0/613489605 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb3806f360 con 0x7fcb380b2170 2024-04-20T11:28:37.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.159+0000 7fcb3ee00640 1 -- 172.21.15.152:0/613489605 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb380689f0 con 0x7fcb380b2170 2024-04-20T11:28:37.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.161+0000 7fcb2fe00640 1 -- 172.21.15.152:0/613489605 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7fcb2802cb20 con 0x7fcb380b2170 2024-04-20T11:28:37.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.161+0000 7fcb2fe00640 1 --2- 172.21.15.152:0/613489605 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fcb1406caa0 0x7fcb1406ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:37.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.162+0000 7fcb2fe00640 1 -- 172.21.15.152:0/613489605 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fcb280aecf0 con 0x7fcb380b2170 2024-04-20T11:28:37.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.162+0000 7fcb37400640 1 --2- 172.21.15.152:0/613489605 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fcb1406caa0 0x7fcb1406ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:37.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.163+0000 7fcb37400640 1 --2- 172.21.15.152:0/613489605 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fcb1406caa0 0x7fcb1406ef60 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fcb20000f60 tx=0x7fcb200079b0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:37.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.166+0000 7fcb2fe00640 1 -- 172.21.15.152:0/613489605 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fcb2807cfb0 con 0x7fcb380b2170 2024-04-20T11:28:37.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.369+0000 7fcb3ee00640 1 -- 172.21.15.152:0/613489605 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fcb380d1200 con 0x7fcb380b2170 2024-04-20T11:28:37.370 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.370+0000 7fcb2fe00640 1 -- 172.21.15.152:0/613489605 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v34) ==== 104+0+11 (secure 0 0 0) 0x7fcb2807c8b0 con 0x7fcb380b2170 2024-04-20T11:28:37.370 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 32 2024-04-20T11:28:37.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.373+0000 7fcb3ee00640 1 -- 172.21.15.152:0/613489605 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fcb1406caa0 msgr2=0x7fcb1406ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:37.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.373+0000 7fcb3ee00640 1 --2- 172.21.15.152:0/613489605 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fcb1406caa0 0x7fcb1406ef60 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fcb20000f60 tx=0x7fcb200079b0 comp rx=0 tx=0).stop 2024-04-20T11:28:37.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.373+0000 7fcb3ee00640 1 -- 172.21.15.152:0/613489605 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcb380b2170 msgr2=0x7fcb3806e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:37.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.373+0000 7fcb3ee00640 1 --2- 172.21.15.152:0/613489605 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcb380b2170 0x7fcb3806e450 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fcb28037040 tx=0x7fcb28004a40 comp rx=0 tx=0).stop 2024-04-20T11:28:37.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.373+0000 7fcb3ee00640 1 -- 172.21.15.152:0/613489605 shutdown_connections 2024-04-20T11:28:37.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.373+0000 7fcb3ee00640 1 --2- 172.21.15.152:0/613489605 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fcb1406caa0 0x7fcb1406ef60 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:37.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.373+0000 7fcb3ee00640 1 --2- 172.21.15.152:0/613489605 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcb38146000 0x7fcb3806dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:37.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.373+0000 7fcb3ee00640 1 --2- 172.21.15.152:0/613489605 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcb380cd8c0 0x7fcb3813e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:37.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.373+0000 7fcb3ee00640 1 --2- 172.21.15.152:0/613489605 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcb380b2170 0x7fcb3806e450 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:37.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.373+0000 7fcb3ee00640 1 -- 172.21.15.152:0/613489605 >> 172.21.15.152:0/613489605 conn(0x7fcb3806b980 msgr2=0x7fcb380b0f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:37.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.373+0000 7fcb3ee00640 1 -- 172.21.15.152:0/613489605 shutdown_connections 2024-04-20T11:28:37.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.373+0000 7fcb3ee00640 1 -- 172.21.15.152:0/613489605 wait complete. 2024-04-20T11:28:37.393 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T11:28:37.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.603+0000 7f7782200640 1 Processor -- start 2024-04-20T11:28:37.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.604+0000 7f7782200640 1 -- start start 2024-04-20T11:28:37.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.605+0000 7f7782200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f777c146000 0x7f777c0b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:37.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.605+0000 7f7782200640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f777c1463e0 con 0x7f777c0cd8c0 2024-04-20T11:28:37.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.606+0000 7f7782200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f777c0b3e20 con 0x7f777c0b2170 2024-04-20T11:28:37.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.606+0000 7f7782200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f777c0b4750 con 0x7f777c146000 2024-04-20T11:28:37.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.606+0000 7f777be00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f777c0cd8c0 0x7f777c15ccf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:47106/0 (socket says 172.21.15.152:47106) 2024-04-20T11:28:37.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.606+0000 7f777be00640 1 -- 172.21.15.152:0/1941610428 learned_addr learned my addr 172.21.15.152:0/1941610428 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:28:37.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.606+0000 7f777b400640 1 --2- 172.21.15.152:0/1941610428 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f777c146000 0x7f777c0b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:37.607 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.607+0000 7f777b400640 1 -- 172.21.15.152:0/1941610428 >> v1:172.21.15.175:6789/0 conn(0x7f777c0b2170 legacy=0x7f777c0cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:37.607 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.607+0000 7f777b400640 1 -- 172.21.15.152:0/1941610428 >> v1:172.21.15.152:6789/0 conn(0x7f777c0cd8c0 legacy=0x7f777c15ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:37.607 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.607+0000 7f777b400640 1 -- 172.21.15.152:0/1941610428 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f777c0b4100 con 0x7f777c146000 2024-04-20T11:28:37.608 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.609+0000 7f777b400640 1 --2- 172.21.15.152:0/1941610428 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f777c146000 0x7f777c0b4be0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f776c0097d0 tx=0x7f776c02b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8548fa49df2e4a01 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:37.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.612+0000 7f777a000640 1 -- 172.21.15.152:0/1941610428 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f776c035020 con 0x7f777c146000 2024-04-20T11:28:37.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.612+0000 7f777a000640 1 -- 172.21.15.152:0/1941610428 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f776c031810 con 0x7f777c146000 2024-04-20T11:28:37.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.612+0000 7f777a000640 1 -- 172.21.15.152:0/1941610428 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f776c031be0 con 0x7f777c146000 2024-04-20T11:28:37.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.612+0000 7f7782200640 1 -- 172.21.15.152:0/1941610428 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f777c146000 msgr2=0x7f777c0b4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:37.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.612+0000 7f7782200640 1 --2- 172.21.15.152:0/1941610428 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f777c146000 0x7f777c0b4be0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f776c0097d0 tx=0x7f776c02b8b0 comp rx=0 tx=0).stop 2024-04-20T11:28:37.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.612+0000 7f7782200640 1 -- 172.21.15.152:0/1941610428 shutdown_connections 2024-04-20T11:28:37.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.612+0000 7f7782200640 1 --2- 172.21.15.152:0/1941610428 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f777c146000 0x7f777c0b4be0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:37.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.613+0000 7f7782200640 1 -- 172.21.15.152:0/1941610428 >> 172.21.15.152:0/1941610428 conn(0x7f777c06b980 msgr2=0x7f777c06bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:37.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.613+0000 7f7782200640 1 -- 172.21.15.152:0/1941610428 shutdown_connections 2024-04-20T11:28:37.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.613+0000 7f7782200640 1 -- 172.21.15.152:0/1941610428 wait complete. 2024-04-20T11:28:37.614 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.615+0000 7f7782200640 1 Processor -- start 2024-04-20T11:28:37.614 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.615+0000 7f7782200640 1 -- start start 2024-04-20T11:28:37.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.616+0000 7f7782200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f777c0b2170 0x7f777c0b1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:37.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.616+0000 7f7782200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f777c0cd8c0 0x7f777c13e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:37.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.616+0000 7f7782200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f777c146000 0x7f777c1518c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:37.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.616+0000 7f7782200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f777c0b2770 con 0x7f777c0cd8c0 2024-04-20T11:28:37.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.616+0000 7f7782200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f777c0b25f0 con 0x7f777c0b2170 2024-04-20T11:28:37.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.616+0000 7f7782200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f777c0b28f0 con 0x7f777c146000 2024-04-20T11:28:37.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.617+0000 7f777aa00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f777c0cd8c0 0x7f777c13e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:37.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.617+0000 7f777aa00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f777c0cd8c0 0x7f777c13e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:59858/0 (socket says 172.21.15.152:59858) 2024-04-20T11:28:37.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.617+0000 7f777aa00640 1 -- 172.21.15.152:0/3730048334 learned_addr learned my addr 172.21.15.152:0/3730048334 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:37.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.617+0000 7f777b400640 1 --2- 172.21.15.152:0/3730048334 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f777c0b2170 0x7f777c0b1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:37.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.617+0000 7f777aa00640 1 -- 172.21.15.152:0/3730048334 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f777c146000 msgr2=0x7f777c1518c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:37.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.617+0000 7f777aa00640 1 --2- 172.21.15.152:0/3730048334 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f777c146000 0x7f777c1518c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:37.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.617+0000 7f777aa00640 1 -- 172.21.15.152:0/3730048334 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f777c0b2170 msgr2=0x7f777c0b1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:37.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.617+0000 7f777aa00640 1 --2- 172.21.15.152:0/3730048334 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f777c0b2170 0x7f777c0b1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:37.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.617+0000 7f777aa00640 1 -- 172.21.15.152:0/3730048334 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f777c142120 con 0x7f777c0cd8c0 2024-04-20T11:28:37.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.617+0000 7f777b400640 1 --2- 172.21.15.152:0/3730048334 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f777c0b2170 0x7f777c0b1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:28:37.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.618+0000 7f777aa00640 1 --2- 172.21.15.152:0/3730048334 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f777c0cd8c0 0x7f777c13e210 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f776400c730 tx=0x7f776400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:37.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.618+0000 7f7773400640 1 -- 172.21.15.152:0/3730048334 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7764003b70 con 0x7f777c0cd8c0 2024-04-20T11:28:37.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.618+0000 7f7773400640 1 -- 172.21.15.152:0/3730048334 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7764003d30 con 0x7f777c0cd8c0 2024-04-20T11:28:37.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.618+0000 7f7782200640 1 -- 172.21.15.152:0/3730048334 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f777c0aee30 con 0x7f777c0cd8c0 2024-04-20T11:28:37.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.618+0000 7f7773400640 1 -- 172.21.15.152:0/3730048334 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f776400aa90 con 0x7f777c0cd8c0 2024-04-20T11:28:37.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.618+0000 7f7782200640 1 -- 172.21.15.152:0/3730048334 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f777c0af3f0 con 0x7f777c0cd8c0 2024-04-20T11:28:37.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.619+0000 7f7782200640 1 -- 172.21.15.152:0/3730048334 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f777c0689f0 con 0x7f777c0cd8c0 2024-04-20T11:28:37.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.620+0000 7f7773400640 1 -- 172.21.15.152:0/3730048334 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f776400ac30 con 0x7f777c0cd8c0 2024-04-20T11:28:37.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.621+0000 7f7773400640 1 --2- 172.21.15.152:0/3730048334 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f775806caa0 0x7f775806ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:37.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.621+0000 7f7773400640 1 -- 172.21.15.152:0/3730048334 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f776408bd80 con 0x7f777c0cd8c0 2024-04-20T11:28:37.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.621+0000 7f777b400640 1 --2- 172.21.15.152:0/3730048334 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f775806caa0 0x7f775806ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:37.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.622+0000 7f777b400640 1 --2- 172.21.15.152:0/3730048334 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f775806caa0 0x7f775806ef60 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f776c036000 tx=0x7f776c008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:37.625 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.626+0000 7f7773400640 1 -- 172.21.15.152:0/3730048334 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f777c142120 con 0x7f777c0cd8c0 2024-04-20T11:28:37.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.824+0000 7f7782200640 1 -- 172.21.15.152:0/3730048334 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f777c06f9a0 con 0x7f777c0cd8c0 2024-04-20T11:28:37.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.825+0000 7f7773400640 1 -- 172.21.15.152:0/3730048334 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v34) ==== 100+0+11 (secure 0 0 0) 0x7f7764059f30 con 0x7f777c0cd8c0 2024-04-20T11:28:37.825 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 64 2024-04-20T11:28:37.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.828+0000 7f7782200640 1 -- 172.21.15.152:0/3730048334 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f775806caa0 msgr2=0x7f775806ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:37.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.828+0000 7f7782200640 1 --2- 172.21.15.152:0/3730048334 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f775806caa0 0x7f775806ef60 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f776c036000 tx=0x7f776c008030 comp rx=0 tx=0).stop 2024-04-20T11:28:37.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.828+0000 7f7782200640 1 -- 172.21.15.152:0/3730048334 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f777c0cd8c0 msgr2=0x7f777c13e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:37.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.828+0000 7f7782200640 1 --2- 172.21.15.152:0/3730048334 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f777c0cd8c0 0x7f777c13e210 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f776400c730 tx=0x7f776400cbf0 comp rx=0 tx=0).stop 2024-04-20T11:28:37.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.828+0000 7f7782200640 1 -- 172.21.15.152:0/3730048334 shutdown_connections 2024-04-20T11:28:37.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.828+0000 7f7782200640 1 --2- 172.21.15.152:0/3730048334 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f775806caa0 0x7f775806ef60 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:37.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.828+0000 7f7782200640 1 --2- 172.21.15.152:0/3730048334 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f777c146000 0x7f777c1518c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:37.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.828+0000 7f7782200640 1 --2- 172.21.15.152:0/3730048334 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f777c0cd8c0 0x7f777c13e210 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:37.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.828+0000 7f7782200640 1 --2- 172.21.15.152:0/3730048334 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f777c0b2170 0x7f777c0b1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:37.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.828+0000 7f7782200640 1 -- 172.21.15.152:0/3730048334 >> 172.21.15.152:0/3730048334 conn(0x7f777c06b980 msgr2=0x7f777c06bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:37.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.828+0000 7f7782200640 1 -- 172.21.15.152:0/3730048334 shutdown_connections 2024-04-20T11:28:37.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:37.828+0000 7f7782200640 1 -- 172.21.15.152:0/3730048334 wait complete. 2024-04-20T11:28:37.847 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-20T11:28:38.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.042+0000 7fd035200640 1 Processor -- start 2024-04-20T11:28:38.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.042+0000 7fd035200640 1 -- start start 2024-04-20T11:28:38.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.043+0000 7fd035200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd03015c460 0x7fd03006de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:38.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.043+0000 7fd035200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fd0300afcc0 0x7fd0300d0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:38.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.043+0000 7fd035200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fd0300d0760 0x7fd030071db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:38.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.043+0000 7fd035200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fd0300d0bb0 con 0x7fd0300d0760 2024-04-20T11:28:38.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.044+0000 7fd035200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fd0301624c0 con 0x7fd0300afcc0 2024-04-20T11:28:38.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.044+0000 7fd035200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd0300af7e0 con 0x7fd03015c460 2024-04-20T11:28:38.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.044+0000 7fd02e800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fd0300d0760 0x7fd030071db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:38.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.044+0000 7fd02e800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fd0300d0760 0x7fd030071db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:59874/0 (socket says 172.21.15.152:59874) 2024-04-20T11:28:38.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.044+0000 7fd02e800640 1 -- 172.21.15.152:0/49608742 learned_addr learned my addr 172.21.15.152:0/49608742 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:38.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.044+0000 7fd02d400640 1 --2- 172.21.15.152:0/49608742 >> v2:172.21.15.175:3300/0 conn(0x7fd0300afcc0 0x7fd0300d0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:38.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.044+0000 7fd02de00640 1 --2- 172.21.15.152:0/49608742 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd03015c460 0x7fd03006de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:38.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.045+0000 7fd02e800640 1 -- 172.21.15.152:0/49608742 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd03015c460 msgr2=0x7fd03006de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:38.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.045+0000 7fd02e800640 1 --2- 172.21.15.152:0/49608742 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd03015c460 0x7fd03006de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:38.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.045+0000 7fd02e800640 1 -- 172.21.15.152:0/49608742 >> v2:172.21.15.175:3300/0 conn(0x7fd0300afcc0 msgr2=0x7fd0300d0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:38.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.045+0000 7fd02e800640 1 --2- 172.21.15.152:0/49608742 >> v2:172.21.15.175:3300/0 conn(0x7fd0300afcc0 0x7fd0300d0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:38.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.045+0000 7fd02e800640 1 -- 172.21.15.152:0/49608742 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd030162710 con 0x7fd0300d0760 2024-04-20T11:28:38.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.046+0000 7fd02e800640 1 --2- 172.21.15.152:0/49608742 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd0300d0760 0x7fd030071db0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fd03006ffb0 tx=0x7fd02002b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d877b939566765a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:38.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.047+0000 7fd02ca00640 1 -- 172.21.15.152:0/49608742 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd020038070 con 0x7fd0300d0760 2024-04-20T11:28:38.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.047+0000 7fd02ca00640 1 -- 172.21.15.152:0/49608742 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd02002f050 con 0x7fd0300d0760 2024-04-20T11:28:38.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.047+0000 7fd02ca00640 1 -- 172.21.15.152:0/49608742 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd020031970 con 0x7fd0300d0760 2024-04-20T11:28:38.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.047+0000 7fd035200640 1 -- 172.21.15.152:0/49608742 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd0300d0760 msgr2=0x7fd030071db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:38.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.047+0000 7fd035200640 1 --2- 172.21.15.152:0/49608742 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd0300d0760 0x7fd030071db0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fd03006ffb0 tx=0x7fd02002b7f0 comp rx=0 tx=0).stop 2024-04-20T11:28:38.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.047+0000 7fd035200640 1 -- 172.21.15.152:0/49608742 shutdown_connections 2024-04-20T11:28:38.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.047+0000 7fd035200640 1 --2- 172.21.15.152:0/49608742 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd0300d0760 0x7fd030071db0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:38.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.047+0000 7fd035200640 1 --2- 172.21.15.152:0/49608742 >> v2:172.21.15.175:3300/0 conn(0x7fd0300afcc0 0x7fd0300d0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:38.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.047+0000 7fd035200640 1 --2- 172.21.15.152:0/49608742 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd03015c460 0x7fd03006de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:38.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.047+0000 7fd035200640 1 -- 172.21.15.152:0/49608742 >> 172.21.15.152:0/49608742 conn(0x7fd030067cf0 msgr2=0x7fd03006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:38.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.047+0000 7fd035200640 1 -- 172.21.15.152:0/49608742 shutdown_connections 2024-04-20T11:28:38.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.047+0000 7fd035200640 1 -- 172.21.15.152:0/49608742 wait complete. 2024-04-20T11:28:38.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.049+0000 7fd035200640 1 Processor -- start 2024-04-20T11:28:38.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.049+0000 7fd035200640 1 -- start start 2024-04-20T11:28:38.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.050+0000 7fd035200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd0300afcc0 0x7fd03015dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:38.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.050+0000 7fd035200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd0300d0760 0x7fd0300cce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:38.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.050+0000 7fd035200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd03015c460 0x7fd030155bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:38.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.050+0000 7fd035200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fd03015c840 con 0x7fd0300afcc0 2024-04-20T11:28:38.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.050+0000 7fd035200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd030162cd0 con 0x7fd0300d0760 2024-04-20T11:28:38.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.050+0000 7fd035200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd0300af490 con 0x7fd03015c460 2024-04-20T11:28:38.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.050+0000 7fd02de00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd0300afcc0 0x7fd03015dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:38.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.050+0000 7fd02de00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd0300afcc0 0x7fd03015dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:59888/0 (socket says 172.21.15.152:59888) 2024-04-20T11:28:38.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.050+0000 7fd02de00640 1 -- 172.21.15.152:0/1464911081 learned_addr learned my addr 172.21.15.152:0/1464911081 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:38.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.051+0000 7fd02d400640 1 --2- 172.21.15.152:0/1464911081 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd0300d0760 0x7fd0300cce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:38.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.051+0000 7fd02de00640 1 -- 172.21.15.152:0/1464911081 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd03015c460 msgr2=0x7fd030155bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:38.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.051+0000 7fd02de00640 1 --2- 172.21.15.152:0/1464911081 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd03015c460 0x7fd030155bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:38.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.051+0000 7fd02de00640 1 -- 172.21.15.152:0/1464911081 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd0300d0760 msgr2=0x7fd0300cce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:38.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.051+0000 7fd02de00640 1 --2- 172.21.15.152:0/1464911081 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd0300d0760 0x7fd0300cce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:38.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.051+0000 7fd02de00640 1 -- 172.21.15.152:0/1464911081 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd03015cb40 con 0x7fd0300afcc0 2024-04-20T11:28:38.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.051+0000 7fd02de00640 1 --2- 172.21.15.152:0/1464911081 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd0300afcc0 0x7fd03015dba0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fd02800efc0 tx=0x7fd02800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:38.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.051+0000 7fd026000640 1 -- 172.21.15.152:0/1464911081 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd028018020 con 0x7fd0300afcc0 2024-04-20T11:28:38.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.051+0000 7fd026000640 1 -- 172.21.15.152:0/1464911081 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd028014480 con 0x7fd0300afcc0 2024-04-20T11:28:38.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.051+0000 7fd026000640 1 -- 172.21.15.152:0/1464911081 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd028014850 con 0x7fd0300afcc0 2024-04-20T11:28:38.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.051+0000 7fd035200640 1 -- 172.21.15.152:0/1464911081 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd03015ce00 con 0x7fd0300afcc0 2024-04-20T11:28:38.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.052+0000 7fd035200640 1 -- 172.21.15.152:0/1464911081 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd03015d3c0 con 0x7fd0300afcc0 2024-04-20T11:28:38.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.053+0000 7fd035200640 1 -- 172.21.15.152:0/1464911081 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0300689f0 con 0x7fd0300afcc0 2024-04-20T11:28:38.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.054+0000 7fd026000640 1 -- 172.21.15.152:0/1464911081 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7fd028005d10 con 0x7fd0300afcc0 2024-04-20T11:28:38.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.055+0000 7fd026000640 1 --2- 172.21.15.152:0/1464911081 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fd00c06caa0 0x7fd00c06ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:38.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.055+0000 7fd026000640 1 -- 172.21.15.152:0/1464911081 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fd02808cd80 con 0x7fd0300afcc0 2024-04-20T11:28:38.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.055+0000 7fd02d400640 1 --2- 172.21.15.152:0/1464911081 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fd00c06caa0 0x7fd00c06ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:38.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.056+0000 7fd02d400640 1 --2- 172.21.15.152:0/1464911081 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fd00c06caa0 0x7fd00c06ef60 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd0300d0e70 tx=0x7fd018008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:38.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.060+0000 7fd026000640 1 -- 172.21.15.152:0/1464911081 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fd028022030 con 0x7fd0300afcc0 2024-04-20T11:28:38.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.273+0000 7fd035200640 1 -- 172.21.15.152:0/1464911081 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fd03014d4b0 con 0x7fd0300afcc0 2024-04-20T11:28:38.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.274+0000 7fd026000640 1 -- 172.21.15.152:0/1464911081 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v34) ==== 80+0+0 (secure 0 0 0) 0x7fd028010040 con 0x7fd0300afcc0 2024-04-20T11:28:38.274 INFO:teuthology.orchestra.run.smithi152.stderr:listed 0 entries 2024-04-20T11:28:38.277 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.277+0000 7fd035200640 1 -- 172.21.15.152:0/1464911081 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fd00c06caa0 msgr2=0x7fd00c06ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:38.277 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.277+0000 7fd035200640 1 --2- 172.21.15.152:0/1464911081 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fd00c06caa0 0x7fd00c06ef60 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd0300d0e70 tx=0x7fd018008040 comp rx=0 tx=0).stop 2024-04-20T11:28:38.277 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.277+0000 7fd035200640 1 -- 172.21.15.152:0/1464911081 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd0300afcc0 msgr2=0x7fd03015dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:38.277 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.277+0000 7fd035200640 1 --2- 172.21.15.152:0/1464911081 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd0300afcc0 0x7fd03015dba0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fd02800efc0 tx=0x7fd02800c490 comp rx=0 tx=0).stop 2024-04-20T11:28:38.277 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.277+0000 7fd035200640 1 -- 172.21.15.152:0/1464911081 shutdown_connections 2024-04-20T11:28:38.277 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.277+0000 7fd035200640 1 --2- 172.21.15.152:0/1464911081 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fd00c06caa0 0x7fd00c06ef60 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:38.277 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.277+0000 7fd035200640 1 --2- 172.21.15.152:0/1464911081 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd03015c460 0x7fd030155bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:38.277 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.277+0000 7fd035200640 1 --2- 172.21.15.152:0/1464911081 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd0300d0760 0x7fd0300cce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:38.277 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.277+0000 7fd035200640 1 --2- 172.21.15.152:0/1464911081 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd0300afcc0 0x7fd03015dba0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:38.277 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.277+0000 7fd035200640 1 -- 172.21.15.152:0/1464911081 >> 172.21.15.152:0/1464911081 conn(0x7fd030067cf0 msgr2=0x7fd0300b3b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:38.277 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.277+0000 7fd035200640 1 -- 172.21.15.152:0/1464911081 shutdown_connections 2024-04-20T11:28:38.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.277+0000 7fd035200640 1 -- 172.21.15.152:0/1464911081 wait complete. 2024-04-20T11:28:38.295 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi152.front.sepia.ceph.com... 2024-04-20T11:28:38.295 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:28:38.295 DEBUG:teuthology.orchestra.run.smithi152:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-20T11:28:38.371 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi152.stderr:ceph-fuse[36577]: fuse finished with error 0 and tester_r 0 2024-04-20T11:28:38.377 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi152.front.sepia.ceph.com 2024-04-20T11:28:38.377 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:28:38.377 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-20T11:28:38.575 INFO:tasks.cephfs.mount:Terminating background process 2024-04-20T11:28:38.575 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T11:28:38.575 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:28:38.625 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:28:38.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.830+0000 7fcd51000640 1 Processor -- start 2024-04-20T11:28:38.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.830+0000 7fcd51000640 1 -- start start 2024-04-20T11:28:38.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.831+0000 7fcd51000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fcd4c13e530 0x7fcd4c06ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:38.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.831+0000 7fcd51000640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fcd4c06f340 0x7fcd4c140dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:38.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.831+0000 7fcd51000640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd4c0cf0a0 con 0x7fcd4c141310 2024-04-20T11:28:38.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.831+0000 7fcd51000640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fcd4c141b00 con 0x7fcd4c13e530 2024-04-20T11:28:38.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.831+0000 7fcd51000640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fcd4c0cfc50 con 0x7fcd4c06f340 2024-04-20T11:28:38.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.831+0000 7fcd4b400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fcd4c13e530 0x7fcd4c06ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:38.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.832+0000 7fcd4b400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fcd4c13e530 0x7fcd4c06ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:59904/0 (socket says 172.21.15.152:59904) 2024-04-20T11:28:38.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.832+0000 7fcd4b400640 1 -- 172.21.15.152:0/3564194351 learned_addr learned my addr 172.21.15.152:0/3564194351 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:38.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.832+0000 7fcd43e00640 1 --2- 172.21.15.152:0/3564194351 >> v2:172.21.15.175:3300/0 conn(0x7fcd4c06f340 0x7fcd4c140dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:38.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.832+0000 7fcd4b400640 1 -- 172.21.15.152:0/3564194351 >> v2:172.21.15.175:3300/0 conn(0x7fcd4c06f340 msgr2=0x7fcd4c140dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:38.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.832+0000 7fcd4b400640 1 --2- 172.21.15.152:0/3564194351 >> v2:172.21.15.175:3300/0 conn(0x7fcd4c06f340 0x7fcd4c140dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:38.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.832+0000 7fcd4b400640 1 -- 172.21.15.152:0/3564194351 >> v1:172.21.15.175:6789/0 conn(0x7fcd4c141310 legacy=0x7fcd4c15efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:38.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.832+0000 7fcd4b400640 1 -- 172.21.15.152:0/3564194351 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd4c0d00e0 con 0x7fcd4c13e530 2024-04-20T11:28:38.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.832+0000 7fcd43e00640 1 --2- 172.21.15.152:0/3564194351 >> v2:172.21.15.175:3300/0 conn(0x7fcd4c06f340 0x7fcd4c140dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:28:38.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.833+0000 7fcd4b400640 1 --2- 172.21.15.152:0/3564194351 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcd4c13e530 0x7fcd4c06ee00 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c06feb0 tx=0x7fcd3c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1abb5fe71c3d8c7e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:38.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.833+0000 7fcd4aa00640 1 -- 172.21.15.152:0/3564194351 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd3c035020 con 0x7fcd4c13e530 2024-04-20T11:28:38.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.833+0000 7fcd4aa00640 1 -- 172.21.15.152:0/3564194351 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd3c031870 con 0x7fcd4c13e530 2024-04-20T11:28:38.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.833+0000 7fcd4aa00640 1 -- 172.21.15.152:0/3564194351 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd3c031c40 con 0x7fcd4c13e530 2024-04-20T11:28:38.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.834+0000 7fcd51000640 1 -- 172.21.15.152:0/3564194351 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcd4c13e530 msgr2=0x7fcd4c06ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:38.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.834+0000 7fcd51000640 1 --2- 172.21.15.152:0/3564194351 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcd4c13e530 0x7fcd4c06ee00 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c06feb0 tx=0x7fcd3c02b910 comp rx=0 tx=0).stop 2024-04-20T11:28:38.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.834+0000 7fcd51000640 1 -- 172.21.15.152:0/3564194351 shutdown_connections 2024-04-20T11:28:38.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.834+0000 7fcd51000640 1 --2- 172.21.15.152:0/3564194351 >> v2:172.21.15.175:3300/0 conn(0x7fcd4c06f340 0x7fcd4c140dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:38.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.834+0000 7fcd51000640 1 --2- 172.21.15.152:0/3564194351 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcd4c13e530 0x7fcd4c06ee00 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:38.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.834+0000 7fcd51000640 1 -- 172.21.15.152:0/3564194351 >> 172.21.15.152:0/3564194351 conn(0x7fcd4c067ae0 msgr2=0x7fcd4c06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:38.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.834+0000 7fcd51000640 1 -- 172.21.15.152:0/3564194351 shutdown_connections 2024-04-20T11:28:38.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.834+0000 7fcd51000640 1 -- 172.21.15.152:0/3564194351 wait complete. 2024-04-20T11:28:38.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.836+0000 7fcd51000640 1 Processor -- start 2024-04-20T11:28:38.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.836+0000 7fcd51000640 1 -- start start 2024-04-20T11:28:38.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.837+0000 7fcd51000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcd4c06f340 0x7fcd4c0b2b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:38.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.837+0000 7fcd51000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcd4c13e530 0x7fcd4c0b1020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:38.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.837+0000 7fcd51000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd4c141310 0x7fcd4c1e7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:38.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.837+0000 7fcd51000640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fcd4c0cf600 con 0x7fcd4c13e530 2024-04-20T11:28:38.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.837+0000 7fcd51000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcd4c072ce0 con 0x7fcd4c141310 2024-04-20T11:28:38.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.837+0000 7fcd51000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fcd4c0cf8e0 con 0x7fcd4c06f340 2024-04-20T11:28:38.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.838+0000 7fcd43e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcd4c13e530 0x7fcd4c0b1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:38.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.838+0000 7fcd43e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcd4c13e530 0x7fcd4c0b1020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:59914/0 (socket says 172.21.15.152:59914) 2024-04-20T11:28:38.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.838+0000 7fcd43e00640 1 -- 172.21.15.152:0/3524909305 learned_addr learned my addr 172.21.15.152:0/3524909305 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:38.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.838+0000 7fcd4be00640 1 --2- 172.21.15.152:0/3524909305 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd4c141310 0x7fcd4c1e7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:38.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.838+0000 7fcd4b400640 1 --2- 172.21.15.152:0/3524909305 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcd4c06f340 0x7fcd4c0b2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:38.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.838+0000 7fcd43e00640 1 -- 172.21.15.152:0/3524909305 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcd4c06f340 msgr2=0x7fcd4c0b2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:38.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.838+0000 7fcd43e00640 1 --2- 172.21.15.152:0/3524909305 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcd4c06f340 0x7fcd4c0b2b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:38.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.838+0000 7fcd43e00640 1 -- 172.21.15.152:0/3524909305 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd4c141310 msgr2=0x7fcd4c1e7ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:38.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.838+0000 7fcd43e00640 1 --2- 172.21.15.152:0/3524909305 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd4c141310 0x7fcd4c1e7ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:38.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.838+0000 7fcd43e00640 1 -- 172.21.15.152:0/3524909305 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd4c0b1560 con 0x7fcd4c13e530 2024-04-20T11:28:38.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.838+0000 7fcd43e00640 1 --2- 172.21.15.152:0/3524909305 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcd4c13e530 0x7fcd4c0b1020 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fcd3400c7f0 tx=0x7fcd3400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:38.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.839+0000 7fcd43400640 1 -- 172.21.15.152:0/3524909305 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd34015070 con 0x7fcd4c13e530 2024-04-20T11:28:38.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.839+0000 7fcd43400640 1 -- 172.21.15.152:0/3524909305 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd34007a30 con 0x7fcd4c13e530 2024-04-20T11:28:38.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.839+0000 7fcd51000640 1 -- 172.21.15.152:0/3524909305 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd4c0b1800 con 0x7fcd4c13e530 2024-04-20T11:28:38.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.839+0000 7fcd43400640 1 -- 172.21.15.152:0/3524909305 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd34007e00 con 0x7fcd4c13e530 2024-04-20T11:28:38.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.839+0000 7fcd51000640 1 -- 172.21.15.152:0/3524909305 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd4c0b1dc0 con 0x7fcd4c13e530 2024-04-20T11:28:38.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.840+0000 7fcd51000640 1 -- 172.21.15.152:0/3524909305 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd4c075e60 con 0x7fcd4c13e530 2024-04-20T11:28:38.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.841+0000 7fcd43400640 1 -- 172.21.15.152:0/3524909305 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7fcd34018070 con 0x7fcd4c13e530 2024-04-20T11:28:38.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.842+0000 7fcd43400640 1 --2- 172.21.15.152:0/3524909305 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fcd2806caa0 0x7fcd2806ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:38.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.842+0000 7fcd43400640 1 -- 172.21.15.152:0/3524909305 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fcd3408bfa0 con 0x7fcd4c13e530 2024-04-20T11:28:38.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.842+0000 7fcd4b400640 1 --2- 172.21.15.152:0/3524909305 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fcd2806caa0 0x7fcd2806ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:38.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.843+0000 7fcd4b400640 1 --2- 172.21.15.152:0/3524909305 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fcd2806caa0 0x7fcd2806ef60 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c036000 tx=0x7fcd3c03c040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:38.845 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:38.846+0000 7fcd43400640 1 -- 172.21.15.152:0/3524909305 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fcd3405a030 con 0x7fcd4c13e530 2024-04-20T11:28:39.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.048+0000 7fcd51000640 1 -- 172.21.15.152:0/3524909305 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcd4c13c540 con 0x7fcd4c13e530 2024-04-20T11:28:39.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.050+0000 7fcd43400640 1 -- 172.21.15.152:0/3524909305 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) ==== 74+0+16729 (secure 0 0 0) 0x7fcd34059930 con 0x7fcd4c13e530 2024-04-20T11:28:39.050 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:28:39.050 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":34,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:28:19.478546+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:28:15.704670+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-20T11:28:16.891101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:28:39.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.053+0000 7fcd51000640 1 -- 172.21.15.152:0/3524909305 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fcd2806caa0 msgr2=0x7fcd2806ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:39.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.053+0000 7fcd51000640 1 --2- 172.21.15.152:0/3524909305 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fcd2806caa0 0x7fcd2806ef60 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c036000 tx=0x7fcd3c03c040 comp rx=0 tx=0).stop 2024-04-20T11:28:39.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.053+0000 7fcd51000640 1 -- 172.21.15.152:0/3524909305 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcd4c13e530 msgr2=0x7fcd4c0b1020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:39.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.053+0000 7fcd51000640 1 --2- 172.21.15.152:0/3524909305 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcd4c13e530 0x7fcd4c0b1020 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fcd3400c7f0 tx=0x7fcd3400cbf0 comp rx=0 tx=0).stop 2024-04-20T11:28:39.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.053+0000 7fcd51000640 1 -- 172.21.15.152:0/3524909305 shutdown_connections 2024-04-20T11:28:39.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.053+0000 7fcd51000640 1 --2- 172.21.15.152:0/3524909305 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fcd2806caa0 0x7fcd2806ef60 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:39.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.053+0000 7fcd51000640 1 --2- 172.21.15.152:0/3524909305 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd4c141310 0x7fcd4c1e7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:39.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.053+0000 7fcd51000640 1 --2- 172.21.15.152:0/3524909305 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcd4c13e530 0x7fcd4c0b1020 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:39.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.053+0000 7fcd51000640 1 --2- 172.21.15.152:0/3524909305 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcd4c06f340 0x7fcd4c0b2b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:39.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.053+0000 7fcd51000640 1 -- 172.21.15.152:0/3524909305 >> 172.21.15.152:0/3524909305 conn(0x7fcd4c067ae0 msgr2=0x7fcd4c06dcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:39.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.053+0000 7fcd51000640 1 -- 172.21.15.152:0/3524909305 shutdown_connections 2024-04-20T11:28:39.054 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.053+0000 7fcd51000640 1 -- 172.21.15.152:0/3524909305 wait complete. 2024-04-20T11:28:39.072 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T11:26:47.246571+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T11:28:15.704670+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T11:28:16.891101+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T11:28:39.072 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T11:28:39.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.280+0000 7f4e47400640 1 Processor -- start 2024-04-20T11:28:39.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.280+0000 7f4e47400640 1 -- start start 2024-04-20T11:28:39.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.282+0000 7f4e47400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f4e4015c050 0x7f4e400b21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:39.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.282+0000 7f4e47400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4e400b4330 0x7f4e40072260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:39.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.282+0000 7f4e47400640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e4015b6a0 con 0x7f4e40071db0 2024-04-20T11:28:39.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.282+0000 7f4e47400640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f4e40155810 con 0x7f4e4015c050 2024-04-20T11:28:39.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.282+0000 7f4e47400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4e4006dc50 con 0x7f4e400b4330 2024-04-20T11:28:39.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.282+0000 7f4e44a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f4e4015c050 0x7f4e400b21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:39.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.282+0000 7f4e37e00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f4e40071db0 0x7f4e400b3c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:47118/0 (socket says 172.21.15.152:47118) 2024-04-20T11:28:39.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.282+0000 7f4e37e00640 1 -- 172.21.15.152:0/3831357998 learned_addr learned my addr 172.21.15.152:0/3831357998 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:28:39.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.282+0000 7f4e44a00640 1 --2- 172.21.15.152:0/3831357998 >> v2:172.21.15.152:3300/0 conn(0x7f4e4015c050 0x7f4e400b21e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:59924/0 (socket says 172.21.15.152:59924) 2024-04-20T11:28:39.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.283+0000 7f4e45400640 1 --2- 172.21.15.152:0/3831357998 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4e400b4330 0x7f4e40072260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:39.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.283+0000 7f4e44a00640 1 -- 172.21.15.152:0/3831357998 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4e400b4330 msgr2=0x7f4e40072260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:39.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.283+0000 7f4e44a00640 1 --2- 172.21.15.152:0/3831357998 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4e400b4330 0x7f4e40072260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:39.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.283+0000 7f4e44a00640 1 -- 172.21.15.152:0/3831357998 >> v1:172.21.15.152:6789/0 conn(0x7f4e40071db0 legacy=0x7f4e400b3c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:39.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.283+0000 7f4e44a00640 1 -- 172.21.15.152:0/3831357998 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e400b4800 con 0x7f4e4015c050 2024-04-20T11:28:39.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.284+0000 7f4e44a00640 1 --2- 172.21.15.152:0/3831357998 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4e4015c050 0x7f4e400b21e0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f4e300097a0 tx=0x7f4e3002b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b31264422ca8b42 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:39.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.284+0000 7f4e3fe00640 1 -- 172.21.15.152:0/3831357998 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e30031780 con 0x7f4e4015c050 2024-04-20T11:28:39.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.284+0000 7f4e3fe00640 1 -- 172.21.15.152:0/3831357998 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e30037070 con 0x7f4e4015c050 2024-04-20T11:28:39.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.284+0000 7f4e3fe00640 1 -- 172.21.15.152:0/3831357998 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e30031b40 con 0x7f4e4015c050 2024-04-20T11:28:39.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.285+0000 7f4e47400640 1 -- 172.21.15.152:0/3831357998 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4e4015c050 msgr2=0x7f4e400b21e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:39.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.285+0000 7f4e47400640 1 --2- 172.21.15.152:0/3831357998 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4e4015c050 0x7f4e400b21e0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f4e300097a0 tx=0x7f4e3002b820 comp rx=0 tx=0).stop 2024-04-20T11:28:39.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.285+0000 7f4e47400640 1 -- 172.21.15.152:0/3831357998 shutdown_connections 2024-04-20T11:28:39.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.285+0000 7f4e47400640 1 --2- 172.21.15.152:0/3831357998 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4e400b4330 0x7f4e40072260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:39.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.285+0000 7f4e47400640 1 --2- 172.21.15.152:0/3831357998 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4e4015c050 0x7f4e400b21e0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:39.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.285+0000 7f4e47400640 1 -- 172.21.15.152:0/3831357998 >> 172.21.15.152:0/3831357998 conn(0x7f4e4006b980 msgr2=0x7f4e4006bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:39.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.285+0000 7f4e47400640 1 -- 172.21.15.152:0/3831357998 shutdown_connections 2024-04-20T11:28:39.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.285+0000 7f4e47400640 1 -- 172.21.15.152:0/3831357998 wait complete. 2024-04-20T11:28:39.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.287+0000 7f4e47400640 1 Processor -- start 2024-04-20T11:28:39.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.287+0000 7f4e47400640 1 -- start start 2024-04-20T11:28:39.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.288+0000 7f4e47400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4e40071db0 0x7f4e4014f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:39.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.288+0000 7f4e47400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4e400b4330 0x7f4e40154300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:39.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.288+0000 7f4e47400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4e4015c050 0x7f4e40157a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:39.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.288+0000 7f4e47400640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f4e4006dea0 con 0x7f4e4015c050 2024-04-20T11:28:39.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.288+0000 7f4e47400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4e400b4dc0 con 0x7f4e40071db0 2024-04-20T11:28:39.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.288+0000 7f4e47400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4e400b2720 con 0x7f4e400b4330 2024-04-20T11:28:39.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.288+0000 7f4e45400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4e4015c050 0x7f4e40157a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:39.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.288+0000 7f4e45400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4e4015c050 0x7f4e40157a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:59940/0 (socket says 172.21.15.152:59940) 2024-04-20T11:28:39.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.288+0000 7f4e45400640 1 -- 172.21.15.152:0/4272795692 learned_addr learned my addr 172.21.15.152:0/4272795692 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:39.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.289+0000 7f4e44a00640 1 --2- 172.21.15.152:0/4272795692 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4e40071db0 0x7f4e4014f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:39.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.289+0000 7f4e45400640 1 -- 172.21.15.152:0/4272795692 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4e400b4330 msgr2=0x7f4e40154300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:39.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.289+0000 7f4e37e00640 1 --2- 172.21.15.152:0/4272795692 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4e400b4330 0x7f4e40154300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:39.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.289+0000 7f4e45400640 1 --2- 172.21.15.152:0/4272795692 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4e400b4330 0x7f4e40154300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:39.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.289+0000 7f4e45400640 1 -- 172.21.15.152:0/4272795692 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4e40071db0 msgr2=0x7f4e4014f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:39.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.289+0000 7f4e45400640 1 --2- 172.21.15.152:0/4272795692 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4e40071db0 0x7f4e4014f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:39.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.289+0000 7f4e45400640 1 -- 172.21.15.152:0/4272795692 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e401566f0 con 0x7f4e4015c050 2024-04-20T11:28:39.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.289+0000 7f4e44a00640 1 --2- 172.21.15.152:0/4272795692 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4e40071db0 0x7f4e4014f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:28:39.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.289+0000 7f4e45400640 1 --2- 172.21.15.152:0/4272795692 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4e4015c050 0x7f4e40157a60 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f4e38007e30 tx=0x7f4e3800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:39.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.289+0000 7f4e3d600640 1 -- 172.21.15.152:0/4272795692 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e38014620 con 0x7f4e4015c050 2024-04-20T11:28:39.291 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.290+0000 7f4e3d600640 1 -- 172.21.15.152:0/4272795692 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e38019070 con 0x7f4e4015c050 2024-04-20T11:28:39.291 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.290+0000 7f4e47400640 1 -- 172.21.15.152:0/4272795692 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e401569d0 con 0x7f4e4015c050 2024-04-20T11:28:39.291 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.290+0000 7f4e3d600640 1 -- 172.21.15.152:0/4272795692 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e38014a00 con 0x7f4e4015c050 2024-04-20T11:28:39.291 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.290+0000 7f4e47400640 1 -- 172.21.15.152:0/4272795692 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e401518b0 con 0x7f4e4015c050 2024-04-20T11:28:39.291 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.291+0000 7f4e47400640 1 -- 172.21.15.152:0/4272795692 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e400689f0 con 0x7f4e4015c050 2024-04-20T11:28:39.291 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.292+0000 7f4e3d600640 1 -- 172.21.15.152:0/4272795692 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f4e38014ba0 con 0x7f4e4015c050 2024-04-20T11:28:39.292 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.293+0000 7f4e3d600640 1 --2- 172.21.15.152:0/4272795692 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f4e1c06caa0 0x7f4e1c06ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:39.292 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.293+0000 7f4e3d600640 1 -- 172.21.15.152:0/4272795692 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f4e3808c7c0 con 0x7f4e4015c050 2024-04-20T11:28:39.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.293+0000 7f4e44a00640 1 --2- 172.21.15.152:0/4272795692 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f4e1c06caa0 0x7f4e1c06ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:39.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.294+0000 7f4e44a00640 1 --2- 172.21.15.152:0/4272795692 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f4e1c06caa0 0x7f4e1c06ef60 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4e30036000 tx=0x7f4e30008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:39.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.298+0000 7f4e3d600640 1 -- 172.21.15.152:0/4272795692 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f4e3805a960 con 0x7f4e4015c050 2024-04-20T11:28:39.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.501+0000 7f4e47400640 1 -- 172.21.15.152:0/4272795692 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4e400b1f50 con 0x7f4e4015c050 2024-04-20T11:28:39.502 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.502+0000 7f4e3d600640 1 -- 172.21.15.152:0/4272795692 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v34) ==== 93+0+10 (secure 0 0 0) 0x7f4e3805a260 con 0x7f4e4015c050 2024-04-20T11:28:39.502 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 1 2024-04-20T11:28:39.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.505+0000 7f4e47400640 1 -- 172.21.15.152:0/4272795692 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f4e1c06caa0 msgr2=0x7f4e1c06ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:39.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.505+0000 7f4e47400640 1 --2- 172.21.15.152:0/4272795692 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f4e1c06caa0 0x7f4e1c06ef60 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4e30036000 tx=0x7f4e30008030 comp rx=0 tx=0).stop 2024-04-20T11:28:39.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.505+0000 7f4e47400640 1 -- 172.21.15.152:0/4272795692 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4e4015c050 msgr2=0x7f4e40157a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:39.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.505+0000 7f4e47400640 1 --2- 172.21.15.152:0/4272795692 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4e4015c050 0x7f4e40157a60 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f4e38007e30 tx=0x7f4e3800e4b0 comp rx=0 tx=0).stop 2024-04-20T11:28:39.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.505+0000 7f4e47400640 1 -- 172.21.15.152:0/4272795692 shutdown_connections 2024-04-20T11:28:39.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.505+0000 7f4e47400640 1 --2- 172.21.15.152:0/4272795692 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f4e1c06caa0 0x7f4e1c06ef60 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:39.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.505+0000 7f4e47400640 1 --2- 172.21.15.152:0/4272795692 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4e4015c050 0x7f4e40157a60 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:39.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.505+0000 7f4e47400640 1 --2- 172.21.15.152:0/4272795692 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4e400b4330 0x7f4e40154300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:39.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.505+0000 7f4e47400640 1 --2- 172.21.15.152:0/4272795692 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4e40071db0 0x7f4e4014f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:39.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.505+0000 7f4e47400640 1 -- 172.21.15.152:0/4272795692 >> 172.21.15.152:0/4272795692 conn(0x7f4e4006b980 msgr2=0x7f4e40154be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:39.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.506+0000 7f4e47400640 1 -- 172.21.15.152:0/4272795692 shutdown_connections 2024-04-20T11:28:39.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.506+0000 7f4e47400640 1 -- 172.21.15.152:0/4272795692 wait complete. 2024-04-20T11:28:39.524 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T11:28:39.722 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.723+0000 7f1121200640 1 Processor -- start 2024-04-20T11:28:39.723 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.723+0000 7f1121200640 1 -- start start 2024-04-20T11:28:39.724 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.725+0000 7f1121200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f111c071cd0 0x7f111c0720b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:39.725 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.725+0000 7f1121200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f111c160220 0x7f111c160680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:39.725 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.725+0000 7f1121200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f111c0725f0 con 0x7f111c06eb30 2024-04-20T11:28:39.725 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.725+0000 7f1121200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f111c13f8e0 con 0x7f111c160220 2024-04-20T11:28:39.725 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.725+0000 7f1121200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f111c160f90 con 0x7f111c071cd0 2024-04-20T11:28:39.725 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.725+0000 7f111aa00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f111c160220 0x7f111c160680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:39.725 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.725+0000 7f111aa00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f111c160220 0x7f111c160680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:59956/0 (socket says 172.21.15.152:59956) 2024-04-20T11:28:39.725 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.725+0000 7f111aa00640 1 -- 172.21.15.152:0/3638793326 learned_addr learned my addr 172.21.15.152:0/3638793326 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:39.725 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.725+0000 7f111b400640 1 --2- 172.21.15.152:0/3638793326 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f111c071cd0 0x7f111c0720b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:39.726 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.726+0000 7f111aa00640 1 -- 172.21.15.152:0/3638793326 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f111c071cd0 msgr2=0x7f111c0720b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:39.726 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.726+0000 7f111aa00640 1 --2- 172.21.15.152:0/3638793326 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f111c071cd0 0x7f111c0720b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:39.726 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.726+0000 7f111aa00640 1 -- 172.21.15.152:0/3638793326 >> v1:172.21.15.175:6789/0 conn(0x7f111c06eb30 legacy=0x7f111c06efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:39.726 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.726+0000 7f111aa00640 1 -- 172.21.15.152:0/3638793326 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f111c06dd90 con 0x7f111c160220 2024-04-20T11:28:39.726 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.726+0000 7f111b400640 1 --2- 172.21.15.152:0/3638793326 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f111c071cd0 0x7f111c0720b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:28:39.727 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.727+0000 7f111aa00640 1 --2- 172.21.15.152:0/3638793326 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f111c160220 0x7f111c160680 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f1108009870 tx=0x7f110802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=feff75476edf54de server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:39.727 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.727+0000 7f111a000640 1 -- 172.21.15.152:0/3638793326 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11080317c0 con 0x7f111c160220 2024-04-20T11:28:39.727 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.727+0000 7f111a000640 1 -- 172.21.15.152:0/3638793326 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1108037070 con 0x7f111c160220 2024-04-20T11:28:39.727 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.727+0000 7f111a000640 1 -- 172.21.15.152:0/3638793326 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1108031b80 con 0x7f111c160220 2024-04-20T11:28:39.728 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.728+0000 7f1121200640 1 -- 172.21.15.152:0/3638793326 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f111c160220 msgr2=0x7f111c160680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:39.728 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.728+0000 7f1121200640 1 --2- 172.21.15.152:0/3638793326 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f111c160220 0x7f111c160680 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f1108009870 tx=0x7f110802b860 comp rx=0 tx=0).stop 2024-04-20T11:28:39.728 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.728+0000 7f1121200640 1 -- 172.21.15.152:0/3638793326 shutdown_connections 2024-04-20T11:28:39.728 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.728+0000 7f1121200640 1 --2- 172.21.15.152:0/3638793326 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f111c160220 0x7f111c160680 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:39.728 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.728+0000 7f1121200640 1 --2- 172.21.15.152:0/3638793326 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f111c071cd0 0x7f111c0720b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:39.728 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.728+0000 7f1121200640 1 -- 172.21.15.152:0/3638793326 >> 172.21.15.152:0/3638793326 conn(0x7f111c067ae0 msgr2=0x7f111c06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:39.728 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.728+0000 7f1121200640 1 -- 172.21.15.152:0/3638793326 shutdown_connections 2024-04-20T11:28:39.728 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.728+0000 7f1121200640 1 -- 172.21.15.152:0/3638793326 wait complete. 2024-04-20T11:28:39.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.730+0000 7f1121200640 1 Processor -- start 2024-04-20T11:28:39.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.730+0000 7f1121200640 1 -- start start 2024-04-20T11:28:39.732 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.731+0000 7f1121200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f111c06eb30 0x7f111c1e6b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:39.732 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.731+0000 7f1121200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f111c071cd0 0x7f111c13a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:39.732 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.731+0000 7f1121200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f111c160220 0x7f111c138e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:39.732 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.731+0000 7f1121200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f111c06e630 con 0x7f111c160220 2024-04-20T11:28:39.732 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.731+0000 7f1121200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f111c06e350 con 0x7f111c071cd0 2024-04-20T11:28:39.732 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.731+0000 7f1121200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f111c13fb90 con 0x7f111c06eb30 2024-04-20T11:28:39.732 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.731+0000 7f111be00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f111c160220 0x7f111c138e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:39.732 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.731+0000 7f111be00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f111c160220 0x7f111c138e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:59966/0 (socket says 172.21.15.152:59966) 2024-04-20T11:28:39.732 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.731+0000 7f111be00640 1 -- 172.21.15.152:0/486513734 learned_addr learned my addr 172.21.15.152:0/486513734 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:39.733 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.732+0000 7f111b400640 1 --2- 172.21.15.152:0/486513734 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f111c06eb30 0x7f111c1e6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:39.733 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.732+0000 7f111be00640 1 -- 172.21.15.152:0/486513734 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f111c06eb30 msgr2=0x7f111c1e6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:39.733 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.732+0000 7f111aa00640 1 --2- 172.21.15.152:0/486513734 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f111c071cd0 0x7f111c13a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:39.733 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.732+0000 7f111be00640 1 --2- 172.21.15.152:0/486513734 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f111c06eb30 0x7f111c1e6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:39.733 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.732+0000 7f111be00640 1 -- 172.21.15.152:0/486513734 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f111c071cd0 msgr2=0x7f111c13a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:39.733 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.732+0000 7f111be00640 1 --2- 172.21.15.152:0/486513734 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f111c071cd0 0x7f111c13a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:39.733 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.732+0000 7f111be00640 1 -- 172.21.15.152:0/486513734 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f111c134730 con 0x7f111c160220 2024-04-20T11:28:39.733 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.732+0000 7f111b400640 1 --2- 172.21.15.152:0/486513734 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f111c06eb30 0x7f111c1e6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:28:39.733 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.732+0000 7f111be00640 1 --2- 172.21.15.152:0/486513734 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f111c160220 0x7f111c138e70 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f111400c780 tx=0x7f111400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:39.733 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.732+0000 7f1113400640 1 -- 172.21.15.152:0/486513734 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1114004b60 con 0x7f111c160220 2024-04-20T11:28:39.733 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.732+0000 7f1113400640 1 -- 172.21.15.152:0/486513734 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1114004d00 con 0x7f111c160220 2024-04-20T11:28:39.733 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.732+0000 7f1113400640 1 -- 172.21.15.152:0/486513734 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f111400aa90 con 0x7f111c160220 2024-04-20T11:28:39.733 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.732+0000 7f1121200640 1 -- 172.21.15.152:0/486513734 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f111c1e73f0 con 0x7f111c160220 2024-04-20T11:28:39.733 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.733+0000 7f1121200640 1 -- 172.21.15.152:0/486513734 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f111c1e79b0 con 0x7f111c160220 2024-04-20T11:28:39.734 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.734+0000 7f1121200640 1 -- 172.21.15.152:0/486513734 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f111c075e60 con 0x7f111c160220 2024-04-20T11:28:39.734 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.735+0000 7f1113400640 1 -- 172.21.15.152:0/486513734 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f111400ac30 con 0x7f111c160220 2024-04-20T11:28:39.735 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.736+0000 7f1113400640 1 --2- 172.21.15.152:0/486513734 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f10fc06caa0 0x7f10fc06ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:39.735 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.736+0000 7f1113400640 1 -- 172.21.15.152:0/486513734 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f111408c1f0 con 0x7f111c160220 2024-04-20T11:28:39.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.736+0000 7f111b400640 1 --2- 172.21.15.152:0/486513734 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f10fc06caa0 0x7f10fc06ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:39.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.737+0000 7f111b400640 1 --2- 172.21.15.152:0/486513734 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f10fc06caa0 0x7f10fc06ef60 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f110c006fd0 tx=0x7f110c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:39.739 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.740+0000 7f1113400640 1 -- 172.21.15.152:0/486513734 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f1114020070 con 0x7f111c160220 2024-04-20T11:28:39.938 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.938+0000 7f1121200640 1 -- 172.21.15.152:0/486513734 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f111c1608d0 con 0x7f111c160220 2024-04-20T11:28:39.939 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.939+0000 7f1113400640 1 -- 172.21.15.152:0/486513734 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v34) ==== 104+0+11 (secure 0 0 0) 0x7f111405a300 con 0x7f111c160220 2024-04-20T11:28:39.939 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 32 2024-04-20T11:28:39.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.942+0000 7f1121200640 1 -- 172.21.15.152:0/486513734 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f10fc06caa0 msgr2=0x7f10fc06ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:39.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.942+0000 7f1121200640 1 --2- 172.21.15.152:0/486513734 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f10fc06caa0 0x7f10fc06ef60 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f110c006fd0 tx=0x7f110c008040 comp rx=0 tx=0).stop 2024-04-20T11:28:39.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.942+0000 7f1121200640 1 -- 172.21.15.152:0/486513734 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f111c160220 msgr2=0x7f111c138e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:39.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.942+0000 7f1121200640 1 --2- 172.21.15.152:0/486513734 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f111c160220 0x7f111c138e70 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f111400c780 tx=0x7f111400cc40 comp rx=0 tx=0).stop 2024-04-20T11:28:39.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.942+0000 7f1121200640 1 -- 172.21.15.152:0/486513734 shutdown_connections 2024-04-20T11:28:39.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.942+0000 7f1121200640 1 --2- 172.21.15.152:0/486513734 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f10fc06caa0 0x7f10fc06ef60 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:39.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.942+0000 7f1121200640 1 --2- 172.21.15.152:0/486513734 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f111c160220 0x7f111c138e70 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:39.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.942+0000 7f1121200640 1 --2- 172.21.15.152:0/486513734 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f111c071cd0 0x7f111c13a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:39.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.942+0000 7f1121200640 1 --2- 172.21.15.152:0/486513734 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f111c06eb30 0x7f111c1e6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:39.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.942+0000 7f1121200640 1 -- 172.21.15.152:0/486513734 >> 172.21.15.152:0/486513734 conn(0x7f111c067ae0 msgr2=0x7f111c15cb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:39.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.942+0000 7f1121200640 1 -- 172.21.15.152:0/486513734 shutdown_connections 2024-04-20T11:28:39.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:39.942+0000 7f1121200640 1 -- 172.21.15.152:0/486513734 wait complete. 2024-04-20T11:28:39.959 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T11:28:40.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.162+0000 7f6b2b000640 1 Processor -- start 2024-04-20T11:28:40.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.163+0000 7f6b2b000640 1 -- start start 2024-04-20T11:28:40.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.164+0000 7f6b2b000640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f6b2415e9e0 0x7f6b2406d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:40.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.164+0000 7f6b2b000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b240d46a0 0x7f6b240d4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:40.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.164+0000 7f6b2b000640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b240d19a0 con 0x7f6b240d0d90 2024-04-20T11:28:40.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.164+0000 7f6b2b000640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f6b240d5040 con 0x7f6b2415e9e0 2024-04-20T11:28:40.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.164+0000 7f6b2b000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6b240d1c80 con 0x7f6b240d46a0 2024-04-20T11:28:40.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.165+0000 7f6b23e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f6b2415e9e0 0x7f6b2406d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:40.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.165+0000 7f6b23400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b240d46a0 0x7f6b240d4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:40.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.165+0000 7f6b23e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f6b2415e9e0 0x7f6b2406d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.152:59852/0 (socket says 172.21.15.152:59852) 2024-04-20T11:28:40.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.165+0000 7f6b23e00640 1 -- 172.21.15.152:0/3497564831 learned_addr learned my addr 172.21.15.152:0/3497564831 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:40.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.166+0000 7f6b23e00640 1 -- 172.21.15.152:0/3497564831 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b240d46a0 msgr2=0x7f6b240d4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:40.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.166+0000 7f6b23e00640 1 --2- 172.21.15.152:0/3497564831 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b240d46a0 0x7f6b240d4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:40.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.166+0000 7f6b23e00640 1 -- 172.21.15.152:0/3497564831 >> v1:172.21.15.175:6789/0 conn(0x7f6b240d0d90 legacy=0x7f6b240cfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:40.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.166+0000 7f6b23e00640 1 -- 172.21.15.152:0/3497564831 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b240d1f60 con 0x7f6b2415e9e0 2024-04-20T11:28:40.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.167+0000 7f6b23e00640 1 --2- 172.21.15.152:0/3497564831 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b2415e9e0 0x7f6b2406d320 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6b14009870 tx=0x7f6b1402b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54cde59f60065c31 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:40.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.168+0000 7f6b22a00640 1 -- 172.21.15.152:0/3497564831 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b14034070 con 0x7f6b2415e9e0 2024-04-20T11:28:40.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.168+0000 7f6b22a00640 1 -- 172.21.15.152:0/3497564831 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b14030840 con 0x7f6b2415e9e0 2024-04-20T11:28:40.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.168+0000 7f6b22a00640 1 -- 172.21.15.152:0/3497564831 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b14030c10 con 0x7f6b2415e9e0 2024-04-20T11:28:40.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.168+0000 7f6b2b000640 1 -- 172.21.15.152:0/3497564831 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b2415e9e0 msgr2=0x7f6b2406d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:40.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.168+0000 7f6b2b000640 1 --2- 172.21.15.152:0/3497564831 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b2415e9e0 0x7f6b2406d320 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6b14009870 tx=0x7f6b1402b980 comp rx=0 tx=0).stop 2024-04-20T11:28:40.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.169+0000 7f6b2b000640 1 -- 172.21.15.152:0/3497564831 shutdown_connections 2024-04-20T11:28:40.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.169+0000 7f6b2b000640 1 --2- 172.21.15.152:0/3497564831 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b240d46a0 0x7f6b240d4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:40.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.169+0000 7f6b2b000640 1 --2- 172.21.15.152:0/3497564831 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b2415e9e0 0x7f6b2406d320 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:40.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.169+0000 7f6b2b000640 1 -- 172.21.15.152:0/3497564831 >> 172.21.15.152:0/3497564831 conn(0x7f6b2409dab0 msgr2=0x7f6b2408bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:40.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.169+0000 7f6b2b000640 1 -- 172.21.15.152:0/3497564831 shutdown_connections 2024-04-20T11:28:40.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.169+0000 7f6b2b000640 1 -- 172.21.15.152:0/3497564831 wait complete. 2024-04-20T11:28:40.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.171+0000 7f6b2b000640 1 Processor -- start 2024-04-20T11:28:40.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.171+0000 7f6b2b000640 1 -- start start 2024-04-20T11:28:40.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.172+0000 7f6b2b000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b240d0d90 0x7f6b241618e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:40.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.172+0000 7f6b2b000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b240d46a0 0x7f6b2415edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:40.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.172+0000 7f6b2b000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6b2415e9e0 0x7f6b24155cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:40.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.172+0000 7f6b2b000640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f6b2406d860 con 0x7f6b2415e9e0 2024-04-20T11:28:40.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.173+0000 7f6b2b000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6b240cf880 con 0x7f6b240d46a0 2024-04-20T11:28:40.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.173+0000 7f6b2b000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6b2406d9e0 con 0x7f6b240d0d90 2024-04-20T11:28:40.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.173+0000 7f6b29000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6b2415e9e0 0x7f6b24155cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:40.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.173+0000 7f6b29000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6b2415e9e0 0x7f6b24155cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:59980/0 (socket says 172.21.15.152:59980) 2024-04-20T11:28:40.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.173+0000 7f6b29000640 1 -- 172.21.15.152:0/3681030494 learned_addr learned my addr 172.21.15.152:0/3681030494 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:40.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.173+0000 7f6b23e00640 1 --2- 172.21.15.152:0/3681030494 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b240d0d90 0x7f6b241618e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:40.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.173+0000 7f6b23400640 1 --2- 172.21.15.152:0/3681030494 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b240d46a0 0x7f6b2415edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:40.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.173+0000 7f6b29000640 1 -- 172.21.15.152:0/3681030494 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b240d0d90 msgr2=0x7f6b241618e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:40.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.173+0000 7f6b29000640 1 --2- 172.21.15.152:0/3681030494 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b240d0d90 0x7f6b241618e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:40.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.173+0000 7f6b29000640 1 -- 172.21.15.152:0/3681030494 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b240d46a0 msgr2=0x7f6b2415edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:40.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.173+0000 7f6b29000640 1 --2- 172.21.15.152:0/3681030494 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b240d46a0 0x7f6b2415edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:40.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.173+0000 7f6b29000640 1 -- 172.21.15.152:0/3681030494 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b240b7400 con 0x7f6b2415e9e0 2024-04-20T11:28:40.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.173+0000 7f6b23400640 1 --2- 172.21.15.152:0/3681030494 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b240d46a0 0x7f6b2415edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:28:40.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.174+0000 7f6b29000640 1 --2- 172.21.15.152:0/3681030494 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6b2415e9e0 0x7f6b24155cd0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c00c7a0 tx=0x7f6b1c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:40.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.174+0000 7f6b1be00640 1 -- 172.21.15.152:0/3681030494 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b1c003b70 con 0x7f6b2415e9e0 2024-04-20T11:28:40.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.174+0000 7f6b1be00640 1 -- 172.21.15.152:0/3681030494 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b1c003d30 con 0x7f6b2415e9e0 2024-04-20T11:28:40.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.174+0000 7f6b1be00640 1 -- 172.21.15.152:0/3681030494 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b1c00aa90 con 0x7f6b2415e9e0 2024-04-20T11:28:40.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.174+0000 7f6b2b000640 1 -- 172.21.15.152:0/3681030494 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b240b76e0 con 0x7f6b2415e9e0 2024-04-20T11:28:40.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.174+0000 7f6b2b000640 1 -- 172.21.15.152:0/3681030494 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b240b7ca0 con 0x7f6b2415e9e0 2024-04-20T11:28:40.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.176+0000 7f6b2b000640 1 -- 172.21.15.152:0/3681030494 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b240689f0 con 0x7f6b2415e9e0 2024-04-20T11:28:40.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.176+0000 7f6b1be00640 1 -- 172.21.15.152:0/3681030494 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f6b1c00ac30 con 0x7f6b2415e9e0 2024-04-20T11:28:40.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.178+0000 7f6b1be00640 1 --2- 172.21.15.152:0/3681030494 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f6b0406caa0 0x7f6b0406ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:40.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.178+0000 7f6b1be00640 1 -- 172.21.15.152:0/3681030494 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f6b1c08be00 con 0x7f6b2415e9e0 2024-04-20T11:28:40.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.178+0000 7f6b23e00640 1 --2- 172.21.15.152:0/3681030494 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f6b0406caa0 0x7f6b0406ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:40.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.179+0000 7f6b23e00640 1 --2- 172.21.15.152:0/3681030494 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f6b0406caa0 0x7f6b0406ef60 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6b14036000 tx=0x7f6b1403b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:40.182 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.182+0000 7f6b1be00640 1 -- 172.21.15.152:0/3681030494 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f6b1c059fb0 con 0x7f6b2415e9e0 2024-04-20T11:28:40.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.387+0000 7f6b2b000640 1 -- 172.21.15.152:0/3681030494 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f6b240d24d0 con 0x7f6b2415e9e0 2024-04-20T11:28:40.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.387+0000 7f6b1be00640 1 -- 172.21.15.152:0/3681030494 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v34) ==== 100+0+11 (secure 0 0 0) 0x7f6b1c0598b0 con 0x7f6b2415e9e0 2024-04-20T11:28:40.387 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 64 2024-04-20T11:28:40.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.390+0000 7f6b2b000640 1 -- 172.21.15.152:0/3681030494 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f6b0406caa0 msgr2=0x7f6b0406ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:40.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.390+0000 7f6b2b000640 1 --2- 172.21.15.152:0/3681030494 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f6b0406caa0 0x7f6b0406ef60 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6b14036000 tx=0x7f6b1403b040 comp rx=0 tx=0).stop 2024-04-20T11:28:40.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.390+0000 7f6b2b000640 1 -- 172.21.15.152:0/3681030494 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6b2415e9e0 msgr2=0x7f6b24155cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:40.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.390+0000 7f6b2b000640 1 --2- 172.21.15.152:0/3681030494 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6b2415e9e0 0x7f6b24155cd0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c00c7a0 tx=0x7f6b1c00cc60 comp rx=0 tx=0).stop 2024-04-20T11:28:40.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.390+0000 7f6b2b000640 1 -- 172.21.15.152:0/3681030494 shutdown_connections 2024-04-20T11:28:40.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.390+0000 7f6b2b000640 1 --2- 172.21.15.152:0/3681030494 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f6b0406caa0 0x7f6b0406ef60 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:40.391 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.390+0000 7f6b2b000640 1 --2- 172.21.15.152:0/3681030494 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6b2415e9e0 0x7f6b24155cd0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:40.391 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.391+0000 7f6b2b000640 1 --2- 172.21.15.152:0/3681030494 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b240d46a0 0x7f6b2415edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:40.391 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.391+0000 7f6b2b000640 1 --2- 172.21.15.152:0/3681030494 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b240d0d90 0x7f6b241618e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:40.391 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.391+0000 7f6b2b000640 1 -- 172.21.15.152:0/3681030494 >> 172.21.15.152:0/3681030494 conn(0x7f6b2409dab0 msgr2=0x7f6b240cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:40.391 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.391+0000 7f6b2b000640 1 -- 172.21.15.152:0/3681030494 shutdown_connections 2024-04-20T11:28:40.391 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.391+0000 7f6b2b000640 1 -- 172.21.15.152:0/3681030494 wait complete. 2024-04-20T11:28:40.410 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-20T11:28:40.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.601+0000 7f8909400640 1 Processor -- start 2024-04-20T11:28:40.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.601+0000 7f8909400640 1 -- start start 2024-04-20T11:28:40.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.603+0000 7f8909400640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f8904145f00 0x7f89040cd0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:40.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.603+0000 7f8909400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f89040cd610 0x7f890406e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:40.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.603+0000 7f8909400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f890406e900 0x7f89041601c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:40.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.603+0000 7f8909400640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f89040cda80 con 0x7f89040cd610 2024-04-20T11:28:40.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.603+0000 7f8909400640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f89040cf9f0 con 0x7f8904145f00 2024-04-20T11:28:40.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.603+0000 7f8909400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f890406f110 con 0x7f890406e900 2024-04-20T11:28:40.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.603+0000 7f8901e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f89040cd610 0x7f890406e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:40.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.603+0000 7f8901e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f89040cd610 0x7f890406e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:59990/0 (socket says 172.21.15.152:59990) 2024-04-20T11:28:40.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.603+0000 7f8901e00640 1 -- 172.21.15.152:0/1581631569 learned_addr learned my addr 172.21.15.152:0/1581631569 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:40.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.604+0000 7f8902800640 1 --2- 172.21.15.152:0/1581631569 >> v2:172.21.15.175:3300/0 conn(0x7f8904145f00 0x7f89040cd0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:40.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.604+0000 7f8903200640 1 --2- 172.21.15.152:0/1581631569 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f890406e900 0x7f89041601c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:40.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.604+0000 7f8901e00640 1 -- 172.21.15.152:0/1581631569 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f890406e900 msgr2=0x7f89041601c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:40.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.604+0000 7f8901e00640 1 --2- 172.21.15.152:0/1581631569 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f890406e900 0x7f89041601c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:40.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.604+0000 7f8901e00640 1 -- 172.21.15.152:0/1581631569 >> v2:172.21.15.175:3300/0 conn(0x7f8904145f00 msgr2=0x7f89040cd0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:40.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.604+0000 7f8901e00640 1 --2- 172.21.15.152:0/1581631569 >> v2:172.21.15.175:3300/0 conn(0x7f8904145f00 0x7f89040cd0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:40.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.604+0000 7f8901e00640 1 -- 172.21.15.152:0/1581631569 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f890406f3f0 con 0x7f89040cd610 2024-04-20T11:28:40.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.605+0000 7f8901e00640 1 --2- 172.21.15.152:0/1581631569 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f89040cd610 0x7f890406e3c0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f890406c840 tx=0x7f88f402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5609c11fdca6921a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:40.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.605+0000 7f8901400640 1 -- 172.21.15.152:0/1581631569 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88f4035020 con 0x7f89040cd610 2024-04-20T11:28:40.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.605+0000 7f8901400640 1 -- 172.21.15.152:0/1581631569 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88f4031870 con 0x7f89040cd610 2024-04-20T11:28:40.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.605+0000 7f8901400640 1 -- 172.21.15.152:0/1581631569 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88f4031c40 con 0x7f89040cd610 2024-04-20T11:28:40.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.606+0000 7f8909400640 1 -- 172.21.15.152:0/1581631569 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f89040cd610 msgr2=0x7f890406e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:40.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.606+0000 7f8909400640 1 --2- 172.21.15.152:0/1581631569 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f89040cd610 0x7f890406e3c0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f890406c840 tx=0x7f88f402b910 comp rx=0 tx=0).stop 2024-04-20T11:28:40.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.606+0000 7f8909400640 1 -- 172.21.15.152:0/1581631569 shutdown_connections 2024-04-20T11:28:40.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.606+0000 7f8909400640 1 --2- 172.21.15.152:0/1581631569 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f890406e900 0x7f89041601c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:40.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.606+0000 7f8909400640 1 --2- 172.21.15.152:0/1581631569 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f89040cd610 0x7f890406e3c0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:40.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.606+0000 7f8909400640 1 --2- 172.21.15.152:0/1581631569 >> v2:172.21.15.175:3300/0 conn(0x7f8904145f00 0x7f89040cd0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:40.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.606+0000 7f8909400640 1 -- 172.21.15.152:0/1581631569 >> 172.21.15.152:0/1581631569 conn(0x7f8904067cf0 msgr2=0x7f890406b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:40.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.606+0000 7f8909400640 1 -- 172.21.15.152:0/1581631569 shutdown_connections 2024-04-20T11:28:40.607 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.606+0000 7f8909400640 1 -- 172.21.15.152:0/1581631569 wait complete. 2024-04-20T11:28:40.607 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.608+0000 7f8909400640 1 Processor -- start 2024-04-20T11:28:40.607 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.608+0000 7f8909400640 1 -- start start 2024-04-20T11:28:40.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.609+0000 7f8909400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f890406e900 0x7f8904136dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:40.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.609+0000 7f8909400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f89040cd610 0x7f890414d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:40.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.609+0000 7f8909400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8904145f00 0x7f8904142bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:40.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.609+0000 7f8909400640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f89041462e0 con 0x7f890406e900 2024-04-20T11:28:40.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.609+0000 7f8909400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f89040cf680 con 0x7f89040cd610 2024-04-20T11:28:40.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.609+0000 7f8909400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f890406ed90 con 0x7f8904145f00 2024-04-20T11:28:40.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.609+0000 7f8902800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f890406e900 0x7f8904136dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:40.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.609+0000 7f8902800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f890406e900 0x7f8904136dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:60000/0 (socket says 172.21.15.152:60000) 2024-04-20T11:28:40.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.609+0000 7f8902800640 1 -- 172.21.15.152:0/1668316242 learned_addr learned my addr 172.21.15.152:0/1668316242 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:40.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.610+0000 7f8902800640 1 -- 172.21.15.152:0/1668316242 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8904145f00 msgr2=0x7f8904142bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:40.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.610+0000 7f8902800640 1 --2- 172.21.15.152:0/1668316242 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8904145f00 0x7f8904142bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:40.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.610+0000 7f8901e00640 1 --2- 172.21.15.152:0/1668316242 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f89040cd610 0x7f890414d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:40.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.610+0000 7f8902800640 1 -- 172.21.15.152:0/1668316242 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f89040cd610 msgr2=0x7f890414d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:40.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.610+0000 7f8902800640 1 --2- 172.21.15.152:0/1668316242 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f89040cd610 0x7f890414d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:40.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.610+0000 7f8902800640 1 -- 172.21.15.152:0/1668316242 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f890414c000 con 0x7f890406e900 2024-04-20T11:28:40.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.610+0000 7f8902800640 1 --2- 172.21.15.152:0/1668316242 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f890406e900 0x7f8904136dc0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f88ec00d5f0 tx=0x7f88ec00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:40.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.610+0000 7f88faa00640 1 -- 172.21.15.152:0/1668316242 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88ec0125f0 con 0x7f890406e900 2024-04-20T11:28:40.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.610+0000 7f88faa00640 1 -- 172.21.15.152:0/1668316242 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88ec019070 con 0x7f890406e900 2024-04-20T11:28:40.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.610+0000 7f88faa00640 1 -- 172.21.15.152:0/1668316242 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88ec0129d0 con 0x7f890406e900 2024-04-20T11:28:40.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.610+0000 7f8909400640 1 -- 172.21.15.152:0/1668316242 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f890414c2e0 con 0x7f890406e900 2024-04-20T11:28:40.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.611+0000 7f8909400640 1 -- 172.21.15.152:0/1668316242 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f890414c8a0 con 0x7f890406e900 2024-04-20T11:28:40.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.612+0000 7f8909400640 1 -- 172.21.15.152:0/1668316242 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89040689f0 con 0x7f890406e900 2024-04-20T11:28:40.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.613+0000 7f88faa00640 1 -- 172.21.15.152:0/1668316242 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f88ec012b70 con 0x7f890406e900 2024-04-20T11:28:40.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.614+0000 7f88faa00640 1 --2- 172.21.15.152:0/1668316242 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f88e006caa0 0x7f88e006ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:40.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.614+0000 7f88faa00640 1 -- 172.21.15.152:0/1668316242 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f88ec08c1f0 con 0x7f890406e900 2024-04-20T11:28:40.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.614+0000 7f8901e00640 1 --2- 172.21.15.152:0/1668316242 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f88e006caa0 0x7f88e006ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:40.614 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.615+0000 7f8901e00640 1 --2- 172.21.15.152:0/1668316242 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f88e006caa0 0x7f88e006ef60 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f88f4036000 tx=0x7f88f4008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:40.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.618+0000 7f88faa00640 1 -- 172.21.15.152:0/1668316242 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f88ec05a390 con 0x7f890406e900 2024-04-20T11:28:40.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.834+0000 7f8909400640 1 -- 172.21.15.152:0/1668316242 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f890415cf10 con 0x7f890406e900 2024-04-20T11:28:40.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.834+0000 7f88faa00640 1 -- 172.21.15.152:0/1668316242 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v34) ==== 80+0+0 (secure 0 0 0) 0x7f88ec059c90 con 0x7f890406e900 2024-04-20T11:28:40.834 INFO:teuthology.orchestra.run.smithi152.stderr:listed 0 entries 2024-04-20T11:28:40.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.837+0000 7f8909400640 1 -- 172.21.15.152:0/1668316242 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f88e006caa0 msgr2=0x7f88e006ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:40.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.837+0000 7f8909400640 1 --2- 172.21.15.152:0/1668316242 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f88e006caa0 0x7f88e006ef60 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f88f4036000 tx=0x7f88f4008030 comp rx=0 tx=0).stop 2024-04-20T11:28:40.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.837+0000 7f8909400640 1 -- 172.21.15.152:0/1668316242 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f890406e900 msgr2=0x7f8904136dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:40.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.837+0000 7f8909400640 1 --2- 172.21.15.152:0/1668316242 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f890406e900 0x7f8904136dc0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f88ec00d5f0 tx=0x7f88ec00dab0 comp rx=0 tx=0).stop 2024-04-20T11:28:40.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.838+0000 7f8909400640 1 -- 172.21.15.152:0/1668316242 shutdown_connections 2024-04-20T11:28:40.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.838+0000 7f8909400640 1 --2- 172.21.15.152:0/1668316242 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f88e006caa0 0x7f88e006ef60 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:40.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.838+0000 7f8909400640 1 --2- 172.21.15.152:0/1668316242 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8904145f00 0x7f8904142bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:40.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.838+0000 7f8909400640 1 --2- 172.21.15.152:0/1668316242 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f89040cd610 0x7f890414d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:40.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.838+0000 7f8909400640 1 --2- 172.21.15.152:0/1668316242 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f890406e900 0x7f8904136dc0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:40.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.838+0000 7f8909400640 1 -- 172.21.15.152:0/1668316242 >> 172.21.15.152:0/1668316242 conn(0x7f8904067cf0 msgr2=0x7f8904071db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:40.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.838+0000 7f8909400640 1 -- 172.21.15.152:0/1668316242 shutdown_connections 2024-04-20T11:28:40.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:40.838+0000 7f8909400640 1 -- 172.21.15.152:0/1668316242 wait complete. 2024-04-20T11:28:40.855 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi175.front.sepia.ceph.com... 2024-04-20T11:28:40.855 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:28:40.855 DEBUG:teuthology.orchestra.run.smithi175:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-04-20T11:28:40.940 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi175.stderr:ceph-fuse[29798]: fuse finished with error 0 and tester_r 0 2024-04-20T11:28:40.943 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi175.front.sepia.ceph.com 2024-04-20T11:28:40.944 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:28:40.944 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-20T11:28:41.122 INFO:tasks.cephfs.mount:Terminating background process 2024-04-20T11:28:41.122 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-20T11:28:41.122 DEBUG:teuthology.orchestra.run.smithi152:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:28:41.181 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-20T11:28:41.181 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi152.front.sepia.ceph.com 2024-04-20T11:28:41.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:28:41.181 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-20T11:28:41.233 INFO:teuthology.orchestra.run.smithi152.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-20T11:28:41.234 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T11:28:41.234 INFO:tasks.cephfs.mount:Terminating background process 2024-04-20T11:28:41.235 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T11:28:41.235 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:28:41.292 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-20T11:28:41.292 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi175.front.sepia.ceph.com 2024-04-20T11:28:41.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:28:41.292 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-20T11:28:41.343 INFO:teuthology.orchestra.run.smithi175.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-20T11:28:41.344 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T11:28:41.345 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:28:41.558 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:41.558+0000 7f39e1000640 1 Processor -- start 2024-04-20T11:28:41.558 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:41.559+0000 7f39e1000640 1 -- start start 2024-04-20T11:28:41.559 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:41.560+0000 7f39e1000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f39dc0ba520 0x7f39dc0d28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:41.559 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:41.560+0000 7f39e1000640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f39dc0d2e30 0x7f39dc0d1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:41.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:41.560+0000 7f39e1000640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39dc0b80b0 con 0x7f39dc15eb80 2024-04-20T11:28:41.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:41.560+0000 7f39e1000640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f39dc0b8330 con 0x7f39dc0d2e30 2024-04-20T11:28:41.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:41.560+0000 7f39e1000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f39dc0b8610 con 0x7f39dc0ba520 2024-04-20T11:28:41.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:41.561+0000 7f39d3e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f39dc0ba520 0x7f39dc0d28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:41.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:41.561+0000 7f39d3e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f39dc0ba520 0x7f39dc0d28f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.152:37716/0 (socket says 172.21.15.152:37716) 2024-04-20T11:28:41.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:41.561+0000 7f39d3e00640 1 -- 172.21.15.152:0/1207307255 learned_addr learned my addr 172.21.15.152:0/1207307255 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:41.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:41.561+0000 7f39da800640 1 --2- 172.21.15.152:0/1207307255 >> v2:172.21.15.175:3300/0 conn(0x7f39dc0d2e30 0x7f39dc0d1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:41.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:41.562+0000 7f39da800640 1 -- 172.21.15.152:0/1207307255 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f39dc0ba520 msgr2=0x7f39dc0d28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:41.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:41.562+0000 7f39da800640 1 --2- 172.21.15.152:0/1207307255 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f39dc0ba520 0x7f39dc0d28f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:41.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:41.562+0000 7f39da800640 1 -- 172.21.15.152:0/1207307255 >> v1:172.21.15.175:6789/0 conn(0x7f39dc15eb80 legacy=0x7f39dc0b79a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:41.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:41.562+0000 7f39da800640 1 -- 172.21.15.152:0/1207307255 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39dc0b88f0 con 0x7f39dc0d2e30 2024-04-20T11:28:41.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:41.563+0000 7f39da800640 1 --2- 172.21.15.152:0/1207307255 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39dc0d2e30 0x7f39dc0d1810 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f39d4009d90 tx=0x7f39d402be20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50a450eb64e93fe4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:41.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:41.564+0000 7f39d9400640 1 -- 172.21.15.152:0/1207307255 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d4039070 con 0x7f39dc0d2e30 2024-04-20T11:28:41.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:41.564+0000 7f39d9400640 1 -- 172.21.15.152:0/1207307255 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39d402cbd0 con 0x7f39dc0d2e30 2024-04-20T11:28:41.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:41.564+0000 7f39d9400640 1 -- 172.21.15.152:0/1207307255 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d4034440 con 0x7f39dc0d2e30 2024-04-20T11:28:41.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:41.564+0000 7f39e1000640 1 -- 172.21.15.152:0/1207307255 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39dc0d2e30 msgr2=0x7f39dc0d1810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:41.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:41.564+0000 7f39e1000640 1 --2- 172.21.15.152:0/1207307255 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39dc0d2e30 0x7f39dc0d1810 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f39d4009d90 tx=0x7f39d402be20 comp rx=0 tx=0).stop 2024-04-20T11:28:41.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:41.564+0000 7f39e1000640 1 -- 172.21.15.152:0/1207307255 shutdown_connections 2024-04-20T11:28:41.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:41.564+0000 7f39e1000640 1 --2- 172.21.15.152:0/1207307255 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39dc0d2e30 0x7f39dc0d1810 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:41.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:41.564+0000 7f39e1000640 1 --2- 172.21.15.152:0/1207307255 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f39dc0ba520 0x7f39dc0d28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:41.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:41.564+0000 7f39e1000640 1 -- 172.21.15.152:0/1207307255 >> 172.21.15.152:0/1207307255 conn(0x7f39dc067ae0 msgr2=0x7f39dc067ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:41.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:41.564+0000 7f39e1000640 1 -- 172.21.15.152:0/1207307255 shutdown_connections 2024-04-20T11:28:41.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:41.564+0000 7f39e1000640 1 -- 172.21.15.152:0/1207307255 wait complete. 2024-04-20T11:28:41.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:41.566+0000 7f39e1000640 1 Processor -- start 2024-04-20T11:28:41.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:41.566+0000 7f39e1000640 1 -- start start 2024-04-20T11:28:41.567 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:41.567+0000 7f39e1000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39dc0ba520 0x7f39dc14eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:41.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:41.567+0000 7f39e1000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f39dc0d2e30 0x7f39dc0cc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:41.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:41.567+0000 7f39e1000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f39dc15eb80 0x7f39dc161130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:41.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:41.567+0000 7f39e1000640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f39dc0d1ed0 con 0x7f39dc15eb80 2024-04-20T11:28:41.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:41.567+0000 7f39e1000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f39dc0d1d50 con 0x7f39dc0ba520 2024-04-20T11:28:41.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:41.567+0000 7f39e1000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f39dc0d2050 con 0x7f39dc0d2e30 2024-04-20T11:28:41.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:41.568+0000 7f39da800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f39dc15eb80 0x7f39dc161130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:41.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:41.568+0000 7f39da800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f39dc15eb80 0x7f39dc161130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:60014/0 (socket says 172.21.15.152:60014) 2024-04-20T11:28:41.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:41.568+0000 7f39da800640 1 -- 172.21.15.152:0/291757511 learned_addr learned my addr 172.21.15.152:0/291757511 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:41.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:41.568+0000 7f39d9e00640 1 --2- 172.21.15.152:0/291757511 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39dc0ba520 0x7f39dc14eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:41.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:41.568+0000 7f39d3e00640 1 --2- 172.21.15.152:0/291757511 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f39dc0d2e30 0x7f39dc0cc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:41.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:41.568+0000 7f39da800640 1 -- 172.21.15.152:0/291757511 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f39dc0d2e30 msgr2=0x7f39dc0cc4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:41.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:41.568+0000 7f39da800640 1 --2- 172.21.15.152:0/291757511 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f39dc0d2e30 0x7f39dc0cc4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:41.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:41.568+0000 7f39da800640 1 -- 172.21.15.152:0/291757511 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39dc0ba520 msgr2=0x7f39dc14eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:41.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:41.568+0000 7f39da800640 1 --2- 172.21.15.152:0/291757511 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39dc0ba520 0x7f39dc14eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:41.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:41.568+0000 7f39da800640 1 -- 172.21.15.152:0/291757511 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39dc1f4f60 con 0x7f39dc15eb80 2024-04-20T11:28:41.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:41.568+0000 7f39d9e00640 1 --2- 172.21.15.152:0/291757511 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39dc0ba520 0x7f39dc14eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:28:41.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:41.569+0000 7f39da800640 1 --2- 172.21.15.152:0/291757511 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f39dc15eb80 0x7f39dc161130 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f39d4009d90 tx=0x7f39d4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:41.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:41.569+0000 7f39d2000640 1 -- 172.21.15.152:0/291757511 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d4039050 con 0x7f39dc15eb80 2024-04-20T11:28:41.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:41.569+0000 7f39d2000640 1 -- 172.21.15.152:0/291757511 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39d4045070 con 0x7f39dc15eb80 2024-04-20T11:28:41.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:41.569+0000 7f39e1000640 1 -- 172.21.15.152:0/291757511 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39dc1f5240 con 0x7f39dc15eb80 2024-04-20T11:28:41.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:41.569+0000 7f39d2000640 1 -- 172.21.15.152:0/291757511 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d403e830 con 0x7f39dc15eb80 2024-04-20T11:28:41.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:41.569+0000 7f39e1000640 1 -- 172.21.15.152:0/291757511 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39dc0cbfa0 con 0x7f39dc15eb80 2024-04-20T11:28:41.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:41.570+0000 7f39e1000640 1 -- 172.21.15.152:0/291757511 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39dc0cbaf0 con 0x7f39dc15eb80 2024-04-20T11:28:41.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:41.571+0000 7f39d2000640 1 -- 172.21.15.152:0/291757511 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f39d403e9d0 con 0x7f39dc15eb80 2024-04-20T11:28:41.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:41.573+0000 7f39d2000640 1 --2- 172.21.15.152:0/291757511 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f39b806caa0 0x7f39b806ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:41.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:41.573+0000 7f39d2000640 1 -- 172.21.15.152:0/291757511 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f39d40aebd0 con 0x7f39dc15eb80 2024-04-20T11:28:41.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:41.573+0000 7f39d9e00640 1 --2- 172.21.15.152:0/291757511 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f39b806caa0 0x7f39b806ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:41.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:41.574+0000 7f39d9e00640 1 --2- 172.21.15.152:0/291757511 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f39b806caa0 0x7f39b806ef60 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f39dc1e8ab0 tx=0x7f39cc009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:41.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:41.577+0000 7f39d2000640 1 -- 172.21.15.152:0/291757511 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f39d407cf30 con 0x7f39dc15eb80 2024-04-20T11:28:41.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:41.819+0000 7f39e1000640 1 -- 172.21.15.152:0/291757511 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f39dc0689f0 con 0x7f39dc15eb80 2024-04-20T11:28:41.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:41.821+0000 7f39d2000640 1 -- 172.21.15.152:0/291757511 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5211 (secure 0 0 0) 0x7f39d407c830 con 0x7f39dc15eb80 2024-04-20T11:28:41.820 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:28:41.820 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4643,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6834/1871483322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1871483322},{"type":"v1","addr":"172.21.15.175:6834","nonce":1871483322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4786,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/2579061164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6835","nonce":2579061164},{"type":"v1","addr":"172.21.15.152:6837","nonce":2579061164}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5208,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6836/3724129680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":3724129680},{"type":"v1","addr":"172.21.15.152:6836","nonce":3724129680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T11:28:17.902738+0000","modified":"2024-04-20T11:28:20.990798+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5223},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5223":{"gid":5223,"name":"b","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.175:6835/3367117968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6833","nonce":3367117968},{"type":"v1","addr":"172.21.15.175:6835","nonce":3367117968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5223,"qdb_cluster":[5223]},"id":2}]} 2024-04-20T11:28:41.821 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 16 2024-04-20T11:28:41.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:41.823+0000 7f39e1000640 1 -- 172.21.15.152:0/291757511 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f39b806caa0 msgr2=0x7f39b806ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:41.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:41.823+0000 7f39e1000640 1 --2- 172.21.15.152:0/291757511 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f39b806caa0 0x7f39b806ef60 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f39dc1e8ab0 tx=0x7f39cc009040 comp rx=0 tx=0).stop 2024-04-20T11:28:41.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:41.823+0000 7f39e1000640 1 -- 172.21.15.152:0/291757511 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f39dc15eb80 msgr2=0x7f39dc161130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:41.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:41.823+0000 7f39e1000640 1 --2- 172.21.15.152:0/291757511 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f39dc15eb80 0x7f39dc161130 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f39d4009d90 tx=0x7f39d4004a40 comp rx=0 tx=0).stop 2024-04-20T11:28:41.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:41.823+0000 7f39e1000640 1 -- 172.21.15.152:0/291757511 shutdown_connections 2024-04-20T11:28:41.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:41.823+0000 7f39e1000640 1 --2- 172.21.15.152:0/291757511 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f39b806caa0 0x7f39b806ef60 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:41.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:41.823+0000 7f39e1000640 1 --2- 172.21.15.152:0/291757511 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f39dc15eb80 0x7f39dc161130 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:41.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:41.823+0000 7f39e1000640 1 --2- 172.21.15.152:0/291757511 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f39dc0d2e30 0x7f39dc0cc4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:41.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:41.823+0000 7f39e1000640 1 --2- 172.21.15.152:0/291757511 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39dc0ba520 0x7f39dc14eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:41.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:41.823+0000 7f39e1000640 1 -- 172.21.15.152:0/291757511 >> 172.21.15.152:0/291757511 conn(0x7f39dc067ae0 msgr2=0x7f39dc0c83a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:41.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:41.824+0000 7f39e1000640 1 -- 172.21.15.152:0/291757511 shutdown_connections 2024-04-20T11:28:41.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:41.824+0000 7f39e1000640 1 -- 172.21.15.152:0/291757511 wait complete. 2024-04-20T11:28:41.841 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:28:42.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.035+0000 7fda40c00640 1 Processor -- start 2024-04-20T11:28:42.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.036+0000 7fda40c00640 1 -- start start 2024-04-20T11:28:42.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.037+0000 7fda40c00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fda3c0ba520 0x7fda3c0d28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:42.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.037+0000 7fda40c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fda3c0d2e30 0x7fda3c0d1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:42.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.037+0000 7fda40c00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda3c0b80b0 con 0x7fda3c15eb80 2024-04-20T11:28:42.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.037+0000 7fda40c00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fda3c0b8330 con 0x7fda3c0ba520 2024-04-20T11:28:42.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.037+0000 7fda40c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fda3c0b8610 con 0x7fda3c0d2e30 2024-04-20T11:28:42.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.037+0000 7fda31400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fda3c0ba520 0x7fda3c0d28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:42.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.037+0000 7fda31400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fda3c0ba520 0x7fda3c0d28f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:60030/0 (socket says 172.21.15.152:60030) 2024-04-20T11:28:42.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.037+0000 7fda31400640 1 -- 172.21.15.152:0/2741171396 learned_addr learned my addr 172.21.15.152:0/2741171396 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:42.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.038+0000 7fda3a800640 1 --2- 172.21.15.152:0/2741171396 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fda3c0d2e30 0x7fda3c0d1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:42.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.038+0000 7fda31400640 1 -- 172.21.15.152:0/2741171396 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fda3c0d2e30 msgr2=0x7fda3c0d1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:42.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.038+0000 7fda31400640 1 --2- 172.21.15.152:0/2741171396 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fda3c0d2e30 0x7fda3c0d1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:42.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.038+0000 7fda31400640 1 -- 172.21.15.152:0/2741171396 >> v1:172.21.15.175:6789/0 conn(0x7fda3c15eb80 legacy=0x7fda3c0b79a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:42.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.038+0000 7fda31400640 1 -- 172.21.15.152:0/2741171396 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda3c0b88f0 con 0x7fda3c0ba520 2024-04-20T11:28:42.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.039+0000 7fda31400640 1 --2- 172.21.15.152:0/2741171396 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fda3c0ba520 0x7fda3c0d28f0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fda24009870 tx=0x7fda2402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5ca46feaed01c70 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:42.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.039+0000 7fda39400640 1 -- 172.21.15.152:0/2741171396 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda240317c0 con 0x7fda3c0ba520 2024-04-20T11:28:42.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.039+0000 7fda39400640 1 -- 172.21.15.152:0/2741171396 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda24037070 con 0x7fda3c0ba520 2024-04-20T11:28:42.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.039+0000 7fda39400640 1 -- 172.21.15.152:0/2741171396 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda24031b80 con 0x7fda3c0ba520 2024-04-20T11:28:42.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.040+0000 7fda40c00640 1 -- 172.21.15.152:0/2741171396 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fda3c0ba520 msgr2=0x7fda3c0d28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:42.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.040+0000 7fda40c00640 1 --2- 172.21.15.152:0/2741171396 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fda3c0ba520 0x7fda3c0d28f0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fda24009870 tx=0x7fda2402b860 comp rx=0 tx=0).stop 2024-04-20T11:28:42.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.040+0000 7fda40c00640 1 -- 172.21.15.152:0/2741171396 shutdown_connections 2024-04-20T11:28:42.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.040+0000 7fda40c00640 1 --2- 172.21.15.152:0/2741171396 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fda3c0d2e30 0x7fda3c0d1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:42.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.040+0000 7fda40c00640 1 --2- 172.21.15.152:0/2741171396 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fda3c0ba520 0x7fda3c0d28f0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:42.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.040+0000 7fda40c00640 1 -- 172.21.15.152:0/2741171396 >> 172.21.15.152:0/2741171396 conn(0x7fda3c067ae0 msgr2=0x7fda3c067ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:42.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.040+0000 7fda40c00640 1 -- 172.21.15.152:0/2741171396 shutdown_connections 2024-04-20T11:28:42.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.040+0000 7fda40c00640 1 -- 172.21.15.152:0/2741171396 wait complete. 2024-04-20T11:28:42.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.042+0000 7fda40c00640 1 Processor -- start 2024-04-20T11:28:42.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.042+0000 7fda40c00640 1 -- start start 2024-04-20T11:28:42.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.043+0000 7fda40c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda3c0ba520 0x7fda3c159090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:42.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.043+0000 7fda40c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fda3c0d2e30 0x7fda3c15a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:42.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.043+0000 7fda40c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fda3c15eb80 0x7fda3c15a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:42.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.043+0000 7fda40c00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fda3c0d1ed0 con 0x7fda3c0d2e30 2024-04-20T11:28:42.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.043+0000 7fda40c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fda3c0d1d50 con 0x7fda3c0ba520 2024-04-20T11:28:42.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.043+0000 7fda40c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fda3c0d2050 con 0x7fda3c15eb80 2024-04-20T11:28:42.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.043+0000 7fda31400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fda3c0d2e30 0x7fda3c15a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:42.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.043+0000 7fda31400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fda3c0d2e30 0x7fda3c15a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:60044/0 (socket says 172.21.15.152:60044) 2024-04-20T11:28:42.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.043+0000 7fda31400640 1 -- 172.21.15.152:0/1191274618 learned_addr learned my addr 172.21.15.152:0/1191274618 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:42.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.044+0000 7fda3a800640 1 --2- 172.21.15.152:0/1191274618 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fda3c15eb80 0x7fda3c15a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:42.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.044+0000 7fda39e00640 1 --2- 172.21.15.152:0/1191274618 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda3c0ba520 0x7fda3c159090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:42.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.044+0000 7fda31400640 1 -- 172.21.15.152:0/1191274618 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fda3c15eb80 msgr2=0x7fda3c15a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:42.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.044+0000 7fda31400640 1 --2- 172.21.15.152:0/1191274618 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fda3c15eb80 0x7fda3c15a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:42.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.044+0000 7fda31400640 1 -- 172.21.15.152:0/1191274618 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda3c0ba520 msgr2=0x7fda3c159090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:42.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.044+0000 7fda31400640 1 --2- 172.21.15.152:0/1191274618 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda3c0ba520 0x7fda3c159090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:42.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.044+0000 7fda31400640 1 -- 172.21.15.152:0/1191274618 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda3c0ccbc0 con 0x7fda3c0d2e30 2024-04-20T11:28:42.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.044+0000 7fda39e00640 1 --2- 172.21.15.152:0/1191274618 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda3c0ba520 0x7fda3c159090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:28:42.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.044+0000 7fda31400640 1 --2- 172.21.15.152:0/1191274618 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fda3c0d2e30 0x7fda3c15a420 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fda24009870 tx=0x7fda240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:42.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.044+0000 7fda32a00640 1 -- 172.21.15.152:0/1191274618 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda24032650 con 0x7fda3c0d2e30 2024-04-20T11:28:42.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.045+0000 7fda32a00640 1 -- 172.21.15.152:0/1191274618 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda240327f0 con 0x7fda3c0d2e30 2024-04-20T11:28:42.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.045+0000 7fda32a00640 1 -- 172.21.15.152:0/1191274618 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda24032be0 con 0x7fda3c0d2e30 2024-04-20T11:28:42.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.045+0000 7fda40c00640 1 -- 172.21.15.152:0/1191274618 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda3c0cce40 con 0x7fda3c0d2e30 2024-04-20T11:28:42.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.045+0000 7fda40c00640 1 -- 172.21.15.152:0/1191274618 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda3c0cd400 con 0x7fda3c0d2e30 2024-04-20T11:28:42.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.046+0000 7fda40c00640 1 -- 172.21.15.152:0/1191274618 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda3c0689f0 con 0x7fda3c0d2e30 2024-04-20T11:28:42.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.047+0000 7fda32a00640 1 -- 172.21.15.152:0/1191274618 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7fda2404c060 con 0x7fda3c0d2e30 2024-04-20T11:28:42.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.048+0000 7fda32a00640 1 --2- 172.21.15.152:0/1191274618 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fda1806caa0 0x7fda1806ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:42.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.048+0000 7fda32a00640 1 -- 172.21.15.152:0/1191274618 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fda240ad710 con 0x7fda3c0d2e30 2024-04-20T11:28:42.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.048+0000 7fda39e00640 1 --2- 172.21.15.152:0/1191274618 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fda1806caa0 0x7fda1806ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:42.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.049+0000 7fda39e00640 1 --2- 172.21.15.152:0/1191274618 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fda1806caa0 0x7fda1806ef60 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fda2c003eb0 tx=0x7fda2c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:42.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.052+0000 7fda32a00640 1 -- 172.21.15.152:0/1191274618 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fda3c0ccbc0 con 0x7fda3c0d2e30 2024-04-20T11:28:42.252 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.252+0000 7fda40c00640 1 -- 172.21.15.152:0/1191274618 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fda3c0d5660 con 0x7fda3c0d2e30 2024-04-20T11:28:42.255 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.255+0000 7fda32a00640 1 -- 172.21.15.152:0/1191274618 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) ==== 74+0+16729 (secure 0 0 0) 0x7fda24040070 con 0x7fda3c0d2e30 2024-04-20T11:28:42.255 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:28:42.255 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":34,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:28:19.478546+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:28:15.704670+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-20T11:28:16.891101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:28:42.258 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.258+0000 7fda40c00640 1 -- 172.21.15.152:0/1191274618 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fda1806caa0 msgr2=0x7fda1806ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:42.258 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.258+0000 7fda40c00640 1 --2- 172.21.15.152:0/1191274618 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fda1806caa0 0x7fda1806ef60 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fda2c003eb0 tx=0x7fda2c009040 comp rx=0 tx=0).stop 2024-04-20T11:28:42.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.258+0000 7fda40c00640 1 -- 172.21.15.152:0/1191274618 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fda3c0d2e30 msgr2=0x7fda3c15a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:42.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.258+0000 7fda40c00640 1 --2- 172.21.15.152:0/1191274618 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fda3c0d2e30 0x7fda3c15a420 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fda24009870 tx=0x7fda240046c0 comp rx=0 tx=0).stop 2024-04-20T11:28:42.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.259+0000 7fda40c00640 1 -- 172.21.15.152:0/1191274618 shutdown_connections 2024-04-20T11:28:42.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.259+0000 7fda40c00640 1 --2- 172.21.15.152:0/1191274618 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fda1806caa0 0x7fda1806ef60 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:42.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.259+0000 7fda40c00640 1 --2- 172.21.15.152:0/1191274618 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fda3c15eb80 0x7fda3c15a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:42.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.259+0000 7fda40c00640 1 --2- 172.21.15.152:0/1191274618 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fda3c0d2e30 0x7fda3c15a420 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:42.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.259+0000 7fda40c00640 1 --2- 172.21.15.152:0/1191274618 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda3c0ba520 0x7fda3c159090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:42.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.259+0000 7fda40c00640 1 -- 172.21.15.152:0/1191274618 >> 172.21.15.152:0/1191274618 conn(0x7fda3c067ae0 msgr2=0x7fda3c0c9720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:42.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.259+0000 7fda40c00640 1 -- 172.21.15.152:0/1191274618 shutdown_connections 2024-04-20T11:28:42.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.259+0000 7fda40c00640 1 -- 172.21.15.152:0/1191274618 wait complete. 2024-04-20T11:28:42.277 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T11:26:47.246571+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T11:28:15.704670+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T11:28:16.891101+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T11:28:42.277 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T11:28:42.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.477+0000 7f5570e00640 1 Processor -- start 2024-04-20T11:28:42.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.478+0000 7f5570e00640 1 -- start start 2024-04-20T11:28:42.482 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.479+0000 7f5570e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f556c0cf5b0 0x7f556c0d4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:42.482 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.479+0000 7f5570e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f556c160230 0x7f556c1606e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:42.482 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.479+0000 7f5570e00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f556c0d2b70 con 0x7f556c0ceeb0 2024-04-20T11:28:42.482 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.479+0000 7f5570e00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f556c0d5420 con 0x7f556c160230 2024-04-20T11:28:42.482 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.479+0000 7f5570e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f556c0d2e50 con 0x7f556c0cf5b0 2024-04-20T11:28:42.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.480+0000 7f556aa00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f556c0ceeb0 0x7f556c0cf990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:47122/0 (socket says 172.21.15.152:47122) 2024-04-20T11:28:42.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.480+0000 7f556aa00640 1 -- 172.21.15.152:0/469123956 learned_addr learned my addr 172.21.15.152:0/469123956 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:28:42.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.480+0000 7f556b400640 1 --2- 172.21.15.152:0/469123956 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f556c0cf5b0 0x7f556c0d4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:42.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.480+0000 7f556be00640 1 --2- 172.21.15.152:0/469123956 >> v2:172.21.15.175:3300/0 conn(0x7f556c160230 0x7f556c1606e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:42.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.480+0000 7f556a000640 1 -- 172.21.15.152:0/469123956 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3327125759 0 0) 0x7f556c0d2b70 con 0x7f556c0ceeb0 2024-04-20T11:28:42.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.480+0000 7f556a000640 1 -- 172.21.15.152:0/469123956 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5550003520 con 0x7f556c0ceeb0 2024-04-20T11:28:42.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.481+0000 7f556a000640 1 -- 172.21.15.152:0/469123956 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3500738875 0 0) 0x7f5550003520 con 0x7f556c0ceeb0 2024-04-20T11:28:42.484 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.481+0000 7f556a000640 1 -- 172.21.15.152:0/469123956 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f556c0cf5b0 msgr2=0x7f556c0d4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:42.484 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.481+0000 7f556a000640 1 --2- 172.21.15.152:0/469123956 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f556c0cf5b0 0x7f556c0d4ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:42.484 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.481+0000 7f556a000640 1 -- 172.21.15.152:0/469123956 >> v2:172.21.15.175:3300/0 conn(0x7f556c160230 msgr2=0x7f556c1606e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:42.484 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.481+0000 7f556a000640 1 --2- 172.21.15.152:0/469123956 >> v2:172.21.15.175:3300/0 conn(0x7f556c160230 0x7f556c1606e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:42.484 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.481+0000 7f556a000640 1 -- 172.21.15.152:0/469123956 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f556c0d3130 con 0x7f556c0ceeb0 2024-04-20T11:28:42.484 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.481+0000 7f556a000640 1 -- 172.21.15.152:0/469123956 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f55580017f0 con 0x7f556c0ceeb0 2024-04-20T11:28:42.484 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.481+0000 7f556a000640 1 -- 172.21.15.152:0/469123956 >> v1:172.21.15.152:6789/0 conn(0x7f556c0ceeb0 legacy=0x7f556c0cf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:42.484 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.481+0000 7f556a000640 1 --2- 172.21.15.152:0/469123956 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f55500039b0 0x7f5550003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:42.485 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.481+0000 7f556a000640 1 --2- 172.21.15.152:0/469123956 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5550004800 0x7f5550004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:42.485 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.481+0000 7f556a000640 1 --2- 172.21.15.152:0/469123956 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f556c0cf5b0 0x7f55500051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:42.485 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.481+0000 7f556a000640 1 -- 172.21.15.152:0/469123956 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f556c0d5420 con 0x7f55500039b0 2024-04-20T11:28:42.485 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.481+0000 7f556a000640 1 -- 172.21.15.152:0/469123956 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f556c0d2e50 con 0x7f5550004800 2024-04-20T11:28:42.485 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.481+0000 7f556a000640 1 -- 172.21.15.152:0/469123956 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f555000b9c0 con 0x7f556c0cf5b0 2024-04-20T11:28:42.485 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.481+0000 7f556b400640 1 --2- 172.21.15.152:0/469123956 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f55500039b0 0x7f5550003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:42.485 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.482+0000 7f556b400640 1 -- 172.21.15.152:0/469123956 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f556c0cf5b0 msgr2=0x7f55500051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:42.485 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.482+0000 7f556be00640 1 --2- 172.21.15.152:0/469123956 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f556c0cf5b0 0x7f55500051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:42.485 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.482+0000 7f556b400640 1 --2- 172.21.15.152:0/469123956 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f556c0cf5b0 0x7f55500051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:42.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.482+0000 7f556b400640 1 -- 172.21.15.152:0/469123956 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5550004800 msgr2=0x7f5550004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:42.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.482+0000 7f556b400640 1 --2- 172.21.15.152:0/469123956 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5550004800 0x7f5550004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:42.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.482+0000 7f556b400640 1 -- 172.21.15.152:0/469123956 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5550003520 con 0x7f55500039b0 2024-04-20T11:28:42.488 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.483+0000 7f556b400640 1 --2- 172.21.15.152:0/469123956 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f55500039b0 0x7f5550003dd0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f555c009ac0 tx=0x7f5550005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=915e5aa0cda8f632 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:42.488 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.483+0000 7f556a000640 1 -- 172.21.15.152:0/469123956 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f555c039070 con 0x7f55500039b0 2024-04-20T11:28:42.489 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.483+0000 7f556a000640 1 -- 172.21.15.152:0/469123956 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f555c02c6b0 con 0x7f55500039b0 2024-04-20T11:28:42.489 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.483+0000 7f556a000640 1 -- 172.21.15.152:0/469123956 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f555c009690 con 0x7f55500039b0 2024-04-20T11:28:42.489 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.484+0000 7f5570e00640 1 -- 172.21.15.152:0/469123956 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f55500039b0 msgr2=0x7f5550003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:42.489 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.484+0000 7f5570e00640 1 --2- 172.21.15.152:0/469123956 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f55500039b0 0x7f5550003dd0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f555c009ac0 tx=0x7f5550005830 comp rx=0 tx=0).stop 2024-04-20T11:28:42.489 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.484+0000 7f556be00640 1 -- 172.21.15.152:0/469123956 reap_dead start 2024-04-20T11:28:42.489 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.484+0000 7f5570e00640 1 -- 172.21.15.152:0/469123956 shutdown_connections 2024-04-20T11:28:42.489 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.484+0000 7f5570e00640 1 -- 172.21.15.152:0/469123956 >> 172.21.15.152:0/469123956 conn(0x7f556c08bfd0 msgr2=0x7f556c0d4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:42.489 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.484+0000 7f5570e00640 1 -- 172.21.15.152:0/469123956 shutdown_connections 2024-04-20T11:28:42.489 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.484+0000 7f5570e00640 1 -- 172.21.15.152:0/469123956 wait complete. 2024-04-20T11:28:42.489 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.487+0000 7f5570e00640 1 Processor -- start 2024-04-20T11:28:42.489 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.487+0000 7f5570e00640 1 -- start start 2024-04-20T11:28:42.489 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.488+0000 7f5570e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f55500039b0 0x7f556c0ca870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:42.489 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.488+0000 7f5570e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f556c0b74c0 0x7f556c1f4d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:42.489 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.488+0000 7f5570e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f556c1f5260 0x7f556c1f8340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:42.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.488+0000 7f5570e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f556c0d3c10 con 0x7f556c0b74c0 2024-04-20T11:28:42.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.488+0000 7f5570e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f556c0d36f0 con 0x7f556c1f5260 2024-04-20T11:28:42.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.488+0000 7f5570e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f556c0d2790 con 0x7f55500039b0 2024-04-20T11:28:42.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.489+0000 7f556aa00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f556c0b74c0 0x7f556c1f4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:42.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.489+0000 7f556aa00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f556c0b74c0 0x7f556c1f4d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:60074/0 (socket says 172.21.15.152:60074) 2024-04-20T11:28:42.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.489+0000 7f556aa00640 1 -- 172.21.15.152:0/739408870 learned_addr learned my addr 172.21.15.152:0/739408870 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:42.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.489+0000 7f556aa00640 1 -- 172.21.15.152:0/739408870 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f55500039b0 msgr2=0x7f556c0ca870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:42.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.489+0000 7f556aa00640 1 --2- 172.21.15.152:0/739408870 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f55500039b0 0x7f556c0ca870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:42.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.489+0000 7f556aa00640 1 -- 172.21.15.152:0/739408870 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f556c1f5260 msgr2=0x7f556c1f8340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:42.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.489+0000 7f556aa00640 1 --2- 172.21.15.152:0/739408870 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f556c1f5260 0x7f556c1f8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:42.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.489+0000 7f556aa00640 1 -- 172.21.15.152:0/739408870 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f556c1f56e0 con 0x7f556c0b74c0 2024-04-20T11:28:42.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.490+0000 7f556aa00640 1 --2- 172.21.15.152:0/739408870 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f556c0b74c0 0x7f556c1f4d20 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f555800c790 tx=0x7f555800cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:42.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.490+0000 7f5563400640 1 -- 172.21.15.152:0/739408870 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5558017070 con 0x7f556c0b74c0 2024-04-20T11:28:42.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.490+0000 7f5563400640 1 -- 172.21.15.152:0/739408870 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5558007480 con 0x7f556c0b74c0 2024-04-20T11:28:42.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.490+0000 7f5563400640 1 -- 172.21.15.152:0/739408870 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55580059c0 con 0x7f556c0b74c0 2024-04-20T11:28:42.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.490+0000 7f5570e00640 1 -- 172.21.15.152:0/739408870 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f556c0b92a0 con 0x7f556c0b74c0 2024-04-20T11:28:42.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.490+0000 7f5570e00640 1 -- 172.21.15.152:0/739408870 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f556c0b9860 con 0x7f556c0b74c0 2024-04-20T11:28:42.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.492+0000 7f5570e00640 1 -- 172.21.15.152:0/739408870 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f556c1491c0 con 0x7f556c0b74c0 2024-04-20T11:28:42.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.493+0000 7f5563400640 1 -- 172.21.15.152:0/739408870 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f555801a070 con 0x7f556c0b74c0 2024-04-20T11:28:42.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.495+0000 7f5563400640 1 --2- 172.21.15.152:0/739408870 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f554c06caa0 0x7f554c06ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:42.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.495+0000 7f5563400640 1 -- 172.21.15.152:0/739408870 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f555808b3d0 con 0x7f556c0b74c0 2024-04-20T11:28:42.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.495+0000 7f556b400640 1 --2- 172.21.15.152:0/739408870 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f554c06caa0 0x7f554c06ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:42.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.496+0000 7f556b400640 1 --2- 172.21.15.152:0/739408870 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f554c06caa0 0x7f554c06ef60 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f555c036000 tx=0x7f555c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:42.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.499+0000 7f5563400640 1 -- 172.21.15.152:0/739408870 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f556c1491c0 con 0x7f556c0b74c0 2024-04-20T11:28:42.697 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.697+0000 7f5570e00640 1 -- 172.21.15.152:0/739408870 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f556c075e60 con 0x7f556c0b74c0 2024-04-20T11:28:42.698 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.698+0000 7f5563400640 1 -- 172.21.15.152:0/739408870 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v34) ==== 93+0+10 (secure 0 0 0) 0x7f5558059580 con 0x7f556c0b74c0 2024-04-20T11:28:42.698 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 1 2024-04-20T11:28:42.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.701+0000 7f5570e00640 1 -- 172.21.15.152:0/739408870 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f554c06caa0 msgr2=0x7f554c06ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:42.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.701+0000 7f5570e00640 1 --2- 172.21.15.152:0/739408870 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f554c06caa0 0x7f554c06ef60 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f555c036000 tx=0x7f555c02e000 comp rx=0 tx=0).stop 2024-04-20T11:28:42.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.701+0000 7f5570e00640 1 -- 172.21.15.152:0/739408870 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f556c0b74c0 msgr2=0x7f556c1f4d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:42.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.701+0000 7f5570e00640 1 --2- 172.21.15.152:0/739408870 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f556c0b74c0 0x7f556c1f4d20 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f555800c790 tx=0x7f555800cc50 comp rx=0 tx=0).stop 2024-04-20T11:28:42.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.701+0000 7f5570e00640 1 -- 172.21.15.152:0/739408870 shutdown_connections 2024-04-20T11:28:42.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.701+0000 7f5570e00640 1 --2- 172.21.15.152:0/739408870 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f554c06caa0 0x7f554c06ef60 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:42.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.701+0000 7f5570e00640 1 --2- 172.21.15.152:0/739408870 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f556c1f5260 0x7f556c1f8340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:42.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.701+0000 7f5570e00640 1 --2- 172.21.15.152:0/739408870 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f556c0b74c0 0x7f556c1f4d20 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:42.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.701+0000 7f5570e00640 1 --2- 172.21.15.152:0/739408870 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f55500039b0 0x7f556c0ca870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:42.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.701+0000 7f5570e00640 1 -- 172.21.15.152:0/739408870 >> 172.21.15.152:0/739408870 conn(0x7f556c08bfd0 msgr2=0x7f556c15a190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:42.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.701+0000 7f5570e00640 1 -- 172.21.15.152:0/739408870 shutdown_connections 2024-04-20T11:28:42.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.701+0000 7f5570e00640 1 -- 172.21.15.152:0/739408870 wait complete. 2024-04-20T11:28:42.718 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T11:28:42.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.918+0000 7f0626800640 1 Processor -- start 2024-04-20T11:28:42.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.918+0000 7f0626800640 1 -- start start 2024-04-20T11:28:42.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.920+0000 7f0626800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f06200b3fa0 0x7f0620162440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:42.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.920+0000 7f0626800640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06200adc30 con 0x7f06200afba0 2024-04-20T11:28:42.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.920+0000 7f0626800640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0620162a70 con 0x7f062015e5d0 2024-04-20T11:28:42.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.920+0000 7f0626800640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f0620162d50 con 0x7f06200b3fa0 2024-04-20T11:28:42.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.920+0000 7f061fe00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f06200b3fa0 0x7f0620162440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:42.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.921+0000 7f0617e00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f06200afba0 0x7f06200ae110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:47124/0 (socket says 172.21.15.152:47124) 2024-04-20T11:28:42.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.921+0000 7f0617e00640 1 -- 172.21.15.152:0/4007013745 learned_addr learned my addr 172.21.15.152:0/4007013745 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:28:42.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.921+0000 7f061fe00640 1 -- 172.21.15.152:0/4007013745 >> v1:172.21.15.175:6789/0 conn(0x7f062015e5d0 legacy=0x7f06200af490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:42.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.921+0000 7f061fe00640 1 -- 172.21.15.152:0/4007013745 >> v1:172.21.15.152:6789/0 conn(0x7f06200afba0 legacy=0x7f06200ae110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:42.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.921+0000 7f061ea00640 1 -- 172.21.15.152:0/4007013745 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2978952231 0 0) 0x7f06200adc30 con 0x7f06200afba0 2024-04-20T11:28:42.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.921+0000 7f061fe00640 1 -- 172.21.15.152:0/4007013745 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0620163030 con 0x7f06200b3fa0 2024-04-20T11:28:42.922 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.922+0000 7f061fe00640 1 --2- 172.21.15.152:0/4007013745 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f06200b3fa0 0x7f0620162440 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f0618009d90 tx=0x7f061802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfa6629b8a9126df server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:42.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.923+0000 7f061ea00640 1 -- 172.21.15.152:0/4007013745 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0618039070 con 0x7f06200b3fa0 2024-04-20T11:28:42.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.923+0000 7f061ea00640 1 -- 172.21.15.152:0/4007013745 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f061802cc60 con 0x7f06200b3fa0 2024-04-20T11:28:42.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.923+0000 7f061ea00640 1 -- 172.21.15.152:0/4007013745 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0618034660 con 0x7f06200b3fa0 2024-04-20T11:28:42.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.923+0000 7f0626800640 1 -- 172.21.15.152:0/4007013745 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f06200b3fa0 msgr2=0x7f0620162440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:42.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.923+0000 7f0626800640 1 --2- 172.21.15.152:0/4007013745 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f06200b3fa0 0x7f0620162440 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f0618009d90 tx=0x7f061802beb0 comp rx=0 tx=0).stop 2024-04-20T11:28:42.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.923+0000 7f0626800640 1 -- 172.21.15.152:0/4007013745 shutdown_connections 2024-04-20T11:28:42.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.923+0000 7f0626800640 1 --2- 172.21.15.152:0/4007013745 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f06200b3fa0 0x7f0620162440 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:42.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.923+0000 7f0626800640 1 -- 172.21.15.152:0/4007013745 >> 172.21.15.152:0/4007013745 conn(0x7f062006b980 msgr2=0x7f062006bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:42.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.923+0000 7f0626800640 1 -- 172.21.15.152:0/4007013745 shutdown_connections 2024-04-20T11:28:42.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.924+0000 7f0626800640 1 -- 172.21.15.152:0/4007013745 wait complete. 2024-04-20T11:28:42.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.925+0000 7f0626800640 1 Processor -- start 2024-04-20T11:28:42.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.925+0000 7f0626800640 1 -- start start 2024-04-20T11:28:42.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.926+0000 7f0626800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f06200afba0 0x7f06201f7c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:42.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.926+0000 7f0626800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f06200b3fa0 0x7f062015f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:42.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.926+0000 7f0626800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f062015e5d0 0x7f06201617a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:42.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.926+0000 7f0626800640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f062006e0a0 con 0x7f06200b3fa0 2024-04-20T11:28:42.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.926+0000 7f0626800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0620163310 con 0x7f06200afba0 2024-04-20T11:28:42.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.926+0000 7f0626800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f062006e220 con 0x7f062015e5d0 2024-04-20T11:28:42.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.926+0000 7f0617e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f06200b3fa0 0x7f062015f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:42.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.927+0000 7f0617e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f06200b3fa0 0x7f062015f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:60094/0 (socket says 172.21.15.152:60094) 2024-04-20T11:28:42.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.927+0000 7f0617e00640 1 -- 172.21.15.152:0/1114529344 learned_addr learned my addr 172.21.15.152:0/1114529344 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:42.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.927+0000 7f0617e00640 1 -- 172.21.15.152:0/1114529344 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f062015e5d0 msgr2=0x7f06201617a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:42.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.927+0000 7f0617e00640 1 --2- 172.21.15.152:0/1114529344 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f062015e5d0 0x7f06201617a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:42.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.927+0000 7f0617e00640 1 -- 172.21.15.152:0/1114529344 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f06200afba0 msgr2=0x7f06201f7c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:42.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.927+0000 7f0617e00640 1 --2- 172.21.15.152:0/1114529344 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f06200afba0 0x7f06201f7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:42.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.927+0000 7f0617e00640 1 -- 172.21.15.152:0/1114529344 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0620161ce0 con 0x7f06200b3fa0 2024-04-20T11:28:42.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.927+0000 7f0617e00640 1 --2- 172.21.15.152:0/1114529344 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f06200b3fa0 0x7f062015f010 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f060800c470 tx=0x7f060800c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:42.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.927+0000 7f0617400640 1 -- 172.21.15.152:0/1114529344 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0608018070 con 0x7f06200b3fa0 2024-04-20T11:28:42.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.927+0000 7f0617400640 1 -- 172.21.15.152:0/1114529344 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06080074e0 con 0x7f06200b3fa0 2024-04-20T11:28:42.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.927+0000 7f0617400640 1 -- 172.21.15.152:0/1114529344 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0608004a20 con 0x7f06200b3fa0 2024-04-20T11:28:42.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.928+0000 7f0626800640 1 -- 172.21.15.152:0/1114529344 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06200b11c0 con 0x7f06200b3fa0 2024-04-20T11:28:42.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.928+0000 7f0626800640 1 -- 172.21.15.152:0/1114529344 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06200b1780 con 0x7f06200b3fa0 2024-04-20T11:28:42.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.929+0000 7f0626800640 1 -- 172.21.15.152:0/1114529344 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06200689f0 con 0x7f06200b3fa0 2024-04-20T11:28:42.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.930+0000 7f0617400640 1 -- 172.21.15.152:0/1114529344 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f0608004bc0 con 0x7f06200b3fa0 2024-04-20T11:28:42.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.931+0000 7f0617400640 1 --2- 172.21.15.152:0/1114529344 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f05fc06caa0 0x7f05fc06ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:42.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.931+0000 7f0617400640 1 -- 172.21.15.152:0/1114529344 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f060808b280 con 0x7f06200b3fa0 2024-04-20T11:28:42.931 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.931+0000 7f061f400640 1 --2- 172.21.15.152:0/1114529344 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f05fc06caa0 0x7f05fc06ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:42.931 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.932+0000 7f061f400640 1 --2- 172.21.15.152:0/1114529344 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f05fc06caa0 0x7f05fc06ef60 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f0610006fd0 tx=0x7f0610006bc0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:42.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:42.935+0000 7f0617400640 1 -- 172.21.15.152:0/1114529344 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f0608059390 con 0x7f06200b3fa0 2024-04-20T11:28:43.136 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.136+0000 7f0626800640 1 -- 172.21.15.152:0/1114529344 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f06200af1c0 con 0x7f06200b3fa0 2024-04-20T11:28:43.137 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.137+0000 7f0617400640 1 -- 172.21.15.152:0/1114529344 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v34) ==== 104+0+11 (secure 0 0 0) 0x7f0608058c90 con 0x7f06200b3fa0 2024-04-20T11:28:43.137 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 32 2024-04-20T11:28:43.139 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.139+0000 7f0626800640 1 -- 172.21.15.152:0/1114529344 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f05fc06caa0 msgr2=0x7f05fc06ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:43.139 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.139+0000 7f0626800640 1 --2- 172.21.15.152:0/1114529344 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f05fc06caa0 0x7f05fc06ef60 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f0610006fd0 tx=0x7f0610006bc0 comp rx=0 tx=0).stop 2024-04-20T11:28:43.140 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.139+0000 7f0626800640 1 -- 172.21.15.152:0/1114529344 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f06200b3fa0 msgr2=0x7f062015f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:43.140 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.139+0000 7f0626800640 1 --2- 172.21.15.152:0/1114529344 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f06200b3fa0 0x7f062015f010 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f060800c470 tx=0x7f060800c930 comp rx=0 tx=0).stop 2024-04-20T11:28:43.140 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.140+0000 7f0626800640 1 -- 172.21.15.152:0/1114529344 shutdown_connections 2024-04-20T11:28:43.140 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.140+0000 7f0626800640 1 --2- 172.21.15.152:0/1114529344 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f05fc06caa0 0x7f05fc06ef60 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:43.140 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.140+0000 7f0626800640 1 --2- 172.21.15.152:0/1114529344 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f062015e5d0 0x7f06201617a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:43.140 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.140+0000 7f0626800640 1 --2- 172.21.15.152:0/1114529344 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f06200b3fa0 0x7f062015f010 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:43.140 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.140+0000 7f0626800640 1 --2- 172.21.15.152:0/1114529344 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f06200afba0 0x7f06201f7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:43.140 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.140+0000 7f0626800640 1 -- 172.21.15.152:0/1114529344 >> 172.21.15.152:0/1114529344 conn(0x7f062006b980 msgr2=0x7f06200ae520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:43.140 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.140+0000 7f0626800640 1 -- 172.21.15.152:0/1114529344 shutdown_connections 2024-04-20T11:28:43.140 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.140+0000 7f0626800640 1 -- 172.21.15.152:0/1114529344 wait complete. 2024-04-20T11:28:43.158 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T11:28:43.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.363+0000 7f5ed9a00640 1 Processor -- start 2024-04-20T11:28:43.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.363+0000 7f5ed9a00640 1 -- start start 2024-04-20T11:28:43.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.365+0000 7f5ed9a00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f5ed40b2880 0x7f5ed415fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:43.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.365+0000 7f5ed9a00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ed415f1c0 con 0x7f5ed40cfc40 2024-04-20T11:28:43.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.365+0000 7f5ed9a00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ed406f670 con 0x7f5ed415b510 2024-04-20T11:28:43.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.365+0000 7f5ed9a00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f5ed406f420 con 0x7f5ed40b2880 2024-04-20T11:28:43.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.365+0000 7f5ecbe00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f5ed40cfc40 0x7f5ed40b4800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:47126/0 (socket says 172.21.15.152:47126) 2024-04-20T11:28:43.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.365+0000 7f5ecbe00640 1 -- 172.21.15.152:0/4119815671 learned_addr learned my addr 172.21.15.152:0/4119815671 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:28:43.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.366+0000 7f5ed3400640 1 --2- 172.21.15.152:0/4119815671 >> v2:172.21.15.175:3300/0 conn(0x7f5ed40b2880 0x7f5ed415fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:43.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.366+0000 7f5ed2000640 1 -- 172.21.15.152:0/4119815671 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2490895812 0 0) 0x7f5ed415f1c0 con 0x7f5ed40cfc40 2024-04-20T11:28:43.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.366+0000 7f5ed2000640 1 -- 172.21.15.152:0/4119815671 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5eb8005730 con 0x7f5ed40cfc40 2024-04-20T11:28:43.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.366+0000 7f5ed2000640 1 -- 172.21.15.152:0/4119815671 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3129625564 0 0) 0x7f5eb8005730 con 0x7f5ed40cfc40 2024-04-20T11:28:43.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.366+0000 7f5ed2000640 1 -- 172.21.15.152:0/4119815671 >> v2:172.21.15.175:3300/0 conn(0x7f5ed40b2880 msgr2=0x7f5ed415fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:43.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.366+0000 7f5ed2000640 1 --2- 172.21.15.152:0/4119815671 >> v2:172.21.15.175:3300/0 conn(0x7f5ed40b2880 0x7f5ed415fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:43.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.366+0000 7f5ed2000640 1 -- 172.21.15.152:0/4119815671 >> v1:172.21.15.175:6789/0 conn(0x7f5ed415b510 legacy=0x7f5ed40b2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:43.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.366+0000 7f5ed2000640 1 -- 172.21.15.152:0/4119815671 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ed415f430 con 0x7f5ed40cfc40 2024-04-20T11:28:43.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.366+0000 7f5ed2000640 1 -- 172.21.15.152:0/4119815671 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f5ebc001670 con 0x7f5ed40cfc40 2024-04-20T11:28:43.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.366+0000 7f5ed2000640 1 -- 172.21.15.152:0/4119815671 >> v1:172.21.15.152:6789/0 conn(0x7f5ed40cfc40 legacy=0x7f5ed40b4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:43.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.367+0000 7f5ed2000640 1 --2- 172.21.15.152:0/4119815671 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5eb8005bc0 0x7f5eb8005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:43.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.367+0000 7f5ed2000640 1 --2- 172.21.15.152:0/4119815671 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5eb8006a10 0x7f5eb8006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:43.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.367+0000 7f5ed2000640 1 --2- 172.21.15.152:0/4119815671 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5eb80073d0 0x7f5eb800dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:43.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.367+0000 7f5ed2000640 1 -- 172.21.15.152:0/4119815671 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed406f420 con 0x7f5eb80073d0 2024-04-20T11:28:43.369 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.367+0000 7f5ed2000640 1 -- 172.21.15.152:0/4119815671 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5eb800e370 con 0x7f5eb8006a10 2024-04-20T11:28:43.369 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.367+0000 7f5ed2000640 1 -- 172.21.15.152:0/4119815671 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5eb800e570 con 0x7f5eb8005bc0 2024-04-20T11:28:43.369 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.367+0000 7f5ed3400640 1 --2- 172.21.15.152:0/4119815671 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5eb80073d0 0x7f5eb800dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:43.369 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.367+0000 7f5ed2a00640 1 --2- 172.21.15.152:0/4119815671 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5eb8005bc0 0x7f5eb8005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:43.369 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.367+0000 7f5ed3400640 1 -- 172.21.15.152:0/4119815671 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5eb8005bc0 msgr2=0x7f5eb8005fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:43.369 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.367+0000 7f5ed3400640 1 --2- 172.21.15.152:0/4119815671 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5eb8005bc0 0x7f5eb8005fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:43.369 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.367+0000 7f5ecbe00640 1 --2- 172.21.15.152:0/4119815671 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5eb8006a10 0x7f5eb8006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:43.369 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.367+0000 7f5ed3400640 1 -- 172.21.15.152:0/4119815671 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5eb8006a10 msgr2=0x7f5eb8006e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:43.369 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.367+0000 7f5ed3400640 1 --2- 172.21.15.152:0/4119815671 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5eb8006a10 0x7f5eb8006e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:43.369 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.367+0000 7f5ed3400640 1 -- 172.21.15.152:0/4119815671 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5eb8005730 con 0x7f5eb80073d0 2024-04-20T11:28:43.369 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.367+0000 7f5ed2a00640 1 --2- 172.21.15.152:0/4119815671 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5eb8005bc0 0x7f5eb8005fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:28:43.369 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.368+0000 7f5ed3400640 1 -- 172.21.15.152:0/4119815671 reap_dead start 2024-04-20T11:28:43.371 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.368+0000 7f5ed3400640 1 --2- 172.21.15.152:0/4119815671 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5eb80073d0 0x7f5eb800dbd0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f5ecc009fc0 tx=0x7f5eb8007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a6da5127f2bf4ac server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:43.371 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.369+0000 7f5ed2000640 1 -- 172.21.15.152:0/4119815671 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ecc009c10 con 0x7f5eb80073d0 2024-04-20T11:28:43.371 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.369+0000 7f5ed2000640 1 -- 172.21.15.152:0/4119815671 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ecc02cca0 con 0x7f5eb80073d0 2024-04-20T11:28:43.371 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.369+0000 7f5ed2000640 1 -- 172.21.15.152:0/4119815671 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ecc003bd0 con 0x7f5eb80073d0 2024-04-20T11:28:43.371 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.369+0000 7f5ed9a00640 1 -- 172.21.15.152:0/4119815671 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5eb80073d0 msgr2=0x7f5eb800dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:43.371 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.369+0000 7f5ed9a00640 1 --2- 172.21.15.152:0/4119815671 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5eb80073d0 0x7f5eb800dbd0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f5ecc009fc0 tx=0x7f5eb8007a40 comp rx=0 tx=0).stop 2024-04-20T11:28:43.371 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.369+0000 7f5ed9a00640 1 -- 172.21.15.152:0/4119815671 shutdown_connections 2024-04-20T11:28:43.371 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.369+0000 7f5ed9a00640 1 --2- 172.21.15.152:0/4119815671 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5eb80073d0 0x7f5eb800dbd0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:43.371 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.369+0000 7f5ed9a00640 1 -- 172.21.15.152:0/4119815671 >> 172.21.15.152:0/4119815671 conn(0x7f5ed406b980 msgr2=0x7f5ed406bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:43.371 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.370+0000 7f5ed9a00640 1 -- 172.21.15.152:0/4119815671 shutdown_connections 2024-04-20T11:28:43.371 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.370+0000 7f5ed9a00640 1 -- 172.21.15.152:0/4119815671 wait complete. 2024-04-20T11:28:43.371 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.371+0000 7f5ed9a00640 1 Processor -- start 2024-04-20T11:28:43.371 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.371+0000 7f5ed9a00640 1 -- start start 2024-04-20T11:28:43.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.372+0000 7f5ed9a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5eb80073d0 0x7f5ed414b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:43.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.372+0000 7f5ed9a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ed415ba70 0x7f5ed415d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:43.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.372+0000 7f5ed9a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5ed41564c0 0x7f5ed4156940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:43.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.372+0000 7f5ed9a00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed406f950 con 0x7f5ed41564c0 2024-04-20T11:28:43.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.372+0000 7f5ed9a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed415b8f0 con 0x7f5eb80073d0 2024-04-20T11:28:43.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.372+0000 7f5ed9a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5ed415fc30 con 0x7f5ed415ba70 2024-04-20T11:28:43.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.373+0000 7f5ed3400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5ed41564c0 0x7f5ed4156940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:43.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.373+0000 7f5ed3400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5ed41564c0 0x7f5ed4156940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:60124/0 (socket says 172.21.15.152:60124) 2024-04-20T11:28:43.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.373+0000 7f5ed3400640 1 -- 172.21.15.152:0/3108870686 learned_addr learned my addr 172.21.15.152:0/3108870686 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:43.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.373+0000 7f5ecbe00640 1 --2- 172.21.15.152:0/3108870686 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ed415ba70 0x7f5ed415d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:43.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.373+0000 7f5ed2a00640 1 --2- 172.21.15.152:0/3108870686 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5eb80073d0 0x7f5ed414b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:43.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.373+0000 7f5ed3400640 1 -- 172.21.15.152:0/3108870686 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ed415ba70 msgr2=0x7f5ed415d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:43.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.373+0000 7f5ed3400640 1 --2- 172.21.15.152:0/3108870686 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ed415ba70 0x7f5ed415d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:43.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.373+0000 7f5ed3400640 1 -- 172.21.15.152:0/3108870686 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5eb80073d0 msgr2=0x7f5ed414b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:43.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.373+0000 7f5ed3400640 1 --2- 172.21.15.152:0/3108870686 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5eb80073d0 0x7f5ed414b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:43.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.373+0000 7f5ed3400640 1 -- 172.21.15.152:0/3108870686 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ed414b690 con 0x7f5ed41564c0 2024-04-20T11:28:43.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.373+0000 7f5ed3400640 1 --2- 172.21.15.152:0/3108870686 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5ed41564c0 0x7f5ed4156940 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f5ecc00e200 tx=0x7f5ecc009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:43.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.374+0000 7f5ecaa00640 1 -- 172.21.15.152:0/3108870686 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ecc009c10 con 0x7f5ed41564c0 2024-04-20T11:28:43.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.374+0000 7f5ecaa00640 1 -- 172.21.15.152:0/3108870686 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ecc044070 con 0x7f5ed41564c0 2024-04-20T11:28:43.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.374+0000 7f5ecaa00640 1 -- 172.21.15.152:0/3108870686 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ecc003bb0 con 0x7f5ed41564c0 2024-04-20T11:28:43.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.374+0000 7f5ed9a00640 1 -- 172.21.15.152:0/3108870686 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ed41f7dc0 con 0x7f5ed41564c0 2024-04-20T11:28:43.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.374+0000 7f5ed9a00640 1 -- 172.21.15.152:0/3108870686 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ed41f8380 con 0x7f5ed41564c0 2024-04-20T11:28:43.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.375+0000 7f5ed9a00640 1 -- 172.21.15.152:0/3108870686 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ed40689f0 con 0x7f5ed41564c0 2024-04-20T11:28:43.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.376+0000 7f5ecaa00640 1 -- 172.21.15.152:0/3108870686 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f5ecc005d10 con 0x7f5ed41564c0 2024-04-20T11:28:43.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.377+0000 7f5ecaa00640 1 --2- 172.21.15.152:0/3108870686 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f5eb006caa0 0x7f5eb006ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:43.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.377+0000 7f5ecaa00640 1 -- 172.21.15.152:0/3108870686 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f5ecc0ae540 con 0x7f5ed41564c0 2024-04-20T11:28:43.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.377+0000 7f5ed2a00640 1 --2- 172.21.15.152:0/3108870686 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f5eb006caa0 0x7f5eb006ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:43.377 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.378+0000 7f5ed2a00640 1 --2- 172.21.15.152:0/3108870686 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f5eb006caa0 0x7f5eb006ef60 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5ec40015b0 tx=0x7f5ec4009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:43.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.381+0000 7f5ecaa00640 1 -- 172.21.15.152:0/3108870686 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f5ecc07c650 con 0x7f5ed41564c0 2024-04-20T11:28:43.582 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.582+0000 7f5ed9a00640 1 -- 172.21.15.152:0/3108870686 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f5ed406ecf0 con 0x7f5ed41564c0 2024-04-20T11:28:43.583 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.583+0000 7f5ecaa00640 1 -- 172.21.15.152:0/3108870686 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v34) ==== 100+0+11 (secure 0 0 0) 0x7f5ecc07bf50 con 0x7f5ed41564c0 2024-04-20T11:28:43.583 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 64 2024-04-20T11:28:43.585 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.585+0000 7f5ed9a00640 1 -- 172.21.15.152:0/3108870686 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f5eb006caa0 msgr2=0x7f5eb006ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:43.585 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.585+0000 7f5ed9a00640 1 --2- 172.21.15.152:0/3108870686 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f5eb006caa0 0x7f5eb006ef60 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5ec40015b0 tx=0x7f5ec4009040 comp rx=0 tx=0).stop 2024-04-20T11:28:43.586 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.586+0000 7f5ed9a00640 1 -- 172.21.15.152:0/3108870686 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5ed41564c0 msgr2=0x7f5ed4156940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:43.586 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.586+0000 7f5ed9a00640 1 --2- 172.21.15.152:0/3108870686 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5ed41564c0 0x7f5ed4156940 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f5ecc00e200 tx=0x7f5ecc009750 comp rx=0 tx=0).stop 2024-04-20T11:28:43.586 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.586+0000 7f5ed9a00640 1 -- 172.21.15.152:0/3108870686 shutdown_connections 2024-04-20T11:28:43.586 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.586+0000 7f5ed9a00640 1 --2- 172.21.15.152:0/3108870686 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f5eb006caa0 0x7f5eb006ef60 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:43.586 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.586+0000 7f5ed9a00640 1 --2- 172.21.15.152:0/3108870686 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5ed41564c0 0x7f5ed4156940 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:43.586 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.586+0000 7f5ed9a00640 1 --2- 172.21.15.152:0/3108870686 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ed415ba70 0x7f5ed415d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:43.586 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.586+0000 7f5ed9a00640 1 --2- 172.21.15.152:0/3108870686 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5eb80073d0 0x7f5ed414b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:43.586 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.586+0000 7f5ed9a00640 1 -- 172.21.15.152:0/3108870686 >> 172.21.15.152:0/3108870686 conn(0x7f5ed406b980 msgr2=0x7f5ed40cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:43.586 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.586+0000 7f5ed9a00640 1 -- 172.21.15.152:0/3108870686 shutdown_connections 2024-04-20T11:28:43.586 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.586+0000 7f5ed9a00640 1 -- 172.21.15.152:0/3108870686 wait complete. 2024-04-20T11:28:43.604 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:28:43.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.807+0000 7f3b5b200640 1 Processor -- start 2024-04-20T11:28:43.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.808+0000 7f3b5b200640 1 -- start start 2024-04-20T11:28:43.809 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.809+0000 7f3b5b200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3b5c0723e0 0x7f3b5c0afa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:43.809 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.809+0000 7f3b5b200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f3b5c06e010 0x7f3b5c06e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:43.809 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.809+0000 7f3b5b200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b5c0aec90 con 0x7f3b5c147410 2024-04-20T11:28:43.809 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.809+0000 7f3b5b200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f3b5c0aef70 con 0x7f3b5c06e010 2024-04-20T11:28:43.809 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.809+0000 7f3b5b200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f3b5c0af250 con 0x7f3b5c0723e0 2024-04-20T11:28:43.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.810+0000 7f3b51400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3b5c0723e0 0x7f3b5c0afa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:43.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.810+0000 7f3b5a800640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f3b5c06e010 0x7f3b5c06e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:43.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.810+0000 7f3b59e00640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7f3b5c147410 0x7f3b5c071cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.152:50816/0 (socket says 172.21.15.152:50816) 2024-04-20T11:28:43.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.810+0000 7f3b59e00640 1 -- 172.21.15.152:0/1280629199 learned_addr learned my addr 172.21.15.152:0/1280629199 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:28:43.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.811+0000 7f3b5a800640 1 -- 172.21.15.152:0/1280629199 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3b5c0723e0 msgr2=0x7f3b5c0afa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:43.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.811+0000 7f3b5a800640 1 --2- 172.21.15.152:0/1280629199 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3b5c0723e0 0x7f3b5c0afa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:43.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.811+0000 7f3b5a800640 1 -- 172.21.15.152:0/1280629199 >> v1:172.21.15.175:6789/0 conn(0x7f3b5c147410 legacy=0x7f3b5c071cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:43.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.811+0000 7f3b5a800640 1 -- 172.21.15.152:0/1280629199 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b5c0af530 con 0x7f3b5c06e010 2024-04-20T11:28:43.812 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.812+0000 7f3b5a800640 1 --2- 172.21.15.152:0/1280629199 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3b5c06e010 0x7f3b5c06e4a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f3b54009d90 tx=0x7f3b5402bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1cfa9bc29e507ec server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:43.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.813+0000 7f3b59400640 1 -- 172.21.15.152:0/1280629199 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b54039070 con 0x7f3b5c06e010 2024-04-20T11:28:43.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.813+0000 7f3b59400640 1 -- 172.21.15.152:0/1280629199 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b5402cc50 con 0x7f3b5c06e010 2024-04-20T11:28:43.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.813+0000 7f3b59400640 1 -- 172.21.15.152:0/1280629199 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b54034440 con 0x7f3b5c06e010 2024-04-20T11:28:43.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.814+0000 7f3b5b200640 1 -- 172.21.15.152:0/1280629199 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3b5c06e010 msgr2=0x7f3b5c06e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:43.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.814+0000 7f3b5b200640 1 --2- 172.21.15.152:0/1280629199 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3b5c06e010 0x7f3b5c06e4a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f3b54009d90 tx=0x7f3b5402bea0 comp rx=0 tx=0).stop 2024-04-20T11:28:43.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.814+0000 7f3b5b200640 1 -- 172.21.15.152:0/1280629199 shutdown_connections 2024-04-20T11:28:43.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.814+0000 7f3b5b200640 1 --2- 172.21.15.152:0/1280629199 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3b5c06e010 0x7f3b5c06e4a0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:43.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.814+0000 7f3b5b200640 1 --2- 172.21.15.152:0/1280629199 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3b5c0723e0 0x7f3b5c0afa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:43.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.814+0000 7f3b5b200640 1 -- 172.21.15.152:0/1280629199 >> 172.21.15.152:0/1280629199 conn(0x7f3b5c067ae0 msgr2=0x7f3b5c06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:43.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.814+0000 7f3b5b200640 1 -- 172.21.15.152:0/1280629199 shutdown_connections 2024-04-20T11:28:43.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.814+0000 7f3b5b200640 1 -- 172.21.15.152:0/1280629199 wait complete. 2024-04-20T11:28:43.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.816+0000 7f3b5b200640 1 Processor -- start 2024-04-20T11:28:43.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.816+0000 7f3b5b200640 1 -- start start 2024-04-20T11:28:43.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.817+0000 7f3b5b200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3b5c06e010 0x7f3b5c13cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:43.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.817+0000 7f3b5b200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3b5c0723e0 0x7f3b5c145380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:43.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.817+0000 7f3b5b200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3b5c147410 0x7f3b5c13f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:43.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.817+0000 7f3b5b200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f3b5c06e9e0 con 0x7f3b5c06e010 2024-04-20T11:28:43.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.817+0000 7f3b5b200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3b5c0b3390 con 0x7f3b5c0723e0 2024-04-20T11:28:43.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.817+0000 7f3b5b200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f3b5c1458c0 con 0x7f3b5c147410 2024-04-20T11:28:43.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.817+0000 7f3b59e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3b5c06e010 0x7f3b5c13cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:43.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.817+0000 7f3b59e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3b5c06e010 0x7f3b5c13cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:60132/0 (socket says 172.21.15.152:60132) 2024-04-20T11:28:43.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.817+0000 7f3b59e00640 1 -- 172.21.15.152:0/1503009073 learned_addr learned my addr 172.21.15.152:0/1503009073 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:43.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.817+0000 7f3b5a800640 1 --2- 172.21.15.152:0/1503009073 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3b5c147410 0x7f3b5c13f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:43.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.818+0000 7f3b59e00640 1 -- 172.21.15.152:0/1503009073 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3b5c147410 msgr2=0x7f3b5c13f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:43.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.818+0000 7f3b51400640 1 --2- 172.21.15.152:0/1503009073 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3b5c0723e0 0x7f3b5c145380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:43.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.818+0000 7f3b59e00640 1 --2- 172.21.15.152:0/1503009073 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3b5c147410 0x7f3b5c13f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:43.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.818+0000 7f3b59e00640 1 -- 172.21.15.152:0/1503009073 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3b5c0723e0 msgr2=0x7f3b5c145380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:43.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.818+0000 7f3b59e00640 1 --2- 172.21.15.152:0/1503009073 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3b5c0723e0 0x7f3b5c145380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:43.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.818+0000 7f3b59e00640 1 -- 172.21.15.152:0/1503009073 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b5c145b60 con 0x7f3b5c06e010 2024-04-20T11:28:43.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.818+0000 7f3b5a800640 1 --2- 172.21.15.152:0/1503009073 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3b5c147410 0x7f3b5c13f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:28:43.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.818+0000 7f3b59e00640 1 --2- 172.21.15.152:0/1503009073 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3b5c06e010 0x7f3b5c13cb00 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c00c480 tx=0x7f3b4c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:43.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.818+0000 7f3b52a00640 1 -- 172.21.15.152:0/1503009073 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b4c017070 con 0x7f3b5c06e010 2024-04-20T11:28:43.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.818+0000 7f3b52a00640 1 -- 172.21.15.152:0/1503009073 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b4c005ce0 con 0x7f3b5c06e010 2024-04-20T11:28:43.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.818+0000 7f3b52a00640 1 -- 172.21.15.152:0/1503009073 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b4c0072f0 con 0x7f3b5c06e010 2024-04-20T11:28:43.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.818+0000 7f3b5b200640 1 -- 172.21.15.152:0/1503009073 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b5c145e40 con 0x7f3b5c06e010 2024-04-20T11:28:43.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.818+0000 7f3b5b200640 1 -- 172.21.15.152:0/1503009073 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b5c146400 con 0x7f3b5c06e010 2024-04-20T11:28:43.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.820+0000 7f3b5b200640 1 -- 172.21.15.152:0/1503009073 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b5c075e60 con 0x7f3b5c06e010 2024-04-20T11:28:43.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.820+0000 7f3b52a00640 1 -- 172.21.15.152:0/1503009073 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f3b4c01a070 con 0x7f3b5c06e010 2024-04-20T11:28:43.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.822+0000 7f3b52a00640 1 --2- 172.21.15.152:0/1503009073 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f3b3806cad0 0x7f3b3806ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:43.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.822+0000 7f3b52a00640 1 -- 172.21.15.152:0/1503009073 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f3b4c08b110 con 0x7f3b5c06e010 2024-04-20T11:28:43.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.822+0000 7f3b51400640 1 --2- 172.21.15.152:0/1503009073 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f3b3806cad0 0x7f3b3806ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:43.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.823+0000 7f3b51400640 1 --2- 172.21.15.152:0/1503009073 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f3b3806cad0 0x7f3b3806ef90 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f3b5c13d410 tx=0x7f3b44008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:43.826 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:43.827+0000 7f3b52a00640 1 -- 172.21.15.152:0/1503009073 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f3b4c058c60 con 0x7f3b5c06e010 2024-04-20T11:28:44.065 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.066+0000 7f3b5b200640 1 -- 172.21.15.152:0/1503009073 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3b5c1405f0 con 0x7f3b5c06e010 2024-04-20T11:28:44.066 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.067+0000 7f3b52a00640 1 -- 172.21.15.152:0/1503009073 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5211 (secure 0 0 0) 0x7f3b4c05cf60 con 0x7f3b5c06e010 2024-04-20T11:28:44.066 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:28:44.067 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4643,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6834/1871483322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1871483322},{"type":"v1","addr":"172.21.15.175:6834","nonce":1871483322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4786,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/2579061164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6835","nonce":2579061164},{"type":"v1","addr":"172.21.15.152:6837","nonce":2579061164}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5208,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6836/3724129680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":3724129680},{"type":"v1","addr":"172.21.15.152:6836","nonce":3724129680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T11:28:17.902738+0000","modified":"2024-04-20T11:28:20.990798+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5223},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5223":{"gid":5223,"name":"b","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.175:6835/3367117968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6833","nonce":3367117968},{"type":"v1","addr":"172.21.15.175:6835","nonce":3367117968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5223,"qdb_cluster":[5223]},"id":2}]} 2024-04-20T11:28:44.067 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 16 2024-04-20T11:28:44.069 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.069+0000 7f3b5b200640 1 -- 172.21.15.152:0/1503009073 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f3b3806cad0 msgr2=0x7f3b3806ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:44.069 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.069+0000 7f3b5b200640 1 --2- 172.21.15.152:0/1503009073 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f3b3806cad0 0x7f3b3806ef90 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f3b5c13d410 tx=0x7f3b44008040 comp rx=0 tx=0).stop 2024-04-20T11:28:44.069 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.069+0000 7f3b5b200640 1 -- 172.21.15.152:0/1503009073 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3b5c06e010 msgr2=0x7f3b5c13cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:44.069 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.069+0000 7f3b5b200640 1 --2- 172.21.15.152:0/1503009073 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3b5c06e010 0x7f3b5c13cb00 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c00c480 tx=0x7f3b4c00c940 comp rx=0 tx=0).stop 2024-04-20T11:28:44.069 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.069+0000 7f3b5b200640 1 -- 172.21.15.152:0/1503009073 shutdown_connections 2024-04-20T11:28:44.069 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.069+0000 7f3b5b200640 1 --2- 172.21.15.152:0/1503009073 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f3b3806cad0 0x7f3b3806ef90 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:44.069 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.069+0000 7f3b5b200640 1 --2- 172.21.15.152:0/1503009073 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3b5c147410 0x7f3b5c13f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:44.069 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.069+0000 7f3b5b200640 1 --2- 172.21.15.152:0/1503009073 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3b5c0723e0 0x7f3b5c145380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:44.070 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.069+0000 7f3b5b200640 1 --2- 172.21.15.152:0/1503009073 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3b5c06e010 0x7f3b5c13cb00 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:44.070 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.069+0000 7f3b5b200640 1 -- 172.21.15.152:0/1503009073 >> 172.21.15.152:0/1503009073 conn(0x7f3b5c067ae0 msgr2=0x7f3b5c06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:44.070 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.070+0000 7f3b5b200640 1 -- 172.21.15.152:0/1503009073 shutdown_connections 2024-04-20T11:28:44.070 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.070+0000 7f3b5b200640 1 -- 172.21.15.152:0/1503009073 wait complete. 2024-04-20T11:28:44.089 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:28:44.292 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.292+0000 7f60eb200640 1 Processor -- start 2024-04-20T11:28:44.292 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.293+0000 7f60eb200640 1 -- start start 2024-04-20T11:28:44.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.294+0000 7f60eb200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f60ec15fcc0 0x7f60ec0d4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:44.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.294+0000 7f60eb200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f60ec0b65c0 0x7f60ec0cf240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:44.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.294+0000 7f60eb200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60ec0d3350 con 0x7f60ec1603d0 2024-04-20T11:28:44.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.294+0000 7f60eb200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f60ec0d3630 con 0x7f60ec15fcc0 2024-04-20T11:28:44.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.294+0000 7f60eb200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f60ec0d3910 con 0x7f60ec0b65c0 2024-04-20T11:28:44.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.294+0000 7f60e9e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f60ec15fcc0 0x7f60ec0d4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:44.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.294+0000 7f60e9e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f60ec15fcc0 0x7f60ec0d4e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:60146/0 (socket says 172.21.15.152:60146) 2024-04-20T11:28:44.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.294+0000 7f60e9e00640 1 -- 172.21.15.152:0/1279233109 learned_addr learned my addr 172.21.15.152:0/1279233109 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:44.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.295+0000 7f60ea800640 1 --2- 172.21.15.152:0/1279233109 >> v2:172.21.15.175:3300/0 conn(0x7f60ec0b65c0 0x7f60ec0cf240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:44.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.295+0000 7f60e9e00640 1 -- 172.21.15.152:0/1279233109 >> v2:172.21.15.175:3300/0 conn(0x7f60ec0b65c0 msgr2=0x7f60ec0cf240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:44.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.295+0000 7f60e9e00640 1 --2- 172.21.15.152:0/1279233109 >> v2:172.21.15.175:3300/0 conn(0x7f60ec0b65c0 0x7f60ec0cf240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:44.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.295+0000 7f60e9e00640 1 -- 172.21.15.152:0/1279233109 >> v1:172.21.15.175:6789/0 conn(0x7f60ec1603d0 legacy=0x7f60ec0b5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:44.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.295+0000 7f60e9e00640 1 -- 172.21.15.152:0/1279233109 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60ec0ced20 con 0x7f60ec15fcc0 2024-04-20T11:28:44.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.296+0000 7f60e9e00640 1 --2- 172.21.15.152:0/1279233109 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f60ec15fcc0 0x7f60ec0d4e70 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f60ec08e7f0 tx=0x7f60dc02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56c32e197503bf77 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:44.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.297+0000 7f60e8a00640 1 -- 172.21.15.152:0/1279233109 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60dc02ca80 con 0x7f60ec15fcc0 2024-04-20T11:28:44.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.297+0000 7f60e8a00640 1 -- 172.21.15.152:0/1279233109 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60dc02cc20 con 0x7f60ec15fcc0 2024-04-20T11:28:44.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.297+0000 7f60e8a00640 1 -- 172.21.15.152:0/1279233109 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60dc0343f0 con 0x7f60ec15fcc0 2024-04-20T11:28:44.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.297+0000 7f60eb200640 1 -- 172.21.15.152:0/1279233109 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f60ec15fcc0 msgr2=0x7f60ec0d4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:44.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.297+0000 7f60eb200640 1 --2- 172.21.15.152:0/1279233109 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f60ec15fcc0 0x7f60ec0d4e70 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f60ec08e7f0 tx=0x7f60dc02bee0 comp rx=0 tx=0).stop 2024-04-20T11:28:44.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.297+0000 7f60eb200640 1 -- 172.21.15.152:0/1279233109 shutdown_connections 2024-04-20T11:28:44.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.297+0000 7f60eb200640 1 --2- 172.21.15.152:0/1279233109 >> v2:172.21.15.175:3300/0 conn(0x7f60ec0b65c0 0x7f60ec0cf240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:44.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.297+0000 7f60eb200640 1 --2- 172.21.15.152:0/1279233109 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f60ec15fcc0 0x7f60ec0d4e70 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:44.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.297+0000 7f60eb200640 1 -- 172.21.15.152:0/1279233109 >> 172.21.15.152:0/1279233109 conn(0x7f60ec08bfd0 msgr2=0x7f60ec0d4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:44.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.298+0000 7f60eb200640 1 -- 172.21.15.152:0/1279233109 shutdown_connections 2024-04-20T11:28:44.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.298+0000 7f60eb200640 1 -- 172.21.15.152:0/1279233109 wait complete. 2024-04-20T11:28:44.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.299+0000 7f60eb200640 1 Processor -- start 2024-04-20T11:28:44.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.299+0000 7f60eb200640 1 -- start start 2024-04-20T11:28:44.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.300+0000 7f60eb200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f60ec0b65c0 0x7f60ec1f4490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:44.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.300+0000 7f60eb200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f60ec15fcc0 0x7f60ec158530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:44.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.300+0000 7f60eb200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f60ec1603d0 0x7f60ec0b9470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:44.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.300+0000 7f60eb200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f60ec0b5600 con 0x7f60ec1603d0 2024-04-20T11:28:44.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.300+0000 7f60eb200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f60ec0b5480 con 0x7f60ec0b65c0 2024-04-20T11:28:44.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.300+0000 7f60eb200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f60ec0cf780 con 0x7f60ec15fcc0 2024-04-20T11:28:44.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.301+0000 7f60ea800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f60ec1603d0 0x7f60ec0b9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:44.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.301+0000 7f60ea800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f60ec1603d0 0x7f60ec0b9470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:60158/0 (socket says 172.21.15.152:60158) 2024-04-20T11:28:44.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.301+0000 7f60ea800640 1 -- 172.21.15.152:0/1477578629 learned_addr learned my addr 172.21.15.152:0/1477578629 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:44.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.301+0000 7f60e9400640 1 --2- 172.21.15.152:0/1477578629 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f60ec15fcc0 0x7f60ec158530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:44.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.301+0000 7f60e9e00640 1 --2- 172.21.15.152:0/1477578629 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f60ec0b65c0 0x7f60ec1f4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:44.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.301+0000 7f60ea800640 1 -- 172.21.15.152:0/1477578629 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f60ec15fcc0 msgr2=0x7f60ec158530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:44.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.301+0000 7f60ea800640 1 --2- 172.21.15.152:0/1477578629 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f60ec15fcc0 0x7f60ec158530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:44.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.301+0000 7f60ea800640 1 -- 172.21.15.152:0/1477578629 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f60ec0b65c0 msgr2=0x7f60ec1f4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:44.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.301+0000 7f60ea800640 1 --2- 172.21.15.152:0/1477578629 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f60ec0b65c0 0x7f60ec1f4490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:44.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.301+0000 7f60ea800640 1 -- 172.21.15.152:0/1477578629 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60ec0b7760 con 0x7f60ec1603d0 2024-04-20T11:28:44.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.302+0000 7f60ea800640 1 --2- 172.21.15.152:0/1477578629 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f60ec1603d0 0x7f60ec0b9470 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f60e400d600 tx=0x7f60e400dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:44.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.302+0000 7f60e2000640 1 -- 172.21.15.152:0/1477578629 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60e40125f0 con 0x7f60ec1603d0 2024-04-20T11:28:44.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.302+0000 7f60e2000640 1 -- 172.21.15.152:0/1477578629 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60e4019070 con 0x7f60ec1603d0 2024-04-20T11:28:44.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.302+0000 7f60e2000640 1 -- 172.21.15.152:0/1477578629 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60e40129d0 con 0x7f60ec1603d0 2024-04-20T11:28:44.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.302+0000 7f60eb200640 1 -- 172.21.15.152:0/1477578629 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60ec0b7a40 con 0x7f60ec1603d0 2024-04-20T11:28:44.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.302+0000 7f60eb200640 1 -- 172.21.15.152:0/1477578629 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60ec15af30 con 0x7f60ec1603d0 2024-04-20T11:28:44.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.303+0000 7f60eb200640 1 -- 172.21.15.152:0/1477578629 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60ec075e60 con 0x7f60ec1603d0 2024-04-20T11:28:44.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.304+0000 7f60e2000640 1 -- 172.21.15.152:0/1477578629 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f60e4012b70 con 0x7f60ec1603d0 2024-04-20T11:28:44.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.305+0000 7f60e2000640 1 --2- 172.21.15.152:0/1477578629 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f60c806caa0 0x7f60c806ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:44.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.305+0000 7f60e2000640 1 -- 172.21.15.152:0/1477578629 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f60e408c290 con 0x7f60ec1603d0 2024-04-20T11:28:44.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.306+0000 7f60e9e00640 1 --2- 172.21.15.152:0/1477578629 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f60c806caa0 0x7f60c806ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:44.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.306+0000 7f60e9e00640 1 --2- 172.21.15.152:0/1477578629 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f60c806caa0 0x7f60c806ef60 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f60dc035000 tx=0x7f60dc030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:44.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.310+0000 7f60e2000640 1 -- 172.21.15.152:0/1477578629 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f60e405a3e0 con 0x7f60ec1603d0 2024-04-20T11:28:44.509 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.510+0000 7f60eb200640 1 -- 172.21.15.152:0/1477578629 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f60ec0b6ce0 con 0x7f60ec1603d0 2024-04-20T11:28:44.511 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.512+0000 7f60e2000640 1 -- 172.21.15.152:0/1477578629 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) ==== 74+0+16729 (secure 0 0 0) 0x7f60e4059ce0 con 0x7f60ec1603d0 2024-04-20T11:28:44.512 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:28:44.512 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":34,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:28:19.478546+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:28:15.704670+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-20T11:28:16.891101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:28:44.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.514+0000 7f60eb200640 1 -- 172.21.15.152:0/1477578629 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f60c806caa0 msgr2=0x7f60c806ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:44.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.514+0000 7f60eb200640 1 --2- 172.21.15.152:0/1477578629 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f60c806caa0 0x7f60c806ef60 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f60dc035000 tx=0x7f60dc030000 comp rx=0 tx=0).stop 2024-04-20T11:28:44.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.514+0000 7f60eb200640 1 -- 172.21.15.152:0/1477578629 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f60ec1603d0 msgr2=0x7f60ec0b9470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:44.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.514+0000 7f60eb200640 1 --2- 172.21.15.152:0/1477578629 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f60ec1603d0 0x7f60ec0b9470 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f60e400d600 tx=0x7f60e400dac0 comp rx=0 tx=0).stop 2024-04-20T11:28:44.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.515+0000 7f60eb200640 1 -- 172.21.15.152:0/1477578629 shutdown_connections 2024-04-20T11:28:44.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.515+0000 7f60eb200640 1 --2- 172.21.15.152:0/1477578629 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f60c806caa0 0x7f60c806ef60 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:44.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.515+0000 7f60eb200640 1 --2- 172.21.15.152:0/1477578629 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f60ec1603d0 0x7f60ec0b9470 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:44.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.515+0000 7f60eb200640 1 --2- 172.21.15.152:0/1477578629 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f60ec15fcc0 0x7f60ec158530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:44.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.515+0000 7f60eb200640 1 --2- 172.21.15.152:0/1477578629 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f60ec0b65c0 0x7f60ec1f4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:44.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.515+0000 7f60eb200640 1 -- 172.21.15.152:0/1477578629 >> 172.21.15.152:0/1477578629 conn(0x7f60ec08bfd0 msgr2=0x7f60ec06dde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:44.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.515+0000 7f60eb200640 1 -- 172.21.15.152:0/1477578629 shutdown_connections 2024-04-20T11:28:44.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.515+0000 7f60eb200640 1 -- 172.21.15.152:0/1477578629 wait complete. 2024-04-20T11:28:44.533 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-20T11:28:44.533 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:28:44.732 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.732+0000 7feb72200640 1 Processor -- start 2024-04-20T11:28:44.732 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.732+0000 7feb72200640 1 -- start start 2024-04-20T11:28:44.733 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.734+0000 7feb72200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7feb6c1603d0 0x7feb6c0b5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:44.733 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.734+0000 7feb72200640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb6c0d3350 con 0x7feb6c0b5480 2024-04-20T11:28:44.733 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.734+0000 7feb72200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb6c0d3630 con 0x7feb6c15fcc0 2024-04-20T11:28:44.734 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.734+0000 7feb72200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7feb6c0d3910 con 0x7feb6c1603d0 2024-04-20T11:28:44.734 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.734+0000 7feb71800640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7feb6c0b5480 0x7feb6c0cf240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:47128/0 (socket says 172.21.15.152:47128) 2024-04-20T11:28:44.734 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.734+0000 7feb71800640 1 -- 172.21.15.152:0/487476706 learned_addr learned my addr 172.21.15.152:0/487476706 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:28:44.734 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.734+0000 7feb6be00640 1 --2- 172.21.15.152:0/487476706 >> v2:172.21.15.152:3300/0 conn(0x7feb6c1603d0 0x7feb6c0b5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:44.734 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.735+0000 7feb6be00640 1 -- 172.21.15.152:0/487476706 >> v1:172.21.15.175:6789/0 conn(0x7feb6c15fcc0 legacy=0x7feb6c0d4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:44.735 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.735+0000 7feb6be00640 1 -- 172.21.15.152:0/487476706 >> v1:172.21.15.152:6789/0 conn(0x7feb6c0b5480 legacy=0x7feb6c0cf240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:44.735 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.735+0000 7feb6be00640 1 -- 172.21.15.152:0/487476706 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb6c0ced20 con 0x7feb6c1603d0 2024-04-20T11:28:44.735 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.736+0000 7feb6be00640 1 --2- 172.21.15.152:0/487476706 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feb6c1603d0 0x7feb6c0b5eb0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7feb58009790 tx=0x7feb5802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e562263587f02964 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:44.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.736+0000 7feb6b400640 1 -- 172.21.15.152:0/487476706 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb58035020 con 0x7feb6c1603d0 2024-04-20T11:28:44.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.736+0000 7feb6b400640 1 -- 172.21.15.152:0/487476706 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb58031810 con 0x7feb6c1603d0 2024-04-20T11:28:44.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.736+0000 7feb6b400640 1 -- 172.21.15.152:0/487476706 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb58031be0 con 0x7feb6c1603d0 2024-04-20T11:28:44.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.737+0000 7feb72200640 1 -- 172.21.15.152:0/487476706 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feb6c1603d0 msgr2=0x7feb6c0b5eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:44.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.737+0000 7feb72200640 1 --2- 172.21.15.152:0/487476706 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feb6c1603d0 0x7feb6c0b5eb0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7feb58009790 tx=0x7feb5802b8b0 comp rx=0 tx=0).stop 2024-04-20T11:28:44.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.737+0000 7feb72200640 1 -- 172.21.15.152:0/487476706 shutdown_connections 2024-04-20T11:28:44.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.737+0000 7feb72200640 1 --2- 172.21.15.152:0/487476706 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feb6c1603d0 0x7feb6c0b5eb0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:44.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.737+0000 7feb72200640 1 -- 172.21.15.152:0/487476706 >> 172.21.15.152:0/487476706 conn(0x7feb6c08bfd0 msgr2=0x7feb6c0d4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:44.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.737+0000 7feb72200640 1 -- 172.21.15.152:0/487476706 shutdown_connections 2024-04-20T11:28:44.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.737+0000 7feb72200640 1 -- 172.21.15.152:0/487476706 wait complete. 2024-04-20T11:28:44.738 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.739+0000 7feb72200640 1 Processor -- start 2024-04-20T11:28:44.738 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.739+0000 7feb72200640 1 -- start start 2024-04-20T11:28:44.741 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.740+0000 7feb72200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feb6c0b5480 0x7feb6c156650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:44.741 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.740+0000 7feb72200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feb6c15fcc0 0x7feb6c0b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:44.741 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.740+0000 7feb72200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feb6c1603d0 0x7feb6c0b8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:44.741 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.740+0000 7feb72200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7feb6c0b6800 con 0x7feb6c15fcc0 2024-04-20T11:28:44.741 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.740+0000 7feb72200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7feb6c0b6680 con 0x7feb6c1603d0 2024-04-20T11:28:44.741 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.740+0000 7feb72200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7feb6c0cf950 con 0x7feb6c0b5480 2024-04-20T11:28:44.741 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.740+0000 7feb6be00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feb6c15fcc0 0x7feb6c0b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:44.741 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.740+0000 7feb6be00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feb6c15fcc0 0x7feb6c0b8ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:60186/0 (socket says 172.21.15.152:60186) 2024-04-20T11:28:44.741 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.740+0000 7feb6be00640 1 -- 172.21.15.152:0/3649123602 learned_addr learned my addr 172.21.15.152:0/3649123602 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:44.741 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.741+0000 7feb6be00640 1 -- 172.21.15.152:0/3649123602 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feb6c0b5480 msgr2=0x7feb6c156650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:44.741 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.741+0000 7feb70e00640 1 --2- 172.21.15.152:0/3649123602 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feb6c0b5480 0x7feb6c156650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:44.741 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.741+0000 7feb71800640 1 --2- 172.21.15.152:0/3649123602 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feb6c1603d0 0x7feb6c0b8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:44.742 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.741+0000 7feb6be00640 1 --2- 172.21.15.152:0/3649123602 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feb6c0b5480 0x7feb6c156650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:44.742 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.741+0000 7feb6be00640 1 -- 172.21.15.152:0/3649123602 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feb6c1603d0 msgr2=0x7feb6c0b8ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:44.742 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.741+0000 7feb6be00640 1 --2- 172.21.15.152:0/3649123602 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feb6c1603d0 0x7feb6c0b8ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:44.742 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.741+0000 7feb6be00640 1 -- 172.21.15.152:0/3649123602 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb6c0b9b50 con 0x7feb6c15fcc0 2024-04-20T11:28:44.742 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.741+0000 7feb6be00640 1 --2- 172.21.15.152:0/3649123602 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feb6c15fcc0 0x7feb6c0b8ab0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7feb580098b0 tx=0x7feb580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:44.743 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.742+0000 7feb68c00640 1 -- 172.21.15.152:0/3649123602 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb58041070 con 0x7feb6c15fcc0 2024-04-20T11:28:44.745 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.742+0000 7feb68c00640 1 -- 172.21.15.152:0/3649123602 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb5803ad00 con 0x7feb6c15fcc0 2024-04-20T11:28:44.745 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.742+0000 7feb68c00640 1 -- 172.21.15.152:0/3649123602 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb58035040 con 0x7feb6c15fcc0 2024-04-20T11:28:44.745 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.742+0000 7feb72200640 1 -- 172.21.15.152:0/3649123602 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb6c0cd930 con 0x7feb6c15fcc0 2024-04-20T11:28:44.745 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.742+0000 7feb72200640 1 -- 172.21.15.152:0/3649123602 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb6c0cdef0 con 0x7feb6c15fcc0 2024-04-20T11:28:44.745 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.743+0000 7feb72200640 1 -- 172.21.15.152:0/3649123602 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb6c075e60 con 0x7feb6c15fcc0 2024-04-20T11:28:44.746 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.744+0000 7feb68c00640 1 -- 172.21.15.152:0/3649123602 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7feb5804c060 con 0x7feb6c15fcc0 2024-04-20T11:28:44.746 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.745+0000 7feb68c00640 1 --2- 172.21.15.152:0/3649123602 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7feb4c06caa0 0x7feb4c06ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:44.746 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.745+0000 7feb68c00640 1 -- 172.21.15.152:0/3649123602 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7feb580adbf0 con 0x7feb6c15fcc0 2024-04-20T11:28:44.746 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.745+0000 7feb70e00640 1 --2- 172.21.15.152:0/3649123602 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7feb4c06caa0 0x7feb4c06ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:44.747 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.746+0000 7feb70e00640 1 --2- 172.21.15.152:0/3649123602 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7feb4c06caa0 0x7feb4c06ef60 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7feb60003d90 tx=0x7feb60009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:44.749 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.750+0000 7feb68c00640 1 -- 172.21.15.152:0/3649123602 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7feb5807bd00 con 0x7feb6c15fcc0 2024-04-20T11:28:44.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.989+0000 7feb72200640 1 -- 172.21.15.152:0/3649123602 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feb6c0ce0c0 con 0x7feb6c15fcc0 2024-04-20T11:28:44.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.989+0000 7feb68c00640 1 -- 172.21.15.152:0/3649123602 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5211 (secure 0 0 0) 0x7feb5807b600 con 0x7feb6c15fcc0 2024-04-20T11:28:44.989 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:28:44.989 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4643,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6834/1871483322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1871483322},{"type":"v1","addr":"172.21.15.175:6834","nonce":1871483322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4786,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/2579061164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6835","nonce":2579061164},{"type":"v1","addr":"172.21.15.152:6837","nonce":2579061164}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5208,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6836/3724129680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":3724129680},{"type":"v1","addr":"172.21.15.152:6836","nonce":3724129680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T11:28:17.902738+0000","modified":"2024-04-20T11:28:20.990798+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5223},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5223":{"gid":5223,"name":"b","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.175:6835/3367117968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6833","nonce":3367117968},{"type":"v1","addr":"172.21.15.175:6835","nonce":3367117968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5223,"qdb_cluster":[5223]},"id":2}]} 2024-04-20T11:28:44.989 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 16 2024-04-20T11:28:44.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.992+0000 7feb72200640 1 -- 172.21.15.152:0/3649123602 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7feb4c06caa0 msgr2=0x7feb4c06ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:44.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.992+0000 7feb72200640 1 --2- 172.21.15.152:0/3649123602 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7feb4c06caa0 0x7feb4c06ef60 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7feb60003d90 tx=0x7feb60009040 comp rx=0 tx=0).stop 2024-04-20T11:28:44.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.992+0000 7feb72200640 1 -- 172.21.15.152:0/3649123602 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feb6c15fcc0 msgr2=0x7feb6c0b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:44.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.992+0000 7feb72200640 1 --2- 172.21.15.152:0/3649123602 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feb6c15fcc0 0x7feb6c0b8ab0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7feb580098b0 tx=0x7feb580046c0 comp rx=0 tx=0).stop 2024-04-20T11:28:44.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.992+0000 7feb72200640 1 -- 172.21.15.152:0/3649123602 shutdown_connections 2024-04-20T11:28:44.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.992+0000 7feb72200640 1 --2- 172.21.15.152:0/3649123602 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7feb4c06caa0 0x7feb4c06ef60 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:44.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.992+0000 7feb72200640 1 --2- 172.21.15.152:0/3649123602 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feb6c1603d0 0x7feb6c0b8ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:44.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.992+0000 7feb72200640 1 --2- 172.21.15.152:0/3649123602 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feb6c15fcc0 0x7feb6c0b8ab0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:44.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.992+0000 7feb72200640 1 --2- 172.21.15.152:0/3649123602 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feb6c0b5480 0x7feb6c156650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:44.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.992+0000 7feb72200640 1 -- 172.21.15.152:0/3649123602 >> 172.21.15.152:0/3649123602 conn(0x7feb6c08bfd0 msgr2=0x7feb6c0d15f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:44.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.993+0000 7feb72200640 1 -- 172.21.15.152:0/3649123602 shutdown_connections 2024-04-20T11:28:44.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:44.993+0000 7feb72200640 1 -- 172.21.15.152:0/3649123602 wait complete. 2024-04-20T11:28:45.011 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:28:45.210 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.211+0000 7f9613e00640 1 Processor -- start 2024-04-20T11:28:45.211 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.211+0000 7f9613e00640 1 -- start start 2024-04-20T11:28:45.212 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.213+0000 7f9613e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f96140723e0 0x7f961415d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:45.212 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.213+0000 7f9613e00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96140ce8c0 con 0x7f961415d1e0 2024-04-20T11:28:45.212 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.213+0000 7f9613e00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96141600b0 con 0x7f9614160b90 2024-04-20T11:28:45.212 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.213+0000 7f9613e00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f9614160390 con 0x7f96140723e0 2024-04-20T11:28:45.213 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.213+0000 7f9612a00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f961415d1e0 0x7f96140cdfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:47132/0 (socket says 172.21.15.152:47132) 2024-04-20T11:28:45.213 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.213+0000 7f9612a00640 1 -- 172.21.15.152:0/920228095 learned_addr learned my addr 172.21.15.152:0/920228095 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:28:45.215 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.213+0000 7f9612000640 1 -- 172.21.15.152:0/920228095 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2426756176 0 0) 0x7f96140ce8c0 con 0x7f961415d1e0 2024-04-20T11:28:45.215 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.213+0000 7f9613400640 1 --2- 172.21.15.152:0/920228095 >> v2:172.21.15.175:3300/0 conn(0x7f96140723e0 0x7f961415d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:45.215 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.213+0000 7f9612000640 1 -- 172.21.15.152:0/920228095 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f95f8003520 con 0x7f961415d1e0 2024-04-20T11:28:45.215 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.214+0000 7f9612000640 1 -- 172.21.15.152:0/920228095 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2244338880 0 0) 0x7f95f8003520 con 0x7f961415d1e0 2024-04-20T11:28:45.215 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.214+0000 7f9612000640 1 -- 172.21.15.152:0/920228095 >> v2:172.21.15.175:3300/0 conn(0x7f96140723e0 msgr2=0x7f961415d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:45.215 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.214+0000 7f9612000640 1 --2- 172.21.15.152:0/920228095 >> v2:172.21.15.175:3300/0 conn(0x7f96140723e0 0x7f961415d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:45.215 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.214+0000 7f9612000640 1 -- 172.21.15.152:0/920228095 >> v1:172.21.15.175:6789/0 conn(0x7f9614160b90 legacy=0x7f9614071cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:45.215 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.214+0000 7f9612000640 1 -- 172.21.15.152:0/920228095 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9614160670 con 0x7f961415d1e0 2024-04-20T11:28:45.215 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.214+0000 7f9612000640 1 -- 172.21.15.152:0/920228095 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f9604001670 con 0x7f961415d1e0 2024-04-20T11:28:45.215 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.214+0000 7f9612000640 1 -- 172.21.15.152:0/920228095 >> v1:172.21.15.152:6789/0 conn(0x7f961415d1e0 legacy=0x7f96140cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:45.215 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.214+0000 7f9612000640 1 --2- 172.21.15.152:0/920228095 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f95f80039b0 0x7f95f8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:45.216 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.214+0000 7f9612000640 1 --2- 172.21.15.152:0/920228095 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f95f8004800 0x7f95f8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:45.216 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.214+0000 7f9612000640 1 --2- 172.21.15.152:0/920228095 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f95f80051c0 0x7f95f800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:45.216 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.214+0000 7f9612000640 1 -- 172.21.15.152:0/920228095 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f9614160390 con 0x7f95f8004800 2024-04-20T11:28:45.216 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.214+0000 7f9612000640 1 -- 172.21.15.152:0/920228095 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f95f800c110 con 0x7f95f80051c0 2024-04-20T11:28:45.216 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.214+0000 7f9612000640 1 -- 172.21.15.152:0/920228095 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f95f800c310 con 0x7f95f80039b0 2024-04-20T11:28:45.216 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.214+0000 7f960be00640 1 --2- 172.21.15.152:0/920228095 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f95f8004800 0x7f95f8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:45.216 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.215+0000 7f9613400640 1 --2- 172.21.15.152:0/920228095 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f95f80051c0 0x7f95f800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:45.216 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.215+0000 7f9612a00640 1 --2- 172.21.15.152:0/920228095 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f95f80039b0 0x7f95f8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:45.216 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.215+0000 7f960be00640 1 -- 172.21.15.152:0/920228095 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f95f80039b0 msgr2=0x7f95f8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:45.216 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.215+0000 7f960be00640 1 --2- 172.21.15.152:0/920228095 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f95f80039b0 0x7f95f8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:45.216 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.215+0000 7f960be00640 1 -- 172.21.15.152:0/920228095 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f95f80051c0 msgr2=0x7f95f800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:45.216 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.215+0000 7f960be00640 1 --2- 172.21.15.152:0/920228095 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f95f80051c0 0x7f95f800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:45.216 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.215+0000 7f960be00640 1 -- 172.21.15.152:0/920228095 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95f8003520 con 0x7f95f8004800 2024-04-20T11:28:45.216 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.215+0000 7f9613400640 1 -- 172.21.15.152:0/920228095 reap_dead start 2024-04-20T11:28:45.217 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.216+0000 7f960be00640 1 --2- 172.21.15.152:0/920228095 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f95f8004800 0x7f95f8004c80 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f95fc001750 tx=0x7f95f8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f2057ef0c5a61e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:45.218 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.216+0000 7f9612000640 1 -- 172.21.15.152:0/920228095 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95fc001370 con 0x7f95f8004800 2024-04-20T11:28:45.218 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.216+0000 7f9612000640 1 -- 172.21.15.152:0/920228095 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95fc0052d0 con 0x7f95f8004800 2024-04-20T11:28:45.218 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.216+0000 7f9612000640 1 -- 172.21.15.152:0/920228095 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95fc003830 con 0x7f95f8004800 2024-04-20T11:28:45.218 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.217+0000 7f9613e00640 1 -- 172.21.15.152:0/920228095 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f95f8004800 msgr2=0x7f95f8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:45.218 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.217+0000 7f9613e00640 1 --2- 172.21.15.152:0/920228095 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f95f8004800 0x7f95f8004c80 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f95fc001750 tx=0x7f95f8005830 comp rx=0 tx=0).stop 2024-04-20T11:28:45.218 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.217+0000 7f9613e00640 1 -- 172.21.15.152:0/920228095 shutdown_connections 2024-04-20T11:28:45.218 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.217+0000 7f9613e00640 1 --2- 172.21.15.152:0/920228095 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f95f8004800 0x7f95f8004c80 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:45.218 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.217+0000 7f9613e00640 1 -- 172.21.15.152:0/920228095 >> 172.21.15.152:0/920228095 conn(0x7f9614067ae0 msgr2=0x7f961406b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:45.218 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.217+0000 7f9613e00640 1 -- 172.21.15.152:0/920228095 shutdown_connections 2024-04-20T11:28:45.218 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.217+0000 7f9613e00640 1 -- 172.21.15.152:0/920228095 wait complete. 2024-04-20T11:28:45.220 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.219+0000 7f9613e00640 1 Processor -- start 2024-04-20T11:28:45.220 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.219+0000 7f9613e00640 1 -- start start 2024-04-20T11:28:45.220 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.220+0000 7f9613e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f95f8004800 0x7f96141f3c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:45.220 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.220+0000 7f9613e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9614159360 0x7f96141597e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:45.220 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.220+0000 7f9613e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9614157220 0x7f96141576a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:45.220 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.220+0000 7f9613e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f961415d740 con 0x7f9614159360 2024-04-20T11:28:45.220 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.220+0000 7f9613e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f961415d5c0 con 0x7f95f8004800 2024-04-20T11:28:45.220 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.220+0000 7f9613e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f961415d8c0 con 0x7f9614157220 2024-04-20T11:28:45.221 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.220+0000 7f960be00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9614159360 0x7f96141597e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:45.221 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.220+0000 7f960be00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9614159360 0x7f96141597e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:60196/0 (socket says 172.21.15.152:60196) 2024-04-20T11:28:45.221 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.220+0000 7f960be00640 1 -- 172.21.15.152:0/4088246443 learned_addr learned my addr 172.21.15.152:0/4088246443 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:45.221 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.220+0000 7f9613400640 1 --2- 172.21.15.152:0/4088246443 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9614157220 0x7f96141576a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:45.221 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.220+0000 7f960be00640 1 -- 172.21.15.152:0/4088246443 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9614157220 msgr2=0x7f96141576a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:45.221 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.220+0000 7f960be00640 1 --2- 172.21.15.152:0/4088246443 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9614157220 0x7f96141576a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:45.221 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.220+0000 7f9612a00640 1 --2- 172.21.15.152:0/4088246443 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f95f8004800 0x7f96141f3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:45.221 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.220+0000 7f960be00640 1 -- 172.21.15.152:0/4088246443 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f95f8004800 msgr2=0x7f96141f3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:45.221 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.220+0000 7f960be00640 1 --2- 172.21.15.152:0/4088246443 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f95f8004800 0x7f96141f3c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:45.221 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.220+0000 7f960be00640 1 -- 172.21.15.152:0/4088246443 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9614157fb0 con 0x7f9614159360 2024-04-20T11:28:45.221 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.220+0000 7f9612a00640 1 --2- 172.21.15.152:0/4088246443 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f95f8004800 0x7f96141f3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:28:45.221 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.221+0000 7f960be00640 1 --2- 172.21.15.152:0/4088246443 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9614159360 0x7f96141597e0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f95fc000c00 tx=0x7f95fc001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:45.222 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.221+0000 7f960aa00640 1 -- 172.21.15.152:0/4088246443 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95fc041020 con 0x7f9614159360 2024-04-20T11:28:45.222 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.221+0000 7f960aa00640 1 -- 172.21.15.152:0/4088246443 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95fc005480 con 0x7f9614159360 2024-04-20T11:28:45.222 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.221+0000 7f9613e00640 1 -- 172.21.15.152:0/4088246443 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9614158290 con 0x7f9614159360 2024-04-20T11:28:45.222 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.221+0000 7f960aa00640 1 -- 172.21.15.152:0/4088246443 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95fc007a30 con 0x7f9614159360 2024-04-20T11:28:45.222 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.221+0000 7f9613e00640 1 -- 172.21.15.152:0/4088246443 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9614158850 con 0x7f9614159360 2024-04-20T11:28:45.222 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.222+0000 7f9613e00640 1 -- 172.21.15.152:0/4088246443 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9614075e60 con 0x7f9614159360 2024-04-20T11:28:45.223 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.223+0000 7f960aa00640 1 -- 172.21.15.152:0/4088246443 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f95fc044020 con 0x7f9614159360 2024-04-20T11:28:45.224 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.224+0000 7f960aa00640 1 --2- 172.21.15.152:0/4088246443 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f95f006cad0 0x7f95f006ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:45.224 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.224+0000 7f960aa00640 1 -- 172.21.15.152:0/4088246443 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f95fc0ada90 con 0x7f9614159360 2024-04-20T11:28:45.224 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.224+0000 7f9612a00640 1 --2- 172.21.15.152:0/4088246443 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f95f006cad0 0x7f95f006ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:45.224 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.225+0000 7f9612a00640 1 --2- 172.21.15.152:0/4088246443 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f95f006cad0 0x7f95f006ef90 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9604001050 tx=0x7f9604001d70 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:45.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.228+0000 7f960aa00640 1 -- 172.21.15.152:0/4088246443 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f95fc04b030 con 0x7f9614159360 2024-04-20T11:28:45.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.470+0000 7f9613e00640 1 -- 172.21.15.152:0/4088246443 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9614072c90 con 0x7f9614159360 2024-04-20T11:28:45.471 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.471+0000 7f960aa00640 1 -- 172.21.15.152:0/4088246443 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5211 (secure 0 0 0) 0x7f95fc0303c0 con 0x7f9614159360 2024-04-20T11:28:45.471 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:28:45.471 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4643,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6834/1871483322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1871483322},{"type":"v1","addr":"172.21.15.175:6834","nonce":1871483322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4786,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/2579061164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6835","nonce":2579061164},{"type":"v1","addr":"172.21.15.152:6837","nonce":2579061164}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5208,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6836/3724129680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":3724129680},{"type":"v1","addr":"172.21.15.152:6836","nonce":3724129680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T11:28:17.902738+0000","modified":"2024-04-20T11:28:20.990798+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5223},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5223":{"gid":5223,"name":"b","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.175:6835/3367117968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6833","nonce":3367117968},{"type":"v1","addr":"172.21.15.175:6835","nonce":3367117968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5223,"qdb_cluster":[5223]},"id":2}]} 2024-04-20T11:28:45.472 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 16 2024-04-20T11:28:45.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.474+0000 7f9613e00640 1 -- 172.21.15.152:0/4088246443 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f95f006cad0 msgr2=0x7f95f006ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:45.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.474+0000 7f9613e00640 1 --2- 172.21.15.152:0/4088246443 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f95f006cad0 0x7f95f006ef90 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9604001050 tx=0x7f9604001d70 comp rx=0 tx=0).stop 2024-04-20T11:28:45.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.474+0000 7f9613e00640 1 -- 172.21.15.152:0/4088246443 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9614159360 msgr2=0x7f96141597e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:45.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.474+0000 7f9613e00640 1 --2- 172.21.15.152:0/4088246443 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9614159360 0x7f96141597e0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f95fc000c00 tx=0x7f95fc001120 comp rx=0 tx=0).stop 2024-04-20T11:28:45.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.474+0000 7f9613e00640 1 -- 172.21.15.152:0/4088246443 shutdown_connections 2024-04-20T11:28:45.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.474+0000 7f9613e00640 1 --2- 172.21.15.152:0/4088246443 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f95f006cad0 0x7f95f006ef90 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:45.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.474+0000 7f9613e00640 1 --2- 172.21.15.152:0/4088246443 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9614157220 0x7f96141576a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:45.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.474+0000 7f9613e00640 1 --2- 172.21.15.152:0/4088246443 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9614159360 0x7f96141597e0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:45.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.474+0000 7f9613e00640 1 --2- 172.21.15.152:0/4088246443 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f95f8004800 0x7f96141f3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:45.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.474+0000 7f9613e00640 1 -- 172.21.15.152:0/4088246443 >> 172.21.15.152:0/4088246443 conn(0x7f9614067ae0 msgr2=0x7f96140ccd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:45.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.475+0000 7f9613e00640 1 -- 172.21.15.152:0/4088246443 shutdown_connections 2024-04-20T11:28:45.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.475+0000 7f9613e00640 1 -- 172.21.15.152:0/4088246443 wait complete. 2024-04-20T11:28:45.492 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:28:45.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.696+0000 7f8146400640 1 Processor -- start 2024-04-20T11:28:45.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.697+0000 7f8146400640 1 -- start start 2024-04-20T11:28:45.697 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.698+0000 7f8146400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f81400ba520 0x7f81400d28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:45.698 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.698+0000 7f8146400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f81400d2e30 0x7f81400d1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:45.698 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.698+0000 7f8146400640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81400b80b0 con 0x7f814015eb80 2024-04-20T11:28:45.698 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.698+0000 7f8146400640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f81400b8330 con 0x7f81400d2e30 2024-04-20T11:28:45.698 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.698+0000 7f8146400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f81400b8610 con 0x7f81400ba520 2024-04-20T11:28:45.698 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.698+0000 7f813f400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f814015eb80 0x7f81400b79a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:47136/0 (socket says 172.21.15.152:47136) 2024-04-20T11:28:45.698 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.698+0000 7f813f400640 1 -- 172.21.15.152:0/3243933785 learned_addr learned my addr 172.21.15.152:0/3243933785 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:28:45.699 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.698+0000 7f813fe00640 1 --2- 172.21.15.152:0/3243933785 >> v2:172.21.15.152:3300/0 conn(0x7f81400d2e30 0x7f81400d1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:45.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.699+0000 7f813ea00640 1 --2- 172.21.15.152:0/3243933785 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f81400ba520 0x7f81400d28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:45.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.699+0000 7f813e000640 1 -- 172.21.15.152:0/3243933785 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1407377365 0 0) 0x7f8130001490 con 0x7f814015eb80 2024-04-20T11:28:45.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.699+0000 7f813e000640 1 -- 172.21.15.152:0/3243933785 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8124003520 con 0x7f814015eb80 2024-04-20T11:28:45.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.699+0000 7f813fe00640 1 -- 172.21.15.152:0/3243933785 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f81400ba520 msgr2=0x7f81400d28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:45.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.699+0000 7f813fe00640 1 --2- 172.21.15.152:0/3243933785 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f81400ba520 0x7f81400d28f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:45.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.699+0000 7f813fe00640 1 -- 172.21.15.152:0/3243933785 >> v1:172.21.15.152:6789/0 conn(0x7f814015eb80 legacy=0x7f81400b79a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:45.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.699+0000 7f813fe00640 1 -- 172.21.15.152:0/3243933785 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81400b88f0 con 0x7f81400d2e30 2024-04-20T11:28:45.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.699+0000 7f813ea00640 1 --2- 172.21.15.152:0/3243933785 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f81400ba520 0x7f81400d28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:28:45.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.700+0000 7f813fe00640 1 --2- 172.21.15.152:0/3243933785 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f81400d2e30 0x7f81400d1810 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f81380097a0 tx=0x7f813802b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26050db31379390a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:45.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.700+0000 7f813e000640 1 -- 172.21.15.152:0/3243933785 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8138031780 con 0x7f81400d2e30 2024-04-20T11:28:45.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.700+0000 7f813e000640 1 -- 172.21.15.152:0/3243933785 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8138037070 con 0x7f81400d2e30 2024-04-20T11:28:45.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.700+0000 7f813e000640 1 -- 172.21.15.152:0/3243933785 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8138031b20 con 0x7f81400d2e30 2024-04-20T11:28:45.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.701+0000 7f8146400640 1 -- 172.21.15.152:0/3243933785 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f81400d2e30 msgr2=0x7f81400d1810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:45.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.701+0000 7f8146400640 1 --2- 172.21.15.152:0/3243933785 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f81400d2e30 0x7f81400d1810 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f81380097a0 tx=0x7f813802b820 comp rx=0 tx=0).stop 2024-04-20T11:28:45.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.701+0000 7f8146400640 1 -- 172.21.15.152:0/3243933785 shutdown_connections 2024-04-20T11:28:45.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.701+0000 7f8146400640 1 --2- 172.21.15.152:0/3243933785 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f81400ba520 0x7f81400d28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:45.703 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.701+0000 7f8146400640 1 --2- 172.21.15.152:0/3243933785 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f81400d2e30 0x7f81400d1810 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:45.703 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.701+0000 7f8146400640 1 -- 172.21.15.152:0/3243933785 >> 172.21.15.152:0/3243933785 conn(0x7f8140067ae0 msgr2=0x7f8140067ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:45.703 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.701+0000 7f8146400640 1 -- 172.21.15.152:0/3243933785 shutdown_connections 2024-04-20T11:28:45.703 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.701+0000 7f8146400640 1 -- 172.21.15.152:0/3243933785 wait complete. 2024-04-20T11:28:45.706 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.703+0000 7f8146400640 1 Processor -- start 2024-04-20T11:28:45.706 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.703+0000 7f8146400640 1 -- start start 2024-04-20T11:28:45.706 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.704+0000 7f8146400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f81400ba520 0x7f8140155ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:45.706 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.704+0000 7f8146400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f81400d2e30 0x7f8140157d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:45.706 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.704+0000 7f8146400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f814015eb80 0x7f81401f3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:45.706 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.704+0000 7f8146400640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f81400d1ed0 con 0x7f81400d2e30 2024-04-20T11:28:45.706 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.704+0000 7f8146400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f81400d1d50 con 0x7f81400ba520 2024-04-20T11:28:45.706 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.704+0000 7f8146400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f81400d2050 con 0x7f814015eb80 2024-04-20T11:28:45.707 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.704+0000 7f813ea00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f81400d2e30 0x7f8140157d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:45.707 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.704+0000 7f813ea00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f81400d2e30 0x7f8140157d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:60212/0 (socket says 172.21.15.152:60212) 2024-04-20T11:28:45.707 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.704+0000 7f813ea00640 1 -- 172.21.15.152:0/463435858 learned_addr learned my addr 172.21.15.152:0/463435858 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:45.707 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.705+0000 7f813fe00640 1 --2- 172.21.15.152:0/463435858 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f814015eb80 0x7f81401f3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:45.707 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.705+0000 7f813f400640 1 --2- 172.21.15.152:0/463435858 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f81400ba520 0x7f8140155ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:45.707 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.705+0000 7f813ea00640 1 -- 172.21.15.152:0/463435858 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f814015eb80 msgr2=0x7f81401f3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:45.707 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.705+0000 7f813ea00640 1 --2- 172.21.15.152:0/463435858 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f814015eb80 0x7f81401f3c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:45.707 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.705+0000 7f813ea00640 1 -- 172.21.15.152:0/463435858 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f81400ba520 msgr2=0x7f8140155ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:45.707 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.705+0000 7f813ea00640 1 --2- 172.21.15.152:0/463435858 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f81400ba520 0x7f8140155ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:45.707 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.705+0000 7f813ea00640 1 -- 172.21.15.152:0/463435858 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81401561e0 con 0x7f81400d2e30 2024-04-20T11:28:45.707 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.705+0000 7f813fe00640 1 --2- 172.21.15.152:0/463435858 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f814015eb80 0x7f81401f3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:28:45.707 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.705+0000 7f813ea00640 1 --2- 172.21.15.152:0/463435858 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f81400d2e30 0x7f8140157d80 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f812800c820 tx=0x7f812800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:45.707 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.705+0000 7f8137400640 1 -- 172.21.15.152:0/463435858 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8128015070 con 0x7f81400d2e30 2024-04-20T11:28:45.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.705+0000 7f8137400640 1 -- 172.21.15.152:0/463435858 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8128007a30 con 0x7f81400d2e30 2024-04-20T11:28:45.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.705+0000 7f8137400640 1 -- 172.21.15.152:0/463435858 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8128007de0 con 0x7f81400d2e30 2024-04-20T11:28:45.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.705+0000 7f8146400640 1 -- 172.21.15.152:0/463435858 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81401f55b0 con 0x7f81400d2e30 2024-04-20T11:28:45.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.705+0000 7f8146400640 1 -- 172.21.15.152:0/463435858 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8140159750 con 0x7f81400d2e30 2024-04-20T11:28:45.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.706+0000 7f8146400640 1 -- 172.21.15.152:0/463435858 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8140159280 con 0x7f81400d2e30 2024-04-20T11:28:45.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.707+0000 7f8137400640 1 -- 172.21.15.152:0/463435858 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f8128018070 con 0x7f81400d2e30 2024-04-20T11:28:45.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.708+0000 7f8137400640 1 --2- 172.21.15.152:0/463435858 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f811c06caa0 0x7f811c06ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:45.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.708+0000 7f8137400640 1 -- 172.21.15.152:0/463435858 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f812808be40 con 0x7f81400d2e30 2024-04-20T11:28:45.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.709+0000 7f813f400640 1 --2- 172.21.15.152:0/463435858 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f811c06caa0 0x7f811c06ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:45.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.709+0000 7f813f400640 1 --2- 172.21.15.152:0/463435858 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f811c06caa0 0x7f811c06ef60 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8130001630 tx=0x7f8130009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:45.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.713+0000 7f8137400640 1 -- 172.21.15.152:0/463435858 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f8128059ed0 con 0x7f81400d2e30 2024-04-20T11:28:45.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.912+0000 7f8146400640 1 -- 172.21.15.152:0/463435858 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f81400689f0 con 0x7f81400d2e30 2024-04-20T11:28:45.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.913+0000 7f8137400640 1 -- 172.21.15.152:0/463435858 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) ==== 74+0+16729 (secure 0 0 0) 0x7f81280597d0 con 0x7f81400d2e30 2024-04-20T11:28:45.914 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:28:45.914 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":34,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:28:19.478546+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:28:15.704670+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-20T11:28:16.891101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:28:45.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.917+0000 7f8146400640 1 -- 172.21.15.152:0/463435858 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f811c06caa0 msgr2=0x7f811c06ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:45.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.917+0000 7f8146400640 1 --2- 172.21.15.152:0/463435858 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f811c06caa0 0x7f811c06ef60 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8130001630 tx=0x7f8130009040 comp rx=0 tx=0).stop 2024-04-20T11:28:45.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.917+0000 7f8146400640 1 -- 172.21.15.152:0/463435858 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f81400d2e30 msgr2=0x7f8140157d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:45.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.917+0000 7f8146400640 1 --2- 172.21.15.152:0/463435858 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f81400d2e30 0x7f8140157d80 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f812800c820 tx=0x7f812800cce0 comp rx=0 tx=0).stop 2024-04-20T11:28:45.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.917+0000 7f8146400640 1 -- 172.21.15.152:0/463435858 shutdown_connections 2024-04-20T11:28:45.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.917+0000 7f8146400640 1 --2- 172.21.15.152:0/463435858 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f811c06caa0 0x7f811c06ef60 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:45.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.917+0000 7f8146400640 1 --2- 172.21.15.152:0/463435858 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f814015eb80 0x7f81401f3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:45.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.917+0000 7f8146400640 1 --2- 172.21.15.152:0/463435858 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f81400d2e30 0x7f8140157d80 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:45.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.917+0000 7f8146400640 1 --2- 172.21.15.152:0/463435858 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f81400ba520 0x7f8140155ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:45.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.917+0000 7f8146400640 1 -- 172.21.15.152:0/463435858 >> 172.21.15.152:0/463435858 conn(0x7f8140067ae0 msgr2=0x7f81400d4d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:45.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.917+0000 7f8146400640 1 -- 172.21.15.152:0/463435858 shutdown_connections 2024-04-20T11:28:45.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:45.917+0000 7f8146400640 1 -- 172.21.15.152:0/463435858 wait complete. 2024-04-20T11:28:45.936 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-20T11:28:46.137 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.137+0000 7f1d72c00640 1 Processor -- start 2024-04-20T11:28:46.137 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.137+0000 7f1d72c00640 1 -- start start 2024-04-20T11:28:46.139 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.139+0000 7f1d72c00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f1d6c15ea40 0x7f1d6c0b46c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:46.139 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.139+0000 7f1d72c00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f1d6c09aa40 0x7f1d6c06b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:46.139 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.139+0000 7f1d72c00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d6c0b2bf0 con 0x7f1d6c06bec0 2024-04-20T11:28:46.139 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.139+0000 7f1d72c00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f1d6c15ee20 con 0x7f1d6c15ea40 2024-04-20T11:28:46.139 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.139+0000 7f1d72c00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f1d6c09ae90 con 0x7f1d6c09aa40 2024-04-20T11:28:46.139 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.139+0000 7f1d71800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f1d6c15ea40 0x7f1d6c0b46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:46.139 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.139+0000 7f1d71800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f1d6c15ea40 0x7f1d6c0b46c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:53974/0 (socket says 172.21.15.152:53974) 2024-04-20T11:28:46.139 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.139+0000 7f1d71800640 1 -- 172.21.15.152:0/1160714672 learned_addr learned my addr 172.21.15.152:0/1160714672 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:46.140 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.140+0000 7f1d71800640 1 -- 172.21.15.152:0/1160714672 >> v2:172.21.15.175:3300/0 conn(0x7f1d6c09aa40 msgr2=0x7f1d6c06b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:46.140 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.140+0000 7f1d71800640 1 --2- 172.21.15.152:0/1160714672 >> v2:172.21.15.175:3300/0 conn(0x7f1d6c09aa40 0x7f1d6c06b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:46.140 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.140+0000 7f1d71800640 1 -- 172.21.15.152:0/1160714672 >> v1:172.21.15.175:6789/0 conn(0x7f1d6c06bec0 legacy=0x7f1d6c0ae250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:46.140 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.140+0000 7f1d71800640 1 -- 172.21.15.152:0/1160714672 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d6c0b6280 con 0x7f1d6c15ea40 2024-04-20T11:28:46.140 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.141+0000 7f1d71800640 1 --2- 172.21.15.152:0/1160714672 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d6c15ea40 0x7f1d6c0b46c0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c073130 tx=0x7f1d6002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22335ae32a5b14ff server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:46.140 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.141+0000 7f1d6be00640 1 -- 172.21.15.152:0/1160714672 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d6002ca80 con 0x7f1d6c15ea40 2024-04-20T11:28:46.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.141+0000 7f1d6be00640 1 -- 172.21.15.152:0/1160714672 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d6002cc20 con 0x7f1d6c15ea40 2024-04-20T11:28:46.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.141+0000 7f1d6be00640 1 -- 172.21.15.152:0/1160714672 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d600343f0 con 0x7f1d6c15ea40 2024-04-20T11:28:46.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.142+0000 7f1d72c00640 1 -- 172.21.15.152:0/1160714672 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d6c15ea40 msgr2=0x7f1d6c0b46c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:46.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.142+0000 7f1d72c00640 1 --2- 172.21.15.152:0/1160714672 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d6c15ea40 0x7f1d6c0b46c0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c073130 tx=0x7f1d6002bee0 comp rx=0 tx=0).stop 2024-04-20T11:28:46.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.142+0000 7f1d72c00640 1 -- 172.21.15.152:0/1160714672 shutdown_connections 2024-04-20T11:28:46.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.142+0000 7f1d72c00640 1 --2- 172.21.15.152:0/1160714672 >> v2:172.21.15.175:3300/0 conn(0x7f1d6c09aa40 0x7f1d6c06b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:46.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.142+0000 7f1d72c00640 1 --2- 172.21.15.152:0/1160714672 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d6c15ea40 0x7f1d6c0b46c0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:46.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.142+0000 7f1d72c00640 1 -- 172.21.15.152:0/1160714672 >> 172.21.15.152:0/1160714672 conn(0x7f1d6c0731a0 msgr2=0x7f1d6c0b2f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:46.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.142+0000 7f1d72c00640 1 -- 172.21.15.152:0/1160714672 shutdown_connections 2024-04-20T11:28:46.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.142+0000 7f1d72c00640 1 -- 172.21.15.152:0/1160714672 wait complete. 2024-04-20T11:28:46.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.144+0000 7f1d72c00640 1 Processor -- start 2024-04-20T11:28:46.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.144+0000 7f1d72c00640 1 -- start start 2024-04-20T11:28:46.146 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.146+0000 7f1d72c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d6c06bec0 0x7f1d6c142b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:46.146 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.146+0000 7f1d72c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d6c09aa40 0x7f1d6c145540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:46.146 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.146+0000 7f1d72c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1d6c15ea40 0x7f1d6c145a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:46.146 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.146+0000 7f1d72c00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f1d6c0b5f50 con 0x7f1d6c09aa40 2024-04-20T11:28:46.146 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.146+0000 7f1d72c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1d6c06c2f0 con 0x7f1d6c06bec0 2024-04-20T11:28:46.146 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.146+0000 7f1d72c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1d6c06c670 con 0x7f1d6c15ea40 2024-04-20T11:28:46.146 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.146+0000 7f1d70e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d6c09aa40 0x7f1d6c145540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:46.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.146+0000 7f1d70e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d6c09aa40 0x7f1d6c145540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:53976/0 (socket says 172.21.15.152:53976) 2024-04-20T11:28:46.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.146+0000 7f1d70e00640 1 -- 172.21.15.152:0/2484658107 learned_addr learned my addr 172.21.15.152:0/2484658107 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:46.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.146+0000 7f1d71800640 1 --2- 172.21.15.152:0/2484658107 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d6c06bec0 0x7f1d6c142b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:46.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.146+0000 7f1d72200640 1 --2- 172.21.15.152:0/2484658107 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1d6c15ea40 0x7f1d6c145a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:46.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.146+0000 7f1d70e00640 1 -- 172.21.15.152:0/2484658107 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1d6c15ea40 msgr2=0x7f1d6c145a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:46.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.146+0000 7f1d70e00640 1 --2- 172.21.15.152:0/2484658107 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1d6c15ea40 0x7f1d6c145a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:46.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.146+0000 7f1d70e00640 1 -- 172.21.15.152:0/2484658107 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d6c06bec0 msgr2=0x7f1d6c142b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:46.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.146+0000 7f1d70e00640 1 --2- 172.21.15.152:0/2484658107 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d6c06bec0 0x7f1d6c142b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:46.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.146+0000 7f1d70e00640 1 -- 172.21.15.152:0/2484658107 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d6c1452a0 con 0x7f1d6c09aa40 2024-04-20T11:28:46.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.146+0000 7f1d71800640 1 --2- 172.21.15.152:0/2484658107 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d6c06bec0 0x7f1d6c142b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:28:46.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.147+0000 7f1d70e00640 1 --2- 172.21.15.152:0/2484658107 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d6c09aa40 0x7f1d6c145540 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f1d5800b4a0 tx=0x7f1d5800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:46.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.147+0000 7f1d69600640 1 -- 172.21.15.152:0/2484658107 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d58014070 con 0x7f1d6c09aa40 2024-04-20T11:28:46.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.147+0000 7f1d69600640 1 -- 172.21.15.152:0/2484658107 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d58010610 con 0x7f1d6c09aa40 2024-04-20T11:28:46.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.147+0000 7f1d69600640 1 -- 172.21.15.152:0/2484658107 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d58010a00 con 0x7f1d6c09aa40 2024-04-20T11:28:46.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.147+0000 7f1d72c00640 1 -- 172.21.15.152:0/2484658107 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d6c144200 con 0x7f1d6c09aa40 2024-04-20T11:28:46.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.147+0000 7f1d72c00640 1 -- 172.21.15.152:0/2484658107 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d6c1447c0 con 0x7f1d6c09aa40 2024-04-20T11:28:46.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.149+0000 7f1d72c00640 1 -- 172.21.15.152:0/2484658107 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d6c162200 con 0x7f1d6c09aa40 2024-04-20T11:28:46.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.149+0000 7f1d69600640 1 -- 172.21.15.152:0/2484658107 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f1d58017070 con 0x7f1d6c09aa40 2024-04-20T11:28:46.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.150+0000 7f1d69600640 1 --2- 172.21.15.152:0/2484658107 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f1d4c06caa0 0x7f1d4c06ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:46.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.150+0000 7f1d69600640 1 -- 172.21.15.152:0/2484658107 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f1d5808b8f0 con 0x7f1d6c09aa40 2024-04-20T11:28:46.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.151+0000 7f1d71800640 1 --2- 172.21.15.152:0/2484658107 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f1d4c06caa0 0x7f1d4c06ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:46.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.152+0000 7f1d71800640 1 --2- 172.21.15.152:0/2484658107 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f1d4c06caa0 0x7f1d4c06ef60 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1d60035000 tx=0x7f1d60030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:46.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.155+0000 7f1d69600640 1 -- 172.21.15.152:0/2484658107 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f1d6c162200 con 0x7f1d6c09aa40 2024-04-20T11:28:46.398 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.398+0000 7f1d72c00640 1 -- 172.21.15.152:0/2484658107 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f1d6c075e60 con 0x7f1d6c09aa40 2024-04-20T11:28:46.399 INFO:tasks.ceph.mon.a.smithi152.stderr:2024-04-20T11:28:46.400+0000 7f8229800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-20T11:28:46.403 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.403+0000 7f1d69600640 1 -- 172.21.15.152:0/2484658107 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v17) ==== 161+0+0 (secure 0 0 0) 0x7f1d58059aa0 con 0x7f1d6c09aa40 2024-04-20T11:28:46.403 INFO:teuthology.orchestra.run.smithi152.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-20T11:28:46.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.407+0000 7f1d72c00640 1 -- 172.21.15.152:0/2484658107 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f1d4c06caa0 msgr2=0x7f1d4c06ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:46.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.407+0000 7f1d72c00640 1 --2- 172.21.15.152:0/2484658107 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f1d4c06caa0 0x7f1d4c06ef60 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1d60035000 tx=0x7f1d60030000 comp rx=0 tx=0).stop 2024-04-20T11:28:46.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.407+0000 7f1d72c00640 1 -- 172.21.15.152:0/2484658107 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d6c09aa40 msgr2=0x7f1d6c145540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:46.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.407+0000 7f1d72c00640 1 --2- 172.21.15.152:0/2484658107 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d6c09aa40 0x7f1d6c145540 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f1d5800b4a0 tx=0x7f1d5800b960 comp rx=0 tx=0).stop 2024-04-20T11:28:46.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.407+0000 7f1d72c00640 1 -- 172.21.15.152:0/2484658107 shutdown_connections 2024-04-20T11:28:46.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.407+0000 7f1d72c00640 1 --2- 172.21.15.152:0/2484658107 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f1d4c06caa0 0x7f1d4c06ef60 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:46.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.407+0000 7f1d72c00640 1 --2- 172.21.15.152:0/2484658107 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1d6c15ea40 0x7f1d6c145a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:46.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.407+0000 7f1d72c00640 1 --2- 172.21.15.152:0/2484658107 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d6c09aa40 0x7f1d6c145540 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:46.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.407+0000 7f1d72c00640 1 --2- 172.21.15.152:0/2484658107 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d6c06bec0 0x7f1d6c142b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:46.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.407+0000 7f1d72c00640 1 -- 172.21.15.152:0/2484658107 >> 172.21.15.152:0/2484658107 conn(0x7f1d6c0731a0 msgr2=0x7f1d6c074a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:46.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.407+0000 7f1d72c00640 1 -- 172.21.15.152:0/2484658107 shutdown_connections 2024-04-20T11:28:46.408 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.407+0000 7f1d72c00640 1 -- 172.21.15.152:0/2484658107 wait complete. 2024-04-20T11:28:46.426 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-20T11:28:46.523 INFO:tasks.ceph.mds.b.smithi175.stdout:starting mds.b at 2024-04-20T11:28:46.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.629+0000 7ffa84a00640 1 Processor -- start 2024-04-20T11:28:46.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.629+0000 7ffa84a00640 1 -- start start 2024-04-20T11:28:46.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.631+0000 7ffa84a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7ffa80160220 0x7ffa801606b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:46.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.631+0000 7ffa84a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa80141450 0x7ffa801418e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:46.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.631+0000 7ffa84a00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa8006fb40 con 0x7ffa80071cd0 2024-04-20T11:28:46.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.631+0000 7ffa84a00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7ffa8013f8e0 con 0x7ffa80160220 2024-04-20T11:28:46.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.631+0000 7ffa84a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ffa80160f90 con 0x7ffa80141450 2024-04-20T11:28:46.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.631+0000 7ffa7de00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7ffa80160220 0x7ffa801606b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:46.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.631+0000 7ffa7de00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7ffa80160220 0x7ffa801606b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:53978/0 (socket says 172.21.15.152:53978) 2024-04-20T11:28:46.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.631+0000 7ffa7de00640 1 -- 172.21.15.152:0/2457873821 learned_addr learned my addr 172.21.15.152:0/2457873821 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:46.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.631+0000 7ffa7f200640 1 --2- 172.21.15.152:0/2457873821 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa80141450 0x7ffa801418e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:46.632 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.632+0000 7ffa7de00640 1 -- 172.21.15.152:0/2457873821 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa80141450 msgr2=0x7ffa801418e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:46.632 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.632+0000 7ffa7de00640 1 --2- 172.21.15.152:0/2457873821 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa80141450 0x7ffa801418e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:46.632 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.632+0000 7ffa7de00640 1 -- 172.21.15.152:0/2457873821 >> v1:172.21.15.175:6789/0 conn(0x7ffa80071cd0 legacy=0x7ffa800720b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:46.632 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.632+0000 7ffa7de00640 1 -- 172.21.15.152:0/2457873821 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa8006dd90 con 0x7ffa80160220 2024-04-20T11:28:46.632 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.632+0000 7ffa7f200640 1 --2- 172.21.15.152:0/2457873821 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa80141450 0x7ffa801418e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:28:46.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.633+0000 7ffa7de00640 1 --2- 172.21.15.152:0/2457873821 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffa80160220 0x7ffa801606b0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c0098a0 tx=0x7ffa6c02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aebcb9f115be849 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:46.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.633+0000 7ffa7d400640 1 -- 172.21.15.152:0/2457873821 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa6c0317c0 con 0x7ffa80160220 2024-04-20T11:28:46.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.633+0000 7ffa7d400640 1 -- 172.21.15.152:0/2457873821 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa6c037070 con 0x7ffa80160220 2024-04-20T11:28:46.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.634+0000 7ffa7d400640 1 -- 172.21.15.152:0/2457873821 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa6c031b80 con 0x7ffa80160220 2024-04-20T11:28:46.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.634+0000 7ffa84a00640 1 -- 172.21.15.152:0/2457873821 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffa80160220 msgr2=0x7ffa801606b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:46.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.634+0000 7ffa84a00640 1 --2- 172.21.15.152:0/2457873821 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffa80160220 0x7ffa801606b0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c0098a0 tx=0x7ffa6c02b860 comp rx=0 tx=0).stop 2024-04-20T11:28:46.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.634+0000 7ffa84a00640 1 -- 172.21.15.152:0/2457873821 shutdown_connections 2024-04-20T11:28:46.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.634+0000 7ffa84a00640 1 --2- 172.21.15.152:0/2457873821 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa80141450 0x7ffa801418e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:46.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.634+0000 7ffa84a00640 1 --2- 172.21.15.152:0/2457873821 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffa80160220 0x7ffa801606b0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:46.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.634+0000 7ffa84a00640 1 -- 172.21.15.152:0/2457873821 >> 172.21.15.152:0/2457873821 conn(0x7ffa80067ae0 msgr2=0x7ffa8006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:46.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.634+0000 7ffa84a00640 1 -- 172.21.15.152:0/2457873821 shutdown_connections 2024-04-20T11:28:46.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.634+0000 7ffa84a00640 1 -- 172.21.15.152:0/2457873821 wait complete. 2024-04-20T11:28:46.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.636+0000 7ffa84a00640 1 Processor -- start 2024-04-20T11:28:46.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.636+0000 7ffa84a00640 1 -- start start 2024-04-20T11:28:46.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.637+0000 7ffa84a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa80071cd0 0x7ffa80137bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:46.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.637+0000 7ffa84a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffa80141450 0x7ffa801f5770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:46.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.637+0000 7ffa84a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa80160220 0x7ffa8013a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:46.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.637+0000 7ffa84a00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7ffa8006e630 con 0x7ffa80141450 2024-04-20T11:28:46.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.637+0000 7ffa84a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ffa8006e350 con 0x7ffa80071cd0 2024-04-20T11:28:46.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.637+0000 7ffa84a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ffa80160bf0 con 0x7ffa80160220 2024-04-20T11:28:46.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.637+0000 7ffa7de00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffa80141450 0x7ffa801f5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:46.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.638+0000 7ffa7de00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffa80141450 0x7ffa801f5770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:53988/0 (socket says 172.21.15.152:53988) 2024-04-20T11:28:46.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.638+0000 7ffa7de00640 1 -- 172.21.15.152:0/878366207 learned_addr learned my addr 172.21.15.152:0/878366207 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:46.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.638+0000 7ffa7e800640 1 --2- 172.21.15.152:0/878366207 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa80071cd0 0x7ffa80137bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:46.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.638+0000 7ffa7f200640 1 --2- 172.21.15.152:0/878366207 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa80160220 0x7ffa8013a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:46.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.638+0000 7ffa7de00640 1 -- 172.21.15.152:0/878366207 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa80160220 msgr2=0x7ffa8013a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:46.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.638+0000 7ffa7de00640 1 --2- 172.21.15.152:0/878366207 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa80160220 0x7ffa8013a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:46.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.638+0000 7ffa7de00640 1 -- 172.21.15.152:0/878366207 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa80071cd0 msgr2=0x7ffa80137bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:46.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.638+0000 7ffa7de00640 1 --2- 172.21.15.152:0/878366207 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa80071cd0 0x7ffa80137bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:46.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.638+0000 7ffa7de00640 1 -- 172.21.15.152:0/878366207 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa80138110 con 0x7ffa80141450 2024-04-20T11:28:46.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.638+0000 7ffa7e800640 1 --2- 172.21.15.152:0/878366207 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa80071cd0 0x7ffa80137bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:28:46.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.638+0000 7ffa7de00640 1 --2- 172.21.15.152:0/878366207 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffa80141450 0x7ffa801f5770 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c0090b0 tx=0x7ffa6c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:46.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.639+0000 7ffa76a00640 1 -- 172.21.15.152:0/878366207 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa6c032650 con 0x7ffa80141450 2024-04-20T11:28:46.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.639+0000 7ffa76a00640 1 -- 172.21.15.152:0/878366207 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa6c0327f0 con 0x7ffa80141450 2024-04-20T11:28:46.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.639+0000 7ffa76a00640 1 -- 172.21.15.152:0/878366207 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa6c032be0 con 0x7ffa80141450 2024-04-20T11:28:46.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.639+0000 7ffa84a00640 1 -- 172.21.15.152:0/878366207 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa80138520 con 0x7ffa80141450 2024-04-20T11:28:46.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.639+0000 7ffa84a00640 1 -- 172.21.15.152:0/878366207 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa80138ae0 con 0x7ffa80141450 2024-04-20T11:28:46.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.640+0000 7ffa84a00640 1 -- 172.21.15.152:0/878366207 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa80075e60 con 0x7ffa80141450 2024-04-20T11:28:46.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.641+0000 7ffa76a00640 1 -- 172.21.15.152:0/878366207 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7ffa6c04c060 con 0x7ffa80141450 2024-04-20T11:28:46.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.642+0000 7ffa76a00640 1 --2- 172.21.15.152:0/878366207 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ffa6006cad0 0x7ffa6006ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:46.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.642+0000 7ffa76a00640 1 -- 172.21.15.152:0/878366207 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5795+0+0 (secure 0 0 0) 0x7ffa6c0ad6d0 con 0x7ffa80141450 2024-04-20T11:28:46.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.642+0000 7ffa7e800640 1 --2- 172.21.15.152:0/878366207 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ffa6006cad0 0x7ffa6006ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:46.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.643+0000 7ffa7e800640 1 --2- 172.21.15.152:0/878366207 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ffa6006cad0 0x7ffa6006ef90 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ffa800cf110 tx=0x7ffa70009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:46.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.647+0000 7ffa76a00640 1 -- 172.21.15.152:0/878366207 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ffa6c040070 con 0x7ffa80141450 2024-04-20T11:28:46.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:46.895+0000 7ffa84a00640 1 -- 172.21.15.152:0/878366207 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7ffa80072d30 con 0x7ffa80141450 2024-04-20T11:28:47.406 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:47.407+0000 7ffa76a00640 1 -- 172.21.15.152:0/878366207 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v18) ==== 104+0+0 (secure 0 0 0) 0x7ffa6c07b780 con 0x7ffa80141450 2024-04-20T11:28:47.410 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:47.410+0000 7ffa84a00640 1 -- 172.21.15.152:0/878366207 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ffa6006cad0 msgr2=0x7ffa6006ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:47.410 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:47.410+0000 7ffa84a00640 1 --2- 172.21.15.152:0/878366207 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ffa6006cad0 0x7ffa6006ef90 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ffa800cf110 tx=0x7ffa70009040 comp rx=0 tx=0).stop 2024-04-20T11:28:47.410 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:47.410+0000 7ffa84a00640 1 -- 172.21.15.152:0/878366207 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffa80141450 msgr2=0x7ffa801f5770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:47.410 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:47.410+0000 7ffa84a00640 1 --2- 172.21.15.152:0/878366207 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffa80141450 0x7ffa801f5770 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c0090b0 tx=0x7ffa6c0046c0 comp rx=0 tx=0).stop 2024-04-20T11:28:47.410 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:47.410+0000 7ffa84a00640 1 -- 172.21.15.152:0/878366207 shutdown_connections 2024-04-20T11:28:47.410 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:47.410+0000 7ffa84a00640 1 --2- 172.21.15.152:0/878366207 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ffa6006cad0 0x7ffa6006ef90 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:47.410 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:47.410+0000 7ffa84a00640 1 --2- 172.21.15.152:0/878366207 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa80160220 0x7ffa8013a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:47.410 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:47.410+0000 7ffa84a00640 1 --2- 172.21.15.152:0/878366207 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffa80141450 0x7ffa801f5770 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:47.410 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:47.410+0000 7ffa84a00640 1 --2- 172.21.15.152:0/878366207 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa80071cd0 0x7ffa80137bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:47.410 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:47.410+0000 7ffa84a00640 1 -- 172.21.15.152:0/878366207 >> 172.21.15.152:0/878366207 conn(0x7ffa80067ae0 msgr2=0x7ffa8015b5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:47.411 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:47.411+0000 7ffa84a00640 1 -- 172.21.15.152:0/878366207 shutdown_connections 2024-04-20T11:28:47.411 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:47.411+0000 7ffa84a00640 1 -- 172.21.15.152:0/878366207 wait complete. 2024-04-20T11:28:47.431 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-20T11:28:47.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:47.629+0000 7fe55a600640 1 Processor -- start 2024-04-20T11:28:47.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:47.630+0000 7fe55a600640 1 -- start start 2024-04-20T11:28:47.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:47.631+0000 7fe55a600640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fe55406f5f0 0x7fe5540cd220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:47.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:47.631+0000 7fe55a600640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe55415f1a0 con 0x7fe5540cd760 2024-04-20T11:28:47.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:47.631+0000 7fe55a600640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe55415f480 con 0x7fe554144f80 2024-04-20T11:28:47.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:47.631+0000 7fe55a600640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fe55415f760 con 0x7fe55406f5f0 2024-04-20T11:28:47.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:47.632+0000 7fe553e00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fe5540cd760 0x7fe554071db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:58292/0 (socket says 172.21.15.152:58292) 2024-04-20T11:28:47.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:47.632+0000 7fe553e00640 1 -- 172.21.15.152:0/2404857215 learned_addr learned my addr 172.21.15.152:0/2404857215 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:28:47.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:47.632+0000 7fe54aa00640 1 --2- 172.21.15.152:0/2404857215 >> v2:172.21.15.175:3300/0 conn(0x7fe55406f5f0 0x7fe5540cd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:47.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:47.632+0000 7fe552a00640 1 -- 172.21.15.152:0/2404857215 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 607330388 0 0) 0x7fe55415f1a0 con 0x7fe5540cd760 2024-04-20T11:28:47.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:47.632+0000 7fe552a00640 1 -- 172.21.15.152:0/2404857215 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe538005730 con 0x7fe5540cd760 2024-04-20T11:28:47.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:47.632+0000 7fe552a00640 1 -- 172.21.15.152:0/2404857215 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3990332808 0 0) 0x7fe538005730 con 0x7fe5540cd760 2024-04-20T11:28:47.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:47.632+0000 7fe552a00640 1 -- 172.21.15.152:0/2404857215 >> v2:172.21.15.175:3300/0 conn(0x7fe55406f5f0 msgr2=0x7fe5540cd220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:47.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:47.632+0000 7fe552a00640 1 --2- 172.21.15.152:0/2404857215 >> v2:172.21.15.175:3300/0 conn(0x7fe55406f5f0 0x7fe5540cd220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:47.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:47.632+0000 7fe552a00640 1 -- 172.21.15.152:0/2404857215 >> v1:172.21.15.175:6789/0 conn(0x7fe554144f80 legacy=0x7fe55406eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:47.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:47.632+0000 7fe552a00640 1 -- 172.21.15.152:0/2404857215 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe55415fa40 con 0x7fe5540cd760 2024-04-20T11:28:47.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:47.632+0000 7fe552a00640 1 -- 172.21.15.152:0/2404857215 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7fe54c0019f0 con 0x7fe5540cd760 2024-04-20T11:28:47.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:47.632+0000 7fe552a00640 1 -- 172.21.15.152:0/2404857215 >> v1:172.21.15.152:6789/0 conn(0x7fe5540cd760 legacy=0x7fe554071db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:47.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:47.632+0000 7fe552a00640 1 --2- 172.21.15.152:0/2404857215 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe538005c30 0x7fe538006030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:47.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:47.633+0000 7fe552a00640 1 --2- 172.21.15.152:0/2404857215 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe538006a10 0x7fe538006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:47.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:47.633+0000 7fe552a00640 1 --2- 172.21.15.152:0/2404857215 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe538007420 0x7fe53800dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:47.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:47.633+0000 7fe552a00640 1 -- 172.21.15.152:0/2404857215 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fe55415f760 con 0x7fe538005c30 2024-04-20T11:28:47.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:47.633+0000 7fe552a00640 1 -- 172.21.15.152:0/2404857215 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe53800e370 con 0x7fe538007420 2024-04-20T11:28:47.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:47.633+0000 7fe552a00640 1 -- 172.21.15.152:0/2404857215 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe53800e570 con 0x7fe538006a10 2024-04-20T11:28:47.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:47.633+0000 7fe553400640 1 --2- 172.21.15.152:0/2404857215 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe538005c30 0x7fe538006030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:47.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:47.633+0000 7fe553400640 1 -- 172.21.15.152:0/2404857215 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe538006a10 msgr2=0x7fe538006e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:47.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:47.633+0000 7fe553400640 1 --2- 172.21.15.152:0/2404857215 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe538006a10 0x7fe538006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:47.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:47.633+0000 7fe553e00640 1 --2- 172.21.15.152:0/2404857215 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe538007420 0x7fe53800dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:47.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:47.633+0000 7fe553400640 1 -- 172.21.15.152:0/2404857215 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe538007420 msgr2=0x7fe53800dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:47.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:47.633+0000 7fe553400640 1 --2- 172.21.15.152:0/2404857215 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe538007420 0x7fe53800dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:47.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:47.633+0000 7fe553400640 1 -- 172.21.15.152:0/2404857215 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe538005730 con 0x7fe538005c30 2024-04-20T11:28:47.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:47.633+0000 7fe553e00640 1 -- 172.21.15.152:0/2404857215 reap_dead start 2024-04-20T11:28:47.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:47.634+0000 7fe553400640 1 --2- 172.21.15.152:0/2404857215 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe538005c30 0x7fe538006030 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fe544003db0 tx=0x7fe538007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea993d973ad3d56c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:47.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:47.635+0000 7fe552a00640 1 -- 172.21.15.152:0/2404857215 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe544037070 con 0x7fe538005c30 2024-04-20T11:28:47.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:47.635+0000 7fe552a00640 1 -- 172.21.15.152:0/2404857215 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe54402be50 con 0x7fe538005c30 2024-04-20T11:28:47.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:47.635+0000 7fe552a00640 1 -- 172.21.15.152:0/2404857215 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe544003980 con 0x7fe538005c30 2024-04-20T11:28:47.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:47.635+0000 7fe55a600640 1 -- 172.21.15.152:0/2404857215 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe538005c30 msgr2=0x7fe538006030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:47.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:47.635+0000 7fe55a600640 1 --2- 172.21.15.152:0/2404857215 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe538005c30 0x7fe538006030 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fe544003db0 tx=0x7fe538007a40 comp rx=0 tx=0).stop 2024-04-20T11:28:47.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:47.635+0000 7fe55a600640 1 -- 172.21.15.152:0/2404857215 shutdown_connections 2024-04-20T11:28:47.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:47.635+0000 7fe55a600640 1 --2- 172.21.15.152:0/2404857215 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe538005c30 0x7fe538006030 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:47.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:47.635+0000 7fe55a600640 1 -- 172.21.15.152:0/2404857215 >> 172.21.15.152:0/2404857215 conn(0x7fe554067cf0 msgr2=0x7fe55406b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:47.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:47.635+0000 7fe55a600640 1 -- 172.21.15.152:0/2404857215 shutdown_connections 2024-04-20T11:28:47.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:47.635+0000 7fe55a600640 1 -- 172.21.15.152:0/2404857215 wait complete. 2024-04-20T11:28:47.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:47.637+0000 7fe55a600640 1 Processor -- start 2024-04-20T11:28:47.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:47.637+0000 7fe55a600640 1 -- start start 2024-04-20T11:28:47.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:47.638+0000 7fe55a600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe538005c30 0x7fe5541f1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:47.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:47.638+0000 7fe55a600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe5541f34e0 0x7fe5541571d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:47.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:47.638+0000 7fe55a600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe554157710 0x7fe5540a9630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:47.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:47.638+0000 7fe55a600640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fe5540cdd70 con 0x7fe5541f34e0 2024-04-20T11:28:47.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:47.638+0000 7fe55a600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe5540cdbf0 con 0x7fe554157710 2024-04-20T11:28:47.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:47.638+0000 7fe55a600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe5540cd220 con 0x7fe538005c30 2024-04-20T11:28:47.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:47.638+0000 7fe54aa00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe5541f34e0 0x7fe5541571d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:47.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:47.638+0000 7fe54aa00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe5541f34e0 0x7fe5541571d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54012/0 (socket says 172.21.15.152:54012) 2024-04-20T11:28:47.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:47.638+0000 7fe54aa00640 1 -- 172.21.15.152:0/990992308 learned_addr learned my addr 172.21.15.152:0/990992308 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:47.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:47.639+0000 7fe553400640 1 --2- 172.21.15.152:0/990992308 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe538005c30 0x7fe5541f1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:47.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:47.639+0000 7fe553e00640 1 --2- 172.21.15.152:0/990992308 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe554157710 0x7fe5540a9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:47.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:47.639+0000 7fe54aa00640 1 -- 172.21.15.152:0/990992308 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe538005c30 msgr2=0x7fe5541f1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:47.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:47.639+0000 7fe54aa00640 1 --2- 172.21.15.152:0/990992308 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe538005c30 0x7fe5541f1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:47.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:47.639+0000 7fe54aa00640 1 -- 172.21.15.152:0/990992308 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe554157710 msgr2=0x7fe5540a9630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:47.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:47.639+0000 7fe54aa00640 1 --2- 172.21.15.152:0/990992308 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe554157710 0x7fe5540a9630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:47.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:47.639+0000 7fe54aa00640 1 -- 172.21.15.152:0/990992308 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe554158c30 con 0x7fe5541f34e0 2024-04-20T11:28:47.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:47.639+0000 7fe54aa00640 1 --2- 172.21.15.152:0/990992308 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe5541f34e0 0x7fe5541571d0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fe53c00d800 tx=0x7fe53c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:47.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:47.639+0000 7fe54be00640 1 -- 172.21.15.152:0/990992308 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe53c00b6c0 con 0x7fe5541f34e0 2024-04-20T11:28:47.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:47.639+0000 7fe54be00640 1 -- 172.21.15.152:0/990992308 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe53c019070 con 0x7fe5541f34e0 2024-04-20T11:28:47.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:47.639+0000 7fe54be00640 1 -- 172.21.15.152:0/990992308 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe53c00baa0 con 0x7fe5541f34e0 2024-04-20T11:28:47.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:47.639+0000 7fe55a600640 1 -- 172.21.15.152:0/990992308 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe554158f10 con 0x7fe5541f34e0 2024-04-20T11:28:47.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:47.640+0000 7fe55a600640 1 -- 172.21.15.152:0/990992308 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5541594d0 con 0x7fe5541f34e0 2024-04-20T11:28:47.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:47.641+0000 7fe55a600640 1 -- 172.21.15.152:0/990992308 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe554068a60 con 0x7fe5541f34e0 2024-04-20T11:28:47.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:47.642+0000 7fe54be00640 1 -- 172.21.15.152:0/990992308 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7fe53c00bc40 con 0x7fe5541f34e0 2024-04-20T11:28:47.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:47.642+0000 7fe54be00640 1 --2- 172.21.15.152:0/990992308 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe53006cad0 0x7fe53006ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:47.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:47.643+0000 7fe54be00640 1 -- 172.21.15.152:0/990992308 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5795+0+0 (secure 0 0 0) 0x7fe53c08c270 con 0x7fe5541f34e0 2024-04-20T11:28:47.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:47.643+0000 7fe553400640 1 --2- 172.21.15.152:0/990992308 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe53006cad0 0x7fe53006ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:47.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:47.643+0000 7fe553400640 1 --2- 172.21.15.152:0/990992308 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe53006cad0 0x7fe53006ef90 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe544034000 tx=0x7fe544009030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:47.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:47.647+0000 7fe54be00640 1 -- 172.21.15.152:0/990992308 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe53c05a3c0 con 0x7fe5541f34e0 2024-04-20T11:28:47.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:47.832+0000 7fe55a600640 1 -- 172.21.15.152:0/990992308 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fe5540b36e0 con 0x7fe5541f34e0 2024-04-20T11:28:48.691 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:48.691+0000 7fe54be00640 1 -- 172.21.15.152:0/990992308 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v36) ==== 181+0+0 (secure 0 0 0) 0x7fe53c059cc0 con 0x7fe5541f34e0 2024-04-20T11:28:48.691 INFO:teuthology.orchestra.run.smithi152.stderr:pool 'cephfs_metadata' removed 2024-04-20T11:28:48.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:48.695+0000 7fe55a600640 1 -- 172.21.15.152:0/990992308 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe53006cad0 msgr2=0x7fe53006ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:48.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:48.695+0000 7fe55a600640 1 --2- 172.21.15.152:0/990992308 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe53006cad0 0x7fe53006ef90 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe544034000 tx=0x7fe544009030 comp rx=0 tx=0).stop 2024-04-20T11:28:48.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:48.695+0000 7fe55a600640 1 -- 172.21.15.152:0/990992308 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe5541f34e0 msgr2=0x7fe5541571d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:48.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:48.695+0000 7fe55a600640 1 --2- 172.21.15.152:0/990992308 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe5541f34e0 0x7fe5541571d0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fe53c00d800 tx=0x7fe53c00dcc0 comp rx=0 tx=0).stop 2024-04-20T11:28:48.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:48.695+0000 7fe55a600640 1 -- 172.21.15.152:0/990992308 shutdown_connections 2024-04-20T11:28:48.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:48.695+0000 7fe55a600640 1 --2- 172.21.15.152:0/990992308 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe53006cad0 0x7fe53006ef90 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:48.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:48.695+0000 7fe55a600640 1 --2- 172.21.15.152:0/990992308 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe554157710 0x7fe5540a9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:48.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:48.695+0000 7fe55a600640 1 --2- 172.21.15.152:0/990992308 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe5541f34e0 0x7fe5541571d0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:48.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:48.695+0000 7fe55a600640 1 --2- 172.21.15.152:0/990992308 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe538005c30 0x7fe5541f1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:48.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:48.695+0000 7fe55a600640 1 -- 172.21.15.152:0/990992308 >> 172.21.15.152:0/990992308 conn(0x7fe554067cf0 msgr2=0x7fe55406b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:48.698 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:48.698+0000 7fe55a600640 1 -- 172.21.15.152:0/990992308 shutdown_connections 2024-04-20T11:28:48.698 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:48.698+0000 7fe55a600640 1 -- 172.21.15.152:0/990992308 wait complete. 2024-04-20T11:28:48.722 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-20T11:28:48.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:48.916+0000 7efe85a00640 1 Processor -- start 2024-04-20T11:28:48.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:48.917+0000 7efe85a00640 1 -- start start 2024-04-20T11:28:48.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:48.918+0000 7efe85a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efe80145340 0x7efe80160db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:48.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:48.918+0000 7efe85a00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe800b4f90 con 0x7efe800b47a0 2024-04-20T11:28:48.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:48.918+0000 7efe85a00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe800b4cb0 con 0x7efe80072530 2024-04-20T11:28:48.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:48.918+0000 7efe85a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7efe80145720 con 0x7efe80145340 2024-04-20T11:28:48.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:48.919+0000 7efe7e000640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7efe800b47a0 0x7efe80071e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:58300/0 (socket says 172.21.15.152:58300) 2024-04-20T11:28:48.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:48.919+0000 7efe7e000640 1 -- 172.21.15.152:0/2389050739 learned_addr learned my addr 172.21.15.152:0/2389050739 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:28:48.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:48.919+0000 7efe7ea00640 1 --2- 172.21.15.152:0/2389050739 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efe80145340 0x7efe80160db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:48.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:48.919+0000 7efe7d600640 1 -- 172.21.15.152:0/2389050739 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 892266593 0 0) 0x7efe800b4f90 con 0x7efe800b47a0 2024-04-20T11:28:48.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:48.919+0000 7efe7d600640 1 -- 172.21.15.152:0/2389050739 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efe64003520 con 0x7efe800b47a0 2024-04-20T11:28:48.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:48.919+0000 7efe7d600640 1 -- 172.21.15.152:0/2389050739 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 121460417 0 0) 0x7efe64003520 con 0x7efe800b47a0 2024-04-20T11:28:48.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:48.919+0000 7efe7d600640 1 -- 172.21.15.152:0/2389050739 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efe80145340 msgr2=0x7efe80160db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:48.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:48.919+0000 7efe7d600640 1 --2- 172.21.15.152:0/2389050739 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efe80145340 0x7efe80160db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:48.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:48.919+0000 7efe7d600640 1 -- 172.21.15.152:0/2389050739 >> v1:172.21.15.175:6789/0 conn(0x7efe80072530 legacy=0x7efe8015cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:48.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:48.919+0000 7efe7d600640 1 -- 172.21.15.152:0/2389050739 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe801602d0 con 0x7efe800b47a0 2024-04-20T11:28:48.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:48.919+0000 7efe7d600640 1 -- 172.21.15.152:0/2389050739 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7efe6c001670 con 0x7efe800b47a0 2024-04-20T11:28:48.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:48.920+0000 7efe7d600640 1 -- 172.21.15.152:0/2389050739 >> v1:172.21.15.152:6789/0 conn(0x7efe800b47a0 legacy=0x7efe80071e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:48.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:48.920+0000 7efe7d600640 1 --2- 172.21.15.152:0/2389050739 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efe64003a50 0x7efe64003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:48.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:48.920+0000 7efe7d600640 1 --2- 172.21.15.152:0/2389050739 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efe80145340 0x7efe64004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:48.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:48.920+0000 7efe7d600640 1 --2- 172.21.15.152:0/2389050739 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efe64004d40 0x7efe640051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:48.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:48.920+0000 7efe7d600640 1 -- 172.21.15.152:0/2389050739 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7efe80145720 con 0x7efe64003a50 2024-04-20T11:28:48.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:48.920+0000 7efe7d600640 1 -- 172.21.15.152:0/2389050739 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7efe6400bbf0 con 0x7efe64004d40 2024-04-20T11:28:48.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:48.920+0000 7efe7d600640 1 -- 172.21.15.152:0/2389050739 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7efe6400bdf0 con 0x7efe80145340 2024-04-20T11:28:48.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:48.920+0000 7efe7ea00640 1 --2- 172.21.15.152:0/2389050739 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efe64003a50 0x7efe64003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:48.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:48.920+0000 7efe7e000640 1 --2- 172.21.15.152:0/2389050739 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efe80145340 0x7efe64004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:48.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:48.920+0000 7efe7ea00640 1 -- 172.21.15.152:0/2389050739 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efe80145340 msgr2=0x7efe64004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:48.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:48.920+0000 7efe7ea00640 1 --2- 172.21.15.152:0/2389050739 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efe80145340 0x7efe64004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:48.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:48.920+0000 7efe7ea00640 1 -- 172.21.15.152:0/2389050739 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efe64004d40 msgr2=0x7efe640051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:48.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:48.920+0000 7efe7f400640 1 --2- 172.21.15.152:0/2389050739 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efe64004d40 0x7efe640051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:48.922 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:48.920+0000 7efe7ea00640 1 --2- 172.21.15.152:0/2389050739 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efe64004d40 0x7efe640051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:48.922 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:48.920+0000 7efe7ea00640 1 -- 172.21.15.152:0/2389050739 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe64003520 con 0x7efe64003a50 2024-04-20T11:28:48.922 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:48.920+0000 7efe7e000640 1 --2- 172.21.15.152:0/2389050739 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efe80145340 0x7efe64004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:28:48.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:48.921+0000 7efe7ea00640 1 --2- 172.21.15.152:0/2389050739 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efe64003a50 0x7efe64003e70 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7efe70009ac0 tx=0x7efe64005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b62c84ee70e7407f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:48.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:48.922+0000 7efe7d600640 1 -- 172.21.15.152:0/2389050739 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe70039070 con 0x7efe64003a50 2024-04-20T11:28:48.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:48.922+0000 7efe7d600640 1 -- 172.21.15.152:0/2389050739 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe7002c6b0 con 0x7efe64003a50 2024-04-20T11:28:48.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:48.922+0000 7efe7d600640 1 -- 172.21.15.152:0/2389050739 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe70009690 con 0x7efe64003a50 2024-04-20T11:28:48.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:48.922+0000 7efe85a00640 1 -- 172.21.15.152:0/2389050739 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efe64003a50 msgr2=0x7efe64003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:48.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:48.922+0000 7efe85a00640 1 --2- 172.21.15.152:0/2389050739 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efe64003a50 0x7efe64003e70 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7efe70009ac0 tx=0x7efe64005830 comp rx=0 tx=0).stop 2024-04-20T11:28:48.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:48.922+0000 7efe7f400640 1 -- 172.21.15.152:0/2389050739 reap_dead start 2024-04-20T11:28:48.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:48.923+0000 7efe85a00640 1 -- 172.21.15.152:0/2389050739 shutdown_connections 2024-04-20T11:28:48.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:48.923+0000 7efe85a00640 1 -- 172.21.15.152:0/2389050739 >> 172.21.15.152:0/2389050739 conn(0x7efe8006b980 msgr2=0x7efe8006bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:48.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:48.923+0000 7efe85a00640 1 -- 172.21.15.152:0/2389050739 shutdown_connections 2024-04-20T11:28:48.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:48.923+0000 7efe85a00640 1 -- 172.21.15.152:0/2389050739 wait complete. 2024-04-20T11:28:48.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:48.925+0000 7efe85a00640 1 Processor -- start 2024-04-20T11:28:48.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:48.925+0000 7efe85a00640 1 -- start start 2024-04-20T11:28:48.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:48.926+0000 7efe85a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efe64003a50 0x7efe8015bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:48.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:48.926+0000 7efe85a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efe801f55f0 0x7efe801411e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:48.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:48.926+0000 7efe85a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efe80141720 0x7efe801396f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:48.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:48.926+0000 7efe85a00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7efe80160db0 con 0x7efe801f55f0 2024-04-20T11:28:48.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:48.926+0000 7efe85a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7efe80146010 con 0x7efe64003a50 2024-04-20T11:28:48.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:48.926+0000 7efe85a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7efe80160f30 con 0x7efe80141720 2024-04-20T11:28:48.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:48.926+0000 7efe7e000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efe801f55f0 0x7efe801411e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:48.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:48.927+0000 7efe7e000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efe801f55f0 0x7efe801411e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54034/0 (socket says 172.21.15.152:54034) 2024-04-20T11:28:48.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:48.927+0000 7efe7e000640 1 -- 172.21.15.152:0/2299350890 learned_addr learned my addr 172.21.15.152:0/2299350890 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:48.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:48.927+0000 7efe7ea00640 1 --2- 172.21.15.152:0/2299350890 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efe64003a50 0x7efe8015bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:48.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:48.927+0000 7efe7f400640 1 --2- 172.21.15.152:0/2299350890 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efe80141720 0x7efe801396f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:48.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:48.927+0000 7efe7e000640 1 -- 172.21.15.152:0/2299350890 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efe80141720 msgr2=0x7efe801396f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:48.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:48.927+0000 7efe7e000640 1 --2- 172.21.15.152:0/2299350890 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efe80141720 0x7efe801396f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:48.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:48.927+0000 7efe7e000640 1 -- 172.21.15.152:0/2299350890 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efe64003a50 msgr2=0x7efe8015bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:48.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:48.927+0000 7efe7e000640 1 --2- 172.21.15.152:0/2299350890 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efe64003a50 0x7efe8015bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:48.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:48.927+0000 7efe7e000640 1 -- 172.21.15.152:0/2299350890 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe800add20 con 0x7efe801f55f0 2024-04-20T11:28:48.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:48.927+0000 7efe7ea00640 1 --2- 172.21.15.152:0/2299350890 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efe64003a50 0x7efe8015bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:28:48.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:48.927+0000 7efe7e000640 1 --2- 172.21.15.152:0/2299350890 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efe801f55f0 0x7efe801411e0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7efe6c001200 tx=0x7efe6c00dc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:48.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:48.928+0000 7efe76a00640 1 -- 172.21.15.152:0/2299350890 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe6c019070 con 0x7efe801f55f0 2024-04-20T11:28:48.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:48.928+0000 7efe76a00640 1 -- 172.21.15.152:0/2299350890 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe6c0074e0 con 0x7efe801f55f0 2024-04-20T11:28:48.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:48.928+0000 7efe85a00640 1 -- 172.21.15.152:0/2299350890 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe800ae000 con 0x7efe801f55f0 2024-04-20T11:28:48.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:48.928+0000 7efe76a00640 1 -- 172.21.15.152:0/2299350890 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe6c004a20 con 0x7efe801f55f0 2024-04-20T11:28:48.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:48.928+0000 7efe85a00640 1 -- 172.21.15.152:0/2299350890 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe800ae5c0 con 0x7efe801f55f0 2024-04-20T11:28:48.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:48.929+0000 7efe85a00640 1 -- 172.21.15.152:0/2299350890 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe80068a60 con 0x7efe801f55f0 2024-04-20T11:28:48.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:48.930+0000 7efe76a00640 1 -- 172.21.15.152:0/2299350890 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7efe6c01c070 con 0x7efe801f55f0 2024-04-20T11:28:48.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:48.931+0000 7efe76a00640 1 --2- 172.21.15.152:0/2299350890 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7efe6006caa0 0x7efe6006ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:48.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:48.931+0000 7efe76a00640 1 -- 172.21.15.152:0/2299350890 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5357+0+0 (secure 0 0 0) 0x7efe6c08b880 con 0x7efe801f55f0 2024-04-20T11:28:48.931 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:48.931+0000 7efe7ea00640 1 --2- 172.21.15.152:0/2299350890 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7efe6006caa0 0x7efe6006ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:48.931 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:48.932+0000 7efe7ea00640 1 --2- 172.21.15.152:0/2299350890 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7efe6006caa0 0x7efe6006ef60 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7efe70036000 tx=0x7efe70030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:48.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:48.936+0000 7efe76a00640 1 -- 172.21.15.152:0/2299350890 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7efe6c059390 con 0x7efe801f55f0 2024-04-20T11:28:49.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:49.149+0000 7efe85a00640 1 -- 172.21.15.152:0/2299350890 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7efe8013e620 con 0x7efe801f55f0 2024-04-20T11:28:49.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:49.701+0000 7efe76a00640 1 -- 172.21.15.152:0/2299350890 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v37) ==== 169+0+0 (secure 0 0 0) 0x7efe6c05d690 con 0x7efe801f55f0 2024-04-20T11:28:49.702 INFO:teuthology.orchestra.run.smithi152.stderr:pool 'cephfs_data' removed 2024-04-20T11:28:49.707 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:49.707+0000 7efe85a00640 1 -- 172.21.15.152:0/2299350890 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7efe6006caa0 msgr2=0x7efe6006ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:49.707 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:49.707+0000 7efe85a00640 1 --2- 172.21.15.152:0/2299350890 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7efe6006caa0 0x7efe6006ef60 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7efe70036000 tx=0x7efe70030000 comp rx=0 tx=0).stop 2024-04-20T11:28:49.707 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:49.707+0000 7efe85a00640 1 -- 172.21.15.152:0/2299350890 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efe801f55f0 msgr2=0x7efe801411e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:49.707 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:49.707+0000 7efe85a00640 1 --2- 172.21.15.152:0/2299350890 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efe801f55f0 0x7efe801411e0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7efe6c001200 tx=0x7efe6c00dc50 comp rx=0 tx=0).stop 2024-04-20T11:28:49.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:49.707+0000 7efe85a00640 1 -- 172.21.15.152:0/2299350890 shutdown_connections 2024-04-20T11:28:49.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:49.707+0000 7efe85a00640 1 --2- 172.21.15.152:0/2299350890 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7efe6006caa0 0x7efe6006ef60 secure :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7efe70036000 tx=0x7efe70030000 comp rx=0 tx=0).stop 2024-04-20T11:28:49.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:49.707+0000 7efe85a00640 1 --2- 172.21.15.152:0/2299350890 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efe80141720 0x7efe801396f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:49.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:49.707+0000 7efe85a00640 1 --2- 172.21.15.152:0/2299350890 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efe801f55f0 0x7efe801411e0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:49.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:49.707+0000 7efe85a00640 1 --2- 172.21.15.152:0/2299350890 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efe64003a50 0x7efe8015bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:49.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:49.707+0000 7efe85a00640 1 -- 172.21.15.152:0/2299350890 >> 172.21.15.152:0/2299350890 conn(0x7efe8006b980 msgr2=0x7efe800b5100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:49.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:49.708+0000 7efe85a00640 1 -- 172.21.15.152:0/2299350890 shutdown_connections 2024-04-20T11:28:49.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:49.708+0000 7efe85a00640 1 -- 172.21.15.152:0/2299350890 wait complete. 2024-04-20T11:28:49.735 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-20T11:28:49.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:49.919+0000 7f580e400640 1 Processor -- start 2024-04-20T11:28:49.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:49.919+0000 7f580e400640 1 -- start start 2024-04-20T11:28:49.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:49.920+0000 7f580e400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f58080b2880 0x7f580815fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:49.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:49.920+0000 7f580e400640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f580815f1c0 con 0x7f58080cfc40 2024-04-20T11:28:49.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:49.920+0000 7f580e400640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f580806f670 con 0x7f580815b510 2024-04-20T11:28:49.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:49.920+0000 7f580e400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f580806f420 con 0x7f58080b2880 2024-04-20T11:28:49.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:49.921+0000 7f5806a00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f58080cfc40 0x7f58080b4800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:58304/0 (socket says 172.21.15.152:58304) 2024-04-20T11:28:49.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:49.921+0000 7f5806a00640 1 -- 172.21.15.152:0/2975575575 learned_addr learned my addr 172.21.15.152:0/2975575575 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:28:49.922 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:49.921+0000 7f5807e00640 1 --2- 172.21.15.152:0/2975575575 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f58080b2880 0x7f580815fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:49.922 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:49.921+0000 7f5806000640 1 -- 172.21.15.152:0/2975575575 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2342988522 0 0) 0x7f580815f1c0 con 0x7f58080cfc40 2024-04-20T11:28:49.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:49.921+0000 7f5806000640 1 -- 172.21.15.152:0/2975575575 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f57ec005730 con 0x7f58080cfc40 2024-04-20T11:28:49.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:49.922+0000 7f5806000640 1 -- 172.21.15.152:0/2975575575 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3977215018 0 0) 0x7f57ec005730 con 0x7f58080cfc40 2024-04-20T11:28:49.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:49.922+0000 7f5806000640 1 -- 172.21.15.152:0/2975575575 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f58080b2880 msgr2=0x7f580815fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:49.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:49.922+0000 7f5806000640 1 --2- 172.21.15.152:0/2975575575 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f58080b2880 0x7f580815fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:49.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:49.922+0000 7f5806000640 1 -- 172.21.15.152:0/2975575575 >> v1:172.21.15.175:6789/0 conn(0x7f580815b510 legacy=0x7f58080b2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:49.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:49.922+0000 7f5806000640 1 -- 172.21.15.152:0/2975575575 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f580815f430 con 0x7f58080cfc40 2024-04-20T11:28:49.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:49.922+0000 7f5806000640 1 -- 172.21.15.152:0/2975575575 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f57f0001670 con 0x7f58080cfc40 2024-04-20T11:28:49.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:49.922+0000 7f5806000640 1 -- 172.21.15.152:0/2975575575 >> v1:172.21.15.152:6789/0 conn(0x7f58080cfc40 legacy=0x7f58080b4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:49.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:49.922+0000 7f5806000640 1 --2- 172.21.15.152:0/2975575575 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f58080b2880 0x7f57ec005bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:49.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:49.922+0000 7f5806000640 1 --2- 172.21.15.152:0/2975575575 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f57ec006170 0x7f57ec006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:49.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:49.922+0000 7f5806000640 1 --2- 172.21.15.152:0/2975575575 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f57ec006f50 0x7f57ec0073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:49.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:49.922+0000 7f5806000640 1 -- 172.21.15.152:0/2975575575 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f580806f420 con 0x7f57ec006170 2024-04-20T11:28:49.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:49.922+0000 7f5806000640 1 -- 172.21.15.152:0/2975575575 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f57ec00dcb0 con 0x7f57ec006f50 2024-04-20T11:28:49.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:49.922+0000 7f5806000640 1 -- 172.21.15.152:0/2975575575 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f57ec00deb0 con 0x7f58080b2880 2024-04-20T11:28:49.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:49.922+0000 7f5806a00640 1 --2- 172.21.15.152:0/2975575575 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f57ec006170 0x7f57ec006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:49.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:49.922+0000 7f5807e00640 1 --2- 172.21.15.152:0/2975575575 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f57ec006f50 0x7f57ec0073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:49.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:49.923+0000 7f5807400640 1 --2- 172.21.15.152:0/2975575575 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f58080b2880 0x7f57ec005bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:49.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:49.923+0000 7f5806a00640 1 -- 172.21.15.152:0/2975575575 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f58080b2880 msgr2=0x7f57ec005bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:49.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:49.923+0000 7f5806a00640 1 --2- 172.21.15.152:0/2975575575 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f58080b2880 0x7f57ec005bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:49.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:49.923+0000 7f5806a00640 1 -- 172.21.15.152:0/2975575575 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f57ec006f50 msgr2=0x7f57ec0073d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:49.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:49.923+0000 7f5806a00640 1 --2- 172.21.15.152:0/2975575575 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f57ec006f50 0x7f57ec0073d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:49.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:49.923+0000 7f5806a00640 1 -- 172.21.15.152:0/2975575575 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57ec005730 con 0x7f57ec006170 2024-04-20T11:28:49.925 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:49.924+0000 7f5806a00640 1 --2- 172.21.15.152:0/2975575575 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f57ec006170 0x7f57ec006a10 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f57f000c240 tx=0x7f57ec007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cb4eee8e28ea973 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:49.925 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:49.924+0000 7f5806000640 1 -- 172.21.15.152:0/2975575575 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f0007d10 con 0x7f57ec006170 2024-04-20T11:28:49.925 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:49.924+0000 7f5806000640 1 -- 172.21.15.152:0/2975575575 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57f0030070 con 0x7f57ec006170 2024-04-20T11:28:49.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:49.924+0000 7f5806000640 1 -- 172.21.15.152:0/2975575575 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f002cda0 con 0x7f57ec006170 2024-04-20T11:28:49.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:49.925+0000 7f580e400640 1 -- 172.21.15.152:0/2975575575 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f57ec006170 msgr2=0x7f57ec006a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:49.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:49.925+0000 7f580e400640 1 --2- 172.21.15.152:0/2975575575 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f57ec006170 0x7f57ec006a10 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f57f000c240 tx=0x7f57ec007a40 comp rx=0 tx=0).stop 2024-04-20T11:28:49.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:49.925+0000 7f5807e00640 1 -- 172.21.15.152:0/2975575575 reap_dead start 2024-04-20T11:28:49.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:49.925+0000 7f580e400640 1 -- 172.21.15.152:0/2975575575 shutdown_connections 2024-04-20T11:28:49.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:49.925+0000 7f580e400640 1 -- 172.21.15.152:0/2975575575 >> 172.21.15.152:0/2975575575 conn(0x7f580806b980 msgr2=0x7f580806bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:49.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:49.925+0000 7f580e400640 1 -- 172.21.15.152:0/2975575575 shutdown_connections 2024-04-20T11:28:49.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:49.925+0000 7f580e400640 1 -- 172.21.15.152:0/2975575575 wait complete. 2024-04-20T11:28:49.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:49.926+0000 7f580e400640 1 Processor -- start 2024-04-20T11:28:49.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:49.927+0000 7f580e400640 1 -- start start 2024-04-20T11:28:49.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:49.928+0000 7f580e400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f57ec006170 0x7f580813d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:49.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:49.928+0000 7f580e400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5808151e50 0x7f580815dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:49.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:49.928+0000 7f580e400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f58081522b0 0x7f580815e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:49.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:49.928+0000 7f580e400640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f580806f950 con 0x7f5808151e50 2024-04-20T11:28:49.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:49.928+0000 7f580e400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f580815b8f0 con 0x7f58081522b0 2024-04-20T11:28:49.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:49.928+0000 7f580e400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f580815fc30 con 0x7f57ec006170 2024-04-20T11:28:49.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:49.928+0000 7f5806a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5808151e50 0x7f580815dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:49.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:49.928+0000 7f5806a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5808151e50 0x7f580815dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54054/0 (socket says 172.21.15.152:54054) 2024-04-20T11:28:49.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:49.928+0000 7f5806a00640 1 -- 172.21.15.152:0/2187649496 learned_addr learned my addr 172.21.15.152:0/2187649496 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:49.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:49.928+0000 7f5807400640 1 --2- 172.21.15.152:0/2187649496 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f57ec006170 0x7f580813d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:49.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:49.928+0000 7f5807e00640 1 --2- 172.21.15.152:0/2187649496 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f58081522b0 0x7f580815e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:49.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:49.928+0000 7f5806a00640 1 -- 172.21.15.152:0/2187649496 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f57ec006170 msgr2=0x7f580813d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:49.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:49.928+0000 7f5806a00640 1 --2- 172.21.15.152:0/2187649496 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f57ec006170 0x7f580813d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:49.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:49.928+0000 7f5806a00640 1 -- 172.21.15.152:0/2187649496 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f58081522b0 msgr2=0x7f580815e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:49.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:49.928+0000 7f5806a00640 1 --2- 172.21.15.152:0/2187649496 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f58081522b0 0x7f580815e180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:49.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:49.928+0000 7f5806a00640 1 -- 172.21.15.152:0/2187649496 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f580814c0c0 con 0x7f5808151e50 2024-04-20T11:28:49.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:49.929+0000 7f5806a00640 1 --2- 172.21.15.152:0/2187649496 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5808151e50 0x7f580815dc40 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f57f000c360 tx=0x7f580815f600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:49.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:49.929+0000 7f57ff400640 1 -- 172.21.15.152:0/2187649496 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f0046070 con 0x7f5808151e50 2024-04-20T11:28:49.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:49.929+0000 7f57ff400640 1 -- 172.21.15.152:0/2187649496 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57f0004bf0 con 0x7f5808151e50 2024-04-20T11:28:49.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:49.929+0000 7f57ff400640 1 -- 172.21.15.152:0/2187649496 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f0007d10 con 0x7f5808151e50 2024-04-20T11:28:49.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:49.929+0000 7f580e400640 1 -- 172.21.15.152:0/2187649496 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f580814c3a0 con 0x7f5808151e50 2024-04-20T11:28:49.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:49.929+0000 7f580e400640 1 -- 172.21.15.152:0/2187649496 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f580814c960 con 0x7f5808151e50 2024-04-20T11:28:49.931 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:49.930+0000 7f580e400640 1 -- 172.21.15.152:0/2187649496 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58080cf860 con 0x7f5808151e50 2024-04-20T11:28:49.931 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:49.931+0000 7f57ff400640 1 -- 172.21.15.152:0/2187649496 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f57f0007070 con 0x7f5808151e50 2024-04-20T11:28:49.932 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:49.932+0000 7f57ff400640 1 --2- 172.21.15.152:0/2187649496 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f57e406caa0 0x7f57e406ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:49.932 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:49.932+0000 7f57ff400640 1 -- 172.21.15.152:0/2187649496 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7f57f00ace50 con 0x7f5808151e50 2024-04-20T11:28:49.932 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:49.933+0000 7f5807400640 1 --2- 172.21.15.152:0/2187649496 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f57e406caa0 0x7f57e406ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:49.932 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:49.933+0000 7f5807400640 1 --2- 172.21.15.152:0/2187649496 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f57e406caa0 0x7f57e406ef60 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f57f8001590 tx=0x7f57f8009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:49.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:49.937+0000 7f57ff400640 1 -- 172.21.15.152:0/2187649496 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f58080cf860 con 0x7f5808151e50 2024-04-20T11:28:50.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.148+0000 7f580e400640 1 -- 172.21.15.152:0/2187649496 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f58080689f0 con 0x7f5808151e50 2024-04-20T11:28:50.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.155+0000 7f57ff400640 1 -- 172.21.15.152:0/2187649496 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7f57f007ac90 con 0x7f5808151e50 2024-04-20T11:28:50.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.159+0000 7f580e400640 1 -- 172.21.15.152:0/2187649496 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f57e406caa0 msgr2=0x7f57e406ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:50.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.159+0000 7f580e400640 1 --2- 172.21.15.152:0/2187649496 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f57e406caa0 0x7f57e406ef60 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f57f8001590 tx=0x7f57f8009040 comp rx=0 tx=0).stop 2024-04-20T11:28:50.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.159+0000 7f580e400640 1 -- 172.21.15.152:0/2187649496 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5808151e50 msgr2=0x7f580815dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:50.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.159+0000 7f580e400640 1 --2- 172.21.15.152:0/2187649496 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5808151e50 0x7f580815dc40 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f57f000c360 tx=0x7f580815f600 comp rx=0 tx=0).stop 2024-04-20T11:28:50.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.159+0000 7f580e400640 1 -- 172.21.15.152:0/2187649496 shutdown_connections 2024-04-20T11:28:50.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.159+0000 7f580e400640 1 --2- 172.21.15.152:0/2187649496 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f57e406caa0 0x7f57e406ef60 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:50.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.159+0000 7f580e400640 1 --2- 172.21.15.152:0/2187649496 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f58081522b0 0x7f580815e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:50.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.159+0000 7f580e400640 1 --2- 172.21.15.152:0/2187649496 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5808151e50 0x7f580815dc40 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:50.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.159+0000 7f580e400640 1 --2- 172.21.15.152:0/2187649496 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f57ec006170 0x7f580813d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:50.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.159+0000 7f580e400640 1 -- 172.21.15.152:0/2187649496 >> 172.21.15.152:0/2187649496 conn(0x7f580806b980 msgr2=0x7f5808071db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:50.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.159+0000 7f580e400640 1 -- 172.21.15.152:0/2187649496 shutdown_connections 2024-04-20T11:28:50.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.159+0000 7f580e400640 1 -- 172.21.15.152:0/2187649496 wait complete. 2024-04-20T11:28:50.178 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade' 2024-04-20T11:28:50.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.374+0000 7f871da00640 1 Processor -- start 2024-04-20T11:28:50.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.374+0000 7f871da00640 1 -- start start 2024-04-20T11:28:50.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.376+0000 7f871da00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f8718145f60 0x7f871806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:50.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.376+0000 7f871da00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8718146340 con 0x7f87180cd0d0 2024-04-20T11:28:50.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.376+0000 7f871da00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f871806f3c0 con 0x7f87180726f0 2024-04-20T11:28:50.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.376+0000 7f871da00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f871806e9f0 con 0x7f8718145f60 2024-04-20T11:28:50.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.376+0000 7f8716a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f8718145f60 0x7f871806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:50.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.376+0000 7f8716000640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f87180cd0d0 0x7f8718071fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:58314/0 (socket says 172.21.15.152:58314) 2024-04-20T11:28:50.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.376+0000 7f8716000640 1 -- 172.21.15.152:0/96922395 learned_addr learned my addr 172.21.15.152:0/96922395 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:28:50.377 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.377+0000 7f8715600640 1 -- 172.21.15.152:0/96922395 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1614312128 0 0) 0x7f8718146340 con 0x7f87180cd0d0 2024-04-20T11:28:50.377 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.377+0000 7f8715600640 1 -- 172.21.15.152:0/96922395 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f86f8003520 con 0x7f87180cd0d0 2024-04-20T11:28:50.377 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.377+0000 7f8716a00640 1 -- 172.21.15.152:0/96922395 >> v1:172.21.15.175:6789/0 conn(0x7f87180726f0 legacy=0x7f87180cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:50.377 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.377+0000 7f8716a00640 1 -- 172.21.15.152:0/96922395 >> v1:172.21.15.152:6789/0 conn(0x7f87180cd0d0 legacy=0x7f8718071fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:50.377 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.377+0000 7f8716a00640 1 -- 172.21.15.152:0/96922395 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8718072d50 con 0x7f8718145f60 2024-04-20T11:28:50.377 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.378+0000 7f8716a00640 1 --2- 172.21.15.152:0/96922395 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8718145f60 0x7f871806ee80 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f87080097a0 tx=0x7f870802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cc90a15d246dd99 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:50.378 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.379+0000 7f8715600640 1 -- 172.21.15.152:0/96922395 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8708035020 con 0x7f8718145f60 2024-04-20T11:28:50.378 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.379+0000 7f8715600640 1 -- 172.21.15.152:0/96922395 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8708031830 con 0x7f8718145f60 2024-04-20T11:28:50.378 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.379+0000 7f8715600640 1 -- 172.21.15.152:0/96922395 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8708031be0 con 0x7f8718145f60 2024-04-20T11:28:50.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.379+0000 7f871da00640 1 -- 172.21.15.152:0/96922395 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8718145f60 msgr2=0x7f871806ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:50.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.379+0000 7f871da00640 1 --2- 172.21.15.152:0/96922395 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8718145f60 0x7f871806ee80 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f87080097a0 tx=0x7f870802b8d0 comp rx=0 tx=0).stop 2024-04-20T11:28:50.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.379+0000 7f871da00640 1 -- 172.21.15.152:0/96922395 shutdown_connections 2024-04-20T11:28:50.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.379+0000 7f871da00640 1 --2- 172.21.15.152:0/96922395 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8718145f60 0x7f871806ee80 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:50.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.379+0000 7f871da00640 1 -- 172.21.15.152:0/96922395 >> 172.21.15.152:0/96922395 conn(0x7f8718067cf0 msgr2=0x7f871806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:50.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.379+0000 7f871da00640 1 -- 172.21.15.152:0/96922395 shutdown_connections 2024-04-20T11:28:50.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.379+0000 7f871da00640 1 -- 172.21.15.152:0/96922395 wait complete. 2024-04-20T11:28:50.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.381+0000 7f871da00640 1 Processor -- start 2024-04-20T11:28:50.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.381+0000 7f871da00640 1 -- start start 2024-04-20T11:28:50.383 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.382+0000 7f871da00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87180726f0 0x7f871815ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:50.383 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.382+0000 7f871da00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f87180cd0d0 0x7f87180ac310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:50.383 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.382+0000 7f871da00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8718145f60 0x7f87181415b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:50.383 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.382+0000 7f871da00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f871806e680 con 0x7f87180726f0 2024-04-20T11:28:50.383 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.382+0000 7f871da00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f871806e3a0 con 0x7f8718145f60 2024-04-20T11:28:50.383 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.382+0000 7f871da00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f87180cd580 con 0x7f87180cd0d0 2024-04-20T11:28:50.383 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.382+0000 7f8716a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87180726f0 0x7f871815ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:50.383 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.382+0000 7f8716a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87180726f0 0x7f871815ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54068/0 (socket says 172.21.15.152:54068) 2024-04-20T11:28:50.383 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.382+0000 7f8716a00640 1 -- 172.21.15.152:0/3653817546 learned_addr learned my addr 172.21.15.152:0/3653817546 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:50.383 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.383+0000 7f8717400640 1 --2- 172.21.15.152:0/3653817546 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8718145f60 0x7f87181415b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:50.383 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.383+0000 7f8716000640 1 --2- 172.21.15.152:0/3653817546 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f87180cd0d0 0x7f87180ac310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:50.384 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.383+0000 7f8716a00640 1 -- 172.21.15.152:0/3653817546 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f87180cd0d0 msgr2=0x7f87180ac310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:50.384 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.383+0000 7f8716a00640 1 --2- 172.21.15.152:0/3653817546 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f87180cd0d0 0x7f87180ac310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:50.384 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.383+0000 7f8716a00640 1 -- 172.21.15.152:0/3653817546 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8718145f60 msgr2=0x7f87181415b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:50.384 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.383+0000 7f8716a00640 1 --2- 172.21.15.152:0/3653817546 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8718145f60 0x7f87181415b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:50.384 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.383+0000 7f8716a00640 1 -- 172.21.15.152:0/3653817546 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f871815efa0 con 0x7f87180726f0 2024-04-20T11:28:50.384 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.383+0000 7f8717400640 1 --2- 172.21.15.152:0/3653817546 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8718145f60 0x7f87181415b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:28:50.384 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.383+0000 7f8716a00640 1 --2- 172.21.15.152:0/3653817546 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87180726f0 0x7f871815ea60 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f8708000c00 tx=0x7f87080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:50.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.384+0000 7f870ea00640 1 -- 172.21.15.152:0/3653817546 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8708040070 con 0x7f87180726f0 2024-04-20T11:28:50.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.384+0000 7f870ea00640 1 -- 172.21.15.152:0/3653817546 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f870803a830 con 0x7f87180726f0 2024-04-20T11:28:50.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.384+0000 7f870ea00640 1 -- 172.21.15.152:0/3653817546 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8708035040 con 0x7f87180726f0 2024-04-20T11:28:50.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.384+0000 7f871da00640 1 -- 172.21.15.152:0/3653817546 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f871815f250 con 0x7f87180726f0 2024-04-20T11:28:50.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.384+0000 7f871da00640 1 -- 172.21.15.152:0/3653817546 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f871815f810 con 0x7f87180726f0 2024-04-20T11:28:50.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.385+0000 7f871da00640 1 -- 172.21.15.152:0/3653817546 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87180689f0 con 0x7f87180726f0 2024-04-20T11:28:50.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.386+0000 7f870ea00640 1 -- 172.21.15.152:0/3653817546 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f870804b060 con 0x7f87180726f0 2024-04-20T11:28:50.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.387+0000 7f870ea00640 1 --2- 172.21.15.152:0/3653817546 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f86f406caa0 0x7f86f406ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:50.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.387+0000 7f870ea00640 1 -- 172.21.15.152:0/3653817546 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7f87080ad520 con 0x7f87180726f0 2024-04-20T11:28:50.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.387+0000 7f8716000640 1 --2- 172.21.15.152:0/3653817546 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f86f406caa0 0x7f86f406ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:50.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.388+0000 7f8716000640 1 --2- 172.21.15.152:0/3653817546 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f86f406caa0 0x7f86f406ef60 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f87180cdda0 tx=0x7f8700009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:50.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.391+0000 7f870ea00640 1 -- 172.21.15.152:0/3653817546 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f8708031800 con 0x7f87180726f0 2024-04-20T11:28:50.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.666+0000 7f871da00640 1 -- 172.21.15.152:0/3653817546 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade"]} v 0) -- 0x7f871815c270 con 0x7f87180726f0 2024-04-20T11:28:50.705 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.705+0000 7f870ea00640 1 -- 172.21.15.152:0/3653817546 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade"]}]=0 v125) ==== 158+0+0 (secure 0 0 0) 0x7f870807b9c0 con 0x7f87180726f0 2024-04-20T11:28:50.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.708+0000 7f871da00640 1 -- 172.21.15.152:0/3653817546 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f86f406caa0 msgr2=0x7f86f406ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:50.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.708+0000 7f871da00640 1 --2- 172.21.15.152:0/3653817546 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f86f406caa0 0x7f86f406ef60 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f87180cdda0 tx=0x7f8700009040 comp rx=0 tx=0).stop 2024-04-20T11:28:50.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.708+0000 7f871da00640 1 -- 172.21.15.152:0/3653817546 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87180726f0 msgr2=0x7f871815ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:50.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.708+0000 7f871da00640 1 --2- 172.21.15.152:0/3653817546 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87180726f0 0x7f871815ea60 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f8708000c00 tx=0x7f87080046c0 comp rx=0 tx=0).stop 2024-04-20T11:28:50.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.708+0000 7f871da00640 1 -- 172.21.15.152:0/3653817546 shutdown_connections 2024-04-20T11:28:50.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.708+0000 7f871da00640 1 --2- 172.21.15.152:0/3653817546 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f86f406caa0 0x7f86f406ef60 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:50.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.708+0000 7f871da00640 1 --2- 172.21.15.152:0/3653817546 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8718145f60 0x7f87181415b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:50.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.708+0000 7f871da00640 1 --2- 172.21.15.152:0/3653817546 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f87180cd0d0 0x7f87180ac310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:50.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.708+0000 7f871da00640 1 --2- 172.21.15.152:0/3653817546 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87180726f0 0x7f871815ea60 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:50.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.708+0000 7f871da00640 1 -- 172.21.15.152:0/3653817546 >> 172.21.15.152:0/3653817546 conn(0x7f8718067cf0 msgr2=0x7f871806f5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:50.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.709+0000 7f871da00640 1 -- 172.21.15.152:0/3653817546 shutdown_connections 2024-04-20T11:28:50.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.709+0000 7f871da00640 1 -- 172.21.15.152:0/3653817546 wait complete. 2024-04-20T11:28:50.728 INFO:tasks.cephfs_test_runner:Validate binary .meta file on legacy subvol root doesn't break the system ... ok 2024-04-20T11:28:50.728 INFO:tasks.cephfs_test_runner:Starting test: test_connection_expiration (tasks.cephfs.test_volumes.TestMisc) 2024-04-20T11:28:50.729 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:28:50.925 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.925+0000 7ff4d1600640 1 Processor -- start 2024-04-20T11:28:50.925 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.925+0000 7ff4d1600640 1 -- start start 2024-04-20T11:28:50.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.927+0000 7ff4d1600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4cc15b5e0 0x7ff4cc06eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:50.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.927+0000 7ff4d1600640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7ff4cc06fa50 0x7ff4cc0b50d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:50.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.927+0000 7ff4d1600640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4cc0b5770 con 0x7ff4cc0b4c50 2024-04-20T11:28:50.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.927+0000 7ff4d1600640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7ff4cc06de10 con 0x7ff4cc06fa50 2024-04-20T11:28:50.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.927+0000 7ff4d1600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff4cc06e060 con 0x7ff4cc15b5e0 2024-04-20T11:28:50.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.927+0000 7ff4cb400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7ff4cc06fa50 0x7ff4cc0b50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:50.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.927+0000 7ff4cb400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7ff4cc06fa50 0x7ff4cc0b50d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54076/0 (socket says 172.21.15.152:54076) 2024-04-20T11:28:50.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.927+0000 7ff4cb400640 1 -- 172.21.15.152:0/3007598043 learned_addr learned my addr 172.21.15.152:0/3007598043 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:50.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.928+0000 7ff4caa00640 1 --2- 172.21.15.152:0/3007598043 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4cc15b5e0 0x7ff4cc06eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:50.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.928+0000 7ff4cb400640 1 -- 172.21.15.152:0/3007598043 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4cc15b5e0 msgr2=0x7ff4cc06eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:50.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.928+0000 7ff4cb400640 1 --2- 172.21.15.152:0/3007598043 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4cc15b5e0 0x7ff4cc06eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:50.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.928+0000 7ff4cb400640 1 -- 172.21.15.152:0/3007598043 >> v1:172.21.15.175:6789/0 conn(0x7ff4cc0b4c50 legacy=0x7ff4cc06f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:50.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.928+0000 7ff4cb400640 1 -- 172.21.15.152:0/3007598043 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4cc06e340 con 0x7ff4cc06fa50 2024-04-20T11:28:50.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.928+0000 7ff4caa00640 1 --2- 172.21.15.152:0/3007598043 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4cc15b5e0 0x7ff4cc06eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:28:50.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.929+0000 7ff4cb400640 1 --2- 172.21.15.152:0/3007598043 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff4cc06fa50 0x7ff4cc0b50d0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc0b0090 tx=0x7ff4c402be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41a2a9cc35e9c2e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:50.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.929+0000 7ff4ca000640 1 -- 172.21.15.152:0/3007598043 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4c402ca30 con 0x7ff4cc06fa50 2024-04-20T11:28:50.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.929+0000 7ff4ca000640 1 -- 172.21.15.152:0/3007598043 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4c402cbd0 con 0x7ff4cc06fa50 2024-04-20T11:28:50.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.929+0000 7ff4ca000640 1 -- 172.21.15.152:0/3007598043 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4c40343f0 con 0x7ff4cc06fa50 2024-04-20T11:28:50.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.930+0000 7ff4d1600640 1 -- 172.21.15.152:0/3007598043 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff4cc06fa50 msgr2=0x7ff4cc0b50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:50.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.930+0000 7ff4d1600640 1 --2- 172.21.15.152:0/3007598043 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff4cc06fa50 0x7ff4cc0b50d0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc0b0090 tx=0x7ff4c402be90 comp rx=0 tx=0).stop 2024-04-20T11:28:50.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.930+0000 7ff4d1600640 1 -- 172.21.15.152:0/3007598043 shutdown_connections 2024-04-20T11:28:50.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.930+0000 7ff4d1600640 1 --2- 172.21.15.152:0/3007598043 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff4cc06fa50 0x7ff4cc0b50d0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:50.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.930+0000 7ff4d1600640 1 --2- 172.21.15.152:0/3007598043 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4cc15b5e0 0x7ff4cc06eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:50.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.930+0000 7ff4d1600640 1 -- 172.21.15.152:0/3007598043 >> 172.21.15.152:0/3007598043 conn(0x7ff4cc067cf0 msgr2=0x7ff4cc06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:50.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.930+0000 7ff4d1600640 1 -- 172.21.15.152:0/3007598043 shutdown_connections 2024-04-20T11:28:50.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.930+0000 7ff4d1600640 1 -- 172.21.15.152:0/3007598043 wait complete. 2024-04-20T11:28:50.931 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.932+0000 7ff4d1600640 1 Processor -- start 2024-04-20T11:28:50.931 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.932+0000 7ff4d1600640 1 -- start start 2024-04-20T11:28:50.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.933+0000 7ff4d1600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff4cc06fa50 0x7ff4cc0cf970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:50.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.933+0000 7ff4d1600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4cc0b4c50 0x7ff4cc141340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:50.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.933+0000 7ff4d1600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4cc15b5e0 0x7ff4cc145bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:50.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.933+0000 7ff4d1600640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7ff4cc0b42d0 con 0x7ff4cc06fa50 2024-04-20T11:28:50.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.933+0000 7ff4d1600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff4cc15c9d0 con 0x7ff4cc0b4c50 2024-04-20T11:28:50.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.933+0000 7ff4d1600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff4cc15cd70 con 0x7ff4cc15b5e0 2024-04-20T11:28:50.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.933+0000 7ff4caa00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff4cc06fa50 0x7ff4cc0cf970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:50.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.933+0000 7ff4caa00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff4cc06fa50 0x7ff4cc0cf970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54084/0 (socket says 172.21.15.152:54084) 2024-04-20T11:28:50.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.933+0000 7ff4caa00640 1 -- 172.21.15.152:0/101684291 learned_addr learned my addr 172.21.15.152:0/101684291 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:50.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.933+0000 7ff4c3e00640 1 --2- 172.21.15.152:0/101684291 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4cc0b4c50 0x7ff4cc141340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:50.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.933+0000 7ff4caa00640 1 -- 172.21.15.152:0/101684291 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4cc15b5e0 msgr2=0x7ff4cc145bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:50.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.933+0000 7ff4caa00640 1 --2- 172.21.15.152:0/101684291 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4cc15b5e0 0x7ff4cc145bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:50.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.933+0000 7ff4caa00640 1 -- 172.21.15.152:0/101684291 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4cc0b4c50 msgr2=0x7ff4cc141340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:50.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.933+0000 7ff4caa00640 1 --2- 172.21.15.152:0/101684291 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4cc0b4c50 0x7ff4cc141340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:50.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.933+0000 7ff4caa00640 1 -- 172.21.15.152:0/101684291 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4cc0b06d0 con 0x7ff4cc06fa50 2024-04-20T11:28:50.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.933+0000 7ff4c3e00640 1 --2- 172.21.15.152:0/101684291 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4cc0b4c50 0x7ff4cc141340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:28:50.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.934+0000 7ff4caa00640 1 --2- 172.21.15.152:0/101684291 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff4cc06fa50 0x7ff4cc0cf970 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc00c7f0 tx=0x7ff4bc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:50.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.934+0000 7ff4c2a00640 1 -- 172.21.15.152:0/101684291 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4bc015070 con 0x7ff4cc06fa50 2024-04-20T11:28:50.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.934+0000 7ff4c2a00640 1 -- 172.21.15.152:0/101684291 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4bc007820 con 0x7ff4cc06fa50 2024-04-20T11:28:50.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.934+0000 7ff4c2a00640 1 -- 172.21.15.152:0/101684291 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4bc007bd0 con 0x7ff4cc06fa50 2024-04-20T11:28:50.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.934+0000 7ff4d1600640 1 -- 172.21.15.152:0/101684291 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4cc0cebb0 con 0x7ff4cc06fa50 2024-04-20T11:28:50.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.934+0000 7ff4d1600640 1 -- 172.21.15.152:0/101684291 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4cc0cf170 con 0x7ff4cc06fa50 2024-04-20T11:28:50.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.936+0000 7ff4d1600640 1 -- 172.21.15.152:0/101684291 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4cc0689f0 con 0x7ff4cc06fa50 2024-04-20T11:28:50.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.936+0000 7ff4c2a00640 1 -- 172.21.15.152:0/101684291 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7ff4bc018070 con 0x7ff4cc06fa50 2024-04-20T11:28:50.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.937+0000 7ff4c2a00640 1 --2- 172.21.15.152:0/101684291 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff4a806caa0 0x7ff4a806ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:50.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.937+0000 7ff4c2a00640 1 -- 172.21.15.152:0/101684291 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7ff4bc08be10 con 0x7ff4cc06fa50 2024-04-20T11:28:50.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.938+0000 7ff4c3e00640 1 --2- 172.21.15.152:0/101684291 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff4a806caa0 0x7ff4a806ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:50.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.938+0000 7ff4c3e00640 1 --2- 172.21.15.152:0/101684291 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff4a806caa0 0x7ff4a806ef60 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc151980 tx=0x7ff4b4009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:50.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:50.942+0000 7ff4c2a00640 1 -- 172.21.15.152:0/101684291 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ff4bc08f030 con 0x7ff4cc06fa50 2024-04-20T11:28:51.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.143+0000 7ff4d1600640 1 -- 172.21.15.152:0/101684291 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff4cc06c450 con 0x7ff4cc06fa50 2024-04-20T11:28:51.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.145+0000 7ff4c2a00640 1 -- 172.21.15.152:0/101684291 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) ==== 74+0+13082 (secure 0 0 0) 0x7ff4bc05a110 con 0x7ff4cc06fa50 2024-04-20T11:28:51.145 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:28:51.145 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":37,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:28:49.692707+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.175:6835/3367117968":"2024-04-21T11:28:46.400052+0000","172.21.15.175:6833/3367117968":"2024-04-21T11:28:46.400052+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:28:51.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.147+0000 7ff4d1600640 1 -- 172.21.15.152:0/101684291 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff4a806caa0 msgr2=0x7ff4a806ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:51.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.147+0000 7ff4d1600640 1 --2- 172.21.15.152:0/101684291 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff4a806caa0 0x7ff4a806ef60 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc151980 tx=0x7ff4b4009040 comp rx=0 tx=0).stop 2024-04-20T11:28:51.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.147+0000 7ff4d1600640 1 -- 172.21.15.152:0/101684291 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff4cc06fa50 msgr2=0x7ff4cc0cf970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:51.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.147+0000 7ff4d1600640 1 --2- 172.21.15.152:0/101684291 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff4cc06fa50 0x7ff4cc0cf970 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc00c7f0 tx=0x7ff4bc00ccb0 comp rx=0 tx=0).stop 2024-04-20T11:28:51.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.148+0000 7ff4d1600640 1 -- 172.21.15.152:0/101684291 shutdown_connections 2024-04-20T11:28:51.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.148+0000 7ff4d1600640 1 --2- 172.21.15.152:0/101684291 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff4a806caa0 0x7ff4a806ef60 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:51.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.148+0000 7ff4d1600640 1 --2- 172.21.15.152:0/101684291 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4cc15b5e0 0x7ff4cc145bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:51.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.148+0000 7ff4d1600640 1 --2- 172.21.15.152:0/101684291 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4cc0b4c50 0x7ff4cc141340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:51.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.148+0000 7ff4d1600640 1 --2- 172.21.15.152:0/101684291 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff4cc06fa50 0x7ff4cc0cf970 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:51.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.148+0000 7ff4d1600640 1 -- 172.21.15.152:0/101684291 >> 172.21.15.152:0/101684291 conn(0x7ff4cc067cf0 msgr2=0x7ff4cc1583c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:51.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.148+0000 7ff4d1600640 1 -- 172.21.15.152:0/101684291 shutdown_connections 2024-04-20T11:28:51.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.148+0000 7ff4d1600640 1 -- 172.21.15.152:0/101684291 wait complete. 2024-04-20T11:28:51.166 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T11:26:47.246571+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T11:28:51.166 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T11:28:51.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.375+0000 7f44a3000640 1 Processor -- start 2024-04-20T11:28:51.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.376+0000 7f44a3000640 1 -- start start 2024-04-20T11:28:51.377 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.377+0000 7f44a3000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f449c15ea20 0x7f449c06dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:51.377 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.377+0000 7f44a3000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f449c06c4b0 0x7f449c0ad5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:51.377 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.377+0000 7f44a3000640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f449c0b5070 con 0x7f449c15e640 2024-04-20T11:28:51.377 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.377+0000 7f44a3000640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f449c06e1a0 con 0x7f449c06c4b0 2024-04-20T11:28:51.377 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.377+0000 7f44a3000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f449c06e480 con 0x7f449c15ea20 2024-04-20T11:28:51.378 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.378+0000 7f44a2600640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f449c06c4b0 0x7f449c0ad5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:51.378 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.378+0000 7f44a1c00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f449c15e640 0x7f449c06bda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:58316/0 (socket says 172.21.15.152:58316) 2024-04-20T11:28:51.378 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.378+0000 7f44a1c00640 1 -- 172.21.15.152:0/2994582079 learned_addr learned my addr 172.21.15.152:0/2994582079 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:28:51.378 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.378+0000 7f44a1200640 1 --2- 172.21.15.152:0/2994582079 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f449c15ea20 0x7f449c06dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:51.378 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.378+0000 7f44a2600640 1 -- 172.21.15.152:0/2994582079 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f449c15ea20 msgr2=0x7f449c06dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:51.378 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.378+0000 7f44a2600640 1 --2- 172.21.15.152:0/2994582079 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f449c15ea20 0x7f449c06dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:51.378 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.378+0000 7f44a2600640 1 -- 172.21.15.152:0/2994582079 >> v1:172.21.15.152:6789/0 conn(0x7f449c15e640 legacy=0x7f449c06bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:51.378 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.378+0000 7f44a2600640 1 -- 172.21.15.152:0/2994582079 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f449c06e760 con 0x7f449c06c4b0 2024-04-20T11:28:51.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.379+0000 7f44a2600640 1 --2- 172.21.15.152:0/2994582079 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f449c06c4b0 0x7f449c0ad5c0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f4494009d90 tx=0x7f449402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3946482d857e2db server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:51.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.380+0000 7f449be00640 1 -- 172.21.15.152:0/2994582079 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4494039070 con 0x7f449c06c4b0 2024-04-20T11:28:51.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.380+0000 7f449be00640 1 -- 172.21.15.152:0/2994582079 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f449402cc60 con 0x7f449c06c4b0 2024-04-20T11:28:51.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.380+0000 7f449be00640 1 -- 172.21.15.152:0/2994582079 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4494034660 con 0x7f449c06c4b0 2024-04-20T11:28:51.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.380+0000 7f44a3000640 1 -- 172.21.15.152:0/2994582079 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f449c06c4b0 msgr2=0x7f449c0ad5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:51.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.380+0000 7f44a3000640 1 --2- 172.21.15.152:0/2994582079 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f449c06c4b0 0x7f449c0ad5c0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f4494009d90 tx=0x7f449402beb0 comp rx=0 tx=0).stop 2024-04-20T11:28:51.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.380+0000 7f44a3000640 1 -- 172.21.15.152:0/2994582079 shutdown_connections 2024-04-20T11:28:51.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.380+0000 7f44a3000640 1 --2- 172.21.15.152:0/2994582079 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f449c15ea20 0x7f449c06dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:51.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.380+0000 7f44a3000640 1 --2- 172.21.15.152:0/2994582079 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f449c06c4b0 0x7f449c0ad5c0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:51.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.380+0000 7f44a3000640 1 -- 172.21.15.152:0/2994582079 >> 172.21.15.152:0/2994582079 conn(0x7f449c0b21e0 msgr2=0x7f449c0b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:51.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.380+0000 7f44a3000640 1 -- 172.21.15.152:0/2994582079 shutdown_connections 2024-04-20T11:28:51.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.381+0000 7f44a3000640 1 -- 172.21.15.152:0/2994582079 wait complete. 2024-04-20T11:28:51.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.382+0000 7f44a3000640 1 Processor -- start 2024-04-20T11:28:51.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.382+0000 7f44a3000640 1 -- start start 2024-04-20T11:28:51.384 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.383+0000 7f44a3000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f449c06c4b0 0x7f449c14b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:51.384 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.383+0000 7f44a3000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f449c15e640 0x7f449c1f7a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:51.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.383+0000 7f44a3000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f449c1f7fc0 0x7f449c1f8470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:51.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.383+0000 7f44a3000640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f449c06f410 con 0x7f449c1f7fc0 2024-04-20T11:28:51.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.383+0000 7f44a3000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f449c0ad0a0 con 0x7f449c15e640 2024-04-20T11:28:51.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.383+0000 7f44a3000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f449c06f590 con 0x7f449c06c4b0 2024-04-20T11:28:51.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.384+0000 7f44a2600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f449c1f7fc0 0x7f449c1f8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:51.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.384+0000 7f44a2600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f449c1f7fc0 0x7f449c1f8470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54110/0 (socket says 172.21.15.152:54110) 2024-04-20T11:28:51.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.384+0000 7f44a2600640 1 -- 172.21.15.152:0/3334145093 learned_addr learned my addr 172.21.15.152:0/3334145093 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:51.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.384+0000 7f44a1c00640 1 --2- 172.21.15.152:0/3334145093 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f449c06c4b0 0x7f449c14b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:51.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.384+0000 7f44a1200640 1 --2- 172.21.15.152:0/3334145093 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f449c15e640 0x7f449c1f7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:51.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.384+0000 7f44a2600640 1 -- 172.21.15.152:0/3334145093 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f449c06c4b0 msgr2=0x7f449c14b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:51.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.384+0000 7f44a2600640 1 --2- 172.21.15.152:0/3334145093 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f449c06c4b0 0x7f449c14b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:51.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.384+0000 7f44a2600640 1 -- 172.21.15.152:0/3334145093 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f449c15e640 msgr2=0x7f449c1f7a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:51.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.384+0000 7f44a2600640 1 --2- 172.21.15.152:0/3334145093 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f449c15e640 0x7f449c1f7a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:51.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.384+0000 7f44a2600640 1 -- 172.21.15.152:0/3334145093 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f449c14b720 con 0x7f449c1f7fc0 2024-04-20T11:28:51.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.384+0000 7f44a2600640 1 --2- 172.21.15.152:0/3334145093 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f449c1f7fc0 0x7f449c1f8470 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f4494009d90 tx=0x7f4494004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:51.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.385+0000 7f4499600640 1 -- 172.21.15.152:0/3334145093 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4494039050 con 0x7f449c1f7fc0 2024-04-20T11:28:51.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.385+0000 7f4499600640 1 -- 172.21.15.152:0/3334145093 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4494045070 con 0x7f449c1f7fc0 2024-04-20T11:28:51.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.385+0000 7f44a3000640 1 -- 172.21.15.152:0/3334145093 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f449c155aa0 con 0x7f449c1f7fc0 2024-04-20T11:28:51.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.385+0000 7f4499600640 1 -- 172.21.15.152:0/3334145093 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f449403e830 con 0x7f449c1f7fc0 2024-04-20T11:28:51.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.385+0000 7f44a3000640 1 -- 172.21.15.152:0/3334145093 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f449c157020 con 0x7f449c1f7fc0 2024-04-20T11:28:51.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.386+0000 7f44a3000640 1 -- 172.21.15.152:0/3334145093 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f449c075e60 con 0x7f449c1f7fc0 2024-04-20T11:28:51.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.387+0000 7f4499600640 1 -- 172.21.15.152:0/3334145093 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f449403e9d0 con 0x7f449c1f7fc0 2024-04-20T11:28:51.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.388+0000 7f4499600640 1 --2- 172.21.15.152:0/3334145093 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f447c06caa0 0x7f447c06ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:51.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.388+0000 7f4499600640 1 -- 172.21.15.152:0/3334145093 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7f44940ae940 con 0x7f449c1f7fc0 2024-04-20T11:28:51.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.388+0000 7f44a1c00640 1 --2- 172.21.15.152:0/3334145093 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f447c06caa0 0x7f447c06ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:51.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.389+0000 7f44a1c00640 1 --2- 172.21.15.152:0/3334145093 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f447c06caa0 0x7f447c06ef60 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4490000f60 tx=0x7f4490009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:51.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.393+0000 7f4499600640 1 -- 172.21.15.152:0/3334145093 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f449407ce70 con 0x7f449c1f7fc0 2024-04-20T11:28:51.451 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:28:51.458 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:28:51.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.595+0000 7f44a3000640 1 -- 172.21.15.152:0/3334145093 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f449c0aee50 con 0x7f449c1f7fc0 2024-04-20T11:28:51.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.596+0000 7f4499600640 1 -- 172.21.15.152:0/3334145093 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v37) ==== 93+0+10 (secure 0 0 0) 0x7f449407c770 con 0x7f449c1f7fc0 2024-04-20T11:28:51.595 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 1 2024-04-20T11:28:51.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.598+0000 7f44a3000640 1 -- 172.21.15.152:0/3334145093 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f447c06caa0 msgr2=0x7f447c06ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:51.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.598+0000 7f44a3000640 1 --2- 172.21.15.152:0/3334145093 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f447c06caa0 0x7f447c06ef60 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4490000f60 tx=0x7f4490009040 comp rx=0 tx=0).stop 2024-04-20T11:28:51.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.598+0000 7f44a3000640 1 -- 172.21.15.152:0/3334145093 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f449c1f7fc0 msgr2=0x7f449c1f8470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:51.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.598+0000 7f44a3000640 1 --2- 172.21.15.152:0/3334145093 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f449c1f7fc0 0x7f449c1f8470 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f4494009d90 tx=0x7f4494004a40 comp rx=0 tx=0).stop 2024-04-20T11:28:51.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.598+0000 7f44a3000640 1 -- 172.21.15.152:0/3334145093 shutdown_connections 2024-04-20T11:28:51.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.598+0000 7f44a3000640 1 --2- 172.21.15.152:0/3334145093 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f447c06caa0 0x7f447c06ef60 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:51.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.598+0000 7f44a3000640 1 --2- 172.21.15.152:0/3334145093 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f449c1f7fc0 0x7f449c1f8470 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:51.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.598+0000 7f44a3000640 1 --2- 172.21.15.152:0/3334145093 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f449c15e640 0x7f449c1f7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:51.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.598+0000 7f44a3000640 1 --2- 172.21.15.152:0/3334145093 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f449c06c4b0 0x7f449c14b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:51.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.598+0000 7f44a3000640 1 -- 172.21.15.152:0/3334145093 >> 172.21.15.152:0/3334145093 conn(0x7f449c0b21e0 msgr2=0x7f449c0ad4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:51.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.598+0000 7f44a3000640 1 -- 172.21.15.152:0/3334145093 shutdown_connections 2024-04-20T11:28:51.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.598+0000 7f44a3000640 1 -- 172.21.15.152:0/3334145093 wait complete. 2024-04-20T11:28:51.617 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration' 2024-04-20T11:28:51.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.865+0000 7f9a15000640 1 Processor -- start 2024-04-20T11:28:51.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.865+0000 7f9a15000640 1 -- start start 2024-04-20T11:28:51.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.867+0000 7f9a15000640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f9a10145f60 0x7f9a1006ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:51.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.867+0000 7f9a15000640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a10146340 con 0x7f9a100cd0d0 2024-04-20T11:28:51.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.867+0000 7f9a15000640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a1006f3c0 con 0x7f9a100726f0 2024-04-20T11:28:51.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.867+0000 7f9a15000640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f9a1006e9f0 con 0x7f9a10145f60 2024-04-20T11:28:51.867 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.867+0000 7f9a07e00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f9a100cd0d0 0x7f9a10071fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:58322/0 (socket says 172.21.15.152:58322) 2024-04-20T11:28:51.867 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.867+0000 7f9a07e00640 1 -- 172.21.15.152:0/3826407267 learned_addr learned my addr 172.21.15.152:0/3826407267 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:28:51.867 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.868+0000 7f9a0de00640 1 --2- 172.21.15.152:0/3826407267 >> v2:172.21.15.175:3300/0 conn(0x7f9a10145f60 0x7f9a1006ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:51.867 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.868+0000 7f9a0d400640 1 -- 172.21.15.152:0/3826407267 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1430606992 0 0) 0x7f9a10146340 con 0x7f9a100cd0d0 2024-04-20T11:28:51.867 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.868+0000 7f9a0d400640 1 -- 172.21.15.152:0/3826407267 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f99f4003520 con 0x7f9a100cd0d0 2024-04-20T11:28:51.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.868+0000 7f9a0d400640 1 -- 172.21.15.152:0/3826407267 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 986058397 0 0) 0x7f99f4003520 con 0x7f9a100cd0d0 2024-04-20T11:28:51.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.868+0000 7f9a0d400640 1 -- 172.21.15.152:0/3826407267 >> v2:172.21.15.175:3300/0 conn(0x7f9a10145f60 msgr2=0x7f9a1006ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:51.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.868+0000 7f9a0d400640 1 --2- 172.21.15.152:0/3826407267 >> v2:172.21.15.175:3300/0 conn(0x7f9a10145f60 0x7f9a1006ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:51.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.868+0000 7f9a0d400640 1 -- 172.21.15.152:0/3826407267 >> v1:172.21.15.175:6789/0 conn(0x7f9a100726f0 legacy=0x7f9a100cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:51.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.868+0000 7f9a0d400640 1 -- 172.21.15.152:0/3826407267 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a10072d50 con 0x7f9a100cd0d0 2024-04-20T11:28:51.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.868+0000 7f9a0d400640 1 -- 172.21.15.152:0/3826407267 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f99f8001670 con 0x7f9a100cd0d0 2024-04-20T11:28:51.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.868+0000 7f9a0d400640 1 -- 172.21.15.152:0/3826407267 >> v1:172.21.15.152:6789/0 conn(0x7f9a100cd0d0 legacy=0x7f9a10071fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:51.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.868+0000 7f9a0d400640 1 --2- 172.21.15.152:0/3826407267 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f99f4003a20 0x7f99f4003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:51.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.868+0000 7f9a0d400640 1 --2- 172.21.15.152:0/3826407267 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f99f4004800 0x7f99f4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:51.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.868+0000 7f9a0d400640 1 --2- 172.21.15.152:0/3826407267 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99f40051c0 0x7f99f400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:51.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.868+0000 7f9a0d400640 1 -- 172.21.15.152:0/3826407267 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f9a1006e9f0 con 0x7f99f4004800 2024-04-20T11:28:51.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.868+0000 7f9a0d400640 1 -- 172.21.15.152:0/3826407267 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f99f400c0f0 con 0x7f99f40051c0 2024-04-20T11:28:51.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.868+0000 7f9a0d400640 1 -- 172.21.15.152:0/3826407267 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f99f400c2f0 con 0x7f99f4003a20 2024-04-20T11:28:51.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.869+0000 7f9a07e00640 1 --2- 172.21.15.152:0/3826407267 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f99f4004800 0x7f99f4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:51.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.869+0000 7f9a0de00640 1 --2- 172.21.15.152:0/3826407267 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f99f4003a20 0x7f99f4003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:51.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.869+0000 7f9a0e800640 1 --2- 172.21.15.152:0/3826407267 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99f40051c0 0x7f99f400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:51.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.869+0000 7f9a07e00640 1 -- 172.21.15.152:0/3826407267 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f99f4003a20 msgr2=0x7f99f4003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:51.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.869+0000 7f9a07e00640 1 --2- 172.21.15.152:0/3826407267 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f99f4003a20 0x7f99f4003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:51.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.869+0000 7f9a07e00640 1 -- 172.21.15.152:0/3826407267 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99f40051c0 msgr2=0x7f99f400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:51.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.869+0000 7f9a07e00640 1 --2- 172.21.15.152:0/3826407267 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99f40051c0 0x7f99f400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:51.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.869+0000 7f9a07e00640 1 -- 172.21.15.152:0/3826407267 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99f4003520 con 0x7f99f4004800 2024-04-20T11:28:51.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.869+0000 7f9a0e800640 1 -- 172.21.15.152:0/3826407267 reap_dead start 2024-04-20T11:28:51.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.870+0000 7f9a07e00640 1 --2- 172.21.15.152:0/3826407267 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f99f4004800 0x7f99f4004c80 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f99f800c240 tx=0x7f99f4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f86b10c5988bba8a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:51.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.871+0000 7f9a0d400640 1 -- 172.21.15.152:0/3826407267 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f8007cf0 con 0x7f99f4004800 2024-04-20T11:28:51.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.871+0000 7f9a0d400640 1 -- 172.21.15.152:0/3826407267 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99f8030070 con 0x7f99f4004800 2024-04-20T11:28:51.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.871+0000 7f9a0d400640 1 -- 172.21.15.152:0/3826407267 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f8005d70 con 0x7f99f4004800 2024-04-20T11:28:51.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.871+0000 7f9a15000640 1 -- 172.21.15.152:0/3826407267 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f99f4004800 msgr2=0x7f99f4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:51.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.871+0000 7f9a15000640 1 --2- 172.21.15.152:0/3826407267 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f99f4004800 0x7f99f4004c80 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f99f800c240 tx=0x7f99f4005830 comp rx=0 tx=0).stop 2024-04-20T11:28:51.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.871+0000 7f9a15000640 1 -- 172.21.15.152:0/3826407267 shutdown_connections 2024-04-20T11:28:51.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.871+0000 7f9a15000640 1 --2- 172.21.15.152:0/3826407267 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f99f4004800 0x7f99f4004c80 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:51.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.871+0000 7f9a15000640 1 -- 172.21.15.152:0/3826407267 >> 172.21.15.152:0/3826407267 conn(0x7f9a10067cf0 msgr2=0x7f9a1006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:51.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.871+0000 7f9a15000640 1 -- 172.21.15.152:0/3826407267 shutdown_connections 2024-04-20T11:28:51.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.871+0000 7f9a15000640 1 -- 172.21.15.152:0/3826407267 wait complete. 2024-04-20T11:28:51.873 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.873+0000 7f9a15000640 1 Processor -- start 2024-04-20T11:28:51.873 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.873+0000 7f9a15000640 1 -- start start 2024-04-20T11:28:51.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.874+0000 7f9a15000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f99f4004800 0x7f9a10144c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:51.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.874+0000 7f9a15000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9a100ae420 0x7f9a100ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:51.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.874+0000 7f9a15000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a100c6090 0x7f9a100c6510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:51.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.874+0000 7f9a15000640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f9a1006e680 con 0x7f9a100ae420 2024-04-20T11:28:51.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.874+0000 7f9a15000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9a1006e3a0 con 0x7f9a100c6090 2024-04-20T11:28:51.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.874+0000 7f9a15000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f9a1006ee80 con 0x7f99f4004800 2024-04-20T11:28:51.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.874+0000 7f9a07e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9a100ae420 0x7f9a100ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:51.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.875+0000 7f9a07e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9a100ae420 0x7f9a100ae880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54116/0 (socket says 172.21.15.152:54116) 2024-04-20T11:28:51.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.875+0000 7f9a07e00640 1 -- 172.21.15.152:0/1115173200 learned_addr learned my addr 172.21.15.152:0/1115173200 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:51.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.875+0000 7f9a0e800640 1 --2- 172.21.15.152:0/1115173200 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a100c6090 0x7f9a100c6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:51.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.875+0000 7f9a0de00640 1 --2- 172.21.15.152:0/1115173200 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f99f4004800 0x7f9a10144c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:51.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.875+0000 7f9a07e00640 1 -- 172.21.15.152:0/1115173200 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f99f4004800 msgr2=0x7f9a10144c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:51.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.875+0000 7f9a07e00640 1 --2- 172.21.15.152:0/1115173200 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f99f4004800 0x7f9a10144c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:51.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.875+0000 7f9a07e00640 1 -- 172.21.15.152:0/1115173200 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a100c6090 msgr2=0x7f9a100c6510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:51.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.875+0000 7f9a07e00640 1 --2- 172.21.15.152:0/1115173200 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a100c6090 0x7f9a100c6510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:51.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.875+0000 7f9a07e00640 1 -- 172.21.15.152:0/1115173200 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a101f4900 con 0x7f9a100ae420 2024-04-20T11:28:51.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.875+0000 7f9a0de00640 1 --2- 172.21.15.152:0/1115173200 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f99f4004800 0x7f9a10144c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:28:51.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.875+0000 7f9a07e00640 1 --2- 172.21.15.152:0/1115173200 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9a100ae420 0x7f9a100ae880 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f99f800c240 tx=0x7f9a1015cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:51.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.876+0000 7f9a06000640 1 -- 172.21.15.152:0/1115173200 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f8045070 con 0x7f9a100ae420 2024-04-20T11:28:51.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.876+0000 7f9a06000640 1 -- 172.21.15.152:0/1115173200 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99f8004bf0 con 0x7f9a100ae420 2024-04-20T11:28:51.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.876+0000 7f9a06000640 1 -- 172.21.15.152:0/1115173200 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f8001c30 con 0x7f9a100ae420 2024-04-20T11:28:51.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.876+0000 7f9a15000640 1 -- 172.21.15.152:0/1115173200 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a100aef90 con 0x7f9a100ae420 2024-04-20T11:28:51.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.876+0000 7f9a15000640 1 -- 172.21.15.152:0/1115173200 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a100af550 con 0x7f9a100ae420 2024-04-20T11:28:51.877 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.877+0000 7f9a15000640 1 -- 172.21.15.152:0/1115173200 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a1015bd30 con 0x7f9a100ae420 2024-04-20T11:28:51.877 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.878+0000 7f9a06000640 1 -- 172.21.15.152:0/1115173200 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f99f8007070 con 0x7f9a100ae420 2024-04-20T11:28:51.879 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.879+0000 7f9a06000640 1 --2- 172.21.15.152:0/1115173200 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f99ec06caa0 0x7f99ec06ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:51.879 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.879+0000 7f9a06000640 1 -- 172.21.15.152:0/1115173200 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7f99f80acbc0 con 0x7f9a100ae420 2024-04-20T11:28:51.879 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.880+0000 7f9a0de00640 1 --2- 172.21.15.152:0/1115173200 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f99ec06caa0 0x7f99ec06ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:51.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.880+0000 7f9a0de00640 1 --2- 172.21.15.152:0/1115173200 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f99ec06caa0 0x7f99ec06ef60 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f9a10076cd0 tx=0x7f9a00008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:51.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:51.884+0000 7f9a06000640 1 -- 172.21.15.152:0/1115173200 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f9a1015bd30 con 0x7f9a100ae420 2024-04-20T11:28:52.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:52.145+0000 7f9a15000640 1 -- 172.21.15.152:0/1115173200 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration"]} v 0) -- 0x7f9a100689f0 con 0x7f9a100ae420 2024-04-20T11:28:52.721 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:52.720+0000 7f9a06000640 1 -- 172.21.15.152:0/1115173200 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration"]}]=0 v127) ==== 143+0+0 (secure 0 0 0) 0x7f99f807aa90 con 0x7f9a100ae420 2024-04-20T11:28:52.724 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:52.724+0000 7f9a15000640 1 -- 172.21.15.152:0/1115173200 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f99ec06caa0 msgr2=0x7f99ec06ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:52.725 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:52.724+0000 7f9a15000640 1 --2- 172.21.15.152:0/1115173200 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f99ec06caa0 0x7f99ec06ef60 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f9a10076cd0 tx=0x7f9a00008040 comp rx=0 tx=0).stop 2024-04-20T11:28:52.725 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:52.724+0000 7f9a15000640 1 -- 172.21.15.152:0/1115173200 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9a100ae420 msgr2=0x7f9a100ae880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:52.725 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:52.724+0000 7f9a15000640 1 --2- 172.21.15.152:0/1115173200 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9a100ae420 0x7f9a100ae880 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f99f800c240 tx=0x7f9a1015cc20 comp rx=0 tx=0).stop 2024-04-20T11:28:52.725 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:52.724+0000 7f9a15000640 1 -- 172.21.15.152:0/1115173200 shutdown_connections 2024-04-20T11:28:52.725 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:52.724+0000 7f9a15000640 1 --2- 172.21.15.152:0/1115173200 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f99ec06caa0 0x7f99ec06ef60 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:52.725 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:52.724+0000 7f9a15000640 1 --2- 172.21.15.152:0/1115173200 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a100c6090 0x7f9a100c6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:52.725 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:52.724+0000 7f9a15000640 1 --2- 172.21.15.152:0/1115173200 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9a100ae420 0x7f9a100ae880 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:52.725 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:52.724+0000 7f9a15000640 1 --2- 172.21.15.152:0/1115173200 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f99f4004800 0x7f9a10144c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:52.725 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:52.724+0000 7f9a15000640 1 -- 172.21.15.152:0/1115173200 >> 172.21.15.152:0/1115173200 conn(0x7f9a10067cf0 msgr2=0x7f9a100cc6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:52.725 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:52.725+0000 7f9a15000640 1 -- 172.21.15.152:0/1115173200 shutdown_connections 2024-04-20T11:28:52.726 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:52.725+0000 7f9a15000640 1 -- 172.21.15.152:0/1115173200 wait complete. 2024-04-20T11:28:52.749 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-20T11:28:52.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:52.949+0000 7ff279c00640 1 Processor -- start 2024-04-20T11:28:52.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:52.950+0000 7ff279c00640 1 -- start start 2024-04-20T11:28:52.951 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:52.951+0000 7ff279c00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7ff27415eb50 0x7ff2740d0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:52.951 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:52.951+0000 7ff279c00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ff274163230 0x7ff2740d2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:52.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:52.951+0000 7ff279c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff2740d1440 0x7ff2740d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:52.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:52.951+0000 7ff279c00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7ff2740d19c0 con 0x7ff27415eb50 2024-04-20T11:28:52.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:52.951+0000 7ff279c00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7ff274162750 con 0x7ff274163230 2024-04-20T11:28:52.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:52.951+0000 7ff279c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff274162a30 con 0x7ff2740d1440 2024-04-20T11:28:52.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:52.951+0000 7ff273e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7ff27415eb50 0x7ff2740d0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:52.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:52.951+0000 7ff273e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7ff27415eb50 0x7ff2740d0f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54118/0 (socket says 172.21.15.152:54118) 2024-04-20T11:28:52.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:52.951+0000 7ff273e00640 1 -- 172.21.15.152:0/3589807959 learned_addr learned my addr 172.21.15.152:0/3589807959 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:52.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:52.952+0000 7ff279200640 1 --2- 172.21.15.152:0/3589807959 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff2740d1440 0x7ff2740d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:52.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:52.952+0000 7ff273e00640 1 -- 172.21.15.152:0/3589807959 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff2740d1440 msgr2=0x7ff2740d33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:52.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:52.952+0000 7ff273e00640 1 --2- 172.21.15.152:0/3589807959 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff2740d1440 0x7ff2740d33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:52.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:52.952+0000 7ff273e00640 1 -- 172.21.15.152:0/3589807959 >> v2:172.21.15.175:3300/0 conn(0x7ff274163230 msgr2=0x7ff2740d2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:52.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:52.952+0000 7ff273400640 1 --2- 172.21.15.152:0/3589807959 >> v2:172.21.15.175:3300/0 conn(0x7ff274163230 0x7ff2740d2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:52.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:52.952+0000 7ff273e00640 1 --2- 172.21.15.152:0/3589807959 >> v2:172.21.15.175:3300/0 conn(0x7ff274163230 0x7ff2740d2e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:52.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:52.952+0000 7ff273e00640 1 -- 172.21.15.152:0/3589807959 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff274162d10 con 0x7ff27415eb50 2024-04-20T11:28:52.953 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:52.952+0000 7ff279200640 1 --2- 172.21.15.152:0/3589807959 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff2740d1440 0x7ff2740d33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:28:52.954 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:52.953+0000 7ff273e00640 1 --2- 172.21.15.152:0/3589807959 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff27415eb50 0x7ff2740d0f00 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7ff27408df30 tx=0x7ff26402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=823b9201a0f14a10 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:52.954 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:52.953+0000 7ff272a00640 1 -- 172.21.15.152:0/3589807959 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff26402e070 con 0x7ff27415eb50 2024-04-20T11:28:52.954 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:52.953+0000 7ff272a00640 1 -- 172.21.15.152:0/3589807959 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff264037070 con 0x7ff27415eb50 2024-04-20T11:28:52.954 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:52.953+0000 7ff272a00640 1 -- 172.21.15.152:0/3589807959 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff264031a00 con 0x7ff27415eb50 2024-04-20T11:28:52.954 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:52.954+0000 7ff279c00640 1 -- 172.21.15.152:0/3589807959 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff27415eb50 msgr2=0x7ff2740d0f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:52.954 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:52.954+0000 7ff279c00640 1 --2- 172.21.15.152:0/3589807959 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff27415eb50 0x7ff2740d0f00 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7ff27408df30 tx=0x7ff26402b880 comp rx=0 tx=0).stop 2024-04-20T11:28:52.954 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:52.954+0000 7ff279c00640 1 -- 172.21.15.152:0/3589807959 shutdown_connections 2024-04-20T11:28:52.954 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:52.954+0000 7ff279c00640 1 --2- 172.21.15.152:0/3589807959 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff2740d1440 0x7ff2740d33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:52.955 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:52.954+0000 7ff279c00640 1 --2- 172.21.15.152:0/3589807959 >> v2:172.21.15.175:3300/0 conn(0x7ff274163230 0x7ff2740d2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:52.955 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:52.954+0000 7ff279c00640 1 --2- 172.21.15.152:0/3589807959 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff27415eb50 0x7ff2740d0f00 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:52.955 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:52.954+0000 7ff279c00640 1 -- 172.21.15.152:0/3589807959 >> 172.21.15.152:0/3589807959 conn(0x7ff27408bfd0 msgr2=0x7ff2740d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:52.955 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:52.954+0000 7ff279c00640 1 -- 172.21.15.152:0/3589807959 shutdown_connections 2024-04-20T11:28:52.955 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:52.954+0000 7ff279c00640 1 -- 172.21.15.152:0/3589807959 wait complete. 2024-04-20T11:28:52.956 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:52.956+0000 7ff279c00640 1 Processor -- start 2024-04-20T11:28:52.956 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:52.957+0000 7ff279c00640 1 -- start start 2024-04-20T11:28:52.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:52.958+0000 7ff279c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff2740d1440 0x7ff274161fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:52.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:52.958+0000 7ff279c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff27415eb50 0x7ff2740b7130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:52.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:52.958+0000 7ff279c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff274163230 0x7ff2740b7670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:52.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:52.958+0000 7ff279c00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7ff27415f0b0 con 0x7ff27415eb50 2024-04-20T11:28:52.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:52.958+0000 7ff279c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff27415ef30 con 0x7ff274163230 2024-04-20T11:28:52.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:52.958+0000 7ff279c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff27415f230 con 0x7ff2740d1440 2024-04-20T11:28:52.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:52.958+0000 7ff273400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff27415eb50 0x7ff2740b7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:52.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:52.958+0000 7ff273400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff27415eb50 0x7ff2740b7130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54122/0 (socket says 172.21.15.152:54122) 2024-04-20T11:28:52.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:52.958+0000 7ff273400640 1 -- 172.21.15.152:0/1980917115 learned_addr learned my addr 172.21.15.152:0/1980917115 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:52.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:52.959+0000 7ff273e00640 1 --2- 172.21.15.152:0/1980917115 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff2740d1440 0x7ff274161fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:52.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:52.959+0000 7ff279200640 1 --2- 172.21.15.152:0/1980917115 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff274163230 0x7ff2740b7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:52.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:52.959+0000 7ff273400640 1 -- 172.21.15.152:0/1980917115 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff2740d1440 msgr2=0x7ff274161fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:52.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:52.959+0000 7ff273400640 1 --2- 172.21.15.152:0/1980917115 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff2740d1440 0x7ff274161fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:52.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:52.959+0000 7ff273400640 1 -- 172.21.15.152:0/1980917115 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff274163230 msgr2=0x7ff2740b7670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:52.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:52.959+0000 7ff273400640 1 --2- 172.21.15.152:0/1980917115 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff274163230 0x7ff2740b7670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:52.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:52.959+0000 7ff273400640 1 -- 172.21.15.152:0/1980917115 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff274162520 con 0x7ff27415eb50 2024-04-20T11:28:52.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:52.959+0000 7ff279200640 1 --2- 172.21.15.152:0/1980917115 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff274163230 0x7ff2740b7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:28:52.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:52.959+0000 7ff273400640 1 --2- 172.21.15.152:0/1980917115 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff27415eb50 0x7ff2740b7130 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7ff25c00cda0 tx=0x7ff25c00a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:52.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:52.959+0000 7ff26be00640 1 -- 172.21.15.152:0/1980917115 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff25c003b90 con 0x7ff27415eb50 2024-04-20T11:28:52.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:52.959+0000 7ff26be00640 1 -- 172.21.15.152:0/1980917115 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff25c003d30 con 0x7ff27415eb50 2024-04-20T11:28:52.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:52.959+0000 7ff26be00640 1 -- 172.21.15.152:0/1980917115 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff25c004240 con 0x7ff27415eb50 2024-04-20T11:28:52.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:52.959+0000 7ff279c00640 1 -- 172.21.15.152:0/1980917115 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2740b9410 con 0x7ff27415eb50 2024-04-20T11:28:52.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:52.960+0000 7ff279c00640 1 -- 172.21.15.152:0/1980917115 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2740b99d0 con 0x7ff27415eb50 2024-04-20T11:28:52.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:52.961+0000 7ff279c00640 1 -- 172.21.15.152:0/1980917115 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff274075e60 con 0x7ff27415eb50 2024-04-20T11:28:52.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:52.962+0000 7ff26be00640 1 -- 172.21.15.152:0/1980917115 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7ff25c00d040 con 0x7ff27415eb50 2024-04-20T11:28:52.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:52.963+0000 7ff26be00640 1 --2- 172.21.15.152:0/1980917115 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff25006caa0 0x7ff25006ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:52.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:52.963+0000 7ff26be00640 1 -- 172.21.15.152:0/1980917115 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7ff25c08c630 con 0x7ff27415eb50 2024-04-20T11:28:52.963 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:52.963+0000 7ff273e00640 1 --2- 172.21.15.152:0/1980917115 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff25006caa0 0x7ff25006ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:52.963 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:52.964+0000 7ff273e00640 1 --2- 172.21.15.152:0/1980917115 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff25006caa0 0x7ff25006ef60 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff264036000 tx=0x7ff26403d040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:52.967 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:52.967+0000 7ff26be00640 1 -- 172.21.15.152:0/1980917115 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ff25c014030 con 0x7ff27415eb50 2024-04-20T11:28:53.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:53.155+0000 7ff279c00640 1 -- 172.21.15.152:0/1980917115 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7ff27415e350 con 0x7ff27415eb50 2024-04-20T11:28:53.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:53.161+0000 7ff26be00640 1 -- 172.21.15.152:0/1980917115 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) ==== 122+0+0 (secure 0 0 0) 0x7ff25c008040 con 0x7ff27415eb50 2024-04-20T11:28:53.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:53.164+0000 7ff279c00640 1 -- 172.21.15.152:0/1980917115 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff25006caa0 msgr2=0x7ff25006ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:53.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:53.164+0000 7ff279c00640 1 --2- 172.21.15.152:0/1980917115 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff25006caa0 0x7ff25006ef60 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff264036000 tx=0x7ff26403d040 comp rx=0 tx=0).stop 2024-04-20T11:28:53.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:53.164+0000 7ff279c00640 1 -- 172.21.15.152:0/1980917115 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff27415eb50 msgr2=0x7ff2740b7130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:53.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:53.164+0000 7ff279c00640 1 --2- 172.21.15.152:0/1980917115 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff27415eb50 0x7ff2740b7130 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7ff25c00cda0 tx=0x7ff25c00a610 comp rx=0 tx=0).stop 2024-04-20T11:28:53.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:53.164+0000 7ff279c00640 1 -- 172.21.15.152:0/1980917115 shutdown_connections 2024-04-20T11:28:53.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:53.164+0000 7ff279c00640 1 --2- 172.21.15.152:0/1980917115 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff25006caa0 0x7ff25006ef60 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:53.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:53.164+0000 7ff279c00640 1 --2- 172.21.15.152:0/1980917115 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff274163230 0x7ff2740b7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:53.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:53.164+0000 7ff279c00640 1 --2- 172.21.15.152:0/1980917115 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff27415eb50 0x7ff2740b7130 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:53.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:53.164+0000 7ff279c00640 1 --2- 172.21.15.152:0/1980917115 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff2740d1440 0x7ff274161fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:53.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:53.164+0000 7ff279c00640 1 -- 172.21.15.152:0/1980917115 >> 172.21.15.152:0/1980917115 conn(0x7ff27408bfd0 msgr2=0x7ff2740b2720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:53.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:53.164+0000 7ff279c00640 1 -- 172.21.15.152:0/1980917115 shutdown_connections 2024-04-20T11:28:53.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:53.164+0000 7ff279c00640 1 -- 172.21.15.152:0/1980917115 wait complete. 2024-04-20T11:28:53.182 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-20T11:28:53.182 DEBUG:teuthology.orchestra.run.smithi152:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-20T11:28:53.271 DEBUG:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-20T11:28:53.387 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-20T11:28:53.387 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-20T11:28:53.387 DEBUG:teuthology.orchestra.run.smithi152:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:28:53.441 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-20T11:28:53.441 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T11:28:53.441 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:28:53.498 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-20T11:28:53.498 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-20T11:28:53.498 DEBUG:teuthology.orchestra.run.smithi152:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:28:53.553 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-20T11:28:53.553 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T11:28:53.553 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:28:53.611 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-20T11:28:53.611 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-20T11:28:53.611 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:28:53.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:53.821+0000 7f7b4fe00640 1 Processor -- start 2024-04-20T11:28:53.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:53.821+0000 7f7b4fe00640 1 -- start start 2024-04-20T11:28:53.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:53.823+0000 7f7b4fe00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f7b5015e920 0x7f7b500d0080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:53.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:53.823+0000 7f7b4fe00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f7b500d2b60 0x7f7b500d2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:53.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:53.823+0000 7f7b4fe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7b50162620 0x7f7b50162ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:53.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:53.823+0000 7f7b4fe00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f7b500d3500 con 0x7f7b5015e920 2024-04-20T11:28:53.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:53.823+0000 7f7b4fe00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f7b50163390 con 0x7f7b500d2b60 2024-04-20T11:28:53.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:53.823+0000 7f7b4fe00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7b500cf2c0 con 0x7f7b50162620 2024-04-20T11:28:53.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:53.823+0000 7f7b4ea00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f7b5015e920 0x7f7b500d0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:53.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:53.823+0000 7f7b4ea00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f7b5015e920 0x7f7b500d0080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54134/0 (socket says 172.21.15.152:54134) 2024-04-20T11:28:53.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:53.823+0000 7f7b4ea00640 1 -- 172.21.15.152:0/2524362146 learned_addr learned my addr 172.21.15.152:0/2524362146 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:53.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:53.824+0000 7f7b4f400640 1 --2- 172.21.15.152:0/2524362146 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7b50162620 0x7f7b50162ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:53.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:53.824+0000 7f7b47e00640 1 --2- 172.21.15.152:0/2524362146 >> v2:172.21.15.175:3300/0 conn(0x7f7b500d2b60 0x7f7b500d2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:53.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:53.824+0000 7f7b4ea00640 1 -- 172.21.15.152:0/2524362146 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7b50162620 msgr2=0x7f7b50162ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:53.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:53.824+0000 7f7b4ea00640 1 --2- 172.21.15.152:0/2524362146 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7b50162620 0x7f7b50162ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:53.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:53.824+0000 7f7b4ea00640 1 -- 172.21.15.152:0/2524362146 >> v2:172.21.15.175:3300/0 conn(0x7f7b500d2b60 msgr2=0x7f7b500d2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:53.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:53.824+0000 7f7b4ea00640 1 --2- 172.21.15.152:0/2524362146 >> v2:172.21.15.175:3300/0 conn(0x7f7b500d2b60 0x7f7b500d2fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:53.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:53.824+0000 7f7b4ea00640 1 -- 172.21.15.152:0/2524362146 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b500cf5a0 con 0x7f7b5015e920 2024-04-20T11:28:53.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:53.824+0000 7f7b47e00640 1 --2- 172.21.15.152:0/2524362146 >> v2:172.21.15.175:3300/0 conn(0x7f7b500d2b60 0x7f7b500d2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:28:53.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:53.825+0000 7f7b4ea00640 1 --2- 172.21.15.152:0/2524362146 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7b5015e920 0x7f7b500d0080 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f7b5006c5f0 tx=0x7f7b4002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa9a5077e7cc7de1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:53.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:53.825+0000 7f7b4e000640 1 -- 172.21.15.152:0/2524362146 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b40035020 con 0x7f7b5015e920 2024-04-20T11:28:53.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:53.825+0000 7f7b4e000640 1 -- 172.21.15.152:0/2524362146 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b40031870 con 0x7f7b5015e920 2024-04-20T11:28:53.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:53.825+0000 7f7b4e000640 1 -- 172.21.15.152:0/2524362146 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b40031c40 con 0x7f7b5015e920 2024-04-20T11:28:53.826 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:53.826+0000 7f7b4fe00640 1 -- 172.21.15.152:0/2524362146 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7b5015e920 msgr2=0x7f7b500d0080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:53.826 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:53.826+0000 7f7b4fe00640 1 --2- 172.21.15.152:0/2524362146 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7b5015e920 0x7f7b500d0080 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f7b5006c5f0 tx=0x7f7b4002b910 comp rx=0 tx=0).stop 2024-04-20T11:28:53.826 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:53.826+0000 7f7b4fe00640 1 -- 172.21.15.152:0/2524362146 shutdown_connections 2024-04-20T11:28:53.826 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:53.826+0000 7f7b4fe00640 1 --2- 172.21.15.152:0/2524362146 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7b50162620 0x7f7b50162ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:53.826 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:53.826+0000 7f7b4fe00640 1 --2- 172.21.15.152:0/2524362146 >> v2:172.21.15.175:3300/0 conn(0x7f7b500d2b60 0x7f7b500d2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:53.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:53.826+0000 7f7b4fe00640 1 --2- 172.21.15.152:0/2524362146 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7b5015e920 0x7f7b500d0080 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:53.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:53.826+0000 7f7b4fe00640 1 -- 172.21.15.152:0/2524362146 >> 172.21.15.152:0/2524362146 conn(0x7f7b5008bfd0 msgr2=0x7f7b500d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:53.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:53.826+0000 7f7b4fe00640 1 -- 172.21.15.152:0/2524362146 shutdown_connections 2024-04-20T11:28:53.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:53.826+0000 7f7b4fe00640 1 -- 172.21.15.152:0/2524362146 wait complete. 2024-04-20T11:28:53.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:53.828+0000 7f7b4fe00640 1 Processor -- start 2024-04-20T11:28:53.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:53.828+0000 7f7b4fe00640 1 -- start start 2024-04-20T11:28:53.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:53.829+0000 7f7b4fe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7b500d2b60 0x7f7b501f77c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:53.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:53.829+0000 7f7b4fe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7b5015e920 0x7f7b5015f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:53.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:53.829+0000 7f7b4fe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b50162620 0x7f7b500ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:53.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:53.829+0000 7f7b4fe00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f7b5015ed00 con 0x7f7b5015e920 2024-04-20T11:28:53.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:53.829+0000 7f7b4fe00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7b500cfb60 con 0x7f7b50162620 2024-04-20T11:28:53.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:53.829+0000 7f7b4fe00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7b50162ff0 con 0x7f7b500d2b60 2024-04-20T11:28:53.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:53.829+0000 7f7b47e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7b5015e920 0x7f7b5015f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:53.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:53.829+0000 7f7b47e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7b5015e920 0x7f7b5015f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54136/0 (socket says 172.21.15.152:54136) 2024-04-20T11:28:53.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:53.829+0000 7f7b47e00640 1 -- 172.21.15.152:0/2858720958 learned_addr learned my addr 172.21.15.152:0/2858720958 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:53.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:53.830+0000 7f7b4ea00640 1 --2- 172.21.15.152:0/2858720958 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7b500d2b60 0x7f7b501f77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:53.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:53.830+0000 7f7b4f400640 1 --2- 172.21.15.152:0/2858720958 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b50162620 0x7f7b500ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:53.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:53.830+0000 7f7b47e00640 1 -- 172.21.15.152:0/2858720958 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7b500d2b60 msgr2=0x7f7b501f77c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:53.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:53.830+0000 7f7b47e00640 1 --2- 172.21.15.152:0/2858720958 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7b500d2b60 0x7f7b501f77c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:53.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:53.830+0000 7f7b47e00640 1 -- 172.21.15.152:0/2858720958 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b50162620 msgr2=0x7f7b500ccc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:53.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:53.830+0000 7f7b47e00640 1 --2- 172.21.15.152:0/2858720958 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b50162620 0x7f7b500ccc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:53.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:53.830+0000 7f7b47e00640 1 -- 172.21.15.152:0/2858720958 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b50160170 con 0x7f7b5015e920 2024-04-20T11:28:53.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:53.830+0000 7f7b4f400640 1 --2- 172.21.15.152:0/2858720958 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b50162620 0x7f7b500ccc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:28:53.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:53.830+0000 7f7b47e00640 1 --2- 172.21.15.152:0/2858720958 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7b5015e920 0x7f7b5015f840 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f7b3800c7f0 tx=0x7f7b3800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:53.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:53.830+0000 7f7b46a00640 1 -- 172.21.15.152:0/2858720958 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b38015070 con 0x7f7b5015e920 2024-04-20T11:28:53.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:53.831+0000 7f7b46a00640 1 -- 172.21.15.152:0/2858720958 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b38007820 con 0x7f7b5015e920 2024-04-20T11:28:53.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:53.831+0000 7f7b46a00640 1 -- 172.21.15.152:0/2858720958 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b38007bd0 con 0x7f7b5015e920 2024-04-20T11:28:53.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:53.831+0000 7f7b4fe00640 1 -- 172.21.15.152:0/2858720958 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b50160450 con 0x7f7b5015e920 2024-04-20T11:28:53.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:53.831+0000 7f7b4fe00640 1 -- 172.21.15.152:0/2858720958 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b50160a10 con 0x7f7b5015e920 2024-04-20T11:28:53.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:53.832+0000 7f7b4fe00640 1 -- 172.21.15.152:0/2858720958 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b50160df0 con 0x7f7b5015e920 2024-04-20T11:28:53.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:53.833+0000 7f7b46a00640 1 -- 172.21.15.152:0/2858720958 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f7b38018070 con 0x7f7b5015e920 2024-04-20T11:28:53.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:53.834+0000 7f7b46a00640 1 --2- 172.21.15.152:0/2858720958 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f7b2c06caa0 0x7f7b2c06ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:53.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:53.834+0000 7f7b46a00640 1 -- 172.21.15.152:0/2858720958 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7f7b3808bdd0 con 0x7f7b5015e920 2024-04-20T11:28:53.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:53.834+0000 7f7b4ea00640 1 --2- 172.21.15.152:0/2858720958 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f7b2c06caa0 0x7f7b2c06ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:53.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:53.835+0000 7f7b4ea00640 1 --2- 172.21.15.152:0/2858720958 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f7b2c06caa0 0x7f7b2c06ef60 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f7b40036000 tx=0x7f7b4003c040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:53.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:53.838+0000 7f7b46a00640 1 -- 172.21.15.152:0/2858720958 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f7b50160df0 con 0x7f7b5015e920 2024-04-20T11:28:54.079 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:54.080+0000 7f7b4fe00640 1 -- 172.21.15.152:0/2858720958 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7b500d2940 con 0x7f7b5015e920 2024-04-20T11:28:54.080 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:54.080+0000 7f7b46a00640 1 -- 172.21.15.152:0/2858720958 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+3881 (secure 0 0 0) 0x7f7b3805a160 con 0x7f7b5015e920 2024-04-20T11:28:54.080 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:28:54.080 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":18,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4643,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6834/1871483322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1871483322},{"type":"v1","addr":"172.21.15.175:6834","nonce":1871483322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4786,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/2579061164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6835","nonce":2579061164},{"type":"v1","addr":"172.21.15.152:6837","nonce":2579061164}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4996,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/3497762024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6833","nonce":3497762024},{"type":"v1","addr":"172.21.15.175:6835","nonce":3497762024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":5208,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6836/3724129680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":3724129680},{"type":"v1","addr":"172.21.15.152:6836","nonce":3724129680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-20T11:28:54.080 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 18 2024-04-20T11:28:54.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:54.083+0000 7f7b4fe00640 1 -- 172.21.15.152:0/2858720958 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f7b2c06caa0 msgr2=0x7f7b2c06ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:54.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:54.083+0000 7f7b4fe00640 1 --2- 172.21.15.152:0/2858720958 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f7b2c06caa0 0x7f7b2c06ef60 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f7b40036000 tx=0x7f7b4003c040 comp rx=0 tx=0).stop 2024-04-20T11:28:54.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:54.083+0000 7f7b4fe00640 1 -- 172.21.15.152:0/2858720958 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7b5015e920 msgr2=0x7f7b5015f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:54.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:54.083+0000 7f7b4fe00640 1 --2- 172.21.15.152:0/2858720958 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7b5015e920 0x7f7b5015f840 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f7b3800c7f0 tx=0x7f7b3800ccb0 comp rx=0 tx=0).stop 2024-04-20T11:28:54.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:54.083+0000 7f7b4fe00640 1 -- 172.21.15.152:0/2858720958 shutdown_connections 2024-04-20T11:28:54.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:54.083+0000 7f7b4fe00640 1 --2- 172.21.15.152:0/2858720958 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f7b2c06caa0 0x7f7b2c06ef60 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:54.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:54.083+0000 7f7b4fe00640 1 --2- 172.21.15.152:0/2858720958 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b50162620 0x7f7b500ccc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:54.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:54.083+0000 7f7b4fe00640 1 --2- 172.21.15.152:0/2858720958 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7b5015e920 0x7f7b5015f840 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:54.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:54.083+0000 7f7b4fe00640 1 --2- 172.21.15.152:0/2858720958 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7b500d2b60 0x7f7b501f77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:54.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:54.083+0000 7f7b4fe00640 1 -- 172.21.15.152:0/2858720958 >> 172.21.15.152:0/2858720958 conn(0x7f7b5008bfd0 msgr2=0x7f7b500b2120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:54.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:54.083+0000 7f7b4fe00640 1 -- 172.21.15.152:0/2858720958 shutdown_connections 2024-04-20T11:28:54.084 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:54.083+0000 7f7b4fe00640 1 -- 172.21.15.152:0/2858720958 wait complete. 2024-04-20T11:28:54.102 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-20T11:28:54.102 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-20T11:28:54.102 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-20T11:28:54.102 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T11:28:54.102 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-20T11:28:54.103 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-20T11:28:54.103 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-20T11:28:54.103 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T11:28:54.103 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-20T11:28:54.103 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-20T11:28:54.103 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-20T11:28:54.103 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T11:28:54.104 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-20T11:28:54.104 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-20T11:28:54.104 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-20T11:28:54.104 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T11:28:54.118 INFO:tasks.ceph.mds.c:Stopped 2024-04-20T11:28:54.119 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-20T11:28:54.121 INFO:tasks.ceph.mds.a:Stopped 2024-04-20T11:28:54.121 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-20T11:28:54.126 INFO:tasks.ceph.mds.d:Stopped 2024-04-20T11:28:54.127 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-20T11:28:54.131 INFO:tasks.ceph.mds.b:Stopped 2024-04-20T11:28:54.131 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-20T11:28:54.168 INFO:tasks.ceph.mds.c:Started 2024-04-20T11:28:54.168 DEBUG:teuthology.parallel:result is None 2024-04-20T11:28:54.173 INFO:tasks.ceph.mds.a:Started 2024-04-20T11:28:54.174 DEBUG:teuthology.parallel:result is None 2024-04-20T11:28:54.177 INFO:tasks.ceph.mds.d:Started 2024-04-20T11:28:54.178 DEBUG:teuthology.parallel:result is None 2024-04-20T11:28:54.184 INFO:tasks.ceph.mds.b:Started 2024-04-20T11:28:54.184 DEBUG:teuthology.parallel:result is None 2024-04-20T11:28:54.184 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-20T11:28:54.309 INFO:tasks.ceph.mds.c.smithi152.stdout:starting mds.c at 2024-04-20T11:28:54.309 INFO:tasks.ceph.mds.a.smithi152.stdout:starting mds.a at 2024-04-20T11:28:54.332 INFO:tasks.ceph.mds.d.smithi175.stdout:starting mds.d at 2024-04-20T11:28:54.332 INFO:tasks.ceph.mds.b.smithi175.stdout:starting mds.b at 2024-04-20T11:28:54.334 INFO:tasks.ceph.mds.d.smithi175.stderr:2024-04-20T11:28:54.334+0000 7f1661200640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2024-04-20T11:28:54.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:54.358+0000 7fcfece00640 1 Processor -- start 2024-04-20T11:28:54.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:54.358+0000 7fcfece00640 1 -- start start 2024-04-20T11:28:54.359 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:54.359+0000 7fcfece00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fcfe815b2b0 0x7fcfe806f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:54.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:54.359+0000 7fcfece00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fcfe815fd60 0x7fcfe806f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:54.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:54.359+0000 7fcfece00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcfe80b4130 con 0x7fcfe815b690 2024-04-20T11:28:54.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:54.359+0000 7fcfece00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fcfe815f8d0 con 0x7fcfe815fd60 2024-04-20T11:28:54.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:54.359+0000 7fcfece00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fcfe815f280 con 0x7fcfe815b2b0 2024-04-20T11:28:54.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:54.360+0000 7fcfdfe00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fcfe815fd60 0x7fcfe806f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:54.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:54.360+0000 7fcfdfe00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fcfe815fd60 0x7fcfe806f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54192/0 (socket says 172.21.15.152:54192) 2024-04-20T11:28:54.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:54.360+0000 7fcfdfe00640 1 -- 172.21.15.152:0/989446027 learned_addr learned my addr 172.21.15.152:0/989446027 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:54.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:54.360+0000 7fcfe5e00640 1 --2- 172.21.15.152:0/989446027 >> v2:172.21.15.175:3300/0 conn(0x7fcfe815b2b0 0x7fcfe806f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:54.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:54.360+0000 7fcfdfe00640 1 -- 172.21.15.152:0/989446027 >> v2:172.21.15.175:3300/0 conn(0x7fcfe815b2b0 msgr2=0x7fcfe806f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:54.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:54.360+0000 7fcfdfe00640 1 --2- 172.21.15.152:0/989446027 >> v2:172.21.15.175:3300/0 conn(0x7fcfe815b2b0 0x7fcfe806f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:54.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:54.360+0000 7fcfdfe00640 1 -- 172.21.15.152:0/989446027 >> v1:172.21.15.175:6789/0 conn(0x7fcfe815b690 legacy=0x7fcfe80b3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:54.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:54.360+0000 7fcfdfe00640 1 -- 172.21.15.152:0/989446027 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfe815f560 con 0x7fcfe815fd60 2024-04-20T11:28:54.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:54.361+0000 7fcfdfe00640 1 --2- 172.21.15.152:0/989446027 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcfe815fd60 0x7fcfe806f880 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fcfe80d14c0 tx=0x7fcfd002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31d6115a72c30a14 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:54.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:54.362+0000 7fcfe5400640 1 -- 172.21.15.152:0/989446027 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd0034070 con 0x7fcfe815fd60 2024-04-20T11:28:54.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:54.362+0000 7fcfe5400640 1 -- 172.21.15.152:0/989446027 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfd0030860 con 0x7fcfe815fd60 2024-04-20T11:28:54.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:54.362+0000 7fcfe5400640 1 -- 172.21.15.152:0/989446027 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd0030c30 con 0x7fcfe815fd60 2024-04-20T11:28:54.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:54.362+0000 7fcfece00640 1 -- 172.21.15.152:0/989446027 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcfe815fd60 msgr2=0x7fcfe806f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:54.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:54.362+0000 7fcfece00640 1 --2- 172.21.15.152:0/989446027 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcfe815fd60 0x7fcfe806f880 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fcfe80d14c0 tx=0x7fcfd002b9a0 comp rx=0 tx=0).stop 2024-04-20T11:28:54.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:54.362+0000 7fcfece00640 1 -- 172.21.15.152:0/989446027 shutdown_connections 2024-04-20T11:28:54.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:54.362+0000 7fcfece00640 1 --2- 172.21.15.152:0/989446027 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcfe815fd60 0x7fcfe806f880 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:54.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:54.362+0000 7fcfece00640 1 --2- 172.21.15.152:0/989446027 >> v2:172.21.15.175:3300/0 conn(0x7fcfe815b2b0 0x7fcfe806f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:54.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:54.362+0000 7fcfece00640 1 -- 172.21.15.152:0/989446027 >> 172.21.15.152:0/989446027 conn(0x7fcfe8067cf0 msgr2=0x7fcfe806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:54.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:54.362+0000 7fcfece00640 1 -- 172.21.15.152:0/989446027 shutdown_connections 2024-04-20T11:28:54.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:54.362+0000 7fcfece00640 1 -- 172.21.15.152:0/989446027 wait complete. 2024-04-20T11:28:54.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:54.364+0000 7fcfece00640 1 Processor -- start 2024-04-20T11:28:54.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:54.364+0000 7fcfece00640 1 -- start start 2024-04-20T11:28:54.366 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:54.365+0000 7fcfece00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcfe815b2b0 0x7fcfe814c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:54.366 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:54.365+0000 7fcfece00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcfe815b690 0x7fcfe80af020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:54.366 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:54.365+0000 7fcfece00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcfe815fd60 0x7fcfe81550a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:54.366 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:54.365+0000 7fcfece00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fcfe80b4790 con 0x7fcfe815b2b0 2024-04-20T11:28:54.366 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:54.365+0000 7fcfece00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcfe80b3dd0 con 0x7fcfe815fd60 2024-04-20T11:28:54.366 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:54.365+0000 7fcfece00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fcfe80b4910 con 0x7fcfe815b690 2024-04-20T11:28:54.366 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:54.365+0000 7fcfe5e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcfe815b2b0 0x7fcfe814c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:54.366 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:54.365+0000 7fcfe6800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcfe815fd60 0x7fcfe81550a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:54.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:54.365+0000 7fcfe5e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcfe815b2b0 0x7fcfe814c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54200/0 (socket says 172.21.15.152:54200) 2024-04-20T11:28:54.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:54.365+0000 7fcfdfe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcfe815b690 0x7fcfe80af020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:54.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:54.365+0000 7fcfe5e00640 1 -- 172.21.15.152:0/97848429 learned_addr learned my addr 172.21.15.152:0/97848429 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:54.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:54.366+0000 7fcfe5e00640 1 -- 172.21.15.152:0/97848429 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcfe815b690 msgr2=0x7fcfe80af020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:54.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:54.366+0000 7fcfe5e00640 1 --2- 172.21.15.152:0/97848429 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcfe815b690 0x7fcfe80af020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:54.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:54.366+0000 7fcfe5e00640 1 -- 172.21.15.152:0/97848429 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcfe815fd60 msgr2=0x7fcfe81550a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:54.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:54.366+0000 7fcfe5e00640 1 --2- 172.21.15.152:0/97848429 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcfe815fd60 0x7fcfe81550a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:54.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:54.366+0000 7fcfe5e00640 1 -- 172.21.15.152:0/97848429 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfe814d2f0 con 0x7fcfe815b2b0 2024-04-20T11:28:54.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:54.366+0000 7fcfe5e00640 1 --2- 172.21.15.152:0/97848429 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcfe815b2b0 0x7fcfe814c9d0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fcfd800e9e0 tx=0x7fcfd800eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:54.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:54.366+0000 7fcfde000640 1 -- 172.21.15.152:0/97848429 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd8017070 con 0x7fcfe815b2b0 2024-04-20T11:28:54.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:54.366+0000 7fcfde000640 1 -- 172.21.15.152:0/97848429 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfd800c9d0 con 0x7fcfe815b2b0 2024-04-20T11:28:54.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:54.367+0000 7fcfece00640 1 -- 172.21.15.152:0/97848429 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfe81513a0 con 0x7fcfe815b2b0 2024-04-20T11:28:54.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:54.367+0000 7fcfde000640 1 -- 172.21.15.152:0/97848429 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd800cdc0 con 0x7fcfe815b2b0 2024-04-20T11:28:54.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:54.367+0000 7fcfece00640 1 -- 172.21.15.152:0/97848429 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfe8151960 con 0x7fcfe815b2b0 2024-04-20T11:28:54.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:54.368+0000 7fcfece00640 1 -- 172.21.15.152:0/97848429 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfe8151ba0 con 0x7fcfe815b2b0 2024-04-20T11:28:54.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:54.369+0000 7fcfde000640 1 -- 172.21.15.152:0/97848429 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7fcfd801a070 con 0x7fcfe815b2b0 2024-04-20T11:28:54.369 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:54.370+0000 7fcfde000640 1 --2- 172.21.15.152:0/97848429 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fcfc406caa0 0x7fcfc406ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:54.369 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:54.370+0000 7fcfde000640 1 -- 172.21.15.152:0/97848429 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7fcfd808bae0 con 0x7fcfe815b2b0 2024-04-20T11:28:54.369 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:54.370+0000 7fcfdfe00640 1 --2- 172.21.15.152:0/97848429 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fcfc406caa0 0x7fcfc406ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:54.370 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:54.371+0000 7fcfdfe00640 1 --2- 172.21.15.152:0/97848429 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fcfc406caa0 0x7fcfc406ef60 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fcfd0036000 tx=0x7fcfd003b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:54.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:54.374+0000 7fcfde000640 1 -- 172.21.15.152:0/97848429 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fcfe8151ba0 con 0x7fcfe815b2b0 2024-04-20T11:28:54.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:54.589+0000 7fcfece00640 1 -- 172.21.15.152:0/97848429 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fcfe80689f0 con 0x7fcfe815b2b0 2024-04-20T11:28:55.764 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:55.764+0000 7fcfde000640 1 -- 172.21.15.152:0/97848429 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v42) ==== 97+0+0 (secure 0 0 0) 0x7fcfd8059950 con 0x7fcfe815b2b0 2024-04-20T11:28:55.764 INFO:teuthology.orchestra.run.smithi152.stderr: removed all blocklist entries 2024-04-20T11:28:55.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:55.768+0000 7fcfece00640 1 -- 172.21.15.152:0/97848429 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fcfc406caa0 msgr2=0x7fcfc406ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:55.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:55.768+0000 7fcfece00640 1 --2- 172.21.15.152:0/97848429 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fcfc406caa0 0x7fcfc406ef60 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fcfd0036000 tx=0x7fcfd003b040 comp rx=0 tx=0).stop 2024-04-20T11:28:55.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:55.768+0000 7fcfece00640 1 -- 172.21.15.152:0/97848429 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcfe815b2b0 msgr2=0x7fcfe814c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:55.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:55.768+0000 7fcfece00640 1 --2- 172.21.15.152:0/97848429 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcfe815b2b0 0x7fcfe814c9d0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fcfd800e9e0 tx=0x7fcfd800eea0 comp rx=0 tx=0).stop 2024-04-20T11:28:55.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:55.769+0000 7fcfece00640 1 -- 172.21.15.152:0/97848429 shutdown_connections 2024-04-20T11:28:55.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:55.769+0000 7fcfece00640 1 --2- 172.21.15.152:0/97848429 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fcfc406caa0 0x7fcfc406ef60 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:55.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:55.769+0000 7fcfece00640 1 --2- 172.21.15.152:0/97848429 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcfe815fd60 0x7fcfe81550a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:55.769 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:55.769+0000 7fcfece00640 1 --2- 172.21.15.152:0/97848429 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcfe815b690 0x7fcfe80af020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:55.769 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:55.769+0000 7fcfece00640 1 --2- 172.21.15.152:0/97848429 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcfe815b2b0 0x7fcfe814c9d0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:55.769 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:55.769+0000 7fcfece00640 1 -- 172.21.15.152:0/97848429 >> 172.21.15.152:0/97848429 conn(0x7fcfe8067cf0 msgr2=0x7fcfe815a360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:55.769 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:55.769+0000 7fcfece00640 1 -- 172.21.15.152:0/97848429 shutdown_connections 2024-04-20T11:28:55.769 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:55.769+0000 7fcfece00640 1 -- 172.21.15.152:0/97848429 wait complete. 2024-04-20T11:28:55.789 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-20T11:28:55.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:55.989+0000 7f6c24400640 1 Processor -- start 2024-04-20T11:28:55.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:55.990+0000 7f6c24400640 1 -- start start 2024-04-20T11:28:55.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:55.991+0000 7f6c24400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f6c1c145f60 0x7f6c1c06ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:55.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:55.991+0000 7f6c24400640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f6c1c0cd0d0 0x7f6c1c0cd580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:55.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:55.991+0000 7f6c24400640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c1c146340 con 0x7f6c1c071fe0 2024-04-20T11:28:55.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:55.991+0000 7f6c24400640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f6c1c06e9f0 con 0x7f6c1c145f60 2024-04-20T11:28:55.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:55.991+0000 7f6c24400640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f6c1c06f3c0 con 0x7f6c1c0cd0d0 2024-04-20T11:28:55.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:55.992+0000 7f6c22400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f6c1c071fe0 0x7f6c1c072470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:42758/0 (socket says 172.21.15.152:42758) 2024-04-20T11:28:55.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:55.992+0000 7f6c21a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f6c1c145f60 0x7f6c1c06ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:55.995 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:55.992+0000 7f6c22400640 1 -- 172.21.15.152:0/194476014 learned_addr learned my addr 172.21.15.152:0/194476014 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:28:55.995 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:55.992+0000 7f6c21000640 1 --2- 172.21.15.152:0/194476014 >> v2:172.21.15.175:3300/0 conn(0x7f6c1c0cd0d0 0x7f6c1c0cd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:55.995 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:55.992+0000 7f6c21a00640 1 -- 172.21.15.152:0/194476014 >> v2:172.21.15.175:3300/0 conn(0x7f6c1c0cd0d0 msgr2=0x7f6c1c0cd580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:55.995 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:55.992+0000 7f6c21a00640 1 --2- 172.21.15.152:0/194476014 >> v2:172.21.15.175:3300/0 conn(0x7f6c1c0cd0d0 0x7f6c1c0cd580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:55.995 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:55.992+0000 7f6c21a00640 1 -- 172.21.15.152:0/194476014 >> v1:172.21.15.152:6789/0 conn(0x7f6c1c071fe0 legacy=0x7f6c1c072470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:55.995 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:55.992+0000 7f6c21a00640 1 -- 172.21.15.152:0/194476014 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c1c072d50 con 0x7f6c1c145f60 2024-04-20T11:28:55.995 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:55.993+0000 7f6c21a00640 1 --2- 172.21.15.152:0/194476014 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6c1c145f60 0x7f6c1c06ee80 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c06eb70 tx=0x7f6c1002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0b2737164ab35bf server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:55.995 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:55.993+0000 7f6c1be00640 1 -- 172.21.15.152:0/194476014 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c10035020 con 0x7f6c1c145f60 2024-04-20T11:28:55.995 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:55.993+0000 7f6c1be00640 1 -- 172.21.15.152:0/194476014 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c10031830 con 0x7f6c1c145f60 2024-04-20T11:28:55.995 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:55.994+0000 7f6c1be00640 1 -- 172.21.15.152:0/194476014 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c10031c00 con 0x7f6c1c145f60 2024-04-20T11:28:55.995 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:55.994+0000 7f6c24400640 1 -- 172.21.15.152:0/194476014 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6c1c145f60 msgr2=0x7f6c1c06ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:55.995 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:55.994+0000 7f6c24400640 1 --2- 172.21.15.152:0/194476014 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6c1c145f60 0x7f6c1c06ee80 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c06eb70 tx=0x7f6c1002b8d0 comp rx=0 tx=0).stop 2024-04-20T11:28:55.995 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:55.994+0000 7f6c24400640 1 -- 172.21.15.152:0/194476014 shutdown_connections 2024-04-20T11:28:55.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:55.994+0000 7f6c24400640 1 --2- 172.21.15.152:0/194476014 >> v2:172.21.15.175:3300/0 conn(0x7f6c1c0cd0d0 0x7f6c1c0cd580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:55.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:55.994+0000 7f6c24400640 1 --2- 172.21.15.152:0/194476014 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6c1c145f60 0x7f6c1c06ee80 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:55.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:55.994+0000 7f6c24400640 1 -- 172.21.15.152:0/194476014 >> 172.21.15.152:0/194476014 conn(0x7f6c1c067cf0 msgr2=0x7f6c1c06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:55.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:55.994+0000 7f6c24400640 1 -- 172.21.15.152:0/194476014 shutdown_connections 2024-04-20T11:28:55.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:55.994+0000 7f6c24400640 1 -- 172.21.15.152:0/194476014 wait complete. 2024-04-20T11:28:55.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:55.996+0000 7f6c24400640 1 Processor -- start 2024-04-20T11:28:55.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:55.996+0000 7f6c24400640 1 -- start start 2024-04-20T11:28:55.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:55.997+0000 7f6c24400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6c1c071fe0 0x7f6c1c1f65b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:55.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:55.997+0000 7f6c24400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6c1c0cd0d0 0x7f6c1c1f2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:55.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:55.997+0000 7f6c24400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6c1c145f60 0x7f6c1c144b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:55.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:55.997+0000 7f6c24400640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f6c1c06e680 con 0x7f6c1c071fe0 2024-04-20T11:28:55.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:55.997+0000 7f6c24400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6c1c06e3a0 con 0x7f6c1c0cd0d0 2024-04-20T11:28:55.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:55.997+0000 7f6c24400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6c1c0cf290 con 0x7f6c1c145f60 2024-04-20T11:28:55.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:55.997+0000 7f6c21a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6c1c071fe0 0x7f6c1c1f65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:55.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:55.997+0000 7f6c21a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6c1c071fe0 0x7f6c1c1f65b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:45040/0 (socket says 172.21.15.152:45040) 2024-04-20T11:28:55.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:55.997+0000 7f6c21a00640 1 -- 172.21.15.152:0/1372674314 learned_addr learned my addr 172.21.15.152:0/1372674314 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:55.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:55.998+0000 7f6c21000640 1 --2- 172.21.15.152:0/1372674314 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6c1c0cd0d0 0x7f6c1c1f2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:55.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:55.998+0000 7f6c21a00640 1 -- 172.21.15.152:0/1372674314 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6c1c145f60 msgr2=0x7f6c1c144b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:55.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:55.998+0000 7f6c21a00640 1 --2- 172.21.15.152:0/1372674314 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6c1c145f60 0x7f6c1c144b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:55.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:55.998+0000 7f6c21a00640 1 -- 172.21.15.152:0/1372674314 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6c1c0cd0d0 msgr2=0x7f6c1c1f2600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:55.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:55.998+0000 7f6c21a00640 1 --2- 172.21.15.152:0/1372674314 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6c1c0cd0d0 0x7f6c1c1f2600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:55.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:55.998+0000 7f6c21a00640 1 -- 172.21.15.152:0/1372674314 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c1c1450d0 con 0x7f6c1c071fe0 2024-04-20T11:28:55.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:55.998+0000 7f6c21a00640 1 --2- 172.21.15.152:0/1372674314 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6c1c071fe0 0x7f6c1c1f65b0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f6c10000c00 tx=0x7f6c100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:56.000 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:55.998+0000 7f6c19600640 1 -- 172.21.15.152:0/1372674314 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c10040070 con 0x7f6c1c071fe0 2024-04-20T11:28:56.000 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:55.998+0000 7f6c19600640 1 -- 172.21.15.152:0/1372674314 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c10039a20 con 0x7f6c1c071fe0 2024-04-20T11:28:56.000 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:55.998+0000 7f6c19600640 1 -- 172.21.15.152:0/1372674314 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c10035040 con 0x7f6c1c071fe0 2024-04-20T11:28:56.000 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:55.999+0000 7f6c24400640 1 -- 172.21.15.152:0/1372674314 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c1c1f2d70 con 0x7f6c1c071fe0 2024-04-20T11:28:56.000 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:55.999+0000 7f6c24400640 1 -- 172.21.15.152:0/1372674314 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c1c1f3330 con 0x7f6c1c071fe0 2024-04-20T11:28:56.000 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.000+0000 7f6c24400640 1 -- 172.21.15.152:0/1372674314 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c1c0689f0 con 0x7f6c1c071fe0 2024-04-20T11:28:56.001 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.001+0000 7f6c19600640 1 -- 172.21.15.152:0/1372674314 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f6c1004b060 con 0x7f6c1c071fe0 2024-04-20T11:28:56.001 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.002+0000 7f6c19600640 1 --2- 172.21.15.152:0/1372674314 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f6bfc06caa0 0x7f6bfc06ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:56.001 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.002+0000 7f6c19600640 1 -- 172.21.15.152:0/1372674314 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 4857+0+0 (secure 0 0 0) 0x7f6c100ad500 con 0x7f6c1c071fe0 2024-04-20T11:28:56.002 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.002+0000 7f6c21000640 1 --2- 172.21.15.152:0/1372674314 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f6bfc06caa0 0x7f6bfc06ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:56.002 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.003+0000 7f6c21000640 1 --2- 172.21.15.152:0/1372674314 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f6bfc06caa0 0x7f6bfc06ef60 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c0ccf10 tx=0x7f6c08008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:56.006 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.007+0000 7f6c19600640 1 -- 172.21.15.152:0/1372674314 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f6c1007b960 con 0x7f6c1c071fe0 2024-04-20T11:28:56.277 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.277+0000 7f6c24400640 1 -- 172.21.15.152:0/1372674314 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f6c1c15ff50 con 0x7f6c1c071fe0 2024-04-20T11:28:56.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.278+0000 7f6c19600640 1 -- 172.21.15.152:0/1372674314 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v9) ==== 80+0+5304 (secure 0 0 0) 0x7f6c1007b260 con 0x7f6c1c071fe0 2024-04-20T11:28:56.278 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:28:56.279 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-04-20T11:28:56.279 INFO:teuthology.orchestra.run.smithi152.stdout: "auth_dump": [ 2024-04-20T11:28:56.279 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:28:56.279 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "mds.a", 2024-04-20T11:28:56.279 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBCpiNmyFRJIBAAj2aIMXdb65nqRgKFDBLLzA==", 2024-04-20T11:28:56.279 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:28:56.279 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow", 2024-04-20T11:28:56.279 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow *", 2024-04-20T11:28:56.279 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow *", 2024-04-20T11:28:56.279 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:28:56.279 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:28:56.280 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:28:56.280 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:28:56.280 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "mds.b", 2024-04-20T11:28:56.280 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBCpiNmVPytOhAAzQkvNOJaNy+MeKETWhYLyg==", 2024-04-20T11:28:56.280 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:28:56.280 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow", 2024-04-20T11:28:56.280 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow *", 2024-04-20T11:28:56.280 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow *", 2024-04-20T11:28:56.280 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:28:56.280 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:28:56.280 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:28:56.281 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:28:56.281 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "mds.c", 2024-04-20T11:28:56.281 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBCpiNm5ViKLRAAg7VBQG4hFTLYUpJMpBAaVA==", 2024-04-20T11:28:56.281 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:28:56.281 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow", 2024-04-20T11:28:56.281 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow *", 2024-04-20T11:28:56.281 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow *", 2024-04-20T11:28:56.281 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:28:56.281 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:28:56.281 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:28:56.281 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:28:56.281 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "mds.d", 2024-04-20T11:28:56.281 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBDpiNmLWhaCxAA2e4qd/7+TKenJlWICIsvGg==", 2024-04-20T11:28:56.282 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:28:56.282 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow", 2024-04-20T11:28:56.282 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow *", 2024-04-20T11:28:56.282 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow *", 2024-04-20T11:28:56.282 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:28:56.282 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:28:56.282 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:28:56.282 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:28:56.282 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "osd.0", 2024-04-20T11:28:56.282 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBHpiNmpuhyDBAAsQXX9DNvWlo0UYh53lb+FA==", 2024-04-20T11:28:56.282 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:28:56.283 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow profile osd", 2024-04-20T11:28:56.283 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile osd", 2024-04-20T11:28:56.283 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:28:56.283 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:28:56.283 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:28:56.283 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:28:56.283 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "osd.1", 2024-04-20T11:28:56.283 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBKpiNmxBiLJhAAHon1adC8G2jbYoDcWdE9jA==", 2024-04-20T11:28:56.283 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:28:56.283 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow profile osd", 2024-04-20T11:28:56.283 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile osd", 2024-04-20T11:28:56.283 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:28:56.283 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:28:56.283 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:28:56.284 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:28:56.284 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "osd.2", 2024-04-20T11:28:56.284 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBNpiNmdn0zMhAAqwIO2kKahdSzTVh8/p3FCg==", 2024-04-20T11:28:56.284 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:28:56.284 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow profile osd", 2024-04-20T11:28:56.284 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile osd", 2024-04-20T11:28:56.284 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:28:56.284 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:28:56.284 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:28:56.284 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:28:56.284 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "osd.3", 2024-04-20T11:28:56.284 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBRpiNmTPCYAhAAHFNHX/xKn21jEFHDkHMjEw==", 2024-04-20T11:28:56.284 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:28:56.284 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow profile osd", 2024-04-20T11:28:56.285 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile osd", 2024-04-20T11:28:56.285 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:28:56.285 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:28:56.285 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:28:56.285 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:28:56.285 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "osd.4", 2024-04-20T11:28:56.285 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBXpiNmM8cVFxAAJl9Hu+MvlkEiMglutI89eg==", 2024-04-20T11:28:56.285 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:28:56.285 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow profile osd", 2024-04-20T11:28:56.285 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile osd", 2024-04-20T11:28:56.285 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:28:56.285 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:28:56.285 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:28:56.286 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:28:56.286 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "osd.5", 2024-04-20T11:28:56.286 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBapiNmnGxJMRAA90P0T+VsTA8LCTcexTVr7w==", 2024-04-20T11:28:56.286 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:28:56.286 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow profile osd", 2024-04-20T11:28:56.286 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile osd", 2024-04-20T11:28:56.286 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:28:56.286 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:28:56.286 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:28:56.286 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:28:56.286 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "osd.6", 2024-04-20T11:28:56.286 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBepiNmyLXWDxAAhtHD5h478mLovE5IejAFng==", 2024-04-20T11:28:56.286 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:28:56.286 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow profile osd", 2024-04-20T11:28:56.287 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile osd", 2024-04-20T11:28:56.287 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:28:56.287 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:28:56.287 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:28:56.287 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:28:56.287 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "osd.7", 2024-04-20T11:28:56.287 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBhpiNmB3jwHBAAqco9om6r2N/GqAV6rVKiaA==", 2024-04-20T11:28:56.287 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:28:56.287 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow profile osd", 2024-04-20T11:28:56.287 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile osd", 2024-04-20T11:28:56.287 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:28:56.287 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:28:56.287 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:28:56.287 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:28:56.288 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "client.0", 2024-04-20T11:28:56.288 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBDpiNmtsBWFhAArq9A7r33CcwgMmOU2pAGPw==", 2024-04-20T11:28:56.288 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:28:56.288 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow", 2024-04-20T11:28:56.288 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow r", 2024-04-20T11:28:56.288 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-20T11:28:56.288 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:28:56.288 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:28:56.288 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:28:56.288 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "client.1", 2024-04-20T11:28:56.288 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBDpiNm6aNnJxAAB431GSGytZ0IKWMh4DMalw==", 2024-04-20T11:28:56.288 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:28:56.288 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow", 2024-04-20T11:28:56.289 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow r", 2024-04-20T11:28:56.289 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow rw", 2024-04-20T11:28:56.289 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow rwx" 2024-04-20T11:28:56.289 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:28:56.289 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:28:56.289 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:28:56.289 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "client.2", 2024-04-20T11:28:56.289 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBDpiNm5QLtHRAAa2kHTDVdgSh/B4XGZaKa5Q==", 2024-04-20T11:28:56.289 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:28:56.289 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow", 2024-04-20T11:28:56.289 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow r", 2024-04-20T11:28:56.289 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow rw", 2024-04-20T11:28:56.289 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow rwx" 2024-04-20T11:28:56.289 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:28:56.290 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:28:56.290 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:28:56.290 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "client.3", 2024-04-20T11:28:56.290 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBDpiNmw95SMBAAdodzTuPp4WdKSEf+jI8Uuw==", 2024-04-20T11:28:56.290 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:28:56.290 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow", 2024-04-20T11:28:56.290 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow r", 2024-04-20T11:28:56.290 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow rw", 2024-04-20T11:28:56.290 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow rwx" 2024-04-20T11:28:56.290 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:28:56.290 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:28:56.290 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:28:56.290 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "client.admin", 2024-04-20T11:28:56.291 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBBpiNm3xz6HhAA82+N+trDW/AmBr3q8l4ECw==", 2024-04-20T11:28:56.291 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:28:56.291 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow *", 2024-04-20T11:28:56.291 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow *", 2024-04-20T11:28:56.291 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow *", 2024-04-20T11:28:56.291 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:28:56.291 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:28:56.291 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:28:56.291 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:28:56.291 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "mgr.x", 2024-04-20T11:28:56.291 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBCpiNmuhalFxAA9dTl/zSbUovCCZAanAipww==", 2024-04-20T11:28:56.291 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:28:56.291 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow *", 2024-04-20T11:28:56.291 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile mgr", 2024-04-20T11:28:56.291 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:28:56.292 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:28:56.292 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:28:56.292 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:28:56.292 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "mgr.y", 2024-04-20T11:28:56.292 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBCpiNmP1+5DRAAs6bVAEOr0i6FdPmSEyo/vQ==", 2024-04-20T11:28:56.292 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:28:56.292 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow *", 2024-04-20T11:28:56.292 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile mgr", 2024-04-20T11:28:56.292 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:28:56.292 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:28:56.292 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:28:56.292 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-04-20T11:28:56.292 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-04-20T11:28:56.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.281+0000 7f6c24400640 1 -- 172.21.15.152:0/1372674314 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f6bfc06caa0 msgr2=0x7f6bfc06ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:56.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.281+0000 7f6c24400640 1 --2- 172.21.15.152:0/1372674314 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f6bfc06caa0 0x7f6bfc06ef60 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c0ccf10 tx=0x7f6c08008040 comp rx=0 tx=0).stop 2024-04-20T11:28:56.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.281+0000 7f6c24400640 1 -- 172.21.15.152:0/1372674314 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6c1c071fe0 msgr2=0x7f6c1c1f65b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:56.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.281+0000 7f6c24400640 1 --2- 172.21.15.152:0/1372674314 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6c1c071fe0 0x7f6c1c1f65b0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f6c10000c00 tx=0x7f6c100046c0 comp rx=0 tx=0).stop 2024-04-20T11:28:56.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.281+0000 7f6c24400640 1 -- 172.21.15.152:0/1372674314 shutdown_connections 2024-04-20T11:28:56.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.281+0000 7f6c24400640 1 --2- 172.21.15.152:0/1372674314 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f6bfc06caa0 0x7f6bfc06ef60 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:56.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.281+0000 7f6c24400640 1 --2- 172.21.15.152:0/1372674314 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6c1c145f60 0x7f6c1c144b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:56.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.281+0000 7f6c24400640 1 --2- 172.21.15.152:0/1372674314 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6c1c0cd0d0 0x7f6c1c1f2600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:56.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.281+0000 7f6c24400640 1 --2- 172.21.15.152:0/1372674314 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6c1c071fe0 0x7f6c1c1f65b0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:56.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.282+0000 7f6c24400640 1 -- 172.21.15.152:0/1372674314 >> 172.21.15.152:0/1372674314 conn(0x7f6c1c067cf0 msgr2=0x7f6c1c15eaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:56.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.282+0000 7f6c24400640 1 -- 172.21.15.152:0/1372674314 shutdown_connections 2024-04-20T11:28:56.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.282+0000 7f6c24400640 1 -- 172.21.15.152:0/1372674314 wait complete. 2024-04-20T11:28:56.299 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:28:56.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.494+0000 7fb57ea00640 1 Processor -- start 2024-04-20T11:28:56.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.494+0000 7fb57ea00640 1 -- start start 2024-04-20T11:28:56.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.495+0000 7fb57ea00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fb578071cd0 0x7fb5780720b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:56.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.495+0000 7fb57ea00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5780725f0 con 0x7fb578162e50 2024-04-20T11:28:56.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.496+0000 7fb57ea00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5780b5510 con 0x7fb57806ea20 2024-04-20T11:28:56.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.496+0000 7fb57ea00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fb5780ceec0 con 0x7fb578071cd0 2024-04-20T11:28:56.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.496+0000 7fb57e000640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fb578162e50 0x7fb578163270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:42768/0 (socket says 172.21.15.152:42768) 2024-04-20T11:28:56.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.496+0000 7fb57e000640 1 -- 172.21.15.152:0/3332500758 learned_addr learned my addr 172.21.15.152:0/3332500758 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:28:56.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.496+0000 7fb57d600640 1 --2- 172.21.15.152:0/3332500758 >> v2:172.21.15.175:3300/0 conn(0x7fb578071cd0 0x7fb5780720b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:56.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.496+0000 7fb577e00640 1 -- 172.21.15.152:0/3332500758 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3856573933 0 0) 0x7fb5780725f0 con 0x7fb578162e50 2024-04-20T11:28:56.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.496+0000 7fb577e00640 1 -- 172.21.15.152:0/3332500758 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb560003520 con 0x7fb578162e50 2024-04-20T11:28:56.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.497+0000 7fb577e00640 1 -- 172.21.15.152:0/3332500758 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3731355675 0 0) 0x7fb560003520 con 0x7fb578162e50 2024-04-20T11:28:56.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.497+0000 7fb577e00640 1 -- 172.21.15.152:0/3332500758 >> v2:172.21.15.175:3300/0 conn(0x7fb578071cd0 msgr2=0x7fb5780720b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:56.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.497+0000 7fb577e00640 1 --2- 172.21.15.152:0/3332500758 >> v2:172.21.15.175:3300/0 conn(0x7fb578071cd0 0x7fb5780720b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:56.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.497+0000 7fb577e00640 1 -- 172.21.15.152:0/3332500758 >> v1:172.21.15.175:6789/0 conn(0x7fb57806ea20 legacy=0x7fb5780b4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:56.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.497+0000 7fb577e00640 1 -- 172.21.15.152:0/3332500758 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb57806df40 con 0x7fb578162e50 2024-04-20T11:28:56.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.497+0000 7fb577e00640 1 -- 172.21.15.152:0/3332500758 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7fb5700019f0 con 0x7fb578162e50 2024-04-20T11:28:56.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.497+0000 7fb577e00640 1 -- 172.21.15.152:0/3332500758 >> v1:172.21.15.152:6789/0 conn(0x7fb578162e50 legacy=0x7fb578163270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:56.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.497+0000 7fb577e00640 1 --2- 172.21.15.152:0/3332500758 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5600039b0 0x7fb560003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:56.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.497+0000 7fb577e00640 1 --2- 172.21.15.152:0/3332500758 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb560004800 0x7fb560004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:56.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.497+0000 7fb577e00640 1 --2- 172.21.15.152:0/3332500758 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb5600051c0 0x7fb56000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:56.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.497+0000 7fb577e00640 1 -- 172.21.15.152:0/3332500758 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fb5780ceec0 con 0x7fb5600051c0 2024-04-20T11:28:56.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.497+0000 7fb577e00640 1 -- 172.21.15.152:0/3332500758 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb56000c060 con 0x7fb5600039b0 2024-04-20T11:28:56.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.497+0000 7fb577e00640 1 -- 172.21.15.152:0/3332500758 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb56000c260 con 0x7fb560004800 2024-04-20T11:28:56.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.497+0000 7fb57e000640 1 --2- 172.21.15.152:0/3332500758 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb5600051c0 0x7fb56000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:56.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.498+0000 7fb57d600640 1 --2- 172.21.15.152:0/3332500758 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5600039b0 0x7fb560003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:56.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.498+0000 7fb57cc00640 1 --2- 172.21.15.152:0/3332500758 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb560004800 0x7fb560004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:56.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.498+0000 7fb57e000640 1 -- 172.21.15.152:0/3332500758 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb560004800 msgr2=0x7fb560004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:56.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.498+0000 7fb57e000640 1 --2- 172.21.15.152:0/3332500758 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb560004800 0x7fb560004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:56.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.498+0000 7fb57e000640 1 -- 172.21.15.152:0/3332500758 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5600039b0 msgr2=0x7fb560003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:56.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.498+0000 7fb57e000640 1 --2- 172.21.15.152:0/3332500758 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5600039b0 0x7fb560003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:56.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.498+0000 7fb57e000640 1 -- 172.21.15.152:0/3332500758 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb560003520 con 0x7fb5600051c0 2024-04-20T11:28:56.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.498+0000 7fb57cc00640 1 --2- 172.21.15.152:0/3332500758 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb560004800 0x7fb560004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:28:56.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.499+0000 7fb57e000640 1 -- 172.21.15.152:0/3332500758 reap_dead start 2024-04-20T11:28:56.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.499+0000 7fb57e000640 1 --2- 172.21.15.152:0/3332500758 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb5600051c0 0x7fb56000b9c0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fb57000a840 tx=0x7fb560005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27f29317ad643791 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:56.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.499+0000 7fb577e00640 1 -- 172.21.15.152:0/3332500758 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb57000a460 con 0x7fb5600051c0 2024-04-20T11:28:56.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.499+0000 7fb577e00640 1 -- 172.21.15.152:0/3332500758 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb570034650 con 0x7fb5600051c0 2024-04-20T11:28:56.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.499+0000 7fb577e00640 1 -- 172.21.15.152:0/3332500758 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb570034a40 con 0x7fb5600051c0 2024-04-20T11:28:56.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.500+0000 7fb57ea00640 1 -- 172.21.15.152:0/3332500758 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb5600051c0 msgr2=0x7fb56000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:56.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.500+0000 7fb57ea00640 1 --2- 172.21.15.152:0/3332500758 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb5600051c0 0x7fb56000b9c0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fb57000a840 tx=0x7fb560005830 comp rx=0 tx=0).stop 2024-04-20T11:28:56.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.500+0000 7fb57ea00640 1 -- 172.21.15.152:0/3332500758 shutdown_connections 2024-04-20T11:28:56.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.500+0000 7fb57ea00640 1 --2- 172.21.15.152:0/3332500758 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb5600051c0 0x7fb56000b9c0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:56.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.500+0000 7fb57ea00640 1 -- 172.21.15.152:0/3332500758 >> 172.21.15.152:0/3332500758 conn(0x7fb578067ae0 msgr2=0x7fb57806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:56.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.500+0000 7fb57ea00640 1 -- 172.21.15.152:0/3332500758 shutdown_connections 2024-04-20T11:28:56.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.500+0000 7fb57ea00640 1 -- 172.21.15.152:0/3332500758 wait complete. 2024-04-20T11:28:56.503 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.501+0000 7fb57ea00640 1 Processor -- start 2024-04-20T11:28:56.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.502+0000 7fb57ea00640 1 -- start start 2024-04-20T11:28:56.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.502+0000 7fb57ea00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb5600051c0 0x7fb57815ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:56.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.503+0000 7fb57ea00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5781f9440 0x7fb57815d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:56.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.503+0000 7fb57ea00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb57815a5f0 0x7fb5781f5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:56.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.503+0000 7fb57ea00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fb57806e500 con 0x7fb5600051c0 2024-04-20T11:28:56.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.503+0000 7fb57ea00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb57806dcf0 con 0x7fb5781f9440 2024-04-20T11:28:56.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.503+0000 7fb57ea00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb5780b4180 con 0x7fb57815a5f0 2024-04-20T11:28:56.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.503+0000 7fb57d600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb5600051c0 0x7fb57815ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:56.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.503+0000 7fb57d600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb5600051c0 0x7fb57815ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:45070/0 (socket says 172.21.15.152:45070) 2024-04-20T11:28:56.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.503+0000 7fb57d600640 1 -- 172.21.15.152:0/962256100 learned_addr learned my addr 172.21.15.152:0/962256100 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:56.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.503+0000 7fb57d600640 1 -- 172.21.15.152:0/962256100 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb57815a5f0 msgr2=0x7fb5781f5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:56.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.503+0000 7fb57d600640 1 --2- 172.21.15.152:0/962256100 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb57815a5f0 0x7fb5781f5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:56.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.503+0000 7fb57cc00640 1 --2- 172.21.15.152:0/962256100 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5781f9440 0x7fb57815d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:56.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.503+0000 7fb57d600640 1 -- 172.21.15.152:0/962256100 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5781f9440 msgr2=0x7fb57815d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:56.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.503+0000 7fb57d600640 1 --2- 172.21.15.152:0/962256100 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5781f9440 0x7fb57815d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:56.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.503+0000 7fb57d600640 1 -- 172.21.15.152:0/962256100 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb57815db10 con 0x7fb5600051c0 2024-04-20T11:28:56.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.504+0000 7fb57d600640 1 --2- 172.21.15.152:0/962256100 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb5600051c0 0x7fb57815ebb0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fb56c011570 tx=0x7fb56c011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:56.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.504+0000 7fb575600640 1 -- 172.21.15.152:0/962256100 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb56c015930 con 0x7fb5600051c0 2024-04-20T11:28:56.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.504+0000 7fb575600640 1 -- 172.21.15.152:0/962256100 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb56c015ad0 con 0x7fb5600051c0 2024-04-20T11:28:56.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.504+0000 7fb57ea00640 1 -- 172.21.15.152:0/962256100 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb57815ddf0 con 0x7fb5600051c0 2024-04-20T11:28:56.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.504+0000 7fb575600640 1 -- 172.21.15.152:0/962256100 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb56c00f460 con 0x7fb5600051c0 2024-04-20T11:28:56.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.504+0000 7fb57ea00640 1 -- 172.21.15.152:0/962256100 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb57815e3b0 con 0x7fb5600051c0 2024-04-20T11:28:56.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.505+0000 7fb57ea00640 1 -- 172.21.15.152:0/962256100 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb578075e60 con 0x7fb5600051c0 2024-04-20T11:28:56.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.506+0000 7fb575600640 1 -- 172.21.15.152:0/962256100 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7fb56c00f600 con 0x7fb5600051c0 2024-04-20T11:28:56.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.507+0000 7fb575600640 1 --2- 172.21.15.152:0/962256100 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fb55806cad0 0x7fb55806ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:56.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.507+0000 7fb575600640 1 -- 172.21.15.152:0/962256100 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 4857+0+0 (secure 0 0 0) 0x7fb56c08c480 con 0x7fb5600051c0 2024-04-20T11:28:56.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.507+0000 7fb57cc00640 1 --2- 172.21.15.152:0/962256100 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fb55806cad0 0x7fb55806ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:56.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.508+0000 7fb57cc00640 1 --2- 172.21.15.152:0/962256100 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fb55806cad0 0x7fb55806ef90 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fb564000f60 tx=0x7fb564009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:56.511 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.512+0000 7fb575600640 1 -- 172.21.15.152:0/962256100 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fb56c019030 con 0x7fb5600051c0 2024-04-20T11:28:56.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.696+0000 7fb57ea00640 1 -- 172.21.15.152:0/962256100 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb5780cff50 con 0x7fb5600051c0 2024-04-20T11:28:56.698 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.698+0000 7fb575600640 1 -- 172.21.15.152:0/962256100 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) ==== 74+0+12951 (secure 0 0 0) 0x7fb56c05a8e0 con 0x7fb5600051c0 2024-04-20T11:28:56.698 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:28:56.698 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":42,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:28:55.761139+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:28:56.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.701+0000 7fb57ea00640 1 -- 172.21.15.152:0/962256100 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fb55806cad0 msgr2=0x7fb55806ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:56.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.701+0000 7fb57ea00640 1 --2- 172.21.15.152:0/962256100 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fb55806cad0 0x7fb55806ef90 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fb564000f60 tx=0x7fb564009040 comp rx=0 tx=0).stop 2024-04-20T11:28:56.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.701+0000 7fb57ea00640 1 -- 172.21.15.152:0/962256100 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb5600051c0 msgr2=0x7fb57815ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:56.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.701+0000 7fb57ea00640 1 --2- 172.21.15.152:0/962256100 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb5600051c0 0x7fb57815ebb0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fb56c011570 tx=0x7fb56c011a30 comp rx=0 tx=0).stop 2024-04-20T11:28:56.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.701+0000 7fb57ea00640 1 -- 172.21.15.152:0/962256100 shutdown_connections 2024-04-20T11:28:56.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.701+0000 7fb57ea00640 1 --2- 172.21.15.152:0/962256100 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fb55806cad0 0x7fb55806ef90 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:56.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.701+0000 7fb57ea00640 1 --2- 172.21.15.152:0/962256100 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb57815a5f0 0x7fb5781f5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:56.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.701+0000 7fb57ea00640 1 --2- 172.21.15.152:0/962256100 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5781f9440 0x7fb57815d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:56.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.701+0000 7fb57ea00640 1 --2- 172.21.15.152:0/962256100 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb5600051c0 0x7fb57815ebb0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:56.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.701+0000 7fb57ea00640 1 -- 172.21.15.152:0/962256100 >> 172.21.15.152:0/962256100 conn(0x7fb578067ae0 msgr2=0x7fb5781620c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:56.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.701+0000 7fb57ea00640 1 -- 172.21.15.152:0/962256100 shutdown_connections 2024-04-20T11:28:56.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.701+0000 7fb57ea00640 1 -- 172.21.15.152:0/962256100 wait complete. 2024-04-20T11:28:56.718 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T11:26:47.246571+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T11:28:56.719 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T11:28:56.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.922+0000 7fe8caa00640 1 Processor -- start 2024-04-20T11:28:56.922 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.922+0000 7fe8caa00640 1 -- start start 2024-04-20T11:28:56.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.923+0000 7fe8caa00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fe8c4146000 0x7fe8c40b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:56.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.923+0000 7fe8caa00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8c40b2170 0x7fe8c40b25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:56.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.923+0000 7fe8caa00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fe8c40cd1b0 0x7fe8c40cd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:56.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.923+0000 7fe8caa00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fe8c40b4750 con 0x7fe8c40cd1b0 2024-04-20T11:28:56.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.923+0000 7fe8caa00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fe8c41463e0 con 0x7fe8c4146000 2024-04-20T11:28:56.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.923+0000 7fe8caa00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe8c40b3e20 con 0x7fe8c40b2170 2024-04-20T11:28:56.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.923+0000 7fe8c8a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fe8c40cd1b0 0x7fe8c40cd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:56.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.923+0000 7fe8c8a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fe8c40cd1b0 0x7fe8c40cd5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:45074/0 (socket says 172.21.15.152:45074) 2024-04-20T11:28:56.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.923+0000 7fe8c8a00640 1 -- 172.21.15.152:0/850327484 learned_addr learned my addr 172.21.15.152:0/850327484 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:56.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.924+0000 7fe8c3e00640 1 --2- 172.21.15.152:0/850327484 >> v2:172.21.15.175:3300/0 conn(0x7fe8c4146000 0x7fe8c40b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:56.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.924+0000 7fe8c8a00640 1 -- 172.21.15.152:0/850327484 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8c40b2170 msgr2=0x7fe8c40b25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:56.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.924+0000 7fe8c8a00640 1 --2- 172.21.15.152:0/850327484 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8c40b2170 0x7fe8c40b25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:56.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.924+0000 7fe8c8a00640 1 -- 172.21.15.152:0/850327484 >> v2:172.21.15.175:3300/0 conn(0x7fe8c4146000 msgr2=0x7fe8c40b4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:56.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.924+0000 7fe8c8a00640 1 --2- 172.21.15.152:0/850327484 >> v2:172.21.15.175:3300/0 conn(0x7fe8c4146000 0x7fe8c40b4be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:56.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.924+0000 7fe8c8a00640 1 -- 172.21.15.152:0/850327484 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8c40b4100 con 0x7fe8c40cd1b0 2024-04-20T11:28:56.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.925+0000 7fe8c8a00640 1 --2- 172.21.15.152:0/850327484 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe8c40cd1b0 0x7fe8c40cd5c0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fe8c4070190 tx=0x7fe8bc02be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e34d6629cfaa7fc9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:56.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.925+0000 7fe8c2a00640 1 -- 172.21.15.152:0/850327484 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8bc02c9f0 con 0x7fe8c40cd1b0 2024-04-20T11:28:56.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.925+0000 7fe8c2a00640 1 -- 172.21.15.152:0/850327484 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8bc02cb90 con 0x7fe8c40cd1b0 2024-04-20T11:28:56.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.925+0000 7fe8c2a00640 1 -- 172.21.15.152:0/850327484 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8bc0343f0 con 0x7fe8c40cd1b0 2024-04-20T11:28:56.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.926+0000 7fe8caa00640 1 -- 172.21.15.152:0/850327484 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe8c40cd1b0 msgr2=0x7fe8c40cd5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:56.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.926+0000 7fe8caa00640 1 --2- 172.21.15.152:0/850327484 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe8c40cd1b0 0x7fe8c40cd5c0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fe8c4070190 tx=0x7fe8bc02be50 comp rx=0 tx=0).stop 2024-04-20T11:28:56.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.926+0000 7fe8caa00640 1 -- 172.21.15.152:0/850327484 shutdown_connections 2024-04-20T11:28:56.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.926+0000 7fe8caa00640 1 --2- 172.21.15.152:0/850327484 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe8c40cd1b0 0x7fe8c40cd5c0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:56.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.926+0000 7fe8caa00640 1 --2- 172.21.15.152:0/850327484 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8c40b2170 0x7fe8c40b25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:56.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.926+0000 7fe8caa00640 1 --2- 172.21.15.152:0/850327484 >> v2:172.21.15.175:3300/0 conn(0x7fe8c4146000 0x7fe8c40b4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:56.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.926+0000 7fe8caa00640 1 -- 172.21.15.152:0/850327484 >> 172.21.15.152:0/850327484 conn(0x7fe8c406b980 msgr2=0x7fe8c406bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:56.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.926+0000 7fe8caa00640 1 -- 172.21.15.152:0/850327484 shutdown_connections 2024-04-20T11:28:56.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.926+0000 7fe8caa00640 1 -- 172.21.15.152:0/850327484 wait complete. 2024-04-20T11:28:56.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.928+0000 7fe8caa00640 1 Processor -- start 2024-04-20T11:28:56.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.928+0000 7fe8caa00640 1 -- start start 2024-04-20T11:28:56.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.929+0000 7fe8caa00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8c40b2170 0x7fe8c40af0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:56.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.929+0000 7fe8caa00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8c40cd1b0 0x7fe8c40ad8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:56.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.929+0000 7fe8caa00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe8c4146000 0x7fe8c40afbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:56.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.929+0000 7fe8caa00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c40cdc80 con 0x7fe8c4146000 2024-04-20T11:28:56.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.929+0000 7fe8caa00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c40cdb00 con 0x7fe8c40b2170 2024-04-20T11:28:56.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.929+0000 7fe8caa00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe8c415ccf0 con 0x7fe8c40cd1b0 2024-04-20T11:28:56.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.929+0000 7fe8c8a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe8c4146000 0x7fe8c40afbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:56.931 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.929+0000 7fe8c8a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe8c4146000 0x7fe8c40afbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:45078/0 (socket says 172.21.15.152:45078) 2024-04-20T11:28:56.931 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.929+0000 7fe8c8a00640 1 -- 172.21.15.152:0/1639618032 learned_addr learned my addr 172.21.15.152:0/1639618032 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:56.931 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.929+0000 7fe8c8a00640 1 -- 172.21.15.152:0/1639618032 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8c40cd1b0 msgr2=0x7fe8c40ad8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:56.931 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.929+0000 7fe8c8a00640 1 --2- 172.21.15.152:0/1639618032 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8c40cd1b0 0x7fe8c40ad8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:56.931 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.929+0000 7fe8c3e00640 1 --2- 172.21.15.152:0/1639618032 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8c40b2170 0x7fe8c40af0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:56.931 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.929+0000 7fe8c8a00640 1 -- 172.21.15.152:0/1639618032 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8c40b2170 msgr2=0x7fe8c40af0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:56.931 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.929+0000 7fe8c8a00640 1 --2- 172.21.15.152:0/1639618032 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8c40b2170 0x7fe8c40af0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:56.931 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.929+0000 7fe8c8a00640 1 -- 172.21.15.152:0/1639618032 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8c40ae100 con 0x7fe8c4146000 2024-04-20T11:28:56.931 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.930+0000 7fe8c8a00640 1 --2- 172.21.15.152:0/1639618032 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe8c4146000 0x7fe8c40afbe0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc036040 tx=0x7fe8bc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:56.932 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.930+0000 7fe8bbe00640 1 -- 172.21.15.152:0/1639618032 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8bc043020 con 0x7fe8c4146000 2024-04-20T11:28:56.932 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.930+0000 7fe8bbe00640 1 -- 172.21.15.152:0/1639618032 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8bc03e450 con 0x7fe8c4146000 2024-04-20T11:28:56.932 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.930+0000 7fe8bbe00640 1 -- 172.21.15.152:0/1639618032 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8bc03e820 con 0x7fe8c4146000 2024-04-20T11:28:56.932 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.930+0000 7fe8caa00640 1 -- 172.21.15.152:0/1639618032 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8c40ae3e0 con 0x7fe8c4146000 2024-04-20T11:28:56.932 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.930+0000 7fe8caa00640 1 -- 172.21.15.152:0/1639618032 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8c414ef10 con 0x7fe8c4146000 2024-04-20T11:28:56.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.931+0000 7fe8caa00640 1 -- 172.21.15.152:0/1639618032 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8c40689f0 con 0x7fe8c4146000 2024-04-20T11:28:56.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.932+0000 7fe8bbe00640 1 -- 172.21.15.152:0/1639618032 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7fe8bc046020 con 0x7fe8c4146000 2024-04-20T11:28:56.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.933+0000 7fe8bbe00640 1 --2- 172.21.15.152:0/1639618032 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe8a406caa0 0x7fe8a406ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:56.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.933+0000 7fe8bbe00640 1 -- 172.21.15.152:0/1639618032 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 4857+0+0 (secure 0 0 0) 0x7fe8bc0ae510 con 0x7fe8c4146000 2024-04-20T11:28:56.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.933+0000 7fe8c3e00640 1 --2- 172.21.15.152:0/1639618032 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe8a406caa0 0x7fe8a406ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:56.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.934+0000 7fe8c3e00640 1 --2- 172.21.15.152:0/1639618032 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe8a406caa0 0x7fe8a406ef60 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fe8c40b07d0 tx=0x7fe8b4008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:56.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:56.938+0000 7fe8bbe00640 1 -- 172.21.15.152:0/1639618032 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe8bc04d030 con 0x7fe8c4146000 2024-04-20T11:28:57.124 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.124+0000 7fe8caa00640 1 -- 172.21.15.152:0/1639618032 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe8c414f0e0 con 0x7fe8c4146000 2024-04-20T11:28:57.125 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.125+0000 7fe8bbe00640 1 -- 172.21.15.152:0/1639618032 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v42) ==== 93+0+10 (secure 0 0 0) 0x7fe8bc07c970 con 0x7fe8c4146000 2024-04-20T11:28:57.125 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 1 2024-04-20T11:28:57.128 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.128+0000 7fe8caa00640 1 -- 172.21.15.152:0/1639618032 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe8a406caa0 msgr2=0x7fe8a406ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:57.128 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.128+0000 7fe8caa00640 1 --2- 172.21.15.152:0/1639618032 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe8a406caa0 0x7fe8a406ef60 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fe8c40b07d0 tx=0x7fe8b4008040 comp rx=0 tx=0).stop 2024-04-20T11:28:57.128 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.128+0000 7fe8caa00640 1 -- 172.21.15.152:0/1639618032 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe8c4146000 msgr2=0x7fe8c40afbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:57.128 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.128+0000 7fe8caa00640 1 --2- 172.21.15.152:0/1639618032 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe8c4146000 0x7fe8c40afbe0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc036040 tx=0x7fe8bc004a40 comp rx=0 tx=0).stop 2024-04-20T11:28:57.128 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.128+0000 7fe8caa00640 1 -- 172.21.15.152:0/1639618032 shutdown_connections 2024-04-20T11:28:57.128 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.128+0000 7fe8caa00640 1 --2- 172.21.15.152:0/1639618032 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe8a406caa0 0x7fe8a406ef60 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:57.128 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.128+0000 7fe8caa00640 1 --2- 172.21.15.152:0/1639618032 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe8c4146000 0x7fe8c40afbe0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:57.128 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.128+0000 7fe8caa00640 1 --2- 172.21.15.152:0/1639618032 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8c40cd1b0 0x7fe8c40ad8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:57.128 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.128+0000 7fe8caa00640 1 --2- 172.21.15.152:0/1639618032 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8c40b2170 0x7fe8c40af0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:57.129 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.128+0000 7fe8caa00640 1 -- 172.21.15.152:0/1639618032 >> 172.21.15.152:0/1639618032 conn(0x7fe8c406b980 msgr2=0x7fe8c4072370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:57.129 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.128+0000 7fe8caa00640 1 -- 172.21.15.152:0/1639618032 shutdown_connections 2024-04-20T11:28:57.129 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.128+0000 7fe8caa00640 1 -- 172.21.15.152:0/1639618032 wait complete. 2024-04-20T11:28:57.146 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-20T11:28:57.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.345+0000 7f2dc6e00640 1 Processor -- start 2024-04-20T11:28:57.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.345+0000 7f2dc6e00640 1 -- start start 2024-04-20T11:28:57.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.347+0000 7f2dc6e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f2dc0071cd0 0x7f2dc00720b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:57.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.347+0000 7f2dc6e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2dc0162e50 0x7f2dc01632c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:57.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.347+0000 7f2dc6e00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2dc00725f0 con 0x7f2dc006ea20 2024-04-20T11:28:57.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.347+0000 7f2dc6e00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f2dc00ceec0 con 0x7f2dc0071cd0 2024-04-20T11:28:57.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.347+0000 7f2dc6e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2dc006dcf0 con 0x7f2dc0162e50 2024-04-20T11:28:57.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.347+0000 7f2dc5a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f2dc0071cd0 0x7f2dc00720b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:57.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.347+0000 7f2dc5a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f2dc0071cd0 0x7f2dc00720b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:45086/0 (socket says 172.21.15.152:45086) 2024-04-20T11:28:57.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.347+0000 7f2dc5a00640 1 -- 172.21.15.152:0/1010064691 learned_addr learned my addr 172.21.15.152:0/1010064691 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:57.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.348+0000 7f2dc6400640 1 --2- 172.21.15.152:0/1010064691 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2dc0162e50 0x7f2dc01632c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:57.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.348+0000 7f2dc5a00640 1 -- 172.21.15.152:0/1010064691 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2dc0162e50 msgr2=0x7f2dc01632c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:57.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.348+0000 7f2dc5a00640 1 --2- 172.21.15.152:0/1010064691 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2dc0162e50 0x7f2dc01632c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:57.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.348+0000 7f2dc5a00640 1 -- 172.21.15.152:0/1010064691 >> v1:172.21.15.175:6789/0 conn(0x7f2dc006ea20 legacy=0x7f2dc00b4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:57.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.348+0000 7f2dc5a00640 1 -- 172.21.15.152:0/1010064691 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dc00b5510 con 0x7f2dc0071cd0 2024-04-20T11:28:57.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.348+0000 7f2dc6400640 1 --2- 172.21.15.152:0/1010064691 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2dc0162e50 0x7f2dc01632c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:28:57.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.349+0000 7f2dc5a00640 1 --2- 172.21.15.152:0/1010064691 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2dc0071cd0 0x7f2dc00720b0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f2dc006ff00 tx=0x7f2db402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8f643647baeb377 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:57.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.349+0000 7f2dbfe00640 1 -- 172.21.15.152:0/1010064691 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2db4035020 con 0x7f2dc0071cd0 2024-04-20T11:28:57.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.350+0000 7f2dbfe00640 1 -- 172.21.15.152:0/1010064691 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2db4031830 con 0x7f2dc0071cd0 2024-04-20T11:28:57.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.350+0000 7f2dbfe00640 1 -- 172.21.15.152:0/1010064691 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2db4031c00 con 0x7f2dc0071cd0 2024-04-20T11:28:57.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.350+0000 7f2dc6e00640 1 -- 172.21.15.152:0/1010064691 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2dc0071cd0 msgr2=0x7f2dc00720b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:57.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.350+0000 7f2dc6e00640 1 --2- 172.21.15.152:0/1010064691 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2dc0071cd0 0x7f2dc00720b0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f2dc006ff00 tx=0x7f2db402b8d0 comp rx=0 tx=0).stop 2024-04-20T11:28:57.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.350+0000 7f2dc6e00640 1 -- 172.21.15.152:0/1010064691 shutdown_connections 2024-04-20T11:28:57.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.350+0000 7f2dc6e00640 1 --2- 172.21.15.152:0/1010064691 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2dc0162e50 0x7f2dc01632c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:57.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.350+0000 7f2dc6e00640 1 --2- 172.21.15.152:0/1010064691 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2dc0071cd0 0x7f2dc00720b0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:57.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.350+0000 7f2dc6e00640 1 -- 172.21.15.152:0/1010064691 >> 172.21.15.152:0/1010064691 conn(0x7f2dc0067ae0 msgr2=0x7f2dc006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:57.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.350+0000 7f2dc6e00640 1 -- 172.21.15.152:0/1010064691 shutdown_connections 2024-04-20T11:28:57.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.350+0000 7f2dc6e00640 1 -- 172.21.15.152:0/1010064691 wait complete. 2024-04-20T11:28:57.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.352+0000 7f2dc6e00640 1 Processor -- start 2024-04-20T11:28:57.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.352+0000 7f2dc6e00640 1 -- start start 2024-04-20T11:28:57.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.353+0000 7f2dc6e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2dc006ea20 0x7f2dc0151010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:57.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.353+0000 7f2dc6e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2dc0071cd0 0x7f2dc0156ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:57.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.353+0000 7f2dc6e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2dc0162e50 0x7f2dc0156ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:57.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.353+0000 7f2dc6e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f2dc006e500 con 0x7f2dc006ea20 2024-04-20T11:28:57.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.353+0000 7f2dc6e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2dc006e220 con 0x7f2dc0071cd0 2024-04-20T11:28:57.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.353+0000 7f2dc6e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2dc00b4180 con 0x7f2dc0162e50 2024-04-20T11:28:57.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.353+0000 7f2dc5a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2dc006ea20 0x7f2dc0151010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:57.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.353+0000 7f2dc5a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2dc006ea20 0x7f2dc0151010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:45100/0 (socket says 172.21.15.152:45100) 2024-04-20T11:28:57.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.353+0000 7f2dc5a00640 1 -- 172.21.15.152:0/367663662 learned_addr learned my addr 172.21.15.152:0/367663662 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:57.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.354+0000 7f2dc5000640 1 --2- 172.21.15.152:0/367663662 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2dc0071cd0 0x7f2dc0156ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:57.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.354+0000 7f2dc6400640 1 --2- 172.21.15.152:0/367663662 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2dc0162e50 0x7f2dc0156ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:57.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.354+0000 7f2dc5a00640 1 -- 172.21.15.152:0/367663662 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2dc0162e50 msgr2=0x7f2dc0156ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:57.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.354+0000 7f2dc5a00640 1 --2- 172.21.15.152:0/367663662 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2dc0162e50 0x7f2dc0156ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:57.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.354+0000 7f2dc5a00640 1 -- 172.21.15.152:0/367663662 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2dc0071cd0 msgr2=0x7f2dc0156ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:57.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.354+0000 7f2dc5a00640 1 --2- 172.21.15.152:0/367663662 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2dc0071cd0 0x7f2dc0156ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:57.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.354+0000 7f2dc5a00640 1 -- 172.21.15.152:0/367663662 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dc0158880 con 0x7f2dc006ea20 2024-04-20T11:28:57.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.354+0000 7f2dc5000640 1 --2- 172.21.15.152:0/367663662 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2dc0071cd0 0x7f2dc0156ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:28:57.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.354+0000 7f2dc5a00640 1 --2- 172.21.15.152:0/367663662 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2dc006ea20 0x7f2dc0151010 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f2db4000c00 tx=0x7f2db40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:57.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.354+0000 7f2dbd600640 1 -- 172.21.15.152:0/367663662 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2db4040070 con 0x7f2dc006ea20 2024-04-20T11:28:57.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.355+0000 7f2dbd600640 1 -- 172.21.15.152:0/367663662 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2db40399c0 con 0x7f2dc006ea20 2024-04-20T11:28:57.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.355+0000 7f2dbd600640 1 -- 172.21.15.152:0/367663662 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2db4035040 con 0x7f2dc006ea20 2024-04-20T11:28:57.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.355+0000 7f2dc6e00640 1 -- 172.21.15.152:0/367663662 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2dc0157530 con 0x7f2dc006ea20 2024-04-20T11:28:57.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.355+0000 7f2dc6e00640 1 -- 172.21.15.152:0/367663662 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2dc0157ac0 con 0x7f2dc006ea20 2024-04-20T11:28:57.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.356+0000 7f2dc6e00640 1 -- 172.21.15.152:0/367663662 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2dc0075e60 con 0x7f2dc006ea20 2024-04-20T11:28:57.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.357+0000 7f2dbd600640 1 -- 172.21.15.152:0/367663662 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f2db4038020 con 0x7f2dc006ea20 2024-04-20T11:28:57.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.358+0000 7f2dbd600640 1 --2- 172.21.15.152:0/367663662 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f2da006caa0 0x7f2da006ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:57.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.358+0000 7f2dbd600640 1 -- 172.21.15.152:0/367663662 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 4857+0+0 (secure 0 0 0) 0x7f2db40ad710 con 0x7f2dc006ea20 2024-04-20T11:28:57.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.358+0000 7f2dc5000640 1 --2- 172.21.15.152:0/367663662 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f2da006caa0 0x7f2da006ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:57.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.359+0000 7f2dc5000640 1 --2- 172.21.15.152:0/367663662 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f2da006caa0 0x7f2da006ef60 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f2dc015fe70 tx=0x7f2dac009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:57.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.362+0000 7f2dbd600640 1 -- 172.21.15.152:0/367663662 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f2db407bae0 con 0x7f2dc006ea20 2024-04-20T11:28:57.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.551+0000 7f2dc6e00640 1 -- 172.21.15.152:0/367663662 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f2dc00b27d0 con 0x7f2dc006ea20 2024-04-20T11:28:57.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.552+0000 7f2dbd600640 1 -- 172.21.15.152:0/367663662 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v42) ==== 84+0+65 (secure 0 0 0) 0x7f2db407b3e0 con 0x7f2dc006ea20 2024-04-20T11:28:57.552 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:28:57.552 INFO:teuthology.orchestra.run.smithi152.stdout:[ 2024-04-20T11:28:57.552 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:28:57.552 INFO:teuthology.orchestra.run.smithi152.stdout: "poolnum": 1, 2024-04-20T11:28:57.552 INFO:teuthology.orchestra.run.smithi152.stdout: "poolname": ".mgr" 2024-04-20T11:28:57.553 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:28:57.553 INFO:teuthology.orchestra.run.smithi152.stdout:] 2024-04-20T11:28:57.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.555+0000 7f2dc6e00640 1 -- 172.21.15.152:0/367663662 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f2da006caa0 msgr2=0x7f2da006ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:57.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.555+0000 7f2dc6e00640 1 --2- 172.21.15.152:0/367663662 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f2da006caa0 0x7f2da006ef60 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f2dc015fe70 tx=0x7f2dac009040 comp rx=0 tx=0).stop 2024-04-20T11:28:57.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.555+0000 7f2dc6e00640 1 -- 172.21.15.152:0/367663662 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2dc006ea20 msgr2=0x7f2dc0151010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:57.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.555+0000 7f2dc6e00640 1 --2- 172.21.15.152:0/367663662 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2dc006ea20 0x7f2dc0151010 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f2db4000c00 tx=0x7f2db40046c0 comp rx=0 tx=0).stop 2024-04-20T11:28:57.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.555+0000 7f2dc6e00640 1 -- 172.21.15.152:0/367663662 shutdown_connections 2024-04-20T11:28:57.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.555+0000 7f2dc6e00640 1 --2- 172.21.15.152:0/367663662 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f2da006caa0 0x7f2da006ef60 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:57.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.555+0000 7f2dc6e00640 1 --2- 172.21.15.152:0/367663662 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2dc0162e50 0x7f2dc0156ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:57.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.555+0000 7f2dc6e00640 1 --2- 172.21.15.152:0/367663662 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2dc0071cd0 0x7f2dc0156ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:57.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.555+0000 7f2dc6e00640 1 --2- 172.21.15.152:0/367663662 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2dc006ea20 0x7f2dc0151010 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:57.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.555+0000 7f2dc6e00640 1 -- 172.21.15.152:0/367663662 >> 172.21.15.152:0/367663662 conn(0x7f2dc0067ae0 msgr2=0x7f2dc0161780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:57.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.555+0000 7f2dc6e00640 1 -- 172.21.15.152:0/367663662 shutdown_connections 2024-04-20T11:28:57.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.555+0000 7f2dc6e00640 1 -- 172.21.15.152:0/367663662 wait complete. 2024-04-20T11:28:57.573 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-20T11:28:57.573 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-20T11:28:57.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.775+0000 7ff56f000640 1 Processor -- start 2024-04-20T11:28:57.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.776+0000 7ff56f000640 1 -- start start 2024-04-20T11:28:57.777 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.777+0000 7ff56f000640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ff56815e6c0 0x7ff56806dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:57.777 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.777+0000 7ff56f000640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff56806ec10 con 0x7ff5680d0d00 2024-04-20T11:28:57.777 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.777+0000 7ff56f000640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff56806e370 con 0x7ff56806f550 2024-04-20T11:28:57.777 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.777+0000 7ff56f000640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7ff5680b50f0 con 0x7ff56815e6c0 2024-04-20T11:28:57.777 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.778+0000 7ff56e600640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7ff5680d0d00 0x7ff5680ad660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:42782/0 (socket says 172.21.15.152:42782) 2024-04-20T11:28:57.777 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.778+0000 7ff56e600640 1 -- 172.21.15.152:0/2892086293 learned_addr learned my addr 172.21.15.152:0/2892086293 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:28:57.778 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.778+0000 7ff56dc00640 1 --2- 172.21.15.152:0/2892086293 >> v2:172.21.15.175:3300/0 conn(0x7ff56815e6c0 0x7ff56806dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:57.778 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.779+0000 7ff56dc00640 1 -- 172.21.15.152:0/2892086293 >> v1:172.21.15.175:6789/0 conn(0x7ff56806f550 legacy=0x7ff5680d05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:57.778 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.779+0000 7ff56dc00640 1 -- 172.21.15.152:0/2892086293 >> v1:172.21.15.152:6789/0 conn(0x7ff5680d0d00 legacy=0x7ff5680ad660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:57.778 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.779+0000 7ff56dc00640 1 -- 172.21.15.152:0/2892086293 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff56806e650 con 0x7ff56815e6c0 2024-04-20T11:28:57.779 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.780+0000 7ff56dc00640 1 --2- 172.21.15.152:0/2892086293 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff56815e6c0 0x7ff56806dc70 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff55c0097a0 tx=0x7ff55c02b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dea62b05640fb4f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:57.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.783+0000 7ff567e00640 1 -- 172.21.15.152:0/2892086293 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff55c034070 con 0x7ff56815e6c0 2024-04-20T11:28:57.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.783+0000 7ff567e00640 1 -- 172.21.15.152:0/2892086293 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff55c030610 con 0x7ff56815e6c0 2024-04-20T11:28:57.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.783+0000 7ff567e00640 1 -- 172.21.15.152:0/2892086293 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff55c0309e0 con 0x7ff56815e6c0 2024-04-20T11:28:57.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.784+0000 7ff56f000640 1 -- 172.21.15.152:0/2892086293 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff56815e6c0 msgr2=0x7ff56806dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:57.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.784+0000 7ff56f000640 1 --2- 172.21.15.152:0/2892086293 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff56815e6c0 0x7ff56806dc70 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff55c0097a0 tx=0x7ff55c02b960 comp rx=0 tx=0).stop 2024-04-20T11:28:57.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.784+0000 7ff56f000640 1 -- 172.21.15.152:0/2892086293 shutdown_connections 2024-04-20T11:28:57.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.784+0000 7ff56f000640 1 --2- 172.21.15.152:0/2892086293 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff56815e6c0 0x7ff56806dc70 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:57.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.784+0000 7ff56f000640 1 -- 172.21.15.152:0/2892086293 >> 172.21.15.152:0/2892086293 conn(0x7ff5680b21e0 msgr2=0x7ff5680b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:57.784 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.784+0000 7ff56f000640 1 -- 172.21.15.152:0/2892086293 shutdown_connections 2024-04-20T11:28:57.784 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.784+0000 7ff56f000640 1 -- 172.21.15.152:0/2892086293 wait complete. 2024-04-20T11:28:57.785 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.785+0000 7ff56f000640 1 Processor -- start 2024-04-20T11:28:57.785 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.786+0000 7ff56f000640 1 -- start start 2024-04-20T11:28:57.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.787+0000 7ff56f000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff56806f550 0x7ff5681f4440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:57.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.787+0000 7ff56f000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5680d0d00 0x7ff5681e76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:57.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.787+0000 7ff56f000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff56815e6c0 0x7ff5681e91f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:57.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.787+0000 7ff56f000640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7ff56815eaa0 con 0x7ff56806f550 2024-04-20T11:28:57.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.787+0000 7ff56f000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff5680ad140 con 0x7ff5680d0d00 2024-04-20T11:28:57.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.787+0000 7ff56f000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff5680add70 con 0x7ff56815e6c0 2024-04-20T11:28:57.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.787+0000 7ff56dc00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff56806f550 0x7ff5681f4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:57.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.787+0000 7ff56dc00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff56806f550 0x7ff5681f4440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:45112/0 (socket says 172.21.15.152:45112) 2024-04-20T11:28:57.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.787+0000 7ff56dc00640 1 -- 172.21.15.152:0/1758299029 learned_addr learned my addr 172.21.15.152:0/1758299029 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:57.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.787+0000 7ff56dc00640 1 -- 172.21.15.152:0/1758299029 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff56815e6c0 msgr2=0x7ff5681e91f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:57.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.787+0000 7ff56dc00640 1 --2- 172.21.15.152:0/1758299029 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff56815e6c0 0x7ff5681e91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:57.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.787+0000 7ff56dc00640 1 -- 172.21.15.152:0/1758299029 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5680d0d00 msgr2=0x7ff5681e76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:57.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.787+0000 7ff56dc00640 1 --2- 172.21.15.152:0/1758299029 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5680d0d00 0x7ff5681e76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:57.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.787+0000 7ff56dc00640 1 -- 172.21.15.152:0/1758299029 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5681f7720 con 0x7ff56806f550 2024-04-20T11:28:57.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.788+0000 7ff56dc00640 1 --2- 172.21.15.152:0/1758299029 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff56806f550 0x7ff5681f4440 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7ff55c009890 tx=0x7ff55c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:57.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.788+0000 7ff565600640 1 -- 172.21.15.152:0/1758299029 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff55c034040 con 0x7ff56806f550 2024-04-20T11:28:57.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.788+0000 7ff565600640 1 -- 172.21.15.152:0/1758299029 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff55c03ad10 con 0x7ff56806f550 2024-04-20T11:28:57.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.788+0000 7ff56f000640 1 -- 172.21.15.152:0/1758299029 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5681e8150 con 0x7ff56806f550 2024-04-20T11:28:57.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.788+0000 7ff565600640 1 -- 172.21.15.152:0/1758299029 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff55c044630 con 0x7ff56806f550 2024-04-20T11:28:57.789 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.788+0000 7ff56f000640 1 -- 172.21.15.152:0/1758299029 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5681e8710 con 0x7ff56806f550 2024-04-20T11:28:57.789 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.789+0000 7ff56f000640 1 -- 172.21.15.152:0/1758299029 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff568075e60 con 0x7ff56806f550 2024-04-20T11:28:57.789 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.790+0000 7ff565600640 1 -- 172.21.15.152:0/1758299029 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7ff55c04b060 con 0x7ff56806f550 2024-04-20T11:28:57.790 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.791+0000 7ff565600640 1 --2- 172.21.15.152:0/1758299029 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff54806caa0 0x7ff54806ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:57.791 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.791+0000 7ff565600640 1 -- 172.21.15.152:0/1758299029 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 4857+0+0 (secure 0 0 0) 0x7ff55c0adc20 con 0x7ff56806f550 2024-04-20T11:28:57.791 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.791+0000 7ff56d200640 1 --2- 172.21.15.152:0/1758299029 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff54806caa0 0x7ff54806ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:57.791 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.792+0000 7ff56d200640 1 --2- 172.21.15.152:0/1758299029 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff54806caa0 0x7ff54806ef60 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7ff554000f60 tx=0x7ff5540079d0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:57.795 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.795+0000 7ff565600640 1 -- 172.21.15.152:0/1758299029 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ff55c03f070 con 0x7ff56806f550 2024-04-20T11:28:57.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:57.995+0000 7ff56f000640 1 -- 172.21.15.152:0/1758299029 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7ff5681f4980 con 0x7ff56806f550 2024-04-20T11:28:58.792 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:58.792+0000 7ff565600640 1 -- 172.21.15.152:0/1758299029 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v43) ==== 138+0+0 (secure 0 0 0) 0x7ff55c04b350 con 0x7ff56806f550 2024-04-20T11:28:58.792 INFO:teuthology.orchestra.run.smithi152.stderr:pool 'cephfs_metadata' created 2024-04-20T11:28:58.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:58.796+0000 7ff56f000640 1 -- 172.21.15.152:0/1758299029 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff54806caa0 msgr2=0x7ff54806ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:58.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:58.796+0000 7ff56f000640 1 --2- 172.21.15.152:0/1758299029 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff54806caa0 0x7ff54806ef60 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7ff554000f60 tx=0x7ff5540079d0 comp rx=0 tx=0).stop 2024-04-20T11:28:58.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:58.796+0000 7ff56f000640 1 -- 172.21.15.152:0/1758299029 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff56806f550 msgr2=0x7ff5681f4440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:58.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:58.796+0000 7ff56f000640 1 --2- 172.21.15.152:0/1758299029 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff56806f550 0x7ff5681f4440 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7ff55c009890 tx=0x7ff55c0046c0 comp rx=0 tx=0).stop 2024-04-20T11:28:58.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:58.796+0000 7ff56f000640 1 -- 172.21.15.152:0/1758299029 shutdown_connections 2024-04-20T11:28:58.797 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:58.796+0000 7ff56f000640 1 --2- 172.21.15.152:0/1758299029 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff54806caa0 0x7ff54806ef60 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:58.797 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:58.796+0000 7ff56f000640 1 --2- 172.21.15.152:0/1758299029 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff56815e6c0 0x7ff5681e91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:58.797 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:58.796+0000 7ff56f000640 1 --2- 172.21.15.152:0/1758299029 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5680d0d00 0x7ff5681e76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:58.797 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:58.796+0000 7ff56f000640 1 --2- 172.21.15.152:0/1758299029 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff56806f550 0x7ff5681f4440 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:58.797 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:58.797+0000 7ff56f000640 1 -- 172.21.15.152:0/1758299029 >> 172.21.15.152:0/1758299029 conn(0x7ff5680b21e0 msgr2=0x7ff56815aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:58.797 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:58.797+0000 7ff56f000640 1 -- 172.21.15.152:0/1758299029 shutdown_connections 2024-04-20T11:28:58.797 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:58.798+0000 7ff56f000640 1 -- 172.21.15.152:0/1758299029 wait complete. 2024-04-20T11:28:58.823 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-20T11:28:59.018 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:59.018+0000 7fe459a00640 1 Processor -- start 2024-04-20T11:28:59.018 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:59.019+0000 7fe459a00640 1 -- start start 2024-04-20T11:28:59.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:59.020+0000 7fe459a00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fe4540d0b80 0x7fe454160d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:59.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:59.020+0000 7fe459a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe4540cd450 0x7fe4540723e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:59.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:59.020+0000 7fe459a00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe454161290 con 0x7fe4540ccff0 2024-04-20T11:28:59.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:59.020+0000 7fe459a00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fe454145830 con 0x7fe4540d0b80 2024-04-20T11:28:59.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:59.020+0000 7fe459a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe45406f070 con 0x7fe4540cd450 2024-04-20T11:28:59.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:59.021+0000 7fe452000640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fe4540ccff0 0x7fe454071cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:42796/0 (socket says 172.21.15.152:42796) 2024-04-20T11:28:59.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:59.021+0000 7fe452000640 1 -- 172.21.15.152:0/4028183074 learned_addr learned my addr 172.21.15.152:0/4028183074 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:28:59.022 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:59.021+0000 7fe452a00640 1 --2- 172.21.15.152:0/4028183074 >> v2:172.21.15.175:3300/0 conn(0x7fe4540d0b80 0x7fe454160d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:59.022 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:59.021+0000 7fe453400640 1 --2- 172.21.15.152:0/4028183074 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe4540cd450 0x7fe4540723e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:59.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:59.021+0000 7fe451600640 1 -- 172.21.15.152:0/4028183074 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1933318026 0 0) 0x7fe454161290 con 0x7fe4540ccff0 2024-04-20T11:28:59.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:59.021+0000 7fe451600640 1 -- 172.21.15.152:0/4028183074 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe438003520 con 0x7fe4540ccff0 2024-04-20T11:28:59.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:59.022+0000 7fe451600640 1 -- 172.21.15.152:0/4028183074 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3685826683 0 0) 0x7fe438003520 con 0x7fe4540ccff0 2024-04-20T11:28:59.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:59.022+0000 7fe451600640 1 -- 172.21.15.152:0/4028183074 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe4540cd450 msgr2=0x7fe4540723e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:59.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:59.022+0000 7fe451600640 1 --2- 172.21.15.152:0/4028183074 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe4540cd450 0x7fe4540723e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:59.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:59.022+0000 7fe451600640 1 -- 172.21.15.152:0/4028183074 >> v2:172.21.15.175:3300/0 conn(0x7fe4540d0b80 msgr2=0x7fe454160d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:59.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:59.022+0000 7fe451600640 1 --2- 172.21.15.152:0/4028183074 >> v2:172.21.15.175:3300/0 conn(0x7fe4540d0b80 0x7fe454160d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:59.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:59.022+0000 7fe451600640 1 -- 172.21.15.152:0/4028183074 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe45406f500 con 0x7fe4540ccff0 2024-04-20T11:28:59.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:59.022+0000 7fe451600640 1 -- 172.21.15.152:0/4028183074 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7fe43c001670 con 0x7fe4540ccff0 2024-04-20T11:28:59.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:59.022+0000 7fe451600640 1 -- 172.21.15.152:0/4028183074 >> v1:172.21.15.152:6789/0 conn(0x7fe4540ccff0 legacy=0x7fe454071cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:59.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:59.022+0000 7fe451600640 1 --2- 172.21.15.152:0/4028183074 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe4380039b0 0x7fe438003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:59.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:59.022+0000 7fe451600640 1 --2- 172.21.15.152:0/4028183074 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe4540cd450 0x7fe438004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:59.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:59.022+0000 7fe451600640 1 --2- 172.21.15.152:0/4028183074 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe438004d40 0x7fe4380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:59.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:59.022+0000 7fe451600640 1 -- 172.21.15.152:0/4028183074 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fe454145830 con 0x7fe4380039b0 2024-04-20T11:28:59.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:59.022+0000 7fe451600640 1 -- 172.21.15.152:0/4028183074 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe45406f070 con 0x7fe438004d40 2024-04-20T11:28:59.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:59.022+0000 7fe451600640 1 -- 172.21.15.152:0/4028183074 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe43800b9c0 con 0x7fe4540cd450 2024-04-20T11:28:59.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:59.022+0000 7fe452a00640 1 --2- 172.21.15.152:0/4028183074 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe4380039b0 0x7fe438003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:59.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:59.022+0000 7fe452000640 1 --2- 172.21.15.152:0/4028183074 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe4540cd450 0x7fe438004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:59.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:59.023+0000 7fe452a00640 1 -- 172.21.15.152:0/4028183074 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe4540cd450 msgr2=0x7fe438004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:59.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:59.023+0000 7fe452a00640 1 --2- 172.21.15.152:0/4028183074 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe4540cd450 0x7fe438004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:59.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:59.023+0000 7fe452a00640 1 -- 172.21.15.152:0/4028183074 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe438004d40 msgr2=0x7fe4380051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:59.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:59.023+0000 7fe452a00640 1 --2- 172.21.15.152:0/4028183074 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe438004d40 0x7fe4380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:59.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:59.023+0000 7fe452a00640 1 -- 172.21.15.152:0/4028183074 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe438003520 con 0x7fe4380039b0 2024-04-20T11:28:59.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:59.023+0000 7fe452000640 1 --2- 172.21.15.152:0/4028183074 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe4540cd450 0x7fe438004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:28:59.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:59.024+0000 7fe452a00640 1 --2- 172.21.15.152:0/4028183074 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe4380039b0 0x7fe438003dd0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fe444009fc0 tx=0x7fe438005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2ba8005f96fce70 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:59.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:59.024+0000 7fe451600640 1 -- 172.21.15.152:0/4028183074 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe444039070 con 0x7fe4380039b0 2024-04-20T11:28:59.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:59.024+0000 7fe451600640 1 -- 172.21.15.152:0/4028183074 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe44402cc30 con 0x7fe4380039b0 2024-04-20T11:28:59.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:59.024+0000 7fe451600640 1 -- 172.21.15.152:0/4028183074 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe444009c10 con 0x7fe4380039b0 2024-04-20T11:28:59.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:59.025+0000 7fe459a00640 1 -- 172.21.15.152:0/4028183074 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe4380039b0 msgr2=0x7fe438003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:59.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:59.025+0000 7fe459a00640 1 --2- 172.21.15.152:0/4028183074 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe4380039b0 0x7fe438003dd0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fe444009fc0 tx=0x7fe438005830 comp rx=0 tx=0).stop 2024-04-20T11:28:59.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:59.025+0000 7fe453400640 1 -- 172.21.15.152:0/4028183074 reap_dead start 2024-04-20T11:28:59.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:59.025+0000 7fe459a00640 1 -- 172.21.15.152:0/4028183074 shutdown_connections 2024-04-20T11:28:59.026 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:59.025+0000 7fe459a00640 1 -- 172.21.15.152:0/4028183074 >> 172.21.15.152:0/4028183074 conn(0x7fe45406b980 msgr2=0x7fe45406bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:59.026 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:59.025+0000 7fe459a00640 1 -- 172.21.15.152:0/4028183074 shutdown_connections 2024-04-20T11:28:59.026 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:59.025+0000 7fe459a00640 1 -- 172.21.15.152:0/4028183074 wait complete. 2024-04-20T11:28:59.026 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:59.027+0000 7fe459a00640 1 Processor -- start 2024-04-20T11:28:59.026 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:59.027+0000 7fe459a00640 1 -- start start 2024-04-20T11:28:59.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:59.028+0000 7fe459a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe4380039b0 0x7fe4540a9d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:59.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:59.028+0000 7fe459a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe4540c9c00 0x7fe454143650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:59.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:59.028+0000 7fe459a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe454143b90 0x7fe4540c8620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:59.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:59.028+0000 7fe459a00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fe45406ea20 con 0x7fe4380039b0 2024-04-20T11:28:59.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:59.028+0000 7fe459a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe45406e740 con 0x7fe454143b90 2024-04-20T11:28:59.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:59.028+0000 7fe459a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe45406ed00 con 0x7fe4540c9c00 2024-04-20T11:28:59.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:59.028+0000 7fe452a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe4380039b0 0x7fe4540a9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:59.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:59.028+0000 7fe452a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe4380039b0 0x7fe4540a9d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:45128/0 (socket says 172.21.15.152:45128) 2024-04-20T11:28:59.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:59.028+0000 7fe452a00640 1 -- 172.21.15.152:0/1004867915 learned_addr learned my addr 172.21.15.152:0/1004867915 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:28:59.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:59.028+0000 7fe453400640 1 --2- 172.21.15.152:0/1004867915 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe454143b90 0x7fe4540c8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:59.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:59.028+0000 7fe452000640 1 --2- 172.21.15.152:0/1004867915 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe4540c9c00 0x7fe454143650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:59.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:59.028+0000 7fe452a00640 1 -- 172.21.15.152:0/1004867915 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe4540c9c00 msgr2=0x7fe454143650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:59.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:59.028+0000 7fe452a00640 1 --2- 172.21.15.152:0/1004867915 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe4540c9c00 0x7fe454143650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:59.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:59.028+0000 7fe452a00640 1 -- 172.21.15.152:0/1004867915 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe454143b90 msgr2=0x7fe4540c8620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:59.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:59.028+0000 7fe452a00640 1 --2- 172.21.15.152:0/1004867915 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe454143b90 0x7fe4540c8620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:59.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:59.028+0000 7fe452a00640 1 -- 172.21.15.152:0/1004867915 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4541322e0 con 0x7fe4380039b0 2024-04-20T11:28:59.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:59.029+0000 7fe452000640 1 --2- 172.21.15.152:0/1004867915 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe4540c9c00 0x7fe454143650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:28:59.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:59.029+0000 7fe452a00640 1 --2- 172.21.15.152:0/1004867915 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe4380039b0 0x7fe4540a9d90 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fe444037040 tx=0x7fe444009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:59.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:59.029+0000 7fe44aa00640 1 -- 172.21.15.152:0/1004867915 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe444039050 con 0x7fe4380039b0 2024-04-20T11:28:59.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:59.029+0000 7fe44aa00640 1 -- 172.21.15.152:0/1004867915 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe44403a040 con 0x7fe4380039b0 2024-04-20T11:28:59.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:59.029+0000 7fe459a00640 1 -- 172.21.15.152:0/1004867915 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4541325c0 con 0x7fe4380039b0 2024-04-20T11:28:59.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:59.029+0000 7fe44aa00640 1 -- 172.21.15.152:0/1004867915 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe444009c10 con 0x7fe4380039b0 2024-04-20T11:28:59.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:59.029+0000 7fe459a00640 1 -- 172.21.15.152:0/1004867915 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe454132b80 con 0x7fe4380039b0 2024-04-20T11:28:59.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:59.030+0000 7fe459a00640 1 -- 172.21.15.152:0/1004867915 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4540689f0 con 0x7fe4380039b0 2024-04-20T11:28:59.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:59.032+0000 7fe44aa00640 1 -- 172.21.15.152:0/1004867915 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7fe444047030 con 0x7fe4380039b0 2024-04-20T11:28:59.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:59.033+0000 7fe44aa00640 1 --2- 172.21.15.152:0/1004867915 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe43006cad0 0x7fe43006ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:59.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:59.033+0000 7fe44aa00640 1 -- 172.21.15.152:0/1004867915 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5227+0+0 (secure 0 0 0) 0x7fe4440ae830 con 0x7fe4380039b0 2024-04-20T11:28:59.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:59.033+0000 7fe452000640 1 --2- 172.21.15.152:0/1004867915 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe43006cad0 0x7fe43006ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:59.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:59.034+0000 7fe452000640 1 --2- 172.21.15.152:0/1004867915 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe43006cad0 0x7fe43006ef90 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fe43c001eb0 tx=0x7fe43c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:59.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:59.037+0000 7fe44aa00640 1 -- 172.21.15.152:0/1004867915 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe44404e060 con 0x7fe4380039b0 2024-04-20T11:28:59.243 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:59.244+0000 7fe459a00640 1 -- 172.21.15.152:0/1004867915 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fe4540d1050 con 0x7fe4380039b0 2024-04-20T11:28:59.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:59.807+0000 7fe44aa00640 1 -- 172.21.15.152:0/1004867915 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v44) ==== 170+0+0 (secure 0 0 0) 0x7fe44407cc30 con 0x7fe4380039b0 2024-04-20T11:28:59.807 INFO:teuthology.orchestra.run.smithi152.stderr:pool 'cephfs_data' created 2024-04-20T11:28:59.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:59.811+0000 7fe459a00640 1 -- 172.21.15.152:0/1004867915 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe43006cad0 msgr2=0x7fe43006ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:59.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:59.811+0000 7fe459a00640 1 --2- 172.21.15.152:0/1004867915 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe43006cad0 0x7fe43006ef90 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fe43c001eb0 tx=0x7fe43c009040 comp rx=0 tx=0).stop 2024-04-20T11:28:59.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:59.811+0000 7fe459a00640 1 -- 172.21.15.152:0/1004867915 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe4380039b0 msgr2=0x7fe4540a9d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:59.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:59.811+0000 7fe459a00640 1 --2- 172.21.15.152:0/1004867915 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe4380039b0 0x7fe4540a9d90 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fe444037040 tx=0x7fe444009750 comp rx=0 tx=0).stop 2024-04-20T11:28:59.812 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:59.811+0000 7fe459a00640 1 -- 172.21.15.152:0/1004867915 shutdown_connections 2024-04-20T11:28:59.812 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:59.811+0000 7fe459a00640 1 --2- 172.21.15.152:0/1004867915 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe43006cad0 0x7fe43006ef90 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:59.812 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:59.811+0000 7fe459a00640 1 --2- 172.21.15.152:0/1004867915 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe454143b90 0x7fe4540c8620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:59.812 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:59.811+0000 7fe459a00640 1 --2- 172.21.15.152:0/1004867915 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe4540c9c00 0x7fe454143650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:59.812 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:59.811+0000 7fe459a00640 1 --2- 172.21.15.152:0/1004867915 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe4380039b0 0x7fe4540a9d90 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:59.812 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:59.811+0000 7fe459a00640 1 -- 172.21.15.152:0/1004867915 >> 172.21.15.152:0/1004867915 conn(0x7fe45406b980 msgr2=0x7fe454144cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:59.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:59.813+0000 7fe459a00640 1 -- 172.21.15.152:0/1004867915 shutdown_connections 2024-04-20T11:28:59.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:28:59.813+0000 7fe459a00640 1 -- 172.21.15.152:0/1004867915 wait complete. 2024-04-20T11:28:59.847 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-20T11:29:00.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:00.023+0000 7f03c7c00640 1 Processor -- start 2024-04-20T11:29:00.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:00.024+0000 7f03c7c00640 1 -- start start 2024-04-20T11:29:00.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:00.025+0000 7f03c7c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03c00b3740 0x7f03c00cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:00.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:00.025+0000 7f03c7c00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03c00d00e0 con 0x7f03c015b7c0 2024-04-20T11:29:00.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:00.025+0000 7f03c7c00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03c00cc260 con 0x7f03c00cdc60 2024-04-20T11:29:00.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:00.025+0000 7f03c7c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f03c00cc030 con 0x7f03c00b3740 2024-04-20T11:29:00.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:00.025+0000 7f03c5200640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f03c015b7c0 0x7f03c00ccd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:42808/0 (socket says 172.21.15.152:42808) 2024-04-20T11:29:00.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:00.025+0000 7f03c5200640 1 -- 172.21.15.152:0/3371332798 learned_addr learned my addr 172.21.15.152:0/3371332798 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:29:00.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:00.026+0000 7f03b7e00640 1 --2- 172.21.15.152:0/3371332798 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03c00b3740 0x7f03c00cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:00.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:00.026+0000 7f03bfe00640 1 -- 172.21.15.152:0/3371332798 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4095614699 0 0) 0x7f03c00d00e0 con 0x7f03c015b7c0 2024-04-20T11:29:00.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:00.026+0000 7f03bfe00640 1 -- 172.21.15.152:0/3371332798 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f03a0003520 con 0x7f03c015b7c0 2024-04-20T11:29:00.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:00.026+0000 7f03bfe00640 1 -- 172.21.15.152:0/3371332798 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2687210326 0 0) 0x7f03a0003520 con 0x7f03c015b7c0 2024-04-20T11:29:00.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:00.026+0000 7f03bfe00640 1 -- 172.21.15.152:0/3371332798 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03c00b3740 msgr2=0x7f03c00cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:00.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:00.026+0000 7f03bfe00640 1 --2- 172.21.15.152:0/3371332798 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03c00b3740 0x7f03c00cfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:00.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:00.026+0000 7f03bfe00640 1 -- 172.21.15.152:0/3371332798 >> v1:172.21.15.175:6789/0 conn(0x7f03c00cdc60 legacy=0x7f03c00b4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:00.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:00.026+0000 7f03bfe00640 1 -- 172.21.15.152:0/3371332798 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03c00cc540 con 0x7f03c015b7c0 2024-04-20T11:29:00.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:00.026+0000 7f03bfe00640 1 -- 172.21.15.152:0/3371332798 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f03a8001670 con 0x7f03c015b7c0 2024-04-20T11:29:00.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:00.026+0000 7f03bfe00640 1 -- 172.21.15.152:0/3371332798 >> v1:172.21.15.152:6789/0 conn(0x7f03c015b7c0 legacy=0x7f03c00ccd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:00.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:00.026+0000 7f03bfe00640 1 --2- 172.21.15.152:0/3371332798 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f03a00039e0 0x7f03a0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:00.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:00.026+0000 7f03bfe00640 1 --2- 172.21.15.152:0/3371332798 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03c00b3740 0x7f03a0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:00.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:00.026+0000 7f03bfe00640 1 --2- 172.21.15.152:0/3371332798 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f03a0004d40 0x7f03a00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:00.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:00.027+0000 7f03bfe00640 1 -- 172.21.15.152:0/3371332798 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f03a000ba70 con 0x7f03a0004d40 2024-04-20T11:29:00.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:00.027+0000 7f03bfe00640 1 -- 172.21.15.152:0/3371332798 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f03a000bbf0 con 0x7f03a00039e0 2024-04-20T11:29:00.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:00.027+0000 7f03bfe00640 1 -- 172.21.15.152:0/3371332798 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f03a000bdf0 con 0x7f03c00b3740 2024-04-20T11:29:00.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:00.027+0000 7f03c5c00640 1 --2- 172.21.15.152:0/3371332798 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f03a0004d40 0x7f03a00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:00.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:00.027+0000 7f03c5c00640 1 -- 172.21.15.152:0/3371332798 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03c00b3740 msgr2=0x7f03a0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:00.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:00.027+0000 7f03c5c00640 1 --2- 172.21.15.152:0/3371332798 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03c00b3740 0x7f03a0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:00.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:00.027+0000 7f03c5c00640 1 -- 172.21.15.152:0/3371332798 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f03a00039e0 msgr2=0x7f03a0003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:00.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:00.027+0000 7f03c5200640 1 --2- 172.21.15.152:0/3371332798 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f03a00039e0 0x7f03a0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:00.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:00.027+0000 7f03c5c00640 1 --2- 172.21.15.152:0/3371332798 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f03a00039e0 0x7f03a0003e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:00.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:00.027+0000 7f03c5c00640 1 -- 172.21.15.152:0/3371332798 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03a0003520 con 0x7f03a0004d40 2024-04-20T11:29:00.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:00.028+0000 7f03c5c00640 1 --2- 172.21.15.152:0/3371332798 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f03a0004d40 0x7f03a00051c0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f03b8001e20 tx=0x7f03a0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=286f38a79a24c3b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:00.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:00.029+0000 7f03bfe00640 1 -- 172.21.15.152:0/3371332798 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b8001a70 con 0x7f03a0004d40 2024-04-20T11:29:00.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:00.029+0000 7f03bfe00640 1 -- 172.21.15.152:0/3371332798 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03b8003b70 con 0x7f03a0004d40 2024-04-20T11:29:00.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:00.029+0000 7f03bfe00640 1 -- 172.21.15.152:0/3371332798 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b80050d0 con 0x7f03a0004d40 2024-04-20T11:29:00.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:00.029+0000 7f03c7c00640 1 -- 172.21.15.152:0/3371332798 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f03a0004d40 msgr2=0x7f03a00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:00.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:00.029+0000 7f03c7c00640 1 --2- 172.21.15.152:0/3371332798 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f03a0004d40 0x7f03a00051c0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f03b8001e20 tx=0x7f03a0005830 comp rx=0 tx=0).stop 2024-04-20T11:29:00.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:00.029+0000 7f03c5c00640 1 -- 172.21.15.152:0/3371332798 reap_dead start 2024-04-20T11:29:00.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:00.029+0000 7f03c7c00640 1 -- 172.21.15.152:0/3371332798 shutdown_connections 2024-04-20T11:29:00.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:00.029+0000 7f03c7c00640 1 -- 172.21.15.152:0/3371332798 >> 172.21.15.152:0/3371332798 conn(0x7f03c009aa40 msgr2=0x7f03c00b3030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:00.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:00.029+0000 7f03c7c00640 1 -- 172.21.15.152:0/3371332798 shutdown_connections 2024-04-20T11:29:00.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:00.029+0000 7f03c7c00640 1 -- 172.21.15.152:0/3371332798 wait complete. 2024-04-20T11:29:00.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:00.031+0000 7f03c7c00640 1 Processor -- start 2024-04-20T11:29:00.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:00.031+0000 7f03c7c00640 1 -- start start 2024-04-20T11:29:00.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:00.033+0000 7f03c7c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03a0004d40 0x7f03c00c7130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:00.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:00.033+0000 7f03c7c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f03c00cb0c0 0x7f03c00cb540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:00.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:00.033+0000 7f03c7c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f03c015a690 0x7f03c015ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:00.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:00.033+0000 7f03c7c00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f03c00cd450 con 0x7f03c00cb0c0 2024-04-20T11:29:00.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:00.033+0000 7f03c7c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f03c00d03c0 con 0x7f03c015a690 2024-04-20T11:29:00.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:00.033+0000 7f03c7c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f03c015bba0 con 0x7f03a0004d40 2024-04-20T11:29:00.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:00.033+0000 7f03b7e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f03c00cb0c0 0x7f03c00cb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:00.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:00.033+0000 7f03b7e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f03c00cb0c0 0x7f03c00cb540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:45154/0 (socket says 172.21.15.152:45154) 2024-04-20T11:29:00.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:00.033+0000 7f03b7e00640 1 -- 172.21.15.152:0/1605919050 learned_addr learned my addr 172.21.15.152:0/1605919050 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:29:00.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:00.033+0000 7f03c5c00640 1 --2- 172.21.15.152:0/1605919050 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f03c015a690 0x7f03c015ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:00.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:00.033+0000 7f03c5200640 1 --2- 172.21.15.152:0/1605919050 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03a0004d40 0x7f03c00c7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:00.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:00.033+0000 7f03b7e00640 1 -- 172.21.15.152:0/1605919050 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03a0004d40 msgr2=0x7f03c00c7130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:00.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:00.033+0000 7f03b7e00640 1 --2- 172.21.15.152:0/1605919050 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03a0004d40 0x7f03c00c7130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:00.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:00.033+0000 7f03b7e00640 1 -- 172.21.15.152:0/1605919050 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f03c015a690 msgr2=0x7f03c015ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:00.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:00.033+0000 7f03b7e00640 1 --2- 172.21.15.152:0/1605919050 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f03c015a690 0x7f03c015ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:00.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:00.033+0000 7f03b7e00640 1 -- 172.21.15.152:0/1605919050 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03c00c9f00 con 0x7f03c00cb0c0 2024-04-20T11:29:00.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:00.033+0000 7f03c5c00640 1 --2- 172.21.15.152:0/1605919050 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f03c015a690 0x7f03c015ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:29:00.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:00.033+0000 7f03c5200640 1 --2- 172.21.15.152:0/1605919050 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03a0004d40 0x7f03c00c7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:29:00.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:00.034+0000 7f03b7e00640 1 --2- 172.21.15.152:0/1605919050 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f03c00cb0c0 0x7f03c00cb540 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f03b000e7f0 tx=0x7f03b000ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:00.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:00.034+0000 7f03bd600640 1 -- 172.21.15.152:0/1605919050 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b0017070 con 0x7f03c00cb0c0 2024-04-20T11:29:00.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:00.034+0000 7f03bd600640 1 -- 172.21.15.152:0/1605919050 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03b000ee40 con 0x7f03c00cb0c0 2024-04-20T11:29:00.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:00.034+0000 7f03bd600640 1 -- 172.21.15.152:0/1605919050 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b000cb60 con 0x7f03c00cb0c0 2024-04-20T11:29:00.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:00.034+0000 7f03c7c00640 1 -- 172.21.15.152:0/1605919050 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03c00c9380 con 0x7f03c00cb0c0 2024-04-20T11:29:00.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:00.034+0000 7f03c7c00640 1 -- 172.21.15.152:0/1605919050 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03c00c9940 con 0x7f03c00cb0c0 2024-04-20T11:29:00.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:00.035+0000 7f03c7c00640 1 -- 172.21.15.152:0/1605919050 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03c00689f0 con 0x7f03c00cb0c0 2024-04-20T11:29:00.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:00.036+0000 7f03bd600640 1 -- 172.21.15.152:0/1605919050 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f03b001a070 con 0x7f03c00cb0c0 2024-04-20T11:29:00.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:00.037+0000 7f03bd600640 1 --2- 172.21.15.152:0/1605919050 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f039c06ca10 0x7f039c06eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:00.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:00.037+0000 7f03bd600640 1 -- 172.21.15.152:0/1605919050 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5609+0+0 (secure 0 0 0) 0x7f03b008bd20 con 0x7f03c00cb0c0 2024-04-20T11:29:00.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:00.038+0000 7f03c5200640 1 --2- 172.21.15.152:0/1605919050 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f039c06ca10 0x7f039c06eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:00.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:00.038+0000 7f03c5200640 1 --2- 172.21.15.152:0/1605919050 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f039c06ca10 0x7f039c06eed0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f03a8001200 tx=0x7f03a8009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:00.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:00.042+0000 7f03bd600640 1 -- 172.21.15.152:0/1605919050 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f03b0059e90 con 0x7f03c00cb0c0 2024-04-20T11:29:00.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:00.275+0000 7f03c7c00640 1 -- 172.21.15.152:0/1605919050 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f03c006c870 con 0x7f03c00cb0c0 2024-04-20T11:29:00.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:00.812+0000 7f03bd600640 1 -- 172.21.15.152:0/1605919050 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v20) ==== 172+0+0 (secure 0 0 0) 0x7f03b0059790 con 0x7f03c00cb0c0 2024-04-20T11:29:00.812 INFO:teuthology.orchestra.run.smithi152.stderr:new fs with metadata pool 6 and data pool 7 2024-04-20T11:29:00.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:00.815+0000 7f03c7c00640 1 -- 172.21.15.152:0/1605919050 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f039c06ca10 msgr2=0x7f039c06eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:00.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:00.815+0000 7f03c7c00640 1 --2- 172.21.15.152:0/1605919050 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f039c06ca10 0x7f039c06eed0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f03a8001200 tx=0x7f03a8009040 comp rx=0 tx=0).stop 2024-04-20T11:29:00.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:00.815+0000 7f03c7c00640 1 -- 172.21.15.152:0/1605919050 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f03c00cb0c0 msgr2=0x7f03c00cb540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:00.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:00.815+0000 7f03c7c00640 1 --2- 172.21.15.152:0/1605919050 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f03c00cb0c0 0x7f03c00cb540 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f03b000e7f0 tx=0x7f03b000ebf0 comp rx=0 tx=0).stop 2024-04-20T11:29:00.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:00.816+0000 7f03c7c00640 1 -- 172.21.15.152:0/1605919050 shutdown_connections 2024-04-20T11:29:00.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:00.816+0000 7f03c7c00640 1 --2- 172.21.15.152:0/1605919050 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f039c06ca10 0x7f039c06eed0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:00.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:00.816+0000 7f03c7c00640 1 --2- 172.21.15.152:0/1605919050 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f03c015a690 0x7f03c015ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:00.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:00.816+0000 7f03c7c00640 1 --2- 172.21.15.152:0/1605919050 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f03c00cb0c0 0x7f03c00cb540 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:00.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:00.816+0000 7f03c7c00640 1 --2- 172.21.15.152:0/1605919050 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03a0004d40 0x7f03c00c7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:00.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:00.816+0000 7f03c7c00640 1 -- 172.21.15.152:0/1605919050 >> 172.21.15.152:0/1605919050 conn(0x7f03c009aa40 msgr2=0x7f03c006fbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:00.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:00.816+0000 7f03c7c00640 1 -- 172.21.15.152:0/1605919050 shutdown_connections 2024-04-20T11:29:00.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:00.816+0000 7f03c7c00640 1 -- 172.21.15.152:0/1605919050 wait complete. 2024-04-20T11:29:00.855 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:29:01.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.025+0000 7f1ba6200640 1 Processor -- start 2024-04-20T11:29:01.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.026+0000 7f1ba6200640 1 -- start start 2024-04-20T11:29:01.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.027+0000 7f1ba6200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f1ba0145f60 0x7f1ba006ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:01.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.027+0000 7f1ba6200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1ba00cd0d0 0x7f1ba00cd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:01.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.027+0000 7f1ba6200640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ba0146340 con 0x7f1ba0071fe0 2024-04-20T11:29:01.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.027+0000 7f1ba6200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f1ba006e9f0 con 0x7f1ba0145f60 2024-04-20T11:29:01.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.027+0000 7f1ba6200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1ba006f3c0 con 0x7f1ba00cd0d0 2024-04-20T11:29:01.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.027+0000 7f1b9fe00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f1ba0071fe0 0x7f1ba0072470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:42820/0 (socket says 172.21.15.152:42820) 2024-04-20T11:29:01.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.027+0000 7f1b9fe00640 1 -- 172.21.15.152:0/1988555946 learned_addr learned my addr 172.21.15.152:0/1988555946 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:29:01.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.028+0000 7f1b9ea00640 1 --2- 172.21.15.152:0/1988555946 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1ba00cd0d0 0x7f1ba00cd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:01.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.028+0000 7f1b9f400640 1 --2- 172.21.15.152:0/1988555946 >> v2:172.21.15.175:3300/0 conn(0x7f1ba0145f60 0x7f1ba006ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:01.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.028+0000 7f1b9e000640 1 -- 172.21.15.152:0/1988555946 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4272998196 0 0) 0x7f1ba0146340 con 0x7f1ba0071fe0 2024-04-20T11:29:01.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.028+0000 7f1b9e000640 1 -- 172.21.15.152:0/1988555946 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1b80003520 con 0x7f1ba0071fe0 2024-04-20T11:29:01.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.028+0000 7f1b9e000640 1 -- 172.21.15.152:0/1988555946 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2977299443 0 0) 0x7f1b80003520 con 0x7f1ba0071fe0 2024-04-20T11:29:01.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.028+0000 7f1b9e000640 1 -- 172.21.15.152:0/1988555946 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1ba00cd0d0 msgr2=0x7f1ba00cd550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:01.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.028+0000 7f1b9e000640 1 --2- 172.21.15.152:0/1988555946 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1ba00cd0d0 0x7f1ba00cd550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:01.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.028+0000 7f1b9e000640 1 -- 172.21.15.152:0/1988555946 >> v2:172.21.15.175:3300/0 conn(0x7f1ba0145f60 msgr2=0x7f1ba006ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:01.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.028+0000 7f1b9e000640 1 --2- 172.21.15.152:0/1988555946 >> v2:172.21.15.175:3300/0 conn(0x7f1ba0145f60 0x7f1ba006ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:01.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.028+0000 7f1b9e000640 1 -- 172.21.15.152:0/1988555946 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ba0072d50 con 0x7f1ba0071fe0 2024-04-20T11:29:01.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.028+0000 7f1b9e000640 1 -- 172.21.15.152:0/1988555946 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f1b98001670 con 0x7f1ba0071fe0 2024-04-20T11:29:01.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.028+0000 7f1b9f400640 1 --2- 172.21.15.152:0/1988555946 >> v2:172.21.15.175:3300/0 conn(0x7f1ba0145f60 0x7f1ba006ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:29:01.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.028+0000 7f1b9e000640 1 -- 172.21.15.152:0/1988555946 >> v1:172.21.15.152:6789/0 conn(0x7f1ba0071fe0 legacy=0x7f1ba0072470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:01.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.028+0000 7f1b9e000640 1 --2- 172.21.15.152:0/1988555946 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1ba00cd0d0 0x7f1b80003a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:01.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.028+0000 7f1b9e000640 1 --2- 172.21.15.152:0/1988555946 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1b80003fc0 0x7f1b80004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:01.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.028+0000 7f1b9e000640 1 --2- 172.21.15.152:0/1988555946 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b80004d40 0x7f1b800051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:01.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.028+0000 7f1b9e000640 1 -- 172.21.15.152:0/1988555946 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba006e9f0 con 0x7f1b80003fc0 2024-04-20T11:29:01.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.028+0000 7f1b9e000640 1 -- 172.21.15.152:0/1988555946 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba006f3c0 con 0x7f1b80004d40 2024-04-20T11:29:01.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.028+0000 7f1b9e000640 1 -- 172.21.15.152:0/1988555946 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1b8000baa0 con 0x7f1ba00cd0d0 2024-04-20T11:29:01.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.029+0000 7f1b9ea00640 1 --2- 172.21.15.152:0/1988555946 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1b80003fc0 0x7f1b80004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:01.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.029+0000 7f1b9f400640 1 --2- 172.21.15.152:0/1988555946 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1ba00cd0d0 0x7f1b80003a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:01.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.029+0000 7f1b9fe00640 1 --2- 172.21.15.152:0/1988555946 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b80004d40 0x7f1b800051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:01.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.029+0000 7f1b9ea00640 1 -- 172.21.15.152:0/1988555946 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1ba00cd0d0 msgr2=0x7f1b80003a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:01.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.029+0000 7f1b9ea00640 1 --2- 172.21.15.152:0/1988555946 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1ba00cd0d0 0x7f1b80003a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:01.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.029+0000 7f1b9ea00640 1 -- 172.21.15.152:0/1988555946 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b80004d40 msgr2=0x7f1b800051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:01.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.029+0000 7f1b9ea00640 1 --2- 172.21.15.152:0/1988555946 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b80004d40 0x7f1b800051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:01.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.029+0000 7f1b9ea00640 1 -- 172.21.15.152:0/1988555946 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b80003520 con 0x7f1b80003fc0 2024-04-20T11:29:01.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.030+0000 7f1b9ea00640 1 --2- 172.21.15.152:0/1988555946 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1b80003fc0 0x7f1b80004800 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f1b88009fc0 tx=0x7f1b80005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9da807968ea0b0a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:01.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.030+0000 7f1b9e000640 1 -- 172.21.15.152:0/1988555946 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b88039070 con 0x7f1b80003fc0 2024-04-20T11:29:01.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.030+0000 7f1b9e000640 1 -- 172.21.15.152:0/1988555946 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b8802cc20 con 0x7f1b80003fc0 2024-04-20T11:29:01.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.030+0000 7f1b9e000640 1 -- 172.21.15.152:0/1988555946 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b88009c10 con 0x7f1b80003fc0 2024-04-20T11:29:01.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.031+0000 7f1ba6200640 1 -- 172.21.15.152:0/1988555946 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1b80003fc0 msgr2=0x7f1b80004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:01.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.031+0000 7f1ba6200640 1 --2- 172.21.15.152:0/1988555946 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1b80003fc0 0x7f1b80004800 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f1b88009fc0 tx=0x7f1b80005830 comp rx=0 tx=0).stop 2024-04-20T11:29:01.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.031+0000 7f1b9fe00640 1 -- 172.21.15.152:0/1988555946 reap_dead start 2024-04-20T11:29:01.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.031+0000 7f1ba6200640 1 -- 172.21.15.152:0/1988555946 shutdown_connections 2024-04-20T11:29:01.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.031+0000 7f1ba6200640 1 -- 172.21.15.152:0/1988555946 >> 172.21.15.152:0/1988555946 conn(0x7f1ba0067cf0 msgr2=0x7f1ba006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:01.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.031+0000 7f1ba6200640 1 -- 172.21.15.152:0/1988555946 shutdown_connections 2024-04-20T11:29:01.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.031+0000 7f1ba6200640 1 -- 172.21.15.152:0/1988555946 wait complete. 2024-04-20T11:29:01.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.033+0000 7f1ba6200640 1 Processor -- start 2024-04-20T11:29:01.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.033+0000 7f1ba6200640 1 -- start start 2024-04-20T11:29:01.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.034+0000 7f1ba6200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b80003fc0 0x7f1ba00ad8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:01.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.034+0000 7f1ba6200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1ba00af120 0x7f1ba00ae580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:01.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.034+0000 7f1ba6200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1ba00af5a0 0x7f1ba0143760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:01.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.034+0000 7f1ba6200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba006e680 con 0x7f1ba00af120 2024-04-20T11:29:01.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.034+0000 7f1ba6200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba006e3a0 con 0x7f1b80003fc0 2024-04-20T11:29:01.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.034+0000 7f1ba6200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1ba00cd550 con 0x7f1ba00af5a0 2024-04-20T11:29:01.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.034+0000 7f1b9ea00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1ba00af120 0x7f1ba00ae580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:01.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.034+0000 7f1b9ea00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1ba00af120 0x7f1ba00ae580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:45166/0 (socket says 172.21.15.152:45166) 2024-04-20T11:29:01.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.034+0000 7f1b9ea00640 1 -- 172.21.15.152:0/1555697890 learned_addr learned my addr 172.21.15.152:0/1555697890 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:29:01.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.035+0000 7f1b9f400640 1 --2- 172.21.15.152:0/1555697890 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b80003fc0 0x7f1ba00ad8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:01.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.035+0000 7f1b9fe00640 1 --2- 172.21.15.152:0/1555697890 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1ba00af5a0 0x7f1ba0143760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:01.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.035+0000 7f1b9ea00640 1 -- 172.21.15.152:0/1555697890 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1ba00af5a0 msgr2=0x7f1ba0143760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:01.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.035+0000 7f1b9ea00640 1 --2- 172.21.15.152:0/1555697890 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1ba00af5a0 0x7f1ba0143760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:01.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.035+0000 7f1b9ea00640 1 -- 172.21.15.152:0/1555697890 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b80003fc0 msgr2=0x7f1ba00ad8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:01.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.035+0000 7f1b9ea00640 1 --2- 172.21.15.152:0/1555697890 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b80003fc0 0x7f1ba00ad8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:01.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.035+0000 7f1b9ea00640 1 -- 172.21.15.152:0/1555697890 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ba00ac540 con 0x7f1ba00af120 2024-04-20T11:29:01.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.035+0000 7f1b9ea00640 1 --2- 172.21.15.152:0/1555697890 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1ba00af120 0x7f1ba00ae580 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f1b8802c690 tx=0x7f1b88009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:01.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.036+0000 7f1b97400640 1 -- 172.21.15.152:0/1555697890 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b88039050 con 0x7f1ba00af120 2024-04-20T11:29:01.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.036+0000 7f1b97400640 1 -- 172.21.15.152:0/1555697890 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b8803a040 con 0x7f1ba00af120 2024-04-20T11:29:01.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.036+0000 7f1b97400640 1 -- 172.21.15.152:0/1555697890 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b88003bb0 con 0x7f1ba00af120 2024-04-20T11:29:01.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.036+0000 7f1ba6200640 1 -- 172.21.15.152:0/1555697890 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ba00ac820 con 0x7f1ba00af120 2024-04-20T11:29:01.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.036+0000 7f1ba6200640 1 -- 172.21.15.152:0/1555697890 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ba00acde0 con 0x7f1ba00af120 2024-04-20T11:29:01.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.037+0000 7f1ba6200640 1 -- 172.21.15.152:0/1555697890 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ba00689f0 con 0x7f1ba00af120 2024-04-20T11:29:01.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.038+0000 7f1b97400640 1 -- 172.21.15.152:0/1555697890 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f1b88004030 con 0x7f1ba00af120 2024-04-20T11:29:01.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.039+0000 7f1b97400640 1 --2- 172.21.15.152:0/1555697890 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f1b7c06cad0 0x7f1b7c06ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:01.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.039+0000 7f1b97400640 1 -- 172.21.15.152:0/1555697890 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f1b880ae6e0 con 0x7f1ba00af120 2024-04-20T11:29:01.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.039+0000 7f1b9f400640 1 --2- 172.21.15.152:0/1555697890 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f1b7c06cad0 0x7f1b7c06ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:01.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.040+0000 7f1b9f400640 1 --2- 172.21.15.152:0/1555697890 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f1b7c06cad0 0x7f1b7c06ef90 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1b90006fd0 tx=0x7f1b90008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:01.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.043+0000 7f1b97400640 1 -- 172.21.15.152:0/1555697890 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f1b8807c7f0 con 0x7f1ba00af120 2024-04-20T11:29:01.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.232+0000 7f1ba6200640 1 -- 172.21.15.152:0/1555697890 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1ba006efb0 con 0x7f1ba00af120 2024-04-20T11:29:01.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.235+0000 7f1b97400640 1 -- 172.21.15.152:0/1555697890 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) ==== 74+0+16686 (secure 0 0 0) 0x7f1b8807c0f0 con 0x7f1ba00af120 2024-04-20T11:29:01.235 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:29:01.235 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":45,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:29:00.276971+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:28:58.007718+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-20T11:28:59.260647+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:29:01.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.238+0000 7f1ba6200640 1 -- 172.21.15.152:0/1555697890 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f1b7c06cad0 msgr2=0x7f1b7c06ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:01.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.238+0000 7f1ba6200640 1 --2- 172.21.15.152:0/1555697890 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f1b7c06cad0 0x7f1b7c06ef90 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1b90006fd0 tx=0x7f1b90008040 comp rx=0 tx=0).stop 2024-04-20T11:29:01.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.238+0000 7f1ba6200640 1 -- 172.21.15.152:0/1555697890 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1ba00af120 msgr2=0x7f1ba00ae580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:01.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.238+0000 7f1ba6200640 1 --2- 172.21.15.152:0/1555697890 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1ba00af120 0x7f1ba00ae580 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f1b8802c690 tx=0x7f1b88009750 comp rx=0 tx=0).stop 2024-04-20T11:29:01.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.238+0000 7f1ba6200640 1 -- 172.21.15.152:0/1555697890 shutdown_connections 2024-04-20T11:29:01.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.238+0000 7f1ba6200640 1 --2- 172.21.15.152:0/1555697890 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f1b7c06cad0 0x7f1b7c06ef90 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:01.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.238+0000 7f1ba6200640 1 --2- 172.21.15.152:0/1555697890 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1ba00af5a0 0x7f1ba0143760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:01.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.238+0000 7f1ba6200640 1 --2- 172.21.15.152:0/1555697890 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1ba00af120 0x7f1ba00ae580 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:01.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.238+0000 7f1ba6200640 1 --2- 172.21.15.152:0/1555697890 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b80003fc0 0x7f1ba00ad8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:01.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.238+0000 7f1ba6200640 1 -- 172.21.15.152:0/1555697890 >> 172.21.15.152:0/1555697890 conn(0x7f1ba0067cf0 msgr2=0x7f1ba006f5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:01.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.238+0000 7f1ba6200640 1 -- 172.21.15.152:0/1555697890 shutdown_connections 2024-04-20T11:29:01.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.238+0000 7f1ba6200640 1 -- 172.21.15.152:0/1555697890 wait complete. 2024-04-20T11:29:01.257 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:29:01.450 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.451+0000 7fa3e6c00640 1 Processor -- start 2024-04-20T11:29:01.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.451+0000 7fa3e6c00640 1 -- start start 2024-04-20T11:29:01.452 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.452+0000 7fa3e6c00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fa3e006f510 0x7fa3e0140dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:01.452 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.452+0000 7fa3e6c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa3e0141310 0x7fa3e0071cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:01.452 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.452+0000 7fa3e6c00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3e00cf0a0 con 0x7fa3e013e530 2024-04-20T11:29:01.452 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.452+0000 7fa3e6c00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fa3e0141b00 con 0x7fa3e006f510 2024-04-20T11:29:01.452 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.452+0000 7fa3e6c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa3e00cfc50 con 0x7fa3e0141310 2024-04-20T11:29:01.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.453+0000 7fa3e4e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fa3e006f510 0x7fa3e0140dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:01.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.453+0000 7fa3e4e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fa3e006f510 0x7fa3e0140dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.152:36614/0 (socket says 172.21.15.152:36614) 2024-04-20T11:29:01.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.453+0000 7fa3e4e00640 1 -- 172.21.15.152:0/228835166 learned_addr learned my addr 172.21.15.152:0/228835166 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:29:01.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.453+0000 7fa3e6200640 1 --2- 172.21.15.152:0/228835166 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa3e0141310 0x7fa3e0071cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:01.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.454+0000 7fa3e4e00640 1 -- 172.21.15.152:0/228835166 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa3e0141310 msgr2=0x7fa3e0071cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:01.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.454+0000 7fa3e4e00640 1 --2- 172.21.15.152:0/228835166 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa3e0141310 0x7fa3e0071cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:01.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.454+0000 7fa3e4e00640 1 -- 172.21.15.152:0/228835166 >> v1:172.21.15.175:6789/0 conn(0x7fa3e013e530 legacy=0x7fa3e006ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:01.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.454+0000 7fa3e4e00640 1 -- 172.21.15.152:0/228835166 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3e00d00e0 con 0x7fa3e006f510 2024-04-20T11:29:01.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.455+0000 7fa3e4e00640 1 --2- 172.21.15.152:0/228835166 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3e006f510 0x7fa3e0140dd0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc009870 tx=0x7fa3cc02b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ac14027dfb2c9b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:01.456 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.456+0000 7fa3dfe00640 1 -- 172.21.15.152:0/228835166 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc034070 con 0x7fa3e006f510 2024-04-20T11:29:01.456 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.456+0000 7fa3dfe00640 1 -- 172.21.15.152:0/228835166 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3cc030840 con 0x7fa3e006f510 2024-04-20T11:29:01.456 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.456+0000 7fa3dfe00640 1 -- 172.21.15.152:0/228835166 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc030c10 con 0x7fa3e006f510 2024-04-20T11:29:01.457 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.457+0000 7fa3e6c00640 1 -- 172.21.15.152:0/228835166 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3e006f510 msgr2=0x7fa3e0140dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:01.457 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.457+0000 7fa3e6c00640 1 --2- 172.21.15.152:0/228835166 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3e006f510 0x7fa3e0140dd0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc009870 tx=0x7fa3cc02b980 comp rx=0 tx=0).stop 2024-04-20T11:29:01.457 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.457+0000 7fa3e6c00640 1 -- 172.21.15.152:0/228835166 shutdown_connections 2024-04-20T11:29:01.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.457+0000 7fa3e6c00640 1 --2- 172.21.15.152:0/228835166 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa3e0141310 0x7fa3e0071cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:01.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.457+0000 7fa3e6c00640 1 --2- 172.21.15.152:0/228835166 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3e006f510 0x7fa3e0140dd0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:01.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.457+0000 7fa3e6c00640 1 -- 172.21.15.152:0/228835166 >> 172.21.15.152:0/228835166 conn(0x7fa3e0067ae0 msgr2=0x7fa3e006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:01.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.457+0000 7fa3e6c00640 1 -- 172.21.15.152:0/228835166 shutdown_connections 2024-04-20T11:29:01.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.457+0000 7fa3e6c00640 1 -- 172.21.15.152:0/228835166 wait complete. 2024-04-20T11:29:01.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.458+0000 7fa3e6c00640 1 Processor -- start 2024-04-20T11:29:01.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.459+0000 7fa3e6c00640 1 -- start start 2024-04-20T11:29:01.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.459+0000 7fa3e6c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa3e006f510 0x7fa3e0152fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:01.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.459+0000 7fa3e6c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3e013e530 0x7fa3e01534e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:01.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.459+0000 7fa3e6c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa3e0141310 0x7fa3e00b21b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:01.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.460+0000 7fa3e6c00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e00cf8e0 con 0x7fa3e006f510 2024-04-20T11:29:01.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.460+0000 7fa3e6c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e00cf600 con 0x7fa3e013e530 2024-04-20T11:29:01.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.460+0000 7fa3e6c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa3e01417c0 con 0x7fa3e0141310 2024-04-20T11:29:01.461 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.460+0000 7fa3e5800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa3e006f510 0x7fa3e0152fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:01.461 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.460+0000 7fa3e5800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa3e006f510 0x7fa3e0152fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:45170/0 (socket says 172.21.15.152:45170) 2024-04-20T11:29:01.462 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.460+0000 7fa3e5800640 1 -- 172.21.15.152:0/2463251116 learned_addr learned my addr 172.21.15.152:0/2463251116 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:29:01.462 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.460+0000 7fa3e4e00640 1 --2- 172.21.15.152:0/2463251116 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3e013e530 0x7fa3e01534e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:01.462 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.460+0000 7fa3e5800640 1 -- 172.21.15.152:0/2463251116 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa3e0141310 msgr2=0x7fa3e00b21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:01.462 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.460+0000 7fa3e6200640 1 --2- 172.21.15.152:0/2463251116 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa3e0141310 0x7fa3e00b21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:01.462 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.460+0000 7fa3e5800640 1 --2- 172.21.15.152:0/2463251116 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa3e0141310 0x7fa3e00b21b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:01.462 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.460+0000 7fa3e5800640 1 -- 172.21.15.152:0/2463251116 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3e013e530 msgr2=0x7fa3e01534e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:01.462 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.460+0000 7fa3e5800640 1 --2- 172.21.15.152:0/2463251116 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3e013e530 0x7fa3e01534e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:01.462 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.460+0000 7fa3e5800640 1 -- 172.21.15.152:0/2463251116 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3e013d3d0 con 0x7fa3e006f510 2024-04-20T11:29:01.462 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.460+0000 7fa3e4e00640 1 --2- 172.21.15.152:0/2463251116 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3e013e530 0x7fa3e01534e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:29:01.462 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.461+0000 7fa3e5800640 1 --2- 172.21.15.152:0/2463251116 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa3e006f510 0x7fa3e0152fa0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fa3d400c430 tx=0x7fa3d400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:01.462 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.461+0000 7fa3dd600640 1 -- 172.21.15.152:0/2463251116 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d40052d0 con 0x7fa3e006f510 2024-04-20T11:29:01.462 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.461+0000 7fa3dd600640 1 -- 172.21.15.152:0/2463251116 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3d4005490 con 0x7fa3e006f510 2024-04-20T11:29:01.463 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.461+0000 7fa3e6c00640 1 -- 172.21.15.152:0/2463251116 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3e013d6b0 con 0x7fa3e006f510 2024-04-20T11:29:01.463 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.461+0000 7fa3dd600640 1 -- 172.21.15.152:0/2463251116 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d4007260 con 0x7fa3e006f510 2024-04-20T11:29:01.463 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.461+0000 7fa3e6c00640 1 -- 172.21.15.152:0/2463251116 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3e013dc70 con 0x7fa3e006f510 2024-04-20T11:29:01.463 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.462+0000 7fa3e6c00640 1 -- 172.21.15.152:0/2463251116 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3e0075e60 con 0x7fa3e006f510 2024-04-20T11:29:01.463 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.463+0000 7fa3dd600640 1 -- 172.21.15.152:0/2463251116 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7fa3d4007400 con 0x7fa3e006f510 2024-04-20T11:29:01.464 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.464+0000 7fa3dd600640 1 --2- 172.21.15.152:0/2463251116 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fa3c006caa0 0x7fa3c006ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:01.464 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.464+0000 7fa3dd600640 1 -- 172.21.15.152:0/2463251116 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7fa3d408b470 con 0x7fa3e006f510 2024-04-20T11:29:01.464 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.464+0000 7fa3e4e00640 1 --2- 172.21.15.152:0/2463251116 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fa3c006caa0 0x7fa3c006ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:01.464 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.465+0000 7fa3e4e00640 1 --2- 172.21.15.152:0/2463251116 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fa3c006caa0 0x7fa3c006ef60 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc036000 tx=0x7fa3cc03b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:01.467 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.468+0000 7fa3dd600640 1 -- 172.21.15.152:0/2463251116 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fa3e00cf0a0 con 0x7fa3e006f510 2024-04-20T11:29:01.671 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.671+0000 7fa3e6c00640 1 -- 172.21.15.152:0/2463251116 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa3e00b3490 con 0x7fa3e006f510 2024-04-20T11:29:01.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.673+0000 7fa3dd600640 1 -- 172.21.15.152:0/2463251116 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) ==== 74+0+16686 (secure 0 0 0) 0x7fa3d4059580 con 0x7fa3e006f510 2024-04-20T11:29:01.674 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:29:01.674 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":45,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:29:00.276971+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:28:58.007718+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-20T11:28:59.260647+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:29:01.677 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.677+0000 7fa3e6c00640 1 -- 172.21.15.152:0/2463251116 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fa3c006caa0 msgr2=0x7fa3c006ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:01.677 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.677+0000 7fa3e6c00640 1 --2- 172.21.15.152:0/2463251116 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fa3c006caa0 0x7fa3c006ef60 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc036000 tx=0x7fa3cc03b040 comp rx=0 tx=0).stop 2024-04-20T11:29:01.677 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.677+0000 7fa3e6c00640 1 -- 172.21.15.152:0/2463251116 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa3e006f510 msgr2=0x7fa3e0152fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:01.677 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.677+0000 7fa3e6c00640 1 --2- 172.21.15.152:0/2463251116 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa3e006f510 0x7fa3e0152fa0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fa3d400c430 tx=0x7fa3d400c8f0 comp rx=0 tx=0).stop 2024-04-20T11:29:01.677 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.677+0000 7fa3e6c00640 1 -- 172.21.15.152:0/2463251116 shutdown_connections 2024-04-20T11:29:01.677 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.677+0000 7fa3e6c00640 1 --2- 172.21.15.152:0/2463251116 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fa3c006caa0 0x7fa3c006ef60 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:01.677 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.677+0000 7fa3e6c00640 1 --2- 172.21.15.152:0/2463251116 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa3e0141310 0x7fa3e00b21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:01.677 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.677+0000 7fa3e6c00640 1 --2- 172.21.15.152:0/2463251116 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3e013e530 0x7fa3e01534e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:01.677 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.677+0000 7fa3e6c00640 1 --2- 172.21.15.152:0/2463251116 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa3e006f510 0x7fa3e0152fa0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:01.678 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.677+0000 7fa3e6c00640 1 -- 172.21.15.152:0/2463251116 >> 172.21.15.152:0/2463251116 conn(0x7fa3e0067ae0 msgr2=0x7fa3e015d720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:01.678 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.677+0000 7fa3e6c00640 1 -- 172.21.15.152:0/2463251116 shutdown_connections 2024-04-20T11:29:01.678 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.677+0000 7fa3e6c00640 1 -- 172.21.15.152:0/2463251116 wait complete. 2024-04-20T11:29:01.695 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-20T11:29:01.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.903+0000 7f389a000640 1 Processor -- start 2024-04-20T11:29:01.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.903+0000 7f389a000640 1 -- start start 2024-04-20T11:29:01.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.905+0000 7f389a000640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f3894163230 0x7f38940d2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:01.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.905+0000 7f389a000640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38940d19c0 con 0x7f389415eb50 2024-04-20T11:29:01.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.905+0000 7f389a000640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3894162750 con 0x7f38940d33d0 2024-04-20T11:29:01.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.905+0000 7f389a000640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f3894162a30 con 0x7f3894163230 2024-04-20T11:29:01.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.905+0000 7f3898c00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f389415eb50 0x7f38940d0f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:42836/0 (socket says 172.21.15.152:42836) 2024-04-20T11:29:01.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.905+0000 7f3898c00640 1 -- 172.21.15.152:0/4152220931 learned_addr learned my addr 172.21.15.152:0/4152220931 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:29:01.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.906+0000 7f3893e00640 1 --2- 172.21.15.152:0/4152220931 >> v2:172.21.15.175:3300/0 conn(0x7f3894163230 0x7f38940d2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:01.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.906+0000 7f3893400640 1 -- 172.21.15.152:0/4152220931 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 350942855 0 0) 0x7f38940d19c0 con 0x7f389415eb50 2024-04-20T11:29:01.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.906+0000 7f3893400640 1 -- 172.21.15.152:0/4152220931 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3878003520 con 0x7f389415eb50 2024-04-20T11:29:01.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.906+0000 7f3893400640 1 -- 172.21.15.152:0/4152220931 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3726071407 0 0) 0x7f3878003520 con 0x7f389415eb50 2024-04-20T11:29:01.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.906+0000 7f3893400640 1 -- 172.21.15.152:0/4152220931 >> v2:172.21.15.175:3300/0 conn(0x7f3894163230 msgr2=0x7f38940d2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:01.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.906+0000 7f3893400640 1 --2- 172.21.15.152:0/4152220931 >> v2:172.21.15.175:3300/0 conn(0x7f3894163230 0x7f38940d2e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:01.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.906+0000 7f3893400640 1 -- 172.21.15.152:0/4152220931 >> v1:172.21.15.175:6789/0 conn(0x7f38940d33d0 legacy=0x7f38940cf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:01.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.906+0000 7f3893400640 1 -- 172.21.15.152:0/4152220931 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3894162d10 con 0x7f389415eb50 2024-04-20T11:29:01.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.906+0000 7f3893400640 1 -- 172.21.15.152:0/4152220931 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f3884001670 con 0x7f389415eb50 2024-04-20T11:29:01.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.906+0000 7f3893400640 1 -- 172.21.15.152:0/4152220931 >> v1:172.21.15.152:6789/0 conn(0x7f389415eb50 legacy=0x7f38940d0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:01.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.906+0000 7f3893400640 1 --2- 172.21.15.152:0/4152220931 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f38780039b0 0x7f3878003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:01.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.906+0000 7f3893400640 1 --2- 172.21.15.152:0/4152220931 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3878004800 0x7f3878004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:01.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.906+0000 7f3893400640 1 --2- 172.21.15.152:0/4152220931 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38780051c0 0x7f387800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:01.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.906+0000 7f3893400640 1 -- 172.21.15.152:0/4152220931 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f3894162a30 con 0x7f3878004800 2024-04-20T11:29:01.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.906+0000 7f3893400640 1 -- 172.21.15.152:0/4152220931 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f387800c0f0 con 0x7f38780051c0 2024-04-20T11:29:01.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.906+0000 7f3893400640 1 -- 172.21.15.152:0/4152220931 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f387800c2f0 con 0x7f38780039b0 2024-04-20T11:29:01.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.907+0000 7f3893e00640 1 --2- 172.21.15.152:0/4152220931 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3878004800 0x7f3878004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:01.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.907+0000 7f3893e00640 1 -- 172.21.15.152:0/4152220931 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f38780039b0 msgr2=0x7f3878003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:01.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.907+0000 7f3893e00640 1 --2- 172.21.15.152:0/4152220931 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f38780039b0 0x7f3878003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:01.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.907+0000 7f3893e00640 1 -- 172.21.15.152:0/4152220931 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38780051c0 msgr2=0x7f387800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:01.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.907+0000 7f3893e00640 1 --2- 172.21.15.152:0/4152220931 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38780051c0 0x7f387800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:01.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.907+0000 7f3893e00640 1 -- 172.21.15.152:0/4152220931 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3878003520 con 0x7f3878004800 2024-04-20T11:29:01.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.907+0000 7f3899600640 1 -- 172.21.15.152:0/4152220931 reap_dead start 2024-04-20T11:29:01.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.908+0000 7f3893e00640 1 --2- 172.21.15.152:0/4152220931 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3878004800 0x7f3878004c80 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f3880009ab0 tx=0x7f3878005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34a906e63764ab72 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:01.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.908+0000 7f3893400640 1 -- 172.21.15.152:0/4152220931 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3880009680 con 0x7f3878004800 2024-04-20T11:29:01.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.908+0000 7f3893400640 1 -- 172.21.15.152:0/4152220931 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3880005990 con 0x7f3878004800 2024-04-20T11:29:01.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.908+0000 7f3893400640 1 -- 172.21.15.152:0/4152220931 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3880005d80 con 0x7f3878004800 2024-04-20T11:29:01.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.909+0000 7f389a000640 1 -- 172.21.15.152:0/4152220931 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3878004800 msgr2=0x7f3878004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:01.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.909+0000 7f389a000640 1 --2- 172.21.15.152:0/4152220931 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3878004800 0x7f3878004c80 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f3880009ab0 tx=0x7f3878005830 comp rx=0 tx=0).stop 2024-04-20T11:29:01.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.909+0000 7f389a000640 1 -- 172.21.15.152:0/4152220931 shutdown_connections 2024-04-20T11:29:01.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.909+0000 7f389a000640 1 --2- 172.21.15.152:0/4152220931 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3878004800 0x7f3878004c80 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:01.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.909+0000 7f389a000640 1 -- 172.21.15.152:0/4152220931 >> 172.21.15.152:0/4152220931 conn(0x7f389408bfd0 msgr2=0x7f38940d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:01.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.909+0000 7f389a000640 1 -- 172.21.15.152:0/4152220931 shutdown_connections 2024-04-20T11:29:01.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.909+0000 7f389a000640 1 -- 172.21.15.152:0/4152220931 wait complete. 2024-04-20T11:29:01.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.911+0000 7f389a000640 1 Processor -- start 2024-04-20T11:29:01.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.911+0000 7f389a000640 1 -- start start 2024-04-20T11:29:01.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.912+0000 7f389a000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3878004800 0x7f38941519f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:01.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.912+0000 7f389a000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f38940b5a20 0x7f38940b5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:01.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.912+0000 7f389a000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38940b9470 0x7f38940b98f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:01.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.912+0000 7f389a000640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f38940d3010 con 0x7f3878004800 2024-04-20T11:29:01.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.912+0000 7f389a000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f38940d2e90 con 0x7f38940b9470 2024-04-20T11:29:01.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.912+0000 7f389a000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f38940d3190 con 0x7f38940b5a20 2024-04-20T11:29:01.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.912+0000 7f3898c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3878004800 0x7f38941519f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:01.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.912+0000 7f3898c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3878004800 0x7f38941519f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:45194/0 (socket says 172.21.15.152:45194) 2024-04-20T11:29:01.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.912+0000 7f3898c00640 1 -- 172.21.15.152:0/223292298 learned_addr learned my addr 172.21.15.152:0/223292298 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:29:01.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.913+0000 7f3898c00640 1 -- 172.21.15.152:0/223292298 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f38940b5a20 msgr2=0x7f38940b5ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:01.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.913+0000 7f3898c00640 1 --2- 172.21.15.152:0/223292298 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f38940b5a20 0x7f38940b5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:01.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.913+0000 7f3898c00640 1 -- 172.21.15.152:0/223292298 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38940b9470 msgr2=0x7f38940b98f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:01.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.913+0000 7f3898c00640 1 --2- 172.21.15.152:0/223292298 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38940b9470 0x7f38940b98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:01.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.913+0000 7f3898c00640 1 -- 172.21.15.152:0/223292298 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38940b64e0 con 0x7f3878004800 2024-04-20T11:29:01.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.913+0000 7f3898c00640 1 --2- 172.21.15.152:0/223292298 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3878004800 0x7f38941519f0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f388400c7c0 tx=0x7f388400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:01.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.913+0000 7f3890c00640 1 -- 172.21.15.152:0/223292298 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3884018070 con 0x7f3878004800 2024-04-20T11:29:01.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.913+0000 7f3890c00640 1 -- 172.21.15.152:0/223292298 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38840074e0 con 0x7f3878004800 2024-04-20T11:29:01.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.913+0000 7f3890c00640 1 -- 172.21.15.152:0/223292298 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3884004a00 con 0x7f3878004800 2024-04-20T11:29:01.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.913+0000 7f389a000640 1 -- 172.21.15.152:0/223292298 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38940b67c0 con 0x7f3878004800 2024-04-20T11:29:01.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.914+0000 7f389a000640 1 -- 172.21.15.152:0/223292298 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3894160150 con 0x7f3878004800 2024-04-20T11:29:01.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.915+0000 7f389a000640 1 -- 172.21.15.152:0/223292298 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3894075e60 con 0x7f3878004800 2024-04-20T11:29:01.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.916+0000 7f3890c00640 1 -- 172.21.15.152:0/223292298 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f3884004ba0 con 0x7f3878004800 2024-04-20T11:29:01.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.917+0000 7f3890c00640 1 --2- 172.21.15.152:0/223292298 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f387406cad0 0x7f387406ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:01.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.917+0000 7f3890c00640 1 -- 172.21.15.152:0/223292298 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f388408b4a0 con 0x7f3878004800 2024-04-20T11:29:01.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.917+0000 7f3893e00640 1 --2- 172.21.15.152:0/223292298 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f387406cad0 0x7f387406ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:01.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.918+0000 7f3893e00640 1 --2- 172.21.15.152:0/223292298 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f387406cad0 0x7f387406ef90 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f3880035000 tx=0x7f388002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:01.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:01.921+0000 7f3890c00640 1 -- 172.21.15.152:0/223292298 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f3884059640 con 0x7f3878004800 2024-04-20T11:29:02.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:02.164+0000 7f389a000640 1 -- 172.21.15.152:0/223292298 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f3894160320 con 0x7f3878004800 2024-04-20T11:29:02.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:02.859+0000 7f3890c00640 1 -- 172.21.15.152:0/223292298 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v23) ==== 118+0+0 (secure 0 0 0) 0x7f3884058f40 con 0x7f3878004800 2024-04-20T11:29:02.863 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:02.864+0000 7f389a000640 1 -- 172.21.15.152:0/223292298 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f387406cad0 msgr2=0x7f387406ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:02.863 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:02.864+0000 7f389a000640 1 --2- 172.21.15.152:0/223292298 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f387406cad0 0x7f387406ef90 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f3880035000 tx=0x7f388002e000 comp rx=0 tx=0).stop 2024-04-20T11:29:02.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:02.864+0000 7f389a000640 1 -- 172.21.15.152:0/223292298 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3878004800 msgr2=0x7f38941519f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:02.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:02.864+0000 7f389a000640 1 --2- 172.21.15.152:0/223292298 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3878004800 0x7f38941519f0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f388400c7c0 tx=0x7f388400cc80 comp rx=0 tx=0).stop 2024-04-20T11:29:02.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:02.864+0000 7f389a000640 1 -- 172.21.15.152:0/223292298 shutdown_connections 2024-04-20T11:29:02.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:02.864+0000 7f389a000640 1 --2- 172.21.15.152:0/223292298 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f387406cad0 0x7f387406ef90 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:02.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:02.864+0000 7f389a000640 1 --2- 172.21.15.152:0/223292298 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38940b9470 0x7f38940b98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:02.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:02.864+0000 7f389a000640 1 --2- 172.21.15.152:0/223292298 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f38940b5a20 0x7f38940b5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:02.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:02.864+0000 7f389a000640 1 --2- 172.21.15.152:0/223292298 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3878004800 0x7f38941519f0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:02.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:02.864+0000 7f389a000640 1 -- 172.21.15.152:0/223292298 >> 172.21.15.152:0/223292298 conn(0x7f389408bfd0 msgr2=0x7f389415c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:02.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:02.864+0000 7f389a000640 1 -- 172.21.15.152:0/223292298 shutdown_connections 2024-04-20T11:29:02.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:02.864+0000 7f389a000640 1 -- 172.21.15.152:0/223292298 wait complete. 2024-04-20T11:29:02.885 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-20T11:29:02.885 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-20T11:29:03.092 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:03.092+0000 7f2acc600640 1 Processor -- start 2024-04-20T11:29:03.092 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:03.093+0000 7f2acc600640 1 -- start start 2024-04-20T11:29:03.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:03.094+0000 7f2acc600640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f2ac40d46a0 0x7f2ac40d4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:03.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:03.094+0000 7f2acc600640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ac40d19a0 con 0x7f2ac40d0d90 2024-04-20T11:29:03.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:03.094+0000 7f2acc600640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ac40d1c80 con 0x7f2ac415e9e0 2024-04-20T11:29:03.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:03.094+0000 7f2acc600640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f2ac406da30 con 0x7f2ac40d46a0 2024-04-20T11:29:03.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:03.095+0000 7f2ac9200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f2ac40d46a0 0x7f2ac40d4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:03.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:03.095+0000 7f2ac9200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f2ac40d46a0 0x7f2ac40d4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:45204/0 (socket says 172.21.15.152:45204) 2024-04-20T11:29:03.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:03.095+0000 7f2ac9200640 1 -- 172.21.15.152:0/2845580834 learned_addr learned my addr 172.21.15.152:0/2845580834 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:29:03.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:03.095+0000 7f2ac9200640 1 -- 172.21.15.152:0/2845580834 >> v1:172.21.15.175:6789/0 conn(0x7f2ac415e9e0 legacy=0x7f2ac406d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:03.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:03.095+0000 7f2ac9200640 1 -- 172.21.15.152:0/2845580834 >> v1:172.21.15.152:6789/0 conn(0x7f2ac40d0d90 legacy=0x7f2ac40cfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:03.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:03.095+0000 7f2ac9200640 1 -- 172.21.15.152:0/2845580834 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ac40d1f60 con 0x7f2ac40d46a0 2024-04-20T11:29:03.096 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:03.096+0000 7f2ac9200640 1 --2- 172.21.15.152:0/2845580834 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ac40d46a0 0x7f2ac40d4b00 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f2ab0009870 tx=0x7f2ab002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d4d7ee260565dbf server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:03.096 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:03.097+0000 7f2ac3e00640 1 -- 172.21.15.152:0/2845580834 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ab0034070 con 0x7f2ac40d46a0 2024-04-20T11:29:03.096 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:03.097+0000 7f2ac3e00640 1 -- 172.21.15.152:0/2845580834 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ab0030860 con 0x7f2ac40d46a0 2024-04-20T11:29:03.096 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:03.097+0000 7f2ac3e00640 1 -- 172.21.15.152:0/2845580834 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ab0030c30 con 0x7f2ac40d46a0 2024-04-20T11:29:03.097 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:03.097+0000 7f2acc600640 1 -- 172.21.15.152:0/2845580834 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ac40d46a0 msgr2=0x7f2ac40d4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:03.097 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:03.097+0000 7f2acc600640 1 --2- 172.21.15.152:0/2845580834 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ac40d46a0 0x7f2ac40d4b00 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f2ab0009870 tx=0x7f2ab002b9a0 comp rx=0 tx=0).stop 2024-04-20T11:29:03.097 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:03.097+0000 7f2acc600640 1 -- 172.21.15.152:0/2845580834 shutdown_connections 2024-04-20T11:29:03.097 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:03.097+0000 7f2acc600640 1 --2- 172.21.15.152:0/2845580834 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ac40d46a0 0x7f2ac40d4b00 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:03.097 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:03.097+0000 7f2acc600640 1 -- 172.21.15.152:0/2845580834 >> 172.21.15.152:0/2845580834 conn(0x7f2ac409dab0 msgr2=0x7f2ac408bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:03.097 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:03.097+0000 7f2acc600640 1 -- 172.21.15.152:0/2845580834 shutdown_connections 2024-04-20T11:29:03.097 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:03.097+0000 7f2acc600640 1 -- 172.21.15.152:0/2845580834 wait complete. 2024-04-20T11:29:03.098 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:03.099+0000 7f2acc600640 1 Processor -- start 2024-04-20T11:29:03.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:03.099+0000 7f2acc600640 1 -- start start 2024-04-20T11:29:03.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:03.100+0000 7f2acc600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2ac40d0d90 0x7f2ac40ce220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:03.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:03.100+0000 7f2acc600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ac40d46a0 0x7f2ac4160b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:03.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:03.100+0000 7f2acc600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ac415e9e0 0x7f2ac406de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:03.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:03.100+0000 7f2acc600640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac40cf880 con 0x7f2ac40d46a0 2024-04-20T11:29:03.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:03.100+0000 7f2acc600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac40d5040 con 0x7f2ac415e9e0 2024-04-20T11:29:03.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:03.100+0000 7f2acc600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2ac415edc0 con 0x7f2ac40d0d90 2024-04-20T11:29:03.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:03.101+0000 7f2ac9200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ac40d46a0 0x7f2ac4160b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:03.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:03.101+0000 7f2ac9200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ac40d46a0 0x7f2ac4160b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:45218/0 (socket says 172.21.15.152:45218) 2024-04-20T11:29:03.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:03.101+0000 7f2ac9200640 1 -- 172.21.15.152:0/2948386447 learned_addr learned my addr 172.21.15.152:0/2948386447 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:29:03.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:03.101+0000 7f2ac9c00640 1 --2- 172.21.15.152:0/2948386447 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2ac40d0d90 0x7f2ac40ce220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:03.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:03.101+0000 7f2ac9200640 1 -- 172.21.15.152:0/2948386447 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2ac40d0d90 msgr2=0x7f2ac40ce220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:03.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:03.101+0000 7f2aca600640 1 --2- 172.21.15.152:0/2948386447 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ac415e9e0 0x7f2ac406de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:03.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:03.101+0000 7f2ac9200640 1 --2- 172.21.15.152:0/2948386447 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2ac40d0d90 0x7f2ac40ce220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:03.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:03.101+0000 7f2ac9200640 1 -- 172.21.15.152:0/2948386447 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ac415e9e0 msgr2=0x7f2ac406de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:03.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:03.101+0000 7f2ac9200640 1 --2- 172.21.15.152:0/2948386447 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ac415e9e0 0x7f2ac406de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:03.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:03.101+0000 7f2ac9200640 1 -- 172.21.15.152:0/2948386447 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ac40cd180 con 0x7f2ac40d46a0 2024-04-20T11:29:03.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:03.101+0000 7f2ac9c00640 1 --2- 172.21.15.152:0/2948386447 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2ac40d0d90 0x7f2ac40ce220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:29:03.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:03.101+0000 7f2ac9200640 1 --2- 172.21.15.152:0/2948386447 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ac40d46a0 0x7f2ac4160b20 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f2ab0000c00 tx=0x7f2ab0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:03.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:03.102+0000 7f2ac1600640 1 -- 172.21.15.152:0/2948386447 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ab0048070 con 0x7f2ac40d46a0 2024-04-20T11:29:03.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:03.102+0000 7f2ac1600640 1 -- 172.21.15.152:0/2948386447 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ab0030660 con 0x7f2ac40d46a0 2024-04-20T11:29:03.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:03.102+0000 7f2ac1600640 1 -- 172.21.15.152:0/2948386447 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ab0034040 con 0x7f2ac40d46a0 2024-04-20T11:29:03.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:03.102+0000 7f2acc600640 1 -- 172.21.15.152:0/2948386447 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ac40cd460 con 0x7f2ac40d46a0 2024-04-20T11:29:03.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:03.102+0000 7f2acc600640 1 -- 172.21.15.152:0/2948386447 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ac40cda20 con 0x7f2ac40d46a0 2024-04-20T11:29:03.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:03.103+0000 7f2acc600640 1 -- 172.21.15.152:0/2948386447 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ac40689f0 con 0x7f2ac40d46a0 2024-04-20T11:29:03.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:03.104+0000 7f2ac1600640 1 -- 172.21.15.152:0/2948386447 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f2ab0037070 con 0x7f2ac40d46a0 2024-04-20T11:29:03.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:03.105+0000 7f2ac1600640 1 --2- 172.21.15.152:0/2948386447 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f2aa406caa0 0x7f2aa406ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:03.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:03.105+0000 7f2ac1600640 1 -- 172.21.15.152:0/2948386447 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f2ab00adfe0 con 0x7f2ac40d46a0 2024-04-20T11:29:03.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:03.105+0000 7f2ac9c00640 1 --2- 172.21.15.152:0/2948386447 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f2aa406caa0 0x7f2aa406ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:03.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:03.106+0000 7f2ac9c00640 1 --2- 172.21.15.152:0/2948386447 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f2aa406caa0 0x7f2aa406ef60 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f2ab4007e90 tx=0x7f2ab40073a0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:03.109 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:03.110+0000 7f2ac1600640 1 -- 172.21.15.152:0/2948386447 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f2ab007c100 con 0x7f2ac40d46a0 2024-04-20T11:29:03.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:03.350+0000 7f2acc600640 1 -- 172.21.15.152:0/2948386447 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f2ac406ea00 con 0x7f2ac40d46a0 2024-04-20T11:29:03.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:03.863+0000 7f2ac1600640 1 -- 172.21.15.152:0/2948386447 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v24) ==== 122+0+0 (secure 0 0 0) 0x7f2ab007ba00 con 0x7f2ac40d46a0 2024-04-20T11:29:03.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:03.866+0000 7f2acc600640 1 -- 172.21.15.152:0/2948386447 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f2aa406caa0 msgr2=0x7f2aa406ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:03.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:03.866+0000 7f2acc600640 1 --2- 172.21.15.152:0/2948386447 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f2aa406caa0 0x7f2aa406ef60 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f2ab4007e90 tx=0x7f2ab40073a0 comp rx=0 tx=0).stop 2024-04-20T11:29:03.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:03.866+0000 7f2acc600640 1 -- 172.21.15.152:0/2948386447 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ac40d46a0 msgr2=0x7f2ac4160b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:03.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:03.866+0000 7f2acc600640 1 --2- 172.21.15.152:0/2948386447 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ac40d46a0 0x7f2ac4160b20 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f2ab0000c00 tx=0x7f2ab0004730 comp rx=0 tx=0).stop 2024-04-20T11:29:03.867 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:03.866+0000 7f2acc600640 1 -- 172.21.15.152:0/2948386447 shutdown_connections 2024-04-20T11:29:03.867 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:03.866+0000 7f2acc600640 1 --2- 172.21.15.152:0/2948386447 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f2aa406caa0 0x7f2aa406ef60 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:03.867 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:03.867+0000 7f2acc600640 1 --2- 172.21.15.152:0/2948386447 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ac415e9e0 0x7f2ac406de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:03.867 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:03.867+0000 7f2acc600640 1 --2- 172.21.15.152:0/2948386447 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ac40d46a0 0x7f2ac4160b20 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:03.867 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:03.867+0000 7f2acc600640 1 --2- 172.21.15.152:0/2948386447 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2ac40d0d90 0x7f2ac40ce220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:03.867 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:03.867+0000 7f2acc600640 1 -- 172.21.15.152:0/2948386447 >> 172.21.15.152:0/2948386447 conn(0x7f2ac409dab0 msgr2=0x7f2ac40d2480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:03.867 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:03.867+0000 7f2acc600640 1 -- 172.21.15.152:0/2948386447 shutdown_connections 2024-04-20T11:29:03.867 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:03.867+0000 7f2acc600640 1 -- 172.21.15.152:0/2948386447 wait complete. 2024-04-20T11:29:03.888 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-20T11:29:03.889 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:29:04.079 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.079+0000 7fa56bc00640 1 Processor -- start 2024-04-20T11:29:04.079 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.080+0000 7fa56bc00640 1 -- start start 2024-04-20T11:29:04.081 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.081+0000 7fa56bc00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fa5640cd0d0 0x7fa5640cd580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:04.081 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.081+0000 7fa56bc00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa564071fe0 0x7fa564072470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:04.081 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.081+0000 7fa56bc00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa564146340 con 0x7fa564145f60 2024-04-20T11:29:04.081 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.081+0000 7fa56bc00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fa56406e9f0 con 0x7fa5640cd0d0 2024-04-20T11:29:04.081 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.081+0000 7fa56bc00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa564072d50 con 0x7fa564071fe0 2024-04-20T11:29:04.082 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.082+0000 7fa569c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa564071fe0 0x7fa564072470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:04.082 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.082+0000 7fa563e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fa5640cd0d0 0x7fa5640cd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:04.082 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.082+0000 7fa569c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa564071fe0 0x7fa564072470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.152:59176/0 (socket says 172.21.15.152:59176) 2024-04-20T11:29:04.082 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.082+0000 7fa569c00640 1 -- 172.21.15.152:0/3101795441 learned_addr learned my addr 172.21.15.152:0/3101795441 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:29:04.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.083+0000 7fa569c00640 1 -- 172.21.15.152:0/3101795441 >> v2:172.21.15.175:3300/0 conn(0x7fa5640cd0d0 msgr2=0x7fa5640cd580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:04.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.083+0000 7fa569c00640 1 --2- 172.21.15.152:0/3101795441 >> v2:172.21.15.175:3300/0 conn(0x7fa5640cd0d0 0x7fa5640cd580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:04.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.083+0000 7fa569c00640 1 -- 172.21.15.152:0/3101795441 >> v1:172.21.15.175:6789/0 conn(0x7fa564145f60 legacy=0x7fa56406ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:04.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.083+0000 7fa569c00640 1 -- 172.21.15.152:0/3101795441 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa56406e0c0 con 0x7fa564071fe0 2024-04-20T11:29:04.084 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.084+0000 7fa569c00640 1 --2- 172.21.15.152:0/3101795441 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa564071fe0 0x7fa564072470 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa55c009870 tx=0x7fa55c02b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a981a24655a95c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:04.084 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.085+0000 7fa563400640 1 -- 172.21.15.152:0/3101795441 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa55c034070 con 0x7fa564071fe0 2024-04-20T11:29:04.085 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.085+0000 7fa563400640 1 -- 172.21.15.152:0/3101795441 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa55c030840 con 0x7fa564071fe0 2024-04-20T11:29:04.085 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.085+0000 7fa563400640 1 -- 172.21.15.152:0/3101795441 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa55c030c10 con 0x7fa564071fe0 2024-04-20T11:29:04.085 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.086+0000 7fa56bc00640 1 -- 172.21.15.152:0/3101795441 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa564071fe0 msgr2=0x7fa564072470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:04.085 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.086+0000 7fa56bc00640 1 --2- 172.21.15.152:0/3101795441 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa564071fe0 0x7fa564072470 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa55c009870 tx=0x7fa55c02b980 comp rx=0 tx=0).stop 2024-04-20T11:29:04.086 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.086+0000 7fa56bc00640 1 -- 172.21.15.152:0/3101795441 shutdown_connections 2024-04-20T11:29:04.086 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.086+0000 7fa56bc00640 1 --2- 172.21.15.152:0/3101795441 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa564071fe0 0x7fa564072470 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:04.086 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.086+0000 7fa56bc00640 1 --2- 172.21.15.152:0/3101795441 >> v2:172.21.15.175:3300/0 conn(0x7fa5640cd0d0 0x7fa5640cd580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:04.086 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.086+0000 7fa56bc00640 1 -- 172.21.15.152:0/3101795441 >> 172.21.15.152:0/3101795441 conn(0x7fa564067cf0 msgr2=0x7fa56406b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:04.086 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.086+0000 7fa56bc00640 1 -- 172.21.15.152:0/3101795441 shutdown_connections 2024-04-20T11:29:04.086 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.086+0000 7fa56bc00640 1 -- 172.21.15.152:0/3101795441 wait complete. 2024-04-20T11:29:04.087 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.088+0000 7fa56bc00640 1 Processor -- start 2024-04-20T11:29:04.087 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.088+0000 7fa56bc00640 1 -- start start 2024-04-20T11:29:04.089 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.089+0000 7fa56bc00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa564071fe0 0x7fa564159410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:04.089 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.089+0000 7fa56bc00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa5640cd0d0 0x7fa5641e6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:04.089 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.089+0000 7fa56bc00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa564145f60 0x7fa5641e67a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:04.089 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.089+0000 7fa56bc00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fa5640729b0 con 0x7fa564071fe0 2024-04-20T11:29:04.090 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.089+0000 7fa56bc00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa56406e680 con 0x7fa5640cd0d0 2024-04-20T11:29:04.090 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.089+0000 7fa56bc00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa564072b30 con 0x7fa564145f60 2024-04-20T11:29:04.090 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.089+0000 7fa569200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa564071fe0 0x7fa564159410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:04.090 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.089+0000 7fa569200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa564071fe0 0x7fa564159410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:45228/0 (socket says 172.21.15.152:45228) 2024-04-20T11:29:04.090 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.089+0000 7fa569200640 1 -- 172.21.15.152:0/2675357347 learned_addr learned my addr 172.21.15.152:0/2675357347 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:29:04.090 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.090+0000 7fa569c00640 1 --2- 172.21.15.152:0/2675357347 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa564145f60 0x7fa5641e67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:04.090 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.090+0000 7fa569200640 1 -- 172.21.15.152:0/2675357347 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa564145f60 msgr2=0x7fa5641e67a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:04.090 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.090+0000 7fa569200640 1 --2- 172.21.15.152:0/2675357347 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa564145f60 0x7fa5641e67a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:04.090 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.090+0000 7fa563e00640 1 --2- 172.21.15.152:0/2675357347 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa5640cd0d0 0x7fa5641e6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:04.090 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.090+0000 7fa569200640 1 -- 172.21.15.152:0/2675357347 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa5640cd0d0 msgr2=0x7fa5641e6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:04.090 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.090+0000 7fa569200640 1 --2- 172.21.15.152:0/2675357347 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa5640cd0d0 0x7fa5641e6260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:04.090 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.090+0000 7fa569200640 1 -- 172.21.15.152:0/2675357347 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5640ad8b0 con 0x7fa564071fe0 2024-04-20T11:29:04.090 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.090+0000 7fa569c00640 1 --2- 172.21.15.152:0/2675357347 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa564145f60 0x7fa5641e67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:29:04.091 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.090+0000 7fa569200640 1 --2- 172.21.15.152:0/2675357347 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa564071fe0 0x7fa564159410 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fa55800c6c0 tx=0x7fa55800cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:04.091 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.090+0000 7fa560c00640 1 -- 172.21.15.152:0/2675357347 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa558003b90 con 0x7fa564071fe0 2024-04-20T11:29:04.091 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.090+0000 7fa560c00640 1 -- 172.21.15.152:0/2675357347 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa558003d30 con 0x7fa564071fe0 2024-04-20T11:29:04.092 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.090+0000 7fa560c00640 1 -- 172.21.15.152:0/2675357347 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa558005310 con 0x7fa564071fe0 2024-04-20T11:29:04.092 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.090+0000 7fa56bc00640 1 -- 172.21.15.152:0/2675357347 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5640adb90 con 0x7fa564071fe0 2024-04-20T11:29:04.092 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.091+0000 7fa56bc00640 1 -- 172.21.15.152:0/2675357347 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5640ae150 con 0x7fa564071fe0 2024-04-20T11:29:04.092 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.092+0000 7fa56bc00640 1 -- 172.21.15.152:0/2675357347 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5640689f0 con 0x7fa564071fe0 2024-04-20T11:29:04.092 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.093+0000 7fa560c00640 1 -- 172.21.15.152:0/2675357347 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7fa5580054b0 con 0x7fa564071fe0 2024-04-20T11:29:04.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.093+0000 7fa560c00640 1 --2- 172.21.15.152:0/2675357347 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fa54406caa0 0x7fa54406ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:04.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.094+0000 7fa560c00640 1 -- 172.21.15.152:0/2675357347 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7fa55808be70 con 0x7fa564071fe0 2024-04-20T11:29:04.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.094+0000 7fa563e00640 1 --2- 172.21.15.152:0/2675357347 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fa54406caa0 0x7fa54406ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:04.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.094+0000 7fa563e00640 1 --2- 172.21.15.152:0/2675357347 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fa54406caa0 0x7fa54406ef60 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa56415a630 tx=0x7fa550008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:04.097 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.098+0000 7fa560c00640 1 -- 172.21.15.152:0/2675357347 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fa558059f30 con 0x7fa564071fe0 2024-04-20T11:29:04.335 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.335+0000 7fa56bc00640 1 -- 172.21.15.152:0/2675357347 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa56415fc60 con 0x7fa564071fe0 2024-04-20T11:29:04.336 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.336+0000 7fa560c00640 1 -- 172.21.15.152:0/2675357347 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+5208 (secure 0 0 0) 0x7fa558059830 con 0x7fa564071fe0 2024-04-20T11:29:04.336 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:29:04.336 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4826,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6834/2855025590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2855025590},{"type":"v1","addr":"172.21.15.175:6834","nonce":2855025590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6836/3887716755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":3887716755},{"type":"v1","addr":"172.21.15.152:6836","nonce":3887716755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5712,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/698387399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6835","nonce":698387399},{"type":"v1","addr":"172.21.15.152:6837","nonce":698387399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T11:29:00.276881+0000","modified":"2024-04-20T11:29:03.352174+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5724},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5724":{"gid":5724,"name":"d","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6835/3704251089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6833","nonce":3704251089},{"type":"v1","addr":"172.21.15.175:6835","nonce":3704251089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5724,"qdb_cluster":[5724]},"id":3}]} 2024-04-20T11:29:04.336 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 24 2024-04-20T11:29:04.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.339+0000 7fa56bc00640 1 -- 172.21.15.152:0/2675357347 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fa54406caa0 msgr2=0x7fa54406ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:04.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.339+0000 7fa56bc00640 1 --2- 172.21.15.152:0/2675357347 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fa54406caa0 0x7fa54406ef60 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa56415a630 tx=0x7fa550008040 comp rx=0 tx=0).stop 2024-04-20T11:29:04.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.339+0000 7fa56bc00640 1 -- 172.21.15.152:0/2675357347 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa564071fe0 msgr2=0x7fa564159410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:04.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.339+0000 7fa56bc00640 1 --2- 172.21.15.152:0/2675357347 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa564071fe0 0x7fa564159410 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fa55800c6c0 tx=0x7fa55800cb80 comp rx=0 tx=0).stop 2024-04-20T11:29:04.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.339+0000 7fa56bc00640 1 -- 172.21.15.152:0/2675357347 shutdown_connections 2024-04-20T11:29:04.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.339+0000 7fa56bc00640 1 --2- 172.21.15.152:0/2675357347 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fa54406caa0 0x7fa54406ef60 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:04.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.339+0000 7fa56bc00640 1 --2- 172.21.15.152:0/2675357347 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa564145f60 0x7fa5641e67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:04.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.339+0000 7fa56bc00640 1 --2- 172.21.15.152:0/2675357347 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa5640cd0d0 0x7fa5641e6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:04.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.339+0000 7fa56bc00640 1 --2- 172.21.15.152:0/2675357347 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa564071fe0 0x7fa564159410 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:04.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.339+0000 7fa56bc00640 1 -- 172.21.15.152:0/2675357347 >> 172.21.15.152:0/2675357347 conn(0x7fa564067cf0 msgr2=0x7fa56415a6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:04.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.339+0000 7fa56bc00640 1 -- 172.21.15.152:0/2675357347 shutdown_connections 2024-04-20T11:29:04.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.339+0000 7fa56bc00640 1 -- 172.21.15.152:0/2675357347 wait complete. 2024-04-20T11:29:04.355 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:29:04.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.552+0000 7fd6d6800640 1 Processor -- start 2024-04-20T11:29:04.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.552+0000 7fd6d6800640 1 -- start start 2024-04-20T11:29:04.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.553+0000 7fd6d6800640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fd6d015b5e0 0x7fd6d006eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:04.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.553+0000 7fd6d6800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fd6d00b4c50 0x7fd6d00b5100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:04.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.553+0000 7fd6d6800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd6d006f340 0x7fd6d006f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:04.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.553+0000 7fd6d6800640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fd6d006de10 con 0x7fd6d00b4c50 2024-04-20T11:29:04.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.553+0000 7fd6d6800640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fd6d006fca0 con 0x7fd6d015b5e0 2024-04-20T11:29:04.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.553+0000 7fd6d6800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd6d00b5770 con 0x7fd6d006f340 2024-04-20T11:29:04.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.554+0000 7fd6c7e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fd6d00b4c50 0x7fd6d00b5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:04.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.554+0000 7fd6c7e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fd6d00b4c50 0x7fd6d00b5100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:45234/0 (socket says 172.21.15.152:45234) 2024-04-20T11:29:04.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.554+0000 7fd6c7e00640 1 -- 172.21.15.152:0/963017579 learned_addr learned my addr 172.21.15.152:0/963017579 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:29:04.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.554+0000 7fd6cfe00640 1 --2- 172.21.15.152:0/963017579 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd6d006f340 0x7fd6d006f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:04.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.554+0000 7fd6c7e00640 1 -- 172.21.15.152:0/963017579 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd6d006f340 msgr2=0x7fd6d006f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:04.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.554+0000 7fd6cf400640 1 --2- 172.21.15.152:0/963017579 >> v2:172.21.15.175:3300/0 conn(0x7fd6d015b5e0 0x7fd6d006eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:04.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.554+0000 7fd6c7e00640 1 --2- 172.21.15.152:0/963017579 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd6d006f340 0x7fd6d006f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:04.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.554+0000 7fd6c7e00640 1 -- 172.21.15.152:0/963017579 >> v2:172.21.15.175:3300/0 conn(0x7fd6d015b5e0 msgr2=0x7fd6d006eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:04.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.554+0000 7fd6c7e00640 1 --2- 172.21.15.152:0/963017579 >> v2:172.21.15.175:3300/0 conn(0x7fd6d015b5e0 0x7fd6d006eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:04.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.554+0000 7fd6c7e00640 1 -- 172.21.15.152:0/963017579 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6d006e060 con 0x7fd6d00b4c50 2024-04-20T11:29:04.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.554+0000 7fd6cf400640 1 --2- 172.21.15.152:0/963017579 >> v2:172.21.15.175:3300/0 conn(0x7fd6d015b5e0 0x7fd6d006eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:29:04.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.555+0000 7fd6c7e00640 1 --2- 172.21.15.152:0/963017579 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd6d00b4c50 0x7fd6d00b5100 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fd6d0072fa0 tx=0x7fd6b802b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7077cbe1cff750f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:04.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.556+0000 7fd6cea00640 1 -- 172.21.15.152:0/963017579 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6b80317d0 con 0x7fd6d00b4c50 2024-04-20T11:29:04.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.556+0000 7fd6cea00640 1 -- 172.21.15.152:0/963017579 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6b8037070 con 0x7fd6d00b4c50 2024-04-20T11:29:04.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.556+0000 7fd6cea00640 1 -- 172.21.15.152:0/963017579 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6b8031b90 con 0x7fd6d00b4c50 2024-04-20T11:29:04.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.556+0000 7fd6d6800640 1 -- 172.21.15.152:0/963017579 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd6d00b4c50 msgr2=0x7fd6d00b5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:04.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.556+0000 7fd6d6800640 1 --2- 172.21.15.152:0/963017579 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd6d00b4c50 0x7fd6d00b5100 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fd6d0072fa0 tx=0x7fd6b802b870 comp rx=0 tx=0).stop 2024-04-20T11:29:04.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.556+0000 7fd6d6800640 1 -- 172.21.15.152:0/963017579 shutdown_connections 2024-04-20T11:29:04.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.556+0000 7fd6d6800640 1 --2- 172.21.15.152:0/963017579 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd6d006f340 0x7fd6d006f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:04.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.556+0000 7fd6d6800640 1 --2- 172.21.15.152:0/963017579 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd6d00b4c50 0x7fd6d00b5100 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:04.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.556+0000 7fd6d6800640 1 --2- 172.21.15.152:0/963017579 >> v2:172.21.15.175:3300/0 conn(0x7fd6d015b5e0 0x7fd6d006eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:04.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.556+0000 7fd6d6800640 1 -- 172.21.15.152:0/963017579 >> 172.21.15.152:0/963017579 conn(0x7fd6d0067cf0 msgr2=0x7fd6d006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:04.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.556+0000 7fd6d6800640 1 -- 172.21.15.152:0/963017579 shutdown_connections 2024-04-20T11:29:04.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.556+0000 7fd6d6800640 1 -- 172.21.15.152:0/963017579 wait complete. 2024-04-20T11:29:04.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.558+0000 7fd6d6800640 1 Processor -- start 2024-04-20T11:29:04.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.558+0000 7fd6d6800640 1 -- start start 2024-04-20T11:29:04.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.559+0000 7fd6d6800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd6d006f340 0x7fd6d01466b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:04.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.559+0000 7fd6d6800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd6d00b4c50 0x7fd6d00aec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:04.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.559+0000 7fd6d6800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd6d015b5e0 0x7fd6d00a9660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:04.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.559+0000 7fd6d6800640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fd6d00b42d0 con 0x7fd6d006f340 2024-04-20T11:29:04.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.559+0000 7fd6d6800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd6d006e620 con 0x7fd6d00b4c50 2024-04-20T11:29:04.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.559+0000 7fd6d6800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd6d015cd70 con 0x7fd6d015b5e0 2024-04-20T11:29:04.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.559+0000 7fd6cf400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd6d006f340 0x7fd6d01466b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:04.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.560+0000 7fd6cf400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd6d006f340 0x7fd6d01466b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:45244/0 (socket says 172.21.15.152:45244) 2024-04-20T11:29:04.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.560+0000 7fd6cf400640 1 -- 172.21.15.152:0/605051927 learned_addr learned my addr 172.21.15.152:0/605051927 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:29:04.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.560+0000 7fd6c7e00640 1 --2- 172.21.15.152:0/605051927 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd6d00b4c50 0x7fd6d00aec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:04.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.560+0000 7fd6cfe00640 1 --2- 172.21.15.152:0/605051927 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd6d015b5e0 0x7fd6d00a9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:04.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.560+0000 7fd6cf400640 1 -- 172.21.15.152:0/605051927 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd6d015b5e0 msgr2=0x7fd6d00a9660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:04.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.560+0000 7fd6cf400640 1 --2- 172.21.15.152:0/605051927 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd6d015b5e0 0x7fd6d00a9660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:04.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.560+0000 7fd6cf400640 1 -- 172.21.15.152:0/605051927 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd6d00b4c50 msgr2=0x7fd6d00aec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:04.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.560+0000 7fd6cf400640 1 --2- 172.21.15.152:0/605051927 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd6d00b4c50 0x7fd6d00aec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:04.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.560+0000 7fd6cf400640 1 -- 172.21.15.152:0/605051927 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6d0146bf0 con 0x7fd6d006f340 2024-04-20T11:29:04.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.560+0000 7fd6cf400640 1 --2- 172.21.15.152:0/605051927 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd6d006f340 0x7fd6d01466b0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fd6c000c7f0 tx=0x7fd6c000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:04.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.561+0000 7fd6c7400640 1 -- 172.21.15.152:0/605051927 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6c0015070 con 0x7fd6d006f340 2024-04-20T11:29:04.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.561+0000 7fd6c7400640 1 -- 172.21.15.152:0/605051927 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6c0007820 con 0x7fd6d006f340 2024-04-20T11:29:04.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.561+0000 7fd6c7400640 1 -- 172.21.15.152:0/605051927 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6c0007bd0 con 0x7fd6d006f340 2024-04-20T11:29:04.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.561+0000 7fd6d6800640 1 -- 172.21.15.152:0/605051927 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6d01f3e60 con 0x7fd6d006f340 2024-04-20T11:29:04.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.561+0000 7fd6d6800640 1 -- 172.21.15.152:0/605051927 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6d01f4420 con 0x7fd6d006f340 2024-04-20T11:29:04.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.562+0000 7fd6d6800640 1 -- 172.21.15.152:0/605051927 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6d00cdd20 con 0x7fd6d006f340 2024-04-20T11:29:04.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.563+0000 7fd6c7400640 1 -- 172.21.15.152:0/605051927 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7fd6c0018070 con 0x7fd6d006f340 2024-04-20T11:29:04.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.564+0000 7fd6c7400640 1 --2- 172.21.15.152:0/605051927 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fd6ac06caa0 0x7fd6ac06ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:04.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.564+0000 7fd6c7400640 1 -- 172.21.15.152:0/605051927 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7fd6c008c090 con 0x7fd6d006f340 2024-04-20T11:29:04.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.564+0000 7fd6c7e00640 1 --2- 172.21.15.152:0/605051927 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fd6ac06caa0 0x7fd6ac06ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:04.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.565+0000 7fd6c7e00640 1 --2- 172.21.15.152:0/605051927 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fd6ac06caa0 0x7fd6ac06ef60 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fd6b8036000 tx=0x7fd6b803d040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:04.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.568+0000 7fd6c7400640 1 -- 172.21.15.152:0/605051927 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fd6d00cdd20 con 0x7fd6d006f340 2024-04-20T11:29:04.752 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.753+0000 7fd6d6800640 1 -- 172.21.15.152:0/605051927 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd6d00689f0 con 0x7fd6d006f340 2024-04-20T11:29:04.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.756+0000 7fd6c7400640 1 -- 172.21.15.152:0/605051927 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) ==== 74+0+16673 (secure 0 0 0) 0x7fd6c005a130 con 0x7fd6d006f340 2024-04-20T11:29:04.756 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:29:04.756 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":46,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:29:01.812130+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:28:58.007718+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-20T11:28:59.260647+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:29:04.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.758+0000 7fd6d6800640 1 -- 172.21.15.152:0/605051927 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fd6ac06caa0 msgr2=0x7fd6ac06ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:04.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.758+0000 7fd6d6800640 1 --2- 172.21.15.152:0/605051927 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fd6ac06caa0 0x7fd6ac06ef60 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fd6b8036000 tx=0x7fd6b803d040 comp rx=0 tx=0).stop 2024-04-20T11:29:04.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.758+0000 7fd6d6800640 1 -- 172.21.15.152:0/605051927 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd6d006f340 msgr2=0x7fd6d01466b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:04.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.758+0000 7fd6d6800640 1 --2- 172.21.15.152:0/605051927 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd6d006f340 0x7fd6d01466b0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fd6c000c7f0 tx=0x7fd6c000ccb0 comp rx=0 tx=0).stop 2024-04-20T11:29:04.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.759+0000 7fd6d6800640 1 -- 172.21.15.152:0/605051927 shutdown_connections 2024-04-20T11:29:04.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.759+0000 7fd6d6800640 1 --2- 172.21.15.152:0/605051927 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fd6ac06caa0 0x7fd6ac06ef60 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:04.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.759+0000 7fd6d6800640 1 --2- 172.21.15.152:0/605051927 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd6d015b5e0 0x7fd6d00a9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:04.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.759+0000 7fd6d6800640 1 --2- 172.21.15.152:0/605051927 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd6d00b4c50 0x7fd6d00aec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:04.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.759+0000 7fd6d6800640 1 --2- 172.21.15.152:0/605051927 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd6d006f340 0x7fd6d01466b0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:04.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.759+0000 7fd6d6800640 1 -- 172.21.15.152:0/605051927 >> 172.21.15.152:0/605051927 conn(0x7fd6d0067cf0 msgr2=0x7fd6d0071db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:04.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.759+0000 7fd6d6800640 1 -- 172.21.15.152:0/605051927 shutdown_connections 2024-04-20T11:29:04.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.759+0000 7fd6d6800640 1 -- 172.21.15.152:0/605051927 wait complete. 2024-04-20T11:29:04.775 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-20T11:29:04.775 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T11:29:04.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.961+0000 7f9eafe00640 1 Processor -- start 2024-04-20T11:29:04.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.961+0000 7f9eafe00640 1 -- start start 2024-04-20T11:29:04.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.962+0000 7f9eafe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9eb01603d0 0x7f9eb00b5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:04.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.962+0000 7f9eafe00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9eb00d3350 con 0x7f9eb00b5480 2024-04-20T11:29:04.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.962+0000 7f9eafe00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9eb00d3630 con 0x7f9eb015fcc0 2024-04-20T11:29:04.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.962+0000 7f9eafe00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f9eb00d3910 con 0x7f9eb01603d0 2024-04-20T11:29:04.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.963+0000 7f9eaf400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f9eb00b5480 0x7f9eb00cf240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:42850/0 (socket says 172.21.15.152:42850) 2024-04-20T11:29:04.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.963+0000 7f9eaf400640 1 -- 172.21.15.152:0/2969740717 learned_addr learned my addr 172.21.15.152:0/2969740717 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:29:04.964 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.963+0000 7f9eae000640 1 --2- 172.21.15.152:0/2969740717 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9eb01603d0 0x7f9eb00b5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:04.964 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.963+0000 7f9ead600640 1 -- 172.21.15.152:0/2969740717 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3637136823 0 0) 0x7f9eb00d3350 con 0x7f9eb00b5480 2024-04-20T11:29:04.964 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.963+0000 7f9ead600640 1 -- 172.21.15.152:0/2969740717 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9e94003520 con 0x7f9eb00b5480 2024-04-20T11:29:04.965 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.964+0000 7f9ead600640 1 -- 172.21.15.152:0/2969740717 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1049292221 0 0) 0x7f9e94003520 con 0x7f9eb00b5480 2024-04-20T11:29:04.965 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.964+0000 7f9ead600640 1 -- 172.21.15.152:0/2969740717 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9eb01603d0 msgr2=0x7f9eb00b5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:04.965 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.964+0000 7f9ead600640 1 --2- 172.21.15.152:0/2969740717 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9eb01603d0 0x7f9eb00b5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:04.965 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.964+0000 7f9ead600640 1 -- 172.21.15.152:0/2969740717 >> v1:172.21.15.175:6789/0 conn(0x7f9eb015fcc0 legacy=0x7f9eb00d4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:04.965 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.964+0000 7f9ead600640 1 -- 172.21.15.152:0/2969740717 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9eb00ced20 con 0x7f9eb00b5480 2024-04-20T11:29:04.965 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.964+0000 7f9ead600640 1 -- 172.21.15.152:0/2969740717 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f9ea80019f0 con 0x7f9eb00b5480 2024-04-20T11:29:04.965 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.964+0000 7f9ead600640 1 -- 172.21.15.152:0/2969740717 >> v1:172.21.15.152:6789/0 conn(0x7f9eb00b5480 legacy=0x7f9eb00cf240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:04.965 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.964+0000 7f9ead600640 1 --2- 172.21.15.152:0/2969740717 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e940039e0 0x7f9e94003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:04.965 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.964+0000 7f9ead600640 1 --2- 172.21.15.152:0/2969740717 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9e94004800 0x7f9e94004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:04.965 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.964+0000 7f9ead600640 1 --2- 172.21.15.152:0/2969740717 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9eb01603d0 0x7f9e940051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:04.965 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.964+0000 7f9ead600640 1 -- 172.21.15.152:0/2969740717 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb00d3910 con 0x7f9e94004800 2024-04-20T11:29:04.965 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.964+0000 7f9ead600640 1 -- 172.21.15.152:0/2969740717 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9e9400baa0 con 0x7f9e940039e0 2024-04-20T11:29:04.965 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.964+0000 7f9ead600640 1 -- 172.21.15.152:0/2969740717 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f9e9400bca0 con 0x7f9eb01603d0 2024-04-20T11:29:04.965 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.964+0000 7f9eae000640 1 --2- 172.21.15.152:0/2969740717 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9e94004800 0x7f9e94004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:04.966 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.964+0000 7f9eae000640 1 -- 172.21.15.152:0/2969740717 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9eb01603d0 msgr2=0x7f9e940051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:04.966 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.964+0000 7f9eae000640 1 --2- 172.21.15.152:0/2969740717 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9eb01603d0 0x7f9e940051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:04.966 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.965+0000 7f9eae000640 1 -- 172.21.15.152:0/2969740717 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e940039e0 msgr2=0x7f9e94003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:04.966 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.965+0000 7f9eae000640 1 --2- 172.21.15.152:0/2969740717 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e940039e0 0x7f9e94003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:04.966 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.965+0000 7f9eae000640 1 -- 172.21.15.152:0/2969740717 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e94003520 con 0x7f9e94004800 2024-04-20T11:29:04.967 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.966+0000 7f9eae000640 1 --2- 172.21.15.152:0/2969740717 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9e94004800 0x7f9e94004c80 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f9eb01607f0 tx=0x7f9e94005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f552b2783b507e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:04.967 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.966+0000 7f9ead600640 1 -- 172.21.15.152:0/2969740717 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c009680 con 0x7f9e94004800 2024-04-20T11:29:04.967 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.966+0000 7f9ead600640 1 -- 172.21.15.152:0/2969740717 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e9c005990 con 0x7f9e94004800 2024-04-20T11:29:04.967 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.966+0000 7f9ead600640 1 -- 172.21.15.152:0/2969740717 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c005d80 con 0x7f9e94004800 2024-04-20T11:29:04.967 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.966+0000 7f9eafe00640 1 -- 172.21.15.152:0/2969740717 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9e94004800 msgr2=0x7f9e94004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:04.967 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.966+0000 7f9eafe00640 1 --2- 172.21.15.152:0/2969740717 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9e94004800 0x7f9e94004c80 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f9eb01607f0 tx=0x7f9e94005830 comp rx=0 tx=0).stop 2024-04-20T11:29:04.967 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.966+0000 7f9eaf400640 1 -- 172.21.15.152:0/2969740717 reap_dead start 2024-04-20T11:29:04.967 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.966+0000 7f9eafe00640 1 -- 172.21.15.152:0/2969740717 shutdown_connections 2024-04-20T11:29:04.968 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.966+0000 7f9eafe00640 1 -- 172.21.15.152:0/2969740717 >> 172.21.15.152:0/2969740717 conn(0x7f9eb008bfd0 msgr2=0x7f9eb00d4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:04.968 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.967+0000 7f9eafe00640 1 -- 172.21.15.152:0/2969740717 shutdown_connections 2024-04-20T11:29:04.968 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.967+0000 7f9eafe00640 1 -- 172.21.15.152:0/2969740717 wait complete. 2024-04-20T11:29:04.968 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.968+0000 7f9eafe00640 1 Processor -- start 2024-04-20T11:29:04.968 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.968+0000 7f9eafe00640 1 -- start start 2024-04-20T11:29:04.970 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.969+0000 7f9eafe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e94004800 0x7f9eb00cacd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:04.970 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.969+0000 7f9eafe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9eb0158550 0x7f9eb00b9570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:04.970 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.969+0000 7f9eafe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9eb00b9ab0 0x7f9eb006cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:04.970 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.969+0000 7f9eafe00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb00cfd20 con 0x7f9eb00b9ab0 2024-04-20T11:29:04.970 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.969+0000 7f9eafe00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb00cfba0 con 0x7f9e94004800 2024-04-20T11:29:04.970 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.969+0000 7f9eafe00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f9eb00cfea0 con 0x7f9eb0158550 2024-04-20T11:29:04.970 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.969+0000 7f9eaf400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9eb00b9ab0 0x7f9eb006cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:04.970 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.970+0000 7f9eaf400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9eb00b9ab0 0x7f9eb006cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:45252/0 (socket says 172.21.15.152:45252) 2024-04-20T11:29:04.971 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.970+0000 7f9eaf400640 1 -- 172.21.15.152:0/2083948543 learned_addr learned my addr 172.21.15.152:0/2083948543 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:29:04.971 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.970+0000 7f9eae000640 1 --2- 172.21.15.152:0/2083948543 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9eb0158550 0x7f9eb00b9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:04.971 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.970+0000 7f9eaf400640 1 -- 172.21.15.152:0/2083948543 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9eb0158550 msgr2=0x7f9eb00b9570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:04.971 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.970+0000 7f9eaf400640 1 --2- 172.21.15.152:0/2083948543 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9eb0158550 0x7f9eb00b9570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:04.971 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.970+0000 7f9eaf400640 1 -- 172.21.15.152:0/2083948543 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e94004800 msgr2=0x7f9eb00cacd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:04.971 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.970+0000 7f9eaea00640 1 --2- 172.21.15.152:0/2083948543 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e94004800 0x7f9eb00cacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:04.971 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.970+0000 7f9eaf400640 1 --2- 172.21.15.152:0/2083948543 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e94004800 0x7f9eb00cacd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:04.971 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.970+0000 7f9eaf400640 1 -- 172.21.15.152:0/2083948543 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9eb00b84d0 con 0x7f9eb00b9ab0 2024-04-20T11:29:04.971 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.970+0000 7f9eaf400640 1 --2- 172.21.15.152:0/2083948543 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9eb00b9ab0 0x7f9eb006cbb0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f9eb00ceb30 tx=0x7f9ea800ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:04.971 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.970+0000 7f9ea6a00640 1 -- 172.21.15.152:0/2083948543 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea800a5e0 con 0x7f9eb00b9ab0 2024-04-20T11:29:04.971 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.971+0000 7f9ea6a00640 1 -- 172.21.15.152:0/2083948543 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ea800a7a0 con 0x7f9eb00b9ab0 2024-04-20T11:29:04.971 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.971+0000 7f9eafe00640 1 -- 172.21.15.152:0/2083948543 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9eb00b87b0 con 0x7f9eb00b9ab0 2024-04-20T11:29:04.971 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.971+0000 7f9ea6a00640 1 -- 172.21.15.152:0/2083948543 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea800ab90 con 0x7f9eb00b9ab0 2024-04-20T11:29:04.971 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.971+0000 7f9eafe00640 1 -- 172.21.15.152:0/2083948543 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9eb00b8d70 con 0x7f9eb00b9ab0 2024-04-20T11:29:04.972 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.972+0000 7f9eafe00640 1 -- 172.21.15.152:0/2083948543 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9eb0075e60 con 0x7f9eb00b9ab0 2024-04-20T11:29:04.972 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.973+0000 7f9ea6a00640 1 -- 172.21.15.152:0/2083948543 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f9ea800ad30 con 0x7f9eb00b9ab0 2024-04-20T11:29:04.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.974+0000 7f9ea6a00640 1 --2- 172.21.15.152:0/2083948543 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f9e9006caa0 0x7f9e9006ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:04.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.974+0000 7f9ea6a00640 1 -- 172.21.15.152:0/2083948543 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f9ea808b8c0 con 0x7f9eb00b9ab0 2024-04-20T11:29:04.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.974+0000 7f9eaea00640 1 --2- 172.21.15.152:0/2083948543 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f9e9006caa0 0x7f9e9006ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:04.974 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.975+0000 7f9eaea00640 1 --2- 172.21.15.152:0/2083948543 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f9e9006caa0 0x7f9e9006ef60 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f9ea00015d0 tx=0x7f9ea0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:04.977 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:04.978+0000 7f9ea6a00640 1 -- 172.21.15.152:0/2083948543 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f9ea8059a60 con 0x7f9eb00b9ab0 2024-04-20T11:29:05.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.174+0000 7f9eafe00640 1 -- 172.21.15.152:0/2083948543 --> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9eb00d5290 con 0x7f9e9006caa0 2024-04-20T11:29:05.184 INFO:tasks.ceph.mds.d.smithi175.stderr:2024-04-20T11:29:05.184+0000 7f1659a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-20T11:29:05.193 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.193+0000 7f9ea6a00640 1 -- 172.21.15.152:0/2083948543 <== mgr.4113 v2:172.21.15.152:6832/2285110240 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251920 (secure 0 0 0) 0x7f9eb00d5290 con 0x7f9e9006caa0 2024-04-20T11:29:05.197 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:29:05.198 INFO:teuthology.orchestra.run.smithi152.stderr:dumped all 2024-04-20T11:29:05.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.199+0000 7f9eafe00640 1 -- 172.21.15.152:0/2083948543 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f9e9006caa0 msgr2=0x7f9e9006ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:05.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.199+0000 7f9eafe00640 1 --2- 172.21.15.152:0/2083948543 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f9e9006caa0 0x7f9e9006ef60 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f9ea00015d0 tx=0x7f9ea0009040 comp rx=0 tx=0).stop 2024-04-20T11:29:05.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.199+0000 7f9eafe00640 1 -- 172.21.15.152:0/2083948543 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9eb00b9ab0 msgr2=0x7f9eb006cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:05.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.199+0000 7f9eafe00640 1 --2- 172.21.15.152:0/2083948543 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9eb00b9ab0 0x7f9eb006cbb0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f9eb00ceb30 tx=0x7f9ea800ccd0 comp rx=0 tx=0).stop 2024-04-20T11:29:05.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.199+0000 7f9eafe00640 1 -- 172.21.15.152:0/2083948543 shutdown_connections 2024-04-20T11:29:05.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.199+0000 7f9eafe00640 1 --2- 172.21.15.152:0/2083948543 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f9e9006caa0 0x7f9e9006ef60 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:05.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.199+0000 7f9eafe00640 1 --2- 172.21.15.152:0/2083948543 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9eb00b9ab0 0x7f9eb006cbb0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:05.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.199+0000 7f9eafe00640 1 --2- 172.21.15.152:0/2083948543 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9eb0158550 0x7f9eb00b9570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:05.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.199+0000 7f9eafe00640 1 --2- 172.21.15.152:0/2083948543 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e94004800 0x7f9eb00cacd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:05.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.199+0000 7f9eafe00640 1 -- 172.21.15.152:0/2083948543 >> 172.21.15.152:0/2083948543 conn(0x7f9eb008bfd0 msgr2=0x7f9eb00d44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:05.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.199+0000 7f9eafe00640 1 -- 172.21.15.152:0/2083948543 shutdown_connections 2024-04-20T11:29:05.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.200+0000 7f9eafe00640 1 -- 172.21.15.152:0/2083948543 wait complete. 2024-04-20T11:29:05.217 INFO:teuthology.orchestra.run.smithi152.stdout:{"pg_ready":true,"pg_map":{"version":107,"stamp":"2024-04-20T11:29:03.251843+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":47,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":221552,"kb_used_data":5376,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753168,"statfs":{"total":773094113280,"available":772867244032,"internally_reserved":0,"allocated":5505024,"data_stored":2916940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12746,"internal_metadata":221236790},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.454134"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T11:29:00.859574+0000","last_change":"2024-04-20T11:29:00.852985+0000","last_active":"2024-04-20T11:29:00.859574+0000","last_peered":"2024-04-20T11:29:00.859574+0000","last_clean":"2024-04-20T11:29:00.859574+0000","last_became_active":"2024-04-20T11:29:00.852677+0000","last_became_peered":"2024-04-20T11:29:00.852677+0000","last_unstale":"2024-04-20T11:29:00.859574+0000","last_undegraded":"2024-04-20T11:29:00.859574+0000","last_fullsized":"2024-04-20T11:29:00.859574+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:57:55.715567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-20T11:29:01.823442+0000","last_change":"2024-04-20T11:29:00.835034+0000","last_active":"2024-04-20T11:29:01.823442+0000","last_peered":"2024-04-20T11:29:01.823442+0000","last_clean":"2024-04-20T11:29:01.823442+0000","last_became_active":"2024-04-20T11:29:00.834736+0000","last_became_peered":"2024-04-20T11:29:00.834736+0000","last_unstale":"2024-04-20T11:29:01.823442+0000","last_undegraded":"2024-04-20T11:29:01.823442+0000","last_fullsized":"2024-04-20T11:29:01.823442+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:57:55.715567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-20T11:29:01.825279+0000","last_change":"2024-04-20T11:29:00.863374+0000","last_active":"2024-04-20T11:29:01.825279+0000","last_peered":"2024-04-20T11:29:01.825279+0000","last_clean":"2024-04-20T11:29:01.825279+0000","last_became_active":"2024-04-20T11:29:00.863070+0000","last_became_peered":"2024-04-20T11:29:00.863070+0000","last_unstale":"2024-04-20T11:29:01.825279+0000","last_undegraded":"2024-04-20T11:29:01.825279+0000","last_fullsized":"2024-04-20T11:29:01.825279+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:22:59.242819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T11:29:00.851044+0000","last_change":"2024-04-20T11:29:00.846863+0000","last_active":"2024-04-20T11:29:00.851044+0000","last_peered":"2024-04-20T11:29:00.851044+0000","last_clean":"2024-04-20T11:29:00.851044+0000","last_became_active":"2024-04-20T11:29:00.845911+0000","last_became_peered":"2024-04-20T11:29:00.845911+0000","last_unstale":"2024-04-20T11:29:00.851044+0000","last_undegraded":"2024-04-20T11:29:00.851044+0000","last_fullsized":"2024-04-20T11:29:00.851044+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:17:30.033031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-20T11:29:01.822679+0000","last_change":"2024-04-20T11:29:00.836049+0000","last_active":"2024-04-20T11:29:01.822679+0000","last_peered":"2024-04-20T11:29:01.822679+0000","last_clean":"2024-04-20T11:29:01.822679+0000","last_became_active":"2024-04-20T11:29:00.833697+0000","last_became_peered":"2024-04-20T11:29:00.833697+0000","last_unstale":"2024-04-20T11:29:01.822679+0000","last_undegraded":"2024-04-20T11:29:01.822679+0000","last_fullsized":"2024-04-20T11:29:01.822679+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:59:51.540993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T11:29:00.837842+0000","last_change":"2024-04-20T11:29:00.830759+0000","last_active":"2024-04-20T11:29:00.837842+0000","last_peered":"2024-04-20T11:29:00.837842+0000","last_clean":"2024-04-20T11:29:00.837842+0000","last_became_active":"2024-04-20T11:29:00.830600+0000","last_became_peered":"2024-04-20T11:29:00.830600+0000","last_unstale":"2024-04-20T11:29:00.837842+0000","last_undegraded":"2024-04-20T11:29:00.837842+0000","last_fullsized":"2024-04-20T11:29:00.837842+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:49:18.547172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T11:29:00.850901+0000","last_change":"2024-04-20T11:29:00.838802+0000","last_active":"2024-04-20T11:29:00.850901+0000","last_peered":"2024-04-20T11:29:00.850901+0000","last_clean":"2024-04-20T11:29:00.850901+0000","last_became_active":"2024-04-20T11:29:00.838530+0000","last_became_peered":"2024-04-20T11:29:00.838530+0000","last_unstale":"2024-04-20T11:29:00.850901+0000","last_undegraded":"2024-04-20T11:29:00.850901+0000","last_fullsized":"2024-04-20T11:29:00.850901+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:38:53.689879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T11:29:00.845300+0000","last_change":"2024-04-20T11:29:00.834424+0000","last_active":"2024-04-20T11:29:00.845300+0000","last_peered":"2024-04-20T11:29:00.845300+0000","last_clean":"2024-04-20T11:29:00.845300+0000","last_became_active":"2024-04-20T11:29:00.834082+0000","last_became_peered":"2024-04-20T11:29:00.834082+0000","last_unstale":"2024-04-20T11:29:00.845300+0000","last_undegraded":"2024-04-20T11:29:00.845300+0000","last_fullsized":"2024-04-20T11:29:00.845300+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:20:11.399706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T11:29:00.850604+0000","last_change":"2024-04-20T11:29:00.845480+0000","last_active":"2024-04-20T11:29:00.850604+0000","last_peered":"2024-04-20T11:29:00.850604+0000","last_clean":"2024-04-20T11:29:00.850604+0000","last_became_active":"2024-04-20T11:29:00.845159+0000","last_became_peered":"2024-04-20T11:29:00.845159+0000","last_unstale":"2024-04-20T11:29:00.850604+0000","last_undegraded":"2024-04-20T11:29:00.850604+0000","last_fullsized":"2024-04-20T11:29:00.850604+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:26:31.816201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T11:29:00.850747+0000","last_change":"2024-04-20T11:29:00.845831+0000","last_active":"2024-04-20T11:29:00.850747+0000","last_peered":"2024-04-20T11:29:00.850747+0000","last_clean":"2024-04-20T11:29:00.850747+0000","last_became_active":"2024-04-20T11:29:00.845168+0000","last_became_peered":"2024-04-20T11:29:00.845168+0000","last_unstale":"2024-04-20T11:29:00.850747+0000","last_undegraded":"2024-04-20T11:29:00.850747+0000","last_fullsized":"2024-04-20T11:29:00.850747+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:41:05.632073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T11:29:00.849139+0000","last_change":"2024-04-20T11:29:00.844286+0000","last_active":"2024-04-20T11:29:00.849139+0000","last_peered":"2024-04-20T11:29:00.849139+0000","last_clean":"2024-04-20T11:29:00.849139+0000","last_became_active":"2024-04-20T11:29:00.844024+0000","last_became_peered":"2024-04-20T11:29:00.844024+0000","last_unstale":"2024-04-20T11:29:00.849139+0000","last_undegraded":"2024-04-20T11:29:00.849139+0000","last_fullsized":"2024-04-20T11:29:00.849139+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:05:12.507611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-20T11:29:01.824063+0000","last_change":"2024-04-20T11:29:00.861729+0000","last_active":"2024-04-20T11:29:01.824063+0000","last_peered":"2024-04-20T11:29:01.824063+0000","last_clean":"2024-04-20T11:29:01.824063+0000","last_became_active":"2024-04-20T11:29:00.861432+0000","last_became_peered":"2024-04-20T11:29:00.861432+0000","last_unstale":"2024-04-20T11:29:01.824063+0000","last_undegraded":"2024-04-20T11:29:01.824063+0000","last_fullsized":"2024-04-20T11:29:01.824063+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:44:23.956892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-20T11:29:01.821547+0000","last_change":"2024-04-20T11:29:00.860956+0000","last_active":"2024-04-20T11:29:01.821547+0000","last_peered":"2024-04-20T11:29:01.821547+0000","last_clean":"2024-04-20T11:29:01.821547+0000","last_became_active":"2024-04-20T11:29:00.860684+0000","last_became_peered":"2024-04-20T11:29:00.860684+0000","last_unstale":"2024-04-20T11:29:01.821547+0000","last_undegraded":"2024-04-20T11:29:01.821547+0000","last_fullsized":"2024-04-20T11:29:01.821547+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:44:13.751283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T11:29:00.847059+0000","last_change":"2024-04-20T11:29:00.837370+0000","last_active":"2024-04-20T11:29:00.847059+0000","last_peered":"2024-04-20T11:29:00.847059+0000","last_clean":"2024-04-20T11:29:00.847059+0000","last_became_active":"2024-04-20T11:29:00.837105+0000","last_became_peered":"2024-04-20T11:29:00.837105+0000","last_unstale":"2024-04-20T11:29:00.847059+0000","last_undegraded":"2024-04-20T11:29:00.847059+0000","last_fullsized":"2024-04-20T11:29:00.847059+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:32:57.103501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T11:29:00.859421+0000","last_change":"2024-04-20T11:29:00.852622+0000","last_active":"2024-04-20T11:29:00.859421+0000","last_peered":"2024-04-20T11:29:00.859421+0000","last_clean":"2024-04-20T11:29:00.859421+0000","last_became_active":"2024-04-20T11:29:00.852322+0000","last_became_peered":"2024-04-20T11:29:00.852322+0000","last_unstale":"2024-04-20T11:29:00.859421+0000","last_undegraded":"2024-04-20T11:29:00.859421+0000","last_fullsized":"2024-04-20T11:29:00.859421+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:23:39.579200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-20T11:29:02.173172+0000","last_change":"2024-04-20T11:29:00.848889+0000","last_active":"2024-04-20T11:29:02.173172+0000","last_peered":"2024-04-20T11:29:02.173172+0000","last_clean":"2024-04-20T11:29:02.173172+0000","last_became_active":"2024-04-20T11:29:00.848614+0000","last_became_peered":"2024-04-20T11:29:00.848614+0000","last_unstale":"2024-04-20T11:29:02.173172+0000","last_undegraded":"2024-04-20T11:29:02.173172+0000","last_fullsized":"2024-04-20T11:29:02.173172+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:10:01.731112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-20T11:29:01.824886+0000","last_change":"2024-04-20T11:29:00.850950+0000","last_active":"2024-04-20T11:29:01.824886+0000","last_peered":"2024-04-20T11:29:01.824886+0000","last_clean":"2024-04-20T11:29:01.824886+0000","last_became_active":"2024-04-20T11:29:00.850638+0000","last_became_peered":"2024-04-20T11:29:00.850638+0000","last_unstale":"2024-04-20T11:29:01.824886+0000","last_undegraded":"2024-04-20T11:29:01.824886+0000","last_fullsized":"2024-04-20T11:29:01.824886+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:57:55.715567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-20T11:29:01.823495+0000","last_change":"2024-04-20T11:29:00.853642+0000","last_active":"2024-04-20T11:29:01.823495+0000","last_peered":"2024-04-20T11:29:01.823495+0000","last_clean":"2024-04-20T11:29:01.823495+0000","last_became_active":"2024-04-20T11:29:00.853084+0000","last_became_peered":"2024-04-20T11:29:00.853084+0000","last_unstale":"2024-04-20T11:29:01.823495+0000","last_undegraded":"2024-04-20T11:29:01.823495+0000","last_fullsized":"2024-04-20T11:29:01.823495+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:38:53.689879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-20T11:29:02.174069+0000","last_change":"2024-04-20T11:29:00.839761+0000","last_active":"2024-04-20T11:29:02.174069+0000","last_peered":"2024-04-20T11:29:02.174069+0000","last_clean":"2024-04-20T11:29:02.174069+0000","last_became_active":"2024-04-20T11:29:00.839145+0000","last_became_peered":"2024-04-20T11:29:00.839145+0000","last_unstale":"2024-04-20T11:29:02.174069+0000","last_undegraded":"2024-04-20T11:29:02.174069+0000","last_fullsized":"2024-04-20T11:29:02.174069+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:03:36.963877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-20T11:29:01.821651+0000","last_change":"2024-04-20T11:29:00.859987+0000","last_active":"2024-04-20T11:29:01.821651+0000","last_peered":"2024-04-20T11:29:01.821651+0000","last_clean":"2024-04-20T11:29:01.821651+0000","last_became_active":"2024-04-20T11:29:00.859721+0000","last_became_peered":"2024-04-20T11:29:00.859721+0000","last_unstale":"2024-04-20T11:29:01.821651+0000","last_undegraded":"2024-04-20T11:29:01.821651+0000","last_fullsized":"2024-04-20T11:29:01.821651+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:20:11.399706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T11:29:00.847174+0000","last_change":"2024-04-20T11:29:00.841913+0000","last_active":"2024-04-20T11:29:00.847174+0000","last_peered":"2024-04-20T11:29:00.847174+0000","last_clean":"2024-04-20T11:29:00.847174+0000","last_became_active":"2024-04-20T11:29:00.841459+0000","last_became_peered":"2024-04-20T11:29:00.841459+0000","last_unstale":"2024-04-20T11:29:00.847174+0000","last_undegraded":"2024-04-20T11:29:00.847174+0000","last_fullsized":"2024-04-20T11:29:00.847174+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:34:13.826288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T11:29:00.844490+0000","last_change":"2024-04-20T11:29:00.833387+0000","last_active":"2024-04-20T11:29:00.844490+0000","last_peered":"2024-04-20T11:29:00.844490+0000","last_clean":"2024-04-20T11:29:00.844490+0000","last_became_active":"2024-04-20T11:29:00.832985+0000","last_became_peered":"2024-04-20T11:29:00.832985+0000","last_unstale":"2024-04-20T11:29:00.844490+0000","last_undegraded":"2024-04-20T11:29:00.844490+0000","last_fullsized":"2024-04-20T11:29:00.844490+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:44:13.751283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-20T11:29:02.173420+0000","last_change":"2024-04-20T11:29:00.844262+0000","last_active":"2024-04-20T11:29:02.173420+0000","last_peered":"2024-04-20T11:29:02.173420+0000","last_clean":"2024-04-20T11:29:02.173420+0000","last_became_active":"2024-04-20T11:29:00.843925+0000","last_became_peered":"2024-04-20T11:29:00.843925+0000","last_unstale":"2024-04-20T11:29:02.173420+0000","last_undegraded":"2024-04-20T11:29:02.173420+0000","last_fullsized":"2024-04-20T11:29:02.173420+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:05:12.507611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-20T11:29:02.173262+0000","last_change":"2024-04-20T11:29:00.832131+0000","last_active":"2024-04-20T11:29:02.173262+0000","last_peered":"2024-04-20T11:29:02.173262+0000","last_clean":"2024-04-20T11:29:02.173262+0000","last_became_active":"2024-04-20T11:29:00.831796+0000","last_became_peered":"2024-04-20T11:29:00.831796+0000","last_unstale":"2024-04-20T11:29:02.173262+0000","last_undegraded":"2024-04-20T11:29:02.173262+0000","last_fullsized":"2024-04-20T11:29:02.173262+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:49:14.491396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-20T11:29:02.174046+0000","last_change":"2024-04-20T11:29:00.841038+0000","last_active":"2024-04-20T11:29:02.174046+0000","last_peered":"2024-04-20T11:29:02.174046+0000","last_clean":"2024-04-20T11:29:02.174046+0000","last_became_active":"2024-04-20T11:29:00.840726+0000","last_became_peered":"2024-04-20T11:29:00.840726+0000","last_unstale":"2024-04-20T11:29:02.174046+0000","last_undegraded":"2024-04-20T11:29:02.174046+0000","last_fullsized":"2024-04-20T11:29:02.174046+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:17:30.033031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-20T11:29:01.821383+0000","last_change":"2024-04-20T11:29:00.849447+0000","last_active":"2024-04-20T11:29:01.821383+0000","last_peered":"2024-04-20T11:29:01.821383+0000","last_clean":"2024-04-20T11:29:01.821383+0000","last_became_active":"2024-04-20T11:29:00.849135+0000","last_became_peered":"2024-04-20T11:29:00.849135+0000","last_unstale":"2024-04-20T11:29:01.821383+0000","last_undegraded":"2024-04-20T11:29:01.821383+0000","last_fullsized":"2024-04-20T11:29:01.821383+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:02:38.545869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-20T11:29:01.823957+0000","last_change":"2024-04-20T11:29:00.851916+0000","last_active":"2024-04-20T11:29:01.823957+0000","last_peered":"2024-04-20T11:29:01.823957+0000","last_clean":"2024-04-20T11:29:01.823957+0000","last_became_active":"2024-04-20T11:29:00.851685+0000","last_became_peered":"2024-04-20T11:29:00.851685+0000","last_unstale":"2024-04-20T11:29:01.823957+0000","last_undegraded":"2024-04-20T11:29:01.823957+0000","last_fullsized":"2024-04-20T11:29:01.823957+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:59:51.540993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-20T11:29:01.824495+0000","last_change":"2024-04-20T11:29:00.861714+0000","last_active":"2024-04-20T11:29:01.824495+0000","last_peered":"2024-04-20T11:29:01.824495+0000","last_clean":"2024-04-20T11:29:01.824495+0000","last_became_active":"2024-04-20T11:29:00.861379+0000","last_became_peered":"2024-04-20T11:29:00.861379+0000","last_unstale":"2024-04-20T11:29:01.824495+0000","last_undegraded":"2024-04-20T11:29:01.824495+0000","last_fullsized":"2024-04-20T11:29:01.824495+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:22:59.242819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-20T11:29:02.173263+0000","last_change":"2024-04-20T11:29:00.842444+0000","last_active":"2024-04-20T11:29:02.173263+0000","last_peered":"2024-04-20T11:29:02.173263+0000","last_clean":"2024-04-20T11:29:02.173263+0000","last_became_active":"2024-04-20T11:29:00.842189+0000","last_became_peered":"2024-04-20T11:29:00.842189+0000","last_unstale":"2024-04-20T11:29:02.173263+0000","last_undegraded":"2024-04-20T11:29:02.173263+0000","last_fullsized":"2024-04-20T11:29:02.173263+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:38:21.618746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-20T11:29:01.823000+0000","last_change":"2024-04-20T11:29:00.852871+0000","last_active":"2024-04-20T11:29:01.823000+0000","last_peered":"2024-04-20T11:29:01.823000+0000","last_clean":"2024-04-20T11:29:01.823000+0000","last_became_active":"2024-04-20T11:29:00.852605+0000","last_became_peered":"2024-04-20T11:29:00.852605+0000","last_unstale":"2024-04-20T11:29:01.823000+0000","last_undegraded":"2024-04-20T11:29:01.823000+0000","last_fullsized":"2024-04-20T11:29:01.823000+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:14:41.729790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T11:29:00.834303+0000","last_change":"2024-04-20T11:29:00.828342+0000","last_active":"2024-04-20T11:29:00.834303+0000","last_peered":"2024-04-20T11:29:00.834303+0000","last_clean":"2024-04-20T11:29:00.834303+0000","last_became_active":"2024-04-20T11:29:00.828065+0000","last_became_peered":"2024-04-20T11:29:00.828065+0000","last_unstale":"2024-04-20T11:29:00.834303+0000","last_undegraded":"2024-04-20T11:29:00.834303+0000","last_fullsized":"2024-04-20T11:29:00.834303+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:26:46.408356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-20T11:29:01.820898+0000","last_change":"2024-04-20T11:29:00.851890+0000","last_active":"2024-04-20T11:29:01.820898+0000","last_peered":"2024-04-20T11:29:01.820898+0000","last_clean":"2024-04-20T11:29:01.820898+0000","last_became_active":"2024-04-20T11:29:00.851562+0000","last_became_peered":"2024-04-20T11:29:00.851562+0000","last_unstale":"2024-04-20T11:29:01.820898+0000","last_undegraded":"2024-04-20T11:29:01.820898+0000","last_fullsized":"2024-04-20T11:29:01.820898+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:53:16.444204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T11:29:00.854998+0000","last_change":"2024-04-20T11:29:00.850498+0000","last_active":"2024-04-20T11:29:00.854998+0000","last_peered":"2024-04-20T11:29:00.854998+0000","last_clean":"2024-04-20T11:29:00.854998+0000","last_became_active":"2024-04-20T11:29:00.850210+0000","last_became_peered":"2024-04-20T11:29:00.850210+0000","last_unstale":"2024-04-20T11:29:00.854998+0000","last_undegraded":"2024-04-20T11:29:00.854998+0000","last_fullsized":"2024-04-20T11:29:00.854998+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:31:39.636876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"45'4","reported_seq":16,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T11:29:00.862974+0000","last_change":"2024-04-20T11:28:59.838071+0000","last_active":"2024-04-20T11:29:00.862974+0000","last_peered":"2024-04-20T11:29:00.862974+0000","last_clean":"2024-04-20T11:29:00.862974+0000","last_became_active":"2024-04-20T11:28:59.836554+0000","last_became_peered":"2024-04-20T11:28:59.836554+0000","last_unstale":"2024-04-20T11:29:00.862974+0000","last_undegraded":"2024-04-20T11:29:00.862974+0000","last_fullsized":"2024-04-20T11:29:00.862974+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_clean_scrub_stamp":"2024-04-20T11:28:58.778154+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:57:54.699702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-20T11:29:02.174085+0000","last_change":"2024-04-20T11:29:00.847627+0000","last_active":"2024-04-20T11:29:02.174085+0000","last_peered":"2024-04-20T11:29:02.174085+0000","last_clean":"2024-04-20T11:29:02.174085+0000","last_became_active":"2024-04-20T11:29:00.847255+0000","last_became_peered":"2024-04-20T11:29:00.847255+0000","last_unstale":"2024-04-20T11:29:02.174085+0000","last_undegraded":"2024-04-20T11:29:02.174085+0000","last_fullsized":"2024-04-20T11:29:02.174085+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:45:37.576931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"45'1","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-20T11:29:01.824413+0000","last_change":"2024-04-20T11:28:59.847780+0000","last_active":"2024-04-20T11:29:01.824413+0000","last_peered":"2024-04-20T11:29:01.824413+0000","last_clean":"2024-04-20T11:29:01.824413+0000","last_became_active":"2024-04-20T11:28:59.847094+0000","last_became_peered":"2024-04-20T11:28:59.847094+0000","last_unstale":"2024-04-20T11:29:01.824413+0000","last_undegraded":"2024-04-20T11:29:01.824413+0000","last_fullsized":"2024-04-20T11:29:01.824413+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_clean_scrub_stamp":"2024-04-20T11:28:58.778154+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:38:20.602881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-20T11:29:01.824874+0000","last_change":"2024-04-20T11:29:00.834021+0000","last_active":"2024-04-20T11:29:01.824874+0000","last_peered":"2024-04-20T11:29:01.824874+0000","last_clean":"2024-04-20T11:29:01.824874+0000","last_became_active":"2024-04-20T11:29:00.833678+0000","last_became_peered":"2024-04-20T11:29:00.833678+0000","last_unstale":"2024-04-20T11:29:01.824874+0000","last_undegraded":"2024-04-20T11:29:01.824874+0000","last_fullsized":"2024-04-20T11:29:01.824874+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:26:46.408356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"45'2","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T11:29:00.862388+0000","last_change":"2024-04-20T11:28:59.843487+0000","last_active":"2024-04-20T11:29:00.862388+0000","last_peered":"2024-04-20T11:29:00.862388+0000","last_clean":"2024-04-20T11:29:00.862388+0000","last_became_active":"2024-04-20T11:28:59.843178+0000","last_became_peered":"2024-04-20T11:28:59.843178+0000","last_unstale":"2024-04-20T11:29:00.862388+0000","last_undegraded":"2024-04-20T11:29:00.862388+0000","last_fullsized":"2024-04-20T11:29:00.862388+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_clean_scrub_stamp":"2024-04-20T11:28:58.778154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:32:56.087636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T11:29:00.856174+0000","last_change":"2024-04-20T11:29:00.847481+0000","last_active":"2024-04-20T11:29:00.856174+0000","last_peered":"2024-04-20T11:29:00.856174+0000","last_clean":"2024-04-20T11:29:00.856174+0000","last_became_active":"2024-04-20T11:29:00.846663+0000","last_became_peered":"2024-04-20T11:29:00.846663+0000","last_unstale":"2024-04-20T11:29:00.856174+0000","last_undegraded":"2024-04-20T11:29:00.856174+0000","last_fullsized":"2024-04-20T11:29:00.856174+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:49:14.491396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"45'2","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T11:29:00.844733+0000","last_change":"2024-04-20T11:28:59.849109+0000","last_active":"2024-04-20T11:29:00.844733+0000","last_peered":"2024-04-20T11:29:00.844733+0000","last_clean":"2024-04-20T11:29:00.844733+0000","last_became_active":"2024-04-20T11:28:59.848783+0000","last_became_peered":"2024-04-20T11:28:59.848783+0000","last_unstale":"2024-04-20T11:29:00.844733+0000","last_undegraded":"2024-04-20T11:29:00.844733+0000","last_fullsized":"2024-04-20T11:29:00.844733+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_clean_scrub_stamp":"2024-04-20T11:28:58.778154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:03:35.948012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-20T11:29:01.824898+0000","last_change":"2024-04-20T11:29:00.853745+0000","last_active":"2024-04-20T11:29:01.824898+0000","last_peered":"2024-04-20T11:29:01.824898+0000","last_clean":"2024-04-20T11:29:01.824898+0000","last_became_active":"2024-04-20T11:29:00.853376+0000","last_became_peered":"2024-04-20T11:29:00.853376+0000","last_unstale":"2024-04-20T11:29:01.824898+0000","last_undegraded":"2024-04-20T11:29:01.824898+0000","last_fullsized":"2024-04-20T11:29:01.824898+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:44:23.956892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"45'1","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T11:29:00.860760+0000","last_change":"2024-04-20T11:28:59.844224+0000","last_active":"2024-04-20T11:29:00.860760+0000","last_peered":"2024-04-20T11:29:00.860760+0000","last_clean":"2024-04-20T11:29:00.860760+0000","last_became_active":"2024-04-20T11:28:59.843984+0000","last_became_peered":"2024-04-20T11:28:59.843984+0000","last_unstale":"2024-04-20T11:29:00.860760+0000","last_undegraded":"2024-04-20T11:29:00.860760+0000","last_fullsized":"2024-04-20T11:29:00.860760+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_clean_scrub_stamp":"2024-04-20T11:28:58.778154+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:22:58.226954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T11:29:00.861899+0000","last_change":"2024-04-20T11:29:00.857690+0000","last_active":"2024-04-20T11:29:00.861899+0000","last_peered":"2024-04-20T11:29:00.861899+0000","last_clean":"2024-04-20T11:29:00.861899+0000","last_became_active":"2024-04-20T11:29:00.857368+0000","last_became_peered":"2024-04-20T11:29:00.857368+0000","last_unstale":"2024-04-20T11:29:00.861899+0000","last_undegraded":"2024-04-20T11:29:00.861899+0000","last_fullsized":"2024-04-20T11:29:00.861899+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:45:37.576931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"45'1","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-20T11:29:01.823300+0000","last_change":"2024-04-20T11:28:59.850482+0000","last_active":"2024-04-20T11:29:01.823300+0000","last_peered":"2024-04-20T11:29:01.823300+0000","last_clean":"2024-04-20T11:29:01.823300+0000","last_became_active":"2024-04-20T11:28:59.850188+0000","last_became_peered":"2024-04-20T11:28:59.850188+0000","last_unstale":"2024-04-20T11:29:01.823300+0000","last_undegraded":"2024-04-20T11:29:01.823300+0000","last_fullsized":"2024-04-20T11:29:01.823300+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_clean_scrub_stamp":"2024-04-20T11:28:58.778154+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:20:10.383841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T11:29:00.845253+0000","last_change":"2024-04-20T11:29:00.833875+0000","last_active":"2024-04-20T11:29:00.845253+0000","last_peered":"2024-04-20T11:29:00.845253+0000","last_clean":"2024-04-20T11:29:00.845253+0000","last_became_active":"2024-04-20T11:29:00.832895+0000","last_became_peered":"2024-04-20T11:29:00.832895+0000","last_unstale":"2024-04-20T11:29:00.845253+0000","last_undegraded":"2024-04-20T11:29:00.845253+0000","last_fullsized":"2024-04-20T11:29:00.845253+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:36:14.165535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-20T11:29:01.822001+0000","last_change":"2024-04-20T11:28:59.844437+0000","last_active":"2024-04-20T11:29:01.822001+0000","last_peered":"2024-04-20T11:29:01.822001+0000","last_clean":"2024-04-20T11:29:01.822001+0000","last_became_active":"2024-04-20T11:28:59.844118+0000","last_became_peered":"2024-04-20T11:28:59.844118+0000","last_unstale":"2024-04-20T11:29:01.822001+0000","last_undegraded":"2024-04-20T11:29:01.822001+0000","last_fullsized":"2024-04-20T11:29:01.822001+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_clean_scrub_stamp":"2024-04-20T11:28:58.778154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:05:11.491745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-20T11:29:02.173203+0000","last_change":"2024-04-20T11:29:00.838698+0000","last_active":"2024-04-20T11:29:02.173203+0000","last_peered":"2024-04-20T11:29:02.173203+0000","last_clean":"2024-04-20T11:29:02.173203+0000","last_became_active":"2024-04-20T11:29:00.838408+0000","last_became_peered":"2024-04-20T11:29:00.838408+0000","last_unstale":"2024-04-20T11:29:02.173203+0000","last_undegraded":"2024-04-20T11:29:02.173203+0000","last_fullsized":"2024-04-20T11:29:02.173203+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:32:57.103501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T11:29:00.818974+0000","last_change":"2024-04-20T11:28:59.834793+0000","last_active":"2024-04-20T11:29:00.818974+0000","last_peered":"2024-04-20T11:29:00.818974+0000","last_clean":"2024-04-20T11:29:00.818974+0000","last_became_active":"2024-04-20T11:28:59.832265+0000","last_became_peered":"2024-04-20T11:28:59.832265+0000","last_unstale":"2024-04-20T11:29:00.818974+0000","last_undegraded":"2024-04-20T11:29:00.818974+0000","last_fullsized":"2024-04-20T11:29:00.818974+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_clean_scrub_stamp":"2024-04-20T11:28:58.778154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:13:54.057269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-20T11:29:01.823948+0000","last_change":"2024-04-20T11:29:00.853396+0000","last_active":"2024-04-20T11:29:01.823948+0000","last_peered":"2024-04-20T11:29:01.823948+0000","last_clean":"2024-04-20T11:29:01.823948+0000","last_became_active":"2024-04-20T11:29:00.852731+0000","last_became_peered":"2024-04-20T11:29:00.852731+0000","last_unstale":"2024-04-20T11:29:01.823948+0000","last_undegraded":"2024-04-20T11:29:01.823948+0000","last_fullsized":"2024-04-20T11:29:01.823948+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:45:21.173004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-20T11:29:01.824959+0000","last_change":"2024-04-20T11:28:59.842441+0000","last_active":"2024-04-20T11:29:01.824959+0000","last_peered":"2024-04-20T11:29:01.824959+0000","last_clean":"2024-04-20T11:29:01.824959+0000","last_became_active":"2024-04-20T11:28:59.842033+0000","last_became_peered":"2024-04-20T11:28:59.842033+0000","last_unstale":"2024-04-20T11:29:01.824959+0000","last_undegraded":"2024-04-20T11:29:01.824959+0000","last_fullsized":"2024-04-20T11:29:01.824959+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_clean_scrub_stamp":"2024-04-20T11:28:58.778154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:32:56.087636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T11:29:00.839278+0000","last_change":"2024-04-20T11:29:00.834705+0000","last_active":"2024-04-20T11:29:00.839278+0000","last_peered":"2024-04-20T11:29:00.839278+0000","last_clean":"2024-04-20T11:29:00.839278+0000","last_became_active":"2024-04-20T11:29:00.834411+0000","last_became_peered":"2024-04-20T11:29:00.834411+0000","last_unstale":"2024-04-20T11:29:00.839278+0000","last_undegraded":"2024-04-20T11:29:00.839278+0000","last_fullsized":"2024-04-20T11:29:00.839278+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:14:41.729790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T11:29:00.826291+0000","last_change":"2024-04-20T11:28:59.843411+0000","last_active":"2024-04-20T11:29:00.826291+0000","last_peered":"2024-04-20T11:29:00.826291+0000","last_clean":"2024-04-20T11:29:00.826291+0000","last_became_active":"2024-04-20T11:28:59.843107+0000","last_became_peered":"2024-04-20T11:28:59.843107+0000","last_unstale":"2024-04-20T11:29:00.826291+0000","last_undegraded":"2024-04-20T11:29:00.826291+0000","last_fullsized":"2024-04-20T11:29:00.826291+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_clean_scrub_stamp":"2024-04-20T11:28:58.778154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:02:37.530004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T11:29:00.860095+0000","last_change":"2024-04-20T11:29:00.854859+0000","last_active":"2024-04-20T11:29:00.860095+0000","last_peered":"2024-04-20T11:29:00.860095+0000","last_clean":"2024-04-20T11:29:00.860095+0000","last_became_active":"2024-04-20T11:29:00.854555+0000","last_became_peered":"2024-04-20T11:29:00.854555+0000","last_unstale":"2024-04-20T11:29:00.860095+0000","last_undegraded":"2024-04-20T11:29:00.860095+0000","last_fullsized":"2024-04-20T11:29:00.860095+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:22:04.188741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"45'3","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T11:29:00.850120+0000","last_change":"2024-04-20T11:28:59.849002+0000","last_active":"2024-04-20T11:29:00.850120+0000","last_peered":"2024-04-20T11:29:00.850120+0000","last_clean":"2024-04-20T11:29:00.850120+0000","last_became_active":"2024-04-20T11:28:59.848231+0000","last_became_peered":"2024-04-20T11:28:59.848231+0000","last_unstale":"2024-04-20T11:29:00.850120+0000","last_undegraded":"2024-04-20T11:29:00.850120+0000","last_fullsized":"2024-04-20T11:29:00.850120+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_clean_scrub_stamp":"2024-04-20T11:28:58.778154+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:47:30.010013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T11:29:00.850835+0000","last_change":"2024-04-20T11:29:00.842900+0000","last_active":"2024-04-20T11:29:00.850835+0000","last_peered":"2024-04-20T11:29:00.850835+0000","last_clean":"2024-04-20T11:29:00.850835+0000","last_became_active":"2024-04-20T11:29:00.842694+0000","last_became_peered":"2024-04-20T11:29:00.842694+0000","last_unstale":"2024-04-20T11:29:00.850835+0000","last_undegraded":"2024-04-20T11:29:00.850835+0000","last_fullsized":"2024-04-20T11:29:00.850835+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:45:21.173004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"45'6","reported_seq":20,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-20T11:29:01.821755+0000","last_change":"2024-04-20T11:28:59.842795+0000","last_active":"2024-04-20T11:29:01.821755+0000","last_peered":"2024-04-20T11:29:01.821755+0000","last_clean":"2024-04-20T11:29:01.821755+0000","last_became_active":"2024-04-20T11:28:59.842461+0000","last_became_peered":"2024-04-20T11:28:59.842461+0000","last_unstale":"2024-04-20T11:29:01.821755+0000","last_undegraded":"2024-04-20T11:29:01.821755+0000","last_fullsized":"2024-04-20T11:29:01.821755+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_clean_scrub_stamp":"2024-04-20T11:28:58.778154+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:05:22.464349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-20T11:29:02.173638+0000","last_change":"2024-04-20T11:29:00.837451+0000","last_active":"2024-04-20T11:29:02.173638+0000","last_peered":"2024-04-20T11:29:02.173638+0000","last_clean":"2024-04-20T11:29:02.173638+0000","last_became_active":"2024-04-20T11:29:00.837235+0000","last_became_peered":"2024-04-20T11:29:00.837235+0000","last_unstale":"2024-04-20T11:29:02.173638+0000","last_undegraded":"2024-04-20T11:29:02.173638+0000","last_fullsized":"2024-04-20T11:29:02.173638+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:26:31.816201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"45'2","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T11:29:00.862683+0000","last_change":"2024-04-20T11:28:59.829037+0000","last_active":"2024-04-20T11:29:00.862683+0000","last_peered":"2024-04-20T11:29:00.862683+0000","last_clean":"2024-04-20T11:29:00.862683+0000","last_became_active":"2024-04-20T11:28:59.828625+0000","last_became_peered":"2024-04-20T11:28:59.828625+0000","last_unstale":"2024-04-20T11:29:00.862683+0000","last_undegraded":"2024-04-20T11:29:00.862683+0000","last_fullsized":"2024-04-20T11:29:00.862683+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_clean_scrub_stamp":"2024-04-20T11:28:58.778154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:38:52.674014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T11:29:00.847768+0000","last_change":"2024-04-20T11:29:00.842669+0000","last_active":"2024-04-20T11:29:00.847768+0000","last_peered":"2024-04-20T11:29:00.847768+0000","last_clean":"2024-04-20T11:29:00.847768+0000","last_became_active":"2024-04-20T11:29:00.839643+0000","last_became_peered":"2024-04-20T11:29:00.839643+0000","last_unstale":"2024-04-20T11:29:00.847768+0000","last_undegraded":"2024-04-20T11:29:00.847768+0000","last_fullsized":"2024-04-20T11:29:00.847768+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:31:39.636876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T11:29:00.828266+0000","last_change":"2024-04-20T11:28:59.844639+0000","last_active":"2024-04-20T11:29:00.828266+0000","last_peered":"2024-04-20T11:29:00.828266+0000","last_clean":"2024-04-20T11:29:00.828266+0000","last_became_active":"2024-04-20T11:28:59.844347+0000","last_became_peered":"2024-04-20T11:28:59.844347+0000","last_unstale":"2024-04-20T11:29:00.828266+0000","last_undegraded":"2024-04-20T11:29:00.828266+0000","last_fullsized":"2024-04-20T11:29:00.828266+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_clean_scrub_stamp":"2024-04-20T11:28:58.778154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:17:29.017166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T11:29:00.846586+0000","last_change":"2024-04-20T11:29:00.839116+0000","last_active":"2024-04-20T11:29:00.846586+0000","last_peered":"2024-04-20T11:29:00.846586+0000","last_clean":"2024-04-20T11:29:00.846586+0000","last_became_active":"2024-04-20T11:29:00.838824+0000","last_became_peered":"2024-04-20T11:29:00.838824+0000","last_unstale":"2024-04-20T11:29:00.846586+0000","last_undegraded":"2024-04-20T11:29:00.846586+0000","last_fullsized":"2024-04-20T11:29:00.846586+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:25:49.258923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T11:29:00.823742+0000","last_change":"2024-04-20T11:28:59.834522+0000","last_active":"2024-04-20T11:29:00.823742+0000","last_peered":"2024-04-20T11:29:00.823742+0000","last_clean":"2024-04-20T11:29:00.823742+0000","last_became_active":"2024-04-20T11:28:59.834257+0000","last_became_peered":"2024-04-20T11:28:59.834257+0000","last_unstale":"2024-04-20T11:29:00.823742+0000","last_undegraded":"2024-04-20T11:29:00.823742+0000","last_fullsized":"2024-04-20T11:29:00.823742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_clean_scrub_stamp":"2024-04-20T11:28:58.778154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:44:22.941027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T11:29:00.850913+0000","last_change":"2024-04-20T11:29:00.844930+0000","last_active":"2024-04-20T11:29:00.850913+0000","last_peered":"2024-04-20T11:29:00.850913+0000","last_clean":"2024-04-20T11:29:00.850913+0000","last_became_active":"2024-04-20T11:29:00.844600+0000","last_became_peered":"2024-04-20T11:29:00.844600+0000","last_unstale":"2024-04-20T11:29:00.850913+0000","last_undegraded":"2024-04-20T11:29:00.850913+0000","last_fullsized":"2024-04-20T11:29:00.850913+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:59:51.540993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"45'8","reported_seq":20,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T11:29:00.850857+0000","last_change":"2024-04-20T11:28:59.837654+0000","last_active":"2024-04-20T11:29:00.850857+0000","last_peered":"2024-04-20T11:29:00.850857+0000","last_clean":"2024-04-20T11:29:00.850857+0000","last_became_active":"2024-04-20T11:28:59.837417+0000","last_became_peered":"2024-04-20T11:28:59.837417+0000","last_unstale":"2024-04-20T11:29:00.850857+0000","last_undegraded":"2024-04-20T11:29:00.850857+0000","last_fullsized":"2024-04-20T11:29:00.850857+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_clean_scrub_stamp":"2024-04-20T11:28:58.778154+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:53:15.428338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-20T11:29:01.821553+0000","last_change":"2024-04-20T11:29:00.854804+0000","last_active":"2024-04-20T11:29:01.821553+0000","last_peered":"2024-04-20T11:29:01.821553+0000","last_clean":"2024-04-20T11:29:01.821553+0000","last_became_active":"2024-04-20T11:29:00.853959+0000","last_became_peered":"2024-04-20T11:29:00.853959+0000","last_unstale":"2024-04-20T11:29:01.821553+0000","last_undegraded":"2024-04-20T11:29:01.821553+0000","last_fullsized":"2024-04-20T11:29:01.821553+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:36:14.165535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"45'1","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-20T11:29:01.821693+0000","last_change":"2024-04-20T11:28:59.834155+0000","last_active":"2024-04-20T11:29:01.821693+0000","last_peered":"2024-04-20T11:29:01.821693+0000","last_clean":"2024-04-20T11:29:01.821693+0000","last_became_active":"2024-04-20T11:28:59.833611+0000","last_became_peered":"2024-04-20T11:28:59.833611+0000","last_unstale":"2024-04-20T11:29:01.821693+0000","last_undegraded":"2024-04-20T11:29:01.821693+0000","last_fullsized":"2024-04-20T11:29:01.821693+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_clean_scrub_stamp":"2024-04-20T11:28:58.778154+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:33:53.368434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T11:29:00.847650+0000","last_change":"2024-04-20T11:29:00.839624+0000","last_active":"2024-04-20T11:29:00.847650+0000","last_peered":"2024-04-20T11:29:00.847650+0000","last_clean":"2024-04-20T11:29:00.847650+0000","last_became_active":"2024-04-20T11:29:00.837690+0000","last_became_peered":"2024-04-20T11:29:00.837690+0000","last_unstale":"2024-04-20T11:29:00.847650+0000","last_undegraded":"2024-04-20T11:29:00.847650+0000","last_fullsized":"2024-04-20T11:29:00.847650+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:49:14.491396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"45'2","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-20T11:29:01.824453+0000","last_change":"2024-04-20T11:28:59.846732+0000","last_active":"2024-04-20T11:29:01.824453+0000","last_peered":"2024-04-20T11:29:01.824453+0000","last_clean":"2024-04-20T11:29:01.824453+0000","last_became_active":"2024-04-20T11:28:59.845963+0000","last_became_peered":"2024-04-20T11:28:59.845963+0000","last_unstale":"2024-04-20T11:29:01.824453+0000","last_undegraded":"2024-04-20T11:29:01.824453+0000","last_fullsized":"2024-04-20T11:29:01.824453+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_clean_scrub_stamp":"2024-04-20T11:28:58.778154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:49:13.475530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-20T11:29:02.173373+0000","last_change":"2024-04-20T11:29:00.844061+0000","last_active":"2024-04-20T11:29:02.173373+0000","last_peered":"2024-04-20T11:29:02.173373+0000","last_clean":"2024-04-20T11:29:02.173373+0000","last_became_active":"2024-04-20T11:29:00.843746+0000","last_became_peered":"2024-04-20T11:29:00.843746+0000","last_unstale":"2024-04-20T11:29:02.173373+0000","last_undegraded":"2024-04-20T11:29:02.173373+0000","last_fullsized":"2024-04-20T11:29:02.173373+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:31:39.636876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"45'1","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-20T11:29:01.823512+0000","last_change":"2024-04-20T11:28:59.843578+0000","last_active":"2024-04-20T11:29:01.823512+0000","last_peered":"2024-04-20T11:29:01.823512+0000","last_clean":"2024-04-20T11:29:01.823512+0000","last_became_active":"2024-04-20T11:28:59.843276+0000","last_became_peered":"2024-04-20T11:28:59.843276+0000","last_unstale":"2024-04-20T11:29:01.823512+0000","last_undegraded":"2024-04-20T11:29:01.823512+0000","last_fullsized":"2024-04-20T11:29:01.823512+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_clean_scrub_stamp":"2024-04-20T11:28:58.778154+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:17:29.017166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-20T11:29:02.173916+0000","last_change":"2024-04-20T11:29:00.843375+0000","last_active":"2024-04-20T11:29:02.173916+0000","last_peered":"2024-04-20T11:29:02.173916+0000","last_clean":"2024-04-20T11:29:02.173916+0000","last_became_active":"2024-04-20T11:29:00.843023+0000","last_became_peered":"2024-04-20T11:29:00.843023+0000","last_unstale":"2024-04-20T11:29:02.173916+0000","last_undegraded":"2024-04-20T11:29:02.173916+0000","last_fullsized":"2024-04-20T11:29:02.173916+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:41:05.632073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T11:29:00.822220+0000","last_change":"2024-04-20T11:28:59.837530+0000","last_active":"2024-04-20T11:29:00.822220+0000","last_peered":"2024-04-20T11:29:00.822220+0000","last_clean":"2024-04-20T11:29:00.822220+0000","last_became_active":"2024-04-20T11:28:59.837317+0000","last_became_peered":"2024-04-20T11:28:59.837317+0000","last_unstale":"2024-04-20T11:29:00.822220+0000","last_undegraded":"2024-04-20T11:29:00.822220+0000","last_fullsized":"2024-04-20T11:29:00.822220+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_clean_scrub_stamp":"2024-04-20T11:28:58.778154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:14:40.713924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T11:29:00.859892+0000","last_change":"2024-04-20T11:29:00.855765+0000","last_active":"2024-04-20T11:29:00.859892+0000","last_peered":"2024-04-20T11:29:00.859892+0000","last_clean":"2024-04-20T11:29:00.859892+0000","last_became_active":"2024-04-20T11:29:00.855408+0000","last_became_peered":"2024-04-20T11:29:00.855408+0000","last_unstale":"2024-04-20T11:29:00.859892+0000","last_undegraded":"2024-04-20T11:29:00.859892+0000","last_fullsized":"2024-04-20T11:29:00.859892+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:38:53.536464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"45'2","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T11:29:00.860723+0000","last_change":"2024-04-20T11:28:59.844376+0000","last_active":"2024-04-20T11:29:00.860723+0000","last_peered":"2024-04-20T11:29:00.860723+0000","last_clean":"2024-04-20T11:29:00.860723+0000","last_became_active":"2024-04-20T11:28:59.843994+0000","last_became_peered":"2024-04-20T11:28:59.843994+0000","last_unstale":"2024-04-20T11:29:00.860723+0000","last_undegraded":"2024-04-20T11:29:00.860723+0000","last_fullsized":"2024-04-20T11:29:00.860723+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_clean_scrub_stamp":"2024-04-20T11:28:58.778154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:59:50.525128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T11:29:00.850401+0000","last_change":"2024-04-20T11:29:00.844159+0000","last_active":"2024-04-20T11:29:00.850401+0000","last_peered":"2024-04-20T11:29:00.850401+0000","last_clean":"2024-04-20T11:29:00.850401+0000","last_became_active":"2024-04-20T11:29:00.843884+0000","last_became_peered":"2024-04-20T11:29:00.843884+0000","last_unstale":"2024-04-20T11:29:00.850401+0000","last_undegraded":"2024-04-20T11:29:00.850401+0000","last_fullsized":"2024-04-20T11:29:00.850401+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:38:21.618746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-20T11:29:01.821951+0000","last_change":"2024-04-20T11:28:59.849414+0000","last_active":"2024-04-20T11:29:01.821951+0000","last_peered":"2024-04-20T11:29:01.821951+0000","last_clean":"2024-04-20T11:29:01.821951+0000","last_became_active":"2024-04-20T11:28:59.847528+0000","last_became_peered":"2024-04-20T11:28:59.847528+0000","last_unstale":"2024-04-20T11:29:01.821951+0000","last_undegraded":"2024-04-20T11:29:01.821951+0000","last_fullsized":"2024-04-20T11:29:01.821951+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_clean_scrub_stamp":"2024-04-20T11:28:58.778154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:41:04.616207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-20T11:29:01.821217+0000","last_change":"2024-04-20T11:29:00.850578+0000","last_active":"2024-04-20T11:29:01.821217+0000","last_peered":"2024-04-20T11:29:01.821217+0000","last_clean":"2024-04-20T11:29:01.821217+0000","last_became_active":"2024-04-20T11:29:00.848473+0000","last_became_peered":"2024-04-20T11:29:00.848473+0000","last_unstale":"2024-04-20T11:29:01.821217+0000","last_undegraded":"2024-04-20T11:29:01.821217+0000","last_fullsized":"2024-04-20T11:29:01.821217+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:25:14.723857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T11:29:00.818698+0000","last_change":"2024-04-20T11:28:59.846947+0000","last_active":"2024-04-20T11:29:00.818698+0000","last_peered":"2024-04-20T11:29:00.818698+0000","last_clean":"2024-04-20T11:29:00.818698+0000","last_became_active":"2024-04-20T11:28:59.846509+0000","last_became_peered":"2024-04-20T11:28:59.846509+0000","last_unstale":"2024-04-20T11:29:00.818698+0000","last_undegraded":"2024-04-20T11:29:00.818698+0000","last_fullsized":"2024-04-20T11:29:00.818698+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_clean_scrub_stamp":"2024-04-20T11:28:58.778154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:28:01.628720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T11:29:00.847193+0000","last_change":"2024-04-20T11:29:00.837320+0000","last_active":"2024-04-20T11:29:00.847193+0000","last_peered":"2024-04-20T11:29:00.847193+0000","last_clean":"2024-04-20T11:29:00.847193+0000","last_became_active":"2024-04-20T11:29:00.837034+0000","last_became_peered":"2024-04-20T11:29:00.837034+0000","last_unstale":"2024-04-20T11:29:00.847193+0000","last_undegraded":"2024-04-20T11:29:00.847193+0000","last_fullsized":"2024-04-20T11:29:00.847193+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:10:01.731112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T11:29:00.826873+0000","last_change":"2024-04-20T11:28:59.844566+0000","last_active":"2024-04-20T11:29:00.826873+0000","last_peered":"2024-04-20T11:29:00.826873+0000","last_clean":"2024-04-20T11:29:00.826873+0000","last_became_active":"2024-04-20T11:28:59.844253+0000","last_became_peered":"2024-04-20T11:28:59.844253+0000","last_unstale":"2024-04-20T11:29:00.826873+0000","last_undegraded":"2024-04-20T11:29:00.826873+0000","last_fullsized":"2024-04-20T11:29:00.826873+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_clean_scrub_stamp":"2024-04-20T11:28:58.778154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:41:04.616207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-20T11:29:02.174310+0000","last_change":"2024-04-20T11:29:00.848797+0000","last_active":"2024-04-20T11:29:02.174310+0000","last_peered":"2024-04-20T11:29:02.174310+0000","last_clean":"2024-04-20T11:29:02.174310+0000","last_became_active":"2024-04-20T11:29:00.848519+0000","last_became_peered":"2024-04-20T11:29:00.848519+0000","last_unstale":"2024-04-20T11:29:02.174310+0000","last_undegraded":"2024-04-20T11:29:02.174310+0000","last_fullsized":"2024-04-20T11:29:02.174310+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:47:31.025879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T11:29:00.823354+0000","last_change":"2024-04-20T11:28:59.850859+0000","last_active":"2024-04-20T11:29:00.823354+0000","last_peered":"2024-04-20T11:29:00.823354+0000","last_clean":"2024-04-20T11:29:00.823354+0000","last_became_active":"2024-04-20T11:28:59.850410+0000","last_became_peered":"2024-04-20T11:28:59.850410+0000","last_unstale":"2024-04-20T11:29:00.823354+0000","last_undegraded":"2024-04-20T11:29:00.823354+0000","last_fullsized":"2024-04-20T11:29:00.823354+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_clean_scrub_stamp":"2024-04-20T11:28:58.778154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:45:20.157138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T11:29:00.861173+0000","last_change":"2024-04-20T11:29:00.854443+0000","last_active":"2024-04-20T11:29:00.861173+0000","last_peered":"2024-04-20T11:29:00.861173+0000","last_clean":"2024-04-20T11:29:00.861173+0000","last_became_active":"2024-04-20T11:29:00.853615+0000","last_became_peered":"2024-04-20T11:29:00.853615+0000","last_unstale":"2024-04-20T11:29:00.861173+0000","last_undegraded":"2024-04-20T11:29:00.861173+0000","last_fullsized":"2024-04-20T11:29:00.861173+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:03:36.963877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T11:29:00.832105+0000","last_change":"2024-04-20T11:28:59.849272+0000","last_active":"2024-04-20T11:29:00.832105+0000","last_peered":"2024-04-20T11:29:00.832105+0000","last_clean":"2024-04-20T11:29:00.832105+0000","last_became_active":"2024-04-20T11:28:59.848656+0000","last_became_peered":"2024-04-20T11:28:59.848656+0000","last_unstale":"2024-04-20T11:29:00.832105+0000","last_undegraded":"2024-04-20T11:29:00.832105+0000","last_fullsized":"2024-04-20T11:29:00.832105+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_clean_scrub_stamp":"2024-04-20T11:28:58.778154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:35:46.785374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T11:29:00.865492+0000","last_change":"2024-04-20T11:29:00.861672+0000","last_active":"2024-04-20T11:29:00.865492+0000","last_peered":"2024-04-20T11:29:00.865492+0000","last_clean":"2024-04-20T11:29:00.865492+0000","last_became_active":"2024-04-20T11:29:00.861272+0000","last_became_peered":"2024-04-20T11:29:00.861272+0000","last_unstale":"2024-04-20T11:29:00.865492+0000","last_undegraded":"2024-04-20T11:29:00.865492+0000","last_fullsized":"2024-04-20T11:29:00.865492+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:44:23.956892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T11:29:00.830282+0000","last_change":"2024-04-20T11:28:59.845063+0000","last_active":"2024-04-20T11:29:00.830282+0000","last_peered":"2024-04-20T11:29:00.830282+0000","last_clean":"2024-04-20T11:29:00.830282+0000","last_became_active":"2024-04-20T11:28:59.844762+0000","last_became_peered":"2024-04-20T11:28:59.844762+0000","last_unstale":"2024-04-20T11:29:00.830282+0000","last_undegraded":"2024-04-20T11:29:00.830282+0000","last_fullsized":"2024-04-20T11:29:00.830282+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_clean_scrub_stamp":"2024-04-20T11:28:58.778154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:20:10.383841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T11:29:00.846567+0000","last_change":"2024-04-20T11:29:00.838933+0000","last_active":"2024-04-20T11:29:00.846567+0000","last_peered":"2024-04-20T11:29:00.846567+0000","last_clean":"2024-04-20T11:29:00.846567+0000","last_became_active":"2024-04-20T11:29:00.838651+0000","last_became_peered":"2024-04-20T11:29:00.838651+0000","last_unstale":"2024-04-20T11:29:00.846567+0000","last_undegraded":"2024-04-20T11:29:00.846567+0000","last_fullsized":"2024-04-20T11:29:00.846567+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:59:22.929015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T11:29:00.828804+0000","last_change":"2024-04-20T11:28:59.848630+0000","last_active":"2024-04-20T11:29:00.828804+0000","last_peered":"2024-04-20T11:29:00.828804+0000","last_clean":"2024-04-20T11:29:00.828804+0000","last_became_active":"2024-04-20T11:28:59.848339+0000","last_became_peered":"2024-04-20T11:28:59.848339+0000","last_unstale":"2024-04-20T11:29:00.828804+0000","last_undegraded":"2024-04-20T11:29:00.828804+0000","last_fullsized":"2024-04-20T11:29:00.828804+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_clean_scrub_stamp":"2024-04-20T11:28:58.778154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:45:36.561066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T11:29:00.859568+0000","last_change":"2024-04-20T11:29:00.855621+0000","last_active":"2024-04-20T11:29:00.859568+0000","last_peered":"2024-04-20T11:29:00.859568+0000","last_clean":"2024-04-20T11:29:00.859568+0000","last_became_active":"2024-04-20T11:29:00.855310+0000","last_became_peered":"2024-04-20T11:29:00.855310+0000","last_unstale":"2024-04-20T11:29:00.859568+0000","last_undegraded":"2024-04-20T11:29:00.859568+0000","last_fullsized":"2024-04-20T11:29:00.859568+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:07:52.578138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-20T11:29:01.823579+0000","last_change":"2024-04-20T11:28:59.845146+0000","last_active":"2024-04-20T11:29:01.823579+0000","last_peered":"2024-04-20T11:29:01.823579+0000","last_clean":"2024-04-20T11:29:01.823579+0000","last_became_active":"2024-04-20T11:28:59.844788+0000","last_became_peered":"2024-04-20T11:28:59.844788+0000","last_unstale":"2024-04-20T11:29:01.823579+0000","last_undegraded":"2024-04-20T11:29:01.823579+0000","last_fullsized":"2024-04-20T11:29:01.823579+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_clean_scrub_stamp":"2024-04-20T11:28:58.778154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:10:00.715247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-20T11:29:01.822179+0000","last_change":"2024-04-20T11:29:00.837423+0000","last_active":"2024-04-20T11:29:01.822179+0000","last_peered":"2024-04-20T11:29:01.822179+0000","last_clean":"2024-04-20T11:29:01.822179+0000","last_became_active":"2024-04-20T11:29:00.836988+0000","last_became_peered":"2024-04-20T11:29:00.836988+0000","last_unstale":"2024-04-20T11:29:01.822179+0000","last_undegraded":"2024-04-20T11:29:01.822179+0000","last_fullsized":"2024-04-20T11:29:01.822179+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:45:21.173004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T11:29:00.831355+0000","last_change":"2024-04-20T11:28:59.842194+0000","last_active":"2024-04-20T11:29:00.831355+0000","last_peered":"2024-04-20T11:29:00.831355+0000","last_clean":"2024-04-20T11:29:00.831355+0000","last_became_active":"2024-04-20T11:28:59.841860+0000","last_became_peered":"2024-04-20T11:28:59.841860+0000","last_unstale":"2024-04-20T11:29:00.831355+0000","last_undegraded":"2024-04-20T11:29:00.831355+0000","last_fullsized":"2024-04-20T11:29:00.831355+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_clean_scrub_stamp":"2024-04-20T11:28:58.778154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:10:00.715247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-20T11:29:01.822782+0000","last_change":"2024-04-20T11:29:00.850105+0000","last_active":"2024-04-20T11:29:01.822782+0000","last_peered":"2024-04-20T11:29:01.822782+0000","last_clean":"2024-04-20T11:29:01.822782+0000","last_became_active":"2024-04-20T11:29:00.849559+0000","last_became_peered":"2024-04-20T11:29:00.849559+0000","last_unstale":"2024-04-20T11:29:01.822782+0000","last_undegraded":"2024-04-20T11:29:01.822782+0000","last_fullsized":"2024-04-20T11:29:01.822782+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:35:47.801239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T11:29:00.819219+0000","last_change":"2024-04-20T11:28:59.825491+0000","last_active":"2024-04-20T11:29:00.819219+0000","last_peered":"2024-04-20T11:29:00.819219+0000","last_clean":"2024-04-20T11:29:00.819219+0000","last_became_active":"2024-04-20T11:28:59.824730+0000","last_became_peered":"2024-04-20T11:28:59.824730+0000","last_unstale":"2024-04-20T11:29:00.819219+0000","last_undegraded":"2024-04-20T11:29:00.819219+0000","last_fullsized":"2024-04-20T11:29:00.819219+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_clean_scrub_stamp":"2024-04-20T11:28:58.778154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:25:13.707992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T11:29:00.847931+0000","last_change":"2024-04-20T11:29:00.839067+0000","last_active":"2024-04-20T11:29:00.847931+0000","last_peered":"2024-04-20T11:29:00.847931+0000","last_clean":"2024-04-20T11:29:00.847931+0000","last_became_active":"2024-04-20T11:29:00.838807+0000","last_became_peered":"2024-04-20T11:29:00.838807+0000","last_unstale":"2024-04-20T11:29:00.847931+0000","last_undegraded":"2024-04-20T11:29:00.847931+0000","last_fullsized":"2024-04-20T11:29:00.847931+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:38:21.618746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"45'2","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-20T11:29:01.824228+0000","last_change":"2024-04-20T11:28:59.839177+0000","last_active":"2024-04-20T11:29:01.824228+0000","last_peered":"2024-04-20T11:29:01.824228+0000","last_clean":"2024-04-20T11:29:01.824228+0000","last_became_active":"2024-04-20T11:28:59.838810+0000","last_became_peered":"2024-04-20T11:28:59.838810+0000","last_unstale":"2024-04-20T11:29:01.824228+0000","last_undegraded":"2024-04-20T11:29:01.824228+0000","last_fullsized":"2024-04-20T11:29:01.824228+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_clean_scrub_stamp":"2024-04-20T11:28:58.778154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:26:30.800335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":215,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T11:29:00.826931+0000","last_change":"2024-04-20T11:26:48.661897+0000","last_active":"2024-04-20T11:29:00.826931+0000","last_peered":"2024-04-20T11:29:00.826931+0000","last_clean":"2024-04-20T11:29:00.826931+0000","last_became_active":"2024-04-20T11:26:48.656535+0000","last_became_peered":"2024-04-20T11:26:48.656535+0000","last_unstale":"2024-04-20T11:29:00.826931+0000","last_undegraded":"2024-04-20T11:29:00.826931+0000","last_fullsized":"2024-04-20T11:29:00.826931+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:47.543004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:47.543004+0000","last_clean_scrub_stamp":"2024-04-20T11:26:47.543004+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:58:35.527105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574880,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28120,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343720,"statfs":{"total":96636764160,"available":96607969280,"internally_reserved":0,"allocated":1126400,"data_stored":805306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574877,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27548,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344292,"statfs":{"total":96636764160,"available":96608555008,"internally_reserved":0,"allocated":536576,"data_stored":215520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574879,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27556,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344284,"statfs":{"total":96636764160,"available":96608546816,"internally_reserved":0,"allocated":544768,"data_stored":221218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574877,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27548,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344292,"statfs":{"total":96636764160,"available":96608555008,"internally_reserved":0,"allocated":540672,"data_stored":216460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574878,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27548,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344292,"statfs":{"total":96636764160,"available":96608555008,"internally_reserved":0,"allocated":540672,"data_stored":215486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574879,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27572,"kb_used_data":544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344268,"statfs":{"total":96636764160,"available":96608530432,"internally_reserved":0,"allocated":557056,"data_stored":222782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574877,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27540,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344300,"statfs":{"total":96636764160,"available":96608563200,"internally_reserved":0,"allocated":532480,"data_stored":214918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574878,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28120,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343720,"statfs":{"total":96636764160,"available":96607969280,"internally_reserved":0,"allocated":1126400,"data_stored":805250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T11:29:05.221 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T11:29:05.447 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.448+0000 7f0f7d200640 1 Processor -- start 2024-04-20T11:29:05.448 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.448+0000 7f0f7d200640 1 -- start start 2024-04-20T11:29:05.449 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.450+0000 7f0f7d200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0f78160870 0x7f0f78163c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:05.450 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.450+0000 7f0f7d200640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f781641b0 con 0x7f0f780b3d80 2024-04-20T11:29:05.450 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.450+0000 7f0f7d200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f78072bd0 con 0x7f0f780b41e0 2024-04-20T11:29:05.450 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.450+0000 7f0f7d200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0f78160c50 con 0x7f0f78160870 2024-04-20T11:29:05.450 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.450+0000 7f0f6fe00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f0f780b3d80 0x7f0f78071db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:42860/0 (socket says 172.21.15.152:42860) 2024-04-20T11:29:05.450 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.450+0000 7f0f6fe00640 1 -- 172.21.15.152:0/2909221500 learned_addr learned my addr 172.21.15.152:0/2909221500 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:29:05.452 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.450+0000 7f0f75e00640 1 --2- 172.21.15.152:0/2909221500 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0f78160870 0x7f0f78163c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:05.452 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.451+0000 7f0f75400640 1 -- 172.21.15.152:0/2909221500 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3432497330 0 0) 0x7f0f60001110 con 0x7f0f780b3d80 2024-04-20T11:29:05.452 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.451+0000 7f0f75400640 1 -- 172.21.15.152:0/2909221500 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0f5c003520 con 0x7f0f780b3d80 2024-04-20T11:29:05.452 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.451+0000 7f0f75400640 1 -- 172.21.15.152:0/2909221500 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2383541500 0 0) 0x7f0f5c003520 con 0x7f0f780b3d80 2024-04-20T11:29:05.452 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.451+0000 7f0f75400640 1 -- 172.21.15.152:0/2909221500 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0f78160870 msgr2=0x7f0f78163c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:05.452 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.451+0000 7f0f75400640 1 --2- 172.21.15.152:0/2909221500 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0f78160870 0x7f0f78163c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:05.452 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.451+0000 7f0f75400640 1 -- 172.21.15.152:0/2909221500 >> v1:172.21.15.175:6789/0 conn(0x7f0f780b41e0 legacy=0x7f0f780724c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:05.452 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.451+0000 7f0f75400640 1 -- 172.21.15.152:0/2909221500 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f781653c0 con 0x7f0f780b3d80 2024-04-20T11:29:05.452 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.451+0000 7f0f75400640 1 -- 172.21.15.152:0/2909221500 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f0f60001840 con 0x7f0f780b3d80 2024-04-20T11:29:05.452 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.451+0000 7f0f75400640 1 -- 172.21.15.152:0/2909221500 >> v1:172.21.15.152:6789/0 conn(0x7f0f780b3d80 legacy=0x7f0f78071db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:05.452 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.451+0000 7f0f75400640 1 --2- 172.21.15.152:0/2909221500 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0f5c003a50 0x7f0f5c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:05.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.451+0000 7f0f75400640 1 --2- 172.21.15.152:0/2909221500 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0f5c004800 0x7f0f5c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:05.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.451+0000 7f0f75400640 1 --2- 172.21.15.152:0/2909221500 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0f78160870 0x7f0f5c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:05.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.451+0000 7f0f75400640 1 -- 172.21.15.152:0/2909221500 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f0f78160c50 con 0x7f0f5c003a50 2024-04-20T11:29:05.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.451+0000 7f0f75400640 1 -- 172.21.15.152:0/2909221500 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0f5c00bac0 con 0x7f0f5c004800 2024-04-20T11:29:05.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.451+0000 7f0f75400640 1 -- 172.21.15.152:0/2909221500 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0f5c00bcc0 con 0x7f0f78160870 2024-04-20T11:29:05.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.451+0000 7f0f75e00640 1 --2- 172.21.15.152:0/2909221500 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0f5c003a50 0x7f0f5c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:05.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.452+0000 7f0f75e00640 1 -- 172.21.15.152:0/2909221500 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0f78160870 msgr2=0x7f0f5c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:05.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.452+0000 7f0f75e00640 1 --2- 172.21.15.152:0/2909221500 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0f78160870 0x7f0f5c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:05.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.452+0000 7f0f75e00640 1 -- 172.21.15.152:0/2909221500 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0f5c004800 msgr2=0x7f0f5c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:05.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.452+0000 7f0f6fe00640 1 --2- 172.21.15.152:0/2909221500 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0f5c004800 0x7f0f5c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:05.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.452+0000 7f0f75e00640 1 --2- 172.21.15.152:0/2909221500 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0f5c004800 0x7f0f5c004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:05.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.452+0000 7f0f75e00640 1 -- 172.21.15.152:0/2909221500 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f5c003520 con 0x7f0f5c003a50 2024-04-20T11:29:05.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.453+0000 7f0f75e00640 1 --2- 172.21.15.152:0/2909221500 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0f5c003a50 0x7f0f5c003e70 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f0f68009ac0 tx=0x7f0f5c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8e663953791c91a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:05.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.453+0000 7f0f75400640 1 -- 172.21.15.152:0/2909221500 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f68039070 con 0x7f0f5c003a50 2024-04-20T11:29:05.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.453+0000 7f0f75400640 1 -- 172.21.15.152:0/2909221500 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f6802c710 con 0x7f0f5c003a50 2024-04-20T11:29:05.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.453+0000 7f0f75400640 1 -- 172.21.15.152:0/2909221500 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f68009690 con 0x7f0f5c003a50 2024-04-20T11:29:05.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.454+0000 7f0f7d200640 1 -- 172.21.15.152:0/2909221500 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0f5c003a50 msgr2=0x7f0f5c003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:05.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.454+0000 7f0f7d200640 1 --2- 172.21.15.152:0/2909221500 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0f5c003a50 0x7f0f5c003e70 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f0f68009ac0 tx=0x7f0f5c005830 comp rx=0 tx=0).stop 2024-04-20T11:29:05.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.454+0000 7f0f76800640 1 -- 172.21.15.152:0/2909221500 reap_dead start 2024-04-20T11:29:05.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.454+0000 7f0f7d200640 1 -- 172.21.15.152:0/2909221500 shutdown_connections 2024-04-20T11:29:05.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.454+0000 7f0f7d200640 1 -- 172.21.15.152:0/2909221500 >> 172.21.15.152:0/2909221500 conn(0x7f0f78067cf0 msgr2=0x7f0f7806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:05.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.454+0000 7f0f7d200640 1 -- 172.21.15.152:0/2909221500 shutdown_connections 2024-04-20T11:29:05.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.454+0000 7f0f7d200640 1 -- 172.21.15.152:0/2909221500 wait complete. 2024-04-20T11:29:05.456 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.456+0000 7f0f7d200640 1 Processor -- start 2024-04-20T11:29:05.456 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.456+0000 7f0f7d200640 1 -- start start 2024-04-20T11:29:05.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.457+0000 7f0f7d200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0f5c003a50 0x7f0f780cfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:05.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.457+0000 7f0f7d200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0f780ce8c0 0x7f0f780cd060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:05.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.457+0000 7f0f7d200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0f7814e490 0x7f0f78161670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:05.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.457+0000 7f0f7d200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f0f7806e700 con 0x7f0f7814e490 2024-04-20T11:29:05.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.457+0000 7f0f7d200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0f7806e420 con 0x7f0f780ce8c0 2024-04-20T11:29:05.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.457+0000 7f0f7d200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0f7806e9e0 con 0x7f0f5c003a50 2024-04-20T11:29:05.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.457+0000 7f0f76800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0f7814e490 0x7f0f78161670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:05.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.457+0000 7f0f76800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0f7814e490 0x7f0f78161670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:45274/0 (socket says 172.21.15.152:45274) 2024-04-20T11:29:05.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.457+0000 7f0f76800640 1 -- 172.21.15.152:0/570679495 learned_addr learned my addr 172.21.15.152:0/570679495 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:29:05.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.458+0000 7f0f76800640 1 -- 172.21.15.152:0/570679495 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0f5c003a50 msgr2=0x7f0f780cfcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:05.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.458+0000 7f0f75e00640 1 --2- 172.21.15.152:0/570679495 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0f5c003a50 0x7f0f780cfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:05.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.458+0000 7f0f6fe00640 1 --2- 172.21.15.152:0/570679495 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0f780ce8c0 0x7f0f780cd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:05.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.458+0000 7f0f76800640 1 --2- 172.21.15.152:0/570679495 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0f5c003a50 0x7f0f780cfcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:05.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.458+0000 7f0f76800640 1 -- 172.21.15.152:0/570679495 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0f780ce8c0 msgr2=0x7f0f780cd060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:05.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.458+0000 7f0f76800640 1 --2- 172.21.15.152:0/570679495 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0f780ce8c0 0x7f0f780cd060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:05.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.458+0000 7f0f76800640 1 -- 172.21.15.152:0/570679495 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f780cd5a0 con 0x7f0f7814e490 2024-04-20T11:29:05.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.458+0000 7f0f76800640 1 --2- 172.21.15.152:0/570679495 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0f7814e490 0x7f0f78161670 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f0f7806ff80 tx=0x7f0f7000cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:05.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.458+0000 7f0f6e000640 1 -- 172.21.15.152:0/570679495 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f70017070 con 0x7f0f7814e490 2024-04-20T11:29:05.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.458+0000 7f0f6e000640 1 -- 172.21.15.152:0/570679495 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f70005d60 con 0x7f0f7814e490 2024-04-20T11:29:05.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.458+0000 7f0f6e000640 1 -- 172.21.15.152:0/570679495 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f7000a850 con 0x7f0f7814e490 2024-04-20T11:29:05.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.458+0000 7f0f7d200640 1 -- 172.21.15.152:0/570679495 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f780cd820 con 0x7f0f7814e490 2024-04-20T11:29:05.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.458+0000 7f0f7d200640 1 -- 172.21.15.152:0/570679495 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f780cdde0 con 0x7f0f7814e490 2024-04-20T11:29:05.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.459+0000 7f0f7d200640 1 -- 172.21.15.152:0/570679495 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f780689f0 con 0x7f0f7814e490 2024-04-20T11:29:05.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.460+0000 7f0f6e000640 1 -- 172.21.15.152:0/570679495 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f0f7000a9f0 con 0x7f0f7814e490 2024-04-20T11:29:05.461 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.461+0000 7f0f6e000640 1 --2- 172.21.15.152:0/570679495 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f0f5406cad0 0x7f0f5406ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:05.461 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.461+0000 7f0f6e000640 1 -- 172.21.15.152:0/570679495 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f0f7008b120 con 0x7f0f7814e490 2024-04-20T11:29:05.461 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.462+0000 7f0f75e00640 1 --2- 172.21.15.152:0/570679495 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f0f5406cad0 0x7f0f5406ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:05.462 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.462+0000 7f0f75e00640 1 --2- 172.21.15.152:0/570679495 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f0f5406cad0 0x7f0f5406ef90 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f0f68036000 tx=0x7f0f68030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:05.465 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.466+0000 7f0f6e000640 1 -- 172.21.15.152:0/570679495 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f0f780cd5a0 con 0x7f0f7814e490 2024-04-20T11:29:05.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.661+0000 7f0f7d200640 1 -- 172.21.15.152:0/570679495 --> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0f780b2d40 con 0x7f0f5406cad0 2024-04-20T11:29:05.672 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.672+0000 7f0f6e000640 1 -- 172.21.15.152:0/570679495 <== mgr.4113 v2:172.21.15.152:6832/2285110240 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251920 (secure 0 0 0) 0x7f0f780b2d40 con 0x7f0f5406cad0 2024-04-20T11:29:05.676 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:29:05.677 INFO:teuthology.orchestra.run.smithi152.stderr:dumped all 2024-04-20T11:29:05.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.679+0000 7f0f7d200640 1 -- 172.21.15.152:0/570679495 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f0f5406cad0 msgr2=0x7f0f5406ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:05.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.679+0000 7f0f7d200640 1 --2- 172.21.15.152:0/570679495 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f0f5406cad0 0x7f0f5406ef90 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f0f68036000 tx=0x7f0f68030000 comp rx=0 tx=0).stop 2024-04-20T11:29:05.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.679+0000 7f0f7d200640 1 -- 172.21.15.152:0/570679495 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0f7814e490 msgr2=0x7f0f78161670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:05.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.679+0000 7f0f7d200640 1 --2- 172.21.15.152:0/570679495 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0f7814e490 0x7f0f78161670 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f0f7806ff80 tx=0x7f0f7000cb90 comp rx=0 tx=0).stop 2024-04-20T11:29:05.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.679+0000 7f0f7d200640 1 -- 172.21.15.152:0/570679495 shutdown_connections 2024-04-20T11:29:05.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.679+0000 7f0f7d200640 1 --2- 172.21.15.152:0/570679495 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f0f5406cad0 0x7f0f5406ef90 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:05.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.679+0000 7f0f7d200640 1 --2- 172.21.15.152:0/570679495 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0f7814e490 0x7f0f78161670 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:05.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.679+0000 7f0f7d200640 1 --2- 172.21.15.152:0/570679495 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0f780ce8c0 0x7f0f780cd060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:05.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.679+0000 7f0f7d200640 1 --2- 172.21.15.152:0/570679495 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0f5c003a50 0x7f0f780cfcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:05.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.679+0000 7f0f7d200640 1 -- 172.21.15.152:0/570679495 >> 172.21.15.152:0/570679495 conn(0x7f0f78067cf0 msgr2=0x7f0f78165ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:05.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.679+0000 7f0f7d200640 1 -- 172.21.15.152:0/570679495 shutdown_connections 2024-04-20T11:29:05.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.680+0000 7f0f7d200640 1 -- 172.21.15.152:0/570679495 wait complete. 2024-04-20T11:29:05.697 INFO:teuthology.orchestra.run.smithi152.stdout:{"pg_ready":true,"pg_map":{"version":108,"stamp":"2024-04-20T11:29:05.252679+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":47,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":221552,"kb_used_data":5376,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753168,"statfs":{"total":773094113280,"available":772867244032,"internally_reserved":0,"allocated":5505024,"data_stored":2916940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12746,"internal_metadata":221236790},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002126"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T11:29:00.859574+0000","last_change":"2024-04-20T11:29:00.852985+0000","last_active":"2024-04-20T11:29:00.859574+0000","last_peered":"2024-04-20T11:29:00.859574+0000","last_clean":"2024-04-20T11:29:00.859574+0000","last_became_active":"2024-04-20T11:29:00.852677+0000","last_became_peered":"2024-04-20T11:29:00.852677+0000","last_unstale":"2024-04-20T11:29:00.859574+0000","last_undegraded":"2024-04-20T11:29:00.859574+0000","last_fullsized":"2024-04-20T11:29:00.859574+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:57:55.715567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-20T11:29:01.823442+0000","last_change":"2024-04-20T11:29:00.835034+0000","last_active":"2024-04-20T11:29:01.823442+0000","last_peered":"2024-04-20T11:29:01.823442+0000","last_clean":"2024-04-20T11:29:01.823442+0000","last_became_active":"2024-04-20T11:29:00.834736+0000","last_became_peered":"2024-04-20T11:29:00.834736+0000","last_unstale":"2024-04-20T11:29:01.823442+0000","last_undegraded":"2024-04-20T11:29:01.823442+0000","last_fullsized":"2024-04-20T11:29:01.823442+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:57:55.715567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-20T11:29:01.825279+0000","last_change":"2024-04-20T11:29:00.863374+0000","last_active":"2024-04-20T11:29:01.825279+0000","last_peered":"2024-04-20T11:29:01.825279+0000","last_clean":"2024-04-20T11:29:01.825279+0000","last_became_active":"2024-04-20T11:29:00.863070+0000","last_became_peered":"2024-04-20T11:29:00.863070+0000","last_unstale":"2024-04-20T11:29:01.825279+0000","last_undegraded":"2024-04-20T11:29:01.825279+0000","last_fullsized":"2024-04-20T11:29:01.825279+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:22:59.242819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T11:29:00.851044+0000","last_change":"2024-04-20T11:29:00.846863+0000","last_active":"2024-04-20T11:29:00.851044+0000","last_peered":"2024-04-20T11:29:00.851044+0000","last_clean":"2024-04-20T11:29:00.851044+0000","last_became_active":"2024-04-20T11:29:00.845911+0000","last_became_peered":"2024-04-20T11:29:00.845911+0000","last_unstale":"2024-04-20T11:29:00.851044+0000","last_undegraded":"2024-04-20T11:29:00.851044+0000","last_fullsized":"2024-04-20T11:29:00.851044+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:17:30.033031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-20T11:29:01.822679+0000","last_change":"2024-04-20T11:29:00.836049+0000","last_active":"2024-04-20T11:29:01.822679+0000","last_peered":"2024-04-20T11:29:01.822679+0000","last_clean":"2024-04-20T11:29:01.822679+0000","last_became_active":"2024-04-20T11:29:00.833697+0000","last_became_peered":"2024-04-20T11:29:00.833697+0000","last_unstale":"2024-04-20T11:29:01.822679+0000","last_undegraded":"2024-04-20T11:29:01.822679+0000","last_fullsized":"2024-04-20T11:29:01.822679+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:59:51.540993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T11:29:00.837842+0000","last_change":"2024-04-20T11:29:00.830759+0000","last_active":"2024-04-20T11:29:00.837842+0000","last_peered":"2024-04-20T11:29:00.837842+0000","last_clean":"2024-04-20T11:29:00.837842+0000","last_became_active":"2024-04-20T11:29:00.830600+0000","last_became_peered":"2024-04-20T11:29:00.830600+0000","last_unstale":"2024-04-20T11:29:00.837842+0000","last_undegraded":"2024-04-20T11:29:00.837842+0000","last_fullsized":"2024-04-20T11:29:00.837842+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:49:18.547172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T11:29:00.850901+0000","last_change":"2024-04-20T11:29:00.838802+0000","last_active":"2024-04-20T11:29:00.850901+0000","last_peered":"2024-04-20T11:29:00.850901+0000","last_clean":"2024-04-20T11:29:00.850901+0000","last_became_active":"2024-04-20T11:29:00.838530+0000","last_became_peered":"2024-04-20T11:29:00.838530+0000","last_unstale":"2024-04-20T11:29:00.850901+0000","last_undegraded":"2024-04-20T11:29:00.850901+0000","last_fullsized":"2024-04-20T11:29:00.850901+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:38:53.689879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T11:29:00.845300+0000","last_change":"2024-04-20T11:29:00.834424+0000","last_active":"2024-04-20T11:29:00.845300+0000","last_peered":"2024-04-20T11:29:00.845300+0000","last_clean":"2024-04-20T11:29:00.845300+0000","last_became_active":"2024-04-20T11:29:00.834082+0000","last_became_peered":"2024-04-20T11:29:00.834082+0000","last_unstale":"2024-04-20T11:29:00.845300+0000","last_undegraded":"2024-04-20T11:29:00.845300+0000","last_fullsized":"2024-04-20T11:29:00.845300+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:20:11.399706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T11:29:00.850604+0000","last_change":"2024-04-20T11:29:00.845480+0000","last_active":"2024-04-20T11:29:00.850604+0000","last_peered":"2024-04-20T11:29:00.850604+0000","last_clean":"2024-04-20T11:29:00.850604+0000","last_became_active":"2024-04-20T11:29:00.845159+0000","last_became_peered":"2024-04-20T11:29:00.845159+0000","last_unstale":"2024-04-20T11:29:00.850604+0000","last_undegraded":"2024-04-20T11:29:00.850604+0000","last_fullsized":"2024-04-20T11:29:00.850604+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:26:31.816201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T11:29:00.850747+0000","last_change":"2024-04-20T11:29:00.845831+0000","last_active":"2024-04-20T11:29:00.850747+0000","last_peered":"2024-04-20T11:29:00.850747+0000","last_clean":"2024-04-20T11:29:00.850747+0000","last_became_active":"2024-04-20T11:29:00.845168+0000","last_became_peered":"2024-04-20T11:29:00.845168+0000","last_unstale":"2024-04-20T11:29:00.850747+0000","last_undegraded":"2024-04-20T11:29:00.850747+0000","last_fullsized":"2024-04-20T11:29:00.850747+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:41:05.632073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T11:29:00.849139+0000","last_change":"2024-04-20T11:29:00.844286+0000","last_active":"2024-04-20T11:29:00.849139+0000","last_peered":"2024-04-20T11:29:00.849139+0000","last_clean":"2024-04-20T11:29:00.849139+0000","last_became_active":"2024-04-20T11:29:00.844024+0000","last_became_peered":"2024-04-20T11:29:00.844024+0000","last_unstale":"2024-04-20T11:29:00.849139+0000","last_undegraded":"2024-04-20T11:29:00.849139+0000","last_fullsized":"2024-04-20T11:29:00.849139+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:05:12.507611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-20T11:29:01.824063+0000","last_change":"2024-04-20T11:29:00.861729+0000","last_active":"2024-04-20T11:29:01.824063+0000","last_peered":"2024-04-20T11:29:01.824063+0000","last_clean":"2024-04-20T11:29:01.824063+0000","last_became_active":"2024-04-20T11:29:00.861432+0000","last_became_peered":"2024-04-20T11:29:00.861432+0000","last_unstale":"2024-04-20T11:29:01.824063+0000","last_undegraded":"2024-04-20T11:29:01.824063+0000","last_fullsized":"2024-04-20T11:29:01.824063+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:44:23.956892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-20T11:29:01.821547+0000","last_change":"2024-04-20T11:29:00.860956+0000","last_active":"2024-04-20T11:29:01.821547+0000","last_peered":"2024-04-20T11:29:01.821547+0000","last_clean":"2024-04-20T11:29:01.821547+0000","last_became_active":"2024-04-20T11:29:00.860684+0000","last_became_peered":"2024-04-20T11:29:00.860684+0000","last_unstale":"2024-04-20T11:29:01.821547+0000","last_undegraded":"2024-04-20T11:29:01.821547+0000","last_fullsized":"2024-04-20T11:29:01.821547+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:44:13.751283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T11:29:00.847059+0000","last_change":"2024-04-20T11:29:00.837370+0000","last_active":"2024-04-20T11:29:00.847059+0000","last_peered":"2024-04-20T11:29:00.847059+0000","last_clean":"2024-04-20T11:29:00.847059+0000","last_became_active":"2024-04-20T11:29:00.837105+0000","last_became_peered":"2024-04-20T11:29:00.837105+0000","last_unstale":"2024-04-20T11:29:00.847059+0000","last_undegraded":"2024-04-20T11:29:00.847059+0000","last_fullsized":"2024-04-20T11:29:00.847059+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:32:57.103501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T11:29:00.859421+0000","last_change":"2024-04-20T11:29:00.852622+0000","last_active":"2024-04-20T11:29:00.859421+0000","last_peered":"2024-04-20T11:29:00.859421+0000","last_clean":"2024-04-20T11:29:00.859421+0000","last_became_active":"2024-04-20T11:29:00.852322+0000","last_became_peered":"2024-04-20T11:29:00.852322+0000","last_unstale":"2024-04-20T11:29:00.859421+0000","last_undegraded":"2024-04-20T11:29:00.859421+0000","last_fullsized":"2024-04-20T11:29:00.859421+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:23:39.579200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-20T11:29:02.173172+0000","last_change":"2024-04-20T11:29:00.848889+0000","last_active":"2024-04-20T11:29:02.173172+0000","last_peered":"2024-04-20T11:29:02.173172+0000","last_clean":"2024-04-20T11:29:02.173172+0000","last_became_active":"2024-04-20T11:29:00.848614+0000","last_became_peered":"2024-04-20T11:29:00.848614+0000","last_unstale":"2024-04-20T11:29:02.173172+0000","last_undegraded":"2024-04-20T11:29:02.173172+0000","last_fullsized":"2024-04-20T11:29:02.173172+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:10:01.731112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-20T11:29:01.824886+0000","last_change":"2024-04-20T11:29:00.850950+0000","last_active":"2024-04-20T11:29:01.824886+0000","last_peered":"2024-04-20T11:29:01.824886+0000","last_clean":"2024-04-20T11:29:01.824886+0000","last_became_active":"2024-04-20T11:29:00.850638+0000","last_became_peered":"2024-04-20T11:29:00.850638+0000","last_unstale":"2024-04-20T11:29:01.824886+0000","last_undegraded":"2024-04-20T11:29:01.824886+0000","last_fullsized":"2024-04-20T11:29:01.824886+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:57:55.715567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-20T11:29:01.823495+0000","last_change":"2024-04-20T11:29:00.853642+0000","last_active":"2024-04-20T11:29:01.823495+0000","last_peered":"2024-04-20T11:29:01.823495+0000","last_clean":"2024-04-20T11:29:01.823495+0000","last_became_active":"2024-04-20T11:29:00.853084+0000","last_became_peered":"2024-04-20T11:29:00.853084+0000","last_unstale":"2024-04-20T11:29:01.823495+0000","last_undegraded":"2024-04-20T11:29:01.823495+0000","last_fullsized":"2024-04-20T11:29:01.823495+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:38:53.689879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-20T11:29:02.174069+0000","last_change":"2024-04-20T11:29:00.839761+0000","last_active":"2024-04-20T11:29:02.174069+0000","last_peered":"2024-04-20T11:29:02.174069+0000","last_clean":"2024-04-20T11:29:02.174069+0000","last_became_active":"2024-04-20T11:29:00.839145+0000","last_became_peered":"2024-04-20T11:29:00.839145+0000","last_unstale":"2024-04-20T11:29:02.174069+0000","last_undegraded":"2024-04-20T11:29:02.174069+0000","last_fullsized":"2024-04-20T11:29:02.174069+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:03:36.963877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-20T11:29:01.821651+0000","last_change":"2024-04-20T11:29:00.859987+0000","last_active":"2024-04-20T11:29:01.821651+0000","last_peered":"2024-04-20T11:29:01.821651+0000","last_clean":"2024-04-20T11:29:01.821651+0000","last_became_active":"2024-04-20T11:29:00.859721+0000","last_became_peered":"2024-04-20T11:29:00.859721+0000","last_unstale":"2024-04-20T11:29:01.821651+0000","last_undegraded":"2024-04-20T11:29:01.821651+0000","last_fullsized":"2024-04-20T11:29:01.821651+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:20:11.399706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T11:29:00.847174+0000","last_change":"2024-04-20T11:29:00.841913+0000","last_active":"2024-04-20T11:29:00.847174+0000","last_peered":"2024-04-20T11:29:00.847174+0000","last_clean":"2024-04-20T11:29:00.847174+0000","last_became_active":"2024-04-20T11:29:00.841459+0000","last_became_peered":"2024-04-20T11:29:00.841459+0000","last_unstale":"2024-04-20T11:29:00.847174+0000","last_undegraded":"2024-04-20T11:29:00.847174+0000","last_fullsized":"2024-04-20T11:29:00.847174+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:34:13.826288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T11:29:00.844490+0000","last_change":"2024-04-20T11:29:00.833387+0000","last_active":"2024-04-20T11:29:00.844490+0000","last_peered":"2024-04-20T11:29:00.844490+0000","last_clean":"2024-04-20T11:29:00.844490+0000","last_became_active":"2024-04-20T11:29:00.832985+0000","last_became_peered":"2024-04-20T11:29:00.832985+0000","last_unstale":"2024-04-20T11:29:00.844490+0000","last_undegraded":"2024-04-20T11:29:00.844490+0000","last_fullsized":"2024-04-20T11:29:00.844490+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:44:13.751283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-20T11:29:02.173420+0000","last_change":"2024-04-20T11:29:00.844262+0000","last_active":"2024-04-20T11:29:02.173420+0000","last_peered":"2024-04-20T11:29:02.173420+0000","last_clean":"2024-04-20T11:29:02.173420+0000","last_became_active":"2024-04-20T11:29:00.843925+0000","last_became_peered":"2024-04-20T11:29:00.843925+0000","last_unstale":"2024-04-20T11:29:02.173420+0000","last_undegraded":"2024-04-20T11:29:02.173420+0000","last_fullsized":"2024-04-20T11:29:02.173420+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:05:12.507611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-20T11:29:02.173262+0000","last_change":"2024-04-20T11:29:00.832131+0000","last_active":"2024-04-20T11:29:02.173262+0000","last_peered":"2024-04-20T11:29:02.173262+0000","last_clean":"2024-04-20T11:29:02.173262+0000","last_became_active":"2024-04-20T11:29:00.831796+0000","last_became_peered":"2024-04-20T11:29:00.831796+0000","last_unstale":"2024-04-20T11:29:02.173262+0000","last_undegraded":"2024-04-20T11:29:02.173262+0000","last_fullsized":"2024-04-20T11:29:02.173262+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:49:14.491396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-20T11:29:02.174046+0000","last_change":"2024-04-20T11:29:00.841038+0000","last_active":"2024-04-20T11:29:02.174046+0000","last_peered":"2024-04-20T11:29:02.174046+0000","last_clean":"2024-04-20T11:29:02.174046+0000","last_became_active":"2024-04-20T11:29:00.840726+0000","last_became_peered":"2024-04-20T11:29:00.840726+0000","last_unstale":"2024-04-20T11:29:02.174046+0000","last_undegraded":"2024-04-20T11:29:02.174046+0000","last_fullsized":"2024-04-20T11:29:02.174046+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:17:30.033031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-20T11:29:01.821383+0000","last_change":"2024-04-20T11:29:00.849447+0000","last_active":"2024-04-20T11:29:01.821383+0000","last_peered":"2024-04-20T11:29:01.821383+0000","last_clean":"2024-04-20T11:29:01.821383+0000","last_became_active":"2024-04-20T11:29:00.849135+0000","last_became_peered":"2024-04-20T11:29:00.849135+0000","last_unstale":"2024-04-20T11:29:01.821383+0000","last_undegraded":"2024-04-20T11:29:01.821383+0000","last_fullsized":"2024-04-20T11:29:01.821383+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:02:38.545869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-20T11:29:01.823957+0000","last_change":"2024-04-20T11:29:00.851916+0000","last_active":"2024-04-20T11:29:01.823957+0000","last_peered":"2024-04-20T11:29:01.823957+0000","last_clean":"2024-04-20T11:29:01.823957+0000","last_became_active":"2024-04-20T11:29:00.851685+0000","last_became_peered":"2024-04-20T11:29:00.851685+0000","last_unstale":"2024-04-20T11:29:01.823957+0000","last_undegraded":"2024-04-20T11:29:01.823957+0000","last_fullsized":"2024-04-20T11:29:01.823957+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:59:51.540993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-20T11:29:01.824495+0000","last_change":"2024-04-20T11:29:00.861714+0000","last_active":"2024-04-20T11:29:01.824495+0000","last_peered":"2024-04-20T11:29:01.824495+0000","last_clean":"2024-04-20T11:29:01.824495+0000","last_became_active":"2024-04-20T11:29:00.861379+0000","last_became_peered":"2024-04-20T11:29:00.861379+0000","last_unstale":"2024-04-20T11:29:01.824495+0000","last_undegraded":"2024-04-20T11:29:01.824495+0000","last_fullsized":"2024-04-20T11:29:01.824495+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:22:59.242819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-20T11:29:02.173263+0000","last_change":"2024-04-20T11:29:00.842444+0000","last_active":"2024-04-20T11:29:02.173263+0000","last_peered":"2024-04-20T11:29:02.173263+0000","last_clean":"2024-04-20T11:29:02.173263+0000","last_became_active":"2024-04-20T11:29:00.842189+0000","last_became_peered":"2024-04-20T11:29:00.842189+0000","last_unstale":"2024-04-20T11:29:02.173263+0000","last_undegraded":"2024-04-20T11:29:02.173263+0000","last_fullsized":"2024-04-20T11:29:02.173263+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:38:21.618746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-20T11:29:01.823000+0000","last_change":"2024-04-20T11:29:00.852871+0000","last_active":"2024-04-20T11:29:01.823000+0000","last_peered":"2024-04-20T11:29:01.823000+0000","last_clean":"2024-04-20T11:29:01.823000+0000","last_became_active":"2024-04-20T11:29:00.852605+0000","last_became_peered":"2024-04-20T11:29:00.852605+0000","last_unstale":"2024-04-20T11:29:01.823000+0000","last_undegraded":"2024-04-20T11:29:01.823000+0000","last_fullsized":"2024-04-20T11:29:01.823000+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:14:41.729790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T11:29:00.834303+0000","last_change":"2024-04-20T11:29:00.828342+0000","last_active":"2024-04-20T11:29:00.834303+0000","last_peered":"2024-04-20T11:29:00.834303+0000","last_clean":"2024-04-20T11:29:00.834303+0000","last_became_active":"2024-04-20T11:29:00.828065+0000","last_became_peered":"2024-04-20T11:29:00.828065+0000","last_unstale":"2024-04-20T11:29:00.834303+0000","last_undegraded":"2024-04-20T11:29:00.834303+0000","last_fullsized":"2024-04-20T11:29:00.834303+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:26:46.408356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-20T11:29:01.820898+0000","last_change":"2024-04-20T11:29:00.851890+0000","last_active":"2024-04-20T11:29:01.820898+0000","last_peered":"2024-04-20T11:29:01.820898+0000","last_clean":"2024-04-20T11:29:01.820898+0000","last_became_active":"2024-04-20T11:29:00.851562+0000","last_became_peered":"2024-04-20T11:29:00.851562+0000","last_unstale":"2024-04-20T11:29:01.820898+0000","last_undegraded":"2024-04-20T11:29:01.820898+0000","last_fullsized":"2024-04-20T11:29:01.820898+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:53:16.444204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T11:29:00.854998+0000","last_change":"2024-04-20T11:29:00.850498+0000","last_active":"2024-04-20T11:29:00.854998+0000","last_peered":"2024-04-20T11:29:00.854998+0000","last_clean":"2024-04-20T11:29:00.854998+0000","last_became_active":"2024-04-20T11:29:00.850210+0000","last_became_peered":"2024-04-20T11:29:00.850210+0000","last_unstale":"2024-04-20T11:29:00.854998+0000","last_undegraded":"2024-04-20T11:29:00.854998+0000","last_fullsized":"2024-04-20T11:29:00.854998+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:31:39.636876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"45'4","reported_seq":16,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T11:29:00.862974+0000","last_change":"2024-04-20T11:28:59.838071+0000","last_active":"2024-04-20T11:29:00.862974+0000","last_peered":"2024-04-20T11:29:00.862974+0000","last_clean":"2024-04-20T11:29:00.862974+0000","last_became_active":"2024-04-20T11:28:59.836554+0000","last_became_peered":"2024-04-20T11:28:59.836554+0000","last_unstale":"2024-04-20T11:29:00.862974+0000","last_undegraded":"2024-04-20T11:29:00.862974+0000","last_fullsized":"2024-04-20T11:29:00.862974+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_clean_scrub_stamp":"2024-04-20T11:28:58.778154+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:57:54.699702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-20T11:29:02.174085+0000","last_change":"2024-04-20T11:29:00.847627+0000","last_active":"2024-04-20T11:29:02.174085+0000","last_peered":"2024-04-20T11:29:02.174085+0000","last_clean":"2024-04-20T11:29:02.174085+0000","last_became_active":"2024-04-20T11:29:00.847255+0000","last_became_peered":"2024-04-20T11:29:00.847255+0000","last_unstale":"2024-04-20T11:29:02.174085+0000","last_undegraded":"2024-04-20T11:29:02.174085+0000","last_fullsized":"2024-04-20T11:29:02.174085+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:45:37.576931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"45'1","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-20T11:29:01.824413+0000","last_change":"2024-04-20T11:28:59.847780+0000","last_active":"2024-04-20T11:29:01.824413+0000","last_peered":"2024-04-20T11:29:01.824413+0000","last_clean":"2024-04-20T11:29:01.824413+0000","last_became_active":"2024-04-20T11:28:59.847094+0000","last_became_peered":"2024-04-20T11:28:59.847094+0000","last_unstale":"2024-04-20T11:29:01.824413+0000","last_undegraded":"2024-04-20T11:29:01.824413+0000","last_fullsized":"2024-04-20T11:29:01.824413+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_clean_scrub_stamp":"2024-04-20T11:28:58.778154+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:38:20.602881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-20T11:29:01.824874+0000","last_change":"2024-04-20T11:29:00.834021+0000","last_active":"2024-04-20T11:29:01.824874+0000","last_peered":"2024-04-20T11:29:01.824874+0000","last_clean":"2024-04-20T11:29:01.824874+0000","last_became_active":"2024-04-20T11:29:00.833678+0000","last_became_peered":"2024-04-20T11:29:00.833678+0000","last_unstale":"2024-04-20T11:29:01.824874+0000","last_undegraded":"2024-04-20T11:29:01.824874+0000","last_fullsized":"2024-04-20T11:29:01.824874+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:26:46.408356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"45'2","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T11:29:00.862388+0000","last_change":"2024-04-20T11:28:59.843487+0000","last_active":"2024-04-20T11:29:00.862388+0000","last_peered":"2024-04-20T11:29:00.862388+0000","last_clean":"2024-04-20T11:29:00.862388+0000","last_became_active":"2024-04-20T11:28:59.843178+0000","last_became_peered":"2024-04-20T11:28:59.843178+0000","last_unstale":"2024-04-20T11:29:00.862388+0000","last_undegraded":"2024-04-20T11:29:00.862388+0000","last_fullsized":"2024-04-20T11:29:00.862388+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_clean_scrub_stamp":"2024-04-20T11:28:58.778154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:32:56.087636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T11:29:00.856174+0000","last_change":"2024-04-20T11:29:00.847481+0000","last_active":"2024-04-20T11:29:00.856174+0000","last_peered":"2024-04-20T11:29:00.856174+0000","last_clean":"2024-04-20T11:29:00.856174+0000","last_became_active":"2024-04-20T11:29:00.846663+0000","last_became_peered":"2024-04-20T11:29:00.846663+0000","last_unstale":"2024-04-20T11:29:00.856174+0000","last_undegraded":"2024-04-20T11:29:00.856174+0000","last_fullsized":"2024-04-20T11:29:00.856174+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:49:14.491396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"45'2","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T11:29:00.844733+0000","last_change":"2024-04-20T11:28:59.849109+0000","last_active":"2024-04-20T11:29:00.844733+0000","last_peered":"2024-04-20T11:29:00.844733+0000","last_clean":"2024-04-20T11:29:00.844733+0000","last_became_active":"2024-04-20T11:28:59.848783+0000","last_became_peered":"2024-04-20T11:28:59.848783+0000","last_unstale":"2024-04-20T11:29:00.844733+0000","last_undegraded":"2024-04-20T11:29:00.844733+0000","last_fullsized":"2024-04-20T11:29:00.844733+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_clean_scrub_stamp":"2024-04-20T11:28:58.778154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:03:35.948012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-20T11:29:01.824898+0000","last_change":"2024-04-20T11:29:00.853745+0000","last_active":"2024-04-20T11:29:01.824898+0000","last_peered":"2024-04-20T11:29:01.824898+0000","last_clean":"2024-04-20T11:29:01.824898+0000","last_became_active":"2024-04-20T11:29:00.853376+0000","last_became_peered":"2024-04-20T11:29:00.853376+0000","last_unstale":"2024-04-20T11:29:01.824898+0000","last_undegraded":"2024-04-20T11:29:01.824898+0000","last_fullsized":"2024-04-20T11:29:01.824898+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:44:23.956892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"45'1","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T11:29:00.860760+0000","last_change":"2024-04-20T11:28:59.844224+0000","last_active":"2024-04-20T11:29:00.860760+0000","last_peered":"2024-04-20T11:29:00.860760+0000","last_clean":"2024-04-20T11:29:00.860760+0000","last_became_active":"2024-04-20T11:28:59.843984+0000","last_became_peered":"2024-04-20T11:28:59.843984+0000","last_unstale":"2024-04-20T11:29:00.860760+0000","last_undegraded":"2024-04-20T11:29:00.860760+0000","last_fullsized":"2024-04-20T11:29:00.860760+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_clean_scrub_stamp":"2024-04-20T11:28:58.778154+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:22:58.226954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T11:29:00.861899+0000","last_change":"2024-04-20T11:29:00.857690+0000","last_active":"2024-04-20T11:29:00.861899+0000","last_peered":"2024-04-20T11:29:00.861899+0000","last_clean":"2024-04-20T11:29:00.861899+0000","last_became_active":"2024-04-20T11:29:00.857368+0000","last_became_peered":"2024-04-20T11:29:00.857368+0000","last_unstale":"2024-04-20T11:29:00.861899+0000","last_undegraded":"2024-04-20T11:29:00.861899+0000","last_fullsized":"2024-04-20T11:29:00.861899+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:45:37.576931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"45'1","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-20T11:29:01.823300+0000","last_change":"2024-04-20T11:28:59.850482+0000","last_active":"2024-04-20T11:29:01.823300+0000","last_peered":"2024-04-20T11:29:01.823300+0000","last_clean":"2024-04-20T11:29:01.823300+0000","last_became_active":"2024-04-20T11:28:59.850188+0000","last_became_peered":"2024-04-20T11:28:59.850188+0000","last_unstale":"2024-04-20T11:29:01.823300+0000","last_undegraded":"2024-04-20T11:29:01.823300+0000","last_fullsized":"2024-04-20T11:29:01.823300+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_clean_scrub_stamp":"2024-04-20T11:28:58.778154+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:20:10.383841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T11:29:00.845253+0000","last_change":"2024-04-20T11:29:00.833875+0000","last_active":"2024-04-20T11:29:00.845253+0000","last_peered":"2024-04-20T11:29:00.845253+0000","last_clean":"2024-04-20T11:29:00.845253+0000","last_became_active":"2024-04-20T11:29:00.832895+0000","last_became_peered":"2024-04-20T11:29:00.832895+0000","last_unstale":"2024-04-20T11:29:00.845253+0000","last_undegraded":"2024-04-20T11:29:00.845253+0000","last_fullsized":"2024-04-20T11:29:00.845253+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:36:14.165535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-20T11:29:01.822001+0000","last_change":"2024-04-20T11:28:59.844437+0000","last_active":"2024-04-20T11:29:01.822001+0000","last_peered":"2024-04-20T11:29:01.822001+0000","last_clean":"2024-04-20T11:29:01.822001+0000","last_became_active":"2024-04-20T11:28:59.844118+0000","last_became_peered":"2024-04-20T11:28:59.844118+0000","last_unstale":"2024-04-20T11:29:01.822001+0000","last_undegraded":"2024-04-20T11:29:01.822001+0000","last_fullsized":"2024-04-20T11:29:01.822001+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_clean_scrub_stamp":"2024-04-20T11:28:58.778154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:05:11.491745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-20T11:29:02.173203+0000","last_change":"2024-04-20T11:29:00.838698+0000","last_active":"2024-04-20T11:29:02.173203+0000","last_peered":"2024-04-20T11:29:02.173203+0000","last_clean":"2024-04-20T11:29:02.173203+0000","last_became_active":"2024-04-20T11:29:00.838408+0000","last_became_peered":"2024-04-20T11:29:00.838408+0000","last_unstale":"2024-04-20T11:29:02.173203+0000","last_undegraded":"2024-04-20T11:29:02.173203+0000","last_fullsized":"2024-04-20T11:29:02.173203+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:32:57.103501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T11:29:00.818974+0000","last_change":"2024-04-20T11:28:59.834793+0000","last_active":"2024-04-20T11:29:00.818974+0000","last_peered":"2024-04-20T11:29:00.818974+0000","last_clean":"2024-04-20T11:29:00.818974+0000","last_became_active":"2024-04-20T11:28:59.832265+0000","last_became_peered":"2024-04-20T11:28:59.832265+0000","last_unstale":"2024-04-20T11:29:00.818974+0000","last_undegraded":"2024-04-20T11:29:00.818974+0000","last_fullsized":"2024-04-20T11:29:00.818974+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_clean_scrub_stamp":"2024-04-20T11:28:58.778154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:13:54.057269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-20T11:29:01.823948+0000","last_change":"2024-04-20T11:29:00.853396+0000","last_active":"2024-04-20T11:29:01.823948+0000","last_peered":"2024-04-20T11:29:01.823948+0000","last_clean":"2024-04-20T11:29:01.823948+0000","last_became_active":"2024-04-20T11:29:00.852731+0000","last_became_peered":"2024-04-20T11:29:00.852731+0000","last_unstale":"2024-04-20T11:29:01.823948+0000","last_undegraded":"2024-04-20T11:29:01.823948+0000","last_fullsized":"2024-04-20T11:29:01.823948+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:45:21.173004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-20T11:29:01.824959+0000","last_change":"2024-04-20T11:28:59.842441+0000","last_active":"2024-04-20T11:29:01.824959+0000","last_peered":"2024-04-20T11:29:01.824959+0000","last_clean":"2024-04-20T11:29:01.824959+0000","last_became_active":"2024-04-20T11:28:59.842033+0000","last_became_peered":"2024-04-20T11:28:59.842033+0000","last_unstale":"2024-04-20T11:29:01.824959+0000","last_undegraded":"2024-04-20T11:29:01.824959+0000","last_fullsized":"2024-04-20T11:29:01.824959+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_clean_scrub_stamp":"2024-04-20T11:28:58.778154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:32:56.087636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T11:29:00.839278+0000","last_change":"2024-04-20T11:29:00.834705+0000","last_active":"2024-04-20T11:29:00.839278+0000","last_peered":"2024-04-20T11:29:00.839278+0000","last_clean":"2024-04-20T11:29:00.839278+0000","last_became_active":"2024-04-20T11:29:00.834411+0000","last_became_peered":"2024-04-20T11:29:00.834411+0000","last_unstale":"2024-04-20T11:29:00.839278+0000","last_undegraded":"2024-04-20T11:29:00.839278+0000","last_fullsized":"2024-04-20T11:29:00.839278+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:14:41.729790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T11:29:00.826291+0000","last_change":"2024-04-20T11:28:59.843411+0000","last_active":"2024-04-20T11:29:00.826291+0000","last_peered":"2024-04-20T11:29:00.826291+0000","last_clean":"2024-04-20T11:29:00.826291+0000","last_became_active":"2024-04-20T11:28:59.843107+0000","last_became_peered":"2024-04-20T11:28:59.843107+0000","last_unstale":"2024-04-20T11:29:00.826291+0000","last_undegraded":"2024-04-20T11:29:00.826291+0000","last_fullsized":"2024-04-20T11:29:00.826291+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_clean_scrub_stamp":"2024-04-20T11:28:58.778154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:02:37.530004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T11:29:00.860095+0000","last_change":"2024-04-20T11:29:00.854859+0000","last_active":"2024-04-20T11:29:00.860095+0000","last_peered":"2024-04-20T11:29:00.860095+0000","last_clean":"2024-04-20T11:29:00.860095+0000","last_became_active":"2024-04-20T11:29:00.854555+0000","last_became_peered":"2024-04-20T11:29:00.854555+0000","last_unstale":"2024-04-20T11:29:00.860095+0000","last_undegraded":"2024-04-20T11:29:00.860095+0000","last_fullsized":"2024-04-20T11:29:00.860095+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:22:04.188741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"45'3","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T11:29:00.850120+0000","last_change":"2024-04-20T11:28:59.849002+0000","last_active":"2024-04-20T11:29:00.850120+0000","last_peered":"2024-04-20T11:29:00.850120+0000","last_clean":"2024-04-20T11:29:00.850120+0000","last_became_active":"2024-04-20T11:28:59.848231+0000","last_became_peered":"2024-04-20T11:28:59.848231+0000","last_unstale":"2024-04-20T11:29:00.850120+0000","last_undegraded":"2024-04-20T11:29:00.850120+0000","last_fullsized":"2024-04-20T11:29:00.850120+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_clean_scrub_stamp":"2024-04-20T11:28:58.778154+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:47:30.010013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T11:29:00.850835+0000","last_change":"2024-04-20T11:29:00.842900+0000","last_active":"2024-04-20T11:29:00.850835+0000","last_peered":"2024-04-20T11:29:00.850835+0000","last_clean":"2024-04-20T11:29:00.850835+0000","last_became_active":"2024-04-20T11:29:00.842694+0000","last_became_peered":"2024-04-20T11:29:00.842694+0000","last_unstale":"2024-04-20T11:29:00.850835+0000","last_undegraded":"2024-04-20T11:29:00.850835+0000","last_fullsized":"2024-04-20T11:29:00.850835+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:45:21.173004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"45'6","reported_seq":20,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-20T11:29:01.821755+0000","last_change":"2024-04-20T11:28:59.842795+0000","last_active":"2024-04-20T11:29:01.821755+0000","last_peered":"2024-04-20T11:29:01.821755+0000","last_clean":"2024-04-20T11:29:01.821755+0000","last_became_active":"2024-04-20T11:28:59.842461+0000","last_became_peered":"2024-04-20T11:28:59.842461+0000","last_unstale":"2024-04-20T11:29:01.821755+0000","last_undegraded":"2024-04-20T11:29:01.821755+0000","last_fullsized":"2024-04-20T11:29:01.821755+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_clean_scrub_stamp":"2024-04-20T11:28:58.778154+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:05:22.464349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-20T11:29:02.173638+0000","last_change":"2024-04-20T11:29:00.837451+0000","last_active":"2024-04-20T11:29:02.173638+0000","last_peered":"2024-04-20T11:29:02.173638+0000","last_clean":"2024-04-20T11:29:02.173638+0000","last_became_active":"2024-04-20T11:29:00.837235+0000","last_became_peered":"2024-04-20T11:29:00.837235+0000","last_unstale":"2024-04-20T11:29:02.173638+0000","last_undegraded":"2024-04-20T11:29:02.173638+0000","last_fullsized":"2024-04-20T11:29:02.173638+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:26:31.816201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"45'2","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T11:29:00.862683+0000","last_change":"2024-04-20T11:28:59.829037+0000","last_active":"2024-04-20T11:29:00.862683+0000","last_peered":"2024-04-20T11:29:00.862683+0000","last_clean":"2024-04-20T11:29:00.862683+0000","last_became_active":"2024-04-20T11:28:59.828625+0000","last_became_peered":"2024-04-20T11:28:59.828625+0000","last_unstale":"2024-04-20T11:29:00.862683+0000","last_undegraded":"2024-04-20T11:29:00.862683+0000","last_fullsized":"2024-04-20T11:29:00.862683+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_clean_scrub_stamp":"2024-04-20T11:28:58.778154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:38:52.674014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T11:29:00.847768+0000","last_change":"2024-04-20T11:29:00.842669+0000","last_active":"2024-04-20T11:29:00.847768+0000","last_peered":"2024-04-20T11:29:00.847768+0000","last_clean":"2024-04-20T11:29:00.847768+0000","last_became_active":"2024-04-20T11:29:00.839643+0000","last_became_peered":"2024-04-20T11:29:00.839643+0000","last_unstale":"2024-04-20T11:29:00.847768+0000","last_undegraded":"2024-04-20T11:29:00.847768+0000","last_fullsized":"2024-04-20T11:29:00.847768+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:31:39.636876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T11:29:00.828266+0000","last_change":"2024-04-20T11:28:59.844639+0000","last_active":"2024-04-20T11:29:00.828266+0000","last_peered":"2024-04-20T11:29:00.828266+0000","last_clean":"2024-04-20T11:29:00.828266+0000","last_became_active":"2024-04-20T11:28:59.844347+0000","last_became_peered":"2024-04-20T11:28:59.844347+0000","last_unstale":"2024-04-20T11:29:00.828266+0000","last_undegraded":"2024-04-20T11:29:00.828266+0000","last_fullsized":"2024-04-20T11:29:00.828266+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_clean_scrub_stamp":"2024-04-20T11:28:58.778154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:17:29.017166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T11:29:00.846586+0000","last_change":"2024-04-20T11:29:00.839116+0000","last_active":"2024-04-20T11:29:00.846586+0000","last_peered":"2024-04-20T11:29:00.846586+0000","last_clean":"2024-04-20T11:29:00.846586+0000","last_became_active":"2024-04-20T11:29:00.838824+0000","last_became_peered":"2024-04-20T11:29:00.838824+0000","last_unstale":"2024-04-20T11:29:00.846586+0000","last_undegraded":"2024-04-20T11:29:00.846586+0000","last_fullsized":"2024-04-20T11:29:00.846586+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:25:49.258923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T11:29:00.823742+0000","last_change":"2024-04-20T11:28:59.834522+0000","last_active":"2024-04-20T11:29:00.823742+0000","last_peered":"2024-04-20T11:29:00.823742+0000","last_clean":"2024-04-20T11:29:00.823742+0000","last_became_active":"2024-04-20T11:28:59.834257+0000","last_became_peered":"2024-04-20T11:28:59.834257+0000","last_unstale":"2024-04-20T11:29:00.823742+0000","last_undegraded":"2024-04-20T11:29:00.823742+0000","last_fullsized":"2024-04-20T11:29:00.823742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_clean_scrub_stamp":"2024-04-20T11:28:58.778154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:44:22.941027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T11:29:00.850913+0000","last_change":"2024-04-20T11:29:00.844930+0000","last_active":"2024-04-20T11:29:00.850913+0000","last_peered":"2024-04-20T11:29:00.850913+0000","last_clean":"2024-04-20T11:29:00.850913+0000","last_became_active":"2024-04-20T11:29:00.844600+0000","last_became_peered":"2024-04-20T11:29:00.844600+0000","last_unstale":"2024-04-20T11:29:00.850913+0000","last_undegraded":"2024-04-20T11:29:00.850913+0000","last_fullsized":"2024-04-20T11:29:00.850913+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:59:51.540993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"45'8","reported_seq":20,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T11:29:00.850857+0000","last_change":"2024-04-20T11:28:59.837654+0000","last_active":"2024-04-20T11:29:00.850857+0000","last_peered":"2024-04-20T11:29:00.850857+0000","last_clean":"2024-04-20T11:29:00.850857+0000","last_became_active":"2024-04-20T11:28:59.837417+0000","last_became_peered":"2024-04-20T11:28:59.837417+0000","last_unstale":"2024-04-20T11:29:00.850857+0000","last_undegraded":"2024-04-20T11:29:00.850857+0000","last_fullsized":"2024-04-20T11:29:00.850857+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_clean_scrub_stamp":"2024-04-20T11:28:58.778154+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:53:15.428338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-20T11:29:01.821553+0000","last_change":"2024-04-20T11:29:00.854804+0000","last_active":"2024-04-20T11:29:01.821553+0000","last_peered":"2024-04-20T11:29:01.821553+0000","last_clean":"2024-04-20T11:29:01.821553+0000","last_became_active":"2024-04-20T11:29:00.853959+0000","last_became_peered":"2024-04-20T11:29:00.853959+0000","last_unstale":"2024-04-20T11:29:01.821553+0000","last_undegraded":"2024-04-20T11:29:01.821553+0000","last_fullsized":"2024-04-20T11:29:01.821553+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:36:14.165535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"45'1","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-20T11:29:01.821693+0000","last_change":"2024-04-20T11:28:59.834155+0000","last_active":"2024-04-20T11:29:01.821693+0000","last_peered":"2024-04-20T11:29:01.821693+0000","last_clean":"2024-04-20T11:29:01.821693+0000","last_became_active":"2024-04-20T11:28:59.833611+0000","last_became_peered":"2024-04-20T11:28:59.833611+0000","last_unstale":"2024-04-20T11:29:01.821693+0000","last_undegraded":"2024-04-20T11:29:01.821693+0000","last_fullsized":"2024-04-20T11:29:01.821693+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_clean_scrub_stamp":"2024-04-20T11:28:58.778154+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:33:53.368434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T11:29:00.847650+0000","last_change":"2024-04-20T11:29:00.839624+0000","last_active":"2024-04-20T11:29:00.847650+0000","last_peered":"2024-04-20T11:29:00.847650+0000","last_clean":"2024-04-20T11:29:00.847650+0000","last_became_active":"2024-04-20T11:29:00.837690+0000","last_became_peered":"2024-04-20T11:29:00.837690+0000","last_unstale":"2024-04-20T11:29:00.847650+0000","last_undegraded":"2024-04-20T11:29:00.847650+0000","last_fullsized":"2024-04-20T11:29:00.847650+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:49:14.491396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"45'2","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-20T11:29:01.824453+0000","last_change":"2024-04-20T11:28:59.846732+0000","last_active":"2024-04-20T11:29:01.824453+0000","last_peered":"2024-04-20T11:29:01.824453+0000","last_clean":"2024-04-20T11:29:01.824453+0000","last_became_active":"2024-04-20T11:28:59.845963+0000","last_became_peered":"2024-04-20T11:28:59.845963+0000","last_unstale":"2024-04-20T11:29:01.824453+0000","last_undegraded":"2024-04-20T11:29:01.824453+0000","last_fullsized":"2024-04-20T11:29:01.824453+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_clean_scrub_stamp":"2024-04-20T11:28:58.778154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:49:13.475530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-20T11:29:02.173373+0000","last_change":"2024-04-20T11:29:00.844061+0000","last_active":"2024-04-20T11:29:02.173373+0000","last_peered":"2024-04-20T11:29:02.173373+0000","last_clean":"2024-04-20T11:29:02.173373+0000","last_became_active":"2024-04-20T11:29:00.843746+0000","last_became_peered":"2024-04-20T11:29:00.843746+0000","last_unstale":"2024-04-20T11:29:02.173373+0000","last_undegraded":"2024-04-20T11:29:02.173373+0000","last_fullsized":"2024-04-20T11:29:02.173373+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:31:39.636876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"45'1","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-20T11:29:01.823512+0000","last_change":"2024-04-20T11:28:59.843578+0000","last_active":"2024-04-20T11:29:01.823512+0000","last_peered":"2024-04-20T11:29:01.823512+0000","last_clean":"2024-04-20T11:29:01.823512+0000","last_became_active":"2024-04-20T11:28:59.843276+0000","last_became_peered":"2024-04-20T11:28:59.843276+0000","last_unstale":"2024-04-20T11:29:01.823512+0000","last_undegraded":"2024-04-20T11:29:01.823512+0000","last_fullsized":"2024-04-20T11:29:01.823512+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_clean_scrub_stamp":"2024-04-20T11:28:58.778154+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:17:29.017166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-20T11:29:02.173916+0000","last_change":"2024-04-20T11:29:00.843375+0000","last_active":"2024-04-20T11:29:02.173916+0000","last_peered":"2024-04-20T11:29:02.173916+0000","last_clean":"2024-04-20T11:29:02.173916+0000","last_became_active":"2024-04-20T11:29:00.843023+0000","last_became_peered":"2024-04-20T11:29:00.843023+0000","last_unstale":"2024-04-20T11:29:02.173916+0000","last_undegraded":"2024-04-20T11:29:02.173916+0000","last_fullsized":"2024-04-20T11:29:02.173916+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:41:05.632073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T11:29:00.822220+0000","last_change":"2024-04-20T11:28:59.837530+0000","last_active":"2024-04-20T11:29:00.822220+0000","last_peered":"2024-04-20T11:29:00.822220+0000","last_clean":"2024-04-20T11:29:00.822220+0000","last_became_active":"2024-04-20T11:28:59.837317+0000","last_became_peered":"2024-04-20T11:28:59.837317+0000","last_unstale":"2024-04-20T11:29:00.822220+0000","last_undegraded":"2024-04-20T11:29:00.822220+0000","last_fullsized":"2024-04-20T11:29:00.822220+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_clean_scrub_stamp":"2024-04-20T11:28:58.778154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:14:40.713924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T11:29:00.859892+0000","last_change":"2024-04-20T11:29:00.855765+0000","last_active":"2024-04-20T11:29:00.859892+0000","last_peered":"2024-04-20T11:29:00.859892+0000","last_clean":"2024-04-20T11:29:00.859892+0000","last_became_active":"2024-04-20T11:29:00.855408+0000","last_became_peered":"2024-04-20T11:29:00.855408+0000","last_unstale":"2024-04-20T11:29:00.859892+0000","last_undegraded":"2024-04-20T11:29:00.859892+0000","last_fullsized":"2024-04-20T11:29:00.859892+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:38:53.536464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"45'2","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T11:29:00.860723+0000","last_change":"2024-04-20T11:28:59.844376+0000","last_active":"2024-04-20T11:29:00.860723+0000","last_peered":"2024-04-20T11:29:00.860723+0000","last_clean":"2024-04-20T11:29:00.860723+0000","last_became_active":"2024-04-20T11:28:59.843994+0000","last_became_peered":"2024-04-20T11:28:59.843994+0000","last_unstale":"2024-04-20T11:29:00.860723+0000","last_undegraded":"2024-04-20T11:29:00.860723+0000","last_fullsized":"2024-04-20T11:29:00.860723+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_clean_scrub_stamp":"2024-04-20T11:28:58.778154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:59:50.525128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T11:29:00.850401+0000","last_change":"2024-04-20T11:29:00.844159+0000","last_active":"2024-04-20T11:29:00.850401+0000","last_peered":"2024-04-20T11:29:00.850401+0000","last_clean":"2024-04-20T11:29:00.850401+0000","last_became_active":"2024-04-20T11:29:00.843884+0000","last_became_peered":"2024-04-20T11:29:00.843884+0000","last_unstale":"2024-04-20T11:29:00.850401+0000","last_undegraded":"2024-04-20T11:29:00.850401+0000","last_fullsized":"2024-04-20T11:29:00.850401+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:38:21.618746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-20T11:29:01.821951+0000","last_change":"2024-04-20T11:28:59.849414+0000","last_active":"2024-04-20T11:29:01.821951+0000","last_peered":"2024-04-20T11:29:01.821951+0000","last_clean":"2024-04-20T11:29:01.821951+0000","last_became_active":"2024-04-20T11:28:59.847528+0000","last_became_peered":"2024-04-20T11:28:59.847528+0000","last_unstale":"2024-04-20T11:29:01.821951+0000","last_undegraded":"2024-04-20T11:29:01.821951+0000","last_fullsized":"2024-04-20T11:29:01.821951+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_clean_scrub_stamp":"2024-04-20T11:28:58.778154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:41:04.616207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-20T11:29:01.821217+0000","last_change":"2024-04-20T11:29:00.850578+0000","last_active":"2024-04-20T11:29:01.821217+0000","last_peered":"2024-04-20T11:29:01.821217+0000","last_clean":"2024-04-20T11:29:01.821217+0000","last_became_active":"2024-04-20T11:29:00.848473+0000","last_became_peered":"2024-04-20T11:29:00.848473+0000","last_unstale":"2024-04-20T11:29:01.821217+0000","last_undegraded":"2024-04-20T11:29:01.821217+0000","last_fullsized":"2024-04-20T11:29:01.821217+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:25:14.723857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T11:29:00.818698+0000","last_change":"2024-04-20T11:28:59.846947+0000","last_active":"2024-04-20T11:29:00.818698+0000","last_peered":"2024-04-20T11:29:00.818698+0000","last_clean":"2024-04-20T11:29:00.818698+0000","last_became_active":"2024-04-20T11:28:59.846509+0000","last_became_peered":"2024-04-20T11:28:59.846509+0000","last_unstale":"2024-04-20T11:29:00.818698+0000","last_undegraded":"2024-04-20T11:29:00.818698+0000","last_fullsized":"2024-04-20T11:29:00.818698+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_clean_scrub_stamp":"2024-04-20T11:28:58.778154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:28:01.628720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T11:29:00.847193+0000","last_change":"2024-04-20T11:29:00.837320+0000","last_active":"2024-04-20T11:29:00.847193+0000","last_peered":"2024-04-20T11:29:00.847193+0000","last_clean":"2024-04-20T11:29:00.847193+0000","last_became_active":"2024-04-20T11:29:00.837034+0000","last_became_peered":"2024-04-20T11:29:00.837034+0000","last_unstale":"2024-04-20T11:29:00.847193+0000","last_undegraded":"2024-04-20T11:29:00.847193+0000","last_fullsized":"2024-04-20T11:29:00.847193+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:10:01.731112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T11:29:00.826873+0000","last_change":"2024-04-20T11:28:59.844566+0000","last_active":"2024-04-20T11:29:00.826873+0000","last_peered":"2024-04-20T11:29:00.826873+0000","last_clean":"2024-04-20T11:29:00.826873+0000","last_became_active":"2024-04-20T11:28:59.844253+0000","last_became_peered":"2024-04-20T11:28:59.844253+0000","last_unstale":"2024-04-20T11:29:00.826873+0000","last_undegraded":"2024-04-20T11:29:00.826873+0000","last_fullsized":"2024-04-20T11:29:00.826873+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_clean_scrub_stamp":"2024-04-20T11:28:58.778154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:41:04.616207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-20T11:29:02.174310+0000","last_change":"2024-04-20T11:29:00.848797+0000","last_active":"2024-04-20T11:29:02.174310+0000","last_peered":"2024-04-20T11:29:02.174310+0000","last_clean":"2024-04-20T11:29:02.174310+0000","last_became_active":"2024-04-20T11:29:00.848519+0000","last_became_peered":"2024-04-20T11:29:00.848519+0000","last_unstale":"2024-04-20T11:29:02.174310+0000","last_undegraded":"2024-04-20T11:29:02.174310+0000","last_fullsized":"2024-04-20T11:29:02.174310+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:47:31.025879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T11:29:00.823354+0000","last_change":"2024-04-20T11:28:59.850859+0000","last_active":"2024-04-20T11:29:00.823354+0000","last_peered":"2024-04-20T11:29:00.823354+0000","last_clean":"2024-04-20T11:29:00.823354+0000","last_became_active":"2024-04-20T11:28:59.850410+0000","last_became_peered":"2024-04-20T11:28:59.850410+0000","last_unstale":"2024-04-20T11:29:00.823354+0000","last_undegraded":"2024-04-20T11:29:00.823354+0000","last_fullsized":"2024-04-20T11:29:00.823354+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_clean_scrub_stamp":"2024-04-20T11:28:58.778154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:45:20.157138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T11:29:00.861173+0000","last_change":"2024-04-20T11:29:00.854443+0000","last_active":"2024-04-20T11:29:00.861173+0000","last_peered":"2024-04-20T11:29:00.861173+0000","last_clean":"2024-04-20T11:29:00.861173+0000","last_became_active":"2024-04-20T11:29:00.853615+0000","last_became_peered":"2024-04-20T11:29:00.853615+0000","last_unstale":"2024-04-20T11:29:00.861173+0000","last_undegraded":"2024-04-20T11:29:00.861173+0000","last_fullsized":"2024-04-20T11:29:00.861173+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:03:36.963877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T11:29:00.832105+0000","last_change":"2024-04-20T11:28:59.849272+0000","last_active":"2024-04-20T11:29:00.832105+0000","last_peered":"2024-04-20T11:29:00.832105+0000","last_clean":"2024-04-20T11:29:00.832105+0000","last_became_active":"2024-04-20T11:28:59.848656+0000","last_became_peered":"2024-04-20T11:28:59.848656+0000","last_unstale":"2024-04-20T11:29:00.832105+0000","last_undegraded":"2024-04-20T11:29:00.832105+0000","last_fullsized":"2024-04-20T11:29:00.832105+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_clean_scrub_stamp":"2024-04-20T11:28:58.778154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:35:46.785374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T11:29:00.865492+0000","last_change":"2024-04-20T11:29:00.861672+0000","last_active":"2024-04-20T11:29:00.865492+0000","last_peered":"2024-04-20T11:29:00.865492+0000","last_clean":"2024-04-20T11:29:00.865492+0000","last_became_active":"2024-04-20T11:29:00.861272+0000","last_became_peered":"2024-04-20T11:29:00.861272+0000","last_unstale":"2024-04-20T11:29:00.865492+0000","last_undegraded":"2024-04-20T11:29:00.865492+0000","last_fullsized":"2024-04-20T11:29:00.865492+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:44:23.956892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T11:29:00.830282+0000","last_change":"2024-04-20T11:28:59.845063+0000","last_active":"2024-04-20T11:29:00.830282+0000","last_peered":"2024-04-20T11:29:00.830282+0000","last_clean":"2024-04-20T11:29:00.830282+0000","last_became_active":"2024-04-20T11:28:59.844762+0000","last_became_peered":"2024-04-20T11:28:59.844762+0000","last_unstale":"2024-04-20T11:29:00.830282+0000","last_undegraded":"2024-04-20T11:29:00.830282+0000","last_fullsized":"2024-04-20T11:29:00.830282+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_clean_scrub_stamp":"2024-04-20T11:28:58.778154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:20:10.383841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T11:29:00.846567+0000","last_change":"2024-04-20T11:29:00.838933+0000","last_active":"2024-04-20T11:29:00.846567+0000","last_peered":"2024-04-20T11:29:00.846567+0000","last_clean":"2024-04-20T11:29:00.846567+0000","last_became_active":"2024-04-20T11:29:00.838651+0000","last_became_peered":"2024-04-20T11:29:00.838651+0000","last_unstale":"2024-04-20T11:29:00.846567+0000","last_undegraded":"2024-04-20T11:29:00.846567+0000","last_fullsized":"2024-04-20T11:29:00.846567+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:59:22.929015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T11:29:00.828804+0000","last_change":"2024-04-20T11:28:59.848630+0000","last_active":"2024-04-20T11:29:00.828804+0000","last_peered":"2024-04-20T11:29:00.828804+0000","last_clean":"2024-04-20T11:29:00.828804+0000","last_became_active":"2024-04-20T11:28:59.848339+0000","last_became_peered":"2024-04-20T11:28:59.848339+0000","last_unstale":"2024-04-20T11:29:00.828804+0000","last_undegraded":"2024-04-20T11:29:00.828804+0000","last_fullsized":"2024-04-20T11:29:00.828804+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_clean_scrub_stamp":"2024-04-20T11:28:58.778154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:45:36.561066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T11:29:00.859568+0000","last_change":"2024-04-20T11:29:00.855621+0000","last_active":"2024-04-20T11:29:00.859568+0000","last_peered":"2024-04-20T11:29:00.859568+0000","last_clean":"2024-04-20T11:29:00.859568+0000","last_became_active":"2024-04-20T11:29:00.855310+0000","last_became_peered":"2024-04-20T11:29:00.855310+0000","last_unstale":"2024-04-20T11:29:00.859568+0000","last_undegraded":"2024-04-20T11:29:00.859568+0000","last_fullsized":"2024-04-20T11:29:00.859568+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:07:52.578138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-20T11:29:01.823579+0000","last_change":"2024-04-20T11:28:59.845146+0000","last_active":"2024-04-20T11:29:01.823579+0000","last_peered":"2024-04-20T11:29:01.823579+0000","last_clean":"2024-04-20T11:29:01.823579+0000","last_became_active":"2024-04-20T11:28:59.844788+0000","last_became_peered":"2024-04-20T11:28:59.844788+0000","last_unstale":"2024-04-20T11:29:01.823579+0000","last_undegraded":"2024-04-20T11:29:01.823579+0000","last_fullsized":"2024-04-20T11:29:01.823579+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_clean_scrub_stamp":"2024-04-20T11:28:58.778154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:10:00.715247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-20T11:29:01.822179+0000","last_change":"2024-04-20T11:29:00.837423+0000","last_active":"2024-04-20T11:29:01.822179+0000","last_peered":"2024-04-20T11:29:01.822179+0000","last_clean":"2024-04-20T11:29:01.822179+0000","last_became_active":"2024-04-20T11:29:00.836988+0000","last_became_peered":"2024-04-20T11:29:00.836988+0000","last_unstale":"2024-04-20T11:29:01.822179+0000","last_undegraded":"2024-04-20T11:29:01.822179+0000","last_fullsized":"2024-04-20T11:29:01.822179+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:45:21.173004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T11:29:00.831355+0000","last_change":"2024-04-20T11:28:59.842194+0000","last_active":"2024-04-20T11:29:00.831355+0000","last_peered":"2024-04-20T11:29:00.831355+0000","last_clean":"2024-04-20T11:29:00.831355+0000","last_became_active":"2024-04-20T11:28:59.841860+0000","last_became_peered":"2024-04-20T11:28:59.841860+0000","last_unstale":"2024-04-20T11:29:00.831355+0000","last_undegraded":"2024-04-20T11:29:00.831355+0000","last_fullsized":"2024-04-20T11:29:00.831355+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_clean_scrub_stamp":"2024-04-20T11:28:58.778154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:10:00.715247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-20T11:29:01.822782+0000","last_change":"2024-04-20T11:29:00.850105+0000","last_active":"2024-04-20T11:29:01.822782+0000","last_peered":"2024-04-20T11:29:01.822782+0000","last_clean":"2024-04-20T11:29:01.822782+0000","last_became_active":"2024-04-20T11:29:00.849559+0000","last_became_peered":"2024-04-20T11:29:00.849559+0000","last_unstale":"2024-04-20T11:29:01.822782+0000","last_undegraded":"2024-04-20T11:29:01.822782+0000","last_fullsized":"2024-04-20T11:29:01.822782+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:35:47.801239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T11:29:00.819219+0000","last_change":"2024-04-20T11:28:59.825491+0000","last_active":"2024-04-20T11:29:00.819219+0000","last_peered":"2024-04-20T11:29:00.819219+0000","last_clean":"2024-04-20T11:29:00.819219+0000","last_became_active":"2024-04-20T11:28:59.824730+0000","last_became_peered":"2024-04-20T11:28:59.824730+0000","last_unstale":"2024-04-20T11:29:00.819219+0000","last_undegraded":"2024-04-20T11:29:00.819219+0000","last_fullsized":"2024-04-20T11:29:00.819219+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_clean_scrub_stamp":"2024-04-20T11:28:58.778154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:25:13.707992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T11:29:00.847931+0000","last_change":"2024-04-20T11:29:00.839067+0000","last_active":"2024-04-20T11:29:00.847931+0000","last_peered":"2024-04-20T11:29:00.847931+0000","last_clean":"2024-04-20T11:29:00.847931+0000","last_became_active":"2024-04-20T11:29:00.838807+0000","last_became_peered":"2024-04-20T11:29:00.838807+0000","last_unstale":"2024-04-20T11:29:00.847931+0000","last_undegraded":"2024-04-20T11:29:00.847931+0000","last_fullsized":"2024-04-20T11:29:00.847931+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:59.794019+0000","last_clean_scrub_stamp":"2024-04-20T11:28:59.794019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:38:21.618746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"45'2","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-20T11:29:01.824228+0000","last_change":"2024-04-20T11:28:59.839177+0000","last_active":"2024-04-20T11:29:01.824228+0000","last_peered":"2024-04-20T11:29:01.824228+0000","last_clean":"2024-04-20T11:29:01.824228+0000","last_became_active":"2024-04-20T11:28:59.838810+0000","last_became_peered":"2024-04-20T11:28:59.838810+0000","last_unstale":"2024-04-20T11:29:01.824228+0000","last_undegraded":"2024-04-20T11:29:01.824228+0000","last_fullsized":"2024-04-20T11:29:01.824228+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:28:58.778154+0000","last_clean_scrub_stamp":"2024-04-20T11:28:58.778154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:26:30.800335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":215,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T11:29:00.826931+0000","last_change":"2024-04-20T11:26:48.661897+0000","last_active":"2024-04-20T11:29:00.826931+0000","last_peered":"2024-04-20T11:29:00.826931+0000","last_clean":"2024-04-20T11:29:00.826931+0000","last_became_active":"2024-04-20T11:26:48.656535+0000","last_became_peered":"2024-04-20T11:26:48.656535+0000","last_unstale":"2024-04-20T11:29:00.826931+0000","last_undegraded":"2024-04-20T11:29:00.826931+0000","last_fullsized":"2024-04-20T11:29:00.826931+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:47.543004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:47.543004+0000","last_clean_scrub_stamp":"2024-04-20T11:26:47.543004+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:58:35.527105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574880,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28120,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343720,"statfs":{"total":96636764160,"available":96607969280,"internally_reserved":0,"allocated":1126400,"data_stored":805306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574877,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27548,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344292,"statfs":{"total":96636764160,"available":96608555008,"internally_reserved":0,"allocated":536576,"data_stored":215520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574879,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27556,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344284,"statfs":{"total":96636764160,"available":96608546816,"internally_reserved":0,"allocated":544768,"data_stored":221218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574877,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27548,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344292,"statfs":{"total":96636764160,"available":96608555008,"internally_reserved":0,"allocated":540672,"data_stored":216460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574878,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27548,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344292,"statfs":{"total":96636764160,"available":96608555008,"internally_reserved":0,"allocated":540672,"data_stored":215486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574879,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27572,"kb_used_data":544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344268,"statfs":{"total":96636764160,"available":96608530432,"internally_reserved":0,"allocated":557056,"data_stored":222782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574877,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27540,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344300,"statfs":{"total":96636764160,"available":96608563200,"internally_reserved":0,"allocated":532480,"data_stored":214918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574878,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28120,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343720,"statfs":{"total":96636764160,"available":96607969280,"internally_reserved":0,"allocated":1126400,"data_stored":805250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T11:29:05.701 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-20T11:29:05.701 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-20T11:29:05.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.952+0000 7f2e88a00640 1 Processor -- start 2024-04-20T11:29:05.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.952+0000 7f2e88a00640 1 -- start start 2024-04-20T11:29:05.954 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.954+0000 7f2e88a00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f2e840b4fb0 0x7f2e8415fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:05.954 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.954+0000 7f2e88a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f2e8406dc60 0x7f2e840b3670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:05.954 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.954+0000 7f2e88a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2e840b5390 0x7f2e840b3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:05.954 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.954+0000 7f2e88a00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f2e84160080 con 0x7f2e8406dc60 2024-04-20T11:29:05.954 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.954+0000 7f2e88a00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f2e8406ec70 con 0x7f2e840b4fb0 2024-04-20T11:29:05.955 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.954+0000 7f2e88a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2e8406fad0 con 0x7f2e840b5390 2024-04-20T11:29:05.955 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.954+0000 7f2e81400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f2e8406dc60 0x7f2e840b3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:05.955 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.954+0000 7f2e81400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f2e8406dc60 0x7f2e840b3670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:46218/0 (socket says 172.21.15.152:46218) 2024-04-20T11:29:05.955 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.954+0000 7f2e81400640 1 -- 172.21.15.152:0/3336359751 learned_addr learned my addr 172.21.15.152:0/3336359751 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:29:05.955 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.955+0000 7f2e82800640 1 --2- 172.21.15.152:0/3336359751 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2e840b5390 0x7f2e840b3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:05.955 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.955+0000 7f2e81e00640 1 --2- 172.21.15.152:0/3336359751 >> v2:172.21.15.175:3300/0 conn(0x7f2e840b4fb0 0x7f2e8415fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:05.955 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.955+0000 7f2e81400640 1 -- 172.21.15.152:0/3336359751 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2e840b5390 msgr2=0x7f2e840b3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:05.955 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.955+0000 7f2e81400640 1 --2- 172.21.15.152:0/3336359751 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2e840b5390 0x7f2e840b3bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:05.955 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.955+0000 7f2e81400640 1 -- 172.21.15.152:0/3336359751 >> v2:172.21.15.175:3300/0 conn(0x7f2e840b4fb0 msgr2=0x7f2e8415fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:05.955 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.955+0000 7f2e81400640 1 --2- 172.21.15.152:0/3336359751 >> v2:172.21.15.175:3300/0 conn(0x7f2e840b4fb0 0x7f2e8415fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:05.955 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.955+0000 7f2e81400640 1 -- 172.21.15.152:0/3336359751 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e8406e0f0 con 0x7f2e8406dc60 2024-04-20T11:29:05.955 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.955+0000 7f2e82800640 1 --2- 172.21.15.152:0/3336359751 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2e840b5390 0x7f2e840b3bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:29:05.956 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.956+0000 7f2e81400640 1 --2- 172.21.15.152:0/3336359751 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2e8406dc60 0x7f2e840b3670 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f2e84073560 tx=0x7f2e6c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb88ee0e476a0fd7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:05.956 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.956+0000 7f2e80a00640 1 -- 172.21.15.152:0/3336359751 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e6c02e070 con 0x7f2e8406dc60 2024-04-20T11:29:05.956 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.956+0000 7f2e80a00640 1 -- 172.21.15.152:0/3336359751 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e6c037070 con 0x7f2e8406dc60 2024-04-20T11:29:05.956 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.956+0000 7f2e80a00640 1 -- 172.21.15.152:0/3336359751 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e6c031a00 con 0x7f2e8406dc60 2024-04-20T11:29:05.957 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.957+0000 7f2e88a00640 1 -- 172.21.15.152:0/3336359751 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2e8406dc60 msgr2=0x7f2e840b3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:05.957 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.957+0000 7f2e88a00640 1 --2- 172.21.15.152:0/3336359751 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2e8406dc60 0x7f2e840b3670 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f2e84073560 tx=0x7f2e6c02b880 comp rx=0 tx=0).stop 2024-04-20T11:29:05.957 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.957+0000 7f2e88a00640 1 -- 172.21.15.152:0/3336359751 shutdown_connections 2024-04-20T11:29:05.957 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.957+0000 7f2e88a00640 1 --2- 172.21.15.152:0/3336359751 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2e840b5390 0x7f2e840b3bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:05.957 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.957+0000 7f2e88a00640 1 --2- 172.21.15.152:0/3336359751 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2e8406dc60 0x7f2e840b3670 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:05.957 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.957+0000 7f2e88a00640 1 --2- 172.21.15.152:0/3336359751 >> v2:172.21.15.175:3300/0 conn(0x7f2e840b4fb0 0x7f2e8415fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:05.957 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.957+0000 7f2e88a00640 1 -- 172.21.15.152:0/3336359751 >> 172.21.15.152:0/3336359751 conn(0x7f2e8406b980 msgr2=0x7f2e8406bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:05.957 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.957+0000 7f2e88a00640 1 -- 172.21.15.152:0/3336359751 shutdown_connections 2024-04-20T11:29:05.957 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.957+0000 7f2e88a00640 1 -- 172.21.15.152:0/3336359751 wait complete. 2024-04-20T11:29:05.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.959+0000 7f2e88a00640 1 Processor -- start 2024-04-20T11:29:05.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.959+0000 7f2e88a00640 1 -- start start 2024-04-20T11:29:05.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.960+0000 7f2e88a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2e840b4fb0 0x7f2e8414ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:05.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.960+0000 7f2e88a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2e840b5390 0x7f2e84160cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:05.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.960+0000 7f2e88a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2e840ad8b0 0x7f2e84156a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:05.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.960+0000 7f2e88a00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f2e8406e990 con 0x7f2e840ad8b0 2024-04-20T11:29:05.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.960+0000 7f2e88a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2e8406e6b0 con 0x7f2e840b4fb0 2024-04-20T11:29:05.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.960+0000 7f2e88a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2e840b40f0 con 0x7f2e840b5390 2024-04-20T11:29:05.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.960+0000 7f2e82800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2e840ad8b0 0x7f2e84156a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:05.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.960+0000 7f2e82800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2e840ad8b0 0x7f2e84156a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:46220/0 (socket says 172.21.15.152:46220) 2024-04-20T11:29:05.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.960+0000 7f2e82800640 1 -- 172.21.15.152:0/743200860 learned_addr learned my addr 172.21.15.152:0/743200860 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:29:05.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.960+0000 7f2e81e00640 1 --2- 172.21.15.152:0/743200860 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2e840b4fb0 0x7f2e8414ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:05.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.960+0000 7f2e81400640 1 --2- 172.21.15.152:0/743200860 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2e840b5390 0x7f2e84160cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:05.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.961+0000 7f2e82800640 1 -- 172.21.15.152:0/743200860 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2e840b5390 msgr2=0x7f2e84160cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:05.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.961+0000 7f2e82800640 1 --2- 172.21.15.152:0/743200860 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2e840b5390 0x7f2e84160cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:05.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.961+0000 7f2e82800640 1 -- 172.21.15.152:0/743200860 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2e840b4fb0 msgr2=0x7f2e8414ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:05.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.961+0000 7f2e82800640 1 --2- 172.21.15.152:0/743200860 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2e840b4fb0 0x7f2e8414ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:05.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.961+0000 7f2e82800640 1 -- 172.21.15.152:0/743200860 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e8414b1c0 con 0x7f2e840ad8b0 2024-04-20T11:29:05.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.961+0000 7f2e81e00640 1 --2- 172.21.15.152:0/743200860 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2e840b4fb0 0x7f2e8414ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:29:05.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.961+0000 7f2e81400640 1 --2- 172.21.15.152:0/743200860 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2e840b5390 0x7f2e84160cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:29:05.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.961+0000 7f2e82800640 1 --2- 172.21.15.152:0/743200860 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2e840ad8b0 0x7f2e84156a10 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c00c5e0 tx=0x7f2e7c00caa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:05.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.961+0000 7f2e7a000640 1 -- 172.21.15.152:0/743200860 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e7c015070 con 0x7f2e840ad8b0 2024-04-20T11:29:05.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.961+0000 7f2e7a000640 1 -- 172.21.15.152:0/743200860 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e7c004240 con 0x7f2e840ad8b0 2024-04-20T11:29:05.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.961+0000 7f2e7a000640 1 -- 172.21.15.152:0/743200860 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e7c003610 con 0x7f2e840ad8b0 2024-04-20T11:29:05.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.961+0000 7f2e88a00640 1 -- 172.21.15.152:0/743200860 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e84161520 con 0x7f2e840ad8b0 2024-04-20T11:29:05.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.961+0000 7f2e88a00640 1 -- 172.21.15.152:0/743200860 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e84161ae0 con 0x7f2e840ad8b0 2024-04-20T11:29:05.963 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.962+0000 7f2e88a00640 1 -- 172.21.15.152:0/743200860 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e840689f0 con 0x7f2e840ad8b0 2024-04-20T11:29:05.963 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.963+0000 7f2e7a000640 1 -- 172.21.15.152:0/743200860 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f2e7c0037b0 con 0x7f2e840ad8b0 2024-04-20T11:29:05.964 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.964+0000 7f2e7a000640 1 --2- 172.21.15.152:0/743200860 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f2e6006caa0 0x7f2e6006ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:05.964 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.964+0000 7f2e7a000640 1 -- 172.21.15.152:0/743200860 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f2e7c08c200 con 0x7f2e840ad8b0 2024-04-20T11:29:05.964 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.965+0000 7f2e81e00640 1 --2- 172.21.15.152:0/743200860 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f2e6006caa0 0x7f2e6006ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:05.964 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.965+0000 7f2e81e00640 1 --2- 172.21.15.152:0/743200860 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f2e6006caa0 0x7f2e6006ef60 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f2e74008f10 tx=0x7f2e74008460 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:05.968 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:05.969+0000 7f2e7a000640 1 -- 172.21.15.152:0/743200860 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f2e7c05a200 con 0x7f2e840ad8b0 2024-04-20T11:29:06.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:06.242+0000 7f2e88a00640 1 -- 172.21.15.152:0/743200860 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f2e840b24f0 con 0x7f2e840ad8b0 2024-04-20T11:29:06.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:06.246+0000 7f2e7a000640 1 -- 172.21.15.152:0/743200860 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v10) ==== 190+0+147 (secure 0 0 0) 0x7f2e7c059b00 con 0x7f2e840ad8b0 2024-04-20T11:29:06.246 INFO:teuthology.orchestra.run.smithi152.stdout:[client.0] 2024-04-20T11:29:06.246 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQBDpiNmtsBWFhAArq9A7r33CcwgMmOU2pAGPw== 2024-04-20T11:29:06.246 INFO:teuthology.orchestra.run.smithi152.stdout: caps mds = "allow" 2024-04-20T11:29:06.246 INFO:teuthology.orchestra.run.smithi152.stdout: caps mon = "allow r" 2024-04-20T11:29:06.246 INFO:teuthology.orchestra.run.smithi152.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-20T11:29:06.247 INFO:teuthology.orchestra.run.smithi152.stderr:updated caps for client.0 2024-04-20T11:29:06.249 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:06.249+0000 7f2e88a00640 1 -- 172.21.15.152:0/743200860 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f2e6006caa0 msgr2=0x7f2e6006ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:06.249 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:06.249+0000 7f2e88a00640 1 --2- 172.21.15.152:0/743200860 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f2e6006caa0 0x7f2e6006ef60 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f2e74008f10 tx=0x7f2e74008460 comp rx=0 tx=0).stop 2024-04-20T11:29:06.249 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:06.249+0000 7f2e88a00640 1 -- 172.21.15.152:0/743200860 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2e840ad8b0 msgr2=0x7f2e84156a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:06.249 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:06.249+0000 7f2e88a00640 1 --2- 172.21.15.152:0/743200860 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2e840ad8b0 0x7f2e84156a10 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c00c5e0 tx=0x7f2e7c00caa0 comp rx=0 tx=0).stop 2024-04-20T11:29:06.249 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:06.249+0000 7f2e88a00640 1 -- 172.21.15.152:0/743200860 shutdown_connections 2024-04-20T11:29:06.250 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:06.250+0000 7f2e88a00640 1 --2- 172.21.15.152:0/743200860 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f2e6006caa0 0x7f2e6006ef60 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:06.250 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:06.250+0000 7f2e88a00640 1 --2- 172.21.15.152:0/743200860 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2e840ad8b0 0x7f2e84156a10 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:06.250 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:06.250+0000 7f2e88a00640 1 --2- 172.21.15.152:0/743200860 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2e840b5390 0x7f2e84160cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:06.250 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:06.250+0000 7f2e88a00640 1 --2- 172.21.15.152:0/743200860 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2e840b4fb0 0x7f2e8414ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:06.250 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:06.250+0000 7f2e88a00640 1 -- 172.21.15.152:0/743200860 >> 172.21.15.152:0/743200860 conn(0x7f2e8406b980 msgr2=0x7f2e84155120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:06.250 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:06.250+0000 7f2e88a00640 1 -- 172.21.15.152:0/743200860 shutdown_connections 2024-04-20T11:29:06.250 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:06.250+0000 7f2e88a00640 1 -- 172.21.15.152:0/743200860 wait complete. 2024-04-20T11:29:06.268 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:29:06.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:06.460+0000 7faae2e00640 1 Processor -- start 2024-04-20T11:29:06.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:06.460+0000 7faae2e00640 1 -- start start 2024-04-20T11:29:06.462 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:06.461+0000 7faae2e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faadc15d1e0 0x7faadc160b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:06.462 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:06.461+0000 7faae2e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7faadc15d5c0 0x7faadc0cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:06.462 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:06.462+0000 7faae2e00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faadc0ce8c0 con 0x7faadc071cd0 2024-04-20T11:29:06.463 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:06.462+0000 7faae2e00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7faadc1600b0 con 0x7faadc15d5c0 2024-04-20T11:29:06.463 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:06.462+0000 7faae2e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7faadc160390 con 0x7faadc15d1e0 2024-04-20T11:29:06.463 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:06.462+0000 7faae1000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7faadc15d5c0 0x7faadc0cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:06.463 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:06.462+0000 7faae2400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7faadc071cd0 0x7faadc0ccd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:35732/0 (socket says 172.21.15.152:35732) 2024-04-20T11:29:06.463 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:06.462+0000 7faae1000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7faadc15d5c0 0x7faadc0cdfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:46232/0 (socket says 172.21.15.152:46232) 2024-04-20T11:29:06.463 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:06.462+0000 7faae2400640 1 -- 172.21.15.152:0/1331139626 learned_addr learned my addr 172.21.15.152:0/1331139626 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:29:06.463 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:06.462+0000 7faae1a00640 1 --2- 172.21.15.152:0/1331139626 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faadc15d1e0 0x7faadc160b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:06.463 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:06.462+0000 7faae1000640 1 -- 172.21.15.152:0/1331139626 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faadc15d1e0 msgr2=0x7faadc160b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:06.463 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:06.462+0000 7faae1000640 1 --2- 172.21.15.152:0/1331139626 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faadc15d1e0 0x7faadc160b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:06.463 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:06.463+0000 7faae1000640 1 -- 172.21.15.152:0/1331139626 >> v1:172.21.15.152:6789/0 conn(0x7faadc071cd0 legacy=0x7faadc0ccd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:06.464 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:06.463+0000 7faae1000640 1 -- 172.21.15.152:0/1331139626 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faadc160670 con 0x7faadc15d5c0 2024-04-20T11:29:06.464 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:06.463+0000 7faadbe00640 1 -- 172.21.15.152:0/1331139626 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4079545212 0 0) 0x7faadc0ce8c0 con 0x7faadc071cd0 2024-04-20T11:29:06.465 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:06.464+0000 7faae1000640 1 --2- 172.21.15.152:0/1331139626 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faadc15d5c0 0x7faadc0cdfe0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7faac80097a0 tx=0x7faac802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c43c1aeb7b39cc4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:06.465 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:06.464+0000 7faadbe00640 1 -- 172.21.15.152:0/1331139626 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac8035020 con 0x7faadc15d5c0 2024-04-20T11:29:06.465 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:06.464+0000 7faadbe00640 1 -- 172.21.15.152:0/1331139626 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faac8031810 con 0x7faadc15d5c0 2024-04-20T11:29:06.465 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:06.464+0000 7faadbe00640 1 -- 172.21.15.152:0/1331139626 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac8031be0 con 0x7faadc15d5c0 2024-04-20T11:29:06.465 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:06.465+0000 7faae2e00640 1 -- 172.21.15.152:0/1331139626 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faadc15d5c0 msgr2=0x7faadc0cdfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:06.465 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:06.465+0000 7faae2e00640 1 --2- 172.21.15.152:0/1331139626 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faadc15d5c0 0x7faadc0cdfe0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7faac80097a0 tx=0x7faac802b8b0 comp rx=0 tx=0).stop 2024-04-20T11:29:06.466 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:06.465+0000 7faae2e00640 1 -- 172.21.15.152:0/1331139626 shutdown_connections 2024-04-20T11:29:06.466 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:06.465+0000 7faae2e00640 1 --2- 172.21.15.152:0/1331139626 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faadc15d5c0 0x7faadc0cdfe0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:06.466 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:06.465+0000 7faae2e00640 1 --2- 172.21.15.152:0/1331139626 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faadc15d1e0 0x7faadc160b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:06.466 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:06.465+0000 7faae2e00640 1 -- 172.21.15.152:0/1331139626 >> 172.21.15.152:0/1331139626 conn(0x7faadc067ae0 msgr2=0x7faadc06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:06.466 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:06.465+0000 7faae2e00640 1 -- 172.21.15.152:0/1331139626 shutdown_connections 2024-04-20T11:29:06.466 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:06.465+0000 7faae2e00640 1 -- 172.21.15.152:0/1331139626 wait complete. 2024-04-20T11:29:06.466 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:06.466+0000 7faae2e00640 1 Processor -- start 2024-04-20T11:29:06.467 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:06.467+0000 7faae2e00640 1 -- start start 2024-04-20T11:29:06.468 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:06.468+0000 7faae2e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faadc071cd0 0x7faadc1f44e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:06.469 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:06.468+0000 7faae2e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faadc15d1e0 0x7faadc0ad8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:06.469 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:06.468+0000 7faae2e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faadc15d5c0 0x7faadc0addf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:06.469 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:06.468+0000 7faae2e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7faadc0ce6a0 con 0x7faadc071cd0 2024-04-20T11:29:06.469 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:06.468+0000 7faae2e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7faadc0ce520 con 0x7faadc15d1e0 2024-04-20T11:29:06.469 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:06.468+0000 7faae2e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7faadc072160 con 0x7faadc15d5c0 2024-04-20T11:29:06.469 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:06.468+0000 7faae1a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faadc071cd0 0x7faadc1f44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:06.469 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:06.468+0000 7faae1a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faadc071cd0 0x7faadc1f44e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:46240/0 (socket says 172.21.15.152:46240) 2024-04-20T11:29:06.469 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:06.468+0000 7faae1a00640 1 -- 172.21.15.152:0/1028849507 learned_addr learned my addr 172.21.15.152:0/1028849507 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:29:06.469 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:06.468+0000 7faae1a00640 1 -- 172.21.15.152:0/1028849507 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faadc15d5c0 msgr2=0x7faadc0addf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:06.469 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:06.468+0000 7faae1a00640 1 --2- 172.21.15.152:0/1028849507 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faadc15d5c0 0x7faadc0addf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:06.469 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:06.468+0000 7faae1a00640 1 -- 172.21.15.152:0/1028849507 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faadc15d1e0 msgr2=0x7faadc0ad8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:06.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:06.468+0000 7faae1a00640 1 --2- 172.21.15.152:0/1028849507 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faadc15d1e0 0x7faadc0ad8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:06.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:06.468+0000 7faae1a00640 1 -- 172.21.15.152:0/1028849507 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faadc0b3e40 con 0x7faadc071cd0 2024-04-20T11:29:06.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:06.469+0000 7faae1a00640 1 --2- 172.21.15.152:0/1028849507 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faadc071cd0 0x7faadc1f44e0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7faad000d830 tx=0x7faad000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:06.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:06.469+0000 7faad9600640 1 -- 172.21.15.152:0/1028849507 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faad000b6c0 con 0x7faadc071cd0 2024-04-20T11:29:06.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:06.469+0000 7faad9600640 1 -- 172.21.15.152:0/1028849507 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faad0019070 con 0x7faadc071cd0 2024-04-20T11:29:06.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:06.469+0000 7faad9600640 1 -- 172.21.15.152:0/1028849507 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faad000bac0 con 0x7faadc071cd0 2024-04-20T11:29:06.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:06.469+0000 7faae2e00640 1 -- 172.21.15.152:0/1028849507 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faadc0b4120 con 0x7faadc071cd0 2024-04-20T11:29:06.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:06.469+0000 7faae2e00640 1 -- 172.21.15.152:0/1028849507 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faadc0b4400 con 0x7faadc071cd0 2024-04-20T11:29:06.471 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:06.470+0000 7faae2e00640 1 -- 172.21.15.152:0/1028849507 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faadc075e60 con 0x7faadc071cd0 2024-04-20T11:29:06.471 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:06.471+0000 7faad9600640 1 -- 172.21.15.152:0/1028849507 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7faad001e460 con 0x7faadc071cd0 2024-04-20T11:29:06.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:06.472+0000 7faad9600640 1 --2- 172.21.15.152:0/1028849507 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7faabc06cad0 0x7faabc06ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:06.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:06.472+0000 7faad9600640 1 -- 172.21.15.152:0/1028849507 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7faad008c6a0 con 0x7faadc071cd0 2024-04-20T11:29:06.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:06.472+0000 7faae1000640 1 --2- 172.21.15.152:0/1028849507 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7faabc06cad0 0x7faabc06ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:06.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:06.473+0000 7faae1000640 1 --2- 172.21.15.152:0/1028849507 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7faabc06cad0 0x7faabc06ef90 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7faac8036000 tx=0x7faac8030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:06.476 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:06.477+0000 7faad9600640 1 -- 172.21.15.152:0/1028849507 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7faad005a720 con 0x7faadc071cd0 2024-04-20T11:29:06.713 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:06.714+0000 7faae2e00640 1 -- 172.21.15.152:0/1028849507 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faadc156190 con 0x7faadc071cd0 2024-04-20T11:29:06.715 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:06.715+0000 7faad9600640 1 -- 172.21.15.152:0/1028849507 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+5208 (secure 0 0 0) 0x7faad005a020 con 0x7faadc071cd0 2024-04-20T11:29:06.715 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:29:06.715 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4826,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6834/2855025590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2855025590},{"type":"v1","addr":"172.21.15.175:6834","nonce":2855025590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6836/3887716755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":3887716755},{"type":"v1","addr":"172.21.15.152:6836","nonce":3887716755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5712,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/698387399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6835","nonce":698387399},{"type":"v1","addr":"172.21.15.152:6837","nonce":698387399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T11:29:00.276881+0000","modified":"2024-04-20T11:29:03.352174+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5724},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5724":{"gid":5724,"name":"d","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6835/3704251089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6833","nonce":3704251089},{"type":"v1","addr":"172.21.15.175:6835","nonce":3704251089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5724,"qdb_cluster":[5724]},"id":3}]} 2024-04-20T11:29:06.715 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 24 2024-04-20T11:29:06.717 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:06.718+0000 7faae2e00640 1 -- 172.21.15.152:0/1028849507 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7faabc06cad0 msgr2=0x7faabc06ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:06.717 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:06.718+0000 7faae2e00640 1 --2- 172.21.15.152:0/1028849507 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7faabc06cad0 0x7faabc06ef90 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7faac8036000 tx=0x7faac8030000 comp rx=0 tx=0).stop 2024-04-20T11:29:06.718 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:06.718+0000 7faae2e00640 1 -- 172.21.15.152:0/1028849507 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faadc071cd0 msgr2=0x7faadc1f44e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:06.718 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:06.718+0000 7faae2e00640 1 --2- 172.21.15.152:0/1028849507 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faadc071cd0 0x7faadc1f44e0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7faad000d830 tx=0x7faad000dcf0 comp rx=0 tx=0).stop 2024-04-20T11:29:06.718 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:06.718+0000 7faae2e00640 1 -- 172.21.15.152:0/1028849507 shutdown_connections 2024-04-20T11:29:06.718 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:06.718+0000 7faae2e00640 1 --2- 172.21.15.152:0/1028849507 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7faabc06cad0 0x7faabc06ef90 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:06.718 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:06.718+0000 7faae2e00640 1 --2- 172.21.15.152:0/1028849507 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faadc15d5c0 0x7faadc0addf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:06.718 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:06.718+0000 7faae2e00640 1 --2- 172.21.15.152:0/1028849507 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faadc15d1e0 0x7faadc0ad8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:06.718 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:06.718+0000 7faae2e00640 1 --2- 172.21.15.152:0/1028849507 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faadc071cd0 0x7faadc1f44e0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:06.718 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:06.718+0000 7faae2e00640 1 -- 172.21.15.152:0/1028849507 >> 172.21.15.152:0/1028849507 conn(0x7faadc067ae0 msgr2=0x7faadc155530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:06.718 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:06.718+0000 7faae2e00640 1 -- 172.21.15.152:0/1028849507 shutdown_connections 2024-04-20T11:29:06.718 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:06.718+0000 7faae2e00640 1 -- 172.21.15.152:0/1028849507 wait complete. 2024-04-20T11:29:07.139 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-20T11:29:00.276881+0000', 'modified': '2024-04-20T11:29:03.352174+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'snap_rstat': True, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5724}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5724': {'gid': 5724, 'name': 'd', 'rank': 0, 'incarnation': 20, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.175:6835/3704251089', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6833', 'nonce': 3704251089}, {'type': 'v1', 'addr': '172.21.15.175:6835', 'nonce': 3704251089}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5724, 'qdb_cluster': [5724]} 2024-04-20T11:29:07.139 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-20T11:29:07.139 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-20T11:29:07.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.351+0000 7f55c6200640 1 Processor -- start 2024-04-20T11:29:07.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.351+0000 7f55c6200640 1 -- start start 2024-04-20T11:29:07.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.352+0000 7f55c6200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f55c00d0930 0x7f55c006de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:07.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.353+0000 7f55c6200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f55c00afcc0 0x7f55c015ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:07.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.353+0000 7f55c6200640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55c00af7e0 con 0x7f55c015c460 2024-04-20T11:29:07.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.353+0000 7f55c6200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f55c01624c0 con 0x7f55c00afcc0 2024-04-20T11:29:07.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.353+0000 7f55c6200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f55c0162710 con 0x7f55c00d0930 2024-04-20T11:29:07.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.353+0000 7f55bf400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f55c015c460 0x7f55c00d0220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:35736/0 (socket says 172.21.15.152:35736) 2024-04-20T11:29:07.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.353+0000 7f55bf400640 1 -- 172.21.15.152:0/2018020932 learned_addr learned my addr 172.21.15.152:0/2018020932 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:29:07.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.353+0000 7f55bea00640 1 -- 172.21.15.152:0/2018020932 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2337717340 0 0) 0x7f55c00af7e0 con 0x7f55c015c460 2024-04-20T11:29:07.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.353+0000 7f55b6a00640 1 --2- 172.21.15.152:0/2018020932 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f55c00d0930 0x7f55c006de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:07.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.353+0000 7f55bfe00640 1 --2- 172.21.15.152:0/2018020932 >> v2:172.21.15.175:3300/0 conn(0x7f55c00afcc0 0x7f55c015ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:07.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.353+0000 7f55bea00640 1 -- 172.21.15.152:0/2018020932 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f55a4003520 con 0x7f55c015c460 2024-04-20T11:29:07.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.354+0000 7f55bea00640 1 -- 172.21.15.152:0/2018020932 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2342671215 0 0) 0x7f55a4003520 con 0x7f55c015c460 2024-04-20T11:29:07.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.354+0000 7f55bea00640 1 -- 172.21.15.152:0/2018020932 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f55c00d0930 msgr2=0x7f55c006de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:07.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.354+0000 7f55bea00640 1 --2- 172.21.15.152:0/2018020932 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f55c00d0930 0x7f55c006de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:07.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.354+0000 7f55bea00640 1 -- 172.21.15.152:0/2018020932 >> v2:172.21.15.175:3300/0 conn(0x7f55c00afcc0 msgr2=0x7f55c015ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:07.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.354+0000 7f55bea00640 1 --2- 172.21.15.152:0/2018020932 >> v2:172.21.15.175:3300/0 conn(0x7f55c00afcc0 0x7f55c015ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:07.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.354+0000 7f55bea00640 1 -- 172.21.15.152:0/2018020932 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55c01629f0 con 0x7f55c015c460 2024-04-20T11:29:07.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.354+0000 7f55bea00640 1 -- 172.21.15.152:0/2018020932 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f55b0001670 con 0x7f55c015c460 2024-04-20T11:29:07.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.354+0000 7f55bea00640 1 -- 172.21.15.152:0/2018020932 >> v1:172.21.15.152:6789/0 conn(0x7f55c015c460 legacy=0x7f55c00d0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:07.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.354+0000 7f55bea00640 1 --2- 172.21.15.152:0/2018020932 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55a40039b0 0x7f55a4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:07.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.354+0000 7f55bea00640 1 --2- 172.21.15.152:0/2018020932 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f55a4004800 0x7f55a4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:07.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.354+0000 7f55bea00640 1 --2- 172.21.15.152:0/2018020932 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f55c00d0930 0x7f55a4005270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:07.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.354+0000 7f55bea00640 1 -- 172.21.15.152:0/2018020932 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f55c01624c0 con 0x7f55a4004800 2024-04-20T11:29:07.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.354+0000 7f55bea00640 1 -- 172.21.15.152:0/2018020932 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f55c0162710 con 0x7f55a40039b0 2024-04-20T11:29:07.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.354+0000 7f55bea00640 1 -- 172.21.15.152:0/2018020932 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f55a400bba0 con 0x7f55c00d0930 2024-04-20T11:29:07.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.354+0000 7f55b6a00640 1 --2- 172.21.15.152:0/2018020932 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f55a4004800 0x7f55a4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:07.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.354+0000 7f55bf400640 1 --2- 172.21.15.152:0/2018020932 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55a40039b0 0x7f55a4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:07.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.355+0000 7f55bfe00640 1 --2- 172.21.15.152:0/2018020932 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f55c00d0930 0x7f55a4005270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:07.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.355+0000 7f55b6a00640 1 -- 172.21.15.152:0/2018020932 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f55c00d0930 msgr2=0x7f55a4005270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:07.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.355+0000 7f55b6a00640 1 --2- 172.21.15.152:0/2018020932 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f55c00d0930 0x7f55a4005270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:07.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.355+0000 7f55b6a00640 1 -- 172.21.15.152:0/2018020932 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55a40039b0 msgr2=0x7f55a4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:07.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.355+0000 7f55b6a00640 1 --2- 172.21.15.152:0/2018020932 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55a40039b0 0x7f55a4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:07.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.355+0000 7f55b6a00640 1 -- 172.21.15.152:0/2018020932 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55a4003520 con 0x7f55a4004800 2024-04-20T11:29:07.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.356+0000 7f55b6a00640 1 --2- 172.21.15.152:0/2018020932 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f55a4004800 0x7f55a4004c80 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f55a800ba30 tx=0x7f55a4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b436e1902d4ca9ab server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:07.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.356+0000 7f55bea00640 1 -- 172.21.15.152:0/2018020932 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55a8036020 con 0x7f55a4004800 2024-04-20T11:29:07.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.356+0000 7f55bea00640 1 -- 172.21.15.152:0/2018020932 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55a8032640 con 0x7f55a4004800 2024-04-20T11:29:07.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.356+0000 7f55bea00640 1 -- 172.21.15.152:0/2018020932 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55a800b600 con 0x7f55a4004800 2024-04-20T11:29:07.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.357+0000 7f55c6200640 1 -- 172.21.15.152:0/2018020932 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f55a4004800 msgr2=0x7f55a4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:07.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.357+0000 7f55c6200640 1 --2- 172.21.15.152:0/2018020932 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f55a4004800 0x7f55a4004c80 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f55a800ba30 tx=0x7f55a4005830 comp rx=0 tx=0).stop 2024-04-20T11:29:07.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.357+0000 7f55bfe00640 1 -- 172.21.15.152:0/2018020932 reap_dead start 2024-04-20T11:29:07.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.357+0000 7f55c6200640 1 -- 172.21.15.152:0/2018020932 shutdown_connections 2024-04-20T11:29:07.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.357+0000 7f55c6200640 1 -- 172.21.15.152:0/2018020932 >> 172.21.15.152:0/2018020932 conn(0x7f55c0067cf0 msgr2=0x7f55c006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:07.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.357+0000 7f55c6200640 1 -- 172.21.15.152:0/2018020932 shutdown_connections 2024-04-20T11:29:07.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.357+0000 7f55c6200640 1 -- 172.21.15.152:0/2018020932 wait complete. 2024-04-20T11:29:07.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.359+0000 7f55c6200640 1 Processor -- start 2024-04-20T11:29:07.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.359+0000 7f55c6200640 1 -- start start 2024-04-20T11:29:07.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.360+0000 7f55c6200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f55a4004800 0x7f55c0161330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:07.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.360+0000 7f55c6200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55c0150100 0x7f55c01553f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:07.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.360+0000 7f55c6200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f55c0155930 0x7f55c014f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:07.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.360+0000 7f55c6200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f55c00af490 con 0x7f55a4004800 2024-04-20T11:29:07.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.360+0000 7f55c6200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f55c015c840 con 0x7f55c0150100 2024-04-20T11:29:07.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.360+0000 7f55c6200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f55c01631f0 con 0x7f55c0155930 2024-04-20T11:29:07.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.360+0000 7f55bf400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f55a4004800 0x7f55c0161330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:07.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.360+0000 7f55bf400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f55a4004800 0x7f55c0161330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:46266/0 (socket says 172.21.15.152:46266) 2024-04-20T11:29:07.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.360+0000 7f55bf400640 1 -- 172.21.15.152:0/2070449152 learned_addr learned my addr 172.21.15.152:0/2070449152 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:29:07.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.361+0000 7f55b6a00640 1 --2- 172.21.15.152:0/2070449152 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55c0150100 0x7f55c01553f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:07.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.361+0000 7f55bfe00640 1 --2- 172.21.15.152:0/2070449152 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f55c0155930 0x7f55c014f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:07.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.361+0000 7f55bf400640 1 -- 172.21.15.152:0/2070449152 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f55c0155930 msgr2=0x7f55c014f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:07.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.361+0000 7f55bf400640 1 --2- 172.21.15.152:0/2070449152 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f55c0155930 0x7f55c014f4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:07.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.361+0000 7f55bf400640 1 -- 172.21.15.152:0/2070449152 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55c0150100 msgr2=0x7f55c01553f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:07.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.361+0000 7f55bf400640 1 --2- 172.21.15.152:0/2070449152 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55c0150100 0x7f55c01553f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:07.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.361+0000 7f55bf400640 1 -- 172.21.15.152:0/2070449152 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55c01560b0 con 0x7f55a4004800 2024-04-20T11:29:07.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.361+0000 7f55bf400640 1 --2- 172.21.15.152:0/2070449152 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f55a4004800 0x7f55c0161330 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f55b000e780 tx=0x7f55b000ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:07.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.361+0000 7f55b7e00640 1 -- 172.21.15.152:0/2070449152 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b000ee90 con 0x7f55a4004800 2024-04-20T11:29:07.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.361+0000 7f55b7e00640 1 -- 172.21.15.152:0/2070449152 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55b0018070 con 0x7f55a4004800 2024-04-20T11:29:07.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.361+0000 7f55c6200640 1 -- 172.21.15.152:0/2070449152 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55c0155dd0 con 0x7f55a4004800 2024-04-20T11:29:07.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.361+0000 7f55b7e00640 1 -- 172.21.15.152:0/2070449152 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b000cb10 con 0x7f55a4004800 2024-04-20T11:29:07.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.361+0000 7f55c6200640 1 -- 172.21.15.152:0/2070449152 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55c015a120 con 0x7f55a4004800 2024-04-20T11:29:07.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.364+0000 7f55b7e00640 1 -- 172.21.15.152:0/2070449152 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f55b000ccb0 con 0x7f55a4004800 2024-04-20T11:29:07.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.365+0000 7f55b7e00640 1 --2- 172.21.15.152:0/2070449152 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f559c06cc40 0x7f559c06f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:07.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.365+0000 7f55b7e00640 1 -- 172.21.15.152:0/2070449152 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f55b008bec0 con 0x7f55a4004800 2024-04-20T11:29:07.366 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.365+0000 7f55b6a00640 1 --2- 172.21.15.152:0/2070449152 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f559c06cc40 0x7f559c06f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:07.366 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.365+0000 7f55c900a740 1 Processor -- start 2024-04-20T11:29:07.366 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.366+0000 7f55c900a740 1 -- start start 2024-04-20T11:29:07.366 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.366+0000 7f55b6a00640 1 --2- 172.21.15.152:0/2070449152 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f559c06cc40 0x7f559c06f100 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f55a800ba00 tx=0x7f55a8030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:07.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.367+0000 7f55c900a740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563a5d60e460 0x563a5d5c5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:07.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.367+0000 7f55c900a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563a5d5c44e0 0x563a5d5c5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:07.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.367+0000 7f55c900a740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x563a5d5c0930 0x563a5d5f0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:07.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.367+0000 7f55c900a740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x563a5d52c9d0 con 0x563a5d60e460 2024-04-20T11:29:07.369 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.367+0000 7f55c900a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x563a5d4d3270 con 0x563a5d5c44e0 2024-04-20T11:29:07.369 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.367+0000 7f55c900a740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x563a5d528990 con 0x563a5d5c0930 2024-04-20T11:29:07.369 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.367+0000 7f55bf400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563a5d60e460 0x563a5d5c5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:07.369 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.367+0000 7f55bf400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563a5d60e460 0x563a5d5c5a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:46268/0 (socket says 172.21.15.152:46268) 2024-04-20T11:29:07.369 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.367+0000 7f55bf400640 1 -- 172.21.15.152:0/2372514215 learned_addr learned my addr 172.21.15.152:0/2372514215 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:29:07.369 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.367+0000 7f55bf400640 1 -- 172.21.15.152:0/2372514215 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x563a5d5c0930 msgr2=0x563a5d5f0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:07.369 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.367+0000 7f55bf400640 1 --2- 172.21.15.152:0/2372514215 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x563a5d5c0930 0x563a5d5f0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:07.369 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.367+0000 7f55bf400640 1 -- 172.21.15.152:0/2372514215 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563a5d5c44e0 msgr2=0x563a5d5c5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:07.369 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.367+0000 7f55bf400640 1 --2- 172.21.15.152:0/2372514215 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563a5d5c44e0 0x563a5d5c5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:07.369 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.367+0000 7f55bf400640 1 -- 172.21.15.152:0/2372514215 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a5d23f2d0 con 0x563a5d60e460 2024-04-20T11:29:07.369 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.368+0000 7f55bf400640 1 --2- 172.21.15.152:0/2372514215 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563a5d60e460 0x563a5d5c5a90 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f55b008a1a0 tx=0x7f55b00594b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b9ad8877790ae1b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:07.369 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.368+0000 7f55af400640 1 -- 172.21.15.152:0/2372514215 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b00624b0 con 0x563a5d60e460 2024-04-20T11:29:07.370 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.368+0000 7f55af400640 1 -- 172.21.15.152:0/2372514215 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55b0055eb0 con 0x563a5d60e460 2024-04-20T11:29:07.370 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.368+0000 7f55af400640 1 -- 172.21.15.152:0/2372514215 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b008a4b0 con 0x563a5d60e460 2024-04-20T11:29:07.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.368+0000 7f55c900a740 1 -- 172.21.15.152:0/2372514215 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563a5d60e460 msgr2=0x563a5d5c5a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:07.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.368+0000 7f55c900a740 1 --2- 172.21.15.152:0/2372514215 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563a5d60e460 0x563a5d5c5a90 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f55b008a1a0 tx=0x7f55b00594b0 comp rx=0 tx=0).stop 2024-04-20T11:29:07.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.369+0000 7f55c900a740 1 -- 172.21.15.152:0/2372514215 shutdown_connections 2024-04-20T11:29:07.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.369+0000 7f55c900a740 1 --2- 172.21.15.152:0/2372514215 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x563a5d5c0930 0x563a5d5f0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:07.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.369+0000 7f55c900a740 1 --2- 172.21.15.152:0/2372514215 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563a5d5c44e0 0x563a5d5c5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:07.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.369+0000 7f55c900a740 1 --2- 172.21.15.152:0/2372514215 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563a5d60e460 0x563a5d5c5a90 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:07.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.369+0000 7f55c900a740 1 -- 172.21.15.152:0/2372514215 >> 172.21.15.152:0/2372514215 conn(0x563a5d5c2bf0 msgr2=0x563a5d60b770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:07.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.369+0000 7f55c900a740 1 -- 172.21.15.152:0/2372514215 shutdown_connections 2024-04-20T11:29:07.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.369+0000 7f55c900a740 1 -- 172.21.15.152:0/2372514215 wait complete. 2024-04-20T11:29:07.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.369+0000 7f55c900a740 1 -- start start 2024-04-20T11:29:07.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.370+0000 7f55c900a740 1 Processor -- start 2024-04-20T11:29:07.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.370+0000 7f55c900a740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x563a5d5c44e0 0x563a5d5dad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:07.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.371+0000 7f55c900a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563a5d60e460 0x563a5d6121c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:07.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.371+0000 7f55c900a740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563a5d5e9280 0x563a5d5ef350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:07.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.371+0000 7f55c900a740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x563a5d5c49f0 con 0x563a5d5e9280 2024-04-20T11:29:07.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.371+0000 7f55c900a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x563a5d4e65e0 con 0x563a5d60e460 2024-04-20T11:29:07.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.371+0000 7f55c900a740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x563a5d5c4cd0 con 0x563a5d5c44e0 2024-04-20T11:29:07.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.371+0000 7f55c900a740 10 client.0 _ll_register_callbacks cb 0x563a5d23eac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:29:07.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.371+0000 7f55bfe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563a5d5e9280 0x563a5d5ef350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:07.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.371+0000 7f55bfe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563a5d5e9280 0x563a5d5ef350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:46282/0 (socket says 172.21.15.152:46282) 2024-04-20T11:29:07.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.371+0000 7f55bfe00640 1 -- 172.21.15.152:0/3372741983 learned_addr learned my addr 172.21.15.152:0/3372741983 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:29:07.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.371+0000 7f55bf400640 1 --2- 172.21.15.152:0/3372741983 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x563a5d5c44e0 0x563a5d5dad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:07.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.371+0000 7f55b6a00640 1 --2- 172.21.15.152:0/3372741983 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563a5d60e460 0x563a5d6121c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:07.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.371+0000 7f55bfe00640 1 -- 172.21.15.152:0/3372741983 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x563a5d5c44e0 msgr2=0x563a5d5dad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:07.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.371+0000 7f55bfe00640 1 --2- 172.21.15.152:0/3372741983 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x563a5d5c44e0 0x563a5d5dad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:07.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.371+0000 7f55bfe00640 1 -- 172.21.15.152:0/3372741983 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563a5d60e460 msgr2=0x563a5d6121c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:07.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.371+0000 7f55bfe00640 1 --2- 172.21.15.152:0/3372741983 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563a5d60e460 0x563a5d6121c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:07.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.371+0000 7f55bfe00640 1 -- 172.21.15.152:0/3372741983 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a5d3096b0 con 0x563a5d5e9280 2024-04-20T11:29:07.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.371+0000 7f55bf400640 1 --2- 172.21.15.152:0/3372741983 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x563a5d5c44e0 0x563a5d5dad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:29:07.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.372+0000 7f55bfe00640 1 --2- 172.21.15.152:0/3372741983 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563a5d5e9280 0x563a5d5ef350 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f55c00d0fa0 tx=0x7f55b80099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=675631897a45607d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:07.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.372+0000 7f55a3e00640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-04-20T11:29:07.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.372+0000 7f55a3e00640 1 -- 172.21.15.152:0/3372741983 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b8007860 con 0x563a5d5e9280 2024-04-20T11:29:07.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.372+0000 7f55a3e00640 1 -- 172.21.15.152:0/3372741983 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f55b8007a00 con 0x563a5d5e9280 2024-04-20T11:29:07.377 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.372+0000 7f55c900a740 1 -- 172.21.15.152:0/3372741983 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55b0055eb0 con 0x563a5d5e9280 2024-04-20T11:29:07.377 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.372+0000 7f55a3e00640 1 -- 172.21.15.152:0/3372741983 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f55b8021050 con 0x563a5d5e9280 2024-04-20T11:29:07.377 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.372+0000 7f55c900a740 10 client.5850 fetch_fsmap learned FSMap version 24 2024-04-20T11:29:07.377 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.372+0000 7f55c900a740 1 -- 172.21.15.152:0/3372741983 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) -- 0x563a5d6133c0 con 0x563a5d5e9280 2024-04-20T11:29:07.377 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.373+0000 7f55a3e00640 1 -- 172.21.15.152:0/3372741983 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55b0055eb0 con 0x563a5d5e9280 2024-04-20T11:29:07.377 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.373+0000 7f55a3e00640 1 -- 172.21.15.152:0/3372741983 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 24) ==== 3495+0+0 (secure 0 0 0) 0x7f55b8024020 con 0x563a5d5e9280 2024-04-20T11:29:07.377 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.373+0000 7f55c900a740 10 client.5850 fetch_fsmap finished waiting for FSMap version 24 2024-04-20T11:29:07.377 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.373+0000 7f55c900a740 10 client.5850 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-20T11:29:07.377 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.373+0000 7f55c900a740 20 client.5850 populate_metadata read hostname 'smithi152' 2024-04-20T11:29:07.377 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.373+0000 7f55c900a740 1 --2- 172.21.15.152:0/3372741983 >> [v2:172.21.15.175:6833/3704251089,v1:172.21.15.175:6835/3704251089] conn(0x563a5d61c860 0x563a5d610690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:07.377 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.373+0000 7f55c900a740 4 client.5850 mds_command: new command op to 5724 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:29:07.377 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.373+0000 7f55c900a740 1 -- 172.21.15.152:0/3372741983 --> [v2:172.21.15.175:6833/3704251089,v1:172.21.15.175:6835/3704251089] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563a5d53a3f0 con 0x563a5d61c860 2024-04-20T11:29:07.378 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.374+0000 7f55bf400640 1 --2- 172.21.15.152:0/3372741983 >> [v2:172.21.15.175:6833/3704251089,v1:172.21.15.175:6835/3704251089] conn(0x563a5d61c860 0x563a5d610690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:07.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.374+0000 7f55bf400640 1 --2- 172.21.15.152:0/3372741983 >> [v2:172.21.15.175:6833/3704251089,v1:172.21.15.175:6835/3704251089] conn(0x563a5d61c860 0x563a5d610690 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=365a6bf87b641a62 server_cookie=a414fe932c5984fd in_seq=0 out_seq=0 2024-04-20T11:29:07.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.374+0000 7f55a3e00640 10 client.5850 ms_handle_connect on v2:172.21.15.175:6833/3704251089 2024-04-20T11:29:07.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.377+0000 7f55a3e00640 1 -- 172.21.15.152:0/3372741983 <== mds.0 v2:172.21.15.175:6833/3704251089 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563a5d53a3f0 con 0x563a5d61c860 2024-04-20T11:29:07.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.377+0000 7f55a3e00640 10 client.5850 handle_command_reply: tid=0 2024-04-20T11:29:07.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.377+0000 7f55c900a740 1 client.5850 shutdown 2024-04-20T11:29:07.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.377+0000 7f55c900a740 1 -- 172.21.15.152:0/3372741983 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563a5d5e9280 msgr2=0x563a5d5ef350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:07.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.377+0000 7f55c900a740 1 --2- 172.21.15.152:0/3372741983 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563a5d5e9280 0x563a5d5ef350 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f55c00d0fa0 tx=0x7f55b80099e0 comp rx=0 tx=0).stop 2024-04-20T11:29:07.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.377+0000 7f55c900a740 1 -- 172.21.15.152:0/3372741983 shutdown_connections 2024-04-20T11:29:07.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.377+0000 7f55c900a740 1 --2- 172.21.15.152:0/3372741983 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563a5d5e9280 0x563a5d5ef350 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:07.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.377+0000 7f55c900a740 1 --2- 172.21.15.152:0/3372741983 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563a5d60e460 0x563a5d6121c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:07.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.377+0000 7f55c900a740 1 --2- 172.21.15.152:0/3372741983 >> [v2:172.21.15.175:6833/3704251089,v1:172.21.15.175:6835/3704251089] conn(0x563a5d61c860 0x563a5d610690 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:07.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.377+0000 7f55c900a740 1 --2- 172.21.15.152:0/3372741983 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x563a5d5c44e0 0x563a5d5dad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:07.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.377+0000 7f55c900a740 1 -- 172.21.15.152:0/3372741983 >> 172.21.15.152:0/3372741983 conn(0x563a5d5c0930 msgr2=0x563a5d60b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:07.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.377+0000 7f55a3e00640 0 client.5850 ms_handle_reset on v2:172.21.15.175:6833/3704251089 2024-04-20T11:29:07.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.377+0000 7f55c900a740 1 -- 172.21.15.152:0/3372741983 shutdown_connections 2024-04-20T11:29:07.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.377+0000 7f55c900a740 1 -- 172.21.15.152:0/3372741983 wait complete. 2024-04-20T11:29:07.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.377+0000 7f55c900a740 20 client.5850 trim_cache size 0 max 16384 2024-04-20T11:29:07.397 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.397+0000 7f55c900a740 1 Processor -- start 2024-04-20T11:29:07.397 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.397+0000 7f55c900a740 1 -- start start 2024-04-20T11:29:07.398 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.398+0000 7f55c900a740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563a5d5c44e0 0x563a5d5bf850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:07.398 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.398+0000 7f55c900a740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x563a5d5e9280 0x563a5d5bfd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:07.398 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.398+0000 7f55c900a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563a5d60e460 0x563a5d5c5600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:07.399 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.398+0000 7f55c900a740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f55c0200450 con 0x563a5d5c44e0 2024-04-20T11:29:07.399 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.398+0000 7f55c900a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f55c015b1e0 con 0x563a5d60e460 2024-04-20T11:29:07.399 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.398+0000 7f55c900a740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f55c015b830 con 0x563a5d5e9280 2024-04-20T11:29:07.401 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.398+0000 7f55bf400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563a5d5c44e0 0x563a5d5bf850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:07.401 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.398+0000 7f55bf400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563a5d5c44e0 0x563a5d5bf850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:46294/0 (socket says 172.21.15.152:46294) 2024-04-20T11:29:07.401 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.398+0000 7f55bf400640 1 -- 172.21.15.152:0/883368372 learned_addr learned my addr 172.21.15.152:0/883368372 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:29:07.401 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.398+0000 7f55bfe00640 1 --2- 172.21.15.152:0/883368372 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563a5d60e460 0x563a5d5c5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:07.401 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.399+0000 7f55b6a00640 1 --2- 172.21.15.152:0/883368372 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x563a5d5e9280 0x563a5d5bfd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:07.401 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.399+0000 7f55bf400640 1 -- 172.21.15.152:0/883368372 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x563a5d5e9280 msgr2=0x563a5d5bfd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:07.401 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.399+0000 7f55bf400640 1 --2- 172.21.15.152:0/883368372 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x563a5d5e9280 0x563a5d5bfd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:07.401 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.399+0000 7f55bf400640 1 -- 172.21.15.152:0/883368372 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563a5d60e460 msgr2=0x563a5d5c5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:07.401 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.399+0000 7f55bf400640 1 --2- 172.21.15.152:0/883368372 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563a5d60e460 0x563a5d5c5600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:07.401 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.399+0000 7f55bf400640 1 -- 172.21.15.152:0/883368372 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a5d612850 con 0x563a5d5c44e0 2024-04-20T11:29:07.401 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.399+0000 7f55bf400640 1 --2- 172.21.15.152:0/883368372 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563a5d5c44e0 0x563a5d5bf850 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f55b0001c00 tx=0x7f55b001e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=145682ae25e126cf server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:07.402 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.399+0000 7f55aea00640 1 -- 172.21.15.152:0/883368372 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b0055eb0 con 0x563a5d5c44e0 2024-04-20T11:29:07.402 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.399+0000 7f55aea00640 1 -- 172.21.15.152:0/883368372 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55b0078070 con 0x563a5d5c44e0 2024-04-20T11:29:07.402 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.399+0000 7f55aea00640 1 -- 172.21.15.152:0/883368372 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b0062020 con 0x563a5d5c44e0 2024-04-20T11:29:07.402 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.400+0000 7f55c900a740 1 -- 172.21.15.152:0/883368372 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563a5d5c44e0 msgr2=0x563a5d5bf850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:07.402 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.400+0000 7f55c900a740 1 --2- 172.21.15.152:0/883368372 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563a5d5c44e0 0x563a5d5bf850 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f55b0001c00 tx=0x7f55b001e000 comp rx=0 tx=0).stop 2024-04-20T11:29:07.402 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.400+0000 7f55c900a740 1 -- 172.21.15.152:0/883368372 shutdown_connections 2024-04-20T11:29:07.402 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.400+0000 7f55c900a740 1 --2- 172.21.15.152:0/883368372 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563a5d60e460 0x563a5d5c5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:07.402 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.400+0000 7f55c900a740 1 --2- 172.21.15.152:0/883368372 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x563a5d5e9280 0x563a5d5bfd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:07.402 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.400+0000 7f55c900a740 1 --2- 172.21.15.152:0/883368372 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563a5d5c44e0 0x563a5d5bf850 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:07.402 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.400+0000 7f55c900a740 1 -- 172.21.15.152:0/883368372 >> 172.21.15.152:0/883368372 conn(0x563a5d5c0930 msgr2=0x563a5d6119d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:07.402 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.400+0000 7f55c900a740 1 -- 172.21.15.152:0/883368372 shutdown_connections 2024-04-20T11:29:07.402 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.400+0000 7f55c900a740 1 -- 172.21.15.152:0/883368372 wait complete. 2024-04-20T11:29:07.406 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.401+0000 7f55c900a740 1 -- start start 2024-04-20T11:29:07.406 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.401+0000 7f55c900a740 1 Processor -- start 2024-04-20T11:29:07.406 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.402+0000 7f55c900a740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x563a5d5c44e0 0x563a5d686fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:07.406 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.402+0000 7f55c900a740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563a5d5e9280 0x563a5d5e7c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:07.406 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.402+0000 7f55c900a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563a5d60e460 0x563a5d60e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:07.406 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.402+0000 7f55c900a740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x563a5d613ca0 con 0x563a5d5e9280 2024-04-20T11:29:07.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.402+0000 7f55c900a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x563a5d61d170 con 0x563a5d60e460 2024-04-20T11:29:07.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.402+0000 7f55c900a740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x563a5d61cce0 con 0x563a5d5c44e0 2024-04-20T11:29:07.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.402+0000 7f55c900a740 10 client.0 _ll_register_callbacks cb 0x7f55b8003a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:29:07.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.402+0000 7f55b6a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563a5d5e9280 0x563a5d5e7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:07.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.402+0000 7f55b6a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563a5d5e9280 0x563a5d5e7c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:46304/0 (socket says 172.21.15.152:46304) 2024-04-20T11:29:07.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.402+0000 7f55b6a00640 1 -- 172.21.15.152:0/16926762 learned_addr learned my addr 172.21.15.152:0/16926762 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:29:07.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.403+0000 7f55bf400640 1 --2- 172.21.15.152:0/16926762 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x563a5d5c44e0 0x563a5d686fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:07.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.403+0000 7f55bfe00640 1 --2- 172.21.15.152:0/16926762 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563a5d60e460 0x563a5d60e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:07.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.403+0000 7f55b6a00640 1 -- 172.21.15.152:0/16926762 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x563a5d5c44e0 msgr2=0x563a5d686fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:07.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.403+0000 7f55b6a00640 1 --2- 172.21.15.152:0/16926762 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x563a5d5c44e0 0x563a5d686fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:07.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.403+0000 7f55b6a00640 1 -- 172.21.15.152:0/16926762 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563a5d60e460 msgr2=0x563a5d60e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:07.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.403+0000 7f55b6a00640 1 --2- 172.21.15.152:0/16926762 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563a5d60e460 0x563a5d60e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:07.408 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.403+0000 7f55b6a00640 1 -- 172.21.15.152:0/16926762 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a5d610800 con 0x563a5d5e9280 2024-04-20T11:29:07.408 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.403+0000 7f55b6a00640 1 --2- 172.21.15.152:0/16926762 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563a5d5e9280 0x563a5d5e7c60 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f55a800bb50 tx=0x7f55a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7e5eb8d3001f72b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:07.408 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.403+0000 7f55a3e00640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-04-20T11:29:07.408 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.404+0000 7f55a3e00640 1 -- 172.21.15.152:0/16926762 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55a8054060 con 0x563a5d5e9280 2024-04-20T11:29:07.408 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.404+0000 7f55c900a740 1 -- 172.21.15.152:0/16926762 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55b0078070 con 0x563a5d5e9280 2024-04-20T11:29:07.408 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.404+0000 7f55a3e00640 1 -- 172.21.15.152:0/16926762 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f55a803baf0 con 0x563a5d5e9280 2024-04-20T11:29:07.408 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.404+0000 7f55a3e00640 1 -- 172.21.15.152:0/16926762 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f55a8036040 con 0x563a5d5e9280 2024-04-20T11:29:07.408 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.404+0000 7f55c900a740 10 client.5856 fetch_fsmap learned FSMap version 24 2024-04-20T11:29:07.408 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.404+0000 7f55c900a740 1 -- 172.21.15.152:0/16926762 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) -- 0x563a5d610b00 con 0x563a5d5e9280 2024-04-20T11:29:07.408 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.404+0000 7f55a3e00640 1 -- 172.21.15.152:0/16926762 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55b0078070 con 0x563a5d5e9280 2024-04-20T11:29:07.408 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.404+0000 7f55a3e00640 1 -- 172.21.15.152:0/16926762 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 24) ==== 3495+0+0 (secure 0 0 0) 0x7f55a804f440 con 0x563a5d5e9280 2024-04-20T11:29:07.408 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.404+0000 7f55c900a740 10 client.5856 fetch_fsmap finished waiting for FSMap version 24 2024-04-20T11:29:07.409 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.404+0000 7f55c900a740 10 client.5856 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-20T11:29:07.409 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.404+0000 7f55c900a740 20 client.5856 populate_metadata read hostname 'smithi152' 2024-04-20T11:29:07.409 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.404+0000 7f55c900a740 1 --2- 172.21.15.152:0/16926762 >> [v2:172.21.15.175:6833/3704251089,v1:172.21.15.175:6835/3704251089] conn(0x563a5d5c5d40 0x563a5d5f3510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:07.409 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.404+0000 7f55c900a740 4 client.5856 mds_command: new command op to 5724 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-20T11:29:07.409 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.404+0000 7f55c900a740 1 -- 172.21.15.152:0/16926762 --> [v2:172.21.15.175:6833/3704251089,v1:172.21.15.175:6835/3704251089] -- command(tid 0: {"prefix": "status"}) -- 0x563a5d534260 con 0x563a5d5c5d40 2024-04-20T11:29:07.409 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.405+0000 7f55bf400640 1 --2- 172.21.15.152:0/16926762 >> [v2:172.21.15.175:6833/3704251089,v1:172.21.15.175:6835/3704251089] conn(0x563a5d5c5d40 0x563a5d5f3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:07.410 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.406+0000 7f55bf400640 1 --2- 172.21.15.152:0/16926762 >> [v2:172.21.15.175:6833/3704251089,v1:172.21.15.175:6835/3704251089] conn(0x563a5d5c5d40 0x563a5d5f3510 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c73d950c9b15d7f9 server_cookie=4ac5748527b510bd in_seq=0 out_seq=0 2024-04-20T11:29:07.411 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.406+0000 7f55a3e00640 10 client.5856 ms_handle_connect on v2:172.21.15.175:6833/3704251089 2024-04-20T11:29:07.411 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.406+0000 7f55a3e00640 1 -- 172.21.15.152:0/16926762 <== mds.0 v2:172.21.15.175:6833/3704251089 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x563a5d534260 con 0x563a5d5c5d40 2024-04-20T11:29:07.411 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.406+0000 7f55a3e00640 10 client.5856 handle_command_reply: tid=0 2024-04-20T11:29:07.411 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.406+0000 7f55c900a740 1 client.5856 shutdown 2024-04-20T11:29:07.411 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.407+0000 7f55c900a740 1 -- 172.21.15.152:0/16926762 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563a5d5e9280 msgr2=0x563a5d5e7c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:07.411 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.407+0000 7f55c900a740 1 --2- 172.21.15.152:0/16926762 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563a5d5e9280 0x563a5d5e7c60 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f55a800bb50 tx=0x7f55a80046c0 comp rx=0 tx=0).stop 2024-04-20T11:29:07.411 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.407+0000 7f55c900a740 1 -- 172.21.15.152:0/16926762 shutdown_connections 2024-04-20T11:29:07.411 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.407+0000 7f55c900a740 1 --2- 172.21.15.152:0/16926762 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563a5d60e460 0x563a5d60e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:07.411 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.407+0000 7f55c900a740 1 --2- 172.21.15.152:0/16926762 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563a5d5e9280 0x563a5d5e7c60 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:07.411 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.407+0000 7f55c900a740 1 --2- 172.21.15.152:0/16926762 >> [v2:172.21.15.175:6833/3704251089,v1:172.21.15.175:6835/3704251089] conn(0x563a5d5c5d40 0x563a5d5f3510 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:07.411 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.407+0000 7f55c900a740 1 --2- 172.21.15.152:0/16926762 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x563a5d5c44e0 0x563a5d686fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:07.411 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.407+0000 7f55c900a740 1 -- 172.21.15.152:0/16926762 >> 172.21.15.152:0/16926762 conn(0x563a5d5c0930 msgr2=0x563a5d60ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:07.411 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.407+0000 7f55a3e00640 0 client.5856 ms_handle_reset on v2:172.21.15.175:6833/3704251089 2024-04-20T11:29:07.412 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.407+0000 7f55c900a740 1 -- 172.21.15.152:0/16926762 shutdown_connections 2024-04-20T11:29:07.412 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.407+0000 7f55c900a740 1 -- 172.21.15.152:0/16926762 wait complete. 2024-04-20T11:29:07.412 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.407+0000 7f55c900a740 20 client.5856 trim_cache size 0 max 16384 2024-04-20T11:29:07.412 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-04-20T11:29:07.412 INFO:teuthology.orchestra.run.smithi152.stdout: "cluster_fsid": "c8a72131-0d4f-42c7-892f-3f3af4d2bd3d", 2024-04-20T11:29:07.412 INFO:teuthology.orchestra.run.smithi152.stdout: "whoami": 0, 2024-04-20T11:29:07.412 INFO:teuthology.orchestra.run.smithi152.stdout: "id": 5724, 2024-04-20T11:29:07.412 INFO:teuthology.orchestra.run.smithi152.stdout: "want_state": "up:active", 2024-04-20T11:29:07.412 INFO:teuthology.orchestra.run.smithi152.stdout: "state": "up:active", 2024-04-20T11:29:07.412 INFO:teuthology.orchestra.run.smithi152.stdout: "fs_name": "cephfs", 2024-04-20T11:29:07.413 INFO:teuthology.orchestra.run.smithi152.stdout: "rank_uptime": 6.5931511370000004, 2024-04-20T11:29:07.413 INFO:teuthology.orchestra.run.smithi152.stdout: "mdsmap_epoch": 24, 2024-04-20T11:29:07.413 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_epoch": 46, 2024-04-20T11:29:07.413 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_epoch_barrier": 46, 2024-04-20T11:29:07.413 INFO:teuthology.orchestra.run.smithi152.stdout: "uptime": 13.070063887 2024-04-20T11:29:07.413 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-04-20T11:29:07.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.408+0000 7f55c6200640 1 -- 172.21.15.152:0/2070449152 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f559c06cc40 msgr2=0x7f559c06f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:07.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.408+0000 7f55c6200640 1 --2- 172.21.15.152:0/2070449152 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f559c06cc40 0x7f559c06f100 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f55a800ba00 tx=0x7f55a8030000 comp rx=0 tx=0).stop 2024-04-20T11:29:07.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.408+0000 7f55c6200640 1 -- 172.21.15.152:0/2070449152 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f55a4004800 msgr2=0x7f55c0161330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:07.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.408+0000 7f55c6200640 1 --2- 172.21.15.152:0/2070449152 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f55a4004800 0x7f55c0161330 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f55b000e780 tx=0x7f55b000ec40 comp rx=0 tx=0).stop 2024-04-20T11:29:07.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.408+0000 7f55c6200640 1 -- 172.21.15.152:0/2070449152 shutdown_connections 2024-04-20T11:29:07.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.408+0000 7f55c6200640 1 --2- 172.21.15.152:0/2070449152 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f559c06cc40 0x7f559c06f100 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:07.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.408+0000 7f55c6200640 1 --2- 172.21.15.152:0/2070449152 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f55c0155930 0x7f55c014f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:07.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.408+0000 7f55c6200640 1 --2- 172.21.15.152:0/2070449152 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55c0150100 0x7f55c01553f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:07.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.408+0000 7f55c6200640 1 --2- 172.21.15.152:0/2070449152 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f55a4004800 0x7f55c0161330 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:07.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.408+0000 7f55c6200640 1 -- 172.21.15.152:0/2070449152 >> 172.21.15.152:0/2070449152 conn(0x7f55c0067cf0 msgr2=0x7f55c00ada50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:07.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.408+0000 7f55c6200640 1 -- 172.21.15.152:0/2070449152 shutdown_connections 2024-04-20T11:29:07.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.408+0000 7f55c6200640 1 -- 172.21.15.152:0/2070449152 wait complete. 2024-04-20T11:29:07.425 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-20T11:29:07.608 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.607+0000 7f1ac3600640 1 Processor -- start 2024-04-20T11:29:07.608 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.608+0000 7f1ac3600640 1 -- start start 2024-04-20T11:29:07.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.609+0000 7f1ac3600640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f1abc0d0810 0x7f1abc1693f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:07.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.609+0000 7f1ac3600640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1abc15fd20 con 0x7f1abc0b4500 2024-04-20T11:29:07.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.609+0000 7f1ac3600640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1abc15ef60 con 0x7f1abc15c560 2024-04-20T11:29:07.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.609+0000 7f1ac3600640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f1abc15f240 con 0x7f1abc0d0810 2024-04-20T11:29:07.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.610+0000 7f1ac1600640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f1abc0d0810 0x7f1abc1693f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:07.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.610+0000 7f1ac1600640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f1abc0d0810 0x7f1abc1693f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:46316/0 (socket says 172.21.15.152:46316) 2024-04-20T11:29:07.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.610+0000 7f1ac1600640 1 -- 172.21.15.152:0/3292341978 learned_addr learned my addr 172.21.15.152:0/3292341978 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:29:07.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.610+0000 7f1ac1600640 1 -- 172.21.15.152:0/3292341978 >> v1:172.21.15.175:6789/0 conn(0x7f1abc15c560 legacy=0x7f1abc0b3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:07.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.610+0000 7f1ac1600640 1 -- 172.21.15.152:0/3292341978 >> v1:172.21.15.152:6789/0 conn(0x7f1abc0b4500 legacy=0x7f1abc0d0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:07.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.610+0000 7f1ac1600640 1 -- 172.21.15.152:0/3292341978 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1abc15f520 con 0x7f1abc0d0810 2024-04-20T11:29:07.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.611+0000 7f1ac1600640 1 --2- 172.21.15.152:0/3292341978 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1abc0d0810 0x7f1abc1693f0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f1ab4009e50 tx=0x7f1ab402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fb90d9abd755688 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:07.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.612+0000 7f1abb400640 1 -- 172.21.15.152:0/3292341978 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab402ca80 con 0x7f1abc0d0810 2024-04-20T11:29:07.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.612+0000 7f1abb400640 1 -- 172.21.15.152:0/3292341978 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ab402cc20 con 0x7f1abc0d0810 2024-04-20T11:29:07.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.612+0000 7f1abb400640 1 -- 172.21.15.152:0/3292341978 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab40343f0 con 0x7f1abc0d0810 2024-04-20T11:29:07.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.612+0000 7f1ac3600640 1 -- 172.21.15.152:0/3292341978 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1abc0d0810 msgr2=0x7f1abc1693f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:07.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.612+0000 7f1ac3600640 1 --2- 172.21.15.152:0/3292341978 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1abc0d0810 0x7f1abc1693f0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f1ab4009e50 tx=0x7f1ab402bee0 comp rx=0 tx=0).stop 2024-04-20T11:29:07.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.612+0000 7f1ac3600640 1 -- 172.21.15.152:0/3292341978 shutdown_connections 2024-04-20T11:29:07.614 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.612+0000 7f1ac3600640 1 --2- 172.21.15.152:0/3292341978 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1abc0d0810 0x7f1abc1693f0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:07.614 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.612+0000 7f1ac3600640 1 -- 172.21.15.152:0/3292341978 >> 172.21.15.152:0/3292341978 conn(0x7f1abc067cf0 msgr2=0x7f1abc06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:07.614 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.613+0000 7f1ac3600640 1 -- 172.21.15.152:0/3292341978 shutdown_connections 2024-04-20T11:29:07.614 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.613+0000 7f1ac3600640 1 -- 172.21.15.152:0/3292341978 wait complete. 2024-04-20T11:29:07.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.614+0000 7f1ac3600640 1 Processor -- start 2024-04-20T11:29:07.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.614+0000 7f1ac3600640 1 -- start start 2024-04-20T11:29:07.616 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.615+0000 7f1ac3600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1abc0b4500 0x7f1abc14f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:07.616 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.615+0000 7f1ac3600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1abc0d0810 0x7f1abc1509f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:07.616 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.615+0000 7f1ac3600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1abc15c560 0x7f1abc152610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:07.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.615+0000 7f1ac3600640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f1abc0aeaa0 con 0x7f1abc15c560 2024-04-20T11:29:07.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.616+0000 7f1ac3600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1abc0d0dc0 con 0x7f1abc0b4500 2024-04-20T11:29:07.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.616+0000 7f1ac3600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1abc0aec20 con 0x7f1abc0d0810 2024-04-20T11:29:07.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.616+0000 7f1ac1600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1abc15c560 0x7f1abc152610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:07.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.616+0000 7f1ac1600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1abc15c560 0x7f1abc152610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:46320/0 (socket says 172.21.15.152:46320) 2024-04-20T11:29:07.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.616+0000 7f1ac1600640 1 -- 172.21.15.152:0/3284127562 learned_addr learned my addr 172.21.15.152:0/3284127562 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:29:07.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.616+0000 7f1abbe00640 1 --2- 172.21.15.152:0/3284127562 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1abc0d0810 0x7f1abc1509f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:07.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.616+0000 7f1ac0c00640 1 --2- 172.21.15.152:0/3284127562 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1abc0b4500 0x7f1abc14f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:07.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.616+0000 7f1ac1600640 1 -- 172.21.15.152:0/3284127562 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1abc0d0810 msgr2=0x7f1abc1509f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:07.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.616+0000 7f1ac1600640 1 --2- 172.21.15.152:0/3284127562 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1abc0d0810 0x7f1abc1509f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:07.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.616+0000 7f1ac1600640 1 -- 172.21.15.152:0/3284127562 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1abc0b4500 msgr2=0x7f1abc14f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:07.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.616+0000 7f1ac1600640 1 --2- 172.21.15.152:0/3284127562 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1abc0b4500 0x7f1abc14f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:07.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.616+0000 7f1ac1600640 1 -- 172.21.15.152:0/3284127562 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1abc1613f0 con 0x7f1abc15c560 2024-04-20T11:29:07.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.616+0000 7f1ac0c00640 1 --2- 172.21.15.152:0/3284127562 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1abc0b4500 0x7f1abc14f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:29:07.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.617+0000 7f1ac1600640 1 --2- 172.21.15.152:0/3284127562 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1abc15c560 0x7f1abc152610 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f1ab4009f70 tx=0x7f1ab4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:07.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.617+0000 7f1ab8c00640 1 -- 172.21.15.152:0/3284127562 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab4043020 con 0x7f1abc15c560 2024-04-20T11:29:07.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.617+0000 7f1ab8c00640 1 -- 172.21.15.152:0/3284127562 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ab403e450 con 0x7f1abc15c560 2024-04-20T11:29:07.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.617+0000 7f1ab8c00640 1 -- 172.21.15.152:0/3284127562 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab4009510 con 0x7f1abc15c560 2024-04-20T11:29:07.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.617+0000 7f1ac3600640 1 -- 172.21.15.152:0/3284127562 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1abc1616d0 con 0x7f1abc15c560 2024-04-20T11:29:07.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.617+0000 7f1ac3600640 1 -- 172.21.15.152:0/3284127562 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1abc161c90 con 0x7f1abc15c560 2024-04-20T11:29:07.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.618+0000 7f1ac3600640 1 -- 172.21.15.152:0/3284127562 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1abc0689f0 con 0x7f1abc15c560 2024-04-20T11:29:07.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.619+0000 7f1ab8c00640 1 -- 172.21.15.152:0/3284127562 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f1ab4046020 con 0x7f1abc15c560 2024-04-20T11:29:07.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.620+0000 7f1ab8c00640 1 --2- 172.21.15.152:0/3284127562 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f1a9c06caa0 0x7f1a9c06ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:07.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.620+0000 7f1ab8c00640 1 -- 172.21.15.152:0/3284127562 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f1ab40ae7c0 con 0x7f1abc15c560 2024-04-20T11:29:07.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.620+0000 7f1ac0c00640 1 --2- 172.21.15.152:0/3284127562 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f1a9c06caa0 0x7f1a9c06ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:07.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.621+0000 7f1ac0c00640 1 --2- 172.21.15.152:0/3284127562 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f1a9c06caa0 0x7f1a9c06ef60 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f1abc159680 tx=0x7f1ab0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:07.625 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.624+0000 7f1ab8c00640 1 -- 172.21.15.152:0/3284127562 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f1ab404d030 con 0x7f1abc15c560 2024-04-20T11:29:07.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.870+0000 7f1ac3600640 1 -- 172.21.15.152:0/3284127562 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f1abc161ed0 con 0x7f1abc15c560 2024-04-20T11:29:07.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.870+0000 7f1ab8c00640 1 -- 172.21.15.152:0/3284127562 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) ==== 53+0+73 (secure 0 0 0) 0x7f1ab407c960 con 0x7f1abc15c560 2024-04-20T11:29:07.871 INFO:teuthology.orchestra.run.smithi152.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-20T11:29:07.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.873+0000 7f1ac3600640 1 -- 172.21.15.152:0/3284127562 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f1a9c06caa0 msgr2=0x7f1a9c06ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:07.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.873+0000 7f1ac3600640 1 --2- 172.21.15.152:0/3284127562 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f1a9c06caa0 0x7f1a9c06ef60 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f1abc159680 tx=0x7f1ab0008040 comp rx=0 tx=0).stop 2024-04-20T11:29:07.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.873+0000 7f1ac3600640 1 -- 172.21.15.152:0/3284127562 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1abc15c560 msgr2=0x7f1abc152610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:07.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.873+0000 7f1ac3600640 1 --2- 172.21.15.152:0/3284127562 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1abc15c560 0x7f1abc152610 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f1ab4009f70 tx=0x7f1ab4004a40 comp rx=0 tx=0).stop 2024-04-20T11:29:07.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.873+0000 7f1ac3600640 1 -- 172.21.15.152:0/3284127562 shutdown_connections 2024-04-20T11:29:07.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.873+0000 7f1ac3600640 1 --2- 172.21.15.152:0/3284127562 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f1a9c06caa0 0x7f1a9c06ef60 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:07.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.873+0000 7f1ac3600640 1 --2- 172.21.15.152:0/3284127562 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1abc15c560 0x7f1abc152610 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:07.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.873+0000 7f1ac3600640 1 --2- 172.21.15.152:0/3284127562 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1abc0d0810 0x7f1abc1509f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:07.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.873+0000 7f1ac3600640 1 --2- 172.21.15.152:0/3284127562 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1abc0b4500 0x7f1abc14f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:07.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.873+0000 7f1ac3600640 1 -- 172.21.15.152:0/3284127562 >> 172.21.15.152:0/3284127562 conn(0x7f1abc067cf0 msgr2=0x7f1abc159ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:07.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.873+0000 7f1ac3600640 1 -- 172.21.15.152:0/3284127562 shutdown_connections 2024-04-20T11:29:07.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:07.873+0000 7f1ac3600640 1 -- 172.21.15.152:0/3284127562 wait complete. 2024-04-20T11:29:07.895 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-20T11:29:07.895 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-20T11:29:07.895 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi152.front.sepia.ceph.com 2024-04-20T11:29:07.895 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-20T11:29:07.895 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-20T11:29:07.895 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-20T11:29:07.895 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-20T11:29:07.895 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-20T11:29:07.895 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-20T11:29:07.895 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:29:07.895 DEBUG:teuthology.orchestra.run.smithi152:> ip addr 2024-04-20T11:29:07.943 INFO:teuthology.orchestra.run.smithi152.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-20T11:29:07.943 INFO:teuthology.orchestra.run.smithi152.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-20T11:29:07.943 INFO:teuthology.orchestra.run.smithi152.stdout: inet 127.0.0.1/8 scope host lo 2024-04-20T11:29:07.943 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:29:07.943 INFO:teuthology.orchestra.run.smithi152.stdout: inet6 ::1/128 scope host 2024-04-20T11:29:07.944 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:29:07.944 INFO:teuthology.orchestra.run.smithi152.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T11:29:07.944 INFO:teuthology.orchestra.run.smithi152.stdout: link/ether 0c:c4:7a:da:6c:44 brd ff:ff:ff:ff:ff:ff 2024-04-20T11:29:07.944 INFO:teuthology.orchestra.run.smithi152.stdout: altname enp7s0f0 2024-04-20T11:29:07.944 INFO:teuthology.orchestra.run.smithi152.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T11:29:07.944 INFO:teuthology.orchestra.run.smithi152.stdout: link/ether 0c:c4:7a:da:6c:45 brd ff:ff:ff:ff:ff:ff 2024-04-20T11:29:07.944 INFO:teuthology.orchestra.run.smithi152.stdout: altname enp7s0f1 2024-04-20T11:29:07.944 INFO:teuthology.orchestra.run.smithi152.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T11:29:07.944 INFO:teuthology.orchestra.run.smithi152.stdout: link/ether 0c:c4:7a:88:6d:e2 brd ff:ff:ff:ff:ff:ff 2024-04-20T11:29:07.944 INFO:teuthology.orchestra.run.smithi152.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-20T11:29:07.944 INFO:teuthology.orchestra.run.smithi152.stdout: link/ether 0c:c4:7a:88:6d:e3 brd ff:ff:ff:ff:ff:ff 2024-04-20T11:29:07.945 INFO:teuthology.orchestra.run.smithi152.stdout: inet 172.21.15.152/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-20T11:29:07.945 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft 42658sec preferred_lft 42658sec 2024-04-20T11:29:07.945 INFO:teuthology.orchestra.run.smithi152.stdout: inet6 fe80::ec4:7aff:fe88:6de3/64 scope link 2024-04-20T11:29:07.945 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:29:07.945 INFO:teuthology.orchestra.run.smithi152.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-20T11:29:07.945 INFO:teuthology.orchestra.run.smithi152.stdout: link/ether 06:3e:9a:d3:49:1e brd ff:ff:ff:ff:ff:ff 2024-04-20T11:29:07.945 INFO:teuthology.orchestra.run.smithi152.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-20T11:29:07.945 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:29:07.945 INFO:teuthology.orchestra.run.smithi152.stdout: inet6 fe80::a019:d8ff:fe70:1185/64 scope link 2024-04-20T11:29:07.945 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:29:07.945 INFO:teuthology.orchestra.run.smithi152.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-20T11:29:07.945 INFO:teuthology.orchestra.run.smithi152.stdout: link/ether 16:e5:08:78:20:47 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T11:29:07.945 INFO:teuthology.orchestra.run.smithi152.stdout: inet6 fe80::14e5:8ff:fe78:2047/64 scope link 2024-04-20T11:29:07.945 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:29:07.946 INFO:teuthology.orchestra.run.smithi152.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-20T11:29:07.946 INFO:teuthology.orchestra.run.smithi152.stdout: link/ether 06:3e:9a:d3:49:1e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T11:29:07.946 INFO:teuthology.orchestra.run.smithi152.stdout: inet6 fe80::43e:9aff:fed3:491e/64 scope link 2024-04-20T11:29:07.946 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:29:07.947 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:29:07.947 DEBUG:teuthology.orchestra.run.smithi152:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-20T11:29:08.068 INFO:teuthology.orchestra.run.smithi152.stdout:1 2024-04-20T11:29:08.073 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:29:08.073 DEBUG:teuthology.orchestra.run.smithi152:> route 2024-04-20T11:29:08.127 INFO:teuthology.orchestra.run.smithi152.stdout:Kernel IP routing table 2024-04-20T11:29:08.128 INFO:teuthology.orchestra.run.smithi152.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-20T11:29:08.128 INFO:teuthology.orchestra.run.smithi152.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-20T11:29:08.128 INFO:teuthology.orchestra.run.smithi152.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-20T11:29:08.128 INFO:teuthology.orchestra.run.smithi152.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-20T11:29:08.128 DEBUG:teuthology.orchestra.run.smithi152:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T11:29:08.128 DEBUG:teuthology.orchestra.run.smithi152:> set -e 2024-04-20T11:29:08.128 DEBUG:teuthology.orchestra.run.smithi152:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-20T11:29:08.128 DEBUG:teuthology.orchestra.run.smithi152:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-20T11:29:08.128 DEBUG:teuthology.orchestra.run.smithi152:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-20T11:29:08.128 DEBUG:teuthology.orchestra.run.smithi152:> ') 2024-04-20T11:29:08.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T11:29:08.331 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T11:29:08.341 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:29:08.341 DEBUG:teuthology.orchestra.run.smithi152:> ip netns list 2024-04-20T11:29:08.390 INFO:teuthology.orchestra.run.smithi152.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-20T11:29:08.391 INFO:teuthology.orchestra.run.smithi152.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-20T11:29:08.391 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-20T11:29:08.391 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:29:08.392 DEBUG:teuthology.orchestra.run.smithi152:> sudo ip link set brx.0 up 2024-04-20T11:29:08.473 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-20T11:29:08.473 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T11:29:08.473 DEBUG:teuthology.orchestra.run.smithi152:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-20T11:29:08.526 INFO:teuthology.orchestra.run.smithi152.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-20T11:29:08.526 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T11:29:08.526 DEBUG:teuthology.orchestra.run.smithi152:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-20T11:29:08.580 DEBUG:teuthology.orchestra.run.smithi152:> sudo modprobe fuse 2024-04-20T11:29:08.664 DEBUG:teuthology.orchestra.run.smithi152:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T11:29:08.715 INFO:teuthology.orchestra.run.smithi152.stdout:/proc 2024-04-20T11:29:08.715 INFO:teuthology.orchestra.run.smithi152.stdout:/sys 2024-04-20T11:29:08.715 INFO:teuthology.orchestra.run.smithi152.stdout:/dev 2024-04-20T11:29:08.715 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/security 2024-04-20T11:29:08.715 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/shm 2024-04-20T11:29:08.715 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/pts 2024-04-20T11:29:08.715 INFO:teuthology.orchestra.run.smithi152.stdout:/run 2024-04-20T11:29:08.715 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/cgroup 2024-04-20T11:29:08.716 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/pstore 2024-04-20T11:29:08.716 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/bpf 2024-04-20T11:29:08.716 INFO:teuthology.orchestra.run.smithi152.stdout:/ 2024-04-20T11:29:08.716 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/selinux 2024-04-20T11:29:08.716 INFO:teuthology.orchestra.run.smithi152.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T11:29:08.716 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/mqueue 2024-04-20T11:29:08.716 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/hugepages 2024-04-20T11:29:08.716 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/debug 2024-04-20T11:29:08.716 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/tracing 2024-04-20T11:29:08.716 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/fuse/connections 2024-04-20T11:29:08.716 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/config 2024-04-20T11:29:08.716 INFO:teuthology.orchestra.run.smithi152.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T11:29:08.716 INFO:teuthology.orchestra.run.smithi152.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T11:29:08.716 INFO:teuthology.orchestra.run.smithi152.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T11:29:08.716 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T11:29:08.717 INFO:teuthology.orchestra.run.smithi152.stdout:/run/user/1000 2024-04-20T11:29:08.717 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph 2024-04-20T11:29:08.717 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph/osd/ceph-0 2024-04-20T11:29:08.717 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph/osd/ceph-1 2024-04-20T11:29:08.717 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph/osd/ceph-2 2024-04-20T11:29:08.717 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph/osd/ceph-3 2024-04-20T11:29:08.717 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns 2024-04-20T11:29:08.717 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T11:29:08.717 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T11:29:08.717 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T11:29:08.717 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T11:29:08.718 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:29:08.718 DEBUG:teuthology.orchestra.run.smithi152:> ls /sys/fs/fuse/connections 2024-04-20T11:29:08.766 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-20T11:29:08.767 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-20T11:29:08.771 DEBUG:teuthology.orchestra.run.smithi152:> sudo modprobe fuse 2024-04-20T11:29:08.841 DEBUG:teuthology.orchestra.run.smithi152:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T11:29:08.882 INFO:teuthology.orchestra.run.smithi152.stdout:/proc 2024-04-20T11:29:08.882 INFO:teuthology.orchestra.run.smithi152.stdout:/sys 2024-04-20T11:29:08.882 INFO:teuthology.orchestra.run.smithi152.stdout:/dev 2024-04-20T11:29:08.883 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/security 2024-04-20T11:29:08.883 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/shm 2024-04-20T11:29:08.883 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/pts 2024-04-20T11:29:08.883 INFO:teuthology.orchestra.run.smithi152.stdout:/run 2024-04-20T11:29:08.883 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/cgroup 2024-04-20T11:29:08.883 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/pstore 2024-04-20T11:29:08.883 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/bpf 2024-04-20T11:29:08.883 INFO:teuthology.orchestra.run.smithi152.stdout:/ 2024-04-20T11:29:08.883 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/selinux 2024-04-20T11:29:08.883 INFO:teuthology.orchestra.run.smithi152.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T11:29:08.883 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/mqueue 2024-04-20T11:29:08.883 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/hugepages 2024-04-20T11:29:08.883 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/debug 2024-04-20T11:29:08.883 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/tracing 2024-04-20T11:29:08.884 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/fuse/connections 2024-04-20T11:29:08.884 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/config 2024-04-20T11:29:08.884 INFO:teuthology.orchestra.run.smithi152.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T11:29:08.884 INFO:teuthology.orchestra.run.smithi152.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T11:29:08.884 INFO:teuthology.orchestra.run.smithi152.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T11:29:08.884 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T11:29:08.884 INFO:teuthology.orchestra.run.smithi152.stdout:/run/user/1000 2024-04-20T11:29:08.884 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph 2024-04-20T11:29:08.884 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph/osd/ceph-0 2024-04-20T11:29:08.884 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph/osd/ceph-1 2024-04-20T11:29:08.884 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph/osd/ceph-2 2024-04-20T11:29:08.884 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph/osd/ceph-3 2024-04-20T11:29:08.884 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns 2024-04-20T11:29:08.884 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T11:29:08.885 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T11:29:08.885 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T11:29:08.885 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T11:29:08.885 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:29:08.886 DEBUG:teuthology.orchestra.run.smithi152:> ls /sys/fs/fuse/connections 2024-04-20T11:29:08.940 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi152.stderr:2024-04-20T11:29:08.940+0000 7fe87a359480 -1 init, newargv = 0x5559fc62a830 newargc=15 2024-04-20T11:29:08.941 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi152.stderr:ceph-fuse[39862]: starting ceph client 2024-04-20T11:29:08.951 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi152.stderr:ceph-fuse[39862]: starting fuse 2024-04-20T11:29:08.968 INFO:teuthology.orchestra.run.smithi152.stdout:42 2024-04-20T11:29:08.968 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-04-20T11:29:08.969 DEBUG:teuthology.orchestra.run.smithi152:> sudo stdin-killer -- python3 -c ' 2024-04-20T11:29:08.969 DEBUG:teuthology.orchestra.run.smithi152:> import glob 2024-04-20T11:29:08.969 DEBUG:teuthology.orchestra.run.smithi152:> import re 2024-04-20T11:29:08.969 DEBUG:teuthology.orchestra.run.smithi152:> import os 2024-04-20T11:29:08.969 DEBUG:teuthology.orchestra.run.smithi152:> import subprocess 2024-04-20T11:29:08.969 DEBUG:teuthology.orchestra.run.smithi152:> 2024-04-20T11:29:08.969 DEBUG:teuthology.orchestra.run.smithi152:> def _find_admin_socket(client_name): 2024-04-20T11:29:08.969 DEBUG:teuthology.orchestra.run.smithi152:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-20T11:29:08.969 DEBUG:teuthology.orchestra.run.smithi152:> files = glob.glob(asok_path) 2024-04-20T11:29:08.969 DEBUG:teuthology.orchestra.run.smithi152:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-20T11:29:08.969 DEBUG:teuthology.orchestra.run.smithi152:> 2024-04-20T11:29:08.969 DEBUG:teuthology.orchestra.run.smithi152:> # Given a non-glob path, it better be there 2024-04-20T11:29:08.969 DEBUG:teuthology.orchestra.run.smithi152:> if "*" not in asok_path: 2024-04-20T11:29:08.969 DEBUG:teuthology.orchestra.run.smithi152:> assert(len(files) == 1) 2024-04-20T11:29:08.969 DEBUG:teuthology.orchestra.run.smithi152:> return files[0] 2024-04-20T11:29:08.970 DEBUG:teuthology.orchestra.run.smithi152:> 2024-04-20T11:29:08.970 DEBUG:teuthology.orchestra.run.smithi152:> for f in files: 2024-04-20T11:29:08.970 DEBUG:teuthology.orchestra.run.smithi152:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-20T11:29:08.970 DEBUG:teuthology.orchestra.run.smithi152:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-20T11:29:08.970 DEBUG:teuthology.orchestra.run.smithi152:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-20T11:29:08.970 DEBUG:teuthology.orchestra.run.smithi152:> contents = proc_f.read() 2024-04-20T11:29:08.970 DEBUG:teuthology.orchestra.run.smithi152:> if mountpoint in contents: 2024-04-20T11:29:08.970 DEBUG:teuthology.orchestra.run.smithi152:> return f 2024-04-20T11:29:08.970 DEBUG:teuthology.orchestra.run.smithi152:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-20T11:29:08.970 DEBUG:teuthology.orchestra.run.smithi152:> 2024-04-20T11:29:08.970 DEBUG:teuthology.orchestra.run.smithi152:> print(_find_admin_socket("client.0")) 2024-04-20T11:29:08.970 DEBUG:teuthology.orchestra.run.smithi152:> ' 2024-04-20T11:29:09.163 INFO:teuthology.orchestra.run.smithi152.stdout:/var/run/ceph/ceph-client.0.39862.asok 2024-04-20T11:29:09.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T11:29:09.180 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.39862.asok 2024-04-20T11:29:09.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:29:09.181 DEBUG:teuthology.orchestra.run.smithi152:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.39862.asok status 2024-04-20T11:29:09.407 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-04-20T11:29:09.408 INFO:teuthology.orchestra.run.smithi152.stdout: "metadata": { 2024-04-20T11:29:09.408 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_sha1": "a8c84f5997950fd7d008204839253291d52e6eda", 2024-04-20T11:29:09.408 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", 2024-04-20T11:29:09.408 INFO:teuthology.orchestra.run.smithi152.stdout: "entity_id": "0", 2024-04-20T11:29:09.408 INFO:teuthology.orchestra.run.smithi152.stdout: "hostname": "smithi152", 2024-04-20T11:29:09.408 INFO:teuthology.orchestra.run.smithi152.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-20T11:29:09.408 INFO:teuthology.orchestra.run.smithi152.stdout: "pid": "39862", 2024-04-20T11:29:09.408 INFO:teuthology.orchestra.run.smithi152.stdout: "root": "/" 2024-04-20T11:29:09.408 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:29:09.408 INFO:teuthology.orchestra.run.smithi152.stdout: "dentry_count": 0, 2024-04-20T11:29:09.408 INFO:teuthology.orchestra.run.smithi152.stdout: "dentry_pinned_count": 0, 2024-04-20T11:29:09.408 INFO:teuthology.orchestra.run.smithi152.stdout: "id": 5871, 2024-04-20T11:29:09.408 INFO:teuthology.orchestra.run.smithi152.stdout: "inst": { 2024-04-20T11:29:09.409 INFO:teuthology.orchestra.run.smithi152.stdout: "name": { 2024-04-20T11:29:09.409 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "client", 2024-04-20T11:29:09.409 INFO:teuthology.orchestra.run.smithi152.stdout: "num": 5871 2024-04-20T11:29:09.409 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:29:09.409 INFO:teuthology.orchestra.run.smithi152.stdout: "addr": { 2024-04-20T11:29:09.409 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "v1", 2024-04-20T11:29:09.409 INFO:teuthology.orchestra.run.smithi152.stdout: "addr": "192.168.0.1:0", 2024-04-20T11:29:09.409 INFO:teuthology.orchestra.run.smithi152.stdout: "nonce": 653639925 2024-04-20T11:29:09.409 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:29:09.409 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:29:09.409 INFO:teuthology.orchestra.run.smithi152.stdout: "addr": { 2024-04-20T11:29:09.409 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "v1", 2024-04-20T11:29:09.409 INFO:teuthology.orchestra.run.smithi152.stdout: "addr": "192.168.0.1:0", 2024-04-20T11:29:09.409 INFO:teuthology.orchestra.run.smithi152.stdout: "nonce": 653639925 2024-04-20T11:29:09.409 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:29:09.409 INFO:teuthology.orchestra.run.smithi152.stdout: "inst_str": "client.5871 192.168.0.1:0/653639925", 2024-04-20T11:29:09.410 INFO:teuthology.orchestra.run.smithi152.stdout: "addr_str": "192.168.0.1:0/653639925", 2024-04-20T11:29:09.410 INFO:teuthology.orchestra.run.smithi152.stdout: "inode_count": 1, 2024-04-20T11:29:09.410 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_epoch": 24, 2024-04-20T11:29:09.410 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_epoch": 46, 2024-04-20T11:29:09.410 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_epoch_barrier": 0, 2024-04-20T11:29:09.410 INFO:teuthology.orchestra.run.smithi152.stdout: "blocklisted": false, 2024-04-20T11:29:09.410 INFO:teuthology.orchestra.run.smithi152.stdout: "fs_name": "cephfs" 2024-04-20T11:29:09.410 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-04-20T11:29:09.422 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:29:09.422 DEBUG:teuthology.orchestra.run.smithi152:> stat --file-system '--printf=%T 2024-04-20T11:29:09.422 DEBUG:teuthology.orchestra.run.smithi152:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-20T11:29:09.475 INFO:teuthology.orchestra.run.smithi152.stdout:fuseblk 2024-04-20T11:29:09.475 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-20T11:29:09.476 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:29:09.476 DEBUG:teuthology.orchestra.run.smithi152:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-20T11:29:09.561 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-20T11:29:09.762 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:09.762+0000 7ff4e9200640 1 Processor -- start 2024-04-20T11:29:09.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:09.762+0000 7ff4e9200640 1 -- start start 2024-04-20T11:29:09.764 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:09.764+0000 7ff4e9200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ff4e406eb40 0x7ff4e406f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:09.764 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:09.764+0000 7ff4e9200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4e406f880 0x7ff4e415cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:09.765 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:09.764+0000 7ff4e9200640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4e406fca0 con 0x7ff4e415b5e0 2024-04-20T11:29:09.765 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:09.764+0000 7ff4e9200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7ff4e406de10 con 0x7ff4e406eb40 2024-04-20T11:29:09.765 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:09.764+0000 7ff4e9200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff4e40b5770 con 0x7ff4e406f880 2024-04-20T11:29:09.765 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:09.764+0000 7ff4e1e00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7ff4e415b5e0 0x7ff4e40b4c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:35746/0 (socket says 172.21.15.152:35746) 2024-04-20T11:29:09.765 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:09.764+0000 7ff4e1e00640 1 -- 172.21.15.152:0/1933985279 learned_addr learned my addr 172.21.15.152:0/1933985279 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:29:09.767 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:09.764+0000 7ff4e2800640 1 --2- 172.21.15.152:0/1933985279 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4e406f880 0x7ff4e415cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:09.767 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:09.764+0000 7ff4d9400640 1 --2- 172.21.15.152:0/1933985279 >> v2:172.21.15.175:3300/0 conn(0x7ff4e406eb40 0x7ff4e406f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:09.767 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:09.765+0000 7ff4e1400640 1 -- 172.21.15.152:0/1933985279 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2102409366 0 0) 0x7ff4e406fca0 con 0x7ff4e415b5e0 2024-04-20T11:29:09.767 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:09.765+0000 7ff4e1400640 1 -- 172.21.15.152:0/1933985279 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff4c8005730 con 0x7ff4e415b5e0 2024-04-20T11:29:09.767 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:09.765+0000 7ff4e1400640 1 -- 172.21.15.152:0/1933985279 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3307188701 0 0) 0x7ff4c8005730 con 0x7ff4e415b5e0 2024-04-20T11:29:09.767 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:09.765+0000 7ff4e1400640 1 -- 172.21.15.152:0/1933985279 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4e406f880 msgr2=0x7ff4e415cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:09.767 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:09.765+0000 7ff4e1400640 1 --2- 172.21.15.152:0/1933985279 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4e406f880 0x7ff4e415cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:09.767 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:09.765+0000 7ff4e1400640 1 -- 172.21.15.152:0/1933985279 >> v2:172.21.15.175:3300/0 conn(0x7ff4e406eb40 msgr2=0x7ff4e406f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:09.767 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:09.765+0000 7ff4e1400640 1 --2- 172.21.15.152:0/1933985279 >> v2:172.21.15.175:3300/0 conn(0x7ff4e406eb40 0x7ff4e406f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:09.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:09.765+0000 7ff4e1400640 1 -- 172.21.15.152:0/1933985279 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4e406e060 con 0x7ff4e415b5e0 2024-04-20T11:29:09.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:09.765+0000 7ff4e1400640 1 -- 172.21.15.152:0/1933985279 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7ff4d4001690 con 0x7ff4e415b5e0 2024-04-20T11:29:09.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:09.765+0000 7ff4e1400640 1 -- 172.21.15.152:0/1933985279 >> v1:172.21.15.152:6789/0 conn(0x7ff4e415b5e0 legacy=0x7ff4e40b4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:09.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:09.765+0000 7ff4e1400640 1 --2- 172.21.15.152:0/1933985279 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4e406f880 0x7ff4c8005c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:09.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:09.765+0000 7ff4e1400640 1 --2- 172.21.15.152:0/1933985279 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff4c80061b0 0x7ff4c8006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:09.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:09.765+0000 7ff4e1400640 1 --2- 172.21.15.152:0/1933985279 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4c8006f50 0x7ff4c80073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:09.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:09.765+0000 7ff4e1400640 1 -- 172.21.15.152:0/1933985279 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e406de10 con 0x7ff4c80061b0 2024-04-20T11:29:09.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:09.765+0000 7ff4e1400640 1 -- 172.21.15.152:0/1933985279 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e40b5770 con 0x7ff4c8006f50 2024-04-20T11:29:09.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:09.765+0000 7ff4e1400640 1 -- 172.21.15.152:0/1933985279 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff4c8003550 con 0x7ff4e406f880 2024-04-20T11:29:09.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:09.766+0000 7ff4d9400640 1 --2- 172.21.15.152:0/1933985279 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff4c80061b0 0x7ff4c8006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:09.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:09.766+0000 7ff4e1e00640 1 --2- 172.21.15.152:0/1933985279 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4e406f880 0x7ff4c8005c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:09.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:09.766+0000 7ff4d9400640 1 -- 172.21.15.152:0/1933985279 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4e406f880 msgr2=0x7ff4c8005c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:09.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:09.766+0000 7ff4d9400640 1 --2- 172.21.15.152:0/1933985279 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4e406f880 0x7ff4c8005c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:09.769 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:09.766+0000 7ff4d9400640 1 -- 172.21.15.152:0/1933985279 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4c8006f50 msgr2=0x7ff4c80073d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:09.769 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:09.766+0000 7ff4d9400640 1 --2- 172.21.15.152:0/1933985279 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4c8006f50 0x7ff4c80073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:09.769 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:09.766+0000 7ff4d9400640 1 -- 172.21.15.152:0/1933985279 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4c8005730 con 0x7ff4c80061b0 2024-04-20T11:29:09.769 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:09.766+0000 7ff4e1e00640 1 --2- 172.21.15.152:0/1933985279 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4e406f880 0x7ff4c8005c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:29:09.770 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:09.767+0000 7ff4d9400640 1 --2- 172.21.15.152:0/1933985279 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff4c80061b0 0x7ff4c8006a10 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc009ac0 tx=0x7ff4c8007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b222a3f937eed689 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:09.770 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:09.767+0000 7ff4e1400640 1 -- 172.21.15.152:0/1933985279 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4cc039070 con 0x7ff4c80061b0 2024-04-20T11:29:09.770 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:09.767+0000 7ff4e1400640 1 -- 172.21.15.152:0/1933985279 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4cc02c710 con 0x7ff4c80061b0 2024-04-20T11:29:09.770 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:09.767+0000 7ff4e1400640 1 -- 172.21.15.152:0/1933985279 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4cc009690 con 0x7ff4c80061b0 2024-04-20T11:29:09.770 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:09.768+0000 7ff4e9200640 1 -- 172.21.15.152:0/1933985279 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff4c80061b0 msgr2=0x7ff4c8006a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:09.770 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:09.768+0000 7ff4e9200640 1 --2- 172.21.15.152:0/1933985279 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff4c80061b0 0x7ff4c8006a10 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc009ac0 tx=0x7ff4c8007a40 comp rx=0 tx=0).stop 2024-04-20T11:29:09.770 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:09.768+0000 7ff4e2800640 1 -- 172.21.15.152:0/1933985279 reap_dead start 2024-04-20T11:29:09.770 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:09.768+0000 7ff4e9200640 1 -- 172.21.15.152:0/1933985279 shutdown_connections 2024-04-20T11:29:09.770 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:09.768+0000 7ff4e9200640 1 -- 172.21.15.152:0/1933985279 >> 172.21.15.152:0/1933985279 conn(0x7ff4e4067cf0 msgr2=0x7ff4e406b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:09.771 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:09.768+0000 7ff4e9200640 1 -- 172.21.15.152:0/1933985279 shutdown_connections 2024-04-20T11:29:09.771 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:09.769+0000 7ff4e9200640 1 -- 172.21.15.152:0/1933985279 wait complete. 2024-04-20T11:29:09.771 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:09.771+0000 7ff4e9200640 1 Processor -- start 2024-04-20T11:29:09.771 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:09.771+0000 7ff4e9200640 1 -- start start 2024-04-20T11:29:09.773 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:09.772+0000 7ff4e9200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4c80061b0 0x7ff4e415ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:09.773 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:09.772+0000 7ff4e9200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4e41f0970 0x7ff4e4142ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:09.773 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:09.772+0000 7ff4e9200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff4e415dba0 0x7ff4e415e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:09.773 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:09.772+0000 7ff4e9200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e415c9d0 con 0x7ff4e415dba0 2024-04-20T11:29:09.773 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:09.772+0000 7ff4e9200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e406e620 con 0x7ff4e41f0970 2024-04-20T11:29:09.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:09.772+0000 7ff4e9200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff4e40b42d0 con 0x7ff4c80061b0 2024-04-20T11:29:09.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:09.772+0000 7ff4e2800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff4e415dba0 0x7ff4e415e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:09.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:09.772+0000 7ff4e2800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff4e415dba0 0x7ff4e415e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:46326/0 (socket says 172.21.15.152:46326) 2024-04-20T11:29:09.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:09.772+0000 7ff4e2800640 1 -- 172.21.15.152:0/3371568598 learned_addr learned my addr 172.21.15.152:0/3371568598 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:29:09.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:09.773+0000 7ff4d9400640 1 --2- 172.21.15.152:0/3371568598 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4e41f0970 0x7ff4e4142ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:09.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:09.773+0000 7ff4e1e00640 1 --2- 172.21.15.152:0/3371568598 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4c80061b0 0x7ff4e415ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:09.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:09.773+0000 7ff4e2800640 1 -- 172.21.15.152:0/3371568598 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4c80061b0 msgr2=0x7ff4e415ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:09.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:09.773+0000 7ff4e2800640 1 --2- 172.21.15.152:0/3371568598 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4c80061b0 0x7ff4e415ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:09.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:09.773+0000 7ff4e2800640 1 -- 172.21.15.152:0/3371568598 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4e41f0970 msgr2=0x7ff4e4142ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:09.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:09.773+0000 7ff4e2800640 1 --2- 172.21.15.152:0/3371568598 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4e41f0970 0x7ff4e4142ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:09.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:09.773+0000 7ff4e2800640 1 -- 172.21.15.152:0/3371568598 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4e41f0df0 con 0x7ff4e415dba0 2024-04-20T11:29:09.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:09.773+0000 7ff4e2800640 1 --2- 172.21.15.152:0/3371568598 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff4e415dba0 0x7ff4e415e020 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7ff4e40b0090 tx=0x7ff4dc00b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:09.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:09.773+0000 7ff4daa00640 1 -- 172.21.15.152:0/3371568598 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4dc018020 con 0x7ff4e415dba0 2024-04-20T11:29:09.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:09.774+0000 7ff4daa00640 1 -- 172.21.15.152:0/3371568598 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4dc00bb10 con 0x7ff4e415dba0 2024-04-20T11:29:09.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:09.774+0000 7ff4daa00640 1 -- 172.21.15.152:0/3371568598 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4dc014400 con 0x7ff4e415dba0 2024-04-20T11:29:09.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:09.774+0000 7ff4e9200640 1 -- 172.21.15.152:0/3371568598 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4e414e080 con 0x7ff4e415dba0 2024-04-20T11:29:09.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:09.774+0000 7ff4e9200640 1 -- 172.21.15.152:0/3371568598 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4e414fd20 con 0x7ff4e415dba0 2024-04-20T11:29:09.776 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:09.775+0000 7ff4e9200640 1 -- 172.21.15.152:0/3371568598 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4e414ff20 con 0x7ff4e415dba0 2024-04-20T11:29:09.776 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:09.776+0000 7ff4daa00640 1 -- 172.21.15.152:0/3371568598 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7ff4dc005d10 con 0x7ff4e415dba0 2024-04-20T11:29:09.777 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:09.777+0000 7ff4daa00640 1 --2- 172.21.15.152:0/3371568598 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff4c006caa0 0x7ff4c006ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:09.778 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:09.777+0000 7ff4daa00640 1 -- 172.21.15.152:0/3371568598 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7ff4dc08c510 con 0x7ff4e415dba0 2024-04-20T11:29:09.778 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:09.777+0000 7ff4e1e00640 1 --2- 172.21.15.152:0/3371568598 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff4c006caa0 0x7ff4c006ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:09.778 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:09.778+0000 7ff4e1e00640 1 --2- 172.21.15.152:0/3371568598 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff4c006caa0 0x7ff4c006ef60 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff4d4001900 tx=0x7ff4d4009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:09.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:09.782+0000 7ff4daa00640 1 -- 172.21.15.152:0/3371568598 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ff4e414ff20 con 0x7ff4e415dba0 2024-04-20T11:29:10.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:10.023+0000 7ff4e9200640 1 -- 172.21.15.152:0/3371568598 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ff4e40689f0 con 0x7ff4e415dba0 2024-04-20T11:29:10.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:10.024+0000 7ff4daa00640 1 -- 172.21.15.152:0/3371568598 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) ==== 53+0+73 (secure 0 0 0) 0x7ff4dc022030 con 0x7ff4e415dba0 2024-04-20T11:29:10.025 INFO:teuthology.orchestra.run.smithi152.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-20T11:29:10.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:10.027+0000 7ff4e9200640 1 -- 172.21.15.152:0/3371568598 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff4c006caa0 msgr2=0x7ff4c006ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:10.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:10.027+0000 7ff4e9200640 1 --2- 172.21.15.152:0/3371568598 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff4c006caa0 0x7ff4c006ef60 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff4d4001900 tx=0x7ff4d4009040 comp rx=0 tx=0).stop 2024-04-20T11:29:10.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:10.027+0000 7ff4e9200640 1 -- 172.21.15.152:0/3371568598 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff4e415dba0 msgr2=0x7ff4e415e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:10.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:10.027+0000 7ff4e9200640 1 --2- 172.21.15.152:0/3371568598 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff4e415dba0 0x7ff4e415e020 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7ff4e40b0090 tx=0x7ff4dc00b480 comp rx=0 tx=0).stop 2024-04-20T11:29:10.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:10.027+0000 7ff4e9200640 1 -- 172.21.15.152:0/3371568598 shutdown_connections 2024-04-20T11:29:10.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:10.027+0000 7ff4e9200640 1 --2- 172.21.15.152:0/3371568598 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff4c006caa0 0x7ff4c006ef60 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:10.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:10.027+0000 7ff4e9200640 1 --2- 172.21.15.152:0/3371568598 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff4e415dba0 0x7ff4e415e020 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:10.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:10.027+0000 7ff4e9200640 1 --2- 172.21.15.152:0/3371568598 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4e41f0970 0x7ff4e4142ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:10.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:10.027+0000 7ff4e9200640 1 --2- 172.21.15.152:0/3371568598 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4c80061b0 0x7ff4e415ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:10.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:10.027+0000 7ff4e9200640 1 -- 172.21.15.152:0/3371568598 >> 172.21.15.152:0/3371568598 conn(0x7ff4e4067cf0 msgr2=0x7ff4e40b49a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:10.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:10.027+0000 7ff4e9200640 1 -- 172.21.15.152:0/3371568598 shutdown_connections 2024-04-20T11:29:10.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:10.027+0000 7ff4e9200640 1 -- 172.21.15.152:0/3371568598 wait complete. 2024-04-20T11:29:10.047 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-20T11:29:10.047 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-20T11:29:10.047 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi175.front.sepia.ceph.com 2024-04-20T11:29:10.047 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-20T11:29:10.047 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-20T11:29:10.047 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-20T11:29:10.047 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-20T11:29:10.047 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-20T11:29:10.047 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-20T11:29:10.048 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:29:10.048 DEBUG:teuthology.orchestra.run.smithi175:> ip addr 2024-04-20T11:29:10.105 INFO:teuthology.orchestra.run.smithi175.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-20T11:29:10.105 INFO:teuthology.orchestra.run.smithi175.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-20T11:29:10.106 INFO:teuthology.orchestra.run.smithi175.stdout: inet 127.0.0.1/8 scope host lo 2024-04-20T11:29:10.106 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:29:10.106 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 ::1/128 scope host 2024-04-20T11:29:10.106 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:29:10.106 INFO:teuthology.orchestra.run.smithi175.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T11:29:10.106 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:d9:39:ee brd ff:ff:ff:ff:ff:ff 2024-04-20T11:29:10.106 INFO:teuthology.orchestra.run.smithi175.stdout: altname enp7s0f0 2024-04-20T11:29:10.106 INFO:teuthology.orchestra.run.smithi175.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T11:29:10.106 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:d9:39:ef brd ff:ff:ff:ff:ff:ff 2024-04-20T11:29:10.106 INFO:teuthology.orchestra.run.smithi175.stdout: altname enp7s0f1 2024-04-20T11:29:10.106 INFO:teuthology.orchestra.run.smithi175.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T11:29:10.106 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:88:71:f0 brd ff:ff:ff:ff:ff:ff 2024-04-20T11:29:10.106 INFO:teuthology.orchestra.run.smithi175.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-20T11:29:10.107 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:88:71:f1 brd ff:ff:ff:ff:ff:ff 2024-04-20T11:29:10.107 INFO:teuthology.orchestra.run.smithi175.stdout: inet 172.21.15.175/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-20T11:29:10.107 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft 42631sec preferred_lft 42631sec 2024-04-20T11:29:10.107 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::ec4:7aff:fe88:71f1/64 scope link 2024-04-20T11:29:10.107 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:29:10.107 INFO:teuthology.orchestra.run.smithi175.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-20T11:29:10.107 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 02:d6:c6:31:26:70 brd ff:ff:ff:ff:ff:ff 2024-04-20T11:29:10.107 INFO:teuthology.orchestra.run.smithi175.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-20T11:29:10.107 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:29:10.107 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::5091:d5ff:fee7:54b3/64 scope link 2024-04-20T11:29:10.107 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:29:10.107 INFO:teuthology.orchestra.run.smithi175.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-20T11:29:10.107 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 02:d6:c6:31:26:70 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-20T11:29:10.107 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::d6:c6ff:fe31:2670/64 scope link 2024-04-20T11:29:10.108 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:29:10.108 INFO:teuthology.orchestra.run.smithi175.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-20T11:29:10.108 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 9e:14:f3:e2:18:1e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-20T11:29:10.108 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::9c14:f3ff:fee2:181e/64 scope link 2024-04-20T11:29:10.108 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:29:10.109 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:29:10.109 DEBUG:teuthology.orchestra.run.smithi175:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-20T11:29:10.185 INFO:teuthology.orchestra.run.smithi175.stdout:1 2024-04-20T11:29:10.189 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:29:10.189 DEBUG:teuthology.orchestra.run.smithi175:> route 2024-04-20T11:29:10.235 INFO:teuthology.orchestra.run.smithi175.stdout:Kernel IP routing table 2024-04-20T11:29:10.236 INFO:teuthology.orchestra.run.smithi175.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-20T11:29:10.236 INFO:teuthology.orchestra.run.smithi175.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-20T11:29:10.236 INFO:teuthology.orchestra.run.smithi175.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-20T11:29:10.236 INFO:teuthology.orchestra.run.smithi175.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-20T11:29:10.236 DEBUG:teuthology.orchestra.run.smithi175:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T11:29:10.236 DEBUG:teuthology.orchestra.run.smithi175:> set -e 2024-04-20T11:29:10.236 DEBUG:teuthology.orchestra.run.smithi175:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-20T11:29:10.236 DEBUG:teuthology.orchestra.run.smithi175:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-20T11:29:10.236 DEBUG:teuthology.orchestra.run.smithi175:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-20T11:29:10.236 DEBUG:teuthology.orchestra.run.smithi175:> ') 2024-04-20T11:29:10.382 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:29:10 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T11:29:10.479 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:29:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T11:29:10.489 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:29:10.489 DEBUG:teuthology.orchestra.run.smithi175:> ip netns list 2024-04-20T11:29:10.538 INFO:teuthology.orchestra.run.smithi175.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-20T11:29:10.538 INFO:teuthology.orchestra.run.smithi175.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-20T11:29:10.539 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-20T11:29:10.539 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:29:10.539 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip link set brx.0 up 2024-04-20T11:29:10.621 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-04-20T11:29:10.621 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T11:29:10.621 DEBUG:teuthology.orchestra.run.smithi175:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-20T11:29:10.674 INFO:teuthology.orchestra.run.smithi175.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-20T11:29:10.674 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T11:29:10.674 DEBUG:teuthology.orchestra.run.smithi175:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-20T11:29:10.722 DEBUG:teuthology.orchestra.run.smithi175:> sudo modprobe fuse 2024-04-20T11:29:10.795 DEBUG:teuthology.orchestra.run.smithi175:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T11:29:10.847 INFO:teuthology.orchestra.run.smithi175.stdout:/proc 2024-04-20T11:29:10.847 INFO:teuthology.orchestra.run.smithi175.stdout:/sys 2024-04-20T11:29:10.847 INFO:teuthology.orchestra.run.smithi175.stdout:/dev 2024-04-20T11:29:10.847 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/security 2024-04-20T11:29:10.847 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/shm 2024-04-20T11:29:10.847 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/pts 2024-04-20T11:29:10.847 INFO:teuthology.orchestra.run.smithi175.stdout:/run 2024-04-20T11:29:10.847 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/cgroup 2024-04-20T11:29:10.847 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/pstore 2024-04-20T11:29:10.848 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/bpf 2024-04-20T11:29:10.848 INFO:teuthology.orchestra.run.smithi175.stdout:/ 2024-04-20T11:29:10.848 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/selinux 2024-04-20T11:29:10.848 INFO:teuthology.orchestra.run.smithi175.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T11:29:10.848 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/mqueue 2024-04-20T11:29:10.848 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/debug 2024-04-20T11:29:10.848 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/hugepages 2024-04-20T11:29:10.848 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/tracing 2024-04-20T11:29:10.848 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/fuse/connections 2024-04-20T11:29:10.848 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/config 2024-04-20T11:29:10.848 INFO:teuthology.orchestra.run.smithi175.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T11:29:10.848 INFO:teuthology.orchestra.run.smithi175.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T11:29:10.848 INFO:teuthology.orchestra.run.smithi175.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T11:29:10.848 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T11:29:10.848 INFO:teuthology.orchestra.run.smithi175.stdout:/run/user/1000 2024-04-20T11:29:10.849 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph 2024-04-20T11:29:10.849 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-4 2024-04-20T11:29:10.849 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-5 2024-04-20T11:29:10.849 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-6 2024-04-20T11:29:10.849 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-7 2024-04-20T11:29:10.849 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns 2024-04-20T11:29:10.849 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-20T11:29:10.849 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-20T11:29:10.849 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-20T11:29:10.849 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-20T11:29:10.850 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:29:10.850 DEBUG:teuthology.orchestra.run.smithi175:> ls /sys/fs/fuse/connections 2024-04-20T11:29:10.902 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-20T11:29:10.903 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-04-20T11:29:10.907 DEBUG:teuthology.orchestra.run.smithi175:> sudo modprobe fuse 2024-04-20T11:29:10.973 DEBUG:teuthology.orchestra.run.smithi175:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T11:29:11.012 INFO:teuthology.orchestra.run.smithi175.stdout:/proc 2024-04-20T11:29:11.012 INFO:teuthology.orchestra.run.smithi175.stdout:/sys 2024-04-20T11:29:11.012 INFO:teuthology.orchestra.run.smithi175.stdout:/dev 2024-04-20T11:29:11.013 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/security 2024-04-20T11:29:11.013 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/shm 2024-04-20T11:29:11.013 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/pts 2024-04-20T11:29:11.013 INFO:teuthology.orchestra.run.smithi175.stdout:/run 2024-04-20T11:29:11.013 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/cgroup 2024-04-20T11:29:11.013 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/pstore 2024-04-20T11:29:11.013 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/bpf 2024-04-20T11:29:11.013 INFO:teuthology.orchestra.run.smithi175.stdout:/ 2024-04-20T11:29:11.013 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/selinux 2024-04-20T11:29:11.013 INFO:teuthology.orchestra.run.smithi175.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T11:29:11.013 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/mqueue 2024-04-20T11:29:11.013 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/debug 2024-04-20T11:29:11.013 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/hugepages 2024-04-20T11:29:11.013 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/tracing 2024-04-20T11:29:11.013 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/fuse/connections 2024-04-20T11:29:11.014 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/config 2024-04-20T11:29:11.014 INFO:teuthology.orchestra.run.smithi175.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T11:29:11.014 INFO:teuthology.orchestra.run.smithi175.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T11:29:11.014 INFO:teuthology.orchestra.run.smithi175.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T11:29:11.014 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T11:29:11.014 INFO:teuthology.orchestra.run.smithi175.stdout:/run/user/1000 2024-04-20T11:29:11.014 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph 2024-04-20T11:29:11.014 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-4 2024-04-20T11:29:11.014 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-5 2024-04-20T11:29:11.014 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-6 2024-04-20T11:29:11.014 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-7 2024-04-20T11:29:11.014 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns 2024-04-20T11:29:11.014 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-20T11:29:11.014 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-20T11:29:11.015 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-20T11:29:11.015 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-20T11:29:11.015 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:29:11.016 DEBUG:teuthology.orchestra.run.smithi175:> ls /sys/fs/fuse/connections 2024-04-20T11:29:11.077 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi175.stderr:2024-04-20T11:29:11.077+0000 7f3a1620c480 -1 init, newargv = 0x55febe987ff0 newargc=15 2024-04-20T11:29:11.078 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi175.stderr:ceph-fuse[30460]: starting ceph client 2024-04-20T11:29:11.087 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi175.stderr:ceph-fuse[30460]: starting fuse 2024-04-20T11:29:11.103 INFO:teuthology.orchestra.run.smithi175.stdout:42 2024-04-20T11:29:11.103 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-04-20T11:29:11.104 DEBUG:teuthology.orchestra.run.smithi175:> sudo stdin-killer -- python3 -c ' 2024-04-20T11:29:11.104 DEBUG:teuthology.orchestra.run.smithi175:> import glob 2024-04-20T11:29:11.104 DEBUG:teuthology.orchestra.run.smithi175:> import re 2024-04-20T11:29:11.104 DEBUG:teuthology.orchestra.run.smithi175:> import os 2024-04-20T11:29:11.104 DEBUG:teuthology.orchestra.run.smithi175:> import subprocess 2024-04-20T11:29:11.104 DEBUG:teuthology.orchestra.run.smithi175:> 2024-04-20T11:29:11.104 DEBUG:teuthology.orchestra.run.smithi175:> def _find_admin_socket(client_name): 2024-04-20T11:29:11.104 DEBUG:teuthology.orchestra.run.smithi175:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-04-20T11:29:11.104 DEBUG:teuthology.orchestra.run.smithi175:> files = glob.glob(asok_path) 2024-04-20T11:29:11.104 DEBUG:teuthology.orchestra.run.smithi175:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-04-20T11:29:11.104 DEBUG:teuthology.orchestra.run.smithi175:> 2024-04-20T11:29:11.104 DEBUG:teuthology.orchestra.run.smithi175:> # Given a non-glob path, it better be there 2024-04-20T11:29:11.104 DEBUG:teuthology.orchestra.run.smithi175:> if "*" not in asok_path: 2024-04-20T11:29:11.104 DEBUG:teuthology.orchestra.run.smithi175:> assert(len(files) == 1) 2024-04-20T11:29:11.105 DEBUG:teuthology.orchestra.run.smithi175:> return files[0] 2024-04-20T11:29:11.105 DEBUG:teuthology.orchestra.run.smithi175:> 2024-04-20T11:29:11.105 DEBUG:teuthology.orchestra.run.smithi175:> for f in files: 2024-04-20T11:29:11.105 DEBUG:teuthology.orchestra.run.smithi175:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-20T11:29:11.105 DEBUG:teuthology.orchestra.run.smithi175:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-20T11:29:11.105 DEBUG:teuthology.orchestra.run.smithi175:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-20T11:29:11.105 DEBUG:teuthology.orchestra.run.smithi175:> contents = proc_f.read() 2024-04-20T11:29:11.105 DEBUG:teuthology.orchestra.run.smithi175:> if mountpoint in contents: 2024-04-20T11:29:11.105 DEBUG:teuthology.orchestra.run.smithi175:> return f 2024-04-20T11:29:11.105 DEBUG:teuthology.orchestra.run.smithi175:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-20T11:29:11.105 DEBUG:teuthology.orchestra.run.smithi175:> 2024-04-20T11:29:11.105 DEBUG:teuthology.orchestra.run.smithi175:> print(_find_admin_socket("client.1")) 2024-04-20T11:29:11.105 DEBUG:teuthology.orchestra.run.smithi175:> ' 2024-04-20T11:29:11.294 INFO:teuthology.orchestra.run.smithi175.stdout:/var/run/ceph/ceph-client.1.30460.asok 2024-04-20T11:29:11.298 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:29:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T11:29:11.310 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.30460.asok 2024-04-20T11:29:11.310 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:29:11.311 DEBUG:teuthology.orchestra.run.smithi175:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.30460.asok status 2024-04-20T11:29:11.529 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-04-20T11:29:11.529 INFO:teuthology.orchestra.run.smithi175.stdout: "metadata": { 2024-04-20T11:29:11.529 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_sha1": "a8c84f5997950fd7d008204839253291d52e6eda", 2024-04-20T11:29:11.530 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", 2024-04-20T11:29:11.530 INFO:teuthology.orchestra.run.smithi175.stdout: "entity_id": "1", 2024-04-20T11:29:11.530 INFO:teuthology.orchestra.run.smithi175.stdout: "hostname": "smithi175", 2024-04-20T11:29:11.530 INFO:teuthology.orchestra.run.smithi175.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-04-20T11:29:11.530 INFO:teuthology.orchestra.run.smithi175.stdout: "pid": "30460", 2024-04-20T11:29:11.530 INFO:teuthology.orchestra.run.smithi175.stdout: "root": "/" 2024-04-20T11:29:11.530 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-20T11:29:11.530 INFO:teuthology.orchestra.run.smithi175.stdout: "dentry_count": 0, 2024-04-20T11:29:11.530 INFO:teuthology.orchestra.run.smithi175.stdout: "dentry_pinned_count": 0, 2024-04-20T11:29:11.530 INFO:teuthology.orchestra.run.smithi175.stdout: "id": 5113, 2024-04-20T11:29:11.530 INFO:teuthology.orchestra.run.smithi175.stdout: "inst": { 2024-04-20T11:29:11.530 INFO:teuthology.orchestra.run.smithi175.stdout: "name": { 2024-04-20T11:29:11.531 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "client", 2024-04-20T11:29:11.531 INFO:teuthology.orchestra.run.smithi175.stdout: "num": 5113 2024-04-20T11:29:11.531 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-20T11:29:11.531 INFO:teuthology.orchestra.run.smithi175.stdout: "addr": { 2024-04-20T11:29:11.531 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "v1", 2024-04-20T11:29:11.531 INFO:teuthology.orchestra.run.smithi175.stdout: "addr": "192.168.0.1:0", 2024-04-20T11:29:11.531 INFO:teuthology.orchestra.run.smithi175.stdout: "nonce": 614736765 2024-04-20T11:29:11.531 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-04-20T11:29:11.531 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-20T11:29:11.531 INFO:teuthology.orchestra.run.smithi175.stdout: "addr": { 2024-04-20T11:29:11.531 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "v1", 2024-04-20T11:29:11.531 INFO:teuthology.orchestra.run.smithi175.stdout: "addr": "192.168.0.1:0", 2024-04-20T11:29:11.531 INFO:teuthology.orchestra.run.smithi175.stdout: "nonce": 614736765 2024-04-20T11:29:11.531 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-20T11:29:11.531 INFO:teuthology.orchestra.run.smithi175.stdout: "inst_str": "client.5113 192.168.0.1:0/614736765", 2024-04-20T11:29:11.531 INFO:teuthology.orchestra.run.smithi175.stdout: "addr_str": "192.168.0.1:0/614736765", 2024-04-20T11:29:11.532 INFO:teuthology.orchestra.run.smithi175.stdout: "inode_count": 1, 2024-04-20T11:29:11.532 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_epoch": 24, 2024-04-20T11:29:11.532 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_epoch": 46, 2024-04-20T11:29:11.532 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_epoch_barrier": 0, 2024-04-20T11:29:11.532 INFO:teuthology.orchestra.run.smithi175.stdout: "blocklisted": false, 2024-04-20T11:29:11.532 INFO:teuthology.orchestra.run.smithi175.stdout: "fs_name": "cephfs" 2024-04-20T11:29:11.532 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-04-20T11:29:11.543 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:29:11.544 DEBUG:teuthology.orchestra.run.smithi175:> stat --file-system '--printf=%T 2024-04-20T11:29:11.544 DEBUG:teuthology.orchestra.run.smithi175:> ' -- /home/ubuntu/cephtest/mnt.1 2024-04-20T11:29:11.598 INFO:teuthology.orchestra.run.smithi175.stdout:fuseblk 2024-04-20T11:29:11.598 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-04-20T11:29:11.598 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:29:11.598 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-20T11:29:11.672 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-04-20T11:29:11.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:11.884+0000 7f1573c00640 1 Processor -- start 2024-04-20T11:29:11.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:11.884+0000 7f1573c00640 1 -- start start 2024-04-20T11:29:11.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:11.886+0000 7f1573c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f156c144dd0 0x7f156c071db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:11.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:11.886+0000 7f1573c00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f156c162c50 0x7f156c0aebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:11.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:11.886+0000 7f1573c00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f156c1632b0 con 0x7f156c0722f0 2024-04-20T11:29:11.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:11.886+0000 7f1573c00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f156c163080 con 0x7f156c162c50 2024-04-20T11:29:11.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:11.886+0000 7f1573c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f156c072730 con 0x7f156c144dd0 2024-04-20T11:29:11.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:11.886+0000 7f156be00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f156c0722f0 0x7f156c162540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:35748/0 (socket says 172.21.15.152:35748) 2024-04-20T11:29:11.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:11.886+0000 7f156be00640 1 -- 172.21.15.152:0/3213206055 learned_addr learned my addr 172.21.15.152:0/3213206055 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:29:11.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:11.886+0000 7f1571200640 1 --2- 172.21.15.152:0/3213206055 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f156c144dd0 0x7f156c071db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:11.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:11.887+0000 7f1571c00640 1 --2- 172.21.15.152:0/3213206055 >> v2:172.21.15.175:3300/0 conn(0x7f156c162c50 0x7f156c0aebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:11.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:11.887+0000 7f156b400640 1 -- 172.21.15.152:0/3213206055 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2618265825 0 0) 0x7f156c1632b0 con 0x7f156c0722f0 2024-04-20T11:29:11.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:11.887+0000 7f156b400640 1 -- 172.21.15.152:0/3213206055 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1554003520 con 0x7f156c0722f0 2024-04-20T11:29:11.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:11.887+0000 7f156b400640 1 -- 172.21.15.152:0/3213206055 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3338741366 0 0) 0x7f1554003520 con 0x7f156c0722f0 2024-04-20T11:29:11.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:11.887+0000 7f156b400640 1 -- 172.21.15.152:0/3213206055 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f156c144dd0 msgr2=0x7f156c071db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:11.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:11.887+0000 7f156b400640 1 --2- 172.21.15.152:0/3213206055 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f156c144dd0 0x7f156c071db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:11.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:11.887+0000 7f156b400640 1 -- 172.21.15.152:0/3213206055 >> v2:172.21.15.175:3300/0 conn(0x7f156c162c50 msgr2=0x7f156c0aebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:11.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:11.887+0000 7f156b400640 1 --2- 172.21.15.152:0/3213206055 >> v2:172.21.15.175:3300/0 conn(0x7f156c162c50 0x7f156c0aebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:11.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:11.887+0000 7f156b400640 1 -- 172.21.15.152:0/3213206055 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f156c0b3510 con 0x7f156c0722f0 2024-04-20T11:29:11.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:11.887+0000 7f156b400640 1 -- 172.21.15.152:0/3213206055 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f1558001670 con 0x7f156c0722f0 2024-04-20T11:29:11.890 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:11.887+0000 7f1571c00640 1 --2- 172.21.15.152:0/3213206055 >> v2:172.21.15.175:3300/0 conn(0x7f156c162c50 0x7f156c0aebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:29:11.890 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:11.887+0000 7f156b400640 1 -- 172.21.15.152:0/3213206055 >> v1:172.21.15.152:6789/0 conn(0x7f156c0722f0 legacy=0x7f156c162540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:11.890 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:11.887+0000 7f156b400640 1 --2- 172.21.15.152:0/3213206055 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f156c144dd0 0x7f1554003a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:11.890 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:11.887+0000 7f156b400640 1 --2- 172.21.15.152:0/3213206055 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1554003fa0 0x7f1554004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:11.890 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:11.887+0000 7f156b400640 1 --2- 172.21.15.152:0/3213206055 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1554004d40 0x7f15540051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:11.890 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:11.888+0000 7f156b400640 1 -- 172.21.15.152:0/3213206055 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f156c072730 con 0x7f1554004d40 2024-04-20T11:29:11.890 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:11.888+0000 7f156b400640 1 -- 172.21.15.152:0/3213206055 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f155400b9c0 con 0x7f1554003fa0 2024-04-20T11:29:11.890 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:11.888+0000 7f156b400640 1 -- 172.21.15.152:0/3213206055 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f155400bb70 con 0x7f156c144dd0 2024-04-20T11:29:11.890 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:11.888+0000 7f1571c00640 1 --2- 172.21.15.152:0/3213206055 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1554004d40 0x7f15540051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:11.890 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:11.888+0000 7f156be00640 1 --2- 172.21.15.152:0/3213206055 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1554003fa0 0x7f1554004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:11.891 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:11.888+0000 7f1571200640 1 --2- 172.21.15.152:0/3213206055 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f156c144dd0 0x7f1554003a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:11.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:11.889+0000 7f1571c00640 1 -- 172.21.15.152:0/3213206055 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f156c144dd0 msgr2=0x7f1554003a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:11.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:11.889+0000 7f1571c00640 1 --2- 172.21.15.152:0/3213206055 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f156c144dd0 0x7f1554003a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:11.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:11.889+0000 7f1571c00640 1 -- 172.21.15.152:0/3213206055 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1554003fa0 msgr2=0x7f1554004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:11.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:11.889+0000 7f1571c00640 1 --2- 172.21.15.152:0/3213206055 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1554003fa0 0x7f1554004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:11.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:11.889+0000 7f1571c00640 1 -- 172.21.15.152:0/3213206055 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1554003520 con 0x7f1554004d40 2024-04-20T11:29:11.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:11.889+0000 7f156be00640 1 --2- 172.21.15.152:0/3213206055 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1554003fa0 0x7f1554004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:29:11.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:11.890+0000 7f1571c00640 1 --2- 172.21.15.152:0/3213206055 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1554004d40 0x7f15540051c0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f156400e350 tx=0x7f1554005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3f526d3d7be2901 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:11.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:11.890+0000 7f156b400640 1 -- 172.21.15.152:0/3213206055 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1564039070 con 0x7f1554004d40 2024-04-20T11:29:11.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:11.890+0000 7f156b400640 1 -- 172.21.15.152:0/3213206055 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f156402ce00 con 0x7f1554004d40 2024-04-20T11:29:11.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:11.890+0000 7f156b400640 1 -- 172.21.15.152:0/3213206055 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1564009de0 con 0x7f1554004d40 2024-04-20T11:29:11.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:11.891+0000 7f1573c00640 1 -- 172.21.15.152:0/3213206055 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1554004d40 msgr2=0x7f15540051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:11.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:11.891+0000 7f1573c00640 1 --2- 172.21.15.152:0/3213206055 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1554004d40 0x7f15540051c0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f156400e350 tx=0x7f1554005830 comp rx=0 tx=0).stop 2024-04-20T11:29:11.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:11.891+0000 7f1571c00640 1 -- 172.21.15.152:0/3213206055 reap_dead start 2024-04-20T11:29:11.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:11.891+0000 7f1573c00640 1 -- 172.21.15.152:0/3213206055 shutdown_connections 2024-04-20T11:29:11.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:11.891+0000 7f1573c00640 1 -- 172.21.15.152:0/3213206055 >> 172.21.15.152:0/3213206055 conn(0x7f156c067cf0 msgr2=0x7f156c06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:11.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:11.891+0000 7f1573c00640 1 -- 172.21.15.152:0/3213206055 shutdown_connections 2024-04-20T11:29:11.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:11.891+0000 7f1573c00640 1 -- 172.21.15.152:0/3213206055 wait complete. 2024-04-20T11:29:11.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:11.893+0000 7f1573c00640 1 Processor -- start 2024-04-20T11:29:11.894 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:11.893+0000 7f1573c00640 1 -- start start 2024-04-20T11:29:11.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:11.894+0000 7f1573c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1554004d40 0x7f156c139030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:11.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:11.894+0000 7f1573c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f156c0b0600 0x7f156c0b0a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:11.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:11.894+0000 7f1573c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f156c1414b0 0x7f156c141930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:11.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:11.894+0000 7f1573c00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f156c0b3db0 con 0x7f156c0b0600 2024-04-20T11:29:11.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:11.894+0000 7f1573c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f156c0b3ad0 con 0x7f1554004d40 2024-04-20T11:29:11.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:11.894+0000 7f1573c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f156c1451b0 con 0x7f156c1414b0 2024-04-20T11:29:11.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:11.894+0000 7f156be00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f156c0b0600 0x7f156c0b0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:11.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:11.895+0000 7f156be00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f156c0b0600 0x7f156c0b0a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:46342/0 (socket says 172.21.15.152:46342) 2024-04-20T11:29:11.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:11.895+0000 7f156be00640 1 -- 172.21.15.152:0/1956901863 learned_addr learned my addr 172.21.15.152:0/1956901863 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:29:11.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:11.895+0000 7f1571200640 1 --2- 172.21.15.152:0/1956901863 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1554004d40 0x7f156c139030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:11.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:11.895+0000 7f1571c00640 1 --2- 172.21.15.152:0/1956901863 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f156c1414b0 0x7f156c141930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:11.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:11.895+0000 7f156be00640 1 -- 172.21.15.152:0/1956901863 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f156c1414b0 msgr2=0x7f156c141930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:11.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:11.895+0000 7f156be00640 1 --2- 172.21.15.152:0/1956901863 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f156c1414b0 0x7f156c141930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:11.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:11.895+0000 7f156be00640 1 -- 172.21.15.152:0/1956901863 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1554004d40 msgr2=0x7f156c139030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:11.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:11.895+0000 7f156be00640 1 --2- 172.21.15.152:0/1956901863 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1554004d40 0x7f156c139030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:11.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:11.895+0000 7f156be00640 1 -- 172.21.15.152:0/1956901863 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f156c13a240 con 0x7f156c0b0600 2024-04-20T11:29:11.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:11.895+0000 7f1571c00640 1 --2- 172.21.15.152:0/1956901863 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f156c1414b0 0x7f156c141930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:29:11.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:11.895+0000 7f156be00640 1 --2- 172.21.15.152:0/1956901863 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f156c0b0600 0x7f156c0b0a80 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f1558007310 tx=0x7f15580077d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:11.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:11.896+0000 7f1568c00640 1 -- 172.21.15.152:0/1956901863 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f155800fc00 con 0x7f156c0b0600 2024-04-20T11:29:11.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:11.896+0000 7f1568c00640 1 -- 172.21.15.152:0/1956901863 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1558018070 con 0x7f156c0b0600 2024-04-20T11:29:11.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:11.896+0000 7f1568c00640 1 -- 172.21.15.152:0/1956901863 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15580133f0 con 0x7f156c0b0600 2024-04-20T11:29:11.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:11.896+0000 7f1573c00640 1 -- 172.21.15.152:0/1956901863 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f156c13a520 con 0x7f156c0b0600 2024-04-20T11:29:11.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:11.896+0000 7f1573c00640 1 -- 172.21.15.152:0/1956901863 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f156c13aae0 con 0x7f156c0b0600 2024-04-20T11:29:11.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:11.897+0000 7f1573c00640 1 -- 172.21.15.152:0/1956901863 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f156c068a60 con 0x7f156c0b0600 2024-04-20T11:29:11.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:11.898+0000 7f1568c00640 1 -- 172.21.15.152:0/1956901863 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f1558013590 con 0x7f156c0b0600 2024-04-20T11:29:11.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:11.899+0000 7f1568c00640 1 --2- 172.21.15.152:0/1956901863 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f154c06cad0 0x7f154c06ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:11.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:11.899+0000 7f1568c00640 1 -- 172.21.15.152:0/1956901863 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f155808c1a0 con 0x7f156c0b0600 2024-04-20T11:29:11.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:11.900+0000 7f1571200640 1 --2- 172.21.15.152:0/1956901863 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f154c06cad0 0x7f154c06ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:11.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:11.900+0000 7f1571200640 1 --2- 172.21.15.152:0/1956901863 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f154c06cad0 0x7f154c06ef90 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f1560009b00 tx=0x7f1560008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:11.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:11.907+0000 7f1568c00640 1 -- 172.21.15.152:0/1956901863 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f155805a3e0 con 0x7f156c0b0600 2024-04-20T11:29:12.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:12.150+0000 7f1573c00640 1 -- 172.21.15.152:0/1956901863 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f156c0720b0 con 0x7f156c0b0600 2024-04-20T11:29:12.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:12.918+0000 7f1568c00640 1 -- 172.21.15.152:0/1956901863 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v25) ==== 136+0+0 (secure 0 0 0) 0x7f1558059ce0 con 0x7f156c0b0600 2024-04-20T11:29:12.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:12.921+0000 7f1573c00640 1 -- 172.21.15.152:0/1956901863 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f154c06cad0 msgr2=0x7f154c06ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:12.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:12.921+0000 7f1573c00640 1 --2- 172.21.15.152:0/1956901863 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f154c06cad0 0x7f154c06ef90 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f1560009b00 tx=0x7f1560008040 comp rx=0 tx=0).stop 2024-04-20T11:29:12.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:12.922+0000 7f1573c00640 1 -- 172.21.15.152:0/1956901863 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f156c0b0600 msgr2=0x7f156c0b0a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:12.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:12.922+0000 7f1573c00640 1 --2- 172.21.15.152:0/1956901863 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f156c0b0600 0x7f156c0b0a80 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f1558007310 tx=0x7f15580077d0 comp rx=0 tx=0).stop 2024-04-20T11:29:12.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:12.922+0000 7f1573c00640 1 -- 172.21.15.152:0/1956901863 shutdown_connections 2024-04-20T11:29:12.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:12.922+0000 7f1573c00640 1 --2- 172.21.15.152:0/1956901863 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f154c06cad0 0x7f154c06ef90 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:12.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:12.922+0000 7f1573c00640 1 --2- 172.21.15.152:0/1956901863 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f156c1414b0 0x7f156c141930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:12.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:12.922+0000 7f1573c00640 1 --2- 172.21.15.152:0/1956901863 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f156c0b0600 0x7f156c0b0a80 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:12.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:12.922+0000 7f1573c00640 1 --2- 172.21.15.152:0/1956901863 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1554004d40 0x7f156c139030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:12.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:12.922+0000 7f1573c00640 1 -- 172.21.15.152:0/1956901863 >> 172.21.15.152:0/1956901863 conn(0x7f156c067cf0 msgr2=0x7f156c142780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:12.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:12.922+0000 7f1573c00640 1 -- 172.21.15.152:0/1956901863 shutdown_connections 2024-04-20T11:29:12.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:12.922+0000 7f1573c00640 1 -- 172.21.15.152:0/1956901863 wait complete. 2024-04-20T11:29:12.943 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-04-20T11:29:13.138 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.137+0000 7f8dc5800640 1 Processor -- start 2024-04-20T11:29:13.138 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.138+0000 7f8dc5800640 1 -- start start 2024-04-20T11:29:13.140 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.139+0000 7f8dc5800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f8dc009b150 0x7f8dc00d06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:13.140 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.139+0000 7f8dc5800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8dc00d0be0 0x7f8dc00cde00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:13.140 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.139+0000 7f8dc5800640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8dc00d1280 con 0x7f8dc015ff30 2024-04-20T11:29:13.140 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.139+0000 7f8dc5800640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f8dc00cd970 con 0x7f8dc009b150 2024-04-20T11:29:13.140 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.139+0000 7f8dc5800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8dc00d1030 con 0x7f8dc00d0be0 2024-04-20T11:29:13.140 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.139+0000 7f8dbea00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f8dc015ff30 0x7f8dc009aa40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:35752/0 (socket says 172.21.15.152:35752) 2024-04-20T11:29:13.140 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.139+0000 7f8dbea00640 1 -- 172.21.15.152:0/178302399 learned_addr learned my addr 172.21.15.152:0/178302399 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:29:13.140 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.139+0000 7f8dbe000640 1 --2- 172.21.15.152:0/178302399 >> v2:172.21.15.152:3300/0 conn(0x7f8dc009b150 0x7f8dc00d06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:13.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.140+0000 7f8dbf400640 1 --2- 172.21.15.152:0/178302399 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8dc00d0be0 0x7f8dc00cde00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:13.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.140+0000 7f8dbe000640 1 -- 172.21.15.152:0/178302399 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8dc00d0be0 msgr2=0x7f8dc00cde00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:13.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.140+0000 7f8dbe000640 1 --2- 172.21.15.152:0/178302399 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8dc00d0be0 0x7f8dc00cde00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:13.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.140+0000 7f8dbe000640 1 -- 172.21.15.152:0/178302399 >> v1:172.21.15.152:6789/0 conn(0x7f8dc015ff30 legacy=0x7f8dc009aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:13.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.140+0000 7f8dbe000640 1 -- 172.21.15.152:0/178302399 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dc00cd320 con 0x7f8dc009b150 2024-04-20T11:29:13.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.140+0000 7f8dbd600640 1 -- 172.21.15.152:0/178302399 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3496260787 0 0) 0x7f8dc00d1280 con 0x7f8dc015ff30 2024-04-20T11:29:13.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.141+0000 7f8dbe000640 1 --2- 172.21.15.152:0/178302399 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8dc009b150 0x7f8dc00d06a0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f8dac0097a0 tx=0x7f8dac02b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff780487c306bf7f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:13.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.141+0000 7f8dbd600640 1 -- 172.21.15.152:0/178302399 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dac031780 con 0x7f8dc009b150 2024-04-20T11:29:13.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.142+0000 7f8dbd600640 1 -- 172.21.15.152:0/178302399 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8dac037070 con 0x7f8dc009b150 2024-04-20T11:29:13.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.142+0000 7f8dbd600640 1 -- 172.21.15.152:0/178302399 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dac031b40 con 0x7f8dc009b150 2024-04-20T11:29:13.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.142+0000 7f8dc5800640 1 -- 172.21.15.152:0/178302399 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8dc009b150 msgr2=0x7f8dc00d06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:13.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.142+0000 7f8dc5800640 1 --2- 172.21.15.152:0/178302399 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8dc009b150 0x7f8dc00d06a0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f8dac0097a0 tx=0x7f8dac02b820 comp rx=0 tx=0).stop 2024-04-20T11:29:13.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.142+0000 7f8dc5800640 1 -- 172.21.15.152:0/178302399 shutdown_connections 2024-04-20T11:29:13.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.142+0000 7f8dc5800640 1 --2- 172.21.15.152:0/178302399 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8dc00d0be0 0x7f8dc00cde00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:13.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.142+0000 7f8dc5800640 1 --2- 172.21.15.152:0/178302399 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8dc009b150 0x7f8dc00d06a0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:13.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.142+0000 7f8dc5800640 1 -- 172.21.15.152:0/178302399 >> 172.21.15.152:0/178302399 conn(0x7f8dc00b4090 msgr2=0x7f8dc00b44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:13.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.142+0000 7f8dc5800640 1 -- 172.21.15.152:0/178302399 shutdown_connections 2024-04-20T11:29:13.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.142+0000 7f8dc5800640 1 -- 172.21.15.152:0/178302399 wait complete. 2024-04-20T11:29:13.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.144+0000 7f8dc5800640 1 Processor -- start 2024-04-20T11:29:13.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.144+0000 7f8dc5800640 1 -- start start 2024-04-20T11:29:13.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.145+0000 7f8dc5800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8dc009b150 0x7f8dc014d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:13.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.145+0000 7f8dc5800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8dc00d0be0 0x7f8dc015cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:13.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.145+0000 7f8dc5800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8dc015ff30 0x7f8dc015d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:13.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.145+0000 7f8dc5800640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f8dc00ccfd0 con 0x7f8dc009b150 2024-04-20T11:29:13.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.145+0000 7f8dc5800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8dc00b4ed0 con 0x7f8dc00d0be0 2024-04-20T11:29:13.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.145+0000 7f8dc5800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8dc00af330 con 0x7f8dc015ff30 2024-04-20T11:29:13.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.145+0000 7f8dbea00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8dc009b150 0x7f8dc014d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:13.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.145+0000 7f8dbea00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8dc009b150 0x7f8dc014d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:46362/0 (socket says 172.21.15.152:46362) 2024-04-20T11:29:13.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.145+0000 7f8dbea00640 1 -- 172.21.15.152:0/141879650 learned_addr learned my addr 172.21.15.152:0/141879650 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:29:13.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.146+0000 7f8dbe000640 1 --2- 172.21.15.152:0/141879650 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8dc00d0be0 0x7f8dc015cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:13.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.146+0000 7f8dbf400640 1 --2- 172.21.15.152:0/141879650 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8dc015ff30 0x7f8dc015d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:13.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.146+0000 7f8dbea00640 1 -- 172.21.15.152:0/141879650 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8dc015ff30 msgr2=0x7f8dc015d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:13.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.146+0000 7f8dbea00640 1 --2- 172.21.15.152:0/141879650 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8dc015ff30 0x7f8dc015d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:13.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.146+0000 7f8dbea00640 1 -- 172.21.15.152:0/141879650 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8dc00d0be0 msgr2=0x7f8dc015cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:13.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.146+0000 7f8dbea00640 1 --2- 172.21.15.152:0/141879650 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8dc00d0be0 0x7f8dc015cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:13.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.146+0000 7f8dbea00640 1 -- 172.21.15.152:0/141879650 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dc014fc90 con 0x7f8dc009b150 2024-04-20T11:29:13.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.146+0000 7f8dbea00640 1 --2- 172.21.15.152:0/141879650 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8dc009b150 0x7f8dc014d370 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f8db000c4d0 tx=0x7f8db000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:13.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.146+0000 7f8db6a00640 1 -- 172.21.15.152:0/141879650 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8db0017070 con 0x7f8dc009b150 2024-04-20T11:29:13.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.147+0000 7f8db6a00640 1 -- 172.21.15.152:0/141879650 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8db0005d00 con 0x7f8dc009b150 2024-04-20T11:29:13.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.147+0000 7f8dc5800640 1 -- 172.21.15.152:0/141879650 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8dc014ff70 con 0x7f8dc009b150 2024-04-20T11:29:13.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.147+0000 7f8db6a00640 1 -- 172.21.15.152:0/141879650 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8db00072b0 con 0x7f8dc009b150 2024-04-20T11:29:13.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.147+0000 7f8dc5800640 1 -- 172.21.15.152:0/141879650 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8dc00c7ce0 con 0x7f8dc009b150 2024-04-20T11:29:13.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.148+0000 7f8dc5800640 1 -- 172.21.15.152:0/141879650 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8dc00689f0 con 0x7f8dc009b150 2024-04-20T11:29:13.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.149+0000 7f8db6a00640 1 -- 172.21.15.152:0/141879650 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f8db001a070 con 0x7f8dc009b150 2024-04-20T11:29:13.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.150+0000 7f8db6a00640 1 --2- 172.21.15.152:0/141879650 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f8da006cad0 0x7f8da006ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:13.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.150+0000 7f8db6a00640 1 -- 172.21.15.152:0/141879650 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f8db008b2a0 con 0x7f8dc009b150 2024-04-20T11:29:13.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.150+0000 7f8dbe000640 1 --2- 172.21.15.152:0/141879650 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f8da006cad0 0x7f8da006ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:13.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.151+0000 7f8dbe000640 1 --2- 172.21.15.152:0/141879650 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f8da006cad0 0x7f8da006ef90 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f8dac036000 tx=0x7f8dac008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:13.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.154+0000 7f8db6a00640 1 -- 172.21.15.152:0/141879650 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f8db00593b0 con 0x7f8dc009b150 2024-04-20T11:29:13.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.387+0000 7f8dc5800640 1 -- 172.21.15.152:0/141879650 --> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f8dc0150630 con 0x7f8da006cad0 2024-04-20T11:29:13.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.389+0000 7f8db6a00640 1 -- 172.21.15.152:0/141879650 <== mgr.4113 v2:172.21.15.152:6832/2285110240 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f8dc0150630 con 0x7f8da006cad0 2024-04-20T11:29:13.390 INFO:teuthology.orchestra.run.smithi152.stdout:[ 2024-04-20T11:29:13.390 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:29:13.390 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "cephfs" 2024-04-20T11:29:13.390 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:29:13.390 INFO:teuthology.orchestra.run.smithi152.stdout:] 2024-04-20T11:29:13.393 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.392+0000 7f8dc5800640 1 -- 172.21.15.152:0/141879650 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f8da006cad0 msgr2=0x7f8da006ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:13.393 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.392+0000 7f8dc5800640 1 --2- 172.21.15.152:0/141879650 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f8da006cad0 0x7f8da006ef90 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f8dac036000 tx=0x7f8dac008030 comp rx=0 tx=0).stop 2024-04-20T11:29:13.393 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.392+0000 7f8dc5800640 1 -- 172.21.15.152:0/141879650 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8dc009b150 msgr2=0x7f8dc014d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:13.393 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.392+0000 7f8dc5800640 1 --2- 172.21.15.152:0/141879650 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8dc009b150 0x7f8dc014d370 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f8db000c4d0 tx=0x7f8db000c990 comp rx=0 tx=0).stop 2024-04-20T11:29:13.393 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.392+0000 7f8dc5800640 1 -- 172.21.15.152:0/141879650 shutdown_connections 2024-04-20T11:29:13.393 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.392+0000 7f8dc5800640 1 --2- 172.21.15.152:0/141879650 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f8da006cad0 0x7f8da006ef90 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:13.393 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.392+0000 7f8dc5800640 1 --2- 172.21.15.152:0/141879650 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8dc015ff30 0x7f8dc015d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:13.393 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.392+0000 7f8dc5800640 1 --2- 172.21.15.152:0/141879650 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8dc00d0be0 0x7f8dc015cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:13.393 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.392+0000 7f8dc5800640 1 --2- 172.21.15.152:0/141879650 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8dc009b150 0x7f8dc014d370 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:13.393 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.392+0000 7f8dc5800640 1 -- 172.21.15.152:0/141879650 >> 172.21.15.152:0/141879650 conn(0x7f8dc00b4090 msgr2=0x7f8dc015e170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:13.394 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.392+0000 7f8dc5800640 1 -- 172.21.15.152:0/141879650 shutdown_connections 2024-04-20T11:29:13.394 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.392+0000 7f8dc5800640 1 -- 172.21.15.152:0/141879650 wait complete. 2024-04-20T11:29:13.411 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-20T11:29:13.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.615+0000 7f41afe00640 1 Processor -- start 2024-04-20T11:29:13.616 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.615+0000 7f41afe00640 1 -- start start 2024-04-20T11:29:13.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.617+0000 7f41afe00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f41b0071cd0 0x7f41b00720b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:13.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.617+0000 7f41afe00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41b006fb40 con 0x7f41b0141450 2024-04-20T11:29:13.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.617+0000 7f41afe00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41b00725f0 con 0x7f41b0160220 2024-04-20T11:29:13.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.617+0000 7f41afe00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f41b013f8e0 con 0x7f41b0071cd0 2024-04-20T11:29:13.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.617+0000 7f41af400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f41b0141450 0x7f41b01418e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:35766/0 (socket says 172.21.15.152:35766) 2024-04-20T11:29:13.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.617+0000 7f41af400640 1 -- 172.21.15.152:0/2577347247 learned_addr learned my addr 172.21.15.152:0/2577347247 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:29:13.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.618+0000 7f41aea00640 1 --2- 172.21.15.152:0/2577347247 >> v2:172.21.15.175:3300/0 conn(0x7f41b0071cd0 0x7f41b00720b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:13.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.618+0000 7f41ad600640 1 -- 172.21.15.152:0/2577347247 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1216887613 0 0) 0x7f41b006fb40 con 0x7f41b0141450 2024-04-20T11:29:13.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.618+0000 7f41ad600640 1 -- 172.21.15.152:0/2577347247 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4194003520 con 0x7f41b0141450 2024-04-20T11:29:13.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.618+0000 7f41ad600640 1 -- 172.21.15.152:0/2577347247 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2161478386 0 0) 0x7f4194003520 con 0x7f41b0141450 2024-04-20T11:29:13.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.618+0000 7f41ad600640 1 -- 172.21.15.152:0/2577347247 >> v2:172.21.15.175:3300/0 conn(0x7f41b0071cd0 msgr2=0x7f41b00720b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:13.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.618+0000 7f41ad600640 1 --2- 172.21.15.152:0/2577347247 >> v2:172.21.15.175:3300/0 conn(0x7f41b0071cd0 0x7f41b00720b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:13.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.618+0000 7f41ad600640 1 -- 172.21.15.152:0/2577347247 >> v1:172.21.15.175:6789/0 conn(0x7f41b0160220 legacy=0x7f41b01606b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:13.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.618+0000 7f41ad600640 1 -- 172.21.15.152:0/2577347247 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41b0160f90 con 0x7f41b0141450 2024-04-20T11:29:13.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.618+0000 7f41ad600640 1 -- 172.21.15.152:0/2577347247 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f41a8001670 con 0x7f41b0141450 2024-04-20T11:29:13.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.618+0000 7f41ad600640 1 -- 172.21.15.152:0/2577347247 >> v1:172.21.15.152:6789/0 conn(0x7f41b0141450 legacy=0x7f41b01418e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:13.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.618+0000 7f41ad600640 1 --2- 172.21.15.152:0/2577347247 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f41940039b0 0x7f4194003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:13.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.618+0000 7f41ad600640 1 --2- 172.21.15.152:0/2577347247 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4194004800 0x7f4194004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:13.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.618+0000 7f41ad600640 1 --2- 172.21.15.152:0/2577347247 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41940051c0 0x7f419400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:13.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.618+0000 7f41ad600640 1 -- 172.21.15.152:0/2577347247 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f41b013f8e0 con 0x7f4194004800 2024-04-20T11:29:13.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.618+0000 7f41ad600640 1 -- 172.21.15.152:0/2577347247 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f419400c160 con 0x7f41940051c0 2024-04-20T11:29:13.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.618+0000 7f41ad600640 1 -- 172.21.15.152:0/2577347247 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f419400c360 con 0x7f41940039b0 2024-04-20T11:29:13.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.619+0000 7f41ae000640 1 --2- 172.21.15.152:0/2577347247 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4194004800 0x7f4194004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:13.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.619+0000 7f41af400640 1 --2- 172.21.15.152:0/2577347247 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41940051c0 0x7f419400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:13.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.619+0000 7f41ae000640 1 -- 172.21.15.152:0/2577347247 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f41940039b0 msgr2=0x7f4194003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:13.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.619+0000 7f41aea00640 1 --2- 172.21.15.152:0/2577347247 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f41940039b0 0x7f4194003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:13.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.619+0000 7f41ae000640 1 --2- 172.21.15.152:0/2577347247 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f41940039b0 0x7f4194003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:13.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.619+0000 7f41ae000640 1 -- 172.21.15.152:0/2577347247 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41940051c0 msgr2=0x7f419400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:13.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.619+0000 7f41ae000640 1 --2- 172.21.15.152:0/2577347247 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41940051c0 0x7f419400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:13.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.619+0000 7f41ae000640 1 -- 172.21.15.152:0/2577347247 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4194003520 con 0x7f4194004800 2024-04-20T11:29:13.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.619+0000 7f41af400640 1 --2- 172.21.15.152:0/2577347247 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41940051c0 0x7f419400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:29:13.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.619+0000 7f41af400640 1 -- 172.21.15.152:0/2577347247 reap_dead start 2024-04-20T11:29:13.623 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.620+0000 7f41ae000640 1 --2- 172.21.15.152:0/2577347247 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4194004800 0x7f4194004c80 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f419c001750 tx=0x7f4194005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ad5babaa058156c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:13.623 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.620+0000 7f41ad600640 1 -- 172.21.15.152:0/2577347247 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f419c001370 con 0x7f4194004800 2024-04-20T11:29:13.623 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.620+0000 7f41ad600640 1 -- 172.21.15.152:0/2577347247 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f419c0052d0 con 0x7f4194004800 2024-04-20T11:29:13.623 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.620+0000 7f41ad600640 1 -- 172.21.15.152:0/2577347247 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f419c003830 con 0x7f4194004800 2024-04-20T11:29:13.623 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.621+0000 7f41afe00640 1 -- 172.21.15.152:0/2577347247 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4194004800 msgr2=0x7f4194004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:13.623 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.621+0000 7f41afe00640 1 --2- 172.21.15.152:0/2577347247 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4194004800 0x7f4194004c80 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f419c001750 tx=0x7f4194005830 comp rx=0 tx=0).stop 2024-04-20T11:29:13.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.621+0000 7f41afe00640 1 -- 172.21.15.152:0/2577347247 shutdown_connections 2024-04-20T11:29:13.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.621+0000 7f41afe00640 1 --2- 172.21.15.152:0/2577347247 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4194004800 0x7f4194004c80 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:13.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.621+0000 7f41afe00640 1 -- 172.21.15.152:0/2577347247 >> 172.21.15.152:0/2577347247 conn(0x7f41b0067ae0 msgr2=0x7f41b006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:13.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.621+0000 7f41afe00640 1 -- 172.21.15.152:0/2577347247 shutdown_connections 2024-04-20T11:29:13.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.621+0000 7f41afe00640 1 -- 172.21.15.152:0/2577347247 wait complete. 2024-04-20T11:29:13.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.623+0000 7f41afe00640 1 Processor -- start 2024-04-20T11:29:13.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.623+0000 7f41afe00640 1 -- start start 2024-04-20T11:29:13.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.624+0000 7f41afe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4194004800 0x7f41b013cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:13.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.624+0000 7f41afe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f41b00aa750 0x7f41b00b35b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:13.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.624+0000 7f41afe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f41b00b3af0 0x7f41b015b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:13.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.624+0000 7f41afe00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f41b006e350 con 0x7f41b00aa750 2024-04-20T11:29:13.627 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.624+0000 7f41afe00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f41b006e070 con 0x7f4194004800 2024-04-20T11:29:13.627 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.624+0000 7f41afe00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f41b006e630 con 0x7f41b00b3af0 2024-04-20T11:29:13.627 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.624+0000 7f41ae000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f41b00aa750 0x7f41b00b35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:13.627 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.624+0000 7f41ae000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f41b00aa750 0x7f41b00b35b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:46386/0 (socket says 172.21.15.152:46386) 2024-04-20T11:29:13.627 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.624+0000 7f41ae000640 1 -- 172.21.15.152:0/857795974 learned_addr learned my addr 172.21.15.152:0/857795974 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:29:13.627 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.624+0000 7f41aea00640 1 --2- 172.21.15.152:0/857795974 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4194004800 0x7f41b013cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:13.627 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.624+0000 7f41af400640 1 --2- 172.21.15.152:0/857795974 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f41b00b3af0 0x7f41b015b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:13.627 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.624+0000 7f41ae000640 1 -- 172.21.15.152:0/857795974 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f41b00b3af0 msgr2=0x7f41b015b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:13.627 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.624+0000 7f41ae000640 1 --2- 172.21.15.152:0/857795974 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f41b00b3af0 0x7f41b015b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:13.627 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.624+0000 7f41ae000640 1 -- 172.21.15.152:0/857795974 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4194004800 msgr2=0x7f41b013cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:13.627 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.624+0000 7f41ae000640 1 --2- 172.21.15.152:0/857795974 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4194004800 0x7f41b013cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:13.627 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.624+0000 7f41ae000640 1 -- 172.21.15.152:0/857795974 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41b00b3f90 con 0x7f41b00aa750 2024-04-20T11:29:13.627 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.625+0000 7f41aea00640 1 --2- 172.21.15.152:0/857795974 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4194004800 0x7f41b013cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:29:13.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.625+0000 7f41ae000640 1 --2- 172.21.15.152:0/857795974 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f41b00aa750 0x7f41b00b35b0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f419c034040 tx=0x7f419c001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:13.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.625+0000 7f41a6a00640 1 -- 172.21.15.152:0/857795974 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f419c041070 con 0x7f41b00aa750 2024-04-20T11:29:13.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.625+0000 7f41a6a00640 1 -- 172.21.15.152:0/857795974 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f419c005480 con 0x7f41b00aa750 2024-04-20T11:29:13.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.625+0000 7f41a6a00640 1 -- 172.21.15.152:0/857795974 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f419c007a30 con 0x7f41b00aa750 2024-04-20T11:29:13.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.625+0000 7f41afe00640 1 -- 172.21.15.152:0/857795974 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41b00b4270 con 0x7f41b00aa750 2024-04-20T11:29:13.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.625+0000 7f41afe00640 1 -- 172.21.15.152:0/857795974 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41b00b4830 con 0x7f41b00aa750 2024-04-20T11:29:13.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.626+0000 7f41afe00640 1 -- 172.21.15.152:0/857795974 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41b0075e60 con 0x7f41b00aa750 2024-04-20T11:29:13.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.627+0000 7f41a6a00640 1 -- 172.21.15.152:0/857795974 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f419c044070 con 0x7f41b00aa750 2024-04-20T11:29:13.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.628+0000 7f41a6a00640 1 --2- 172.21.15.152:0/857795974 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f419006caa0 0x7f419006ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:13.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.628+0000 7f41a6a00640 1 -- 172.21.15.152:0/857795974 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f419c0adb00 con 0x7f41b00aa750 2024-04-20T11:29:13.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.628+0000 7f41aea00640 1 --2- 172.21.15.152:0/857795974 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f419006caa0 0x7f419006ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:13.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.629+0000 7f41aea00640 1 --2- 172.21.15.152:0/857795974 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f419006caa0 0x7f419006ef60 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f41b013e8d0 tx=0x7f41a000b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:13.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.633+0000 7f41a6a00640 1 -- 172.21.15.152:0/857795974 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f419c07bb00 con 0x7f41b00aa750 2024-04-20T11:29:13.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.835+0000 7f41afe00640 1 -- 172.21.15.152:0/857795974 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f41b013e340 con 0x7f41b00aa750 2024-04-20T11:29:13.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.836+0000 7f41a6a00640 1 -- 172.21.15.152:0/857795974 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) ==== 122+0+0 (secure 0 0 0) 0x7f419c07b400 con 0x7f41b00aa750 2024-04-20T11:29:13.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.839+0000 7f41afe00640 1 -- 172.21.15.152:0/857795974 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f419006caa0 msgr2=0x7f419006ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:13.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.839+0000 7f41afe00640 1 --2- 172.21.15.152:0/857795974 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f419006caa0 0x7f419006ef60 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f41b013e8d0 tx=0x7f41a000b040 comp rx=0 tx=0).stop 2024-04-20T11:29:13.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.839+0000 7f41afe00640 1 -- 172.21.15.152:0/857795974 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f41b00aa750 msgr2=0x7f41b00b35b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:13.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.839+0000 7f41afe00640 1 --2- 172.21.15.152:0/857795974 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f41b00aa750 0x7f41b00b35b0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f419c034040 tx=0x7f419c001120 comp rx=0 tx=0).stop 2024-04-20T11:29:13.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.839+0000 7f41afe00640 1 -- 172.21.15.152:0/857795974 shutdown_connections 2024-04-20T11:29:13.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.839+0000 7f41afe00640 1 --2- 172.21.15.152:0/857795974 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f419006caa0 0x7f419006ef60 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:13.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.840+0000 7f41afe00640 1 --2- 172.21.15.152:0/857795974 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f41b00b3af0 0x7f41b015b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:13.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.840+0000 7f41afe00640 1 --2- 172.21.15.152:0/857795974 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f41b00aa750 0x7f41b00b35b0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:13.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.840+0000 7f41afe00640 1 --2- 172.21.15.152:0/857795974 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4194004800 0x7f41b013cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:13.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.840+0000 7f41afe00640 1 -- 172.21.15.152:0/857795974 >> 172.21.15.152:0/857795974 conn(0x7f41b0067ae0 msgr2=0x7f41b013d6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:13.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.840+0000 7f41afe00640 1 -- 172.21.15.152:0/857795974 shutdown_connections 2024-04-20T11:29:13.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:13.840+0000 7f41afe00640 1 -- 172.21.15.152:0/857795974 wait complete. 2024-04-20T11:29:13.859 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-20T11:29:13.860 DEBUG:teuthology.orchestra.run.smithi152:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:29:13.901 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:29:14.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.147+0000 7f0434a00640 1 Processor -- start 2024-04-20T11:29:14.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.147+0000 7f0434a00640 1 -- start start 2024-04-20T11:29:14.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.149+0000 7f0434a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f043006eb40 0x7f043006f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:14.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.149+0000 7f0434a00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f043006f880 0x7f043015cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:14.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.149+0000 7f0434a00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04300b5770 con 0x7f043015b5e0 2024-04-20T11:29:14.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.149+0000 7f0434a00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f043006fcd0 con 0x7f043006f880 2024-04-20T11:29:14.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.149+0000 7f0434a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f043006de10 con 0x7f043006eb40 2024-04-20T11:29:14.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.150+0000 7f0427e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f043006eb40 0x7f043006f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:14.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.150+0000 7f0427e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f043006eb40 0x7f043006f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.152:36102/0 (socket says 172.21.15.152:36102) 2024-04-20T11:29:14.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.150+0000 7f0427e00640 1 -- 172.21.15.152:0/4000903841 learned_addr learned my addr 172.21.15.152:0/4000903841 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:29:14.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.150+0000 7f042e800640 1 --2- 172.21.15.152:0/4000903841 >> v2:172.21.15.175:3300/0 conn(0x7f043006f880 0x7f043015cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:14.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.151+0000 7f0427e00640 1 -- 172.21.15.152:0/4000903841 >> v2:172.21.15.175:3300/0 conn(0x7f043006f880 msgr2=0x7f043015cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:14.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.151+0000 7f0427e00640 1 --2- 172.21.15.152:0/4000903841 >> v2:172.21.15.175:3300/0 conn(0x7f043006f880 0x7f043015cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:14.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.151+0000 7f0427e00640 1 -- 172.21.15.152:0/4000903841 >> v1:172.21.15.175:6789/0 conn(0x7f043015b5e0 legacy=0x7f04300b4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:14.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.151+0000 7f0427e00640 1 -- 172.21.15.152:0/4000903841 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f043006e060 con 0x7f043006eb40 2024-04-20T11:29:14.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.151+0000 7f042e800640 1 --2- 172.21.15.152:0/4000903841 >> v2:172.21.15.175:3300/0 conn(0x7f043006f880 0x7f043015cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:29:14.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.152+0000 7f0427e00640 1 --2- 172.21.15.152:0/4000903841 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f043006eb40 0x7f043006f340 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0418009870 tx=0x7f041802b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e95da0405e1b3d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:14.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.153+0000 7f042d400640 1 -- 172.21.15.152:0/4000903841 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0418034070 con 0x7f043006eb40 2024-04-20T11:29:14.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.153+0000 7f042d400640 1 -- 172.21.15.152:0/4000903841 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0418030840 con 0x7f043006eb40 2024-04-20T11:29:14.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.153+0000 7f042d400640 1 -- 172.21.15.152:0/4000903841 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0418030c10 con 0x7f043006eb40 2024-04-20T11:29:14.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.153+0000 7f0434a00640 1 -- 172.21.15.152:0/4000903841 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f043006eb40 msgr2=0x7f043006f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:14.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.153+0000 7f0434a00640 1 --2- 172.21.15.152:0/4000903841 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f043006eb40 0x7f043006f340 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0418009870 tx=0x7f041802b980 comp rx=0 tx=0).stop 2024-04-20T11:29:14.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.153+0000 7f0434a00640 1 -- 172.21.15.152:0/4000903841 shutdown_connections 2024-04-20T11:29:14.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.153+0000 7f0434a00640 1 --2- 172.21.15.152:0/4000903841 >> v2:172.21.15.175:3300/0 conn(0x7f043006f880 0x7f043015cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:14.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.153+0000 7f0434a00640 1 --2- 172.21.15.152:0/4000903841 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f043006eb40 0x7f043006f340 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:14.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.153+0000 7f0434a00640 1 -- 172.21.15.152:0/4000903841 >> 172.21.15.152:0/4000903841 conn(0x7f0430067cf0 msgr2=0x7f043006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:14.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.153+0000 7f0434a00640 1 -- 172.21.15.152:0/4000903841 shutdown_connections 2024-04-20T11:29:14.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.153+0000 7f0434a00640 1 -- 172.21.15.152:0/4000903841 wait complete. 2024-04-20T11:29:14.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.156+0000 7f0434a00640 1 Processor -- start 2024-04-20T11:29:14.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.156+0000 7f0434a00640 1 -- start start 2024-04-20T11:29:14.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.157+0000 7f0434a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f043006eb40 0x7f0430148de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:14.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.157+0000 7f0434a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f043006f880 0x7f04301e5820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:14.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.157+0000 7f0434a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f043015b5e0 0x7f0430159a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:14.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.157+0000 7f0434a00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f043015c9d0 con 0x7f043006eb40 2024-04-20T11:29:14.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.157+0000 7f0434a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f043006e620 con 0x7f043015b5e0 2024-04-20T11:29:14.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.157+0000 7f0434a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f04300b42d0 con 0x7f043006f880 2024-04-20T11:29:14.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.157+0000 7f042de00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f043006eb40 0x7f0430148de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:14.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.157+0000 7f042de00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f043006eb40 0x7f0430148de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:46392/0 (socket says 172.21.15.152:46392) 2024-04-20T11:29:14.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.157+0000 7f042de00640 1 -- 172.21.15.152:0/1108254383 learned_addr learned my addr 172.21.15.152:0/1108254383 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:29:14.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.158+0000 7f042e800640 1 --2- 172.21.15.152:0/1108254383 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f043015b5e0 0x7f0430159a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:14.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.158+0000 7f0427e00640 1 --2- 172.21.15.152:0/1108254383 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f043006f880 0x7f04301e5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:14.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.158+0000 7f042de00640 1 -- 172.21.15.152:0/1108254383 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f043006f880 msgr2=0x7f04301e5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:14.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.158+0000 7f042de00640 1 --2- 172.21.15.152:0/1108254383 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f043006f880 0x7f04301e5820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:14.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.158+0000 7f042de00640 1 -- 172.21.15.152:0/1108254383 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f043015b5e0 msgr2=0x7f0430159a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:14.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.158+0000 7f042de00640 1 --2- 172.21.15.152:0/1108254383 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f043015b5e0 0x7f0430159a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:14.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.158+0000 7f042de00640 1 -- 172.21.15.152:0/1108254383 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04301e61b0 con 0x7f043006eb40 2024-04-20T11:29:14.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.158+0000 7f042de00640 1 --2- 172.21.15.152:0/1108254383 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f043006eb40 0x7f0430148de0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f042000c3c0 tx=0x7f042000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:14.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.158+0000 7f0426000640 1 -- 172.21.15.152:0/1108254383 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04200052d0 con 0x7f043006eb40 2024-04-20T11:29:14.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.158+0000 7f0426000640 1 -- 172.21.15.152:0/1108254383 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0420005490 con 0x7f043006eb40 2024-04-20T11:29:14.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.158+0000 7f0426000640 1 -- 172.21.15.152:0/1108254383 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0420007260 con 0x7f043006eb40 2024-04-20T11:29:14.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.158+0000 7f0434a00640 1 -- 172.21.15.152:0/1108254383 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04301e5d60 con 0x7f043006eb40 2024-04-20T11:29:14.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.159+0000 7f0434a00640 1 -- 172.21.15.152:0/1108254383 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04301f6330 con 0x7f043006eb40 2024-04-20T11:29:14.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.160+0000 7f0434a00640 1 -- 172.21.15.152:0/1108254383 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04301e6380 con 0x7f043006eb40 2024-04-20T11:29:14.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.161+0000 7f0426000640 1 -- 172.21.15.152:0/1108254383 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f0420007400 con 0x7f043006eb40 2024-04-20T11:29:14.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.161+0000 7f0426000640 1 --2- 172.21.15.152:0/1108254383 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f040c06caa0 0x7f040c06ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:14.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.162+0000 7f0426000640 1 -- 172.21.15.152:0/1108254383 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f042008b400 con 0x7f043006eb40 2024-04-20T11:29:14.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.162+0000 7f0427e00640 1 --2- 172.21.15.152:0/1108254383 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f040c06caa0 0x7f040c06ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:14.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.162+0000 7f0427e00640 1 --2- 172.21.15.152:0/1108254383 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f040c06caa0 0x7f040c06ef60 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f0418036000 tx=0x7f041803b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:14.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.166+0000 7f0426000640 1 -- 172.21.15.152:0/1108254383 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f0420059510 con 0x7f043006eb40 2024-04-20T11:29:14.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.368+0000 7f0434a00640 1 -- 172.21.15.152:0/1108254383 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f04300689f0 con 0x7f043006eb40 2024-04-20T11:29:14.370 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.369+0000 7f0426000640 1 -- 172.21.15.152:0/1108254383 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) ==== 74+0+16673 (secure 0 0 0) 0x7f0420058e10 con 0x7f043006eb40 2024-04-20T11:29:14.371 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:29:14.371 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":46,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:29:01.812130+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:28:58.007718+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-20T11:28:59.260647+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:29:14.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.372+0000 7f0434a00640 1 -- 172.21.15.152:0/1108254383 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f040c06caa0 msgr2=0x7f040c06ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:14.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.372+0000 7f0434a00640 1 --2- 172.21.15.152:0/1108254383 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f040c06caa0 0x7f040c06ef60 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f0418036000 tx=0x7f041803b040 comp rx=0 tx=0).stop 2024-04-20T11:29:14.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.372+0000 7f0434a00640 1 -- 172.21.15.152:0/1108254383 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f043006eb40 msgr2=0x7f0430148de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:14.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.372+0000 7f0434a00640 1 --2- 172.21.15.152:0/1108254383 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f043006eb40 0x7f0430148de0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f042000c3c0 tx=0x7f042000c880 comp rx=0 tx=0).stop 2024-04-20T11:29:14.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.372+0000 7f0434a00640 1 -- 172.21.15.152:0/1108254383 shutdown_connections 2024-04-20T11:29:14.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.372+0000 7f0434a00640 1 --2- 172.21.15.152:0/1108254383 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f040c06caa0 0x7f040c06ef60 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:14.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.372+0000 7f0434a00640 1 --2- 172.21.15.152:0/1108254383 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f043015b5e0 0x7f0430159a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:14.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.372+0000 7f0434a00640 1 --2- 172.21.15.152:0/1108254383 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f043006f880 0x7f04301e5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:14.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.373+0000 7f0434a00640 1 --2- 172.21.15.152:0/1108254383 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f043006eb40 0x7f0430148de0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:14.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.373+0000 7f0434a00640 1 -- 172.21.15.152:0/1108254383 >> 172.21.15.152:0/1108254383 conn(0x7f0430067cf0 msgr2=0x7f04300b00e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:14.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.373+0000 7f0434a00640 1 -- 172.21.15.152:0/1108254383 shutdown_connections 2024-04-20T11:29:14.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.373+0000 7f0434a00640 1 -- 172.21.15.152:0/1108254383 wait complete. 2024-04-20T11:29:14.391 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T11:26:47.246571+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T11:28:58.007718+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T11:28:59.260647+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T11:29:14.392 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T11:29:14.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.597+0000 7ff6adc00640 1 Processor -- start 2024-04-20T11:29:14.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.597+0000 7ff6adc00640 1 -- start start 2024-04-20T11:29:14.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.599+0000 7ff6adc00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ff6a815e5d0 0x7ff6a8162440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:14.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.599+0000 7ff6adc00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff6a80b3fa0 0x7ff6a80af490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:14.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.599+0000 7ff6adc00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7ff6a815e9b0 0x7ff6a80af9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:14.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.599+0000 7ff6adc00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7ff6a80adc30 con 0x7ff6a815e9b0 2024-04-20T11:29:14.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.599+0000 7ff6adc00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7ff6a8162a70 con 0x7ff6a815e5d0 2024-04-20T11:29:14.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.599+0000 7ff6adc00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff6a8162d50 con 0x7ff6a80b3fa0 2024-04-20T11:29:14.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.599+0000 7ff6a7400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7ff6a815e9b0 0x7ff6a80af9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:14.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.599+0000 7ff6a7400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7ff6a815e9b0 0x7ff6a80af9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:46400/0 (socket says 172.21.15.152:46400) 2024-04-20T11:29:14.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.599+0000 7ff6a7400640 1 -- 172.21.15.152:0/2331741303 learned_addr learned my addr 172.21.15.152:0/2331741303 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:29:14.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.600+0000 7ff6a6a00640 1 --2- 172.21.15.152:0/2331741303 >> v2:172.21.15.175:3300/0 conn(0x7ff6a815e5d0 0x7ff6a8162440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:14.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.600+0000 7ff6a6000640 1 --2- 172.21.15.152:0/2331741303 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff6a80b3fa0 0x7ff6a80af490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:14.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.600+0000 7ff6a7400640 1 -- 172.21.15.152:0/2331741303 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff6a80b3fa0 msgr2=0x7ff6a80af490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:14.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.600+0000 7ff6a7400640 1 --2- 172.21.15.152:0/2331741303 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff6a80b3fa0 0x7ff6a80af490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:14.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.600+0000 7ff6a7400640 1 -- 172.21.15.152:0/2331741303 >> v2:172.21.15.175:3300/0 conn(0x7ff6a815e5d0 msgr2=0x7ff6a8162440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:14.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.600+0000 7ff6a7400640 1 --2- 172.21.15.152:0/2331741303 >> v2:172.21.15.175:3300/0 conn(0x7ff6a815e5d0 0x7ff6a8162440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:14.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.600+0000 7ff6a7400640 1 -- 172.21.15.152:0/2331741303 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6a8163030 con 0x7ff6a815e9b0 2024-04-20T11:29:14.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.600+0000 7ff6a6a00640 1 --2- 172.21.15.152:0/2331741303 >> v2:172.21.15.175:3300/0 conn(0x7ff6a815e5d0 0x7ff6a8162440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:29:14.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.600+0000 7ff6a6000640 1 --2- 172.21.15.152:0/2331741303 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff6a80b3fa0 0x7ff6a80af490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:29:14.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.601+0000 7ff6a7400640 1 --2- 172.21.15.152:0/2331741303 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff6a815e9b0 0x7ff6a80af9d0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7ff6a806ff00 tx=0x7ff6a002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d834560147383db server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:14.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.601+0000 7ff6a5600640 1 -- 172.21.15.152:0/2331741303 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6a0035020 con 0x7ff6a815e9b0 2024-04-20T11:29:14.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.601+0000 7ff6a5600640 1 -- 172.21.15.152:0/2331741303 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6a0031870 con 0x7ff6a815e9b0 2024-04-20T11:29:14.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.601+0000 7ff6a5600640 1 -- 172.21.15.152:0/2331741303 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6a0031c40 con 0x7ff6a815e9b0 2024-04-20T11:29:14.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.602+0000 7ff6adc00640 1 -- 172.21.15.152:0/2331741303 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff6a815e9b0 msgr2=0x7ff6a80af9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:14.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.602+0000 7ff6adc00640 1 --2- 172.21.15.152:0/2331741303 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff6a815e9b0 0x7ff6a80af9d0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7ff6a806ff00 tx=0x7ff6a002b910 comp rx=0 tx=0).stop 2024-04-20T11:29:14.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.602+0000 7ff6adc00640 1 -- 172.21.15.152:0/2331741303 shutdown_connections 2024-04-20T11:29:14.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.602+0000 7ff6adc00640 1 --2- 172.21.15.152:0/2331741303 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff6a815e9b0 0x7ff6a80af9d0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:14.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.602+0000 7ff6adc00640 1 --2- 172.21.15.152:0/2331741303 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff6a80b3fa0 0x7ff6a80af490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:14.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.602+0000 7ff6adc00640 1 --2- 172.21.15.152:0/2331741303 >> v2:172.21.15.175:3300/0 conn(0x7ff6a815e5d0 0x7ff6a8162440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:14.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.602+0000 7ff6adc00640 1 -- 172.21.15.152:0/2331741303 >> 172.21.15.152:0/2331741303 conn(0x7ff6a806b980 msgr2=0x7ff6a806bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:14.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.602+0000 7ff6adc00640 1 -- 172.21.15.152:0/2331741303 shutdown_connections 2024-04-20T11:29:14.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.602+0000 7ff6adc00640 1 -- 172.21.15.152:0/2331741303 wait complete. 2024-04-20T11:29:14.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.604+0000 7ff6adc00640 1 Processor -- start 2024-04-20T11:29:14.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.604+0000 7ff6adc00640 1 -- start start 2024-04-20T11:29:14.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.605+0000 7ff6adc00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff6a80b3fa0 0x7ff6a80cdf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:14.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.605+0000 7ff6adc00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff6a815e5d0 0x7ff6a80ce480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:14.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.605+0000 7ff6adc00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff6a815e9b0 0x7ff6a80ccec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:14.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.605+0000 7ff6adc00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7ff6a806e0a0 con 0x7ff6a80b3fa0 2024-04-20T11:29:14.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.605+0000 7ff6adc00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff6a8163310 con 0x7ff6a815e5d0 2024-04-20T11:29:14.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.605+0000 7ff6adc00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff6a806e220 con 0x7ff6a815e9b0 2024-04-20T11:29:14.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.605+0000 7ff6a6a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff6a80b3fa0 0x7ff6a80cdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:14.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.605+0000 7ff6a6a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff6a80b3fa0 0x7ff6a80cdf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:46416/0 (socket says 172.21.15.152:46416) 2024-04-20T11:29:14.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.605+0000 7ff6a6a00640 1 -- 172.21.15.152:0/2710050565 learned_addr learned my addr 172.21.15.152:0/2710050565 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:29:14.608 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.606+0000 7ff6a6000640 1 --2- 172.21.15.152:0/2710050565 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff6a815e5d0 0x7ff6a80ce480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:14.608 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.606+0000 7ff6a6a00640 1 -- 172.21.15.152:0/2710050565 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff6a815e9b0 msgr2=0x7ff6a80ccec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:14.608 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.606+0000 7ff6a7400640 1 --2- 172.21.15.152:0/2710050565 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff6a815e9b0 0x7ff6a80ccec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:14.608 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.606+0000 7ff6a6a00640 1 --2- 172.21.15.152:0/2710050565 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff6a815e9b0 0x7ff6a80ccec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:14.608 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.606+0000 7ff6a6a00640 1 -- 172.21.15.152:0/2710050565 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff6a815e5d0 msgr2=0x7ff6a80ce480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:14.608 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.606+0000 7ff6a6a00640 1 --2- 172.21.15.152:0/2710050565 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff6a815e5d0 0x7ff6a80ce480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:14.608 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.606+0000 7ff6a6a00640 1 -- 172.21.15.152:0/2710050565 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6a8151e10 con 0x7ff6a80b3fa0 2024-04-20T11:29:14.608 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.606+0000 7ff6a6000640 1 --2- 172.21.15.152:0/2710050565 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff6a815e5d0 0x7ff6a80ce480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:29:14.608 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.606+0000 7ff6a6a00640 1 --2- 172.21.15.152:0/2710050565 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff6a80b3fa0 0x7ff6a80cdf40 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7ff69800ce00 tx=0x7ff69800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:14.608 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.606+0000 7ff69ea00640 1 -- 172.21.15.152:0/2710050565 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff69800f890 con 0x7ff6a80b3fa0 2024-04-20T11:29:14.608 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.606+0000 7ff69ea00640 1 -- 172.21.15.152:0/2710050565 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff69800fa30 con 0x7ff6a80b3fa0 2024-04-20T11:29:14.608 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.606+0000 7ff69ea00640 1 -- 172.21.15.152:0/2710050565 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff69800fe40 con 0x7ff6a80b3fa0 2024-04-20T11:29:14.608 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.606+0000 7ff6adc00640 1 -- 172.21.15.152:0/2710050565 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6a81520f0 con 0x7ff6a80b3fa0 2024-04-20T11:29:14.608 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.607+0000 7ff6adc00640 1 -- 172.21.15.152:0/2710050565 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6a81526b0 con 0x7ff6a80b3fa0 2024-04-20T11:29:14.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.608+0000 7ff6adc00640 1 -- 172.21.15.152:0/2710050565 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6a80689f0 con 0x7ff6a80b3fa0 2024-04-20T11:29:14.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.609+0000 7ff69ea00640 1 -- 172.21.15.152:0/2710050565 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7ff698018400 con 0x7ff6a80b3fa0 2024-04-20T11:29:14.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.610+0000 7ff69ea00640 1 --2- 172.21.15.152:0/2710050565 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff68406cad0 0x7ff68406ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:14.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.610+0000 7ff69ea00640 1 -- 172.21.15.152:0/2710050565 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7ff69808cc70 con 0x7ff6a80b3fa0 2024-04-20T11:29:14.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.610+0000 7ff6a6000640 1 --2- 172.21.15.152:0/2710050565 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff68406cad0 0x7ff68406ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:14.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.611+0000 7ff6a6000640 1 --2- 172.21.15.152:0/2710050565 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff68406cad0 0x7ff68406ef90 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ff690006fd0 tx=0x7ff690008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:14.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.615+0000 7ff69ea00640 1 -- 172.21.15.152:0/2710050565 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ff698014030 con 0x7ff6a80b3fa0 2024-04-20T11:29:14.812 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.812+0000 7ff6adc00640 1 -- 172.21.15.152:0/2710050565 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff6a806f570 con 0x7ff6a80b3fa0 2024-04-20T11:29:14.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.812+0000 7ff69ea00640 1 -- 172.21.15.152:0/2710050565 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v46) ==== 93+0+10 (secure 0 0 0) 0x7ff69800ae20 con 0x7ff6a80b3fa0 2024-04-20T11:29:14.813 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 1 2024-04-20T11:29:14.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.815+0000 7ff6adc00640 1 -- 172.21.15.152:0/2710050565 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff68406cad0 msgr2=0x7ff68406ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:14.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.815+0000 7ff6adc00640 1 --2- 172.21.15.152:0/2710050565 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff68406cad0 0x7ff68406ef90 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ff690006fd0 tx=0x7ff690008040 comp rx=0 tx=0).stop 2024-04-20T11:29:14.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.815+0000 7ff6adc00640 1 -- 172.21.15.152:0/2710050565 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff6a80b3fa0 msgr2=0x7ff6a80cdf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:14.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.815+0000 7ff6adc00640 1 --2- 172.21.15.152:0/2710050565 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff6a80b3fa0 0x7ff6a80cdf40 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7ff69800ce00 tx=0x7ff69800a650 comp rx=0 tx=0).stop 2024-04-20T11:29:14.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.815+0000 7ff6adc00640 1 -- 172.21.15.152:0/2710050565 shutdown_connections 2024-04-20T11:29:14.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.815+0000 7ff6adc00640 1 --2- 172.21.15.152:0/2710050565 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff68406cad0 0x7ff68406ef90 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:14.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.815+0000 7ff6adc00640 1 --2- 172.21.15.152:0/2710050565 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff6a815e9b0 0x7ff6a80ccec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:14.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.815+0000 7ff6adc00640 1 --2- 172.21.15.152:0/2710050565 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff6a815e5d0 0x7ff6a80ce480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:14.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.815+0000 7ff6adc00640 1 --2- 172.21.15.152:0/2710050565 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff6a80b3fa0 0x7ff6a80cdf40 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:14.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.815+0000 7ff6adc00640 1 -- 172.21.15.152:0/2710050565 >> 172.21.15.152:0/2710050565 conn(0x7ff6a806b980 msgr2=0x7ff6a80b0110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:14.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.815+0000 7ff6adc00640 1 -- 172.21.15.152:0/2710050565 shutdown_connections 2024-04-20T11:29:14.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:14.815+0000 7ff6adc00640 1 -- 172.21.15.152:0/2710050565 wait complete. 2024-04-20T11:29:14.835 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T11:29:15.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.044+0000 7f0d64200640 1 Processor -- start 2024-04-20T11:29:15.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.045+0000 7f0d64200640 1 -- start start 2024-04-20T11:29:15.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.046+0000 7f0d64200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f0d5c0ba430 0x7f0d5c0b7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:15.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.046+0000 7f0d64200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f0d5c0b7cc0 0x7f0d5c0d1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:15.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.046+0000 7f0d64200640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d5c15ec70 con 0x7f0d5c15e890 2024-04-20T11:29:15.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.046+0000 7f0d64200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f0d5c0b8e20 con 0x7f0d5c0ba430 2024-04-20T11:29:15.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.046+0000 7f0d64200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f0d5c0d2ef0 con 0x7f0d5c0b7cc0 2024-04-20T11:29:15.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.046+0000 7f0d61800640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f0d5c15e890 0x7f0d5c0d27e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:35778/0 (socket says 172.21.15.152:35778) 2024-04-20T11:29:15.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.046+0000 7f0d61800640 1 -- 172.21.15.152:0/39293146 learned_addr learned my addr 172.21.15.152:0/39293146 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:29:15.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.046+0000 7f0d60e00640 1 --2- 172.21.15.152:0/39293146 >> v2:172.21.15.152:3300/0 conn(0x7f0d5c0ba430 0x7f0d5c0b7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:15.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.047+0000 7f0d62200640 1 --2- 172.21.15.152:0/39293146 >> v2:172.21.15.175:3300/0 conn(0x7f0d5c0b7cc0 0x7f0d5c0d1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:15.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.047+0000 7f0d60e00640 1 -- 172.21.15.152:0/39293146 >> v2:172.21.15.175:3300/0 conn(0x7f0d5c0b7cc0 msgr2=0x7f0d5c0d1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:15.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.047+0000 7f0d5be00640 1 -- 172.21.15.152:0/39293146 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2985567121 0 0) 0x7f0d5c15ec70 con 0x7f0d5c15e890 2024-04-20T11:29:15.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.047+0000 7f0d60e00640 1 --2- 172.21.15.152:0/39293146 >> v2:172.21.15.175:3300/0 conn(0x7f0d5c0b7cc0 0x7f0d5c0d1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:15.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.047+0000 7f0d60e00640 1 -- 172.21.15.152:0/39293146 >> v1:172.21.15.152:6789/0 conn(0x7f0d5c15e890 legacy=0x7f0d5c0d27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:15.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.047+0000 7f0d60e00640 1 -- 172.21.15.152:0/39293146 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d5c0b8210 con 0x7f0d5c0ba430 2024-04-20T11:29:15.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.048+0000 7f0d60e00640 1 --2- 172.21.15.152:0/39293146 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0d5c0ba430 0x7f0d5c0b7780 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f0d5c0b8fa0 tx=0x7f0d4802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ee6eccb59ee5fbd server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:15.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.048+0000 7f0d5be00640 1 -- 172.21.15.152:0/39293146 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d48034070 con 0x7f0d5c0ba430 2024-04-20T11:29:15.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.048+0000 7f0d5be00640 1 -- 172.21.15.152:0/39293146 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d48030610 con 0x7f0d5c0ba430 2024-04-20T11:29:15.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.048+0000 7f0d5be00640 1 -- 172.21.15.152:0/39293146 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d480309e0 con 0x7f0d5c0ba430 2024-04-20T11:29:15.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.049+0000 7f0d64200640 1 -- 172.21.15.152:0/39293146 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0d5c0ba430 msgr2=0x7f0d5c0b7780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:15.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.049+0000 7f0d64200640 1 --2- 172.21.15.152:0/39293146 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0d5c0ba430 0x7f0d5c0b7780 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f0d5c0b8fa0 tx=0x7f0d4802b960 comp rx=0 tx=0).stop 2024-04-20T11:29:15.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.049+0000 7f0d64200640 1 -- 172.21.15.152:0/39293146 shutdown_connections 2024-04-20T11:29:15.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.049+0000 7f0d64200640 1 --2- 172.21.15.152:0/39293146 >> v2:172.21.15.175:3300/0 conn(0x7f0d5c0b7cc0 0x7f0d5c0d1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:15.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.049+0000 7f0d64200640 1 --2- 172.21.15.152:0/39293146 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0d5c0ba430 0x7f0d5c0b7780 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:15.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.049+0000 7f0d64200640 1 -- 172.21.15.152:0/39293146 >> 172.21.15.152:0/39293146 conn(0x7f0d5c067ae0 msgr2=0x7f0d5c067ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:15.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.049+0000 7f0d64200640 1 -- 172.21.15.152:0/39293146 shutdown_connections 2024-04-20T11:29:15.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.049+0000 7f0d64200640 1 -- 172.21.15.152:0/39293146 wait complete. 2024-04-20T11:29:15.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.051+0000 7f0d64200640 1 Processor -- start 2024-04-20T11:29:15.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.051+0000 7f0d64200640 1 -- start start 2024-04-20T11:29:15.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.052+0000 7f0d64200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0d5c0b7cc0 0x7f0d5c15ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:15.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.052+0000 7f0d64200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d5c0ba430 0x7f0d5c0cdff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:15.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.052+0000 7f0d64200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0d5c15e890 0x7f0d5c1f32c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:15.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.052+0000 7f0d64200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f0d5c0b8ab0 con 0x7f0d5c0b7cc0 2024-04-20T11:29:15.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.052+0000 7f0d64200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0d5c0b87d0 con 0x7f0d5c0ba430 2024-04-20T11:29:15.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.052+0000 7f0d64200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0d5c0d1cd0 con 0x7f0d5c15e890 2024-04-20T11:29:15.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.052+0000 7f0d61800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0d5c0b7cc0 0x7f0d5c15ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:15.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.052+0000 7f0d61800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0d5c0b7cc0 0x7f0d5c15ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:46436/0 (socket says 172.21.15.152:46436) 2024-04-20T11:29:15.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.052+0000 7f0d61800640 1 -- 172.21.15.152:0/3671669939 learned_addr learned my addr 172.21.15.152:0/3671669939 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:29:15.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.053+0000 7f0d60e00640 1 --2- 172.21.15.152:0/3671669939 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d5c0ba430 0x7f0d5c0cdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:15.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.053+0000 7f0d61800640 1 -- 172.21.15.152:0/3671669939 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0d5c15e890 msgr2=0x7f0d5c1f32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:15.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.053+0000 7f0d62200640 1 --2- 172.21.15.152:0/3671669939 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0d5c15e890 0x7f0d5c1f32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:15.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.053+0000 7f0d61800640 1 --2- 172.21.15.152:0/3671669939 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0d5c15e890 0x7f0d5c1f32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:15.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.053+0000 7f0d61800640 1 -- 172.21.15.152:0/3671669939 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d5c0ba430 msgr2=0x7f0d5c0cdff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:15.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.053+0000 7f0d61800640 1 --2- 172.21.15.152:0/3671669939 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d5c0ba430 0x7f0d5c0cdff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:15.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.053+0000 7f0d61800640 1 -- 172.21.15.152:0/3671669939 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d5c15f570 con 0x7f0d5c0b7cc0 2024-04-20T11:29:15.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.053+0000 7f0d60e00640 1 --2- 172.21.15.152:0/3671669939 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d5c0ba430 0x7f0d5c0cdff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:29:15.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.053+0000 7f0d62200640 1 --2- 172.21.15.152:0/3671669939 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0d5c15e890 0x7f0d5c1f32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:29:15.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.053+0000 7f0d61800640 1 --2- 172.21.15.152:0/3671669939 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0d5c0b7cc0 0x7f0d5c15ee70 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c00c430 tx=0x7f0d4c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:15.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.053+0000 7f0d59600640 1 -- 172.21.15.152:0/3671669939 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d4c0074e0 con 0x7f0d5c0b7cc0 2024-04-20T11:29:15.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.053+0000 7f0d59600640 1 -- 172.21.15.152:0/3671669939 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d4c0047e0 con 0x7f0d5c0b7cc0 2024-04-20T11:29:15.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.053+0000 7f0d64200640 1 -- 172.21.15.152:0/3671669939 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d5c161680 con 0x7f0d5c0b7cc0 2024-04-20T11:29:15.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.053+0000 7f0d59600640 1 -- 172.21.15.152:0/3671669939 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d4c004bd0 con 0x7f0d5c0b7cc0 2024-04-20T11:29:15.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.054+0000 7f0d64200640 1 -- 172.21.15.152:0/3671669939 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d5c161c40 con 0x7f0d5c0b7cc0 2024-04-20T11:29:15.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.055+0000 7f0d64200640 1 -- 172.21.15.152:0/3671669939 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d5c0689f0 con 0x7f0d5c0b7cc0 2024-04-20T11:29:15.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.056+0000 7f0d59600640 1 -- 172.21.15.152:0/3671669939 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f0d4c011070 con 0x7f0d5c0b7cc0 2024-04-20T11:29:15.057 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.057+0000 7f0d59600640 1 --2- 172.21.15.152:0/3671669939 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f0d3c06cad0 0x7f0d3c06ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:15.057 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.057+0000 7f0d59600640 1 -- 172.21.15.152:0/3671669939 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f0d4c08bb30 con 0x7f0d5c0b7cc0 2024-04-20T11:29:15.057 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.057+0000 7f0d60e00640 1 --2- 172.21.15.152:0/3671669939 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f0d3c06cad0 0x7f0d3c06ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:15.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.058+0000 7f0d60e00640 1 --2- 172.21.15.152:0/3671669939 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f0d3c06cad0 0x7f0d3c06ef90 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f0d48036000 tx=0x7f0d4803b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:15.061 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.061+0000 7f0d59600640 1 -- 172.21.15.152:0/3671669939 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f0d5c15f570 con 0x7f0d5c0b7cc0 2024-04-20T11:29:15.263 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.262+0000 7f0d64200640 1 -- 172.21.15.152:0/3671669939 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f0d5c161370 con 0x7f0d5c0b7cc0 2024-04-20T11:29:15.263 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.263+0000 7f0d59600640 1 -- 172.21.15.152:0/3671669939 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v46) ==== 104+0+11 (secure 0 0 0) 0x7f0d4c00e280 con 0x7f0d5c0b7cc0 2024-04-20T11:29:15.264 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 32 2024-04-20T11:29:15.266 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.266+0000 7f0d64200640 1 -- 172.21.15.152:0/3671669939 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f0d3c06cad0 msgr2=0x7f0d3c06ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:15.266 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.266+0000 7f0d64200640 1 --2- 172.21.15.152:0/3671669939 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f0d3c06cad0 0x7f0d3c06ef90 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f0d48036000 tx=0x7f0d4803b040 comp rx=0 tx=0).stop 2024-04-20T11:29:15.266 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.266+0000 7f0d64200640 1 -- 172.21.15.152:0/3671669939 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0d5c0b7cc0 msgr2=0x7f0d5c15ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:15.266 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.266+0000 7f0d64200640 1 --2- 172.21.15.152:0/3671669939 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0d5c0b7cc0 0x7f0d5c15ee70 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c00c430 tx=0x7f0d4c00c8f0 comp rx=0 tx=0).stop 2024-04-20T11:29:15.266 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.266+0000 7f0d64200640 1 -- 172.21.15.152:0/3671669939 shutdown_connections 2024-04-20T11:29:15.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.266+0000 7f0d64200640 1 --2- 172.21.15.152:0/3671669939 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f0d3c06cad0 0x7f0d3c06ef90 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:15.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.266+0000 7f0d64200640 1 --2- 172.21.15.152:0/3671669939 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0d5c15e890 0x7f0d5c1f32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:15.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.266+0000 7f0d64200640 1 --2- 172.21.15.152:0/3671669939 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d5c0ba430 0x7f0d5c0cdff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:15.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.266+0000 7f0d64200640 1 --2- 172.21.15.152:0/3671669939 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0d5c0b7cc0 0x7f0d5c15ee70 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:15.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.266+0000 7f0d64200640 1 -- 172.21.15.152:0/3671669939 >> 172.21.15.152:0/3671669939 conn(0x7f0d5c067ae0 msgr2=0x7f0d5c0b57e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:15.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.266+0000 7f0d64200640 1 -- 172.21.15.152:0/3671669939 shutdown_connections 2024-04-20T11:29:15.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.266+0000 7f0d64200640 1 -- 172.21.15.152:0/3671669939 wait complete. 2024-04-20T11:29:15.285 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T11:29:15.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.481+0000 7fc158800640 1 Processor -- start 2024-04-20T11:29:15.482 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.481+0000 7fc158800640 1 -- start start 2024-04-20T11:29:15.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.482+0000 7fc158800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fc1500b2880 0x7fc1500b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:15.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.482+0000 7fc158800640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1501463e0 con 0x7fc1500cd1b0 2024-04-20T11:29:15.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.483+0000 7fc158800640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1500b3e20 con 0x7fc150146000 2024-04-20T11:29:15.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.483+0000 7fc158800640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fc1500b4750 con 0x7fc1500b2880 2024-04-20T11:29:15.484 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.483+0000 7fc155400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fc1500b2880 0x7fc1500b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:15.484 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.483+0000 7fc156800640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fc1500cd1b0 0x7fc1500cd5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:35784/0 (socket says 172.21.15.152:35784) 2024-04-20T11:29:15.484 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.483+0000 7fc156800640 1 -- 172.21.15.152:0/3656654459 learned_addr learned my addr 172.21.15.152:0/3656654459 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:29:15.484 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.483+0000 7fc154a00640 1 -- 172.21.15.152:0/3656654459 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 537731190 0 0) 0x7fc1501463e0 con 0x7fc1500cd1b0 2024-04-20T11:29:15.485 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.483+0000 7fc154a00640 1 -- 172.21.15.152:0/3656654459 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc138003520 con 0x7fc1500cd1b0 2024-04-20T11:29:15.485 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.484+0000 7fc155400640 1 -- 172.21.15.152:0/3656654459 >> v1:172.21.15.175:6789/0 conn(0x7fc150146000 legacy=0x7fc1500b2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:15.485 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.484+0000 7fc155400640 1 -- 172.21.15.152:0/3656654459 >> v1:172.21.15.152:6789/0 conn(0x7fc1500cd1b0 legacy=0x7fc1500cd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:15.485 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.484+0000 7fc155400640 1 -- 172.21.15.152:0/3656654459 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1500b4100 con 0x7fc1500b2880 2024-04-20T11:29:15.485 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.485+0000 7fc155400640 1 --2- 172.21.15.152:0/3656654459 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc1500b2880 0x7fc1500b4be0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fc1400097a0 tx=0x7fc14002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea86b5ee3a0d237f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:15.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.485+0000 7fc154a00640 1 -- 172.21.15.152:0/3656654459 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc140035020 con 0x7fc1500b2880 2024-04-20T11:29:15.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.485+0000 7fc154a00640 1 -- 172.21.15.152:0/3656654459 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc140031830 con 0x7fc1500b2880 2024-04-20T11:29:15.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.485+0000 7fc154a00640 1 -- 172.21.15.152:0/3656654459 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc140031c00 con 0x7fc1500b2880 2024-04-20T11:29:15.487 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.486+0000 7fc158800640 1 -- 172.21.15.152:0/3656654459 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc1500b2880 msgr2=0x7fc1500b4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:15.487 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.486+0000 7fc158800640 1 --2- 172.21.15.152:0/3656654459 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc1500b2880 0x7fc1500b4be0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fc1400097a0 tx=0x7fc14002b8d0 comp rx=0 tx=0).stop 2024-04-20T11:29:15.487 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.486+0000 7fc158800640 1 -- 172.21.15.152:0/3656654459 shutdown_connections 2024-04-20T11:29:15.487 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.486+0000 7fc158800640 1 --2- 172.21.15.152:0/3656654459 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc1500b2880 0x7fc1500b4be0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:15.487 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.486+0000 7fc158800640 1 -- 172.21.15.152:0/3656654459 >> 172.21.15.152:0/3656654459 conn(0x7fc15006b980 msgr2=0x7fc15006bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:15.487 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.486+0000 7fc158800640 1 -- 172.21.15.152:0/3656654459 shutdown_connections 2024-04-20T11:29:15.487 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.486+0000 7fc158800640 1 -- 172.21.15.152:0/3656654459 wait complete. 2024-04-20T11:29:15.488 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.488+0000 7fc158800640 1 Processor -- start 2024-04-20T11:29:15.488 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.488+0000 7fc158800640 1 -- start start 2024-04-20T11:29:15.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.489+0000 7fc158800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc1500b2880 0x7fc15015c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:15.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.489+0000 7fc158800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc1500cd1b0 0x7fc1500cbd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:15.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.489+0000 7fc158800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc150146000 0x7fc150143a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:15.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.489+0000 7fc158800640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fc15015ce70 con 0x7fc1500b2880 2024-04-20T11:29:15.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.489+0000 7fc158800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc15015ccf0 con 0x7fc150146000 2024-04-20T11:29:15.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.489+0000 7fc158800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc15015cff0 con 0x7fc1500cd1b0 2024-04-20T11:29:15.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.489+0000 7fc155e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc1500b2880 0x7fc15015c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:15.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.489+0000 7fc155e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc1500b2880 0x7fc15015c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:46458/0 (socket says 172.21.15.152:46458) 2024-04-20T11:29:15.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.489+0000 7fc155e00640 1 -- 172.21.15.152:0/3395893314 learned_addr learned my addr 172.21.15.152:0/3395893314 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:29:15.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.490+0000 7fc155400640 1 --2- 172.21.15.152:0/3395893314 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc1500cd1b0 0x7fc1500cbd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:15.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.490+0000 7fc156800640 1 --2- 172.21.15.152:0/3395893314 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc150146000 0x7fc150143a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:15.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.490+0000 7fc155e00640 1 -- 172.21.15.152:0/3395893314 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc1500cd1b0 msgr2=0x7fc1500cbd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:15.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.490+0000 7fc155e00640 1 --2- 172.21.15.152:0/3395893314 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc1500cd1b0 0x7fc1500cbd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:15.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.490+0000 7fc155e00640 1 -- 172.21.15.152:0/3395893314 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc150146000 msgr2=0x7fc150143a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:15.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.490+0000 7fc155e00640 1 --2- 172.21.15.152:0/3395893314 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc150146000 0x7fc150143a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:15.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.490+0000 7fc155e00640 1 -- 172.21.15.152:0/3395893314 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc15013ffb0 con 0x7fc1500b2880 2024-04-20T11:29:15.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.490+0000 7fc155e00640 1 --2- 172.21.15.152:0/3395893314 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc1500b2880 0x7fc15015c140 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fc14400c3f0 tx=0x7fc14400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:15.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.490+0000 7fc14a000640 1 -- 172.21.15.152:0/3395893314 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1440052d0 con 0x7fc1500b2880 2024-04-20T11:29:15.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.490+0000 7fc14a000640 1 -- 172.21.15.152:0/3395893314 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc144005490 con 0x7fc1500b2880 2024-04-20T11:29:15.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.490+0000 7fc14a000640 1 -- 172.21.15.152:0/3395893314 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc144007260 con 0x7fc1500b2880 2024-04-20T11:29:15.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.491+0000 7fc158800640 1 -- 172.21.15.152:0/3395893314 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc150140290 con 0x7fc1500b2880 2024-04-20T11:29:15.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.491+0000 7fc158800640 1 -- 172.21.15.152:0/3395893314 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc150140850 con 0x7fc1500b2880 2024-04-20T11:29:15.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.492+0000 7fc158800640 1 -- 172.21.15.152:0/3395893314 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1500689f0 con 0x7fc1500b2880 2024-04-20T11:29:15.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.493+0000 7fc14a000640 1 -- 172.21.15.152:0/3395893314 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7fc144007400 con 0x7fc1500b2880 2024-04-20T11:29:15.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.494+0000 7fc14a000640 1 --2- 172.21.15.152:0/3395893314 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc13406caa0 0x7fc13406ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:15.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.494+0000 7fc14a000640 1 -- 172.21.15.152:0/3395893314 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7fc14408b490 con 0x7fc1500b2880 2024-04-20T11:29:15.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.494+0000 7fc155400640 1 --2- 172.21.15.152:0/3395893314 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc13406caa0 0x7fc13406ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:15.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.495+0000 7fc155400640 1 --2- 172.21.15.152:0/3395893314 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc13406caa0 0x7fc13406ef60 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc140036000 tx=0x7fc140008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:15.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.498+0000 7fc14a000640 1 -- 172.21.15.152:0/3395893314 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fc144059510 con 0x7fc1500b2880 2024-04-20T11:29:15.697 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.696+0000 7fc158800640 1 -- 172.21.15.152:0/3395893314 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc1500b0ab0 con 0x7fc1500b2880 2024-04-20T11:29:15.698 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.697+0000 7fc14a000640 1 -- 172.21.15.152:0/3395893314 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v46) ==== 100+0+11 (secure 0 0 0) 0x7fc144058e10 con 0x7fc1500b2880 2024-04-20T11:29:15.698 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 64 2024-04-20T11:29:15.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.700+0000 7fc158800640 1 -- 172.21.15.152:0/3395893314 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc13406caa0 msgr2=0x7fc13406ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:15.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.700+0000 7fc158800640 1 --2- 172.21.15.152:0/3395893314 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc13406caa0 0x7fc13406ef60 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc140036000 tx=0x7fc140008030 comp rx=0 tx=0).stop 2024-04-20T11:29:15.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.700+0000 7fc158800640 1 -- 172.21.15.152:0/3395893314 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc1500b2880 msgr2=0x7fc15015c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:15.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.700+0000 7fc158800640 1 --2- 172.21.15.152:0/3395893314 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc1500b2880 0x7fc15015c140 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fc14400c3f0 tx=0x7fc14400c8b0 comp rx=0 tx=0).stop 2024-04-20T11:29:15.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.700+0000 7fc158800640 1 -- 172.21.15.152:0/3395893314 shutdown_connections 2024-04-20T11:29:15.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.700+0000 7fc158800640 1 --2- 172.21.15.152:0/3395893314 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc13406caa0 0x7fc13406ef60 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:15.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.700+0000 7fc158800640 1 --2- 172.21.15.152:0/3395893314 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc150146000 0x7fc150143a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:15.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.700+0000 7fc158800640 1 --2- 172.21.15.152:0/3395893314 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc1500cd1b0 0x7fc1500cbd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:15.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.700+0000 7fc158800640 1 --2- 172.21.15.152:0/3395893314 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc1500b2880 0x7fc15015c140 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:15.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.700+0000 7fc158800640 1 -- 172.21.15.152:0/3395893314 >> 172.21.15.152:0/3395893314 conn(0x7fc15006b980 msgr2=0x7fc150074a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:15.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.700+0000 7fc158800640 1 -- 172.21.15.152:0/3395893314 shutdown_connections 2024-04-20T11:29:15.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.700+0000 7fc158800640 1 -- 172.21.15.152:0/3395893314 wait complete. 2024-04-20T11:29:15.720 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-20T11:29:15.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.910+0000 7f8b73200640 1 Processor -- start 2024-04-20T11:29:15.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.911+0000 7f8b73200640 1 -- start start 2024-04-20T11:29:15.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.912+0000 7f8b73200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8b6c159040 0x7f8b6c0cf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:15.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.912+0000 7f8b73200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f8b6c06f6a0 0x7f8b6c0ce140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:15.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.912+0000 7f8b73200640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b6c159420 con 0x7f8b6c162530 2024-04-20T11:29:15.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.912+0000 7f8b73200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f8b6c0ce680 con 0x7f8b6c06f6a0 2024-04-20T11:29:15.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.912+0000 7f8b73200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8b6c06fb00 con 0x7f8b6c159040 2024-04-20T11:29:15.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.912+0000 7f8b71400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f8b6c06f6a0 0x7f8b6c0ce140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:15.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.912+0000 7f8b72800640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f8b6c162530 0x7f8b6c0b32d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:52574/0 (socket says 172.21.15.152:52574) 2024-04-20T11:29:15.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.912+0000 7f8b72800640 1 -- 172.21.15.152:0/657402509 learned_addr learned my addr 172.21.15.152:0/657402509 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:29:15.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.912+0000 7f8b71e00640 1 --2- 172.21.15.152:0/657402509 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8b6c159040 0x7f8b6c0cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:15.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.913+0000 7f8b71400640 1 -- 172.21.15.152:0/657402509 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8b6c159040 msgr2=0x7f8b6c0cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:15.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.913+0000 7f8b71400640 1 --2- 172.21.15.152:0/657402509 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8b6c159040 0x7f8b6c0cf950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:15.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.913+0000 7f8b71400640 1 -- 172.21.15.152:0/657402509 >> v1:172.21.15.152:6789/0 conn(0x7f8b6c162530 legacy=0x7f8b6c0b32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:15.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.913+0000 7f8b71400640 1 -- 172.21.15.152:0/657402509 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b6c0ce8b0 con 0x7f8b6c06f6a0 2024-04-20T11:29:15.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.913+0000 7f8b70a00640 1 -- 172.21.15.152:0/657402509 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2749859144 0 0) 0x7f8b68001490 con 0x7f8b6c162530 2024-04-20T11:29:15.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.914+0000 7f8b71400640 1 --2- 172.21.15.152:0/657402509 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8b6c06f6a0 0x7f8b6c0ce140 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c0097a0 tx=0x7f8b5c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71fee0bde0afee19 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:15.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.914+0000 7f8b70a00640 1 -- 172.21.15.152:0/657402509 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b5c035020 con 0x7f8b6c06f6a0 2024-04-20T11:29:15.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.914+0000 7f8b70a00640 1 -- 172.21.15.152:0/657402509 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b5c031830 con 0x7f8b6c06f6a0 2024-04-20T11:29:15.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.914+0000 7f8b70a00640 1 -- 172.21.15.152:0/657402509 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b5c031c00 con 0x7f8b6c06f6a0 2024-04-20T11:29:15.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.915+0000 7f8b73200640 1 -- 172.21.15.152:0/657402509 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8b6c06f6a0 msgr2=0x7f8b6c0ce140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:15.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.915+0000 7f8b73200640 1 --2- 172.21.15.152:0/657402509 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8b6c06f6a0 0x7f8b6c0ce140 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c0097a0 tx=0x7f8b5c02b8d0 comp rx=0 tx=0).stop 2024-04-20T11:29:15.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.915+0000 7f8b73200640 1 -- 172.21.15.152:0/657402509 shutdown_connections 2024-04-20T11:29:15.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.915+0000 7f8b73200640 1 --2- 172.21.15.152:0/657402509 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8b6c06f6a0 0x7f8b6c0ce140 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:15.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.915+0000 7f8b73200640 1 --2- 172.21.15.152:0/657402509 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8b6c159040 0x7f8b6c0cf950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:15.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.915+0000 7f8b73200640 1 -- 172.21.15.152:0/657402509 >> 172.21.15.152:0/657402509 conn(0x7f8b6c0d0260 msgr2=0x7f8b6c0d0670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:15.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.915+0000 7f8b73200640 1 -- 172.21.15.152:0/657402509 shutdown_connections 2024-04-20T11:29:15.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.915+0000 7f8b73200640 1 -- 172.21.15.152:0/657402509 wait complete. 2024-04-20T11:29:15.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.916+0000 7f8b73200640 1 Processor -- start 2024-04-20T11:29:15.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.917+0000 7f8b73200640 1 -- start start 2024-04-20T11:29:15.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.917+0000 7f8b73200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8b6c06f6a0 0x7f8b6c1466b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:15.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.917+0000 7f8b73200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8b6c159040 0x7f8b6c151030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:15.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.917+0000 7f8b73200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b6c162530 0x7f8b6c151570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:15.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.917+0000 7f8b73200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f8b6c0cf150 con 0x7f8b6c06f6a0 2024-04-20T11:29:15.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.917+0000 7f8b73200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8b6c0cee70 con 0x7f8b6c162530 2024-04-20T11:29:15.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.917+0000 7f8b73200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8b6c0cf430 con 0x7f8b6c159040 2024-04-20T11:29:15.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.918+0000 7f8b71e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8b6c06f6a0 0x7f8b6c1466b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:15.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.918+0000 7f8b71e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8b6c06f6a0 0x7f8b6c1466b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:50712/0 (socket says 172.21.15.152:50712) 2024-04-20T11:29:15.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.918+0000 7f8b71e00640 1 -- 172.21.15.152:0/2771732230 learned_addr learned my addr 172.21.15.152:0/2771732230 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:29:15.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.918+0000 7f8b71e00640 1 -- 172.21.15.152:0/2771732230 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8b6c159040 msgr2=0x7f8b6c151030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:15.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.918+0000 7f8b71e00640 1 --2- 172.21.15.152:0/2771732230 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8b6c159040 0x7f8b6c151030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:15.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.918+0000 7f8b71e00640 1 -- 172.21.15.152:0/2771732230 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b6c162530 msgr2=0x7f8b6c151570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:15.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.918+0000 7f8b71e00640 1 --2- 172.21.15.152:0/2771732230 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b6c162530 0x7f8b6c151570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:15.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.918+0000 7f8b71e00640 1 -- 172.21.15.152:0/2771732230 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b6c160070 con 0x7f8b6c06f6a0 2024-04-20T11:29:15.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.918+0000 7f8b71e00640 1 --2- 172.21.15.152:0/2771732230 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8b6c06f6a0 0x7f8b6c1466b0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f8b6000d5f0 tx=0x7f8b6000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:15.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.918+0000 7f8b66000640 1 -- 172.21.15.152:0/2771732230 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b600125f0 con 0x7f8b6c06f6a0 2024-04-20T11:29:15.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.919+0000 7f8b66000640 1 -- 172.21.15.152:0/2771732230 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b60019070 con 0x7f8b6c06f6a0 2024-04-20T11:29:15.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.919+0000 7f8b73200640 1 -- 172.21.15.152:0/2771732230 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b6c073460 con 0x7f8b6c06f6a0 2024-04-20T11:29:15.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.919+0000 7f8b66000640 1 -- 172.21.15.152:0/2771732230 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b600129d0 con 0x7f8b6c06f6a0 2024-04-20T11:29:15.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.919+0000 7f8b73200640 1 -- 172.21.15.152:0/2771732230 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b6c073a20 con 0x7f8b6c06f6a0 2024-04-20T11:29:15.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.920+0000 7f8b73200640 1 -- 172.21.15.152:0/2771732230 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b6c0ade00 con 0x7f8b6c06f6a0 2024-04-20T11:29:15.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.921+0000 7f8b66000640 1 -- 172.21.15.152:0/2771732230 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f8b60012b70 con 0x7f8b6c06f6a0 2024-04-20T11:29:15.922 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.922+0000 7f8b66000640 1 --2- 172.21.15.152:0/2771732230 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f8b5006caa0 0x7f8b5006ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:15.922 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.922+0000 7f8b66000640 1 -- 172.21.15.152:0/2771732230 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f8b6008c2f0 con 0x7f8b6c06f6a0 2024-04-20T11:29:15.922 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.922+0000 7f8b71400640 1 --2- 172.21.15.152:0/2771732230 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f8b5006caa0 0x7f8b5006ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:15.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.923+0000 7f8b71400640 1 --2- 172.21.15.152:0/2771732230 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f8b5006caa0 0x7f8b5006ef60 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c036000 tx=0x7f8b5c030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:15.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:15.926+0000 7f8b66000640 1 -- 172.21.15.152:0/2771732230 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f8b6c0ade00 con 0x7f8b6c06f6a0 2024-04-20T11:29:16.139 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.139+0000 7f8b73200640 1 -- 172.21.15.152:0/2771732230 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f8b6c0cf8c0 con 0x7f8b6c06f6a0 2024-04-20T11:29:16.140 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.139+0000 7f8b66000640 1 -- 172.21.15.152:0/2771732230 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v46) ==== 80+0+0 (secure 0 0 0) 0x7f8b6c0cf8c0 con 0x7f8b6c06f6a0 2024-04-20T11:29:16.140 INFO:teuthology.orchestra.run.smithi152.stderr:listed 0 entries 2024-04-20T11:29:16.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.142+0000 7f8b73200640 1 -- 172.21.15.152:0/2771732230 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f8b5006caa0 msgr2=0x7f8b5006ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:16.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.142+0000 7f8b73200640 1 --2- 172.21.15.152:0/2771732230 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f8b5006caa0 0x7f8b5006ef60 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c036000 tx=0x7f8b5c030000 comp rx=0 tx=0).stop 2024-04-20T11:29:16.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.142+0000 7f8b73200640 1 -- 172.21.15.152:0/2771732230 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8b6c06f6a0 msgr2=0x7f8b6c1466b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:16.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.142+0000 7f8b73200640 1 --2- 172.21.15.152:0/2771732230 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8b6c06f6a0 0x7f8b6c1466b0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f8b6000d5f0 tx=0x7f8b6000dab0 comp rx=0 tx=0).stop 2024-04-20T11:29:16.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.142+0000 7f8b73200640 1 -- 172.21.15.152:0/2771732230 shutdown_connections 2024-04-20T11:29:16.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.142+0000 7f8b73200640 1 --2- 172.21.15.152:0/2771732230 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f8b5006caa0 0x7f8b5006ef60 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:16.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.142+0000 7f8b73200640 1 --2- 172.21.15.152:0/2771732230 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b6c162530 0x7f8b6c151570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:16.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.142+0000 7f8b73200640 1 --2- 172.21.15.152:0/2771732230 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8b6c159040 0x7f8b6c151030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:16.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.142+0000 7f8b73200640 1 --2- 172.21.15.152:0/2771732230 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8b6c06f6a0 0x7f8b6c1466b0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:16.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.142+0000 7f8b73200640 1 -- 172.21.15.152:0/2771732230 >> 172.21.15.152:0/2771732230 conn(0x7f8b6c0d0260 msgr2=0x7f8b6c1552f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:16.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.142+0000 7f8b73200640 1 -- 172.21.15.152:0/2771732230 shutdown_connections 2024-04-20T11:29:16.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.142+0000 7f8b73200640 1 -- 172.21.15.152:0/2771732230 wait complete. 2024-04-20T11:29:16.161 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-20T11:29:16.161 DEBUG:teuthology.orchestra.run.smithi152:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:29:16.208 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:29:16.452 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.452+0000 7fc22f400640 1 Processor -- start 2024-04-20T11:29:16.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.452+0000 7fc22f400640 1 -- start start 2024-04-20T11:29:16.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.454+0000 7fc22f400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc22815c560 0x7fc2280b3df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:16.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.454+0000 7fc22f400640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc22815fd20 con 0x7fc2280b4330 2024-04-20T11:29:16.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.454+0000 7fc22f400640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc22815ef60 con 0x7fc2280d0810 2024-04-20T11:29:16.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.454+0000 7fc22f400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc22815f240 con 0x7fc22815c560 2024-04-20T11:29:16.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.454+0000 7fc227e00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fc2280b4330 0x7fc2280d0100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:52584/0 (socket says 172.21.15.152:52584) 2024-04-20T11:29:16.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.454+0000 7fc227e00640 1 -- 172.21.15.152:0/2884582050 learned_addr learned my addr 172.21.15.152:0/2884582050 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:29:16.456 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.454+0000 7fc22ca00640 1 --2- 172.21.15.152:0/2884582050 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc22815c560 0x7fc2280b3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:16.456 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.455+0000 7fc227400640 1 -- 172.21.15.152:0/2884582050 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3593561334 0 0) 0x7fc22815fd20 con 0x7fc2280b4330 2024-04-20T11:29:16.456 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.455+0000 7fc227400640 1 -- 172.21.15.152:0/2884582050 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc210003520 con 0x7fc2280b4330 2024-04-20T11:29:16.456 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.455+0000 7fc227400640 1 -- 172.21.15.152:0/2884582050 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1108836367 0 0) 0x7fc210003520 con 0x7fc2280b4330 2024-04-20T11:29:16.457 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.455+0000 7fc227400640 1 -- 172.21.15.152:0/2884582050 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc22815c560 msgr2=0x7fc2280b3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:16.457 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.455+0000 7fc227400640 1 --2- 172.21.15.152:0/2884582050 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc22815c560 0x7fc2280b3df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:16.457 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.455+0000 7fc227400640 1 -- 172.21.15.152:0/2884582050 >> v1:172.21.15.175:6789/0 conn(0x7fc2280d0810 legacy=0x7fc2280ada90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:16.457 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.455+0000 7fc227400640 1 -- 172.21.15.152:0/2884582050 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc22815f520 con 0x7fc2280b4330 2024-04-20T11:29:16.457 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.455+0000 7fc227400640 1 -- 172.21.15.152:0/2884582050 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7fc214001670 con 0x7fc2280b4330 2024-04-20T11:29:16.457 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.455+0000 7fc227400640 1 -- 172.21.15.152:0/2884582050 >> v1:172.21.15.152:6789/0 conn(0x7fc2280b4330 legacy=0x7fc2280d0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:16.457 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.455+0000 7fc227400640 1 --2- 172.21.15.152:0/2884582050 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc22815c560 0x7fc210003a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:16.457 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.455+0000 7fc227400640 1 --2- 172.21.15.152:0/2884582050 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc210003fd0 0x7fc210004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:16.457 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.455+0000 7fc227400640 1 --2- 172.21.15.152:0/2884582050 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc210004d40 0x7fc2100051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:16.457 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.455+0000 7fc227400640 1 -- 172.21.15.152:0/2884582050 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fc22815f240 con 0x7fc210003fd0 2024-04-20T11:29:16.457 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.455+0000 7fc227400640 1 -- 172.21.15.152:0/2884582050 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc21000bbb0 con 0x7fc210004d40 2024-04-20T11:29:16.457 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.455+0000 7fc227400640 1 -- 172.21.15.152:0/2884582050 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc21000bdb0 con 0x7fc22815c560 2024-04-20T11:29:16.457 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.455+0000 7fc227e00640 1 --2- 172.21.15.152:0/2884582050 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc210003fd0 0x7fc210004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:16.457 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.456+0000 7fc22ca00640 1 --2- 172.21.15.152:0/2884582050 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc22815c560 0x7fc210003a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:16.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.456+0000 7fc22d400640 1 --2- 172.21.15.152:0/2884582050 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc210004d40 0x7fc2100051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:16.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.456+0000 7fc227e00640 1 -- 172.21.15.152:0/2884582050 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc22815c560 msgr2=0x7fc210003a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:16.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.456+0000 7fc227e00640 1 --2- 172.21.15.152:0/2884582050 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc22815c560 0x7fc210003a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:16.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.456+0000 7fc227e00640 1 -- 172.21.15.152:0/2884582050 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc210004d40 msgr2=0x7fc2100051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:16.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.456+0000 7fc227e00640 1 --2- 172.21.15.152:0/2884582050 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc210004d40 0x7fc2100051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:16.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.456+0000 7fc227e00640 1 -- 172.21.15.152:0/2884582050 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc210003520 con 0x7fc210003fd0 2024-04-20T11:29:16.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.457+0000 7fc227e00640 1 --2- 172.21.15.152:0/2884582050 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc210003fd0 0x7fc210004800 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fc21400c240 tx=0x7fc210005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19e2839fe4d54874 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:16.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.457+0000 7fc227400640 1 -- 172.21.15.152:0/2884582050 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc214007d10 con 0x7fc210003fd0 2024-04-20T11:29:16.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.457+0000 7fc227400640 1 -- 172.21.15.152:0/2884582050 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc214030070 con 0x7fc210003fd0 2024-04-20T11:29:16.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.457+0000 7fc227400640 1 -- 172.21.15.152:0/2884582050 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc21402cdc0 con 0x7fc210003fd0 2024-04-20T11:29:16.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.458+0000 7fc22f400640 1 -- 172.21.15.152:0/2884582050 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc210003fd0 msgr2=0x7fc210004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:16.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.458+0000 7fc22f400640 1 --2- 172.21.15.152:0/2884582050 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc210003fd0 0x7fc210004800 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fc21400c240 tx=0x7fc210005830 comp rx=0 tx=0).stop 2024-04-20T11:29:16.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.458+0000 7fc22d400640 1 -- 172.21.15.152:0/2884582050 reap_dead start 2024-04-20T11:29:16.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.458+0000 7fc22f400640 1 -- 172.21.15.152:0/2884582050 shutdown_connections 2024-04-20T11:29:16.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.458+0000 7fc22f400640 1 -- 172.21.15.152:0/2884582050 >> 172.21.15.152:0/2884582050 conn(0x7fc228067cf0 msgr2=0x7fc22806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:16.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.458+0000 7fc22f400640 1 -- 172.21.15.152:0/2884582050 shutdown_connections 2024-04-20T11:29:16.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.458+0000 7fc22f400640 1 -- 172.21.15.152:0/2884582050 wait complete. 2024-04-20T11:29:16.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.460+0000 7fc22f400640 1 Processor -- start 2024-04-20T11:29:16.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.460+0000 7fc22f400640 1 -- start start 2024-04-20T11:29:16.462 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.461+0000 7fc22f400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc210003fd0 0x7fc2281f7660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:16.462 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.461+0000 7fc22f400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc2281619c0 0x7fc228160900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:16.462 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.461+0000 7fc22f400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc2281f48a0 0x7fc2281f4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:16.462 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.461+0000 7fc22f400640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fc2280b3df0 con 0x7fc2281f48a0 2024-04-20T11:29:16.462 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.461+0000 7fc22f400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc2280d0dc0 con 0x7fc210003fd0 2024-04-20T11:29:16.462 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.461+0000 7fc22f400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc2280b3f70 con 0x7fc2281619c0 2024-04-20T11:29:16.462 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.461+0000 7fc22d400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc2281f48a0 0x7fc2281f4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:16.462 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.461+0000 7fc22d400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc2281f48a0 0x7fc2281f4d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:50732/0 (socket says 172.21.15.152:50732) 2024-04-20T11:29:16.462 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.461+0000 7fc22d400640 1 -- 172.21.15.152:0/4244718125 learned_addr learned my addr 172.21.15.152:0/4244718125 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:29:16.463 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.462+0000 7fc22ca00640 1 --2- 172.21.15.152:0/4244718125 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc210003fd0 0x7fc2281f7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:16.463 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.462+0000 7fc227e00640 1 --2- 172.21.15.152:0/4244718125 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc2281619c0 0x7fc228160900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:16.464 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.462+0000 7fc22d400640 1 -- 172.21.15.152:0/4244718125 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc2281619c0 msgr2=0x7fc228160900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:16.464 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.462+0000 7fc22d400640 1 --2- 172.21.15.152:0/4244718125 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc2281619c0 0x7fc228160900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:16.464 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.462+0000 7fc22d400640 1 -- 172.21.15.152:0/4244718125 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc210003fd0 msgr2=0x7fc2281f7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:16.464 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.462+0000 7fc22d400640 1 --2- 172.21.15.152:0/4244718125 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc210003fd0 0x7fc2281f7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:16.464 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.462+0000 7fc22d400640 1 -- 172.21.15.152:0/4244718125 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc22815d3f0 con 0x7fc2281f48a0 2024-04-20T11:29:16.464 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.462+0000 7fc22d400640 1 --2- 172.21.15.152:0/4244718125 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc2281f48a0 0x7fc2281f4d50 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fc228072e10 tx=0x7fc22000ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:16.464 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.463+0000 7fc224c00640 1 -- 172.21.15.152:0/4244718125 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc220003970 con 0x7fc2281f48a0 2024-04-20T11:29:16.464 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.463+0000 7fc224c00640 1 -- 172.21.15.152:0/4244718125 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc220018070 con 0x7fc2281f48a0 2024-04-20T11:29:16.464 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.463+0000 7fc224c00640 1 -- 172.21.15.152:0/4244718125 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc220003d50 con 0x7fc2281f48a0 2024-04-20T11:29:16.464 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.463+0000 7fc22f400640 1 -- 172.21.15.152:0/4244718125 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc22815d660 con 0x7fc2281f48a0 2024-04-20T11:29:16.464 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.463+0000 7fc22f400640 1 -- 172.21.15.152:0/4244718125 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc22815dc20 con 0x7fc2281f48a0 2024-04-20T11:29:16.465 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.464+0000 7fc22f400640 1 -- 172.21.15.152:0/4244718125 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2280689f0 con 0x7fc2281f48a0 2024-04-20T11:29:16.465 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.465+0000 7fc224c00640 1 -- 172.21.15.152:0/4244718125 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7fc2200050d0 con 0x7fc2281f48a0 2024-04-20T11:29:16.467 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.466+0000 7fc224c00640 1 --2- 172.21.15.152:0/4244718125 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc20806caa0 0x7fc20806ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:16.467 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.466+0000 7fc224c00640 1 -- 172.21.15.152:0/4244718125 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7fc22008bbb0 con 0x7fc2281f48a0 2024-04-20T11:29:16.467 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.467+0000 7fc22ca00640 1 --2- 172.21.15.152:0/4244718125 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc20806caa0 0x7fc20806ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:16.467 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.467+0000 7fc22ca00640 1 --2- 172.21.15.152:0/4244718125 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc20806caa0 0x7fc20806ef60 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fc21c009ac0 tx=0x7fc21c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:16.471 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.471+0000 7fc224c00640 1 -- 172.21.15.152:0/4244718125 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fc220059cc0 con 0x7fc2281f48a0 2024-04-20T11:29:16.675 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.675+0000 7fc22f400640 1 -- 172.21.15.152:0/4244718125 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc2280b1c60 con 0x7fc2281f48a0 2024-04-20T11:29:16.678 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.678+0000 7fc224c00640 1 -- 172.21.15.152:0/4244718125 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) ==== 74+0+16673 (secure 0 0 0) 0x7fc2200595c0 con 0x7fc2281f48a0 2024-04-20T11:29:16.679 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:29:16.680 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":46,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:29:01.812130+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:28:58.007718+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-20T11:28:59.260647+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:29:16.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.681+0000 7fc22f400640 1 -- 172.21.15.152:0/4244718125 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc20806caa0 msgr2=0x7fc20806ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:16.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.681+0000 7fc22f400640 1 --2- 172.21.15.152:0/4244718125 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc20806caa0 0x7fc20806ef60 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fc21c009ac0 tx=0x7fc21c008040 comp rx=0 tx=0).stop 2024-04-20T11:29:16.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.681+0000 7fc22f400640 1 -- 172.21.15.152:0/4244718125 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc2281f48a0 msgr2=0x7fc2281f4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:16.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.681+0000 7fc22f400640 1 --2- 172.21.15.152:0/4244718125 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc2281f48a0 0x7fc2281f4d50 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fc228072e10 tx=0x7fc22000ec10 comp rx=0 tx=0).stop 2024-04-20T11:29:16.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.681+0000 7fc22f400640 1 -- 172.21.15.152:0/4244718125 shutdown_connections 2024-04-20T11:29:16.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.681+0000 7fc22f400640 1 --2- 172.21.15.152:0/4244718125 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc20806caa0 0x7fc20806ef60 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:16.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.681+0000 7fc22f400640 1 --2- 172.21.15.152:0/4244718125 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc2281f48a0 0x7fc2281f4d50 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:16.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.681+0000 7fc22f400640 1 --2- 172.21.15.152:0/4244718125 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc2281619c0 0x7fc228160900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:16.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.681+0000 7fc22f400640 1 --2- 172.21.15.152:0/4244718125 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc210003fd0 0x7fc2281f7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:16.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.681+0000 7fc22f400640 1 -- 172.21.15.152:0/4244718125 >> 172.21.15.152:0/4244718125 conn(0x7fc228067cf0 msgr2=0x7fc2280b0bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:16.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.681+0000 7fc22f400640 1 -- 172.21.15.152:0/4244718125 shutdown_connections 2024-04-20T11:29:16.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.681+0000 7fc22f400640 1 -- 172.21.15.152:0/4244718125 wait complete. 2024-04-20T11:29:16.701 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T11:26:47.246571+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T11:28:58.007718+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T11:28:59.260647+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T11:29:16.701 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T11:29:16.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.906+0000 7f716dc00640 1 Processor -- start 2024-04-20T11:29:16.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.906+0000 7f716dc00640 1 -- start start 2024-04-20T11:29:16.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.908+0000 7f716dc00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f716815e640 0x7f716806dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:16.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.908+0000 7f716dc00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71680b5070 con 0x7f716815ea20 2024-04-20T11:29:16.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.908+0000 7f716dc00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f716806e1a0 con 0x7f716806c4b0 2024-04-20T11:29:16.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.908+0000 7f716dc00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f716806e480 con 0x7f716815e640 2024-04-20T11:29:16.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.908+0000 7f7167e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f716815e640 0x7f716806dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:16.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.908+0000 7f7167e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f716815e640 0x7f716806dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:50736/0 (socket says 172.21.15.152:50736) 2024-04-20T11:29:16.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.908+0000 7f7167e00640 1 -- 172.21.15.152:0/3259694315 learned_addr learned my addr 172.21.15.152:0/3259694315 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:29:16.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.909+0000 7f7167e00640 1 -- 172.21.15.152:0/3259694315 >> v1:172.21.15.175:6789/0 conn(0x7f716806c4b0 legacy=0x7f71680ad5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:16.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.909+0000 7f7167e00640 1 -- 172.21.15.152:0/3259694315 >> v1:172.21.15.152:6789/0 conn(0x7f716815ea20 legacy=0x7f716806bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:16.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.909+0000 7f7167e00640 1 -- 172.21.15.152:0/3259694315 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f716806e760 con 0x7f716815e640 2024-04-20T11:29:16.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.909+0000 7f7166a00640 1 -- 172.21.15.152:0/3259694315 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1689116393 0 0) 0x7f71680b5070 con 0x7f716815ea20 2024-04-20T11:29:16.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.910+0000 7f7167e00640 1 --2- 172.21.15.152:0/3259694315 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f716815e640 0x7f716806dbc0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f71580098a0 tx=0x7f715802ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2022912cfa80066 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:16.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.910+0000 7f7166a00640 1 -- 172.21.15.152:0/3259694315 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f715802f840 con 0x7f716815e640 2024-04-20T11:29:16.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.911+0000 7f7166a00640 1 -- 172.21.15.152:0/3259694315 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f715802f9e0 con 0x7f716815e640 2024-04-20T11:29:16.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.911+0000 7f7166a00640 1 -- 172.21.15.152:0/3259694315 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f715802fdb0 con 0x7f716815e640 2024-04-20T11:29:16.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.911+0000 7f716dc00640 1 -- 172.21.15.152:0/3259694315 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f716815e640 msgr2=0x7f716806dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:16.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.911+0000 7f716dc00640 1 --2- 172.21.15.152:0/3259694315 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f716815e640 0x7f716806dbc0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f71580098a0 tx=0x7f715802ba20 comp rx=0 tx=0).stop 2024-04-20T11:29:16.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.911+0000 7f716dc00640 1 -- 172.21.15.152:0/3259694315 shutdown_connections 2024-04-20T11:29:16.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.911+0000 7f716dc00640 1 --2- 172.21.15.152:0/3259694315 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f716815e640 0x7f716806dbc0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:16.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.911+0000 7f716dc00640 1 -- 172.21.15.152:0/3259694315 >> 172.21.15.152:0/3259694315 conn(0x7f71680b21e0 msgr2=0x7f71680b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:16.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.911+0000 7f716dc00640 1 -- 172.21.15.152:0/3259694315 shutdown_connections 2024-04-20T11:29:16.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.911+0000 7f716dc00640 1 -- 172.21.15.152:0/3259694315 wait complete. 2024-04-20T11:29:16.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.913+0000 7f716dc00640 1 Processor -- start 2024-04-20T11:29:16.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.913+0000 7f716dc00640 1 -- start start 2024-04-20T11:29:16.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.914+0000 7f716dc00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f716806c4b0 0x7f7168157b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:16.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.914+0000 7f716dc00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f716815e640 0x7f71681467a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:16.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.914+0000 7f716dc00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f716815a500 0x7f71680994f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:16.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.914+0000 7f716dc00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f716806f410 con 0x7f716815a500 2024-04-20T11:29:16.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.914+0000 7f716dc00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f71680ad0a0 con 0x7f716806c4b0 2024-04-20T11:29:16.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.914+0000 7f716dc00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f716806f590 con 0x7f716815e640 2024-04-20T11:29:16.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.915+0000 7f716d200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f716815a500 0x7f71680994f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:16.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.915+0000 7f716d200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f716815a500 0x7f71680994f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:50740/0 (socket says 172.21.15.152:50740) 2024-04-20T11:29:16.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.915+0000 7f716d200640 1 -- 172.21.15.152:0/3884396988 learned_addr learned my addr 172.21.15.152:0/3884396988 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:29:16.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.915+0000 7f7167e00640 1 --2- 172.21.15.152:0/3884396988 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f716806c4b0 0x7f7168157b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:16.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.915+0000 7f7167400640 1 --2- 172.21.15.152:0/3884396988 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f716815e640 0x7f71681467a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:16.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.915+0000 7f716d200640 1 -- 172.21.15.152:0/3884396988 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f716815e640 msgr2=0x7f71681467a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:16.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.915+0000 7f716d200640 1 --2- 172.21.15.152:0/3884396988 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f716815e640 0x7f71681467a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:16.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.915+0000 7f716d200640 1 -- 172.21.15.152:0/3884396988 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f716806c4b0 msgr2=0x7f7168157b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:16.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.915+0000 7f716d200640 1 --2- 172.21.15.152:0/3884396988 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f716806c4b0 0x7f7168157b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:16.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.915+0000 7f716d200640 1 -- 172.21.15.152:0/3884396988 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7168146ce0 con 0x7f716815a500 2024-04-20T11:29:16.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.915+0000 7f7167400640 1 --2- 172.21.15.152:0/3884396988 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f716815e640 0x7f71681467a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:29:16.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.916+0000 7f716d200640 1 --2- 172.21.15.152:0/3884396988 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f716815a500 0x7f71680994f0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f716000c7a0 tx=0x7f716000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:16.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.916+0000 7f715fe00640 1 -- 172.21.15.152:0/3884396988 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7160003b70 con 0x7f716815a500 2024-04-20T11:29:16.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.916+0000 7f715fe00640 1 -- 172.21.15.152:0/3884396988 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7160003d30 con 0x7f716815a500 2024-04-20T11:29:16.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.916+0000 7f715fe00640 1 -- 172.21.15.152:0/3884396988 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f716000aa90 con 0x7f716815a500 2024-04-20T11:29:16.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.916+0000 7f716dc00640 1 -- 172.21.15.152:0/3884396988 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7168158130 con 0x7f716815a500 2024-04-20T11:29:16.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.916+0000 7f716dc00640 1 -- 172.21.15.152:0/3884396988 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71681586f0 con 0x7f716815a500 2024-04-20T11:29:16.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.917+0000 7f716dc00640 1 -- 172.21.15.152:0/3884396988 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7168075e60 con 0x7f716815a500 2024-04-20T11:29:16.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.918+0000 7f715fe00640 1 -- 172.21.15.152:0/3884396988 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f716000ac30 con 0x7f716815a500 2024-04-20T11:29:16.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.919+0000 7f715fe00640 1 --2- 172.21.15.152:0/3884396988 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f714806caa0 0x7f714806ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:16.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.919+0000 7f715fe00640 1 -- 172.21.15.152:0/3884396988 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f716008bd80 con 0x7f716815a500 2024-04-20T11:29:16.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.919+0000 7f7167e00640 1 --2- 172.21.15.152:0/3884396988 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f714806caa0 0x7f714806ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:16.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.920+0000 7f7167e00640 1 --2- 172.21.15.152:0/3884396988 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f714806caa0 0x7f714806ef60 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f7158036000 tx=0x7f7158008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:16.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:16.923+0000 7f715fe00640 1 -- 172.21.15.152:0/3884396988 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f7160059f20 con 0x7f716815a500 2024-04-20T11:29:17.109 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.109+0000 7f716dc00640 1 -- 172.21.15.152:0/3884396988 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f71680b4340 con 0x7f716815a500 2024-04-20T11:29:17.110 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.110+0000 7f715fe00640 1 -- 172.21.15.152:0/3884396988 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v46) ==== 93+0+10 (secure 0 0 0) 0x7f7160059820 con 0x7f716815a500 2024-04-20T11:29:17.110 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 1 2024-04-20T11:29:17.113 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.112+0000 7f716dc00640 1 -- 172.21.15.152:0/3884396988 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f714806caa0 msgr2=0x7f714806ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:17.113 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.112+0000 7f716dc00640 1 --2- 172.21.15.152:0/3884396988 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f714806caa0 0x7f714806ef60 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f7158036000 tx=0x7f7158008030 comp rx=0 tx=0).stop 2024-04-20T11:29:17.113 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.112+0000 7f716dc00640 1 -- 172.21.15.152:0/3884396988 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f716815a500 msgr2=0x7f71680994f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:17.113 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.112+0000 7f716dc00640 1 --2- 172.21.15.152:0/3884396988 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f716815a500 0x7f71680994f0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f716000c7a0 tx=0x7f716000cc60 comp rx=0 tx=0).stop 2024-04-20T11:29:17.114 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.113+0000 7f716dc00640 1 -- 172.21.15.152:0/3884396988 shutdown_connections 2024-04-20T11:29:17.114 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.113+0000 7f716dc00640 1 --2- 172.21.15.152:0/3884396988 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f714806caa0 0x7f714806ef60 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:17.114 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.113+0000 7f716dc00640 1 --2- 172.21.15.152:0/3884396988 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f716815a500 0x7f71680994f0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:17.114 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.113+0000 7f716dc00640 1 --2- 172.21.15.152:0/3884396988 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f716815e640 0x7f71681467a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:17.114 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.113+0000 7f716dc00640 1 --2- 172.21.15.152:0/3884396988 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f716806c4b0 0x7f7168157b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:17.114 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.113+0000 7f716dc00640 1 -- 172.21.15.152:0/3884396988 >> 172.21.15.152:0/3884396988 conn(0x7f71680b21e0 msgr2=0x7f71680af150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:17.114 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.113+0000 7f716dc00640 1 -- 172.21.15.152:0/3884396988 shutdown_connections 2024-04-20T11:29:17.114 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.113+0000 7f716dc00640 1 -- 172.21.15.152:0/3884396988 wait complete. 2024-04-20T11:29:17.132 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T11:29:17.330 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.329+0000 7ff5a8200640 1 Processor -- start 2024-04-20T11:29:17.330 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.329+0000 7ff5a8200640 1 -- start start 2024-04-20T11:29:17.332 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.331+0000 7ff5a8200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7ff5a00d0800 0x7ff5a0162680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:17.332 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.331+0000 7ff5a8200640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5a0162fd0 con 0x7ff5a0162bc0 2024-04-20T11:29:17.332 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.331+0000 7ff5a8200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5a00d0dc0 con 0x7ff5a00b3930 2024-04-20T11:29:17.332 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.331+0000 7ff5a8200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7ff5a00d19d0 con 0x7ff5a00d0800 2024-04-20T11:29:17.332 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.331+0000 7ff5a4e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7ff5a00d0800 0x7ff5a0162680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:17.332 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.332+0000 7ff5a6200640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7ff5a0162bc0 0x7ff5a015e1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:52608/0 (socket says 172.21.15.152:52608) 2024-04-20T11:29:17.333 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.332+0000 7ff5a6200640 1 -- 172.21.15.152:0/2379273751 learned_addr learned my addr 172.21.15.152:0/2379273751 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:29:17.333 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.332+0000 7ff5a4e00640 1 -- 172.21.15.152:0/2379273751 >> v1:172.21.15.175:6789/0 conn(0x7ff5a00b3930 legacy=0x7ff5a009b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:17.333 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.332+0000 7ff5a4e00640 1 -- 172.21.15.152:0/2379273751 >> v1:172.21.15.152:6789/0 conn(0x7ff5a0162bc0 legacy=0x7ff5a015e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:17.333 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.332+0000 7ff59fe00640 1 -- 172.21.15.152:0/2379273751 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1256140497 0 0) 0x7ff598001490 con 0x7ff5a0162bc0 2024-04-20T11:29:17.333 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.332+0000 7ff5a4e00640 1 -- 172.21.15.152:0/2379273751 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5a00d10a0 con 0x7ff5a00d0800 2024-04-20T11:29:17.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.334+0000 7ff5a4e00640 1 --2- 172.21.15.152:0/2379273751 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff5a00d0800 0x7ff5a0162680 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7ff58c0097a0 tx=0x7ff58c02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac4e1105c1497a05 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:17.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.334+0000 7ff59fe00640 1 -- 172.21.15.152:0/2379273751 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff58c034070 con 0x7ff5a00d0800 2024-04-20T11:29:17.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.334+0000 7ff59fe00640 1 -- 172.21.15.152:0/2379273751 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff58c030610 con 0x7ff5a00d0800 2024-04-20T11:29:17.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.334+0000 7ff59fe00640 1 -- 172.21.15.152:0/2379273751 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff58c0309e0 con 0x7ff5a00d0800 2024-04-20T11:29:17.335 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.334+0000 7ff5a8200640 1 -- 172.21.15.152:0/2379273751 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff5a00d0800 msgr2=0x7ff5a0162680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:17.335 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.334+0000 7ff5a8200640 1 --2- 172.21.15.152:0/2379273751 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff5a00d0800 0x7ff5a0162680 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7ff58c0097a0 tx=0x7ff58c02b960 comp rx=0 tx=0).stop 2024-04-20T11:29:17.335 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.335+0000 7ff5a8200640 1 -- 172.21.15.152:0/2379273751 shutdown_connections 2024-04-20T11:29:17.335 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.335+0000 7ff5a8200640 1 --2- 172.21.15.152:0/2379273751 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff5a00d0800 0x7ff5a0162680 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:17.335 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.335+0000 7ff5a8200640 1 -- 172.21.15.152:0/2379273751 >> 172.21.15.152:0/2379273751 conn(0x7ff5a00b4020 msgr2=0x7ff5a00b4430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:17.336 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.335+0000 7ff5a8200640 1 -- 172.21.15.152:0/2379273751 shutdown_connections 2024-04-20T11:29:17.336 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.335+0000 7ff5a8200640 1 -- 172.21.15.152:0/2379273751 wait complete. 2024-04-20T11:29:17.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.336+0000 7ff5a8200640 1 Processor -- start 2024-04-20T11:29:17.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.336+0000 7ff5a8200640 1 -- start start 2024-04-20T11:29:17.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.337+0000 7ff5a8200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff5a00b3930 0x7ff5a01f4540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:17.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.337+0000 7ff5a8200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff5a00d0800 0x7ff5a015f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:17.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.337+0000 7ff5a8200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5a0162bc0 0x7ff5a015fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:17.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.337+0000 7ff5a8200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a00d1660 con 0x7ff5a00b3930 2024-04-20T11:29:17.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.337+0000 7ff5a8200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a009bdb0 con 0x7ff5a0162bc0 2024-04-20T11:29:17.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.337+0000 7ff5a8200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff5a00b3d10 con 0x7ff5a00d0800 2024-04-20T11:29:17.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.338+0000 7ff5a5800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff5a00b3930 0x7ff5a01f4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:17.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.338+0000 7ff5a5800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff5a00b3930 0x7ff5a01f4540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:50760/0 (socket says 172.21.15.152:50760) 2024-04-20T11:29:17.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.338+0000 7ff5a5800640 1 -- 172.21.15.152:0/4086407370 learned_addr learned my addr 172.21.15.152:0/4086407370 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:29:17.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.338+0000 7ff5a5800640 1 -- 172.21.15.152:0/4086407370 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff5a00d0800 msgr2=0x7ff5a015f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:17.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.338+0000 7ff5a5800640 1 --2- 172.21.15.152:0/4086407370 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff5a00d0800 0x7ff5a015f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:17.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.338+0000 7ff5a5800640 1 -- 172.21.15.152:0/4086407370 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5a0162bc0 msgr2=0x7ff5a015fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:17.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.338+0000 7ff5a5800640 1 --2- 172.21.15.152:0/4086407370 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5a0162bc0 0x7ff5a015fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:17.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.338+0000 7ff5a5800640 1 -- 172.21.15.152:0/4086407370 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5a0145690 con 0x7ff5a00b3930 2024-04-20T11:29:17.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.338+0000 7ff5a5800640 1 --2- 172.21.15.152:0/4086407370 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff5a00b3930 0x7ff5a01f4540 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7ff59400c450 tx=0x7ff59400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:17.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.339+0000 7ff59d600640 1 -- 172.21.15.152:0/4086407370 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5940052d0 con 0x7ff5a00b3930 2024-04-20T11:29:17.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.339+0000 7ff59d600640 1 -- 172.21.15.152:0/4086407370 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff594005470 con 0x7ff5a00b3930 2024-04-20T11:29:17.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.339+0000 7ff5a8200640 1 -- 172.21.15.152:0/4086407370 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5a0145900 con 0x7ff5a00b3930 2024-04-20T11:29:17.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.339+0000 7ff59d600640 1 -- 172.21.15.152:0/4086407370 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff594012870 con 0x7ff5a00b3930 2024-04-20T11:29:17.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.339+0000 7ff5a8200640 1 -- 172.21.15.152:0/4086407370 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5a0145ec0 con 0x7ff5a00b3930 2024-04-20T11:29:17.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.340+0000 7ff5a8200640 1 -- 172.21.15.152:0/4086407370 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5a00689f0 con 0x7ff5a00b3930 2024-04-20T11:29:17.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.341+0000 7ff59d600640 1 -- 172.21.15.152:0/4086407370 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7ff594012a10 con 0x7ff5a00b3930 2024-04-20T11:29:17.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.343+0000 7ff59d600640 1 --2- 172.21.15.152:0/4086407370 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff58006caa0 0x7ff58006ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:17.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.344+0000 7ff59d600640 1 -- 172.21.15.152:0/4086407370 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7ff59408bbd0 con 0x7ff5a00b3930 2024-04-20T11:29:17.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.344+0000 7ff5a4e00640 1 --2- 172.21.15.152:0/4086407370 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff58006caa0 0x7ff58006ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:17.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.344+0000 7ff5a4e00640 1 --2- 172.21.15.152:0/4086407370 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff58006caa0 0x7ff58006ef60 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7ff58c036000 tx=0x7ff58c030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:17.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.351+0000 7ff59d600640 1 -- 172.21.15.152:0/4086407370 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ff594059c50 con 0x7ff5a00b3930 2024-04-20T11:29:17.548 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.548+0000 7ff5a8200640 1 -- 172.21.15.152:0/4086407370 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff5a00b4c80 con 0x7ff5a00b3930 2024-04-20T11:29:17.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.549+0000 7ff59d600640 1 -- 172.21.15.152:0/4086407370 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v46) ==== 104+0+11 (secure 0 0 0) 0x7ff594059550 con 0x7ff5a00b3930 2024-04-20T11:29:17.549 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 32 2024-04-20T11:29:17.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.551+0000 7ff5a8200640 1 -- 172.21.15.152:0/4086407370 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff58006caa0 msgr2=0x7ff58006ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:17.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.551+0000 7ff5a8200640 1 --2- 172.21.15.152:0/4086407370 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff58006caa0 0x7ff58006ef60 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7ff58c036000 tx=0x7ff58c030000 comp rx=0 tx=0).stop 2024-04-20T11:29:17.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.551+0000 7ff5a8200640 1 -- 172.21.15.152:0/4086407370 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff5a00b3930 msgr2=0x7ff5a01f4540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:17.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.551+0000 7ff5a8200640 1 --2- 172.21.15.152:0/4086407370 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff5a00b3930 0x7ff5a01f4540 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7ff59400c450 tx=0x7ff59400c910 comp rx=0 tx=0).stop 2024-04-20T11:29:17.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.551+0000 7ff5a8200640 1 -- 172.21.15.152:0/4086407370 shutdown_connections 2024-04-20T11:29:17.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.551+0000 7ff5a8200640 1 --2- 172.21.15.152:0/4086407370 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff58006caa0 0x7ff58006ef60 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:17.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.551+0000 7ff5a8200640 1 --2- 172.21.15.152:0/4086407370 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5a0162bc0 0x7ff5a015fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:17.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.551+0000 7ff5a8200640 1 --2- 172.21.15.152:0/4086407370 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff5a00d0800 0x7ff5a015f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:17.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.551+0000 7ff5a8200640 1 --2- 172.21.15.152:0/4086407370 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff5a00b3930 0x7ff5a01f4540 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:17.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.551+0000 7ff5a8200640 1 -- 172.21.15.152:0/4086407370 >> 172.21.15.152:0/4086407370 conn(0x7ff5a00b4020 msgr2=0x7ff5a0160780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:17.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.552+0000 7ff5a8200640 1 -- 172.21.15.152:0/4086407370 shutdown_connections 2024-04-20T11:29:17.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.552+0000 7ff5a8200640 1 -- 172.21.15.152:0/4086407370 wait complete. 2024-04-20T11:29:17.571 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T11:29:17.777 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.776+0000 7f226f200640 1 Processor -- start 2024-04-20T11:29:17.777 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.776+0000 7f226f200640 1 -- start start 2024-04-20T11:29:17.779 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.777+0000 7f226f200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f22700cf5b0 0x7f22700d4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:17.779 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.777+0000 7f226f200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f22700ceeb0 0x7f22700cf990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:17.779 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.777+0000 7f226f200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2270162430 0x7f22701628e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:17.779 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.778+0000 7f226f200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f22700d5420 con 0x7f22700ceeb0 2024-04-20T11:29:17.779 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.778+0000 7f226f200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f22700cfed0 con 0x7f22700cf5b0 2024-04-20T11:29:17.779 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.778+0000 7f226f200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f22700d2b70 con 0x7f2270162430 2024-04-20T11:29:17.779 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.778+0000 7f2265400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f22700ceeb0 0x7f22700cf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:17.779 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.778+0000 7f2265400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f22700ceeb0 0x7f22700cf990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:50762/0 (socket says 172.21.15.152:50762) 2024-04-20T11:29:17.779 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.778+0000 7f2265400640 1 -- 172.21.15.152:0/3951216659 learned_addr learned my addr 172.21.15.152:0/3951216659 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:29:17.779 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.778+0000 7f226de00640 1 --2- 172.21.15.152:0/3951216659 >> v2:172.21.15.175:3300/0 conn(0x7f22700cf5b0 0x7f22700d4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:17.779 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.778+0000 7f226e800640 1 --2- 172.21.15.152:0/3951216659 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2270162430 0x7f22701628e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:17.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.778+0000 7f2265400640 1 -- 172.21.15.152:0/3951216659 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2270162430 msgr2=0x7f22701628e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:17.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.778+0000 7f2265400640 1 --2- 172.21.15.152:0/3951216659 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2270162430 0x7f22701628e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:17.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.778+0000 7f2265400640 1 -- 172.21.15.152:0/3951216659 >> v2:172.21.15.175:3300/0 conn(0x7f22700cf5b0 msgr2=0x7f22700d4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:17.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.778+0000 7f2265400640 1 --2- 172.21.15.152:0/3951216659 >> v2:172.21.15.175:3300/0 conn(0x7f22700cf5b0 0x7f22700d4ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:17.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.778+0000 7f2265400640 1 -- 172.21.15.152:0/3951216659 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22700d2e50 con 0x7f22700ceeb0 2024-04-20T11:29:17.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.780+0000 7f2265400640 1 --2- 172.21.15.152:0/3951216659 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f22700ceeb0 0x7f22700cf990 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f227008cc10 tx=0x7f225802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e48744e0475c454c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:17.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.780+0000 7f226d400640 1 -- 172.21.15.152:0/3951216659 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2258004230 con 0x7f22700ceeb0 2024-04-20T11:29:17.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.780+0000 7f226d400640 1 -- 172.21.15.152:0/3951216659 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22580043d0 con 0x7f22700ceeb0 2024-04-20T11:29:17.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.780+0000 7f226d400640 1 -- 172.21.15.152:0/3951216659 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2258030840 con 0x7f22700ceeb0 2024-04-20T11:29:17.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.780+0000 7f226f200640 1 -- 172.21.15.152:0/3951216659 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f22700ceeb0 msgr2=0x7f22700cf990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:17.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.780+0000 7f226f200640 1 --2- 172.21.15.152:0/3951216659 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f22700ceeb0 0x7f22700cf990 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f227008cc10 tx=0x7f225802b980 comp rx=0 tx=0).stop 2024-04-20T11:29:17.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.781+0000 7f226f200640 1 -- 172.21.15.152:0/3951216659 shutdown_connections 2024-04-20T11:29:17.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.781+0000 7f226f200640 1 --2- 172.21.15.152:0/3951216659 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2270162430 0x7f22701628e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:17.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.781+0000 7f226f200640 1 --2- 172.21.15.152:0/3951216659 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f22700ceeb0 0x7f22700cf990 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:17.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.781+0000 7f226f200640 1 --2- 172.21.15.152:0/3951216659 >> v2:172.21.15.175:3300/0 conn(0x7f22700cf5b0 0x7f22700d4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:17.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.781+0000 7f226f200640 1 -- 172.21.15.152:0/3951216659 >> 172.21.15.152:0/3951216659 conn(0x7f227008bfd0 msgr2=0x7f22700d4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:17.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.781+0000 7f226f200640 1 -- 172.21.15.152:0/3951216659 shutdown_connections 2024-04-20T11:29:17.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.781+0000 7f226f200640 1 -- 172.21.15.152:0/3951216659 wait complete. 2024-04-20T11:29:17.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.782+0000 7f226f200640 1 Processor -- start 2024-04-20T11:29:17.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.782+0000 7f226f200640 1 -- start start 2024-04-20T11:29:17.785 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.783+0000 7f226f200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f22700ceeb0 0x7f2270159f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:17.785 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.783+0000 7f226f200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f22700cf5b0 0x7f22700b8d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:17.785 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.783+0000 7f226f200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2270162430 0x7f22700b92b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:17.785 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.783+0000 7f226f200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f22700d36f0 con 0x7f22700cf5b0 2024-04-20T11:29:17.785 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.783+0000 7f226f200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f22700d3410 con 0x7f2270162430 2024-04-20T11:29:17.785 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.783+0000 7f226f200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f22700d3c10 con 0x7f22700ceeb0 2024-04-20T11:29:17.785 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.784+0000 7f2265400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f22700cf5b0 0x7f22700b8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:17.785 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.784+0000 7f2265400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f22700cf5b0 0x7f22700b8d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:50776/0 (socket says 172.21.15.152:50776) 2024-04-20T11:29:17.785 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.784+0000 7f2265400640 1 -- 172.21.15.152:0/3524525662 learned_addr learned my addr 172.21.15.152:0/3524525662 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:29:17.785 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.784+0000 7f226de00640 1 --2- 172.21.15.152:0/3524525662 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f22700ceeb0 0x7f2270159f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:17.785 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.784+0000 7f226e800640 1 --2- 172.21.15.152:0/3524525662 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2270162430 0x7f22700b92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:17.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.784+0000 7f2265400640 1 -- 172.21.15.152:0/3524525662 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f22700ceeb0 msgr2=0x7f2270159f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:17.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.784+0000 7f2265400640 1 --2- 172.21.15.152:0/3524525662 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f22700ceeb0 0x7f2270159f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:17.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.784+0000 7f2265400640 1 -- 172.21.15.152:0/3524525662 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2270162430 msgr2=0x7f22700b92b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:17.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.784+0000 7f2265400640 1 --2- 172.21.15.152:0/3524525662 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2270162430 0x7f22700b92b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:17.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.784+0000 7f2265400640 1 -- 172.21.15.152:0/3524525662 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f227015aaf0 con 0x7f22700cf5b0 2024-04-20T11:29:17.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.785+0000 7f2265400640 1 --2- 172.21.15.152:0/3524525662 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f22700cf5b0 0x7f22700b8d70 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f22580090b0 tx=0x7f22580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:17.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.785+0000 7f2266a00640 1 -- 172.21.15.152:0/3524525662 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2258047070 con 0x7f22700cf5b0 2024-04-20T11:29:17.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.785+0000 7f2266a00640 1 -- 172.21.15.152:0/3524525662 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2258030c00 con 0x7f22700cf5b0 2024-04-20T11:29:17.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.785+0000 7f226f200640 1 -- 172.21.15.152:0/3524525662 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22701e8e10 con 0x7f22700cf5b0 2024-04-20T11:29:17.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.785+0000 7f2266a00640 1 -- 172.21.15.152:0/3524525662 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2258003610 con 0x7f22700cf5b0 2024-04-20T11:29:17.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.785+0000 7f226f200640 1 -- 172.21.15.152:0/3524525662 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22701e93d0 con 0x7f22700cf5b0 2024-04-20T11:29:17.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.786+0000 7f226f200640 1 -- 172.21.15.152:0/3524525662 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2270075e60 con 0x7f22700cf5b0 2024-04-20T11:29:17.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.787+0000 7f2266a00640 1 -- 172.21.15.152:0/3524525662 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f225804a070 con 0x7f22700cf5b0 2024-04-20T11:29:17.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.788+0000 7f2266a00640 1 --2- 172.21.15.152:0/3524525662 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f224c06caa0 0x7f224c06ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:17.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.788+0000 7f2266a00640 1 -- 172.21.15.152:0/3524525662 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f22580adec0 con 0x7f22700cf5b0 2024-04-20T11:29:17.789 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.788+0000 7f226de00640 1 --2- 172.21.15.152:0/3524525662 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f224c06caa0 0x7f224c06ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:17.789 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.789+0000 7f226de00640 1 --2- 172.21.15.152:0/3524525662 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f224c06caa0 0x7f224c06ef60 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f2260006fd0 tx=0x7f2260008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:17.793 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.793+0000 7f2266a00640 1 -- 172.21.15.152:0/3524525662 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f227015aaf0 con 0x7f22700cf5b0 2024-04-20T11:29:17.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.994+0000 7f226f200640 1 -- 172.21.15.152:0/3524525662 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f22700ba4d0 con 0x7f22700cf5b0 2024-04-20T11:29:17.995 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.995+0000 7f2266a00640 1 -- 172.21.15.152:0/3524525662 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v46) ==== 100+0+11 (secure 0 0 0) 0x7f2258034020 con 0x7f22700cf5b0 2024-04-20T11:29:17.995 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 64 2024-04-20T11:29:17.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.997+0000 7f226f200640 1 -- 172.21.15.152:0/3524525662 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f224c06caa0 msgr2=0x7f224c06ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:17.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.997+0000 7f226f200640 1 --2- 172.21.15.152:0/3524525662 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f224c06caa0 0x7f224c06ef60 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f2260006fd0 tx=0x7f2260008040 comp rx=0 tx=0).stop 2024-04-20T11:29:17.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.997+0000 7f226f200640 1 -- 172.21.15.152:0/3524525662 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f22700cf5b0 msgr2=0x7f22700b8d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:17.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.997+0000 7f226f200640 1 --2- 172.21.15.152:0/3524525662 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f22700cf5b0 0x7f22700b8d70 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f22580090b0 tx=0x7f22580046c0 comp rx=0 tx=0).stop 2024-04-20T11:29:17.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.998+0000 7f226f200640 1 -- 172.21.15.152:0/3524525662 shutdown_connections 2024-04-20T11:29:17.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.998+0000 7f226f200640 1 --2- 172.21.15.152:0/3524525662 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f224c06caa0 0x7f224c06ef60 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:17.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.998+0000 7f226f200640 1 --2- 172.21.15.152:0/3524525662 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2270162430 0x7f22700b92b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:17.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.998+0000 7f226f200640 1 --2- 172.21.15.152:0/3524525662 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f22700cf5b0 0x7f22700b8d70 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:17.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.998+0000 7f226f200640 1 --2- 172.21.15.152:0/3524525662 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f22700ceeb0 0x7f2270159f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:17.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.998+0000 7f226f200640 1 -- 172.21.15.152:0/3524525662 >> 172.21.15.152:0/3524525662 conn(0x7f227008bfd0 msgr2=0x7f22700c9690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:17.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.998+0000 7f226f200640 1 -- 172.21.15.152:0/3524525662 shutdown_connections 2024-04-20T11:29:17.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:17.998+0000 7f226f200640 1 -- 172.21.15.152:0/3524525662 wait complete. 2024-04-20T11:29:18.018 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-20T11:29:18.219 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:18.218+0000 7fced4e00640 1 Processor -- start 2024-04-20T11:29:18.219 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:18.218+0000 7fced4e00640 1 -- start start 2024-04-20T11:29:18.220 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:18.220+0000 7fced4e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fced0161d30 0x7fced00d0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:18.220 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:18.220+0000 7fced4e00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fced00d26a0 con 0x7fced00d4ec0 2024-04-20T11:29:18.220 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:18.220+0000 7fced4e00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fced00d53c0 con 0x7fced00d2b80 2024-04-20T11:29:18.220 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:18.220+0000 7fced4e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fced00d0540 con 0x7fced0161d30 2024-04-20T11:29:18.221 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:18.220+0000 7fcecea00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fced00d4ec0 0x7fced00d15e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:52624/0 (socket says 172.21.15.152:52624) 2024-04-20T11:29:18.221 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:18.220+0000 7fcecea00640 1 -- 172.21.15.152:0/1636468585 learned_addr learned my addr 172.21.15.152:0/1636468585 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:29:18.222 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:18.220+0000 7fcecf400640 1 --2- 172.21.15.152:0/1636468585 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fced0161d30 0x7fced00d0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:18.223 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:18.220+0000 7fcece000640 1 -- 172.21.15.152:0/1636468585 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2930019597 0 0) 0x7fced00d26a0 con 0x7fced00d4ec0 2024-04-20T11:29:18.223 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:18.221+0000 7fcece000640 1 -- 172.21.15.152:0/1636468585 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fceb0003520 con 0x7fced00d4ec0 2024-04-20T11:29:18.223 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:18.221+0000 7fcece000640 1 -- 172.21.15.152:0/1636468585 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 771033736 0 0) 0x7fceb0003520 con 0x7fced00d4ec0 2024-04-20T11:29:18.223 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:18.221+0000 7fcece000640 1 -- 172.21.15.152:0/1636468585 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fced0161d30 msgr2=0x7fced00d0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:18.223 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:18.221+0000 7fcece000640 1 --2- 172.21.15.152:0/1636468585 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fced0161d30 0x7fced00d0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:18.223 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:18.221+0000 7fcece000640 1 -- 172.21.15.152:0/1636468585 >> v1:172.21.15.175:6789/0 conn(0x7fced00d2b80 legacy=0x7fced006c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:18.223 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:18.221+0000 7fcece000640 1 -- 172.21.15.152:0/1636468585 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fced00cf520 con 0x7fced00d4ec0 2024-04-20T11:29:18.223 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:18.221+0000 7fcece000640 1 -- 172.21.15.152:0/1636468585 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7fceb8001b70 con 0x7fced00d4ec0 2024-04-20T11:29:18.223 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:18.221+0000 7fcece000640 1 -- 172.21.15.152:0/1636468585 >> v1:172.21.15.152:6789/0 conn(0x7fced00d4ec0 legacy=0x7fced00d15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:18.223 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:18.221+0000 7fcece000640 1 --2- 172.21.15.152:0/1636468585 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fced0161d30 0x7fceb00039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:18.223 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:18.221+0000 7fcece000640 1 --2- 172.21.15.152:0/1636468585 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fceb0003fb0 0x7fceb0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:18.223 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:18.221+0000 7fcece000640 1 --2- 172.21.15.152:0/1636468585 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fceb0004d40 0x7fceb00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:18.223 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:18.221+0000 7fcece000640 1 -- 172.21.15.152:0/1636468585 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fced00d0540 con 0x7fceb0003fb0 2024-04-20T11:29:18.224 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:18.221+0000 7fcece000640 1 -- 172.21.15.152:0/1636468585 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fceb000bb00 con 0x7fceb0004d40 2024-04-20T11:29:18.224 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:18.221+0000 7fcece000640 1 -- 172.21.15.152:0/1636468585 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fceb000bd00 con 0x7fced0161d30 2024-04-20T11:29:18.224 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:18.221+0000 7fcecea00640 1 --2- 172.21.15.152:0/1636468585 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fceb0003fb0 0x7fceb0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:18.224 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:18.222+0000 7fcecfe00640 1 --2- 172.21.15.152:0/1636468585 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fceb0004d40 0x7fceb00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:18.224 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:18.222+0000 7fcecf400640 1 --2- 172.21.15.152:0/1636468585 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fced0161d30 0x7fceb00039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:18.224 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:18.222+0000 7fcecea00640 1 -- 172.21.15.152:0/1636468585 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fced0161d30 msgr2=0x7fceb00039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:18.224 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:18.222+0000 7fcecea00640 1 --2- 172.21.15.152:0/1636468585 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fced0161d30 0x7fceb00039e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:18.224 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:18.222+0000 7fcecea00640 1 -- 172.21.15.152:0/1636468585 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fceb0004d40 msgr2=0x7fceb00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:18.224 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:18.222+0000 7fcecea00640 1 --2- 172.21.15.152:0/1636468585 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fceb0004d40 0x7fceb00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:18.224 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:18.222+0000 7fcecea00640 1 -- 172.21.15.152:0/1636468585 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fceb0003520 con 0x7fceb0003fb0 2024-04-20T11:29:18.224 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:18.222+0000 7fcecfe00640 1 --2- 172.21.15.152:0/1636468585 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fceb0004d40 0x7fceb00051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:29:18.225 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:18.223+0000 7fcecea00640 1 --2- 172.21.15.152:0/1636468585 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fceb0003fb0 0x7fceb0004800 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fceb800c490 tx=0x7fceb0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fa789b8a1e83f12 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:18.225 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:18.223+0000 7fcece000640 1 -- 172.21.15.152:0/1636468585 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceb800c0e0 con 0x7fceb0003fb0 2024-04-20T11:29:18.225 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:18.223+0000 7fcece000640 1 -- 172.21.15.152:0/1636468585 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fceb8030070 con 0x7fceb0003fb0 2024-04-20T11:29:18.225 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:18.223+0000 7fcece000640 1 -- 172.21.15.152:0/1636468585 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceb802cab0 con 0x7fceb0003fb0 2024-04-20T11:29:18.225 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:18.224+0000 7fced4e00640 1 -- 172.21.15.152:0/1636468585 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fceb0003fb0 msgr2=0x7fceb0004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:18.225 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:18.224+0000 7fced4e00640 1 --2- 172.21.15.152:0/1636468585 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fceb0003fb0 0x7fceb0004800 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fceb800c490 tx=0x7fceb0005830 comp rx=0 tx=0).stop 2024-04-20T11:29:18.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:18.224+0000 7fcecfe00640 1 -- 172.21.15.152:0/1636468585 reap_dead start 2024-04-20T11:29:18.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:18.224+0000 7fced4e00640 1 -- 172.21.15.152:0/1636468585 shutdown_connections 2024-04-20T11:29:18.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:18.224+0000 7fced4e00640 1 -- 172.21.15.152:0/1636468585 >> 172.21.15.152:0/1636468585 conn(0x7fced008bfd0 msgr2=0x7fced00d45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:18.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:18.224+0000 7fced4e00640 1 -- 172.21.15.152:0/1636468585 shutdown_connections 2024-04-20T11:29:18.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:18.224+0000 7fced4e00640 1 -- 172.21.15.152:0/1636468585 wait complete. 2024-04-20T11:29:18.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:18.226+0000 7fced4e00640 1 Processor -- start 2024-04-20T11:29:18.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:18.226+0000 7fced4e00640 1 -- start start 2024-04-20T11:29:18.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:18.227+0000 7fced4e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fceb0003fb0 0x7fced014e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:18.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:18.227+0000 7fced4e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fced00b6790 0x7fced01f8250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:18.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:18.227+0000 7fced4e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fced01f8790 0x7fced01f8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:18.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:18.227+0000 7fced4e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fced00cf1b0 con 0x7fced00b6790 2024-04-20T11:29:18.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:18.227+0000 7fced4e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fced00cfb70 con 0x7fced01f8790 2024-04-20T11:29:18.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:18.227+0000 7fced4e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fced0162110 con 0x7fceb0003fb0 2024-04-20T11:29:18.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:18.227+0000 7fcecea00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fced00b6790 0x7fced01f8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:18.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:18.227+0000 7fcecea00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fced00b6790 0x7fced01f8250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:50786/0 (socket says 172.21.15.152:50786) 2024-04-20T11:29:18.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:18.227+0000 7fcecea00640 1 -- 172.21.15.152:0/1975179170 learned_addr learned my addr 172.21.15.152:0/1975179170 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:29:18.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:18.228+0000 7fcecf400640 1 --2- 172.21.15.152:0/1975179170 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fceb0003fb0 0x7fced014e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:18.230 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:18.228+0000 7fcecfe00640 1 --2- 172.21.15.152:0/1975179170 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fced01f8790 0x7fced01f8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:18.230 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:18.228+0000 7fcecea00640 1 -- 172.21.15.152:0/1975179170 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fceb0003fb0 msgr2=0x7fced014e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:18.230 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:18.228+0000 7fcecea00640 1 --2- 172.21.15.152:0/1975179170 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fceb0003fb0 0x7fced014e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:18.230 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:18.228+0000 7fcecea00640 1 -- 172.21.15.152:0/1975179170 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fced01f8790 msgr2=0x7fced01f8c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:18.230 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:18.228+0000 7fcecea00640 1 --2- 172.21.15.152:0/1975179170 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fced01f8790 0x7fced01f8c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:18.230 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:18.228+0000 7fcecea00640 1 -- 172.21.15.152:0/1975179170 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fced00cd940 con 0x7fced00b6790 2024-04-20T11:29:18.230 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:18.228+0000 7fcecea00640 1 --2- 172.21.15.152:0/1975179170 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fced00b6790 0x7fced01f8250 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fceb8001270 tx=0x7fced008d840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:18.230 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:18.228+0000 7fcec7400640 1 -- 172.21.15.152:0/1975179170 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceb8046070 con 0x7fced00b6790 2024-04-20T11:29:18.230 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:18.229+0000 7fcec7400640 1 -- 172.21.15.152:0/1975179170 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fceb8007070 con 0x7fced00b6790 2024-04-20T11:29:18.230 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:18.229+0000 7fcec7400640 1 -- 172.21.15.152:0/1975179170 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceb8007440 con 0x7fced00b6790 2024-04-20T11:29:18.230 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:18.229+0000 7fced4e00640 1 -- 172.21.15.152:0/1975179170 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fced00cdc00 con 0x7fced00b6790 2024-04-20T11:29:18.230 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:18.229+0000 7fced4e00640 1 -- 172.21.15.152:0/1975179170 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fced00ce1c0 con 0x7fced00b6790 2024-04-20T11:29:18.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:18.230+0000 7fced4e00640 1 -- 172.21.15.152:0/1975179170 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fced0075e60 con 0x7fced00b6790 2024-04-20T11:29:18.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:18.231+0000 7fcec7400640 1 -- 172.21.15.152:0/1975179170 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7fceb8049070 con 0x7fced00b6790 2024-04-20T11:29:18.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:18.232+0000 7fcec7400640 1 --2- 172.21.15.152:0/1975179170 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fceac06caa0 0x7fceac06ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:18.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:18.232+0000 7fcec7400640 1 -- 172.21.15.152:0/1975179170 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7fceb80ad820 con 0x7fced00b6790 2024-04-20T11:29:18.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:18.232+0000 7fcecf400640 1 --2- 172.21.15.152:0/1975179170 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fceac06caa0 0x7fceac06ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:18.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:18.233+0000 7fcecf400640 1 --2- 172.21.15.152:0/1975179170 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fceac06caa0 0x7fceac06ef60 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fced00b9050 tx=0x7fcec0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:18.237 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:18.237+0000 7fcec7400640 1 -- 172.21.15.152:0/1975179170 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fceb807b930 con 0x7fced00b6790 2024-04-20T11:29:18.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:18.450+0000 7fced4e00640 1 -- 172.21.15.152:0/1975179170 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fced00d1d70 con 0x7fced00b6790 2024-04-20T11:29:18.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:18.451+0000 7fcec7400640 1 -- 172.21.15.152:0/1975179170 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v46) ==== 80+0+0 (secure 0 0 0) 0x7fceb807b230 con 0x7fced00b6790 2024-04-20T11:29:18.451 INFO:teuthology.orchestra.run.smithi152.stderr:listed 0 entries 2024-04-20T11:29:18.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:18.453+0000 7fced4e00640 1 -- 172.21.15.152:0/1975179170 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fceac06caa0 msgr2=0x7fceac06ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:18.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:18.454+0000 7fced4e00640 1 --2- 172.21.15.152:0/1975179170 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fceac06caa0 0x7fceac06ef60 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fced00b9050 tx=0x7fcec0008040 comp rx=0 tx=0).stop 2024-04-20T11:29:18.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:18.454+0000 7fced4e00640 1 -- 172.21.15.152:0/1975179170 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fced00b6790 msgr2=0x7fced01f8250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:18.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:18.454+0000 7fced4e00640 1 --2- 172.21.15.152:0/1975179170 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fced00b6790 0x7fced01f8250 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fceb8001270 tx=0x7fced008d840 comp rx=0 tx=0).stop 2024-04-20T11:29:18.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:18.454+0000 7fced4e00640 1 -- 172.21.15.152:0/1975179170 shutdown_connections 2024-04-20T11:29:18.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:18.454+0000 7fced4e00640 1 --2- 172.21.15.152:0/1975179170 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fceac06caa0 0x7fceac06ef60 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:18.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:18.454+0000 7fced4e00640 1 --2- 172.21.15.152:0/1975179170 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fced01f8790 0x7fced01f8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:18.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:18.454+0000 7fced4e00640 1 --2- 172.21.15.152:0/1975179170 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fced00b6790 0x7fced01f8250 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:18.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:18.454+0000 7fced4e00640 1 --2- 172.21.15.152:0/1975179170 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fceb0003fb0 0x7fced014e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:18.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:18.454+0000 7fced4e00640 1 -- 172.21.15.152:0/1975179170 >> 172.21.15.152:0/1975179170 conn(0x7fced008bfd0 msgr2=0x7fced00c80d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:18.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:18.454+0000 7fced4e00640 1 -- 172.21.15.152:0/1975179170 shutdown_connections 2024-04-20T11:29:18.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:18.454+0000 7fced4e00640 1 -- 172.21.15.152:0/1975179170 wait complete. 2024-04-20T11:29:18.473 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi152.front.sepia.ceph.com... 2024-04-20T11:29:18.473 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:29:18.474 DEBUG:teuthology.orchestra.run.smithi152:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-20T11:29:18.548 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi152.stderr:ceph-fuse[39862]: fuse finished with error 0 and tester_r 0 2024-04-20T11:29:18.553 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T11:29:21.607 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:29:21.615 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:29:24.655 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi152.front.sepia.ceph.com 2024-04-20T11:29:24.655 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:29:24.656 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-20T11:29:24.714 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T11:29:24.714 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:29:24.772 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:29:24.978 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:24.978+0000 7ff3e3200640 1 Processor -- start 2024-04-20T11:29:24.978 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:24.978+0000 7ff3e3200640 1 -- start start 2024-04-20T11:29:24.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:24.980+0000 7ff3e3200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7ff3e415ca20 0x7ff3e40d1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:24.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:24.980+0000 7ff3e3200640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3e40b5380 con 0x7ff3e40cecb0 2024-04-20T11:29:24.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:24.980+0000 7ff3e3200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3e40b5660 con 0x7ff3e409aa40 2024-04-20T11:29:24.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:24.980+0000 7ff3e3200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7ff3e40b5940 con 0x7ff3e415ca20 2024-04-20T11:29:24.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:24.980+0000 7ff3e1400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7ff3e415ca20 0x7ff3e40d1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:24.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:24.980+0000 7ff3e1400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7ff3e415ca20 0x7ff3e40d1ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:50788/0 (socket says 172.21.15.152:50788) 2024-04-20T11:29:24.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:24.980+0000 7ff3e1400640 1 -- 172.21.15.152:0/750257105 learned_addr learned my addr 172.21.15.152:0/750257105 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:29:24.982 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:24.981+0000 7ff3e1400640 1 -- 172.21.15.152:0/750257105 >> v1:172.21.15.175:6789/0 conn(0x7ff3e409aa40 legacy=0x7ff3e409ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:24.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:24.981+0000 7ff3e1400640 1 -- 172.21.15.152:0/750257105 >> v1:172.21.15.152:6789/0 conn(0x7ff3e40cecb0 legacy=0x7ff3e40cf140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:24.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:24.981+0000 7ff3e1400640 1 -- 172.21.15.152:0/750257105 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3e40b5c20 con 0x7ff3e415ca20 2024-04-20T11:29:24.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:24.982+0000 7ff3e1400640 1 --2- 172.21.15.152:0/750257105 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff3e415ca20 0x7ff3e40d1ad0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7ff3d0009870 tx=0x7ff3d002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a09350ea4cffd38a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:24.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:24.982+0000 7ff3e0a00640 1 -- 172.21.15.152:0/750257105 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3d0034070 con 0x7ff3e415ca20 2024-04-20T11:29:24.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:24.982+0000 7ff3e0a00640 1 -- 172.21.15.152:0/750257105 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3d0030860 con 0x7ff3e415ca20 2024-04-20T11:29:24.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:24.982+0000 7ff3e0a00640 1 -- 172.21.15.152:0/750257105 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3d0030c30 con 0x7ff3e415ca20 2024-04-20T11:29:24.984 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:24.983+0000 7ff3e3200640 1 -- 172.21.15.152:0/750257105 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff3e415ca20 msgr2=0x7ff3e40d1ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:24.984 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:24.983+0000 7ff3e3200640 1 --2- 172.21.15.152:0/750257105 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff3e415ca20 0x7ff3e40d1ad0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7ff3d0009870 tx=0x7ff3d002b9a0 comp rx=0 tx=0).stop 2024-04-20T11:29:24.984 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:24.983+0000 7ff3e3200640 1 -- 172.21.15.152:0/750257105 shutdown_connections 2024-04-20T11:29:24.984 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:24.983+0000 7ff3e3200640 1 --2- 172.21.15.152:0/750257105 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff3e415ca20 0x7ff3e40d1ad0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:24.984 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:24.983+0000 7ff3e3200640 1 -- 172.21.15.152:0/750257105 >> 172.21.15.152:0/750257105 conn(0x7ff3e40b3fe0 msgr2=0x7ff3e40b43f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:24.984 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:24.983+0000 7ff3e3200640 1 -- 172.21.15.152:0/750257105 shutdown_connections 2024-04-20T11:29:24.984 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:24.983+0000 7ff3e3200640 1 -- 172.21.15.152:0/750257105 wait complete. 2024-04-20T11:29:24.985 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:24.985+0000 7ff3e3200640 1 Processor -- start 2024-04-20T11:29:24.985 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:24.985+0000 7ff3e3200640 1 -- start start 2024-04-20T11:29:24.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:24.986+0000 7ff3e3200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff3e409aa40 0x7ff3e40cbec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:24.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:24.986+0000 7ff3e3200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff3e40cecb0 0x7ff3e41538d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:24.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:24.986+0000 7ff3e3200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff3e415ca20 0x7ff3e415ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:24.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:24.986+0000 7ff3e3200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7ff3e40b7430 con 0x7ff3e409aa40 2024-04-20T11:29:24.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:24.986+0000 7ff3e3200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff3e40d2010 con 0x7ff3e40cecb0 2024-04-20T11:29:24.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:24.986+0000 7ff3e3200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff3e40b75b0 con 0x7ff3e415ca20 2024-04-20T11:29:24.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:24.986+0000 7ff3e1e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff3e409aa40 0x7ff3e40cbec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:24.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:24.986+0000 7ff3e1e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff3e409aa40 0x7ff3e40cbec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:50804/0 (socket says 172.21.15.152:50804) 2024-04-20T11:29:24.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:24.986+0000 7ff3e1e00640 1 -- 172.21.15.152:0/2657734583 learned_addr learned my addr 172.21.15.152:0/2657734583 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:29:24.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:24.987+0000 7ff3e2800640 1 --2- 172.21.15.152:0/2657734583 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff3e415ca20 0x7ff3e415ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:24.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:24.987+0000 7ff3e1e00640 1 -- 172.21.15.152:0/2657734583 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff3e415ca20 msgr2=0x7ff3e415ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:24.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:24.987+0000 7ff3e1400640 1 --2- 172.21.15.152:0/2657734583 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff3e40cecb0 0x7ff3e41538d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:24.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:24.987+0000 7ff3e1e00640 1 --2- 172.21.15.152:0/2657734583 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff3e415ca20 0x7ff3e415ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:24.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:24.987+0000 7ff3e1e00640 1 -- 172.21.15.152:0/2657734583 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff3e40cecb0 msgr2=0x7ff3e41538d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:24.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:24.987+0000 7ff3e1e00640 1 --2- 172.21.15.152:0/2657734583 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff3e40cecb0 0x7ff3e41538d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:24.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:24.987+0000 7ff3e1e00640 1 -- 172.21.15.152:0/2657734583 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3e40cc5f0 con 0x7ff3e409aa40 2024-04-20T11:29:24.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:24.987+0000 7ff3e2800640 1 --2- 172.21.15.152:0/2657734583 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff3e415ca20 0x7ff3e415ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:29:24.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:24.987+0000 7ff3e1e00640 1 --2- 172.21.15.152:0/2657734583 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff3e409aa40 0x7ff3e40cbec0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7ff3d400c460 tx=0x7ff3d400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:24.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:24.987+0000 7ff3da000640 1 -- 172.21.15.152:0/2657734583 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3d4016020 con 0x7ff3e409aa40 2024-04-20T11:29:24.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:24.987+0000 7ff3da000640 1 -- 172.21.15.152:0/2657734583 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3d4012870 con 0x7ff3e409aa40 2024-04-20T11:29:24.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:24.987+0000 7ff3da000640 1 -- 172.21.15.152:0/2657734583 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3d4012c60 con 0x7ff3e409aa40 2024-04-20T11:29:24.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:24.988+0000 7ff3e3200640 1 -- 172.21.15.152:0/2657734583 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3e40cc8d0 con 0x7ff3e409aa40 2024-04-20T11:29:24.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:24.988+0000 7ff3e3200640 1 -- 172.21.15.152:0/2657734583 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3e40cce90 con 0x7ff3e409aa40 2024-04-20T11:29:24.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:24.989+0000 7ff3e3200640 1 -- 172.21.15.152:0/2657734583 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3e4075e60 con 0x7ff3e409aa40 2024-04-20T11:29:24.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:24.990+0000 7ff3da000640 1 -- 172.21.15.152:0/2657734583 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7ff3d4019020 con 0x7ff3e409aa40 2024-04-20T11:29:24.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:24.991+0000 7ff3da000640 1 --2- 172.21.15.152:0/2657734583 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff3c406caa0 0x7ff3c406ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:24.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:24.991+0000 7ff3da000640 1 -- 172.21.15.152:0/2657734583 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7ff3d408bb80 con 0x7ff3e409aa40 2024-04-20T11:29:24.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:24.991+0000 7ff3e1400640 1 --2- 172.21.15.152:0/2657734583 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff3c406caa0 0x7ff3c406ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:24.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:24.992+0000 7ff3e1400640 1 --2- 172.21.15.152:0/2657734583 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff3c406caa0 0x7ff3c406ef60 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7ff3d0036000 tx=0x7ff3d003b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:24.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:24.996+0000 7ff3da000640 1 -- 172.21.15.152:0/2657734583 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ff3d4020030 con 0x7ff3e409aa40 2024-04-20T11:29:25.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.195+0000 7ff3e3200640 1 -- 172.21.15.152:0/2657734583 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff3e415d340 con 0x7ff3e409aa40 2024-04-20T11:29:25.199 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.198+0000 7ff3da000640 1 -- 172.21.15.152:0/2657734583 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) ==== 74+0+16673 (secure 0 0 0) 0x7ff3d4059d30 con 0x7ff3e409aa40 2024-04-20T11:29:25.200 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:29:25.200 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":46,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:29:01.812130+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:28:58.007718+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-20T11:28:59.260647+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:29:25.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.201+0000 7ff3e3200640 1 -- 172.21.15.152:0/2657734583 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff3c406caa0 msgr2=0x7ff3c406ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:25.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.201+0000 7ff3e3200640 1 --2- 172.21.15.152:0/2657734583 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff3c406caa0 0x7ff3c406ef60 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7ff3d0036000 tx=0x7ff3d003b040 comp rx=0 tx=0).stop 2024-04-20T11:29:25.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.201+0000 7ff3e3200640 1 -- 172.21.15.152:0/2657734583 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff3e409aa40 msgr2=0x7ff3e40cbec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:25.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.201+0000 7ff3e3200640 1 --2- 172.21.15.152:0/2657734583 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff3e409aa40 0x7ff3e40cbec0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7ff3d400c460 tx=0x7ff3d400c920 comp rx=0 tx=0).stop 2024-04-20T11:29:25.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.201+0000 7ff3e3200640 1 -- 172.21.15.152:0/2657734583 shutdown_connections 2024-04-20T11:29:25.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.201+0000 7ff3e3200640 1 --2- 172.21.15.152:0/2657734583 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff3c406caa0 0x7ff3c406ef60 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:25.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.201+0000 7ff3e3200640 1 --2- 172.21.15.152:0/2657734583 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff3e415ca20 0x7ff3e415ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:25.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.201+0000 7ff3e3200640 1 --2- 172.21.15.152:0/2657734583 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff3e40cecb0 0x7ff3e41538d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:25.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.201+0000 7ff3e3200640 1 --2- 172.21.15.152:0/2657734583 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff3e409aa40 0x7ff3e40cbec0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:25.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.201+0000 7ff3e3200640 1 -- 172.21.15.152:0/2657734583 >> 172.21.15.152:0/2657734583 conn(0x7ff3e40b3fe0 msgr2=0x7ff3e40cf120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:25.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.202+0000 7ff3e3200640 1 -- 172.21.15.152:0/2657734583 shutdown_connections 2024-04-20T11:29:25.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.202+0000 7ff3e3200640 1 -- 172.21.15.152:0/2657734583 wait complete. 2024-04-20T11:29:25.222 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T11:26:47.246571+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T11:28:58.007718+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T11:28:59.260647+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T11:29:25.222 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T11:29:25.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.414+0000 7f3ea0400640 1 Processor -- start 2024-04-20T11:29:25.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.414+0000 7f3ea0400640 1 -- start start 2024-04-20T11:29:25.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.415+0000 7f3ea0400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f3e9815e9e0 0x7f3e9806d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:25.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.416+0000 7f3ea0400640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e980d19a0 con 0x7f3e980d14a0 2024-04-20T11:29:25.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.416+0000 7f3ea0400640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e980d1c80 con 0x7f3e980d46a0 2024-04-20T11:29:25.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.416+0000 7f3ea0400640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f3e980d1f60 con 0x7f3e9815e9e0 2024-04-20T11:29:25.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.416+0000 7f3e9da00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f3e9815e9e0 0x7f3e9806d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:25.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.416+0000 7f3e9e400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f3e980d14a0 0x7f3e980cfda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:52638/0 (socket says 172.21.15.152:52638) 2024-04-20T11:29:25.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.416+0000 7f3e9e400640 1 -- 172.21.15.152:0/2076998353 learned_addr learned my addr 172.21.15.152:0/2076998353 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:29:25.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.416+0000 7f3e9da00640 1 -- 172.21.15.152:0/2076998353 >> v1:172.21.15.175:6789/0 conn(0x7f3e980d46a0 legacy=0x7f3e980d0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:25.418 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.416+0000 7f3e9da00640 1 -- 172.21.15.152:0/2076998353 >> v1:172.21.15.152:6789/0 conn(0x7f3e980d14a0 legacy=0x7f3e980cfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:25.418 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.416+0000 7f3e9da00640 1 -- 172.21.15.152:0/2076998353 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e980cf5a0 con 0x7f3e9815e9e0 2024-04-20T11:29:25.418 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.416+0000 7f3e97e00640 1 -- 172.21.15.152:0/2076998353 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4270416797 0 0) 0x7f3e980d19a0 con 0x7f3e980d14a0 2024-04-20T11:29:25.419 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.417+0000 7f3e9da00640 1 --2- 172.21.15.152:0/2076998353 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3e9815e9e0 0x7f3e9806d320 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f3e880097a0 tx=0x7f3e8802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acacb4c64a73656 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:25.419 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.418+0000 7f3e97e00640 1 -- 172.21.15.152:0/2076998353 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e88035020 con 0x7f3e9815e9e0 2024-04-20T11:29:25.419 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.418+0000 7f3e97e00640 1 -- 172.21.15.152:0/2076998353 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e88031830 con 0x7f3e9815e9e0 2024-04-20T11:29:25.419 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.418+0000 7f3e97e00640 1 -- 172.21.15.152:0/2076998353 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e88031c00 con 0x7f3e9815e9e0 2024-04-20T11:29:25.419 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.418+0000 7f3ea0400640 1 -- 172.21.15.152:0/2076998353 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3e9815e9e0 msgr2=0x7f3e9806d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:25.419 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.418+0000 7f3ea0400640 1 --2- 172.21.15.152:0/2076998353 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3e9815e9e0 0x7f3e9806d320 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f3e880097a0 tx=0x7f3e8802b8d0 comp rx=0 tx=0).stop 2024-04-20T11:29:25.419 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.418+0000 7f3ea0400640 1 -- 172.21.15.152:0/2076998353 shutdown_connections 2024-04-20T11:29:25.419 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.418+0000 7f3ea0400640 1 --2- 172.21.15.152:0/2076998353 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3e9815e9e0 0x7f3e9806d320 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:25.419 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.418+0000 7f3ea0400640 1 -- 172.21.15.152:0/2076998353 >> 172.21.15.152:0/2076998353 conn(0x7f3e9809dab0 msgr2=0x7f3e9808bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:25.420 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.418+0000 7f3ea0400640 1 -- 172.21.15.152:0/2076998353 shutdown_connections 2024-04-20T11:29:25.420 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.418+0000 7f3ea0400640 1 -- 172.21.15.152:0/2076998353 wait complete. 2024-04-20T11:29:25.420 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.420+0000 7f3ea0400640 1 Processor -- start 2024-04-20T11:29:25.420 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.420+0000 7f3ea0400640 1 -- start start 2024-04-20T11:29:25.423 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.421+0000 7f3ea0400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3e980d14a0 0x7f3e980b9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:25.423 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.421+0000 7f3ea0400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3e980d46a0 0x7f3e98152b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:25.423 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.421+0000 7f3ea0400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3e9815e9e0 0x7f3e98162ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:25.423 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.421+0000 7f3ea0400640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f3e9806d9e0 con 0x7f3e980d14a0 2024-04-20T11:29:25.423 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.421+0000 7f3ea0400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3e9806d860 con 0x7f3e9815e9e0 2024-04-20T11:29:25.423 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.421+0000 7f3ea0400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f3e9815edc0 con 0x7f3e980d46a0 2024-04-20T11:29:25.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.421+0000 7f3e9da00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3e980d14a0 0x7f3e980b9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:25.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.421+0000 7f3e9da00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3e980d14a0 0x7f3e980b9cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:50824/0 (socket says 172.21.15.152:50824) 2024-04-20T11:29:25.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.421+0000 7f3e9da00640 1 -- 172.21.15.152:0/2541517749 learned_addr learned my addr 172.21.15.152:0/2541517749 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:29:25.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.422+0000 7f3e9da00640 1 -- 172.21.15.152:0/2541517749 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3e980d46a0 msgr2=0x7f3e98152b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:25.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.422+0000 7f3e9d000640 1 --2- 172.21.15.152:0/2541517749 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3e980d46a0 0x7f3e98152b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:25.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.422+0000 7f3e9da00640 1 --2- 172.21.15.152:0/2541517749 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3e980d46a0 0x7f3e98152b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:25.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.422+0000 7f3e9da00640 1 -- 172.21.15.152:0/2541517749 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3e9815e9e0 msgr2=0x7f3e98162ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:25.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.422+0000 7f3e9da00640 1 --2- 172.21.15.152:0/2541517749 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3e9815e9e0 0x7f3e98162ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:25.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.422+0000 7f3e9da00640 1 -- 172.21.15.152:0/2541517749 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e980b9990 con 0x7f3e980d14a0 2024-04-20T11:29:25.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.422+0000 7f3e9da00640 1 --2- 172.21.15.152:0/2541517749 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3e980d14a0 0x7f3e980b9cb0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f3e880097a0 tx=0x7f3e880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:25.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.422+0000 7f3e95600640 1 -- 172.21.15.152:0/2541517749 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e8803aa70 con 0x7f3e980d14a0 2024-04-20T11:29:25.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.422+0000 7f3e95600640 1 -- 172.21.15.152:0/2541517749 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e8803ac10 con 0x7f3e980d14a0 2024-04-20T11:29:25.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.422+0000 7f3e95600640 1 -- 172.21.15.152:0/2541517749 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e880443f0 con 0x7f3e980d14a0 2024-04-20T11:29:25.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.422+0000 7f3ea0400640 1 -- 172.21.15.152:0/2541517749 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e9815fe20 con 0x7f3e980d14a0 2024-04-20T11:29:25.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.423+0000 7f3ea0400640 1 -- 172.21.15.152:0/2541517749 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e981606c0 con 0x7f3e980d14a0 2024-04-20T11:29:25.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.424+0000 7f3ea0400640 1 -- 172.21.15.152:0/2541517749 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e98160210 con 0x7f3e980d14a0 2024-04-20T11:29:25.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.425+0000 7f3e95600640 1 -- 172.21.15.152:0/2541517749 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f3e8804b060 con 0x7f3e980d14a0 2024-04-20T11:29:25.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.425+0000 7f3e95600640 1 --2- 172.21.15.152:0/2541517749 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f3e7806cad0 0x7f3e7806ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:25.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.425+0000 7f3e95600640 1 -- 172.21.15.152:0/2541517749 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f3e880adca0 con 0x7f3e980d14a0 2024-04-20T11:29:25.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.426+0000 7f3e9d000640 1 --2- 172.21.15.152:0/2541517749 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f3e7806cad0 0x7f3e7806ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:25.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.426+0000 7f3e9d000640 1 --2- 172.21.15.152:0/2541517749 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f3e7806cad0 0x7f3e7806ef90 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f3e980ba770 tx=0x7f3e84009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:25.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.430+0000 7f3e95600640 1 -- 172.21.15.152:0/2541517749 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f3e8803f070 con 0x7f3e980d14a0 2024-04-20T11:29:25.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.626+0000 7f3ea0400640 1 -- 172.21.15.152:0/2541517749 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3e9806cdb0 con 0x7f3e980d14a0 2024-04-20T11:29:25.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.627+0000 7f3e95600640 1 -- 172.21.15.152:0/2541517749 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v46) ==== 93+0+10 (secure 0 0 0) 0x7f3e8802f2e0 con 0x7f3e980d14a0 2024-04-20T11:29:25.630 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 1 2024-04-20T11:29:25.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.629+0000 7f3ea0400640 1 -- 172.21.15.152:0/2541517749 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f3e7806cad0 msgr2=0x7f3e7806ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:25.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.629+0000 7f3ea0400640 1 --2- 172.21.15.152:0/2541517749 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f3e7806cad0 0x7f3e7806ef90 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f3e980ba770 tx=0x7f3e84009040 comp rx=0 tx=0).stop 2024-04-20T11:29:25.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.629+0000 7f3ea0400640 1 -- 172.21.15.152:0/2541517749 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3e980d14a0 msgr2=0x7f3e980b9cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:25.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.629+0000 7f3ea0400640 1 --2- 172.21.15.152:0/2541517749 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3e980d14a0 0x7f3e980b9cb0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f3e880097a0 tx=0x7f3e880046c0 comp rx=0 tx=0).stop 2024-04-20T11:29:25.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.630+0000 7f3ea0400640 1 -- 172.21.15.152:0/2541517749 shutdown_connections 2024-04-20T11:29:25.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.630+0000 7f3ea0400640 1 --2- 172.21.15.152:0/2541517749 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f3e7806cad0 0x7f3e7806ef90 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:25.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.630+0000 7f3ea0400640 1 --2- 172.21.15.152:0/2541517749 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3e9815e9e0 0x7f3e98162ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:25.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.630+0000 7f3ea0400640 1 --2- 172.21.15.152:0/2541517749 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3e980d46a0 0x7f3e98152b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:25.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.630+0000 7f3ea0400640 1 --2- 172.21.15.152:0/2541517749 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3e980d14a0 0x7f3e980b9cb0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:25.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.630+0000 7f3ea0400640 1 -- 172.21.15.152:0/2541517749 >> 172.21.15.152:0/2541517749 conn(0x7f3e9809dab0 msgr2=0x7f3e9808bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:25.632 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.630+0000 7f3ea0400640 1 -- 172.21.15.152:0/2541517749 shutdown_connections 2024-04-20T11:29:25.632 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.630+0000 7f3ea0400640 1 -- 172.21.15.152:0/2541517749 wait complete. 2024-04-20T11:29:25.649 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T11:29:25.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.848+0000 7fcfaf800640 1 Processor -- start 2024-04-20T11:29:25.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.849+0000 7fcfaf800640 1 -- start start 2024-04-20T11:29:25.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.850+0000 7fcfaf800640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fcfa80afba0 0x7fcfa8162440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:25.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.850+0000 7fcfaf800640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcfa80adc30 con 0x7fcfa815e5d0 2024-04-20T11:29:25.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.850+0000 7fcfaf800640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcfa8162a70 con 0x7fcfa80b3fa0 2024-04-20T11:29:25.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.850+0000 7fcfaf800640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fcfa8162d50 con 0x7fcfa80afba0 2024-04-20T11:29:25.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.850+0000 7fcface00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fcfa815e5d0 0x7fcfa80af490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:42462/0 (socket says 172.21.15.152:42462) 2024-04-20T11:29:25.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.850+0000 7fcface00640 1 -- 172.21.15.152:0/3874644709 learned_addr learned my addr 172.21.15.152:0/3874644709 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:29:25.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.851+0000 7fcfa7400640 1 -- 172.21.15.152:0/3874644709 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 781694356 0 0) 0x7fcfa80adc30 con 0x7fcfa815e5d0 2024-04-20T11:29:25.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.851+0000 7fcfa7e00640 1 --2- 172.21.15.152:0/3874644709 >> v2:172.21.15.175:3300/0 conn(0x7fcfa80afba0 0x7fcfa8162440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:25.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.851+0000 7fcfa7400640 1 -- 172.21.15.152:0/3874644709 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcf90003520 con 0x7fcfa815e5d0 2024-04-20T11:29:25.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.851+0000 7fcfa7400640 1 -- 172.21.15.152:0/3874644709 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2495641572 0 0) 0x7fcf90003520 con 0x7fcfa815e5d0 2024-04-20T11:29:25.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.851+0000 7fcfa7400640 1 -- 172.21.15.152:0/3874644709 >> v2:172.21.15.175:3300/0 conn(0x7fcfa80afba0 msgr2=0x7fcfa8162440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:25.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.851+0000 7fcfa7400640 1 --2- 172.21.15.152:0/3874644709 >> v2:172.21.15.175:3300/0 conn(0x7fcfa80afba0 0x7fcfa8162440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:25.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.851+0000 7fcfa7400640 1 -- 172.21.15.152:0/3874644709 >> v1:172.21.15.175:6789/0 conn(0x7fcfa80b3fa0 legacy=0x7fcfa80ae110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:25.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.851+0000 7fcfa7400640 1 -- 172.21.15.152:0/3874644709 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfa8163030 con 0x7fcfa815e5d0 2024-04-20T11:29:25.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.851+0000 7fcfa7400640 1 -- 172.21.15.152:0/3874644709 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7fcf9c001670 con 0x7fcfa815e5d0 2024-04-20T11:29:25.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.851+0000 7fcfa7400640 1 -- 172.21.15.152:0/3874644709 >> v1:172.21.15.152:6789/0 conn(0x7fcfa815e5d0 legacy=0x7fcfa80af490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:25.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.852+0000 7fcfa7400640 1 --2- 172.21.15.152:0/3874644709 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcf90003a00 0x7fcf90003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:25.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.852+0000 7fcfa7400640 1 --2- 172.21.15.152:0/3874644709 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcf90004800 0x7fcf90004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:25.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.852+0000 7fcfa7400640 1 --2- 172.21.15.152:0/3874644709 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcf900051c0 0x7fcf9000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:25.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.852+0000 7fcfa7400640 1 -- 172.21.15.152:0/3874644709 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fcfa8162d50 con 0x7fcf90003a00 2024-04-20T11:29:25.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.852+0000 7fcfa7400640 1 -- 172.21.15.152:0/3874644709 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcf9000c0c0 con 0x7fcf90004800 2024-04-20T11:29:25.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.852+0000 7fcfa7400640 1 -- 172.21.15.152:0/3874644709 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fcf9000c2c0 con 0x7fcf900051c0 2024-04-20T11:29:25.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.852+0000 7fcface00640 1 --2- 172.21.15.152:0/3874644709 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcf90003a00 0x7fcf90003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:25.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.852+0000 7fcface00640 1 -- 172.21.15.152:0/3874644709 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcf900051c0 msgr2=0x7fcf9000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:25.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.852+0000 7fcface00640 1 --2- 172.21.15.152:0/3874644709 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcf900051c0 0x7fcf9000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:25.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.852+0000 7fcface00640 1 -- 172.21.15.152:0/3874644709 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcf90004800 msgr2=0x7fcf90004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:25.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.852+0000 7fcface00640 1 --2- 172.21.15.152:0/3874644709 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcf90004800 0x7fcf90004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:25.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.852+0000 7fcface00640 1 -- 172.21.15.152:0/3874644709 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf90003520 con 0x7fcf90003a00 2024-04-20T11:29:25.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.852+0000 7fcfad800640 1 -- 172.21.15.152:0/3874644709 reap_dead start 2024-04-20T11:29:25.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.853+0000 7fcface00640 1 --2- 172.21.15.152:0/3874644709 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcf90003a00 0x7fcf90003e20 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fcf9c00c110 tx=0x7fcf90005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a07ec7df44f225d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:25.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.854+0000 7fcfa7400640 1 -- 172.21.15.152:0/3874644709 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf9c007c40 con 0x7fcf90003a00 2024-04-20T11:29:25.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.854+0000 7fcfa7400640 1 -- 172.21.15.152:0/3874644709 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf9c030070 con 0x7fcf90003a00 2024-04-20T11:29:25.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.854+0000 7fcfa7400640 1 -- 172.21.15.152:0/3874644709 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf9c005d70 con 0x7fcf90003a00 2024-04-20T11:29:25.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.854+0000 7fcfaf800640 1 -- 172.21.15.152:0/3874644709 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcf90003a00 msgr2=0x7fcf90003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:25.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.854+0000 7fcfaf800640 1 --2- 172.21.15.152:0/3874644709 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcf90003a00 0x7fcf90003e20 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fcf9c00c110 tx=0x7fcf90005830 comp rx=0 tx=0).stop 2024-04-20T11:29:25.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.854+0000 7fcfaf800640 1 -- 172.21.15.152:0/3874644709 shutdown_connections 2024-04-20T11:29:25.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.854+0000 7fcfaf800640 1 --2- 172.21.15.152:0/3874644709 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcf90003a00 0x7fcf90003e20 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:25.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.854+0000 7fcfaf800640 1 -- 172.21.15.152:0/3874644709 >> 172.21.15.152:0/3874644709 conn(0x7fcfa806b980 msgr2=0x7fcfa806bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:25.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.854+0000 7fcfaf800640 1 -- 172.21.15.152:0/3874644709 shutdown_connections 2024-04-20T11:29:25.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.854+0000 7fcfaf800640 1 -- 172.21.15.152:0/3874644709 wait complete. 2024-04-20T11:29:25.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.856+0000 7fcfaf800640 1 Processor -- start 2024-04-20T11:29:25.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.856+0000 7fcfaf800640 1 -- start start 2024-04-20T11:29:25.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.857+0000 7fcfaf800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcf90003a00 0x7fcfa81560b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:25.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.857+0000 7fcfaf800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcfa8145300 0x7fcfa8161100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:25.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.857+0000 7fcfaf800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcfa80b1160 0x7fcfa80b15e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:25.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.857+0000 7fcfaf800640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fcfa806e0a0 con 0x7fcfa80b1160 2024-04-20T11:29:25.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.857+0000 7fcfaf800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcfa8163310 con 0x7fcfa8145300 2024-04-20T11:29:25.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.857+0000 7fcfaf800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fcfa806e220 con 0x7fcf90003a00 2024-04-20T11:29:25.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.857+0000 7fcfad800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcfa80b1160 0x7fcfa80b15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:25.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.857+0000 7fcfad800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcfa80b1160 0x7fcfa80b15e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35256/0 (socket says 172.21.15.152:35256) 2024-04-20T11:29:25.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.858+0000 7fcfad800640 1 -- 172.21.15.152:0/336620750 learned_addr learned my addr 172.21.15.152:0/336620750 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:29:25.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.858+0000 7fcface00640 1 --2- 172.21.15.152:0/336620750 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcf90003a00 0x7fcfa81560b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:25.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.858+0000 7fcfa7e00640 1 --2- 172.21.15.152:0/336620750 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcfa8145300 0x7fcfa8161100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:25.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.858+0000 7fcfad800640 1 -- 172.21.15.152:0/336620750 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcf90003a00 msgr2=0x7fcfa81560b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:25.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.858+0000 7fcfad800640 1 --2- 172.21.15.152:0/336620750 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcf90003a00 0x7fcfa81560b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:25.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.858+0000 7fcfad800640 1 -- 172.21.15.152:0/336620750 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcfa8145300 msgr2=0x7fcfa8161100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:25.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.858+0000 7fcfad800640 1 --2- 172.21.15.152:0/336620750 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcfa8145300 0x7fcfa8161100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:25.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.858+0000 7fcfad800640 1 -- 172.21.15.152:0/336620750 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfa81f8210 con 0x7fcfa80b1160 2024-04-20T11:29:25.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.858+0000 7fcfad800640 1 --2- 172.21.15.152:0/336620750 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcfa80b1160 0x7fcfa80b15e0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fcfa80b4460 tx=0x7fcfa000cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:25.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.858+0000 7fcfa4c00640 1 -- 172.21.15.152:0/336620750 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa0017070 con 0x7fcfa80b1160 2024-04-20T11:29:25.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.859+0000 7fcfa4c00640 1 -- 172.21.15.152:0/336620750 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfa0005d60 con 0x7fcfa80b1160 2024-04-20T11:29:25.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.859+0000 7fcfa4c00640 1 -- 172.21.15.152:0/336620750 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa000a850 con 0x7fcfa80b1160 2024-04-20T11:29:25.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.859+0000 7fcfaf800640 1 -- 172.21.15.152:0/336620750 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfa81f84f0 con 0x7fcfa80b1160 2024-04-20T11:29:25.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.859+0000 7fcfaf800640 1 -- 172.21.15.152:0/336620750 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfa81f8ab0 con 0x7fcfa80b1160 2024-04-20T11:29:25.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.860+0000 7fcfaf800640 1 -- 172.21.15.152:0/336620750 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfa80689f0 con 0x7fcfa80b1160 2024-04-20T11:29:25.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.861+0000 7fcfa4c00640 1 -- 172.21.15.152:0/336620750 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7fcfa001a070 con 0x7fcfa80b1160 2024-04-20T11:29:25.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.862+0000 7fcfa4c00640 1 --2- 172.21.15.152:0/336620750 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fcf8806cad0 0x7fcf8806ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:25.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.862+0000 7fcfa4c00640 1 -- 172.21.15.152:0/336620750 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7fcfa008ae10 con 0x7fcfa80b1160 2024-04-20T11:29:25.863 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.862+0000 7fcface00640 1 --2- 172.21.15.152:0/336620750 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fcf8806cad0 0x7fcf8806ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:25.863 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.863+0000 7fcface00640 1 --2- 172.21.15.152:0/336620750 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fcf8806cad0 0x7fcf8806ef90 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fcf9c034000 tx=0x7fcf9c009030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:25.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:25.866+0000 7fcfa4c00640 1 -- 172.21.15.152:0/336620750 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fcfa0058fc0 con 0x7fcfa80b1160 2024-04-20T11:29:26.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.051+0000 7fcfaf800640 1 -- 172.21.15.152:0/336620750 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fcfa8161a70 con 0x7fcfa80b1160 2024-04-20T11:29:26.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.052+0000 7fcfa4c00640 1 -- 172.21.15.152:0/336620750 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v46) ==== 104+0+11 (secure 0 0 0) 0x7fcfa00588c0 con 0x7fcfa80b1160 2024-04-20T11:29:26.053 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 32 2024-04-20T11:29:26.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.054+0000 7fcfaf800640 1 -- 172.21.15.152:0/336620750 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fcf8806cad0 msgr2=0x7fcf8806ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:26.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.054+0000 7fcfaf800640 1 --2- 172.21.15.152:0/336620750 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fcf8806cad0 0x7fcf8806ef90 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fcf9c034000 tx=0x7fcf9c009030 comp rx=0 tx=0).stop 2024-04-20T11:29:26.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.054+0000 7fcfaf800640 1 -- 172.21.15.152:0/336620750 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcfa80b1160 msgr2=0x7fcfa80b15e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:26.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.054+0000 7fcfaf800640 1 --2- 172.21.15.152:0/336620750 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcfa80b1160 0x7fcfa80b15e0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fcfa80b4460 tx=0x7fcfa000cb90 comp rx=0 tx=0).stop 2024-04-20T11:29:26.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.055+0000 7fcfaf800640 1 -- 172.21.15.152:0/336620750 shutdown_connections 2024-04-20T11:29:26.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.055+0000 7fcfaf800640 1 --2- 172.21.15.152:0/336620750 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fcf8806cad0 0x7fcf8806ef90 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:26.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.055+0000 7fcfaf800640 1 --2- 172.21.15.152:0/336620750 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcfa80b1160 0x7fcfa80b15e0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:26.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.055+0000 7fcfaf800640 1 --2- 172.21.15.152:0/336620750 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcfa8145300 0x7fcfa8161100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:26.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.055+0000 7fcfaf800640 1 --2- 172.21.15.152:0/336620750 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcf90003a00 0x7fcfa81560b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:26.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.055+0000 7fcfaf800640 1 -- 172.21.15.152:0/336620750 >> 172.21.15.152:0/336620750 conn(0x7fcfa806b980 msgr2=0x7fcfa815bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:26.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.055+0000 7fcfaf800640 1 -- 172.21.15.152:0/336620750 shutdown_connections 2024-04-20T11:29:26.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.055+0000 7fcfaf800640 1 -- 172.21.15.152:0/336620750 wait complete. 2024-04-20T11:29:26.072 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T11:29:26.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.272+0000 7f5a97e00640 1 Processor -- start 2024-04-20T11:29:26.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.272+0000 7f5a97e00640 1 -- start start 2024-04-20T11:29:26.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.274+0000 7f5a97e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f5a9015b510 0x7f5a9015fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:26.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.274+0000 7f5a97e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f5a900b2880 0x7f5a900b3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:26.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.274+0000 7f5a97e00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a9015f1c0 con 0x7f5a900cfc40 2024-04-20T11:29:26.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.274+0000 7f5a97e00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f5a9006f420 con 0x7f5a900b2880 2024-04-20T11:29:26.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.274+0000 7f5a97e00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f5a900b4430 con 0x7f5a9015b510 2024-04-20T11:29:26.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.274+0000 7f5a95e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f5a900b2880 0x7f5a900b3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:26.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.274+0000 7f5a95e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f5a900b2880 0x7f5a900b3ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35264/0 (socket says 172.21.15.152:35264) 2024-04-20T11:29:26.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.274+0000 7f5a95e00640 1 -- 172.21.15.152:0/4084587125 learned_addr learned my addr 172.21.15.152:0/4084587125 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:29:26.276 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.275+0000 7f5a95400640 1 --2- 172.21.15.152:0/4084587125 >> v2:172.21.15.175:3300/0 conn(0x7f5a9015b510 0x7f5a9015fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:26.276 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.275+0000 7f5a95e00640 1 -- 172.21.15.152:0/4084587125 >> v2:172.21.15.175:3300/0 conn(0x7f5a9015b510 msgr2=0x7f5a9015fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:26.276 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.275+0000 7f5a95e00640 1 --2- 172.21.15.152:0/4084587125 >> v2:172.21.15.175:3300/0 conn(0x7f5a9015b510 0x7f5a9015fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:26.276 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.275+0000 7f5a95e00640 1 -- 172.21.15.152:0/4084587125 >> v1:172.21.15.175:6789/0 conn(0x7f5a900cfc40 legacy=0x7f5a900b2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:26.276 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.275+0000 7f5a95e00640 1 -- 172.21.15.152:0/4084587125 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a9006f670 con 0x7f5a900b2880 2024-04-20T11:29:26.277 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.276+0000 7f5a95e00640 1 --2- 172.21.15.152:0/4084587125 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5a900b2880 0x7f5a900b3ef0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f5a9006c520 tx=0x7f5a8802be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee8053e19c086a76 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:26.277 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.276+0000 7f5a8fe00640 1 -- 172.21.15.152:0/4084587125 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a8802c9f0 con 0x7f5a900b2880 2024-04-20T11:29:26.277 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.276+0000 7f5a8fe00640 1 -- 172.21.15.152:0/4084587125 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a8802cb90 con 0x7f5a900b2880 2024-04-20T11:29:26.277 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.276+0000 7f5a8fe00640 1 -- 172.21.15.152:0/4084587125 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a880343f0 con 0x7f5a900b2880 2024-04-20T11:29:26.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.277+0000 7f5a97e00640 1 -- 172.21.15.152:0/4084587125 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5a900b2880 msgr2=0x7f5a900b3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:26.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.277+0000 7f5a97e00640 1 --2- 172.21.15.152:0/4084587125 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5a900b2880 0x7f5a900b3ef0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f5a9006c520 tx=0x7f5a8802be50 comp rx=0 tx=0).stop 2024-04-20T11:29:26.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.277+0000 7f5a97e00640 1 -- 172.21.15.152:0/4084587125 shutdown_connections 2024-04-20T11:29:26.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.277+0000 7f5a97e00640 1 --2- 172.21.15.152:0/4084587125 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5a900b2880 0x7f5a900b3ef0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:26.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.277+0000 7f5a97e00640 1 --2- 172.21.15.152:0/4084587125 >> v2:172.21.15.175:3300/0 conn(0x7f5a9015b510 0x7f5a9015fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:26.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.277+0000 7f5a97e00640 1 -- 172.21.15.152:0/4084587125 >> 172.21.15.152:0/4084587125 conn(0x7f5a9006b980 msgr2=0x7f5a9006bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:26.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.277+0000 7f5a97e00640 1 -- 172.21.15.152:0/4084587125 shutdown_connections 2024-04-20T11:29:26.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.277+0000 7f5a97e00640 1 -- 172.21.15.152:0/4084587125 wait complete. 2024-04-20T11:29:26.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.279+0000 7f5a97e00640 1 Processor -- start 2024-04-20T11:29:26.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.279+0000 7f5a97e00640 1 -- start start 2024-04-20T11:29:26.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.280+0000 7f5a97e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5a900b2880 0x7f5a900b0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:26.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.280+0000 7f5a97e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5a900cfc40 0x7f5a90143660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:26.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.280+0000 7f5a97e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5a9015b510 0x7f5a90144e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:26.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.280+0000 7f5a97e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f5a9015b8f0 con 0x7f5a9015b510 2024-04-20T11:29:26.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.280+0000 7f5a97e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5a9015f710 con 0x7f5a900cfc40 2024-04-20T11:29:26.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.280+0000 7f5a97e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5a9006f950 con 0x7f5a900b2880 2024-04-20T11:29:26.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.280+0000 7f5a95e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5a9015b510 0x7f5a90144e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:26.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.280+0000 7f5a95e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5a9015b510 0x7f5a90144e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35276/0 (socket says 172.21.15.152:35276) 2024-04-20T11:29:26.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.280+0000 7f5a95e00640 1 -- 172.21.15.152:0/708824660 learned_addr learned my addr 172.21.15.152:0/708824660 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:29:26.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.280+0000 7f5a95e00640 1 -- 172.21.15.152:0/708824660 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5a900b2880 msgr2=0x7f5a900b0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:26.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.280+0000 7f5a95e00640 1 --2- 172.21.15.152:0/708824660 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5a900b2880 0x7f5a900b0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:26.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.280+0000 7f5a95e00640 1 -- 172.21.15.152:0/708824660 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5a900cfc40 msgr2=0x7f5a90143660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:26.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.280+0000 7f5a94a00640 1 --2- 172.21.15.152:0/708824660 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5a900cfc40 0x7f5a90143660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:26.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.280+0000 7f5a95e00640 1 --2- 172.21.15.152:0/708824660 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5a900cfc40 0x7f5a90143660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:26.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.280+0000 7f5a95e00640 1 -- 172.21.15.152:0/708824660 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a90143d90 con 0x7f5a9015b510 2024-04-20T11:29:26.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.281+0000 7f5a95e00640 1 --2- 172.21.15.152:0/708824660 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5a9015b510 0x7f5a90144e30 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f5a88009f40 tx=0x7f5a88004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:26.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.281+0000 7f5a8d600640 1 -- 172.21.15.152:0/708824660 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a88043020 con 0x7f5a9015b510 2024-04-20T11:29:26.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.281+0000 7f5a8d600640 1 -- 172.21.15.152:0/708824660 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a8803e450 con 0x7f5a9015b510 2024-04-20T11:29:26.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.281+0000 7f5a8d600640 1 -- 172.21.15.152:0/708824660 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a8803e820 con 0x7f5a9015b510 2024-04-20T11:29:26.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.281+0000 7f5a97e00640 1 -- 172.21.15.152:0/708824660 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a90144070 con 0x7f5a9015b510 2024-04-20T11:29:26.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.281+0000 7f5a97e00640 1 -- 172.21.15.152:0/708824660 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a90144630 con 0x7f5a9015b510 2024-04-20T11:29:26.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.282+0000 7f5a97e00640 1 -- 172.21.15.152:0/708824660 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a900689f0 con 0x7f5a9015b510 2024-04-20T11:29:26.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.283+0000 7f5a8d600640 1 -- 172.21.15.152:0/708824660 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f5a88046020 con 0x7f5a9015b510 2024-04-20T11:29:26.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.284+0000 7f5a8d600640 1 --2- 172.21.15.152:0/708824660 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f5a7006caa0 0x7f5a7006ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:26.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.284+0000 7f5a8d600640 1 -- 172.21.15.152:0/708824660 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f5a880ae860 con 0x7f5a9015b510 2024-04-20T11:29:26.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.284+0000 7f5a95400640 1 --2- 172.21.15.152:0/708824660 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f5a7006caa0 0x7f5a7006ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:26.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.285+0000 7f5a95400640 1 --2- 172.21.15.152:0/708824660 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f5a7006caa0 0x7f5a7006ef60 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f5a8400a810 tx=0x7f5a84008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:26.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.288+0000 7f5a8d600640 1 -- 172.21.15.152:0/708824660 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f5a8804d030 con 0x7f5a9015b510 2024-04-20T11:29:26.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.478+0000 7f5a97e00640 1 -- 172.21.15.152:0/708824660 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f5a9006ecf0 con 0x7f5a9015b510 2024-04-20T11:29:26.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.478+0000 7f5a8d600640 1 -- 172.21.15.152:0/708824660 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v46) ==== 100+0+11 (secure 0 0 0) 0x7f5a8807c970 con 0x7f5a9015b510 2024-04-20T11:29:26.479 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 64 2024-04-20T11:29:26.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.481+0000 7f5a97e00640 1 -- 172.21.15.152:0/708824660 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f5a7006caa0 msgr2=0x7f5a7006ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:26.482 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.481+0000 7f5a97e00640 1 --2- 172.21.15.152:0/708824660 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f5a7006caa0 0x7f5a7006ef60 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f5a8400a810 tx=0x7f5a84008040 comp rx=0 tx=0).stop 2024-04-20T11:29:26.482 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.481+0000 7f5a97e00640 1 -- 172.21.15.152:0/708824660 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5a9015b510 msgr2=0x7f5a90144e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:26.482 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.481+0000 7f5a97e00640 1 --2- 172.21.15.152:0/708824660 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5a9015b510 0x7f5a90144e30 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f5a88009f40 tx=0x7f5a88004a40 comp rx=0 tx=0).stop 2024-04-20T11:29:26.482 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.481+0000 7f5a97e00640 1 -- 172.21.15.152:0/708824660 shutdown_connections 2024-04-20T11:29:26.482 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.481+0000 7f5a97e00640 1 --2- 172.21.15.152:0/708824660 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f5a7006caa0 0x7f5a7006ef60 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:26.482 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.481+0000 7f5a97e00640 1 --2- 172.21.15.152:0/708824660 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5a9015b510 0x7f5a90144e30 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:26.482 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.481+0000 7f5a97e00640 1 --2- 172.21.15.152:0/708824660 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5a900cfc40 0x7f5a90143660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:26.482 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.481+0000 7f5a97e00640 1 --2- 172.21.15.152:0/708824660 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5a900b2880 0x7f5a900b0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:26.482 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.481+0000 7f5a97e00640 1 -- 172.21.15.152:0/708824660 >> 172.21.15.152:0/708824660 conn(0x7f5a9006b980 msgr2=0x7f5a900d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:26.482 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.481+0000 7f5a97e00640 1 -- 172.21.15.152:0/708824660 shutdown_connections 2024-04-20T11:29:26.482 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.481+0000 7f5a97e00640 1 -- 172.21.15.152:0/708824660 wait complete. 2024-04-20T11:29:26.499 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-20T11:29:26.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.693+0000 7fbda2000640 1 Processor -- start 2024-04-20T11:29:26.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.693+0000 7fbda2000640 1 -- start start 2024-04-20T11:29:26.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.695+0000 7fbda2000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fbd9c06fa50 0x7fbd9c0b3470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:26.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.695+0000 7fbda2000640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fbd9c15fd60 0x7fbd9c0b39b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:26.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.695+0000 7fbda2000640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd9c0b4130 con 0x7fbd9c15b2b0 2024-04-20T11:29:26.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.695+0000 7fbda2000640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fbd9c0b3ef0 con 0x7fbd9c06fa50 2024-04-20T11:29:26.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.695+0000 7fbda2000640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fbd9c15f8d0 con 0x7fbd9c15fd60 2024-04-20T11:29:26.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.695+0000 7fbd9aa00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fbd9c06fa50 0x7fbd9c0b3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:26.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.695+0000 7fbd9aa00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fbd9c06fa50 0x7fbd9c0b3470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35290/0 (socket says 172.21.15.152:35290) 2024-04-20T11:29:26.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.695+0000 7fbd9aa00640 1 -- 172.21.15.152:0/2545352473 learned_addr learned my addr 172.21.15.152:0/2545352473 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:29:26.697 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.696+0000 7fbd9be00640 1 --2- 172.21.15.152:0/2545352473 >> v2:172.21.15.175:3300/0 conn(0x7fbd9c15fd60 0x7fbd9c0b39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:26.697 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.696+0000 7fbd9aa00640 1 -- 172.21.15.152:0/2545352473 >> v2:172.21.15.175:3300/0 conn(0x7fbd9c15fd60 msgr2=0x7fbd9c0b39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:26.697 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.696+0000 7fbd9aa00640 1 --2- 172.21.15.152:0/2545352473 >> v2:172.21.15.175:3300/0 conn(0x7fbd9c15fd60 0x7fbd9c0b39b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:26.697 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.696+0000 7fbd9aa00640 1 -- 172.21.15.152:0/2545352473 >> v1:172.21.15.175:6789/0 conn(0x7fbd9c15b2b0 legacy=0x7fbd9c06f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:26.697 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.696+0000 7fbd9aa00640 1 -- 172.21.15.152:0/2545352473 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd9c15f280 con 0x7fbd9c06fa50 2024-04-20T11:29:26.698 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.697+0000 7fbd9aa00640 1 --2- 172.21.15.152:0/2545352473 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd9c06fa50 0x7fbd9c0b3470 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c0ccef0 tx=0x7fbd8802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3901c5ff62d5b821 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:26.698 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.697+0000 7fbd9a000640 1 -- 172.21.15.152:0/2545352473 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd88035020 con 0x7fbd9c06fa50 2024-04-20T11:29:26.698 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.697+0000 7fbd9a000640 1 -- 172.21.15.152:0/2545352473 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd88031870 con 0x7fbd9c06fa50 2024-04-20T11:29:26.698 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.697+0000 7fbd9a000640 1 -- 172.21.15.152:0/2545352473 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd88031c40 con 0x7fbd9c06fa50 2024-04-20T11:29:26.699 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.698+0000 7fbda2000640 1 -- 172.21.15.152:0/2545352473 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd9c06fa50 msgr2=0x7fbd9c0b3470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:26.699 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.698+0000 7fbda2000640 1 --2- 172.21.15.152:0/2545352473 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd9c06fa50 0x7fbd9c0b3470 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c0ccef0 tx=0x7fbd8802b910 comp rx=0 tx=0).stop 2024-04-20T11:29:26.699 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.698+0000 7fbda2000640 1 -- 172.21.15.152:0/2545352473 shutdown_connections 2024-04-20T11:29:26.699 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.698+0000 7fbda2000640 1 --2- 172.21.15.152:0/2545352473 >> v2:172.21.15.175:3300/0 conn(0x7fbd9c15fd60 0x7fbd9c0b39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:26.699 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.698+0000 7fbda2000640 1 --2- 172.21.15.152:0/2545352473 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd9c06fa50 0x7fbd9c0b3470 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:26.699 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.698+0000 7fbda2000640 1 -- 172.21.15.152:0/2545352473 >> 172.21.15.152:0/2545352473 conn(0x7fbd9c067cf0 msgr2=0x7fbd9c06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:26.699 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.698+0000 7fbda2000640 1 -- 172.21.15.152:0/2545352473 shutdown_connections 2024-04-20T11:29:26.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.698+0000 7fbda2000640 1 -- 172.21.15.152:0/2545352473 wait complete. 2024-04-20T11:29:26.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.700+0000 7fbda2000640 1 Processor -- start 2024-04-20T11:29:26.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.701+0000 7fbda2000640 1 -- start start 2024-04-20T11:29:26.704 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.702+0000 7fbda2000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd9c06fa50 0x7fbd9c1540e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:26.704 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.702+0000 7fbda2000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd9c15b2b0 0x7fbd9c158d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:26.704 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.702+0000 7fbda2000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd9c15fd60 0x7fbd9c1e6b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:26.704 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.702+0000 7fbda2000640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fbd9c15b810 con 0x7fbd9c15b2b0 2024-04-20T11:29:26.704 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.702+0000 7fbda2000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbd9c15b690 con 0x7fbd9c06fa50 2024-04-20T11:29:26.704 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.702+0000 7fbda2000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fbd9c15b990 con 0x7fbd9c15fd60 2024-04-20T11:29:26.704 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.702+0000 7fbd9aa00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd9c15b2b0 0x7fbd9c158d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:26.704 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.702+0000 7fbd9aa00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd9c15b2b0 0x7fbd9c158d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35302/0 (socket says 172.21.15.152:35302) 2024-04-20T11:29:26.704 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.702+0000 7fbd9aa00640 1 -- 172.21.15.152:0/2516533778 learned_addr learned my addr 172.21.15.152:0/2516533778 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:29:26.704 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.702+0000 7fbd9b400640 1 --2- 172.21.15.152:0/2516533778 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd9c06fa50 0x7fbd9c1540e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:26.705 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.702+0000 7fbd9be00640 1 --2- 172.21.15.152:0/2516533778 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd9c15fd60 0x7fbd9c1e6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:26.705 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.703+0000 7fbd9aa00640 1 -- 172.21.15.152:0/2516533778 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd9c15fd60 msgr2=0x7fbd9c1e6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:26.705 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.703+0000 7fbd9aa00640 1 --2- 172.21.15.152:0/2516533778 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd9c15fd60 0x7fbd9c1e6b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:26.705 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.703+0000 7fbd9aa00640 1 -- 172.21.15.152:0/2516533778 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd9c06fa50 msgr2=0x7fbd9c1540e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:26.705 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.703+0000 7fbd9aa00640 1 --2- 172.21.15.152:0/2516533778 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd9c06fa50 0x7fbd9c1540e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:26.705 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.703+0000 7fbd9aa00640 1 -- 172.21.15.152:0/2516533778 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd9c157cc0 con 0x7fbd9c15b2b0 2024-04-20T11:29:26.705 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.703+0000 7fbd9aa00640 1 --2- 172.21.15.152:0/2516533778 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd9c15b2b0 0x7fbd9c158d60 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fbd8802b950 tx=0x7fbd880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:26.705 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.703+0000 7fbd93400640 1 -- 172.21.15.152:0/2516533778 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd88040070 con 0x7fbd9c15b2b0 2024-04-20T11:29:26.705 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.703+0000 7fbd93400640 1 -- 172.21.15.152:0/2516533778 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd8803a420 con 0x7fbd9c15b2b0 2024-04-20T11:29:26.705 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.703+0000 7fbd93400640 1 -- 172.21.15.152:0/2516533778 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd88035040 con 0x7fbd9c15b2b0 2024-04-20T11:29:26.705 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.703+0000 7fbda2000640 1 -- 172.21.15.152:0/2516533778 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd9c157fa0 con 0x7fbd9c15b2b0 2024-04-20T11:29:26.705 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.704+0000 7fbda2000640 1 -- 172.21.15.152:0/2516533778 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd9c158560 con 0x7fbd9c15b2b0 2024-04-20T11:29:26.706 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.705+0000 7fbda2000640 1 -- 172.21.15.152:0/2516533778 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd9c0689f0 con 0x7fbd9c15b2b0 2024-04-20T11:29:26.706 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.706+0000 7fbd93400640 1 -- 172.21.15.152:0/2516533778 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7fbd88038020 con 0x7fbd9c15b2b0 2024-04-20T11:29:26.707 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.706+0000 7fbd93400640 1 --2- 172.21.15.152:0/2516533778 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fbd7c06caa0 0x7fbd7c06ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:26.707 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.707+0000 7fbd93400640 1 -- 172.21.15.152:0/2516533778 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7fbd880adf50 con 0x7fbd9c15b2b0 2024-04-20T11:29:26.707 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.707+0000 7fbd9b400640 1 --2- 172.21.15.152:0/2516533778 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fbd7c06caa0 0x7fbd7c06ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:26.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.708+0000 7fbd9b400640 1 --2- 172.21.15.152:0/2516533778 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fbd7c06caa0 0x7fbd7c06ef60 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c1549d0 tx=0x7fbd8c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:26.711 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.711+0000 7fbd93400640 1 -- 172.21.15.152:0/2516533778 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fbd8807c060 con 0x7fbd9c15b2b0 2024-04-20T11:29:26.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.911+0000 7fbda2000640 1 -- 172.21.15.152:0/2516533778 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fbd9c06eef0 con 0x7fbd9c15b2b0 2024-04-20T11:29:26.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.912+0000 7fbd93400640 1 -- 172.21.15.152:0/2516533778 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v46) ==== 80+0+0 (secure 0 0 0) 0x7fbd8807b960 con 0x7fbd9c15b2b0 2024-04-20T11:29:26.913 INFO:teuthology.orchestra.run.smithi152.stderr:listed 0 entries 2024-04-20T11:29:26.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.915+0000 7fbda2000640 1 -- 172.21.15.152:0/2516533778 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fbd7c06caa0 msgr2=0x7fbd7c06ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:26.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.915+0000 7fbda2000640 1 --2- 172.21.15.152:0/2516533778 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fbd7c06caa0 0x7fbd7c06ef60 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c1549d0 tx=0x7fbd8c009040 comp rx=0 tx=0).stop 2024-04-20T11:29:26.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.915+0000 7fbda2000640 1 -- 172.21.15.152:0/2516533778 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd9c15b2b0 msgr2=0x7fbd9c158d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:26.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.915+0000 7fbda2000640 1 --2- 172.21.15.152:0/2516533778 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd9c15b2b0 0x7fbd9c158d60 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fbd8802b950 tx=0x7fbd880046c0 comp rx=0 tx=0).stop 2024-04-20T11:29:26.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.915+0000 7fbda2000640 1 -- 172.21.15.152:0/2516533778 shutdown_connections 2024-04-20T11:29:26.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.915+0000 7fbda2000640 1 --2- 172.21.15.152:0/2516533778 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fbd7c06caa0 0x7fbd7c06ef60 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:26.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.915+0000 7fbda2000640 1 --2- 172.21.15.152:0/2516533778 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd9c15fd60 0x7fbd9c1e6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:26.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.915+0000 7fbda2000640 1 --2- 172.21.15.152:0/2516533778 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd9c15b2b0 0x7fbd9c158d60 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:26.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.915+0000 7fbda2000640 1 --2- 172.21.15.152:0/2516533778 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd9c06fa50 0x7fbd9c1540e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:26.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.915+0000 7fbda2000640 1 -- 172.21.15.152:0/2516533778 >> 172.21.15.152:0/2516533778 conn(0x7fbd9c067cf0 msgr2=0x7fbd9c0b3240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:26.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.915+0000 7fbda2000640 1 -- 172.21.15.152:0/2516533778 shutdown_connections 2024-04-20T11:29:26.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:26.915+0000 7fbda2000640 1 -- 172.21.15.152:0/2516533778 wait complete. 2024-04-20T11:29:26.935 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T11:29:26.935 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:29:26.992 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:29:27.193 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.193+0000 7fdd2b800640 1 Processor -- start 2024-04-20T11:29:27.194 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.193+0000 7fdd2b800640 1 -- start start 2024-04-20T11:29:27.195 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.194+0000 7fdd2b800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fdd2406eb40 0x7fdd2406f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:27.195 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.194+0000 7fdd2b800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd2406f880 0x7fdd2415cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:27.195 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.194+0000 7fdd2b800640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd2406fca0 con 0x7fdd2415b5e0 2024-04-20T11:29:27.195 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.194+0000 7fdd2b800640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fdd2406de10 con 0x7fdd2406eb40 2024-04-20T11:29:27.195 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.194+0000 7fdd2b800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fdd240b5770 con 0x7fdd2406f880 2024-04-20T11:29:27.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.195+0000 7fdd23e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fdd2406eb40 0x7fdd2406f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:27.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.195+0000 7fdd28e00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fdd2415b5e0 0x7fdd240b4c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:42464/0 (socket says 172.21.15.152:42464) 2024-04-20T11:29:27.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.195+0000 7fdd28e00640 1 -- 172.21.15.152:0/139032652 learned_addr learned my addr 172.21.15.152:0/139032652 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:29:27.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.195+0000 7fdd23e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fdd2406eb40 0x7fdd2406f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35310/0 (socket says 172.21.15.152:35310) 2024-04-20T11:29:27.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.195+0000 7fdd23e00640 1 -- 172.21.15.152:0/139032652 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd2406f880 msgr2=0x7fdd2415cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:27.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.195+0000 7fdd23e00640 1 --2- 172.21.15.152:0/139032652 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd2406f880 0x7fdd2415cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:27.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.195+0000 7fdd23e00640 1 -- 172.21.15.152:0/139032652 >> v1:172.21.15.152:6789/0 conn(0x7fdd2415b5e0 legacy=0x7fdd240b4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:27.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.195+0000 7fdd23e00640 1 -- 172.21.15.152:0/139032652 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd2406e060 con 0x7fdd2406eb40 2024-04-20T11:29:27.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.196+0000 7fdd23e00640 1 --2- 172.21.15.152:0/139032652 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdd2406eb40 0x7fdd2406f340 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fdd100097a0 tx=0x7fdd1002dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e88c7b869bd5490d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:27.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.197+0000 7fdd23400640 1 -- 172.21.15.152:0/139032652 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd1002f8a0 con 0x7fdd2406eb40 2024-04-20T11:29:27.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.197+0000 7fdd23400640 1 -- 172.21.15.152:0/139032652 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd1002fa40 con 0x7fdd2406eb40 2024-04-20T11:29:27.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.197+0000 7fdd23400640 1 -- 172.21.15.152:0/139032652 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd1002fe10 con 0x7fdd2406eb40 2024-04-20T11:29:27.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.197+0000 7fdd2b800640 1 -- 172.21.15.152:0/139032652 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdd2406eb40 msgr2=0x7fdd2406f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:27.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.197+0000 7fdd2b800640 1 --2- 172.21.15.152:0/139032652 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdd2406eb40 0x7fdd2406f340 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fdd100097a0 tx=0x7fdd1002dae0 comp rx=0 tx=0).stop 2024-04-20T11:29:27.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.197+0000 7fdd2b800640 1 -- 172.21.15.152:0/139032652 shutdown_connections 2024-04-20T11:29:27.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.197+0000 7fdd2b800640 1 --2- 172.21.15.152:0/139032652 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd2406f880 0x7fdd2415cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:27.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.197+0000 7fdd2b800640 1 --2- 172.21.15.152:0/139032652 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdd2406eb40 0x7fdd2406f340 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:27.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.197+0000 7fdd2b800640 1 -- 172.21.15.152:0/139032652 >> 172.21.15.152:0/139032652 conn(0x7fdd24067cf0 msgr2=0x7fdd2406b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:27.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.197+0000 7fdd2b800640 1 -- 172.21.15.152:0/139032652 shutdown_connections 2024-04-20T11:29:27.199 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.198+0000 7fdd2b800640 1 -- 172.21.15.152:0/139032652 wait complete. 2024-04-20T11:29:27.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.199+0000 7fdd2b800640 1 Processor -- start 2024-04-20T11:29:27.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.199+0000 7fdd2b800640 1 -- start start 2024-04-20T11:29:27.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.200+0000 7fdd2b800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd2406eb40 0x7fdd240cfc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:27.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.200+0000 7fdd2b800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdd2406f880 0x7fdd24146de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:27.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.200+0000 7fdd2b800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd2415b5e0 0x7fdd240ae900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:27.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.200+0000 7fdd2b800640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fdd2415c9d0 con 0x7fdd2406f880 2024-04-20T11:29:27.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.200+0000 7fdd2b800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdd2406e620 con 0x7fdd2415b5e0 2024-04-20T11:29:27.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.200+0000 7fdd2b800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fdd240b42d0 con 0x7fdd2406eb40 2024-04-20T11:29:27.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.201+0000 7fdd23e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdd2406f880 0x7fdd24146de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:27.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.201+0000 7fdd23e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdd2406f880 0x7fdd24146de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35314/0 (socket says 172.21.15.152:35314) 2024-04-20T11:29:27.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.201+0000 7fdd23e00640 1 -- 172.21.15.152:0/497390337 learned_addr learned my addr 172.21.15.152:0/497390337 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:29:27.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.201+0000 7fdd28e00640 1 --2- 172.21.15.152:0/497390337 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd2406eb40 0x7fdd240cfc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:27.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.201+0000 7fdd29800640 1 --2- 172.21.15.152:0/497390337 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd2415b5e0 0x7fdd240ae900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:27.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.201+0000 7fdd23e00640 1 -- 172.21.15.152:0/497390337 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd2406eb40 msgr2=0x7fdd240cfc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:27.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.201+0000 7fdd23e00640 1 --2- 172.21.15.152:0/497390337 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd2406eb40 0x7fdd240cfc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:27.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.201+0000 7fdd23e00640 1 -- 172.21.15.152:0/497390337 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd2415b5e0 msgr2=0x7fdd240ae900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:27.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.201+0000 7fdd23e00640 1 --2- 172.21.15.152:0/497390337 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd2415b5e0 0x7fdd240ae900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:27.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.201+0000 7fdd23e00640 1 -- 172.21.15.152:0/497390337 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd24155ca0 con 0x7fdd2406f880 2024-04-20T11:29:27.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.201+0000 7fdd29800640 1 --2- 172.21.15.152:0/497390337 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd2415b5e0 0x7fdd240ae900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:29:27.204 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.202+0000 7fdd23e00640 1 --2- 172.21.15.152:0/497390337 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdd2406f880 0x7fdd24146de0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fdd100098c0 tx=0x7fdd100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:27.204 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.202+0000 7fdd20c00640 1 -- 172.21.15.152:0/497390337 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd10047070 con 0x7fdd2406f880 2024-04-20T11:29:27.204 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.202+0000 7fdd20c00640 1 -- 172.21.15.152:0/497390337 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd10030c40 con 0x7fdd2406f880 2024-04-20T11:29:27.204 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.202+0000 7fdd20c00640 1 -- 172.21.15.152:0/497390337 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd10042410 con 0x7fdd2406f880 2024-04-20T11:29:27.204 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.202+0000 7fdd2b800640 1 -- 172.21.15.152:0/497390337 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd2415e0f0 con 0x7fdd2406f880 2024-04-20T11:29:27.204 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.202+0000 7fdd2b800640 1 -- 172.21.15.152:0/497390337 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd2415e6b0 con 0x7fdd2406f880 2024-04-20T11:29:27.204 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.203+0000 7fdd2b800640 1 -- 172.21.15.152:0/497390337 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd240689f0 con 0x7fdd2406f880 2024-04-20T11:29:27.205 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.204+0000 7fdd20c00640 1 -- 172.21.15.152:0/497390337 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7fdd10039400 con 0x7fdd2406f880 2024-04-20T11:29:27.206 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.205+0000 7fdd20c00640 1 --2- 172.21.15.152:0/497390337 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fdd0406caa0 0x7fdd0406ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:27.206 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.205+0000 7fdd20c00640 1 -- 172.21.15.152:0/497390337 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7fdd100ada20 con 0x7fdd2406f880 2024-04-20T11:29:27.206 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.205+0000 7fdd28e00640 1 --2- 172.21.15.152:0/497390337 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fdd0406caa0 0x7fdd0406ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:27.206 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.206+0000 7fdd28e00640 1 --2- 172.21.15.152:0/497390337 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fdd0406caa0 0x7fdd0406ef60 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fdd18000f60 tx=0x7fdd18009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:27.210 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.209+0000 7fdd20c00640 1 -- 172.21.15.152:0/497390337 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fdd10035030 con 0x7fdd2406f880 2024-04-20T11:29:27.412 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.411+0000 7fdd2b800640 1 -- 172.21.15.152:0/497390337 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdd2415ac00 con 0x7fdd2406f880 2024-04-20T11:29:27.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.413+0000 7fdd20c00640 1 -- 172.21.15.152:0/497390337 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) ==== 74+0+16673 (secure 0 0 0) 0x7fdd1007ba20 con 0x7fdd2406f880 2024-04-20T11:29:27.414 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:29:27.414 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":46,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:29:01.812130+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:28:58.007718+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-20T11:28:59.260647+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:29:27.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.415+0000 7fdd2b800640 1 -- 172.21.15.152:0/497390337 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fdd0406caa0 msgr2=0x7fdd0406ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:27.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.415+0000 7fdd2b800640 1 --2- 172.21.15.152:0/497390337 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fdd0406caa0 0x7fdd0406ef60 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fdd18000f60 tx=0x7fdd18009040 comp rx=0 tx=0).stop 2024-04-20T11:29:27.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.415+0000 7fdd2b800640 1 -- 172.21.15.152:0/497390337 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdd2406f880 msgr2=0x7fdd24146de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:27.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.415+0000 7fdd2b800640 1 --2- 172.21.15.152:0/497390337 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdd2406f880 0x7fdd24146de0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fdd100098c0 tx=0x7fdd100046c0 comp rx=0 tx=0).stop 2024-04-20T11:29:27.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.416+0000 7fdd2b800640 1 -- 172.21.15.152:0/497390337 shutdown_connections 2024-04-20T11:29:27.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.416+0000 7fdd2b800640 1 --2- 172.21.15.152:0/497390337 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fdd0406caa0 0x7fdd0406ef60 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:27.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.416+0000 7fdd2b800640 1 --2- 172.21.15.152:0/497390337 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd2415b5e0 0x7fdd240ae900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:27.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.416+0000 7fdd2b800640 1 --2- 172.21.15.152:0/497390337 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdd2406f880 0x7fdd24146de0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:27.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.416+0000 7fdd2b800640 1 --2- 172.21.15.152:0/497390337 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd2406eb40 0x7fdd240cfc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:27.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.416+0000 7fdd2b800640 1 -- 172.21.15.152:0/497390337 >> 172.21.15.152:0/497390337 conn(0x7fdd24067cf0 msgr2=0x7fdd240d0520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:27.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.416+0000 7fdd2b800640 1 -- 172.21.15.152:0/497390337 shutdown_connections 2024-04-20T11:29:27.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.416+0000 7fdd2b800640 1 -- 172.21.15.152:0/497390337 wait complete. 2024-04-20T11:29:27.436 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T11:26:47.246571+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T11:28:58.007718+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T11:28:59.260647+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T11:29:27.436 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T11:29:27.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.643+0000 7f30a6400640 1 Processor -- start 2024-04-20T11:29:27.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.643+0000 7f30a6400640 1 -- start start 2024-04-20T11:29:27.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.645+0000 7f30a6400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f30a00cd8c0 0x7f30a00b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:27.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.645+0000 7f30a6400640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30a01463e0 con 0x7f30a0146000 2024-04-20T11:29:27.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.645+0000 7f30a6400640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30a00b3e20 con 0x7f30a00b2880 2024-04-20T11:29:27.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.645+0000 7f30a6400640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f30a00b4750 con 0x7f30a00cd8c0 2024-04-20T11:29:27.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.645+0000 7f309fe00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f30a00cd8c0 0x7f30a00b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:27.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.645+0000 7f309fe00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f30a00cd8c0 0x7f30a00b4be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35320/0 (socket says 172.21.15.152:35320) 2024-04-20T11:29:27.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.645+0000 7f309f400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f30a0146000 0x7f30a00b2170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:42466/0 (socket says 172.21.15.152:42466) 2024-04-20T11:29:27.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.645+0000 7f309fe00640 1 -- 172.21.15.152:0/1626426687 learned_addr learned my addr 172.21.15.152:0/1626426687 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:29:27.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.646+0000 7f309fe00640 1 -- 172.21.15.152:0/1626426687 >> v1:172.21.15.175:6789/0 conn(0x7f30a00b2880 legacy=0x7f30a00cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:27.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.646+0000 7f309fe00640 1 -- 172.21.15.152:0/1626426687 >> v1:172.21.15.152:6789/0 conn(0x7f30a0146000 legacy=0x7f30a00b2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:27.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.646+0000 7f309fe00640 1 -- 172.21.15.152:0/1626426687 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30a00b4100 con 0x7f30a00cd8c0 2024-04-20T11:29:27.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.646+0000 7f309ea00640 1 -- 172.21.15.152:0/1626426687 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3880030382 0 0) 0x7f30a01463e0 con 0x7f30a0146000 2024-04-20T11:29:27.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.647+0000 7f309fe00640 1 --2- 172.21.15.152:0/1626426687 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f30a00cd8c0 0x7f30a00b4be0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f3098009e50 tx=0x7f309802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf9d689270060396 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:27.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.647+0000 7f309ea00640 1 -- 172.21.15.152:0/1626426687 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f309802ca80 con 0x7f30a00cd8c0 2024-04-20T11:29:27.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.647+0000 7f309ea00640 1 -- 172.21.15.152:0/1626426687 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f309802cc20 con 0x7f30a00cd8c0 2024-04-20T11:29:27.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.647+0000 7f309ea00640 1 -- 172.21.15.152:0/1626426687 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30980343f0 con 0x7f30a00cd8c0 2024-04-20T11:29:27.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.648+0000 7f30a6400640 1 -- 172.21.15.152:0/1626426687 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f30a00cd8c0 msgr2=0x7f30a00b4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:27.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.648+0000 7f30a6400640 1 --2- 172.21.15.152:0/1626426687 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f30a00cd8c0 0x7f30a00b4be0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f3098009e50 tx=0x7f309802bee0 comp rx=0 tx=0).stop 2024-04-20T11:29:27.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.648+0000 7f30a6400640 1 -- 172.21.15.152:0/1626426687 shutdown_connections 2024-04-20T11:29:27.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.648+0000 7f30a6400640 1 --2- 172.21.15.152:0/1626426687 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f30a00cd8c0 0x7f30a00b4be0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:27.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.648+0000 7f30a6400640 1 -- 172.21.15.152:0/1626426687 >> 172.21.15.152:0/1626426687 conn(0x7f30a006b980 msgr2=0x7f30a006bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:27.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.648+0000 7f30a6400640 1 -- 172.21.15.152:0/1626426687 shutdown_connections 2024-04-20T11:29:27.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.648+0000 7f30a6400640 1 -- 172.21.15.152:0/1626426687 wait complete. 2024-04-20T11:29:27.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.650+0000 7f30a6400640 1 Processor -- start 2024-04-20T11:29:27.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.650+0000 7f30a6400640 1 -- start start 2024-04-20T11:29:27.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.651+0000 7f30a6400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f30a00b2880 0x7f30a013e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:27.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.651+0000 7f30a6400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f30a00cd8c0 0x7f30a00aedc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:27.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.651+0000 7f30a6400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f30a0146000 0x7f30a00af300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:27.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.651+0000 7f30a6400640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f30a015ce70 con 0x7f30a0146000 2024-04-20T11:29:27.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.651+0000 7f30a6400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f30a015ccf0 con 0x7f30a00cd8c0 2024-04-20T11:29:27.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.651+0000 7f30a6400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f30a015cff0 con 0x7f30a00b2880 2024-04-20T11:29:27.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.651+0000 7f309fe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f30a0146000 0x7f30a00af300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:27.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.651+0000 7f309fe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f30a0146000 0x7f30a00af300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35330/0 (socket says 172.21.15.152:35330) 2024-04-20T11:29:27.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.651+0000 7f309fe00640 1 -- 172.21.15.152:0/598634588 learned_addr learned my addr 172.21.15.152:0/598634588 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:29:27.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.652+0000 7f309f400640 1 --2- 172.21.15.152:0/598634588 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f30a00b2880 0x7f30a013e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:27.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.652+0000 7f3096a00640 1 --2- 172.21.15.152:0/598634588 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f30a00cd8c0 0x7f30a00aedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:27.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.652+0000 7f309fe00640 1 -- 172.21.15.152:0/598634588 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f30a00b2880 msgr2=0x7f30a013e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:27.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.652+0000 7f309fe00640 1 --2- 172.21.15.152:0/598634588 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f30a00b2880 0x7f30a013e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:27.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.652+0000 7f309fe00640 1 -- 172.21.15.152:0/598634588 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f30a00cd8c0 msgr2=0x7f30a00aedc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:27.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.652+0000 7f309fe00640 1 --2- 172.21.15.152:0/598634588 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f30a00cd8c0 0x7f30a00aedc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:27.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.652+0000 7f309fe00640 1 -- 172.21.15.152:0/598634588 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30a01f3d40 con 0x7f30a0146000 2024-04-20T11:29:27.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.652+0000 7f309fe00640 1 --2- 172.21.15.152:0/598634588 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f30a0146000 0x7f30a00af300 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f3098036040 tx=0x7f3098004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:27.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.653+0000 7f3097e00640 1 -- 172.21.15.152:0/598634588 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3098043020 con 0x7f30a0146000 2024-04-20T11:29:27.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.653+0000 7f3097e00640 1 -- 172.21.15.152:0/598634588 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f309803e450 con 0x7f30a0146000 2024-04-20T11:29:27.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.653+0000 7f3097e00640 1 -- 172.21.15.152:0/598634588 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f309803e820 con 0x7f30a0146000 2024-04-20T11:29:27.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.653+0000 7f30a6400640 1 -- 172.21.15.152:0/598634588 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30a01f4020 con 0x7f30a0146000 2024-04-20T11:29:27.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.653+0000 7f30a6400640 1 -- 172.21.15.152:0/598634588 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30a015c4f0 con 0x7f30a0146000 2024-04-20T11:29:27.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.654+0000 7f30a6400640 1 -- 172.21.15.152:0/598634588 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30a00689f0 con 0x7f30a0146000 2024-04-20T11:29:27.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.655+0000 7f3097e00640 1 -- 172.21.15.152:0/598634588 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f3098046020 con 0x7f30a0146000 2024-04-20T11:29:27.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.656+0000 7f3097e00640 1 --2- 172.21.15.152:0/598634588 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f307c06caa0 0x7f307c06ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:27.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.656+0000 7f3097e00640 1 -- 172.21.15.152:0/598634588 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f30980ae880 con 0x7f30a0146000 2024-04-20T11:29:27.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.656+0000 7f309f400640 1 --2- 172.21.15.152:0/598634588 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f307c06caa0 0x7f307c06ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:27.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.657+0000 7f309f400640 1 --2- 172.21.15.152:0/598634588 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f307c06caa0 0x7f307c06ef60 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f3090000f60 tx=0x7f3090007c00 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:27.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.661+0000 7f3097e00640 1 -- 172.21.15.152:0/598634588 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f309804d030 con 0x7f30a0146000 2024-04-20T11:29:27.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.864+0000 7f30a6400640 1 -- 172.21.15.152:0/598634588 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f30a00b0f30 con 0x7f30a0146000 2024-04-20T11:29:27.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.865+0000 7f3097e00640 1 -- 172.21.15.152:0/598634588 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v46) ==== 93+0+10 (secure 0 0 0) 0x7f309807c990 con 0x7f30a0146000 2024-04-20T11:29:27.865 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 1 2024-04-20T11:29:27.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.867+0000 7f30a6400640 1 -- 172.21.15.152:0/598634588 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f307c06caa0 msgr2=0x7f307c06ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:27.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.867+0000 7f30a6400640 1 --2- 172.21.15.152:0/598634588 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f307c06caa0 0x7f307c06ef60 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f3090000f60 tx=0x7f3090007c00 comp rx=0 tx=0).stop 2024-04-20T11:29:27.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.867+0000 7f30a6400640 1 -- 172.21.15.152:0/598634588 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f30a0146000 msgr2=0x7f30a00af300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:27.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.867+0000 7f30a6400640 1 --2- 172.21.15.152:0/598634588 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f30a0146000 0x7f30a00af300 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f3098036040 tx=0x7f3098004a40 comp rx=0 tx=0).stop 2024-04-20T11:29:27.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.867+0000 7f30a6400640 1 -- 172.21.15.152:0/598634588 shutdown_connections 2024-04-20T11:29:27.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.867+0000 7f30a6400640 1 --2- 172.21.15.152:0/598634588 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f307c06caa0 0x7f307c06ef60 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:27.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.867+0000 7f30a6400640 1 --2- 172.21.15.152:0/598634588 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f30a0146000 0x7f30a00af300 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:27.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.867+0000 7f30a6400640 1 --2- 172.21.15.152:0/598634588 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f30a00cd8c0 0x7f30a00aedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:27.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.867+0000 7f30a6400640 1 --2- 172.21.15.152:0/598634588 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f30a00b2880 0x7f30a013e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:27.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.868+0000 7f30a6400640 1 -- 172.21.15.152:0/598634588 >> 172.21.15.152:0/598634588 conn(0x7f30a006b980 msgr2=0x7f30a01441f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:27.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.868+0000 7f30a6400640 1 -- 172.21.15.152:0/598634588 shutdown_connections 2024-04-20T11:29:27.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:27.868+0000 7f30a6400640 1 -- 172.21.15.152:0/598634588 wait complete. 2024-04-20T11:29:27.887 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T11:29:28.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.092+0000 7fdb8b000640 1 Processor -- start 2024-04-20T11:29:28.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.092+0000 7fdb8b000640 1 -- start start 2024-04-20T11:29:28.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.094+0000 7fdb8b000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdb84146000 0x7fdb840b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:28.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.094+0000 7fdb8b000640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fdb840b2170 0x7fdb840b25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:28.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.094+0000 7fdb8b000640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb841463e0 con 0x7fdb840cd1b0 2024-04-20T11:29:28.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.094+0000 7fdb8b000640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fdb840b4750 con 0x7fdb840b2170 2024-04-20T11:29:28.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.094+0000 7fdb8b000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fdb840b3e20 con 0x7fdb84146000 2024-04-20T11:29:28.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.094+0000 7fdb89000640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fdb840cd1b0 0x7fdb840cd5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:42480/0 (socket says 172.21.15.152:42480) 2024-04-20T11:29:28.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.094+0000 7fdb89000640 1 -- 172.21.15.152:0/2631306299 learned_addr learned my addr 172.21.15.152:0/2631306299 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:29:28.097 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.095+0000 7fdb83e00640 1 --2- 172.21.15.152:0/2631306299 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdb84146000 0x7fdb840b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:28.097 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.095+0000 7fdb83400640 1 --2- 172.21.15.152:0/2631306299 >> v2:172.21.15.175:3300/0 conn(0x7fdb840b2170 0x7fdb840b25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:28.097 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.095+0000 7fdb82a00640 1 -- 172.21.15.152:0/2631306299 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3218629187 0 0) 0x7fdb841463e0 con 0x7fdb840cd1b0 2024-04-20T11:29:28.098 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.095+0000 7fdb82a00640 1 -- 172.21.15.152:0/2631306299 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdb68003520 con 0x7fdb840cd1b0 2024-04-20T11:29:28.098 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.095+0000 7fdb82a00640 1 -- 172.21.15.152:0/2631306299 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1781113432 0 0) 0x7fdb68003520 con 0x7fdb840cd1b0 2024-04-20T11:29:28.098 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.095+0000 7fdb82a00640 1 -- 172.21.15.152:0/2631306299 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdb84146000 msgr2=0x7fdb840b4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:28.098 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.095+0000 7fdb82a00640 1 --2- 172.21.15.152:0/2631306299 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdb84146000 0x7fdb840b4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:28.098 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.095+0000 7fdb82a00640 1 -- 172.21.15.152:0/2631306299 >> v2:172.21.15.175:3300/0 conn(0x7fdb840b2170 msgr2=0x7fdb840b25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:28.098 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.095+0000 7fdb82a00640 1 --2- 172.21.15.152:0/2631306299 >> v2:172.21.15.175:3300/0 conn(0x7fdb840b2170 0x7fdb840b25f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:28.098 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.095+0000 7fdb82a00640 1 -- 172.21.15.152:0/2631306299 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb840b4100 con 0x7fdb840cd1b0 2024-04-20T11:29:28.098 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.095+0000 7fdb82a00640 1 -- 172.21.15.152:0/2631306299 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7fdb7c0019f0 con 0x7fdb840cd1b0 2024-04-20T11:29:28.098 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.095+0000 7fdb82a00640 1 -- 172.21.15.152:0/2631306299 >> v1:172.21.15.152:6789/0 conn(0x7fdb840cd1b0 legacy=0x7fdb840cd5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:28.098 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.095+0000 7fdb82a00640 1 --2- 172.21.15.152:0/2631306299 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb680039b0 0x7fdb68003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:28.098 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.095+0000 7fdb82a00640 1 --2- 172.21.15.152:0/2631306299 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdb68004800 0x7fdb68004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:28.098 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.096+0000 7fdb82a00640 1 --2- 172.21.15.152:0/2631306299 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdb84146000 0x7fdb680051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:28.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.096+0000 7fdb82a00640 1 -- 172.21.15.152:0/2631306299 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fdb840b4750 con 0x7fdb68004800 2024-04-20T11:29:28.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.096+0000 7fdb82a00640 1 -- 172.21.15.152:0/2631306299 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdb840b3e20 con 0x7fdb680039b0 2024-04-20T11:29:28.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.096+0000 7fdb82a00640 1 -- 172.21.15.152:0/2631306299 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fdb6800b9c0 con 0x7fdb84146000 2024-04-20T11:29:28.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.096+0000 7fdb83400640 1 --2- 172.21.15.152:0/2631306299 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdb68004800 0x7fdb68004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:28.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.096+0000 7fdb89000640 1 --2- 172.21.15.152:0/2631306299 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdb84146000 0x7fdb680051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:28.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.096+0000 7fdb83e00640 1 --2- 172.21.15.152:0/2631306299 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb680039b0 0x7fdb68003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:28.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.096+0000 7fdb83400640 1 -- 172.21.15.152:0/2631306299 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdb84146000 msgr2=0x7fdb680051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:28.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.096+0000 7fdb83400640 1 --2- 172.21.15.152:0/2631306299 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdb84146000 0x7fdb680051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:28.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.096+0000 7fdb83400640 1 -- 172.21.15.152:0/2631306299 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb680039b0 msgr2=0x7fdb68003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:28.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.096+0000 7fdb83400640 1 --2- 172.21.15.152:0/2631306299 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb680039b0 0x7fdb68003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:28.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.096+0000 7fdb83400640 1 -- 172.21.15.152:0/2631306299 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb68003520 con 0x7fdb68004800 2024-04-20T11:29:28.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.096+0000 7fdb83e00640 1 --2- 172.21.15.152:0/2631306299 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb680039b0 0x7fdb68003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:29:28.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.097+0000 7fdb83400640 1 --2- 172.21.15.152:0/2631306299 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdb68004800 0x7fdb68004c80 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fdb70009af0 tx=0x7fdb68005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=655a9c23aa18fcf8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:28.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.098+0000 7fdb82a00640 1 -- 172.21.15.152:0/2631306299 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb70039070 con 0x7fdb68004800 2024-04-20T11:29:28.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.098+0000 7fdb82a00640 1 -- 172.21.15.152:0/2631306299 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb7002c710 con 0x7fdb68004800 2024-04-20T11:29:28.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.098+0000 7fdb82a00640 1 -- 172.21.15.152:0/2631306299 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb70009690 con 0x7fdb68004800 2024-04-20T11:29:28.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.098+0000 7fdb8b000640 1 -- 172.21.15.152:0/2631306299 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdb68004800 msgr2=0x7fdb68004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:28.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.098+0000 7fdb8b000640 1 --2- 172.21.15.152:0/2631306299 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdb68004800 0x7fdb68004c80 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fdb70009af0 tx=0x7fdb68005830 comp rx=0 tx=0).stop 2024-04-20T11:29:28.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.098+0000 7fdb89000640 1 -- 172.21.15.152:0/2631306299 reap_dead start 2024-04-20T11:29:28.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.098+0000 7fdb8b000640 1 -- 172.21.15.152:0/2631306299 shutdown_connections 2024-04-20T11:29:28.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.098+0000 7fdb8b000640 1 -- 172.21.15.152:0/2631306299 >> 172.21.15.152:0/2631306299 conn(0x7fdb8406b980 msgr2=0x7fdb8406bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:28.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.099+0000 7fdb8b000640 1 -- 172.21.15.152:0/2631306299 shutdown_connections 2024-04-20T11:29:28.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.099+0000 7fdb8b000640 1 -- 172.21.15.152:0/2631306299 wait complete. 2024-04-20T11:29:28.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.100+0000 7fdb8b000640 1 Processor -- start 2024-04-20T11:29:28.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.100+0000 7fdb8b000640 1 -- start start 2024-04-20T11:29:28.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.101+0000 7fdb8b000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb68004800 0x7fdb84141a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:28.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.101+0000 7fdb8b000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdb8413d5f0 0x7fdb8415c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:28.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.101+0000 7fdb8b000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdb840ae350 0x7fdb840ae7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:28.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.101+0000 7fdb8b000640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fdb840b4d60 con 0x7fdb840ae350 2024-04-20T11:29:28.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.101+0000 7fdb8b000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdb840b4be0 con 0x7fdb68004800 2024-04-20T11:29:28.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.101+0000 7fdb8b000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fdb840b4ee0 con 0x7fdb8413d5f0 2024-04-20T11:29:28.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.102+0000 7fdb89000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdb840ae350 0x7fdb840ae7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:28.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.102+0000 7fdb89000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdb840ae350 0x7fdb840ae7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35354/0 (socket says 172.21.15.152:35354) 2024-04-20T11:29:28.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.102+0000 7fdb89000640 1 -- 172.21.15.152:0/1155468393 learned_addr learned my addr 172.21.15.152:0/1155468393 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:29:28.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.102+0000 7fdb83400640 1 --2- 172.21.15.152:0/1155468393 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdb8413d5f0 0x7fdb8415c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:28.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.102+0000 7fdb89000640 1 -- 172.21.15.152:0/1155468393 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdb8413d5f0 msgr2=0x7fdb8415c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:28.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.102+0000 7fdb89000640 1 --2- 172.21.15.152:0/1155468393 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdb8413d5f0 0x7fdb8415c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:28.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.102+0000 7fdb89000640 1 -- 172.21.15.152:0/1155468393 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb68004800 msgr2=0x7fdb84141a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:28.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.102+0000 7fdb89000640 1 --2- 172.21.15.152:0/1155468393 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb68004800 0x7fdb84141a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:28.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.102+0000 7fdb89000640 1 -- 172.21.15.152:0/1155468393 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb8415c5c0 con 0x7fdb840ae350 2024-04-20T11:29:28.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.102+0000 7fdb89000640 1 --2- 172.21.15.152:0/1155468393 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdb840ae350 0x7fdb840ae7d0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fdb841601e0 tx=0x7fdb7c007bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:28.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.103+0000 7fdb7be00640 1 -- 172.21.15.152:0/1155468393 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb7c00ea80 con 0x7fdb840ae350 2024-04-20T11:29:28.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.103+0000 7fdb7be00640 1 -- 172.21.15.152:0/1155468393 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb7c00ec20 con 0x7fdb840ae350 2024-04-20T11:29:28.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.103+0000 7fdb7be00640 1 -- 172.21.15.152:0/1155468393 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb7c014480 con 0x7fdb840ae350 2024-04-20T11:29:28.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.103+0000 7fdb8b000640 1 -- 172.21.15.152:0/1155468393 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb8413db40 con 0x7fdb840ae350 2024-04-20T11:29:28.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.103+0000 7fdb8b000640 1 -- 172.21.15.152:0/1155468393 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb8413e100 con 0x7fdb840ae350 2024-04-20T11:29:28.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.104+0000 7fdb8b000640 1 -- 172.21.15.152:0/1155468393 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb840689f0 con 0x7fdb840ae350 2024-04-20T11:29:28.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.106+0000 7fdb7be00640 1 -- 172.21.15.152:0/1155468393 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7fdb7c014620 con 0x7fdb840ae350 2024-04-20T11:29:28.107 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.107+0000 7fdb7be00640 1 --2- 172.21.15.152:0/1155468393 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fdb6406caa0 0x7fdb6406ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:28.107 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.107+0000 7fdb7be00640 1 -- 172.21.15.152:0/1155468393 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7fdb7c08bdc0 con 0x7fdb840ae350 2024-04-20T11:29:28.108 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.107+0000 7fdb83e00640 1 --2- 172.21.15.152:0/1155468393 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fdb6406caa0 0x7fdb6406ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:28.108 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.108+0000 7fdb83e00640 1 --2- 172.21.15.152:0/1155468393 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fdb6406caa0 0x7fdb6406ef60 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fdb74009910 tx=0x7fdb74008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:28.111 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.111+0000 7fdb7be00640 1 -- 172.21.15.152:0/1155468393 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fdb8415c5c0 con 0x7fdb840ae350 2024-04-20T11:29:28.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.315+0000 7fdb8b000640 1 -- 172.21.15.152:0/1155468393 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fdb841423b0 con 0x7fdb840ae350 2024-04-20T11:29:28.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.316+0000 7fdb7be00640 1 -- 172.21.15.152:0/1155468393 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v46) ==== 104+0+11 (secure 0 0 0) 0x7fdb7c059f60 con 0x7fdb840ae350 2024-04-20T11:29:28.317 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 32 2024-04-20T11:29:28.320 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.319+0000 7fdb8b000640 1 -- 172.21.15.152:0/1155468393 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fdb6406caa0 msgr2=0x7fdb6406ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:28.320 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.319+0000 7fdb8b000640 1 --2- 172.21.15.152:0/1155468393 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fdb6406caa0 0x7fdb6406ef60 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fdb74009910 tx=0x7fdb74008040 comp rx=0 tx=0).stop 2024-04-20T11:29:28.320 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.319+0000 7fdb8b000640 1 -- 172.21.15.152:0/1155468393 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdb840ae350 msgr2=0x7fdb840ae7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:28.320 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.319+0000 7fdb8b000640 1 --2- 172.21.15.152:0/1155468393 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdb840ae350 0x7fdb840ae7d0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fdb841601e0 tx=0x7fdb7c007bc0 comp rx=0 tx=0).stop 2024-04-20T11:29:28.320 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.319+0000 7fdb8b000640 1 -- 172.21.15.152:0/1155468393 shutdown_connections 2024-04-20T11:29:28.320 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.319+0000 7fdb8b000640 1 --2- 172.21.15.152:0/1155468393 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fdb6406caa0 0x7fdb6406ef60 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:28.320 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.319+0000 7fdb8b000640 1 --2- 172.21.15.152:0/1155468393 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdb840ae350 0x7fdb840ae7d0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:28.320 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.319+0000 7fdb8b000640 1 --2- 172.21.15.152:0/1155468393 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdb8413d5f0 0x7fdb8415c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:28.320 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.319+0000 7fdb8b000640 1 --2- 172.21.15.152:0/1155468393 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb68004800 0x7fdb84141a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:28.320 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.319+0000 7fdb8b000640 1 -- 172.21.15.152:0/1155468393 >> 172.21.15.152:0/1155468393 conn(0x7fdb8406b980 msgr2=0x7fdb8415cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:28.320 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.319+0000 7fdb8b000640 1 -- 172.21.15.152:0/1155468393 shutdown_connections 2024-04-20T11:29:28.320 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.319+0000 7fdb8b000640 1 -- 172.21.15.152:0/1155468393 wait complete. 2024-04-20T11:29:28.338 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T11:29:28.536 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.535+0000 7fb21bc00640 1 Processor -- start 2024-04-20T11:29:28.536 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.535+0000 7fb21bc00640 1 -- start start 2024-04-20T11:29:28.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.537+0000 7fb21bc00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fb214146000 0x7fb2140b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:28.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.537+0000 7fb21bc00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fb2140cd8c0 0x7fb2140b2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:28.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.537+0000 7fb21bc00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2141463e0 con 0x7fb2140b2170 2024-04-20T11:29:28.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.537+0000 7fb21bc00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fb2140cdca0 con 0x7fb214146000 2024-04-20T11:29:28.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.537+0000 7fb21bc00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fb2140b4750 con 0x7fb2140cd8c0 2024-04-20T11:29:28.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.537+0000 7fb219200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fb214146000 0x7fb2140b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:28.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.537+0000 7fb213e00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fb2140b2170 0x7fb2140cd1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:42484/0 (socket says 172.21.15.152:42484) 2024-04-20T11:29:28.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.537+0000 7fb213e00640 1 -- 172.21.15.152:0/4243350718 learned_addr learned my addr 172.21.15.152:0/4243350718 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:29:28.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.537+0000 7fb219c00640 1 --2- 172.21.15.152:0/4243350718 >> v2:172.21.15.175:3300/0 conn(0x7fb2140cd8c0 0x7fb2140b2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:28.539 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.538+0000 7fb219200640 1 -- 172.21.15.152:0/4243350718 >> v2:172.21.15.175:3300/0 conn(0x7fb2140cd8c0 msgr2=0x7fb2140b2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:28.539 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.538+0000 7fb219200640 1 --2- 172.21.15.152:0/4243350718 >> v2:172.21.15.175:3300/0 conn(0x7fb2140cd8c0 0x7fb2140b2620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:28.539 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.538+0000 7fb219200640 1 -- 172.21.15.152:0/4243350718 >> v1:172.21.15.152:6789/0 conn(0x7fb2140b2170 legacy=0x7fb2140cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:28.539 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.538+0000 7fb219200640 1 -- 172.21.15.152:0/4243350718 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2140b3e20 con 0x7fb214146000 2024-04-20T11:29:28.539 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.538+0000 7fb219c00640 1 --2- 172.21.15.152:0/4243350718 >> v2:172.21.15.175:3300/0 conn(0x7fb2140cd8c0 0x7fb2140b2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:29:28.540 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.539+0000 7fb219200640 1 --2- 172.21.15.152:0/4243350718 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb214146000 0x7fb2140b4be0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fb2141601b0 tx=0x7fb20802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64db94968483a6b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:28.540 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.539+0000 7fb213400640 1 -- 172.21.15.152:0/4243350718 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2080315a0 con 0x7fb214146000 2024-04-20T11:29:28.540 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.539+0000 7fb213400640 1 -- 172.21.15.152:0/4243350718 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb208037070 con 0x7fb214146000 2024-04-20T11:29:28.540 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.539+0000 7fb213400640 1 -- 172.21.15.152:0/4243350718 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb208031960 con 0x7fb214146000 2024-04-20T11:29:28.541 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.540+0000 7fb21bc00640 1 -- 172.21.15.152:0/4243350718 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb214146000 msgr2=0x7fb2140b4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:28.541 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.540+0000 7fb21bc00640 1 --2- 172.21.15.152:0/4243350718 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb214146000 0x7fb2140b4be0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fb2141601b0 tx=0x7fb20802b840 comp rx=0 tx=0).stop 2024-04-20T11:29:28.541 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.540+0000 7fb21bc00640 1 -- 172.21.15.152:0/4243350718 shutdown_connections 2024-04-20T11:29:28.541 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.540+0000 7fb21bc00640 1 --2- 172.21.15.152:0/4243350718 >> v2:172.21.15.175:3300/0 conn(0x7fb2140cd8c0 0x7fb2140b2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:28.541 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.540+0000 7fb21bc00640 1 --2- 172.21.15.152:0/4243350718 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb214146000 0x7fb2140b4be0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:28.541 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.540+0000 7fb21bc00640 1 -- 172.21.15.152:0/4243350718 >> 172.21.15.152:0/4243350718 conn(0x7fb21406b980 msgr2=0x7fb21406bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:28.541 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.540+0000 7fb21bc00640 1 -- 172.21.15.152:0/4243350718 shutdown_connections 2024-04-20T11:29:28.541 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.540+0000 7fb21bc00640 1 -- 172.21.15.152:0/4243350718 wait complete. 2024-04-20T11:29:28.542 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.542+0000 7fb21bc00640 1 Processor -- start 2024-04-20T11:29:28.542 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.542+0000 7fb21bc00640 1 -- start start 2024-04-20T11:29:28.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.543+0000 7fb21bc00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb2140b2170 0x7fb21413b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:28.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.543+0000 7fb21bc00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb2140cd8c0 0x7fb21406dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:28.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.543+0000 7fb21bc00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb214146000 0x7fb21406e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:28.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.543+0000 7fb21bc00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fb21415ccf0 con 0x7fb2140cd8c0 2024-04-20T11:29:28.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.543+0000 7fb21bc00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb2140b43e0 con 0x7fb214146000 2024-04-20T11:29:28.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.543+0000 7fb21bc00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb21415ce70 con 0x7fb2140b2170 2024-04-20T11:29:28.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.543+0000 7fb213e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb2140cd8c0 0x7fb21406dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:28.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.543+0000 7fb213e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb2140cd8c0 0x7fb21406dc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35366/0 (socket says 172.21.15.152:35366) 2024-04-20T11:29:28.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.543+0000 7fb213e00640 1 -- 172.21.15.152:0/1927763675 learned_addr learned my addr 172.21.15.152:0/1927763675 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:29:28.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.544+0000 7fb219200640 1 --2- 172.21.15.152:0/1927763675 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb2140b2170 0x7fb21413b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:28.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.544+0000 7fb213e00640 1 -- 172.21.15.152:0/1927763675 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb2140b2170 msgr2=0x7fb21413b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:28.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.544+0000 7fb213e00640 1 --2- 172.21.15.152:0/1927763675 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb2140b2170 0x7fb21413b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:28.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.544+0000 7fb213e00640 1 -- 172.21.15.152:0/1927763675 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb214146000 msgr2=0x7fb21406e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:28.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.544+0000 7fb213e00640 1 --2- 172.21.15.152:0/1927763675 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb214146000 0x7fb21406e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:28.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.544+0000 7fb213e00640 1 -- 172.21.15.152:0/1927763675 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2140c9be0 con 0x7fb2140cd8c0 2024-04-20T11:29:28.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.544+0000 7fb219200640 1 --2- 172.21.15.152:0/1927763675 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb2140b2170 0x7fb21413b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:29:28.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.544+0000 7fb213e00640 1 --2- 172.21.15.152:0/1927763675 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb2140cd8c0 0x7fb21406dc50 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fb20000c400 tx=0x7fb20000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:28.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.544+0000 7fb210c00640 1 -- 172.21.15.152:0/1927763675 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb200017070 con 0x7fb2140cd8c0 2024-04-20T11:29:28.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.544+0000 7fb210c00640 1 -- 172.21.15.152:0/1927763675 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb200005ce0 con 0x7fb2140cd8c0 2024-04-20T11:29:28.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.544+0000 7fb210c00640 1 -- 172.21.15.152:0/1927763675 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2000072f0 con 0x7fb2140cd8c0 2024-04-20T11:29:28.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.544+0000 7fb21bc00640 1 -- 172.21.15.152:0/1927763675 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2140c9ec0 con 0x7fb2140cd8c0 2024-04-20T11:29:28.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.545+0000 7fb21bc00640 1 -- 172.21.15.152:0/1927763675 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb21413ba20 con 0x7fb2140cd8c0 2024-04-20T11:29:28.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.546+0000 7fb21bc00640 1 -- 172.21.15.152:0/1927763675 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2140689f0 con 0x7fb2140cd8c0 2024-04-20T11:29:28.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.547+0000 7fb210c00640 1 -- 172.21.15.152:0/1927763675 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7fb20001a070 con 0x7fb2140cd8c0 2024-04-20T11:29:28.548 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.548+0000 7fb210c00640 1 --2- 172.21.15.152:0/1927763675 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fb1f406caa0 0x7fb1f406ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:28.548 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.548+0000 7fb210c00640 1 -- 172.21.15.152:0/1927763675 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7fb20008b260 con 0x7fb2140cd8c0 2024-04-20T11:29:28.548 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.548+0000 7fb219200640 1 --2- 172.21.15.152:0/1927763675 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fb1f406caa0 0x7fb1f406ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:28.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.549+0000 7fb219200640 1 --2- 172.21.15.152:0/1927763675 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fb1f406caa0 0x7fb1f406ef60 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fb208036000 tx=0x7fb208008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:28.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.552+0000 7fb210c00640 1 -- 172.21.15.152:0/1927763675 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fb200059440 con 0x7fb2140cd8c0 2024-04-20T11:29:28.753 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.752+0000 7fb21bc00640 1 -- 172.21.15.152:0/1927763675 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb2140ca250 con 0x7fb2140cd8c0 2024-04-20T11:29:28.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.753+0000 7fb210c00640 1 -- 172.21.15.152:0/1927763675 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v46) ==== 100+0+11 (secure 0 0 0) 0x7fb200058d40 con 0x7fb2140cd8c0 2024-04-20T11:29:28.754 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 64 2024-04-20T11:29:28.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.756+0000 7fb21bc00640 1 -- 172.21.15.152:0/1927763675 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fb1f406caa0 msgr2=0x7fb1f406ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:28.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.756+0000 7fb21bc00640 1 --2- 172.21.15.152:0/1927763675 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fb1f406caa0 0x7fb1f406ef60 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fb208036000 tx=0x7fb208008030 comp rx=0 tx=0).stop 2024-04-20T11:29:28.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.756+0000 7fb21bc00640 1 -- 172.21.15.152:0/1927763675 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb2140cd8c0 msgr2=0x7fb21406dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:28.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.756+0000 7fb21bc00640 1 --2- 172.21.15.152:0/1927763675 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb2140cd8c0 0x7fb21406dc50 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fb20000c400 tx=0x7fb20000c8c0 comp rx=0 tx=0).stop 2024-04-20T11:29:28.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.756+0000 7fb21bc00640 1 -- 172.21.15.152:0/1927763675 shutdown_connections 2024-04-20T11:29:28.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.756+0000 7fb21bc00640 1 --2- 172.21.15.152:0/1927763675 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fb1f406caa0 0x7fb1f406ef60 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:28.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.756+0000 7fb21bc00640 1 --2- 172.21.15.152:0/1927763675 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb214146000 0x7fb21406e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:28.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.756+0000 7fb21bc00640 1 --2- 172.21.15.152:0/1927763675 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb2140cd8c0 0x7fb21406dc50 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:28.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.756+0000 7fb21bc00640 1 --2- 172.21.15.152:0/1927763675 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb2140b2170 0x7fb21413b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:28.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.756+0000 7fb21bc00640 1 -- 172.21.15.152:0/1927763675 >> 172.21.15.152:0/1927763675 conn(0x7fb21406b980 msgr2=0x7fb21415d230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:28.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.756+0000 7fb21bc00640 1 -- 172.21.15.152:0/1927763675 shutdown_connections 2024-04-20T11:29:28.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.756+0000 7fb21bc00640 1 -- 172.21.15.152:0/1927763675 wait complete. 2024-04-20T11:29:28.774 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-20T11:29:28.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.958+0000 7f27e2800640 1 Processor -- start 2024-04-20T11:29:28.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.958+0000 7f27e2800640 1 -- start start 2024-04-20T11:29:28.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.960+0000 7f27e2800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f27dc161d30 0x7f27dc0d0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:28.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.960+0000 7f27e2800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f27dc0d2b80 0x7f27dc0d3030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:28.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.960+0000 7f27e2800640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27dc0d26a0 con 0x7f27dc0d4ec0 2024-04-20T11:29:28.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.960+0000 7f27e2800640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f27dc0d0540 con 0x7f27dc161d30 2024-04-20T11:29:28.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.960+0000 7f27e2800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f27dc0cfb70 con 0x7f27dc0d2b80 2024-04-20T11:29:28.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.960+0000 7f27e1400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f27dc161d30 0x7f27dc0d0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:28.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.960+0000 7f27e1400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f27dc161d30 0x7f27dc0d0000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35376/0 (socket says 172.21.15.152:35376) 2024-04-20T11:29:28.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.960+0000 7f27e1400640 1 -- 172.21.15.152:0/2533557984 learned_addr learned my addr 172.21.15.152:0/2533557984 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:29:28.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.961+0000 7f27e1400640 1 -- 172.21.15.152:0/2533557984 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f27dc0d2b80 msgr2=0x7f27dc0d3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:28.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.961+0000 7f27e1e00640 1 --2- 172.21.15.152:0/2533557984 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f27dc0d2b80 0x7f27dc0d3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:28.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.961+0000 7f27e1400640 1 --2- 172.21.15.152:0/2533557984 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f27dc0d2b80 0x7f27dc0d3030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:28.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.961+0000 7f27e1400640 1 -- 172.21.15.152:0/2533557984 >> v1:172.21.15.175:6789/0 conn(0x7f27dc0d4ec0 legacy=0x7f27dc0d15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:28.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.961+0000 7f27e1400640 1 -- 172.21.15.152:0/2533557984 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27dc0d53c0 con 0x7f27dc161d30 2024-04-20T11:29:28.963 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.962+0000 7f27e1400640 1 --2- 172.21.15.152:0/2533557984 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f27dc161d30 0x7f27dc0d0000 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f27d0009870 tx=0x7f27d002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7273545c19ba56b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:28.963 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.962+0000 7f27dbe00640 1 -- 172.21.15.152:0/2533557984 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d0035020 con 0x7f27dc161d30 2024-04-20T11:29:28.963 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.962+0000 7f27dbe00640 1 -- 172.21.15.152:0/2533557984 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27d0031640 con 0x7f27dc161d30 2024-04-20T11:29:28.964 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.962+0000 7f27dbe00640 1 -- 172.21.15.152:0/2533557984 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d002d040 con 0x7f27dc161d30 2024-04-20T11:29:28.964 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.963+0000 7f27e2800640 1 -- 172.21.15.152:0/2533557984 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f27dc161d30 msgr2=0x7f27dc0d0000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:28.964 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.963+0000 7f27e2800640 1 --2- 172.21.15.152:0/2533557984 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f27dc161d30 0x7f27dc0d0000 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f27d0009870 tx=0x7f27d002b8f0 comp rx=0 tx=0).stop 2024-04-20T11:29:28.964 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.963+0000 7f27e2800640 1 -- 172.21.15.152:0/2533557984 shutdown_connections 2024-04-20T11:29:28.964 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.963+0000 7f27e2800640 1 --2- 172.21.15.152:0/2533557984 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f27dc0d2b80 0x7f27dc0d3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:28.964 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.963+0000 7f27e2800640 1 --2- 172.21.15.152:0/2533557984 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f27dc161d30 0x7f27dc0d0000 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:28.964 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.963+0000 7f27e2800640 1 -- 172.21.15.152:0/2533557984 >> 172.21.15.152:0/2533557984 conn(0x7f27dc08bfd0 msgr2=0x7f27dc0d45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:28.964 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.963+0000 7f27e2800640 1 -- 172.21.15.152:0/2533557984 shutdown_connections 2024-04-20T11:29:28.964 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.963+0000 7f27e2800640 1 -- 172.21.15.152:0/2533557984 wait complete. 2024-04-20T11:29:28.965 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.965+0000 7f27e2800640 1 Processor -- start 2024-04-20T11:29:28.965 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.965+0000 7f27e2800640 1 -- start start 2024-04-20T11:29:28.967 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.966+0000 7f27e2800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f27dc0d2b80 0x7f27dc0ccc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:28.967 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.966+0000 7f27e2800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27dc0d4ec0 0x7f27dc151850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:28.967 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.966+0000 7f27e2800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f27dc161d30 0x7f27dc15cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:28.967 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.966+0000 7f27e2800640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f27dc0cf1b0 con 0x7f27dc0d2b80 2024-04-20T11:29:28.967 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.966+0000 7f27e2800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f27dc0cf800 con 0x7f27dc0d4ec0 2024-04-20T11:29:28.967 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.966+0000 7f27e2800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f27dc162110 con 0x7f27dc161d30 2024-04-20T11:29:28.967 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.966+0000 7f27e1400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f27dc0d2b80 0x7f27dc0ccc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:28.967 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.966+0000 7f27e1400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f27dc0d2b80 0x7f27dc0ccc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35388/0 (socket says 172.21.15.152:35388) 2024-04-20T11:29:28.967 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.966+0000 7f27e1400640 1 -- 172.21.15.152:0/1067162800 learned_addr learned my addr 172.21.15.152:0/1067162800 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:29:28.968 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.966+0000 7f27e1400640 1 -- 172.21.15.152:0/1067162800 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f27dc161d30 msgr2=0x7f27dc15cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:28.968 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.966+0000 7f27e1400640 1 --2- 172.21.15.152:0/1067162800 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f27dc161d30 0x7f27dc15cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:28.968 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.966+0000 7f27e1400640 1 -- 172.21.15.152:0/1067162800 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27dc0d4ec0 msgr2=0x7f27dc151850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:28.968 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.966+0000 7f27e1400640 1 --2- 172.21.15.152:0/1067162800 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27dc0d4ec0 0x7f27dc151850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:28.968 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.966+0000 7f27e1400640 1 -- 172.21.15.152:0/1067162800 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27dc0cbbc0 con 0x7f27dc0d2b80 2024-04-20T11:29:28.969 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.967+0000 7f27e1400640 1 --2- 172.21.15.152:0/1067162800 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f27dc0d2b80 0x7f27dc0ccc60 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f27d002b930 tx=0x7f27d00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:28.969 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.967+0000 7f27d9600640 1 -- 172.21.15.152:0/1067162800 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d0040070 con 0x7f27dc0d2b80 2024-04-20T11:29:28.969 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.967+0000 7f27d9600640 1 -- 172.21.15.152:0/1067162800 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27d0039c10 con 0x7f27dc0d2b80 2024-04-20T11:29:28.969 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.967+0000 7f27d9600640 1 -- 172.21.15.152:0/1067162800 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d0035040 con 0x7f27dc0d2b80 2024-04-20T11:29:28.969 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.967+0000 7f27e2800640 1 -- 172.21.15.152:0/1067162800 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27dc0cbea0 con 0x7f27dc0d2b80 2024-04-20T11:29:28.969 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.967+0000 7f27e2800640 1 -- 172.21.15.152:0/1067162800 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27dc0cc460 con 0x7f27dc0d2b80 2024-04-20T11:29:28.969 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.968+0000 7f27e2800640 1 -- 172.21.15.152:0/1067162800 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27dc075e60 con 0x7f27dc0d2b80 2024-04-20T11:29:28.970 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.969+0000 7f27d9600640 1 -- 172.21.15.152:0/1067162800 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f27d0038020 con 0x7f27dc0d2b80 2024-04-20T11:29:28.971 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.970+0000 7f27d9600640 1 --2- 172.21.15.152:0/1067162800 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f27c006caa0 0x7f27c006ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:28.971 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.971+0000 7f27d9600640 1 -- 172.21.15.152:0/1067162800 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f27d00adaa0 con 0x7f27dc0d2b80 2024-04-20T11:29:28.971 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.971+0000 7f27e0a00640 1 --2- 172.21.15.152:0/1067162800 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f27c006caa0 0x7f27c006ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:28.971 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.971+0000 7f27e0a00640 1 --2- 172.21.15.152:0/1067162800 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f27c006caa0 0x7f27c006ef60 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f27cc007fd0 tx=0x7f27cc007d80 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:28.975 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:28.975+0000 7f27d9600640 1 -- 172.21.15.152:0/1067162800 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f27d007bc40 con 0x7f27dc0d2b80 2024-04-20T11:29:29.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:29.173+0000 7f27e2800640 1 -- 172.21.15.152:0/1067162800 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f27dc0d1d70 con 0x7f27dc0d2b80 2024-04-20T11:29:29.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:29.174+0000 7f27d9600640 1 -- 172.21.15.152:0/1067162800 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v46) ==== 80+0+0 (secure 0 0 0) 0x7f27d007b540 con 0x7f27dc0d2b80 2024-04-20T11:29:29.174 INFO:teuthology.orchestra.run.smithi152.stderr:listed 0 entries 2024-04-20T11:29:29.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:29.176+0000 7f27e2800640 1 -- 172.21.15.152:0/1067162800 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f27c006caa0 msgr2=0x7f27c006ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:29.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:29.176+0000 7f27e2800640 1 --2- 172.21.15.152:0/1067162800 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f27c006caa0 0x7f27c006ef60 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f27cc007fd0 tx=0x7f27cc007d80 comp rx=0 tx=0).stop 2024-04-20T11:29:29.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:29.176+0000 7f27e2800640 1 -- 172.21.15.152:0/1067162800 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f27dc0d2b80 msgr2=0x7f27dc0ccc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:29.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:29.176+0000 7f27e2800640 1 --2- 172.21.15.152:0/1067162800 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f27dc0d2b80 0x7f27dc0ccc60 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f27d002b930 tx=0x7f27d00046c0 comp rx=0 tx=0).stop 2024-04-20T11:29:29.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:29.176+0000 7f27e2800640 1 -- 172.21.15.152:0/1067162800 shutdown_connections 2024-04-20T11:29:29.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:29.176+0000 7f27e2800640 1 --2- 172.21.15.152:0/1067162800 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f27c006caa0 0x7f27c006ef60 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:29.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:29.176+0000 7f27e2800640 1 --2- 172.21.15.152:0/1067162800 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f27dc161d30 0x7f27dc15cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:29.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:29.176+0000 7f27e2800640 1 --2- 172.21.15.152:0/1067162800 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27dc0d4ec0 0x7f27dc151850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:29.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:29.176+0000 7f27e2800640 1 --2- 172.21.15.152:0/1067162800 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f27dc0d2b80 0x7f27dc0ccc60 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:29.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:29.176+0000 7f27e2800640 1 -- 172.21.15.152:0/1067162800 >> 172.21.15.152:0/1067162800 conn(0x7f27dc08bfd0 msgr2=0x7f27dc0ca280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:29.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:29.177+0000 7f27e2800640 1 -- 172.21.15.152:0/1067162800 shutdown_connections 2024-04-20T11:29:29.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:29.177+0000 7f27e2800640 1 -- 172.21.15.152:0/1067162800 wait complete. 2024-04-20T11:29:29.195 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi175.front.sepia.ceph.com... 2024-04-20T11:29:29.195 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:29:29.195 DEBUG:teuthology.orchestra.run.smithi175:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-04-20T11:29:29.279 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi175.stderr:ceph-fuse[30460]: fuse finished with error 0 and tester_r 0 2024-04-20T11:29:29.283 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T11:29:35.384 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi175.front.sepia.ceph.com 2024-04-20T11:29:35.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:29:35.385 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-20T11:29:35.443 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:29:35.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:35.654+0000 7f93b0400640 1 Processor -- start 2024-04-20T11:29:35.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:35.654+0000 7f93b0400640 1 -- start start 2024-04-20T11:29:35.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:35.655+0000 7f93b0400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f93a815c560 0x7f93a80b3df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:35.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:35.655+0000 7f93b0400640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93a815fd20 con 0x7f93a80b4330 2024-04-20T11:29:35.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:35.655+0000 7f93b0400640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93a815ef60 con 0x7f93a80d0810 2024-04-20T11:29:35.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:35.655+0000 7f93b0400640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f93a815f240 con 0x7f93a815c560 2024-04-20T11:29:35.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:35.656+0000 7f93ada00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f93a815c560 0x7f93a80b3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:35.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:35.656+0000 7f93ada00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f93a815c560 0x7f93a80b3df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35404/0 (socket says 172.21.15.152:35404) 2024-04-20T11:29:35.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:35.656+0000 7f93ada00640 1 -- 172.21.15.152:0/3472864262 learned_addr learned my addr 172.21.15.152:0/3472864262 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:29:35.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:35.656+0000 7f93ad000640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f93a80b4330 0x7f93a80d0100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:42492/0 (socket says 172.21.15.152:42492) 2024-04-20T11:29:35.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:35.656+0000 7f93ada00640 1 -- 172.21.15.152:0/3472864262 >> v1:172.21.15.175:6789/0 conn(0x7f93a80d0810 legacy=0x7f93a80ada90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:35.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:35.656+0000 7f93ada00640 1 -- 172.21.15.152:0/3472864262 >> v1:172.21.15.152:6789/0 conn(0x7f93a80b4330 legacy=0x7f93a80d0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:35.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:35.656+0000 7f93ada00640 1 -- 172.21.15.152:0/3472864262 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93a815f520 con 0x7f93a815c560 2024-04-20T11:29:35.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:35.657+0000 7f93ada00640 1 --2- 172.21.15.152:0/3472864262 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f93a815c560 0x7f93a80b3df0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f939c009870 tx=0x7f939c02b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc5cbc2567196856 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:35.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:35.658+0000 7f93a7e00640 1 -- 172.21.15.152:0/3472864262 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f939c035020 con 0x7f93a815c560 2024-04-20T11:29:35.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:35.658+0000 7f93a7e00640 1 -- 172.21.15.152:0/3472864262 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f939c02d040 con 0x7f93a815c560 2024-04-20T11:29:35.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:35.658+0000 7f93a7e00640 1 -- 172.21.15.152:0/3472864262 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f939c031a80 con 0x7f93a815c560 2024-04-20T11:29:35.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:35.658+0000 7f93b0400640 1 -- 172.21.15.152:0/3472864262 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f93a815c560 msgr2=0x7f93a80b3df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:35.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:35.658+0000 7f93b0400640 1 --2- 172.21.15.152:0/3472864262 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f93a815c560 0x7f93a80b3df0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f939c009870 tx=0x7f939c02b900 comp rx=0 tx=0).stop 2024-04-20T11:29:35.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:35.659+0000 7f93b0400640 1 -- 172.21.15.152:0/3472864262 shutdown_connections 2024-04-20T11:29:35.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:35.659+0000 7f93b0400640 1 --2- 172.21.15.152:0/3472864262 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f93a815c560 0x7f93a80b3df0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:35.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:35.659+0000 7f93b0400640 1 -- 172.21.15.152:0/3472864262 >> 172.21.15.152:0/3472864262 conn(0x7f93a8067cf0 msgr2=0x7f93a806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:35.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:35.659+0000 7f93b0400640 1 -- 172.21.15.152:0/3472864262 shutdown_connections 2024-04-20T11:29:35.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:35.659+0000 7f93b0400640 1 -- 172.21.15.152:0/3472864262 wait complete. 2024-04-20T11:29:35.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:35.660+0000 7f93b0400640 1 Processor -- start 2024-04-20T11:29:35.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:35.661+0000 7f93b0400640 1 -- start start 2024-04-20T11:29:35.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:35.661+0000 7f93b0400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f93a80b4330 0x7f93a80cf6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:35.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:35.662+0000 7f93b0400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93a80d0810 0x7f93a80c5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:35.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:35.662+0000 7f93b0400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f93a815c560 0x7f93a814b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:35.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:35.662+0000 7f93b0400640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f93a80ae1a0 con 0x7f93a80b4330 2024-04-20T11:29:35.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:35.662+0000 7f93b0400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f93a80d0dc0 con 0x7f93a80d0810 2024-04-20T11:29:35.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:35.662+0000 7f93b0400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f93a80ae320 con 0x7f93a815c560 2024-04-20T11:29:35.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:35.662+0000 7f93ada00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f93a80b4330 0x7f93a80cf6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:35.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:35.662+0000 7f93ada00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f93a80b4330 0x7f93a80cf6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35418/0 (socket says 172.21.15.152:35418) 2024-04-20T11:29:35.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:35.662+0000 7f93ada00640 1 -- 172.21.15.152:0/1156293532 learned_addr learned my addr 172.21.15.152:0/1156293532 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:29:35.664 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:35.662+0000 7f93ad000640 1 --2- 172.21.15.152:0/1156293532 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93a80d0810 0x7f93a80c5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:35.664 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:35.662+0000 7f93ada00640 1 -- 172.21.15.152:0/1156293532 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f93a815c560 msgr2=0x7f93a814b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:35.664 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:35.662+0000 7f93ada00640 1 --2- 172.21.15.152:0/1156293532 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f93a815c560 0x7f93a814b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:35.664 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:35.662+0000 7f93ada00640 1 -- 172.21.15.152:0/1156293532 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93a80d0810 msgr2=0x7f93a80c5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:35.664 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:35.662+0000 7f93ada00640 1 --2- 172.21.15.152:0/1156293532 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93a80d0810 0x7f93a80c5450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:35.664 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:35.662+0000 7f93ada00640 1 -- 172.21.15.152:0/1156293532 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93a8160ce0 con 0x7f93a80b4330 2024-04-20T11:29:35.664 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:35.663+0000 7f93ada00640 1 --2- 172.21.15.152:0/1156293532 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f93a80b4330 0x7f93a80cf6b0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f939c009870 tx=0x7f939c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:35.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:35.663+0000 7f93a5600640 1 -- 172.21.15.152:0/1156293532 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f939c03ac60 con 0x7f93a80b4330 2024-04-20T11:29:35.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:35.663+0000 7f93a5600640 1 -- 172.21.15.152:0/1156293532 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f939c03ae00 con 0x7f93a80b4330 2024-04-20T11:29:35.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:35.663+0000 7f93b0400640 1 -- 172.21.15.152:0/1156293532 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93a8160fc0 con 0x7f93a80b4330 2024-04-20T11:29:35.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:35.663+0000 7f93a5600640 1 -- 172.21.15.152:0/1156293532 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f939c044630 con 0x7f93a80b4330 2024-04-20T11:29:35.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:35.663+0000 7f93b0400640 1 -- 172.21.15.152:0/1156293532 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93a8161860 con 0x7f93a80b4330 2024-04-20T11:29:35.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:35.664+0000 7f93b0400640 1 -- 172.21.15.152:0/1156293532 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93a8161190 con 0x7f93a80b4330 2024-04-20T11:29:35.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:35.665+0000 7f93a5600640 1 -- 172.21.15.152:0/1156293532 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f939c04b060 con 0x7f93a80b4330 2024-04-20T11:29:35.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:35.666+0000 7f93a5600640 1 --2- 172.21.15.152:0/1156293532 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f938806caa0 0x7f938806ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:35.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:35.666+0000 7f93a5600640 1 -- 172.21.15.152:0/1156293532 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f939c0ae0e0 con 0x7f93a80b4330 2024-04-20T11:29:35.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:35.667+0000 7f93ad000640 1 --2- 172.21.15.152:0/1156293532 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f938806caa0 0x7f938806ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:35.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:35.667+0000 7f93ad000640 1 --2- 172.21.15.152:0/1156293532 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f938806caa0 0x7f938806ef60 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f93a80b4240 tx=0x7f9394009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:35.672 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:35.672+0000 7f93a5600640 1 -- 172.21.15.152:0/1156293532 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f939c03f070 con 0x7f93a80b4330 2024-04-20T11:29:35.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:35.904+0000 7f93b0400640 1 -- 172.21.15.152:0/1156293532 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f93a80689f0 con 0x7f93a80b4330 2024-04-20T11:29:35.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:35.905+0000 7f93a5600640 1 -- 172.21.15.152:0/1156293532 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+5208 (secure 0 0 0) 0x7f939c07c0e0 con 0x7f93a80b4330 2024-04-20T11:29:35.906 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:29:35.907 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4826,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6834/2855025590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2855025590},{"type":"v1","addr":"172.21.15.175:6834","nonce":2855025590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6836/3887716755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":3887716755},{"type":"v1","addr":"172.21.15.152:6836","nonce":3887716755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5712,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/698387399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6835","nonce":698387399},{"type":"v1","addr":"172.21.15.152:6837","nonce":698387399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T11:29:00.276881+0000","modified":"2024-04-20T11:29:03.352174+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5724},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5724":{"gid":5724,"name":"d","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6835/3704251089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6833","nonce":3704251089},{"type":"v1","addr":"172.21.15.175:6835","nonce":3704251089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5724,"qdb_cluster":[5724]},"id":3}]} 2024-04-20T11:29:35.907 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 25 2024-04-20T11:29:35.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:35.908+0000 7f93b0400640 1 -- 172.21.15.152:0/1156293532 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f938806caa0 msgr2=0x7f938806ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:35.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:35.908+0000 7f93b0400640 1 --2- 172.21.15.152:0/1156293532 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f938806caa0 0x7f938806ef60 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f93a80b4240 tx=0x7f9394009040 comp rx=0 tx=0).stop 2024-04-20T11:29:35.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:35.908+0000 7f93b0400640 1 -- 172.21.15.152:0/1156293532 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f93a80b4330 msgr2=0x7f93a80cf6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:35.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:35.908+0000 7f93b0400640 1 --2- 172.21.15.152:0/1156293532 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f93a80b4330 0x7f93a80cf6b0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f939c009870 tx=0x7f939c0046c0 comp rx=0 tx=0).stop 2024-04-20T11:29:35.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:35.909+0000 7f93b0400640 1 -- 172.21.15.152:0/1156293532 shutdown_connections 2024-04-20T11:29:35.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:35.909+0000 7f93b0400640 1 --2- 172.21.15.152:0/1156293532 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f938806caa0 0x7f938806ef60 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:35.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:35.909+0000 7f93b0400640 1 --2- 172.21.15.152:0/1156293532 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f93a815c560 0x7f93a814b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:35.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:35.909+0000 7f93b0400640 1 --2- 172.21.15.152:0/1156293532 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93a80d0810 0x7f93a80c5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:35.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:35.909+0000 7f93b0400640 1 --2- 172.21.15.152:0/1156293532 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f93a80b4330 0x7f93a80cf6b0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:35.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:35.909+0000 7f93b0400640 1 -- 172.21.15.152:0/1156293532 >> 172.21.15.152:0/1156293532 conn(0x7f93a8067cf0 msgr2=0x7f93a8159700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:35.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:35.909+0000 7f93b0400640 1 -- 172.21.15.152:0/1156293532 shutdown_connections 2024-04-20T11:29:35.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:35.909+0000 7f93b0400640 1 -- 172.21.15.152:0/1156293532 wait complete. 2024-04-20T11:29:35.929 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json session ls 2024-04-20T11:29:36.159 INFO:teuthology.orchestra.run.smithi175.stdout:[] 2024-04-20T11:29:36.159 DEBUG:tasks.cephfs.filesystem:_json_asok output [] 2024-04-20T11:29:36.160 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol62000 2024-04-20T11:29:36.369 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.368+0000 7ff7b0e00640 1 Processor -- start 2024-04-20T11:29:36.369 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.369+0000 7ff7b0e00640 1 -- start start 2024-04-20T11:29:36.371 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.370+0000 7ff7b0e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7ff7ac15eb50 0x7ff7ac0d0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:36.371 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.370+0000 7ff7b0e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff7ac163230 0x7ff7ac0d2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:36.371 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.370+0000 7ff7b0e00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7ac0d19c0 con 0x7ff7ac0d1440 2024-04-20T11:29:36.371 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.370+0000 7ff7b0e00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7ff7ac162750 con 0x7ff7ac15eb50 2024-04-20T11:29:36.371 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.370+0000 7ff7b0e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff7ac162a30 con 0x7ff7ac163230 2024-04-20T11:29:36.371 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.370+0000 7ff7ab400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7ff7ac15eb50 0x7ff7ac0d0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:36.371 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.371+0000 7ff7ab400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7ff7ac15eb50 0x7ff7ac0d0f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52936/0 (socket says 172.21.15.152:52936) 2024-04-20T11:29:36.371 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.371+0000 7ff7ab400640 1 -- 172.21.15.152:0/2514161936 learned_addr learned my addr 172.21.15.152:0/2514161936 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:29:36.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.371+0000 7ff7aaa00640 1 --2- 172.21.15.152:0/2514161936 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff7ac163230 0x7ff7ac0d2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:36.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.371+0000 7ff7ab400640 1 -- 172.21.15.152:0/2514161936 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff7ac163230 msgr2=0x7ff7ac0d2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:36.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.371+0000 7ff7ab400640 1 --2- 172.21.15.152:0/2514161936 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff7ac163230 0x7ff7ac0d2e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:36.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.371+0000 7ff7ab400640 1 -- 172.21.15.152:0/2514161936 >> v1:172.21.15.175:6789/0 conn(0x7ff7ac0d1440 legacy=0x7ff7ac15ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:36.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.371+0000 7ff7ab400640 1 -- 172.21.15.152:0/2514161936 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7ac162d10 con 0x7ff7ac15eb50 2024-04-20T11:29:36.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.371+0000 7ff7aaa00640 1 --2- 172.21.15.152:0/2514161936 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff7ac163230 0x7ff7ac0d2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:29:36.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.372+0000 7ff7ab400640 1 --2- 172.21.15.152:0/2514161936 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff7ac15eb50 0x7ff7ac0d0f00 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7ff79c009e50 tx=0x7ff79c02be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=368d44260eb5e03d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:36.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.372+0000 7ff7aa000640 1 -- 172.21.15.152:0/2514161936 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff79c02c9f0 con 0x7ff7ac15eb50 2024-04-20T11:29:36.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.373+0000 7ff7aa000640 1 -- 172.21.15.152:0/2514161936 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff79c02cb90 con 0x7ff7ac15eb50 2024-04-20T11:29:36.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.373+0000 7ff7aa000640 1 -- 172.21.15.152:0/2514161936 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff79c0343f0 con 0x7ff7ac15eb50 2024-04-20T11:29:36.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.373+0000 7ff7b0e00640 1 -- 172.21.15.152:0/2514161936 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff7ac15eb50 msgr2=0x7ff7ac0d0f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:36.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.373+0000 7ff7b0e00640 1 --2- 172.21.15.152:0/2514161936 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff7ac15eb50 0x7ff7ac0d0f00 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7ff79c009e50 tx=0x7ff79c02be50 comp rx=0 tx=0).stop 2024-04-20T11:29:36.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.373+0000 7ff7b0e00640 1 -- 172.21.15.152:0/2514161936 shutdown_connections 2024-04-20T11:29:36.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.373+0000 7ff7b0e00640 1 --2- 172.21.15.152:0/2514161936 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff7ac163230 0x7ff7ac0d2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:36.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.373+0000 7ff7b0e00640 1 --2- 172.21.15.152:0/2514161936 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff7ac15eb50 0x7ff7ac0d0f00 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:36.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.373+0000 7ff7b0e00640 1 -- 172.21.15.152:0/2514161936 >> 172.21.15.152:0/2514161936 conn(0x7ff7ac08bfd0 msgr2=0x7ff7ac0d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:36.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.373+0000 7ff7b0e00640 1 -- 172.21.15.152:0/2514161936 shutdown_connections 2024-04-20T11:29:36.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.373+0000 7ff7b0e00640 1 -- 172.21.15.152:0/2514161936 wait complete. 2024-04-20T11:29:36.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.375+0000 7ff7b0e00640 1 Processor -- start 2024-04-20T11:29:36.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.376+0000 7ff7b0e00640 1 -- start start 2024-04-20T11:29:36.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.377+0000 7ff7b0e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff7ac0d1440 0x7ff7ac0b6a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:36.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.377+0000 7ff7b0e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff7ac15eb50 0x7ff7ac0b7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:36.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.377+0000 7ff7b0e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7ac163230 0x7ff7ac0b9740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:36.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.377+0000 7ff7b0e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7ff7ac0d3550 con 0x7ff7ac0d1440 2024-04-20T11:29:36.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.377+0000 7ff7b0e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff7ac0d33d0 con 0x7ff7ac163230 2024-04-20T11:29:36.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.377+0000 7ff7b0e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff7ac0d36d0 con 0x7ff7ac15eb50 2024-04-20T11:29:36.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.377+0000 7ff7ab400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff7ac0d1440 0x7ff7ac0b6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:36.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.377+0000 7ff7ab400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff7ac0d1440 0x7ff7ac0b6a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52948/0 (socket says 172.21.15.152:52948) 2024-04-20T11:29:36.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.377+0000 7ff7ab400640 1 -- 172.21.15.152:0/3081537128 learned_addr learned my addr 172.21.15.152:0/3081537128 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:29:36.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.378+0000 7ff7ab400640 1 -- 172.21.15.152:0/3081537128 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff7ac15eb50 msgr2=0x7ff7ac0b7ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:36.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.378+0000 7ff7ab400640 1 --2- 172.21.15.152:0/3081537128 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff7ac15eb50 0x7ff7ac0b7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:36.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.378+0000 7ff7ab400640 1 -- 172.21.15.152:0/3081537128 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7ac163230 msgr2=0x7ff7ac0b9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:36.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.378+0000 7ff7ab400640 1 --2- 172.21.15.152:0/3081537128 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7ac163230 0x7ff7ac0b9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:36.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.378+0000 7ff7ab400640 1 -- 172.21.15.152:0/3081537128 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7ac0cd4a0 con 0x7ff7ac0d1440 2024-04-20T11:29:36.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.378+0000 7ff7ab400640 1 --2- 172.21.15.152:0/3081537128 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff7ac0d1440 0x7ff7ac0b6a20 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7ff79c000f80 tx=0x7ff79c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:36.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.378+0000 7ff7a3400640 1 -- 172.21.15.152:0/3081537128 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff79c043020 con 0x7ff7ac0d1440 2024-04-20T11:29:36.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.378+0000 7ff7a3400640 1 -- 172.21.15.152:0/3081537128 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff79c03e450 con 0x7ff7ac0d1440 2024-04-20T11:29:36.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.378+0000 7ff7a3400640 1 -- 172.21.15.152:0/3081537128 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff79c03e840 con 0x7ff7ac0d1440 2024-04-20T11:29:36.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.378+0000 7ff7b0e00640 1 -- 172.21.15.152:0/3081537128 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7ac0cd780 con 0x7ff7ac0d1440 2024-04-20T11:29:36.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.379+0000 7ff7b0e00640 1 -- 172.21.15.152:0/3081537128 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7ac0cdd40 con 0x7ff7ac0d1440 2024-04-20T11:29:36.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.380+0000 7ff7b0e00640 1 -- 172.21.15.152:0/3081537128 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7ac075e60 con 0x7ff7ac0d1440 2024-04-20T11:29:36.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.381+0000 7ff7a3400640 1 -- 172.21.15.152:0/3081537128 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7ff79c046020 con 0x7ff7ac0d1440 2024-04-20T11:29:36.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.381+0000 7ff7a3400640 1 --2- 172.21.15.152:0/3081537128 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff78806caa0 0x7ff78806ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:36.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.382+0000 7ff7a3400640 1 -- 172.21.15.152:0/3081537128 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7ff79c0ae7d0 con 0x7ff7ac0d1440 2024-04-20T11:29:36.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.382+0000 7ff7aaa00640 1 --2- 172.21.15.152:0/3081537128 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff78806caa0 0x7ff78806ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:36.383 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.382+0000 7ff7aaa00640 1 --2- 172.21.15.152:0/3081537128 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff78806caa0 0x7ff78806ef60 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac0d01c0 tx=0x7ff794008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:36.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.387+0000 7ff7a3400640 1 -- 172.21.15.152:0/3081537128 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ff79c04d030 con 0x7ff7ac0d1440 2024-04-20T11:29:36.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.612+0000 7ff7b0e00640 1 -- 172.21.15.152:0/3081537128 --> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvol62000", "target": ["mon-mgr", ""]}) -- 0x7ff7ac0d3de0 con 0x7ff78806caa0 2024-04-20T11:29:36.622 INFO:tasks.ceph.mgr.y.smithi152.stderr:2024-04-20T11:29:36.622+0000 7fd165e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:29:36.622 INFO:tasks.ceph.mgr.y.smithi152.stderr:2024-04-20T11:29:36.622+0000 7fd165e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:29:36.622 INFO:tasks.ceph.mgr.y.smithi152.stderr:2024-04-20T11:29:36.622+0000 7fd165e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:29:36.622 INFO:tasks.ceph.mgr.y.smithi152.stderr:2024-04-20T11:29:36.622+0000 7fd165e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:29:36.623 INFO:tasks.ceph.mgr.y.smithi152.stderr:2024-04-20T11:29:36.622+0000 7fd165e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:29:36.703 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.703+0000 7ff7a3400640 1 -- 172.21.15.152:0/3081537128 <== mgr.4113 v2:172.21.15.152:6832/2285110240 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff7ac0d3de0 con 0x7ff78806caa0 2024-04-20T11:29:36.707 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.706+0000 7ff7b0e00640 1 -- 172.21.15.152:0/3081537128 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff78806caa0 msgr2=0x7ff78806ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:36.707 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.706+0000 7ff7b0e00640 1 --2- 172.21.15.152:0/3081537128 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff78806caa0 0x7ff78806ef60 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac0d01c0 tx=0x7ff794008040 comp rx=0 tx=0).stop 2024-04-20T11:29:36.707 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.706+0000 7ff7b0e00640 1 -- 172.21.15.152:0/3081537128 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff7ac0d1440 msgr2=0x7ff7ac0b6a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:36.707 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.706+0000 7ff7b0e00640 1 --2- 172.21.15.152:0/3081537128 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff7ac0d1440 0x7ff7ac0b6a20 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7ff79c000f80 tx=0x7ff79c004a40 comp rx=0 tx=0).stop 2024-04-20T11:29:36.707 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.706+0000 7ff7b0e00640 1 -- 172.21.15.152:0/3081537128 shutdown_connections 2024-04-20T11:29:36.707 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.706+0000 7ff7b0e00640 1 --2- 172.21.15.152:0/3081537128 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff78806caa0 0x7ff78806ef60 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:36.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.706+0000 7ff7b0e00640 1 --2- 172.21.15.152:0/3081537128 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7ac163230 0x7ff7ac0b9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:36.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.706+0000 7ff7b0e00640 1 --2- 172.21.15.152:0/3081537128 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff7ac15eb50 0x7ff7ac0b7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:36.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.706+0000 7ff7b0e00640 1 --2- 172.21.15.152:0/3081537128 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff7ac0d1440 0x7ff7ac0b6a20 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:36.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.706+0000 7ff7b0e00640 1 -- 172.21.15.152:0/3081537128 >> 172.21.15.152:0/3081537128 conn(0x7ff7ac08bfd0 msgr2=0x7ff7ac0b2cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:36.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.706+0000 7ff7b0e00640 1 -- 172.21.15.152:0/3081537128 shutdown_connections 2024-04-20T11:29:36.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.706+0000 7ff7b0e00640 1 -- 172.21.15.152:0/3081537128 wait complete. 2024-04-20T11:29:36.726 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:29:36.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.916+0000 7f0e2f600640 1 Processor -- start 2024-04-20T11:29:36.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.916+0000 7f0e2f600640 1 -- start start 2024-04-20T11:29:36.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.917+0000 7f0e2f600640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f0e28145f60 0x7f0e2806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:36.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.918+0000 7f0e2f600640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e28146340 con 0x7f0e280cd0d0 2024-04-20T11:29:36.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.918+0000 7f0e2f600640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e2806f3c0 con 0x7f0e280726f0 2024-04-20T11:29:36.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.918+0000 7f0e2f600640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f0e2806e9f0 con 0x7f0e28145f60 2024-04-20T11:29:36.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.918+0000 7f0e1fe00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f0e280cd0d0 0x7f0e28071fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:36860/0 (socket says 172.21.15.152:36860) 2024-04-20T11:29:36.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.918+0000 7f0e1fe00640 1 -- 172.21.15.152:0/3769559198 learned_addr learned my addr 172.21.15.152:0/3769559198 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:29:36.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.918+0000 7f0e2cc00640 1 --2- 172.21.15.152:0/3769559198 >> v2:172.21.15.175:3300/0 conn(0x7f0e28145f60 0x7f0e2806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:36.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.918+0000 7f0e27e00640 1 -- 172.21.15.152:0/3769559198 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3386728855 0 0) 0x7f0e28146340 con 0x7f0e280cd0d0 2024-04-20T11:29:36.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.919+0000 7f0e27e00640 1 -- 172.21.15.152:0/3769559198 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0e08003520 con 0x7f0e280cd0d0 2024-04-20T11:29:36.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.919+0000 7f0e27e00640 1 -- 172.21.15.152:0/3769559198 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1650822639 0 0) 0x7f0e08003520 con 0x7f0e280cd0d0 2024-04-20T11:29:36.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.919+0000 7f0e27e00640 1 -- 172.21.15.152:0/3769559198 >> v2:172.21.15.175:3300/0 conn(0x7f0e28145f60 msgr2=0x7f0e2806ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:36.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.919+0000 7f0e27e00640 1 --2- 172.21.15.152:0/3769559198 >> v2:172.21.15.175:3300/0 conn(0x7f0e28145f60 0x7f0e2806ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:36.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.919+0000 7f0e27e00640 1 -- 172.21.15.152:0/3769559198 >> v1:172.21.15.175:6789/0 conn(0x7f0e280726f0 legacy=0x7f0e280cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:36.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.919+0000 7f0e27e00640 1 -- 172.21.15.152:0/3769559198 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e28072d50 con 0x7f0e280cd0d0 2024-04-20T11:29:36.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.919+0000 7f0e27e00640 1 -- 172.21.15.152:0/3769559198 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f0e18001670 con 0x7f0e280cd0d0 2024-04-20T11:29:36.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.919+0000 7f0e27e00640 1 -- 172.21.15.152:0/3769559198 >> v1:172.21.15.152:6789/0 conn(0x7f0e280cd0d0 legacy=0x7f0e28071fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:36.922 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.919+0000 7f0e27e00640 1 --2- 172.21.15.152:0/3769559198 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e08003a20 0x7f0e08003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:36.922 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.919+0000 7f0e27e00640 1 --2- 172.21.15.152:0/3769559198 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0e08004800 0x7f0e08004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:36.922 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.919+0000 7f0e27e00640 1 --2- 172.21.15.152:0/3769559198 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0e080051c0 0x7f0e0800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:36.922 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.919+0000 7f0e27e00640 1 -- 172.21.15.152:0/3769559198 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f0e2806e9f0 con 0x7f0e080051c0 2024-04-20T11:29:36.922 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.919+0000 7f0e27e00640 1 -- 172.21.15.152:0/3769559198 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0e0800c0f0 con 0x7f0e08003a20 2024-04-20T11:29:36.922 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.919+0000 7f0e27e00640 1 -- 172.21.15.152:0/3769559198 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0e0800c2f0 con 0x7f0e08004800 2024-04-20T11:29:36.922 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.919+0000 7f0e2d600640 1 --2- 172.21.15.152:0/3769559198 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0e080051c0 0x7f0e0800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:36.922 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.920+0000 7f0e2cc00640 1 --2- 172.21.15.152:0/3769559198 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e08003a20 0x7f0e08003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:36.922 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.920+0000 7f0e1fe00640 1 --2- 172.21.15.152:0/3769559198 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0e08004800 0x7f0e08004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:36.922 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.920+0000 7f0e2d600640 1 -- 172.21.15.152:0/3769559198 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0e08004800 msgr2=0x7f0e08004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:36.922 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.920+0000 7f0e2d600640 1 --2- 172.21.15.152:0/3769559198 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0e08004800 0x7f0e08004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:36.922 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.920+0000 7f0e2d600640 1 -- 172.21.15.152:0/3769559198 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e08003a20 msgr2=0x7f0e08003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:36.922 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.920+0000 7f0e2d600640 1 --2- 172.21.15.152:0/3769559198 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e08003a20 0x7f0e08003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:36.922 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.920+0000 7f0e2d600640 1 -- 172.21.15.152:0/3769559198 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e08003520 con 0x7f0e080051c0 2024-04-20T11:29:36.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.920+0000 7f0e1fe00640 1 --2- 172.21.15.152:0/3769559198 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0e08004800 0x7f0e08004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:29:36.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.921+0000 7f0e2d600640 1 -- 172.21.15.152:0/3769559198 reap_dead start 2024-04-20T11:29:36.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.921+0000 7f0e2d600640 1 --2- 172.21.15.152:0/3769559198 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0e080051c0 0x7f0e0800b9c0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f0e20001e50 tx=0x7f0e08005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86e7e66aa9f37035 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:36.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.921+0000 7f0e27e00640 1 -- 172.21.15.152:0/3769559198 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e20001a70 con 0x7f0e080051c0 2024-04-20T11:29:36.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.921+0000 7f0e27e00640 1 -- 172.21.15.152:0/3769559198 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e20003bb0 con 0x7f0e080051c0 2024-04-20T11:29:36.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.921+0000 7f0e27e00640 1 -- 172.21.15.152:0/3769559198 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e200050d0 con 0x7f0e080051c0 2024-04-20T11:29:36.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.922+0000 7f0e2f600640 1 -- 172.21.15.152:0/3769559198 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0e080051c0 msgr2=0x7f0e0800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:36.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.922+0000 7f0e2f600640 1 --2- 172.21.15.152:0/3769559198 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0e080051c0 0x7f0e0800b9c0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f0e20001e50 tx=0x7f0e08005830 comp rx=0 tx=0).stop 2024-04-20T11:29:36.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.922+0000 7f0e2f600640 1 -- 172.21.15.152:0/3769559198 shutdown_connections 2024-04-20T11:29:36.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.922+0000 7f0e2f600640 1 --2- 172.21.15.152:0/3769559198 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0e080051c0 0x7f0e0800b9c0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:36.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.922+0000 7f0e2f600640 1 -- 172.21.15.152:0/3769559198 >> 172.21.15.152:0/3769559198 conn(0x7f0e28067cf0 msgr2=0x7f0e2806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:36.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.922+0000 7f0e2f600640 1 -- 172.21.15.152:0/3769559198 shutdown_connections 2024-04-20T11:29:36.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.922+0000 7f0e2f600640 1 -- 172.21.15.152:0/3769559198 wait complete. 2024-04-20T11:29:36.925 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.924+0000 7f0e2f600640 1 Processor -- start 2024-04-20T11:29:36.925 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.924+0000 7f0e2f600640 1 -- start start 2024-04-20T11:29:36.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.925+0000 7f0e2f600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0e080051c0 0x7f0e280b4830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:36.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.925+0000 7f0e2f600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e281444b0 0x7f0e28144910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:36.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.925+0000 7f0e2f600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0e28159bb0 0x7f0e28159ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:36.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.925+0000 7f0e2f600640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f0e2806e680 con 0x7f0e080051c0 2024-04-20T11:29:36.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.925+0000 7f0e2f600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0e2806e3a0 con 0x7f0e281444b0 2024-04-20T11:29:36.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.925+0000 7f0e2f600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0e2806ee80 con 0x7f0e28159bb0 2024-04-20T11:29:36.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.925+0000 7f0e2cc00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0e080051c0 0x7f0e280b4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:36.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.925+0000 7f0e2cc00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0e080051c0 0x7f0e280b4830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52970/0 (socket says 172.21.15.152:52970) 2024-04-20T11:29:36.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.925+0000 7f0e2cc00640 1 -- 172.21.15.152:0/3189933246 learned_addr learned my addr 172.21.15.152:0/3189933246 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:29:36.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.925+0000 7f0e2cc00640 1 -- 172.21.15.152:0/3189933246 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0e28159bb0 msgr2=0x7f0e28159ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:36.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.925+0000 7f0e2cc00640 1 --2- 172.21.15.152:0/3189933246 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0e28159bb0 0x7f0e28159ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:36.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.925+0000 7f0e2cc00640 1 -- 172.21.15.152:0/3189933246 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e281444b0 msgr2=0x7f0e28144910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:36.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.925+0000 7f0e2cc00640 1 --2- 172.21.15.152:0/3189933246 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e281444b0 0x7f0e28144910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:36.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.925+0000 7f0e2cc00640 1 -- 172.21.15.152:0/3189933246 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e2815a530 con 0x7f0e080051c0 2024-04-20T11:29:36.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.926+0000 7f0e2cc00640 1 --2- 172.21.15.152:0/3189933246 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0e080051c0 0x7f0e280b4830 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f0e10011540 tx=0x7f0e10011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:36.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.926+0000 7f0e25600640 1 -- 172.21.15.152:0/3189933246 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e10015930 con 0x7f0e080051c0 2024-04-20T11:29:36.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.926+0000 7f0e25600640 1 -- 172.21.15.152:0/3189933246 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e10015ad0 con 0x7f0e080051c0 2024-04-20T11:29:36.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.926+0000 7f0e25600640 1 -- 172.21.15.152:0/3189933246 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e1000f460 con 0x7f0e080051c0 2024-04-20T11:29:36.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.926+0000 7f0e2f600640 1 -- 172.21.15.152:0/3189933246 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e28141c60 con 0x7f0e080051c0 2024-04-20T11:29:36.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.926+0000 7f0e2f600640 1 -- 172.21.15.152:0/3189933246 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e2815de50 con 0x7f0e080051c0 2024-04-20T11:29:36.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.927+0000 7f0e2f600640 1 -- 172.21.15.152:0/3189933246 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e280b6410 con 0x7f0e080051c0 2024-04-20T11:29:36.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.928+0000 7f0e25600640 1 -- 172.21.15.152:0/3189933246 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f0e1000f600 con 0x7f0e080051c0 2024-04-20T11:29:36.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.929+0000 7f0e25600640 1 --2- 172.21.15.152:0/3189933246 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f0e0406caa0 0x7f0e0406ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:36.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.929+0000 7f0e25600640 1 -- 172.21.15.152:0/3189933246 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f0e1008c890 con 0x7f0e080051c0 2024-04-20T11:29:36.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.929+0000 7f0e1fe00640 1 --2- 172.21.15.152:0/3189933246 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f0e0406caa0 0x7f0e0406ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:36.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.930+0000 7f0e1fe00640 1 --2- 172.21.15.152:0/3189933246 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f0e0406caa0 0x7f0e0406ef60 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f0e18001eb0 tx=0x7f0e18009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:36.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:36.933+0000 7f0e25600640 1 -- 172.21.15.152:0/3189933246 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f0e280b6410 con 0x7f0e080051c0 2024-04-20T11:29:37.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:37.172+0000 7f0e2f600640 1 -- 172.21.15.152:0/3189933246 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0e280689f0 con 0x7f0e080051c0 2024-04-20T11:29:37.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:37.173+0000 7f0e25600640 1 -- 172.21.15.152:0/3189933246 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+5208 (secure 0 0 0) 0x7f0e10019030 con 0x7f0e080051c0 2024-04-20T11:29:37.174 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:29:37.174 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4826,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6834/2855025590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2855025590},{"type":"v1","addr":"172.21.15.175:6834","nonce":2855025590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6836/3887716755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":3887716755},{"type":"v1","addr":"172.21.15.152:6836","nonce":3887716755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5712,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/698387399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6835","nonce":698387399},{"type":"v1","addr":"172.21.15.152:6837","nonce":698387399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T11:29:00.276881+0000","modified":"2024-04-20T11:29:03.352174+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5724},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5724":{"gid":5724,"name":"d","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6835/3704251089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6833","nonce":3704251089},{"type":"v1","addr":"172.21.15.175:6835","nonce":3704251089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5724,"qdb_cluster":[5724]},"id":3}]} 2024-04-20T11:29:37.175 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 25 2024-04-20T11:29:37.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:37.176+0000 7f0e2f600640 1 -- 172.21.15.152:0/3189933246 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f0e0406caa0 msgr2=0x7f0e0406ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:37.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:37.176+0000 7f0e2f600640 1 --2- 172.21.15.152:0/3189933246 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f0e0406caa0 0x7f0e0406ef60 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f0e18001eb0 tx=0x7f0e18009040 comp rx=0 tx=0).stop 2024-04-20T11:29:37.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:37.176+0000 7f0e2f600640 1 -- 172.21.15.152:0/3189933246 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0e080051c0 msgr2=0x7f0e280b4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:37.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:37.176+0000 7f0e2f600640 1 --2- 172.21.15.152:0/3189933246 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0e080051c0 0x7f0e280b4830 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f0e10011540 tx=0x7f0e10011a00 comp rx=0 tx=0).stop 2024-04-20T11:29:37.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:37.176+0000 7f0e2f600640 1 -- 172.21.15.152:0/3189933246 shutdown_connections 2024-04-20T11:29:37.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:37.176+0000 7f0e2f600640 1 --2- 172.21.15.152:0/3189933246 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f0e0406caa0 0x7f0e0406ef60 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:37.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:37.176+0000 7f0e2f600640 1 --2- 172.21.15.152:0/3189933246 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0e28159bb0 0x7f0e28159ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:37.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:37.176+0000 7f0e2f600640 1 --2- 172.21.15.152:0/3189933246 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e281444b0 0x7f0e28144910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:37.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:37.176+0000 7f0e2f600640 1 --2- 172.21.15.152:0/3189933246 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0e080051c0 0x7f0e280b4830 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:37.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:37.176+0000 7f0e2f600640 1 -- 172.21.15.152:0/3189933246 >> 172.21.15.152:0/3189933246 conn(0x7f0e28067cf0 msgr2=0x7f0e28160170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:37.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:37.177+0000 7f0e2f600640 1 -- 172.21.15.152:0/3189933246 shutdown_connections 2024-04-20T11:29:37.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:37.177+0000 7f0e2f600640 1 -- 172.21.15.152:0/3189933246 wait complete. 2024-04-20T11:29:37.197 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json session ls 2024-04-20T11:29:37.432 INFO:teuthology.orchestra.run.smithi175.stdout:[{"id":6069,"entity":{"name":{"type":"client","num":6069},"addr":{"type":"any","addr":"172.21.15.152:0","nonce":83509866}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":14,"uptime":0.79098775700000001,"requests_in_flight":2,"num_completed_requests":0,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":21.961306249031054,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.6069 172.21.15.152:0/83509866","completed_requests":[],"prealloc_inos":[{"start":"0x10000000005","length":996}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"a8c84f5997950fd7d008204839253291d52e6eda","ceph_version":"ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)","entity_id":"y","hostname":"smithi152","pid":"22712","root":"/"}}] 2024-04-20T11:29:37.433 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "a8c84f5997950fd7d008204839253291d52e6eda", "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi152", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "22712", "root": "/" }, "completed_requests": [], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.152:0", "nonce": 83509866, "type": "any" }, "name": { "num": 6069, "type": "client" } }, "id": 6069, "inst": "client.6069 172.21.15.152:0/83509866", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 0, "num_leases": 1, "prealloc_inos": [ { "length": 996, "start": "0x10000000005" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 14, "requests_in_flight": 2, "session_cache_liveness": { "halflife": 300, "value": 21.961306249031054 }, "state": "open", "uptime": 0.790987757 } ] 2024-04-20T11:29:37.433 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:29:37.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:37.642+0000 7fd92de00640 1 Processor -- start 2024-04-20T11:29:37.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:37.642+0000 7fd92de00640 1 -- start start 2024-04-20T11:29:37.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:37.644+0000 7fd92de00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd92815d780 0x7fd928160110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:37.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:37.644+0000 7fd92de00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9281539e0 con 0x7fd92815e820 2024-04-20T11:29:37.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:37.644+0000 7fd92de00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd928160fd0 con 0x7fd9280b6000 2024-04-20T11:29:37.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:37.644+0000 7fd92de00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd92815e390 con 0x7fd92815d780 2024-04-20T11:29:37.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:37.644+0000 7fd92d400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fd92815e820 0x7fd92815ecd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:36862/0 (socket says 172.21.15.152:36862) 2024-04-20T11:29:37.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:37.644+0000 7fd92d400640 1 -- 172.21.15.152:0/49998051 learned_addr learned my addr 172.21.15.152:0/49998051 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:29:37.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:37.645+0000 7fd927e00640 1 --2- 172.21.15.152:0/49998051 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd92815d780 0x7fd928160110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:37.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:37.645+0000 7fd927400640 1 -- 172.21.15.152:0/49998051 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1983241244 0 0) 0x7fd9281539e0 con 0x7fd92815e820 2024-04-20T11:29:37.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:37.645+0000 7fd927400640 1 -- 172.21.15.152:0/49998051 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd90c003520 con 0x7fd92815e820 2024-04-20T11:29:37.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:37.645+0000 7fd927400640 1 -- 172.21.15.152:0/49998051 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4035701127 0 0) 0x7fd90c003520 con 0x7fd92815e820 2024-04-20T11:29:37.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:37.645+0000 7fd927400640 1 -- 172.21.15.152:0/49998051 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd92815d780 msgr2=0x7fd928160110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:37.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:37.645+0000 7fd927400640 1 --2- 172.21.15.152:0/49998051 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd92815d780 0x7fd928160110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:37.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:37.645+0000 7fd927400640 1 -- 172.21.15.152:0/49998051 >> v1:172.21.15.175:6789/0 conn(0x7fd9280b6000 legacy=0x7fd9280b63e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:37.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:37.645+0000 7fd927400640 1 -- 172.21.15.152:0/49998051 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd928160a10 con 0x7fd92815e820 2024-04-20T11:29:37.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:37.645+0000 7fd927400640 1 -- 172.21.15.152:0/49998051 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7fd9200019f0 con 0x7fd92815e820 2024-04-20T11:29:37.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:37.645+0000 7fd927400640 1 -- 172.21.15.152:0/49998051 >> v1:172.21.15.152:6789/0 conn(0x7fd92815e820 legacy=0x7fd92815ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:37.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:37.645+0000 7fd927400640 1 --2- 172.21.15.152:0/49998051 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd90c0039e0 0x7fd90c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:37.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:37.645+0000 7fd927400640 1 --2- 172.21.15.152:0/49998051 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd92815d780 0x7fd90c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:37.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:37.645+0000 7fd927400640 1 --2- 172.21.15.152:0/49998051 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd90c004d40 0x7fd90c0051a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:37.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:37.645+0000 7fd927400640 1 -- 172.21.15.152:0/49998051 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fd92815e390 con 0x7fd90c004d40 2024-04-20T11:29:37.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:37.645+0000 7fd927400640 1 -- 172.21.15.152:0/49998051 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd90c00bbd0 con 0x7fd90c0039e0 2024-04-20T11:29:37.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:37.645+0000 7fd927400640 1 -- 172.21.15.152:0/49998051 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd90c00bdd0 con 0x7fd92815d780 2024-04-20T11:29:37.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:37.646+0000 7fd92d400640 1 --2- 172.21.15.152:0/49998051 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd90c004d40 0x7fd90c0051a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:37.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:37.646+0000 7fd92ca00640 1 --2- 172.21.15.152:0/49998051 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd90c0039e0 0x7fd90c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:37.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:37.646+0000 7fd92d400640 1 -- 172.21.15.152:0/49998051 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd92815d780 msgr2=0x7fd90c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:37.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:37.646+0000 7fd92d400640 1 --2- 172.21.15.152:0/49998051 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd92815d780 0x7fd90c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:37.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:37.646+0000 7fd92d400640 1 -- 172.21.15.152:0/49998051 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd90c0039e0 msgr2=0x7fd90c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:37.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:37.646+0000 7fd92d400640 1 --2- 172.21.15.152:0/49998051 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd90c0039e0 0x7fd90c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:37.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:37.646+0000 7fd92d400640 1 -- 172.21.15.152:0/49998051 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd90c003520 con 0x7fd90c004d40 2024-04-20T11:29:37.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:37.647+0000 7fd92d400640 1 --2- 172.21.15.152:0/49998051 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd90c004d40 0x7fd90c0051a0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fd92000a770 tx=0x7fd90c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=805740433f1d2b8e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:37.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:37.647+0000 7fd927400640 1 -- 172.21.15.152:0/49998051 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd92000a390 con 0x7fd90c004d40 2024-04-20T11:29:37.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:37.647+0000 7fd927400640 1 -- 172.21.15.152:0/49998051 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd920034650 con 0x7fd90c004d40 2024-04-20T11:29:37.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:37.647+0000 7fd927400640 1 -- 172.21.15.152:0/49998051 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd920034ac0 con 0x7fd90c004d40 2024-04-20T11:29:37.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:37.648+0000 7fd92de00640 1 -- 172.21.15.152:0/49998051 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd90c004d40 msgr2=0x7fd90c0051a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:37.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:37.648+0000 7fd92de00640 1 --2- 172.21.15.152:0/49998051 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd90c004d40 0x7fd90c0051a0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fd92000a770 tx=0x7fd90c005830 comp rx=0 tx=0).stop 2024-04-20T11:29:37.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:37.648+0000 7fd92d400640 1 -- 172.21.15.152:0/49998051 reap_dead start 2024-04-20T11:29:37.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:37.648+0000 7fd92de00640 1 -- 172.21.15.152:0/49998051 shutdown_connections 2024-04-20T11:29:37.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:37.648+0000 7fd92de00640 1 -- 172.21.15.152:0/49998051 >> 172.21.15.152:0/49998051 conn(0x7fd928073340 msgr2=0x7fd92806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:37.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:37.648+0000 7fd92de00640 1 -- 172.21.15.152:0/49998051 shutdown_connections 2024-04-20T11:29:37.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:37.648+0000 7fd92de00640 1 -- 172.21.15.152:0/49998051 wait complete. 2024-04-20T11:29:37.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:37.650+0000 7fd92de00640 1 Processor -- start 2024-04-20T11:29:37.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:37.650+0000 7fd92de00640 1 -- start start 2024-04-20T11:29:37.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:37.651+0000 7fd92de00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd9280b5140 0x7fd928154d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:37.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:37.651+0000 7fd92de00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd9281f5760 0x7fd928155270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:37.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:37.651+0000 7fd92de00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd9280b5520 0x7fd9281539e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:37.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:37.651+0000 7fd92de00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fd928160110 con 0x7fd9281f5760 2024-04-20T11:29:37.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:37.651+0000 7fd92de00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd92815e020 con 0x7fd9280b5140 2024-04-20T11:29:37.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:37.651+0000 7fd92de00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd928160290 con 0x7fd9280b5520 2024-04-20T11:29:37.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:37.651+0000 7fd927e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd9281f5760 0x7fd928155270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:37.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:37.651+0000 7fd927e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd9281f5760 0x7fd928155270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52978/0 (socket says 172.21.15.152:52978) 2024-04-20T11:29:37.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:37.651+0000 7fd927e00640 1 -- 172.21.15.152:0/1839328756 learned_addr learned my addr 172.21.15.152:0/1839328756 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:29:37.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:37.652+0000 7fd92ca00640 1 --2- 172.21.15.152:0/1839328756 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd9280b5140 0x7fd928154d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:37.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:37.652+0000 7fd92d400640 1 --2- 172.21.15.152:0/1839328756 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd9280b5520 0x7fd9281539e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:37.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:37.652+0000 7fd927e00640 1 -- 172.21.15.152:0/1839328756 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd9280b5520 msgr2=0x7fd9281539e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:37.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:37.652+0000 7fd927e00640 1 --2- 172.21.15.152:0/1839328756 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd9280b5520 0x7fd9281539e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:37.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:37.652+0000 7fd927e00640 1 -- 172.21.15.152:0/1839328756 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd9280b5140 msgr2=0x7fd928154d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:37.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:37.652+0000 7fd927e00640 1 --2- 172.21.15.152:0/1839328756 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd9280b5140 0x7fd928154d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:37.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:37.652+0000 7fd927e00640 1 -- 172.21.15.152:0/1839328756 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9281f4cc0 con 0x7fd9281f5760 2024-04-20T11:29:37.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:37.652+0000 7fd92ca00640 1 --2- 172.21.15.152:0/1839328756 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd9280b5140 0x7fd928154d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:29:37.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:37.652+0000 7fd92d400640 1 --2- 172.21.15.152:0/1839328756 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd9280b5520 0x7fd9281539e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:29:37.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:37.652+0000 7fd927e00640 1 --2- 172.21.15.152:0/1839328756 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd9281f5760 0x7fd928155270 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fd91400e850 tx=0x7fd91400ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:37.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:37.653+0000 7fd924c00640 1 -- 172.21.15.152:0/1839328756 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd914017070 con 0x7fd9281f5760 2024-04-20T11:29:37.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:37.653+0000 7fd924c00640 1 -- 172.21.15.152:0/1839328756 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd91400c940 con 0x7fd9281f5760 2024-04-20T11:29:37.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:37.653+0000 7fd924c00640 1 -- 172.21.15.152:0/1839328756 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd91400cd10 con 0x7fd9281f5760 2024-04-20T11:29:37.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:37.653+0000 7fd92de00640 1 -- 172.21.15.152:0/1839328756 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd928154680 con 0x7fd9281f5760 2024-04-20T11:29:37.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:37.653+0000 7fd92de00640 1 -- 172.21.15.152:0/1839328756 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd928153f20 con 0x7fd9281f5760 2024-04-20T11:29:37.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:37.654+0000 7fd92de00640 1 -- 172.21.15.152:0/1839328756 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd928075e60 con 0x7fd9281f5760 2024-04-20T11:29:37.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:37.655+0000 7fd924c00640 1 -- 172.21.15.152:0/1839328756 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7fd91401a070 con 0x7fd9281f5760 2024-04-20T11:29:37.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:37.656+0000 7fd924c00640 1 --2- 172.21.15.152:0/1839328756 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fd90806caa0 0x7fd90806ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:37.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:37.656+0000 7fd924c00640 1 -- 172.21.15.152:0/1839328756 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7fd91408bcc0 con 0x7fd9281f5760 2024-04-20T11:29:37.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:37.656+0000 7fd92ca00640 1 --2- 172.21.15.152:0/1839328756 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fd90806caa0 0x7fd90806ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:37.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:37.657+0000 7fd92ca00640 1 --2- 172.21.15.152:0/1839328756 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fd90806caa0 0x7fd90806ef60 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fd9180016b0 tx=0x7fd918009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:37.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:37.661+0000 7fd924c00640 1 -- 172.21.15.152:0/1839328756 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fd914092080 con 0x7fd9281f5760 2024-04-20T11:29:37.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:37.899+0000 7fd92de00640 1 -- 172.21.15.152:0/1839328756 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd9280679d0 con 0x7fd9281f5760 2024-04-20T11:29:37.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:37.900+0000 7fd924c00640 1 -- 172.21.15.152:0/1839328756 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+5208 (secure 0 0 0) 0x7fd914059e60 con 0x7fd9281f5760 2024-04-20T11:29:37.901 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:29:37.901 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4826,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6834/2855025590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2855025590},{"type":"v1","addr":"172.21.15.175:6834","nonce":2855025590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6836/3887716755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":3887716755},{"type":"v1","addr":"172.21.15.152:6836","nonce":3887716755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5712,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/698387399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6835","nonce":698387399},{"type":"v1","addr":"172.21.15.152:6837","nonce":698387399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T11:29:00.276881+0000","modified":"2024-04-20T11:29:03.352174+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5724},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5724":{"gid":5724,"name":"d","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6835/3704251089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6833","nonce":3704251089},{"type":"v1","addr":"172.21.15.175:6835","nonce":3704251089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5724,"qdb_cluster":[5724]},"id":3}]} 2024-04-20T11:29:37.901 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 25 2024-04-20T11:29:37.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:37.903+0000 7fd92de00640 1 -- 172.21.15.152:0/1839328756 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fd90806caa0 msgr2=0x7fd90806ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:37.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:37.903+0000 7fd92de00640 1 --2- 172.21.15.152:0/1839328756 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fd90806caa0 0x7fd90806ef60 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fd9180016b0 tx=0x7fd918009040 comp rx=0 tx=0).stop 2024-04-20T11:29:37.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:37.903+0000 7fd92de00640 1 -- 172.21.15.152:0/1839328756 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd9281f5760 msgr2=0x7fd928155270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:37.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:37.903+0000 7fd92de00640 1 --2- 172.21.15.152:0/1839328756 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd9281f5760 0x7fd928155270 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fd91400e850 tx=0x7fd91400ed10 comp rx=0 tx=0).stop 2024-04-20T11:29:37.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:37.904+0000 7fd92de00640 1 -- 172.21.15.152:0/1839328756 shutdown_connections 2024-04-20T11:29:37.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:37.904+0000 7fd92de00640 1 --2- 172.21.15.152:0/1839328756 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fd90806caa0 0x7fd90806ef60 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:37.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:37.904+0000 7fd92de00640 1 --2- 172.21.15.152:0/1839328756 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd9280b5520 0x7fd9281539e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:37.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:37.904+0000 7fd92de00640 1 --2- 172.21.15.152:0/1839328756 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd9281f5760 0x7fd928155270 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:37.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:37.904+0000 7fd92de00640 1 --2- 172.21.15.152:0/1839328756 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd9280b5140 0x7fd928154d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:37.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:37.904+0000 7fd92de00640 1 -- 172.21.15.152:0/1839328756 >> 172.21.15.152:0/1839328756 conn(0x7fd928073340 msgr2=0x7fd92814efb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:37.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:37.904+0000 7fd92de00640 1 -- 172.21.15.152:0/1839328756 shutdown_connections 2024-04-20T11:29:37.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:37.904+0000 7fd92de00640 1 -- 172.21.15.152:0/1839328756 wait complete. 2024-04-20T11:29:37.923 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json session ls 2024-04-20T11:29:38.161 INFO:teuthology.orchestra.run.smithi175.stdout:[{"id":6069,"entity":{"name":{"type":"client","num":6069},"addr":{"type":"any","addr":"172.21.15.152:0","nonce":83509866}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":14,"uptime":1.51897649,"requests_in_flight":2,"num_completed_requests":0,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":21.924398136211519,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.6069 172.21.15.152:0/83509866","completed_requests":[],"prealloc_inos":[{"start":"0x10000000005","length":996}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"a8c84f5997950fd7d008204839253291d52e6eda","ceph_version":"ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)","entity_id":"y","hostname":"smithi152","pid":"22712","root":"/"}}] 2024-04-20T11:29:38.161 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "a8c84f5997950fd7d008204839253291d52e6eda", "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi152", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "22712", "root": "/" }, "completed_requests": [], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.152:0", "nonce": 83509866, "type": "any" }, "name": { "num": 6069, "type": "client" } }, "id": 6069, "inst": "client.6069 172.21.15.152:0/83509866", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 0, "num_leases": 1, "prealloc_inos": [ { "length": 996, "start": "0x10000000005" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 14, "requests_in_flight": 2, "session_cache_liveness": { "halflife": 300, "value": 21.92439813621152 }, "state": "open", "uptime": 1.51897649 } ] 2024-04-20T11:29:38.161 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-04-20T11:29:43.162 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:29:43.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:43.374+0000 7f6e17c00640 1 Processor -- start 2024-04-20T11:29:43.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:43.375+0000 7f6e17c00640 1 -- start start 2024-04-20T11:29:43.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:43.376+0000 7f6e17c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e100b4500 0x7f6e100d0100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:43.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:43.376+0000 7f6e17c00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f6e100d0640 0x7f6e100ada90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:43.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:43.376+0000 7f6e17c00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e1015fd20 con 0x7f6e1015c560 2024-04-20T11:29:43.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:43.376+0000 7f6e17c00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f6e1015ef60 con 0x7f6e100d0640 2024-04-20T11:29:43.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:43.376+0000 7f6e17c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6e1015f240 con 0x7f6e100b4500 2024-04-20T11:29:43.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:43.377+0000 7f6e15200640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f6e1015c560 0x7f6e100b3df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:36866/0 (socket says 172.21.15.152:36866) 2024-04-20T11:29:43.377 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:43.377+0000 7f6e15200640 1 -- 172.21.15.152:0/2284038698 learned_addr learned my addr 172.21.15.152:0/2284038698 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:29:43.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:43.377+0000 7f6e0fe00640 1 --2- 172.21.15.152:0/2284038698 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e100b4500 0x7f6e100d0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:43.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:43.377+0000 7f6e15c00640 1 --2- 172.21.15.152:0/2284038698 >> v2:172.21.15.175:3300/0 conn(0x7f6e100d0640 0x7f6e100ada90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:43.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:43.377+0000 7f6e0f400640 1 -- 172.21.15.152:0/2284038698 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2609181688 0 0) 0x7f6e1015fd20 con 0x7f6e1015c560 2024-04-20T11:29:43.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:43.377+0000 7f6e0f400640 1 -- 172.21.15.152:0/2284038698 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6df4003520 con 0x7f6e1015c560 2024-04-20T11:29:43.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:43.377+0000 7f6e0f400640 1 -- 172.21.15.152:0/2284038698 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2014343274 0 0) 0x7f6df4003520 con 0x7f6e1015c560 2024-04-20T11:29:43.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:43.378+0000 7f6e0f400640 1 -- 172.21.15.152:0/2284038698 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e100b4500 msgr2=0x7f6e100d0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:43.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:43.378+0000 7f6e0f400640 1 --2- 172.21.15.152:0/2284038698 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e100b4500 0x7f6e100d0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:43.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:43.378+0000 7f6e0f400640 1 -- 172.21.15.152:0/2284038698 >> v2:172.21.15.175:3300/0 conn(0x7f6e100d0640 msgr2=0x7f6e100ada90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:43.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:43.378+0000 7f6e0f400640 1 --2- 172.21.15.152:0/2284038698 >> v2:172.21.15.175:3300/0 conn(0x7f6e100d0640 0x7f6e100ada90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:43.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:43.378+0000 7f6e0f400640 1 -- 172.21.15.152:0/2284038698 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e1015f520 con 0x7f6e1015c560 2024-04-20T11:29:43.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:43.378+0000 7f6e0f400640 1 -- 172.21.15.152:0/2284038698 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f6df8001670 con 0x7f6e1015c560 2024-04-20T11:29:43.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:43.378+0000 7f6e0f400640 1 -- 172.21.15.152:0/2284038698 >> v1:172.21.15.152:6789/0 conn(0x7f6e1015c560 legacy=0x7f6e100b3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:43.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:43.378+0000 7f6e0f400640 1 --2- 172.21.15.152:0/2284038698 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e100b4500 0x7f6df4003a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:43.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:43.378+0000 7f6e0f400640 1 --2- 172.21.15.152:0/2284038698 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6df4003ff0 0x7f6df4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:43.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:43.378+0000 7f6e0f400640 1 --2- 172.21.15.152:0/2284038698 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6df4004d40 0x7f6df40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:43.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:43.378+0000 7f6e0f400640 1 -- 172.21.15.152:0/2284038698 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f6e1015ef60 con 0x7f6df4003ff0 2024-04-20T11:29:43.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:43.378+0000 7f6e0f400640 1 -- 172.21.15.152:0/2284038698 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6e1015f240 con 0x7f6df4004d40 2024-04-20T11:29:43.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:43.378+0000 7f6e0f400640 1 -- 172.21.15.152:0/2284038698 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6df400bac0 con 0x7f6e100b4500 2024-04-20T11:29:43.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:43.378+0000 7f6e0fe00640 1 --2- 172.21.15.152:0/2284038698 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6df4003ff0 0x7f6df4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:43.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:43.379+0000 7f6e15c00640 1 --2- 172.21.15.152:0/2284038698 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6df4004d40 0x7f6df40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:43.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:43.379+0000 7f6e15200640 1 --2- 172.21.15.152:0/2284038698 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e100b4500 0x7f6df4003a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:43.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:43.379+0000 7f6e0fe00640 1 -- 172.21.15.152:0/2284038698 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e100b4500 msgr2=0x7f6df4003a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:43.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:43.379+0000 7f6e0fe00640 1 --2- 172.21.15.152:0/2284038698 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e100b4500 0x7f6df4003a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:43.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:43.379+0000 7f6e0fe00640 1 -- 172.21.15.152:0/2284038698 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6df4004d40 msgr2=0x7f6df40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:43.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:43.379+0000 7f6e0fe00640 1 --2- 172.21.15.152:0/2284038698 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6df4004d40 0x7f6df40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:43.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:43.379+0000 7f6e0fe00640 1 -- 172.21.15.152:0/2284038698 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6df4003520 con 0x7f6df4003ff0 2024-04-20T11:29:43.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:43.379+0000 7f6e15c00640 1 --2- 172.21.15.152:0/2284038698 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6df4004d40 0x7f6df40051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:29:43.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:43.380+0000 7f6e0fe00640 1 --2- 172.21.15.152:0/2284038698 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6df4003ff0 0x7f6df4004800 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f6e00009fc0 tx=0x7f6df4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f99875d52a7b43f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:43.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:43.380+0000 7f6e0f400640 1 -- 172.21.15.152:0/2284038698 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e00039070 con 0x7f6df4003ff0 2024-04-20T11:29:43.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:43.380+0000 7f6e0f400640 1 -- 172.21.15.152:0/2284038698 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e0002cc20 con 0x7f6df4003ff0 2024-04-20T11:29:43.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:43.380+0000 7f6e0f400640 1 -- 172.21.15.152:0/2284038698 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e00009c10 con 0x7f6df4003ff0 2024-04-20T11:29:43.383 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:43.381+0000 7f6e17c00640 1 -- 172.21.15.152:0/2284038698 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6df4003ff0 msgr2=0x7f6df4004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:43.383 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:43.381+0000 7f6e17c00640 1 --2- 172.21.15.152:0/2284038698 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6df4003ff0 0x7f6df4004800 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f6e00009fc0 tx=0x7f6df4005830 comp rx=0 tx=0).stop 2024-04-20T11:29:43.383 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:43.381+0000 7f6e15c00640 1 -- 172.21.15.152:0/2284038698 reap_dead start 2024-04-20T11:29:43.383 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:43.381+0000 7f6e17c00640 1 -- 172.21.15.152:0/2284038698 shutdown_connections 2024-04-20T11:29:43.383 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:43.381+0000 7f6e17c00640 1 -- 172.21.15.152:0/2284038698 >> 172.21.15.152:0/2284038698 conn(0x7f6e10067cf0 msgr2=0x7f6e1006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:43.383 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:43.381+0000 7f6e17c00640 1 -- 172.21.15.152:0/2284038698 shutdown_connections 2024-04-20T11:29:43.383 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:43.381+0000 7f6e17c00640 1 -- 172.21.15.152:0/2284038698 wait complete. 2024-04-20T11:29:43.383 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:43.383+0000 7f6e17c00640 1 Processor -- start 2024-04-20T11:29:43.383 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:43.383+0000 7f6e17c00640 1 -- start start 2024-04-20T11:29:43.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:43.384+0000 7f6e17c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6df4003ff0 0x7f6e10161510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:43.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:43.384+0000 7f6e17c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6e100ce5f0 0x7f6e100cce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:43.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:43.384+0000 7f6e17c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e101f4510 0x7f6e101f4990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:43.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:43.384+0000 7f6e17c00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f6e100d0aa0 con 0x7f6e100ce5f0 2024-04-20T11:29:43.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:43.384+0000 7f6e17c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6e100d0dc0 con 0x7f6e101f4510 2024-04-20T11:29:43.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:43.384+0000 7f6e17c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6e100d0100 con 0x7f6df4003ff0 2024-04-20T11:29:43.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:43.384+0000 7f6e0fe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6e100ce5f0 0x7f6e100cce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:43.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:43.384+0000 7f6e0fe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6e100ce5f0 0x7f6e100cce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:53002/0 (socket says 172.21.15.152:53002) 2024-04-20T11:29:43.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:43.384+0000 7f6e0fe00640 1 -- 172.21.15.152:0/1711552096 learned_addr learned my addr 172.21.15.152:0/1711552096 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:29:43.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:43.384+0000 7f6e15200640 1 --2- 172.21.15.152:0/1711552096 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6df4003ff0 0x7f6e10161510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:43.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:43.384+0000 7f6e15c00640 1 --2- 172.21.15.152:0/1711552096 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e101f4510 0x7f6e101f4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:43.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:43.384+0000 7f6e0fe00640 1 -- 172.21.15.152:0/1711552096 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6df4003ff0 msgr2=0x7f6e10161510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:43.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:43.384+0000 7f6e0fe00640 1 --2- 172.21.15.152:0/1711552096 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6df4003ff0 0x7f6e10161510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:43.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:43.385+0000 7f6e0fe00640 1 -- 172.21.15.152:0/1711552096 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e101f4510 msgr2=0x7f6e101f4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:43.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:43.385+0000 7f6e0fe00640 1 --2- 172.21.15.152:0/1711552096 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e101f4510 0x7f6e101f4990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:43.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:43.385+0000 7f6e0fe00640 1 -- 172.21.15.152:0/1711552096 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e100cd550 con 0x7f6e100ce5f0 2024-04-20T11:29:43.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:43.385+0000 7f6e0fe00640 1 --2- 172.21.15.152:0/1711552096 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6e100ce5f0 0x7f6e100cce50 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f6e0000e230 tx=0x7f6e00009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:43.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:43.385+0000 7f6e0cc00640 1 -- 172.21.15.152:0/1711552096 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e00039050 con 0x7f6e100ce5f0 2024-04-20T11:29:43.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:43.385+0000 7f6e0cc00640 1 -- 172.21.15.152:0/1711552096 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e0003a040 con 0x7f6e100ce5f0 2024-04-20T11:29:43.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:43.385+0000 7f6e0cc00640 1 -- 172.21.15.152:0/1711552096 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e00003bb0 con 0x7f6e100ce5f0 2024-04-20T11:29:43.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:43.385+0000 7f6e17c00640 1 -- 172.21.15.152:0/1711552096 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e100cd830 con 0x7f6e100ce5f0 2024-04-20T11:29:43.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:43.385+0000 7f6e17c00640 1 -- 172.21.15.152:0/1711552096 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e100cddf0 con 0x7f6e100ce5f0 2024-04-20T11:29:43.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:43.387+0000 7f6e17c00640 1 -- 172.21.15.152:0/1711552096 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e10161260 con 0x7f6e100ce5f0 2024-04-20T11:29:43.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:43.388+0000 7f6e0cc00640 1 -- 172.21.15.152:0/1711552096 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 8) ==== 91244+0+0 (secure 0 0 0) 0x7f6e00004030 con 0x7f6e100ce5f0 2024-04-20T11:29:43.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:43.388+0000 7f6e0cc00640 1 --2- 172.21.15.152:0/1711552096 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f6dec06cb30 0x7f6dec06eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:43.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:43.389+0000 7f6e0cc00640 1 -- 172.21.15.152:0/1711552096 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7f6e000ae960 con 0x7f6e100ce5f0 2024-04-20T11:29:43.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:43.389+0000 7f6e15200640 1 --2- 172.21.15.152:0/1711552096 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f6dec06cb30 0x7f6dec06eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:43.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:43.389+0000 7f6e15200640 1 --2- 172.21.15.152:0/1711552096 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f6dec06cb30 0x7f6dec06eff0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f6df8000c00 tx=0x7f6df8009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:43.393 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:43.394+0000 7f6e0cc00640 1 -- 172.21.15.152:0/1711552096 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f6e0007c950 con 0x7f6e100ce5f0 2024-04-20T11:29:43.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:43.628+0000 7f6e17c00640 1 -- 172.21.15.152:0/1711552096 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6e100689f0 con 0x7f6e100ce5f0 2024-04-20T11:29:43.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:43.629+0000 7f6e0cc00640 1 -- 172.21.15.152:0/1711552096 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+5208 (secure 0 0 0) 0x7f6e0007c250 con 0x7f6e100ce5f0 2024-04-20T11:29:43.631 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:29:43.631 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4826,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6834/2855025590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2855025590},{"type":"v1","addr":"172.21.15.175:6834","nonce":2855025590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6836/3887716755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":3887716755},{"type":"v1","addr":"172.21.15.152:6836","nonce":3887716755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5712,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/698387399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6835","nonce":698387399},{"type":"v1","addr":"172.21.15.152:6837","nonce":698387399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T11:29:00.276881+0000","modified":"2024-04-20T11:29:03.352174+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5724},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5724":{"gid":5724,"name":"d","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6835/3704251089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6833","nonce":3704251089},{"type":"v1","addr":"172.21.15.175:6835","nonce":3704251089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5724,"qdb_cluster":[5724]},"id":3}]} 2024-04-20T11:29:43.632 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 25 2024-04-20T11:29:43.632 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:43.632+0000 7f6e17c00640 1 -- 172.21.15.152:0/1711552096 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f6dec06cb30 msgr2=0x7f6dec06eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:43.632 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:43.632+0000 7f6e17c00640 1 --2- 172.21.15.152:0/1711552096 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f6dec06cb30 0x7f6dec06eff0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f6df8000c00 tx=0x7f6df8009040 comp rx=0 tx=0).stop 2024-04-20T11:29:43.632 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:43.632+0000 7f6e17c00640 1 -- 172.21.15.152:0/1711552096 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6e100ce5f0 msgr2=0x7f6e100cce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:43.632 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:43.632+0000 7f6e17c00640 1 --2- 172.21.15.152:0/1711552096 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6e100ce5f0 0x7f6e100cce50 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f6e0000e230 tx=0x7f6e00009750 comp rx=0 tx=0).stop 2024-04-20T11:29:43.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:43.632+0000 7f6e17c00640 1 -- 172.21.15.152:0/1711552096 shutdown_connections 2024-04-20T11:29:43.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:43.632+0000 7f6e17c00640 1 --2- 172.21.15.152:0/1711552096 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f6dec06cb30 0x7f6dec06eff0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:43.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:43.632+0000 7f6e17c00640 1 --2- 172.21.15.152:0/1711552096 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e101f4510 0x7f6e101f4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:43.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:43.632+0000 7f6e17c00640 1 --2- 172.21.15.152:0/1711552096 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6e100ce5f0 0x7f6e100cce50 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:43.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:43.632+0000 7f6e17c00640 1 --2- 172.21.15.152:0/1711552096 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6df4003ff0 0x7f6e10161510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:43.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:43.632+0000 7f6e17c00640 1 -- 172.21.15.152:0/1711552096 >> 172.21.15.152:0/1711552096 conn(0x7f6e10067cf0 msgr2=0x7f6e100adbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:43.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:43.632+0000 7f6e17c00640 1 -- 172.21.15.152:0/1711552096 shutdown_connections 2024-04-20T11:29:43.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:43.632+0000 7f6e17c00640 1 -- 172.21.15.152:0/1711552096 wait complete. 2024-04-20T11:29:43.649 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json session ls 2024-04-20T11:29:43.879 INFO:teuthology.orchestra.run.smithi175.stdout:[{"id":6069,"entity":{"name":{"type":"client","num":6069},"addr":{"type":"any","addr":"172.21.15.152:0","nonce":83509866}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":13,"uptime":7.2368879269999997,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":21.636655844846448,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.6069 172.21.15.152:0/83509866","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"a8c84f5997950fd7d008204839253291d52e6eda","ceph_version":"ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)","entity_id":"y","hostname":"smithi152","pid":"22712","root":"/"}}] 2024-04-20T11:29:43.879 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "a8c84f5997950fd7d008204839253291d52e6eda", "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi152", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "22712", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.152:0", "nonce": 83509866, "type": "any" }, "name": { "num": 6069, "type": "client" } }, "id": 6069, "inst": "client.6069 172.21.15.152:0/83509866", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 13, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 21.636655844846448 }, "state": "open", "uptime": 7.236887927 } ] 2024-04-20T11:29:43.880 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-04-20T11:29:48.881 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:29:49.089 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:49.089+0000 7f55df000640 1 Processor -- start 2024-04-20T11:29:49.089 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:49.089+0000 7f55df000640 1 -- start start 2024-04-20T11:29:49.090 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:49.091+0000 7f55df000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f55d8145f60 0x7f55d806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:49.090 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:49.091+0000 7f55df000640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f55d80726f0 0x7f55d80cd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:49.090 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:49.091+0000 7f55df000640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55d8146340 con 0x7f55d80cd0d0 2024-04-20T11:29:49.090 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:49.091+0000 7f55df000640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f55d806e9f0 con 0x7f55d80726f0 2024-04-20T11:29:49.090 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:49.091+0000 7f55df000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f55d806f3c0 con 0x7f55d8145f60 2024-04-20T11:29:49.091 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:49.091+0000 7f55dd000640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f55d80726f0 0x7f55d80cd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:49.091 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:49.091+0000 7f55d7e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f55d8145f60 0x7f55d806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:49.091 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:49.092+0000 7f55dd000640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f55d80726f0 0x7f55d80cd550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.152:43754/0 (socket says 172.21.15.152:43754) 2024-04-20T11:29:49.091 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:49.092+0000 7f55dd000640 1 -- 172.21.15.152:0/3577049086 learned_addr learned my addr 172.21.15.152:0/3577049086 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:29:49.091 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:49.092+0000 7f55d7400640 1 --1- 172.21.15.152:0/3577049086 >> v1:172.21.15.175:6789/0 conn(0x7f55d80cd0d0 0x7f55d8071fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.152:40896/0 (socket says 172.21.15.152:40896) 2024-04-20T11:29:49.092 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:49.092+0000 7f55dd000640 1 -- 172.21.15.152:0/3577049086 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f55d8145f60 msgr2=0x7f55d806ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:49.092 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:49.092+0000 7f55dd000640 1 --2- 172.21.15.152:0/3577049086 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f55d8145f60 0x7f55d806ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:49.092 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:49.092+0000 7f55dd000640 1 -- 172.21.15.152:0/3577049086 >> v1:172.21.15.175:6789/0 conn(0x7f55d80cd0d0 legacy=0x7f55d8071fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:49.092 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:49.092+0000 7f55dd000640 1 -- 172.21.15.152:0/3577049086 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55d8072d50 con 0x7f55d80726f0 2024-04-20T11:29:49.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:49.094+0000 7f55dd000640 1 --2- 172.21.15.152:0/3577049086 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55d80726f0 0x7f55d80cd550 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f55d806eb70 tx=0x7f55d002bf50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91cc11f612307434 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:49.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:49.094+0000 7f55d6a00640 1 -- 172.21.15.152:0/3577049086 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d002caf0 con 0x7f55d80726f0 2024-04-20T11:29:49.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:49.094+0000 7f55d6a00640 1 -- 172.21.15.152:0/3577049086 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55d002cc90 con 0x7f55d80726f0 2024-04-20T11:29:49.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:49.094+0000 7f55d6a00640 1 -- 172.21.15.152:0/3577049086 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d0034610 con 0x7f55d80726f0 2024-04-20T11:29:49.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:49.095+0000 7f55df000640 1 -- 172.21.15.152:0/3577049086 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55d80726f0 msgr2=0x7f55d80cd550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:49.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:49.095+0000 7f55df000640 1 --2- 172.21.15.152:0/3577049086 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55d80726f0 0x7f55d80cd550 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f55d806eb70 tx=0x7f55d002bf50 comp rx=0 tx=0).stop 2024-04-20T11:29:49.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:49.095+0000 7f55df000640 1 -- 172.21.15.152:0/3577049086 shutdown_connections 2024-04-20T11:29:49.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:49.095+0000 7f55df000640 1 --2- 172.21.15.152:0/3577049086 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55d80726f0 0x7f55d80cd550 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:49.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:49.095+0000 7f55df000640 1 --2- 172.21.15.152:0/3577049086 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f55d8145f60 0x7f55d806ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:49.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:49.095+0000 7f55df000640 1 -- 172.21.15.152:0/3577049086 >> 172.21.15.152:0/3577049086 conn(0x7f55d8067cf0 msgr2=0x7f55d806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:49.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:49.095+0000 7f55df000640 1 -- 172.21.15.152:0/3577049086 shutdown_connections 2024-04-20T11:29:49.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:49.095+0000 7f55df000640 1 -- 172.21.15.152:0/3577049086 wait complete. 2024-04-20T11:29:49.097 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:49.097+0000 7f55df000640 1 Processor -- start 2024-04-20T11:29:49.097 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:49.097+0000 7f55df000640 1 -- start start 2024-04-20T11:29:49.098 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:49.098+0000 7f55df000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f55d80726f0 0x7f55d813ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:49.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:49.099+0000 7f55df000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55d80cd0d0 0x7f55d81f4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:49.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:49.099+0000 7f55df000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f55d8145f60 0x7f55d80ca000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:49.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:49.099+0000 7f55df000640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f55d806e680 con 0x7f55d8145f60 2024-04-20T11:29:49.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:49.099+0000 7f55df000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f55d806e3a0 con 0x7f55d80cd0d0 2024-04-20T11:29:49.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:49.099+0000 7f55df000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f55d80cf290 con 0x7f55d80726f0 2024-04-20T11:29:49.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:49.099+0000 7f55dd000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f55d8145f60 0x7f55d80ca000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:49.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:49.099+0000 7f55dd000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f55d8145f60 0x7f55d80ca000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43804/0 (socket says 172.21.15.152:43804) 2024-04-20T11:29:49.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:49.099+0000 7f55dd000640 1 -- 172.21.15.152:0/192493017 learned_addr learned my addr 172.21.15.152:0/192493017 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:29:49.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:49.099+0000 7f55d7e00640 1 --2- 172.21.15.152:0/192493017 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f55d80726f0 0x7f55d813ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:49.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:49.099+0000 7f55d7400640 1 --2- 172.21.15.152:0/192493017 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55d80cd0d0 0x7f55d81f4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:49.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:49.100+0000 7f55dd000640 1 -- 172.21.15.152:0/192493017 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f55d80726f0 msgr2=0x7f55d813ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:49.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:49.100+0000 7f55dd000640 1 --2- 172.21.15.152:0/192493017 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f55d80726f0 0x7f55d813ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:49.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:49.100+0000 7f55dd000640 1 -- 172.21.15.152:0/192493017 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55d80cd0d0 msgr2=0x7f55d81f4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:49.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:49.100+0000 7f55dd000640 1 --2- 172.21.15.152:0/192493017 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55d80cd0d0 0x7f55d81f4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:49.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:49.100+0000 7f55dd000640 1 -- 172.21.15.152:0/192493017 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55d80b5190 con 0x7f55d8145f60 2024-04-20T11:29:49.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:49.100+0000 7f55d7400640 1 --2- 172.21.15.152:0/192493017 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55d80cd0d0 0x7f55d81f4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:29:49.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:49.100+0000 7f55dd000640 1 --2- 172.21.15.152:0/192493017 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f55d8145f60 0x7f55d80ca000 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f55d002c560 tx=0x7f55d0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:49.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:49.100+0000 7f55cfe00640 1 -- 172.21.15.152:0/192493017 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d0043020 con 0x7f55d8145f60 2024-04-20T11:29:49.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:49.101+0000 7f55cfe00640 1 -- 172.21.15.152:0/192493017 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55d003e450 con 0x7f55d8145f60 2024-04-20T11:29:49.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:49.101+0000 7f55df000640 1 -- 172.21.15.152:0/192493017 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55d815ddd0 con 0x7f55d8145f60 2024-04-20T11:29:49.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:49.101+0000 7f55cfe00640 1 -- 172.21.15.152:0/192493017 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d0009510 con 0x7f55d8145f60 2024-04-20T11:29:49.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:49.101+0000 7f55df000640 1 -- 172.21.15.152:0/192493017 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55d815e390 con 0x7f55d8145f60 2024-04-20T11:29:49.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:49.102+0000 7f55df000640 1 -- 172.21.15.152:0/192493017 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55d80689f0 con 0x7f55d8145f60 2024-04-20T11:29:49.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:49.103+0000 7f55cfe00640 1 -- 172.21.15.152:0/192493017 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 8) ==== 91244+0+0 (secure 0 0 0) 0x7f55d0046020 con 0x7f55d8145f60 2024-04-20T11:29:49.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:49.104+0000 7f55cfe00640 1 --2- 172.21.15.152:0/192493017 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f55b806cb30 0x7f55b806eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:49.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:49.104+0000 7f55cfe00640 1 -- 172.21.15.152:0/192493017 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7f55d00ae650 con 0x7f55d8145f60 2024-04-20T11:29:49.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:49.104+0000 7f55d7e00640 1 --2- 172.21.15.152:0/192493017 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f55b806cb30 0x7f55b806eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:49.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:49.105+0000 7f55d7e00640 1 --2- 172.21.15.152:0/192493017 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f55b806cb30 0x7f55b806eff0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f55c8009aa0 tx=0x7f55c8009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:49.109 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:49.109+0000 7f55cfe00640 1 -- 172.21.15.152:0/192493017 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f55d004d030 con 0x7f55d8145f60 2024-04-20T11:29:49.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:49.356+0000 7f55df000640 1 -- 172.21.15.152:0/192493017 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f55d81f4c80 con 0x7f55d8145f60 2024-04-20T11:29:49.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:49.357+0000 7f55cfe00640 1 -- 172.21.15.152:0/192493017 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+5208 (secure 0 0 0) 0x7f55d007c770 con 0x7f55d8145f60 2024-04-20T11:29:49.357 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:29:49.357 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4826,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6834/2855025590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2855025590},{"type":"v1","addr":"172.21.15.175:6834","nonce":2855025590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6836/3887716755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":3887716755},{"type":"v1","addr":"172.21.15.152:6836","nonce":3887716755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5712,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/698387399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6835","nonce":698387399},{"type":"v1","addr":"172.21.15.152:6837","nonce":698387399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T11:29:00.276881+0000","modified":"2024-04-20T11:29:03.352174+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5724},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5724":{"gid":5724,"name":"d","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6835/3704251089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6833","nonce":3704251089},{"type":"v1","addr":"172.21.15.175:6835","nonce":3704251089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5724,"qdb_cluster":[5724]},"id":3}]} 2024-04-20T11:29:49.357 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 25 2024-04-20T11:29:49.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:49.360+0000 7f55df000640 1 -- 172.21.15.152:0/192493017 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f55b806cb30 msgr2=0x7f55b806eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:49.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:49.360+0000 7f55df000640 1 --2- 172.21.15.152:0/192493017 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f55b806cb30 0x7f55b806eff0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f55c8009aa0 tx=0x7f55c8009190 comp rx=0 tx=0).stop 2024-04-20T11:29:49.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:49.360+0000 7f55df000640 1 -- 172.21.15.152:0/192493017 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f55d8145f60 msgr2=0x7f55d80ca000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:49.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:49.360+0000 7f55df000640 1 --2- 172.21.15.152:0/192493017 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f55d8145f60 0x7f55d80ca000 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f55d002c560 tx=0x7f55d0004a40 comp rx=0 tx=0).stop 2024-04-20T11:29:49.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:49.360+0000 7f55df000640 1 -- 172.21.15.152:0/192493017 shutdown_connections 2024-04-20T11:29:49.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:49.360+0000 7f55df000640 1 --2- 172.21.15.152:0/192493017 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f55b806cb30 0x7f55b806eff0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:49.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:49.360+0000 7f55df000640 1 --2- 172.21.15.152:0/192493017 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f55d8145f60 0x7f55d80ca000 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:49.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:49.360+0000 7f55df000640 1 --2- 172.21.15.152:0/192493017 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55d80cd0d0 0x7f55d81f4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:49.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:49.360+0000 7f55df000640 1 --2- 172.21.15.152:0/192493017 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f55d80726f0 0x7f55d813ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:49.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:49.360+0000 7f55df000640 1 -- 172.21.15.152:0/192493017 >> 172.21.15.152:0/192493017 conn(0x7f55d8067cf0 msgr2=0x7f55d806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:49.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:49.360+0000 7f55df000640 1 -- 172.21.15.152:0/192493017 shutdown_connections 2024-04-20T11:29:49.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:49.360+0000 7f55df000640 1 -- 172.21.15.152:0/192493017 wait complete. 2024-04-20T11:29:49.379 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json session ls 2024-04-20T11:29:49.613 INFO:teuthology.orchestra.run.smithi175.stdout:[{"id":6069,"entity":{"name":{"type":"client","num":6069},"addr":{"type":"any","addr":"172.21.15.152:0","nonce":83509866}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":12,"uptime":12.970798896,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":21.351900643139302,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.6069 172.21.15.152:0/83509866","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"a8c84f5997950fd7d008204839253291d52e6eda","ceph_version":"ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)","entity_id":"y","hostname":"smithi152","pid":"22712","root":"/"}}] 2024-04-20T11:29:49.613 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "a8c84f5997950fd7d008204839253291d52e6eda", "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi152", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "22712", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.152:0", "nonce": 83509866, "type": "any" }, "name": { "num": 6069, "type": "client" } }, "id": 6069, "inst": "client.6069 172.21.15.152:0/83509866", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 12, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 21.351900643139302 }, "state": "open", "uptime": 12.970798896 } ] 2024-04-20T11:29:49.613 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-04-20T11:29:51.746 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:29:51.755 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:29:54.614 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:29:54.826 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:54.826+0000 7f74e0600640 1 Processor -- start 2024-04-20T11:29:54.826 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:54.827+0000 7f74e0600640 1 -- start start 2024-04-20T11:29:54.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:54.828+0000 7f74e0600640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f74d815c810 0x7f74d8160470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:54.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:54.828+0000 7f74e0600640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74d81612a0 con 0x7f74d809bb80 2024-04-20T11:29:54.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:54.828+0000 7f74e0600640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74d809aae0 con 0x7f74d80d1280 2024-04-20T11:29:54.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:54.828+0000 7f74e0600640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f74d809adc0 con 0x7f74d815c810 2024-04-20T11:29:54.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:54.828+0000 7f74dd200640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f74d809bb80 0x7f74d80d0b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:51166/0 (socket says 172.21.15.152:51166) 2024-04-20T11:29:54.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:54.828+0000 7f74dd200640 1 -- 172.21.15.152:0/4155332340 learned_addr learned my addr 172.21.15.152:0/4155332340 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:29:54.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:54.829+0000 7f74ddc00640 1 --2- 172.21.15.152:0/4155332340 >> v2:172.21.15.175:3300/0 conn(0x7f74d815c810 0x7f74d8160470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:54.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:54.829+0000 7f74d7e00640 1 -- 172.21.15.152:0/4155332340 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4019781792 0 0) 0x7f74d81612a0 con 0x7f74d809bb80 2024-04-20T11:29:54.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:54.829+0000 7f74d7e00640 1 -- 172.21.15.152:0/4155332340 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f74bc003520 con 0x7f74d809bb80 2024-04-20T11:29:54.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:54.829+0000 7f74d7e00640 1 -- 172.21.15.152:0/4155332340 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 713181492 0 0) 0x7f74bc003520 con 0x7f74d809bb80 2024-04-20T11:29:54.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:54.829+0000 7f74d7e00640 1 -- 172.21.15.152:0/4155332340 >> v2:172.21.15.175:3300/0 conn(0x7f74d815c810 msgr2=0x7f74d8160470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:54.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:54.829+0000 7f74d7e00640 1 --2- 172.21.15.152:0/4155332340 >> v2:172.21.15.175:3300/0 conn(0x7f74d815c810 0x7f74d8160470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:54.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:54.829+0000 7f74d7e00640 1 -- 172.21.15.152:0/4155332340 >> v1:172.21.15.175:6789/0 conn(0x7f74d80d1280 legacy=0x7f74d80c8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:54.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:54.829+0000 7f74d7e00640 1 -- 172.21.15.152:0/4155332340 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74d809b0a0 con 0x7f74d809bb80 2024-04-20T11:29:54.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:54.829+0000 7f74d7e00640 1 -- 172.21.15.152:0/4155332340 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f74c40019f0 con 0x7f74d809bb80 2024-04-20T11:29:54.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:54.829+0000 7f74d7e00640 1 -- 172.21.15.152:0/4155332340 >> v1:172.21.15.152:6789/0 conn(0x7f74d809bb80 legacy=0x7f74d80d0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:54.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:54.829+0000 7f74d7e00640 1 --2- 172.21.15.152:0/4155332340 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74bc003a20 0x7f74bc003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:54.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:54.829+0000 7f74d7e00640 1 --2- 172.21.15.152:0/4155332340 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f74bc004800 0x7f74bc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:54.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:54.829+0000 7f74d7e00640 1 --2- 172.21.15.152:0/4155332340 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74bc0051c0 0x7f74bc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:54.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:54.829+0000 7f74d7e00640 1 -- 172.21.15.152:0/4155332340 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f74d809adc0 con 0x7f74bc004800 2024-04-20T11:29:54.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:54.830+0000 7f74d7e00640 1 -- 172.21.15.152:0/4155332340 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f74bc00c0a0 con 0x7f74bc0051c0 2024-04-20T11:29:54.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:54.830+0000 7f74d7e00640 1 -- 172.21.15.152:0/4155332340 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f74bc00c2a0 con 0x7f74bc003a20 2024-04-20T11:29:54.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:54.830+0000 7f74dd200640 1 --2- 172.21.15.152:0/4155332340 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f74bc004800 0x7f74bc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:54.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:54.830+0000 7f74de600640 1 --2- 172.21.15.152:0/4155332340 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74bc0051c0 0x7f74bc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:54.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:54.830+0000 7f74ddc00640 1 --2- 172.21.15.152:0/4155332340 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74bc003a20 0x7f74bc003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:54.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:54.830+0000 7f74dd200640 1 -- 172.21.15.152:0/4155332340 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74bc003a20 msgr2=0x7f74bc003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:54.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:54.830+0000 7f74dd200640 1 --2- 172.21.15.152:0/4155332340 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74bc003a20 0x7f74bc003e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:54.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:54.830+0000 7f74dd200640 1 -- 172.21.15.152:0/4155332340 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74bc0051c0 msgr2=0x7f74bc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:54.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:54.830+0000 7f74dd200640 1 --2- 172.21.15.152:0/4155332340 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74bc0051c0 0x7f74bc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:54.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:54.830+0000 7f74dd200640 1 -- 172.21.15.152:0/4155332340 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74bc003520 con 0x7f74bc004800 2024-04-20T11:29:54.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:54.830+0000 7f74de600640 1 -- 172.21.15.152:0/4155332340 reap_dead start 2024-04-20T11:29:54.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:54.831+0000 7f74dd200640 1 --2- 172.21.15.152:0/4155332340 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f74bc004800 0x7f74bc004c80 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f74c400a920 tx=0x7f74bc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce23db146f7c355d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:54.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:54.832+0000 7f74d7e00640 1 -- 172.21.15.152:0/4155332340 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c400a4c0 con 0x7f74bc004800 2024-04-20T11:29:54.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:54.832+0000 7f74d7e00640 1 -- 172.21.15.152:0/4155332340 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74c4034650 con 0x7f74bc004800 2024-04-20T11:29:54.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:54.832+0000 7f74d7e00640 1 -- 172.21.15.152:0/4155332340 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c4034a80 con 0x7f74bc004800 2024-04-20T11:29:54.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:54.832+0000 7f74e0600640 1 -- 172.21.15.152:0/4155332340 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f74bc004800 msgr2=0x7f74bc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:54.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:54.832+0000 7f74e0600640 1 --2- 172.21.15.152:0/4155332340 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f74bc004800 0x7f74bc004c80 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f74c400a920 tx=0x7f74bc005830 comp rx=0 tx=0).stop 2024-04-20T11:29:54.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:54.832+0000 7f74e0600640 1 -- 172.21.15.152:0/4155332340 shutdown_connections 2024-04-20T11:29:54.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:54.832+0000 7f74e0600640 1 --2- 172.21.15.152:0/4155332340 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f74bc004800 0x7f74bc004c80 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:54.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:54.832+0000 7f74e0600640 1 -- 172.21.15.152:0/4155332340 >> 172.21.15.152:0/4155332340 conn(0x7f74d80b4080 msgr2=0x7f74d80b4490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:54.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:54.832+0000 7f74e0600640 1 -- 172.21.15.152:0/4155332340 shutdown_connections 2024-04-20T11:29:54.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:54.832+0000 7f74e0600640 1 -- 172.21.15.152:0/4155332340 wait complete. 2024-04-20T11:29:54.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:54.834+0000 7f74e0600640 1 Processor -- start 2024-04-20T11:29:54.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:54.834+0000 7f74e0600640 1 -- start start 2024-04-20T11:29:54.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:54.835+0000 7f74e0600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74bc004800 0x7f74d81f4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:54.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:54.835+0000 7f74e0600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f74d815d860 0x7f74d815dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:54.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:54.835+0000 7f74e0600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74d815e200 0x7f74d80caa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:54.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:54.835+0000 7f74e0600640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f74d815cd70 con 0x7f74d815d860 2024-04-20T11:29:54.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:54.835+0000 7f74e0600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f74d815cbf0 con 0x7f74d815e200 2024-04-20T11:29:54.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:54.835+0000 7f74e0600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f74d80d0b70 con 0x7f74bc004800 2024-04-20T11:29:54.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:54.836+0000 7f74dd200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f74d815d860 0x7f74d815dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:54.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:54.836+0000 7f74dd200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f74d815d860 0x7f74d815dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43824/0 (socket says 172.21.15.152:43824) 2024-04-20T11:29:54.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:54.836+0000 7f74dd200640 1 -- 172.21.15.152:0/1081163970 learned_addr learned my addr 172.21.15.152:0/1081163970 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:29:54.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:54.836+0000 7f74de600640 1 --2- 172.21.15.152:0/1081163970 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74d815e200 0x7f74d80caa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:54.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:54.836+0000 7f74ddc00640 1 --2- 172.21.15.152:0/1081163970 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74bc004800 0x7f74d81f4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:54.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:54.836+0000 7f74dd200640 1 -- 172.21.15.152:0/1081163970 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74bc004800 msgr2=0x7f74d81f4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:54.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:54.836+0000 7f74dd200640 1 --2- 172.21.15.152:0/1081163970 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74bc004800 0x7f74d81f4740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:54.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:54.836+0000 7f74dd200640 1 -- 172.21.15.152:0/1081163970 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74d815e200 msgr2=0x7f74d80caa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:54.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:54.836+0000 7f74dd200640 1 --2- 172.21.15.152:0/1081163970 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74d815e200 0x7f74d80caa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:54.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:54.836+0000 7f74dd200640 1 -- 172.21.15.152:0/1081163970 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74d815e680 con 0x7f74d815d860 2024-04-20T11:29:54.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:54.836+0000 7f74de600640 1 --2- 172.21.15.152:0/1081163970 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74d815e200 0x7f74d80caa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:29:54.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:54.836+0000 7f74dd200640 1 --2- 172.21.15.152:0/1081163970 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f74d815d860 0x7f74d815dcc0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f74c402cc90 tx=0x7f74c4001dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:54.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:54.837+0000 7f74d5600640 1 -- 172.21.15.152:0/1081163970 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c4039040 con 0x7f74d815d860 2024-04-20T11:29:54.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:54.837+0000 7f74d5600640 1 -- 172.21.15.152:0/1081163970 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74c4001870 con 0x7f74d815d860 2024-04-20T11:29:54.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:54.837+0000 7f74e0600640 1 -- 172.21.15.152:0/1081163970 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74d80c9c80 con 0x7f74d815d860 2024-04-20T11:29:54.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:54.837+0000 7f74d5600640 1 -- 172.21.15.152:0/1081163970 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c4004d60 con 0x7f74d815d860 2024-04-20T11:29:54.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:54.837+0000 7f74e0600640 1 -- 172.21.15.152:0/1081163970 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74d80ca240 con 0x7f74d815d860 2024-04-20T11:29:54.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:54.838+0000 7f74e0600640 1 -- 172.21.15.152:0/1081163970 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74d80689f0 con 0x7f74d815d860 2024-04-20T11:29:54.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:54.839+0000 7f74d5600640 1 -- 172.21.15.152:0/1081163970 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 8) ==== 91244+0+0 (secure 0 0 0) 0x7f74c4045020 con 0x7f74d815d860 2024-04-20T11:29:54.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:54.840+0000 7f74d5600640 1 --2- 172.21.15.152:0/1081163970 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f74b806cb30 0x7f74b806eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:54.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:54.841+0000 7f74d5600640 1 -- 172.21.15.152:0/1081163970 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7f74c40add60 con 0x7f74d815d860 2024-04-20T11:29:54.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:54.841+0000 7f74ddc00640 1 --2- 172.21.15.152:0/1081163970 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f74b806cb30 0x7f74b806eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:54.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:54.841+0000 7f74ddc00640 1 --2- 172.21.15.152:0/1081163970 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f74b806cb30 0x7f74b806eff0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f74d8076cd0 tx=0x7f74c8008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:54.845 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:54.846+0000 7f74d5600640 1 -- 172.21.15.152:0/1081163970 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f74c407bc80 con 0x7f74d815d860 2024-04-20T11:29:55.090 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:55.089+0000 7f74e0600640 1 -- 172.21.15.152:0/1081163970 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f74d80b4dc0 con 0x7f74d815d860 2024-04-20T11:29:55.090 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:55.090+0000 7f74d5600640 1 -- 172.21.15.152:0/1081163970 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+5208 (secure 0 0 0) 0x7f74c407b580 con 0x7f74d815d860 2024-04-20T11:29:55.091 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:29:55.091 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4826,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6834/2855025590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2855025590},{"type":"v1","addr":"172.21.15.175:6834","nonce":2855025590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6836/3887716755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":3887716755},{"type":"v1","addr":"172.21.15.152:6836","nonce":3887716755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5712,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/698387399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6835","nonce":698387399},{"type":"v1","addr":"172.21.15.152:6837","nonce":698387399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T11:29:00.276881+0000","modified":"2024-04-20T11:29:03.352174+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5724},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5724":{"gid":5724,"name":"d","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6835/3704251089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6833","nonce":3704251089},{"type":"v1","addr":"172.21.15.175:6835","nonce":3704251089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5724,"qdb_cluster":[5724]},"id":3}]} 2024-04-20T11:29:55.091 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 25 2024-04-20T11:29:55.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:55.093+0000 7f74e0600640 1 -- 172.21.15.152:0/1081163970 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f74b806cb30 msgr2=0x7f74b806eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:55.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:55.093+0000 7f74e0600640 1 --2- 172.21.15.152:0/1081163970 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f74b806cb30 0x7f74b806eff0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f74d8076cd0 tx=0x7f74c8008040 comp rx=0 tx=0).stop 2024-04-20T11:29:55.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:55.093+0000 7f74e0600640 1 -- 172.21.15.152:0/1081163970 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f74d815d860 msgr2=0x7f74d815dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:55.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:55.093+0000 7f74e0600640 1 --2- 172.21.15.152:0/1081163970 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f74d815d860 0x7f74d815dcc0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f74c402cc90 tx=0x7f74c4001dc0 comp rx=0 tx=0).stop 2024-04-20T11:29:55.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:55.093+0000 7f74e0600640 1 -- 172.21.15.152:0/1081163970 shutdown_connections 2024-04-20T11:29:55.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:55.093+0000 7f74e0600640 1 --2- 172.21.15.152:0/1081163970 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f74b806cb30 0x7f74b806eff0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:55.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:55.093+0000 7f74e0600640 1 --2- 172.21.15.152:0/1081163970 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74d815e200 0x7f74d80caa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:55.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:55.093+0000 7f74e0600640 1 --2- 172.21.15.152:0/1081163970 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f74d815d860 0x7f74d815dcc0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:55.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:55.093+0000 7f74e0600640 1 --2- 172.21.15.152:0/1081163970 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74bc004800 0x7f74d81f4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:55.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:55.093+0000 7f74e0600640 1 -- 172.21.15.152:0/1081163970 >> 172.21.15.152:0/1081163970 conn(0x7f74d80b4080 msgr2=0x7f74d80c8230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:55.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:55.093+0000 7f74e0600640 1 -- 172.21.15.152:0/1081163970 shutdown_connections 2024-04-20T11:29:55.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:29:55.093+0000 7f74e0600640 1 -- 172.21.15.152:0/1081163970 wait complete. 2024-04-20T11:29:55.112 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json session ls 2024-04-20T11:29:55.347 INFO:teuthology.orchestra.run.smithi175.stdout:[{"id":6069,"entity":{"name":{"type":"client","num":6069},"addr":{"type":"any","addr":"172.21.15.152:0","nonce":83509866}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":12,"uptime":18.704709866000002,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":21.070893041080986,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.6069 172.21.15.152:0/83509866","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"a8c84f5997950fd7d008204839253291d52e6eda","ceph_version":"ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)","entity_id":"y","hostname":"smithi152","pid":"22712","root":"/"}}] 2024-04-20T11:29:55.347 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "a8c84f5997950fd7d008204839253291d52e6eda", "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi152", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "22712", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.152:0", "nonce": 83509866, "type": "any" }, "name": { "num": 6069, "type": "client" } }, "id": 6069, "inst": "client.6069 172.21.15.152:0/83509866", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 12, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 21.070893041080986 }, "state": "open", "uptime": 18.704709866 } ] 2024-04-20T11:29:55.347 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-04-20T11:30:00.348 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:30:00.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:00.561+0000 7fbe93e00640 1 Processor -- start 2024-04-20T11:30:00.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:00.561+0000 7fbe93e00640 1 -- start start 2024-04-20T11:30:00.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:00.563+0000 7fbe93e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fbe9413e530 0x7fbe9406ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:00.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:00.563+0000 7fbe93e00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe940cf0a0 con 0x7fbe9406f340 2024-04-20T11:30:00.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:00.563+0000 7fbe93e00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe940d00e0 con 0x7fbe941414e0 2024-04-20T11:30:00.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:00.563+0000 7fbe93e00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fbe94141b00 con 0x7fbe9413e530 2024-04-20T11:30:00.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:00.563+0000 7fbe92000640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fbe9406f340 0x7fbe94140dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:58118/0 (socket says 172.21.15.152:58118) 2024-04-20T11:30:00.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:00.563+0000 7fbe92000640 1 -- 172.21.15.152:0/3761936335 learned_addr learned my addr 172.21.15.152:0/3761936335 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:30:00.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:00.564+0000 7fbe92a00640 1 --2- 172.21.15.152:0/3761936335 >> v2:172.21.15.175:3300/0 conn(0x7fbe9413e530 0x7fbe9406ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:00.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:00.564+0000 7fbe91600640 1 -- 172.21.15.152:0/3761936335 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 525936793 0 0) 0x7fbe80001110 con 0x7fbe9406f340 2024-04-20T11:30:00.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:00.564+0000 7fbe91600640 1 -- 172.21.15.152:0/3761936335 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbe78003520 con 0x7fbe9406f340 2024-04-20T11:30:00.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:00.564+0000 7fbe91600640 1 -- 172.21.15.152:0/3761936335 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4184782032 0 0) 0x7fbe78003520 con 0x7fbe9406f340 2024-04-20T11:30:00.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:00.564+0000 7fbe91600640 1 -- 172.21.15.152:0/3761936335 >> v2:172.21.15.175:3300/0 conn(0x7fbe9413e530 msgr2=0x7fbe9406ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:00.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:00.564+0000 7fbe91600640 1 --2- 172.21.15.152:0/3761936335 >> v2:172.21.15.175:3300/0 conn(0x7fbe9413e530 0x7fbe9406ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:00.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:00.564+0000 7fbe91600640 1 -- 172.21.15.152:0/3761936335 >> v1:172.21.15.175:6789/0 conn(0x7fbe941414e0 legacy=0x7fbe94071cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:00.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:00.564+0000 7fbe91600640 1 -- 172.21.15.152:0/3761936335 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe940cf320 con 0x7fbe9406f340 2024-04-20T11:30:00.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:00.564+0000 7fbe91600640 1 -- 172.21.15.152:0/3761936335 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7fbe80001840 con 0x7fbe9406f340 2024-04-20T11:30:00.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:00.564+0000 7fbe91600640 1 -- 172.21.15.152:0/3761936335 >> v1:172.21.15.152:6789/0 conn(0x7fbe9406f340 legacy=0x7fbe94140dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:00.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:00.564+0000 7fbe91600640 1 --2- 172.21.15.152:0/3761936335 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbe78003a00 0x7fbe78003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:00.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:00.564+0000 7fbe91600640 1 --2- 172.21.15.152:0/3761936335 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbe78004800 0x7fbe78004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:00.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:00.564+0000 7fbe91600640 1 --2- 172.21.15.152:0/3761936335 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbe780051c0 0x7fbe7800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:00.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:00.564+0000 7fbe91600640 1 -- 172.21.15.152:0/3761936335 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fbe7800c090 con 0x7fbe780051c0 2024-04-20T11:30:00.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:00.564+0000 7fbe91600640 1 -- 172.21.15.152:0/3761936335 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbe7800c210 con 0x7fbe78003a00 2024-04-20T11:30:00.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:00.564+0000 7fbe91600640 1 -- 172.21.15.152:0/3761936335 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fbe7800c410 con 0x7fbe78004800 2024-04-20T11:30:00.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:00.565+0000 7fbe93400640 1 --2- 172.21.15.152:0/3761936335 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbe780051c0 0x7fbe7800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:00.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:00.565+0000 7fbe92a00640 1 --2- 172.21.15.152:0/3761936335 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbe78003a00 0x7fbe78003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:00.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:00.565+0000 7fbe92000640 1 --2- 172.21.15.152:0/3761936335 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbe78004800 0x7fbe78004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:00.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:00.565+0000 7fbe93400640 1 -- 172.21.15.152:0/3761936335 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbe78004800 msgr2=0x7fbe78004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:00.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:00.565+0000 7fbe93400640 1 --2- 172.21.15.152:0/3761936335 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbe78004800 0x7fbe78004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:00.567 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:00.565+0000 7fbe93400640 1 -- 172.21.15.152:0/3761936335 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbe78003a00 msgr2=0x7fbe78003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:00.567 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:00.565+0000 7fbe93400640 1 --2- 172.21.15.152:0/3761936335 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbe78003a00 0x7fbe78003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:00.567 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:00.565+0000 7fbe93400640 1 -- 172.21.15.152:0/3761936335 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe78003520 con 0x7fbe780051c0 2024-04-20T11:30:00.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:00.566+0000 7fbe93400640 1 -- 172.21.15.152:0/3761936335 reap_dead start 2024-04-20T11:30:00.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:00.566+0000 7fbe93400640 1 --2- 172.21.15.152:0/3761936335 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbe780051c0 0x7fbe7800b9c0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c0044b0 tx=0x7fbe78005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f667923039c90a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:00.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:00.567+0000 7fbe91600640 1 -- 172.21.15.152:0/3761936335 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe8c034070 con 0x7fbe780051c0 2024-04-20T11:30:00.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:00.567+0000 7fbe91600640 1 -- 172.21.15.152:0/3761936335 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe8c030620 con 0x7fbe780051c0 2024-04-20T11:30:00.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:00.567+0000 7fbe91600640 1 -- 172.21.15.152:0/3761936335 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe8c004080 con 0x7fbe780051c0 2024-04-20T11:30:00.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:00.567+0000 7fbe93e00640 1 -- 172.21.15.152:0/3761936335 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbe780051c0 msgr2=0x7fbe7800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:00.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:00.567+0000 7fbe93e00640 1 --2- 172.21.15.152:0/3761936335 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbe780051c0 0x7fbe7800b9c0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c0044b0 tx=0x7fbe78005830 comp rx=0 tx=0).stop 2024-04-20T11:30:00.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:00.567+0000 7fbe93e00640 1 -- 172.21.15.152:0/3761936335 shutdown_connections 2024-04-20T11:30:00.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:00.567+0000 7fbe93e00640 1 --2- 172.21.15.152:0/3761936335 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbe780051c0 0x7fbe7800b9c0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:00.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:00.567+0000 7fbe93e00640 1 -- 172.21.15.152:0/3761936335 >> 172.21.15.152:0/3761936335 conn(0x7fbe94067ae0 msgr2=0x7fbe9406b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:00.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:00.567+0000 7fbe93e00640 1 -- 172.21.15.152:0/3761936335 shutdown_connections 2024-04-20T11:30:00.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:00.568+0000 7fbe93e00640 1 -- 172.21.15.152:0/3761936335 wait complete. 2024-04-20T11:30:00.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:00.570+0000 7fbe93e00640 1 Processor -- start 2024-04-20T11:30:00.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:00.570+0000 7fbe93e00640 1 -- start start 2024-04-20T11:30:00.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:00.571+0000 7fbe93e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbe780051c0 0x7fbe940a9e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:00.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:00.571+0000 7fbe93e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbe941f4dc0 0x7fbe940b3920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:00.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:00.571+0000 7fbe93e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbe941512e0 0x7fbe94151770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:00.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:00.571+0000 7fbe93e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fbe940cf8e0 con 0x7fbe941f4dc0 2024-04-20T11:30:00.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:00.571+0000 7fbe93e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbe940cfc50 con 0x7fbe780051c0 2024-04-20T11:30:00.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:00.571+0000 7fbe93e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fbe94140dd0 con 0x7fbe941512e0 2024-04-20T11:30:00.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:00.571+0000 7fbe92000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbe941f4dc0 0x7fbe940b3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:00.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:00.571+0000 7fbe92000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbe941f4dc0 0x7fbe940b3920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:56612/0 (socket says 172.21.15.152:56612) 2024-04-20T11:30:00.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:00.571+0000 7fbe92000640 1 -- 172.21.15.152:0/2519468426 learned_addr learned my addr 172.21.15.152:0/2519468426 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:30:00.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:00.571+0000 7fbe92a00640 1 --2- 172.21.15.152:0/2519468426 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbe780051c0 0x7fbe940a9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:00.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:00.571+0000 7fbe93400640 1 --2- 172.21.15.152:0/2519468426 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbe941512e0 0x7fbe94151770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:00.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:00.571+0000 7fbe92000640 1 -- 172.21.15.152:0/2519468426 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbe941512e0 msgr2=0x7fbe94151770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:00.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:00.571+0000 7fbe92000640 1 --2- 172.21.15.152:0/2519468426 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbe941512e0 0x7fbe94151770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:00.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:00.571+0000 7fbe92000640 1 -- 172.21.15.152:0/2519468426 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbe780051c0 msgr2=0x7fbe940a9e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:00.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:00.571+0000 7fbe92000640 1 --2- 172.21.15.152:0/2519468426 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbe780051c0 0x7fbe940a9e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:00.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:00.571+0000 7fbe92000640 1 -- 172.21.15.152:0/2519468426 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe94153c20 con 0x7fbe941f4dc0 2024-04-20T11:30:00.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:00.571+0000 7fbe93400640 1 --2- 172.21.15.152:0/2519468426 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbe941512e0 0x7fbe94151770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:30:00.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:00.572+0000 7fbe92000640 1 --2- 172.21.15.152:0/2519468426 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbe941f4dc0 0x7fbe940b3920 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fbe8000e6f0 tx=0x7fbe8000ebb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:00.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:00.572+0000 7fbe8aa00640 1 -- 172.21.15.152:0/2519468426 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe8000c6c0 con 0x7fbe941f4dc0 2024-04-20T11:30:00.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:00.572+0000 7fbe8aa00640 1 -- 172.21.15.152:0/2519468426 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe80018070 con 0x7fbe941f4dc0 2024-04-20T11:30:00.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:00.572+0000 7fbe93e00640 1 -- 172.21.15.152:0/2519468426 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe94153e90 con 0x7fbe941f4dc0 2024-04-20T11:30:00.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:00.572+0000 7fbe8aa00640 1 -- 172.21.15.152:0/2519468426 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe8000cae0 con 0x7fbe941f4dc0 2024-04-20T11:30:00.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:00.572+0000 7fbe93e00640 1 -- 172.21.15.152:0/2519468426 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe94154450 con 0x7fbe941f4dc0 2024-04-20T11:30:00.574 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:00.573+0000 7fbe93e00640 1 -- 172.21.15.152:0/2519468426 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe94075e60 con 0x7fbe941f4dc0 2024-04-20T11:30:00.574 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:00.574+0000 7fbe8aa00640 1 -- 172.21.15.152:0/2519468426 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 8) ==== 91244+0+0 (secure 0 0 0) 0x7fbe8000cc80 con 0x7fbe941f4dc0 2024-04-20T11:30:00.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:00.576+0000 7fbe8aa00640 1 --2- 172.21.15.152:0/2519468426 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fbe7406cad0 0x7fbe7406ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:00.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:00.576+0000 7fbe8aa00640 1 -- 172.21.15.152:0/2519468426 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7fbe8008bd90 con 0x7fbe941f4dc0 2024-04-20T11:30:00.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:00.576+0000 7fbe92a00640 1 --2- 172.21.15.152:0/2519468426 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fbe7406cad0 0x7fbe7406ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:00.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:00.577+0000 7fbe92a00640 1 --2- 172.21.15.152:0/2519468426 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fbe7406cad0 0x7fbe7406ef90 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fbe84009990 tx=0x7fbe84008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:00.580 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:00.581+0000 7fbe8aa00640 1 -- 172.21.15.152:0/2519468426 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fbe940cf0a0 con 0x7fbe941f4dc0 2024-04-20T11:30:00.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:00.824+0000 7fbe93e00640 1 -- 172.21.15.152:0/2519468426 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbe940b3490 con 0x7fbe941f4dc0 2024-04-20T11:30:00.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:00.824+0000 7fbe8aa00640 1 -- 172.21.15.152:0/2519468426 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+5208 (secure 0 0 0) 0x7fbe80059ed0 con 0x7fbe941f4dc0 2024-04-20T11:30:00.824 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:30:00.824 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4826,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6834/2855025590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2855025590},{"type":"v1","addr":"172.21.15.175:6834","nonce":2855025590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6836/3887716755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":3887716755},{"type":"v1","addr":"172.21.15.152:6836","nonce":3887716755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5712,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/698387399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6835","nonce":698387399},{"type":"v1","addr":"172.21.15.152:6837","nonce":698387399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T11:29:00.276881+0000","modified":"2024-04-20T11:29:03.352174+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5724},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5724":{"gid":5724,"name":"d","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6835/3704251089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6833","nonce":3704251089},{"type":"v1","addr":"172.21.15.175:6835","nonce":3704251089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5724,"qdb_cluster":[5724]},"id":3}]} 2024-04-20T11:30:00.824 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 25 2024-04-20T11:30:00.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:00.827+0000 7fbe93e00640 1 -- 172.21.15.152:0/2519468426 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fbe7406cad0 msgr2=0x7fbe7406ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:00.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:00.827+0000 7fbe93e00640 1 --2- 172.21.15.152:0/2519468426 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fbe7406cad0 0x7fbe7406ef90 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fbe84009990 tx=0x7fbe84008040 comp rx=0 tx=0).stop 2024-04-20T11:30:00.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:00.827+0000 7fbe93e00640 1 -- 172.21.15.152:0/2519468426 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbe941f4dc0 msgr2=0x7fbe940b3920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:00.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:00.827+0000 7fbe93e00640 1 --2- 172.21.15.152:0/2519468426 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbe941f4dc0 0x7fbe940b3920 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fbe8000e6f0 tx=0x7fbe8000ebb0 comp rx=0 tx=0).stop 2024-04-20T11:30:00.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:00.827+0000 7fbe93e00640 1 -- 172.21.15.152:0/2519468426 shutdown_connections 2024-04-20T11:30:00.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:00.827+0000 7fbe93e00640 1 --2- 172.21.15.152:0/2519468426 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fbe7406cad0 0x7fbe7406ef90 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:00.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:00.827+0000 7fbe93e00640 1 --2- 172.21.15.152:0/2519468426 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbe941512e0 0x7fbe94151770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:00.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:00.827+0000 7fbe93e00640 1 --2- 172.21.15.152:0/2519468426 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbe941f4dc0 0x7fbe940b3920 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:00.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:00.827+0000 7fbe93e00640 1 --2- 172.21.15.152:0/2519468426 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbe780051c0 0x7fbe940a9e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:00.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:00.827+0000 7fbe93e00640 1 -- 172.21.15.152:0/2519468426 >> 172.21.15.152:0/2519468426 conn(0x7fbe94067ae0 msgr2=0x7fbe9413f950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:00.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:00.827+0000 7fbe93e00640 1 -- 172.21.15.152:0/2519468426 shutdown_connections 2024-04-20T11:30:00.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:00.827+0000 7fbe93e00640 1 -- 172.21.15.152:0/2519468426 wait complete. 2024-04-20T11:30:00.845 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json session ls 2024-04-20T11:30:01.076 INFO:teuthology.orchestra.run.smithi175.stdout:[{"id":6069,"entity":{"name":{"type":"client","num":6069},"addr":{"type":"any","addr":"172.21.15.152:0","nonce":83509866}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":11,"uptime":24.433620914999999,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":20.793823931888575,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.6069 172.21.15.152:0/83509866","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"a8c84f5997950fd7d008204839253291d52e6eda","ceph_version":"ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)","entity_id":"y","hostname":"smithi152","pid":"22712","root":"/"}}] 2024-04-20T11:30:01.076 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "a8c84f5997950fd7d008204839253291d52e6eda", "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi152", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "22712", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.152:0", "nonce": 83509866, "type": "any" }, "name": { "num": 6069, "type": "client" } }, "id": 6069, "inst": "client.6069 172.21.15.152:0/83509866", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 11, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 20.793823931888575 }, "state": "open", "uptime": 24.433620915 } ] 2024-04-20T11:30:01.077 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-04-20T11:30:06.078 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:30:06.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:06.293+0000 7fa772c00640 1 Processor -- start 2024-04-20T11:30:06.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:06.293+0000 7fa772c00640 1 -- start start 2024-04-20T11:30:06.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:06.295+0000 7fa772c00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fa76c0723e0 0x7fa76c15d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:06.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:06.295+0000 7fa772c00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa76c0ce8c0 con 0x7fa76c15d1e0 2024-04-20T11:30:06.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:06.295+0000 7fa772c00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa76c1600b0 con 0x7fa76c160b90 2024-04-20T11:30:06.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:06.295+0000 7fa772c00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fa76c160390 con 0x7fa76c0723e0 2024-04-20T11:30:06.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:06.295+0000 7fa771800640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fa76c15d1e0 0x7fa76c0cdfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:44576/0 (socket says 172.21.15.152:44576) 2024-04-20T11:30:06.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:06.295+0000 7fa771800640 1 -- 172.21.15.152:0/505395698 learned_addr learned my addr 172.21.15.152:0/505395698 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:30:06.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:06.295+0000 7fa772200640 1 --2- 172.21.15.152:0/505395698 >> v2:172.21.15.152:3300/0 conn(0x7fa76c0723e0 0x7fa76c15d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:06.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:06.296+0000 7fa76be00640 1 -- 172.21.15.152:0/505395698 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1749698273 0 0) 0x7fa76c0ce8c0 con 0x7fa76c15d1e0 2024-04-20T11:30:06.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:06.296+0000 7fa76be00640 1 -- 172.21.15.152:0/505395698 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa750003520 con 0x7fa76c15d1e0 2024-04-20T11:30:06.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:06.296+0000 7fa772200640 1 -- 172.21.15.152:0/505395698 >> v1:172.21.15.175:6789/0 conn(0x7fa76c160b90 legacy=0x7fa76c071cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:06.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:06.296+0000 7fa772200640 1 -- 172.21.15.152:0/505395698 >> v1:172.21.15.152:6789/0 conn(0x7fa76c15d1e0 legacy=0x7fa76c0cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:06.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:06.296+0000 7fa772200640 1 -- 172.21.15.152:0/505395698 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa76c160670 con 0x7fa76c0723e0 2024-04-20T11:30:06.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:06.297+0000 7fa772200640 1 --2- 172.21.15.152:0/505395698 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa76c0723e0 0x7fa76c15d5c0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fa75c0097d0 tx=0x7fa75c02b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1de684dd9ad2640 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:06.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:06.297+0000 7fa76be00640 1 -- 172.21.15.152:0/505395698 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa75c034070 con 0x7fa76c0723e0 2024-04-20T11:30:06.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:06.297+0000 7fa76be00640 1 -- 172.21.15.152:0/505395698 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa75c030800 con 0x7fa76c0723e0 2024-04-20T11:30:06.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:06.297+0000 7fa76be00640 1 -- 172.21.15.152:0/505395698 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa75c030bb0 con 0x7fa76c0723e0 2024-04-20T11:30:06.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:06.298+0000 7fa772c00640 1 -- 172.21.15.152:0/505395698 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa76c0723e0 msgr2=0x7fa76c15d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:06.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:06.298+0000 7fa772c00640 1 --2- 172.21.15.152:0/505395698 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa76c0723e0 0x7fa76c15d5c0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fa75c0097d0 tx=0x7fa75c02b940 comp rx=0 tx=0).stop 2024-04-20T11:30:06.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:06.298+0000 7fa772c00640 1 -- 172.21.15.152:0/505395698 shutdown_connections 2024-04-20T11:30:06.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:06.298+0000 7fa772c00640 1 --2- 172.21.15.152:0/505395698 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa76c0723e0 0x7fa76c15d5c0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:06.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:06.298+0000 7fa772c00640 1 -- 172.21.15.152:0/505395698 >> 172.21.15.152:0/505395698 conn(0x7fa76c067ae0 msgr2=0x7fa76c06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:06.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:06.298+0000 7fa772c00640 1 -- 172.21.15.152:0/505395698 shutdown_connections 2024-04-20T11:30:06.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:06.298+0000 7fa772c00640 1 -- 172.21.15.152:0/505395698 wait complete. 2024-04-20T11:30:06.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:06.300+0000 7fa772c00640 1 Processor -- start 2024-04-20T11:30:06.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:06.300+0000 7fa772c00640 1 -- start start 2024-04-20T11:30:06.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:06.301+0000 7fa772c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa76c0723e0 0x7fa76c140bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:06.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:06.301+0000 7fa772c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa76c15d1e0 0x7fa76c157050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:06.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:06.301+0000 7fa772c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa76c160b90 0x7fa76c152240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:06.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:06.301+0000 7fa772c00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fa76c0ccef0 con 0x7fa76c0723e0 2024-04-20T11:30:06.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:06.301+0000 7fa772c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa76c0ccd70 con 0x7fa76c160b90 2024-04-20T11:30:06.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:06.301+0000 7fa772c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa76c0cd070 con 0x7fa76c15d1e0 2024-04-20T11:30:06.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:06.301+0000 7fa771800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa76c0723e0 0x7fa76c140bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:06.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:06.301+0000 7fa771800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa76c0723e0 0x7fa76c140bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:58026/0 (socket says 172.21.15.152:58026) 2024-04-20T11:30:06.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:06.301+0000 7fa771800640 1 -- 172.21.15.152:0/1529143795 learned_addr learned my addr 172.21.15.152:0/1529143795 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:30:06.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:06.302+0000 7fa772200640 1 --2- 172.21.15.152:0/1529143795 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa76c160b90 0x7fa76c152240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:06.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:06.302+0000 7fa770e00640 1 --2- 172.21.15.152:0/1529143795 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa76c15d1e0 0x7fa76c157050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:06.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:06.302+0000 7fa771800640 1 -- 172.21.15.152:0/1529143795 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa76c15d1e0 msgr2=0x7fa76c157050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:06.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:06.302+0000 7fa771800640 1 --2- 172.21.15.152:0/1529143795 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa76c15d1e0 0x7fa76c157050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:06.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:06.302+0000 7fa771800640 1 -- 172.21.15.152:0/1529143795 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa76c160b90 msgr2=0x7fa76c152240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:06.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:06.302+0000 7fa771800640 1 --2- 172.21.15.152:0/1529143795 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa76c160b90 0x7fa76c152240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:06.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:06.302+0000 7fa771800640 1 -- 172.21.15.152:0/1529143795 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa76c157590 con 0x7fa76c0723e0 2024-04-20T11:30:06.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:06.302+0000 7fa770e00640 1 --2- 172.21.15.152:0/1529143795 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa76c15d1e0 0x7fa76c157050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:30:06.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:06.302+0000 7fa771800640 1 --2- 172.21.15.152:0/1529143795 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa76c0723e0 0x7fa76c140bb0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fa76400c500 tx=0x7fa76400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:06.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:06.302+0000 7fa769600640 1 -- 172.21.15.152:0/1529143795 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa764017070 con 0x7fa76c0723e0 2024-04-20T11:30:06.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:06.303+0000 7fa769600640 1 -- 172.21.15.152:0/1529143795 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa764005ce0 con 0x7fa76c0723e0 2024-04-20T11:30:06.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:06.303+0000 7fa769600640 1 -- 172.21.15.152:0/1529143795 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa764007330 con 0x7fa76c0723e0 2024-04-20T11:30:06.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:06.303+0000 7fa772c00640 1 -- 172.21.15.152:0/1529143795 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa76c1e7a90 con 0x7fa76c0723e0 2024-04-20T11:30:06.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:06.303+0000 7fa772c00640 1 -- 172.21.15.152:0/1529143795 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa76c1e77b0 con 0x7fa76c0723e0 2024-04-20T11:30:06.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:06.304+0000 7fa772c00640 1 -- 172.21.15.152:0/1529143795 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa76c15cd20 con 0x7fa76c0723e0 2024-04-20T11:30:06.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:06.305+0000 7fa769600640 1 -- 172.21.15.152:0/1529143795 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 8) ==== 91244+0+0 (secure 0 0 0) 0x7fa76401a070 con 0x7fa76c0723e0 2024-04-20T11:30:06.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:06.306+0000 7fa769600640 1 --2- 172.21.15.152:0/1529143795 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fa74c06cb30 0x7fa74c06eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:06.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:06.307+0000 7fa769600640 1 -- 172.21.15.152:0/1529143795 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7fa76408b450 con 0x7fa76c0723e0 2024-04-20T11:30:06.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:06.307+0000 7fa770e00640 1 --2- 172.21.15.152:0/1529143795 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fa74c06cb30 0x7fa74c06eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:06.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:06.307+0000 7fa770e00640 1 --2- 172.21.15.152:0/1529143795 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fa74c06cb30 0x7fa74c06eff0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa76c072080 tx=0x7fa758009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:06.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:06.311+0000 7fa769600640 1 -- 172.21.15.152:0/1529143795 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fa76c157590 con 0x7fa76c0723e0 2024-04-20T11:30:06.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:06.551+0000 7fa772c00640 1 -- 172.21.15.152:0/1529143795 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa76c075e60 con 0x7fa76c0723e0 2024-04-20T11:30:06.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:06.552+0000 7fa769600640 1 -- 172.21.15.152:0/1529143795 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+5208 (secure 0 0 0) 0x7fa764059590 con 0x7fa76c0723e0 2024-04-20T11:30:06.552 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:30:06.553 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4826,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6834/2855025590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2855025590},{"type":"v1","addr":"172.21.15.175:6834","nonce":2855025590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6836/3887716755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":3887716755},{"type":"v1","addr":"172.21.15.152:6836","nonce":3887716755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5712,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/698387399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6835","nonce":698387399},{"type":"v1","addr":"172.21.15.152:6837","nonce":698387399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T11:29:00.276881+0000","modified":"2024-04-20T11:29:03.352174+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5724},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5724":{"gid":5724,"name":"d","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6835/3704251089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6833","nonce":3704251089},{"type":"v1","addr":"172.21.15.175:6835","nonce":3704251089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5724,"qdb_cluster":[5724]},"id":3}]} 2024-04-20T11:30:06.553 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 25 2024-04-20T11:30:06.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:06.555+0000 7fa772c00640 1 -- 172.21.15.152:0/1529143795 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fa74c06cb30 msgr2=0x7fa74c06eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:06.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:06.555+0000 7fa772c00640 1 --2- 172.21.15.152:0/1529143795 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fa74c06cb30 0x7fa74c06eff0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa76c072080 tx=0x7fa758009040 comp rx=0 tx=0).stop 2024-04-20T11:30:06.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:06.555+0000 7fa772c00640 1 -- 172.21.15.152:0/1529143795 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa76c0723e0 msgr2=0x7fa76c140bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:06.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:06.555+0000 7fa772c00640 1 --2- 172.21.15.152:0/1529143795 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa76c0723e0 0x7fa76c140bb0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fa76400c500 tx=0x7fa76400c9c0 comp rx=0 tx=0).stop 2024-04-20T11:30:06.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:06.555+0000 7fa772c00640 1 -- 172.21.15.152:0/1529143795 shutdown_connections 2024-04-20T11:30:06.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:06.555+0000 7fa772c00640 1 --2- 172.21.15.152:0/1529143795 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fa74c06cb30 0x7fa74c06eff0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:06.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:06.555+0000 7fa772c00640 1 --2- 172.21.15.152:0/1529143795 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa76c160b90 0x7fa76c152240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:06.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:06.555+0000 7fa772c00640 1 --2- 172.21.15.152:0/1529143795 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa76c15d1e0 0x7fa76c157050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:06.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:06.555+0000 7fa772c00640 1 --2- 172.21.15.152:0/1529143795 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa76c0723e0 0x7fa76c140bb0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:06.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:06.555+0000 7fa772c00640 1 -- 172.21.15.152:0/1529143795 >> 172.21.15.152:0/1529143795 conn(0x7fa76c067ae0 msgr2=0x7fa76c06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:06.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:06.556+0000 7fa772c00640 1 -- 172.21.15.152:0/1529143795 shutdown_connections 2024-04-20T11:30:06.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:06.556+0000 7fa772c00640 1 -- 172.21.15.152:0/1529143795 wait complete. 2024-04-20T11:30:06.575 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json session ls 2024-04-20T11:30:06.806 INFO:teuthology.orchestra.run.smithi175.stdout:[{"id":6069,"entity":{"name":{"type":"client","num":6069},"addr":{"type":"any","addr":"172.21.15.152:0","nonce":83509866}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":10,"uptime":30.163531948999999,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":20.520350697353198,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.6069 172.21.15.152:0/83509866","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"a8c84f5997950fd7d008204839253291d52e6eda","ceph_version":"ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)","entity_id":"y","hostname":"smithi152","pid":"22712","root":"/"}}] 2024-04-20T11:30:06.806 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "a8c84f5997950fd7d008204839253291d52e6eda", "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi152", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "22712", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.152:0", "nonce": 83509866, "type": "any" }, "name": { "num": 6069, "type": "client" } }, "id": 6069, "inst": "client.6069 172.21.15.152:0/83509866", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 10, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 20.520350697353198 }, "state": "open", "uptime": 30.163531949 } ] 2024-04-20T11:30:06.806 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-04-20T11:30:11.807 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:30:12.006 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:12.007+0000 7fe541c00640 1 Processor -- start 2024-04-20T11:30:12.007 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:12.007+0000 7fe541c00640 1 -- start start 2024-04-20T11:30:12.008 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:12.008+0000 7fe541c00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fe53c0d1790 0x7fe53c0d3a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:12.008 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:12.008+0000 7fe541c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe53c0d1c20 0x7fe53c0d3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:12.008 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:12.008+0000 7fe541c00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe53c0cf470 con 0x7fe53c15e9b0 2024-04-20T11:30:12.008 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:12.008+0000 7fe541c00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fe53c0d0010 con 0x7fe53c0d1790 2024-04-20T11:30:12.008 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:12.008+0000 7fe541c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe53c0cf800 con 0x7fe53c0d1c20 2024-04-20T11:30:12.009 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:12.009+0000 7fe53a000640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fe53c0d1790 0x7fe53c0d3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:12.009 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:12.009+0000 7fe53b400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe53c0d1c20 0x7fe53c0d3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:12.009 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:12.009+0000 7fe53a000640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fe53c0d1790 0x7fe53c0d3a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.152:58248/0 (socket says 172.21.15.152:58248) 2024-04-20T11:30:12.009 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:12.009+0000 7fe53a000640 1 -- 172.21.15.152:0/3273704603 learned_addr learned my addr 172.21.15.152:0/3273704603 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:30:12.010 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:12.010+0000 7fe53b400640 1 -- 172.21.15.152:0/3273704603 >> v2:172.21.15.175:3300/0 conn(0x7fe53c0d1790 msgr2=0x7fe53c0d3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:12.010 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:12.010+0000 7fe53b400640 1 --2- 172.21.15.152:0/3273704603 >> v2:172.21.15.175:3300/0 conn(0x7fe53c0d1790 0x7fe53c0d3a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:12.010 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:12.010+0000 7fe53b400640 1 -- 172.21.15.152:0/3273704603 >> v1:172.21.15.175:6789/0 conn(0x7fe53c15e9b0 legacy=0x7fe53c0d1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:12.010 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:12.010+0000 7fe53b400640 1 -- 172.21.15.152:0/3273704603 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe53c0cfa50 con 0x7fe53c0d1c20 2024-04-20T11:30:12.010 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:12.010+0000 7fe53a000640 1 --2- 172.21.15.152:0/3273704603 >> v2:172.21.15.175:3300/0 conn(0x7fe53c0d1790 0x7fe53c0d3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:30:12.010 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:12.011+0000 7fe53b400640 1 --2- 172.21.15.152:0/3273704603 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe53c0d1c20 0x7fe53c0d3fc0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe534009d90 tx=0x7fe53402be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3eb04b08d55c0f34 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:12.011 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:12.012+0000 7fe539600640 1 -- 172.21.15.152:0/3273704603 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe534039070 con 0x7fe53c0d1c20 2024-04-20T11:30:12.011 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:12.012+0000 7fe539600640 1 -- 172.21.15.152:0/3273704603 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe53402cbc0 con 0x7fe53c0d1c20 2024-04-20T11:30:12.012 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:12.012+0000 7fe539600640 1 -- 172.21.15.152:0/3273704603 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe534034440 con 0x7fe53c0d1c20 2024-04-20T11:30:12.012 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:12.012+0000 7fe541c00640 1 -- 172.21.15.152:0/3273704603 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe53c0d1c20 msgr2=0x7fe53c0d3fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:12.012 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:12.012+0000 7fe541c00640 1 --2- 172.21.15.152:0/3273704603 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe53c0d1c20 0x7fe53c0d3fc0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe534009d90 tx=0x7fe53402be10 comp rx=0 tx=0).stop 2024-04-20T11:30:12.012 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:12.013+0000 7fe541c00640 1 -- 172.21.15.152:0/3273704603 shutdown_connections 2024-04-20T11:30:12.013 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:12.013+0000 7fe541c00640 1 --2- 172.21.15.152:0/3273704603 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe53c0d1c20 0x7fe53c0d3fc0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:12.013 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:12.013+0000 7fe541c00640 1 --2- 172.21.15.152:0/3273704603 >> v2:172.21.15.175:3300/0 conn(0x7fe53c0d1790 0x7fe53c0d3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:12.013 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:12.013+0000 7fe541c00640 1 -- 172.21.15.152:0/3273704603 >> 172.21.15.152:0/3273704603 conn(0x7fe53c06cc70 msgr2=0x7fe53c06d080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:12.013 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:12.013+0000 7fe541c00640 1 -- 172.21.15.152:0/3273704603 shutdown_connections 2024-04-20T11:30:12.013 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:12.013+0000 7fe541c00640 1 -- 172.21.15.152:0/3273704603 wait complete. 2024-04-20T11:30:12.014 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:12.014+0000 7fe541c00640 1 Processor -- start 2024-04-20T11:30:12.014 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:12.014+0000 7fe541c00640 1 -- start start 2024-04-20T11:30:12.015 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:12.015+0000 7fe541c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe53c0d1790 0x7fe53c1f3d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:12.015 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:12.015+0000 7fe541c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe53c0d1c20 0x7fe53c0ca9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:12.015 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:12.015+0000 7fe541c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe53c15e9b0 0x7fe53c15a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:12.015 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:12.015+0000 7fe541c00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fe53c0b3e20 con 0x7fe53c0d1790 2024-04-20T11:30:12.016 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:12.015+0000 7fe541c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe53c0b3ca0 con 0x7fe53c15e9b0 2024-04-20T11:30:12.016 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:12.015+0000 7fe541c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe53c15ed90 con 0x7fe53c0d1c20 2024-04-20T11:30:12.016 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:12.016+0000 7fe53aa00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe53c0d1790 0x7fe53c1f3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:12.016 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:12.016+0000 7fe53aa00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe53c0d1790 0x7fe53c1f3d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:58034/0 (socket says 172.21.15.152:58034) 2024-04-20T11:30:12.016 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:12.016+0000 7fe53aa00640 1 -- 172.21.15.152:0/116742320 learned_addr learned my addr 172.21.15.152:0/116742320 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:30:12.018 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:12.016+0000 7fe53a000640 1 --2- 172.21.15.152:0/116742320 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe53c0d1c20 0x7fe53c0ca9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:12.018 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:12.016+0000 7fe53aa00640 1 -- 172.21.15.152:0/116742320 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe53c0d1c20 msgr2=0x7fe53c0ca9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:12.018 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:12.016+0000 7fe53aa00640 1 --2- 172.21.15.152:0/116742320 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe53c0d1c20 0x7fe53c0ca9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:12.018 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:12.016+0000 7fe53b400640 1 --2- 172.21.15.152:0/116742320 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe53c15e9b0 0x7fe53c15a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:12.018 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:12.016+0000 7fe53aa00640 1 -- 172.21.15.152:0/116742320 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe53c15e9b0 msgr2=0x7fe53c15a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:12.018 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:12.016+0000 7fe53aa00640 1 --2- 172.21.15.152:0/116742320 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe53c15e9b0 0x7fe53c15a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:12.018 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:12.016+0000 7fe53aa00640 1 -- 172.21.15.152:0/116742320 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe53c1f7930 con 0x7fe53c0d1790 2024-04-20T11:30:12.018 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:12.017+0000 7fe53aa00640 1 --2- 172.21.15.152:0/116742320 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe53c0d1790 0x7fe53c1f3d30 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fe52c00c3c0 tx=0x7fe52c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:12.019 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:12.017+0000 7fe532a00640 1 -- 172.21.15.152:0/116742320 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe52c0052d0 con 0x7fe53c0d1790 2024-04-20T11:30:12.019 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:12.017+0000 7fe532a00640 1 -- 172.21.15.152:0/116742320 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe52c005490 con 0x7fe53c0d1790 2024-04-20T11:30:12.019 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:12.017+0000 7fe541c00640 1 -- 172.21.15.152:0/116742320 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe53c1f7c10 con 0x7fe53c0d1790 2024-04-20T11:30:12.019 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:12.017+0000 7fe532a00640 1 -- 172.21.15.152:0/116742320 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe52c007260 con 0x7fe53c0d1790 2024-04-20T11:30:12.019 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:12.017+0000 7fe541c00640 1 -- 172.21.15.152:0/116742320 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe53c1f81d0 con 0x7fe53c0d1790 2024-04-20T11:30:12.019 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:12.018+0000 7fe541c00640 1 -- 172.21.15.152:0/116742320 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe53c069100 con 0x7fe53c0d1790 2024-04-20T11:30:12.019 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:12.019+0000 7fe532a00640 1 -- 172.21.15.152:0/116742320 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 8) ==== 91244+0+0 (secure 0 0 0) 0x7fe52c007400 con 0x7fe53c0d1790 2024-04-20T11:30:12.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:12.020+0000 7fe532a00640 1 --2- 172.21.15.152:0/116742320 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe51c06cb30 0x7fe51c06eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:12.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:12.021+0000 7fe532a00640 1 -- 172.21.15.152:0/116742320 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7fe52c08b470 con 0x7fe53c0d1790 2024-04-20T11:30:12.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:12.021+0000 7fe53a000640 1 --2- 172.21.15.152:0/116742320 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe51c06cb30 0x7fe51c06eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:12.021 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:12.022+0000 7fe53a000640 1 --2- 172.21.15.152:0/116742320 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe51c06cb30 0x7fe51c06eff0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fe53c0b7b20 tx=0x7fe528008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:12.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:12.026+0000 7fe532a00640 1 -- 172.21.15.152:0/116742320 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe52c059530 con 0x7fe53c0d1790 2024-04-20T11:30:12.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:12.264+0000 7fe541c00640 1 -- 172.21.15.152:0/116742320 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe53c160fd0 con 0x7fe53c0d1790 2024-04-20T11:30:12.265 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:12.265+0000 7fe532a00640 1 -- 172.21.15.152:0/116742320 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+5208 (secure 0 0 0) 0x7fe52c058e30 con 0x7fe53c0d1790 2024-04-20T11:30:12.265 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:30:12.265 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4826,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6834/2855025590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2855025590},{"type":"v1","addr":"172.21.15.175:6834","nonce":2855025590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6836/3887716755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":3887716755},{"type":"v1","addr":"172.21.15.152:6836","nonce":3887716755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5712,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/698387399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6835","nonce":698387399},{"type":"v1","addr":"172.21.15.152:6837","nonce":698387399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T11:29:00.276881+0000","modified":"2024-04-20T11:29:03.352174+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5724},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5724":{"gid":5724,"name":"d","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6835/3704251089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6833","nonce":3704251089},{"type":"v1","addr":"172.21.15.175:6835","nonce":3704251089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5724,"qdb_cluster":[5724]},"id":3}]} 2024-04-20T11:30:12.265 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 25 2024-04-20T11:30:12.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:12.268+0000 7fe541c00640 1 -- 172.21.15.152:0/116742320 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe51c06cb30 msgr2=0x7fe51c06eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:12.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:12.268+0000 7fe541c00640 1 --2- 172.21.15.152:0/116742320 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe51c06cb30 0x7fe51c06eff0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fe53c0b7b20 tx=0x7fe528008040 comp rx=0 tx=0).stop 2024-04-20T11:30:12.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:12.268+0000 7fe541c00640 1 -- 172.21.15.152:0/116742320 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe53c0d1790 msgr2=0x7fe53c1f3d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:12.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:12.268+0000 7fe541c00640 1 --2- 172.21.15.152:0/116742320 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe53c0d1790 0x7fe53c1f3d30 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fe52c00c3c0 tx=0x7fe52c00c880 comp rx=0 tx=0).stop 2024-04-20T11:30:12.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:12.268+0000 7fe541c00640 1 -- 172.21.15.152:0/116742320 shutdown_connections 2024-04-20T11:30:12.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:12.268+0000 7fe541c00640 1 --2- 172.21.15.152:0/116742320 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe51c06cb30 0x7fe51c06eff0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:12.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:12.268+0000 7fe541c00640 1 --2- 172.21.15.152:0/116742320 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe53c15e9b0 0x7fe53c15a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:12.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:12.268+0000 7fe541c00640 1 --2- 172.21.15.152:0/116742320 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe53c0d1c20 0x7fe53c0ca9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:12.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:12.268+0000 7fe541c00640 1 --2- 172.21.15.152:0/116742320 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe53c0d1790 0x7fe53c1f3d30 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:12.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:12.268+0000 7fe541c00640 1 -- 172.21.15.152:0/116742320 >> 172.21.15.152:0/116742320 conn(0x7fe53c06cc70 msgr2=0x7fe53c06d050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:12.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:12.268+0000 7fe541c00640 1 -- 172.21.15.152:0/116742320 shutdown_connections 2024-04-20T11:30:12.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:12.268+0000 7fe541c00640 1 -- 172.21.15.152:0/116742320 wait complete. 2024-04-20T11:30:12.287 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json session ls 2024-04-20T11:30:12.518 INFO:teuthology.orchestra.run.smithi175.stdout:[{"id":6069,"entity":{"name":{"type":"client","num":6069},"addr":{"type":"any","addr":"172.21.15.152:0","nonce":83509866}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":9,"uptime":35.876443248999998,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":20.251269497110499,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.6069 172.21.15.152:0/83509866","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"a8c84f5997950fd7d008204839253291d52e6eda","ceph_version":"ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)","entity_id":"y","hostname":"smithi152","pid":"22712","root":"/"}}] 2024-04-20T11:30:12.519 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "a8c84f5997950fd7d008204839253291d52e6eda", "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi152", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "22712", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.152:0", "nonce": 83509866, "type": "any" }, "name": { "num": 6069, "type": "client" } }, "id": 6069, "inst": "client.6069 172.21.15.152:0/83509866", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 9, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 20.2512694971105 }, "state": "open", "uptime": 35.876443249 } ] 2024-04-20T11:30:12.519 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-04-20T11:30:17.520 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:30:17.731 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:17.731+0000 7fbfadc00640 1 Processor -- start 2024-04-20T11:30:17.731 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:17.731+0000 7fbfadc00640 1 -- start start 2024-04-20T11:30:17.732 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:17.733+0000 7fbfadc00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fbfa815d1e0 0x7fbfa8160b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:17.733 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:17.733+0000 7fbfadc00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbfa8071cd0 0x7fbfa8072160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:17.733 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:17.733+0000 7fbfadc00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbfa80ce8c0 con 0x7fbfa815d5c0 2024-04-20T11:30:17.733 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:17.733+0000 7fbfadc00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fbfa80726a0 con 0x7fbfa815d1e0 2024-04-20T11:30:17.733 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:17.733+0000 7fbfadc00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fbfa81600b0 con 0x7fbfa8071cd0 2024-04-20T11:30:17.733 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:17.733+0000 7fbfa7e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fbfa815d1e0 0x7fbfa8160b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:17.733 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:17.733+0000 7fbfa7e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fbfa815d1e0 0x7fbfa8160b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:51620/0 (socket says 172.21.15.152:51620) 2024-04-20T11:30:17.733 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:17.733+0000 7fbfa7e00640 1 -- 172.21.15.152:0/130652114 learned_addr learned my addr 172.21.15.152:0/130652114 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:30:17.734 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:17.733+0000 7fbfad200640 1 --2- 172.21.15.152:0/130652114 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbfa8071cd0 0x7fbfa8072160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:17.734 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:17.734+0000 7fbfa7e00640 1 -- 172.21.15.152:0/130652114 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbfa8071cd0 msgr2=0x7fbfa8072160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:17.734 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:17.734+0000 7fbfa7e00640 1 --2- 172.21.15.152:0/130652114 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbfa8071cd0 0x7fbfa8072160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:17.734 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:17.734+0000 7fbfa7e00640 1 -- 172.21.15.152:0/130652114 >> v1:172.21.15.175:6789/0 conn(0x7fbfa815d5c0 legacy=0x7fbfa80cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:17.734 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:17.734+0000 7fbfa7e00640 1 -- 172.21.15.152:0/130652114 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfa8160390 con 0x7fbfa815d1e0 2024-04-20T11:30:17.735 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:17.735+0000 7fbfa7e00640 1 --2- 172.21.15.152:0/130652114 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbfa815d1e0 0x7fbfa8160b90 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fbf980098a0 tx=0x7fbf9802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbd1b5133a4a720c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:17.735 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:17.735+0000 7fbfa7400640 1 -- 172.21.15.152:0/130652114 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf98035020 con 0x7fbfa815d1e0 2024-04-20T11:30:17.735 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:17.735+0000 7fbfa7400640 1 -- 172.21.15.152:0/130652114 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf98031640 con 0x7fbfa815d1e0 2024-04-20T11:30:17.735 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:17.735+0000 7fbfa7400640 1 -- 172.21.15.152:0/130652114 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf9802d040 con 0x7fbfa815d1e0 2024-04-20T11:30:17.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:17.736+0000 7fbfadc00640 1 -- 172.21.15.152:0/130652114 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbfa815d1e0 msgr2=0x7fbfa8160b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:17.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:17.736+0000 7fbfadc00640 1 --2- 172.21.15.152:0/130652114 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbfa815d1e0 0x7fbfa8160b90 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fbf980098a0 tx=0x7fbf9802b8f0 comp rx=0 tx=0).stop 2024-04-20T11:30:17.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:17.736+0000 7fbfadc00640 1 -- 172.21.15.152:0/130652114 shutdown_connections 2024-04-20T11:30:17.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:17.736+0000 7fbfadc00640 1 --2- 172.21.15.152:0/130652114 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbfa8071cd0 0x7fbfa8072160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:17.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:17.736+0000 7fbfadc00640 1 --2- 172.21.15.152:0/130652114 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbfa815d1e0 0x7fbfa8160b90 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:17.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:17.736+0000 7fbfadc00640 1 -- 172.21.15.152:0/130652114 >> 172.21.15.152:0/130652114 conn(0x7fbfa8067ae0 msgr2=0x7fbfa806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:17.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:17.736+0000 7fbfadc00640 1 -- 172.21.15.152:0/130652114 shutdown_connections 2024-04-20T11:30:17.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:17.736+0000 7fbfadc00640 1 -- 172.21.15.152:0/130652114 wait complete. 2024-04-20T11:30:17.738 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:17.738+0000 7fbfadc00640 1 Processor -- start 2024-04-20T11:30:17.738 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:17.739+0000 7fbfadc00640 1 -- start start 2024-04-20T11:30:17.741 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:17.740+0000 7fbfadc00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbfa8071cd0 0x7fbfa81447c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:17.741 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:17.740+0000 7fbfadc00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbfa815d1e0 0x7fbfa8147920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:17.741 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:17.740+0000 7fbfadc00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbfa815d5c0 0x7fbfa80b42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:17.741 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:17.740+0000 7fbfadc00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fbfa80ccd70 con 0x7fbfa815d5c0 2024-04-20T11:30:17.741 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:17.740+0000 7fbfadc00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbfa80ceba0 con 0x7fbfa815d1e0 2024-04-20T11:30:17.741 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:17.740+0000 7fbfadc00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fbfa80ccef0 con 0x7fbfa8071cd0 2024-04-20T11:30:17.741 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:17.740+0000 7fbfad200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbfa815d5c0 0x7fbfa80b42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:17.741 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:17.740+0000 7fbfad200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbfa815d5c0 0x7fbfa80b42d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:51634/0 (socket says 172.21.15.152:51634) 2024-04-20T11:30:17.742 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:17.740+0000 7fbfad200640 1 -- 172.21.15.152:0/2492799625 learned_addr learned my addr 172.21.15.152:0/2492799625 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:30:17.742 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:17.740+0000 7fbfa7e00640 1 --2- 172.21.15.152:0/2492799625 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbfa8071cd0 0x7fbfa81447c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:17.742 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:17.741+0000 7fbf9f400640 1 --2- 172.21.15.152:0/2492799625 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbfa815d1e0 0x7fbfa8147920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:17.742 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:17.741+0000 7fbfad200640 1 -- 172.21.15.152:0/2492799625 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbfa8071cd0 msgr2=0x7fbfa81447c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:17.742 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:17.741+0000 7fbfad200640 1 --2- 172.21.15.152:0/2492799625 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbfa8071cd0 0x7fbfa81447c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:17.742 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:17.741+0000 7fbfad200640 1 -- 172.21.15.152:0/2492799625 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbfa815d1e0 msgr2=0x7fbfa8147920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:17.742 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:17.741+0000 7fbfad200640 1 --2- 172.21.15.152:0/2492799625 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbfa815d1e0 0x7fbfa8147920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:17.742 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:17.741+0000 7fbfad200640 1 -- 172.21.15.152:0/2492799625 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfa8155c90 con 0x7fbfa815d5c0 2024-04-20T11:30:17.742 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:17.741+0000 7fbf9f400640 1 --2- 172.21.15.152:0/2492799625 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbfa815d1e0 0x7fbfa8147920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:30:17.742 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:17.741+0000 7fbfad200640 1 --2- 172.21.15.152:0/2492799625 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbfa815d5c0 0x7fbfa80b42d0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fbfa0007e40 tx=0x7fbfa000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:17.742 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:17.741+0000 7fbfa4c00640 1 -- 172.21.15.152:0/2492799625 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa0014620 con 0x7fbfa815d5c0 2024-04-20T11:30:17.743 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:17.741+0000 7fbfa4c00640 1 -- 172.21.15.152:0/2492799625 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfa0019070 con 0x7fbfa815d5c0 2024-04-20T11:30:17.743 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:17.741+0000 7fbfadc00640 1 -- 172.21.15.152:0/2492799625 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfa8155f70 con 0x7fbfa815d5c0 2024-04-20T11:30:17.743 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:17.741+0000 7fbfa4c00640 1 -- 172.21.15.152:0/2492799625 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa0014a20 con 0x7fbfa815d5c0 2024-04-20T11:30:17.743 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:17.742+0000 7fbfadc00640 1 -- 172.21.15.152:0/2492799625 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfa8156530 con 0x7fbfa815d5c0 2024-04-20T11:30:17.743 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:17.743+0000 7fbfadc00640 1 -- 172.21.15.152:0/2492799625 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfa8075e60 con 0x7fbfa815d5c0 2024-04-20T11:30:17.743 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:17.744+0000 7fbfa4c00640 1 -- 172.21.15.152:0/2492799625 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 8) ==== 91244+0+0 (secure 0 0 0) 0x7fbfa0014bc0 con 0x7fbfa815d5c0 2024-04-20T11:30:17.744 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:17.745+0000 7fbfa4c00640 1 --2- 172.21.15.152:0/2492799625 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fbf8406cb60 0x7fbf8406f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:17.744 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:17.745+0000 7fbfa4c00640 1 -- 172.21.15.152:0/2492799625 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7fbfa008c860 con 0x7fbfa815d5c0 2024-04-20T11:30:17.744 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:17.745+0000 7fbfa7e00640 1 --2- 172.21.15.152:0/2492799625 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fbf8406cb60 0x7fbf8406f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:17.745 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:17.746+0000 7fbfa7e00640 1 --2- 172.21.15.152:0/2492799625 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fbf8406cb60 0x7fbf8406f020 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fbf98036000 tx=0x7fbf9803c040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:17.749 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:17.750+0000 7fbfa4c00640 1 -- 172.21.15.152:0/2492799625 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fbfa005a920 con 0x7fbfa815d5c0 2024-04-20T11:30:17.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:17.988+0000 7fbfadc00640 1 -- 172.21.15.152:0/2492799625 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbfa8072c90 con 0x7fbfa815d5c0 2024-04-20T11:30:17.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:17.989+0000 7fbfa4c00640 1 -- 172.21.15.152:0/2492799625 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+5208 (secure 0 0 0) 0x7fbfa005a220 con 0x7fbfa815d5c0 2024-04-20T11:30:17.989 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:30:17.989 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4826,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6834/2855025590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2855025590},{"type":"v1","addr":"172.21.15.175:6834","nonce":2855025590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6836/3887716755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":3887716755},{"type":"v1","addr":"172.21.15.152:6836","nonce":3887716755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5712,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/698387399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6835","nonce":698387399},{"type":"v1","addr":"172.21.15.152:6837","nonce":698387399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T11:29:00.276881+0000","modified":"2024-04-20T11:29:03.352174+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5724},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5724":{"gid":5724,"name":"d","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6835/3704251089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6833","nonce":3704251089},{"type":"v1","addr":"172.21.15.175:6835","nonce":3704251089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5724,"qdb_cluster":[5724]},"id":3}]} 2024-04-20T11:30:17.989 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 25 2024-04-20T11:30:17.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:17.991+0000 7fbfadc00640 1 -- 172.21.15.152:0/2492799625 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fbf8406cb60 msgr2=0x7fbf8406f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:17.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:17.991+0000 7fbfadc00640 1 --2- 172.21.15.152:0/2492799625 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fbf8406cb60 0x7fbf8406f020 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fbf98036000 tx=0x7fbf9803c040 comp rx=0 tx=0).stop 2024-04-20T11:30:17.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:17.991+0000 7fbfadc00640 1 -- 172.21.15.152:0/2492799625 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbfa815d5c0 msgr2=0x7fbfa80b42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:17.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:17.991+0000 7fbfadc00640 1 --2- 172.21.15.152:0/2492799625 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbfa815d5c0 0x7fbfa80b42d0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fbfa0007e40 tx=0x7fbfa000e4b0 comp rx=0 tx=0).stop 2024-04-20T11:30:17.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:17.992+0000 7fbfadc00640 1 -- 172.21.15.152:0/2492799625 shutdown_connections 2024-04-20T11:30:17.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:17.992+0000 7fbfadc00640 1 --2- 172.21.15.152:0/2492799625 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fbf8406cb60 0x7fbf8406f020 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:17.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:17.992+0000 7fbfadc00640 1 --2- 172.21.15.152:0/2492799625 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbfa815d5c0 0x7fbfa80b42d0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:17.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:17.992+0000 7fbfadc00640 1 --2- 172.21.15.152:0/2492799625 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbfa815d1e0 0x7fbfa8147920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:17.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:17.992+0000 7fbfadc00640 1 --2- 172.21.15.152:0/2492799625 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbfa8071cd0 0x7fbfa81447c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:17.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:17.992+0000 7fbfadc00640 1 -- 172.21.15.152:0/2492799625 >> 172.21.15.152:0/2492799625 conn(0x7fbfa8067ae0 msgr2=0x7fbfa80b0ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:17.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:17.992+0000 7fbfadc00640 1 -- 172.21.15.152:0/2492799625 shutdown_connections 2024-04-20T11:30:17.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:17.992+0000 7fbfadc00640 1 -- 172.21.15.152:0/2492799625 wait complete. 2024-04-20T11:30:18.011 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json session ls 2024-04-20T11:30:18.242 INFO:teuthology.orchestra.run.smithi175.stdout:[{"id":6069,"entity":{"name":{"type":"client","num":6069},"addr":{"type":"any","addr":"172.21.15.152:0","nonce":83509866}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":9,"uptime":41.599354394999999,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":19.98525497468184,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.6069 172.21.15.152:0/83509866","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"a8c84f5997950fd7d008204839253291d52e6eda","ceph_version":"ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)","entity_id":"y","hostname":"smithi152","pid":"22712","root":"/"}}] 2024-04-20T11:30:18.243 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "a8c84f5997950fd7d008204839253291d52e6eda", "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi152", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "22712", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.152:0", "nonce": 83509866, "type": "any" }, "name": { "num": 6069, "type": "client" } }, "id": 6069, "inst": "client.6069 172.21.15.152:0/83509866", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 9, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 19.98525497468184 }, "state": "open", "uptime": 41.599354395 } ] 2024-04-20T11:30:18.243 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-04-20T11:30:21.884 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:30:21.892 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:30:23.244 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:30:23.440 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:23.441+0000 7f4a87200640 1 Processor -- start 2024-04-20T11:30:23.441 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:23.441+0000 7f4a87200640 1 -- start start 2024-04-20T11:30:23.442 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:23.442+0000 7f4a87200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f4a8813e530 0x7f4a8806ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:23.442 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:23.442+0000 7f4a87200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f4a8806f340 0x7f4a88140dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:23.442 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:23.442+0000 7f4a87200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4a88141310 0x7f4a8815efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:23.442 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:23.442+0000 7f4a87200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f4a88141b00 con 0x7f4a8813e530 2024-04-20T11:30:23.443 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:23.442+0000 7f4a87200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f4a880cfc50 con 0x7f4a8806f340 2024-04-20T11:30:23.443 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:23.442+0000 7f4a87200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4a880cf0a0 con 0x7f4a88141310 2024-04-20T11:30:23.443 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:23.443+0000 7f4a85e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f4a8813e530 0x7f4a8806ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:23.443 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:23.443+0000 7f4a85e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f4a8813e530 0x7f4a8806ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:51646/0 (socket says 172.21.15.152:51646) 2024-04-20T11:30:23.443 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:23.443+0000 7f4a85e00640 1 -- 172.21.15.152:0/1565010318 learned_addr learned my addr 172.21.15.152:0/1565010318 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:30:23.444 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:23.443+0000 7f4a7d400640 1 --2- 172.21.15.152:0/1565010318 >> v2:172.21.15.175:3300/0 conn(0x7f4a8806f340 0x7f4a88140dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:23.444 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:23.443+0000 7f4a86800640 1 --2- 172.21.15.152:0/1565010318 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4a88141310 0x7f4a8815efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:23.444 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:23.443+0000 7f4a85e00640 1 -- 172.21.15.152:0/1565010318 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4a88141310 msgr2=0x7f4a8815efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:23.444 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:23.443+0000 7f4a85e00640 1 --2- 172.21.15.152:0/1565010318 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4a88141310 0x7f4a8815efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:23.444 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:23.443+0000 7f4a85e00640 1 -- 172.21.15.152:0/1565010318 >> v2:172.21.15.175:3300/0 conn(0x7f4a8806f340 msgr2=0x7f4a88140dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:23.444 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:23.443+0000 7f4a85e00640 1 --2- 172.21.15.152:0/1565010318 >> v2:172.21.15.175:3300/0 conn(0x7f4a8806f340 0x7f4a88140dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:23.444 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:23.443+0000 7f4a85e00640 1 -- 172.21.15.152:0/1565010318 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a880d00e0 con 0x7f4a8813e530 2024-04-20T11:30:23.444 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:23.443+0000 7f4a7d400640 1 --2- 172.21.15.152:0/1565010318 >> v2:172.21.15.175:3300/0 conn(0x7f4a8806f340 0x7f4a88140dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:30:23.444 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:23.444+0000 7f4a85e00640 1 --2- 172.21.15.152:0/1565010318 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4a8813e530 0x7f4a8806ee00 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f4a8806feb0 tx=0x7f4a7802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7d41ec3af130101 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:23.446 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:23.445+0000 7f4a85400640 1 -- 172.21.15.152:0/1565010318 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a78035020 con 0x7f4a8813e530 2024-04-20T11:30:23.446 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:23.445+0000 7f4a85400640 1 -- 172.21.15.152:0/1565010318 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a78031870 con 0x7f4a8813e530 2024-04-20T11:30:23.446 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:23.445+0000 7f4a85400640 1 -- 172.21.15.152:0/1565010318 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a78031c40 con 0x7f4a8813e530 2024-04-20T11:30:23.446 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:23.445+0000 7f4a87200640 1 -- 172.21.15.152:0/1565010318 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4a8813e530 msgr2=0x7f4a8806ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:23.446 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:23.445+0000 7f4a87200640 1 --2- 172.21.15.152:0/1565010318 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4a8813e530 0x7f4a8806ee00 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f4a8806feb0 tx=0x7f4a7802b910 comp rx=0 tx=0).stop 2024-04-20T11:30:23.446 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:23.446+0000 7f4a87200640 1 -- 172.21.15.152:0/1565010318 shutdown_connections 2024-04-20T11:30:23.446 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:23.446+0000 7f4a87200640 1 --2- 172.21.15.152:0/1565010318 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4a88141310 0x7f4a8815efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:23.446 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:23.446+0000 7f4a87200640 1 --2- 172.21.15.152:0/1565010318 >> v2:172.21.15.175:3300/0 conn(0x7f4a8806f340 0x7f4a88140dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:23.446 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:23.446+0000 7f4a87200640 1 --2- 172.21.15.152:0/1565010318 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4a8813e530 0x7f4a8806ee00 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:23.446 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:23.446+0000 7f4a87200640 1 -- 172.21.15.152:0/1565010318 >> 172.21.15.152:0/1565010318 conn(0x7f4a88067ae0 msgr2=0x7f4a8806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:23.446 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:23.446+0000 7f4a87200640 1 -- 172.21.15.152:0/1565010318 shutdown_connections 2024-04-20T11:30:23.447 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:23.446+0000 7f4a87200640 1 -- 172.21.15.152:0/1565010318 wait complete. 2024-04-20T11:30:23.447 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:23.447+0000 7f4a87200640 1 Processor -- start 2024-04-20T11:30:23.447 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:23.448+0000 7f4a87200640 1 -- start start 2024-04-20T11:30:23.449 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:23.448+0000 7f4a87200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4a8806f340 0x7f4a88138600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:23.450 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:23.448+0000 7f4a87200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4a8813e530 0x7f4a88140510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:23.450 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:23.448+0000 7f4a87200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4a88141310 0x7f4a88154dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:23.450 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:23.448+0000 7f4a87200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f4a880cf8e0 con 0x7f4a88141310 2024-04-20T11:30:23.450 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:23.448+0000 7f4a87200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4a880cf600 con 0x7f4a8806f340 2024-04-20T11:30:23.450 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:23.448+0000 7f4a87200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4a881417a0 con 0x7f4a8813e530 2024-04-20T11:30:23.450 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:23.449+0000 7f4a86800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4a88141310 0x7f4a88154dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:23.450 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:23.449+0000 7f4a86800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4a88141310 0x7f4a88154dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:51654/0 (socket says 172.21.15.152:51654) 2024-04-20T11:30:23.450 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:23.449+0000 7f4a86800640 1 -- 172.21.15.152:0/2311447570 learned_addr learned my addr 172.21.15.152:0/2311447570 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:30:23.450 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:23.449+0000 7f4a85e00640 1 --2- 172.21.15.152:0/2311447570 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4a8806f340 0x7f4a88138600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:23.450 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:23.449+0000 7f4a7d400640 1 --2- 172.21.15.152:0/2311447570 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4a8813e530 0x7f4a88140510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:23.450 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:23.449+0000 7f4a86800640 1 -- 172.21.15.152:0/2311447570 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4a8813e530 msgr2=0x7f4a88140510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:23.450 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:23.449+0000 7f4a86800640 1 --2- 172.21.15.152:0/2311447570 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4a8813e530 0x7f4a88140510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:23.450 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:23.449+0000 7f4a86800640 1 -- 172.21.15.152:0/2311447570 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4a8806f340 msgr2=0x7f4a88138600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:23.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:23.449+0000 7f4a86800640 1 --2- 172.21.15.152:0/2311447570 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4a8806f340 0x7f4a88138600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:23.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:23.449+0000 7f4a86800640 1 -- 172.21.15.152:0/2311447570 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a88140a50 con 0x7f4a88141310 2024-04-20T11:30:23.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:23.450+0000 7f4a86800640 1 --2- 172.21.15.152:0/2311447570 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4a88141310 0x7f4a88154dc0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f4a8000cc60 tx=0x7f4a8000a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:23.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:23.450+0000 7f4a7ea00640 1 -- 172.21.15.152:0/2311447570 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a80003990 con 0x7f4a88141310 2024-04-20T11:30:23.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:23.450+0000 7f4a7ea00640 1 -- 172.21.15.152:0/2311447570 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a80003b30 con 0x7f4a88141310 2024-04-20T11:30:23.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:23.450+0000 7f4a87200640 1 -- 172.21.15.152:0/2311447570 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a88140cf0 con 0x7f4a88141310 2024-04-20T11:30:23.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:23.450+0000 7f4a7ea00640 1 -- 172.21.15.152:0/2311447570 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a80004020 con 0x7f4a88141310 2024-04-20T11:30:23.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:23.450+0000 7f4a87200640 1 -- 172.21.15.152:0/2311447570 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a8813b7b0 con 0x7f4a88141310 2024-04-20T11:30:23.452 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:23.451+0000 7f4a87200640 1 -- 172.21.15.152:0/2311447570 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a88075e60 con 0x7f4a88141310 2024-04-20T11:30:23.452 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:23.452+0000 7f4a7ea00640 1 -- 172.21.15.152:0/2311447570 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 8) ==== 91244+0+0 (secure 0 0 0) 0x7f4a800041c0 con 0x7f4a88141310 2024-04-20T11:30:23.452 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:23.453+0000 7f4a7ea00640 1 --2- 172.21.15.152:0/2311447570 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f4a6406cb30 0x7f4a6406eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:23.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:23.453+0000 7f4a7ea00640 1 -- 172.21.15.152:0/2311447570 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7f4a8008c580 con 0x7f4a88141310 2024-04-20T11:30:23.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:23.453+0000 7f4a85e00640 1 --2- 172.21.15.152:0/2311447570 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f4a6406cb30 0x7f4a6406eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:23.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:23.454+0000 7f4a85e00640 1 --2- 172.21.15.152:0/2311447570 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f4a6406cb30 0x7f4a6406eff0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f4a78036000 tx=0x7f4a7803c040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:23.457 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:23.458+0000 7f4a7ea00640 1 -- 172.21.15.152:0/2311447570 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f4a80014030 con 0x7f4a88141310 2024-04-20T11:30:23.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:23.700+0000 7f4a87200640 1 -- 172.21.15.152:0/2311447570 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4a8815e9a0 con 0x7f4a88141310 2024-04-20T11:30:23.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:23.701+0000 7f4a7ea00640 1 -- 172.21.15.152:0/2311447570 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+5208 (secure 0 0 0) 0x7f4a8005a4f0 con 0x7f4a88141310 2024-04-20T11:30:23.701 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:30:23.701 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4826,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6834/2855025590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2855025590},{"type":"v1","addr":"172.21.15.175:6834","nonce":2855025590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6836/3887716755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":3887716755},{"type":"v1","addr":"172.21.15.152:6836","nonce":3887716755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5712,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/698387399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6835","nonce":698387399},{"type":"v1","addr":"172.21.15.152:6837","nonce":698387399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T11:29:00.276881+0000","modified":"2024-04-20T11:29:03.352174+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5724},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5724":{"gid":5724,"name":"d","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6835/3704251089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6833","nonce":3704251089},{"type":"v1","addr":"172.21.15.175:6835","nonce":3704251089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5724,"qdb_cluster":[5724]},"id":3}]} 2024-04-20T11:30:23.701 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 25 2024-04-20T11:30:23.703 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:23.704+0000 7f4a87200640 1 -- 172.21.15.152:0/2311447570 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f4a6406cb30 msgr2=0x7f4a6406eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:23.704 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:23.704+0000 7f4a87200640 1 --2- 172.21.15.152:0/2311447570 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f4a6406cb30 0x7f4a6406eff0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f4a78036000 tx=0x7f4a7803c040 comp rx=0 tx=0).stop 2024-04-20T11:30:23.704 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:23.704+0000 7f4a87200640 1 -- 172.21.15.152:0/2311447570 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4a88141310 msgr2=0x7f4a88154dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:23.704 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:23.704+0000 7f4a87200640 1 --2- 172.21.15.152:0/2311447570 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4a88141310 0x7f4a88154dc0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f4a8000cc60 tx=0x7f4a8000a5c0 comp rx=0 tx=0).stop 2024-04-20T11:30:23.704 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:23.704+0000 7f4a87200640 1 -- 172.21.15.152:0/2311447570 shutdown_connections 2024-04-20T11:30:23.704 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:23.704+0000 7f4a87200640 1 --2- 172.21.15.152:0/2311447570 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f4a6406cb30 0x7f4a6406eff0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:23.704 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:23.704+0000 7f4a87200640 1 --2- 172.21.15.152:0/2311447570 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4a88141310 0x7f4a88154dc0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:23.704 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:23.704+0000 7f4a87200640 1 --2- 172.21.15.152:0/2311447570 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4a8813e530 0x7f4a88140510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:23.704 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:23.704+0000 7f4a87200640 1 --2- 172.21.15.152:0/2311447570 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4a8806f340 0x7f4a88138600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:23.704 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:23.704+0000 7f4a87200640 1 -- 172.21.15.152:0/2311447570 >> 172.21.15.152:0/2311447570 conn(0x7f4a88067ae0 msgr2=0x7f4a8813d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:23.705 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:23.704+0000 7f4a87200640 1 -- 172.21.15.152:0/2311447570 shutdown_connections 2024-04-20T11:30:23.705 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:23.704+0000 7f4a87200640 1 -- 172.21.15.152:0/2311447570 wait complete. 2024-04-20T11:30:23.724 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json session ls 2024-04-20T11:30:23.955 INFO:teuthology.orchestra.run.smithi175.stdout:[{"id":6069,"entity":{"name":{"type":"client","num":6069},"addr":{"type":"any","addr":"172.21.15.152:0","nonce":83509866}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":8,"uptime":47.312265693000001,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":19.723190428457137,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.6069 172.21.15.152:0/83509866","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"a8c84f5997950fd7d008204839253291d52e6eda","ceph_version":"ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)","entity_id":"y","hostname":"smithi152","pid":"22712","root":"/"}}] 2024-04-20T11:30:23.956 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "a8c84f5997950fd7d008204839253291d52e6eda", "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi152", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "22712", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.152:0", "nonce": 83509866, "type": "any" }, "name": { "num": 6069, "type": "client" } }, "id": 6069, "inst": "client.6069 172.21.15.152:0/83509866", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 8, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 19.723190428457137 }, "state": "open", "uptime": 47.312265693 } ] 2024-04-20T11:30:23.956 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-04-20T11:30:28.957 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:30:29.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:29.169+0000 7fd517e00640 1 Processor -- start 2024-04-20T11:30:29.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:29.169+0000 7fd517e00640 1 -- start start 2024-04-20T11:30:29.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:29.170+0000 7fd517e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fd51813e530 0x7fd51806ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:29.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:29.170+0000 7fd517e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd51806f340 0x7fd518140dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:29.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:29.171+0000 7fd517e00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5180cf0a0 con 0x7fd518141310 2024-04-20T11:30:29.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:29.171+0000 7fd517e00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fd518141b00 con 0x7fd51813e530 2024-04-20T11:30:29.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:29.171+0000 7fd517e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd5180cfc50 con 0x7fd51806f340 2024-04-20T11:30:29.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:29.171+0000 7fd517400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fd518141310 0x7fd51815efd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:54728/0 (socket says 172.21.15.152:54728) 2024-04-20T11:30:29.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:29.171+0000 7fd517400640 1 -- 172.21.15.152:0/1489536907 learned_addr learned my addr 172.21.15.152:0/1489536907 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:30:29.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:29.171+0000 7fd516a00640 1 --2- 172.21.15.152:0/1489536907 >> v2:172.21.15.175:3300/0 conn(0x7fd51813e530 0x7fd51806ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:29.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:29.171+0000 7fd50e000640 1 --2- 172.21.15.152:0/1489536907 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd51806f340 0x7fd518140dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:29.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:29.171+0000 7fd516000640 1 -- 172.21.15.152:0/1489536907 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4058581132 0 0) 0x7fd510001490 con 0x7fd518141310 2024-04-20T11:30:29.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:29.171+0000 7fd516000640 1 -- 172.21.15.152:0/1489536907 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd4f8003520 con 0x7fd518141310 2024-04-20T11:30:29.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:29.172+0000 7fd516000640 1 -- 172.21.15.152:0/1489536907 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3220051033 0 0) 0x7fd4f8003520 con 0x7fd518141310 2024-04-20T11:30:29.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:29.172+0000 7fd516000640 1 -- 172.21.15.152:0/1489536907 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd51806f340 msgr2=0x7fd518140dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:29.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:29.172+0000 7fd516000640 1 --2- 172.21.15.152:0/1489536907 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd51806f340 0x7fd518140dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:29.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:29.172+0000 7fd516000640 1 -- 172.21.15.152:0/1489536907 >> v2:172.21.15.175:3300/0 conn(0x7fd51813e530 msgr2=0x7fd51806ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:29.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:29.172+0000 7fd516000640 1 --2- 172.21.15.152:0/1489536907 >> v2:172.21.15.175:3300/0 conn(0x7fd51813e530 0x7fd51806ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:29.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:29.172+0000 7fd516000640 1 -- 172.21.15.152:0/1489536907 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5180d00e0 con 0x7fd518141310 2024-04-20T11:30:29.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:29.172+0000 7fd516000640 1 -- 172.21.15.152:0/1489536907 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7fd510001bc0 con 0x7fd518141310 2024-04-20T11:30:29.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:29.172+0000 7fd516000640 1 -- 172.21.15.152:0/1489536907 >> v1:172.21.15.152:6789/0 conn(0x7fd518141310 legacy=0x7fd51815efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:29.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:29.172+0000 7fd516000640 1 --2- 172.21.15.152:0/1489536907 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4f80039b0 0x7fd4f8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:29.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:29.172+0000 7fd516000640 1 --2- 172.21.15.152:0/1489536907 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd4f8004800 0x7fd4f8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:29.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:29.172+0000 7fd516000640 1 --2- 172.21.15.152:0/1489536907 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd51806f340 0x7fd4f80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:29.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:29.172+0000 7fd516000640 1 -- 172.21.15.152:0/1489536907 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fd5180cfc50 con 0x7fd4f8004800 2024-04-20T11:30:29.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:29.172+0000 7fd516000640 1 -- 172.21.15.152:0/1489536907 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f800dbd0 con 0x7fd4f80039b0 2024-04-20T11:30:29.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:29.172+0000 7fd516000640 1 -- 172.21.15.152:0/1489536907 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd4f800dd50 con 0x7fd51806f340 2024-04-20T11:30:29.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:29.172+0000 7fd50e000640 1 --2- 172.21.15.152:0/1489536907 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd4f8004800 0x7fd4f8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:29.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:29.173+0000 7fd516a00640 1 --2- 172.21.15.152:0/1489536907 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4f80039b0 0x7fd4f8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:29.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:29.173+0000 7fd517400640 1 --2- 172.21.15.152:0/1489536907 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd51806f340 0x7fd4f80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:29.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:29.173+0000 7fd50e000640 1 -- 172.21.15.152:0/1489536907 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd51806f340 msgr2=0x7fd4f80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:29.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:29.173+0000 7fd50e000640 1 --2- 172.21.15.152:0/1489536907 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd51806f340 0x7fd4f80051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:29.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:29.173+0000 7fd50e000640 1 -- 172.21.15.152:0/1489536907 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4f80039b0 msgr2=0x7fd4f8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:29.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:29.173+0000 7fd50e000640 1 --2- 172.21.15.152:0/1489536907 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4f80039b0 0x7fd4f8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:29.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:29.173+0000 7fd50e000640 1 -- 172.21.15.152:0/1489536907 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4f8003520 con 0x7fd4f8004800 2024-04-20T11:30:29.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:29.173+0000 7fd516a00640 1 --2- 172.21.15.152:0/1489536907 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4f80039b0 0x7fd4f8003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:30:29.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:29.174+0000 7fd50e000640 1 --2- 172.21.15.152:0/1489536907 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd4f8004800 0x7fd4f8004c80 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fd50000ba30 tx=0x7fd4f8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c85f6510bc66e63 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:29.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:29.174+0000 7fd516000640 1 -- 172.21.15.152:0/1489536907 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd500036020 con 0x7fd4f8004800 2024-04-20T11:30:29.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:29.174+0000 7fd516000640 1 -- 172.21.15.152:0/1489536907 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd500032640 con 0x7fd4f8004800 2024-04-20T11:30:29.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:29.174+0000 7fd516000640 1 -- 172.21.15.152:0/1489536907 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd50000b600 con 0x7fd4f8004800 2024-04-20T11:30:29.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:29.175+0000 7fd517e00640 1 -- 172.21.15.152:0/1489536907 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd4f8004800 msgr2=0x7fd4f8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:29.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:29.175+0000 7fd517e00640 1 --2- 172.21.15.152:0/1489536907 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd4f8004800 0x7fd4f8004c80 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fd50000ba30 tx=0x7fd4f8005830 comp rx=0 tx=0).stop 2024-04-20T11:30:29.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:29.175+0000 7fd517400640 1 -- 172.21.15.152:0/1489536907 reap_dead start 2024-04-20T11:30:29.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:29.175+0000 7fd517e00640 1 -- 172.21.15.152:0/1489536907 shutdown_connections 2024-04-20T11:30:29.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:29.175+0000 7fd517e00640 1 -- 172.21.15.152:0/1489536907 >> 172.21.15.152:0/1489536907 conn(0x7fd518067ae0 msgr2=0x7fd51806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:29.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:29.175+0000 7fd517e00640 1 -- 172.21.15.152:0/1489536907 shutdown_connections 2024-04-20T11:30:29.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:29.175+0000 7fd517e00640 1 -- 172.21.15.152:0/1489536907 wait complete. 2024-04-20T11:30:29.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:29.177+0000 7fd517e00640 1 Processor -- start 2024-04-20T11:30:29.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:29.177+0000 7fd517e00640 1 -- start start 2024-04-20T11:30:29.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:29.178+0000 7fd517e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd4f8004800 0x7fd5180a94a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:29.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:29.178+0000 7fd517e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd51813d150 0x7fd518156440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:29.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:29.178+0000 7fd517e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd518156980 0x7fd518158690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:29.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:29.178+0000 7fd517e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fd5180cf600 con 0x7fd4f8004800 2024-04-20T11:30:29.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:29.178+0000 7fd517e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd518072ce0 con 0x7fd518156980 2024-04-20T11:30:29.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:29.178+0000 7fd517e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd5180cf8e0 con 0x7fd51813d150 2024-04-20T11:30:29.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:29.178+0000 7fd516a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd4f8004800 0x7fd5180a94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:29.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:29.178+0000 7fd516a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd4f8004800 0x7fd5180a94a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:53538/0 (socket says 172.21.15.152:53538) 2024-04-20T11:30:29.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:29.178+0000 7fd516a00640 1 -- 172.21.15.152:0/2198846312 learned_addr learned my addr 172.21.15.152:0/2198846312 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:30:29.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:29.179+0000 7fd50e000640 1 --2- 172.21.15.152:0/2198846312 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd51813d150 0x7fd518156440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:29.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:29.179+0000 7fd517400640 1 --2- 172.21.15.152:0/2198846312 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd518156980 0x7fd518158690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:29.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:29.179+0000 7fd516a00640 1 -- 172.21.15.152:0/2198846312 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd51813d150 msgr2=0x7fd518156440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:29.181 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:29.179+0000 7fd516a00640 1 --2- 172.21.15.152:0/2198846312 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd51813d150 0x7fd518156440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:29.181 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:29.179+0000 7fd516a00640 1 -- 172.21.15.152:0/2198846312 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd518156980 msgr2=0x7fd518158690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:29.181 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:29.179+0000 7fd516a00640 1 --2- 172.21.15.152:0/2198846312 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd518156980 0x7fd518158690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:29.181 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:29.179+0000 7fd516a00640 1 -- 172.21.15.152:0/2198846312 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd518156f00 con 0x7fd4f8004800 2024-04-20T11:30:29.181 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:29.179+0000 7fd517400640 1 --2- 172.21.15.152:0/2198846312 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd518156980 0x7fd518158690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:30:29.181 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:29.179+0000 7fd516a00640 1 --2- 172.21.15.152:0/2198846312 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd4f8004800 0x7fd5180a94a0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fd5080109d0 tx=0x7fd508010e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:29.181 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:29.179+0000 7fd50fe00640 1 -- 172.21.15.152:0/2198846312 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd50800f040 con 0x7fd4f8004800 2024-04-20T11:30:29.181 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:29.179+0000 7fd50fe00640 1 -- 172.21.15.152:0/2198846312 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd50800ebc0 con 0x7fd4f8004800 2024-04-20T11:30:29.181 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:29.179+0000 7fd517e00640 1 -- 172.21.15.152:0/2198846312 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd51815c4f0 con 0x7fd4f8004800 2024-04-20T11:30:29.181 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:29.179+0000 7fd50fe00640 1 -- 172.21.15.152:0/2198846312 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd50801c400 con 0x7fd4f8004800 2024-04-20T11:30:29.181 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:29.180+0000 7fd517e00640 1 -- 172.21.15.152:0/2198846312 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd51813db10 con 0x7fd4f8004800 2024-04-20T11:30:29.181 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:29.181+0000 7fd517e00640 1 -- 172.21.15.152:0/2198846312 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5180cea00 con 0x7fd4f8004800 2024-04-20T11:30:29.182 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:29.182+0000 7fd50fe00640 1 -- 172.21.15.152:0/2198846312 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 8) ==== 91244+0+0 (secure 0 0 0) 0x7fd50801c5a0 con 0x7fd4f8004800 2024-04-20T11:30:29.182 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:29.183+0000 7fd50fe00640 1 --2- 172.21.15.152:0/2198846312 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fd4f406caa0 0x7fd4f406ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:29.182 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:29.183+0000 7fd50fe00640 1 -- 172.21.15.152:0/2198846312 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7fd50808c7c0 con 0x7fd4f8004800 2024-04-20T11:30:29.182 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:29.183+0000 7fd50e000640 1 --2- 172.21.15.152:0/2198846312 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fd4f406caa0 0x7fd4f406ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:29.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:29.184+0000 7fd50e000640 1 --2- 172.21.15.152:0/2198846312 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fd4f406caa0 0x7fd4f406ef60 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fd50000ba00 tx=0x7fd500030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:29.187 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:29.188+0000 7fd50fe00640 1 -- 172.21.15.152:0/2198846312 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fd5180cea00 con 0x7fd4f8004800 2024-04-20T11:30:29.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:29.430+0000 7fd517e00640 1 -- 172.21.15.152:0/2198846312 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd518075e60 con 0x7fd4f8004800 2024-04-20T11:30:29.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:29.431+0000 7fd50fe00640 1 -- 172.21.15.152:0/2198846312 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+5208 (secure 0 0 0) 0x7fd508018030 con 0x7fd4f8004800 2024-04-20T11:30:29.431 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:30:29.431 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4826,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6834/2855025590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2855025590},{"type":"v1","addr":"172.21.15.175:6834","nonce":2855025590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6836/3887716755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":3887716755},{"type":"v1","addr":"172.21.15.152:6836","nonce":3887716755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5712,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/698387399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6835","nonce":698387399},{"type":"v1","addr":"172.21.15.152:6837","nonce":698387399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T11:29:00.276881+0000","modified":"2024-04-20T11:29:03.352174+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5724},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5724":{"gid":5724,"name":"d","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6835/3704251089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6833","nonce":3704251089},{"type":"v1","addr":"172.21.15.175:6835","nonce":3704251089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5724,"qdb_cluster":[5724]},"id":3}]} 2024-04-20T11:30:29.431 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 25 2024-04-20T11:30:29.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:29.434+0000 7fd517e00640 1 -- 172.21.15.152:0/2198846312 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fd4f406caa0 msgr2=0x7fd4f406ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:29.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:29.434+0000 7fd517e00640 1 --2- 172.21.15.152:0/2198846312 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fd4f406caa0 0x7fd4f406ef60 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fd50000ba00 tx=0x7fd500030000 comp rx=0 tx=0).stop 2024-04-20T11:30:29.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:29.434+0000 7fd517e00640 1 -- 172.21.15.152:0/2198846312 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd4f8004800 msgr2=0x7fd5180a94a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:29.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:29.434+0000 7fd517e00640 1 --2- 172.21.15.152:0/2198846312 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd4f8004800 0x7fd5180a94a0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fd5080109d0 tx=0x7fd508010e90 comp rx=0 tx=0).stop 2024-04-20T11:30:29.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:29.434+0000 7fd517e00640 1 -- 172.21.15.152:0/2198846312 shutdown_connections 2024-04-20T11:30:29.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:29.434+0000 7fd517e00640 1 --2- 172.21.15.152:0/2198846312 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fd4f406caa0 0x7fd4f406ef60 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:29.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:29.434+0000 7fd517e00640 1 --2- 172.21.15.152:0/2198846312 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd518156980 0x7fd518158690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:29.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:29.434+0000 7fd517e00640 1 --2- 172.21.15.152:0/2198846312 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd51813d150 0x7fd518156440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:29.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:29.434+0000 7fd517e00640 1 --2- 172.21.15.152:0/2198846312 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd4f8004800 0x7fd5180a94a0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:29.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:29.434+0000 7fd517e00640 1 -- 172.21.15.152:0/2198846312 >> 172.21.15.152:0/2198846312 conn(0x7fd518067ae0 msgr2=0x7fd51806eb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:29.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:29.434+0000 7fd517e00640 1 -- 172.21.15.152:0/2198846312 shutdown_connections 2024-04-20T11:30:29.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:29.434+0000 7fd517e00640 1 -- 172.21.15.152:0/2198846312 wait complete. 2024-04-20T11:30:29.452 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json session ls 2024-04-20T11:30:29.684 INFO:teuthology.orchestra.run.smithi175.stdout:[{"id":6069,"entity":{"name":{"type":"client","num":6069},"addr":{"type":"any","addr":"172.21.15.152:0","nonce":83509866}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":8,"uptime":53.040176756999998,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":19.463887738949055,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.6069 172.21.15.152:0/83509866","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"a8c84f5997950fd7d008204839253291d52e6eda","ceph_version":"ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)","entity_id":"y","hostname":"smithi152","pid":"22712","root":"/"}}] 2024-04-20T11:30:29.684 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "a8c84f5997950fd7d008204839253291d52e6eda", "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi152", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "22712", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.152:0", "nonce": 83509866, "type": "any" }, "name": { "num": 6069, "type": "client" } }, "id": 6069, "inst": "client.6069 172.21.15.152:0/83509866", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 8, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 19.463887738949055 }, "state": "open", "uptime": 53.040176757 } ] 2024-04-20T11:30:29.685 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-04-20T11:30:34.685 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:30:34.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:34.896+0000 7f4a20600640 1 Processor -- start 2024-04-20T11:30:34.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:34.896+0000 7f4a20600640 1 -- start start 2024-04-20T11:30:34.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:34.897+0000 7f4a20600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4a18145f60 0x7f4a1806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:34.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:34.898+0000 7f4a20600640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a18146340 con 0x7f4a180cd0d0 2024-04-20T11:30:34.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:34.898+0000 7f4a20600640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a1806f3c0 con 0x7f4a180726f0 2024-04-20T11:30:34.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:34.898+0000 7f4a20600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4a1806e9f0 con 0x7f4a18145f60 2024-04-20T11:30:34.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:34.898+0000 7f4a1d200640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f4a180cd0d0 0x7f4a18071fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:54732/0 (socket says 172.21.15.152:54732) 2024-04-20T11:30:34.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:34.898+0000 7f4a1d200640 1 -- 172.21.15.152:0/2183520896 learned_addr learned my addr 172.21.15.152:0/2183520896 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:30:34.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:34.898+0000 7f4a17e00640 1 -- 172.21.15.152:0/2183520896 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2346239390 0 0) 0x7f4a18146340 con 0x7f4a180cd0d0 2024-04-20T11:30:34.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:34.898+0000 7f4a1dc00640 1 --2- 172.21.15.152:0/2183520896 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4a18145f60 0x7f4a1806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:34.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:34.898+0000 7f4a17e00640 1 -- 172.21.15.152:0/2183520896 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4a00003520 con 0x7f4a180cd0d0 2024-04-20T11:30:34.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:34.899+0000 7f4a17e00640 1 -- 172.21.15.152:0/2183520896 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 717785523 0 0) 0x7f4a00003520 con 0x7f4a180cd0d0 2024-04-20T11:30:34.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:34.899+0000 7f4a17e00640 1 -- 172.21.15.152:0/2183520896 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4a18145f60 msgr2=0x7f4a1806ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:34.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:34.899+0000 7f4a17e00640 1 --2- 172.21.15.152:0/2183520896 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4a18145f60 0x7f4a1806ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:34.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:34.899+0000 7f4a17e00640 1 -- 172.21.15.152:0/2183520896 >> v1:172.21.15.175:6789/0 conn(0x7f4a180726f0 legacy=0x7f4a180cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:34.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:34.899+0000 7f4a17e00640 1 -- 172.21.15.152:0/2183520896 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a18072d50 con 0x7f4a180cd0d0 2024-04-20T11:30:34.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:34.899+0000 7f4a17e00640 1 -- 172.21.15.152:0/2183520896 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f4a04001670 con 0x7f4a180cd0d0 2024-04-20T11:30:34.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:34.899+0000 7f4a17e00640 1 -- 172.21.15.152:0/2183520896 >> v1:172.21.15.152:6789/0 conn(0x7f4a180cd0d0 legacy=0x7f4a18071fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:34.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:34.899+0000 7f4a17e00640 1 --2- 172.21.15.152:0/2183520896 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4a00003a50 0x7f4a00003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:34.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:34.899+0000 7f4a17e00640 1 --2- 172.21.15.152:0/2183520896 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4a00004800 0x7f4a00004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:34.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:34.899+0000 7f4a17e00640 1 --2- 172.21.15.152:0/2183520896 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4a000051c0 0x7f4a0000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:34.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:34.899+0000 7f4a17e00640 1 -- 172.21.15.152:0/2183520896 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f4a1806e9f0 con 0x7f4a000051c0 2024-04-20T11:30:34.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:34.899+0000 7f4a17e00640 1 -- 172.21.15.152:0/2183520896 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4a0000c0f0 con 0x7f4a00004800 2024-04-20T11:30:34.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:34.899+0000 7f4a17e00640 1 -- 172.21.15.152:0/2183520896 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4a0000c2f0 con 0x7f4a00003a50 2024-04-20T11:30:34.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:34.899+0000 7f4a1e600640 1 --2- 172.21.15.152:0/2183520896 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4a000051c0 0x7f4a0000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:34.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:34.900+0000 7f4a1dc00640 1 --2- 172.21.15.152:0/2183520896 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4a00003a50 0x7f4a00003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:34.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:34.900+0000 7f4a1e600640 1 -- 172.21.15.152:0/2183520896 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4a00003a50 msgr2=0x7f4a00003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:34.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:34.900+0000 7f4a1e600640 1 --2- 172.21.15.152:0/2183520896 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4a00003a50 0x7f4a00003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:34.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:34.900+0000 7f4a1e600640 1 -- 172.21.15.152:0/2183520896 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4a00004800 msgr2=0x7f4a00004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:34.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:34.900+0000 7f4a1e600640 1 --2- 172.21.15.152:0/2183520896 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4a00004800 0x7f4a00004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:34.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:34.900+0000 7f4a1e600640 1 -- 172.21.15.152:0/2183520896 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a00003520 con 0x7f4a000051c0 2024-04-20T11:30:34.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:34.900+0000 7f4a1dc00640 1 --2- 172.21.15.152:0/2183520896 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4a00003a50 0x7f4a00003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:30:34.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:34.901+0000 7f4a1e600640 1 --2- 172.21.15.152:0/2183520896 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4a000051c0 0x7f4a0000b9c0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f4a10001e50 tx=0x7f4a00005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5977224bd335947 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:34.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:34.901+0000 7f4a17e00640 1 -- 172.21.15.152:0/2183520896 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a10001a70 con 0x7f4a000051c0 2024-04-20T11:30:34.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:34.901+0000 7f4a17e00640 1 -- 172.21.15.152:0/2183520896 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a10003b70 con 0x7f4a000051c0 2024-04-20T11:30:34.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:34.901+0000 7f4a17e00640 1 -- 172.21.15.152:0/2183520896 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a100050d0 con 0x7f4a000051c0 2024-04-20T11:30:34.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:34.902+0000 7f4a20600640 1 -- 172.21.15.152:0/2183520896 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4a000051c0 msgr2=0x7f4a0000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:34.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:34.902+0000 7f4a20600640 1 --2- 172.21.15.152:0/2183520896 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4a000051c0 0x7f4a0000b9c0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f4a10001e50 tx=0x7f4a00005830 comp rx=0 tx=0).stop 2024-04-20T11:30:34.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:34.902+0000 7f4a1e600640 1 -- 172.21.15.152:0/2183520896 reap_dead start 2024-04-20T11:30:34.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:34.902+0000 7f4a20600640 1 -- 172.21.15.152:0/2183520896 shutdown_connections 2024-04-20T11:30:34.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:34.902+0000 7f4a20600640 1 -- 172.21.15.152:0/2183520896 >> 172.21.15.152:0/2183520896 conn(0x7f4a18067cf0 msgr2=0x7f4a1806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:34.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:34.902+0000 7f4a20600640 1 -- 172.21.15.152:0/2183520896 shutdown_connections 2024-04-20T11:30:34.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:34.902+0000 7f4a20600640 1 -- 172.21.15.152:0/2183520896 wait complete. 2024-04-20T11:30:34.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:34.904+0000 7f4a20600640 1 Processor -- start 2024-04-20T11:30:34.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:34.904+0000 7f4a20600640 1 -- start start 2024-04-20T11:30:34.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:34.905+0000 7f4a20600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4a000051c0 0x7f4a181413c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:34.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:34.905+0000 7f4a20600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4a181f4460 0x7f4a1815a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:34.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:34.905+0000 7f4a20600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4a181445b0 0x7f4a180b45c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:34.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:34.905+0000 7f4a20600640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f4a1806e680 con 0x7f4a181f4460 2024-04-20T11:30:34.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:34.905+0000 7f4a20600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4a1806e3a0 con 0x7f4a000051c0 2024-04-20T11:30:34.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:34.905+0000 7f4a20600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4a1806ee80 con 0x7f4a181445b0 2024-04-20T11:30:34.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:34.905+0000 7f4a1d200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4a181f4460 0x7f4a1815a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:34.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:34.905+0000 7f4a1d200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4a181f4460 0x7f4a1815a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:53560/0 (socket says 172.21.15.152:53560) 2024-04-20T11:30:34.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:34.905+0000 7f4a1d200640 1 -- 172.21.15.152:0/4026213319 learned_addr learned my addr 172.21.15.152:0/4026213319 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:30:34.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:34.905+0000 7f4a1dc00640 1 --2- 172.21.15.152:0/4026213319 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4a000051c0 0x7f4a181413c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:34.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:34.905+0000 7f4a1e600640 1 --2- 172.21.15.152:0/4026213319 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4a181445b0 0x7f4a180b45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:34.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:34.905+0000 7f4a1d200640 1 -- 172.21.15.152:0/4026213319 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4a181445b0 msgr2=0x7f4a180b45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:34.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:34.905+0000 7f4a1d200640 1 --2- 172.21.15.152:0/4026213319 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4a181445b0 0x7f4a180b45c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:34.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:34.905+0000 7f4a1d200640 1 -- 172.21.15.152:0/4026213319 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4a000051c0 msgr2=0x7f4a181413c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:34.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:34.905+0000 7f4a1d200640 1 --2- 172.21.15.152:0/4026213319 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4a000051c0 0x7f4a181413c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:34.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:34.905+0000 7f4a1d200640 1 -- 172.21.15.152:0/4026213319 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a1815de90 con 0x7f4a181f4460 2024-04-20T11:30:34.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:34.906+0000 7f4a1dc00640 1 --2- 172.21.15.152:0/4026213319 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4a000051c0 0x7f4a181413c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:30:34.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:34.906+0000 7f4a1d200640 1 --2- 172.21.15.152:0/4026213319 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4a181f4460 0x7f4a1815a050 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f4a0400c7d0 tx=0x7f4a0400cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:34.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:34.906+0000 7f4a15600640 1 -- 172.21.15.152:0/4026213319 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a04018070 con 0x7f4a181f4460 2024-04-20T11:30:34.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:34.906+0000 7f4a15600640 1 -- 172.21.15.152:0/4026213319 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a040074e0 con 0x7f4a181f4460 2024-04-20T11:30:34.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:34.906+0000 7f4a20600640 1 -- 172.21.15.152:0/4026213319 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a1815e170 con 0x7f4a181f4460 2024-04-20T11:30:34.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:34.906+0000 7f4a15600640 1 -- 172.21.15.152:0/4026213319 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a04004a20 con 0x7f4a181f4460 2024-04-20T11:30:34.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:34.906+0000 7f4a20600640 1 -- 172.21.15.152:0/4026213319 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a1815e730 con 0x7f4a181f4460 2024-04-20T11:30:34.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:34.907+0000 7f4a20600640 1 -- 172.21.15.152:0/4026213319 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a180689f0 con 0x7f4a181f4460 2024-04-20T11:30:34.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:34.908+0000 7f4a15600640 1 -- 172.21.15.152:0/4026213319 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 8) ==== 91244+0+0 (secure 0 0 0) 0x7f4a0401b070 con 0x7f4a181f4460 2024-04-20T11:30:34.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:34.909+0000 7f4a15600640 1 --2- 172.21.15.152:0/4026213319 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f49f806cb30 0x7f49f806eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:34.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:34.909+0000 7f4a15600640 1 -- 172.21.15.152:0/4026213319 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7f4a0408b540 con 0x7f4a181f4460 2024-04-20T11:30:34.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:34.910+0000 7f4a1dc00640 1 --2- 172.21.15.152:0/4026213319 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f49f806cb30 0x7f49f806eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:34.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:34.910+0000 7f4a1dc00640 1 --2- 172.21.15.152:0/4026213319 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f49f806cb30 0x7f49f806eff0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f4a18140e40 tx=0x7f4a0c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:34.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:34.915+0000 7f4a15600640 1 -- 172.21.15.152:0/4026213319 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f4a040595c0 con 0x7f4a181f4460 2024-04-20T11:30:35.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:35.149+0000 7f4a20600640 1 -- 172.21.15.152:0/4026213319 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4a1806f770 con 0x7f4a181f4460 2024-04-20T11:30:35.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:35.150+0000 7f4a15600640 1 -- 172.21.15.152:0/4026213319 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+5208 (secure 0 0 0) 0x7f4a04058ec0 con 0x7f4a181f4460 2024-04-20T11:30:35.151 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:30:35.151 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4826,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6834/2855025590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2855025590},{"type":"v1","addr":"172.21.15.175:6834","nonce":2855025590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6836/3887716755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":3887716755},{"type":"v1","addr":"172.21.15.152:6836","nonce":3887716755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5712,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/698387399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6835","nonce":698387399},{"type":"v1","addr":"172.21.15.152:6837","nonce":698387399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T11:29:00.276881+0000","modified":"2024-04-20T11:29:03.352174+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5724},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5724":{"gid":5724,"name":"d","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6835/3704251089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6833","nonce":3704251089},{"type":"v1","addr":"172.21.15.175:6835","nonce":3704251089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5724,"qdb_cluster":[5724]},"id":3}]} 2024-04-20T11:30:35.151 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 25 2024-04-20T11:30:35.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:35.152+0000 7f4a20600640 1 -- 172.21.15.152:0/4026213319 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f49f806cb30 msgr2=0x7f49f806eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:35.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:35.152+0000 7f4a20600640 1 --2- 172.21.15.152:0/4026213319 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f49f806cb30 0x7f49f806eff0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f4a18140e40 tx=0x7f4a0c008040 comp rx=0 tx=0).stop 2024-04-20T11:30:35.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:35.152+0000 7f4a20600640 1 -- 172.21.15.152:0/4026213319 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4a181f4460 msgr2=0x7f4a1815a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:35.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:35.152+0000 7f4a20600640 1 --2- 172.21.15.152:0/4026213319 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4a181f4460 0x7f4a1815a050 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f4a0400c7d0 tx=0x7f4a0400cc90 comp rx=0 tx=0).stop 2024-04-20T11:30:35.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:35.152+0000 7f4a20600640 1 -- 172.21.15.152:0/4026213319 shutdown_connections 2024-04-20T11:30:35.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:35.152+0000 7f4a20600640 1 --2- 172.21.15.152:0/4026213319 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f49f806cb30 0x7f49f806eff0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:35.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:35.152+0000 7f4a20600640 1 --2- 172.21.15.152:0/4026213319 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4a181445b0 0x7f4a180b45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:35.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:35.152+0000 7f4a20600640 1 --2- 172.21.15.152:0/4026213319 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4a181f4460 0x7f4a1815a050 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:35.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:35.152+0000 7f4a20600640 1 --2- 172.21.15.152:0/4026213319 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4a000051c0 0x7f4a181413c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:35.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:35.152+0000 7f4a20600640 1 -- 172.21.15.152:0/4026213319 >> 172.21.15.152:0/4026213319 conn(0x7f4a18067cf0 msgr2=0x7f4a1815b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:35.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:35.153+0000 7f4a20600640 1 -- 172.21.15.152:0/4026213319 shutdown_connections 2024-04-20T11:30:35.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:35.153+0000 7f4a20600640 1 -- 172.21.15.152:0/4026213319 wait complete. 2024-04-20T11:30:35.170 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json session ls 2024-04-20T11:30:35.403 INFO:teuthology.orchestra.run.smithi175.stdout:[{"id":6069,"entity":{"name":{"type":"client","num":6069},"addr":{"type":"any","addr":"172.21.15.152:0","nonce":83509866}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":7,"uptime":58.761087932000002,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":19.208304783759786,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.6069 172.21.15.152:0/83509866","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"a8c84f5997950fd7d008204839253291d52e6eda","ceph_version":"ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)","entity_id":"y","hostname":"smithi152","pid":"22712","root":"/"}}] 2024-04-20T11:30:35.404 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "a8c84f5997950fd7d008204839253291d52e6eda", "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi152", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "22712", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.152:0", "nonce": 83509866, "type": "any" }, "name": { "num": 6069, "type": "client" } }, "id": 6069, "inst": "client.6069 172.21.15.152:0/83509866", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 7, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 19.208304783759786 }, "state": "open", "uptime": 58.761087932 } ] 2024-04-20T11:30:35.404 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-04-20T11:30:40.405 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:30:40.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:40.614+0000 7f5eef200640 1 Processor -- start 2024-04-20T11:30:40.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:40.614+0000 7f5eef200640 1 -- start start 2024-04-20T11:30:40.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:40.616+0000 7f5eef200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f5ee806eb40 0x7f5ee806f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:40.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:40.616+0000 7f5eef200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ee806f880 0x7f5ee815cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:40.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:40.616+0000 7f5eef200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ee806fca0 con 0x7f5ee815b5e0 2024-04-20T11:30:40.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:40.616+0000 7f5eef200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f5ee806de10 con 0x7f5ee806eb40 2024-04-20T11:30:40.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:40.616+0000 7f5eef200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5ee80b5770 con 0x7f5ee806f880 2024-04-20T11:30:40.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:40.616+0000 7f5ee7400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f5ee806eb40 0x7f5ee806f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:40.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:40.616+0000 7f5ee7400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f5ee806eb40 0x7f5ee806f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:58756/0 (socket says 172.21.15.152:58756) 2024-04-20T11:30:40.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:40.616+0000 7f5ee7400640 1 -- 172.21.15.152:0/589377950 learned_addr learned my addr 172.21.15.152:0/589377950 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:30:40.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:40.617+0000 7f5eed200640 1 --2- 172.21.15.152:0/589377950 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ee806f880 0x7f5ee815cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:40.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:40.617+0000 7f5ee7400640 1 -- 172.21.15.152:0/589377950 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ee806f880 msgr2=0x7f5ee815cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:40.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:40.617+0000 7f5ee7400640 1 --2- 172.21.15.152:0/589377950 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ee806f880 0x7f5ee815cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:40.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:40.617+0000 7f5ee7400640 1 -- 172.21.15.152:0/589377950 >> v1:172.21.15.175:6789/0 conn(0x7f5ee815b5e0 legacy=0x7f5ee80b4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:40.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:40.617+0000 7f5ee7400640 1 -- 172.21.15.152:0/589377950 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ee806e060 con 0x7f5ee806eb40 2024-04-20T11:30:40.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:40.618+0000 7f5ee7400640 1 --2- 172.21.15.152:0/589377950 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5ee806eb40 0x7f5ee806f340 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f5ed0009870 tx=0x7f5ed002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce67c8c145be2eff server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:40.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:40.618+0000 7f5ee6a00640 1 -- 172.21.15.152:0/589377950 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ed0034070 con 0x7f5ee806eb40 2024-04-20T11:30:40.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:40.618+0000 7f5ee6a00640 1 -- 172.21.15.152:0/589377950 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ed0030840 con 0x7f5ee806eb40 2024-04-20T11:30:40.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:40.618+0000 7f5ee6a00640 1 -- 172.21.15.152:0/589377950 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ed0030c10 con 0x7f5ee806eb40 2024-04-20T11:30:40.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:40.619+0000 7f5eef200640 1 -- 172.21.15.152:0/589377950 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5ee806eb40 msgr2=0x7f5ee806f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:40.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:40.619+0000 7f5eef200640 1 --2- 172.21.15.152:0/589377950 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5ee806eb40 0x7f5ee806f340 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f5ed0009870 tx=0x7f5ed002b980 comp rx=0 tx=0).stop 2024-04-20T11:30:40.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:40.619+0000 7f5eef200640 1 -- 172.21.15.152:0/589377950 shutdown_connections 2024-04-20T11:30:40.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:40.619+0000 7f5eef200640 1 --2- 172.21.15.152:0/589377950 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ee806f880 0x7f5ee815cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:40.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:40.619+0000 7f5eef200640 1 --2- 172.21.15.152:0/589377950 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5ee806eb40 0x7f5ee806f340 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:40.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:40.619+0000 7f5eef200640 1 -- 172.21.15.152:0/589377950 >> 172.21.15.152:0/589377950 conn(0x7f5ee8067cf0 msgr2=0x7f5ee806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:40.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:40.619+0000 7f5eef200640 1 -- 172.21.15.152:0/589377950 shutdown_connections 2024-04-20T11:30:40.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:40.619+0000 7f5eef200640 1 -- 172.21.15.152:0/589377950 wait complete. 2024-04-20T11:30:40.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:40.621+0000 7f5eef200640 1 Processor -- start 2024-04-20T11:30:40.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:40.621+0000 7f5eef200640 1 -- start start 2024-04-20T11:30:40.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:40.622+0000 7f5eef200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ee806eb40 0x7f5ee814f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:40.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:40.622+0000 7f5eef200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5ee806f880 0x7f5ee8155e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:40.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:40.622+0000 7f5eef200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ee815b5e0 0x7f5ee8146c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:40.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:40.622+0000 7f5eef200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f5ee815c9d0 con 0x7f5ee806f880 2024-04-20T11:30:40.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:40.622+0000 7f5eef200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5ee806e620 con 0x7f5ee806eb40 2024-04-20T11:30:40.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:40.622+0000 7f5eef200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5ee80b42d0 con 0x7f5ee815b5e0 2024-04-20T11:30:40.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:40.622+0000 7f5ee7400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5ee806f880 0x7f5ee8155e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:40.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:40.623+0000 7f5ee7400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5ee806f880 0x7f5ee8155e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:58764/0 (socket says 172.21.15.152:58764) 2024-04-20T11:30:40.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:40.623+0000 7f5ee7e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ee806eb40 0x7f5ee814f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:40.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:40.623+0000 7f5ee7400640 1 -- 172.21.15.152:0/2777099099 learned_addr learned my addr 172.21.15.152:0/2777099099 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:30:40.625 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:40.623+0000 7f5eed200640 1 --2- 172.21.15.152:0/2777099099 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ee815b5e0 0x7f5ee8146c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:40.625 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:40.623+0000 7f5ee7400640 1 -- 172.21.15.152:0/2777099099 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ee815b5e0 msgr2=0x7f5ee8146c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:40.625 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:40.623+0000 7f5ee7400640 1 --2- 172.21.15.152:0/2777099099 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ee815b5e0 0x7f5ee8146c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:40.625 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:40.623+0000 7f5ee7400640 1 -- 172.21.15.152:0/2777099099 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ee806eb40 msgr2=0x7f5ee814f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:40.625 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:40.623+0000 7f5ee7400640 1 --2- 172.21.15.152:0/2777099099 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ee806eb40 0x7f5ee814f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:40.625 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:40.623+0000 7f5ee7400640 1 -- 172.21.15.152:0/2777099099 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ee8156380 con 0x7f5ee806f880 2024-04-20T11:30:40.625 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:40.623+0000 7f5ee7400640 1 --2- 172.21.15.152:0/2777099099 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5ee806f880 0x7f5ee8155e40 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f5ed0000c00 tx=0x7f5ed00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:40.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:40.624+0000 7f5edfe00640 1 -- 172.21.15.152:0/2777099099 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ed0034040 con 0x7f5ee806f880 2024-04-20T11:30:40.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:40.624+0000 7f5edfe00640 1 -- 172.21.15.152:0/2777099099 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ed0039d90 con 0x7f5ee806f880 2024-04-20T11:30:40.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:40.624+0000 7f5edfe00640 1 -- 172.21.15.152:0/2777099099 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ed0043610 con 0x7f5ee806f880 2024-04-20T11:30:40.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:40.624+0000 7f5eef200640 1 -- 172.21.15.152:0/2777099099 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ee8156640 con 0x7f5ee806f880 2024-04-20T11:30:40.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:40.624+0000 7f5eef200640 1 -- 172.21.15.152:0/2777099099 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ee8151170 con 0x7f5ee806f880 2024-04-20T11:30:40.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:40.625+0000 7f5eef200640 1 -- 172.21.15.152:0/2777099099 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ee80689f0 con 0x7f5ee806f880 2024-04-20T11:30:40.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:40.626+0000 7f5edfe00640 1 -- 172.21.15.152:0/2777099099 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 9) ==== 91193+0+0 (secure 0 0 0) 0x7f5ed004a060 con 0x7f5ee806f880 2024-04-20T11:30:40.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:40.627+0000 7f5edfe00640 1 --2- 172.21.15.152:0/2777099099 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f5ec406cb20 0x7f5ec406efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:40.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:40.627+0000 7f5edfe00640 1 -- 172.21.15.152:0/2777099099 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7f5ed00adfe0 con 0x7f5ee806f880 2024-04-20T11:30:40.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:40.627+0000 7f5ee7e00640 1 --2- 172.21.15.152:0/2777099099 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f5ec406cb20 0x7f5ec406efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:40.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:40.628+0000 7f5ee7e00640 1 --2- 172.21.15.152:0/2777099099 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f5ec406cb20 0x7f5ec406efe0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f5ed80012e0 tx=0x7f5ed8009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:40.632 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:40.632+0000 7f5edfe00640 1 -- 172.21.15.152:0/2777099099 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f5ed003e070 con 0x7f5ee806f880 2024-04-20T11:30:40.867 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:40.866+0000 7f5eef200640 1 -- 172.21.15.152:0/2777099099 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5ee806ba20 con 0x7f5ee806f880 2024-04-20T11:30:40.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:40.867+0000 7f5edfe00640 1 -- 172.21.15.152:0/2777099099 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+5208 (secure 0 0 0) 0x7f5ed007c020 con 0x7f5ee806f880 2024-04-20T11:30:40.868 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:30:40.868 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4826,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6834/2855025590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2855025590},{"type":"v1","addr":"172.21.15.175:6834","nonce":2855025590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6836/3887716755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":3887716755},{"type":"v1","addr":"172.21.15.152:6836","nonce":3887716755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5712,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/698387399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6835","nonce":698387399},{"type":"v1","addr":"172.21.15.152:6837","nonce":698387399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T11:29:00.276881+0000","modified":"2024-04-20T11:29:03.352174+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5724},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5724":{"gid":5724,"name":"d","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6835/3704251089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6833","nonce":3704251089},{"type":"v1","addr":"172.21.15.175:6835","nonce":3704251089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5724,"qdb_cluster":[5724]},"id":3}]} 2024-04-20T11:30:40.869 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 25 2024-04-20T11:30:40.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:40.870+0000 7f5eef200640 1 -- 172.21.15.152:0/2777099099 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f5ec406cb20 msgr2=0x7f5ec406efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:40.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:40.870+0000 7f5eef200640 1 --2- 172.21.15.152:0/2777099099 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f5ec406cb20 0x7f5ec406efe0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f5ed80012e0 tx=0x7f5ed8009040 comp rx=0 tx=0).stop 2024-04-20T11:30:40.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:40.870+0000 7f5eef200640 1 -- 172.21.15.152:0/2777099099 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5ee806f880 msgr2=0x7f5ee8155e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:40.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:40.870+0000 7f5eef200640 1 --2- 172.21.15.152:0/2777099099 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5ee806f880 0x7f5ee8155e40 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f5ed0000c00 tx=0x7f5ed00046c0 comp rx=0 tx=0).stop 2024-04-20T11:30:40.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:40.870+0000 7f5eef200640 1 -- 172.21.15.152:0/2777099099 shutdown_connections 2024-04-20T11:30:40.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:40.870+0000 7f5eef200640 1 --2- 172.21.15.152:0/2777099099 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f5ec406cb20 0x7f5ec406efe0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:40.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:40.870+0000 7f5eef200640 1 --2- 172.21.15.152:0/2777099099 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ee815b5e0 0x7f5ee8146c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:40.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:40.870+0000 7f5eef200640 1 --2- 172.21.15.152:0/2777099099 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5ee806f880 0x7f5ee8155e40 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:40.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:40.870+0000 7f5eef200640 1 --2- 172.21.15.152:0/2777099099 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ee806eb40 0x7f5ee814f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:40.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:40.870+0000 7f5eef200640 1 -- 172.21.15.152:0/2777099099 >> 172.21.15.152:0/2777099099 conn(0x7f5ee8067cf0 msgr2=0x7f5ee8156c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:40.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:40.870+0000 7f5eef200640 1 -- 172.21.15.152:0/2777099099 shutdown_connections 2024-04-20T11:30:40.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:40.870+0000 7f5eef200640 1 -- 172.21.15.152:0/2777099099 wait complete. 2024-04-20T11:30:40.889 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json session ls 2024-04-20T11:30:41.120 INFO:teuthology.orchestra.run.smithi175.stdout:[] 2024-04-20T11:30:41.120 DEBUG:tasks.cephfs.filesystem:_json_asok output [] 2024-04-20T11:30:41.120 DEBUG:tasks.ceph_test_case:wait_until_true: success in 55s and 0 retries 2024-04-20T11:30:41.120 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-20T11:30:41.121 DEBUG:teuthology.orchestra.run.smithi152:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-20T11:30:41.227 DEBUG:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-20T11:30:41.333 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-20T11:30:41.333 INFO:tasks.cephfs.mount:Terminating background process 2024-04-20T11:30:41.333 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-20T11:30:41.333 DEBUG:teuthology.orchestra.run.smithi152:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:30:41.391 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-20T11:30:41.391 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi152.front.sepia.ceph.com 2024-04-20T11:30:41.391 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:30:41.392 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-20T11:30:41.444 INFO:teuthology.orchestra.run.smithi152.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-20T11:30:41.445 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T11:30:41.446 INFO:tasks.cephfs.mount:Terminating background process 2024-04-20T11:30:41.446 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T11:30:41.446 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:30:41.504 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-20T11:30:41.504 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi175.front.sepia.ceph.com 2024-04-20T11:30:41.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:30:41.505 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-20T11:30:41.554 INFO:teuthology.orchestra.run.smithi175.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-04-20T11:30:41.555 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T11:30:41.555 INFO:tasks.cephfs.mount:Terminating background process 2024-04-20T11:30:41.555 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-20T11:30:41.556 DEBUG:teuthology.orchestra.run.smithi152:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:30:41.611 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-20T11:30:41.611 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi152.front.sepia.ceph.com 2024-04-20T11:30:41.611 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:30:41.612 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-20T11:30:41.661 INFO:teuthology.orchestra.run.smithi152.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-20T11:30:41.662 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T11:30:41.662 INFO:tasks.cephfs.mount:Terminating background process 2024-04-20T11:30:41.662 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T11:30:41.662 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:30:41.720 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-20T11:30:41.720 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi175.front.sepia.ceph.com 2024-04-20T11:30:41.720 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:30:41.720 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-20T11:30:41.775 INFO:teuthology.orchestra.run.smithi175.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-20T11:30:41.776 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T11:30:41.777 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:30:41.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:41.987+0000 7f3419800640 1 Processor -- start 2024-04-20T11:30:41.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:41.987+0000 7f3419800640 1 -- start start 2024-04-20T11:30:41.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:41.989+0000 7f3419800640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f34140723e0 0x7f341415d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:41.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:41.989+0000 7f3419800640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34140ce8c0 con 0x7f3414160b90 2024-04-20T11:30:41.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:41.989+0000 7f3419800640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34141600b0 con 0x7f341415d1e0 2024-04-20T11:30:41.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:41.989+0000 7f3419800640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f3414160390 con 0x7f34140723e0 2024-04-20T11:30:41.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:41.989+0000 7f3413400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f3414160b90 0x7f3414071cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:37562/0 (socket says 172.21.15.152:37562) 2024-04-20T11:30:41.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:41.989+0000 7f3413400640 1 -- 172.21.15.152:0/1375663589 learned_addr learned my addr 172.21.15.152:0/1375663589 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:30:41.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:41.989+0000 7f3418e00640 1 --2- 172.21.15.152:0/1375663589 >> v2:172.21.15.175:3300/0 conn(0x7f34140723e0 0x7f341415d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:41.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:41.989+0000 7f3412a00640 1 -- 172.21.15.152:0/1375663589 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 958931815 0 0) 0x7f34140ce8c0 con 0x7f3414160b90 2024-04-20T11:30:41.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:41.990+0000 7f3412a00640 1 -- 172.21.15.152:0/1375663589 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f33f8003520 con 0x7f3414160b90 2024-04-20T11:30:41.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:41.990+0000 7f3412a00640 1 -- 172.21.15.152:0/1375663589 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3651982927 0 0) 0x7f33f8003520 con 0x7f3414160b90 2024-04-20T11:30:41.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:41.990+0000 7f3412a00640 1 -- 172.21.15.152:0/1375663589 >> v2:172.21.15.175:3300/0 conn(0x7f34140723e0 msgr2=0x7f341415d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:41.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:41.990+0000 7f3412a00640 1 --2- 172.21.15.152:0/1375663589 >> v2:172.21.15.175:3300/0 conn(0x7f34140723e0 0x7f341415d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:41.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:41.990+0000 7f3412a00640 1 -- 172.21.15.152:0/1375663589 >> v1:172.21.15.175:6789/0 conn(0x7f341415d1e0 legacy=0x7f34140cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:41.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:41.990+0000 7f3412a00640 1 -- 172.21.15.152:0/1375663589 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3414160670 con 0x7f3414160b90 2024-04-20T11:30:41.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:41.990+0000 7f3412a00640 1 -- 172.21.15.152:0/1375663589 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f3400001670 con 0x7f3414160b90 2024-04-20T11:30:41.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:41.990+0000 7f3418e00640 1 --2- 172.21.15.152:0/1375663589 >> v2:172.21.15.175:3300/0 conn(0x7f34140723e0 0x7f341415d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:30:41.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:41.990+0000 7f3412a00640 1 -- 172.21.15.152:0/1375663589 >> v1:172.21.15.152:6789/0 conn(0x7f3414160b90 legacy=0x7f3414071cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:41.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:41.990+0000 7f3412a00640 1 --2- 172.21.15.152:0/1375663589 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33f8003a00 0x7f33f8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:41.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:41.990+0000 7f3412a00640 1 --2- 172.21.15.152:0/1375663589 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f33f8004800 0x7f33f8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:41.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:41.990+0000 7f3412a00640 1 --2- 172.21.15.152:0/1375663589 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33f8005210 0x7f33f800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:41.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:41.990+0000 7f3412a00640 1 -- 172.21.15.152:0/1375663589 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f3414160390 con 0x7f33f8004800 2024-04-20T11:30:41.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:41.990+0000 7f3412a00640 1 -- 172.21.15.152:0/1375663589 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f33f800c160 con 0x7f33f8005210 2024-04-20T11:30:41.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:41.990+0000 7f3412a00640 1 -- 172.21.15.152:0/1375663589 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f33f800c360 con 0x7f33f8003a00 2024-04-20T11:30:41.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:41.990+0000 7f3413400640 1 --2- 172.21.15.152:0/1375663589 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f33f8004800 0x7f33f8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:41.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:41.991+0000 7f3413e00640 1 --2- 172.21.15.152:0/1375663589 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33f8003a00 0x7f33f8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:41.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:41.991+0000 7f3418e00640 1 --2- 172.21.15.152:0/1375663589 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33f8005210 0x7f33f800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:41.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:41.991+0000 7f3413400640 1 -- 172.21.15.152:0/1375663589 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33f8003a00 msgr2=0x7f33f8003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:41.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:41.991+0000 7f3413400640 1 --2- 172.21.15.152:0/1375663589 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33f8003a00 0x7f33f8003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:41.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:41.991+0000 7f3413400640 1 -- 172.21.15.152:0/1375663589 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33f8005210 msgr2=0x7f33f800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:41.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:41.991+0000 7f3413400640 1 --2- 172.21.15.152:0/1375663589 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33f8005210 0x7f33f800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:41.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:41.991+0000 7f3413400640 1 -- 172.21.15.152:0/1375663589 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33f8003520 con 0x7f33f8004800 2024-04-20T11:30:41.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:41.991+0000 7f3418e00640 1 --2- 172.21.15.152:0/1375663589 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33f8005210 0x7f33f800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:30:41.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:41.991+0000 7f3418e00640 1 -- 172.21.15.152:0/1375663589 reap_dead start 2024-04-20T11:30:41.995 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:41.992+0000 7f3413400640 1 --2- 172.21.15.152:0/1375663589 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f33f8004800 0x7f33f8004c80 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f3400007fb0 tx=0x7f33f8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21c607ad992ca698 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:41.995 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:41.992+0000 7f3412a00640 1 -- 172.21.15.152:0/1375663589 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3400007c00 con 0x7f33f8004800 2024-04-20T11:30:41.995 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:41.992+0000 7f3412a00640 1 -- 172.21.15.152:0/1375663589 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3400030070 con 0x7f33f8004800 2024-04-20T11:30:41.995 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:41.992+0000 7f3412a00640 1 -- 172.21.15.152:0/1375663589 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3400005d70 con 0x7f33f8004800 2024-04-20T11:30:41.995 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:41.993+0000 7f3419800640 1 -- 172.21.15.152:0/1375663589 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f33f8004800 msgr2=0x7f33f8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:41.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:41.993+0000 7f3419800640 1 --2- 172.21.15.152:0/1375663589 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f33f8004800 0x7f33f8004c80 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f3400007fb0 tx=0x7f33f8005830 comp rx=0 tx=0).stop 2024-04-20T11:30:41.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:41.993+0000 7f3419800640 1 -- 172.21.15.152:0/1375663589 shutdown_connections 2024-04-20T11:30:41.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:41.993+0000 7f3419800640 1 --2- 172.21.15.152:0/1375663589 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f33f8004800 0x7f33f8004c80 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:41.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:41.993+0000 7f3419800640 1 -- 172.21.15.152:0/1375663589 >> 172.21.15.152:0/1375663589 conn(0x7f3414067ae0 msgr2=0x7f341406b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:41.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:41.993+0000 7f3419800640 1 -- 172.21.15.152:0/1375663589 shutdown_connections 2024-04-20T11:30:41.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:41.993+0000 7f3419800640 1 -- 172.21.15.152:0/1375663589 wait complete. 2024-04-20T11:30:41.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:41.995+0000 7f3419800640 1 Processor -- start 2024-04-20T11:30:41.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:41.995+0000 7f3419800640 1 -- start start 2024-04-20T11:30:41.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:41.996+0000 7f3419800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33f8004800 0x7f3414152230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:41.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:41.996+0000 7f3419800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3414157d80 0x7f3414158c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:41.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:41.996+0000 7f3419800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f34141591c0 0x7f34140b4300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:41.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:41.996+0000 7f3419800640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f341415d740 con 0x7f3414157d80 2024-04-20T11:30:41.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:41.996+0000 7f3419800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f341415d5c0 con 0x7f33f8004800 2024-04-20T11:30:41.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:41.996+0000 7f3419800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f341415d8c0 con 0x7f34141591c0 2024-04-20T11:30:41.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:41.996+0000 7f3413400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3414157d80 0x7f3414158c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:41.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:41.996+0000 7f3413400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3414157d80 0x7f3414158c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:58780/0 (socket says 172.21.15.152:58780) 2024-04-20T11:30:41.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:41.996+0000 7f3413400640 1 -- 172.21.15.152:0/1553567754 learned_addr learned my addr 172.21.15.152:0/1553567754 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:30:41.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:41.996+0000 7f3413e00640 1 --2- 172.21.15.152:0/1553567754 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33f8004800 0x7f3414152230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:41.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:41.997+0000 7f3418e00640 1 --2- 172.21.15.152:0/1553567754 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f34141591c0 0x7f34140b4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:41.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:41.997+0000 7f3413400640 1 -- 172.21.15.152:0/1553567754 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f34141591c0 msgr2=0x7f34140b4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:41.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:41.997+0000 7f3413400640 1 --2- 172.21.15.152:0/1553567754 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f34141591c0 0x7f34140b4300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:41.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:41.997+0000 7f3413400640 1 -- 172.21.15.152:0/1553567754 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33f8004800 msgr2=0x7f3414152230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:41.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:41.997+0000 7f3413400640 1 --2- 172.21.15.152:0/1553567754 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33f8004800 0x7f3414152230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:41.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:41.997+0000 7f3413400640 1 -- 172.21.15.152:0/1553567754 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3414152770 con 0x7f3414157d80 2024-04-20T11:30:41.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:41.997+0000 7f3413400640 1 --2- 172.21.15.152:0/1553567754 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3414157d80 0x7f3414158c80 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f340000c1a0 tx=0x7f34000018b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:41.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:41.997+0000 7f340be00640 1 -- 172.21.15.152:0/1553567754 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f340002cbd0 con 0x7f3414157d80 2024-04-20T11:30:41.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:41.997+0000 7f340be00640 1 -- 172.21.15.152:0/1553567754 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f340002cd70 con 0x7f3414157d80 2024-04-20T11:30:41.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:41.997+0000 7f340be00640 1 -- 172.21.15.152:0/1553567754 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3400007270 con 0x7f3414157d80 2024-04-20T11:30:41.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:41.997+0000 7f3419800640 1 -- 172.21.15.152:0/1553567754 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3414159700 con 0x7f3414157d80 2024-04-20T11:30:41.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:41.998+0000 7f3419800640 1 -- 172.21.15.152:0/1553567754 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3414157580 con 0x7f3414157d80 2024-04-20T11:30:42.000 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:41.999+0000 7f3419800640 1 -- 172.21.15.152:0/1553567754 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f341415ad50 con 0x7f3414157d80 2024-04-20T11:30:42.000 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.000+0000 7f340be00640 1 -- 172.21.15.152:0/1553567754 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 9) ==== 91193+0+0 (secure 0 0 0) 0x7f3400007410 con 0x7f3414157d80 2024-04-20T11:30:42.001 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.000+0000 7f340be00640 1 --2- 172.21.15.152:0/1553567754 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f33f406cad0 0x7f33f406ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:42.001 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.001+0000 7f340be00640 1 -- 172.21.15.152:0/1553567754 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7f34000ad5f0 con 0x7f3414157d80 2024-04-20T11:30:42.001 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.001+0000 7f3413e00640 1 --2- 172.21.15.152:0/1553567754 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f33f406cad0 0x7f33f406ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:42.002 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.001+0000 7f3413e00640 1 --2- 172.21.15.152:0/1553567754 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f33f406cad0 0x7f33f406ef90 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f34040054d0 tx=0x7f3404009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:42.005 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.005+0000 7f340be00640 1 -- 172.21.15.152:0/1553567754 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f341415ad50 con 0x7f3414157d80 2024-04-20T11:30:42.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.235+0000 7f3419800640 1 -- 172.21.15.152:0/1553567754 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3414075e60 con 0x7f3414157d80 2024-04-20T11:30:42.237 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.236+0000 7f340be00640 1 -- 172.21.15.152:0/1553567754 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+5208 (secure 0 0 0) 0x7f340007b750 con 0x7f3414157d80 2024-04-20T11:30:42.237 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:30:42.237 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4826,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6834/2855025590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2855025590},{"type":"v1","addr":"172.21.15.175:6834","nonce":2855025590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6836/3887716755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":3887716755},{"type":"v1","addr":"172.21.15.152:6836","nonce":3887716755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5712,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/698387399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6835","nonce":698387399},{"type":"v1","addr":"172.21.15.152:6837","nonce":698387399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T11:29:00.276881+0000","modified":"2024-04-20T11:29:03.352174+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5724},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5724":{"gid":5724,"name":"d","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6835/3704251089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6833","nonce":3704251089},{"type":"v1","addr":"172.21.15.175:6835","nonce":3704251089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5724,"qdb_cluster":[5724]},"id":3}]} 2024-04-20T11:30:42.238 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 25 2024-04-20T11:30:42.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.239+0000 7f3419800640 1 -- 172.21.15.152:0/1553567754 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f33f406cad0 msgr2=0x7f33f406ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:42.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.239+0000 7f3419800640 1 --2- 172.21.15.152:0/1553567754 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f33f406cad0 0x7f33f406ef90 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f34040054d0 tx=0x7f3404009040 comp rx=0 tx=0).stop 2024-04-20T11:30:42.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.239+0000 7f3419800640 1 -- 172.21.15.152:0/1553567754 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3414157d80 msgr2=0x7f3414158c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:42.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.239+0000 7f3419800640 1 --2- 172.21.15.152:0/1553567754 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3414157d80 0x7f3414158c80 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f340000c1a0 tx=0x7f34000018b0 comp rx=0 tx=0).stop 2024-04-20T11:30:42.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.239+0000 7f3419800640 1 -- 172.21.15.152:0/1553567754 shutdown_connections 2024-04-20T11:30:42.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.239+0000 7f3419800640 1 --2- 172.21.15.152:0/1553567754 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f33f406cad0 0x7f33f406ef90 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:42.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.239+0000 7f3419800640 1 --2- 172.21.15.152:0/1553567754 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f34141591c0 0x7f34140b4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:42.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.239+0000 7f3419800640 1 --2- 172.21.15.152:0/1553567754 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3414157d80 0x7f3414158c80 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:42.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.239+0000 7f3419800640 1 --2- 172.21.15.152:0/1553567754 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33f8004800 0x7f3414152230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:42.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.239+0000 7f3419800640 1 -- 172.21.15.152:0/1553567754 >> 172.21.15.152:0/1553567754 conn(0x7f3414067ae0 msgr2=0x7f341406b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:42.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.239+0000 7f3419800640 1 -- 172.21.15.152:0/1553567754 shutdown_connections 2024-04-20T11:30:42.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.239+0000 7f3419800640 1 -- 172.21.15.152:0/1553567754 wait complete. 2024-04-20T11:30:42.259 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:30:42.452 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.451+0000 7f9cf0a00640 1 Processor -- start 2024-04-20T11:30:42.452 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.451+0000 7f9cf0a00640 1 -- start start 2024-04-20T11:30:42.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.453+0000 7f9cf0a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9cec15b5e0 0x7f9cec06eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:42.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.453+0000 7f9cf0a00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f9cec0b4c50 0x7f9cec0b50d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:42.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.453+0000 7f9cf0a00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9cec0b5770 con 0x7f9cec06f340 2024-04-20T11:30:42.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.453+0000 7f9cf0a00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f9cec06de10 con 0x7f9cec0b4c50 2024-04-20T11:30:42.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.453+0000 7f9cf0a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f9cec06e060 con 0x7f9cec15b5e0 2024-04-20T11:30:42.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.453+0000 7f9ce9400640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f9cec0b4c50 0x7f9cec0b50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:42.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.454+0000 7f9ce9e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9cec15b5e0 0x7f9cec06eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:42.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.454+0000 7f9ce9400640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f9cec0b4c50 0x7f9cec0b50d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.152:56534/0 (socket says 172.21.15.152:56534) 2024-04-20T11:30:42.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.454+0000 7f9ce9400640 1 -- 172.21.15.152:0/3509606905 learned_addr learned my addr 172.21.15.152:0/3509606905 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:30:42.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.454+0000 7f9ce9400640 1 -- 172.21.15.152:0/3509606905 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9cec15b5e0 msgr2=0x7f9cec06eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:42.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.454+0000 7f9ce9400640 1 --2- 172.21.15.152:0/3509606905 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9cec15b5e0 0x7f9cec06eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:42.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.454+0000 7f9ce9400640 1 -- 172.21.15.152:0/3509606905 >> v1:172.21.15.175:6789/0 conn(0x7f9cec06f340 legacy=0x7f9cec15cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:42.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.454+0000 7f9ce9400640 1 -- 172.21.15.152:0/3509606905 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cec06e340 con 0x7f9cec0b4c50 2024-04-20T11:30:42.456 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.456+0000 7f9ce9400640 1 --2- 172.21.15.152:0/3509606905 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9cec0b4c50 0x7f9cec0b50d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9cd800bb10 tx=0x7f9cd802e150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e2a6f6def4bed94 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:42.457 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.456+0000 7f9ce8a00640 1 -- 172.21.15.152:0/3509606905 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cd803b070 con 0x7f9cec0b4c50 2024-04-20T11:30:42.457 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.456+0000 7f9ce8a00640 1 -- 172.21.15.152:0/3509606905 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cd8036440 con 0x7f9cec0b4c50 2024-04-20T11:30:42.457 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.456+0000 7f9ce8a00640 1 -- 172.21.15.152:0/3509606905 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cd8036810 con 0x7f9cec0b4c50 2024-04-20T11:30:42.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.457+0000 7f9cf0a00640 1 -- 172.21.15.152:0/3509606905 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9cec0b4c50 msgr2=0x7f9cec0b50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:42.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.457+0000 7f9cf0a00640 1 --2- 172.21.15.152:0/3509606905 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9cec0b4c50 0x7f9cec0b50d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9cd800bb10 tx=0x7f9cd802e150 comp rx=0 tx=0).stop 2024-04-20T11:30:42.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.457+0000 7f9cf0a00640 1 -- 172.21.15.152:0/3509606905 shutdown_connections 2024-04-20T11:30:42.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.457+0000 7f9cf0a00640 1 --2- 172.21.15.152:0/3509606905 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9cec0b4c50 0x7f9cec0b50d0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:42.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.457+0000 7f9cf0a00640 1 --2- 172.21.15.152:0/3509606905 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9cec15b5e0 0x7f9cec06eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:42.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.457+0000 7f9cf0a00640 1 -- 172.21.15.152:0/3509606905 >> 172.21.15.152:0/3509606905 conn(0x7f9cec067cf0 msgr2=0x7f9cec06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:42.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.457+0000 7f9cf0a00640 1 -- 172.21.15.152:0/3509606905 shutdown_connections 2024-04-20T11:30:42.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.457+0000 7f9cf0a00640 1 -- 172.21.15.152:0/3509606905 wait complete. 2024-04-20T11:30:42.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.459+0000 7f9cf0a00640 1 Processor -- start 2024-04-20T11:30:42.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.459+0000 7f9cf0a00640 1 -- start start 2024-04-20T11:30:42.461 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.460+0000 7f9cf0a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9cec06f340 0x7f9cec0ced30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:42.462 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.460+0000 7f9cf0a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9cec0b4c50 0x7f9cec14dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:42.462 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.460+0000 7f9cf0a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9cec15b5e0 0x7f9cec0ce520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:42.462 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.460+0000 7f9cf0a00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f9cec0b42d0 con 0x7f9cec0b4c50 2024-04-20T11:30:42.462 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.460+0000 7f9cf0a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9cec15c9d0 con 0x7f9cec15b5e0 2024-04-20T11:30:42.462 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.460+0000 7f9cf0a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f9cec06f790 con 0x7f9cec06f340 2024-04-20T11:30:42.462 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.460+0000 7f9ce9400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9cec0b4c50 0x7f9cec14dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:42.462 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.461+0000 7f9ce9400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9cec0b4c50 0x7f9cec14dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:58788/0 (socket says 172.21.15.152:58788) 2024-04-20T11:30:42.462 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.461+0000 7f9ce9400640 1 -- 172.21.15.152:0/2449748887 learned_addr learned my addr 172.21.15.152:0/2449748887 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:30:42.462 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.461+0000 7f9ce9e00640 1 --2- 172.21.15.152:0/2449748887 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9cec06f340 0x7f9cec0ced30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:42.462 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.461+0000 7f9cea800640 1 --2- 172.21.15.152:0/2449748887 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9cec15b5e0 0x7f9cec0ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:42.462 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.461+0000 7f9ce9400640 1 -- 172.21.15.152:0/2449748887 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9cec06f340 msgr2=0x7f9cec0ced30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:42.463 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.461+0000 7f9ce9400640 1 --2- 172.21.15.152:0/2449748887 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9cec06f340 0x7f9cec0ced30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:42.463 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.461+0000 7f9ce9400640 1 -- 172.21.15.152:0/2449748887 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9cec15b5e0 msgr2=0x7f9cec0ce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:42.465 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.461+0000 7f9ce9400640 1 --2- 172.21.15.152:0/2449748887 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9cec15b5e0 0x7f9cec0ce520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:42.465 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.461+0000 7f9ce9400640 1 -- 172.21.15.152:0/2449748887 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cec0cf270 con 0x7f9cec0b4c50 2024-04-20T11:30:42.465 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.461+0000 7f9ce9400640 1 --2- 172.21.15.152:0/2449748887 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9cec0b4c50 0x7f9cec14dfc0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f9cd802e190 tx=0x7f9cd80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:42.466 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.463+0000 7f9ce2000640 1 -- 172.21.15.152:0/2449748887 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cd803b030 con 0x7f9cec0b4c50 2024-04-20T11:30:42.466 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.463+0000 7f9ce2000640 1 -- 172.21.15.152:0/2449748887 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cd803d050 con 0x7f9cec0b4c50 2024-04-20T11:30:42.466 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.463+0000 7f9ce2000640 1 -- 172.21.15.152:0/2449748887 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cd8034870 con 0x7f9cec0b4c50 2024-04-20T11:30:42.466 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.463+0000 7f9cf0a00640 1 -- 172.21.15.152:0/2449748887 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cec14e500 con 0x7f9cec0b4c50 2024-04-20T11:30:42.466 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.463+0000 7f9cf0a00640 1 -- 172.21.15.152:0/2449748887 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cec1f7dc0 con 0x7f9cec0b4c50 2024-04-20T11:30:42.466 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.464+0000 7f9cf0a00640 1 -- 172.21.15.152:0/2449748887 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cec0689f0 con 0x7f9cec0b4c50 2024-04-20T11:30:42.466 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.465+0000 7f9ce2000640 1 -- 172.21.15.152:0/2449748887 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 9) ==== 91193+0+0 (secure 0 0 0) 0x7f9cd8048060 con 0x7f9cec0b4c50 2024-04-20T11:30:42.467 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.466+0000 7f9ce2000640 1 --2- 172.21.15.152:0/2449748887 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f9ccc06caa0 0x7f9ccc06ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:42.467 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.466+0000 7f9ce2000640 1 -- 172.21.15.152:0/2449748887 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7f9cd80ad7b0 con 0x7f9cec0b4c50 2024-04-20T11:30:42.467 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.466+0000 7f9ce9e00640 1 --2- 172.21.15.152:0/2449748887 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f9ccc06caa0 0x7f9ccc06ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:42.467 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.467+0000 7f9ce9e00640 1 --2- 172.21.15.152:0/2449748887 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f9ccc06caa0 0x7f9ccc06ef60 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc009a30 tx=0x7f9cdc008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:42.471 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.471+0000 7f9ce2000640 1 -- 172.21.15.152:0/2449748887 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f9cd807b9b0 con 0x7f9cec0b4c50 2024-04-20T11:30:42.671 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.670+0000 7f9cf0a00640 1 -- 172.21.15.152:0/2449748887 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9cec072980 con 0x7f9cec0b4c50 2024-04-20T11:30:42.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.672+0000 7f9ce2000640 1 -- 172.21.15.152:0/2449748887 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) ==== 74+0+16824 (secure 0 0 0) 0x7f9cd807b2b0 con 0x7f9cec0b4c50 2024-04-20T11:30:42.674 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:30:42.674 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":48,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:29:42.078051+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:28:58.007718+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-20T11:28:59.260647+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.6","mappings":[{"from":0,"to":4}]},{"pgid":"7.2c","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:30:42.676 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.675+0000 7f9cf0a00640 1 -- 172.21.15.152:0/2449748887 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f9ccc06caa0 msgr2=0x7f9ccc06ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:42.676 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.675+0000 7f9cf0a00640 1 --2- 172.21.15.152:0/2449748887 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f9ccc06caa0 0x7f9ccc06ef60 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc009a30 tx=0x7f9cdc008040 comp rx=0 tx=0).stop 2024-04-20T11:30:42.676 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.675+0000 7f9cf0a00640 1 -- 172.21.15.152:0/2449748887 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9cec0b4c50 msgr2=0x7f9cec14dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:42.676 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.675+0000 7f9cf0a00640 1 --2- 172.21.15.152:0/2449748887 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9cec0b4c50 0x7f9cec14dfc0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f9cd802e190 tx=0x7f9cd80046c0 comp rx=0 tx=0).stop 2024-04-20T11:30:42.676 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.675+0000 7f9cf0a00640 1 -- 172.21.15.152:0/2449748887 shutdown_connections 2024-04-20T11:30:42.676 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.675+0000 7f9cf0a00640 1 --2- 172.21.15.152:0/2449748887 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f9ccc06caa0 0x7f9ccc06ef60 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:42.676 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.675+0000 7f9cf0a00640 1 --2- 172.21.15.152:0/2449748887 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9cec15b5e0 0x7f9cec0ce520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:42.676 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.675+0000 7f9cf0a00640 1 --2- 172.21.15.152:0/2449748887 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9cec0b4c50 0x7f9cec14dfc0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:42.676 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.675+0000 7f9cf0a00640 1 --2- 172.21.15.152:0/2449748887 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9cec06f340 0x7f9cec0ced30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:42.677 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.675+0000 7f9cf0a00640 1 -- 172.21.15.152:0/2449748887 >> 172.21.15.152:0/2449748887 conn(0x7f9cec067cf0 msgr2=0x7f9cec15a840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:42.677 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.675+0000 7f9cf0a00640 1 -- 172.21.15.152:0/2449748887 shutdown_connections 2024-04-20T11:30:42.677 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.676+0000 7f9cf0a00640 1 -- 172.21.15.152:0/2449748887 wait complete. 2024-04-20T11:30:42.694 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T11:26:47.246571+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T11:28:58.007718+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T11:28:59.260647+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T11:30:42.695 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T11:30:42.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.901+0000 7f602b000640 1 Processor -- start 2024-04-20T11:30:42.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.901+0000 7f602b000640 1 -- start start 2024-04-20T11:30:42.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.903+0000 7f602b000640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f60240ceeb0 0x7f60240d4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:42.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.903+0000 7f602b000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6024160230 0x7f60241606e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:42.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.903+0000 7f602b000640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60240d2b70 con 0x7f60240cf5b0 2024-04-20T11:30:42.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.903+0000 7f602b000640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f60240d5420 con 0x7f60240ceeb0 2024-04-20T11:30:42.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.903+0000 7f602b000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f60240d2e50 con 0x7f6024160230 2024-04-20T11:30:42.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.904+0000 7f6029200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f60240ceeb0 0x7f60240d4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:42.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.904+0000 7f602a600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6024160230 0x7f60241606e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:42.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.904+0000 7f6029200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f60240ceeb0 0x7f60240d4ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.152:56558/0 (socket says 172.21.15.152:56558) 2024-04-20T11:30:42.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.904+0000 7f6029200640 1 -- 172.21.15.152:0/3191790624 learned_addr learned my addr 172.21.15.152:0/3191790624 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:30:42.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.905+0000 7f6029200640 1 -- 172.21.15.152:0/3191790624 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6024160230 msgr2=0x7f60241606e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:42.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.905+0000 7f6029200640 1 --2- 172.21.15.152:0/3191790624 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6024160230 0x7f60241606e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:42.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.905+0000 7f6029200640 1 -- 172.21.15.152:0/3191790624 >> v1:172.21.15.175:6789/0 conn(0x7f60240cf5b0 legacy=0x7f60240cf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:42.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.905+0000 7f6029200640 1 -- 172.21.15.152:0/3191790624 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60240d3130 con 0x7f60240ceeb0 2024-04-20T11:30:42.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.906+0000 7f6029200640 1 --2- 172.21.15.152:0/3191790624 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f60240ceeb0 0x7f60240d4ee0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f6014009850 tx=0x7f601402b9f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2039ef2b7622c7c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:42.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.907+0000 7f6023e00640 1 -- 172.21.15.152:0/3191790624 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6014004230 con 0x7f60240ceeb0 2024-04-20T11:30:42.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.907+0000 7f6023e00640 1 -- 172.21.15.152:0/3191790624 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60140043d0 con 0x7f60240ceeb0 2024-04-20T11:30:42.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.907+0000 7f6023e00640 1 -- 172.21.15.152:0/3191790624 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6014003810 con 0x7f60240ceeb0 2024-04-20T11:30:42.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.907+0000 7f602b000640 1 -- 172.21.15.152:0/3191790624 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f60240ceeb0 msgr2=0x7f60240d4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:42.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.907+0000 7f602b000640 1 --2- 172.21.15.152:0/3191790624 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f60240ceeb0 0x7f60240d4ee0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f6014009850 tx=0x7f601402b9f0 comp rx=0 tx=0).stop 2024-04-20T11:30:42.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.907+0000 7f602b000640 1 -- 172.21.15.152:0/3191790624 shutdown_connections 2024-04-20T11:30:42.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.907+0000 7f602b000640 1 --2- 172.21.15.152:0/3191790624 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6024160230 0x7f60241606e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:42.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.907+0000 7f602b000640 1 --2- 172.21.15.152:0/3191790624 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f60240ceeb0 0x7f60240d4ee0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:42.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.907+0000 7f602b000640 1 -- 172.21.15.152:0/3191790624 >> 172.21.15.152:0/3191790624 conn(0x7f602408bfd0 msgr2=0x7f60240d4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:42.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.907+0000 7f602b000640 1 -- 172.21.15.152:0/3191790624 shutdown_connections 2024-04-20T11:30:42.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.908+0000 7f602b000640 1 -- 172.21.15.152:0/3191790624 wait complete. 2024-04-20T11:30:42.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.909+0000 7f602b000640 1 Processor -- start 2024-04-20T11:30:42.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.909+0000 7f602b000640 1 -- start start 2024-04-20T11:30:42.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.910+0000 7f602b000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f60240ceeb0 0x7f602414d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:42.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.910+0000 7f602b000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f60240cf5b0 0x7f60240b8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:42.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.910+0000 7f602b000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6024160230 0x7f60240b9500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:42.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.910+0000 7f602b000640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f60240d3c10 con 0x7f6024160230 2024-04-20T11:30:42.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.910+0000 7f602b000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f60240d36f0 con 0x7f60240ceeb0 2024-04-20T11:30:42.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.910+0000 7f602b000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f60240d2790 con 0x7f60240cf5b0 2024-04-20T11:30:42.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.911+0000 7f602a600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6024160230 0x7f60240b9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:42.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.911+0000 7f602a600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6024160230 0x7f60240b9500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:58796/0 (socket says 172.21.15.152:58796) 2024-04-20T11:30:42.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.911+0000 7f602a600640 1 -- 172.21.15.152:0/1057692233 learned_addr learned my addr 172.21.15.152:0/1057692233 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:30:42.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.911+0000 7f6029c00640 1 --2- 172.21.15.152:0/1057692233 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f60240ceeb0 0x7f602414d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:42.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.911+0000 7f6029200640 1 --2- 172.21.15.152:0/1057692233 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f60240cf5b0 0x7f60240b8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:42.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.911+0000 7f602a600640 1 -- 172.21.15.152:0/1057692233 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f60240cf5b0 msgr2=0x7f60240b8fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:42.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.911+0000 7f602a600640 1 --2- 172.21.15.152:0/1057692233 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f60240cf5b0 0x7f60240b8fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:42.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.911+0000 7f602a600640 1 -- 172.21.15.152:0/1057692233 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f60240ceeb0 msgr2=0x7f602414d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:42.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.911+0000 7f602a600640 1 --2- 172.21.15.152:0/1057692233 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f60240ceeb0 0x7f602414d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:42.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.911+0000 7f602a600640 1 -- 172.21.15.152:0/1057692233 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60241e91c0 con 0x7f6024160230 2024-04-20T11:30:42.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.911+0000 7f602a600640 1 --2- 172.21.15.152:0/1057692233 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6024160230 0x7f60240b9500 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f601c009dd0 tx=0x7f601c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:42.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.912+0000 7f6021600640 1 -- 172.21.15.152:0/1057692233 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f601c018020 con 0x7f6024160230 2024-04-20T11:30:42.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.912+0000 7f6021600640 1 -- 172.21.15.152:0/1057692233 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f601c014480 con 0x7f6024160230 2024-04-20T11:30:42.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.912+0000 7f602b000640 1 -- 172.21.15.152:0/1057692233 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60241e94a0 con 0x7f6024160230 2024-04-20T11:30:42.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.912+0000 7f6021600640 1 -- 172.21.15.152:0/1057692233 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f601c014870 con 0x7f6024160230 2024-04-20T11:30:42.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.912+0000 7f602b000640 1 -- 172.21.15.152:0/1057692233 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60241e9a60 con 0x7f6024160230 2024-04-20T11:30:42.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.913+0000 7f602b000640 1 -- 172.21.15.152:0/1057692233 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6024075e60 con 0x7f6024160230 2024-04-20T11:30:42.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.914+0000 7f6021600640 1 -- 172.21.15.152:0/1057692233 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 9) ==== 91193+0+0 (secure 0 0 0) 0x7f601c01b020 con 0x7f6024160230 2024-04-20T11:30:42.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.915+0000 7f6021600640 1 --2- 172.21.15.152:0/1057692233 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f600806caa0 0x7f600806ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:42.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.915+0000 7f6021600640 1 -- 172.21.15.152:0/1057692233 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7f601c08ccc0 con 0x7f6024160230 2024-04-20T11:30:42.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.915+0000 7f6029c00640 1 --2- 172.21.15.152:0/1057692233 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f600806caa0 0x7f600806ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:42.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.916+0000 7f6029c00640 1 --2- 172.21.15.152:0/1057692233 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f600806caa0 0x7f600806ef60 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f6018000f60 tx=0x7f6018009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:42.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:42.919+0000 7f6021600640 1 -- 172.21.15.152:0/1057692233 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f601c093080 con 0x7f6024160230 2024-04-20T11:30:43.118 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.117+0000 7f602b000640 1 -- 172.21.15.152:0/1057692233 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f60240cf390 con 0x7f6024160230 2024-04-20T11:30:43.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.119+0000 7f6021600640 1 -- 172.21.15.152:0/1057692233 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v48) ==== 93+0+10 (secure 0 0 0) 0x7f601c022030 con 0x7f6024160230 2024-04-20T11:30:43.119 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 1 2024-04-20T11:30:43.122 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.121+0000 7f602b000640 1 -- 172.21.15.152:0/1057692233 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f600806caa0 msgr2=0x7f600806ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:43.122 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.121+0000 7f602b000640 1 --2- 172.21.15.152:0/1057692233 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f600806caa0 0x7f600806ef60 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f6018000f60 tx=0x7f6018009040 comp rx=0 tx=0).stop 2024-04-20T11:30:43.122 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.121+0000 7f602b000640 1 -- 172.21.15.152:0/1057692233 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6024160230 msgr2=0x7f60240b9500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:43.122 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.121+0000 7f602b000640 1 --2- 172.21.15.152:0/1057692233 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6024160230 0x7f60240b9500 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f601c009dd0 tx=0x7f601c00c490 comp rx=0 tx=0).stop 2024-04-20T11:30:43.122 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.121+0000 7f602b000640 1 -- 172.21.15.152:0/1057692233 shutdown_connections 2024-04-20T11:30:43.122 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.121+0000 7f602b000640 1 --2- 172.21.15.152:0/1057692233 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f600806caa0 0x7f600806ef60 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:43.122 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.121+0000 7f602b000640 1 --2- 172.21.15.152:0/1057692233 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6024160230 0x7f60240b9500 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:43.122 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.121+0000 7f602b000640 1 --2- 172.21.15.152:0/1057692233 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f60240cf5b0 0x7f60240b8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:43.122 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.121+0000 7f602b000640 1 --2- 172.21.15.152:0/1057692233 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f60240ceeb0 0x7f602414d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:43.123 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.121+0000 7f602b000640 1 -- 172.21.15.152:0/1057692233 >> 172.21.15.152:0/1057692233 conn(0x7f602408bfd0 msgr2=0x7f6024161b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:43.123 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.121+0000 7f602b000640 1 -- 172.21.15.152:0/1057692233 shutdown_connections 2024-04-20T11:30:43.123 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.121+0000 7f602b000640 1 -- 172.21.15.152:0/1057692233 wait complete. 2024-04-20T11:30:43.141 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T11:30:43.332 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.332+0000 7f462ce00640 1 Processor -- start 2024-04-20T11:30:43.333 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.332+0000 7f462ce00640 1 -- start start 2024-04-20T11:30:43.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.333+0000 7f462ce00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f462815e9c0 0x7f4628162530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:43.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.333+0000 7f462ce00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f46280d1980 0x7f46280d3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:43.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.333+0000 7f462ce00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46280b7920 con 0x7f46280ba560 2024-04-20T11:30:43.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.333+0000 7f462ce00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f4628163460 con 0x7f462815e9c0 2024-04-20T11:30:43.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.333+0000 7f462ce00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4628162ea0 con 0x7f46280d1980 2024-04-20T11:30:43.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.334+0000 7f4625400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f46280ba560 0x7f46280d1270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:37574/0 (socket says 172.21.15.152:37574) 2024-04-20T11:30:43.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.334+0000 7f4625400640 1 -- 172.21.15.152:0/3343691823 learned_addr learned my addr 172.21.15.152:0/3343691823 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:30:43.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.334+0000 7f4625e00640 1 --2- 172.21.15.152:0/3343691823 >> v2:172.21.15.175:3300/0 conn(0x7f462815e9c0 0x7f4628162530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:43.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.334+0000 7f4626800640 1 --2- 172.21.15.152:0/3343691823 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f46280d1980 0x7f46280d3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:43.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.334+0000 7f4624a00640 1 -- 172.21.15.152:0/3343691823 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 956495786 0 0) 0x7f4618001490 con 0x7f46280ba560 2024-04-20T11:30:43.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.334+0000 7f4624a00640 1 -- 172.21.15.152:0/3343691823 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f460c003520 con 0x7f46280ba560 2024-04-20T11:30:43.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.335+0000 7f4624a00640 1 -- 172.21.15.152:0/3343691823 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3987713481 0 0) 0x7f460c003520 con 0x7f46280ba560 2024-04-20T11:30:43.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.335+0000 7f4624a00640 1 -- 172.21.15.152:0/3343691823 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f46280d1980 msgr2=0x7f46280d3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:43.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.335+0000 7f4624a00640 1 --2- 172.21.15.152:0/3343691823 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f46280d1980 0x7f46280d3490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:43.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.335+0000 7f4624a00640 1 -- 172.21.15.152:0/3343691823 >> v2:172.21.15.175:3300/0 conn(0x7f462815e9c0 msgr2=0x7f4628162530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:43.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.335+0000 7f4624a00640 1 --2- 172.21.15.152:0/3343691823 >> v2:172.21.15.175:3300/0 conn(0x7f462815e9c0 0x7f4628162530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:43.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.335+0000 7f4624a00640 1 -- 172.21.15.152:0/3343691823 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4628163180 con 0x7f46280ba560 2024-04-20T11:30:43.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.335+0000 7f4624a00640 1 -- 172.21.15.152:0/3343691823 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f4618001be0 con 0x7f46280ba560 2024-04-20T11:30:43.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.335+0000 7f4624a00640 1 -- 172.21.15.152:0/3343691823 >> v1:172.21.15.152:6789/0 conn(0x7f46280ba560 legacy=0x7f46280d1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:43.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.335+0000 7f4624a00640 1 --2- 172.21.15.152:0/3343691823 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f460c0039b0 0x7f460c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:43.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.335+0000 7f4624a00640 1 --2- 172.21.15.152:0/3343691823 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f460c004800 0x7f460c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:43.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.335+0000 7f4624a00640 1 --2- 172.21.15.152:0/3343691823 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f46280d1980 0x7f460c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:43.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.335+0000 7f4624a00640 1 -- 172.21.15.152:0/3343691823 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f4628163460 con 0x7f460c004800 2024-04-20T11:30:43.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.335+0000 7f4624a00640 1 -- 172.21.15.152:0/3343691823 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4628162ea0 con 0x7f460c0039b0 2024-04-20T11:30:43.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.335+0000 7f4624a00640 1 -- 172.21.15.152:0/3343691823 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f460c00ba40 con 0x7f46280d1980 2024-04-20T11:30:43.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.335+0000 7f4625400640 1 --2- 172.21.15.152:0/3343691823 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f460c004800 0x7f460c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:43.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.336+0000 7f4626800640 1 --2- 172.21.15.152:0/3343691823 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f46280d1980 0x7f460c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:43.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.336+0000 7f4625e00640 1 --2- 172.21.15.152:0/3343691823 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f460c0039b0 0x7f460c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:43.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.336+0000 7f4625400640 1 -- 172.21.15.152:0/3343691823 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f46280d1980 msgr2=0x7f460c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:43.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.336+0000 7f4625400640 1 --2- 172.21.15.152:0/3343691823 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f46280d1980 0x7f460c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:43.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.336+0000 7f4625400640 1 -- 172.21.15.152:0/3343691823 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f460c0039b0 msgr2=0x7f460c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:43.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.336+0000 7f4625400640 1 --2- 172.21.15.152:0/3343691823 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f460c0039b0 0x7f460c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:43.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.336+0000 7f4625400640 1 -- 172.21.15.152:0/3343691823 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f460c003520 con 0x7f460c004800 2024-04-20T11:30:43.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.336+0000 7f4625e00640 1 --2- 172.21.15.152:0/3343691823 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f460c0039b0 0x7f460c003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:30:43.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.336+0000 7f4626800640 1 --2- 172.21.15.152:0/3343691823 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f46280d1980 0x7f460c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:30:43.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.337+0000 7f4625400640 1 --2- 172.21.15.152:0/3343691823 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f460c004800 0x7f460c004c80 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f461800a980 tx=0x7f460c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce88ba3aaad528c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:43.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.337+0000 7f4624a00640 1 -- 172.21.15.152:0/3343691823 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f461800a5d0 con 0x7f460c004800 2024-04-20T11:30:43.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.337+0000 7f4624a00640 1 -- 172.21.15.152:0/3343691823 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4618034650 con 0x7f460c004800 2024-04-20T11:30:43.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.337+0000 7f4624a00640 1 -- 172.21.15.152:0/3343691823 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4618034a40 con 0x7f460c004800 2024-04-20T11:30:43.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.338+0000 7f462ce00640 1 -- 172.21.15.152:0/3343691823 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f460c004800 msgr2=0x7f460c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:43.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.338+0000 7f462ce00640 1 --2- 172.21.15.152:0/3343691823 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f460c004800 0x7f460c004c80 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f461800a980 tx=0x7f460c005830 comp rx=0 tx=0).stop 2024-04-20T11:30:43.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.338+0000 7f4626800640 1 -- 172.21.15.152:0/3343691823 reap_dead start 2024-04-20T11:30:43.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.338+0000 7f462ce00640 1 -- 172.21.15.152:0/3343691823 shutdown_connections 2024-04-20T11:30:43.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.338+0000 7f462ce00640 1 -- 172.21.15.152:0/3343691823 >> 172.21.15.152:0/3343691823 conn(0x7f462806cc40 msgr2=0x7f462806d050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:43.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.338+0000 7f462ce00640 1 -- 172.21.15.152:0/3343691823 shutdown_connections 2024-04-20T11:30:43.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.338+0000 7f462ce00640 1 -- 172.21.15.152:0/3343691823 wait complete. 2024-04-20T11:30:43.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.340+0000 7f462ce00640 1 Processor -- start 2024-04-20T11:30:43.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.340+0000 7f462ce00640 1 -- start start 2024-04-20T11:30:43.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.341+0000 7f462ce00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f460c004800 0x7f4628146bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:43.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.341+0000 7f462ce00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f46280b5d90 0x7f462815a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:43.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.341+0000 7f462ce00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f462815d520 0x7f462815d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:43.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.341+0000 7f462ce00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f462815eda0 con 0x7f46280b5d90 2024-04-20T11:30:43.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.341+0000 7f462ce00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f46280d2f70 con 0x7f460c004800 2024-04-20T11:30:43.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.341+0000 7f462ce00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4628162a70 con 0x7f462815d520 2024-04-20T11:30:43.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.341+0000 7f4625400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f46280b5d90 0x7f462815a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:43.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.341+0000 7f4625400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f46280b5d90 0x7f462815a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:58814/0 (socket says 172.21.15.152:58814) 2024-04-20T11:30:43.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.341+0000 7f4625400640 1 -- 172.21.15.152:0/4090383370 learned_addr learned my addr 172.21.15.152:0/4090383370 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:30:43.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.341+0000 7f4625e00640 1 --2- 172.21.15.152:0/4090383370 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f460c004800 0x7f4628146bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:43.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.341+0000 7f4626800640 1 --2- 172.21.15.152:0/4090383370 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f462815d520 0x7f462815d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:43.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.341+0000 7f4625400640 1 -- 172.21.15.152:0/4090383370 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f462815d520 msgr2=0x7f462815d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:43.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.341+0000 7f4625400640 1 --2- 172.21.15.152:0/4090383370 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f462815d520 0x7f462815d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:43.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.341+0000 7f4625400640 1 -- 172.21.15.152:0/4090383370 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f460c004800 msgr2=0x7f4628146bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:43.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.342+0000 7f4625400640 1 --2- 172.21.15.152:0/4090383370 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f460c004800 0x7f4628146bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:43.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.342+0000 7f4625400640 1 -- 172.21.15.152:0/4090383370 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f462815c4c0 con 0x7f46280b5d90 2024-04-20T11:30:43.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.342+0000 7f4626800640 1 --2- 172.21.15.152:0/4090383370 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f462815d520 0x7f462815d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:30:43.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.342+0000 7f4625400640 1 --2- 172.21.15.152:0/4090383370 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f46280b5d90 0x7f462815a950 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f461802cd70 tx=0x7f4628163350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:43.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.342+0000 7f461e000640 1 -- 172.21.15.152:0/4090383370 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4618039040 con 0x7f46280b5d90 2024-04-20T11:30:43.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.342+0000 7f461e000640 1 -- 172.21.15.152:0/4090383370 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4618043070 con 0x7f46280b5d90 2024-04-20T11:30:43.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.342+0000 7f462ce00640 1 -- 172.21.15.152:0/4090383370 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f462815ae90 con 0x7f46280b5d90 2024-04-20T11:30:43.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.342+0000 7f461e000640 1 -- 172.21.15.152:0/4090383370 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46180019a0 con 0x7f46280b5d90 2024-04-20T11:30:43.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.342+0000 7f462ce00640 1 -- 172.21.15.152:0/4090383370 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f462815ca40 con 0x7f46280b5d90 2024-04-20T11:30:43.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.344+0000 7f462ce00640 1 -- 172.21.15.152:0/4090383370 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46280689f0 con 0x7f46280b5d90 2024-04-20T11:30:43.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.344+0000 7f461e000640 1 -- 172.21.15.152:0/4090383370 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 9) ==== 91193+0+0 (secure 0 0 0) 0x7f4618001b40 con 0x7f46280b5d90 2024-04-20T11:30:43.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.345+0000 7f461e000640 1 --2- 172.21.15.152:0/4090383370 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f460406caa0 0x7f460406ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:43.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.345+0000 7f461e000640 1 -- 172.21.15.152:0/4090383370 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7f46180adf80 con 0x7f46280b5d90 2024-04-20T11:30:43.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.346+0000 7f4625e00640 1 --2- 172.21.15.152:0/4090383370 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f460406caa0 0x7f460406ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:43.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.347+0000 7f4625e00640 1 --2- 172.21.15.152:0/4090383370 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f460406caa0 0x7f460406ef60 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f4610009b20 tx=0x7f4610008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:43.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.350+0000 7f461e000640 1 -- 172.21.15.152:0/4090383370 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f46280b7920 con 0x7f46280b5d90 2024-04-20T11:30:43.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.553+0000 7f462ce00640 1 -- 172.21.15.152:0/4090383370 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f46280d2430 con 0x7f46280b5d90 2024-04-20T11:30:43.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.554+0000 7f461e000640 1 -- 172.21.15.152:0/4090383370 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v48) ==== 104+0+11 (secure 0 0 0) 0x7f461807bff0 con 0x7f46280b5d90 2024-04-20T11:30:43.555 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 32 2024-04-20T11:30:43.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.556+0000 7f462ce00640 1 -- 172.21.15.152:0/4090383370 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f460406caa0 msgr2=0x7f460406ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:43.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.556+0000 7f462ce00640 1 --2- 172.21.15.152:0/4090383370 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f460406caa0 0x7f460406ef60 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f4610009b20 tx=0x7f4610008040 comp rx=0 tx=0).stop 2024-04-20T11:30:43.558 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.556+0000 7f462ce00640 1 -- 172.21.15.152:0/4090383370 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f46280b5d90 msgr2=0x7f462815a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:43.558 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.556+0000 7f462ce00640 1 --2- 172.21.15.152:0/4090383370 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f46280b5d90 0x7f462815a950 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f461802cd70 tx=0x7f4628163350 comp rx=0 tx=0).stop 2024-04-20T11:30:43.558 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.557+0000 7f462ce00640 1 -- 172.21.15.152:0/4090383370 shutdown_connections 2024-04-20T11:30:43.558 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.557+0000 7f462ce00640 1 --2- 172.21.15.152:0/4090383370 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f460406caa0 0x7f460406ef60 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:43.558 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.557+0000 7f462ce00640 1 --2- 172.21.15.152:0/4090383370 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f462815d520 0x7f462815d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:43.558 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.557+0000 7f462ce00640 1 --2- 172.21.15.152:0/4090383370 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f46280b5d90 0x7f462815a950 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:43.558 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.557+0000 7f462ce00640 1 --2- 172.21.15.152:0/4090383370 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f460c004800 0x7f4628146bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:43.558 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.557+0000 7f462ce00640 1 -- 172.21.15.152:0/4090383370 >> 172.21.15.152:0/4090383370 conn(0x7f462806cc40 msgr2=0x7f462806d050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:43.558 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.557+0000 7f462ce00640 1 -- 172.21.15.152:0/4090383370 shutdown_connections 2024-04-20T11:30:43.558 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.557+0000 7f462ce00640 1 -- 172.21.15.152:0/4090383370 wait complete. 2024-04-20T11:30:43.576 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T11:30:43.773 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.773+0000 7f9bb5a00640 1 Processor -- start 2024-04-20T11:30:43.773 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.773+0000 7f9bb5a00640 1 -- start start 2024-04-20T11:30:43.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.774+0000 7f9bb5a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f9bb0156c00 0x7f9bb0160a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:43.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.774+0000 7f9bb5a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9bb015ca40 0x7f9bb015cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:43.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.774+0000 7f9bb5a00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9bb0160fd0 con 0x7f9bb006bda0 2024-04-20T11:30:43.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.774+0000 7f9bb5a00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f9bb015d5c0 con 0x7f9bb0156c00 2024-04-20T11:30:43.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.774+0000 7f9bb5a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f9bb015d8a0 con 0x7f9bb015ca40 2024-04-20T11:30:43.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.774+0000 7f9bafe00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f9bb0156c00 0x7f9bb0160a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:43.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.775+0000 7f9bafe00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f9bb0156c00 0x7f9bb0160a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:58824/0 (socket says 172.21.15.152:58824) 2024-04-20T11:30:43.776 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.775+0000 7f9bafe00640 1 -- 172.21.15.152:0/2715532610 learned_addr learned my addr 172.21.15.152:0/2715532610 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:30:43.776 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.775+0000 7f9ba7400640 1 --2- 172.21.15.152:0/2715532610 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9bb015ca40 0x7f9bb015cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:43.776 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.775+0000 7f9bafe00640 1 -- 172.21.15.152:0/2715532610 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9bb015ca40 msgr2=0x7f9bb015cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:43.776 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.775+0000 7f9bafe00640 1 --2- 172.21.15.152:0/2715532610 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9bb015ca40 0x7f9bb015cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:43.776 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.775+0000 7f9bafe00640 1 -- 172.21.15.152:0/2715532610 >> v1:172.21.15.175:6789/0 conn(0x7f9bb006bda0 legacy=0x7f9bb00b30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:43.776 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.775+0000 7f9bafe00640 1 -- 172.21.15.152:0/2715532610 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bb0160290 con 0x7f9bb0156c00 2024-04-20T11:30:43.778 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.776+0000 7f9bafe00640 1 --2- 172.21.15.152:0/2715532610 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9bb0156c00 0x7f9bb0160a90 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f9ba0009870 tx=0x7f9ba002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccb03c20d9b9a6fd server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:43.778 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.777+0000 7f9baf400640 1 -- 172.21.15.152:0/2715532610 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ba0035020 con 0x7f9bb0156c00 2024-04-20T11:30:43.778 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.777+0000 7f9baf400640 1 -- 172.21.15.152:0/2715532610 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ba0031640 con 0x7f9bb0156c00 2024-04-20T11:30:43.778 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.777+0000 7f9baf400640 1 -- 172.21.15.152:0/2715532610 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ba002d040 con 0x7f9bb0156c00 2024-04-20T11:30:43.778 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.777+0000 7f9bb5a00640 1 -- 172.21.15.152:0/2715532610 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9bb0156c00 msgr2=0x7f9bb0160a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:43.778 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.777+0000 7f9bb5a00640 1 --2- 172.21.15.152:0/2715532610 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9bb0156c00 0x7f9bb0160a90 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f9ba0009870 tx=0x7f9ba002b8f0 comp rx=0 tx=0).stop 2024-04-20T11:30:43.778 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.777+0000 7f9bb5a00640 1 -- 172.21.15.152:0/2715532610 shutdown_connections 2024-04-20T11:30:43.778 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.777+0000 7f9bb5a00640 1 --2- 172.21.15.152:0/2715532610 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9bb015ca40 0x7f9bb015cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:43.778 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.777+0000 7f9bb5a00640 1 --2- 172.21.15.152:0/2715532610 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9bb0156c00 0x7f9bb0160a90 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:43.778 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.777+0000 7f9bb5a00640 1 -- 172.21.15.152:0/2715532610 >> 172.21.15.152:0/2715532610 conn(0x7f9bb00b21e0 msgr2=0x7f9bb00b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:43.778 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.777+0000 7f9bb5a00640 1 -- 172.21.15.152:0/2715532610 shutdown_connections 2024-04-20T11:30:43.779 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.777+0000 7f9bb5a00640 1 -- 172.21.15.152:0/2715532610 wait complete. 2024-04-20T11:30:43.779 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.779+0000 7f9bb5a00640 1 Processor -- start 2024-04-20T11:30:43.779 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.779+0000 7f9bb5a00640 1 -- start start 2024-04-20T11:30:43.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.780+0000 7f9bb5a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9bb006bda0 0x7f9bb01f89f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:43.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.780+0000 7f9bb5a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9bb0156c00 0x7f9bb00cf620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:43.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.780+0000 7f9bb5a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9bb015ca40 0x7f9bb00b0210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:43.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.780+0000 7f9bb5a00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f9bb00b3950 con 0x7f9bb006bda0 2024-04-20T11:30:43.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.780+0000 7f9bb5a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9bb00b37d0 con 0x7f9bb0156c00 2024-04-20T11:30:43.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.780+0000 7f9bb5a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f9bb00b3ad0 con 0x7f9bb015ca40 2024-04-20T11:30:43.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.780+0000 7f9bafe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9bb006bda0 0x7f9bb01f89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:43.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.780+0000 7f9bafe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9bb006bda0 0x7f9bb01f89f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:58838/0 (socket says 172.21.15.152:58838) 2024-04-20T11:30:43.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.780+0000 7f9bafe00640 1 -- 172.21.15.152:0/3738444694 learned_addr learned my addr 172.21.15.152:0/3738444694 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:30:43.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.781+0000 7f9ba7400640 1 --2- 172.21.15.152:0/3738444694 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9bb0156c00 0x7f9bb00cf620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:43.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.781+0000 7f9bafe00640 1 -- 172.21.15.152:0/3738444694 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9bb015ca40 msgr2=0x7f9bb00b0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:43.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.781+0000 7f9bafe00640 1 --2- 172.21.15.152:0/3738444694 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9bb015ca40 0x7f9bb00b0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:43.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.781+0000 7f9bafe00640 1 -- 172.21.15.152:0/3738444694 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9bb0156c00 msgr2=0x7f9bb00cf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:43.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.781+0000 7f9bafe00640 1 --2- 172.21.15.152:0/3738444694 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9bb0156c00 0x7f9bb00cf620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:43.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.781+0000 7f9bafe00640 1 -- 172.21.15.152:0/3738444694 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bb01f9940 con 0x7f9bb006bda0 2024-04-20T11:30:43.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.781+0000 7f9bafe00640 1 --2- 172.21.15.152:0/3738444694 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9bb006bda0 0x7f9bb01f89f0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f9ba0000c00 tx=0x7f9ba00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:43.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.781+0000 7f9bacc00640 1 -- 172.21.15.152:0/3738444694 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ba0040070 con 0x7f9bb006bda0 2024-04-20T11:30:43.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.781+0000 7f9bacc00640 1 -- 172.21.15.152:0/3738444694 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ba0039bf0 con 0x7f9bb006bda0 2024-04-20T11:30:43.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.781+0000 7f9bb5a00640 1 -- 172.21.15.152:0/3738444694 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bb00cfb60 con 0x7f9bb006bda0 2024-04-20T11:30:43.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.781+0000 7f9bacc00640 1 -- 172.21.15.152:0/3738444694 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ba0035040 con 0x7f9bb006bda0 2024-04-20T11:30:43.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.782+0000 7f9bb5a00640 1 -- 172.21.15.152:0/3738444694 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bb01f9380 con 0x7f9bb006bda0 2024-04-20T11:30:43.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.783+0000 7f9bb5a00640 1 -- 172.21.15.152:0/3738444694 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bb0075e60 con 0x7f9bb006bda0 2024-04-20T11:30:43.784 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.784+0000 7f9bacc00640 1 -- 172.21.15.152:0/3738444694 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 9) ==== 91193+0+0 (secure 0 0 0) 0x7f9ba004b060 con 0x7f9bb006bda0 2024-04-20T11:30:43.785 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.784+0000 7f9bacc00640 1 --2- 172.21.15.152:0/3738444694 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f9b8c06caa0 0x7f9b8c06ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:43.785 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.785+0000 7f9bacc00640 1 -- 172.21.15.152:0/3738444694 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7f9ba00ada00 con 0x7f9bb006bda0 2024-04-20T11:30:43.785 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.785+0000 7f9ba7400640 1 --2- 172.21.15.152:0/3738444694 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f9b8c06caa0 0x7f9b8c06ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:43.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.785+0000 7f9ba7400640 1 --2- 172.21.15.152:0/3738444694 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f9b8c06caa0 0x7f9b8c06ef60 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f9b9800b7c0 tx=0x7f9b98008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:43.789 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.789+0000 7f9bacc00640 1 -- 172.21.15.152:0/3738444694 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f9ba0031610 con 0x7f9bb006bda0 2024-04-20T11:30:43.979 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.979+0000 7f9bb5a00640 1 -- 172.21.15.152:0/3738444694 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f9bb00b3470 con 0x7f9bb006bda0 2024-04-20T11:30:43.980 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.980+0000 7f9bacc00640 1 -- 172.21.15.152:0/3738444694 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v48) ==== 100+0+11 (secure 0 0 0) 0x7f9ba007bb60 con 0x7f9bb006bda0 2024-04-20T11:30:43.980 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 64 2024-04-20T11:30:43.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.982+0000 7f9bb5a00640 1 -- 172.21.15.152:0/3738444694 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f9b8c06caa0 msgr2=0x7f9b8c06ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:43.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.982+0000 7f9bb5a00640 1 --2- 172.21.15.152:0/3738444694 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f9b8c06caa0 0x7f9b8c06ef60 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f9b9800b7c0 tx=0x7f9b98008040 comp rx=0 tx=0).stop 2024-04-20T11:30:43.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.982+0000 7f9bb5a00640 1 -- 172.21.15.152:0/3738444694 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9bb006bda0 msgr2=0x7f9bb01f89f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:43.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.982+0000 7f9bb5a00640 1 --2- 172.21.15.152:0/3738444694 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9bb006bda0 0x7f9bb01f89f0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f9ba0000c00 tx=0x7f9ba00046c0 comp rx=0 tx=0).stop 2024-04-20T11:30:43.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.982+0000 7f9bb5a00640 1 -- 172.21.15.152:0/3738444694 shutdown_connections 2024-04-20T11:30:43.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.982+0000 7f9bb5a00640 1 --2- 172.21.15.152:0/3738444694 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f9b8c06caa0 0x7f9b8c06ef60 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:43.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.982+0000 7f9bb5a00640 1 --2- 172.21.15.152:0/3738444694 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9bb015ca40 0x7f9bb00b0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:43.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.982+0000 7f9bb5a00640 1 --2- 172.21.15.152:0/3738444694 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9bb0156c00 0x7f9bb00cf620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:43.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.982+0000 7f9bb5a00640 1 --2- 172.21.15.152:0/3738444694 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9bb006bda0 0x7f9bb01f89f0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:43.984 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.982+0000 7f9bb5a00640 1 -- 172.21.15.152:0/3738444694 >> 172.21.15.152:0/3738444694 conn(0x7f9bb00b21e0 msgr2=0x7f9bb015b150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:43.984 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.982+0000 7f9bb5a00640 1 -- 172.21.15.152:0/3738444694 shutdown_connections 2024-04-20T11:30:43.984 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:43.982+0000 7f9bb5a00640 1 -- 172.21.15.152:0/3738444694 wait complete. 2024-04-20T11:30:44.000 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:30:44.184 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.183+0000 7fe739000640 1 Processor -- start 2024-04-20T11:30:44.184 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.183+0000 7fe739000640 1 -- start start 2024-04-20T11:30:44.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.184+0000 7fe739000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fe73415fcc0 0x7fe7340d4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:44.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.184+0000 7fe739000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7340b65c0 0x7fe7340cf240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:44.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.184+0000 7fe739000640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7340d3350 con 0x7fe7341603d0 2024-04-20T11:30:44.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.185+0000 7fe739000640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fe7340d3630 con 0x7fe73415fcc0 2024-04-20T11:30:44.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.185+0000 7fe739000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe7340d3910 con 0x7fe7340b65c0 2024-04-20T11:30:44.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.185+0000 7fe733400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fe73415fcc0 0x7fe7340d4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:44.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.185+0000 7fe733400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fe73415fcc0 0x7fe7340d4e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:58840/0 (socket says 172.21.15.152:58840) 2024-04-20T11:30:44.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.185+0000 7fe733400640 1 -- 172.21.15.152:0/3537521341 learned_addr learned my addr 172.21.15.152:0/3537521341 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:30:44.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.185+0000 7fe733e00640 1 --2- 172.21.15.152:0/3537521341 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7340b65c0 0x7fe7340cf240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:44.187 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.185+0000 7fe733400640 1 -- 172.21.15.152:0/3537521341 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7340b65c0 msgr2=0x7fe7340cf240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:44.187 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.185+0000 7fe733400640 1 --2- 172.21.15.152:0/3537521341 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7340b65c0 0x7fe7340cf240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:44.187 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.185+0000 7fe733400640 1 -- 172.21.15.152:0/3537521341 >> v1:172.21.15.175:6789/0 conn(0x7fe7341603d0 legacy=0x7fe7340b5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:44.187 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.185+0000 7fe733400640 1 -- 172.21.15.152:0/3537521341 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7340ced20 con 0x7fe73415fcc0 2024-04-20T11:30:44.187 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.186+0000 7fe733400640 1 --2- 172.21.15.152:0/3537521341 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe73415fcc0 0x7fe7340d4e70 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fe73408cb80 tx=0x7fe72402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b0878d5864a052 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:44.188 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.187+0000 7fe732000640 1 -- 172.21.15.152:0/3537521341 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe724035020 con 0x7fe73415fcc0 2024-04-20T11:30:44.188 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.187+0000 7fe732000640 1 -- 172.21.15.152:0/3537521341 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe724031640 con 0x7fe73415fcc0 2024-04-20T11:30:44.188 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.187+0000 7fe732000640 1 -- 172.21.15.152:0/3537521341 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe72402d040 con 0x7fe73415fcc0 2024-04-20T11:30:44.188 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.187+0000 7fe739000640 1 -- 172.21.15.152:0/3537521341 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe73415fcc0 msgr2=0x7fe7340d4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:44.188 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.187+0000 7fe739000640 1 --2- 172.21.15.152:0/3537521341 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe73415fcc0 0x7fe7340d4e70 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fe73408cb80 tx=0x7fe72402b8f0 comp rx=0 tx=0).stop 2024-04-20T11:30:44.188 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.187+0000 7fe739000640 1 -- 172.21.15.152:0/3537521341 shutdown_connections 2024-04-20T11:30:44.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.187+0000 7fe739000640 1 --2- 172.21.15.152:0/3537521341 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7340b65c0 0x7fe7340cf240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:44.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.187+0000 7fe739000640 1 --2- 172.21.15.152:0/3537521341 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe73415fcc0 0x7fe7340d4e70 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:44.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.187+0000 7fe739000640 1 -- 172.21.15.152:0/3537521341 >> 172.21.15.152:0/3537521341 conn(0x7fe73408bfd0 msgr2=0x7fe7340d4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:44.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.188+0000 7fe739000640 1 -- 172.21.15.152:0/3537521341 shutdown_connections 2024-04-20T11:30:44.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.188+0000 7fe739000640 1 -- 172.21.15.152:0/3537521341 wait complete. 2024-04-20T11:30:44.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.189+0000 7fe739000640 1 Processor -- start 2024-04-20T11:30:44.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.189+0000 7fe739000640 1 -- start start 2024-04-20T11:30:44.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.190+0000 7fe739000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7340b65c0 0x7fe7341f7220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:44.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.190+0000 7fe739000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe73415fcc0 0x7fe7340cb620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:44.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.190+0000 7fe739000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe7341603d0 0x7fe734160de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:44.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.190+0000 7fe739000640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fe7340b5600 con 0x7fe7341603d0 2024-04-20T11:30:44.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.190+0000 7fe739000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe7340b5480 con 0x7fe73415fcc0 2024-04-20T11:30:44.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.190+0000 7fe739000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe7340cf780 con 0x7fe7340b65c0 2024-04-20T11:30:44.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.190+0000 7fe733e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe7341603d0 0x7fe734160de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:44.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.191+0000 7fe733e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe7341603d0 0x7fe734160de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:58844/0 (socket says 172.21.15.152:58844) 2024-04-20T11:30:44.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.191+0000 7fe733e00640 1 -- 172.21.15.152:0/3010324479 learned_addr learned my addr 172.21.15.152:0/3010324479 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:30:44.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.191+0000 7fe732a00640 1 --2- 172.21.15.152:0/3010324479 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe73415fcc0 0x7fe7340cb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:44.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.191+0000 7fe733400640 1 --2- 172.21.15.152:0/3010324479 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7340b65c0 0x7fe7341f7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:44.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.191+0000 7fe733e00640 1 -- 172.21.15.152:0/3010324479 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7340b65c0 msgr2=0x7fe7341f7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:44.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.191+0000 7fe733e00640 1 --2- 172.21.15.152:0/3010324479 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7340b65c0 0x7fe7341f7220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:44.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.191+0000 7fe733e00640 1 -- 172.21.15.152:0/3010324479 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe73415fcc0 msgr2=0x7fe7340cb620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:44.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.191+0000 7fe733e00640 1 --2- 172.21.15.152:0/3010324479 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe73415fcc0 0x7fe7340cb620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:44.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.191+0000 7fe733e00640 1 -- 172.21.15.152:0/3010324479 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7341f4490 con 0x7fe7341603d0 2024-04-20T11:30:44.193 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.191+0000 7fe733e00640 1 --2- 172.21.15.152:0/3010324479 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe7341603d0 0x7fe734160de0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fe72c007de0 tx=0x7fe72c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:44.193 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.191+0000 7fe72b400640 1 -- 172.21.15.152:0/3010324479 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe72c014620 con 0x7fe7341603d0 2024-04-20T11:30:44.193 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.191+0000 7fe72b400640 1 -- 172.21.15.152:0/3010324479 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe72c019070 con 0x7fe7341603d0 2024-04-20T11:30:44.194 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.191+0000 7fe72b400640 1 -- 172.21.15.152:0/3010324479 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe72c014a20 con 0x7fe7341603d0 2024-04-20T11:30:44.194 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.192+0000 7fe739000640 1 -- 172.21.15.152:0/3010324479 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7341f4730 con 0x7fe7341603d0 2024-04-20T11:30:44.194 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.192+0000 7fe739000640 1 -- 172.21.15.152:0/3010324479 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7341f4cf0 con 0x7fe7341603d0 2024-04-20T11:30:44.194 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.193+0000 7fe739000640 1 -- 172.21.15.152:0/3010324479 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe734075e60 con 0x7fe7341603d0 2024-04-20T11:30:44.194 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.194+0000 7fe72b400640 1 -- 172.21.15.152:0/3010324479 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 9) ==== 91193+0+0 (secure 0 0 0) 0x7fe72c014bc0 con 0x7fe7341603d0 2024-04-20T11:30:44.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.195+0000 7fe72b400640 1 --2- 172.21.15.152:0/3010324479 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe71406caa0 0x7fe71406ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:44.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.195+0000 7fe72b400640 1 -- 172.21.15.152:0/3010324479 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7fe72c08c840 con 0x7fe7341603d0 2024-04-20T11:30:44.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.195+0000 7fe733400640 1 --2- 172.21.15.152:0/3010324479 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe71406caa0 0x7fe71406ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:44.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.196+0000 7fe733400640 1 --2- 172.21.15.152:0/3010324479 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe71406caa0 0x7fe71406ef60 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fe724036000 tx=0x7fe72403c040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:44.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.200+0000 7fe72b400640 1 -- 172.21.15.152:0/3010324479 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe72c05a9a0 con 0x7fe7341603d0 2024-04-20T11:30:44.433 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.433+0000 7fe739000640 1 -- 172.21.15.152:0/3010324479 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe7340b6ce0 con 0x7fe7341603d0 2024-04-20T11:30:44.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.434+0000 7fe72b400640 1 -- 172.21.15.152:0/3010324479 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+5208 (secure 0 0 0) 0x7fe72c05a2a0 con 0x7fe7341603d0 2024-04-20T11:30:44.435 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:30:44.435 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4826,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6834/2855025590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2855025590},{"type":"v1","addr":"172.21.15.175:6834","nonce":2855025590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6836/3887716755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":3887716755},{"type":"v1","addr":"172.21.15.152:6836","nonce":3887716755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5712,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/698387399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6835","nonce":698387399},{"type":"v1","addr":"172.21.15.152:6837","nonce":698387399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T11:29:00.276881+0000","modified":"2024-04-20T11:29:03.352174+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5724},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5724":{"gid":5724,"name":"d","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6835/3704251089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6833","nonce":3704251089},{"type":"v1","addr":"172.21.15.175:6835","nonce":3704251089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5724,"qdb_cluster":[5724]},"id":3}]} 2024-04-20T11:30:44.435 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 25 2024-04-20T11:30:44.437 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.436+0000 7fe739000640 1 -- 172.21.15.152:0/3010324479 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe71406caa0 msgr2=0x7fe71406ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:44.438 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.436+0000 7fe739000640 1 --2- 172.21.15.152:0/3010324479 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe71406caa0 0x7fe71406ef60 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fe724036000 tx=0x7fe72403c040 comp rx=0 tx=0).stop 2024-04-20T11:30:44.438 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.436+0000 7fe739000640 1 -- 172.21.15.152:0/3010324479 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe7341603d0 msgr2=0x7fe734160de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:44.438 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.436+0000 7fe739000640 1 --2- 172.21.15.152:0/3010324479 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe7341603d0 0x7fe734160de0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fe72c007de0 tx=0x7fe72c00e4b0 comp rx=0 tx=0).stop 2024-04-20T11:30:44.438 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.437+0000 7fe739000640 1 -- 172.21.15.152:0/3010324479 shutdown_connections 2024-04-20T11:30:44.438 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.437+0000 7fe739000640 1 --2- 172.21.15.152:0/3010324479 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe71406caa0 0x7fe71406ef60 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:44.438 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.437+0000 7fe739000640 1 --2- 172.21.15.152:0/3010324479 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe7341603d0 0x7fe734160de0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:44.438 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.437+0000 7fe739000640 1 --2- 172.21.15.152:0/3010324479 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe73415fcc0 0x7fe7340cb620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:44.438 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.437+0000 7fe739000640 1 --2- 172.21.15.152:0/3010324479 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7340b65c0 0x7fe7341f7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:44.438 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.437+0000 7fe739000640 1 -- 172.21.15.152:0/3010324479 >> 172.21.15.152:0/3010324479 conn(0x7fe73408bfd0 msgr2=0x7fe7340b5d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:44.438 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.437+0000 7fe739000640 1 -- 172.21.15.152:0/3010324479 shutdown_connections 2024-04-20T11:30:44.439 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.437+0000 7fe739000640 1 -- 172.21.15.152:0/3010324479 wait complete. 2024-04-20T11:30:44.455 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:30:44.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.656+0000 7f6b98a00640 1 Processor -- start 2024-04-20T11:30:44.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.656+0000 7f6b98a00640 1 -- start start 2024-04-20T11:30:44.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.658+0000 7f6b98a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b941414e0 0x7f6b94166360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:44.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.658+0000 7f6b98a00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b940cf0a0 con 0x7f6b9413e530 2024-04-20T11:30:44.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.658+0000 7f6b98a00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b940d00e0 con 0x7f6b9406f510 2024-04-20T11:30:44.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.658+0000 7f6b98a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6b94141b00 con 0x7f6b941414e0 2024-04-20T11:30:44.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.658+0000 7f6b92800640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f6b9413e530 0x7f6b9406ee00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:37584/0 (socket says 172.21.15.152:37584) 2024-04-20T11:30:44.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.658+0000 7f6b92800640 1 -- 172.21.15.152:0/3747816209 learned_addr learned my addr 172.21.15.152:0/3747816209 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:30:44.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.658+0000 7f6b91e00640 1 -- 172.21.15.152:0/3747816209 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1911795060 0 0) 0x7f6b84001110 con 0x7f6b9413e530 2024-04-20T11:30:44.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.658+0000 7f6b93200640 1 --2- 172.21.15.152:0/3747816209 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b941414e0 0x7f6b94166360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:44.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.658+0000 7f6b91e00640 1 -- 172.21.15.152:0/3747816209 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6b78005730 con 0x7f6b9413e530 2024-04-20T11:30:44.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.659+0000 7f6b91e00640 1 -- 172.21.15.152:0/3747816209 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 678894669 0 0) 0x7f6b78005730 con 0x7f6b9413e530 2024-04-20T11:30:44.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.659+0000 7f6b91e00640 1 -- 172.21.15.152:0/3747816209 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b941414e0 msgr2=0x7f6b94166360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:44.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.659+0000 7f6b91e00640 1 --2- 172.21.15.152:0/3747816209 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b941414e0 0x7f6b94166360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:44.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.659+0000 7f6b91e00640 1 -- 172.21.15.152:0/3747816209 >> v1:172.21.15.175:6789/0 conn(0x7f6b9406f510 legacy=0x7f6b94140dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:44.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.659+0000 7f6b91e00640 1 -- 172.21.15.152:0/3747816209 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b940cf320 con 0x7f6b9413e530 2024-04-20T11:30:44.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.659+0000 7f6b91e00640 1 -- 172.21.15.152:0/3747816209 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f6b84001840 con 0x7f6b9413e530 2024-04-20T11:30:44.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.659+0000 7f6b91e00640 1 -- 172.21.15.152:0/3747816209 >> v1:172.21.15.152:6789/0 conn(0x7f6b9413e530 legacy=0x7f6b9406ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:44.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.659+0000 7f6b91e00640 1 --2- 172.21.15.152:0/3747816209 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6b78005bf0 0x7f6b78006010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:44.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.659+0000 7f6b91e00640 1 --2- 172.21.15.152:0/3747816209 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b941414e0 0x7f6b78006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:44.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.659+0000 7f6b91e00640 1 --2- 172.21.15.152:0/3747816209 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b78006f50 0x7f6b780073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:44.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.659+0000 7f6b91e00640 1 -- 172.21.15.152:0/3747816209 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f6b7800dd60 con 0x7f6b78005bf0 2024-04-20T11:30:44.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.659+0000 7f6b91e00640 1 -- 172.21.15.152:0/3747816209 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6b7800dee0 con 0x7f6b78006f50 2024-04-20T11:30:44.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.659+0000 7f6b91e00640 1 -- 172.21.15.152:0/3747816209 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6b7800e0b0 con 0x7f6b941414e0 2024-04-20T11:30:44.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.659+0000 7f6b92800640 1 --2- 172.21.15.152:0/3747816209 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6b78005bf0 0x7f6b78006010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:44.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.660+0000 7f6b92800640 1 -- 172.21.15.152:0/3747816209 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b941414e0 msgr2=0x7f6b78006a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:44.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.660+0000 7f6b92800640 1 --2- 172.21.15.152:0/3747816209 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b941414e0 0x7f6b78006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:44.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.660+0000 7f6b92800640 1 -- 172.21.15.152:0/3747816209 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b78006f50 msgr2=0x7f6b780073d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:44.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.660+0000 7f6b92800640 1 --2- 172.21.15.152:0/3747816209 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b78006f50 0x7f6b780073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:44.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.660+0000 7f6b92800640 1 -- 172.21.15.152:0/3747816209 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b78005730 con 0x7f6b78005bf0 2024-04-20T11:30:44.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.661+0000 7f6b92800640 1 --2- 172.21.15.152:0/3747816209 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6b78005bf0 0x7f6b78006010 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f6b8400c4f0 tx=0x7f6b78007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=948c1a3ea1eff57b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:44.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.661+0000 7f6b91e00640 1 -- 172.21.15.152:0/3747816209 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b8400c0e0 con 0x7f6b78005bf0 2024-04-20T11:30:44.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.661+0000 7f6b91e00640 1 -- 172.21.15.152:0/3747816209 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b84030070 con 0x7f6b78005bf0 2024-04-20T11:30:44.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.661+0000 7f6b91e00640 1 -- 172.21.15.152:0/3747816209 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b840058d0 con 0x7f6b78005bf0 2024-04-20T11:30:44.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.662+0000 7f6b98a00640 1 -- 172.21.15.152:0/3747816209 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6b78005bf0 msgr2=0x7f6b78006010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:44.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.662+0000 7f6b98a00640 1 --2- 172.21.15.152:0/3747816209 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6b78005bf0 0x7f6b78006010 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f6b8400c4f0 tx=0x7f6b78007a40 comp rx=0 tx=0).stop 2024-04-20T11:30:44.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.662+0000 7f6b93200640 1 -- 172.21.15.152:0/3747816209 reap_dead start 2024-04-20T11:30:44.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.662+0000 7f6b98a00640 1 -- 172.21.15.152:0/3747816209 shutdown_connections 2024-04-20T11:30:44.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.662+0000 7f6b98a00640 1 -- 172.21.15.152:0/3747816209 >> 172.21.15.152:0/3747816209 conn(0x7f6b94067ae0 msgr2=0x7f6b9406b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:44.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.662+0000 7f6b98a00640 1 -- 172.21.15.152:0/3747816209 shutdown_connections 2024-04-20T11:30:44.664 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.662+0000 7f6b98a00640 1 -- 172.21.15.152:0/3747816209 wait complete. 2024-04-20T11:30:44.664 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.664+0000 7f6b98a00640 1 Processor -- start 2024-04-20T11:30:44.664 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.664+0000 7f6b98a00640 1 -- start start 2024-04-20T11:30:44.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.665+0000 7f6b98a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6b78005bf0 0x7f6b941e5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:44.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.665+0000 7f6b98a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b941e3ae0 0x7f6b9413dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:44.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.665+0000 7f6b98a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b941f0aa0 0x7f6b94143630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:44.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.665+0000 7f6b98a00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f6b94072ce0 con 0x7f6b78005bf0 2024-04-20T11:30:44.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.665+0000 7f6b98a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6b940cfc50 con 0x7f6b941e3ae0 2024-04-20T11:30:44.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.665+0000 7f6b98a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6b940cf8e0 con 0x7f6b941f0aa0 2024-04-20T11:30:44.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.665+0000 7f6b92800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6b78005bf0 0x7f6b941e5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:44.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.665+0000 7f6b92800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6b78005bf0 0x7f6b941e5080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:58862/0 (socket says 172.21.15.152:58862) 2024-04-20T11:30:44.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.665+0000 7f6b92800640 1 -- 172.21.15.152:0/1579133043 learned_addr learned my addr 172.21.15.152:0/1579133043 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:30:44.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.665+0000 7f6b92800640 1 -- 172.21.15.152:0/1579133043 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b941f0aa0 msgr2=0x7f6b94143630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:44.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.665+0000 7f6b92800640 1 --2- 172.21.15.152:0/1579133043 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b941f0aa0 0x7f6b94143630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:44.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.665+0000 7f6b92800640 1 -- 172.21.15.152:0/1579133043 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b941e3ae0 msgr2=0x7f6b9413dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:44.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.665+0000 7f6b8be00640 1 --2- 172.21.15.152:0/1579133043 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b941e3ae0 0x7f6b9413dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:44.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.665+0000 7f6b92800640 1 --2- 172.21.15.152:0/1579133043 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b941e3ae0 0x7f6b9413dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:44.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.666+0000 7f6b92800640 1 -- 172.21.15.152:0/1579133043 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b941e3fe0 con 0x7f6b78005bf0 2024-04-20T11:30:44.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.666+0000 7f6b92800640 1 --2- 172.21.15.152:0/1579133043 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6b78005bf0 0x7f6b941e5080 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f6b84001b40 tx=0x7f6b94155fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:44.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.666+0000 7f6b8aa00640 1 -- 172.21.15.152:0/1579133043 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b84004bf0 con 0x7f6b78005bf0 2024-04-20T11:30:44.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.666+0000 7f6b8aa00640 1 -- 172.21.15.152:0/1579133043 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b84004d90 con 0x7f6b78005bf0 2024-04-20T11:30:44.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.666+0000 7f6b8aa00640 1 -- 172.21.15.152:0/1579133043 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b84007270 con 0x7f6b78005bf0 2024-04-20T11:30:44.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.666+0000 7f6b98a00640 1 -- 172.21.15.152:0/1579133043 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b941e42c0 con 0x7f6b78005bf0 2024-04-20T11:30:44.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.666+0000 7f6b98a00640 1 -- 172.21.15.152:0/1579133043 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b941e4880 con 0x7f6b78005bf0 2024-04-20T11:30:44.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.667+0000 7f6b98a00640 1 -- 172.21.15.152:0/1579133043 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b94075e60 con 0x7f6b78005bf0 2024-04-20T11:30:44.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.668+0000 7f6b8aa00640 1 -- 172.21.15.152:0/1579133043 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 9) ==== 91193+0+0 (secure 0 0 0) 0x7f6b84007410 con 0x7f6b78005bf0 2024-04-20T11:30:44.670 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.669+0000 7f6b8aa00640 1 --2- 172.21.15.152:0/1579133043 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f6b7006caa0 0x7f6b7006ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:44.670 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.669+0000 7f6b8aa00640 1 -- 172.21.15.152:0/1579133043 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7f6b840ad4e0 con 0x7f6b78005bf0 2024-04-20T11:30:44.670 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.669+0000 7f6b8be00640 1 --2- 172.21.15.152:0/1579133043 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f6b7006caa0 0x7f6b7006ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:44.670 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.670+0000 7f6b8be00640 1 --2- 172.21.15.152:0/1579133043 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f6b7006caa0 0x7f6b7006ef60 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f6b940cdcf0 tx=0x7f6b7c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:44.674 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.674+0000 7f6b8aa00640 1 -- 172.21.15.152:0/1579133043 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f6b940cf0a0 con 0x7f6b78005bf0 2024-04-20T11:30:44.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.880+0000 7f6b98a00640 1 -- 172.21.15.152:0/1579133043 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6b940b3490 con 0x7f6b78005bf0 2024-04-20T11:30:44.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.882+0000 7f6b8aa00640 1 -- 172.21.15.152:0/1579133043 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) ==== 74+0+16824 (secure 0 0 0) 0x7f6b8407b5b0 con 0x7f6b78005bf0 2024-04-20T11:30:44.883 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:30:44.883 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":48,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:29:42.078051+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:28:58.007718+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-20T11:28:59.260647+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.6","mappings":[{"from":0,"to":4}]},{"pgid":"7.2c","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:30:44.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.884+0000 7f6b98a00640 1 -- 172.21.15.152:0/1579133043 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f6b7006caa0 msgr2=0x7f6b7006ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:44.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.884+0000 7f6b98a00640 1 --2- 172.21.15.152:0/1579133043 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f6b7006caa0 0x7f6b7006ef60 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f6b940cdcf0 tx=0x7f6b7c009040 comp rx=0 tx=0).stop 2024-04-20T11:30:44.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.884+0000 7f6b98a00640 1 -- 172.21.15.152:0/1579133043 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6b78005bf0 msgr2=0x7f6b941e5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:44.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.884+0000 7f6b98a00640 1 --2- 172.21.15.152:0/1579133043 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6b78005bf0 0x7f6b941e5080 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f6b84001b40 tx=0x7f6b94155fa0 comp rx=0 tx=0).stop 2024-04-20T11:30:44.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.885+0000 7f6b98a00640 1 -- 172.21.15.152:0/1579133043 shutdown_connections 2024-04-20T11:30:44.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.885+0000 7f6b98a00640 1 --2- 172.21.15.152:0/1579133043 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f6b7006caa0 0x7f6b7006ef60 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:44.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.885+0000 7f6b98a00640 1 --2- 172.21.15.152:0/1579133043 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b941f0aa0 0x7f6b94143630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:44.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.885+0000 7f6b98a00640 1 --2- 172.21.15.152:0/1579133043 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b941e3ae0 0x7f6b9413dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:44.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.885+0000 7f6b98a00640 1 --2- 172.21.15.152:0/1579133043 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6b78005bf0 0x7f6b941e5080 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:44.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.885+0000 7f6b98a00640 1 -- 172.21.15.152:0/1579133043 >> 172.21.15.152:0/1579133043 conn(0x7f6b94067ae0 msgr2=0x7f6b940b36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:44.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.885+0000 7f6b98a00640 1 -- 172.21.15.152:0/1579133043 shutdown_connections 2024-04-20T11:30:44.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:44.885+0000 7f6b98a00640 1 -- 172.21.15.152:0/1579133043 wait complete. 2024-04-20T11:30:44.903 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-20T11:30:44.903 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:30:45.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.102+0000 7f8099200640 1 Processor -- start 2024-04-20T11:30:45.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.102+0000 7f8099200640 1 -- start start 2024-04-20T11:30:45.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.104+0000 7f8099200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8094147410 0x7f80940afa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:45.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.104+0000 7f8099200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f8094071cd0 0x7f8094072130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:45.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.104+0000 7f8099200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f809406e010 0x7f809406e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:45.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.104+0000 7f8099200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f8094072670 con 0x7f809406e010 2024-04-20T11:30:45.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.104+0000 7f8099200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f80940aec90 con 0x7f8094071cd0 2024-04-20T11:30:45.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.104+0000 7f8099200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f80940aef70 con 0x7f8094147410 2024-04-20T11:30:45.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.104+0000 7f8093e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f809406e010 0x7f809406e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:45.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.104+0000 7f8093e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f809406e010 0x7f809406e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:58878/0 (socket says 172.21.15.152:58878) 2024-04-20T11:30:45.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.104+0000 7f8093e00640 1 -- 172.21.15.152:0/1853382453 learned_addr learned my addr 172.21.15.152:0/1853382453 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:30:45.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.104+0000 7f8092a00640 1 --2- 172.21.15.152:0/1853382453 >> v2:172.21.15.175:3300/0 conn(0x7f8094071cd0 0x7f8094072130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:45.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.104+0000 7f8093400640 1 --2- 172.21.15.152:0/1853382453 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8094147410 0x7f80940afa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:45.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.105+0000 7f8093e00640 1 -- 172.21.15.152:0/1853382453 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8094147410 msgr2=0x7f80940afa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:45.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.105+0000 7f8093e00640 1 --2- 172.21.15.152:0/1853382453 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8094147410 0x7f80940afa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:45.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.105+0000 7f8093e00640 1 -- 172.21.15.152:0/1853382453 >> v2:172.21.15.175:3300/0 conn(0x7f8094071cd0 msgr2=0x7f8094072130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:45.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.105+0000 7f8093e00640 1 --2- 172.21.15.152:0/1853382453 >> v2:172.21.15.175:3300/0 conn(0x7f8094071cd0 0x7f8094072130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:45.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.105+0000 7f8093e00640 1 -- 172.21.15.152:0/1853382453 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80940af250 con 0x7f809406e010 2024-04-20T11:30:45.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.105+0000 7f8093400640 1 --2- 172.21.15.152:0/1853382453 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8094147410 0x7f80940afa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:30:45.107 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.106+0000 7f8093e00640 1 --2- 172.21.15.152:0/1853382453 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f809406e010 0x7f809406e4a0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f809406ff30 tx=0x7f808c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=599d63e3719dacc8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:45.108 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.106+0000 7f8092000640 1 -- 172.21.15.152:0/1853382453 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f808c02ca70 con 0x7f809406e010 2024-04-20T11:30:45.108 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.106+0000 7f8092000640 1 -- 172.21.15.152:0/1853382453 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f808c02cc10 con 0x7f809406e010 2024-04-20T11:30:45.108 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.106+0000 7f8092000640 1 -- 172.21.15.152:0/1853382453 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f808c0343f0 con 0x7f809406e010 2024-04-20T11:30:45.108 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.107+0000 7f8099200640 1 -- 172.21.15.152:0/1853382453 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f809406e010 msgr2=0x7f809406e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:45.108 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.107+0000 7f8099200640 1 --2- 172.21.15.152:0/1853382453 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f809406e010 0x7f809406e4a0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f809406ff30 tx=0x7f808c02bed0 comp rx=0 tx=0).stop 2024-04-20T11:30:45.108 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.107+0000 7f8099200640 1 -- 172.21.15.152:0/1853382453 shutdown_connections 2024-04-20T11:30:45.108 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.107+0000 7f8099200640 1 --2- 172.21.15.152:0/1853382453 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f809406e010 0x7f809406e4a0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:45.108 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.107+0000 7f8099200640 1 --2- 172.21.15.152:0/1853382453 >> v2:172.21.15.175:3300/0 conn(0x7f8094071cd0 0x7f8094072130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:45.108 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.107+0000 7f8099200640 1 --2- 172.21.15.152:0/1853382453 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8094147410 0x7f80940afa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:45.108 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.107+0000 7f8099200640 1 -- 172.21.15.152:0/1853382453 >> 172.21.15.152:0/1853382453 conn(0x7f8094067ae0 msgr2=0x7f809406b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:45.108 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.107+0000 7f8099200640 1 -- 172.21.15.152:0/1853382453 shutdown_connections 2024-04-20T11:30:45.108 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.107+0000 7f8099200640 1 -- 172.21.15.152:0/1853382453 wait complete. 2024-04-20T11:30:45.109 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.109+0000 7f8099200640 1 Processor -- start 2024-04-20T11:30:45.109 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.109+0000 7f8099200640 1 -- start start 2024-04-20T11:30:45.111 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.110+0000 7f8099200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f809406e010 0x7f809413dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:45.111 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.110+0000 7f8099200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8094071cd0 0x7f80940b5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:45.111 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.110+0000 7f8099200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8094147410 0x7f809413f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:45.111 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.110+0000 7f8099200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f809406e9e0 con 0x7f8094071cd0 2024-04-20T11:30:45.111 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.110+0000 7f8099200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f80940ae980 con 0x7f809406e010 2024-04-20T11:30:45.111 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.110+0000 7f8099200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f80941605c0 con 0x7f8094147410 2024-04-20T11:30:45.111 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.110+0000 7f8092a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8094071cd0 0x7f80940b5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:45.111 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.110+0000 7f8092a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8094071cd0 0x7f80940b5120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:58884/0 (socket says 172.21.15.152:58884) 2024-04-20T11:30:45.111 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.110+0000 7f8092a00640 1 -- 172.21.15.152:0/1465780203 learned_addr learned my addr 172.21.15.152:0/1465780203 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:30:45.112 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.111+0000 7f8093400640 1 --2- 172.21.15.152:0/1465780203 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f809406e010 0x7f809413dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:45.112 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.111+0000 7f8093e00640 1 --2- 172.21.15.152:0/1465780203 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8094147410 0x7f809413f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:45.112 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.111+0000 7f8092a00640 1 -- 172.21.15.152:0/1465780203 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8094147410 msgr2=0x7f809413f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:45.113 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.111+0000 7f8092a00640 1 --2- 172.21.15.152:0/1465780203 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8094147410 0x7f809413f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:45.113 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.111+0000 7f8092a00640 1 -- 172.21.15.152:0/1465780203 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f809406e010 msgr2=0x7f809413dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:45.113 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.111+0000 7f8092a00640 1 --2- 172.21.15.152:0/1465780203 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f809406e010 0x7f809413dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:45.113 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.111+0000 7f8092a00640 1 -- 172.21.15.152:0/1465780203 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80940b34f0 con 0x7f8094071cd0 2024-04-20T11:30:45.113 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.111+0000 7f8092a00640 1 --2- 172.21.15.152:0/1465780203 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8094071cd0 0x7f80940b5120 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f808000d800 tx=0x7f808000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:45.113 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.111+0000 7f808b400640 1 -- 172.21.15.152:0/1465780203 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f808000b6c0 con 0x7f8094071cd0 2024-04-20T11:30:45.113 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.111+0000 7f808b400640 1 -- 172.21.15.152:0/1465780203 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8080019070 con 0x7f8094071cd0 2024-04-20T11:30:45.113 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.111+0000 7f808b400640 1 -- 172.21.15.152:0/1465780203 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f808000ba80 con 0x7f8094071cd0 2024-04-20T11:30:45.113 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.111+0000 7f8099200640 1 -- 172.21.15.152:0/1465780203 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80940b37d0 con 0x7f8094071cd0 2024-04-20T11:30:45.113 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.112+0000 7f8099200640 1 -- 172.21.15.152:0/1465780203 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80940b3d90 con 0x7f8094071cd0 2024-04-20T11:30:45.114 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.113+0000 7f8099200640 1 -- 172.21.15.152:0/1465780203 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8094075e60 con 0x7f8094071cd0 2024-04-20T11:30:45.114 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.114+0000 7f808b400640 1 -- 172.21.15.152:0/1465780203 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 9) ==== 91193+0+0 (secure 0 0 0) 0x7f808000bc20 con 0x7f8094071cd0 2024-04-20T11:30:45.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.114+0000 7f808b400640 1 --2- 172.21.15.152:0/1465780203 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f807406caa0 0x7f807406ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:45.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.115+0000 7f808b400640 1 -- 172.21.15.152:0/1465780203 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7f808008c3c0 con 0x7f8094071cd0 2024-04-20T11:30:45.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.115+0000 7f8093400640 1 --2- 172.21.15.152:0/1465780203 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f807406caa0 0x7f807406ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:45.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.115+0000 7f8093400640 1 --2- 172.21.15.152:0/1465780203 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f807406caa0 0x7f807406ef60 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f808400a910 tx=0x7f8084008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:45.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.119+0000 7f808b400640 1 -- 172.21.15.152:0/1465780203 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f808005a520 con 0x7f8094071cd0 2024-04-20T11:30:45.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.353+0000 7f8099200640 1 -- 172.21.15.152:0/1465780203 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f80941455d0 con 0x7f8094071cd0 2024-04-20T11:30:45.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.354+0000 7f808b400640 1 -- 172.21.15.152:0/1465780203 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+5208 (secure 0 0 0) 0x7f8080059e20 con 0x7f8094071cd0 2024-04-20T11:30:45.355 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:30:45.355 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4826,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6834/2855025590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2855025590},{"type":"v1","addr":"172.21.15.175:6834","nonce":2855025590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6836/3887716755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":3887716755},{"type":"v1","addr":"172.21.15.152:6836","nonce":3887716755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5712,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/698387399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6835","nonce":698387399},{"type":"v1","addr":"172.21.15.152:6837","nonce":698387399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T11:29:00.276881+0000","modified":"2024-04-20T11:29:03.352174+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5724},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5724":{"gid":5724,"name":"d","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6835/3704251089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6833","nonce":3704251089},{"type":"v1","addr":"172.21.15.175:6835","nonce":3704251089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5724,"qdb_cluster":[5724]},"id":3}]} 2024-04-20T11:30:45.355 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 25 2024-04-20T11:30:45.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.356+0000 7f8099200640 1 -- 172.21.15.152:0/1465780203 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f807406caa0 msgr2=0x7f807406ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:45.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.356+0000 7f8099200640 1 --2- 172.21.15.152:0/1465780203 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f807406caa0 0x7f807406ef60 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f808400a910 tx=0x7f8084008040 comp rx=0 tx=0).stop 2024-04-20T11:30:45.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.356+0000 7f8099200640 1 -- 172.21.15.152:0/1465780203 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8094071cd0 msgr2=0x7f80940b5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:45.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.356+0000 7f8099200640 1 --2- 172.21.15.152:0/1465780203 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8094071cd0 0x7f80940b5120 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f808000d800 tx=0x7f808000dcc0 comp rx=0 tx=0).stop 2024-04-20T11:30:45.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.357+0000 7f8099200640 1 -- 172.21.15.152:0/1465780203 shutdown_connections 2024-04-20T11:30:45.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.357+0000 7f8099200640 1 --2- 172.21.15.152:0/1465780203 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f807406caa0 0x7f807406ef60 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:45.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.357+0000 7f8099200640 1 --2- 172.21.15.152:0/1465780203 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8094147410 0x7f809413f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:45.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.357+0000 7f8099200640 1 --2- 172.21.15.152:0/1465780203 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8094071cd0 0x7f80940b5120 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:45.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.357+0000 7f8099200640 1 --2- 172.21.15.152:0/1465780203 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f809406e010 0x7f809413dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:45.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.357+0000 7f8099200640 1 -- 172.21.15.152:0/1465780203 >> 172.21.15.152:0/1465780203 conn(0x7f8094067ae0 msgr2=0x7f80941628f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:45.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.357+0000 7f8099200640 1 -- 172.21.15.152:0/1465780203 shutdown_connections 2024-04-20T11:30:45.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.357+0000 7f8099200640 1 -- 172.21.15.152:0/1465780203 wait complete. 2024-04-20T11:30:45.376 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:30:45.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.565+0000 7fc532400640 1 Processor -- start 2024-04-20T11:30:45.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.565+0000 7fc532400640 1 -- start start 2024-04-20T11:30:45.567 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.566+0000 7fc532400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc52c145f60 0x7fc52c06ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:45.567 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.566+0000 7fc532400640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fc52c0cd0d0 0x7fc52c0cd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:45.567 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.567+0000 7fc532400640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc52c146340 con 0x7fc52c071fe0 2024-04-20T11:30:45.567 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.567+0000 7fc532400640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fc52c06e9f0 con 0x7fc52c0cd0d0 2024-04-20T11:30:45.567 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.567+0000 7fc532400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc52c06f3c0 con 0x7fc52c145f60 2024-04-20T11:30:45.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.567+0000 7fc523e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fc52c0cd0d0 0x7fc52c0cd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:45.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.567+0000 7fc52b400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc52c145f60 0x7fc52c06ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:45.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.567+0000 7fc52b400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc52c145f60 0x7fc52c06ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.152:54028/0 (socket says 172.21.15.152:54028) 2024-04-20T11:30:45.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.567+0000 7fc52b400640 1 -- 172.21.15.152:0/481750909 learned_addr learned my addr 172.21.15.152:0/481750909 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:30:45.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.568+0000 7fc523e00640 1 -- 172.21.15.152:0/481750909 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc52c145f60 msgr2=0x7fc52c06ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:45.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.568+0000 7fc523e00640 1 --2- 172.21.15.152:0/481750909 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc52c145f60 0x7fc52c06ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:45.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.568+0000 7fc523e00640 1 -- 172.21.15.152:0/481750909 >> v1:172.21.15.175:6789/0 conn(0x7fc52c071fe0 legacy=0x7fc52c072470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:45.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.568+0000 7fc523e00640 1 -- 172.21.15.152:0/481750909 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc52c072d50 con 0x7fc52c0cd0d0 2024-04-20T11:30:45.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.570+0000 7fc523e00640 1 --2- 172.21.15.152:0/481750909 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc52c0cd0d0 0x7fc52c0cd550 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc52c06eb70 tx=0x7fc51402b8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=379e26c6018fc6f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:45.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.570+0000 7fc52aa00640 1 -- 172.21.15.152:0/481750909 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc514035020 con 0x7fc52c0cd0d0 2024-04-20T11:30:45.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.570+0000 7fc52aa00640 1 -- 172.21.15.152:0/481750909 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc514031800 con 0x7fc52c0cd0d0 2024-04-20T11:30:45.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.570+0000 7fc52aa00640 1 -- 172.21.15.152:0/481750909 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc514031bd0 con 0x7fc52c0cd0d0 2024-04-20T11:30:45.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.571+0000 7fc532400640 1 -- 172.21.15.152:0/481750909 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc52c0cd0d0 msgr2=0x7fc52c0cd550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:45.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.571+0000 7fc532400640 1 --2- 172.21.15.152:0/481750909 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc52c0cd0d0 0x7fc52c0cd550 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc52c06eb70 tx=0x7fc51402b8a0 comp rx=0 tx=0).stop 2024-04-20T11:30:45.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.571+0000 7fc532400640 1 -- 172.21.15.152:0/481750909 shutdown_connections 2024-04-20T11:30:45.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.571+0000 7fc532400640 1 --2- 172.21.15.152:0/481750909 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc52c0cd0d0 0x7fc52c0cd550 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:45.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.571+0000 7fc532400640 1 --2- 172.21.15.152:0/481750909 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc52c145f60 0x7fc52c06ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:45.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.571+0000 7fc532400640 1 -- 172.21.15.152:0/481750909 >> 172.21.15.152:0/481750909 conn(0x7fc52c067cf0 msgr2=0x7fc52c06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:45.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.571+0000 7fc532400640 1 -- 172.21.15.152:0/481750909 shutdown_connections 2024-04-20T11:30:45.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.571+0000 7fc532400640 1 -- 172.21.15.152:0/481750909 wait complete. 2024-04-20T11:30:45.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.573+0000 7fc532400640 1 Processor -- start 2024-04-20T11:30:45.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.573+0000 7fc532400640 1 -- start start 2024-04-20T11:30:45.574 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.574+0000 7fc532400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc52c071fe0 0x7fc52c0aef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:45.574 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.574+0000 7fc532400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc52c0cd0d0 0x7fc52c138100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:45.574 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.574+0000 7fc532400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc52c145f60 0x7fc52c1e37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:45.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.574+0000 7fc532400640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fc52c06e680 con 0x7fc52c0cd0d0 2024-04-20T11:30:45.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.574+0000 7fc532400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc52c06e3a0 con 0x7fc52c145f60 2024-04-20T11:30:45.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.574+0000 7fc532400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc52c0cf290 con 0x7fc52c071fe0 2024-04-20T11:30:45.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.574+0000 7fc523e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc52c0cd0d0 0x7fc52c138100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:45.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.574+0000 7fc523e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc52c0cd0d0 0x7fc52c138100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:58892/0 (socket says 172.21.15.152:58892) 2024-04-20T11:30:45.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.574+0000 7fc523e00640 1 -- 172.21.15.152:0/2698617967 learned_addr learned my addr 172.21.15.152:0/2698617967 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:30:45.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.575+0000 7fc52b400640 1 --2- 172.21.15.152:0/2698617967 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc52c071fe0 0x7fc52c0aef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:45.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.575+0000 7fc52be00640 1 --2- 172.21.15.152:0/2698617967 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc52c145f60 0x7fc52c1e37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:45.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.575+0000 7fc523e00640 1 -- 172.21.15.152:0/2698617967 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc52c071fe0 msgr2=0x7fc52c0aef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:45.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.575+0000 7fc523e00640 1 --2- 172.21.15.152:0/2698617967 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc52c071fe0 0x7fc52c0aef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:45.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.575+0000 7fc523e00640 1 -- 172.21.15.152:0/2698617967 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc52c145f60 msgr2=0x7fc52c1e37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:45.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.575+0000 7fc523e00640 1 --2- 172.21.15.152:0/2698617967 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc52c145f60 0x7fc52c1e37a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:45.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.575+0000 7fc523e00640 1 -- 172.21.15.152:0/2698617967 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc52c0af530 con 0x7fc52c0cd0d0 2024-04-20T11:30:45.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.575+0000 7fc523e00640 1 --2- 172.21.15.152:0/2698617967 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc52c0cd0d0 0x7fc52c138100 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fc514009890 tx=0x7fc5140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:45.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.576+0000 7fc523400640 1 -- 172.21.15.152:0/2698617967 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc514041070 con 0x7fc52c0cd0d0 2024-04-20T11:30:45.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.576+0000 7fc523400640 1 -- 172.21.15.152:0/2698617967 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc51403ad00 con 0x7fc52c0cd0d0 2024-04-20T11:30:45.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.576+0000 7fc523400640 1 -- 172.21.15.152:0/2698617967 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc514035040 con 0x7fc52c0cd0d0 2024-04-20T11:30:45.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.576+0000 7fc532400640 1 -- 172.21.15.152:0/2698617967 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc52c0adb00 con 0x7fc52c0cd0d0 2024-04-20T11:30:45.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.576+0000 7fc532400640 1 -- 172.21.15.152:0/2698617967 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc52c0ae0c0 con 0x7fc52c0cd0d0 2024-04-20T11:30:45.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.577+0000 7fc532400640 1 -- 172.21.15.152:0/2698617967 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc52c0cddc0 con 0x7fc52c0cd0d0 2024-04-20T11:30:45.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.578+0000 7fc523400640 1 -- 172.21.15.152:0/2698617967 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 9) ==== 91193+0+0 (secure 0 0 0) 0x7fc514038020 con 0x7fc52c0cd0d0 2024-04-20T11:30:45.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.579+0000 7fc523400640 1 --2- 172.21.15.152:0/2698617967 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc50806caa0 0x7fc50806ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:45.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.579+0000 7fc523400640 1 -- 172.21.15.152:0/2698617967 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7fc5140adb60 con 0x7fc52c0cd0d0 2024-04-20T11:30:45.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.579+0000 7fc52b400640 1 --2- 172.21.15.152:0/2698617967 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc50806caa0 0x7fc50806ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:45.580 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.580+0000 7fc52b400640 1 --2- 172.21.15.152:0/2698617967 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc50806caa0 0x7fc50806ef60 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fc51c009ac0 tx=0x7fc51c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:45.583 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.583+0000 7fc523400640 1 -- 172.21.15.152:0/2698617967 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fc52c0cddc0 con 0x7fc52c0cd0d0 2024-04-20T11:30:45.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.820+0000 7fc532400640 1 -- 172.21.15.152:0/2698617967 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc52c137f00 con 0x7fc52c0cd0d0 2024-04-20T11:30:45.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.821+0000 7fc523400640 1 -- 172.21.15.152:0/2698617967 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+5208 (secure 0 0 0) 0x7fc52c137f00 con 0x7fc52c0cd0d0 2024-04-20T11:30:45.822 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:30:45.822 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4826,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6834/2855025590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2855025590},{"type":"v1","addr":"172.21.15.175:6834","nonce":2855025590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6836/3887716755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":3887716755},{"type":"v1","addr":"172.21.15.152:6836","nonce":3887716755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5712,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/698387399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6835","nonce":698387399},{"type":"v1","addr":"172.21.15.152:6837","nonce":698387399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T11:29:00.276881+0000","modified":"2024-04-20T11:29:03.352174+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5724},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5724":{"gid":5724,"name":"d","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6835/3704251089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6833","nonce":3704251089},{"type":"v1","addr":"172.21.15.175:6835","nonce":3704251089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5724,"qdb_cluster":[5724]},"id":3}]} 2024-04-20T11:30:45.822 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 25 2024-04-20T11:30:45.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.823+0000 7fc532400640 1 -- 172.21.15.152:0/2698617967 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc50806caa0 msgr2=0x7fc50806ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:45.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.823+0000 7fc532400640 1 --2- 172.21.15.152:0/2698617967 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc50806caa0 0x7fc50806ef60 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fc51c009ac0 tx=0x7fc51c008040 comp rx=0 tx=0).stop 2024-04-20T11:30:45.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.823+0000 7fc532400640 1 -- 172.21.15.152:0/2698617967 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc52c0cd0d0 msgr2=0x7fc52c138100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:45.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.823+0000 7fc532400640 1 --2- 172.21.15.152:0/2698617967 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc52c0cd0d0 0x7fc52c138100 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fc514009890 tx=0x7fc5140046c0 comp rx=0 tx=0).stop 2024-04-20T11:30:45.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.824+0000 7fc532400640 1 -- 172.21.15.152:0/2698617967 shutdown_connections 2024-04-20T11:30:45.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.824+0000 7fc532400640 1 --2- 172.21.15.152:0/2698617967 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc50806caa0 0x7fc50806ef60 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:45.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.824+0000 7fc532400640 1 --2- 172.21.15.152:0/2698617967 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc52c145f60 0x7fc52c1e37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:45.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.824+0000 7fc532400640 1 --2- 172.21.15.152:0/2698617967 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc52c0cd0d0 0x7fc52c138100 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:45.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.824+0000 7fc532400640 1 --2- 172.21.15.152:0/2698617967 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc52c071fe0 0x7fc52c0aef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:45.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.824+0000 7fc532400640 1 -- 172.21.15.152:0/2698617967 >> 172.21.15.152:0/2698617967 conn(0x7fc52c067cf0 msgr2=0x7fc52c0cc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:45.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.824+0000 7fc532400640 1 -- 172.21.15.152:0/2698617967 shutdown_connections 2024-04-20T11:30:45.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:45.824+0000 7fc532400640 1 -- 172.21.15.152:0/2698617967 wait complete. 2024-04-20T11:30:45.843 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:30:46.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.046+0000 7f88d9200640 1 Processor -- start 2024-04-20T11:30:46.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.046+0000 7f88d9200640 1 -- start start 2024-04-20T11:30:46.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.048+0000 7f88d9200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f88d4145f60 0x7f88d406ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:46.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.048+0000 7f88d9200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f88d40cd0d0 0x7f88d40cd580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:46.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.048+0000 7f88d9200640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88d4146340 con 0x7f88d4071fe0 2024-04-20T11:30:46.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.048+0000 7f88d9200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f88d406e9f0 con 0x7f88d40cd0d0 2024-04-20T11:30:46.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.048+0000 7f88d9200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f88d406f3c0 con 0x7f88d4145f60 2024-04-20T11:30:46.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.048+0000 7f88d2800640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f88d4071fe0 0x7f88d4072470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:57920/0 (socket says 172.21.15.152:57920) 2024-04-20T11:30:46.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.048+0000 7f88d2800640 1 -- 172.21.15.152:0/3384697534 learned_addr learned my addr 172.21.15.152:0/3384697534 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:30:46.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.048+0000 7f88d1400640 1 --2- 172.21.15.152:0/3384697534 >> v2:172.21.15.152:3300/0 conn(0x7f88d40cd0d0 0x7f88d40cd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:46.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.049+0000 7f88d1e00640 1 --2- 172.21.15.152:0/3384697534 >> v2:172.21.15.175:3300/0 conn(0x7f88d4145f60 0x7f88d406ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:46.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.049+0000 7f88d0a00640 1 -- 172.21.15.152:0/3384697534 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1280492505 0 0) 0x7f88d4146340 con 0x7f88d4071fe0 2024-04-20T11:30:46.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.049+0000 7f88d1400640 1 -- 172.21.15.152:0/3384697534 >> v2:172.21.15.175:3300/0 conn(0x7f88d4145f60 msgr2=0x7f88d406ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:46.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.049+0000 7f88d1400640 1 --2- 172.21.15.152:0/3384697534 >> v2:172.21.15.175:3300/0 conn(0x7f88d4145f60 0x7f88d406ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:46.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.049+0000 7f88d1400640 1 -- 172.21.15.152:0/3384697534 >> v1:172.21.15.152:6789/0 conn(0x7f88d4071fe0 legacy=0x7f88d4072470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:46.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.049+0000 7f88d1400640 1 -- 172.21.15.152:0/3384697534 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88d4072d50 con 0x7f88d40cd0d0 2024-04-20T11:30:46.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.050+0000 7f88d1400640 1 --2- 172.21.15.152:0/3384697534 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f88d40cd0d0 0x7f88d40cd580 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f88d406eb70 tx=0x7f88c002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e27d882eb4fce366 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:46.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.050+0000 7f88d0a00640 1 -- 172.21.15.152:0/3384697534 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c0035020 con 0x7f88d40cd0d0 2024-04-20T11:30:46.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.050+0000 7f88d0a00640 1 -- 172.21.15.152:0/3384697534 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88c0031830 con 0x7f88d40cd0d0 2024-04-20T11:30:46.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.050+0000 7f88d0a00640 1 -- 172.21.15.152:0/3384697534 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c0031c00 con 0x7f88d40cd0d0 2024-04-20T11:30:46.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.051+0000 7f88d9200640 1 -- 172.21.15.152:0/3384697534 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f88d40cd0d0 msgr2=0x7f88d40cd580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:46.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.051+0000 7f88d9200640 1 --2- 172.21.15.152:0/3384697534 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f88d40cd0d0 0x7f88d40cd580 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f88d406eb70 tx=0x7f88c002b8d0 comp rx=0 tx=0).stop 2024-04-20T11:30:46.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.051+0000 7f88d9200640 1 -- 172.21.15.152:0/3384697534 shutdown_connections 2024-04-20T11:30:46.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.051+0000 7f88d9200640 1 --2- 172.21.15.152:0/3384697534 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f88d40cd0d0 0x7f88d40cd580 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:46.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.051+0000 7f88d9200640 1 --2- 172.21.15.152:0/3384697534 >> v2:172.21.15.175:3300/0 conn(0x7f88d4145f60 0x7f88d406ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:46.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.051+0000 7f88d9200640 1 -- 172.21.15.152:0/3384697534 >> 172.21.15.152:0/3384697534 conn(0x7f88d4067cf0 msgr2=0x7f88d406b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:46.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.051+0000 7f88d9200640 1 -- 172.21.15.152:0/3384697534 shutdown_connections 2024-04-20T11:30:46.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.051+0000 7f88d9200640 1 -- 172.21.15.152:0/3384697534 wait complete. 2024-04-20T11:30:46.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.053+0000 7f88d9200640 1 Processor -- start 2024-04-20T11:30:46.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.053+0000 7f88d9200640 1 -- start start 2024-04-20T11:30:46.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.054+0000 7f88d9200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f88d4071fe0 0x7f88d40c53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:46.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.054+0000 7f88d9200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f88d40cd0d0 0x7f88d415fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:46.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.054+0000 7f88d9200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88d4145f60 0x7f88d415e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:46.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.054+0000 7f88d9200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f88d406e680 con 0x7f88d40cd0d0 2024-04-20T11:30:46.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.054+0000 7f88d9200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f88d406e3a0 con 0x7f88d4145f60 2024-04-20T11:30:46.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.054+0000 7f88d9200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f88d40cf290 con 0x7f88d4071fe0 2024-04-20T11:30:46.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.054+0000 7f88d1400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f88d40cd0d0 0x7f88d415fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:46.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.054+0000 7f88d1400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f88d40cd0d0 0x7f88d415fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:55592/0 (socket says 172.21.15.152:55592) 2024-04-20T11:30:46.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.054+0000 7f88d1400640 1 -- 172.21.15.152:0/3052792266 learned_addr learned my addr 172.21.15.152:0/3052792266 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:30:46.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.054+0000 7f88d1e00640 1 --2- 172.21.15.152:0/3052792266 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f88d4071fe0 0x7f88d40c53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:46.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.054+0000 7f88d2800640 1 --2- 172.21.15.152:0/3052792266 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88d4145f60 0x7f88d415e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:46.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.055+0000 7f88d1400640 1 -- 172.21.15.152:0/3052792266 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f88d4071fe0 msgr2=0x7f88d40c53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:46.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.055+0000 7f88d1400640 1 --2- 172.21.15.152:0/3052792266 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f88d4071fe0 0x7f88d40c53e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:46.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.055+0000 7f88d1400640 1 -- 172.21.15.152:0/3052792266 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88d4145f60 msgr2=0x7f88d415e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:46.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.055+0000 7f88d1400640 1 --2- 172.21.15.152:0/3052792266 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88d4145f60 0x7f88d415e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:46.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.055+0000 7f88d1400640 1 -- 172.21.15.152:0/3052792266 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88d415eae0 con 0x7f88d40cd0d0 2024-04-20T11:30:46.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.055+0000 7f88d1400640 1 --2- 172.21.15.152:0/3052792266 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f88d40cd0d0 0x7f88d415fb80 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f88c0009860 tx=0x7f88c00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:46.057 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.055+0000 7f88ca000640 1 -- 172.21.15.152:0/3052792266 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c0041070 con 0x7f88d40cd0d0 2024-04-20T11:30:46.057 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.055+0000 7f88ca000640 1 -- 172.21.15.152:0/3052792266 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88c003aa30 con 0x7f88d40cd0d0 2024-04-20T11:30:46.057 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.055+0000 7f88d9200640 1 -- 172.21.15.152:0/3052792266 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88d415edc0 con 0x7f88d40cd0d0 2024-04-20T11:30:46.057 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.055+0000 7f88ca000640 1 -- 172.21.15.152:0/3052792266 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c0035040 con 0x7f88d40cd0d0 2024-04-20T11:30:46.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.055+0000 7f88d9200640 1 -- 172.21.15.152:0/3052792266 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88d415f380 con 0x7f88d40cd0d0 2024-04-20T11:30:46.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.057+0000 7f88d9200640 1 -- 172.21.15.152:0/3052792266 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88d40689f0 con 0x7f88d40cd0d0 2024-04-20T11:30:46.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.057+0000 7f88ca000640 1 -- 172.21.15.152:0/3052792266 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 9) ==== 91193+0+0 (secure 0 0 0) 0x7f88c0038020 con 0x7f88d40cd0d0 2024-04-20T11:30:46.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.058+0000 7f88ca000640 1 --2- 172.21.15.152:0/3052792266 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f88b406cad0 0x7f88b406ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:46.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.058+0000 7f88ca000640 1 -- 172.21.15.152:0/3052792266 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7f88c00ad980 con 0x7f88d40cd0d0 2024-04-20T11:30:46.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.059+0000 7f88d1e00640 1 --2- 172.21.15.152:0/3052792266 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f88b406cad0 0x7f88b406ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:46.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.059+0000 7f88d1e00640 1 --2- 172.21.15.152:0/3052792266 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f88b406cad0 0x7f88b406ef90 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f88c4006fd0 tx=0x7f88c4008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:46.063 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.063+0000 7f88ca000640 1 -- 172.21.15.152:0/3052792266 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f88c007baf0 con 0x7f88d40cd0d0 2024-04-20T11:30:46.249 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.249+0000 7f88d9200640 1 -- 172.21.15.152:0/3052792266 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f88d40b35c0 con 0x7f88d40cd0d0 2024-04-20T11:30:46.252 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.252+0000 7f88ca000640 1 -- 172.21.15.152:0/3052792266 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) ==== 74+0+16824 (secure 0 0 0) 0x7f88c007b3f0 con 0x7f88d40cd0d0 2024-04-20T11:30:46.253 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:30:46.253 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":48,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:29:42.078051+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:28:58.007718+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-20T11:28:59.260647+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.6","mappings":[{"from":0,"to":4}]},{"pgid":"7.2c","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:30:46.255 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.254+0000 7f88d9200640 1 -- 172.21.15.152:0/3052792266 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f88b406cad0 msgr2=0x7f88b406ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:46.255 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.254+0000 7f88d9200640 1 --2- 172.21.15.152:0/3052792266 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f88b406cad0 0x7f88b406ef90 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f88c4006fd0 tx=0x7f88c4008040 comp rx=0 tx=0).stop 2024-04-20T11:30:46.255 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.255+0000 7f88d9200640 1 -- 172.21.15.152:0/3052792266 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f88d40cd0d0 msgr2=0x7f88d415fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:46.256 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.255+0000 7f88d9200640 1 --2- 172.21.15.152:0/3052792266 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f88d40cd0d0 0x7f88d415fb80 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f88c0009860 tx=0x7f88c00046c0 comp rx=0 tx=0).stop 2024-04-20T11:30:46.256 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.255+0000 7f88d9200640 1 -- 172.21.15.152:0/3052792266 shutdown_connections 2024-04-20T11:30:46.256 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.255+0000 7f88d9200640 1 --2- 172.21.15.152:0/3052792266 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f88b406cad0 0x7f88b406ef90 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:46.256 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.255+0000 7f88d9200640 1 --2- 172.21.15.152:0/3052792266 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88d4145f60 0x7f88d415e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:46.256 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.255+0000 7f88d9200640 1 --2- 172.21.15.152:0/3052792266 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f88d40cd0d0 0x7f88d415fb80 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:46.256 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.255+0000 7f88d9200640 1 --2- 172.21.15.152:0/3052792266 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f88d4071fe0 0x7f88d40c53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:46.256 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.255+0000 7f88d9200640 1 -- 172.21.15.152:0/3052792266 >> 172.21.15.152:0/3052792266 conn(0x7f88d4067cf0 msgr2=0x7f88d415bbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:46.256 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.255+0000 7f88d9200640 1 -- 172.21.15.152:0/3052792266 shutdown_connections 2024-04-20T11:30:46.256 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.255+0000 7f88d9200640 1 -- 172.21.15.152:0/3052792266 wait complete. 2024-04-20T11:30:46.276 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-20T11:30:46.469 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.469+0000 7efe91c00640 1 Processor -- start 2024-04-20T11:30:46.469 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.469+0000 7efe91c00640 1 -- start start 2024-04-20T11:30:46.471 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.471+0000 7efe91c00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7efe8c0d0fe0 0x7efe8c0d13c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:46.471 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.471+0000 7efe91c00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7efe8c1622a0 0x7efe8c0cf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:46.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.471+0000 7efe91c00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe8c0d1900 con 0x7efe8c0cf820 2024-04-20T11:30:46.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.471+0000 7efe91c00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7efe8c0d4370 con 0x7efe8c0d0fe0 2024-04-20T11:30:46.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.471+0000 7efe91c00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7efe8c15ef20 con 0x7efe8c1622a0 2024-04-20T11:30:46.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.471+0000 7efe8be00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7efe8c0d0fe0 0x7efe8c0d13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:46.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.471+0000 7efe8be00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7efe8c0d0fe0 0x7efe8c0d13c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:55598/0 (socket says 172.21.15.152:55598) 2024-04-20T11:30:46.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.471+0000 7efe8be00640 1 -- 172.21.15.152:0/4270261479 learned_addr learned my addr 172.21.15.152:0/4270261479 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:30:46.473 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.471+0000 7efe83400640 1 --2- 172.21.15.152:0/4270261479 >> v2:172.21.15.175:3300/0 conn(0x7efe8c1622a0 0x7efe8c0cf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:46.473 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.472+0000 7efe8be00640 1 -- 172.21.15.152:0/4270261479 >> v2:172.21.15.175:3300/0 conn(0x7efe8c1622a0 msgr2=0x7efe8c0cf2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:46.473 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.472+0000 7efe8be00640 1 --2- 172.21.15.152:0/4270261479 >> v2:172.21.15.175:3300/0 conn(0x7efe8c1622a0 0x7efe8c0cf2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:46.473 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.472+0000 7efe8be00640 1 -- 172.21.15.152:0/4270261479 >> v1:172.21.15.175:6789/0 conn(0x7efe8c0cf820 legacy=0x7efe8c0cfc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:46.473 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.472+0000 7efe8be00640 1 -- 172.21.15.152:0/4270261479 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe8c15f200 con 0x7efe8c0d0fe0 2024-04-20T11:30:46.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.473+0000 7efe8be00640 1 --2- 172.21.15.152:0/4270261479 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efe8c0d0fe0 0x7efe8c0d13c0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7efe8c08c950 tx=0x7efe7c02be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84b21af4ff497d7a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:46.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.473+0000 7efe8b400640 1 -- 172.21.15.152:0/4270261479 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe7c02c9f0 con 0x7efe8c0d0fe0 2024-04-20T11:30:46.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.473+0000 7efe8b400640 1 -- 172.21.15.152:0/4270261479 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe7c02cb90 con 0x7efe8c0d0fe0 2024-04-20T11:30:46.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.473+0000 7efe8b400640 1 -- 172.21.15.152:0/4270261479 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe7c0343f0 con 0x7efe8c0d0fe0 2024-04-20T11:30:46.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.474+0000 7efe91c00640 1 -- 172.21.15.152:0/4270261479 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efe8c0d0fe0 msgr2=0x7efe8c0d13c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:46.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.474+0000 7efe91c00640 1 --2- 172.21.15.152:0/4270261479 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efe8c0d0fe0 0x7efe8c0d13c0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7efe8c08c950 tx=0x7efe7c02be50 comp rx=0 tx=0).stop 2024-04-20T11:30:46.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.474+0000 7efe91c00640 1 -- 172.21.15.152:0/4270261479 shutdown_connections 2024-04-20T11:30:46.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.474+0000 7efe91c00640 1 --2- 172.21.15.152:0/4270261479 >> v2:172.21.15.175:3300/0 conn(0x7efe8c1622a0 0x7efe8c0cf2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:46.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.474+0000 7efe91c00640 1 --2- 172.21.15.152:0/4270261479 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efe8c0d0fe0 0x7efe8c0d13c0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:46.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.474+0000 7efe91c00640 1 -- 172.21.15.152:0/4270261479 >> 172.21.15.152:0/4270261479 conn(0x7efe8c08bfd0 msgr2=0x7efe8c0d45d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:46.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.474+0000 7efe91c00640 1 -- 172.21.15.152:0/4270261479 shutdown_connections 2024-04-20T11:30:46.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.474+0000 7efe91c00640 1 -- 172.21.15.152:0/4270261479 wait complete. 2024-04-20T11:30:46.476 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.476+0000 7efe91c00640 1 Processor -- start 2024-04-20T11:30:46.476 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.476+0000 7efe91c00640 1 -- start start 2024-04-20T11:30:46.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.477+0000 7efe91c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efe8c0cf820 0x7efe8c0ca0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:46.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.477+0000 7efe91c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efe8c0d0fe0 0x7efe8c1f8770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:46.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.477+0000 7efe91c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efe8c1622a0 0x7efe8c06e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:46.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.477+0000 7efe91c00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7efe8c15eba0 con 0x7efe8c0d0fe0 2024-04-20T11:30:46.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.477+0000 7efe91c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7efe8c15f7c0 con 0x7efe8c1622a0 2024-04-20T11:30:46.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.477+0000 7efe91c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7efe8c0d03a0 con 0x7efe8c0cf820 2024-04-20T11:30:46.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.477+0000 7efe83400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efe8c0d0fe0 0x7efe8c1f8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:46.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.477+0000 7efe83400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efe8c0d0fe0 0x7efe8c1f8770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:55612/0 (socket says 172.21.15.152:55612) 2024-04-20T11:30:46.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.477+0000 7efe83400640 1 -- 172.21.15.152:0/3595774174 learned_addr learned my addr 172.21.15.152:0/3595774174 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:30:46.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.478+0000 7efe8be00640 1 --2- 172.21.15.152:0/3595774174 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efe8c0cf820 0x7efe8c0ca0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:46.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.478+0000 7efe91200640 1 --2- 172.21.15.152:0/3595774174 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efe8c1622a0 0x7efe8c06e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:46.480 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.478+0000 7efe83400640 1 -- 172.21.15.152:0/3595774174 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efe8c0cf820 msgr2=0x7efe8c0ca0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:46.480 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.478+0000 7efe83400640 1 --2- 172.21.15.152:0/3595774174 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efe8c0cf820 0x7efe8c0ca0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:46.480 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.478+0000 7efe83400640 1 -- 172.21.15.152:0/3595774174 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efe8c1622a0 msgr2=0x7efe8c06e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:46.480 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.478+0000 7efe83400640 1 --2- 172.21.15.152:0/3595774174 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efe8c1622a0 0x7efe8c06e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:46.480 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.478+0000 7efe83400640 1 -- 172.21.15.152:0/3595774174 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe8c14e3a0 con 0x7efe8c0d0fe0 2024-04-20T11:30:46.480 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.478+0000 7efe8be00640 1 --2- 172.21.15.152:0/3595774174 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efe8c0cf820 0x7efe8c0ca0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:30:46.480 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.478+0000 7efe91200640 1 --2- 172.21.15.152:0/3595774174 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efe8c1622a0 0x7efe8c06e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:30:46.480 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.478+0000 7efe83400640 1 --2- 172.21.15.152:0/3595774174 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efe8c0d0fe0 0x7efe8c1f8770 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7efe7400d620 tx=0x7efe7400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:46.480 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.478+0000 7efe88c00640 1 -- 172.21.15.152:0/3595774174 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe7400b620 con 0x7efe8c0d0fe0 2024-04-20T11:30:46.480 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.478+0000 7efe88c00640 1 -- 172.21.15.152:0/3595774174 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe74019070 con 0x7efe8c0d0fe0 2024-04-20T11:30:46.480 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.478+0000 7efe91c00640 1 -- 172.21.15.152:0/3595774174 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe8c14e680 con 0x7efe8c0d0fe0 2024-04-20T11:30:46.480 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.478+0000 7efe88c00640 1 -- 172.21.15.152:0/3595774174 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe7400ba00 con 0x7efe8c0d0fe0 2024-04-20T11:30:46.480 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.478+0000 7efe91c00640 1 -- 172.21.15.152:0/3595774174 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe8c14ec40 con 0x7efe8c0d0fe0 2024-04-20T11:30:46.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.479+0000 7efe91c00640 1 -- 172.21.15.152:0/3595774174 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe8c075e60 con 0x7efe8c0d0fe0 2024-04-20T11:30:46.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.481+0000 7efe88c00640 1 -- 172.21.15.152:0/3595774174 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 9) ==== 91193+0+0 (secure 0 0 0) 0x7efe7400bba0 con 0x7efe8c0d0fe0 2024-04-20T11:30:46.482 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.481+0000 7efe88c00640 1 --2- 172.21.15.152:0/3595774174 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7efe6806caa0 0x7efe6806ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:46.482 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.482+0000 7efe88c00640 1 -- 172.21.15.152:0/3595774174 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7efe7408c2b0 con 0x7efe8c0d0fe0 2024-04-20T11:30:46.482 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.482+0000 7efe8be00640 1 --2- 172.21.15.152:0/3595774174 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7efe6806caa0 0x7efe6806ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:46.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.483+0000 7efe8be00640 1 --2- 172.21.15.152:0/3595774174 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7efe6806caa0 0x7efe6806ef60 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7efe7c035000 tx=0x7efe7c030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:46.487 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.486+0000 7efe88c00640 1 -- 172.21.15.152:0/3595774174 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7efe7405a410 con 0x7efe8c0d0fe0 2024-04-20T11:30:46.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.708+0000 7efe91c00640 1 -- 172.21.15.152:0/3595774174 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7efe8c0b5190 con 0x7efe8c0d0fe0 2024-04-20T11:30:46.710 INFO:tasks.ceph.mon.a.smithi152.stderr:2024-04-20T11:30:46.709+0000 7f8229800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-20T11:30:46.714 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.713+0000 7efe88c00640 1 -- 172.21.15.152:0/3595774174 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v26) ==== 161+0+0 (secure 0 0 0) 0x7efe74059d10 con 0x7efe8c0d0fe0 2024-04-20T11:30:46.714 INFO:teuthology.orchestra.run.smithi152.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-20T11:30:46.719 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.718+0000 7efe91c00640 1 -- 172.21.15.152:0/3595774174 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7efe6806caa0 msgr2=0x7efe6806ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:46.719 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.718+0000 7efe91c00640 1 --2- 172.21.15.152:0/3595774174 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7efe6806caa0 0x7efe6806ef60 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7efe7c035000 tx=0x7efe7c030000 comp rx=0 tx=0).stop 2024-04-20T11:30:46.719 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.718+0000 7efe91c00640 1 -- 172.21.15.152:0/3595774174 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efe8c0d0fe0 msgr2=0x7efe8c1f8770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:46.719 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.718+0000 7efe91c00640 1 --2- 172.21.15.152:0/3595774174 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efe8c0d0fe0 0x7efe8c1f8770 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7efe7400d620 tx=0x7efe7400dae0 comp rx=0 tx=0).stop 2024-04-20T11:30:46.719 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.718+0000 7efe91c00640 1 -- 172.21.15.152:0/3595774174 shutdown_connections 2024-04-20T11:30:46.719 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.718+0000 7efe91c00640 1 --2- 172.21.15.152:0/3595774174 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7efe6806caa0 0x7efe6806ef60 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:46.719 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.718+0000 7efe91c00640 1 --2- 172.21.15.152:0/3595774174 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efe8c1622a0 0x7efe8c06e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:46.719 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.718+0000 7efe91c00640 1 --2- 172.21.15.152:0/3595774174 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efe8c0d0fe0 0x7efe8c1f8770 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:46.719 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.718+0000 7efe91c00640 1 --2- 172.21.15.152:0/3595774174 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efe8c0cf820 0x7efe8c0ca0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:46.719 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.718+0000 7efe91c00640 1 -- 172.21.15.152:0/3595774174 >> 172.21.15.152:0/3595774174 conn(0x7efe8c08bfd0 msgr2=0x7efe8c0b56b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:46.720 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.719+0000 7efe91c00640 1 -- 172.21.15.152:0/3595774174 shutdown_connections 2024-04-20T11:30:46.720 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.719+0000 7efe91c00640 1 -- 172.21.15.152:0/3595774174 wait complete. 2024-04-20T11:30:46.738 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-20T11:30:46.749 INFO:tasks.ceph.mds.d.smithi175.stderr: -9745> 2024-04-20T11:29:05.184+0000 7f1659a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-20T11:30:46.828 INFO:tasks.ceph.mds.d.smithi175.stdout:starting mds.d at 2024-04-20T11:30:46.932 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.931+0000 7f139ba00640 1 Processor -- start 2024-04-20T11:30:46.932 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.932+0000 7f139ba00640 1 -- start start 2024-04-20T11:30:46.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.933+0000 7f139ba00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1394146000 0x7f13940b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:46.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.933+0000 7f139ba00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f13940cd8c0 0x7f13940b25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:46.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.933+0000 7f139ba00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13941463e0 con 0x7f13940b2170 2024-04-20T11:30:46.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.933+0000 7f139ba00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f13940b4750 con 0x7f13940cd8c0 2024-04-20T11:30:46.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.933+0000 7f139ba00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f13940b3e20 con 0x7f1394146000 2024-04-20T11:30:46.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.934+0000 7f1393e00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f13940b2170 0x7f13940cd1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:57928/0 (socket says 172.21.15.152:57928) 2024-04-20T11:30:46.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.934+0000 7f1393e00640 1 -- 172.21.15.152:0/3496675184 learned_addr learned my addr 172.21.15.152:0/3496675184 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:30:46.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.934+0000 7f1399000640 1 --2- 172.21.15.152:0/3496675184 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1394146000 0x7f13940b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:46.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.934+0000 7f1399a00640 1 --2- 172.21.15.152:0/3496675184 >> v2:172.21.15.175:3300/0 conn(0x7f13940cd8c0 0x7f13940b25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:46.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.934+0000 7f1393400640 1 -- 172.21.15.152:0/3496675184 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1852493767 0 0) 0x7f13941463e0 con 0x7f13940b2170 2024-04-20T11:30:46.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.934+0000 7f1393400640 1 -- 172.21.15.152:0/3496675184 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f137c003520 con 0x7f13940b2170 2024-04-20T11:30:46.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.934+0000 7f1393400640 1 -- 172.21.15.152:0/3496675184 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2154229602 0 0) 0x7f137c003520 con 0x7f13940b2170 2024-04-20T11:30:46.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.934+0000 7f1393400640 1 -- 172.21.15.152:0/3496675184 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1394146000 msgr2=0x7f13940b4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:46.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.934+0000 7f1393400640 1 --2- 172.21.15.152:0/3496675184 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1394146000 0x7f13940b4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:46.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.934+0000 7f1393400640 1 -- 172.21.15.152:0/3496675184 >> v2:172.21.15.175:3300/0 conn(0x7f13940cd8c0 msgr2=0x7f13940b25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:46.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.934+0000 7f1393400640 1 --2- 172.21.15.152:0/3496675184 >> v2:172.21.15.175:3300/0 conn(0x7f13940cd8c0 0x7f13940b25f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:46.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.934+0000 7f1393400640 1 -- 172.21.15.152:0/3496675184 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13940b4100 con 0x7f13940b2170 2024-04-20T11:30:46.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.934+0000 7f1393400640 1 -- 172.21.15.152:0/3496675184 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f1380001670 con 0x7f13940b2170 2024-04-20T11:30:46.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.934+0000 7f1393400640 1 -- 172.21.15.152:0/3496675184 >> v1:172.21.15.152:6789/0 conn(0x7f13940b2170 legacy=0x7f13940cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:46.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.935+0000 7f1393400640 1 --2- 172.21.15.152:0/3496675184 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1394146000 0x7f137c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:46.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.935+0000 7f1393400640 1 --2- 172.21.15.152:0/3496675184 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f137c003f80 0x7f137c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:46.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.935+0000 7f1393400640 1 --2- 172.21.15.152:0/3496675184 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f137c004d40 0x7f137c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:46.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.935+0000 7f1393400640 1 -- 172.21.15.152:0/3496675184 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f13940b4750 con 0x7f137c004d40 2024-04-20T11:30:46.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.935+0000 7f1393400640 1 -- 172.21.15.152:0/3496675184 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f13940b3e20 con 0x7f137c003f80 2024-04-20T11:30:46.938 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.935+0000 7f1393400640 1 -- 172.21.15.152:0/3496675184 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f137c00bb20 con 0x7f1394146000 2024-04-20T11:30:46.938 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.935+0000 7f1399a00640 1 --2- 172.21.15.152:0/3496675184 >> v2:172.21.15.175:3300/0 conn(0x7f13940cd8c0 0x7f13940b25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:30:46.938 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.935+0000 7f1399a00640 1 --2- 172.21.15.152:0/3496675184 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f137c004d40 0x7f137c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:46.938 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.935+0000 7f1393e00640 1 --2- 172.21.15.152:0/3496675184 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f137c003f80 0x7f137c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:46.938 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.935+0000 7f1399000640 1 --2- 172.21.15.152:0/3496675184 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1394146000 0x7f137c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:46.938 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.936+0000 7f1399a00640 1 -- 172.21.15.152:0/3496675184 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1394146000 msgr2=0x7f137c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:46.938 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.936+0000 7f1399a00640 1 --2- 172.21.15.152:0/3496675184 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1394146000 0x7f137c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:46.938 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.936+0000 7f1399a00640 1 -- 172.21.15.152:0/3496675184 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f137c003f80 msgr2=0x7f137c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:46.938 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.936+0000 7f1399a00640 1 --2- 172.21.15.152:0/3496675184 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f137c003f80 0x7f137c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:46.938 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.936+0000 7f1399a00640 1 -- 172.21.15.152:0/3496675184 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f137c003520 con 0x7f137c004d40 2024-04-20T11:30:46.938 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.936+0000 7f1399000640 1 --2- 172.21.15.152:0/3496675184 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1394146000 0x7f137c0039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:30:46.939 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.937+0000 7f1399a00640 1 --2- 172.21.15.152:0/3496675184 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f137c004d40 0x7f137c0051c0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f138c0073e0 tx=0x7f137c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a248f74f3fa79d7d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:46.939 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.937+0000 7f1393400640 1 -- 172.21.15.152:0/3496675184 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f138c039070 con 0x7f137c004d40 2024-04-20T11:30:46.939 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.937+0000 7f1393400640 1 -- 172.21.15.152:0/3496675184 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f138c02cdc0 con 0x7f137c004d40 2024-04-20T11:30:46.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.937+0000 7f1393400640 1 -- 172.21.15.152:0/3496675184 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f138c009c70 con 0x7f137c004d40 2024-04-20T11:30:46.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.938+0000 7f139ba00640 1 -- 172.21.15.152:0/3496675184 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f137c004d40 msgr2=0x7f137c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:46.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.938+0000 7f139ba00640 1 --2- 172.21.15.152:0/3496675184 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f137c004d40 0x7f137c0051c0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f138c0073e0 tx=0x7f137c005830 comp rx=0 tx=0).stop 2024-04-20T11:30:46.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.938+0000 7f1399a00640 1 -- 172.21.15.152:0/3496675184 reap_dead start 2024-04-20T11:30:46.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.938+0000 7f139ba00640 1 -- 172.21.15.152:0/3496675184 shutdown_connections 2024-04-20T11:30:46.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.938+0000 7f139ba00640 1 -- 172.21.15.152:0/3496675184 >> 172.21.15.152:0/3496675184 conn(0x7f139406b980 msgr2=0x7f139406bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:46.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.938+0000 7f139ba00640 1 -- 172.21.15.152:0/3496675184 shutdown_connections 2024-04-20T11:30:46.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.938+0000 7f139ba00640 1 -- 172.21.15.152:0/3496675184 wait complete. 2024-04-20T11:30:46.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.940+0000 7f139ba00640 1 Processor -- start 2024-04-20T11:30:46.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.940+0000 7f139ba00640 1 -- start start 2024-04-20T11:30:46.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.941+0000 7f139ba00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f137c004d40 0x7f139415c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:46.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.941+0000 7f139ba00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f13940c6ee0 0x7f13940c7360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:46.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.941+0000 7f139ba00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1394141330 0x7f13941417b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:46.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.941+0000 7f139ba00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f13940b4d60 con 0x7f13940c6ee0 2024-04-20T11:30:46.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.941+0000 7f139ba00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f13940b4be0 con 0x7f1394141330 2024-04-20T11:30:46.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.941+0000 7f139ba00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f13940b4ee0 con 0x7f137c004d40 2024-04-20T11:30:46.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.941+0000 7f1393e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f13940c6ee0 0x7f13940c7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:46.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.941+0000 7f1393e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f13940c6ee0 0x7f13940c7360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:55636/0 (socket says 172.21.15.152:55636) 2024-04-20T11:30:46.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.941+0000 7f1393e00640 1 -- 172.21.15.152:0/2248262663 learned_addr learned my addr 172.21.15.152:0/2248262663 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:30:46.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.942+0000 7f1399000640 1 --2- 172.21.15.152:0/2248262663 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f137c004d40 0x7f139415c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:46.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.942+0000 7f1393e00640 1 -- 172.21.15.152:0/2248262663 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f137c004d40 msgr2=0x7f139415c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:46.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.942+0000 7f1393e00640 1 --2- 172.21.15.152:0/2248262663 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f137c004d40 0x7f139415c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:46.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.942+0000 7f1393e00640 1 -- 172.21.15.152:0/2248262663 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1394141330 msgr2=0x7f13941417b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:46.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.942+0000 7f1399a00640 1 --2- 172.21.15.152:0/2248262663 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1394141330 0x7f13941417b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:46.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.942+0000 7f1393e00640 1 --2- 172.21.15.152:0/2248262663 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1394141330 0x7f13941417b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:46.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.942+0000 7f1393e00640 1 -- 172.21.15.152:0/2248262663 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f139413e9a0 con 0x7f13940c6ee0 2024-04-20T11:30:46.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.942+0000 7f1399a00640 1 --2- 172.21.15.152:0/2248262663 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1394141330 0x7f13941417b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:30:46.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.942+0000 7f1393e00640 1 --2- 172.21.15.152:0/2248262663 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f13940c6ee0 0x7f13940c7360 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f1380007340 tx=0x7f1380007800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:46.944 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.942+0000 7f1390c00640 1 -- 172.21.15.152:0/2248262663 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f138000f8b0 con 0x7f13940c6ee0 2024-04-20T11:30:46.944 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.942+0000 7f1390c00640 1 -- 172.21.15.152:0/2248262663 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1380018070 con 0x7f13940c6ee0 2024-04-20T11:30:46.944 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.942+0000 7f1390c00640 1 -- 172.21.15.152:0/2248262663 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f138000fcb0 con 0x7f13940c6ee0 2024-04-20T11:30:46.944 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.942+0000 7f139ba00640 1 -- 172.21.15.152:0/2248262663 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f139413db40 con 0x7f13940c6ee0 2024-04-20T11:30:46.944 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.942+0000 7f139ba00640 1 -- 172.21.15.152:0/2248262663 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f139413e100 con 0x7f13940c6ee0 2024-04-20T11:30:46.944 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.944+0000 7f139ba00640 1 -- 172.21.15.152:0/2248262663 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13940689f0 con 0x7f13940c6ee0 2024-04-20T11:30:46.945 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.945+0000 7f1390c00640 1 -- 172.21.15.152:0/2248262663 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 9) ==== 91193+0+0 (secure 0 0 0) 0x7f138001d460 con 0x7f13940c6ee0 2024-04-20T11:30:46.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.945+0000 7f1390c00640 1 --2- 172.21.15.152:0/2248262663 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f137406caa0 0x7f137406ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:46.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.946+0000 7f1390c00640 1 -- 172.21.15.152:0/2248262663 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5853+0+0 (secure 0 0 0) 0x7f138008c0d0 con 0x7f13940c6ee0 2024-04-20T11:30:46.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.946+0000 7f1399000640 1 --2- 172.21.15.152:0/2248262663 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f137406caa0 0x7f137406ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:46.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.946+0000 7f1399000640 1 --2- 172.21.15.152:0/2248262663 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f137406caa0 0x7f137406ef60 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f13880099b0 tx=0x7f1388008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:46.950 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:46.950+0000 7f1390c00640 1 -- 172.21.15.152:0/2248262663 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f138005a150 con 0x7f13940c6ee0 2024-04-20T11:30:47.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:47.191+0000 7f139ba00640 1 -- 172.21.15.152:0/2248262663 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f139415e740 con 0x7f13940c6ee0 2024-04-20T11:30:47.725 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:47.724+0000 7f1390c00640 1 -- 172.21.15.152:0/2248262663 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v27) ==== 104+0+0 (secure 0 0 0) 0x7f1380059a50 con 0x7f13940c6ee0 2024-04-20T11:30:47.728 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:47.728+0000 7f139ba00640 1 -- 172.21.15.152:0/2248262663 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f137406caa0 msgr2=0x7f137406ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:47.728 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:47.728+0000 7f139ba00640 1 --2- 172.21.15.152:0/2248262663 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f137406caa0 0x7f137406ef60 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f13880099b0 tx=0x7f1388008040 comp rx=0 tx=0).stop 2024-04-20T11:30:47.728 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:47.728+0000 7f139ba00640 1 -- 172.21.15.152:0/2248262663 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f13940c6ee0 msgr2=0x7f13940c7360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:47.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:47.728+0000 7f139ba00640 1 --2- 172.21.15.152:0/2248262663 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f13940c6ee0 0x7f13940c7360 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f1380007340 tx=0x7f1380007800 comp rx=0 tx=0).stop 2024-04-20T11:30:47.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:47.728+0000 7f139ba00640 1 -- 172.21.15.152:0/2248262663 shutdown_connections 2024-04-20T11:30:47.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:47.728+0000 7f139ba00640 1 --2- 172.21.15.152:0/2248262663 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f137406caa0 0x7f137406ef60 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:47.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:47.728+0000 7f139ba00640 1 --2- 172.21.15.152:0/2248262663 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1394141330 0x7f13941417b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:47.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:47.728+0000 7f139ba00640 1 --2- 172.21.15.152:0/2248262663 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f13940c6ee0 0x7f13940c7360 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:47.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:47.728+0000 7f139ba00640 1 --2- 172.21.15.152:0/2248262663 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f137c004d40 0x7f139415c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:47.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:47.728+0000 7f139ba00640 1 -- 172.21.15.152:0/2248262663 >> 172.21.15.152:0/2248262663 conn(0x7f139406b980 msgr2=0x7f139415cf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:47.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:47.728+0000 7f139ba00640 1 -- 172.21.15.152:0/2248262663 shutdown_connections 2024-04-20T11:30:47.730 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:47.728+0000 7f139ba00640 1 -- 172.21.15.152:0/2248262663 wait complete. 2024-04-20T11:30:47.750 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-20T11:30:47.953 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:47.953+0000 7f5997e00640 1 Processor -- start 2024-04-20T11:30:47.954 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:47.953+0000 7f5997e00640 1 -- start start 2024-04-20T11:30:47.955 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:47.954+0000 7f5997e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f599815b5c0 0x7f5998071f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:47.955 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:47.955+0000 7f5997e00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f599815b9a0 con 0x7f59980cebd0 2024-04-20T11:30:47.955 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:47.955+0000 7f5997e00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5998072610 con 0x7f59980d05c0 2024-04-20T11:30:47.955 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:47.955+0000 7f5997e00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f59980728f0 con 0x7f599815b5c0 2024-04-20T11:30:47.956 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:47.955+0000 7f5996a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f599815b5c0 0x7f5998071f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:47.956 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:47.955+0000 7f5996000640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f59980cebd0 0x7f59980b3d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:57942/0 (socket says 172.21.15.152:57942) 2024-04-20T11:30:47.956 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:47.955+0000 7f5996000640 1 -- 172.21.15.152:0/1507046531 learned_addr learned my addr 172.21.15.152:0/1507046531 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:30:47.956 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:47.955+0000 7f5996a00640 1 -- 172.21.15.152:0/1507046531 >> v1:172.21.15.175:6789/0 conn(0x7f59980d05c0 legacy=0x7f599806db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:47.956 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:47.955+0000 7f5996a00640 1 -- 172.21.15.152:0/1507046531 >> v1:172.21.15.152:6789/0 conn(0x7f59980cebd0 legacy=0x7f59980b3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:47.956 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:47.955+0000 7f5996a00640 1 -- 172.21.15.152:0/1507046531 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59980b3500 con 0x7f599815b5c0 2024-04-20T11:30:47.957 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:47.957+0000 7f5996a00640 1 --2- 172.21.15.152:0/1507046531 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f599815b5c0 0x7f5998071f10 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f59880097a0 tx=0x7f598802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15d5d762bd0843ab server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:47.957 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:47.957+0000 7f5995600640 1 -- 172.21.15.152:0/1507046531 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59880315a0 con 0x7f599815b5c0 2024-04-20T11:30:47.957 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:47.957+0000 7f5995600640 1 -- 172.21.15.152:0/1507046531 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5988037070 con 0x7f599815b5c0 2024-04-20T11:30:47.957 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:47.957+0000 7f5995600640 1 -- 172.21.15.152:0/1507046531 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5988031960 con 0x7f599815b5c0 2024-04-20T11:30:47.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:47.957+0000 7f5997e00640 1 -- 172.21.15.152:0/1507046531 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f599815b5c0 msgr2=0x7f5998071f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:47.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:47.957+0000 7f5997e00640 1 --2- 172.21.15.152:0/1507046531 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f599815b5c0 0x7f5998071f10 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f59880097a0 tx=0x7f598802b840 comp rx=0 tx=0).stop 2024-04-20T11:30:47.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:47.957+0000 7f5997e00640 1 -- 172.21.15.152:0/1507046531 shutdown_connections 2024-04-20T11:30:47.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:47.957+0000 7f5997e00640 1 --2- 172.21.15.152:0/1507046531 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f599815b5c0 0x7f5998071f10 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:47.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:47.957+0000 7f5997e00640 1 -- 172.21.15.152:0/1507046531 >> 172.21.15.152:0/1507046531 conn(0x7f599806bda0 msgr2=0x7f599806c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:47.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:47.958+0000 7f5997e00640 1 -- 172.21.15.152:0/1507046531 shutdown_connections 2024-04-20T11:30:47.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:47.958+0000 7f5997e00640 1 -- 172.21.15.152:0/1507046531 wait complete. 2024-04-20T11:30:47.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:47.959+0000 7f5997e00640 1 Processor -- start 2024-04-20T11:30:47.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:47.959+0000 7f5997e00640 1 -- start start 2024-04-20T11:30:47.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:47.960+0000 7f5997e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59980cebd0 0x7f599809d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:47.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:47.960+0000 7f5997e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f59980d05c0 0x7f59980b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:47.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:47.960+0000 7f5997e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f599815b5c0 0x7f5998159bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:47.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:47.960+0000 7f5997e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f59980b4410 con 0x7f59980d05c0 2024-04-20T11:30:47.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:47.960+0000 7f5997e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f59980cf030 con 0x7f59980cebd0 2024-04-20T11:30:47.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:47.960+0000 7f5997e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f599806e220 con 0x7f599815b5c0 2024-04-20T11:30:47.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:47.961+0000 7f5996000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f59980d05c0 0x7f59980b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:47.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:47.961+0000 7f5996000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f59980d05c0 0x7f59980b15c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:55658/0 (socket says 172.21.15.152:55658) 2024-04-20T11:30:47.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:47.961+0000 7f5996000640 1 -- 172.21.15.152:0/3857509734 learned_addr learned my addr 172.21.15.152:0/3857509734 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:30:47.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:47.961+0000 7f5996a00640 1 --2- 172.21.15.152:0/3857509734 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59980cebd0 0x7f599809d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:47.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:47.961+0000 7f5996000640 1 -- 172.21.15.152:0/3857509734 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f599815b5c0 msgr2=0x7f5998159bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:47.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:47.961+0000 7f5996000640 1 --2- 172.21.15.152:0/3857509734 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f599815b5c0 0x7f5998159bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:47.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:47.961+0000 7f5996000640 1 -- 172.21.15.152:0/3857509734 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59980cebd0 msgr2=0x7f599809d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:47.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:47.961+0000 7f5996000640 1 --2- 172.21.15.152:0/3857509734 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59980cebd0 0x7f599809d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:47.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:47.961+0000 7f5996000640 1 -- 172.21.15.152:0/3857509734 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f599809dc50 con 0x7f59980d05c0 2024-04-20T11:30:47.963 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:47.961+0000 7f5996000640 1 --2- 172.21.15.152:0/3857509734 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f59980d05c0 0x7f59980b15c0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f598400c400 tx=0x7f598400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:47.963 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:47.962+0000 7f598f400640 1 -- 172.21.15.152:0/3857509734 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5984017070 con 0x7f59980d05c0 2024-04-20T11:30:47.963 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:47.962+0000 7f598f400640 1 -- 172.21.15.152:0/3857509734 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5984005ce0 con 0x7f59980d05c0 2024-04-20T11:30:47.963 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:47.962+0000 7f598f400640 1 -- 172.21.15.152:0/3857509734 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59840072f0 con 0x7f59980d05c0 2024-04-20T11:30:47.964 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:47.962+0000 7f5997e00640 1 -- 172.21.15.152:0/3857509734 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59980b1e10 con 0x7f59980d05c0 2024-04-20T11:30:47.964 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:47.962+0000 7f5997e00640 1 -- 172.21.15.152:0/3857509734 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59980b23d0 con 0x7f59980d05c0 2024-04-20T11:30:47.964 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:47.963+0000 7f5997e00640 1 -- 172.21.15.152:0/3857509734 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59980689f0 con 0x7f59980d05c0 2024-04-20T11:30:47.964 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:47.964+0000 7f598f400640 1 -- 172.21.15.152:0/3857509734 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 9) ==== 91193+0+0 (secure 0 0 0) 0x7f598401a070 con 0x7f59980d05c0 2024-04-20T11:30:47.965 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:47.965+0000 7f598f400640 1 --2- 172.21.15.152:0/3857509734 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f597806caa0 0x7f597806ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:47.965 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:47.965+0000 7f598f400640 1 -- 172.21.15.152:0/3857509734 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5853+0+0 (secure 0 0 0) 0x7f598408b330 con 0x7f59980d05c0 2024-04-20T11:30:47.965 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:47.965+0000 7f5996a00640 1 --2- 172.21.15.152:0/3857509734 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f597806caa0 0x7f597806ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:47.966 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:47.966+0000 7f5996a00640 1 --2- 172.21.15.152:0/3857509734 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f597806caa0 0x7f597806ef60 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f5988036000 tx=0x7f5988008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:47.970 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:47.970+0000 7f598f400640 1 -- 172.21.15.152:0/3857509734 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f5984059440 con 0x7f59980d05c0 2024-04-20T11:30:48.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:48.173+0000 7f5997e00640 1 -- 172.21.15.152:0/3857509734 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f59980cfe30 con 0x7f59980d05c0 2024-04-20T11:30:48.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:48.728+0000 7f598f400640 1 -- 172.21.15.152:0/3857509734 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v50) ==== 181+0+0 (secure 0 0 0) 0x7f5984058d40 con 0x7f59980d05c0 2024-04-20T11:30:48.729 INFO:teuthology.orchestra.run.smithi152.stderr:pool 'cephfs_metadata' removed 2024-04-20T11:30:48.733 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:48.732+0000 7f5997e00640 1 -- 172.21.15.152:0/3857509734 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f597806caa0 msgr2=0x7f597806ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:48.733 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:48.732+0000 7f5997e00640 1 --2- 172.21.15.152:0/3857509734 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f597806caa0 0x7f597806ef60 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f5988036000 tx=0x7f5988008030 comp rx=0 tx=0).stop 2024-04-20T11:30:48.733 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:48.732+0000 7f5997e00640 1 -- 172.21.15.152:0/3857509734 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f59980d05c0 msgr2=0x7f59980b15c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:48.733 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:48.732+0000 7f5997e00640 1 --2- 172.21.15.152:0/3857509734 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f59980d05c0 0x7f59980b15c0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f598400c400 tx=0x7f598400c8c0 comp rx=0 tx=0).stop 2024-04-20T11:30:48.733 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:48.732+0000 7f5997e00640 1 -- 172.21.15.152:0/3857509734 shutdown_connections 2024-04-20T11:30:48.733 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:48.732+0000 7f5997e00640 1 --2- 172.21.15.152:0/3857509734 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f597806caa0 0x7f597806ef60 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:48.733 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:48.732+0000 7f5997e00640 1 --2- 172.21.15.152:0/3857509734 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f599815b5c0 0x7f5998159bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:48.733 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:48.732+0000 7f5997e00640 1 --2- 172.21.15.152:0/3857509734 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f59980d05c0 0x7f59980b15c0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:48.733 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:48.732+0000 7f5997e00640 1 --2- 172.21.15.152:0/3857509734 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59980cebd0 0x7f599809d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:48.733 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:48.732+0000 7f5997e00640 1 -- 172.21.15.152:0/3857509734 >> 172.21.15.152:0/3857509734 conn(0x7f599806bda0 msgr2=0x7f599815c890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:48.734 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:48.732+0000 7f5997e00640 1 -- 172.21.15.152:0/3857509734 shutdown_connections 2024-04-20T11:30:48.734 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:48.733+0000 7f5997e00640 1 -- 172.21.15.152:0/3857509734 wait complete. 2024-04-20T11:30:48.762 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-20T11:30:48.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:48.957+0000 7fe0bfe00640 1 Processor -- start 2024-04-20T11:30:48.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:48.957+0000 7fe0bfe00640 1 -- start start 2024-04-20T11:30:48.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:48.959+0000 7fe0bfe00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fe0c00ad760 0x7fe0c015ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:48.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:48.959+0000 7fe0bfe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe0c00d05f0 0x7fe0c00d0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:48.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:48.959+0000 7fe0bfe00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0c015d8d0 con 0x7fe0c01569d0 2024-04-20T11:30:48.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:48.959+0000 7fe0bfe00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fe0c015d310 con 0x7fe0c00ad760 2024-04-20T11:30:48.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:48.959+0000 7fe0bfe00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe0c015d5f0 con 0x7fe0c00d05f0 2024-04-20T11:30:48.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:48.959+0000 7fe0bea00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fe0c01569d0 0x7fe0c006e510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:57950/0 (socket says 172.21.15.152:57950) 2024-04-20T11:30:48.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:48.959+0000 7fe0bea00640 1 -- 172.21.15.152:0/3569634589 learned_addr learned my addr 172.21.15.152:0/3569634589 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:30:48.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:48.959+0000 7fe0be000640 1 --2- 172.21.15.152:0/3569634589 >> v2:172.21.15.175:3300/0 conn(0x7fe0c00ad760 0x7fe0c015ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:48.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:48.960+0000 7fe0bf400640 1 --2- 172.21.15.152:0/3569634589 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe0c00d05f0 0x7fe0c00d0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:48.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:48.960+0000 7fe0bd600640 1 -- 172.21.15.152:0/3569634589 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2377198718 0 0) 0x7fe0c015d8d0 con 0x7fe0c01569d0 2024-04-20T11:30:48.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:48.960+0000 7fe0bd600640 1 -- 172.21.15.152:0/3569634589 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe0a4003520 con 0x7fe0c01569d0 2024-04-20T11:30:48.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:48.960+0000 7fe0bd600640 1 -- 172.21.15.152:0/3569634589 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 473975815 0 0) 0x7fe0a4003520 con 0x7fe0c01569d0 2024-04-20T11:30:48.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:48.960+0000 7fe0bd600640 1 -- 172.21.15.152:0/3569634589 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe0c00d05f0 msgr2=0x7fe0c00d0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:48.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:48.960+0000 7fe0bd600640 1 --2- 172.21.15.152:0/3569634589 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe0c00d05f0 0x7fe0c00d0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:48.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:48.960+0000 7fe0bd600640 1 -- 172.21.15.152:0/3569634589 >> v2:172.21.15.175:3300/0 conn(0x7fe0c00ad760 msgr2=0x7fe0c015ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:48.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:48.960+0000 7fe0bd600640 1 --2- 172.21.15.152:0/3569634589 >> v2:172.21.15.175:3300/0 conn(0x7fe0c00ad760 0x7fe0c015ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:48.963 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:48.960+0000 7fe0bd600640 1 -- 172.21.15.152:0/3569634589 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0c006dd10 con 0x7fe0c01569d0 2024-04-20T11:30:48.963 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:48.960+0000 7fe0bd600640 1 -- 172.21.15.152:0/3569634589 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7fe0b00019f0 con 0x7fe0c01569d0 2024-04-20T11:30:48.963 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:48.960+0000 7fe0bd600640 1 -- 172.21.15.152:0/3569634589 >> v1:172.21.15.152:6789/0 conn(0x7fe0c01569d0 legacy=0x7fe0c006e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:48.963 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:48.960+0000 7fe0bd600640 1 --2- 172.21.15.152:0/3569634589 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe0c00d05f0 0x7fe0a40039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:48.963 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:48.960+0000 7fe0bd600640 1 --2- 172.21.15.152:0/3569634589 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0a4003f30 0x7fe0a4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:48.963 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:48.960+0000 7fe0bd600640 1 --2- 172.21.15.152:0/3569634589 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe0a4004d40 0x7fe0a40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:48.963 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:48.960+0000 7fe0bd600640 1 -- 172.21.15.152:0/3569634589 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c015d310 con 0x7fe0a4004d40 2024-04-20T11:30:48.963 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:48.960+0000 7fe0bd600640 1 -- 172.21.15.152:0/3569634589 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c015d5f0 con 0x7fe0a4003f30 2024-04-20T11:30:48.963 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:48.960+0000 7fe0bd600640 1 -- 172.21.15.152:0/3569634589 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe0a400b9c0 con 0x7fe0c00d05f0 2024-04-20T11:30:48.963 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:48.961+0000 7fe0bf400640 1 --2- 172.21.15.152:0/3569634589 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe0a4004d40 0x7fe0a40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:48.963 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:48.961+0000 7fe0bea00640 1 --2- 172.21.15.152:0/3569634589 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe0c00d05f0 0x7fe0a40039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:48.963 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:48.961+0000 7fe0be000640 1 --2- 172.21.15.152:0/3569634589 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0a4003f30 0x7fe0a4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:48.963 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:48.961+0000 7fe0bf400640 1 -- 172.21.15.152:0/3569634589 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe0c00d05f0 msgr2=0x7fe0a40039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:48.963 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:48.961+0000 7fe0bf400640 1 --2- 172.21.15.152:0/3569634589 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe0c00d05f0 0x7fe0a40039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:48.963 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:48.961+0000 7fe0bf400640 1 -- 172.21.15.152:0/3569634589 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0a4003f30 msgr2=0x7fe0a4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:48.964 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:48.961+0000 7fe0bf400640 1 --2- 172.21.15.152:0/3569634589 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0a4003f30 0x7fe0a4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:48.964 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:48.961+0000 7fe0bf400640 1 -- 172.21.15.152:0/3569634589 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0a4003520 con 0x7fe0a4004d40 2024-04-20T11:30:48.965 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:48.962+0000 7fe0bf400640 1 --2- 172.21.15.152:0/3569634589 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe0a4004d40 0x7fe0a40051c0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fe0b8009ac0 tx=0x7fe0a4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b47fb63a072d211a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:48.965 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:48.962+0000 7fe0bd600640 1 -- 172.21.15.152:0/3569634589 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0b8039070 con 0x7fe0a4004d40 2024-04-20T11:30:48.965 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:48.963+0000 7fe0bd600640 1 -- 172.21.15.152:0/3569634589 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0b802c6b0 con 0x7fe0a4004d40 2024-04-20T11:30:48.965 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:48.963+0000 7fe0bd600640 1 -- 172.21.15.152:0/3569634589 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0b8009690 con 0x7fe0a4004d40 2024-04-20T11:30:48.965 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:48.963+0000 7fe0bfe00640 1 -- 172.21.15.152:0/3569634589 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe0a4004d40 msgr2=0x7fe0a40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:48.965 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:48.963+0000 7fe0bfe00640 1 --2- 172.21.15.152:0/3569634589 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe0a4004d40 0x7fe0a40051c0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fe0b8009ac0 tx=0x7fe0a4005830 comp rx=0 tx=0).stop 2024-04-20T11:30:48.965 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:48.963+0000 7fe0bf400640 1 -- 172.21.15.152:0/3569634589 reap_dead start 2024-04-20T11:30:48.965 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:48.963+0000 7fe0bfe00640 1 -- 172.21.15.152:0/3569634589 shutdown_connections 2024-04-20T11:30:48.965 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:48.963+0000 7fe0bfe00640 1 -- 172.21.15.152:0/3569634589 >> 172.21.15.152:0/3569634589 conn(0x7fe0c00b21e0 msgr2=0x7fe0c00b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:48.965 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:48.963+0000 7fe0bfe00640 1 -- 172.21.15.152:0/3569634589 shutdown_connections 2024-04-20T11:30:48.965 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:48.963+0000 7fe0bfe00640 1 -- 172.21.15.152:0/3569634589 wait complete. 2024-04-20T11:30:48.966 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:48.965+0000 7fe0bfe00640 1 Processor -- start 2024-04-20T11:30:48.967 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:48.965+0000 7fe0bfe00640 1 -- start start 2024-04-20T11:30:48.968 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:48.966+0000 7fe0bfe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe0a4004d40 0x7fe0c009dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:48.968 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:48.966+0000 7fe0bfe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe0c0157d90 0x7fe0c0156f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:48.968 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:48.966+0000 7fe0bfe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0c0157470 0x7fe0c00b4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:48.968 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:48.966+0000 7fe0bfe00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c015cf50 con 0x7fe0a4004d40 2024-04-20T11:30:48.969 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:48.966+0000 7fe0bfe00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c0156db0 con 0x7fe0c0157470 2024-04-20T11:30:48.969 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:48.966+0000 7fe0bfe00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe0c015d0d0 con 0x7fe0c0157d90 2024-04-20T11:30:48.969 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:48.967+0000 7fe0bea00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe0a4004d40 0x7fe0c009dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:48.969 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:48.967+0000 7fe0bea00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe0a4004d40 0x7fe0c009dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:55666/0 (socket says 172.21.15.152:55666) 2024-04-20T11:30:48.969 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:48.967+0000 7fe0bea00640 1 -- 172.21.15.152:0/623449299 learned_addr learned my addr 172.21.15.152:0/623449299 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:30:48.969 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:48.967+0000 7fe0bf400640 1 --2- 172.21.15.152:0/623449299 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0c0157470 0x7fe0c00b4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:48.969 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:48.967+0000 7fe0be000640 1 --2- 172.21.15.152:0/623449299 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe0c0157d90 0x7fe0c0156f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:48.969 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:48.967+0000 7fe0bea00640 1 -- 172.21.15.152:0/623449299 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe0c0157d90 msgr2=0x7fe0c0156f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:48.969 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:48.967+0000 7fe0bea00640 1 --2- 172.21.15.152:0/623449299 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe0c0157d90 0x7fe0c0156f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:48.969 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:48.967+0000 7fe0bea00640 1 -- 172.21.15.152:0/623449299 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0c0157470 msgr2=0x7fe0c00b4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:48.969 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:48.967+0000 7fe0bea00640 1 --2- 172.21.15.152:0/623449299 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0c0157470 0x7fe0c00b4590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:48.969 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:48.967+0000 7fe0bea00640 1 -- 172.21.15.152:0/623449299 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0c01f5440 con 0x7fe0a4004d40 2024-04-20T11:30:48.969 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:48.967+0000 7fe0bea00640 1 --2- 172.21.15.152:0/623449299 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe0a4004d40 0x7fe0c009dd60 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fe0b0007bb0 tx=0x7fe0b000e5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:48.969 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:48.968+0000 7fe0b6a00640 1 -- 172.21.15.152:0/623449299 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0b0019070 con 0x7fe0a4004d40 2024-04-20T11:30:48.970 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:48.968+0000 7fe0b6a00640 1 -- 172.21.15.152:0/623449299 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0b0004b60 con 0x7fe0a4004d40 2024-04-20T11:30:48.970 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:48.968+0000 7fe0b6a00640 1 -- 172.21.15.152:0/623449299 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0b0014480 con 0x7fe0a4004d40 2024-04-20T11:30:48.970 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:48.968+0000 7fe0bfe00640 1 -- 172.21.15.152:0/623449299 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0c01f56b0 con 0x7fe0a4004d40 2024-04-20T11:30:48.970 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:48.968+0000 7fe0bfe00640 1 -- 172.21.15.152:0/623449299 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0c01f5c70 con 0x7fe0a4004d40 2024-04-20T11:30:48.970 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:48.969+0000 7fe0bfe00640 1 -- 172.21.15.152:0/623449299 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0c0074c30 con 0x7fe0a4004d40 2024-04-20T11:30:48.970 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:48.970+0000 7fe0b6a00640 1 -- 172.21.15.152:0/623449299 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 9) ==== 91193+0+0 (secure 0 0 0) 0x7fe0b001c070 con 0x7fe0a4004d40 2024-04-20T11:30:48.972 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:48.971+0000 7fe0b6a00640 1 --2- 172.21.15.152:0/623449299 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe0a006caa0 0x7fe0a006ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:48.972 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:48.971+0000 7fe0b6a00640 1 -- 172.21.15.152:0/623449299 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5415+0+0 (secure 0 0 0) 0x7fe0b008c110 con 0x7fe0a4004d40 2024-04-20T11:30:48.972 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:48.971+0000 7fe0be000640 1 --2- 172.21.15.152:0/623449299 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe0a006caa0 0x7fe0a006ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:48.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:48.972+0000 7fe0be000640 1 --2- 172.21.15.152:0/623449299 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe0a006caa0 0x7fe0a006ef60 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fe0c015e890 tx=0x7fe0ac008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:48.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:48.976+0000 7fe0b6a00640 1 -- 172.21.15.152:0/623449299 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe0c0074c30 con 0x7fe0a4004d40 2024-04-20T11:30:49.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:49.177+0000 7fe0bfe00640 1 -- 172.21.15.152:0/623449299 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fe0c0075e60 con 0x7fe0a4004d40 2024-04-20T11:30:49.739 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:49.738+0000 7fe0b6a00640 1 -- 172.21.15.152:0/623449299 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v51) ==== 169+0+0 (secure 0 0 0) 0x7fe0b0059cf0 con 0x7fe0a4004d40 2024-04-20T11:30:49.740 INFO:teuthology.orchestra.run.smithi152.stderr:pool 'cephfs_data' removed 2024-04-20T11:30:49.743 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:49.742+0000 7fe0bfe00640 1 -- 172.21.15.152:0/623449299 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe0a006caa0 msgr2=0x7fe0a006ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:49.743 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:49.742+0000 7fe0bfe00640 1 --2- 172.21.15.152:0/623449299 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe0a006caa0 0x7fe0a006ef60 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fe0c015e890 tx=0x7fe0ac008040 comp rx=0 tx=0).stop 2024-04-20T11:30:49.743 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:49.743+0000 7fe0bfe00640 1 -- 172.21.15.152:0/623449299 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe0a4004d40 msgr2=0x7fe0c009dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:49.743 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:49.743+0000 7fe0bfe00640 1 --2- 172.21.15.152:0/623449299 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe0a4004d40 0x7fe0c009dd60 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fe0b0007bb0 tx=0x7fe0b000e5f0 comp rx=0 tx=0).stop 2024-04-20T11:30:49.744 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:49.743+0000 7fe0bfe00640 1 -- 172.21.15.152:0/623449299 shutdown_connections 2024-04-20T11:30:49.744 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:49.743+0000 7fe0bfe00640 1 --2- 172.21.15.152:0/623449299 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe0a006caa0 0x7fe0a006ef60 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:49.745 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:49.743+0000 7fe0bfe00640 1 --2- 172.21.15.152:0/623449299 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0c0157470 0x7fe0c00b4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:49.745 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:49.743+0000 7fe0bfe00640 1 --2- 172.21.15.152:0/623449299 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe0c0157d90 0x7fe0c0156f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:49.745 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:49.743+0000 7fe0bfe00640 1 --2- 172.21.15.152:0/623449299 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe0a4004d40 0x7fe0c009dd60 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:49.745 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:49.743+0000 7fe0bfe00640 1 -- 172.21.15.152:0/623449299 >> 172.21.15.152:0/623449299 conn(0x7fe0c00b21e0 msgr2=0x7fe0c00afe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:49.745 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:49.743+0000 7fe0bfe00640 1 -- 172.21.15.152:0/623449299 shutdown_connections 2024-04-20T11:30:49.745 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:49.743+0000 7fe0bfe00640 1 -- 172.21.15.152:0/623449299 wait complete. 2024-04-20T11:30:49.771 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-20T11:30:49.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:49.960+0000 7f8a88800640 1 Processor -- start 2024-04-20T11:30:49.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:49.960+0000 7f8a88800640 1 -- start start 2024-04-20T11:30:49.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:49.961+0000 7f8a88800640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f8a80146000 0x7f8a800b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:49.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:49.961+0000 7f8a88800640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a801463e0 con 0x7f8a800cd8c0 2024-04-20T11:30:49.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:49.961+0000 7f8a88800640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a800b3e20 con 0x7f8a800b2170 2024-04-20T11:30:49.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:49.961+0000 7f8a88800640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f8a800cdca0 con 0x7f8a80146000 2024-04-20T11:30:49.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:49.961+0000 7f8a86800640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f8a800cd8c0 0x7f8a8015ccf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:57964/0 (socket says 172.21.15.152:57964) 2024-04-20T11:30:49.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:49.961+0000 7f8a86800640 1 -- 172.21.15.152:0/2946605171 learned_addr learned my addr 172.21.15.152:0/2946605171 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:30:49.964 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:49.962+0000 7f8a84a00640 1 -- 172.21.15.152:0/2946605171 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 694848996 0 0) 0x7f8a801463e0 con 0x7f8a800cd8c0 2024-04-20T11:30:49.964 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:49.962+0000 7f8a84a00640 1 -- 172.21.15.152:0/2946605171 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8a68003520 con 0x7f8a800cd8c0 2024-04-20T11:30:49.964 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:49.962+0000 7f8a85e00640 1 --2- 172.21.15.152:0/2946605171 >> v2:172.21.15.175:3300/0 conn(0x7f8a80146000 0x7f8a800b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:49.964 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:49.962+0000 7f8a84a00640 1 -- 172.21.15.152:0/2946605171 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3217661189 0 0) 0x7f8a68003520 con 0x7f8a800cd8c0 2024-04-20T11:30:49.964 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:49.962+0000 7f8a84a00640 1 -- 172.21.15.152:0/2946605171 >> v2:172.21.15.175:3300/0 conn(0x7f8a80146000 msgr2=0x7f8a800b4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:49.964 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:49.962+0000 7f8a84a00640 1 --2- 172.21.15.152:0/2946605171 >> v2:172.21.15.175:3300/0 conn(0x7f8a80146000 0x7f8a800b4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:49.964 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:49.962+0000 7f8a84a00640 1 -- 172.21.15.152:0/2946605171 >> v1:172.21.15.175:6789/0 conn(0x7f8a800b2170 legacy=0x7f8a800cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:49.964 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:49.962+0000 7f8a84a00640 1 -- 172.21.15.152:0/2946605171 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a800b4100 con 0x7f8a800cd8c0 2024-04-20T11:30:49.964 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:49.962+0000 7f8a84a00640 1 -- 172.21.15.152:0/2946605171 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f8a7c0019f0 con 0x7f8a800cd8c0 2024-04-20T11:30:49.965 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:49.962+0000 7f8a84a00640 1 -- 172.21.15.152:0/2946605171 >> v1:172.21.15.152:6789/0 conn(0x7f8a800cd8c0 legacy=0x7f8a8015ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:49.965 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:49.962+0000 7f8a84a00640 1 --2- 172.21.15.152:0/2946605171 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8a68003a00 0x7f8a68003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:49.965 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:49.963+0000 7f8a84a00640 1 --2- 172.21.15.152:0/2946605171 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8a68004800 0x7f8a68004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:49.965 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:49.963+0000 7f8a84a00640 1 --2- 172.21.15.152:0/2946605171 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8a68005210 0x7f8a6800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:49.965 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:49.963+0000 7f8a84a00640 1 -- 172.21.15.152:0/2946605171 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f8a800cdca0 con 0x7f8a68005210 2024-04-20T11:30:49.965 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:49.963+0000 7f8a84a00640 1 -- 172.21.15.152:0/2946605171 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8a6800c160 con 0x7f8a68003a00 2024-04-20T11:30:49.965 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:49.963+0000 7f8a84a00640 1 -- 172.21.15.152:0/2946605171 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8a6800c360 con 0x7f8a68004800 2024-04-20T11:30:49.965 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:49.963+0000 7f8a86800640 1 --2- 172.21.15.152:0/2946605171 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8a68005210 0x7f8a6800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:49.965 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:49.963+0000 7f8a85e00640 1 --2- 172.21.15.152:0/2946605171 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8a68003a00 0x7f8a68003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:49.965 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:49.963+0000 7f8a85400640 1 --2- 172.21.15.152:0/2946605171 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8a68004800 0x7f8a68004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:49.965 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:49.963+0000 7f8a86800640 1 -- 172.21.15.152:0/2946605171 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8a68004800 msgr2=0x7f8a68004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:49.965 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:49.963+0000 7f8a86800640 1 --2- 172.21.15.152:0/2946605171 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8a68004800 0x7f8a68004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:49.965 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:49.963+0000 7f8a86800640 1 -- 172.21.15.152:0/2946605171 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8a68003a00 msgr2=0x7f8a68003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:49.965 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:49.963+0000 7f8a86800640 1 --2- 172.21.15.152:0/2946605171 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8a68003a00 0x7f8a68003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:49.966 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:49.963+0000 7f8a86800640 1 -- 172.21.15.152:0/2946605171 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a68003520 con 0x7f8a68005210 2024-04-20T11:30:49.966 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:49.963+0000 7f8a85e00640 1 --2- 172.21.15.152:0/2946605171 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8a68003a00 0x7f8a68003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:30:49.967 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:49.964+0000 7f8a86800640 1 -- 172.21.15.152:0/2946605171 reap_dead start 2024-04-20T11:30:49.967 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:49.964+0000 7f8a86800640 1 --2- 172.21.15.152:0/2946605171 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8a68005210 0x7f8a6800b9c0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f8a7c00a740 tx=0x7f8a68005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b54542f8d770f1a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:49.967 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:49.965+0000 7f8a84a00640 1 -- 172.21.15.152:0/2946605171 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a7c00a390 con 0x7f8a68005210 2024-04-20T11:30:49.967 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:49.965+0000 7f8a84a00640 1 -- 172.21.15.152:0/2946605171 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a7c034650 con 0x7f8a68005210 2024-04-20T11:30:49.967 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:49.965+0000 7f8a84a00640 1 -- 172.21.15.152:0/2946605171 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a7c034a40 con 0x7f8a68005210 2024-04-20T11:30:49.967 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:49.965+0000 7f8a88800640 1 -- 172.21.15.152:0/2946605171 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8a68005210 msgr2=0x7f8a6800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:49.967 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:49.965+0000 7f8a88800640 1 --2- 172.21.15.152:0/2946605171 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8a68005210 0x7f8a6800b9c0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f8a7c00a740 tx=0x7f8a68005830 comp rx=0 tx=0).stop 2024-04-20T11:30:49.967 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:49.966+0000 7f8a88800640 1 -- 172.21.15.152:0/2946605171 shutdown_connections 2024-04-20T11:30:49.967 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:49.966+0000 7f8a88800640 1 --2- 172.21.15.152:0/2946605171 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8a68005210 0x7f8a6800b9c0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:49.967 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:49.966+0000 7f8a88800640 1 -- 172.21.15.152:0/2946605171 >> 172.21.15.152:0/2946605171 conn(0x7f8a8006b980 msgr2=0x7f8a8006bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:49.967 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:49.966+0000 7f8a88800640 1 -- 172.21.15.152:0/2946605171 shutdown_connections 2024-04-20T11:30:49.968 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:49.966+0000 7f8a88800640 1 -- 172.21.15.152:0/2946605171 wait complete. 2024-04-20T11:30:49.968 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:49.967+0000 7f8a88800640 1 Processor -- start 2024-04-20T11:30:49.968 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:49.967+0000 7f8a88800640 1 -- start start 2024-04-20T11:30:49.970 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:49.968+0000 7f8a88800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8a68005210 0x7f8a801f41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:49.970 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:49.968+0000 7f8a88800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8a8006dc50 0x7f8a80143740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:49.970 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:49.968+0000 7f8a88800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8a800c8160 0x7f8a800c85e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:49.970 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:49.968+0000 7f8a88800640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f8a800b4be0 con 0x7f8a8006dc50 2024-04-20T11:30:49.970 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:49.968+0000 7f8a88800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8a800b4750 con 0x7f8a68005210 2024-04-20T11:30:49.970 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:49.968+0000 7f8a88800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8a800b4d60 con 0x7f8a800c8160 2024-04-20T11:30:49.970 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:49.969+0000 7f8a85400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8a8006dc50 0x7f8a80143740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:49.970 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:49.969+0000 7f8a85400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8a8006dc50 0x7f8a80143740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:55692/0 (socket says 172.21.15.152:55692) 2024-04-20T11:30:49.970 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:49.969+0000 7f8a85400640 1 -- 172.21.15.152:0/2080292799 learned_addr learned my addr 172.21.15.152:0/2080292799 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:30:49.970 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:49.969+0000 7f8a85e00640 1 --2- 172.21.15.152:0/2080292799 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8a68005210 0x7f8a801f41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:49.970 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:49.969+0000 7f8a86800640 1 --2- 172.21.15.152:0/2080292799 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8a800c8160 0x7f8a800c85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:49.970 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:49.969+0000 7f8a85400640 1 -- 172.21.15.152:0/2080292799 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8a800c8160 msgr2=0x7f8a800c85e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:49.971 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:49.969+0000 7f8a85400640 1 --2- 172.21.15.152:0/2080292799 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8a800c8160 0x7f8a800c85e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:49.971 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:49.969+0000 7f8a85400640 1 -- 172.21.15.152:0/2080292799 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8a68005210 msgr2=0x7f8a801f41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:49.971 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:49.969+0000 7f8a85400640 1 --2- 172.21.15.152:0/2080292799 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8a68005210 0x7f8a801f41c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:49.971 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:49.969+0000 7f8a85400640 1 -- 172.21.15.152:0/2080292799 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a8006e0d0 con 0x7f8a8006dc50 2024-04-20T11:30:49.971 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:49.969+0000 7f8a86800640 1 --2- 172.21.15.152:0/2080292799 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8a800c8160 0x7f8a800c85e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:30:49.971 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:49.969+0000 7f8a85400640 1 --2- 172.21.15.152:0/2080292799 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8a8006dc50 0x7f8a80143740 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f8a7000e4e0 tx=0x7f8a7000e9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:49.972 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:49.970+0000 7f8a7a000640 1 -- 172.21.15.152:0/2080292799 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a700052e0 con 0x7f8a8006dc50 2024-04-20T11:30:49.972 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:49.970+0000 7f8a7a000640 1 -- 172.21.15.152:0/2080292799 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a70018070 con 0x7f8a8006dc50 2024-04-20T11:30:49.972 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:49.970+0000 7f8a7a000640 1 -- 172.21.15.152:0/2080292799 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a70003830 con 0x7f8a8006dc50 2024-04-20T11:30:49.972 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:49.970+0000 7f8a88800640 1 -- 172.21.15.152:0/2080292799 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a80144090 con 0x7f8a8006dc50 2024-04-20T11:30:49.972 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:49.970+0000 7f8a88800640 1 -- 172.21.15.152:0/2080292799 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a8006e630 con 0x7f8a8006dc50 2024-04-20T11:30:49.972 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:49.971+0000 7f8a88800640 1 -- 172.21.15.152:0/2080292799 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a800689f0 con 0x7f8a8006dc50 2024-04-20T11:30:49.972 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:49.972+0000 7f8a7a000640 1 -- 172.21.15.152:0/2080292799 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 9) ==== 91193+0+0 (secure 0 0 0) 0x7f8a700039d0 con 0x7f8a8006dc50 2024-04-20T11:30:49.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:49.973+0000 7f8a7a000640 1 --2- 172.21.15.152:0/2080292799 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f8a6406caa0 0x7f8a6406ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:49.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:49.973+0000 7f8a7a000640 1 -- 172.21.15.152:0/2080292799 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 4943+0+0 (secure 0 0 0) 0x7f8a7008b840 con 0x7f8a8006dc50 2024-04-20T11:30:49.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:49.973+0000 7f8a85e00640 1 --2- 172.21.15.152:0/2080292799 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f8a6406caa0 0x7f8a6406ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:49.974 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:49.974+0000 7f8a85e00640 1 --2- 172.21.15.152:0/2080292799 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f8a6406caa0 0x7f8a6406ef60 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f8a74009990 tx=0x7f8a74008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:49.978 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:49.978+0000 7f8a7a000640 1 -- 172.21.15.152:0/2080292799 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f8a8006e0d0 con 0x7f8a8006dc50 2024-04-20T11:30:50.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.178+0000 7f8a88800640 1 -- 172.21.15.152:0/2080292799 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f8a800b0770 con 0x7f8a8006dc50 2024-04-20T11:30:50.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.183+0000 7f8a7a000640 1 -- 172.21.15.152:0/2080292799 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) ==== 104+0+0 (secure 0 0 0) 0x7f8a70059c50 con 0x7f8a8006dc50 2024-04-20T11:30:50.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.186+0000 7f8a88800640 1 -- 172.21.15.152:0/2080292799 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f8a6406caa0 msgr2=0x7f8a6406ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:50.187 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.186+0000 7f8a88800640 1 --2- 172.21.15.152:0/2080292799 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f8a6406caa0 0x7f8a6406ef60 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f8a74009990 tx=0x7f8a74008040 comp rx=0 tx=0).stop 2024-04-20T11:30:50.187 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.186+0000 7f8a88800640 1 -- 172.21.15.152:0/2080292799 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8a8006dc50 msgr2=0x7f8a80143740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:50.187 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.186+0000 7f8a88800640 1 --2- 172.21.15.152:0/2080292799 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8a8006dc50 0x7f8a80143740 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f8a7000e4e0 tx=0x7f8a7000e9a0 comp rx=0 tx=0).stop 2024-04-20T11:30:50.187 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.186+0000 7f8a88800640 1 -- 172.21.15.152:0/2080292799 shutdown_connections 2024-04-20T11:30:50.187 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.186+0000 7f8a88800640 1 --2- 172.21.15.152:0/2080292799 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f8a6406caa0 0x7f8a6406ef60 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:50.187 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.186+0000 7f8a88800640 1 --2- 172.21.15.152:0/2080292799 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8a800c8160 0x7f8a800c85e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:50.187 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.186+0000 7f8a88800640 1 --2- 172.21.15.152:0/2080292799 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8a8006dc50 0x7f8a80143740 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:50.187 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.186+0000 7f8a88800640 1 --2- 172.21.15.152:0/2080292799 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8a68005210 0x7f8a801f41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:50.187 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.186+0000 7f8a88800640 1 -- 172.21.15.152:0/2080292799 >> 172.21.15.152:0/2080292799 conn(0x7f8a8006b980 msgr2=0x7f8a8015ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:50.187 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.186+0000 7f8a88800640 1 -- 172.21.15.152:0/2080292799 shutdown_connections 2024-04-20T11:30:50.187 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.186+0000 7f8a88800640 1 -- 172.21.15.152:0/2080292799 wait complete. 2024-04-20T11:30:50.205 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration' 2024-04-20T11:30:50.399 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.399+0000 7fa63f200640 1 Processor -- start 2024-04-20T11:30:50.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.399+0000 7fa63f200640 1 -- start start 2024-04-20T11:30:50.403 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.401+0000 7fa63f200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fa63806ea20 0x7fa6380b4a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:50.403 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.401+0000 7fa63f200640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6380b5510 con 0x7fa638071cd0 2024-04-20T11:30:50.403 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.401+0000 7fa63f200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa63806df40 con 0x7fa6380b4f90 2024-04-20T11:30:50.403 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.401+0000 7fa63f200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fa6380ceec0 con 0x7fa63806ea20 2024-04-20T11:30:50.403 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.401+0000 7fa63de00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fa638071cd0 0x7fa6380720b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:57980/0 (socket says 172.21.15.152:57980) 2024-04-20T11:30:50.403 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.401+0000 7fa63de00640 1 -- 172.21.15.152:0/3979942161 learned_addr learned my addr 172.21.15.152:0/3979942161 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:30:50.403 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.401+0000 7fa63d400640 1 --2- 172.21.15.152:0/3979942161 >> v2:172.21.15.152:3300/0 conn(0x7fa63806ea20 0x7fa6380b4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:50.403 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.402+0000 7fa63ca00640 1 -- 172.21.15.152:0/3979942161 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 53265413 0 0) 0x7fa6380b5510 con 0x7fa638071cd0 2024-04-20T11:30:50.404 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.402+0000 7fa63ca00640 1 -- 172.21.15.152:0/3979942161 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa620003520 con 0x7fa638071cd0 2024-04-20T11:30:50.404 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.402+0000 7fa63d400640 1 -- 172.21.15.152:0/3979942161 >> v1:172.21.15.175:6789/0 conn(0x7fa6380b4f90 legacy=0x7fa638162e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:50.404 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.402+0000 7fa63d400640 1 -- 172.21.15.152:0/3979942161 >> v1:172.21.15.152:6789/0 conn(0x7fa638071cd0 legacy=0x7fa6380720b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:50.404 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.402+0000 7fa63d400640 1 -- 172.21.15.152:0/3979942161 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa63806e220 con 0x7fa63806ea20 2024-04-20T11:30:50.411 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.403+0000 7fa63d400640 1 --2- 172.21.15.152:0/3979942161 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa63806ea20 0x7fa6380b4a50 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fa6280097a0 tx=0x7fa62802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ba1e8ea56ffbe78 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:50.412 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.403+0000 7fa63ca00640 1 -- 172.21.15.152:0/3979942161 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6280315a0 con 0x7fa63806ea20 2024-04-20T11:30:50.412 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.403+0000 7fa63ca00640 1 -- 172.21.15.152:0/3979942161 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa628037070 con 0x7fa63806ea20 2024-04-20T11:30:50.412 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.403+0000 7fa63ca00640 1 -- 172.21.15.152:0/3979942161 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa628031960 con 0x7fa63806ea20 2024-04-20T11:30:50.412 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.404+0000 7fa63f200640 1 -- 172.21.15.152:0/3979942161 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa63806ea20 msgr2=0x7fa6380b4a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:50.412 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.404+0000 7fa63f200640 1 --2- 172.21.15.152:0/3979942161 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa63806ea20 0x7fa6380b4a50 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fa6280097a0 tx=0x7fa62802b840 comp rx=0 tx=0).stop 2024-04-20T11:30:50.412 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.404+0000 7fa63f200640 1 -- 172.21.15.152:0/3979942161 shutdown_connections 2024-04-20T11:30:50.412 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.404+0000 7fa63f200640 1 --2- 172.21.15.152:0/3979942161 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa63806ea20 0x7fa6380b4a50 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:50.412 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.404+0000 7fa63f200640 1 -- 172.21.15.152:0/3979942161 >> 172.21.15.152:0/3979942161 conn(0x7fa638067ae0 msgr2=0x7fa63806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:50.412 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.404+0000 7fa63f200640 1 -- 172.21.15.152:0/3979942161 shutdown_connections 2024-04-20T11:30:50.412 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.404+0000 7fa63f200640 1 -- 172.21.15.152:0/3979942161 wait complete. 2024-04-20T11:30:50.412 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.406+0000 7fa63f200640 1 Processor -- start 2024-04-20T11:30:50.412 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.406+0000 7fa63f200640 1 -- start start 2024-04-20T11:30:50.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.407+0000 7fa63f200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa63806ea20 0x7fa63815f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:50.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.407+0000 7fa63f200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa638071cd0 0x7fa6380b33f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:50.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.407+0000 7fa63f200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6380b4f90 0x7fa638145f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:50.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.407+0000 7fa63f200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fa6380b4180 con 0x7fa63806ea20 2024-04-20T11:30:50.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.407+0000 7fa63f200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa63806dcf0 con 0x7fa6380b4f90 2024-04-20T11:30:50.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.407+0000 7fa63f200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa638163560 con 0x7fa638071cd0 2024-04-20T11:30:50.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.407+0000 7fa63de00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa63806ea20 0x7fa63815f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:50.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.407+0000 7fa63de00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa63806ea20 0x7fa63815f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:55716/0 (socket says 172.21.15.152:55716) 2024-04-20T11:30:50.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.407+0000 7fa63de00640 1 -- 172.21.15.152:0/1220507863 learned_addr learned my addr 172.21.15.152:0/1220507863 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:30:50.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.408+0000 7fa63de00640 1 -- 172.21.15.152:0/1220507863 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa638071cd0 msgr2=0x7fa6380b33f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:50.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.408+0000 7fa63de00640 1 --2- 172.21.15.152:0/1220507863 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa638071cd0 0x7fa6380b33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:50.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.408+0000 7fa63de00640 1 -- 172.21.15.152:0/1220507863 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6380b4f90 msgr2=0x7fa638145f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:50.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.408+0000 7fa63e800640 1 --2- 172.21.15.152:0/1220507863 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6380b4f90 0x7fa638145f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:50.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.408+0000 7fa63de00640 1 --2- 172.21.15.152:0/1220507863 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6380b4f90 0x7fa638145f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:50.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.408+0000 7fa63de00640 1 -- 172.21.15.152:0/1220507863 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa638146c70 con 0x7fa63806ea20 2024-04-20T11:30:50.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.408+0000 7fa63de00640 1 --2- 172.21.15.152:0/1220507863 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa63806ea20 0x7fa63815f010 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fa62c00c460 tx=0x7fa62c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:50.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.408+0000 7fa632000640 1 -- 172.21.15.152:0/1220507863 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa62c017070 con 0x7fa63806ea20 2024-04-20T11:30:50.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.408+0000 7fa632000640 1 -- 172.21.15.152:0/1220507863 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa62c005ce0 con 0x7fa63806ea20 2024-04-20T11:30:50.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.408+0000 7fa63f200640 1 -- 172.21.15.152:0/1220507863 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa638146f10 con 0x7fa63806ea20 2024-04-20T11:30:50.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.408+0000 7fa632000640 1 -- 172.21.15.152:0/1220507863 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa62c007290 con 0x7fa63806ea20 2024-04-20T11:30:50.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.408+0000 7fa63f200640 1 -- 172.21.15.152:0/1220507863 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6381474d0 con 0x7fa63806ea20 2024-04-20T11:30:50.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.410+0000 7fa63f200640 1 -- 172.21.15.152:0/1220507863 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa638075e60 con 0x7fa63806ea20 2024-04-20T11:30:50.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.411+0000 7fa632000640 1 -- 172.21.15.152:0/1220507863 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 9) ==== 91193+0+0 (secure 0 0 0) 0x7fa62c01a070 con 0x7fa63806ea20 2024-04-20T11:30:50.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.411+0000 7fa632000640 1 --2- 172.21.15.152:0/1220507863 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fa61c06caa0 0x7fa61c06ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:50.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.412+0000 7fa632000640 1 -- 172.21.15.152:0/1220507863 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 4943+0+0 (secure 0 0 0) 0x7fa62c08af40 con 0x7fa63806ea20 2024-04-20T11:30:50.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.412+0000 7fa63d400640 1 --2- 172.21.15.152:0/1220507863 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fa61c06caa0 0x7fa61c06ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:50.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.412+0000 7fa63d400640 1 --2- 172.21.15.152:0/1220507863 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fa61c06caa0 0x7fa61c06ef60 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fa628036000 tx=0x7fa628030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:50.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.416+0000 7fa632000640 1 -- 172.21.15.152:0/1220507863 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fa62c0594a0 con 0x7fa63806ea20 2024-04-20T11:30:50.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.688+0000 7fa63f200640 1 -- 172.21.15.152:0/1220507863 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration"]} v 0) -- 0x7fa638075070 con 0x7fa63806ea20 2024-04-20T11:30:50.742 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.742+0000 7fa632000640 1 -- 172.21.15.152:0/1220507863 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration"]}]=0 v212) ==== 140+0+0 (secure 0 0 0) 0x7fa62c058da0 con 0x7fa63806ea20 2024-04-20T11:30:50.746 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.745+0000 7fa63f200640 1 -- 172.21.15.152:0/1220507863 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fa61c06caa0 msgr2=0x7fa61c06ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:50.746 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.745+0000 7fa63f200640 1 --2- 172.21.15.152:0/1220507863 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fa61c06caa0 0x7fa61c06ef60 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fa628036000 tx=0x7fa628030000 comp rx=0 tx=0).stop 2024-04-20T11:30:50.746 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.745+0000 7fa63f200640 1 -- 172.21.15.152:0/1220507863 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa63806ea20 msgr2=0x7fa63815f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:50.746 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.745+0000 7fa63f200640 1 --2- 172.21.15.152:0/1220507863 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa63806ea20 0x7fa63815f010 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fa62c00c460 tx=0x7fa62c00c920 comp rx=0 tx=0).stop 2024-04-20T11:30:50.746 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.745+0000 7fa63f200640 1 -- 172.21.15.152:0/1220507863 shutdown_connections 2024-04-20T11:30:50.746 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.745+0000 7fa63f200640 1 --2- 172.21.15.152:0/1220507863 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fa61c06caa0 0x7fa61c06ef60 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:50.746 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.745+0000 7fa63f200640 1 --2- 172.21.15.152:0/1220507863 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6380b4f90 0x7fa638145f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:50.746 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.745+0000 7fa63f200640 1 --2- 172.21.15.152:0/1220507863 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa638071cd0 0x7fa6380b33f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:50.746 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.745+0000 7fa63f200640 1 --2- 172.21.15.152:0/1220507863 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa63806ea20 0x7fa63815f010 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:50.746 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.745+0000 7fa63f200640 1 -- 172.21.15.152:0/1220507863 >> 172.21.15.152:0/1220507863 conn(0x7fa638067ae0 msgr2=0x7fa6380b0000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:50.746 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.745+0000 7fa63f200640 1 -- 172.21.15.152:0/1220507863 shutdown_connections 2024-04-20T11:30:50.746 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.745+0000 7fa63f200640 1 -- 172.21.15.152:0/1220507863 wait complete. 2024-04-20T11:30:50.767 INFO:tasks.cephfs_test_runner:test_connection_expiration (tasks.cephfs.test_volumes.TestMisc) ... ok 2024-04-20T11:30:50.767 INFO:tasks.cephfs_test_runner:Starting test: test_malicious_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc) Validate handcrafted .meta file on legacy subvol root doesn't break the system 2024-04-20T11:30:50.767 INFO:tasks.cephfs_test_runner:test_malicious_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc) 2024-04-20T11:30:50.767 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:30:50.971 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.971+0000 7fc6c7600640 1 Processor -- start 2024-04-20T11:30:50.972 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.971+0000 7fc6c7600640 1 -- start start 2024-04-20T11:30:50.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.972+0000 7fc6c7600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc6c00cd0d0 0x7fc6c00cd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:50.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.973+0000 7fc6c7600640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fc6c0071fe0 0x7fc6c0072470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:50.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.973+0000 7fc6c7600640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6c0146340 con 0x7fc6c0145f60 2024-04-20T11:30:50.974 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.973+0000 7fc6c7600640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fc6c006e9f0 con 0x7fc6c0071fe0 2024-04-20T11:30:50.974 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.973+0000 7fc6c7600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc6c0072d50 con 0x7fc6c00cd0d0 2024-04-20T11:30:50.974 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.973+0000 7fc6c4c00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fc6c0145f60 0x7fc6c006ee80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:57988/0 (socket says 172.21.15.152:57988) 2024-04-20T11:30:50.974 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.973+0000 7fc6c4c00640 1 -- 172.21.15.152:0/297988741 learned_addr learned my addr 172.21.15.152:0/297988741 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:30:50.974 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.973+0000 7fc6c5600640 1 --2- 172.21.15.152:0/297988741 >> v2:172.21.15.152:3300/0 conn(0x7fc6c0071fe0 0x7fc6c0072470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:50.975 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.973+0000 7fc6b7e00640 1 --2- 172.21.15.152:0/297988741 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc6c00cd0d0 0x7fc6c00cd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:50.975 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.973+0000 7fc6bfe00640 1 -- 172.21.15.152:0/297988741 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2077340497 0 0) 0x7fc6c0146340 con 0x7fc6c0145f60 2024-04-20T11:30:50.975 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.973+0000 7fc6bfe00640 1 -- 172.21.15.152:0/297988741 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc6a4003520 con 0x7fc6c0145f60 2024-04-20T11:30:50.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.974+0000 7fc6c5600640 1 -- 172.21.15.152:0/297988741 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc6c00cd0d0 msgr2=0x7fc6c00cd550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:50.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.974+0000 7fc6c5600640 1 --2- 172.21.15.152:0/297988741 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc6c00cd0d0 0x7fc6c00cd550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:50.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.974+0000 7fc6c5600640 1 -- 172.21.15.152:0/297988741 >> v1:172.21.15.152:6789/0 conn(0x7fc6c0145f60 legacy=0x7fc6c006ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:50.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.974+0000 7fc6c5600640 1 -- 172.21.15.152:0/297988741 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6c006e0c0 con 0x7fc6c0071fe0 2024-04-20T11:30:50.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.974+0000 7fc6b7e00640 1 --2- 172.21.15.152:0/297988741 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc6c00cd0d0 0x7fc6c00cd550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:30:50.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.975+0000 7fc6c5600640 1 --2- 172.21.15.152:0/297988741 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc6c0071fe0 0x7fc6c0072470 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fc6c006eb70 tx=0x7fc6b802be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1abd1a197f04f4f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:50.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.975+0000 7fc6bfe00640 1 -- 172.21.15.152:0/297988741 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6b8039070 con 0x7fc6c0071fe0 2024-04-20T11:30:50.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.975+0000 7fc6bfe00640 1 -- 172.21.15.152:0/297988741 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6b802cc40 con 0x7fc6c0071fe0 2024-04-20T11:30:50.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.975+0000 7fc6bfe00640 1 -- 172.21.15.152:0/297988741 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6b8034440 con 0x7fc6c0071fe0 2024-04-20T11:30:50.977 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.976+0000 7fc6c7600640 1 -- 172.21.15.152:0/297988741 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc6c0071fe0 msgr2=0x7fc6c0072470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:50.977 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.976+0000 7fc6c7600640 1 --2- 172.21.15.152:0/297988741 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc6c0071fe0 0x7fc6c0072470 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fc6c006eb70 tx=0x7fc6b802be90 comp rx=0 tx=0).stop 2024-04-20T11:30:50.977 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.976+0000 7fc6c7600640 1 -- 172.21.15.152:0/297988741 shutdown_connections 2024-04-20T11:30:50.977 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.976+0000 7fc6c7600640 1 --2- 172.21.15.152:0/297988741 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc6c00cd0d0 0x7fc6c00cd550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:50.977 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.976+0000 7fc6c7600640 1 --2- 172.21.15.152:0/297988741 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc6c0071fe0 0x7fc6c0072470 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:50.977 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.976+0000 7fc6c7600640 1 -- 172.21.15.152:0/297988741 >> 172.21.15.152:0/297988741 conn(0x7fc6c0067cf0 msgr2=0x7fc6c006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:50.977 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.976+0000 7fc6c7600640 1 -- 172.21.15.152:0/297988741 shutdown_connections 2024-04-20T11:30:50.977 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.976+0000 7fc6c7600640 1 -- 172.21.15.152:0/297988741 wait complete. 2024-04-20T11:30:50.978 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.977+0000 7fc6c7600640 1 Processor -- start 2024-04-20T11:30:50.978 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.978+0000 7fc6c7600640 1 -- start start 2024-04-20T11:30:50.979 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.978+0000 7fc6c7600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc6c0071fe0 0x7fc6c01f2010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:50.979 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.978+0000 7fc6c7600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc6c00cd0d0 0x7fc6c00ae680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:50.979 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.978+0000 7fc6c7600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc6c0145f60 0x7fc6c01f4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:50.979 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.978+0000 7fc6c7600640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fc6c00729b0 con 0x7fc6c00cd0d0 2024-04-20T11:30:50.979 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.978+0000 7fc6c7600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc6c006e680 con 0x7fc6c0145f60 2024-04-20T11:30:50.979 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.978+0000 7fc6c7600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc6c0072b30 con 0x7fc6c0071fe0 2024-04-20T11:30:50.979 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.979+0000 7fc6b7e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc6c00cd0d0 0x7fc6c00ae680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:50.980 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.979+0000 7fc6b7e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc6c00cd0d0 0x7fc6c00ae680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:55728/0 (socket says 172.21.15.152:55728) 2024-04-20T11:30:50.980 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.979+0000 7fc6b7e00640 1 -- 172.21.15.152:0/1928852607 learned_addr learned my addr 172.21.15.152:0/1928852607 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:30:50.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.979+0000 7fc6c4c00640 1 --2- 172.21.15.152:0/1928852607 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc6c0071fe0 0x7fc6c01f2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:50.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.979+0000 7fc6b7e00640 1 -- 172.21.15.152:0/1928852607 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc6c0071fe0 msgr2=0x7fc6c01f2010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:50.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.979+0000 7fc6c5600640 1 --2- 172.21.15.152:0/1928852607 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc6c0145f60 0x7fc6c01f4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:50.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.979+0000 7fc6b7e00640 1 --2- 172.21.15.152:0/1928852607 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc6c0071fe0 0x7fc6c01f2010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:50.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.979+0000 7fc6b7e00640 1 -- 172.21.15.152:0/1928852607 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc6c0145f60 msgr2=0x7fc6c01f4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:50.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.979+0000 7fc6b7e00640 1 --2- 172.21.15.152:0/1928852607 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc6c0145f60 0x7fc6c01f4f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:50.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.979+0000 7fc6b7e00640 1 -- 172.21.15.152:0/1928852607 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6c00aaa70 con 0x7fc6c00cd0d0 2024-04-20T11:30:50.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.980+0000 7fc6b7e00640 1 --2- 172.21.15.152:0/1928852607 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc6c00cd0d0 0x7fc6c00ae680 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fc6a800c820 tx=0x7fc6a800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:50.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.980+0000 7fc6bd600640 1 -- 172.21.15.152:0/1928852607 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6a8015070 con 0x7fc6c00cd0d0 2024-04-20T11:30:50.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.980+0000 7fc6bd600640 1 -- 172.21.15.152:0/1928852607 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6a8007a30 con 0x7fc6c00cd0d0 2024-04-20T11:30:50.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.980+0000 7fc6c7600640 1 -- 172.21.15.152:0/1928852607 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6c00aedf0 con 0x7fc6c00cd0d0 2024-04-20T11:30:50.982 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.980+0000 7fc6bd600640 1 -- 172.21.15.152:0/1928852607 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6a8007e00 con 0x7fc6c00cd0d0 2024-04-20T11:30:50.982 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.980+0000 7fc6c7600640 1 -- 172.21.15.152:0/1928852607 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6c00aafd0 con 0x7fc6c00cd0d0 2024-04-20T11:30:50.982 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.981+0000 7fc6c7600640 1 -- 172.21.15.152:0/1928852607 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6c00689f0 con 0x7fc6c00cd0d0 2024-04-20T11:30:50.982 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.982+0000 7fc6bd600640 1 -- 172.21.15.152:0/1928852607 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 9) ==== 91193+0+0 (secure 0 0 0) 0x7fc6a8018070 con 0x7fc6c00cd0d0 2024-04-20T11:30:50.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.983+0000 7fc6bd600640 1 --2- 172.21.15.152:0/1928852607 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc69c06cad0 0x7fc69c06ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:50.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.983+0000 7fc6bd600640 1 -- 172.21.15.152:0/1928852607 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 4943+0+0 (secure 0 0 0) 0x7fc6a808bc90 con 0x7fc6c00cd0d0 2024-04-20T11:30:50.984 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.983+0000 7fc6c4c00640 1 --2- 172.21.15.152:0/1928852607 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc69c06cad0 0x7fc69c06ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:50.984 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.984+0000 7fc6c4c00640 1 --2- 172.21.15.152:0/1928852607 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc69c06cad0 0x7fc69c06ef90 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fc6b0000f60 tx=0x7fc6b0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:50.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:50.988+0000 7fc6bd600640 1 -- 172.21.15.152:0/1928852607 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fc6c00aaa70 con 0x7fc6c00cd0d0 2024-04-20T11:30:51.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.185+0000 7fc6c7600640 1 -- 172.21.15.152:0/1928852607 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc6c006bde0 con 0x7fc6c00cd0d0 2024-04-20T11:30:51.187 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.186+0000 7fc6bd600640 1 -- 172.21.15.152:0/1928852607 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) ==== 74+0+13082 (secure 0 0 0) 0x7fc6a8059f90 con 0x7fc6c00cd0d0 2024-04-20T11:30:51.187 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:30:51.187 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":51,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:30:49.730696+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.175:6835/3704251089":"2024-04-21T11:30:46.710566+0000","172.21.15.175:6833/3704251089":"2024-04-21T11:30:46.710566+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:30:51.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.188+0000 7fc6c7600640 1 -- 172.21.15.152:0/1928852607 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc69c06cad0 msgr2=0x7fc69c06ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:51.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.188+0000 7fc6c7600640 1 --2- 172.21.15.152:0/1928852607 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc69c06cad0 0x7fc69c06ef90 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fc6b0000f60 tx=0x7fc6b0009040 comp rx=0 tx=0).stop 2024-04-20T11:30:51.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.189+0000 7fc6c7600640 1 -- 172.21.15.152:0/1928852607 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc6c00cd0d0 msgr2=0x7fc6c00ae680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:51.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.189+0000 7fc6c7600640 1 --2- 172.21.15.152:0/1928852607 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc6c00cd0d0 0x7fc6c00ae680 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fc6a800c820 tx=0x7fc6a800cc20 comp rx=0 tx=0).stop 2024-04-20T11:30:51.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.189+0000 7fc6c7600640 1 -- 172.21.15.152:0/1928852607 shutdown_connections 2024-04-20T11:30:51.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.189+0000 7fc6c7600640 1 --2- 172.21.15.152:0/1928852607 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc69c06cad0 0x7fc69c06ef90 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:51.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.189+0000 7fc6c7600640 1 --2- 172.21.15.152:0/1928852607 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc6c0145f60 0x7fc6c01f4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:51.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.189+0000 7fc6c7600640 1 --2- 172.21.15.152:0/1928852607 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc6c00cd0d0 0x7fc6c00ae680 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:51.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.189+0000 7fc6c7600640 1 --2- 172.21.15.152:0/1928852607 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc6c0071fe0 0x7fc6c01f2010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:51.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.189+0000 7fc6c7600640 1 -- 172.21.15.152:0/1928852607 >> 172.21.15.152:0/1928852607 conn(0x7fc6c0067cf0 msgr2=0x7fc6c015a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:51.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.189+0000 7fc6c7600640 1 -- 172.21.15.152:0/1928852607 shutdown_connections 2024-04-20T11:30:51.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.189+0000 7fc6c7600640 1 -- 172.21.15.152:0/1928852607 wait complete. 2024-04-20T11:30:51.206 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T11:26:47.246571+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T11:30:51.207 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T11:30:51.404 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.403+0000 7f7a30c00640 1 Processor -- start 2024-04-20T11:30:51.404 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.403+0000 7f7a30c00640 1 -- start start 2024-04-20T11:30:51.406 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.405+0000 7f7a30c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7a2c15b510 0x7f7a2c15fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:51.406 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.405+0000 7f7a30c00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f7a2c0cfc40 0x7f7a2c0b3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:51.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.405+0000 7f7a30c00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f7a2c0b2170 0x7f7a2c0b2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:51.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.405+0000 7f7a30c00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f7a2c06f420 con 0x7f7a2c0b2170 2024-04-20T11:30:51.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.405+0000 7f7a30c00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f7a2c0b4430 con 0x7f7a2c0cfc40 2024-04-20T11:30:51.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.405+0000 7f7a30c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7a2c15f1c0 con 0x7f7a2c15b510 2024-04-20T11:30:51.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.405+0000 7f7a2a800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f7a2c0b2170 0x7f7a2c0b2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:51.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.406+0000 7f7a2a800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f7a2c0b2170 0x7f7a2c0b2620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:55730/0 (socket says 172.21.15.152:55730) 2024-04-20T11:30:51.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.406+0000 7f7a2a800640 1 -- 172.21.15.152:0/2534322107 learned_addr learned my addr 172.21.15.152:0/2534322107 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:30:51.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.406+0000 7f7a29e00640 1 --2- 172.21.15.152:0/2534322107 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7a2c15b510 0x7f7a2c15fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:51.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.406+0000 7f7a29400640 1 --2- 172.21.15.152:0/2534322107 >> v2:172.21.15.175:3300/0 conn(0x7f7a2c0cfc40 0x7f7a2c0b3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:51.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.406+0000 7f7a2a800640 1 -- 172.21.15.152:0/2534322107 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7a2c15b510 msgr2=0x7f7a2c15fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:51.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.406+0000 7f7a2a800640 1 --2- 172.21.15.152:0/2534322107 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7a2c15b510 0x7f7a2c15fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:51.408 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.406+0000 7f7a2a800640 1 -- 172.21.15.152:0/2534322107 >> v2:172.21.15.175:3300/0 conn(0x7f7a2c0cfc40 msgr2=0x7f7a2c0b3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:51.408 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.406+0000 7f7a2a800640 1 --2- 172.21.15.152:0/2534322107 >> v2:172.21.15.175:3300/0 conn(0x7f7a2c0cfc40 0x7f7a2c0b3ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:51.408 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.406+0000 7f7a2a800640 1 -- 172.21.15.152:0/2534322107 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a2c06f670 con 0x7f7a2c0b2170 2024-04-20T11:30:51.409 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.407+0000 7f7a2a800640 1 --2- 172.21.15.152:0/2534322107 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7a2c0b2170 0x7f7a2c0b2620 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c072fd0 tx=0x7f7a2402e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be61aba583141ed0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:51.409 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.407+0000 7f7a28a00640 1 -- 172.21.15.152:0/2534322107 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a2402ec80 con 0x7f7a2c0b2170 2024-04-20T11:30:51.409 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.407+0000 7f7a28a00640 1 -- 172.21.15.152:0/2534322107 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a2402ee20 con 0x7f7a2c0b2170 2024-04-20T11:30:51.409 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.407+0000 7f7a28a00640 1 -- 172.21.15.152:0/2534322107 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a24036610 con 0x7f7a2c0b2170 2024-04-20T11:30:51.410 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.408+0000 7f7a30c00640 1 -- 172.21.15.152:0/2534322107 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7a2c0b2170 msgr2=0x7f7a2c0b2620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:51.410 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.408+0000 7f7a30c00640 1 --2- 172.21.15.152:0/2534322107 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7a2c0b2170 0x7f7a2c0b2620 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c072fd0 tx=0x7f7a2402e0e0 comp rx=0 tx=0).stop 2024-04-20T11:30:51.410 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.408+0000 7f7a30c00640 1 -- 172.21.15.152:0/2534322107 shutdown_connections 2024-04-20T11:30:51.410 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.408+0000 7f7a30c00640 1 --2- 172.21.15.152:0/2534322107 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7a2c0b2170 0x7f7a2c0b2620 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:51.410 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.408+0000 7f7a30c00640 1 --2- 172.21.15.152:0/2534322107 >> v2:172.21.15.175:3300/0 conn(0x7f7a2c0cfc40 0x7f7a2c0b3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:51.410 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.408+0000 7f7a30c00640 1 --2- 172.21.15.152:0/2534322107 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7a2c15b510 0x7f7a2c15fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:51.410 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.408+0000 7f7a30c00640 1 -- 172.21.15.152:0/2534322107 >> 172.21.15.152:0/2534322107 conn(0x7f7a2c06b980 msgr2=0x7f7a2c06bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:51.410 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.408+0000 7f7a30c00640 1 -- 172.21.15.152:0/2534322107 shutdown_connections 2024-04-20T11:30:51.410 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.408+0000 7f7a30c00640 1 -- 172.21.15.152:0/2534322107 wait complete. 2024-04-20T11:30:51.411 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.410+0000 7f7a30c00640 1 Processor -- start 2024-04-20T11:30:51.411 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.410+0000 7f7a30c00640 1 -- start start 2024-04-20T11:30:51.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.411+0000 7f7a30c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7a2c0b2170 0x7f7a2c0ae9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:51.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.411+0000 7f7a30c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7a2c0cfc40 0x7f7a2c0b0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:51.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.411+0000 7f7a30c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a2c15b510 0x7f7a2c0b0e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:51.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.411+0000 7f7a30c00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f7a2c15b8f0 con 0x7f7a2c0cfc40 2024-04-20T11:30:51.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.411+0000 7f7a30c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7a2c15f710 con 0x7f7a2c15b510 2024-04-20T11:30:51.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.411+0000 7f7a30c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7a2c06f950 con 0x7f7a2c0b2170 2024-04-20T11:30:51.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.411+0000 7f7a29400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7a2c0cfc40 0x7f7a2c0b0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:51.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.411+0000 7f7a29400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7a2c0cfc40 0x7f7a2c0b0930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:55738/0 (socket says 172.21.15.152:55738) 2024-04-20T11:30:51.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.411+0000 7f7a29400640 1 -- 172.21.15.152:0/3590804275 learned_addr learned my addr 172.21.15.152:0/3590804275 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:30:51.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.412+0000 7f7a29e00640 1 --2- 172.21.15.152:0/3590804275 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7a2c0b2170 0x7f7a2c0ae9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:51.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.412+0000 7f7a29400640 1 -- 172.21.15.152:0/3590804275 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7a2c0b2170 msgr2=0x7f7a2c0ae9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:51.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.412+0000 7f7a29400640 1 --2- 172.21.15.152:0/3590804275 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7a2c0b2170 0x7f7a2c0ae9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:51.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.412+0000 7f7a29400640 1 -- 172.21.15.152:0/3590804275 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a2c15b510 msgr2=0x7f7a2c0b0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:51.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.412+0000 7f7a2a800640 1 --2- 172.21.15.152:0/3590804275 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a2c15b510 0x7f7a2c0b0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:51.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.412+0000 7f7a29400640 1 --2- 172.21.15.152:0/3590804275 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a2c15b510 0x7f7a2c0b0e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:51.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.412+0000 7f7a29400640 1 -- 172.21.15.152:0/3590804275 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a2c15bbf0 con 0x7f7a2c0cfc40 2024-04-20T11:30:51.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.412+0000 7f7a29e00640 1 --2- 172.21.15.152:0/3590804275 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7a2c0b2170 0x7f7a2c0ae9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:30:51.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.412+0000 7f7a29400640 1 --2- 172.21.15.152:0/3590804275 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7a2c0cfc40 0x7f7a2c0b0930 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f7a1800d800 tx=0x7f7a1800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:51.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.412+0000 7f7a22000640 1 -- 172.21.15.152:0/3590804275 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a18018020 con 0x7f7a2c0cfc40 2024-04-20T11:30:51.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.413+0000 7f7a22000640 1 -- 172.21.15.152:0/3590804275 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a180096a0 con 0x7f7a2c0cfc40 2024-04-20T11:30:51.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.413+0000 7f7a22000640 1 -- 172.21.15.152:0/3590804275 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a18009a70 con 0x7f7a2c0cfc40 2024-04-20T11:30:51.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.413+0000 7f7a30c00640 1 -- 172.21.15.152:0/3590804275 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a2c15bed0 con 0x7f7a2c0cfc40 2024-04-20T11:30:51.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.413+0000 7f7a30c00640 1 -- 172.21.15.152:0/3590804275 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a2c15c490 con 0x7f7a2c0cfc40 2024-04-20T11:30:51.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.414+0000 7f7a30c00640 1 -- 172.21.15.152:0/3590804275 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a2c0689f0 con 0x7f7a2c0cfc40 2024-04-20T11:30:51.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.415+0000 7f7a22000640 1 -- 172.21.15.152:0/3590804275 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 9) ==== 91193+0+0 (secure 0 0 0) 0x7f7a1801b020 con 0x7f7a2c0cfc40 2024-04-20T11:30:51.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.416+0000 7f7a22000640 1 --2- 172.21.15.152:0/3590804275 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f7a0c06caa0 0x7f7a0c06ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:51.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.416+0000 7f7a22000640 1 -- 172.21.15.152:0/3590804275 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 4943+0+0 (secure 0 0 0) 0x7f7a1808beb0 con 0x7f7a2c0cfc40 2024-04-20T11:30:51.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.416+0000 7f7a29e00640 1 --2- 172.21.15.152:0/3590804275 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f7a0c06caa0 0x7f7a0c06ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:51.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.417+0000 7f7a29e00640 1 --2- 172.21.15.152:0/3590804275 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f7a0c06caa0 0x7f7a0c06ef60 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c006fd0 tx=0x7f7a1c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:51.420 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.420+0000 7f7a22000640 1 -- 172.21.15.152:0/3590804275 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f7a18022030 con 0x7f7a2c0cfc40 2024-04-20T11:30:51.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.618+0000 7f7a30c00640 1 -- 172.21.15.152:0/3590804275 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7a2c0af130 con 0x7f7a2c0cfc40 2024-04-20T11:30:51.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.619+0000 7f7a22000640 1 -- 172.21.15.152:0/3590804275 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v51) ==== 93+0+10 (secure 0 0 0) 0x7f7a1805a2c0 con 0x7f7a2c0cfc40 2024-04-20T11:30:51.620 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 1 2024-04-20T11:30:51.623 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.622+0000 7f7a30c00640 1 -- 172.21.15.152:0/3590804275 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f7a0c06caa0 msgr2=0x7f7a0c06ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:51.623 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.622+0000 7f7a30c00640 1 --2- 172.21.15.152:0/3590804275 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f7a0c06caa0 0x7f7a0c06ef60 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c006fd0 tx=0x7f7a1c008040 comp rx=0 tx=0).stop 2024-04-20T11:30:51.623 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.622+0000 7f7a30c00640 1 -- 172.21.15.152:0/3590804275 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7a2c0cfc40 msgr2=0x7f7a2c0b0930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:51.623 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.622+0000 7f7a30c00640 1 --2- 172.21.15.152:0/3590804275 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7a2c0cfc40 0x7f7a2c0b0930 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f7a1800d800 tx=0x7f7a1800dcc0 comp rx=0 tx=0).stop 2024-04-20T11:30:51.623 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.622+0000 7f7a30c00640 1 -- 172.21.15.152:0/3590804275 shutdown_connections 2024-04-20T11:30:51.623 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.622+0000 7f7a30c00640 1 --2- 172.21.15.152:0/3590804275 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f7a0c06caa0 0x7f7a0c06ef60 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:51.623 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.622+0000 7f7a30c00640 1 --2- 172.21.15.152:0/3590804275 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a2c15b510 0x7f7a2c0b0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:51.623 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.622+0000 7f7a30c00640 1 --2- 172.21.15.152:0/3590804275 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7a2c0cfc40 0x7f7a2c0b0930 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:51.623 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.622+0000 7f7a30c00640 1 --2- 172.21.15.152:0/3590804275 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7a2c0b2170 0x7f7a2c0ae9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:51.623 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.622+0000 7f7a30c00640 1 -- 172.21.15.152:0/3590804275 >> 172.21.15.152:0/3590804275 conn(0x7f7a2c06b980 msgr2=0x7f7a2c071db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:51.623 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.622+0000 7f7a30c00640 1 -- 172.21.15.152:0/3590804275 shutdown_connections 2024-04-20T11:30:51.623 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.622+0000 7f7a30c00640 1 -- 172.21.15.152:0/3590804275 wait complete. 2024-04-20T11:30:51.642 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_malicious_metafile_on_legacy_to_v1_upgrade' 2024-04-20T11:30:51.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.831+0000 7fd96b200640 1 Processor -- start 2024-04-20T11:30:51.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.832+0000 7fd96b200640 1 -- start start 2024-04-20T11:30:51.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.833+0000 7fd96b200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fd96c15d1e0 0x7fd96c160b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:51.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.833+0000 7fd96b200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd96c071cd0 0x7fd96c072160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:51.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.833+0000 7fd96b200640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd96c0ce8c0 con 0x7fd96c15d5c0 2024-04-20T11:30:51.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.833+0000 7fd96b200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fd96c0726a0 con 0x7fd96c15d1e0 2024-04-20T11:30:51.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.833+0000 7fd96b200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd96c1600b0 con 0x7fd96c071cd0 2024-04-20T11:30:51.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.833+0000 7fd963e00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fd96c15d5c0 0x7fd96c0cdfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:57992/0 (socket says 172.21.15.152:57992) 2024-04-20T11:30:51.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.833+0000 7fd963e00640 1 -- 172.21.15.152:0/3897500813 learned_addr learned my addr 172.21.15.152:0/3897500813 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:30:51.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.834+0000 7fd969e00640 1 --2- 172.21.15.152:0/3897500813 >> v2:172.21.15.175:3300/0 conn(0x7fd96c15d1e0 0x7fd96c160b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:51.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.834+0000 7fd96a800640 1 --2- 172.21.15.152:0/3897500813 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd96c071cd0 0x7fd96c072160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:51.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.834+0000 7fd969400640 1 -- 172.21.15.152:0/3897500813 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1422442803 0 0) 0x7fd96c0ce8c0 con 0x7fd96c15d5c0 2024-04-20T11:30:51.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.834+0000 7fd969400640 1 -- 172.21.15.152:0/3897500813 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd950003520 con 0x7fd96c15d5c0 2024-04-20T11:30:51.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.834+0000 7fd969400640 1 -- 172.21.15.152:0/3897500813 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 35625515 0 0) 0x7fd950003520 con 0x7fd96c15d5c0 2024-04-20T11:30:51.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.834+0000 7fd969400640 1 -- 172.21.15.152:0/3897500813 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd96c071cd0 msgr2=0x7fd96c072160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:51.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.834+0000 7fd969400640 1 --2- 172.21.15.152:0/3897500813 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd96c071cd0 0x7fd96c072160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:51.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.834+0000 7fd969400640 1 -- 172.21.15.152:0/3897500813 >> v2:172.21.15.175:3300/0 conn(0x7fd96c15d1e0 msgr2=0x7fd96c160b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:51.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.834+0000 7fd969400640 1 --2- 172.21.15.152:0/3897500813 >> v2:172.21.15.175:3300/0 conn(0x7fd96c15d1e0 0x7fd96c160b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:51.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.834+0000 7fd969400640 1 -- 172.21.15.152:0/3897500813 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd96c160390 con 0x7fd96c15d5c0 2024-04-20T11:30:51.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.834+0000 7fd969400640 1 -- 172.21.15.152:0/3897500813 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7fd954001670 con 0x7fd96c15d5c0 2024-04-20T11:30:51.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.834+0000 7fd969400640 1 -- 172.21.15.152:0/3897500813 >> v1:172.21.15.152:6789/0 conn(0x7fd96c15d5c0 legacy=0x7fd96c0cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:51.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.834+0000 7fd969400640 1 --2- 172.21.15.152:0/3897500813 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd9500039b0 0x7fd950003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:51.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.834+0000 7fd969400640 1 --2- 172.21.15.152:0/3897500813 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd950004800 0x7fd950004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:51.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.834+0000 7fd969400640 1 --2- 172.21.15.152:0/3897500813 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd96c071cd0 0x7fd9500051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:51.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.834+0000 7fd969400640 1 -- 172.21.15.152:0/3897500813 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fd96c0726a0 con 0x7fd950004800 2024-04-20T11:30:51.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.834+0000 7fd969400640 1 -- 172.21.15.152:0/3897500813 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd96c1600b0 con 0x7fd9500039b0 2024-04-20T11:30:51.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.834+0000 7fd969400640 1 -- 172.21.15.152:0/3897500813 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd95000b9c0 con 0x7fd96c071cd0 2024-04-20T11:30:51.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.835+0000 7fd963e00640 1 --2- 172.21.15.152:0/3897500813 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd950004800 0x7fd950004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:51.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.835+0000 7fd96a800640 1 --2- 172.21.15.152:0/3897500813 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd96c071cd0 0x7fd9500051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:51.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.835+0000 7fd969e00640 1 --2- 172.21.15.152:0/3897500813 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd9500039b0 0x7fd950003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:51.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.835+0000 7fd963e00640 1 -- 172.21.15.152:0/3897500813 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd96c071cd0 msgr2=0x7fd9500051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:51.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.835+0000 7fd963e00640 1 --2- 172.21.15.152:0/3897500813 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd96c071cd0 0x7fd9500051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:51.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.835+0000 7fd963e00640 1 -- 172.21.15.152:0/3897500813 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd9500039b0 msgr2=0x7fd950003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:51.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.835+0000 7fd963e00640 1 --2- 172.21.15.152:0/3897500813 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd9500039b0 0x7fd950003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:51.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.835+0000 7fd963e00640 1 -- 172.21.15.152:0/3897500813 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd950003520 con 0x7fd950004800 2024-04-20T11:30:51.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.836+0000 7fd963e00640 1 --2- 172.21.15.152:0/3897500813 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd950004800 0x7fd950004c80 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fd95400c110 tx=0x7fd950005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c83c3daeb517645 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:51.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.836+0000 7fd969400640 1 -- 172.21.15.152:0/3897500813 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd954007c40 con 0x7fd950004800 2024-04-20T11:30:51.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.837+0000 7fd969400640 1 -- 172.21.15.152:0/3897500813 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd954030070 con 0x7fd950004800 2024-04-20T11:30:51.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.837+0000 7fd969400640 1 -- 172.21.15.152:0/3897500813 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd954005d30 con 0x7fd950004800 2024-04-20T11:30:51.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.837+0000 7fd96b200640 1 -- 172.21.15.152:0/3897500813 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd950004800 msgr2=0x7fd950004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:51.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.837+0000 7fd96b200640 1 --2- 172.21.15.152:0/3897500813 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd950004800 0x7fd950004c80 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fd95400c110 tx=0x7fd950005830 comp rx=0 tx=0).stop 2024-04-20T11:30:51.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.837+0000 7fd96a800640 1 -- 172.21.15.152:0/3897500813 reap_dead start 2024-04-20T11:30:51.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.837+0000 7fd96b200640 1 -- 172.21.15.152:0/3897500813 shutdown_connections 2024-04-20T11:30:51.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.837+0000 7fd96b200640 1 -- 172.21.15.152:0/3897500813 >> 172.21.15.152:0/3897500813 conn(0x7fd96c067ae0 msgr2=0x7fd96c06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:51.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.837+0000 7fd96b200640 1 -- 172.21.15.152:0/3897500813 shutdown_connections 2024-04-20T11:30:51.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.837+0000 7fd96b200640 1 -- 172.21.15.152:0/3897500813 wait complete. 2024-04-20T11:30:51.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.839+0000 7fd96b200640 1 Processor -- start 2024-04-20T11:30:51.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.839+0000 7fd96b200640 1 -- start start 2024-04-20T11:30:51.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.840+0000 7fd96b200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd950004800 0x7fd96c14cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:51.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.840+0000 7fd96b200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd96c1590f0 0x7fd96c159570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:51.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.840+0000 7fd96b200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd96c0b3b80 0x7fd96c0b4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:51.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.840+0000 7fd96b200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fd96c072160 con 0x7fd950004800 2024-04-20T11:30:51.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.840+0000 7fd96b200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd96c0ceba0 con 0x7fd96c1590f0 2024-04-20T11:30:51.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.840+0000 7fd96b200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd96c0722e0 con 0x7fd96c0b3b80 2024-04-20T11:30:51.843 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.840+0000 7fd969e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd950004800 0x7fd96c14cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:51.843 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.840+0000 7fd969e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd950004800 0x7fd96c14cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:55750/0 (socket says 172.21.15.152:55750) 2024-04-20T11:30:51.843 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.840+0000 7fd969e00640 1 -- 172.21.15.152:0/3258648330 learned_addr learned my addr 172.21.15.152:0/3258648330 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:30:51.843 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.841+0000 7fd969e00640 1 -- 172.21.15.152:0/3258648330 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd96c0b3b80 msgr2=0x7fd96c0b4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:51.843 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.841+0000 7fd969e00640 1 --2- 172.21.15.152:0/3258648330 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd96c0b3b80 0x7fd96c0b4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:51.843 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.841+0000 7fd969e00640 1 -- 172.21.15.152:0/3258648330 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd96c1590f0 msgr2=0x7fd96c159570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:51.843 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.841+0000 7fd969e00640 1 --2- 172.21.15.152:0/3258648330 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd96c1590f0 0x7fd96c159570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:51.843 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.841+0000 7fd969e00640 1 -- 172.21.15.152:0/3258648330 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd96c14d1d0 con 0x7fd950004800 2024-04-20T11:30:51.843 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.841+0000 7fd969e00640 1 --2- 172.21.15.152:0/3258648330 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd950004800 0x7fd96c14cc90 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fd95c010900 tx=0x7fd95c010dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:51.843 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.841+0000 7fd962000640 1 -- 172.21.15.152:0/3258648330 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd95c00f040 con 0x7fd950004800 2024-04-20T11:30:51.843 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.841+0000 7fd962000640 1 -- 172.21.15.152:0/3258648330 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd95c00eba0 con 0x7fd950004800 2024-04-20T11:30:51.843 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.841+0000 7fd962000640 1 -- 172.21.15.152:0/3258648330 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd95c01c400 con 0x7fd950004800 2024-04-20T11:30:51.843 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.841+0000 7fd96b200640 1 -- 172.21.15.152:0/3258648330 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd96c159b70 con 0x7fd950004800 2024-04-20T11:30:51.843 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.841+0000 7fd96b200640 1 -- 172.21.15.152:0/3258648330 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd96c0adce0 con 0x7fd950004800 2024-04-20T11:30:51.844 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.843+0000 7fd96b200640 1 -- 172.21.15.152:0/3258648330 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd96c075e60 con 0x7fd950004800 2024-04-20T11:30:51.844 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.844+0000 7fd962000640 1 -- 172.21.15.152:0/3258648330 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 9) ==== 91193+0+0 (secure 0 0 0) 0x7fd95c01c5a0 con 0x7fd950004800 2024-04-20T11:30:51.845 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.844+0000 7fd962000640 1 --2- 172.21.15.152:0/3258648330 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fd94806caa0 0x7fd94806ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:51.845 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.845+0000 7fd962000640 1 -- 172.21.15.152:0/3258648330 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 4943+0+0 (secure 0 0 0) 0x7fd95c08c2a0 con 0x7fd950004800 2024-04-20T11:30:51.845 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.845+0000 7fd963e00640 1 --2- 172.21.15.152:0/3258648330 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fd94806caa0 0x7fd94806ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:51.846 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.845+0000 7fd963e00640 1 --2- 172.21.15.152:0/3258648330 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fd94806caa0 0x7fd94806ef60 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fd954034000 tx=0x7fd954039040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:51.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:51.850+0000 7fd962000640 1 -- 172.21.15.152:0/3258648330 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fd96c14d1d0 con 0x7fd950004800 2024-04-20T11:30:52.032 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:30:52.038 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:30:52.125 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:52.125+0000 7fd96b200640 1 -- 172.21.15.152:0/3258648330 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_malicious_metafile_on_legacy_to_v1_upgrade"]} v 0) -- 0x7fd96c072c90 con 0x7fd950004800 2024-04-20T11:30:52.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:52.754+0000 7fd962000640 1 -- 172.21.15.152:0/3258648330 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_malicious_metafile_on_legacy_to_v1_upgrade"]}]=0 v214) ==== 164+0+0 (secure 0 0 0) 0x7fd95c018030 con 0x7fd950004800 2024-04-20T11:30:52.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:52.758+0000 7fd96b200640 1 -- 172.21.15.152:0/3258648330 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fd94806caa0 msgr2=0x7fd94806ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:52.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:52.758+0000 7fd96b200640 1 --2- 172.21.15.152:0/3258648330 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fd94806caa0 0x7fd94806ef60 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fd954034000 tx=0x7fd954039040 comp rx=0 tx=0).stop 2024-04-20T11:30:52.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:52.758+0000 7fd96b200640 1 -- 172.21.15.152:0/3258648330 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd950004800 msgr2=0x7fd96c14cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:52.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:52.758+0000 7fd96b200640 1 --2- 172.21.15.152:0/3258648330 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd950004800 0x7fd96c14cc90 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fd95c010900 tx=0x7fd95c010dc0 comp rx=0 tx=0).stop 2024-04-20T11:30:52.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:52.758+0000 7fd96b200640 1 -- 172.21.15.152:0/3258648330 shutdown_connections 2024-04-20T11:30:52.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:52.758+0000 7fd96b200640 1 --2- 172.21.15.152:0/3258648330 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fd94806caa0 0x7fd94806ef60 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:52.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:52.758+0000 7fd96b200640 1 --2- 172.21.15.152:0/3258648330 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd96c0b3b80 0x7fd96c0b4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:52.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:52.758+0000 7fd96b200640 1 --2- 172.21.15.152:0/3258648330 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd96c1590f0 0x7fd96c159570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:52.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:52.758+0000 7fd96b200640 1 --2- 172.21.15.152:0/3258648330 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd950004800 0x7fd96c14cc90 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:52.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:52.758+0000 7fd96b200640 1 -- 172.21.15.152:0/3258648330 >> 172.21.15.152:0/3258648330 conn(0x7fd96c067ae0 msgr2=0x7fd96c06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:52.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:52.758+0000 7fd96b200640 1 -- 172.21.15.152:0/3258648330 shutdown_connections 2024-04-20T11:30:52.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:52.758+0000 7fd96b200640 1 -- 172.21.15.152:0/3258648330 wait complete. 2024-04-20T11:30:52.780 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-20T11:30:52.985 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:52.984+0000 7fdb4e200640 1 Processor -- start 2024-04-20T11:30:52.985 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:52.984+0000 7fdb4e200640 1 -- start start 2024-04-20T11:30:52.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:52.986+0000 7fdb4e200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fdb4806f1c0 0x7fdb4806f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:52.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:52.986+0000 7fdb4e200640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb4815f7e0 con 0x7fdb481480b0 2024-04-20T11:30:52.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:52.986+0000 7fdb4e200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb4815fac0 con 0x7fdb48147cd0 2024-04-20T11:30:52.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:52.986+0000 7fdb4e200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fdb4806fb70 con 0x7fdb4806f1c0 2024-04-20T11:30:52.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:52.986+0000 7fdb47e00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fdb481480b0 0x7fdb4815ecb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:57998/0 (socket says 172.21.15.152:57998) 2024-04-20T11:30:52.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:52.986+0000 7fdb47e00640 1 -- 172.21.15.152:0/3397424975 learned_addr learned my addr 172.21.15.152:0/3397424975 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:30:52.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:52.987+0000 7fdb47400640 1 -- 172.21.15.152:0/3397424975 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4220578452 0 0) 0x7fdb4815f7e0 con 0x7fdb481480b0 2024-04-20T11:30:52.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:52.987+0000 7fdb47400640 1 -- 172.21.15.152:0/3397424975 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdb30003520 con 0x7fdb481480b0 2024-04-20T11:30:52.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:52.987+0000 7fdb4d800640 1 --2- 172.21.15.152:0/3397424975 >> v2:172.21.15.175:3300/0 conn(0x7fdb4806f1c0 0x7fdb4806f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:52.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:52.987+0000 7fdb47400640 1 -- 172.21.15.152:0/3397424975 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3891221304 0 0) 0x7fdb30003520 con 0x7fdb481480b0 2024-04-20T11:30:52.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:52.987+0000 7fdb47400640 1 -- 172.21.15.152:0/3397424975 >> v2:172.21.15.175:3300/0 conn(0x7fdb4806f1c0 msgr2=0x7fdb4806f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:52.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:52.987+0000 7fdb47400640 1 --2- 172.21.15.152:0/3397424975 >> v2:172.21.15.175:3300/0 conn(0x7fdb4806f1c0 0x7fdb4806f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:52.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:52.987+0000 7fdb47400640 1 -- 172.21.15.152:0/3397424975 >> v1:172.21.15.175:6789/0 conn(0x7fdb48147cd0 legacy=0x7fdb4806e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:52.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:52.987+0000 7fdb47400640 1 -- 172.21.15.152:0/3397424975 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb4806df00 con 0x7fdb481480b0 2024-04-20T11:30:52.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:52.987+0000 7fdb47400640 1 -- 172.21.15.152:0/3397424975 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7fdb34001670 con 0x7fdb481480b0 2024-04-20T11:30:52.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:52.987+0000 7fdb47400640 1 -- 172.21.15.152:0/3397424975 >> v1:172.21.15.152:6789/0 conn(0x7fdb481480b0 legacy=0x7fdb4815ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:52.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:52.988+0000 7fdb47400640 1 --2- 172.21.15.152:0/3397424975 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb300039b0 0x7fdb30003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:52.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:52.988+0000 7fdb47400640 1 --2- 172.21.15.152:0/3397424975 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdb30004800 0x7fdb30004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:52.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:52.988+0000 7fdb47400640 1 --2- 172.21.15.152:0/3397424975 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdb300051c0 0x7fdb3000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:52.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:52.988+0000 7fdb47400640 1 -- 172.21.15.152:0/3397424975 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fdb4806fb70 con 0x7fdb300051c0 2024-04-20T11:30:52.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:52.988+0000 7fdb47400640 1 -- 172.21.15.152:0/3397424975 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdb3000c110 con 0x7fdb300039b0 2024-04-20T11:30:52.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:52.988+0000 7fdb47400640 1 -- 172.21.15.152:0/3397424975 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fdb3000c310 con 0x7fdb30004800 2024-04-20T11:30:52.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:52.988+0000 7fdb4d800640 1 --2- 172.21.15.152:0/3397424975 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdb300051c0 0x7fdb3000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:52.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:52.988+0000 7fdb4ce00640 1 --2- 172.21.15.152:0/3397424975 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb300039b0 0x7fdb30003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:52.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:52.988+0000 7fdb47e00640 1 --2- 172.21.15.152:0/3397424975 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdb30004800 0x7fdb30004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:52.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:52.988+0000 7fdb4d800640 1 -- 172.21.15.152:0/3397424975 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdb30004800 msgr2=0x7fdb30004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:52.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:52.988+0000 7fdb4d800640 1 --2- 172.21.15.152:0/3397424975 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdb30004800 0x7fdb30004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:52.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:52.989+0000 7fdb4d800640 1 -- 172.21.15.152:0/3397424975 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb300039b0 msgr2=0x7fdb30003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:52.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:52.989+0000 7fdb4d800640 1 --2- 172.21.15.152:0/3397424975 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb300039b0 0x7fdb30003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:52.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:52.989+0000 7fdb4d800640 1 -- 172.21.15.152:0/3397424975 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb30003520 con 0x7fdb300051c0 2024-04-20T11:30:52.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:52.989+0000 7fdb47e00640 1 --2- 172.21.15.152:0/3397424975 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdb30004800 0x7fdb30004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:30:52.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:52.989+0000 7fdb4d800640 1 -- 172.21.15.152:0/3397424975 reap_dead start 2024-04-20T11:30:52.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:52.990+0000 7fdb4d800640 1 --2- 172.21.15.152:0/3397424975 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdb300051c0 0x7fdb3000b9c0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fdb40007360 tx=0x7fdb30005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16d2cdadf82fd75c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:52.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:52.990+0000 7fdb47400640 1 -- 172.21.15.152:0/3397424975 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb40009be0 con 0x7fdb300051c0 2024-04-20T11:30:52.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:52.990+0000 7fdb47400640 1 -- 172.21.15.152:0/3397424975 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb4002cd30 con 0x7fdb300051c0 2024-04-20T11:30:52.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:52.990+0000 7fdb47400640 1 -- 172.21.15.152:0/3397424975 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb40003bd0 con 0x7fdb300051c0 2024-04-20T11:30:52.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:52.990+0000 7fdb4e200640 1 -- 172.21.15.152:0/3397424975 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdb300051c0 msgr2=0x7fdb3000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:52.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:52.990+0000 7fdb4e200640 1 --2- 172.21.15.152:0/3397424975 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdb300051c0 0x7fdb3000b9c0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fdb40007360 tx=0x7fdb30005830 comp rx=0 tx=0).stop 2024-04-20T11:30:52.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:52.991+0000 7fdb4e200640 1 -- 172.21.15.152:0/3397424975 shutdown_connections 2024-04-20T11:30:52.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:52.991+0000 7fdb4e200640 1 --2- 172.21.15.152:0/3397424975 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdb300051c0 0x7fdb3000b9c0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:52.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:52.991+0000 7fdb4e200640 1 -- 172.21.15.152:0/3397424975 >> 172.21.15.152:0/3397424975 conn(0x7fdb48067ae0 msgr2=0x7fdb4806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:52.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:52.991+0000 7fdb4e200640 1 -- 172.21.15.152:0/3397424975 shutdown_connections 2024-04-20T11:30:52.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:52.991+0000 7fdb4e200640 1 -- 172.21.15.152:0/3397424975 wait complete. 2024-04-20T11:30:52.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:52.992+0000 7fdb4e200640 1 Processor -- start 2024-04-20T11:30:52.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:52.992+0000 7fdb4e200640 1 -- start start 2024-04-20T11:30:52.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:52.993+0000 7fdb4e200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb300051c0 0x7fdb48142560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:52.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:52.993+0000 7fdb4e200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdb48145c50 0x7fdb481460d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:52.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:52.993+0000 7fdb4e200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdb481e5b00 0x7fdb481e5f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:52.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:52.993+0000 7fdb4e200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fdb4806f630 con 0x7fdb48145c50 2024-04-20T11:30:52.995 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:52.993+0000 7fdb4e200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdb48071ed0 con 0x7fdb300051c0 2024-04-20T11:30:52.995 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:52.993+0000 7fdb4e200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fdb4806f7b0 con 0x7fdb481e5b00 2024-04-20T11:30:52.995 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:52.994+0000 7fdb47e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdb48145c50 0x7fdb481460d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:52.995 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:52.994+0000 7fdb47e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdb48145c50 0x7fdb481460d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:55770/0 (socket says 172.21.15.152:55770) 2024-04-20T11:30:52.995 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:52.994+0000 7fdb47e00640 1 -- 172.21.15.152:0/3294772231 learned_addr learned my addr 172.21.15.152:0/3294772231 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:30:52.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:52.994+0000 7fdb4ce00640 1 --2- 172.21.15.152:0/3294772231 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb300051c0 0x7fdb48142560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:52.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:52.994+0000 7fdb4d800640 1 --2- 172.21.15.152:0/3294772231 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdb481e5b00 0x7fdb481e5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:52.997 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:52.994+0000 7fdb47e00640 1 -- 172.21.15.152:0/3294772231 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdb481e5b00 msgr2=0x7fdb481e5f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:52.997 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:52.994+0000 7fdb47e00640 1 --2- 172.21.15.152:0/3294772231 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdb481e5b00 0x7fdb481e5f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:52.997 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:52.994+0000 7fdb47e00640 1 -- 172.21.15.152:0/3294772231 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb300051c0 msgr2=0x7fdb48142560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:52.997 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:52.994+0000 7fdb47e00640 1 --2- 172.21.15.152:0/3294772231 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb300051c0 0x7fdb48142560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:52.997 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:52.994+0000 7fdb47e00640 1 -- 172.21.15.152:0/3294772231 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb48144440 con 0x7fdb48145c50 2024-04-20T11:30:52.997 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:52.994+0000 7fdb47e00640 1 --2- 172.21.15.152:0/3294772231 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdb48145c50 0x7fdb481460d0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fdb3400d800 tx=0x7fdb3400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:52.997 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:52.995+0000 7fdb44c00640 1 -- 172.21.15.152:0/3294772231 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb34019070 con 0x7fdb48145c50 2024-04-20T11:30:52.997 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:52.995+0000 7fdb44c00640 1 -- 172.21.15.152:0/3294772231 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb340074e0 con 0x7fdb48145c50 2024-04-20T11:30:52.997 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:52.995+0000 7fdb4e200640 1 -- 172.21.15.152:0/3294772231 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb48144720 con 0x7fdb48145c50 2024-04-20T11:30:52.997 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:52.995+0000 7fdb44c00640 1 -- 172.21.15.152:0/3294772231 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb340049e0 con 0x7fdb48145c50 2024-04-20T11:30:52.997 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:52.995+0000 7fdb4e200640 1 -- 172.21.15.152:0/3294772231 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb481f4750 con 0x7fdb48145c50 2024-04-20T11:30:52.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:52.996+0000 7fdb4e200640 1 -- 172.21.15.152:0/3294772231 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb48075e60 con 0x7fdb48145c50 2024-04-20T11:30:52.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:52.997+0000 7fdb44c00640 1 -- 172.21.15.152:0/3294772231 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 9) ==== 91193+0+0 (secure 0 0 0) 0x7fdb3401c070 con 0x7fdb48145c50 2024-04-20T11:30:52.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:52.998+0000 7fdb44c00640 1 --2- 172.21.15.152:0/3294772231 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fdb2806cad0 0x7fdb2806ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:52.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:52.998+0000 7fdb44c00640 1 -- 172.21.15.152:0/3294772231 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 4943+0+0 (secure 0 0 0) 0x7fdb3408b740 con 0x7fdb48145c50 2024-04-20T11:30:52.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:52.998+0000 7fdb4ce00640 1 --2- 172.21.15.152:0/3294772231 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fdb2806cad0 0x7fdb2806ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:52.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:52.999+0000 7fdb4ce00640 1 --2- 172.21.15.152:0/3294772231 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fdb2806cad0 0x7fdb2806ef90 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c007fd0 tx=0x7fdb3c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:53.003 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:53.002+0000 7fdb44c00640 1 -- 172.21.15.152:0/3294772231 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fdb34059a40 con 0x7fdb48145c50 2024-04-20T11:30:53.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:53.186+0000 7fdb4e200640 1 -- 172.21.15.152:0/3294772231 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fdb480ccf10 con 0x7fdb48145c50 2024-04-20T11:30:53.193 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:53.192+0000 7fdb44c00640 1 -- 172.21.15.152:0/3294772231 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) ==== 122+0+0 (secure 0 0 0) 0x7fdb34059340 con 0x7fdb48145c50 2024-04-20T11:30:53.195 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:53.195+0000 7fdb4e200640 1 -- 172.21.15.152:0/3294772231 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fdb2806cad0 msgr2=0x7fdb2806ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:53.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:53.195+0000 7fdb4e200640 1 --2- 172.21.15.152:0/3294772231 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fdb2806cad0 0x7fdb2806ef90 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c007fd0 tx=0x7fdb3c009040 comp rx=0 tx=0).stop 2024-04-20T11:30:53.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:53.195+0000 7fdb4e200640 1 -- 172.21.15.152:0/3294772231 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdb48145c50 msgr2=0x7fdb481460d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:53.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:53.195+0000 7fdb4e200640 1 --2- 172.21.15.152:0/3294772231 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdb48145c50 0x7fdb481460d0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fdb3400d800 tx=0x7fdb3400dcc0 comp rx=0 tx=0).stop 2024-04-20T11:30:53.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:53.195+0000 7fdb4e200640 1 -- 172.21.15.152:0/3294772231 shutdown_connections 2024-04-20T11:30:53.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:53.195+0000 7fdb4e200640 1 --2- 172.21.15.152:0/3294772231 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fdb2806cad0 0x7fdb2806ef90 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:53.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:53.195+0000 7fdb4e200640 1 --2- 172.21.15.152:0/3294772231 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdb481e5b00 0x7fdb481e5f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:53.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:53.195+0000 7fdb4e200640 1 --2- 172.21.15.152:0/3294772231 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdb48145c50 0x7fdb481460d0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:53.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:53.195+0000 7fdb4e200640 1 --2- 172.21.15.152:0/3294772231 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb300051c0 0x7fdb48142560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:53.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:53.195+0000 7fdb4e200640 1 -- 172.21.15.152:0/3294772231 >> 172.21.15.152:0/3294772231 conn(0x7fdb48067ae0 msgr2=0x7fdb4815e350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:53.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:53.195+0000 7fdb4e200640 1 -- 172.21.15.152:0/3294772231 shutdown_connections 2024-04-20T11:30:53.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:53.195+0000 7fdb4e200640 1 -- 172.21.15.152:0/3294772231 wait complete. 2024-04-20T11:30:53.212 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-20T11:30:53.212 DEBUG:teuthology.orchestra.run.smithi152:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-20T11:30:53.301 DEBUG:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-20T11:30:53.416 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-20T11:30:53.417 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-20T11:30:53.417 DEBUG:teuthology.orchestra.run.smithi152:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:30:53.472 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-20T11:30:53.472 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T11:30:53.473 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:30:53.530 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-20T11:30:53.531 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-20T11:30:53.531 DEBUG:teuthology.orchestra.run.smithi152:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:30:53.588 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-20T11:30:53.588 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T11:30:53.588 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:30:53.644 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-20T11:30:53.645 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-20T11:30:53.645 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:30:53.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:53.854+0000 7fdf03200640 1 Processor -- start 2024-04-20T11:30:53.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:53.854+0000 7fdf03200640 1 -- start start 2024-04-20T11:30:53.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:53.856+0000 7fdf03200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdefc145f60 0x7fdefc06ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:53.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:53.856+0000 7fdf03200640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdefc146340 con 0x7fdefc0cd0d0 2024-04-20T11:30:53.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:53.856+0000 7fdf03200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdefc06f3c0 con 0x7fdefc0726f0 2024-04-20T11:30:53.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:53.856+0000 7fdf03200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fdefc06e9f0 con 0x7fdefc145f60 2024-04-20T11:30:53.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:53.856+0000 7fdefb400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fdefc0cd0d0 0x7fdefc071fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:58006/0 (socket says 172.21.15.152:58006) 2024-04-20T11:30:53.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:53.856+0000 7fdefb400640 1 -- 172.21.15.152:0/2815257006 learned_addr learned my addr 172.21.15.152:0/2815257006 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:30:53.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:53.856+0000 7fdefaa00640 1 -- 172.21.15.152:0/2815257006 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2194265988 0 0) 0x7fdefc146340 con 0x7fdefc0cd0d0 2024-04-20T11:30:53.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:53.856+0000 7fdefbe00640 1 --2- 172.21.15.152:0/2815257006 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdefc145f60 0x7fdefc06ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:53.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:53.856+0000 7fdefaa00640 1 -- 172.21.15.152:0/2815257006 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdee0003520 con 0x7fdefc0cd0d0 2024-04-20T11:30:53.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:53.857+0000 7fdefaa00640 1 -- 172.21.15.152:0/2815257006 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3634178741 0 0) 0x7fdee0003520 con 0x7fdefc0cd0d0 2024-04-20T11:30:53.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:53.857+0000 7fdefaa00640 1 -- 172.21.15.152:0/2815257006 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdefc145f60 msgr2=0x7fdefc06ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:53.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:53.857+0000 7fdefaa00640 1 --2- 172.21.15.152:0/2815257006 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdefc145f60 0x7fdefc06ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:53.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:53.857+0000 7fdefaa00640 1 -- 172.21.15.152:0/2815257006 >> v1:172.21.15.175:6789/0 conn(0x7fdefc0726f0 legacy=0x7fdefc0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:53.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:53.857+0000 7fdefaa00640 1 -- 172.21.15.152:0/2815257006 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdefc072d50 con 0x7fdefc0cd0d0 2024-04-20T11:30:53.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:53.857+0000 7fdefaa00640 1 -- 172.21.15.152:0/2815257006 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7fdee8001670 con 0x7fdefc0cd0d0 2024-04-20T11:30:53.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:53.857+0000 7fdefbe00640 1 --2- 172.21.15.152:0/2815257006 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdefc145f60 0x7fdefc06ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:30:53.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:53.857+0000 7fdefaa00640 1 -- 172.21.15.152:0/2815257006 >> v1:172.21.15.152:6789/0 conn(0x7fdefc0cd0d0 legacy=0x7fdefc071fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:53.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:53.857+0000 7fdefaa00640 1 --2- 172.21.15.152:0/2815257006 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdefc145f60 0x7fdee0003a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:53.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:53.857+0000 7fdefaa00640 1 --2- 172.21.15.152:0/2815257006 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdee0004040 0x7fdee0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:53.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:53.857+0000 7fdefaa00640 1 --2- 172.21.15.152:0/2815257006 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdee0004d40 0x7fdee00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:53.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:53.857+0000 7fdefaa00640 1 -- 172.21.15.152:0/2815257006 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fdefc06e9f0 con 0x7fdee0004040 2024-04-20T11:30:53.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:53.857+0000 7fdefaa00640 1 -- 172.21.15.152:0/2815257006 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdee000bbb0 con 0x7fdee0004d40 2024-04-20T11:30:53.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:53.857+0000 7fdefaa00640 1 -- 172.21.15.152:0/2815257006 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fdee000bdb0 con 0x7fdefc145f60 2024-04-20T11:30:53.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:53.857+0000 7fdefb400640 1 --2- 172.21.15.152:0/2815257006 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdee0004040 0x7fdee0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:53.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:53.858+0000 7fdefbe00640 1 --2- 172.21.15.152:0/2815257006 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdefc145f60 0x7fdee0003a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:53.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:53.858+0000 7fdefb400640 1 -- 172.21.15.152:0/2815257006 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdefc145f60 msgr2=0x7fdee0003a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:53.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:53.858+0000 7fdefb400640 1 --2- 172.21.15.152:0/2815257006 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdefc145f60 0x7fdee0003a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:53.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:53.858+0000 7fdefb400640 1 -- 172.21.15.152:0/2815257006 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdee0004d40 msgr2=0x7fdee00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:53.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:53.858+0000 7fdefb400640 1 --2- 172.21.15.152:0/2815257006 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdee0004d40 0x7fdee00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:53.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:53.858+0000 7fdefb400640 1 -- 172.21.15.152:0/2815257006 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdee0003520 con 0x7fdee0004040 2024-04-20T11:30:53.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:53.859+0000 7fdefb400640 1 --2- 172.21.15.152:0/2815257006 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdee0004040 0x7fdee0004800 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fdee800c240 tx=0x7fdee0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4de9e5a2b61fdefd server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:53.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:53.859+0000 7fdefaa00640 1 -- 172.21.15.152:0/2815257006 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdee8007cf0 con 0x7fdee0004040 2024-04-20T11:30:53.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:53.859+0000 7fdefaa00640 1 -- 172.21.15.152:0/2815257006 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdee8030070 con 0x7fdee0004040 2024-04-20T11:30:53.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:53.859+0000 7fdefaa00640 1 -- 172.21.15.152:0/2815257006 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdee8005d70 con 0x7fdee0004040 2024-04-20T11:30:53.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:53.859+0000 7fdf03200640 1 -- 172.21.15.152:0/2815257006 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdee0004040 msgr2=0x7fdee0004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:53.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:53.859+0000 7fdf03200640 1 --2- 172.21.15.152:0/2815257006 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdee0004040 0x7fdee0004800 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fdee800c240 tx=0x7fdee0005830 comp rx=0 tx=0).stop 2024-04-20T11:30:53.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:53.859+0000 7fdf01200640 1 -- 172.21.15.152:0/2815257006 reap_dead start 2024-04-20T11:30:53.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:53.860+0000 7fdf03200640 1 -- 172.21.15.152:0/2815257006 shutdown_connections 2024-04-20T11:30:53.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:53.860+0000 7fdf03200640 1 -- 172.21.15.152:0/2815257006 >> 172.21.15.152:0/2815257006 conn(0x7fdefc067cf0 msgr2=0x7fdefc06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:53.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:53.860+0000 7fdf03200640 1 -- 172.21.15.152:0/2815257006 shutdown_connections 2024-04-20T11:30:53.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:53.860+0000 7fdf03200640 1 -- 172.21.15.152:0/2815257006 wait complete. 2024-04-20T11:30:53.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:53.861+0000 7fdf03200640 1 Processor -- start 2024-04-20T11:30:53.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:53.861+0000 7fdf03200640 1 -- start start 2024-04-20T11:30:53.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:53.862+0000 7fdf03200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdee0004040 0x7fdefc1e6c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:53.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:53.862+0000 7fdf03200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdefc13f710 0x7fdefc13e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:53.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:53.862+0000 7fdf03200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdefc140f00 0x7fdefc15f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:53.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:53.862+0000 7fdf03200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fdefc06e680 con 0x7fdefc13f710 2024-04-20T11:30:53.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:53.862+0000 7fdf03200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdefc06e3a0 con 0x7fdefc140f00 2024-04-20T11:30:53.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:53.862+0000 7fdf03200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fdefc06ee80 con 0x7fdee0004040 2024-04-20T11:30:53.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:53.862+0000 7fdefb400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdefc13f710 0x7fdefc13e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:53.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:53.863+0000 7fdefb400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdefc13f710 0x7fdefc13e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:55788/0 (socket says 172.21.15.152:55788) 2024-04-20T11:30:53.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:53.863+0000 7fdefb400640 1 -- 172.21.15.152:0/1699126296 learned_addr learned my addr 172.21.15.152:0/1699126296 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:30:53.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:53.863+0000 7fdefbe00640 1 --2- 172.21.15.152:0/1699126296 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdee0004040 0x7fdefc1e6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:53.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:53.863+0000 7fdefb400640 1 -- 172.21.15.152:0/1699126296 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdee0004040 msgr2=0x7fdefc1e6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:53.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:53.863+0000 7fdf01200640 1 --2- 172.21.15.152:0/1699126296 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdefc140f00 0x7fdefc15f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:53.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:53.863+0000 7fdefb400640 1 --2- 172.21.15.152:0/1699126296 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdee0004040 0x7fdefc1e6c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:53.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:53.863+0000 7fdefb400640 1 -- 172.21.15.152:0/1699126296 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdefc140f00 msgr2=0x7fdefc15f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:53.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:53.863+0000 7fdefb400640 1 --2- 172.21.15.152:0/1699126296 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdefc140f00 0x7fdefc15f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:53.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:53.863+0000 7fdefb400640 1 -- 172.21.15.152:0/1699126296 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdefc13fb90 con 0x7fdefc13f710 2024-04-20T11:30:53.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:53.863+0000 7fdefb400640 1 --2- 172.21.15.152:0/1699126296 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdefc13f710 0x7fdefc13e750 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fdee800c360 tx=0x7fdefc06e960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:53.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:53.864+0000 7fdef3e00640 1 -- 172.21.15.152:0/1699126296 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdee8007cf0 con 0x7fdefc13f710 2024-04-20T11:30:53.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:53.864+0000 7fdef3e00640 1 -- 172.21.15.152:0/1699126296 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdee8004bf0 con 0x7fdefc13f710 2024-04-20T11:30:53.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:53.864+0000 7fdf03200640 1 -- 172.21.15.152:0/1699126296 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdefc13fe60 con 0x7fdefc13f710 2024-04-20T11:30:53.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:53.864+0000 7fdef3e00640 1 -- 172.21.15.152:0/1699126296 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdee8001c30 con 0x7fdefc13f710 2024-04-20T11:30:53.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:53.864+0000 7fdf03200640 1 -- 172.21.15.152:0/1699126296 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdefc140420 con 0x7fdefc13f710 2024-04-20T11:30:53.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:53.865+0000 7fdf03200640 1 -- 172.21.15.152:0/1699126296 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdefc0682e0 con 0x7fdefc13f710 2024-04-20T11:30:53.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:53.866+0000 7fdef3e00640 1 -- 172.21.15.152:0/1699126296 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 9) ==== 91193+0+0 (secure 0 0 0) 0x7fdee8047020 con 0x7fdefc13f710 2024-04-20T11:30:53.867 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:53.867+0000 7fdef3e00640 1 --2- 172.21.15.152:0/1699126296 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fdedc06caa0 0x7fdedc06ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:53.867 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:53.867+0000 7fdef3e00640 1 -- 172.21.15.152:0/1699126296 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 4943+0+0 (secure 0 0 0) 0x7fdee80acd00 con 0x7fdefc13f710 2024-04-20T11:30:53.867 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:53.867+0000 7fdefbe00640 1 --2- 172.21.15.152:0/1699126296 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fdedc06caa0 0x7fdedc06ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:53.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:53.868+0000 7fdefbe00640 1 --2- 172.21.15.152:0/1699126296 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fdedc06caa0 0x7fdedc06ef60 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fdeec006fd0 tx=0x7fdeec008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:53.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:53.871+0000 7fdef3e00640 1 -- 172.21.15.152:0/1699126296 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fdefc0682e0 con 0x7fdefc13f710 2024-04-20T11:30:54.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:54.101+0000 7fdf03200640 1 -- 172.21.15.152:0/1699126296 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdefc06f950 con 0x7fdefc13f710 2024-04-20T11:30:54.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:54.102+0000 7fdef3e00640 1 -- 172.21.15.152:0/1699126296 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+3878 (secure 0 0 0) 0x7fdee807b1b0 con 0x7fdefc13f710 2024-04-20T11:30:54.103 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:30:54.103 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":27,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4826,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6834/2855025590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2855025590},{"type":"v1","addr":"172.21.15.175:6834","nonce":2855025590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5299,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1354923132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6833","nonce":1354923132},{"type":"v1","addr":"172.21.15.175:6835","nonce":1354923132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":5709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6836/3887716755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":3887716755},{"type":"v1","addr":"172.21.15.152:6836","nonce":3887716755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5712,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/698387399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6835","nonce":698387399},{"type":"v1","addr":"172.21.15.152:6837","nonce":698387399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[]} 2024-04-20T11:30:54.104 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 27 2024-04-20T11:30:54.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:54.105+0000 7fdf03200640 1 -- 172.21.15.152:0/1699126296 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fdedc06caa0 msgr2=0x7fdedc06ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:54.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:54.105+0000 7fdf03200640 1 --2- 172.21.15.152:0/1699126296 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fdedc06caa0 0x7fdedc06ef60 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fdeec006fd0 tx=0x7fdeec008040 comp rx=0 tx=0).stop 2024-04-20T11:30:54.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:54.105+0000 7fdf03200640 1 -- 172.21.15.152:0/1699126296 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdefc13f710 msgr2=0x7fdefc13e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:54.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:54.105+0000 7fdf03200640 1 --2- 172.21.15.152:0/1699126296 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdefc13f710 0x7fdefc13e750 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fdee800c360 tx=0x7fdefc06e960 comp rx=0 tx=0).stop 2024-04-20T11:30:54.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:54.105+0000 7fdf03200640 1 -- 172.21.15.152:0/1699126296 shutdown_connections 2024-04-20T11:30:54.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:54.105+0000 7fdf03200640 1 --2- 172.21.15.152:0/1699126296 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fdedc06caa0 0x7fdedc06ef60 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:54.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:54.105+0000 7fdf03200640 1 --2- 172.21.15.152:0/1699126296 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdefc140f00 0x7fdefc15f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:54.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:54.105+0000 7fdf03200640 1 --2- 172.21.15.152:0/1699126296 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdefc13f710 0x7fdefc13e750 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:54.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:54.105+0000 7fdf03200640 1 --2- 172.21.15.152:0/1699126296 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdee0004040 0x7fdefc1e6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:54.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:54.105+0000 7fdf03200640 1 -- 172.21.15.152:0/1699126296 >> 172.21.15.152:0/1699126296 conn(0x7fdefc067cf0 msgr2=0x7fdefc15a840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:54.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:54.105+0000 7fdf03200640 1 -- 172.21.15.152:0/1699126296 shutdown_connections 2024-04-20T11:30:54.107 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:54.105+0000 7fdf03200640 1 -- 172.21.15.152:0/1699126296 wait complete. 2024-04-20T11:30:54.122 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-20T11:30:54.122 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-20T11:30:54.122 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-20T11:30:54.123 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T11:30:54.123 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-20T11:30:54.123 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-20T11:30:54.123 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-20T11:30:54.123 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T11:30:54.123 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-20T11:30:54.123 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-20T11:30:54.124 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-20T11:30:54.124 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T11:30:54.124 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-20T11:30:54.124 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-20T11:30:54.124 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-20T11:30:54.124 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T11:30:54.140 INFO:tasks.ceph.mds.a:Stopped 2024-04-20T11:30:54.141 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-20T11:30:54.142 INFO:tasks.ceph.mds.c:Stopped 2024-04-20T11:30:54.142 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-20T11:30:54.144 INFO:tasks.ceph.mds.b:Stopped 2024-04-20T11:30:54.144 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-20T11:30:54.146 INFO:tasks.ceph.mds.d:Stopped 2024-04-20T11:30:54.146 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-20T11:30:54.188 INFO:tasks.ceph.mds.a:Started 2024-04-20T11:30:54.189 DEBUG:teuthology.parallel:result is None 2024-04-20T11:30:54.191 INFO:tasks.ceph.mds.b:Started 2024-04-20T11:30:54.191 DEBUG:teuthology.parallel:result is None 2024-04-20T11:30:54.195 INFO:tasks.ceph.mds.c:Started 2024-04-20T11:30:54.195 DEBUG:teuthology.parallel:result is None 2024-04-20T11:30:54.197 INFO:tasks.ceph.mds.d:Started 2024-04-20T11:30:54.198 DEBUG:teuthology.parallel:result is None 2024-04-20T11:30:54.198 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-20T11:30:54.334 INFO:tasks.ceph.mds.a.smithi152.stdout:starting mds.a at 2024-04-20T11:30:54.334 INFO:tasks.ceph.mds.c.smithi152.stdout:starting mds.c at 2024-04-20T11:30:54.346 INFO:tasks.ceph.mds.b.smithi175.stdout:starting mds.b at 2024-04-20T11:30:54.348 INFO:tasks.ceph.mds.d.smithi175.stdout:starting mds.d at 2024-04-20T11:30:54.370 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:54.370+0000 7fbe75600640 1 Processor -- start 2024-04-20T11:30:54.371 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:54.370+0000 7fbe75600640 1 -- start start 2024-04-20T11:30:54.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:54.371+0000 7fbe75600640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fbe70161d30 0x7fbe700d0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:54.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:54.371+0000 7fbe75600640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe700d26a0 con 0x7fbe700d2b80 2024-04-20T11:30:54.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:54.371+0000 7fbe75600640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe700d53c0 con 0x7fbe700d4ec0 2024-04-20T11:30:54.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:54.371+0000 7fbe75600640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fbe700d0540 con 0x7fbe70161d30 2024-04-20T11:30:54.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:54.371+0000 7fbe6fe00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fbe70161d30 0x7fbe700d0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:54.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:54.371+0000 7fbe74c00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fbe700d2b80 0x7fbe7006c830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:58034/0 (socket says 172.21.15.152:58034) 2024-04-20T11:30:54.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:54.371+0000 7fbe74c00640 1 -- 172.21.15.152:0/3345705074 learned_addr learned my addr 172.21.15.152:0/3345705074 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:30:54.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:54.372+0000 7fbe6fe00640 1 -- 172.21.15.152:0/3345705074 >> v1:172.21.15.175:6789/0 conn(0x7fbe700d4ec0 legacy=0x7fbe700d15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:54.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:54.372+0000 7fbe6fe00640 1 -- 172.21.15.152:0/3345705074 >> v1:172.21.15.152:6789/0 conn(0x7fbe700d2b80 legacy=0x7fbe7006c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:54.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:54.372+0000 7fbe6fe00640 1 -- 172.21.15.152:0/3345705074 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe700cf520 con 0x7fbe70161d30 2024-04-20T11:30:54.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:54.373+0000 7fbe6fe00640 1 --2- 172.21.15.152:0/3345705074 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbe70161d30 0x7fbe700d0000 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fbe600097a0 tx=0x7fbe6002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d592e70568b203a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:54.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:54.373+0000 7fbe6ea00640 1 -- 172.21.15.152:0/3345705074 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe60034070 con 0x7fbe70161d30 2024-04-20T11:30:54.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:54.373+0000 7fbe6ea00640 1 -- 172.21.15.152:0/3345705074 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe60030610 con 0x7fbe70161d30 2024-04-20T11:30:54.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:54.373+0000 7fbe6ea00640 1 -- 172.21.15.152:0/3345705074 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe600309e0 con 0x7fbe70161d30 2024-04-20T11:30:54.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:54.374+0000 7fbe75600640 1 -- 172.21.15.152:0/3345705074 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbe70161d30 msgr2=0x7fbe700d0000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:54.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:54.374+0000 7fbe75600640 1 --2- 172.21.15.152:0/3345705074 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbe70161d30 0x7fbe700d0000 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fbe600097a0 tx=0x7fbe6002b960 comp rx=0 tx=0).stop 2024-04-20T11:30:54.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:54.374+0000 7fbe75600640 1 -- 172.21.15.152:0/3345705074 shutdown_connections 2024-04-20T11:30:54.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:54.374+0000 7fbe75600640 1 --2- 172.21.15.152:0/3345705074 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbe70161d30 0x7fbe700d0000 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:54.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:54.374+0000 7fbe75600640 1 -- 172.21.15.152:0/3345705074 >> 172.21.15.152:0/3345705074 conn(0x7fbe7008bfd0 msgr2=0x7fbe700d45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:54.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:54.374+0000 7fbe75600640 1 -- 172.21.15.152:0/3345705074 shutdown_connections 2024-04-20T11:30:54.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:54.374+0000 7fbe75600640 1 -- 172.21.15.152:0/3345705074 wait complete. 2024-04-20T11:30:54.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:54.375+0000 7fbe75600640 1 Processor -- start 2024-04-20T11:30:54.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:54.376+0000 7fbe75600640 1 -- start start 2024-04-20T11:30:54.377 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:54.376+0000 7fbe75600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbe700d2b80 0x7fbe700ce6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:54.378 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:54.376+0000 7fbe75600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbe700d4ec0 0x7fbe7015a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:54.378 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:54.377+0000 7fbe75600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbe70161d30 0x7fbe700ccec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:54.378 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:54.377+0000 7fbe75600640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fbe700cf1b0 con 0x7fbe70161d30 2024-04-20T11:30:54.378 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:54.377+0000 7fbe75600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbe700cfb70 con 0x7fbe700d4ec0 2024-04-20T11:30:54.378 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:54.377+0000 7fbe75600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fbe70162110 con 0x7fbe700d2b80 2024-04-20T11:30:54.378 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:54.377+0000 7fbe74c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbe70161d30 0x7fbe700ccec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:54.378 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:54.377+0000 7fbe74c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbe70161d30 0x7fbe700ccec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:55840/0 (socket says 172.21.15.152:55840) 2024-04-20T11:30:54.378 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:54.377+0000 7fbe74c00640 1 -- 172.21.15.152:0/306041170 learned_addr learned my addr 172.21.15.152:0/306041170 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:30:54.378 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:54.377+0000 7fbe6fe00640 1 --2- 172.21.15.152:0/306041170 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbe700d2b80 0x7fbe700ce6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:54.378 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:54.377+0000 7fbe6f400640 1 --2- 172.21.15.152:0/306041170 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbe700d4ec0 0x7fbe7015a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:54.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:54.377+0000 7fbe74c00640 1 -- 172.21.15.152:0/306041170 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbe700d2b80 msgr2=0x7fbe700ce6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:54.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:54.377+0000 7fbe74c00640 1 --2- 172.21.15.152:0/306041170 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbe700d2b80 0x7fbe700ce6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:54.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:54.377+0000 7fbe74c00640 1 -- 172.21.15.152:0/306041170 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbe700d4ec0 msgr2=0x7fbe7015a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:54.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:54.377+0000 7fbe74c00640 1 --2- 172.21.15.152:0/306041170 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbe700d4ec0 0x7fbe7015a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:54.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:54.378+0000 7fbe74c00640 1 -- 172.21.15.152:0/306041170 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe700cd610 con 0x7fbe70161d30 2024-04-20T11:30:54.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:54.378+0000 7fbe74c00640 1 --2- 172.21.15.152:0/306041170 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbe70161d30 0x7fbe700ccec0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fbe6800ca80 tx=0x7fbe6800cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:54.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:54.378+0000 7fbe67e00640 1 -- 172.21.15.152:0/306041170 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe68017070 con 0x7fbe70161d30 2024-04-20T11:30:54.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:54.378+0000 7fbe67e00640 1 -- 172.21.15.152:0/306041170 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe6800ac30 con 0x7fbe70161d30 2024-04-20T11:30:54.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:54.378+0000 7fbe67e00640 1 -- 172.21.15.152:0/306041170 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe68012600 con 0x7fbe70161d30 2024-04-20T11:30:54.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:54.379+0000 7fbe75600640 1 -- 172.21.15.152:0/306041170 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe700cd8f0 con 0x7fbe70161d30 2024-04-20T11:30:54.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:54.379+0000 7fbe75600640 1 -- 172.21.15.152:0/306041170 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe700cdeb0 con 0x7fbe70161d30 2024-04-20T11:30:54.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:54.380+0000 7fbe75600640 1 -- 172.21.15.152:0/306041170 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe70075e60 con 0x7fbe70161d30 2024-04-20T11:30:54.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:54.381+0000 7fbe67e00640 1 -- 172.21.15.152:0/306041170 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 9) ==== 91193+0+0 (secure 0 0 0) 0x7fbe6801a070 con 0x7fbe70161d30 2024-04-20T11:30:54.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:54.382+0000 7fbe67e00640 1 --2- 172.21.15.152:0/306041170 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fbe5006caa0 0x7fbe5006ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:54.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:54.382+0000 7fbe67e00640 1 -- 172.21.15.152:0/306041170 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 4943+0+0 (secure 0 0 0) 0x7fbe6808b7c0 con 0x7fbe70161d30 2024-04-20T11:30:54.383 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:54.382+0000 7fbe6fe00640 1 --2- 172.21.15.152:0/306041170 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fbe5006caa0 0x7fbe5006ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:54.383 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:54.383+0000 7fbe6fe00640 1 --2- 172.21.15.152:0/306041170 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fbe5006caa0 0x7fbe5006ef60 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fbe60036000 tx=0x7fbe6003b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:54.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:54.387+0000 7fbe67e00640 1 -- 172.21.15.152:0/306041170 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fbe6800ae20 con 0x7fbe70161d30 2024-04-20T11:30:54.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:54.596+0000 7fbe75600640 1 -- 172.21.15.152:0/306041170 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fbe700d1d70 con 0x7fbe70161d30 2024-04-20T11:30:55.798 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:55.797+0000 7fbe67e00640 1 -- 172.21.15.152:0/306041170 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v56) ==== 97+0+0 (secure 0 0 0) 0x7fbe68059630 con 0x7fbe70161d30 2024-04-20T11:30:55.798 INFO:teuthology.orchestra.run.smithi152.stderr: removed all blocklist entries 2024-04-20T11:30:55.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:55.802+0000 7fbe75600640 1 -- 172.21.15.152:0/306041170 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fbe5006caa0 msgr2=0x7fbe5006ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:55.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:55.802+0000 7fbe75600640 1 --2- 172.21.15.152:0/306041170 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fbe5006caa0 0x7fbe5006ef60 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fbe60036000 tx=0x7fbe6003b040 comp rx=0 tx=0).stop 2024-04-20T11:30:55.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:55.802+0000 7fbe75600640 1 -- 172.21.15.152:0/306041170 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbe70161d30 msgr2=0x7fbe700ccec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:55.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:55.802+0000 7fbe75600640 1 --2- 172.21.15.152:0/306041170 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbe70161d30 0x7fbe700ccec0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fbe6800ca80 tx=0x7fbe6800cf40 comp rx=0 tx=0).stop 2024-04-20T11:30:55.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:55.802+0000 7fbe75600640 1 -- 172.21.15.152:0/306041170 shutdown_connections 2024-04-20T11:30:55.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:55.802+0000 7fbe75600640 1 --2- 172.21.15.152:0/306041170 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fbe5006caa0 0x7fbe5006ef60 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:55.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:55.802+0000 7fbe75600640 1 --2- 172.21.15.152:0/306041170 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbe70161d30 0x7fbe700ccec0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:55.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:55.802+0000 7fbe75600640 1 --2- 172.21.15.152:0/306041170 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbe700d4ec0 0x7fbe7015a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:55.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:55.802+0000 7fbe75600640 1 --2- 172.21.15.152:0/306041170 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbe700d2b80 0x7fbe700ce6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:55.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:55.802+0000 7fbe75600640 1 -- 172.21.15.152:0/306041170 >> 172.21.15.152:0/306041170 conn(0x7fbe7008bfd0 msgr2=0x7fbe700b11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:55.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:55.802+0000 7fbe75600640 1 -- 172.21.15.152:0/306041170 shutdown_connections 2024-04-20T11:30:55.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:55.802+0000 7fbe75600640 1 -- 172.21.15.152:0/306041170 wait complete. 2024-04-20T11:30:55.823 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-20T11:30:56.018 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.017+0000 7f6029e00640 1 Processor -- start 2024-04-20T11:30:56.018 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.018+0000 7f6029e00640 1 -- start start 2024-04-20T11:30:56.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.019+0000 7f6029e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f60240d33a0 0x7f60240cfcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:56.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.019+0000 7f6029e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f60240d01f0 0x7f60240b3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:56.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.019+0000 7f6029e00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60240d2320 con 0x7f60240d0f60 2024-04-20T11:30:56.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.019+0000 7f6029e00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f60240d25e0 con 0x7f60240d01f0 2024-04-20T11:30:56.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.019+0000 7f6029e00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f60240d28c0 con 0x7f60240d33a0 2024-04-20T11:30:56.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.019+0000 7f6028a00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f60240d0f60 0x7f60240d1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:50882/0 (socket says 172.21.15.152:50882) 2024-04-20T11:30:56.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.020+0000 7f6028a00640 1 -- 172.21.15.152:0/3832184051 learned_addr learned my addr 172.21.15.152:0/3832184051 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:30:56.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.020+0000 7f6029400640 1 --2- 172.21.15.152:0/3832184051 >> v2:172.21.15.152:3300/0 conn(0x7f60240d01f0 0x7f60240b3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:56.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.020+0000 7f6023400640 1 -- 172.21.15.152:0/3832184051 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3634342898 0 0) 0x7f60240d2320 con 0x7f60240d0f60 2024-04-20T11:30:56.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.020+0000 7f6023e00640 1 --2- 172.21.15.152:0/3832184051 >> v2:172.21.15.175:3300/0 conn(0x7f60240d33a0 0x7f60240cfcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:56.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.020+0000 7f6023400640 1 -- 172.21.15.152:0/3832184051 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6008003520 con 0x7f60240d0f60 2024-04-20T11:30:56.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.020+0000 7f6029400640 1 -- 172.21.15.152:0/3832184051 >> v2:172.21.15.175:3300/0 conn(0x7f60240d33a0 msgr2=0x7f60240cfcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:56.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.020+0000 7f6029400640 1 --2- 172.21.15.152:0/3832184051 >> v2:172.21.15.175:3300/0 conn(0x7f60240d33a0 0x7f60240cfcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:56.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.020+0000 7f6029400640 1 -- 172.21.15.152:0/3832184051 >> v1:172.21.15.152:6789/0 conn(0x7f60240d0f60 legacy=0x7f60240d1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:56.026 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.020+0000 7f6029400640 1 -- 172.21.15.152:0/3832184051 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60240d2ba0 con 0x7f60240d01f0 2024-04-20T11:30:56.026 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.021+0000 7f6029400640 1 --2- 172.21.15.152:0/3832184051 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f60240d01f0 0x7f60240b3d00 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f602408d0f0 tx=0x7f601c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7d90d3956855af5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:56.026 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.022+0000 7f6023400640 1 -- 172.21.15.152:0/3832184051 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f601c035020 con 0x7f60240d01f0 2024-04-20T11:30:56.026 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.022+0000 7f6023400640 1 -- 172.21.15.152:0/3832184051 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f601c031830 con 0x7f60240d01f0 2024-04-20T11:30:56.026 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.022+0000 7f6023400640 1 -- 172.21.15.152:0/3832184051 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f601c031c00 con 0x7f60240d01f0 2024-04-20T11:30:56.026 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.022+0000 7f6029e00640 1 -- 172.21.15.152:0/3832184051 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f60240d01f0 msgr2=0x7f60240b3d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:56.026 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.022+0000 7f6029e00640 1 --2- 172.21.15.152:0/3832184051 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f60240d01f0 0x7f60240b3d00 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f602408d0f0 tx=0x7f601c02b8d0 comp rx=0 tx=0).stop 2024-04-20T11:30:56.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.022+0000 7f6029e00640 1 -- 172.21.15.152:0/3832184051 shutdown_connections 2024-04-20T11:30:56.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.022+0000 7f6029e00640 1 --2- 172.21.15.152:0/3832184051 >> v2:172.21.15.175:3300/0 conn(0x7f60240d33a0 0x7f60240cfcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:56.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.022+0000 7f6029e00640 1 --2- 172.21.15.152:0/3832184051 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f60240d01f0 0x7f60240b3d00 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:56.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.022+0000 7f6029e00640 1 -- 172.21.15.152:0/3832184051 >> 172.21.15.152:0/3832184051 conn(0x7f602408bfd0 msgr2=0x7f60240d45d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:56.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.022+0000 7f6029e00640 1 -- 172.21.15.152:0/3832184051 shutdown_connections 2024-04-20T11:30:56.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.022+0000 7f6029e00640 1 -- 172.21.15.152:0/3832184051 wait complete. 2024-04-20T11:30:56.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.024+0000 7f6029e00640 1 Processor -- start 2024-04-20T11:30:56.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.024+0000 7f6029e00640 1 -- start start 2024-04-20T11:30:56.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.025+0000 7f6029e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f60240d01f0 0x7f6024158bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:56.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.025+0000 7f6029e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f60240d0f60 0x7f60240c8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:56.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.025+0000 7f6029e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f60240d33a0 0x7f6024159fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:56.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.025+0000 7f6029e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f6024161fe0 con 0x7f60240d33a0 2024-04-20T11:30:56.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.025+0000 7f6029e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6024161e60 con 0x7f60240d01f0 2024-04-20T11:30:56.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.025+0000 7f6029e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f60240b4240 con 0x7f60240d0f60 2024-04-20T11:30:56.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.025+0000 7f6029400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f60240d33a0 0x7f6024159fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:56.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.025+0000 7f6029400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f60240d33a0 0x7f6024159fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:47484/0 (socket says 172.21.15.152:47484) 2024-04-20T11:30:56.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.025+0000 7f6029400640 1 -- 172.21.15.152:0/4100924930 learned_addr learned my addr 172.21.15.152:0/4100924930 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:30:56.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.026+0000 7f6028a00640 1 --2- 172.21.15.152:0/4100924930 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f60240d01f0 0x7f6024158bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:56.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.026+0000 7f6023e00640 1 --2- 172.21.15.152:0/4100924930 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f60240d0f60 0x7f60240c8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:56.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.026+0000 7f6029400640 1 -- 172.21.15.152:0/4100924930 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f60240d0f60 msgr2=0x7f60240c8ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:56.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.026+0000 7f6029400640 1 --2- 172.21.15.152:0/4100924930 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f60240d0f60 0x7f60240c8ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:56.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.026+0000 7f6029400640 1 -- 172.21.15.152:0/4100924930 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f60240d01f0 msgr2=0x7f6024158bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:56.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.026+0000 7f6029400640 1 --2- 172.21.15.152:0/4100924930 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f60240d01f0 0x7f6024158bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:56.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.026+0000 7f6029400640 1 -- 172.21.15.152:0/4100924930 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60240cb7d0 con 0x7f60240d33a0 2024-04-20T11:30:56.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.026+0000 7f6028a00640 1 --2- 172.21.15.152:0/4100924930 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f60240d01f0 0x7f6024158bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:30:56.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.026+0000 7f6029400640 1 --2- 172.21.15.152:0/4100924930 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f60240d33a0 0x7f6024159fb0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f601c009860 tx=0x7f601c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:56.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.026+0000 7f6020c00640 1 -- 172.21.15.152:0/4100924930 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f601c035040 con 0x7f60240d33a0 2024-04-20T11:30:56.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.027+0000 7f6020c00640 1 -- 172.21.15.152:0/4100924930 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f601c03f070 con 0x7f60240d33a0 2024-04-20T11:30:56.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.027+0000 7f6029e00640 1 -- 172.21.15.152:0/4100924930 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60240cbab0 con 0x7f60240d33a0 2024-04-20T11:30:56.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.027+0000 7f6020c00640 1 -- 172.21.15.152:0/4100924930 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f601c03baa0 con 0x7f60240d33a0 2024-04-20T11:30:56.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.027+0000 7f6029e00640 1 -- 172.21.15.152:0/4100924930 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60241590f0 con 0x7f60240d33a0 2024-04-20T11:30:56.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.028+0000 7f6029e00640 1 -- 172.21.15.152:0/4100924930 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6024075e60 con 0x7f60240d33a0 2024-04-20T11:30:56.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.029+0000 7f6020c00640 1 -- 172.21.15.152:0/4100924930 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 9) ==== 91193+0+0 (secure 0 0 0) 0x7f601c038020 con 0x7f60240d33a0 2024-04-20T11:30:56.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.030+0000 7f6020c00640 1 --2- 172.21.15.152:0/4100924930 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f600406caa0 0x7f600406ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:56.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.030+0000 7f6020c00640 1 -- 172.21.15.152:0/4100924930 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 4857+0+0 (secure 0 0 0) 0x7f601c0ad790 con 0x7f60240d33a0 2024-04-20T11:30:56.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.030+0000 7f6028a00640 1 --2- 172.21.15.152:0/4100924930 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f600406caa0 0x7f600406ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:56.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.031+0000 7f6028a00640 1 --2- 172.21.15.152:0/4100924930 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f600406caa0 0x7f600406ef60 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f6014005880 tx=0x7f6014009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:56.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.035+0000 7f6020c00640 1 -- 172.21.15.152:0/4100924930 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f601c031800 con 0x7f60240d33a0 2024-04-20T11:30:56.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.304+0000 7f6029e00640 1 -- 172.21.15.152:0/4100924930 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f60240d1cf0 con 0x7f60240d33a0 2024-04-20T11:30:56.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.306+0000 7f6020c00640 1 -- 172.21.15.152:0/4100924930 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v10) ==== 80+0+5304 (secure 0 0 0) 0x7f601c07bc80 con 0x7f60240d33a0 2024-04-20T11:30:56.306 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:30:56.306 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-04-20T11:30:56.307 INFO:teuthology.orchestra.run.smithi152.stdout: "auth_dump": [ 2024-04-20T11:30:56.307 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:30:56.307 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "mds.a", 2024-04-20T11:30:56.307 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBCpiNmyFRJIBAAj2aIMXdb65nqRgKFDBLLzA==", 2024-04-20T11:30:56.307 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:30:56.307 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow", 2024-04-20T11:30:56.307 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow *", 2024-04-20T11:30:56.307 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow *", 2024-04-20T11:30:56.307 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:30:56.307 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:30:56.307 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:30:56.307 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:30:56.308 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "mds.b", 2024-04-20T11:30:56.308 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBCpiNmVPytOhAAzQkvNOJaNy+MeKETWhYLyg==", 2024-04-20T11:30:56.308 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:30:56.308 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow", 2024-04-20T11:30:56.308 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow *", 2024-04-20T11:30:56.308 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow *", 2024-04-20T11:30:56.308 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:30:56.308 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:30:56.308 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:30:56.308 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:30:56.308 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "mds.c", 2024-04-20T11:30:56.309 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBCpiNm5ViKLRAAg7VBQG4hFTLYUpJMpBAaVA==", 2024-04-20T11:30:56.309 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:30:56.309 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow", 2024-04-20T11:30:56.309 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow *", 2024-04-20T11:30:56.309 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow *", 2024-04-20T11:30:56.309 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:30:56.309 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:30:56.309 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:30:56.309 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:30:56.309 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "mds.d", 2024-04-20T11:30:56.309 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBDpiNmLWhaCxAA2e4qd/7+TKenJlWICIsvGg==", 2024-04-20T11:30:56.309 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:30:56.310 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow", 2024-04-20T11:30:56.310 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow *", 2024-04-20T11:30:56.310 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow *", 2024-04-20T11:30:56.310 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:30:56.310 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:30:56.310 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:30:56.310 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:30:56.310 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "osd.0", 2024-04-20T11:30:56.310 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBHpiNmpuhyDBAAsQXX9DNvWlo0UYh53lb+FA==", 2024-04-20T11:30:56.310 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:30:56.310 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow profile osd", 2024-04-20T11:30:56.310 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile osd", 2024-04-20T11:30:56.310 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:30:56.311 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:30:56.311 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:30:56.311 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:30:56.311 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "osd.1", 2024-04-20T11:30:56.311 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBKpiNmxBiLJhAAHon1adC8G2jbYoDcWdE9jA==", 2024-04-20T11:30:56.311 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:30:56.311 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow profile osd", 2024-04-20T11:30:56.311 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile osd", 2024-04-20T11:30:56.311 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:30:56.311 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:30:56.311 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:30:56.311 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:30:56.312 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "osd.2", 2024-04-20T11:30:56.312 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBNpiNmdn0zMhAAqwIO2kKahdSzTVh8/p3FCg==", 2024-04-20T11:30:56.312 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:30:56.312 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow profile osd", 2024-04-20T11:30:56.312 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile osd", 2024-04-20T11:30:56.312 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:30:56.312 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:30:56.312 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:30:56.312 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:30:56.312 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "osd.3", 2024-04-20T11:30:56.312 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBRpiNmTPCYAhAAHFNHX/xKn21jEFHDkHMjEw==", 2024-04-20T11:30:56.312 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:30:56.312 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow profile osd", 2024-04-20T11:30:56.313 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile osd", 2024-04-20T11:30:56.313 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:30:56.313 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:30:56.313 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:30:56.313 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:30:56.313 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "osd.4", 2024-04-20T11:30:56.313 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBXpiNmM8cVFxAAJl9Hu+MvlkEiMglutI89eg==", 2024-04-20T11:30:56.313 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:30:56.313 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow profile osd", 2024-04-20T11:30:56.313 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile osd", 2024-04-20T11:30:56.313 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:30:56.313 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:30:56.314 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:30:56.314 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:30:56.314 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "osd.5", 2024-04-20T11:30:56.314 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBapiNmnGxJMRAA90P0T+VsTA8LCTcexTVr7w==", 2024-04-20T11:30:56.314 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:30:56.314 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow profile osd", 2024-04-20T11:30:56.314 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile osd", 2024-04-20T11:30:56.314 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:30:56.314 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:30:56.314 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:30:56.314 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:30:56.314 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "osd.6", 2024-04-20T11:30:56.315 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBepiNmyLXWDxAAhtHD5h478mLovE5IejAFng==", 2024-04-20T11:30:56.315 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:30:56.315 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow profile osd", 2024-04-20T11:30:56.315 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile osd", 2024-04-20T11:30:56.315 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:30:56.315 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:30:56.315 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:30:56.315 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:30:56.315 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "osd.7", 2024-04-20T11:30:56.315 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBhpiNmB3jwHBAAqco9om6r2N/GqAV6rVKiaA==", 2024-04-20T11:30:56.315 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:30:56.315 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow profile osd", 2024-04-20T11:30:56.315 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile osd", 2024-04-20T11:30:56.316 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:30:56.316 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:30:56.316 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:30:56.316 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:30:56.316 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "client.0", 2024-04-20T11:30:56.316 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBDpiNmtsBWFhAArq9A7r33CcwgMmOU2pAGPw==", 2024-04-20T11:30:56.316 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:30:56.316 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow", 2024-04-20T11:30:56.316 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow r", 2024-04-20T11:30:56.316 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-20T11:30:56.316 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:30:56.316 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:30:56.317 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:30:56.317 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "client.1", 2024-04-20T11:30:56.317 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBDpiNm6aNnJxAAB431GSGytZ0IKWMh4DMalw==", 2024-04-20T11:30:56.317 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:30:56.317 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow", 2024-04-20T11:30:56.317 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow r", 2024-04-20T11:30:56.317 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow rw", 2024-04-20T11:30:56.317 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow rwx" 2024-04-20T11:30:56.317 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:30:56.317 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:30:56.317 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:30:56.317 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "client.2", 2024-04-20T11:30:56.318 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBDpiNm5QLtHRAAa2kHTDVdgSh/B4XGZaKa5Q==", 2024-04-20T11:30:56.318 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:30:56.318 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow", 2024-04-20T11:30:56.318 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow r", 2024-04-20T11:30:56.318 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow rw", 2024-04-20T11:30:56.318 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow rwx" 2024-04-20T11:30:56.318 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:30:56.318 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:30:56.318 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:30:56.318 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "client.3", 2024-04-20T11:30:56.318 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBDpiNmw95SMBAAdodzTuPp4WdKSEf+jI8Uuw==", 2024-04-20T11:30:56.318 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:30:56.319 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow", 2024-04-20T11:30:56.319 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow r", 2024-04-20T11:30:56.319 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow rw", 2024-04-20T11:30:56.319 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow rwx" 2024-04-20T11:30:56.319 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:30:56.319 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:30:56.319 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:30:56.319 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "client.admin", 2024-04-20T11:30:56.319 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBBpiNm3xz6HhAA82+N+trDW/AmBr3q8l4ECw==", 2024-04-20T11:30:56.319 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:30:56.319 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow *", 2024-04-20T11:30:56.319 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow *", 2024-04-20T11:30:56.319 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow *", 2024-04-20T11:30:56.320 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:30:56.320 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:30:56.320 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:30:56.320 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:30:56.320 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "mgr.x", 2024-04-20T11:30:56.320 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBCpiNmuhalFxAA9dTl/zSbUovCCZAanAipww==", 2024-04-20T11:30:56.320 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:30:56.320 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow *", 2024-04-20T11:30:56.320 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile mgr", 2024-04-20T11:30:56.320 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:30:56.320 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:30:56.320 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:30:56.320 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:30:56.321 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "mgr.y", 2024-04-20T11:30:56.321 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBCpiNmP1+5DRAAs6bVAEOr0i6FdPmSEyo/vQ==", 2024-04-20T11:30:56.321 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:30:56.321 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow *", 2024-04-20T11:30:56.321 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile mgr", 2024-04-20T11:30:56.321 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:30:56.321 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:30:56.321 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:30:56.321 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-04-20T11:30:56.321 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-04-20T11:30:56.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.308+0000 7f6029e00640 1 -- 172.21.15.152:0/4100924930 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f600406caa0 msgr2=0x7f600406ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:56.323 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.308+0000 7f6029e00640 1 --2- 172.21.15.152:0/4100924930 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f600406caa0 0x7f600406ef60 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f6014005880 tx=0x7f6014009040 comp rx=0 tx=0).stop 2024-04-20T11:30:56.323 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.308+0000 7f6029e00640 1 -- 172.21.15.152:0/4100924930 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f60240d33a0 msgr2=0x7f6024159fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:56.323 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.308+0000 7f6029e00640 1 --2- 172.21.15.152:0/4100924930 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f60240d33a0 0x7f6024159fb0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f601c009860 tx=0x7f601c0046c0 comp rx=0 tx=0).stop 2024-04-20T11:30:56.323 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.308+0000 7f6029e00640 1 -- 172.21.15.152:0/4100924930 shutdown_connections 2024-04-20T11:30:56.323 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.308+0000 7f6029e00640 1 --2- 172.21.15.152:0/4100924930 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f600406caa0 0x7f600406ef60 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:56.323 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.308+0000 7f6029e00640 1 --2- 172.21.15.152:0/4100924930 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f60240d33a0 0x7f6024159fb0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:56.323 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.308+0000 7f6029e00640 1 --2- 172.21.15.152:0/4100924930 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f60240d0f60 0x7f60240c8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:56.323 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.308+0000 7f6029e00640 1 --2- 172.21.15.152:0/4100924930 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f60240d01f0 0x7f6024158bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:56.323 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.308+0000 7f6029e00640 1 -- 172.21.15.152:0/4100924930 >> 172.21.15.152:0/4100924930 conn(0x7f602408bfd0 msgr2=0x7f602406e120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:56.323 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.308+0000 7f6029e00640 1 -- 172.21.15.152:0/4100924930 shutdown_connections 2024-04-20T11:30:56.323 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.308+0000 7f6029e00640 1 -- 172.21.15.152:0/4100924930 wait complete. 2024-04-20T11:30:56.328 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:30:56.527 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.526+0000 7f82d8c00640 1 Processor -- start 2024-04-20T11:30:56.527 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.526+0000 7f82d8c00640 1 -- start start 2024-04-20T11:30:56.528 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.528+0000 7f82d8c00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f82d4147410 0x7f82d40afa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:56.529 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.528+0000 7f82d8c00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f82d406e720 0x7f82d4072130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:56.529 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.528+0000 7f82d8c00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82d40aec90 con 0x7f82d4071cd0 2024-04-20T11:30:56.529 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.528+0000 7f82d8c00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f82d4072670 con 0x7f82d406e720 2024-04-20T11:30:56.529 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.528+0000 7f82d8c00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f82d40aef70 con 0x7f82d4147410 2024-04-20T11:30:56.529 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.528+0000 7f82d3e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f82d406e720 0x7f82d4072130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:56.529 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.528+0000 7f82d3e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f82d406e720 0x7f82d4072130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:47488/0 (socket says 172.21.15.152:47488) 2024-04-20T11:30:56.529 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.528+0000 7f82d3e00640 1 -- 172.21.15.152:0/2366576858 learned_addr learned my addr 172.21.15.152:0/2366576858 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:30:56.530 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.529+0000 7f82d3400640 1 --2- 172.21.15.152:0/2366576858 >> v2:172.21.15.175:3300/0 conn(0x7f82d4147410 0x7f82d40afa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:56.530 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.529+0000 7f82d3e00640 1 -- 172.21.15.152:0/2366576858 >> v2:172.21.15.175:3300/0 conn(0x7f82d4147410 msgr2=0x7f82d40afa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:56.530 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.529+0000 7f82d3e00640 1 --2- 172.21.15.152:0/2366576858 >> v2:172.21.15.175:3300/0 conn(0x7f82d4147410 0x7f82d40afa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:56.530 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.529+0000 7f82d3e00640 1 -- 172.21.15.152:0/2366576858 >> v1:172.21.15.175:6789/0 conn(0x7f82d4071cd0 legacy=0x7f82d406e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:56.530 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.529+0000 7f82d3e00640 1 -- 172.21.15.152:0/2366576858 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82d40af250 con 0x7f82d406e720 2024-04-20T11:30:56.530 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.530+0000 7f82d3e00640 1 --2- 172.21.15.152:0/2366576858 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f82d406e720 0x7f82d4072130 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f82d40b30e0 tx=0x7f82cc02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d2724bff51f3f0e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:56.531 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.530+0000 7f82d2000640 1 -- 172.21.15.152:0/2366576858 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82cc02ca80 con 0x7f82d406e720 2024-04-20T11:30:56.531 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.530+0000 7f82d2000640 1 -- 172.21.15.152:0/2366576858 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82cc02cc20 con 0x7f82d406e720 2024-04-20T11:30:56.531 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.530+0000 7f82d2000640 1 -- 172.21.15.152:0/2366576858 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82cc0343f0 con 0x7f82d406e720 2024-04-20T11:30:56.531 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.531+0000 7f82d8c00640 1 -- 172.21.15.152:0/2366576858 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f82d406e720 msgr2=0x7f82d4072130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:56.532 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.531+0000 7f82d8c00640 1 --2- 172.21.15.152:0/2366576858 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f82d406e720 0x7f82d4072130 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f82d40b30e0 tx=0x7f82cc02bee0 comp rx=0 tx=0).stop 2024-04-20T11:30:56.532 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.531+0000 7f82d8c00640 1 -- 172.21.15.152:0/2366576858 shutdown_connections 2024-04-20T11:30:56.532 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.531+0000 7f82d8c00640 1 --2- 172.21.15.152:0/2366576858 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f82d406e720 0x7f82d4072130 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:56.532 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.531+0000 7f82d8c00640 1 --2- 172.21.15.152:0/2366576858 >> v2:172.21.15.175:3300/0 conn(0x7f82d4147410 0x7f82d40afa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:56.532 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.531+0000 7f82d8c00640 1 -- 172.21.15.152:0/2366576858 >> 172.21.15.152:0/2366576858 conn(0x7f82d4067ae0 msgr2=0x7f82d406b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:56.532 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.531+0000 7f82d8c00640 1 -- 172.21.15.152:0/2366576858 shutdown_connections 2024-04-20T11:30:56.532 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.531+0000 7f82d8c00640 1 -- 172.21.15.152:0/2366576858 wait complete. 2024-04-20T11:30:56.533 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.533+0000 7f82d8c00640 1 Processor -- start 2024-04-20T11:30:56.533 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.533+0000 7f82d8c00640 1 -- start start 2024-04-20T11:30:56.535 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.534+0000 7f82d8c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f82d406e720 0x7f82d40cee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:56.535 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.534+0000 7f82d8c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f82d4071cd0 0x7f82d40b02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:56.535 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.534+0000 7f82d8c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f82d4147410 0x7f82d4161030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:56.535 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.534+0000 7f82d8c00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f82d40b3390 con 0x7f82d4071cd0 2024-04-20T11:30:56.535 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.534+0000 7f82d8c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f82d40ae980 con 0x7f82d4147410 2024-04-20T11:30:56.535 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.534+0000 7f82d8c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f82d40b3510 con 0x7f82d406e720 2024-04-20T11:30:56.535 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.534+0000 7f82d2a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f82d4071cd0 0x7f82d40b02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:56.536 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.534+0000 7f82d2a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f82d4071cd0 0x7f82d40b02b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:47494/0 (socket says 172.21.15.152:47494) 2024-04-20T11:30:56.536 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.534+0000 7f82d2a00640 1 -- 172.21.15.152:0/1572701446 learned_addr learned my addr 172.21.15.152:0/1572701446 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:30:56.536 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.534+0000 7f82d3400640 1 --2- 172.21.15.152:0/1572701446 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f82d406e720 0x7f82d40cee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:56.536 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.534+0000 7f82d2a00640 1 -- 172.21.15.152:0/1572701446 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f82d406e720 msgr2=0x7f82d40cee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:56.536 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.534+0000 7f82d3e00640 1 --2- 172.21.15.152:0/1572701446 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f82d4147410 0x7f82d4161030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:56.536 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.534+0000 7f82d2a00640 1 --2- 172.21.15.152:0/1572701446 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f82d406e720 0x7f82d40cee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:56.536 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.534+0000 7f82d2a00640 1 -- 172.21.15.152:0/1572701446 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f82d4147410 msgr2=0x7f82d4161030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:56.536 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.534+0000 7f82d2a00640 1 --2- 172.21.15.152:0/1572701446 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f82d4147410 0x7f82d4161030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:56.536 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.534+0000 7f82d2a00640 1 -- 172.21.15.152:0/1572701446 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82d40cf340 con 0x7f82d4071cd0 2024-04-20T11:30:56.536 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.534+0000 7f82d3400640 1 --2- 172.21.15.152:0/1572701446 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f82d406e720 0x7f82d40cee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:30:56.537 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.535+0000 7f82d2a00640 1 --2- 172.21.15.152:0/1572701446 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f82d4071cd0 0x7f82d40b02b0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f82c000c430 tx=0x7f82c000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:56.537 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.535+0000 7f82cb400640 1 -- 172.21.15.152:0/1572701446 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c0011790 con 0x7f82d4071cd0 2024-04-20T11:30:56.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.535+0000 7f82cb400640 1 -- 172.21.15.152:0/1572701446 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82c0017070 con 0x7f82d4071cd0 2024-04-20T11:30:56.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.535+0000 7f82cb400640 1 -- 172.21.15.152:0/1572701446 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c0011bb0 con 0x7f82d4071cd0 2024-04-20T11:30:56.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.535+0000 7f82d8c00640 1 -- 172.21.15.152:0/1572701446 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82d40b0d30 con 0x7f82d4071cd0 2024-04-20T11:30:56.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.535+0000 7f82d8c00640 1 -- 172.21.15.152:0/1572701446 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82d40b12f0 con 0x7f82d4071cd0 2024-04-20T11:30:56.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.536+0000 7f82d8c00640 1 -- 172.21.15.152:0/1572701446 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82d4075e60 con 0x7f82d4071cd0 2024-04-20T11:30:56.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.537+0000 7f82cb400640 1 -- 172.21.15.152:0/1572701446 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 9) ==== 91193+0+0 (secure 0 0 0) 0x7f82c001c460 con 0x7f82d4071cd0 2024-04-20T11:30:56.539 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.538+0000 7f82cb400640 1 --2- 172.21.15.152:0/1572701446 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f82b406caa0 0x7f82b406ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:56.539 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.538+0000 7f82cb400640 1 -- 172.21.15.152:0/1572701446 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 4857+0+0 (secure 0 0 0) 0x7f82c008bad0 con 0x7f82d4071cd0 2024-04-20T11:30:56.539 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.538+0000 7f82d3400640 1 --2- 172.21.15.152:0/1572701446 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f82b406caa0 0x7f82b406ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:56.539 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.539+0000 7f82d3400640 1 --2- 172.21.15.152:0/1572701446 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f82b406caa0 0x7f82b406ef60 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f82c4006fd0 tx=0x7f82c4008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:56.543 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.543+0000 7f82cb400640 1 -- 172.21.15.152:0/1572701446 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f82c0059f40 con 0x7f82d4071cd0 2024-04-20T11:30:56.743 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.742+0000 7f82d8c00640 1 -- 172.21.15.152:0/1572701446 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f82d4074c50 con 0x7f82d4071cd0 2024-04-20T11:30:56.744 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.744+0000 7f82cb400640 1 -- 172.21.15.152:0/1572701446 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) ==== 74+0+12951 (secure 0 0 0) 0x7f82c0059840 con 0x7f82d4071cd0 2024-04-20T11:30:56.745 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:30:56.745 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":56,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:30:55.795325+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:30:56.747 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.747+0000 7f82d8c00640 1 -- 172.21.15.152:0/1572701446 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f82b406caa0 msgr2=0x7f82b406ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:56.747 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.747+0000 7f82d8c00640 1 --2- 172.21.15.152:0/1572701446 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f82b406caa0 0x7f82b406ef60 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f82c4006fd0 tx=0x7f82c4008040 comp rx=0 tx=0).stop 2024-04-20T11:30:56.747 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.747+0000 7f82d8c00640 1 -- 172.21.15.152:0/1572701446 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f82d4071cd0 msgr2=0x7f82d40b02b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:56.748 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.747+0000 7f82d8c00640 1 --2- 172.21.15.152:0/1572701446 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f82d4071cd0 0x7f82d40b02b0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f82c000c430 tx=0x7f82c000c8f0 comp rx=0 tx=0).stop 2024-04-20T11:30:56.748 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.747+0000 7f82d8c00640 1 -- 172.21.15.152:0/1572701446 shutdown_connections 2024-04-20T11:30:56.748 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.747+0000 7f82d8c00640 1 --2- 172.21.15.152:0/1572701446 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f82b406caa0 0x7f82b406ef60 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:56.748 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.747+0000 7f82d8c00640 1 --2- 172.21.15.152:0/1572701446 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f82d4147410 0x7f82d4161030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:56.748 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.747+0000 7f82d8c00640 1 --2- 172.21.15.152:0/1572701446 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f82d4071cd0 0x7f82d40b02b0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:56.748 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.747+0000 7f82d8c00640 1 --2- 172.21.15.152:0/1572701446 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f82d406e720 0x7f82d40cee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:56.748 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.747+0000 7f82d8c00640 1 -- 172.21.15.152:0/1572701446 >> 172.21.15.152:0/1572701446 conn(0x7f82d4067ae0 msgr2=0x7f82d41477f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:56.748 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.747+0000 7f82d8c00640 1 -- 172.21.15.152:0/1572701446 shutdown_connections 2024-04-20T11:30:56.748 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.747+0000 7f82d8c00640 1 -- 172.21.15.152:0/1572701446 wait complete. 2024-04-20T11:30:56.767 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T11:26:47.246571+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T11:30:56.767 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T11:30:56.970 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.968+0000 7f984e200640 1 Processor -- start 2024-04-20T11:30:56.970 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.969+0000 7f984e200640 1 -- start start 2024-04-20T11:30:56.972 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.970+0000 7f984e200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f98480afba0 0x7f9848162440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:56.972 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.970+0000 7f984e200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f98480b3fa0 0x7f98480ae110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:56.972 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.970+0000 7f984e200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98480adc30 con 0x7f984815e5d0 2024-04-20T11:30:56.972 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.970+0000 7f984e200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f9848162a70 con 0x7f98480afba0 2024-04-20T11:30:56.972 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.970+0000 7f984e200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f9848162d50 con 0x7f98480b3fa0 2024-04-20T11:30:56.972 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.971+0000 7f9846a00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f98480afba0 0x7f9848162440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:56.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.971+0000 7f9847e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f98480b3fa0 0x7f98480ae110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:56.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.971+0000 7f9846a00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f98480afba0 0x7f9848162440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.152:38614/0 (socket says 172.21.15.152:38614) 2024-04-20T11:30:56.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.971+0000 7f9846a00640 1 -- 172.21.15.152:0/2474106988 learned_addr learned my addr 172.21.15.152:0/2474106988 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:30:56.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.972+0000 7f9847e00640 1 -- 172.21.15.152:0/2474106988 >> v2:172.21.15.175:3300/0 conn(0x7f98480afba0 msgr2=0x7f9848162440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:56.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.972+0000 7f9847e00640 1 --2- 172.21.15.152:0/2474106988 >> v2:172.21.15.175:3300/0 conn(0x7f98480afba0 0x7f9848162440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:56.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.972+0000 7f9847e00640 1 -- 172.21.15.152:0/2474106988 >> v1:172.21.15.175:6789/0 conn(0x7f984815e5d0 legacy=0x7f98480af490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:56.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.972+0000 7f9847e00640 1 -- 172.21.15.152:0/2474106988 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9848163030 con 0x7f98480b3fa0 2024-04-20T11:30:56.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.973+0000 7f9847e00640 1 --2- 172.21.15.152:0/2474106988 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f98480b3fa0 0x7f98480ae110 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f98400097a0 tx=0x7f984002b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4bdfe2f0ffda4117 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:56.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.974+0000 7f9846000640 1 -- 172.21.15.152:0/2474106988 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9840035020 con 0x7f98480b3fa0 2024-04-20T11:30:56.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.974+0000 7f9846000640 1 -- 172.21.15.152:0/2474106988 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9840031830 con 0x7f98480b3fa0 2024-04-20T11:30:56.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.974+0000 7f9846000640 1 -- 172.21.15.152:0/2474106988 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9840031c00 con 0x7f98480b3fa0 2024-04-20T11:30:56.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.975+0000 7f984e200640 1 -- 172.21.15.152:0/2474106988 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f98480b3fa0 msgr2=0x7f98480ae110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:56.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.975+0000 7f984e200640 1 --2- 172.21.15.152:0/2474106988 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f98480b3fa0 0x7f98480ae110 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f98400097a0 tx=0x7f984002b8d0 comp rx=0 tx=0).stop 2024-04-20T11:30:56.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.975+0000 7f984e200640 1 -- 172.21.15.152:0/2474106988 shutdown_connections 2024-04-20T11:30:56.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.975+0000 7f984e200640 1 --2- 172.21.15.152:0/2474106988 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f98480b3fa0 0x7f98480ae110 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:56.982 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.975+0000 7f984e200640 1 --2- 172.21.15.152:0/2474106988 >> v2:172.21.15.175:3300/0 conn(0x7f98480afba0 0x7f9848162440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:56.982 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.975+0000 7f984e200640 1 -- 172.21.15.152:0/2474106988 >> 172.21.15.152:0/2474106988 conn(0x7f984806b980 msgr2=0x7f984806bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:56.982 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.975+0000 7f984e200640 1 -- 172.21.15.152:0/2474106988 shutdown_connections 2024-04-20T11:30:56.982 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.975+0000 7f984e200640 1 -- 172.21.15.152:0/2474106988 wait complete. 2024-04-20T11:30:56.982 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.977+0000 7f984e200640 1 Processor -- start 2024-04-20T11:30:56.982 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.977+0000 7f984e200640 1 -- start start 2024-04-20T11:30:56.982 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.978+0000 7f984e200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98480afba0 0x7f98480cd5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:56.982 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.978+0000 7f984e200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f98480b3fa0 0x7f9848147a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:56.982 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.978+0000 7f984e200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f984815e5d0 0x7f98480cffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:56.982 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.978+0000 7f984e200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f984806e0a0 con 0x7f98480b3fa0 2024-04-20T11:30:56.982 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.978+0000 7f984e200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9848163310 con 0x7f98480afba0 2024-04-20T11:30:56.982 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.978+0000 7f984e200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f984806e220 con 0x7f984815e5d0 2024-04-20T11:30:56.982 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.978+0000 7f9846a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f98480b3fa0 0x7f9848147a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:56.982 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.978+0000 7f9846a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f98480b3fa0 0x7f9848147a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:47510/0 (socket says 172.21.15.152:47510) 2024-04-20T11:30:56.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.978+0000 7f9846a00640 1 -- 172.21.15.152:0/1124177533 learned_addr learned my addr 172.21.15.152:0/1124177533 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:30:56.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.978+0000 7f9847400640 1 --2- 172.21.15.152:0/1124177533 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98480afba0 0x7f98480cd5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:56.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.978+0000 7f9847e00640 1 --2- 172.21.15.152:0/1124177533 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f984815e5d0 0x7f98480cffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:56.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.978+0000 7f9846a00640 1 -- 172.21.15.152:0/1124177533 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f984815e5d0 msgr2=0x7f98480cffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:56.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.978+0000 7f9846a00640 1 --2- 172.21.15.152:0/1124177533 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f984815e5d0 0x7f98480cffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:56.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.978+0000 7f9846a00640 1 -- 172.21.15.152:0/1124177533 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98480afba0 msgr2=0x7f98480cd5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:56.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.978+0000 7f9846a00640 1 --2- 172.21.15.152:0/1124177533 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98480afba0 0x7f98480cd5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:56.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.978+0000 7f9846a00640 1 -- 172.21.15.152:0/1124177533 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98481482b0 con 0x7f98480b3fa0 2024-04-20T11:30:56.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.978+0000 7f9847400640 1 --2- 172.21.15.152:0/1124177533 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98480afba0 0x7f98480cd5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:30:56.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.979+0000 7f9846a00640 1 --2- 172.21.15.152:0/1124177533 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f98480b3fa0 0x7f9848147a00 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f983400efc0 tx=0x7f983400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:56.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.979+0000 7f983f400640 1 -- 172.21.15.152:0/1124177533 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f983400cda0 con 0x7f98480b3fa0 2024-04-20T11:30:56.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.979+0000 7f983f400640 1 -- 172.21.15.152:0/1124177533 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9834019070 con 0x7f98480b3fa0 2024-04-20T11:30:56.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.979+0000 7f984e200640 1 -- 172.21.15.152:0/1124177533 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9848148590 con 0x7f98480b3fa0 2024-04-20T11:30:56.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.979+0000 7f983f400640 1 -- 172.21.15.152:0/1124177533 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98340146a0 con 0x7f98480b3fa0 2024-04-20T11:30:56.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.979+0000 7f984e200640 1 -- 172.21.15.152:0/1124177533 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9848148b50 con 0x7f98480b3fa0 2024-04-20T11:30:56.984 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.980+0000 7f984e200640 1 -- 172.21.15.152:0/1124177533 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98480689f0 con 0x7f98480b3fa0 2024-04-20T11:30:56.984 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.981+0000 7f983f400640 1 -- 172.21.15.152:0/1124177533 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 9) ==== 91193+0+0 (secure 0 0 0) 0x7f9834005d10 con 0x7f98480b3fa0 2024-04-20T11:30:56.984 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.983+0000 7f983f400640 1 --2- 172.21.15.152:0/1124177533 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f982806cad0 0x7f982806ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:56.985 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.983+0000 7f983f400640 1 -- 172.21.15.152:0/1124177533 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 4857+0+0 (secure 0 0 0) 0x7f983408c1e0 con 0x7f98480b3fa0 2024-04-20T11:30:56.985 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.983+0000 7f9847400640 1 --2- 172.21.15.152:0/1124177533 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f982806cad0 0x7f982806ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:56.985 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.983+0000 7f9847400640 1 --2- 172.21.15.152:0/1124177533 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f982806cad0 0x7f982806ef90 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f98380015e0 tx=0x7f9838009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:56.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:56.987+0000 7f983f400640 1 -- 172.21.15.152:0/1124177533 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f9834010040 con 0x7f98480b3fa0 2024-04-20T11:30:57.187 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.186+0000 7f984e200640 1 -- 172.21.15.152:0/1124177533 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f984815abc0 con 0x7f98480b3fa0 2024-04-20T11:30:57.187 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.187+0000 7f983f400640 1 -- 172.21.15.152:0/1124177533 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v56) ==== 93+0+10 (secure 0 0 0) 0x7f983405a530 con 0x7f98480b3fa0 2024-04-20T11:30:57.188 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 1 2024-04-20T11:30:57.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.189+0000 7f984e200640 1 -- 172.21.15.152:0/1124177533 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f982806cad0 msgr2=0x7f982806ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:57.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.189+0000 7f984e200640 1 --2- 172.21.15.152:0/1124177533 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f982806cad0 0x7f982806ef90 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f98380015e0 tx=0x7f9838009040 comp rx=0 tx=0).stop 2024-04-20T11:30:57.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.189+0000 7f984e200640 1 -- 172.21.15.152:0/1124177533 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f98480b3fa0 msgr2=0x7f9848147a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:57.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.189+0000 7f984e200640 1 --2- 172.21.15.152:0/1124177533 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f98480b3fa0 0x7f9848147a00 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f983400efc0 tx=0x7f983400c490 comp rx=0 tx=0).stop 2024-04-20T11:30:57.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.190+0000 7f984e200640 1 -- 172.21.15.152:0/1124177533 shutdown_connections 2024-04-20T11:30:57.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.190+0000 7f984e200640 1 --2- 172.21.15.152:0/1124177533 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f982806cad0 0x7f982806ef90 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:57.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.190+0000 7f984e200640 1 --2- 172.21.15.152:0/1124177533 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f984815e5d0 0x7f98480cffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:57.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.190+0000 7f984e200640 1 --2- 172.21.15.152:0/1124177533 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f98480b3fa0 0x7f9848147a00 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:57.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.190+0000 7f984e200640 1 --2- 172.21.15.152:0/1124177533 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98480afba0 0x7f98480cd5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:57.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.190+0000 7f984e200640 1 -- 172.21.15.152:0/1124177533 >> 172.21.15.152:0/1124177533 conn(0x7f984806b980 msgr2=0x7f984815b3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:57.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.190+0000 7f984e200640 1 -- 172.21.15.152:0/1124177533 shutdown_connections 2024-04-20T11:30:57.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.190+0000 7f984e200640 1 -- 172.21.15.152:0/1124177533 wait complete. 2024-04-20T11:30:57.208 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-20T11:30:57.405 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.404+0000 7f1379400640 1 Processor -- start 2024-04-20T11:30:57.406 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.405+0000 7f1379400640 1 -- start start 2024-04-20T11:30:57.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.406+0000 7f1379400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1374071cd0 0x7f13740720b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:57.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.406+0000 7f1379400640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f1374162e50 0x7f1374163300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:57.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.406+0000 7f1379400640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13740725f0 con 0x7f137406ea20 2024-04-20T11:30:57.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.406+0000 7f1379400640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f13740ceec0 con 0x7f1374162e50 2024-04-20T11:30:57.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.406+0000 7f1379400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f137406dcf0 con 0x7f1374071cd0 2024-04-20T11:30:57.408 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.407+0000 7f1378a00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f1374162e50 0x7f1374163300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:57.408 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.407+0000 7f136b400640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7f137406ea20 0x7f13740b4a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.152:36318/0 (socket says 172.21.15.152:36318) 2024-04-20T11:30:57.408 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.407+0000 7f136b400640 1 -- 172.21.15.152:0/3520033464 learned_addr learned my addr 172.21.15.152:0/3520033464 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:30:57.408 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.407+0000 7f1373e00640 1 --2- 172.21.15.152:0/3520033464 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1374071cd0 0x7f13740720b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:57.409 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.408+0000 7f1378a00640 1 -- 172.21.15.152:0/3520033464 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1374071cd0 msgr2=0x7f13740720b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:57.409 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.408+0000 7f1378a00640 1 --2- 172.21.15.152:0/3520033464 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1374071cd0 0x7f13740720b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:57.409 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.408+0000 7f1378a00640 1 -- 172.21.15.152:0/3520033464 >> v1:172.21.15.175:6789/0 conn(0x7f137406ea20 legacy=0x7f13740b4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:57.409 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.408+0000 7f1378a00640 1 -- 172.21.15.152:0/3520033464 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13740b5510 con 0x7f1374162e50 2024-04-20T11:30:57.410 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.410+0000 7f1378a00640 1 --2- 172.21.15.152:0/3520033464 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1374162e50 0x7f1374163300 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f136c009d90 tx=0x7f136c02bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=883885e64cddbdc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:57.411 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.410+0000 7f1373400640 1 -- 172.21.15.152:0/3520033464 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f136c039070 con 0x7f1374162e50 2024-04-20T11:30:57.411 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.410+0000 7f1373400640 1 -- 172.21.15.152:0/3520033464 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f136c02cc50 con 0x7f1374162e50 2024-04-20T11:30:57.411 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.410+0000 7f1373400640 1 -- 172.21.15.152:0/3520033464 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f136c034440 con 0x7f1374162e50 2024-04-20T11:30:57.412 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.411+0000 7f1379400640 1 -- 172.21.15.152:0/3520033464 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1374162e50 msgr2=0x7f1374163300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:57.412 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.411+0000 7f1379400640 1 --2- 172.21.15.152:0/3520033464 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1374162e50 0x7f1374163300 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f136c009d90 tx=0x7f136c02bea0 comp rx=0 tx=0).stop 2024-04-20T11:30:57.412 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.411+0000 7f1379400640 1 -- 172.21.15.152:0/3520033464 shutdown_connections 2024-04-20T11:30:57.412 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.411+0000 7f1379400640 1 --2- 172.21.15.152:0/3520033464 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1374162e50 0x7f1374163300 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:57.412 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.411+0000 7f1379400640 1 --2- 172.21.15.152:0/3520033464 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1374071cd0 0x7f13740720b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:57.412 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.411+0000 7f1379400640 1 -- 172.21.15.152:0/3520033464 >> 172.21.15.152:0/3520033464 conn(0x7f1374067ae0 msgr2=0x7f137406b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:57.412 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.411+0000 7f1379400640 1 -- 172.21.15.152:0/3520033464 shutdown_connections 2024-04-20T11:30:57.412 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.411+0000 7f1379400640 1 -- 172.21.15.152:0/3520033464 wait complete. 2024-04-20T11:30:57.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.413+0000 7f1379400640 1 Processor -- start 2024-04-20T11:30:57.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.414+0000 7f1379400640 1 -- start start 2024-04-20T11:30:57.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.415+0000 7f1379400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f137406ea20 0x7f1374148420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:57.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.415+0000 7f1379400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1374071cd0 0x7f137415c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:57.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.415+0000 7f1379400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1374162e50 0x7f137415c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:57.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.415+0000 7f1379400640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f137406e500 con 0x7f137406ea20 2024-04-20T11:30:57.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.415+0000 7f1379400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f137406e220 con 0x7f1374071cd0 2024-04-20T11:30:57.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.415+0000 7f1379400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f13740b4180 con 0x7f1374162e50 2024-04-20T11:30:57.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.415+0000 7f1373e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f137406ea20 0x7f1374148420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:57.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.415+0000 7f1373e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f137406ea20 0x7f1374148420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:47512/0 (socket says 172.21.15.152:47512) 2024-04-20T11:30:57.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.415+0000 7f1373e00640 1 -- 172.21.15.152:0/3464607824 learned_addr learned my addr 172.21.15.152:0/3464607824 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:30:57.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.415+0000 7f136b400640 1 --2- 172.21.15.152:0/3464607824 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1374071cd0 0x7f137415c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:57.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.415+0000 7f1373e00640 1 -- 172.21.15.152:0/3464607824 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1374162e50 msgr2=0x7f137415c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:57.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.415+0000 7f1373e00640 1 --2- 172.21.15.152:0/3464607824 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1374162e50 0x7f137415c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:57.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.415+0000 7f1373e00640 1 -- 172.21.15.152:0/3464607824 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1374071cd0 msgr2=0x7f137415c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:57.418 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.416+0000 7f1373e00640 1 --2- 172.21.15.152:0/3464607824 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1374071cd0 0x7f137415c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:57.418 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.416+0000 7f1373e00640 1 -- 172.21.15.152:0/3464607824 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1374148960 con 0x7f137406ea20 2024-04-20T11:30:57.418 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.416+0000 7f136b400640 1 --2- 172.21.15.152:0/3464607824 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1374071cd0 0x7f137415c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:30:57.418 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.416+0000 7f1373e00640 1 --2- 172.21.15.152:0/3464607824 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f137406ea20 0x7f1374148420 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f136400e930 tx=0x7f136400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:57.418 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.416+0000 7f1370c00640 1 -- 172.21.15.152:0/3464607824 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1364017070 con 0x7f137406ea20 2024-04-20T11:30:57.418 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.416+0000 7f1370c00640 1 -- 172.21.15.152:0/3464607824 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f136400ca00 con 0x7f137406ea20 2024-04-20T11:30:57.418 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.416+0000 7f1379400640 1 -- 172.21.15.152:0/3464607824 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f137415eef0 con 0x7f137406ea20 2024-04-20T11:30:57.418 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.416+0000 7f1370c00640 1 -- 172.21.15.152:0/3464607824 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f136400cdf0 con 0x7f137406ea20 2024-04-20T11:30:57.418 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.416+0000 7f1379400640 1 -- 172.21.15.152:0/3464607824 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f137415f4b0 con 0x7f137406ea20 2024-04-20T11:30:57.419 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.417+0000 7f1379400640 1 -- 172.21.15.152:0/3464607824 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1374075e60 con 0x7f137406ea20 2024-04-20T11:30:57.419 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.418+0000 7f1370c00640 1 -- 172.21.15.152:0/3464607824 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 9) ==== 91193+0+0 (secure 0 0 0) 0x7f136401a070 con 0x7f137406ea20 2024-04-20T11:30:57.420 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.419+0000 7f1370c00640 1 --2- 172.21.15.152:0/3464607824 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f135006caa0 0x7f135006ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:57.420 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.419+0000 7f1370c00640 1 -- 172.21.15.152:0/3464607824 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 4857+0+0 (secure 0 0 0) 0x7f136408bfe0 con 0x7f137406ea20 2024-04-20T11:30:57.420 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.420+0000 7f136b400640 1 --2- 172.21.15.152:0/3464607824 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f135006caa0 0x7f135006ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:57.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.420+0000 7f136b400640 1 --2- 172.21.15.152:0/3464607824 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f135006caa0 0x7f135006ef60 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f13741662e0 tx=0x7f135c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:57.424 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.424+0000 7f1370c00640 1 -- 172.21.15.152:0/3464607824 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f136405a540 con 0x7f137406ea20 2024-04-20T11:30:57.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.624+0000 7f1379400640 1 -- 172.21.15.152:0/3464607824 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f1374162000 con 0x7f137406ea20 2024-04-20T11:30:57.625 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.625+0000 7f1370c00640 1 -- 172.21.15.152:0/3464607824 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v56) ==== 84+0+65 (secure 0 0 0) 0x7f1364059e40 con 0x7f137406ea20 2024-04-20T11:30:57.626 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:30:57.626 INFO:teuthology.orchestra.run.smithi152.stdout:[ 2024-04-20T11:30:57.626 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:30:57.626 INFO:teuthology.orchestra.run.smithi152.stdout: "poolnum": 1, 2024-04-20T11:30:57.626 INFO:teuthology.orchestra.run.smithi152.stdout: "poolname": ".mgr" 2024-04-20T11:30:57.626 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:30:57.626 INFO:teuthology.orchestra.run.smithi152.stdout:] 2024-04-20T11:30:57.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.627+0000 7f1379400640 1 -- 172.21.15.152:0/3464607824 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f135006caa0 msgr2=0x7f135006ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:57.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.627+0000 7f1379400640 1 --2- 172.21.15.152:0/3464607824 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f135006caa0 0x7f135006ef60 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f13741662e0 tx=0x7f135c009040 comp rx=0 tx=0).stop 2024-04-20T11:30:57.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.628+0000 7f1379400640 1 -- 172.21.15.152:0/3464607824 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f137406ea20 msgr2=0x7f1374148420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:57.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.628+0000 7f1379400640 1 --2- 172.21.15.152:0/3464607824 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f137406ea20 0x7f1374148420 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f136400e930 tx=0x7f136400edf0 comp rx=0 tx=0).stop 2024-04-20T11:30:57.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.628+0000 7f1379400640 1 -- 172.21.15.152:0/3464607824 shutdown_connections 2024-04-20T11:30:57.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.628+0000 7f1379400640 1 --2- 172.21.15.152:0/3464607824 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f135006caa0 0x7f135006ef60 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:57.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.628+0000 7f1379400640 1 --2- 172.21.15.152:0/3464607824 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1374162e50 0x7f137415c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:57.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.628+0000 7f1379400640 1 --2- 172.21.15.152:0/3464607824 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1374071cd0 0x7f137415c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:57.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.628+0000 7f1379400640 1 --2- 172.21.15.152:0/3464607824 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f137406ea20 0x7f1374148420 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:57.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.628+0000 7f1379400640 1 -- 172.21.15.152:0/3464607824 >> 172.21.15.152:0/3464607824 conn(0x7f1374067ae0 msgr2=0x7f13740cc580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:57.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.628+0000 7f1379400640 1 -- 172.21.15.152:0/3464607824 shutdown_connections 2024-04-20T11:30:57.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.628+0000 7f1379400640 1 -- 172.21.15.152:0/3464607824 wait complete. 2024-04-20T11:30:57.648 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-20T11:30:57.649 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-20T11:30:57.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.852+0000 7f94b3e00640 1 Processor -- start 2024-04-20T11:30:57.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.852+0000 7f94b3e00640 1 -- start start 2024-04-20T11:30:57.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.854+0000 7f94b3e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f94b415e6c0 0x7f94b406dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:57.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.854+0000 7f94b3e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94b406f550 0x7f94b40d05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:57.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.854+0000 7f94b3e00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94b406ec10 con 0x7f94b40d0b30 2024-04-20T11:30:57.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.854+0000 7f94b3e00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f94b40b50f0 con 0x7f94b415e6c0 2024-04-20T11:30:57.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.854+0000 7f94b3e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f94b406e370 con 0x7f94b406f550 2024-04-20T11:30:57.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.854+0000 7f94b3400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f94b40d0b30 0x7f94b40b36d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:50898/0 (socket says 172.21.15.152:50898) 2024-04-20T11:30:57.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.854+0000 7f94b3400640 1 -- 172.21.15.152:0/1310952817 learned_addr learned my addr 172.21.15.152:0/1310952817 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:30:57.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.855+0000 7f94b2a00640 1 --2- 172.21.15.152:0/1310952817 >> v2:172.21.15.175:3300/0 conn(0x7f94b415e6c0 0x7f94b406dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:57.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.855+0000 7f94aa000640 1 --2- 172.21.15.152:0/1310952817 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94b406f550 0x7f94b40d05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:57.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.855+0000 7f94b2000640 1 -- 172.21.15.152:0/1310952817 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4202546609 0 0) 0x7f94b406ec10 con 0x7f94b40d0b30 2024-04-20T11:30:57.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.855+0000 7f94b2000640 1 -- 172.21.15.152:0/1310952817 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9498003520 con 0x7f94b40d0b30 2024-04-20T11:30:57.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.855+0000 7f94b2000640 1 -- 172.21.15.152:0/1310952817 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1647401085 0 0) 0x7f9498003520 con 0x7f94b40d0b30 2024-04-20T11:30:57.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.855+0000 7f94b2000640 1 -- 172.21.15.152:0/1310952817 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94b406f550 msgr2=0x7f94b40d05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:57.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.855+0000 7f94b2000640 1 --2- 172.21.15.152:0/1310952817 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94b406f550 0x7f94b40d05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:57.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.855+0000 7f94b2000640 1 -- 172.21.15.152:0/1310952817 >> v2:172.21.15.175:3300/0 conn(0x7f94b415e6c0 msgr2=0x7f94b406dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:57.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.855+0000 7f94b2000640 1 --2- 172.21.15.152:0/1310952817 >> v2:172.21.15.175:3300/0 conn(0x7f94b415e6c0 0x7f94b406dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:57.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.855+0000 7f94b2000640 1 -- 172.21.15.152:0/1310952817 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94b406e650 con 0x7f94b40d0b30 2024-04-20T11:30:57.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.855+0000 7f94b2000640 1 -- 172.21.15.152:0/1310952817 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f94ac0019f0 con 0x7f94b40d0b30 2024-04-20T11:30:57.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.855+0000 7f94b2000640 1 -- 172.21.15.152:0/1310952817 >> v1:172.21.15.152:6789/0 conn(0x7f94b40d0b30 legacy=0x7f94b40b36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:57.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.855+0000 7f94b2000640 1 --2- 172.21.15.152:0/1310952817 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94980039b0 0x7f9498003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:57.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.855+0000 7f94b2000640 1 --2- 172.21.15.152:0/1310952817 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94b406f550 0x7f9498004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:57.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.855+0000 7f94b2000640 1 --2- 172.21.15.152:0/1310952817 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9498004d40 0x7f94980051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:57.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.855+0000 7f94b2000640 1 -- 172.21.15.152:0/1310952817 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f94b40b50f0 con 0x7f9498004d40 2024-04-20T11:30:57.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.855+0000 7f94b2000640 1 -- 172.21.15.152:0/1310952817 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f94b406e370 con 0x7f94980039b0 2024-04-20T11:30:57.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.855+0000 7f94b2000640 1 -- 172.21.15.152:0/1310952817 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f949800ba40 con 0x7f94b406f550 2024-04-20T11:30:57.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.856+0000 7f94b3400640 1 --2- 172.21.15.152:0/1310952817 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9498004d40 0x7f94980051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:57.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.856+0000 7f94aa000640 1 --2- 172.21.15.152:0/1310952817 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94b406f550 0x7f9498004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:57.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.856+0000 7f94b3400640 1 -- 172.21.15.152:0/1310952817 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94b406f550 msgr2=0x7f9498004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:57.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.856+0000 7f94b3400640 1 --2- 172.21.15.152:0/1310952817 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94b406f550 0x7f9498004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:57.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.856+0000 7f94b3400640 1 -- 172.21.15.152:0/1310952817 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94980039b0 msgr2=0x7f9498003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:57.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.856+0000 7f94b3400640 1 --2- 172.21.15.152:0/1310952817 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94980039b0 0x7f9498003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:57.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.856+0000 7f94b3400640 1 -- 172.21.15.152:0/1310952817 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9498003520 con 0x7f9498004d40 2024-04-20T11:30:57.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.857+0000 7f94b3400640 1 --2- 172.21.15.152:0/1310952817 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9498004d40 0x7f94980051c0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f94ac00a7a0 tx=0x7f9498005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4397b7e52cf6a320 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:57.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.857+0000 7f94b2000640 1 -- 172.21.15.152:0/1310952817 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94ac00a3f0 con 0x7f9498004d40 2024-04-20T11:30:57.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.857+0000 7f94b2000640 1 -- 172.21.15.152:0/1310952817 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94ac034630 con 0x7f9498004d40 2024-04-20T11:30:57.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.857+0000 7f94b2000640 1 -- 172.21.15.152:0/1310952817 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94ac034a40 con 0x7f9498004d40 2024-04-20T11:30:57.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.858+0000 7f94b3e00640 1 -- 172.21.15.152:0/1310952817 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9498004d40 msgr2=0x7f94980051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:57.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.858+0000 7f94b3e00640 1 --2- 172.21.15.152:0/1310952817 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9498004d40 0x7f94980051c0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f94ac00a7a0 tx=0x7f9498005830 comp rx=0 tx=0).stop 2024-04-20T11:30:57.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.858+0000 7f94b3400640 1 -- 172.21.15.152:0/1310952817 reap_dead start 2024-04-20T11:30:57.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.858+0000 7f94b3e00640 1 -- 172.21.15.152:0/1310952817 shutdown_connections 2024-04-20T11:30:57.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.858+0000 7f94b3e00640 1 -- 172.21.15.152:0/1310952817 >> 172.21.15.152:0/1310952817 conn(0x7f94b40b21e0 msgr2=0x7f94b40b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:57.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.858+0000 7f94b3e00640 1 -- 172.21.15.152:0/1310952817 shutdown_connections 2024-04-20T11:30:57.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.858+0000 7f94b3e00640 1 -- 172.21.15.152:0/1310952817 wait complete. 2024-04-20T11:30:57.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.860+0000 7f94b3e00640 1 Processor -- start 2024-04-20T11:30:57.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.860+0000 7f94b3e00640 1 -- start start 2024-04-20T11:30:57.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.861+0000 7f94b3e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9498004d40 0x7f94b41f6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:57.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.861+0000 7f94b3e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94b40cef00 0x7f94b41f4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:57.863 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.861+0000 7f94b3e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94b41f4ba0 0x7f94b41f5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:57.863 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.861+0000 7f94b3e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f94b415eaa0 con 0x7f9498004d40 2024-04-20T11:30:57.863 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.861+0000 7f94b3e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f94b40ad140 con 0x7f94b41f4ba0 2024-04-20T11:30:57.863 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.861+0000 7f94b3e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f94b40ae670 con 0x7f94b40cef00 2024-04-20T11:30:57.863 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.861+0000 7f94b2a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9498004d40 0x7f94b41f6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:57.863 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.861+0000 7f94b2a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9498004d40 0x7f94b41f6360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:47532/0 (socket says 172.21.15.152:47532) 2024-04-20T11:30:57.863 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.861+0000 7f94b2a00640 1 -- 172.21.15.152:0/200353628 learned_addr learned my addr 172.21.15.152:0/200353628 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:30:57.863 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.862+0000 7f94aa000640 1 --2- 172.21.15.152:0/200353628 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94b40cef00 0x7f94b41f4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:57.863 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.862+0000 7f94b3400640 1 --2- 172.21.15.152:0/200353628 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94b41f4ba0 0x7f94b41f5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:57.863 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.862+0000 7f94b2a00640 1 -- 172.21.15.152:0/200353628 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94b40cef00 msgr2=0x7f94b41f4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:57.863 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.862+0000 7f94b2a00640 1 --2- 172.21.15.152:0/200353628 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94b40cef00 0x7f94b41f4660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:57.863 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.862+0000 7f94b2a00640 1 -- 172.21.15.152:0/200353628 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94b41f4ba0 msgr2=0x7f94b41f5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:57.863 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.862+0000 7f94b2a00640 1 --2- 172.21.15.152:0/200353628 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94b41f4ba0 0x7f94b41f5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:57.863 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.862+0000 7f94b2a00640 1 -- 172.21.15.152:0/200353628 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94b41f6bb0 con 0x7f9498004d40 2024-04-20T11:30:57.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.862+0000 7f94b3400640 1 --2- 172.21.15.152:0/200353628 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94b41f4ba0 0x7f94b41f5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:30:57.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.862+0000 7f94b2a00640 1 --2- 172.21.15.152:0/200353628 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9498004d40 0x7f94b41f6360 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f94a400e7c0 tx=0x7f94a400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:57.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.862+0000 7f94ab400640 1 -- 172.21.15.152:0/200353628 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a4017070 con 0x7f9498004d40 2024-04-20T11:30:57.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.862+0000 7f94ab400640 1 -- 172.21.15.152:0/200353628 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94a400c940 con 0x7f9498004d40 2024-04-20T11:30:57.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.862+0000 7f94ab400640 1 -- 172.21.15.152:0/200353628 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a400cd30 con 0x7f9498004d40 2024-04-20T11:30:57.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.862+0000 7f94b3e00640 1 -- 172.21.15.152:0/200353628 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94b41f6e90 con 0x7f9498004d40 2024-04-20T11:30:57.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.863+0000 7f94b3e00640 1 -- 172.21.15.152:0/200353628 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94b41f7450 con 0x7f9498004d40 2024-04-20T11:30:57.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.864+0000 7f94b3e00640 1 -- 172.21.15.152:0/200353628 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94b4075e60 con 0x7f9498004d40 2024-04-20T11:30:57.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.865+0000 7f94ab400640 1 -- 172.21.15.152:0/200353628 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 9) ==== 91193+0+0 (secure 0 0 0) 0x7f94a401a070 con 0x7f9498004d40 2024-04-20T11:30:57.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.866+0000 7f94ab400640 1 --2- 172.21.15.152:0/200353628 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f949006cad0 0x7f949006ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:57.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.866+0000 7f94ab400640 1 -- 172.21.15.152:0/200353628 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 4857+0+0 (secure 0 0 0) 0x7f94a408bdd0 con 0x7f9498004d40 2024-04-20T11:30:57.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.866+0000 7f94aa000640 1 --2- 172.21.15.152:0/200353628 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f949006cad0 0x7f949006ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:57.867 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.867+0000 7f94aa000640 1 --2- 172.21.15.152:0/200353628 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f949006cad0 0x7f949006ef90 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f94b415ba80 tx=0x7f949c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:57.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:57.870+0000 7f94ab400640 1 -- 172.21.15.152:0/200353628 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f94a405a230 con 0x7f9498004d40 2024-04-20T11:30:58.070 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:58.069+0000 7f94b3e00640 1 -- 172.21.15.152:0/200353628 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f94b40b3e50 con 0x7f9498004d40 2024-04-20T11:30:58.826 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:58.825+0000 7f94ab400640 1 -- 172.21.15.152:0/200353628 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v57) ==== 138+0+0 (secure 0 0 0) 0x7f94a4059b30 con 0x7f9498004d40 2024-04-20T11:30:58.826 INFO:teuthology.orchestra.run.smithi152.stderr:pool 'cephfs_metadata' created 2024-04-20T11:30:58.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:58.829+0000 7f94b3e00640 1 -- 172.21.15.152:0/200353628 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f949006cad0 msgr2=0x7f949006ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:58.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:58.829+0000 7f94b3e00640 1 --2- 172.21.15.152:0/200353628 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f949006cad0 0x7f949006ef90 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f94b415ba80 tx=0x7f949c008040 comp rx=0 tx=0).stop 2024-04-20T11:30:58.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:58.830+0000 7f94b3e00640 1 -- 172.21.15.152:0/200353628 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9498004d40 msgr2=0x7f94b41f6360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:58.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:58.830+0000 7f94b3e00640 1 --2- 172.21.15.152:0/200353628 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9498004d40 0x7f94b41f6360 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f94a400e7c0 tx=0x7f94a400ec80 comp rx=0 tx=0).stop 2024-04-20T11:30:58.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:58.830+0000 7f94b3e00640 1 -- 172.21.15.152:0/200353628 shutdown_connections 2024-04-20T11:30:58.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:58.830+0000 7f94b3e00640 1 --2- 172.21.15.152:0/200353628 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f949006cad0 0x7f949006ef90 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:58.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:58.830+0000 7f94b3e00640 1 --2- 172.21.15.152:0/200353628 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94b41f4ba0 0x7f94b41f5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:58.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:58.830+0000 7f94b3e00640 1 --2- 172.21.15.152:0/200353628 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94b40cef00 0x7f94b41f4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:58.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:58.830+0000 7f94b3e00640 1 --2- 172.21.15.152:0/200353628 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9498004d40 0x7f94b41f6360 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:58.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:58.830+0000 7f94b3e00640 1 -- 172.21.15.152:0/200353628 >> 172.21.15.152:0/200353628 conn(0x7f94b40b21e0 msgr2=0x7f94b40b0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:58.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:58.830+0000 7f94b3e00640 1 -- 172.21.15.152:0/200353628 shutdown_connections 2024-04-20T11:30:58.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:58.830+0000 7f94b3e00640 1 -- 172.21.15.152:0/200353628 wait complete. 2024-04-20T11:30:58.860 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-20T11:30:59.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:59.046+0000 7f42ff200640 1 Processor -- start 2024-04-20T11:30:59.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:59.047+0000 7f42ff200640 1 -- start start 2024-04-20T11:30:59.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:59.048+0000 7f42ff200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f4300159040 0x7f43000cf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:59.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:59.048+0000 7f42ff200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4300162c40 0x7f43000ce140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:59.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:59.048+0000 7f42ff200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4300159420 con 0x7f430006f6a0 2024-04-20T11:30:59.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:59.048+0000 7f42ff200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f43000ce680 con 0x7f4300159040 2024-04-20T11:30:59.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:59.048+0000 7f42ff200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f430006fb30 con 0x7f4300162c40 2024-04-20T11:30:59.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:59.049+0000 7f42fe800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4300162c40 0x7f43000ce140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:59.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:59.049+0000 7f42fe800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4300162c40 0x7f43000ce140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.152:33374/0 (socket says 172.21.15.152:33374) 2024-04-20T11:30:59.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:59.049+0000 7f42fe800640 1 -- 172.21.15.152:0/1695215137 learned_addr learned my addr 172.21.15.152:0/1695215137 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:30:59.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:59.049+0000 7f42fde00640 1 --2- 172.21.15.152:0/1695215137 >> v2:172.21.15.175:3300/0 conn(0x7f4300159040 0x7f43000cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:59.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:59.050+0000 7f42fe800640 1 -- 172.21.15.152:0/1695215137 >> v2:172.21.15.175:3300/0 conn(0x7f4300159040 msgr2=0x7f43000cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:59.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:59.050+0000 7f42fe800640 1 --2- 172.21.15.152:0/1695215137 >> v2:172.21.15.175:3300/0 conn(0x7f4300159040 0x7f43000cf950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:59.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:59.050+0000 7f42fe800640 1 -- 172.21.15.152:0/1695215137 >> v1:172.21.15.175:6789/0 conn(0x7f430006f6a0 legacy=0x7f4300162530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:59.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:59.050+0000 7f42fe800640 1 -- 172.21.15.152:0/1695215137 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43000ce8b0 con 0x7f4300162c40 2024-04-20T11:30:59.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:59.050+0000 7f42fde00640 1 --2- 172.21.15.152:0/1695215137 >> v2:172.21.15.175:3300/0 conn(0x7f4300159040 0x7f43000cf950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:30:59.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:59.051+0000 7f42fe800640 1 --2- 172.21.15.152:0/1695215137 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4300162c40 0x7f43000ce140 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f42f8009e50 tx=0x7f42f802bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b6dfff897c97b4d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:59.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:59.052+0000 7f42fd400640 1 -- 172.21.15.152:0/1695215137 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f802cb00 con 0x7f4300162c40 2024-04-20T11:30:59.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:59.052+0000 7f42fd400640 1 -- 172.21.15.152:0/1695215137 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42f802cca0 con 0x7f4300162c40 2024-04-20T11:30:59.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:59.052+0000 7f42fd400640 1 -- 172.21.15.152:0/1695215137 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f8034610 con 0x7f4300162c40 2024-04-20T11:30:59.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:59.052+0000 7f42ff200640 1 -- 172.21.15.152:0/1695215137 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4300162c40 msgr2=0x7f43000ce140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:59.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:59.052+0000 7f42ff200640 1 --2- 172.21.15.152:0/1695215137 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4300162c40 0x7f43000ce140 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f42f8009e50 tx=0x7f42f802bf60 comp rx=0 tx=0).stop 2024-04-20T11:30:59.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:59.053+0000 7f42ff200640 1 -- 172.21.15.152:0/1695215137 shutdown_connections 2024-04-20T11:30:59.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:59.053+0000 7f42ff200640 1 --2- 172.21.15.152:0/1695215137 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4300162c40 0x7f43000ce140 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:59.054 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:59.053+0000 7f42ff200640 1 --2- 172.21.15.152:0/1695215137 >> v2:172.21.15.175:3300/0 conn(0x7f4300159040 0x7f43000cf950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:59.054 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:59.053+0000 7f42ff200640 1 -- 172.21.15.152:0/1695215137 >> 172.21.15.152:0/1695215137 conn(0x7f43000d0260 msgr2=0x7f43000d0670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:59.054 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:59.053+0000 7f42ff200640 1 -- 172.21.15.152:0/1695215137 shutdown_connections 2024-04-20T11:30:59.054 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:59.053+0000 7f42ff200640 1 -- 172.21.15.152:0/1695215137 wait complete. 2024-04-20T11:30:59.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:59.055+0000 7f42ff200640 1 Processor -- start 2024-04-20T11:30:59.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:59.055+0000 7f42ff200640 1 -- start start 2024-04-20T11:30:59.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:59.056+0000 7f42ff200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f430006f6a0 0x7f4300148530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:59.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:59.056+0000 7f42ff200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4300159040 0x7f4300072cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:59.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:59.056+0000 7f42ff200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4300162c40 0x7f43000731f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:59.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:59.056+0000 7f42ff200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f43000cf150 con 0x7f430006f6a0 2024-04-20T11:30:59.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:59.056+0000 7f42ff200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f43000cee70 con 0x7f4300162c40 2024-04-20T11:30:59.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:59.056+0000 7f42ff200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f43000cf430 con 0x7f4300159040 2024-04-20T11:30:59.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:59.056+0000 7f42fde00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f430006f6a0 0x7f4300148530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:59.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:59.056+0000 7f42fde00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f430006f6a0 0x7f4300148530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:47540/0 (socket says 172.21.15.152:47540) 2024-04-20T11:30:59.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:59.056+0000 7f42fde00640 1 -- 172.21.15.152:0/3159214423 learned_addr learned my addr 172.21.15.152:0/3159214423 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:30:59.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:59.056+0000 7f42fe800640 1 --2- 172.21.15.152:0/3159214423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4300162c40 0x7f43000731f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:59.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:59.056+0000 7f42f7e00640 1 --2- 172.21.15.152:0/3159214423 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4300159040 0x7f4300072cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:59.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:59.057+0000 7f42fde00640 1 -- 172.21.15.152:0/3159214423 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4300159040 msgr2=0x7f4300072cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:59.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:59.057+0000 7f42fde00640 1 --2- 172.21.15.152:0/3159214423 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4300159040 0x7f4300072cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:59.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:59.057+0000 7f42fde00640 1 -- 172.21.15.152:0/3159214423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4300162c40 msgr2=0x7f43000731f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:59.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:59.057+0000 7f42fde00640 1 --2- 172.21.15.152:0/3159214423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4300162c40 0x7f43000731f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:59.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:59.057+0000 7f42fde00640 1 -- 172.21.15.152:0/3159214423 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4300155240 con 0x7f430006f6a0 2024-04-20T11:30:59.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:59.057+0000 7f42f7e00640 1 --2- 172.21.15.152:0/3159214423 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4300159040 0x7f4300072cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:30:59.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:59.057+0000 7f42fde00640 1 --2- 172.21.15.152:0/3159214423 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f430006f6a0 0x7f4300148530 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f42f000d800 tx=0x7f42f000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:59.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:59.057+0000 7f42f6000640 1 -- 172.21.15.152:0/3159214423 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f0004030 con 0x7f430006f6a0 2024-04-20T11:30:59.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:59.057+0000 7f42f6000640 1 -- 172.21.15.152:0/3159214423 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42f00041d0 con 0x7f430006f6a0 2024-04-20T11:30:59.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:59.057+0000 7f42f6000640 1 -- 172.21.15.152:0/3159214423 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f00096a0 con 0x7f430006f6a0 2024-04-20T11:30:59.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:59.057+0000 7f42ff200640 1 -- 172.21.15.152:0/3159214423 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4300155520 con 0x7f430006f6a0 2024-04-20T11:30:59.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:59.057+0000 7f42ff200640 1 -- 172.21.15.152:0/3159214423 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4300155ae0 con 0x7f430006f6a0 2024-04-20T11:30:59.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:59.059+0000 7f42ff200640 1 -- 172.21.15.152:0/3159214423 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f430015eaa0 con 0x7f430006f6a0 2024-04-20T11:30:59.061 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:59.060+0000 7f42f6000640 1 -- 172.21.15.152:0/3159214423 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 9) ==== 91193+0+0 (secure 0 0 0) 0x7f42f0009840 con 0x7f430006f6a0 2024-04-20T11:30:59.061 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:59.061+0000 7f42f6000640 1 --2- 172.21.15.152:0/3159214423 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f42dc06caa0 0x7f42dc06ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:59.061 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:59.061+0000 7f42f6000640 1 -- 172.21.15.152:0/3159214423 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5227+0+0 (secure 0 0 0) 0x7f42f008ca50 con 0x7f430006f6a0 2024-04-20T11:30:59.061 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:59.061+0000 7f42f7e00640 1 --2- 172.21.15.152:0/3159214423 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f42dc06caa0 0x7f42dc06ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:59.062 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:59.062+0000 7f42f7e00640 1 --2- 172.21.15.152:0/3159214423 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f42dc06caa0 0x7f42dc06ef60 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f43001567a0 tx=0x7f42e8009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:59.066 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:59.065+0000 7f42f6000640 1 -- 172.21.15.152:0/3159214423 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f430015eaa0 con 0x7f430006f6a0 2024-04-20T11:30:59.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:59.263+0000 7f42ff200640 1 -- 172.21.15.152:0/3159214423 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f4300075e60 con 0x7f430006f6a0 2024-04-20T11:30:59.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:59.840+0000 7f42f6000640 1 -- 172.21.15.152:0/3159214423 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v58) ==== 170+0+0 (secure 0 0 0) 0x7f42f001c360 con 0x7f430006f6a0 2024-04-20T11:30:59.841 INFO:teuthology.orchestra.run.smithi152.stderr:pool 'cephfs_data' created 2024-04-20T11:30:59.845 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:59.844+0000 7f42ff200640 1 -- 172.21.15.152:0/3159214423 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f42dc06caa0 msgr2=0x7f42dc06ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:59.845 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:59.844+0000 7f42ff200640 1 --2- 172.21.15.152:0/3159214423 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f42dc06caa0 0x7f42dc06ef60 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f43001567a0 tx=0x7f42e8009040 comp rx=0 tx=0).stop 2024-04-20T11:30:59.845 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:59.844+0000 7f42ff200640 1 -- 172.21.15.152:0/3159214423 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f430006f6a0 msgr2=0x7f4300148530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:59.845 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:59.844+0000 7f42ff200640 1 --2- 172.21.15.152:0/3159214423 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f430006f6a0 0x7f4300148530 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f42f000d800 tx=0x7f42f000dcc0 comp rx=0 tx=0).stop 2024-04-20T11:30:59.846 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:59.845+0000 7f42ff200640 1 -- 172.21.15.152:0/3159214423 shutdown_connections 2024-04-20T11:30:59.846 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:59.845+0000 7f42ff200640 1 --2- 172.21.15.152:0/3159214423 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f42dc06caa0 0x7f42dc06ef60 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:59.846 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:59.845+0000 7f42ff200640 1 --2- 172.21.15.152:0/3159214423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4300162c40 0x7f43000731f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:59.846 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:59.845+0000 7f42ff200640 1 --2- 172.21.15.152:0/3159214423 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4300159040 0x7f4300072cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:59.846 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:59.845+0000 7f42ff200640 1 --2- 172.21.15.152:0/3159214423 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f430006f6a0 0x7f4300148530 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:59.846 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:59.845+0000 7f42ff200640 1 -- 172.21.15.152:0/3159214423 >> 172.21.15.152:0/3159214423 conn(0x7f43000d0260 msgr2=0x7f43000adfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:59.846 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:59.845+0000 7f42ff200640 1 -- 172.21.15.152:0/3159214423 shutdown_connections 2024-04-20T11:30:59.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:30:59.846+0000 7f42ff200640 1 -- 172.21.15.152:0/3159214423 wait complete. 2024-04-20T11:30:59.885 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-20T11:31:00.063 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:00.062+0000 7fd062a00640 1 Processor -- start 2024-04-20T11:31:00.063 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:00.062+0000 7fd062a00640 1 -- start start 2024-04-20T11:31:00.065 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:00.064+0000 7fd062a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fd05c06f720 0x7fd05c0b2170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:00.065 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:00.064+0000 7fd062a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd05c0b26b0 0x7fd05c06e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:00.065 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:00.064+0000 7fd062a00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd05c06fc30 con 0x7fd05c162bc0 2024-04-20T11:31:00.065 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:00.064+0000 7fd062a00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fd05c0ccb40 con 0x7fd05c06f720 2024-04-20T11:31:00.065 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:00.064+0000 7fd062a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd05c0b3610 con 0x7fd05c0b26b0 2024-04-20T11:31:00.065 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:00.064+0000 7fd053400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fd05c06f720 0x7fd05c0b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:00.065 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:00.064+0000 7fd053400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fd05c06f720 0x7fd05c0b2170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:47544/0 (socket says 172.21.15.152:47544) 2024-04-20T11:31:00.065 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:00.064+0000 7fd053400640 1 -- 172.21.15.152:0/3096477378 learned_addr learned my addr 172.21.15.152:0/3096477378 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:00.065 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:00.064+0000 7fd053400640 1 -- 172.21.15.152:0/3096477378 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd05c0b26b0 msgr2=0x7fd05c06e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:00.066 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:00.064+0000 7fd053400640 1 --2- 172.21.15.152:0/3096477378 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd05c0b26b0 0x7fd05c06e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:00.066 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:00.064+0000 7fd053400640 1 -- 172.21.15.152:0/3096477378 >> v1:172.21.15.175:6789/0 conn(0x7fd05c162bc0 legacy=0x7fd05c167240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:00.066 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:00.064+0000 7fd053400640 1 -- 172.21.15.152:0/3096477378 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd05c0b4090 con 0x7fd05c06f720 2024-04-20T11:31:00.066 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:00.066+0000 7fd053400640 1 --2- 172.21.15.152:0/3096477378 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd05c06f720 0x7fd05c0b2170 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fd044009870 tx=0x7fd04402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baf2c3cb44608847 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:00.067 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:00.066+0000 7fd05b400640 1 -- 172.21.15.152:0/3096477378 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd044035020 con 0x7fd05c06f720 2024-04-20T11:31:00.067 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:00.066+0000 7fd05b400640 1 -- 172.21.15.152:0/3096477378 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd044031640 con 0x7fd05c06f720 2024-04-20T11:31:00.067 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:00.066+0000 7fd05b400640 1 -- 172.21.15.152:0/3096477378 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd04402d040 con 0x7fd05c06f720 2024-04-20T11:31:00.068 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:00.067+0000 7fd062a00640 1 -- 172.21.15.152:0/3096477378 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd05c06f720 msgr2=0x7fd05c0b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:00.068 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:00.067+0000 7fd062a00640 1 --2- 172.21.15.152:0/3096477378 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd05c06f720 0x7fd05c0b2170 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fd044009870 tx=0x7fd04402b8f0 comp rx=0 tx=0).stop 2024-04-20T11:31:00.068 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:00.067+0000 7fd062a00640 1 -- 172.21.15.152:0/3096477378 shutdown_connections 2024-04-20T11:31:00.068 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:00.067+0000 7fd062a00640 1 --2- 172.21.15.152:0/3096477378 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd05c0b26b0 0x7fd05c06e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:00.068 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:00.067+0000 7fd062a00640 1 --2- 172.21.15.152:0/3096477378 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd05c06f720 0x7fd05c0b2170 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:00.068 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:00.067+0000 7fd062a00640 1 -- 172.21.15.152:0/3096477378 >> 172.21.15.152:0/3096477378 conn(0x7fd05c06b980 msgr2=0x7fd05c06bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:00.068 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:00.067+0000 7fd062a00640 1 -- 172.21.15.152:0/3096477378 shutdown_connections 2024-04-20T11:31:00.068 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:00.067+0000 7fd062a00640 1 -- 172.21.15.152:0/3096477378 wait complete. 2024-04-20T11:31:00.069 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:00.069+0000 7fd062a00640 1 Processor -- start 2024-04-20T11:31:00.069 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:00.069+0000 7fd062a00640 1 -- start start 2024-04-20T11:31:00.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:00.070+0000 7fd062a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd05c06f720 0x7fd05c158cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:00.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:00.070+0000 7fd062a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd05c0b26b0 0x7fd05c1fa160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:00.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:00.070+0000 7fd062a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd05c162bc0 0x7fd05c1545b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:00.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:00.070+0000 7fd062a00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fd05c162fa0 con 0x7fd05c162bc0 2024-04-20T11:31:00.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:00.070+0000 7fd062a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd05c0b4650 con 0x7fd05c06f720 2024-04-20T11:31:00.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:00.070+0000 7fd062a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd05c163120 con 0x7fd05c0b26b0 2024-04-20T11:31:00.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:00.070+0000 7fd060a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd05c162bc0 0x7fd05c1545b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:00.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:00.070+0000 7fd060a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd05c162bc0 0x7fd05c1545b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:47558/0 (socket says 172.21.15.152:47558) 2024-04-20T11:31:00.073 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:00.070+0000 7fd060a00640 1 -- 172.21.15.152:0/1158013076 learned_addr learned my addr 172.21.15.152:0/1158013076 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:00.073 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:00.070+0000 7fd053400640 1 --2- 172.21.15.152:0/1158013076 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd05c0b26b0 0x7fd05c1fa160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:00.073 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:00.070+0000 7fd05be00640 1 --2- 172.21.15.152:0/1158013076 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd05c06f720 0x7fd05c158cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:00.073 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:00.070+0000 7fd060a00640 1 -- 172.21.15.152:0/1158013076 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd05c0b26b0 msgr2=0x7fd05c1fa160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:00.073 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:00.070+0000 7fd060a00640 1 --2- 172.21.15.152:0/1158013076 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd05c0b26b0 0x7fd05c1fa160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:00.073 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:00.071+0000 7fd060a00640 1 -- 172.21.15.152:0/1158013076 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd05c06f720 msgr2=0x7fd05c158cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:00.073 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:00.071+0000 7fd060a00640 1 --2- 172.21.15.152:0/1158013076 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd05c06f720 0x7fd05c158cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:00.073 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:00.071+0000 7fd060a00640 1 -- 172.21.15.152:0/1158013076 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd05c159760 con 0x7fd05c162bc0 2024-04-20T11:31:00.073 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:00.071+0000 7fd05be00640 1 --2- 172.21.15.152:0/1158013076 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd05c06f720 0x7fd05c158cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:31:00.073 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:00.071+0000 7fd060a00640 1 --2- 172.21.15.152:0/1158013076 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd05c162bc0 0x7fd05c1545b0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fd05400b970 tx=0x7fd05400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:00.073 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:00.071+0000 7fd058c00640 1 -- 172.21.15.152:0/1158013076 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd054019070 con 0x7fd05c162bc0 2024-04-20T11:31:00.073 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:00.071+0000 7fd058c00640 1 -- 172.21.15.152:0/1158013076 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd05400cb60 con 0x7fd05c162bc0 2024-04-20T11:31:00.073 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:00.071+0000 7fd058c00640 1 -- 172.21.15.152:0/1158013076 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd054014440 con 0x7fd05c162bc0 2024-04-20T11:31:00.073 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:00.071+0000 7fd062a00640 1 -- 172.21.15.152:0/1158013076 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd05c159a40 con 0x7fd05c162bc0 2024-04-20T11:31:00.074 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:00.071+0000 7fd062a00640 1 -- 172.21.15.152:0/1158013076 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd05c1fa6a0 con 0x7fd05c162bc0 2024-04-20T11:31:00.074 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:00.073+0000 7fd062a00640 1 -- 172.21.15.152:0/1158013076 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd05c0689f0 con 0x7fd05c162bc0 2024-04-20T11:31:00.074 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:00.074+0000 7fd058c00640 1 -- 172.21.15.152:0/1158013076 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 9) ==== 91193+0+0 (secure 0 0 0) 0x7fd05401c070 con 0x7fd05c162bc0 2024-04-20T11:31:00.075 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:00.075+0000 7fd058c00640 1 --2- 172.21.15.152:0/1158013076 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fd03806caa0 0x7fd03806ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:00.075 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:00.075+0000 7fd058c00640 1 -- 172.21.15.152:0/1158013076 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5609+0+0 (secure 0 0 0) 0x7fd05408bc20 con 0x7fd05c162bc0 2024-04-20T11:31:00.075 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:00.075+0000 7fd05be00640 1 --2- 172.21.15.152:0/1158013076 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fd03806caa0 0x7fd03806ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:00.076 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:00.076+0000 7fd05be00640 1 --2- 172.21.15.152:0/1158013076 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fd03806caa0 0x7fd03806ef60 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fd04c003eb0 tx=0x7fd04c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:00.080 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:00.080+0000 7fd058c00640 1 -- 172.21.15.152:0/1158013076 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fd054059d90 con 0x7fd05c162bc0 2024-04-20T11:31:00.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:00.306+0000 7fd062a00640 1 -- 172.21.15.152:0/1158013076 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fd05c0af800 con 0x7fd05c162bc0 2024-04-20T11:31:00.845 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:00.844+0000 7fd058c00640 1 -- 172.21.15.152:0/1158013076 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v29) ==== 172+0+0 (secure 0 0 0) 0x7fd054059690 con 0x7fd05c162bc0 2024-04-20T11:31:00.845 INFO:teuthology.orchestra.run.smithi152.stderr:new fs with metadata pool 8 and data pool 9 2024-04-20T11:31:00.848 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:00.848+0000 7fd062a00640 1 -- 172.21.15.152:0/1158013076 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fd03806caa0 msgr2=0x7fd03806ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:00.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:00.848+0000 7fd062a00640 1 --2- 172.21.15.152:0/1158013076 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fd03806caa0 0x7fd03806ef60 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fd04c003eb0 tx=0x7fd04c009040 comp rx=0 tx=0).stop 2024-04-20T11:31:00.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:00.848+0000 7fd062a00640 1 -- 172.21.15.152:0/1158013076 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd05c162bc0 msgr2=0x7fd05c1545b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:00.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:00.848+0000 7fd062a00640 1 --2- 172.21.15.152:0/1158013076 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd05c162bc0 0x7fd05c1545b0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fd05400b970 tx=0x7fd05400be30 comp rx=0 tx=0).stop 2024-04-20T11:31:00.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:00.848+0000 7fd062a00640 1 -- 172.21.15.152:0/1158013076 shutdown_connections 2024-04-20T11:31:00.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:00.848+0000 7fd062a00640 1 --2- 172.21.15.152:0/1158013076 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fd03806caa0 0x7fd03806ef60 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:00.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:00.848+0000 7fd062a00640 1 --2- 172.21.15.152:0/1158013076 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd05c162bc0 0x7fd05c1545b0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:00.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:00.848+0000 7fd062a00640 1 --2- 172.21.15.152:0/1158013076 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd05c0b26b0 0x7fd05c1fa160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:00.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:00.848+0000 7fd062a00640 1 --2- 172.21.15.152:0/1158013076 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd05c06f720 0x7fd05c158cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:00.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:00.848+0000 7fd062a00640 1 -- 172.21.15.152:0/1158013076 >> 172.21.15.152:0/1158013076 conn(0x7fd05c06b980 msgr2=0x7fd05c0cd7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:00.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:00.849+0000 7fd062a00640 1 -- 172.21.15.152:0/1158013076 shutdown_connections 2024-04-20T11:31:00.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:00.849+0000 7fd062a00640 1 -- 172.21.15.152:0/1158013076 wait complete. 2024-04-20T11:31:00.892 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:31:01.066 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.064+0000 7f486a000640 1 Processor -- start 2024-04-20T11:31:01.067 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.064+0000 7f486a000640 1 -- start start 2024-04-20T11:31:01.071 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.066+0000 7f486a000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f486406fa50 0x7f486415cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:01.071 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.066+0000 7f486a000640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48640b5770 con 0x7f486415b5e0 2024-04-20T11:31:01.071 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.066+0000 7f486a000640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f486406e060 con 0x7f486406eb40 2024-04-20T11:31:01.071 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.066+0000 7f486a000640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f486406de10 con 0x7f486406fa50 2024-04-20T11:31:01.071 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.066+0000 7f4863e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f486406fa50 0x7f486415cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:01.071 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.066+0000 7f4863e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f486406fa50 0x7f486415cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:47562/0 (socket says 172.21.15.152:47562) 2024-04-20T11:31:01.071 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.066+0000 7f4863e00640 1 -- 172.21.15.152:0/1501865495 learned_addr learned my addr 172.21.15.152:0/1501865495 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:01.071 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.067+0000 7f4863e00640 1 -- 172.21.15.152:0/1501865495 >> v1:172.21.15.175:6789/0 conn(0x7f486406eb40 legacy=0x7f486406f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:01.071 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.067+0000 7f4863e00640 1 -- 172.21.15.152:0/1501865495 >> v1:172.21.15.152:6789/0 conn(0x7f486415b5e0 legacy=0x7f48640b4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:01.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.067+0000 7f4863e00640 1 -- 172.21.15.152:0/1501865495 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f486406e340 con 0x7f486406fa50 2024-04-20T11:31:01.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.068+0000 7f4863e00640 1 --2- 172.21.15.152:0/1501865495 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f486406fa50 0x7f486415cd70 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f486406df90 tx=0x7f485c02e160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db1ebd7091d6c545 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:01.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.068+0000 7f4862a00640 1 -- 172.21.15.152:0/1501865495 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f485c02ed00 con 0x7f486406fa50 2024-04-20T11:31:01.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.068+0000 7f4862a00640 1 -- 172.21.15.152:0/1501865495 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f485c0363f0 con 0x7f486406fa50 2024-04-20T11:31:01.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.068+0000 7f4862a00640 1 -- 172.21.15.152:0/1501865495 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f485c0367c0 con 0x7f486406fa50 2024-04-20T11:31:01.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.069+0000 7f486a000640 1 -- 172.21.15.152:0/1501865495 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f486406fa50 msgr2=0x7f486415cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:01.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.069+0000 7f486a000640 1 --2- 172.21.15.152:0/1501865495 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f486406fa50 0x7f486415cd70 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f486406df90 tx=0x7f485c02e160 comp rx=0 tx=0).stop 2024-04-20T11:31:01.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.069+0000 7f486a000640 1 -- 172.21.15.152:0/1501865495 shutdown_connections 2024-04-20T11:31:01.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.069+0000 7f486a000640 1 --2- 172.21.15.152:0/1501865495 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f486406fa50 0x7f486415cd70 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:01.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.069+0000 7f486a000640 1 -- 172.21.15.152:0/1501865495 >> 172.21.15.152:0/1501865495 conn(0x7f4864067cf0 msgr2=0x7f486406b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:01.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.069+0000 7f486a000640 1 -- 172.21.15.152:0/1501865495 shutdown_connections 2024-04-20T11:31:01.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.069+0000 7f486a000640 1 -- 172.21.15.152:0/1501865495 wait complete. 2024-04-20T11:31:01.073 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.071+0000 7f486a000640 1 Processor -- start 2024-04-20T11:31:01.073 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.071+0000 7f486a000640 1 -- start start 2024-04-20T11:31:01.075 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.072+0000 7f486a000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f486406eb40 0x7f486414dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:01.075 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.072+0000 7f486a000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f486406fa50 0x7f4864154960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:01.075 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.072+0000 7f486a000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f486415b5e0 0x7f48640cf190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:01.075 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.072+0000 7f486a000640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f48640b42d0 con 0x7f486406fa50 2024-04-20T11:31:01.075 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.072+0000 7f486a000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f486415c9d0 con 0x7f486406eb40 2024-04-20T11:31:01.075 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.072+0000 7f486a000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f48640b5360 con 0x7f486415b5e0 2024-04-20T11:31:01.075 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.072+0000 7f485be00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f486406fa50 0x7f4864154960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:01.076 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.072+0000 7f485be00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f486406fa50 0x7f4864154960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:47576/0 (socket says 172.21.15.152:47576) 2024-04-20T11:31:01.076 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.072+0000 7f485be00640 1 -- 172.21.15.152:0/1523523968 learned_addr learned my addr 172.21.15.152:0/1523523968 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:01.076 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.072+0000 7f4863400640 1 --2- 172.21.15.152:0/1523523968 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f486406eb40 0x7f486414dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:01.076 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.072+0000 7f485be00640 1 -- 172.21.15.152:0/1523523968 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f486415b5e0 msgr2=0x7f48640cf190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:01.076 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.072+0000 7f485be00640 1 --2- 172.21.15.152:0/1523523968 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f486415b5e0 0x7f48640cf190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:01.076 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.072+0000 7f485be00640 1 -- 172.21.15.152:0/1523523968 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f486406eb40 msgr2=0x7f486414dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:01.076 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.073+0000 7f485be00640 1 --2- 172.21.15.152:0/1523523968 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f486406eb40 0x7f486414dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:01.076 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.073+0000 7f485be00640 1 -- 172.21.15.152:0/1523523968 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4864155460 con 0x7f486406fa50 2024-04-20T11:31:01.076 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.073+0000 7f4863400640 1 --2- 172.21.15.152:0/1523523968 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f486406eb40 0x7f486414dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:31:01.076 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.073+0000 7f485be00640 1 --2- 172.21.15.152:0/1523523968 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f486406fa50 0x7f4864154960 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f484c00b3f0 tx=0x7f484c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:01.077 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.073+0000 7f485b400640 1 -- 172.21.15.152:0/1523523968 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f484c015020 con 0x7f486406fa50 2024-04-20T11:31:01.077 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.073+0000 7f485b400640 1 -- 172.21.15.152:0/1523523968 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f484c011830 con 0x7f486406fa50 2024-04-20T11:31:01.077 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.073+0000 7f485b400640 1 -- 172.21.15.152:0/1523523968 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f484c011c20 con 0x7f486406fa50 2024-04-20T11:31:01.077 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.073+0000 7f486a000640 1 -- 172.21.15.152:0/1523523968 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4864155740 con 0x7f486406fa50 2024-04-20T11:31:01.077 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.073+0000 7f486a000640 1 -- 172.21.15.152:0/1523523968 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4864155d00 con 0x7f486406fa50 2024-04-20T11:31:01.077 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.075+0000 7f486a000640 1 -- 172.21.15.152:0/1523523968 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48640689f0 con 0x7f486406fa50 2024-04-20T11:31:01.078 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.076+0000 7f485b400640 1 -- 172.21.15.152:0/1523523968 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 9) ==== 91193+0+0 (secure 0 0 0) 0x7f484c018020 con 0x7f486406fa50 2024-04-20T11:31:01.078 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.076+0000 7f485b400640 1 --2- 172.21.15.152:0/1523523968 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f484006caa0 0x7f484006ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:01.078 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.077+0000 7f485b400640 1 -- 172.21.15.152:0/1523523968 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f484c08bab0 con 0x7f486406fa50 2024-04-20T11:31:01.078 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.077+0000 7f4863400640 1 --2- 172.21.15.152:0/1523523968 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f484006caa0 0x7f484006ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:01.079 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.077+0000 7f4863400640 1 --2- 172.21.15.152:0/1523523968 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f484006caa0 0x7f484006ef60 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f4854001680 tx=0x7f4854009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:01.081 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.081+0000 7f485b400640 1 -- 172.21.15.152:0/1523523968 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f484c01f030 con 0x7f486406fa50 2024-04-20T11:31:01.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.280+0000 7f486a000640 1 -- 172.21.15.152:0/1523523968 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f486406cd90 con 0x7f486406fa50 2024-04-20T11:31:01.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.283+0000 7f485b400640 1 -- 172.21.15.152:0/1523523968 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) ==== 74+0+16738 (secure 0 0 0) 0x7f484c0172c0 con 0x7f486406fa50 2024-04-20T11:31:01.284 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:31:01.284 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":59,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:31:00.308769+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:30:58.087822+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-20T11:30:59.281767+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:31:01.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.285+0000 7f486a000640 1 -- 172.21.15.152:0/1523523968 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f484006caa0 msgr2=0x7f484006ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:01.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.285+0000 7f486a000640 1 --2- 172.21.15.152:0/1523523968 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f484006caa0 0x7f484006ef60 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f4854001680 tx=0x7f4854009040 comp rx=0 tx=0).stop 2024-04-20T11:31:01.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.286+0000 7f486a000640 1 -- 172.21.15.152:0/1523523968 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f486406fa50 msgr2=0x7f4864154960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:01.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.286+0000 7f486a000640 1 --2- 172.21.15.152:0/1523523968 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f486406fa50 0x7f4864154960 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f484c00b3f0 tx=0x7f484c00b8b0 comp rx=0 tx=0).stop 2024-04-20T11:31:01.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.286+0000 7f486a000640 1 -- 172.21.15.152:0/1523523968 shutdown_connections 2024-04-20T11:31:01.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.286+0000 7f486a000640 1 --2- 172.21.15.152:0/1523523968 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f484006caa0 0x7f484006ef60 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:01.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.286+0000 7f486a000640 1 --2- 172.21.15.152:0/1523523968 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f486415b5e0 0x7f48640cf190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:01.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.286+0000 7f486a000640 1 --2- 172.21.15.152:0/1523523968 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f486406fa50 0x7f4864154960 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:01.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.286+0000 7f486a000640 1 --2- 172.21.15.152:0/1523523968 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f486406eb40 0x7f486414dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:01.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.286+0000 7f486a000640 1 -- 172.21.15.152:0/1523523968 >> 172.21.15.152:0/1523523968 conn(0x7f4864067cf0 msgr2=0x7f4864071db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:01.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.286+0000 7f486a000640 1 -- 172.21.15.152:0/1523523968 shutdown_connections 2024-04-20T11:31:01.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.286+0000 7f486a000640 1 -- 172.21.15.152:0/1523523968 wait complete. 2024-04-20T11:31:01.304 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:31:01.488 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.488+0000 7f5cbfe00640 1 Processor -- start 2024-04-20T11:31:01.489 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.488+0000 7f5cbfe00640 1 -- start start 2024-04-20T11:31:01.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.489+0000 7f5cbfe00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f5cc00723e0 0x7f5cc00afa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:01.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.490+0000 7f5cbfe00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5cc00aec90 con 0x7f5cc0147410 2024-04-20T11:31:01.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.490+0000 7f5cbfe00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5cc00aef70 con 0x7f5cc006e010 2024-04-20T11:31:01.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.490+0000 7f5cbfe00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f5cc00af250 con 0x7f5cc00723e0 2024-04-20T11:31:01.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.490+0000 7f5cb7e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f5cc00723e0 0x7f5cc00afa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:01.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.490+0000 7f5cb7e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f5cc00723e0 0x7f5cc00afa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:47588/0 (socket says 172.21.15.152:47588) 2024-04-20T11:31:01.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.490+0000 7f5cb7e00640 1 -- 172.21.15.152:0/318122066 learned_addr learned my addr 172.21.15.152:0/318122066 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:01.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.490+0000 7f5cb7e00640 1 -- 172.21.15.152:0/318122066 >> v1:172.21.15.175:6789/0 conn(0x7f5cc006e010 legacy=0x7f5cc006e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:01.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.490+0000 7f5cb7e00640 1 -- 172.21.15.152:0/318122066 >> v1:172.21.15.152:6789/0 conn(0x7f5cc0147410 legacy=0x7f5cc0071cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:01.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.490+0000 7f5cb7e00640 1 -- 172.21.15.152:0/318122066 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cc00af530 con 0x7f5cc00723e0 2024-04-20T11:31:01.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.491+0000 7f5cb7e00640 1 --2- 172.21.15.152:0/318122066 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5cc00723e0 0x7f5cc00afa50 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f5ca8009870 tx=0x7f5ca802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b70bb48566c5480a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:01.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.492+0000 7f5cbe000640 1 -- 172.21.15.152:0/318122066 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca8034070 con 0x7f5cc00723e0 2024-04-20T11:31:01.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.492+0000 7f5cbe000640 1 -- 172.21.15.152:0/318122066 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ca8030860 con 0x7f5cc00723e0 2024-04-20T11:31:01.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.492+0000 7f5cbe000640 1 -- 172.21.15.152:0/318122066 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca8030c30 con 0x7f5cc00723e0 2024-04-20T11:31:01.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.492+0000 7f5cbfe00640 1 -- 172.21.15.152:0/318122066 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5cc00723e0 msgr2=0x7f5cc00afa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:01.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.492+0000 7f5cbfe00640 1 --2- 172.21.15.152:0/318122066 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5cc00723e0 0x7f5cc00afa50 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f5ca8009870 tx=0x7f5ca802b9a0 comp rx=0 tx=0).stop 2024-04-20T11:31:01.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.492+0000 7f5cbfe00640 1 -- 172.21.15.152:0/318122066 shutdown_connections 2024-04-20T11:31:01.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.492+0000 7f5cbfe00640 1 --2- 172.21.15.152:0/318122066 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5cc00723e0 0x7f5cc00afa50 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:01.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.492+0000 7f5cbfe00640 1 -- 172.21.15.152:0/318122066 >> 172.21.15.152:0/318122066 conn(0x7f5cc0067ae0 msgr2=0x7f5cc006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:01.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.492+0000 7f5cbfe00640 1 -- 172.21.15.152:0/318122066 shutdown_connections 2024-04-20T11:31:01.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.492+0000 7f5cbfe00640 1 -- 172.21.15.152:0/318122066 wait complete. 2024-04-20T11:31:01.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.494+0000 7f5cbfe00640 1 Processor -- start 2024-04-20T11:31:01.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.494+0000 7f5cbfe00640 1 -- start start 2024-04-20T11:31:01.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.495+0000 7f5cbfe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5cc006e010 0x7f5cc0143140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:01.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.495+0000 7f5cbfe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5cc00723e0 0x7f5cc0160300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:01.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.495+0000 7f5cbfe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5cc0147410 0x7f5cc015ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:01.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.495+0000 7f5cbfe00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc00b3510 con 0x7f5cc00723e0 2024-04-20T11:31:01.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.495+0000 7f5cbfe00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc00b3390 con 0x7f5cc006e010 2024-04-20T11:31:01.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.495+0000 7f5cbfe00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5cc00b3690 con 0x7f5cc0147410 2024-04-20T11:31:01.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.495+0000 7f5cb7e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5cc00723e0 0x7f5cc0160300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:01.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.495+0000 7f5cb7e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5cc00723e0 0x7f5cc0160300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:47596/0 (socket says 172.21.15.152:47596) 2024-04-20T11:31:01.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.495+0000 7f5cb7e00640 1 -- 172.21.15.152:0/1004521194 learned_addr learned my addr 172.21.15.152:0/1004521194 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:01.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.496+0000 7f5cb7e00640 1 -- 172.21.15.152:0/1004521194 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5cc0147410 msgr2=0x7f5cc015ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:01.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.496+0000 7f5cb7e00640 1 --2- 172.21.15.152:0/1004521194 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5cc0147410 0x7f5cc015ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:01.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.496+0000 7f5cb7e00640 1 -- 172.21.15.152:0/1004521194 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5cc006e010 msgr2=0x7f5cc0143140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:01.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.496+0000 7f5cb7e00640 1 --2- 172.21.15.152:0/1004521194 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5cc006e010 0x7f5cc0143140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:01.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.496+0000 7f5cb7e00640 1 -- 172.21.15.152:0/1004521194 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cc0160840 con 0x7f5cc00723e0 2024-04-20T11:31:01.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.496+0000 7f5cb7e00640 1 --2- 172.21.15.152:0/1004521194 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5cc00723e0 0x7f5cc0160300 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f5ca8009870 tx=0x7f5ca8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:01.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.496+0000 7f5cb7400640 1 -- 172.21.15.152:0/1004521194 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca8048070 con 0x7f5cc00723e0 2024-04-20T11:31:01.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.496+0000 7f5cb7400640 1 -- 172.21.15.152:0/1004521194 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ca8039d90 con 0x7f5cc00723e0 2024-04-20T11:31:01.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.496+0000 7f5cbfe00640 1 -- 172.21.15.152:0/1004521194 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cc015ef90 con 0x7f5cc00723e0 2024-04-20T11:31:01.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.496+0000 7f5cb7400640 1 -- 172.21.15.152:0/1004521194 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca8034040 con 0x7f5cc00723e0 2024-04-20T11:31:01.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.496+0000 7f5cbfe00640 1 -- 172.21.15.152:0/1004521194 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cc015f540 con 0x7f5cc00723e0 2024-04-20T11:31:01.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.497+0000 7f5cbfe00640 1 -- 172.21.15.152:0/1004521194 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cc006bc50 con 0x7f5cc00723e0 2024-04-20T11:31:01.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.498+0000 7f5cb7400640 1 -- 172.21.15.152:0/1004521194 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 9) ==== 91193+0+0 (secure 0 0 0) 0x7f5ca804b070 con 0x7f5cc00723e0 2024-04-20T11:31:01.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.499+0000 7f5cb7400640 1 --2- 172.21.15.152:0/1004521194 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f5c9c06caa0 0x7f5c9c06ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:01.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.499+0000 7f5cb7400640 1 -- 172.21.15.152:0/1004521194 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f5ca80ae1c0 con 0x7f5cc00723e0 2024-04-20T11:31:01.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.499+0000 7f5cbea00640 1 --2- 172.21.15.152:0/1004521194 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f5c9c06caa0 0x7f5c9c06ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:01.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.500+0000 7f5cbea00640 1 --2- 172.21.15.152:0/1004521194 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f5c9c06caa0 0x7f5c9c06ef60 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f5cb0000f60 tx=0x7f5cb00079d0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:01.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.503+0000 7f5cb7400640 1 -- 172.21.15.152:0/1004521194 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f5cc006bc50 con 0x7f5cc00723e0 2024-04-20T11:31:01.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.702+0000 7f5cbfe00640 1 -- 172.21.15.152:0/1004521194 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5cc01410e0 con 0x7f5cc00723e0 2024-04-20T11:31:01.704 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.704+0000 7f5cb7400640 1 -- 172.21.15.152:0/1004521194 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) ==== 74+0+16738 (secure 0 0 0) 0x7f5ca807c1c0 con 0x7f5cc00723e0 2024-04-20T11:31:01.705 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:31:01.705 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":59,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:31:00.308769+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:30:58.087822+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-20T11:30:59.281767+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:31:01.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.707+0000 7f5cbfe00640 1 -- 172.21.15.152:0/1004521194 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f5c9c06caa0 msgr2=0x7f5c9c06ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:01.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.707+0000 7f5cbfe00640 1 --2- 172.21.15.152:0/1004521194 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f5c9c06caa0 0x7f5c9c06ef60 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f5cb0000f60 tx=0x7f5cb00079d0 comp rx=0 tx=0).stop 2024-04-20T11:31:01.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.707+0000 7f5cbfe00640 1 -- 172.21.15.152:0/1004521194 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5cc00723e0 msgr2=0x7f5cc0160300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:01.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.707+0000 7f5cbfe00640 1 --2- 172.21.15.152:0/1004521194 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5cc00723e0 0x7f5cc0160300 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f5ca8009870 tx=0x7f5ca8004730 comp rx=0 tx=0).stop 2024-04-20T11:31:01.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.707+0000 7f5cbfe00640 1 -- 172.21.15.152:0/1004521194 shutdown_connections 2024-04-20T11:31:01.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.707+0000 7f5cbfe00640 1 --2- 172.21.15.152:0/1004521194 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f5c9c06caa0 0x7f5c9c06ef60 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:01.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.707+0000 7f5cbfe00640 1 --2- 172.21.15.152:0/1004521194 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5cc0147410 0x7f5cc015ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:01.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.707+0000 7f5cbfe00640 1 --2- 172.21.15.152:0/1004521194 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5cc00723e0 0x7f5cc0160300 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:01.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.707+0000 7f5cbfe00640 1 --2- 172.21.15.152:0/1004521194 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5cc006e010 0x7f5cc0143140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:01.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.707+0000 7f5cbfe00640 1 -- 172.21.15.152:0/1004521194 >> 172.21.15.152:0/1004521194 conn(0x7f5cc0067ae0 msgr2=0x7f5cc00b48d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:01.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.707+0000 7f5cbfe00640 1 -- 172.21.15.152:0/1004521194 shutdown_connections 2024-04-20T11:31:01.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.707+0000 7f5cbfe00640 1 -- 172.21.15.152:0/1004521194 wait complete. 2024-04-20T11:31:01.725 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-20T11:31:01.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.927+0000 7fd628800640 1 Processor -- start 2024-04-20T11:31:01.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.927+0000 7fd628800640 1 -- start start 2024-04-20T11:31:01.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.929+0000 7fd628800640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fd6200cfc40 0x7fd62015fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:01.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.929+0000 7fd628800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd6200b2880 0x7fd6200b3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:01.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.929+0000 7fd628800640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd62015f1c0 con 0x7fd62015b510 2024-04-20T11:31:01.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.929+0000 7fd628800640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fd62006f420 con 0x7fd6200cfc40 2024-04-20T11:31:01.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.929+0000 7fd628800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd6200b4430 con 0x7fd6200b2880 2024-04-20T11:31:01.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.929+0000 7fd625e00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fd62015b510 0x7fd6200b2170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:50916/0 (socket says 172.21.15.152:50916) 2024-04-20T11:31:01.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.929+0000 7fd625e00640 1 -- 172.21.15.152:0/1395094415 learned_addr learned my addr 172.21.15.152:0/1395094415 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:31:01.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.929+0000 7fd624a00640 1 -- 172.21.15.152:0/1395094415 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3560921813 0 0) 0x7fd62015f1c0 con 0x7fd62015b510 2024-04-20T11:31:01.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.929+0000 7fd624a00640 1 -- 172.21.15.152:0/1395094415 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd608005730 con 0x7fd62015b510 2024-04-20T11:31:01.931 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.929+0000 7fd626800640 1 --2- 172.21.15.152:0/1395094415 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd6200b2880 0x7fd6200b3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:01.931 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.929+0000 7fd625400640 1 --2- 172.21.15.152:0/1395094415 >> v2:172.21.15.175:3300/0 conn(0x7fd6200cfc40 0x7fd62015fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:01.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.930+0000 7fd624a00640 1 -- 172.21.15.152:0/1395094415 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3580051469 0 0) 0x7fd608005730 con 0x7fd62015b510 2024-04-20T11:31:01.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.930+0000 7fd624a00640 1 -- 172.21.15.152:0/1395094415 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd6200b2880 msgr2=0x7fd6200b3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:01.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.930+0000 7fd624a00640 1 --2- 172.21.15.152:0/1395094415 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd6200b2880 0x7fd6200b3ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:01.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.930+0000 7fd624a00640 1 -- 172.21.15.152:0/1395094415 >> v2:172.21.15.175:3300/0 conn(0x7fd6200cfc40 msgr2=0x7fd62015fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:01.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.930+0000 7fd624a00640 1 --2- 172.21.15.152:0/1395094415 >> v2:172.21.15.175:3300/0 conn(0x7fd6200cfc40 0x7fd62015fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:01.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.930+0000 7fd624a00640 1 -- 172.21.15.152:0/1395094415 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd62006f670 con 0x7fd62015b510 2024-04-20T11:31:01.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.930+0000 7fd624a00640 1 -- 172.21.15.152:0/1395094415 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7fd614001670 con 0x7fd62015b510 2024-04-20T11:31:01.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.930+0000 7fd624a00640 1 -- 172.21.15.152:0/1395094415 >> v1:172.21.15.152:6789/0 conn(0x7fd62015b510 legacy=0x7fd6200b2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:01.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.930+0000 7fd624a00640 1 --2- 172.21.15.152:0/1395094415 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd6200b2880 0x7fd608005bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:01.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.930+0000 7fd624a00640 1 --2- 172.21.15.152:0/1395094415 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd608006240 0x7fd608006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:01.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.930+0000 7fd624a00640 1 --2- 172.21.15.152:0/1395094415 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd608006f50 0x7fd6080073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:01.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.930+0000 7fd624a00640 1 -- 172.21.15.152:0/1395094415 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fd62006f420 con 0x7fd608006f50 2024-04-20T11:31:01.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.930+0000 7fd624a00640 1 -- 172.21.15.152:0/1395094415 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd6200b4430 con 0x7fd608006240 2024-04-20T11:31:01.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.930+0000 7fd624a00640 1 -- 172.21.15.152:0/1395094415 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd60800dd30 con 0x7fd6200b2880 2024-04-20T11:31:01.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.930+0000 7fd626800640 1 --2- 172.21.15.152:0/1395094415 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd608006f50 0x7fd6080073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:01.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.931+0000 7fd625e00640 1 --2- 172.21.15.152:0/1395094415 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd6200b2880 0x7fd608005bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:01.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.931+0000 7fd625400640 1 --2- 172.21.15.152:0/1395094415 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd608006240 0x7fd608006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:01.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.931+0000 7fd626800640 1 -- 172.21.15.152:0/1395094415 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd6200b2880 msgr2=0x7fd608005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:01.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.931+0000 7fd626800640 1 --2- 172.21.15.152:0/1395094415 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd6200b2880 0x7fd608005bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:01.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.931+0000 7fd626800640 1 -- 172.21.15.152:0/1395094415 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd608006240 msgr2=0x7fd608006a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:01.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.931+0000 7fd626800640 1 --2- 172.21.15.152:0/1395094415 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd608006240 0x7fd608006a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:01.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.931+0000 7fd626800640 1 -- 172.21.15.152:0/1395094415 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd608005730 con 0x7fd608006f50 2024-04-20T11:31:01.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.931+0000 7fd625400640 1 --2- 172.21.15.152:0/1395094415 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd608006240 0x7fd608006a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:31:01.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.932+0000 7fd626800640 1 --2- 172.21.15.152:0/1395094415 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd608006f50 0x7fd6080073d0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fd61c007360 tx=0x7fd608007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a83f4f7b07fba453 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:01.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.932+0000 7fd624a00640 1 -- 172.21.15.152:0/1395094415 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd61c039070 con 0x7fd608006f50 2024-04-20T11:31:01.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.932+0000 7fd624a00640 1 -- 172.21.15.152:0/1395094415 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd61c02ccd0 con 0x7fd608006f50 2024-04-20T11:31:01.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.932+0000 7fd624a00640 1 -- 172.21.15.152:0/1395094415 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd61c009be0 con 0x7fd608006f50 2024-04-20T11:31:01.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.933+0000 7fd628800640 1 -- 172.21.15.152:0/1395094415 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd608006f50 msgr2=0x7fd6080073d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:01.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.933+0000 7fd628800640 1 --2- 172.21.15.152:0/1395094415 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd608006f50 0x7fd6080073d0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fd61c007360 tx=0x7fd608007a40 comp rx=0 tx=0).stop 2024-04-20T11:31:01.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.933+0000 7fd626800640 1 -- 172.21.15.152:0/1395094415 reap_dead start 2024-04-20T11:31:01.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.933+0000 7fd628800640 1 -- 172.21.15.152:0/1395094415 shutdown_connections 2024-04-20T11:31:01.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.933+0000 7fd628800640 1 -- 172.21.15.152:0/1395094415 >> 172.21.15.152:0/1395094415 conn(0x7fd62006b980 msgr2=0x7fd62006bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:01.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.933+0000 7fd628800640 1 -- 172.21.15.152:0/1395094415 shutdown_connections 2024-04-20T11:31:01.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.933+0000 7fd628800640 1 -- 172.21.15.152:0/1395094415 wait complete. 2024-04-20T11:31:01.938 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.935+0000 7fd628800640 1 Processor -- start 2024-04-20T11:31:01.939 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.935+0000 7fd628800640 1 -- start start 2024-04-20T11:31:01.939 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.936+0000 7fd628800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd608006f50 0x7fd6201e6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:01.939 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.936+0000 7fd628800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd6201f5600 0x7fd620146090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:01.939 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.936+0000 7fd628800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd62015ba70 0x7fd6200b0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:01.939 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.936+0000 7fd628800640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fd62015b8f0 con 0x7fd608006f50 2024-04-20T11:31:01.939 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.936+0000 7fd628800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd62015f710 con 0x7fd62015ba70 2024-04-20T11:31:01.939 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.936+0000 7fd628800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd62006f950 con 0x7fd6201f5600 2024-04-20T11:31:01.939 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.936+0000 7fd625e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd608006f50 0x7fd6201e6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:01.939 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.936+0000 7fd625e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd608006f50 0x7fd6201e6410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:47608/0 (socket says 172.21.15.152:47608) 2024-04-20T11:31:01.939 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.936+0000 7fd625e00640 1 -- 172.21.15.152:0/917739493 learned_addr learned my addr 172.21.15.152:0/917739493 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:01.939 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.936+0000 7fd625e00640 1 -- 172.21.15.152:0/917739493 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd6201f5600 msgr2=0x7fd620146090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:01.939 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.936+0000 7fd625e00640 1 --2- 172.21.15.152:0/917739493 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd6201f5600 0x7fd620146090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:01.939 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.936+0000 7fd625e00640 1 -- 172.21.15.152:0/917739493 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd62015ba70 msgr2=0x7fd6200b0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:01.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.936+0000 7fd625e00640 1 --2- 172.21.15.152:0/917739493 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd62015ba70 0x7fd6200b0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:01.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.936+0000 7fd625e00640 1 -- 172.21.15.152:0/917739493 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6201465d0 con 0x7fd608006f50 2024-04-20T11:31:01.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.937+0000 7fd625e00640 1 --2- 172.21.15.152:0/917739493 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd608006f50 0x7fd6201e6410 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fd61400e7f0 tx=0x7fd61400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:01.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.937+0000 7fd61a000640 1 -- 172.21.15.152:0/917739493 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd61400c7f0 con 0x7fd608006f50 2024-04-20T11:31:01.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.937+0000 7fd61a000640 1 -- 172.21.15.152:0/917739493 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd614018070 con 0x7fd608006f50 2024-04-20T11:31:01.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.937+0000 7fd628800640 1 -- 172.21.15.152:0/917739493 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd62015c2c0 con 0x7fd608006f50 2024-04-20T11:31:01.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.937+0000 7fd61a000640 1 -- 172.21.15.152:0/917739493 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd61400cbd0 con 0x7fd608006f50 2024-04-20T11:31:01.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.937+0000 7fd628800640 1 -- 172.21.15.152:0/917739493 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6201f4b20 con 0x7fd608006f50 2024-04-20T11:31:01.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.938+0000 7fd628800640 1 -- 172.21.15.152:0/917739493 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd620068300 con 0x7fd608006f50 2024-04-20T11:31:01.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.939+0000 7fd61a000640 1 -- 172.21.15.152:0/917739493 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 9) ==== 91193+0+0 (secure 0 0 0) 0x7fd61401d460 con 0x7fd608006f50 2024-04-20T11:31:01.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.940+0000 7fd61a000640 1 --2- 172.21.15.152:0/917739493 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fd60406cad0 0x7fd60406ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:01.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.940+0000 7fd61a000640 1 -- 172.21.15.152:0/917739493 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5709+0+0 (secure 0 0 0) 0x7fd61408bcf0 con 0x7fd608006f50 2024-04-20T11:31:01.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.940+0000 7fd625400640 1 --2- 172.21.15.152:0/917739493 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fd60406cad0 0x7fd60406ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:01.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.941+0000 7fd625400640 1 --2- 172.21.15.152:0/917739493 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fd60406cad0 0x7fd60406ef90 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fd6200b3af0 tx=0x7fd6100093d0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:01.944 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:01.944+0000 7fd61a000640 1 -- 172.21.15.152:0/917739493 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fd620068300 con 0x7fd608006f50 2024-04-20T11:31:02.182 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:02.182+0000 7fd628800640 1 -- 172.21.15.152:0/917739493 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fd6200689f0 con 0x7fd608006f50 2024-04-20T11:31:02.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:02.891+0000 7fd61a000640 1 -- 172.21.15.152:0/917739493 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v32) ==== 118+0+0 (secure 0 0 0) 0x7fd614059e90 con 0x7fd608006f50 2024-04-20T11:31:02.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:02.895+0000 7fd628800640 1 -- 172.21.15.152:0/917739493 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fd60406cad0 msgr2=0x7fd60406ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:02.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:02.895+0000 7fd628800640 1 --2- 172.21.15.152:0/917739493 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fd60406cad0 0x7fd60406ef90 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fd6200b3af0 tx=0x7fd6100093d0 comp rx=0 tx=0).stop 2024-04-20T11:31:02.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:02.895+0000 7fd628800640 1 -- 172.21.15.152:0/917739493 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd608006f50 msgr2=0x7fd6201e6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:02.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:02.895+0000 7fd628800640 1 --2- 172.21.15.152:0/917739493 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd608006f50 0x7fd6201e6410 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fd61400e7f0 tx=0x7fd61400ecb0 comp rx=0 tx=0).stop 2024-04-20T11:31:02.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:02.895+0000 7fd628800640 1 -- 172.21.15.152:0/917739493 shutdown_connections 2024-04-20T11:31:02.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:02.895+0000 7fd628800640 1 --2- 172.21.15.152:0/917739493 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fd60406cad0 0x7fd60406ef90 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:02.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:02.895+0000 7fd628800640 1 --2- 172.21.15.152:0/917739493 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd62015ba70 0x7fd6200b0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:02.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:02.895+0000 7fd628800640 1 --2- 172.21.15.152:0/917739493 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd6201f5600 0x7fd620146090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:02.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:02.895+0000 7fd628800640 1 --2- 172.21.15.152:0/917739493 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd608006f50 0x7fd6201e6410 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:02.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:02.895+0000 7fd628800640 1 -- 172.21.15.152:0/917739493 >> 172.21.15.152:0/917739493 conn(0x7fd62006b980 msgr2=0x7fd620157960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:02.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:02.895+0000 7fd628800640 1 -- 172.21.15.152:0/917739493 shutdown_connections 2024-04-20T11:31:02.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:02.895+0000 7fd628800640 1 -- 172.21.15.152:0/917739493 wait complete. 2024-04-20T11:31:02.916 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-20T11:31:02.917 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-20T11:31:03.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:03.114+0000 7f9130a00640 1 Processor -- start 2024-04-20T11:31:03.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:03.115+0000 7f9130a00640 1 -- start start 2024-04-20T11:31:03.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:03.116+0000 7f9130a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f912c06be10 0x7f912c145f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:03.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:03.116+0000 7f9130a00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f912c0b5cd0 con 0x7f912c0b1b70 2024-04-20T11:31:03.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:03.116+0000 7f9130a00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f912c0b2aa0 con 0x7f912c06b980 2024-04-20T11:31:03.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:03.116+0000 7f9130a00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f912c0b5f70 con 0x7f912c06be10 2024-04-20T11:31:03.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:03.116+0000 7f912a800640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f912c0b1b70 0x7f912c162d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:50922/0 (socket says 172.21.15.152:50922) 2024-04-20T11:31:03.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:03.116+0000 7f912a800640 1 -- 172.21.15.152:0/2551293681 learned_addr learned my addr 172.21.15.152:0/2551293681 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:31:03.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:03.116+0000 7f912b200640 1 --2- 172.21.15.152:0/2551293681 >> v2:172.21.15.152:3300/0 conn(0x7f912c06be10 0x7f912c145f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:03.118 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:03.117+0000 7f912b200640 1 -- 172.21.15.152:0/2551293681 >> v1:172.21.15.175:6789/0 conn(0x7f912c06b980 legacy=0x7f912c145810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:03.118 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:03.117+0000 7f912b200640 1 -- 172.21.15.152:0/2551293681 >> v1:172.21.15.152:6789/0 conn(0x7f912c0b1b70 legacy=0x7f912c162d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:03.118 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:03.117+0000 7f912b200640 1 -- 172.21.15.152:0/2551293681 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f912c0b6250 con 0x7f912c06be10 2024-04-20T11:31:03.118 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:03.117+0000 7f9129400640 1 -- 172.21.15.152:0/2551293681 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1702192120 0 0) 0x7f912c0b5cd0 con 0x7f912c0b1b70 2024-04-20T11:31:03.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:03.118+0000 7f912b200640 1 --2- 172.21.15.152:0/2551293681 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f912c06be10 0x7f912c145f20 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f91240097a0 tx=0x7f912402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c2338b20832ce26 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:03.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:03.118+0000 7f9129400640 1 -- 172.21.15.152:0/2551293681 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9124035020 con 0x7f912c06be10 2024-04-20T11:31:03.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:03.118+0000 7f9129400640 1 -- 172.21.15.152:0/2551293681 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9124031830 con 0x7f912c06be10 2024-04-20T11:31:03.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:03.118+0000 7f9129400640 1 -- 172.21.15.152:0/2551293681 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9124031c00 con 0x7f912c06be10 2024-04-20T11:31:03.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:03.119+0000 7f9130a00640 1 -- 172.21.15.152:0/2551293681 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f912c06be10 msgr2=0x7f912c145f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:03.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:03.119+0000 7f9130a00640 1 --2- 172.21.15.152:0/2551293681 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f912c06be10 0x7f912c145f20 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f91240097a0 tx=0x7f912402b8d0 comp rx=0 tx=0).stop 2024-04-20T11:31:03.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:03.119+0000 7f9130a00640 1 -- 172.21.15.152:0/2551293681 shutdown_connections 2024-04-20T11:31:03.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:03.119+0000 7f9130a00640 1 --2- 172.21.15.152:0/2551293681 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f912c06be10 0x7f912c145f20 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:03.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:03.119+0000 7f9130a00640 1 -- 172.21.15.152:0/2551293681 >> 172.21.15.152:0/2551293681 conn(0x7f912c0b6790 msgr2=0x7f912c0b6ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:03.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:03.119+0000 7f9130a00640 1 -- 172.21.15.152:0/2551293681 shutdown_connections 2024-04-20T11:31:03.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:03.119+0000 7f9130a00640 1 -- 172.21.15.152:0/2551293681 wait complete. 2024-04-20T11:31:03.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:03.121+0000 7f9130a00640 1 Processor -- start 2024-04-20T11:31:03.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:03.121+0000 7f9130a00640 1 -- start start 2024-04-20T11:31:03.124 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:03.122+0000 7f9130a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f912c06b980 0x7f912c15f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:03.124 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:03.122+0000 7f9130a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f912c06be10 0x7f912c15d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:03.124 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:03.122+0000 7f9130a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f912c0b1b70 0x7f912c0afae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:03.124 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:03.122+0000 7f9130a00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f912c0b44a0 con 0x7f912c06be10 2024-04-20T11:31:03.124 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:03.122+0000 7f9130a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f912c06c260 con 0x7f912c0b1b70 2024-04-20T11:31:03.124 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:03.122+0000 7f9130a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f912c0b1f50 con 0x7f912c06b980 2024-04-20T11:31:03.124 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:03.122+0000 7f9129e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f912c06be10 0x7f912c15d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:03.125 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:03.122+0000 7f9129e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f912c06be10 0x7f912c15d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:47618/0 (socket says 172.21.15.152:47618) 2024-04-20T11:31:03.125 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:03.122+0000 7f9129e00640 1 -- 172.21.15.152:0/403640277 learned_addr learned my addr 172.21.15.152:0/403640277 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:03.125 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:03.123+0000 7f912a800640 1 --2- 172.21.15.152:0/403640277 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f912c06b980 0x7f912c15f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:03.125 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:03.123+0000 7f912b200640 1 --2- 172.21.15.152:0/403640277 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f912c0b1b70 0x7f912c0afae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:03.125 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:03.123+0000 7f9129e00640 1 -- 172.21.15.152:0/403640277 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f912c06b980 msgr2=0x7f912c15f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:03.125 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:03.123+0000 7f9129e00640 1 --2- 172.21.15.152:0/403640277 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f912c06b980 0x7f912c15f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:03.125 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:03.123+0000 7f9129e00640 1 -- 172.21.15.152:0/403640277 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f912c0b1b70 msgr2=0x7f912c0afae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:03.125 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:03.123+0000 7f9129e00640 1 --2- 172.21.15.152:0/403640277 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f912c0b1b70 0x7f912c0afae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:03.125 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:03.123+0000 7f9129e00640 1 -- 172.21.15.152:0/403640277 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f912c15d940 con 0x7f912c06be10 2024-04-20T11:31:03.125 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:03.123+0000 7f9129e00640 1 --2- 172.21.15.152:0/403640277 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f912c06be10 0x7f912c15d400 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f911400c7c0 tx=0x7f911400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:03.125 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:03.123+0000 7f9122a00640 1 -- 172.21.15.152:0/403640277 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9114016020 con 0x7f912c06be10 2024-04-20T11:31:03.125 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:03.123+0000 7f9122a00640 1 -- 172.21.15.152:0/403640277 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f911400aa50 con 0x7f912c06be10 2024-04-20T11:31:03.126 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:03.123+0000 7f9122a00640 1 -- 172.21.15.152:0/403640277 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9114012420 con 0x7f912c06be10 2024-04-20T11:31:03.126 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:03.123+0000 7f9130a00640 1 -- 172.21.15.152:0/403640277 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f912c15feb0 con 0x7f912c06be10 2024-04-20T11:31:03.126 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:03.124+0000 7f9130a00640 1 -- 172.21.15.152:0/403640277 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f912c160470 con 0x7f912c06be10 2024-04-20T11:31:03.126 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:03.124+0000 7f9130a00640 1 -- 172.21.15.152:0/403640277 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f912c075e60 con 0x7f912c06be10 2024-04-20T11:31:03.126 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:03.126+0000 7f9122a00640 1 -- 172.21.15.152:0/403640277 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 9) ==== 91193+0+0 (secure 0 0 0) 0x7f9114019020 con 0x7f912c06be10 2024-04-20T11:31:03.128 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:03.127+0000 7f9122a00640 1 --2- 172.21.15.152:0/403640277 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f910806cad0 0x7f910806ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:03.128 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:03.127+0000 7f9122a00640 1 -- 172.21.15.152:0/403640277 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5709+0+0 (secure 0 0 0) 0x7f911408bb90 con 0x7f912c06be10 2024-04-20T11:31:03.128 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:03.127+0000 7f912a800640 1 --2- 172.21.15.152:0/403640277 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f910806cad0 0x7f910806ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:03.128 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:03.128+0000 7f912a800640 1 --2- 172.21.15.152:0/403640277 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f910806cad0 0x7f910806ef90 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f911c000f60 tx=0x7f911c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:03.132 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:03.132+0000 7f9122a00640 1 -- 172.21.15.152:0/403640277 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f9114020070 con 0x7f912c06be10 2024-04-20T11:31:03.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:03.367+0000 7f9130a00640 1 -- 172.21.15.152:0/403640277 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f912c160810 con 0x7f912c06be10 2024-04-20T11:31:03.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:03.906+0000 7f9122a00640 1 -- 172.21.15.152:0/403640277 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v33) ==== 122+0+0 (secure 0 0 0) 0x7f9114059d30 con 0x7f912c06be10 2024-04-20T11:31:03.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:03.910+0000 7f9130a00640 1 -- 172.21.15.152:0/403640277 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f910806cad0 msgr2=0x7f910806ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:03.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:03.910+0000 7f9130a00640 1 --2- 172.21.15.152:0/403640277 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f910806cad0 0x7f910806ef90 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f911c000f60 tx=0x7f911c009040 comp rx=0 tx=0).stop 2024-04-20T11:31:03.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:03.910+0000 7f9130a00640 1 -- 172.21.15.152:0/403640277 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f912c06be10 msgr2=0x7f912c15d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:03.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:03.910+0000 7f9130a00640 1 --2- 172.21.15.152:0/403640277 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f912c06be10 0x7f912c15d400 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f911400c7c0 tx=0x7f911400cc80 comp rx=0 tx=0).stop 2024-04-20T11:31:03.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:03.910+0000 7f9130a00640 1 -- 172.21.15.152:0/403640277 shutdown_connections 2024-04-20T11:31:03.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:03.910+0000 7f9130a00640 1 --2- 172.21.15.152:0/403640277 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f910806cad0 0x7f910806ef90 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:03.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:03.910+0000 7f9130a00640 1 --2- 172.21.15.152:0/403640277 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f912c0b1b70 0x7f912c0afae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:03.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:03.910+0000 7f9130a00640 1 --2- 172.21.15.152:0/403640277 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f912c06be10 0x7f912c15d400 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:03.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:03.910+0000 7f9130a00640 1 --2- 172.21.15.152:0/403640277 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f912c06b980 0x7f912c15f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:03.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:03.910+0000 7f9130a00640 1 -- 172.21.15.152:0/403640277 >> 172.21.15.152:0/403640277 conn(0x7f912c0b6790 msgr2=0x7f912c15e1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:03.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:03.910+0000 7f9130a00640 1 -- 172.21.15.152:0/403640277 shutdown_connections 2024-04-20T11:31:03.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:03.910+0000 7f9130a00640 1 -- 172.21.15.152:0/403640277 wait complete. 2024-04-20T11:31:03.933 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-20T11:31:03.933 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:31:04.135 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.131+0000 7f3395600640 1 Processor -- start 2024-04-20T11:31:04.135 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.131+0000 7f3395600640 1 -- start start 2024-04-20T11:31:04.135 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.132+0000 7f3395600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f339015b5e0 0x7f339006eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:04.136 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.132+0000 7f3395600640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f339006fa50 0x7f33900b50d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:04.136 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.132+0000 7f3395600640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33900b5770 con 0x7f33900b4c50 2024-04-20T11:31:04.136 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.132+0000 7f3395600640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f339006de10 con 0x7f339006fa50 2024-04-20T11:31:04.136 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.132+0000 7f3395600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f339006e060 con 0x7f339015b5e0 2024-04-20T11:31:04.136 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.133+0000 7f338f400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f339006fa50 0x7f33900b50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:04.136 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.133+0000 7f338f400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f339006fa50 0x7f33900b50d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:47624/0 (socket says 172.21.15.152:47624) 2024-04-20T11:31:04.136 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.133+0000 7f338f400640 1 -- 172.21.15.152:0/552743646 learned_addr learned my addr 172.21.15.152:0/552743646 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:04.136 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.133+0000 7f338ea00640 1 --2- 172.21.15.152:0/552743646 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f339015b5e0 0x7f339006eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:04.136 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.133+0000 7f338f400640 1 -- 172.21.15.152:0/552743646 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f339015b5e0 msgr2=0x7f339006eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:04.136 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.133+0000 7f338f400640 1 --2- 172.21.15.152:0/552743646 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f339015b5e0 0x7f339006eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:04.136 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.133+0000 7f338f400640 1 -- 172.21.15.152:0/552743646 >> v1:172.21.15.175:6789/0 conn(0x7f33900b4c50 legacy=0x7f339006f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:04.136 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.133+0000 7f338f400640 1 -- 172.21.15.152:0/552743646 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f339006e340 con 0x7f339006fa50 2024-04-20T11:31:04.136 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.134+0000 7f338f400640 1 --2- 172.21.15.152:0/552743646 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f339006fa50 0x7f33900b50d0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f33900b0090 tx=0x7f338802be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c75936b446cac5c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:04.136 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.135+0000 7f338e000640 1 -- 172.21.15.152:0/552743646 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f338802ca30 con 0x7f339006fa50 2024-04-20T11:31:04.137 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.135+0000 7f338e000640 1 -- 172.21.15.152:0/552743646 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f338802cbd0 con 0x7f339006fa50 2024-04-20T11:31:04.137 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.135+0000 7f338e000640 1 -- 172.21.15.152:0/552743646 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33880343f0 con 0x7f339006fa50 2024-04-20T11:31:04.137 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.135+0000 7f3395600640 1 -- 172.21.15.152:0/552743646 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f339006fa50 msgr2=0x7f33900b50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:04.137 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.135+0000 7f3395600640 1 --2- 172.21.15.152:0/552743646 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f339006fa50 0x7f33900b50d0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f33900b0090 tx=0x7f338802be90 comp rx=0 tx=0).stop 2024-04-20T11:31:04.138 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.136+0000 7f3395600640 1 -- 172.21.15.152:0/552743646 shutdown_connections 2024-04-20T11:31:04.138 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.136+0000 7f3395600640 1 --2- 172.21.15.152:0/552743646 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f339006fa50 0x7f33900b50d0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:04.138 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.136+0000 7f3395600640 1 --2- 172.21.15.152:0/552743646 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f339015b5e0 0x7f339006eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:04.138 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.136+0000 7f3395600640 1 -- 172.21.15.152:0/552743646 >> 172.21.15.152:0/552743646 conn(0x7f3390067cf0 msgr2=0x7f339006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:04.138 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.136+0000 7f3395600640 1 -- 172.21.15.152:0/552743646 shutdown_connections 2024-04-20T11:31:04.138 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.136+0000 7f3395600640 1 -- 172.21.15.152:0/552743646 wait complete. 2024-04-20T11:31:04.138 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.138+0000 7f3395600640 1 Processor -- start 2024-04-20T11:31:04.139 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.138+0000 7f3395600640 1 -- start start 2024-04-20T11:31:04.140 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.139+0000 7f3395600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f339006fa50 0x7f339014b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:04.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.139+0000 7f3395600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33900b4c50 0x7f3390158cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:04.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.139+0000 7f3395600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f339015b5e0 0x7f3390159200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:04.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.139+0000 7f3395600640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f33900b42d0 con 0x7f339015b5e0 2024-04-20T11:31:04.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.139+0000 7f3395600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f339015c9d0 con 0x7f339006fa50 2024-04-20T11:31:04.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.139+0000 7f3395600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f339015cd70 con 0x7f33900b4c50 2024-04-20T11:31:04.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.139+0000 7f338f400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f339015b5e0 0x7f3390159200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:04.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.139+0000 7f338f400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f339015b5e0 0x7f3390159200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:47634/0 (socket says 172.21.15.152:47634) 2024-04-20T11:31:04.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.139+0000 7f338f400640 1 -- 172.21.15.152:0/2600636089 learned_addr learned my addr 172.21.15.152:0/2600636089 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:04.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.140+0000 7f338ea00640 1 --2- 172.21.15.152:0/2600636089 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f339006fa50 0x7f339014b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:04.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.140+0000 7f3386000640 1 --2- 172.21.15.152:0/2600636089 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33900b4c50 0x7f3390158cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:04.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.140+0000 7f338f400640 1 -- 172.21.15.152:0/2600636089 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33900b4c50 msgr2=0x7f3390158cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:04.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.140+0000 7f338f400640 1 --2- 172.21.15.152:0/2600636089 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33900b4c50 0x7f3390158cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:04.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.140+0000 7f338f400640 1 -- 172.21.15.152:0/2600636089 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f339006fa50 msgr2=0x7f339014b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:04.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.140+0000 7f338f400640 1 --2- 172.21.15.152:0/2600636089 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f339006fa50 0x7f339014b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:04.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.140+0000 7f338f400640 1 -- 172.21.15.152:0/2600636089 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3390151fb0 con 0x7f339015b5e0 2024-04-20T11:31:04.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.140+0000 7f338f400640 1 --2- 172.21.15.152:0/2600636089 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f339015b5e0 0x7f3390159200 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f3388009f40 tx=0x7f3388004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:04.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.140+0000 7f3387400640 1 -- 172.21.15.152:0/2600636089 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3388043020 con 0x7f339015b5e0 2024-04-20T11:31:04.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.141+0000 7f3387400640 1 -- 172.21.15.152:0/2600636089 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f338803e450 con 0x7f339015b5e0 2024-04-20T11:31:04.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.141+0000 7f3387400640 1 -- 172.21.15.152:0/2600636089 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3388009510 con 0x7f339015b5e0 2024-04-20T11:31:04.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.141+0000 7f3395600640 1 -- 172.21.15.152:0/2600636089 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3390152290 con 0x7f339015b5e0 2024-04-20T11:31:04.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.141+0000 7f3395600640 1 -- 172.21.15.152:0/2600636089 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3390152850 con 0x7f339015b5e0 2024-04-20T11:31:04.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.142+0000 7f3395600640 1 -- 172.21.15.152:0/2600636089 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33900689f0 con 0x7f339015b5e0 2024-04-20T11:31:04.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.143+0000 7f3387400640 1 -- 172.21.15.152:0/2600636089 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 9) ==== 91193+0+0 (secure 0 0 0) 0x7f3388046020 con 0x7f339015b5e0 2024-04-20T11:31:04.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.144+0000 7f3387400640 1 --2- 172.21.15.152:0/2600636089 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f336c06caa0 0x7f336c06ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:04.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.144+0000 7f3387400640 1 -- 172.21.15.152:0/2600636089 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5709+0+0 (secure 0 0 0) 0x7f33880aea30 con 0x7f339015b5e0 2024-04-20T11:31:04.145 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.144+0000 7f338ea00640 1 --2- 172.21.15.152:0/2600636089 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f336c06caa0 0x7f336c06ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:04.145 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.145+0000 7f338ea00640 1 --2- 172.21.15.152:0/2600636089 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f336c06caa0 0x7f336c06ef60 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f3380006fd0 tx=0x7f3380008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:04.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.150+0000 7f3387400640 1 -- 172.21.15.152:0/2600636089 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f338804d020 con 0x7f339015b5e0 2024-04-20T11:31:04.377 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.377+0000 7f3395600640 1 -- 172.21.15.152:0/2600636089 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3390154da0 con 0x7f339015b5e0 2024-04-20T11:31:04.378 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.377+0000 7f3387400640 1 -- 172.21.15.152:0/2600636089 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+5205 (secure 0 0 0) 0x7f338807cb40 con 0x7f339015b5e0 2024-04-20T11:31:04.378 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:31:04.378 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":33,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5045,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/3268046291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3268046291},{"type":"v1","addr":"172.21.15.175:6835","nonce":3268046291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5347,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/845526345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":845526345},{"type":"v1","addr":"172.21.15.175:6833","nonce":845526345}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":6288,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6836/3582785355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":3582785355},{"type":"v1","addr":"172.21.15.152:6836","nonce":3582785355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T11:31:00.308725+0000","modified":"2024-04-20T11:31:03.369713+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6291":{"gid":6291,"name":"c","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6837/488955771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6835","nonce":488955771},{"type":"v1","addr":"172.21.15.152:6837","nonce":488955771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6291,"qdb_cluster":[6291]},"id":4}]} 2024-04-20T11:31:04.379 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 33 2024-04-20T11:31:04.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.380+0000 7f3395600640 1 -- 172.21.15.152:0/2600636089 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f336c06caa0 msgr2=0x7f336c06ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:04.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.380+0000 7f3395600640 1 --2- 172.21.15.152:0/2600636089 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f336c06caa0 0x7f336c06ef60 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f3380006fd0 tx=0x7f3380008040 comp rx=0 tx=0).stop 2024-04-20T11:31:04.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.380+0000 7f3395600640 1 -- 172.21.15.152:0/2600636089 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f339015b5e0 msgr2=0x7f3390159200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:04.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.380+0000 7f3395600640 1 --2- 172.21.15.152:0/2600636089 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f339015b5e0 0x7f3390159200 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f3388009f40 tx=0x7f3388004a40 comp rx=0 tx=0).stop 2024-04-20T11:31:04.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.380+0000 7f3395600640 1 -- 172.21.15.152:0/2600636089 shutdown_connections 2024-04-20T11:31:04.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.380+0000 7f3395600640 1 --2- 172.21.15.152:0/2600636089 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f336c06caa0 0x7f336c06ef60 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:04.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.380+0000 7f3395600640 1 --2- 172.21.15.152:0/2600636089 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f339015b5e0 0x7f3390159200 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:04.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.380+0000 7f3395600640 1 --2- 172.21.15.152:0/2600636089 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33900b4c50 0x7f3390158cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:04.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.380+0000 7f3395600640 1 --2- 172.21.15.152:0/2600636089 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f339006fa50 0x7f339014b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:04.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.380+0000 7f3395600640 1 -- 172.21.15.152:0/2600636089 >> 172.21.15.152:0/2600636089 conn(0x7f3390067cf0 msgr2=0x7f3390150ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:04.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.380+0000 7f3395600640 1 -- 172.21.15.152:0/2600636089 shutdown_connections 2024-04-20T11:31:04.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.380+0000 7f3395600640 1 -- 172.21.15.152:0/2600636089 wait complete. 2024-04-20T11:31:04.400 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:31:04.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.594+0000 7efc63e00640 1 Processor -- start 2024-04-20T11:31:04.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.594+0000 7efc63e00640 1 -- start start 2024-04-20T11:31:04.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.595+0000 7efc63e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7efc6415d1e0 0x7efc64160b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:04.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.595+0000 7efc63e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efc6415d5c0 0x7efc640cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:04.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.596+0000 7efc63e00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc640ce8c0 con 0x7efc64071cd0 2024-04-20T11:31:04.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.596+0000 7efc63e00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7efc641600b0 con 0x7efc6415d1e0 2024-04-20T11:31:04.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.596+0000 7efc63e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7efc64160390 con 0x7efc6415d5c0 2024-04-20T11:31:04.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.596+0000 7efc63400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7efc64071cd0 0x7efc640ccd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:50924/0 (socket says 172.21.15.152:50924) 2024-04-20T11:31:04.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.596+0000 7efc63400640 1 -- 172.21.15.152:0/1501318237 learned_addr learned my addr 172.21.15.152:0/1501318237 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:31:04.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.596+0000 7efc62a00640 1 --2- 172.21.15.152:0/1501318237 >> v2:172.21.15.175:3300/0 conn(0x7efc6415d1e0 0x7efc64160b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:04.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.596+0000 7efc61600640 1 -- 172.21.15.152:0/1501318237 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3681263931 0 0) 0x7efc640ce8c0 con 0x7efc64071cd0 2024-04-20T11:31:04.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.596+0000 7efc62000640 1 --2- 172.21.15.152:0/1501318237 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efc6415d5c0 0x7efc640cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:04.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.596+0000 7efc61600640 1 -- 172.21.15.152:0/1501318237 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efc48003520 con 0x7efc64071cd0 2024-04-20T11:31:04.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.597+0000 7efc61600640 1 -- 172.21.15.152:0/1501318237 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2065272709 0 0) 0x7efc48003520 con 0x7efc64071cd0 2024-04-20T11:31:04.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.597+0000 7efc61600640 1 -- 172.21.15.152:0/1501318237 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efc6415d5c0 msgr2=0x7efc640cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:04.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.597+0000 7efc61600640 1 --2- 172.21.15.152:0/1501318237 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efc6415d5c0 0x7efc640cdfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:04.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.597+0000 7efc61600640 1 -- 172.21.15.152:0/1501318237 >> v2:172.21.15.175:3300/0 conn(0x7efc6415d1e0 msgr2=0x7efc64160b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:04.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.597+0000 7efc61600640 1 --2- 172.21.15.152:0/1501318237 >> v2:172.21.15.175:3300/0 conn(0x7efc6415d1e0 0x7efc64160b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:04.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.597+0000 7efc61600640 1 -- 172.21.15.152:0/1501318237 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc64160670 con 0x7efc64071cd0 2024-04-20T11:31:04.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.597+0000 7efc61600640 1 -- 172.21.15.152:0/1501318237 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7efc5c0019f0 con 0x7efc64071cd0 2024-04-20T11:31:04.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.597+0000 7efc61600640 1 -- 172.21.15.152:0/1501318237 >> v1:172.21.15.152:6789/0 conn(0x7efc64071cd0 legacy=0x7efc640ccd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:04.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.597+0000 7efc61600640 1 --2- 172.21.15.152:0/1501318237 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efc480039b0 0x7efc48003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:04.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.597+0000 7efc61600640 1 --2- 172.21.15.152:0/1501318237 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efc48004800 0x7efc48004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:04.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.597+0000 7efc61600640 1 --2- 172.21.15.152:0/1501318237 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efc480051c0 0x7efc4800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:04.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.597+0000 7efc61600640 1 -- 172.21.15.152:0/1501318237 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7efc641600b0 con 0x7efc480039b0 2024-04-20T11:31:04.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.597+0000 7efc61600640 1 -- 172.21.15.152:0/1501318237 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7efc64160390 con 0x7efc48004800 2024-04-20T11:31:04.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.597+0000 7efc61600640 1 -- 172.21.15.152:0/1501318237 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7efc4800bfb0 con 0x7efc480051c0 2024-04-20T11:31:04.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.597+0000 7efc62000640 1 --2- 172.21.15.152:0/1501318237 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efc6415d5c0 0x7efc640cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:31:04.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.597+0000 7efc62a00640 1 --2- 172.21.15.152:0/1501318237 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efc480039b0 0x7efc48003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:04.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.597+0000 7efc62a00640 1 -- 172.21.15.152:0/1501318237 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efc480051c0 msgr2=0x7efc4800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:04.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.597+0000 7efc62a00640 1 --2- 172.21.15.152:0/1501318237 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efc480051c0 0x7efc4800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:04.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.598+0000 7efc62a00640 1 -- 172.21.15.152:0/1501318237 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efc48004800 msgr2=0x7efc48004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:04.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.598+0000 7efc62a00640 1 --2- 172.21.15.152:0/1501318237 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efc48004800 0x7efc48004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:04.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.598+0000 7efc62a00640 1 -- 172.21.15.152:0/1501318237 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc48003520 con 0x7efc480039b0 2024-04-20T11:31:04.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.599+0000 7efc62a00640 1 --2- 172.21.15.152:0/1501318237 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efc480039b0 0x7efc48003dd0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7efc54009ac0 tx=0x7efc48005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5569e47ae0885ac4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:04.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.599+0000 7efc61600640 1 -- 172.21.15.152:0/1501318237 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc54039070 con 0x7efc480039b0 2024-04-20T11:31:04.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.599+0000 7efc61600640 1 -- 172.21.15.152:0/1501318237 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc5402c710 con 0x7efc480039b0 2024-04-20T11:31:04.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.599+0000 7efc61600640 1 -- 172.21.15.152:0/1501318237 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc54009690 con 0x7efc480039b0 2024-04-20T11:31:04.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.599+0000 7efc63e00640 1 -- 172.21.15.152:0/1501318237 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efc480039b0 msgr2=0x7efc48003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:04.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.599+0000 7efc63e00640 1 --2- 172.21.15.152:0/1501318237 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efc480039b0 0x7efc48003dd0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7efc54009ac0 tx=0x7efc48005830 comp rx=0 tx=0).stop 2024-04-20T11:31:04.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.599+0000 7efc63400640 1 -- 172.21.15.152:0/1501318237 reap_dead start 2024-04-20T11:31:04.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.599+0000 7efc63e00640 1 -- 172.21.15.152:0/1501318237 shutdown_connections 2024-04-20T11:31:04.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.599+0000 7efc63e00640 1 -- 172.21.15.152:0/1501318237 >> 172.21.15.152:0/1501318237 conn(0x7efc64067ae0 msgr2=0x7efc6406b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:04.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.599+0000 7efc63e00640 1 -- 172.21.15.152:0/1501318237 shutdown_connections 2024-04-20T11:31:04.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.600+0000 7efc63e00640 1 -- 172.21.15.152:0/1501318237 wait complete. 2024-04-20T11:31:04.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.601+0000 7efc63e00640 1 Processor -- start 2024-04-20T11:31:04.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.601+0000 7efc63e00640 1 -- start start 2024-04-20T11:31:04.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.602+0000 7efc63e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efc480039b0 0x7efc641e78c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:04.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.602+0000 7efc63e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efc641e62b0 0x7efc6414abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:04.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.602+0000 7efc63e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efc640aecb0 0x7efc640abc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:04.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.602+0000 7efc63e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7efc640722e0 con 0x7efc641e62b0 2024-04-20T11:31:04.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.602+0000 7efc63e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7efc64072160 con 0x7efc480039b0 2024-04-20T11:31:04.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.602+0000 7efc63e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7efc64072460 con 0x7efc640aecb0 2024-04-20T11:31:04.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.603+0000 7efc62000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efc641e62b0 0x7efc6414abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:04.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.603+0000 7efc62000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efc641e62b0 0x7efc6414abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:47664/0 (socket says 172.21.15.152:47664) 2024-04-20T11:31:04.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.603+0000 7efc62000640 1 -- 172.21.15.152:0/1400228763 learned_addr learned my addr 172.21.15.152:0/1400228763 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:04.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.603+0000 7efc62a00640 1 --2- 172.21.15.152:0/1400228763 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efc480039b0 0x7efc641e78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:04.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.603+0000 7efc62000640 1 -- 172.21.15.152:0/1400228763 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efc640aecb0 msgr2=0x7efc640abc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:04.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.603+0000 7efc62000640 1 --2- 172.21.15.152:0/1400228763 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efc640aecb0 0x7efc640abc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:04.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.603+0000 7efc62000640 1 -- 172.21.15.152:0/1400228763 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efc480039b0 msgr2=0x7efc641e78c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:04.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.603+0000 7efc62000640 1 --2- 172.21.15.152:0/1400228763 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efc480039b0 0x7efc641e78c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:04.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.603+0000 7efc62000640 1 -- 172.21.15.152:0/1400228763 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc641e6820 con 0x7efc641e62b0 2024-04-20T11:31:04.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.603+0000 7efc62000640 1 --2- 172.21.15.152:0/1400228763 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efc641e62b0 0x7efc6414abe0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7efc5000d4b0 tx=0x7efc5000d970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:04.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.603+0000 7efc5aa00640 1 -- 172.21.15.152:0/1400228763 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc500125f0 con 0x7efc641e62b0 2024-04-20T11:31:04.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.603+0000 7efc5aa00640 1 -- 172.21.15.152:0/1400228763 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc50019070 con 0x7efc641e62b0 2024-04-20T11:31:04.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.603+0000 7efc5aa00640 1 -- 172.21.15.152:0/1400228763 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc500129f0 con 0x7efc641e62b0 2024-04-20T11:31:04.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.604+0000 7efc63e00640 1 -- 172.21.15.152:0/1400228763 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc641e6b00 con 0x7efc641e62b0 2024-04-20T11:31:04.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.604+0000 7efc63e00640 1 -- 172.21.15.152:0/1400228763 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc641e70c0 con 0x7efc641e62b0 2024-04-20T11:31:04.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.605+0000 7efc63e00640 1 -- 172.21.15.152:0/1400228763 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc64075e60 con 0x7efc641e62b0 2024-04-20T11:31:04.607 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.606+0000 7efc5aa00640 1 -- 172.21.15.152:0/1400228763 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 9) ==== 91193+0+0 (secure 0 0 0) 0x7efc50012b90 con 0x7efc641e62b0 2024-04-20T11:31:04.607 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.607+0000 7efc5aa00640 1 --2- 172.21.15.152:0/1400228763 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7efc4406caa0 0x7efc4406ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:04.607 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.607+0000 7efc5aa00640 1 -- 172.21.15.152:0/1400228763 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5709+0+0 (secure 0 0 0) 0x7efc5008bfe0 con 0x7efc641e62b0 2024-04-20T11:31:04.607 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.607+0000 7efc62a00640 1 --2- 172.21.15.152:0/1400228763 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7efc4406caa0 0x7efc4406ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:04.608 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.608+0000 7efc62a00640 1 --2- 172.21.15.152:0/1400228763 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7efc4406caa0 0x7efc4406ef60 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7efc54036000 tx=0x7efc54030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:04.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.612+0000 7efc5aa00640 1 -- 172.21.15.152:0/1400228763 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7efc50023020 con 0x7efc641e62b0 2024-04-20T11:31:04.798 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.798+0000 7efc63e00640 1 -- 172.21.15.152:0/1400228763 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efc64072c90 con 0x7efc641e62b0 2024-04-20T11:31:04.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.801+0000 7efc5aa00640 1 -- 172.21.15.152:0/1400228763 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) ==== 74+0+16725 (secure 0 0 0) 0x7efc5005a180 con 0x7efc641e62b0 2024-04-20T11:31:04.802 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:31:04.802 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":60,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:31:01.845268+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:30:58.087822+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-20T11:30:59.281767+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:31:04.805 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.804+0000 7efc63e00640 1 -- 172.21.15.152:0/1400228763 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7efc4406caa0 msgr2=0x7efc4406ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:04.805 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.804+0000 7efc63e00640 1 --2- 172.21.15.152:0/1400228763 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7efc4406caa0 0x7efc4406ef60 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7efc54036000 tx=0x7efc54030000 comp rx=0 tx=0).stop 2024-04-20T11:31:04.805 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.804+0000 7efc63e00640 1 -- 172.21.15.152:0/1400228763 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efc641e62b0 msgr2=0x7efc6414abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:04.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.804+0000 7efc63e00640 1 --2- 172.21.15.152:0/1400228763 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efc641e62b0 0x7efc6414abe0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7efc5000d4b0 tx=0x7efc5000d970 comp rx=0 tx=0).stop 2024-04-20T11:31:04.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.805+0000 7efc63e00640 1 -- 172.21.15.152:0/1400228763 shutdown_connections 2024-04-20T11:31:04.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.805+0000 7efc63e00640 1 --2- 172.21.15.152:0/1400228763 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7efc4406caa0 0x7efc4406ef60 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:04.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.805+0000 7efc63e00640 1 --2- 172.21.15.152:0/1400228763 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efc640aecb0 0x7efc640abc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:04.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.805+0000 7efc63e00640 1 --2- 172.21.15.152:0/1400228763 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efc641e62b0 0x7efc6414abe0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:04.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.805+0000 7efc63e00640 1 --2- 172.21.15.152:0/1400228763 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efc480039b0 0x7efc641e78c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:04.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.805+0000 7efc63e00640 1 -- 172.21.15.152:0/1400228763 >> 172.21.15.152:0/1400228763 conn(0x7efc64067ae0 msgr2=0x7efc64159b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:04.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.805+0000 7efc63e00640 1 -- 172.21.15.152:0/1400228763 shutdown_connections 2024-04-20T11:31:04.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:04.805+0000 7efc63e00640 1 -- 172.21.15.152:0/1400228763 wait complete. 2024-04-20T11:31:04.825 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-20T11:31:04.826 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T11:31:05.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.024+0000 7f6bd7e00640 1 Processor -- start 2024-04-20T11:31:05.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.024+0000 7f6bd7e00640 1 -- start start 2024-04-20T11:31:05.026 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.025+0000 7f6bd7e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6bd815d1e0 0x7f6bd8160b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:05.026 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.025+0000 7f6bd7e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f6bd815d5c0 0x7f6bd80cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:05.026 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.025+0000 7f6bd7e00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6bd80ce8c0 con 0x7f6bd8071cd0 2024-04-20T11:31:05.026 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.025+0000 7f6bd7e00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f6bd81600b0 con 0x7f6bd815d5c0 2024-04-20T11:31:05.026 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.025+0000 7f6bd7e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6bd8160390 con 0x7f6bd815d1e0 2024-04-20T11:31:05.026 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.026+0000 7f6bd7400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f6bd8071cd0 0x7f6bd80ccd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:50934/0 (socket says 172.21.15.152:50934) 2024-04-20T11:31:05.026 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.026+0000 7f6bd7400640 1 -- 172.21.15.152:0/3175814875 learned_addr learned my addr 172.21.15.152:0/3175814875 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:31:05.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.026+0000 7f6bd6000640 1 -- 172.21.15.152:0/3175814875 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 453805953 0 0) 0x7f6bd80ce8c0 con 0x7f6bd8071cd0 2024-04-20T11:31:05.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.026+0000 7f6bd6000640 1 -- 172.21.15.152:0/3175814875 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6bbc003520 con 0x7f6bd8071cd0 2024-04-20T11:31:05.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.026+0000 7f6bce000640 1 --2- 172.21.15.152:0/3175814875 >> v2:172.21.15.175:3300/0 conn(0x7f6bd815d5c0 0x7f6bd80cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:05.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.026+0000 7f6bd6a00640 1 --2- 172.21.15.152:0/3175814875 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6bd815d1e0 0x7f6bd8160b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:05.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.026+0000 7f6bd6000640 1 -- 172.21.15.152:0/3175814875 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3305618999 0 0) 0x7f6bbc003520 con 0x7f6bd8071cd0 2024-04-20T11:31:05.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.026+0000 7f6bd6000640 1 -- 172.21.15.152:0/3175814875 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6bd815d1e0 msgr2=0x7f6bd8160b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:05.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.026+0000 7f6bd6000640 1 --2- 172.21.15.152:0/3175814875 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6bd815d1e0 0x7f6bd8160b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:05.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.026+0000 7f6bd6000640 1 -- 172.21.15.152:0/3175814875 >> v2:172.21.15.175:3300/0 conn(0x7f6bd815d5c0 msgr2=0x7f6bd80cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:05.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.026+0000 7f6bd6000640 1 --2- 172.21.15.152:0/3175814875 >> v2:172.21.15.175:3300/0 conn(0x7f6bd815d5c0 0x7f6bd80cdfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:05.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.026+0000 7f6bd6000640 1 -- 172.21.15.152:0/3175814875 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bd8160670 con 0x7f6bd8071cd0 2024-04-20T11:31:05.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.026+0000 7f6bd6000640 1 -- 172.21.15.152:0/3175814875 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f6bd0001670 con 0x7f6bd8071cd0 2024-04-20T11:31:05.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.026+0000 7f6bd6000640 1 -- 172.21.15.152:0/3175814875 >> v1:172.21.15.152:6789/0 conn(0x7f6bd8071cd0 legacy=0x7f6bd80ccd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:05.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.026+0000 7f6bd6000640 1 --2- 172.21.15.152:0/3175814875 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6bbc0039b0 0x7f6bbc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:05.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.026+0000 7f6bd6000640 1 --2- 172.21.15.152:0/3175814875 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6bd815d1e0 0x7f6bbc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:05.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.026+0000 7f6bd6000640 1 --2- 172.21.15.152:0/3175814875 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6bbc004d40 0x7f6bbc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:05.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.027+0000 7f6bd6000640 1 -- 172.21.15.152:0/3175814875 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f6bd81600b0 con 0x7f6bbc004d40 2024-04-20T11:31:05.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.027+0000 7f6bd6000640 1 -- 172.21.15.152:0/3175814875 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6bd8160390 con 0x7f6bbc0039b0 2024-04-20T11:31:05.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.027+0000 7f6bd6000640 1 -- 172.21.15.152:0/3175814875 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6bbc00baf0 con 0x7f6bd815d1e0 2024-04-20T11:31:05.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.027+0000 7f6bd7400640 1 --2- 172.21.15.152:0/3175814875 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6bbc004d40 0x7f6bbc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:05.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.027+0000 7f6bd6a00640 1 --2- 172.21.15.152:0/3175814875 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6bbc0039b0 0x7f6bbc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:05.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.027+0000 7f6bd7400640 1 -- 172.21.15.152:0/3175814875 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6bd815d1e0 msgr2=0x7f6bbc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:05.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.027+0000 7f6bd7400640 1 --2- 172.21.15.152:0/3175814875 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6bd815d1e0 0x7f6bbc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:05.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.027+0000 7f6bd7400640 1 -- 172.21.15.152:0/3175814875 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6bbc0039b0 msgr2=0x7f6bbc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:05.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.027+0000 7f6bd7400640 1 --2- 172.21.15.152:0/3175814875 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6bbc0039b0 0x7f6bbc003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:05.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.027+0000 7f6bd7400640 1 -- 172.21.15.152:0/3175814875 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bbc003520 con 0x7f6bbc004d40 2024-04-20T11:31:05.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.027+0000 7f6bd6a00640 1 --2- 172.21.15.152:0/3175814875 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6bbc0039b0 0x7f6bbc003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:31:05.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.028+0000 7f6bd7400640 1 --2- 172.21.15.152:0/3175814875 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6bbc004d40 0x7f6bbc0051c0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f6bd000c160 tx=0x7f6bbc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffdbcc8aef3819c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:05.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.029+0000 7f6bd6000640 1 -- 172.21.15.152:0/3175814875 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bd0007c80 con 0x7f6bbc004d40 2024-04-20T11:31:05.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.029+0000 7f6bd6000640 1 -- 172.21.15.152:0/3175814875 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bd0030070 con 0x7f6bbc004d40 2024-04-20T11:31:05.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.029+0000 7f6bd6000640 1 -- 172.21.15.152:0/3175814875 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bd0005d30 con 0x7f6bbc004d40 2024-04-20T11:31:05.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.029+0000 7f6bd7e00640 1 -- 172.21.15.152:0/3175814875 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6bbc004d40 msgr2=0x7f6bbc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:05.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.029+0000 7f6bd7e00640 1 --2- 172.21.15.152:0/3175814875 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6bbc004d40 0x7f6bbc0051c0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f6bd000c160 tx=0x7f6bbc005830 comp rx=0 tx=0).stop 2024-04-20T11:31:05.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.029+0000 7f6bd7400640 1 -- 172.21.15.152:0/3175814875 reap_dead start 2024-04-20T11:31:05.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.029+0000 7f6bd7e00640 1 -- 172.21.15.152:0/3175814875 shutdown_connections 2024-04-20T11:31:05.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.029+0000 7f6bd7e00640 1 -- 172.21.15.152:0/3175814875 >> 172.21.15.152:0/3175814875 conn(0x7f6bd8067ae0 msgr2=0x7f6bd806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:05.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.029+0000 7f6bd7e00640 1 -- 172.21.15.152:0/3175814875 shutdown_connections 2024-04-20T11:31:05.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.030+0000 7f6bd7e00640 1 -- 172.21.15.152:0/3175814875 wait complete. 2024-04-20T11:31:05.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.031+0000 7f6bd7e00640 1 Processor -- start 2024-04-20T11:31:05.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.031+0000 7f6bd7e00640 1 -- start start 2024-04-20T11:31:05.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.032+0000 7f6bd7e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6bbc004d40 0x7f6bd81f6040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:05.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.032+0000 7f6bd7e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6bd81473e0 0x7f6bd8144ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:05.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.032+0000 7f6bd7e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6bd80ac7a0 0x7f6bd80acc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:05.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.032+0000 7f6bd7e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f6bd80ce6a0 con 0x7f6bd81473e0 2024-04-20T11:31:05.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.032+0000 7f6bd7e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6bd80ce520 con 0x7f6bbc004d40 2024-04-20T11:31:05.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.032+0000 7f6bd7e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6bd8160b90 con 0x7f6bd80ac7a0 2024-04-20T11:31:05.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.032+0000 7f6bce000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6bd81473e0 0x7f6bd8144ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:05.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.033+0000 7f6bce000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6bd81473e0 0x7f6bd8144ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:47678/0 (socket says 172.21.15.152:47678) 2024-04-20T11:31:05.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.033+0000 7f6bce000640 1 -- 172.21.15.152:0/1121146119 learned_addr learned my addr 172.21.15.152:0/1121146119 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:05.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.033+0000 7f6bce000640 1 -- 172.21.15.152:0/1121146119 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6bd80ac7a0 msgr2=0x7f6bd80acc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:05.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.033+0000 7f6bce000640 1 --2- 172.21.15.152:0/1121146119 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6bd80ac7a0 0x7f6bd80acc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:05.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.033+0000 7f6bce000640 1 -- 172.21.15.152:0/1121146119 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6bbc004d40 msgr2=0x7f6bd81f6040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:05.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.033+0000 7f6bce000640 1 --2- 172.21.15.152:0/1121146119 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6bbc004d40 0x7f6bd81f6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:05.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.033+0000 7f6bce000640 1 -- 172.21.15.152:0/1121146119 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bd8145020 con 0x7f6bd81473e0 2024-04-20T11:31:05.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.033+0000 7f6bce000640 1 --2- 172.21.15.152:0/1121146119 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6bd81473e0 0x7f6bd8144ae0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f6bc0007e10 tx=0x7f6bc000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:05.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.033+0000 7f6bcf400640 1 -- 172.21.15.152:0/1121146119 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bc0014620 con 0x7f6bd81473e0 2024-04-20T11:31:05.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.033+0000 7f6bcf400640 1 -- 172.21.15.152:0/1121146119 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bc0019070 con 0x7f6bd81473e0 2024-04-20T11:31:05.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.033+0000 7f6bd7e00640 1 -- 172.21.15.152:0/1121146119 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bd8147870 con 0x7f6bd81473e0 2024-04-20T11:31:05.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.034+0000 7f6bcf400640 1 -- 172.21.15.152:0/1121146119 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bc0014a20 con 0x7f6bd81473e0 2024-04-20T11:31:05.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.034+0000 7f6bd7e00640 1 -- 172.21.15.152:0/1121146119 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bd8147e30 con 0x7f6bd81473e0 2024-04-20T11:31:05.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.035+0000 7f6bd7e00640 1 -- 172.21.15.152:0/1121146119 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bd8075e60 con 0x7f6bd81473e0 2024-04-20T11:31:05.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.036+0000 7f6bcf400640 1 -- 172.21.15.152:0/1121146119 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 9) ==== 91193+0+0 (secure 0 0 0) 0x7f6bc0014bc0 con 0x7f6bd81473e0 2024-04-20T11:31:05.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.037+0000 7f6bcf400640 1 --2- 172.21.15.152:0/1121146119 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f6bb406caa0 0x7f6bb406ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:05.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.037+0000 7f6bcf400640 1 -- 172.21.15.152:0/1121146119 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5709+0+0 (secure 0 0 0) 0x7f6bc008c7a0 con 0x7f6bd81473e0 2024-04-20T11:31:05.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.037+0000 7f6bd6a00640 1 --2- 172.21.15.152:0/1121146119 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f6bb406caa0 0x7f6bb406ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:05.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.038+0000 7f6bd6a00640 1 --2- 172.21.15.152:0/1121146119 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f6bb406caa0 0x7f6bb406ef60 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f6bc8006fd0 tx=0x7f6bc8009350 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:05.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.041+0000 7f6bcf400640 1 -- 172.21.15.152:0/1121146119 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f6bc005a8b0 con 0x7f6bd81473e0 2024-04-20T11:31:05.209 INFO:tasks.ceph.mds.c.smithi152.stderr:2024-04-20T11:31:05.208+0000 7f01c6400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-20T11:31:05.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.238+0000 7f6bd7e00640 1 -- 172.21.15.152:0/1121146119 --> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6bd8072c90 con 0x7f6bb406caa0 2024-04-20T11:31:05.257 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.257+0000 7f6bcf400640 1 -- 172.21.15.152:0/1121146119 <== mgr.4113 v2:172.21.15.152:6832/2285110240 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251911 (secure 0 0 0) 0x7f6bd8072c90 con 0x7f6bb406caa0 2024-04-20T11:31:05.259 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:31:05.263 INFO:teuthology.orchestra.run.smithi152.stderr:dumped all 2024-04-20T11:31:05.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.264+0000 7f6bd7e00640 1 -- 172.21.15.152:0/1121146119 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f6bb406caa0 msgr2=0x7f6bb406ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:05.265 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.264+0000 7f6bd7e00640 1 --2- 172.21.15.152:0/1121146119 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f6bb406caa0 0x7f6bb406ef60 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f6bc8006fd0 tx=0x7f6bc8009350 comp rx=0 tx=0).stop 2024-04-20T11:31:05.265 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.264+0000 7f6bd7e00640 1 -- 172.21.15.152:0/1121146119 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6bd81473e0 msgr2=0x7f6bd8144ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:05.265 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.264+0000 7f6bd7e00640 1 --2- 172.21.15.152:0/1121146119 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6bd81473e0 0x7f6bd8144ae0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f6bc0007e10 tx=0x7f6bc000e4b0 comp rx=0 tx=0).stop 2024-04-20T11:31:05.265 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.264+0000 7f6bd7e00640 1 -- 172.21.15.152:0/1121146119 shutdown_connections 2024-04-20T11:31:05.265 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.264+0000 7f6bd7e00640 1 --2- 172.21.15.152:0/1121146119 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f6bb406caa0 0x7f6bb406ef60 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:05.265 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.264+0000 7f6bd7e00640 1 --2- 172.21.15.152:0/1121146119 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6bd80ac7a0 0x7f6bd80acc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:05.265 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.264+0000 7f6bd7e00640 1 --2- 172.21.15.152:0/1121146119 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6bd81473e0 0x7f6bd8144ae0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:05.265 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.264+0000 7f6bd7e00640 1 --2- 172.21.15.152:0/1121146119 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6bbc004d40 0x7f6bd81f6040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:05.265 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.264+0000 7f6bd7e00640 1 -- 172.21.15.152:0/1121146119 >> 172.21.15.152:0/1121146119 conn(0x7f6bd8067ae0 msgr2=0x7f6bd806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:05.265 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.264+0000 7f6bd7e00640 1 -- 172.21.15.152:0/1121146119 shutdown_connections 2024-04-20T11:31:05.265 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.264+0000 7f6bd7e00640 1 -- 172.21.15.152:0/1121146119 wait complete. 2024-04-20T11:31:05.284 INFO:teuthology.orchestra.run.smithi152.stdout:{"pg_ready":true,"pg_map":{"version":181,"stamp":"2024-04-20T11:31:03.295865+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":243744,"kb_used_data":6772,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754730976,"statfs":{"total":773094113280,"available":772844519424,"internally_reserved":0,"allocated":6934528,"data_stored":3555960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12746,"internal_metadata":242535990},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":44,"apply_latency_ms":44,"commit_latency_ns":44000000,"apply_latency_ns":44000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.462443"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-20T11:31:01.854480+0000","last_change":"2024-04-20T11:31:00.879013+0000","last_active":"2024-04-20T11:31:01.854480+0000","last_peered":"2024-04-20T11:31:01.854480+0000","last_clean":"2024-04-20T11:31:01.854480+0000","last_became_active":"2024-04-20T11:31:00.878751+0000","last_became_peered":"2024-04-20T11:31:00.878751+0000","last_unstale":"2024-04-20T11:31:01.854480+0000","last_undegraded":"2024-04-20T11:31:01.854480+0000","last_fullsized":"2024-04-20T11:31:01.854480+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:14:19.396693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-20T11:31:01.857367+0000","last_change":"2024-04-20T11:31:00.872086+0000","last_active":"2024-04-20T11:31:01.857367+0000","last_peered":"2024-04-20T11:31:01.857367+0000","last_clean":"2024-04-20T11:31:01.857367+0000","last_became_active":"2024-04-20T11:31:00.871806+0000","last_became_peered":"2024-04-20T11:31:00.871806+0000","last_unstale":"2024-04-20T11:31:01.857367+0000","last_undegraded":"2024-04-20T11:31:01.857367+0000","last_fullsized":"2024-04-20T11:31:01.857367+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:20:01.273117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.903978+0000","last_change":"2024-04-20T11:31:00.902101+0000","last_active":"2024-04-20T11:31:00.903978+0000","last_peered":"2024-04-20T11:31:00.903978+0000","last_clean":"2024-04-20T11:31:00.903978+0000","last_became_active":"2024-04-20T11:31:00.901339+0000","last_became_peered":"2024-04-20T11:31:00.901339+0000","last_unstale":"2024-04-20T11:31:00.903978+0000","last_undegraded":"2024-04-20T11:31:00.903978+0000","last_fullsized":"2024-04-20T11:31:00.903978+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:21:51.908569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-20T11:31:01.856405+0000","last_change":"2024-04-20T11:31:00.883684+0000","last_active":"2024-04-20T11:31:01.856405+0000","last_peered":"2024-04-20T11:31:01.856405+0000","last_clean":"2024-04-20T11:31:01.856405+0000","last_became_active":"2024-04-20T11:31:00.883381+0000","last_became_peered":"2024-04-20T11:31:00.883381+0000","last_unstale":"2024-04-20T11:31:01.856405+0000","last_undegraded":"2024-04-20T11:31:01.856405+0000","last_fullsized":"2024-04-20T11:31:01.856405+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:42:53.717064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-20T11:31:01.854373+0000","last_change":"2024-04-20T11:31:00.869648+0000","last_active":"2024-04-20T11:31:01.854373+0000","last_peered":"2024-04-20T11:31:01.854373+0000","last_clean":"2024-04-20T11:31:01.854373+0000","last_became_active":"2024-04-20T11:31:00.869335+0000","last_became_peered":"2024-04-20T11:31:00.869335+0000","last_unstale":"2024-04-20T11:31:01.854373+0000","last_undegraded":"2024-04-20T11:31:01.854373+0000","last_fullsized":"2024-04-20T11:31:01.854373+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:35:08.994877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.882526+0000","last_change":"2024-04-20T11:31:00.875405+0000","last_active":"2024-04-20T11:31:00.882526+0000","last_peered":"2024-04-20T11:31:00.882526+0000","last_clean":"2024-04-20T11:31:00.882526+0000","last_became_active":"2024-04-20T11:31:00.875110+0000","last_became_peered":"2024-04-20T11:31:00.875110+0000","last_unstale":"2024-04-20T11:31:00.882526+0000","last_undegraded":"2024-04-20T11:31:00.882526+0000","last_fullsized":"2024-04-20T11:31:00.882526+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:36:24.761945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.877585+0000","last_change":"2024-04-20T11:31:00.868812+0000","last_active":"2024-04-20T11:31:00.877585+0000","last_peered":"2024-04-20T11:31:00.877585+0000","last_clean":"2024-04-20T11:31:00.877585+0000","last_became_active":"2024-04-20T11:31:00.868378+0000","last_became_peered":"2024-04-20T11:31:00.868378+0000","last_unstale":"2024-04-20T11:31:00.877585+0000","last_undegraded":"2024-04-20T11:31:00.877585+0000","last_fullsized":"2024-04-20T11:31:00.877585+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:24:45.359945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-20T11:31:01.856909+0000","last_change":"2024-04-20T11:31:00.879298+0000","last_active":"2024-04-20T11:31:01.856909+0000","last_peered":"2024-04-20T11:31:01.856909+0000","last_clean":"2024-04-20T11:31:01.856909+0000","last_became_active":"2024-04-20T11:31:00.879024+0000","last_became_peered":"2024-04-20T11:31:00.879024+0000","last_unstale":"2024-04-20T11:31:01.856909+0000","last_undegraded":"2024-04-20T11:31:01.856909+0000","last_fullsized":"2024-04-20T11:31:01.856909+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:51:52.380825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.878664+0000","last_change":"2024-04-20T11:31:00.873632+0000","last_active":"2024-04-20T11:31:00.878664+0000","last_peered":"2024-04-20T11:31:00.878664+0000","last_clean":"2024-04-20T11:31:00.878664+0000","last_became_active":"2024-04-20T11:31:00.873353+0000","last_became_peered":"2024-04-20T11:31:00.873353+0000","last_unstale":"2024-04-20T11:31:00.878664+0000","last_undegraded":"2024-04-20T11:31:00.878664+0000","last_fullsized":"2024-04-20T11:31:00.878664+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:06:49.119572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.893272+0000","last_change":"2024-04-20T11:31:00.886226+0000","last_active":"2024-04-20T11:31:00.893272+0000","last_peered":"2024-04-20T11:31:00.893272+0000","last_clean":"2024-04-20T11:31:00.893272+0000","last_became_active":"2024-04-20T11:31:00.885947+0000","last_became_peered":"2024-04-20T11:31:00.885947+0000","last_unstale":"2024-04-20T11:31:00.893272+0000","last_undegraded":"2024-04-20T11:31:00.893272+0000","last_fullsized":"2024-04-20T11:31:00.893272+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:18:25.731153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.894018+0000","last_change":"2024-04-20T11:31:00.891786+0000","last_active":"2024-04-20T11:31:00.894018+0000","last_peered":"2024-04-20T11:31:00.894018+0000","last_clean":"2024-04-20T11:31:00.894018+0000","last_became_active":"2024-04-20T11:31:00.891520+0000","last_became_peered":"2024-04-20T11:31:00.891520+0000","last_unstale":"2024-04-20T11:31:00.894018+0000","last_undegraded":"2024-04-20T11:31:00.894018+0000","last_fullsized":"2024-04-20T11:31:00.894018+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:20:10.518047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-20T11:31:01.856518+0000","last_change":"2024-04-20T11:31:00.891872+0000","last_active":"2024-04-20T11:31:01.856518+0000","last_peered":"2024-04-20T11:31:01.856518+0000","last_clean":"2024-04-20T11:31:01.856518+0000","last_became_active":"2024-04-20T11:31:00.891551+0000","last_became_peered":"2024-04-20T11:31:00.891551+0000","last_unstale":"2024-04-20T11:31:01.856518+0000","last_undegraded":"2024-04-20T11:31:01.856518+0000","last_fullsized":"2024-04-20T11:31:01.856518+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:39:57.271994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.886628+0000","last_change":"2024-04-20T11:31:00.880969+0000","last_active":"2024-04-20T11:31:00.886628+0000","last_peered":"2024-04-20T11:31:00.886628+0000","last_clean":"2024-04-20T11:31:00.886628+0000","last_became_active":"2024-04-20T11:31:00.880640+0000","last_became_peered":"2024-04-20T11:31:00.880640+0000","last_unstale":"2024-04-20T11:31:00.886628+0000","last_undegraded":"2024-04-20T11:31:00.886628+0000","last_fullsized":"2024-04-20T11:31:00.886628+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:30:16.658442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.889307+0000","last_change":"2024-04-20T11:31:00.877890+0000","last_active":"2024-04-20T11:31:00.889307+0000","last_peered":"2024-04-20T11:31:00.889307+0000","last_clean":"2024-04-20T11:31:00.889307+0000","last_became_active":"2024-04-20T11:31:00.877468+0000","last_became_peered":"2024-04-20T11:31:00.877468+0000","last_unstale":"2024-04-20T11:31:00.889307+0000","last_undegraded":"2024-04-20T11:31:00.889307+0000","last_fullsized":"2024-04-20T11:31:00.889307+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:22:28.047678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.877614+0000","last_change":"2024-04-20T11:31:00.873622+0000","last_active":"2024-04-20T11:31:00.877614+0000","last_peered":"2024-04-20T11:31:00.877614+0000","last_clean":"2024-04-20T11:31:00.877614+0000","last_became_active":"2024-04-20T11:31:00.873174+0000","last_became_peered":"2024-04-20T11:31:00.873174+0000","last_unstale":"2024-04-20T11:31:00.877614+0000","last_undegraded":"2024-04-20T11:31:00.877614+0000","last_fullsized":"2024-04-20T11:31:00.877614+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:54:15.217821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.893306+0000","last_change":"2024-04-20T11:31:00.886578+0000","last_active":"2024-04-20T11:31:00.893306+0000","last_peered":"2024-04-20T11:31:00.893306+0000","last_clean":"2024-04-20T11:31:00.893306+0000","last_became_active":"2024-04-20T11:31:00.886200+0000","last_became_peered":"2024-04-20T11:31:00.886200+0000","last_unstale":"2024-04-20T11:31:00.893306+0000","last_undegraded":"2024-04-20T11:31:00.893306+0000","last_fullsized":"2024-04-20T11:31:00.893306+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:39:57.271994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-20T11:31:01.858562+0000","last_change":"2024-04-20T11:31:00.891337+0000","last_active":"2024-04-20T11:31:01.858562+0000","last_peered":"2024-04-20T11:31:01.858562+0000","last_clean":"2024-04-20T11:31:01.858562+0000","last_became_active":"2024-04-20T11:31:00.890773+0000","last_became_peered":"2024-04-20T11:31:00.890773+0000","last_unstale":"2024-04-20T11:31:01.858562+0000","last_undegraded":"2024-04-20T11:31:01.858562+0000","last_fullsized":"2024-04-20T11:31:01.858562+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:32:40.001175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.903429+0000","last_change":"2024-04-20T11:31:00.895120+0000","last_active":"2024-04-20T11:31:00.903429+0000","last_peered":"2024-04-20T11:31:00.903429+0000","last_clean":"2024-04-20T11:31:00.903429+0000","last_became_active":"2024-04-20T11:31:00.894826+0000","last_became_peered":"2024-04-20T11:31:00.894826+0000","last_unstale":"2024-04-20T11:31:00.903429+0000","last_undegraded":"2024-04-20T11:31:00.903429+0000","last_fullsized":"2024-04-20T11:31:00.903429+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:26:37.214795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-20T11:31:01.855003+0000","last_change":"2024-04-20T11:31:00.878393+0000","last_active":"2024-04-20T11:31:01.855003+0000","last_peered":"2024-04-20T11:31:01.855003+0000","last_clean":"2024-04-20T11:31:01.855003+0000","last_became_active":"2024-04-20T11:31:00.878113+0000","last_became_peered":"2024-04-20T11:31:00.878113+0000","last_unstale":"2024-04-20T11:31:01.855003+0000","last_undegraded":"2024-04-20T11:31:01.855003+0000","last_fullsized":"2024-04-20T11:31:01.855003+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:21:07.403914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-20T11:31:01.857937+0000","last_change":"2024-04-20T11:31:00.870156+0000","last_active":"2024-04-20T11:31:01.857937+0000","last_peered":"2024-04-20T11:31:01.857937+0000","last_clean":"2024-04-20T11:31:01.857937+0000","last_became_active":"2024-04-20T11:31:00.869890+0000","last_became_peered":"2024-04-20T11:31:00.869890+0000","last_unstale":"2024-04-20T11:31:01.857937+0000","last_undegraded":"2024-04-20T11:31:01.857937+0000","last_fullsized":"2024-04-20T11:31:01.857937+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:29:11.139899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.903482+0000","last_change":"2024-04-20T11:31:00.896450+0000","last_active":"2024-04-20T11:31:00.903482+0000","last_peered":"2024-04-20T11:31:00.903482+0000","last_clean":"2024-04-20T11:31:00.903482+0000","last_became_active":"2024-04-20T11:31:00.896066+0000","last_became_peered":"2024-04-20T11:31:00.896066+0000","last_unstale":"2024-04-20T11:31:00.903482+0000","last_undegraded":"2024-04-20T11:31:00.903482+0000","last_fullsized":"2024-04-20T11:31:00.903482+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:36:24.761945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-20T11:31:01.858121+0000","last_change":"2024-04-20T11:31:00.883409+0000","last_active":"2024-04-20T11:31:01.858121+0000","last_peered":"2024-04-20T11:31:01.858121+0000","last_clean":"2024-04-20T11:31:01.858121+0000","last_became_active":"2024-04-20T11:31:00.883100+0000","last_became_peered":"2024-04-20T11:31:00.883100+0000","last_unstale":"2024-04-20T11:31:01.858121+0000","last_undegraded":"2024-04-20T11:31:01.858121+0000","last_fullsized":"2024-04-20T11:31:01.858121+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:55:05.047669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.903388+0000","last_change":"2024-04-20T11:31:00.893500+0000","last_active":"2024-04-20T11:31:00.903388+0000","last_peered":"2024-04-20T11:31:00.903388+0000","last_clean":"2024-04-20T11:31:00.903388+0000","last_became_active":"2024-04-20T11:31:00.893204+0000","last_became_peered":"2024-04-20T11:31:00.893204+0000","last_unstale":"2024-04-20T11:31:00.903388+0000","last_undegraded":"2024-04-20T11:31:00.903388+0000","last_fullsized":"2024-04-20T11:31:00.903388+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:02:57.616897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.886593+0000","last_change":"2024-04-20T11:31:00.883232+0000","last_active":"2024-04-20T11:31:00.886593+0000","last_peered":"2024-04-20T11:31:00.886593+0000","last_clean":"2024-04-20T11:31:00.886593+0000","last_became_active":"2024-04-20T11:31:00.882820+0000","last_became_peered":"2024-04-20T11:31:00.882820+0000","last_unstale":"2024-04-20T11:31:00.886593+0000","last_undegraded":"2024-04-20T11:31:00.886593+0000","last_fullsized":"2024-04-20T11:31:00.886593+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:13:11.698885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-20T11:31:01.854772+0000","last_change":"2024-04-20T11:31:00.861347+0000","last_active":"2024-04-20T11:31:01.854772+0000","last_peered":"2024-04-20T11:31:01.854772+0000","last_clean":"2024-04-20T11:31:01.854772+0000","last_became_active":"2024-04-20T11:31:00.861067+0000","last_became_peered":"2024-04-20T11:31:00.861067+0000","last_unstale":"2024-04-20T11:31:01.854772+0000","last_undegraded":"2024-04-20T11:31:01.854772+0000","last_fullsized":"2024-04-20T11:31:01.854772+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:06:39.367242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-20T11:31:01.856802+0000","last_change":"2024-04-20T11:31:00.879243+0000","last_active":"2024-04-20T11:31:01.856802+0000","last_peered":"2024-04-20T11:31:01.856802+0000","last_clean":"2024-04-20T11:31:01.856802+0000","last_became_active":"2024-04-20T11:31:00.878706+0000","last_became_peered":"2024-04-20T11:31:00.878706+0000","last_unstale":"2024-04-20T11:31:01.856802+0000","last_undegraded":"2024-04-20T11:31:01.856802+0000","last_fullsized":"2024-04-20T11:31:01.856802+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:23:22.124914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.893526+0000","last_change":"2024-04-20T11:31:00.888228+0000","last_active":"2024-04-20T11:31:00.893526+0000","last_peered":"2024-04-20T11:31:00.893526+0000","last_clean":"2024-04-20T11:31:00.893526+0000","last_became_active":"2024-04-20T11:31:00.887818+0000","last_became_peered":"2024-04-20T11:31:00.887818+0000","last_unstale":"2024-04-20T11:31:00.893526+0000","last_undegraded":"2024-04-20T11:31:00.893526+0000","last_fullsized":"2024-04-20T11:31:00.893526+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:04:46.835057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-20T11:31:01.854801+0000","last_change":"2024-04-20T11:31:00.873526+0000","last_active":"2024-04-20T11:31:01.854801+0000","last_peered":"2024-04-20T11:31:01.854801+0000","last_clean":"2024-04-20T11:31:01.854801+0000","last_became_active":"2024-04-20T11:31:00.873268+0000","last_became_peered":"2024-04-20T11:31:00.873268+0000","last_unstale":"2024-04-20T11:31:01.854801+0000","last_undegraded":"2024-04-20T11:31:01.854801+0000","last_fullsized":"2024-04-20T11:31:01.854801+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:53:40.226736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-20T11:31:01.857124+0000","last_change":"2024-04-20T11:31:00.875115+0000","last_active":"2024-04-20T11:31:01.857124+0000","last_peered":"2024-04-20T11:31:01.857124+0000","last_clean":"2024-04-20T11:31:01.857124+0000","last_became_active":"2024-04-20T11:31:00.874772+0000","last_became_peered":"2024-04-20T11:31:00.874772+0000","last_unstale":"2024-04-20T11:31:01.857124+0000","last_undegraded":"2024-04-20T11:31:01.857124+0000","last_fullsized":"2024-04-20T11:31:01.857124+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:22:28.866085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.903522+0000","last_change":"2024-04-20T11:31:00.898319+0000","last_active":"2024-04-20T11:31:00.903522+0000","last_peered":"2024-04-20T11:31:00.903522+0000","last_clean":"2024-04-20T11:31:00.903522+0000","last_became_active":"2024-04-20T11:31:00.897940+0000","last_became_peered":"2024-04-20T11:31:00.897940+0000","last_unstale":"2024-04-20T11:31:00.903522+0000","last_undegraded":"2024-04-20T11:31:00.903522+0000","last_fullsized":"2024-04-20T11:31:00.903522+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:47:44.049807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-20T11:31:01.854159+0000","last_change":"2024-04-20T11:31:00.870669+0000","last_active":"2024-04-20T11:31:01.854159+0000","last_peered":"2024-04-20T11:31:01.854159+0000","last_clean":"2024-04-20T11:31:01.854159+0000","last_became_active":"2024-04-20T11:31:00.870362+0000","last_became_peered":"2024-04-20T11:31:00.870362+0000","last_unstale":"2024-04-20T11:31:01.854159+0000","last_undegraded":"2024-04-20T11:31:01.854159+0000","last_fullsized":"2024-04-20T11:31:01.854159+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:50:01.584349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.884365+0000","last_change":"2024-04-20T11:31:00.881176+0000","last_active":"2024-04-20T11:31:00.884365+0000","last_peered":"2024-04-20T11:31:00.884365+0000","last_clean":"2024-04-20T11:31:00.884365+0000","last_became_active":"2024-04-20T11:31:00.880875+0000","last_became_peered":"2024-04-20T11:31:00.880875+0000","last_unstale":"2024-04-20T11:31:00.884365+0000","last_undegraded":"2024-04-20T11:31:00.884365+0000","last_fullsized":"2024-04-20T11:31:00.884365+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:42:53.717064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.893323+0000","last_change":"2024-04-20T11:31:00.888829+0000","last_active":"2024-04-20T11:31:00.893323+0000","last_peered":"2024-04-20T11:31:00.893323+0000","last_clean":"2024-04-20T11:31:00.893323+0000","last_became_active":"2024-04-20T11:31:00.888548+0000","last_became_peered":"2024-04-20T11:31:00.888548+0000","last_unstale":"2024-04-20T11:31:00.893323+0000","last_undegraded":"2024-04-20T11:31:00.893323+0000","last_fullsized":"2024-04-20T11:31:00.893323+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:22:38.955596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"59'4","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.890262+0000","last_change":"2024-04-20T11:30:59.878500+0000","last_active":"2024-04-20T11:31:00.890262+0000","last_peered":"2024-04-20T11:31:00.890262+0000","last_clean":"2024-04-20T11:31:00.890262+0000","last_became_active":"2024-04-20T11:30:59.878178+0000","last_became_peered":"2024-04-20T11:30:59.878178+0000","last_unstale":"2024-04-20T11:31:00.890262+0000","last_undegraded":"2024-04-20T11:31:00.890262+0000","last_fullsized":"2024-04-20T11:31:00.890262+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_clean_scrub_stamp":"2024-04-20T11:30:58.811830+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:02:56.600968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.893441+0000","last_change":"2024-04-20T11:31:00.887702+0000","last_active":"2024-04-20T11:31:00.893441+0000","last_peered":"2024-04-20T11:31:00.893441+0000","last_clean":"2024-04-20T11:31:00.893441+0000","last_became_active":"2024-04-20T11:31:00.887402+0000","last_became_peered":"2024-04-20T11:31:00.887402+0000","last_unstale":"2024-04-20T11:31:00.893441+0000","last_undegraded":"2024-04-20T11:31:00.893441+0000","last_fullsized":"2024-04-20T11:31:00.893441+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:42:15.418923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"59'1","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-20T11:31:01.858195+0000","last_change":"2024-04-20T11:30:59.880713+0000","last_active":"2024-04-20T11:31:01.858195+0000","last_peered":"2024-04-20T11:31:01.858195+0000","last_clean":"2024-04-20T11:31:01.858195+0000","last_became_active":"2024-04-20T11:30:59.880447+0000","last_became_peered":"2024-04-20T11:30:59.880447+0000","last_unstale":"2024-04-20T11:31:01.858195+0000","last_undegraded":"2024-04-20T11:31:01.858195+0000","last_fullsized":"2024-04-20T11:31:01.858195+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_clean_scrub_stamp":"2024-04-20T11:30:58.811830+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:21:50.892640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.879678+0000","last_change":"2024-04-20T11:31:00.872261+0000","last_active":"2024-04-20T11:31:00.879678+0000","last_peered":"2024-04-20T11:31:00.879678+0000","last_clean":"2024-04-20T11:31:00.879678+0000","last_became_active":"2024-04-20T11:31:00.872001+0000","last_became_peered":"2024-04-20T11:31:00.872001+0000","last_unstale":"2024-04-20T11:31:00.879678+0000","last_undegraded":"2024-04-20T11:31:00.879678+0000","last_fullsized":"2024-04-20T11:31:00.879678+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:49:17.613752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"59'2","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.888407+0000","last_change":"2024-04-20T11:30:59.873756+0000","last_active":"2024-04-20T11:31:00.888407+0000","last_peered":"2024-04-20T11:31:00.888407+0000","last_clean":"2024-04-20T11:31:00.888407+0000","last_became_active":"2024-04-20T11:30:59.873042+0000","last_became_peered":"2024-04-20T11:30:59.873042+0000","last_unstale":"2024-04-20T11:31:00.888407+0000","last_undegraded":"2024-04-20T11:31:00.888407+0000","last_fullsized":"2024-04-20T11:31:00.888407+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_clean_scrub_stamp":"2024-04-20T11:30:58.811830+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:26:36.198866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.882719+0000","last_change":"2024-04-20T11:31:00.877357+0000","last_active":"2024-04-20T11:31:00.882719+0000","last_peered":"2024-04-20T11:31:00.882719+0000","last_clean":"2024-04-20T11:31:00.882719+0000","last_became_active":"2024-04-20T11:31:00.877067+0000","last_became_peered":"2024-04-20T11:31:00.877067+0000","last_unstale":"2024-04-20T11:31:00.882719+0000","last_undegraded":"2024-04-20T11:31:00.882719+0000","last_fullsized":"2024-04-20T11:31:00.882719+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:47:44.049807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"59'2","reported_seq":16,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-20T11:31:01.856685+0000","last_change":"2024-04-20T11:30:59.878107+0000","last_active":"2024-04-20T11:31:01.856685+0000","last_peered":"2024-04-20T11:31:01.856685+0000","last_clean":"2024-04-20T11:31:01.856685+0000","last_became_active":"2024-04-20T11:30:59.877737+0000","last_became_peered":"2024-04-20T11:30:59.877737+0000","last_unstale":"2024-04-20T11:31:01.856685+0000","last_undegraded":"2024-04-20T11:31:01.856685+0000","last_fullsized":"2024-04-20T11:31:01.856685+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_clean_scrub_stamp":"2024-04-20T11:30:58.811830+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:54:14.201891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-20T11:31:01.857718+0000","last_change":"2024-04-20T11:31:00.884954+0000","last_active":"2024-04-20T11:31:01.857718+0000","last_peered":"2024-04-20T11:31:01.857718+0000","last_clean":"2024-04-20T11:31:01.857718+0000","last_became_active":"2024-04-20T11:31:00.884681+0000","last_became_peered":"2024-04-20T11:31:00.884681+0000","last_unstale":"2024-04-20T11:31:01.857718+0000","last_undegraded":"2024-04-20T11:31:01.857718+0000","last_fullsized":"2024-04-20T11:31:01.857718+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:38:19.059089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"59'1","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.893083+0000","last_change":"2024-04-20T11:30:59.881895+0000","last_active":"2024-04-20T11:31:00.893083+0000","last_peered":"2024-04-20T11:31:00.893083+0000","last_clean":"2024-04-20T11:31:00.893083+0000","last_became_active":"2024-04-20T11:30:59.881025+0000","last_became_peered":"2024-04-20T11:30:59.881025+0000","last_unstale":"2024-04-20T11:31:00.893083+0000","last_undegraded":"2024-04-20T11:31:00.893083+0000","last_fullsized":"2024-04-20T11:31:00.893083+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_clean_scrub_stamp":"2024-04-20T11:30:58.811830+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:27:21.889130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-20T11:31:01.857282+0000","last_change":"2024-04-20T11:31:00.871560+0000","last_active":"2024-04-20T11:31:01.857282+0000","last_peered":"2024-04-20T11:31:01.857282+0000","last_clean":"2024-04-20T11:31:01.857282+0000","last_became_active":"2024-04-20T11:31:00.871247+0000","last_became_peered":"2024-04-20T11:31:00.871247+0000","last_unstale":"2024-04-20T11:31:01.857282+0000","last_undegraded":"2024-04-20T11:31:01.857282+0000","last_fullsized":"2024-04-20T11:31:01.857282+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:11:35.588117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"59'1","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.887643+0000","last_change":"2024-04-20T11:30:59.868051+0000","last_active":"2024-04-20T11:31:00.887643+0000","last_peered":"2024-04-20T11:31:00.887643+0000","last_clean":"2024-04-20T11:31:00.887643+0000","last_became_active":"2024-04-20T11:30:59.867699+0000","last_became_peered":"2024-04-20T11:30:59.867699+0000","last_unstale":"2024-04-20T11:31:00.887643+0000","last_undegraded":"2024-04-20T11:31:00.887643+0000","last_fullsized":"2024-04-20T11:31:00.887643+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_clean_scrub_stamp":"2024-04-20T11:30:58.811830+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:22:27.031748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.902356+0000","last_change":"2024-04-20T11:31:00.892797+0000","last_active":"2024-04-20T11:31:00.902356+0000","last_peered":"2024-04-20T11:31:00.902356+0000","last_clean":"2024-04-20T11:31:00.902356+0000","last_became_active":"2024-04-20T11:31:00.892484+0000","last_became_peered":"2024-04-20T11:31:00.892484+0000","last_unstale":"2024-04-20T11:31:00.902356+0000","last_undegraded":"2024-04-20T11:31:00.902356+0000","last_fullsized":"2024-04-20T11:31:00.902356+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:47:35.633763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-20T11:31:01.855697+0000","last_change":"2024-04-20T11:30:59.878984+0000","last_active":"2024-04-20T11:31:01.855697+0000","last_peered":"2024-04-20T11:31:01.855697+0000","last_clean":"2024-04-20T11:31:01.855697+0000","last_became_active":"2024-04-20T11:30:59.878458+0000","last_became_peered":"2024-04-20T11:30:59.878458+0000","last_unstale":"2024-04-20T11:31:01.855697+0000","last_undegraded":"2024-04-20T11:31:01.855697+0000","last_fullsized":"2024-04-20T11:31:01.855697+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_clean_scrub_stamp":"2024-04-20T11:30:58.811830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:24:44.344016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-20T11:31:01.857523+0000","last_change":"2024-04-20T11:31:00.889857+0000","last_active":"2024-04-20T11:31:01.857523+0000","last_peered":"2024-04-20T11:31:01.857523+0000","last_clean":"2024-04-20T11:31:01.857523+0000","last_became_active":"2024-04-20T11:31:00.889606+0000","last_became_peered":"2024-04-20T11:31:00.889606+0000","last_unstale":"2024-04-20T11:31:01.857523+0000","last_undegraded":"2024-04-20T11:31:01.857523+0000","last_fullsized":"2024-04-20T11:31:01.857523+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:06:58.334912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.855995+0000","last_change":"2024-04-20T11:30:59.872604+0000","last_active":"2024-04-20T11:31:00.855995+0000","last_peered":"2024-04-20T11:31:00.855995+0000","last_clean":"2024-04-20T11:31:00.855995+0000","last_became_active":"2024-04-20T11:30:59.872298+0000","last_became_peered":"2024-04-20T11:30:59.872298+0000","last_unstale":"2024-04-20T11:31:00.855995+0000","last_undegraded":"2024-04-20T11:31:00.855995+0000","last_fullsized":"2024-04-20T11:31:00.855995+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_clean_scrub_stamp":"2024-04-20T11:30:58.811830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:52:08.145439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.881828+0000","last_change":"2024-04-20T11:31:00.875883+0000","last_active":"2024-04-20T11:31:00.881828+0000","last_peered":"2024-04-20T11:31:00.881828+0000","last_clean":"2024-04-20T11:31:00.881828+0000","last_became_active":"2024-04-20T11:31:00.874155+0000","last_became_peered":"2024-04-20T11:31:00.874155+0000","last_unstale":"2024-04-20T11:31:00.881828+0000","last_undegraded":"2024-04-20T11:31:00.881828+0000","last_fullsized":"2024-04-20T11:31:00.881828+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:31:41.278482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.882622+0000","last_change":"2024-04-20T11:30:59.880360+0000","last_active":"2024-04-20T11:31:00.882622+0000","last_peered":"2024-04-20T11:31:00.882622+0000","last_clean":"2024-04-20T11:31:00.882622+0000","last_became_active":"2024-04-20T11:30:59.880008+0000","last_became_peered":"2024-04-20T11:30:59.880008+0000","last_unstale":"2024-04-20T11:31:00.882622+0000","last_undegraded":"2024-04-20T11:31:00.882622+0000","last_fullsized":"2024-04-20T11:31:00.882622+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_clean_scrub_stamp":"2024-04-20T11:30:58.811830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:54:14.201891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.903435+0000","last_change":"2024-04-20T11:31:00.895533+0000","last_active":"2024-04-20T11:31:00.903435+0000","last_peered":"2024-04-20T11:31:00.903435+0000","last_clean":"2024-04-20T11:31:00.903435+0000","last_became_active":"2024-04-20T11:31:00.894689+0000","last_became_peered":"2024-04-20T11:31:00.894689+0000","last_unstale":"2024-04-20T11:31:00.903435+0000","last_undegraded":"2024-04-20T11:31:00.903435+0000","last_fullsized":"2024-04-20T11:31:00.903435+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:26:22.574534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-20T11:31:01.858182+0000","last_change":"2024-04-20T11:30:59.859762+0000","last_active":"2024-04-20T11:31:01.858182+0000","last_peered":"2024-04-20T11:31:01.858182+0000","last_clean":"2024-04-20T11:31:01.858182+0000","last_became_active":"2024-04-20T11:30:59.859432+0000","last_became_peered":"2024-04-20T11:30:59.859432+0000","last_unstale":"2024-04-20T11:31:01.858182+0000","last_undegraded":"2024-04-20T11:31:01.858182+0000","last_fullsized":"2024-04-20T11:31:01.858182+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_clean_scrub_stamp":"2024-04-20T11:30:58.811830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:39:56.256065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.884101+0000","last_change":"2024-04-20T11:31:00.879677+0000","last_active":"2024-04-20T11:31:00.884101+0000","last_peered":"2024-04-20T11:31:00.884101+0000","last_clean":"2024-04-20T11:31:00.884101+0000","last_became_active":"2024-04-20T11:31:00.879195+0000","last_became_peered":"2024-04-20T11:31:00.879195+0000","last_unstale":"2024-04-20T11:31:00.884101+0000","last_undegraded":"2024-04-20T11:31:00.884101+0000","last_fullsized":"2024-04-20T11:31:00.884101+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:47:35.633763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"59'3","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.902789+0000","last_change":"2024-04-20T11:30:59.876927+0000","last_active":"2024-04-20T11:31:00.902789+0000","last_peered":"2024-04-20T11:31:00.902789+0000","last_clean":"2024-04-20T11:31:00.902789+0000","last_became_active":"2024-04-20T11:30:59.876550+0000","last_became_peered":"2024-04-20T11:30:59.876550+0000","last_unstale":"2024-04-20T11:31:00.902789+0000","last_undegraded":"2024-04-20T11:31:00.902789+0000","last_fullsized":"2024-04-20T11:31:00.902789+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_clean_scrub_stamp":"2024-04-20T11:30:58.811830+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:31:40.262552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.903994+0000","last_change":"2024-04-20T11:31:00.902398+0000","last_active":"2024-04-20T11:31:00.903994+0000","last_peered":"2024-04-20T11:31:00.903994+0000","last_clean":"2024-04-20T11:31:00.903994+0000","last_became_active":"2024-04-20T11:31:00.901554+0000","last_became_peered":"2024-04-20T11:31:00.901554+0000","last_unstale":"2024-04-20T11:31:00.903994+0000","last_undegraded":"2024-04-20T11:31:00.903994+0000","last_fullsized":"2024-04-20T11:31:00.903994+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:42:53.717064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"59'6","reported_seq":18,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.905357+0000","last_change":"2024-04-20T11:30:59.871898+0000","last_active":"2024-04-20T11:31:00.905357+0000","last_peered":"2024-04-20T11:31:00.905357+0000","last_clean":"2024-04-20T11:31:00.905357+0000","last_became_active":"2024-04-20T11:30:59.871581+0000","last_became_peered":"2024-04-20T11:30:59.871581+0000","last_unstale":"2024-04-20T11:31:00.905357+0000","last_undegraded":"2024-04-20T11:31:00.905357+0000","last_fullsized":"2024-04-20T11:31:00.905357+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_clean_scrub_stamp":"2024-04-20T11:30:58.811830+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:43:03.824977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-20T11:31:01.856266+0000","last_change":"2024-04-20T11:31:00.877030+0000","last_active":"2024-04-20T11:31:01.856266+0000","last_peered":"2024-04-20T11:31:01.856266+0000","last_clean":"2024-04-20T11:31:01.856266+0000","last_became_active":"2024-04-20T11:31:00.876522+0000","last_became_peered":"2024-04-20T11:31:00.876522+0000","last_unstale":"2024-04-20T11:31:01.856266+0000","last_undegraded":"2024-04-20T11:31:01.856266+0000","last_fullsized":"2024-04-20T11:31:01.856266+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:21:51.908569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"59'2","reported_seq":16,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-20T11:31:01.856956+0000","last_change":"2024-04-20T11:30:59.857714+0000","last_active":"2024-04-20T11:31:01.856956+0000","last_peered":"2024-04-20T11:31:01.856956+0000","last_clean":"2024-04-20T11:31:01.856956+0000","last_became_active":"2024-04-20T11:30:59.856452+0000","last_became_peered":"2024-04-20T11:30:59.856452+0000","last_unstale":"2024-04-20T11:31:01.856956+0000","last_undegraded":"2024-04-20T11:31:01.856956+0000","last_fullsized":"2024-04-20T11:31:01.856956+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_clean_scrub_stamp":"2024-04-20T11:30:58.811830+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:42:52.701135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.901283+0000","last_change":"2024-04-20T11:31:00.890228+0000","last_active":"2024-04-20T11:31:00.901283+0000","last_peered":"2024-04-20T11:31:00.901283+0000","last_clean":"2024-04-20T11:31:00.901283+0000","last_became_active":"2024-04-20T11:31:00.889916+0000","last_became_peered":"2024-04-20T11:31:00.889916+0000","last_unstale":"2024-04-20T11:31:00.901283+0000","last_undegraded":"2024-04-20T11:31:00.901283+0000","last_fullsized":"2024-04-20T11:31:00.901283+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:22:28.047678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.873753+0000","last_change":"2024-04-20T11:30:59.874179+0000","last_active":"2024-04-20T11:31:00.873753+0000","last_peered":"2024-04-20T11:31:00.873753+0000","last_clean":"2024-04-20T11:31:00.873753+0000","last_became_active":"2024-04-20T11:30:59.872895+0000","last_became_peered":"2024-04-20T11:30:59.872895+0000","last_unstale":"2024-04-20T11:31:00.873753+0000","last_undegraded":"2024-04-20T11:31:00.873753+0000","last_fullsized":"2024-04-20T11:31:00.873753+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_clean_scrub_stamp":"2024-04-20T11:30:58.811830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:32:29.028175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.897544+0000","last_change":"2024-04-20T11:31:00.889922+0000","last_active":"2024-04-20T11:31:00.897544+0000","last_peered":"2024-04-20T11:31:00.897544+0000","last_clean":"2024-04-20T11:31:00.897544+0000","last_became_active":"2024-04-20T11:31:00.888986+0000","last_became_peered":"2024-04-20T11:31:00.888986+0000","last_unstale":"2024-04-20T11:31:00.897544+0000","last_undegraded":"2024-04-20T11:31:00.897544+0000","last_fullsized":"2024-04-20T11:31:00.897544+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:02:57.616897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-20T11:31:01.855785+0000","last_change":"2024-04-20T11:30:59.878746+0000","last_active":"2024-04-20T11:31:01.855785+0000","last_peered":"2024-04-20T11:31:01.855785+0000","last_clean":"2024-04-20T11:31:01.855785+0000","last_became_active":"2024-04-20T11:30:59.877899+0000","last_became_peered":"2024-04-20T11:30:59.877899+0000","last_unstale":"2024-04-20T11:31:01.855785+0000","last_undegraded":"2024-04-20T11:31:01.855785+0000","last_fullsized":"2024-04-20T11:31:01.855785+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_clean_scrub_stamp":"2024-04-20T11:30:58.811830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:43:03.824977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.903363+0000","last_change":"2024-04-20T11:31:00.893527+0000","last_active":"2024-04-20T11:31:00.903363+0000","last_peered":"2024-04-20T11:31:00.903363+0000","last_clean":"2024-04-20T11:31:00.903363+0000","last_became_active":"2024-04-20T11:31:00.893249+0000","last_became_peered":"2024-04-20T11:31:00.893249+0000","last_unstale":"2024-04-20T11:31:00.903363+0000","last_undegraded":"2024-04-20T11:31:00.903363+0000","last_fullsized":"2024-04-20T11:31:00.903363+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:50:43.035072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"59'8","reported_seq":20,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.903123+0000","last_change":"2024-04-20T11:30:59.879029+0000","last_active":"2024-04-20T11:31:00.903123+0000","last_peered":"2024-04-20T11:31:00.903123+0000","last_clean":"2024-04-20T11:31:00.903123+0000","last_became_active":"2024-04-20T11:30:59.878708+0000","last_became_peered":"2024-04-20T11:30:59.878708+0000","last_unstale":"2024-04-20T11:31:00.903123+0000","last_undegraded":"2024-04-20T11:31:00.903123+0000","last_fullsized":"2024-04-20T11:31:00.903123+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_clean_scrub_stamp":"2024-04-20T11:30:58.811830+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:06:48.103643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.903450+0000","last_change":"2024-04-20T11:31:00.895176+0000","last_active":"2024-04-20T11:31:00.903450+0000","last_peered":"2024-04-20T11:31:00.903450+0000","last_clean":"2024-04-20T11:31:00.903450+0000","last_became_active":"2024-04-20T11:31:00.894853+0000","last_became_peered":"2024-04-20T11:31:00.894853+0000","last_unstale":"2024-04-20T11:31:00.903450+0000","last_undegraded":"2024-04-20T11:31:00.903450+0000","last_fullsized":"2024-04-20T11:31:00.903450+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:46:17.185811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"59'1","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-20T11:31:01.858418+0000","last_change":"2024-04-20T11:30:59.861988+0000","last_active":"2024-04-20T11:31:01.858418+0000","last_peered":"2024-04-20T11:31:01.858418+0000","last_clean":"2024-04-20T11:31:01.858418+0000","last_became_active":"2024-04-20T11:30:59.861699+0000","last_became_peered":"2024-04-20T11:30:59.861699+0000","last_unstale":"2024-04-20T11:31:01.858418+0000","last_undegraded":"2024-04-20T11:31:01.858418+0000","last_fullsized":"2024-04-20T11:31:01.858418+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_clean_scrub_stamp":"2024-04-20T11:30:58.811830+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:26:08.927093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-20T11:31:01.857316+0000","last_change":"2024-04-20T11:31:00.875699+0000","last_active":"2024-04-20T11:31:01.857316+0000","last_peered":"2024-04-20T11:31:01.857316+0000","last_clean":"2024-04-20T11:31:01.857316+0000","last_became_active":"2024-04-20T11:31:00.875215+0000","last_became_peered":"2024-04-20T11:31:00.875215+0000","last_unstale":"2024-04-20T11:31:01.857316+0000","last_undegraded":"2024-04-20T11:31:01.857316+0000","last_fullsized":"2024-04-20T11:31:01.857316+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:26:09.943022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"59'2","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.884584+0000","last_change":"2024-04-20T11:30:59.868667+0000","last_active":"2024-04-20T11:31:00.884584+0000","last_peered":"2024-04-20T11:31:00.884584+0000","last_clean":"2024-04-20T11:31:00.884584+0000","last_became_active":"2024-04-20T11:30:59.868217+0000","last_became_peered":"2024-04-20T11:30:59.868217+0000","last_unstale":"2024-04-20T11:31:00.884584+0000","last_undegraded":"2024-04-20T11:31:00.884584+0000","last_fullsized":"2024-04-20T11:31:00.884584+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_clean_scrub_stamp":"2024-04-20T11:30:58.811830+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:46:16.169882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-20T11:31:01.857523+0000","last_change":"2024-04-20T11:31:00.881414+0000","last_active":"2024-04-20T11:31:01.857523+0000","last_peered":"2024-04-20T11:31:01.857523+0000","last_clean":"2024-04-20T11:31:01.857523+0000","last_became_active":"2024-04-20T11:31:00.880204+0000","last_became_peered":"2024-04-20T11:31:00.880204+0000","last_unstale":"2024-04-20T11:31:01.857523+0000","last_undegraded":"2024-04-20T11:31:01.857523+0000","last_fullsized":"2024-04-20T11:31:01.857523+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:37:46.224157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"59'1","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-20T11:31:01.856723+0000","last_change":"2024-04-20T11:30:59.881525+0000","last_active":"2024-04-20T11:31:01.856723+0000","last_peered":"2024-04-20T11:31:01.856723+0000","last_clean":"2024-04-20T11:31:01.856723+0000","last_became_active":"2024-04-20T11:30:59.880812+0000","last_became_peered":"2024-04-20T11:30:59.880812+0000","last_unstale":"2024-04-20T11:31:01.856723+0000","last_undegraded":"2024-04-20T11:31:01.856723+0000","last_fullsized":"2024-04-20T11:31:01.856723+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_clean_scrub_stamp":"2024-04-20T11:30:58.811830+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:06:48.103643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.904005+0000","last_change":"2024-04-20T11:31:00.903090+0000","last_active":"2024-04-20T11:31:00.904005+0000","last_peered":"2024-04-20T11:31:00.904005+0000","last_clean":"2024-04-20T11:31:00.904005+0000","last_became_active":"2024-04-20T11:31:00.902825+0000","last_became_peered":"2024-04-20T11:31:00.902825+0000","last_unstale":"2024-04-20T11:31:00.904005+0000","last_undegraded":"2024-04-20T11:31:00.904005+0000","last_fullsized":"2024-04-20T11:31:00.904005+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:39:57.271994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.861903+0000","last_change":"2024-04-20T11:30:59.871645+0000","last_active":"2024-04-20T11:31:00.861903+0000","last_peered":"2024-04-20T11:31:00.861903+0000","last_clean":"2024-04-20T11:31:00.861903+0000","last_became_active":"2024-04-20T11:30:59.871342+0000","last_became_peered":"2024-04-20T11:30:59.871342+0000","last_unstale":"2024-04-20T11:31:00.861903+0000","last_undegraded":"2024-04-20T11:31:00.861903+0000","last_fullsized":"2024-04-20T11:31:00.861903+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_clean_scrub_stamp":"2024-04-20T11:30:58.811830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:50:42.019142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-20T11:31:01.856040+0000","last_change":"2024-04-20T11:31:00.873059+0000","last_active":"2024-04-20T11:31:01.856040+0000","last_peered":"2024-04-20T11:31:01.856040+0000","last_clean":"2024-04-20T11:31:01.856040+0000","last_became_active":"2024-04-20T11:31:00.872349+0000","last_became_peered":"2024-04-20T11:31:00.872349+0000","last_unstale":"2024-04-20T11:31:01.856040+0000","last_undegraded":"2024-04-20T11:31:01.856040+0000","last_fullsized":"2024-04-20T11:31:01.856040+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:47:44.049807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"59'2","reported_seq":16,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-20T11:31:01.857575+0000","last_change":"2024-04-20T11:30:59.869576+0000","last_active":"2024-04-20T11:31:01.857575+0000","last_peered":"2024-04-20T11:31:01.857575+0000","last_clean":"2024-04-20T11:31:01.857575+0000","last_became_active":"2024-04-20T11:30:59.869303+0000","last_became_peered":"2024-04-20T11:30:59.869303+0000","last_unstale":"2024-04-20T11:31:01.857575+0000","last_undegraded":"2024-04-20T11:31:01.857575+0000","last_fullsized":"2024-04-20T11:31:01.857575+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_clean_scrub_stamp":"2024-04-20T11:30:58.811830+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:23:21.108985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.893495+0000","last_change":"2024-04-20T11:31:00.886909+0000","last_active":"2024-04-20T11:31:00.893495+0000","last_peered":"2024-04-20T11:31:00.893495+0000","last_clean":"2024-04-20T11:31:00.893495+0000","last_became_active":"2024-04-20T11:31:00.886609+0000","last_became_peered":"2024-04-20T11:31:00.886609+0000","last_unstale":"2024-04-20T11:31:00.893495+0000","last_undegraded":"2024-04-20T11:31:00.893495+0000","last_fullsized":"2024-04-20T11:31:00.893495+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:23:22.124914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.866182+0000","last_change":"2024-04-20T11:30:59.863261+0000","last_active":"2024-04-20T11:31:00.866182+0000","last_peered":"2024-04-20T11:31:00.866182+0000","last_clean":"2024-04-20T11:31:00.866182+0000","last_became_active":"2024-04-20T11:30:59.862977+0000","last_became_peered":"2024-04-20T11:30:59.862977+0000","last_unstale":"2024-04-20T11:31:00.866182+0000","last_undegraded":"2024-04-20T11:31:00.866182+0000","last_fullsized":"2024-04-20T11:31:00.866182+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_clean_scrub_stamp":"2024-04-20T11:30:58.811830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:30:15.642512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.893351+0000","last_change":"2024-04-20T11:31:00.888491+0000","last_active":"2024-04-20T11:31:00.893351+0000","last_peered":"2024-04-20T11:31:00.893351+0000","last_clean":"2024-04-20T11:31:00.893351+0000","last_became_active":"2024-04-20T11:31:00.888215+0000","last_became_peered":"2024-04-20T11:31:00.888215+0000","last_unstale":"2024-04-20T11:31:00.893351+0000","last_undegraded":"2024-04-20T11:31:00.893351+0000","last_fullsized":"2024-04-20T11:31:00.893351+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:26:09.943022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-20T11:31:01.855481+0000","last_change":"2024-04-20T11:30:59.877636+0000","last_active":"2024-04-20T11:31:01.855481+0000","last_peered":"2024-04-20T11:31:01.855481+0000","last_clean":"2024-04-20T11:31:01.855481+0000","last_became_active":"2024-04-20T11:30:59.877279+0000","last_became_peered":"2024-04-20T11:30:59.877279+0000","last_unstale":"2024-04-20T11:31:01.855481+0000","last_undegraded":"2024-04-20T11:31:01.855481+0000","last_fullsized":"2024-04-20T11:31:01.855481+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_clean_scrub_stamp":"2024-04-20T11:30:58.811830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:27:21.889130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-20T11:31:01.857130+0000","last_change":"2024-04-20T11:31:00.885922+0000","last_active":"2024-04-20T11:31:01.857130+0000","last_peered":"2024-04-20T11:31:01.857130+0000","last_clean":"2024-04-20T11:31:01.857130+0000","last_became_active":"2024-04-20T11:31:00.885648+0000","last_became_peered":"2024-04-20T11:31:00.885648+0000","last_unstale":"2024-04-20T11:31:01.857130+0000","last_undegraded":"2024-04-20T11:31:01.857130+0000","last_fullsized":"2024-04-20T11:31:01.857130+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:04:46.835057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"59'2","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.885278+0000","last_change":"2024-04-20T11:30:59.869998+0000","last_active":"2024-04-20T11:31:00.885278+0000","last_peered":"2024-04-20T11:31:00.885278+0000","last_clean":"2024-04-20T11:31:00.885278+0000","last_became_active":"2024-04-20T11:30:59.869559+0000","last_became_peered":"2024-04-20T11:30:59.869559+0000","last_unstale":"2024-04-20T11:31:00.885278+0000","last_undegraded":"2024-04-20T11:31:00.885278+0000","last_fullsized":"2024-04-20T11:31:00.885278+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_clean_scrub_stamp":"2024-04-20T11:30:58.811830+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:17:11.482838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-20T11:31:01.857303+0000","last_change":"2024-04-20T11:31:00.888344+0000","last_active":"2024-04-20T11:31:01.857303+0000","last_peered":"2024-04-20T11:31:01.857303+0000","last_clean":"2024-04-20T11:31:01.857303+0000","last_became_active":"2024-04-20T11:31:00.888049+0000","last_became_peered":"2024-04-20T11:31:00.888049+0000","last_unstale":"2024-04-20T11:31:01.857303+0000","last_undegraded":"2024-04-20T11:31:01.857303+0000","last_fullsized":"2024-04-20T11:31:01.857303+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:22:38.955596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.864564+0000","last_change":"2024-04-20T11:30:59.867927+0000","last_active":"2024-04-20T11:31:00.864564+0000","last_peered":"2024-04-20T11:31:00.864564+0000","last_clean":"2024-04-20T11:31:00.864564+0000","last_became_active":"2024-04-20T11:30:59.867580+0000","last_became_peered":"2024-04-20T11:30:59.867580+0000","last_unstale":"2024-04-20T11:31:00.864564+0000","last_undegraded":"2024-04-20T11:31:00.864564+0000","last_fullsized":"2024-04-20T11:31:00.864564+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_clean_scrub_stamp":"2024-04-20T11:30:58.811830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:26:21.558604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.884078+0000","last_change":"2024-04-20T11:31:00.879445+0000","last_active":"2024-04-20T11:31:00.884078+0000","last_peered":"2024-04-20T11:31:00.884078+0000","last_clean":"2024-04-20T11:31:00.884078+0000","last_became_active":"2024-04-20T11:31:00.878871+0000","last_became_peered":"2024-04-20T11:31:00.878871+0000","last_unstale":"2024-04-20T11:31:00.884078+0000","last_undegraded":"2024-04-20T11:31:00.884078+0000","last_fullsized":"2024-04-20T11:31:00.884078+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:13:11.698885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.880639+0000","last_change":"2024-04-20T11:30:59.873959+0000","last_active":"2024-04-20T11:31:00.880639+0000","last_peered":"2024-04-20T11:31:00.880639+0000","last_clean":"2024-04-20T11:31:00.880639+0000","last_became_active":"2024-04-20T11:30:59.868495+0000","last_became_peered":"2024-04-20T11:30:59.868495+0000","last_unstale":"2024-04-20T11:31:00.880639+0000","last_undegraded":"2024-04-20T11:31:00.880639+0000","last_fullsized":"2024-04-20T11:31:00.880639+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_clean_scrub_stamp":"2024-04-20T11:30:58.811830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:17:11.482838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.882740+0000","last_change":"2024-04-20T11:31:00.877496+0000","last_active":"2024-04-20T11:31:00.882740+0000","last_peered":"2024-04-20T11:31:00.882740+0000","last_clean":"2024-04-20T11:31:00.882740+0000","last_became_active":"2024-04-20T11:31:00.876527+0000","last_became_peered":"2024-04-20T11:31:00.876527+0000","last_unstale":"2024-04-20T11:31:00.882740+0000","last_undegraded":"2024-04-20T11:31:00.882740+0000","last_fullsized":"2024-04-20T11:31:00.882740+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:21:51.908569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-20T11:31:01.856746+0000","last_change":"2024-04-20T11:30:59.882562+0000","last_active":"2024-04-20T11:31:01.856746+0000","last_peered":"2024-04-20T11:31:01.856746+0000","last_clean":"2024-04-20T11:31:01.856746+0000","last_became_active":"2024-04-20T11:30:59.882277+0000","last_became_peered":"2024-04-20T11:30:59.882277+0000","last_unstale":"2024-04-20T11:31:01.856746+0000","last_undegraded":"2024-04-20T11:31:01.856746+0000","last_fullsized":"2024-04-20T11:31:01.856746+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_clean_scrub_stamp":"2024-04-20T11:30:58.811830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:31:40.262552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-20T11:31:01.857886+0000","last_change":"2024-04-20T11:31:00.875951+0000","last_active":"2024-04-20T11:31:01.857886+0000","last_peered":"2024-04-20T11:31:01.857886+0000","last_clean":"2024-04-20T11:31:01.857886+0000","last_became_active":"2024-04-20T11:31:00.875700+0000","last_became_peered":"2024-04-20T11:31:00.875700+0000","last_unstale":"2024-04-20T11:31:01.857886+0000","last_undegraded":"2024-04-20T11:31:01.857886+0000","last_fullsized":"2024-04-20T11:31:01.857886+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:23:42.556940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.861010+0000","last_change":"2024-04-20T11:30:59.876568+0000","last_active":"2024-04-20T11:31:00.861010+0000","last_peered":"2024-04-20T11:31:00.861010+0000","last_clean":"2024-04-20T11:31:00.861010+0000","last_became_active":"2024-04-20T11:30:59.876145+0000","last_became_peered":"2024-04-20T11:30:59.876145+0000","last_unstale":"2024-04-20T11:31:00.861010+0000","last_undegraded":"2024-04-20T11:31:00.861010+0000","last_fullsized":"2024-04-20T11:31:00.861010+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_clean_scrub_stamp":"2024-04-20T11:30:58.811830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:52:51.929802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.903460+0000","last_change":"2024-04-20T11:31:00.896169+0000","last_active":"2024-04-20T11:31:00.903460+0000","last_peered":"2024-04-20T11:31:00.903460+0000","last_clean":"2024-04-20T11:31:00.903460+0000","last_became_active":"2024-04-20T11:31:00.895761+0000","last_became_peered":"2024-04-20T11:31:00.895761+0000","last_unstale":"2024-04-20T11:31:00.903460+0000","last_undegraded":"2024-04-20T11:31:00.903460+0000","last_fullsized":"2024-04-20T11:31:00.903460+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:49:17.613752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.859559+0000","last_change":"2024-04-20T11:30:59.873436+0000","last_active":"2024-04-20T11:31:00.859559+0000","last_peered":"2024-04-20T11:31:00.859559+0000","last_clean":"2024-04-20T11:31:00.859559+0000","last_became_active":"2024-04-20T11:30:59.873162+0000","last_became_peered":"2024-04-20T11:30:59.873162+0000","last_unstale":"2024-04-20T11:31:00.859559+0000","last_undegraded":"2024-04-20T11:31:00.859559+0000","last_fullsized":"2024-04-20T11:31:00.859559+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_clean_scrub_stamp":"2024-04-20T11:30:58.811830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:20:28.268336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.893224+0000","last_change":"2024-04-20T11:31:00.886086+0000","last_active":"2024-04-20T11:31:00.893224+0000","last_peered":"2024-04-20T11:31:00.893224+0000","last_clean":"2024-04-20T11:31:00.893224+0000","last_became_active":"2024-04-20T11:31:00.885821+0000","last_became_peered":"2024-04-20T11:31:00.885821+0000","last_unstale":"2024-04-20T11:31:00.893224+0000","last_undegraded":"2024-04-20T11:31:00.893224+0000","last_fullsized":"2024-04-20T11:31:00.893224+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:27:06.123345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.857691+0000","last_change":"2024-04-20T11:30:59.871396+0000","last_active":"2024-04-20T11:31:00.857691+0000","last_peered":"2024-04-20T11:31:00.857691+0000","last_clean":"2024-04-20T11:31:00.857691+0000","last_became_active":"2024-04-20T11:30:59.871101+0000","last_became_peered":"2024-04-20T11:30:59.871101+0000","last_unstale":"2024-04-20T11:31:00.857691+0000","last_undegraded":"2024-04-20T11:31:00.857691+0000","last_fullsized":"2024-04-20T11:31:00.857691+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_clean_scrub_stamp":"2024-04-20T11:30:58.811830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:32:29.028175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-20T11:31:01.857783+0000","last_change":"2024-04-20T11:31:00.889262+0000","last_active":"2024-04-20T11:31:01.857783+0000","last_peered":"2024-04-20T11:31:01.857783+0000","last_clean":"2024-04-20T11:31:01.857783+0000","last_became_active":"2024-04-20T11:31:00.888999+0000","last_became_peered":"2024-04-20T11:31:00.888999+0000","last_unstale":"2024-04-20T11:31:01.857783+0000","last_undegraded":"2024-04-20T11:31:01.857783+0000","last_fullsized":"2024-04-20T11:31:01.857783+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:43:11.158343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-20T11:31:01.857147+0000","last_change":"2024-04-20T11:30:59.875128+0000","last_active":"2024-04-20T11:31:01.857147+0000","last_peered":"2024-04-20T11:31:01.857147+0000","last_clean":"2024-04-20T11:31:01.857147+0000","last_became_active":"2024-04-20T11:30:59.874728+0000","last_became_peered":"2024-04-20T11:30:59.874728+0000","last_unstale":"2024-04-20T11:31:01.857147+0000","last_undegraded":"2024-04-20T11:31:01.857147+0000","last_fullsized":"2024-04-20T11:31:01.857147+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_clean_scrub_stamp":"2024-04-20T11:30:58.811830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:04:45.819127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-20T11:31:01.857004+0000","last_change":"2024-04-20T11:31:00.870992+0000","last_active":"2024-04-20T11:31:01.857004+0000","last_peered":"2024-04-20T11:31:01.857004+0000","last_clean":"2024-04-20T11:31:01.857004+0000","last_became_active":"2024-04-20T11:31:00.870724+0000","last_became_peered":"2024-04-20T11:31:00.870724+0000","last_unstale":"2024-04-20T11:31:01.857004+0000","last_undegraded":"2024-04-20T11:31:01.857004+0000","last_fullsized":"2024-04-20T11:31:01.857004+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:24:23.886081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.876286+0000","last_change":"2024-04-20T11:30:59.874922+0000","last_active":"2024-04-20T11:31:00.876286+0000","last_peered":"2024-04-20T11:31:00.876286+0000","last_clean":"2024-04-20T11:31:00.876286+0000","last_became_active":"2024-04-20T11:30:59.874492+0000","last_became_peered":"2024-04-20T11:30:59.874492+0000","last_unstale":"2024-04-20T11:31:00.876286+0000","last_undegraded":"2024-04-20T11:31:00.876286+0000","last_fullsized":"2024-04-20T11:31:00.876286+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_clean_scrub_stamp":"2024-04-20T11:30:58.811830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:52:51.929802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":240,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.879541+0000","last_change":"2024-04-20T11:26:48.661897+0000","last_active":"2024-04-20T11:31:00.879541+0000","last_peered":"2024-04-20T11:31:00.879541+0000","last_clean":"2024-04-20T11:31:00.879541+0000","last_became_active":"2024-04-20T11:26:48.656535+0000","last_became_peered":"2024-04-20T11:26:48.656535+0000","last_unstale":"2024-04-20T11:31:00.879541+0000","last_undegraded":"2024-04-20T11:31:00.879541+0000","last_fullsized":"2024-04-20T11:31:00.879541+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:47.543004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:47.543004+0000","last_clean_scrub_stamp":"2024-04-20T11:26:47.543004+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:58:35.527105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574904,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32460,"kb_used_data":1280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339380,"statfs":{"total":96636764160,"available":96603525120,"internally_reserved":0,"allocated":1310720,"data_stored":886041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574901,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27716,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344124,"statfs":{"total":96636764160,"available":96608382976,"internally_reserved":0,"allocated":708608,"data_stored":295513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574903,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31880,"kb_used_data":700,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339960,"statfs":{"total":96636764160,"available":96604119040,"internally_reserved":0,"allocated":716800,"data_stored":296165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574901,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27724,"kb_used_data":704,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344116,"statfs":{"total":96636764160,"available":96608374784,"internally_reserved":0,"allocated":720896,"data_stored":297125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574902,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31880,"kb_used_data":700,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339960,"statfs":{"total":96636764160,"available":96604119040,"internally_reserved":0,"allocated":716800,"data_stored":295569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574903,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27740,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344100,"statfs":{"total":96636764160,"available":96608358400,"internally_reserved":0,"allocated":729088,"data_stored":301869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574901,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31876,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339964,"statfs":{"total":96636764160,"available":96604123136,"internally_reserved":0,"allocated":712704,"data_stored":295603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574902,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32468,"kb_used_data":1288,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339372,"statfs":{"total":96636764160,"available":96603516928,"internally_reserved":0,"allocated":1318912,"data_stored":888075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T11:31:05.289 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T11:31:05.532 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.532+0000 7f7c47200640 1 Processor -- start 2024-04-20T11:31:05.533 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.532+0000 7f7c47200640 1 -- start start 2024-04-20T11:31:05.535 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.534+0000 7f7c47200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f7c48147410 0x7f7c480afa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:05.535 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.534+0000 7f7c47200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7c48071cd0 0x7f7c48072130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:05.535 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.534+0000 7f7c47200640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c480aec90 con 0x7f7c4806e010 2024-04-20T11:31:05.535 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.534+0000 7f7c47200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f7c48072670 con 0x7f7c48147410 2024-04-20T11:31:05.535 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.534+0000 7f7c47200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7c480aef70 con 0x7f7c48071cd0 2024-04-20T11:31:05.535 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.534+0000 7f7c46800640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f7c4806e010 0x7f7c4806e4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:50946/0 (socket says 172.21.15.152:50946) 2024-04-20T11:31:05.535 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.534+0000 7f7c46800640 1 -- 172.21.15.152:0/261962348 learned_addr learned my addr 172.21.15.152:0/261962348 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:31:05.537 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.534+0000 7f7c45400640 1 --2- 172.21.15.152:0/261962348 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7c48071cd0 0x7f7c48072130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:05.537 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.534+0000 7f7c45e00640 1 --2- 172.21.15.152:0/261962348 >> v2:172.21.15.175:3300/0 conn(0x7f7c48147410 0x7f7c480afa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:05.537 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.534+0000 7f7c44a00640 1 -- 172.21.15.152:0/261962348 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2861878839 0 0) 0x7f7c480aec90 con 0x7f7c4806e010 2024-04-20T11:31:05.537 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.535+0000 7f7c44a00640 1 -- 172.21.15.152:0/261962348 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7c2c003520 con 0x7f7c4806e010 2024-04-20T11:31:05.537 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.535+0000 7f7c44a00640 1 -- 172.21.15.152:0/261962348 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3041082800 0 0) 0x7f7c2c003520 con 0x7f7c4806e010 2024-04-20T11:31:05.537 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.535+0000 7f7c44a00640 1 -- 172.21.15.152:0/261962348 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7c48071cd0 msgr2=0x7f7c48072130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:05.537 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.535+0000 7f7c44a00640 1 --2- 172.21.15.152:0/261962348 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7c48071cd0 0x7f7c48072130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:05.537 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.535+0000 7f7c44a00640 1 -- 172.21.15.152:0/261962348 >> v2:172.21.15.175:3300/0 conn(0x7f7c48147410 msgr2=0x7f7c480afa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:05.537 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.535+0000 7f7c44a00640 1 --2- 172.21.15.152:0/261962348 >> v2:172.21.15.175:3300/0 conn(0x7f7c48147410 0x7f7c480afa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:05.537 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.535+0000 7f7c44a00640 1 -- 172.21.15.152:0/261962348 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c480af250 con 0x7f7c4806e010 2024-04-20T11:31:05.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.535+0000 7f7c44a00640 1 -- 172.21.15.152:0/261962348 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f7c400019f0 con 0x7f7c4806e010 2024-04-20T11:31:05.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.535+0000 7f7c45e00640 1 --2- 172.21.15.152:0/261962348 >> v2:172.21.15.175:3300/0 conn(0x7f7c48147410 0x7f7c480afa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:31:05.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.535+0000 7f7c44a00640 1 -- 172.21.15.152:0/261962348 >> v1:172.21.15.152:6789/0 conn(0x7f7c4806e010 legacy=0x7f7c4806e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:05.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.535+0000 7f7c44a00640 1 --2- 172.21.15.152:0/261962348 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c2c0039b0 0x7f7c2c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:05.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.535+0000 7f7c44a00640 1 --2- 172.21.15.152:0/261962348 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7c2c004800 0x7f7c2c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:05.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.535+0000 7f7c44a00640 1 --2- 172.21.15.152:0/261962348 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7c48071cd0 0x7f7c2c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:05.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.535+0000 7f7c44a00640 1 -- 172.21.15.152:0/261962348 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f7c480aef70 con 0x7f7c2c004800 2024-04-20T11:31:05.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.535+0000 7f7c44a00640 1 -- 172.21.15.152:0/261962348 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7c2c00ba70 con 0x7f7c2c0039b0 2024-04-20T11:31:05.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.535+0000 7f7c44a00640 1 -- 172.21.15.152:0/261962348 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7c2c00bc40 con 0x7f7c48071cd0 2024-04-20T11:31:05.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.535+0000 7f7c45400640 1 --2- 172.21.15.152:0/261962348 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7c2c004800 0x7f7c2c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:05.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.536+0000 7f7c45e00640 1 --2- 172.21.15.152:0/261962348 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c2c0039b0 0x7f7c2c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:05.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.536+0000 7f7c45400640 1 -- 172.21.15.152:0/261962348 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7c48071cd0 msgr2=0x7f7c2c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:05.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.536+0000 7f7c45400640 1 --2- 172.21.15.152:0/261962348 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7c48071cd0 0x7f7c2c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:05.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.536+0000 7f7c45400640 1 -- 172.21.15.152:0/261962348 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c2c0039b0 msgr2=0x7f7c2c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:05.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.536+0000 7f7c45400640 1 --2- 172.21.15.152:0/261962348 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c2c0039b0 0x7f7c2c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:05.539 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.536+0000 7f7c45400640 1 -- 172.21.15.152:0/261962348 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c2c003520 con 0x7f7c2c004800 2024-04-20T11:31:05.540 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.537+0000 7f7c45400640 1 --2- 172.21.15.152:0/261962348 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7c2c004800 0x7f7c2c004c80 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f7c3400ba00 tx=0x7f7c2c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc639e2deec11852 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:05.540 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.537+0000 7f7c44a00640 1 -- 172.21.15.152:0/261962348 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c34036020 con 0x7f7c2c004800 2024-04-20T11:31:05.540 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.537+0000 7f7c44a00640 1 -- 172.21.15.152:0/261962348 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c34032630 con 0x7f7c2c004800 2024-04-20T11:31:05.540 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.537+0000 7f7c44a00640 1 -- 172.21.15.152:0/261962348 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c3400b600 con 0x7f7c2c004800 2024-04-20T11:31:05.540 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.538+0000 7f7c47200640 1 -- 172.21.15.152:0/261962348 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7c2c004800 msgr2=0x7f7c2c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:05.540 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.538+0000 7f7c47200640 1 --2- 172.21.15.152:0/261962348 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7c2c004800 0x7f7c2c004c80 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f7c3400ba00 tx=0x7f7c2c005830 comp rx=0 tx=0).stop 2024-04-20T11:31:05.540 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.538+0000 7f7c46800640 1 -- 172.21.15.152:0/261962348 reap_dead start 2024-04-20T11:31:05.540 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.538+0000 7f7c47200640 1 -- 172.21.15.152:0/261962348 shutdown_connections 2024-04-20T11:31:05.540 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.538+0000 7f7c47200640 1 -- 172.21.15.152:0/261962348 >> 172.21.15.152:0/261962348 conn(0x7f7c48067ae0 msgr2=0x7f7c4806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:05.541 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.538+0000 7f7c47200640 1 -- 172.21.15.152:0/261962348 shutdown_connections 2024-04-20T11:31:05.541 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.538+0000 7f7c47200640 1 -- 172.21.15.152:0/261962348 wait complete. 2024-04-20T11:31:05.541 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.540+0000 7f7c47200640 1 Processor -- start 2024-04-20T11:31:05.541 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.540+0000 7f7c47200640 1 -- start start 2024-04-20T11:31:05.543 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.541+0000 7f7c47200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c2c004800 0x7f7c4815fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:05.543 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.541+0000 7f7c47200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7c481619a0 0x7f7c480b4540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:05.543 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.541+0000 7f7c47200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7c4815ec70 0x7f7c4815f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:05.543 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.541+0000 7f7c47200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f7c48072130 con 0x7f7c4815ec70 2024-04-20T11:31:05.543 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.541+0000 7f7c47200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7c480ae980 con 0x7f7c2c004800 2024-04-20T11:31:05.543 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.541+0000 7f7c47200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7c480722b0 con 0x7f7c481619a0 2024-04-20T11:31:05.543 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.541+0000 7f7c46800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7c4815ec70 0x7f7c4815f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:05.543 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.541+0000 7f7c46800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7c4815ec70 0x7f7c4815f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:47692/0 (socket says 172.21.15.152:47692) 2024-04-20T11:31:05.543 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.541+0000 7f7c46800640 1 -- 172.21.15.152:0/3881478333 learned_addr learned my addr 172.21.15.152:0/3881478333 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:05.543 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.541+0000 7f7c45e00640 1 --2- 172.21.15.152:0/3881478333 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c2c004800 0x7f7c4815fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:05.543 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.541+0000 7f7c46800640 1 -- 172.21.15.152:0/3881478333 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7c481619a0 msgr2=0x7f7c480b4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:05.543 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.541+0000 7f7c46800640 1 --2- 172.21.15.152:0/3881478333 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7c481619a0 0x7f7c480b4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:05.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.541+0000 7f7c46800640 1 -- 172.21.15.152:0/3881478333 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c2c004800 msgr2=0x7f7c4815fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:05.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.541+0000 7f7c46800640 1 --2- 172.21.15.152:0/3881478333 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c2c004800 0x7f7c4815fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:05.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.541+0000 7f7c46800640 1 -- 172.21.15.152:0/3881478333 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c48140370 con 0x7f7c4815ec70 2024-04-20T11:31:05.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.542+0000 7f7c46800640 1 --2- 172.21.15.152:0/3881478333 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7c4815ec70 0x7f7c4815f0d0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f7c480700c0 tx=0x7f7c4000cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:05.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.542+0000 7f7c3e000640 1 -- 172.21.15.152:0/3881478333 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c4000cea0 con 0x7f7c4815ec70 2024-04-20T11:31:05.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.542+0000 7f7c3e000640 1 -- 172.21.15.152:0/3881478333 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c4000a5c0 con 0x7f7c4815ec70 2024-04-20T11:31:05.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.542+0000 7f7c47200640 1 -- 172.21.15.152:0/3881478333 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c48160900 con 0x7f7c4815ec70 2024-04-20T11:31:05.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.542+0000 7f7c3e000640 1 -- 172.21.15.152:0/3881478333 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c4000a990 con 0x7f7c4815ec70 2024-04-20T11:31:05.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.542+0000 7f7c47200640 1 -- 172.21.15.152:0/3881478333 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c48160ec0 con 0x7f7c4815ec70 2024-04-20T11:31:05.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.543+0000 7f7c47200640 1 -- 172.21.15.152:0/3881478333 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c48075e60 con 0x7f7c4815ec70 2024-04-20T11:31:05.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.544+0000 7f7c3e000640 1 -- 172.21.15.152:0/3881478333 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 9) ==== 91193+0+0 (secure 0 0 0) 0x7f7c4000ab30 con 0x7f7c4815ec70 2024-04-20T11:31:05.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.545+0000 7f7c3e000640 1 --2- 172.21.15.152:0/3881478333 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f7c2806caa0 0x7f7c2806ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:05.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.546+0000 7f7c3e000640 1 -- 172.21.15.152:0/3881478333 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5709+0+0 (secure 0 0 0) 0x7f7c4008b720 con 0x7f7c4815ec70 2024-04-20T11:31:05.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.546+0000 7f7c45e00640 1 --2- 172.21.15.152:0/3881478333 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f7c2806caa0 0x7f7c2806ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:05.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.546+0000 7f7c45e00640 1 --2- 172.21.15.152:0/3881478333 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f7c2806caa0 0x7f7c2806ef60 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f7c38006fd0 tx=0x7f7c38008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:05.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.550+0000 7f7c3e000640 1 -- 172.21.15.152:0/3881478333 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f7c40059830 con 0x7f7c4815ec70 2024-04-20T11:31:05.747 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.746+0000 7f7c47200640 1 -- 172.21.15.152:0/3881478333 --> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7c480b17d0 con 0x7f7c2806caa0 2024-04-20T11:31:05.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.760+0000 7f7c3e000640 1 -- 172.21.15.152:0/3881478333 <== mgr.4113 v2:172.21.15.152:6832/2285110240 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251911 (secure 0 0 0) 0x7f7c480b17d0 con 0x7f7c2806caa0 2024-04-20T11:31:05.767 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:31:05.768 INFO:teuthology.orchestra.run.smithi152.stderr:dumped all 2024-04-20T11:31:05.769 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.765+0000 7f7c47200640 1 -- 172.21.15.152:0/3881478333 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f7c2806caa0 msgr2=0x7f7c2806ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:05.770 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.765+0000 7f7c47200640 1 --2- 172.21.15.152:0/3881478333 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f7c2806caa0 0x7f7c2806ef60 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f7c38006fd0 tx=0x7f7c38008040 comp rx=0 tx=0).stop 2024-04-20T11:31:05.770 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.765+0000 7f7c47200640 1 -- 172.21.15.152:0/3881478333 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7c4815ec70 msgr2=0x7f7c4815f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:05.770 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.765+0000 7f7c47200640 1 --2- 172.21.15.152:0/3881478333 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7c4815ec70 0x7f7c4815f0d0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f7c480700c0 tx=0x7f7c4000cc50 comp rx=0 tx=0).stop 2024-04-20T11:31:05.770 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.765+0000 7f7c47200640 1 -- 172.21.15.152:0/3881478333 shutdown_connections 2024-04-20T11:31:05.770 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.765+0000 7f7c47200640 1 --2- 172.21.15.152:0/3881478333 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f7c2806caa0 0x7f7c2806ef60 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:05.770 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.766+0000 7f7c47200640 1 --2- 172.21.15.152:0/3881478333 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7c4815ec70 0x7f7c4815f0d0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:05.770 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.766+0000 7f7c47200640 1 --2- 172.21.15.152:0/3881478333 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7c481619a0 0x7f7c480b4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:05.770 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.766+0000 7f7c47200640 1 --2- 172.21.15.152:0/3881478333 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c2c004800 0x7f7c4815fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:05.770 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.766+0000 7f7c47200640 1 -- 172.21.15.152:0/3881478333 >> 172.21.15.152:0/3881478333 conn(0x7f7c48067ae0 msgr2=0x7f7c48162060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:05.770 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.766+0000 7f7c47200640 1 -- 172.21.15.152:0/3881478333 shutdown_connections 2024-04-20T11:31:05.770 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:05.766+0000 7f7c47200640 1 -- 172.21.15.152:0/3881478333 wait complete. 2024-04-20T11:31:05.787 INFO:teuthology.orchestra.run.smithi152.stdout:{"pg_ready":true,"pg_map":{"version":182,"stamp":"2024-04-20T11:31:05.296750+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":243744,"kb_used_data":6772,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754730976,"statfs":{"total":773094113280,"available":772844519424,"internally_reserved":0,"allocated":6934528,"data_stored":3555960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12746,"internal_metadata":242535990},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":44,"apply_latency_ms":44,"commit_latency_ns":44000000,"apply_latency_ns":44000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001986"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-20T11:31:01.854480+0000","last_change":"2024-04-20T11:31:00.879013+0000","last_active":"2024-04-20T11:31:01.854480+0000","last_peered":"2024-04-20T11:31:01.854480+0000","last_clean":"2024-04-20T11:31:01.854480+0000","last_became_active":"2024-04-20T11:31:00.878751+0000","last_became_peered":"2024-04-20T11:31:00.878751+0000","last_unstale":"2024-04-20T11:31:01.854480+0000","last_undegraded":"2024-04-20T11:31:01.854480+0000","last_fullsized":"2024-04-20T11:31:01.854480+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:14:19.396693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-20T11:31:01.857367+0000","last_change":"2024-04-20T11:31:00.872086+0000","last_active":"2024-04-20T11:31:01.857367+0000","last_peered":"2024-04-20T11:31:01.857367+0000","last_clean":"2024-04-20T11:31:01.857367+0000","last_became_active":"2024-04-20T11:31:00.871806+0000","last_became_peered":"2024-04-20T11:31:00.871806+0000","last_unstale":"2024-04-20T11:31:01.857367+0000","last_undegraded":"2024-04-20T11:31:01.857367+0000","last_fullsized":"2024-04-20T11:31:01.857367+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:20:01.273117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.903978+0000","last_change":"2024-04-20T11:31:00.902101+0000","last_active":"2024-04-20T11:31:00.903978+0000","last_peered":"2024-04-20T11:31:00.903978+0000","last_clean":"2024-04-20T11:31:00.903978+0000","last_became_active":"2024-04-20T11:31:00.901339+0000","last_became_peered":"2024-04-20T11:31:00.901339+0000","last_unstale":"2024-04-20T11:31:00.903978+0000","last_undegraded":"2024-04-20T11:31:00.903978+0000","last_fullsized":"2024-04-20T11:31:00.903978+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:21:51.908569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-20T11:31:01.856405+0000","last_change":"2024-04-20T11:31:00.883684+0000","last_active":"2024-04-20T11:31:01.856405+0000","last_peered":"2024-04-20T11:31:01.856405+0000","last_clean":"2024-04-20T11:31:01.856405+0000","last_became_active":"2024-04-20T11:31:00.883381+0000","last_became_peered":"2024-04-20T11:31:00.883381+0000","last_unstale":"2024-04-20T11:31:01.856405+0000","last_undegraded":"2024-04-20T11:31:01.856405+0000","last_fullsized":"2024-04-20T11:31:01.856405+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:42:53.717064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-20T11:31:01.854373+0000","last_change":"2024-04-20T11:31:00.869648+0000","last_active":"2024-04-20T11:31:01.854373+0000","last_peered":"2024-04-20T11:31:01.854373+0000","last_clean":"2024-04-20T11:31:01.854373+0000","last_became_active":"2024-04-20T11:31:00.869335+0000","last_became_peered":"2024-04-20T11:31:00.869335+0000","last_unstale":"2024-04-20T11:31:01.854373+0000","last_undegraded":"2024-04-20T11:31:01.854373+0000","last_fullsized":"2024-04-20T11:31:01.854373+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:35:08.994877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.882526+0000","last_change":"2024-04-20T11:31:00.875405+0000","last_active":"2024-04-20T11:31:00.882526+0000","last_peered":"2024-04-20T11:31:00.882526+0000","last_clean":"2024-04-20T11:31:00.882526+0000","last_became_active":"2024-04-20T11:31:00.875110+0000","last_became_peered":"2024-04-20T11:31:00.875110+0000","last_unstale":"2024-04-20T11:31:00.882526+0000","last_undegraded":"2024-04-20T11:31:00.882526+0000","last_fullsized":"2024-04-20T11:31:00.882526+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:36:24.761945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.877585+0000","last_change":"2024-04-20T11:31:00.868812+0000","last_active":"2024-04-20T11:31:00.877585+0000","last_peered":"2024-04-20T11:31:00.877585+0000","last_clean":"2024-04-20T11:31:00.877585+0000","last_became_active":"2024-04-20T11:31:00.868378+0000","last_became_peered":"2024-04-20T11:31:00.868378+0000","last_unstale":"2024-04-20T11:31:00.877585+0000","last_undegraded":"2024-04-20T11:31:00.877585+0000","last_fullsized":"2024-04-20T11:31:00.877585+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:24:45.359945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-20T11:31:01.856909+0000","last_change":"2024-04-20T11:31:00.879298+0000","last_active":"2024-04-20T11:31:01.856909+0000","last_peered":"2024-04-20T11:31:01.856909+0000","last_clean":"2024-04-20T11:31:01.856909+0000","last_became_active":"2024-04-20T11:31:00.879024+0000","last_became_peered":"2024-04-20T11:31:00.879024+0000","last_unstale":"2024-04-20T11:31:01.856909+0000","last_undegraded":"2024-04-20T11:31:01.856909+0000","last_fullsized":"2024-04-20T11:31:01.856909+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:51:52.380825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.878664+0000","last_change":"2024-04-20T11:31:00.873632+0000","last_active":"2024-04-20T11:31:00.878664+0000","last_peered":"2024-04-20T11:31:00.878664+0000","last_clean":"2024-04-20T11:31:00.878664+0000","last_became_active":"2024-04-20T11:31:00.873353+0000","last_became_peered":"2024-04-20T11:31:00.873353+0000","last_unstale":"2024-04-20T11:31:00.878664+0000","last_undegraded":"2024-04-20T11:31:00.878664+0000","last_fullsized":"2024-04-20T11:31:00.878664+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:06:49.119572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.893272+0000","last_change":"2024-04-20T11:31:00.886226+0000","last_active":"2024-04-20T11:31:00.893272+0000","last_peered":"2024-04-20T11:31:00.893272+0000","last_clean":"2024-04-20T11:31:00.893272+0000","last_became_active":"2024-04-20T11:31:00.885947+0000","last_became_peered":"2024-04-20T11:31:00.885947+0000","last_unstale":"2024-04-20T11:31:00.893272+0000","last_undegraded":"2024-04-20T11:31:00.893272+0000","last_fullsized":"2024-04-20T11:31:00.893272+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:18:25.731153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.894018+0000","last_change":"2024-04-20T11:31:00.891786+0000","last_active":"2024-04-20T11:31:00.894018+0000","last_peered":"2024-04-20T11:31:00.894018+0000","last_clean":"2024-04-20T11:31:00.894018+0000","last_became_active":"2024-04-20T11:31:00.891520+0000","last_became_peered":"2024-04-20T11:31:00.891520+0000","last_unstale":"2024-04-20T11:31:00.894018+0000","last_undegraded":"2024-04-20T11:31:00.894018+0000","last_fullsized":"2024-04-20T11:31:00.894018+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:20:10.518047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-20T11:31:01.856518+0000","last_change":"2024-04-20T11:31:00.891872+0000","last_active":"2024-04-20T11:31:01.856518+0000","last_peered":"2024-04-20T11:31:01.856518+0000","last_clean":"2024-04-20T11:31:01.856518+0000","last_became_active":"2024-04-20T11:31:00.891551+0000","last_became_peered":"2024-04-20T11:31:00.891551+0000","last_unstale":"2024-04-20T11:31:01.856518+0000","last_undegraded":"2024-04-20T11:31:01.856518+0000","last_fullsized":"2024-04-20T11:31:01.856518+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:39:57.271994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.886628+0000","last_change":"2024-04-20T11:31:00.880969+0000","last_active":"2024-04-20T11:31:00.886628+0000","last_peered":"2024-04-20T11:31:00.886628+0000","last_clean":"2024-04-20T11:31:00.886628+0000","last_became_active":"2024-04-20T11:31:00.880640+0000","last_became_peered":"2024-04-20T11:31:00.880640+0000","last_unstale":"2024-04-20T11:31:00.886628+0000","last_undegraded":"2024-04-20T11:31:00.886628+0000","last_fullsized":"2024-04-20T11:31:00.886628+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:30:16.658442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.889307+0000","last_change":"2024-04-20T11:31:00.877890+0000","last_active":"2024-04-20T11:31:00.889307+0000","last_peered":"2024-04-20T11:31:00.889307+0000","last_clean":"2024-04-20T11:31:00.889307+0000","last_became_active":"2024-04-20T11:31:00.877468+0000","last_became_peered":"2024-04-20T11:31:00.877468+0000","last_unstale":"2024-04-20T11:31:00.889307+0000","last_undegraded":"2024-04-20T11:31:00.889307+0000","last_fullsized":"2024-04-20T11:31:00.889307+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:22:28.047678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.877614+0000","last_change":"2024-04-20T11:31:00.873622+0000","last_active":"2024-04-20T11:31:00.877614+0000","last_peered":"2024-04-20T11:31:00.877614+0000","last_clean":"2024-04-20T11:31:00.877614+0000","last_became_active":"2024-04-20T11:31:00.873174+0000","last_became_peered":"2024-04-20T11:31:00.873174+0000","last_unstale":"2024-04-20T11:31:00.877614+0000","last_undegraded":"2024-04-20T11:31:00.877614+0000","last_fullsized":"2024-04-20T11:31:00.877614+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:54:15.217821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.893306+0000","last_change":"2024-04-20T11:31:00.886578+0000","last_active":"2024-04-20T11:31:00.893306+0000","last_peered":"2024-04-20T11:31:00.893306+0000","last_clean":"2024-04-20T11:31:00.893306+0000","last_became_active":"2024-04-20T11:31:00.886200+0000","last_became_peered":"2024-04-20T11:31:00.886200+0000","last_unstale":"2024-04-20T11:31:00.893306+0000","last_undegraded":"2024-04-20T11:31:00.893306+0000","last_fullsized":"2024-04-20T11:31:00.893306+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:39:57.271994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-20T11:31:01.858562+0000","last_change":"2024-04-20T11:31:00.891337+0000","last_active":"2024-04-20T11:31:01.858562+0000","last_peered":"2024-04-20T11:31:01.858562+0000","last_clean":"2024-04-20T11:31:01.858562+0000","last_became_active":"2024-04-20T11:31:00.890773+0000","last_became_peered":"2024-04-20T11:31:00.890773+0000","last_unstale":"2024-04-20T11:31:01.858562+0000","last_undegraded":"2024-04-20T11:31:01.858562+0000","last_fullsized":"2024-04-20T11:31:01.858562+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:32:40.001175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.903429+0000","last_change":"2024-04-20T11:31:00.895120+0000","last_active":"2024-04-20T11:31:00.903429+0000","last_peered":"2024-04-20T11:31:00.903429+0000","last_clean":"2024-04-20T11:31:00.903429+0000","last_became_active":"2024-04-20T11:31:00.894826+0000","last_became_peered":"2024-04-20T11:31:00.894826+0000","last_unstale":"2024-04-20T11:31:00.903429+0000","last_undegraded":"2024-04-20T11:31:00.903429+0000","last_fullsized":"2024-04-20T11:31:00.903429+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:26:37.214795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-20T11:31:01.855003+0000","last_change":"2024-04-20T11:31:00.878393+0000","last_active":"2024-04-20T11:31:01.855003+0000","last_peered":"2024-04-20T11:31:01.855003+0000","last_clean":"2024-04-20T11:31:01.855003+0000","last_became_active":"2024-04-20T11:31:00.878113+0000","last_became_peered":"2024-04-20T11:31:00.878113+0000","last_unstale":"2024-04-20T11:31:01.855003+0000","last_undegraded":"2024-04-20T11:31:01.855003+0000","last_fullsized":"2024-04-20T11:31:01.855003+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:21:07.403914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-20T11:31:01.857937+0000","last_change":"2024-04-20T11:31:00.870156+0000","last_active":"2024-04-20T11:31:01.857937+0000","last_peered":"2024-04-20T11:31:01.857937+0000","last_clean":"2024-04-20T11:31:01.857937+0000","last_became_active":"2024-04-20T11:31:00.869890+0000","last_became_peered":"2024-04-20T11:31:00.869890+0000","last_unstale":"2024-04-20T11:31:01.857937+0000","last_undegraded":"2024-04-20T11:31:01.857937+0000","last_fullsized":"2024-04-20T11:31:01.857937+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:29:11.139899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.903482+0000","last_change":"2024-04-20T11:31:00.896450+0000","last_active":"2024-04-20T11:31:00.903482+0000","last_peered":"2024-04-20T11:31:00.903482+0000","last_clean":"2024-04-20T11:31:00.903482+0000","last_became_active":"2024-04-20T11:31:00.896066+0000","last_became_peered":"2024-04-20T11:31:00.896066+0000","last_unstale":"2024-04-20T11:31:00.903482+0000","last_undegraded":"2024-04-20T11:31:00.903482+0000","last_fullsized":"2024-04-20T11:31:00.903482+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:36:24.761945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-20T11:31:01.858121+0000","last_change":"2024-04-20T11:31:00.883409+0000","last_active":"2024-04-20T11:31:01.858121+0000","last_peered":"2024-04-20T11:31:01.858121+0000","last_clean":"2024-04-20T11:31:01.858121+0000","last_became_active":"2024-04-20T11:31:00.883100+0000","last_became_peered":"2024-04-20T11:31:00.883100+0000","last_unstale":"2024-04-20T11:31:01.858121+0000","last_undegraded":"2024-04-20T11:31:01.858121+0000","last_fullsized":"2024-04-20T11:31:01.858121+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:55:05.047669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.903388+0000","last_change":"2024-04-20T11:31:00.893500+0000","last_active":"2024-04-20T11:31:00.903388+0000","last_peered":"2024-04-20T11:31:00.903388+0000","last_clean":"2024-04-20T11:31:00.903388+0000","last_became_active":"2024-04-20T11:31:00.893204+0000","last_became_peered":"2024-04-20T11:31:00.893204+0000","last_unstale":"2024-04-20T11:31:00.903388+0000","last_undegraded":"2024-04-20T11:31:00.903388+0000","last_fullsized":"2024-04-20T11:31:00.903388+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:02:57.616897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.886593+0000","last_change":"2024-04-20T11:31:00.883232+0000","last_active":"2024-04-20T11:31:00.886593+0000","last_peered":"2024-04-20T11:31:00.886593+0000","last_clean":"2024-04-20T11:31:00.886593+0000","last_became_active":"2024-04-20T11:31:00.882820+0000","last_became_peered":"2024-04-20T11:31:00.882820+0000","last_unstale":"2024-04-20T11:31:00.886593+0000","last_undegraded":"2024-04-20T11:31:00.886593+0000","last_fullsized":"2024-04-20T11:31:00.886593+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:13:11.698885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-20T11:31:01.854772+0000","last_change":"2024-04-20T11:31:00.861347+0000","last_active":"2024-04-20T11:31:01.854772+0000","last_peered":"2024-04-20T11:31:01.854772+0000","last_clean":"2024-04-20T11:31:01.854772+0000","last_became_active":"2024-04-20T11:31:00.861067+0000","last_became_peered":"2024-04-20T11:31:00.861067+0000","last_unstale":"2024-04-20T11:31:01.854772+0000","last_undegraded":"2024-04-20T11:31:01.854772+0000","last_fullsized":"2024-04-20T11:31:01.854772+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:06:39.367242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-20T11:31:01.856802+0000","last_change":"2024-04-20T11:31:00.879243+0000","last_active":"2024-04-20T11:31:01.856802+0000","last_peered":"2024-04-20T11:31:01.856802+0000","last_clean":"2024-04-20T11:31:01.856802+0000","last_became_active":"2024-04-20T11:31:00.878706+0000","last_became_peered":"2024-04-20T11:31:00.878706+0000","last_unstale":"2024-04-20T11:31:01.856802+0000","last_undegraded":"2024-04-20T11:31:01.856802+0000","last_fullsized":"2024-04-20T11:31:01.856802+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:23:22.124914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.893526+0000","last_change":"2024-04-20T11:31:00.888228+0000","last_active":"2024-04-20T11:31:00.893526+0000","last_peered":"2024-04-20T11:31:00.893526+0000","last_clean":"2024-04-20T11:31:00.893526+0000","last_became_active":"2024-04-20T11:31:00.887818+0000","last_became_peered":"2024-04-20T11:31:00.887818+0000","last_unstale":"2024-04-20T11:31:00.893526+0000","last_undegraded":"2024-04-20T11:31:00.893526+0000","last_fullsized":"2024-04-20T11:31:00.893526+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:04:46.835057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-20T11:31:01.854801+0000","last_change":"2024-04-20T11:31:00.873526+0000","last_active":"2024-04-20T11:31:01.854801+0000","last_peered":"2024-04-20T11:31:01.854801+0000","last_clean":"2024-04-20T11:31:01.854801+0000","last_became_active":"2024-04-20T11:31:00.873268+0000","last_became_peered":"2024-04-20T11:31:00.873268+0000","last_unstale":"2024-04-20T11:31:01.854801+0000","last_undegraded":"2024-04-20T11:31:01.854801+0000","last_fullsized":"2024-04-20T11:31:01.854801+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:53:40.226736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-20T11:31:01.857124+0000","last_change":"2024-04-20T11:31:00.875115+0000","last_active":"2024-04-20T11:31:01.857124+0000","last_peered":"2024-04-20T11:31:01.857124+0000","last_clean":"2024-04-20T11:31:01.857124+0000","last_became_active":"2024-04-20T11:31:00.874772+0000","last_became_peered":"2024-04-20T11:31:00.874772+0000","last_unstale":"2024-04-20T11:31:01.857124+0000","last_undegraded":"2024-04-20T11:31:01.857124+0000","last_fullsized":"2024-04-20T11:31:01.857124+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:22:28.866085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.903522+0000","last_change":"2024-04-20T11:31:00.898319+0000","last_active":"2024-04-20T11:31:00.903522+0000","last_peered":"2024-04-20T11:31:00.903522+0000","last_clean":"2024-04-20T11:31:00.903522+0000","last_became_active":"2024-04-20T11:31:00.897940+0000","last_became_peered":"2024-04-20T11:31:00.897940+0000","last_unstale":"2024-04-20T11:31:00.903522+0000","last_undegraded":"2024-04-20T11:31:00.903522+0000","last_fullsized":"2024-04-20T11:31:00.903522+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:47:44.049807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-20T11:31:01.854159+0000","last_change":"2024-04-20T11:31:00.870669+0000","last_active":"2024-04-20T11:31:01.854159+0000","last_peered":"2024-04-20T11:31:01.854159+0000","last_clean":"2024-04-20T11:31:01.854159+0000","last_became_active":"2024-04-20T11:31:00.870362+0000","last_became_peered":"2024-04-20T11:31:00.870362+0000","last_unstale":"2024-04-20T11:31:01.854159+0000","last_undegraded":"2024-04-20T11:31:01.854159+0000","last_fullsized":"2024-04-20T11:31:01.854159+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:50:01.584349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.884365+0000","last_change":"2024-04-20T11:31:00.881176+0000","last_active":"2024-04-20T11:31:00.884365+0000","last_peered":"2024-04-20T11:31:00.884365+0000","last_clean":"2024-04-20T11:31:00.884365+0000","last_became_active":"2024-04-20T11:31:00.880875+0000","last_became_peered":"2024-04-20T11:31:00.880875+0000","last_unstale":"2024-04-20T11:31:00.884365+0000","last_undegraded":"2024-04-20T11:31:00.884365+0000","last_fullsized":"2024-04-20T11:31:00.884365+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:42:53.717064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.893323+0000","last_change":"2024-04-20T11:31:00.888829+0000","last_active":"2024-04-20T11:31:00.893323+0000","last_peered":"2024-04-20T11:31:00.893323+0000","last_clean":"2024-04-20T11:31:00.893323+0000","last_became_active":"2024-04-20T11:31:00.888548+0000","last_became_peered":"2024-04-20T11:31:00.888548+0000","last_unstale":"2024-04-20T11:31:00.893323+0000","last_undegraded":"2024-04-20T11:31:00.893323+0000","last_fullsized":"2024-04-20T11:31:00.893323+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:22:38.955596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"59'4","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.890262+0000","last_change":"2024-04-20T11:30:59.878500+0000","last_active":"2024-04-20T11:31:00.890262+0000","last_peered":"2024-04-20T11:31:00.890262+0000","last_clean":"2024-04-20T11:31:00.890262+0000","last_became_active":"2024-04-20T11:30:59.878178+0000","last_became_peered":"2024-04-20T11:30:59.878178+0000","last_unstale":"2024-04-20T11:31:00.890262+0000","last_undegraded":"2024-04-20T11:31:00.890262+0000","last_fullsized":"2024-04-20T11:31:00.890262+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_clean_scrub_stamp":"2024-04-20T11:30:58.811830+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:02:56.600968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.893441+0000","last_change":"2024-04-20T11:31:00.887702+0000","last_active":"2024-04-20T11:31:00.893441+0000","last_peered":"2024-04-20T11:31:00.893441+0000","last_clean":"2024-04-20T11:31:00.893441+0000","last_became_active":"2024-04-20T11:31:00.887402+0000","last_became_peered":"2024-04-20T11:31:00.887402+0000","last_unstale":"2024-04-20T11:31:00.893441+0000","last_undegraded":"2024-04-20T11:31:00.893441+0000","last_fullsized":"2024-04-20T11:31:00.893441+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:42:15.418923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"59'1","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-20T11:31:01.858195+0000","last_change":"2024-04-20T11:30:59.880713+0000","last_active":"2024-04-20T11:31:01.858195+0000","last_peered":"2024-04-20T11:31:01.858195+0000","last_clean":"2024-04-20T11:31:01.858195+0000","last_became_active":"2024-04-20T11:30:59.880447+0000","last_became_peered":"2024-04-20T11:30:59.880447+0000","last_unstale":"2024-04-20T11:31:01.858195+0000","last_undegraded":"2024-04-20T11:31:01.858195+0000","last_fullsized":"2024-04-20T11:31:01.858195+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_clean_scrub_stamp":"2024-04-20T11:30:58.811830+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:21:50.892640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.879678+0000","last_change":"2024-04-20T11:31:00.872261+0000","last_active":"2024-04-20T11:31:00.879678+0000","last_peered":"2024-04-20T11:31:00.879678+0000","last_clean":"2024-04-20T11:31:00.879678+0000","last_became_active":"2024-04-20T11:31:00.872001+0000","last_became_peered":"2024-04-20T11:31:00.872001+0000","last_unstale":"2024-04-20T11:31:00.879678+0000","last_undegraded":"2024-04-20T11:31:00.879678+0000","last_fullsized":"2024-04-20T11:31:00.879678+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:49:17.613752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"59'2","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.888407+0000","last_change":"2024-04-20T11:30:59.873756+0000","last_active":"2024-04-20T11:31:00.888407+0000","last_peered":"2024-04-20T11:31:00.888407+0000","last_clean":"2024-04-20T11:31:00.888407+0000","last_became_active":"2024-04-20T11:30:59.873042+0000","last_became_peered":"2024-04-20T11:30:59.873042+0000","last_unstale":"2024-04-20T11:31:00.888407+0000","last_undegraded":"2024-04-20T11:31:00.888407+0000","last_fullsized":"2024-04-20T11:31:00.888407+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_clean_scrub_stamp":"2024-04-20T11:30:58.811830+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:26:36.198866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.882719+0000","last_change":"2024-04-20T11:31:00.877357+0000","last_active":"2024-04-20T11:31:00.882719+0000","last_peered":"2024-04-20T11:31:00.882719+0000","last_clean":"2024-04-20T11:31:00.882719+0000","last_became_active":"2024-04-20T11:31:00.877067+0000","last_became_peered":"2024-04-20T11:31:00.877067+0000","last_unstale":"2024-04-20T11:31:00.882719+0000","last_undegraded":"2024-04-20T11:31:00.882719+0000","last_fullsized":"2024-04-20T11:31:00.882719+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:47:44.049807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"59'2","reported_seq":16,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-20T11:31:01.856685+0000","last_change":"2024-04-20T11:30:59.878107+0000","last_active":"2024-04-20T11:31:01.856685+0000","last_peered":"2024-04-20T11:31:01.856685+0000","last_clean":"2024-04-20T11:31:01.856685+0000","last_became_active":"2024-04-20T11:30:59.877737+0000","last_became_peered":"2024-04-20T11:30:59.877737+0000","last_unstale":"2024-04-20T11:31:01.856685+0000","last_undegraded":"2024-04-20T11:31:01.856685+0000","last_fullsized":"2024-04-20T11:31:01.856685+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_clean_scrub_stamp":"2024-04-20T11:30:58.811830+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:54:14.201891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-20T11:31:01.857718+0000","last_change":"2024-04-20T11:31:00.884954+0000","last_active":"2024-04-20T11:31:01.857718+0000","last_peered":"2024-04-20T11:31:01.857718+0000","last_clean":"2024-04-20T11:31:01.857718+0000","last_became_active":"2024-04-20T11:31:00.884681+0000","last_became_peered":"2024-04-20T11:31:00.884681+0000","last_unstale":"2024-04-20T11:31:01.857718+0000","last_undegraded":"2024-04-20T11:31:01.857718+0000","last_fullsized":"2024-04-20T11:31:01.857718+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:38:19.059089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"59'1","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.893083+0000","last_change":"2024-04-20T11:30:59.881895+0000","last_active":"2024-04-20T11:31:00.893083+0000","last_peered":"2024-04-20T11:31:00.893083+0000","last_clean":"2024-04-20T11:31:00.893083+0000","last_became_active":"2024-04-20T11:30:59.881025+0000","last_became_peered":"2024-04-20T11:30:59.881025+0000","last_unstale":"2024-04-20T11:31:00.893083+0000","last_undegraded":"2024-04-20T11:31:00.893083+0000","last_fullsized":"2024-04-20T11:31:00.893083+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_clean_scrub_stamp":"2024-04-20T11:30:58.811830+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:27:21.889130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-20T11:31:01.857282+0000","last_change":"2024-04-20T11:31:00.871560+0000","last_active":"2024-04-20T11:31:01.857282+0000","last_peered":"2024-04-20T11:31:01.857282+0000","last_clean":"2024-04-20T11:31:01.857282+0000","last_became_active":"2024-04-20T11:31:00.871247+0000","last_became_peered":"2024-04-20T11:31:00.871247+0000","last_unstale":"2024-04-20T11:31:01.857282+0000","last_undegraded":"2024-04-20T11:31:01.857282+0000","last_fullsized":"2024-04-20T11:31:01.857282+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:11:35.588117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"59'1","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.887643+0000","last_change":"2024-04-20T11:30:59.868051+0000","last_active":"2024-04-20T11:31:00.887643+0000","last_peered":"2024-04-20T11:31:00.887643+0000","last_clean":"2024-04-20T11:31:00.887643+0000","last_became_active":"2024-04-20T11:30:59.867699+0000","last_became_peered":"2024-04-20T11:30:59.867699+0000","last_unstale":"2024-04-20T11:31:00.887643+0000","last_undegraded":"2024-04-20T11:31:00.887643+0000","last_fullsized":"2024-04-20T11:31:00.887643+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_clean_scrub_stamp":"2024-04-20T11:30:58.811830+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:22:27.031748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.902356+0000","last_change":"2024-04-20T11:31:00.892797+0000","last_active":"2024-04-20T11:31:00.902356+0000","last_peered":"2024-04-20T11:31:00.902356+0000","last_clean":"2024-04-20T11:31:00.902356+0000","last_became_active":"2024-04-20T11:31:00.892484+0000","last_became_peered":"2024-04-20T11:31:00.892484+0000","last_unstale":"2024-04-20T11:31:00.902356+0000","last_undegraded":"2024-04-20T11:31:00.902356+0000","last_fullsized":"2024-04-20T11:31:00.902356+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:47:35.633763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-20T11:31:01.855697+0000","last_change":"2024-04-20T11:30:59.878984+0000","last_active":"2024-04-20T11:31:01.855697+0000","last_peered":"2024-04-20T11:31:01.855697+0000","last_clean":"2024-04-20T11:31:01.855697+0000","last_became_active":"2024-04-20T11:30:59.878458+0000","last_became_peered":"2024-04-20T11:30:59.878458+0000","last_unstale":"2024-04-20T11:31:01.855697+0000","last_undegraded":"2024-04-20T11:31:01.855697+0000","last_fullsized":"2024-04-20T11:31:01.855697+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_clean_scrub_stamp":"2024-04-20T11:30:58.811830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:24:44.344016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-20T11:31:01.857523+0000","last_change":"2024-04-20T11:31:00.889857+0000","last_active":"2024-04-20T11:31:01.857523+0000","last_peered":"2024-04-20T11:31:01.857523+0000","last_clean":"2024-04-20T11:31:01.857523+0000","last_became_active":"2024-04-20T11:31:00.889606+0000","last_became_peered":"2024-04-20T11:31:00.889606+0000","last_unstale":"2024-04-20T11:31:01.857523+0000","last_undegraded":"2024-04-20T11:31:01.857523+0000","last_fullsized":"2024-04-20T11:31:01.857523+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:06:58.334912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.855995+0000","last_change":"2024-04-20T11:30:59.872604+0000","last_active":"2024-04-20T11:31:00.855995+0000","last_peered":"2024-04-20T11:31:00.855995+0000","last_clean":"2024-04-20T11:31:00.855995+0000","last_became_active":"2024-04-20T11:30:59.872298+0000","last_became_peered":"2024-04-20T11:30:59.872298+0000","last_unstale":"2024-04-20T11:31:00.855995+0000","last_undegraded":"2024-04-20T11:31:00.855995+0000","last_fullsized":"2024-04-20T11:31:00.855995+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_clean_scrub_stamp":"2024-04-20T11:30:58.811830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:52:08.145439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.881828+0000","last_change":"2024-04-20T11:31:00.875883+0000","last_active":"2024-04-20T11:31:00.881828+0000","last_peered":"2024-04-20T11:31:00.881828+0000","last_clean":"2024-04-20T11:31:00.881828+0000","last_became_active":"2024-04-20T11:31:00.874155+0000","last_became_peered":"2024-04-20T11:31:00.874155+0000","last_unstale":"2024-04-20T11:31:00.881828+0000","last_undegraded":"2024-04-20T11:31:00.881828+0000","last_fullsized":"2024-04-20T11:31:00.881828+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:31:41.278482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.882622+0000","last_change":"2024-04-20T11:30:59.880360+0000","last_active":"2024-04-20T11:31:00.882622+0000","last_peered":"2024-04-20T11:31:00.882622+0000","last_clean":"2024-04-20T11:31:00.882622+0000","last_became_active":"2024-04-20T11:30:59.880008+0000","last_became_peered":"2024-04-20T11:30:59.880008+0000","last_unstale":"2024-04-20T11:31:00.882622+0000","last_undegraded":"2024-04-20T11:31:00.882622+0000","last_fullsized":"2024-04-20T11:31:00.882622+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_clean_scrub_stamp":"2024-04-20T11:30:58.811830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:54:14.201891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.903435+0000","last_change":"2024-04-20T11:31:00.895533+0000","last_active":"2024-04-20T11:31:00.903435+0000","last_peered":"2024-04-20T11:31:00.903435+0000","last_clean":"2024-04-20T11:31:00.903435+0000","last_became_active":"2024-04-20T11:31:00.894689+0000","last_became_peered":"2024-04-20T11:31:00.894689+0000","last_unstale":"2024-04-20T11:31:00.903435+0000","last_undegraded":"2024-04-20T11:31:00.903435+0000","last_fullsized":"2024-04-20T11:31:00.903435+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:26:22.574534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-20T11:31:01.858182+0000","last_change":"2024-04-20T11:30:59.859762+0000","last_active":"2024-04-20T11:31:01.858182+0000","last_peered":"2024-04-20T11:31:01.858182+0000","last_clean":"2024-04-20T11:31:01.858182+0000","last_became_active":"2024-04-20T11:30:59.859432+0000","last_became_peered":"2024-04-20T11:30:59.859432+0000","last_unstale":"2024-04-20T11:31:01.858182+0000","last_undegraded":"2024-04-20T11:31:01.858182+0000","last_fullsized":"2024-04-20T11:31:01.858182+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_clean_scrub_stamp":"2024-04-20T11:30:58.811830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:39:56.256065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.884101+0000","last_change":"2024-04-20T11:31:00.879677+0000","last_active":"2024-04-20T11:31:00.884101+0000","last_peered":"2024-04-20T11:31:00.884101+0000","last_clean":"2024-04-20T11:31:00.884101+0000","last_became_active":"2024-04-20T11:31:00.879195+0000","last_became_peered":"2024-04-20T11:31:00.879195+0000","last_unstale":"2024-04-20T11:31:00.884101+0000","last_undegraded":"2024-04-20T11:31:00.884101+0000","last_fullsized":"2024-04-20T11:31:00.884101+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:47:35.633763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"59'3","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.902789+0000","last_change":"2024-04-20T11:30:59.876927+0000","last_active":"2024-04-20T11:31:00.902789+0000","last_peered":"2024-04-20T11:31:00.902789+0000","last_clean":"2024-04-20T11:31:00.902789+0000","last_became_active":"2024-04-20T11:30:59.876550+0000","last_became_peered":"2024-04-20T11:30:59.876550+0000","last_unstale":"2024-04-20T11:31:00.902789+0000","last_undegraded":"2024-04-20T11:31:00.902789+0000","last_fullsized":"2024-04-20T11:31:00.902789+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_clean_scrub_stamp":"2024-04-20T11:30:58.811830+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:31:40.262552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.903994+0000","last_change":"2024-04-20T11:31:00.902398+0000","last_active":"2024-04-20T11:31:00.903994+0000","last_peered":"2024-04-20T11:31:00.903994+0000","last_clean":"2024-04-20T11:31:00.903994+0000","last_became_active":"2024-04-20T11:31:00.901554+0000","last_became_peered":"2024-04-20T11:31:00.901554+0000","last_unstale":"2024-04-20T11:31:00.903994+0000","last_undegraded":"2024-04-20T11:31:00.903994+0000","last_fullsized":"2024-04-20T11:31:00.903994+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:42:53.717064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"59'6","reported_seq":18,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.905357+0000","last_change":"2024-04-20T11:30:59.871898+0000","last_active":"2024-04-20T11:31:00.905357+0000","last_peered":"2024-04-20T11:31:00.905357+0000","last_clean":"2024-04-20T11:31:00.905357+0000","last_became_active":"2024-04-20T11:30:59.871581+0000","last_became_peered":"2024-04-20T11:30:59.871581+0000","last_unstale":"2024-04-20T11:31:00.905357+0000","last_undegraded":"2024-04-20T11:31:00.905357+0000","last_fullsized":"2024-04-20T11:31:00.905357+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_clean_scrub_stamp":"2024-04-20T11:30:58.811830+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:43:03.824977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-20T11:31:01.856266+0000","last_change":"2024-04-20T11:31:00.877030+0000","last_active":"2024-04-20T11:31:01.856266+0000","last_peered":"2024-04-20T11:31:01.856266+0000","last_clean":"2024-04-20T11:31:01.856266+0000","last_became_active":"2024-04-20T11:31:00.876522+0000","last_became_peered":"2024-04-20T11:31:00.876522+0000","last_unstale":"2024-04-20T11:31:01.856266+0000","last_undegraded":"2024-04-20T11:31:01.856266+0000","last_fullsized":"2024-04-20T11:31:01.856266+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:21:51.908569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"59'2","reported_seq":16,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-20T11:31:01.856956+0000","last_change":"2024-04-20T11:30:59.857714+0000","last_active":"2024-04-20T11:31:01.856956+0000","last_peered":"2024-04-20T11:31:01.856956+0000","last_clean":"2024-04-20T11:31:01.856956+0000","last_became_active":"2024-04-20T11:30:59.856452+0000","last_became_peered":"2024-04-20T11:30:59.856452+0000","last_unstale":"2024-04-20T11:31:01.856956+0000","last_undegraded":"2024-04-20T11:31:01.856956+0000","last_fullsized":"2024-04-20T11:31:01.856956+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_clean_scrub_stamp":"2024-04-20T11:30:58.811830+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:42:52.701135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.901283+0000","last_change":"2024-04-20T11:31:00.890228+0000","last_active":"2024-04-20T11:31:00.901283+0000","last_peered":"2024-04-20T11:31:00.901283+0000","last_clean":"2024-04-20T11:31:00.901283+0000","last_became_active":"2024-04-20T11:31:00.889916+0000","last_became_peered":"2024-04-20T11:31:00.889916+0000","last_unstale":"2024-04-20T11:31:00.901283+0000","last_undegraded":"2024-04-20T11:31:00.901283+0000","last_fullsized":"2024-04-20T11:31:00.901283+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:22:28.047678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.873753+0000","last_change":"2024-04-20T11:30:59.874179+0000","last_active":"2024-04-20T11:31:00.873753+0000","last_peered":"2024-04-20T11:31:00.873753+0000","last_clean":"2024-04-20T11:31:00.873753+0000","last_became_active":"2024-04-20T11:30:59.872895+0000","last_became_peered":"2024-04-20T11:30:59.872895+0000","last_unstale":"2024-04-20T11:31:00.873753+0000","last_undegraded":"2024-04-20T11:31:00.873753+0000","last_fullsized":"2024-04-20T11:31:00.873753+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_clean_scrub_stamp":"2024-04-20T11:30:58.811830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:32:29.028175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.897544+0000","last_change":"2024-04-20T11:31:00.889922+0000","last_active":"2024-04-20T11:31:00.897544+0000","last_peered":"2024-04-20T11:31:00.897544+0000","last_clean":"2024-04-20T11:31:00.897544+0000","last_became_active":"2024-04-20T11:31:00.888986+0000","last_became_peered":"2024-04-20T11:31:00.888986+0000","last_unstale":"2024-04-20T11:31:00.897544+0000","last_undegraded":"2024-04-20T11:31:00.897544+0000","last_fullsized":"2024-04-20T11:31:00.897544+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:02:57.616897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-20T11:31:01.855785+0000","last_change":"2024-04-20T11:30:59.878746+0000","last_active":"2024-04-20T11:31:01.855785+0000","last_peered":"2024-04-20T11:31:01.855785+0000","last_clean":"2024-04-20T11:31:01.855785+0000","last_became_active":"2024-04-20T11:30:59.877899+0000","last_became_peered":"2024-04-20T11:30:59.877899+0000","last_unstale":"2024-04-20T11:31:01.855785+0000","last_undegraded":"2024-04-20T11:31:01.855785+0000","last_fullsized":"2024-04-20T11:31:01.855785+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_clean_scrub_stamp":"2024-04-20T11:30:58.811830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:43:03.824977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.903363+0000","last_change":"2024-04-20T11:31:00.893527+0000","last_active":"2024-04-20T11:31:00.903363+0000","last_peered":"2024-04-20T11:31:00.903363+0000","last_clean":"2024-04-20T11:31:00.903363+0000","last_became_active":"2024-04-20T11:31:00.893249+0000","last_became_peered":"2024-04-20T11:31:00.893249+0000","last_unstale":"2024-04-20T11:31:00.903363+0000","last_undegraded":"2024-04-20T11:31:00.903363+0000","last_fullsized":"2024-04-20T11:31:00.903363+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:50:43.035072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"59'8","reported_seq":20,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.903123+0000","last_change":"2024-04-20T11:30:59.879029+0000","last_active":"2024-04-20T11:31:00.903123+0000","last_peered":"2024-04-20T11:31:00.903123+0000","last_clean":"2024-04-20T11:31:00.903123+0000","last_became_active":"2024-04-20T11:30:59.878708+0000","last_became_peered":"2024-04-20T11:30:59.878708+0000","last_unstale":"2024-04-20T11:31:00.903123+0000","last_undegraded":"2024-04-20T11:31:00.903123+0000","last_fullsized":"2024-04-20T11:31:00.903123+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_clean_scrub_stamp":"2024-04-20T11:30:58.811830+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:06:48.103643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.903450+0000","last_change":"2024-04-20T11:31:00.895176+0000","last_active":"2024-04-20T11:31:00.903450+0000","last_peered":"2024-04-20T11:31:00.903450+0000","last_clean":"2024-04-20T11:31:00.903450+0000","last_became_active":"2024-04-20T11:31:00.894853+0000","last_became_peered":"2024-04-20T11:31:00.894853+0000","last_unstale":"2024-04-20T11:31:00.903450+0000","last_undegraded":"2024-04-20T11:31:00.903450+0000","last_fullsized":"2024-04-20T11:31:00.903450+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:46:17.185811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"59'1","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-20T11:31:01.858418+0000","last_change":"2024-04-20T11:30:59.861988+0000","last_active":"2024-04-20T11:31:01.858418+0000","last_peered":"2024-04-20T11:31:01.858418+0000","last_clean":"2024-04-20T11:31:01.858418+0000","last_became_active":"2024-04-20T11:30:59.861699+0000","last_became_peered":"2024-04-20T11:30:59.861699+0000","last_unstale":"2024-04-20T11:31:01.858418+0000","last_undegraded":"2024-04-20T11:31:01.858418+0000","last_fullsized":"2024-04-20T11:31:01.858418+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_clean_scrub_stamp":"2024-04-20T11:30:58.811830+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:26:08.927093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-20T11:31:01.857316+0000","last_change":"2024-04-20T11:31:00.875699+0000","last_active":"2024-04-20T11:31:01.857316+0000","last_peered":"2024-04-20T11:31:01.857316+0000","last_clean":"2024-04-20T11:31:01.857316+0000","last_became_active":"2024-04-20T11:31:00.875215+0000","last_became_peered":"2024-04-20T11:31:00.875215+0000","last_unstale":"2024-04-20T11:31:01.857316+0000","last_undegraded":"2024-04-20T11:31:01.857316+0000","last_fullsized":"2024-04-20T11:31:01.857316+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:26:09.943022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"59'2","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.884584+0000","last_change":"2024-04-20T11:30:59.868667+0000","last_active":"2024-04-20T11:31:00.884584+0000","last_peered":"2024-04-20T11:31:00.884584+0000","last_clean":"2024-04-20T11:31:00.884584+0000","last_became_active":"2024-04-20T11:30:59.868217+0000","last_became_peered":"2024-04-20T11:30:59.868217+0000","last_unstale":"2024-04-20T11:31:00.884584+0000","last_undegraded":"2024-04-20T11:31:00.884584+0000","last_fullsized":"2024-04-20T11:31:00.884584+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_clean_scrub_stamp":"2024-04-20T11:30:58.811830+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:46:16.169882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-20T11:31:01.857523+0000","last_change":"2024-04-20T11:31:00.881414+0000","last_active":"2024-04-20T11:31:01.857523+0000","last_peered":"2024-04-20T11:31:01.857523+0000","last_clean":"2024-04-20T11:31:01.857523+0000","last_became_active":"2024-04-20T11:31:00.880204+0000","last_became_peered":"2024-04-20T11:31:00.880204+0000","last_unstale":"2024-04-20T11:31:01.857523+0000","last_undegraded":"2024-04-20T11:31:01.857523+0000","last_fullsized":"2024-04-20T11:31:01.857523+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:37:46.224157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"59'1","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-20T11:31:01.856723+0000","last_change":"2024-04-20T11:30:59.881525+0000","last_active":"2024-04-20T11:31:01.856723+0000","last_peered":"2024-04-20T11:31:01.856723+0000","last_clean":"2024-04-20T11:31:01.856723+0000","last_became_active":"2024-04-20T11:30:59.880812+0000","last_became_peered":"2024-04-20T11:30:59.880812+0000","last_unstale":"2024-04-20T11:31:01.856723+0000","last_undegraded":"2024-04-20T11:31:01.856723+0000","last_fullsized":"2024-04-20T11:31:01.856723+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_clean_scrub_stamp":"2024-04-20T11:30:58.811830+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:06:48.103643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.904005+0000","last_change":"2024-04-20T11:31:00.903090+0000","last_active":"2024-04-20T11:31:00.904005+0000","last_peered":"2024-04-20T11:31:00.904005+0000","last_clean":"2024-04-20T11:31:00.904005+0000","last_became_active":"2024-04-20T11:31:00.902825+0000","last_became_peered":"2024-04-20T11:31:00.902825+0000","last_unstale":"2024-04-20T11:31:00.904005+0000","last_undegraded":"2024-04-20T11:31:00.904005+0000","last_fullsized":"2024-04-20T11:31:00.904005+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:39:57.271994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.861903+0000","last_change":"2024-04-20T11:30:59.871645+0000","last_active":"2024-04-20T11:31:00.861903+0000","last_peered":"2024-04-20T11:31:00.861903+0000","last_clean":"2024-04-20T11:31:00.861903+0000","last_became_active":"2024-04-20T11:30:59.871342+0000","last_became_peered":"2024-04-20T11:30:59.871342+0000","last_unstale":"2024-04-20T11:31:00.861903+0000","last_undegraded":"2024-04-20T11:31:00.861903+0000","last_fullsized":"2024-04-20T11:31:00.861903+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_clean_scrub_stamp":"2024-04-20T11:30:58.811830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:50:42.019142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-20T11:31:01.856040+0000","last_change":"2024-04-20T11:31:00.873059+0000","last_active":"2024-04-20T11:31:01.856040+0000","last_peered":"2024-04-20T11:31:01.856040+0000","last_clean":"2024-04-20T11:31:01.856040+0000","last_became_active":"2024-04-20T11:31:00.872349+0000","last_became_peered":"2024-04-20T11:31:00.872349+0000","last_unstale":"2024-04-20T11:31:01.856040+0000","last_undegraded":"2024-04-20T11:31:01.856040+0000","last_fullsized":"2024-04-20T11:31:01.856040+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:47:44.049807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"59'2","reported_seq":16,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-20T11:31:01.857575+0000","last_change":"2024-04-20T11:30:59.869576+0000","last_active":"2024-04-20T11:31:01.857575+0000","last_peered":"2024-04-20T11:31:01.857575+0000","last_clean":"2024-04-20T11:31:01.857575+0000","last_became_active":"2024-04-20T11:30:59.869303+0000","last_became_peered":"2024-04-20T11:30:59.869303+0000","last_unstale":"2024-04-20T11:31:01.857575+0000","last_undegraded":"2024-04-20T11:31:01.857575+0000","last_fullsized":"2024-04-20T11:31:01.857575+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_clean_scrub_stamp":"2024-04-20T11:30:58.811830+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:23:21.108985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.893495+0000","last_change":"2024-04-20T11:31:00.886909+0000","last_active":"2024-04-20T11:31:00.893495+0000","last_peered":"2024-04-20T11:31:00.893495+0000","last_clean":"2024-04-20T11:31:00.893495+0000","last_became_active":"2024-04-20T11:31:00.886609+0000","last_became_peered":"2024-04-20T11:31:00.886609+0000","last_unstale":"2024-04-20T11:31:00.893495+0000","last_undegraded":"2024-04-20T11:31:00.893495+0000","last_fullsized":"2024-04-20T11:31:00.893495+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:23:22.124914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.866182+0000","last_change":"2024-04-20T11:30:59.863261+0000","last_active":"2024-04-20T11:31:00.866182+0000","last_peered":"2024-04-20T11:31:00.866182+0000","last_clean":"2024-04-20T11:31:00.866182+0000","last_became_active":"2024-04-20T11:30:59.862977+0000","last_became_peered":"2024-04-20T11:30:59.862977+0000","last_unstale":"2024-04-20T11:31:00.866182+0000","last_undegraded":"2024-04-20T11:31:00.866182+0000","last_fullsized":"2024-04-20T11:31:00.866182+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_clean_scrub_stamp":"2024-04-20T11:30:58.811830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:30:15.642512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.893351+0000","last_change":"2024-04-20T11:31:00.888491+0000","last_active":"2024-04-20T11:31:00.893351+0000","last_peered":"2024-04-20T11:31:00.893351+0000","last_clean":"2024-04-20T11:31:00.893351+0000","last_became_active":"2024-04-20T11:31:00.888215+0000","last_became_peered":"2024-04-20T11:31:00.888215+0000","last_unstale":"2024-04-20T11:31:00.893351+0000","last_undegraded":"2024-04-20T11:31:00.893351+0000","last_fullsized":"2024-04-20T11:31:00.893351+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:26:09.943022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-20T11:31:01.855481+0000","last_change":"2024-04-20T11:30:59.877636+0000","last_active":"2024-04-20T11:31:01.855481+0000","last_peered":"2024-04-20T11:31:01.855481+0000","last_clean":"2024-04-20T11:31:01.855481+0000","last_became_active":"2024-04-20T11:30:59.877279+0000","last_became_peered":"2024-04-20T11:30:59.877279+0000","last_unstale":"2024-04-20T11:31:01.855481+0000","last_undegraded":"2024-04-20T11:31:01.855481+0000","last_fullsized":"2024-04-20T11:31:01.855481+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_clean_scrub_stamp":"2024-04-20T11:30:58.811830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:27:21.889130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-20T11:31:01.857130+0000","last_change":"2024-04-20T11:31:00.885922+0000","last_active":"2024-04-20T11:31:01.857130+0000","last_peered":"2024-04-20T11:31:01.857130+0000","last_clean":"2024-04-20T11:31:01.857130+0000","last_became_active":"2024-04-20T11:31:00.885648+0000","last_became_peered":"2024-04-20T11:31:00.885648+0000","last_unstale":"2024-04-20T11:31:01.857130+0000","last_undegraded":"2024-04-20T11:31:01.857130+0000","last_fullsized":"2024-04-20T11:31:01.857130+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:04:46.835057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"59'2","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.885278+0000","last_change":"2024-04-20T11:30:59.869998+0000","last_active":"2024-04-20T11:31:00.885278+0000","last_peered":"2024-04-20T11:31:00.885278+0000","last_clean":"2024-04-20T11:31:00.885278+0000","last_became_active":"2024-04-20T11:30:59.869559+0000","last_became_peered":"2024-04-20T11:30:59.869559+0000","last_unstale":"2024-04-20T11:31:00.885278+0000","last_undegraded":"2024-04-20T11:31:00.885278+0000","last_fullsized":"2024-04-20T11:31:00.885278+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_clean_scrub_stamp":"2024-04-20T11:30:58.811830+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:17:11.482838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-20T11:31:01.857303+0000","last_change":"2024-04-20T11:31:00.888344+0000","last_active":"2024-04-20T11:31:01.857303+0000","last_peered":"2024-04-20T11:31:01.857303+0000","last_clean":"2024-04-20T11:31:01.857303+0000","last_became_active":"2024-04-20T11:31:00.888049+0000","last_became_peered":"2024-04-20T11:31:00.888049+0000","last_unstale":"2024-04-20T11:31:01.857303+0000","last_undegraded":"2024-04-20T11:31:01.857303+0000","last_fullsized":"2024-04-20T11:31:01.857303+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:22:38.955596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.864564+0000","last_change":"2024-04-20T11:30:59.867927+0000","last_active":"2024-04-20T11:31:00.864564+0000","last_peered":"2024-04-20T11:31:00.864564+0000","last_clean":"2024-04-20T11:31:00.864564+0000","last_became_active":"2024-04-20T11:30:59.867580+0000","last_became_peered":"2024-04-20T11:30:59.867580+0000","last_unstale":"2024-04-20T11:31:00.864564+0000","last_undegraded":"2024-04-20T11:31:00.864564+0000","last_fullsized":"2024-04-20T11:31:00.864564+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_clean_scrub_stamp":"2024-04-20T11:30:58.811830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:26:21.558604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.884078+0000","last_change":"2024-04-20T11:31:00.879445+0000","last_active":"2024-04-20T11:31:00.884078+0000","last_peered":"2024-04-20T11:31:00.884078+0000","last_clean":"2024-04-20T11:31:00.884078+0000","last_became_active":"2024-04-20T11:31:00.878871+0000","last_became_peered":"2024-04-20T11:31:00.878871+0000","last_unstale":"2024-04-20T11:31:00.884078+0000","last_undegraded":"2024-04-20T11:31:00.884078+0000","last_fullsized":"2024-04-20T11:31:00.884078+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:13:11.698885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.880639+0000","last_change":"2024-04-20T11:30:59.873959+0000","last_active":"2024-04-20T11:31:00.880639+0000","last_peered":"2024-04-20T11:31:00.880639+0000","last_clean":"2024-04-20T11:31:00.880639+0000","last_became_active":"2024-04-20T11:30:59.868495+0000","last_became_peered":"2024-04-20T11:30:59.868495+0000","last_unstale":"2024-04-20T11:31:00.880639+0000","last_undegraded":"2024-04-20T11:31:00.880639+0000","last_fullsized":"2024-04-20T11:31:00.880639+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_clean_scrub_stamp":"2024-04-20T11:30:58.811830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:17:11.482838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.882740+0000","last_change":"2024-04-20T11:31:00.877496+0000","last_active":"2024-04-20T11:31:00.882740+0000","last_peered":"2024-04-20T11:31:00.882740+0000","last_clean":"2024-04-20T11:31:00.882740+0000","last_became_active":"2024-04-20T11:31:00.876527+0000","last_became_peered":"2024-04-20T11:31:00.876527+0000","last_unstale":"2024-04-20T11:31:00.882740+0000","last_undegraded":"2024-04-20T11:31:00.882740+0000","last_fullsized":"2024-04-20T11:31:00.882740+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:21:51.908569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-20T11:31:01.856746+0000","last_change":"2024-04-20T11:30:59.882562+0000","last_active":"2024-04-20T11:31:01.856746+0000","last_peered":"2024-04-20T11:31:01.856746+0000","last_clean":"2024-04-20T11:31:01.856746+0000","last_became_active":"2024-04-20T11:30:59.882277+0000","last_became_peered":"2024-04-20T11:30:59.882277+0000","last_unstale":"2024-04-20T11:31:01.856746+0000","last_undegraded":"2024-04-20T11:31:01.856746+0000","last_fullsized":"2024-04-20T11:31:01.856746+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_clean_scrub_stamp":"2024-04-20T11:30:58.811830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:31:40.262552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-20T11:31:01.857886+0000","last_change":"2024-04-20T11:31:00.875951+0000","last_active":"2024-04-20T11:31:01.857886+0000","last_peered":"2024-04-20T11:31:01.857886+0000","last_clean":"2024-04-20T11:31:01.857886+0000","last_became_active":"2024-04-20T11:31:00.875700+0000","last_became_peered":"2024-04-20T11:31:00.875700+0000","last_unstale":"2024-04-20T11:31:01.857886+0000","last_undegraded":"2024-04-20T11:31:01.857886+0000","last_fullsized":"2024-04-20T11:31:01.857886+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:23:42.556940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.861010+0000","last_change":"2024-04-20T11:30:59.876568+0000","last_active":"2024-04-20T11:31:00.861010+0000","last_peered":"2024-04-20T11:31:00.861010+0000","last_clean":"2024-04-20T11:31:00.861010+0000","last_became_active":"2024-04-20T11:30:59.876145+0000","last_became_peered":"2024-04-20T11:30:59.876145+0000","last_unstale":"2024-04-20T11:31:00.861010+0000","last_undegraded":"2024-04-20T11:31:00.861010+0000","last_fullsized":"2024-04-20T11:31:00.861010+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_clean_scrub_stamp":"2024-04-20T11:30:58.811830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:52:51.929802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.903460+0000","last_change":"2024-04-20T11:31:00.896169+0000","last_active":"2024-04-20T11:31:00.903460+0000","last_peered":"2024-04-20T11:31:00.903460+0000","last_clean":"2024-04-20T11:31:00.903460+0000","last_became_active":"2024-04-20T11:31:00.895761+0000","last_became_peered":"2024-04-20T11:31:00.895761+0000","last_unstale":"2024-04-20T11:31:00.903460+0000","last_undegraded":"2024-04-20T11:31:00.903460+0000","last_fullsized":"2024-04-20T11:31:00.903460+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:49:17.613752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.859559+0000","last_change":"2024-04-20T11:30:59.873436+0000","last_active":"2024-04-20T11:31:00.859559+0000","last_peered":"2024-04-20T11:31:00.859559+0000","last_clean":"2024-04-20T11:31:00.859559+0000","last_became_active":"2024-04-20T11:30:59.873162+0000","last_became_peered":"2024-04-20T11:30:59.873162+0000","last_unstale":"2024-04-20T11:31:00.859559+0000","last_undegraded":"2024-04-20T11:31:00.859559+0000","last_fullsized":"2024-04-20T11:31:00.859559+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_clean_scrub_stamp":"2024-04-20T11:30:58.811830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:20:28.268336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.893224+0000","last_change":"2024-04-20T11:31:00.886086+0000","last_active":"2024-04-20T11:31:00.893224+0000","last_peered":"2024-04-20T11:31:00.893224+0000","last_clean":"2024-04-20T11:31:00.893224+0000","last_became_active":"2024-04-20T11:31:00.885821+0000","last_became_peered":"2024-04-20T11:31:00.885821+0000","last_unstale":"2024-04-20T11:31:00.893224+0000","last_undegraded":"2024-04-20T11:31:00.893224+0000","last_fullsized":"2024-04-20T11:31:00.893224+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:27:06.123345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.857691+0000","last_change":"2024-04-20T11:30:59.871396+0000","last_active":"2024-04-20T11:31:00.857691+0000","last_peered":"2024-04-20T11:31:00.857691+0000","last_clean":"2024-04-20T11:31:00.857691+0000","last_became_active":"2024-04-20T11:30:59.871101+0000","last_became_peered":"2024-04-20T11:30:59.871101+0000","last_unstale":"2024-04-20T11:31:00.857691+0000","last_undegraded":"2024-04-20T11:31:00.857691+0000","last_fullsized":"2024-04-20T11:31:00.857691+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_clean_scrub_stamp":"2024-04-20T11:30:58.811830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:32:29.028175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-20T11:31:01.857783+0000","last_change":"2024-04-20T11:31:00.889262+0000","last_active":"2024-04-20T11:31:01.857783+0000","last_peered":"2024-04-20T11:31:01.857783+0000","last_clean":"2024-04-20T11:31:01.857783+0000","last_became_active":"2024-04-20T11:31:00.888999+0000","last_became_peered":"2024-04-20T11:31:00.888999+0000","last_unstale":"2024-04-20T11:31:01.857783+0000","last_undegraded":"2024-04-20T11:31:01.857783+0000","last_fullsized":"2024-04-20T11:31:01.857783+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:43:11.158343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-20T11:31:01.857147+0000","last_change":"2024-04-20T11:30:59.875128+0000","last_active":"2024-04-20T11:31:01.857147+0000","last_peered":"2024-04-20T11:31:01.857147+0000","last_clean":"2024-04-20T11:31:01.857147+0000","last_became_active":"2024-04-20T11:30:59.874728+0000","last_became_peered":"2024-04-20T11:30:59.874728+0000","last_unstale":"2024-04-20T11:31:01.857147+0000","last_undegraded":"2024-04-20T11:31:01.857147+0000","last_fullsized":"2024-04-20T11:31:01.857147+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_clean_scrub_stamp":"2024-04-20T11:30:58.811830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:04:45.819127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-20T11:31:01.857004+0000","last_change":"2024-04-20T11:31:00.870992+0000","last_active":"2024-04-20T11:31:01.857004+0000","last_peered":"2024-04-20T11:31:01.857004+0000","last_clean":"2024-04-20T11:31:01.857004+0000","last_became_active":"2024-04-20T11:31:00.870724+0000","last_became_peered":"2024-04-20T11:31:00.870724+0000","last_unstale":"2024-04-20T11:31:01.857004+0000","last_undegraded":"2024-04-20T11:31:01.857004+0000","last_fullsized":"2024-04-20T11:31:01.857004+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:59.827759+0000","last_clean_scrub_stamp":"2024-04-20T11:30:59.827759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:24:23.886081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.876286+0000","last_change":"2024-04-20T11:30:59.874922+0000","last_active":"2024-04-20T11:31:00.876286+0000","last_peered":"2024-04-20T11:31:00.876286+0000","last_clean":"2024-04-20T11:31:00.876286+0000","last_became_active":"2024-04-20T11:30:59.874492+0000","last_became_peered":"2024-04-20T11:30:59.874492+0000","last_unstale":"2024-04-20T11:31:00.876286+0000","last_undegraded":"2024-04-20T11:31:00.876286+0000","last_fullsized":"2024-04-20T11:31:00.876286+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:30:58.811830+0000","last_clean_scrub_stamp":"2024-04-20T11:30:58.811830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:52:51.929802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":240,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T11:31:00.879541+0000","last_change":"2024-04-20T11:26:48.661897+0000","last_active":"2024-04-20T11:31:00.879541+0000","last_peered":"2024-04-20T11:31:00.879541+0000","last_clean":"2024-04-20T11:31:00.879541+0000","last_became_active":"2024-04-20T11:26:48.656535+0000","last_became_peered":"2024-04-20T11:26:48.656535+0000","last_unstale":"2024-04-20T11:31:00.879541+0000","last_undegraded":"2024-04-20T11:31:00.879541+0000","last_fullsized":"2024-04-20T11:31:00.879541+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:47.543004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:47.543004+0000","last_clean_scrub_stamp":"2024-04-20T11:26:47.543004+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:58:35.527105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574904,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32460,"kb_used_data":1280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339380,"statfs":{"total":96636764160,"available":96603525120,"internally_reserved":0,"allocated":1310720,"data_stored":886041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574901,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27716,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344124,"statfs":{"total":96636764160,"available":96608382976,"internally_reserved":0,"allocated":708608,"data_stored":295513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574903,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31880,"kb_used_data":700,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339960,"statfs":{"total":96636764160,"available":96604119040,"internally_reserved":0,"allocated":716800,"data_stored":296165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574901,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27724,"kb_used_data":704,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344116,"statfs":{"total":96636764160,"available":96608374784,"internally_reserved":0,"allocated":720896,"data_stored":297125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574902,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31880,"kb_used_data":700,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339960,"statfs":{"total":96636764160,"available":96604119040,"internally_reserved":0,"allocated":716800,"data_stored":295569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574903,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27740,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344100,"statfs":{"total":96636764160,"available":96608358400,"internally_reserved":0,"allocated":729088,"data_stored":301869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574901,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31876,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339964,"statfs":{"total":96636764160,"available":96604123136,"internally_reserved":0,"allocated":712704,"data_stored":295603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574902,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32468,"kb_used_data":1288,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339372,"statfs":{"total":96636764160,"available":96603516928,"internally_reserved":0,"allocated":1318912,"data_stored":888075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T11:31:05.791 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-20T11:31:05.791 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-20T11:31:06.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.041+0000 7ff272e00640 1 Processor -- start 2024-04-20T11:31:06.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.041+0000 7ff272e00640 1 -- start start 2024-04-20T11:31:06.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.043+0000 7ff272e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff26c0b4fb0 0x7ff26c15fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:06.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.043+0000 7ff272e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ff26c06dc60 0x7ff26c0b3670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:06.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.043+0000 7ff272e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7ff26c0b5390 0x7ff26c0b3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:06.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.043+0000 7ff272e00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7ff26c160080 con 0x7ff26c0b5390 2024-04-20T11:31:06.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.043+0000 7ff272e00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7ff26c06ec70 con 0x7ff26c06dc60 2024-04-20T11:31:06.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.043+0000 7ff272e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff26c06fad0 con 0x7ff26c0b4fb0 2024-04-20T11:31:06.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.043+0000 7ff270e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7ff26c0b5390 0x7ff26c0b3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:06.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.043+0000 7ff270e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7ff26c0b5390 0x7ff26c0b3bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36842/0 (socket says 172.21.15.152:36842) 2024-04-20T11:31:06.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.043+0000 7ff270e00640 1 -- 172.21.15.152:0/2134374641 learned_addr learned my addr 172.21.15.152:0/2134374641 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:06.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.043+0000 7ff26b400640 1 --2- 172.21.15.152:0/2134374641 >> v2:172.21.15.175:3300/0 conn(0x7ff26c06dc60 0x7ff26c0b3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:06.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.043+0000 7ff26be00640 1 --2- 172.21.15.152:0/2134374641 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff26c0b4fb0 0x7ff26c15fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:06.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.044+0000 7ff270e00640 1 -- 172.21.15.152:0/2134374641 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff26c0b4fb0 msgr2=0x7ff26c15fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:06.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.044+0000 7ff270e00640 1 --2- 172.21.15.152:0/2134374641 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff26c0b4fb0 0x7ff26c15fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:06.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.044+0000 7ff270e00640 1 -- 172.21.15.152:0/2134374641 >> v2:172.21.15.175:3300/0 conn(0x7ff26c06dc60 msgr2=0x7ff26c0b3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:06.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.044+0000 7ff270e00640 1 --2- 172.21.15.152:0/2134374641 >> v2:172.21.15.175:3300/0 conn(0x7ff26c06dc60 0x7ff26c0b3670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:06.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.044+0000 7ff270e00640 1 -- 172.21.15.152:0/2134374641 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff26c06e0f0 con 0x7ff26c0b5390 2024-04-20T11:31:06.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.044+0000 7ff26b400640 1 --2- 172.21.15.152:0/2134374641 >> v2:172.21.15.175:3300/0 conn(0x7ff26c06dc60 0x7ff26c0b3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:31:06.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.045+0000 7ff270e00640 1 --2- 172.21.15.152:0/2134374641 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff26c0b5390 0x7ff26c0b3bb0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7ff26c0735c0 tx=0x7ff26402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4735f6e432f1598c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:06.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.045+0000 7ff26aa00640 1 -- 172.21.15.152:0/2134374641 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff26402c9e0 con 0x7ff26c0b5390 2024-04-20T11:31:06.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.045+0000 7ff26aa00640 1 -- 172.21.15.152:0/2134374641 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff26402cb80 con 0x7ff26c0b5390 2024-04-20T11:31:06.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.045+0000 7ff26aa00640 1 -- 172.21.15.152:0/2134374641 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2640343f0 con 0x7ff26c0b5390 2024-04-20T11:31:06.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.046+0000 7ff272e00640 1 -- 172.21.15.152:0/2134374641 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff26c0b5390 msgr2=0x7ff26c0b3bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:06.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.046+0000 7ff272e00640 1 --2- 172.21.15.152:0/2134374641 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff26c0b5390 0x7ff26c0b3bb0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7ff26c0735c0 tx=0x7ff26402be40 comp rx=0 tx=0).stop 2024-04-20T11:31:06.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.046+0000 7ff272e00640 1 -- 172.21.15.152:0/2134374641 shutdown_connections 2024-04-20T11:31:06.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.046+0000 7ff272e00640 1 --2- 172.21.15.152:0/2134374641 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff26c0b5390 0x7ff26c0b3bb0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:06.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.046+0000 7ff272e00640 1 --2- 172.21.15.152:0/2134374641 >> v2:172.21.15.175:3300/0 conn(0x7ff26c06dc60 0x7ff26c0b3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:06.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.046+0000 7ff272e00640 1 --2- 172.21.15.152:0/2134374641 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff26c0b4fb0 0x7ff26c15fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:06.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.046+0000 7ff272e00640 1 -- 172.21.15.152:0/2134374641 >> 172.21.15.152:0/2134374641 conn(0x7ff26c06b980 msgr2=0x7ff26c06bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:06.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.046+0000 7ff272e00640 1 -- 172.21.15.152:0/2134374641 shutdown_connections 2024-04-20T11:31:06.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.046+0000 7ff272e00640 1 -- 172.21.15.152:0/2134374641 wait complete. 2024-04-20T11:31:06.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.048+0000 7ff272e00640 1 Processor -- start 2024-04-20T11:31:06.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.049+0000 7ff272e00640 1 -- start start 2024-04-20T11:31:06.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.050+0000 7ff272e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff26c0b4fb0 0x7ff26c0ae530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:06.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.050+0000 7ff272e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff26c0b5390 0x7ff26c158ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:06.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.050+0000 7ff272e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff26c1f7960 0x7ff26c1f7e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:06.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.050+0000 7ff272e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7ff26c06e990 con 0x7ff26c0b5390 2024-04-20T11:31:06.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.050+0000 7ff272e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff26c06e6b0 con 0x7ff26c0b4fb0 2024-04-20T11:31:06.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.050+0000 7ff272e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff26c0b40f0 con 0x7ff26c1f7960 2024-04-20T11:31:06.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.050+0000 7ff26b400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff26c0b5390 0x7ff26c158ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:06.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.050+0000 7ff26b400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff26c0b5390 0x7ff26c158ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36850/0 (socket says 172.21.15.152:36850) 2024-04-20T11:31:06.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.050+0000 7ff26b400640 1 -- 172.21.15.152:0/770495214 learned_addr learned my addr 172.21.15.152:0/770495214 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:06.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.050+0000 7ff26b400640 1 -- 172.21.15.152:0/770495214 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff26c1f7960 msgr2=0x7ff26c1f7e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:06.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.050+0000 7ff270e00640 1 --2- 172.21.15.152:0/770495214 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff26c1f7960 0x7ff26c1f7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:06.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.050+0000 7ff26be00640 1 --2- 172.21.15.152:0/770495214 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff26c0b4fb0 0x7ff26c0ae530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:06.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.050+0000 7ff26b400640 1 --2- 172.21.15.152:0/770495214 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff26c1f7960 0x7ff26c1f7e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:06.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.050+0000 7ff26b400640 1 -- 172.21.15.152:0/770495214 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff26c0b4fb0 msgr2=0x7ff26c0ae530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:06.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.050+0000 7ff26b400640 1 --2- 172.21.15.152:0/770495214 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff26c0b4fb0 0x7ff26c0ae530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:06.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.050+0000 7ff26b400640 1 -- 172.21.15.152:0/770495214 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff26c1f8350 con 0x7ff26c0b5390 2024-04-20T11:31:06.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.051+0000 7ff26b400640 1 --2- 172.21.15.152:0/770495214 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff26c0b5390 0x7ff26c158ef0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7ff25800c7f0 tx=0x7ff25800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:06.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.051+0000 7ff263e00640 1 -- 172.21.15.152:0/770495214 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff258004230 con 0x7ff26c0b5390 2024-04-20T11:31:06.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.051+0000 7ff263e00640 1 -- 172.21.15.152:0/770495214 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2580043f0 con 0x7ff26c0b5390 2024-04-20T11:31:06.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.051+0000 7ff272e00640 1 -- 172.21.15.152:0/770495214 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff26c1f85e0 con 0x7ff26c0b5390 2024-04-20T11:31:06.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.051+0000 7ff263e00640 1 -- 172.21.15.152:0/770495214 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff258007860 con 0x7ff26c0b5390 2024-04-20T11:31:06.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.051+0000 7ff272e00640 1 -- 172.21.15.152:0/770495214 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff26c0add30 con 0x7ff26c0b5390 2024-04-20T11:31:06.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.052+0000 7ff272e00640 1 -- 172.21.15.152:0/770495214 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff26c0689f0 con 0x7ff26c0b5390 2024-04-20T11:31:06.054 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.053+0000 7ff263e00640 1 -- 172.21.15.152:0/770495214 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 9) ==== 91193+0+0 (secure 0 0 0) 0x7ff258007c10 con 0x7ff26c0b5390 2024-04-20T11:31:06.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.054+0000 7ff263e00640 1 --2- 172.21.15.152:0/770495214 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff24c06caa0 0x7ff24c06ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:06.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.054+0000 7ff263e00640 1 -- 172.21.15.152:0/770495214 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5709+0+0 (secure 0 0 0) 0x7ff25808c260 con 0x7ff26c0b5390 2024-04-20T11:31:06.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.055+0000 7ff26be00640 1 --2- 172.21.15.152:0/770495214 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff24c06caa0 0x7ff24c06ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:06.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.055+0000 7ff26be00640 1 --2- 172.21.15.152:0/770495214 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff24c06caa0 0x7ff24c06ef60 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7ff25c006fd0 tx=0x7ff25c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:06.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.059+0000 7ff263e00640 1 -- 172.21.15.152:0/770495214 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ff258015020 con 0x7ff26c0b5390 2024-04-20T11:31:06.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.334+0000 7ff272e00640 1 -- 172.21.15.152:0/770495214 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7ff26c06ee90 con 0x7ff26c0b5390 2024-04-20T11:31:06.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.339+0000 7ff263e00640 1 -- 172.21.15.152:0/770495214 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) ==== 190+0+147 (secure 0 0 0) 0x7ff25805a270 con 0x7ff26c0b5390 2024-04-20T11:31:06.340 INFO:teuthology.orchestra.run.smithi152.stdout:[client.0] 2024-04-20T11:31:06.340 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQBDpiNmtsBWFhAArq9A7r33CcwgMmOU2pAGPw== 2024-04-20T11:31:06.340 INFO:teuthology.orchestra.run.smithi152.stdout: caps mds = "allow" 2024-04-20T11:31:06.340 INFO:teuthology.orchestra.run.smithi152.stdout: caps mon = "allow r" 2024-04-20T11:31:06.340 INFO:teuthology.orchestra.run.smithi152.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-20T11:31:06.340 INFO:teuthology.orchestra.run.smithi152.stderr:updated caps for client.0 2024-04-20T11:31:06.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.341+0000 7ff272e00640 1 -- 172.21.15.152:0/770495214 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff24c06caa0 msgr2=0x7ff24c06ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:06.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.341+0000 7ff272e00640 1 --2- 172.21.15.152:0/770495214 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff24c06caa0 0x7ff24c06ef60 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7ff25c006fd0 tx=0x7ff25c008040 comp rx=0 tx=0).stop 2024-04-20T11:31:06.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.342+0000 7ff272e00640 1 -- 172.21.15.152:0/770495214 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff26c0b5390 msgr2=0x7ff26c158ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:06.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.342+0000 7ff272e00640 1 --2- 172.21.15.152:0/770495214 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff26c0b5390 0x7ff26c158ef0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7ff25800c7f0 tx=0x7ff25800ccb0 comp rx=0 tx=0).stop 2024-04-20T11:31:06.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.342+0000 7ff272e00640 1 -- 172.21.15.152:0/770495214 shutdown_connections 2024-04-20T11:31:06.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.342+0000 7ff272e00640 1 --2- 172.21.15.152:0/770495214 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff24c06caa0 0x7ff24c06ef60 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:06.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.342+0000 7ff272e00640 1 --2- 172.21.15.152:0/770495214 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff26c1f7960 0x7ff26c1f7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:06.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.342+0000 7ff272e00640 1 --2- 172.21.15.152:0/770495214 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff26c0b5390 0x7ff26c158ef0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:06.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.342+0000 7ff272e00640 1 --2- 172.21.15.152:0/770495214 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff26c0b4fb0 0x7ff26c0ae530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:06.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.342+0000 7ff272e00640 1 -- 172.21.15.152:0/770495214 >> 172.21.15.152:0/770495214 conn(0x7ff26c06b980 msgr2=0x7ff26c157b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:06.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.342+0000 7ff272e00640 1 -- 172.21.15.152:0/770495214 shutdown_connections 2024-04-20T11:31:06.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.342+0000 7ff272e00640 1 -- 172.21.15.152:0/770495214 wait complete. 2024-04-20T11:31:06.361 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:31:06.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.557+0000 7fbd9a200640 1 Processor -- start 2024-04-20T11:31:06.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.557+0000 7fbd9a200640 1 -- start start 2024-04-20T11:31:06.559 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.558+0000 7fbd9a200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd94145f60 0x7fbd9406ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:06.559 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.558+0000 7fbd9a200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fbd940726f0 0x7fbd940cd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:06.559 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.558+0000 7fbd9a200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd94146340 con 0x7fbd940cd0d0 2024-04-20T11:31:06.559 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.558+0000 7fbd9a200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fbd9406e9f0 con 0x7fbd940726f0 2024-04-20T11:31:06.559 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.558+0000 7fbd9a200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fbd9406f3c0 con 0x7fbd94145f60 2024-04-20T11:31:06.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.559+0000 7fbd93e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fbd940726f0 0x7fbd940cd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:06.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.559+0000 7fbd93e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fbd940726f0 0x7fbd940cd550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36866/0 (socket says 172.21.15.152:36866) 2024-04-20T11:31:06.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.559+0000 7fbd93e00640 1 -- 172.21.15.152:0/849178216 learned_addr learned my addr 172.21.15.152:0/849178216 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:06.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.559+0000 7fbd93400640 1 --2- 172.21.15.152:0/849178216 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd94145f60 0x7fbd9406ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:06.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.559+0000 7fbd93e00640 1 -- 172.21.15.152:0/849178216 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd94145f60 msgr2=0x7fbd9406ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:06.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.559+0000 7fbd93e00640 1 --2- 172.21.15.152:0/849178216 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd94145f60 0x7fbd9406ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:06.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.559+0000 7fbd93e00640 1 -- 172.21.15.152:0/849178216 >> v1:172.21.15.175:6789/0 conn(0x7fbd940cd0d0 legacy=0x7fbd94071fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:06.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.559+0000 7fbd93e00640 1 -- 172.21.15.152:0/849178216 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd94072d50 con 0x7fbd940726f0 2024-04-20T11:31:06.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.560+0000 7fbd93e00640 1 --2- 172.21.15.152:0/849178216 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd940726f0 0x7fbd940cd550 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fbd9406eb70 tx=0x7fbd8c02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cb306ccc83bf1e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:06.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.561+0000 7fbd92a00640 1 -- 172.21.15.152:0/849178216 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd8c02ca50 con 0x7fbd940726f0 2024-04-20T11:31:06.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.561+0000 7fbd92a00640 1 -- 172.21.15.152:0/849178216 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd8c02cbf0 con 0x7fbd940726f0 2024-04-20T11:31:06.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.561+0000 7fbd92a00640 1 -- 172.21.15.152:0/849178216 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd8c0343f0 con 0x7fbd940726f0 2024-04-20T11:31:06.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.561+0000 7fbd9a200640 1 -- 172.21.15.152:0/849178216 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd940726f0 msgr2=0x7fbd940cd550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:06.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.561+0000 7fbd9a200640 1 --2- 172.21.15.152:0/849178216 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd940726f0 0x7fbd940cd550 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fbd9406eb70 tx=0x7fbd8c02beb0 comp rx=0 tx=0).stop 2024-04-20T11:31:06.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.562+0000 7fbd9a200640 1 -- 172.21.15.152:0/849178216 shutdown_connections 2024-04-20T11:31:06.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.562+0000 7fbd9a200640 1 --2- 172.21.15.152:0/849178216 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd940726f0 0x7fbd940cd550 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:06.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.562+0000 7fbd9a200640 1 --2- 172.21.15.152:0/849178216 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd94145f60 0x7fbd9406ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:06.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.562+0000 7fbd9a200640 1 -- 172.21.15.152:0/849178216 >> 172.21.15.152:0/849178216 conn(0x7fbd94067cf0 msgr2=0x7fbd9406b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:06.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.562+0000 7fbd9a200640 1 -- 172.21.15.152:0/849178216 shutdown_connections 2024-04-20T11:31:06.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.562+0000 7fbd9a200640 1 -- 172.21.15.152:0/849178216 wait complete. 2024-04-20T11:31:06.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.563+0000 7fbd9a200640 1 Processor -- start 2024-04-20T11:31:06.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.564+0000 7fbd9a200640 1 -- start start 2024-04-20T11:31:06.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.564+0000 7fbd9a200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd940726f0 0x7fbd941e5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:06.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.564+0000 7fbd9a200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd940cd0d0 0x7fbd941e8080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:06.567 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.564+0000 7fbd9a200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd94145f60 0x7fbd941e7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:06.567 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.564+0000 7fbd9a200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fbd9406e680 con 0x7fbd940cd0d0 2024-04-20T11:31:06.567 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.564+0000 7fbd9a200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbd9406e3a0 con 0x7fbd94145f60 2024-04-20T11:31:06.567 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.564+0000 7fbd9a200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fbd940cf290 con 0x7fbd940726f0 2024-04-20T11:31:06.567 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.565+0000 7fbd8aa00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd940cd0d0 0x7fbd941e8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:06.567 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.565+0000 7fbd8aa00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd940cd0d0 0x7fbd941e8080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36870/0 (socket says 172.21.15.152:36870) 2024-04-20T11:31:06.567 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.565+0000 7fbd8aa00640 1 -- 172.21.15.152:0/2078874958 learned_addr learned my addr 172.21.15.152:0/2078874958 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:06.567 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.565+0000 7fbd93e00640 1 --2- 172.21.15.152:0/2078874958 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd94145f60 0x7fbd941e7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:06.567 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.565+0000 7fbd8aa00640 1 -- 172.21.15.152:0/2078874958 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd940726f0 msgr2=0x7fbd941e5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:06.567 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.565+0000 7fbd8aa00640 1 --2- 172.21.15.152:0/2078874958 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd940726f0 0x7fbd941e5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:06.567 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.565+0000 7fbd8aa00640 1 -- 172.21.15.152:0/2078874958 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd94145f60 msgr2=0x7fbd941e7230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:06.567 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.565+0000 7fbd8aa00640 1 --2- 172.21.15.152:0/2078874958 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd94145f60 0x7fbd941e7230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:06.567 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.565+0000 7fbd8aa00640 1 -- 172.21.15.152:0/2078874958 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd941e6190 con 0x7fbd940cd0d0 2024-04-20T11:31:06.567 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.565+0000 7fbd93e00640 1 --2- 172.21.15.152:0/2078874958 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd94145f60 0x7fbd941e7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:31:06.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.566+0000 7fbd8aa00640 1 --2- 172.21.15.152:0/2078874958 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd940cd0d0 0x7fbd941e8080 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c00c430 tx=0x7fbd7c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:06.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.566+0000 7fbd8be00640 1 -- 172.21.15.152:0/2078874958 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd7c011790 con 0x7fbd940cd0d0 2024-04-20T11:31:06.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.566+0000 7fbd8be00640 1 -- 172.21.15.152:0/2078874958 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd7c017070 con 0x7fbd940cd0d0 2024-04-20T11:31:06.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.566+0000 7fbd8be00640 1 -- 172.21.15.152:0/2078874958 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd7c011bb0 con 0x7fbd940cd0d0 2024-04-20T11:31:06.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.566+0000 7fbd9a200640 1 -- 172.21.15.152:0/2078874958 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd941e6470 con 0x7fbd940cd0d0 2024-04-20T11:31:06.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.566+0000 7fbd9a200640 1 -- 172.21.15.152:0/2078874958 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd941e6a30 con 0x7fbd940cd0d0 2024-04-20T11:31:06.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.567+0000 7fbd9a200640 1 -- 172.21.15.152:0/2078874958 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd94069360 con 0x7fbd940cd0d0 2024-04-20T11:31:06.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.568+0000 7fbd8be00640 1 -- 172.21.15.152:0/2078874958 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 9) ==== 91193+0+0 (secure 0 0 0) 0x7fbd7c01c460 con 0x7fbd940cd0d0 2024-04-20T11:31:06.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.569+0000 7fbd8be00640 1 --2- 172.21.15.152:0/2078874958 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fbd7006cad0 0x7fbd7006ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:06.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.569+0000 7fbd8be00640 1 -- 172.21.15.152:0/2078874958 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5709+0+0 (secure 0 0 0) 0x7fbd7c08be20 con 0x7fbd940cd0d0 2024-04-20T11:31:06.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.569+0000 7fbd93400640 1 --2- 172.21.15.152:0/2078874958 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fbd7006cad0 0x7fbd7006ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:06.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.570+0000 7fbd93400640 1 --2- 172.21.15.152:0/2078874958 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fbd7006cad0 0x7fbd7006ef90 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fbd84006fd0 tx=0x7fbd84008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:06.574 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.573+0000 7fbd8be00640 1 -- 172.21.15.152:0/2078874958 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fbd94069360 con 0x7fbd940cd0d0 2024-04-20T11:31:06.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.816+0000 7fbd9a200640 1 -- 172.21.15.152:0/2078874958 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbd940689f0 con 0x7fbd940cd0d0 2024-04-20T11:31:06.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.817+0000 7fbd8be00640 1 -- 172.21.15.152:0/2078874958 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+5205 (secure 0 0 0) 0x7fbd7c059ea0 con 0x7fbd940cd0d0 2024-04-20T11:31:06.818 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:31:06.818 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":33,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5045,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/3268046291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3268046291},{"type":"v1","addr":"172.21.15.175:6835","nonce":3268046291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5347,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/845526345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":845526345},{"type":"v1","addr":"172.21.15.175:6833","nonce":845526345}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":6288,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6836/3582785355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":3582785355},{"type":"v1","addr":"172.21.15.152:6836","nonce":3582785355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T11:31:00.308725+0000","modified":"2024-04-20T11:31:03.369713+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6291":{"gid":6291,"name":"c","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6837/488955771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6835","nonce":488955771},{"type":"v1","addr":"172.21.15.152:6837","nonce":488955771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6291,"qdb_cluster":[6291]},"id":4}]} 2024-04-20T11:31:06.818 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 33 2024-04-20T11:31:06.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.820+0000 7fbd9a200640 1 -- 172.21.15.152:0/2078874958 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fbd7006cad0 msgr2=0x7fbd7006ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:06.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.820+0000 7fbd9a200640 1 --2- 172.21.15.152:0/2078874958 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fbd7006cad0 0x7fbd7006ef90 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fbd84006fd0 tx=0x7fbd84008040 comp rx=0 tx=0).stop 2024-04-20T11:31:06.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.820+0000 7fbd9a200640 1 -- 172.21.15.152:0/2078874958 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd940cd0d0 msgr2=0x7fbd941e8080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:06.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.820+0000 7fbd9a200640 1 --2- 172.21.15.152:0/2078874958 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd940cd0d0 0x7fbd941e8080 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c00c430 tx=0x7fbd7c00c8f0 comp rx=0 tx=0).stop 2024-04-20T11:31:06.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.820+0000 7fbd9a200640 1 -- 172.21.15.152:0/2078874958 shutdown_connections 2024-04-20T11:31:06.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.820+0000 7fbd9a200640 1 --2- 172.21.15.152:0/2078874958 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fbd7006cad0 0x7fbd7006ef90 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:06.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.820+0000 7fbd9a200640 1 --2- 172.21.15.152:0/2078874958 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd94145f60 0x7fbd941e7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:06.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.820+0000 7fbd9a200640 1 --2- 172.21.15.152:0/2078874958 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd940cd0d0 0x7fbd941e8080 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:06.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.820+0000 7fbd9a200640 1 --2- 172.21.15.152:0/2078874958 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd940726f0 0x7fbd941e5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:06.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.820+0000 7fbd9a200640 1 -- 172.21.15.152:0/2078874958 >> 172.21.15.152:0/2078874958 conn(0x7fbd94067cf0 msgr2=0x7fbd9415f8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:06.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.820+0000 7fbd9a200640 1 -- 172.21.15.152:0/2078874958 shutdown_connections 2024-04-20T11:31:06.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:06.820+0000 7fbd9a200640 1 -- 172.21.15.152:0/2078874958 wait complete. 2024-04-20T11:31:07.245 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 33, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-20T11:31:00.308725+0000', 'modified': '2024-04-20T11:31:03.369713+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'snap_rstat': True, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6291}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6291': {'gid': 6291, 'name': 'c', 'rank': 0, 'incarnation': 29, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.152:6837/488955771', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6835', 'nonce': 488955771}, {'type': 'v1', 'addr': '172.21.15.152:6837', 'nonce': 488955771}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6291, 'qdb_cluster': [6291]} 2024-04-20T11:31:07.245 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-20T11:31:07.245 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-20T11:31:07.457 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.456+0000 7f9c72200640 1 Processor -- start 2024-04-20T11:31:07.457 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.457+0000 7f9c72200640 1 -- start start 2024-04-20T11:31:07.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.458+0000 7f9c72200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9c6c15c7f0 0x7f9c6c072b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:07.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.458+0000 7f9c72200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f9c6c0b4870 0x7f9c6c0ad2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:07.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.458+0000 7f9c72200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c6c073fc0 con 0x7f9c6c073b60 2024-04-20T11:31:07.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.458+0000 7f9c72200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f9c6c0ce140 con 0x7f9c6c0b4870 2024-04-20T11:31:07.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.458+0000 7f9c72200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f9c6c0ce390 con 0x7f9c6c15c7f0 2024-04-20T11:31:07.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.459+0000 7f9c70e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9c6c15c7f0 0x7f9c6c072b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:07.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.459+0000 7f9c70e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9c6c15c7f0 0x7f9c6c072b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.152:37354/0 (socket says 172.21.15.152:37354) 2024-04-20T11:31:07.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.459+0000 7f9c6be00640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7f9c6c073b60 0x7f9c6c0b4160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.152:36224/0 (socket says 172.21.15.152:36224) 2024-04-20T11:31:07.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.459+0000 7f9c71800640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f9c6c0b4870 0x7f9c6c0ad2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:07.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.459+0000 7f9c70e00640 1 -- 172.21.15.152:0/1869667516 learned_addr learned my addr 172.21.15.152:0/1869667516 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:07.461 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.460+0000 7f9c71800640 1 -- 172.21.15.152:0/1869667516 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9c6c15c7f0 msgr2=0x7f9c6c072b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:07.461 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.460+0000 7f9c71800640 1 --2- 172.21.15.152:0/1869667516 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9c6c15c7f0 0x7f9c6c072b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:07.461 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.460+0000 7f9c71800640 1 -- 172.21.15.152:0/1869667516 >> v1:172.21.15.175:6789/0 conn(0x7f9c6c073b60 legacy=0x7f9c6c0b4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:07.461 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.460+0000 7f9c71800640 1 -- 172.21.15.152:0/1869667516 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c6c0ce670 con 0x7f9c6c0b4870 2024-04-20T11:31:07.462 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.461+0000 7f9c71800640 1 --2- 172.21.15.152:0/1869667516 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c6c0b4870 0x7f9c6c0ad2b0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f9c64009d90 tx=0x7f9c6402be20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2318074d70ecb63 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:07.463 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.462+0000 7f9c6b400640 1 -- 172.21.15.152:0/1869667516 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c64039070 con 0x7f9c6c0b4870 2024-04-20T11:31:07.463 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.462+0000 7f9c6b400640 1 -- 172.21.15.152:0/1869667516 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c6402cbd0 con 0x7f9c6c0b4870 2024-04-20T11:31:07.463 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.462+0000 7f9c6b400640 1 -- 172.21.15.152:0/1869667516 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c64034440 con 0x7f9c6c0b4870 2024-04-20T11:31:07.464 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.463+0000 7f9c72200640 1 -- 172.21.15.152:0/1869667516 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c6c0b4870 msgr2=0x7f9c6c0ad2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:07.464 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.463+0000 7f9c72200640 1 --2- 172.21.15.152:0/1869667516 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c6c0b4870 0x7f9c6c0ad2b0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f9c64009d90 tx=0x7f9c6402be20 comp rx=0 tx=0).stop 2024-04-20T11:31:07.464 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.463+0000 7f9c72200640 1 -- 172.21.15.152:0/1869667516 shutdown_connections 2024-04-20T11:31:07.464 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.463+0000 7f9c72200640 1 --2- 172.21.15.152:0/1869667516 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c6c0b4870 0x7f9c6c0ad2b0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:07.464 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.463+0000 7f9c72200640 1 --2- 172.21.15.152:0/1869667516 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9c6c15c7f0 0x7f9c6c072b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:07.464 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.463+0000 7f9c72200640 1 -- 172.21.15.152:0/1869667516 >> 172.21.15.152:0/1869667516 conn(0x7f9c6c0d0260 msgr2=0x7f9c6c0d0670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:07.464 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.463+0000 7f9c72200640 1 -- 172.21.15.152:0/1869667516 shutdown_connections 2024-04-20T11:31:07.464 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.463+0000 7f9c72200640 1 -- 172.21.15.152:0/1869667516 wait complete. 2024-04-20T11:31:07.466 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.465+0000 7f9c72200640 1 Processor -- start 2024-04-20T11:31:07.466 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.466+0000 7f9c72200640 1 -- start start 2024-04-20T11:31:07.468 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.467+0000 7f9c72200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9c6c073b60 0x7f9c6c1505f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:07.468 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.467+0000 7f9c72200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c6c0b4870 0x7f9c6c14c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:07.468 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.467+0000 7f9c72200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9c6c150b30 0x7f9c6c14e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:07.468 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.467+0000 7f9c72200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f9c6c0cef10 con 0x7f9c6c150b30 2024-04-20T11:31:07.468 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.467+0000 7f9c72200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9c6c0cec30 con 0x7f9c6c0b4870 2024-04-20T11:31:07.468 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.467+0000 7f9c72200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f9c6c0cf430 con 0x7f9c6c073b60 2024-04-20T11:31:07.468 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.467+0000 7f9c71800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9c6c150b30 0x7f9c6c14e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:07.468 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.467+0000 7f9c71800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9c6c150b30 0x7f9c6c14e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36874/0 (socket says 172.21.15.152:36874) 2024-04-20T11:31:07.468 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.467+0000 7f9c71800640 1 -- 172.21.15.152:0/4122849435 learned_addr learned my addr 172.21.15.152:0/4122849435 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:07.469 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.468+0000 7f9c70e00640 1 --2- 172.21.15.152:0/4122849435 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9c6c073b60 0x7f9c6c1505f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:07.469 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.468+0000 7f9c6be00640 1 --2- 172.21.15.152:0/4122849435 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c6c0b4870 0x7f9c6c14c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:07.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.468+0000 7f9c71800640 1 -- 172.21.15.152:0/4122849435 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9c6c073b60 msgr2=0x7f9c6c1505f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:07.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.468+0000 7f9c71800640 1 --2- 172.21.15.152:0/4122849435 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9c6c073b60 0x7f9c6c1505f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:07.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.468+0000 7f9c71800640 1 -- 172.21.15.152:0/4122849435 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c6c0b4870 msgr2=0x7f9c6c14c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:07.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.468+0000 7f9c71800640 1 --2- 172.21.15.152:0/4122849435 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c6c0b4870 0x7f9c6c14c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:07.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.468+0000 7f9c71800640 1 -- 172.21.15.152:0/4122849435 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c6c1f7b90 con 0x7f9c6c150b30 2024-04-20T11:31:07.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.468+0000 7f9c71800640 1 --2- 172.21.15.152:0/4122849435 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9c6c150b30 0x7f9c6c14e860 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f9c64009eb0 tx=0x7f9c64004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:07.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.468+0000 7f9c68c00640 1 -- 172.21.15.152:0/4122849435 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c64044020 con 0x7f9c6c150b30 2024-04-20T11:31:07.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.468+0000 7f9c68c00640 1 -- 172.21.15.152:0/4122849435 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c6403f470 con 0x7f9c6c150b30 2024-04-20T11:31:07.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.468+0000 7f9c68c00640 1 -- 172.21.15.152:0/4122849435 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c64039040 con 0x7f9c6c150b30 2024-04-20T11:31:07.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.468+0000 7f9c72200640 1 -- 172.21.15.152:0/4122849435 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c6c1f7e70 con 0x7f9c6c150b30 2024-04-20T11:31:07.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.469+0000 7f9c72200640 1 -- 172.21.15.152:0/4122849435 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c6c1f8430 con 0x7f9c6c150b30 2024-04-20T11:31:07.471 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.471+0000 7f9c68c00640 1 -- 172.21.15.152:0/4122849435 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 9) ==== 91193+0+0 (secure 0 0 0) 0x7f9c64047020 con 0x7f9c6c150b30 2024-04-20T11:31:07.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.472+0000 7f9c68c00640 1 --2- 172.21.15.152:0/4122849435 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f9c4806caa0 0x7f9c4806ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:07.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.472+0000 7f9c68c00640 1 -- 172.21.15.152:0/4122849435 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5709+0+0 (secure 0 0 0) 0x7f9c640ae960 con 0x7f9c6c150b30 2024-04-20T11:31:07.473 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.472+0000 7f9c70e00640 1 --2- 172.21.15.152:0/4122849435 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f9c4806caa0 0x7f9c4806ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:07.473 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.473+0000 7f9c765d9740 1 Processor -- start 2024-04-20T11:31:07.473 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.473+0000 7f9c765d9740 1 -- start start 2024-04-20T11:31:07.473 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.473+0000 7f9c70e00640 1 --2- 172.21.15.152:0/4122849435 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f9c4806caa0 0x7f9c4806ef60 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c009ac0 tx=0x7f9c5c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:07.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.474+0000 7f9c765d9740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f94aa6a670 0x55f94aa21ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:07.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.474+0000 7f9c765d9740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f94aa206f0 0x55f94aa221e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:07.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.474+0000 7f9c765d9740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f94aa1cb90 0x55f94aa4c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:07.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.474+0000 7f9c765d9740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55f94a9889d0 con 0x55f94aa206f0 2024-04-20T11:31:07.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.474+0000 7f9c765d9740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55f94a92f270 con 0x55f94aa1cb90 2024-04-20T11:31:07.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.474+0000 7f9c765d9740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x55f94a984990 con 0x55f94aa6a670 2024-04-20T11:31:07.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.474+0000 7f9c6be00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f94aa206f0 0x55f94aa221e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:07.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.474+0000 7f9c6be00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f94aa206f0 0x55f94aa221e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36886/0 (socket says 172.21.15.152:36886) 2024-04-20T11:31:07.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.474+0000 7f9c6be00640 1 -- 172.21.15.152:0/263268949 learned_addr learned my addr 172.21.15.152:0/263268949 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:07.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.474+0000 7f9c70e00640 1 --2- 172.21.15.152:0/263268949 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f94aa6a670 0x55f94aa21ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:07.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.475+0000 7f9c71800640 1 --2- 172.21.15.152:0/263268949 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f94aa1cb90 0x55f94aa4c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:07.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.475+0000 7f9c6be00640 1 -- 172.21.15.152:0/263268949 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f94aa6a670 msgr2=0x55f94aa21ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:07.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.475+0000 7f9c6be00640 1 --2- 172.21.15.152:0/263268949 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f94aa6a670 0x55f94aa21ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:07.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.475+0000 7f9c6be00640 1 -- 172.21.15.152:0/263268949 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f94aa1cb90 msgr2=0x55f94aa4c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:07.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.475+0000 7f9c6be00640 1 --2- 172.21.15.152:0/263268949 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f94aa1cb90 0x55f94aa4c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:07.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.475+0000 7f9c6be00640 1 -- 172.21.15.152:0/263268949 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f94a69b2d0 con 0x55f94aa206f0 2024-04-20T11:31:07.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.475+0000 7f9c6be00640 1 --2- 172.21.15.152:0/263268949 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f94aa206f0 0x55f94aa221e0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f9c54005cf0 tx=0x7f9c54007420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=887cf202e3f7d56a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:07.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.475+0000 7f9c61600640 1 -- 172.21.15.152:0/263268949 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c54019070 con 0x55f94aa206f0 2024-04-20T11:31:07.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.475+0000 7f9c61600640 1 -- 172.21.15.152:0/263268949 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c54001700 con 0x55f94aa206f0 2024-04-20T11:31:07.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.475+0000 7f9c61600640 1 -- 172.21.15.152:0/263268949 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c54001ad0 con 0x55f94aa206f0 2024-04-20T11:31:07.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.476+0000 7f9c765d9740 1 -- 172.21.15.152:0/263268949 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f94aa206f0 msgr2=0x55f94aa221e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:07.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.476+0000 7f9c765d9740 1 --2- 172.21.15.152:0/263268949 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f94aa206f0 0x55f94aa221e0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f9c54005cf0 tx=0x7f9c54007420 comp rx=0 tx=0).stop 2024-04-20T11:31:07.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.476+0000 7f9c765d9740 1 -- 172.21.15.152:0/263268949 shutdown_connections 2024-04-20T11:31:07.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.476+0000 7f9c765d9740 1 --2- 172.21.15.152:0/263268949 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f94aa1cb90 0x55f94aa4c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:07.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.476+0000 7f9c765d9740 1 --2- 172.21.15.152:0/263268949 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f94aa206f0 0x55f94aa221e0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:07.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.476+0000 7f9c765d9740 1 --2- 172.21.15.152:0/263268949 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f94aa6a670 0x55f94aa21ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:07.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.476+0000 7f9c765d9740 1 -- 172.21.15.152:0/263268949 >> 172.21.15.152:0/263268949 conn(0x55f94aa1ee00 msgr2=0x55f94aa67980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:07.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.476+0000 7f9c765d9740 1 -- 172.21.15.152:0/263268949 shutdown_connections 2024-04-20T11:31:07.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.476+0000 7f9c765d9740 1 -- 172.21.15.152:0/263268949 wait complete. 2024-04-20T11:31:07.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.477+0000 7f9c765d9740 1 -- start start 2024-04-20T11:31:07.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.477+0000 7f9c765d9740 1 Processor -- start 2024-04-20T11:31:07.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.478+0000 7f9c765d9740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f94aa206f0 0x55f94aa6e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:07.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.478+0000 7f9c765d9740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f94aa6a670 0x55f94aa4b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:07.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.478+0000 7f9c765d9740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f94aa45170 0x55f94aa4bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:07.482 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.478+0000 7f9c765d9740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55f94aa20c00 con 0x55f94aa6a670 2024-04-20T11:31:07.482 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.478+0000 7f9c765d9740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55f94a9425e0 con 0x55f94aa45170 2024-04-20T11:31:07.482 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.478+0000 7f9c765d9740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x55f94aa20ee0 con 0x55f94aa206f0 2024-04-20T11:31:07.482 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.478+0000 7f9c765d9740 10 client.0 _ll_register_callbacks cb 0x55f94a69aac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:31:07.482 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.478+0000 7f9c6be00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f94aa6a670 0x55f94aa4b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:07.482 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.478+0000 7f9c6be00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f94aa6a670 0x55f94aa4b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36898/0 (socket says 172.21.15.152:36898) 2024-04-20T11:31:07.482 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.478+0000 7f9c6be00640 1 -- 172.21.15.152:0/2598935355 learned_addr learned my addr 172.21.15.152:0/2598935355 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:07.482 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.479+0000 7f9c70e00640 1 --2- 172.21.15.152:0/2598935355 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f94aa206f0 0x55f94aa6e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:07.482 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.479+0000 7f9c71800640 1 --2- 172.21.15.152:0/2598935355 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f94aa45170 0x55f94aa4bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:07.482 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.479+0000 7f9c6be00640 1 -- 172.21.15.152:0/2598935355 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f94aa206f0 msgr2=0x55f94aa6e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:07.482 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.479+0000 7f9c6be00640 1 --2- 172.21.15.152:0/2598935355 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f94aa206f0 0x55f94aa6e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:07.482 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.479+0000 7f9c6be00640 1 -- 172.21.15.152:0/2598935355 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f94aa45170 msgr2=0x55f94aa4bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:07.482 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.479+0000 7f9c6be00640 1 --2- 172.21.15.152:0/2598935355 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f94aa45170 0x55f94aa4bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:07.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.479+0000 7f9c6be00640 1 -- 172.21.15.152:0/2598935355 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f94a7656b0 con 0x55f94aa6a670 2024-04-20T11:31:07.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.479+0000 7f9c71800640 1 --2- 172.21.15.152:0/2598935355 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f94aa45170 0x55f94aa4bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:31:07.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.479+0000 7f9c6be00640 1 --2- 172.21.15.152:0/2598935355 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f94aa6a670 0x55f94aa4b9a0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f9c54013000 tx=0x7f9c54000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bf309bbf23e6fda server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:07.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.479+0000 7f9c5a000640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-04-20T11:31:07.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.479+0000 7f9c5a000640 1 -- 172.21.15.152:0/2598935355 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c54019040 con 0x55f94aa6a670 2024-04-20T11:31:07.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.479+0000 7f9c5a000640 1 -- 172.21.15.152:0/2598935355 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(60..60 src has 1..60) ==== 5709+0+0 (secure 0 0 0) 0x7f9c54001c60 con 0x55f94aa6a670 2024-04-20T11:31:07.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.479+0000 7f9c765d9740 1 -- 172.21.15.152:0/2598935355 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c54001700 con 0x55f94aa6a670 2024-04-20T11:31:07.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.480+0000 7f9c5a000640 1 -- 172.21.15.152:0/2598935355 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f9c54027050 con 0x55f94aa6a670 2024-04-20T11:31:07.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.480+0000 7f9c765d9740 10 client.6429 fetch_fsmap learned FSMap version 33 2024-04-20T11:31:07.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.480+0000 7f9c765d9740 1 -- 172.21.15.152:0/2598935355 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) -- 0x55f94aa4de30 con 0x55f94aa6a670 2024-04-20T11:31:07.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.480+0000 7f9c5a000640 1 -- 172.21.15.152:0/2598935355 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c54001700 con 0x55f94aa6a670 2024-04-20T11:31:07.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.480+0000 7f9c5a000640 1 -- 172.21.15.152:0/2598935355 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 33) ==== 3495+0+0 (secure 0 0 0) 0x7f9c5402a020 con 0x55f94aa6a670 2024-04-20T11:31:07.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.480+0000 7f9c765d9740 10 client.6429 fetch_fsmap finished waiting for FSMap version 33 2024-04-20T11:31:07.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.480+0000 7f9c765d9740 10 client.6429 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-20T11:31:07.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.480+0000 7f9c765d9740 20 client.6429 populate_metadata read hostname 'smithi152' 2024-04-20T11:31:07.484 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.480+0000 7f9c765d9740 1 --2- 172.21.15.152:0/2598935355 >> [v2:172.21.15.152:6835/488955771,v1:172.21.15.152:6837/488955771] conn(0x55f94aa84b90 0x55f94aa86f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:07.484 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.480+0000 7f9c765d9740 4 client.6429 mds_command: new command op to 6291 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:31:07.484 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.480+0000 7f9c765d9740 1 -- 172.21.15.152:0/2598935355 --> [v2:172.21.15.152:6835/488955771,v1:172.21.15.152:6837/488955771] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f94a9963f0 con 0x55f94aa84b90 2024-04-20T11:31:07.484 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.481+0000 7f9c70e00640 1 --2- 172.21.15.152:0/2598935355 >> [v2:172.21.15.152:6835/488955771,v1:172.21.15.152:6837/488955771] conn(0x55f94aa84b90 0x55f94aa86f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:07.485 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.481+0000 7f9c70e00640 1 --2- 172.21.15.152:0/2598935355 >> [v2:172.21.15.152:6835/488955771,v1:172.21.15.152:6837/488955771] conn(0x55f94aa84b90 0x55f94aa86f80 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e41fa3f4220efcaa server_cookie=21b68934a6fa2c8e in_seq=0 out_seq=0 2024-04-20T11:31:07.485 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.481+0000 7f9c5a000640 10 client.6429 ms_handle_connect on v2:172.21.15.152:6835/488955771 2024-04-20T11:31:07.485 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.482+0000 7f9c5a000640 1 -- 172.21.15.152:0/2598935355 <== mds.0 v2:172.21.15.152:6835/488955771 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f94a9963f0 con 0x55f94aa84b90 2024-04-20T11:31:07.485 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.482+0000 7f9c5a000640 10 client.6429 handle_command_reply: tid=0 2024-04-20T11:31:07.485 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.482+0000 7f9c765d9740 1 client.6429 shutdown 2024-04-20T11:31:07.485 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.482+0000 7f9c765d9740 1 -- 172.21.15.152:0/2598935355 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f94aa6a670 msgr2=0x55f94aa4b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:07.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.482+0000 7f9c765d9740 1 --2- 172.21.15.152:0/2598935355 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f94aa6a670 0x55f94aa4b9a0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f9c54013000 tx=0x7f9c54000bc0 comp rx=0 tx=0).stop 2024-04-20T11:31:07.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.482+0000 7f9c765d9740 1 -- 172.21.15.152:0/2598935355 shutdown_connections 2024-04-20T11:31:07.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.482+0000 7f9c765d9740 1 --2- 172.21.15.152:0/2598935355 >> [v2:172.21.15.152:6835/488955771,v1:172.21.15.152:6837/488955771] conn(0x55f94aa84b90 0x55f94aa86f80 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:07.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.482+0000 7f9c765d9740 1 --2- 172.21.15.152:0/2598935355 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f94aa45170 0x55f94aa4bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:07.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.482+0000 7f9c765d9740 1 --2- 172.21.15.152:0/2598935355 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f94aa6a670 0x55f94aa4b9a0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:07.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.482+0000 7f9c765d9740 1 --2- 172.21.15.152:0/2598935355 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f94aa206f0 0x55f94aa6e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:07.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.482+0000 7f9c5a000640 0 client.6429 ms_handle_reset on v2:172.21.15.152:6835/488955771 2024-04-20T11:31:07.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.482+0000 7f9c765d9740 1 -- 172.21.15.152:0/2598935355 >> 172.21.15.152:0/2598935355 conn(0x55f94aa1cb90 msgr2=0x55f94aa67780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:07.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.483+0000 7f9c765d9740 1 -- 172.21.15.152:0/2598935355 shutdown_connections 2024-04-20T11:31:07.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.483+0000 7f9c765d9740 1 -- 172.21.15.152:0/2598935355 wait complete. 2024-04-20T11:31:07.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.483+0000 7f9c765d9740 20 client.6429 trim_cache size 0 max 16384 2024-04-20T11:31:07.503 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.503+0000 7f9c765d9740 1 Processor -- start 2024-04-20T11:31:07.503 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.503+0000 7f9c765d9740 1 -- start start 2024-04-20T11:31:07.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.503+0000 7f9c765d9740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f94aa206f0 0x55f94aa21810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:07.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.503+0000 7f9c765d9740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f94aa45170 0x55f94aa68de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:07.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.504+0000 7f9c765d9740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f94aa6a670 0x55f94aa1bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:07.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.504+0000 7f9c765d9740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f9c6c1ffd20 con 0x55f94aa45170 2024-04-20T11:31:07.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.504+0000 7f9c765d9740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9c6c155580 con 0x55f94aa6a670 2024-04-20T11:31:07.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.504+0000 7f9c765d9740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f9c6c155bd0 con 0x55f94aa206f0 2024-04-20T11:31:07.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.504+0000 7f9c6be00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f94aa45170 0x55f94aa68de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:07.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.504+0000 7f9c6be00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f94aa45170 0x55f94aa68de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36914/0 (socket says 172.21.15.152:36914) 2024-04-20T11:31:07.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.504+0000 7f9c6be00640 1 -- 172.21.15.152:0/25817258 learned_addr learned my addr 172.21.15.152:0/25817258 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:07.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.504+0000 7f9c6be00640 1 -- 172.21.15.152:0/25817258 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f94aa206f0 msgr2=0x55f94aa21810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:07.508 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.504+0000 7f9c6be00640 1 --2- 172.21.15.152:0/25817258 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f94aa206f0 0x55f94aa21810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:07.508 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.504+0000 7f9c6be00640 1 -- 172.21.15.152:0/25817258 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f94aa6a670 msgr2=0x55f94aa1bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:07.508 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.504+0000 7f9c71800640 1 --2- 172.21.15.152:0/25817258 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f94aa6a670 0x55f94aa1bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:07.508 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.504+0000 7f9c6be00640 1 --2- 172.21.15.152:0/25817258 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f94aa6a670 0x55f94aa1bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:07.508 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.504+0000 7f9c6be00640 1 -- 172.21.15.152:0/25817258 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f94aa44540 con 0x55f94aa45170 2024-04-20T11:31:07.508 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.505+0000 7f9c6be00640 1 --2- 172.21.15.152:0/25817258 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f94aa45170 0x55f94aa68de0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f9c54005bb0 tx=0x7f9c5401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7da73f06e79a9628 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:07.508 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.505+0000 7f9c60c00640 1 -- 172.21.15.152:0/25817258 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c54019070 con 0x55f94aa45170 2024-04-20T11:31:07.508 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.505+0000 7f9c60c00640 1 -- 172.21.15.152:0/25817258 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c54001ac0 con 0x55f94aa45170 2024-04-20T11:31:07.508 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.505+0000 7f9c60c00640 1 -- 172.21.15.152:0/25817258 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c54001c70 con 0x55f94aa45170 2024-04-20T11:31:07.509 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.505+0000 7f9c765d9740 1 -- 172.21.15.152:0/25817258 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f94aa45170 msgr2=0x55f94aa68de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:07.509 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.505+0000 7f9c765d9740 1 --2- 172.21.15.152:0/25817258 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f94aa45170 0x55f94aa68de0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f9c54005bb0 tx=0x7f9c5401a000 comp rx=0 tx=0).stop 2024-04-20T11:31:07.509 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.505+0000 7f9c765d9740 1 -- 172.21.15.152:0/25817258 shutdown_connections 2024-04-20T11:31:07.509 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.505+0000 7f9c765d9740 1 --2- 172.21.15.152:0/25817258 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f94aa6a670 0x55f94aa1bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:07.509 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.505+0000 7f9c765d9740 1 --2- 172.21.15.152:0/25817258 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f94aa45170 0x55f94aa68de0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:07.509 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.505+0000 7f9c765d9740 1 --2- 172.21.15.152:0/25817258 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f94aa206f0 0x55f94aa21810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:07.509 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.505+0000 7f9c765d9740 1 -- 172.21.15.152:0/25817258 >> 172.21.15.152:0/25817258 conn(0x55f94aa1cb90 msgr2=0x55f94aa504a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:07.509 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.506+0000 7f9c765d9740 1 -- 172.21.15.152:0/25817258 shutdown_connections 2024-04-20T11:31:07.509 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.506+0000 7f9c765d9740 1 -- 172.21.15.152:0/25817258 wait complete. 2024-04-20T11:31:07.509 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.506+0000 7f9c765d9740 1 -- start start 2024-04-20T11:31:07.510 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.506+0000 7f9c765d9740 1 Processor -- start 2024-04-20T11:31:07.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.507+0000 7f9c765d9740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f94aa206f0 0x55f94aae2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:07.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.507+0000 7f9c765d9740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f94aa45170 0x55f94aa47c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:07.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.507+0000 7f9c765d9740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f94aa6a670 0x55f94aa4b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:07.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.507+0000 7f9c765d9740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55f94aa50c00 con 0x55f94aa6a670 2024-04-20T11:31:07.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.507+0000 7f9c765d9740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55f94aa6ee50 con 0x55f94aa45170 2024-04-20T11:31:07.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.507+0000 7f9c765d9740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x55f94aa6d890 con 0x55f94aa206f0 2024-04-20T11:31:07.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.507+0000 7f9c765d9740 10 client.0 _ll_register_callbacks cb 0x7f9c54017070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:31:07.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.508+0000 7f9c71800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f94aa6a670 0x55f94aa4b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:07.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.508+0000 7f9c71800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f94aa6a670 0x55f94aa4b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36918/0 (socket says 172.21.15.152:36918) 2024-04-20T11:31:07.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.508+0000 7f9c71800640 1 -- 172.21.15.152:0/4280426863 learned_addr learned my addr 172.21.15.152:0/4280426863 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:07.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.508+0000 7f9c6be00640 1 --2- 172.21.15.152:0/4280426863 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f94aa45170 0x55f94aa47c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:07.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.508+0000 7f9c70e00640 1 --2- 172.21.15.152:0/4280426863 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f94aa206f0 0x55f94aae2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:07.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.508+0000 7f9c71800640 1 -- 172.21.15.152:0/4280426863 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f94aa206f0 msgr2=0x55f94aae2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:07.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.508+0000 7f9c71800640 1 --2- 172.21.15.152:0/4280426863 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f94aa206f0 0x55f94aae2a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:07.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.508+0000 7f9c71800640 1 -- 172.21.15.152:0/4280426863 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f94aa45170 msgr2=0x55f94aa47c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:07.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.508+0000 7f9c71800640 1 --2- 172.21.15.152:0/4280426863 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f94aa45170 0x55f94aa47c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:07.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.508+0000 7f9c71800640 1 -- 172.21.15.152:0/4280426863 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f94aa664e0 con 0x55f94aa6a670 2024-04-20T11:31:07.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.508+0000 7f9c6be00640 1 --2- 172.21.15.152:0/4280426863 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f94aa45170 0x55f94aa47c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:31:07.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.508+0000 7f9c71800640 1 --2- 172.21.15.152:0/4280426863 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f94aa6a670 0x55f94aa4b510 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f9c64045000 tx=0x7f9c6407bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=395803481d1fca94 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:07.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.508+0000 7f9c5a000640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-04-20T11:31:07.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.509+0000 7f9c5a000640 1 -- 172.21.15.152:0/4280426863 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c640ab060 con 0x55f94aa6a670 2024-04-20T11:31:07.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.509+0000 7f9c5a000640 1 -- 172.21.15.152:0/4280426863 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(60..60 src has 1..60) ==== 5709+0+0 (secure 0 0 0) 0x7f9c640ac8b0 con 0x55f94aa6a670 2024-04-20T11:31:07.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.509+0000 7f9c765d9740 1 -- 172.21.15.152:0/4280426863 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c54001ac0 con 0x55f94aa6a670 2024-04-20T11:31:07.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.509+0000 7f9c5a000640 1 -- 172.21.15.152:0/4280426863 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f9c64099050 con 0x55f94aa6a670 2024-04-20T11:31:07.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.509+0000 7f9c765d9740 10 client.6435 fetch_fsmap learned FSMap version 33 2024-04-20T11:31:07.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.509+0000 7f9c765d9740 1 -- 172.21.15.152:0/4280426863 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) -- 0x55f94aa6eba0 con 0x55f94aa6a670 2024-04-20T11:31:07.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.509+0000 7f9c5a000640 1 -- 172.21.15.152:0/4280426863 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c54001ac0 con 0x55f94aa6a670 2024-04-20T11:31:07.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.509+0000 7f9c5a000640 1 -- 172.21.15.152:0/4280426863 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 33) ==== 3495+0+0 (secure 0 0 0) 0x7f9c64094be0 con 0x55f94aa6a670 2024-04-20T11:31:07.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.509+0000 7f9c765d9740 10 client.6435 fetch_fsmap finished waiting for FSMap version 33 2024-04-20T11:31:07.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.509+0000 7f9c765d9740 10 client.6435 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-20T11:31:07.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.509+0000 7f9c765d9740 20 client.6435 populate_metadata read hostname 'smithi152' 2024-04-20T11:31:07.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.509+0000 7f9c765d9740 1 --2- 172.21.15.152:0/4280426863 >> [v2:172.21.15.152:6835/488955771,v1:172.21.15.152:6837/488955771] conn(0x55f94aa6e560 0x55f94aae6540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:07.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.509+0000 7f9c765d9740 4 client.6435 mds_command: new command op to 6291 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-20T11:31:07.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.509+0000 7f9c765d9740 1 -- 172.21.15.152:0/4280426863 --> [v2:172.21.15.152:6835/488955771,v1:172.21.15.152:6837/488955771] -- command(tid 0: {"prefix": "status"}) -- 0x55f94a990260 con 0x55f94aa6e560 2024-04-20T11:31:07.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.510+0000 7f9c70e00640 1 --2- 172.21.15.152:0/4280426863 >> [v2:172.21.15.152:6835/488955771,v1:172.21.15.152:6837/488955771] conn(0x55f94aa6e560 0x55f94aae6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:07.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.510+0000 7f9c70e00640 1 --2- 172.21.15.152:0/4280426863 >> [v2:172.21.15.152:6835/488955771,v1:172.21.15.152:6837/488955771] conn(0x55f94aa6e560 0x55f94aae6540 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1893897999ff447b server_cookie=f6a207b8b4cdb599 in_seq=0 out_seq=0 2024-04-20T11:31:07.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.510+0000 7f9c5a000640 10 client.6435 ms_handle_connect on v2:172.21.15.152:6835/488955771 2024-04-20T11:31:07.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.510+0000 7f9c5a000640 1 -- 172.21.15.152:0/4280426863 <== mds.0 v2:172.21.15.152:6835/488955771 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x55f94a990260 con 0x55f94aa6e560 2024-04-20T11:31:07.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.510+0000 7f9c5a000640 10 client.6435 handle_command_reply: tid=0 2024-04-20T11:31:07.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.510+0000 7f9c765d9740 1 client.6435 shutdown 2024-04-20T11:31:07.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.510+0000 7f9c765d9740 1 -- 172.21.15.152:0/4280426863 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f94aa6a670 msgr2=0x55f94aa4b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:07.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.510+0000 7f9c765d9740 1 --2- 172.21.15.152:0/4280426863 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f94aa6a670 0x55f94aa4b510 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f9c64045000 tx=0x7f9c6407bd60 comp rx=0 tx=0).stop 2024-04-20T11:31:07.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.511+0000 7f9c765d9740 1 -- 172.21.15.152:0/4280426863 shutdown_connections 2024-04-20T11:31:07.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.511+0000 7f9c765d9740 1 --2- 172.21.15.152:0/4280426863 >> [v2:172.21.15.152:6835/488955771,v1:172.21.15.152:6837/488955771] conn(0x55f94aa6e560 0x55f94aae6540 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:07.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.511+0000 7f9c765d9740 1 --2- 172.21.15.152:0/4280426863 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f94aa6a670 0x55f94aa4b510 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:07.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.511+0000 7f9c765d9740 1 --2- 172.21.15.152:0/4280426863 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f94aa45170 0x55f94aa47c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:07.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.511+0000 7f9c5a000640 0 client.6435 ms_handle_reset on v2:172.21.15.152:6835/488955771 2024-04-20T11:31:07.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.511+0000 7f9c765d9740 1 --2- 172.21.15.152:0/4280426863 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55f94aa206f0 0x55f94aae2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:07.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.511+0000 7f9c765d9740 1 -- 172.21.15.152:0/4280426863 >> 172.21.15.152:0/4280426863 conn(0x55f94aa1cb90 msgr2=0x55f94aa504a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:07.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.511+0000 7f9c765d9740 1 -- 172.21.15.152:0/4280426863 shutdown_connections 2024-04-20T11:31:07.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.511+0000 7f9c765d9740 1 -- 172.21.15.152:0/4280426863 wait complete. 2024-04-20T11:31:07.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.511+0000 7f9c765d9740 20 client.6435 trim_cache size 0 max 16384 2024-04-20T11:31:07.521 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-04-20T11:31:07.521 INFO:teuthology.orchestra.run.smithi152.stdout: "cluster_fsid": "c8a72131-0d4f-42c7-892f-3f3af4d2bd3d", 2024-04-20T11:31:07.521 INFO:teuthology.orchestra.run.smithi152.stdout: "whoami": 0, 2024-04-20T11:31:07.521 INFO:teuthology.orchestra.run.smithi152.stdout: "id": 6291, 2024-04-20T11:31:07.521 INFO:teuthology.orchestra.run.smithi152.stdout: "want_state": "up:active", 2024-04-20T11:31:07.521 INFO:teuthology.orchestra.run.smithi152.stdout: "state": "up:active", 2024-04-20T11:31:07.522 INFO:teuthology.orchestra.run.smithi152.stdout: "fs_name": "cephfs", 2024-04-20T11:31:07.522 INFO:teuthology.orchestra.run.smithi152.stdout: "rank_uptime": 6.6644749860000001, 2024-04-20T11:31:07.522 INFO:teuthology.orchestra.run.smithi152.stdout: "mdsmap_epoch": 33, 2024-04-20T11:31:07.522 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_epoch": 60, 2024-04-20T11:31:07.522 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_epoch_barrier": 60, 2024-04-20T11:31:07.522 INFO:teuthology.orchestra.run.smithi152.stdout: "uptime": 13.172111735 2024-04-20T11:31:07.522 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-04-20T11:31:07.523 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.511+0000 7f9c72200640 1 -- 172.21.15.152:0/4122849435 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f9c4806caa0 msgr2=0x7f9c4806ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:07.523 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.511+0000 7f9c72200640 1 --2- 172.21.15.152:0/4122849435 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f9c4806caa0 0x7f9c4806ef60 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c009ac0 tx=0x7f9c5c008040 comp rx=0 tx=0).stop 2024-04-20T11:31:07.524 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.511+0000 7f9c72200640 1 -- 172.21.15.152:0/4122849435 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9c6c150b30 msgr2=0x7f9c6c14e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:07.524 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.511+0000 7f9c72200640 1 --2- 172.21.15.152:0/4122849435 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9c6c150b30 0x7f9c6c14e860 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f9c64009eb0 tx=0x7f9c64004a40 comp rx=0 tx=0).stop 2024-04-20T11:31:07.524 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.512+0000 7f9c72200640 1 -- 172.21.15.152:0/4122849435 shutdown_connections 2024-04-20T11:31:07.524 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.512+0000 7f9c72200640 1 --2- 172.21.15.152:0/4122849435 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f9c4806caa0 0x7f9c4806ef60 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:07.524 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.512+0000 7f9c72200640 1 --2- 172.21.15.152:0/4122849435 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9c6c150b30 0x7f9c6c14e860 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:07.524 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.512+0000 7f9c72200640 1 --2- 172.21.15.152:0/4122849435 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c6c0b4870 0x7f9c6c14c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:07.524 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.512+0000 7f9c72200640 1 --2- 172.21.15.152:0/4122849435 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9c6c073b60 0x7f9c6c1505f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:07.524 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.512+0000 7f9c72200640 1 -- 172.21.15.152:0/4122849435 >> 172.21.15.152:0/4122849435 conn(0x7f9c6c0d0260 msgr2=0x7f9c6c06ecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:07.524 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.512+0000 7f9c72200640 1 -- 172.21.15.152:0/4122849435 shutdown_connections 2024-04-20T11:31:07.524 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.512+0000 7f9c72200640 1 -- 172.21.15.152:0/4122849435 wait complete. 2024-04-20T11:31:07.529 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-20T11:31:07.724 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.723+0000 7fd1a2e00640 1 Processor -- start 2024-04-20T11:31:07.724 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.723+0000 7fd1a2e00640 1 -- start start 2024-04-20T11:31:07.725 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.725+0000 7fd1a2e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fd19c0b3cd0 0x7fd19c0d03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:07.726 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.725+0000 7fd1a2e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fd19c0b43f0 0x7fd19c0b4820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:07.726 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.725+0000 7fd1a2e00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd19c15b890 con 0x7fd19c15b320 2024-04-20T11:31:07.726 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.725+0000 7fd1a2e00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fd19c0b4d60 con 0x7fd19c0b43f0 2024-04-20T11:31:07.726 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.725+0000 7fd1a2e00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fd19c15bb70 con 0x7fd19c0b3cd0 2024-04-20T11:31:07.726 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.725+0000 7fd19be00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fd19c15b320 0x7fd19c0cc9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:50558/0 (socket says 172.21.15.152:50558) 2024-04-20T11:31:07.726 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.725+0000 7fd19be00640 1 -- 172.21.15.152:0/1669131929 learned_addr learned my addr 172.21.15.152:0/1669131929 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:31:07.726 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.725+0000 7fd1a0e00640 1 --2- 172.21.15.152:0/1669131929 >> v2:172.21.15.152:3300/0 conn(0x7fd19c0b43f0 0x7fd19c0b4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:07.727 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.725+0000 7fd193400640 1 --2- 172.21.15.152:0/1669131929 >> v2:172.21.15.175:3300/0 conn(0x7fd19c0b3cd0 0x7fd19c0d03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:07.727 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.726+0000 7fd19b400640 1 -- 172.21.15.152:0/1669131929 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3832052701 0 0) 0x7fd19c15b890 con 0x7fd19c15b320 2024-04-20T11:31:07.727 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.726+0000 7fd19b400640 1 -- 172.21.15.152:0/1669131929 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd180003520 con 0x7fd19c15b320 2024-04-20T11:31:07.727 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.726+0000 7fd1a0e00640 1 -- 172.21.15.152:0/1669131929 >> v2:172.21.15.175:3300/0 conn(0x7fd19c0b3cd0 msgr2=0x7fd19c0d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:07.727 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.726+0000 7fd1a0e00640 1 --2- 172.21.15.152:0/1669131929 >> v2:172.21.15.175:3300/0 conn(0x7fd19c0b3cd0 0x7fd19c0d03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:07.727 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.726+0000 7fd1a0e00640 1 -- 172.21.15.152:0/1669131929 >> v1:172.21.15.152:6789/0 conn(0x7fd19c15b320 legacy=0x7fd19c0cc9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:07.727 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.726+0000 7fd1a0e00640 1 -- 172.21.15.152:0/1669131929 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd19c15be50 con 0x7fd19c0b43f0 2024-04-20T11:31:07.728 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.727+0000 7fd1a0e00640 1 --2- 172.21.15.152:0/1669131929 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd19c0b43f0 0x7fd19c0b4820 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fd19c0d1360 tx=0x7fd19402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=713ec9dadd7326b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:07.728 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.727+0000 7fd19b400640 1 -- 172.21.15.152:0/1669131929 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd194039070 con 0x7fd19c0b43f0 2024-04-20T11:31:07.728 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.727+0000 7fd19b400640 1 -- 172.21.15.152:0/1669131929 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd19402cc80 con 0x7fd19c0b43f0 2024-04-20T11:31:07.728 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.727+0000 7fd19b400640 1 -- 172.21.15.152:0/1669131929 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd194034660 con 0x7fd19c0b43f0 2024-04-20T11:31:07.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.728+0000 7fd1a2e00640 1 -- 172.21.15.152:0/1669131929 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd19c0b43f0 msgr2=0x7fd19c0b4820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:07.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.728+0000 7fd1a2e00640 1 --2- 172.21.15.152:0/1669131929 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd19c0b43f0 0x7fd19c0b4820 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fd19c0d1360 tx=0x7fd19402bed0 comp rx=0 tx=0).stop 2024-04-20T11:31:07.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.728+0000 7fd1a2e00640 1 -- 172.21.15.152:0/1669131929 shutdown_connections 2024-04-20T11:31:07.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.728+0000 7fd1a2e00640 1 --2- 172.21.15.152:0/1669131929 >> v2:172.21.15.175:3300/0 conn(0x7fd19c0b3cd0 0x7fd19c0d03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:07.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.728+0000 7fd1a2e00640 1 --2- 172.21.15.152:0/1669131929 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd19c0b43f0 0x7fd19c0b4820 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:07.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.728+0000 7fd1a2e00640 1 -- 172.21.15.152:0/1669131929 >> 172.21.15.152:0/1669131929 conn(0x7fd19c067cf0 msgr2=0x7fd19c06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:07.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.728+0000 7fd1a2e00640 1 -- 172.21.15.152:0/1669131929 shutdown_connections 2024-04-20T11:31:07.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.728+0000 7fd1a2e00640 1 -- 172.21.15.152:0/1669131929 wait complete. 2024-04-20T11:31:07.730 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.730+0000 7fd1a2e00640 1 Processor -- start 2024-04-20T11:31:07.730 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.730+0000 7fd1a2e00640 1 -- start start 2024-04-20T11:31:07.731 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.731+0000 7fd1a2e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd19c0b3cd0 0x7fd19c152620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:07.732 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.731+0000 7fd1a2e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd19c0b43f0 0x7fd19c153040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:07.732 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.731+0000 7fd1a2e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd19c15b320 0x7fd19c14f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:07.732 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.731+0000 7fd1a2e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fd19c0d0a80 con 0x7fd19c15b320 2024-04-20T11:31:07.732 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.731+0000 7fd1a2e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd19c0d0900 con 0x7fd19c0b43f0 2024-04-20T11:31:07.732 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.731+0000 7fd1a2e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd19c0728e0 con 0x7fd19c0b3cd0 2024-04-20T11:31:07.732 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.731+0000 7fd1a0e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd19c15b320 0x7fd19c14f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:07.732 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.731+0000 7fd1a0e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd19c15b320 0x7fd19c14f8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36944/0 (socket says 172.21.15.152:36944) 2024-04-20T11:31:07.732 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.731+0000 7fd1a0e00640 1 -- 172.21.15.152:0/3717823258 learned_addr learned my addr 172.21.15.152:0/3717823258 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:07.733 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.732+0000 7fd193400640 1 --2- 172.21.15.152:0/3717823258 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd19c0b43f0 0x7fd19c153040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:07.734 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.732+0000 7fd19be00640 1 --2- 172.21.15.152:0/3717823258 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd19c0b3cd0 0x7fd19c152620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:07.734 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.732+0000 7fd1a0e00640 1 -- 172.21.15.152:0/3717823258 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd19c0b3cd0 msgr2=0x7fd19c152620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:07.734 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.732+0000 7fd1a0e00640 1 --2- 172.21.15.152:0/3717823258 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd19c0b3cd0 0x7fd19c152620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:07.734 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.732+0000 7fd1a0e00640 1 -- 172.21.15.152:0/3717823258 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd19c0b43f0 msgr2=0x7fd19c153040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:07.734 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.732+0000 7fd1a0e00640 1 --2- 172.21.15.152:0/3717823258 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd19c0b43f0 0x7fd19c153040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:07.734 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.732+0000 7fd1a0e00640 1 -- 172.21.15.152:0/3717823258 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd19c153580 con 0x7fd19c15b320 2024-04-20T11:31:07.734 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.732+0000 7fd19be00640 1 --2- 172.21.15.152:0/3717823258 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd19c0b3cd0 0x7fd19c152620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:31:07.734 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.732+0000 7fd1a0e00640 1 --2- 172.21.15.152:0/3717823258 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd19c15b320 0x7fd19c14f8e0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fd194009e80 tx=0x7fd194004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:07.734 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.732+0000 7fd198c00640 1 -- 172.21.15.152:0/3717823258 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd194039050 con 0x7fd19c15b320 2024-04-20T11:31:07.734 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.732+0000 7fd198c00640 1 -- 172.21.15.152:0/3717823258 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd194045070 con 0x7fd19c15b320 2024-04-20T11:31:07.734 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.732+0000 7fd198c00640 1 -- 172.21.15.152:0/3717823258 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd194009450 con 0x7fd19c15b320 2024-04-20T11:31:07.734 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.732+0000 7fd1a2e00640 1 -- 172.21.15.152:0/3717823258 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd19c153c20 con 0x7fd19c15b320 2024-04-20T11:31:07.735 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.733+0000 7fd1a2e00640 1 -- 172.21.15.152:0/3717823258 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd19c1541e0 con 0x7fd19c15b320 2024-04-20T11:31:07.735 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.734+0000 7fd1a2e00640 1 -- 172.21.15.152:0/3717823258 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd19c0689f0 con 0x7fd19c15b320 2024-04-20T11:31:07.735 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.735+0000 7fd198c00640 1 -- 172.21.15.152:0/3717823258 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 9) ==== 91193+0+0 (secure 0 0 0) 0x7fd19403e850 con 0x7fd19c15b320 2024-04-20T11:31:07.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.735+0000 7fd198c00640 1 --2- 172.21.15.152:0/3717823258 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fd17806caa0 0x7fd17806ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:07.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.736+0000 7fd198c00640 1 -- 172.21.15.152:0/3717823258 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5709+0+0 (secure 0 0 0) 0x7fd1940aedd0 con 0x7fd19c15b320 2024-04-20T11:31:07.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.736+0000 7fd19be00640 1 --2- 172.21.15.152:0/3717823258 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fd17806caa0 0x7fd17806ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:07.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.737+0000 7fd19be00640 1 --2- 172.21.15.152:0/3717823258 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fd17806caa0 0x7fd17806ef60 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fd18c005880 tx=0x7fd18c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:07.740 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.741+0000 7fd198c00640 1 -- 172.21.15.152:0/3717823258 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fd19407d090 con 0x7fd19c15b320 2024-04-20T11:31:07.986 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.985+0000 7fd1a2e00640 1 -- 172.21.15.152:0/3717823258 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fd19c0b3500 con 0x7fd19c15b320 2024-04-20T11:31:07.986 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.986+0000 7fd198c00640 1 -- 172.21.15.152:0/3717823258 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v33) ==== 53+0+73 (secure 0 0 0) 0x7fd19407c990 con 0x7fd19c15b320 2024-04-20T11:31:07.986 INFO:teuthology.orchestra.run.smithi152.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-20T11:31:07.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.988+0000 7fd1a2e00640 1 -- 172.21.15.152:0/3717823258 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fd17806caa0 msgr2=0x7fd17806ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:07.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.988+0000 7fd1a2e00640 1 --2- 172.21.15.152:0/3717823258 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fd17806caa0 0x7fd17806ef60 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fd18c005880 tx=0x7fd18c009040 comp rx=0 tx=0).stop 2024-04-20T11:31:07.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.988+0000 7fd1a2e00640 1 -- 172.21.15.152:0/3717823258 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd19c15b320 msgr2=0x7fd19c14f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:07.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.988+0000 7fd1a2e00640 1 --2- 172.21.15.152:0/3717823258 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd19c15b320 0x7fd19c14f8e0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fd194009e80 tx=0x7fd194004a40 comp rx=0 tx=0).stop 2024-04-20T11:31:07.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.989+0000 7fd1a2e00640 1 -- 172.21.15.152:0/3717823258 shutdown_connections 2024-04-20T11:31:07.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.989+0000 7fd1a2e00640 1 --2- 172.21.15.152:0/3717823258 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fd17806caa0 0x7fd17806ef60 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:07.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.989+0000 7fd1a2e00640 1 --2- 172.21.15.152:0/3717823258 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd19c15b320 0x7fd19c14f8e0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:07.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.989+0000 7fd1a2e00640 1 --2- 172.21.15.152:0/3717823258 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd19c0b43f0 0x7fd19c153040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:07.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.989+0000 7fd1a2e00640 1 --2- 172.21.15.152:0/3717823258 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd19c0b3cd0 0x7fd19c152620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:07.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.989+0000 7fd1a2e00640 1 -- 172.21.15.152:0/3717823258 >> 172.21.15.152:0/3717823258 conn(0x7fd19c067cf0 msgr2=0x7fd19c06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:07.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.989+0000 7fd1a2e00640 1 -- 172.21.15.152:0/3717823258 shutdown_connections 2024-04-20T11:31:07.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:07.989+0000 7fd1a2e00640 1 -- 172.21.15.152:0/3717823258 wait complete. 2024-04-20T11:31:08.009 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-20T11:31:08.009 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-20T11:31:08.009 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi152.front.sepia.ceph.com 2024-04-20T11:31:08.009 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-20T11:31:08.009 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-20T11:31:08.009 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-20T11:31:08.009 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-20T11:31:08.009 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-20T11:31:08.009 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-20T11:31:08.009 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:31:08.010 DEBUG:teuthology.orchestra.run.smithi152:> ip addr 2024-04-20T11:31:08.059 INFO:teuthology.orchestra.run.smithi152.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-20T11:31:08.059 INFO:teuthology.orchestra.run.smithi152.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-20T11:31:08.059 INFO:teuthology.orchestra.run.smithi152.stdout: inet 127.0.0.1/8 scope host lo 2024-04-20T11:31:08.059 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:31:08.060 INFO:teuthology.orchestra.run.smithi152.stdout: inet6 ::1/128 scope host 2024-04-20T11:31:08.060 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:31:08.060 INFO:teuthology.orchestra.run.smithi152.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T11:31:08.060 INFO:teuthology.orchestra.run.smithi152.stdout: link/ether 0c:c4:7a:da:6c:44 brd ff:ff:ff:ff:ff:ff 2024-04-20T11:31:08.060 INFO:teuthology.orchestra.run.smithi152.stdout: altname enp7s0f0 2024-04-20T11:31:08.060 INFO:teuthology.orchestra.run.smithi152.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T11:31:08.060 INFO:teuthology.orchestra.run.smithi152.stdout: link/ether 0c:c4:7a:da:6c:45 brd ff:ff:ff:ff:ff:ff 2024-04-20T11:31:08.060 INFO:teuthology.orchestra.run.smithi152.stdout: altname enp7s0f1 2024-04-20T11:31:08.060 INFO:teuthology.orchestra.run.smithi152.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T11:31:08.060 INFO:teuthology.orchestra.run.smithi152.stdout: link/ether 0c:c4:7a:88:6d:e2 brd ff:ff:ff:ff:ff:ff 2024-04-20T11:31:08.060 INFO:teuthology.orchestra.run.smithi152.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-20T11:31:08.060 INFO:teuthology.orchestra.run.smithi152.stdout: link/ether 0c:c4:7a:88:6d:e3 brd ff:ff:ff:ff:ff:ff 2024-04-20T11:31:08.060 INFO:teuthology.orchestra.run.smithi152.stdout: inet 172.21.15.152/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-20T11:31:08.061 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft 42538sec preferred_lft 42538sec 2024-04-20T11:31:08.061 INFO:teuthology.orchestra.run.smithi152.stdout: inet6 fe80::ec4:7aff:fe88:6de3/64 scope link 2024-04-20T11:31:08.061 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:31:08.061 INFO:teuthology.orchestra.run.smithi152.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-20T11:31:08.061 INFO:teuthology.orchestra.run.smithi152.stdout: link/ether 06:3e:9a:d3:49:1e brd ff:ff:ff:ff:ff:ff 2024-04-20T11:31:08.061 INFO:teuthology.orchestra.run.smithi152.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-20T11:31:08.061 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:31:08.061 INFO:teuthology.orchestra.run.smithi152.stdout: inet6 fe80::a019:d8ff:fe70:1185/64 scope link 2024-04-20T11:31:08.061 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:31:08.061 INFO:teuthology.orchestra.run.smithi152.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-20T11:31:08.061 INFO:teuthology.orchestra.run.smithi152.stdout: link/ether 16:e5:08:78:20:47 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T11:31:08.061 INFO:teuthology.orchestra.run.smithi152.stdout: inet6 fe80::14e5:8ff:fe78:2047/64 scope link 2024-04-20T11:31:08.061 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:31:08.061 INFO:teuthology.orchestra.run.smithi152.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-20T11:31:08.061 INFO:teuthology.orchestra.run.smithi152.stdout: link/ether 06:3e:9a:d3:49:1e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T11:31:08.062 INFO:teuthology.orchestra.run.smithi152.stdout: inet6 fe80::43e:9aff:fed3:491e/64 scope link 2024-04-20T11:31:08.062 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:31:08.063 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:31:08.063 DEBUG:teuthology.orchestra.run.smithi152:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-20T11:31:08.141 INFO:teuthology.orchestra.run.smithi152.stdout:1 2024-04-20T11:31:08.145 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:31:08.145 DEBUG:teuthology.orchestra.run.smithi152:> route 2024-04-20T11:31:08.197 INFO:teuthology.orchestra.run.smithi152.stdout:Kernel IP routing table 2024-04-20T11:31:08.197 INFO:teuthology.orchestra.run.smithi152.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-20T11:31:08.198 INFO:teuthology.orchestra.run.smithi152.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-20T11:31:08.198 INFO:teuthology.orchestra.run.smithi152.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-20T11:31:08.198 INFO:teuthology.orchestra.run.smithi152.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-20T11:31:08.198 DEBUG:teuthology.orchestra.run.smithi152:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T11:31:08.198 DEBUG:teuthology.orchestra.run.smithi152:> set -e 2024-04-20T11:31:08.198 DEBUG:teuthology.orchestra.run.smithi152:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-20T11:31:08.198 DEBUG:teuthology.orchestra.run.smithi152:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-20T11:31:08.198 DEBUG:teuthology.orchestra.run.smithi152:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-20T11:31:08.198 DEBUG:teuthology.orchestra.run.smithi152:> ') 2024-04-20T11:31:08.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T11:31:08.399 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T11:31:08.408 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:31:08.408 DEBUG:teuthology.orchestra.run.smithi152:> ip netns list 2024-04-20T11:31:08.456 INFO:teuthology.orchestra.run.smithi152.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-20T11:31:08.456 INFO:teuthology.orchestra.run.smithi152.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-20T11:31:08.456 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-20T11:31:08.457 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:31:08.457 DEBUG:teuthology.orchestra.run.smithi152:> sudo ip link set brx.0 up 2024-04-20T11:31:08.539 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-20T11:31:08.539 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T11:31:08.540 DEBUG:teuthology.orchestra.run.smithi152:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-20T11:31:08.591 INFO:teuthology.orchestra.run.smithi152.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-20T11:31:08.592 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T11:31:08.592 DEBUG:teuthology.orchestra.run.smithi152:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-20T11:31:08.644 DEBUG:teuthology.orchestra.run.smithi152:> sudo modprobe fuse 2024-04-20T11:31:08.725 DEBUG:teuthology.orchestra.run.smithi152:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T11:31:08.781 INFO:teuthology.orchestra.run.smithi152.stdout:/proc 2024-04-20T11:31:08.781 INFO:teuthology.orchestra.run.smithi152.stdout:/sys 2024-04-20T11:31:08.782 INFO:teuthology.orchestra.run.smithi152.stdout:/dev 2024-04-20T11:31:08.782 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/security 2024-04-20T11:31:08.782 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/shm 2024-04-20T11:31:08.782 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/pts 2024-04-20T11:31:08.782 INFO:teuthology.orchestra.run.smithi152.stdout:/run 2024-04-20T11:31:08.782 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/cgroup 2024-04-20T11:31:08.782 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/pstore 2024-04-20T11:31:08.782 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/bpf 2024-04-20T11:31:08.782 INFO:teuthology.orchestra.run.smithi152.stdout:/ 2024-04-20T11:31:08.782 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/selinux 2024-04-20T11:31:08.782 INFO:teuthology.orchestra.run.smithi152.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T11:31:08.782 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/mqueue 2024-04-20T11:31:08.782 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/hugepages 2024-04-20T11:31:08.783 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/debug 2024-04-20T11:31:08.783 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/tracing 2024-04-20T11:31:08.783 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/fuse/connections 2024-04-20T11:31:08.783 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/config 2024-04-20T11:31:08.783 INFO:teuthology.orchestra.run.smithi152.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T11:31:08.783 INFO:teuthology.orchestra.run.smithi152.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T11:31:08.783 INFO:teuthology.orchestra.run.smithi152.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T11:31:08.783 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T11:31:08.783 INFO:teuthology.orchestra.run.smithi152.stdout:/run/user/1000 2024-04-20T11:31:08.783 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph 2024-04-20T11:31:08.783 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph/osd/ceph-0 2024-04-20T11:31:08.783 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph/osd/ceph-1 2024-04-20T11:31:08.783 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph/osd/ceph-2 2024-04-20T11:31:08.783 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph/osd/ceph-3 2024-04-20T11:31:08.783 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns 2024-04-20T11:31:08.784 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T11:31:08.784 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T11:31:08.784 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T11:31:08.784 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T11:31:08.785 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:31:08.785 DEBUG:teuthology.orchestra.run.smithi152:> ls /sys/fs/fuse/connections 2024-04-20T11:31:08.839 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-20T11:31:08.839 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-20T11:31:08.846 DEBUG:teuthology.orchestra.run.smithi152:> sudo modprobe fuse 2024-04-20T11:31:08.919 DEBUG:teuthology.orchestra.run.smithi152:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T11:31:08.963 INFO:teuthology.orchestra.run.smithi152.stdout:/proc 2024-04-20T11:31:08.963 INFO:teuthology.orchestra.run.smithi152.stdout:/sys 2024-04-20T11:31:08.963 INFO:teuthology.orchestra.run.smithi152.stdout:/dev 2024-04-20T11:31:08.963 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/security 2024-04-20T11:31:08.963 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/shm 2024-04-20T11:31:08.963 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/pts 2024-04-20T11:31:08.963 INFO:teuthology.orchestra.run.smithi152.stdout:/run 2024-04-20T11:31:08.963 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/cgroup 2024-04-20T11:31:08.964 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/pstore 2024-04-20T11:31:08.964 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/bpf 2024-04-20T11:31:08.964 INFO:teuthology.orchestra.run.smithi152.stdout:/ 2024-04-20T11:31:08.964 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/selinux 2024-04-20T11:31:08.964 INFO:teuthology.orchestra.run.smithi152.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T11:31:08.964 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/mqueue 2024-04-20T11:31:08.964 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/hugepages 2024-04-20T11:31:08.964 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/debug 2024-04-20T11:31:08.964 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/tracing 2024-04-20T11:31:08.964 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/fuse/connections 2024-04-20T11:31:08.964 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/config 2024-04-20T11:31:08.964 INFO:teuthology.orchestra.run.smithi152.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T11:31:08.964 INFO:teuthology.orchestra.run.smithi152.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T11:31:08.964 INFO:teuthology.orchestra.run.smithi152.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T11:31:08.965 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T11:31:08.965 INFO:teuthology.orchestra.run.smithi152.stdout:/run/user/1000 2024-04-20T11:31:08.965 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph 2024-04-20T11:31:08.965 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph/osd/ceph-0 2024-04-20T11:31:08.965 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph/osd/ceph-1 2024-04-20T11:31:08.965 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph/osd/ceph-2 2024-04-20T11:31:08.965 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph/osd/ceph-3 2024-04-20T11:31:08.965 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns 2024-04-20T11:31:08.965 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T11:31:08.965 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T11:31:08.965 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T11:31:08.965 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T11:31:08.966 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:31:08.967 DEBUG:teuthology.orchestra.run.smithi152:> ls /sys/fs/fuse/connections 2024-04-20T11:31:09.020 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi152.stderr:2024-04-20T11:31:09.019+0000 7f573064c480 -1 init, newargv = 0x55a9a286c4b0 newargc=15 2024-04-20T11:31:09.020 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi152.stderr:ceph-fuse[44009]: starting ceph client 2024-04-20T11:31:09.029 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi152.stderr:ceph-fuse[44009]: starting fuse 2024-04-20T11:31:09.048 INFO:teuthology.orchestra.run.smithi152.stdout:42 2024-04-20T11:31:09.049 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-04-20T11:31:09.049 DEBUG:teuthology.orchestra.run.smithi152:> sudo stdin-killer -- python3 -c ' 2024-04-20T11:31:09.049 DEBUG:teuthology.orchestra.run.smithi152:> import glob 2024-04-20T11:31:09.049 DEBUG:teuthology.orchestra.run.smithi152:> import re 2024-04-20T11:31:09.049 DEBUG:teuthology.orchestra.run.smithi152:> import os 2024-04-20T11:31:09.049 DEBUG:teuthology.orchestra.run.smithi152:> import subprocess 2024-04-20T11:31:09.049 DEBUG:teuthology.orchestra.run.smithi152:> 2024-04-20T11:31:09.050 DEBUG:teuthology.orchestra.run.smithi152:> def _find_admin_socket(client_name): 2024-04-20T11:31:09.050 DEBUG:teuthology.orchestra.run.smithi152:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-20T11:31:09.050 DEBUG:teuthology.orchestra.run.smithi152:> files = glob.glob(asok_path) 2024-04-20T11:31:09.050 DEBUG:teuthology.orchestra.run.smithi152:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-20T11:31:09.050 DEBUG:teuthology.orchestra.run.smithi152:> 2024-04-20T11:31:09.050 DEBUG:teuthology.orchestra.run.smithi152:> # Given a non-glob path, it better be there 2024-04-20T11:31:09.050 DEBUG:teuthology.orchestra.run.smithi152:> if "*" not in asok_path: 2024-04-20T11:31:09.050 DEBUG:teuthology.orchestra.run.smithi152:> assert(len(files) == 1) 2024-04-20T11:31:09.050 DEBUG:teuthology.orchestra.run.smithi152:> return files[0] 2024-04-20T11:31:09.050 DEBUG:teuthology.orchestra.run.smithi152:> 2024-04-20T11:31:09.050 DEBUG:teuthology.orchestra.run.smithi152:> for f in files: 2024-04-20T11:31:09.050 DEBUG:teuthology.orchestra.run.smithi152:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-20T11:31:09.050 DEBUG:teuthology.orchestra.run.smithi152:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-20T11:31:09.050 DEBUG:teuthology.orchestra.run.smithi152:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-20T11:31:09.051 DEBUG:teuthology.orchestra.run.smithi152:> contents = proc_f.read() 2024-04-20T11:31:09.051 DEBUG:teuthology.orchestra.run.smithi152:> if mountpoint in contents: 2024-04-20T11:31:09.051 DEBUG:teuthology.orchestra.run.smithi152:> return f 2024-04-20T11:31:09.051 DEBUG:teuthology.orchestra.run.smithi152:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-20T11:31:09.051 DEBUG:teuthology.orchestra.run.smithi152:> 2024-04-20T11:31:09.051 DEBUG:teuthology.orchestra.run.smithi152:> print(_find_admin_socket("client.0")) 2024-04-20T11:31:09.051 DEBUG:teuthology.orchestra.run.smithi152:> ' 2024-04-20T11:31:09.236 INFO:teuthology.orchestra.run.smithi152.stdout:/var/run/ceph/ceph-client.0.44009.asok 2024-04-20T11:31:09.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T11:31:09.252 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.44009.asok 2024-04-20T11:31:09.252 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:31:09.253 DEBUG:teuthology.orchestra.run.smithi152:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.44009.asok status 2024-04-20T11:31:09.479 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-04-20T11:31:09.479 INFO:teuthology.orchestra.run.smithi152.stdout: "metadata": { 2024-04-20T11:31:09.480 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_sha1": "a8c84f5997950fd7d008204839253291d52e6eda", 2024-04-20T11:31:09.480 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", 2024-04-20T11:31:09.480 INFO:teuthology.orchestra.run.smithi152.stdout: "entity_id": "0", 2024-04-20T11:31:09.480 INFO:teuthology.orchestra.run.smithi152.stdout: "hostname": "smithi152", 2024-04-20T11:31:09.480 INFO:teuthology.orchestra.run.smithi152.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-20T11:31:09.480 INFO:teuthology.orchestra.run.smithi152.stdout: "pid": "44009", 2024-04-20T11:31:09.480 INFO:teuthology.orchestra.run.smithi152.stdout: "root": "/" 2024-04-20T11:31:09.480 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:31:09.480 INFO:teuthology.orchestra.run.smithi152.stdout: "dentry_count": 0, 2024-04-20T11:31:09.480 INFO:teuthology.orchestra.run.smithi152.stdout: "dentry_pinned_count": 0, 2024-04-20T11:31:09.480 INFO:teuthology.orchestra.run.smithi152.stdout: "id": 6450, 2024-04-20T11:31:09.480 INFO:teuthology.orchestra.run.smithi152.stdout: "inst": { 2024-04-20T11:31:09.480 INFO:teuthology.orchestra.run.smithi152.stdout: "name": { 2024-04-20T11:31:09.480 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "client", 2024-04-20T11:31:09.481 INFO:teuthology.orchestra.run.smithi152.stdout: "num": 6450 2024-04-20T11:31:09.481 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:31:09.481 INFO:teuthology.orchestra.run.smithi152.stdout: "addr": { 2024-04-20T11:31:09.481 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "v1", 2024-04-20T11:31:09.481 INFO:teuthology.orchestra.run.smithi152.stdout: "addr": "192.168.0.1:0", 2024-04-20T11:31:09.481 INFO:teuthology.orchestra.run.smithi152.stdout: "nonce": 892980810 2024-04-20T11:31:09.481 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:31:09.481 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:31:09.481 INFO:teuthology.orchestra.run.smithi152.stdout: "addr": { 2024-04-20T11:31:09.481 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "v1", 2024-04-20T11:31:09.481 INFO:teuthology.orchestra.run.smithi152.stdout: "addr": "192.168.0.1:0", 2024-04-20T11:31:09.481 INFO:teuthology.orchestra.run.smithi152.stdout: "nonce": 892980810 2024-04-20T11:31:09.481 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:31:09.482 INFO:teuthology.orchestra.run.smithi152.stdout: "inst_str": "client.6450 192.168.0.1:0/892980810", 2024-04-20T11:31:09.482 INFO:teuthology.orchestra.run.smithi152.stdout: "addr_str": "192.168.0.1:0/892980810", 2024-04-20T11:31:09.482 INFO:teuthology.orchestra.run.smithi152.stdout: "inode_count": 1, 2024-04-20T11:31:09.482 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_epoch": 33, 2024-04-20T11:31:09.482 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_epoch": 60, 2024-04-20T11:31:09.482 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_epoch_barrier": 0, 2024-04-20T11:31:09.482 INFO:teuthology.orchestra.run.smithi152.stdout: "blocklisted": false, 2024-04-20T11:31:09.482 INFO:teuthology.orchestra.run.smithi152.stdout: "fs_name": "cephfs" 2024-04-20T11:31:09.483 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-04-20T11:31:09.494 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:31:09.495 DEBUG:teuthology.orchestra.run.smithi152:> stat --file-system '--printf=%T 2024-04-20T11:31:09.495 DEBUG:teuthology.orchestra.run.smithi152:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-20T11:31:09.543 INFO:teuthology.orchestra.run.smithi152.stdout:fuseblk 2024-04-20T11:31:09.543 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-20T11:31:09.543 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:31:09.543 DEBUG:teuthology.orchestra.run.smithi152:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-20T11:31:09.616 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-20T11:31:09.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:09.814+0000 7fd321a00640 1 Processor -- start 2024-04-20T11:31:09.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:09.814+0000 7fd321a00640 1 -- start start 2024-04-20T11:31:09.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:09.816+0000 7fd321a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fd31c15d1e0 0x7fd31c160b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:09.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:09.816+0000 7fd321a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd31c15d5c0 0x7fd31c0cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:09.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:09.816+0000 7fd321a00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd31c0ce8c0 con 0x7fd31c071cd0 2024-04-20T11:31:09.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:09.816+0000 7fd321a00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fd31c1600b0 con 0x7fd31c15d1e0 2024-04-20T11:31:09.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:09.816+0000 7fd321a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd31c160390 con 0x7fd31c15d5c0 2024-04-20T11:31:09.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:09.816+0000 7fd31be00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fd31c15d1e0 0x7fd31c160b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:09.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:09.816+0000 7fd31be00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fd31c15d1e0 0x7fd31c160b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36956/0 (socket says 172.21.15.152:36956) 2024-04-20T11:31:09.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:09.816+0000 7fd31be00640 1 -- 172.21.15.152:0/1527532907 learned_addr learned my addr 172.21.15.152:0/1527532907 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:09.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:09.816+0000 7fd31b400640 1 --2- 172.21.15.152:0/1527532907 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd31c15d5c0 0x7fd31c0cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:09.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:09.817+0000 7fd31be00640 1 -- 172.21.15.152:0/1527532907 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd31c15d5c0 msgr2=0x7fd31c0cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:09.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:09.817+0000 7fd31be00640 1 --2- 172.21.15.152:0/1527532907 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd31c15d5c0 0x7fd31c0cdfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:09.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:09.817+0000 7fd31be00640 1 -- 172.21.15.152:0/1527532907 >> v1:172.21.15.175:6789/0 conn(0x7fd31c071cd0 legacy=0x7fd31c0ccd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:09.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:09.817+0000 7fd31be00640 1 -- 172.21.15.152:0/1527532907 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd31c160670 con 0x7fd31c15d1e0 2024-04-20T11:31:09.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:09.818+0000 7fd31be00640 1 --2- 172.21.15.152:0/1527532907 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd31c15d1e0 0x7fd31c160b90 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fd30c009870 tx=0x7fd30c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9df7cf1b789fe16 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:09.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:09.818+0000 7fd31aa00640 1 -- 172.21.15.152:0/1527532907 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd30c035020 con 0x7fd31c15d1e0 2024-04-20T11:31:09.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:09.818+0000 7fd31aa00640 1 -- 172.21.15.152:0/1527532907 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd30c031640 con 0x7fd31c15d1e0 2024-04-20T11:31:09.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:09.818+0000 7fd31aa00640 1 -- 172.21.15.152:0/1527532907 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd30c02d040 con 0x7fd31c15d1e0 2024-04-20T11:31:09.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:09.819+0000 7fd321a00640 1 -- 172.21.15.152:0/1527532907 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd31c15d1e0 msgr2=0x7fd31c160b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:09.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:09.819+0000 7fd321a00640 1 --2- 172.21.15.152:0/1527532907 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd31c15d1e0 0x7fd31c160b90 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fd30c009870 tx=0x7fd30c02b8f0 comp rx=0 tx=0).stop 2024-04-20T11:31:09.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:09.819+0000 7fd321a00640 1 -- 172.21.15.152:0/1527532907 shutdown_connections 2024-04-20T11:31:09.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:09.819+0000 7fd321a00640 1 --2- 172.21.15.152:0/1527532907 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd31c15d5c0 0x7fd31c0cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:09.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:09.819+0000 7fd321a00640 1 --2- 172.21.15.152:0/1527532907 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd31c15d1e0 0x7fd31c160b90 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:09.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:09.819+0000 7fd321a00640 1 -- 172.21.15.152:0/1527532907 >> 172.21.15.152:0/1527532907 conn(0x7fd31c067ae0 msgr2=0x7fd31c06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:09.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:09.819+0000 7fd321a00640 1 -- 172.21.15.152:0/1527532907 shutdown_connections 2024-04-20T11:31:09.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:09.819+0000 7fd321a00640 1 -- 172.21.15.152:0/1527532907 wait complete. 2024-04-20T11:31:09.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:09.821+0000 7fd321a00640 1 Processor -- start 2024-04-20T11:31:09.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:09.821+0000 7fd321a00640 1 -- start start 2024-04-20T11:31:09.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:09.822+0000 7fd321a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd31c071cd0 0x7fd31c146390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:09.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:09.822+0000 7fd321a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd31c15d1e0 0x7fd31c149580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:09.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:09.822+0000 7fd321a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd31c15d5c0 0x7fd31c1e6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:09.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:09.822+0000 7fd321a00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fd31c0ce6a0 con 0x7fd31c15d1e0 2024-04-20T11:31:09.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:09.822+0000 7fd321a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd31c0ce520 con 0x7fd31c15d5c0 2024-04-20T11:31:09.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:09.822+0000 7fd321a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd31c072160 con 0x7fd31c071cd0 2024-04-20T11:31:09.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:09.822+0000 7fd31b400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd31c15d1e0 0x7fd31c149580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:09.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:09.822+0000 7fd31b400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd31c15d1e0 0x7fd31c149580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36960/0 (socket says 172.21.15.152:36960) 2024-04-20T11:31:09.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:09.822+0000 7fd31b400640 1 -- 172.21.15.152:0/1243874326 learned_addr learned my addr 172.21.15.152:0/1243874326 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:09.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:09.822+0000 7fd31be00640 1 --2- 172.21.15.152:0/1243874326 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd31c071cd0 0x7fd31c146390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:09.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:09.822+0000 7fd31b400640 1 -- 172.21.15.152:0/1243874326 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd31c071cd0 msgr2=0x7fd31c146390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:09.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:09.822+0000 7fd31b400640 1 --2- 172.21.15.152:0/1243874326 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd31c071cd0 0x7fd31c146390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:09.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:09.822+0000 7fd31b400640 1 -- 172.21.15.152:0/1243874326 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd31c15d5c0 msgr2=0x7fd31c1e6cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:09.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:09.823+0000 7fd31b400640 1 --2- 172.21.15.152:0/1243874326 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd31c15d5c0 0x7fd31c1e6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:09.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:09.823+0000 7fd31b400640 1 -- 172.21.15.152:0/1243874326 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd31c149ac0 con 0x7fd31c15d1e0 2024-04-20T11:31:09.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:09.823+0000 7fd31b400640 1 --2- 172.21.15.152:0/1243874326 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd31c15d1e0 0x7fd31c149580 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fd30800d620 tx=0x7fd30800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:09.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:09.823+0000 7fd313e00640 1 -- 172.21.15.152:0/1243874326 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd30800b620 con 0x7fd31c15d1e0 2024-04-20T11:31:09.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:09.823+0000 7fd313e00640 1 -- 172.21.15.152:0/1243874326 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd308019070 con 0x7fd31c15d1e0 2024-04-20T11:31:09.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:09.823+0000 7fd313e00640 1 -- 172.21.15.152:0/1243874326 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd30800ba00 con 0x7fd31c15d1e0 2024-04-20T11:31:09.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:09.823+0000 7fd321a00640 1 -- 172.21.15.152:0/1243874326 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd31c146e50 con 0x7fd31c15d1e0 2024-04-20T11:31:09.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:09.823+0000 7fd321a00640 1 -- 172.21.15.152:0/1243874326 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd31c147410 con 0x7fd31c15d1e0 2024-04-20T11:31:09.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:09.824+0000 7fd321a00640 1 -- 172.21.15.152:0/1243874326 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd31c075e60 con 0x7fd31c15d1e0 2024-04-20T11:31:09.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:09.825+0000 7fd313e00640 1 -- 172.21.15.152:0/1243874326 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 9) ==== 91193+0+0 (secure 0 0 0) 0x7fd30800bba0 con 0x7fd31c15d1e0 2024-04-20T11:31:09.826 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:09.826+0000 7fd313e00640 1 --2- 172.21.15.152:0/1243874326 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fd2fc06caa0 0x7fd2fc06ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:09.826 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:09.826+0000 7fd313e00640 1 -- 172.21.15.152:0/1243874326 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5709+0+0 (secure 0 0 0) 0x7fd30808c270 con 0x7fd31c15d1e0 2024-04-20T11:31:09.826 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:09.827+0000 7fd31be00640 1 --2- 172.21.15.152:0/1243874326 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fd2fc06caa0 0x7fd2fc06ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:09.826 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:09.827+0000 7fd31be00640 1 --2- 172.21.15.152:0/1243874326 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fd2fc06caa0 0x7fd2fc06ef60 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fd30c036000 tx=0x7fd30c03c040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:09.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:09.832+0000 7fd313e00640 1 -- 172.21.15.152:0/1243874326 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fd31c149ac0 con 0x7fd31c15d1e0 2024-04-20T11:31:10.078 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:10.078+0000 7fd321a00640 1 -- 172.21.15.152:0/1243874326 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fd31c06f6b0 con 0x7fd31c15d1e0 2024-04-20T11:31:10.078 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:10.079+0000 7fd313e00640 1 -- 172.21.15.152:0/1243874326 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v33) ==== 53+0+73 (secure 0 0 0) 0x7fd30805a410 con 0x7fd31c15d1e0 2024-04-20T11:31:10.078 INFO:teuthology.orchestra.run.smithi152.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-20T11:31:10.081 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:10.081+0000 7fd321a00640 1 -- 172.21.15.152:0/1243874326 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fd2fc06caa0 msgr2=0x7fd2fc06ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:10.082 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:10.081+0000 7fd321a00640 1 --2- 172.21.15.152:0/1243874326 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fd2fc06caa0 0x7fd2fc06ef60 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fd30c036000 tx=0x7fd30c03c040 comp rx=0 tx=0).stop 2024-04-20T11:31:10.082 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:10.081+0000 7fd321a00640 1 -- 172.21.15.152:0/1243874326 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd31c15d1e0 msgr2=0x7fd31c149580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:10.082 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:10.081+0000 7fd321a00640 1 --2- 172.21.15.152:0/1243874326 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd31c15d1e0 0x7fd31c149580 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fd30800d620 tx=0x7fd30800dae0 comp rx=0 tx=0).stop 2024-04-20T11:31:10.082 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:10.082+0000 7fd321a00640 1 -- 172.21.15.152:0/1243874326 shutdown_connections 2024-04-20T11:31:10.082 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:10.082+0000 7fd321a00640 1 --2- 172.21.15.152:0/1243874326 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fd2fc06caa0 0x7fd2fc06ef60 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:10.082 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:10.082+0000 7fd321a00640 1 --2- 172.21.15.152:0/1243874326 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd31c15d5c0 0x7fd31c1e6cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:10.082 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:10.082+0000 7fd321a00640 1 --2- 172.21.15.152:0/1243874326 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd31c15d1e0 0x7fd31c149580 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:10.082 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:10.082+0000 7fd321a00640 1 --2- 172.21.15.152:0/1243874326 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd31c071cd0 0x7fd31c146390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:10.082 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:10.082+0000 7fd321a00640 1 -- 172.21.15.152:0/1243874326 >> 172.21.15.152:0/1243874326 conn(0x7fd31c067ae0 msgr2=0x7fd31c15b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:10.082 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:10.082+0000 7fd321a00640 1 -- 172.21.15.152:0/1243874326 shutdown_connections 2024-04-20T11:31:10.082 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:10.082+0000 7fd321a00640 1 -- 172.21.15.152:0/1243874326 wait complete. 2024-04-20T11:31:10.100 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-20T11:31:10.100 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-20T11:31:10.100 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi175.front.sepia.ceph.com 2024-04-20T11:31:10.100 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-20T11:31:10.100 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-20T11:31:10.100 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-20T11:31:10.101 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-20T11:31:10.101 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-20T11:31:10.101 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-20T11:31:10.101 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:31:10.101 DEBUG:teuthology.orchestra.run.smithi175:> ip addr 2024-04-20T11:31:10.159 INFO:teuthology.orchestra.run.smithi175.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-20T11:31:10.159 INFO:teuthology.orchestra.run.smithi175.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-20T11:31:10.159 INFO:teuthology.orchestra.run.smithi175.stdout: inet 127.0.0.1/8 scope host lo 2024-04-20T11:31:10.159 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:31:10.159 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 ::1/128 scope host 2024-04-20T11:31:10.159 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:31:10.159 INFO:teuthology.orchestra.run.smithi175.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T11:31:10.159 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:d9:39:ee brd ff:ff:ff:ff:ff:ff 2024-04-20T11:31:10.159 INFO:teuthology.orchestra.run.smithi175.stdout: altname enp7s0f0 2024-04-20T11:31:10.159 INFO:teuthology.orchestra.run.smithi175.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T11:31:10.160 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:d9:39:ef brd ff:ff:ff:ff:ff:ff 2024-04-20T11:31:10.160 INFO:teuthology.orchestra.run.smithi175.stdout: altname enp7s0f1 2024-04-20T11:31:10.160 INFO:teuthology.orchestra.run.smithi175.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T11:31:10.160 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:88:71:f0 brd ff:ff:ff:ff:ff:ff 2024-04-20T11:31:10.160 INFO:teuthology.orchestra.run.smithi175.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-20T11:31:10.160 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:88:71:f1 brd ff:ff:ff:ff:ff:ff 2024-04-20T11:31:10.160 INFO:teuthology.orchestra.run.smithi175.stdout: inet 172.21.15.175/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-20T11:31:10.160 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft 42511sec preferred_lft 42511sec 2024-04-20T11:31:10.160 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::ec4:7aff:fe88:71f1/64 scope link 2024-04-20T11:31:10.160 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:31:10.160 INFO:teuthology.orchestra.run.smithi175.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-20T11:31:10.160 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 02:d6:c6:31:26:70 brd ff:ff:ff:ff:ff:ff 2024-04-20T11:31:10.160 INFO:teuthology.orchestra.run.smithi175.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-20T11:31:10.160 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:31:10.161 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::5091:d5ff:fee7:54b3/64 scope link 2024-04-20T11:31:10.161 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:31:10.161 INFO:teuthology.orchestra.run.smithi175.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-20T11:31:10.161 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 02:d6:c6:31:26:70 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-20T11:31:10.161 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::d6:c6ff:fe31:2670/64 scope link 2024-04-20T11:31:10.161 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:31:10.161 INFO:teuthology.orchestra.run.smithi175.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-20T11:31:10.161 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 9e:14:f3:e2:18:1e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-20T11:31:10.161 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::9c14:f3ff:fee2:181e/64 scope link 2024-04-20T11:31:10.161 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:31:10.162 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:31:10.162 DEBUG:teuthology.orchestra.run.smithi175:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-20T11:31:10.242 INFO:teuthology.orchestra.run.smithi175.stdout:1 2024-04-20T11:31:10.246 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:31:10.246 DEBUG:teuthology.orchestra.run.smithi175:> route 2024-04-20T11:31:10.297 INFO:teuthology.orchestra.run.smithi175.stdout:Kernel IP routing table 2024-04-20T11:31:10.298 INFO:teuthology.orchestra.run.smithi175.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-20T11:31:10.298 INFO:teuthology.orchestra.run.smithi175.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-20T11:31:10.298 INFO:teuthology.orchestra.run.smithi175.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-20T11:31:10.298 INFO:teuthology.orchestra.run.smithi175.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-20T11:31:10.298 DEBUG:teuthology.orchestra.run.smithi175:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T11:31:10.298 DEBUG:teuthology.orchestra.run.smithi175:> set -e 2024-04-20T11:31:10.298 DEBUG:teuthology.orchestra.run.smithi175:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-20T11:31:10.298 DEBUG:teuthology.orchestra.run.smithi175:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-20T11:31:10.298 DEBUG:teuthology.orchestra.run.smithi175:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-20T11:31:10.299 DEBUG:teuthology.orchestra.run.smithi175:> ') 2024-04-20T11:31:10.400 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:31:10 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T11:31:10.496 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:31:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T11:31:10.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:31:10.505 DEBUG:teuthology.orchestra.run.smithi175:> ip netns list 2024-04-20T11:31:10.553 INFO:teuthology.orchestra.run.smithi175.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-20T11:31:10.553 INFO:teuthology.orchestra.run.smithi175.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-20T11:31:10.554 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-20T11:31:10.554 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:31:10.554 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip link set brx.0 up 2024-04-20T11:31:10.638 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-04-20T11:31:10.638 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T11:31:10.638 DEBUG:teuthology.orchestra.run.smithi175:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-20T11:31:10.692 INFO:teuthology.orchestra.run.smithi175.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-20T11:31:10.692 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T11:31:10.692 DEBUG:teuthology.orchestra.run.smithi175:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-20T11:31:10.741 DEBUG:teuthology.orchestra.run.smithi175:> sudo modprobe fuse 2024-04-20T11:31:10.822 DEBUG:teuthology.orchestra.run.smithi175:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T11:31:10.879 INFO:teuthology.orchestra.run.smithi175.stdout:/proc 2024-04-20T11:31:10.879 INFO:teuthology.orchestra.run.smithi175.stdout:/sys 2024-04-20T11:31:10.879 INFO:teuthology.orchestra.run.smithi175.stdout:/dev 2024-04-20T11:31:10.880 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/security 2024-04-20T11:31:10.880 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/shm 2024-04-20T11:31:10.880 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/pts 2024-04-20T11:31:10.880 INFO:teuthology.orchestra.run.smithi175.stdout:/run 2024-04-20T11:31:10.880 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/cgroup 2024-04-20T11:31:10.880 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/pstore 2024-04-20T11:31:10.880 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/bpf 2024-04-20T11:31:10.880 INFO:teuthology.orchestra.run.smithi175.stdout:/ 2024-04-20T11:31:10.880 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/selinux 2024-04-20T11:31:10.880 INFO:teuthology.orchestra.run.smithi175.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T11:31:10.880 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/mqueue 2024-04-20T11:31:10.880 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/debug 2024-04-20T11:31:10.881 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/hugepages 2024-04-20T11:31:10.881 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/tracing 2024-04-20T11:31:10.881 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/fuse/connections 2024-04-20T11:31:10.881 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/config 2024-04-20T11:31:10.881 INFO:teuthology.orchestra.run.smithi175.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T11:31:10.881 INFO:teuthology.orchestra.run.smithi175.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T11:31:10.881 INFO:teuthology.orchestra.run.smithi175.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T11:31:10.881 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T11:31:10.881 INFO:teuthology.orchestra.run.smithi175.stdout:/run/user/1000 2024-04-20T11:31:10.881 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph 2024-04-20T11:31:10.881 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-4 2024-04-20T11:31:10.881 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-5 2024-04-20T11:31:10.881 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-6 2024-04-20T11:31:10.881 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-7 2024-04-20T11:31:10.882 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns 2024-04-20T11:31:10.882 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-20T11:31:10.882 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-20T11:31:10.882 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-20T11:31:10.882 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-20T11:31:10.883 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:31:10.883 DEBUG:teuthology.orchestra.run.smithi175:> ls /sys/fs/fuse/connections 2024-04-20T11:31:10.936 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-20T11:31:10.937 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-04-20T11:31:10.941 DEBUG:teuthology.orchestra.run.smithi175:> sudo modprobe fuse 2024-04-20T11:31:11.006 DEBUG:teuthology.orchestra.run.smithi175:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T11:31:11.047 INFO:teuthology.orchestra.run.smithi175.stdout:/proc 2024-04-20T11:31:11.047 INFO:teuthology.orchestra.run.smithi175.stdout:/sys 2024-04-20T11:31:11.047 INFO:teuthology.orchestra.run.smithi175.stdout:/dev 2024-04-20T11:31:11.047 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/security 2024-04-20T11:31:11.048 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/shm 2024-04-20T11:31:11.048 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/pts 2024-04-20T11:31:11.048 INFO:teuthology.orchestra.run.smithi175.stdout:/run 2024-04-20T11:31:11.048 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/cgroup 2024-04-20T11:31:11.048 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/pstore 2024-04-20T11:31:11.048 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/bpf 2024-04-20T11:31:11.048 INFO:teuthology.orchestra.run.smithi175.stdout:/ 2024-04-20T11:31:11.048 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/selinux 2024-04-20T11:31:11.048 INFO:teuthology.orchestra.run.smithi175.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T11:31:11.048 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/mqueue 2024-04-20T11:31:11.048 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/debug 2024-04-20T11:31:11.048 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/hugepages 2024-04-20T11:31:11.048 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/tracing 2024-04-20T11:31:11.048 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/fuse/connections 2024-04-20T11:31:11.049 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/config 2024-04-20T11:31:11.049 INFO:teuthology.orchestra.run.smithi175.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T11:31:11.049 INFO:teuthology.orchestra.run.smithi175.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T11:31:11.049 INFO:teuthology.orchestra.run.smithi175.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T11:31:11.049 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T11:31:11.049 INFO:teuthology.orchestra.run.smithi175.stdout:/run/user/1000 2024-04-20T11:31:11.049 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph 2024-04-20T11:31:11.049 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-4 2024-04-20T11:31:11.049 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-5 2024-04-20T11:31:11.049 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-6 2024-04-20T11:31:11.049 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-7 2024-04-20T11:31:11.049 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns 2024-04-20T11:31:11.049 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-20T11:31:11.049 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-20T11:31:11.050 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-20T11:31:11.050 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-20T11:31:11.051 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:31:11.051 DEBUG:teuthology.orchestra.run.smithi175:> ls /sys/fs/fuse/connections 2024-04-20T11:31:11.109 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi175.stderr:2024-04-20T11:31:11.109+0000 7f558fad1480 -1 init, newargv = 0x55a5b2eaede0 newargc=15 2024-04-20T11:31:11.109 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi175.stderr:ceph-fuse[31493]: starting ceph client 2024-04-20T11:31:11.120 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi175.stderr:ceph-fuse[31493]: starting fuse 2024-04-20T11:31:11.134 INFO:teuthology.orchestra.run.smithi175.stdout:42 2024-04-20T11:31:11.134 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-04-20T11:31:11.135 DEBUG:teuthology.orchestra.run.smithi175:> sudo stdin-killer -- python3 -c ' 2024-04-20T11:31:11.135 DEBUG:teuthology.orchestra.run.smithi175:> import glob 2024-04-20T11:31:11.135 DEBUG:teuthology.orchestra.run.smithi175:> import re 2024-04-20T11:31:11.135 DEBUG:teuthology.orchestra.run.smithi175:> import os 2024-04-20T11:31:11.135 DEBUG:teuthology.orchestra.run.smithi175:> import subprocess 2024-04-20T11:31:11.135 DEBUG:teuthology.orchestra.run.smithi175:> 2024-04-20T11:31:11.135 DEBUG:teuthology.orchestra.run.smithi175:> def _find_admin_socket(client_name): 2024-04-20T11:31:11.135 DEBUG:teuthology.orchestra.run.smithi175:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-04-20T11:31:11.135 DEBUG:teuthology.orchestra.run.smithi175:> files = glob.glob(asok_path) 2024-04-20T11:31:11.135 DEBUG:teuthology.orchestra.run.smithi175:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-04-20T11:31:11.135 DEBUG:teuthology.orchestra.run.smithi175:> 2024-04-20T11:31:11.136 DEBUG:teuthology.orchestra.run.smithi175:> # Given a non-glob path, it better be there 2024-04-20T11:31:11.136 DEBUG:teuthology.orchestra.run.smithi175:> if "*" not in asok_path: 2024-04-20T11:31:11.136 DEBUG:teuthology.orchestra.run.smithi175:> assert(len(files) == 1) 2024-04-20T11:31:11.136 DEBUG:teuthology.orchestra.run.smithi175:> return files[0] 2024-04-20T11:31:11.136 DEBUG:teuthology.orchestra.run.smithi175:> 2024-04-20T11:31:11.136 DEBUG:teuthology.orchestra.run.smithi175:> for f in files: 2024-04-20T11:31:11.136 DEBUG:teuthology.orchestra.run.smithi175:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-20T11:31:11.136 DEBUG:teuthology.orchestra.run.smithi175:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-20T11:31:11.136 DEBUG:teuthology.orchestra.run.smithi175:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-20T11:31:11.136 DEBUG:teuthology.orchestra.run.smithi175:> contents = proc_f.read() 2024-04-20T11:31:11.136 DEBUG:teuthology.orchestra.run.smithi175:> if mountpoint in contents: 2024-04-20T11:31:11.136 DEBUG:teuthology.orchestra.run.smithi175:> return f 2024-04-20T11:31:11.136 DEBUG:teuthology.orchestra.run.smithi175:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-20T11:31:11.136 DEBUG:teuthology.orchestra.run.smithi175:> 2024-04-20T11:31:11.137 DEBUG:teuthology.orchestra.run.smithi175:> print(_find_admin_socket("client.1")) 2024-04-20T11:31:11.137 DEBUG:teuthology.orchestra.run.smithi175:> ' 2024-04-20T11:31:11.316 INFO:teuthology.orchestra.run.smithi175.stdout:/var/run/ceph/ceph-client.1.31493.asok 2024-04-20T11:31:11.321 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:31:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T11:31:11.334 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.31493.asok 2024-04-20T11:31:11.334 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:31:11.334 DEBUG:teuthology.orchestra.run.smithi175:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.31493.asok status 2024-04-20T11:31:11.560 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-04-20T11:31:11.560 INFO:teuthology.orchestra.run.smithi175.stdout: "metadata": { 2024-04-20T11:31:11.560 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_sha1": "a8c84f5997950fd7d008204839253291d52e6eda", 2024-04-20T11:31:11.560 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", 2024-04-20T11:31:11.561 INFO:teuthology.orchestra.run.smithi175.stdout: "entity_id": "1", 2024-04-20T11:31:11.561 INFO:teuthology.orchestra.run.smithi175.stdout: "hostname": "smithi175", 2024-04-20T11:31:11.561 INFO:teuthology.orchestra.run.smithi175.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-04-20T11:31:11.561 INFO:teuthology.orchestra.run.smithi175.stdout: "pid": "31493", 2024-04-20T11:31:11.561 INFO:teuthology.orchestra.run.smithi175.stdout: "root": "/" 2024-04-20T11:31:11.561 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-20T11:31:11.561 INFO:teuthology.orchestra.run.smithi175.stdout: "dentry_count": 0, 2024-04-20T11:31:11.561 INFO:teuthology.orchestra.run.smithi175.stdout: "dentry_pinned_count": 0, 2024-04-20T11:31:11.561 INFO:teuthology.orchestra.run.smithi175.stdout: "id": 5428, 2024-04-20T11:31:11.561 INFO:teuthology.orchestra.run.smithi175.stdout: "inst": { 2024-04-20T11:31:11.561 INFO:teuthology.orchestra.run.smithi175.stdout: "name": { 2024-04-20T11:31:11.561 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "client", 2024-04-20T11:31:11.561 INFO:teuthology.orchestra.run.smithi175.stdout: "num": 5428 2024-04-20T11:31:11.562 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-20T11:31:11.562 INFO:teuthology.orchestra.run.smithi175.stdout: "addr": { 2024-04-20T11:31:11.562 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "v1", 2024-04-20T11:31:11.562 INFO:teuthology.orchestra.run.smithi175.stdout: "addr": "192.168.0.1:0", 2024-04-20T11:31:11.562 INFO:teuthology.orchestra.run.smithi175.stdout: "nonce": 3732046460 2024-04-20T11:31:11.562 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-04-20T11:31:11.562 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-20T11:31:11.562 INFO:teuthology.orchestra.run.smithi175.stdout: "addr": { 2024-04-20T11:31:11.562 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "v1", 2024-04-20T11:31:11.562 INFO:teuthology.orchestra.run.smithi175.stdout: "addr": "192.168.0.1:0", 2024-04-20T11:31:11.562 INFO:teuthology.orchestra.run.smithi175.stdout: "nonce": 3732046460 2024-04-20T11:31:11.562 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-20T11:31:11.562 INFO:teuthology.orchestra.run.smithi175.stdout: "inst_str": "client.5428 192.168.0.1:0/3732046460", 2024-04-20T11:31:11.562 INFO:teuthology.orchestra.run.smithi175.stdout: "addr_str": "192.168.0.1:0/3732046460", 2024-04-20T11:31:11.563 INFO:teuthology.orchestra.run.smithi175.stdout: "inode_count": 1, 2024-04-20T11:31:11.563 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_epoch": 33, 2024-04-20T11:31:11.563 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_epoch": 60, 2024-04-20T11:31:11.563 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_epoch_barrier": 0, 2024-04-20T11:31:11.563 INFO:teuthology.orchestra.run.smithi175.stdout: "blocklisted": false, 2024-04-20T11:31:11.563 INFO:teuthology.orchestra.run.smithi175.stdout: "fs_name": "cephfs" 2024-04-20T11:31:11.563 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-04-20T11:31:11.574 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:31:11.575 DEBUG:teuthology.orchestra.run.smithi175:> stat --file-system '--printf=%T 2024-04-20T11:31:11.575 DEBUG:teuthology.orchestra.run.smithi175:> ' -- /home/ubuntu/cephtest/mnt.1 2024-04-20T11:31:11.625 INFO:teuthology.orchestra.run.smithi175.stdout:fuseblk 2024-04-20T11:31:11.625 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-04-20T11:31:11.625 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:31:11.625 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-20T11:31:11.708 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-04-20T11:31:11.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:11.920+0000 7fdb3e200640 1 Processor -- start 2024-04-20T11:31:11.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:11.921+0000 7fdb3e200640 1 -- start start 2024-04-20T11:31:11.922 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:11.922+0000 7fdb3e200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdb380d0cd0 0x7fdb3815efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:11.922 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:11.922+0000 7fdb3e200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fdb3815f520 0x7fdb380b3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:11.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:11.922+0000 7fdb3e200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb3815fcf0 con 0x7fdb38158fc0 2024-04-20T11:31:11.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:11.922+0000 7fdb3e200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fdb381593a0 con 0x7fdb3815f520 2024-04-20T11:31:11.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:11.922+0000 7fdb3e200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fdb38162db0 con 0x7fdb380d0cd0 2024-04-20T11:31:11.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:11.922+0000 7fdb3d800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fdb3815f520 0x7fdb380b3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:11.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:11.923+0000 7fdb3d800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fdb3815f520 0x7fdb380b3050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36972/0 (socket says 172.21.15.152:36972) 2024-04-20T11:31:11.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:11.923+0000 7fdb3d800640 1 -- 172.21.15.152:0/1222735657 learned_addr learned my addr 172.21.15.152:0/1222735657 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:11.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:11.923+0000 7fdb3d800640 1 -- 172.21.15.152:0/1222735657 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdb380d0cd0 msgr2=0x7fdb3815efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:11.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:11.923+0000 7fdb3d800640 1 --2- 172.21.15.152:0/1222735657 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdb380d0cd0 0x7fdb3815efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:11.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:11.923+0000 7fdb3d800640 1 -- 172.21.15.152:0/1222735657 >> v1:172.21.15.175:6789/0 conn(0x7fdb38158fc0 legacy=0x7fdb380d05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:11.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:11.923+0000 7fdb3d800640 1 -- 172.21.15.152:0/1222735657 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb38162480 con 0x7fdb3815f520 2024-04-20T11:31:11.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:11.924+0000 7fdb3d800640 1 --2- 172.21.15.152:0/1222735657 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdb3815f520 0x7fdb380b3050 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fdb30009e50 tx=0x7fdb3002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d906307d669dabe1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:11.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:11.924+0000 7fdb37e00640 1 -- 172.21.15.152:0/1222735657 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb3002ca80 con 0x7fdb3815f520 2024-04-20T11:31:11.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:11.924+0000 7fdb37e00640 1 -- 172.21.15.152:0/1222735657 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb3002cc20 con 0x7fdb3815f520 2024-04-20T11:31:11.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:11.924+0000 7fdb37e00640 1 -- 172.21.15.152:0/1222735657 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb300343f0 con 0x7fdb3815f520 2024-04-20T11:31:11.925 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:11.925+0000 7fdb3e200640 1 -- 172.21.15.152:0/1222735657 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdb3815f520 msgr2=0x7fdb380b3050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:11.925 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:11.925+0000 7fdb3e200640 1 --2- 172.21.15.152:0/1222735657 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdb3815f520 0x7fdb380b3050 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fdb30009e50 tx=0x7fdb3002bee0 comp rx=0 tx=0).stop 2024-04-20T11:31:11.925 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:11.925+0000 7fdb3e200640 1 -- 172.21.15.152:0/1222735657 shutdown_connections 2024-04-20T11:31:11.925 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:11.925+0000 7fdb3e200640 1 --2- 172.21.15.152:0/1222735657 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdb3815f520 0x7fdb380b3050 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:11.925 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:11.925+0000 7fdb3e200640 1 --2- 172.21.15.152:0/1222735657 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdb380d0cd0 0x7fdb3815efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:11.925 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:11.925+0000 7fdb3e200640 1 -- 172.21.15.152:0/1222735657 >> 172.21.15.152:0/1222735657 conn(0x7fdb3806bda0 msgr2=0x7fdb3806c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:11.925 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:11.925+0000 7fdb3e200640 1 -- 172.21.15.152:0/1222735657 shutdown_connections 2024-04-20T11:31:11.925 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:11.925+0000 7fdb3e200640 1 -- 172.21.15.152:0/1222735657 wait complete. 2024-04-20T11:31:11.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:11.927+0000 7fdb3e200640 1 Processor -- start 2024-04-20T11:31:11.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:11.927+0000 7fdb3e200640 1 -- start start 2024-04-20T11:31:11.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:11.928+0000 7fdb3e200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdb380d0cd0 0x7fdb3814cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:11.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:11.928+0000 7fdb3e200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb38158fc0 0x7fdb381575e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:11.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:11.928+0000 7fdb3e200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdb3815f520 0x7fdb380b2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:11.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:11.928+0000 7fdb3e200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fdb3815f9a0 con 0x7fdb3815f520 2024-04-20T11:31:11.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:11.928+0000 7fdb3e200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdb38162a40 con 0x7fdb38158fc0 2024-04-20T11:31:11.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:11.928+0000 7fdb3e200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fdb38163240 con 0x7fdb380d0cd0 2024-04-20T11:31:11.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:11.928+0000 7fdb3d800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdb3815f520 0x7fdb380b2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:11.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:11.928+0000 7fdb3d800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdb3815f520 0x7fdb380b2fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36988/0 (socket says 172.21.15.152:36988) 2024-04-20T11:31:11.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:11.928+0000 7fdb3d800640 1 -- 172.21.15.152:0/1718039903 learned_addr learned my addr 172.21.15.152:0/1718039903 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:11.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:11.928+0000 7fdb3d800640 1 -- 172.21.15.152:0/1718039903 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdb380d0cd0 msgr2=0x7fdb3814cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:11.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:11.929+0000 7fdb3d800640 1 --2- 172.21.15.152:0/1718039903 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdb380d0cd0 0x7fdb3814cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:11.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:11.929+0000 7fdb3d800640 1 -- 172.21.15.152:0/1718039903 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb38158fc0 msgr2=0x7fdb381575e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:11.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:11.929+0000 7fdb3d800640 1 --2- 172.21.15.152:0/1718039903 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb38158fc0 0x7fdb381575e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:11.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:11.929+0000 7fdb3d800640 1 -- 172.21.15.152:0/1718039903 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb38155310 con 0x7fdb3815f520 2024-04-20T11:31:11.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:11.929+0000 7fdb3d800640 1 --2- 172.21.15.152:0/1718039903 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdb3815f520 0x7fdb380b2fe0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fdb30009e50 tx=0x7fdb30004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:11.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:11.931+0000 7fdb35600640 1 -- 172.21.15.152:0/1718039903 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb30043020 con 0x7fdb3815f520 2024-04-20T11:31:11.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:11.931+0000 7fdb35600640 1 -- 172.21.15.152:0/1718039903 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb3003e450 con 0x7fdb3815f520 2024-04-20T11:31:11.931 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:11.931+0000 7fdb35600640 1 -- 172.21.15.152:0/1718039903 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb3003e820 con 0x7fdb3815f520 2024-04-20T11:31:11.931 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:11.931+0000 7fdb3e200640 1 -- 172.21.15.152:0/1718039903 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb381555f0 con 0x7fdb3815f520 2024-04-20T11:31:11.931 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:11.931+0000 7fdb3e200640 1 -- 172.21.15.152:0/1718039903 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb38155bb0 con 0x7fdb3815f520 2024-04-20T11:31:11.932 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:11.932+0000 7fdb3e200640 1 -- 172.21.15.152:0/1718039903 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb38155d80 con 0x7fdb3815f520 2024-04-20T11:31:11.932 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:11.933+0000 7fdb35600640 1 -- 172.21.15.152:0/1718039903 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 9) ==== 91193+0+0 (secure 0 0 0) 0x7fdb30046020 con 0x7fdb3815f520 2024-04-20T11:31:11.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:11.934+0000 7fdb35600640 1 --2- 172.21.15.152:0/1718039903 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fdb1406cad0 0x7fdb1406ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:11.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:11.934+0000 7fdb35600640 1 -- 172.21.15.152:0/1718039903 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5709+0+0 (secure 0 0 0) 0x7fdb300ae580 con 0x7fdb3815f520 2024-04-20T11:31:11.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:11.934+0000 7fdb3ce00640 1 --2- 172.21.15.152:0/1718039903 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fdb1406cad0 0x7fdb1406ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:11.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:11.935+0000 7fdb3ce00640 1 --2- 172.21.15.152:0/1718039903 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fdb1406cad0 0x7fdb1406ef90 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fdb28001560 tx=0x7fdb28007990 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:11.938 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:11.939+0000 7fdb35600640 1 -- 172.21.15.152:0/1718039903 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fdb3004d070 con 0x7fdb3815f520 2024-04-20T11:31:12.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:12.166+0000 7fdb3e200640 1 -- 172.21.15.152:0/1718039903 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7fdb380cecd0 con 0x7fdb3815f520 2024-04-20T11:31:12.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:12.942+0000 7fdb35600640 1 -- 172.21.15.152:0/1718039903 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v34) ==== 136+0+0 (secure 0 0 0) 0x7fdb3007c690 con 0x7fdb3815f520 2024-04-20T11:31:12.945 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:12.945+0000 7fdb3e200640 1 -- 172.21.15.152:0/1718039903 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fdb1406cad0 msgr2=0x7fdb1406ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:12.945 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:12.945+0000 7fdb3e200640 1 --2- 172.21.15.152:0/1718039903 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fdb1406cad0 0x7fdb1406ef90 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fdb28001560 tx=0x7fdb28007990 comp rx=0 tx=0).stop 2024-04-20T11:31:12.945 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:12.945+0000 7fdb3e200640 1 -- 172.21.15.152:0/1718039903 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdb3815f520 msgr2=0x7fdb380b2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:12.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:12.945+0000 7fdb3e200640 1 --2- 172.21.15.152:0/1718039903 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdb3815f520 0x7fdb380b2fe0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fdb30009e50 tx=0x7fdb30004a40 comp rx=0 tx=0).stop 2024-04-20T11:31:12.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:12.946+0000 7fdb3e200640 1 -- 172.21.15.152:0/1718039903 shutdown_connections 2024-04-20T11:31:12.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:12.946+0000 7fdb3e200640 1 --2- 172.21.15.152:0/1718039903 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fdb1406cad0 0x7fdb1406ef90 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:12.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:12.946+0000 7fdb3e200640 1 --2- 172.21.15.152:0/1718039903 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdb3815f520 0x7fdb380b2fe0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:12.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:12.946+0000 7fdb3e200640 1 --2- 172.21.15.152:0/1718039903 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb38158fc0 0x7fdb381575e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:12.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:12.946+0000 7fdb3e200640 1 --2- 172.21.15.152:0/1718039903 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdb380d0cd0 0x7fdb3814cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:12.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:12.946+0000 7fdb3e200640 1 -- 172.21.15.152:0/1718039903 >> 172.21.15.152:0/1718039903 conn(0x7fdb3806bda0 msgr2=0x7fdb380ccbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:12.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:12.946+0000 7fdb3e200640 1 -- 172.21.15.152:0/1718039903 shutdown_connections 2024-04-20T11:31:12.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:12.946+0000 7fdb3e200640 1 -- 172.21.15.152:0/1718039903 wait complete. 2024-04-20T11:31:12.967 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-04-20T11:31:13.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.171+0000 7fa9a7000640 1 Processor -- start 2024-04-20T11:31:13.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.172+0000 7fa9a7000640 1 -- start start 2024-04-20T11:31:13.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.173+0000 7fa9a7000640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fa9a00d4ec0 0x7fa9a00d0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:13.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.173+0000 7fa9a7000640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9a00d26a0 con 0x7fa9a0161d30 2024-04-20T11:31:13.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.173+0000 7fa9a7000640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9a00d53c0 con 0x7fa9a00d2b80 2024-04-20T11:31:13.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.173+0000 7fa9a7000640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fa9a00d0540 con 0x7fa9a00d4ec0 2024-04-20T11:31:13.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.174+0000 7fa9a5c00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fa9a0161d30 0x7fa9a00d15e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:50560/0 (socket says 172.21.15.152:50560) 2024-04-20T11:31:13.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.174+0000 7fa9a5c00640 1 -- 172.21.15.152:0/1804534053 learned_addr learned my addr 172.21.15.152:0/1804534053 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:31:13.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.174+0000 7fa9a5200640 1 --2- 172.21.15.152:0/1804534053 >> v2:172.21.15.175:3300/0 conn(0x7fa9a00d4ec0 0x7fa9a00d0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:13.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.174+0000 7fa99fe00640 1 -- 172.21.15.152:0/1804534053 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3535424449 0 0) 0x7fa9a00d26a0 con 0x7fa9a0161d30 2024-04-20T11:31:13.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.174+0000 7fa99fe00640 1 -- 172.21.15.152:0/1804534053 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa988003520 con 0x7fa9a0161d30 2024-04-20T11:31:13.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.174+0000 7fa99fe00640 1 -- 172.21.15.152:0/1804534053 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 19962370 0 0) 0x7fa988003520 con 0x7fa9a0161d30 2024-04-20T11:31:13.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.174+0000 7fa99fe00640 1 -- 172.21.15.152:0/1804534053 >> v2:172.21.15.175:3300/0 conn(0x7fa9a00d4ec0 msgr2=0x7fa9a00d0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:13.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.174+0000 7fa99fe00640 1 --2- 172.21.15.152:0/1804534053 >> v2:172.21.15.175:3300/0 conn(0x7fa9a00d4ec0 0x7fa9a00d0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.174+0000 7fa99fe00640 1 -- 172.21.15.152:0/1804534053 >> v1:172.21.15.175:6789/0 conn(0x7fa9a00d2b80 legacy=0x7fa9a006c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:13.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.174+0000 7fa99fe00640 1 -- 172.21.15.152:0/1804534053 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9a00cf520 con 0x7fa9a0161d30 2024-04-20T11:31:13.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.174+0000 7fa99fe00640 1 -- 172.21.15.152:0/1804534053 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7fa994001670 con 0x7fa9a0161d30 2024-04-20T11:31:13.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.175+0000 7fa99fe00640 1 -- 172.21.15.152:0/1804534053 >> v1:172.21.15.152:6789/0 conn(0x7fa9a0161d30 legacy=0x7fa9a00d15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:13.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.175+0000 7fa99fe00640 1 --2- 172.21.15.152:0/1804534053 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa9880039b0 0x7fa988003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:13.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.175+0000 7fa99fe00640 1 --2- 172.21.15.152:0/1804534053 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa988004800 0x7fa988004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:13.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.175+0000 7fa99fe00640 1 --2- 172.21.15.152:0/1804534053 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa9880051c0 0x7fa98800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:13.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.175+0000 7fa99fe00640 1 -- 172.21.15.152:0/1804534053 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fa9a00d0540 con 0x7fa9880039b0 2024-04-20T11:31:13.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.175+0000 7fa99fe00640 1 -- 172.21.15.152:0/1804534053 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa98800c060 con 0x7fa988004800 2024-04-20T11:31:13.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.175+0000 7fa99fe00640 1 -- 172.21.15.152:0/1804534053 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa98800c260 con 0x7fa9880051c0 2024-04-20T11:31:13.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.175+0000 7fa9a5c00640 1 --2- 172.21.15.152:0/1804534053 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa9880039b0 0x7fa988003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:13.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.175+0000 7fa9a5c00640 1 -- 172.21.15.152:0/1804534053 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa9880051c0 msgr2=0x7fa98800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:13.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.175+0000 7fa9a5c00640 1 --2- 172.21.15.152:0/1804534053 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa9880051c0 0x7fa98800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.175+0000 7fa9a5c00640 1 -- 172.21.15.152:0/1804534053 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa988004800 msgr2=0x7fa988004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:13.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.175+0000 7fa9a5c00640 1 --2- 172.21.15.152:0/1804534053 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa988004800 0x7fa988004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.175+0000 7fa9a5c00640 1 -- 172.21.15.152:0/1804534053 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa988003520 con 0x7fa9880039b0 2024-04-20T11:31:13.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.175+0000 7fa9a6600640 1 -- 172.21.15.152:0/1804534053 reap_dead start 2024-04-20T11:31:13.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.176+0000 7fa9a5c00640 1 --2- 172.21.15.152:0/1804534053 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa9880039b0 0x7fa988003dd0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fa99400c240 tx=0x7fa988005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f73d705268407f17 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:13.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.177+0000 7fa99fe00640 1 -- 172.21.15.152:0/1804534053 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa994007cf0 con 0x7fa9880039b0 2024-04-20T11:31:13.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.177+0000 7fa99fe00640 1 -- 172.21.15.152:0/1804534053 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa994030070 con 0x7fa9880039b0 2024-04-20T11:31:13.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.177+0000 7fa99fe00640 1 -- 172.21.15.152:0/1804534053 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa994005d70 con 0x7fa9880039b0 2024-04-20T11:31:13.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.177+0000 7fa9a7000640 1 -- 172.21.15.152:0/1804534053 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa9880039b0 msgr2=0x7fa988003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:13.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.177+0000 7fa9a7000640 1 --2- 172.21.15.152:0/1804534053 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa9880039b0 0x7fa988003dd0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fa99400c240 tx=0x7fa988005830 comp rx=0 tx=0).stop 2024-04-20T11:31:13.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.177+0000 7fa9a7000640 1 -- 172.21.15.152:0/1804534053 shutdown_connections 2024-04-20T11:31:13.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.177+0000 7fa9a7000640 1 --2- 172.21.15.152:0/1804534053 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa9880039b0 0x7fa988003dd0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.177+0000 7fa9a7000640 1 -- 172.21.15.152:0/1804534053 >> 172.21.15.152:0/1804534053 conn(0x7fa9a008bfd0 msgr2=0x7fa9a00d45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:13.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.177+0000 7fa9a7000640 1 -- 172.21.15.152:0/1804534053 shutdown_connections 2024-04-20T11:31:13.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.178+0000 7fa9a7000640 1 -- 172.21.15.152:0/1804534053 wait complete. 2024-04-20T11:31:13.182 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.179+0000 7fa9a7000640 1 Processor -- start 2024-04-20T11:31:13.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.179+0000 7fa9a7000640 1 -- start start 2024-04-20T11:31:13.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.180+0000 7fa9a7000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa9880039b0 0x7fa9a0158050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:13.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.180+0000 7fa9a7000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9a00b7180 0x7fa9a00b2d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:13.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.180+0000 7fa9a7000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa9a00b7600 0x7fa9a00b32b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:13.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.180+0000 7fa9a7000640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fa9a00cf1b0 con 0x7fa9880039b0 2024-04-20T11:31:13.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.180+0000 7fa9a7000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa9a00cfb70 con 0x7fa9a00b7180 2024-04-20T11:31:13.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.180+0000 7fa9a7000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa9a0162110 con 0x7fa9a00b7600 2024-04-20T11:31:13.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.180+0000 7fa9a5c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa9880039b0 0x7fa9a0158050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:13.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.181+0000 7fa9a5c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa9880039b0 0x7fa9a0158050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36998/0 (socket says 172.21.15.152:36998) 2024-04-20T11:31:13.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.181+0000 7fa9a5c00640 1 -- 172.21.15.152:0/418532403 learned_addr learned my addr 172.21.15.152:0/418532403 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:13.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.181+0000 7fa9a5200640 1 --2- 172.21.15.152:0/418532403 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9a00b7180 0x7fa9a00b2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:13.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.181+0000 7fa9a5c00640 1 -- 172.21.15.152:0/418532403 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa9a00b7600 msgr2=0x7fa9a00b32b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:13.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.181+0000 7fa9a5c00640 1 --2- 172.21.15.152:0/418532403 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa9a00b7600 0x7fa9a00b32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.184 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.181+0000 7fa9a5c00640 1 -- 172.21.15.152:0/418532403 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9a00b7180 msgr2=0x7fa9a00b2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:13.184 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.181+0000 7fa9a5c00640 1 --2- 172.21.15.152:0/418532403 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9a00b7180 0x7fa9a00b2d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.184 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.181+0000 7fa9a5c00640 1 -- 172.21.15.152:0/418532403 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9a00b7bd0 con 0x7fa9880039b0 2024-04-20T11:31:13.184 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.181+0000 7fa9a5c00640 1 --2- 172.21.15.152:0/418532403 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa9880039b0 0x7fa9a0158050 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fa99400c360 tx=0x7fa9a008d840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:13.184 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.182+0000 7fa99d600640 1 -- 172.21.15.152:0/418532403 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa994045070 con 0x7fa9880039b0 2024-04-20T11:31:13.184 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.182+0000 7fa99d600640 1 -- 172.21.15.152:0/418532403 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa994004bf0 con 0x7fa9880039b0 2024-04-20T11:31:13.184 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.182+0000 7fa99d600640 1 -- 172.21.15.152:0/418532403 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa994001c30 con 0x7fa9880039b0 2024-04-20T11:31:13.184 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.182+0000 7fa9a7000640 1 -- 172.21.15.152:0/418532403 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9a00b7eb0 con 0x7fa9880039b0 2024-04-20T11:31:13.184 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.182+0000 7fa9a7000640 1 -- 172.21.15.152:0/418532403 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9a00b4420 con 0x7fa9880039b0 2024-04-20T11:31:13.184 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.183+0000 7fa9a7000640 1 -- 172.21.15.152:0/418532403 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9a0075e60 con 0x7fa9880039b0 2024-04-20T11:31:13.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.184+0000 7fa99d600640 1 -- 172.21.15.152:0/418532403 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 9) ==== 91193+0+0 (secure 0 0 0) 0x7fa994007070 con 0x7fa9880039b0 2024-04-20T11:31:13.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.186+0000 7fa99d600640 1 --2- 172.21.15.152:0/418532403 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fa98406cad0 0x7fa98406ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:13.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.186+0000 7fa99d600640 1 -- 172.21.15.152:0/418532403 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5709+0+0 (secure 0 0 0) 0x7fa9940aceb0 con 0x7fa9880039b0 2024-04-20T11:31:13.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.186+0000 7fa9a5200640 1 --2- 172.21.15.152:0/418532403 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fa98406cad0 0x7fa98406ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:13.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.187+0000 7fa9a5200640 1 --2- 172.21.15.152:0/418532403 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fa98406cad0 0x7fa98406ef90 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fa9a00ca3e0 tx=0x7fa99000b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:13.188 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.189+0000 7fa99d600640 1 -- 172.21.15.152:0/418532403 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fa99407aa20 con 0x7fa9880039b0 2024-04-20T11:31:13.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.414+0000 7fa9a7000640 1 -- 172.21.15.152:0/418532403 --> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7fa9a0158590 con 0x7fa98406cad0 2024-04-20T11:31:13.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.417+0000 7fa99d600640 1 -- 172.21.15.152:0/418532403 <== mgr.4113 v2:172.21.15.152:6832/2285110240 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7fa990009ca0 con 0x7fa98406cad0 2024-04-20T11:31:13.417 INFO:teuthology.orchestra.run.smithi152.stdout:[ 2024-04-20T11:31:13.417 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:31:13.417 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "cephfs" 2024-04-20T11:31:13.417 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:31:13.418 INFO:teuthology.orchestra.run.smithi152.stdout:] 2024-04-20T11:31:13.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.420+0000 7fa9a7000640 1 -- 172.21.15.152:0/418532403 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fa98406cad0 msgr2=0x7fa98406ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:13.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.420+0000 7fa9a7000640 1 --2- 172.21.15.152:0/418532403 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fa98406cad0 0x7fa98406ef90 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fa9a00ca3e0 tx=0x7fa99000b040 comp rx=0 tx=0).stop 2024-04-20T11:31:13.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.420+0000 7fa9a7000640 1 -- 172.21.15.152:0/418532403 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa9880039b0 msgr2=0x7fa9a0158050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:13.422 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.420+0000 7fa9a7000640 1 --2- 172.21.15.152:0/418532403 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa9880039b0 0x7fa9a0158050 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fa99400c360 tx=0x7fa9a008d840 comp rx=0 tx=0).stop 2024-04-20T11:31:13.422 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.420+0000 7fa9a7000640 1 -- 172.21.15.152:0/418532403 shutdown_connections 2024-04-20T11:31:13.422 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.420+0000 7fa9a7000640 1 --2- 172.21.15.152:0/418532403 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fa98406cad0 0x7fa98406ef90 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.422 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.420+0000 7fa9a7000640 1 --2- 172.21.15.152:0/418532403 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa9a00b7600 0x7fa9a00b32b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.422 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.420+0000 7fa9a7000640 1 --2- 172.21.15.152:0/418532403 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9a00b7180 0x7fa9a00b2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.422 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.420+0000 7fa9a7000640 1 --2- 172.21.15.152:0/418532403 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa9880039b0 0x7fa9a0158050 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.423 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.420+0000 7fa9a7000640 1 -- 172.21.15.152:0/418532403 >> 172.21.15.152:0/418532403 conn(0x7fa9a008bfd0 msgr2=0x7fa9a00c8d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:13.423 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.421+0000 7fa9a7000640 1 -- 172.21.15.152:0/418532403 shutdown_connections 2024-04-20T11:31:13.423 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.421+0000 7fa9a7000640 1 -- 172.21.15.152:0/418532403 wait complete. 2024-04-20T11:31:13.438 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-20T11:31:13.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.640+0000 7fbdca800640 1 Processor -- start 2024-04-20T11:31:13.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.640+0000 7fbdca800640 1 -- start start 2024-04-20T11:31:13.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.642+0000 7fbdca800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fbdc4146000 0x7fbdc40b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:13.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.642+0000 7fbdca800640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fbdc40b2170 0x7fbdc40b2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:13.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.642+0000 7fbdca800640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbdc41463e0 con 0x7fbdc40cd1b0 2024-04-20T11:31:13.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.642+0000 7fbdca800640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fbdc40cdcc0 con 0x7fbdc4146000 2024-04-20T11:31:13.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.642+0000 7fbdca800640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fbdc40b4750 con 0x7fbdc40b2170 2024-04-20T11:31:13.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.642+0000 7fbdc3400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fbdc4146000 0x7fbdc40b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:13.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.642+0000 7fbdc3400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fbdc4146000 0x7fbdc40b4be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:37010/0 (socket says 172.21.15.152:37010) 2024-04-20T11:31:13.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.642+0000 7fbdc3400640 1 -- 172.21.15.152:0/590764674 learned_addr learned my addr 172.21.15.152:0/590764674 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:13.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.642+0000 7fbdc3e00640 1 --1- 172.21.15.152:0/590764674 >> v1:172.21.15.152:6789/0 conn(0x7fbdc40cd1b0 0x7fbdc40cd5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:50568/0 (socket says 172.21.15.152:50568) 2024-04-20T11:31:13.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.642+0000 7fbdc2a00640 1 --2- 172.21.15.152:0/590764674 >> v2:172.21.15.175:3300/0 conn(0x7fbdc40b2170 0x7fbdc40b2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:13.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.643+0000 7fbdc3400640 1 -- 172.21.15.152:0/590764674 >> v2:172.21.15.175:3300/0 conn(0x7fbdc40b2170 msgr2=0x7fbdc40b2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:13.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.643+0000 7fbdc3400640 1 --2- 172.21.15.152:0/590764674 >> v2:172.21.15.175:3300/0 conn(0x7fbdc40b2170 0x7fbdc40b2620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.643+0000 7fbdc3400640 1 -- 172.21.15.152:0/590764674 >> v1:172.21.15.152:6789/0 conn(0x7fbdc40cd1b0 legacy=0x7fbdc40cd5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:13.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.643+0000 7fbdc3400640 1 -- 172.21.15.152:0/590764674 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdc40b3e20 con 0x7fbdc4146000 2024-04-20T11:31:13.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.643+0000 7fbdc2000640 1 -- 172.21.15.152:0/590764674 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1513979422 0 0) 0x7fbdc41463e0 con 0x7fbdc40cd1b0 2024-04-20T11:31:13.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.644+0000 7fbdc3400640 1 --2- 172.21.15.152:0/590764674 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbdc4146000 0x7fbdc40b4be0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fbdc40b3280 tx=0x7fbdb402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=651b23c49d279441 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:13.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.644+0000 7fbdc2000640 1 -- 172.21.15.152:0/590764674 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdb4035020 con 0x7fbdc4146000 2024-04-20T11:31:13.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.644+0000 7fbdc2000640 1 -- 172.21.15.152:0/590764674 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbdb4031870 con 0x7fbdc4146000 2024-04-20T11:31:13.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.644+0000 7fbdc2000640 1 -- 172.21.15.152:0/590764674 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdb4031c40 con 0x7fbdc4146000 2024-04-20T11:31:13.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.645+0000 7fbdca800640 1 -- 172.21.15.152:0/590764674 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbdc4146000 msgr2=0x7fbdc40b4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:13.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.645+0000 7fbdca800640 1 --2- 172.21.15.152:0/590764674 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbdc4146000 0x7fbdc40b4be0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fbdc40b3280 tx=0x7fbdb402b910 comp rx=0 tx=0).stop 2024-04-20T11:31:13.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.645+0000 7fbdca800640 1 -- 172.21.15.152:0/590764674 shutdown_connections 2024-04-20T11:31:13.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.645+0000 7fbdca800640 1 --2- 172.21.15.152:0/590764674 >> v2:172.21.15.175:3300/0 conn(0x7fbdc40b2170 0x7fbdc40b2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.645+0000 7fbdca800640 1 --2- 172.21.15.152:0/590764674 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbdc4146000 0x7fbdc40b4be0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.645+0000 7fbdca800640 1 -- 172.21.15.152:0/590764674 >> 172.21.15.152:0/590764674 conn(0x7fbdc406b980 msgr2=0x7fbdc406bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:13.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.645+0000 7fbdca800640 1 -- 172.21.15.152:0/590764674 shutdown_connections 2024-04-20T11:31:13.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.645+0000 7fbdca800640 1 -- 172.21.15.152:0/590764674 wait complete. 2024-04-20T11:31:13.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.647+0000 7fbdca800640 1 Processor -- start 2024-04-20T11:31:13.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.647+0000 7fbdca800640 1 -- start start 2024-04-20T11:31:13.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.648+0000 7fbdca800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbdc40b2170 0x7fbdc4141e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:13.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.648+0000 7fbdca800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbdc40cd1b0 0x7fbdc41f7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:13.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.648+0000 7fbdca800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbdc4146000 0x7fbdc41f4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:13.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.648+0000 7fbdca800640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc415ccf0 con 0x7fbdc40b2170 2024-04-20T11:31:13.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.648+0000 7fbdca800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc40b43e0 con 0x7fbdc40cd1b0 2024-04-20T11:31:13.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.648+0000 7fbdca800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fbdc415ce70 con 0x7fbdc4146000 2024-04-20T11:31:13.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.648+0000 7fbdc3400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbdc40b2170 0x7fbdc4141e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:13.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.648+0000 7fbdc3400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbdc40b2170 0x7fbdc4141e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:37026/0 (socket says 172.21.15.152:37026) 2024-04-20T11:31:13.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.648+0000 7fbdc3400640 1 -- 172.21.15.152:0/704381348 learned_addr learned my addr 172.21.15.152:0/704381348 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:13.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.648+0000 7fbdc3400640 1 -- 172.21.15.152:0/704381348 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbdc4146000 msgr2=0x7fbdc41f4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:13.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.648+0000 7fbdc3400640 1 --2- 172.21.15.152:0/704381348 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbdc4146000 0x7fbdc41f4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.648+0000 7fbdc3400640 1 -- 172.21.15.152:0/704381348 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbdc40cd1b0 msgr2=0x7fbdc41f7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:13.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.648+0000 7fbdc3400640 1 --2- 172.21.15.152:0/704381348 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbdc40cd1b0 0x7fbdc41f7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.648+0000 7fbdc3400640 1 -- 172.21.15.152:0/704381348 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdc41f84d0 con 0x7fbdc40b2170 2024-04-20T11:31:13.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.649+0000 7fbdc3400640 1 --2- 172.21.15.152:0/704381348 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbdc40b2170 0x7fbdc4141e80 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fbdb402b950 tx=0x7fbdb40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:13.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.649+0000 7fbdbb400640 1 -- 172.21.15.152:0/704381348 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdb4040070 con 0x7fbdc40b2170 2024-04-20T11:31:13.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.649+0000 7fbdbb400640 1 -- 172.21.15.152:0/704381348 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbdb4030d00 con 0x7fbdc40b2170 2024-04-20T11:31:13.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.649+0000 7fbdbb400640 1 -- 172.21.15.152:0/704381348 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdb4035040 con 0x7fbdc40b2170 2024-04-20T11:31:13.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.649+0000 7fbdca800640 1 -- 172.21.15.152:0/704381348 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdc41f87b0 con 0x7fbdc40b2170 2024-04-20T11:31:13.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.649+0000 7fbdca800640 1 -- 172.21.15.152:0/704381348 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdc41f46f0 con 0x7fbdc40b2170 2024-04-20T11:31:13.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.650+0000 7fbdca800640 1 -- 172.21.15.152:0/704381348 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbdc40689f0 con 0x7fbdc40b2170 2024-04-20T11:31:13.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.651+0000 7fbdbb400640 1 -- 172.21.15.152:0/704381348 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 9) ==== 91193+0+0 (secure 0 0 0) 0x7fbdb404b060 con 0x7fbdc40b2170 2024-04-20T11:31:13.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.652+0000 7fbdbb400640 1 --2- 172.21.15.152:0/704381348 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fbda406caa0 0x7fbda406ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:13.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.652+0000 7fbdbb400640 1 -- 172.21.15.152:0/704381348 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5709+0+0 (secure 0 0 0) 0x7fbdb40add40 con 0x7fbdc40b2170 2024-04-20T11:31:13.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.652+0000 7fbdc2a00640 1 --2- 172.21.15.152:0/704381348 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fbda406caa0 0x7fbda406ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:13.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.653+0000 7fbdc2a00640 1 --2- 172.21.15.152:0/704381348 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fbda406caa0 0x7fbda406ef60 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fbdc40cbc70 tx=0x7fbdb0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:13.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.657+0000 7fbdbb400640 1 -- 172.21.15.152:0/704381348 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fbdb407be50 con 0x7fbdc40b2170 2024-04-20T11:31:13.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.857+0000 7fbdca800640 1 -- 172.21.15.152:0/704381348 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fbdc40d0ee0 con 0x7fbdc40b2170 2024-04-20T11:31:13.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.857+0000 7fbdbb400640 1 -- 172.21.15.152:0/704381348 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) ==== 122+0+0 (secure 0 0 0) 0x7fbdb407b750 con 0x7fbdc40b2170 2024-04-20T11:31:13.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.860+0000 7fbdca800640 1 -- 172.21.15.152:0/704381348 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fbda406caa0 msgr2=0x7fbda406ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:13.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.860+0000 7fbdca800640 1 --2- 172.21.15.152:0/704381348 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fbda406caa0 0x7fbda406ef60 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fbdc40cbc70 tx=0x7fbdb0008040 comp rx=0 tx=0).stop 2024-04-20T11:31:13.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.860+0000 7fbdca800640 1 -- 172.21.15.152:0/704381348 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbdc40b2170 msgr2=0x7fbdc4141e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:13.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.860+0000 7fbdca800640 1 --2- 172.21.15.152:0/704381348 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbdc40b2170 0x7fbdc4141e80 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fbdb402b950 tx=0x7fbdb40046c0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.860+0000 7fbdca800640 1 -- 172.21.15.152:0/704381348 shutdown_connections 2024-04-20T11:31:13.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.860+0000 7fbdca800640 1 --2- 172.21.15.152:0/704381348 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fbda406caa0 0x7fbda406ef60 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.860+0000 7fbdca800640 1 --2- 172.21.15.152:0/704381348 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbdc4146000 0x7fbdc41f4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.860+0000 7fbdca800640 1 --2- 172.21.15.152:0/704381348 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbdc40cd1b0 0x7fbdc41f7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.860+0000 7fbdca800640 1 --2- 172.21.15.152:0/704381348 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbdc40b2170 0x7fbdc4141e80 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.860+0000 7fbdca800640 1 -- 172.21.15.152:0/704381348 >> 172.21.15.152:0/704381348 conn(0x7fbdc406b980 msgr2=0x7fbdc40b17e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:13.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.860+0000 7fbdca800640 1 -- 172.21.15.152:0/704381348 shutdown_connections 2024-04-20T11:31:13.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:13.860+0000 7fbdca800640 1 -- 172.21.15.152:0/704381348 wait complete. 2024-04-20T11:31:13.878 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:31:13.878 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p ./volumes/_nogroup/subvol99630) 2024-04-20T11:31:13.977 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:31:13.977 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2024-04-20T11:31:14.034 INFO:teuthology.orchestra.run.smithi152.stdout:cephfs_data 2024-04-20T11:31:14.034 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:31:14.035 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data ./volumes/_nogroup/subvol99630) 2024-04-20T11:31:14.124 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol77311 2024-04-20T11:31:14.326 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:14.326+0000 7facb3e00640 1 Processor -- start 2024-04-20T11:31:14.326 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:14.327+0000 7facb3e00640 1 -- start start 2024-04-20T11:31:14.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:14.328+0000 7facb3e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7facb4163230 0x7facb40d2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:14.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:14.328+0000 7facb3e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7facb40d33d0 0x7facb40cf470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:14.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:14.328+0000 7facb3e00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7facb40d19c0 con 0x7facb415eb50 2024-04-20T11:31:14.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:14.328+0000 7facb3e00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7facb4162750 con 0x7facb4163230 2024-04-20T11:31:14.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:14.328+0000 7facb3e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7facb4162a30 con 0x7facb40d33d0 2024-04-20T11:31:14.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:14.329+0000 7facb2a00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7facb415eb50 0x7facb40d0f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:50572/0 (socket says 172.21.15.152:50572) 2024-04-20T11:31:14.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:14.329+0000 7facb2a00640 1 -- 172.21.15.152:0/1783713076 learned_addr learned my addr 172.21.15.152:0/1783713076 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:31:14.330 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:14.329+0000 7facb2000640 1 --2- 172.21.15.152:0/1783713076 >> v2:172.21.15.175:3300/0 conn(0x7facb4163230 0x7facb40d2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:14.330 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:14.329+0000 7facb1600640 1 -- 172.21.15.152:0/1783713076 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3525961789 0 0) 0x7facb40d19c0 con 0x7facb415eb50 2024-04-20T11:31:14.331 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:14.329+0000 7facb1600640 1 -- 172.21.15.152:0/1783713076 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fac98003520 con 0x7facb415eb50 2024-04-20T11:31:14.331 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:14.329+0000 7facb3400640 1 --2- 172.21.15.152:0/1783713076 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7facb40d33d0 0x7facb40cf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:14.331 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:14.330+0000 7facb1600640 1 -- 172.21.15.152:0/1783713076 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2944720430 0 0) 0x7fac98003520 con 0x7facb415eb50 2024-04-20T11:31:14.331 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:14.330+0000 7facb1600640 1 -- 172.21.15.152:0/1783713076 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7facb40d33d0 msgr2=0x7facb40cf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:14.331 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:14.330+0000 7facb1600640 1 --2- 172.21.15.152:0/1783713076 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7facb40d33d0 0x7facb40cf470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:14.331 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:14.330+0000 7facb1600640 1 -- 172.21.15.152:0/1783713076 >> v2:172.21.15.175:3300/0 conn(0x7facb4163230 msgr2=0x7facb40d2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:14.331 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:14.330+0000 7facb1600640 1 --2- 172.21.15.152:0/1783713076 >> v2:172.21.15.175:3300/0 conn(0x7facb4163230 0x7facb40d2e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:14.331 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:14.330+0000 7facb1600640 1 -- 172.21.15.152:0/1783713076 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facb4162d10 con 0x7facb415eb50 2024-04-20T11:31:14.331 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:14.330+0000 7facb1600640 1 -- 172.21.15.152:0/1783713076 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7faca4001670 con 0x7facb415eb50 2024-04-20T11:31:14.331 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:14.330+0000 7facb1600640 1 -- 172.21.15.152:0/1783713076 >> v1:172.21.15.152:6789/0 conn(0x7facb415eb50 legacy=0x7facb40d0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:14.331 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:14.330+0000 7facb1600640 1 --2- 172.21.15.152:0/1783713076 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fac980039b0 0x7fac98003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:14.331 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:14.330+0000 7facb1600640 1 --2- 172.21.15.152:0/1783713076 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fac98004800 0x7fac98004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:14.331 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:14.330+0000 7facb1600640 1 --2- 172.21.15.152:0/1783713076 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fac980051c0 0x7fac9800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:14.331 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:14.330+0000 7facb1600640 1 -- 172.21.15.152:0/1783713076 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7facb4162750 con 0x7fac98004800 2024-04-20T11:31:14.332 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:14.330+0000 7facb1600640 1 -- 172.21.15.152:0/1783713076 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7facb4162a30 con 0x7fac980039b0 2024-04-20T11:31:14.332 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:14.330+0000 7facb1600640 1 -- 172.21.15.152:0/1783713076 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fac9800c030 con 0x7fac980051c0 2024-04-20T11:31:14.332 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:14.330+0000 7facb3400640 1 --2- 172.21.15.152:0/1783713076 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7facb40d33d0 0x7facb40cf470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:31:14.332 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:14.330+0000 7facb2000640 1 --2- 172.21.15.152:0/1783713076 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fac98004800 0x7fac98004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:14.332 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:14.330+0000 7facb2a00640 1 --2- 172.21.15.152:0/1783713076 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fac980039b0 0x7fac98003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:14.332 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:14.330+0000 7facb2000640 1 -- 172.21.15.152:0/1783713076 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fac980051c0 msgr2=0x7fac9800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:14.332 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:14.330+0000 7facb2000640 1 --2- 172.21.15.152:0/1783713076 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fac980051c0 0x7fac9800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:14.332 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:14.331+0000 7facb2000640 1 -- 172.21.15.152:0/1783713076 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fac980039b0 msgr2=0x7fac98003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:14.332 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:14.331+0000 7facb2000640 1 --2- 172.21.15.152:0/1783713076 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fac980039b0 0x7fac98003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:14.332 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:14.331+0000 7facb2000640 1 -- 172.21.15.152:0/1783713076 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac98003520 con 0x7fac98004800 2024-04-20T11:31:14.333 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:14.332+0000 7facb2000640 1 --2- 172.21.15.152:0/1783713076 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fac98004800 0x7fac98004c80 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fac9c00e170 tx=0x7fac98005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7c6d6009eff8b77 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:14.333 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:14.332+0000 7facb1600640 1 -- 172.21.15.152:0/1783713076 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac9c039070 con 0x7fac98004800 2024-04-20T11:31:14.333 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:14.332+0000 7facb1600640 1 -- 172.21.15.152:0/1783713076 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac9c02cca0 con 0x7fac98004800 2024-04-20T11:31:14.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:14.332+0000 7facb1600640 1 -- 172.21.15.152:0/1783713076 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac9c009c60 con 0x7fac98004800 2024-04-20T11:31:14.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:14.332+0000 7facb3e00640 1 -- 172.21.15.152:0/1783713076 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fac98004800 msgr2=0x7fac98004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:14.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:14.332+0000 7facb3e00640 1 --2- 172.21.15.152:0/1783713076 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fac98004800 0x7fac98004c80 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fac9c00e170 tx=0x7fac98005830 comp rx=0 tx=0).stop 2024-04-20T11:31:14.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:14.333+0000 7facb3400640 1 -- 172.21.15.152:0/1783713076 reap_dead start 2024-04-20T11:31:14.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:14.333+0000 7facb3e00640 1 -- 172.21.15.152:0/1783713076 shutdown_connections 2024-04-20T11:31:14.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:14.333+0000 7facb3e00640 1 -- 172.21.15.152:0/1783713076 >> 172.21.15.152:0/1783713076 conn(0x7facb408bfd0 msgr2=0x7facb40d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:14.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:14.333+0000 7facb3e00640 1 -- 172.21.15.152:0/1783713076 shutdown_connections 2024-04-20T11:31:14.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:14.333+0000 7facb3e00640 1 -- 172.21.15.152:0/1783713076 wait complete. 2024-04-20T11:31:14.335 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:14.335+0000 7facb3e00640 1 Processor -- start 2024-04-20T11:31:14.335 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:14.335+0000 7facb3e00640 1 -- start start 2024-04-20T11:31:14.335 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:14.336+0000 7facb3e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fac98004800 0x7facb40b61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:14.336 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:14.336+0000 7facb3e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7facb40b6700 0x7facb415ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:14.336 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:14.336+0000 7facb3e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7facb415f470 0x7facb414bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:14.336 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:14.336+0000 7facb3e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7facb40d1610 con 0x7facb415f470 2024-04-20T11:31:14.336 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:14.336+0000 7facb3e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7facb40cf470 con 0x7fac98004800 2024-04-20T11:31:14.336 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:14.336+0000 7facb3e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7facb40b5120 con 0x7facb40b6700 2024-04-20T11:31:14.336 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:14.336+0000 7facb3400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7facb415f470 0x7facb414bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:14.336 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:14.336+0000 7facb3400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7facb415f470 0x7facb414bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:37048/0 (socket says 172.21.15.152:37048) 2024-04-20T11:31:14.336 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:14.336+0000 7facb3400640 1 -- 172.21.15.152:0/2588225755 learned_addr learned my addr 172.21.15.152:0/2588225755 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:14.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:14.336+0000 7facb2a00640 1 --2- 172.21.15.152:0/2588225755 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fac98004800 0x7facb40b61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:14.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:14.336+0000 7facb3400640 1 -- 172.21.15.152:0/2588225755 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7facb40b6700 msgr2=0x7facb415ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:14.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:14.336+0000 7facb3400640 1 --2- 172.21.15.152:0/2588225755 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7facb40b6700 0x7facb415ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:14.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:14.336+0000 7facb3400640 1 -- 172.21.15.152:0/2588225755 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fac98004800 msgr2=0x7facb40b61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:14.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:14.336+0000 7facb3400640 1 --2- 172.21.15.152:0/2588225755 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fac98004800 0x7facb40b61c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:14.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:14.337+0000 7facb3400640 1 -- 172.21.15.152:0/2588225755 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facb40b5400 con 0x7facb415f470 2024-04-20T11:31:14.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:14.337+0000 7facb3400640 1 --2- 172.21.15.152:0/2588225755 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7facb415f470 0x7facb414bcf0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7facac006fd0 tx=0x7facac00e8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:14.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:14.337+0000 7facaaa00640 1 -- 172.21.15.152:0/2588225755 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facac01a070 con 0x7facb415f470 2024-04-20T11:31:14.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:14.337+0000 7facaaa00640 1 -- 172.21.15.152:0/2588225755 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facac004230 con 0x7facb415f470 2024-04-20T11:31:14.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:14.337+0000 7facb3e00640 1 -- 172.21.15.152:0/2588225755 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facb40b56e0 con 0x7facb415f470 2024-04-20T11:31:14.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:14.337+0000 7facaaa00640 1 -- 172.21.15.152:0/2588225755 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facac00f040 con 0x7facb415f470 2024-04-20T11:31:14.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:14.337+0000 7facb3e00640 1 -- 172.21.15.152:0/2588225755 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facb40b5ca0 con 0x7facb415f470 2024-04-20T11:31:14.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:14.338+0000 7facb3e00640 1 -- 172.21.15.152:0/2588225755 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facb4075e60 con 0x7facb415f470 2024-04-20T11:31:14.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:14.340+0000 7facaaa00640 1 -- 172.21.15.152:0/2588225755 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 9) ==== 91193+0+0 (secure 0 0 0) 0x7facac01d070 con 0x7facb415f470 2024-04-20T11:31:14.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:14.340+0000 7facaaa00640 1 --2- 172.21.15.152:0/2588225755 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fac9006caa0 0x7fac9006ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:14.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:14.341+0000 7facaaa00640 1 -- 172.21.15.152:0/2588225755 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5709+0+0 (secure 0 0 0) 0x7facac08ad80 con 0x7facb415f470 2024-04-20T11:31:14.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:14.341+0000 7facb2a00640 1 --2- 172.21.15.152:0/2588225755 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fac9006caa0 0x7fac9006ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:14.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:14.341+0000 7facb2a00640 1 --2- 172.21.15.152:0/2588225755 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fac9006caa0 0x7fac9006ef60 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7faca4001050 tx=0x7faca4009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:14.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:14.345+0000 7facaaa00640 1 -- 172.21.15.152:0/2588225755 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7facac051b90 con 0x7facb415f470 2024-04-20T11:31:14.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:14.569+0000 7facb3e00640 1 -- 172.21.15.152:0/2588225755 --> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvol77311", "target": ["mon-mgr", ""]}) -- 0x7facb408c680 con 0x7fac9006caa0 2024-04-20T11:31:14.578 INFO:tasks.ceph.mgr.y.smithi152.stderr:2024-04-20T11:31:14.579+0000 7fd165e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:31:14.578 INFO:tasks.ceph.mgr.y.smithi152.stderr:2024-04-20T11:31:14.579+0000 7fd165e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:31:14.579 INFO:tasks.ceph.mgr.y.smithi152.stderr:2024-04-20T11:31:14.579+0000 7fd165e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:31:14.579 INFO:tasks.ceph.mgr.y.smithi152.stderr:2024-04-20T11:31:14.579+0000 7fd165e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:31:14.579 INFO:tasks.ceph.mgr.y.smithi152.stderr:2024-04-20T11:31:14.579+0000 7fd165e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:31:14.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:14.657+0000 7facaaa00640 1 -- 172.21.15.152:0/2588225755 <== mgr.4113 v2:172.21.15.152:6832/2285110240 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7facb408c680 con 0x7fac9006caa0 2024-04-20T11:31:14.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:14.661+0000 7facb3e00640 1 -- 172.21.15.152:0/2588225755 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fac9006caa0 msgr2=0x7fac9006ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:14.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:14.661+0000 7facb3e00640 1 --2- 172.21.15.152:0/2588225755 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fac9006caa0 0x7fac9006ef60 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7faca4001050 tx=0x7faca4009040 comp rx=0 tx=0).stop 2024-04-20T11:31:14.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:14.661+0000 7facb3e00640 1 -- 172.21.15.152:0/2588225755 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7facb415f470 msgr2=0x7facb414bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:14.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:14.661+0000 7facb3e00640 1 --2- 172.21.15.152:0/2588225755 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7facb415f470 0x7facb414bcf0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7facac006fd0 tx=0x7facac00e8a0 comp rx=0 tx=0).stop 2024-04-20T11:31:14.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:14.661+0000 7facb3e00640 1 -- 172.21.15.152:0/2588225755 shutdown_connections 2024-04-20T11:31:14.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:14.661+0000 7facb3e00640 1 --2- 172.21.15.152:0/2588225755 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fac9006caa0 0x7fac9006ef60 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:14.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:14.661+0000 7facb3e00640 1 --2- 172.21.15.152:0/2588225755 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7facb415f470 0x7facb414bcf0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:14.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:14.661+0000 7facb3e00640 1 --2- 172.21.15.152:0/2588225755 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7facb40b6700 0x7facb415ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:14.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:14.661+0000 7facb3e00640 1 --2- 172.21.15.152:0/2588225755 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fac98004800 0x7facb40b61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:14.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:14.661+0000 7facb3e00640 1 -- 172.21.15.152:0/2588225755 >> 172.21.15.152:0/2588225755 conn(0x7facb408bfd0 msgr2=0x7facb40d37e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:14.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:14.661+0000 7facb3e00640 1 -- 172.21.15.152:0/2588225755 shutdown_connections 2024-04-20T11:31:14.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:14.661+0000 7facb3e00640 1 -- 172.21.15.152:0/2588225755 wait complete. 2024-04-20T11:31:14.682 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:31:14.682 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cp ./volumes/_nogroup/subvol77311/.meta ./volumes/_nogroup/subvol99630) 2024-04-20T11:31:19.381 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol99630 2024-04-20T11:31:19.590 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:19.591+0000 7f3d90e00640 1 Processor -- start 2024-04-20T11:31:19.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:19.591+0000 7f3d90e00640 1 -- start start 2024-04-20T11:31:19.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:19.592+0000 7f3d90e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f3d8c071cd0 0x7f3d8c0720b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:19.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:19.592+0000 7f3d90e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f3d8c160220 0x7f3d8c1606b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:19.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:19.593+0000 7f3d90e00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d8c0725f0 con 0x7f3d8c06eb30 2024-04-20T11:31:19.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:19.593+0000 7f3d90e00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f3d8c13f8e0 con 0x7f3d8c160220 2024-04-20T11:31:19.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:19.593+0000 7f3d90e00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f3d8c160f90 con 0x7f3d8c071cd0 2024-04-20T11:31:19.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:19.593+0000 7f3d8be00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f3d8c06eb30 0x7f3d8c06efc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:39732/0 (socket says 172.21.15.152:39732) 2024-04-20T11:31:19.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:19.593+0000 7f3d8be00640 1 -- 172.21.15.152:0/3229266700 learned_addr learned my addr 172.21.15.152:0/3229266700 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:31:19.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:19.593+0000 7f3d83e00640 1 --2- 172.21.15.152:0/3229266700 >> v2:172.21.15.152:3300/0 conn(0x7f3d8c160220 0x7f3d8c1606b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:19.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:19.593+0000 7f3d8b400640 1 --2- 172.21.15.152:0/3229266700 >> v2:172.21.15.175:3300/0 conn(0x7f3d8c071cd0 0x7f3d8c0720b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:19.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:19.593+0000 7f3d8aa00640 1 -- 172.21.15.152:0/3229266700 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 526314619 0 0) 0x7f3d8c0725f0 con 0x7f3d8c06eb30 2024-04-20T11:31:19.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:19.594+0000 7f3d8aa00640 1 -- 172.21.15.152:0/3229266700 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3d70003520 con 0x7f3d8c06eb30 2024-04-20T11:31:19.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:19.594+0000 7f3d83e00640 1 -- 172.21.15.152:0/3229266700 >> v2:172.21.15.175:3300/0 conn(0x7f3d8c071cd0 msgr2=0x7f3d8c0720b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:19.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:19.594+0000 7f3d83e00640 1 --2- 172.21.15.152:0/3229266700 >> v2:172.21.15.175:3300/0 conn(0x7f3d8c071cd0 0x7f3d8c0720b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:19.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:19.594+0000 7f3d83e00640 1 -- 172.21.15.152:0/3229266700 >> v1:172.21.15.152:6789/0 conn(0x7f3d8c06eb30 legacy=0x7f3d8c06efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:19.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:19.594+0000 7f3d83e00640 1 -- 172.21.15.152:0/3229266700 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d8c06dd90 con 0x7f3d8c160220 2024-04-20T11:31:19.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:19.595+0000 7f3d83e00640 1 --2- 172.21.15.152:0/3229266700 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3d8c160220 0x7f3d8c1606b0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f3d8c06c460 tx=0x7f3d7402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=373f43f5d8114314 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:19.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:19.595+0000 7f3d8aa00640 1 -- 172.21.15.152:0/3229266700 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d740315a0 con 0x7f3d8c160220 2024-04-20T11:31:19.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:19.595+0000 7f3d8aa00640 1 -- 172.21.15.152:0/3229266700 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d74037070 con 0x7f3d8c160220 2024-04-20T11:31:19.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:19.595+0000 7f3d8aa00640 1 -- 172.21.15.152:0/3229266700 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d74031960 con 0x7f3d8c160220 2024-04-20T11:31:19.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:19.596+0000 7f3d90e00640 1 -- 172.21.15.152:0/3229266700 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3d8c160220 msgr2=0x7f3d8c1606b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:19.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:19.596+0000 7f3d90e00640 1 --2- 172.21.15.152:0/3229266700 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3d8c160220 0x7f3d8c1606b0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f3d8c06c460 tx=0x7f3d7402b840 comp rx=0 tx=0).stop 2024-04-20T11:31:19.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:19.596+0000 7f3d90e00640 1 -- 172.21.15.152:0/3229266700 shutdown_connections 2024-04-20T11:31:19.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:19.596+0000 7f3d90e00640 1 --2- 172.21.15.152:0/3229266700 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3d8c160220 0x7f3d8c1606b0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:19.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:19.596+0000 7f3d90e00640 1 --2- 172.21.15.152:0/3229266700 >> v2:172.21.15.175:3300/0 conn(0x7f3d8c071cd0 0x7f3d8c0720b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:19.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:19.596+0000 7f3d90e00640 1 -- 172.21.15.152:0/3229266700 >> 172.21.15.152:0/3229266700 conn(0x7f3d8c067ae0 msgr2=0x7f3d8c06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:19.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:19.596+0000 7f3d90e00640 1 -- 172.21.15.152:0/3229266700 shutdown_connections 2024-04-20T11:31:19.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:19.596+0000 7f3d90e00640 1 -- 172.21.15.152:0/3229266700 wait complete. 2024-04-20T11:31:19.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:19.598+0000 7f3d90e00640 1 Processor -- start 2024-04-20T11:31:19.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:19.598+0000 7f3d90e00640 1 -- start start 2024-04-20T11:31:19.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:19.598+0000 7f3d90e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3d8c06eb30 0x7f3d8c1f2730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:19.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:19.599+0000 7f3d90e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3d8c071cd0 0x7f3d8c133c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:19.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:19.599+0000 7f3d90e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3d8c160220 0x7f3d8c1560c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:19.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:19.599+0000 7f3d90e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f3d8c06e630 con 0x7f3d8c160220 2024-04-20T11:31:19.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:19.599+0000 7f3d90e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3d8c06e350 con 0x7f3d8c06eb30 2024-04-20T11:31:19.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:19.599+0000 7f3d90e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f3d8c160bf0 con 0x7f3d8c071cd0 2024-04-20T11:31:19.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:19.599+0000 7f3d8be00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3d8c160220 0x7f3d8c1560c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:19.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:19.599+0000 7f3d8be00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3d8c160220 0x7f3d8c1560c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:55522/0 (socket says 172.21.15.152:55522) 2024-04-20T11:31:19.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:19.599+0000 7f3d8be00640 1 -- 172.21.15.152:0/563390472 learned_addr learned my addr 172.21.15.152:0/563390472 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:19.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:19.599+0000 7f3d8b400640 1 --2- 172.21.15.152:0/563390472 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3d8c06eb30 0x7f3d8c1f2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:19.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:19.599+0000 7f3d8be00640 1 -- 172.21.15.152:0/563390472 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3d8c071cd0 msgr2=0x7f3d8c133c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:19.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:19.599+0000 7f3d8be00640 1 --2- 172.21.15.152:0/563390472 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3d8c071cd0 0x7f3d8c133c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:19.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:19.599+0000 7f3d8be00640 1 -- 172.21.15.152:0/563390472 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3d8c06eb30 msgr2=0x7f3d8c1f2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:19.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:19.599+0000 7f3d8be00640 1 --2- 172.21.15.152:0/563390472 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3d8c06eb30 0x7f3d8c1f2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:19.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:19.599+0000 7f3d8be00640 1 -- 172.21.15.152:0/563390472 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d8c156600 con 0x7f3d8c160220 2024-04-20T11:31:19.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:19.599+0000 7f3d8b400640 1 --2- 172.21.15.152:0/563390472 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3d8c06eb30 0x7f3d8c1f2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:31:19.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:19.600+0000 7f3d8be00640 1 --2- 172.21.15.152:0/563390472 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3d8c160220 0x7f3d8c1560c0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f3d8400cb40 tx=0x7f3d8400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:19.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:19.600+0000 7f3d83400640 1 -- 172.21.15.152:0/563390472 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d84017070 con 0x7f3d8c160220 2024-04-20T11:31:19.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:19.600+0000 7f3d83400640 1 -- 172.21.15.152:0/563390472 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d8400aca0 con 0x7f3d8c160220 2024-04-20T11:31:19.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:19.600+0000 7f3d90e00640 1 -- 172.21.15.152:0/563390472 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d8c1f2d80 con 0x7f3d8c160220 2024-04-20T11:31:19.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:19.600+0000 7f3d83400640 1 -- 172.21.15.152:0/563390472 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d84012620 con 0x7f3d8c160220 2024-04-20T11:31:19.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:19.600+0000 7f3d90e00640 1 -- 172.21.15.152:0/563390472 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d8c1f3340 con 0x7f3d8c160220 2024-04-20T11:31:19.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:19.601+0000 7f3d90e00640 1 -- 172.21.15.152:0/563390472 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d8c075e60 con 0x7f3d8c160220 2024-04-20T11:31:19.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:19.602+0000 7f3d83400640 1 -- 172.21.15.152:0/563390472 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 10) ==== 91244+0+0 (secure 0 0 0) 0x7f3d8401a070 con 0x7f3d8c160220 2024-04-20T11:31:19.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:19.603+0000 7f3d83400640 1 --2- 172.21.15.152:0/563390472 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f3d6806cb30 0x7f3d6806eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:19.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:19.603+0000 7f3d83400640 1 -- 172.21.15.152:0/563390472 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5709+0+0 (secure 0 0 0) 0x7f3d8408beb0 con 0x7f3d8c160220 2024-04-20T11:31:19.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:19.603+0000 7f3d8b400640 1 --2- 172.21.15.152:0/563390472 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f3d6806cb30 0x7f3d6806eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:19.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:19.604+0000 7f3d8b400640 1 --2- 172.21.15.152:0/563390472 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f3d6806cb30 0x7f3d6806eff0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c006fd0 tx=0x7f3d7c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:19.607 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:19.608+0000 7f3d83400640 1 -- 172.21.15.152:0/563390472 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f3d84059ee0 con 0x7f3d8c160220 2024-04-20T11:31:19.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:19.842+0000 7f3d90e00640 1 -- 172.21.15.152:0/563390472 --> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol99630", "target": ["mon-mgr", ""]}) -- 0x7f3d8c072d30 con 0x7f3d6806cb30 2024-04-20T11:31:19.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:19.905+0000 7f3d83400640 1 -- 172.21.15.152:0/563390472 <== mgr.4113 v2:172.21.15.152:6832/2285110240 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+30 (secure 0 0 0) 0x7f3d8c072d30 con 0x7f3d6806cb30 2024-04-20T11:31:19.906 INFO:teuthology.orchestra.run.smithi152.stdout:/volumes/_nogroup/subvol99630 2024-04-20T11:31:19.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:19.908+0000 7f3d90e00640 1 -- 172.21.15.152:0/563390472 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f3d6806cb30 msgr2=0x7f3d6806eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:19.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:19.908+0000 7f3d90e00640 1 --2- 172.21.15.152:0/563390472 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f3d6806cb30 0x7f3d6806eff0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c006fd0 tx=0x7f3d7c008040 comp rx=0 tx=0).stop 2024-04-20T11:31:19.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:19.908+0000 7f3d90e00640 1 -- 172.21.15.152:0/563390472 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3d8c160220 msgr2=0x7f3d8c1560c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:19.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:19.909+0000 7f3d90e00640 1 --2- 172.21.15.152:0/563390472 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3d8c160220 0x7f3d8c1560c0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f3d8400cb40 tx=0x7f3d8400a060 comp rx=0 tx=0).stop 2024-04-20T11:31:19.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:19.909+0000 7f3d90e00640 1 -- 172.21.15.152:0/563390472 shutdown_connections 2024-04-20T11:31:19.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:19.909+0000 7f3d90e00640 1 --2- 172.21.15.152:0/563390472 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f3d6806cb30 0x7f3d6806eff0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:19.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:19.909+0000 7f3d90e00640 1 --2- 172.21.15.152:0/563390472 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3d8c160220 0x7f3d8c1560c0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:19.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:19.909+0000 7f3d90e00640 1 --2- 172.21.15.152:0/563390472 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3d8c071cd0 0x7f3d8c133c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:19.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:19.909+0000 7f3d90e00640 1 --2- 172.21.15.152:0/563390472 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3d8c06eb30 0x7f3d8c1f2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:19.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:19.909+0000 7f3d90e00640 1 -- 172.21.15.152:0/563390472 >> 172.21.15.152:0/563390472 conn(0x7f3d8c067ae0 msgr2=0x7f3d8c15d1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:19.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:19.909+0000 7f3d90e00640 1 -- 172.21.15.152:0/563390472 shutdown_connections 2024-04-20T11:31:19.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:19.909+0000 7f3d90e00640 1 -- 172.21.15.152:0/563390472 wait complete. 2024-04-20T11:31:19.930 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol99630 2024-04-20T11:31:20.137 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:20.137+0000 7f7509e00640 1 Processor -- start 2024-04-20T11:31:20.137 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:20.138+0000 7f7509e00640 1 -- start start 2024-04-20T11:31:20.139 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:20.139+0000 7f7509e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f7504147cd0 0x7f750406e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:20.139 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:20.139+0000 7f7509e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f750406f1c0 0x7f750406f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:20.139 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:20.139+0000 7f7509e00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f750415f7e0 con 0x7f75041480b0 2024-04-20T11:31:20.139 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:20.139+0000 7f7509e00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f750406fb70 con 0x7f750406f1c0 2024-04-20T11:31:20.139 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:20.139+0000 7f7509e00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f750415fac0 con 0x7f7504147cd0 2024-04-20T11:31:20.139 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:20.140+0000 7f7509400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f750406f1c0 0x7f750406f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:20.139 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:20.140+0000 7f7509400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f750406f1c0 0x7f750406f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:55530/0 (socket says 172.21.15.152:55530) 2024-04-20T11:31:20.140 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:20.140+0000 7f7509400640 1 -- 172.21.15.152:0/995399059 learned_addr learned my addr 172.21.15.152:0/995399059 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:20.140 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:20.140+0000 7f7508a00640 1 --2- 172.21.15.152:0/995399059 >> v2:172.21.15.175:3300/0 conn(0x7f7504147cd0 0x7f750406e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:20.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:20.140+0000 7f7509400640 1 -- 172.21.15.152:0/995399059 >> v2:172.21.15.175:3300/0 conn(0x7f7504147cd0 msgr2=0x7f750406e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:20.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:20.140+0000 7f7509400640 1 --2- 172.21.15.152:0/995399059 >> v2:172.21.15.175:3300/0 conn(0x7f7504147cd0 0x7f750406e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:20.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:20.140+0000 7f7509400640 1 -- 172.21.15.152:0/995399059 >> v1:172.21.15.175:6789/0 conn(0x7f75041480b0 legacy=0x7f750415ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:20.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:20.140+0000 7f7509400640 1 -- 172.21.15.152:0/995399059 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f750406df00 con 0x7f750406f1c0 2024-04-20T11:31:20.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:20.140+0000 7f7508a00640 1 --2- 172.21.15.152:0/995399059 >> v2:172.21.15.175:3300/0 conn(0x7f7504147cd0 0x7f750406e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:31:20.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:20.142+0000 7f7509400640 1 --2- 172.21.15.152:0/995399059 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f750406f1c0 0x7f750406f630 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f750406ffc0 tx=0x7f74fc02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37d533d43d27757f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:20.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:20.142+0000 7f7503e00640 1 -- 172.21.15.152:0/995399059 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74fc02ca80 con 0x7f750406f1c0 2024-04-20T11:31:20.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:20.142+0000 7f7503e00640 1 -- 172.21.15.152:0/995399059 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74fc02cc20 con 0x7f750406f1c0 2024-04-20T11:31:20.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:20.142+0000 7f7503e00640 1 -- 172.21.15.152:0/995399059 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74fc0343f0 con 0x7f750406f1c0 2024-04-20T11:31:20.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:20.143+0000 7f7509e00640 1 -- 172.21.15.152:0/995399059 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f750406f1c0 msgr2=0x7f750406f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:20.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:20.143+0000 7f7509e00640 1 --2- 172.21.15.152:0/995399059 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f750406f1c0 0x7f750406f630 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f750406ffc0 tx=0x7f74fc02bee0 comp rx=0 tx=0).stop 2024-04-20T11:31:20.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:20.143+0000 7f7509e00640 1 -- 172.21.15.152:0/995399059 shutdown_connections 2024-04-20T11:31:20.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:20.143+0000 7f7509e00640 1 --2- 172.21.15.152:0/995399059 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f750406f1c0 0x7f750406f630 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:20.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:20.143+0000 7f7509e00640 1 --2- 172.21.15.152:0/995399059 >> v2:172.21.15.175:3300/0 conn(0x7f7504147cd0 0x7f750406e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:20.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:20.143+0000 7f7509e00640 1 -- 172.21.15.152:0/995399059 >> 172.21.15.152:0/995399059 conn(0x7f7504067ae0 msgr2=0x7f750406b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:20.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:20.143+0000 7f7509e00640 1 -- 172.21.15.152:0/995399059 shutdown_connections 2024-04-20T11:31:20.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:20.143+0000 7f7509e00640 1 -- 172.21.15.152:0/995399059 wait complete. 2024-04-20T11:31:20.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:20.145+0000 7f7509e00640 1 Processor -- start 2024-04-20T11:31:20.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:20.145+0000 7f7509e00640 1 -- start start 2024-04-20T11:31:20.146 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:20.146+0000 7f7509e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f750406f1c0 0x7f750413d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:20.146 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:20.146+0000 7f7509e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7504147cd0 0x7f7504143bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:20.146 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:20.146+0000 7f7509e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f75041480b0 0x7f7504145720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:20.146 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:20.146+0000 7f7509e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f750406ec40 con 0x7f750406f1c0 2024-04-20T11:31:20.146 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:20.146+0000 7f7509e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7504071ed0 con 0x7f7504147cd0 2024-04-20T11:31:20.146 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:20.146+0000 7f7509e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f750406edc0 con 0x7f75041480b0 2024-04-20T11:31:20.146 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:20.146+0000 7f7508a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f750406f1c0 0x7f750413d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:20.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:20.146+0000 7f7508a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f750406f1c0 0x7f750413d750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:55542/0 (socket says 172.21.15.152:55542) 2024-04-20T11:31:20.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:20.146+0000 7f7508a00640 1 -- 172.21.15.152:0/4006042060 learned_addr learned my addr 172.21.15.152:0/4006042060 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:20.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:20.146+0000 7f7509400640 1 --2- 172.21.15.152:0/4006042060 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f75041480b0 0x7f7504145720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:20.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:20.146+0000 7f7508a00640 1 -- 172.21.15.152:0/4006042060 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f75041480b0 msgr2=0x7f7504145720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:20.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:20.146+0000 7f7508a00640 1 --2- 172.21.15.152:0/4006042060 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f75041480b0 0x7f7504145720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:20.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:20.146+0000 7f7508a00640 1 -- 172.21.15.152:0/4006042060 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7504147cd0 msgr2=0x7f7504143bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:20.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:20.146+0000 7f7508a00640 1 --2- 172.21.15.152:0/4006042060 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7504147cd0 0x7f7504143bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:20.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:20.146+0000 7f7508a00640 1 -- 172.21.15.152:0/4006042060 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f750413dc90 con 0x7f750406f1c0 2024-04-20T11:31:20.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:20.147+0000 7f7508a00640 1 --2- 172.21.15.152:0/4006042060 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f750406f1c0 0x7f750413d750 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f74f400c7f0 tx=0x7f74f400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:20.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:20.147+0000 7f7501600640 1 -- 172.21.15.152:0/4006042060 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74f4015070 con 0x7f750406f1c0 2024-04-20T11:31:20.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:20.147+0000 7f7501600640 1 -- 172.21.15.152:0/4006042060 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74f4007820 con 0x7f750406f1c0 2024-04-20T11:31:20.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:20.147+0000 7f7501600640 1 -- 172.21.15.152:0/4006042060 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74f4007bd0 con 0x7f750406f1c0 2024-04-20T11:31:20.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:20.147+0000 7f7509e00640 1 -- 172.21.15.152:0/4006042060 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7504144680 con 0x7f750406f1c0 2024-04-20T11:31:20.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:20.147+0000 7f7509e00640 1 -- 172.21.15.152:0/4006042060 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7504144c40 con 0x7f750406f1c0 2024-04-20T11:31:20.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:20.148+0000 7f7509e00640 1 -- 172.21.15.152:0/4006042060 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7504075e60 con 0x7f750406f1c0 2024-04-20T11:31:20.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:20.149+0000 7f7501600640 1 -- 172.21.15.152:0/4006042060 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 10) ==== 91244+0+0 (secure 0 0 0) 0x7f74f4018070 con 0x7f750406f1c0 2024-04-20T11:31:20.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:20.150+0000 7f7501600640 1 --2- 172.21.15.152:0/4006042060 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f74e006cb30 0x7f74e006eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:20.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:20.150+0000 7f7501600640 1 -- 172.21.15.152:0/4006042060 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5709+0+0 (secure 0 0 0) 0x7f74f408c1b0 con 0x7f750406f1c0 2024-04-20T11:31:20.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:20.150+0000 7f74fbe00640 1 --2- 172.21.15.152:0/4006042060 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f74e006cb30 0x7f74e006eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:20.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:20.151+0000 7f74fbe00640 1 --2- 172.21.15.152:0/4006042060 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f74e006cb30 0x7f74e006eff0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f75041f8910 tx=0x7f74ec0079b0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:20.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:20.155+0000 7f7501600640 1 -- 172.21.15.152:0/4006042060 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f74f405a160 con 0x7f750406f1c0 2024-04-20T11:31:20.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:20.381+0000 7f7509e00640 1 -- 172.21.15.152:0/4006042060 --> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol99630", "target": ["mon-mgr", ""]}) -- 0x7f750415f1c0 con 0x7f74e006cb30 2024-04-20T11:31:20.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:20.386+0000 7f7501600640 1 -- 172.21.15.152:0/4006042060 <== mgr.4113 v2:172.21.15.152:6832/2285110240 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+30 (secure 0 0 0) 0x7f750415f1c0 con 0x7f74e006cb30 2024-04-20T11:31:20.386 INFO:teuthology.orchestra.run.smithi152.stdout:/volumes/_nogroup/subvol99630 2024-04-20T11:31:20.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:20.389+0000 7f7509e00640 1 -- 172.21.15.152:0/4006042060 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f74e006cb30 msgr2=0x7f74e006eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:20.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:20.389+0000 7f7509e00640 1 --2- 172.21.15.152:0/4006042060 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f74e006cb30 0x7f74e006eff0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f75041f8910 tx=0x7f74ec0079b0 comp rx=0 tx=0).stop 2024-04-20T11:31:20.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:20.389+0000 7f7509e00640 1 -- 172.21.15.152:0/4006042060 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f750406f1c0 msgr2=0x7f750413d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:20.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:20.389+0000 7f7509e00640 1 --2- 172.21.15.152:0/4006042060 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f750406f1c0 0x7f750413d750 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f74f400c7f0 tx=0x7f74f400ccb0 comp rx=0 tx=0).stop 2024-04-20T11:31:20.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:20.389+0000 7f7509e00640 1 -- 172.21.15.152:0/4006042060 shutdown_connections 2024-04-20T11:31:20.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:20.389+0000 7f7509e00640 1 --2- 172.21.15.152:0/4006042060 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f74e006cb30 0x7f74e006eff0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:20.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:20.389+0000 7f7509e00640 1 --2- 172.21.15.152:0/4006042060 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f75041480b0 0x7f7504145720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:20.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:20.389+0000 7f7509e00640 1 --2- 172.21.15.152:0/4006042060 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7504147cd0 0x7f7504143bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:20.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:20.389+0000 7f7509e00640 1 --2- 172.21.15.152:0/4006042060 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f750406f1c0 0x7f750413d750 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:20.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:20.389+0000 7f7509e00640 1 -- 172.21.15.152:0/4006042060 >> 172.21.15.152:0/4006042060 conn(0x7f7504067ae0 msgr2=0x7f750415dfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:20.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:20.389+0000 7f7509e00640 1 -- 172.21.15.152:0/4006042060 shutdown_connections 2024-04-20T11:31:20.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:20.389+0000 7f7509e00640 1 -- 172.21.15.152:0/4006042060 wait complete. 2024-04-20T11:31:20.409 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:31:20.409 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_legacy/02de0fac64e401ea88645bda9cb34688.meta) 2024-04-20T11:31:22.169 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:31:22.177 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:31:24.359 INFO:teuthology.orchestra.run.smithi152.stdout:[GLOBAL] 2024-04-20T11:31:24.359 INFO:teuthology.orchestra.run.smithi152.stdout:version = 1 2024-04-20T11:31:24.359 INFO:teuthology.orchestra.run.smithi152.stdout:type = subvolume 2024-04-20T11:31:24.359 INFO:teuthology.orchestra.run.smithi152.stdout:path = /volumes/_nogroup/subvol99630 2024-04-20T11:31:24.360 INFO:teuthology.orchestra.run.smithi152.stdout:state = complete 2024-04-20T11:31:24.360 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:31:24.365 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume authorize cephfs subvol99630 alice 2024-04-20T11:31:24.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:24.576+0000 7f4d27c00640 1 Processor -- start 2024-04-20T11:31:24.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:24.576+0000 7f4d27c00640 1 -- start start 2024-04-20T11:31:24.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:24.577+0000 7f4d27c00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f4d2015e910 0x7f4d200d3870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:24.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:24.577+0000 7f4d27c00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f4d200d22d0 0x7f4d200cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:24.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:24.577+0000 7f4d27c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4d200cfa10 0x7f4d200b6180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:24.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:24.578+0000 7f4d27c00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f4d200d40e0 con 0x7f4d2015e910 2024-04-20T11:31:24.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:24.578+0000 7f4d27c00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f4d200d43c0 con 0x7f4d200d22d0 2024-04-20T11:31:24.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:24.578+0000 7f4d27c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4d200d46a0 con 0x7f4d200cfa10 2024-04-20T11:31:24.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:24.578+0000 7f4d25200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f4d2015e910 0x7f4d200d3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:24.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:24.578+0000 7f4d25200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f4d2015e910 0x7f4d200d3870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:55548/0 (socket says 172.21.15.152:55548) 2024-04-20T11:31:24.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:24.578+0000 7f4d25200640 1 -- 172.21.15.152:0/1175195317 learned_addr learned my addr 172.21.15.152:0/1175195317 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:24.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:24.578+0000 7f4d25200640 1 -- 172.21.15.152:0/1175195317 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4d200cfa10 msgr2=0x7f4d200b6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:24.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:24.578+0000 7f4d25c00640 1 --2- 172.21.15.152:0/1175195317 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4d200cfa10 0x7f4d200b6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:24.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:24.578+0000 7f4d1fe00640 1 --2- 172.21.15.152:0/1175195317 >> v2:172.21.15.175:3300/0 conn(0x7f4d200d22d0 0x7f4d200cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:24.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:24.578+0000 7f4d25200640 1 --2- 172.21.15.152:0/1175195317 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4d200cfa10 0x7f4d200b6180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:24.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:24.578+0000 7f4d25200640 1 -- 172.21.15.152:0/1175195317 >> v2:172.21.15.175:3300/0 conn(0x7f4d200d22d0 msgr2=0x7f4d200cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:24.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:24.578+0000 7f4d25200640 1 --2- 172.21.15.152:0/1175195317 >> v2:172.21.15.175:3300/0 conn(0x7f4d200d22d0 0x7f4d200cf4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:24.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:24.578+0000 7f4d25200640 1 -- 172.21.15.152:0/1175195317 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d200d4980 con 0x7f4d2015e910 2024-04-20T11:31:24.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:24.578+0000 7f4d1fe00640 1 --2- 172.21.15.152:0/1175195317 >> v2:172.21.15.175:3300/0 conn(0x7f4d200d22d0 0x7f4d200cf4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:31:24.580 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:24.579+0000 7f4d25200640 1 --2- 172.21.15.152:0/1175195317 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d2015e910 0x7f4d200d3870 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f4d200d20a0 tx=0x7f4d1402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf666369387a8a84 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:24.580 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:24.580+0000 7f4d1f400640 1 -- 172.21.15.152:0/1175195317 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d14035020 con 0x7f4d2015e910 2024-04-20T11:31:24.580 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:24.580+0000 7f4d1f400640 1 -- 172.21.15.152:0/1175195317 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d14031870 con 0x7f4d2015e910 2024-04-20T11:31:24.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:24.580+0000 7f4d1f400640 1 -- 172.21.15.152:0/1175195317 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d14031c40 con 0x7f4d2015e910 2024-04-20T11:31:24.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:24.580+0000 7f4d27c00640 1 -- 172.21.15.152:0/1175195317 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d2015e910 msgr2=0x7f4d200d3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:24.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:24.580+0000 7f4d27c00640 1 --2- 172.21.15.152:0/1175195317 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d2015e910 0x7f4d200d3870 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f4d200d20a0 tx=0x7f4d1402b910 comp rx=0 tx=0).stop 2024-04-20T11:31:24.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:24.581+0000 7f4d27c00640 1 -- 172.21.15.152:0/1175195317 shutdown_connections 2024-04-20T11:31:24.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:24.581+0000 7f4d27c00640 1 --2- 172.21.15.152:0/1175195317 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4d200cfa10 0x7f4d200b6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:24.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:24.581+0000 7f4d27c00640 1 --2- 172.21.15.152:0/1175195317 >> v2:172.21.15.175:3300/0 conn(0x7f4d200d22d0 0x7f4d200cf4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:24.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:24.581+0000 7f4d27c00640 1 --2- 172.21.15.152:0/1175195317 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d2015e910 0x7f4d200d3870 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:24.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:24.581+0000 7f4d27c00640 1 -- 172.21.15.152:0/1175195317 >> 172.21.15.152:0/1175195317 conn(0x7f4d20067ae0 msgr2=0x7f4d20067ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:24.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:24.581+0000 7f4d27c00640 1 -- 172.21.15.152:0/1175195317 shutdown_connections 2024-04-20T11:31:24.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:24.581+0000 7f4d27c00640 1 -- 172.21.15.152:0/1175195317 wait complete. 2024-04-20T11:31:24.582 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:24.582+0000 7f4d27c00640 1 Processor -- start 2024-04-20T11:31:24.582 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:24.583+0000 7f4d27c00640 1 -- start start 2024-04-20T11:31:24.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:24.583+0000 7f4d27c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d200cfa10 0x7f4d200c7260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:24.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:24.583+0000 7f4d27c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4d200d22d0 0x7f4d200c7e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:24.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:24.583+0000 7f4d27c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d2015e910 0x7f4d200c8360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:24.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:24.583+0000 7f4d27c00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f4d200cfdf0 con 0x7f4d2015e910 2024-04-20T11:31:24.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:24.583+0000 7f4d27c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4d200d3db0 con 0x7f4d200cfa10 2024-04-20T11:31:24.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:24.584+0000 7f4d27c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4d2015ecf0 con 0x7f4d200d22d0 2024-04-20T11:31:24.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:24.584+0000 7f4d25c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d2015e910 0x7f4d200c8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:24.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:24.584+0000 7f4d25c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d2015e910 0x7f4d200c8360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:55552/0 (socket says 172.21.15.152:55552) 2024-04-20T11:31:24.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:24.584+0000 7f4d25c00640 1 -- 172.21.15.152:0/1739030929 learned_addr learned my addr 172.21.15.152:0/1739030929 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:24.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:24.584+0000 7f4d25200640 1 --2- 172.21.15.152:0/1739030929 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d200cfa10 0x7f4d200c7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:24.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:24.584+0000 7f4d25c00640 1 -- 172.21.15.152:0/1739030929 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4d200d22d0 msgr2=0x7f4d200c7e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:24.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:24.584+0000 7f4d25c00640 1 --2- 172.21.15.152:0/1739030929 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4d200d22d0 0x7f4d200c7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:24.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:24.584+0000 7f4d25c00640 1 -- 172.21.15.152:0/1739030929 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d200cfa10 msgr2=0x7f4d200c7260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:24.585 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:24.584+0000 7f4d25c00640 1 --2- 172.21.15.152:0/1739030929 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d200cfa10 0x7f4d200c7260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:24.585 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:24.584+0000 7f4d25c00640 1 -- 172.21.15.152:0/1739030929 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d20156610 con 0x7f4d2015e910 2024-04-20T11:31:24.585 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:24.585+0000 7f4d25c00640 1 --2- 172.21.15.152:0/1739030929 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d2015e910 0x7f4d200c8360 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f4d1800cda0 tx=0x7f4d1800a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:24.585 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:24.585+0000 7f4d1cc00640 1 -- 172.21.15.152:0/1739030929 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d18003b90 con 0x7f4d2015e910 2024-04-20T11:31:24.586 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:24.585+0000 7f4d1cc00640 1 -- 172.21.15.152:0/1739030929 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d18003d30 con 0x7f4d2015e910 2024-04-20T11:31:24.586 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:24.585+0000 7f4d1cc00640 1 -- 172.21.15.152:0/1739030929 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d18004240 con 0x7f4d2015e910 2024-04-20T11:31:24.586 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:24.585+0000 7f4d27c00640 1 -- 172.21.15.152:0/1739030929 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d201568f0 con 0x7f4d2015e910 2024-04-20T11:31:24.586 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:24.585+0000 7f4d27c00640 1 -- 172.21.15.152:0/1739030929 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d20156eb0 con 0x7f4d2015e910 2024-04-20T11:31:24.586 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:24.586+0000 7f4d27c00640 1 -- 172.21.15.152:0/1739030929 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d20068a60 con 0x7f4d2015e910 2024-04-20T11:31:24.586 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:24.587+0000 7f4d1cc00640 1 -- 172.21.15.152:0/1739030929 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 10) ==== 91244+0+0 (secure 0 0 0) 0x7f4d180043e0 con 0x7f4d2015e910 2024-04-20T11:31:24.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:24.588+0000 7f4d1cc00640 1 --2- 172.21.15.152:0/1739030929 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f4d0006cb30 0x7f4d0006eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:24.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:24.588+0000 7f4d1cc00640 1 -- 172.21.15.152:0/1739030929 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5709+0+0 (secure 0 0 0) 0x7f4d1808cc70 con 0x7f4d2015e910 2024-04-20T11:31:24.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:24.588+0000 7f4d25200640 1 --2- 172.21.15.152:0/1739030929 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f4d0006cb30 0x7f4d0006eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:24.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:24.589+0000 7f4d25200640 1 --2- 172.21.15.152:0/1739030929 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f4d0006cb30 0x7f4d0006eff0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f4d14036000 tx=0x7f4d1403c040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:24.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:24.593+0000 7f4d1cc00640 1 -- 172.21.15.152:0/1739030929 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f4d1805aba0 con 0x7f4d2015e910 2024-04-20T11:31:24.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:24.819+0000 7f4d27c00640 1 -- 172.21.15.152:0/1739030929 --> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] -- mgr_command(tid 0: {"prefix": "fs subvolume authorize", "vol_name": "cephfs", "sub_name": "subvol99630", "auth_id": "alice", "target": ["mon-mgr", ""]}) -- 0x7f4d200682d0 con 0x7f4d0006cb30 2024-04-20T11:31:24.835 INFO:tasks.ceph.mgr.y.smithi152.stderr:Creating meta for ID alice with tenant None 2024-04-20T11:31:24.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:24.920+0000 7f4d1cc00640 1 -- 172.21.15.152:0/1739030929 <== mgr.4113 v2:172.21.15.152:6832/2285110240 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4d200682d0 con 0x7f4d0006cb30 2024-04-20T11:31:24.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:24.924+0000 7f4d27c00640 1 -- 172.21.15.152:0/1739030929 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f4d0006cb30 msgr2=0x7f4d0006eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:24.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:24.924+0000 7f4d27c00640 1 --2- 172.21.15.152:0/1739030929 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f4d0006cb30 0x7f4d0006eff0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f4d14036000 tx=0x7f4d1403c040 comp rx=0 tx=0).stop 2024-04-20T11:31:24.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:24.924+0000 7f4d27c00640 1 -- 172.21.15.152:0/1739030929 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d2015e910 msgr2=0x7f4d200c8360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:24.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:24.924+0000 7f4d27c00640 1 --2- 172.21.15.152:0/1739030929 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d2015e910 0x7f4d200c8360 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f4d1800cda0 tx=0x7f4d1800a660 comp rx=0 tx=0).stop 2024-04-20T11:31:24.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:24.924+0000 7f4d27c00640 1 -- 172.21.15.152:0/1739030929 shutdown_connections 2024-04-20T11:31:24.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:24.924+0000 7f4d27c00640 1 --2- 172.21.15.152:0/1739030929 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f4d0006cb30 0x7f4d0006eff0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:24.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:24.924+0000 7f4d27c00640 1 --2- 172.21.15.152:0/1739030929 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d2015e910 0x7f4d200c8360 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:24.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:24.924+0000 7f4d27c00640 1 --2- 172.21.15.152:0/1739030929 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4d200d22d0 0x7f4d200c7e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:24.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:24.924+0000 7f4d27c00640 1 --2- 172.21.15.152:0/1739030929 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d200cfa10 0x7f4d200c7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:24.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:24.924+0000 7f4d27c00640 1 -- 172.21.15.152:0/1739030929 >> 172.21.15.152:0/1739030929 conn(0x7f4d20067ae0 msgr2=0x7f4d2015dd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:24.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:24.924+0000 7f4d27c00640 1 -- 172.21.15.152:0/1739030929 shutdown_connections 2024-04-20T11:31:24.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:24.924+0000 7f4d27c00640 1 -- 172.21.15.152:0/1739030929 wait complete. 2024-04-20T11:31:24.945 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth get client.alice --format=json-pretty 2024-04-20T11:31:25.139 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.139+0000 7fc754c00640 1 Processor -- start 2024-04-20T11:31:25.139 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.139+0000 7fc754c00640 1 -- start start 2024-04-20T11:31:25.140 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.141+0000 7fc754c00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fc7500b75e0 0x7fc7500b7a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:25.140 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.141+0000 7fc754c00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7500b63b0 con 0x7fc7500d0b30 2024-04-20T11:31:25.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.141+0000 7fc754c00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7500cd250 con 0x7fc75015c860 2024-04-20T11:31:25.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.141+0000 7fc754c00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fc7500cd020 con 0x7fc7500b75e0 2024-04-20T11:31:25.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.141+0000 7fc74d400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fc7500d0b30 0x7fc7500d0f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:39734/0 (socket says 172.21.15.152:39734) 2024-04-20T11:31:25.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.141+0000 7fc74d400640 1 -- 172.21.15.152:0/2686653515 learned_addr learned my addr 172.21.15.152:0/2686653515 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:31:25.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.142+0000 7fc74e800640 1 --2- 172.21.15.152:0/2686653515 >> v2:172.21.15.175:3300/0 conn(0x7fc7500b75e0 0x7fc7500b7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:25.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.142+0000 7fc74ca00640 1 -- 172.21.15.152:0/2686653515 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 349108197 0 0) 0x7fc7500b63b0 con 0x7fc7500d0b30 2024-04-20T11:31:25.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.142+0000 7fc74ca00640 1 -- 172.21.15.152:0/2686653515 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc730003520 con 0x7fc7500d0b30 2024-04-20T11:31:25.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.142+0000 7fc74ca00640 1 -- 172.21.15.152:0/2686653515 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1073125919 0 0) 0x7fc730003520 con 0x7fc7500d0b30 2024-04-20T11:31:25.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.142+0000 7fc74ca00640 1 -- 172.21.15.152:0/2686653515 >> v2:172.21.15.175:3300/0 conn(0x7fc7500b75e0 msgr2=0x7fc7500b7a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:25.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.142+0000 7fc74ca00640 1 --2- 172.21.15.152:0/2686653515 >> v2:172.21.15.175:3300/0 conn(0x7fc7500b75e0 0x7fc7500b7a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:25.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.142+0000 7fc74ca00640 1 -- 172.21.15.152:0/2686653515 >> v1:172.21.15.175:6789/0 conn(0x7fc75015c860 legacy=0x7fc75009b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:25.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.142+0000 7fc74ca00640 1 -- 172.21.15.152:0/2686653515 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7500cd530 con 0x7fc7500d0b30 2024-04-20T11:31:25.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.142+0000 7fc74ca00640 1 -- 172.21.15.152:0/2686653515 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7fc7380019f0 con 0x7fc7500d0b30 2024-04-20T11:31:25.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.142+0000 7fc74ca00640 1 -- 172.21.15.152:0/2686653515 >> v1:172.21.15.152:6789/0 conn(0x7fc7500d0b30 legacy=0x7fc7500d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:25.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.142+0000 7fc74ca00640 1 --2- 172.21.15.152:0/2686653515 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc7300039b0 0x7fc730003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:25.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.142+0000 7fc74ca00640 1 --2- 172.21.15.152:0/2686653515 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc730004800 0x7fc730004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:25.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.142+0000 7fc74ca00640 1 --2- 172.21.15.152:0/2686653515 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc7300051c0 0x7fc73000fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:25.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.142+0000 7fc74ca00640 1 -- 172.21.15.152:0/2686653515 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fc730010440 con 0x7fc730004800 2024-04-20T11:31:25.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.142+0000 7fc74ca00640 1 -- 172.21.15.152:0/2686653515 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc7300105f0 con 0x7fc7300051c0 2024-04-20T11:31:25.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.142+0000 7fc74ca00640 1 -- 172.21.15.152:0/2686653515 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc7300107f0 con 0x7fc7300039b0 2024-04-20T11:31:25.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.143+0000 7fc74d400640 1 --2- 172.21.15.152:0/2686653515 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc730004800 0x7fc730004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:25.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.143+0000 7fc74e800640 1 --2- 172.21.15.152:0/2686653515 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc7300051c0 0x7fc73000fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:25.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.143+0000 7fc74de00640 1 --2- 172.21.15.152:0/2686653515 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc7300039b0 0x7fc730003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:25.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.143+0000 7fc74d400640 1 -- 172.21.15.152:0/2686653515 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc7300039b0 msgr2=0x7fc730003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:25.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.143+0000 7fc74d400640 1 --2- 172.21.15.152:0/2686653515 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc7300039b0 0x7fc730003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:25.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.143+0000 7fc74d400640 1 -- 172.21.15.152:0/2686653515 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc7300051c0 msgr2=0x7fc73000fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:25.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.143+0000 7fc74d400640 1 --2- 172.21.15.152:0/2686653515 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc7300051c0 0x7fc73000fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:25.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.143+0000 7fc74d400640 1 -- 172.21.15.152:0/2686653515 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc730003520 con 0x7fc730004800 2024-04-20T11:31:25.145 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.143+0000 7fc74e800640 1 --2- 172.21.15.152:0/2686653515 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc7300051c0 0x7fc73000fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:31:25.145 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.143+0000 7fc74e800640 1 -- 172.21.15.152:0/2686653515 reap_dead start 2024-04-20T11:31:25.145 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.143+0000 7fc74de00640 1 --2- 172.21.15.152:0/2686653515 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc7300039b0 0x7fc730003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:31:25.146 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.144+0000 7fc74d400640 1 --2- 172.21.15.152:0/2686653515 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc730004800 0x7fc730004c80 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fc73800a850 tx=0x7fc730005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e07919fedb7557b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:25.146 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.144+0000 7fc74ca00640 1 -- 172.21.15.152:0/2686653515 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc73800a4a0 con 0x7fc730004800 2024-04-20T11:31:25.146 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.145+0000 7fc74ca00640 1 -- 172.21.15.152:0/2686653515 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc738034650 con 0x7fc730004800 2024-04-20T11:31:25.146 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.145+0000 7fc74ca00640 1 -- 172.21.15.152:0/2686653515 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc738034a80 con 0x7fc730004800 2024-04-20T11:31:25.146 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.145+0000 7fc754c00640 1 -- 172.21.15.152:0/2686653515 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc730004800 msgr2=0x7fc730004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:25.146 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.145+0000 7fc754c00640 1 --2- 172.21.15.152:0/2686653515 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc730004800 0x7fc730004c80 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fc73800a850 tx=0x7fc730005830 comp rx=0 tx=0).stop 2024-04-20T11:31:25.146 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.145+0000 7fc754c00640 1 -- 172.21.15.152:0/2686653515 shutdown_connections 2024-04-20T11:31:25.146 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.145+0000 7fc754c00640 1 --2- 172.21.15.152:0/2686653515 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc730004800 0x7fc730004c80 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:25.146 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.145+0000 7fc754c00640 1 -- 172.21.15.152:0/2686653515 >> 172.21.15.152:0/2686653515 conn(0x7fc7500b4040 msgr2=0x7fc7500b4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:25.146 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.145+0000 7fc754c00640 1 -- 172.21.15.152:0/2686653515 shutdown_connections 2024-04-20T11:31:25.146 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.145+0000 7fc754c00640 1 -- 172.21.15.152:0/2686653515 wait complete. 2024-04-20T11:31:25.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.147+0000 7fc754c00640 1 Processor -- start 2024-04-20T11:31:25.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.147+0000 7fc754c00640 1 -- start start 2024-04-20T11:31:25.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.148+0000 7fc754c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc730004800 0x7fc750148500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:25.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.148+0000 7fc754c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc7500cc3e0 0x7fc7500713b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:25.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.148+0000 7fc754c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc7500718f0 0x7fc75014ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:25.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.148+0000 7fc754c00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fc7500cddd0 con 0x7fc7500718f0 2024-04-20T11:31:25.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.148+0000 7fc754c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc7500cdaf0 con 0x7fc730004800 2024-04-20T11:31:25.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.148+0000 7fc754c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc7500d03b0 con 0x7fc7500cc3e0 2024-04-20T11:31:25.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.148+0000 7fc74e800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc7500718f0 0x7fc75014ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:25.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.148+0000 7fc74e800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc7500718f0 0x7fc75014ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:55558/0 (socket says 172.21.15.152:55558) 2024-04-20T11:31:25.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.148+0000 7fc74e800640 1 -- 172.21.15.152:0/1205119657 learned_addr learned my addr 172.21.15.152:0/1205119657 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:25.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.149+0000 7fc74de00640 1 --2- 172.21.15.152:0/1205119657 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc730004800 0x7fc750148500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:25.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.149+0000 7fc74d400640 1 --2- 172.21.15.152:0/1205119657 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc7500cc3e0 0x7fc7500713b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:25.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.149+0000 7fc74e800640 1 -- 172.21.15.152:0/1205119657 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc7500cc3e0 msgr2=0x7fc7500713b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:25.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.149+0000 7fc74e800640 1 --2- 172.21.15.152:0/1205119657 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc7500cc3e0 0x7fc7500713b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:25.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.149+0000 7fc74e800640 1 -- 172.21.15.152:0/1205119657 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc730004800 msgr2=0x7fc750148500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:25.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.149+0000 7fc74e800640 1 --2- 172.21.15.152:0/1205119657 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc730004800 0x7fc750148500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:25.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.149+0000 7fc74e800640 1 -- 172.21.15.152:0/1205119657 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc75014b9b0 con 0x7fc7500718f0 2024-04-20T11:31:25.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.149+0000 7fc74e800640 1 --2- 172.21.15.152:0/1205119657 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc7500718f0 0x7fc75014ac10 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fc75006fcc0 tx=0x7fc748010c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:25.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.149+0000 7fc746000640 1 -- 172.21.15.152:0/1205119657 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc748018070 con 0x7fc7500718f0 2024-04-20T11:31:25.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.150+0000 7fc746000640 1 -- 172.21.15.152:0/1205119657 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc748004230 con 0x7fc7500718f0 2024-04-20T11:31:25.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.150+0000 7fc754c00640 1 -- 172.21.15.152:0/1205119657 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7500cb900 con 0x7fc7500718f0 2024-04-20T11:31:25.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.150+0000 7fc746000640 1 -- 172.21.15.152:0/1205119657 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc74800f040 con 0x7fc7500718f0 2024-04-20T11:31:25.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.150+0000 7fc754c00640 1 -- 172.21.15.152:0/1205119657 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7500cb5d0 con 0x7fc7500718f0 2024-04-20T11:31:25.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.151+0000 7fc754c00640 1 -- 172.21.15.152:0/1205119657 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7500689f0 con 0x7fc7500718f0 2024-04-20T11:31:25.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.152+0000 7fc746000640 1 -- 172.21.15.152:0/1205119657 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 10) ==== 91244+0+0 (secure 0 0 0) 0x7fc74801b070 con 0x7fc7500718f0 2024-04-20T11:31:25.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.153+0000 7fc746000640 1 --2- 172.21.15.152:0/1205119657 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc72c06cb60 0x7fc72c06f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:25.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.153+0000 7fc746000640 1 -- 172.21.15.152:0/1205119657 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5709+0+0 (secure 0 0 0) 0x7fc74808be50 con 0x7fc7500718f0 2024-04-20T11:31:25.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.153+0000 7fc74de00640 1 --2- 172.21.15.152:0/1205119657 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc72c06cb60 0x7fc72c06f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:25.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.153+0000 7fc74de00640 1 --2- 172.21.15.152:0/1205119657 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc72c06cb60 0x7fc72c06f020 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fc750076cd0 tx=0x7fc740009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:25.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.157+0000 7fc746000640 1 -- 172.21.15.152:0/1205119657 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fc748059e80 con 0x7fc7500718f0 2024-04-20T11:31:25.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.429+0000 7fc754c00640 1 -- 172.21.15.152:0/1205119657 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "auth get", "entity": "client.alice", "format": "json-pretty"} v 0) -- 0x7fc7500b4ca0 con 0x7fc7500718f0 2024-04-20T11:31:25.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.430+0000 7fc746000640 1 -- 172.21.15.152:0/1205119657 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "auth get", "entity": "client.alice", "format": "json-pretty"}]=0 v12) ==== 107+0+280 (secure 0 0 0) 0x7fc748059780 con 0x7fc7500718f0 2024-04-20T11:31:25.429 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:31:25.429 INFO:teuthology.orchestra.run.smithi152.stdout:[ 2024-04-20T11:31:25.429 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:31:25.429 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "client.alice", 2024-04-20T11:31:25.430 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQCMpyNmP7+RMxAAw/JhZWU6U/0MpqThf8E3TA==", 2024-04-20T11:31:25.430 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:31:25.430 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow rw path=/volumes/_nogroup/subvol99630", 2024-04-20T11:31:25.430 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow r", 2024-04-20T11:31:25.430 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow rw pool=cephfs_data" 2024-04-20T11:31:25.430 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:31:25.430 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:31:25.430 INFO:teuthology.orchestra.run.smithi152.stdout:] 2024-04-20T11:31:25.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.432+0000 7fc754c00640 1 -- 172.21.15.152:0/1205119657 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc72c06cb60 msgr2=0x7fc72c06f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:25.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.432+0000 7fc754c00640 1 --2- 172.21.15.152:0/1205119657 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc72c06cb60 0x7fc72c06f020 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fc750076cd0 tx=0x7fc740009040 comp rx=0 tx=0).stop 2024-04-20T11:31:25.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.432+0000 7fc754c00640 1 -- 172.21.15.152:0/1205119657 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc7500718f0 msgr2=0x7fc75014ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:25.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.432+0000 7fc754c00640 1 --2- 172.21.15.152:0/1205119657 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc7500718f0 0x7fc75014ac10 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fc75006fcc0 tx=0x7fc748010c20 comp rx=0 tx=0).stop 2024-04-20T11:31:25.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.432+0000 7fc754c00640 1 -- 172.21.15.152:0/1205119657 shutdown_connections 2024-04-20T11:31:25.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.432+0000 7fc754c00640 1 --2- 172.21.15.152:0/1205119657 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc72c06cb60 0x7fc72c06f020 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:25.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.432+0000 7fc754c00640 1 --2- 172.21.15.152:0/1205119657 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc7500718f0 0x7fc75014ac10 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:25.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.432+0000 7fc754c00640 1 --2- 172.21.15.152:0/1205119657 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc7500cc3e0 0x7fc7500713b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:25.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.432+0000 7fc754c00640 1 --2- 172.21.15.152:0/1205119657 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc730004800 0x7fc750148500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:25.433 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.432+0000 7fc754c00640 1 -- 172.21.15.152:0/1205119657 >> 172.21.15.152:0/1205119657 conn(0x7fc7500b4040 msgr2=0x7fc7500b5a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:25.433 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.432+0000 7fc754c00640 1 -- 172.21.15.152:0/1205119657 shutdown_connections 2024-04-20T11:31:25.433 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.433+0000 7fc754c00640 1 -- 172.21.15.152:0/1205119657 wait complete. 2024-04-20T11:31:25.451 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol99630 2024-04-20T11:31:25.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.656+0000 7f8955e00640 1 Processor -- start 2024-04-20T11:31:25.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.656+0000 7f8955e00640 1 -- start start 2024-04-20T11:31:25.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.658+0000 7f8955e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8950146000 0x7f89500b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:25.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.658+0000 7f8955e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f89500cd8c0 0x7f89500b25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:25.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.658+0000 7f8955e00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89501463e0 con 0x7f89500b2170 2024-04-20T11:31:25.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.658+0000 7f8955e00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f89500b4750 con 0x7f89500cd8c0 2024-04-20T11:31:25.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.658+0000 7f8955e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f89500b3e20 con 0x7f8950146000 2024-04-20T11:31:25.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.658+0000 7f894f400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f89500cd8c0 0x7f89500b25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:25.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.658+0000 7f894f400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f89500cd8c0 0x7f89500b25f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:55574/0 (socket says 172.21.15.152:55574) 2024-04-20T11:31:25.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.658+0000 7f894f400640 1 -- 172.21.15.152:0/4033966308 learned_addr learned my addr 172.21.15.152:0/4033966308 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:25.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.659+0000 7f894ea00640 1 --2- 172.21.15.152:0/4033966308 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8950146000 0x7f89500b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:25.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.659+0000 7f894f400640 1 -- 172.21.15.152:0/4033966308 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8950146000 msgr2=0x7f89500b4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:25.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.659+0000 7f894f400640 1 --2- 172.21.15.152:0/4033966308 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8950146000 0x7f89500b4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:25.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.659+0000 7f894f400640 1 -- 172.21.15.152:0/4033966308 >> v1:172.21.15.175:6789/0 conn(0x7f89500b2170 legacy=0x7f89500cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:25.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.659+0000 7f894f400640 1 -- 172.21.15.152:0/4033966308 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89500b4100 con 0x7f89500cd8c0 2024-04-20T11:31:25.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.660+0000 7f894f400640 1 --2- 172.21.15.152:0/4033966308 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f89500cd8c0 0x7f89500b25f0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f8948009e50 tx=0x7f894802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddd715663023172c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:25.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.661+0000 7f894e000640 1 -- 172.21.15.152:0/4033966308 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f894802ca70 con 0x7f89500cd8c0 2024-04-20T11:31:25.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.661+0000 7f894e000640 1 -- 172.21.15.152:0/4033966308 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f894802cc10 con 0x7f89500cd8c0 2024-04-20T11:31:25.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.661+0000 7f894e000640 1 -- 172.21.15.152:0/4033966308 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89480343f0 con 0x7f89500cd8c0 2024-04-20T11:31:25.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.661+0000 7f8955e00640 1 -- 172.21.15.152:0/4033966308 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f89500cd8c0 msgr2=0x7f89500b25f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:25.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.661+0000 7f8955e00640 1 --2- 172.21.15.152:0/4033966308 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f89500cd8c0 0x7f89500b25f0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f8948009e50 tx=0x7f894802bed0 comp rx=0 tx=0).stop 2024-04-20T11:31:25.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.662+0000 7f8955e00640 1 -- 172.21.15.152:0/4033966308 shutdown_connections 2024-04-20T11:31:25.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.662+0000 7f8955e00640 1 --2- 172.21.15.152:0/4033966308 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f89500cd8c0 0x7f89500b25f0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:25.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.662+0000 7f8955e00640 1 --2- 172.21.15.152:0/4033966308 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8950146000 0x7f89500b4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:25.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.662+0000 7f8955e00640 1 -- 172.21.15.152:0/4033966308 >> 172.21.15.152:0/4033966308 conn(0x7f895006b980 msgr2=0x7f895006bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:25.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.662+0000 7f8955e00640 1 -- 172.21.15.152:0/4033966308 shutdown_connections 2024-04-20T11:31:25.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.662+0000 7f8955e00640 1 -- 172.21.15.152:0/4033966308 wait complete. 2024-04-20T11:31:25.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.664+0000 7f8955e00640 1 Processor -- start 2024-04-20T11:31:25.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.664+0000 7f8955e00640 1 -- start start 2024-04-20T11:31:25.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.665+0000 7f8955e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f89500b2170 0x7f89500b0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:25.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.665+0000 7f8955e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f89500cd8c0 0x7f89500ab300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:25.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.665+0000 7f8955e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8950146000 0x7f89500cf330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:25.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.665+0000 7f8955e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f895015ce70 con 0x7f89500cd8c0 2024-04-20T11:31:25.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.665+0000 7f8955e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f895015ccf0 con 0x7f89500b2170 2024-04-20T11:31:25.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.665+0000 7f8955e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f895015cff0 con 0x7f8950146000 2024-04-20T11:31:25.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.665+0000 7f8947e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f89500cd8c0 0x7f89500ab300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:25.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.666+0000 7f8947e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f89500cd8c0 0x7f89500ab300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:55576/0 (socket says 172.21.15.152:55576) 2024-04-20T11:31:25.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.666+0000 7f8947e00640 1 -- 172.21.15.152:0/1269187294 learned_addr learned my addr 172.21.15.152:0/1269187294 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:25.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.666+0000 7f894ea00640 1 --2- 172.21.15.152:0/1269187294 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f89500b2170 0x7f89500b0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:25.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.666+0000 7f8947e00640 1 -- 172.21.15.152:0/1269187294 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8950146000 msgr2=0x7f89500cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:25.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.666+0000 7f894f400640 1 --2- 172.21.15.152:0/1269187294 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8950146000 0x7f89500cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:25.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.666+0000 7f8947e00640 1 --2- 172.21.15.152:0/1269187294 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8950146000 0x7f89500cf330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:25.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.666+0000 7f8947e00640 1 -- 172.21.15.152:0/1269187294 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f89500b2170 msgr2=0x7f89500b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:25.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.666+0000 7f8947e00640 1 --2- 172.21.15.152:0/1269187294 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f89500b2170 0x7f89500b0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:25.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.666+0000 7f8947e00640 1 -- 172.21.15.152:0/1269187294 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89500a9b20 con 0x7f89500cd8c0 2024-04-20T11:31:25.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.666+0000 7f894ea00640 1 --2- 172.21.15.152:0/1269187294 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f89500b2170 0x7f89500b0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:31:25.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.666+0000 7f8947e00640 1 --2- 172.21.15.152:0/1269187294 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f89500cd8c0 0x7f89500ab300 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f893800c400 tx=0x7f893800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:25.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.667+0000 7f8947400640 1 -- 172.21.15.152:0/1269187294 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89380052d0 con 0x7f89500cd8c0 2024-04-20T11:31:25.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.667+0000 7f8947400640 1 -- 172.21.15.152:0/1269187294 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8938005490 con 0x7f89500cd8c0 2024-04-20T11:31:25.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.667+0000 7f8947400640 1 -- 172.21.15.152:0/1269187294 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8938007260 con 0x7f89500cd8c0 2024-04-20T11:31:25.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.667+0000 7f8955e00640 1 -- 172.21.15.152:0/1269187294 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89500a9e60 con 0x7f89500cd8c0 2024-04-20T11:31:25.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.667+0000 7f8955e00640 1 -- 172.21.15.152:0/1269187294 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89500aa8e0 con 0x7f89500cd8c0 2024-04-20T11:31:25.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.668+0000 7f8955e00640 1 -- 172.21.15.152:0/1269187294 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89500689f0 con 0x7f89500cd8c0 2024-04-20T11:31:25.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.669+0000 7f8947400640 1 -- 172.21.15.152:0/1269187294 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 10) ==== 91244+0+0 (secure 0 0 0) 0x7f8938007400 con 0x7f89500cd8c0 2024-04-20T11:31:25.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.670+0000 7f8947400640 1 --2- 172.21.15.152:0/1269187294 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f892c06cb30 0x7f892c06eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:25.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.670+0000 7f8947400640 1 -- 172.21.15.152:0/1269187294 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5709+0+0 (secure 0 0 0) 0x7f893808b500 con 0x7f89500cd8c0 2024-04-20T11:31:25.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.670+0000 7f894ea00640 1 --2- 172.21.15.152:0/1269187294 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f892c06cb30 0x7f892c06eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:25.670 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.671+0000 7f894ea00640 1 --2- 172.21.15.152:0/1269187294 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f892c06cb30 0x7f892c06eff0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f8940006fd0 tx=0x7f8940008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:25.674 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.675+0000 7f8947400640 1 -- 172.21.15.152:0/1269187294 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f89500a9e60 con 0x7f89500cd8c0 2024-04-20T11:31:25.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.907+0000 7f8955e00640 1 -- 172.21.15.152:0/1269187294 --> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol99630", "target": ["mon-mgr", ""]}) -- 0x7f89500b1a60 con 0x7f892c06cb30 2024-04-20T11:31:25.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.943+0000 7f8947400640 1 -- 172.21.15.152:0/1269187294 <== mgr.4113 v2:172.21.15.152:6832/2285110240 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f89500b1a60 con 0x7f892c06cb30 2024-04-20T11:31:25.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.946+0000 7f8955e00640 1 -- 172.21.15.152:0/1269187294 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f892c06cb30 msgr2=0x7f892c06eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:25.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.946+0000 7f8955e00640 1 --2- 172.21.15.152:0/1269187294 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f892c06cb30 0x7f892c06eff0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f8940006fd0 tx=0x7f8940008040 comp rx=0 tx=0).stop 2024-04-20T11:31:25.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.946+0000 7f8955e00640 1 -- 172.21.15.152:0/1269187294 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f89500cd8c0 msgr2=0x7f89500ab300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:25.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.946+0000 7f8955e00640 1 --2- 172.21.15.152:0/1269187294 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f89500cd8c0 0x7f89500ab300 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f893800c400 tx=0x7f893800c8c0 comp rx=0 tx=0).stop 2024-04-20T11:31:25.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.947+0000 7f8955e00640 1 -- 172.21.15.152:0/1269187294 shutdown_connections 2024-04-20T11:31:25.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.947+0000 7f8955e00640 1 --2- 172.21.15.152:0/1269187294 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f892c06cb30 0x7f892c06eff0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:25.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.947+0000 7f8955e00640 1 --2- 172.21.15.152:0/1269187294 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8950146000 0x7f89500cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:25.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.947+0000 7f8955e00640 1 --2- 172.21.15.152:0/1269187294 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f89500cd8c0 0x7f89500ab300 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:25.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.947+0000 7f8955e00640 1 --2- 172.21.15.152:0/1269187294 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f89500b2170 0x7f89500b0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:25.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.947+0000 7f8955e00640 1 -- 172.21.15.152:0/1269187294 >> 172.21.15.152:0/1269187294 conn(0x7f895006b980 msgr2=0x7f8950142aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:25.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.947+0000 7f8955e00640 1 -- 172.21.15.152:0/1269187294 shutdown_connections 2024-04-20T11:31:25.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:25.947+0000 7f8955e00640 1 -- 172.21.15.152:0/1269187294 wait complete. 2024-04-20T11:31:25.950 INFO:tasks.ceph.mgr.y.smithi152.stderr:2024-04-20T11:31:25.951+0000 7fd167c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:31:25.951 INFO:tasks.ceph.mgr.y.smithi152.stderr:2024-04-20T11:31:25.951+0000 7fd167c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:31:25.951 INFO:tasks.ceph.mgr.y.smithi152.stderr:2024-04-20T11:31:25.951+0000 7fd167c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:31:25.951 INFO:tasks.ceph.mgr.y.smithi152.stderr:2024-04-20T11:31:25.951+0000 7fd167c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:31:25.951 INFO:tasks.ceph.mgr.y.smithi152.stderr:2024-04-20T11:31:25.951+0000 7fd167c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:31:25.969 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol77311 2024-04-20T11:31:25.974 INFO:tasks.ceph.mgr.y.smithi152.stderr:2024-04-20T11:31:25.973+0000 7fd169000640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:31:25.975 INFO:tasks.ceph.mgr.y.smithi152.stderr:2024-04-20T11:31:25.973+0000 7fd169000640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:31:25.975 INFO:tasks.ceph.mgr.y.smithi152.stderr:2024-04-20T11:31:25.973+0000 7fd169000640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:31:25.975 INFO:tasks.ceph.mgr.y.smithi152.stderr:2024-04-20T11:31:25.973+0000 7fd169000640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:31:25.975 INFO:tasks.ceph.mgr.y.smithi152.stderr:2024-04-20T11:31:25.973+0000 7fd169000640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:31:26.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:26.150+0000 7fee03c00640 1 Processor -- start 2024-04-20T11:31:26.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:26.150+0000 7fee03c00640 1 -- start start 2024-04-20T11:31:26.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:26.152+0000 7fee03c00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fedfc146000 0x7fedfc0b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:26.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:26.152+0000 7fee03c00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fedfc0cd8c0 0x7fedfc0b2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:26.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:26.152+0000 7fee03c00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fedfc1463e0 con 0x7fedfc0b2170 2024-04-20T11:31:26.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:26.152+0000 7fee03c00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fedfc0cdca0 con 0x7fedfc146000 2024-04-20T11:31:26.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:26.152+0000 7fee03c00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fedfc0b4750 con 0x7fedfc0cd8c0 2024-04-20T11:31:26.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:26.152+0000 7fee01200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fedfc146000 0x7fedfc0b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:26.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:26.152+0000 7fedfbe00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fedfc0b2170 0x7fedfc0cd1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:54776/0 (socket says 172.21.15.152:54776) 2024-04-20T11:31:26.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:26.152+0000 7fedfbe00640 1 -- 172.21.15.152:0/1480420828 learned_addr learned my addr 172.21.15.152:0/1480420828 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:31:26.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:26.152+0000 7fee01200640 1 -- 172.21.15.152:0/1480420828 >> v2:172.21.15.175:3300/0 conn(0x7fedfc0cd8c0 msgr2=0x7fedfc0b2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:26.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:26.152+0000 7fee01c00640 1 --2- 172.21.15.152:0/1480420828 >> v2:172.21.15.175:3300/0 conn(0x7fedfc0cd8c0 0x7fedfc0b2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:26.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:26.152+0000 7fee01200640 1 --2- 172.21.15.152:0/1480420828 >> v2:172.21.15.175:3300/0 conn(0x7fedfc0cd8c0 0x7fedfc0b2620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:26.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:26.152+0000 7fee01200640 1 -- 172.21.15.152:0/1480420828 >> v1:172.21.15.152:6789/0 conn(0x7fedfc0b2170 legacy=0x7fedfc0cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:26.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:26.152+0000 7fee01200640 1 -- 172.21.15.152:0/1480420828 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedfc0b3e20 con 0x7fedfc146000 2024-04-20T11:31:26.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:26.152+0000 7fedfb400640 1 -- 172.21.15.152:0/1480420828 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 779708279 0 0) 0x7fedfc1463e0 con 0x7fedfc0b2170 2024-04-20T11:31:26.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:26.154+0000 7fee01200640 1 --2- 172.21.15.152:0/1480420828 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fedfc146000 0x7fedfc0b4be0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fedfc1601b0 tx=0x7fedf002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dfef9cd4588cebc server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:26.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:26.154+0000 7fedfb400640 1 -- 172.21.15.152:0/1480420828 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf00315a0 con 0x7fedfc146000 2024-04-20T11:31:26.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:26.154+0000 7fedfb400640 1 -- 172.21.15.152:0/1480420828 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fedf0037070 con 0x7fedfc146000 2024-04-20T11:31:26.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:26.154+0000 7fedfb400640 1 -- 172.21.15.152:0/1480420828 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf0031960 con 0x7fedfc146000 2024-04-20T11:31:26.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:26.154+0000 7fee03c00640 1 -- 172.21.15.152:0/1480420828 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fedfc146000 msgr2=0x7fedfc0b4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:26.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:26.154+0000 7fee03c00640 1 --2- 172.21.15.152:0/1480420828 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fedfc146000 0x7fedfc0b4be0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fedfc1601b0 tx=0x7fedf002b840 comp rx=0 tx=0).stop 2024-04-20T11:31:26.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:26.154+0000 7fee03c00640 1 -- 172.21.15.152:0/1480420828 shutdown_connections 2024-04-20T11:31:26.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:26.154+0000 7fee03c00640 1 --2- 172.21.15.152:0/1480420828 >> v2:172.21.15.175:3300/0 conn(0x7fedfc0cd8c0 0x7fedfc0b2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:26.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:26.154+0000 7fee03c00640 1 --2- 172.21.15.152:0/1480420828 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fedfc146000 0x7fedfc0b4be0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:26.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:26.154+0000 7fee03c00640 1 -- 172.21.15.152:0/1480420828 >> 172.21.15.152:0/1480420828 conn(0x7fedfc06b980 msgr2=0x7fedfc06bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:26.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:26.155+0000 7fee03c00640 1 -- 172.21.15.152:0/1480420828 shutdown_connections 2024-04-20T11:31:26.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:26.155+0000 7fee03c00640 1 -- 172.21.15.152:0/1480420828 wait complete. 2024-04-20T11:31:26.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:26.156+0000 7fee03c00640 1 Processor -- start 2024-04-20T11:31:26.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:26.156+0000 7fee03c00640 1 -- start start 2024-04-20T11:31:26.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:26.157+0000 7fee03c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fedfc0b2170 0x7fedfc06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:26.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:26.157+0000 7fee03c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fedfc0cd8c0 0x7fedfc0c9860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:26.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:26.157+0000 7fee03c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fedfc146000 0x7fedfc0c9da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:26.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:26.157+0000 7fee03c00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fedfc15ccf0 con 0x7fedfc0cd8c0 2024-04-20T11:31:26.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:26.157+0000 7fee03c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fedfc0b43e0 con 0x7fedfc0b2170 2024-04-20T11:31:26.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:26.157+0000 7fee03c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fedfc15ce70 con 0x7fedfc146000 2024-04-20T11:31:26.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:26.158+0000 7fedfbe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fedfc0cd8c0 0x7fedfc0c9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:26.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:26.158+0000 7fedfbe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fedfc0cd8c0 0x7fedfc0c9860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54314/0 (socket says 172.21.15.152:54314) 2024-04-20T11:31:26.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:26.158+0000 7fedfbe00640 1 -- 172.21.15.152:0/984324693 learned_addr learned my addr 172.21.15.152:0/984324693 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:26.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:26.158+0000 7fee01200640 1 --2- 172.21.15.152:0/984324693 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fedfc0b2170 0x7fedfc06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:26.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:26.158+0000 7fedfbe00640 1 -- 172.21.15.152:0/984324693 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fedfc146000 msgr2=0x7fedfc0c9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:26.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:26.158+0000 7fedfbe00640 1 --2- 172.21.15.152:0/984324693 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fedfc146000 0x7fedfc0c9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:26.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:26.158+0000 7fedfbe00640 1 -- 172.21.15.152:0/984324693 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fedfc0b2170 msgr2=0x7fedfc06f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:26.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:26.158+0000 7fedfbe00640 1 --2- 172.21.15.152:0/984324693 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fedfc0b2170 0x7fedfc06f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:26.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:26.158+0000 7fedfbe00640 1 -- 172.21.15.152:0/984324693 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedfc06f610 con 0x7fedfc0cd8c0 2024-04-20T11:31:26.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:26.158+0000 7fee01200640 1 --2- 172.21.15.152:0/984324693 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fedfc0b2170 0x7fedfc06f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:31:26.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:26.158+0000 7fedfbe00640 1 --2- 172.21.15.152:0/984324693 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fedfc0cd8c0 0x7fedfc0c9860 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fedec00c500 tx=0x7fedec00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:26.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:26.158+0000 7fedf8c00640 1 -- 172.21.15.152:0/984324693 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedec018070 con 0x7fedfc0cd8c0 2024-04-20T11:31:26.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:26.159+0000 7fedf8c00640 1 -- 172.21.15.152:0/984324693 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fedec007580 con 0x7fedfc0cd8c0 2024-04-20T11:31:26.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:26.159+0000 7fee03c00640 1 -- 172.21.15.152:0/984324693 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedfc06e030 con 0x7fedfc0cd8c0 2024-04-20T11:31:26.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:26.159+0000 7fedf8c00640 1 -- 172.21.15.152:0/984324693 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedec004a20 con 0x7fedfc0cd8c0 2024-04-20T11:31:26.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:26.159+0000 7fee03c00640 1 -- 172.21.15.152:0/984324693 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedfc06e5f0 con 0x7fedfc0cd8c0 2024-04-20T11:31:26.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:26.160+0000 7fee03c00640 1 -- 172.21.15.152:0/984324693 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fedfc0689f0 con 0x7fedfc0cd8c0 2024-04-20T11:31:26.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:26.161+0000 7fedf8c00640 1 -- 172.21.15.152:0/984324693 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 10) ==== 91244+0+0 (secure 0 0 0) 0x7fedec01b070 con 0x7fedfc0cd8c0 2024-04-20T11:31:26.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:26.161+0000 7fedf8c00640 1 --2- 172.21.15.152:0/984324693 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fede006cb30 0x7fede006eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:26.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:26.162+0000 7fedf8c00640 1 -- 172.21.15.152:0/984324693 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5709+0+0 (secure 0 0 0) 0x7fedec08b320 con 0x7fedfc0cd8c0 2024-04-20T11:31:26.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:26.162+0000 7fee01200640 1 --2- 172.21.15.152:0/984324693 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fede006cb30 0x7fede006eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:26.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:26.162+0000 7fee01200640 1 --2- 172.21.15.152:0/984324693 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fede006cb30 0x7fede006eff0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fedf0036000 tx=0x7fedf0008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:26.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:26.166+0000 7fedf8c00640 1 -- 172.21.15.152:0/984324693 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fedec0593e0 con 0x7fedfc0cd8c0 2024-04-20T11:31:26.395 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:26.396+0000 7fee03c00640 1 -- 172.21.15.152:0/984324693 --> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol77311", "target": ["mon-mgr", ""]}) -- 0x7fedfc0b0860 con 0x7fede006cb30 2024-04-20T11:31:26.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:26.428+0000 7fedf8c00640 1 -- 172.21.15.152:0/984324693 <== mgr.4113 v2:172.21.15.152:6832/2285110240 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fedfc0b0860 con 0x7fede006cb30 2024-04-20T11:31:26.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:26.432+0000 7fee03c00640 1 -- 172.21.15.152:0/984324693 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fede006cb30 msgr2=0x7fede006eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:26.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:26.432+0000 7fee03c00640 1 --2- 172.21.15.152:0/984324693 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fede006cb30 0x7fede006eff0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fedf0036000 tx=0x7fedf0008030 comp rx=0 tx=0).stop 2024-04-20T11:31:26.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:26.432+0000 7fee03c00640 1 -- 172.21.15.152:0/984324693 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fedfc0cd8c0 msgr2=0x7fedfc0c9860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:26.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:26.432+0000 7fee03c00640 1 --2- 172.21.15.152:0/984324693 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fedfc0cd8c0 0x7fedfc0c9860 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fedec00c500 tx=0x7fedec00c9c0 comp rx=0 tx=0).stop 2024-04-20T11:31:26.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:26.432+0000 7fee03c00640 1 -- 172.21.15.152:0/984324693 shutdown_connections 2024-04-20T11:31:26.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:26.432+0000 7fee03c00640 1 --2- 172.21.15.152:0/984324693 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fede006cb30 0x7fede006eff0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:26.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:26.432+0000 7fee03c00640 1 --2- 172.21.15.152:0/984324693 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fedfc146000 0x7fedfc0c9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:26.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:26.432+0000 7fee03c00640 1 --2- 172.21.15.152:0/984324693 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fedfc0cd8c0 0x7fedfc0c9860 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:26.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:26.432+0000 7fee03c00640 1 --2- 172.21.15.152:0/984324693 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fedfc0b2170 0x7fedfc06f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:26.433 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:26.432+0000 7fee03c00640 1 -- 172.21.15.152:0/984324693 >> 172.21.15.152:0/984324693 conn(0x7fedfc06b980 msgr2=0x7fedfc1452a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:26.433 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:26.432+0000 7fee03c00640 1 -- 172.21.15.152:0/984324693 shutdown_connections 2024-04-20T11:31:26.433 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:26.432+0000 7fee03c00640 1 -- 172.21.15.152:0/984324693 wait complete. 2024-04-20T11:31:26.452 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-04-20T11:31:26.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:26 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-20T11:31:26.570 INFO:teuthology.orchestra.run.smithi152.stdout:2 2024-04-20T11:31:26.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T11:31:26.580 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-20T11:31:26.580 DEBUG:teuthology.orchestra.run.smithi152:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-20T11:31:26.677 DEBUG:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-20T11:31:26.790 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-20T11:31:26.790 INFO:tasks.cephfs.mount:Terminating background process 2024-04-20T11:31:26.790 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-20T11:31:26.790 DEBUG:teuthology.orchestra.run.smithi152:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:31:26.847 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:31:27.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.048+0000 7f7a29800640 1 Processor -- start 2024-04-20T11:31:27.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.048+0000 7f7a29800640 1 -- start start 2024-04-20T11:31:27.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.050+0000 7f7a29800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f7a240726f0 0x7f7a240cd580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:27.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.050+0000 7f7a29800640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a24146340 con 0x7f7a24145f60 2024-04-20T11:31:27.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.050+0000 7f7a29800640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a24072d50 con 0x7f7a240cd0d0 2024-04-20T11:31:27.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.050+0000 7f7a29800640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f7a2406e9f0 con 0x7f7a240726f0 2024-04-20T11:31:27.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.050+0000 7f7a23400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f7a240726f0 0x7f7a240cd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:27.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.050+0000 7f7a23400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f7a240726f0 0x7f7a240cd580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54326/0 (socket says 172.21.15.152:54326) 2024-04-20T11:31:27.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.050+0000 7f7a23400640 1 -- 172.21.15.152:0/3363073908 learned_addr learned my addr 172.21.15.152:0/3363073908 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:27.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.050+0000 7f7a23400640 1 -- 172.21.15.152:0/3363073908 >> v1:172.21.15.175:6789/0 conn(0x7f7a240cd0d0 legacy=0x7f7a24071fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:27.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.051+0000 7f7a23400640 1 -- 172.21.15.152:0/3363073908 >> v1:172.21.15.152:6789/0 conn(0x7f7a24145f60 legacy=0x7f7a2406ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:27.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.051+0000 7f7a23400640 1 -- 172.21.15.152:0/3363073908 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a2406e0c0 con 0x7f7a240726f0 2024-04-20T11:31:27.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.052+0000 7f7a23400640 1 --2- 172.21.15.152:0/3363073908 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7a240726f0 0x7f7a240cd580 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c009e50 tx=0x7f7a1c02bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90a207aa38c4950a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:27.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.052+0000 7f7a22000640 1 -- 172.21.15.152:0/3363073908 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a1c02cb10 con 0x7f7a240726f0 2024-04-20T11:31:27.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.052+0000 7f7a22000640 1 -- 172.21.15.152:0/3363073908 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a1c02ccb0 con 0x7f7a240726f0 2024-04-20T11:31:27.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.052+0000 7f7a22000640 1 -- 172.21.15.152:0/3363073908 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a1c02ce60 con 0x7f7a240726f0 2024-04-20T11:31:27.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.052+0000 7f7a29800640 1 -- 172.21.15.152:0/3363073908 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7a240726f0 msgr2=0x7f7a240cd580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:27.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.052+0000 7f7a29800640 1 --2- 172.21.15.152:0/3363073908 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7a240726f0 0x7f7a240cd580 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c009e50 tx=0x7f7a1c02bf70 comp rx=0 tx=0).stop 2024-04-20T11:31:27.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.053+0000 7f7a29800640 1 -- 172.21.15.152:0/3363073908 shutdown_connections 2024-04-20T11:31:27.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.053+0000 7f7a29800640 1 --2- 172.21.15.152:0/3363073908 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7a240726f0 0x7f7a240cd580 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:27.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.053+0000 7f7a29800640 1 -- 172.21.15.152:0/3363073908 >> 172.21.15.152:0/3363073908 conn(0x7f7a24067cf0 msgr2=0x7f7a2406b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:27.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.053+0000 7f7a29800640 1 -- 172.21.15.152:0/3363073908 shutdown_connections 2024-04-20T11:31:27.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.053+0000 7f7a29800640 1 -- 172.21.15.152:0/3363073908 wait complete. 2024-04-20T11:31:27.054 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.054+0000 7f7a29800640 1 Processor -- start 2024-04-20T11:31:27.054 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.055+0000 7f7a29800640 1 -- start start 2024-04-20T11:31:27.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.055+0000 7f7a29800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7a240726f0 0x7f7a241386a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:27.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.055+0000 7f7a29800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7a240cd0d0 0x7f7a2415ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:27.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.055+0000 7f7a29800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a24145f60 0x7f7a24135b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:27.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.056+0000 7f7a29800640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f7a240cf290 con 0x7f7a240cd0d0 2024-04-20T11:31:27.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.056+0000 7f7a29800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7a2406e680 con 0x7f7a24145f60 2024-04-20T11:31:27.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.056+0000 7f7a29800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7a240cf410 con 0x7f7a240726f0 2024-04-20T11:31:27.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.056+0000 7f7a1be00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7a240cd0d0 0x7f7a2415ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:27.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.056+0000 7f7a1be00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7a240cd0d0 0x7f7a2415ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54330/0 (socket says 172.21.15.152:54330) 2024-04-20T11:31:27.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.056+0000 7f7a1be00640 1 -- 172.21.15.152:0/3764078572 learned_addr learned my addr 172.21.15.152:0/3764078572 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:27.057 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.056+0000 7f7a23400640 1 --2- 172.21.15.152:0/3764078572 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a24145f60 0x7f7a24135b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:27.057 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.056+0000 7f7a22a00640 1 --2- 172.21.15.152:0/3764078572 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7a240726f0 0x7f7a241386a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:27.057 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.056+0000 7f7a1be00640 1 -- 172.21.15.152:0/3764078572 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7a240726f0 msgr2=0x7f7a241386a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:27.057 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.057+0000 7f7a1be00640 1 --2- 172.21.15.152:0/3764078572 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7a240726f0 0x7f7a241386a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:27.057 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.057+0000 7f7a1be00640 1 -- 172.21.15.152:0/3764078572 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a24145f60 msgr2=0x7f7a24135b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:27.057 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.057+0000 7f7a1be00640 1 --2- 172.21.15.152:0/3764078572 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a24145f60 0x7f7a24135b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:27.057 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.057+0000 7f7a1be00640 1 -- 172.21.15.152:0/3764078572 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a24138f80 con 0x7f7a240cd0d0 2024-04-20T11:31:27.057 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.057+0000 7f7a1be00640 1 --2- 172.21.15.152:0/3764078572 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7a240cd0d0 0x7f7a2415ac00 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c00c430 tx=0x7f7a0c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:27.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.057+0000 7f7a1aa00640 1 -- 172.21.15.152:0/3764078572 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a0c011790 con 0x7f7a240cd0d0 2024-04-20T11:31:27.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.057+0000 7f7a1aa00640 1 -- 172.21.15.152:0/3764078572 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a0c017070 con 0x7f7a240cd0d0 2024-04-20T11:31:27.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.057+0000 7f7a1aa00640 1 -- 172.21.15.152:0/3764078572 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a0c011bb0 con 0x7f7a240cd0d0 2024-04-20T11:31:27.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.057+0000 7f7a29800640 1 -- 172.21.15.152:0/3764078572 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a24139260 con 0x7f7a240cd0d0 2024-04-20T11:31:27.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.057+0000 7f7a29800640 1 -- 172.21.15.152:0/3764078572 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a24139820 con 0x7f7a240cd0d0 2024-04-20T11:31:27.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.058+0000 7f7a29800640 1 -- 172.21.15.152:0/3764078572 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a240689f0 con 0x7f7a240cd0d0 2024-04-20T11:31:27.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.059+0000 7f7a1aa00640 1 -- 172.21.15.152:0/3764078572 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 10) ==== 91244+0+0 (secure 0 0 0) 0x7f7a0c01c460 con 0x7f7a240cd0d0 2024-04-20T11:31:27.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.060+0000 7f7a1aa00640 1 --2- 172.21.15.152:0/3764078572 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f7a0006cb60 0x7f7a0006f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:27.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.060+0000 7f7a1aa00640 1 -- 172.21.15.152:0/3764078572 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5709+0+0 (secure 0 0 0) 0x7f7a0c08bd40 con 0x7f7a240cd0d0 2024-04-20T11:31:27.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.061+0000 7f7a22a00640 1 --2- 172.21.15.152:0/3764078572 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f7a0006cb60 0x7f7a0006f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:27.061 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.061+0000 7f7a22a00640 1 --2- 172.21.15.152:0/3764078572 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f7a0006cb60 0x7f7a0006f020 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f7a14007fd0 tx=0x7f7a14009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:27.064 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.065+0000 7f7a1aa00640 1 -- 172.21.15.152:0/3764078572 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f7a0c059ed0 con 0x7f7a240cd0d0 2024-04-20T11:31:27.266 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.266+0000 7f7a29800640 1 -- 172.21.15.152:0/3764078572 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7a240af590 con 0x7f7a240cd0d0 2024-04-20T11:31:27.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.268+0000 7f7a1aa00640 1 -- 172.21.15.152:0/3764078572 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) ==== 74+0+16725 (secure 0 0 0) 0x7f7a0c0597d0 con 0x7f7a240cd0d0 2024-04-20T11:31:27.269 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:31:27.269 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":60,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:31:01.845268+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:30:58.087822+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-20T11:30:59.281767+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:31:27.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.272+0000 7f7a29800640 1 -- 172.21.15.152:0/3764078572 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f7a0006cb60 msgr2=0x7f7a0006f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:27.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.272+0000 7f7a29800640 1 --2- 172.21.15.152:0/3764078572 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f7a0006cb60 0x7f7a0006f020 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f7a14007fd0 tx=0x7f7a14009040 comp rx=0 tx=0).stop 2024-04-20T11:31:27.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.272+0000 7f7a29800640 1 -- 172.21.15.152:0/3764078572 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7a240cd0d0 msgr2=0x7f7a2415ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:27.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.272+0000 7f7a29800640 1 --2- 172.21.15.152:0/3764078572 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7a240cd0d0 0x7f7a2415ac00 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c00c430 tx=0x7f7a0c00c8f0 comp rx=0 tx=0).stop 2024-04-20T11:31:27.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.272+0000 7f7a29800640 1 -- 172.21.15.152:0/3764078572 shutdown_connections 2024-04-20T11:31:27.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.272+0000 7f7a29800640 1 --2- 172.21.15.152:0/3764078572 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f7a0006cb60 0x7f7a0006f020 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:27.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.272+0000 7f7a29800640 1 --2- 172.21.15.152:0/3764078572 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a24145f60 0x7f7a24135b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:27.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.272+0000 7f7a29800640 1 --2- 172.21.15.152:0/3764078572 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7a240cd0d0 0x7f7a2415ac00 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:27.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.272+0000 7f7a29800640 1 --2- 172.21.15.152:0/3764078572 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7a240726f0 0x7f7a241386a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:27.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.272+0000 7f7a29800640 1 -- 172.21.15.152:0/3764078572 >> 172.21.15.152:0/3764078572 conn(0x7f7a24067cf0 msgr2=0x7f7a240cf780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:27.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.272+0000 7f7a29800640 1 -- 172.21.15.152:0/3764078572 shutdown_connections 2024-04-20T11:31:27.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.272+0000 7f7a29800640 1 -- 172.21.15.152:0/3764078572 wait complete. 2024-04-20T11:31:27.291 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T11:26:47.246571+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T11:30:58.087822+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '59', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T11:30:59.281767+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '60', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T11:31:27.291 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T11:31:27.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.494+0000 7fb79ce00640 1 Processor -- start 2024-04-20T11:31:27.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.494+0000 7fb79ce00640 1 -- start start 2024-04-20T11:31:27.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.496+0000 7fb79ce00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb7980ceeb0 0x7fb7980d4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:27.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.496+0000 7fb79ce00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7980d2b70 con 0x7fb7980cf5b0 2024-04-20T11:31:27.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.496+0000 7fb79ce00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7980d2e50 con 0x7fb798160230 2024-04-20T11:31:27.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.496+0000 7fb79ce00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb7980d5420 con 0x7fb7980ceeb0 2024-04-20T11:31:27.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.496+0000 7fb797400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fb7980cf5b0 0x7fb7980cf990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:54788/0 (socket says 172.21.15.152:54788) 2024-04-20T11:31:27.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.496+0000 7fb797400640 1 -- 172.21.15.152:0/500402081 learned_addr learned my addr 172.21.15.152:0/500402081 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:31:27.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.497+0000 7fb796a00640 1 --2- 172.21.15.152:0/500402081 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb7980ceeb0 0x7fb7980d4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:27.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.497+0000 7fb796000640 1 -- 172.21.15.152:0/500402081 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3815020841 0 0) 0x7fb7980d2b70 con 0x7fb7980cf5b0 2024-04-20T11:31:27.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.497+0000 7fb796000640 1 -- 172.21.15.152:0/500402081 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb77c003520 con 0x7fb7980cf5b0 2024-04-20T11:31:27.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.497+0000 7fb796000640 1 -- 172.21.15.152:0/500402081 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3771158960 0 0) 0x7fb77c003520 con 0x7fb7980cf5b0 2024-04-20T11:31:27.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.497+0000 7fb796000640 1 -- 172.21.15.152:0/500402081 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb7980ceeb0 msgr2=0x7fb7980d4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:27.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.497+0000 7fb796000640 1 --2- 172.21.15.152:0/500402081 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb7980ceeb0 0x7fb7980d4ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:27.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.497+0000 7fb796000640 1 -- 172.21.15.152:0/500402081 >> v1:172.21.15.175:6789/0 conn(0x7fb798160230 legacy=0x7fb7981606e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:27.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.497+0000 7fb796000640 1 -- 172.21.15.152:0/500402081 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7980d3130 con 0x7fb7980cf5b0 2024-04-20T11:31:27.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.497+0000 7fb796000640 1 -- 172.21.15.152:0/500402081 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7fb7880019f0 con 0x7fb7980cf5b0 2024-04-20T11:31:27.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.497+0000 7fb796000640 1 -- 172.21.15.152:0/500402081 >> v1:172.21.15.152:6789/0 conn(0x7fb7980cf5b0 legacy=0x7fb7980cf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:27.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.497+0000 7fb796000640 1 --2- 172.21.15.152:0/500402081 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb77c0039e0 0x7fb77c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:27.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.497+0000 7fb796000640 1 --2- 172.21.15.152:0/500402081 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb77c004800 0x7fb77c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:27.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.497+0000 7fb796000640 1 --2- 172.21.15.152:0/500402081 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb7980ceeb0 0x7fb77c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:27.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.497+0000 7fb796000640 1 -- 172.21.15.152:0/500402081 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fb7980d5420 con 0x7fb77c004800 2024-04-20T11:31:27.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.497+0000 7fb796000640 1 -- 172.21.15.152:0/500402081 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb77c00bb00 con 0x7fb77c0039e0 2024-04-20T11:31:27.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.497+0000 7fb796000640 1 -- 172.21.15.152:0/500402081 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb77c00bd00 con 0x7fb7980ceeb0 2024-04-20T11:31:27.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.498+0000 7fb796a00640 1 --2- 172.21.15.152:0/500402081 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb77c004800 0x7fb77c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:27.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.498+0000 7fb797400640 1 --2- 172.21.15.152:0/500402081 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb77c0039e0 0x7fb77c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:27.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.498+0000 7fb796a00640 1 -- 172.21.15.152:0/500402081 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb7980ceeb0 msgr2=0x7fb77c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:27.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.498+0000 7fb797e00640 1 --2- 172.21.15.152:0/500402081 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb7980ceeb0 0x7fb77c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:27.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.498+0000 7fb796a00640 1 --2- 172.21.15.152:0/500402081 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb7980ceeb0 0x7fb77c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:27.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.498+0000 7fb796a00640 1 -- 172.21.15.152:0/500402081 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb77c0039e0 msgr2=0x7fb77c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:27.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.498+0000 7fb796a00640 1 --2- 172.21.15.152:0/500402081 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb77c0039e0 0x7fb77c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:27.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.498+0000 7fb796a00640 1 -- 172.21.15.152:0/500402081 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb77c003520 con 0x7fb77c004800 2024-04-20T11:31:27.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.498+0000 7fb797e00640 1 --2- 172.21.15.152:0/500402081 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb7980ceeb0 0x7fb77c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:31:27.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.499+0000 7fb796a00640 1 --2- 172.21.15.152:0/500402081 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb77c004800 0x7fb77c004c80 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fb780009a80 tx=0x7fb77c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e594dd3a0a405e65 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:27.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.500+0000 7fb796000640 1 -- 172.21.15.152:0/500402081 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb780009680 con 0x7fb77c004800 2024-04-20T11:31:27.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.500+0000 7fb796000640 1 -- 172.21.15.152:0/500402081 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb780005990 con 0x7fb77c004800 2024-04-20T11:31:27.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.500+0000 7fb796000640 1 -- 172.21.15.152:0/500402081 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb780005d80 con 0x7fb77c004800 2024-04-20T11:31:27.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.500+0000 7fb79ce00640 1 -- 172.21.15.152:0/500402081 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb77c004800 msgr2=0x7fb77c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:27.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.500+0000 7fb79ce00640 1 --2- 172.21.15.152:0/500402081 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb77c004800 0x7fb77c004c80 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fb780009a80 tx=0x7fb77c005830 comp rx=0 tx=0).stop 2024-04-20T11:31:27.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.500+0000 7fb797e00640 1 -- 172.21.15.152:0/500402081 reap_dead start 2024-04-20T11:31:27.502 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.500+0000 7fb79ce00640 1 -- 172.21.15.152:0/500402081 shutdown_connections 2024-04-20T11:31:27.502 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.500+0000 7fb79ce00640 1 -- 172.21.15.152:0/500402081 >> 172.21.15.152:0/500402081 conn(0x7fb79808bfd0 msgr2=0x7fb7980d4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:27.502 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.500+0000 7fb79ce00640 1 -- 172.21.15.152:0/500402081 shutdown_connections 2024-04-20T11:31:27.502 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.500+0000 7fb79ce00640 1 -- 172.21.15.152:0/500402081 wait complete. 2024-04-20T11:31:27.502 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.502+0000 7fb79ce00640 1 Processor -- start 2024-04-20T11:31:27.502 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.502+0000 7fb79ce00640 1 -- start start 2024-04-20T11:31:27.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.503+0000 7fb79ce00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb77c004800 0x7fb79815c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:27.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.503+0000 7fb79ce00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb798155d70 0x7fb7980cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:27.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.503+0000 7fb79ce00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb79815cf10 0x7fb7980cbf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:27.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.503+0000 7fb79ce00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fb7980d3c10 con 0x7fb77c004800 2024-04-20T11:31:27.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.503+0000 7fb79ce00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb7980d36f0 con 0x7fb798155d70 2024-04-20T11:31:27.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.503+0000 7fb79ce00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb7980d2790 con 0x7fb79815cf10 2024-04-20T11:31:27.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.504+0000 7fb797400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb77c004800 0x7fb79815c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:27.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.504+0000 7fb797400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb77c004800 0x7fb79815c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54348/0 (socket says 172.21.15.152:54348) 2024-04-20T11:31:27.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.504+0000 7fb797400640 1 -- 172.21.15.152:0/3931890550 learned_addr learned my addr 172.21.15.152:0/3931890550 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:27.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.504+0000 7fb796a00640 1 --2- 172.21.15.152:0/3931890550 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb798155d70 0x7fb7980cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:27.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.504+0000 7fb797400640 1 -- 172.21.15.152:0/3931890550 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb79815cf10 msgr2=0x7fb7980cbf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:27.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.504+0000 7fb797400640 1 --2- 172.21.15.152:0/3931890550 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb79815cf10 0x7fb7980cbf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:27.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.504+0000 7fb797400640 1 -- 172.21.15.152:0/3931890550 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb798155d70 msgr2=0x7fb7980cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:27.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.504+0000 7fb797400640 1 --2- 172.21.15.152:0/3931890550 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb798155d70 0x7fb7980cdfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:27.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.504+0000 7fb797400640 1 -- 172.21.15.152:0/3931890550 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7980c9730 con 0x7fb77c004800 2024-04-20T11:31:27.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.504+0000 7fb796a00640 1 --2- 172.21.15.152:0/3931890550 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb798155d70 0x7fb7980cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:31:27.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.504+0000 7fb797400640 1 --2- 172.21.15.152:0/3931890550 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb77c004800 0x7fb79815c9d0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fb788007ab0 tx=0x7fb78800e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:27.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.504+0000 7fb78f400640 1 -- 172.21.15.152:0/3931890550 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb788013420 con 0x7fb77c004800 2024-04-20T11:31:27.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.505+0000 7fb78f400640 1 -- 172.21.15.152:0/3931890550 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb788018070 con 0x7fb77c004800 2024-04-20T11:31:27.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.505+0000 7fb78f400640 1 -- 172.21.15.152:0/3931890550 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb788013800 con 0x7fb77c004800 2024-04-20T11:31:27.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.505+0000 7fb79ce00640 1 -- 172.21.15.152:0/3931890550 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7980c90e0 con 0x7fb77c004800 2024-04-20T11:31:27.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.505+0000 7fb79ce00640 1 -- 172.21.15.152:0/3931890550 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7980c9c80 con 0x7fb77c004800 2024-04-20T11:31:27.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.506+0000 7fb79ce00640 1 -- 172.21.15.152:0/3931890550 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb798075e60 con 0x7fb77c004800 2024-04-20T11:31:27.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.507+0000 7fb78f400640 1 -- 172.21.15.152:0/3931890550 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 10) ==== 91244+0+0 (secure 0 0 0) 0x7fb7880139a0 con 0x7fb77c004800 2024-04-20T11:31:27.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.508+0000 7fb78f400640 1 --2- 172.21.15.152:0/3931890550 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fb77406cb30 0x7fb77406eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:27.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.508+0000 7fb78f400640 1 -- 172.21.15.152:0/3931890550 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5709+0+0 (secure 0 0 0) 0x7fb78808c660 con 0x7fb77c004800 2024-04-20T11:31:27.508 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.508+0000 7fb796a00640 1 --2- 172.21.15.152:0/3931890550 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fb77406cb30 0x7fb77406eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:27.508 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.509+0000 7fb796a00640 1 --2- 172.21.15.152:0/3931890550 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fb77406cb30 0x7fb77406eff0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fb780035000 tx=0x7fb780030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:27.512 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.513+0000 7fb78f400640 1 -- 172.21.15.152:0/3931890550 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fb78805a7f0 con 0x7fb77c004800 2024-04-20T11:31:27.711 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.712+0000 7fb79ce00640 1 -- 172.21.15.152:0/3931890550 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb7980cf390 con 0x7fb77c004800 2024-04-20T11:31:27.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.713+0000 7fb78f400640 1 -- 172.21.15.152:0/3931890550 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v60) ==== 93+0+10 (secure 0 0 0) 0x7fb78805a0f0 con 0x7fb77c004800 2024-04-20T11:31:27.712 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 1 2024-04-20T11:31:27.715 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.715+0000 7fb79ce00640 1 -- 172.21.15.152:0/3931890550 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fb77406cb30 msgr2=0x7fb77406eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:27.715 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.715+0000 7fb79ce00640 1 --2- 172.21.15.152:0/3931890550 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fb77406cb30 0x7fb77406eff0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fb780035000 tx=0x7fb780030000 comp rx=0 tx=0).stop 2024-04-20T11:31:27.716 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.715+0000 7fb79ce00640 1 -- 172.21.15.152:0/3931890550 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb77c004800 msgr2=0x7fb79815c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:27.716 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.715+0000 7fb79ce00640 1 --2- 172.21.15.152:0/3931890550 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb77c004800 0x7fb79815c9d0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fb788007ab0 tx=0x7fb78800e530 comp rx=0 tx=0).stop 2024-04-20T11:31:27.716 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.716+0000 7fb79ce00640 1 -- 172.21.15.152:0/3931890550 shutdown_connections 2024-04-20T11:31:27.716 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.716+0000 7fb79ce00640 1 --2- 172.21.15.152:0/3931890550 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fb77406cb30 0x7fb77406eff0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:27.716 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.716+0000 7fb79ce00640 1 --2- 172.21.15.152:0/3931890550 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb79815cf10 0x7fb7980cbf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:27.716 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.716+0000 7fb79ce00640 1 --2- 172.21.15.152:0/3931890550 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb798155d70 0x7fb7980cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:27.716 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.716+0000 7fb79ce00640 1 --2- 172.21.15.152:0/3931890550 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb77c004800 0x7fb79815c9d0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:27.716 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.716+0000 7fb79ce00640 1 -- 172.21.15.152:0/3931890550 >> 172.21.15.152:0/3931890550 conn(0x7fb79808bfd0 msgr2=0x7fb79806df10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:27.716 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.716+0000 7fb79ce00640 1 -- 172.21.15.152:0/3931890550 shutdown_connections 2024-04-20T11:31:27.716 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.716+0000 7fb79ce00640 1 -- 172.21.15.152:0/3931890550 wait complete. 2024-04-20T11:31:27.734 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T11:31:27.931 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.932+0000 7f21efc00640 1 Processor -- start 2024-04-20T11:31:27.931 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.932+0000 7f21efc00640 1 -- start start 2024-04-20T11:31:27.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.934+0000 7f21efc00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f21e80cd8c0 0x7f21e80b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:27.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.934+0000 7f21efc00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21e81463e0 con 0x7f21e80b2880 2024-04-20T11:31:27.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.934+0000 7f21efc00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21e80b3e20 con 0x7f21e8146000 2024-04-20T11:31:27.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.934+0000 7f21efc00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f21e80b4750 con 0x7f21e80cd8c0 2024-04-20T11:31:27.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.934+0000 7f21edc00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f21e80cd8c0 0x7f21e80b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:27.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.934+0000 7f21e7e00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f21e80b2880 0x7f21e80cd1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:54798/0 (socket says 172.21.15.152:54798) 2024-04-20T11:31:27.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.934+0000 7f21edc00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f21e80cd8c0 0x7f21e80b4be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54358/0 (socket says 172.21.15.152:54358) 2024-04-20T11:31:27.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.934+0000 7f21e7e00640 1 -- 172.21.15.152:0/2789662202 learned_addr learned my addr 172.21.15.152:0/2789662202 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:31:27.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.935+0000 7f21edc00640 1 -- 172.21.15.152:0/2789662202 >> v1:172.21.15.175:6789/0 conn(0x7f21e8146000 legacy=0x7f21e80b2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:27.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.935+0000 7f21edc00640 1 -- 172.21.15.152:0/2789662202 >> v1:172.21.15.152:6789/0 conn(0x7f21e80b2880 legacy=0x7f21e80cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:27.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.935+0000 7f21edc00640 1 -- 172.21.15.152:0/2789662202 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21e80b4100 con 0x7f21e80cd8c0 2024-04-20T11:31:27.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.936+0000 7f21edc00640 1 --2- 172.21.15.152:0/2789662202 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f21e80cd8c0 0x7f21e80b4be0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f21e0009d90 tx=0x7f21e002be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dc4ba54a63976f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:27.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.936+0000 7f21e7400640 1 -- 172.21.15.152:0/2789662202 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e0039070 con 0x7f21e80cd8c0 2024-04-20T11:31:27.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.936+0000 7f21e7400640 1 -- 172.21.15.152:0/2789662202 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21e002cbd0 con 0x7f21e80cd8c0 2024-04-20T11:31:27.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.936+0000 7f21e7400640 1 -- 172.21.15.152:0/2789662202 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e0034440 con 0x7f21e80cd8c0 2024-04-20T11:31:27.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.937+0000 7f21efc00640 1 -- 172.21.15.152:0/2789662202 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f21e80cd8c0 msgr2=0x7f21e80b4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:27.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.937+0000 7f21efc00640 1 --2- 172.21.15.152:0/2789662202 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f21e80cd8c0 0x7f21e80b4be0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f21e0009d90 tx=0x7f21e002be20 comp rx=0 tx=0).stop 2024-04-20T11:31:27.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.937+0000 7f21efc00640 1 -- 172.21.15.152:0/2789662202 shutdown_connections 2024-04-20T11:31:27.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.937+0000 7f21efc00640 1 --2- 172.21.15.152:0/2789662202 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f21e80cd8c0 0x7f21e80b4be0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:27.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.937+0000 7f21efc00640 1 -- 172.21.15.152:0/2789662202 >> 172.21.15.152:0/2789662202 conn(0x7f21e806b980 msgr2=0x7f21e806bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:27.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.937+0000 7f21efc00640 1 -- 172.21.15.152:0/2789662202 shutdown_connections 2024-04-20T11:31:27.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.937+0000 7f21efc00640 1 -- 172.21.15.152:0/2789662202 wait complete. 2024-04-20T11:31:27.938 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.939+0000 7f21efc00640 1 Processor -- start 2024-04-20T11:31:27.938 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.939+0000 7f21efc00640 1 -- start start 2024-04-20T11:31:27.939 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.940+0000 7f21efc00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f21e80b2880 0x7f21e813b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:27.939 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.940+0000 7f21efc00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f21e80cd8c0 0x7f21e813dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:27.939 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.940+0000 7f21efc00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f21e8146000 0x7f21e8139e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:27.939 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.940+0000 7f21efc00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f21e815ce70 con 0x7f21e80b2880 2024-04-20T11:31:27.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.940+0000 7f21efc00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f21e815ccf0 con 0x7f21e8146000 2024-04-20T11:31:27.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.940+0000 7f21efc00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f21e815cff0 con 0x7f21e80cd8c0 2024-04-20T11:31:27.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.940+0000 7f21ed200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f21e80b2880 0x7f21e813b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:27.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.940+0000 7f21ed200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f21e80b2880 0x7f21e813b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54364/0 (socket says 172.21.15.152:54364) 2024-04-20T11:31:27.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.940+0000 7f21ed200640 1 -- 172.21.15.152:0/1338818697 learned_addr learned my addr 172.21.15.152:0/1338818697 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:27.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.940+0000 7f21edc00640 1 --2- 172.21.15.152:0/1338818697 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f21e8146000 0x7f21e8139e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:27.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.940+0000 7f21ed200640 1 -- 172.21.15.152:0/1338818697 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f21e80cd8c0 msgr2=0x7f21e813dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:27.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.940+0000 7f21ed200640 1 --2- 172.21.15.152:0/1338818697 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f21e80cd8c0 0x7f21e813dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:27.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.940+0000 7f21ed200640 1 -- 172.21.15.152:0/1338818697 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f21e8146000 msgr2=0x7f21e8139e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:27.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.940+0000 7f21ed200640 1 --2- 172.21.15.152:0/1338818697 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f21e8146000 0x7f21e8139e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:27.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.940+0000 7f21ed200640 1 -- 172.21.15.152:0/1338818697 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21e813a340 con 0x7f21e80b2880 2024-04-20T11:31:27.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.941+0000 7f21ed200640 1 --2- 172.21.15.152:0/1338818697 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f21e80b2880 0x7f21e813b710 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f21dc00b3f0 tx=0x7f21dc00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:27.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.941+0000 7f21e4c00640 1 -- 172.21.15.152:0/1338818697 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21dc015020 con 0x7f21e80b2880 2024-04-20T11:31:27.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.941+0000 7f21e4c00640 1 -- 172.21.15.152:0/1338818697 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21dc011830 con 0x7f21e80b2880 2024-04-20T11:31:27.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.941+0000 7f21e4c00640 1 -- 172.21.15.152:0/1338818697 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21dc011c20 con 0x7f21e80b2880 2024-04-20T11:31:27.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.941+0000 7f21efc00640 1 -- 172.21.15.152:0/1338818697 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21e813e280 con 0x7f21e80b2880 2024-04-20T11:31:27.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.941+0000 7f21efc00640 1 -- 172.21.15.152:0/1338818697 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21e813e840 con 0x7f21e80b2880 2024-04-20T11:31:27.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.943+0000 7f21efc00640 1 -- 172.21.15.152:0/1338818697 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21e80689f0 con 0x7f21e80b2880 2024-04-20T11:31:27.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.943+0000 7f21e4c00640 1 -- 172.21.15.152:0/1338818697 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 10) ==== 91244+0+0 (secure 0 0 0) 0x7f21dc018020 con 0x7f21e80b2880 2024-04-20T11:31:27.944 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.944+0000 7f21e4c00640 1 --2- 172.21.15.152:0/1338818697 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f21c806cb60 0x7f21c806f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:27.944 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.944+0000 7f21e4c00640 1 -- 172.21.15.152:0/1338818697 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5709+0+0 (secure 0 0 0) 0x7f21dc08bbd0 con 0x7f21e80b2880 2024-04-20T11:31:27.944 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.945+0000 7f21e7e00640 1 --2- 172.21.15.152:0/1338818697 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f21c806cb60 0x7f21c806f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:27.945 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.945+0000 7f21e7e00640 1 --2- 172.21.15.152:0/1338818697 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f21c806cb60 0x7f21c806f020 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f21e81f57a0 tx=0x7f21d4007a60 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:27.948 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:27.949+0000 7f21e4c00640 1 -- 172.21.15.152:0/1338818697 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f21dc01f030 con 0x7f21e80b2880 2024-04-20T11:31:28.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.042+0000 7f21e4c00640 1 -- 172.21.15.152:0/1338818697 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mgrmap(e 11) ==== 91346+0+0 (secure 0 0 0) 0x7f21dc055b70 con 0x7f21e80b2880 2024-04-20T11:31:28.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.152+0000 7f21efc00640 1 -- 172.21.15.152:0/1338818697 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f21e813ea40 con 0x7f21e80b2880 2024-04-20T11:31:28.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.153+0000 7f21e4c00640 1 -- 172.21.15.152:0/1338818697 <== mon.0 v2:172.21.15.152:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v60) ==== 104+0+11 (secure 0 0 0) 0x7f21dc059c00 con 0x7f21e80b2880 2024-04-20T11:31:28.153 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 32 2024-04-20T11:31:28.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.156+0000 7f21efc00640 1 -- 172.21.15.152:0/1338818697 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f21c806cb60 msgr2=0x7f21c806f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:28.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.156+0000 7f21efc00640 1 --2- 172.21.15.152:0/1338818697 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f21c806cb60 0x7f21c806f020 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f21e81f57a0 tx=0x7f21d4007a60 comp rx=0 tx=0).stop 2024-04-20T11:31:28.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.156+0000 7f21efc00640 1 -- 172.21.15.152:0/1338818697 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f21e80b2880 msgr2=0x7f21e813b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:28.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.156+0000 7f21efc00640 1 --2- 172.21.15.152:0/1338818697 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f21e80b2880 0x7f21e813b710 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f21dc00b3f0 tx=0x7f21dc00b8b0 comp rx=0 tx=0).stop 2024-04-20T11:31:28.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.156+0000 7f21efc00640 1 -- 172.21.15.152:0/1338818697 shutdown_connections 2024-04-20T11:31:28.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.156+0000 7f21efc00640 1 --2- 172.21.15.152:0/1338818697 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f21c806cb60 0x7f21c806f020 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:28.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.156+0000 7f21efc00640 1 --2- 172.21.15.152:0/1338818697 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f21e8146000 0x7f21e8139e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:28.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.156+0000 7f21efc00640 1 --2- 172.21.15.152:0/1338818697 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f21e80cd8c0 0x7f21e813dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:28.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.156+0000 7f21efc00640 1 --2- 172.21.15.152:0/1338818697 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f21e80b2880 0x7f21e813b710 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:28.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.156+0000 7f21efc00640 1 -- 172.21.15.152:0/1338818697 >> 172.21.15.152:0/1338818697 conn(0x7f21e806b980 msgr2=0x7f21e815ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:28.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.156+0000 7f21efc00640 1 -- 172.21.15.152:0/1338818697 shutdown_connections 2024-04-20T11:31:28.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.156+0000 7f21efc00640 1 -- 172.21.15.152:0/1338818697 wait complete. 2024-04-20T11:31:28.174 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T11:31:28.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.381+0000 7fe467000640 1 Processor -- start 2024-04-20T11:31:28.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.381+0000 7fe467000640 1 -- start start 2024-04-20T11:31:28.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.383+0000 7fe467000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe46015b510 0x7fe46015fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:28.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.383+0000 7fe467000640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe46015f1c0 con 0x7fe4600b2880 2024-04-20T11:31:28.383 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.383+0000 7fe467000640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe46006f670 con 0x7fe4600cfc40 2024-04-20T11:31:28.383 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.383+0000 7fe467000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe46006f420 con 0x7fe46015b510 2024-04-20T11:31:28.383 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.383+0000 7fe465000640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fe4600b2880 0x7fe4600b4800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:54812/0 (socket says 172.21.15.152:54812) 2024-04-20T11:31:28.383 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.383+0000 7fe465000640 1 -- 172.21.15.152:0/1938181384 learned_addr learned my addr 172.21.15.152:0/1938181384 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:31:28.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.383+0000 7fe45fe00640 1 --2- 172.21.15.152:0/1938181384 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe46015b510 0x7fe46015fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:28.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.384+0000 7fe45ea00640 1 -- 172.21.15.152:0/1938181384 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 222355357 0 0) 0x7fe46015f1c0 con 0x7fe4600b2880 2024-04-20T11:31:28.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.384+0000 7fe45ea00640 1 -- 172.21.15.152:0/1938181384 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe444003520 con 0x7fe4600b2880 2024-04-20T11:31:28.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.384+0000 7fe45ea00640 1 -- 172.21.15.152:0/1938181384 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2602350380 0 0) 0x7fe444003520 con 0x7fe4600b2880 2024-04-20T11:31:28.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.384+0000 7fe45ea00640 1 -- 172.21.15.152:0/1938181384 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe46015b510 msgr2=0x7fe46015fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:28.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.384+0000 7fe45ea00640 1 --2- 172.21.15.152:0/1938181384 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe46015b510 0x7fe46015fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:28.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.384+0000 7fe45ea00640 1 -- 172.21.15.152:0/1938181384 >> v1:172.21.15.175:6789/0 conn(0x7fe4600cfc40 legacy=0x7fe4600b2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:28.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.384+0000 7fe45ea00640 1 -- 172.21.15.152:0/1938181384 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe46015f430 con 0x7fe4600b2880 2024-04-20T11:31:28.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.384+0000 7fe45ea00640 1 -- 172.21.15.152:0/1938181384 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7fe4580019f0 con 0x7fe4600b2880 2024-04-20T11:31:28.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.384+0000 7fe45ea00640 1 -- 172.21.15.152:0/1938181384 >> v1:172.21.15.152:6789/0 conn(0x7fe4600b2880 legacy=0x7fe4600b4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:28.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.384+0000 7fe45ea00640 1 --2- 172.21.15.152:0/1938181384 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe46015b510 0x7fe4440039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:28.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.384+0000 7fe45ea00640 1 --2- 172.21.15.152:0/1938181384 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe444003fb0 0x7fe444004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:28.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.384+0000 7fe45ea00640 1 --2- 172.21.15.152:0/1938181384 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe444004d40 0x7fe4440051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:28.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.384+0000 7fe45ea00640 1 -- 172.21.15.152:0/1938181384 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fe46006f420 con 0x7fe444003fb0 2024-04-20T11:31:28.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.384+0000 7fe45ea00640 1 -- 172.21.15.152:0/1938181384 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe44400bbb0 con 0x7fe444004d40 2024-04-20T11:31:28.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.384+0000 7fe45ea00640 1 -- 172.21.15.152:0/1938181384 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe44400bdb0 con 0x7fe46015b510 2024-04-20T11:31:28.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.384+0000 7fe45f400640 1 --2- 172.21.15.152:0/1938181384 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe444003fb0 0x7fe444004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:28.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.385+0000 7fe45fe00640 1 --2- 172.21.15.152:0/1938181384 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe46015b510 0x7fe4440039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:28.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.385+0000 7fe465000640 1 --2- 172.21.15.152:0/1938181384 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe444004d40 0x7fe4440051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:28.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.385+0000 7fe45f400640 1 -- 172.21.15.152:0/1938181384 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe46015b510 msgr2=0x7fe4440039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:28.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.385+0000 7fe45f400640 1 --2- 172.21.15.152:0/1938181384 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe46015b510 0x7fe4440039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:28.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.385+0000 7fe45f400640 1 -- 172.21.15.152:0/1938181384 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe444004d40 msgr2=0x7fe4440051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:28.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.385+0000 7fe45f400640 1 --2- 172.21.15.152:0/1938181384 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe444004d40 0x7fe4440051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:28.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.385+0000 7fe45f400640 1 -- 172.21.15.152:0/1938181384 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe444003520 con 0x7fe444003fb0 2024-04-20T11:31:28.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.385+0000 7fe465000640 1 --2- 172.21.15.152:0/1938181384 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe444004d40 0x7fe4440051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:31:28.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.386+0000 7fe45f400640 1 --2- 172.21.15.152:0/1938181384 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe444003fb0 0x7fe444004800 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fe44c001750 tx=0x7fe444005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3264b6a61ad17d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:28.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.386+0000 7fe45ea00640 1 -- 172.21.15.152:0/1938181384 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe44c001370 con 0x7fe444003fb0 2024-04-20T11:31:28.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.386+0000 7fe45ea00640 1 -- 172.21.15.152:0/1938181384 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe44c0052d0 con 0x7fe444003fb0 2024-04-20T11:31:28.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.386+0000 7fe45ea00640 1 -- 172.21.15.152:0/1938181384 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe44c003830 con 0x7fe444003fb0 2024-04-20T11:31:28.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.387+0000 7fe467000640 1 -- 172.21.15.152:0/1938181384 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe444003fb0 msgr2=0x7fe444004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:28.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.387+0000 7fe467000640 1 --2- 172.21.15.152:0/1938181384 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe444003fb0 0x7fe444004800 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fe44c001750 tx=0x7fe444005830 comp rx=0 tx=0).stop 2024-04-20T11:31:28.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.387+0000 7fe465000640 1 -- 172.21.15.152:0/1938181384 reap_dead start 2024-04-20T11:31:28.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.387+0000 7fe467000640 1 -- 172.21.15.152:0/1938181384 shutdown_connections 2024-04-20T11:31:28.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.387+0000 7fe467000640 1 -- 172.21.15.152:0/1938181384 >> 172.21.15.152:0/1938181384 conn(0x7fe46006b980 msgr2=0x7fe46006bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:28.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.387+0000 7fe467000640 1 -- 172.21.15.152:0/1938181384 shutdown_connections 2024-04-20T11:31:28.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.387+0000 7fe467000640 1 -- 172.21.15.152:0/1938181384 wait complete. 2024-04-20T11:31:28.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.389+0000 7fe467000640 1 Processor -- start 2024-04-20T11:31:28.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.389+0000 7fe467000640 1 -- start start 2024-04-20T11:31:28.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.390+0000 7fe467000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe444003fb0 0x7fe46014bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:28.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.390+0000 7fe467000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe460152d80 0x7fe460156d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:28.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.390+0000 7fe467000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe4601572c0 0x7fe46015df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:28.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.390+0000 7fe467000640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fe46006f950 con 0x7fe460152d80 2024-04-20T11:31:28.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.390+0000 7fe467000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe46015b8f0 con 0x7fe444003fb0 2024-04-20T11:31:28.391 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.390+0000 7fe467000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe46015fc30 con 0x7fe4601572c0 2024-04-20T11:31:28.391 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.390+0000 7fe45f400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe460152d80 0x7fe460156d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:28.391 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.390+0000 7fe45f400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe460152d80 0x7fe460156d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54376/0 (socket says 172.21.15.152:54376) 2024-04-20T11:31:28.391 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.390+0000 7fe45f400640 1 -- 172.21.15.152:0/1328271846 learned_addr learned my addr 172.21.15.152:0/1328271846 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:28.391 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.390+0000 7fe465000640 1 --2- 172.21.15.152:0/1328271846 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe4601572c0 0x7fe46015df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:28.391 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.390+0000 7fe45f400640 1 -- 172.21.15.152:0/1328271846 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe4601572c0 msgr2=0x7fe46015df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:28.391 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.390+0000 7fe45f400640 1 --2- 172.21.15.152:0/1328271846 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe4601572c0 0x7fe46015df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:28.391 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.390+0000 7fe45f400640 1 -- 172.21.15.152:0/1328271846 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe444003fb0 msgr2=0x7fe46014bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:28.391 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.390+0000 7fe45f400640 1 --2- 172.21.15.152:0/1328271846 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe444003fb0 0x7fe46014bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:28.391 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.390+0000 7fe45f400640 1 -- 172.21.15.152:0/1328271846 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4601523f0 con 0x7fe460152d80 2024-04-20T11:31:28.391 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.390+0000 7fe465000640 1 --2- 172.21.15.152:0/1328271846 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe4601572c0 0x7fe46015df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:31:28.391 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.391+0000 7fe45f400640 1 --2- 172.21.15.152:0/1328271846 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe460152d80 0x7fe460156d80 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fe44c008fd0 tx=0x7fe44c001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:28.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.391+0000 7fe457e00640 1 -- 172.21.15.152:0/1328271846 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe44c041070 con 0x7fe460152d80 2024-04-20T11:31:28.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.391+0000 7fe457e00640 1 -- 172.21.15.152:0/1328271846 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe44c005480 con 0x7fe460152d80 2024-04-20T11:31:28.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.391+0000 7fe457e00640 1 -- 172.21.15.152:0/1328271846 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe44c001370 con 0x7fe460152d80 2024-04-20T11:31:28.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.391+0000 7fe467000640 1 -- 172.21.15.152:0/1328271846 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe46015a6e0 con 0x7fe460152d80 2024-04-20T11:31:28.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.391+0000 7fe467000640 1 -- 172.21.15.152:0/1328271846 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe46015da70 con 0x7fe460152d80 2024-04-20T11:31:28.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.392+0000 7fe467000640 1 -- 172.21.15.152:0/1328271846 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4600689f0 con 0x7fe460152d80 2024-04-20T11:31:28.393 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.394+0000 7fe457e00640 1 -- 172.21.15.152:0/1328271846 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 11) ==== 91346+0+0 (secure 0 0 0) 0x7fe44c044070 con 0x7fe460152d80 2024-04-20T11:31:28.394 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.394+0000 7fe457e00640 1 --2- 172.21.15.152:0/1328271846 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe44006cc50 0x7fe44006f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:28.394 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.395+0000 7fe457e00640 1 -- 172.21.15.152:0/1328271846 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5709+0+0 (secure 0 0 0) 0x7fe44c0adf90 con 0x7fe460152d80 2024-04-20T11:31:28.394 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.395+0000 7fe45fe00640 1 --2- 172.21.15.152:0/1328271846 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe44006cc50 0x7fe44006f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:28.395 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.395+0000 7fe45fe00640 1 --2- 172.21.15.152:0/1328271846 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe44006cc50 0x7fe44006f110 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fe46015da00 tx=0x7fe450008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:28.398 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.399+0000 7fe457e00640 1 -- 172.21.15.152:0/1328271846 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe44c07bef0 con 0x7fe460152d80 2024-04-20T11:31:28.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.600+0000 7fe467000640 1 -- 172.21.15.152:0/1328271846 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fe46006ecf0 con 0x7fe460152d80 2024-04-20T11:31:28.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.601+0000 7fe457e00640 1 -- 172.21.15.152:0/1328271846 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v60) ==== 100+0+11 (secure 0 0 0) 0x7fe44c07b7f0 con 0x7fe460152d80 2024-04-20T11:31:28.601 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 64 2024-04-20T11:31:28.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.604+0000 7fe467000640 1 -- 172.21.15.152:0/1328271846 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe44006cc50 msgr2=0x7fe44006f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:28.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.604+0000 7fe467000640 1 --2- 172.21.15.152:0/1328271846 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe44006cc50 0x7fe44006f110 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fe46015da00 tx=0x7fe450008040 comp rx=0 tx=0).stop 2024-04-20T11:31:28.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.604+0000 7fe467000640 1 -- 172.21.15.152:0/1328271846 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe460152d80 msgr2=0x7fe460156d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:28.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.604+0000 7fe467000640 1 --2- 172.21.15.152:0/1328271846 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe460152d80 0x7fe460156d80 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fe44c008fd0 tx=0x7fe44c001120 comp rx=0 tx=0).stop 2024-04-20T11:31:28.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.604+0000 7fe467000640 1 -- 172.21.15.152:0/1328271846 shutdown_connections 2024-04-20T11:31:28.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.604+0000 7fe467000640 1 --2- 172.21.15.152:0/1328271846 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe44006cc50 0x7fe44006f110 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:28.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.604+0000 7fe467000640 1 --2- 172.21.15.152:0/1328271846 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe4601572c0 0x7fe46015df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:28.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.604+0000 7fe467000640 1 --2- 172.21.15.152:0/1328271846 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe460152d80 0x7fe460156d80 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:28.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.604+0000 7fe467000640 1 --2- 172.21.15.152:0/1328271846 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe444003fb0 0x7fe46014bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:28.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.604+0000 7fe467000640 1 -- 172.21.15.152:0/1328271846 >> 172.21.15.152:0/1328271846 conn(0x7fe46006b980 msgr2=0x7fe4600cf230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:28.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.604+0000 7fe467000640 1 -- 172.21.15.152:0/1328271846 shutdown_connections 2024-04-20T11:31:28.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.605+0000 7fe467000640 1 -- 172.21.15.152:0/1328271846 wait complete. 2024-04-20T11:31:28.624 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-20T11:31:28.826 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.826+0000 7fe86ec00640 1 Processor -- start 2024-04-20T11:31:28.826 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.827+0000 7fe86ec00640 1 -- start start 2024-04-20T11:31:28.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.828+0000 7fe86ec00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe868161d30 0x7fe8680d0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:28.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.828+0000 7fe86ec00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fe8680d4ec0 0x7fe8680d15e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:28.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.828+0000 7fe86ec00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8680d1b20 con 0x7fe8680d2b80 2024-04-20T11:31:28.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.828+0000 7fe86ec00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fe8680d0540 con 0x7fe8680d4ec0 2024-04-20T11:31:28.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.828+0000 7fe86ec00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe8680cfb70 con 0x7fe868161d30 2024-04-20T11:31:28.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.829+0000 7fe86e200640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fe8680d2b80 0x7fe86806c830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:54824/0 (socket says 172.21.15.152:54824) 2024-04-20T11:31:28.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.829+0000 7fe86e200640 1 -- 172.21.15.152:0/3345263656 learned_addr learned my addr 172.21.15.152:0/3345263656 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:31:28.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.829+0000 7fe86d800640 1 --2- 172.21.15.152:0/3345263656 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe868161d30 0x7fe8680d0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:28.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.829+0000 7fe86ce00640 1 --2- 172.21.15.152:0/3345263656 >> v2:172.21.15.175:3300/0 conn(0x7fe8680d4ec0 0x7fe8680d15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:28.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.829+0000 7fe867e00640 1 -- 172.21.15.152:0/3345263656 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3185039385 0 0) 0x7fe8680d1b20 con 0x7fe8680d2b80 2024-04-20T11:31:28.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.829+0000 7fe867e00640 1 -- 172.21.15.152:0/3345263656 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe84c003520 con 0x7fe8680d2b80 2024-04-20T11:31:28.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.830+0000 7fe867e00640 1 -- 172.21.15.152:0/3345263656 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1229610518 0 0) 0x7fe84c003520 con 0x7fe8680d2b80 2024-04-20T11:31:28.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.830+0000 7fe867e00640 1 -- 172.21.15.152:0/3345263656 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe868161d30 msgr2=0x7fe8680d0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:28.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.830+0000 7fe867e00640 1 --2- 172.21.15.152:0/3345263656 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe868161d30 0x7fe8680d0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:28.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.830+0000 7fe867e00640 1 -- 172.21.15.152:0/3345263656 >> v2:172.21.15.175:3300/0 conn(0x7fe8680d4ec0 msgr2=0x7fe8680d15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:28.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.830+0000 7fe867e00640 1 --2- 172.21.15.152:0/3345263656 >> v2:172.21.15.175:3300/0 conn(0x7fe8680d4ec0 0x7fe8680d15e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:28.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.830+0000 7fe867e00640 1 -- 172.21.15.152:0/3345263656 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8680d26a0 con 0x7fe8680d2b80 2024-04-20T11:31:28.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.830+0000 7fe867e00640 1 -- 172.21.15.152:0/3345263656 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7fe8600019f0 con 0x7fe8680d2b80 2024-04-20T11:31:28.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.830+0000 7fe867e00640 1 -- 172.21.15.152:0/3345263656 >> v1:172.21.15.152:6789/0 conn(0x7fe8680d2b80 legacy=0x7fe86806c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:28.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.830+0000 7fe867e00640 1 --2- 172.21.15.152:0/3345263656 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe84c0039b0 0x7fe84c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:28.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.830+0000 7fe867e00640 1 --2- 172.21.15.152:0/3345263656 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe84c004800 0x7fe84c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:28.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.830+0000 7fe867e00640 1 --2- 172.21.15.152:0/3345263656 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe868161d30 0x7fe84c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:28.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.830+0000 7fe867e00640 1 -- 172.21.15.152:0/3345263656 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fe8680d0540 con 0x7fe84c0039b0 2024-04-20T11:31:28.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.830+0000 7fe867e00640 1 -- 172.21.15.152:0/3345263656 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe8680cfb70 con 0x7fe84c004800 2024-04-20T11:31:28.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.830+0000 7fe867e00640 1 -- 172.21.15.152:0/3345263656 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe84c00b9c0 con 0x7fe868161d30 2024-04-20T11:31:28.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.830+0000 7fe86d800640 1 --2- 172.21.15.152:0/3345263656 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe84c0039b0 0x7fe84c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:28.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.830+0000 7fe86ce00640 1 --2- 172.21.15.152:0/3345263656 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe84c004800 0x7fe84c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:28.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.830+0000 7fe86d800640 1 -- 172.21.15.152:0/3345263656 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe868161d30 msgr2=0x7fe84c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:28.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.830+0000 7fe86e200640 1 --2- 172.21.15.152:0/3345263656 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe868161d30 0x7fe84c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:28.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.831+0000 7fe86d800640 1 --2- 172.21.15.152:0/3345263656 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe868161d30 0x7fe84c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:28.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.831+0000 7fe86d800640 1 -- 172.21.15.152:0/3345263656 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe84c004800 msgr2=0x7fe84c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:28.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.831+0000 7fe86d800640 1 --2- 172.21.15.152:0/3345263656 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe84c004800 0x7fe84c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:28.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.831+0000 7fe86d800640 1 -- 172.21.15.152:0/3345263656 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe84c003520 con 0x7fe84c0039b0 2024-04-20T11:31:28.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.831+0000 7fe86e200640 1 --2- 172.21.15.152:0/3345263656 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe868161d30 0x7fe84c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:31:28.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.831+0000 7fe86ce00640 1 --2- 172.21.15.152:0/3345263656 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe84c004800 0x7fe84c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:31:28.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.832+0000 7fe86d800640 1 --2- 172.21.15.152:0/3345263656 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe84c0039b0 0x7fe84c003dd0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fe858009ac0 tx=0x7fe84c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5afca7e716f56359 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:28.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.832+0000 7fe867e00640 1 -- 172.21.15.152:0/3345263656 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe858039070 con 0x7fe84c0039b0 2024-04-20T11:31:28.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.832+0000 7fe867e00640 1 -- 172.21.15.152:0/3345263656 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe85802c6b0 con 0x7fe84c0039b0 2024-04-20T11:31:28.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.832+0000 7fe867e00640 1 -- 172.21.15.152:0/3345263656 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe858009690 con 0x7fe84c0039b0 2024-04-20T11:31:28.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.832+0000 7fe86ec00640 1 -- 172.21.15.152:0/3345263656 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe84c0039b0 msgr2=0x7fe84c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:28.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.832+0000 7fe86ec00640 1 --2- 172.21.15.152:0/3345263656 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe84c0039b0 0x7fe84c003dd0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fe858009ac0 tx=0x7fe84c005830 comp rx=0 tx=0).stop 2024-04-20T11:31:28.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.833+0000 7fe86e200640 1 -- 172.21.15.152:0/3345263656 reap_dead start 2024-04-20T11:31:28.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.833+0000 7fe86ec00640 1 -- 172.21.15.152:0/3345263656 shutdown_connections 2024-04-20T11:31:28.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.833+0000 7fe86ec00640 1 -- 172.21.15.152:0/3345263656 >> 172.21.15.152:0/3345263656 conn(0x7fe86808bfd0 msgr2=0x7fe8680d45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:28.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.833+0000 7fe86ec00640 1 -- 172.21.15.152:0/3345263656 shutdown_connections 2024-04-20T11:31:28.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.833+0000 7fe86ec00640 1 -- 172.21.15.152:0/3345263656 wait complete. 2024-04-20T11:31:28.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.834+0000 7fe86ec00640 1 Processor -- start 2024-04-20T11:31:28.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.835+0000 7fe86ec00640 1 -- start start 2024-04-20T11:31:28.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.835+0000 7fe86ec00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe84c0039b0 0x7fe86815db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:28.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.835+0000 7fe86ec00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe86809e350 0x7fe8680a18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:28.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.836+0000 7fe86ec00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe86809e7d0 0x7fe8680a1e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:28.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.836+0000 7fe86ec00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fe8680cf800 con 0x7fe84c0039b0 2024-04-20T11:31:28.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.836+0000 7fe86ec00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe8680cf520 con 0x7fe86809e350 2024-04-20T11:31:28.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.836+0000 7fe86ec00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe8680cf1b0 con 0x7fe86809e7d0 2024-04-20T11:31:28.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.836+0000 7fe86d800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe84c0039b0 0x7fe86815db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:28.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.836+0000 7fe86d800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe84c0039b0 0x7fe86815db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54400/0 (socket says 172.21.15.152:54400) 2024-04-20T11:31:28.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.836+0000 7fe86d800640 1 -- 172.21.15.152:0/1488383389 learned_addr learned my addr 172.21.15.152:0/1488383389 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:28.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.836+0000 7fe86d800640 1 -- 172.21.15.152:0/1488383389 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe86809e7d0 msgr2=0x7fe8680a1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:28.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.836+0000 7fe86d800640 1 --2- 172.21.15.152:0/1488383389 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe86809e7d0 0x7fe8680a1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:28.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.836+0000 7fe86d800640 1 -- 172.21.15.152:0/1488383389 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe86809e350 msgr2=0x7fe8680a18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:28.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.836+0000 7fe86d800640 1 --2- 172.21.15.152:0/1488383389 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe86809e350 0x7fe8680a18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:28.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.836+0000 7fe86d800640 1 -- 172.21.15.152:0/1488383389 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8680cb670 con 0x7fe84c0039b0 2024-04-20T11:31:28.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.837+0000 7fe86d800640 1 --2- 172.21.15.152:0/1488383389 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe84c0039b0 0x7fe86815db60 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fe858037040 tx=0x7fe858004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:28.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.837+0000 7fe865600640 1 -- 172.21.15.152:0/1488383389 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe858042020 con 0x7fe84c0039b0 2024-04-20T11:31:28.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.837+0000 7fe865600640 1 -- 172.21.15.152:0/1488383389 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe85803a040 con 0x7fe84c0039b0 2024-04-20T11:31:28.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.837+0000 7fe865600640 1 -- 172.21.15.152:0/1488383389 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe858039070 con 0x7fe84c0039b0 2024-04-20T11:31:28.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.837+0000 7fe86ec00640 1 -- 172.21.15.152:0/1488383389 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8680cb950 con 0x7fe84c0039b0 2024-04-20T11:31:28.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.837+0000 7fe86ec00640 1 -- 172.21.15.152:0/1488383389 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8680cbf10 con 0x7fe84c0039b0 2024-04-20T11:31:28.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.838+0000 7fe86ec00640 1 -- 172.21.15.152:0/1488383389 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe868075e60 con 0x7fe84c0039b0 2024-04-20T11:31:28.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.839+0000 7fe865600640 1 -- 172.21.15.152:0/1488383389 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 11) ==== 91346+0+0 (secure 0 0 0) 0x7fe858045020 con 0x7fe84c0039b0 2024-04-20T11:31:28.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.841+0000 7fe865600640 1 --2- 172.21.15.152:0/1488383389 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe84806cc50 0x7fe84806f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:28.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.841+0000 7fe865600640 1 -- 172.21.15.152:0/1488383389 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5709+0+0 (secure 0 0 0) 0x7fe8580ad5f0 con 0x7fe84c0039b0 2024-04-20T11:31:28.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.841+0000 7fe86ce00640 1 --2- 172.21.15.152:0/1488383389 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe84806cc50 0x7fe84806f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:28.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.842+0000 7fe86ce00640 1 --2- 172.21.15.152:0/1488383389 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe84806cc50 0x7fe84806f110 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fe8681eabb0 tx=0x7fe854008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:28.845 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:28.845+0000 7fe865600640 1 -- 172.21.15.152:0/1488383389 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe85804c070 con 0x7fe84c0039b0 2024-04-20T11:31:29.066 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:29.067+0000 7fe86ec00640 1 -- 172.21.15.152:0/1488383389 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe8680d1d70 con 0x7fe84c0039b0 2024-04-20T11:31:29.067 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:29.067+0000 7fe865600640 1 -- 172.21.15.152:0/1488383389 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v60) ==== 80+0+0 (secure 0 0 0) 0x7fe85807b5d0 con 0x7fe84c0039b0 2024-04-20T11:31:29.067 INFO:teuthology.orchestra.run.smithi152.stderr:listed 0 entries 2024-04-20T11:31:29.070 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:29.070+0000 7fe86ec00640 1 -- 172.21.15.152:0/1488383389 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe84806cc50 msgr2=0x7fe84806f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:29.070 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:29.070+0000 7fe86ec00640 1 --2- 172.21.15.152:0/1488383389 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe84806cc50 0x7fe84806f110 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fe8681eabb0 tx=0x7fe854008040 comp rx=0 tx=0).stop 2024-04-20T11:31:29.070 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:29.070+0000 7fe86ec00640 1 -- 172.21.15.152:0/1488383389 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe84c0039b0 msgr2=0x7fe86815db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:29.071 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:29.070+0000 7fe86ec00640 1 --2- 172.21.15.152:0/1488383389 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe84c0039b0 0x7fe86815db60 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fe858037040 tx=0x7fe858004730 comp rx=0 tx=0).stop 2024-04-20T11:31:29.071 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:29.071+0000 7fe86ec00640 1 -- 172.21.15.152:0/1488383389 shutdown_connections 2024-04-20T11:31:29.071 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:29.071+0000 7fe86ec00640 1 --2- 172.21.15.152:0/1488383389 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe84806cc50 0x7fe84806f110 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:29.071 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:29.071+0000 7fe86ec00640 1 --2- 172.21.15.152:0/1488383389 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe86809e7d0 0x7fe8680a1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:29.071 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:29.071+0000 7fe86ec00640 1 --2- 172.21.15.152:0/1488383389 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe86809e350 0x7fe8680a18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:29.071 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:29.071+0000 7fe86ec00640 1 --2- 172.21.15.152:0/1488383389 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe84c0039b0 0x7fe86815db60 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:29.071 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:29.071+0000 7fe86ec00640 1 -- 172.21.15.152:0/1488383389 >> 172.21.15.152:0/1488383389 conn(0x7fe86808bfd0 msgr2=0x7fe8680b71c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:29.071 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:29.071+0000 7fe86ec00640 1 -- 172.21.15.152:0/1488383389 shutdown_connections 2024-04-20T11:31:29.071 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:29.071+0000 7fe86ec00640 1 -- 172.21.15.152:0/1488383389 wait complete. 2024-04-20T11:31:29.089 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi152.front.sepia.ceph.com... 2024-04-20T11:31:29.089 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:31:29.090 DEBUG:teuthology.orchestra.run.smithi152:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-20T11:31:29.166 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi152.stderr:ceph-fuse[44009]: fuse finished with error 0 and tester_r 0 2024-04-20T11:31:29.171 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi152.front.sepia.ceph.com 2024-04-20T11:31:29.171 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:31:29.172 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-20T11:31:29.379 INFO:tasks.cephfs.mount:Terminating background process 2024-04-20T11:31:29.380 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T11:31:29.380 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:31:29.433 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:31:29.638 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:29.638+0000 7fef18200640 1 Processor -- start 2024-04-20T11:31:29.638 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:29.638+0000 7fef18200640 1 -- start start 2024-04-20T11:31:29.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:29.640+0000 7fef18200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fef10145f60 0x7fef1006ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:29.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:29.640+0000 7fef18200640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef10146340 con 0x7fef100cd0d0 2024-04-20T11:31:29.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:29.640+0000 7fef18200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef1006f3c0 con 0x7fef100726f0 2024-04-20T11:31:29.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:29.640+0000 7fef18200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fef1006e9f0 con 0x7fef10145f60 2024-04-20T11:31:29.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:29.640+0000 7fef15800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fef10145f60 0x7fef1006ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:29.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:29.640+0000 7fef15800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fef10145f60 0x7fef1006ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54408/0 (socket says 172.21.15.152:54408) 2024-04-20T11:31:29.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:29.640+0000 7fef14e00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fef100cd0d0 0x7fef10071fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:54828/0 (socket says 172.21.15.152:54828) 2024-04-20T11:31:29.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:29.640+0000 7fef15800640 1 -- 172.21.15.152:0/1654362254 learned_addr learned my addr 172.21.15.152:0/1654362254 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:29.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:29.641+0000 7fef15800640 1 -- 172.21.15.152:0/1654362254 >> v1:172.21.15.175:6789/0 conn(0x7fef100726f0 legacy=0x7fef100cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:29.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:29.641+0000 7fef15800640 1 -- 172.21.15.152:0/1654362254 >> v1:172.21.15.152:6789/0 conn(0x7fef100cd0d0 legacy=0x7fef10071fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:29.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:29.641+0000 7fef15800640 1 -- 172.21.15.152:0/1654362254 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef10072d50 con 0x7fef10145f60 2024-04-20T11:31:29.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:29.642+0000 7fef15800640 1 --2- 172.21.15.152:0/1654362254 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef10145f60 0x7fef1006ee80 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fef04009870 tx=0x7fef0402b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bda9907397f1ed45 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:29.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:29.642+0000 7fef0fe00640 1 -- 172.21.15.152:0/1654362254 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef04034070 con 0x7fef10145f60 2024-04-20T11:31:29.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:29.642+0000 7fef0fe00640 1 -- 172.21.15.152:0/1654362254 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef04030850 con 0x7fef10145f60 2024-04-20T11:31:29.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:29.642+0000 7fef0fe00640 1 -- 172.21.15.152:0/1654362254 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef04030c20 con 0x7fef10145f60 2024-04-20T11:31:29.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:29.643+0000 7fef18200640 1 -- 172.21.15.152:0/1654362254 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef10145f60 msgr2=0x7fef1006ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:29.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:29.643+0000 7fef18200640 1 --2- 172.21.15.152:0/1654362254 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef10145f60 0x7fef1006ee80 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fef04009870 tx=0x7fef0402b990 comp rx=0 tx=0).stop 2024-04-20T11:31:29.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:29.643+0000 7fef18200640 1 -- 172.21.15.152:0/1654362254 shutdown_connections 2024-04-20T11:31:29.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:29.643+0000 7fef18200640 1 --2- 172.21.15.152:0/1654362254 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef10145f60 0x7fef1006ee80 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:29.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:29.643+0000 7fef18200640 1 -- 172.21.15.152:0/1654362254 >> 172.21.15.152:0/1654362254 conn(0x7fef10067cf0 msgr2=0x7fef1006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:29.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:29.643+0000 7fef18200640 1 -- 172.21.15.152:0/1654362254 shutdown_connections 2024-04-20T11:31:29.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:29.643+0000 7fef18200640 1 -- 172.21.15.152:0/1654362254 wait complete. 2024-04-20T11:31:29.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:29.645+0000 7fef18200640 1 Processor -- start 2024-04-20T11:31:29.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:29.645+0000 7fef18200640 1 -- start start 2024-04-20T11:31:29.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:29.646+0000 7fef18200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef100726f0 0x7fef10144fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:29.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:29.646+0000 7fef18200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef100cd0d0 0x7fef100ae2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:29.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:29.646+0000 7fef18200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef10145f60 0x7fef1015ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:29.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:29.646+0000 7fef18200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fef1006e680 con 0x7fef100cd0d0 2024-04-20T11:31:29.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:29.646+0000 7fef18200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fef1006e3a0 con 0x7fef100726f0 2024-04-20T11:31:29.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:29.646+0000 7fef18200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fef100cd580 con 0x7fef10145f60 2024-04-20T11:31:29.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:29.646+0000 7fef14e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef100cd0d0 0x7fef100ae2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:29.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:29.646+0000 7fef14e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef100cd0d0 0x7fef100ae2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54424/0 (socket says 172.21.15.152:54424) 2024-04-20T11:31:29.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:29.646+0000 7fef14e00640 1 -- 172.21.15.152:0/3065188172 learned_addr learned my addr 172.21.15.152:0/3065188172 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:29.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:29.646+0000 7fef14e00640 1 -- 172.21.15.152:0/3065188172 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef10145f60 msgr2=0x7fef1015ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:29.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:29.646+0000 7fef14e00640 1 --2- 172.21.15.152:0/3065188172 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef10145f60 0x7fef1015ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:29.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:29.646+0000 7fef14e00640 1 -- 172.21.15.152:0/3065188172 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef100726f0 msgr2=0x7fef10144fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:29.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:29.646+0000 7fef14e00640 1 --2- 172.21.15.152:0/3065188172 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef100726f0 0x7fef10144fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:29.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:29.646+0000 7fef14e00640 1 -- 172.21.15.152:0/3065188172 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef101e66d0 con 0x7fef100cd0d0 2024-04-20T11:31:29.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:29.647+0000 7fef14e00640 1 --2- 172.21.15.152:0/3065188172 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef100cd0d0 0x7fef100ae2c0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7feefc00c430 tx=0x7feefc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:29.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:29.647+0000 7fef0d600640 1 -- 172.21.15.152:0/3065188172 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feefc0074e0 con 0x7fef100cd0d0 2024-04-20T11:31:29.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:29.647+0000 7fef0d600640 1 -- 172.21.15.152:0/3065188172 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feefc007680 con 0x7fef100cd0d0 2024-04-20T11:31:29.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:29.647+0000 7fef18200640 1 -- 172.21.15.152:0/3065188172 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef100aeaf0 con 0x7fef100cd0d0 2024-04-20T11:31:29.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:29.647+0000 7fef0d600640 1 -- 172.21.15.152:0/3065188172 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feefc004a00 con 0x7fef100cd0d0 2024-04-20T11:31:29.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:29.647+0000 7fef18200640 1 -- 172.21.15.152:0/3065188172 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef100af0b0 con 0x7fef100cd0d0 2024-04-20T11:31:29.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:29.648+0000 7fef18200640 1 -- 172.21.15.152:0/3065188172 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef100689f0 con 0x7fef100cd0d0 2024-04-20T11:31:29.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:29.649+0000 7fef0d600640 1 -- 172.21.15.152:0/3065188172 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 11) ==== 91346+0+0 (secure 0 0 0) 0x7feefc01a020 con 0x7fef100cd0d0 2024-04-20T11:31:29.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:29.650+0000 7fef0d600640 1 --2- 172.21.15.152:0/3065188172 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7feef006cc50 0x7feef006f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:29.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:29.650+0000 7fef0d600640 1 -- 172.21.15.152:0/3065188172 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5709+0+0 (secure 0 0 0) 0x7feefc08b850 con 0x7fef100cd0d0 2024-04-20T11:31:29.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:29.651+0000 7fef15800640 1 --2- 172.21.15.152:0/3065188172 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7feef006cc50 0x7feef006f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:29.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:29.651+0000 7fef15800640 1 --2- 172.21.15.152:0/3065188172 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7feef006cc50 0x7feef006f110 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fef04036000 tx=0x7fef04030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:29.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:29.655+0000 7fef0d600640 1 -- 172.21.15.152:0/3065188172 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7feefc059900 con 0x7fef100cd0d0 2024-04-20T11:31:29.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:29.858+0000 7fef18200640 1 -- 172.21.15.152:0/3065188172 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fef100ae800 con 0x7fef100cd0d0 2024-04-20T11:31:29.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:29.860+0000 7fef0d600640 1 -- 172.21.15.152:0/3065188172 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) ==== 74+0+16725 (secure 0 0 0) 0x7feefc059200 con 0x7fef100cd0d0 2024-04-20T11:31:29.860 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:31:29.861 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":60,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:31:01.845268+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:30:58.087822+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-20T11:30:59.281767+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:31:29.863 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:29.863+0000 7fef18200640 1 -- 172.21.15.152:0/3065188172 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7feef006cc50 msgr2=0x7feef006f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:29.863 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:29.863+0000 7fef18200640 1 --2- 172.21.15.152:0/3065188172 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7feef006cc50 0x7feef006f110 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fef04036000 tx=0x7fef04030000 comp rx=0 tx=0).stop 2024-04-20T11:31:29.863 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:29.863+0000 7fef18200640 1 -- 172.21.15.152:0/3065188172 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef100cd0d0 msgr2=0x7fef100ae2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:29.863 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:29.863+0000 7fef18200640 1 --2- 172.21.15.152:0/3065188172 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef100cd0d0 0x7fef100ae2c0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7feefc00c430 tx=0x7feefc00c8f0 comp rx=0 tx=0).stop 2024-04-20T11:31:29.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:29.864+0000 7fef18200640 1 -- 172.21.15.152:0/3065188172 shutdown_connections 2024-04-20T11:31:29.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:29.864+0000 7fef18200640 1 --2- 172.21.15.152:0/3065188172 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7feef006cc50 0x7feef006f110 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:29.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:29.864+0000 7fef18200640 1 --2- 172.21.15.152:0/3065188172 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef10145f60 0x7fef1015ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:29.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:29.864+0000 7fef18200640 1 --2- 172.21.15.152:0/3065188172 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef100cd0d0 0x7fef100ae2c0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:29.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:29.864+0000 7fef18200640 1 --2- 172.21.15.152:0/3065188172 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef100726f0 0x7fef10144fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:29.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:29.864+0000 7fef18200640 1 -- 172.21.15.152:0/3065188172 >> 172.21.15.152:0/3065188172 conn(0x7fef10067cf0 msgr2=0x7fef1015a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:29.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:29.864+0000 7fef18200640 1 -- 172.21.15.152:0/3065188172 shutdown_connections 2024-04-20T11:31:29.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:29.864+0000 7fef18200640 1 -- 172.21.15.152:0/3065188172 wait complete. 2024-04-20T11:31:29.882 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T11:26:47.246571+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T11:30:58.087822+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '59', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T11:30:59.281767+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '60', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T11:31:29.883 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T11:31:30.090 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.090+0000 7f42b5c00640 1 Processor -- start 2024-04-20T11:31:30.090 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.090+0000 7f42b5c00640 1 -- start start 2024-04-20T11:31:30.091 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.092+0000 7f42b5c00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f42b015b510 0x7f42b015fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:30.091 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.092+0000 7f42b5c00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42b015f1c0 con 0x7f42b00cfc40 2024-04-20T11:31:30.091 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.092+0000 7f42b5c00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42b006f670 con 0x7f42b00b2880 2024-04-20T11:31:30.092 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.092+0000 7f42b5c00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f42b006f420 con 0x7f42b015b510 2024-04-20T11:31:30.092 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.092+0000 7f42ae000640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f42b00cfc40 0x7f42b00b2170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:54832/0 (socket says 172.21.15.152:54832) 2024-04-20T11:31:30.092 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.092+0000 7f42ae000640 1 -- 172.21.15.152:0/427604561 learned_addr learned my addr 172.21.15.152:0/427604561 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:31:30.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.092+0000 7f42aea00640 1 --2- 172.21.15.152:0/427604561 >> v2:172.21.15.175:3300/0 conn(0x7f42b015b510 0x7f42b015fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:30.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.093+0000 7f42ad600640 1 -- 172.21.15.152:0/427604561 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3583058842 0 0) 0x7f42b015f1c0 con 0x7f42b00cfc40 2024-04-20T11:31:30.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.093+0000 7f42ad600640 1 -- 172.21.15.152:0/427604561 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4294003520 con 0x7f42b00cfc40 2024-04-20T11:31:30.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.093+0000 7f42ad600640 1 -- 172.21.15.152:0/427604561 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2723733625 0 0) 0x7f4294003520 con 0x7f42b00cfc40 2024-04-20T11:31:30.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.093+0000 7f42ad600640 1 -- 172.21.15.152:0/427604561 >> v2:172.21.15.175:3300/0 conn(0x7f42b015b510 msgr2=0x7f42b015fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:30.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.093+0000 7f42ad600640 1 --2- 172.21.15.152:0/427604561 >> v2:172.21.15.175:3300/0 conn(0x7f42b015b510 0x7f42b015fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:30.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.093+0000 7f42ad600640 1 -- 172.21.15.152:0/427604561 >> v1:172.21.15.175:6789/0 conn(0x7f42b00b2880 legacy=0x7f42b00b4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:30.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.093+0000 7f42ad600640 1 -- 172.21.15.152:0/427604561 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42b015f430 con 0x7f42b00cfc40 2024-04-20T11:31:30.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.093+0000 7f42ad600640 1 -- 172.21.15.152:0/427604561 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f429c001670 con 0x7f42b00cfc40 2024-04-20T11:31:30.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.093+0000 7f42ad600640 1 -- 172.21.15.152:0/427604561 >> v1:172.21.15.152:6789/0 conn(0x7f42b00cfc40 legacy=0x7f42b00b2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:30.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.093+0000 7f42ad600640 1 --2- 172.21.15.152:0/427604561 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f42940039b0 0x7f4294003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:30.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.093+0000 7f42ad600640 1 --2- 172.21.15.152:0/427604561 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4294004800 0x7f4294004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:30.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.093+0000 7f42ad600640 1 --2- 172.21.15.152:0/427604561 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f42940051c0 0x7f429400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:30.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.093+0000 7f42ad600640 1 -- 172.21.15.152:0/427604561 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f42b006f420 con 0x7f4294004800 2024-04-20T11:31:30.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.093+0000 7f42ad600640 1 -- 172.21.15.152:0/427604561 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f429400c160 con 0x7f42940039b0 2024-04-20T11:31:30.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.093+0000 7f42ad600640 1 -- 172.21.15.152:0/427604561 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f429400c360 con 0x7f42940051c0 2024-04-20T11:31:30.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.094+0000 7f42ae000640 1 --2- 172.21.15.152:0/427604561 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4294004800 0x7f4294004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:30.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.094+0000 7f42aea00640 1 --2- 172.21.15.152:0/427604561 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f42940039b0 0x7f4294003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:30.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.094+0000 7f42af400640 1 --2- 172.21.15.152:0/427604561 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f42940051c0 0x7f429400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:30.096 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.094+0000 7f42ae000640 1 -- 172.21.15.152:0/427604561 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f42940051c0 msgr2=0x7f429400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:30.096 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.094+0000 7f42ae000640 1 --2- 172.21.15.152:0/427604561 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f42940051c0 0x7f429400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:30.096 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.094+0000 7f42ae000640 1 -- 172.21.15.152:0/427604561 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f42940039b0 msgr2=0x7f4294003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:30.096 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.094+0000 7f42ae000640 1 --2- 172.21.15.152:0/427604561 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f42940039b0 0x7f4294003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:30.096 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.094+0000 7f42ae000640 1 -- 172.21.15.152:0/427604561 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4294003520 con 0x7f4294004800 2024-04-20T11:31:30.096 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.094+0000 7f42af400640 1 --2- 172.21.15.152:0/427604561 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f42940051c0 0x7f429400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:31:30.096 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.094+0000 7f42af400640 1 -- 172.21.15.152:0/427604561 reap_dead start 2024-04-20T11:31:30.097 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.095+0000 7f42ae000640 1 --2- 172.21.15.152:0/427604561 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4294004800 0x7f4294004c80 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f429c00c240 tx=0x7f4294005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33ce687a192159b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:30.097 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.095+0000 7f42ad600640 1 -- 172.21.15.152:0/427604561 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f429c007d10 con 0x7f4294004800 2024-04-20T11:31:30.097 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.095+0000 7f42ad600640 1 -- 172.21.15.152:0/427604561 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f429c030070 con 0x7f4294004800 2024-04-20T11:31:30.098 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.095+0000 7f42ad600640 1 -- 172.21.15.152:0/427604561 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f429c02cdc0 con 0x7f4294004800 2024-04-20T11:31:30.098 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.096+0000 7f42b5c00640 1 -- 172.21.15.152:0/427604561 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4294004800 msgr2=0x7f4294004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:30.098 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.096+0000 7f42b5c00640 1 --2- 172.21.15.152:0/427604561 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4294004800 0x7f4294004c80 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f429c00c240 tx=0x7f4294005830 comp rx=0 tx=0).stop 2024-04-20T11:31:30.098 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.096+0000 7f42b5c00640 1 -- 172.21.15.152:0/427604561 shutdown_connections 2024-04-20T11:31:30.098 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.096+0000 7f42b5c00640 1 --2- 172.21.15.152:0/427604561 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4294004800 0x7f4294004c80 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:30.098 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.096+0000 7f42b5c00640 1 -- 172.21.15.152:0/427604561 >> 172.21.15.152:0/427604561 conn(0x7f42b006b980 msgr2=0x7f42b006bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:30.098 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.096+0000 7f42b5c00640 1 -- 172.21.15.152:0/427604561 shutdown_connections 2024-04-20T11:31:30.098 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.096+0000 7f42b5c00640 1 -- 172.21.15.152:0/427604561 wait complete. 2024-04-20T11:31:30.098 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.098+0000 7f42b5c00640 1 Processor -- start 2024-04-20T11:31:30.098 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.098+0000 7f42b5c00640 1 -- start start 2024-04-20T11:31:30.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.099+0000 7f42b5c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4294004800 0x7f42b014fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:30.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.099+0000 7f42b5c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f42b015a240 0x7f42b01517c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:30.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.099+0000 7f42b5c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f42b015a6a0 0x7f42b0151d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:30.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.099+0000 7f42b5c00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f42b006f950 con 0x7f4294004800 2024-04-20T11:31:30.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.099+0000 7f42b5c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f42b015b8f0 con 0x7f42b015a6a0 2024-04-20T11:31:30.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.099+0000 7f42b5c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f42b015fc30 con 0x7f42b015a240 2024-04-20T11:31:30.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.099+0000 7f42aea00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4294004800 0x7f42b014fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:30.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.099+0000 7f42aea00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4294004800 0x7f42b014fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54442/0 (socket says 172.21.15.152:54442) 2024-04-20T11:31:30.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.099+0000 7f42aea00640 1 -- 172.21.15.152:0/1139062951 learned_addr learned my addr 172.21.15.152:0/1139062951 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:30.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.100+0000 7f42ae000640 1 --2- 172.21.15.152:0/1139062951 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f42b015a240 0x7f42b01517c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:30.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.100+0000 7f42af400640 1 --2- 172.21.15.152:0/1139062951 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f42b015a6a0 0x7f42b0151d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:30.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.100+0000 7f42aea00640 1 -- 172.21.15.152:0/1139062951 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f42b015a240 msgr2=0x7f42b01517c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:30.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.100+0000 7f42aea00640 1 --2- 172.21.15.152:0/1139062951 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f42b015a240 0x7f42b01517c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:30.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.100+0000 7f42aea00640 1 -- 172.21.15.152:0/1139062951 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f42b015a6a0 msgr2=0x7f42b0151d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:30.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.100+0000 7f42aea00640 1 --2- 172.21.15.152:0/1139062951 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f42b015a6a0 0x7f42b0151d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:30.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.100+0000 7f42aea00640 1 -- 172.21.15.152:0/1139062951 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42b01503e0 con 0x7f4294004800 2024-04-20T11:31:30.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.100+0000 7f42af400640 1 --2- 172.21.15.152:0/1139062951 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f42b015a6a0 0x7f42b0151d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:31:30.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.100+0000 7f42aea00640 1 --2- 172.21.15.152:0/1139062951 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4294004800 0x7f42b014fea0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f42a00116d0 tx=0x7f42a0011b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:30.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.100+0000 7f42a6a00640 1 -- 172.21.15.152:0/1139062951 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42a000f9c0 con 0x7f4294004800 2024-04-20T11:31:30.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.101+0000 7f42a6a00640 1 -- 172.21.15.152:0/1139062951 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42a000fb60 con 0x7f4294004800 2024-04-20T11:31:30.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.101+0000 7f42b5c00640 1 -- 172.21.15.152:0/1139062951 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42b01591a0 con 0x7f4294004800 2024-04-20T11:31:30.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.101+0000 7f42a6a00640 1 -- 172.21.15.152:0/1139062951 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42a0015460 con 0x7f4294004800 2024-04-20T11:31:30.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.101+0000 7f42b5c00640 1 -- 172.21.15.152:0/1139062951 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42b0159760 con 0x7f4294004800 2024-04-20T11:31:30.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.102+0000 7f42b5c00640 1 -- 172.21.15.152:0/1139062951 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42b00689f0 con 0x7f4294004800 2024-04-20T11:31:30.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.103+0000 7f42a6a00640 1 -- 172.21.15.152:0/1139062951 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 11) ==== 91346+0+0 (secure 0 0 0) 0x7f42a0015600 con 0x7f4294004800 2024-04-20T11:31:30.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.104+0000 7f42a6a00640 1 --2- 172.21.15.152:0/1139062951 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f429006cc80 0x7f429006f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:30.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.105+0000 7f42a6a00640 1 -- 172.21.15.152:0/1139062951 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5709+0+0 (secure 0 0 0) 0x7f42a008ca30 con 0x7f4294004800 2024-04-20T11:31:30.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.105+0000 7f42ae000640 1 --2- 172.21.15.152:0/1139062951 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f429006cc80 0x7f429006f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:30.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.105+0000 7f42ae000640 1 --2- 172.21.15.152:0/1139062951 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f429006cc80 0x7f429006f140 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f429c034000 tx=0x7f429c039040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:30.108 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.109+0000 7f42a6a00640 1 -- 172.21.15.152:0/1139062951 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f42a0019030 con 0x7f4294004800 2024-04-20T11:31:30.319 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.319+0000 7f42b5c00640 1 -- 172.21.15.152:0/1139062951 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f42b006ecf0 con 0x7f4294004800 2024-04-20T11:31:30.319 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.320+0000 7f42a6a00640 1 -- 172.21.15.152:0/1139062951 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v60) ==== 93+0+10 (secure 0 0 0) 0x7f42a005aaa0 con 0x7f4294004800 2024-04-20T11:31:30.320 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 1 2024-04-20T11:31:30.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.322+0000 7f42b5c00640 1 -- 172.21.15.152:0/1139062951 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f429006cc80 msgr2=0x7f429006f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:30.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.322+0000 7f42b5c00640 1 --2- 172.21.15.152:0/1139062951 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f429006cc80 0x7f429006f140 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f429c034000 tx=0x7f429c039040 comp rx=0 tx=0).stop 2024-04-20T11:31:30.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.322+0000 7f42b5c00640 1 -- 172.21.15.152:0/1139062951 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4294004800 msgr2=0x7f42b014fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:30.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.322+0000 7f42b5c00640 1 --2- 172.21.15.152:0/1139062951 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4294004800 0x7f42b014fea0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f42a00116d0 tx=0x7f42a0011b90 comp rx=0 tx=0).stop 2024-04-20T11:31:30.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.322+0000 7f42b5c00640 1 -- 172.21.15.152:0/1139062951 shutdown_connections 2024-04-20T11:31:30.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.322+0000 7f42b5c00640 1 --2- 172.21.15.152:0/1139062951 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f429006cc80 0x7f429006f140 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:30.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.322+0000 7f42b5c00640 1 --2- 172.21.15.152:0/1139062951 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f42b015a6a0 0x7f42b0151d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:30.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.322+0000 7f42b5c00640 1 --2- 172.21.15.152:0/1139062951 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f42b015a240 0x7f42b01517c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:30.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.322+0000 7f42b5c00640 1 --2- 172.21.15.152:0/1139062951 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4294004800 0x7f42b014fea0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:30.323 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.322+0000 7f42b5c00640 1 -- 172.21.15.152:0/1139062951 >> 172.21.15.152:0/1139062951 conn(0x7f42b006b980 msgr2=0x7f42b00ce210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:30.323 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.322+0000 7f42b5c00640 1 -- 172.21.15.152:0/1139062951 shutdown_connections 2024-04-20T11:31:30.323 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.322+0000 7f42b5c00640 1 -- 172.21.15.152:0/1139062951 wait complete. 2024-04-20T11:31:30.340 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T11:31:30.535 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.535+0000 7f2dffe00640 1 Processor -- start 2024-04-20T11:31:30.535 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.536+0000 7f2dffe00640 1 -- start start 2024-04-20T11:31:30.537 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.537+0000 7f2dffe00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f2e001480b0 0x7f2e0015ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:30.537 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.537+0000 7f2dffe00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f2e0015f1f0 0x7f2e0006f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:30.537 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.537+0000 7f2dffe00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e0015f7e0 con 0x7f2e00147cd0 2024-04-20T11:31:30.537 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.537+0000 7f2dffe00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f2e0006ee10 con 0x7f2e0015f1f0 2024-04-20T11:31:30.537 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.537+0000 7f2dffe00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f2e0015fac0 con 0x7f2e001480b0 2024-04-20T11:31:30.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.538+0000 7f2dfea00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f2e00147cd0 0x7f2e0006e700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:54840/0 (socket says 172.21.15.152:54840) 2024-04-20T11:31:30.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.538+0000 7f2dfea00640 1 -- 172.21.15.152:0/1441324552 learned_addr learned my addr 172.21.15.152:0/1441324552 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:31:30.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.538+0000 7f2dff400640 1 --2- 172.21.15.152:0/1441324552 >> v2:172.21.15.152:3300/0 conn(0x7f2e0015f1f0 0x7f2e0006f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:30.539 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.538+0000 7f2df7e00640 1 --2- 172.21.15.152:0/1441324552 >> v2:172.21.15.175:3300/0 conn(0x7f2e001480b0 0x7f2e0015ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:30.539 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.538+0000 7f2dff400640 1 -- 172.21.15.152:0/1441324552 >> v2:172.21.15.175:3300/0 conn(0x7f2e001480b0 msgr2=0x7f2e0015ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:30.539 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.538+0000 7f2dff400640 1 --2- 172.21.15.152:0/1441324552 >> v2:172.21.15.175:3300/0 conn(0x7f2e001480b0 0x7f2e0015ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:30.539 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.538+0000 7f2dff400640 1 -- 172.21.15.152:0/1441324552 >> v1:172.21.15.152:6789/0 conn(0x7f2e00147cd0 legacy=0x7f2e0006e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:30.539 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.538+0000 7f2dff400640 1 -- 172.21.15.152:0/1441324552 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e0006df00 con 0x7f2e0015f1f0 2024-04-20T11:31:30.539 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.538+0000 7f2dfe000640 1 -- 172.21.15.152:0/1441324552 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3915918851 0 0) 0x7f2e0015f7e0 con 0x7f2e00147cd0 2024-04-20T11:31:30.539 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.539+0000 7f2dff400640 1 --2- 172.21.15.152:0/1441324552 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2e0015f1f0 0x7f2e0006f1c0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f2e0006fd60 tx=0x7f2df802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62a19bfeb58c007b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:30.540 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.540+0000 7f2dfe000640 1 -- 172.21.15.152:0/1441324552 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df8039070 con 0x7f2e0015f1f0 2024-04-20T11:31:30.541 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.540+0000 7f2dfe000640 1 -- 172.21.15.152:0/1441324552 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2df802cc80 con 0x7f2e0015f1f0 2024-04-20T11:31:30.541 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.540+0000 7f2dfe000640 1 -- 172.21.15.152:0/1441324552 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df8034660 con 0x7f2e0015f1f0 2024-04-20T11:31:30.541 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.540+0000 7f2dffe00640 1 -- 172.21.15.152:0/1441324552 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2e0015f1f0 msgr2=0x7f2e0006f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:30.541 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.540+0000 7f2dffe00640 1 --2- 172.21.15.152:0/1441324552 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2e0015f1f0 0x7f2e0006f1c0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f2e0006fd60 tx=0x7f2df802bed0 comp rx=0 tx=0).stop 2024-04-20T11:31:30.541 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.540+0000 7f2dffe00640 1 -- 172.21.15.152:0/1441324552 shutdown_connections 2024-04-20T11:31:30.541 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.540+0000 7f2dffe00640 1 --2- 172.21.15.152:0/1441324552 >> v2:172.21.15.175:3300/0 conn(0x7f2e001480b0 0x7f2e0015ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:30.541 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.540+0000 7f2dffe00640 1 --2- 172.21.15.152:0/1441324552 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2e0015f1f0 0x7f2e0006f1c0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:30.541 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.540+0000 7f2dffe00640 1 -- 172.21.15.152:0/1441324552 >> 172.21.15.152:0/1441324552 conn(0x7f2e00067ae0 msgr2=0x7f2e0006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:30.541 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.541+0000 7f2dffe00640 1 -- 172.21.15.152:0/1441324552 shutdown_connections 2024-04-20T11:31:30.541 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.541+0000 7f2dffe00640 1 -- 172.21.15.152:0/1441324552 wait complete. 2024-04-20T11:31:30.542 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.542+0000 7f2dffe00640 1 Processor -- start 2024-04-20T11:31:30.542 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.542+0000 7f2dffe00640 1 -- start start 2024-04-20T11:31:30.543 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.543+0000 7f2dffe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2e00147cd0 0x7f2e0015c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:30.543 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.543+0000 7f2dffe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2e001480b0 0x7f2e001e64b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:30.543 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.543+0000 7f2dffe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2e0015f1f0 0x7f2e001e69f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:30.543 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.543+0000 7f2dffe00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f2e0006f700 con 0x7f2e0015f1f0 2024-04-20T11:31:30.543 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.543+0000 7f2dffe00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2e00071ed0 con 0x7f2e001480b0 2024-04-20T11:31:30.543 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.543+0000 7f2dffe00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2e0006f880 con 0x7f2e00147cd0 2024-04-20T11:31:30.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.544+0000 7f2dff400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2e0015f1f0 0x7f2e001e69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:30.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.544+0000 7f2dff400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2e0015f1f0 0x7f2e001e69f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54450/0 (socket says 172.21.15.152:54450) 2024-04-20T11:31:30.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.544+0000 7f2dff400640 1 -- 172.21.15.152:0/4177969603 learned_addr learned my addr 172.21.15.152:0/4177969603 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:30.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.544+0000 7f2df7e00640 1 --2- 172.21.15.152:0/4177969603 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2e001480b0 0x7f2e001e64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:30.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.544+0000 7f2dfea00640 1 --2- 172.21.15.152:0/4177969603 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2e00147cd0 0x7f2e0015c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:30.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.544+0000 7f2dff400640 1 -- 172.21.15.152:0/4177969603 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2e00147cd0 msgr2=0x7f2e0015c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:30.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.544+0000 7f2dff400640 1 --2- 172.21.15.152:0/4177969603 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2e00147cd0 0x7f2e0015c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:30.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.544+0000 7f2dff400640 1 -- 172.21.15.152:0/4177969603 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2e001480b0 msgr2=0x7f2e001e64b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:30.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.544+0000 7f2dff400640 1 --2- 172.21.15.152:0/4177969603 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2e001480b0 0x7f2e001e64b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:30.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.544+0000 7f2dff400640 1 -- 172.21.15.152:0/4177969603 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e00146250 con 0x7f2e0015f1f0 2024-04-20T11:31:30.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.544+0000 7f2dff400640 1 --2- 172.21.15.152:0/4177969603 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2e0015f1f0 0x7f2e001e69f0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f2df8037040 tx=0x7f2df8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:30.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.545+0000 7f2df6a00640 1 -- 172.21.15.152:0/4177969603 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df8044020 con 0x7f2e0015f1f0 2024-04-20T11:31:30.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.545+0000 7f2df6a00640 1 -- 172.21.15.152:0/4177969603 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2df803a040 con 0x7f2e0015f1f0 2024-04-20T11:31:30.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.545+0000 7f2df6a00640 1 -- 172.21.15.152:0/4177969603 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df8039050 con 0x7f2e0015f1f0 2024-04-20T11:31:30.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.545+0000 7f2dffe00640 1 -- 172.21.15.152:0/4177969603 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e001e7020 con 0x7f2e0015f1f0 2024-04-20T11:31:30.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.545+0000 7f2dffe00640 1 -- 172.21.15.152:0/4177969603 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e001e75e0 con 0x7f2e0015f1f0 2024-04-20T11:31:30.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.546+0000 7f2dffe00640 1 -- 172.21.15.152:0/4177969603 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e00075e60 con 0x7f2e0015f1f0 2024-04-20T11:31:30.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.547+0000 7f2df6a00640 1 -- 172.21.15.152:0/4177969603 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 11) ==== 91346+0+0 (secure 0 0 0) 0x7f2df8003b90 con 0x7f2e0015f1f0 2024-04-20T11:31:30.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.548+0000 7f2df6a00640 1 --2- 172.21.15.152:0/4177969603 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f2ddc06cc80 0x7f2ddc06f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:30.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.548+0000 7f2df6a00640 1 -- 172.21.15.152:0/4177969603 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5709+0+0 (secure 0 0 0) 0x7f2df80ae950 con 0x7f2e0015f1f0 2024-04-20T11:31:30.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.548+0000 7f2dfea00640 1 --2- 172.21.15.152:0/4177969603 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f2ddc06cc80 0x7f2ddc06f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:30.548 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.549+0000 7f2dfea00640 1 --2- 172.21.15.152:0/4177969603 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f2ddc06cc80 0x7f2ddc06f140 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f2df0000f60 tx=0x7f2df0007b50 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:30.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.552+0000 7f2df6a00640 1 -- 172.21.15.152:0/4177969603 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f2df804e020 con 0x7f2e0015f1f0 2024-04-20T11:31:30.751 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.751+0000 7f2dffe00640 1 -- 172.21.15.152:0/4177969603 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f2e0015c670 con 0x7f2e0015f1f0 2024-04-20T11:31:30.752 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.752+0000 7f2df6a00640 1 -- 172.21.15.152:0/4177969603 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v60) ==== 104+0+11 (secure 0 0 0) 0x7f2df807c9c0 con 0x7f2e0015f1f0 2024-04-20T11:31:30.752 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 32 2024-04-20T11:31:30.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.754+0000 7f2dffe00640 1 -- 172.21.15.152:0/4177969603 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f2ddc06cc80 msgr2=0x7f2ddc06f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:30.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.754+0000 7f2dffe00640 1 --2- 172.21.15.152:0/4177969603 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f2ddc06cc80 0x7f2ddc06f140 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f2df0000f60 tx=0x7f2df0007b50 comp rx=0 tx=0).stop 2024-04-20T11:31:30.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.755+0000 7f2dffe00640 1 -- 172.21.15.152:0/4177969603 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2e0015f1f0 msgr2=0x7f2e001e69f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:30.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.755+0000 7f2dffe00640 1 --2- 172.21.15.152:0/4177969603 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2e0015f1f0 0x7f2e001e69f0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f2df8037040 tx=0x7f2df8004a40 comp rx=0 tx=0).stop 2024-04-20T11:31:30.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.755+0000 7f2dffe00640 1 -- 172.21.15.152:0/4177969603 shutdown_connections 2024-04-20T11:31:30.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.755+0000 7f2dffe00640 1 --2- 172.21.15.152:0/4177969603 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f2ddc06cc80 0x7f2ddc06f140 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:30.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.755+0000 7f2dffe00640 1 --2- 172.21.15.152:0/4177969603 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2e0015f1f0 0x7f2e001e69f0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:30.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.755+0000 7f2dffe00640 1 --2- 172.21.15.152:0/4177969603 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2e001480b0 0x7f2e001e64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:30.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.755+0000 7f2dffe00640 1 --2- 172.21.15.152:0/4177969603 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2e00147cd0 0x7f2e0015c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:30.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.755+0000 7f2dffe00640 1 -- 172.21.15.152:0/4177969603 >> 172.21.15.152:0/4177969603 conn(0x7f2e00067ae0 msgr2=0x7f2e000b41c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:30.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.755+0000 7f2dffe00640 1 -- 172.21.15.152:0/4177969603 shutdown_connections 2024-04-20T11:31:30.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.755+0000 7f2dffe00640 1 -- 172.21.15.152:0/4177969603 wait complete. 2024-04-20T11:31:30.773 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T11:31:30.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.976+0000 7f24b3c00640 1 Processor -- start 2024-04-20T11:31:30.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.976+0000 7f24b3c00640 1 -- start start 2024-04-20T11:31:30.978 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.978+0000 7f24b3c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f24ac0b3fa0 0x7f24ac162440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:30.978 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.978+0000 7f24b3c00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24ac0adc30 con 0x7f24ac0afba0 2024-04-20T11:31:30.978 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.978+0000 7f24b3c00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24ac162a70 con 0x7f24ac15e5d0 2024-04-20T11:31:30.978 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.978+0000 7f24b3c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f24ac162d50 con 0x7f24ac0b3fa0 2024-04-20T11:31:30.978 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.978+0000 7f24a3e00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f24ac0afba0 0x7f24ac0ae110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:54850/0 (socket says 172.21.15.152:54850) 2024-04-20T11:31:30.978 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.979+0000 7f24a3e00640 1 -- 172.21.15.152:0/2405668286 learned_addr learned my addr 172.21.15.152:0/2405668286 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:31:30.980 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.979+0000 7f24b1c00640 1 --2- 172.21.15.152:0/2405668286 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f24ac0b3fa0 0x7f24ac162440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:30.980 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.979+0000 7f24abe00640 1 -- 172.21.15.152:0/2405668286 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 16395316 0 0) 0x7f24ac0adc30 con 0x7f24ac0afba0 2024-04-20T11:31:30.980 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.979+0000 7f24abe00640 1 -- 172.21.15.152:0/2405668286 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2490003520 con 0x7f24ac0afba0 2024-04-20T11:31:30.980 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.979+0000 7f24abe00640 1 -- 172.21.15.152:0/2405668286 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1591850922 0 0) 0x7f2490003520 con 0x7f24ac0afba0 2024-04-20T11:31:30.980 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.979+0000 7f24abe00640 1 -- 172.21.15.152:0/2405668286 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f24ac0b3fa0 msgr2=0x7f24ac162440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:30.980 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.979+0000 7f24abe00640 1 --2- 172.21.15.152:0/2405668286 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f24ac0b3fa0 0x7f24ac162440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:30.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.979+0000 7f24abe00640 1 -- 172.21.15.152:0/2405668286 >> v1:172.21.15.175:6789/0 conn(0x7f24ac15e5d0 legacy=0x7f24ac0af490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:30.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.979+0000 7f24abe00640 1 -- 172.21.15.152:0/2405668286 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24ac163030 con 0x7f24ac0afba0 2024-04-20T11:31:30.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.979+0000 7f24abe00640 1 -- 172.21.15.152:0/2405668286 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f2494001690 con 0x7f24ac0afba0 2024-04-20T11:31:30.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.980+0000 7f24abe00640 1 -- 172.21.15.152:0/2405668286 >> v1:172.21.15.152:6789/0 conn(0x7f24ac0afba0 legacy=0x7f24ac0ae110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:30.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.980+0000 7f24abe00640 1 --2- 172.21.15.152:0/2405668286 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f24900039e0 0x7f2490003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:30.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.980+0000 7f24abe00640 1 --2- 172.21.15.152:0/2405668286 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f24ac0b3fa0 0x7f2490004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:30.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.980+0000 7f24abe00640 1 --2- 172.21.15.152:0/2405668286 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2490004d40 0x7f24900051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:30.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.980+0000 7f24abe00640 1 -- 172.21.15.152:0/2405668286 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f24ac162d50 con 0x7f24900039e0 2024-04-20T11:31:30.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.980+0000 7f24abe00640 1 -- 172.21.15.152:0/2405668286 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f249000ba70 con 0x7f2490004d40 2024-04-20T11:31:30.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.980+0000 7f24abe00640 1 -- 172.21.15.152:0/2405668286 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f249000bc70 con 0x7f24ac0b3fa0 2024-04-20T11:31:30.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.980+0000 7f24b1200640 1 --2- 172.21.15.152:0/2405668286 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f24900039e0 0x7f2490003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:30.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.980+0000 7f24a3e00640 1 --2- 172.21.15.152:0/2405668286 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f24ac0b3fa0 0x7f2490004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:30.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.980+0000 7f24b1200640 1 -- 172.21.15.152:0/2405668286 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f24ac0b3fa0 msgr2=0x7f2490004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:30.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.980+0000 7f24b1200640 1 --2- 172.21.15.152:0/2405668286 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f24ac0b3fa0 0x7f2490004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:30.982 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.980+0000 7f24b1200640 1 -- 172.21.15.152:0/2405668286 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2490004d40 msgr2=0x7f24900051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:30.982 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.980+0000 7f24b1c00640 1 --2- 172.21.15.152:0/2405668286 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2490004d40 0x7f24900051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:30.982 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.980+0000 7f24b1200640 1 --2- 172.21.15.152:0/2405668286 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2490004d40 0x7f24900051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:30.982 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.980+0000 7f24b1200640 1 -- 172.21.15.152:0/2405668286 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2490003520 con 0x7f24900039e0 2024-04-20T11:31:30.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.981+0000 7f24b1200640 1 --2- 172.21.15.152:0/2405668286 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f24900039e0 0x7f2490003e00 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f249c001720 tx=0x7f2490005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5944787dd774b096 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:30.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.982+0000 7f24abe00640 1 -- 172.21.15.152:0/2405668286 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f249c001370 con 0x7f24900039e0 2024-04-20T11:31:30.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.982+0000 7f24abe00640 1 -- 172.21.15.152:0/2405668286 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f249c0052d0 con 0x7f24900039e0 2024-04-20T11:31:30.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.982+0000 7f24abe00640 1 -- 172.21.15.152:0/2405668286 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f249c003830 con 0x7f24900039e0 2024-04-20T11:31:30.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.982+0000 7f24b3c00640 1 -- 172.21.15.152:0/2405668286 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f24900039e0 msgr2=0x7f2490003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:30.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.982+0000 7f24b3c00640 1 --2- 172.21.15.152:0/2405668286 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f24900039e0 0x7f2490003e00 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f249c001720 tx=0x7f2490005830 comp rx=0 tx=0).stop 2024-04-20T11:31:30.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.982+0000 7f24b1c00640 1 -- 172.21.15.152:0/2405668286 reap_dead start 2024-04-20T11:31:30.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.982+0000 7f24b3c00640 1 -- 172.21.15.152:0/2405668286 shutdown_connections 2024-04-20T11:31:30.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.982+0000 7f24b3c00640 1 -- 172.21.15.152:0/2405668286 >> 172.21.15.152:0/2405668286 conn(0x7f24ac06b980 msgr2=0x7f24ac06bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:30.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.983+0000 7f24b3c00640 1 -- 172.21.15.152:0/2405668286 shutdown_connections 2024-04-20T11:31:30.984 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.983+0000 7f24b3c00640 1 -- 172.21.15.152:0/2405668286 wait complete. 2024-04-20T11:31:30.984 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.984+0000 7f24b3c00640 1 Processor -- start 2024-04-20T11:31:30.984 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.984+0000 7f24b3c00640 1 -- start start 2024-04-20T11:31:30.985 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.985+0000 7f24b3c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f24900039e0 0x7f24ac1f4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:30.985 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.985+0000 7f24b3c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f24ac15a140 0x7f24ac15c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:30.985 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.985+0000 7f24b3c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f24ac144950 0x7f24ac144dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:30.985 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.985+0000 7f24b3c00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f24ac06e0a0 con 0x7f24900039e0 2024-04-20T11:31:30.985 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.985+0000 7f24b3c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f24ac163310 con 0x7f24ac144950 2024-04-20T11:31:30.986 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.985+0000 7f24b3c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f24ac06e220 con 0x7f24ac15a140 2024-04-20T11:31:30.986 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.986+0000 7f24b1200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f24900039e0 0x7f24ac1f4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:30.986 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.986+0000 7f24b1200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f24900039e0 0x7f24ac1f4f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54472/0 (socket says 172.21.15.152:54472) 2024-04-20T11:31:30.986 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.986+0000 7f24b1200640 1 -- 172.21.15.152:0/103435889 learned_addr learned my addr 172.21.15.152:0/103435889 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:30.986 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.986+0000 7f24b1200640 1 -- 172.21.15.152:0/103435889 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f24ac15a140 msgr2=0x7f24ac15c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:30.986 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.986+0000 7f24b1200640 1 --2- 172.21.15.152:0/103435889 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f24ac15a140 0x7f24ac15c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:30.986 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.986+0000 7f24b1200640 1 -- 172.21.15.152:0/103435889 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f24ac144950 msgr2=0x7f24ac144dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:30.986 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.986+0000 7f24b1200640 1 --2- 172.21.15.152:0/103435889 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f24ac144950 0x7f24ac144dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:30.986 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.986+0000 7f24b1200640 1 -- 172.21.15.152:0/103435889 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24ac1f54d0 con 0x7f24900039e0 2024-04-20T11:31:30.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.986+0000 7f24b1200640 1 --2- 172.21.15.152:0/103435889 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f24900039e0 0x7f24ac1f4f90 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f249c008fd0 tx=0x7f249c001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:30.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.987+0000 7f24a9600640 1 -- 172.21.15.152:0/103435889 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f249c041070 con 0x7f24900039e0 2024-04-20T11:31:30.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.987+0000 7f24a9600640 1 -- 172.21.15.152:0/103435889 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f249c005480 con 0x7f24900039e0 2024-04-20T11:31:30.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.987+0000 7f24a9600640 1 -- 172.21.15.152:0/103435889 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f249c007a30 con 0x7f24900039e0 2024-04-20T11:31:30.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.987+0000 7f24b3c00640 1 -- 172.21.15.152:0/103435889 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24ac15a6c0 con 0x7f24900039e0 2024-04-20T11:31:30.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.987+0000 7f24b3c00640 1 -- 172.21.15.152:0/103435889 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24ac15ac80 con 0x7f24900039e0 2024-04-20T11:31:30.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.988+0000 7f24b3c00640 1 -- 172.21.15.152:0/103435889 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24ac0689f0 con 0x7f24900039e0 2024-04-20T11:31:30.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.989+0000 7f24a9600640 1 -- 172.21.15.152:0/103435889 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 11) ==== 91346+0+0 (secure 0 0 0) 0x7f249c044070 con 0x7f24900039e0 2024-04-20T11:31:30.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.990+0000 7f24a9600640 1 --2- 172.21.15.152:0/103435889 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f248806cc50 0x7f248806f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:30.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.990+0000 7f24a9600640 1 -- 172.21.15.152:0/103435889 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5709+0+0 (secure 0 0 0) 0x7f249c0add40 con 0x7f24900039e0 2024-04-20T11:31:30.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.990+0000 7f24a3e00640 1 --2- 172.21.15.152:0/103435889 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f248806cc50 0x7f248806f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:30.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.991+0000 7f24a3e00640 1 --2- 172.21.15.152:0/103435889 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f248806cc50 0x7f248806f110 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f2494001050 tx=0x7f24940011e0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:30.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:30.995+0000 7f24a9600640 1 -- 172.21.15.152:0/103435889 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f24ac1f54d0 con 0x7f24900039e0 2024-04-20T11:31:31.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:31.192+0000 7f24b3c00640 1 -- 172.21.15.152:0/103435889 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f24ac06f540 con 0x7f24900039e0 2024-04-20T11:31:31.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:31.193+0000 7f24a9600640 1 -- 172.21.15.152:0/103435889 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v60) ==== 100+0+11 (secure 0 0 0) 0x7f249c07bc10 con 0x7f24900039e0 2024-04-20T11:31:31.192 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 64 2024-04-20T11:31:31.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:31.196+0000 7f24b3c00640 1 -- 172.21.15.152:0/103435889 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f248806cc50 msgr2=0x7f248806f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:31.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:31.196+0000 7f24b3c00640 1 --2- 172.21.15.152:0/103435889 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f248806cc50 0x7f248806f110 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f2494001050 tx=0x7f24940011e0 comp rx=0 tx=0).stop 2024-04-20T11:31:31.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:31.196+0000 7f24b3c00640 1 -- 172.21.15.152:0/103435889 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f24900039e0 msgr2=0x7f24ac1f4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:31.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:31.196+0000 7f24b3c00640 1 --2- 172.21.15.152:0/103435889 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f24900039e0 0x7f24ac1f4f90 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f249c008fd0 tx=0x7f249c001120 comp rx=0 tx=0).stop 2024-04-20T11:31:31.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:31.196+0000 7f24b3c00640 1 -- 172.21.15.152:0/103435889 shutdown_connections 2024-04-20T11:31:31.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:31.196+0000 7f24b3c00640 1 --2- 172.21.15.152:0/103435889 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f248806cc50 0x7f248806f110 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:31.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:31.196+0000 7f24b3c00640 1 --2- 172.21.15.152:0/103435889 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f24ac144950 0x7f24ac144dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:31.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:31.196+0000 7f24b3c00640 1 --2- 172.21.15.152:0/103435889 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f24ac15a140 0x7f24ac15c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:31.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:31.196+0000 7f24b3c00640 1 --2- 172.21.15.152:0/103435889 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f24900039e0 0x7f24ac1f4f90 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:31.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:31.196+0000 7f24b3c00640 1 -- 172.21.15.152:0/103435889 >> 172.21.15.152:0/103435889 conn(0x7f24ac06b980 msgr2=0x7f24ac06e6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:31.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:31.196+0000 7f24b3c00640 1 -- 172.21.15.152:0/103435889 shutdown_connections 2024-04-20T11:31:31.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:31.196+0000 7f24b3c00640 1 -- 172.21.15.152:0/103435889 wait complete. 2024-04-20T11:31:31.214 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-20T11:31:31.419 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:31.419+0000 7fbd84e00640 1 Processor -- start 2024-04-20T11:31:31.419 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:31.419+0000 7fbd84e00640 1 -- start start 2024-04-20T11:31:31.420 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:31.421+0000 7fbd84e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fbd80159040 0x7fbd800cf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:31.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:31.421+0000 7fbd84e00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd80159420 con 0x7fbd8006f6a0 2024-04-20T11:31:31.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:31.421+0000 7fbd84e00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd800ce8b0 con 0x7fbd80162c40 2024-04-20T11:31:31.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:31.421+0000 7fbd84e00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fbd8006fb30 con 0x7fbd80159040 2024-04-20T11:31:31.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:31.421+0000 7fbd77e00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fbd8006f6a0 0x7fbd80162530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:54854/0 (socket says 172.21.15.152:54854) 2024-04-20T11:31:31.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:31.421+0000 7fbd77e00640 1 -- 172.21.15.152:0/3663706113 learned_addr learned my addr 172.21.15.152:0/3663706113 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:31:31.423 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:31.422+0000 7fbd7f400640 1 --2- 172.21.15.152:0/3663706113 >> v2:172.21.15.175:3300/0 conn(0x7fbd80159040 0x7fbd800cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:31.423 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:31.422+0000 7fbd7ea00640 1 -- 172.21.15.152:0/3663706113 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2979552896 0 0) 0x7fbd68001110 con 0x7fbd8006f6a0 2024-04-20T11:31:31.423 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:31.422+0000 7fbd7ea00640 1 -- 172.21.15.152:0/3663706113 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbd64003520 con 0x7fbd8006f6a0 2024-04-20T11:31:31.423 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:31.422+0000 7fbd7ea00640 1 -- 172.21.15.152:0/3663706113 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2044738233 0 0) 0x7fbd64003520 con 0x7fbd8006f6a0 2024-04-20T11:31:31.423 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:31.422+0000 7fbd7ea00640 1 -- 172.21.15.152:0/3663706113 >> v2:172.21.15.175:3300/0 conn(0x7fbd80159040 msgr2=0x7fbd800cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:31.424 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:31.422+0000 7fbd7ea00640 1 --2- 172.21.15.152:0/3663706113 >> v2:172.21.15.175:3300/0 conn(0x7fbd80159040 0x7fbd800cf950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:31.424 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:31.422+0000 7fbd7ea00640 1 -- 172.21.15.152:0/3663706113 >> v1:172.21.15.175:6789/0 conn(0x7fbd80162c40 legacy=0x7fbd800b32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:31.424 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:31.422+0000 7fbd7ea00640 1 -- 172.21.15.152:0/3663706113 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd800ceb90 con 0x7fbd8006f6a0 2024-04-20T11:31:31.424 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:31.422+0000 7fbd7ea00640 1 -- 172.21.15.152:0/3663706113 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7fbd68001840 con 0x7fbd8006f6a0 2024-04-20T11:31:31.424 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:31.422+0000 7fbd7ea00640 1 -- 172.21.15.152:0/3663706113 >> v1:172.21.15.152:6789/0 conn(0x7fbd8006f6a0 legacy=0x7fbd80162530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:31.424 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:31.422+0000 7fbd7ea00640 1 --2- 172.21.15.152:0/3663706113 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd64003a00 0x7fbd64003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:31.424 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:31.422+0000 7fbd7ea00640 1 --2- 172.21.15.152:0/3663706113 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd64004800 0x7fbd64004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:31.424 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:31.423+0000 7fbd7ea00640 1 --2- 172.21.15.152:0/3663706113 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd640051c0 0x7fbd6400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:31.424 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:31.423+0000 7fbd7ea00640 1 -- 172.21.15.152:0/3663706113 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fbd8006fb30 con 0x7fbd64003a00 2024-04-20T11:31:31.424 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:31.423+0000 7fbd7ea00640 1 -- 172.21.15.152:0/3663706113 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbd6400c0c0 con 0x7fbd64004800 2024-04-20T11:31:31.424 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:31.423+0000 7fbd7ea00640 1 -- 172.21.15.152:0/3663706113 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fbd6400c2c0 con 0x7fbd640051c0 2024-04-20T11:31:31.424 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:31.423+0000 7fbd7f400640 1 --2- 172.21.15.152:0/3663706113 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd64003a00 0x7fbd64003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:31.424 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:31.423+0000 7fbd77e00640 1 --2- 172.21.15.152:0/3663706113 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd64004800 0x7fbd64004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:31.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:31.423+0000 7fbd7fe00640 1 --2- 172.21.15.152:0/3663706113 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd640051c0 0x7fbd6400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:31.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:31.423+0000 7fbd7f400640 1 -- 172.21.15.152:0/3663706113 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd640051c0 msgr2=0x7fbd6400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:31.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:31.423+0000 7fbd7f400640 1 --2- 172.21.15.152:0/3663706113 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd640051c0 0x7fbd6400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:31.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:31.423+0000 7fbd7f400640 1 -- 172.21.15.152:0/3663706113 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd64004800 msgr2=0x7fbd64004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:31.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:31.423+0000 7fbd7f400640 1 --2- 172.21.15.152:0/3663706113 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd64004800 0x7fbd64004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:31.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:31.423+0000 7fbd7f400640 1 -- 172.21.15.152:0/3663706113 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd64003520 con 0x7fbd64003a00 2024-04-20T11:31:31.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:31.423+0000 7fbd7fe00640 1 --2- 172.21.15.152:0/3663706113 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd640051c0 0x7fbd6400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:31:31.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:31.423+0000 7fbd77e00640 1 --2- 172.21.15.152:0/3663706113 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd64004800 0x7fbd64004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:31:31.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:31.423+0000 7fbd7fe00640 1 -- 172.21.15.152:0/3663706113 reap_dead start 2024-04-20T11:31:31.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:31.424+0000 7fbd7f400640 1 --2- 172.21.15.152:0/3663706113 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd64003a00 0x7fbd64003e20 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fbd70009ac0 tx=0x7fbd64005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a59023ae9e09bcc server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:31.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:31.425+0000 7fbd7ea00640 1 -- 172.21.15.152:0/3663706113 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd70039070 con 0x7fbd64003a00 2024-04-20T11:31:31.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:31.425+0000 7fbd7ea00640 1 -- 172.21.15.152:0/3663706113 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd7002c6b0 con 0x7fbd64003a00 2024-04-20T11:31:31.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:31.425+0000 7fbd7ea00640 1 -- 172.21.15.152:0/3663706113 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd70009690 con 0x7fbd64003a00 2024-04-20T11:31:31.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:31.425+0000 7fbd84e00640 1 -- 172.21.15.152:0/3663706113 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd64003a00 msgr2=0x7fbd64003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:31.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:31.425+0000 7fbd84e00640 1 --2- 172.21.15.152:0/3663706113 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd64003a00 0x7fbd64003e20 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fbd70009ac0 tx=0x7fbd64005830 comp rx=0 tx=0).stop 2024-04-20T11:31:31.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:31.425+0000 7fbd84e00640 1 -- 172.21.15.152:0/3663706113 shutdown_connections 2024-04-20T11:31:31.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:31.425+0000 7fbd84e00640 1 --2- 172.21.15.152:0/3663706113 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd64003a00 0x7fbd64003e20 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:31.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:31.425+0000 7fbd84e00640 1 -- 172.21.15.152:0/3663706113 >> 172.21.15.152:0/3663706113 conn(0x7fbd800d0260 msgr2=0x7fbd800d0670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:31.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:31.425+0000 7fbd84e00640 1 -- 172.21.15.152:0/3663706113 shutdown_connections 2024-04-20T11:31:31.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:31.425+0000 7fbd84e00640 1 -- 172.21.15.152:0/3663706113 wait complete. 2024-04-20T11:31:31.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:31.427+0000 7fbd84e00640 1 Processor -- start 2024-04-20T11:31:31.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:31.427+0000 7fbd84e00640 1 -- start start 2024-04-20T11:31:31.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:31.428+0000 7fbd84e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd64003a00 0x7fbd800cd040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:31.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:31.428+0000 7fbd84e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd8009bd20 0x7fbd8009a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:31.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:31.428+0000 7fbd84e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd800cb660 0x7fbd800cbae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:31.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:31.428+0000 7fbd84e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fbd800cf430 con 0x7fbd8009bd20 2024-04-20T11:31:31.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:31.428+0000 7fbd84e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbd800cf150 con 0x7fbd800cb660 2024-04-20T11:31:31.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:31.428+0000 7fbd84e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fbd800cf950 con 0x7fbd64003a00 2024-04-20T11:31:31.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:31.429+0000 7fbd77e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd8009bd20 0x7fbd8009a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:31.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:31.429+0000 7fbd77e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd8009bd20 0x7fbd8009a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54484/0 (socket says 172.21.15.152:54484) 2024-04-20T11:31:31.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:31.429+0000 7fbd77e00640 1 -- 172.21.15.152:0/4024414316 learned_addr learned my addr 172.21.15.152:0/4024414316 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:31.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:31.429+0000 7fbd7f400640 1 --2- 172.21.15.152:0/4024414316 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd64003a00 0x7fbd800cd040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:31.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:31.429+0000 7fbd7fe00640 1 --2- 172.21.15.152:0/4024414316 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd800cb660 0x7fbd800cbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:31.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:31.429+0000 7fbd77e00640 1 -- 172.21.15.152:0/4024414316 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd64003a00 msgr2=0x7fbd800cd040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:31.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:31.429+0000 7fbd77e00640 1 --2- 172.21.15.152:0/4024414316 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd64003a00 0x7fbd800cd040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:31.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:31.429+0000 7fbd77e00640 1 -- 172.21.15.152:0/4024414316 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd800cb660 msgr2=0x7fbd800cbae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:31.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:31.429+0000 7fbd77e00640 1 --2- 172.21.15.152:0/4024414316 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd800cb660 0x7fbd800cbae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:31.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:31.429+0000 7fbd77e00640 1 -- 172.21.15.152:0/4024414316 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd80148ff0 con 0x7fbd8009bd20 2024-04-20T11:31:31.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:31.430+0000 7fbd77e00640 1 --2- 172.21.15.152:0/4024414316 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd8009bd20 0x7fbd8009a3c0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fbd6800d6c0 tx=0x7fbd6800db80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:31.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:31.430+0000 7fbd77400640 1 -- 172.21.15.152:0/4024414316 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd68019070 con 0x7fbd8009bd20 2024-04-20T11:31:31.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:31.430+0000 7fbd77400640 1 -- 172.21.15.152:0/4024414316 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd680072d0 con 0x7fbd8009bd20 2024-04-20T11:31:31.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:31.430+0000 7fbd77400640 1 -- 172.21.15.152:0/4024414316 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd680047e0 con 0x7fbd8009bd20 2024-04-20T11:31:31.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:31.430+0000 7fbd84e00640 1 -- 172.21.15.152:0/4024414316 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd800ca8a0 con 0x7fbd8009bd20 2024-04-20T11:31:31.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:31.430+0000 7fbd84e00640 1 -- 172.21.15.152:0/4024414316 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd800cae60 con 0x7fbd8009bd20 2024-04-20T11:31:31.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:31.431+0000 7fbd84e00640 1 -- 172.21.15.152:0/4024414316 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd80075e60 con 0x7fbd8009bd20 2024-04-20T11:31:31.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:31.432+0000 7fbd77400640 1 -- 172.21.15.152:0/4024414316 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 11) ==== 91346+0+0 (secure 0 0 0) 0x7fbd68004980 con 0x7fbd8009bd20 2024-04-20T11:31:31.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:31.433+0000 7fbd77400640 1 --2- 172.21.15.152:0/4024414316 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fbd5c06cc50 0x7fbd5c06f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:31.433 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:31.433+0000 7fbd77400640 1 -- 172.21.15.152:0/4024414316 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5709+0+0 (secure 0 0 0) 0x7fbd6808b990 con 0x7fbd8009bd20 2024-04-20T11:31:31.433 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:31.433+0000 7fbd7f400640 1 --2- 172.21.15.152:0/4024414316 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fbd5c06cc50 0x7fbd5c06f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:31.433 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:31.434+0000 7fbd7f400640 1 --2- 172.21.15.152:0/4024414316 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fbd5c06cc50 0x7fbd5c06f110 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fbd70036000 tx=0x7fbd70030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:31.437 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:31.437+0000 7fbd77400640 1 -- 172.21.15.152:0/4024414316 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fbd80159420 con 0x7fbd8009bd20 2024-04-20T11:31:31.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:31.651+0000 7fbd84e00640 1 -- 172.21.15.152:0/4024414316 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fbd800af870 con 0x7fbd8009bd20 2024-04-20T11:31:31.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:31.652+0000 7fbd77400640 1 -- 172.21.15.152:0/4024414316 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v60) ==== 80+0+0 (secure 0 0 0) 0x7fbd680592a0 con 0x7fbd8009bd20 2024-04-20T11:31:31.652 INFO:teuthology.orchestra.run.smithi152.stderr:listed 0 entries 2024-04-20T11:31:31.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:31.655+0000 7fbd84e00640 1 -- 172.21.15.152:0/4024414316 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fbd5c06cc50 msgr2=0x7fbd5c06f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:31.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:31.655+0000 7fbd84e00640 1 --2- 172.21.15.152:0/4024414316 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fbd5c06cc50 0x7fbd5c06f110 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fbd70036000 tx=0x7fbd70030000 comp rx=0 tx=0).stop 2024-04-20T11:31:31.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:31.655+0000 7fbd84e00640 1 -- 172.21.15.152:0/4024414316 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd8009bd20 msgr2=0x7fbd8009a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:31.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:31.655+0000 7fbd84e00640 1 --2- 172.21.15.152:0/4024414316 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd8009bd20 0x7fbd8009a3c0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fbd6800d6c0 tx=0x7fbd6800db80 comp rx=0 tx=0).stop 2024-04-20T11:31:31.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:31.655+0000 7fbd84e00640 1 -- 172.21.15.152:0/4024414316 shutdown_connections 2024-04-20T11:31:31.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:31.655+0000 7fbd84e00640 1 --2- 172.21.15.152:0/4024414316 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fbd5c06cc50 0x7fbd5c06f110 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:31.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:31.655+0000 7fbd84e00640 1 --2- 172.21.15.152:0/4024414316 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd800cb660 0x7fbd800cbae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:31.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:31.655+0000 7fbd84e00640 1 --2- 172.21.15.152:0/4024414316 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd8009bd20 0x7fbd8009a3c0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:31.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:31.655+0000 7fbd84e00640 1 --2- 172.21.15.152:0/4024414316 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd64003a00 0x7fbd800cd040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:31.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:31.655+0000 7fbd84e00640 1 -- 172.21.15.152:0/4024414316 >> 172.21.15.152:0/4024414316 conn(0x7fbd800d0260 msgr2=0x7fbd80153850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:31.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:31.655+0000 7fbd84e00640 1 -- 172.21.15.152:0/4024414316 shutdown_connections 2024-04-20T11:31:31.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:31.655+0000 7fbd84e00640 1 -- 172.21.15.152:0/4024414316 wait complete. 2024-04-20T11:31:31.673 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi175.front.sepia.ceph.com... 2024-04-20T11:31:31.673 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:31:31.673 DEBUG:teuthology.orchestra.run.smithi175:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-04-20T11:31:31.750 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi175.stderr:ceph-fuse[31493]: fuse finished with error 0 and tester_r 0 2024-04-20T11:31:31.755 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi175.front.sepia.ceph.com 2024-04-20T11:31:31.755 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:31:31.756 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-20T11:31:31.929 INFO:tasks.cephfs.mount:Terminating background process 2024-04-20T11:31:31.929 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-20T11:31:31.930 DEBUG:teuthology.orchestra.run.smithi152:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:31:31.986 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-20T11:31:31.986 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi152.front.sepia.ceph.com 2024-04-20T11:31:31.986 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:31:31.986 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-20T11:31:32.033 INFO:teuthology.orchestra.run.smithi152.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-20T11:31:32.034 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T11:31:32.035 INFO:tasks.cephfs.mount:Terminating background process 2024-04-20T11:31:32.035 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T11:31:32.035 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:31:32.092 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-20T11:31:32.092 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi175.front.sepia.ceph.com 2024-04-20T11:31:32.092 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:31:32.092 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-20T11:31:32.142 INFO:teuthology.orchestra.run.smithi175.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-20T11:31:32.143 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T11:31:32.143 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:31:32.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.348+0000 7fc3a0000640 1 Processor -- start 2024-04-20T11:31:32.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.348+0000 7fc3a0000640 1 -- start start 2024-04-20T11:31:32.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.350+0000 7fc3a0000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc398145f60 0x7fc39806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:32.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.350+0000 7fc3a0000640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fc3980726f0 0x7fc3980cd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:32.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.350+0000 7fc3a0000640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc398146340 con 0x7fc3980cd0d0 2024-04-20T11:31:32.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.350+0000 7fc3a0000640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fc39806e9f0 con 0x7fc3980726f0 2024-04-20T11:31:32.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.350+0000 7fc3a0000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc39806f3c0 con 0x7fc398145f60 2024-04-20T11:31:32.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.350+0000 7fc39e000640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fc3980726f0 0x7fc3980cd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:32.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.351+0000 7fc39e000640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fc3980726f0 0x7fc3980cd550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.152:45800/0 (socket says 172.21.15.152:45800) 2024-04-20T11:31:32.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.351+0000 7fc39e000640 1 -- 172.21.15.152:0/2018032335 learned_addr learned my addr 172.21.15.152:0/2018032335 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:32.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.351+0000 7fc39cc00640 1 --1- 172.21.15.152:0/2018032335 >> v1:172.21.15.175:6789/0 conn(0x7fc3980cd0d0 0x7fc398071fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.152:44294/0 (socket says 172.21.15.152:44294) 2024-04-20T11:31:32.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.351+0000 7fc39d600640 1 --2- 172.21.15.152:0/2018032335 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc398145f60 0x7fc39806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:32.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.351+0000 7fc39e000640 1 -- 172.21.15.152:0/2018032335 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc398145f60 msgr2=0x7fc39806ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:32.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.351+0000 7fc39e000640 1 --2- 172.21.15.152:0/2018032335 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc398145f60 0x7fc39806ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:32.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.351+0000 7fc39e000640 1 -- 172.21.15.152:0/2018032335 >> v1:172.21.15.175:6789/0 conn(0x7fc3980cd0d0 legacy=0x7fc398071fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:32.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.351+0000 7fc39e000640 1 -- 172.21.15.152:0/2018032335 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc398072d50 con 0x7fc3980726f0 2024-04-20T11:31:32.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.353+0000 7fc39e000640 1 --2- 172.21.15.152:0/2018032335 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc3980726f0 0x7fc3980cd550 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc39806eb70 tx=0x7fc39002bf50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=86f0bf9c33a68d0d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:32.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.353+0000 7fc397e00640 1 -- 172.21.15.152:0/2018032335 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc39002caf0 con 0x7fc3980726f0 2024-04-20T11:31:32.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.353+0000 7fc397e00640 1 -- 172.21.15.152:0/2018032335 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc39002cc90 con 0x7fc3980726f0 2024-04-20T11:31:32.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.353+0000 7fc397e00640 1 -- 172.21.15.152:0/2018032335 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc390034610 con 0x7fc3980726f0 2024-04-20T11:31:32.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.354+0000 7fc3a0000640 1 -- 172.21.15.152:0/2018032335 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc3980726f0 msgr2=0x7fc3980cd550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:32.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.354+0000 7fc3a0000640 1 --2- 172.21.15.152:0/2018032335 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc3980726f0 0x7fc3980cd550 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc39806eb70 tx=0x7fc39002bf50 comp rx=0 tx=0).stop 2024-04-20T11:31:32.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.354+0000 7fc3a0000640 1 -- 172.21.15.152:0/2018032335 shutdown_connections 2024-04-20T11:31:32.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.354+0000 7fc3a0000640 1 --2- 172.21.15.152:0/2018032335 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc3980726f0 0x7fc3980cd550 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:32.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.354+0000 7fc3a0000640 1 --2- 172.21.15.152:0/2018032335 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc398145f60 0x7fc39806ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:32.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.354+0000 7fc3a0000640 1 -- 172.21.15.152:0/2018032335 >> 172.21.15.152:0/2018032335 conn(0x7fc398067cf0 msgr2=0x7fc39806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:32.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.354+0000 7fc3a0000640 1 -- 172.21.15.152:0/2018032335 shutdown_connections 2024-04-20T11:31:32.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.354+0000 7fc3a0000640 1 -- 172.21.15.152:0/2018032335 wait complete. 2024-04-20T11:31:32.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.356+0000 7fc3a0000640 1 Processor -- start 2024-04-20T11:31:32.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.356+0000 7fc3a0000640 1 -- start start 2024-04-20T11:31:32.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.357+0000 7fc3a0000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc3980726f0 0x7fc39815a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:32.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.357+0000 7fc3a0000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc3980cd0d0 0x7fc3980cb910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:32.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.357+0000 7fc3a0000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc398145f60 0x7fc3981f57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:32.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.357+0000 7fc3a0000640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fc39806e680 con 0x7fc3980cd0d0 2024-04-20T11:31:32.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.357+0000 7fc3a0000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc39806e3a0 con 0x7fc398145f60 2024-04-20T11:31:32.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.357+0000 7fc3a0000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc3980cf290 con 0x7fc3980726f0 2024-04-20T11:31:32.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.357+0000 7fc39cc00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc3980cd0d0 0x7fc3980cb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:32.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.357+0000 7fc39cc00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc3980cd0d0 0x7fc3980cb910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54498/0 (socket says 172.21.15.152:54498) 2024-04-20T11:31:32.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.357+0000 7fc39cc00640 1 -- 172.21.15.152:0/1433741444 learned_addr learned my addr 172.21.15.152:0/1433741444 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:32.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.357+0000 7fc39d600640 1 --2- 172.21.15.152:0/1433741444 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc3980726f0 0x7fc39815a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:32.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.357+0000 7fc39e000640 1 --2- 172.21.15.152:0/1433741444 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc398145f60 0x7fc3981f57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:32.359 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.358+0000 7fc39cc00640 1 -- 172.21.15.152:0/1433741444 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc3980726f0 msgr2=0x7fc39815a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:32.359 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.358+0000 7fc39cc00640 1 --2- 172.21.15.152:0/1433741444 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc3980726f0 0x7fc39815a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:32.359 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.358+0000 7fc39cc00640 1 -- 172.21.15.152:0/1433741444 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc398145f60 msgr2=0x7fc3981f57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:32.359 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.358+0000 7fc39cc00640 1 --2- 172.21.15.152:0/1433741444 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc398145f60 0x7fc3981f57b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:32.359 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.358+0000 7fc39cc00640 1 -- 172.21.15.152:0/1433741444 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3981f5cf0 con 0x7fc3980cd0d0 2024-04-20T11:31:32.359 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.358+0000 7fc39e000640 1 --2- 172.21.15.152:0/1433741444 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc398145f60 0x7fc3981f57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:31:32.359 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.358+0000 7fc39cc00640 1 --2- 172.21.15.152:0/1433741444 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc3980cd0d0 0x7fc3980cb910 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fc38400c400 tx=0x7fc38400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:32.359 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.358+0000 7fc395600640 1 -- 172.21.15.152:0/1433741444 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3840074e0 con 0x7fc3980cd0d0 2024-04-20T11:31:32.359 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.358+0000 7fc395600640 1 -- 172.21.15.152:0/1433741444 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc384007680 con 0x7fc3980cd0d0 2024-04-20T11:31:32.359 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.358+0000 7fc3a0000640 1 -- 172.21.15.152:0/1433741444 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3981f4b60 con 0x7fc3980cd0d0 2024-04-20T11:31:32.359 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.358+0000 7fc395600640 1 -- 172.21.15.152:0/1433741444 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc384004a00 con 0x7fc3980cd0d0 2024-04-20T11:31:32.359 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.359+0000 7fc3a0000640 1 -- 172.21.15.152:0/1433741444 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3981f5120 con 0x7fc3980cd0d0 2024-04-20T11:31:32.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.360+0000 7fc3a0000640 1 -- 172.21.15.152:0/1433741444 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3980689f0 con 0x7fc3980cd0d0 2024-04-20T11:31:32.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.361+0000 7fc395600640 1 -- 172.21.15.152:0/1433741444 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 11) ==== 91346+0+0 (secure 0 0 0) 0x7fc38401a020 con 0x7fc3980cd0d0 2024-04-20T11:31:32.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.362+0000 7fc395600640 1 --2- 172.21.15.152:0/1433741444 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc37806cc50 0x7fc37806f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:32.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.362+0000 7fc395600640 1 -- 172.21.15.152:0/1433741444 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5709+0+0 (secure 0 0 0) 0x7fc38408b880 con 0x7fc3980cd0d0 2024-04-20T11:31:32.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.362+0000 7fc39d600640 1 --2- 172.21.15.152:0/1433741444 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc37806cc50 0x7fc37806f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:32.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.363+0000 7fc39d600640 1 --2- 172.21.15.152:0/1433741444 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc37806cc50 0x7fc37806f110 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fc398145cc0 tx=0x7fc38c009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:32.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.366+0000 7fc395600640 1 -- 172.21.15.152:0/1433741444 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fc3840598a0 con 0x7fc3980cd0d0 2024-04-20T11:31:32.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.606+0000 7fc3a0000640 1 -- 172.21.15.152:0/1433741444 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc39806f820 con 0x7fc3980cd0d0 2024-04-20T11:31:32.607 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.607+0000 7fc395600640 1 -- 172.21.15.152:0/1433741444 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+5205 (secure 0 0 0) 0x7fc3840591a0 con 0x7fc3980cd0d0 2024-04-20T11:31:32.607 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:31:32.607 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5045,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/3268046291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3268046291},{"type":"v1","addr":"172.21.15.175:6835","nonce":3268046291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5347,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/845526345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":845526345},{"type":"v1","addr":"172.21.15.175:6833","nonce":845526345}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":6288,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6836/3582785355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":3582785355},{"type":"v1","addr":"172.21.15.152:6836","nonce":3582785355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T11:31:00.308725+0000","modified":"2024-04-20T11:31:03.369713+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6291":{"gid":6291,"name":"c","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6837/488955771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6835","nonce":488955771},{"type":"v1","addr":"172.21.15.152:6837","nonce":488955771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6291,"qdb_cluster":[6291]},"id":4}]} 2024-04-20T11:31:32.607 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 34 2024-04-20T11:31:32.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.610+0000 7fc3a0000640 1 -- 172.21.15.152:0/1433741444 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc37806cc50 msgr2=0x7fc37806f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:32.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.610+0000 7fc3a0000640 1 --2- 172.21.15.152:0/1433741444 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc37806cc50 0x7fc37806f110 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fc398145cc0 tx=0x7fc38c009190 comp rx=0 tx=0).stop 2024-04-20T11:31:32.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.610+0000 7fc3a0000640 1 -- 172.21.15.152:0/1433741444 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc3980cd0d0 msgr2=0x7fc3980cb910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:32.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.610+0000 7fc3a0000640 1 --2- 172.21.15.152:0/1433741444 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc3980cd0d0 0x7fc3980cb910 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fc38400c400 tx=0x7fc38400c8c0 comp rx=0 tx=0).stop 2024-04-20T11:31:32.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.610+0000 7fc3a0000640 1 -- 172.21.15.152:0/1433741444 shutdown_connections 2024-04-20T11:31:32.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.610+0000 7fc3a0000640 1 --2- 172.21.15.152:0/1433741444 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc37806cc50 0x7fc37806f110 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:32.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.610+0000 7fc3a0000640 1 --2- 172.21.15.152:0/1433741444 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc398145f60 0x7fc3981f57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:32.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.610+0000 7fc3a0000640 1 --2- 172.21.15.152:0/1433741444 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc3980cd0d0 0x7fc3980cb910 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:32.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.610+0000 7fc3a0000640 1 --2- 172.21.15.152:0/1433741444 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc3980726f0 0x7fc39815a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:32.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.610+0000 7fc3a0000640 1 -- 172.21.15.152:0/1433741444 >> 172.21.15.152:0/1433741444 conn(0x7fc398067cf0 msgr2=0x7fc39815dd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:32.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.610+0000 7fc3a0000640 1 -- 172.21.15.152:0/1433741444 shutdown_connections 2024-04-20T11:31:32.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.610+0000 7fc3a0000640 1 -- 172.21.15.152:0/1433741444 wait complete. 2024-04-20T11:31:32.629 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:31:32.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.823+0000 7fe7bf200640 1 Processor -- start 2024-04-20T11:31:32.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.823+0000 7fe7bf200640 1 -- start start 2024-04-20T11:31:32.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.825+0000 7fe7bf200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fe7b8145f60 0x7fe7b806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:32.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.825+0000 7fe7bf200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7b80cd0d0 0x7fe7b80cd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:32.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.825+0000 7fe7bf200640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7b8146340 con 0x7fe7b8071fe0 2024-04-20T11:31:32.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.825+0000 7fe7bf200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fe7b806e9f0 con 0x7fe7b8145f60 2024-04-20T11:31:32.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.825+0000 7fe7bf200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe7b806f3c0 con 0x7fe7b80cd0d0 2024-04-20T11:31:32.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.825+0000 7fe7b7e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fe7b8145f60 0x7fe7b806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:32.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.825+0000 7fe7bd200640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fe7b8071fe0 0x7fe7b8072470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:54856/0 (socket says 172.21.15.152:54856) 2024-04-20T11:31:32.826 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.826+0000 7fe7bd200640 1 -- 172.21.15.152:0/310230044 learned_addr learned my addr 172.21.15.152:0/310230044 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:31:32.826 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.826+0000 7fe7b7e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fe7b8145f60 0x7fe7b806ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54502/0 (socket says 172.21.15.152:54502) 2024-04-20T11:31:32.826 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.826+0000 7fe7b7400640 1 --2- 172.21.15.152:0/310230044 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7b80cd0d0 0x7fe7b80cd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:32.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.826+0000 7fe7b7e00640 1 -- 172.21.15.152:0/310230044 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7b80cd0d0 msgr2=0x7fe7b80cd550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:32.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.826+0000 7fe7b7e00640 1 --2- 172.21.15.152:0/310230044 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7b80cd0d0 0x7fe7b80cd550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:32.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.826+0000 7fe7b7e00640 1 -- 172.21.15.152:0/310230044 >> v1:172.21.15.152:6789/0 conn(0x7fe7b8071fe0 legacy=0x7fe7b8072470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:32.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.826+0000 7fe7b7e00640 1 -- 172.21.15.152:0/310230044 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7b8072d50 con 0x7fe7b8145f60 2024-04-20T11:31:32.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.827+0000 7fe7b7e00640 1 --2- 172.21.15.152:0/310230044 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe7b8145f60 0x7fe7b806ee80 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fe7b806eb70 tx=0x7fe7a802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5da1b09f4e20133f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:32.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.828+0000 7fe7b6a00640 1 -- 172.21.15.152:0/310230044 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a8035020 con 0x7fe7b8145f60 2024-04-20T11:31:32.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.828+0000 7fe7b6a00640 1 -- 172.21.15.152:0/310230044 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7a8031810 con 0x7fe7b8145f60 2024-04-20T11:31:32.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.828+0000 7fe7b6a00640 1 -- 172.21.15.152:0/310230044 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a8031be0 con 0x7fe7b8145f60 2024-04-20T11:31:32.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.828+0000 7fe7bf200640 1 -- 172.21.15.152:0/310230044 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe7b8145f60 msgr2=0x7fe7b806ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:32.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.828+0000 7fe7bf200640 1 --2- 172.21.15.152:0/310230044 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe7b8145f60 0x7fe7b806ee80 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fe7b806eb70 tx=0x7fe7a802b8b0 comp rx=0 tx=0).stop 2024-04-20T11:31:32.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.829+0000 7fe7bf200640 1 -- 172.21.15.152:0/310230044 shutdown_connections 2024-04-20T11:31:32.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.829+0000 7fe7bf200640 1 --2- 172.21.15.152:0/310230044 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7b80cd0d0 0x7fe7b80cd550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:32.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.829+0000 7fe7bf200640 1 --2- 172.21.15.152:0/310230044 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe7b8145f60 0x7fe7b806ee80 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:32.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.829+0000 7fe7bf200640 1 -- 172.21.15.152:0/310230044 >> 172.21.15.152:0/310230044 conn(0x7fe7b8067cf0 msgr2=0x7fe7b806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:32.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.829+0000 7fe7bf200640 1 -- 172.21.15.152:0/310230044 shutdown_connections 2024-04-20T11:31:32.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.829+0000 7fe7bf200640 1 -- 172.21.15.152:0/310230044 wait complete. 2024-04-20T11:31:32.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.830+0000 7fe7bf200640 1 Processor -- start 2024-04-20T11:31:32.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.831+0000 7fe7bf200640 1 -- start start 2024-04-20T11:31:32.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.832+0000 7fe7bf200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe7b8071fe0 0x7fe7b80b3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:32.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.832+0000 7fe7bf200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7b80cd0d0 0x7fe7b80b5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:32.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.832+0000 7fe7bf200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7b8145f60 0x7fe7b813e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:32.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.832+0000 7fe7bf200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b806e680 con 0x7fe7b8071fe0 2024-04-20T11:31:32.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.832+0000 7fe7bf200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b806e3a0 con 0x7fe7b8145f60 2024-04-20T11:31:32.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.832+0000 7fe7bf200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe7b80cf290 con 0x7fe7b80cd0d0 2024-04-20T11:31:32.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.832+0000 7fe7b7e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe7b8071fe0 0x7fe7b80b3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:32.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.832+0000 7fe7b7e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe7b8071fe0 0x7fe7b80b3f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54514/0 (socket says 172.21.15.152:54514) 2024-04-20T11:31:32.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.832+0000 7fe7b7e00640 1 -- 172.21.15.152:0/660420549 learned_addr learned my addr 172.21.15.152:0/660420549 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:32.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.832+0000 7fe7b7e00640 1 -- 172.21.15.152:0/660420549 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7b80cd0d0 msgr2=0x7fe7b80b5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:32.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.832+0000 7fe7b7e00640 1 --2- 172.21.15.152:0/660420549 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7b80cd0d0 0x7fe7b80b5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:32.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.832+0000 7fe7b7e00640 1 -- 172.21.15.152:0/660420549 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7b8145f60 msgr2=0x7fe7b813e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:32.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.832+0000 7fe7b7e00640 1 --2- 172.21.15.152:0/660420549 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7b8145f60 0x7fe7b813e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:32.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.832+0000 7fe7b7e00640 1 -- 172.21.15.152:0/660420549 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7b80ada20 con 0x7fe7b8071fe0 2024-04-20T11:31:32.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.833+0000 7fe7b7e00640 1 --2- 172.21.15.152:0/660420549 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe7b8071fe0 0x7fe7b80b3f80 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fe7a8000c00 tx=0x7fe7a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:32.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.833+0000 7fe7afe00640 1 -- 172.21.15.152:0/660420549 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a8040070 con 0x7fe7b8071fe0 2024-04-20T11:31:32.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.833+0000 7fe7afe00640 1 -- 172.21.15.152:0/660420549 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7a8039cf0 con 0x7fe7b8071fe0 2024-04-20T11:31:32.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.833+0000 7fe7afe00640 1 -- 172.21.15.152:0/660420549 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a8035040 con 0x7fe7b8071fe0 2024-04-20T11:31:32.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.833+0000 7fe7bf200640 1 -- 172.21.15.152:0/660420549 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7b80add00 con 0x7fe7b8071fe0 2024-04-20T11:31:32.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.833+0000 7fe7bf200640 1 -- 172.21.15.152:0/660420549 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7b80ae2c0 con 0x7fe7b8071fe0 2024-04-20T11:31:32.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.834+0000 7fe7bf200640 1 -- 172.21.15.152:0/660420549 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7b80689f0 con 0x7fe7b8071fe0 2024-04-20T11:31:32.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.835+0000 7fe7afe00640 1 -- 172.21.15.152:0/660420549 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 11) ==== 91346+0+0 (secure 0 0 0) 0x7fe7a804b060 con 0x7fe7b8071fe0 2024-04-20T11:31:32.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.836+0000 7fe7afe00640 1 --2- 172.21.15.152:0/660420549 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe79806cc50 0x7fe79806f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:32.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.836+0000 7fe7afe00640 1 -- 172.21.15.152:0/660420549 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5709+0+0 (secure 0 0 0) 0x7fe7a80adc90 con 0x7fe7b8071fe0 2024-04-20T11:31:32.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.836+0000 7fe7b7400640 1 --2- 172.21.15.152:0/660420549 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe79806cc50 0x7fe79806f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:32.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.837+0000 7fe7b7400640 1 --2- 172.21.15.152:0/660420549 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe79806cc50 0x7fe79806f110 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fe7a40070b0 tx=0x7fe7a4007040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:32.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:32.840+0000 7fe7afe00640 1 -- 172.21.15.152:0/660420549 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe7a804a2c0 con 0x7fe7b8071fe0 2024-04-20T11:31:33.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.048+0000 7fe7bf200640 1 -- 172.21.15.152:0/660420549 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe7b8144f70 con 0x7fe7b8071fe0 2024-04-20T11:31:33.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.049+0000 7fe7afe00640 1 -- 172.21.15.152:0/660420549 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) ==== 74+0+16725 (secure 0 0 0) 0x7fe7a807bd00 con 0x7fe7b8071fe0 2024-04-20T11:31:33.050 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:31:33.050 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":60,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:31:01.845268+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:30:58.087822+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-20T11:30:59.281767+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:31:33.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.052+0000 7fe7bf200640 1 -- 172.21.15.152:0/660420549 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe79806cc50 msgr2=0x7fe79806f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:33.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.052+0000 7fe7bf200640 1 --2- 172.21.15.152:0/660420549 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe79806cc50 0x7fe79806f110 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fe7a40070b0 tx=0x7fe7a4007040 comp rx=0 tx=0).stop 2024-04-20T11:31:33.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.053+0000 7fe7bf200640 1 -- 172.21.15.152:0/660420549 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe7b8071fe0 msgr2=0x7fe7b80b3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:33.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.053+0000 7fe7bf200640 1 --2- 172.21.15.152:0/660420549 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe7b8071fe0 0x7fe7b80b3f80 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fe7a8000c00 tx=0x7fe7a80046c0 comp rx=0 tx=0).stop 2024-04-20T11:31:33.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.053+0000 7fe7bf200640 1 -- 172.21.15.152:0/660420549 shutdown_connections 2024-04-20T11:31:33.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.053+0000 7fe7bf200640 1 --2- 172.21.15.152:0/660420549 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe79806cc50 0x7fe79806f110 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:33.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.053+0000 7fe7bf200640 1 --2- 172.21.15.152:0/660420549 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7b8145f60 0x7fe7b813e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:33.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.053+0000 7fe7bf200640 1 --2- 172.21.15.152:0/660420549 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7b80cd0d0 0x7fe7b80b5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:33.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.053+0000 7fe7bf200640 1 --2- 172.21.15.152:0/660420549 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe7b8071fe0 0x7fe7b80b3f80 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:33.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.053+0000 7fe7bf200640 1 -- 172.21.15.152:0/660420549 >> 172.21.15.152:0/660420549 conn(0x7fe7b8067cf0 msgr2=0x7fe7b80cc6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:33.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.053+0000 7fe7bf200640 1 -- 172.21.15.152:0/660420549 shutdown_connections 2024-04-20T11:31:33.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.053+0000 7fe7bf200640 1 -- 172.21.15.152:0/660420549 wait complete. 2024-04-20T11:31:33.072 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T11:26:47.246571+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T11:30:58.087822+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '59', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T11:30:59.281767+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '60', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T11:31:33.072 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T11:31:33.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.268+0000 7f7ca8000640 1 Processor -- start 2024-04-20T11:31:33.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.268+0000 7f7ca8000640 1 -- start start 2024-04-20T11:31:33.270 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.270+0000 7f7ca8000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7ca0146000 0x7f7ca00b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:33.270 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.270+0000 7f7ca8000640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ca01463e0 con 0x7f7ca00cd8c0 2024-04-20T11:31:33.270 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.270+0000 7f7ca8000640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ca00b3e20 con 0x7f7ca00b2170 2024-04-20T11:31:33.270 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.270+0000 7f7ca8000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7ca00b4750 con 0x7f7ca0146000 2024-04-20T11:31:33.270 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.270+0000 7f7ca6000640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f7ca00cd8c0 0x7f7ca015ccf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:54870/0 (socket says 172.21.15.152:54870) 2024-04-20T11:31:33.270 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.270+0000 7f7ca6000640 1 -- 172.21.15.152:0/386415664 learned_addr learned my addr 172.21.15.152:0/386415664 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:31:33.271 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.270+0000 7f7ca5600640 1 --2- 172.21.15.152:0/386415664 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7ca0146000 0x7f7ca00b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:33.271 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.271+0000 7f7c9fe00640 1 -- 172.21.15.152:0/386415664 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3372013123 0 0) 0x7f7ca01463e0 con 0x7f7ca00cd8c0 2024-04-20T11:31:33.271 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.271+0000 7f7c9fe00640 1 -- 172.21.15.152:0/386415664 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7c88003520 con 0x7f7ca00cd8c0 2024-04-20T11:31:33.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.271+0000 7f7c9fe00640 1 -- 172.21.15.152:0/386415664 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2206616901 0 0) 0x7f7c88003520 con 0x7f7ca00cd8c0 2024-04-20T11:31:33.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.271+0000 7f7c9fe00640 1 -- 172.21.15.152:0/386415664 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7ca0146000 msgr2=0x7f7ca00b4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:33.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.271+0000 7f7c9fe00640 1 --2- 172.21.15.152:0/386415664 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7ca0146000 0x7f7ca00b4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:33.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.271+0000 7f7c9fe00640 1 -- 172.21.15.152:0/386415664 >> v1:172.21.15.175:6789/0 conn(0x7f7ca00b2170 legacy=0x7f7ca00cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:33.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.271+0000 7f7c9fe00640 1 -- 172.21.15.152:0/386415664 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ca00b4100 con 0x7f7ca00cd8c0 2024-04-20T11:31:33.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.271+0000 7f7c9fe00640 1 -- 172.21.15.152:0/386415664 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f7c980019f0 con 0x7f7ca00cd8c0 2024-04-20T11:31:33.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.271+0000 7f7c9fe00640 1 -- 172.21.15.152:0/386415664 >> v1:172.21.15.152:6789/0 conn(0x7f7ca00cd8c0 legacy=0x7f7ca015ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:33.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.271+0000 7f7c9fe00640 1 --2- 172.21.15.152:0/386415664 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c880039e0 0x7f7c88003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:33.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.271+0000 7f7c9fe00640 1 --2- 172.21.15.152:0/386415664 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7ca0146000 0x7f7c88004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:33.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.271+0000 7f7c9fe00640 1 --2- 172.21.15.152:0/386415664 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7c88004d40 0x7f7c880051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:33.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.271+0000 7f7c9fe00640 1 -- 172.21.15.152:0/386415664 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f7ca00b4750 con 0x7f7c88004d40 2024-04-20T11:31:33.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.271+0000 7f7c9fe00640 1 -- 172.21.15.152:0/386415664 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7c8800baf0 con 0x7f7c880039e0 2024-04-20T11:31:33.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.271+0000 7f7c9fe00640 1 -- 172.21.15.152:0/386415664 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7c8800bcf0 con 0x7f7ca0146000 2024-04-20T11:31:33.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.272+0000 7f7ca6000640 1 --2- 172.21.15.152:0/386415664 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7c88004d40 0x7f7c880051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:33.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.272+0000 7f7ca5600640 1 --2- 172.21.15.152:0/386415664 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c880039e0 0x7f7c88003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:33.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.272+0000 7f7ca4c00640 1 --2- 172.21.15.152:0/386415664 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7ca0146000 0x7f7c88004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:33.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.272+0000 7f7ca6000640 1 -- 172.21.15.152:0/386415664 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7ca0146000 msgr2=0x7f7c88004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:33.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.272+0000 7f7ca6000640 1 --2- 172.21.15.152:0/386415664 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7ca0146000 0x7f7c88004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:33.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.272+0000 7f7ca6000640 1 -- 172.21.15.152:0/386415664 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c880039e0 msgr2=0x7f7c88003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:33.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.272+0000 7f7ca6000640 1 --2- 172.21.15.152:0/386415664 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c880039e0 0x7f7c88003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:33.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.272+0000 7f7ca6000640 1 -- 172.21.15.152:0/386415664 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c88003520 con 0x7f7c88004d40 2024-04-20T11:31:33.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.273+0000 7f7ca6000640 1 --2- 172.21.15.152:0/386415664 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7c88004d40 0x7f7c880051c0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f7c9800a850 tx=0x7f7c88005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac4e3be357457832 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:33.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.273+0000 7f7c9fe00640 1 -- 172.21.15.152:0/386415664 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c9800a4a0 con 0x7f7c88004d40 2024-04-20T11:31:33.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.273+0000 7f7c9fe00640 1 -- 172.21.15.152:0/386415664 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c98034650 con 0x7f7c88004d40 2024-04-20T11:31:33.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.273+0000 7f7c9fe00640 1 -- 172.21.15.152:0/386415664 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c98034a40 con 0x7f7c88004d40 2024-04-20T11:31:33.276 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.274+0000 7f7ca8000640 1 -- 172.21.15.152:0/386415664 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7c88004d40 msgr2=0x7f7c880051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:33.276 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.274+0000 7f7ca8000640 1 --2- 172.21.15.152:0/386415664 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7c88004d40 0x7f7c880051c0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f7c9800a850 tx=0x7f7c88005830 comp rx=0 tx=0).stop 2024-04-20T11:31:33.276 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.274+0000 7f7ca6000640 1 -- 172.21.15.152:0/386415664 reap_dead start 2024-04-20T11:31:33.276 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.274+0000 7f7ca8000640 1 -- 172.21.15.152:0/386415664 shutdown_connections 2024-04-20T11:31:33.276 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.274+0000 7f7ca8000640 1 -- 172.21.15.152:0/386415664 >> 172.21.15.152:0/386415664 conn(0x7f7ca006b980 msgr2=0x7f7ca006bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:33.276 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.274+0000 7f7ca8000640 1 -- 172.21.15.152:0/386415664 shutdown_connections 2024-04-20T11:31:33.276 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.274+0000 7f7ca8000640 1 -- 172.21.15.152:0/386415664 wait complete. 2024-04-20T11:31:33.276 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.276+0000 7f7ca8000640 1 Processor -- start 2024-04-20T11:31:33.276 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.276+0000 7f7ca8000640 1 -- start start 2024-04-20T11:31:33.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.277+0000 7f7ca8000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c88004d40 0x7f7ca01e57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:33.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.277+0000 7f7ca8000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7ca00cc080 0x7f7ca01e7130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:33.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.277+0000 7f7ca8000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7ca015bbb0 0x7f7ca015c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:33.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.277+0000 7f7ca8000640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f7ca00b4d60 con 0x7f7ca015bbb0 2024-04-20T11:31:33.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.277+0000 7f7ca8000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7ca00b4be0 con 0x7f7c88004d40 2024-04-20T11:31:33.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.277+0000 7f7ca8000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7ca00b4ee0 con 0x7f7ca00cc080 2024-04-20T11:31:33.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.277+0000 7f7ca6000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7ca015bbb0 0x7f7ca015c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:33.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.277+0000 7f7ca6000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7ca015bbb0 0x7f7ca015c030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54534/0 (socket says 172.21.15.152:54534) 2024-04-20T11:31:33.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.277+0000 7f7ca6000640 1 -- 172.21.15.152:0/1641312676 learned_addr learned my addr 172.21.15.152:0/1641312676 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:33.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.278+0000 7f7ca4c00640 1 --2- 172.21.15.152:0/1641312676 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7ca00cc080 0x7f7ca01e7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:33.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.278+0000 7f7ca5600640 1 --2- 172.21.15.152:0/1641312676 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c88004d40 0x7f7ca01e57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:33.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.278+0000 7f7ca6000640 1 -- 172.21.15.152:0/1641312676 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7ca00cc080 msgr2=0x7f7ca01e7130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:33.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.278+0000 7f7ca6000640 1 --2- 172.21.15.152:0/1641312676 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7ca00cc080 0x7f7ca01e7130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:33.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.278+0000 7f7ca6000640 1 -- 172.21.15.152:0/1641312676 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c88004d40 msgr2=0x7f7ca01e57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:33.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.278+0000 7f7ca6000640 1 --2- 172.21.15.152:0/1641312676 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c88004d40 0x7f7ca01e57b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:33.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.278+0000 7f7ca6000640 1 -- 172.21.15.152:0/1641312676 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ca015c570 con 0x7f7ca015bbb0 2024-04-20T11:31:33.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.278+0000 7f7ca6000640 1 --2- 172.21.15.152:0/1641312676 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7ca015bbb0 0x7f7ca015c030 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f7c9800a970 tx=0x7f7c98001dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:33.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.278+0000 7f7c9d600640 1 -- 172.21.15.152:0/1641312676 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c98034a00 con 0x7f7ca015bbb0 2024-04-20T11:31:33.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.278+0000 7f7c9d600640 1 -- 172.21.15.152:0/1641312676 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c98044070 con 0x7f7ca015bbb0 2024-04-20T11:31:33.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.278+0000 7f7c9d600640 1 -- 172.21.15.152:0/1641312676 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c98034da0 con 0x7f7ca015bbb0 2024-04-20T11:31:33.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.278+0000 7f7ca8000640 1 -- 172.21.15.152:0/1641312676 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ca00cc500 con 0x7f7ca015bbb0 2024-04-20T11:31:33.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.279+0000 7f7ca8000640 1 -- 172.21.15.152:0/1641312676 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ca01e6150 con 0x7f7ca015bbb0 2024-04-20T11:31:33.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.280+0000 7f7ca8000640 1 -- 172.21.15.152:0/1641312676 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ca00689f0 con 0x7f7ca015bbb0 2024-04-20T11:31:33.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.281+0000 7f7c9d600640 1 -- 172.21.15.152:0/1641312676 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 11) ==== 91346+0+0 (secure 0 0 0) 0x7f7c98049460 con 0x7f7ca015bbb0 2024-04-20T11:31:33.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.282+0000 7f7c9d600640 1 --2- 172.21.15.152:0/1641312676 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f7c8006cc50 0x7f7c8006f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:33.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.282+0000 7f7c9d600640 1 -- 172.21.15.152:0/1641312676 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5709+0+0 (secure 0 0 0) 0x7f7c980ae650 con 0x7f7ca015bbb0 2024-04-20T11:31:33.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.282+0000 7f7ca5600640 1 --2- 172.21.15.152:0/1641312676 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f7c8006cc50 0x7f7c8006f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:33.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.283+0000 7f7ca5600640 1 --2- 172.21.15.152:0/1641312676 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f7c8006cc50 0x7f7c8006f110 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f7c94009a10 tx=0x7f7c94008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:33.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.286+0000 7f7c9d600640 1 -- 172.21.15.152:0/1641312676 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f7c98032380 con 0x7f7ca015bbb0 2024-04-20T11:31:33.488 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.489+0000 7f7ca8000640 1 -- 172.21.15.152:0/1641312676 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7ca00d0d70 con 0x7f7ca015bbb0 2024-04-20T11:31:33.489 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.490+0000 7f7c9d600640 1 -- 172.21.15.152:0/1641312676 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v60) ==== 93+0+10 (secure 0 0 0) 0x7f7c9807c520 con 0x7f7ca015bbb0 2024-04-20T11:31:33.489 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 1 2024-04-20T11:31:33.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.493+0000 7f7ca8000640 1 -- 172.21.15.152:0/1641312676 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f7c8006cc50 msgr2=0x7f7c8006f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:33.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.493+0000 7f7ca8000640 1 --2- 172.21.15.152:0/1641312676 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f7c8006cc50 0x7f7c8006f110 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f7c94009a10 tx=0x7f7c94008040 comp rx=0 tx=0).stop 2024-04-20T11:31:33.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.493+0000 7f7ca8000640 1 -- 172.21.15.152:0/1641312676 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7ca015bbb0 msgr2=0x7f7ca015c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:33.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.493+0000 7f7ca8000640 1 --2- 172.21.15.152:0/1641312676 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7ca015bbb0 0x7f7ca015c030 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f7c9800a970 tx=0x7f7c98001dc0 comp rx=0 tx=0).stop 2024-04-20T11:31:33.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.493+0000 7f7ca8000640 1 -- 172.21.15.152:0/1641312676 shutdown_connections 2024-04-20T11:31:33.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.493+0000 7f7ca8000640 1 --2- 172.21.15.152:0/1641312676 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f7c8006cc50 0x7f7c8006f110 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:33.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.493+0000 7f7ca8000640 1 --2- 172.21.15.152:0/1641312676 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7ca015bbb0 0x7f7ca015c030 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:33.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.493+0000 7f7ca8000640 1 --2- 172.21.15.152:0/1641312676 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7ca00cc080 0x7f7ca01e7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:33.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.493+0000 7f7ca8000640 1 --2- 172.21.15.152:0/1641312676 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c88004d40 0x7f7ca01e57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:33.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.493+0000 7f7ca8000640 1 -- 172.21.15.152:0/1641312676 >> 172.21.15.152:0/1641312676 conn(0x7f7ca006b980 msgr2=0x7f7ca00b1240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:33.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.493+0000 7f7ca8000640 1 -- 172.21.15.152:0/1641312676 shutdown_connections 2024-04-20T11:31:33.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.493+0000 7f7ca8000640 1 -- 172.21.15.152:0/1641312676 wait complete. 2024-04-20T11:31:33.511 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T11:31:33.713 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.713+0000 7f247aa00640 1 Processor -- start 2024-04-20T11:31:33.713 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.714+0000 7f247aa00640 1 -- start start 2024-04-20T11:31:33.715 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.715+0000 7f247aa00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f247415eb50 0x7f24740d0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:33.715 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.715+0000 7f247aa00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f2474163230 0x7f24740d2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:33.715 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.715+0000 7f247aa00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24740d19c0 con 0x7f24740d1440 2024-04-20T11:31:33.716 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.715+0000 7f247aa00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f2474162750 con 0x7f2474163230 2024-04-20T11:31:33.716 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.715+0000 7f247aa00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2474162a30 con 0x7f247415eb50 2024-04-20T11:31:33.716 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.715+0000 7f2478c00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f2474163230 0x7f24740d2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:33.716 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.716+0000 7f2478c00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f2474163230 0x7f24740d2e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54540/0 (socket says 172.21.15.152:54540) 2024-04-20T11:31:33.716 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.716+0000 7f2478c00640 1 -- 172.21.15.152:0/3245833603 learned_addr learned my addr 172.21.15.152:0/3245833603 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:33.716 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.716+0000 7f2479600640 1 --2- 172.21.15.152:0/3245833603 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f247415eb50 0x7f24740d0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:33.716 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.716+0000 7f2478c00640 1 -- 172.21.15.152:0/3245833603 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f247415eb50 msgr2=0x7f24740d0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:33.716 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.716+0000 7f2478c00640 1 --2- 172.21.15.152:0/3245833603 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f247415eb50 0x7f24740d0f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:33.716 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.716+0000 7f2478c00640 1 -- 172.21.15.152:0/3245833603 >> v1:172.21.15.175:6789/0 conn(0x7f24740d1440 legacy=0x7f247415ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:33.716 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.716+0000 7f2478c00640 1 -- 172.21.15.152:0/3245833603 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2474162d10 con 0x7f2474163230 2024-04-20T11:31:33.717 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.717+0000 7f2478c00640 1 --2- 172.21.15.152:0/3245833603 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2474163230 0x7f24740d2e90 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f2468009850 tx=0x7f246802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2c7df815529590b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:33.717 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.717+0000 7f2473e00640 1 -- 172.21.15.152:0/3245833603 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2468037070 con 0x7f2474163230 2024-04-20T11:31:33.717 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.717+0000 7f2473e00640 1 -- 172.21.15.152:0/3245833603 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2468004030 con 0x7f2474163230 2024-04-20T11:31:33.717 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.717+0000 7f2473e00640 1 -- 172.21.15.152:0/3245833603 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2468004460 con 0x7f2474163230 2024-04-20T11:31:33.718 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.718+0000 7f247aa00640 1 -- 172.21.15.152:0/3245833603 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2474163230 msgr2=0x7f24740d2e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:33.718 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.718+0000 7f247aa00640 1 --2- 172.21.15.152:0/3245833603 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2474163230 0x7f24740d2e90 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f2468009850 tx=0x7f246802b840 comp rx=0 tx=0).stop 2024-04-20T11:31:33.718 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.718+0000 7f247aa00640 1 -- 172.21.15.152:0/3245833603 shutdown_connections 2024-04-20T11:31:33.718 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.718+0000 7f247aa00640 1 --2- 172.21.15.152:0/3245833603 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2474163230 0x7f24740d2e90 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:33.718 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.718+0000 7f247aa00640 1 --2- 172.21.15.152:0/3245833603 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f247415eb50 0x7f24740d0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:33.718 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.718+0000 7f247aa00640 1 -- 172.21.15.152:0/3245833603 >> 172.21.15.152:0/3245833603 conn(0x7f247408bfd0 msgr2=0x7f24740d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:33.718 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.718+0000 7f247aa00640 1 -- 172.21.15.152:0/3245833603 shutdown_connections 2024-04-20T11:31:33.718 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.718+0000 7f247aa00640 1 -- 172.21.15.152:0/3245833603 wait complete. 2024-04-20T11:31:33.719 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.720+0000 7f247aa00640 1 Processor -- start 2024-04-20T11:31:33.720 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.720+0000 7f247aa00640 1 -- start start 2024-04-20T11:31:33.721 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.721+0000 7f247aa00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f24740d1440 0x7f2474156150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:33.722 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.721+0000 7f247aa00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f247415eb50 0x7f247415b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:33.722 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.721+0000 7f247aa00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2474163230 0x7f247415bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:33.722 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.721+0000 7f247aa00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f24740d3550 con 0x7f24740d1440 2024-04-20T11:31:33.722 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.721+0000 7f247aa00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f24740d33d0 con 0x7f2474163230 2024-04-20T11:31:33.722 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.721+0000 7f247aa00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f24740d36d0 con 0x7f247415eb50 2024-04-20T11:31:33.722 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.721+0000 7f2479600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f24740d1440 0x7f2474156150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:33.722 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.721+0000 7f2479600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f24740d1440 0x7f2474156150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54542/0 (socket says 172.21.15.152:54542) 2024-04-20T11:31:33.722 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.721+0000 7f2479600640 1 -- 172.21.15.152:0/256548281 learned_addr learned my addr 172.21.15.152:0/256548281 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:33.722 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.722+0000 7f2478c00640 1 --2- 172.21.15.152:0/256548281 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f247415eb50 0x7f247415b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:33.722 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.722+0000 7f247a000640 1 --2- 172.21.15.152:0/256548281 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2474163230 0x7f247415bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:33.722 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.722+0000 7f2479600640 1 -- 172.21.15.152:0/256548281 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f247415eb50 msgr2=0x7f247415b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:33.722 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.722+0000 7f2479600640 1 --2- 172.21.15.152:0/256548281 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f247415eb50 0x7f247415b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:33.722 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.722+0000 7f2479600640 1 -- 172.21.15.152:0/256548281 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2474163230 msgr2=0x7f247415bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:33.723 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.722+0000 7f2479600640 1 --2- 172.21.15.152:0/256548281 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2474163230 0x7f247415bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:33.723 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.722+0000 7f2479600640 1 -- 172.21.15.152:0/256548281 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24741f8c60 con 0x7f24740d1440 2024-04-20T11:31:33.723 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.722+0000 7f2479600640 1 --2- 172.21.15.152:0/256548281 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f24740d1440 0x7f2474156150 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f246c007ef0 tx=0x7f246c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:33.724 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.723+0000 7f2471600640 1 -- 172.21.15.152:0/256548281 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f246c014620 con 0x7f24740d1440 2024-04-20T11:31:33.724 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.723+0000 7f2471600640 1 -- 172.21.15.152:0/256548281 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f246c019070 con 0x7f24740d1440 2024-04-20T11:31:33.724 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.723+0000 7f247aa00640 1 -- 172.21.15.152:0/256548281 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24740b72d0 con 0x7f24740d1440 2024-04-20T11:31:33.724 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.723+0000 7f2471600640 1 -- 172.21.15.152:0/256548281 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f246c014a00 con 0x7f24740d1440 2024-04-20T11:31:33.724 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.723+0000 7f247aa00640 1 -- 172.21.15.152:0/256548281 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24740b7890 con 0x7f24740d1440 2024-04-20T11:31:33.724 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.724+0000 7f247aa00640 1 -- 172.21.15.152:0/256548281 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2474075e60 con 0x7f24740d1440 2024-04-20T11:31:33.724 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.725+0000 7f2471600640 1 -- 172.21.15.152:0/256548281 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 11) ==== 91346+0+0 (secure 0 0 0) 0x7f246c014ba0 con 0x7f24740d1440 2024-04-20T11:31:33.725 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.726+0000 7f2471600640 1 --2- 172.21.15.152:0/256548281 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f245406cc50 0x7f245406f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:33.725 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.726+0000 7f2471600640 1 -- 172.21.15.152:0/256548281 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5709+0+0 (secure 0 0 0) 0x7f246c08c9a0 con 0x7f24740d1440 2024-04-20T11:31:33.726 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.726+0000 7f2478c00640 1 --2- 172.21.15.152:0/256548281 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f245406cc50 0x7f245406f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:33.726 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.727+0000 7f2478c00640 1 --2- 172.21.15.152:0/256548281 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f245406cc50 0x7f245406f110 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f2468034000 tx=0x7f2468008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:33.730 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.731+0000 7f2471600640 1 -- 172.21.15.152:0/256548281 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f246c05aa50 con 0x7f24740d1440 2024-04-20T11:31:33.932 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.932+0000 7f247aa00640 1 -- 172.21.15.152:0/256548281 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f247408c680 con 0x7f24740d1440 2024-04-20T11:31:33.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.933+0000 7f2471600640 1 -- 172.21.15.152:0/256548281 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v60) ==== 104+0+11 (secure 0 0 0) 0x7f246c05a350 con 0x7f24740d1440 2024-04-20T11:31:33.933 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 32 2024-04-20T11:31:33.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.935+0000 7f247aa00640 1 -- 172.21.15.152:0/256548281 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f245406cc50 msgr2=0x7f245406f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:33.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.935+0000 7f247aa00640 1 --2- 172.21.15.152:0/256548281 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f245406cc50 0x7f245406f110 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f2468034000 tx=0x7f2468008030 comp rx=0 tx=0).stop 2024-04-20T11:31:33.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.935+0000 7f247aa00640 1 -- 172.21.15.152:0/256548281 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f24740d1440 msgr2=0x7f2474156150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:33.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.935+0000 7f247aa00640 1 --2- 172.21.15.152:0/256548281 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f24740d1440 0x7f2474156150 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f246c007ef0 tx=0x7f246c00e4b0 comp rx=0 tx=0).stop 2024-04-20T11:31:33.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.936+0000 7f247aa00640 1 -- 172.21.15.152:0/256548281 shutdown_connections 2024-04-20T11:31:33.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.936+0000 7f247aa00640 1 --2- 172.21.15.152:0/256548281 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f245406cc50 0x7f245406f110 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:33.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.936+0000 7f247aa00640 1 --2- 172.21.15.152:0/256548281 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2474163230 0x7f247415bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:33.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.936+0000 7f247aa00640 1 --2- 172.21.15.152:0/256548281 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f247415eb50 0x7f247415b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:33.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.936+0000 7f247aa00640 1 --2- 172.21.15.152:0/256548281 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f24740d1440 0x7f2474156150 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:33.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.936+0000 7f247aa00640 1 -- 172.21.15.152:0/256548281 >> 172.21.15.152:0/256548281 conn(0x7f247408bfd0 msgr2=0x7f24740b1810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:33.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.936+0000 7f247aa00640 1 -- 172.21.15.152:0/256548281 shutdown_connections 2024-04-20T11:31:33.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:33.936+0000 7f247aa00640 1 -- 172.21.15.152:0/256548281 wait complete. 2024-04-20T11:31:33.954 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T11:31:34.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.163+0000 7f452d000640 1 Processor -- start 2024-04-20T11:31:34.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.163+0000 7f452d000640 1 -- start start 2024-04-20T11:31:34.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.164+0000 7f452d000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f4528156c00 0x7f4528160a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:34.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.165+0000 7f452d000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f452806c4b0 0x7f452815cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:34.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.165+0000 7f452d000640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4528160fd0 con 0x7f452815ca40 2024-04-20T11:31:34.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.165+0000 7f452d000640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f452815d5c0 con 0x7f4528156c00 2024-04-20T11:31:34.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.165+0000 7f452d000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f452815d8a0 con 0x7f452806c4b0 2024-04-20T11:31:34.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.165+0000 7f4527400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f4528156c00 0x7f4528160a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:34.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.165+0000 7f4527400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f4528156c00 0x7f4528160a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54546/0 (socket says 172.21.15.152:54546) 2024-04-20T11:31:34.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.165+0000 7f4527400640 1 -- 172.21.15.152:0/2846889953 learned_addr learned my addr 172.21.15.152:0/2846889953 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:34.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.165+0000 7f4527400640 1 -- 172.21.15.152:0/2846889953 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f452806c4b0 msgr2=0x7f452815cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:34.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.165+0000 7f4527400640 1 --2- 172.21.15.152:0/2846889953 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f452806c4b0 0x7f452815cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:34.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.165+0000 7f4527400640 1 -- 172.21.15.152:0/2846889953 >> v1:172.21.15.175:6789/0 conn(0x7f452815ca40 legacy=0x7f452806bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:34.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.165+0000 7f4527400640 1 -- 172.21.15.152:0/2846889953 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4528160290 con 0x7f4528156c00 2024-04-20T11:31:34.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.166+0000 7f4527400640 1 --2- 172.21.15.152:0/2846889953 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4528156c00 0x7f4528160a90 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f45280d1250 tx=0x7f451802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe5badfdf08d1c77 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:34.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.167+0000 7f4526a00640 1 -- 172.21.15.152:0/2846889953 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4518035020 con 0x7f4528156c00 2024-04-20T11:31:34.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.167+0000 7f4526a00640 1 -- 172.21.15.152:0/2846889953 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4518031640 con 0x7f4528156c00 2024-04-20T11:31:34.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.167+0000 7f4526a00640 1 -- 172.21.15.152:0/2846889953 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f451802d040 con 0x7f4528156c00 2024-04-20T11:31:34.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.167+0000 7f452d000640 1 -- 172.21.15.152:0/2846889953 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4528156c00 msgr2=0x7f4528160a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:34.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.167+0000 7f452d000640 1 --2- 172.21.15.152:0/2846889953 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4528156c00 0x7f4528160a90 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f45280d1250 tx=0x7f451802b8f0 comp rx=0 tx=0).stop 2024-04-20T11:31:34.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.168+0000 7f452d000640 1 -- 172.21.15.152:0/2846889953 shutdown_connections 2024-04-20T11:31:34.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.168+0000 7f452d000640 1 --2- 172.21.15.152:0/2846889953 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f452806c4b0 0x7f452815cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:34.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.168+0000 7f452d000640 1 --2- 172.21.15.152:0/2846889953 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4528156c00 0x7f4528160a90 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:34.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.168+0000 7f452d000640 1 -- 172.21.15.152:0/2846889953 >> 172.21.15.152:0/2846889953 conn(0x7f45280b21e0 msgr2=0x7f45280b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:34.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.168+0000 7f452d000640 1 -- 172.21.15.152:0/2846889953 shutdown_connections 2024-04-20T11:31:34.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.168+0000 7f452d000640 1 -- 172.21.15.152:0/2846889953 wait complete. 2024-04-20T11:31:34.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.169+0000 7f452d000640 1 Processor -- start 2024-04-20T11:31:34.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.170+0000 7f452d000640 1 -- start start 2024-04-20T11:31:34.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.170+0000 7f452d000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f452806c4b0 0x7f4528149a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:34.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.170+0000 7f452d000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4528156c00 0x7f45281f45a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:34.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.170+0000 7f452d000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f452815ca40 0x7f452809ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:34.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.171+0000 7f452d000640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f45280b3240 con 0x7f452815ca40 2024-04-20T11:31:34.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.171+0000 7f452d000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f45280b30c0 con 0x7f4528156c00 2024-04-20T11:31:34.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.171+0000 7f452d000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f45280b33c0 con 0x7f452806c4b0 2024-04-20T11:31:34.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.171+0000 7f4527e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f452815ca40 0x7f452809ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:34.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.171+0000 7f451fe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4528156c00 0x7f45281f45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:34.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.171+0000 7f4527e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f452815ca40 0x7f452809ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54550/0 (socket says 172.21.15.152:54550) 2024-04-20T11:31:34.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.171+0000 7f4527e00640 1 -- 172.21.15.152:0/3986348501 learned_addr learned my addr 172.21.15.152:0/3986348501 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:34.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.171+0000 7f4527400640 1 --2- 172.21.15.152:0/3986348501 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f452806c4b0 0x7f4528149a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:34.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.172+0000 7f4527e00640 1 -- 172.21.15.152:0/3986348501 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f452806c4b0 msgr2=0x7f4528149a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:34.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.172+0000 7f4527e00640 1 --2- 172.21.15.152:0/3986348501 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f452806c4b0 0x7f4528149a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:34.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.172+0000 7f4527e00640 1 -- 172.21.15.152:0/3986348501 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4528156c00 msgr2=0x7f45281f45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:34.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.172+0000 7f4527e00640 1 --2- 172.21.15.152:0/3986348501 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4528156c00 0x7f45281f45a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:34.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.172+0000 7f4527e00640 1 -- 172.21.15.152:0/3986348501 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45281f4ff0 con 0x7f452815ca40 2024-04-20T11:31:34.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.172+0000 7f4527e00640 1 --2- 172.21.15.152:0/3986348501 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f452815ca40 0x7f452809ab50 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f452000b9e0 tx=0x7f452000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:34.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.172+0000 7f451f400640 1 -- 172.21.15.152:0/3986348501 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4520019070 con 0x7f452815ca40 2024-04-20T11:31:34.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.172+0000 7f451f400640 1 -- 172.21.15.152:0/3986348501 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f452000cb40 con 0x7f452815ca40 2024-04-20T11:31:34.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.172+0000 7f451f400640 1 -- 172.21.15.152:0/3986348501 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4520014440 con 0x7f452815ca40 2024-04-20T11:31:34.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.172+0000 7f452d000640 1 -- 172.21.15.152:0/3986348501 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45281f52d0 con 0x7f452815ca40 2024-04-20T11:31:34.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.173+0000 7f452d000640 1 -- 172.21.15.152:0/3986348501 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4528159630 con 0x7f452815ca40 2024-04-20T11:31:34.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.174+0000 7f452d000640 1 -- 172.21.15.152:0/3986348501 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4528075e60 con 0x7f452815ca40 2024-04-20T11:31:34.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.175+0000 7f451f400640 1 -- 172.21.15.152:0/3986348501 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 11) ==== 91346+0+0 (secure 0 0 0) 0x7f452001c070 con 0x7f452815ca40 2024-04-20T11:31:34.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.176+0000 7f451f400640 1 --2- 172.21.15.152:0/3986348501 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f450406cc50 0x7f450406f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:34.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.176+0000 7f451f400640 1 -- 172.21.15.152:0/3986348501 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5709+0+0 (secure 0 0 0) 0x7f452008bcd0 con 0x7f452815ca40 2024-04-20T11:31:34.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.176+0000 7f4527400640 1 --2- 172.21.15.152:0/3986348501 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f450406cc50 0x7f450406f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:34.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.177+0000 7f4527400640 1 --2- 172.21.15.152:0/3986348501 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f450406cc50 0x7f450406f110 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f4518036000 tx=0x7f451803c040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:34.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.181+0000 7f451f400640 1 -- 172.21.15.152:0/3986348501 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f45200596f0 con 0x7f452815ca40 2024-04-20T11:31:34.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.382+0000 7f452d000640 1 -- 172.21.15.152:0/3986348501 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f452806f390 con 0x7f452815ca40 2024-04-20T11:31:34.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.383+0000 7f451f400640 1 -- 172.21.15.152:0/3986348501 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v60) ==== 100+0+11 (secure 0 0 0) 0x7f452005d9f0 con 0x7f452815ca40 2024-04-20T11:31:34.383 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 64 2024-04-20T11:31:34.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.385+0000 7f452d000640 1 -- 172.21.15.152:0/3986348501 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f450406cc50 msgr2=0x7f450406f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:34.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.385+0000 7f452d000640 1 --2- 172.21.15.152:0/3986348501 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f450406cc50 0x7f450406f110 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f4518036000 tx=0x7f451803c040 comp rx=0 tx=0).stop 2024-04-20T11:31:34.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.385+0000 7f452d000640 1 -- 172.21.15.152:0/3986348501 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f452815ca40 msgr2=0x7f452809ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:34.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.385+0000 7f452d000640 1 --2- 172.21.15.152:0/3986348501 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f452815ca40 0x7f452809ab50 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f452000b9e0 tx=0x7f452000bea0 comp rx=0 tx=0).stop 2024-04-20T11:31:34.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.386+0000 7f452d000640 1 -- 172.21.15.152:0/3986348501 shutdown_connections 2024-04-20T11:31:34.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.386+0000 7f452d000640 1 --2- 172.21.15.152:0/3986348501 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f450406cc50 0x7f450406f110 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:34.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.386+0000 7f452d000640 1 --2- 172.21.15.152:0/3986348501 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f452815ca40 0x7f452809ab50 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:34.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.386+0000 7f452d000640 1 --2- 172.21.15.152:0/3986348501 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4528156c00 0x7f45281f45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:34.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.386+0000 7f452d000640 1 --2- 172.21.15.152:0/3986348501 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f452806c4b0 0x7f4528149a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:34.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.386+0000 7f452d000640 1 -- 172.21.15.152:0/3986348501 >> 172.21.15.152:0/3986348501 conn(0x7f45280b21e0 msgr2=0x7f45280730b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:34.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.386+0000 7f452d000640 1 -- 172.21.15.152:0/3986348501 shutdown_connections 2024-04-20T11:31:34.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.386+0000 7f452d000640 1 -- 172.21.15.152:0/3986348501 wait complete. 2024-04-20T11:31:34.405 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:31:34.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.612+0000 7f46d4200640 1 Processor -- start 2024-04-20T11:31:34.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.612+0000 7f46d4200640 1 -- start start 2024-04-20T11:31:34.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.614+0000 7f46d4200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f46cc15c560 0x7f46cc0b3df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:34.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.614+0000 7f46d4200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f46cc0d0810 0x7f46cc0ada90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:34.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.614+0000 7f46d4200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46cc15fd20 con 0x7f46cc0b4330 2024-04-20T11:31:34.614 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.614+0000 7f46d4200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f46cc15ef60 con 0x7f46cc15c560 2024-04-20T11:31:34.614 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.614+0000 7f46d4200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f46cc15f240 con 0x7f46cc0d0810 2024-04-20T11:31:34.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.614+0000 7f46d1800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f46cc15c560 0x7f46cc0b3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:34.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.614+0000 7f46d1800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f46cc15c560 0x7f46cc0b3df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54552/0 (socket says 172.21.15.152:54552) 2024-04-20T11:31:34.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.614+0000 7f46d1800640 1 -- 172.21.15.152:0/3384651414 learned_addr learned my addr 172.21.15.152:0/3384651414 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:34.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.615+0000 7f46d2200640 1 --2- 172.21.15.152:0/3384651414 >> v2:172.21.15.175:3300/0 conn(0x7f46cc0d0810 0x7f46cc0ada90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:34.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.615+0000 7f46d1800640 1 -- 172.21.15.152:0/3384651414 >> v2:172.21.15.175:3300/0 conn(0x7f46cc0d0810 msgr2=0x7f46cc0ada90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:34.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.615+0000 7f46d1800640 1 --2- 172.21.15.152:0/3384651414 >> v2:172.21.15.175:3300/0 conn(0x7f46cc0d0810 0x7f46cc0ada90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:34.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.615+0000 7f46d1800640 1 -- 172.21.15.152:0/3384651414 >> v1:172.21.15.175:6789/0 conn(0x7f46cc0b4330 legacy=0x7f46cc0d0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:34.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.615+0000 7f46d1800640 1 -- 172.21.15.152:0/3384651414 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46cc15f520 con 0x7f46cc15c560 2024-04-20T11:31:34.616 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.616+0000 7f46d1800640 1 --2- 172.21.15.152:0/3384651414 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f46cc15c560 0x7f46cc0b3df0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f46cc0b58c0 tx=0x7f46c002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c72f7c3ee053914 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:34.616 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.616+0000 7f46cbe00640 1 -- 172.21.15.152:0/3384651414 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c0035020 con 0x7f46cc15c560 2024-04-20T11:31:34.616 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.616+0000 7f46cbe00640 1 -- 172.21.15.152:0/3384651414 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46c0031870 con 0x7f46cc15c560 2024-04-20T11:31:34.616 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.616+0000 7f46cbe00640 1 -- 172.21.15.152:0/3384651414 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c0031c40 con 0x7f46cc15c560 2024-04-20T11:31:34.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.617+0000 7f46d4200640 1 -- 172.21.15.152:0/3384651414 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f46cc15c560 msgr2=0x7f46cc0b3df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:34.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.617+0000 7f46d4200640 1 --2- 172.21.15.152:0/3384651414 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f46cc15c560 0x7f46cc0b3df0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f46cc0b58c0 tx=0x7f46c002b910 comp rx=0 tx=0).stop 2024-04-20T11:31:34.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.617+0000 7f46d4200640 1 -- 172.21.15.152:0/3384651414 shutdown_connections 2024-04-20T11:31:34.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.617+0000 7f46d4200640 1 --2- 172.21.15.152:0/3384651414 >> v2:172.21.15.175:3300/0 conn(0x7f46cc0d0810 0x7f46cc0ada90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:34.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.617+0000 7f46d4200640 1 --2- 172.21.15.152:0/3384651414 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f46cc15c560 0x7f46cc0b3df0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:34.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.617+0000 7f46d4200640 1 -- 172.21.15.152:0/3384651414 >> 172.21.15.152:0/3384651414 conn(0x7f46cc067cf0 msgr2=0x7f46cc06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:34.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.617+0000 7f46d4200640 1 -- 172.21.15.152:0/3384651414 shutdown_connections 2024-04-20T11:31:34.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.617+0000 7f46d4200640 1 -- 172.21.15.152:0/3384651414 wait complete. 2024-04-20T11:31:34.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.619+0000 7f46d4200640 1 Processor -- start 2024-04-20T11:31:34.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.619+0000 7f46d4200640 1 -- start start 2024-04-20T11:31:34.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.620+0000 7f46d4200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f46cc0b4330 0x7f46cc14be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:34.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.620+0000 7f46d4200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f46cc0d0810 0x7f46cc15d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:34.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.620+0000 7f46d4200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f46cc15c560 0x7f46cc0cd610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:34.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.620+0000 7f46d4200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f46cc0adfd0 con 0x7f46cc0d0810 2024-04-20T11:31:34.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.620+0000 7f46d4200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f46cc0d0dc0 con 0x7f46cc15c560 2024-04-20T11:31:34.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.620+0000 7f46d4200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f46cc0ae150 con 0x7f46cc0b4330 2024-04-20T11:31:34.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.620+0000 7f46d0e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f46cc0d0810 0x7f46cc15d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:34.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.620+0000 7f46d0e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f46cc0d0810 0x7f46cc15d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54558/0 (socket says 172.21.15.152:54558) 2024-04-20T11:31:34.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.620+0000 7f46d0e00640 1 -- 172.21.15.152:0/1893406810 learned_addr learned my addr 172.21.15.152:0/1893406810 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:34.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.621+0000 7f46d1800640 1 --2- 172.21.15.152:0/1893406810 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f46cc0b4330 0x7f46cc14be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:34.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.621+0000 7f46d2200640 1 --2- 172.21.15.152:0/1893406810 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f46cc15c560 0x7f46cc0cd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:34.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.621+0000 7f46d0e00640 1 -- 172.21.15.152:0/1893406810 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f46cc0b4330 msgr2=0x7f46cc14be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:34.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.621+0000 7f46d0e00640 1 --2- 172.21.15.152:0/1893406810 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f46cc0b4330 0x7f46cc14be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:34.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.621+0000 7f46d0e00640 1 -- 172.21.15.152:0/1893406810 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f46cc15c560 msgr2=0x7f46cc0cd610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:34.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.621+0000 7f46d0e00640 1 --2- 172.21.15.152:0/1893406810 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f46cc15c560 0x7f46cc0cd610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:34.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.621+0000 7f46d0e00640 1 -- 172.21.15.152:0/1893406810 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46cc15ddf0 con 0x7f46cc0d0810 2024-04-20T11:31:34.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.621+0000 7f46d0e00640 1 --2- 172.21.15.152:0/1893406810 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f46cc0d0810 0x7f46cc15d350 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f46b800c400 tx=0x7f46b800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:34.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.621+0000 7f46c9600640 1 -- 172.21.15.152:0/1893406810 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b80052d0 con 0x7f46cc0d0810 2024-04-20T11:31:34.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.621+0000 7f46c9600640 1 -- 172.21.15.152:0/1893406810 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46b8005490 con 0x7f46cc0d0810 2024-04-20T11:31:34.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.621+0000 7f46d4200640 1 -- 172.21.15.152:0/1893406810 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46cc15e0d0 con 0x7f46cc0d0810 2024-04-20T11:31:34.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.622+0000 7f46c9600640 1 -- 172.21.15.152:0/1893406810 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b8007260 con 0x7f46cc0d0810 2024-04-20T11:31:34.623 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.622+0000 7f46d4200640 1 -- 172.21.15.152:0/1893406810 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46cc14c370 con 0x7f46cc0d0810 2024-04-20T11:31:34.623 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.623+0000 7f46d4200640 1 -- 172.21.15.152:0/1893406810 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46cc15e2a0 con 0x7f46cc0d0810 2024-04-20T11:31:34.623 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.624+0000 7f46c9600640 1 -- 172.21.15.152:0/1893406810 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 11) ==== 91346+0+0 (secure 0 0 0) 0x7f46b8007400 con 0x7f46cc0d0810 2024-04-20T11:31:34.625 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.625+0000 7f46c9600640 1 --2- 172.21.15.152:0/1893406810 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f46ac06cc50 0x7f46ac06f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:34.625 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.625+0000 7f46c9600640 1 -- 172.21.15.152:0/1893406810 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5709+0+0 (secure 0 0 0) 0x7f46b808b640 con 0x7f46cc0d0810 2024-04-20T11:31:34.625 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.625+0000 7f46d1800640 1 --2- 172.21.15.152:0/1893406810 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f46ac06cc50 0x7f46ac06f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:34.625 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.626+0000 7f46d1800640 1 --2- 172.21.15.152:0/1893406810 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f46ac06cc50 0x7f46ac06f110 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f46c0036000 tx=0x7f46c003c040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:34.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.630+0000 7f46c9600640 1 -- 172.21.15.152:0/1893406810 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f46cc15e2a0 con 0x7f46cc0d0810 2024-04-20T11:31:34.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.865+0000 7f46d4200640 1 -- 172.21.15.152:0/1893406810 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f46cc160030 con 0x7f46cc0d0810 2024-04-20T11:31:34.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.866+0000 7f46c9600640 1 -- 172.21.15.152:0/1893406810 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+5205 (secure 0 0 0) 0x7f46b8059660 con 0x7f46cc0d0810 2024-04-20T11:31:34.866 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:31:34.866 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5045,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/3268046291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3268046291},{"type":"v1","addr":"172.21.15.175:6835","nonce":3268046291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5347,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/845526345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":845526345},{"type":"v1","addr":"172.21.15.175:6833","nonce":845526345}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":6288,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6836/3582785355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":3582785355},{"type":"v1","addr":"172.21.15.152:6836","nonce":3582785355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T11:31:00.308725+0000","modified":"2024-04-20T11:31:03.369713+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6291":{"gid":6291,"name":"c","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6837/488955771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6835","nonce":488955771},{"type":"v1","addr":"172.21.15.152:6837","nonce":488955771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6291,"qdb_cluster":[6291]},"id":4}]} 2024-04-20T11:31:34.866 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 34 2024-04-20T11:31:34.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.868+0000 7f46d4200640 1 -- 172.21.15.152:0/1893406810 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f46ac06cc50 msgr2=0x7f46ac06f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:34.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.868+0000 7f46d4200640 1 --2- 172.21.15.152:0/1893406810 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f46ac06cc50 0x7f46ac06f110 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f46c0036000 tx=0x7f46c003c040 comp rx=0 tx=0).stop 2024-04-20T11:31:34.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.868+0000 7f46d4200640 1 -- 172.21.15.152:0/1893406810 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f46cc0d0810 msgr2=0x7f46cc15d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:34.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.868+0000 7f46d4200640 1 --2- 172.21.15.152:0/1893406810 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f46cc0d0810 0x7f46cc15d350 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f46b800c400 tx=0x7f46b800c8c0 comp rx=0 tx=0).stop 2024-04-20T11:31:34.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.868+0000 7f46d4200640 1 -- 172.21.15.152:0/1893406810 shutdown_connections 2024-04-20T11:31:34.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.868+0000 7f46d4200640 1 --2- 172.21.15.152:0/1893406810 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f46ac06cc50 0x7f46ac06f110 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:34.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.868+0000 7f46d4200640 1 --2- 172.21.15.152:0/1893406810 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f46cc15c560 0x7f46cc0cd610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:34.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.868+0000 7f46d4200640 1 --2- 172.21.15.152:0/1893406810 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f46cc0d0810 0x7f46cc15d350 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:34.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.868+0000 7f46d4200640 1 --2- 172.21.15.152:0/1893406810 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f46cc0b4330 0x7f46cc14be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:34.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.868+0000 7f46d4200640 1 -- 172.21.15.152:0/1893406810 >> 172.21.15.152:0/1893406810 conn(0x7f46cc067cf0 msgr2=0x7f46cc0b2150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:34.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.869+0000 7f46d4200640 1 -- 172.21.15.152:0/1893406810 shutdown_connections 2024-04-20T11:31:34.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:34.869+0000 7f46d4200640 1 -- 172.21.15.152:0/1893406810 wait complete. 2024-04-20T11:31:34.887 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:31:35.078 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.078+0000 7f5021000640 1 Processor -- start 2024-04-20T11:31:35.078 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.078+0000 7f5021000640 1 -- start start 2024-04-20T11:31:35.080 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.080+0000 7f5021000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f501c06f510 0x7f501c140dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:35.080 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.080+0000 7f5021000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f501c141310 0x7f501c071cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:35.080 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.080+0000 7f5021000640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f501c0cf0a0 con 0x7f501c13e530 2024-04-20T11:31:35.080 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.080+0000 7f5021000640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f501c141b00 con 0x7f501c06f510 2024-04-20T11:31:35.080 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.080+0000 7f5021000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f501c0cfc50 con 0x7f501c141310 2024-04-20T11:31:35.081 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.080+0000 7f501b400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f501c13e530 0x7f501c06ee00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:54878/0 (socket says 172.21.15.152:54878) 2024-04-20T11:31:35.081 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.080+0000 7f501b400640 1 -- 172.21.15.152:0/2295631364 learned_addr learned my addr 172.21.15.152:0/2295631364 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:31:35.081 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.080+0000 7f5012a00640 1 --2- 172.21.15.152:0/2295631364 >> v2:172.21.15.152:3300/0 conn(0x7f501c06f510 0x7f501c140dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:35.081 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.081+0000 7f501be00640 1 --2- 172.21.15.152:0/2295631364 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f501c141310 0x7f501c071cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:35.081 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.081+0000 7f5012a00640 1 -- 172.21.15.152:0/2295631364 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f501c141310 msgr2=0x7f501c071cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:35.081 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.081+0000 7f501aa00640 1 -- 172.21.15.152:0/2295631364 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3259389950 0 0) 0x7f500c001490 con 0x7f501c13e530 2024-04-20T11:31:35.081 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.081+0000 7f5012a00640 1 --2- 172.21.15.152:0/2295631364 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f501c141310 0x7f501c071cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:35.081 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.081+0000 7f5012a00640 1 -- 172.21.15.152:0/2295631364 >> v1:172.21.15.152:6789/0 conn(0x7f501c13e530 legacy=0x7f501c06ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:35.081 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.081+0000 7f5012a00640 1 -- 172.21.15.152:0/2295631364 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f501c0d00e0 con 0x7f501c06f510 2024-04-20T11:31:35.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.082+0000 7f5012a00640 1 --2- 172.21.15.152:0/2295631364 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f501c06f510 0x7f501c140dd0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f5004009800 tx=0x7f500402dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=831d81ab7fe4901c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:35.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.083+0000 7f501aa00640 1 -- 172.21.15.152:0/2295631364 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f500402f8a0 con 0x7f501c06f510 2024-04-20T11:31:35.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.083+0000 7f501aa00640 1 -- 172.21.15.152:0/2295631364 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f500402fa40 con 0x7f501c06f510 2024-04-20T11:31:35.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.083+0000 7f501aa00640 1 -- 172.21.15.152:0/2295631364 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f500402fe10 con 0x7f501c06f510 2024-04-20T11:31:35.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.083+0000 7f5021000640 1 -- 172.21.15.152:0/2295631364 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f501c06f510 msgr2=0x7f501c140dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:35.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.083+0000 7f5021000640 1 --2- 172.21.15.152:0/2295631364 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f501c06f510 0x7f501c140dd0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f5004009800 tx=0x7f500402dae0 comp rx=0 tx=0).stop 2024-04-20T11:31:35.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.083+0000 7f5021000640 1 -- 172.21.15.152:0/2295631364 shutdown_connections 2024-04-20T11:31:35.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.083+0000 7f5021000640 1 --2- 172.21.15.152:0/2295631364 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f501c141310 0x7f501c071cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:35.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.083+0000 7f5021000640 1 --2- 172.21.15.152:0/2295631364 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f501c06f510 0x7f501c140dd0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:35.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.083+0000 7f5021000640 1 -- 172.21.15.152:0/2295631364 >> 172.21.15.152:0/2295631364 conn(0x7f501c067ae0 msgr2=0x7f501c06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:35.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.083+0000 7f5021000640 1 -- 172.21.15.152:0/2295631364 shutdown_connections 2024-04-20T11:31:35.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.083+0000 7f5021000640 1 -- 172.21.15.152:0/2295631364 wait complete. 2024-04-20T11:31:35.084 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.085+0000 7f5021000640 1 Processor -- start 2024-04-20T11:31:35.084 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.085+0000 7f5021000640 1 -- start start 2024-04-20T11:31:35.086 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.086+0000 7f5021000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f501c06f510 0x7f501c0b2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:35.086 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.086+0000 7f5021000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f501c13e530 0x7f501c13db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:35.086 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.086+0000 7f5021000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f501c141310 0x7f501c0b09c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:35.086 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.086+0000 7f5021000640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f501c0cf8e0 con 0x7f501c06f510 2024-04-20T11:31:35.086 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.086+0000 7f5021000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f501c0cf600 con 0x7f501c13e530 2024-04-20T11:31:35.086 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.086+0000 7f5021000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f501c1417c0 con 0x7f501c141310 2024-04-20T11:31:35.086 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.086+0000 7f501b400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f501c06f510 0x7f501c0b2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:35.086 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.086+0000 7f501b400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f501c06f510 0x7f501c0b2490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54576/0 (socket says 172.21.15.152:54576) 2024-04-20T11:31:35.086 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.087+0000 7f501b400640 1 -- 172.21.15.152:0/4085827447 learned_addr learned my addr 172.21.15.152:0/4085827447 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:35.088 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.087+0000 7f501be00640 1 --2- 172.21.15.152:0/4085827447 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f501c141310 0x7f501c0b09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:35.088 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.087+0000 7f5012a00640 1 --2- 172.21.15.152:0/4085827447 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f501c13e530 0x7f501c13db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:35.088 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.087+0000 7f501b400640 1 -- 172.21.15.152:0/4085827447 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f501c141310 msgr2=0x7f501c0b09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:35.088 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.087+0000 7f501b400640 1 --2- 172.21.15.152:0/4085827447 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f501c141310 0x7f501c0b09c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:35.088 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.087+0000 7f501b400640 1 -- 172.21.15.152:0/4085827447 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f501c13e530 msgr2=0x7f501c13db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:35.088 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.087+0000 7f501b400640 1 --2- 172.21.15.152:0/4085827447 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f501c13e530 0x7f501c13db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:35.088 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.087+0000 7f501b400640 1 -- 172.21.15.152:0/4085827447 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f501c0b13f0 con 0x7f501c06f510 2024-04-20T11:31:35.088 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.087+0000 7f5012a00640 1 --2- 172.21.15.152:0/4085827447 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f501c13e530 0x7f501c13db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:31:35.088 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.087+0000 7f501b400640 1 --2- 172.21.15.152:0/4085827447 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f501c06f510 0x7f501c0b2490 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f500c00c850 tx=0x7f500c00cd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:35.088 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.088+0000 7f5013e00640 1 -- 172.21.15.152:0/4085827447 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f500c018070 con 0x7f501c06f510 2024-04-20T11:31:35.088 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.088+0000 7f5013e00640 1 -- 172.21.15.152:0/4085827447 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f500c00a640 con 0x7f501c06f510 2024-04-20T11:31:35.088 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.088+0000 7f5013e00640 1 -- 172.21.15.152:0/4085827447 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f500c00aa30 con 0x7f501c06f510 2024-04-20T11:31:35.088 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.088+0000 7f5021000640 1 -- 172.21.15.152:0/4085827447 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f501c0b16d0 con 0x7f501c06f510 2024-04-20T11:31:35.089 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.088+0000 7f5021000640 1 -- 172.21.15.152:0/4085827447 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f501c0b1c90 con 0x7f501c06f510 2024-04-20T11:31:35.089 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.089+0000 7f5021000640 1 -- 172.21.15.152:0/4085827447 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f501c075e60 con 0x7f501c06f510 2024-04-20T11:31:35.090 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.090+0000 7f5013e00640 1 -- 172.21.15.152:0/4085827447 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 11) ==== 91346+0+0 (secure 0 0 0) 0x7f500c00abd0 con 0x7f501c06f510 2024-04-20T11:31:35.090 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.091+0000 7f5013e00640 1 --2- 172.21.15.152:0/4085827447 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f4ff806cc50 0x7f4ff806f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:35.091 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.091+0000 7f5013e00640 1 -- 172.21.15.152:0/4085827447 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5709+0+0 (secure 0 0 0) 0x7f500c08b7d0 con 0x7f501c06f510 2024-04-20T11:31:35.091 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.091+0000 7f5012a00640 1 --2- 172.21.15.152:0/4085827447 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f4ff806cc50 0x7f4ff806f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:35.092 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.092+0000 7f5012a00640 1 --2- 172.21.15.152:0/4085827447 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f4ff806cc50 0x7f4ff806f110 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f5004038000 tx=0x7f5004008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:35.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.096+0000 7f5013e00640 1 -- 172.21.15.152:0/4085827447 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f501c0cf0a0 con 0x7f501c06f510 2024-04-20T11:31:35.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.296+0000 7f5021000640 1 -- 172.21.15.152:0/4085827447 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f501c0b3a00 con 0x7f501c06f510 2024-04-20T11:31:35.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.298+0000 7f5013e00640 1 -- 172.21.15.152:0/4085827447 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) ==== 74+0+16725 (secure 0 0 0) 0x7f500c0597f0 con 0x7f501c06f510 2024-04-20T11:31:35.298 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:31:35.298 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":60,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:31:01.845268+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:30:58.087822+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-20T11:30:59.281767+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:31:35.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.301+0000 7f5021000640 1 -- 172.21.15.152:0/4085827447 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f4ff806cc50 msgr2=0x7f4ff806f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:35.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.301+0000 7f5021000640 1 --2- 172.21.15.152:0/4085827447 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f4ff806cc50 0x7f4ff806f110 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f5004038000 tx=0x7f5004008030 comp rx=0 tx=0).stop 2024-04-20T11:31:35.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.301+0000 7f5021000640 1 -- 172.21.15.152:0/4085827447 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f501c06f510 msgr2=0x7f501c0b2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:35.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.301+0000 7f5021000640 1 --2- 172.21.15.152:0/4085827447 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f501c06f510 0x7f501c0b2490 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f500c00c850 tx=0x7f500c00cd10 comp rx=0 tx=0).stop 2024-04-20T11:31:35.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.301+0000 7f5021000640 1 -- 172.21.15.152:0/4085827447 shutdown_connections 2024-04-20T11:31:35.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.301+0000 7f5021000640 1 --2- 172.21.15.152:0/4085827447 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f4ff806cc50 0x7f4ff806f110 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:35.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.301+0000 7f5021000640 1 --2- 172.21.15.152:0/4085827447 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f501c141310 0x7f501c0b09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:35.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.301+0000 7f5021000640 1 --2- 172.21.15.152:0/4085827447 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f501c13e530 0x7f501c13db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:35.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.301+0000 7f5021000640 1 --2- 172.21.15.152:0/4085827447 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f501c06f510 0x7f501c0b2490 secure :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f500c00c850 tx=0x7f500c00cd10 comp rx=0 tx=0).stop 2024-04-20T11:31:35.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.301+0000 7f5021000640 1 -- 172.21.15.152:0/4085827447 >> 172.21.15.152:0/4085827447 conn(0x7f501c067ae0 msgr2=0x7f501c0b5170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:35.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.301+0000 7f5021000640 1 -- 172.21.15.152:0/4085827447 shutdown_connections 2024-04-20T11:31:35.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.301+0000 7f5021000640 1 -- 172.21.15.152:0/4085827447 wait complete. 2024-04-20T11:31:35.320 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-20T11:31:35.320 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:31:35.510 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.511+0000 7f4888c00640 1 Processor -- start 2024-04-20T11:31:35.511 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.511+0000 7f4888c00640 1 -- start start 2024-04-20T11:31:35.512 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.512+0000 7f4888c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4884151ab0 0x7f4884072560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:35.512 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.513+0000 7f4888c00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f488415b340 con 0x7f4884072aa0 2024-04-20T11:31:35.512 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.513+0000 7f4888c00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f488415b620 con 0x7f48840b3cd0 2024-04-20T11:31:35.512 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.513+0000 7f4888c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f488415c100 con 0x7f4884151ab0 2024-04-20T11:31:35.513 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.513+0000 7f487aa00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f4884072aa0 0x7f48840b35c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:54880/0 (socket says 172.21.15.152:54880) 2024-04-20T11:31:35.513 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.513+0000 7f487aa00640 1 -- 172.21.15.152:0/920754813 learned_addr learned my addr 172.21.15.152:0/920754813 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:31:35.514 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.513+0000 7f4882a00640 1 -- 172.21.15.152:0/920754813 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 168054014 0 0) 0x7f488415b340 con 0x7f4884072aa0 2024-04-20T11:31:35.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.513+0000 7f4882a00640 1 -- 172.21.15.152:0/920754813 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4868003520 con 0x7f4884072aa0 2024-04-20T11:31:35.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.513+0000 7f4883400640 1 --2- 172.21.15.152:0/920754813 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4884151ab0 0x7f4884072560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:35.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.514+0000 7f4882a00640 1 -- 172.21.15.152:0/920754813 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2916535306 0 0) 0x7f4868003520 con 0x7f4884072aa0 2024-04-20T11:31:35.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.514+0000 7f4882a00640 1 -- 172.21.15.152:0/920754813 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4884151ab0 msgr2=0x7f4884072560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:35.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.514+0000 7f4882a00640 1 --2- 172.21.15.152:0/920754813 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4884151ab0 0x7f4884072560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:35.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.514+0000 7f4882a00640 1 -- 172.21.15.152:0/920754813 >> v1:172.21.15.175:6789/0 conn(0x7f48840b3cd0 legacy=0x7f488415e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:35.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.514+0000 7f4882a00640 1 -- 172.21.15.152:0/920754813 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f488415b900 con 0x7f4884072aa0 2024-04-20T11:31:35.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.514+0000 7f4882a00640 1 -- 172.21.15.152:0/920754813 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f486c001670 con 0x7f4884072aa0 2024-04-20T11:31:35.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.514+0000 7f4882a00640 1 -- 172.21.15.152:0/920754813 >> v1:172.21.15.152:6789/0 conn(0x7f4884072aa0 legacy=0x7f48840b35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:35.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.514+0000 7f4882a00640 1 --2- 172.21.15.152:0/920754813 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48680039e0 0x7f4868003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:35.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.514+0000 7f4882a00640 1 --2- 172.21.15.152:0/920754813 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4884151ab0 0x7f4868004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:35.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.514+0000 7f4882a00640 1 --2- 172.21.15.152:0/920754813 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4868004d40 0x7f48680051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:35.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.514+0000 7f4882a00640 1 -- 172.21.15.152:0/920754813 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f488415c100 con 0x7f4868004d40 2024-04-20T11:31:35.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.514+0000 7f4882a00640 1 -- 172.21.15.152:0/920754813 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4868005b70 con 0x7f48680039e0 2024-04-20T11:31:35.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.514+0000 7f4882a00640 1 -- 172.21.15.152:0/920754813 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4868005d70 con 0x7f4884151ab0 2024-04-20T11:31:35.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.514+0000 7f4883e00640 1 --2- 172.21.15.152:0/920754813 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4868004d40 0x7f48680051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:35.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.514+0000 7f4883400640 1 --2- 172.21.15.152:0/920754813 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48680039e0 0x7f4868003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:35.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.514+0000 7f487aa00640 1 --2- 172.21.15.152:0/920754813 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4884151ab0 0x7f4868004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:35.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.515+0000 7f4883e00640 1 -- 172.21.15.152:0/920754813 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4884151ab0 msgr2=0x7f4868004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:35.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.515+0000 7f4883e00640 1 --2- 172.21.15.152:0/920754813 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4884151ab0 0x7f4868004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:35.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.515+0000 7f4883e00640 1 -- 172.21.15.152:0/920754813 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48680039e0 msgr2=0x7f4868003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:35.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.515+0000 7f4883e00640 1 --2- 172.21.15.152:0/920754813 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48680039e0 0x7f4868003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:35.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.515+0000 7f4883e00640 1 -- 172.21.15.152:0/920754813 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4868003520 con 0x7f4868004d40 2024-04-20T11:31:35.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.515+0000 7f487aa00640 1 --2- 172.21.15.152:0/920754813 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4884151ab0 0x7f4868004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:31:35.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.516+0000 7f4883e00640 1 --2- 172.21.15.152:0/920754813 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4868004d40 0x7f48680051c0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f487c001e50 tx=0x7f4868005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f46bc2d9cbdfadde server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:35.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.516+0000 7f4882a00640 1 -- 172.21.15.152:0/920754813 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f487c001a70 con 0x7f4868004d40 2024-04-20T11:31:35.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.516+0000 7f4882a00640 1 -- 172.21.15.152:0/920754813 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f487c003b70 con 0x7f4868004d40 2024-04-20T11:31:35.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.516+0000 7f4882a00640 1 -- 172.21.15.152:0/920754813 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f487c0050d0 con 0x7f4868004d40 2024-04-20T11:31:35.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.516+0000 7f4888c00640 1 -- 172.21.15.152:0/920754813 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4868004d40 msgr2=0x7f48680051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:35.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.517+0000 7f4888c00640 1 --2- 172.21.15.152:0/920754813 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4868004d40 0x7f48680051c0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f487c001e50 tx=0x7f4868005830 comp rx=0 tx=0).stop 2024-04-20T11:31:35.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.517+0000 7f4883e00640 1 -- 172.21.15.152:0/920754813 reap_dead start 2024-04-20T11:31:35.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.517+0000 7f4888c00640 1 -- 172.21.15.152:0/920754813 shutdown_connections 2024-04-20T11:31:35.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.517+0000 7f4888c00640 1 -- 172.21.15.152:0/920754813 >> 172.21.15.152:0/920754813 conn(0x7f4884067ae0 msgr2=0x7f488406b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:35.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.517+0000 7f4888c00640 1 -- 172.21.15.152:0/920754813 shutdown_connections 2024-04-20T11:31:35.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.517+0000 7f4888c00640 1 -- 172.21.15.152:0/920754813 wait complete. 2024-04-20T11:31:35.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.518+0000 7f4888c00640 1 Processor -- start 2024-04-20T11:31:35.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.519+0000 7f4888c00640 1 -- start start 2024-04-20T11:31:35.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.519+0000 7f4888c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4868004d40 0x7f48841e5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:35.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.519+0000 7f4888c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48840a9660 0x7f48841e5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:35.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.519+0000 7f4888c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f488414fcc0 0x7f4884146600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:35.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.519+0000 7f4888c00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f488415bc70 con 0x7f4868004d40 2024-04-20T11:31:35.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.519+0000 7f4888c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4884151e90 con 0x7f48840a9660 2024-04-20T11:31:35.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.519+0000 7f4888c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4884152010 con 0x7f488414fcc0 2024-04-20T11:31:35.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.520+0000 7f4883400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4868004d40 0x7f48841e5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:35.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.520+0000 7f4883400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4868004d40 0x7f48841e5140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54598/0 (socket says 172.21.15.152:54598) 2024-04-20T11:31:35.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.520+0000 7f4883400640 1 -- 172.21.15.152:0/2126940828 learned_addr learned my addr 172.21.15.152:0/2126940828 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:35.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.520+0000 7f4883400640 1 -- 172.21.15.152:0/2126940828 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f488414fcc0 msgr2=0x7f4884146600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:35.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.520+0000 7f4883400640 1 --2- 172.21.15.152:0/2126940828 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f488414fcc0 0x7f4884146600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:35.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.520+0000 7f4883400640 1 -- 172.21.15.152:0/2126940828 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48840a9660 msgr2=0x7f48841e5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:35.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.520+0000 7f4883400640 1 --2- 172.21.15.152:0/2126940828 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48840a9660 0x7f48841e5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:35.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.520+0000 7f4883400640 1 -- 172.21.15.152:0/2126940828 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48841e40a0 con 0x7f4868004d40 2024-04-20T11:31:35.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.520+0000 7f4883400640 1 --2- 172.21.15.152:0/2126940828 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4868004d40 0x7f48841e5140 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f487400f580 tx=0x7f487400fa40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:35.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.521+0000 7f487be00640 1 -- 172.21.15.152:0/2126940828 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4874019020 con 0x7f4868004d40 2024-04-20T11:31:35.522 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.521+0000 7f487be00640 1 -- 172.21.15.152:0/2126940828 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4874014870 con 0x7f4868004d40 2024-04-20T11:31:35.522 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.521+0000 7f4888c00640 1 -- 172.21.15.152:0/2126940828 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48841e4380 con 0x7f4868004d40 2024-04-20T11:31:35.522 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.521+0000 7f487be00640 1 -- 172.21.15.152:0/2126940828 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4874014c60 con 0x7f4868004d40 2024-04-20T11:31:35.522 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.521+0000 7f4888c00640 1 -- 172.21.15.152:0/2126940828 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48841e4940 con 0x7f4868004d40 2024-04-20T11:31:35.522 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.522+0000 7f4888c00640 1 -- 172.21.15.152:0/2126940828 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4884075e60 con 0x7f4868004d40 2024-04-20T11:31:35.522 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.523+0000 7f487be00640 1 -- 172.21.15.152:0/2126940828 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 11) ==== 91346+0+0 (secure 0 0 0) 0x7f487401c020 con 0x7f4868004d40 2024-04-20T11:31:35.523 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.524+0000 7f487be00640 1 --2- 172.21.15.152:0/2126940828 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f486006cc50 0x7f486006f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:35.523 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.524+0000 7f487be00640 1 -- 172.21.15.152:0/2126940828 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5709+0+0 (secure 0 0 0) 0x7f487408c9a0 con 0x7f4868004d40 2024-04-20T11:31:35.523 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.524+0000 7f487aa00640 1 --2- 172.21.15.152:0/2126940828 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f486006cc50 0x7f486006f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:35.524 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.525+0000 7f487aa00640 1 --2- 172.21.15.152:0/2126940828 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f486006cc50 0x7f486006f110 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f486c001200 tx=0x7f486c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:35.527 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.528+0000 7f487be00640 1 -- 172.21.15.152:0/2126940828 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f4874023020 con 0x7f4868004d40 2024-04-20T11:31:35.770 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.771+0000 7f4888c00640 1 -- 172.21.15.152:0/2126940828 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f48840b2ec0 con 0x7f4868004d40 2024-04-20T11:31:35.772 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.772+0000 7f487be00640 1 -- 172.21.15.152:0/2126940828 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+5205 (secure 0 0 0) 0x7f4874026350 con 0x7f4868004d40 2024-04-20T11:31:35.772 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:31:35.772 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5045,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/3268046291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3268046291},{"type":"v1","addr":"172.21.15.175:6835","nonce":3268046291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5347,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/845526345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":845526345},{"type":"v1","addr":"172.21.15.175:6833","nonce":845526345}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":6288,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6836/3582785355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":3582785355},{"type":"v1","addr":"172.21.15.152:6836","nonce":3582785355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T11:31:00.308725+0000","modified":"2024-04-20T11:31:03.369713+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6291":{"gid":6291,"name":"c","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6837/488955771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6835","nonce":488955771},{"type":"v1","addr":"172.21.15.152:6837","nonce":488955771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6291,"qdb_cluster":[6291]},"id":4}]} 2024-04-20T11:31:35.773 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 34 2024-04-20T11:31:35.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.775+0000 7f4888c00640 1 -- 172.21.15.152:0/2126940828 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f486006cc50 msgr2=0x7f486006f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:35.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.775+0000 7f4888c00640 1 --2- 172.21.15.152:0/2126940828 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f486006cc50 0x7f486006f110 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f486c001200 tx=0x7f486c009040 comp rx=0 tx=0).stop 2024-04-20T11:31:35.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.775+0000 7f4888c00640 1 -- 172.21.15.152:0/2126940828 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4868004d40 msgr2=0x7f48841e5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:35.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.775+0000 7f4888c00640 1 --2- 172.21.15.152:0/2126940828 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4868004d40 0x7f48841e5140 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f487400f580 tx=0x7f487400fa40 comp rx=0 tx=0).stop 2024-04-20T11:31:35.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.775+0000 7f4888c00640 1 -- 172.21.15.152:0/2126940828 shutdown_connections 2024-04-20T11:31:35.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.775+0000 7f4888c00640 1 --2- 172.21.15.152:0/2126940828 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f486006cc50 0x7f486006f110 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:35.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.775+0000 7f4888c00640 1 --2- 172.21.15.152:0/2126940828 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f488414fcc0 0x7f4884146600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:35.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.775+0000 7f4888c00640 1 --2- 172.21.15.152:0/2126940828 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48840a9660 0x7f48841e5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:35.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.775+0000 7f4888c00640 1 --2- 172.21.15.152:0/2126940828 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4868004d40 0x7f48841e5140 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:35.776 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.775+0000 7f4888c00640 1 -- 172.21.15.152:0/2126940828 >> 172.21.15.152:0/2126940828 conn(0x7f4884067ae0 msgr2=0x7f48840cea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:35.776 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.775+0000 7f4888c00640 1 -- 172.21.15.152:0/2126940828 shutdown_connections 2024-04-20T11:31:35.776 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.776+0000 7f4888c00640 1 -- 172.21.15.152:0/2126940828 wait complete. 2024-04-20T11:31:35.795 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:31:35.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.988+0000 7f06b7400640 1 Processor -- start 2024-04-20T11:31:35.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.988+0000 7f06b7400640 1 -- start start 2024-04-20T11:31:35.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.989+0000 7f06b7400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f06b006eb40 0x7f06b006f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:35.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.989+0000 7f06b7400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f06b006f880 0x7f06b015cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:35.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.989+0000 7f06b7400640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06b00b5770 con 0x7f06b015b5e0 2024-04-20T11:31:35.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.989+0000 7f06b7400640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f06b006fcd0 con 0x7f06b006f880 2024-04-20T11:31:35.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.989+0000 7f06b7400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f06b006de10 con 0x7f06b006eb40 2024-04-20T11:31:35.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.990+0000 7f06b5400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f06b006f880 0x7f06b015cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:35.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.990+0000 7f06b5400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f06b006f880 0x7f06b015cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49476/0 (socket says 172.21.15.152:49476) 2024-04-20T11:31:35.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.990+0000 7f06b5400640 1 -- 172.21.15.152:0/3989960768 learned_addr learned my addr 172.21.15.152:0/3989960768 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:35.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.990+0000 7f06afe00640 1 --2- 172.21.15.152:0/3989960768 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f06b006eb40 0x7f06b006f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:35.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.990+0000 7f06b5400640 1 -- 172.21.15.152:0/3989960768 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f06b006eb40 msgr2=0x7f06b006f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:35.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.990+0000 7f06b5400640 1 --2- 172.21.15.152:0/3989960768 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f06b006eb40 0x7f06b006f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:35.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.990+0000 7f06b5400640 1 -- 172.21.15.152:0/3989960768 >> v1:172.21.15.152:6789/0 conn(0x7f06b015b5e0 legacy=0x7f06b00b4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:35.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.990+0000 7f06b5400640 1 -- 172.21.15.152:0/3989960768 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06b006e060 con 0x7f06b006f880 2024-04-20T11:31:35.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.990+0000 7f06af400640 1 -- 172.21.15.152:0/3989960768 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2310032538 0 0) 0x7f06b00b5770 con 0x7f06b015b5e0 2024-04-20T11:31:35.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.990+0000 7f06afe00640 1 --2- 172.21.15.152:0/3989960768 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f06b006eb40 0x7f06b006f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:31:35.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.991+0000 7f06b5400640 1 --2- 172.21.15.152:0/3989960768 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f06b006f880 0x7f06b015cd70 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f06a8009e50 tx=0x7f06a802bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de2bb06f93edc4c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:35.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.992+0000 7f06af400640 1 -- 172.21.15.152:0/3989960768 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06a802cb00 con 0x7f06b006f880 2024-04-20T11:31:35.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.992+0000 7f06af400640 1 -- 172.21.15.152:0/3989960768 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06a802cca0 con 0x7f06b006f880 2024-04-20T11:31:35.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.992+0000 7f06af400640 1 -- 172.21.15.152:0/3989960768 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06a8034610 con 0x7f06b006f880 2024-04-20T11:31:35.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.992+0000 7f06b7400640 1 -- 172.21.15.152:0/3989960768 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f06b006f880 msgr2=0x7f06b015cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:35.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.992+0000 7f06b7400640 1 --2- 172.21.15.152:0/3989960768 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f06b006f880 0x7f06b015cd70 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f06a8009e50 tx=0x7f06a802bf60 comp rx=0 tx=0).stop 2024-04-20T11:31:35.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.992+0000 7f06b7400640 1 -- 172.21.15.152:0/3989960768 shutdown_connections 2024-04-20T11:31:35.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.992+0000 7f06b7400640 1 --2- 172.21.15.152:0/3989960768 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f06b006eb40 0x7f06b006f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:35.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.992+0000 7f06b7400640 1 --2- 172.21.15.152:0/3989960768 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f06b006f880 0x7f06b015cd70 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:35.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.992+0000 7f06b7400640 1 -- 172.21.15.152:0/3989960768 >> 172.21.15.152:0/3989960768 conn(0x7f06b0067cf0 msgr2=0x7f06b006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:35.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.992+0000 7f06b7400640 1 -- 172.21.15.152:0/3989960768 shutdown_connections 2024-04-20T11:31:35.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.992+0000 7f06b7400640 1 -- 172.21.15.152:0/3989960768 wait complete. 2024-04-20T11:31:35.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.994+0000 7f06b7400640 1 Processor -- start 2024-04-20T11:31:35.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.994+0000 7f06b7400640 1 -- start start 2024-04-20T11:31:35.995 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.995+0000 7f06b7400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f06b006eb40 0x7f06b00aee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:35.995 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.995+0000 7f06b7400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f06b006f880 0x7f06b00cf8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:35.995 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.995+0000 7f06b7400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f06b015b5e0 0x7f06b01568a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:35.995 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.995+0000 7f06b7400640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f06b015c9d0 con 0x7f06b006eb40 2024-04-20T11:31:35.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.995+0000 7f06b7400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f06b006e620 con 0x7f06b015b5e0 2024-04-20T11:31:35.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.995+0000 7f06b7400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f06b00b42d0 con 0x7f06b006f880 2024-04-20T11:31:35.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.995+0000 7f06b4a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f06b006eb40 0x7f06b00aee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:35.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.996+0000 7f06b4a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f06b006eb40 0x7f06b00aee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49478/0 (socket says 172.21.15.152:49478) 2024-04-20T11:31:35.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.996+0000 7f06b4a00640 1 -- 172.21.15.152:0/678824710 learned_addr learned my addr 172.21.15.152:0/678824710 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:35.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.996+0000 7f06b4a00640 1 -- 172.21.15.152:0/678824710 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f06b006f880 msgr2=0x7f06b00cf8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:35.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.996+0000 7f06b4a00640 1 --2- 172.21.15.152:0/678824710 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f06b006f880 0x7f06b00cf8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:35.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.996+0000 7f06b4a00640 1 -- 172.21.15.152:0/678824710 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f06b015b5e0 msgr2=0x7f06b01568a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:35.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.996+0000 7f06b4a00640 1 --2- 172.21.15.152:0/678824710 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f06b015b5e0 0x7f06b01568a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:35.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.996+0000 7f06b4a00640 1 -- 172.21.15.152:0/678824710 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06b00ceb00 con 0x7f06b006eb40 2024-04-20T11:31:35.997 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.996+0000 7f06b4a00640 1 --2- 172.21.15.152:0/678824710 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f06b006eb40 0x7f06b00aee90 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f06a400c4b0 tx=0x7f06a400c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:35.997 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.996+0000 7f06acc00640 1 -- 172.21.15.152:0/678824710 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06a4017070 con 0x7f06b006eb40 2024-04-20T11:31:35.997 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.996+0000 7f06acc00640 1 -- 172.21.15.152:0/678824710 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06a4005480 con 0x7f06b006eb40 2024-04-20T11:31:35.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.997+0000 7f06acc00640 1 -- 172.21.15.152:0/678824710 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06a4003830 con 0x7f06b006eb40 2024-04-20T11:31:35.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.997+0000 7f06b7400640 1 -- 172.21.15.152:0/678824710 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06b00cede0 con 0x7f06b006eb40 2024-04-20T11:31:35.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.997+0000 7f06b7400640 1 -- 172.21.15.152:0/678824710 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06b00cf3a0 con 0x7f06b006eb40 2024-04-20T11:31:35.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.998+0000 7f06b7400640 1 -- 172.21.15.152:0/678824710 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06b00689f0 con 0x7f06b006eb40 2024-04-20T11:31:35.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:35.999+0000 7f06acc00640 1 -- 172.21.15.152:0/678824710 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 11) ==== 91346+0+0 (secure 0 0 0) 0x7f06a401a070 con 0x7f06b006eb40 2024-04-20T11:31:35.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.000+0000 7f06acc00640 1 --2- 172.21.15.152:0/678824710 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f069006cc50 0x7f069006f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:35.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.000+0000 7f06acc00640 1 -- 172.21.15.152:0/678824710 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5709+0+0 (secure 0 0 0) 0x7f06a408b3f0 con 0x7f06b006eb40 2024-04-20T11:31:35.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.000+0000 7f06afe00640 1 --2- 172.21.15.152:0/678824710 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f069006cc50 0x7f069006f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:36.000 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.001+0000 7f06afe00640 1 --2- 172.21.15.152:0/678824710 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f069006cc50 0x7f069006f110 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f069c006fd0 tx=0x7f069c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:36.003 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.004+0000 7f06acc00640 1 -- 172.21.15.152:0/678824710 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f06a4059460 con 0x7f06b006eb40 2024-04-20T11:31:36.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.242+0000 7f06b7400640 1 -- 172.21.15.152:0/678824710 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f06b00cf5e0 con 0x7f06b006eb40 2024-04-20T11:31:36.243 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.243+0000 7f06acc00640 1 -- 172.21.15.152:0/678824710 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+5205 (secure 0 0 0) 0x7f06a4058d60 con 0x7f06b006eb40 2024-04-20T11:31:36.243 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:31:36.243 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5045,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/3268046291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3268046291},{"type":"v1","addr":"172.21.15.175:6835","nonce":3268046291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5347,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/845526345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":845526345},{"type":"v1","addr":"172.21.15.175:6833","nonce":845526345}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":6288,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6836/3582785355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":3582785355},{"type":"v1","addr":"172.21.15.152:6836","nonce":3582785355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T11:31:00.308725+0000","modified":"2024-04-20T11:31:03.369713+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6291":{"gid":6291,"name":"c","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6837/488955771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6835","nonce":488955771},{"type":"v1","addr":"172.21.15.152:6837","nonce":488955771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6291,"qdb_cluster":[6291]},"id":4}]} 2024-04-20T11:31:36.243 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 34 2024-04-20T11:31:36.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.246+0000 7f06b7400640 1 -- 172.21.15.152:0/678824710 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f069006cc50 msgr2=0x7f069006f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:36.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.246+0000 7f06b7400640 1 --2- 172.21.15.152:0/678824710 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f069006cc50 0x7f069006f110 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f069c006fd0 tx=0x7f069c008040 comp rx=0 tx=0).stop 2024-04-20T11:31:36.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.246+0000 7f06b7400640 1 -- 172.21.15.152:0/678824710 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f06b006eb40 msgr2=0x7f06b00aee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:36.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.246+0000 7f06b7400640 1 --2- 172.21.15.152:0/678824710 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f06b006eb40 0x7f06b00aee90 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f06a400c4b0 tx=0x7f06a400c970 comp rx=0 tx=0).stop 2024-04-20T11:31:36.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.246+0000 7f06b7400640 1 -- 172.21.15.152:0/678824710 shutdown_connections 2024-04-20T11:31:36.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.246+0000 7f06b7400640 1 --2- 172.21.15.152:0/678824710 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f069006cc50 0x7f069006f110 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:36.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.246+0000 7f06b7400640 1 --2- 172.21.15.152:0/678824710 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f06b015b5e0 0x7f06b01568a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:36.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.246+0000 7f06b7400640 1 --2- 172.21.15.152:0/678824710 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f06b006f880 0x7f06b00cf8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:36.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.246+0000 7f06b7400640 1 --2- 172.21.15.152:0/678824710 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f06b006eb40 0x7f06b00aee90 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:36.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.246+0000 7f06b7400640 1 -- 172.21.15.152:0/678824710 >> 172.21.15.152:0/678824710 conn(0x7f06b0067cf0 msgr2=0x7f06b00b18c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:36.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.246+0000 7f06b7400640 1 -- 172.21.15.152:0/678824710 shutdown_connections 2024-04-20T11:31:36.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.246+0000 7f06b7400640 1 -- 172.21.15.152:0/678824710 wait complete. 2024-04-20T11:31:36.265 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:31:36.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.460+0000 7f6c48600640 1 Processor -- start 2024-04-20T11:31:36.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.461+0000 7f6c48600640 1 -- start start 2024-04-20T11:31:36.462 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.462+0000 7f6c48600640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f6c40160b80 0x7f6c400d0b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:36.462 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.462+0000 7f6c48600640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f6c4009bb80 0x7f6c400d10b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:36.462 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.462+0000 7f6c48600640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c401612a0 con 0x7f6c4015c810 2024-04-20T11:31:36.462 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.462+0000 7f6c48600640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f6c4009aae0 con 0x7f6c4009bb80 2024-04-20T11:31:36.462 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.462+0000 7f6c48600640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f6c4009adc0 con 0x7f6c40160b80 2024-04-20T11:31:36.462 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.462+0000 7f6c46600640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f6c4009bb80 0x7f6c400d10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:36.462 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.462+0000 7f6c46600640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f6c4009bb80 0x7f6c400d10b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49492/0 (socket says 172.21.15.152:49492) 2024-04-20T11:31:36.462 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.462+0000 7f6c46600640 1 -- 172.21.15.152:0/2089499893 learned_addr learned my addr 172.21.15.152:0/2089499893 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:36.462 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.462+0000 7f6c45c00640 1 --1- 172.21.15.152:0/2089499893 >> v1:172.21.15.152:6789/0 conn(0x7f6c4015c810 0x7f6c40160470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:41314/0 (socket says 172.21.15.152:41314) 2024-04-20T11:31:36.463 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.463+0000 7f6c45200640 1 --2- 172.21.15.152:0/2089499893 >> v2:172.21.15.175:3300/0 conn(0x7f6c40160b80 0x7f6c400d0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:36.463 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.463+0000 7f6c46600640 1 -- 172.21.15.152:0/2089499893 >> v2:172.21.15.175:3300/0 conn(0x7f6c40160b80 msgr2=0x7f6c400d0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:36.463 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.463+0000 7f6c46600640 1 --2- 172.21.15.152:0/2089499893 >> v2:172.21.15.175:3300/0 conn(0x7f6c40160b80 0x7f6c400d0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:36.463 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.463+0000 7f6c46600640 1 -- 172.21.15.152:0/2089499893 >> v1:172.21.15.152:6789/0 conn(0x7f6c4015c810 legacy=0x7f6c40160470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:36.463 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.463+0000 7f6c46600640 1 -- 172.21.15.152:0/2089499893 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c4009b0a0 con 0x7f6c4009bb80 2024-04-20T11:31:36.463 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.463+0000 7f6c45200640 1 --2- 172.21.15.152:0/2089499893 >> v2:172.21.15.175:3300/0 conn(0x7f6c40160b80 0x7f6c400d0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:31:36.464 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.464+0000 7f6c46600640 1 --2- 172.21.15.152:0/2089499893 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6c4009bb80 0x7f6c400d10b0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f6c400736a0 tx=0x7f6c3802c010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65fd767baee7230f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:36.465 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.464+0000 7f6c3fe00640 1 -- 172.21.15.152:0/2089499893 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c3802cbb0 con 0x7f6c4009bb80 2024-04-20T11:31:36.465 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.464+0000 7f6c3fe00640 1 -- 172.21.15.152:0/2089499893 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c3802cd50 con 0x7f6c4009bb80 2024-04-20T11:31:36.465 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.464+0000 7f6c3fe00640 1 -- 172.21.15.152:0/2089499893 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c38034610 con 0x7f6c4009bb80 2024-04-20T11:31:36.465 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.465+0000 7f6c48600640 1 -- 172.21.15.152:0/2089499893 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6c4009bb80 msgr2=0x7f6c400d10b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:36.465 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.465+0000 7f6c48600640 1 --2- 172.21.15.152:0/2089499893 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6c4009bb80 0x7f6c400d10b0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f6c400736a0 tx=0x7f6c3802c010 comp rx=0 tx=0).stop 2024-04-20T11:31:36.465 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.465+0000 7f6c48600640 1 -- 172.21.15.152:0/2089499893 shutdown_connections 2024-04-20T11:31:36.465 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.465+0000 7f6c48600640 1 --2- 172.21.15.152:0/2089499893 >> v2:172.21.15.175:3300/0 conn(0x7f6c40160b80 0x7f6c400d0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:36.465 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.465+0000 7f6c48600640 1 --2- 172.21.15.152:0/2089499893 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6c4009bb80 0x7f6c400d10b0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:36.465 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.465+0000 7f6c48600640 1 -- 172.21.15.152:0/2089499893 >> 172.21.15.152:0/2089499893 conn(0x7f6c400b4080 msgr2=0x7f6c400b4490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:36.465 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.465+0000 7f6c48600640 1 -- 172.21.15.152:0/2089499893 shutdown_connections 2024-04-20T11:31:36.465 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.465+0000 7f6c48600640 1 -- 172.21.15.152:0/2089499893 wait complete. 2024-04-20T11:31:36.466 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.467+0000 7f6c48600640 1 Processor -- start 2024-04-20T11:31:36.466 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.467+0000 7f6c48600640 1 -- start start 2024-04-20T11:31:36.468 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.468+0000 7f6c48600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6c4009bb80 0x7f6c40159dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:36.469 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.468+0000 7f6c48600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6c4015c810 0x7f6c4015cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:36.469 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.468+0000 7f6c48600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6c40160b80 0x7f6c4015d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:36.469 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.468+0000 7f6c48600640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f6c4015cd70 con 0x7f6c4009bb80 2024-04-20T11:31:36.469 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.468+0000 7f6c48600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6c4015cbf0 con 0x7f6c40160b80 2024-04-20T11:31:36.469 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.468+0000 7f6c48600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6c400c8230 con 0x7f6c4015c810 2024-04-20T11:31:36.469 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.468+0000 7f6c45c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6c4009bb80 0x7f6c40159dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:36.469 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.468+0000 7f6c45c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6c4009bb80 0x7f6c40159dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49494/0 (socket says 172.21.15.152:49494) 2024-04-20T11:31:36.469 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.468+0000 7f6c45c00640 1 -- 172.21.15.152:0/1309828637 learned_addr learned my addr 172.21.15.152:0/1309828637 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:36.469 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.468+0000 7f6c46600640 1 --2- 172.21.15.152:0/1309828637 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6c40160b80 0x7f6c4015d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:36.469 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.468+0000 7f6c45c00640 1 -- 172.21.15.152:0/1309828637 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6c4015c810 msgr2=0x7f6c4015cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:36.469 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.468+0000 7f6c45c00640 1 --2- 172.21.15.152:0/1309828637 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6c4015c810 0x7f6c4015cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:36.469 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.469+0000 7f6c45c00640 1 -- 172.21.15.152:0/1309828637 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6c40160b80 msgr2=0x7f6c4015d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:36.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.469+0000 7f6c45c00640 1 --2- 172.21.15.152:0/1309828637 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6c40160b80 0x7f6c4015d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:36.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.469+0000 7f6c45c00640 1 -- 172.21.15.152:0/1309828637 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c4015a300 con 0x7f6c4009bb80 2024-04-20T11:31:36.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.469+0000 7f6c45c00640 1 --2- 172.21.15.152:0/1309828637 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6c4009bb80 0x7f6c40159dc0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f6c3000c430 tx=0x7f6c3000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:36.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.469+0000 7f6c3d600640 1 -- 172.21.15.152:0/1309828637 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c300074e0 con 0x7f6c4009bb80 2024-04-20T11:31:36.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.469+0000 7f6c3d600640 1 -- 172.21.15.152:0/1309828637 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c30007680 con 0x7f6c4009bb80 2024-04-20T11:31:36.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.469+0000 7f6c3d600640 1 -- 172.21.15.152:0/1309828637 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c30004a20 con 0x7f6c4009bb80 2024-04-20T11:31:36.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.469+0000 7f6c48600640 1 -- 172.21.15.152:0/1309828637 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c4015da60 con 0x7f6c4009bb80 2024-04-20T11:31:36.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.469+0000 7f6c48600640 1 -- 172.21.15.152:0/1309828637 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c4015e520 con 0x7f6c4009bb80 2024-04-20T11:31:36.471 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.470+0000 7f6c48600640 1 -- 172.21.15.152:0/1309828637 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c400689f0 con 0x7f6c4009bb80 2024-04-20T11:31:36.471 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.471+0000 7f6c3d600640 1 -- 172.21.15.152:0/1309828637 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 11) ==== 91346+0+0 (secure 0 0 0) 0x7f6c3001a020 con 0x7f6c4009bb80 2024-04-20T11:31:36.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.473+0000 7f6c3d600640 1 --2- 172.21.15.152:0/1309828637 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f6c2006cc50 0x7f6c2006f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:36.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.473+0000 7f6c3d600640 1 -- 172.21.15.152:0/1309828637 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5709+0+0 (secure 0 0 0) 0x7f6c3008b950 con 0x7f6c4009bb80 2024-04-20T11:31:36.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.473+0000 7f6c45200640 1 --2- 172.21.15.152:0/1309828637 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f6c2006cc50 0x7f6c2006f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:36.473 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.474+0000 7f6c45200640 1 --2- 172.21.15.152:0/1309828637 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f6c2006cc50 0x7f6c2006f110 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c006fd0 tx=0x7f6c2c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:36.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.477+0000 7f6c3d600640 1 -- 172.21.15.152:0/1309828637 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f6c30010300 con 0x7f6c4009bb80 2024-04-20T11:31:36.674 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.674+0000 7f6c48600640 1 -- 172.21.15.152:0/1309828637 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6c400cc750 con 0x7f6c4009bb80 2024-04-20T11:31:36.676 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.677+0000 7f6c3d600640 1 -- 172.21.15.152:0/1309828637 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) ==== 74+0+16725 (secure 0 0 0) 0x7f6c30059930 con 0x7f6c4009bb80 2024-04-20T11:31:36.677 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:31:36.677 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":60,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:31:01.845268+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:30:58.087822+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-20T11:30:59.281767+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:31:36.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.679+0000 7f6c48600640 1 -- 172.21.15.152:0/1309828637 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f6c2006cc50 msgr2=0x7f6c2006f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:36.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.679+0000 7f6c48600640 1 --2- 172.21.15.152:0/1309828637 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f6c2006cc50 0x7f6c2006f110 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c006fd0 tx=0x7f6c2c008040 comp rx=0 tx=0).stop 2024-04-20T11:31:36.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.679+0000 7f6c48600640 1 -- 172.21.15.152:0/1309828637 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6c4009bb80 msgr2=0x7f6c40159dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:36.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.679+0000 7f6c48600640 1 --2- 172.21.15.152:0/1309828637 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6c4009bb80 0x7f6c40159dc0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f6c3000c430 tx=0x7f6c3000c8f0 comp rx=0 tx=0).stop 2024-04-20T11:31:36.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.679+0000 7f6c48600640 1 -- 172.21.15.152:0/1309828637 shutdown_connections 2024-04-20T11:31:36.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.679+0000 7f6c48600640 1 --2- 172.21.15.152:0/1309828637 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f6c2006cc50 0x7f6c2006f110 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:36.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.679+0000 7f6c48600640 1 --2- 172.21.15.152:0/1309828637 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6c40160b80 0x7f6c4015d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:36.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.679+0000 7f6c48600640 1 --2- 172.21.15.152:0/1309828637 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6c4015c810 0x7f6c4015cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:36.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.679+0000 7f6c48600640 1 --2- 172.21.15.152:0/1309828637 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6c4009bb80 0x7f6c40159dc0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:36.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.680+0000 7f6c48600640 1 -- 172.21.15.152:0/1309828637 >> 172.21.15.152:0/1309828637 conn(0x7f6c400b4080 msgr2=0x7f6c400cd5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:36.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.680+0000 7f6c48600640 1 -- 172.21.15.152:0/1309828637 shutdown_connections 2024-04-20T11:31:36.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.680+0000 7f6c48600640 1 -- 172.21.15.152:0/1309828637 wait complete. 2024-04-20T11:31:36.697 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-20T11:31:36.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.884+0000 7f7217e00640 1 Processor -- start 2024-04-20T11:31:36.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.885+0000 7f7217e00640 1 -- start start 2024-04-20T11:31:36.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.886+0000 7f7217e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f72180d4ec0 0x7f72180d0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:36.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.886+0000 7f7217e00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72180d26a0 con 0x7f72180d2b80 2024-04-20T11:31:36.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.886+0000 7f7217e00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72180d53c0 con 0x7f7218161d30 2024-04-20T11:31:36.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.886+0000 7f7217e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f72180d0540 con 0x7f72180d4ec0 2024-04-20T11:31:36.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.887+0000 7f7217400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f72180d2b80 0x7f721806c830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:41326/0 (socket says 172.21.15.152:41326) 2024-04-20T11:31:36.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.887+0000 7f7217400640 1 -- 172.21.15.152:0/2159416370 learned_addr learned my addr 172.21.15.152:0/2159416370 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:31:36.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.887+0000 7f7216000640 1 --2- 172.21.15.152:0/2159416370 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f72180d4ec0 0x7f72180d0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:36.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.887+0000 7f7215600640 1 -- 172.21.15.152:0/2159416370 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1733319437 0 0) 0x7f72180d26a0 con 0x7f72180d2b80 2024-04-20T11:31:36.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.887+0000 7f7215600640 1 -- 172.21.15.152:0/2159416370 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f71fc003520 con 0x7f72180d2b80 2024-04-20T11:31:36.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.887+0000 7f7215600640 1 -- 172.21.15.152:0/2159416370 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3479818794 0 0) 0x7f71fc003520 con 0x7f72180d2b80 2024-04-20T11:31:36.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.887+0000 7f7215600640 1 -- 172.21.15.152:0/2159416370 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f72180d4ec0 msgr2=0x7f72180d0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:36.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.887+0000 7f7215600640 1 --2- 172.21.15.152:0/2159416370 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f72180d4ec0 0x7f72180d0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:36.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.887+0000 7f7215600640 1 -- 172.21.15.152:0/2159416370 >> v1:172.21.15.175:6789/0 conn(0x7f7218161d30 legacy=0x7f72180d15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:36.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.887+0000 7f7215600640 1 -- 172.21.15.152:0/2159416370 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72180cf520 con 0x7f72180d2b80 2024-04-20T11:31:36.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.887+0000 7f7215600640 1 -- 172.21.15.152:0/2159416370 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f72100019f0 con 0x7f72180d2b80 2024-04-20T11:31:36.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.888+0000 7f7215600640 1 -- 172.21.15.152:0/2159416370 >> v1:172.21.15.152:6789/0 conn(0x7f72180d2b80 legacy=0x7f721806c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:36.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.888+0000 7f7215600640 1 --2- 172.21.15.152:0/2159416370 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f71fc0039e0 0x7f71fc003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:36.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.888+0000 7f7215600640 1 --2- 172.21.15.152:0/2159416370 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71fc004390 0x7f71fc004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:36.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.888+0000 7f7215600640 1 --2- 172.21.15.152:0/2159416370 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f72180d4ec0 0x7f71fc0050c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:36.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.888+0000 7f7215600640 1 -- 172.21.15.152:0/2159416370 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f72180d0540 con 0x7f71fc0039e0 2024-04-20T11:31:36.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.888+0000 7f7215600640 1 -- 172.21.15.152:0/2159416370 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f71fc00b9c0 con 0x7f71fc004390 2024-04-20T11:31:36.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.888+0000 7f7215600640 1 -- 172.21.15.152:0/2159416370 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f71fc00bbc0 con 0x7f72180d4ec0 2024-04-20T11:31:36.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.888+0000 7f7216a00640 1 --2- 172.21.15.152:0/2159416370 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f71fc0039e0 0x7f71fc003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:36.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.888+0000 7f7216000640 1 --2- 172.21.15.152:0/2159416370 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71fc004390 0x7f71fc004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:36.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.888+0000 7f7216a00640 1 -- 172.21.15.152:0/2159416370 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f72180d4ec0 msgr2=0x7f71fc0050c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:36.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.888+0000 7f7216a00640 1 --2- 172.21.15.152:0/2159416370 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f72180d4ec0 0x7f71fc0050c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:36.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.888+0000 7f7216a00640 1 -- 172.21.15.152:0/2159416370 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71fc004390 msgr2=0x7f71fc004860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:36.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.888+0000 7f7216a00640 1 --2- 172.21.15.152:0/2159416370 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71fc004390 0x7f71fc004860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:36.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.888+0000 7f7216a00640 1 -- 172.21.15.152:0/2159416370 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71fc003520 con 0x7f71fc0039e0 2024-04-20T11:31:36.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.889+0000 7f7216a00640 1 --2- 172.21.15.152:0/2159416370 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f71fc0039e0 0x7f71fc003dc0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f7208001860 tx=0x7f71fc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8acc686ab8af096 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:36.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.890+0000 7f7215600640 1 -- 172.21.15.152:0/2159416370 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f720802f050 con 0x7f71fc0039e0 2024-04-20T11:31:36.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.890+0000 7f7215600640 1 -- 172.21.15.152:0/2159416370 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7208033650 con 0x7f71fc0039e0 2024-04-20T11:31:36.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.890+0000 7f7215600640 1 -- 172.21.15.152:0/2159416370 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7208001430 con 0x7f71fc0039e0 2024-04-20T11:31:36.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.890+0000 7f7217e00640 1 -- 172.21.15.152:0/2159416370 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f71fc0039e0 msgr2=0x7f71fc003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:36.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.890+0000 7f7217e00640 1 --2- 172.21.15.152:0/2159416370 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f71fc0039e0 0x7f71fc003dc0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f7208001860 tx=0x7f71fc005830 comp rx=0 tx=0).stop 2024-04-20T11:31:36.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.890+0000 7f7217400640 1 -- 172.21.15.152:0/2159416370 reap_dead start 2024-04-20T11:31:36.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.890+0000 7f7217e00640 1 -- 172.21.15.152:0/2159416370 shutdown_connections 2024-04-20T11:31:36.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.890+0000 7f7217e00640 1 -- 172.21.15.152:0/2159416370 >> 172.21.15.152:0/2159416370 conn(0x7f721808bfd0 msgr2=0x7f72180d45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:36.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.890+0000 7f7217e00640 1 -- 172.21.15.152:0/2159416370 shutdown_connections 2024-04-20T11:31:36.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.891+0000 7f7217e00640 1 -- 172.21.15.152:0/2159416370 wait complete. 2024-04-20T11:31:36.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.892+0000 7f7217e00640 1 Processor -- start 2024-04-20T11:31:36.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.892+0000 7f7217e00640 1 -- start start 2024-04-20T11:31:36.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.893+0000 7f7217e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f71fc0039e0 0x7f7218154ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:36.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.893+0000 7f7217e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f721815d9b0 0x7f721809fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:36.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.893+0000 7f7217e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72180a00e0 0x7f72180cbd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:36.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.893+0000 7f7217e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f72180cf1b0 con 0x7f71fc0039e0 2024-04-20T11:31:36.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.893+0000 7f7217e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f72180cfb70 con 0x7f72180a00e0 2024-04-20T11:31:36.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.893+0000 7f7217e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7218162110 con 0x7f721815d9b0 2024-04-20T11:31:36.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.893+0000 7f7216a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f71fc0039e0 0x7f7218154ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:36.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.893+0000 7f7216a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f71fc0039e0 0x7f7218154ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49516/0 (socket says 172.21.15.152:49516) 2024-04-20T11:31:36.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.893+0000 7f7216a00640 1 -- 172.21.15.152:0/920421312 learned_addr learned my addr 172.21.15.152:0/920421312 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:36.944 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.894+0000 7f7216a00640 1 -- 172.21.15.152:0/920421312 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f721815d9b0 msgr2=0x7f721809fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:36.944 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.894+0000 7f7216a00640 1 --2- 172.21.15.152:0/920421312 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f721815d9b0 0x7f721809fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:36.944 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.894+0000 7f7216a00640 1 -- 172.21.15.152:0/920421312 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72180a00e0 msgr2=0x7f72180cbd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:36.944 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.894+0000 7f7216a00640 1 --2- 172.21.15.152:0/920421312 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72180a00e0 0x7f72180cbd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:36.944 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.894+0000 7f7216a00640 1 -- 172.21.15.152:0/920421312 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72180a1bd0 con 0x7f71fc0039e0 2024-04-20T11:31:36.944 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.894+0000 7f7216a00640 1 --2- 172.21.15.152:0/920421312 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f71fc0039e0 0x7f7218154ad0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f720802bdf0 tx=0x7f7208001230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:36.944 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.894+0000 7f720ea00640 1 -- 172.21.15.152:0/920421312 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72080035f0 con 0x7f71fc0039e0 2024-04-20T11:31:36.944 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.894+0000 7f720ea00640 1 -- 172.21.15.152:0/920421312 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7208003790 con 0x7f71fc0039e0 2024-04-20T11:31:36.944 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.894+0000 7f720ea00640 1 -- 172.21.15.152:0/920421312 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7208003b60 con 0x7f71fc0039e0 2024-04-20T11:31:36.944 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.895+0000 7f7217e00640 1 -- 172.21.15.152:0/920421312 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72180a1e50 con 0x7f71fc0039e0 2024-04-20T11:31:36.944 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.895+0000 7f7217e00640 1 -- 172.21.15.152:0/920421312 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72180ce570 con 0x7f71fc0039e0 2024-04-20T11:31:36.944 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.896+0000 7f7217e00640 1 -- 172.21.15.152:0/920421312 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72180a0500 con 0x7f71fc0039e0 2024-04-20T11:31:36.944 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.897+0000 7f720ea00640 1 -- 172.21.15.152:0/920421312 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 11) ==== 91346+0+0 (secure 0 0 0) 0x7f7208003d00 con 0x7f71fc0039e0 2024-04-20T11:31:36.944 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.898+0000 7f720ea00640 1 --2- 172.21.15.152:0/920421312 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f71f806cc50 0x7f71f806f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:36.944 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.898+0000 7f720ea00640 1 -- 172.21.15.152:0/920421312 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5709+0+0 (secure 0 0 0) 0x7f72080ae370 con 0x7f71fc0039e0 2024-04-20T11:31:36.944 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.898+0000 7f7216000640 1 --2- 172.21.15.152:0/920421312 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f71f806cc50 0x7f71f806f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:36.945 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.899+0000 7f7216000640 1 --2- 172.21.15.152:0/920421312 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f71f806cc50 0x7f71f806f110 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f72181f7c50 tx=0x7f7204008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:36.945 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:36.902+0000 7f720ea00640 1 -- 172.21.15.152:0/920421312 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f72180a1bd0 con 0x7f71fc0039e0 2024-04-20T11:31:37.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:37.149+0000 7f7217e00640 1 -- 172.21.15.152:0/920421312 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f7218075e60 con 0x7f71fc0039e0 2024-04-20T11:31:37.150 INFO:tasks.ceph.mon.a.smithi152.stderr:2024-04-20T11:31:37.150+0000 7f8229800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-20T11:31:37.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:37.154+0000 7f720ea00640 1 -- 172.21.15.152:0/920421312 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v35) ==== 161+0+0 (secure 0 0 0) 0x7f7208041030 con 0x7f71fc0039e0 2024-04-20T11:31:37.153 INFO:teuthology.orchestra.run.smithi152.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-20T11:31:37.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:37.157+0000 7f7217e00640 1 -- 172.21.15.152:0/920421312 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f71f806cc50 msgr2=0x7f71f806f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:37.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:37.157+0000 7f7217e00640 1 --2- 172.21.15.152:0/920421312 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f71f806cc50 0x7f71f806f110 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f72181f7c50 tx=0x7f7204008040 comp rx=0 tx=0).stop 2024-04-20T11:31:37.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:37.157+0000 7f7217e00640 1 -- 172.21.15.152:0/920421312 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f71fc0039e0 msgr2=0x7f7218154ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:37.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:37.157+0000 7f7217e00640 1 --2- 172.21.15.152:0/920421312 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f71fc0039e0 0x7f7218154ad0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f720802bdf0 tx=0x7f7208001230 comp rx=0 tx=0).stop 2024-04-20T11:31:37.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:37.157+0000 7f7217e00640 1 -- 172.21.15.152:0/920421312 shutdown_connections 2024-04-20T11:31:37.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:37.157+0000 7f7217e00640 1 --2- 172.21.15.152:0/920421312 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f71f806cc50 0x7f71f806f110 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:37.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:37.157+0000 7f7217e00640 1 --2- 172.21.15.152:0/920421312 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72180a00e0 0x7f72180cbd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:37.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:37.157+0000 7f7217e00640 1 --2- 172.21.15.152:0/920421312 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f721815d9b0 0x7f721809fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:37.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:37.157+0000 7f7217e00640 1 --2- 172.21.15.152:0/920421312 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f71fc0039e0 0x7f7218154ad0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:37.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:37.157+0000 7f7217e00640 1 -- 172.21.15.152:0/920421312 >> 172.21.15.152:0/920421312 conn(0x7f721808bfd0 msgr2=0x7f721806d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:37.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:37.158+0000 7f7217e00640 1 -- 172.21.15.152:0/920421312 shutdown_connections 2024-04-20T11:31:37.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:37.158+0000 7f7217e00640 1 -- 172.21.15.152:0/920421312 wait complete. 2024-04-20T11:31:37.177 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-20T11:31:37.237 INFO:tasks.ceph.mds.c.smithi152.stdout:starting mds.c at 2024-04-20T11:31:37.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:37.351+0000 7fe8bf400640 1 Processor -- start 2024-04-20T11:31:37.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:37.351+0000 7fe8bf400640 1 -- start start 2024-04-20T11:31:37.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:37.353+0000 7fe8bf400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fe8b815b510 0x7fe8b815fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:37.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:37.353+0000 7fe8bf400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8b80cfc40 0x7fe8b80b3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:37.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:37.353+0000 7fe8bf400640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fe8b80b2170 0x7fe8b80b2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:37.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:37.353+0000 7fe8bf400640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fe8b806f420 con 0x7fe8b815b510 2024-04-20T11:31:37.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:37.353+0000 7fe8bf400640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fe8b80b4430 con 0x7fe8b80b2170 2024-04-20T11:31:37.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:37.353+0000 7fe8bf400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe8b815f1c0 con 0x7fe8b80cfc40 2024-04-20T11:31:37.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:37.353+0000 7fe8bca00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fe8b815b510 0x7fe8b815fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:37.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:37.353+0000 7fe8bca00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fe8b815b510 0x7fe8b815fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49530/0 (socket says 172.21.15.152:49530) 2024-04-20T11:31:37.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:37.353+0000 7fe8bca00640 1 -- 172.21.15.152:0/3892865022 learned_addr learned my addr 172.21.15.152:0/3892865022 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:37.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:37.353+0000 7fe8bca00640 1 -- 172.21.15.152:0/3892865022 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8b80cfc40 msgr2=0x7fe8b80b3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:37.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:37.353+0000 7fe8bca00640 1 --2- 172.21.15.152:0/3892865022 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8b80cfc40 0x7fe8b80b3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:37.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:37.353+0000 7fe8bca00640 1 -- 172.21.15.152:0/3892865022 >> v2:172.21.15.175:3300/0 conn(0x7fe8b80b2170 msgr2=0x7fe8b80b2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:37.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:37.353+0000 7fe8bca00640 1 --2- 172.21.15.152:0/3892865022 >> v2:172.21.15.175:3300/0 conn(0x7fe8b80b2170 0x7fe8b80b2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:37.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:37.353+0000 7fe8bca00640 1 -- 172.21.15.152:0/3892865022 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8b806f670 con 0x7fe8b815b510 2024-04-20T11:31:37.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:37.354+0000 7fe8bca00640 1 --2- 172.21.15.152:0/3892865022 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe8b815b510 0x7fe8b815fc30 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fe8b80cdab0 tx=0x7fe8ac02db00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f96f82f2aa42053d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:37.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:37.355+0000 7fe8b7400640 1 -- 172.21.15.152:0/3892865022 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8ac02f8c0 con 0x7fe8b815b510 2024-04-20T11:31:37.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:37.355+0000 7fe8b7400640 1 -- 172.21.15.152:0/3892865022 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8ac02fa60 con 0x7fe8b815b510 2024-04-20T11:31:37.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:37.355+0000 7fe8b7400640 1 -- 172.21.15.152:0/3892865022 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8ac02fe30 con 0x7fe8b815b510 2024-04-20T11:31:37.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:37.355+0000 7fe8bf400640 1 -- 172.21.15.152:0/3892865022 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe8b815b510 msgr2=0x7fe8b815fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:37.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:37.355+0000 7fe8bf400640 1 --2- 172.21.15.152:0/3892865022 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe8b815b510 0x7fe8b815fc30 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fe8b80cdab0 tx=0x7fe8ac02db00 comp rx=0 tx=0).stop 2024-04-20T11:31:37.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:37.355+0000 7fe8bf400640 1 -- 172.21.15.152:0/3892865022 shutdown_connections 2024-04-20T11:31:37.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:37.355+0000 7fe8bf400640 1 --2- 172.21.15.152:0/3892865022 >> v2:172.21.15.175:3300/0 conn(0x7fe8b80b2170 0x7fe8b80b2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:37.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:37.355+0000 7fe8bf400640 1 --2- 172.21.15.152:0/3892865022 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8b80cfc40 0x7fe8b80b3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:37.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:37.355+0000 7fe8bf400640 1 --2- 172.21.15.152:0/3892865022 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe8b815b510 0x7fe8b815fc30 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:37.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:37.355+0000 7fe8bf400640 1 -- 172.21.15.152:0/3892865022 >> 172.21.15.152:0/3892865022 conn(0x7fe8b806b980 msgr2=0x7fe8b806bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:37.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:37.355+0000 7fe8bf400640 1 -- 172.21.15.152:0/3892865022 shutdown_connections 2024-04-20T11:31:37.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:37.355+0000 7fe8bf400640 1 -- 172.21.15.152:0/3892865022 wait complete. 2024-04-20T11:31:37.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:37.357+0000 7fe8bf400640 1 Processor -- start 2024-04-20T11:31:37.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:37.357+0000 7fe8bf400640 1 -- start start 2024-04-20T11:31:37.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:37.358+0000 7fe8bf400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8b80b2170 0x7fe8b8152420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:37.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:37.358+0000 7fe8bf400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8b80cfc40 0x7fe8b814f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:37.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:37.358+0000 7fe8bf400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe8b815b510 0x7fe8b814d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:37.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:37.358+0000 7fe8bf400640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b815b8f0 con 0x7fe8b815b510 2024-04-20T11:31:37.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:37.358+0000 7fe8bf400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b815f710 con 0x7fe8b80cfc40 2024-04-20T11:31:37.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:37.358+0000 7fe8bf400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe8b806f950 con 0x7fe8b80b2170 2024-04-20T11:31:37.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:37.358+0000 7fe8bd400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe8b815b510 0x7fe8b814d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:37.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:37.358+0000 7fe8bd400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe8b815b510 0x7fe8b814d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49542/0 (socket says 172.21.15.152:49542) 2024-04-20T11:31:37.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:37.358+0000 7fe8bd400640 1 -- 172.21.15.152:0/4028690665 learned_addr learned my addr 172.21.15.152:0/4028690665 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:37.359 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:37.359+0000 7fe8bca00640 1 --2- 172.21.15.152:0/4028690665 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8b80b2170 0x7fe8b8152420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:37.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:37.359+0000 7fe8b7e00640 1 --2- 172.21.15.152:0/4028690665 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8b80cfc40 0x7fe8b814f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:37.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:37.359+0000 7fe8bd400640 1 -- 172.21.15.152:0/4028690665 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8b80b2170 msgr2=0x7fe8b8152420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:37.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:37.359+0000 7fe8bd400640 1 --2- 172.21.15.152:0/4028690665 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8b80b2170 0x7fe8b8152420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:37.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:37.359+0000 7fe8bd400640 1 -- 172.21.15.152:0/4028690665 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8b80cfc40 msgr2=0x7fe8b814f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:37.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:37.359+0000 7fe8bd400640 1 --2- 172.21.15.152:0/4028690665 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8b80cfc40 0x7fe8b814f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:37.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:37.359+0000 7fe8bd400640 1 -- 172.21.15.152:0/4028690665 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8b814f8d0 con 0x7fe8b815b510 2024-04-20T11:31:37.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:37.359+0000 7fe8bca00640 1 --2- 172.21.15.152:0/4028690665 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8b80b2170 0x7fe8b8152420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:31:37.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:37.359+0000 7fe8bd400640 1 --2- 172.21.15.152:0/4028690665 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe8b815b510 0x7fe8b814d890 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fe8b000b970 tx=0x7fe8b000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:37.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:37.359+0000 7fe8b4c00640 1 -- 172.21.15.152:0/4028690665 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b0019070 con 0x7fe8b815b510 2024-04-20T11:31:37.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:37.359+0000 7fe8b4c00640 1 -- 172.21.15.152:0/4028690665 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8b000cb60 con 0x7fe8b815b510 2024-04-20T11:31:37.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:37.359+0000 7fe8bf400640 1 -- 172.21.15.152:0/4028690665 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8b8151380 con 0x7fe8b815b510 2024-04-20T11:31:37.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:37.359+0000 7fe8b4c00640 1 -- 172.21.15.152:0/4028690665 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b0014440 con 0x7fe8b815b510 2024-04-20T11:31:37.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:37.359+0000 7fe8bf400640 1 -- 172.21.15.152:0/4028690665 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8b8151940 con 0x7fe8b815b510 2024-04-20T11:31:37.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:37.361+0000 7fe8bf400640 1 -- 172.21.15.152:0/4028690665 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8b80689f0 con 0x7fe8b815b510 2024-04-20T11:31:37.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:37.362+0000 7fe8b4c00640 1 -- 172.21.15.152:0/4028690665 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 11) ==== 91346+0+0 (secure 0 0 0) 0x7fe8b001c070 con 0x7fe8b815b510 2024-04-20T11:31:37.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:37.362+0000 7fe8b4c00640 1 --2- 172.21.15.152:0/4028690665 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe89806cc50 0x7fe89806f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:37.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:37.363+0000 7fe8b4c00640 1 -- 172.21.15.152:0/4028690665 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5795+0+0 (secure 0 0 0) 0x7fe8b008be80 con 0x7fe8b815b510 2024-04-20T11:31:37.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:37.363+0000 7fe8bca00640 1 --2- 172.21.15.152:0/4028690665 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe89806cc50 0x7fe89806f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:37.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:37.364+0000 7fe8bca00640 1 --2- 172.21.15.152:0/4028690665 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe89806cc50 0x7fe89806f110 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac038000 tx=0x7fe8ac008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:37.366 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:37.367+0000 7fe8b4c00640 1 -- 172.21.15.152:0/4028690665 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe8b0059e00 con 0x7fe8b815b510 2024-04-20T11:31:37.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:37.590+0000 7fe8bf400640 1 -- 172.21.15.152:0/4028690665 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fe8b806ecf0 con 0x7fe8b815b510 2024-04-20T11:31:38.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:38.158+0000 7fe8b4c00640 1 -- 172.21.15.152:0/4028690665 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v36) ==== 104+0+0 (secure 0 0 0) 0x7fe8b0059700 con 0x7fe8b815b510 2024-04-20T11:31:38.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:38.162+0000 7fe8bf400640 1 -- 172.21.15.152:0/4028690665 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe89806cc50 msgr2=0x7fe89806f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:38.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:38.162+0000 7fe8bf400640 1 --2- 172.21.15.152:0/4028690665 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe89806cc50 0x7fe89806f110 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac038000 tx=0x7fe8ac008030 comp rx=0 tx=0).stop 2024-04-20T11:31:38.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:38.162+0000 7fe8bf400640 1 -- 172.21.15.152:0/4028690665 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe8b815b510 msgr2=0x7fe8b814d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:38.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:38.162+0000 7fe8bf400640 1 --2- 172.21.15.152:0/4028690665 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe8b815b510 0x7fe8b814d890 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fe8b000b970 tx=0x7fe8b000be30 comp rx=0 tx=0).stop 2024-04-20T11:31:38.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:38.162+0000 7fe8bf400640 1 -- 172.21.15.152:0/4028690665 shutdown_connections 2024-04-20T11:31:38.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:38.162+0000 7fe8bf400640 1 --2- 172.21.15.152:0/4028690665 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe89806cc50 0x7fe89806f110 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:38.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:38.162+0000 7fe8bf400640 1 --2- 172.21.15.152:0/4028690665 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe8b815b510 0x7fe8b814d890 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:38.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:38.162+0000 7fe8bf400640 1 --2- 172.21.15.152:0/4028690665 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8b80cfc40 0x7fe8b814f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:38.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:38.162+0000 7fe8bf400640 1 --2- 172.21.15.152:0/4028690665 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8b80b2170 0x7fe8b8152420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:38.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:38.162+0000 7fe8bf400640 1 -- 172.21.15.152:0/4028690665 >> 172.21.15.152:0/4028690665 conn(0x7fe8b806b980 msgr2=0x7fe8b80ced00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:38.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:38.162+0000 7fe8bf400640 1 -- 172.21.15.152:0/4028690665 shutdown_connections 2024-04-20T11:31:38.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:38.162+0000 7fe8bf400640 1 -- 172.21.15.152:0/4028690665 wait complete. 2024-04-20T11:31:38.186 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-20T11:31:38.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:38.386+0000 7fe360a00640 1 Processor -- start 2024-04-20T11:31:38.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:38.387+0000 7fe360a00640 1 -- start start 2024-04-20T11:31:38.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:38.388+0000 7fe360a00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fe35c161d50 0x7fe35c0d4c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:38.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:38.388+0000 7fe360a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe35c0d51a0 0x7fe35c1622c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:38.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:38.388+0000 7fe360a00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe35c0cf6e0 con 0x7fe35c162800 2024-04-20T11:31:38.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:38.388+0000 7fe360a00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fe35c0b11b0 con 0x7fe35c161d50 2024-04-20T11:31:38.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:38.388+0000 7fe360a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe35c0cf9c0 con 0x7fe35c0d51a0 2024-04-20T11:31:38.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:38.389+0000 7fe359e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe35c0d51a0 0x7fe35c1622c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:38.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:38.389+0000 7fe359e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe35c0d51a0 0x7fe35c1622c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.152:52282/0 (socket says 172.21.15.152:52282) 2024-04-20T11:31:38.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:38.389+0000 7fe359e00640 1 -- 172.21.15.152:0/2277234611 learned_addr learned my addr 172.21.15.152:0/2277234611 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:38.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:38.389+0000 7fe35a800640 1 --2- 172.21.15.152:0/2277234611 >> v2:172.21.15.175:3300/0 conn(0x7fe35c161d50 0x7fe35c0d4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:38.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:38.390+0000 7fe359400640 1 -- 172.21.15.152:0/2277234611 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2588875238 0 0) 0x7fe35c0cf6e0 con 0x7fe35c162800 2024-04-20T11:31:38.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:38.390+0000 7fe359e00640 1 -- 172.21.15.152:0/2277234611 >> v2:172.21.15.175:3300/0 conn(0x7fe35c161d50 msgr2=0x7fe35c0d4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:38.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:38.390+0000 7fe359e00640 1 --2- 172.21.15.152:0/2277234611 >> v2:172.21.15.175:3300/0 conn(0x7fe35c161d50 0x7fe35c0d4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:38.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:38.390+0000 7fe359e00640 1 -- 172.21.15.152:0/2277234611 >> v1:172.21.15.175:6789/0 conn(0x7fe35c162800 legacy=0x7fe35c0d2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:38.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:38.390+0000 7fe359e00640 1 -- 172.21.15.152:0/2277234611 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe35c0cfca0 con 0x7fe35c0d51a0 2024-04-20T11:31:38.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:38.391+0000 7fe359e00640 1 --2- 172.21.15.152:0/2277234611 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe35c0d51a0 0x7fe35c1622c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe344009870 tx=0x7fe34402b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=660e34ae4511f539 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:38.391 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:38.391+0000 7fe359400640 1 -- 172.21.15.152:0/2277234611 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe344034070 con 0x7fe35c0d51a0 2024-04-20T11:31:38.391 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:38.391+0000 7fe359400640 1 -- 172.21.15.152:0/2277234611 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe344030840 con 0x7fe35c0d51a0 2024-04-20T11:31:38.391 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:38.391+0000 7fe359400640 1 -- 172.21.15.152:0/2277234611 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe344030c10 con 0x7fe35c0d51a0 2024-04-20T11:31:38.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:38.392+0000 7fe360a00640 1 -- 172.21.15.152:0/2277234611 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe35c0d51a0 msgr2=0x7fe35c1622c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:38.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:38.392+0000 7fe360a00640 1 --2- 172.21.15.152:0/2277234611 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe35c0d51a0 0x7fe35c1622c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe344009870 tx=0x7fe34402b980 comp rx=0 tx=0).stop 2024-04-20T11:31:38.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:38.392+0000 7fe360a00640 1 -- 172.21.15.152:0/2277234611 shutdown_connections 2024-04-20T11:31:38.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:38.392+0000 7fe360a00640 1 --2- 172.21.15.152:0/2277234611 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe35c0d51a0 0x7fe35c1622c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:38.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:38.392+0000 7fe360a00640 1 --2- 172.21.15.152:0/2277234611 >> v2:172.21.15.175:3300/0 conn(0x7fe35c161d50 0x7fe35c0d4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:38.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:38.392+0000 7fe360a00640 1 -- 172.21.15.152:0/2277234611 >> 172.21.15.152:0/2277234611 conn(0x7fe35c08bfd0 msgr2=0x7fe35c0d4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:38.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:38.392+0000 7fe360a00640 1 -- 172.21.15.152:0/2277234611 shutdown_connections 2024-04-20T11:31:38.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:38.392+0000 7fe360a00640 1 -- 172.21.15.152:0/2277234611 wait complete. 2024-04-20T11:31:38.393 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:38.394+0000 7fe360a00640 1 Processor -- start 2024-04-20T11:31:38.393 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:38.394+0000 7fe360a00640 1 -- start start 2024-04-20T11:31:38.394 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:38.395+0000 7fe360a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe35c0d51a0 0x7fe35c0b58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:38.394 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:38.395+0000 7fe360a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe35c161d50 0x7fe35c14e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:38.395 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:38.395+0000 7fe360a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe35c162800 0x7fe35c0b45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:38.395 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:38.395+0000 7fe360a00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fe35c0b2650 con 0x7fe35c162800 2024-04-20T11:31:38.395 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:38.395+0000 7fe360a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe35c163020 con 0x7fe35c0d51a0 2024-04-20T11:31:38.395 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:38.395+0000 7fe360a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe35c0b27d0 con 0x7fe35c161d50 2024-04-20T11:31:38.396 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:38.395+0000 7fe35b200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe35c162800 0x7fe35c0b45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:38.396 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:38.396+0000 7fe35b200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe35c162800 0x7fe35c0b45f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49556/0 (socket says 172.21.15.152:49556) 2024-04-20T11:31:38.396 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:38.396+0000 7fe35b200640 1 -- 172.21.15.152:0/3632511954 learned_addr learned my addr 172.21.15.152:0/3632511954 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:38.397 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:38.396+0000 7fe359e00640 1 --2- 172.21.15.152:0/3632511954 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe35c161d50 0x7fe35c14e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:38.397 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:38.396+0000 7fe35a800640 1 --2- 172.21.15.152:0/3632511954 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe35c0d51a0 0x7fe35c0b58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:38.397 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:38.396+0000 7fe35b200640 1 -- 172.21.15.152:0/3632511954 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe35c161d50 msgr2=0x7fe35c14e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:38.397 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:38.396+0000 7fe35b200640 1 --2- 172.21.15.152:0/3632511954 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe35c161d50 0x7fe35c14e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:38.397 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:38.396+0000 7fe35b200640 1 -- 172.21.15.152:0/3632511954 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe35c0d51a0 msgr2=0x7fe35c0b58f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:38.397 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:38.396+0000 7fe35b200640 1 --2- 172.21.15.152:0/3632511954 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe35c0d51a0 0x7fe35c0b58f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:38.397 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:38.396+0000 7fe35b200640 1 -- 172.21.15.152:0/3632511954 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe35c0b5e30 con 0x7fe35c162800 2024-04-20T11:31:38.397 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:38.396+0000 7fe35b200640 1 --2- 172.21.15.152:0/3632511954 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe35c162800 0x7fe35c0b45f0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fe35400c480 tx=0x7fe35400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:38.397 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:38.397+0000 7fe352a00640 1 -- 172.21.15.152:0/3632511954 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe354005480 con 0x7fe35c162800 2024-04-20T11:31:38.397 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:38.397+0000 7fe352a00640 1 -- 172.21.15.152:0/3632511954 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3540035f0 con 0x7fe35c162800 2024-04-20T11:31:38.397 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:38.397+0000 7fe352a00640 1 -- 172.21.15.152:0/3632511954 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe354003a00 con 0x7fe35c162800 2024-04-20T11:31:38.397 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:38.397+0000 7fe360a00640 1 -- 172.21.15.152:0/3632511954 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe35c0ca500 con 0x7fe35c162800 2024-04-20T11:31:38.397 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:38.397+0000 7fe360a00640 1 -- 172.21.15.152:0/3632511954 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe35c0caac0 con 0x7fe35c162800 2024-04-20T11:31:38.398 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:38.398+0000 7fe360a00640 1 -- 172.21.15.152:0/3632511954 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe35c0689f0 con 0x7fe35c162800 2024-04-20T11:31:38.398 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:38.399+0000 7fe352a00640 1 -- 172.21.15.152:0/3632511954 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 11) ==== 91346+0+0 (secure 0 0 0) 0x7fe354003ba0 con 0x7fe35c162800 2024-04-20T11:31:38.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:38.400+0000 7fe352a00640 1 --2- 172.21.15.152:0/3632511954 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe33806cdf0 0x7fe33806f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:38.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:38.400+0000 7fe352a00640 1 -- 172.21.15.152:0/3632511954 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5795+0+0 (secure 0 0 0) 0x7fe35408bcd0 con 0x7fe35c162800 2024-04-20T11:31:38.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:38.400+0000 7fe35a800640 1 --2- 172.21.15.152:0/3632511954 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe33806cdf0 0x7fe33806f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:38.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:38.401+0000 7fe35a800640 1 --2- 172.21.15.152:0/3632511954 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe33806cdf0 0x7fe33806f2b0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fe34c0113e0 tx=0x7fe34c00b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:38.404 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:38.405+0000 7fe352a00640 1 -- 172.21.15.152:0/3632511954 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe35c0b5e30 con 0x7fe35c162800 2024-04-20T11:31:38.608 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:38.608+0000 7fe360a00640 1 -- 172.21.15.152:0/3632511954 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fe35c0b6bd0 con 0x7fe35c162800 2024-04-20T11:31:39.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:39.169+0000 7fe352a00640 1 -- 172.21.15.152:0/3632511954 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v62) ==== 181+0+0 (secure 0 0 0) 0x7fe354059590 con 0x7fe35c162800 2024-04-20T11:31:39.170 INFO:teuthology.orchestra.run.smithi152.stderr:pool 'cephfs_metadata' removed 2024-04-20T11:31:39.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:39.174+0000 7fe360a00640 1 -- 172.21.15.152:0/3632511954 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe33806cdf0 msgr2=0x7fe33806f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:39.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:39.174+0000 7fe360a00640 1 --2- 172.21.15.152:0/3632511954 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe33806cdf0 0x7fe33806f2b0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fe34c0113e0 tx=0x7fe34c00b040 comp rx=0 tx=0).stop 2024-04-20T11:31:39.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:39.174+0000 7fe360a00640 1 -- 172.21.15.152:0/3632511954 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe35c162800 msgr2=0x7fe35c0b45f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:39.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:39.174+0000 7fe360a00640 1 --2- 172.21.15.152:0/3632511954 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe35c162800 0x7fe35c0b45f0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fe35400c480 tx=0x7fe35400c940 comp rx=0 tx=0).stop 2024-04-20T11:31:39.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:39.178+0000 7fe360a00640 1 -- 172.21.15.152:0/3632511954 shutdown_connections 2024-04-20T11:31:39.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:39.178+0000 7fe360a00640 1 --2- 172.21.15.152:0/3632511954 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe33806cdf0 0x7fe33806f2b0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:39.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:39.178+0000 7fe360a00640 1 --2- 172.21.15.152:0/3632511954 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe35c162800 0x7fe35c0b45f0 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:39.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:39.178+0000 7fe360a00640 1 --2- 172.21.15.152:0/3632511954 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe35c161d50 0x7fe35c14e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:39.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:39.178+0000 7fe360a00640 1 --2- 172.21.15.152:0/3632511954 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe35c0d51a0 0x7fe35c0b58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:39.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:39.178+0000 7fe360a00640 1 -- 172.21.15.152:0/3632511954 >> 172.21.15.152:0/3632511954 conn(0x7fe35c08bfd0 msgr2=0x7fe35c0d26c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:39.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:39.179+0000 7fe360a00640 1 -- 172.21.15.152:0/3632511954 shutdown_connections 2024-04-20T11:31:39.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:39.180+0000 7fe360a00640 1 -- 172.21.15.152:0/3632511954 wait complete. 2024-04-20T11:31:39.202 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-20T11:31:39.398 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:39.398+0000 7f6f06800640 1 Processor -- start 2024-04-20T11:31:39.398 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:39.398+0000 7f6f06800640 1 -- start start 2024-04-20T11:31:39.399 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:39.400+0000 7f6f06800640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f6f001569d0 0x7f6f000ad760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:39.399 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:39.400+0000 7f6f06800640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f0015d8d0 con 0x7f6f0006e510 2024-04-20T11:31:39.399 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:39.400+0000 7f6f06800640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f0015d310 con 0x7f6f0006e970 2024-04-20T11:31:39.399 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:39.400+0000 7f6f06800640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f6f0015d5f0 con 0x7f6f001569d0 2024-04-20T11:31:39.399 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:39.400+0000 7f6f04a00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f6f0006e510 0x7f6f0015ca10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:41354/0 (socket says 172.21.15.152:41354) 2024-04-20T11:31:39.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:39.400+0000 7f6f04a00640 1 -- 172.21.15.152:0/3090530272 learned_addr learned my addr 172.21.15.152:0/3090530272 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:31:39.401 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:39.400+0000 7f6effe00640 1 -- 172.21.15.152:0/3090530272 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 42715194 0 0) 0x7f6f0015d8d0 con 0x7f6f0006e510 2024-04-20T11:31:39.401 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:39.400+0000 7f6effe00640 1 -- 172.21.15.152:0/3090530272 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6ee8003520 con 0x7f6f0006e510 2024-04-20T11:31:39.401 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:39.401+0000 7f6f05400640 1 --2- 172.21.15.152:0/3090530272 >> v2:172.21.15.175:3300/0 conn(0x7f6f001569d0 0x7f6f000ad760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:39.402 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:39.401+0000 7f6effe00640 1 -- 172.21.15.152:0/3090530272 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 814069060 0 0) 0x7f6ee8003520 con 0x7f6f0006e510 2024-04-20T11:31:39.402 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:39.401+0000 7f6effe00640 1 -- 172.21.15.152:0/3090530272 >> v2:172.21.15.175:3300/0 conn(0x7f6f001569d0 msgr2=0x7f6f000ad760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:39.402 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:39.401+0000 7f6effe00640 1 --2- 172.21.15.152:0/3090530272 >> v2:172.21.15.175:3300/0 conn(0x7f6f001569d0 0x7f6f000ad760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:39.402 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:39.401+0000 7f6effe00640 1 -- 172.21.15.152:0/3090530272 >> v1:172.21.15.175:6789/0 conn(0x7f6f0006e970 legacy=0x7f6f000d05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:39.402 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:39.401+0000 7f6effe00640 1 -- 172.21.15.152:0/3090530272 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f0006dd10 con 0x7f6f0006e510 2024-04-20T11:31:39.402 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:39.401+0000 7f6effe00640 1 -- 172.21.15.152:0/3090530272 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f6eec001670 con 0x7f6f0006e510 2024-04-20T11:31:39.402 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:39.401+0000 7f6effe00640 1 -- 172.21.15.152:0/3090530272 >> v1:172.21.15.152:6789/0 conn(0x7f6f0006e510 legacy=0x7f6f0015ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:39.402 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:39.401+0000 7f6effe00640 1 --2- 172.21.15.152:0/3090530272 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6ee80039b0 0x7f6ee8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:39.402 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:39.401+0000 7f6effe00640 1 --2- 172.21.15.152:0/3090530272 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6ee8004800 0x7f6ee8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:39.402 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:39.401+0000 7f6effe00640 1 --2- 172.21.15.152:0/3090530272 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ee80052e0 0x7f6ee800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:39.402 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:39.401+0000 7f6effe00640 1 -- 172.21.15.152:0/3090530272 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f6f0015d5f0 con 0x7f6ee8004800 2024-04-20T11:31:39.402 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:39.401+0000 7f6effe00640 1 -- 172.21.15.152:0/3090530272 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6ee800c1b0 con 0x7f6ee80052e0 2024-04-20T11:31:39.402 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:39.401+0000 7f6effe00640 1 -- 172.21.15.152:0/3090530272 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6ee800c3b0 con 0x7f6ee80039b0 2024-04-20T11:31:39.403 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:39.401+0000 7f6f04a00640 1 --2- 172.21.15.152:0/3090530272 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6ee8004800 0x7f6ee8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:39.403 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:39.402+0000 7f6f05400640 1 --2- 172.21.15.152:0/3090530272 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6ee80039b0 0x7f6ee8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:39.403 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:39.402+0000 7f6f05e00640 1 --2- 172.21.15.152:0/3090530272 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ee80052e0 0x7f6ee800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:39.403 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:39.402+0000 7f6f04a00640 1 -- 172.21.15.152:0/3090530272 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6ee80039b0 msgr2=0x7f6ee8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:39.403 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:39.402+0000 7f6f04a00640 1 --2- 172.21.15.152:0/3090530272 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6ee80039b0 0x7f6ee8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:39.403 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:39.402+0000 7f6f04a00640 1 -- 172.21.15.152:0/3090530272 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ee80052e0 msgr2=0x7f6ee800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:39.403 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:39.402+0000 7f6f04a00640 1 --2- 172.21.15.152:0/3090530272 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ee80052e0 0x7f6ee800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:39.403 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:39.402+0000 7f6f04a00640 1 -- 172.21.15.152:0/3090530272 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ee8003520 con 0x7f6ee8004800 2024-04-20T11:31:39.403 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:39.402+0000 7f6f05e00640 1 --2- 172.21.15.152:0/3090530272 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ee80052e0 0x7f6ee800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:31:39.403 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:39.402+0000 7f6f05e00640 1 -- 172.21.15.152:0/3090530272 reap_dead start 2024-04-20T11:31:39.404 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:39.403+0000 7f6f04a00640 1 --2- 172.21.15.152:0/3090530272 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6ee8004800 0x7f6ee8004c80 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f6eec00c210 tx=0x7f6ee8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80bc8febb4296c2b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:39.405 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:39.403+0000 7f6effe00640 1 -- 172.21.15.152:0/3090530272 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eec007cf0 con 0x7f6ee8004800 2024-04-20T11:31:39.405 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:39.403+0000 7f6effe00640 1 -- 172.21.15.152:0/3090530272 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6eec030070 con 0x7f6ee8004800 2024-04-20T11:31:39.405 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:39.403+0000 7f6effe00640 1 -- 172.21.15.152:0/3090530272 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eec005d70 con 0x7f6ee8004800 2024-04-20T11:31:39.405 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:39.404+0000 7f6f06800640 1 -- 172.21.15.152:0/3090530272 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6ee8004800 msgr2=0x7f6ee8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:39.405 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:39.404+0000 7f6f06800640 1 --2- 172.21.15.152:0/3090530272 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6ee8004800 0x7f6ee8004c80 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f6eec00c210 tx=0x7f6ee8005830 comp rx=0 tx=0).stop 2024-04-20T11:31:39.405 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:39.404+0000 7f6f06800640 1 -- 172.21.15.152:0/3090530272 shutdown_connections 2024-04-20T11:31:39.405 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:39.404+0000 7f6f06800640 1 --2- 172.21.15.152:0/3090530272 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6ee8004800 0x7f6ee8004c80 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:39.405 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:39.404+0000 7f6f06800640 1 -- 172.21.15.152:0/3090530272 >> 172.21.15.152:0/3090530272 conn(0x7f6f000b21e0 msgr2=0x7f6f000b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:39.405 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:39.404+0000 7f6f06800640 1 -- 172.21.15.152:0/3090530272 shutdown_connections 2024-04-20T11:31:39.405 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:39.404+0000 7f6f06800640 1 -- 172.21.15.152:0/3090530272 wait complete. 2024-04-20T11:31:39.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:39.406+0000 7f6f06800640 1 Processor -- start 2024-04-20T11:31:39.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:39.406+0000 7f6f06800640 1 -- start start 2024-04-20T11:31:39.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:39.407+0000 7f6f06800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6ee8004800 0x7f6f000b1300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:39.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:39.407+0000 7f6f06800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6f000aefb0 0x7f6f0015c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:39.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:39.407+0000 7f6f06800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6f000b0b20 0x7f6f0015ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:39.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:39.407+0000 7f6f06800640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f6f000ad760 con 0x7f6ee8004800 2024-04-20T11:31:39.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:39.407+0000 7f6f06800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6f00156db0 con 0x7f6f000b0b20 2024-04-20T11:31:39.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:39.407+0000 7f6f06800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6f000ad8e0 con 0x7f6f000aefb0 2024-04-20T11:31:39.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:39.407+0000 7f6f05400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6ee8004800 0x7f6f000b1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:39.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:39.407+0000 7f6f05400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6ee8004800 0x7f6f000b1300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49572/0 (socket says 172.21.15.152:49572) 2024-04-20T11:31:39.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:39.407+0000 7f6f05400640 1 -- 172.21.15.152:0/1323719945 learned_addr learned my addr 172.21.15.152:0/1323719945 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:39.408 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:39.408+0000 7f6f04a00640 1 --2- 172.21.15.152:0/1323719945 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6f000aefb0 0x7f6f0015c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:39.409 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:39.408+0000 7f6f05e00640 1 --2- 172.21.15.152:0/1323719945 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6f000b0b20 0x7f6f0015ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:39.409 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:39.408+0000 7f6f05400640 1 -- 172.21.15.152:0/1323719945 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6f000aefb0 msgr2=0x7f6f0015c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:39.409 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:39.408+0000 7f6f05400640 1 --2- 172.21.15.152:0/1323719945 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6f000aefb0 0x7f6f0015c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:39.409 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:39.408+0000 7f6f05400640 1 -- 172.21.15.152:0/1323719945 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6f000b0b20 msgr2=0x7f6f0015ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:39.409 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:39.408+0000 7f6f05400640 1 --2- 172.21.15.152:0/1323719945 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6f000b0b20 0x7f6f0015ec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:39.409 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:39.408+0000 7f6f05400640 1 -- 172.21.15.152:0/1323719945 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f000b0fa0 con 0x7f6ee8004800 2024-04-20T11:31:39.409 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:39.408+0000 7f6f04a00640 1 --2- 172.21.15.152:0/1323719945 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6f000aefb0 0x7f6f0015c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:31:39.409 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:39.408+0000 7f6f05400640 1 --2- 172.21.15.152:0/1323719945 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6ee8004800 0x7f6f000b1300 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f6ef4009d10 tx=0x7f6ef4007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:39.409 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:39.408+0000 7f6efd600640 1 -- 172.21.15.152:0/1323719945 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ef4013580 con 0x7f6ee8004800 2024-04-20T11:31:39.409 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:39.408+0000 7f6efd600640 1 -- 172.21.15.152:0/1323719945 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ef4019070 con 0x7f6ee8004800 2024-04-20T11:31:39.409 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:39.408+0000 7f6efd600640 1 -- 172.21.15.152:0/1323719945 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ef4013980 con 0x7f6ee8004800 2024-04-20T11:31:39.409 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:39.408+0000 7f6f06800640 1 -- 172.21.15.152:0/1323719945 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f00138b40 con 0x7f6ee8004800 2024-04-20T11:31:39.409 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:39.409+0000 7f6f06800640 1 -- 172.21.15.152:0/1323719945 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f00138860 con 0x7f6ee8004800 2024-04-20T11:31:39.410 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:39.410+0000 7f6f06800640 1 -- 172.21.15.152:0/1323719945 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f00075e60 con 0x7f6ee8004800 2024-04-20T11:31:39.410 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:39.411+0000 7f6efd600640 1 -- 172.21.15.152:0/1323719945 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 11) ==== 91346+0+0 (secure 0 0 0) 0x7f6ef4013b20 con 0x7f6ee8004800 2024-04-20T11:31:39.411 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:39.412+0000 7f6efd600640 1 --2- 172.21.15.152:0/1323719945 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f6ee006cc50 0x7f6ee006f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:39.411 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:39.412+0000 7f6efd600640 1 -- 172.21.15.152:0/1323719945 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5357+0+0 (secure 0 0 0) 0x7f6ef408cb10 con 0x7f6ee8004800 2024-04-20T11:31:39.411 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:39.412+0000 7f6f04a00640 1 --2- 172.21.15.152:0/1323719945 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f6ee006cc50 0x7f6ee006f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:39.412 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:39.413+0000 7f6f04a00640 1 --2- 172.21.15.152:0/1323719945 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f6ee006cc50 0x7f6ee006f110 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f6eec034000 tx=0x7f6eec039040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:39.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:39.416+0000 7f6efd600640 1 -- 172.21.15.152:0/1323719945 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f6ef4003ec0 con 0x7f6ee8004800 2024-04-20T11:31:39.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:39.617+0000 7f6f06800640 1 -- 172.21.15.152:0/1323719945 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f6f0006f440 con 0x7f6ee8004800 2024-04-20T11:31:40.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.180+0000 7f6efd600640 1 -- 172.21.15.152:0/1323719945 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v63) ==== 169+0+0 (secure 0 0 0) 0x7f6ef405ab40 con 0x7f6ee8004800 2024-04-20T11:31:40.180 INFO:teuthology.orchestra.run.smithi152.stderr:pool 'cephfs_data' removed 2024-04-20T11:31:40.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.183+0000 7f6f06800640 1 -- 172.21.15.152:0/1323719945 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f6ee006cc50 msgr2=0x7f6ee006f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:40.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.183+0000 7f6f06800640 1 --2- 172.21.15.152:0/1323719945 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f6ee006cc50 0x7f6ee006f110 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f6eec034000 tx=0x7f6eec039040 comp rx=0 tx=0).stop 2024-04-20T11:31:40.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.183+0000 7f6f06800640 1 -- 172.21.15.152:0/1323719945 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6ee8004800 msgr2=0x7f6f000b1300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:40.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.183+0000 7f6f06800640 1 --2- 172.21.15.152:0/1323719945 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6ee8004800 0x7f6f000b1300 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f6ef4009d10 tx=0x7f6ef4007700 comp rx=0 tx=0).stop 2024-04-20T11:31:40.184 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.184+0000 7f6f06800640 1 -- 172.21.15.152:0/1323719945 shutdown_connections 2024-04-20T11:31:40.184 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.184+0000 7f6f06800640 1 --2- 172.21.15.152:0/1323719945 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f6ee006cc50 0x7f6ee006f110 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:40.184 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.184+0000 7f6f06800640 1 --2- 172.21.15.152:0/1323719945 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6f000b0b20 0x7f6f0015ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:40.184 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.184+0000 7f6f06800640 1 --2- 172.21.15.152:0/1323719945 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6f000aefb0 0x7f6f0015c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:40.184 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.184+0000 7f6f06800640 1 --2- 172.21.15.152:0/1323719945 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6ee8004800 0x7f6f000b1300 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:40.184 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.184+0000 7f6f06800640 1 -- 172.21.15.152:0/1323719945 >> 172.21.15.152:0/1323719945 conn(0x7f6f000b21e0 msgr2=0x7f6f0015b100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:40.184 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.184+0000 7f6f06800640 1 -- 172.21.15.152:0/1323719945 shutdown_connections 2024-04-20T11:31:40.184 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.184+0000 7f6f06800640 1 -- 172.21.15.152:0/1323719945 wait complete. 2024-04-20T11:31:40.217 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-20T11:31:40.396 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.396+0000 7fae56800640 1 Processor -- start 2024-04-20T11:31:40.396 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.396+0000 7fae56800640 1 -- start start 2024-04-20T11:31:40.398 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.398+0000 7fae56800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fae500b2880 0x7fae500b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:40.398 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.398+0000 7fae56800640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae501463e0 con 0x7fae50146000 2024-04-20T11:31:40.398 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.398+0000 7fae56800640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae500b3e20 con 0x7fae500cd1b0 2024-04-20T11:31:40.398 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.398+0000 7fae56800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fae500b4750 con 0x7fae500b2880 2024-04-20T11:31:40.398 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.398+0000 7fae4f400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fae50146000 0x7fae500b2170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:41360/0 (socket says 172.21.15.152:41360) 2024-04-20T11:31:40.398 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.398+0000 7fae4f400640 1 -- 172.21.15.152:0/1699604246 learned_addr learned my addr 172.21.15.152:0/1699604246 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:31:40.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.399+0000 7fae4ea00640 1 -- 172.21.15.152:0/1699604246 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 407178487 0 0) 0x7fae501463e0 con 0x7fae50146000 2024-04-20T11:31:40.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.399+0000 7fae4ea00640 1 -- 172.21.15.152:0/1699604246 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fae34003520 con 0x7fae50146000 2024-04-20T11:31:40.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.399+0000 7fae46a00640 1 --2- 172.21.15.152:0/1699604246 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fae500b2880 0x7fae500b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:40.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.399+0000 7fae4ea00640 1 -- 172.21.15.152:0/1699604246 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2908927447 0 0) 0x7fae34003520 con 0x7fae50146000 2024-04-20T11:31:40.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.399+0000 7fae4ea00640 1 -- 172.21.15.152:0/1699604246 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fae500b2880 msgr2=0x7fae500b4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:40.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.399+0000 7fae4ea00640 1 --2- 172.21.15.152:0/1699604246 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fae500b2880 0x7fae500b4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:40.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.399+0000 7fae4ea00640 1 -- 172.21.15.152:0/1699604246 >> v1:172.21.15.175:6789/0 conn(0x7fae500cd1b0 legacy=0x7fae500cd600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:40.401 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.399+0000 7fae4ea00640 1 -- 172.21.15.152:0/1699604246 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae500b4100 con 0x7fae50146000 2024-04-20T11:31:40.401 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.399+0000 7fae4ea00640 1 -- 172.21.15.152:0/1699604246 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7fae400019f0 con 0x7fae50146000 2024-04-20T11:31:40.401 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.399+0000 7fae4ea00640 1 -- 172.21.15.152:0/1699604246 >> v1:172.21.15.152:6789/0 conn(0x7fae50146000 legacy=0x7fae500b2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:40.401 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.399+0000 7fae4ea00640 1 --2- 172.21.15.152:0/1699604246 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fae500b2880 0x7fae340039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:40.401 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.399+0000 7fae4ea00640 1 --2- 172.21.15.152:0/1699604246 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae34004010 0x7fae34004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:40.401 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.399+0000 7fae4ea00640 1 --2- 172.21.15.152:0/1699604246 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fae34004d40 0x7fae340051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:40.401 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.400+0000 7fae4ea00640 1 -- 172.21.15.152:0/1699604246 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fae500b4750 con 0x7fae34004d40 2024-04-20T11:31:40.401 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.400+0000 7fae4ea00640 1 -- 172.21.15.152:0/1699604246 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fae3400bc00 con 0x7fae34004010 2024-04-20T11:31:40.401 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.400+0000 7fae4ea00640 1 -- 172.21.15.152:0/1699604246 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fae3400be00 con 0x7fae500b2880 2024-04-20T11:31:40.401 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.400+0000 7fae4fe00640 1 --2- 172.21.15.152:0/1699604246 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fae34004d40 0x7fae340051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:40.401 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.400+0000 7fae4f400640 1 --2- 172.21.15.152:0/1699604246 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fae500b2880 0x7fae340039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:40.401 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.400+0000 7fae46a00640 1 --2- 172.21.15.152:0/1699604246 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae34004010 0x7fae34004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:40.401 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.400+0000 7fae4fe00640 1 -- 172.21.15.152:0/1699604246 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fae500b2880 msgr2=0x7fae340039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:40.401 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.400+0000 7fae4fe00640 1 --2- 172.21.15.152:0/1699604246 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fae500b2880 0x7fae340039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:40.402 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.400+0000 7fae4fe00640 1 -- 172.21.15.152:0/1699604246 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae34004010 msgr2=0x7fae34004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:40.402 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.400+0000 7fae4fe00640 1 --2- 172.21.15.152:0/1699604246 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae34004010 0x7fae34004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:40.402 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.400+0000 7fae4fe00640 1 -- 172.21.15.152:0/1699604246 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae34003520 con 0x7fae34004d40 2024-04-20T11:31:40.402 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.400+0000 7fae46a00640 1 --2- 172.21.15.152:0/1699604246 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae34004010 0x7fae34004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:31:40.403 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.401+0000 7fae4fe00640 1 --2- 172.21.15.152:0/1699604246 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fae34004d40 0x7fae340051c0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fae48001750 tx=0x7fae34005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d85627f31f5c9549 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:40.403 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.402+0000 7fae4ea00640 1 -- 172.21.15.152:0/1699604246 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae48001370 con 0x7fae34004d40 2024-04-20T11:31:40.403 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.402+0000 7fae4ea00640 1 -- 172.21.15.152:0/1699604246 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae480052d0 con 0x7fae34004d40 2024-04-20T11:31:40.403 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.402+0000 7fae4ea00640 1 -- 172.21.15.152:0/1699604246 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae48003830 con 0x7fae34004d40 2024-04-20T11:31:40.403 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.402+0000 7fae56800640 1 -- 172.21.15.152:0/1699604246 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fae34004d40 msgr2=0x7fae340051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:40.403 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.402+0000 7fae56800640 1 --2- 172.21.15.152:0/1699604246 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fae34004d40 0x7fae340051c0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fae48001750 tx=0x7fae34005830 comp rx=0 tx=0).stop 2024-04-20T11:31:40.403 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.402+0000 7fae4fe00640 1 -- 172.21.15.152:0/1699604246 reap_dead start 2024-04-20T11:31:40.403 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.402+0000 7fae56800640 1 -- 172.21.15.152:0/1699604246 shutdown_connections 2024-04-20T11:31:40.403 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.402+0000 7fae56800640 1 -- 172.21.15.152:0/1699604246 >> 172.21.15.152:0/1699604246 conn(0x7fae5006b980 msgr2=0x7fae5006bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:40.404 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.402+0000 7fae56800640 1 -- 172.21.15.152:0/1699604246 shutdown_connections 2024-04-20T11:31:40.404 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.403+0000 7fae56800640 1 -- 172.21.15.152:0/1699604246 wait complete. 2024-04-20T11:31:40.404 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.404+0000 7fae56800640 1 Processor -- start 2024-04-20T11:31:40.404 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.404+0000 7fae56800640 1 -- start start 2024-04-20T11:31:40.406 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.405+0000 7fae56800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fae34004d40 0x7fae500ad8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:40.406 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.405+0000 7fae56800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fae500aff40 0x7fae50141ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:40.406 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.405+0000 7fae56800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae501f80b0 0x7fae501f8530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:40.406 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.405+0000 7fae56800640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fae500b4d60 con 0x7fae500aff40 2024-04-20T11:31:40.406 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.405+0000 7fae56800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fae500b4be0 con 0x7fae501f80b0 2024-04-20T11:31:40.406 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.405+0000 7fae56800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fae500b4ee0 con 0x7fae34004d40 2024-04-20T11:31:40.406 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.406+0000 7fae46a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fae500aff40 0x7fae50141ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:40.406 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.406+0000 7fae46a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fae500aff40 0x7fae50141ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49580/0 (socket says 172.21.15.152:49580) 2024-04-20T11:31:40.406 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.406+0000 7fae46a00640 1 -- 172.21.15.152:0/3368965227 learned_addr learned my addr 172.21.15.152:0/3368965227 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:40.406 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.406+0000 7fae4f400640 1 --2- 172.21.15.152:0/3368965227 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fae34004d40 0x7fae500ad8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:40.406 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.406+0000 7fae4fe00640 1 --2- 172.21.15.152:0/3368965227 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae501f80b0 0x7fae501f8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:40.406 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.406+0000 7fae46a00640 1 -- 172.21.15.152:0/3368965227 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fae34004d40 msgr2=0x7fae500ad8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:40.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.406+0000 7fae46a00640 1 --2- 172.21.15.152:0/3368965227 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fae34004d40 0x7fae500ad8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:40.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.406+0000 7fae46a00640 1 -- 172.21.15.152:0/3368965227 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae501f80b0 msgr2=0x7fae501f8530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:40.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.406+0000 7fae46a00640 1 --2- 172.21.15.152:0/3368965227 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae501f80b0 0x7fae501f8530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:40.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.406+0000 7fae46a00640 1 -- 172.21.15.152:0/3368965227 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae500adf70 con 0x7fae500aff40 2024-04-20T11:31:40.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.406+0000 7fae4fe00640 1 --2- 172.21.15.152:0/3368965227 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae501f80b0 0x7fae501f8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:31:40.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.406+0000 7fae46a00640 1 --2- 172.21.15.152:0/3368965227 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fae500aff40 0x7fae50141ee0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fae3800e7e0 tx=0x7fae3800eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:40.408 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.407+0000 7fae47e00640 1 -- 172.21.15.152:0/3368965227 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae38017070 con 0x7fae500aff40 2024-04-20T11:31:40.408 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.407+0000 7fae47e00640 1 -- 172.21.15.152:0/3368965227 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae3800c6d0 con 0x7fae500aff40 2024-04-20T11:31:40.408 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.407+0000 7fae47e00640 1 -- 172.21.15.152:0/3368965227 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae3800f040 con 0x7fae500aff40 2024-04-20T11:31:40.408 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.407+0000 7fae56800640 1 -- 172.21.15.152:0/3368965227 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae500ae250 con 0x7fae500aff40 2024-04-20T11:31:40.408 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.407+0000 7fae56800640 1 -- 172.21.15.152:0/3368965227 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae501f7b90 con 0x7fae500aff40 2024-04-20T11:31:40.408 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.408+0000 7fae56800640 1 -- 172.21.15.152:0/3368965227 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae501f7980 con 0x7fae500aff40 2024-04-20T11:31:40.408 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.409+0000 7fae47e00640 1 -- 172.21.15.152:0/3368965227 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 11) ==== 91346+0+0 (secure 0 0 0) 0x7fae3801a070 con 0x7fae500aff40 2024-04-20T11:31:40.410 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.410+0000 7fae47e00640 1 --2- 172.21.15.152:0/3368965227 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fae2c06cc50 0x7fae2c06f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:40.410 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.410+0000 7fae47e00640 1 -- 172.21.15.152:0/3368965227 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 4943+0+0 (secure 0 0 0) 0x7fae3808bb40 con 0x7fae500aff40 2024-04-20T11:31:40.410 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.410+0000 7fae4f400640 1 --2- 172.21.15.152:0/3368965227 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fae2c06cc50 0x7fae2c06f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:40.410 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.411+0000 7fae4f400640 1 --2- 172.21.15.152:0/3368965227 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fae2c06cc50 0x7fae2c06f110 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fae400013d0 tx=0x7fae40009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:40.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.415+0000 7fae47e00640 1 -- 172.21.15.152:0/3368965227 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fae38059d50 con 0x7fae500aff40 2024-04-20T11:31:40.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.619+0000 7fae56800640 1 -- 172.21.15.152:0/3368965227 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fae500689f0 con 0x7fae500aff40 2024-04-20T11:31:40.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.626+0000 7fae47e00640 1 -- 172.21.15.152:0/3368965227 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) ==== 104+0+0 (secure 0 0 0) 0x7fae38059650 con 0x7fae500aff40 2024-04-20T11:31:40.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.629+0000 7fae56800640 1 -- 172.21.15.152:0/3368965227 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fae2c06cc50 msgr2=0x7fae2c06f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:40.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.629+0000 7fae56800640 1 --2- 172.21.15.152:0/3368965227 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fae2c06cc50 0x7fae2c06f110 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fae400013d0 tx=0x7fae40009040 comp rx=0 tx=0).stop 2024-04-20T11:31:40.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.629+0000 7fae56800640 1 -- 172.21.15.152:0/3368965227 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fae500aff40 msgr2=0x7fae50141ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:40.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.629+0000 7fae56800640 1 --2- 172.21.15.152:0/3368965227 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fae500aff40 0x7fae50141ee0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fae3800e7e0 tx=0x7fae3800eca0 comp rx=0 tx=0).stop 2024-04-20T11:31:40.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.629+0000 7fae56800640 1 -- 172.21.15.152:0/3368965227 shutdown_connections 2024-04-20T11:31:40.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.629+0000 7fae56800640 1 --2- 172.21.15.152:0/3368965227 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fae2c06cc50 0x7fae2c06f110 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:40.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.629+0000 7fae56800640 1 --2- 172.21.15.152:0/3368965227 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae501f80b0 0x7fae501f8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:40.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.629+0000 7fae56800640 1 --2- 172.21.15.152:0/3368965227 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fae500aff40 0x7fae50141ee0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:40.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.629+0000 7fae56800640 1 --2- 172.21.15.152:0/3368965227 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fae34004d40 0x7fae500ad8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:40.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.629+0000 7fae56800640 1 -- 172.21.15.152:0/3368965227 >> 172.21.15.152:0/3368965227 conn(0x7fae5006b980 msgr2=0x7fae50145110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:40.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.629+0000 7fae56800640 1 -- 172.21.15.152:0/3368965227 shutdown_connections 2024-04-20T11:31:40.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.630+0000 7fae56800640 1 -- 172.21.15.152:0/3368965227 wait complete. 2024-04-20T11:31:40.650 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_malicious_metafile_on_legacy_to_v1_upgrade' 2024-04-20T11:31:40.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.853+0000 7facdaa00640 1 Processor -- start 2024-04-20T11:31:40.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.853+0000 7facdaa00640 1 -- start start 2024-04-20T11:31:40.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.854+0000 7facdaa00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7facd415d1e0 0x7facd4160b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:40.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.854+0000 7facdaa00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7facd40ce8c0 con 0x7facd4071cd0 2024-04-20T11:31:40.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.855+0000 7facdaa00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7facd41600b0 con 0x7facd415d5c0 2024-04-20T11:31:40.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.855+0000 7facdaa00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7facd4160390 con 0x7facd415d1e0 2024-04-20T11:31:40.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.855+0000 7facd9600640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7facd415d1e0 0x7facd4160b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:40.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.855+0000 7facd9600640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7facd415d1e0 0x7facd4160b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49596/0 (socket says 172.21.15.152:49596) 2024-04-20T11:31:40.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.855+0000 7facd9600640 1 -- 172.21.15.152:0/438821073 learned_addr learned my addr 172.21.15.152:0/438821073 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:40.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.855+0000 7facda000640 1 --1- 172.21.15.152:0/438821073 >> v1:172.21.15.152:6789/0 conn(0x7facd4071cd0 0x7facd4072120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:41362/0 (socket says 172.21.15.152:41362) 2024-04-20T11:31:40.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.856+0000 7facd9600640 1 -- 172.21.15.152:0/438821073 >> v1:172.21.15.175:6789/0 conn(0x7facd415d5c0 legacy=0x7facd40cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:40.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.856+0000 7facd9600640 1 -- 172.21.15.152:0/438821073 >> v1:172.21.15.152:6789/0 conn(0x7facd4071cd0 legacy=0x7facd4072120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:40.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.856+0000 7facd9600640 1 -- 172.21.15.152:0/438821073 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facd4160670 con 0x7facd415d1e0 2024-04-20T11:31:40.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.856+0000 7facd3e00640 1 -- 172.21.15.152:0/438821073 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 272355 0 0) 0x7facd40ce8c0 con 0x7facd4071cd0 2024-04-20T11:31:40.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.857+0000 7facd9600640 1 --2- 172.21.15.152:0/438821073 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7facd415d1e0 0x7facd4160b90 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7facc40098a0 tx=0x7facc402ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cd6e9b24fb85185 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:40.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.857+0000 7facd3e00640 1 -- 172.21.15.152:0/438821073 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facc402f830 con 0x7facd415d1e0 2024-04-20T11:31:40.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.857+0000 7facd3e00640 1 -- 172.21.15.152:0/438821073 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facc402f9d0 con 0x7facd415d1e0 2024-04-20T11:31:40.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.857+0000 7facd3e00640 1 -- 172.21.15.152:0/438821073 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facc402fda0 con 0x7facd415d1e0 2024-04-20T11:31:40.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.858+0000 7facdaa00640 1 -- 172.21.15.152:0/438821073 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7facd415d1e0 msgr2=0x7facd4160b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:40.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.858+0000 7facdaa00640 1 --2- 172.21.15.152:0/438821073 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7facd415d1e0 0x7facd4160b90 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7facc40098a0 tx=0x7facc402ba10 comp rx=0 tx=0).stop 2024-04-20T11:31:40.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.858+0000 7facdaa00640 1 -- 172.21.15.152:0/438821073 shutdown_connections 2024-04-20T11:31:40.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.858+0000 7facdaa00640 1 --2- 172.21.15.152:0/438821073 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7facd415d1e0 0x7facd4160b90 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:40.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.858+0000 7facdaa00640 1 -- 172.21.15.152:0/438821073 >> 172.21.15.152:0/438821073 conn(0x7facd4067ae0 msgr2=0x7facd406b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:40.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.858+0000 7facdaa00640 1 -- 172.21.15.152:0/438821073 shutdown_connections 2024-04-20T11:31:40.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.858+0000 7facdaa00640 1 -- 172.21.15.152:0/438821073 wait complete. 2024-04-20T11:31:40.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.859+0000 7facdaa00640 1 Processor -- start 2024-04-20T11:31:40.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.860+0000 7facdaa00640 1 -- start start 2024-04-20T11:31:40.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.860+0000 7facdaa00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7facd4071cd0 0x7facd40a9d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:40.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.860+0000 7facdaa00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7facd415d1e0 0x7facd4152090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:40.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.861+0000 7facdaa00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7facd415d5c0 0x7facd41585b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:40.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.861+0000 7facdaa00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7facd40ccef0 con 0x7facd4071cd0 2024-04-20T11:31:40.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.861+0000 7facdaa00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7facd40ccd70 con 0x7facd415d1e0 2024-04-20T11:31:40.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.861+0000 7facdaa00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7facd40cd070 con 0x7facd415d5c0 2024-04-20T11:31:40.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.861+0000 7facd9600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7facd4071cd0 0x7facd40a9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:40.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.861+0000 7facd9600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7facd4071cd0 0x7facd40a9d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49606/0 (socket says 172.21.15.152:49606) 2024-04-20T11:31:40.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.861+0000 7facd9600640 1 -- 172.21.15.152:0/3209478467 learned_addr learned my addr 172.21.15.152:0/3209478467 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:40.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.861+0000 7facd9600640 1 -- 172.21.15.152:0/3209478467 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7facd415d5c0 msgr2=0x7facd41585b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:40.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.861+0000 7facd9600640 1 --2- 172.21.15.152:0/3209478467 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7facd415d5c0 0x7facd41585b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:40.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.861+0000 7facd9600640 1 -- 172.21.15.152:0/3209478467 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7facd415d1e0 msgr2=0x7facd4152090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:40.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.861+0000 7facd9600640 1 --2- 172.21.15.152:0/3209478467 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7facd415d1e0 0x7facd4152090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:40.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.861+0000 7facd9600640 1 -- 172.21.15.152:0/3209478467 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facd40aa240 con 0x7facd4071cd0 2024-04-20T11:31:40.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.862+0000 7facd9600640 1 --2- 172.21.15.152:0/3209478467 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7facd4071cd0 0x7facd40a9d00 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7facc40090b0 tx=0x7facc4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:40.863 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.862+0000 7facd1600640 1 -- 172.21.15.152:0/3209478467 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facc4047070 con 0x7facd4071cd0 2024-04-20T11:31:40.863 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.862+0000 7facd1600640 1 -- 172.21.15.152:0/3209478467 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facc4037cd0 con 0x7facd4071cd0 2024-04-20T11:31:40.863 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.862+0000 7facd1600640 1 -- 172.21.15.152:0/3209478467 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facc4004250 con 0x7facd4071cd0 2024-04-20T11:31:40.863 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.862+0000 7facdaa00640 1 -- 172.21.15.152:0/3209478467 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facd41549f0 con 0x7facd4071cd0 2024-04-20T11:31:40.863 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.862+0000 7facdaa00640 1 -- 172.21.15.152:0/3209478467 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facd4154fb0 con 0x7facd4071cd0 2024-04-20T11:31:40.863 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.863+0000 7facdaa00640 1 -- 172.21.15.152:0/3209478467 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facd4075e60 con 0x7facd4071cd0 2024-04-20T11:31:40.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.864+0000 7facd1600640 1 -- 172.21.15.152:0/3209478467 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 11) ==== 91346+0+0 (secure 0 0 0) 0x7facc404a070 con 0x7facd4071cd0 2024-04-20T11:31:40.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.865+0000 7facd1600640 1 --2- 172.21.15.152:0/3209478467 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7facb406cc50 0x7facb406f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:40.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.865+0000 7facd1600640 1 -- 172.21.15.152:0/3209478467 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 4943+0+0 (secure 0 0 0) 0x7facc40ae080 con 0x7facd4071cd0 2024-04-20T11:31:40.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.865+0000 7facd8c00640 1 --2- 172.21.15.152:0/3209478467 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7facb406cc50 0x7facb406f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:40.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.866+0000 7facd8c00640 1 --2- 172.21.15.152:0/3209478467 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7facb406cc50 0x7facb406f110 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7facc0000f60 tx=0x7facc00079b0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:40.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:40.870+0000 7facd1600640 1 -- 172.21.15.152:0/3209478467 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7facd40aa240 con 0x7facd4071cd0 2024-04-20T11:31:41.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.150+0000 7facdaa00640 1 -- 172.21.15.152:0/3209478467 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_malicious_metafile_on_legacy_to_v1_upgrade"]} v 0) -- 0x7facd4072c90 con 0x7facd4071cd0 2024-04-20T11:31:41.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.183+0000 7facd1600640 1 -- 172.21.15.152:0/3209478467 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_malicious_metafile_on_legacy_to_v1_upgrade"]}]=0 v255) ==== 161+0+0 (secure 0 0 0) 0x7facc4033030 con 0x7facd4071cd0 2024-04-20T11:31:41.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.186+0000 7facdaa00640 1 -- 172.21.15.152:0/3209478467 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7facb406cc50 msgr2=0x7facb406f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:41.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.186+0000 7facdaa00640 1 --2- 172.21.15.152:0/3209478467 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7facb406cc50 0x7facb406f110 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7facc0000f60 tx=0x7facc00079b0 comp rx=0 tx=0).stop 2024-04-20T11:31:41.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.186+0000 7facdaa00640 1 -- 172.21.15.152:0/3209478467 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7facd4071cd0 msgr2=0x7facd40a9d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:41.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.186+0000 7facdaa00640 1 --2- 172.21.15.152:0/3209478467 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7facd4071cd0 0x7facd40a9d00 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7facc40090b0 tx=0x7facc4004730 comp rx=0 tx=0).stop 2024-04-20T11:31:41.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.186+0000 7facdaa00640 1 -- 172.21.15.152:0/3209478467 shutdown_connections 2024-04-20T11:31:41.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.186+0000 7facdaa00640 1 --2- 172.21.15.152:0/3209478467 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7facb406cc50 0x7facb406f110 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:41.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.186+0000 7facdaa00640 1 --2- 172.21.15.152:0/3209478467 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7facd415d5c0 0x7facd41585b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:41.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.186+0000 7facdaa00640 1 --2- 172.21.15.152:0/3209478467 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7facd415d1e0 0x7facd4152090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:41.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.186+0000 7facdaa00640 1 --2- 172.21.15.152:0/3209478467 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7facd4071cd0 0x7facd40a9d00 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:41.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.186+0000 7facdaa00640 1 -- 172.21.15.152:0/3209478467 >> 172.21.15.152:0/3209478467 conn(0x7facd4067ae0 msgr2=0x7facd406b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:41.187 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.186+0000 7facdaa00640 1 -- 172.21.15.152:0/3209478467 shutdown_connections 2024-04-20T11:31:41.187 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.186+0000 7facdaa00640 1 -- 172.21.15.152:0/3209478467 wait complete. 2024-04-20T11:31:41.205 INFO:tasks.cephfs_test_runner:Validate handcrafted .meta file on legacy subvol root doesn't break the system ... ok 2024-04-20T11:31:41.205 INFO:tasks.cephfs_test_runner:Starting test: test_mgr_eviction (tasks.cephfs.test_volumes.TestMisc) 2024-04-20T11:31:41.206 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:31:41.403 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.400+0000 7fac95e00640 1 Processor -- start 2024-04-20T11:31:41.403 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.400+0000 7fac95e00640 1 -- start start 2024-04-20T11:31:41.403 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.402+0000 7fac95e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fac9015c560 0x7fac900b3df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:41.403 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.402+0000 7fac95e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fac900d0810 0x7fac900ada90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:41.403 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.402+0000 7fac95e00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac9015fd20 con 0x7fac900b4330 2024-04-20T11:31:41.404 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.402+0000 7fac95e00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fac9015ef60 con 0x7fac9015c560 2024-04-20T11:31:41.404 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.402+0000 7fac95e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fac9015f240 con 0x7fac900d0810 2024-04-20T11:31:41.404 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.403+0000 7fac8f400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fac900d0810 0x7fac900ada90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:41.404 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.403+0000 7fac8f400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fac900d0810 0x7fac900ada90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.152:52352/0 (socket says 172.21.15.152:52352) 2024-04-20T11:31:41.404 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.403+0000 7fac8f400640 1 -- 172.21.15.152:0/1000258803 learned_addr learned my addr 172.21.15.152:0/1000258803 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:41.404 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.403+0000 7fac8ea00640 1 --2- 172.21.15.152:0/1000258803 >> v2:172.21.15.175:3300/0 conn(0x7fac9015c560 0x7fac900b3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:41.404 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.404+0000 7fac8f400640 1 -- 172.21.15.152:0/1000258803 >> v2:172.21.15.175:3300/0 conn(0x7fac9015c560 msgr2=0x7fac900b3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:41.404 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.404+0000 7fac8f400640 1 --2- 172.21.15.152:0/1000258803 >> v2:172.21.15.175:3300/0 conn(0x7fac9015c560 0x7fac900b3df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:41.404 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.404+0000 7fac8f400640 1 -- 172.21.15.152:0/1000258803 >> v1:172.21.15.175:6789/0 conn(0x7fac900b4330 legacy=0x7fac900d0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:41.404 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.404+0000 7fac8f400640 1 -- 172.21.15.152:0/1000258803 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac9015f520 con 0x7fac900d0810 2024-04-20T11:31:41.405 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.405+0000 7fac8f400640 1 --2- 172.21.15.152:0/1000258803 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fac900d0810 0x7fac900ada90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fac88009e50 tx=0x7fac8802bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db8226cf8d7ed821 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:41.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.406+0000 7fac8e000640 1 -- 172.21.15.152:0/1000258803 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac8802ca70 con 0x7fac900d0810 2024-04-20T11:31:41.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.406+0000 7fac8e000640 1 -- 172.21.15.152:0/1000258803 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac8802cc10 con 0x7fac900d0810 2024-04-20T11:31:41.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.406+0000 7fac8e000640 1 -- 172.21.15.152:0/1000258803 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac880343f0 con 0x7fac900d0810 2024-04-20T11:31:41.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.406+0000 7fac95e00640 1 -- 172.21.15.152:0/1000258803 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fac900d0810 msgr2=0x7fac900ada90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:41.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.406+0000 7fac95e00640 1 --2- 172.21.15.152:0/1000258803 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fac900d0810 0x7fac900ada90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fac88009e50 tx=0x7fac8802bed0 comp rx=0 tx=0).stop 2024-04-20T11:31:41.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.407+0000 7fac95e00640 1 -- 172.21.15.152:0/1000258803 shutdown_connections 2024-04-20T11:31:41.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.407+0000 7fac95e00640 1 --2- 172.21.15.152:0/1000258803 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fac900d0810 0x7fac900ada90 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:41.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.407+0000 7fac95e00640 1 --2- 172.21.15.152:0/1000258803 >> v2:172.21.15.175:3300/0 conn(0x7fac9015c560 0x7fac900b3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:41.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.407+0000 7fac95e00640 1 -- 172.21.15.152:0/1000258803 >> 172.21.15.152:0/1000258803 conn(0x7fac90067cf0 msgr2=0x7fac9006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:41.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.407+0000 7fac95e00640 1 -- 172.21.15.152:0/1000258803 shutdown_connections 2024-04-20T11:31:41.408 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.407+0000 7fac95e00640 1 -- 172.21.15.152:0/1000258803 wait complete. 2024-04-20T11:31:41.408 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.408+0000 7fac95e00640 1 Processor -- start 2024-04-20T11:31:41.408 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.409+0000 7fac95e00640 1 -- start start 2024-04-20T11:31:41.409 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.410+0000 7fac95e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fac900b4330 0x7fac90161b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:41.409 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.410+0000 7fac95e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fac900d0810 0x7fac900cef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:41.409 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.410+0000 7fac95e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fac9015c560 0x7fac900cf490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:41.409 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.410+0000 7fac95e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fac900adfd0 con 0x7fac9015c560 2024-04-20T11:31:41.410 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.410+0000 7fac95e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fac900d0dc0 con 0x7fac900b4330 2024-04-20T11:31:41.410 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.410+0000 7fac95e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fac900ae150 con 0x7fac900d0810 2024-04-20T11:31:41.410 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.410+0000 7fac8f400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fac9015c560 0x7fac900cf490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:41.410 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.410+0000 7fac8f400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fac9015c560 0x7fac900cf490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49608/0 (socket says 172.21.15.152:49608) 2024-04-20T11:31:41.410 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.410+0000 7fac8f400640 1 -- 172.21.15.152:0/732735720 learned_addr learned my addr 172.21.15.152:0/732735720 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:41.411 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.410+0000 7fac8ea00640 1 --2- 172.21.15.152:0/732735720 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fac900b4330 0x7fac90161b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:41.411 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.410+0000 7fac86000640 1 --2- 172.21.15.152:0/732735720 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fac900d0810 0x7fac900cef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:41.411 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.410+0000 7fac8f400640 1 -- 172.21.15.152:0/732735720 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fac900d0810 msgr2=0x7fac900cef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:41.411 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.410+0000 7fac8f400640 1 --2- 172.21.15.152:0/732735720 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fac900d0810 0x7fac900cef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:41.411 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.410+0000 7fac8f400640 1 -- 172.21.15.152:0/732735720 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fac900b4330 msgr2=0x7fac90161b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:41.412 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.410+0000 7fac8f400640 1 --2- 172.21.15.152:0/732735720 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fac900b4330 0x7fac90161b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:41.412 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.410+0000 7fac8f400640 1 -- 172.21.15.152:0/732735720 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac900cdca0 con 0x7fac9015c560 2024-04-20T11:31:41.412 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.411+0000 7fac8f400640 1 --2- 172.21.15.152:0/732735720 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fac9015c560 0x7fac900cf490 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fac88009f70 tx=0x7fac88004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:41.412 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.411+0000 7fac87e00640 1 -- 172.21.15.152:0/732735720 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac88043020 con 0x7fac9015c560 2024-04-20T11:31:41.412 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.411+0000 7fac87e00640 1 -- 172.21.15.152:0/732735720 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac8803e450 con 0x7fac9015c560 2024-04-20T11:31:41.412 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.411+0000 7fac87e00640 1 -- 172.21.15.152:0/732735720 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac88009510 con 0x7fac9015c560 2024-04-20T11:31:41.412 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.411+0000 7fac95e00640 1 -- 172.21.15.152:0/732735720 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac900cdf80 con 0x7fac9015c560 2024-04-20T11:31:41.412 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.411+0000 7fac95e00640 1 -- 172.21.15.152:0/732735720 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac900ce540 con 0x7fac9015c560 2024-04-20T11:31:41.412 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.412+0000 7fac95e00640 1 -- 172.21.15.152:0/732735720 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac900689f0 con 0x7fac9015c560 2024-04-20T11:31:41.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.413+0000 7fac87e00640 1 -- 172.21.15.152:0/732735720 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 11) ==== 91346+0+0 (secure 0 0 0) 0x7fac88046020 con 0x7fac9015c560 2024-04-20T11:31:41.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.414+0000 7fac87e00640 1 --2- 172.21.15.152:0/732735720 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fac6c06cc50 0x7fac6c06f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:41.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.414+0000 7fac87e00640 1 -- 172.21.15.152:0/732735720 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 4943+0+0 (secure 0 0 0) 0x7fac880ae570 con 0x7fac9015c560 2024-04-20T11:31:41.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.415+0000 7fac8ea00640 1 --2- 172.21.15.152:0/732735720 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fac6c06cc50 0x7fac6c06f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:41.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.415+0000 7fac8ea00640 1 --2- 172.21.15.152:0/732735720 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fac6c06cc50 0x7fac6c06f110 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fac80009a30 tx=0x7fac80008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:41.418 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.419+0000 7fac87e00640 1 -- 172.21.15.152:0/732735720 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fac8804d030 con 0x7fac9015c560 2024-04-20T11:31:41.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.621+0000 7fac95e00640 1 -- 172.21.15.152:0/732735720 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fac9015ba40 con 0x7fac9015c560 2024-04-20T11:31:41.623 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.623+0000 7fac87e00640 1 -- 172.21.15.152:0/732735720 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) ==== 74+0+13080 (secure 0 0 0) 0x7fac8807c850 con 0x7fac9015c560 2024-04-20T11:31:41.623 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:31:41.623 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":63,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:31:40.170555+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.152:6837/488955771":"2024-04-21T11:31:37.150733+0000","172.21.15.152:6835/488955771":"2024-04-21T11:31:37.150733+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:31:41.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.626+0000 7fac95e00640 1 -- 172.21.15.152:0/732735720 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fac6c06cc50 msgr2=0x7fac6c06f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:41.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.626+0000 7fac95e00640 1 --2- 172.21.15.152:0/732735720 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fac6c06cc50 0x7fac6c06f110 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fac80009a30 tx=0x7fac80008040 comp rx=0 tx=0).stop 2024-04-20T11:31:41.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.626+0000 7fac95e00640 1 -- 172.21.15.152:0/732735720 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fac9015c560 msgr2=0x7fac900cf490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:41.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.626+0000 7fac95e00640 1 --2- 172.21.15.152:0/732735720 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fac9015c560 0x7fac900cf490 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fac88009f70 tx=0x7fac88004a40 comp rx=0 tx=0).stop 2024-04-20T11:31:41.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.626+0000 7fac95e00640 1 -- 172.21.15.152:0/732735720 shutdown_connections 2024-04-20T11:31:41.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.626+0000 7fac95e00640 1 --2- 172.21.15.152:0/732735720 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fac6c06cc50 0x7fac6c06f110 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:41.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.626+0000 7fac95e00640 1 --2- 172.21.15.152:0/732735720 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fac9015c560 0x7fac900cf490 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:41.627 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.626+0000 7fac95e00640 1 --2- 172.21.15.152:0/732735720 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fac900d0810 0x7fac900cef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:41.627 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.626+0000 7fac95e00640 1 --2- 172.21.15.152:0/732735720 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fac900b4330 0x7fac90161b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:41.627 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.626+0000 7fac95e00640 1 -- 172.21.15.152:0/732735720 >> 172.21.15.152:0/732735720 conn(0x7fac90067cf0 msgr2=0x7fac900b3710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:41.627 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.626+0000 7fac95e00640 1 -- 172.21.15.152:0/732735720 shutdown_connections 2024-04-20T11:31:41.627 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.626+0000 7fac95e00640 1 -- 172.21.15.152:0/732735720 wait complete. 2024-04-20T11:31:41.644 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T11:26:47.246571+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T11:31:41.644 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T11:31:41.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.827+0000 7ffa2d400640 1 Processor -- start 2024-04-20T11:31:41.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.827+0000 7ffa2d400640 1 -- start start 2024-04-20T11:31:41.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.829+0000 7ffa2d400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa28156c00 0x7ffa28160a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:41.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.829+0000 7ffa2d400640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ffa2815ca40 0x7ffa2815cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:41.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.829+0000 7ffa2d400640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa28160fd0 con 0x7ffa2806bda0 2024-04-20T11:31:41.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.829+0000 7ffa2d400640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7ffa2815d5c0 con 0x7ffa2815ca40 2024-04-20T11:31:41.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.829+0000 7ffa2d400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ffa2815d8a0 con 0x7ffa28156c00 2024-04-20T11:31:41.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.829+0000 7ffa2ca00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7ffa2806bda0 0x7ffa280b30c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:41366/0 (socket says 172.21.15.152:41366) 2024-04-20T11:31:41.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.829+0000 7ffa2ca00640 1 -- 172.21.15.152:0/2920696942 learned_addr learned my addr 172.21.15.152:0/2920696942 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:31:41.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.830+0000 7ffa27e00640 1 --2- 172.21.15.152:0/2920696942 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa28156c00 0x7ffa28160a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:41.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.830+0000 7ffa1f400640 1 --2- 172.21.15.152:0/2920696942 >> v2:172.21.15.175:3300/0 conn(0x7ffa2815ca40 0x7ffa2815cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:41.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.830+0000 7ffa27400640 1 -- 172.21.15.152:0/2920696942 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1333177720 0 0) 0x7ffa28160fd0 con 0x7ffa2806bda0 2024-04-20T11:31:41.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.830+0000 7ffa27400640 1 -- 172.21.15.152:0/2920696942 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffa0c003520 con 0x7ffa2806bda0 2024-04-20T11:31:41.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.830+0000 7ffa27400640 1 -- 172.21.15.152:0/2920696942 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 454572315 0 0) 0x7ffa0c003520 con 0x7ffa2806bda0 2024-04-20T11:31:41.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.830+0000 7ffa27400640 1 -- 172.21.15.152:0/2920696942 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa28156c00 msgr2=0x7ffa28160a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:41.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.830+0000 7ffa27400640 1 --2- 172.21.15.152:0/2920696942 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa28156c00 0x7ffa28160a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:41.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.830+0000 7ffa27400640 1 -- 172.21.15.152:0/2920696942 >> v2:172.21.15.175:3300/0 conn(0x7ffa2815ca40 msgr2=0x7ffa2815cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:41.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.830+0000 7ffa27400640 1 --2- 172.21.15.152:0/2920696942 >> v2:172.21.15.175:3300/0 conn(0x7ffa2815ca40 0x7ffa2815cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:41.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.830+0000 7ffa27400640 1 -- 172.21.15.152:0/2920696942 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa28160290 con 0x7ffa2806bda0 2024-04-20T11:31:41.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.830+0000 7ffa27400640 1 -- 172.21.15.152:0/2920696942 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7ffa20001670 con 0x7ffa2806bda0 2024-04-20T11:31:41.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.830+0000 7ffa27400640 1 -- 172.21.15.152:0/2920696942 >> v1:172.21.15.152:6789/0 conn(0x7ffa2806bda0 legacy=0x7ffa280b30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:41.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.830+0000 7ffa27400640 1 --2- 172.21.15.152:0/2920696942 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffa0c0039b0 0x7ffa0c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:41.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.830+0000 7ffa27400640 1 --2- 172.21.15.152:0/2920696942 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa28156c00 0x7ffa0c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:41.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.830+0000 7ffa27400640 1 --2- 172.21.15.152:0/2920696942 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa0c004d40 0x7ffa0c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:41.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.831+0000 7ffa27400640 1 -- 172.21.15.152:0/2920696942 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7ffa2815d5c0 con 0x7ffa0c0039b0 2024-04-20T11:31:41.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.831+0000 7ffa27400640 1 -- 172.21.15.152:0/2920696942 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ffa2815d8a0 con 0x7ffa0c004d40 2024-04-20T11:31:41.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.831+0000 7ffa27400640 1 -- 172.21.15.152:0/2920696942 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ffa0c00b9c0 con 0x7ffa28156c00 2024-04-20T11:31:41.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.831+0000 7ffa27e00640 1 --2- 172.21.15.152:0/2920696942 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffa0c0039b0 0x7ffa0c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:41.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.831+0000 7ffa1f400640 1 --2- 172.21.15.152:0/2920696942 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa28156c00 0x7ffa0c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:41.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.831+0000 7ffa2ca00640 1 --2- 172.21.15.152:0/2920696942 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa0c004d40 0x7ffa0c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:41.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.831+0000 7ffa27e00640 1 -- 172.21.15.152:0/2920696942 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa28156c00 msgr2=0x7ffa0c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:41.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.831+0000 7ffa27e00640 1 --2- 172.21.15.152:0/2920696942 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa28156c00 0x7ffa0c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:41.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.831+0000 7ffa27e00640 1 -- 172.21.15.152:0/2920696942 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa0c004d40 msgr2=0x7ffa0c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:41.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.831+0000 7ffa27e00640 1 --2- 172.21.15.152:0/2920696942 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa0c004d40 0x7ffa0c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:41.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.831+0000 7ffa27e00640 1 -- 172.21.15.152:0/2920696942 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa0c003520 con 0x7ffa0c0039b0 2024-04-20T11:31:41.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.831+0000 7ffa2ca00640 1 --2- 172.21.15.152:0/2920696942 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa0c004d40 0x7ffa0c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:31:41.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.833+0000 7ffa27e00640 1 --2- 172.21.15.152:0/2920696942 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffa0c0039b0 0x7ffa0c003dd0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7ffa18009fc0 tx=0x7ffa0c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb0a66a7e38c719c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:41.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.833+0000 7ffa27400640 1 -- 172.21.15.152:0/2920696942 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa18039070 con 0x7ffa0c0039b0 2024-04-20T11:31:41.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.833+0000 7ffa27400640 1 -- 172.21.15.152:0/2920696942 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa1802cc20 con 0x7ffa0c0039b0 2024-04-20T11:31:41.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.833+0000 7ffa27400640 1 -- 172.21.15.152:0/2920696942 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa18009c10 con 0x7ffa0c0039b0 2024-04-20T11:31:41.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.833+0000 7ffa2d400640 1 -- 172.21.15.152:0/2920696942 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffa0c0039b0 msgr2=0x7ffa0c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:41.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.833+0000 7ffa2d400640 1 --2- 172.21.15.152:0/2920696942 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffa0c0039b0 0x7ffa0c003dd0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7ffa18009fc0 tx=0x7ffa0c005830 comp rx=0 tx=0).stop 2024-04-20T11:31:41.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.833+0000 7ffa2ca00640 1 -- 172.21.15.152:0/2920696942 reap_dead start 2024-04-20T11:31:41.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.834+0000 7ffa2d400640 1 -- 172.21.15.152:0/2920696942 shutdown_connections 2024-04-20T11:31:41.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.834+0000 7ffa2d400640 1 -- 172.21.15.152:0/2920696942 >> 172.21.15.152:0/2920696942 conn(0x7ffa280b21e0 msgr2=0x7ffa280b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:41.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.834+0000 7ffa2d400640 1 -- 172.21.15.152:0/2920696942 shutdown_connections 2024-04-20T11:31:41.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.834+0000 7ffa2d400640 1 -- 172.21.15.152:0/2920696942 wait complete. 2024-04-20T11:31:41.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.835+0000 7ffa2d400640 1 Processor -- start 2024-04-20T11:31:41.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.835+0000 7ffa2d400640 1 -- start start 2024-04-20T11:31:41.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.836+0000 7ffa2d400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa0c0039b0 0x7ffa2814b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:41.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.836+0000 7ffa2d400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffa2814ef60 0x7ffa280b45e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:41.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.836+0000 7ffa2d400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa2814fb70 0x7ffa28158cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:41.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.836+0000 7ffa2d400640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7ffa28160c10 con 0x7ffa2814ef60 2024-04-20T11:31:41.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.836+0000 7ffa2d400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ffa28160a90 con 0x7ffa0c0039b0 2024-04-20T11:31:41.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.836+0000 7ffa2d400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ffa28160d90 con 0x7ffa2814fb70 2024-04-20T11:31:41.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.837+0000 7ffa1f400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffa2814ef60 0x7ffa280b45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:41.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.837+0000 7ffa1f400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffa2814ef60 0x7ffa280b45e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49632/0 (socket says 172.21.15.152:49632) 2024-04-20T11:31:41.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.837+0000 7ffa1f400640 1 -- 172.21.15.152:0/3349813448 learned_addr learned my addr 172.21.15.152:0/3349813448 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:41.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.837+0000 7ffa27e00640 1 --2- 172.21.15.152:0/3349813448 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa0c0039b0 0x7ffa2814b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:41.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.837+0000 7ffa2ca00640 1 --2- 172.21.15.152:0/3349813448 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa2814fb70 0x7ffa28158cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:41.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.837+0000 7ffa1f400640 1 -- 172.21.15.152:0/3349813448 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa2814fb70 msgr2=0x7ffa28158cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:41.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.837+0000 7ffa1f400640 1 --2- 172.21.15.152:0/3349813448 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa2814fb70 0x7ffa28158cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:41.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.837+0000 7ffa1f400640 1 -- 172.21.15.152:0/3349813448 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa0c0039b0 msgr2=0x7ffa2814b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:41.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.837+0000 7ffa1f400640 1 --2- 172.21.15.152:0/3349813448 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa0c0039b0 0x7ffa2814b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:41.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.837+0000 7ffa1f400640 1 -- 172.21.15.152:0/3349813448 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa2814bf20 con 0x7ffa2814ef60 2024-04-20T11:31:41.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.837+0000 7ffa1f400640 1 --2- 172.21.15.152:0/3349813448 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffa2814ef60 0x7ffa280b45e0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7ffa10011a70 tx=0x7ffa10011f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:41.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.837+0000 7ffa24c00640 1 -- 172.21.15.152:0/3349813448 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa1000fb50 con 0x7ffa2814ef60 2024-04-20T11:31:41.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.838+0000 7ffa24c00640 1 -- 172.21.15.152:0/3349813448 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa1000fcf0 con 0x7ffa2814ef60 2024-04-20T11:31:41.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.838+0000 7ffa24c00640 1 -- 172.21.15.152:0/3349813448 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa1000fea0 con 0x7ffa2814ef60 2024-04-20T11:31:41.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.838+0000 7ffa2d400640 1 -- 172.21.15.152:0/3349813448 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa2814f4b0 con 0x7ffa2814ef60 2024-04-20T11:31:41.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.838+0000 7ffa2d400640 1 -- 172.21.15.152:0/3349813448 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa281584b0 con 0x7ffa2814ef60 2024-04-20T11:31:41.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.839+0000 7ffa2d400640 1 -- 172.21.15.152:0/3349813448 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa28075e60 con 0x7ffa2814ef60 2024-04-20T11:31:41.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.840+0000 7ffa24c00640 1 -- 172.21.15.152:0/3349813448 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 11) ==== 91346+0+0 (secure 0 0 0) 0x7ffa10012040 con 0x7ffa2814ef60 2024-04-20T11:31:41.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.841+0000 7ffa24c00640 1 --2- 172.21.15.152:0/3349813448 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ffa0406cc80 0x7ffa0406f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:41.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.841+0000 7ffa24c00640 1 -- 172.21.15.152:0/3349813448 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 4943+0+0 (secure 0 0 0) 0x7ffa1008c390 con 0x7ffa2814ef60 2024-04-20T11:31:41.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.842+0000 7ffa27e00640 1 --2- 172.21.15.152:0/3349813448 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ffa0406cc80 0x7ffa0406f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:41.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.842+0000 7ffa27e00640 1 --2- 172.21.15.152:0/3349813448 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ffa0406cc80 0x7ffa0406f140 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7ffa18036000 tx=0x7ffa1802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:41.845 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:41.846+0000 7ffa24c00640 1 -- 172.21.15.152:0/3349813448 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ffa10019030 con 0x7ffa2814ef60 2024-04-20T11:31:42.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:42.045+0000 7ffa2d400640 1 -- 172.21.15.152:0/3349813448 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ffa2806c3d0 con 0x7ffa2814ef60 2024-04-20T11:31:42.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:42.046+0000 7ffa24c00640 1 -- 172.21.15.152:0/3349813448 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v63) ==== 93+0+10 (secure 0 0 0) 0x7ffa1005a690 con 0x7ffa2814ef60 2024-04-20T11:31:42.045 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 1 2024-04-20T11:31:42.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:42.048+0000 7ffa2d400640 1 -- 172.21.15.152:0/3349813448 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ffa0406cc80 msgr2=0x7ffa0406f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:42.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:42.048+0000 7ffa2d400640 1 --2- 172.21.15.152:0/3349813448 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ffa0406cc80 0x7ffa0406f140 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7ffa18036000 tx=0x7ffa1802e000 comp rx=0 tx=0).stop 2024-04-20T11:31:42.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:42.048+0000 7ffa2d400640 1 -- 172.21.15.152:0/3349813448 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffa2814ef60 msgr2=0x7ffa280b45e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:42.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:42.048+0000 7ffa2d400640 1 --2- 172.21.15.152:0/3349813448 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffa2814ef60 0x7ffa280b45e0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7ffa10011a70 tx=0x7ffa10011f30 comp rx=0 tx=0).stop 2024-04-20T11:31:42.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:42.048+0000 7ffa2d400640 1 -- 172.21.15.152:0/3349813448 shutdown_connections 2024-04-20T11:31:42.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:42.048+0000 7ffa2d400640 1 --2- 172.21.15.152:0/3349813448 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ffa0406cc80 0x7ffa0406f140 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:42.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:42.048+0000 7ffa2d400640 1 --2- 172.21.15.152:0/3349813448 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa2814fb70 0x7ffa28158cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:42.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:42.048+0000 7ffa2d400640 1 --2- 172.21.15.152:0/3349813448 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffa2814ef60 0x7ffa280b45e0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:42.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:42.048+0000 7ffa2d400640 1 --2- 172.21.15.152:0/3349813448 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa0c0039b0 0x7ffa2814b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:42.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:42.048+0000 7ffa2d400640 1 -- 172.21.15.152:0/3349813448 >> 172.21.15.152:0/3349813448 conn(0x7ffa280b21e0 msgr2=0x7ffa2815dc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:42.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:42.049+0000 7ffa2d400640 1 -- 172.21.15.152:0/3349813448 shutdown_connections 2024-04-20T11:31:42.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:42.049+0000 7ffa2d400640 1 -- 172.21.15.152:0/3349813448 wait complete. 2024-04-20T11:31:42.068 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_mgr_eviction' 2024-04-20T11:31:42.257 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:42.257+0000 7f27ea000640 1 Processor -- start 2024-04-20T11:31:42.257 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:42.258+0000 7f27ea000640 1 -- start start 2024-04-20T11:31:42.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:42.259+0000 7f27ea000640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f27e415d1e0 0x7f27e4160b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:42.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:42.259+0000 7f27ea000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f27e4071cd0 0x7f27e4072160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:42.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:42.259+0000 7f27ea000640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27e40ce8c0 con 0x7f27e415d5c0 2024-04-20T11:31:42.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:42.259+0000 7f27ea000640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f27e40726a0 con 0x7f27e415d1e0 2024-04-20T11:31:42.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:42.259+0000 7f27ea000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f27e41600b0 con 0x7f27e4071cd0 2024-04-20T11:31:42.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:42.260+0000 7f27e9600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f27e4071cd0 0x7f27e4072160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:42.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:42.260+0000 7f27e9600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f27e4071cd0 0x7f27e4072160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.152:52390/0 (socket says 172.21.15.152:52390) 2024-04-20T11:31:42.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:42.260+0000 7f27e9600640 1 -- 172.21.15.152:0/3912469646 learned_addr learned my addr 172.21.15.152:0/3912469646 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:42.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:42.260+0000 7f27e8c00640 1 --2- 172.21.15.152:0/3912469646 >> v2:172.21.15.175:3300/0 conn(0x7f27e415d1e0 0x7f27e4160b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:42.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:42.261+0000 7f27e8c00640 1 -- 172.21.15.152:0/3912469646 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f27e4071cd0 msgr2=0x7f27e4072160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:42.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:42.261+0000 7f27e8c00640 1 --2- 172.21.15.152:0/3912469646 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f27e4071cd0 0x7f27e4072160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:42.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:42.261+0000 7f27e8c00640 1 -- 172.21.15.152:0/3912469646 >> v1:172.21.15.175:6789/0 conn(0x7f27e415d5c0 legacy=0x7f27e40cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:42.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:42.261+0000 7f27e8c00640 1 -- 172.21.15.152:0/3912469646 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27e4160390 con 0x7f27e415d1e0 2024-04-20T11:31:42.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:42.262+0000 7f27e8c00640 1 --2- 172.21.15.152:0/3912469646 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27e415d1e0 0x7f27e4160b90 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f27d40097d0 tx=0x7f27d402b8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=45514ea624c0fbaf server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:42.262 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:42.262+0000 7f27e3e00640 1 -- 172.21.15.152:0/3912469646 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d4035020 con 0x7f27e415d1e0 2024-04-20T11:31:42.262 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:42.262+0000 7f27e3e00640 1 -- 172.21.15.152:0/3912469646 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27d4031800 con 0x7f27e415d1e0 2024-04-20T11:31:42.262 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:42.262+0000 7f27e3e00640 1 -- 172.21.15.152:0/3912469646 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d4031bd0 con 0x7f27e415d1e0 2024-04-20T11:31:42.263 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:42.263+0000 7f27ea000640 1 -- 172.21.15.152:0/3912469646 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27e415d1e0 msgr2=0x7f27e4160b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:42.263 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:42.263+0000 7f27ea000640 1 --2- 172.21.15.152:0/3912469646 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27e415d1e0 0x7f27e4160b90 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f27d40097d0 tx=0x7f27d402b8a0 comp rx=0 tx=0).stop 2024-04-20T11:31:42.263 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:42.263+0000 7f27ea000640 1 -- 172.21.15.152:0/3912469646 shutdown_connections 2024-04-20T11:31:42.263 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:42.263+0000 7f27ea000640 1 --2- 172.21.15.152:0/3912469646 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f27e4071cd0 0x7f27e4072160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:42.263 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:42.263+0000 7f27ea000640 1 --2- 172.21.15.152:0/3912469646 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27e415d1e0 0x7f27e4160b90 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:42.263 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:42.263+0000 7f27ea000640 1 -- 172.21.15.152:0/3912469646 >> 172.21.15.152:0/3912469646 conn(0x7f27e4067ae0 msgr2=0x7f27e406b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:42.263 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:42.263+0000 7f27ea000640 1 -- 172.21.15.152:0/3912469646 shutdown_connections 2024-04-20T11:31:42.263 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:42.263+0000 7f27ea000640 1 -- 172.21.15.152:0/3912469646 wait complete. 2024-04-20T11:31:42.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:42.265+0000 7f27ea000640 1 Processor -- start 2024-04-20T11:31:42.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:42.265+0000 7f27ea000640 1 -- start start 2024-04-20T11:31:42.266 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:42.266+0000 7f27ea000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f27e4071cd0 0x7f27e4157050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:42.266 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:42.266+0000 7f27ea000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27e415d1e0 0x7f27e4153a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:42.266 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:42.266+0000 7f27ea000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f27e415d5c0 0x7f27e41f42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:42.266 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:42.266+0000 7f27ea000640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f27e40ccd70 con 0x7f27e4071cd0 2024-04-20T11:31:42.266 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:42.266+0000 7f27ea000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f27e40ceba0 con 0x7f27e415d1e0 2024-04-20T11:31:42.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:42.266+0000 7f27ea000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f27e40ccef0 con 0x7f27e415d5c0 2024-04-20T11:31:42.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:42.266+0000 7f27e8c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f27e4071cd0 0x7f27e4157050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:42.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:42.266+0000 7f27e8c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f27e4071cd0 0x7f27e4157050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49646/0 (socket says 172.21.15.152:49646) 2024-04-20T11:31:42.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:42.266+0000 7f27e8c00640 1 -- 172.21.15.152:0/761137875 learned_addr learned my addr 172.21.15.152:0/761137875 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:42.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:42.267+0000 7f27dbe00640 1 --2- 172.21.15.152:0/761137875 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27e415d1e0 0x7f27e4153a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:42.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:42.267+0000 7f27e8c00640 1 -- 172.21.15.152:0/761137875 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f27e415d5c0 msgr2=0x7f27e41f42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:42.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:42.267+0000 7f27e8c00640 1 --2- 172.21.15.152:0/761137875 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f27e415d5c0 0x7f27e41f42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:42.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:42.267+0000 7f27e8c00640 1 -- 172.21.15.152:0/761137875 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27e415d1e0 msgr2=0x7f27e4153a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:42.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:42.267+0000 7f27e8c00640 1 --2- 172.21.15.152:0/761137875 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27e415d1e0 0x7f27e4153a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:42.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:42.267+0000 7f27e8c00640 1 -- 172.21.15.152:0/761137875 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27e40add70 con 0x7f27e4071cd0 2024-04-20T11:31:42.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:42.267+0000 7f27dbe00640 1 --2- 172.21.15.152:0/761137875 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27e415d1e0 0x7f27e4153a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:31:42.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:42.267+0000 7f27e8c00640 1 --2- 172.21.15.152:0/761137875 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f27e4071cd0 0x7f27e4157050 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f27d40098f0 tx=0x7f27d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:42.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:42.267+0000 7f27e1600640 1 -- 172.21.15.152:0/761137875 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d4041070 con 0x7f27e4071cd0 2024-04-20T11:31:42.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:42.267+0000 7f27e1600640 1 -- 172.21.15.152:0/761137875 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27d403ac80 con 0x7f27e4071cd0 2024-04-20T11:31:42.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:42.267+0000 7f27ea000640 1 -- 172.21.15.152:0/761137875 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27e40ae050 con 0x7f27e4071cd0 2024-04-20T11:31:42.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:42.267+0000 7f27e1600640 1 -- 172.21.15.152:0/761137875 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d4035040 con 0x7f27e4071cd0 2024-04-20T11:31:42.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:42.267+0000 7f27ea000640 1 -- 172.21.15.152:0/761137875 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27e40ae610 con 0x7f27e4071cd0 2024-04-20T11:31:42.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:42.268+0000 7f27ea000640 1 -- 172.21.15.152:0/761137875 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27e41530d0 con 0x7f27e4071cd0 2024-04-20T11:31:42.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:42.269+0000 7f27e1600640 1 -- 172.21.15.152:0/761137875 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 11) ==== 91346+0+0 (secure 0 0 0) 0x7f27d404c060 con 0x7f27e4071cd0 2024-04-20T11:31:42.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:42.270+0000 7f27e1600640 1 --2- 172.21.15.152:0/761137875 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f27c006cc50 0x7f27c006f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:42.270 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:42.270+0000 7f27e1600640 1 -- 172.21.15.152:0/761137875 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 4943+0+0 (secure 0 0 0) 0x7f27d40ad870 con 0x7f27e4071cd0 2024-04-20T11:31:42.270 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:42.270+0000 7f27dbe00640 1 --2- 172.21.15.152:0/761137875 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f27c006cc50 0x7f27c006f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:42.270 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:42.271+0000 7f27dbe00640 1 --2- 172.21.15.152:0/761137875 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f27c006cc50 0x7f27c006f110 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f27e40b46e0 tx=0x7f27cc009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:42.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:42.274+0000 7f27e1600640 1 -- 172.21.15.152:0/761137875 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f27e41530d0 con 0x7f27e4071cd0 2024-04-20T11:31:42.541 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:42.542+0000 7f27ea000640 1 -- 172.21.15.152:0/761137875 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_mgr_eviction"]} v 0) -- 0x7f27e4075e60 con 0x7f27e4071cd0 2024-04-20T11:31:43.195 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:43.195+0000 7f27e1600640 1 -- 172.21.15.152:0/761137875 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_mgr_eviction"]}]=0 v257) ==== 134+0+0 (secure 0 0 0) 0x7f27d407bbe0 con 0x7f27e4071cd0 2024-04-20T11:31:43.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:43.199+0000 7f27ea000640 1 -- 172.21.15.152:0/761137875 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f27c006cc50 msgr2=0x7f27c006f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:43.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:43.199+0000 7f27ea000640 1 --2- 172.21.15.152:0/761137875 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f27c006cc50 0x7f27c006f110 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f27e40b46e0 tx=0x7f27cc009040 comp rx=0 tx=0).stop 2024-04-20T11:31:43.199 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:43.199+0000 7f27ea000640 1 -- 172.21.15.152:0/761137875 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f27e4071cd0 msgr2=0x7f27e4157050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:43.199 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:43.199+0000 7f27ea000640 1 --2- 172.21.15.152:0/761137875 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f27e4071cd0 0x7f27e4157050 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f27d40098f0 tx=0x7f27d40046c0 comp rx=0 tx=0).stop 2024-04-20T11:31:43.199 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:43.199+0000 7f27ea000640 1 -- 172.21.15.152:0/761137875 shutdown_connections 2024-04-20T11:31:43.199 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:43.199+0000 7f27ea000640 1 --2- 172.21.15.152:0/761137875 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f27c006cc50 0x7f27c006f110 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:43.199 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:43.199+0000 7f27ea000640 1 --2- 172.21.15.152:0/761137875 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f27e415d5c0 0x7f27e41f42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:43.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:43.199+0000 7f27ea000640 1 --2- 172.21.15.152:0/761137875 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27e415d1e0 0x7f27e4153a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:43.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:43.199+0000 7f27ea000640 1 --2- 172.21.15.152:0/761137875 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f27e4071cd0 0x7f27e4157050 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:43.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:43.199+0000 7f27ea000640 1 -- 172.21.15.152:0/761137875 >> 172.21.15.152:0/761137875 conn(0x7f27e4067ae0 msgr2=0x7f27e415a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:43.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:43.199+0000 7f27ea000640 1 -- 172.21.15.152:0/761137875 shutdown_connections 2024-04-20T11:31:43.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:43.199+0000 7f27ea000640 1 -- 172.21.15.152:0/761137875 wait complete. 2024-04-20T11:31:43.223 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-20T11:31:43.422 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:43.422+0000 7f9d76200640 1 Processor -- start 2024-04-20T11:31:43.422 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:43.422+0000 7f9d76200640 1 -- start start 2024-04-20T11:31:43.424 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:43.424+0000 7f9d76200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d7006ee00 0x7f9d7006f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:43.424 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:43.424+0000 7f9d76200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f9d7015fbf0 0x7f9d7015c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:43.424 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:43.424+0000 7f9d76200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d700d00d0 con 0x7f9d700cf110 2024-04-20T11:31:43.424 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:43.424+0000 7f9d76200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f9d7006f720 con 0x7f9d7015fbf0 2024-04-20T11:31:43.424 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:43.424+0000 7f9d76200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f9d700cfb10 con 0x7f9d7006ee00 2024-04-20T11:31:43.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:43.424+0000 7f9d67e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f9d7015fbf0 0x7f9d7015c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:43.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:43.424+0000 7f9d67e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f9d7015fbf0 0x7f9d7015c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49652/0 (socket says 172.21.15.152:49652) 2024-04-20T11:31:43.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:43.424+0000 7f9d67e00640 1 -- 172.21.15.152:0/1182235967 learned_addr learned my addr 172.21.15.152:0/1182235967 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:43.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:43.425+0000 7f9d74e00640 1 --2- 172.21.15.152:0/1182235967 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d7006ee00 0x7f9d7006f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:43.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:43.425+0000 7f9d67e00640 1 -- 172.21.15.152:0/1182235967 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d7006ee00 msgr2=0x7f9d7006f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:43.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:43.425+0000 7f9d67e00640 1 --2- 172.21.15.152:0/1182235967 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d7006ee00 0x7f9d7006f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:43.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:43.425+0000 7f9d67e00640 1 -- 172.21.15.152:0/1182235967 >> v1:172.21.15.175:6789/0 conn(0x7f9d700cf110 legacy=0x7f9d700b3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:43.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:43.425+0000 7f9d67e00640 1 -- 172.21.15.152:0/1182235967 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d700cfdf0 con 0x7f9d7015fbf0 2024-04-20T11:31:43.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:43.426+0000 7f9d67e00640 1 --2- 172.21.15.152:0/1182235967 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d7015fbf0 0x7f9d7015c390 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f9d58009870 tx=0x7f9d5802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed17763a8a4be1be server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:43.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:43.426+0000 7f9d6fe00640 1 -- 172.21.15.152:0/1182235967 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d58035020 con 0x7f9d7015fbf0 2024-04-20T11:31:43.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:43.426+0000 7f9d6fe00640 1 -- 172.21.15.152:0/1182235967 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d58031640 con 0x7f9d7015fbf0 2024-04-20T11:31:43.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:43.426+0000 7f9d6fe00640 1 -- 172.21.15.152:0/1182235967 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d5802d040 con 0x7f9d7015fbf0 2024-04-20T11:31:43.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:43.427+0000 7f9d76200640 1 -- 172.21.15.152:0/1182235967 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d7015fbf0 msgr2=0x7f9d7015c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:43.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:43.427+0000 7f9d76200640 1 --2- 172.21.15.152:0/1182235967 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d7015fbf0 0x7f9d7015c390 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f9d58009870 tx=0x7f9d5802b8f0 comp rx=0 tx=0).stop 2024-04-20T11:31:43.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:43.427+0000 7f9d76200640 1 -- 172.21.15.152:0/1182235967 shutdown_connections 2024-04-20T11:31:43.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:43.427+0000 7f9d76200640 1 --2- 172.21.15.152:0/1182235967 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d7015fbf0 0x7f9d7015c390 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:43.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:43.427+0000 7f9d76200640 1 --2- 172.21.15.152:0/1182235967 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d7006ee00 0x7f9d7006f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:43.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:43.427+0000 7f9d76200640 1 -- 172.21.15.152:0/1182235967 >> 172.21.15.152:0/1182235967 conn(0x7f9d70067ae0 msgr2=0x7f9d7006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:43.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:43.427+0000 7f9d76200640 1 -- 172.21.15.152:0/1182235967 shutdown_connections 2024-04-20T11:31:43.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:43.427+0000 7f9d76200640 1 -- 172.21.15.152:0/1182235967 wait complete. 2024-04-20T11:31:43.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:43.429+0000 7f9d76200640 1 Processor -- start 2024-04-20T11:31:43.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:43.429+0000 7f9d76200640 1 -- start start 2024-04-20T11:31:43.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:43.430+0000 7f9d76200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d7006ee00 0x7f9d7015af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:43.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:43.430+0000 7f9d76200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d700cf110 0x7f9d70136c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:43.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:43.430+0000 7f9d76200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d7015fbf0 0x7f9d70135db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:43.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:43.430+0000 7f9d76200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f9d7015c8d0 con 0x7f9d7006ee00 2024-04-20T11:31:43.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:43.430+0000 7f9d76200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9d7015be70 con 0x7f9d7015fbf0 2024-04-20T11:31:43.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:43.430+0000 7f9d76200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f9d70144970 con 0x7f9d700cf110 2024-04-20T11:31:43.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:43.430+0000 7f9d74e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d7006ee00 0x7f9d7015af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:43.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:43.430+0000 7f9d74e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d7006ee00 0x7f9d7015af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49654/0 (socket says 172.21.15.152:49654) 2024-04-20T11:31:43.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:43.430+0000 7f9d74e00640 1 -- 172.21.15.152:0/1235076303 learned_addr learned my addr 172.21.15.152:0/1235076303 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:43.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:43.430+0000 7f9d74e00640 1 -- 172.21.15.152:0/1235076303 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d700cf110 msgr2=0x7f9d70136c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:43.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:43.430+0000 7f9d74e00640 1 --2- 172.21.15.152:0/1235076303 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d700cf110 0x7f9d70136c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:43.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:43.430+0000 7f9d74e00640 1 -- 172.21.15.152:0/1235076303 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d7015fbf0 msgr2=0x7f9d70135db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:43.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:43.430+0000 7f9d74e00640 1 --2- 172.21.15.152:0/1235076303 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d7015fbf0 0x7f9d70135db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:43.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:43.430+0000 7f9d74e00640 1 -- 172.21.15.152:0/1235076303 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d700b03e0 con 0x7f9d7006ee00 2024-04-20T11:31:43.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:43.431+0000 7f9d74e00640 1 --2- 172.21.15.152:0/1235076303 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d7006ee00 0x7f9d7015af90 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f9d6000d800 tx=0x7f9d6000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:43.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:43.431+0000 7f9d6d600640 1 -- 172.21.15.152:0/1235076303 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d6000b6c0 con 0x7f9d7006ee00 2024-04-20T11:31:43.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:43.431+0000 7f9d6d600640 1 -- 172.21.15.152:0/1235076303 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d60019070 con 0x7f9d7006ee00 2024-04-20T11:31:43.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:43.431+0000 7f9d76200640 1 -- 172.21.15.152:0/1235076303 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d700b06c0 con 0x7f9d7006ee00 2024-04-20T11:31:43.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:43.431+0000 7f9d6d600640 1 -- 172.21.15.152:0/1235076303 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d6000ba80 con 0x7f9d7006ee00 2024-04-20T11:31:43.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:43.431+0000 7f9d76200640 1 -- 172.21.15.152:0/1235076303 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d70135890 con 0x7f9d7006ee00 2024-04-20T11:31:43.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:43.432+0000 7f9d76200640 1 -- 172.21.15.152:0/1235076303 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d70075e60 con 0x7f9d7006ee00 2024-04-20T11:31:43.433 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:43.433+0000 7f9d6d600640 1 -- 172.21.15.152:0/1235076303 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 11) ==== 91346+0+0 (secure 0 0 0) 0x7f9d6000bc20 con 0x7f9d7006ee00 2024-04-20T11:31:43.433 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:43.434+0000 7f9d6d600640 1 --2- 172.21.15.152:0/1235076303 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f9d4c06cc80 0x7f9d4c06f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:43.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:43.434+0000 7f9d6d600640 1 -- 172.21.15.152:0/1235076303 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 4943+0+0 (secure 0 0 0) 0x7f9d6008c1e0 con 0x7f9d7006ee00 2024-04-20T11:31:43.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:43.434+0000 7f9d67e00640 1 --2- 172.21.15.152:0/1235076303 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f9d4c06cc80 0x7f9d4c06f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:43.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:43.435+0000 7f9d67e00640 1 --2- 172.21.15.152:0/1235076303 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f9d4c06cc80 0x7f9d4c06f140 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f9d58036000 tx=0x7f9d58030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:43.438 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:43.439+0000 7f9d6d600640 1 -- 172.21.15.152:0/1235076303 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f9d6001c360 con 0x7f9d7006ee00 2024-04-20T11:31:43.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:43.620+0000 7f9d76200640 1 -- 172.21.15.152:0/1235076303 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f9d700b3740 con 0x7f9d7006ee00 2024-04-20T11:31:43.627 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:43.627+0000 7f9d6d600640 1 -- 172.21.15.152:0/1235076303 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) ==== 122+0+0 (secure 0 0 0) 0x7f9d6005a590 con 0x7f9d7006ee00 2024-04-20T11:31:43.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:43.630+0000 7f9d76200640 1 -- 172.21.15.152:0/1235076303 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f9d4c06cc80 msgr2=0x7f9d4c06f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:43.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:43.630+0000 7f9d76200640 1 --2- 172.21.15.152:0/1235076303 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f9d4c06cc80 0x7f9d4c06f140 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f9d58036000 tx=0x7f9d58030000 comp rx=0 tx=0).stop 2024-04-20T11:31:43.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:43.630+0000 7f9d76200640 1 -- 172.21.15.152:0/1235076303 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d7006ee00 msgr2=0x7f9d7015af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:43.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:43.630+0000 7f9d76200640 1 --2- 172.21.15.152:0/1235076303 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d7006ee00 0x7f9d7015af90 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f9d6000d800 tx=0x7f9d6000dcc0 comp rx=0 tx=0).stop 2024-04-20T11:31:43.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:43.630+0000 7f9d76200640 1 -- 172.21.15.152:0/1235076303 shutdown_connections 2024-04-20T11:31:43.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:43.630+0000 7f9d76200640 1 --2- 172.21.15.152:0/1235076303 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f9d4c06cc80 0x7f9d4c06f140 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:43.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:43.630+0000 7f9d76200640 1 --2- 172.21.15.152:0/1235076303 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d7015fbf0 0x7f9d70135db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:43.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:43.630+0000 7f9d76200640 1 --2- 172.21.15.152:0/1235076303 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d700cf110 0x7f9d70136c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:43.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:43.630+0000 7f9d76200640 1 --2- 172.21.15.152:0/1235076303 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d7006ee00 0x7f9d7015af90 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:43.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:43.630+0000 7f9d76200640 1 -- 172.21.15.152:0/1235076303 >> 172.21.15.152:0/1235076303 conn(0x7f9d70067ae0 msgr2=0x7f9d7006e440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:43.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:43.630+0000 7f9d76200640 1 -- 172.21.15.152:0/1235076303 shutdown_connections 2024-04-20T11:31:43.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:43.630+0000 7f9d76200640 1 -- 172.21.15.152:0/1235076303 wait complete. 2024-04-20T11:31:43.647 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-20T11:31:43.648 DEBUG:teuthology.orchestra.run.smithi152:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-20T11:31:43.746 DEBUG:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-20T11:31:43.865 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-20T11:31:43.866 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-20T11:31:43.866 DEBUG:teuthology.orchestra.run.smithi152:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:31:43.920 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-20T11:31:43.921 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T11:31:43.921 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:31:43.977 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-20T11:31:43.978 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-20T11:31:43.978 DEBUG:teuthology.orchestra.run.smithi152:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:31:44.032 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-20T11:31:44.032 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T11:31:44.032 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:31:44.088 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-20T11:31:44.089 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-20T11:31:44.089 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:31:44.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.300+0000 7f8d14e00640 1 Processor -- start 2024-04-20T11:31:44.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.300+0000 7f8d14e00640 1 -- start start 2024-04-20T11:31:44.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.302+0000 7f8d14e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8d100b3d80 0x7f8d100b4210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:44.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.302+0000 7f8d14e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f8d101643b0 0x7f8d10164800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:44.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.302+0000 7f8d14e00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d1006e140 con 0x7f8d10160870 2024-04-20T11:31:44.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.302+0000 7f8d14e00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f8d10160c50 con 0x7f8d101643b0 2024-04-20T11:31:44.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.302+0000 7f8d14e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8d1006e420 con 0x7f8d100b3d80 2024-04-20T11:31:44.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.302+0000 7f8d0e800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f8d101643b0 0x7f8d10164800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:44.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.303+0000 7f8d0e800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f8d101643b0 0x7f8d10164800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49658/0 (socket says 172.21.15.152:49658) 2024-04-20T11:31:44.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.303+0000 7f8d0e800640 1 -- 172.21.15.152:0/1940293221 learned_addr learned my addr 172.21.15.152:0/1940293221 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:44.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.303+0000 7f8d0e800640 1 -- 172.21.15.152:0/1940293221 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8d100b3d80 msgr2=0x7f8d100b4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:44.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.303+0000 7f8d0d400640 1 --2- 172.21.15.152:0/1940293221 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8d100b3d80 0x7f8d100b4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:44.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.303+0000 7f8d0e800640 1 --2- 172.21.15.152:0/1940293221 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8d100b3d80 0x7f8d100b4210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:44.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.303+0000 7f8d0e800640 1 -- 172.21.15.152:0/1940293221 >> v1:172.21.15.175:6789/0 conn(0x7f8d10160870 legacy=0x7f8d10163c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:44.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.303+0000 7f8d0e800640 1 -- 172.21.15.152:0/1940293221 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d1006e700 con 0x7f8d101643b0 2024-04-20T11:31:44.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.304+0000 7f8d0e800640 1 --2- 172.21.15.152:0/1940293221 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8d101643b0 0x7f8d10164800 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f8d08009870 tx=0x7f8d0802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a7b4d4ee0357b4e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:44.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.305+0000 7f8d0ca00640 1 -- 172.21.15.152:0/1940293221 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d08034070 con 0x7f8d101643b0 2024-04-20T11:31:44.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.305+0000 7f8d0ca00640 1 -- 172.21.15.152:0/1940293221 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d08030840 con 0x7f8d101643b0 2024-04-20T11:31:44.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.305+0000 7f8d0ca00640 1 -- 172.21.15.152:0/1940293221 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d08030c10 con 0x7f8d101643b0 2024-04-20T11:31:44.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.305+0000 7f8d14e00640 1 -- 172.21.15.152:0/1940293221 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8d101643b0 msgr2=0x7f8d10164800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:44.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.305+0000 7f8d14e00640 1 --2- 172.21.15.152:0/1940293221 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8d101643b0 0x7f8d10164800 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f8d08009870 tx=0x7f8d0802b980 comp rx=0 tx=0).stop 2024-04-20T11:31:44.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.305+0000 7f8d14e00640 1 -- 172.21.15.152:0/1940293221 shutdown_connections 2024-04-20T11:31:44.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.305+0000 7f8d14e00640 1 --2- 172.21.15.152:0/1940293221 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8d101643b0 0x7f8d10164800 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:44.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.305+0000 7f8d14e00640 1 --2- 172.21.15.152:0/1940293221 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8d100b3d80 0x7f8d100b4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:44.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.305+0000 7f8d14e00640 1 -- 172.21.15.152:0/1940293221 >> 172.21.15.152:0/1940293221 conn(0x7f8d10067cf0 msgr2=0x7f8d1006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:44.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.306+0000 7f8d14e00640 1 -- 172.21.15.152:0/1940293221 shutdown_connections 2024-04-20T11:31:44.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.306+0000 7f8d14e00640 1 -- 172.21.15.152:0/1940293221 wait complete. 2024-04-20T11:31:44.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.307+0000 7f8d14e00640 1 Processor -- start 2024-04-20T11:31:44.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.307+0000 7f8d14e00640 1 -- start start 2024-04-20T11:31:44.308 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.308+0000 7f8d14e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8d100b3d80 0x7f8d10162090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:44.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.308+0000 7f8d14e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8d10160870 0x7f8d10150260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:44.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.308+0000 7f8d14e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8d101643b0 0x7f8d101507a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:44.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.308+0000 7f8d14e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f8d1006de10 con 0x7f8d101643b0 2024-04-20T11:31:44.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.308+0000 7f8d14e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8d1006ecc0 con 0x7f8d100b3d80 2024-04-20T11:31:44.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.308+0000 7f8d14e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8d1006f1e0 con 0x7f8d10160870 2024-04-20T11:31:44.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.309+0000 7f8d0e800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8d101643b0 0x7f8d101507a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:44.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.309+0000 7f8d0e800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8d101643b0 0x7f8d101507a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49668/0 (socket says 172.21.15.152:49668) 2024-04-20T11:31:44.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.309+0000 7f8d0e800640 1 -- 172.21.15.152:0/3973898286 learned_addr learned my addr 172.21.15.152:0/3973898286 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:44.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.309+0000 7f8d0de00640 1 --2- 172.21.15.152:0/3973898286 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8d100b3d80 0x7f8d10162090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:44.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.309+0000 7f8d0d400640 1 --2- 172.21.15.152:0/3973898286 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8d10160870 0x7f8d10150260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:44.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.309+0000 7f8d0e800640 1 -- 172.21.15.152:0/3973898286 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8d10160870 msgr2=0x7f8d10150260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:44.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.309+0000 7f8d0e800640 1 --2- 172.21.15.152:0/3973898286 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8d10160870 0x7f8d10150260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:44.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.309+0000 7f8d0e800640 1 -- 172.21.15.152:0/3973898286 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8d100b3d80 msgr2=0x7f8d10162090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:44.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.309+0000 7f8d0e800640 1 --2- 172.21.15.152:0/3973898286 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8d100b3d80 0x7f8d10162090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:44.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.309+0000 7f8d0e800640 1 -- 172.21.15.152:0/3973898286 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d101fd440 con 0x7f8d101643b0 2024-04-20T11:31:44.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.309+0000 7f8d0e800640 1 --2- 172.21.15.152:0/3973898286 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8d101643b0 0x7f8d101507a0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f8d0802b9c0 tx=0x7f8d080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:44.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.310+0000 7f8d06000640 1 -- 172.21.15.152:0/3973898286 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d08034040 con 0x7f8d101643b0 2024-04-20T11:31:44.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.310+0000 7f8d06000640 1 -- 172.21.15.152:0/3973898286 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d080443f0 con 0x7f8d101643b0 2024-04-20T11:31:44.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.310+0000 7f8d06000640 1 -- 172.21.15.152:0/3973898286 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d080447e0 con 0x7f8d101643b0 2024-04-20T11:31:44.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.310+0000 7f8d14e00640 1 -- 172.21.15.152:0/3973898286 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d101fd160 con 0x7f8d101643b0 2024-04-20T11:31:44.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.310+0000 7f8d14e00640 1 -- 172.21.15.152:0/3973898286 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d10151280 con 0x7f8d101643b0 2024-04-20T11:31:44.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.311+0000 7f8d14e00640 1 -- 172.21.15.152:0/3973898286 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d100689f0 con 0x7f8d101643b0 2024-04-20T11:31:44.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.312+0000 7f8d06000640 1 -- 172.21.15.152:0/3973898286 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 11) ==== 91346+0+0 (secure 0 0 0) 0x7f8d08037070 con 0x7f8d101643b0 2024-04-20T11:31:44.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.313+0000 7f8d06000640 1 --2- 172.21.15.152:0/3973898286 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f8cec06cc50 0x7f8cec06f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:44.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.313+0000 7f8d06000640 1 -- 172.21.15.152:0/3973898286 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 4943+0+0 (secure 0 0 0) 0x7f8d080ae170 con 0x7f8d101643b0 2024-04-20T11:31:44.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.313+0000 7f8d0de00640 1 --2- 172.21.15.152:0/3973898286 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f8cec06cc50 0x7f8cec06f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:44.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.314+0000 7f8d0de00640 1 --2- 172.21.15.152:0/3973898286 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f8cec06cc50 0x7f8cec06f110 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f8d00001d40 tx=0x7f8d00009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:44.317 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.318+0000 7f8d06000640 1 -- 172.21.15.152:0/3973898286 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f8d0803f070 con 0x7f8d101643b0 2024-04-20T11:31:44.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.556+0000 7f8d14e00640 1 -- 172.21.15.152:0/3973898286 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8d10069100 con 0x7f8d101643b0 2024-04-20T11:31:44.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.558+0000 7f8d06000640 1 -- 172.21.15.152:0/3973898286 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+3878 (secure 0 0 0) 0x7f8d0807c380 con 0x7f8d101643b0 2024-04-20T11:31:44.557 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:31:44.557 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":36,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5045,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/3268046291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3268046291},{"type":"v1","addr":"172.21.15.175:6835","nonce":3268046291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5347,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/845526345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":845526345},{"type":"v1","addr":"172.21.15.175:6833","nonce":845526345}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":6288,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6836/3582785355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":3582785355},{"type":"v1","addr":"172.21.15.152:6836","nonce":3582785355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":6702,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/1842160674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6835","nonce":1842160674},{"type":"v1","addr":"172.21.15.152:6837","nonce":1842160674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36}],"filesystems":[]} 2024-04-20T11:31:44.558 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 36 2024-04-20T11:31:44.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.560+0000 7f8d14e00640 1 -- 172.21.15.152:0/3973898286 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f8cec06cc50 msgr2=0x7f8cec06f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:44.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.560+0000 7f8d14e00640 1 --2- 172.21.15.152:0/3973898286 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f8cec06cc50 0x7f8cec06f110 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f8d00001d40 tx=0x7f8d00009040 comp rx=0 tx=0).stop 2024-04-20T11:31:44.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.560+0000 7f8d14e00640 1 -- 172.21.15.152:0/3973898286 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8d101643b0 msgr2=0x7f8d101507a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:44.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.560+0000 7f8d14e00640 1 --2- 172.21.15.152:0/3973898286 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8d101643b0 0x7f8d101507a0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f8d0802b9c0 tx=0x7f8d080046c0 comp rx=0 tx=0).stop 2024-04-20T11:31:44.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.560+0000 7f8d14e00640 1 -- 172.21.15.152:0/3973898286 shutdown_connections 2024-04-20T11:31:44.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.560+0000 7f8d14e00640 1 --2- 172.21.15.152:0/3973898286 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f8cec06cc50 0x7f8cec06f110 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:44.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.560+0000 7f8d14e00640 1 --2- 172.21.15.152:0/3973898286 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8d101643b0 0x7f8d101507a0 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:44.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.560+0000 7f8d14e00640 1 --2- 172.21.15.152:0/3973898286 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8d10160870 0x7f8d10150260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:44.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.560+0000 7f8d14e00640 1 --2- 172.21.15.152:0/3973898286 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8d100b3d80 0x7f8d10162090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:44.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.560+0000 7f8d14e00640 1 -- 172.21.15.152:0/3973898286 >> 172.21.15.152:0/3973898286 conn(0x7f8d10067cf0 msgr2=0x7f8d10165d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:44.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.560+0000 7f8d14e00640 1 -- 172.21.15.152:0/3973898286 shutdown_connections 2024-04-20T11:31:44.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.560+0000 7f8d14e00640 1 -- 172.21.15.152:0/3973898286 wait complete. 2024-04-20T11:31:44.576 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-20T11:31:44.576 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-20T11:31:44.577 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-20T11:31:44.577 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T11:31:44.577 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-20T11:31:44.577 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-20T11:31:44.577 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-20T11:31:44.577 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T11:31:44.577 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-20T11:31:44.577 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-20T11:31:44.578 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-20T11:31:44.578 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T11:31:44.578 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-20T11:31:44.578 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-20T11:31:44.578 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-20T11:31:44.578 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T11:31:44.596 INFO:tasks.ceph.mds.c:Stopped 2024-04-20T11:31:44.596 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-20T11:31:44.598 INFO:tasks.ceph.mds.a:Stopped 2024-04-20T11:31:44.598 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-20T11:31:44.600 INFO:tasks.ceph.mds.d:Stopped 2024-04-20T11:31:44.600 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-20T11:31:44.603 INFO:tasks.ceph.mds.b:Stopped 2024-04-20T11:31:44.603 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-20T11:31:44.644 INFO:tasks.ceph.mds.c:Started 2024-04-20T11:31:44.645 DEBUG:teuthology.parallel:result is None 2024-04-20T11:31:44.647 INFO:tasks.ceph.mds.d:Started 2024-04-20T11:31:44.648 DEBUG:teuthology.parallel:result is None 2024-04-20T11:31:44.650 INFO:tasks.ceph.mds.a:Started 2024-04-20T11:31:44.650 DEBUG:teuthology.parallel:result is None 2024-04-20T11:31:44.651 INFO:tasks.ceph.mds.b:Started 2024-04-20T11:31:44.651 DEBUG:teuthology.parallel:result is None 2024-04-20T11:31:44.652 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-20T11:31:44.787 INFO:tasks.ceph.mds.c.smithi152.stdout:starting mds.c at 2024-04-20T11:31:44.788 INFO:tasks.ceph.mds.a.smithi152.stdout:starting mds.a at 2024-04-20T11:31:44.800 INFO:tasks.ceph.mds.d.smithi175.stdout:starting mds.d at 2024-04-20T11:31:44.800 INFO:tasks.ceph.mds.b.smithi175.stdout:starting mds.b at 2024-04-20T11:31:44.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.826+0000 7fefdae00640 1 Processor -- start 2024-04-20T11:31:44.826 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.826+0000 7fefdae00640 1 -- start start 2024-04-20T11:31:44.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.827+0000 7fefdae00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fefd40ccf30 0x7fefd40705f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:44.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.827+0000 7fefdae00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fefd406c390 0x7fefd40ce4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:44.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.827+0000 7fefdae00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fefd40cd310 con 0x7fefd40ce9f0 2024-04-20T11:31:44.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.827+0000 7fefdae00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fefd40ce020 con 0x7fefd406c390 2024-04-20T11:31:44.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.827+0000 7fefdae00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fefd4070b30 con 0x7fefd40ccf30 2024-04-20T11:31:44.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.828+0000 7fefd3400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fefd406c390 0x7fefd40ce4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:44.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.828+0000 7fefd8e00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fefd40ce9f0 0x7fefd40cbab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:41384/0 (socket says 172.21.15.152:41384) 2024-04-20T11:31:44.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.828+0000 7fefd8e00640 1 -- 172.21.15.152:0/3796335088 learned_addr learned my addr 172.21.15.152:0/3796335088 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:31:44.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.828+0000 7fefd3e00640 1 --2- 172.21.15.152:0/3796335088 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fefd40ccf30 0x7fefd40705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:44.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.828+0000 7fefd3400640 1 -- 172.21.15.152:0/3796335088 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fefd40ccf30 msgr2=0x7fefd40705f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:44.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.828+0000 7fefd3400640 1 --2- 172.21.15.152:0/3796335088 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fefd40ccf30 0x7fefd40705f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:44.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.828+0000 7fefd3400640 1 -- 172.21.15.152:0/3796335088 >> v1:172.21.15.152:6789/0 conn(0x7fefd40ce9f0 legacy=0x7fefd40cbab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:44.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.828+0000 7fefd3400640 1 -- 172.21.15.152:0/3796335088 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefd4070e00 con 0x7fefd406c390 2024-04-20T11:31:44.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.829+0000 7fefd3400640 1 --2- 172.21.15.152:0/3796335088 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fefd406c390 0x7fefd40ce4b0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fefc00097a0 tx=0x7fefc002b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc339129ea33d617 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:44.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.829+0000 7fefd2a00640 1 -- 172.21.15.152:0/3796335088 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefc0031780 con 0x7fefd406c390 2024-04-20T11:31:44.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.829+0000 7fefd2a00640 1 -- 172.21.15.152:0/3796335088 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefc0037070 con 0x7fefd406c390 2024-04-20T11:31:44.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.829+0000 7fefd2a00640 1 -- 172.21.15.152:0/3796335088 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefc0031b40 con 0x7fefd406c390 2024-04-20T11:31:44.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.830+0000 7fefdae00640 1 -- 172.21.15.152:0/3796335088 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fefd406c390 msgr2=0x7fefd40ce4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:44.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.830+0000 7fefdae00640 1 --2- 172.21.15.152:0/3796335088 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fefd406c390 0x7fefd40ce4b0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fefc00097a0 tx=0x7fefc002b820 comp rx=0 tx=0).stop 2024-04-20T11:31:44.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.830+0000 7fefdae00640 1 -- 172.21.15.152:0/3796335088 shutdown_connections 2024-04-20T11:31:44.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.830+0000 7fefdae00640 1 --2- 172.21.15.152:0/3796335088 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fefd406c390 0x7fefd40ce4b0 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:44.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.830+0000 7fefdae00640 1 --2- 172.21.15.152:0/3796335088 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fefd40ccf30 0x7fefd40705f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:44.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.830+0000 7fefdae00640 1 -- 172.21.15.152:0/3796335088 >> 172.21.15.152:0/3796335088 conn(0x7fefd40b3030 msgr2=0x7fefd40b3440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:44.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.830+0000 7fefdae00640 1 -- 172.21.15.152:0/3796335088 shutdown_connections 2024-04-20T11:31:44.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.830+0000 7fefdae00640 1 -- 172.21.15.152:0/3796335088 wait complete. 2024-04-20T11:31:44.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.832+0000 7fefdae00640 1 Processor -- start 2024-04-20T11:31:44.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.832+0000 7fefdae00640 1 -- start start 2024-04-20T11:31:44.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.833+0000 7fefdae00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fefd406c390 0x7fefd40caa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:44.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.833+0000 7fefdae00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fefd40ccf30 0x7fefd40cafa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:44.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.833+0000 7fefdae00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fefd40ce9f0 0x7fefd41f7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:44.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.833+0000 7fefdae00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fefd40cdcb0 con 0x7fefd40ce9f0 2024-04-20T11:31:44.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.833+0000 7fefdae00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fefd40cd9d0 con 0x7fefd40ccf30 2024-04-20T11:31:44.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.833+0000 7fefdae00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fefd40cc1c0 con 0x7fefd406c390 2024-04-20T11:31:44.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.833+0000 7fefd8e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fefd40ce9f0 0x7fefd41f7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:44.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.833+0000 7fefd8e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fefd40ce9f0 0x7fefd41f7bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49722/0 (socket says 172.21.15.152:49722) 2024-04-20T11:31:44.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.833+0000 7fefd8e00640 1 -- 172.21.15.152:0/4283091693 learned_addr learned my addr 172.21.15.152:0/4283091693 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:44.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.833+0000 7fefd3e00640 1 --2- 172.21.15.152:0/4283091693 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fefd406c390 0x7fefd40caa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:44.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.833+0000 7fefd3400640 1 --2- 172.21.15.152:0/4283091693 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fefd40ccf30 0x7fefd40cafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:44.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.834+0000 7fefd8e00640 1 -- 172.21.15.152:0/4283091693 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fefd406c390 msgr2=0x7fefd40caa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:44.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.834+0000 7fefd8e00640 1 --2- 172.21.15.152:0/4283091693 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fefd406c390 0x7fefd40caa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:44.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.834+0000 7fefd8e00640 1 -- 172.21.15.152:0/4283091693 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fefd40ccf30 msgr2=0x7fefd40cafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:44.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.834+0000 7fefd8e00640 1 --2- 172.21.15.152:0/4283091693 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fefd40ccf30 0x7fefd40cafa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:44.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.834+0000 7fefd8e00640 1 -- 172.21.15.152:0/4283091693 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefd40c9220 con 0x7fefd40ce9f0 2024-04-20T11:31:44.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.834+0000 7fefd3400640 1 --2- 172.21.15.152:0/4283091693 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fefd40ccf30 0x7fefd40cafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:31:44.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.834+0000 7fefd8e00640 1 --2- 172.21.15.152:0/4283091693 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fefd40ce9f0 0x7fefd41f7bd0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fefcc00ca80 tx=0x7fefcc00cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:44.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.834+0000 7fefcbe00640 1 -- 172.21.15.152:0/4283091693 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefcc017070 con 0x7fefd40ce9f0 2024-04-20T11:31:44.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.834+0000 7fefcbe00640 1 -- 172.21.15.152:0/4283091693 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefcc00ac30 con 0x7fefd40ce9f0 2024-04-20T11:31:44.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.834+0000 7fefcbe00640 1 -- 172.21.15.152:0/4283091693 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefcc012600 con 0x7fefd40ce9f0 2024-04-20T11:31:44.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.834+0000 7fefdae00640 1 -- 172.21.15.152:0/4283091693 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefd40c9500 con 0x7fefd40ce9f0 2024-04-20T11:31:44.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.834+0000 7fefdae00640 1 -- 172.21.15.152:0/4283091693 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefd415b5c0 con 0x7fefd40ce9f0 2024-04-20T11:31:44.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.836+0000 7fefdae00640 1 -- 172.21.15.152:0/4283091693 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefd4069100 con 0x7fefd40ce9f0 2024-04-20T11:31:44.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.837+0000 7fefcbe00640 1 -- 172.21.15.152:0/4283091693 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 11) ==== 91346+0+0 (secure 0 0 0) 0x7fefcc01a070 con 0x7fefd40ce9f0 2024-04-20T11:31:44.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.838+0000 7fefcbe00640 1 --2- 172.21.15.152:0/4283091693 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fefb406cc50 0x7fefb406f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:44.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.838+0000 7fefcbe00640 1 -- 172.21.15.152:0/4283091693 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 4943+0+0 (secure 0 0 0) 0x7fefcc08bac0 con 0x7fefd40ce9f0 2024-04-20T11:31:44.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.838+0000 7fefd3e00640 1 --2- 172.21.15.152:0/4283091693 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fefb406cc50 0x7fefb406f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:44.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.839+0000 7fefd3e00640 1 --2- 172.21.15.152:0/4283091693 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fefb406cc50 0x7fefb406f110 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fefc4009b40 tx=0x7fefc4009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:44.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:44.842+0000 7fefcbe00640 1 -- 172.21.15.152:0/4283091693 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fefcc00ae20 con 0x7fefd40ce9f0 2024-04-20T11:31:45.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.057+0000 7fefdae00640 1 -- 172.21.15.152:0/4283091693 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fefd40cbd70 con 0x7fefd40ce9f0 2024-04-20T11:31:45.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.233+0000 7fefcbe00640 1 -- 172.21.15.152:0/4283091693 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v65) ==== 97+0+0 (secure 0 0 0) 0x7fefcc0596e0 con 0x7fefd40ce9f0 2024-04-20T11:31:45.233 INFO:teuthology.orchestra.run.smithi152.stderr: removed all blocklist entries 2024-04-20T11:31:45.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.236+0000 7fefdae00640 1 -- 172.21.15.152:0/4283091693 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fefb406cc50 msgr2=0x7fefb406f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:45.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.236+0000 7fefdae00640 1 --2- 172.21.15.152:0/4283091693 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fefb406cc50 0x7fefb406f110 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fefc4009b40 tx=0x7fefc4009190 comp rx=0 tx=0).stop 2024-04-20T11:31:45.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.236+0000 7fefdae00640 1 -- 172.21.15.152:0/4283091693 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fefd40ce9f0 msgr2=0x7fefd41f7bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:45.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.236+0000 7fefdae00640 1 --2- 172.21.15.152:0/4283091693 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fefd40ce9f0 0x7fefd41f7bd0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fefcc00ca80 tx=0x7fefcc00cf40 comp rx=0 tx=0).stop 2024-04-20T11:31:45.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.236+0000 7fefdae00640 1 -- 172.21.15.152:0/4283091693 shutdown_connections 2024-04-20T11:31:45.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.236+0000 7fefdae00640 1 --2- 172.21.15.152:0/4283091693 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fefb406cc50 0x7fefb406f110 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:45.237 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.236+0000 7fefdae00640 1 --2- 172.21.15.152:0/4283091693 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fefd40ce9f0 0x7fefd41f7bd0 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:45.237 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.236+0000 7fefdae00640 1 --2- 172.21.15.152:0/4283091693 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fefd40ccf30 0x7fefd40cafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:45.237 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.236+0000 7fefdae00640 1 --2- 172.21.15.152:0/4283091693 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fefd406c390 0x7fefd40caa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:45.237 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.236+0000 7fefdae00640 1 -- 172.21.15.152:0/4283091693 >> 172.21.15.152:0/4283091693 conn(0x7fefd40b3030 msgr2=0x7fefd406bb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:45.237 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.237+0000 7fefdae00640 1 -- 172.21.15.152:0/4283091693 shutdown_connections 2024-04-20T11:31:45.237 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.237+0000 7fefdae00640 1 -- 172.21.15.152:0/4283091693 wait complete. 2024-04-20T11:31:45.259 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-20T11:31:45.449 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.449+0000 7f26f7e00640 1 Processor -- start 2024-04-20T11:31:45.449 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.449+0000 7f26f7e00640 1 -- start start 2024-04-20T11:31:45.450 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.450+0000 7f26f7e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f26f8142b60 0x7f26f8160c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:45.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.450+0000 7f26f7e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f26f806dcf0 0x7f26f806e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:45.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.450+0000 7f26f7e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f26f8071cd0 0x7f26f8072140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:45.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.450+0000 7f26f7e00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f26f8072680 con 0x7f26f8142b60 2024-04-20T11:31:45.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.450+0000 7f26f7e00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f26f81611a0 con 0x7f26f806dcf0 2024-04-20T11:31:45.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.450+0000 7f26f7e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f26f806e7b0 con 0x7f26f8071cd0 2024-04-20T11:31:45.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.451+0000 7f26f6a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f26f8142b60 0x7f26f8160c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:45.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.451+0000 7f26f6a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f26f8142b60 0x7f26f8160c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49730/0 (socket says 172.21.15.152:49730) 2024-04-20T11:31:45.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.451+0000 7f26f6a00640 1 -- 172.21.15.152:0/1413692785 learned_addr learned my addr 172.21.15.152:0/1413692785 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:45.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.451+0000 7f26f6a00640 1 -- 172.21.15.152:0/1413692785 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f26f8071cd0 msgr2=0x7f26f8072140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:45.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.451+0000 7f26f6a00640 1 --2- 172.21.15.152:0/1413692785 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f26f8071cd0 0x7f26f8072140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:45.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.451+0000 7f26f6a00640 1 -- 172.21.15.152:0/1413692785 >> v2:172.21.15.175:3300/0 conn(0x7f26f806dcf0 msgr2=0x7f26f806e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:45.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.451+0000 7f26f6a00640 1 --2- 172.21.15.152:0/1413692785 >> v2:172.21.15.175:3300/0 conn(0x7f26f806dcf0 0x7f26f806e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:45.452 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.451+0000 7f26f6a00640 1 -- 172.21.15.152:0/1413692785 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26f806ea90 con 0x7f26f8142b60 2024-04-20T11:31:45.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.452+0000 7f26f6a00640 1 --2- 172.21.15.152:0/1413692785 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f26f8142b60 0x7f26f8160c60 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f26f80b59c0 tx=0x7f26e802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a9e26f03fa4c063 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:45.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.452+0000 7f26f5600640 1 -- 172.21.15.152:0/1413692785 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e8035020 con 0x7f26f8142b60 2024-04-20T11:31:45.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.452+0000 7f26f5600640 1 -- 172.21.15.152:0/1413692785 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26e8031870 con 0x7f26f8142b60 2024-04-20T11:31:45.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.453+0000 7f26f5600640 1 -- 172.21.15.152:0/1413692785 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e8031c40 con 0x7f26f8142b60 2024-04-20T11:31:45.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.453+0000 7f26f7e00640 1 -- 172.21.15.152:0/1413692785 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f26f8142b60 msgr2=0x7f26f8160c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:45.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.453+0000 7f26f7e00640 1 --2- 172.21.15.152:0/1413692785 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f26f8142b60 0x7f26f8160c60 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f26f80b59c0 tx=0x7f26e802b910 comp rx=0 tx=0).stop 2024-04-20T11:31:45.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.453+0000 7f26f7e00640 1 -- 172.21.15.152:0/1413692785 shutdown_connections 2024-04-20T11:31:45.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.453+0000 7f26f7e00640 1 --2- 172.21.15.152:0/1413692785 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f26f8071cd0 0x7f26f8072140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:45.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.453+0000 7f26f7e00640 1 --2- 172.21.15.152:0/1413692785 >> v2:172.21.15.175:3300/0 conn(0x7f26f806dcf0 0x7f26f806e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:45.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.453+0000 7f26f7e00640 1 --2- 172.21.15.152:0/1413692785 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f26f8142b60 0x7f26f8160c60 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:45.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.453+0000 7f26f7e00640 1 -- 172.21.15.152:0/1413692785 >> 172.21.15.152:0/1413692785 conn(0x7f26f8067ae0 msgr2=0x7f26f806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:45.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.453+0000 7f26f7e00640 1 -- 172.21.15.152:0/1413692785 shutdown_connections 2024-04-20T11:31:45.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.453+0000 7f26f7e00640 1 -- 172.21.15.152:0/1413692785 wait complete. 2024-04-20T11:31:45.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.455+0000 7f26f7e00640 1 Processor -- start 2024-04-20T11:31:45.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.455+0000 7f26f7e00640 1 -- start start 2024-04-20T11:31:45.456 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.456+0000 7f26f7e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26f806dcf0 0x7f26f813f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:45.457 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.456+0000 7f26f7e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f26f8071cd0 0x7f26f81445c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:45.457 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.456+0000 7f26f7e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f26f8142b60 0x7f26f81e9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:45.457 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.456+0000 7f26f7e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f26f806f460 con 0x7f26f8142b60 2024-04-20T11:31:45.457 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.456+0000 7f26f7e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f26f8160740 con 0x7f26f806dcf0 2024-04-20T11:31:45.457 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.456+0000 7f26f7e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f26f8142f40 con 0x7f26f8071cd0 2024-04-20T11:31:45.457 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.456+0000 7f26f7400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f26f8142b60 0x7f26f81e9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:45.457 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.456+0000 7f26f7400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f26f8142b60 0x7f26f81e9ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49738/0 (socket says 172.21.15.152:49738) 2024-04-20T11:31:45.457 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.456+0000 7f26f7400640 1 -- 172.21.15.152:0/3043762311 learned_addr learned my addr 172.21.15.152:0/3043762311 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:45.457 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.457+0000 7f26f6a00640 1 --2- 172.21.15.152:0/3043762311 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26f806dcf0 0x7f26f813f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:45.457 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.457+0000 7f26f7400640 1 -- 172.21.15.152:0/3043762311 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f26f8071cd0 msgr2=0x7f26f81445c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:45.457 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.457+0000 7f26f6000640 1 --2- 172.21.15.152:0/3043762311 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f26f8071cd0 0x7f26f81445c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:45.457 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.457+0000 7f26f7400640 1 --2- 172.21.15.152:0/3043762311 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f26f8071cd0 0x7f26f81445c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:45.457 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.457+0000 7f26f7400640 1 -- 172.21.15.152:0/3043762311 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26f806dcf0 msgr2=0x7f26f813f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:45.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.457+0000 7f26f7400640 1 --2- 172.21.15.152:0/3043762311 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26f806dcf0 0x7f26f813f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:45.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.457+0000 7f26f7400640 1 -- 172.21.15.152:0/3043762311 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26f8143520 con 0x7f26f8142b60 2024-04-20T11:31:45.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.457+0000 7f26f6a00640 1 --2- 172.21.15.152:0/3043762311 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26f806dcf0 0x7f26f813f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:31:45.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.457+0000 7f26f7400640 1 --2- 172.21.15.152:0/3043762311 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f26f8142b60 0x7f26f81e9ad0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f26f000b9a0 tx=0x7f26f000be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:45.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.457+0000 7f26eea00640 1 -- 172.21.15.152:0/3043762311 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26f0019070 con 0x7f26f8142b60 2024-04-20T11:31:45.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.457+0000 7f26eea00640 1 -- 172.21.15.152:0/3043762311 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26f000cbc0 con 0x7f26f8142b60 2024-04-20T11:31:45.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.457+0000 7f26eea00640 1 -- 172.21.15.152:0/3043762311 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26f0014440 con 0x7f26f8142b60 2024-04-20T11:31:45.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.457+0000 7f26f7e00640 1 -- 172.21.15.152:0/3043762311 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26f8143800 con 0x7f26f8142b60 2024-04-20T11:31:45.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.458+0000 7f26f7e00640 1 -- 172.21.15.152:0/3043762311 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26f8143dc0 con 0x7f26f8142b60 2024-04-20T11:31:45.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.458+0000 7f26f7e00640 1 -- 172.21.15.152:0/3043762311 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26f8075e60 con 0x7f26f8142b60 2024-04-20T11:31:45.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.459+0000 7f26eea00640 1 -- 172.21.15.152:0/3043762311 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 11) ==== 91346+0+0 (secure 0 0 0) 0x7f26f001c070 con 0x7f26f8142b60 2024-04-20T11:31:45.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.460+0000 7f26eea00640 1 --2- 172.21.15.152:0/3043762311 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f26d806cc50 0x7f26d806f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:45.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.460+0000 7f26eea00640 1 -- 172.21.15.152:0/3043762311 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 4857+0+0 (secure 0 0 0) 0x7f26f008bad0 con 0x7f26f8142b60 2024-04-20T11:31:45.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.461+0000 7f26f6a00640 1 --2- 172.21.15.152:0/3043762311 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f26d806cc50 0x7f26d806f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:45.461 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.461+0000 7f26f6a00640 1 --2- 172.21.15.152:0/3043762311 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f26d806cc50 0x7f26d806f110 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f26e8036000 tx=0x7f26e8008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:45.464 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.465+0000 7f26eea00640 1 -- 172.21.15.152:0/3043762311 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f26f0059e40 con 0x7f26f8142b60 2024-04-20T11:31:45.749 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.750+0000 7f26f7e00640 1 -- 172.21.15.152:0/3043762311 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f26f815c770 con 0x7f26f8142b60 2024-04-20T11:31:45.750 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.751+0000 7f26eea00640 1 -- 172.21.15.152:0/3043762311 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v12) ==== 80+0+5617 (secure 0 0 0) 0x7f26f0059740 con 0x7f26f8142b60 2024-04-20T11:31:45.750 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:31:45.750 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-04-20T11:31:45.751 INFO:teuthology.orchestra.run.smithi152.stdout: "auth_dump": [ 2024-04-20T11:31:45.751 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:31:45.751 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "mds.a", 2024-04-20T11:31:45.751 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBCpiNmyFRJIBAAj2aIMXdb65nqRgKFDBLLzA==", 2024-04-20T11:31:45.751 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:31:45.751 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow", 2024-04-20T11:31:45.751 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow *", 2024-04-20T11:31:45.751 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow *", 2024-04-20T11:31:45.751 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:31:45.751 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:31:45.751 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:31:45.751 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:31:45.751 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "mds.b", 2024-04-20T11:31:45.752 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBCpiNmVPytOhAAzQkvNOJaNy+MeKETWhYLyg==", 2024-04-20T11:31:45.752 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:31:45.752 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow", 2024-04-20T11:31:45.752 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow *", 2024-04-20T11:31:45.752 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow *", 2024-04-20T11:31:45.752 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:31:45.752 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:31:45.752 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:31:45.752 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:31:45.752 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "mds.c", 2024-04-20T11:31:45.752 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBCpiNm5ViKLRAAg7VBQG4hFTLYUpJMpBAaVA==", 2024-04-20T11:31:45.752 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:31:45.752 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow", 2024-04-20T11:31:45.752 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow *", 2024-04-20T11:31:45.753 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow *", 2024-04-20T11:31:45.753 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:31:45.753 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:31:45.753 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:31:45.753 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:31:45.753 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "mds.d", 2024-04-20T11:31:45.753 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBDpiNmLWhaCxAA2e4qd/7+TKenJlWICIsvGg==", 2024-04-20T11:31:45.753 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:31:45.753 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow", 2024-04-20T11:31:45.753 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow *", 2024-04-20T11:31:45.753 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow *", 2024-04-20T11:31:45.753 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:31:45.753 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:31:45.754 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:31:45.754 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:31:45.754 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "osd.0", 2024-04-20T11:31:45.754 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBHpiNmpuhyDBAAsQXX9DNvWlo0UYh53lb+FA==", 2024-04-20T11:31:45.754 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:31:45.754 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow profile osd", 2024-04-20T11:31:45.754 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile osd", 2024-04-20T11:31:45.754 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:31:45.754 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:31:45.754 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:31:45.754 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:31:45.754 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "osd.1", 2024-04-20T11:31:45.754 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBKpiNmxBiLJhAAHon1adC8G2jbYoDcWdE9jA==", 2024-04-20T11:31:45.755 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:31:45.755 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow profile osd", 2024-04-20T11:31:45.755 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile osd", 2024-04-20T11:31:45.755 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:31:45.755 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:31:45.755 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:31:45.755 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:31:45.755 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "osd.2", 2024-04-20T11:31:45.755 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBNpiNmdn0zMhAAqwIO2kKahdSzTVh8/p3FCg==", 2024-04-20T11:31:45.755 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:31:45.755 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow profile osd", 2024-04-20T11:31:45.755 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile osd", 2024-04-20T11:31:45.755 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:31:45.756 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:31:45.756 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:31:45.756 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:31:45.756 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "osd.3", 2024-04-20T11:31:45.756 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBRpiNmTPCYAhAAHFNHX/xKn21jEFHDkHMjEw==", 2024-04-20T11:31:45.756 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:31:45.756 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow profile osd", 2024-04-20T11:31:45.756 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile osd", 2024-04-20T11:31:45.756 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:31:45.756 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:31:45.756 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:31:45.756 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:31:45.756 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "osd.4", 2024-04-20T11:31:45.757 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBXpiNmM8cVFxAAJl9Hu+MvlkEiMglutI89eg==", 2024-04-20T11:31:45.757 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:31:45.757 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow profile osd", 2024-04-20T11:31:45.757 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile osd", 2024-04-20T11:31:45.757 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:31:45.757 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:31:45.757 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:31:45.757 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:31:45.757 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "osd.5", 2024-04-20T11:31:45.757 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBapiNmnGxJMRAA90P0T+VsTA8LCTcexTVr7w==", 2024-04-20T11:31:45.757 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:31:45.757 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow profile osd", 2024-04-20T11:31:45.757 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile osd", 2024-04-20T11:31:45.757 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:31:45.758 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:31:45.758 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:31:45.758 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:31:45.758 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "osd.6", 2024-04-20T11:31:45.758 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBepiNmyLXWDxAAhtHD5h478mLovE5IejAFng==", 2024-04-20T11:31:45.758 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:31:45.758 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow profile osd", 2024-04-20T11:31:45.758 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile osd", 2024-04-20T11:31:45.758 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:31:45.758 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:31:45.758 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:31:45.758 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:31:45.758 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "osd.7", 2024-04-20T11:31:45.758 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBhpiNmB3jwHBAAqco9om6r2N/GqAV6rVKiaA==", 2024-04-20T11:31:45.759 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:31:45.759 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow profile osd", 2024-04-20T11:31:45.759 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile osd", 2024-04-20T11:31:45.759 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:31:45.759 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:31:45.759 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:31:45.759 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:31:45.759 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "client.0", 2024-04-20T11:31:45.759 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBDpiNmtsBWFhAArq9A7r33CcwgMmOU2pAGPw==", 2024-04-20T11:31:45.759 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:31:45.759 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow", 2024-04-20T11:31:45.759 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow r", 2024-04-20T11:31:45.759 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-20T11:31:45.759 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:31:45.760 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:31:45.760 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:31:45.760 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "client.1", 2024-04-20T11:31:45.760 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBDpiNm6aNnJxAAB431GSGytZ0IKWMh4DMalw==", 2024-04-20T11:31:45.760 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:31:45.760 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow", 2024-04-20T11:31:45.760 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow r", 2024-04-20T11:31:45.760 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow rw", 2024-04-20T11:31:45.760 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow rwx" 2024-04-20T11:31:45.760 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:31:45.760 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:31:45.760 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:31:45.760 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "client.2", 2024-04-20T11:31:45.761 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBDpiNm5QLtHRAAa2kHTDVdgSh/B4XGZaKa5Q==", 2024-04-20T11:31:45.761 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:31:45.761 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow", 2024-04-20T11:31:45.761 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow r", 2024-04-20T11:31:45.761 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow rw", 2024-04-20T11:31:45.761 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow rwx" 2024-04-20T11:31:45.761 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:31:45.761 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:31:45.761 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:31:45.761 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "client.3", 2024-04-20T11:31:45.761 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBDpiNmw95SMBAAdodzTuPp4WdKSEf+jI8Uuw==", 2024-04-20T11:31:45.761 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:31:45.761 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow", 2024-04-20T11:31:45.761 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow r", 2024-04-20T11:31:45.762 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow rw", 2024-04-20T11:31:45.762 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow rwx" 2024-04-20T11:31:45.762 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:31:45.762 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:31:45.762 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:31:45.762 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "client.admin", 2024-04-20T11:31:45.762 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBBpiNm3xz6HhAA82+N+trDW/AmBr3q8l4ECw==", 2024-04-20T11:31:45.762 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:31:45.762 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow *", 2024-04-20T11:31:45.762 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow *", 2024-04-20T11:31:45.762 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow *", 2024-04-20T11:31:45.762 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:31:45.762 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:31:45.762 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:31:45.763 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:31:45.763 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "client.alice", 2024-04-20T11:31:45.763 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQCMpyNmP7+RMxAAw/JhZWU6U/0MpqThf8E3TA==", 2024-04-20T11:31:45.763 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:31:45.763 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow rw path=/volumes/_nogroup/subvol99630", 2024-04-20T11:31:45.763 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow r", 2024-04-20T11:31:45.763 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow rw pool=cephfs_data" 2024-04-20T11:31:45.763 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:31:45.763 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:31:45.763 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:31:45.763 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "mgr.x", 2024-04-20T11:31:45.763 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBCpiNmuhalFxAA9dTl/zSbUovCCZAanAipww==", 2024-04-20T11:31:45.764 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:31:45.764 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow *", 2024-04-20T11:31:45.764 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile mgr", 2024-04-20T11:31:45.764 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:31:45.764 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:31:45.764 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:31:45.764 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:31:45.764 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "mgr.y", 2024-04-20T11:31:45.764 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBCpiNmP1+5DRAAs6bVAEOr0i6FdPmSEyo/vQ==", 2024-04-20T11:31:45.764 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:31:45.764 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow *", 2024-04-20T11:31:45.764 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile mgr", 2024-04-20T11:31:45.764 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:31:45.764 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:31:45.765 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:31:45.765 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-04-20T11:31:45.765 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-04-20T11:31:45.765 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.753+0000 7f26f7e00640 1 -- 172.21.15.152:0/3043762311 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f26d806cc50 msgr2=0x7f26d806f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:45.765 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.753+0000 7f26f7e00640 1 --2- 172.21.15.152:0/3043762311 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f26d806cc50 0x7f26d806f110 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f26e8036000 tx=0x7f26e8008030 comp rx=0 tx=0).stop 2024-04-20T11:31:45.766 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.753+0000 7f26f7e00640 1 -- 172.21.15.152:0/3043762311 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f26f8142b60 msgr2=0x7f26f81e9ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:45.766 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.753+0000 7f26f7e00640 1 --2- 172.21.15.152:0/3043762311 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f26f8142b60 0x7f26f81e9ad0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f26f000b9a0 tx=0x7f26f000be60 comp rx=0 tx=0).stop 2024-04-20T11:31:45.766 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.753+0000 7f26f7e00640 1 -- 172.21.15.152:0/3043762311 shutdown_connections 2024-04-20T11:31:45.766 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.754+0000 7f26f7e00640 1 --2- 172.21.15.152:0/3043762311 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f26d806cc50 0x7f26d806f110 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:45.766 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.754+0000 7f26f7e00640 1 --2- 172.21.15.152:0/3043762311 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f26f8142b60 0x7f26f81e9ad0 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:45.766 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.754+0000 7f26f7e00640 1 --2- 172.21.15.152:0/3043762311 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f26f8071cd0 0x7f26f81445c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:45.766 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.754+0000 7f26f7e00640 1 --2- 172.21.15.152:0/3043762311 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26f806dcf0 0x7f26f813f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:45.766 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.754+0000 7f26f7e00640 1 -- 172.21.15.152:0/3043762311 >> 172.21.15.152:0/3043762311 conn(0x7f26f8067ae0 msgr2=0x7f26f81409f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:45.766 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.754+0000 7f26f7e00640 1 -- 172.21.15.152:0/3043762311 shutdown_connections 2024-04-20T11:31:45.766 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.754+0000 7f26f7e00640 1 -- 172.21.15.152:0/3043762311 wait complete. 2024-04-20T11:31:45.772 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.alice 2024-04-20T11:31:45.970 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.971+0000 7f6df3e00640 1 Processor -- start 2024-04-20T11:31:45.971 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.971+0000 7f6df3e00640 1 -- start start 2024-04-20T11:31:45.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.973+0000 7f6df3e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f6df415c7f0 0x7f6df4072b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:45.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.973+0000 7f6df3e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6df4073b60 0x7f6df40ad2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:45.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.973+0000 7f6df3e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f6df40b4160 0x7f6df40b4590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:45.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.973+0000 7f6df3e00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f6df40ce140 con 0x7f6df40b4160 2024-04-20T11:31:45.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.973+0000 7f6df3e00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f6df40b4ad0 con 0x7f6df415c7f0 2024-04-20T11:31:45.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.973+0000 7f6df3e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6df4073fc0 con 0x7f6df4073b60 2024-04-20T11:31:45.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.973+0000 7f6df3400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f6df40b4160 0x7f6df40b4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:45.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.973+0000 7f6df3400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f6df40b4160 0x7f6df40b4590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36440/0 (socket says 172.21.15.152:36440) 2024-04-20T11:31:45.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.973+0000 7f6df3400640 1 -- 172.21.15.152:0/2023185635 learned_addr learned my addr 172.21.15.152:0/2023185635 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:45.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.974+0000 7f6debe00640 1 --2- 172.21.15.152:0/2023185635 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6df4073b60 0x7f6df40ad2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:45.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.974+0000 7f6df2a00640 1 --2- 172.21.15.152:0/2023185635 >> v2:172.21.15.175:3300/0 conn(0x7f6df415c7f0 0x7f6df4072b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:45.974 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.974+0000 7f6df3400640 1 -- 172.21.15.152:0/2023185635 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6df4073b60 msgr2=0x7f6df40ad2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:45.974 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.974+0000 7f6df3400640 1 --2- 172.21.15.152:0/2023185635 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6df4073b60 0x7f6df40ad2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:45.974 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.974+0000 7f6df3400640 1 -- 172.21.15.152:0/2023185635 >> v2:172.21.15.175:3300/0 conn(0x7f6df415c7f0 msgr2=0x7f6df4072b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:45.974 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.974+0000 7f6df3400640 1 --2- 172.21.15.152:0/2023185635 >> v2:172.21.15.175:3300/0 conn(0x7f6df415c7f0 0x7f6df4072b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:45.974 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.974+0000 7f6df3400640 1 -- 172.21.15.152:0/2023185635 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6df40ce390 con 0x7f6df40b4160 2024-04-20T11:31:45.975 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.975+0000 7f6df3400640 1 --2- 172.21.15.152:0/2023185635 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6df40b4160 0x7f6df40b4590 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f6df4070c60 tx=0x7f6dec02be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=776bd696b3fd0f64 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:45.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.975+0000 7f6df2000640 1 -- 172.21.15.152:0/2023185635 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dec02c9d0 con 0x7f6df40b4160 2024-04-20T11:31:45.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.975+0000 7f6df2000640 1 -- 172.21.15.152:0/2023185635 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6dec02cb70 con 0x7f6df40b4160 2024-04-20T11:31:45.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.975+0000 7f6df2000640 1 -- 172.21.15.152:0/2023185635 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dec0343f0 con 0x7f6df40b4160 2024-04-20T11:31:45.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.976+0000 7f6df3e00640 1 -- 172.21.15.152:0/2023185635 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6df40b4160 msgr2=0x7f6df40b4590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:45.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.976+0000 7f6df3e00640 1 --2- 172.21.15.152:0/2023185635 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6df40b4160 0x7f6df40b4590 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f6df4070c60 tx=0x7f6dec02be30 comp rx=0 tx=0).stop 2024-04-20T11:31:45.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.976+0000 7f6df3e00640 1 -- 172.21.15.152:0/2023185635 shutdown_connections 2024-04-20T11:31:45.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.976+0000 7f6df3e00640 1 --2- 172.21.15.152:0/2023185635 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6df40b4160 0x7f6df40b4590 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:45.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.976+0000 7f6df3e00640 1 --2- 172.21.15.152:0/2023185635 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6df4073b60 0x7f6df40ad2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:45.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.976+0000 7f6df3e00640 1 --2- 172.21.15.152:0/2023185635 >> v2:172.21.15.175:3300/0 conn(0x7f6df415c7f0 0x7f6df4072b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:45.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.976+0000 7f6df3e00640 1 -- 172.21.15.152:0/2023185635 >> 172.21.15.152:0/2023185635 conn(0x7f6df40d0260 msgr2=0x7f6df40d0670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:45.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.976+0000 7f6df3e00640 1 -- 172.21.15.152:0/2023185635 shutdown_connections 2024-04-20T11:31:45.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.976+0000 7f6df3e00640 1 -- 172.21.15.152:0/2023185635 wait complete. 2024-04-20T11:31:45.977 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.978+0000 7f6df3e00640 1 Processor -- start 2024-04-20T11:31:45.977 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.978+0000 7f6df3e00640 1 -- start start 2024-04-20T11:31:45.979 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.979+0000 7f6df3e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6df4073b60 0x7f6df40b07b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:45.979 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.979+0000 7f6df3e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6df40b4160 0x7f6df40b0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:45.979 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.979+0000 7f6df3e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6df4151ff0 0x7f6df41fc1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:45.979 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.979+0000 7f6df3e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f6df40cec30 con 0x7f6df4151ff0 2024-04-20T11:31:45.979 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.979+0000 7f6df3e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6df40ce950 con 0x7f6df40b4160 2024-04-20T11:31:45.979 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.979+0000 7f6df3e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6df40cef10 con 0x7f6df4073b60 2024-04-20T11:31:45.979 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.979+0000 7f6df3400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6df4151ff0 0x7f6df41fc1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:45.979 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.979+0000 7f6df3400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6df4151ff0 0x7f6df41fc1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36444/0 (socket says 172.21.15.152:36444) 2024-04-20T11:31:45.979 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.979+0000 7f6df3400640 1 -- 172.21.15.152:0/1439878618 learned_addr learned my addr 172.21.15.152:0/1439878618 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:45.980 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.980+0000 7f6df2a00640 1 --2- 172.21.15.152:0/1439878618 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6df4073b60 0x7f6df40b07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:45.980 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.980+0000 7f6debe00640 1 --2- 172.21.15.152:0/1439878618 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6df40b4160 0x7f6df40b0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:45.980 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.980+0000 7f6df3400640 1 -- 172.21.15.152:0/1439878618 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6df4073b60 msgr2=0x7f6df40b07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:45.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.980+0000 7f6df3400640 1 --2- 172.21.15.152:0/1439878618 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6df4073b60 0x7f6df40b07b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:45.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.980+0000 7f6df3400640 1 -- 172.21.15.152:0/1439878618 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6df40b4160 msgr2=0x7f6df40b0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:45.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.980+0000 7f6df3400640 1 --2- 172.21.15.152:0/1439878618 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6df40b4160 0x7f6df40b0cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:45.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.980+0000 7f6df3400640 1 -- 172.21.15.152:0/1439878618 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6df41524d0 con 0x7f6df4151ff0 2024-04-20T11:31:45.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.980+0000 7f6df3400640 1 --2- 172.21.15.152:0/1439878618 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6df4151ff0 0x7f6df41fc1b0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f6dec036040 tx=0x7f6dec004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:45.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.980+0000 7f6deaa00640 1 -- 172.21.15.152:0/1439878618 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dec043020 con 0x7f6df4151ff0 2024-04-20T11:31:45.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.980+0000 7f6deaa00640 1 -- 172.21.15.152:0/1439878618 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6dec03e450 con 0x7f6df4151ff0 2024-04-20T11:31:45.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.980+0000 7f6deaa00640 1 -- 172.21.15.152:0/1439878618 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dec009510 con 0x7f6df4151ff0 2024-04-20T11:31:45.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.980+0000 7f6df3e00640 1 -- 172.21.15.152:0/1439878618 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6df41527b0 con 0x7f6df4151ff0 2024-04-20T11:31:45.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.980+0000 7f6df3e00640 1 -- 172.21.15.152:0/1439878618 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6df4152d70 con 0x7f6df4151ff0 2024-04-20T11:31:45.982 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.981+0000 7f6df3e00640 1 -- 172.21.15.152:0/1439878618 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6df4075e60 con 0x7f6df4151ff0 2024-04-20T11:31:45.982 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.982+0000 7f6deaa00640 1 -- 172.21.15.152:0/1439878618 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 11) ==== 91346+0+0 (secure 0 0 0) 0x7f6dec046020 con 0x7f6df4151ff0 2024-04-20T11:31:45.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.983+0000 7f6deaa00640 1 --2- 172.21.15.152:0/1439878618 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f6dd006cc50 0x7f6dd006f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:45.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.983+0000 7f6deaa00640 1 -- 172.21.15.152:0/1439878618 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 4857+0+0 (secure 0 0 0) 0x7f6dec0ae650 con 0x7f6df4151ff0 2024-04-20T11:31:45.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.983+0000 7f6df2a00640 1 --2- 172.21.15.152:0/1439878618 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f6dd006cc50 0x7f6dd006f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:45.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.984+0000 7f6df2a00640 1 --2- 172.21.15.152:0/1439878618 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f6dd006cc50 0x7f6dd006f110 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f6de4006fd0 tx=0x7f6de4008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:45.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:45.987+0000 7f6deaa00640 1 -- 172.21.15.152:0/1439878618 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f6dec04d030 con 0x7f6df4151ff0 2024-04-20T11:31:46.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.239+0000 7f6df3e00640 1 -- 172.21.15.152:0/1439878618 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.alice"} v 0) -- 0x7f6df4153d80 con 0x7f6df4151ff0 2024-04-20T11:31:46.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.244+0000 7f6deaa00640 1 -- 172.21.15.152:0/1439878618 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.alice"}]=0 v13) ==== 82+0+0 (secure 0 0 0) 0x7f6dec07c8f0 con 0x7f6df4151ff0 2024-04-20T11:31:46.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.247+0000 7f6df3e00640 1 -- 172.21.15.152:0/1439878618 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f6dd006cc50 msgr2=0x7f6dd006f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:46.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.247+0000 7f6df3e00640 1 --2- 172.21.15.152:0/1439878618 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f6dd006cc50 0x7f6dd006f110 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f6de4006fd0 tx=0x7f6de4008040 comp rx=0 tx=0).stop 2024-04-20T11:31:46.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.247+0000 7f6df3e00640 1 -- 172.21.15.152:0/1439878618 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6df4151ff0 msgr2=0x7f6df41fc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:46.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.247+0000 7f6df3e00640 1 --2- 172.21.15.152:0/1439878618 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6df4151ff0 0x7f6df41fc1b0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f6dec036040 tx=0x7f6dec004a40 comp rx=0 tx=0).stop 2024-04-20T11:31:46.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.247+0000 7f6df3e00640 1 -- 172.21.15.152:0/1439878618 shutdown_connections 2024-04-20T11:31:46.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.247+0000 7f6df3e00640 1 --2- 172.21.15.152:0/1439878618 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f6dd006cc50 0x7f6dd006f110 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:46.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.247+0000 7f6df3e00640 1 --2- 172.21.15.152:0/1439878618 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6df4151ff0 0x7f6df41fc1b0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:46.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.247+0000 7f6df3e00640 1 --2- 172.21.15.152:0/1439878618 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6df40b4160 0x7f6df40b0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:46.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.247+0000 7f6df3e00640 1 --2- 172.21.15.152:0/1439878618 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6df4073b60 0x7f6df40b07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:46.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.247+0000 7f6df3e00640 1 -- 172.21.15.152:0/1439878618 >> 172.21.15.152:0/1439878618 conn(0x7f6df40d0260 msgr2=0x7f6df406c210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:46.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.247+0000 7f6df3e00640 1 -- 172.21.15.152:0/1439878618 shutdown_connections 2024-04-20T11:31:46.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.248+0000 7f6df3e00640 1 -- 172.21.15.152:0/1439878618 wait complete. 2024-04-20T11:31:46.267 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:31:46.457 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.457+0000 7fe8df000640 1 Processor -- start 2024-04-20T11:31:46.457 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.457+0000 7fe8df000640 1 -- start start 2024-04-20T11:31:46.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.459+0000 7fe8df000640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fe8d8147410 0x7fe8d80afa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:46.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.459+0000 7fe8df000640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8d80aec90 con 0x7fe8d8071cd0 2024-04-20T11:31:46.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.459+0000 7fe8df000640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8d80aef70 con 0x7fe8d806e720 2024-04-20T11:31:46.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.459+0000 7fe8df000640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fe8d80af250 con 0x7fe8d8147410 2024-04-20T11:31:46.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.459+0000 7fe8dd200640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fe8d8071cd0 0x7fe8d806e010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:56344/0 (socket says 172.21.15.152:56344) 2024-04-20T11:31:46.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.459+0000 7fe8dd200640 1 -- 172.21.15.152:0/1758448110 learned_addr learned my addr 172.21.15.152:0/1758448110 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:31:46.462 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.460+0000 7fe8ddc00640 1 --2- 172.21.15.152:0/1758448110 >> v2:172.21.15.175:3300/0 conn(0x7fe8d8147410 0x7fe8d80afa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:46.462 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.460+0000 7fe8d7e00640 1 -- 172.21.15.152:0/1758448110 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1475117141 0 0) 0x7fe8d80aec90 con 0x7fe8d8071cd0 2024-04-20T11:31:46.462 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.460+0000 7fe8d7e00640 1 -- 172.21.15.152:0/1758448110 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe8c0003520 con 0x7fe8d8071cd0 2024-04-20T11:31:46.462 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.460+0000 7fe8d7e00640 1 -- 172.21.15.152:0/1758448110 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1012199528 0 0) 0x7fe8c0003520 con 0x7fe8d8071cd0 2024-04-20T11:31:46.462 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.460+0000 7fe8d7e00640 1 -- 172.21.15.152:0/1758448110 >> v2:172.21.15.175:3300/0 conn(0x7fe8d8147410 msgr2=0x7fe8d80afa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:46.462 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.460+0000 7fe8d7e00640 1 --2- 172.21.15.152:0/1758448110 >> v2:172.21.15.175:3300/0 conn(0x7fe8d8147410 0x7fe8d80afa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:46.462 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.460+0000 7fe8d7e00640 1 -- 172.21.15.152:0/1758448110 >> v1:172.21.15.175:6789/0 conn(0x7fe8d806e720 legacy=0x7fe8d80b3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:46.462 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.460+0000 7fe8d7e00640 1 -- 172.21.15.152:0/1758448110 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8d80af530 con 0x7fe8d8071cd0 2024-04-20T11:31:46.463 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.460+0000 7fe8d7e00640 1 -- 172.21.15.152:0/1758448110 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7fe8c4001670 con 0x7fe8d8071cd0 2024-04-20T11:31:46.463 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.460+0000 7fe8d7e00640 1 -- 172.21.15.152:0/1758448110 >> v1:172.21.15.152:6789/0 conn(0x7fe8d8071cd0 legacy=0x7fe8d806e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:46.463 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.461+0000 7fe8d7e00640 1 --2- 172.21.15.152:0/1758448110 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe8c0003a00 0x7fe8c0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:46.463 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.461+0000 7fe8d7e00640 1 --2- 172.21.15.152:0/1758448110 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8c0004800 0x7fe8c0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:46.463 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.461+0000 7fe8d7e00640 1 --2- 172.21.15.152:0/1758448110 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8c00051c0 0x7fe8c000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:46.463 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.461+0000 7fe8d7e00640 1 -- 172.21.15.152:0/1758448110 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fe8d80af250 con 0x7fe8c0003a00 2024-04-20T11:31:46.463 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.461+0000 7fe8d7e00640 1 -- 172.21.15.152:0/1758448110 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c000c0c0 con 0x7fe8c00051c0 2024-04-20T11:31:46.463 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.461+0000 7fe8d7e00640 1 -- 172.21.15.152:0/1758448110 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe8c000c2c0 con 0x7fe8c0004800 2024-04-20T11:31:46.463 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.461+0000 7fe8ddc00640 1 --2- 172.21.15.152:0/1758448110 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe8c0003a00 0x7fe8c0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:46.463 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.461+0000 7fe8de600640 1 --2- 172.21.15.152:0/1758448110 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8c00051c0 0x7fe8c000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:46.463 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.461+0000 7fe8dd200640 1 --2- 172.21.15.152:0/1758448110 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8c0004800 0x7fe8c0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:46.463 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.461+0000 7fe8ddc00640 1 -- 172.21.15.152:0/1758448110 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8c0004800 msgr2=0x7fe8c0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:46.464 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.461+0000 7fe8ddc00640 1 --2- 172.21.15.152:0/1758448110 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8c0004800 0x7fe8c0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:46.464 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.461+0000 7fe8ddc00640 1 -- 172.21.15.152:0/1758448110 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8c00051c0 msgr2=0x7fe8c000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:46.464 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.461+0000 7fe8ddc00640 1 --2- 172.21.15.152:0/1758448110 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8c00051c0 0x7fe8c000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:46.464 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.461+0000 7fe8ddc00640 1 -- 172.21.15.152:0/1758448110 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8c0003520 con 0x7fe8c0003a00 2024-04-20T11:31:46.464 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.461+0000 7fe8de600640 1 --2- 172.21.15.152:0/1758448110 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8c00051c0 0x7fe8c000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:31:46.464 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.461+0000 7fe8de600640 1 -- 172.21.15.152:0/1758448110 reap_dead start 2024-04-20T11:31:46.464 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.461+0000 7fe8dd200640 1 --2- 172.21.15.152:0/1758448110 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8c0004800 0x7fe8c0004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:31:46.464 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.463+0000 7fe8ddc00640 1 --2- 172.21.15.152:0/1758448110 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe8c0003a00 0x7fe8c0003e20 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc009ac0 tx=0x7fe8c0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e084273286512d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:46.468 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.463+0000 7fe8d7e00640 1 -- 172.21.15.152:0/1758448110 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8cc039070 con 0x7fe8c0003a00 2024-04-20T11:31:46.468 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.463+0000 7fe8d7e00640 1 -- 172.21.15.152:0/1758448110 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8cc02c6b0 con 0x7fe8c0003a00 2024-04-20T11:31:46.468 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.463+0000 7fe8d7e00640 1 -- 172.21.15.152:0/1758448110 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8cc009690 con 0x7fe8c0003a00 2024-04-20T11:31:46.468 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.463+0000 7fe8df000640 1 -- 172.21.15.152:0/1758448110 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe8c0003a00 msgr2=0x7fe8c0003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:46.468 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.463+0000 7fe8df000640 1 --2- 172.21.15.152:0/1758448110 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe8c0003a00 0x7fe8c0003e20 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc009ac0 tx=0x7fe8c0005830 comp rx=0 tx=0).stop 2024-04-20T11:31:46.468 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.464+0000 7fe8df000640 1 -- 172.21.15.152:0/1758448110 shutdown_connections 2024-04-20T11:31:46.468 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.464+0000 7fe8df000640 1 --2- 172.21.15.152:0/1758448110 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe8c0003a00 0x7fe8c0003e20 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:46.469 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.464+0000 7fe8df000640 1 -- 172.21.15.152:0/1758448110 >> 172.21.15.152:0/1758448110 conn(0x7fe8d8067ae0 msgr2=0x7fe8d806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:46.469 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.464+0000 7fe8df000640 1 -- 172.21.15.152:0/1758448110 shutdown_connections 2024-04-20T11:31:46.469 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.464+0000 7fe8df000640 1 -- 172.21.15.152:0/1758448110 wait complete. 2024-04-20T11:31:46.469 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.465+0000 7fe8df000640 1 Processor -- start 2024-04-20T11:31:46.469 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.466+0000 7fe8df000640 1 -- start start 2024-04-20T11:31:46.469 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.466+0000 7fe8df000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8c0003a00 0x7fe8d80c7750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:46.469 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.466+0000 7fe8df000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8d80c5300 0x7fe8d80b3270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:46.469 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.466+0000 7fe8df000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe8d80c5780 0x7fe8d80b37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:46.469 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.467+0000 7fe8df000640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fe8d80afbd0 con 0x7fe8d80c5780 2024-04-20T11:31:46.469 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.467+0000 7fe8df000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe8d80afa50 con 0x7fe8c0003a00 2024-04-20T11:31:46.469 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.467+0000 7fe8df000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe8d80afd50 con 0x7fe8d80c5300 2024-04-20T11:31:46.469 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.467+0000 7fe8de600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe8d80c5780 0x7fe8d80b37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:46.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.467+0000 7fe8de600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe8d80c5780 0x7fe8d80b37b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36462/0 (socket says 172.21.15.152:36462) 2024-04-20T11:31:46.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.467+0000 7fe8de600640 1 -- 172.21.15.152:0/2595419935 learned_addr learned my addr 172.21.15.152:0/2595419935 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:46.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.467+0000 7fe8dd200640 1 --2- 172.21.15.152:0/2595419935 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8d80c5300 0x7fe8d80b3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:46.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.467+0000 7fe8ddc00640 1 --2- 172.21.15.152:0/2595419935 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8c0003a00 0x7fe8d80c7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:46.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.467+0000 7fe8de600640 1 -- 172.21.15.152:0/2595419935 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8d80c5300 msgr2=0x7fe8d80b3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:46.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.467+0000 7fe8de600640 1 --2- 172.21.15.152:0/2595419935 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8d80c5300 0x7fe8d80b3270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:46.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.467+0000 7fe8de600640 1 -- 172.21.15.152:0/2595419935 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8c0003a00 msgr2=0x7fe8d80c7750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:46.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.467+0000 7fe8de600640 1 --2- 172.21.15.152:0/2595419935 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8c0003a00 0x7fe8d80c7750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:46.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.467+0000 7fe8de600640 1 -- 172.21.15.152:0/2595419935 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8d80c5d50 con 0x7fe8d80c5780 2024-04-20T11:31:46.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.467+0000 7fe8dd200640 1 --2- 172.21.15.152:0/2595419935 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8d80c5300 0x7fe8d80b3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:31:46.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.467+0000 7fe8ddc00640 1 --2- 172.21.15.152:0/2595419935 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8c0003a00 0x7fe8d80c7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:31:46.471 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.468+0000 7fe8de600640 1 --2- 172.21.15.152:0/2595419935 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe8d80c5780 0x7fe8d80b37b0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fe8d80d0290 tx=0x7fe8d000d9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:46.471 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.468+0000 7fe8d5600640 1 -- 172.21.15.152:0/2595419935 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d0018070 con 0x7fe8d80c5780 2024-04-20T11:31:46.471 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.468+0000 7fe8d5600640 1 -- 172.21.15.152:0/2595419935 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8d0003da0 con 0x7fe8d80c5780 2024-04-20T11:31:46.471 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.468+0000 7fe8df000640 1 -- 172.21.15.152:0/2595419935 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8d80c6030 con 0x7fe8d80c5780 2024-04-20T11:31:46.471 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.468+0000 7fe8d5600640 1 -- 172.21.15.152:0/2595419935 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d0006d90 con 0x7fe8d80c5780 2024-04-20T11:31:46.471 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.468+0000 7fe8df000640 1 -- 172.21.15.152:0/2595419935 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8d80c65f0 con 0x7fe8d80c5780 2024-04-20T11:31:46.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.469+0000 7fe8df000640 1 -- 172.21.15.152:0/2595419935 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8d8075e60 con 0x7fe8d80c5780 2024-04-20T11:31:46.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.470+0000 7fe8d5600640 1 -- 172.21.15.152:0/2595419935 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 11) ==== 91346+0+0 (secure 0 0 0) 0x7fe8d001b070 con 0x7fe8d80c5780 2024-04-20T11:31:46.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.471+0000 7fe8d5600640 1 --2- 172.21.15.152:0/2595419935 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe8b806cc80 0x7fe8b806f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:46.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.471+0000 7fe8d5600640 1 -- 172.21.15.152:0/2595419935 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 4857+0+0 (secure 0 0 0) 0x7fe8d008af30 con 0x7fe8d80c5780 2024-04-20T11:31:46.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.471+0000 7fe8ddc00640 1 --2- 172.21.15.152:0/2595419935 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe8b806cc80 0x7fe8b806f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:46.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.472+0000 7fe8ddc00640 1 --2- 172.21.15.152:0/2595419935 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe8b806cc80 0x7fe8b806f140 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc036000 tx=0x7fe8cc030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:46.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.476+0000 7fe8d5600640 1 -- 172.21.15.152:0/2595419935 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe8d0059150 con 0x7fe8d80c5780 2024-04-20T11:31:46.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.681+0000 7fe8df000640 1 -- 172.21.15.152:0/2595419935 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe8d8141f80 con 0x7fe8d80c5780 2024-04-20T11:31:46.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.683+0000 7fe8d5600640 1 -- 172.21.15.152:0/2595419935 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v67) ==== 74+0+12951 (secure 0 0 0) 0x7fe8d0058a50 con 0x7fe8d80c5780 2024-04-20T11:31:46.682 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:31:46.683 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":67,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:31:45.249816+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:31:46.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.685+0000 7fe8df000640 1 -- 172.21.15.152:0/2595419935 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe8b806cc80 msgr2=0x7fe8b806f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:46.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.686+0000 7fe8df000640 1 --2- 172.21.15.152:0/2595419935 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe8b806cc80 0x7fe8b806f140 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc036000 tx=0x7fe8cc030000 comp rx=0 tx=0).stop 2024-04-20T11:31:46.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.686+0000 7fe8df000640 1 -- 172.21.15.152:0/2595419935 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe8d80c5780 msgr2=0x7fe8d80b37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:46.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.686+0000 7fe8df000640 1 --2- 172.21.15.152:0/2595419935 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe8d80c5780 0x7fe8d80b37b0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fe8d80d0290 tx=0x7fe8d000d9d0 comp rx=0 tx=0).stop 2024-04-20T11:31:46.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.686+0000 7fe8df000640 1 -- 172.21.15.152:0/2595419935 shutdown_connections 2024-04-20T11:31:46.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.686+0000 7fe8df000640 1 --2- 172.21.15.152:0/2595419935 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe8b806cc80 0x7fe8b806f140 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:46.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.686+0000 7fe8df000640 1 --2- 172.21.15.152:0/2595419935 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe8d80c5780 0x7fe8d80b37b0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:46.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.686+0000 7fe8df000640 1 --2- 172.21.15.152:0/2595419935 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8d80c5300 0x7fe8d80b3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:46.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.686+0000 7fe8df000640 1 --2- 172.21.15.152:0/2595419935 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8c0003a00 0x7fe8d80c7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:46.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.686+0000 7fe8df000640 1 -- 172.21.15.152:0/2595419935 >> 172.21.15.152:0/2595419935 conn(0x7fe8d8067ae0 msgr2=0x7fe8d81477f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:46.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.686+0000 7fe8df000640 1 -- 172.21.15.152:0/2595419935 shutdown_connections 2024-04-20T11:31:46.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.686+0000 7fe8df000640 1 -- 172.21.15.152:0/2595419935 wait complete. 2024-04-20T11:31:46.704 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T11:26:47.246571+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T11:31:46.704 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T11:31:46.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.904+0000 7fee21a00640 1 Processor -- start 2024-04-20T11:31:46.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.904+0000 7fee21a00640 1 -- start start 2024-04-20T11:31:46.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.906+0000 7fee21a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fee1c15ca40 0x7fee1c15cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:46.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.906+0000 7fee21a00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fee1c06bda0 0x7fee1c06c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:46.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.906+0000 7fee21a00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee1c15d5c0 con 0x7fee1c156c00 2024-04-20T11:31:46.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.906+0000 7fee21a00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fee1c06c750 con 0x7fee1c15ca40 2024-04-20T11:31:46.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.906+0000 7fee21a00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fee1c15d8a0 con 0x7fee1c06bda0 2024-04-20T11:31:46.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.906+0000 7fee1b400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fee1c15ca40 0x7fee1c15cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:46.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.906+0000 7fee1b400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fee1c15ca40 0x7fee1c15cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36472/0 (socket says 172.21.15.152:36472) 2024-04-20T11:31:46.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.906+0000 7fee1b400640 1 -- 172.21.15.152:0/2433301643 learned_addr learned my addr 172.21.15.152:0/2433301643 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:46.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.907+0000 7fee21000640 1 --2- 172.21.15.152:0/2433301643 >> v2:172.21.15.175:3300/0 conn(0x7fee1c06bda0 0x7fee1c06c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:46.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.907+0000 7fee1b400640 1 -- 172.21.15.152:0/2433301643 >> v2:172.21.15.175:3300/0 conn(0x7fee1c06bda0 msgr2=0x7fee1c06c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:46.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.907+0000 7fee1b400640 1 --2- 172.21.15.152:0/2433301643 >> v2:172.21.15.175:3300/0 conn(0x7fee1c06bda0 0x7fee1c06c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:46.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.907+0000 7fee1b400640 1 -- 172.21.15.152:0/2433301643 >> v1:172.21.15.152:6789/0 conn(0x7fee1c156c00 legacy=0x7fee1c160a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:46.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.907+0000 7fee1b400640 1 -- 172.21.15.152:0/2433301643 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee1c160290 con 0x7fee1c15ca40 2024-04-20T11:31:46.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.907+0000 7fee1aa00640 1 -- 172.21.15.152:0/2433301643 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 349366951 0 0) 0x7fee1c15d5c0 con 0x7fee1c156c00 2024-04-20T11:31:46.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.907+0000 7fee21000640 1 --2- 172.21.15.152:0/2433301643 >> v2:172.21.15.175:3300/0 conn(0x7fee1c06bda0 0x7fee1c06c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:31:46.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.908+0000 7fee1b400640 1 --2- 172.21.15.152:0/2433301643 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fee1c15ca40 0x7fee1c15cec0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fee1c071b40 tx=0x7fee0802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de60f3c264c67251 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:46.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.908+0000 7fee1aa00640 1 -- 172.21.15.152:0/2433301643 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee08034070 con 0x7fee1c15ca40 2024-04-20T11:31:46.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.908+0000 7fee1aa00640 1 -- 172.21.15.152:0/2433301643 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee08030860 con 0x7fee1c15ca40 2024-04-20T11:31:46.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.908+0000 7fee1aa00640 1 -- 172.21.15.152:0/2433301643 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee08030c30 con 0x7fee1c15ca40 2024-04-20T11:31:46.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.909+0000 7fee21a00640 1 -- 172.21.15.152:0/2433301643 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fee1c15ca40 msgr2=0x7fee1c15cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:46.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.909+0000 7fee21a00640 1 --2- 172.21.15.152:0/2433301643 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fee1c15ca40 0x7fee1c15cec0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fee1c071b40 tx=0x7fee0802b9a0 comp rx=0 tx=0).stop 2024-04-20T11:31:46.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.909+0000 7fee21a00640 1 -- 172.21.15.152:0/2433301643 shutdown_connections 2024-04-20T11:31:46.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.909+0000 7fee21a00640 1 --2- 172.21.15.152:0/2433301643 >> v2:172.21.15.175:3300/0 conn(0x7fee1c06bda0 0x7fee1c06c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:46.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.909+0000 7fee21a00640 1 --2- 172.21.15.152:0/2433301643 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fee1c15ca40 0x7fee1c15cec0 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:46.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.909+0000 7fee21a00640 1 -- 172.21.15.152:0/2433301643 >> 172.21.15.152:0/2433301643 conn(0x7fee1c0b21e0 msgr2=0x7fee1c0b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:46.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.909+0000 7fee21a00640 1 -- 172.21.15.152:0/2433301643 shutdown_connections 2024-04-20T11:31:46.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.909+0000 7fee21a00640 1 -- 172.21.15.152:0/2433301643 wait complete. 2024-04-20T11:31:46.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.911+0000 7fee21a00640 1 Processor -- start 2024-04-20T11:31:46.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.911+0000 7fee21a00640 1 -- start start 2024-04-20T11:31:46.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.912+0000 7fee21a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee1c06bda0 0x7fee1c14c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:46.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.912+0000 7fee21a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee1c156c00 0x7fee1c0af960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:46.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.912+0000 7fee21a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fee1c15ca40 0x7fee1c14a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:46.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.912+0000 7fee21a00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fee1c0b3240 con 0x7fee1c15ca40 2024-04-20T11:31:46.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.912+0000 7fee21a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fee1c0b30c0 con 0x7fee1c06bda0 2024-04-20T11:31:46.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.912+0000 7fee21a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fee1c0b33c0 con 0x7fee1c156c00 2024-04-20T11:31:46.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.912+0000 7fee21000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fee1c15ca40 0x7fee1c14a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:46.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.912+0000 7fee21000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fee1c15ca40 0x7fee1c14a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36474/0 (socket says 172.21.15.152:36474) 2024-04-20T11:31:46.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.912+0000 7fee21000640 1 -- 172.21.15.152:0/812273264 learned_addr learned my addr 172.21.15.152:0/812273264 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:46.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.912+0000 7fee1be00640 1 --2- 172.21.15.152:0/812273264 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee1c06bda0 0x7fee1c14c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:46.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.912+0000 7fee21000640 1 -- 172.21.15.152:0/812273264 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee1c156c00 msgr2=0x7fee1c0af960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:46.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.912+0000 7fee1b400640 1 --2- 172.21.15.152:0/812273264 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee1c156c00 0x7fee1c0af960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:46.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.912+0000 7fee21000640 1 --2- 172.21.15.152:0/812273264 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee1c156c00 0x7fee1c0af960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:46.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.912+0000 7fee21000640 1 -- 172.21.15.152:0/812273264 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee1c06bda0 msgr2=0x7fee1c14c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:46.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.912+0000 7fee21000640 1 --2- 172.21.15.152:0/812273264 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee1c06bda0 0x7fee1c14c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:46.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.912+0000 7fee21000640 1 -- 172.21.15.152:0/812273264 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee1c14ce90 con 0x7fee1c15ca40 2024-04-20T11:31:46.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.912+0000 7fee1be00640 1 --2- 172.21.15.152:0/812273264 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee1c06bda0 0x7fee1c14c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:31:46.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.913+0000 7fee21000640 1 --2- 172.21.15.152:0/812273264 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fee1c15ca40 0x7fee1c14a300 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fee1400cef0 tx=0x7fee1400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:46.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.913+0000 7fee13e00640 1 -- 172.21.15.152:0/812273264 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee1401a070 con 0x7fee1c15ca40 2024-04-20T11:31:46.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.913+0000 7fee13e00640 1 -- 172.21.15.152:0/812273264 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee1400d860 con 0x7fee1c15ca40 2024-04-20T11:31:46.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.913+0000 7fee13e00640 1 -- 172.21.15.152:0/812273264 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee1400dc50 con 0x7fee1c15ca40 2024-04-20T11:31:46.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.913+0000 7fee21a00640 1 -- 172.21.15.152:0/812273264 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee1c14d170 con 0x7fee1c15ca40 2024-04-20T11:31:46.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.913+0000 7fee21a00640 1 -- 172.21.15.152:0/812273264 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee1c0b0b80 con 0x7fee1c15ca40 2024-04-20T11:31:46.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.914+0000 7fee21a00640 1 -- 172.21.15.152:0/812273264 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee1c068d80 con 0x7fee1c15ca40 2024-04-20T11:31:46.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.915+0000 7fee13e00640 1 -- 172.21.15.152:0/812273264 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 11) ==== 91346+0+0 (secure 0 0 0) 0x7fee1401d070 con 0x7fee1c15ca40 2024-04-20T11:31:46.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.916+0000 7fee13e00640 1 --2- 172.21.15.152:0/812273264 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fedfc06cc50 0x7fedfc06f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:46.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.916+0000 7fee13e00640 1 -- 172.21.15.152:0/812273264 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 4857+0+0 (secure 0 0 0) 0x7fee1408c5b0 con 0x7fee1c15ca40 2024-04-20T11:31:46.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.916+0000 7fee1be00640 1 --2- 172.21.15.152:0/812273264 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fedfc06cc50 0x7fedfc06f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:46.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.917+0000 7fee1be00640 1 --2- 172.21.15.152:0/812273264 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fedfc06cc50 0x7fedfc06f110 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fee0c000f60 tx=0x7fee0c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:46.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:46.920+0000 7fee13e00640 1 -- 172.21.15.152:0/812273264 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fee1c068d80 con 0x7fee1c15ca40 2024-04-20T11:31:47.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.121+0000 7fee21a00640 1 -- 172.21.15.152:0/812273264 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fee1c155a20 con 0x7fee1c15ca40 2024-04-20T11:31:47.122 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.122+0000 7fee13e00640 1 -- 172.21.15.152:0/812273264 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v67) ==== 93+0+10 (secure 0 0 0) 0x7fee1c155a20 con 0x7fee1c15ca40 2024-04-20T11:31:47.122 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 1 2024-04-20T11:31:47.126 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.124+0000 7fee21a00640 1 -- 172.21.15.152:0/812273264 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fedfc06cc50 msgr2=0x7fedfc06f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:47.126 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.125+0000 7fee21a00640 1 --2- 172.21.15.152:0/812273264 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fedfc06cc50 0x7fedfc06f110 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fee0c000f60 tx=0x7fee0c009040 comp rx=0 tx=0).stop 2024-04-20T11:31:47.126 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.125+0000 7fee21a00640 1 -- 172.21.15.152:0/812273264 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fee1c15ca40 msgr2=0x7fee1c14a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:47.126 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.125+0000 7fee21a00640 1 --2- 172.21.15.152:0/812273264 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fee1c15ca40 0x7fee1c14a300 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fee1400cef0 tx=0x7fee1400a710 comp rx=0 tx=0).stop 2024-04-20T11:31:47.127 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.125+0000 7fee21a00640 1 -- 172.21.15.152:0/812273264 shutdown_connections 2024-04-20T11:31:47.127 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.125+0000 7fee21a00640 1 --2- 172.21.15.152:0/812273264 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fedfc06cc50 0x7fedfc06f110 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.127 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.125+0000 7fee21a00640 1 --2- 172.21.15.152:0/812273264 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fee1c15ca40 0x7fee1c14a300 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.127 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.125+0000 7fee21a00640 1 --2- 172.21.15.152:0/812273264 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee1c156c00 0x7fee1c0af960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.127 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.125+0000 7fee21a00640 1 --2- 172.21.15.152:0/812273264 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee1c06bda0 0x7fee1c14c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.127 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.125+0000 7fee21a00640 1 -- 172.21.15.152:0/812273264 >> 172.21.15.152:0/812273264 conn(0x7fee1c0b21e0 msgr2=0x7fee1c14aed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:47.127 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.125+0000 7fee21a00640 1 -- 172.21.15.152:0/812273264 shutdown_connections 2024-04-20T11:31:47.127 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.125+0000 7fee21a00640 1 -- 172.21.15.152:0/812273264 wait complete. 2024-04-20T11:31:47.144 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-20T11:31:47.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.344+0000 7f2b05000640 1 Processor -- start 2024-04-20T11:31:47.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.344+0000 7f2b05000640 1 -- start start 2024-04-20T11:31:47.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.345+0000 7f2b05000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f2b0015d1e0 0x7f2b00160b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:47.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.345+0000 7f2b05000640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b000ce8c0 con 0x7f2b0015d5c0 2024-04-20T11:31:47.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.345+0000 7f2b05000640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b001600b0 con 0x7f2b00071cd0 2024-04-20T11:31:47.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.345+0000 7f2b05000640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f2b00160390 con 0x7f2b0015d1e0 2024-04-20T11:31:47.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.346+0000 7f2afea00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f2b0015d5c0 0x7f2b000cdfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:56366/0 (socket says 172.21.15.152:56366) 2024-04-20T11:31:47.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.346+0000 7f2afea00640 1 -- 172.21.15.152:0/4023677984 learned_addr learned my addr 172.21.15.152:0/4023677984 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:31:47.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.346+0000 7f2aff400640 1 --2- 172.21.15.152:0/4023677984 >> v2:172.21.15.152:3300/0 conn(0x7f2b0015d1e0 0x7f2b00160b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:47.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.346+0000 7f2afe000640 1 -- 172.21.15.152:0/4023677984 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3953363829 0 0) 0x7f2b000ce8c0 con 0x7f2b0015d5c0 2024-04-20T11:31:47.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.346+0000 7f2afe000640 1 -- 172.21.15.152:0/4023677984 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2ae4003520 con 0x7f2b0015d5c0 2024-04-20T11:31:47.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.346+0000 7f2aff400640 1 -- 172.21.15.152:0/4023677984 >> v1:172.21.15.175:6789/0 conn(0x7f2b00071cd0 legacy=0x7f2b00072120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:47.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.346+0000 7f2aff400640 1 -- 172.21.15.152:0/4023677984 >> v1:172.21.15.152:6789/0 conn(0x7f2b0015d5c0 legacy=0x7f2b000cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:47.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.346+0000 7f2aff400640 1 -- 172.21.15.152:0/4023677984 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b00160670 con 0x7f2b0015d1e0 2024-04-20T11:31:47.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.348+0000 7f2aff400640 1 --2- 172.21.15.152:0/4023677984 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2b0015d1e0 0x7f2b00160b90 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f2af00097d0 tx=0x7f2af002b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3f90de8181f1692 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:47.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.348+0000 7f2afe000640 1 -- 172.21.15.152:0/4023677984 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af0035020 con 0x7f2b0015d1e0 2024-04-20T11:31:47.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.348+0000 7f2afe000640 1 -- 172.21.15.152:0/4023677984 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2af0031810 con 0x7f2b0015d1e0 2024-04-20T11:31:47.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.348+0000 7f2afe000640 1 -- 172.21.15.152:0/4023677984 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af0031be0 con 0x7f2b0015d1e0 2024-04-20T11:31:47.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.349+0000 7f2b05000640 1 -- 172.21.15.152:0/4023677984 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2b0015d1e0 msgr2=0x7f2b00160b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:47.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.349+0000 7f2b05000640 1 --2- 172.21.15.152:0/4023677984 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2b0015d1e0 0x7f2b00160b90 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f2af00097d0 tx=0x7f2af002b8b0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.349+0000 7f2b05000640 1 -- 172.21.15.152:0/4023677984 shutdown_connections 2024-04-20T11:31:47.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.349+0000 7f2b05000640 1 --2- 172.21.15.152:0/4023677984 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2b0015d1e0 0x7f2b00160b90 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.349+0000 7f2b05000640 1 -- 172.21.15.152:0/4023677984 >> 172.21.15.152:0/4023677984 conn(0x7f2b00067ae0 msgr2=0x7f2b0006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:47.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.349+0000 7f2b05000640 1 -- 172.21.15.152:0/4023677984 shutdown_connections 2024-04-20T11:31:47.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.349+0000 7f2b05000640 1 -- 172.21.15.152:0/4023677984 wait complete. 2024-04-20T11:31:47.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.350+0000 7f2b05000640 1 Processor -- start 2024-04-20T11:31:47.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.351+0000 7f2b05000640 1 -- start start 2024-04-20T11:31:47.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.351+0000 7f2b05000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2b00071cd0 0x7f2b000b28c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:47.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.352+0000 7f2b05000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2b0015d1e0 0x7f2b0014f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:47.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.352+0000 7f2b05000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b0015d5c0 0x7f2b0014fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:47.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.352+0000 7f2b05000640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f2b000ccef0 con 0x7f2b00071cd0 2024-04-20T11:31:47.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.352+0000 7f2b05000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2b000ccd70 con 0x7f2b0015d5c0 2024-04-20T11:31:47.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.352+0000 7f2b05000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2b000cd070 con 0x7f2b0015d1e0 2024-04-20T11:31:47.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.352+0000 7f2aff400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2b00071cd0 0x7f2b000b28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:47.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.352+0000 7f2aff400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2b00071cd0 0x7f2b000b28c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36492/0 (socket says 172.21.15.152:36492) 2024-04-20T11:31:47.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.352+0000 7f2aff400640 1 -- 172.21.15.152:0/2523941651 learned_addr learned my addr 172.21.15.152:0/2523941651 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:47.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.352+0000 7f2aff400640 1 -- 172.21.15.152:0/2523941651 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2b0015d1e0 msgr2=0x7f2b0014f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:47.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.352+0000 7f2aff400640 1 --2- 172.21.15.152:0/2523941651 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2b0015d1e0 0x7f2b0014f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.352+0000 7f2aff400640 1 -- 172.21.15.152:0/2523941651 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b0015d5c0 msgr2=0x7f2b0014fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:47.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.352+0000 7f2aff400640 1 --2- 172.21.15.152:0/2523941651 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b0015d5c0 0x7f2b0014fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.352+0000 7f2aff400640 1 -- 172.21.15.152:0/2523941651 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b001527f0 con 0x7f2b00071cd0 2024-04-20T11:31:47.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.353+0000 7f2aff400640 1 --2- 172.21.15.152:0/2523941651 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2b00071cd0 0x7f2b000b28c0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f2af00098f0 tx=0x7f2af00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:47.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.353+0000 7f2af7400640 1 -- 172.21.15.152:0/2523941651 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af0041070 con 0x7f2b00071cd0 2024-04-20T11:31:47.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.353+0000 7f2af7400640 1 -- 172.21.15.152:0/2523941651 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2af003ad00 con 0x7f2b00071cd0 2024-04-20T11:31:47.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.353+0000 7f2af7400640 1 -- 172.21.15.152:0/2523941651 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af0035040 con 0x7f2b00071cd0 2024-04-20T11:31:47.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.353+0000 7f2b05000640 1 -- 172.21.15.152:0/2523941651 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b00152510 con 0x7f2b00071cd0 2024-04-20T11:31:47.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.353+0000 7f2b05000640 1 -- 172.21.15.152:0/2523941651 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b00157f60 con 0x7f2b00071cd0 2024-04-20T11:31:47.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.354+0000 7f2b05000640 1 -- 172.21.15.152:0/2523941651 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b00075e60 con 0x7f2b00071cd0 2024-04-20T11:31:47.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.355+0000 7f2af7400640 1 -- 172.21.15.152:0/2523941651 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 11) ==== 91346+0+0 (secure 0 0 0) 0x7f2af0038020 con 0x7f2b00071cd0 2024-04-20T11:31:47.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.356+0000 7f2af7400640 1 --2- 172.21.15.152:0/2523941651 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f2ae006cc50 0x7f2ae006f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:47.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.356+0000 7f2af7400640 1 -- 172.21.15.152:0/2523941651 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 4857+0+0 (secure 0 0 0) 0x7f2af00ada00 con 0x7f2b00071cd0 2024-04-20T11:31:47.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.356+0000 7f2afea00640 1 --2- 172.21.15.152:0/2523941651 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f2ae006cc50 0x7f2ae006f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:47.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.357+0000 7f2afea00640 1 --2- 172.21.15.152:0/2523941651 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f2ae006cc50 0x7f2ae006f110 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f2aec000f60 tx=0x7f2aec0078f0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:47.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.360+0000 7f2af7400640 1 -- 172.21.15.152:0/2523941651 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f2af007bd30 con 0x7f2b00071cd0 2024-04-20T11:31:47.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.560+0000 7f2b05000640 1 -- 172.21.15.152:0/2523941651 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f2b00072c90 con 0x7f2b00071cd0 2024-04-20T11:31:47.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.561+0000 7f2af7400640 1 -- 172.21.15.152:0/2523941651 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v67) ==== 84+0+65 (secure 0 0 0) 0x7f2af007b630 con 0x7f2b00071cd0 2024-04-20T11:31:47.561 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:31:47.561 INFO:teuthology.orchestra.run.smithi152.stdout:[ 2024-04-20T11:31:47.561 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:31:47.561 INFO:teuthology.orchestra.run.smithi152.stdout: "poolnum": 1, 2024-04-20T11:31:47.561 INFO:teuthology.orchestra.run.smithi152.stdout: "poolname": ".mgr" 2024-04-20T11:31:47.561 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:31:47.561 INFO:teuthology.orchestra.run.smithi152.stdout:] 2024-04-20T11:31:47.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.563+0000 7f2b05000640 1 -- 172.21.15.152:0/2523941651 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f2ae006cc50 msgr2=0x7f2ae006f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:47.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.563+0000 7f2b05000640 1 --2- 172.21.15.152:0/2523941651 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f2ae006cc50 0x7f2ae006f110 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f2aec000f60 tx=0x7f2aec0078f0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.563+0000 7f2b05000640 1 -- 172.21.15.152:0/2523941651 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2b00071cd0 msgr2=0x7f2b000b28c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:47.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.563+0000 7f2b05000640 1 --2- 172.21.15.152:0/2523941651 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2b00071cd0 0x7f2b000b28c0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f2af00098f0 tx=0x7f2af00046c0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.563+0000 7f2b05000640 1 -- 172.21.15.152:0/2523941651 shutdown_connections 2024-04-20T11:31:47.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.563+0000 7f2b05000640 1 --2- 172.21.15.152:0/2523941651 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f2ae006cc50 0x7f2ae006f110 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.563+0000 7f2b05000640 1 --2- 172.21.15.152:0/2523941651 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b0015d5c0 0x7f2b0014fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.563+0000 7f2b05000640 1 --2- 172.21.15.152:0/2523941651 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2b0015d1e0 0x7f2b0014f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.563+0000 7f2b05000640 1 --2- 172.21.15.152:0/2523941651 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2b00071cd0 0x7f2b000b28c0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.563+0000 7f2b05000640 1 -- 172.21.15.152:0/2523941651 >> 172.21.15.152:0/2523941651 conn(0x7f2b00067ae0 msgr2=0x7f2b0006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:47.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.564+0000 7f2b05000640 1 -- 172.21.15.152:0/2523941651 shutdown_connections 2024-04-20T11:31:47.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.564+0000 7f2b05000640 1 -- 172.21.15.152:0/2523941651 wait complete. 2024-04-20T11:31:47.582 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-20T11:31:47.582 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-20T11:31:47.791 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.791+0000 7efe0fe00640 1 Processor -- start 2024-04-20T11:31:47.791 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.792+0000 7efe0fe00640 1 -- start start 2024-04-20T11:31:47.793 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.793+0000 7efe0fe00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7efe100ae180 0x7efe100ae600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:47.793 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.793+0000 7efe0fe00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7efe10071f20 0x7efe10072390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:47.793 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.793+0000 7efe0fe00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe100aeef0 con 0x7efe100d05f0 2024-04-20T11:31:47.793 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.793+0000 7efe0fe00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7efe100af1d0 con 0x7efe10071f20 2024-04-20T11:31:47.793 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.793+0000 7efe0fe00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7efe100af4b0 con 0x7efe100ae180 2024-04-20T11:31:47.793 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.793+0000 7efe0f400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7efe10071f20 0x7efe10072390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:47.794 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.793+0000 7efe0f400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7efe10071f20 0x7efe10072390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36508/0 (socket says 172.21.15.152:36508) 2024-04-20T11:31:47.794 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.793+0000 7efe0f400640 1 -- 172.21.15.152:0/982373196 learned_addr learned my addr 172.21.15.152:0/982373196 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:47.794 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.793+0000 7efe0e000640 1 --2- 172.21.15.152:0/982373196 >> v2:172.21.15.175:3300/0 conn(0x7efe100ae180 0x7efe100ae600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:47.794 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.794+0000 7efe0f400640 1 -- 172.21.15.152:0/982373196 >> v2:172.21.15.175:3300/0 conn(0x7efe100ae180 msgr2=0x7efe100ae600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:47.794 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.794+0000 7efe0f400640 1 --2- 172.21.15.152:0/982373196 >> v2:172.21.15.175:3300/0 conn(0x7efe100ae180 0x7efe100ae600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.794 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.794+0000 7efe0f400640 1 -- 172.21.15.152:0/982373196 >> v1:172.21.15.175:6789/0 conn(0x7efe100d05f0 legacy=0x7efe100d09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:47.794 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.794+0000 7efe0f400640 1 -- 172.21.15.152:0/982373196 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe100af790 con 0x7efe10071f20 2024-04-20T11:31:47.795 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.795+0000 7efe0f400640 1 --2- 172.21.15.152:0/982373196 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efe10071f20 0x7efe10072390 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7efe100b6360 tx=0x7efe0802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f83f045e81776f97 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:47.795 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.795+0000 7efe0d600640 1 -- 172.21.15.152:0/982373196 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe08035020 con 0x7efe10071f20 2024-04-20T11:31:47.795 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.795+0000 7efe0d600640 1 -- 172.21.15.152:0/982373196 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe08031870 con 0x7efe10071f20 2024-04-20T11:31:47.795 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.795+0000 7efe0d600640 1 -- 172.21.15.152:0/982373196 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe08031c40 con 0x7efe10071f20 2024-04-20T11:31:47.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.796+0000 7efe0fe00640 1 -- 172.21.15.152:0/982373196 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efe10071f20 msgr2=0x7efe10072390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:47.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.796+0000 7efe0fe00640 1 --2- 172.21.15.152:0/982373196 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efe10071f20 0x7efe10072390 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7efe100b6360 tx=0x7efe0802b910 comp rx=0 tx=0).stop 2024-04-20T11:31:47.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.796+0000 7efe0fe00640 1 -- 172.21.15.152:0/982373196 shutdown_connections 2024-04-20T11:31:47.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.796+0000 7efe0fe00640 1 --2- 172.21.15.152:0/982373196 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efe10071f20 0x7efe10072390 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.796+0000 7efe0fe00640 1 --2- 172.21.15.152:0/982373196 >> v2:172.21.15.175:3300/0 conn(0x7efe100ae180 0x7efe100ae600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.796+0000 7efe0fe00640 1 -- 172.21.15.152:0/982373196 >> 172.21.15.152:0/982373196 conn(0x7efe100b21e0 msgr2=0x7efe100b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:47.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.796+0000 7efe0fe00640 1 -- 172.21.15.152:0/982373196 shutdown_connections 2024-04-20T11:31:47.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.796+0000 7efe0fe00640 1 -- 172.21.15.152:0/982373196 wait complete. 2024-04-20T11:31:47.797 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.798+0000 7efe0fe00640 1 Processor -- start 2024-04-20T11:31:47.797 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.798+0000 7efe0fe00640 1 -- start start 2024-04-20T11:31:47.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.799+0000 7efe0fe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efe10071f20 0x7efe1006e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:47.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.799+0000 7efe0fe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efe100ae180 0x7efe1009bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:47.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.799+0000 7efe0fe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efe100d05f0 0x7efe1009c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:47.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.799+0000 7efe0fe00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7efe10155ca0 con 0x7efe100d05f0 2024-04-20T11:31:47.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.799+0000 7efe0fe00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7efe100aeb40 con 0x7efe100ae180 2024-04-20T11:31:47.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.799+0000 7efe0fe00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7efe101f2270 con 0x7efe10071f20 2024-04-20T11:31:47.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.799+0000 7efe0f400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efe100d05f0 0x7efe1009c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:47.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.799+0000 7efe0f400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efe100d05f0 0x7efe1009c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36520/0 (socket says 172.21.15.152:36520) 2024-04-20T11:31:47.800 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.799+0000 7efe0f400640 1 -- 172.21.15.152:0/3127756982 learned_addr learned my addr 172.21.15.152:0/3127756982 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:47.800 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.799+0000 7efe0ea00640 1 --2- 172.21.15.152:0/3127756982 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efe10071f20 0x7efe1006e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:47.800 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.799+0000 7efe0e000640 1 --2- 172.21.15.152:0/3127756982 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efe100ae180 0x7efe1009bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:47.800 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.799+0000 7efe0f400640 1 -- 172.21.15.152:0/3127756982 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efe10071f20 msgr2=0x7efe1006e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:47.800 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.799+0000 7efe0f400640 1 --2- 172.21.15.152:0/3127756982 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efe10071f20 0x7efe1006e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.800 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.799+0000 7efe0f400640 1 -- 172.21.15.152:0/3127756982 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efe100ae180 msgr2=0x7efe1009bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:47.800 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.799+0000 7efe0f400640 1 --2- 172.21.15.152:0/3127756982 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efe100ae180 0x7efe1009bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.800 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.799+0000 7efe0f400640 1 -- 172.21.15.152:0/3127756982 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe1009ca50 con 0x7efe100d05f0 2024-04-20T11:31:47.800 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.799+0000 7efe0ea00640 1 --2- 172.21.15.152:0/3127756982 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efe10071f20 0x7efe1006e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:31:47.800 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.800+0000 7efe0f400640 1 --2- 172.21.15.152:0/3127756982 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efe100d05f0 0x7efe1009c510 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7efe080090b0 tx=0x7efe080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:47.801 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.800+0000 7efe06a00640 1 -- 172.21.15.152:0/3127756982 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe08040070 con 0x7efe100d05f0 2024-04-20T11:31:47.801 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.800+0000 7efe06a00640 1 -- 172.21.15.152:0/3127756982 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe08030d00 con 0x7efe100d05f0 2024-04-20T11:31:47.801 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.800+0000 7efe0fe00640 1 -- 172.21.15.152:0/3127756982 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe1015dff0 con 0x7efe100d05f0 2024-04-20T11:31:47.801 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.800+0000 7efe06a00640 1 -- 172.21.15.152:0/3127756982 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe08035040 con 0x7efe100d05f0 2024-04-20T11:31:47.801 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.800+0000 7efe0fe00640 1 -- 172.21.15.152:0/3127756982 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe1015e5b0 con 0x7efe100d05f0 2024-04-20T11:31:47.801 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.802+0000 7efe0fe00640 1 -- 172.21.15.152:0/3127756982 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe10075e60 con 0x7efe100d05f0 2024-04-20T11:31:47.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.802+0000 7efe06a00640 1 -- 172.21.15.152:0/3127756982 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 11) ==== 91346+0+0 (secure 0 0 0) 0x7efe0804b060 con 0x7efe100d05f0 2024-04-20T11:31:47.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.803+0000 7efe06a00640 1 --2- 172.21.15.152:0/3127756982 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7efdf006cc50 0x7efdf006f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:47.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.804+0000 7efe06a00640 1 -- 172.21.15.152:0/3127756982 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 4857+0+0 (secure 0 0 0) 0x7efe080adaf0 con 0x7efe100d05f0 2024-04-20T11:31:47.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.804+0000 7efe0ea00640 1 --2- 172.21.15.152:0/3127756982 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7efdf006cc50 0x7efdf006f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:47.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.804+0000 7efe0ea00640 1 --2- 172.21.15.152:0/3127756982 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7efdf006cc50 0x7efdf006f110 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7efe00004520 tx=0x7efe00009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:47.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:47.808+0000 7efe06a00640 1 -- 172.21.15.152:0/3127756982 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7efe0807be60 con 0x7efe100d05f0 2024-04-20T11:31:48.007 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:48.007+0000 7efe0fe00640 1 -- 172.21.15.152:0/3127756982 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7efe1006f3a0 con 0x7efe100d05f0 2024-04-20T11:31:48.253 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:48.254+0000 7efe06a00640 1 -- 172.21.15.152:0/3127756982 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v68) ==== 138+0+0 (secure 0 0 0) 0x7efe0807b760 con 0x7efe100d05f0 2024-04-20T11:31:48.254 INFO:teuthology.orchestra.run.smithi152.stderr:pool 'cephfs_metadata' created 2024-04-20T11:31:48.258 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:48.258+0000 7efe0fe00640 1 -- 172.21.15.152:0/3127756982 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7efdf006cc50 msgr2=0x7efdf006f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:48.258 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:48.258+0000 7efe0fe00640 1 --2- 172.21.15.152:0/3127756982 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7efdf006cc50 0x7efdf006f110 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7efe00004520 tx=0x7efe00009040 comp rx=0 tx=0).stop 2024-04-20T11:31:48.258 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:48.258+0000 7efe0fe00640 1 -- 172.21.15.152:0/3127756982 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efe100d05f0 msgr2=0x7efe1009c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:48.258 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:48.258+0000 7efe0fe00640 1 --2- 172.21.15.152:0/3127756982 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efe100d05f0 0x7efe1009c510 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7efe080090b0 tx=0x7efe080046c0 comp rx=0 tx=0).stop 2024-04-20T11:31:48.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:48.259+0000 7efe0fe00640 1 -- 172.21.15.152:0/3127756982 shutdown_connections 2024-04-20T11:31:48.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:48.259+0000 7efe0fe00640 1 --2- 172.21.15.152:0/3127756982 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7efdf006cc50 0x7efdf006f110 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:48.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:48.259+0000 7efe0fe00640 1 --2- 172.21.15.152:0/3127756982 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efe100d05f0 0x7efe1009c510 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:48.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:48.259+0000 7efe0fe00640 1 --2- 172.21.15.152:0/3127756982 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efe100ae180 0x7efe1009bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:48.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:48.259+0000 7efe0fe00640 1 --2- 172.21.15.152:0/3127756982 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efe10071f20 0x7efe1006e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:48.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:48.259+0000 7efe0fe00640 1 -- 172.21.15.152:0/3127756982 >> 172.21.15.152:0/3127756982 conn(0x7efe100b21e0 msgr2=0x7efe10072300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:48.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:48.259+0000 7efe0fe00640 1 -- 172.21.15.152:0/3127756982 shutdown_connections 2024-04-20T11:31:48.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:48.259+0000 7efe0fe00640 1 -- 172.21.15.152:0/3127756982 wait complete. 2024-04-20T11:31:48.288 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-20T11:31:48.487 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:48.488+0000 7f4381400640 1 Processor -- start 2024-04-20T11:31:48.487 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:48.488+0000 7f4381400640 1 -- start start 2024-04-20T11:31:48.489 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:48.489+0000 7f4381400640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f437c0ccff0 0x7f437c0cd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:48.489 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:48.489+0000 7f4381400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f437c0cf1b0 0x7f437c0cf5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:48.489 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:48.489+0000 7f4381400640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f437c06f500 con 0x7f437c0d0b80 2024-04-20T11:31:48.489 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:48.489+0000 7f4381400640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f437c145830 con 0x7f437c0cf1b0 2024-04-20T11:31:48.489 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:48.489+0000 7f4381400640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f437c06f070 con 0x7f437c0ccff0 2024-04-20T11:31:48.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:48.490+0000 7f437b200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f437c0cf1b0 0x7f437c0cf5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:48.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:48.490+0000 7f437a800640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f437c0d0b80 0x7f437c160d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:56380/0 (socket says 172.21.15.152:56380) 2024-04-20T11:31:48.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:48.490+0000 7f437a800640 1 -- 172.21.15.152:0/510126452 learned_addr learned my addr 172.21.15.152:0/510126452 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:31:48.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:48.490+0000 7f4371e00640 1 --2- 172.21.15.152:0/510126452 >> v2:172.21.15.175:3300/0 conn(0x7f437c0ccff0 0x7f437c0cd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:48.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:48.490+0000 7f437b200640 1 -- 172.21.15.152:0/510126452 >> v2:172.21.15.175:3300/0 conn(0x7f437c0ccff0 msgr2=0x7f437c0cd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:48.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:48.490+0000 7f437b200640 1 --2- 172.21.15.152:0/510126452 >> v2:172.21.15.175:3300/0 conn(0x7f437c0ccff0 0x7f437c0cd430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:48.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:48.490+0000 7f437b200640 1 -- 172.21.15.152:0/510126452 >> v1:172.21.15.152:6789/0 conn(0x7f437c0d0b80 legacy=0x7f437c160d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:48.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:48.490+0000 7f437b200640 1 -- 172.21.15.152:0/510126452 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f437c06e460 con 0x7f437c0cf1b0 2024-04-20T11:31:48.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:48.490+0000 7f4371e00640 1 --2- 172.21.15.152:0/510126452 >> v2:172.21.15.175:3300/0 conn(0x7f437c0ccff0 0x7f437c0cd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:31:48.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:48.492+0000 7f437b200640 1 --2- 172.21.15.152:0/510126452 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f437c0cf1b0 0x7f437c0cf5e0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f437c0afe50 tx=0x7f437402bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85c29ae33f85435d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:48.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:48.492+0000 7f4379e00640 1 -- 172.21.15.152:0/510126452 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4374039070 con 0x7f437c0cf1b0 2024-04-20T11:31:48.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:48.492+0000 7f4379e00640 1 -- 172.21.15.152:0/510126452 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f437402cc70 con 0x7f437c0cf1b0 2024-04-20T11:31:48.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:48.492+0000 7f4379e00640 1 -- 172.21.15.152:0/510126452 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4374034660 con 0x7f437c0cf1b0 2024-04-20T11:31:48.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:48.493+0000 7f4381400640 1 -- 172.21.15.152:0/510126452 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f437c0cf1b0 msgr2=0x7f437c0cf5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:48.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:48.493+0000 7f4381400640 1 --2- 172.21.15.152:0/510126452 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f437c0cf1b0 0x7f437c0cf5e0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f437c0afe50 tx=0x7f437402bec0 comp rx=0 tx=0).stop 2024-04-20T11:31:48.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:48.493+0000 7f4381400640 1 -- 172.21.15.152:0/510126452 shutdown_connections 2024-04-20T11:31:48.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:48.493+0000 7f4381400640 1 --2- 172.21.15.152:0/510126452 >> v2:172.21.15.175:3300/0 conn(0x7f437c0ccff0 0x7f437c0cd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:48.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:48.493+0000 7f4381400640 1 --2- 172.21.15.152:0/510126452 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f437c0cf1b0 0x7f437c0cf5e0 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:48.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:48.493+0000 7f4381400640 1 -- 172.21.15.152:0/510126452 >> 172.21.15.152:0/510126452 conn(0x7f437c06b980 msgr2=0x7f437c06bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:48.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:48.493+0000 7f4381400640 1 -- 172.21.15.152:0/510126452 shutdown_connections 2024-04-20T11:31:48.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:48.493+0000 7f4381400640 1 -- 172.21.15.152:0/510126452 wait complete. 2024-04-20T11:31:48.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:48.495+0000 7f4381400640 1 Processor -- start 2024-04-20T11:31:48.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:48.495+0000 7f4381400640 1 -- start start 2024-04-20T11:31:48.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:48.496+0000 7f4381400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f437c0ccff0 0x7f437c13a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:48.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:48.496+0000 7f4381400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f437c0cf1b0 0x7f437c0c63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:48.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:48.496+0000 7f4381400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f437c0d0b80 0x7f437c0aeea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:48.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:48.496+0000 7f4381400640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f437c06ea20 con 0x7f437c0ccff0 2024-04-20T11:31:48.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:48.496+0000 7f4381400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f437c0cd970 con 0x7f437c0cf1b0 2024-04-20T11:31:48.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:48.496+0000 7f4381400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f437c06ed00 con 0x7f437c0d0b80 2024-04-20T11:31:48.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:48.496+0000 7f437a800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f437c0ccff0 0x7f437c13a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:48.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:48.496+0000 7f437a800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f437c0ccff0 0x7f437c13a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36526/0 (socket says 172.21.15.152:36526) 2024-04-20T11:31:48.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:48.496+0000 7f437a800640 1 -- 172.21.15.152:0/3366429577 learned_addr learned my addr 172.21.15.152:0/3366429577 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:48.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:48.497+0000 7f4371e00640 1 --2- 172.21.15.152:0/3366429577 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f437c0cf1b0 0x7f437c0c63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:48.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:48.497+0000 7f437b200640 1 --2- 172.21.15.152:0/3366429577 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f437c0d0b80 0x7f437c0aeea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:48.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:48.497+0000 7f437a800640 1 -- 172.21.15.152:0/3366429577 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f437c0d0b80 msgr2=0x7f437c0aeea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:48.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:48.497+0000 7f437a800640 1 --2- 172.21.15.152:0/3366429577 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f437c0d0b80 0x7f437c0aeea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:48.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:48.497+0000 7f437a800640 1 -- 172.21.15.152:0/3366429577 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f437c0cf1b0 msgr2=0x7f437c0c63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:48.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:48.497+0000 7f437a800640 1 --2- 172.21.15.152:0/3366429577 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f437c0cf1b0 0x7f437c0c63a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:48.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:48.497+0000 7f437a800640 1 -- 172.21.15.152:0/3366429577 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f437c0c5300 con 0x7f437c0ccff0 2024-04-20T11:31:48.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:48.497+0000 7f4371e00640 1 --2- 172.21.15.152:0/3366429577 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f437c0cf1b0 0x7f437c0c63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:31:48.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:48.497+0000 7f437a800640 1 --2- 172.21.15.152:0/3366429577 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f437c0ccff0 0x7f437c13a7b0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f436c00c430 tx=0x7f436c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:48.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:48.497+0000 7f4373400640 1 -- 172.21.15.152:0/3366429577 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f436c017070 con 0x7f437c0ccff0 2024-04-20T11:31:48.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:48.497+0000 7f4373400640 1 -- 172.21.15.152:0/3366429577 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f436c005ce0 con 0x7f437c0ccff0 2024-04-20T11:31:48.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:48.497+0000 7f4373400640 1 -- 172.21.15.152:0/3366429577 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f436c0072f0 con 0x7f437c0ccff0 2024-04-20T11:31:48.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:48.497+0000 7f4381400640 1 -- 172.21.15.152:0/3366429577 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f437c0c55e0 con 0x7f437c0ccff0 2024-04-20T11:31:48.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:48.498+0000 7f4381400640 1 -- 172.21.15.152:0/3366429577 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f437c0c5e80 con 0x7f437c0ccff0 2024-04-20T11:31:48.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:48.499+0000 7f4381400640 1 -- 172.21.15.152:0/3366429577 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f437c0c57b0 con 0x7f437c0ccff0 2024-04-20T11:31:48.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:48.501+0000 7f4373400640 1 -- 172.21.15.152:0/3366429577 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 11) ==== 91346+0+0 (secure 0 0 0) 0x7f436c01a070 con 0x7f437c0ccff0 2024-04-20T11:31:48.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:48.502+0000 7f4373400640 1 --2- 172.21.15.152:0/3366429577 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f435806cc50 0x7f435806f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:48.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:48.502+0000 7f4373400640 1 -- 172.21.15.152:0/3366429577 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5227+0+0 (secure 0 0 0) 0x7f436c08b690 con 0x7f437c0ccff0 2024-04-20T11:31:48.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:48.502+0000 7f4371e00640 1 --2- 172.21.15.152:0/3366429577 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f435806cc50 0x7f435806f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:48.502 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:48.503+0000 7f4371e00640 1 --2- 172.21.15.152:0/3366429577 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f435806cc50 0x7f435806f110 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f4364006fd0 tx=0x7f4364008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:48.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:48.506+0000 7f4373400640 1 -- 172.21.15.152:0/3366429577 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f436c021020 con 0x7f437c0ccff0 2024-04-20T11:31:48.707 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:48.708+0000 7f4381400640 1 -- 172.21.15.152:0/3366429577 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f437c0d1050 con 0x7f437c0ccff0 2024-04-20T11:31:49.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:49.260+0000 7f4373400640 1 -- 172.21.15.152:0/3366429577 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v69) ==== 170+0+0 (secure 0 0 0) 0x7f436c059990 con 0x7f437c0ccff0 2024-04-20T11:31:49.260 INFO:teuthology.orchestra.run.smithi152.stderr:pool 'cephfs_data' created 2024-04-20T11:31:49.263 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:49.264+0000 7f4381400640 1 -- 172.21.15.152:0/3366429577 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f435806cc50 msgr2=0x7f435806f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:49.263 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:49.264+0000 7f4381400640 1 --2- 172.21.15.152:0/3366429577 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f435806cc50 0x7f435806f110 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f4364006fd0 tx=0x7f4364008040 comp rx=0 tx=0).stop 2024-04-20T11:31:49.263 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:49.264+0000 7f4381400640 1 -- 172.21.15.152:0/3366429577 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f437c0ccff0 msgr2=0x7f437c13a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:49.263 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:49.264+0000 7f4381400640 1 --2- 172.21.15.152:0/3366429577 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f437c0ccff0 0x7f437c13a7b0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f436c00c430 tx=0x7f436c00c8f0 comp rx=0 tx=0).stop 2024-04-20T11:31:49.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:49.264+0000 7f4381400640 1 -- 172.21.15.152:0/3366429577 shutdown_connections 2024-04-20T11:31:49.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:49.264+0000 7f4381400640 1 --2- 172.21.15.152:0/3366429577 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f435806cc50 0x7f435806f110 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:49.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:49.264+0000 7f4381400640 1 --2- 172.21.15.152:0/3366429577 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f437c0d0b80 0x7f437c0aeea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:49.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:49.264+0000 7f4381400640 1 --2- 172.21.15.152:0/3366429577 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f437c0cf1b0 0x7f437c0c63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:49.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:49.264+0000 7f4381400640 1 --2- 172.21.15.152:0/3366429577 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f437c0ccff0 0x7f437c13a7b0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:49.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:49.264+0000 7f4381400640 1 -- 172.21.15.152:0/3366429577 >> 172.21.15.152:0/3366429577 conn(0x7f437c06b980 msgr2=0x7f437c0b0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:49.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:49.264+0000 7f4381400640 1 -- 172.21.15.152:0/3366429577 shutdown_connections 2024-04-20T11:31:49.265 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:49.264+0000 7f4381400640 1 -- 172.21.15.152:0/3366429577 wait complete. 2024-04-20T11:31:49.303 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-20T11:31:49.480 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:49.480+0000 7f9e9e800640 1 Processor -- start 2024-04-20T11:31:49.480 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:49.481+0000 7f9e9e800640 1 -- start start 2024-04-20T11:31:49.482 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:49.482+0000 7f9e9e800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f9e9815e9e0 0x7f9e9806d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:49.482 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:49.482+0000 7f9e9e800640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f9e980d14a0 0x7f9e980d4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:49.482 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:49.482+0000 7f9e9e800640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e980d19a0 con 0x7f9e980d46a0 2024-04-20T11:31:49.482 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:49.482+0000 7f9e9e800640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f9e980d5040 con 0x7f9e9815e9e0 2024-04-20T11:31:49.482 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:49.482+0000 7f9e9e800640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f9e980d1c80 con 0x7f9e980d14a0 2024-04-20T11:31:49.482 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:49.482+0000 7f9e97400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f9e9815e9e0 0x7f9e9806d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:49.482 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:49.482+0000 7f9e96a00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f9e980d46a0 0x7f9e980d0d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:56390/0 (socket says 172.21.15.152:56390) 2024-04-20T11:31:49.482 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:49.482+0000 7f9e96a00640 1 -- 172.21.15.152:0/2523213957 learned_addr learned my addr 172.21.15.152:0/2523213957 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:31:49.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:49.483+0000 7f9e97e00640 1 --2- 172.21.15.152:0/2523213957 >> v2:172.21.15.175:3300/0 conn(0x7f9e980d14a0 0x7f9e980d4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:49.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:49.483+0000 7f9e96000640 1 -- 172.21.15.152:0/2523213957 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1808562819 0 0) 0x7f9e980d19a0 con 0x7f9e980d46a0 2024-04-20T11:31:49.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:49.483+0000 7f9e97400640 1 -- 172.21.15.152:0/2523213957 >> v2:172.21.15.175:3300/0 conn(0x7f9e980d14a0 msgr2=0x7f9e980d4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:49.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:49.483+0000 7f9e97400640 1 --2- 172.21.15.152:0/2523213957 >> v2:172.21.15.175:3300/0 conn(0x7f9e980d14a0 0x7f9e980d4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:49.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:49.483+0000 7f9e97400640 1 -- 172.21.15.152:0/2523213957 >> v1:172.21.15.152:6789/0 conn(0x7f9e980d46a0 legacy=0x7f9e980d0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:49.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:49.483+0000 7f9e97400640 1 -- 172.21.15.152:0/2523213957 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e980d1f60 con 0x7f9e9815e9e0 2024-04-20T11:31:49.484 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:49.485+0000 7f9e97400640 1 --2- 172.21.15.152:0/2523213957 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9e9815e9e0 0x7f9e9806d320 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f9e980cee20 tx=0x7f9e8802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdf03f8cd7fb5c5f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:49.484 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:49.485+0000 7f9e96000640 1 -- 172.21.15.152:0/2523213957 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e88035020 con 0x7f9e9815e9e0 2024-04-20T11:31:49.484 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:49.485+0000 7f9e96000640 1 -- 172.21.15.152:0/2523213957 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e88031830 con 0x7f9e9815e9e0 2024-04-20T11:31:49.485 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:49.485+0000 7f9e96000640 1 -- 172.21.15.152:0/2523213957 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e88031c00 con 0x7f9e9815e9e0 2024-04-20T11:31:49.485 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:49.486+0000 7f9e9e800640 1 -- 172.21.15.152:0/2523213957 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9e9815e9e0 msgr2=0x7f9e9806d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:49.485 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:49.486+0000 7f9e9e800640 1 --2- 172.21.15.152:0/2523213957 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9e9815e9e0 0x7f9e9806d320 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f9e980cee20 tx=0x7f9e8802b8d0 comp rx=0 tx=0).stop 2024-04-20T11:31:49.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:49.486+0000 7f9e9e800640 1 -- 172.21.15.152:0/2523213957 shutdown_connections 2024-04-20T11:31:49.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:49.486+0000 7f9e9e800640 1 --2- 172.21.15.152:0/2523213957 >> v2:172.21.15.175:3300/0 conn(0x7f9e980d14a0 0x7f9e980d4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:49.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:49.486+0000 7f9e9e800640 1 --2- 172.21.15.152:0/2523213957 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9e9815e9e0 0x7f9e9806d320 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:49.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:49.486+0000 7f9e9e800640 1 -- 172.21.15.152:0/2523213957 >> 172.21.15.152:0/2523213957 conn(0x7f9e9809dab0 msgr2=0x7f9e9808bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:49.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:49.486+0000 7f9e9e800640 1 -- 172.21.15.152:0/2523213957 shutdown_connections 2024-04-20T11:31:49.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:49.486+0000 7f9e9e800640 1 -- 172.21.15.152:0/2523213957 wait complete. 2024-04-20T11:31:49.487 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:49.488+0000 7f9e9e800640 1 Processor -- start 2024-04-20T11:31:49.487 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:49.488+0000 7f9e9e800640 1 -- start start 2024-04-20T11:31:49.489 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:49.489+0000 7f9e9e800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9e980d14a0 0x7f9e9815fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:49.489 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:49.489+0000 7f9e9e800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e980d46a0 0x7f9e980b76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:49.489 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:49.489+0000 7f9e9e800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9e9815e9e0 0x7f9e98155ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:49.489 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:49.489+0000 7f9e9e800640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f9e9806d860 con 0x7f9e9815e9e0 2024-04-20T11:31:49.489 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:49.489+0000 7f9e9e800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9e980cf880 con 0x7f9e980d46a0 2024-04-20T11:31:49.489 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:49.489+0000 7f9e9e800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f9e9806d9e0 con 0x7f9e980d14a0 2024-04-20T11:31:49.489 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:49.489+0000 7f9e97e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9e9815e9e0 0x7f9e98155ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:49.489 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:49.489+0000 7f9e97e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9e9815e9e0 0x7f9e98155ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36536/0 (socket says 172.21.15.152:36536) 2024-04-20T11:31:49.489 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:49.489+0000 7f9e97e00640 1 -- 172.21.15.152:0/933680947 learned_addr learned my addr 172.21.15.152:0/933680947 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:49.489 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:49.489+0000 7f9e97e00640 1 -- 172.21.15.152:0/933680947 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9e980d14a0 msgr2=0x7f9e9815fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:49.489 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:49.489+0000 7f9e97e00640 1 --2- 172.21.15.152:0/933680947 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9e980d14a0 0x7f9e9815fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:49.489 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:49.489+0000 7f9e97e00640 1 -- 172.21.15.152:0/933680947 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e980d46a0 msgr2=0x7f9e980b76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:49.489 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:49.489+0000 7f9e97e00640 1 --2- 172.21.15.152:0/933680947 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e980d46a0 0x7f9e980b76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:49.489 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:49.489+0000 7f9e97e00640 1 -- 172.21.15.152:0/933680947 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e981603f0 con 0x7f9e9815e9e0 2024-04-20T11:31:49.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:49.490+0000 7f9e97e00640 1 --2- 172.21.15.152:0/933680947 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9e9815e9e0 0x7f9e98155ea0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f9e90007e60 tx=0x7f9e9000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:49.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:49.490+0000 7f9e8f400640 1 -- 172.21.15.152:0/933680947 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e90014620 con 0x7f9e9815e9e0 2024-04-20T11:31:49.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:49.490+0000 7f9e8f400640 1 -- 172.21.15.152:0/933680947 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e90019070 con 0x7f9e9815e9e0 2024-04-20T11:31:49.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:49.490+0000 7f9e8f400640 1 -- 172.21.15.152:0/933680947 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e90014a00 con 0x7f9e9815e9e0 2024-04-20T11:31:49.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:49.490+0000 7f9e9e800640 1 -- 172.21.15.152:0/933680947 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e981606d0 con 0x7f9e9815e9e0 2024-04-20T11:31:49.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:49.490+0000 7f9e9e800640 1 -- 172.21.15.152:0/933680947 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e98160c90 con 0x7f9e9815e9e0 2024-04-20T11:31:49.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:49.491+0000 7f9e9e800640 1 -- 172.21.15.152:0/933680947 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e9815da90 con 0x7f9e9815e9e0 2024-04-20T11:31:49.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:49.492+0000 7f9e8f400640 1 -- 172.21.15.152:0/933680947 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 11) ==== 91346+0+0 (secure 0 0 0) 0x7f9e90014ba0 con 0x7f9e9815e9e0 2024-04-20T11:31:49.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:49.493+0000 7f9e8f400640 1 --2- 172.21.15.152:0/933680947 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f9e7806cc50 0x7f9e7806f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:49.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:49.493+0000 7f9e8f400640 1 -- 172.21.15.152:0/933680947 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5609+0+0 (secure 0 0 0) 0x7f9e9008c950 con 0x7f9e9815e9e0 2024-04-20T11:31:49.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:49.493+0000 7f9e97400640 1 --2- 172.21.15.152:0/933680947 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f9e7806cc50 0x7f9e7806f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:49.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:49.494+0000 7f9e97400640 1 --2- 172.21.15.152:0/933680947 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f9e7806cc50 0x7f9e7806f110 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f9e88036000 tx=0x7f9e88030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:49.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:49.499+0000 7f9e8f400640 1 -- 172.21.15.152:0/933680947 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f9e9815da90 con 0x7f9e9815e9e0 2024-04-20T11:31:49.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:49.737+0000 7f9e9e800640 1 -- 172.21.15.152:0/933680947 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f9e980d3ac0 con 0x7f9e9815e9e0 2024-04-20T11:31:50.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.302+0000 7f9e8f400640 1 -- 172.21.15.152:0/933680947 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 10 and data pool 11 v39) ==== 174+0+0 (secure 0 0 0) 0x7f9e980d3ac0 con 0x7f9e9815e9e0 2024-04-20T11:31:50.304 INFO:teuthology.orchestra.run.smithi152.stderr:new fs with metadata pool 10 and data pool 11 2024-04-20T11:31:50.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.310+0000 7f9e9e800640 1 -- 172.21.15.152:0/933680947 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f9e7806cc50 msgr2=0x7f9e7806f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:50.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.310+0000 7f9e9e800640 1 --2- 172.21.15.152:0/933680947 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f9e7806cc50 0x7f9e7806f110 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f9e88036000 tx=0x7f9e88030000 comp rx=0 tx=0).stop 2024-04-20T11:31:50.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.310+0000 7f9e9e800640 1 -- 172.21.15.152:0/933680947 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9e9815e9e0 msgr2=0x7f9e98155ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:50.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.310+0000 7f9e9e800640 1 --2- 172.21.15.152:0/933680947 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9e9815e9e0 0x7f9e98155ea0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f9e90007e60 tx=0x7f9e9000e4b0 comp rx=0 tx=0).stop 2024-04-20T11:31:50.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.311+0000 7f9e9e800640 1 -- 172.21.15.152:0/933680947 shutdown_connections 2024-04-20T11:31:50.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.311+0000 7f9e9e800640 1 --2- 172.21.15.152:0/933680947 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f9e7806cc50 0x7f9e7806f110 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:50.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.311+0000 7f9e9e800640 1 --2- 172.21.15.152:0/933680947 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9e9815e9e0 0x7f9e98155ea0 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:50.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.311+0000 7f9e9e800640 1 --2- 172.21.15.152:0/933680947 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e980d46a0 0x7f9e980b76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:50.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.311+0000 7f9e9e800640 1 --2- 172.21.15.152:0/933680947 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9e980d14a0 0x7f9e9815fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:50.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.311+0000 7f9e9e800640 1 -- 172.21.15.152:0/933680947 >> 172.21.15.152:0/933680947 conn(0x7f9e9809dab0 msgr2=0x7f9e98162360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:50.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.312+0000 7f9e9e800640 1 -- 172.21.15.152:0/933680947 shutdown_connections 2024-04-20T11:31:50.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.312+0000 7f9e9e800640 1 -- 172.21.15.152:0/933680947 wait complete. 2024-04-20T11:31:50.345 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:31:50.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.506+0000 7fc14a400640 1 Processor -- start 2024-04-20T11:31:50.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.507+0000 7fc14a400640 1 -- start start 2024-04-20T11:31:50.508 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.508+0000 7fc14a400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fc14415d1e0 0x7fc144160b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:50.508 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.508+0000 7fc14a400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc144071cd0 0x7fc144072160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:50.508 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.508+0000 7fc14a400640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1440ce8c0 con 0x7fc14415d5c0 2024-04-20T11:31:50.508 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.508+0000 7fc14a400640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fc1440726a0 con 0x7fc14415d1e0 2024-04-20T11:31:50.508 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.508+0000 7fc14a400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc1441600b0 con 0x7fc144071cd0 2024-04-20T11:31:50.509 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.508+0000 7fc149000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fc14415d1e0 0x7fc144160b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:50.509 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.508+0000 7fc149000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fc14415d1e0 0x7fc144160b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36546/0 (socket says 172.21.15.152:36546) 2024-04-20T11:31:50.509 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.508+0000 7fc149000640 1 -- 172.21.15.152:0/2490044949 learned_addr learned my addr 172.21.15.152:0/2490044949 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:50.509 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.509+0000 7fc149a00640 1 --2- 172.21.15.152:0/2490044949 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc144071cd0 0x7fc144072160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:50.509 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.509+0000 7fc149000640 1 -- 172.21.15.152:0/2490044949 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc144071cd0 msgr2=0x7fc144072160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:50.509 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.509+0000 7fc149000640 1 --2- 172.21.15.152:0/2490044949 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc144071cd0 0x7fc144072160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:50.509 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.509+0000 7fc149000640 1 -- 172.21.15.152:0/2490044949 >> v1:172.21.15.175:6789/0 conn(0x7fc14415d5c0 legacy=0x7fc1440cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:50.509 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.509+0000 7fc149000640 1 -- 172.21.15.152:0/2490044949 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc144160390 con 0x7fc14415d1e0 2024-04-20T11:31:50.509 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.509+0000 7fc149a00640 1 --2- 172.21.15.152:0/2490044949 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc144071cd0 0x7fc144072160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:31:50.510 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.510+0000 7fc149000640 1 --2- 172.21.15.152:0/2490044949 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc14415d1e0 0x7fc144160b90 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fc1380098a0 tx=0x7fc13802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beaebf068a469dcf server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:50.510 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.510+0000 7fc143400640 1 -- 172.21.15.152:0/2490044949 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc138035020 con 0x7fc14415d1e0 2024-04-20T11:31:50.510 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.510+0000 7fc143400640 1 -- 172.21.15.152:0/2490044949 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc138031640 con 0x7fc14415d1e0 2024-04-20T11:31:50.511 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.510+0000 7fc143400640 1 -- 172.21.15.152:0/2490044949 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc13802d040 con 0x7fc14415d1e0 2024-04-20T11:31:50.511 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.511+0000 7fc14a400640 1 -- 172.21.15.152:0/2490044949 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc14415d1e0 msgr2=0x7fc144160b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:50.511 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.511+0000 7fc14a400640 1 --2- 172.21.15.152:0/2490044949 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc14415d1e0 0x7fc144160b90 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fc1380098a0 tx=0x7fc13802b8f0 comp rx=0 tx=0).stop 2024-04-20T11:31:50.511 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.511+0000 7fc14a400640 1 -- 172.21.15.152:0/2490044949 shutdown_connections 2024-04-20T11:31:50.511 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.511+0000 7fc14a400640 1 --2- 172.21.15.152:0/2490044949 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc144071cd0 0x7fc144072160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:50.511 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.511+0000 7fc14a400640 1 --2- 172.21.15.152:0/2490044949 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc14415d1e0 0x7fc144160b90 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:50.511 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.511+0000 7fc14a400640 1 -- 172.21.15.152:0/2490044949 >> 172.21.15.152:0/2490044949 conn(0x7fc144067ae0 msgr2=0x7fc14406b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:50.511 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.511+0000 7fc14a400640 1 -- 172.21.15.152:0/2490044949 shutdown_connections 2024-04-20T11:31:50.511 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.511+0000 7fc14a400640 1 -- 172.21.15.152:0/2490044949 wait complete. 2024-04-20T11:31:50.512 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.512+0000 7fc14a400640 1 Processor -- start 2024-04-20T11:31:50.512 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.513+0000 7fc14a400640 1 -- start start 2024-04-20T11:31:50.514 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.513+0000 7fc14a400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc144071cd0 0x7fc14415e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:50.514 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.513+0000 7fc14a400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc14415d1e0 0x7fc144155870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:50.514 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.513+0000 7fc14a400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc14415d5c0 0x7fc144153bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:50.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.513+0000 7fc14a400640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fc1440ccd70 con 0x7fc14415d1e0 2024-04-20T11:31:50.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.513+0000 7fc14a400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc1440ceba0 con 0x7fc144071cd0 2024-04-20T11:31:50.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.513+0000 7fc14a400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc1440ccef0 con 0x7fc14415d5c0 2024-04-20T11:31:50.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.514+0000 7fc143e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc14415d1e0 0x7fc144155870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:50.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.514+0000 7fc143e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc14415d1e0 0x7fc144155870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36552/0 (socket says 172.21.15.152:36552) 2024-04-20T11:31:50.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.514+0000 7fc143e00640 1 -- 172.21.15.152:0/3825710202 learned_addr learned my addr 172.21.15.152:0/3825710202 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:50.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.514+0000 7fc149000640 1 --2- 172.21.15.152:0/3825710202 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc144071cd0 0x7fc14415e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:50.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.514+0000 7fc143e00640 1 -- 172.21.15.152:0/3825710202 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc14415d5c0 msgr2=0x7fc144153bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:50.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.514+0000 7fc143e00640 1 --2- 172.21.15.152:0/3825710202 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc14415d5c0 0x7fc144153bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:50.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.514+0000 7fc143e00640 1 -- 172.21.15.152:0/3825710202 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc144071cd0 msgr2=0x7fc14415e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:50.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.514+0000 7fc143e00640 1 --2- 172.21.15.152:0/3825710202 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc144071cd0 0x7fc14415e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:50.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.514+0000 7fc143e00640 1 -- 172.21.15.152:0/3825710202 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc144155fe0 con 0x7fc14415d1e0 2024-04-20T11:31:50.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.514+0000 7fc143e00640 1 --2- 172.21.15.152:0/3825710202 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc14415d1e0 0x7fc144155870 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fc13000b420 tx=0x7fc13000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:50.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.515+0000 7fc140c00640 1 -- 172.21.15.152:0/3825710202 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc130011830 con 0x7fc14415d1e0 2024-04-20T11:31:50.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.515+0000 7fc140c00640 1 -- 172.21.15.152:0/3825710202 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc130016070 con 0x7fc14415d1e0 2024-04-20T11:31:50.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.515+0000 7fc140c00640 1 -- 172.21.15.152:0/3825710202 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc130011c10 con 0x7fc14415d1e0 2024-04-20T11:31:50.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.515+0000 7fc14a400640 1 -- 172.21.15.152:0/3825710202 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1441562c0 con 0x7fc14415d1e0 2024-04-20T11:31:50.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.515+0000 7fc14a400640 1 -- 172.21.15.152:0/3825710202 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc144156880 con 0x7fc14415d1e0 2024-04-20T11:31:50.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.516+0000 7fc14a400640 1 -- 172.21.15.152:0/3825710202 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc144075e60 con 0x7fc14415d1e0 2024-04-20T11:31:50.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.517+0000 7fc140c00640 1 -- 172.21.15.152:0/3825710202 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 11) ==== 91346+0+0 (secure 0 0 0) 0x7fc13001b460 con 0x7fc14415d1e0 2024-04-20T11:31:50.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.518+0000 7fc140c00640 1 --2- 172.21.15.152:0/3825710202 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc12406cc50 0x7fc12406f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:50.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.519+0000 7fc140c00640 1 -- 172.21.15.152:0/3825710202 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7fc13008bb80 con 0x7fc14415d1e0 2024-04-20T11:31:50.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.519+0000 7fc149000640 1 --2- 172.21.15.152:0/3825710202 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc12406cc50 0x7fc12406f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:50.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.519+0000 7fc149000640 1 --2- 172.21.15.152:0/3825710202 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc12406cc50 0x7fc12406f110 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fc138036000 tx=0x7fc138008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:50.522 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.523+0000 7fc140c00640 1 -- 172.21.15.152:0/3825710202 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fc130020020 con 0x7fc14415d1e0 2024-04-20T11:31:50.718 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.718+0000 7fc14a400640 1 -- 172.21.15.152:0/3825710202 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc1440b1c10 con 0x7fc14415d1e0 2024-04-20T11:31:50.721 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.722+0000 7fc140c00640 1 -- 172.21.15.152:0/3825710202 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) ==== 74+0+16741 (secure 0 0 0) 0x7fc130059bf0 con 0x7fc14415d1e0 2024-04-20T11:31:50.721 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:31:50.722 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":70,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:31:49.738850+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:31:48.024528+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-20T11:31:48.725648+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:31:50.723 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.724+0000 7fc14a400640 1 -- 172.21.15.152:0/3825710202 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc12406cc50 msgr2=0x7fc12406f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:50.724 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.724+0000 7fc14a400640 1 --2- 172.21.15.152:0/3825710202 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc12406cc50 0x7fc12406f110 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fc138036000 tx=0x7fc138008030 comp rx=0 tx=0).stop 2024-04-20T11:31:50.724 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.724+0000 7fc14a400640 1 -- 172.21.15.152:0/3825710202 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc14415d1e0 msgr2=0x7fc144155870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:50.724 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.724+0000 7fc14a400640 1 --2- 172.21.15.152:0/3825710202 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc14415d1e0 0x7fc144155870 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fc13000b420 tx=0x7fc13000b8e0 comp rx=0 tx=0).stop 2024-04-20T11:31:50.724 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.724+0000 7fc14a400640 1 -- 172.21.15.152:0/3825710202 shutdown_connections 2024-04-20T11:31:50.724 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.724+0000 7fc14a400640 1 --2- 172.21.15.152:0/3825710202 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc12406cc50 0x7fc12406f110 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:50.724 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.724+0000 7fc14a400640 1 --2- 172.21.15.152:0/3825710202 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc14415d5c0 0x7fc144153bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:50.724 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.724+0000 7fc14a400640 1 --2- 172.21.15.152:0/3825710202 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc14415d1e0 0x7fc144155870 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:50.725 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.724+0000 7fc14a400640 1 --2- 172.21.15.152:0/3825710202 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc144071cd0 0x7fc14415e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:50.725 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.724+0000 7fc14a400640 1 -- 172.21.15.152:0/3825710202 >> 172.21.15.152:0/3825710202 conn(0x7fc144067ae0 msgr2=0x7fc1440b0980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:50.725 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.725+0000 7fc14a400640 1 -- 172.21.15.152:0/3825710202 shutdown_connections 2024-04-20T11:31:50.725 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.725+0000 7fc14a400640 1 -- 172.21.15.152:0/3825710202 wait complete. 2024-04-20T11:31:50.741 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:31:50.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.933+0000 7f7102200640 1 Processor -- start 2024-04-20T11:31:50.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.934+0000 7f7102200640 1 -- start start 2024-04-20T11:31:50.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.935+0000 7f7102200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f70fc145f60 0x7f70fc06ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:50.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.935+0000 7f7102200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f70fc0cd0d0 0x7f70fc0cd580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:50.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.935+0000 7f7102200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f70fc071fe0 0x7f70fc072470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:50.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.935+0000 7f7102200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f70fc06e9f0 con 0x7f70fc0cd0d0 2024-04-20T11:31:50.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.935+0000 7f7102200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f70fc146340 con 0x7f70fc145f60 2024-04-20T11:31:50.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.935+0000 7f7102200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f70fc06f3c0 con 0x7f70fc071fe0 2024-04-20T11:31:50.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.936+0000 7f70f2a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f70fc0cd0d0 0x7f70fc0cd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:50.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.936+0000 7f70f2a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f70fc0cd0d0 0x7f70fc0cd580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36558/0 (socket says 172.21.15.152:36558) 2024-04-20T11:31:50.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.936+0000 7f70f2a00640 1 -- 172.21.15.152:0/3107856075 learned_addr learned my addr 172.21.15.152:0/3107856075 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:50.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.936+0000 7f70fb400640 1 --2- 172.21.15.152:0/3107856075 >> v2:172.21.15.175:3300/0 conn(0x7f70fc145f60 0x7f70fc06ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:50.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.936+0000 7f70fbe00640 1 --2- 172.21.15.152:0/3107856075 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f70fc071fe0 0x7f70fc072470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:50.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.936+0000 7f70f2a00640 1 -- 172.21.15.152:0/3107856075 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f70fc071fe0 msgr2=0x7f70fc072470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:50.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.936+0000 7f70f2a00640 1 --2- 172.21.15.152:0/3107856075 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f70fc071fe0 0x7f70fc072470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:50.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.936+0000 7f70f2a00640 1 -- 172.21.15.152:0/3107856075 >> v2:172.21.15.175:3300/0 conn(0x7f70fc145f60 msgr2=0x7f70fc06ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:50.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.936+0000 7f70f2a00640 1 --2- 172.21.15.152:0/3107856075 >> v2:172.21.15.175:3300/0 conn(0x7f70fc145f60 0x7f70fc06ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:50.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.936+0000 7f70f2a00640 1 -- 172.21.15.152:0/3107856075 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70fc072d50 con 0x7f70fc0cd0d0 2024-04-20T11:31:50.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.937+0000 7f70f2a00640 1 --2- 172.21.15.152:0/3107856075 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f70fc0cd0d0 0x7f70fc0cd580 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f70fc073230 tx=0x7f70e402b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ee187afb5a15ebd server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:50.939 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.938+0000 7f70faa00640 1 -- 172.21.15.152:0/3107856075 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70e4035020 con 0x7f70fc0cd0d0 2024-04-20T11:31:50.939 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.938+0000 7f70faa00640 1 -- 172.21.15.152:0/3107856075 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70e402d040 con 0x7f70fc0cd0d0 2024-04-20T11:31:50.939 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.938+0000 7f70faa00640 1 -- 172.21.15.152:0/3107856075 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70e4031a80 con 0x7f70fc0cd0d0 2024-04-20T11:31:50.939 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.938+0000 7f7102200640 1 -- 172.21.15.152:0/3107856075 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f70fc0cd0d0 msgr2=0x7f70fc0cd580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:50.939 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.938+0000 7f7102200640 1 --2- 172.21.15.152:0/3107856075 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f70fc0cd0d0 0x7f70fc0cd580 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f70fc073230 tx=0x7f70e402b900 comp rx=0 tx=0).stop 2024-04-20T11:31:50.939 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.938+0000 7f7102200640 1 -- 172.21.15.152:0/3107856075 shutdown_connections 2024-04-20T11:31:50.939 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.938+0000 7f7102200640 1 --2- 172.21.15.152:0/3107856075 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f70fc071fe0 0x7f70fc072470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:50.939 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.938+0000 7f7102200640 1 --2- 172.21.15.152:0/3107856075 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f70fc0cd0d0 0x7f70fc0cd580 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:50.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.939+0000 7f7102200640 1 --2- 172.21.15.152:0/3107856075 >> v2:172.21.15.175:3300/0 conn(0x7f70fc145f60 0x7f70fc06ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:50.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.939+0000 7f7102200640 1 -- 172.21.15.152:0/3107856075 >> 172.21.15.152:0/3107856075 conn(0x7f70fc067cf0 msgr2=0x7f70fc06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:50.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.939+0000 7f7102200640 1 -- 172.21.15.152:0/3107856075 shutdown_connections 2024-04-20T11:31:50.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.939+0000 7f7102200640 1 -- 172.21.15.152:0/3107856075 wait complete. 2024-04-20T11:31:50.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.940+0000 7f7102200640 1 Processor -- start 2024-04-20T11:31:50.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.940+0000 7f7102200640 1 -- start start 2024-04-20T11:31:50.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.941+0000 7f7102200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f70fc071fe0 0x7f70fc13cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:50.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.941+0000 7f7102200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f70fc0cd0d0 0x7f70fc134e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:50.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.941+0000 7f7102200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f70fc145f60 0x7f70fc13d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:50.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.941+0000 7f7102200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f70fc06e680 con 0x7f70fc145f60 2024-04-20T11:31:50.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.941+0000 7f7102200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f70fc06e3a0 con 0x7f70fc071fe0 2024-04-20T11:31:50.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.941+0000 7f7102200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f70fc0729b0 con 0x7f70fc0cd0d0 2024-04-20T11:31:50.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.942+0000 7f70fbe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f70fc145f60 0x7f70fc13d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:50.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.942+0000 7f70fbe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f70fc145f60 0x7f70fc13d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36574/0 (socket says 172.21.15.152:36574) 2024-04-20T11:31:50.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.942+0000 7f70fbe00640 1 -- 172.21.15.152:0/334522052 learned_addr learned my addr 172.21.15.152:0/334522052 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:50.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.942+0000 7f70fb400640 1 --2- 172.21.15.152:0/334522052 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f70fc071fe0 0x7f70fc13cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:50.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.942+0000 7f70f2a00640 1 --2- 172.21.15.152:0/334522052 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f70fc0cd0d0 0x7f70fc134e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:50.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.942+0000 7f70fbe00640 1 -- 172.21.15.152:0/334522052 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f70fc0cd0d0 msgr2=0x7f70fc134e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:50.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.942+0000 7f70fbe00640 1 --2- 172.21.15.152:0/334522052 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f70fc0cd0d0 0x7f70fc134e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:50.944 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.942+0000 7f70fbe00640 1 -- 172.21.15.152:0/334522052 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f70fc071fe0 msgr2=0x7f70fc13cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:50.944 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.942+0000 7f70fbe00640 1 --2- 172.21.15.152:0/334522052 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f70fc071fe0 0x7f70fc13cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:50.944 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.942+0000 7f70fbe00640 1 -- 172.21.15.152:0/334522052 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70fc13d0b0 con 0x7f70fc145f60 2024-04-20T11:31:50.944 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.942+0000 7f70fbe00640 1 --2- 172.21.15.152:0/334522052 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f70fc145f60 0x7f70fc13d5e0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f70f4007e60 tx=0x7f70f400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:50.944 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.943+0000 7f70f3e00640 1 -- 172.21.15.152:0/334522052 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f4014620 con 0x7f70fc145f60 2024-04-20T11:31:50.944 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.943+0000 7f70f3e00640 1 -- 172.21.15.152:0/334522052 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70f4019070 con 0x7f70fc145f60 2024-04-20T11:31:50.944 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.943+0000 7f70f3e00640 1 -- 172.21.15.152:0/334522052 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f4014a20 con 0x7f70fc145f60 2024-04-20T11:31:50.944 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.943+0000 7f7102200640 1 -- 172.21.15.152:0/334522052 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70fc136e30 con 0x7f70fc145f60 2024-04-20T11:31:50.944 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.943+0000 7f7102200640 1 -- 172.21.15.152:0/334522052 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70fc1373f0 con 0x7f70fc145f60 2024-04-20T11:31:50.945 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.944+0000 7f7102200640 1 -- 172.21.15.152:0/334522052 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70fc0689f0 con 0x7f70fc145f60 2024-04-20T11:31:50.945 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.945+0000 7f70f3e00640 1 -- 172.21.15.152:0/334522052 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 11) ==== 91346+0+0 (secure 0 0 0) 0x7f70f4014bc0 con 0x7f70fc145f60 2024-04-20T11:31:50.945 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.946+0000 7f70f3e00640 1 --2- 172.21.15.152:0/334522052 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f70d806cc50 0x7f70d806f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:50.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.946+0000 7f70f3e00640 1 -- 172.21.15.152:0/334522052 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7f70f408c8f0 con 0x7f70fc145f60 2024-04-20T11:31:50.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.946+0000 7f70fb400640 1 --2- 172.21.15.152:0/334522052 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f70d806cc50 0x7f70d806f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:50.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.947+0000 7f70fb400640 1 --2- 172.21.15.152:0/334522052 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f70d806cc50 0x7f70d806f110 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f70ec009970 tx=0x7f70ec008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:50.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:50.950+0000 7f70f3e00640 1 -- 172.21.15.152:0/334522052 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f70f405a960 con 0x7f70fc145f60 2024-04-20T11:31:51.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:51.155+0000 7f7102200640 1 -- 172.21.15.152:0/334522052 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f70fc06f690 con 0x7f70fc145f60 2024-04-20T11:31:51.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:51.159+0000 7f70f3e00640 1 -- 172.21.15.152:0/334522052 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) ==== 74+0+16741 (secure 0 0 0) 0x7f70f405a260 con 0x7f70fc145f60 2024-04-20T11:31:51.159 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:31:51.159 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":70,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:31:49.738850+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:31:48.024528+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-20T11:31:48.725648+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:31:51.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:51.161+0000 7f7102200640 1 -- 172.21.15.152:0/334522052 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f70d806cc50 msgr2=0x7f70d806f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:51.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:51.161+0000 7f7102200640 1 --2- 172.21.15.152:0/334522052 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f70d806cc50 0x7f70d806f110 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f70ec009970 tx=0x7f70ec008040 comp rx=0 tx=0).stop 2024-04-20T11:31:51.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:51.161+0000 7f7102200640 1 -- 172.21.15.152:0/334522052 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f70fc145f60 msgr2=0x7f70fc13d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:51.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:51.161+0000 7f7102200640 1 --2- 172.21.15.152:0/334522052 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f70fc145f60 0x7f70fc13d5e0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f70f4007e60 tx=0x7f70f400e4b0 comp rx=0 tx=0).stop 2024-04-20T11:31:51.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:51.161+0000 7f7102200640 1 -- 172.21.15.152:0/334522052 shutdown_connections 2024-04-20T11:31:51.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:51.161+0000 7f7102200640 1 --2- 172.21.15.152:0/334522052 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f70d806cc50 0x7f70d806f110 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:51.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:51.161+0000 7f7102200640 1 --2- 172.21.15.152:0/334522052 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f70fc145f60 0x7f70fc13d5e0 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:51.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:51.161+0000 7f7102200640 1 --2- 172.21.15.152:0/334522052 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f70fc0cd0d0 0x7f70fc134e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:51.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:51.161+0000 7f7102200640 1 --2- 172.21.15.152:0/334522052 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f70fc071fe0 0x7f70fc13cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:51.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:51.161+0000 7f7102200640 1 -- 172.21.15.152:0/334522052 >> 172.21.15.152:0/334522052 conn(0x7f70fc067cf0 msgr2=0x7f70fc0af710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:51.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:51.161+0000 7f7102200640 1 -- 172.21.15.152:0/334522052 shutdown_connections 2024-04-20T11:31:51.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:51.162+0000 7f7102200640 1 -- 172.21.15.152:0/334522052 wait complete. 2024-04-20T11:31:51.180 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-20T11:31:51.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:51.382+0000 7f3d38000640 1 Processor -- start 2024-04-20T11:31:51.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:51.383+0000 7f3d38000640 1 -- start start 2024-04-20T11:31:51.384 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:51.384+0000 7f3d38000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3d3015e5d0 0x7f3d30162440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:51.384 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:51.384+0000 7f3d38000640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f3d300b3fa0 0x7f3d300af490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:51.384 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:51.384+0000 7f3d38000640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d300adc30 con 0x7f3d3015e9b0 2024-04-20T11:31:51.384 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:51.384+0000 7f3d38000640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f3d30162a70 con 0x7f3d300b3fa0 2024-04-20T11:31:51.384 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:51.384+0000 7f3d38000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f3d30162d50 con 0x7f3d3015e5d0 2024-04-20T11:31:51.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:51.385+0000 7f3d35600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3d3015e5d0 0x7f3d30162440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:51.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:51.385+0000 7f3d36000640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7f3d3015e9b0 0x7f3d30071db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.152:60320/0 (socket says 172.21.15.152:60320) 2024-04-20T11:31:51.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:51.385+0000 7f3d36000640 1 -- 172.21.15.152:0/1390715607 learned_addr learned my addr 172.21.15.152:0/1390715607 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:31:51.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:51.385+0000 7f3d34c00640 1 --2- 172.21.15.152:0/1390715607 >> v2:172.21.15.175:3300/0 conn(0x7f3d300b3fa0 0x7f3d300af490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:51.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:51.386+0000 7f3d35600640 1 -- 172.21.15.152:0/1390715607 >> v2:172.21.15.175:3300/0 conn(0x7f3d300b3fa0 msgr2=0x7f3d300af490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:51.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:51.386+0000 7f3d35600640 1 --2- 172.21.15.152:0/1390715607 >> v2:172.21.15.175:3300/0 conn(0x7f3d300b3fa0 0x7f3d300af490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:51.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:51.386+0000 7f3d35600640 1 -- 172.21.15.152:0/1390715607 >> v1:172.21.15.175:6789/0 conn(0x7f3d3015e9b0 legacy=0x7f3d30071db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:51.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:51.386+0000 7f3d35600640 1 -- 172.21.15.152:0/1390715607 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d30163030 con 0x7f3d3015e5d0 2024-04-20T11:31:51.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:51.387+0000 7f3d35600640 1 --2- 172.21.15.152:0/1390715607 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3d3015e5d0 0x7f3d30162440 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3d240097a0 tx=0x7f3d2402b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=813239570bde6837 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:51.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:51.388+0000 7f3d2fe00640 1 -- 172.21.15.152:0/1390715607 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d24035020 con 0x7f3d3015e5d0 2024-04-20T11:31:51.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:51.388+0000 7f3d2fe00640 1 -- 172.21.15.152:0/1390715607 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d24031810 con 0x7f3d3015e5d0 2024-04-20T11:31:51.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:51.388+0000 7f3d2fe00640 1 -- 172.21.15.152:0/1390715607 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d24031be0 con 0x7f3d3015e5d0 2024-04-20T11:31:51.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:51.388+0000 7f3d38000640 1 -- 172.21.15.152:0/1390715607 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3d3015e5d0 msgr2=0x7f3d30162440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:51.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:51.388+0000 7f3d38000640 1 --2- 172.21.15.152:0/1390715607 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3d3015e5d0 0x7f3d30162440 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3d240097a0 tx=0x7f3d2402b8b0 comp rx=0 tx=0).stop 2024-04-20T11:31:51.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:51.388+0000 7f3d38000640 1 -- 172.21.15.152:0/1390715607 shutdown_connections 2024-04-20T11:31:51.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:51.388+0000 7f3d38000640 1 --2- 172.21.15.152:0/1390715607 >> v2:172.21.15.175:3300/0 conn(0x7f3d300b3fa0 0x7f3d300af490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:51.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:51.388+0000 7f3d38000640 1 --2- 172.21.15.152:0/1390715607 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3d3015e5d0 0x7f3d30162440 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:51.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:51.388+0000 7f3d38000640 1 -- 172.21.15.152:0/1390715607 >> 172.21.15.152:0/1390715607 conn(0x7f3d3006b980 msgr2=0x7f3d3006bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:51.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:51.388+0000 7f3d38000640 1 -- 172.21.15.152:0/1390715607 shutdown_connections 2024-04-20T11:31:51.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:51.389+0000 7f3d38000640 1 -- 172.21.15.152:0/1390715607 wait complete. 2024-04-20T11:31:51.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:51.390+0000 7f3d38000640 1 Processor -- start 2024-04-20T11:31:51.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:51.390+0000 7f3d38000640 1 -- start start 2024-04-20T11:31:51.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:51.391+0000 7f3d38000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3d300b3fa0 0x7f3d30160070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:51.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:51.391+0000 7f3d38000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3d3015e5d0 0x7f3d30161720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:51.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:51.391+0000 7f3d38000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3d3015e9b0 0x7f3d30155ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:51.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:51.391+0000 7f3d38000640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f3d3006e0a0 con 0x7f3d300b3fa0 2024-04-20T11:31:51.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:51.391+0000 7f3d38000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3d30163310 con 0x7f3d3015e5d0 2024-04-20T11:31:51.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:51.391+0000 7f3d38000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f3d3006e220 con 0x7f3d3015e9b0 2024-04-20T11:31:51.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:51.392+0000 7f3d35600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3d300b3fa0 0x7f3d30160070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:51.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:51.392+0000 7f3d35600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3d300b3fa0 0x7f3d30160070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36584/0 (socket says 172.21.15.152:36584) 2024-04-20T11:31:51.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:51.392+0000 7f3d35600640 1 -- 172.21.15.152:0/3239368087 learned_addr learned my addr 172.21.15.152:0/3239368087 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:51.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:51.392+0000 7f3d34c00640 1 --2- 172.21.15.152:0/3239368087 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3d3015e5d0 0x7f3d30161720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:51.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:51.392+0000 7f3d36000640 1 --2- 172.21.15.152:0/3239368087 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3d3015e9b0 0x7f3d30155ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:51.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:51.392+0000 7f3d35600640 1 -- 172.21.15.152:0/3239368087 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3d3015e9b0 msgr2=0x7f3d30155ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:51.393 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:51.392+0000 7f3d35600640 1 --2- 172.21.15.152:0/3239368087 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3d3015e9b0 0x7f3d30155ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:51.393 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:51.392+0000 7f3d35600640 1 -- 172.21.15.152:0/3239368087 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3d3015e5d0 msgr2=0x7f3d30161720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:51.393 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:51.392+0000 7f3d35600640 1 --2- 172.21.15.152:0/3239368087 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3d3015e5d0 0x7f3d30161720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:51.393 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:51.392+0000 7f3d35600640 1 -- 172.21.15.152:0/3239368087 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d30160680 con 0x7f3d300b3fa0 2024-04-20T11:31:51.393 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:51.393+0000 7f3d35600640 1 --2- 172.21.15.152:0/3239368087 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3d300b3fa0 0x7f3d30160070 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f3d2402bec0 tx=0x7f3d240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:51.394 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:51.393+0000 7f3d2d600640 1 -- 172.21.15.152:0/3239368087 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d24041070 con 0x7f3d300b3fa0 2024-04-20T11:31:51.394 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:51.393+0000 7f3d2d600640 1 -- 172.21.15.152:0/3239368087 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d2403ad00 con 0x7f3d300b3fa0 2024-04-20T11:31:51.394 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:51.393+0000 7f3d2d600640 1 -- 172.21.15.152:0/3239368087 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d24035040 con 0x7f3d300b3fa0 2024-04-20T11:31:51.394 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:51.393+0000 7f3d38000640 1 -- 172.21.15.152:0/3239368087 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d30160960 con 0x7f3d300b3fa0 2024-04-20T11:31:51.394 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:51.393+0000 7f3d38000640 1 -- 172.21.15.152:0/3239368087 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d30160f20 con 0x7f3d300b3fa0 2024-04-20T11:31:51.394 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:51.394+0000 7f3d38000640 1 -- 172.21.15.152:0/3239368087 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d300689f0 con 0x7f3d300b3fa0 2024-04-20T11:31:51.394 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:51.395+0000 7f3d2d600640 1 -- 172.21.15.152:0/3239368087 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 11) ==== 91346+0+0 (secure 0 0 0) 0x7f3d24038020 con 0x7f3d300b3fa0 2024-04-20T11:31:51.395 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:51.396+0000 7f3d2d600640 1 --2- 172.21.15.152:0/3239368087 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f3d1406cc50 0x7f3d1406f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:51.396 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:51.396+0000 7f3d2d600640 1 -- 172.21.15.152:0/3239368087 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5709+0+0 (secure 0 0 0) 0x7f3d240add20 con 0x7f3d300b3fa0 2024-04-20T11:31:51.396 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:51.397+0000 7f3d34c00640 1 --2- 172.21.15.152:0/3239368087 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f3d1406cc50 0x7f3d1406f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:51.396 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:51.397+0000 7f3d34c00640 1 --2- 172.21.15.152:0/3239368087 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f3d1406cc50 0x7f3d1406f110 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f3d301605b0 tx=0x7f3d20008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:51.399 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:51.400+0000 7f3d2d600640 1 -- 172.21.15.152:0/3239368087 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f3d2407bd00 con 0x7f3d300b3fa0 2024-04-20T11:31:51.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:51.641+0000 7f3d38000640 1 -- 172.21.15.152:0/3239368087 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f3d3006f960 con 0x7f3d300b3fa0 2024-04-20T11:31:52.297 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:31:52.305 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:31:52.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:52.322+0000 7f3d2d600640 1 -- 172.21.15.152:0/3239368087 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v42) ==== 118+0+0 (secure 0 0 0) 0x7f3d2407b600 con 0x7f3d300b3fa0 2024-04-20T11:31:52.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:52.326+0000 7f3d38000640 1 -- 172.21.15.152:0/3239368087 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f3d1406cc50 msgr2=0x7f3d1406f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:52.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:52.326+0000 7f3d38000640 1 --2- 172.21.15.152:0/3239368087 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f3d1406cc50 0x7f3d1406f110 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f3d301605b0 tx=0x7f3d20008040 comp rx=0 tx=0).stop 2024-04-20T11:31:52.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:52.326+0000 7f3d38000640 1 -- 172.21.15.152:0/3239368087 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3d300b3fa0 msgr2=0x7f3d30160070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:52.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:52.326+0000 7f3d38000640 1 --2- 172.21.15.152:0/3239368087 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3d300b3fa0 0x7f3d30160070 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f3d2402bec0 tx=0x7f3d240046c0 comp rx=0 tx=0).stop 2024-04-20T11:31:52.326 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:52.326+0000 7f3d38000640 1 -- 172.21.15.152:0/3239368087 shutdown_connections 2024-04-20T11:31:52.326 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:52.326+0000 7f3d38000640 1 --2- 172.21.15.152:0/3239368087 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f3d1406cc50 0x7f3d1406f110 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:52.326 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:52.326+0000 7f3d38000640 1 --2- 172.21.15.152:0/3239368087 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3d3015e9b0 0x7f3d30155ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:52.326 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:52.326+0000 7f3d38000640 1 --2- 172.21.15.152:0/3239368087 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3d3015e5d0 0x7f3d30161720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:52.327 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:52.326+0000 7f3d38000640 1 --2- 172.21.15.152:0/3239368087 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3d300b3fa0 0x7f3d30160070 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:52.327 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:52.326+0000 7f3d38000640 1 -- 172.21.15.152:0/3239368087 >> 172.21.15.152:0/3239368087 conn(0x7f3d3006b980 msgr2=0x7f3d300b04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:52.327 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:52.326+0000 7f3d38000640 1 -- 172.21.15.152:0/3239368087 shutdown_connections 2024-04-20T11:31:52.327 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:52.326+0000 7f3d38000640 1 -- 172.21.15.152:0/3239368087 wait complete. 2024-04-20T11:31:52.347 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-20T11:31:52.347 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-20T11:31:52.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:52.595+0000 7fe13be00640 1 Processor -- start 2024-04-20T11:31:52.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:52.596+0000 7fe13be00640 1 -- start start 2024-04-20T11:31:52.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:52.597+0000 7fe13be00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe13415e890 0x7fe1340d27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:52.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:52.597+0000 7fe13be00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fe1340d2d20 0x7fe1340b7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:52.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:52.597+0000 7fe13be00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fe1340b7cc0 0x7fe1340d1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:52.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:52.597+0000 7fe13be00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fe1340b8e20 con 0x7fe1340b7cc0 2024-04-20T11:31:52.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:52.597+0000 7fe13be00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fe13415ec70 con 0x7fe1340d2d20 2024-04-20T11:31:52.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:52.597+0000 7fe13be00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe1340b8210 con 0x7fe13415e890 2024-04-20T11:31:52.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:52.598+0000 7fe139e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fe1340b7cc0 0x7fe1340d1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:52.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:52.598+0000 7fe139e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fe1340b7cc0 0x7fe1340d1790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36594/0 (socket says 172.21.15.152:36594) 2024-04-20T11:31:52.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:52.598+0000 7fe139e00640 1 -- 172.21.15.152:0/3814827615 learned_addr learned my addr 172.21.15.152:0/3814827615 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:52.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:52.598+0000 7fe138a00640 1 --2- 172.21.15.152:0/3814827615 >> v2:172.21.15.175:3300/0 conn(0x7fe1340d2d20 0x7fe1340b7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:52.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:52.598+0000 7fe139e00640 1 -- 172.21.15.152:0/3814827615 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe13415e890 msgr2=0x7fe1340d27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:52.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:52.598+0000 7fe139400640 1 --2- 172.21.15.152:0/3814827615 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe13415e890 0x7fe1340d27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:52.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:52.598+0000 7fe139e00640 1 --2- 172.21.15.152:0/3814827615 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe13415e890 0x7fe1340d27e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:52.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:52.598+0000 7fe139e00640 1 -- 172.21.15.152:0/3814827615 >> v2:172.21.15.175:3300/0 conn(0x7fe1340d2d20 msgr2=0x7fe1340b7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:52.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:52.598+0000 7fe139e00640 1 --2- 172.21.15.152:0/3814827615 >> v2:172.21.15.175:3300/0 conn(0x7fe1340d2d20 0x7fe1340b7780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:52.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:52.598+0000 7fe139e00640 1 -- 172.21.15.152:0/3814827615 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1340b84f0 con 0x7fe1340b7cc0 2024-04-20T11:31:52.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:52.598+0000 7fe138a00640 1 --2- 172.21.15.152:0/3814827615 >> v2:172.21.15.175:3300/0 conn(0x7fe1340d2d20 0x7fe1340b7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:31:52.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:52.599+0000 7fe139e00640 1 --2- 172.21.15.152:0/3814827615 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe1340b7cc0 0x7fe1340d1790 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7fe1340d57d0 tx=0x7fe12c02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4649dda91dc8dc8c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:52.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:52.600+0000 7fe133e00640 1 -- 172.21.15.152:0/3814827615 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe12c02cb00 con 0x7fe1340b7cc0 2024-04-20T11:31:52.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:52.600+0000 7fe133e00640 1 -- 172.21.15.152:0/3814827615 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe12c02cca0 con 0x7fe1340b7cc0 2024-04-20T11:31:52.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:52.600+0000 7fe133e00640 1 -- 172.21.15.152:0/3814827615 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe12c034610 con 0x7fe1340b7cc0 2024-04-20T11:31:52.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:52.600+0000 7fe13be00640 1 -- 172.21.15.152:0/3814827615 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe1340b7cc0 msgr2=0x7fe1340d1790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:52.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:52.600+0000 7fe13be00640 1 --2- 172.21.15.152:0/3814827615 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe1340b7cc0 0x7fe1340d1790 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7fe1340d57d0 tx=0x7fe12c02bf60 comp rx=0 tx=0).stop 2024-04-20T11:31:52.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:52.600+0000 7fe13be00640 1 -- 172.21.15.152:0/3814827615 shutdown_connections 2024-04-20T11:31:52.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:52.600+0000 7fe13be00640 1 --2- 172.21.15.152:0/3814827615 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe1340b7cc0 0x7fe1340d1790 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:52.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:52.600+0000 7fe13be00640 1 --2- 172.21.15.152:0/3814827615 >> v2:172.21.15.175:3300/0 conn(0x7fe1340d2d20 0x7fe1340b7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:52.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:52.600+0000 7fe13be00640 1 --2- 172.21.15.152:0/3814827615 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe13415e890 0x7fe1340d27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:52.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:52.600+0000 7fe13be00640 1 -- 172.21.15.152:0/3814827615 >> 172.21.15.152:0/3814827615 conn(0x7fe134067ae0 msgr2=0x7fe134067ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:52.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:52.600+0000 7fe13be00640 1 -- 172.21.15.152:0/3814827615 shutdown_connections 2024-04-20T11:31:52.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:52.600+0000 7fe13be00640 1 -- 172.21.15.152:0/3814827615 wait complete. 2024-04-20T11:31:52.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:52.602+0000 7fe13be00640 1 Processor -- start 2024-04-20T11:31:52.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:52.602+0000 7fe13be00640 1 -- start start 2024-04-20T11:31:52.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:52.603+0000 7fe13be00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1340b7cc0 0x7fe1341f8150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:52.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:52.603+0000 7fe13be00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe13415e890 0x7fe1341606e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:52.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:52.603+0000 7fe13be00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe1340d3620 0x7fe1341f34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:52.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:52.603+0000 7fe13be00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fe1340ba430 con 0x7fe13415e890 2024-04-20T11:31:52.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:52.603+0000 7fe13be00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe1340b8ab0 con 0x7fe1340b7cc0 2024-04-20T11:31:52.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:52.603+0000 7fe13be00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe1340ba5b0 con 0x7fe1340d3620 2024-04-20T11:31:52.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:52.603+0000 7fe138a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe13415e890 0x7fe1341606e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:52.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:52.604+0000 7fe138a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe13415e890 0x7fe1341606e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36600/0 (socket says 172.21.15.152:36600) 2024-04-20T11:31:52.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:52.604+0000 7fe138a00640 1 -- 172.21.15.152:0/3837047918 learned_addr learned my addr 172.21.15.152:0/3837047918 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:52.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:52.604+0000 7fe139400640 1 --2- 172.21.15.152:0/3837047918 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1340b7cc0 0x7fe1341f8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:52.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:52.604+0000 7fe138a00640 1 -- 172.21.15.152:0/3837047918 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe1340d3620 msgr2=0x7fe1341f34f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:52.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:52.604+0000 7fe138a00640 1 --2- 172.21.15.152:0/3837047918 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe1340d3620 0x7fe1341f34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:52.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:52.604+0000 7fe138a00640 1 -- 172.21.15.152:0/3837047918 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1340b7cc0 msgr2=0x7fe1341f8150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:52.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:52.604+0000 7fe138a00640 1 --2- 172.21.15.152:0/3837047918 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1340b7cc0 0x7fe1341f8150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:52.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:52.604+0000 7fe138a00640 1 -- 172.21.15.152:0/3837047918 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe134160e70 con 0x7fe13415e890 2024-04-20T11:31:52.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:52.604+0000 7fe139400640 1 --2- 172.21.15.152:0/3837047918 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1340b7cc0 0x7fe1341f8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:31:52.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:52.604+0000 7fe138a00640 1 --2- 172.21.15.152:0/3837047918 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe13415e890 0x7fe1341606e0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fe12000c7f0 tx=0x7fe12000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:52.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:52.604+0000 7fe131600640 1 -- 172.21.15.152:0/3837047918 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe120015070 con 0x7fe13415e890 2024-04-20T11:31:52.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:52.605+0000 7fe131600640 1 -- 172.21.15.152:0/3837047918 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe120007820 con 0x7fe13415e890 2024-04-20T11:31:52.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:52.605+0000 7fe131600640 1 -- 172.21.15.152:0/3837047918 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe120007c10 con 0x7fe13415e890 2024-04-20T11:31:52.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:52.605+0000 7fe13be00640 1 -- 172.21.15.152:0/3837047918 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe134161150 con 0x7fe13415e890 2024-04-20T11:31:52.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:52.605+0000 7fe13be00640 1 -- 172.21.15.152:0/3837047918 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe134161710 con 0x7fe13415e890 2024-04-20T11:31:52.607 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:52.606+0000 7fe13be00640 1 -- 172.21.15.152:0/3837047918 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1340689f0 con 0x7fe13415e890 2024-04-20T11:31:52.607 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:52.607+0000 7fe131600640 1 -- 172.21.15.152:0/3837047918 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 11) ==== 91346+0+0 (secure 0 0 0) 0x7fe120018070 con 0x7fe13415e890 2024-04-20T11:31:52.607 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:52.608+0000 7fe131600640 1 --2- 172.21.15.152:0/3837047918 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe11406cc50 0x7fe11406f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:52.607 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:52.608+0000 7fe131600640 1 -- 172.21.15.152:0/3837047918 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5709+0+0 (secure 0 0 0) 0x7fe12008c3a0 con 0x7fe13415e890 2024-04-20T11:31:52.608 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:52.608+0000 7fe139400640 1 --2- 172.21.15.152:0/3837047918 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe11406cc50 0x7fe11406f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:52.608 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:52.609+0000 7fe139400640 1 --2- 172.21.15.152:0/3837047918 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe11406cc50 0x7fe11406f110 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fe128008890 tx=0x7fe128008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:52.614 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:52.615+0000 7fe131600640 1 -- 172.21.15.152:0/3837047918 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe12005a1f0 con 0x7fe13415e890 2024-04-20T11:31:52.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:52.856+0000 7fe13be00640 1 -- 172.21.15.152:0/3837047918 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fe1340b9bc0 con 0x7fe13415e890 2024-04-20T11:31:53.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:53.334+0000 7fe131600640 1 -- 172.21.15.152:0/3837047918 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v43) ==== 122+0+0 (secure 0 0 0) 0x7fe120059af0 con 0x7fe13415e890 2024-04-20T11:31:53.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:53.338+0000 7fe13be00640 1 -- 172.21.15.152:0/3837047918 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe11406cc50 msgr2=0x7fe11406f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:53.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:53.338+0000 7fe13be00640 1 --2- 172.21.15.152:0/3837047918 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe11406cc50 0x7fe11406f110 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fe128008890 tx=0x7fe128008040 comp rx=0 tx=0).stop 2024-04-20T11:31:53.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:53.338+0000 7fe13be00640 1 -- 172.21.15.152:0/3837047918 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe13415e890 msgr2=0x7fe1341606e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:53.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:53.338+0000 7fe13be00640 1 --2- 172.21.15.152:0/3837047918 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe13415e890 0x7fe1341606e0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fe12000c7f0 tx=0x7fe12000ccb0 comp rx=0 tx=0).stop 2024-04-20T11:31:53.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:53.338+0000 7fe13be00640 1 -- 172.21.15.152:0/3837047918 shutdown_connections 2024-04-20T11:31:53.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:53.338+0000 7fe13be00640 1 --2- 172.21.15.152:0/3837047918 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe11406cc50 0x7fe11406f110 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:53.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:53.338+0000 7fe13be00640 1 --2- 172.21.15.152:0/3837047918 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe1340d3620 0x7fe1341f34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:53.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:53.338+0000 7fe13be00640 1 --2- 172.21.15.152:0/3837047918 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe13415e890 0x7fe1341606e0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:53.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:53.338+0000 7fe13be00640 1 --2- 172.21.15.152:0/3837047918 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1340b7cc0 0x7fe1341f8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:53.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:53.339+0000 7fe13be00640 1 -- 172.21.15.152:0/3837047918 >> 172.21.15.152:0/3837047918 conn(0x7fe134067ae0 msgr2=0x7fe1340cf520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:53.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:53.339+0000 7fe13be00640 1 -- 172.21.15.152:0/3837047918 shutdown_connections 2024-04-20T11:31:53.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:53.339+0000 7fe13be00640 1 -- 172.21.15.152:0/3837047918 wait complete. 2024-04-20T11:31:53.359 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-20T11:31:53.360 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:31:53.559 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:53.560+0000 7fd35d000640 1 Processor -- start 2024-04-20T11:31:53.559 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:53.560+0000 7fd35d000640 1 -- start start 2024-04-20T11:31:53.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:53.561+0000 7fd35d000640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fd35815b5e0 0x7fd35806eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:53.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:53.561+0000 7fd35d000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd35806fa50 0x7fd3580b5100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:53.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:53.561+0000 7fd35d000640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3580b5770 con 0x7fd3580b4c50 2024-04-20T11:31:53.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:53.561+0000 7fd35d000640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fd35806de10 con 0x7fd35815b5e0 2024-04-20T11:31:53.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:53.561+0000 7fd35d000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd35806e060 con 0x7fd35806fa50 2024-04-20T11:31:53.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:53.562+0000 7fd355e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fd35815b5e0 0x7fd35806eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:53.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:53.562+0000 7fd356800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd35806fa50 0x7fd3580b5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:53.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:53.562+0000 7fd356800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd35806fa50 0x7fd3580b5100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.152:56472/0 (socket says 172.21.15.152:56472) 2024-04-20T11:31:53.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:53.562+0000 7fd356800640 1 -- 172.21.15.152:0/1310472051 learned_addr learned my addr 172.21.15.152:0/1310472051 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:53.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:53.563+0000 7fd356800640 1 -- 172.21.15.152:0/1310472051 >> v2:172.21.15.175:3300/0 conn(0x7fd35815b5e0 msgr2=0x7fd35806eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:53.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:53.563+0000 7fd356800640 1 --2- 172.21.15.152:0/1310472051 >> v2:172.21.15.175:3300/0 conn(0x7fd35815b5e0 0x7fd35806eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:53.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:53.563+0000 7fd356800640 1 -- 172.21.15.152:0/1310472051 >> v1:172.21.15.175:6789/0 conn(0x7fd3580b4c50 legacy=0x7fd35806f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:53.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:53.563+0000 7fd356800640 1 -- 172.21.15.152:0/1310472051 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd35806e340 con 0x7fd35806fa50 2024-04-20T11:31:53.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:53.564+0000 7fd356800640 1 --2- 172.21.15.152:0/1310472051 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd35806fa50 0x7fd3580b5100 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd350009e50 tx=0x7fd35002bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=379dd8ab5981206c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:53.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:53.565+0000 7fd355400640 1 -- 172.21.15.152:0/1310472051 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd35002ca70 con 0x7fd35806fa50 2024-04-20T11:31:53.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:53.565+0000 7fd355400640 1 -- 172.21.15.152:0/1310472051 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd35002cc10 con 0x7fd35806fa50 2024-04-20T11:31:53.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:53.565+0000 7fd355400640 1 -- 172.21.15.152:0/1310472051 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3500343f0 con 0x7fd35806fa50 2024-04-20T11:31:53.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:53.565+0000 7fd35d000640 1 -- 172.21.15.152:0/1310472051 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd35806fa50 msgr2=0x7fd3580b5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:53.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:53.565+0000 7fd35d000640 1 --2- 172.21.15.152:0/1310472051 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd35806fa50 0x7fd3580b5100 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd350009e50 tx=0x7fd35002bed0 comp rx=0 tx=0).stop 2024-04-20T11:31:53.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:53.566+0000 7fd35d000640 1 -- 172.21.15.152:0/1310472051 shutdown_connections 2024-04-20T11:31:53.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:53.566+0000 7fd35d000640 1 --2- 172.21.15.152:0/1310472051 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd35806fa50 0x7fd3580b5100 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:53.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:53.566+0000 7fd35d000640 1 --2- 172.21.15.152:0/1310472051 >> v2:172.21.15.175:3300/0 conn(0x7fd35815b5e0 0x7fd35806eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:53.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:53.566+0000 7fd35d000640 1 -- 172.21.15.152:0/1310472051 >> 172.21.15.152:0/1310472051 conn(0x7fd358067cf0 msgr2=0x7fd35806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:53.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:53.566+0000 7fd35d000640 1 -- 172.21.15.152:0/1310472051 shutdown_connections 2024-04-20T11:31:53.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:53.566+0000 7fd35d000640 1 -- 172.21.15.152:0/1310472051 wait complete. 2024-04-20T11:31:53.567 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:53.567+0000 7fd35d000640 1 Processor -- start 2024-04-20T11:31:53.567 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:53.568+0000 7fd35d000640 1 -- start start 2024-04-20T11:31:53.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:53.568+0000 7fd35d000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd35806fa50 0x7fd35815a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:53.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:53.568+0000 7fd35d000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd3580b4c50 0x7fd3580a95c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:53.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:53.568+0000 7fd35d000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd35815b5e0 0x7fd3580b0540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:53.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:53.568+0000 7fd35d000640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fd3580b42d0 con 0x7fd35806fa50 2024-04-20T11:31:53.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:53.568+0000 7fd35d000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd35815c9d0 con 0x7fd35815b5e0 2024-04-20T11:31:53.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:53.568+0000 7fd35d000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd35815cd70 con 0x7fd3580b4c50 2024-04-20T11:31:53.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:53.569+0000 7fd355e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd35806fa50 0x7fd35815a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:53.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:53.569+0000 7fd355e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd35806fa50 0x7fd35815a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36608/0 (socket says 172.21.15.152:36608) 2024-04-20T11:31:53.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:53.569+0000 7fd355e00640 1 -- 172.21.15.152:0/1696102095 learned_addr learned my addr 172.21.15.152:0/1696102095 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:53.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:53.569+0000 7fd355e00640 1 -- 172.21.15.152:0/1696102095 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd3580b4c50 msgr2=0x7fd3580a95c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:53.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:53.569+0000 7fd355e00640 1 --2- 172.21.15.152:0/1696102095 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd3580b4c50 0x7fd3580a95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:53.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:53.569+0000 7fd355e00640 1 -- 172.21.15.152:0/1696102095 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd35815b5e0 msgr2=0x7fd3580b0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:53.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:53.569+0000 7fd355e00640 1 --2- 172.21.15.152:0/1696102095 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd35815b5e0 0x7fd3580b0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:53.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:53.569+0000 7fd355e00640 1 -- 172.21.15.152:0/1696102095 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd35814c050 con 0x7fd35806fa50 2024-04-20T11:31:53.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:53.570+0000 7fd355e00640 1 --2- 172.21.15.152:0/1696102095 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd35806fa50 0x7fd35815a9a0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fd34800e8a0 tx=0x7fd34800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:53.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:53.570+0000 7fd34e000640 1 -- 172.21.15.152:0/1696102095 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd348017070 con 0x7fd35806fa50 2024-04-20T11:31:53.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:53.570+0000 7fd34e000640 1 -- 172.21.15.152:0/1696102095 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd34800c940 con 0x7fd35806fa50 2024-04-20T11:31:53.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:53.570+0000 7fd34e000640 1 -- 172.21.15.152:0/1696102095 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd34800cd30 con 0x7fd35806fa50 2024-04-20T11:31:53.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:53.570+0000 7fd35d000640 1 -- 172.21.15.152:0/1696102095 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd35814c330 con 0x7fd35806fa50 2024-04-20T11:31:53.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:53.570+0000 7fd35d000640 1 -- 172.21.15.152:0/1696102095 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3580ce630 con 0x7fd35806fa50 2024-04-20T11:31:53.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:53.571+0000 7fd35d000640 1 -- 172.21.15.152:0/1696102095 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3580d0f10 con 0x7fd35806fa50 2024-04-20T11:31:53.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:53.572+0000 7fd34e000640 1 -- 172.21.15.152:0/1696102095 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 11) ==== 91346+0+0 (secure 0 0 0) 0x7fd34801a070 con 0x7fd35806fa50 2024-04-20T11:31:53.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:53.573+0000 7fd34e000640 1 --2- 172.21.15.152:0/1696102095 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fd33406cc50 0x7fd33406f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:53.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:53.573+0000 7fd34e000640 1 -- 172.21.15.152:0/1696102095 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5709+0+0 (secure 0 0 0) 0x7fd34808bf20 con 0x7fd35806fa50 2024-04-20T11:31:53.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:53.573+0000 7fd34fe00640 1 --2- 172.21.15.152:0/1696102095 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fd33406cc50 0x7fd33406f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:53.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:53.574+0000 7fd34fe00640 1 --2- 172.21.15.152:0/1696102095 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fd33406cc50 0x7fd33406f110 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fd3580af350 tx=0x7fd340007970 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:53.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:53.577+0000 7fd34e000640 1 -- 172.21.15.152:0/1696102095 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fd3580d0f10 con 0x7fd35806fa50 2024-04-20T11:31:53.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:53.826+0000 7fd35d000640 1 -- 172.21.15.152:0/1696102095 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd3580689f0 con 0x7fd35806fa50 2024-04-20T11:31:53.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:53.827+0000 7fd34e000640 1 -- 172.21.15.152:0/1696102095 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+5210 (secure 0 0 0) 0x7fd348059f90 con 0x7fd35806fa50 2024-04-20T11:31:53.827 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:31:53.827 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":43,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5234,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6836/359391273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6835","nonce":359391273},{"type":"v1","addr":"172.21.15.152:6836","nonce":359391273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":5237,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/4269836436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":4269836436},{"type":"v1","addr":"172.21.15.152:6837","nonce":4269836436}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":5611,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6834/4153485754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":4153485754},{"type":"v1","addr":"172.21.15.175:6834","nonce":4153485754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T11:31:49.738765+0000","modified":"2024-04-20T11:31:52.857955+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5614},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5614":{"gid":5614,"name":"b","rank":0,"incarnation":39,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6835/2057501014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6833","nonce":2057501014},{"type":"v1","addr":"172.21.15.175:6835","nonce":2057501014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5614,"qdb_cluster":[5614]},"id":5}]} 2024-04-20T11:31:53.827 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 43 2024-04-20T11:31:53.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:53.830+0000 7fd35d000640 1 -- 172.21.15.152:0/1696102095 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fd33406cc50 msgr2=0x7fd33406f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:53.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:53.830+0000 7fd35d000640 1 --2- 172.21.15.152:0/1696102095 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fd33406cc50 0x7fd33406f110 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fd3580af350 tx=0x7fd340007970 comp rx=0 tx=0).stop 2024-04-20T11:31:53.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:53.830+0000 7fd35d000640 1 -- 172.21.15.152:0/1696102095 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd35806fa50 msgr2=0x7fd35815a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:53.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:53.830+0000 7fd35d000640 1 --2- 172.21.15.152:0/1696102095 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd35806fa50 0x7fd35815a9a0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fd34800e8a0 tx=0x7fd34800ed60 comp rx=0 tx=0).stop 2024-04-20T11:31:53.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:53.830+0000 7fd35d000640 1 -- 172.21.15.152:0/1696102095 shutdown_connections 2024-04-20T11:31:53.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:53.830+0000 7fd35d000640 1 --2- 172.21.15.152:0/1696102095 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fd33406cc50 0x7fd33406f110 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:53.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:53.830+0000 7fd35d000640 1 --2- 172.21.15.152:0/1696102095 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd35815b5e0 0x7fd3580b0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:53.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:53.830+0000 7fd35d000640 1 --2- 172.21.15.152:0/1696102095 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd3580b4c50 0x7fd3580a95c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:53.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:53.830+0000 7fd35d000640 1 --2- 172.21.15.152:0/1696102095 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd35806fa50 0x7fd35815a9a0 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:53.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:53.830+0000 7fd35d000640 1 -- 172.21.15.152:0/1696102095 >> 172.21.15.152:0/1696102095 conn(0x7fd358067cf0 msgr2=0x7fd3580b3830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:53.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:53.831+0000 7fd35d000640 1 -- 172.21.15.152:0/1696102095 shutdown_connections 2024-04-20T11:31:53.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:53.831+0000 7fd35d000640 1 -- 172.21.15.152:0/1696102095 wait complete. 2024-04-20T11:31:53.850 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:31:54.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.048+0000 7f4cfc600640 1 Processor -- start 2024-04-20T11:31:54.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.048+0000 7f4cfc600640 1 -- start start 2024-04-20T11:31:54.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.050+0000 7f4cfc600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4cf415edc0 0x7f4cf409b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:54.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.050+0000 7f4cfc600640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f4cf40b4be0 0x7f4cf406e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:54.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.050+0000 7f4cfc600640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4cf409b710 con 0x7f4cf40d0bc0 2024-04-20T11:31:54.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.050+0000 7f4cfc600640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f4cf406d050 con 0x7f4cf40b4be0 2024-04-20T11:31:54.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.050+0000 7f4cfc600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4cf415f1a0 con 0x7f4cf415edc0 2024-04-20T11:31:54.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.050+0000 7f4cf9200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f4cf40b4be0 0x7f4cf406e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:54.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.050+0000 7f4cf9200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f4cf40b4be0 0x7f4cf406e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36616/0 (socket says 172.21.15.152:36616) 2024-04-20T11:31:54.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.050+0000 7f4cf9200640 1 -- 172.21.15.152:0/72007645 learned_addr learned my addr 172.21.15.152:0/72007645 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:54.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.051+0000 7f4cf9c00640 1 --2- 172.21.15.152:0/72007645 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4cf415edc0 0x7f4cf409b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:54.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.051+0000 7f4cf9200640 1 -- 172.21.15.152:0/72007645 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4cf415edc0 msgr2=0x7f4cf409b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:54.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.051+0000 7f4cf9200640 1 --2- 172.21.15.152:0/72007645 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4cf415edc0 0x7f4cf409b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:54.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.051+0000 7f4cf9200640 1 -- 172.21.15.152:0/72007645 >> v1:172.21.15.175:6789/0 conn(0x7f4cf40d0bc0 legacy=0x7f4cf40ccc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:54.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.051+0000 7f4cf9200640 1 -- 172.21.15.152:0/72007645 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cf406d2f0 con 0x7f4cf40b4be0 2024-04-20T11:31:54.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.051+0000 7f4cf9c00640 1 --2- 172.21.15.152:0/72007645 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4cf415edc0 0x7f4cf409b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:31:54.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.052+0000 7f4cf9200640 1 --2- 172.21.15.152:0/72007645 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4cf40b4be0 0x7f4cf406e390 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f4ce4009870 tx=0x7f4ce402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a537ddb758a62aa server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:54.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.052+0000 7f4cf3e00640 1 -- 172.21.15.152:0/72007645 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce4035020 con 0x7f4cf40b4be0 2024-04-20T11:31:54.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.052+0000 7f4cf3e00640 1 -- 172.21.15.152:0/72007645 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ce4031640 con 0x7f4cf40b4be0 2024-04-20T11:31:54.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.052+0000 7f4cf3e00640 1 -- 172.21.15.152:0/72007645 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce402d040 con 0x7f4cf40b4be0 2024-04-20T11:31:54.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.053+0000 7f4cfc600640 1 -- 172.21.15.152:0/72007645 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4cf40b4be0 msgr2=0x7f4cf406e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:54.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.053+0000 7f4cfc600640 1 --2- 172.21.15.152:0/72007645 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4cf40b4be0 0x7f4cf406e390 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f4ce4009870 tx=0x7f4ce402b8f0 comp rx=0 tx=0).stop 2024-04-20T11:31:54.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.053+0000 7f4cfc600640 1 -- 172.21.15.152:0/72007645 shutdown_connections 2024-04-20T11:31:54.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.053+0000 7f4cfc600640 1 --2- 172.21.15.152:0/72007645 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4cf40b4be0 0x7f4cf406e390 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:54.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.053+0000 7f4cfc600640 1 --2- 172.21.15.152:0/72007645 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4cf415edc0 0x7f4cf409b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:54.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.053+0000 7f4cfc600640 1 -- 172.21.15.152:0/72007645 >> 172.21.15.152:0/72007645 conn(0x7f4cf40b40c0 msgr2=0x7f4cf40b44d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:54.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.053+0000 7f4cfc600640 1 -- 172.21.15.152:0/72007645 shutdown_connections 2024-04-20T11:31:54.054 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.053+0000 7f4cfc600640 1 -- 172.21.15.152:0/72007645 wait complete. 2024-04-20T11:31:54.054 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.055+0000 7f4cfc600640 1 Processor -- start 2024-04-20T11:31:54.054 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.055+0000 7f4cfc600640 1 -- start start 2024-04-20T11:31:54.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.056+0000 7f4cfc600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4cf40b4be0 0x7f4cf40cd630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:54.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.056+0000 7f4cfc600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4cf40d0bc0 0x7f4cf41f3d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:54.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.056+0000 7f4cfc600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4cf415edc0 0x7f4cf41f4250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:54.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.056+0000 7f4cfc600640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf406db90 con 0x7f4cf40b4be0 2024-04-20T11:31:54.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.056+0000 7f4cfc600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf406d8b0 con 0x7f4cf40d0bc0 2024-04-20T11:31:54.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.056+0000 7f4cfc600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4cf406de70 con 0x7f4cf415edc0 2024-04-20T11:31:54.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.056+0000 7f4cf9c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4cf40b4be0 0x7f4cf40cd630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:54.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.056+0000 7f4cf9c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4cf40b4be0 0x7f4cf40cd630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36622/0 (socket says 172.21.15.152:36622) 2024-04-20T11:31:54.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.056+0000 7f4cf9c00640 1 -- 172.21.15.152:0/2672924260 learned_addr learned my addr 172.21.15.152:0/2672924260 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:54.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.057+0000 7f4cf9200640 1 --2- 172.21.15.152:0/2672924260 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4cf40d0bc0 0x7f4cf41f3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:54.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.057+0000 7f4cf9c00640 1 -- 172.21.15.152:0/2672924260 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4cf415edc0 msgr2=0x7f4cf41f4250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:54.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.057+0000 7f4cf9c00640 1 --2- 172.21.15.152:0/2672924260 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4cf415edc0 0x7f4cf41f4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:54.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.057+0000 7f4cf9c00640 1 -- 172.21.15.152:0/2672924260 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4cf40d0bc0 msgr2=0x7f4cf41f3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:54.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.057+0000 7f4cf9c00640 1 --2- 172.21.15.152:0/2672924260 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4cf40d0bc0 0x7f4cf41f3d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:54.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.057+0000 7f4cf9c00640 1 -- 172.21.15.152:0/2672924260 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cf40b7f90 con 0x7f4cf40b4be0 2024-04-20T11:31:54.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.057+0000 7f4cf9c00640 1 --2- 172.21.15.152:0/2672924260 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4cf40b4be0 0x7f4cf40cd630 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f4ce800cf60 tx=0x7f4ce800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:54.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.057+0000 7f4cf1600640 1 -- 172.21.15.152:0/2672924260 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce801a070 con 0x7f4cf40b4be0 2024-04-20T11:31:54.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.057+0000 7f4cf1600640 1 -- 172.21.15.152:0/2672924260 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ce800d840 con 0x7f4cf40b4be0 2024-04-20T11:31:54.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.057+0000 7f4cfc600640 1 -- 172.21.15.152:0/2672924260 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cf415df30 con 0x7f4cf40b4be0 2024-04-20T11:31:54.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.057+0000 7f4cf1600640 1 -- 172.21.15.152:0/2672924260 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce800dc10 con 0x7f4cf40b4be0 2024-04-20T11:31:54.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.058+0000 7f4cfc600640 1 -- 172.21.15.152:0/2672924260 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cf40c96a0 con 0x7f4cf40b4be0 2024-04-20T11:31:54.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.059+0000 7f4cfc600640 1 -- 172.21.15.152:0/2672924260 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cf415e320 con 0x7f4cf40b4be0 2024-04-20T11:31:54.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.060+0000 7f4cf1600640 1 -- 172.21.15.152:0/2672924260 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 11) ==== 91346+0+0 (secure 0 0 0) 0x7f4ce801d070 con 0x7f4cf40b4be0 2024-04-20T11:31:54.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.061+0000 7f4cf1600640 1 --2- 172.21.15.152:0/2672924260 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f4cd806cc50 0x7f4cd806f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:54.061 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.061+0000 7f4cf1600640 1 -- 172.21.15.152:0/2672924260 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5709+0+0 (secure 0 0 0) 0x7f4ce808c990 con 0x7f4cf40b4be0 2024-04-20T11:31:54.061 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.061+0000 7f4cf9200640 1 --2- 172.21.15.152:0/2672924260 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f4cd806cc50 0x7f4cd806f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:54.061 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.062+0000 7f4cf9200640 1 --2- 172.21.15.152:0/2672924260 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f4cd806cc50 0x7f4cd806f110 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f4ce4036000 tx=0x7f4ce4008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:54.065 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.066+0000 7f4cf1600640 1 -- 172.21.15.152:0/2672924260 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f4ce805a220 con 0x7f4cf40b4be0 2024-04-20T11:31:54.265 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.265+0000 7f4cfc600640 1 -- 172.21.15.152:0/2672924260 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4cf40689f0 con 0x7f4cf40b4be0 2024-04-20T11:31:54.266 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.267+0000 7f4cf1600640 1 -- 172.21.15.152:0/2672924260 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) ==== 74+0+16728 (secure 0 0 0) 0x7f4ce805e520 con 0x7f4cf40b4be0 2024-04-20T11:31:54.267 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:31:54.267 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":71,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:31:51.301720+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:31:48.024528+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-20T11:31:48.725648+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:31:54.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.270+0000 7f4cfc600640 1 -- 172.21.15.152:0/2672924260 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f4cd806cc50 msgr2=0x7f4cd806f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:54.270 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.270+0000 7f4cfc600640 1 --2- 172.21.15.152:0/2672924260 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f4cd806cc50 0x7f4cd806f110 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f4ce4036000 tx=0x7f4ce4008030 comp rx=0 tx=0).stop 2024-04-20T11:31:54.270 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.270+0000 7f4cfc600640 1 -- 172.21.15.152:0/2672924260 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4cf40b4be0 msgr2=0x7f4cf40cd630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:54.270 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.270+0000 7f4cfc600640 1 --2- 172.21.15.152:0/2672924260 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4cf40b4be0 0x7f4cf40cd630 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f4ce800cf60 tx=0x7f4ce800cf90 comp rx=0 tx=0).stop 2024-04-20T11:31:54.270 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.270+0000 7f4cfc600640 1 -- 172.21.15.152:0/2672924260 shutdown_connections 2024-04-20T11:31:54.270 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.270+0000 7f4cfc600640 1 --2- 172.21.15.152:0/2672924260 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f4cd806cc50 0x7f4cd806f110 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:54.270 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.270+0000 7f4cfc600640 1 --2- 172.21.15.152:0/2672924260 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4cf415edc0 0x7f4cf41f4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:54.270 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.270+0000 7f4cfc600640 1 --2- 172.21.15.152:0/2672924260 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4cf40d0bc0 0x7f4cf41f3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:54.270 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.270+0000 7f4cfc600640 1 --2- 172.21.15.152:0/2672924260 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4cf40b4be0 0x7f4cf40cd630 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:54.270 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.270+0000 7f4cfc600640 1 -- 172.21.15.152:0/2672924260 >> 172.21.15.152:0/2672924260 conn(0x7f4cf40b40c0 msgr2=0x7f4cf40704a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:54.270 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.270+0000 7f4cfc600640 1 -- 172.21.15.152:0/2672924260 shutdown_connections 2024-04-20T11:31:54.270 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.270+0000 7f4cfc600640 1 -- 172.21.15.152:0/2672924260 wait complete. 2024-04-20T11:31:54.289 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-20T11:31:54.289 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T11:31:54.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.491+0000 7fb94c000640 1 Processor -- start 2024-04-20T11:31:54.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.491+0000 7fb94c000640 1 -- start start 2024-04-20T11:31:54.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.493+0000 7fb94c000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fb94406eb40 0x7fb94406f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:54.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.493+0000 7fb94c000640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fb94406f880 0x7fb94415cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:54.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.493+0000 7fb94c000640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb94406fca0 con 0x7fb94415b5e0 2024-04-20T11:31:54.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.493+0000 7fb94c000640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fb94406de10 con 0x7fb94406eb40 2024-04-20T11:31:54.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.493+0000 7fb94c000640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fb9440b5770 con 0x7fb94406f880 2024-04-20T11:31:54.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.493+0000 7fb948c00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fb94406eb40 0x7fb94406f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:54.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.493+0000 7fb949600640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fb94415b5e0 0x7fb9440b4c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:56396/0 (socket says 172.21.15.152:56396) 2024-04-20T11:31:54.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.493+0000 7fb949600640 1 -- 172.21.15.152:0/338254314 learned_addr learned my addr 172.21.15.152:0/338254314 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:31:54.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.493+0000 7fb94a000640 1 --2- 172.21.15.152:0/338254314 >> v2:172.21.15.175:3300/0 conn(0x7fb94406f880 0x7fb94415cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:54.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.494+0000 7fb948c00640 1 -- 172.21.15.152:0/338254314 >> v2:172.21.15.175:3300/0 conn(0x7fb94406f880 msgr2=0x7fb94415cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:54.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.494+0000 7fb948c00640 1 --2- 172.21.15.152:0/338254314 >> v2:172.21.15.175:3300/0 conn(0x7fb94406f880 0x7fb94415cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:54.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.494+0000 7fb948c00640 1 -- 172.21.15.152:0/338254314 >> v1:172.21.15.152:6789/0 conn(0x7fb94415b5e0 legacy=0x7fb9440b4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:54.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.494+0000 7fb948c00640 1 -- 172.21.15.152:0/338254314 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb94406e060 con 0x7fb94406eb40 2024-04-20T11:31:54.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.494+0000 7fb943e00640 1 -- 172.21.15.152:0/338254314 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2477491317 0 0) 0x7fb94406fca0 con 0x7fb94415b5e0 2024-04-20T11:31:54.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.495+0000 7fb948c00640 1 --2- 172.21.15.152:0/338254314 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb94406eb40 0x7fb94406f340 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fb9440d0e90 tx=0x7fb93002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ba2b15f9dcc7849 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:54.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.495+0000 7fb943e00640 1 -- 172.21.15.152:0/338254314 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb930035020 con 0x7fb94406eb40 2024-04-20T11:31:54.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.495+0000 7fb943e00640 1 -- 172.21.15.152:0/338254314 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb930031830 con 0x7fb94406eb40 2024-04-20T11:31:54.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.495+0000 7fb943e00640 1 -- 172.21.15.152:0/338254314 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb930031c00 con 0x7fb94406eb40 2024-04-20T11:31:54.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.496+0000 7fb94c000640 1 -- 172.21.15.152:0/338254314 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb94406eb40 msgr2=0x7fb94406f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:54.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.496+0000 7fb94c000640 1 --2- 172.21.15.152:0/338254314 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb94406eb40 0x7fb94406f340 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fb9440d0e90 tx=0x7fb93002b8d0 comp rx=0 tx=0).stop 2024-04-20T11:31:54.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.496+0000 7fb94c000640 1 -- 172.21.15.152:0/338254314 shutdown_connections 2024-04-20T11:31:54.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.496+0000 7fb94c000640 1 --2- 172.21.15.152:0/338254314 >> v2:172.21.15.175:3300/0 conn(0x7fb94406f880 0x7fb94415cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:54.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.496+0000 7fb94c000640 1 --2- 172.21.15.152:0/338254314 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb94406eb40 0x7fb94406f340 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:54.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.496+0000 7fb94c000640 1 -- 172.21.15.152:0/338254314 >> 172.21.15.152:0/338254314 conn(0x7fb944067cf0 msgr2=0x7fb94406b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:54.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.496+0000 7fb94c000640 1 -- 172.21.15.152:0/338254314 shutdown_connections 2024-04-20T11:31:54.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.496+0000 7fb94c000640 1 -- 172.21.15.152:0/338254314 wait complete. 2024-04-20T11:31:54.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.498+0000 7fb94c000640 1 Processor -- start 2024-04-20T11:31:54.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.498+0000 7fb94c000640 1 -- start start 2024-04-20T11:31:54.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.498+0000 7fb94c000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb94406eb40 0x7fb944152ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:54.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.499+0000 7fb94c000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb94406f880 0x7fb9440a9650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:54.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.499+0000 7fb94c000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb94415b5e0 0x7fb9440aa340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:54.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.499+0000 7fb94c000640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fb94415c9d0 con 0x7fb94415b5e0 2024-04-20T11:31:54.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.499+0000 7fb94c000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb94406e620 con 0x7fb94406eb40 2024-04-20T11:31:54.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.499+0000 7fb94c000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb9440b42d0 con 0x7fb94406f880 2024-04-20T11:31:54.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.499+0000 7fb94a000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb94415b5e0 0x7fb9440aa340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:54.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.499+0000 7fb94a000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb94415b5e0 0x7fb9440aa340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36638/0 (socket says 172.21.15.152:36638) 2024-04-20T11:31:54.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.499+0000 7fb94a000640 1 -- 172.21.15.152:0/3192351166 learned_addr learned my addr 172.21.15.152:0/3192351166 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:54.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.499+0000 7fb94a000640 1 -- 172.21.15.152:0/3192351166 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb94406f880 msgr2=0x7fb9440a9650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:54.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.499+0000 7fb94a000640 1 --2- 172.21.15.152:0/3192351166 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb94406f880 0x7fb9440a9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:54.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.499+0000 7fb94a000640 1 -- 172.21.15.152:0/3192351166 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb94406eb40 msgr2=0x7fb944152ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:54.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.499+0000 7fb94a000640 1 --2- 172.21.15.152:0/3192351166 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb94406eb40 0x7fb944152ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:54.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.499+0000 7fb94a000640 1 -- 172.21.15.152:0/3192351166 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9440a9b90 con 0x7fb94415b5e0 2024-04-20T11:31:54.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.500+0000 7fb94a000640 1 --2- 172.21.15.152:0/3192351166 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb94415b5e0 0x7fb9440aa340 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fb93c007e40 tx=0x7fb93c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:54.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.500+0000 7fb941600640 1 -- 172.21.15.152:0/3192351166 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb93c014620 con 0x7fb94415b5e0 2024-04-20T11:31:54.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.500+0000 7fb941600640 1 -- 172.21.15.152:0/3192351166 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb93c019070 con 0x7fb94415b5e0 2024-04-20T11:31:54.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.500+0000 7fb941600640 1 -- 172.21.15.152:0/3192351166 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb93c014a20 con 0x7fb94415b5e0 2024-04-20T11:31:54.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.500+0000 7fb94c000640 1 -- 172.21.15.152:0/3192351166 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb944153150 con 0x7fb94415b5e0 2024-04-20T11:31:54.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.500+0000 7fb94c000640 1 -- 172.21.15.152:0/3192351166 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb944153710 con 0x7fb94415b5e0 2024-04-20T11:31:54.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.501+0000 7fb94c000640 1 -- 172.21.15.152:0/3192351166 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9440689f0 con 0x7fb94415b5e0 2024-04-20T11:31:54.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.502+0000 7fb941600640 1 -- 172.21.15.152:0/3192351166 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 11) ==== 91346+0+0 (secure 0 0 0) 0x7fb93c014bc0 con 0x7fb94415b5e0 2024-04-20T11:31:54.503 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.503+0000 7fb941600640 1 --2- 172.21.15.152:0/3192351166 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fb92406cc50 0x7fb92406f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:54.503 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.504+0000 7fb941600640 1 -- 172.21.15.152:0/3192351166 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5709+0+0 (secure 0 0 0) 0x7fb93c08c910 con 0x7fb94415b5e0 2024-04-20T11:31:54.503 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.504+0000 7fb949600640 1 --2- 172.21.15.152:0/3192351166 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fb92406cc50 0x7fb92406f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:54.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.505+0000 7fb949600640 1 --2- 172.21.15.152:0/3192351166 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fb92406cc50 0x7fb92406f110 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fb938007fd0 tx=0x7fb938007960 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:54.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.508+0000 7fb941600640 1 -- 172.21.15.152:0/3192351166 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fb93c05a9c0 con 0x7fb94415b5e0 2024-04-20T11:31:54.704 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.704+0000 7fb94c000640 1 -- 172.21.15.152:0/3192351166 --> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb94406c610 con 0x7fb92406cc50 2024-04-20T11:31:54.714 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.715+0000 7fb941600640 1 -- 172.21.15.152:0/3192351166 <== mgr.4113 v2:172.21.15.152:6832/2285110240 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252083 (secure 0 0 0) 0x7fb94406c610 con 0x7fb92406cc50 2024-04-20T11:31:54.717 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:31:54.719 INFO:teuthology.orchestra.run.smithi152.stderr:dumped all 2024-04-20T11:31:54.721 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.721+0000 7fb94c000640 1 -- 172.21.15.152:0/3192351166 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fb92406cc50 msgr2=0x7fb92406f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:54.721 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.721+0000 7fb94c000640 1 --2- 172.21.15.152:0/3192351166 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fb92406cc50 0x7fb92406f110 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fb938007fd0 tx=0x7fb938007960 comp rx=0 tx=0).stop 2024-04-20T11:31:54.721 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.721+0000 7fb94c000640 1 -- 172.21.15.152:0/3192351166 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb94415b5e0 msgr2=0x7fb9440aa340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:54.721 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.721+0000 7fb94c000640 1 --2- 172.21.15.152:0/3192351166 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb94415b5e0 0x7fb9440aa340 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fb93c007e40 tx=0x7fb93c00e4b0 comp rx=0 tx=0).stop 2024-04-20T11:31:54.721 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.721+0000 7fb94c000640 1 -- 172.21.15.152:0/3192351166 shutdown_connections 2024-04-20T11:31:54.721 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.721+0000 7fb94c000640 1 --2- 172.21.15.152:0/3192351166 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fb92406cc50 0x7fb92406f110 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:54.721 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.721+0000 7fb94c000640 1 --2- 172.21.15.152:0/3192351166 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb94415b5e0 0x7fb9440aa340 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:54.721 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.721+0000 7fb94c000640 1 --2- 172.21.15.152:0/3192351166 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb94406f880 0x7fb9440a9650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:54.721 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.721+0000 7fb94c000640 1 --2- 172.21.15.152:0/3192351166 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb94406eb40 0x7fb944152ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:54.721 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.721+0000 7fb94c000640 1 -- 172.21.15.152:0/3192351166 >> 172.21.15.152:0/3192351166 conn(0x7fb944067cf0 msgr2=0x7fb9440725b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:54.721 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.721+0000 7fb94c000640 1 -- 172.21.15.152:0/3192351166 shutdown_connections 2024-04-20T11:31:54.722 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.722+0000 7fb94c000640 1 -- 172.21.15.152:0/3192351166 wait complete. 2024-04-20T11:31:54.741 INFO:teuthology.orchestra.run.smithi152.stdout:{"pg_ready":true,"pg_map":{"version":217,"stamp":"2024-04-20T11:31:53.311510+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":148,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":257432,"kb_used_data":7952,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754717288,"statfs":{"total":773094113280,"available":772830502912,"internally_reserved":0,"allocated":8142848,"data_stored":4093060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12746,"internal_metadata":255315510},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.054763"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:52.144504+0000","last_change":"2024-04-20T11:31:50.325580+0000","last_active":"2024-04-20T11:31:52.144504+0000","last_peered":"2024-04-20T11:31:52.144504+0000","last_clean":"2024-04-20T11:31:52.144504+0000","last_became_active":"2024-04-20T11:31:50.324926+0000","last_became_peered":"2024-04-20T11:31:50.324926+0000","last_unstale":"2024-04-20T11:31:52.144504+0000","last_undegraded":"2024-04-20T11:31:52.144504+0000","last_fullsized":"2024-04-20T11:31:52.144504+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:03:24.930741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.413081+0000","last_change":"2024-04-20T11:31:50.328037+0000","last_active":"2024-04-20T11:31:51.413081+0000","last_peered":"2024-04-20T11:31:51.413081+0000","last_clean":"2024-04-20T11:31:51.413081+0000","last_became_active":"2024-04-20T11:31:50.327723+0000","last_became_peered":"2024-04-20T11:31:50.327723+0000","last_unstale":"2024-04-20T11:31:51.413081+0000","last_undegraded":"2024-04-20T11:31:51.413081+0000","last_fullsized":"2024-04-20T11:31:51.413081+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:33:29.427506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.320228+0000","last_change":"2024-04-20T11:31:50.310201+0000","last_active":"2024-04-20T11:31:51.320228+0000","last_peered":"2024-04-20T11:31:51.320228+0000","last_clean":"2024-04-20T11:31:51.320228+0000","last_became_active":"2024-04-20T11:31:50.309302+0000","last_became_peered":"2024-04-20T11:31:50.309302+0000","last_unstale":"2024-04-20T11:31:51.320228+0000","last_undegraded":"2024-04-20T11:31:51.320228+0000","last_fullsized":"2024-04-20T11:31:51.320228+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:37:56.915958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.318473+0000","last_change":"2024-04-20T11:31:50.330202+0000","last_active":"2024-04-20T11:31:51.318473+0000","last_peered":"2024-04-20T11:31:51.318473+0000","last_clean":"2024-04-20T11:31:51.318473+0000","last_became_active":"2024-04-20T11:31:50.329914+0000","last_became_peered":"2024-04-20T11:31:50.329914+0000","last_unstale":"2024-04-20T11:31:51.318473+0000","last_undegraded":"2024-04-20T11:31:51.318473+0000","last_fullsized":"2024-04-20T11:31:51.318473+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:28:05.113257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:52.071888+0000","last_change":"2024-04-20T11:31:50.323466+0000","last_active":"2024-04-20T11:31:52.071888+0000","last_peered":"2024-04-20T11:31:52.071888+0000","last_clean":"2024-04-20T11:31:52.071888+0000","last_became_active":"2024-04-20T11:31:50.323204+0000","last_became_peered":"2024-04-20T11:31:50.323204+0000","last_unstale":"2024-04-20T11:31:52.071888+0000","last_undegraded":"2024-04-20T11:31:52.071888+0000","last_fullsized":"2024-04-20T11:31:52.071888+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:22:41.334900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.411815+0000","last_change":"2024-04-20T11:31:50.341288+0000","last_active":"2024-04-20T11:31:51.411815+0000","last_peered":"2024-04-20T11:31:51.411815+0000","last_clean":"2024-04-20T11:31:51.411815+0000","last_became_active":"2024-04-20T11:31:50.339864+0000","last_became_peered":"2024-04-20T11:31:50.339864+0000","last_unstale":"2024-04-20T11:31:51.411815+0000","last_undegraded":"2024-04-20T11:31:51.411815+0000","last_fullsized":"2024-04-20T11:31:51.411815+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:38:35.650488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.317481+0000","last_change":"2024-04-20T11:31:50.311897+0000","last_active":"2024-04-20T11:31:51.317481+0000","last_peered":"2024-04-20T11:31:51.317481+0000","last_clean":"2024-04-20T11:31:51.317481+0000","last_became_active":"2024-04-20T11:31:50.308939+0000","last_became_peered":"2024-04-20T11:31:50.308939+0000","last_unstale":"2024-04-20T11:31:51.317481+0000","last_undegraded":"2024-04-20T11:31:51.317481+0000","last_fullsized":"2024-04-20T11:31:51.317481+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:14:01.991182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.411848+0000","last_change":"2024-04-20T11:31:50.321035+0000","last_active":"2024-04-20T11:31:51.411848+0000","last_peered":"2024-04-20T11:31:51.411848+0000","last_clean":"2024-04-20T11:31:51.411848+0000","last_became_active":"2024-04-20T11:31:50.320724+0000","last_became_peered":"2024-04-20T11:31:50.320724+0000","last_unstale":"2024-04-20T11:31:51.411848+0000","last_undegraded":"2024-04-20T11:31:51.411848+0000","last_fullsized":"2024-04-20T11:31:51.411848+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:07:47.761243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:52.072428+0000","last_change":"2024-04-20T11:31:50.322571+0000","last_active":"2024-04-20T11:31:52.072428+0000","last_peered":"2024-04-20T11:31:52.072428+0000","last_clean":"2024-04-20T11:31:52.072428+0000","last_became_active":"2024-04-20T11:31:50.321867+0000","last_became_peered":"2024-04-20T11:31:50.321867+0000","last_unstale":"2024-04-20T11:31:52.072428+0000","last_undegraded":"2024-04-20T11:31:52.072428+0000","last_fullsized":"2024-04-20T11:31:52.072428+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:05:36.261388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:52.144364+0000","last_change":"2024-04-20T11:31:50.321085+0000","last_active":"2024-04-20T11:31:52.144364+0000","last_peered":"2024-04-20T11:31:52.144364+0000","last_clean":"2024-04-20T11:31:52.144364+0000","last_became_active":"2024-04-20T11:31:50.320791+0000","last_became_peered":"2024-04-20T11:31:50.320791+0000","last_unstale":"2024-04-20T11:31:52.144364+0000","last_undegraded":"2024-04-20T11:31:52.144364+0000","last_fullsized":"2024-04-20T11:31:52.144364+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:28:39.542749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:52.143127+0000","last_change":"2024-04-20T11:31:50.323050+0000","last_active":"2024-04-20T11:31:52.143127+0000","last_peered":"2024-04-20T11:31:52.143127+0000","last_clean":"2024-04-20T11:31:52.143127+0000","last_became_active":"2024-04-20T11:31:50.322778+0000","last_became_peered":"2024-04-20T11:31:50.322778+0000","last_unstale":"2024-04-20T11:31:52.143127+0000","last_undegraded":"2024-04-20T11:31:52.143127+0000","last_fullsized":"2024-04-20T11:31:52.143127+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:44:55.157437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.318871+0000","last_change":"2024-04-20T11:31:50.315955+0000","last_active":"2024-04-20T11:31:51.318871+0000","last_peered":"2024-04-20T11:31:51.318871+0000","last_clean":"2024-04-20T11:31:51.318871+0000","last_became_active":"2024-04-20T11:31:50.313938+0000","last_became_peered":"2024-04-20T11:31:50.313938+0000","last_unstale":"2024-04-20T11:31:51.318871+0000","last_undegraded":"2024-04-20T11:31:51.318871+0000","last_fullsized":"2024-04-20T11:31:51.318871+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:33:45.092493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.333207+0000","last_change":"2024-04-20T11:31:50.330639+0000","last_active":"2024-04-20T11:31:51.333207+0000","last_peered":"2024-04-20T11:31:51.333207+0000","last_clean":"2024-04-20T11:31:51.333207+0000","last_became_active":"2024-04-20T11:31:50.330276+0000","last_became_peered":"2024-04-20T11:31:50.330276+0000","last_unstale":"2024-04-20T11:31:51.333207+0000","last_undegraded":"2024-04-20T11:31:51.333207+0000","last_fullsized":"2024-04-20T11:31:51.333207+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:38:43.528257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:52.071818+0000","last_change":"2024-04-20T11:31:50.312741+0000","last_active":"2024-04-20T11:31:52.071818+0000","last_peered":"2024-04-20T11:31:52.071818+0000","last_clean":"2024-04-20T11:31:52.071818+0000","last_became_active":"2024-04-20T11:31:50.312315+0000","last_became_peered":"2024-04-20T11:31:50.312315+0000","last_unstale":"2024-04-20T11:31:52.071818+0000","last_undegraded":"2024-04-20T11:31:52.071818+0000","last_fullsized":"2024-04-20T11:31:52.071818+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:48:33.476138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.332656+0000","last_change":"2024-04-20T11:31:50.316408+0000","last_active":"2024-04-20T11:31:51.332656+0000","last_peered":"2024-04-20T11:31:51.332656+0000","last_clean":"2024-04-20T11:31:51.332656+0000","last_became_active":"2024-04-20T11:31:50.316099+0000","last_became_peered":"2024-04-20T11:31:50.316099+0000","last_unstale":"2024-04-20T11:31:51.332656+0000","last_undegraded":"2024-04-20T11:31:51.332656+0000","last_fullsized":"2024-04-20T11:31:51.332656+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:49:05.141517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:52.142303+0000","last_change":"2024-04-20T11:31:50.342752+0000","last_active":"2024-04-20T11:31:52.142303+0000","last_peered":"2024-04-20T11:31:52.142303+0000","last_clean":"2024-04-20T11:31:52.142303+0000","last_became_active":"2024-04-20T11:31:50.342243+0000","last_became_peered":"2024-04-20T11:31:50.342243+0000","last_unstale":"2024-04-20T11:31:52.142303+0000","last_undegraded":"2024-04-20T11:31:52.142303+0000","last_fullsized":"2024-04-20T11:31:52.142303+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:30:00.566230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:52.072688+0000","last_change":"2024-04-20T11:31:50.321768+0000","last_active":"2024-04-20T11:31:52.072688+0000","last_peered":"2024-04-20T11:31:52.072688+0000","last_clean":"2024-04-20T11:31:52.072688+0000","last_became_active":"2024-04-20T11:31:50.319324+0000","last_became_peered":"2024-04-20T11:31:50.319324+0000","last_unstale":"2024-04-20T11:31:52.072688+0000","last_undegraded":"2024-04-20T11:31:52.072688+0000","last_fullsized":"2024-04-20T11:31:52.072688+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:26:59.369353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.318705+0000","last_change":"2024-04-20T11:31:50.330892+0000","last_active":"2024-04-20T11:31:51.318705+0000","last_peered":"2024-04-20T11:31:51.318705+0000","last_clean":"2024-04-20T11:31:51.318705+0000","last_became_active":"2024-04-20T11:31:50.330542+0000","last_became_peered":"2024-04-20T11:31:50.330542+0000","last_unstale":"2024-04-20T11:31:51.318705+0000","last_undegraded":"2024-04-20T11:31:51.318705+0000","last_fullsized":"2024-04-20T11:31:51.318705+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:52:00.356433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.412538+0000","last_change":"2024-04-20T11:31:50.338478+0000","last_active":"2024-04-20T11:31:51.412538+0000","last_peered":"2024-04-20T11:31:51.412538+0000","last_clean":"2024-04-20T11:31:51.412538+0000","last_became_active":"2024-04-20T11:31:50.329097+0000","last_became_peered":"2024-04-20T11:31:50.329097+0000","last_unstale":"2024-04-20T11:31:51.412538+0000","last_undegraded":"2024-04-20T11:31:51.412538+0000","last_fullsized":"2024-04-20T11:31:51.412538+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:30:00.566230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.333267+0000","last_change":"2024-04-20T11:31:50.320732+0000","last_active":"2024-04-20T11:31:51.333267+0000","last_peered":"2024-04-20T11:31:51.333267+0000","last_clean":"2024-04-20T11:31:51.333267+0000","last_became_active":"2024-04-20T11:31:50.320369+0000","last_became_peered":"2024-04-20T11:31:50.320369+0000","last_unstale":"2024-04-20T11:31:51.333267+0000","last_undegraded":"2024-04-20T11:31:51.333267+0000","last_fullsized":"2024-04-20T11:31:51.333267+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:18:48.610827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:52.143451+0000","last_change":"2024-04-20T11:31:50.320620+0000","last_active":"2024-04-20T11:31:52.143451+0000","last_peered":"2024-04-20T11:31:52.143451+0000","last_clean":"2024-04-20T11:31:52.143451+0000","last_became_active":"2024-04-20T11:31:50.319513+0000","last_became_peered":"2024-04-20T11:31:50.319513+0000","last_unstale":"2024-04-20T11:31:52.143451+0000","last_undegraded":"2024-04-20T11:31:52.143451+0000","last_fullsized":"2024-04-20T11:31:52.143451+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:33:29.427506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:52.143342+0000","last_change":"2024-04-20T11:31:50.323667+0000","last_active":"2024-04-20T11:31:52.143342+0000","last_peered":"2024-04-20T11:31:52.143342+0000","last_clean":"2024-04-20T11:31:52.143342+0000","last_became_active":"2024-04-20T11:31:50.323294+0000","last_became_peered":"2024-04-20T11:31:50.323294+0000","last_unstale":"2024-04-20T11:31:52.143342+0000","last_undegraded":"2024-04-20T11:31:52.143342+0000","last_fullsized":"2024-04-20T11:31:52.143342+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:55:54.473999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:52.142561+0000","last_change":"2024-04-20T11:31:50.329025+0000","last_active":"2024-04-20T11:31:52.142561+0000","last_peered":"2024-04-20T11:31:52.142561+0000","last_clean":"2024-04-20T11:31:52.142561+0000","last_became_active":"2024-04-20T11:31:50.328741+0000","last_became_peered":"2024-04-20T11:31:50.328741+0000","last_unstale":"2024-04-20T11:31:52.142561+0000","last_undegraded":"2024-04-20T11:31:52.142561+0000","last_fullsized":"2024-04-20T11:31:52.142561+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:47:37.822407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:52.142088+0000","last_change":"2024-04-20T11:31:50.331933+0000","last_active":"2024-04-20T11:31:52.142088+0000","last_peered":"2024-04-20T11:31:52.142088+0000","last_clean":"2024-04-20T11:31:52.142088+0000","last_became_active":"2024-04-20T11:31:50.331730+0000","last_became_peered":"2024-04-20T11:31:50.331730+0000","last_unstale":"2024-04-20T11:31:52.142088+0000","last_undegraded":"2024-04-20T11:31:52.142088+0000","last_fullsized":"2024-04-20T11:31:52.142088+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:38:43.528257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:52.144059+0000","last_change":"2024-04-20T11:31:50.341703+0000","last_active":"2024-04-20T11:31:52.144059+0000","last_peered":"2024-04-20T11:31:52.144059+0000","last_clean":"2024-04-20T11:31:52.144059+0000","last_became_active":"2024-04-20T11:31:50.341371+0000","last_became_peered":"2024-04-20T11:31:50.341371+0000","last_unstale":"2024-04-20T11:31:52.144059+0000","last_undegraded":"2024-04-20T11:31:52.144059+0000","last_fullsized":"2024-04-20T11:31:52.144059+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:29:41.481297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.318323+0000","last_change":"2024-04-20T11:31:50.329710+0000","last_active":"2024-04-20T11:31:51.318323+0000","last_peered":"2024-04-20T11:31:51.318323+0000","last_clean":"2024-04-20T11:31:51.318323+0000","last_became_active":"2024-04-20T11:31:50.329406+0000","last_became_peered":"2024-04-20T11:31:50.329406+0000","last_unstale":"2024-04-20T11:31:51.318323+0000","last_undegraded":"2024-04-20T11:31:51.318323+0000","last_fullsized":"2024-04-20T11:31:51.318323+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:03:26.976387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.320657+0000","last_change":"2024-04-20T11:31:50.314168+0000","last_active":"2024-04-20T11:31:51.320657+0000","last_peered":"2024-04-20T11:31:51.320657+0000","last_clean":"2024-04-20T11:31:51.320657+0000","last_became_active":"2024-04-20T11:31:50.313856+0000","last_became_peered":"2024-04-20T11:31:50.313856+0000","last_unstale":"2024-04-20T11:31:51.320657+0000","last_undegraded":"2024-04-20T11:31:51.320657+0000","last_fullsized":"2024-04-20T11:31:51.320657+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:25:20.652223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:52.144808+0000","last_change":"2024-04-20T11:31:50.321610+0000","last_active":"2024-04-20T11:31:52.144808+0000","last_peered":"2024-04-20T11:31:52.144808+0000","last_clean":"2024-04-20T11:31:52.144808+0000","last_became_active":"2024-04-20T11:31:50.321214+0000","last_became_peered":"2024-04-20T11:31:50.321214+0000","last_unstale":"2024-04-20T11:31:52.144808+0000","last_undegraded":"2024-04-20T11:31:52.144808+0000","last_fullsized":"2024-04-20T11:31:52.144808+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:45:40.359700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.319845+0000","last_change":"2024-04-20T11:31:50.319039+0000","last_active":"2024-04-20T11:31:51.319845+0000","last_peered":"2024-04-20T11:31:51.319845+0000","last_clean":"2024-04-20T11:31:51.319845+0000","last_became_active":"2024-04-20T11:31:50.318707+0000","last_became_peered":"2024-04-20T11:31:50.318707+0000","last_unstale":"2024-04-20T11:31:51.319845+0000","last_undegraded":"2024-04-20T11:31:51.319845+0000","last_fullsized":"2024-04-20T11:31:51.319845+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:55:29.750493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.317877+0000","last_change":"2024-04-20T11:31:50.335499+0000","last_active":"2024-04-20T11:31:51.317877+0000","last_peered":"2024-04-20T11:31:51.317877+0000","last_clean":"2024-04-20T11:31:51.317877+0000","last_became_active":"2024-04-20T11:31:50.332697+0000","last_became_peered":"2024-04-20T11:31:50.332697+0000","last_unstale":"2024-04-20T11:31:51.317877+0000","last_undegraded":"2024-04-20T11:31:51.317877+0000","last_fullsized":"2024-04-20T11:31:51.317877+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:01:47.414565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.317530+0000","last_change":"2024-04-20T11:31:50.313012+0000","last_active":"2024-04-20T11:31:51.317530+0000","last_peered":"2024-04-20T11:31:51.317530+0000","last_clean":"2024-04-20T11:31:51.317530+0000","last_became_active":"2024-04-20T11:31:50.312581+0000","last_became_peered":"2024-04-20T11:31:50.312581+0000","last_unstale":"2024-04-20T11:31:51.317530+0000","last_undegraded":"2024-04-20T11:31:51.317530+0000","last_fullsized":"2024-04-20T11:31:51.317530+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:45:40.359700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.412052+0000","last_change":"2024-04-20T11:31:50.330657+0000","last_active":"2024-04-20T11:31:51.412052+0000","last_peered":"2024-04-20T11:31:51.412052+0000","last_clean":"2024-04-20T11:31:51.412052+0000","last_became_active":"2024-04-20T11:31:50.330371+0000","last_became_peered":"2024-04-20T11:31:50.330371+0000","last_unstale":"2024-04-20T11:31:51.412052+0000","last_undegraded":"2024-04-20T11:31:51.412052+0000","last_fullsized":"2024-04-20T11:31:51.412052+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:44:00.584674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:52.144132+0000","last_change":"2024-04-20T11:31:50.341124+0000","last_active":"2024-04-20T11:31:52.144132+0000","last_peered":"2024-04-20T11:31:52.144132+0000","last_clean":"2024-04-20T11:31:52.144132+0000","last_became_active":"2024-04-20T11:31:50.340851+0000","last_became_peered":"2024-04-20T11:31:50.340851+0000","last_unstale":"2024-04-20T11:31:52.144132+0000","last_undegraded":"2024-04-20T11:31:52.144132+0000","last_fullsized":"2024-04-20T11:31:52.144132+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:46:50.995840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"70'4","reported_seq":18,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:52.145009+0000","last_change":"2024-04-20T11:31:49.302275+0000","last_active":"2024-04-20T11:31:52.145009+0000","last_peered":"2024-04-20T11:31:52.145009+0000","last_clean":"2024-04-20T11:31:52.145009+0000","last_became_active":"2024-04-20T11:31:49.301374+0000","last_became_peered":"2024-04-20T11:31:49.301374+0000","last_unstale":"2024-04-20T11:31:52.145009+0000","last_undegraded":"2024-04-20T11:31:52.145009+0000","last_fullsized":"2024-04-20T11:31:52.145009+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_clean_scrub_stamp":"2024-04-20T11:31:48.241496+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:29:59.553636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:52.072791+0000","last_change":"2024-04-20T11:31:50.322007+0000","last_active":"2024-04-20T11:31:52.072791+0000","last_peered":"2024-04-20T11:31:52.072791+0000","last_clean":"2024-04-20T11:31:52.072791+0000","last_became_active":"2024-04-20T11:31:50.319703+0000","last_became_peered":"2024-04-20T11:31:50.319703+0000","last_unstale":"2024-04-20T11:31:52.072791+0000","last_undegraded":"2024-04-20T11:31:52.072791+0000","last_fullsized":"2024-04-20T11:31:52.072791+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:23:28.381927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"70'1","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.333424+0000","last_change":"2024-04-20T11:31:49.288268+0000","last_active":"2024-04-20T11:31:51.333424+0000","last_peered":"2024-04-20T11:31:51.333424+0000","last_clean":"2024-04-20T11:31:51.333424+0000","last_became_active":"2024-04-20T11:31:49.287810+0000","last_became_peered":"2024-04-20T11:31:49.287810+0000","last_unstale":"2024-04-20T11:31:51.333424+0000","last_undegraded":"2024-04-20T11:31:51.333424+0000","last_fullsized":"2024-04-20T11:31:51.333424+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_clean_scrub_stamp":"2024-04-20T11:31:48.241496+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:22:21.850770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.412123+0000","last_change":"2024-04-20T11:31:50.339803+0000","last_active":"2024-04-20T11:31:51.412123+0000","last_peered":"2024-04-20T11:31:51.412123+0000","last_clean":"2024-04-20T11:31:51.412123+0000","last_became_active":"2024-04-20T11:31:50.339500+0000","last_became_peered":"2024-04-20T11:31:50.339500+0000","last_unstale":"2024-04-20T11:31:51.412123+0000","last_undegraded":"2024-04-20T11:31:51.412123+0000","last_fullsized":"2024-04-20T11:31:51.412123+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:24:31.983271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"70'2","reported_seq":16,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:52.144120+0000","last_change":"2024-04-20T11:31:49.304762+0000","last_active":"2024-04-20T11:31:52.144120+0000","last_peered":"2024-04-20T11:31:52.144120+0000","last_clean":"2024-04-20T11:31:52.144120+0000","last_became_active":"2024-04-20T11:31:49.304416+0000","last_became_peered":"2024-04-20T11:31:49.304416+0000","last_unstale":"2024-04-20T11:31:52.144120+0000","last_undegraded":"2024-04-20T11:31:52.144120+0000","last_fullsized":"2024-04-20T11:31:52.144120+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_clean_scrub_stamp":"2024-04-20T11:31:48.241496+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:52:40.794561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:52.142972+0000","last_change":"2024-04-20T11:31:50.329274+0000","last_active":"2024-04-20T11:31:52.142972+0000","last_peered":"2024-04-20T11:31:52.142972+0000","last_clean":"2024-04-20T11:31:52.142972+0000","last_became_active":"2024-04-20T11:31:50.328551+0000","last_became_peered":"2024-04-20T11:31:50.328551+0000","last_unstale":"2024-04-20T11:31:52.142972+0000","last_undegraded":"2024-04-20T11:31:52.142972+0000","last_fullsized":"2024-04-20T11:31:52.142972+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:05:09.098414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"70'2","reported_seq":16,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.412861+0000","last_change":"2024-04-20T11:31:49.297823+0000","last_active":"2024-04-20T11:31:51.412861+0000","last_peered":"2024-04-20T11:31:51.412861+0000","last_clean":"2024-04-20T11:31:51.412861+0000","last_became_active":"2024-04-20T11:31:49.297474+0000","last_became_peered":"2024-04-20T11:31:49.297474+0000","last_unstale":"2024-04-20T11:31:51.412861+0000","last_undegraded":"2024-04-20T11:31:51.412861+0000","last_fullsized":"2024-04-20T11:31:51.412861+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_clean_scrub_stamp":"2024-04-20T11:31:48.241496+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:35:13.023313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.318295+0000","last_change":"2024-04-20T11:31:50.311206+0000","last_active":"2024-04-20T11:31:51.318295+0000","last_peered":"2024-04-20T11:31:51.318295+0000","last_clean":"2024-04-20T11:31:51.318295+0000","last_became_active":"2024-04-20T11:31:50.310613+0000","last_became_peered":"2024-04-20T11:31:50.310613+0000","last_unstale":"2024-04-20T11:31:51.318295+0000","last_undegraded":"2024-04-20T11:31:51.318295+0000","last_fullsized":"2024-04-20T11:31:51.318295+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:49:05.141517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"70'1","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.319957+0000","last_change":"2024-04-20T11:31:49.302337+0000","last_active":"2024-04-20T11:31:51.319957+0000","last_peered":"2024-04-20T11:31:51.319957+0000","last_clean":"2024-04-20T11:31:51.319957+0000","last_became_active":"2024-04-20T11:31:49.301130+0000","last_became_peered":"2024-04-20T11:31:49.301130+0000","last_unstale":"2024-04-20T11:31:51.319957+0000","last_undegraded":"2024-04-20T11:31:51.319957+0000","last_fullsized":"2024-04-20T11:31:51.319957+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_clean_scrub_stamp":"2024-04-20T11:31:48.241496+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:25:19.188163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:52.143901+0000","last_change":"2024-04-20T11:31:50.322874+0000","last_active":"2024-04-20T11:31:52.143901+0000","last_peered":"2024-04-20T11:31:52.143901+0000","last_clean":"2024-04-20T11:31:52.143901+0000","last_became_active":"2024-04-20T11:31:50.322543+0000","last_became_peered":"2024-04-20T11:31:50.322543+0000","last_unstale":"2024-04-20T11:31:52.143901+0000","last_undegraded":"2024-04-20T11:31:52.143901+0000","last_fullsized":"2024-04-20T11:31:52.143901+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:22:22.863365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"70'1","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.412218+0000","last_change":"2024-04-20T11:31:49.302612+0000","last_active":"2024-04-20T11:31:51.412218+0000","last_peered":"2024-04-20T11:31:51.412218+0000","last_clean":"2024-04-20T11:31:51.412218+0000","last_became_active":"2024-04-20T11:31:49.302072+0000","last_became_peered":"2024-04-20T11:31:49.302072+0000","last_unstale":"2024-04-20T11:31:51.412218+0000","last_undegraded":"2024-04-20T11:31:51.412218+0000","last_fullsized":"2024-04-20T11:31:51.412218+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_clean_scrub_stamp":"2024-04-20T11:31:48.241496+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:23:27.369333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:52.071280+0000","last_change":"2024-04-20T11:31:50.319154+0000","last_active":"2024-04-20T11:31:52.071280+0000","last_peered":"2024-04-20T11:31:52.071280+0000","last_clean":"2024-04-20T11:31:52.071280+0000","last_became_active":"2024-04-20T11:31:50.318848+0000","last_became_peered":"2024-04-20T11:31:50.318848+0000","last_unstale":"2024-04-20T11:31:52.071280+0000","last_undegraded":"2024-04-20T11:31:52.071280+0000","last_fullsized":"2024-04-20T11:31:52.071280+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:50:07.040083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.317690+0000","last_change":"2024-04-20T11:31:49.302035+0000","last_active":"2024-04-20T11:31:51.317690+0000","last_peered":"2024-04-20T11:31:51.317690+0000","last_clean":"2024-04-20T11:31:51.317690+0000","last_became_active":"2024-04-20T11:31:49.301766+0000","last_became_peered":"2024-04-20T11:31:49.301766+0000","last_unstale":"2024-04-20T11:31:51.317690+0000","last_undegraded":"2024-04-20T11:31:51.317690+0000","last_fullsized":"2024-04-20T11:31:51.317690+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_clean_scrub_stamp":"2024-04-20T11:31:48.241496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:47:47.119919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.332858+0000","last_change":"2024-04-20T11:31:50.335909+0000","last_active":"2024-04-20T11:31:51.332858+0000","last_peered":"2024-04-20T11:31:51.332858+0000","last_clean":"2024-04-20T11:31:51.332858+0000","last_became_active":"2024-04-20T11:31:50.335467+0000","last_became_peered":"2024-04-20T11:31:50.335467+0000","last_unstale":"2024-04-20T11:31:51.332858+0000","last_undegraded":"2024-04-20T11:31:51.332858+0000","last_fullsized":"2024-04-20T11:31:51.332858+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:03:26.976387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:52.071358+0000","last_change":"2024-04-20T11:31:49.283257+0000","last_active":"2024-04-20T11:31:52.071358+0000","last_peered":"2024-04-20T11:31:52.071358+0000","last_clean":"2024-04-20T11:31:52.071358+0000","last_became_active":"2024-04-20T11:31:49.282924+0000","last_became_peered":"2024-04-20T11:31:49.282924+0000","last_unstale":"2024-04-20T11:31:52.071358+0000","last_undegraded":"2024-04-20T11:31:52.071358+0000","last_fullsized":"2024-04-20T11:31:52.071358+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_clean_scrub_stamp":"2024-04-20T11:31:48.241496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:52:57.575105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.318544+0000","last_change":"2024-04-20T11:31:50.335086+0000","last_active":"2024-04-20T11:31:51.318544+0000","last_peered":"2024-04-20T11:31:51.318544+0000","last_clean":"2024-04-20T11:31:51.318544+0000","last_became_active":"2024-04-20T11:31:50.334427+0000","last_became_peered":"2024-04-20T11:31:50.334427+0000","last_unstale":"2024-04-20T11:31:51.318544+0000","last_undegraded":"2024-04-20T11:31:51.318544+0000","last_fullsized":"2024-04-20T11:31:51.318544+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:38:43.528257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:52.144345+0000","last_change":"2024-04-20T11:31:49.305014+0000","last_active":"2024-04-20T11:31:52.144345+0000","last_peered":"2024-04-20T11:31:52.144345+0000","last_clean":"2024-04-20T11:31:52.144345+0000","last_became_active":"2024-04-20T11:31:49.304677+0000","last_became_peered":"2024-04-20T11:31:49.304677+0000","last_unstale":"2024-04-20T11:31:52.144345+0000","last_undegraded":"2024-04-20T11:31:52.144345+0000","last_fullsized":"2024-04-20T11:31:52.144345+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_clean_scrub_stamp":"2024-04-20T11:31:48.241496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:33:44.079899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:52.144721+0000","last_change":"2024-04-20T11:31:50.340468+0000","last_active":"2024-04-20T11:31:52.144721+0000","last_peered":"2024-04-20T11:31:52.144721+0000","last_clean":"2024-04-20T11:31:52.144721+0000","last_became_active":"2024-04-20T11:31:50.340166+0000","last_became_peered":"2024-04-20T11:31:50.340166+0000","last_unstale":"2024-04-20T11:31:52.144721+0000","last_undegraded":"2024-04-20T11:31:52.144721+0000","last_fullsized":"2024-04-20T11:31:52.144721+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:46:12.753823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.413078+0000","last_change":"2024-04-20T11:31:49.300474+0000","last_active":"2024-04-20T11:31:51.413078+0000","last_peered":"2024-04-20T11:31:51.413078+0000","last_clean":"2024-04-20T11:31:51.413078+0000","last_became_active":"2024-04-20T11:31:49.300179+0000","last_became_peered":"2024-04-20T11:31:49.300179+0000","last_unstale":"2024-04-20T11:31:51.413078+0000","last_undegraded":"2024-04-20T11:31:51.413078+0000","last_fullsized":"2024-04-20T11:31:51.413078+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_clean_scrub_stamp":"2024-04-20T11:31:48.241496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:57:54.788106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.412584+0000","last_change":"2024-04-20T11:31:50.344041+0000","last_active":"2024-04-20T11:31:51.412584+0000","last_peered":"2024-04-20T11:31:51.412584+0000","last_clean":"2024-04-20T11:31:51.412584+0000","last_became_active":"2024-04-20T11:31:50.343739+0000","last_became_peered":"2024-04-20T11:31:50.343739+0000","last_unstale":"2024-04-20T11:31:51.412584+0000","last_undegraded":"2024-04-20T11:31:51.412584+0000","last_fullsized":"2024-04-20T11:31:51.412584+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:55:54.473999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"70'3","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.320777+0000","last_change":"2024-04-20T11:31:49.282326+0000","last_active":"2024-04-20T11:31:51.320777+0000","last_peered":"2024-04-20T11:31:51.320777+0000","last_clean":"2024-04-20T11:31:51.320777+0000","last_became_active":"2024-04-20T11:31:49.282026+0000","last_became_peered":"2024-04-20T11:31:49.282026+0000","last_unstale":"2024-04-20T11:31:51.320777+0000","last_undegraded":"2024-04-20T11:31:51.320777+0000","last_fullsized":"2024-04-20T11:31:51.320777+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_clean_scrub_stamp":"2024-04-20T11:31:48.241496+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:46:49.983246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:52.072403+0000","last_change":"2024-04-20T11:31:50.310816+0000","last_active":"2024-04-20T11:31:52.072403+0000","last_peered":"2024-04-20T11:31:52.072403+0000","last_clean":"2024-04-20T11:31:52.072403+0000","last_became_active":"2024-04-20T11:31:50.310455+0000","last_became_peered":"2024-04-20T11:31:50.310455+0000","last_unstale":"2024-04-20T11:31:52.072403+0000","last_undegraded":"2024-04-20T11:31:52.072403+0000","last_fullsized":"2024-04-20T11:31:52.072403+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:24:11.551245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"70'6","reported_seq":20,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.319119+0000","last_change":"2024-04-20T11:31:49.290097+0000","last_active":"2024-04-20T11:31:51.319119+0000","last_peered":"2024-04-20T11:31:51.319119+0000","last_clean":"2024-04-20T11:31:51.319119+0000","last_became_active":"2024-04-20T11:31:49.289048+0000","last_became_peered":"2024-04-20T11:31:49.289048+0000","last_unstale":"2024-04-20T11:31:51.319119+0000","last_undegraded":"2024-04-20T11:31:51.319119+0000","last_fullsized":"2024-04-20T11:31:51.319119+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_clean_scrub_stamp":"2024-04-20T11:31:48.241496+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:35:13.023313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:52.142516+0000","last_change":"2024-04-20T11:31:50.326503+0000","last_active":"2024-04-20T11:31:52.142516+0000","last_peered":"2024-04-20T11:31:52.142516+0000","last_clean":"2024-04-20T11:31:52.142516+0000","last_became_active":"2024-04-20T11:31:50.318766+0000","last_became_peered":"2024-04-20T11:31:50.318766+0000","last_unstale":"2024-04-20T11:31:52.142516+0000","last_undegraded":"2024-04-20T11:31:52.142516+0000","last_fullsized":"2024-04-20T11:31:52.142516+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:18:48.610827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"70'2","reported_seq":16,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.318542+0000","last_change":"2024-04-20T11:31:49.296194+0000","last_active":"2024-04-20T11:31:51.318542+0000","last_peered":"2024-04-20T11:31:51.318542+0000","last_clean":"2024-04-20T11:31:51.318542+0000","last_became_active":"2024-04-20T11:31:49.295903+0000","last_became_peered":"2024-04-20T11:31:49.295903+0000","last_unstale":"2024-04-20T11:31:51.318542+0000","last_undegraded":"2024-04-20T11:31:51.318542+0000","last_fullsized":"2024-04-20T11:31:51.318542+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_clean_scrub_stamp":"2024-04-20T11:31:48.241496+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:19:33.664899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.317847+0000","last_change":"2024-04-20T11:31:50.319043+0000","last_active":"2024-04-20T11:31:51.317847+0000","last_peered":"2024-04-20T11:31:51.317847+0000","last_clean":"2024-04-20T11:31:51.317847+0000","last_became_active":"2024-04-20T11:31:50.318727+0000","last_became_peered":"2024-04-20T11:31:50.318727+0000","last_unstale":"2024-04-20T11:31:51.317847+0000","last_undegraded":"2024-04-20T11:31:51.317847+0000","last_fullsized":"2024-04-20T11:31:51.317847+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:40:25.463996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.412423+0000","last_change":"2024-04-20T11:31:49.300082+0000","last_active":"2024-04-20T11:31:51.412423+0000","last_peered":"2024-04-20T11:31:51.412423+0000","last_clean":"2024-04-20T11:31:51.412423+0000","last_became_active":"2024-04-20T11:31:49.298466+0000","last_became_peered":"2024-04-20T11:31:49.298466+0000","last_unstale":"2024-04-20T11:31:51.412423+0000","last_undegraded":"2024-04-20T11:31:51.412423+0000","last_fullsized":"2024-04-20T11:31:51.412423+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_clean_scrub_stamp":"2024-04-20T11:31:48.241496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:47:47.119919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.333404+0000","last_change":"2024-04-20T11:31:50.332812+0000","last_active":"2024-04-20T11:31:51.333404+0000","last_peered":"2024-04-20T11:31:51.333404+0000","last_clean":"2024-04-20T11:31:51.333404+0000","last_became_active":"2024-04-20T11:31:50.331836+0000","last_became_peered":"2024-04-20T11:31:50.331836+0000","last_unstale":"2024-04-20T11:31:51.333404+0000","last_undegraded":"2024-04-20T11:31:51.333404+0000","last_fullsized":"2024-04-20T11:31:51.333404+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:05:09.098414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.317303+0000","last_change":"2024-04-20T11:31:49.286829+0000","last_active":"2024-04-20T11:31:51.317303+0000","last_peered":"2024-04-20T11:31:51.317303+0000","last_clean":"2024-04-20T11:31:51.317303+0000","last_became_active":"2024-04-20T11:31:49.285716+0000","last_became_peered":"2024-04-20T11:31:49.285716+0000","last_unstale":"2024-04-20T11:31:51.317303+0000","last_undegraded":"2024-04-20T11:31:51.317303+0000","last_fullsized":"2024-04-20T11:31:51.317303+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_clean_scrub_stamp":"2024-04-20T11:31:48.241496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:20:49.686853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:52.071473+0000","last_change":"2024-04-20T11:31:50.318154+0000","last_active":"2024-04-20T11:31:52.071473+0000","last_peered":"2024-04-20T11:31:52.071473+0000","last_clean":"2024-04-20T11:31:52.071473+0000","last_became_active":"2024-04-20T11:31:50.317714+0000","last_became_peered":"2024-04-20T11:31:50.317714+0000","last_unstale":"2024-04-20T11:31:52.071473+0000","last_undegraded":"2024-04-20T11:31:52.071473+0000","last_fullsized":"2024-04-20T11:31:52.071473+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:37:14.188276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"70'8","reported_seq":22,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:52.142377+0000","last_change":"2024-04-20T11:31:49.300103+0000","last_active":"2024-04-20T11:31:52.142377+0000","last_peered":"2024-04-20T11:31:52.142377+0000","last_clean":"2024-04-20T11:31:52.142377+0000","last_became_active":"2024-04-20T11:31:49.299818+0000","last_became_peered":"2024-04-20T11:31:49.299818+0000","last_unstale":"2024-04-20T11:31:52.142377+0000","last_undegraded":"2024-04-20T11:31:52.142377+0000","last_fullsized":"2024-04-20T11:31:52.142377+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_clean_scrub_stamp":"2024-04-20T11:31:48.241496+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:28:04.100663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:52.144646+0000","last_change":"2024-04-20T11:31:50.320441+0000","last_active":"2024-04-20T11:31:52.144646+0000","last_peered":"2024-04-20T11:31:52.144646+0000","last_clean":"2024-04-20T11:31:52.144646+0000","last_became_active":"2024-04-20T11:31:50.320064+0000","last_became_peered":"2024-04-20T11:31:50.320064+0000","last_unstale":"2024-04-20T11:31:52.144646+0000","last_undegraded":"2024-04-20T11:31:52.144646+0000","last_fullsized":"2024-04-20T11:31:52.144646+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:14:01.991182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"70'1","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.412982+0000","last_change":"2024-04-20T11:31:49.299871+0000","last_active":"2024-04-20T11:31:51.412982+0000","last_peered":"2024-04-20T11:31:51.412982+0000","last_clean":"2024-04-20T11:31:51.412982+0000","last_became_active":"2024-04-20T11:31:49.298727+0000","last_became_peered":"2024-04-20T11:31:49.298727+0000","last_unstale":"2024-04-20T11:31:51.412982+0000","last_undegraded":"2024-04-20T11:31:51.412982+0000","last_fullsized":"2024-04-20T11:31:51.412982+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_clean_scrub_stamp":"2024-04-20T11:31:48.241496+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:15:59.087516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:52.144864+0000","last_change":"2024-04-20T11:31:50.325834+0000","last_active":"2024-04-20T11:31:52.144864+0000","last_peered":"2024-04-20T11:31:52.144864+0000","last_clean":"2024-04-20T11:31:52.144864+0000","last_became_active":"2024-04-20T11:31:50.325261+0000","last_became_peered":"2024-04-20T11:31:50.325261+0000","last_unstale":"2024-04-20T11:31:52.144864+0000","last_undegraded":"2024-04-20T11:31:52.144864+0000","last_fullsized":"2024-04-20T11:31:52.144864+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:40:15.054162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"70'2","reported_seq":16,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:52.144674+0000","last_change":"2024-04-20T11:31:49.305964+0000","last_active":"2024-04-20T11:31:52.144674+0000","last_peered":"2024-04-20T11:31:52.144674+0000","last_clean":"2024-04-20T11:31:52.144674+0000","last_became_active":"2024-04-20T11:31:49.305614+0000","last_became_peered":"2024-04-20T11:31:49.305614+0000","last_unstale":"2024-04-20T11:31:52.144674+0000","last_undegraded":"2024-04-20T11:31:52.144674+0000","last_fullsized":"2024-04-20T11:31:52.144674+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_clean_scrub_stamp":"2024-04-20T11:31:48.241496+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:51:19.490704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.319028+0000","last_change":"2024-04-20T11:31:50.313100+0000","last_active":"2024-04-20T11:31:51.319028+0000","last_peered":"2024-04-20T11:31:51.319028+0000","last_clean":"2024-04-20T11:31:51.319028+0000","last_became_active":"2024-04-20T11:31:50.311305+0000","last_became_peered":"2024-04-20T11:31:50.311305+0000","last_unstale":"2024-04-20T11:31:51.319028+0000","last_undegraded":"2024-04-20T11:31:51.319028+0000","last_fullsized":"2024-04-20T11:31:51.319028+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:51:20.503298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"70'1","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.412604+0000","last_change":"2024-04-20T11:31:49.305428+0000","last_active":"2024-04-20T11:31:51.412604+0000","last_peered":"2024-04-20T11:31:51.412604+0000","last_clean":"2024-04-20T11:31:51.412604+0000","last_became_active":"2024-04-20T11:31:49.305001+0000","last_became_peered":"2024-04-20T11:31:49.305001+0000","last_unstale":"2024-04-20T11:31:51.412604+0000","last_undegraded":"2024-04-20T11:31:51.412604+0000","last_fullsized":"2024-04-20T11:31:51.412604+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_clean_scrub_stamp":"2024-04-20T11:31:48.241496+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:28:04.100663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:52.072330+0000","last_change":"2024-04-20T11:31:50.319726+0000","last_active":"2024-04-20T11:31:52.072330+0000","last_peered":"2024-04-20T11:31:52.072330+0000","last_clean":"2024-04-20T11:31:52.072330+0000","last_became_active":"2024-04-20T11:31:50.319411+0000","last_became_peered":"2024-04-20T11:31:50.319411+0000","last_unstale":"2024-04-20T11:31:52.072330+0000","last_undegraded":"2024-04-20T11:31:52.072330+0000","last_fullsized":"2024-04-20T11:31:52.072330+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:40:46.698325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:52.143858+0000","last_change":"2024-04-20T11:31:49.306476+0000","last_active":"2024-04-20T11:31:52.143858+0000","last_peered":"2024-04-20T11:31:52.143858+0000","last_clean":"2024-04-20T11:31:52.143858+0000","last_became_active":"2024-04-20T11:31:49.306151+0000","last_became_peered":"2024-04-20T11:31:49.306151+0000","last_unstale":"2024-04-20T11:31:52.143858+0000","last_undegraded":"2024-04-20T11:31:52.143858+0000","last_fullsized":"2024-04-20T11:31:52.143858+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_clean_scrub_stamp":"2024-04-20T11:31:48.241496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:24:30.970676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.320073+0000","last_change":"2024-04-20T11:31:50.312195+0000","last_active":"2024-04-20T11:31:51.320073+0000","last_peered":"2024-04-20T11:31:51.320073+0000","last_clean":"2024-04-20T11:31:51.320073+0000","last_became_active":"2024-04-20T11:31:50.311289+0000","last_became_peered":"2024-04-20T11:31:50.311289+0000","last_unstale":"2024-04-20T11:31:51.320073+0000","last_undegraded":"2024-04-20T11:31:51.320073+0000","last_fullsized":"2024-04-20T11:31:51.320073+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:34:47.042949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.412504+0000","last_change":"2024-04-20T11:31:49.302373+0000","last_active":"2024-04-20T11:31:51.412504+0000","last_peered":"2024-04-20T11:31:51.412504+0000","last_clean":"2024-04-20T11:31:51.412504+0000","last_became_active":"2024-04-20T11:31:49.301685+0000","last_became_peered":"2024-04-20T11:31:49.301685+0000","last_unstale":"2024-04-20T11:31:51.412504+0000","last_undegraded":"2024-04-20T11:31:51.412504+0000","last_fullsized":"2024-04-20T11:31:51.412504+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_clean_scrub_stamp":"2024-04-20T11:31:48.241496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:43:34.162743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.319787+0000","last_change":"2024-04-20T11:31:50.324173+0000","last_active":"2024-04-20T11:31:51.319787+0000","last_peered":"2024-04-20T11:31:51.319787+0000","last_clean":"2024-04-20T11:31:51.319787+0000","last_became_active":"2024-04-20T11:31:50.323748+0000","last_became_peered":"2024-04-20T11:31:50.323748+0000","last_unstale":"2024-04-20T11:31:51.319787+0000","last_undegraded":"2024-04-20T11:31:51.319787+0000","last_fullsized":"2024-04-20T11:31:51.319787+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:13:51.696049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.317774+0000","last_change":"2024-04-20T11:31:49.290048+0000","last_active":"2024-04-20T11:31:51.317774+0000","last_peered":"2024-04-20T11:31:51.317774+0000","last_clean":"2024-04-20T11:31:51.317774+0000","last_became_active":"2024-04-20T11:31:49.288399+0000","last_became_peered":"2024-04-20T11:31:49.288399+0000","last_unstale":"2024-04-20T11:31:51.317774+0000","last_undegraded":"2024-04-20T11:31:51.317774+0000","last_fullsized":"2024-04-20T11:31:51.317774+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_clean_scrub_stamp":"2024-04-20T11:31:48.241496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:38:34.637894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.319605+0000","last_change":"2024-04-20T11:31:50.339749+0000","last_active":"2024-04-20T11:31:51.319605+0000","last_peered":"2024-04-20T11:31:51.319605+0000","last_clean":"2024-04-20T11:31:51.319605+0000","last_became_active":"2024-04-20T11:31:50.339445+0000","last_became_peered":"2024-04-20T11:31:50.339445+0000","last_unstale":"2024-04-20T11:31:51.319605+0000","last_undegraded":"2024-04-20T11:31:51.319605+0000","last_fullsized":"2024-04-20T11:31:51.319605+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:36:35.730829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.332529+0000","last_change":"2024-04-20T11:31:49.289202+0000","last_active":"2024-04-20T11:31:51.332529+0000","last_peered":"2024-04-20T11:31:51.332529+0000","last_clean":"2024-04-20T11:31:51.332529+0000","last_became_active":"2024-04-20T11:31:49.288904+0000","last_became_peered":"2024-04-20T11:31:49.288904+0000","last_unstale":"2024-04-20T11:31:51.332529+0000","last_undegraded":"2024-04-20T11:31:51.332529+0000","last_fullsized":"2024-04-20T11:31:51.332529+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_clean_scrub_stamp":"2024-04-20T11:31:48.241496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:55:53.461405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.333307+0000","last_change":"2024-04-20T11:31:50.316459+0000","last_active":"2024-04-20T11:31:51.333307+0000","last_peered":"2024-04-20T11:31:51.333307+0000","last_clean":"2024-04-20T11:31:51.333307+0000","last_became_active":"2024-04-20T11:31:50.316146+0000","last_became_peered":"2024-04-20T11:31:50.316146+0000","last_unstale":"2024-04-20T11:31:51.333307+0000","last_undegraded":"2024-04-20T11:31:51.333307+0000","last_fullsized":"2024-04-20T11:31:51.333307+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:47:37.822407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"70'2","reported_seq":16,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.412900+0000","last_change":"2024-04-20T11:31:49.297024+0000","last_active":"2024-04-20T11:31:51.412900+0000","last_peered":"2024-04-20T11:31:51.412900+0000","last_clean":"2024-04-20T11:31:51.412900+0000","last_became_active":"2024-04-20T11:31:49.296579+0000","last_became_peered":"2024-04-20T11:31:49.296579+0000","last_unstale":"2024-04-20T11:31:51.412900+0000","last_undegraded":"2024-04-20T11:31:51.412900+0000","last_fullsized":"2024-04-20T11:31:51.412900+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_clean_scrub_stamp":"2024-04-20T11:31:48.241496+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:58:13.811653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.317457+0000","last_change":"2024-04-20T11:31:50.313760+0000","last_active":"2024-04-20T11:31:51.317457+0000","last_peered":"2024-04-20T11:31:51.317457+0000","last_clean":"2024-04-20T11:31:51.317457+0000","last_became_active":"2024-04-20T11:31:50.313446+0000","last_became_peered":"2024-04-20T11:31:50.313446+0000","last_unstale":"2024-04-20T11:31:51.317457+0000","last_undegraded":"2024-04-20T11:31:51.317457+0000","last_fullsized":"2024-04-20T11:31:51.317457+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:28:39.542749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"71'1","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.331489+0000","last_change":"2024-04-20T11:31:49.291746+0000","last_active":"2024-04-20T11:31:51.331489+0000","last_peered":"2024-04-20T11:31:51.331489+0000","last_clean":"2024-04-20T11:31:51.331489+0000","last_became_active":"2024-04-20T11:31:49.291308+0000","last_became_peered":"2024-04-20T11:31:49.291308+0000","last_unstale":"2024-04-20T11:31:51.331489+0000","last_undegraded":"2024-04-20T11:31:51.331489+0000","last_fullsized":"2024-04-20T11:31:51.331489+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_clean_scrub_stamp":"2024-04-20T11:31:48.241496+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:07:46.748648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:52.143583+0000","last_change":"2024-04-20T11:31:50.331466+0000","last_active":"2024-04-20T11:31:52.143583+0000","last_peered":"2024-04-20T11:31:52.143583+0000","last_clean":"2024-04-20T11:31:52.143583+0000","last_became_active":"2024-04-20T11:31:50.331103+0000","last_became_peered":"2024-04-20T11:31:50.331103+0000","last_unstale":"2024-04-20T11:31:52.143583+0000","last_undegraded":"2024-04-20T11:31:52.143583+0000","last_fullsized":"2024-04-20T11:31:52.143583+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:25:20.200757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:52.072643+0000","last_change":"2024-04-20T11:31:49.287119+0000","last_active":"2024-04-20T11:31:52.072643+0000","last_peered":"2024-04-20T11:31:52.072643+0000","last_clean":"2024-04-20T11:31:52.072643+0000","last_became_active":"2024-04-20T11:31:49.286840+0000","last_became_peered":"2024-04-20T11:31:49.286840+0000","last_unstale":"2024-04-20T11:31:52.072643+0000","last_undegraded":"2024-04-20T11:31:52.072643+0000","last_fullsized":"2024-04-20T11:31:52.072643+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_clean_scrub_stamp":"2024-04-20T11:31:48.241496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:28:11.318796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:52.143383+0000","last_change":"2024-04-20T11:31:50.332447+0000","last_active":"2024-04-20T11:31:52.143383+0000","last_peered":"2024-04-20T11:31:52.143383+0000","last_clean":"2024-04-20T11:31:52.143383+0000","last_became_active":"2024-04-20T11:31:50.328942+0000","last_became_peered":"2024-04-20T11:31:50.328942+0000","last_unstale":"2024-04-20T11:31:52.143383+0000","last_undegraded":"2024-04-20T11:31:52.143383+0000","last_fullsized":"2024-04-20T11:31:52.143383+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:43:53.681553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.333168+0000","last_change":"2024-04-20T11:31:49.301336+0000","last_active":"2024-04-20T11:31:51.333168+0000","last_peered":"2024-04-20T11:31:51.333168+0000","last_clean":"2024-04-20T11:31:51.333168+0000","last_became_active":"2024-04-20T11:31:49.301018+0000","last_became_peered":"2024-04-20T11:31:49.301018+0000","last_unstale":"2024-04-20T11:31:51.333168+0000","last_undegraded":"2024-04-20T11:31:51.333168+0000","last_fullsized":"2024-04-20T11:31:51.333168+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_clean_scrub_stamp":"2024-04-20T11:31:48.241496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:33:28.414911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:52.071917+0000","last_change":"2024-04-20T11:31:50.315308+0000","last_active":"2024-04-20T11:31:52.071917+0000","last_peered":"2024-04-20T11:31:52.071917+0000","last_clean":"2024-04-20T11:31:52.071917+0000","last_became_active":"2024-04-20T11:31:50.314606+0000","last_became_peered":"2024-04-20T11:31:50.314606+0000","last_unstale":"2024-04-20T11:31:52.071917+0000","last_undegraded":"2024-04-20T11:31:52.071917+0000","last_fullsized":"2024-04-20T11:31:52.071917+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:43:43.143395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:52.071824+0000","last_change":"2024-04-20T11:31:49.282480+0000","last_active":"2024-04-20T11:31:52.071824+0000","last_peered":"2024-04-20T11:31:52.071824+0000","last_clean":"2024-04-20T11:31:52.071824+0000","last_became_active":"2024-04-20T11:31:49.282211+0000","last_became_peered":"2024-04-20T11:31:49.282211+0000","last_unstale":"2024-04-20T11:31:52.071824+0000","last_undegraded":"2024-04-20T11:31:52.071824+0000","last_fullsized":"2024-04-20T11:31:52.071824+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_clean_scrub_stamp":"2024-04-20T11:31:48.241496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:21:17.698002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.411955+0000","last_change":"2024-04-20T11:31:50.339536+0000","last_active":"2024-04-20T11:31:51.411955+0000","last_peered":"2024-04-20T11:31:51.411955+0000","last_clean":"2024-04-20T11:31:51.411955+0000","last_became_active":"2024-04-20T11:31:50.339190+0000","last_became_peered":"2024-04-20T11:31:50.339190+0000","last_unstale":"2024-04-20T11:31:51.411955+0000","last_undegraded":"2024-04-20T11:31:51.411955+0000","last_fullsized":"2024-04-20T11:31:51.411955+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:39:08.485420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.318001+0000","last_change":"2024-04-20T11:31:49.303460+0000","last_active":"2024-04-20T11:31:51.318001+0000","last_peered":"2024-04-20T11:31:51.318001+0000","last_clean":"2024-04-20T11:31:51.318001+0000","last_became_active":"2024-04-20T11:31:49.303195+0000","last_became_peered":"2024-04-20T11:31:49.303195+0000","last_unstale":"2024-04-20T11:31:51.318001+0000","last_undegraded":"2024-04-20T11:31:51.318001+0000","last_fullsized":"2024-04-20T11:31:51.318001+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_clean_scrub_stamp":"2024-04-20T11:31:48.241496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:43:34.162743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.317839+0000","last_change":"2024-04-20T11:31:50.315834+0000","last_active":"2024-04-20T11:31:51.317839+0000","last_peered":"2024-04-20T11:31:51.317839+0000","last_clean":"2024-04-20T11:31:51.317839+0000","last_became_active":"2024-04-20T11:31:50.314796+0000","last_became_peered":"2024-04-20T11:31:50.314796+0000","last_unstale":"2024-04-20T11:31:51.317839+0000","last_undegraded":"2024-04-20T11:31:51.317839+0000","last_fullsized":"2024-04-20T11:31:51.317839+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:57:55.800700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:52.143215+0000","last_change":"2024-04-20T11:31:49.298984+0000","last_active":"2024-04-20T11:31:52.143215+0000","last_peered":"2024-04-20T11:31:52.143215+0000","last_clean":"2024-04-20T11:31:52.143215+0000","last_became_active":"2024-04-20T11:31:49.298683+0000","last_became_peered":"2024-04-20T11:31:49.298683+0000","last_unstale":"2024-04-20T11:31:52.143215+0000","last_undegraded":"2024-04-20T11:31:52.143215+0000","last_fullsized":"2024-04-20T11:31:52.143215+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_clean_scrub_stamp":"2024-04-20T11:31:48.241496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:51:59.343839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.411680+0000","last_change":"2024-04-20T11:31:50.336409+0000","last_active":"2024-04-20T11:31:51.411680+0000","last_peered":"2024-04-20T11:31:51.411680+0000","last_clean":"2024-04-20T11:31:51.411680+0000","last_became_active":"2024-04-20T11:31:50.331829+0000","last_became_peered":"2024-04-20T11:31:50.331829+0000","last_unstale":"2024-04-20T11:31:51.411680+0000","last_undegraded":"2024-04-20T11:31:51.411680+0000","last_fullsized":"2024-04-20T11:31:51.411680+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:20:50.699448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.412164+0000","last_change":"2024-04-20T11:31:49.299441+0000","last_active":"2024-04-20T11:31:51.412164+0000","last_peered":"2024-04-20T11:31:51.412164+0000","last_clean":"2024-04-20T11:31:51.412164+0000","last_became_active":"2024-04-20T11:31:49.299154+0000","last_became_peered":"2024-04-20T11:31:49.299154+0000","last_unstale":"2024-04-20T11:31:51.412164+0000","last_undegraded":"2024-04-20T11:31:51.412164+0000","last_fullsized":"2024-04-20T11:31:51.412164+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_clean_scrub_stamp":"2024-04-20T11:31:48.241496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:08:44.355167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.320576+0000","last_change":"2024-04-20T11:31:50.331634+0000","last_active":"2024-04-20T11:31:51.320576+0000","last_peered":"2024-04-20T11:31:51.320576+0000","last_clean":"2024-04-20T11:31:51.320576+0000","last_became_active":"2024-04-20T11:31:50.331197+0000","last_became_peered":"2024-04-20T11:31:50.331197+0000","last_unstale":"2024-04-20T11:31:51.320576+0000","last_undegraded":"2024-04-20T11:31:51.320576+0000","last_fullsized":"2024-04-20T11:31:51.320576+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:57:09.923909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"70'2","reported_seq":16,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.411920+0000","last_change":"2024-04-20T11:31:49.303190+0000","last_active":"2024-04-20T11:31:51.411920+0000","last_peered":"2024-04-20T11:31:51.411920+0000","last_clean":"2024-04-20T11:31:51.411920+0000","last_became_active":"2024-04-20T11:31:49.301475+0000","last_became_peered":"2024-04-20T11:31:49.301475+0000","last_unstale":"2024-04-20T11:31:51.411920+0000","last_undegraded":"2024-04-20T11:31:51.411920+0000","last_fullsized":"2024-04-20T11:31:51.411920+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_clean_scrub_stamp":"2024-04-20T11:31:48.241496+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:19:33.664899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":261,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:52.142028+0000","last_change":"2024-04-20T11:26:48.661897+0000","last_active":"2024-04-20T11:31:52.142028+0000","last_peered":"2024-04-20T11:31:52.142028+0000","last_clean":"2024-04-20T11:31:52.142028+0000","last_became_active":"2024-04-20T11:26:48.656535+0000","last_became_peered":"2024-04-20T11:26:48.656535+0000","last_unstale":"2024-04-20T11:31:52.142028+0000","last_undegraded":"2024-04-20T11:31:52.142028+0000","last_fullsized":"2024-04-20T11:31:52.142028+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:47.543004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:47.543004+0000","last_clean_scrub_stamp":"2024-04-20T11:26:47.543004+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:58:35.527105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574914,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32604,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339236,"statfs":{"total":96636764160,"available":96603377664,"internally_reserved":0,"allocated":1454080,"data_stored":949697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574911,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32032,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339808,"statfs":{"total":96636764160,"available":96603963392,"internally_reserved":0,"allocated":872448,"data_stored":365105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574913,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32036,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339804,"statfs":{"total":96636764160,"available":96603959296,"internally_reserved":0,"allocated":868352,"data_stored":365103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574911,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32020,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339820,"statfs":{"total":96636764160,"available":96603975680,"internally_reserved":0,"allocated":856064,"data_stored":358783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574912,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32040,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339800,"statfs":{"total":96636764160,"available":96603955200,"internally_reserved":0,"allocated":872448,"data_stored":365125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574913,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32036,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339804,"statfs":{"total":96636764160,"available":96603959296,"internally_reserved":0,"allocated":876544,"data_stored":366625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574911,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32052,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339788,"statfs":{"total":96636764160,"available":96603942912,"internally_reserved":0,"allocated":884736,"data_stored":367241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574912,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32612,"kb_used_data":1424,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339228,"statfs":{"total":96636764160,"available":96603369472,"internally_reserved":0,"allocated":1458176,"data_stored":955381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T11:31:54.746 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T11:31:54.985 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.985+0000 7f8087e00640 1 Processor -- start 2024-04-20T11:31:54.985 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.985+0000 7f8087e00640 1 -- start start 2024-04-20T11:31:54.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.987+0000 7f8087e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f80880b5fe0 0x7f80880d2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:54.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.987+0000 7f8087e00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80880d2de0 con 0x7f808815f850 2024-04-20T11:31:54.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.987+0000 7f8087e00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80880d30c0 con 0x7f80880cec90 2024-04-20T11:31:54.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.987+0000 7f8087e00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f80880d39f0 con 0x7f80880b5fe0 2024-04-20T11:31:54.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.987+0000 7f807e000640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f808815f850 0x7f80880b63c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:56412/0 (socket says 172.21.15.152:56412) 2024-04-20T11:31:54.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.987+0000 7f807e000640 1 -- 172.21.15.152:0/4269697443 learned_addr learned my addr 172.21.15.152:0/4269697443 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:31:54.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.988+0000 7f8086a00640 1 --2- 172.21.15.152:0/4269697443 >> v2:172.21.15.175:3300/0 conn(0x7f80880b5fe0 0x7f80880d2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:54.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.988+0000 7f8086000640 1 -- 172.21.15.152:0/4269697443 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 550838304 0 0) 0x7f80880d2de0 con 0x7f808815f850 2024-04-20T11:31:54.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.988+0000 7f8086000640 1 -- 172.21.15.152:0/4269697443 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f806c003520 con 0x7f808815f850 2024-04-20T11:31:54.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.988+0000 7f8086000640 1 -- 172.21.15.152:0/4269697443 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3807128091 0 0) 0x7f806c003520 con 0x7f808815f850 2024-04-20T11:31:54.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.988+0000 7f8086a00640 1 -- 172.21.15.152:0/4269697443 >> v1:172.21.15.175:6789/0 conn(0x7f80880cec90 legacy=0x7f80880cf0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:54.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.988+0000 7f8086a00640 1 -- 172.21.15.152:0/4269697443 >> v1:172.21.15.152:6789/0 conn(0x7f808815f850 legacy=0x7f80880b63c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:54.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.988+0000 7f8086a00640 1 -- 172.21.15.152:0/4269697443 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80880d33a0 con 0x7f80880b5fe0 2024-04-20T11:31:54.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.990+0000 7f8086a00640 1 --2- 172.21.15.152:0/4269697443 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f80880b5fe0 0x7f80880d2710 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f80780097a0 tx=0x7f807802b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e0a50ebf66695df server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:54.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.990+0000 7f8086000640 1 -- 172.21.15.152:0/4269697443 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8078037070 con 0x7f80880b5fe0 2024-04-20T11:31:54.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.990+0000 7f8086000640 1 -- 172.21.15.152:0/4269697443 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8078030510 con 0x7f80880b5fe0 2024-04-20T11:31:54.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.990+0000 7f8086000640 1 -- 172.21.15.152:0/4269697443 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8070001670 con 0x7f80880b5fe0 2024-04-20T11:31:54.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.991+0000 7f8087e00640 1 -- 172.21.15.152:0/4269697443 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f80880b5fe0 msgr2=0x7f80880d2710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:54.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.991+0000 7f8087e00640 1 --2- 172.21.15.152:0/4269697443 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f80880b5fe0 0x7f80880d2710 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f80780097a0 tx=0x7f807802b790 comp rx=0 tx=0).stop 2024-04-20T11:31:54.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.991+0000 7f8087e00640 1 -- 172.21.15.152:0/4269697443 shutdown_connections 2024-04-20T11:31:54.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.991+0000 7f8087e00640 1 --2- 172.21.15.152:0/4269697443 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f80880b5fe0 0x7f80880d2710 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:54.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.991+0000 7f8087e00640 1 -- 172.21.15.152:0/4269697443 >> 172.21.15.152:0/4269697443 conn(0x7f808808bfd0 msgr2=0x7f80880d4560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:54.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.991+0000 7f8087e00640 1 -- 172.21.15.152:0/4269697443 shutdown_connections 2024-04-20T11:31:54.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.991+0000 7f8087e00640 1 -- 172.21.15.152:0/4269697443 wait complete. 2024-04-20T11:31:54.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.993+0000 7f8087e00640 1 Processor -- start 2024-04-20T11:31:54.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.993+0000 7f8087e00640 1 -- start start 2024-04-20T11:31:54.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.994+0000 7f8087e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f80880b5fe0 0x7f80880c8070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:54.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.994+0000 7f8087e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f80880cec90 0x7f80880cdd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:54.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.994+0000 7f8087e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f808815f850 0x7f808814f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:54.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.994+0000 7f8087e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f808815eaa0 con 0x7f808815f850 2024-04-20T11:31:54.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.994+0000 7f8087e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f808815fc70 con 0x7f80880cec90 2024-04-20T11:31:54.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.994+0000 7f8087e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f808815ec20 con 0x7f80880b5fe0 2024-04-20T11:31:54.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.994+0000 7f8087400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f808815f850 0x7f808814f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:54.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.994+0000 7f8087400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f808815f850 0x7f808814f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36646/0 (socket says 172.21.15.152:36646) 2024-04-20T11:31:54.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.994+0000 7f8087400640 1 -- 172.21.15.152:0/3357681313 learned_addr learned my addr 172.21.15.152:0/3357681313 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:54.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.995+0000 7f807e000640 1 --2- 172.21.15.152:0/3357681313 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f80880cec90 0x7f80880cdd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:54.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.995+0000 7f8086a00640 1 --2- 172.21.15.152:0/3357681313 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f80880b5fe0 0x7f80880c8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:54.995 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.995+0000 7f8087400640 1 -- 172.21.15.152:0/3357681313 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f80880b5fe0 msgr2=0x7f80880c8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:54.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.995+0000 7f8087400640 1 --2- 172.21.15.152:0/3357681313 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f80880b5fe0 0x7f80880c8070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:54.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.995+0000 7f8087400640 1 -- 172.21.15.152:0/3357681313 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f80880cec90 msgr2=0x7f80880cdd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:54.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.995+0000 7f8087400640 1 --2- 172.21.15.152:0/3357681313 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f80880cec90 0x7f80880cdd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:54.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.995+0000 7f8087400640 1 -- 172.21.15.152:0/3357681313 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80880cc240 con 0x7f808815f850 2024-04-20T11:31:54.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.995+0000 7f8087400640 1 --2- 172.21.15.152:0/3357681313 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f808815f850 0x7f808814f810 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f808000c7a0 tx=0x7f808000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:54.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.995+0000 7f807fe00640 1 -- 172.21.15.152:0/3357681313 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8080003b70 con 0x7f808815f850 2024-04-20T11:31:54.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.995+0000 7f807fe00640 1 -- 172.21.15.152:0/3357681313 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8080003d30 con 0x7f808815f850 2024-04-20T11:31:54.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.995+0000 7f807fe00640 1 -- 172.21.15.152:0/3357681313 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f808000aa90 con 0x7f808815f850 2024-04-20T11:31:54.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.995+0000 7f8087e00640 1 -- 172.21.15.152:0/3357681313 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80880cc520 con 0x7f808815f850 2024-04-20T11:31:54.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.996+0000 7f8087e00640 1 -- 172.21.15.152:0/3357681313 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80880b27a0 con 0x7f808815f850 2024-04-20T11:31:54.997 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.997+0000 7f8087e00640 1 -- 172.21.15.152:0/3357681313 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8088075e60 con 0x7f808815f850 2024-04-20T11:31:54.997 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.998+0000 7f807fe00640 1 -- 172.21.15.152:0/3357681313 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 11) ==== 91346+0+0 (secure 0 0 0) 0x7f808000ac30 con 0x7f808815f850 2024-04-20T11:31:54.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.998+0000 7f807fe00640 1 --2- 172.21.15.152:0/3357681313 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f806406cc50 0x7f806406f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:54.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.999+0000 7f807fe00640 1 -- 172.21.15.152:0/3357681313 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5709+0+0 (secure 0 0 0) 0x7f808008c100 con 0x7f808815f850 2024-04-20T11:31:54.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:54.999+0000 7f8086a00640 1 --2- 172.21.15.152:0/3357681313 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f806406cc50 0x7f806406f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:54.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.000+0000 7f8086a00640 1 --2- 172.21.15.152:0/3357681313 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f806406cc50 0x7f806406f110 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f8078034000 tx=0x7f807802f000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:55.002 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.003+0000 7f807fe00640 1 -- 172.21.15.152:0/3357681313 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f808005a070 con 0x7f808815f850 2024-04-20T11:31:55.199 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.199+0000 7f8087e00640 1 -- 172.21.15.152:0/3357681313 --> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f80880ba4e0 con 0x7f806406cc50 2024-04-20T11:31:55.207 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.208+0000 7f807fe00640 1 -- 172.21.15.152:0/3357681313 <== mgr.4113 v2:172.21.15.152:6832/2285110240 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252083 (secure 0 0 0) 0x7f80880ba4e0 con 0x7f806406cc50 2024-04-20T11:31:55.211 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:31:55.212 INFO:teuthology.orchestra.run.smithi152.stderr:dumped all 2024-04-20T11:31:55.214 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.213+0000 7f8087e00640 1 -- 172.21.15.152:0/3357681313 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f806406cc50 msgr2=0x7f806406f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:55.214 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.213+0000 7f8087e00640 1 --2- 172.21.15.152:0/3357681313 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f806406cc50 0x7f806406f110 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f8078034000 tx=0x7f807802f000 comp rx=0 tx=0).stop 2024-04-20T11:31:55.214 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.213+0000 7f8087e00640 1 -- 172.21.15.152:0/3357681313 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f808815f850 msgr2=0x7f808814f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:55.214 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.213+0000 7f8087e00640 1 --2- 172.21.15.152:0/3357681313 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f808815f850 0x7f808814f810 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f808000c7a0 tx=0x7f808000cc60 comp rx=0 tx=0).stop 2024-04-20T11:31:55.214 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.214+0000 7f8087e00640 1 -- 172.21.15.152:0/3357681313 shutdown_connections 2024-04-20T11:31:55.214 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.214+0000 7f8087e00640 1 --2- 172.21.15.152:0/3357681313 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f806406cc50 0x7f806406f110 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:55.214 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.214+0000 7f8087e00640 1 --2- 172.21.15.152:0/3357681313 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f808815f850 0x7f808814f810 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:55.214 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.214+0000 7f8087e00640 1 --2- 172.21.15.152:0/3357681313 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f80880cec90 0x7f80880cdd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:55.214 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.214+0000 7f8087e00640 1 --2- 172.21.15.152:0/3357681313 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f80880b5fe0 0x7f80880c8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:55.214 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.214+0000 7f8087e00640 1 -- 172.21.15.152:0/3357681313 >> 172.21.15.152:0/3357681313 conn(0x7f808808bfd0 msgr2=0x7f808806de70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:55.214 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.214+0000 7f8087e00640 1 -- 172.21.15.152:0/3357681313 shutdown_connections 2024-04-20T11:31:55.215 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.214+0000 7f8087e00640 1 -- 172.21.15.152:0/3357681313 wait complete. 2024-04-20T11:31:55.232 INFO:teuthology.orchestra.run.smithi152.stdout:{"pg_ready":true,"pg_map":{"version":217,"stamp":"2024-04-20T11:31:53.311510+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":148,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":257432,"kb_used_data":7952,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754717288,"statfs":{"total":773094113280,"available":772830502912,"internally_reserved":0,"allocated":8142848,"data_stored":4093060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12746,"internal_metadata":255315510},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.054763"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:52.144504+0000","last_change":"2024-04-20T11:31:50.325580+0000","last_active":"2024-04-20T11:31:52.144504+0000","last_peered":"2024-04-20T11:31:52.144504+0000","last_clean":"2024-04-20T11:31:52.144504+0000","last_became_active":"2024-04-20T11:31:50.324926+0000","last_became_peered":"2024-04-20T11:31:50.324926+0000","last_unstale":"2024-04-20T11:31:52.144504+0000","last_undegraded":"2024-04-20T11:31:52.144504+0000","last_fullsized":"2024-04-20T11:31:52.144504+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:03:24.930741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.413081+0000","last_change":"2024-04-20T11:31:50.328037+0000","last_active":"2024-04-20T11:31:51.413081+0000","last_peered":"2024-04-20T11:31:51.413081+0000","last_clean":"2024-04-20T11:31:51.413081+0000","last_became_active":"2024-04-20T11:31:50.327723+0000","last_became_peered":"2024-04-20T11:31:50.327723+0000","last_unstale":"2024-04-20T11:31:51.413081+0000","last_undegraded":"2024-04-20T11:31:51.413081+0000","last_fullsized":"2024-04-20T11:31:51.413081+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:33:29.427506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.320228+0000","last_change":"2024-04-20T11:31:50.310201+0000","last_active":"2024-04-20T11:31:51.320228+0000","last_peered":"2024-04-20T11:31:51.320228+0000","last_clean":"2024-04-20T11:31:51.320228+0000","last_became_active":"2024-04-20T11:31:50.309302+0000","last_became_peered":"2024-04-20T11:31:50.309302+0000","last_unstale":"2024-04-20T11:31:51.320228+0000","last_undegraded":"2024-04-20T11:31:51.320228+0000","last_fullsized":"2024-04-20T11:31:51.320228+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:37:56.915958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.318473+0000","last_change":"2024-04-20T11:31:50.330202+0000","last_active":"2024-04-20T11:31:51.318473+0000","last_peered":"2024-04-20T11:31:51.318473+0000","last_clean":"2024-04-20T11:31:51.318473+0000","last_became_active":"2024-04-20T11:31:50.329914+0000","last_became_peered":"2024-04-20T11:31:50.329914+0000","last_unstale":"2024-04-20T11:31:51.318473+0000","last_undegraded":"2024-04-20T11:31:51.318473+0000","last_fullsized":"2024-04-20T11:31:51.318473+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:28:05.113257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:52.071888+0000","last_change":"2024-04-20T11:31:50.323466+0000","last_active":"2024-04-20T11:31:52.071888+0000","last_peered":"2024-04-20T11:31:52.071888+0000","last_clean":"2024-04-20T11:31:52.071888+0000","last_became_active":"2024-04-20T11:31:50.323204+0000","last_became_peered":"2024-04-20T11:31:50.323204+0000","last_unstale":"2024-04-20T11:31:52.071888+0000","last_undegraded":"2024-04-20T11:31:52.071888+0000","last_fullsized":"2024-04-20T11:31:52.071888+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:22:41.334900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.411815+0000","last_change":"2024-04-20T11:31:50.341288+0000","last_active":"2024-04-20T11:31:51.411815+0000","last_peered":"2024-04-20T11:31:51.411815+0000","last_clean":"2024-04-20T11:31:51.411815+0000","last_became_active":"2024-04-20T11:31:50.339864+0000","last_became_peered":"2024-04-20T11:31:50.339864+0000","last_unstale":"2024-04-20T11:31:51.411815+0000","last_undegraded":"2024-04-20T11:31:51.411815+0000","last_fullsized":"2024-04-20T11:31:51.411815+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:38:35.650488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.317481+0000","last_change":"2024-04-20T11:31:50.311897+0000","last_active":"2024-04-20T11:31:51.317481+0000","last_peered":"2024-04-20T11:31:51.317481+0000","last_clean":"2024-04-20T11:31:51.317481+0000","last_became_active":"2024-04-20T11:31:50.308939+0000","last_became_peered":"2024-04-20T11:31:50.308939+0000","last_unstale":"2024-04-20T11:31:51.317481+0000","last_undegraded":"2024-04-20T11:31:51.317481+0000","last_fullsized":"2024-04-20T11:31:51.317481+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:14:01.991182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.411848+0000","last_change":"2024-04-20T11:31:50.321035+0000","last_active":"2024-04-20T11:31:51.411848+0000","last_peered":"2024-04-20T11:31:51.411848+0000","last_clean":"2024-04-20T11:31:51.411848+0000","last_became_active":"2024-04-20T11:31:50.320724+0000","last_became_peered":"2024-04-20T11:31:50.320724+0000","last_unstale":"2024-04-20T11:31:51.411848+0000","last_undegraded":"2024-04-20T11:31:51.411848+0000","last_fullsized":"2024-04-20T11:31:51.411848+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:07:47.761243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:52.072428+0000","last_change":"2024-04-20T11:31:50.322571+0000","last_active":"2024-04-20T11:31:52.072428+0000","last_peered":"2024-04-20T11:31:52.072428+0000","last_clean":"2024-04-20T11:31:52.072428+0000","last_became_active":"2024-04-20T11:31:50.321867+0000","last_became_peered":"2024-04-20T11:31:50.321867+0000","last_unstale":"2024-04-20T11:31:52.072428+0000","last_undegraded":"2024-04-20T11:31:52.072428+0000","last_fullsized":"2024-04-20T11:31:52.072428+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:05:36.261388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:52.144364+0000","last_change":"2024-04-20T11:31:50.321085+0000","last_active":"2024-04-20T11:31:52.144364+0000","last_peered":"2024-04-20T11:31:52.144364+0000","last_clean":"2024-04-20T11:31:52.144364+0000","last_became_active":"2024-04-20T11:31:50.320791+0000","last_became_peered":"2024-04-20T11:31:50.320791+0000","last_unstale":"2024-04-20T11:31:52.144364+0000","last_undegraded":"2024-04-20T11:31:52.144364+0000","last_fullsized":"2024-04-20T11:31:52.144364+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:28:39.542749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:52.143127+0000","last_change":"2024-04-20T11:31:50.323050+0000","last_active":"2024-04-20T11:31:52.143127+0000","last_peered":"2024-04-20T11:31:52.143127+0000","last_clean":"2024-04-20T11:31:52.143127+0000","last_became_active":"2024-04-20T11:31:50.322778+0000","last_became_peered":"2024-04-20T11:31:50.322778+0000","last_unstale":"2024-04-20T11:31:52.143127+0000","last_undegraded":"2024-04-20T11:31:52.143127+0000","last_fullsized":"2024-04-20T11:31:52.143127+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:44:55.157437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.318871+0000","last_change":"2024-04-20T11:31:50.315955+0000","last_active":"2024-04-20T11:31:51.318871+0000","last_peered":"2024-04-20T11:31:51.318871+0000","last_clean":"2024-04-20T11:31:51.318871+0000","last_became_active":"2024-04-20T11:31:50.313938+0000","last_became_peered":"2024-04-20T11:31:50.313938+0000","last_unstale":"2024-04-20T11:31:51.318871+0000","last_undegraded":"2024-04-20T11:31:51.318871+0000","last_fullsized":"2024-04-20T11:31:51.318871+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:33:45.092493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.333207+0000","last_change":"2024-04-20T11:31:50.330639+0000","last_active":"2024-04-20T11:31:51.333207+0000","last_peered":"2024-04-20T11:31:51.333207+0000","last_clean":"2024-04-20T11:31:51.333207+0000","last_became_active":"2024-04-20T11:31:50.330276+0000","last_became_peered":"2024-04-20T11:31:50.330276+0000","last_unstale":"2024-04-20T11:31:51.333207+0000","last_undegraded":"2024-04-20T11:31:51.333207+0000","last_fullsized":"2024-04-20T11:31:51.333207+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:38:43.528257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:52.071818+0000","last_change":"2024-04-20T11:31:50.312741+0000","last_active":"2024-04-20T11:31:52.071818+0000","last_peered":"2024-04-20T11:31:52.071818+0000","last_clean":"2024-04-20T11:31:52.071818+0000","last_became_active":"2024-04-20T11:31:50.312315+0000","last_became_peered":"2024-04-20T11:31:50.312315+0000","last_unstale":"2024-04-20T11:31:52.071818+0000","last_undegraded":"2024-04-20T11:31:52.071818+0000","last_fullsized":"2024-04-20T11:31:52.071818+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:48:33.476138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.332656+0000","last_change":"2024-04-20T11:31:50.316408+0000","last_active":"2024-04-20T11:31:51.332656+0000","last_peered":"2024-04-20T11:31:51.332656+0000","last_clean":"2024-04-20T11:31:51.332656+0000","last_became_active":"2024-04-20T11:31:50.316099+0000","last_became_peered":"2024-04-20T11:31:50.316099+0000","last_unstale":"2024-04-20T11:31:51.332656+0000","last_undegraded":"2024-04-20T11:31:51.332656+0000","last_fullsized":"2024-04-20T11:31:51.332656+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:49:05.141517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:52.142303+0000","last_change":"2024-04-20T11:31:50.342752+0000","last_active":"2024-04-20T11:31:52.142303+0000","last_peered":"2024-04-20T11:31:52.142303+0000","last_clean":"2024-04-20T11:31:52.142303+0000","last_became_active":"2024-04-20T11:31:50.342243+0000","last_became_peered":"2024-04-20T11:31:50.342243+0000","last_unstale":"2024-04-20T11:31:52.142303+0000","last_undegraded":"2024-04-20T11:31:52.142303+0000","last_fullsized":"2024-04-20T11:31:52.142303+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:30:00.566230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:52.072688+0000","last_change":"2024-04-20T11:31:50.321768+0000","last_active":"2024-04-20T11:31:52.072688+0000","last_peered":"2024-04-20T11:31:52.072688+0000","last_clean":"2024-04-20T11:31:52.072688+0000","last_became_active":"2024-04-20T11:31:50.319324+0000","last_became_peered":"2024-04-20T11:31:50.319324+0000","last_unstale":"2024-04-20T11:31:52.072688+0000","last_undegraded":"2024-04-20T11:31:52.072688+0000","last_fullsized":"2024-04-20T11:31:52.072688+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:26:59.369353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.318705+0000","last_change":"2024-04-20T11:31:50.330892+0000","last_active":"2024-04-20T11:31:51.318705+0000","last_peered":"2024-04-20T11:31:51.318705+0000","last_clean":"2024-04-20T11:31:51.318705+0000","last_became_active":"2024-04-20T11:31:50.330542+0000","last_became_peered":"2024-04-20T11:31:50.330542+0000","last_unstale":"2024-04-20T11:31:51.318705+0000","last_undegraded":"2024-04-20T11:31:51.318705+0000","last_fullsized":"2024-04-20T11:31:51.318705+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:52:00.356433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.412538+0000","last_change":"2024-04-20T11:31:50.338478+0000","last_active":"2024-04-20T11:31:51.412538+0000","last_peered":"2024-04-20T11:31:51.412538+0000","last_clean":"2024-04-20T11:31:51.412538+0000","last_became_active":"2024-04-20T11:31:50.329097+0000","last_became_peered":"2024-04-20T11:31:50.329097+0000","last_unstale":"2024-04-20T11:31:51.412538+0000","last_undegraded":"2024-04-20T11:31:51.412538+0000","last_fullsized":"2024-04-20T11:31:51.412538+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:30:00.566230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.333267+0000","last_change":"2024-04-20T11:31:50.320732+0000","last_active":"2024-04-20T11:31:51.333267+0000","last_peered":"2024-04-20T11:31:51.333267+0000","last_clean":"2024-04-20T11:31:51.333267+0000","last_became_active":"2024-04-20T11:31:50.320369+0000","last_became_peered":"2024-04-20T11:31:50.320369+0000","last_unstale":"2024-04-20T11:31:51.333267+0000","last_undegraded":"2024-04-20T11:31:51.333267+0000","last_fullsized":"2024-04-20T11:31:51.333267+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:18:48.610827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:52.143451+0000","last_change":"2024-04-20T11:31:50.320620+0000","last_active":"2024-04-20T11:31:52.143451+0000","last_peered":"2024-04-20T11:31:52.143451+0000","last_clean":"2024-04-20T11:31:52.143451+0000","last_became_active":"2024-04-20T11:31:50.319513+0000","last_became_peered":"2024-04-20T11:31:50.319513+0000","last_unstale":"2024-04-20T11:31:52.143451+0000","last_undegraded":"2024-04-20T11:31:52.143451+0000","last_fullsized":"2024-04-20T11:31:52.143451+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:33:29.427506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:52.143342+0000","last_change":"2024-04-20T11:31:50.323667+0000","last_active":"2024-04-20T11:31:52.143342+0000","last_peered":"2024-04-20T11:31:52.143342+0000","last_clean":"2024-04-20T11:31:52.143342+0000","last_became_active":"2024-04-20T11:31:50.323294+0000","last_became_peered":"2024-04-20T11:31:50.323294+0000","last_unstale":"2024-04-20T11:31:52.143342+0000","last_undegraded":"2024-04-20T11:31:52.143342+0000","last_fullsized":"2024-04-20T11:31:52.143342+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:55:54.473999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:52.142561+0000","last_change":"2024-04-20T11:31:50.329025+0000","last_active":"2024-04-20T11:31:52.142561+0000","last_peered":"2024-04-20T11:31:52.142561+0000","last_clean":"2024-04-20T11:31:52.142561+0000","last_became_active":"2024-04-20T11:31:50.328741+0000","last_became_peered":"2024-04-20T11:31:50.328741+0000","last_unstale":"2024-04-20T11:31:52.142561+0000","last_undegraded":"2024-04-20T11:31:52.142561+0000","last_fullsized":"2024-04-20T11:31:52.142561+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:47:37.822407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:52.142088+0000","last_change":"2024-04-20T11:31:50.331933+0000","last_active":"2024-04-20T11:31:52.142088+0000","last_peered":"2024-04-20T11:31:52.142088+0000","last_clean":"2024-04-20T11:31:52.142088+0000","last_became_active":"2024-04-20T11:31:50.331730+0000","last_became_peered":"2024-04-20T11:31:50.331730+0000","last_unstale":"2024-04-20T11:31:52.142088+0000","last_undegraded":"2024-04-20T11:31:52.142088+0000","last_fullsized":"2024-04-20T11:31:52.142088+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:38:43.528257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:52.144059+0000","last_change":"2024-04-20T11:31:50.341703+0000","last_active":"2024-04-20T11:31:52.144059+0000","last_peered":"2024-04-20T11:31:52.144059+0000","last_clean":"2024-04-20T11:31:52.144059+0000","last_became_active":"2024-04-20T11:31:50.341371+0000","last_became_peered":"2024-04-20T11:31:50.341371+0000","last_unstale":"2024-04-20T11:31:52.144059+0000","last_undegraded":"2024-04-20T11:31:52.144059+0000","last_fullsized":"2024-04-20T11:31:52.144059+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:29:41.481297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.318323+0000","last_change":"2024-04-20T11:31:50.329710+0000","last_active":"2024-04-20T11:31:51.318323+0000","last_peered":"2024-04-20T11:31:51.318323+0000","last_clean":"2024-04-20T11:31:51.318323+0000","last_became_active":"2024-04-20T11:31:50.329406+0000","last_became_peered":"2024-04-20T11:31:50.329406+0000","last_unstale":"2024-04-20T11:31:51.318323+0000","last_undegraded":"2024-04-20T11:31:51.318323+0000","last_fullsized":"2024-04-20T11:31:51.318323+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:03:26.976387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.320657+0000","last_change":"2024-04-20T11:31:50.314168+0000","last_active":"2024-04-20T11:31:51.320657+0000","last_peered":"2024-04-20T11:31:51.320657+0000","last_clean":"2024-04-20T11:31:51.320657+0000","last_became_active":"2024-04-20T11:31:50.313856+0000","last_became_peered":"2024-04-20T11:31:50.313856+0000","last_unstale":"2024-04-20T11:31:51.320657+0000","last_undegraded":"2024-04-20T11:31:51.320657+0000","last_fullsized":"2024-04-20T11:31:51.320657+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:25:20.652223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:52.144808+0000","last_change":"2024-04-20T11:31:50.321610+0000","last_active":"2024-04-20T11:31:52.144808+0000","last_peered":"2024-04-20T11:31:52.144808+0000","last_clean":"2024-04-20T11:31:52.144808+0000","last_became_active":"2024-04-20T11:31:50.321214+0000","last_became_peered":"2024-04-20T11:31:50.321214+0000","last_unstale":"2024-04-20T11:31:52.144808+0000","last_undegraded":"2024-04-20T11:31:52.144808+0000","last_fullsized":"2024-04-20T11:31:52.144808+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:45:40.359700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.319845+0000","last_change":"2024-04-20T11:31:50.319039+0000","last_active":"2024-04-20T11:31:51.319845+0000","last_peered":"2024-04-20T11:31:51.319845+0000","last_clean":"2024-04-20T11:31:51.319845+0000","last_became_active":"2024-04-20T11:31:50.318707+0000","last_became_peered":"2024-04-20T11:31:50.318707+0000","last_unstale":"2024-04-20T11:31:51.319845+0000","last_undegraded":"2024-04-20T11:31:51.319845+0000","last_fullsized":"2024-04-20T11:31:51.319845+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:55:29.750493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.317877+0000","last_change":"2024-04-20T11:31:50.335499+0000","last_active":"2024-04-20T11:31:51.317877+0000","last_peered":"2024-04-20T11:31:51.317877+0000","last_clean":"2024-04-20T11:31:51.317877+0000","last_became_active":"2024-04-20T11:31:50.332697+0000","last_became_peered":"2024-04-20T11:31:50.332697+0000","last_unstale":"2024-04-20T11:31:51.317877+0000","last_undegraded":"2024-04-20T11:31:51.317877+0000","last_fullsized":"2024-04-20T11:31:51.317877+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:01:47.414565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.317530+0000","last_change":"2024-04-20T11:31:50.313012+0000","last_active":"2024-04-20T11:31:51.317530+0000","last_peered":"2024-04-20T11:31:51.317530+0000","last_clean":"2024-04-20T11:31:51.317530+0000","last_became_active":"2024-04-20T11:31:50.312581+0000","last_became_peered":"2024-04-20T11:31:50.312581+0000","last_unstale":"2024-04-20T11:31:51.317530+0000","last_undegraded":"2024-04-20T11:31:51.317530+0000","last_fullsized":"2024-04-20T11:31:51.317530+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:45:40.359700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.412052+0000","last_change":"2024-04-20T11:31:50.330657+0000","last_active":"2024-04-20T11:31:51.412052+0000","last_peered":"2024-04-20T11:31:51.412052+0000","last_clean":"2024-04-20T11:31:51.412052+0000","last_became_active":"2024-04-20T11:31:50.330371+0000","last_became_peered":"2024-04-20T11:31:50.330371+0000","last_unstale":"2024-04-20T11:31:51.412052+0000","last_undegraded":"2024-04-20T11:31:51.412052+0000","last_fullsized":"2024-04-20T11:31:51.412052+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:44:00.584674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:52.144132+0000","last_change":"2024-04-20T11:31:50.341124+0000","last_active":"2024-04-20T11:31:52.144132+0000","last_peered":"2024-04-20T11:31:52.144132+0000","last_clean":"2024-04-20T11:31:52.144132+0000","last_became_active":"2024-04-20T11:31:50.340851+0000","last_became_peered":"2024-04-20T11:31:50.340851+0000","last_unstale":"2024-04-20T11:31:52.144132+0000","last_undegraded":"2024-04-20T11:31:52.144132+0000","last_fullsized":"2024-04-20T11:31:52.144132+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:46:50.995840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"70'4","reported_seq":18,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:52.145009+0000","last_change":"2024-04-20T11:31:49.302275+0000","last_active":"2024-04-20T11:31:52.145009+0000","last_peered":"2024-04-20T11:31:52.145009+0000","last_clean":"2024-04-20T11:31:52.145009+0000","last_became_active":"2024-04-20T11:31:49.301374+0000","last_became_peered":"2024-04-20T11:31:49.301374+0000","last_unstale":"2024-04-20T11:31:52.145009+0000","last_undegraded":"2024-04-20T11:31:52.145009+0000","last_fullsized":"2024-04-20T11:31:52.145009+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_clean_scrub_stamp":"2024-04-20T11:31:48.241496+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:29:59.553636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:52.072791+0000","last_change":"2024-04-20T11:31:50.322007+0000","last_active":"2024-04-20T11:31:52.072791+0000","last_peered":"2024-04-20T11:31:52.072791+0000","last_clean":"2024-04-20T11:31:52.072791+0000","last_became_active":"2024-04-20T11:31:50.319703+0000","last_became_peered":"2024-04-20T11:31:50.319703+0000","last_unstale":"2024-04-20T11:31:52.072791+0000","last_undegraded":"2024-04-20T11:31:52.072791+0000","last_fullsized":"2024-04-20T11:31:52.072791+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:23:28.381927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"70'1","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.333424+0000","last_change":"2024-04-20T11:31:49.288268+0000","last_active":"2024-04-20T11:31:51.333424+0000","last_peered":"2024-04-20T11:31:51.333424+0000","last_clean":"2024-04-20T11:31:51.333424+0000","last_became_active":"2024-04-20T11:31:49.287810+0000","last_became_peered":"2024-04-20T11:31:49.287810+0000","last_unstale":"2024-04-20T11:31:51.333424+0000","last_undegraded":"2024-04-20T11:31:51.333424+0000","last_fullsized":"2024-04-20T11:31:51.333424+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_clean_scrub_stamp":"2024-04-20T11:31:48.241496+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:22:21.850770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.412123+0000","last_change":"2024-04-20T11:31:50.339803+0000","last_active":"2024-04-20T11:31:51.412123+0000","last_peered":"2024-04-20T11:31:51.412123+0000","last_clean":"2024-04-20T11:31:51.412123+0000","last_became_active":"2024-04-20T11:31:50.339500+0000","last_became_peered":"2024-04-20T11:31:50.339500+0000","last_unstale":"2024-04-20T11:31:51.412123+0000","last_undegraded":"2024-04-20T11:31:51.412123+0000","last_fullsized":"2024-04-20T11:31:51.412123+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:24:31.983271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"70'2","reported_seq":16,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:52.144120+0000","last_change":"2024-04-20T11:31:49.304762+0000","last_active":"2024-04-20T11:31:52.144120+0000","last_peered":"2024-04-20T11:31:52.144120+0000","last_clean":"2024-04-20T11:31:52.144120+0000","last_became_active":"2024-04-20T11:31:49.304416+0000","last_became_peered":"2024-04-20T11:31:49.304416+0000","last_unstale":"2024-04-20T11:31:52.144120+0000","last_undegraded":"2024-04-20T11:31:52.144120+0000","last_fullsized":"2024-04-20T11:31:52.144120+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_clean_scrub_stamp":"2024-04-20T11:31:48.241496+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:52:40.794561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:52.142972+0000","last_change":"2024-04-20T11:31:50.329274+0000","last_active":"2024-04-20T11:31:52.142972+0000","last_peered":"2024-04-20T11:31:52.142972+0000","last_clean":"2024-04-20T11:31:52.142972+0000","last_became_active":"2024-04-20T11:31:50.328551+0000","last_became_peered":"2024-04-20T11:31:50.328551+0000","last_unstale":"2024-04-20T11:31:52.142972+0000","last_undegraded":"2024-04-20T11:31:52.142972+0000","last_fullsized":"2024-04-20T11:31:52.142972+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:05:09.098414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"70'2","reported_seq":16,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.412861+0000","last_change":"2024-04-20T11:31:49.297823+0000","last_active":"2024-04-20T11:31:51.412861+0000","last_peered":"2024-04-20T11:31:51.412861+0000","last_clean":"2024-04-20T11:31:51.412861+0000","last_became_active":"2024-04-20T11:31:49.297474+0000","last_became_peered":"2024-04-20T11:31:49.297474+0000","last_unstale":"2024-04-20T11:31:51.412861+0000","last_undegraded":"2024-04-20T11:31:51.412861+0000","last_fullsized":"2024-04-20T11:31:51.412861+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_clean_scrub_stamp":"2024-04-20T11:31:48.241496+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:35:13.023313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.318295+0000","last_change":"2024-04-20T11:31:50.311206+0000","last_active":"2024-04-20T11:31:51.318295+0000","last_peered":"2024-04-20T11:31:51.318295+0000","last_clean":"2024-04-20T11:31:51.318295+0000","last_became_active":"2024-04-20T11:31:50.310613+0000","last_became_peered":"2024-04-20T11:31:50.310613+0000","last_unstale":"2024-04-20T11:31:51.318295+0000","last_undegraded":"2024-04-20T11:31:51.318295+0000","last_fullsized":"2024-04-20T11:31:51.318295+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:49:05.141517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"70'1","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.319957+0000","last_change":"2024-04-20T11:31:49.302337+0000","last_active":"2024-04-20T11:31:51.319957+0000","last_peered":"2024-04-20T11:31:51.319957+0000","last_clean":"2024-04-20T11:31:51.319957+0000","last_became_active":"2024-04-20T11:31:49.301130+0000","last_became_peered":"2024-04-20T11:31:49.301130+0000","last_unstale":"2024-04-20T11:31:51.319957+0000","last_undegraded":"2024-04-20T11:31:51.319957+0000","last_fullsized":"2024-04-20T11:31:51.319957+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_clean_scrub_stamp":"2024-04-20T11:31:48.241496+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:25:19.188163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:52.143901+0000","last_change":"2024-04-20T11:31:50.322874+0000","last_active":"2024-04-20T11:31:52.143901+0000","last_peered":"2024-04-20T11:31:52.143901+0000","last_clean":"2024-04-20T11:31:52.143901+0000","last_became_active":"2024-04-20T11:31:50.322543+0000","last_became_peered":"2024-04-20T11:31:50.322543+0000","last_unstale":"2024-04-20T11:31:52.143901+0000","last_undegraded":"2024-04-20T11:31:52.143901+0000","last_fullsized":"2024-04-20T11:31:52.143901+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:22:22.863365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"70'1","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.412218+0000","last_change":"2024-04-20T11:31:49.302612+0000","last_active":"2024-04-20T11:31:51.412218+0000","last_peered":"2024-04-20T11:31:51.412218+0000","last_clean":"2024-04-20T11:31:51.412218+0000","last_became_active":"2024-04-20T11:31:49.302072+0000","last_became_peered":"2024-04-20T11:31:49.302072+0000","last_unstale":"2024-04-20T11:31:51.412218+0000","last_undegraded":"2024-04-20T11:31:51.412218+0000","last_fullsized":"2024-04-20T11:31:51.412218+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_clean_scrub_stamp":"2024-04-20T11:31:48.241496+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:23:27.369333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:52.071280+0000","last_change":"2024-04-20T11:31:50.319154+0000","last_active":"2024-04-20T11:31:52.071280+0000","last_peered":"2024-04-20T11:31:52.071280+0000","last_clean":"2024-04-20T11:31:52.071280+0000","last_became_active":"2024-04-20T11:31:50.318848+0000","last_became_peered":"2024-04-20T11:31:50.318848+0000","last_unstale":"2024-04-20T11:31:52.071280+0000","last_undegraded":"2024-04-20T11:31:52.071280+0000","last_fullsized":"2024-04-20T11:31:52.071280+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:50:07.040083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.317690+0000","last_change":"2024-04-20T11:31:49.302035+0000","last_active":"2024-04-20T11:31:51.317690+0000","last_peered":"2024-04-20T11:31:51.317690+0000","last_clean":"2024-04-20T11:31:51.317690+0000","last_became_active":"2024-04-20T11:31:49.301766+0000","last_became_peered":"2024-04-20T11:31:49.301766+0000","last_unstale":"2024-04-20T11:31:51.317690+0000","last_undegraded":"2024-04-20T11:31:51.317690+0000","last_fullsized":"2024-04-20T11:31:51.317690+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_clean_scrub_stamp":"2024-04-20T11:31:48.241496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:47:47.119919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.332858+0000","last_change":"2024-04-20T11:31:50.335909+0000","last_active":"2024-04-20T11:31:51.332858+0000","last_peered":"2024-04-20T11:31:51.332858+0000","last_clean":"2024-04-20T11:31:51.332858+0000","last_became_active":"2024-04-20T11:31:50.335467+0000","last_became_peered":"2024-04-20T11:31:50.335467+0000","last_unstale":"2024-04-20T11:31:51.332858+0000","last_undegraded":"2024-04-20T11:31:51.332858+0000","last_fullsized":"2024-04-20T11:31:51.332858+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:03:26.976387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:52.071358+0000","last_change":"2024-04-20T11:31:49.283257+0000","last_active":"2024-04-20T11:31:52.071358+0000","last_peered":"2024-04-20T11:31:52.071358+0000","last_clean":"2024-04-20T11:31:52.071358+0000","last_became_active":"2024-04-20T11:31:49.282924+0000","last_became_peered":"2024-04-20T11:31:49.282924+0000","last_unstale":"2024-04-20T11:31:52.071358+0000","last_undegraded":"2024-04-20T11:31:52.071358+0000","last_fullsized":"2024-04-20T11:31:52.071358+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_clean_scrub_stamp":"2024-04-20T11:31:48.241496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:52:57.575105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.318544+0000","last_change":"2024-04-20T11:31:50.335086+0000","last_active":"2024-04-20T11:31:51.318544+0000","last_peered":"2024-04-20T11:31:51.318544+0000","last_clean":"2024-04-20T11:31:51.318544+0000","last_became_active":"2024-04-20T11:31:50.334427+0000","last_became_peered":"2024-04-20T11:31:50.334427+0000","last_unstale":"2024-04-20T11:31:51.318544+0000","last_undegraded":"2024-04-20T11:31:51.318544+0000","last_fullsized":"2024-04-20T11:31:51.318544+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:38:43.528257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:52.144345+0000","last_change":"2024-04-20T11:31:49.305014+0000","last_active":"2024-04-20T11:31:52.144345+0000","last_peered":"2024-04-20T11:31:52.144345+0000","last_clean":"2024-04-20T11:31:52.144345+0000","last_became_active":"2024-04-20T11:31:49.304677+0000","last_became_peered":"2024-04-20T11:31:49.304677+0000","last_unstale":"2024-04-20T11:31:52.144345+0000","last_undegraded":"2024-04-20T11:31:52.144345+0000","last_fullsized":"2024-04-20T11:31:52.144345+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_clean_scrub_stamp":"2024-04-20T11:31:48.241496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:33:44.079899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:52.144721+0000","last_change":"2024-04-20T11:31:50.340468+0000","last_active":"2024-04-20T11:31:52.144721+0000","last_peered":"2024-04-20T11:31:52.144721+0000","last_clean":"2024-04-20T11:31:52.144721+0000","last_became_active":"2024-04-20T11:31:50.340166+0000","last_became_peered":"2024-04-20T11:31:50.340166+0000","last_unstale":"2024-04-20T11:31:52.144721+0000","last_undegraded":"2024-04-20T11:31:52.144721+0000","last_fullsized":"2024-04-20T11:31:52.144721+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:46:12.753823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.413078+0000","last_change":"2024-04-20T11:31:49.300474+0000","last_active":"2024-04-20T11:31:51.413078+0000","last_peered":"2024-04-20T11:31:51.413078+0000","last_clean":"2024-04-20T11:31:51.413078+0000","last_became_active":"2024-04-20T11:31:49.300179+0000","last_became_peered":"2024-04-20T11:31:49.300179+0000","last_unstale":"2024-04-20T11:31:51.413078+0000","last_undegraded":"2024-04-20T11:31:51.413078+0000","last_fullsized":"2024-04-20T11:31:51.413078+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_clean_scrub_stamp":"2024-04-20T11:31:48.241496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:57:54.788106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.412584+0000","last_change":"2024-04-20T11:31:50.344041+0000","last_active":"2024-04-20T11:31:51.412584+0000","last_peered":"2024-04-20T11:31:51.412584+0000","last_clean":"2024-04-20T11:31:51.412584+0000","last_became_active":"2024-04-20T11:31:50.343739+0000","last_became_peered":"2024-04-20T11:31:50.343739+0000","last_unstale":"2024-04-20T11:31:51.412584+0000","last_undegraded":"2024-04-20T11:31:51.412584+0000","last_fullsized":"2024-04-20T11:31:51.412584+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:55:54.473999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"70'3","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.320777+0000","last_change":"2024-04-20T11:31:49.282326+0000","last_active":"2024-04-20T11:31:51.320777+0000","last_peered":"2024-04-20T11:31:51.320777+0000","last_clean":"2024-04-20T11:31:51.320777+0000","last_became_active":"2024-04-20T11:31:49.282026+0000","last_became_peered":"2024-04-20T11:31:49.282026+0000","last_unstale":"2024-04-20T11:31:51.320777+0000","last_undegraded":"2024-04-20T11:31:51.320777+0000","last_fullsized":"2024-04-20T11:31:51.320777+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_clean_scrub_stamp":"2024-04-20T11:31:48.241496+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:46:49.983246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:52.072403+0000","last_change":"2024-04-20T11:31:50.310816+0000","last_active":"2024-04-20T11:31:52.072403+0000","last_peered":"2024-04-20T11:31:52.072403+0000","last_clean":"2024-04-20T11:31:52.072403+0000","last_became_active":"2024-04-20T11:31:50.310455+0000","last_became_peered":"2024-04-20T11:31:50.310455+0000","last_unstale":"2024-04-20T11:31:52.072403+0000","last_undegraded":"2024-04-20T11:31:52.072403+0000","last_fullsized":"2024-04-20T11:31:52.072403+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:24:11.551245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"70'6","reported_seq":20,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.319119+0000","last_change":"2024-04-20T11:31:49.290097+0000","last_active":"2024-04-20T11:31:51.319119+0000","last_peered":"2024-04-20T11:31:51.319119+0000","last_clean":"2024-04-20T11:31:51.319119+0000","last_became_active":"2024-04-20T11:31:49.289048+0000","last_became_peered":"2024-04-20T11:31:49.289048+0000","last_unstale":"2024-04-20T11:31:51.319119+0000","last_undegraded":"2024-04-20T11:31:51.319119+0000","last_fullsized":"2024-04-20T11:31:51.319119+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_clean_scrub_stamp":"2024-04-20T11:31:48.241496+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:35:13.023313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:52.142516+0000","last_change":"2024-04-20T11:31:50.326503+0000","last_active":"2024-04-20T11:31:52.142516+0000","last_peered":"2024-04-20T11:31:52.142516+0000","last_clean":"2024-04-20T11:31:52.142516+0000","last_became_active":"2024-04-20T11:31:50.318766+0000","last_became_peered":"2024-04-20T11:31:50.318766+0000","last_unstale":"2024-04-20T11:31:52.142516+0000","last_undegraded":"2024-04-20T11:31:52.142516+0000","last_fullsized":"2024-04-20T11:31:52.142516+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:18:48.610827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"70'2","reported_seq":16,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.318542+0000","last_change":"2024-04-20T11:31:49.296194+0000","last_active":"2024-04-20T11:31:51.318542+0000","last_peered":"2024-04-20T11:31:51.318542+0000","last_clean":"2024-04-20T11:31:51.318542+0000","last_became_active":"2024-04-20T11:31:49.295903+0000","last_became_peered":"2024-04-20T11:31:49.295903+0000","last_unstale":"2024-04-20T11:31:51.318542+0000","last_undegraded":"2024-04-20T11:31:51.318542+0000","last_fullsized":"2024-04-20T11:31:51.318542+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_clean_scrub_stamp":"2024-04-20T11:31:48.241496+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:19:33.664899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.317847+0000","last_change":"2024-04-20T11:31:50.319043+0000","last_active":"2024-04-20T11:31:51.317847+0000","last_peered":"2024-04-20T11:31:51.317847+0000","last_clean":"2024-04-20T11:31:51.317847+0000","last_became_active":"2024-04-20T11:31:50.318727+0000","last_became_peered":"2024-04-20T11:31:50.318727+0000","last_unstale":"2024-04-20T11:31:51.317847+0000","last_undegraded":"2024-04-20T11:31:51.317847+0000","last_fullsized":"2024-04-20T11:31:51.317847+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:40:25.463996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.412423+0000","last_change":"2024-04-20T11:31:49.300082+0000","last_active":"2024-04-20T11:31:51.412423+0000","last_peered":"2024-04-20T11:31:51.412423+0000","last_clean":"2024-04-20T11:31:51.412423+0000","last_became_active":"2024-04-20T11:31:49.298466+0000","last_became_peered":"2024-04-20T11:31:49.298466+0000","last_unstale":"2024-04-20T11:31:51.412423+0000","last_undegraded":"2024-04-20T11:31:51.412423+0000","last_fullsized":"2024-04-20T11:31:51.412423+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_clean_scrub_stamp":"2024-04-20T11:31:48.241496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:47:47.119919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.333404+0000","last_change":"2024-04-20T11:31:50.332812+0000","last_active":"2024-04-20T11:31:51.333404+0000","last_peered":"2024-04-20T11:31:51.333404+0000","last_clean":"2024-04-20T11:31:51.333404+0000","last_became_active":"2024-04-20T11:31:50.331836+0000","last_became_peered":"2024-04-20T11:31:50.331836+0000","last_unstale":"2024-04-20T11:31:51.333404+0000","last_undegraded":"2024-04-20T11:31:51.333404+0000","last_fullsized":"2024-04-20T11:31:51.333404+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:05:09.098414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.317303+0000","last_change":"2024-04-20T11:31:49.286829+0000","last_active":"2024-04-20T11:31:51.317303+0000","last_peered":"2024-04-20T11:31:51.317303+0000","last_clean":"2024-04-20T11:31:51.317303+0000","last_became_active":"2024-04-20T11:31:49.285716+0000","last_became_peered":"2024-04-20T11:31:49.285716+0000","last_unstale":"2024-04-20T11:31:51.317303+0000","last_undegraded":"2024-04-20T11:31:51.317303+0000","last_fullsized":"2024-04-20T11:31:51.317303+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_clean_scrub_stamp":"2024-04-20T11:31:48.241496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:20:49.686853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:52.071473+0000","last_change":"2024-04-20T11:31:50.318154+0000","last_active":"2024-04-20T11:31:52.071473+0000","last_peered":"2024-04-20T11:31:52.071473+0000","last_clean":"2024-04-20T11:31:52.071473+0000","last_became_active":"2024-04-20T11:31:50.317714+0000","last_became_peered":"2024-04-20T11:31:50.317714+0000","last_unstale":"2024-04-20T11:31:52.071473+0000","last_undegraded":"2024-04-20T11:31:52.071473+0000","last_fullsized":"2024-04-20T11:31:52.071473+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:37:14.188276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"70'8","reported_seq":22,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:52.142377+0000","last_change":"2024-04-20T11:31:49.300103+0000","last_active":"2024-04-20T11:31:52.142377+0000","last_peered":"2024-04-20T11:31:52.142377+0000","last_clean":"2024-04-20T11:31:52.142377+0000","last_became_active":"2024-04-20T11:31:49.299818+0000","last_became_peered":"2024-04-20T11:31:49.299818+0000","last_unstale":"2024-04-20T11:31:52.142377+0000","last_undegraded":"2024-04-20T11:31:52.142377+0000","last_fullsized":"2024-04-20T11:31:52.142377+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_clean_scrub_stamp":"2024-04-20T11:31:48.241496+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:28:04.100663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:52.144646+0000","last_change":"2024-04-20T11:31:50.320441+0000","last_active":"2024-04-20T11:31:52.144646+0000","last_peered":"2024-04-20T11:31:52.144646+0000","last_clean":"2024-04-20T11:31:52.144646+0000","last_became_active":"2024-04-20T11:31:50.320064+0000","last_became_peered":"2024-04-20T11:31:50.320064+0000","last_unstale":"2024-04-20T11:31:52.144646+0000","last_undegraded":"2024-04-20T11:31:52.144646+0000","last_fullsized":"2024-04-20T11:31:52.144646+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:14:01.991182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"70'1","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.412982+0000","last_change":"2024-04-20T11:31:49.299871+0000","last_active":"2024-04-20T11:31:51.412982+0000","last_peered":"2024-04-20T11:31:51.412982+0000","last_clean":"2024-04-20T11:31:51.412982+0000","last_became_active":"2024-04-20T11:31:49.298727+0000","last_became_peered":"2024-04-20T11:31:49.298727+0000","last_unstale":"2024-04-20T11:31:51.412982+0000","last_undegraded":"2024-04-20T11:31:51.412982+0000","last_fullsized":"2024-04-20T11:31:51.412982+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_clean_scrub_stamp":"2024-04-20T11:31:48.241496+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:15:59.087516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:52.144864+0000","last_change":"2024-04-20T11:31:50.325834+0000","last_active":"2024-04-20T11:31:52.144864+0000","last_peered":"2024-04-20T11:31:52.144864+0000","last_clean":"2024-04-20T11:31:52.144864+0000","last_became_active":"2024-04-20T11:31:50.325261+0000","last_became_peered":"2024-04-20T11:31:50.325261+0000","last_unstale":"2024-04-20T11:31:52.144864+0000","last_undegraded":"2024-04-20T11:31:52.144864+0000","last_fullsized":"2024-04-20T11:31:52.144864+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:40:15.054162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"70'2","reported_seq":16,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:52.144674+0000","last_change":"2024-04-20T11:31:49.305964+0000","last_active":"2024-04-20T11:31:52.144674+0000","last_peered":"2024-04-20T11:31:52.144674+0000","last_clean":"2024-04-20T11:31:52.144674+0000","last_became_active":"2024-04-20T11:31:49.305614+0000","last_became_peered":"2024-04-20T11:31:49.305614+0000","last_unstale":"2024-04-20T11:31:52.144674+0000","last_undegraded":"2024-04-20T11:31:52.144674+0000","last_fullsized":"2024-04-20T11:31:52.144674+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_clean_scrub_stamp":"2024-04-20T11:31:48.241496+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:51:19.490704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.319028+0000","last_change":"2024-04-20T11:31:50.313100+0000","last_active":"2024-04-20T11:31:51.319028+0000","last_peered":"2024-04-20T11:31:51.319028+0000","last_clean":"2024-04-20T11:31:51.319028+0000","last_became_active":"2024-04-20T11:31:50.311305+0000","last_became_peered":"2024-04-20T11:31:50.311305+0000","last_unstale":"2024-04-20T11:31:51.319028+0000","last_undegraded":"2024-04-20T11:31:51.319028+0000","last_fullsized":"2024-04-20T11:31:51.319028+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:51:20.503298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"70'1","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.412604+0000","last_change":"2024-04-20T11:31:49.305428+0000","last_active":"2024-04-20T11:31:51.412604+0000","last_peered":"2024-04-20T11:31:51.412604+0000","last_clean":"2024-04-20T11:31:51.412604+0000","last_became_active":"2024-04-20T11:31:49.305001+0000","last_became_peered":"2024-04-20T11:31:49.305001+0000","last_unstale":"2024-04-20T11:31:51.412604+0000","last_undegraded":"2024-04-20T11:31:51.412604+0000","last_fullsized":"2024-04-20T11:31:51.412604+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_clean_scrub_stamp":"2024-04-20T11:31:48.241496+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:28:04.100663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:52.072330+0000","last_change":"2024-04-20T11:31:50.319726+0000","last_active":"2024-04-20T11:31:52.072330+0000","last_peered":"2024-04-20T11:31:52.072330+0000","last_clean":"2024-04-20T11:31:52.072330+0000","last_became_active":"2024-04-20T11:31:50.319411+0000","last_became_peered":"2024-04-20T11:31:50.319411+0000","last_unstale":"2024-04-20T11:31:52.072330+0000","last_undegraded":"2024-04-20T11:31:52.072330+0000","last_fullsized":"2024-04-20T11:31:52.072330+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:40:46.698325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:52.143858+0000","last_change":"2024-04-20T11:31:49.306476+0000","last_active":"2024-04-20T11:31:52.143858+0000","last_peered":"2024-04-20T11:31:52.143858+0000","last_clean":"2024-04-20T11:31:52.143858+0000","last_became_active":"2024-04-20T11:31:49.306151+0000","last_became_peered":"2024-04-20T11:31:49.306151+0000","last_unstale":"2024-04-20T11:31:52.143858+0000","last_undegraded":"2024-04-20T11:31:52.143858+0000","last_fullsized":"2024-04-20T11:31:52.143858+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_clean_scrub_stamp":"2024-04-20T11:31:48.241496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:24:30.970676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.320073+0000","last_change":"2024-04-20T11:31:50.312195+0000","last_active":"2024-04-20T11:31:51.320073+0000","last_peered":"2024-04-20T11:31:51.320073+0000","last_clean":"2024-04-20T11:31:51.320073+0000","last_became_active":"2024-04-20T11:31:50.311289+0000","last_became_peered":"2024-04-20T11:31:50.311289+0000","last_unstale":"2024-04-20T11:31:51.320073+0000","last_undegraded":"2024-04-20T11:31:51.320073+0000","last_fullsized":"2024-04-20T11:31:51.320073+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:34:47.042949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.412504+0000","last_change":"2024-04-20T11:31:49.302373+0000","last_active":"2024-04-20T11:31:51.412504+0000","last_peered":"2024-04-20T11:31:51.412504+0000","last_clean":"2024-04-20T11:31:51.412504+0000","last_became_active":"2024-04-20T11:31:49.301685+0000","last_became_peered":"2024-04-20T11:31:49.301685+0000","last_unstale":"2024-04-20T11:31:51.412504+0000","last_undegraded":"2024-04-20T11:31:51.412504+0000","last_fullsized":"2024-04-20T11:31:51.412504+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_clean_scrub_stamp":"2024-04-20T11:31:48.241496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:43:34.162743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.319787+0000","last_change":"2024-04-20T11:31:50.324173+0000","last_active":"2024-04-20T11:31:51.319787+0000","last_peered":"2024-04-20T11:31:51.319787+0000","last_clean":"2024-04-20T11:31:51.319787+0000","last_became_active":"2024-04-20T11:31:50.323748+0000","last_became_peered":"2024-04-20T11:31:50.323748+0000","last_unstale":"2024-04-20T11:31:51.319787+0000","last_undegraded":"2024-04-20T11:31:51.319787+0000","last_fullsized":"2024-04-20T11:31:51.319787+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:13:51.696049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.317774+0000","last_change":"2024-04-20T11:31:49.290048+0000","last_active":"2024-04-20T11:31:51.317774+0000","last_peered":"2024-04-20T11:31:51.317774+0000","last_clean":"2024-04-20T11:31:51.317774+0000","last_became_active":"2024-04-20T11:31:49.288399+0000","last_became_peered":"2024-04-20T11:31:49.288399+0000","last_unstale":"2024-04-20T11:31:51.317774+0000","last_undegraded":"2024-04-20T11:31:51.317774+0000","last_fullsized":"2024-04-20T11:31:51.317774+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_clean_scrub_stamp":"2024-04-20T11:31:48.241496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:38:34.637894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.319605+0000","last_change":"2024-04-20T11:31:50.339749+0000","last_active":"2024-04-20T11:31:51.319605+0000","last_peered":"2024-04-20T11:31:51.319605+0000","last_clean":"2024-04-20T11:31:51.319605+0000","last_became_active":"2024-04-20T11:31:50.339445+0000","last_became_peered":"2024-04-20T11:31:50.339445+0000","last_unstale":"2024-04-20T11:31:51.319605+0000","last_undegraded":"2024-04-20T11:31:51.319605+0000","last_fullsized":"2024-04-20T11:31:51.319605+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:36:35.730829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.332529+0000","last_change":"2024-04-20T11:31:49.289202+0000","last_active":"2024-04-20T11:31:51.332529+0000","last_peered":"2024-04-20T11:31:51.332529+0000","last_clean":"2024-04-20T11:31:51.332529+0000","last_became_active":"2024-04-20T11:31:49.288904+0000","last_became_peered":"2024-04-20T11:31:49.288904+0000","last_unstale":"2024-04-20T11:31:51.332529+0000","last_undegraded":"2024-04-20T11:31:51.332529+0000","last_fullsized":"2024-04-20T11:31:51.332529+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_clean_scrub_stamp":"2024-04-20T11:31:48.241496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:55:53.461405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.333307+0000","last_change":"2024-04-20T11:31:50.316459+0000","last_active":"2024-04-20T11:31:51.333307+0000","last_peered":"2024-04-20T11:31:51.333307+0000","last_clean":"2024-04-20T11:31:51.333307+0000","last_became_active":"2024-04-20T11:31:50.316146+0000","last_became_peered":"2024-04-20T11:31:50.316146+0000","last_unstale":"2024-04-20T11:31:51.333307+0000","last_undegraded":"2024-04-20T11:31:51.333307+0000","last_fullsized":"2024-04-20T11:31:51.333307+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:47:37.822407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"70'2","reported_seq":16,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.412900+0000","last_change":"2024-04-20T11:31:49.297024+0000","last_active":"2024-04-20T11:31:51.412900+0000","last_peered":"2024-04-20T11:31:51.412900+0000","last_clean":"2024-04-20T11:31:51.412900+0000","last_became_active":"2024-04-20T11:31:49.296579+0000","last_became_peered":"2024-04-20T11:31:49.296579+0000","last_unstale":"2024-04-20T11:31:51.412900+0000","last_undegraded":"2024-04-20T11:31:51.412900+0000","last_fullsized":"2024-04-20T11:31:51.412900+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_clean_scrub_stamp":"2024-04-20T11:31:48.241496+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:58:13.811653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.317457+0000","last_change":"2024-04-20T11:31:50.313760+0000","last_active":"2024-04-20T11:31:51.317457+0000","last_peered":"2024-04-20T11:31:51.317457+0000","last_clean":"2024-04-20T11:31:51.317457+0000","last_became_active":"2024-04-20T11:31:50.313446+0000","last_became_peered":"2024-04-20T11:31:50.313446+0000","last_unstale":"2024-04-20T11:31:51.317457+0000","last_undegraded":"2024-04-20T11:31:51.317457+0000","last_fullsized":"2024-04-20T11:31:51.317457+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:28:39.542749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"71'1","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.331489+0000","last_change":"2024-04-20T11:31:49.291746+0000","last_active":"2024-04-20T11:31:51.331489+0000","last_peered":"2024-04-20T11:31:51.331489+0000","last_clean":"2024-04-20T11:31:51.331489+0000","last_became_active":"2024-04-20T11:31:49.291308+0000","last_became_peered":"2024-04-20T11:31:49.291308+0000","last_unstale":"2024-04-20T11:31:51.331489+0000","last_undegraded":"2024-04-20T11:31:51.331489+0000","last_fullsized":"2024-04-20T11:31:51.331489+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_clean_scrub_stamp":"2024-04-20T11:31:48.241496+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:07:46.748648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:52.143583+0000","last_change":"2024-04-20T11:31:50.331466+0000","last_active":"2024-04-20T11:31:52.143583+0000","last_peered":"2024-04-20T11:31:52.143583+0000","last_clean":"2024-04-20T11:31:52.143583+0000","last_became_active":"2024-04-20T11:31:50.331103+0000","last_became_peered":"2024-04-20T11:31:50.331103+0000","last_unstale":"2024-04-20T11:31:52.143583+0000","last_undegraded":"2024-04-20T11:31:52.143583+0000","last_fullsized":"2024-04-20T11:31:52.143583+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:25:20.200757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:52.072643+0000","last_change":"2024-04-20T11:31:49.287119+0000","last_active":"2024-04-20T11:31:52.072643+0000","last_peered":"2024-04-20T11:31:52.072643+0000","last_clean":"2024-04-20T11:31:52.072643+0000","last_became_active":"2024-04-20T11:31:49.286840+0000","last_became_peered":"2024-04-20T11:31:49.286840+0000","last_unstale":"2024-04-20T11:31:52.072643+0000","last_undegraded":"2024-04-20T11:31:52.072643+0000","last_fullsized":"2024-04-20T11:31:52.072643+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_clean_scrub_stamp":"2024-04-20T11:31:48.241496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:28:11.318796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:52.143383+0000","last_change":"2024-04-20T11:31:50.332447+0000","last_active":"2024-04-20T11:31:52.143383+0000","last_peered":"2024-04-20T11:31:52.143383+0000","last_clean":"2024-04-20T11:31:52.143383+0000","last_became_active":"2024-04-20T11:31:50.328942+0000","last_became_peered":"2024-04-20T11:31:50.328942+0000","last_unstale":"2024-04-20T11:31:52.143383+0000","last_undegraded":"2024-04-20T11:31:52.143383+0000","last_fullsized":"2024-04-20T11:31:52.143383+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:43:53.681553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.333168+0000","last_change":"2024-04-20T11:31:49.301336+0000","last_active":"2024-04-20T11:31:51.333168+0000","last_peered":"2024-04-20T11:31:51.333168+0000","last_clean":"2024-04-20T11:31:51.333168+0000","last_became_active":"2024-04-20T11:31:49.301018+0000","last_became_peered":"2024-04-20T11:31:49.301018+0000","last_unstale":"2024-04-20T11:31:51.333168+0000","last_undegraded":"2024-04-20T11:31:51.333168+0000","last_fullsized":"2024-04-20T11:31:51.333168+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_clean_scrub_stamp":"2024-04-20T11:31:48.241496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:33:28.414911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:52.071917+0000","last_change":"2024-04-20T11:31:50.315308+0000","last_active":"2024-04-20T11:31:52.071917+0000","last_peered":"2024-04-20T11:31:52.071917+0000","last_clean":"2024-04-20T11:31:52.071917+0000","last_became_active":"2024-04-20T11:31:50.314606+0000","last_became_peered":"2024-04-20T11:31:50.314606+0000","last_unstale":"2024-04-20T11:31:52.071917+0000","last_undegraded":"2024-04-20T11:31:52.071917+0000","last_fullsized":"2024-04-20T11:31:52.071917+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:43:43.143395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:52.071824+0000","last_change":"2024-04-20T11:31:49.282480+0000","last_active":"2024-04-20T11:31:52.071824+0000","last_peered":"2024-04-20T11:31:52.071824+0000","last_clean":"2024-04-20T11:31:52.071824+0000","last_became_active":"2024-04-20T11:31:49.282211+0000","last_became_peered":"2024-04-20T11:31:49.282211+0000","last_unstale":"2024-04-20T11:31:52.071824+0000","last_undegraded":"2024-04-20T11:31:52.071824+0000","last_fullsized":"2024-04-20T11:31:52.071824+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_clean_scrub_stamp":"2024-04-20T11:31:48.241496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:21:17.698002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.411955+0000","last_change":"2024-04-20T11:31:50.339536+0000","last_active":"2024-04-20T11:31:51.411955+0000","last_peered":"2024-04-20T11:31:51.411955+0000","last_clean":"2024-04-20T11:31:51.411955+0000","last_became_active":"2024-04-20T11:31:50.339190+0000","last_became_peered":"2024-04-20T11:31:50.339190+0000","last_unstale":"2024-04-20T11:31:51.411955+0000","last_undegraded":"2024-04-20T11:31:51.411955+0000","last_fullsized":"2024-04-20T11:31:51.411955+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:39:08.485420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.318001+0000","last_change":"2024-04-20T11:31:49.303460+0000","last_active":"2024-04-20T11:31:51.318001+0000","last_peered":"2024-04-20T11:31:51.318001+0000","last_clean":"2024-04-20T11:31:51.318001+0000","last_became_active":"2024-04-20T11:31:49.303195+0000","last_became_peered":"2024-04-20T11:31:49.303195+0000","last_unstale":"2024-04-20T11:31:51.318001+0000","last_undegraded":"2024-04-20T11:31:51.318001+0000","last_fullsized":"2024-04-20T11:31:51.318001+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_clean_scrub_stamp":"2024-04-20T11:31:48.241496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:43:34.162743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.317839+0000","last_change":"2024-04-20T11:31:50.315834+0000","last_active":"2024-04-20T11:31:51.317839+0000","last_peered":"2024-04-20T11:31:51.317839+0000","last_clean":"2024-04-20T11:31:51.317839+0000","last_became_active":"2024-04-20T11:31:50.314796+0000","last_became_peered":"2024-04-20T11:31:50.314796+0000","last_unstale":"2024-04-20T11:31:51.317839+0000","last_undegraded":"2024-04-20T11:31:51.317839+0000","last_fullsized":"2024-04-20T11:31:51.317839+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:57:55.800700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:52.143215+0000","last_change":"2024-04-20T11:31:49.298984+0000","last_active":"2024-04-20T11:31:52.143215+0000","last_peered":"2024-04-20T11:31:52.143215+0000","last_clean":"2024-04-20T11:31:52.143215+0000","last_became_active":"2024-04-20T11:31:49.298683+0000","last_became_peered":"2024-04-20T11:31:49.298683+0000","last_unstale":"2024-04-20T11:31:52.143215+0000","last_undegraded":"2024-04-20T11:31:52.143215+0000","last_fullsized":"2024-04-20T11:31:52.143215+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_clean_scrub_stamp":"2024-04-20T11:31:48.241496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:51:59.343839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.411680+0000","last_change":"2024-04-20T11:31:50.336409+0000","last_active":"2024-04-20T11:31:51.411680+0000","last_peered":"2024-04-20T11:31:51.411680+0000","last_clean":"2024-04-20T11:31:51.411680+0000","last_became_active":"2024-04-20T11:31:50.331829+0000","last_became_peered":"2024-04-20T11:31:50.331829+0000","last_unstale":"2024-04-20T11:31:51.411680+0000","last_undegraded":"2024-04-20T11:31:51.411680+0000","last_fullsized":"2024-04-20T11:31:51.411680+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:20:50.699448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.412164+0000","last_change":"2024-04-20T11:31:49.299441+0000","last_active":"2024-04-20T11:31:51.412164+0000","last_peered":"2024-04-20T11:31:51.412164+0000","last_clean":"2024-04-20T11:31:51.412164+0000","last_became_active":"2024-04-20T11:31:49.299154+0000","last_became_peered":"2024-04-20T11:31:49.299154+0000","last_unstale":"2024-04-20T11:31:51.412164+0000","last_undegraded":"2024-04-20T11:31:51.412164+0000","last_fullsized":"2024-04-20T11:31:51.412164+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_clean_scrub_stamp":"2024-04-20T11:31:48.241496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:08:44.355167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.320576+0000","last_change":"2024-04-20T11:31:50.331634+0000","last_active":"2024-04-20T11:31:51.320576+0000","last_peered":"2024-04-20T11:31:51.320576+0000","last_clean":"2024-04-20T11:31:51.320576+0000","last_became_active":"2024-04-20T11:31:50.331197+0000","last_became_peered":"2024-04-20T11:31:50.331197+0000","last_unstale":"2024-04-20T11:31:51.320576+0000","last_undegraded":"2024-04-20T11:31:51.320576+0000","last_fullsized":"2024-04-20T11:31:51.320576+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:49.254090+0000","last_clean_scrub_stamp":"2024-04-20T11:31:49.254090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:57:09.923909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"70'2","reported_seq":16,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:51.411920+0000","last_change":"2024-04-20T11:31:49.303190+0000","last_active":"2024-04-20T11:31:51.411920+0000","last_peered":"2024-04-20T11:31:51.411920+0000","last_clean":"2024-04-20T11:31:51.411920+0000","last_became_active":"2024-04-20T11:31:49.301475+0000","last_became_peered":"2024-04-20T11:31:49.301475+0000","last_unstale":"2024-04-20T11:31:51.411920+0000","last_undegraded":"2024-04-20T11:31:51.411920+0000","last_fullsized":"2024-04-20T11:31:51.411920+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:31:48.241496+0000","last_clean_scrub_stamp":"2024-04-20T11:31:48.241496+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:19:33.664899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":261,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T11:31:52.142028+0000","last_change":"2024-04-20T11:26:48.661897+0000","last_active":"2024-04-20T11:31:52.142028+0000","last_peered":"2024-04-20T11:31:52.142028+0000","last_clean":"2024-04-20T11:31:52.142028+0000","last_became_active":"2024-04-20T11:26:48.656535+0000","last_became_peered":"2024-04-20T11:26:48.656535+0000","last_unstale":"2024-04-20T11:31:52.142028+0000","last_undegraded":"2024-04-20T11:31:52.142028+0000","last_fullsized":"2024-04-20T11:31:52.142028+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:47.543004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:47.543004+0000","last_clean_scrub_stamp":"2024-04-20T11:26:47.543004+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:58:35.527105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574914,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32604,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339236,"statfs":{"total":96636764160,"available":96603377664,"internally_reserved":0,"allocated":1454080,"data_stored":949697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574911,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32032,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339808,"statfs":{"total":96636764160,"available":96603963392,"internally_reserved":0,"allocated":872448,"data_stored":365105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574913,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32036,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339804,"statfs":{"total":96636764160,"available":96603959296,"internally_reserved":0,"allocated":868352,"data_stored":365103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574911,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32020,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339820,"statfs":{"total":96636764160,"available":96603975680,"internally_reserved":0,"allocated":856064,"data_stored":358783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574912,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32040,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339800,"statfs":{"total":96636764160,"available":96603955200,"internally_reserved":0,"allocated":872448,"data_stored":365125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574913,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32036,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339804,"statfs":{"total":96636764160,"available":96603959296,"internally_reserved":0,"allocated":876544,"data_stored":366625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574911,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32052,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339788,"statfs":{"total":96636764160,"available":96603942912,"internally_reserved":0,"allocated":884736,"data_stored":367241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574912,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32612,"kb_used_data":1424,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339228,"statfs":{"total":96636764160,"available":96603369472,"internally_reserved":0,"allocated":1458176,"data_stored":955381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T11:31:55.236 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-20T11:31:55.236 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-20T11:31:55.488 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.488+0000 7fe6aec00640 1 Processor -- start 2024-04-20T11:31:55.488 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.488+0000 7fe6aec00640 1 -- start start 2024-04-20T11:31:55.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.490+0000 7fe6aec00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fe6a80d0b80 0x7fe6a8160d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:55.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.490+0000 7fe6aec00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fe6a80ccff0 0x7fe6a80cd450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:55.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.490+0000 7fe6aec00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe6a8071cd0 0x7fe6a8072140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:55.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.490+0000 7fe6aec00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fe6a80cd990 con 0x7fe6a80d0b80 2024-04-20T11:31:55.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.490+0000 7fe6aec00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fe6a8145830 con 0x7fe6a80ccff0 2024-04-20T11:31:55.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.490+0000 7fe6aec00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe6a806f070 con 0x7fe6a8071cd0 2024-04-20T11:31:55.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.490+0000 7fe6a7e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fe6a80d0b80 0x7fe6a8160d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:55.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.491+0000 7fe6a7e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fe6a80d0b80 0x7fe6a8160d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36660/0 (socket says 172.21.15.152:36660) 2024-04-20T11:31:55.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.491+0000 7fe6a7e00640 1 -- 172.21.15.152:0/2822621172 learned_addr learned my addr 172.21.15.152:0/2822621172 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:55.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.491+0000 7fe6acc00640 1 --2- 172.21.15.152:0/2822621172 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe6a8071cd0 0x7fe6a8072140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:55.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.491+0000 7fe6a7400640 1 --2- 172.21.15.152:0/2822621172 >> v2:172.21.15.175:3300/0 conn(0x7fe6a80ccff0 0x7fe6a80cd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:55.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.491+0000 7fe6a7e00640 1 -- 172.21.15.152:0/2822621172 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe6a8071cd0 msgr2=0x7fe6a8072140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:55.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.491+0000 7fe6a7e00640 1 --2- 172.21.15.152:0/2822621172 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe6a8071cd0 0x7fe6a8072140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:55.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.491+0000 7fe6a7e00640 1 -- 172.21.15.152:0/2822621172 >> v2:172.21.15.175:3300/0 conn(0x7fe6a80ccff0 msgr2=0x7fe6a80cd450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:55.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.491+0000 7fe6a7e00640 1 --2- 172.21.15.152:0/2822621172 >> v2:172.21.15.175:3300/0 conn(0x7fe6a80ccff0 0x7fe6a80cd450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:55.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.491+0000 7fe6a7e00640 1 -- 172.21.15.152:0/2822621172 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6a8161290 con 0x7fe6a80d0b80 2024-04-20T11:31:55.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.491+0000 7fe6acc00640 1 --2- 172.21.15.152:0/2822621172 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe6a8071cd0 0x7fe6a8072140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:31:55.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.492+0000 7fe6a7e00640 1 --2- 172.21.15.152:0/2822621172 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe6a80d0b80 0x7fe6a8160d50 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fe6a80afe50 tx=0x7fe69802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=751149442f7efba1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:55.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.492+0000 7fe6a6a00640 1 -- 172.21.15.152:0/2822621172 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe698035020 con 0x7fe6a80d0b80 2024-04-20T11:31:55.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.492+0000 7fe6a6a00640 1 -- 172.21.15.152:0/2822621172 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe698031870 con 0x7fe6a80d0b80 2024-04-20T11:31:55.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.493+0000 7fe6a6a00640 1 -- 172.21.15.152:0/2822621172 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe698031c40 con 0x7fe6a80d0b80 2024-04-20T11:31:55.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.493+0000 7fe6aec00640 1 -- 172.21.15.152:0/2822621172 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe6a80d0b80 msgr2=0x7fe6a8160d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:55.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.493+0000 7fe6aec00640 1 --2- 172.21.15.152:0/2822621172 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe6a80d0b80 0x7fe6a8160d50 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fe6a80afe50 tx=0x7fe69802b910 comp rx=0 tx=0).stop 2024-04-20T11:31:55.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.493+0000 7fe6aec00640 1 -- 172.21.15.152:0/2822621172 shutdown_connections 2024-04-20T11:31:55.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.493+0000 7fe6aec00640 1 --2- 172.21.15.152:0/2822621172 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe6a8071cd0 0x7fe6a8072140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:55.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.493+0000 7fe6aec00640 1 --2- 172.21.15.152:0/2822621172 >> v2:172.21.15.175:3300/0 conn(0x7fe6a80ccff0 0x7fe6a80cd450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:55.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.493+0000 7fe6aec00640 1 --2- 172.21.15.152:0/2822621172 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe6a80d0b80 0x7fe6a8160d50 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:55.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.493+0000 7fe6aec00640 1 -- 172.21.15.152:0/2822621172 >> 172.21.15.152:0/2822621172 conn(0x7fe6a806b980 msgr2=0x7fe6a806bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:55.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.493+0000 7fe6aec00640 1 -- 172.21.15.152:0/2822621172 shutdown_connections 2024-04-20T11:31:55.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.494+0000 7fe6aec00640 1 -- 172.21.15.152:0/2822621172 wait complete. 2024-04-20T11:31:55.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.495+0000 7fe6aec00640 1 Processor -- start 2024-04-20T11:31:55.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.495+0000 7fe6aec00640 1 -- start start 2024-04-20T11:31:55.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.496+0000 7fe6aec00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe6a8071cd0 0x7fe6a815c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:55.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.496+0000 7fe6aec00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe6a80ccff0 0x7fe6a81423b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:55.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.496+0000 7fe6aec00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe6a80d0b80 0x7fe6a81428f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:55.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.496+0000 7fe6aec00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a806e740 con 0x7fe6a8071cd0 2024-04-20T11:31:55.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.496+0000 7fe6aec00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a806e460 con 0x7fe6a80d0b80 2024-04-20T11:31:55.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.496+0000 7fe6aec00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe6a806ea20 con 0x7fe6a80ccff0 2024-04-20T11:31:55.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.496+0000 7fe6a7e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe6a8071cd0 0x7fe6a815c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:55.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.497+0000 7fe6a7e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe6a8071cd0 0x7fe6a815c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36668/0 (socket says 172.21.15.152:36668) 2024-04-20T11:31:55.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.497+0000 7fe6a7e00640 1 -- 172.21.15.152:0/1175809744 learned_addr learned my addr 172.21.15.152:0/1175809744 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:55.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.497+0000 7fe6a7400640 1 --2- 172.21.15.152:0/1175809744 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe6a80ccff0 0x7fe6a81423b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:55.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.497+0000 7fe6acc00640 1 --2- 172.21.15.152:0/1175809744 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe6a80d0b80 0x7fe6a81428f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:55.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.497+0000 7fe6a7e00640 1 -- 172.21.15.152:0/1175809744 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe6a80ccff0 msgr2=0x7fe6a81423b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:55.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.497+0000 7fe6a7e00640 1 --2- 172.21.15.152:0/1175809744 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe6a80ccff0 0x7fe6a81423b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:55.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.497+0000 7fe6a7e00640 1 -- 172.21.15.152:0/1175809744 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe6a80d0b80 msgr2=0x7fe6a81428f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:55.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.497+0000 7fe6a7e00640 1 --2- 172.21.15.152:0/1175809744 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe6a80d0b80 0x7fe6a81428f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:55.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.497+0000 7fe6a7e00640 1 -- 172.21.15.152:0/1175809744 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6a815c6b0 con 0x7fe6a8071cd0 2024-04-20T11:31:55.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.497+0000 7fe6a7e00640 1 --2- 172.21.15.152:0/1175809744 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe6a8071cd0 0x7fe6a815c170 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fe69802b950 tx=0x7fe6980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:55.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.498+0000 7fe69fe00640 1 -- 172.21.15.152:0/1175809744 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe698035040 con 0x7fe6a8071cd0 2024-04-20T11:31:55.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.498+0000 7fe69fe00640 1 -- 172.21.15.152:0/1175809744 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe698045480 con 0x7fe6a8071cd0 2024-04-20T11:31:55.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.498+0000 7fe69fe00640 1 -- 172.21.15.152:0/1175809744 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe698045870 con 0x7fe6a8071cd0 2024-04-20T11:31:55.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.498+0000 7fe6aec00640 1 -- 172.21.15.152:0/1175809744 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6a8141740 con 0x7fe6a8071cd0 2024-04-20T11:31:55.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.498+0000 7fe6aec00640 1 -- 172.21.15.152:0/1175809744 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6a8141d00 con 0x7fe6a8071cd0 2024-04-20T11:31:55.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.499+0000 7fe6aec00640 1 -- 172.21.15.152:0/1175809744 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6a80689f0 con 0x7fe6a8071cd0 2024-04-20T11:31:55.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.500+0000 7fe69fe00640 1 -- 172.21.15.152:0/1175809744 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 11) ==== 91346+0+0 (secure 0 0 0) 0x7fe698038020 con 0x7fe6a8071cd0 2024-04-20T11:31:55.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.501+0000 7fe69fe00640 1 --2- 172.21.15.152:0/1175809744 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe68406cc80 0x7fe68406f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:55.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.501+0000 7fe69fe00640 1 -- 172.21.15.152:0/1175809744 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5709+0+0 (secure 0 0 0) 0x7fe6980ae4d0 con 0x7fe6a8071cd0 2024-04-20T11:31:55.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.501+0000 7fe6a7400640 1 --2- 172.21.15.152:0/1175809744 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe68406cc80 0x7fe68406f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:55.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.502+0000 7fe6a7400640 1 --2- 172.21.15.152:0/1175809744 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe68406cc80 0x7fe68406f140 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fe6a80b4e90 tx=0x7fe690008440 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:55.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.505+0000 7fe69fe00640 1 -- 172.21.15.152:0/1175809744 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe69807c540 con 0x7fe6a8071cd0 2024-04-20T11:31:55.777 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.777+0000 7fe6aec00640 1 -- 172.21.15.152:0/1175809744 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fe6a80d1050 con 0x7fe6a8071cd0 2024-04-20T11:31:55.784 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.784+0000 7fe69fe00640 1 -- 172.21.15.152:0/1175809744 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v14) ==== 190+0+147 (secure 0 0 0) 0x7fe69807be40 con 0x7fe6a8071cd0 2024-04-20T11:31:55.784 INFO:teuthology.orchestra.run.smithi152.stdout:[client.0] 2024-04-20T11:31:55.784 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQBDpiNmtsBWFhAArq9A7r33CcwgMmOU2pAGPw== 2024-04-20T11:31:55.784 INFO:teuthology.orchestra.run.smithi152.stdout: caps mds = "allow" 2024-04-20T11:31:55.784 INFO:teuthology.orchestra.run.smithi152.stdout: caps mon = "allow r" 2024-04-20T11:31:55.784 INFO:teuthology.orchestra.run.smithi152.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-20T11:31:55.784 INFO:teuthology.orchestra.run.smithi152.stderr:updated caps for client.0 2024-04-20T11:31:55.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.787+0000 7fe6aec00640 1 -- 172.21.15.152:0/1175809744 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe68406cc80 msgr2=0x7fe68406f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:55.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.787+0000 7fe6aec00640 1 --2- 172.21.15.152:0/1175809744 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe68406cc80 0x7fe68406f140 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fe6a80b4e90 tx=0x7fe690008440 comp rx=0 tx=0).stop 2024-04-20T11:31:55.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.787+0000 7fe6aec00640 1 -- 172.21.15.152:0/1175809744 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe6a8071cd0 msgr2=0x7fe6a815c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:55.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.787+0000 7fe6aec00640 1 --2- 172.21.15.152:0/1175809744 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe6a8071cd0 0x7fe6a815c170 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fe69802b950 tx=0x7fe6980046c0 comp rx=0 tx=0).stop 2024-04-20T11:31:55.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.787+0000 7fe6aec00640 1 -- 172.21.15.152:0/1175809744 shutdown_connections 2024-04-20T11:31:55.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.787+0000 7fe6aec00640 1 --2- 172.21.15.152:0/1175809744 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fe68406cc80 0x7fe68406f140 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:55.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.787+0000 7fe6aec00640 1 --2- 172.21.15.152:0/1175809744 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe6a80d0b80 0x7fe6a81428f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:55.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.787+0000 7fe6aec00640 1 --2- 172.21.15.152:0/1175809744 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe6a80ccff0 0x7fe6a81423b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:55.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.787+0000 7fe6aec00640 1 --2- 172.21.15.152:0/1175809744 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe6a8071cd0 0x7fe6a815c170 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:55.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.787+0000 7fe6aec00640 1 -- 172.21.15.152:0/1175809744 >> 172.21.15.152:0/1175809744 conn(0x7fe6a806b980 msgr2=0x7fe6a806f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:55.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.787+0000 7fe6aec00640 1 -- 172.21.15.152:0/1175809744 shutdown_connections 2024-04-20T11:31:55.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.787+0000 7fe6aec00640 1 -- 172.21.15.152:0/1175809744 wait complete. 2024-04-20T11:31:55.807 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:31:55.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.996+0000 7ff1bb200640 1 Processor -- start 2024-04-20T11:31:55.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.997+0000 7ff1bb200640 1 -- start start 2024-04-20T11:31:55.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.998+0000 7ff1bb200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff1bc155c10 0x7ff1bc0b6b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:55.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.998+0000 7ff1bb200640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1bc0b5da0 con 0x7ff1bc0b70a0 2024-04-20T11:31:55.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.998+0000 7ff1bb200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1bc0b6080 con 0x7ff1bc155830 2024-04-20T11:31:55.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.998+0000 7ff1bb200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff1bc0b6360 con 0x7ff1bc155c10 2024-04-20T11:31:55.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.999+0000 7ff1ba800640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7ff1bc0b70a0 0x7ff1bc09aa40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:34696/0 (socket says 172.21.15.152:34696) 2024-04-20T11:31:55.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.999+0000 7ff1ba800640 1 -- 172.21.15.152:0/1727876973 learned_addr learned my addr 172.21.15.152:0/1727876973 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:31:56.000 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.999+0000 7ff1b1400640 1 --2- 172.21.15.152:0/1727876973 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff1bc155c10 0x7ff1bc0b6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:56.000 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.999+0000 7ff1b9400640 1 -- 172.21.15.152:0/1727876973 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3969496922 0 0) 0x7ff1bc0b5da0 con 0x7ff1bc0b70a0 2024-04-20T11:31:56.000 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.999+0000 7ff1b9400640 1 -- 172.21.15.152:0/1727876973 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff1a0003520 con 0x7ff1bc0b70a0 2024-04-20T11:31:56.000 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.999+0000 7ff1b9400640 1 -- 172.21.15.152:0/1727876973 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2170937705 0 0) 0x7ff1a0003520 con 0x7ff1bc0b70a0 2024-04-20T11:31:56.000 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.999+0000 7ff1b9400640 1 -- 172.21.15.152:0/1727876973 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff1bc155c10 msgr2=0x7ff1bc0b6b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:56.000 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.999+0000 7ff1b9400640 1 --2- 172.21.15.152:0/1727876973 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff1bc155c10 0x7ff1bc0b6b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:56.001 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.999+0000 7ff1b9400640 1 -- 172.21.15.152:0/1727876973 >> v1:172.21.15.175:6789/0 conn(0x7ff1bc155830 legacy=0x7ff1bc06ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:56.001 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.999+0000 7ff1b9400640 1 -- 172.21.15.152:0/1727876973 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1bc06d1a0 con 0x7ff1bc0b70a0 2024-04-20T11:31:56.001 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.999+0000 7ff1b9400640 1 -- 172.21.15.152:0/1727876973 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7ff1b40019f0 con 0x7ff1bc0b70a0 2024-04-20T11:31:56.001 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.999+0000 7ff1b9400640 1 -- 172.21.15.152:0/1727876973 >> v1:172.21.15.152:6789/0 conn(0x7ff1bc0b70a0 legacy=0x7ff1bc09aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:56.001 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:55.999+0000 7ff1b9400640 1 --2- 172.21.15.152:0/1727876973 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff1a00039e0 0x7ff1a0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:56.001 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.000+0000 7ff1b9400640 1 --2- 172.21.15.152:0/1727876973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1a0004800 0x7ff1a0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:56.001 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.000+0000 7ff1b9400640 1 --2- 172.21.15.152:0/1727876973 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff1bc155c10 0x7ff1a0005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:56.001 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.000+0000 7ff1b9400640 1 -- 172.21.15.152:0/1727876973 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7ff1bc0b6360 con 0x7ff1a00039e0 2024-04-20T11:31:56.001 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.000+0000 7ff1b9400640 1 -- 172.21.15.152:0/1727876973 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff1a000baf0 con 0x7ff1a0004800 2024-04-20T11:31:56.001 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.000+0000 7ff1b9400640 1 -- 172.21.15.152:0/1727876973 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff1a000bcf0 con 0x7ff1bc155c10 2024-04-20T11:31:56.001 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.000+0000 7ff1b9e00640 1 --2- 172.21.15.152:0/1727876973 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff1a00039e0 0x7ff1a0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:56.001 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.000+0000 7ff1b9e00640 1 -- 172.21.15.152:0/1727876973 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff1bc155c10 msgr2=0x7ff1a0005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:56.001 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.000+0000 7ff1b9e00640 1 --2- 172.21.15.152:0/1727876973 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff1bc155c10 0x7ff1a0005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:56.001 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.000+0000 7ff1b9e00640 1 -- 172.21.15.152:0/1727876973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1a0004800 msgr2=0x7ff1a0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:56.001 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.000+0000 7ff1b1400640 1 --2- 172.21.15.152:0/1727876973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1a0004800 0x7ff1a0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:56.002 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.000+0000 7ff1b9e00640 1 --2- 172.21.15.152:0/1727876973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1a0004800 0x7ff1a0004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:56.002 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.000+0000 7ff1b9e00640 1 -- 172.21.15.152:0/1727876973 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1a0003520 con 0x7ff1a00039e0 2024-04-20T11:31:56.003 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.001+0000 7ff1b9e00640 1 --2- 172.21.15.152:0/1727876973 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff1a00039e0 0x7ff1a0003e00 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac001750 tx=0x7ff1a0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3e683cbd615f0a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:56.003 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.002+0000 7ff1b9400640 1 -- 172.21.15.152:0/1727876973 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1ac001370 con 0x7ff1a00039e0 2024-04-20T11:31:56.003 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.002+0000 7ff1b9400640 1 -- 172.21.15.152:0/1727876973 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1ac0052d0 con 0x7ff1a00039e0 2024-04-20T11:31:56.003 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.002+0000 7ff1b9400640 1 -- 172.21.15.152:0/1727876973 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1ac003830 con 0x7ff1a00039e0 2024-04-20T11:31:56.003 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.002+0000 7ff1bb200640 1 -- 172.21.15.152:0/1727876973 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff1a00039e0 msgr2=0x7ff1a0003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:56.003 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.002+0000 7ff1bb200640 1 --2- 172.21.15.152:0/1727876973 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff1a00039e0 0x7ff1a0003e00 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac001750 tx=0x7ff1a0005830 comp rx=0 tx=0).stop 2024-04-20T11:31:56.003 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.002+0000 7ff1ba800640 1 -- 172.21.15.152:0/1727876973 reap_dead start 2024-04-20T11:31:56.003 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.002+0000 7ff1bb200640 1 -- 172.21.15.152:0/1727876973 shutdown_connections 2024-04-20T11:31:56.003 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.002+0000 7ff1bb200640 1 -- 172.21.15.152:0/1727876973 >> 172.21.15.152:0/1727876973 conn(0x7ff1bc0b3ed0 msgr2=0x7ff1bc0b42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:56.003 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.002+0000 7ff1bb200640 1 -- 172.21.15.152:0/1727876973 shutdown_connections 2024-04-20T11:31:56.003 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.002+0000 7ff1bb200640 1 -- 172.21.15.152:0/1727876973 wait complete. 2024-04-20T11:31:56.005 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.004+0000 7ff1bb200640 1 Processor -- start 2024-04-20T11:31:56.005 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.004+0000 7ff1bb200640 1 -- start start 2024-04-20T11:31:56.005 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.005+0000 7ff1bb200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff1a00039e0 0x7ff1bc15d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:56.005 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.005+0000 7ff1bb200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff1bc154a90 0x7ff1bc154f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:56.005 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.005+0000 7ff1bb200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1bc09c330 0x7ff1bc09c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:56.006 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.005+0000 7ff1bb200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7ff1bc09b2d0 con 0x7ff1bc154a90 2024-04-20T11:31:56.006 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.005+0000 7ff1bb200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff1bc09b150 con 0x7ff1bc09c330 2024-04-20T11:31:56.006 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.005+0000 7ff1bb200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff1bc0b6b60 con 0x7ff1a00039e0 2024-04-20T11:31:56.006 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.005+0000 7ff1b1400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff1bc154a90 0x7ff1bc154f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:56.006 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.005+0000 7ff1b1400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff1bc154a90 0x7ff1bc154f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35300/0 (socket says 172.21.15.152:35300) 2024-04-20T11:31:56.006 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.005+0000 7ff1b1400640 1 -- 172.21.15.152:0/3225096847 learned_addr learned my addr 172.21.15.152:0/3225096847 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:56.006 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.006+0000 7ff1ba800640 1 --2- 172.21.15.152:0/3225096847 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1bc09c330 0x7ff1bc09c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:56.006 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.006+0000 7ff1b9e00640 1 --2- 172.21.15.152:0/3225096847 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff1a00039e0 0x7ff1bc15d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:56.006 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.006+0000 7ff1b1400640 1 -- 172.21.15.152:0/3225096847 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff1a00039e0 msgr2=0x7ff1bc15d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:56.006 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.006+0000 7ff1b1400640 1 --2- 172.21.15.152:0/3225096847 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff1a00039e0 0x7ff1bc15d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:56.006 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.006+0000 7ff1b1400640 1 -- 172.21.15.152:0/3225096847 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1bc09c330 msgr2=0x7ff1bc09c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:56.006 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.006+0000 7ff1b1400640 1 --2- 172.21.15.152:0/3225096847 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1bc09c330 0x7ff1bc09c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:56.006 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.006+0000 7ff1b1400640 1 -- 172.21.15.152:0/3225096847 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1bc09ccf0 con 0x7ff1bc154a90 2024-04-20T11:31:56.006 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.006+0000 7ff1b1400640 1 --2- 172.21.15.152:0/3225096847 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff1bc154a90 0x7ff1bc154f10 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7ff1a4010860 tx=0x7ff1a4010d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:56.007 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.006+0000 7ff1b2a00640 1 -- 172.21.15.152:0/3225096847 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a400f040 con 0x7ff1bc154a90 2024-04-20T11:31:56.007 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.006+0000 7ff1b2a00640 1 -- 172.21.15.152:0/3225096847 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1a400eb60 con 0x7ff1bc154a90 2024-04-20T11:31:56.007 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.006+0000 7ff1b2a00640 1 -- 172.21.15.152:0/3225096847 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a401c400 con 0x7ff1bc154a90 2024-04-20T11:31:56.007 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.006+0000 7ff1bb200640 1 -- 172.21.15.152:0/3225096847 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1bc09cf80 con 0x7ff1bc154a90 2024-04-20T11:31:56.008 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.006+0000 7ff1bb200640 1 -- 172.21.15.152:0/3225096847 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1bc09d540 con 0x7ff1bc154a90 2024-04-20T11:31:56.008 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.008+0000 7ff1bb200640 1 -- 172.21.15.152:0/3225096847 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1bc075e60 con 0x7ff1bc154a90 2024-04-20T11:31:56.008 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.009+0000 7ff1b2a00640 1 -- 172.21.15.152:0/3225096847 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 11) ==== 91346+0+0 (secure 0 0 0) 0x7ff1a401c5a0 con 0x7ff1bc154a90 2024-04-20T11:31:56.009 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.010+0000 7ff1b2a00640 1 --2- 172.21.15.152:0/3225096847 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff19806cc80 0x7ff19806f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:56.009 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.010+0000 7ff1b2a00640 1 -- 172.21.15.152:0/3225096847 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5709+0+0 (secure 0 0 0) 0x7ff1a408c620 con 0x7ff1bc154a90 2024-04-20T11:31:56.009 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.010+0000 7ff1b9e00640 1 --2- 172.21.15.152:0/3225096847 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff19806cc80 0x7ff19806f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:56.010 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.011+0000 7ff1b9e00640 1 --2- 172.21.15.152:0/3225096847 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff19806cc80 0x7ff19806f140 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac033000 tx=0x7ff1ac009030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:56.014 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.014+0000 7ff1b2a00640 1 -- 172.21.15.152:0/3225096847 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ff1a4018030 con 0x7ff1bc154a90 2024-04-20T11:31:56.253 INFO:tasks.ceph.mds.b.smithi175.stderr:2024-04-20T11:31:56.252+0000 7f1661600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-20T11:31:56.256 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.256+0000 7ff1bb200640 1 -- 172.21.15.152:0/3225096847 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff1bc1f4880 con 0x7ff1bc154a90 2024-04-20T11:31:56.257 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.257+0000 7ff1b2a00640 1 -- 172.21.15.152:0/3225096847 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+5210 (secure 0 0 0) 0x7ff1a4059ff0 con 0x7ff1bc154a90 2024-04-20T11:31:56.257 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:31:56.257 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":43,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5234,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6836/359391273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6835","nonce":359391273},{"type":"v1","addr":"172.21.15.152:6836","nonce":359391273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":5237,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/4269836436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":4269836436},{"type":"v1","addr":"172.21.15.152:6837","nonce":4269836436}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":5611,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6834/4153485754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":4153485754},{"type":"v1","addr":"172.21.15.175:6834","nonce":4153485754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T11:31:49.738765+0000","modified":"2024-04-20T11:31:52.857955+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5614},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5614":{"gid":5614,"name":"b","rank":0,"incarnation":39,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6835/2057501014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6833","nonce":2057501014},{"type":"v1","addr":"172.21.15.175:6835","nonce":2057501014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5614,"qdb_cluster":[5614]},"id":5}]} 2024-04-20T11:31:56.258 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 43 2024-04-20T11:31:56.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.260+0000 7ff1bb200640 1 -- 172.21.15.152:0/3225096847 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff19806cc80 msgr2=0x7ff19806f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:56.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.260+0000 7ff1bb200640 1 --2- 172.21.15.152:0/3225096847 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff19806cc80 0x7ff19806f140 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac033000 tx=0x7ff1ac009030 comp rx=0 tx=0).stop 2024-04-20T11:31:56.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.260+0000 7ff1bb200640 1 -- 172.21.15.152:0/3225096847 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff1bc154a90 msgr2=0x7ff1bc154f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:56.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.260+0000 7ff1bb200640 1 --2- 172.21.15.152:0/3225096847 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff1bc154a90 0x7ff1bc154f10 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7ff1a4010860 tx=0x7ff1a4010d20 comp rx=0 tx=0).stop 2024-04-20T11:31:56.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.260+0000 7ff1bb200640 1 -- 172.21.15.152:0/3225096847 shutdown_connections 2024-04-20T11:31:56.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.260+0000 7ff1bb200640 1 --2- 172.21.15.152:0/3225096847 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff19806cc80 0x7ff19806f140 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:56.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.260+0000 7ff1bb200640 1 --2- 172.21.15.152:0/3225096847 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1bc09c330 0x7ff1bc09c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:56.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.260+0000 7ff1bb200640 1 --2- 172.21.15.152:0/3225096847 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff1bc154a90 0x7ff1bc154f10 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:56.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.260+0000 7ff1bb200640 1 --2- 172.21.15.152:0/3225096847 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff1a00039e0 0x7ff1bc15d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:56.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.260+0000 7ff1bb200640 1 -- 172.21.15.152:0/3225096847 >> 172.21.15.152:0/3225096847 conn(0x7ff1bc0b3ed0 msgr2=0x7ff1bc0b11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:56.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.261+0000 7ff1bb200640 1 -- 172.21.15.152:0/3225096847 shutdown_connections 2024-04-20T11:31:56.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.261+0000 7ff1bb200640 1 -- 172.21.15.152:0/3225096847 wait complete. 2024-04-20T11:31:56.682 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 43, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-20T11:31:49.738765+0000', 'modified': '2024-04-20T11:31:52.857955+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'snap_rstat': True, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5614}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5614': {'gid': 5614, 'name': 'b', 'rank': 0, 'incarnation': 39, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.175:6835/2057501014', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6833', 'nonce': 2057501014}, {'type': 'v1', 'addr': '172.21.15.175:6835', 'nonce': 2057501014}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5614, 'qdb_cluster': [5614]} 2024-04-20T11:31:56.682 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-20T11:31:56.682 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-20T11:31:56.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.892+0000 7f08b7e00640 1 Processor -- start 2024-04-20T11:31:56.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.892+0000 7f08b7e00640 1 -- start start 2024-04-20T11:31:56.894 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.894+0000 7f08b7e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f08b00d0750 0x7f08b006d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:56.894 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.894+0000 7f08b7e00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08b00d0b90 con 0x7f08b006d6b0 2024-04-20T11:31:56.894 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.894+0000 7f08b7e00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08b006dac0 con 0x7f08b00ce9a0 2024-04-20T11:31:56.894 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.894+0000 7f08b7e00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f08b00d1a20 con 0x7f08b00d0750 2024-04-20T11:31:56.894 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.894+0000 7f08b4a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f08b00d0750 0x7f08b006d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:56.894 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.894+0000 7f08b5e00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f08b006d6b0 0x7f08b015e1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:34700/0 (socket says 172.21.15.152:34700) 2024-04-20T11:31:56.894 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.894+0000 7f08b5e00640 1 -- 172.21.15.152:0/535135200 learned_addr learned my addr 172.21.15.152:0/535135200 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:31:56.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.895+0000 7f08afe00640 1 -- 172.21.15.152:0/535135200 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 116968272 0 0) 0x7f08a8001110 con 0x7f08b006d6b0 2024-04-20T11:31:56.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.895+0000 7f08b4a00640 1 -- 172.21.15.152:0/535135200 >> v1:172.21.15.175:6789/0 conn(0x7f08b00ce9a0 legacy=0x7f08b009b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:56.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.895+0000 7f08b4a00640 1 -- 172.21.15.152:0/535135200 >> v1:172.21.15.152:6789/0 conn(0x7f08b006d6b0 legacy=0x7f08b015e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:56.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.895+0000 7f08b4a00640 1 -- 172.21.15.152:0/535135200 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08b00d0e10 con 0x7f08b00d0750 2024-04-20T11:31:56.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.896+0000 7f08b4a00640 1 --2- 172.21.15.152:0/535135200 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f08b00d0750 0x7f08b006d170 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f08a0009d90 tx=0x7f08a002be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=531c7b59195b8e93 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:56.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.896+0000 7f08afe00640 1 -- 172.21.15.152:0/535135200 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a0039070 con 0x7f08b00d0750 2024-04-20T11:31:56.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.896+0000 7f08afe00640 1 -- 172.21.15.152:0/535135200 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08a002cbd0 con 0x7f08b00d0750 2024-04-20T11:31:56.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.896+0000 7f08afe00640 1 -- 172.21.15.152:0/535135200 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a0034440 con 0x7f08b00d0750 2024-04-20T11:31:56.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.897+0000 7f08b7e00640 1 -- 172.21.15.152:0/535135200 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f08b00d0750 msgr2=0x7f08b006d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:56.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.897+0000 7f08b7e00640 1 --2- 172.21.15.152:0/535135200 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f08b00d0750 0x7f08b006d170 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f08a0009d90 tx=0x7f08a002be20 comp rx=0 tx=0).stop 2024-04-20T11:31:56.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.897+0000 7f08b7e00640 1 -- 172.21.15.152:0/535135200 shutdown_connections 2024-04-20T11:31:56.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.897+0000 7f08b7e00640 1 --2- 172.21.15.152:0/535135200 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f08b00d0750 0x7f08b006d170 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:56.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.897+0000 7f08b7e00640 1 -- 172.21.15.152:0/535135200 >> 172.21.15.152:0/535135200 conn(0x7f08b00b4040 msgr2=0x7f08b00b4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:56.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.897+0000 7f08b7e00640 1 -- 172.21.15.152:0/535135200 shutdown_connections 2024-04-20T11:31:56.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.897+0000 7f08b7e00640 1 -- 172.21.15.152:0/535135200 wait complete. 2024-04-20T11:31:56.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.899+0000 7f08b7e00640 1 Processor -- start 2024-04-20T11:31:56.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.899+0000 7f08b7e00640 1 -- start start 2024-04-20T11:31:56.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.900+0000 7f08b7e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f08b006d6b0 0x7f08b00cd520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:56.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.900+0000 7f08b7e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f08b00ce9a0 0x7f08b01e90c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:56.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.900+0000 7f08b7e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f08b00d0750 0x7f08b01e9600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:56.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.900+0000 7f08b7e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f08b00d16b0 con 0x7f08b00d0750 2024-04-20T11:31:56.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.900+0000 7f08b7e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f08b00d13d0 con 0x7f08b006d6b0 2024-04-20T11:31:56.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.900+0000 7f08b7e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f08b00ced80 con 0x7f08b00ce9a0 2024-04-20T11:31:56.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.901+0000 7f08b5e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f08b00d0750 0x7f08b01e9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:56.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.901+0000 7f08b5e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f08b00d0750 0x7f08b01e9600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35316/0 (socket says 172.21.15.152:35316) 2024-04-20T11:31:56.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.901+0000 7f08b5e00640 1 -- 172.21.15.152:0/984187946 learned_addr learned my addr 172.21.15.152:0/984187946 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:56.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.901+0000 7f08b5400640 1 --2- 172.21.15.152:0/984187946 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f08b006d6b0 0x7f08b00cd520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:56.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.901+0000 7f08b4a00640 1 --2- 172.21.15.152:0/984187946 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f08b00ce9a0 0x7f08b01e90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:56.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.901+0000 7f08b5e00640 1 -- 172.21.15.152:0/984187946 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f08b00ce9a0 msgr2=0x7f08b01e90c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:56.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.901+0000 7f08b5e00640 1 --2- 172.21.15.152:0/984187946 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f08b00ce9a0 0x7f08b01e90c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:56.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.901+0000 7f08b5e00640 1 -- 172.21.15.152:0/984187946 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f08b006d6b0 msgr2=0x7f08b00cd520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:56.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.901+0000 7f08b5e00640 1 --2- 172.21.15.152:0/984187946 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f08b006d6b0 0x7f08b00cd520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:56.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.901+0000 7f08b5e00640 1 -- 172.21.15.152:0/984187946 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08b01e9b40 con 0x7f08b00d0750 2024-04-20T11:31:56.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.901+0000 7f08b5400640 1 --2- 172.21.15.152:0/984187946 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f08b006d6b0 0x7f08b00cd520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:31:56.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.901+0000 7f08b5e00640 1 --2- 172.21.15.152:0/984187946 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f08b00d0750 0x7f08b01e9600 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f08a800c410 tx=0x7f08a800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:56.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.902+0000 7f08ad600640 1 -- 172.21.15.152:0/984187946 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a80074e0 con 0x7f08b00d0750 2024-04-20T11:31:56.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.902+0000 7f08ad600640 1 -- 172.21.15.152:0/984187946 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08a80047e0 con 0x7f08b00d0750 2024-04-20T11:31:56.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.902+0000 7f08ad600640 1 -- 172.21.15.152:0/984187946 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a8004bd0 con 0x7f08b00d0750 2024-04-20T11:31:56.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.902+0000 7f08b7e00640 1 -- 172.21.15.152:0/984187946 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08b01e9dd0 con 0x7f08b00d0750 2024-04-20T11:31:56.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.902+0000 7f08b7e00640 1 -- 172.21.15.152:0/984187946 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08b00b5b10 con 0x7f08b00d0750 2024-04-20T11:31:56.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.904+0000 7f08ad600640 1 -- 172.21.15.152:0/984187946 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 11) ==== 91346+0+0 (secure 0 0 0) 0x7f08a8011070 con 0x7f08b00d0750 2024-04-20T11:31:56.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.905+0000 7f08ad600640 1 --2- 172.21.15.152:0/984187946 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f089406cc50 0x7f089406f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:56.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.905+0000 7f08ad600640 1 -- 172.21.15.152:0/984187946 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5709+0+0 (secure 0 0 0) 0x7f08a808bdd0 con 0x7f08b00d0750 2024-04-20T11:31:56.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.905+0000 7f08b5400640 1 --2- 172.21.15.152:0/984187946 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f089406cc50 0x7f089406f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:56.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.906+0000 7f08baa5c740 1 Processor -- start 2024-04-20T11:31:56.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.906+0000 7f08baa5c740 1 -- start start 2024-04-20T11:31:56.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.906+0000 7f08b5400640 1 --2- 172.21.15.152:0/984187946 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f089406cc50 0x7f089406f110 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f08a400a830 tx=0x7f08a4008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:56.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.907+0000 7f08baa5c740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x563039fd9460 0x563039f90a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:56.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.907+0000 7f08baa5c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563039f8f4e0 0x563039f90fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:56.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.907+0000 7f08baa5c740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563039f8b930 0x563039fbb770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:56.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.907+0000 7f08baa5c740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x563039ef79d0 con 0x563039f8b930 2024-04-20T11:31:56.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.907+0000 7f08baa5c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x563039e9e270 con 0x563039f8f4e0 2024-04-20T11:31:56.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.907+0000 7f08baa5c740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x563039ef3990 con 0x563039fd9460 2024-04-20T11:31:56.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.907+0000 7f08b5e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563039f8b930 0x563039fbb770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:56.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.907+0000 7f08b5e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563039f8b930 0x563039fbb770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35320/0 (socket says 172.21.15.152:35320) 2024-04-20T11:31:56.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.907+0000 7f08b5e00640 1 -- 172.21.15.152:0/3524897162 learned_addr learned my addr 172.21.15.152:0/3524897162 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:56.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.907+0000 7f08b5400640 1 --2- 172.21.15.152:0/3524897162 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x563039fd9460 0x563039f90a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:56.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.908+0000 7f08b5e00640 1 -- 172.21.15.152:0/3524897162 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x563039fd9460 msgr2=0x563039f90a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:56.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.908+0000 7f08b5e00640 1 --2- 172.21.15.152:0/3524897162 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x563039fd9460 0x563039f90a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:56.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.908+0000 7f08b4a00640 1 --2- 172.21.15.152:0/3524897162 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563039f8f4e0 0x563039f90fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:56.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.908+0000 7f08b5e00640 1 -- 172.21.15.152:0/3524897162 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563039f8f4e0 msgr2=0x563039f90fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:56.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.908+0000 7f08b5e00640 1 --2- 172.21.15.152:0/3524897162 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563039f8f4e0 0x563039f90fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:56.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.908+0000 7f08b5e00640 1 -- 172.21.15.152:0/3524897162 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563039c0a2d0 con 0x563039f8b930 2024-04-20T11:31:56.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.908+0000 7f08b5e00640 1 --2- 172.21.15.152:0/3524897162 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563039f8b930 0x563039fbb770 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f08a800cdd0 tx=0x7f08a8001190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef045325cdd0424b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:56.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.908+0000 7f089dc00640 1 -- 172.21.15.152:0/3524897162 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a805d8d0 con 0x563039f8b930 2024-04-20T11:31:56.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.908+0000 7f089dc00640 1 -- 172.21.15.152:0/3524897162 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08a8062190 con 0x563039f8b930 2024-04-20T11:31:56.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.908+0000 7f089dc00640 1 -- 172.21.15.152:0/3524897162 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a8055b90 con 0x563039f8b930 2024-04-20T11:31:56.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.909+0000 7f08baa5c740 1 -- 172.21.15.152:0/3524897162 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563039f8b930 msgr2=0x563039fbb770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:56.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.909+0000 7f08baa5c740 1 --2- 172.21.15.152:0/3524897162 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563039f8b930 0x563039fbb770 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f08a800cdd0 tx=0x7f08a8001190 comp rx=0 tx=0).stop 2024-04-20T11:31:56.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.909+0000 7f08baa5c740 1 -- 172.21.15.152:0/3524897162 shutdown_connections 2024-04-20T11:31:56.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.909+0000 7f08baa5c740 1 --2- 172.21.15.152:0/3524897162 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563039f8b930 0x563039fbb770 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:56.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.909+0000 7f08baa5c740 1 --2- 172.21.15.152:0/3524897162 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563039f8f4e0 0x563039f90fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:56.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.909+0000 7f08baa5c740 1 --2- 172.21.15.152:0/3524897162 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x563039fd9460 0x563039f90a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:56.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.909+0000 7f08baa5c740 1 -- 172.21.15.152:0/3524897162 >> 172.21.15.152:0/3524897162 conn(0x563039f8dbf0 msgr2=0x563039fd6770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:56.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.909+0000 7f08baa5c740 1 -- 172.21.15.152:0/3524897162 shutdown_connections 2024-04-20T11:31:56.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.909+0000 7f08baa5c740 1 -- 172.21.15.152:0/3524897162 wait complete. 2024-04-20T11:31:56.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.910+0000 7f08baa5c740 1 -- start start 2024-04-20T11:31:56.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.910+0000 7f08baa5c740 1 Processor -- start 2024-04-20T11:31:56.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.911+0000 7f08baa5c740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x563039f8f4e0 0x563039fb9650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:56.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.911+0000 7f08baa5c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563039fd9460 0x563039fb2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:56.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.911+0000 7f08baa5c740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563039fb2f70 0x563039fd8260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:56.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.911+0000 7f08baa5c740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x563039f8f9f0 con 0x563039fb2f70 2024-04-20T11:31:56.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.911+0000 7f08baa5c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x563039eb15e0 con 0x563039fd9460 2024-04-20T11:31:56.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.911+0000 7f08baa5c740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x563039f8fcd0 con 0x563039f8f4e0 2024-04-20T11:31:56.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.911+0000 7f08baa5c740 10 client.0 _ll_register_callbacks cb 0x563039c09ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:31:56.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.911+0000 7f08b5e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563039fb2f70 0x563039fd8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:56.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.911+0000 7f08b5e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563039fb2f70 0x563039fd8260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35324/0 (socket says 172.21.15.152:35324) 2024-04-20T11:31:56.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.911+0000 7f08b5e00640 1 -- 172.21.15.152:0/1043053620 learned_addr learned my addr 172.21.15.152:0/1043053620 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:56.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.912+0000 7f08b5400640 1 --2- 172.21.15.152:0/1043053620 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x563039f8f4e0 0x563039fb9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:56.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.912+0000 7f08b4a00640 1 --2- 172.21.15.152:0/1043053620 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563039fd9460 0x563039fb2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:56.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.912+0000 7f08b5e00640 1 -- 172.21.15.152:0/1043053620 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x563039f8f4e0 msgr2=0x563039fb9650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:56.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.912+0000 7f08b5e00640 1 --2- 172.21.15.152:0/1043053620 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x563039f8f4e0 0x563039fb9650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:56.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.912+0000 7f08b5e00640 1 -- 172.21.15.152:0/1043053620 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563039fd9460 msgr2=0x563039fb2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:56.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.912+0000 7f08b5e00640 1 --2- 172.21.15.152:0/1043053620 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563039fd9460 0x563039fb2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:56.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.912+0000 7f08b5e00640 1 -- 172.21.15.152:0/1043053620 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563039cd46b0 con 0x563039fb2f70 2024-04-20T11:31:56.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.912+0000 7f08b4a00640 1 --2- 172.21.15.152:0/1043053620 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563039fd9460 0x563039fb2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:31:56.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.912+0000 7f08b5e00640 1 --2- 172.21.15.152:0/1043053620 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563039fb2f70 0x563039fd8260 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f08a80591d0 tx=0x7f08a805d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d38ec3e3b585b567 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:56.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.912+0000 7f0892000640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-04-20T11:31:56.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.912+0000 7f0892000640 1 -- 172.21.15.152:0/1043053620 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a8089060 con 0x563039fb2f70 2024-04-20T11:31:56.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.912+0000 7f0892000640 1 -- 172.21.15.152:0/1043053620 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(71..71 src has 1..71) ==== 5709+0+0 (secure 0 0 0) 0x7f08a8055270 con 0x563039fb2f70 2024-04-20T11:31:56.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.912+0000 7f08baa5c740 1 -- 172.21.15.152:0/1043053620 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08a8062190 con 0x563039fb2f70 2024-04-20T11:31:56.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.912+0000 7f0892000640 1 -- 172.21.15.152:0/1043053620 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f08a8055470 con 0x563039fb2f70 2024-04-20T11:31:56.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.913+0000 7f08baa5c740 10 client.6918 fetch_fsmap learned FSMap version 43 2024-04-20T11:31:56.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.913+0000 7f08baa5c740 1 -- 172.21.15.152:0/1043053620 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=2+}) -- 0x563039fdd5b0 con 0x563039fb2f70 2024-04-20T11:31:56.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.913+0000 7f0892000640 1 -- 172.21.15.152:0/1043053620 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08a8062190 con 0x563039fb2f70 2024-04-20T11:31:56.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.913+0000 7f0892000640 1 -- 172.21.15.152:0/1043053620 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 43) ==== 3495+0+0 (secure 0 0 0) 0x7f08a80583e0 con 0x563039fb2f70 2024-04-20T11:31:56.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.913+0000 7f08baa5c740 10 client.6918 fetch_fsmap finished waiting for FSMap version 43 2024-04-20T11:31:56.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.913+0000 7f08baa5c740 10 client.6918 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-20T11:31:56.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.913+0000 7f08baa5c740 20 client.6918 populate_metadata read hostname 'smithi152' 2024-04-20T11:31:56.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.913+0000 7f08baa5c740 1 --2- 172.21.15.152:0/1043053620 >> [v2:172.21.15.175:6833/2057501014,v1:172.21.15.175:6835/2057501014] conn(0x563039fddff0 0x563039fde3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:56.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.913+0000 7f08baa5c740 4 client.6918 mds_command: new command op to 5614 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:31:56.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.913+0000 7f08baa5c740 1 -- 172.21.15.152:0/1043053620 --> [v2:172.21.15.175:6833/2057501014,v1:172.21.15.175:6835/2057501014] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563039f053f0 con 0x563039fddff0 2024-04-20T11:31:56.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.914+0000 7f08b5400640 1 --2- 172.21.15.152:0/1043053620 >> [v2:172.21.15.175:6833/2057501014,v1:172.21.15.175:6835/2057501014] conn(0x563039fddff0 0x563039fde3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:56.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.915+0000 7f08b5400640 1 --2- 172.21.15.152:0/1043053620 >> [v2:172.21.15.175:6833/2057501014,v1:172.21.15.175:6835/2057501014] conn(0x563039fddff0 0x563039fde3d0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=afcd7a1276df825e server_cookie=53ed5b523858fa09 in_seq=0 out_seq=0 2024-04-20T11:31:56.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.915+0000 7f0892000640 10 client.6918 ms_handle_connect on v2:172.21.15.175:6833/2057501014 2024-04-20T11:31:56.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.917+0000 7f0892000640 1 -- 172.21.15.152:0/1043053620 <== mds.0 v2:172.21.15.175:6833/2057501014 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563039f053f0 con 0x563039fddff0 2024-04-20T11:31:56.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.917+0000 7f0892000640 10 client.6918 handle_command_reply: tid=0 2024-04-20T11:31:56.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.917+0000 7f08baa5c740 1 client.6918 shutdown 2024-04-20T11:31:56.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.917+0000 7f08baa5c740 1 -- 172.21.15.152:0/1043053620 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563039fb2f70 msgr2=0x563039fd8260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:56.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.917+0000 7f08baa5c740 1 --2- 172.21.15.152:0/1043053620 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563039fb2f70 0x563039fd8260 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f08a80591d0 tx=0x7f08a805d040 comp rx=0 tx=0).stop 2024-04-20T11:31:56.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.917+0000 7f08baa5c740 1 -- 172.21.15.152:0/1043053620 shutdown_connections 2024-04-20T11:31:56.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.917+0000 7f08baa5c740 1 --2- 172.21.15.152:0/1043053620 >> [v2:172.21.15.175:6833/2057501014,v1:172.21.15.175:6835/2057501014] conn(0x563039fddff0 0x563039fde3d0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:56.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.917+0000 7f08baa5c740 1 --2- 172.21.15.152:0/1043053620 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563039fb2f70 0x563039fd8260 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:56.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.917+0000 7f08baa5c740 1 --2- 172.21.15.152:0/1043053620 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563039fd9460 0x563039fb2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:56.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.917+0000 7f0892000640 0 client.6918 ms_handle_reset on v2:172.21.15.175:6833/2057501014 2024-04-20T11:31:56.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.917+0000 7f08baa5c740 1 --2- 172.21.15.152:0/1043053620 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x563039f8f4e0 0x563039fb9650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:56.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.917+0000 7f08baa5c740 1 -- 172.21.15.152:0/1043053620 >> 172.21.15.152:0/1043053620 conn(0x563039f8b930 msgr2=0x563039f90880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:56.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.917+0000 7f08baa5c740 1 -- 172.21.15.152:0/1043053620 shutdown_connections 2024-04-20T11:31:56.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.918+0000 7f08baa5c740 1 -- 172.21.15.152:0/1043053620 wait complete. 2024-04-20T11:31:56.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.918+0000 7f08baa5c740 20 client.6918 trim_cache size 0 max 16384 2024-04-20T11:31:56.938 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.938+0000 7f08baa5c740 1 Processor -- start 2024-04-20T11:31:56.938 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.938+0000 7f08baa5c740 1 -- start start 2024-04-20T11:31:56.939 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.939+0000 7f08baa5c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563039f8f4e0 0x563039ffae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:56.939 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.939+0000 7f08baa5c740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563039fb2f70 0x563039fbb090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:56.939 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.939+0000 7f08baa5c740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x563039fd9460 0x563039fbb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:56.939 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.939+0000 7f08baa5c740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f08b00b7390 con 0x563039fb2f70 2024-04-20T11:31:56.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.939+0000 7f08baa5c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f08b01f8c90 con 0x563039f8f4e0 2024-04-20T11:31:56.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.939+0000 7f08baa5c740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f08b00b6d20 con 0x563039fd9460 2024-04-20T11:31:56.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.939+0000 7f08b4a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563039fb2f70 0x563039fbb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:56.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.939+0000 7f08b4a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563039fb2f70 0x563039fbb090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35340/0 (socket says 172.21.15.152:35340) 2024-04-20T11:31:56.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.939+0000 7f08b4a00640 1 -- 172.21.15.152:0/2304601765 learned_addr learned my addr 172.21.15.152:0/2304601765 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:56.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.939+0000 7f08b5400640 1 --2- 172.21.15.152:0/2304601765 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563039f8f4e0 0x563039ffae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:56.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.940+0000 7f08b5e00640 1 --2- 172.21.15.152:0/2304601765 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x563039fd9460 0x563039fbb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:56.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.940+0000 7f08b4a00640 1 -- 172.21.15.152:0/2304601765 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x563039fd9460 msgr2=0x563039fbb5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:56.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.940+0000 7f08b4a00640 1 --2- 172.21.15.152:0/2304601765 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x563039fd9460 0x563039fbb5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:56.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.940+0000 7f08b4a00640 1 -- 172.21.15.152:0/2304601765 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563039f8f4e0 msgr2=0x563039ffae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:56.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.940+0000 7f08b4a00640 1 --2- 172.21.15.152:0/2304601765 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563039f8f4e0 0x563039ffae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:56.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.940+0000 7f08b4a00640 1 -- 172.21.15.152:0/2304601765 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563039f8d980 con 0x563039fb2f70 2024-04-20T11:31:56.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.940+0000 7f08b4a00640 1 --2- 172.21.15.152:0/2304601765 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563039fb2f70 0x563039fbb090 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f08a0009eb0 tx=0x7f08a002e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19a08b778fc6d2fb server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:56.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.940+0000 7f089d200640 1 -- 172.21.15.152:0/2304601765 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a0044070 con 0x563039fb2f70 2024-04-20T11:31:56.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.940+0000 7f089d200640 1 -- 172.21.15.152:0/2304601765 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08a0003990 con 0x563039fb2f70 2024-04-20T11:31:56.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.940+0000 7f089d200640 1 -- 172.21.15.152:0/2304601765 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a0039040 con 0x563039fb2f70 2024-04-20T11:31:56.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.941+0000 7f08baa5c740 1 -- 172.21.15.152:0/2304601765 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563039fb2f70 msgr2=0x563039fbb090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:56.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.941+0000 7f08baa5c740 1 --2- 172.21.15.152:0/2304601765 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563039fb2f70 0x563039fbb090 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f08a0009eb0 tx=0x7f08a002e000 comp rx=0 tx=0).stop 2024-04-20T11:31:56.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.941+0000 7f08baa5c740 1 -- 172.21.15.152:0/2304601765 shutdown_connections 2024-04-20T11:31:56.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.941+0000 7f08baa5c740 1 --2- 172.21.15.152:0/2304601765 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x563039fd9460 0x563039fbb5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:56.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.941+0000 7f08baa5c740 1 --2- 172.21.15.152:0/2304601765 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563039fb2f70 0x563039fbb090 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:56.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.941+0000 7f08baa5c740 1 --2- 172.21.15.152:0/2304601765 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563039f8f4e0 0x563039ffae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:56.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.941+0000 7f08baa5c740 1 -- 172.21.15.152:0/2304601765 >> 172.21.15.152:0/2304601765 conn(0x563039f8b930 msgr2=0x563039fbf630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:56.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.941+0000 7f08baa5c740 1 -- 172.21.15.152:0/2304601765 shutdown_connections 2024-04-20T11:31:56.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.941+0000 7f08baa5c740 1 -- 172.21.15.152:0/2304601765 wait complete. 2024-04-20T11:31:56.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.942+0000 7f08baa5c740 1 -- start start 2024-04-20T11:31:56.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.942+0000 7f08baa5c740 1 Processor -- start 2024-04-20T11:31:56.945 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.943+0000 7f08baa5c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563039f8f4e0 0x563039f8c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:56.945 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.943+0000 7f08baa5c740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x563039fb2f70 0x563039fbb090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:56.945 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.943+0000 7f08baa5c740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563039fd9460 0x563039fbb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:56.945 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.943+0000 7f08baa5c740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x563039f8f9f0 con 0x563039fd9460 2024-04-20T11:31:56.945 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.943+0000 7f08baa5c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x563039eb15e0 con 0x563039f8f4e0 2024-04-20T11:31:56.945 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.943+0000 7f08baa5c740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x563039fdb400 con 0x563039fb2f70 2024-04-20T11:31:56.945 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.943+0000 7f08baa5c740 10 client.0 _ll_register_callbacks cb 0x7f08a8007680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:31:56.945 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.943+0000 7f08b5e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563039fd9460 0x563039fbb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:56.945 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.943+0000 7f08b5e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563039fd9460 0x563039fbb5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35350/0 (socket says 172.21.15.152:35350) 2024-04-20T11:31:56.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.943+0000 7f08b5e00640 1 -- 172.21.15.152:0/3474658676 learned_addr learned my addr 172.21.15.152:0/3474658676 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:56.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.944+0000 7f08b5400640 1 --2- 172.21.15.152:0/3474658676 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563039f8f4e0 0x563039f8c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:56.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.944+0000 7f08b4a00640 1 --2- 172.21.15.152:0/3474658676 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x563039fb2f70 0x563039fbb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:56.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.944+0000 7f08b5e00640 1 -- 172.21.15.152:0/3474658676 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x563039fb2f70 msgr2=0x563039fbb090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:56.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.944+0000 7f08b5e00640 1 --2- 172.21.15.152:0/3474658676 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x563039fb2f70 0x563039fbb090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:56.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.944+0000 7f08b5e00640 1 -- 172.21.15.152:0/3474658676 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563039f8f4e0 msgr2=0x563039f8c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:56.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.944+0000 7f08b5e00640 1 --2- 172.21.15.152:0/3474658676 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563039f8f4e0 0x563039f8c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:56.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.944+0000 7f08b5e00640 1 -- 172.21.15.152:0/3474658676 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563039f8ca80 con 0x563039fd9460 2024-04-20T11:31:56.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.944+0000 7f08b5400640 1 --2- 172.21.15.152:0/3474658676 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563039f8f4e0 0x563039f8c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:31:56.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.944+0000 7f08b5e00640 1 --2- 172.21.15.152:0/3474658676 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563039fd9460 0x563039fbb5d0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f08a80011d0 tx=0x7f08a8009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c26072b18aaba32 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:56.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.944+0000 7f0892000640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-04-20T11:31:56.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.944+0000 7f0892000640 1 -- 172.21.15.152:0/3474658676 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a8089070 con 0x563039fd9460 2024-04-20T11:31:56.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.944+0000 7f0892000640 1 -- 172.21.15.152:0/3474658676 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(71..71 src has 1..71) ==== 5709+0+0 (secure 0 0 0) 0x7f08a8058b80 con 0x563039fd9460 2024-04-20T11:31:56.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.944+0000 7f08baa5c740 1 -- 172.21.15.152:0/3474658676 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08a0003990 con 0x563039fd9460 2024-04-20T11:31:56.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.944+0000 7f0892000640 1 -- 172.21.15.152:0/3474658676 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f08a8062190 con 0x563039fd9460 2024-04-20T11:31:56.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.944+0000 7f08baa5c740 10 client.6924 fetch_fsmap learned FSMap version 43 2024-04-20T11:31:56.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.945+0000 7f08baa5c740 1 -- 172.21.15.152:0/3474658676 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=2+}) -- 0x563039f8ca80 con 0x563039fd9460 2024-04-20T11:31:56.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.945+0000 7f0892000640 1 -- 172.21.15.152:0/3474658676 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08a0003990 con 0x563039fd9460 2024-04-20T11:31:56.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.945+0000 7f0892000640 1 -- 172.21.15.152:0/3474658676 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 43) ==== 3495+0+0 (secure 0 0 0) 0x7f08a805ec70 con 0x563039fd9460 2024-04-20T11:31:56.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.945+0000 7f08baa5c740 10 client.6924 fetch_fsmap finished waiting for FSMap version 43 2024-04-20T11:31:56.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.945+0000 7f08baa5c740 10 client.6924 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-20T11:31:56.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.945+0000 7f08baa5c740 20 client.6924 populate_metadata read hostname 'smithi152' 2024-04-20T11:31:56.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.945+0000 7f08baa5c740 1 --2- 172.21.15.152:0/3474658676 >> [v2:172.21.15.175:6833/2057501014,v1:172.21.15.175:6835/2057501014] conn(0x56303a05a230 0x56303a05a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:56.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.945+0000 7f08baa5c740 4 client.6924 mds_command: new command op to 5614 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-20T11:31:56.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.945+0000 7f08baa5c740 1 -- 172.21.15.152:0/3474658676 --> [v2:172.21.15.175:6833/2057501014,v1:172.21.15.175:6835/2057501014] -- command(tid 0: {"prefix": "status"}) -- 0x563039eff260 con 0x56303a05a230 2024-04-20T11:31:56.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.945+0000 7f08b5400640 1 --2- 172.21.15.152:0/3474658676 >> [v2:172.21.15.175:6833/2057501014,v1:172.21.15.175:6835/2057501014] conn(0x56303a05a230 0x56303a05a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:56.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.946+0000 7f08b5400640 1 --2- 172.21.15.152:0/3474658676 >> [v2:172.21.15.175:6833/2057501014,v1:172.21.15.175:6835/2057501014] conn(0x56303a05a230 0x56303a05a610 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e1f7bcab174c55c server_cookie=24b4fa1cb84d8ab0 in_seq=0 out_seq=0 2024-04-20T11:31:56.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.946+0000 7f0892000640 10 client.6924 ms_handle_connect on v2:172.21.15.175:6833/2057501014 2024-04-20T11:31:56.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.947+0000 7f0892000640 1 -- 172.21.15.152:0/3474658676 <== mds.0 v2:172.21.15.175:6833/2057501014 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x563039eff260 con 0x56303a05a230 2024-04-20T11:31:56.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.947+0000 7f0892000640 10 client.6924 handle_command_reply: tid=0 2024-04-20T11:31:56.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.947+0000 7f08baa5c740 1 client.6924 shutdown 2024-04-20T11:31:56.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.947+0000 7f08baa5c740 1 -- 172.21.15.152:0/3474658676 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563039fd9460 msgr2=0x563039fbb5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:56.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.947+0000 7f08baa5c740 1 --2- 172.21.15.152:0/3474658676 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563039fd9460 0x563039fbb5d0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f08a80011d0 tx=0x7f08a8009040 comp rx=0 tx=0).stop 2024-04-20T11:31:56.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.947+0000 7f08baa5c740 1 -- 172.21.15.152:0/3474658676 shutdown_connections 2024-04-20T11:31:56.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.947+0000 7f08baa5c740 1 --2- 172.21.15.152:0/3474658676 >> [v2:172.21.15.175:6833/2057501014,v1:172.21.15.175:6835/2057501014] conn(0x56303a05a230 0x56303a05a610 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:56.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.947+0000 7f08baa5c740 1 --2- 172.21.15.152:0/3474658676 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563039fd9460 0x563039fbb5d0 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:56.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.947+0000 7f08baa5c740 1 --2- 172.21.15.152:0/3474658676 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x563039fb2f70 0x563039fbb090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:56.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.947+0000 7f08baa5c740 1 --2- 172.21.15.152:0/3474658676 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563039f8f4e0 0x563039f8c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:56.950 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.947+0000 7f0892000640 0 client.6924 ms_handle_reset on v2:172.21.15.175:6833/2057501014 2024-04-20T11:31:56.950 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.947+0000 7f08baa5c740 1 -- 172.21.15.152:0/3474658676 >> 172.21.15.152:0/3474658676 conn(0x563039f8b930 msgr2=0x563039fde6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:56.950 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.947+0000 7f08baa5c740 1 -- 172.21.15.152:0/3474658676 shutdown_connections 2024-04-20T11:31:56.950 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.947+0000 7f08baa5c740 1 -- 172.21.15.152:0/3474658676 wait complete. 2024-04-20T11:31:56.950 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.947+0000 7f08baa5c740 20 client.6924 trim_cache size 0 max 16384 2024-04-20T11:31:56.950 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-04-20T11:31:56.950 INFO:teuthology.orchestra.run.smithi152.stdout: "cluster_fsid": "c8a72131-0d4f-42c7-892f-3f3af4d2bd3d", 2024-04-20T11:31:56.950 INFO:teuthology.orchestra.run.smithi152.stdout: "whoami": 0, 2024-04-20T11:31:56.950 INFO:teuthology.orchestra.run.smithi152.stdout: "id": 5614, 2024-04-20T11:31:56.950 INFO:teuthology.orchestra.run.smithi152.stdout: "want_state": "up:active", 2024-04-20T11:31:56.951 INFO:teuthology.orchestra.run.smithi152.stdout: "state": "up:active", 2024-04-20T11:31:56.951 INFO:teuthology.orchestra.run.smithi152.stdout: "fs_name": "cephfs", 2024-04-20T11:31:56.951 INFO:teuthology.orchestra.run.smithi152.stdout: "rank_uptime": 6.6376374010000001, 2024-04-20T11:31:56.951 INFO:teuthology.orchestra.run.smithi152.stdout: "mdsmap_epoch": 43, 2024-04-20T11:31:56.951 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_epoch": 71, 2024-04-20T11:31:56.951 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_epoch_barrier": 71, 2024-04-20T11:31:56.951 INFO:teuthology.orchestra.run.smithi152.stdout: "uptime": 12.141636177000001 2024-04-20T11:31:56.951 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-04-20T11:31:56.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.948+0000 7f08b7e00640 1 -- 172.21.15.152:0/984187946 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f089406cc50 msgr2=0x7f089406f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:56.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.948+0000 7f08b7e00640 1 --2- 172.21.15.152:0/984187946 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f089406cc50 0x7f089406f110 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f08a400a830 tx=0x7f08a4008040 comp rx=0 tx=0).stop 2024-04-20T11:31:56.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.948+0000 7f08b7e00640 1 -- 172.21.15.152:0/984187946 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f08b00d0750 msgr2=0x7f08b01e9600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:56.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.948+0000 7f08b7e00640 1 --2- 172.21.15.152:0/984187946 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f08b00d0750 0x7f08b01e9600 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f08a800c410 tx=0x7f08a800c8d0 comp rx=0 tx=0).stop 2024-04-20T11:31:56.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.948+0000 7f08b7e00640 1 -- 172.21.15.152:0/984187946 shutdown_connections 2024-04-20T11:31:56.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.948+0000 7f08b7e00640 1 --2- 172.21.15.152:0/984187946 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f089406cc50 0x7f089406f110 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:56.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.948+0000 7f08b7e00640 1 --2- 172.21.15.152:0/984187946 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f08b00d0750 0x7f08b01e9600 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:56.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.948+0000 7f08b7e00640 1 --2- 172.21.15.152:0/984187946 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f08b00ce9a0 0x7f08b01e90c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:56.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.948+0000 7f08b7e00640 1 --2- 172.21.15.152:0/984187946 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f08b006d6b0 0x7f08b00cd520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:56.953 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.948+0000 7f08b7e00640 1 -- 172.21.15.152:0/984187946 >> 172.21.15.152:0/984187946 conn(0x7f08b00b4040 msgr2=0x7f08b006cd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:56.953 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.948+0000 7f08b7e00640 1 -- 172.21.15.152:0/984187946 shutdown_connections 2024-04-20T11:31:56.953 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:56.948+0000 7f08b7e00640 1 -- 172.21.15.152:0/984187946 wait complete. 2024-04-20T11:31:56.966 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-20T11:31:57.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:57.166+0000 7f048f200640 1 Processor -- start 2024-04-20T11:31:57.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:57.166+0000 7f048f200640 1 -- start start 2024-04-20T11:31:57.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:57.168+0000 7f048f200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f049013e530 0x7f049006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:57.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:57.168+0000 7f048f200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f049006f340 0x7f0490140dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:57.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:57.168+0000 7f048f200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04900cf0a0 con 0x7f0490141310 2024-04-20T11:31:57.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:57.168+0000 7f048f200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f0490141b00 con 0x7f049006f340 2024-04-20T11:31:57.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:57.168+0000 7f048f200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f04900cfc50 con 0x7f049013e530 2024-04-20T11:31:57.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:57.168+0000 7f048d400640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f049006f340 0x7f0490140dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:57.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:57.168+0000 7f048de00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f049013e530 0x7f049006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:57.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:57.169+0000 7f048de00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f049013e530 0x7f049006ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.152:53772/0 (socket says 172.21.15.152:53772) 2024-04-20T11:31:57.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:57.169+0000 7f048de00640 1 -- 172.21.15.152:0/2877143352 learned_addr learned my addr 172.21.15.152:0/2877143352 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:57.172 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:57.169+0000 7f048e800640 1 --1- 172.21.15.152:0/2877143352 >> v1:172.21.15.175:6789/0 conn(0x7f0490141310 0x7f049015efd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.152:39068/0 (socket says 172.21.15.152:39068) 2024-04-20T11:31:57.172 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:57.169+0000 7f048de00640 1 -- 172.21.15.152:0/2877143352 >> v2:172.21.15.175:3300/0 conn(0x7f049006f340 msgr2=0x7f0490140dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:57.172 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:57.169+0000 7f048de00640 1 --2- 172.21.15.152:0/2877143352 >> v2:172.21.15.175:3300/0 conn(0x7f049006f340 0x7f0490140dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:57.172 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:57.169+0000 7f048de00640 1 -- 172.21.15.152:0/2877143352 >> v1:172.21.15.175:6789/0 conn(0x7f0490141310 legacy=0x7f049015efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:57.172 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:57.169+0000 7f048de00640 1 -- 172.21.15.152:0/2877143352 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04900d00e0 con 0x7f049013e530 2024-04-20T11:31:57.172 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:57.171+0000 7f048de00640 1 --2- 172.21.15.152:0/2877143352 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f049013e530 0x7f049006ee00 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f048000b920 tx=0x7f048002bf50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c1e08dc3f382634 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:57.172 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:57.171+0000 7f048ca00640 1 -- 172.21.15.152:0/2877143352 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0480039070 con 0x7f049013e530 2024-04-20T11:31:57.172 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:57.171+0000 7f048ca00640 1 -- 172.21.15.152:0/2877143352 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f048002cd00 con 0x7f049013e530 2024-04-20T11:31:57.172 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:57.171+0000 7f048ca00640 1 -- 172.21.15.152:0/2877143352 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0480034660 con 0x7f049013e530 2024-04-20T11:31:57.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:57.172+0000 7f048f200640 1 -- 172.21.15.152:0/2877143352 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f049013e530 msgr2=0x7f049006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:57.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:57.172+0000 7f048f200640 1 --2- 172.21.15.152:0/2877143352 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f049013e530 0x7f049006ee00 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f048000b920 tx=0x7f048002bf50 comp rx=0 tx=0).stop 2024-04-20T11:31:57.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:57.172+0000 7f048f200640 1 -- 172.21.15.152:0/2877143352 shutdown_connections 2024-04-20T11:31:57.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:57.172+0000 7f048f200640 1 --2- 172.21.15.152:0/2877143352 >> v2:172.21.15.175:3300/0 conn(0x7f049006f340 0x7f0490140dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:57.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:57.172+0000 7f048f200640 1 --2- 172.21.15.152:0/2877143352 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f049013e530 0x7f049006ee00 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:57.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:57.172+0000 7f048f200640 1 -- 172.21.15.152:0/2877143352 >> 172.21.15.152:0/2877143352 conn(0x7f0490067ae0 msgr2=0x7f049006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:57.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:57.172+0000 7f048f200640 1 -- 172.21.15.152:0/2877143352 shutdown_connections 2024-04-20T11:31:57.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:57.172+0000 7f048f200640 1 -- 172.21.15.152:0/2877143352 wait complete. 2024-04-20T11:31:57.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:57.174+0000 7f048f200640 1 Processor -- start 2024-04-20T11:31:57.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:57.175+0000 7f048f200640 1 -- start start 2024-04-20T11:31:57.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:57.176+0000 7f048f200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f049006f340 0x7f049013d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:57.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:57.176+0000 7f048f200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f049013e530 0x7f049013fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:57.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:57.176+0000 7f048f200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0490141310 0x7f04901402e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:57.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:57.176+0000 7f048f200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f04900cf600 con 0x7f049006f340 2024-04-20T11:31:57.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:57.176+0000 7f048f200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0490072ce0 con 0x7f0490141310 2024-04-20T11:31:57.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:57.176+0000 7f048f200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f04900cf8e0 con 0x7f049013e530 2024-04-20T11:31:57.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:57.176+0000 7f048de00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f049006f340 0x7f049013d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:57.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:57.176+0000 7f048de00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f049006f340 0x7f049013d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35362/0 (socket says 172.21.15.152:35362) 2024-04-20T11:31:57.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:57.176+0000 7f048de00640 1 -- 172.21.15.152:0/381391708 learned_addr learned my addr 172.21.15.152:0/381391708 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:57.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:57.176+0000 7f048d400640 1 --2- 172.21.15.152:0/381391708 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f049013e530 0x7f049013fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:57.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:57.176+0000 7f048e800640 1 --2- 172.21.15.152:0/381391708 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0490141310 0x7f04901402e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:57.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:57.177+0000 7f048de00640 1 -- 172.21.15.152:0/381391708 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f049013e530 msgr2=0x7f049013fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:57.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:57.177+0000 7f048de00640 1 --2- 172.21.15.152:0/381391708 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f049013e530 0x7f049013fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:57.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:57.177+0000 7f048de00640 1 -- 172.21.15.152:0/381391708 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0490141310 msgr2=0x7f04901402e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:57.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:57.177+0000 7f048de00640 1 --2- 172.21.15.152:0/381391708 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0490141310 0x7f04901402e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:57.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:57.177+0000 7f048de00640 1 -- 172.21.15.152:0/381391708 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04900b19c0 con 0x7f049006f340 2024-04-20T11:31:57.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:57.177+0000 7f048d400640 1 --2- 172.21.15.152:0/381391708 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f049013e530 0x7f049013fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:31:57.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:57.177+0000 7f048de00640 1 --2- 172.21.15.152:0/381391708 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f049006f340 0x7f049013d520 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f0480000c00 tx=0x7f04800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:57.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:57.177+0000 7f0486000640 1 -- 172.21.15.152:0/381391708 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0480039040 con 0x7f049006f340 2024-04-20T11:31:57.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:57.177+0000 7f0486000640 1 -- 172.21.15.152:0/381391708 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0480044070 con 0x7f049006f340 2024-04-20T11:31:57.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:57.177+0000 7f0486000640 1 -- 172.21.15.152:0/381391708 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f048003f660 con 0x7f049006f340 2024-04-20T11:31:57.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:57.177+0000 7f048f200640 1 -- 172.21.15.152:0/381391708 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04900b1ca0 con 0x7f049006f340 2024-04-20T11:31:57.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:57.177+0000 7f048f200640 1 -- 172.21.15.152:0/381391708 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04901538b0 con 0x7f049006f340 2024-04-20T11:31:57.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:57.179+0000 7f048f200640 1 -- 172.21.15.152:0/381391708 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0490075e60 con 0x7f049006f340 2024-04-20T11:31:57.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:57.180+0000 7f0486000640 1 -- 172.21.15.152:0/381391708 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 11) ==== 91346+0+0 (secure 0 0 0) 0x7f048003f800 con 0x7f049006f340 2024-04-20T11:31:57.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:57.180+0000 7f0486000640 1 --2- 172.21.15.152:0/381391708 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f047006cc50 0x7f047006f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:57.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:57.181+0000 7f0486000640 1 -- 172.21.15.152:0/381391708 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5709+0+0 (secure 0 0 0) 0x7f04800ae870 con 0x7f049006f340 2024-04-20T11:31:57.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:57.181+0000 7f048d400640 1 --2- 172.21.15.152:0/381391708 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f047006cc50 0x7f047006f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:57.181 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:57.181+0000 7f048d400640 1 --2- 172.21.15.152:0/381391708 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f047006cc50 0x7f047006f110 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f047c00d440 tx=0x7f047c00f040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:57.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:57.186+0000 7f0486000640 1 -- 172.21.15.152:0/381391708 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f0480032e20 con 0x7f049006f340 2024-04-20T11:31:57.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:57.414+0000 7f048f200640 1 -- 172.21.15.152:0/381391708 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f04900b3490 con 0x7f049006f340 2024-04-20T11:31:57.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:57.414+0000 7f0486000640 1 -- 172.21.15.152:0/381391708 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v43) ==== 53+0+73 (secure 0 0 0) 0x7f048007c8e0 con 0x7f049006f340 2024-04-20T11:31:57.414 INFO:teuthology.orchestra.run.smithi152.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-20T11:31:57.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:57.417+0000 7f048f200640 1 -- 172.21.15.152:0/381391708 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f047006cc50 msgr2=0x7f047006f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:57.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:57.417+0000 7f048f200640 1 --2- 172.21.15.152:0/381391708 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f047006cc50 0x7f047006f110 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f047c00d440 tx=0x7f047c00f040 comp rx=0 tx=0).stop 2024-04-20T11:31:57.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:57.417+0000 7f048f200640 1 -- 172.21.15.152:0/381391708 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f049006f340 msgr2=0x7f049013d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:57.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:57.417+0000 7f048f200640 1 --2- 172.21.15.152:0/381391708 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f049006f340 0x7f049013d520 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f0480000c00 tx=0x7f04800046c0 comp rx=0 tx=0).stop 2024-04-20T11:31:57.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:57.417+0000 7f048f200640 1 -- 172.21.15.152:0/381391708 shutdown_connections 2024-04-20T11:31:57.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:57.417+0000 7f048f200640 1 --2- 172.21.15.152:0/381391708 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f047006cc50 0x7f047006f110 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:57.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:57.417+0000 7f048f200640 1 --2- 172.21.15.152:0/381391708 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0490141310 0x7f04901402e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:57.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:57.417+0000 7f048f200640 1 --2- 172.21.15.152:0/381391708 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f049013e530 0x7f049013fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:57.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:57.417+0000 7f048f200640 1 --2- 172.21.15.152:0/381391708 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f049006f340 0x7f049013d520 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:57.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:57.417+0000 7f048f200640 1 -- 172.21.15.152:0/381391708 >> 172.21.15.152:0/381391708 conn(0x7f0490067ae0 msgr2=0x7f0490159570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:57.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:57.417+0000 7f048f200640 1 -- 172.21.15.152:0/381391708 shutdown_connections 2024-04-20T11:31:57.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:57.417+0000 7f048f200640 1 -- 172.21.15.152:0/381391708 wait complete. 2024-04-20T11:31:57.434 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-20T11:31:57.434 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-20T11:31:57.434 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi152.front.sepia.ceph.com 2024-04-20T11:31:57.434 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-20T11:31:57.434 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-20T11:31:57.434 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-20T11:31:57.435 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-20T11:31:57.435 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-20T11:31:57.435 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-20T11:31:57.435 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:31:57.435 DEBUG:teuthology.orchestra.run.smithi152:> ip addr 2024-04-20T11:31:57.483 INFO:teuthology.orchestra.run.smithi152.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-20T11:31:57.483 INFO:teuthology.orchestra.run.smithi152.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-20T11:31:57.483 INFO:teuthology.orchestra.run.smithi152.stdout: inet 127.0.0.1/8 scope host lo 2024-04-20T11:31:57.484 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:31:57.484 INFO:teuthology.orchestra.run.smithi152.stdout: inet6 ::1/128 scope host 2024-04-20T11:31:57.484 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:31:57.484 INFO:teuthology.orchestra.run.smithi152.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T11:31:57.484 INFO:teuthology.orchestra.run.smithi152.stdout: link/ether 0c:c4:7a:da:6c:44 brd ff:ff:ff:ff:ff:ff 2024-04-20T11:31:57.484 INFO:teuthology.orchestra.run.smithi152.stdout: altname enp7s0f0 2024-04-20T11:31:57.484 INFO:teuthology.orchestra.run.smithi152.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T11:31:57.484 INFO:teuthology.orchestra.run.smithi152.stdout: link/ether 0c:c4:7a:da:6c:45 brd ff:ff:ff:ff:ff:ff 2024-04-20T11:31:57.484 INFO:teuthology.orchestra.run.smithi152.stdout: altname enp7s0f1 2024-04-20T11:31:57.484 INFO:teuthology.orchestra.run.smithi152.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T11:31:57.484 INFO:teuthology.orchestra.run.smithi152.stdout: link/ether 0c:c4:7a:88:6d:e2 brd ff:ff:ff:ff:ff:ff 2024-04-20T11:31:57.484 INFO:teuthology.orchestra.run.smithi152.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-20T11:31:57.484 INFO:teuthology.orchestra.run.smithi152.stdout: link/ether 0c:c4:7a:88:6d:e3 brd ff:ff:ff:ff:ff:ff 2024-04-20T11:31:57.484 INFO:teuthology.orchestra.run.smithi152.stdout: inet 172.21.15.152/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-20T11:31:57.485 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft 42489sec preferred_lft 42489sec 2024-04-20T11:31:57.485 INFO:teuthology.orchestra.run.smithi152.stdout: inet6 fe80::ec4:7aff:fe88:6de3/64 scope link 2024-04-20T11:31:57.485 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:31:57.485 INFO:teuthology.orchestra.run.smithi152.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-20T11:31:57.485 INFO:teuthology.orchestra.run.smithi152.stdout: link/ether 06:3e:9a:d3:49:1e brd ff:ff:ff:ff:ff:ff 2024-04-20T11:31:57.485 INFO:teuthology.orchestra.run.smithi152.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-20T11:31:57.485 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:31:57.485 INFO:teuthology.orchestra.run.smithi152.stdout: inet6 fe80::a019:d8ff:fe70:1185/64 scope link 2024-04-20T11:31:57.485 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:31:57.485 INFO:teuthology.orchestra.run.smithi152.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-20T11:31:57.485 INFO:teuthology.orchestra.run.smithi152.stdout: link/ether 16:e5:08:78:20:47 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T11:31:57.485 INFO:teuthology.orchestra.run.smithi152.stdout: inet6 fe80::14e5:8ff:fe78:2047/64 scope link 2024-04-20T11:31:57.485 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:31:57.485 INFO:teuthology.orchestra.run.smithi152.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-20T11:31:57.485 INFO:teuthology.orchestra.run.smithi152.stdout: link/ether 06:3e:9a:d3:49:1e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T11:31:57.486 INFO:teuthology.orchestra.run.smithi152.stdout: inet6 fe80::43e:9aff:fed3:491e/64 scope link 2024-04-20T11:31:57.486 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:31:57.486 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:31:57.486 DEBUG:teuthology.orchestra.run.smithi152:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-20T11:31:57.599 INFO:teuthology.orchestra.run.smithi152.stdout:1 2024-04-20T11:31:57.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:31:57.603 DEBUG:teuthology.orchestra.run.smithi152:> route 2024-04-20T11:31:57.652 INFO:teuthology.orchestra.run.smithi152.stdout:Kernel IP routing table 2024-04-20T11:31:57.652 INFO:teuthology.orchestra.run.smithi152.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-20T11:31:57.652 INFO:teuthology.orchestra.run.smithi152.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-20T11:31:57.652 INFO:teuthology.orchestra.run.smithi152.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-20T11:31:57.652 INFO:teuthology.orchestra.run.smithi152.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-20T11:31:57.652 DEBUG:teuthology.orchestra.run.smithi152:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T11:31:57.653 DEBUG:teuthology.orchestra.run.smithi152:> set -e 2024-04-20T11:31:57.653 DEBUG:teuthology.orchestra.run.smithi152:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-20T11:31:57.653 DEBUG:teuthology.orchestra.run.smithi152:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-20T11:31:57.653 DEBUG:teuthology.orchestra.run.smithi152:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-20T11:31:57.653 DEBUG:teuthology.orchestra.run.smithi152:> ') 2024-04-20T11:31:57.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T11:31:57.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T11:31:57.864 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:31:57.864 DEBUG:teuthology.orchestra.run.smithi152:> ip netns list 2024-04-20T11:31:57.913 INFO:teuthology.orchestra.run.smithi152.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-20T11:31:57.914 INFO:teuthology.orchestra.run.smithi152.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-20T11:31:57.914 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-20T11:31:57.915 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:31:57.915 DEBUG:teuthology.orchestra.run.smithi152:> sudo ip link set brx.0 up 2024-04-20T11:31:57.990 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-20T11:31:57.991 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T11:31:57.991 DEBUG:teuthology.orchestra.run.smithi152:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-20T11:31:58.044 INFO:teuthology.orchestra.run.smithi152.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-20T11:31:58.044 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T11:31:58.044 DEBUG:teuthology.orchestra.run.smithi152:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-20T11:31:58.097 DEBUG:teuthology.orchestra.run.smithi152:> sudo modprobe fuse 2024-04-20T11:31:58.182 DEBUG:teuthology.orchestra.run.smithi152:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T11:31:58.238 INFO:teuthology.orchestra.run.smithi152.stdout:/proc 2024-04-20T11:31:58.238 INFO:teuthology.orchestra.run.smithi152.stdout:/sys 2024-04-20T11:31:58.238 INFO:teuthology.orchestra.run.smithi152.stdout:/dev 2024-04-20T11:31:58.238 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/security 2024-04-20T11:31:58.238 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/shm 2024-04-20T11:31:58.239 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/pts 2024-04-20T11:31:58.239 INFO:teuthology.orchestra.run.smithi152.stdout:/run 2024-04-20T11:31:58.239 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/cgroup 2024-04-20T11:31:58.239 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/pstore 2024-04-20T11:31:58.239 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/bpf 2024-04-20T11:31:58.239 INFO:teuthology.orchestra.run.smithi152.stdout:/ 2024-04-20T11:31:58.239 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/selinux 2024-04-20T11:31:58.239 INFO:teuthology.orchestra.run.smithi152.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T11:31:58.239 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/mqueue 2024-04-20T11:31:58.239 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/hugepages 2024-04-20T11:31:58.239 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/debug 2024-04-20T11:31:58.239 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/tracing 2024-04-20T11:31:58.239 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/fuse/connections 2024-04-20T11:31:58.239 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/config 2024-04-20T11:31:58.240 INFO:teuthology.orchestra.run.smithi152.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T11:31:58.240 INFO:teuthology.orchestra.run.smithi152.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T11:31:58.240 INFO:teuthology.orchestra.run.smithi152.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T11:31:58.240 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T11:31:58.240 INFO:teuthology.orchestra.run.smithi152.stdout:/run/user/1000 2024-04-20T11:31:58.240 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph 2024-04-20T11:31:58.240 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph/osd/ceph-0 2024-04-20T11:31:58.240 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph/osd/ceph-1 2024-04-20T11:31:58.240 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph/osd/ceph-2 2024-04-20T11:31:58.240 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph/osd/ceph-3 2024-04-20T11:31:58.240 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns 2024-04-20T11:31:58.240 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T11:31:58.240 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T11:31:58.240 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T11:31:58.241 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T11:31:58.241 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:31:58.242 DEBUG:teuthology.orchestra.run.smithi152:> ls /sys/fs/fuse/connections 2024-04-20T11:31:58.292 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-20T11:31:58.292 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-20T11:31:58.296 DEBUG:teuthology.orchestra.run.smithi152:> sudo modprobe fuse 2024-04-20T11:31:58.367 DEBUG:teuthology.orchestra.run.smithi152:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T11:31:58.410 INFO:teuthology.orchestra.run.smithi152.stdout:/proc 2024-04-20T11:31:58.411 INFO:teuthology.orchestra.run.smithi152.stdout:/sys 2024-04-20T11:31:58.411 INFO:teuthology.orchestra.run.smithi152.stdout:/dev 2024-04-20T11:31:58.411 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/security 2024-04-20T11:31:58.411 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/shm 2024-04-20T11:31:58.411 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/pts 2024-04-20T11:31:58.411 INFO:teuthology.orchestra.run.smithi152.stdout:/run 2024-04-20T11:31:58.411 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/cgroup 2024-04-20T11:31:58.411 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/pstore 2024-04-20T11:31:58.411 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/bpf 2024-04-20T11:31:58.411 INFO:teuthology.orchestra.run.smithi152.stdout:/ 2024-04-20T11:31:58.411 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/selinux 2024-04-20T11:31:58.411 INFO:teuthology.orchestra.run.smithi152.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T11:31:58.411 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/mqueue 2024-04-20T11:31:58.411 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/hugepages 2024-04-20T11:31:58.412 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/debug 2024-04-20T11:31:58.412 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/tracing 2024-04-20T11:31:58.412 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/fuse/connections 2024-04-20T11:31:58.412 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/config 2024-04-20T11:31:58.412 INFO:teuthology.orchestra.run.smithi152.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T11:31:58.412 INFO:teuthology.orchestra.run.smithi152.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T11:31:58.412 INFO:teuthology.orchestra.run.smithi152.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T11:31:58.412 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T11:31:58.412 INFO:teuthology.orchestra.run.smithi152.stdout:/run/user/1000 2024-04-20T11:31:58.412 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph 2024-04-20T11:31:58.412 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph/osd/ceph-0 2024-04-20T11:31:58.412 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph/osd/ceph-1 2024-04-20T11:31:58.412 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph/osd/ceph-2 2024-04-20T11:31:58.413 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph/osd/ceph-3 2024-04-20T11:31:58.413 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns 2024-04-20T11:31:58.413 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T11:31:58.413 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T11:31:58.413 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T11:31:58.413 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T11:31:58.414 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:31:58.414 DEBUG:teuthology.orchestra.run.smithi152:> ls /sys/fs/fuse/connections 2024-04-20T11:31:58.470 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi152.stderr:2024-04-20T11:31:58.470+0000 7efdace38480 -1 init, newargv = 0x56006d672ec0 newargc=15 2024-04-20T11:31:58.470 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi152.stderr:ceph-fuse[47498]: starting ceph client 2024-04-20T11:31:58.480 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi152.stderr:ceph-fuse[47498]: starting fuse 2024-04-20T11:31:58.498 INFO:teuthology.orchestra.run.smithi152.stdout:42 2024-04-20T11:31:58.499 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-04-20T11:31:58.499 DEBUG:teuthology.orchestra.run.smithi152:> sudo stdin-killer -- python3 -c ' 2024-04-20T11:31:58.499 DEBUG:teuthology.orchestra.run.smithi152:> import glob 2024-04-20T11:31:58.499 DEBUG:teuthology.orchestra.run.smithi152:> import re 2024-04-20T11:31:58.499 DEBUG:teuthology.orchestra.run.smithi152:> import os 2024-04-20T11:31:58.499 DEBUG:teuthology.orchestra.run.smithi152:> import subprocess 2024-04-20T11:31:58.499 DEBUG:teuthology.orchestra.run.smithi152:> 2024-04-20T11:31:58.499 DEBUG:teuthology.orchestra.run.smithi152:> def _find_admin_socket(client_name): 2024-04-20T11:31:58.499 DEBUG:teuthology.orchestra.run.smithi152:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-20T11:31:58.499 DEBUG:teuthology.orchestra.run.smithi152:> files = glob.glob(asok_path) 2024-04-20T11:31:58.499 DEBUG:teuthology.orchestra.run.smithi152:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-20T11:31:58.499 DEBUG:teuthology.orchestra.run.smithi152:> 2024-04-20T11:31:58.500 DEBUG:teuthology.orchestra.run.smithi152:> # Given a non-glob path, it better be there 2024-04-20T11:31:58.500 DEBUG:teuthology.orchestra.run.smithi152:> if "*" not in asok_path: 2024-04-20T11:31:58.500 DEBUG:teuthology.orchestra.run.smithi152:> assert(len(files) == 1) 2024-04-20T11:31:58.500 DEBUG:teuthology.orchestra.run.smithi152:> return files[0] 2024-04-20T11:31:58.500 DEBUG:teuthology.orchestra.run.smithi152:> 2024-04-20T11:31:58.500 DEBUG:teuthology.orchestra.run.smithi152:> for f in files: 2024-04-20T11:31:58.500 DEBUG:teuthology.orchestra.run.smithi152:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-20T11:31:58.500 DEBUG:teuthology.orchestra.run.smithi152:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-20T11:31:58.500 DEBUG:teuthology.orchestra.run.smithi152:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-20T11:31:58.500 DEBUG:teuthology.orchestra.run.smithi152:> contents = proc_f.read() 2024-04-20T11:31:58.500 DEBUG:teuthology.orchestra.run.smithi152:> if mountpoint in contents: 2024-04-20T11:31:58.500 DEBUG:teuthology.orchestra.run.smithi152:> return f 2024-04-20T11:31:58.500 DEBUG:teuthology.orchestra.run.smithi152:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-20T11:31:58.500 DEBUG:teuthology.orchestra.run.smithi152:> 2024-04-20T11:31:58.500 DEBUG:teuthology.orchestra.run.smithi152:> print(_find_admin_socket("client.0")) 2024-04-20T11:31:58.500 DEBUG:teuthology.orchestra.run.smithi152:> ' 2024-04-20T11:31:58.692 INFO:teuthology.orchestra.run.smithi152.stdout:/var/run/ceph/ceph-client.0.47498.asok 2024-04-20T11:31:58.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T11:31:58.709 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.47498.asok 2024-04-20T11:31:58.709 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:31:58.709 DEBUG:teuthology.orchestra.run.smithi152:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.47498.asok status 2024-04-20T11:31:58.938 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-04-20T11:31:58.938 INFO:teuthology.orchestra.run.smithi152.stdout: "metadata": { 2024-04-20T11:31:58.938 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_sha1": "a8c84f5997950fd7d008204839253291d52e6eda", 2024-04-20T11:31:58.938 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", 2024-04-20T11:31:58.938 INFO:teuthology.orchestra.run.smithi152.stdout: "entity_id": "0", 2024-04-20T11:31:58.938 INFO:teuthology.orchestra.run.smithi152.stdout: "hostname": "smithi152", 2024-04-20T11:31:58.939 INFO:teuthology.orchestra.run.smithi152.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-20T11:31:58.939 INFO:teuthology.orchestra.run.smithi152.stdout: "pid": "47498", 2024-04-20T11:31:58.939 INFO:teuthology.orchestra.run.smithi152.stdout: "root": "/" 2024-04-20T11:31:58.939 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:31:58.939 INFO:teuthology.orchestra.run.smithi152.stdout: "dentry_count": 0, 2024-04-20T11:31:58.939 INFO:teuthology.orchestra.run.smithi152.stdout: "dentry_pinned_count": 0, 2024-04-20T11:31:58.939 INFO:teuthology.orchestra.run.smithi152.stdout: "id": 6933, 2024-04-20T11:31:58.939 INFO:teuthology.orchestra.run.smithi152.stdout: "inst": { 2024-04-20T11:31:58.939 INFO:teuthology.orchestra.run.smithi152.stdout: "name": { 2024-04-20T11:31:58.939 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "client", 2024-04-20T11:31:58.939 INFO:teuthology.orchestra.run.smithi152.stdout: "num": 6933 2024-04-20T11:31:58.939 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:31:58.939 INFO:teuthology.orchestra.run.smithi152.stdout: "addr": { 2024-04-20T11:31:58.940 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "v1", 2024-04-20T11:31:58.940 INFO:teuthology.orchestra.run.smithi152.stdout: "addr": "192.168.0.1:0", 2024-04-20T11:31:58.940 INFO:teuthology.orchestra.run.smithi152.stdout: "nonce": 3956992320 2024-04-20T11:31:58.940 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:31:58.940 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:31:58.940 INFO:teuthology.orchestra.run.smithi152.stdout: "addr": { 2024-04-20T11:31:58.940 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "v1", 2024-04-20T11:31:58.940 INFO:teuthology.orchestra.run.smithi152.stdout: "addr": "192.168.0.1:0", 2024-04-20T11:31:58.940 INFO:teuthology.orchestra.run.smithi152.stdout: "nonce": 3956992320 2024-04-20T11:31:58.940 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:31:58.940 INFO:teuthology.orchestra.run.smithi152.stdout: "inst_str": "client.6933 192.168.0.1:0/3956992320", 2024-04-20T11:31:58.940 INFO:teuthology.orchestra.run.smithi152.stdout: "addr_str": "192.168.0.1:0/3956992320", 2024-04-20T11:31:58.940 INFO:teuthology.orchestra.run.smithi152.stdout: "inode_count": 1, 2024-04-20T11:31:58.940 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_epoch": 43, 2024-04-20T11:31:58.941 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_epoch": 71, 2024-04-20T11:31:58.941 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_epoch_barrier": 0, 2024-04-20T11:31:58.941 INFO:teuthology.orchestra.run.smithi152.stdout: "blocklisted": false, 2024-04-20T11:31:58.941 INFO:teuthology.orchestra.run.smithi152.stdout: "fs_name": "cephfs" 2024-04-20T11:31:58.941 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-04-20T11:31:58.952 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:31:58.953 DEBUG:teuthology.orchestra.run.smithi152:> stat --file-system '--printf=%T 2024-04-20T11:31:58.953 DEBUG:teuthology.orchestra.run.smithi152:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-20T11:31:59.004 INFO:teuthology.orchestra.run.smithi152.stdout:fuseblk 2024-04-20T11:31:59.005 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-20T11:31:59.005 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:31:59.005 DEBUG:teuthology.orchestra.run.smithi152:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-20T11:31:59.087 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-20T11:31:59.291 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:59.291+0000 7f5a2a800640 1 Processor -- start 2024-04-20T11:31:59.291 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:59.291+0000 7f5a2a800640 1 -- start start 2024-04-20T11:31:59.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:59.293+0000 7f5a2a800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f5a240cd0d0 0x7f5a240cd580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:59.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:59.293+0000 7f5a2a800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5a24071fe0 0x7f5a24072470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:59.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:59.293+0000 7f5a2a800640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a24146340 con 0x7f5a24145f60 2024-04-20T11:31:59.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:59.293+0000 7f5a2a800640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f5a2406e9f0 con 0x7f5a240cd0d0 2024-04-20T11:31:59.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:59.293+0000 7f5a2a800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5a24072d50 con 0x7f5a24071fe0 2024-04-20T11:31:59.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:59.294+0000 7f5a1be00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f5a240cd0d0 0x7f5a240cd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:59.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:59.294+0000 7f5a1be00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f5a240cd0d0 0x7f5a240cd580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35376/0 (socket says 172.21.15.152:35376) 2024-04-20T11:31:59.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:59.294+0000 7f5a1be00640 1 -- 172.21.15.152:0/2364326239 learned_addr learned my addr 172.21.15.152:0/2364326239 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:59.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:59.294+0000 7f5a23e00640 1 --2- 172.21.15.152:0/2364326239 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5a24071fe0 0x7f5a24072470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:59.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:59.294+0000 7f5a1be00640 1 -- 172.21.15.152:0/2364326239 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5a24071fe0 msgr2=0x7f5a24072470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:59.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:59.294+0000 7f5a1be00640 1 --2- 172.21.15.152:0/2364326239 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5a24071fe0 0x7f5a24072470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:59.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:59.294+0000 7f5a1be00640 1 -- 172.21.15.152:0/2364326239 >> v1:172.21.15.175:6789/0 conn(0x7f5a24145f60 legacy=0x7f5a2406ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:59.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:59.294+0000 7f5a1be00640 1 -- 172.21.15.152:0/2364326239 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a2406e0c0 con 0x7f5a240cd0d0 2024-04-20T11:31:59.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:59.295+0000 7f5a1be00640 1 --2- 172.21.15.152:0/2364326239 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5a240cd0d0 0x7f5a240cd580 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f5a240ce480 tx=0x7f5a0c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=861608e608ee9991 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:59.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:59.295+0000 7f5a22a00640 1 -- 172.21.15.152:0/2364326239 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a0c035020 con 0x7f5a240cd0d0 2024-04-20T11:31:59.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:59.295+0000 7f5a22a00640 1 -- 172.21.15.152:0/2364326239 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a0c031830 con 0x7f5a240cd0d0 2024-04-20T11:31:59.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:59.296+0000 7f5a22a00640 1 -- 172.21.15.152:0/2364326239 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a0c031c00 con 0x7f5a240cd0d0 2024-04-20T11:31:59.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:59.296+0000 7f5a2a800640 1 -- 172.21.15.152:0/2364326239 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5a240cd0d0 msgr2=0x7f5a240cd580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:59.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:59.296+0000 7f5a2a800640 1 --2- 172.21.15.152:0/2364326239 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5a240cd0d0 0x7f5a240cd580 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f5a240ce480 tx=0x7f5a0c02b8d0 comp rx=0 tx=0).stop 2024-04-20T11:31:59.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:59.296+0000 7f5a2a800640 1 -- 172.21.15.152:0/2364326239 shutdown_connections 2024-04-20T11:31:59.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:59.296+0000 7f5a2a800640 1 --2- 172.21.15.152:0/2364326239 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5a24071fe0 0x7f5a24072470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:59.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:59.296+0000 7f5a2a800640 1 --2- 172.21.15.152:0/2364326239 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5a240cd0d0 0x7f5a240cd580 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:59.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:59.296+0000 7f5a2a800640 1 -- 172.21.15.152:0/2364326239 >> 172.21.15.152:0/2364326239 conn(0x7f5a24067cf0 msgr2=0x7f5a2406b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:59.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:59.296+0000 7f5a2a800640 1 -- 172.21.15.152:0/2364326239 shutdown_connections 2024-04-20T11:31:59.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:59.296+0000 7f5a2a800640 1 -- 172.21.15.152:0/2364326239 wait complete. 2024-04-20T11:31:59.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:59.298+0000 7f5a2a800640 1 Processor -- start 2024-04-20T11:31:59.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:59.298+0000 7f5a2a800640 1 -- start start 2024-04-20T11:31:59.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:59.299+0000 7f5a2a800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5a24071fe0 0x7f5a240cc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:59.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:59.299+0000 7f5a2a800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5a240cd0d0 0x7f5a240c7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:59.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:59.299+0000 7f5a2a800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5a24145f60 0x7f5a241f8130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:59.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:59.299+0000 7f5a2a800640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f5a240729b0 con 0x7f5a24071fe0 2024-04-20T11:31:59.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:59.299+0000 7f5a2a800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5a2406e680 con 0x7f5a24145f60 2024-04-20T11:31:59.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:59.299+0000 7f5a2a800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5a24072b30 con 0x7f5a240cd0d0 2024-04-20T11:31:59.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:59.299+0000 7f5a23400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5a24071fe0 0x7f5a240cc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:59.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:59.299+0000 7f5a23400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5a24071fe0 0x7f5a240cc1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35380/0 (socket says 172.21.15.152:35380) 2024-04-20T11:31:59.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:59.299+0000 7f5a23400640 1 -- 172.21.15.152:0/2657138874 learned_addr learned my addr 172.21.15.152:0/2657138874 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:31:59.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:59.300+0000 7f5a1be00640 1 --2- 172.21.15.152:0/2657138874 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5a240cd0d0 0x7f5a240c7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:59.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:59.300+0000 7f5a23e00640 1 --2- 172.21.15.152:0/2657138874 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5a24145f60 0x7f5a241f8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:59.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:59.300+0000 7f5a23400640 1 -- 172.21.15.152:0/2657138874 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5a240cd0d0 msgr2=0x7f5a240c7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:59.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:59.300+0000 7f5a23400640 1 --2- 172.21.15.152:0/2657138874 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5a240cd0d0 0x7f5a240c7ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:59.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:59.300+0000 7f5a23400640 1 -- 172.21.15.152:0/2657138874 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5a24145f60 msgr2=0x7f5a241f8130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:59.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:59.300+0000 7f5a23400640 1 --2- 172.21.15.152:0/2657138874 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5a24145f60 0x7f5a241f8130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:59.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:59.300+0000 7f5a23400640 1 -- 172.21.15.152:0/2657138874 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a240c57c0 con 0x7f5a24071fe0 2024-04-20T11:31:59.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:59.300+0000 7f5a23e00640 1 --2- 172.21.15.152:0/2657138874 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5a24145f60 0x7f5a241f8130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:31:59.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:59.300+0000 7f5a23400640 1 --2- 172.21.15.152:0/2657138874 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5a24071fe0 0x7f5a240cc1e0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f5a1400b3f0 tx=0x7f5a1400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:59.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:59.301+0000 7f5a1b400640 1 -- 172.21.15.152:0/2657138874 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a14015020 con 0x7f5a24071fe0 2024-04-20T11:31:59.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:59.301+0000 7f5a1b400640 1 -- 172.21.15.152:0/2657138874 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a14011830 con 0x7f5a24071fe0 2024-04-20T11:31:59.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:59.301+0000 7f5a2a800640 1 -- 172.21.15.152:0/2657138874 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a240c5aa0 con 0x7f5a24071fe0 2024-04-20T11:31:59.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:59.301+0000 7f5a1b400640 1 -- 172.21.15.152:0/2657138874 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a14011c20 con 0x7f5a24071fe0 2024-04-20T11:31:59.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:59.301+0000 7f5a2a800640 1 -- 172.21.15.152:0/2657138874 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a240c6060 con 0x7f5a24071fe0 2024-04-20T11:31:59.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:59.302+0000 7f5a2a800640 1 -- 172.21.15.152:0/2657138874 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a240689f0 con 0x7f5a24071fe0 2024-04-20T11:31:59.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:59.303+0000 7f5a1b400640 1 -- 172.21.15.152:0/2657138874 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 11) ==== 91346+0+0 (secure 0 0 0) 0x7f5a14018020 con 0x7f5a24071fe0 2024-04-20T11:31:59.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:59.304+0000 7f5a1b400640 1 --2- 172.21.15.152:0/2657138874 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f5a0006cc50 0x7f5a0006f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:59.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:59.304+0000 7f5a1b400640 1 -- 172.21.15.152:0/2657138874 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5709+0+0 (secure 0 0 0) 0x7f5a1408bd80 con 0x7f5a24071fe0 2024-04-20T11:31:59.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:59.304+0000 7f5a1be00640 1 --2- 172.21.15.152:0/2657138874 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f5a0006cc50 0x7f5a0006f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:59.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:59.305+0000 7f5a1be00640 1 --2- 172.21.15.152:0/2657138874 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f5a0006cc50 0x7f5a0006f110 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c036000 tx=0x7f5a0c03c040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:59.308 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:59.309+0000 7f5a1b400640 1 -- 172.21.15.152:0/2657138874 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f5a1401f030 con 0x7f5a24071fe0 2024-04-20T11:31:59.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:59.552+0000 7f5a2a800640 1 -- 172.21.15.152:0/2657138874 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f5a240b5560 con 0x7f5a24071fe0 2024-04-20T11:31:59.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:59.552+0000 7f5a1b400640 1 -- 172.21.15.152:0/2657138874 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v43) ==== 53+0+73 (secure 0 0 0) 0x7f5a140172c0 con 0x7f5a24071fe0 2024-04-20T11:31:59.552 INFO:teuthology.orchestra.run.smithi152.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-20T11:31:59.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:59.555+0000 7f5a2a800640 1 -- 172.21.15.152:0/2657138874 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f5a0006cc50 msgr2=0x7f5a0006f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:59.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:59.555+0000 7f5a2a800640 1 --2- 172.21.15.152:0/2657138874 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f5a0006cc50 0x7f5a0006f110 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c036000 tx=0x7f5a0c03c040 comp rx=0 tx=0).stop 2024-04-20T11:31:59.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:59.555+0000 7f5a2a800640 1 -- 172.21.15.152:0/2657138874 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5a24071fe0 msgr2=0x7f5a240cc1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:59.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:59.555+0000 7f5a2a800640 1 --2- 172.21.15.152:0/2657138874 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5a24071fe0 0x7f5a240cc1e0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f5a1400b3f0 tx=0x7f5a1400b8b0 comp rx=0 tx=0).stop 2024-04-20T11:31:59.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:59.555+0000 7f5a2a800640 1 -- 172.21.15.152:0/2657138874 shutdown_connections 2024-04-20T11:31:59.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:59.555+0000 7f5a2a800640 1 --2- 172.21.15.152:0/2657138874 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f5a0006cc50 0x7f5a0006f110 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:59.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:59.555+0000 7f5a2a800640 1 --2- 172.21.15.152:0/2657138874 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5a24145f60 0x7f5a241f8130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:59.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:59.555+0000 7f5a2a800640 1 --2- 172.21.15.152:0/2657138874 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5a240cd0d0 0x7f5a240c7ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:59.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:59.555+0000 7f5a2a800640 1 --2- 172.21.15.152:0/2657138874 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5a24071fe0 0x7f5a240cc1e0 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:59.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:59.555+0000 7f5a2a800640 1 -- 172.21.15.152:0/2657138874 >> 172.21.15.152:0/2657138874 conn(0x7f5a24067cf0 msgr2=0x7f5a240d05a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:59.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:59.555+0000 7f5a2a800640 1 -- 172.21.15.152:0/2657138874 shutdown_connections 2024-04-20T11:31:59.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:31:59.555+0000 7f5a2a800640 1 -- 172.21.15.152:0/2657138874 wait complete. 2024-04-20T11:31:59.575 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-20T11:31:59.575 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-20T11:31:59.575 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi175.front.sepia.ceph.com 2024-04-20T11:31:59.576 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-20T11:31:59.576 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-20T11:31:59.576 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-20T11:31:59.576 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-20T11:31:59.576 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-20T11:31:59.576 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-20T11:31:59.576 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:31:59.576 DEBUG:teuthology.orchestra.run.smithi175:> ip addr 2024-04-20T11:31:59.635 INFO:teuthology.orchestra.run.smithi175.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-20T11:31:59.635 INFO:teuthology.orchestra.run.smithi175.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-20T11:31:59.635 INFO:teuthology.orchestra.run.smithi175.stdout: inet 127.0.0.1/8 scope host lo 2024-04-20T11:31:59.635 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:31:59.635 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 ::1/128 scope host 2024-04-20T11:31:59.636 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:31:59.636 INFO:teuthology.orchestra.run.smithi175.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T11:31:59.636 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:d9:39:ee brd ff:ff:ff:ff:ff:ff 2024-04-20T11:31:59.636 INFO:teuthology.orchestra.run.smithi175.stdout: altname enp7s0f0 2024-04-20T11:31:59.636 INFO:teuthology.orchestra.run.smithi175.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T11:31:59.636 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:d9:39:ef brd ff:ff:ff:ff:ff:ff 2024-04-20T11:31:59.636 INFO:teuthology.orchestra.run.smithi175.stdout: altname enp7s0f1 2024-04-20T11:31:59.636 INFO:teuthology.orchestra.run.smithi175.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T11:31:59.636 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:88:71:f0 brd ff:ff:ff:ff:ff:ff 2024-04-20T11:31:59.636 INFO:teuthology.orchestra.run.smithi175.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-20T11:31:59.636 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:88:71:f1 brd ff:ff:ff:ff:ff:ff 2024-04-20T11:31:59.636 INFO:teuthology.orchestra.run.smithi175.stdout: inet 172.21.15.175/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-20T11:31:59.636 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft 42461sec preferred_lft 42461sec 2024-04-20T11:31:59.636 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::ec4:7aff:fe88:71f1/64 scope link 2024-04-20T11:31:59.636 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:31:59.637 INFO:teuthology.orchestra.run.smithi175.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-20T11:31:59.637 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 02:d6:c6:31:26:70 brd ff:ff:ff:ff:ff:ff 2024-04-20T11:31:59.637 INFO:teuthology.orchestra.run.smithi175.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-20T11:31:59.637 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:31:59.637 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::5091:d5ff:fee7:54b3/64 scope link 2024-04-20T11:31:59.637 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:31:59.637 INFO:teuthology.orchestra.run.smithi175.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-20T11:31:59.637 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 02:d6:c6:31:26:70 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-20T11:31:59.637 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::d6:c6ff:fe31:2670/64 scope link 2024-04-20T11:31:59.637 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:31:59.637 INFO:teuthology.orchestra.run.smithi175.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-20T11:31:59.637 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 9e:14:f3:e2:18:1e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-20T11:31:59.637 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::9c14:f3ff:fee2:181e/64 scope link 2024-04-20T11:31:59.637 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:31:59.638 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:31:59.638 DEBUG:teuthology.orchestra.run.smithi175:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-20T11:31:59.718 INFO:teuthology.orchestra.run.smithi175.stdout:1 2024-04-20T11:31:59.723 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:31:59.723 DEBUG:teuthology.orchestra.run.smithi175:> route 2024-04-20T11:31:59.776 INFO:teuthology.orchestra.run.smithi175.stdout:Kernel IP routing table 2024-04-20T11:31:59.776 INFO:teuthology.orchestra.run.smithi175.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-20T11:31:59.776 INFO:teuthology.orchestra.run.smithi175.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-20T11:31:59.776 INFO:teuthology.orchestra.run.smithi175.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-20T11:31:59.776 INFO:teuthology.orchestra.run.smithi175.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-20T11:31:59.777 DEBUG:teuthology.orchestra.run.smithi175:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T11:31:59.777 DEBUG:teuthology.orchestra.run.smithi175:> set -e 2024-04-20T11:31:59.777 DEBUG:teuthology.orchestra.run.smithi175:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-20T11:31:59.777 DEBUG:teuthology.orchestra.run.smithi175:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-20T11:31:59.777 DEBUG:teuthology.orchestra.run.smithi175:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-20T11:31:59.777 DEBUG:teuthology.orchestra.run.smithi175:> ') 2024-04-20T11:31:59.880 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:31:59 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T11:31:59.976 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:31:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T11:31:59.986 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:31:59.986 DEBUG:teuthology.orchestra.run.smithi175:> ip netns list 2024-04-20T11:32:00.032 INFO:teuthology.orchestra.run.smithi175.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-20T11:32:00.032 INFO:teuthology.orchestra.run.smithi175.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-20T11:32:00.032 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-20T11:32:00.032 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:32:00.032 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip link set brx.0 up 2024-04-20T11:32:00.160 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-04-20T11:32:00.160 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T11:32:00.160 DEBUG:teuthology.orchestra.run.smithi175:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-20T11:32:00.214 INFO:teuthology.orchestra.run.smithi175.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-20T11:32:00.214 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T11:32:00.214 DEBUG:teuthology.orchestra.run.smithi175:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-20T11:32:00.269 DEBUG:teuthology.orchestra.run.smithi175:> sudo modprobe fuse 2024-04-20T11:32:00.351 DEBUG:teuthology.orchestra.run.smithi175:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T11:32:00.403 INFO:teuthology.orchestra.run.smithi175.stdout:/proc 2024-04-20T11:32:00.403 INFO:teuthology.orchestra.run.smithi175.stdout:/sys 2024-04-20T11:32:00.403 INFO:teuthology.orchestra.run.smithi175.stdout:/dev 2024-04-20T11:32:00.403 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/security 2024-04-20T11:32:00.403 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/shm 2024-04-20T11:32:00.404 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/pts 2024-04-20T11:32:00.404 INFO:teuthology.orchestra.run.smithi175.stdout:/run 2024-04-20T11:32:00.404 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/cgroup 2024-04-20T11:32:00.404 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/pstore 2024-04-20T11:32:00.404 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/bpf 2024-04-20T11:32:00.404 INFO:teuthology.orchestra.run.smithi175.stdout:/ 2024-04-20T11:32:00.404 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/selinux 2024-04-20T11:32:00.404 INFO:teuthology.orchestra.run.smithi175.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T11:32:00.404 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/mqueue 2024-04-20T11:32:00.404 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/debug 2024-04-20T11:32:00.404 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/hugepages 2024-04-20T11:32:00.404 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/tracing 2024-04-20T11:32:00.404 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/fuse/connections 2024-04-20T11:32:00.404 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/config 2024-04-20T11:32:00.404 INFO:teuthology.orchestra.run.smithi175.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T11:32:00.404 INFO:teuthology.orchestra.run.smithi175.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T11:32:00.405 INFO:teuthology.orchestra.run.smithi175.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T11:32:00.405 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T11:32:00.405 INFO:teuthology.orchestra.run.smithi175.stdout:/run/user/1000 2024-04-20T11:32:00.405 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph 2024-04-20T11:32:00.405 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-4 2024-04-20T11:32:00.405 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-5 2024-04-20T11:32:00.405 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-6 2024-04-20T11:32:00.405 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-7 2024-04-20T11:32:00.405 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns 2024-04-20T11:32:00.405 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-20T11:32:00.405 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-20T11:32:00.405 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-20T11:32:00.405 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-20T11:32:00.406 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:32:00.406 DEBUG:teuthology.orchestra.run.smithi175:> ls /sys/fs/fuse/connections 2024-04-20T11:32:00.457 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-20T11:32:00.457 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-04-20T11:32:00.462 DEBUG:teuthology.orchestra.run.smithi175:> sudo modprobe fuse 2024-04-20T11:32:00.528 DEBUG:teuthology.orchestra.run.smithi175:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T11:32:00.568 INFO:teuthology.orchestra.run.smithi175.stdout:/proc 2024-04-20T11:32:00.569 INFO:teuthology.orchestra.run.smithi175.stdout:/sys 2024-04-20T11:32:00.569 INFO:teuthology.orchestra.run.smithi175.stdout:/dev 2024-04-20T11:32:00.569 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/security 2024-04-20T11:32:00.569 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/shm 2024-04-20T11:32:00.569 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/pts 2024-04-20T11:32:00.569 INFO:teuthology.orchestra.run.smithi175.stdout:/run 2024-04-20T11:32:00.569 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/cgroup 2024-04-20T11:32:00.569 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/pstore 2024-04-20T11:32:00.569 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/bpf 2024-04-20T11:32:00.569 INFO:teuthology.orchestra.run.smithi175.stdout:/ 2024-04-20T11:32:00.569 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/selinux 2024-04-20T11:32:00.569 INFO:teuthology.orchestra.run.smithi175.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T11:32:00.570 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/mqueue 2024-04-20T11:32:00.570 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/debug 2024-04-20T11:32:00.570 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/hugepages 2024-04-20T11:32:00.570 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/tracing 2024-04-20T11:32:00.570 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/fuse/connections 2024-04-20T11:32:00.570 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/config 2024-04-20T11:32:00.570 INFO:teuthology.orchestra.run.smithi175.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T11:32:00.570 INFO:teuthology.orchestra.run.smithi175.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T11:32:00.570 INFO:teuthology.orchestra.run.smithi175.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T11:32:00.570 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T11:32:00.570 INFO:teuthology.orchestra.run.smithi175.stdout:/run/user/1000 2024-04-20T11:32:00.570 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph 2024-04-20T11:32:00.570 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-4 2024-04-20T11:32:00.571 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-5 2024-04-20T11:32:00.571 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-6 2024-04-20T11:32:00.571 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-7 2024-04-20T11:32:00.571 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns 2024-04-20T11:32:00.571 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-20T11:32:00.571 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-20T11:32:00.571 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-20T11:32:00.571 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-20T11:32:00.572 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:32:00.572 DEBUG:teuthology.orchestra.run.smithi175:> ls /sys/fs/fuse/connections 2024-04-20T11:32:00.629 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi175.stderr:2024-04-20T11:32:00.629+0000 7f3bc643c480 -1 init, newargv = 0x564eb66ab500 newargc=15 2024-04-20T11:32:00.629 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi175.stderr:ceph-fuse[32167]: starting ceph client 2024-04-20T11:32:00.640 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi175.stderr:ceph-fuse[32167]: starting fuse 2024-04-20T11:32:00.658 INFO:teuthology.orchestra.run.smithi175.stdout:42 2024-04-20T11:32:00.658 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-04-20T11:32:00.659 DEBUG:teuthology.orchestra.run.smithi175:> sudo stdin-killer -- python3 -c ' 2024-04-20T11:32:00.659 DEBUG:teuthology.orchestra.run.smithi175:> import glob 2024-04-20T11:32:00.659 DEBUG:teuthology.orchestra.run.smithi175:> import re 2024-04-20T11:32:00.659 DEBUG:teuthology.orchestra.run.smithi175:> import os 2024-04-20T11:32:00.659 DEBUG:teuthology.orchestra.run.smithi175:> import subprocess 2024-04-20T11:32:00.659 DEBUG:teuthology.orchestra.run.smithi175:> 2024-04-20T11:32:00.659 DEBUG:teuthology.orchestra.run.smithi175:> def _find_admin_socket(client_name): 2024-04-20T11:32:00.659 DEBUG:teuthology.orchestra.run.smithi175:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-04-20T11:32:00.659 DEBUG:teuthology.orchestra.run.smithi175:> files = glob.glob(asok_path) 2024-04-20T11:32:00.659 DEBUG:teuthology.orchestra.run.smithi175:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-04-20T11:32:00.659 DEBUG:teuthology.orchestra.run.smithi175:> 2024-04-20T11:32:00.659 DEBUG:teuthology.orchestra.run.smithi175:> # Given a non-glob path, it better be there 2024-04-20T11:32:00.659 DEBUG:teuthology.orchestra.run.smithi175:> if "*" not in asok_path: 2024-04-20T11:32:00.660 DEBUG:teuthology.orchestra.run.smithi175:> assert(len(files) == 1) 2024-04-20T11:32:00.660 DEBUG:teuthology.orchestra.run.smithi175:> return files[0] 2024-04-20T11:32:00.660 DEBUG:teuthology.orchestra.run.smithi175:> 2024-04-20T11:32:00.660 DEBUG:teuthology.orchestra.run.smithi175:> for f in files: 2024-04-20T11:32:00.660 DEBUG:teuthology.orchestra.run.smithi175:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-20T11:32:00.660 DEBUG:teuthology.orchestra.run.smithi175:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-20T11:32:00.660 DEBUG:teuthology.orchestra.run.smithi175:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-20T11:32:00.660 DEBUG:teuthology.orchestra.run.smithi175:> contents = proc_f.read() 2024-04-20T11:32:00.660 DEBUG:teuthology.orchestra.run.smithi175:> if mountpoint in contents: 2024-04-20T11:32:00.660 DEBUG:teuthology.orchestra.run.smithi175:> return f 2024-04-20T11:32:00.660 DEBUG:teuthology.orchestra.run.smithi175:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-20T11:32:00.660 DEBUG:teuthology.orchestra.run.smithi175:> 2024-04-20T11:32:00.660 DEBUG:teuthology.orchestra.run.smithi175:> print(_find_admin_socket("client.1")) 2024-04-20T11:32:00.660 DEBUG:teuthology.orchestra.run.smithi175:> ' 2024-04-20T11:32:00.844 INFO:teuthology.orchestra.run.smithi175.stdout:/var/run/ceph/ceph-client.1.32167.asok 2024-04-20T11:32:00.849 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:32:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T11:32:00.862 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.32167.asok 2024-04-20T11:32:00.862 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:32:00.862 DEBUG:teuthology.orchestra.run.smithi175:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.32167.asok status 2024-04-20T11:32:01.088 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-04-20T11:32:01.089 INFO:teuthology.orchestra.run.smithi175.stdout: "metadata": { 2024-04-20T11:32:01.089 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_sha1": "a8c84f5997950fd7d008204839253291d52e6eda", 2024-04-20T11:32:01.089 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", 2024-04-20T11:32:01.089 INFO:teuthology.orchestra.run.smithi175.stdout: "entity_id": "1", 2024-04-20T11:32:01.089 INFO:teuthology.orchestra.run.smithi175.stdout: "hostname": "smithi175", 2024-04-20T11:32:01.089 INFO:teuthology.orchestra.run.smithi175.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-04-20T11:32:01.089 INFO:teuthology.orchestra.run.smithi175.stdout: "pid": "32167", 2024-04-20T11:32:01.089 INFO:teuthology.orchestra.run.smithi175.stdout: "root": "/" 2024-04-20T11:32:01.089 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-20T11:32:01.089 INFO:teuthology.orchestra.run.smithi175.stdout: "dentry_count": 0, 2024-04-20T11:32:01.089 INFO:teuthology.orchestra.run.smithi175.stdout: "dentry_pinned_count": 0, 2024-04-20T11:32:01.089 INFO:teuthology.orchestra.run.smithi175.stdout: "id": 5686, 2024-04-20T11:32:01.090 INFO:teuthology.orchestra.run.smithi175.stdout: "inst": { 2024-04-20T11:32:01.090 INFO:teuthology.orchestra.run.smithi175.stdout: "name": { 2024-04-20T11:32:01.090 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "client", 2024-04-20T11:32:01.090 INFO:teuthology.orchestra.run.smithi175.stdout: "num": 5686 2024-04-20T11:32:01.090 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-20T11:32:01.090 INFO:teuthology.orchestra.run.smithi175.stdout: "addr": { 2024-04-20T11:32:01.090 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "v1", 2024-04-20T11:32:01.090 INFO:teuthology.orchestra.run.smithi175.stdout: "addr": "192.168.0.1:0", 2024-04-20T11:32:01.090 INFO:teuthology.orchestra.run.smithi175.stdout: "nonce": 2332682925 2024-04-20T11:32:01.090 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-04-20T11:32:01.090 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-20T11:32:01.090 INFO:teuthology.orchestra.run.smithi175.stdout: "addr": { 2024-04-20T11:32:01.090 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "v1", 2024-04-20T11:32:01.090 INFO:teuthology.orchestra.run.smithi175.stdout: "addr": "192.168.0.1:0", 2024-04-20T11:32:01.091 INFO:teuthology.orchestra.run.smithi175.stdout: "nonce": 2332682925 2024-04-20T11:32:01.091 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-20T11:32:01.091 INFO:teuthology.orchestra.run.smithi175.stdout: "inst_str": "client.5686 192.168.0.1:0/2332682925", 2024-04-20T11:32:01.091 INFO:teuthology.orchestra.run.smithi175.stdout: "addr_str": "192.168.0.1:0/2332682925", 2024-04-20T11:32:01.091 INFO:teuthology.orchestra.run.smithi175.stdout: "inode_count": 1, 2024-04-20T11:32:01.091 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_epoch": 43, 2024-04-20T11:32:01.091 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_epoch": 71, 2024-04-20T11:32:01.091 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_epoch_barrier": 0, 2024-04-20T11:32:01.091 INFO:teuthology.orchestra.run.smithi175.stdout: "blocklisted": false, 2024-04-20T11:32:01.091 INFO:teuthology.orchestra.run.smithi175.stdout: "fs_name": "cephfs" 2024-04-20T11:32:01.091 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-04-20T11:32:01.103 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:32:01.103 DEBUG:teuthology.orchestra.run.smithi175:> stat --file-system '--printf=%T 2024-04-20T11:32:01.103 DEBUG:teuthology.orchestra.run.smithi175:> ' -- /home/ubuntu/cephtest/mnt.1 2024-04-20T11:32:01.154 INFO:teuthology.orchestra.run.smithi175.stdout:fuseblk 2024-04-20T11:32:01.154 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-04-20T11:32:01.154 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:32:01.154 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-20T11:32:01.234 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-04-20T11:32:01.449 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:01.448+0000 7f42a1a00640 1 Processor -- start 2024-04-20T11:32:01.449 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:01.449+0000 7f42a1a00640 1 -- start start 2024-04-20T11:32:01.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:01.450+0000 7f42a1a00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f429c0b4ac0 0x7f429c0b3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:01.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:01.450+0000 7f42a1a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f429c15d1f0 0x7f429c0d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:01.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:01.450+0000 7f42a1a00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f429c15da60 con 0x7f429c0d05c0 2024-04-20T11:32:01.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:01.450+0000 7f42a1a00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f429c0d0f60 con 0x7f429c0b4ac0 2024-04-20T11:32:01.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:01.450+0000 7f42a1a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f429c0b3740 con 0x7f429c15d1f0 2024-04-20T11:32:01.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:01.451+0000 7f4293400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f429c0d05c0 0x7f429c15cae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:34714/0 (socket says 172.21.15.152:34714) 2024-04-20T11:32:01.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:01.451+0000 7f4293400640 1 -- 172.21.15.152:0/4035575631 learned_addr learned my addr 172.21.15.152:0/4035575631 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:32:01.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:01.451+0000 7f42a1000640 1 --2- 172.21.15.152:0/4035575631 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f429c15d1f0 0x7f429c0d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:01.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:01.451+0000 7f429be00640 1 --2- 172.21.15.152:0/4035575631 >> v2:172.21.15.175:3300/0 conn(0x7f429c0b4ac0 0x7f429c0b3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:01.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:01.451+0000 7f429b400640 1 -- 172.21.15.152:0/4035575631 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1422535541 0 0) 0x7f429c15da60 con 0x7f429c0d05c0 2024-04-20T11:32:01.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:01.451+0000 7f429b400640 1 -- 172.21.15.152:0/4035575631 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4280003520 con 0x7f429c0d05c0 2024-04-20T11:32:01.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:01.451+0000 7f429b400640 1 -- 172.21.15.152:0/4035575631 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 76954751 0 0) 0x7f4280003520 con 0x7f429c0d05c0 2024-04-20T11:32:01.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:01.451+0000 7f429b400640 1 -- 172.21.15.152:0/4035575631 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f429c15d1f0 msgr2=0x7f429c0d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:01.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:01.451+0000 7f429b400640 1 --2- 172.21.15.152:0/4035575631 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f429c15d1f0 0x7f429c0d0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:01.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:01.451+0000 7f429b400640 1 -- 172.21.15.152:0/4035575631 >> v2:172.21.15.175:3300/0 conn(0x7f429c0b4ac0 msgr2=0x7f429c0b3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:01.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:01.451+0000 7f429b400640 1 --2- 172.21.15.152:0/4035575631 >> v2:172.21.15.175:3300/0 conn(0x7f429c0b4ac0 0x7f429c0b3bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:01.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:01.451+0000 7f429b400640 1 -- 172.21.15.152:0/4035575631 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f429c15d780 con 0x7f429c0d05c0 2024-04-20T11:32:01.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:01.452+0000 7f429b400640 1 -- 172.21.15.152:0/4035575631 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f4284001670 con 0x7f429c0d05c0 2024-04-20T11:32:01.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:01.452+0000 7f429be00640 1 --2- 172.21.15.152:0/4035575631 >> v2:172.21.15.175:3300/0 conn(0x7f429c0b4ac0 0x7f429c0b3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:32:01.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:01.452+0000 7f429b400640 1 -- 172.21.15.152:0/4035575631 >> v1:172.21.15.152:6789/0 conn(0x7f429c0d05c0 legacy=0x7f429c15cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:01.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:01.452+0000 7f429b400640 1 --2- 172.21.15.152:0/4035575631 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f42800039b0 0x7f4280003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:01.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:01.452+0000 7f429b400640 1 --2- 172.21.15.152:0/4035575631 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4280004800 0x7f4280004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:01.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:01.452+0000 7f429b400640 1 --2- 172.21.15.152:0/4035575631 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f429c15d1f0 0x7f42800051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:01.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:01.452+0000 7f429b400640 1 -- 172.21.15.152:0/4035575631 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f429c0b3740 con 0x7f4280004800 2024-04-20T11:32:01.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:01.452+0000 7f429b400640 1 -- 172.21.15.152:0/4035575631 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f428000b9c0 con 0x7f42800039b0 2024-04-20T11:32:01.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:01.452+0000 7f429b400640 1 -- 172.21.15.152:0/4035575631 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f428000bb70 con 0x7f429c15d1f0 2024-04-20T11:32:01.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:01.452+0000 7f4293400640 1 --2- 172.21.15.152:0/4035575631 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4280004800 0x7f4280004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:01.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:01.452+0000 7f429be00640 1 --2- 172.21.15.152:0/4035575631 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f42800039b0 0x7f4280003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:01.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:01.452+0000 7f42a1000640 1 --2- 172.21.15.152:0/4035575631 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f429c15d1f0 0x7f42800051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:01.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:01.453+0000 7f4293400640 1 -- 172.21.15.152:0/4035575631 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f429c15d1f0 msgr2=0x7f42800051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:01.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:01.453+0000 7f4293400640 1 --2- 172.21.15.152:0/4035575631 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f429c15d1f0 0x7f42800051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:01.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:01.453+0000 7f4293400640 1 -- 172.21.15.152:0/4035575631 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f42800039b0 msgr2=0x7f4280003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:01.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:01.453+0000 7f4293400640 1 --2- 172.21.15.152:0/4035575631 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f42800039b0 0x7f4280003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:01.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:01.453+0000 7f4293400640 1 -- 172.21.15.152:0/4035575631 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f429c15d780 con 0x7f4280004800 2024-04-20T11:32:01.456 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:01.454+0000 7f4293400640 1 --2- 172.21.15.152:0/4035575631 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4280004800 0x7f4280004c80 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f428400c260 tx=0x7f4280005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e073919084e8bda3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:01.456 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:01.454+0000 7f429b400640 1 -- 172.21.15.152:0/4035575631 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4284007d70 con 0x7f4280004800 2024-04-20T11:32:01.457 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:01.454+0000 7f429b400640 1 -- 172.21.15.152:0/4035575631 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4284030070 con 0x7f4280004800 2024-04-20T11:32:01.457 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:01.454+0000 7f429b400640 1 -- 172.21.15.152:0/4035575631 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f428402cd50 con 0x7f4280004800 2024-04-20T11:32:01.457 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:01.454+0000 7f42a1a00640 1 -- 172.21.15.152:0/4035575631 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4280004800 msgr2=0x7f4280004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:01.457 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:01.454+0000 7f42a1a00640 1 --2- 172.21.15.152:0/4035575631 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4280004800 0x7f4280004c80 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f428400c260 tx=0x7f4280005830 comp rx=0 tx=0).stop 2024-04-20T11:32:01.457 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:01.455+0000 7f42a1000640 1 -- 172.21.15.152:0/4035575631 reap_dead start 2024-04-20T11:32:01.457 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:01.455+0000 7f42a1a00640 1 -- 172.21.15.152:0/4035575631 shutdown_connections 2024-04-20T11:32:01.457 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:01.455+0000 7f42a1a00640 1 -- 172.21.15.152:0/4035575631 >> 172.21.15.152:0/4035575631 conn(0x7f429c06bda0 msgr2=0x7f429c06c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:01.457 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:01.455+0000 7f42a1a00640 1 -- 172.21.15.152:0/4035575631 shutdown_connections 2024-04-20T11:32:01.457 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:01.455+0000 7f42a1a00640 1 -- 172.21.15.152:0/4035575631 wait complete. 2024-04-20T11:32:01.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:01.456+0000 7f42a1a00640 1 Processor -- start 2024-04-20T11:32:01.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:01.457+0000 7f42a1a00640 1 -- start start 2024-04-20T11:32:01.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:01.457+0000 7f42a1a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4280004800 0x7f429c09b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:01.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:01.457+0000 7f42a1a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f429c09c7b0 0x7f429c1f5430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:01.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:01.457+0000 7f42a1a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f429c09cc30 0x7f429c1f5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:01.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:01.457+0000 7f42a1a00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f429c0b4ea0 con 0x7f429c09c7b0 2024-04-20T11:32:01.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:01.458+0000 7f42a1a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f429c0b33d0 con 0x7f4280004800 2024-04-20T11:32:01.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:01.458+0000 7f42a1a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f429c0d0a20 con 0x7f429c09cc30 2024-04-20T11:32:01.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:01.458+0000 7f4293400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f429c09c7b0 0x7f429c1f5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:01.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:01.458+0000 7f4293400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f429c09c7b0 0x7f429c1f5430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35398/0 (socket says 172.21.15.152:35398) 2024-04-20T11:32:01.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:01.458+0000 7f4293400640 1 -- 172.21.15.152:0/3082960531 learned_addr learned my addr 172.21.15.152:0/3082960531 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:32:01.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:01.458+0000 7f429be00640 1 --2- 172.21.15.152:0/3082960531 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4280004800 0x7f429c09b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:01.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:01.458+0000 7f4293400640 1 -- 172.21.15.152:0/3082960531 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f429c09cc30 msgr2=0x7f429c1f5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:01.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:01.458+0000 7f42a1000640 1 --2- 172.21.15.152:0/3082960531 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f429c09cc30 0x7f429c1f5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:01.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:01.458+0000 7f4293400640 1 --2- 172.21.15.152:0/3082960531 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f429c09cc30 0x7f429c1f5970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:01.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:01.458+0000 7f4293400640 1 -- 172.21.15.152:0/3082960531 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4280004800 msgr2=0x7f429c09b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:01.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:01.458+0000 7f4293400640 1 --2- 172.21.15.152:0/3082960531 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4280004800 0x7f429c09b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:01.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:01.458+0000 7f4293400640 1 -- 172.21.15.152:0/3082960531 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f429c09d200 con 0x7f429c09c7b0 2024-04-20T11:32:01.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:01.458+0000 7f429be00640 1 --2- 172.21.15.152:0/3082960531 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4280004800 0x7f429c09b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:32:01.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:01.459+0000 7f4293400640 1 --2- 172.21.15.152:0/3082960531 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f429c09c7b0 0x7f429c1f5430 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f428400c260 tx=0x7f4284000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:01.461 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:01.459+0000 7f4298c00640 1 -- 172.21.15.152:0/3082960531 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4284004bf0 con 0x7f429c09c7b0 2024-04-20T11:32:01.461 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:01.459+0000 7f4298c00640 1 -- 172.21.15.152:0/3082960531 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4284004d90 con 0x7f429c09c7b0 2024-04-20T11:32:01.461 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:01.459+0000 7f4298c00640 1 -- 172.21.15.152:0/3082960531 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4284007290 con 0x7f429c09c7b0 2024-04-20T11:32:01.461 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:01.459+0000 7f42a1a00640 1 -- 172.21.15.152:0/3082960531 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f429c09d4e0 con 0x7f429c09c7b0 2024-04-20T11:32:01.461 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:01.459+0000 7f42a1a00640 1 -- 172.21.15.152:0/3082960531 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f429c09daa0 con 0x7f429c09c7b0 2024-04-20T11:32:01.461 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:01.460+0000 7f42a1a00640 1 -- 172.21.15.152:0/3082960531 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f429c0689f0 con 0x7f429c09c7b0 2024-04-20T11:32:01.461 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:01.461+0000 7f4298c00640 1 -- 172.21.15.152:0/3082960531 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 11) ==== 91346+0+0 (secure 0 0 0) 0x7f428403f070 con 0x7f429c09c7b0 2024-04-20T11:32:01.462 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:01.462+0000 7f4298c00640 1 --2- 172.21.15.152:0/3082960531 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f427806cca0 0x7f427806f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:01.462 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:01.462+0000 7f4298c00640 1 -- 172.21.15.152:0/3082960531 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5709+0+0 (secure 0 0 0) 0x7f42840ad2c0 con 0x7f429c09c7b0 2024-04-20T11:32:01.463 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:01.462+0000 7f429be00640 1 --2- 172.21.15.152:0/3082960531 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f427806cca0 0x7f427806f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:01.463 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:01.463+0000 7f429be00640 1 --2- 172.21.15.152:0/3082960531 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f427806cca0 0x7f427806f160 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f428c006fd0 tx=0x7f428c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:01.467 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:01.467+0000 7f4298c00640 1 -- 172.21.15.152:0/3082960531 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f428407b3b0 con 0x7f429c09c7b0 2024-04-20T11:32:01.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:01.708+0000 7f42a1a00640 1 -- 172.21.15.152:0/3082960531 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f429c15da60 con 0x7f429c09c7b0 2024-04-20T11:32:02.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:02.367+0000 7f4298c00640 1 -- 172.21.15.152:0/3082960531 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v44) ==== 136+0+0 (secure 0 0 0) 0x7f428407acb0 con 0x7f429c09c7b0 2024-04-20T11:32:02.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:02.371+0000 7f42a1a00640 1 -- 172.21.15.152:0/3082960531 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f427806cca0 msgr2=0x7f427806f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:02.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:02.371+0000 7f42a1a00640 1 --2- 172.21.15.152:0/3082960531 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f427806cca0 0x7f427806f160 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f428c006fd0 tx=0x7f428c008040 comp rx=0 tx=0).stop 2024-04-20T11:32:02.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:02.371+0000 7f42a1a00640 1 -- 172.21.15.152:0/3082960531 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f429c09c7b0 msgr2=0x7f429c1f5430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:02.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:02.371+0000 7f42a1a00640 1 --2- 172.21.15.152:0/3082960531 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f429c09c7b0 0x7f429c1f5430 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f428400c260 tx=0x7f4284000fe0 comp rx=0 tx=0).stop 2024-04-20T11:32:02.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:02.371+0000 7f42a1a00640 1 -- 172.21.15.152:0/3082960531 shutdown_connections 2024-04-20T11:32:02.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:02.371+0000 7f42a1a00640 1 --2- 172.21.15.152:0/3082960531 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f427806cca0 0x7f427806f160 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:02.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:02.371+0000 7f42a1a00640 1 --2- 172.21.15.152:0/3082960531 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f429c09cc30 0x7f429c1f5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:02.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:02.371+0000 7f42a1a00640 1 --2- 172.21.15.152:0/3082960531 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f429c09c7b0 0x7f429c1f5430 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:02.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:02.371+0000 7f42a1a00640 1 --2- 172.21.15.152:0/3082960531 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4280004800 0x7f429c09b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:02.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:02.371+0000 7f42a1a00640 1 -- 172.21.15.152:0/3082960531 >> 172.21.15.152:0/3082960531 conn(0x7f429c06bda0 msgr2=0x7f429c0b12a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:02.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:02.372+0000 7f42a1a00640 1 -- 172.21.15.152:0/3082960531 shutdown_connections 2024-04-20T11:32:02.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:02.372+0000 7f42a1a00640 1 -- 172.21.15.152:0/3082960531 wait complete. 2024-04-20T11:32:02.398 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-04-20T11:32:02.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:02.599+0000 7f1513200640 1 Processor -- start 2024-04-20T11:32:02.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:02.599+0000 7f1513200640 1 -- start start 2024-04-20T11:32:02.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:02.600+0000 7f1513200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f151415c7f0 0x7f1514072b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:02.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:02.601+0000 7f1513200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f1514073b60 0x7f15140ad2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:02.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:02.601+0000 7f1513200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f15140b4160 0x7f15140b4570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:02.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:02.601+0000 7f1513200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f1514073ff0 con 0x7f1514073b60 2024-04-20T11:32:02.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:02.601+0000 7f1513200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f15140ce140 con 0x7f151415c7f0 2024-04-20T11:32:02.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:02.601+0000 7f1513200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f15140b4ab0 con 0x7f15140b4160 2024-04-20T11:32:02.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:02.601+0000 7f1511400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f1514073b60 0x7f15140ad2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:02.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:02.601+0000 7f1511400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f1514073b60 0x7f15140ad2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35406/0 (socket says 172.21.15.152:35406) 2024-04-20T11:32:02.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:02.601+0000 7f1511400640 1 -- 172.21.15.152:0/202921857 learned_addr learned my addr 172.21.15.152:0/202921857 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:32:02.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:02.601+0000 7f1511e00640 1 --2- 172.21.15.152:0/202921857 >> v2:172.21.15.175:3300/0 conn(0x7f151415c7f0 0x7f1514072b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:02.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:02.601+0000 7f1512800640 1 --2- 172.21.15.152:0/202921857 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f15140b4160 0x7f15140b4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:02.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:02.602+0000 7f1511400640 1 -- 172.21.15.152:0/202921857 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f15140b4160 msgr2=0x7f15140b4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:02.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:02.602+0000 7f1511400640 1 --2- 172.21.15.152:0/202921857 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f15140b4160 0x7f15140b4570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:02.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:02.602+0000 7f1511400640 1 -- 172.21.15.152:0/202921857 >> v2:172.21.15.175:3300/0 conn(0x7f151415c7f0 msgr2=0x7f1514072b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:02.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:02.602+0000 7f1511400640 1 --2- 172.21.15.152:0/202921857 >> v2:172.21.15.175:3300/0 conn(0x7f151415c7f0 0x7f1514072b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:02.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:02.602+0000 7f1511400640 1 -- 172.21.15.152:0/202921857 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15140ce390 con 0x7f1514073b60 2024-04-20T11:32:02.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:02.603+0000 7f1511400640 1 --2- 172.21.15.152:0/202921857 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1514073b60 0x7f15140ad2b0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f1514070c60 tx=0x7f150002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd0b0dbf4d3dc6f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:02.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:02.603+0000 7f1510a00640 1 -- 172.21.15.152:0/202921857 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f150002e070 con 0x7f1514073b60 2024-04-20T11:32:02.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:02.603+0000 7f1510a00640 1 -- 172.21.15.152:0/202921857 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1500037070 con 0x7f1514073b60 2024-04-20T11:32:02.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:02.603+0000 7f1510a00640 1 -- 172.21.15.152:0/202921857 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1500031a00 con 0x7f1514073b60 2024-04-20T11:32:02.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:02.604+0000 7f1513200640 1 -- 172.21.15.152:0/202921857 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1514073b60 msgr2=0x7f15140ad2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:02.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:02.604+0000 7f1513200640 1 --2- 172.21.15.152:0/202921857 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1514073b60 0x7f15140ad2b0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f1514070c60 tx=0x7f150002b880 comp rx=0 tx=0).stop 2024-04-20T11:32:02.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:02.604+0000 7f1513200640 1 -- 172.21.15.152:0/202921857 shutdown_connections 2024-04-20T11:32:02.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:02.604+0000 7f1513200640 1 --2- 172.21.15.152:0/202921857 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f15140b4160 0x7f15140b4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:02.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:02.604+0000 7f1513200640 1 --2- 172.21.15.152:0/202921857 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1514073b60 0x7f15140ad2b0 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:02.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:02.604+0000 7f1513200640 1 --2- 172.21.15.152:0/202921857 >> v2:172.21.15.175:3300/0 conn(0x7f151415c7f0 0x7f1514072b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:02.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:02.604+0000 7f1513200640 1 -- 172.21.15.152:0/202921857 >> 172.21.15.152:0/202921857 conn(0x7f15140d0260 msgr2=0x7f15140d0670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:02.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:02.604+0000 7f1513200640 1 -- 172.21.15.152:0/202921857 shutdown_connections 2024-04-20T11:32:02.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:02.604+0000 7f1513200640 1 -- 172.21.15.152:0/202921857 wait complete. 2024-04-20T11:32:02.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:02.606+0000 7f1513200640 1 Processor -- start 2024-04-20T11:32:02.607 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:02.606+0000 7f1513200640 1 -- start start 2024-04-20T11:32:02.607 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:02.607+0000 7f1513200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1514073b60 0x7f1514154d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:02.607 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:02.607+0000 7f1513200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f15140b4160 0x7f1514156d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:02.608 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:02.607+0000 7f1513200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f151406f180 0x7f151414e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:02.608 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:02.607+0000 7f1513200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f15140cec30 con 0x7f151406f180 2024-04-20T11:32:02.608 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:02.607+0000 7f1513200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f15140ce950 con 0x7f1514073b60 2024-04-20T11:32:02.608 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:02.607+0000 7f1513200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f15140cef10 con 0x7f15140b4160 2024-04-20T11:32:02.608 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:02.607+0000 7f1512800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f151406f180 0x7f151414e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:02.608 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:02.607+0000 7f1512800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f151406f180 0x7f151414e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35420/0 (socket says 172.21.15.152:35420) 2024-04-20T11:32:02.608 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:02.607+0000 7f1512800640 1 -- 172.21.15.152:0/1251652768 learned_addr learned my addr 172.21.15.152:0/1251652768 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:32:02.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:02.608+0000 7f1511400640 1 --2- 172.21.15.152:0/1251652768 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f15140b4160 0x7f1514156d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:02.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:02.608+0000 7f1511e00640 1 --2- 172.21.15.152:0/1251652768 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1514073b60 0x7f1514154d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:02.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:02.608+0000 7f1512800640 1 -- 172.21.15.152:0/1251652768 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f15140b4160 msgr2=0x7f1514156d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:02.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:02.608+0000 7f1512800640 1 --2- 172.21.15.152:0/1251652768 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f15140b4160 0x7f1514156d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:02.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:02.608+0000 7f1512800640 1 -- 172.21.15.152:0/1251652768 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1514073b60 msgr2=0x7f1514154d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:02.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:02.608+0000 7f1512800640 1 --2- 172.21.15.152:0/1251652768 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1514073b60 0x7f1514154d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:02.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:02.608+0000 7f1512800640 1 -- 172.21.15.152:0/1251652768 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1514155370 con 0x7f151406f180 2024-04-20T11:32:02.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:02.608+0000 7f1511400640 1 --2- 172.21.15.152:0/1251652768 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f15140b4160 0x7f1514156d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:32:02.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:02.608+0000 7f1511e00640 1 --2- 172.21.15.152:0/1251652768 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1514073b60 0x7f1514154d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:32:02.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:02.608+0000 7f1512800640 1 --2- 172.21.15.152:0/1251652768 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f151406f180 0x7f151414e5d0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f150c007d20 tx=0x7f150c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:02.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:02.608+0000 7f150a000640 1 -- 172.21.15.152:0/1251652768 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f150c014420 con 0x7f151406f180 2024-04-20T11:32:02.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:02.608+0000 7f150a000640 1 -- 172.21.15.152:0/1251652768 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f150c019070 con 0x7f151406f180 2024-04-20T11:32:02.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:02.608+0000 7f1513200640 1 -- 172.21.15.152:0/1251652768 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1514155650 con 0x7f151406f180 2024-04-20T11:32:02.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:02.608+0000 7f150a000640 1 -- 172.21.15.152:0/1251652768 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f150c014820 con 0x7f151406f180 2024-04-20T11:32:02.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:02.608+0000 7f1513200640 1 -- 172.21.15.152:0/1251652768 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1514155c10 con 0x7f151406f180 2024-04-20T11:32:02.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:02.610+0000 7f1513200640 1 -- 172.21.15.152:0/1251652768 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1514075e60 con 0x7f151406f180 2024-04-20T11:32:02.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:02.610+0000 7f150a000640 1 -- 172.21.15.152:0/1251652768 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 11) ==== 91346+0+0 (secure 0 0 0) 0x7f150c0149c0 con 0x7f151406f180 2024-04-20T11:32:02.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:02.611+0000 7f150a000640 1 --2- 172.21.15.152:0/1251652768 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f14f406cc50 0x7f14f406f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:02.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:02.611+0000 7f150a000640 1 -- 172.21.15.152:0/1251652768 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5709+0+0 (secure 0 0 0) 0x7f150c08c850 con 0x7f151406f180 2024-04-20T11:32:02.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:02.612+0000 7f1511e00640 1 --2- 172.21.15.152:0/1251652768 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f14f406cc50 0x7f14f406f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:02.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:02.612+0000 7f1511e00640 1 --2- 172.21.15.152:0/1251652768 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f14f406cc50 0x7f14f406f110 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f1504006fd0 tx=0x7f1504008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:02.616 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:02.616+0000 7f150a000640 1 -- 172.21.15.152:0/1251652768 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f150c05a900 con 0x7f151406f180 2024-04-20T11:32:02.844 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:02.844+0000 7f1513200640 1 -- 172.21.15.152:0/1251652768 --> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f15140af710 con 0x7f14f406cc50 2024-04-20T11:32:02.847 INFO:teuthology.orchestra.run.smithi152.stdout:[ 2024-04-20T11:32:02.847 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:32:02.847 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "cephfs" 2024-04-20T11:32:02.847 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:02.847 INFO:teuthology.orchestra.run.smithi152.stdout:] 2024-04-20T11:32:02.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:02.846+0000 7f150a000640 1 -- 172.21.15.152:0/1251652768 <== mgr.4113 v2:172.21.15.152:6832/2285110240 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f15140af710 con 0x7f14f406cc50 2024-04-20T11:32:02.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:02.849+0000 7f1513200640 1 -- 172.21.15.152:0/1251652768 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f14f406cc50 msgr2=0x7f14f406f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:02.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:02.849+0000 7f1513200640 1 --2- 172.21.15.152:0/1251652768 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f14f406cc50 0x7f14f406f110 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f1504006fd0 tx=0x7f1504008040 comp rx=0 tx=0).stop 2024-04-20T11:32:02.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:02.849+0000 7f1513200640 1 -- 172.21.15.152:0/1251652768 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f151406f180 msgr2=0x7f151414e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:02.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:02.849+0000 7f1513200640 1 --2- 172.21.15.152:0/1251652768 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f151406f180 0x7f151414e5d0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f150c007d20 tx=0x7f150c00e4b0 comp rx=0 tx=0).stop 2024-04-20T11:32:02.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:02.849+0000 7f1513200640 1 -- 172.21.15.152:0/1251652768 shutdown_connections 2024-04-20T11:32:02.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:02.849+0000 7f1513200640 1 --2- 172.21.15.152:0/1251652768 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f14f406cc50 0x7f14f406f110 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:02.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:02.849+0000 7f1513200640 1 --2- 172.21.15.152:0/1251652768 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f151406f180 0x7f151414e5d0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:02.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:02.849+0000 7f1513200640 1 --2- 172.21.15.152:0/1251652768 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f15140b4160 0x7f1514156d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:02.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:02.849+0000 7f1513200640 1 --2- 172.21.15.152:0/1251652768 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1514073b60 0x7f1514154d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:02.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:02.849+0000 7f1513200640 1 -- 172.21.15.152:0/1251652768 >> 172.21.15.152:0/1251652768 conn(0x7f15140d0260 msgr2=0x7f1514072b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:02.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:02.849+0000 7f1513200640 1 -- 172.21.15.152:0/1251652768 shutdown_connections 2024-04-20T11:32:02.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:02.849+0000 7f1513200640 1 -- 172.21.15.152:0/1251652768 wait complete. 2024-04-20T11:32:02.869 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-20T11:32:03.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.071+0000 7fd8f9a00640 1 Processor -- start 2024-04-20T11:32:03.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.071+0000 7fd8f9a00640 1 -- start start 2024-04-20T11:32:03.074 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.073+0000 7fd8f9a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd8f415fbf0 0x7fd8f415c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:03.074 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.073+0000 7fd8f9a00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8f40d00d0 con 0x7fd8f40cf110 2024-04-20T11:32:03.074 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.073+0000 7fd8f9a00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8f40cfb10 con 0x7fd8f406ee00 2024-04-20T11:32:03.075 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.073+0000 7fd8f9a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd8f40cfdf0 con 0x7fd8f415fbf0 2024-04-20T11:32:03.075 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.073+0000 7fd8f9000640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fd8f40cf110 0x7fd8f40b3ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:34716/0 (socket says 172.21.15.152:34716) 2024-04-20T11:32:03.075 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.073+0000 7fd8f9000640 1 -- 172.21.15.152:0/3865394438 learned_addr learned my addr 172.21.15.152:0/3865394438 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:32:03.075 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.074+0000 7fd8eb400640 1 --2- 172.21.15.152:0/3865394438 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd8f415fbf0 0x7fd8f415c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:03.080 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.074+0000 7fd8f3400640 1 -- 172.21.15.152:0/3865394438 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2598293972 0 0) 0x7fd8f40d00d0 con 0x7fd8f40cf110 2024-04-20T11:32:03.080 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.074+0000 7fd8f3400640 1 -- 172.21.15.152:0/3865394438 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd8d8005730 con 0x7fd8f40cf110 2024-04-20T11:32:03.080 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.074+0000 7fd8f3400640 1 -- 172.21.15.152:0/3865394438 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1274053642 0 0) 0x7fd8f40cfb10 con 0x7fd8f406ee00 2024-04-20T11:32:03.080 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.074+0000 7fd8f3400640 1 -- 172.21.15.152:0/3865394438 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd8f40d00d0 con 0x7fd8f406ee00 2024-04-20T11:32:03.080 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.074+0000 7fd8f3400640 1 -- 172.21.15.152:0/3865394438 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2723627629 0 0) 0x7fd8d8005730 con 0x7fd8f40cf110 2024-04-20T11:32:03.081 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.074+0000 7fd8f3400640 1 -- 172.21.15.152:0/3865394438 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd8f415fbf0 msgr2=0x7fd8f415c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:03.081 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.074+0000 7fd8f3400640 1 --2- 172.21.15.152:0/3865394438 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd8f415fbf0 0x7fd8f415c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:03.081 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.074+0000 7fd8f3400640 1 -- 172.21.15.152:0/3865394438 >> v1:172.21.15.175:6789/0 conn(0x7fd8f406ee00 legacy=0x7fd8f406f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:03.081 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.075+0000 7fd8f3400640 1 -- 172.21.15.152:0/3865394438 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8f415bb90 con 0x7fd8f40cf110 2024-04-20T11:32:03.081 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.075+0000 7fd8f3400640 1 -- 172.21.15.152:0/3865394438 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7fd8ec001970 con 0x7fd8f40cf110 2024-04-20T11:32:03.081 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.075+0000 7fd8f3400640 1 -- 172.21.15.152:0/3865394438 >> v1:172.21.15.152:6789/0 conn(0x7fd8f40cf110 legacy=0x7fd8f40b3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:03.081 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.075+0000 7fd8f3400640 1 --2- 172.21.15.152:0/3865394438 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd8d8005d80 0x7fd8d8006180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:03.082 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.075+0000 7fd8f3400640 1 --2- 172.21.15.152:0/3865394438 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd8d8006a10 0x7fd8d8006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:03.082 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.075+0000 7fd8f3400640 1 --2- 172.21.15.152:0/3865394438 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd8f415fbf0 0x7fd8d80073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:03.082 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.075+0000 7fd8f3400640 1 -- 172.21.15.152:0/3865394438 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f40cfdf0 con 0x7fd8d8006a10 2024-04-20T11:32:03.082 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.075+0000 7fd8f3400640 1 -- 172.21.15.152:0/3865394438 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd8d800dc80 con 0x7fd8d8005d80 2024-04-20T11:32:03.082 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.075+0000 7fd8f3400640 1 -- 172.21.15.152:0/3865394438 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd8d800de80 con 0x7fd8f415fbf0 2024-04-20T11:32:03.082 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.075+0000 7fd8eb400640 1 --2- 172.21.15.152:0/3865394438 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd8d8006a10 0x7fd8d8006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:03.082 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.075+0000 7fd8f3e00640 1 --2- 172.21.15.152:0/3865394438 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd8d8005d80 0x7fd8d8006180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:03.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.075+0000 7fd8f9000640 1 --2- 172.21.15.152:0/3865394438 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd8f415fbf0 0x7fd8d80073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:03.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.075+0000 7fd8eb400640 1 -- 172.21.15.152:0/3865394438 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd8f415fbf0 msgr2=0x7fd8d80073d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:03.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.075+0000 7fd8eb400640 1 --2- 172.21.15.152:0/3865394438 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd8f415fbf0 0x7fd8d80073d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:03.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.075+0000 7fd8eb400640 1 -- 172.21.15.152:0/3865394438 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd8d8005d80 msgr2=0x7fd8d8006180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:03.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.075+0000 7fd8eb400640 1 --2- 172.21.15.152:0/3865394438 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd8d8005d80 0x7fd8d8006180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:03.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.076+0000 7fd8eb400640 1 -- 172.21.15.152:0/3865394438 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8d8005730 con 0x7fd8d8006a10 2024-04-20T11:32:03.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.076+0000 7fd8f9000640 1 --2- 172.21.15.152:0/3865394438 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd8f415fbf0 0x7fd8d80073d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:32:03.084 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.076+0000 7fd8f3e00640 1 --2- 172.21.15.152:0/3865394438 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd8d8005d80 0x7fd8d8006180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:32:03.084 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.077+0000 7fd8eb400640 1 --2- 172.21.15.152:0/3865394438 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd8d8006a10 0x7fd8d8006e90 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc009ac0 tx=0x7fd8d8007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b03019498adaada9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:03.084 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.077+0000 7fd8f3400640 1 -- 172.21.15.152:0/3865394438 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8dc039070 con 0x7fd8d8006a10 2024-04-20T11:32:03.084 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.077+0000 7fd8f3400640 1 -- 172.21.15.152:0/3865394438 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8dc02c6b0 con 0x7fd8d8006a10 2024-04-20T11:32:03.084 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.077+0000 7fd8f3400640 1 -- 172.21.15.152:0/3865394438 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8dc009690 con 0x7fd8d8006a10 2024-04-20T11:32:03.084 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.077+0000 7fd8f9a00640 1 -- 172.21.15.152:0/3865394438 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd8d8006a10 msgr2=0x7fd8d8006e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:03.084 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.077+0000 7fd8f9a00640 1 --2- 172.21.15.152:0/3865394438 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd8d8006a10 0x7fd8d8006e90 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc009ac0 tx=0x7fd8d8007a40 comp rx=0 tx=0).stop 2024-04-20T11:32:03.084 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.077+0000 7fd8f9000640 1 -- 172.21.15.152:0/3865394438 reap_dead start 2024-04-20T11:32:03.085 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.078+0000 7fd8f9a00640 1 -- 172.21.15.152:0/3865394438 shutdown_connections 2024-04-20T11:32:03.085 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.078+0000 7fd8f9a00640 1 -- 172.21.15.152:0/3865394438 >> 172.21.15.152:0/3865394438 conn(0x7fd8f4067ae0 msgr2=0x7fd8f406b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:03.085 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.078+0000 7fd8f9a00640 1 -- 172.21.15.152:0/3865394438 shutdown_connections 2024-04-20T11:32:03.085 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.078+0000 7fd8f9a00640 1 -- 172.21.15.152:0/3865394438 wait complete. 2024-04-20T11:32:03.089 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.080+0000 7fd8f9a00640 1 Processor -- start 2024-04-20T11:32:03.089 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.080+0000 7fd8f9a00640 1 -- start start 2024-04-20T11:32:03.090 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.081+0000 7fd8f9a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd8d8006a10 0x7fd8f4143240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:03.090 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.081+0000 7fd8f9a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd8f41437a0 0x7fd8f40abdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:03.090 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.081+0000 7fd8f9a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd8f40ae710 0x7fd8f40aeb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:03.090 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.081+0000 7fd8f9a00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f4144af0 con 0x7fd8f40ae710 2024-04-20T11:32:03.090 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.081+0000 7fd8f9a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f4144970 con 0x7fd8f41437a0 2024-04-20T11:32:03.090 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.081+0000 7fd8f9a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd8f40cf5a0 con 0x7fd8d8006a10 2024-04-20T11:32:03.090 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.082+0000 7fd8f9000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd8f40ae710 0x7fd8f40aeb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:03.090 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.082+0000 7fd8f9000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd8f40ae710 0x7fd8f40aeb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35440/0 (socket says 172.21.15.152:35440) 2024-04-20T11:32:03.090 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.082+0000 7fd8f9000640 1 -- 172.21.15.152:0/3679623255 learned_addr learned my addr 172.21.15.152:0/3679623255 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:32:03.090 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.082+0000 7fd8f3e00640 1 --2- 172.21.15.152:0/3679623255 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd8d8006a10 0x7fd8f4143240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:03.090 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.082+0000 7fd8eb400640 1 --2- 172.21.15.152:0/3679623255 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd8f41437a0 0x7fd8f40abdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:03.091 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.082+0000 7fd8f9000640 1 -- 172.21.15.152:0/3679623255 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd8d8006a10 msgr2=0x7fd8f4143240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:03.091 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.082+0000 7fd8f9000640 1 --2- 172.21.15.152:0/3679623255 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd8d8006a10 0x7fd8f4143240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:03.091 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.082+0000 7fd8f9000640 1 -- 172.21.15.152:0/3679623255 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd8f41437a0 msgr2=0x7fd8f40abdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:03.091 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.082+0000 7fd8f9000640 1 --2- 172.21.15.152:0/3679623255 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd8f41437a0 0x7fd8f40abdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:03.091 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.082+0000 7fd8f9000640 1 -- 172.21.15.152:0/3679623255 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8f41f0440 con 0x7fd8f40ae710 2024-04-20T11:32:03.091 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.083+0000 7fd8f9000640 1 --2- 172.21.15.152:0/3679623255 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd8f40ae710 0x7fd8f40aeb90 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fd8f40b51a0 tx=0x7fd8ec00b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:03.091 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.083+0000 7fd8f0c00640 1 -- 172.21.15.152:0/3679623255 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8ec004d60 con 0x7fd8f40ae710 2024-04-20T11:32:03.091 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.083+0000 7fd8f0c00640 1 -- 172.21.15.152:0/3679623255 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8ec00bb20 con 0x7fd8f40ae710 2024-04-20T11:32:03.091 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.083+0000 7fd8f9a00640 1 -- 172.21.15.152:0/3679623255 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8f40ac300 con 0x7fd8f40ae710 2024-04-20T11:32:03.091 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.083+0000 7fd8f0c00640 1 -- 172.21.15.152:0/3679623255 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8ec007070 con 0x7fd8f40ae710 2024-04-20T11:32:03.091 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.083+0000 7fd8f9a00640 1 -- 172.21.15.152:0/3679623255 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8f40ad4e0 con 0x7fd8f40ae710 2024-04-20T11:32:03.091 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.084+0000 7fd8f9a00640 1 -- 172.21.15.152:0/3679623255 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8f4075e60 con 0x7fd8f40ae710 2024-04-20T11:32:03.092 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.086+0000 7fd8f0c00640 1 -- 172.21.15.152:0/3679623255 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 11) ==== 91346+0+0 (secure 0 0 0) 0x7fd8ec01b020 con 0x7fd8f40ae710 2024-04-20T11:32:03.092 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.086+0000 7fd8f0c00640 1 --2- 172.21.15.152:0/3679623255 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fd8d006cc50 0x7fd8d006f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:03.092 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.087+0000 7fd8f0c00640 1 -- 172.21.15.152:0/3679623255 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5709+0+0 (secure 0 0 0) 0x7fd8ec08c390 con 0x7fd8f40ae710 2024-04-20T11:32:03.092 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.087+0000 7fd8f3e00640 1 --2- 172.21.15.152:0/3679623255 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fd8d006cc50 0x7fd8d006f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:03.092 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.087+0000 7fd8f3e00640 1 --2- 172.21.15.152:0/3679623255 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fd8d006cc50 0x7fd8d006f110 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fd8f40b1eb0 tx=0x7fd8e4009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:03.092 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.091+0000 7fd8f0c00640 1 -- 172.21.15.152:0/3679623255 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fd8ec05a260 con 0x7fd8f40ae710 2024-04-20T11:32:03.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.294+0000 7fd8f9a00640 1 -- 172.21.15.152:0/3679623255 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fd8f40ac530 con 0x7fd8f40ae710 2024-04-20T11:32:03.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.295+0000 7fd8f0c00640 1 -- 172.21.15.152:0/3679623255 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) ==== 122+0+0 (secure 0 0 0) 0x7fd8ec059b60 con 0x7fd8f40ae710 2024-04-20T11:32:03.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.298+0000 7fd8f9a00640 1 -- 172.21.15.152:0/3679623255 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fd8d006cc50 msgr2=0x7fd8d006f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:03.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.298+0000 7fd8f9a00640 1 --2- 172.21.15.152:0/3679623255 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fd8d006cc50 0x7fd8d006f110 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fd8f40b1eb0 tx=0x7fd8e4009040 comp rx=0 tx=0).stop 2024-04-20T11:32:03.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.298+0000 7fd8f9a00640 1 -- 172.21.15.152:0/3679623255 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd8f40ae710 msgr2=0x7fd8f40aeb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:03.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.298+0000 7fd8f9a00640 1 --2- 172.21.15.152:0/3679623255 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd8f40ae710 0x7fd8f40aeb90 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fd8f40b51a0 tx=0x7fd8ec00b4a0 comp rx=0 tx=0).stop 2024-04-20T11:32:03.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.298+0000 7fd8f9a00640 1 -- 172.21.15.152:0/3679623255 shutdown_connections 2024-04-20T11:32:03.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.298+0000 7fd8f9a00640 1 --2- 172.21.15.152:0/3679623255 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fd8d006cc50 0x7fd8d006f110 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:03.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.298+0000 7fd8f9a00640 1 --2- 172.21.15.152:0/3679623255 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd8f40ae710 0x7fd8f40aeb90 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:03.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.298+0000 7fd8f9a00640 1 --2- 172.21.15.152:0/3679623255 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd8f41437a0 0x7fd8f40abdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:03.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.298+0000 7fd8f9a00640 1 --2- 172.21.15.152:0/3679623255 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd8d8006a10 0x7fd8f4143240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:03.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.298+0000 7fd8f9a00640 1 -- 172.21.15.152:0/3679623255 >> 172.21.15.152:0/3679623255 conn(0x7fd8f4067ae0 msgr2=0x7fd8f406e440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:03.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.298+0000 7fd8f9a00640 1 -- 172.21.15.152:0/3679623255 shutdown_connections 2024-04-20T11:32:03.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.298+0000 7fd8f9a00640 1 -- 172.21.15.152:0/3679623255 wait complete. 2024-04-20T11:32:03.319 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-20T11:32:03.319 DEBUG:teuthology.orchestra.run.smithi152:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:32:03.368 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:32:03.607 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.607+0000 7eff06800640 1 Processor -- start 2024-04-20T11:32:03.608 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.607+0000 7eff06800640 1 -- start start 2024-04-20T11:32:03.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.608+0000 7eff06800640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7eff0006f510 0x7eff00140dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:03.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.609+0000 7eff06800640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff000cf0a0 con 0x7eff0013e530 2024-04-20T11:32:03.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.609+0000 7eff06800640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff000d00e0 con 0x7eff00141310 2024-04-20T11:32:03.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.609+0000 7eff06800640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7eff00141b00 con 0x7eff0006f510 2024-04-20T11:32:03.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.609+0000 7eff05400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7eff0013e530 0x7eff0006ee00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:34718/0 (socket says 172.21.15.152:34718) 2024-04-20T11:32:03.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.609+0000 7eff05400640 1 -- 172.21.15.152:0/1449598380 learned_addr learned my addr 172.21.15.152:0/1449598380 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:32:03.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.609+0000 7efeffe00640 1 -- 172.21.15.152:0/1449598380 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2019338467 0 0) 0x7efef0001490 con 0x7eff0013e530 2024-04-20T11:32:03.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.609+0000 7efeffe00640 1 -- 172.21.15.152:0/1449598380 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efee4005730 con 0x7eff0013e530 2024-04-20T11:32:03.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.609+0000 7eff04a00640 1 --2- 172.21.15.152:0/1449598380 >> v2:172.21.15.175:3300/0 conn(0x7eff0006f510 0x7eff00140dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:03.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.610+0000 7efeffe00640 1 -- 172.21.15.152:0/1449598380 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1804852946 0 0) 0x7efee4005730 con 0x7eff0013e530 2024-04-20T11:32:03.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.610+0000 7efeffe00640 1 -- 172.21.15.152:0/1449598380 >> v2:172.21.15.175:3300/0 conn(0x7eff0006f510 msgr2=0x7eff00140dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:03.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.610+0000 7efeffe00640 1 --2- 172.21.15.152:0/1449598380 >> v2:172.21.15.175:3300/0 conn(0x7eff0006f510 0x7eff00140dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:03.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.610+0000 7efeffe00640 1 -- 172.21.15.152:0/1449598380 >> v1:172.21.15.175:6789/0 conn(0x7eff00141310 legacy=0x7eff00071cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:03.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.610+0000 7efeffe00640 1 -- 172.21.15.152:0/1449598380 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff000cf320 con 0x7eff0013e530 2024-04-20T11:32:03.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.610+0000 7efeffe00640 1 -- 172.21.15.152:0/1449598380 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7efef0001bc0 con 0x7eff0013e530 2024-04-20T11:32:03.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.610+0000 7efeffe00640 1 -- 172.21.15.152:0/1449598380 >> v1:172.21.15.152:6789/0 conn(0x7eff0013e530 legacy=0x7eff0006ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:03.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.610+0000 7efeffe00640 1 --2- 172.21.15.152:0/1449598380 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efee4005c10 0x7efee4006010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:03.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.610+0000 7efeffe00640 1 --2- 172.21.15.152:0/1449598380 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efee4006a10 0x7efee4006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:03.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.610+0000 7efeffe00640 1 --2- 172.21.15.152:0/1449598380 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efee40073d0 0x7efee400dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:03.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.610+0000 7efeffe00640 1 -- 172.21.15.152:0/1449598380 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7efee400e2a0 con 0x7efee4006a10 2024-04-20T11:32:03.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.610+0000 7efeffe00640 1 -- 172.21.15.152:0/1449598380 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7efee400e420 con 0x7efee4005c10 2024-04-20T11:32:03.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.610+0000 7efeffe00640 1 -- 172.21.15.152:0/1449598380 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7efee400e620 con 0x7efee40073d0 2024-04-20T11:32:03.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.610+0000 7eff04a00640 1 --2- 172.21.15.152:0/1449598380 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efee4006a10 0x7efee4006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:03.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.611+0000 7eff05400640 1 --2- 172.21.15.152:0/1449598380 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efee4005c10 0x7efee4006010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:03.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.611+0000 7eff04a00640 1 -- 172.21.15.152:0/1449598380 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efee40073d0 msgr2=0x7efee400dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:03.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.611+0000 7eff04a00640 1 --2- 172.21.15.152:0/1449598380 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efee40073d0 0x7efee400dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:03.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.611+0000 7eff04a00640 1 -- 172.21.15.152:0/1449598380 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efee4005c10 msgr2=0x7efee4006010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:03.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.611+0000 7eff04a00640 1 --2- 172.21.15.152:0/1449598380 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efee4005c10 0x7efee4006010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:03.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.611+0000 7eff04a00640 1 -- 172.21.15.152:0/1449598380 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efee4005730 con 0x7efee4006a10 2024-04-20T11:32:03.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.611+0000 7eff05400640 1 --2- 172.21.15.152:0/1449598380 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efee4005c10 0x7efee4006010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:32:03.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.611+0000 7eff05e00640 1 -- 172.21.15.152:0/1449598380 reap_dead start 2024-04-20T11:32:03.614 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.612+0000 7eff04a00640 1 --2- 172.21.15.152:0/1449598380 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efee4006a10 0x7efee4006e90 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7efeec009ac0 tx=0x7efee4007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fd1fb57e6517433 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:03.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.612+0000 7efeffe00640 1 -- 172.21.15.152:0/1449598380 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeec039070 con 0x7efee4006a10 2024-04-20T11:32:03.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.612+0000 7efeffe00640 1 -- 172.21.15.152:0/1449598380 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efeec02c6b0 con 0x7efee4006a10 2024-04-20T11:32:03.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.612+0000 7efeffe00640 1 -- 172.21.15.152:0/1449598380 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeec009690 con 0x7efee4006a10 2024-04-20T11:32:03.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.613+0000 7eff06800640 1 -- 172.21.15.152:0/1449598380 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efee4006a10 msgr2=0x7efee4006e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:03.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.613+0000 7eff06800640 1 --2- 172.21.15.152:0/1449598380 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efee4006a10 0x7efee4006e90 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7efeec009ac0 tx=0x7efee4007a40 comp rx=0 tx=0).stop 2024-04-20T11:32:03.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.613+0000 7eff06800640 1 -- 172.21.15.152:0/1449598380 shutdown_connections 2024-04-20T11:32:03.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.613+0000 7eff06800640 1 --2- 172.21.15.152:0/1449598380 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efee4006a10 0x7efee4006e90 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:03.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.613+0000 7eff06800640 1 -- 172.21.15.152:0/1449598380 >> 172.21.15.152:0/1449598380 conn(0x7eff00067ae0 msgr2=0x7eff0006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:03.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.613+0000 7eff06800640 1 -- 172.21.15.152:0/1449598380 shutdown_connections 2024-04-20T11:32:03.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.613+0000 7eff06800640 1 -- 172.21.15.152:0/1449598380 wait complete. 2024-04-20T11:32:03.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.615+0000 7eff06800640 1 Processor -- start 2024-04-20T11:32:03.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.615+0000 7eff06800640 1 -- start start 2024-04-20T11:32:03.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.616+0000 7eff06800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efee4006a10 0x7eff00155ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:03.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.616+0000 7eff06800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff001f5740 0x7eff001f4310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:03.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.616+0000 7eff06800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7eff00142150 0x7eff0015d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:03.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.616+0000 7eff06800640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7eff000cf8e0 con 0x7eff00142150 2024-04-20T11:32:03.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.616+0000 7eff06800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7eff000cfc50 con 0x7eff001f5740 2024-04-20T11:32:03.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.616+0000 7eff06800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7eff001417a0 con 0x7efee4006a10 2024-04-20T11:32:03.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.616+0000 7eff05e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7eff00142150 0x7eff0015d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:03.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.616+0000 7eff05e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7eff00142150 0x7eff0015d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35460/0 (socket says 172.21.15.152:35460) 2024-04-20T11:32:03.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.616+0000 7eff05e00640 1 -- 172.21.15.152:0/3616493368 learned_addr learned my addr 172.21.15.152:0/3616493368 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:32:03.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.616+0000 7eff04a00640 1 --2- 172.21.15.152:0/3616493368 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff001f5740 0x7eff001f4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:03.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.616+0000 7eff05400640 1 --2- 172.21.15.152:0/3616493368 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efee4006a10 0x7eff00155ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:03.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.616+0000 7eff05e00640 1 -- 172.21.15.152:0/3616493368 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efee4006a10 msgr2=0x7eff00155ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:03.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.616+0000 7eff05e00640 1 --2- 172.21.15.152:0/3616493368 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efee4006a10 0x7eff00155ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:03.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.616+0000 7eff05e00640 1 -- 172.21.15.152:0/3616493368 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff001f5740 msgr2=0x7eff001f4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:03.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.616+0000 7eff05e00640 1 --2- 172.21.15.152:0/3616493368 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff001f5740 0x7eff001f4310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:03.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.616+0000 7eff05e00640 1 -- 172.21.15.152:0/3616493368 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff00156820 con 0x7eff00142150 2024-04-20T11:32:03.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.616+0000 7eff05400640 1 --2- 172.21.15.152:0/3616493368 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efee4006a10 0x7eff00155ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:32:03.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.616+0000 7eff04a00640 1 --2- 172.21.15.152:0/3616493368 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff001f5740 0x7eff001f4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:32:03.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.617+0000 7eff05e00640 1 --2- 172.21.15.152:0/3616493368 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7eff00142150 0x7eff0015d080 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7eff000cee10 tx=0x7efef800c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:03.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.617+0000 7efefd600640 1 -- 172.21.15.152:0/3616493368 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef8007280 con 0x7eff00142150 2024-04-20T11:32:03.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.617+0000 7efefd600640 1 -- 172.21.15.152:0/3616493368 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efef8007420 con 0x7eff00142150 2024-04-20T11:32:03.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.617+0000 7efefd600640 1 -- 172.21.15.152:0/3616493368 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef8005800 con 0x7eff00142150 2024-04-20T11:32:03.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.617+0000 7eff06800640 1 -- 172.21.15.152:0/3616493368 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff00156b00 con 0x7eff00142150 2024-04-20T11:32:03.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.617+0000 7eff06800640 1 -- 172.21.15.152:0/3616493368 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff00156530 con 0x7eff00142150 2024-04-20T11:32:03.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.618+0000 7eff06800640 1 -- 172.21.15.152:0/3616493368 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff00157020 con 0x7eff00142150 2024-04-20T11:32:03.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.619+0000 7efefd600640 1 -- 172.21.15.152:0/3616493368 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 11) ==== 91346+0+0 (secure 0 0 0) 0x7efef8011070 con 0x7eff00142150 2024-04-20T11:32:03.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.620+0000 7efefd600640 1 --2- 172.21.15.152:0/3616493368 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7efee006cc50 0x7efee006f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:03.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.620+0000 7efefd600640 1 -- 172.21.15.152:0/3616493368 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5709+0+0 (secure 0 0 0) 0x7efef808abb0 con 0x7eff00142150 2024-04-20T11:32:03.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.621+0000 7eff05400640 1 --2- 172.21.15.152:0/3616493368 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7efee006cc50 0x7efee006f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:03.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.621+0000 7eff05400640 1 --2- 172.21.15.152:0/3616493368 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7efee006cc50 0x7efee006f110 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7efef0001630 tx=0x7efef0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:03.625 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.625+0000 7efefd600640 1 -- 172.21.15.152:0/3616493368 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7eff00157020 con 0x7eff00142150 2024-04-20T11:32:03.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.809+0000 7eff06800640 1 -- 172.21.15.152:0/3616493368 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7eff00075e60 con 0x7eff00142150 2024-04-20T11:32:03.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.812+0000 7efefd600640 1 -- 172.21.15.152:0/3616493368 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) ==== 74+0+16728 (secure 0 0 0) 0x7efef8058a90 con 0x7eff00142150 2024-04-20T11:32:03.814 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:32:03.814 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":71,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:31:51.301720+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:31:48.024528+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-20T11:31:48.725648+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:32:03.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.815+0000 7eff06800640 1 -- 172.21.15.152:0/3616493368 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7efee006cc50 msgr2=0x7efee006f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:03.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.815+0000 7eff06800640 1 --2- 172.21.15.152:0/3616493368 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7efee006cc50 0x7efee006f110 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7efef0001630 tx=0x7efef0009040 comp rx=0 tx=0).stop 2024-04-20T11:32:03.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.815+0000 7eff06800640 1 -- 172.21.15.152:0/3616493368 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7eff00142150 msgr2=0x7eff0015d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:03.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.815+0000 7eff06800640 1 --2- 172.21.15.152:0/3616493368 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7eff00142150 0x7eff0015d080 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7eff000cee10 tx=0x7efef800c820 comp rx=0 tx=0).stop 2024-04-20T11:32:03.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.815+0000 7eff06800640 1 -- 172.21.15.152:0/3616493368 shutdown_connections 2024-04-20T11:32:03.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.815+0000 7eff06800640 1 --2- 172.21.15.152:0/3616493368 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7efee006cc50 0x7efee006f110 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:03.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.815+0000 7eff06800640 1 --2- 172.21.15.152:0/3616493368 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7eff00142150 0x7eff0015d080 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:03.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.815+0000 7eff06800640 1 --2- 172.21.15.152:0/3616493368 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff001f5740 0x7eff001f4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:03.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.815+0000 7eff06800640 1 --2- 172.21.15.152:0/3616493368 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efee4006a10 0x7eff00155ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:03.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.815+0000 7eff06800640 1 -- 172.21.15.152:0/3616493368 >> 172.21.15.152:0/3616493368 conn(0x7eff00067ae0 msgr2=0x7eff00155280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:03.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.815+0000 7eff06800640 1 -- 172.21.15.152:0/3616493368 shutdown_connections 2024-04-20T11:32:03.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:03.815+0000 7eff06800640 1 -- 172.21.15.152:0/3616493368 wait complete. 2024-04-20T11:32:03.835 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T11:26:47.246571+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T11:31:48.024528+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '70', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T11:31:48.725648+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '71', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T11:32:03.835 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T11:32:04.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.030+0000 7f6ccd600640 1 Processor -- start 2024-04-20T11:32:04.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.030+0000 7f6ccd600640 1 -- start start 2024-04-20T11:32:04.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.031+0000 7f6ccd600640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f6cc80cf5b0 0x7f6cc80d4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:04.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.031+0000 7f6ccd600640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f6cc80ceeb0 0x7f6cc80cf990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:04.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.031+0000 7f6ccd600640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6cc80cfed0 con 0x7f6cc8162430 2024-04-20T11:32:04.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.032+0000 7f6ccd600640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f6cc80d5420 con 0x7f6cc80cf5b0 2024-04-20T11:32:04.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.032+0000 7f6ccd600640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f6cc80d2b70 con 0x7f6cc80ceeb0 2024-04-20T11:32:04.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.032+0000 7f6cccc00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f6cc8162430 0x7f6cc81628c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:34730/0 (socket says 172.21.15.152:34730) 2024-04-20T11:32:04.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.032+0000 7f6cccc00640 1 -- 172.21.15.152:0/3238223054 learned_addr learned my addr 172.21.15.152:0/3238223054 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:32:04.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.032+0000 7f6cc7e00640 1 --2- 172.21.15.152:0/3238223054 >> v2:172.21.15.152:3300/0 conn(0x7f6cc80cf5b0 0x7f6cc80d4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:04.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.032+0000 7f6cbf400640 1 --2- 172.21.15.152:0/3238223054 >> v2:172.21.15.175:3300/0 conn(0x7f6cc80ceeb0 0x7f6cc80cf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:04.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.032+0000 7f6cc7400640 1 -- 172.21.15.152:0/3238223054 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1139070678 0 0) 0x7f6cc80cfed0 con 0x7f6cc8162430 2024-04-20T11:32:04.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.032+0000 7f6cc7e00640 1 -- 172.21.15.152:0/3238223054 >> v2:172.21.15.175:3300/0 conn(0x7f6cc80ceeb0 msgr2=0x7f6cc80cf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:04.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.032+0000 7f6cc7e00640 1 --2- 172.21.15.152:0/3238223054 >> v2:172.21.15.175:3300/0 conn(0x7f6cc80ceeb0 0x7f6cc80cf990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:04.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.032+0000 7f6cc7e00640 1 -- 172.21.15.152:0/3238223054 >> v1:172.21.15.152:6789/0 conn(0x7f6cc8162430 legacy=0x7f6cc81628c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:04.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.032+0000 7f6cc7e00640 1 -- 172.21.15.152:0/3238223054 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cc80d2e50 con 0x7f6cc80cf5b0 2024-04-20T11:32:04.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.033+0000 7f6cc7e00640 1 --2- 172.21.15.152:0/3238223054 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6cc80cf5b0 0x7f6cc80d4ee0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f6cc808cb50 tx=0x7f6cb802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d0a883ce8c34a22 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:04.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.034+0000 7f6cc7400640 1 -- 172.21.15.152:0/3238223054 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb8034070 con 0x7f6cc80cf5b0 2024-04-20T11:32:04.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.034+0000 7f6cc7400640 1 -- 172.21.15.152:0/3238223054 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cb8030610 con 0x7f6cc80cf5b0 2024-04-20T11:32:04.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.034+0000 7f6cc7400640 1 -- 172.21.15.152:0/3238223054 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb80309e0 con 0x7f6cc80cf5b0 2024-04-20T11:32:04.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.034+0000 7f6ccd600640 1 -- 172.21.15.152:0/3238223054 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6cc80cf5b0 msgr2=0x7f6cc80d4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:04.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.034+0000 7f6ccd600640 1 --2- 172.21.15.152:0/3238223054 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6cc80cf5b0 0x7f6cc80d4ee0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f6cc808cb50 tx=0x7f6cb802b960 comp rx=0 tx=0).stop 2024-04-20T11:32:04.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.034+0000 7f6ccd600640 1 -- 172.21.15.152:0/3238223054 shutdown_connections 2024-04-20T11:32:04.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.034+0000 7f6ccd600640 1 --2- 172.21.15.152:0/3238223054 >> v2:172.21.15.175:3300/0 conn(0x7f6cc80ceeb0 0x7f6cc80cf990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:04.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.034+0000 7f6ccd600640 1 --2- 172.21.15.152:0/3238223054 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6cc80cf5b0 0x7f6cc80d4ee0 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:04.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.034+0000 7f6ccd600640 1 -- 172.21.15.152:0/3238223054 >> 172.21.15.152:0/3238223054 conn(0x7f6cc808bfd0 msgr2=0x7f6cc80d4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:04.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.034+0000 7f6ccd600640 1 -- 172.21.15.152:0/3238223054 shutdown_connections 2024-04-20T11:32:04.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.035+0000 7f6ccd600640 1 -- 172.21.15.152:0/3238223054 wait complete. 2024-04-20T11:32:04.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.036+0000 7f6ccd600640 1 Processor -- start 2024-04-20T11:32:04.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.036+0000 7f6ccd600640 1 -- start start 2024-04-20T11:32:04.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.037+0000 7f6ccd600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6cc80ceeb0 0x7f6cc80cb620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:04.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.037+0000 7f6ccd600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6cc80cf5b0 0x7f6cc8153ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:04.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.037+0000 7f6ccd600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6cc8162430 0x7f6cc8157880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:04.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.037+0000 7f6ccd600640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f6cc80d36f0 con 0x7f6cc80cf5b0 2024-04-20T11:32:04.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.037+0000 7f6ccd600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6cc80d3410 con 0x7f6cc8162430 2024-04-20T11:32:04.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.037+0000 7f6ccd600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6cc8161240 con 0x7f6cc80ceeb0 2024-04-20T11:32:04.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.037+0000 7f6cbf400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6cc80cf5b0 0x7f6cc8153ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:04.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.038+0000 7f6cbf400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6cc80cf5b0 0x7f6cc8153ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35484/0 (socket says 172.21.15.152:35484) 2024-04-20T11:32:04.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.038+0000 7f6cbf400640 1 -- 172.21.15.152:0/26224867 learned_addr learned my addr 172.21.15.152:0/26224867 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:32:04.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.038+0000 7f6cc7e00640 1 --2- 172.21.15.152:0/26224867 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6cc80ceeb0 0x7f6cc80cb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:04.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.038+0000 7f6cccc00640 1 --2- 172.21.15.152:0/26224867 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6cc8162430 0x7f6cc8157880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:04.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.038+0000 7f6cbf400640 1 -- 172.21.15.152:0/26224867 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6cc80ceeb0 msgr2=0x7f6cc80cb620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:04.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.038+0000 7f6cbf400640 1 --2- 172.21.15.152:0/26224867 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6cc80ceeb0 0x7f6cc80cb620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:04.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.038+0000 7f6cbf400640 1 -- 172.21.15.152:0/26224867 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6cc8162430 msgr2=0x7f6cc8157880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:04.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.038+0000 7f6cbf400640 1 --2- 172.21.15.152:0/26224867 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6cc8162430 0x7f6cc8157880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:04.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.038+0000 7f6cbf400640 1 -- 172.21.15.152:0/26224867 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cc80cbb60 con 0x7f6cc80cf5b0 2024-04-20T11:32:04.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.038+0000 7f6cbf400640 1 --2- 172.21.15.152:0/26224867 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6cc80cf5b0 0x7f6cc8153ae0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f6cb000d830 tx=0x7f6cb000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:04.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.038+0000 7f6cc4c00640 1 -- 172.21.15.152:0/26224867 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb000b6c0 con 0x7f6cc80cf5b0 2024-04-20T11:32:04.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.038+0000 7f6cc4c00640 1 -- 172.21.15.152:0/26224867 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cb0019070 con 0x7f6cc80cf5b0 2024-04-20T11:32:04.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.038+0000 7f6cc4c00640 1 -- 172.21.15.152:0/26224867 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb000baa0 con 0x7f6cc80cf5b0 2024-04-20T11:32:04.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.039+0000 7f6ccd600640 1 -- 172.21.15.152:0/26224867 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cc8154020 con 0x7f6cc80cf5b0 2024-04-20T11:32:04.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.039+0000 7f6ccd600640 1 -- 172.21.15.152:0/26224867 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cc815c740 con 0x7f6cc80cf5b0 2024-04-20T11:32:04.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.040+0000 7f6ccd600640 1 -- 172.21.15.152:0/26224867 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cc8075e60 con 0x7f6cc80cf5b0 2024-04-20T11:32:04.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.041+0000 7f6cc4c00640 1 -- 172.21.15.152:0/26224867 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 11) ==== 91346+0+0 (secure 0 0 0) 0x7f6cb001e460 con 0x7f6cc80cf5b0 2024-04-20T11:32:04.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.042+0000 7f6cc4c00640 1 --2- 172.21.15.152:0/26224867 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f6ca406cc50 0x7f6ca406f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:04.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.042+0000 7f6cc4c00640 1 -- 172.21.15.152:0/26224867 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5709+0+0 (secure 0 0 0) 0x7f6cb008c850 con 0x7f6cc80cf5b0 2024-04-20T11:32:04.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.042+0000 7f6cc7e00640 1 --2- 172.21.15.152:0/26224867 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f6ca406cc50 0x7f6ca406f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:04.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.043+0000 7f6cc7e00640 1 --2- 172.21.15.152:0/26224867 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f6ca406cc50 0x7f6ca406f110 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f6cb8036000 tx=0x7f6cb803b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:04.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.046+0000 7f6cc4c00640 1 -- 172.21.15.152:0/26224867 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f6cc8154020 con 0x7f6cc80cf5b0 2024-04-20T11:32:04.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.228+0000 7f6ccd600640 1 -- 172.21.15.152:0/26224867 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6cc80cf390 con 0x7f6cc80cf5b0 2024-04-20T11:32:04.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.229+0000 7f6cc4c00640 1 -- 172.21.15.152:0/26224867 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v71) ==== 93+0+10 (secure 0 0 0) 0x7f6cb005a7a0 con 0x7f6cc80cf5b0 2024-04-20T11:32:04.230 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 1 2024-04-20T11:32:04.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.232+0000 7f6ccd600640 1 -- 172.21.15.152:0/26224867 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f6ca406cc50 msgr2=0x7f6ca406f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:04.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.232+0000 7f6ccd600640 1 --2- 172.21.15.152:0/26224867 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f6ca406cc50 0x7f6ca406f110 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f6cb8036000 tx=0x7f6cb803b040 comp rx=0 tx=0).stop 2024-04-20T11:32:04.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.232+0000 7f6ccd600640 1 -- 172.21.15.152:0/26224867 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6cc80cf5b0 msgr2=0x7f6cc8153ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:04.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.232+0000 7f6ccd600640 1 --2- 172.21.15.152:0/26224867 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6cc80cf5b0 0x7f6cc8153ae0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f6cb000d830 tx=0x7f6cb000dcf0 comp rx=0 tx=0).stop 2024-04-20T11:32:04.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.232+0000 7f6ccd600640 1 -- 172.21.15.152:0/26224867 shutdown_connections 2024-04-20T11:32:04.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.232+0000 7f6ccd600640 1 --2- 172.21.15.152:0/26224867 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f6ca406cc50 0x7f6ca406f110 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:04.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.232+0000 7f6ccd600640 1 --2- 172.21.15.152:0/26224867 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6cc8162430 0x7f6cc8157880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:04.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.232+0000 7f6ccd600640 1 --2- 172.21.15.152:0/26224867 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6cc80cf5b0 0x7f6cc8153ae0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:04.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.232+0000 7f6ccd600640 1 --2- 172.21.15.152:0/26224867 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6cc80ceeb0 0x7f6cc80cb620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:04.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.232+0000 7f6ccd600640 1 -- 172.21.15.152:0/26224867 >> 172.21.15.152:0/26224867 conn(0x7f6cc808bfd0 msgr2=0x7f6cc81601b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:04.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.232+0000 7f6ccd600640 1 -- 172.21.15.152:0/26224867 shutdown_connections 2024-04-20T11:32:04.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.232+0000 7f6ccd600640 1 -- 172.21.15.152:0/26224867 wait complete. 2024-04-20T11:32:04.251 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T11:32:04.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.450+0000 7fac5f000640 1 Processor -- start 2024-04-20T11:32:04.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.450+0000 7fac5f000640 1 -- start start 2024-04-20T11:32:04.452 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.452+0000 7fac5f000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fac580b4330 0x7fac58165c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:04.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.452+0000 7fac5f000640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac5815b6a0 con 0x7fac5815c050 2024-04-20T11:32:04.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.452+0000 7fac5f000640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac580b4800 con 0x7fac580724c0 2024-04-20T11:32:04.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.452+0000 7fac5f000640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fac58155810 con 0x7fac580b4330 2024-04-20T11:32:04.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.452+0000 7fac57e00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fac5815c050 0x7fac58071db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:34744/0 (socket says 172.21.15.152:34744) 2024-04-20T11:32:04.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.452+0000 7fac57e00640 1 -- 172.21.15.152:0/3543772266 learned_addr learned my addr 172.21.15.152:0/3543772266 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:32:04.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.452+0000 7fac5d000640 1 --2- 172.21.15.152:0/3543772266 >> v2:172.21.15.152:3300/0 conn(0x7fac580b4330 0x7fac58165c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:04.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.453+0000 7fac57400640 1 -- 172.21.15.152:0/3543772266 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2268846733 0 0) 0x7fac48001110 con 0x7fac5815c050 2024-04-20T11:32:04.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.453+0000 7fac57400640 1 -- 172.21.15.152:0/3543772266 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fac3c003520 con 0x7fac5815c050 2024-04-20T11:32:04.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.453+0000 7fac5d000640 1 -- 172.21.15.152:0/3543772266 >> v1:172.21.15.175:6789/0 conn(0x7fac580724c0 legacy=0x7fac580b3c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:04.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.453+0000 7fac5d000640 1 -- 172.21.15.152:0/3543772266 >> v1:172.21.15.152:6789/0 conn(0x7fac5815c050 legacy=0x7fac58071db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:04.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.453+0000 7fac5d000640 1 -- 172.21.15.152:0/3543772266 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac580b4ae0 con 0x7fac580b4330 2024-04-20T11:32:04.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.454+0000 7fac5d000640 1 --2- 172.21.15.152:0/3543772266 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fac580b4330 0x7fac58165c80 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fac50009d90 tx=0x7fac5002bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35af019912687b97 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:04.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.454+0000 7fac57400640 1 -- 172.21.15.152:0/3543772266 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac50039070 con 0x7fac580b4330 2024-04-20T11:32:04.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.454+0000 7fac57400640 1 -- 172.21.15.152:0/3543772266 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac5002cc70 con 0x7fac580b4330 2024-04-20T11:32:04.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.454+0000 7fac57400640 1 -- 172.21.15.152:0/3543772266 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac50034640 con 0x7fac580b4330 2024-04-20T11:32:04.456 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.455+0000 7fac5f000640 1 -- 172.21.15.152:0/3543772266 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fac580b4330 msgr2=0x7fac58165c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:04.456 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.455+0000 7fac5f000640 1 --2- 172.21.15.152:0/3543772266 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fac580b4330 0x7fac58165c80 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fac50009d90 tx=0x7fac5002bec0 comp rx=0 tx=0).stop 2024-04-20T11:32:04.456 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.455+0000 7fac5f000640 1 -- 172.21.15.152:0/3543772266 shutdown_connections 2024-04-20T11:32:04.456 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.455+0000 7fac5f000640 1 --2- 172.21.15.152:0/3543772266 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fac580b4330 0x7fac58165c80 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:04.456 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.455+0000 7fac5f000640 1 -- 172.21.15.152:0/3543772266 >> 172.21.15.152:0/3543772266 conn(0x7fac5806b980 msgr2=0x7fac5806bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:04.456 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.455+0000 7fac5f000640 1 -- 172.21.15.152:0/3543772266 shutdown_connections 2024-04-20T11:32:04.456 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.455+0000 7fac5f000640 1 -- 172.21.15.152:0/3543772266 wait complete. 2024-04-20T11:32:04.457 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.457+0000 7fac5f000640 1 Processor -- start 2024-04-20T11:32:04.457 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.457+0000 7fac5f000640 1 -- start start 2024-04-20T11:32:04.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.458+0000 7fac5f000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fac580724c0 0x7fac5814fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:04.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.458+0000 7fac5f000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fac580b4330 0x7fac58150c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:04.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.458+0000 7fac5f000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fac5815c050 0x7fac581549f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:04.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.458+0000 7fac5f000640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fac5806dea0 con 0x7fac580724c0 2024-04-20T11:32:04.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.458+0000 7fac5f000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fac5806dc50 con 0x7fac5815c050 2024-04-20T11:32:04.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.458+0000 7fac5f000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fac5806f3d0 con 0x7fac580b4330 2024-04-20T11:32:04.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.458+0000 7fac57e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fac580724c0 0x7fac5814fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:04.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.458+0000 7fac57e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fac580724c0 0x7fac5814fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35500/0 (socket says 172.21.15.152:35500) 2024-04-20T11:32:04.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.458+0000 7fac57e00640 1 -- 172.21.15.152:0/1223930451 learned_addr learned my addr 172.21.15.152:0/1223930451 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:32:04.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.458+0000 7fac5d000640 1 --2- 172.21.15.152:0/1223930451 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fac5815c050 0x7fac581549f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:04.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.458+0000 7fac57e00640 1 -- 172.21.15.152:0/1223930451 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fac580b4330 msgr2=0x7fac58150c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:04.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.458+0000 7fac57e00640 1 --2- 172.21.15.152:0/1223930451 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fac580b4330 0x7fac58150c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:04.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.458+0000 7fac57e00640 1 -- 172.21.15.152:0/1223930451 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fac5815c050 msgr2=0x7fac581549f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:04.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.458+0000 7fac57e00640 1 --2- 172.21.15.152:0/1223930451 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fac5815c050 0x7fac581549f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:04.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.458+0000 7fac57e00640 1 -- 172.21.15.152:0/1223930451 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac58151550 con 0x7fac580724c0 2024-04-20T11:32:04.461 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.459+0000 7fac57e00640 1 --2- 172.21.15.152:0/1223930451 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fac580724c0 0x7fac5814fa10 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fac4800c430 tx=0x7fac4800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:04.461 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.459+0000 7fac54c00640 1 -- 172.21.15.152:0/1223930451 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac48017070 con 0x7fac580724c0 2024-04-20T11:32:04.461 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.459+0000 7fac54c00640 1 -- 172.21.15.152:0/1223930451 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac48005ce0 con 0x7fac580724c0 2024-04-20T11:32:04.461 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.459+0000 7fac54c00640 1 -- 172.21.15.152:0/1223930451 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac480072d0 con 0x7fac580724c0 2024-04-20T11:32:04.461 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.459+0000 7fac5f000640 1 -- 172.21.15.152:0/1223930451 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac58157470 con 0x7fac580724c0 2024-04-20T11:32:04.461 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.459+0000 7fac5f000640 1 -- 172.21.15.152:0/1223930451 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac58157a30 con 0x7fac580724c0 2024-04-20T11:32:04.461 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.460+0000 7fac5f000640 1 -- 172.21.15.152:0/1223930451 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac580689f0 con 0x7fac580724c0 2024-04-20T11:32:04.462 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.461+0000 7fac54c00640 1 -- 172.21.15.152:0/1223930451 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 11) ==== 91346+0+0 (secure 0 0 0) 0x7fac4801a070 con 0x7fac580724c0 2024-04-20T11:32:04.463 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.462+0000 7fac54c00640 1 --2- 172.21.15.152:0/1223930451 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fac3406cc50 0x7fac3406f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:04.463 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.463+0000 7fac54c00640 1 -- 172.21.15.152:0/1223930451 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5709+0+0 (secure 0 0 0) 0x7fac4808b4e0 con 0x7fac580724c0 2024-04-20T11:32:04.463 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.463+0000 7fac4f400640 1 --2- 172.21.15.152:0/1223930451 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fac3406cc50 0x7fac3406f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:04.464 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.464+0000 7fac4f400640 1 --2- 172.21.15.152:0/1223930451 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fac3406cc50 0x7fac3406f110 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fac40000f60 tx=0x7fac400079d0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:04.467 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.467+0000 7fac54c00640 1 -- 172.21.15.152:0/1223930451 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fac5815b6a0 con 0x7fac580724c0 2024-04-20T11:32:04.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.666+0000 7fac5f000640 1 -- 172.21.15.152:0/1223930451 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fac58072a90 con 0x7fac580724c0 2024-04-20T11:32:04.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.667+0000 7fac54c00640 1 -- 172.21.15.152:0/1223930451 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v71) ==== 104+0+11 (secure 0 0 0) 0x7fac48058d70 con 0x7fac580724c0 2024-04-20T11:32:04.668 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 32 2024-04-20T11:32:04.671 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.670+0000 7fac5f000640 1 -- 172.21.15.152:0/1223930451 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fac3406cc50 msgr2=0x7fac3406f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:04.671 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.670+0000 7fac5f000640 1 --2- 172.21.15.152:0/1223930451 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fac3406cc50 0x7fac3406f110 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fac40000f60 tx=0x7fac400079d0 comp rx=0 tx=0).stop 2024-04-20T11:32:04.671 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.670+0000 7fac5f000640 1 -- 172.21.15.152:0/1223930451 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fac580724c0 msgr2=0x7fac5814fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:04.671 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.670+0000 7fac5f000640 1 --2- 172.21.15.152:0/1223930451 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fac580724c0 0x7fac5814fa10 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fac4800c430 tx=0x7fac4800c8f0 comp rx=0 tx=0).stop 2024-04-20T11:32:04.671 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.670+0000 7fac5f000640 1 -- 172.21.15.152:0/1223930451 shutdown_connections 2024-04-20T11:32:04.671 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.670+0000 7fac5f000640 1 --2- 172.21.15.152:0/1223930451 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fac3406cc50 0x7fac3406f110 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:04.671 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.670+0000 7fac5f000640 1 --2- 172.21.15.152:0/1223930451 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fac5815c050 0x7fac581549f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:04.672 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.670+0000 7fac5f000640 1 --2- 172.21.15.152:0/1223930451 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fac580b4330 0x7fac58150c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:04.672 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.670+0000 7fac5f000640 1 --2- 172.21.15.152:0/1223930451 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fac580724c0 0x7fac5814fa10 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:04.672 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.670+0000 7fac5f000640 1 -- 172.21.15.152:0/1223930451 >> 172.21.15.152:0/1223930451 conn(0x7fac5806b980 msgr2=0x7fac5806f760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:04.672 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.670+0000 7fac5f000640 1 -- 172.21.15.152:0/1223930451 shutdown_connections 2024-04-20T11:32:04.672 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.670+0000 7fac5f000640 1 -- 172.21.15.152:0/1223930451 wait complete. 2024-04-20T11:32:04.690 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T11:32:04.891 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.890+0000 7f9d2f200640 1 Processor -- start 2024-04-20T11:32:04.891 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.890+0000 7f9d2f200640 1 -- start start 2024-04-20T11:32:04.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.892+0000 7f9d2f200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d280b5e30 0x7f9d280b6210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:04.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.892+0000 7f9d2f200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f9d2815ed60 0x7f9d280b7610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:04.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.892+0000 7f9d2f200640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d280cffc0 con 0x7f9d280b5670 2024-04-20T11:32:04.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.892+0000 7f9d2f200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f9d280d4f30 con 0x7f9d2815ed60 2024-04-20T11:32:04.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.892+0000 7f9d2f200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f9d280d02a0 con 0x7f9d280b5e30 2024-04-20T11:32:04.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.892+0000 7f9d2d400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f9d280b5670 0x7f9d2815e650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:34748/0 (socket says 172.21.15.152:34748) 2024-04-20T11:32:04.894 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.892+0000 7f9d2d400640 1 -- 172.21.15.152:0/2683944194 learned_addr learned my addr 172.21.15.152:0/2683944194 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:32:04.894 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.892+0000 7f9d2e800640 1 --2- 172.21.15.152:0/2683944194 >> v2:172.21.15.152:3300/0 conn(0x7f9d2815ed60 0x7f9d280b7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:04.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.893+0000 7f9d2de00640 1 --2- 172.21.15.152:0/2683944194 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d280b5e30 0x7f9d280b6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:04.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.893+0000 7f9d2ca00640 1 -- 172.21.15.152:0/2683944194 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4144047368 0 0) 0x7f9d280cffc0 con 0x7f9d280b5670 2024-04-20T11:32:04.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.893+0000 7f9d2ca00640 1 -- 172.21.15.152:0/2683944194 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9d10003520 con 0x7f9d280b5670 2024-04-20T11:32:04.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.893+0000 7f9d2e800640 1 -- 172.21.15.152:0/2683944194 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d280b5e30 msgr2=0x7f9d280b6210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:04.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.893+0000 7f9d2e800640 1 --2- 172.21.15.152:0/2683944194 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d280b5e30 0x7f9d280b6210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:04.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.893+0000 7f9d2e800640 1 -- 172.21.15.152:0/2683944194 >> v1:172.21.15.152:6789/0 conn(0x7f9d280b5670 legacy=0x7f9d2815e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:04.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.893+0000 7f9d2e800640 1 -- 172.21.15.152:0/2683944194 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d280d0580 con 0x7f9d2815ed60 2024-04-20T11:32:04.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.895+0000 7f9d2e800640 1 --2- 172.21.15.152:0/2683944194 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d2815ed60 0x7f9d280b7610 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f9d24009d90 tx=0x7f9d2402bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9937e1b1e2d51ff server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:04.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.895+0000 7f9d2ca00640 1 -- 172.21.15.152:0/2683944194 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d24039070 con 0x7f9d2815ed60 2024-04-20T11:32:04.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.895+0000 7f9d2ca00640 1 -- 172.21.15.152:0/2683944194 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d2402cc50 con 0x7f9d2815ed60 2024-04-20T11:32:04.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.895+0000 7f9d2ca00640 1 -- 172.21.15.152:0/2683944194 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d24034440 con 0x7f9d2815ed60 2024-04-20T11:32:04.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.896+0000 7f9d2f200640 1 -- 172.21.15.152:0/2683944194 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d2815ed60 msgr2=0x7f9d280b7610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:04.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.896+0000 7f9d2f200640 1 --2- 172.21.15.152:0/2683944194 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d2815ed60 0x7f9d280b7610 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f9d24009d90 tx=0x7f9d2402bea0 comp rx=0 tx=0).stop 2024-04-20T11:32:04.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.896+0000 7f9d2f200640 1 -- 172.21.15.152:0/2683944194 shutdown_connections 2024-04-20T11:32:04.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.896+0000 7f9d2f200640 1 --2- 172.21.15.152:0/2683944194 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d2815ed60 0x7f9d280b7610 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:04.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.896+0000 7f9d2f200640 1 --2- 172.21.15.152:0/2683944194 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d280b5e30 0x7f9d280b6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:04.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.896+0000 7f9d2f200640 1 -- 172.21.15.152:0/2683944194 >> 172.21.15.152:0/2683944194 conn(0x7f9d2808bfd0 msgr2=0x7f9d280d4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:04.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.896+0000 7f9d2f200640 1 -- 172.21.15.152:0/2683944194 shutdown_connections 2024-04-20T11:32:04.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.896+0000 7f9d2f200640 1 -- 172.21.15.152:0/2683944194 wait complete. 2024-04-20T11:32:04.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.898+0000 7f9d2f200640 1 Processor -- start 2024-04-20T11:32:04.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.898+0000 7f9d2f200640 1 -- start start 2024-04-20T11:32:04.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.899+0000 7f9d2f200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d280b5670 0x7f9d281f4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:04.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.899+0000 7f9d2f200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d280b5e30 0x7f9d2815ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:04.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.899+0000 7f9d2f200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d2815ed60 0x7f9d2815b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:04.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.899+0000 7f9d2f200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f9d280b6750 con 0x7f9d280b5670 2024-04-20T11:32:04.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.899+0000 7f9d2f200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9d280d0b40 con 0x7f9d2815ed60 2024-04-20T11:32:04.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.899+0000 7f9d2f200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f9d280b68d0 con 0x7f9d280b5e30 2024-04-20T11:32:04.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.899+0000 7f9d2de00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d280b5670 0x7f9d281f4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:04.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.899+0000 7f9d2de00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d280b5670 0x7f9d281f4bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35512/0 (socket says 172.21.15.152:35512) 2024-04-20T11:32:04.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.899+0000 7f9d2de00640 1 -- 172.21.15.152:0/1943055112 learned_addr learned my addr 172.21.15.152:0/1943055112 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:32:04.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.899+0000 7f9d2e800640 1 --2- 172.21.15.152:0/1943055112 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d2815ed60 0x7f9d2815b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:04.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.900+0000 7f9d2d400640 1 --2- 172.21.15.152:0/1943055112 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d280b5e30 0x7f9d2815ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:04.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.900+0000 7f9d2de00640 1 -- 172.21.15.152:0/1943055112 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d280b5e30 msgr2=0x7f9d2815ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:04.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.900+0000 7f9d2de00640 1 --2- 172.21.15.152:0/1943055112 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d280b5e30 0x7f9d2815ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:04.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.900+0000 7f9d2de00640 1 -- 172.21.15.152:0/1943055112 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d2815ed60 msgr2=0x7f9d2815b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:04.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.900+0000 7f9d2de00640 1 --2- 172.21.15.152:0/1943055112 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d2815ed60 0x7f9d2815b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:04.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.900+0000 7f9d2de00640 1 -- 172.21.15.152:0/1943055112 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d281f5110 con 0x7f9d280b5670 2024-04-20T11:32:04.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.900+0000 7f9d2e800640 1 --2- 172.21.15.152:0/1943055112 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d2815ed60 0x7f9d2815b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:32:04.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.900+0000 7f9d2de00640 1 --2- 172.21.15.152:0/1943055112 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d280b5670 0x7f9d281f4bd0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f9d1c00d5f0 tx=0x7f9d1c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:04.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.900+0000 7f9d22000640 1 -- 172.21.15.152:0/1943055112 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d1c0125f0 con 0x7f9d280b5670 2024-04-20T11:32:04.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.900+0000 7f9d22000640 1 -- 172.21.15.152:0/1943055112 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d1c019070 con 0x7f9d280b5670 2024-04-20T11:32:04.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.900+0000 7f9d22000640 1 -- 172.21.15.152:0/1943055112 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d1c0129d0 con 0x7f9d280b5670 2024-04-20T11:32:04.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.900+0000 7f9d2f200640 1 -- 172.21.15.152:0/1943055112 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d280c8d40 con 0x7f9d280b5670 2024-04-20T11:32:04.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.900+0000 7f9d2f200640 1 -- 172.21.15.152:0/1943055112 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d281f5790 con 0x7f9d280b5670 2024-04-20T11:32:04.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.902+0000 7f9d2f200640 1 -- 172.21.15.152:0/1943055112 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d28075e60 con 0x7f9d280b5670 2024-04-20T11:32:04.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.902+0000 7f9d22000640 1 -- 172.21.15.152:0/1943055112 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 11) ==== 91346+0+0 (secure 0 0 0) 0x7f9d1c012b70 con 0x7f9d280b5670 2024-04-20T11:32:04.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.903+0000 7f9d22000640 1 --2- 172.21.15.152:0/1943055112 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f9d0c06cc50 0x7f9d0c06f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:04.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.903+0000 7f9d22000640 1 -- 172.21.15.152:0/1943055112 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5709+0+0 (secure 0 0 0) 0x7f9d1c08c530 con 0x7f9d280b5670 2024-04-20T11:32:04.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.904+0000 7f9d2d400640 1 --2- 172.21.15.152:0/1943055112 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f9d0c06cc50 0x7f9d0c06f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:04.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.904+0000 7f9d2d400640 1 --2- 172.21.15.152:0/1943055112 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f9d0c06cc50 0x7f9d0c06f110 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f9d280b5200 tx=0x7f9d18009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:04.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:04.908+0000 7f9d22000640 1 -- 172.21.15.152:0/1943055112 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f9d281f5110 con 0x7f9d280b5670 2024-04-20T11:32:05.108 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.107+0000 7f9d2f200640 1 -- 172.21.15.152:0/1943055112 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f9d280cf840 con 0x7f9d280b5670 2024-04-20T11:32:05.108 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.108+0000 7f9d22000640 1 -- 172.21.15.152:0/1943055112 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v71) ==== 100+0+11 (secure 0 0 0) 0x7f9d1c05a510 con 0x7f9d280b5670 2024-04-20T11:32:05.109 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 64 2024-04-20T11:32:05.112 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.111+0000 7f9d2f200640 1 -- 172.21.15.152:0/1943055112 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f9d0c06cc50 msgr2=0x7f9d0c06f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:05.112 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.111+0000 7f9d2f200640 1 --2- 172.21.15.152:0/1943055112 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f9d0c06cc50 0x7f9d0c06f110 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f9d280b5200 tx=0x7f9d18009040 comp rx=0 tx=0).stop 2024-04-20T11:32:05.112 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.111+0000 7f9d2f200640 1 -- 172.21.15.152:0/1943055112 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d280b5670 msgr2=0x7f9d281f4bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:05.112 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.111+0000 7f9d2f200640 1 --2- 172.21.15.152:0/1943055112 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d280b5670 0x7f9d281f4bd0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f9d1c00d5f0 tx=0x7f9d1c00dab0 comp rx=0 tx=0).stop 2024-04-20T11:32:05.112 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.111+0000 7f9d2f200640 1 -- 172.21.15.152:0/1943055112 shutdown_connections 2024-04-20T11:32:05.112 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.111+0000 7f9d2f200640 1 --2- 172.21.15.152:0/1943055112 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f9d0c06cc50 0x7f9d0c06f110 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:05.112 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.111+0000 7f9d2f200640 1 --2- 172.21.15.152:0/1943055112 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d2815ed60 0x7f9d2815b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:05.112 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.111+0000 7f9d2f200640 1 --2- 172.21.15.152:0/1943055112 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d280b5e30 0x7f9d2815ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:05.112 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.111+0000 7f9d2f200640 1 --2- 172.21.15.152:0/1943055112 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d280b5670 0x7f9d281f4bd0 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:05.112 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.111+0000 7f9d2f200640 1 -- 172.21.15.152:0/1943055112 >> 172.21.15.152:0/1943055112 conn(0x7f9d2808bfd0 msgr2=0x7f9d2815e600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:05.112 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.111+0000 7f9d2f200640 1 -- 172.21.15.152:0/1943055112 shutdown_connections 2024-04-20T11:32:05.112 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.111+0000 7f9d2f200640 1 -- 172.21.15.152:0/1943055112 wait complete. 2024-04-20T11:32:05.134 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-20T11:32:05.335 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.334+0000 7f4e03200640 1 Processor -- start 2024-04-20T11:32:05.336 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.335+0000 7f4e03200640 1 -- start start 2024-04-20T11:32:05.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.336+0000 7f4e03200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4dfc15c7f0 0x7f4dfc072b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:05.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.336+0000 7f4e03200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f4dfc0b4870 0x7f4dfc0ad2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:05.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.336+0000 7f4e03200640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4dfc073fc0 con 0x7f4dfc073b60 2024-04-20T11:32:05.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.336+0000 7f4e03200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f4dfc0ce140 con 0x7f4dfc0b4870 2024-04-20T11:32:05.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.336+0000 7f4e03200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4dfc0ce390 con 0x7f4dfc15c7f0 2024-04-20T11:32:05.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.336+0000 7f4e01400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f4dfc073b60 0x7f4dfc0b4160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:34760/0 (socket says 172.21.15.152:34760) 2024-04-20T11:32:05.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.336+0000 7f4e02800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f4dfc0b4870 0x7f4dfc0ad2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:05.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.336+0000 7f4e01400640 1 -- 172.21.15.152:0/4196526590 learned_addr learned my addr 172.21.15.152:0/4196526590 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:32:05.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.337+0000 7f4e01e00640 1 --2- 172.21.15.152:0/4196526590 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4dfc15c7f0 0x7f4dfc072b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:05.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.337+0000 7f4e02800640 1 -- 172.21.15.152:0/4196526590 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4dfc15c7f0 msgr2=0x7f4dfc072b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:05.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.337+0000 7f4e02800640 1 --2- 172.21.15.152:0/4196526590 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4dfc15c7f0 0x7f4dfc072b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:05.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.337+0000 7f4e02800640 1 -- 172.21.15.152:0/4196526590 >> v1:172.21.15.152:6789/0 conn(0x7f4dfc073b60 legacy=0x7f4dfc0b4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:05.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.337+0000 7f4e02800640 1 -- 172.21.15.152:0/4196526590 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dfc0ce670 con 0x7f4dfc0b4870 2024-04-20T11:32:05.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.337+0000 7f4e00a00640 1 -- 172.21.15.152:0/4196526590 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3884668841 0 0) 0x7f4dfc073fc0 con 0x7f4dfc073b60 2024-04-20T11:32:05.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.338+0000 7f4e02800640 1 --2- 172.21.15.152:0/4196526590 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4dfc0b4870 0x7f4dfc0ad2b0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f4df8009d90 tx=0x7f4df802be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e05272e908a62a96 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:05.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.338+0000 7f4e00a00640 1 -- 172.21.15.152:0/4196526590 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4df8039070 con 0x7f4dfc0b4870 2024-04-20T11:32:05.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.338+0000 7f4e00a00640 1 -- 172.21.15.152:0/4196526590 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4df802cbd0 con 0x7f4dfc0b4870 2024-04-20T11:32:05.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.338+0000 7f4e00a00640 1 -- 172.21.15.152:0/4196526590 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4df8034440 con 0x7f4dfc0b4870 2024-04-20T11:32:05.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.339+0000 7f4e03200640 1 -- 172.21.15.152:0/4196526590 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4dfc0b4870 msgr2=0x7f4dfc0ad2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:05.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.339+0000 7f4e03200640 1 --2- 172.21.15.152:0/4196526590 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4dfc0b4870 0x7f4dfc0ad2b0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f4df8009d90 tx=0x7f4df802be20 comp rx=0 tx=0).stop 2024-04-20T11:32:05.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.339+0000 7f4e03200640 1 -- 172.21.15.152:0/4196526590 shutdown_connections 2024-04-20T11:32:05.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.339+0000 7f4e03200640 1 --2- 172.21.15.152:0/4196526590 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4dfc0b4870 0x7f4dfc0ad2b0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:05.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.339+0000 7f4e03200640 1 --2- 172.21.15.152:0/4196526590 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4dfc15c7f0 0x7f4dfc072b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:05.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.339+0000 7f4e03200640 1 -- 172.21.15.152:0/4196526590 >> 172.21.15.152:0/4196526590 conn(0x7f4dfc0d0260 msgr2=0x7f4dfc0d0670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:05.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.339+0000 7f4e03200640 1 -- 172.21.15.152:0/4196526590 shutdown_connections 2024-04-20T11:32:05.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.339+0000 7f4e03200640 1 -- 172.21.15.152:0/4196526590 wait complete. 2024-04-20T11:32:05.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.341+0000 7f4e03200640 1 Processor -- start 2024-04-20T11:32:05.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.341+0000 7f4e03200640 1 -- start start 2024-04-20T11:32:05.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.342+0000 7f4e03200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4dfc073b60 0x7f4dfc15a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:05.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.342+0000 7f4e03200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4dfc0b4870 0x7f4dfc157870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:05.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.342+0000 7f4e03200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4dfc1430e0 0x7f4dfc072da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:05.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.342+0000 7f4e03200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f4dfc0cef10 con 0x7f4dfc1430e0 2024-04-20T11:32:05.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.342+0000 7f4e03200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4dfc0cec30 con 0x7f4dfc0b4870 2024-04-20T11:32:05.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.342+0000 7f4e03200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4dfc0cf430 con 0x7f4dfc073b60 2024-04-20T11:32:05.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.343+0000 7f4e02800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4dfc1430e0 0x7f4dfc072da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:05.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.343+0000 7f4e02800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4dfc1430e0 0x7f4dfc072da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35524/0 (socket says 172.21.15.152:35524) 2024-04-20T11:32:05.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.343+0000 7f4e02800640 1 -- 172.21.15.152:0/2393483958 learned_addr learned my addr 172.21.15.152:0/2393483958 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:32:05.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.343+0000 7f4e01400640 1 --2- 172.21.15.152:0/2393483958 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4dfc0b4870 0x7f4dfc157870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:05.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.343+0000 7f4e01e00640 1 --2- 172.21.15.152:0/2393483958 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4dfc073b60 0x7f4dfc15a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:05.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.343+0000 7f4e02800640 1 -- 172.21.15.152:0/2393483958 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4dfc073b60 msgr2=0x7f4dfc15a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:05.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.343+0000 7f4e02800640 1 --2- 172.21.15.152:0/2393483958 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4dfc073b60 0x7f4dfc15a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:05.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.343+0000 7f4e02800640 1 -- 172.21.15.152:0/2393483958 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4dfc0b4870 msgr2=0x7f4dfc157870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:05.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.343+0000 7f4e02800640 1 --2- 172.21.15.152:0/2393483958 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4dfc0b4870 0x7f4dfc157870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:05.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.343+0000 7f4e02800640 1 -- 172.21.15.152:0/2393483958 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dfc155550 con 0x7f4dfc1430e0 2024-04-20T11:32:05.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.343+0000 7f4e01400640 1 --2- 172.21.15.152:0/2393483958 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4dfc0b4870 0x7f4dfc157870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:32:05.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.344+0000 7f4e02800640 1 --2- 172.21.15.152:0/2393483958 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4dfc1430e0 0x7f4dfc072da0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f4df8009eb0 tx=0x7f4df8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:05.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.344+0000 7f4df6000640 1 -- 172.21.15.152:0/2393483958 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4df8039050 con 0x7f4dfc1430e0 2024-04-20T11:32:05.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.344+0000 7f4df6000640 1 -- 172.21.15.152:0/2393483958 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4df8045070 con 0x7f4dfc1430e0 2024-04-20T11:32:05.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.344+0000 7f4e03200640 1 -- 172.21.15.152:0/2393483958 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4dfc155830 con 0x7f4dfc1430e0 2024-04-20T11:32:05.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.344+0000 7f4df6000640 1 -- 172.21.15.152:0/2393483958 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4df803e830 con 0x7f4dfc1430e0 2024-04-20T11:32:05.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.344+0000 7f4e03200640 1 -- 172.21.15.152:0/2393483958 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4dfc155df0 con 0x7f4dfc1430e0 2024-04-20T11:32:05.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.345+0000 7f4e03200640 1 -- 172.21.15.152:0/2393483958 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4dfc075e60 con 0x7f4dfc1430e0 2024-04-20T11:32:05.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.347+0000 7f4df6000640 1 -- 172.21.15.152:0/2393483958 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 11) ==== 91346+0+0 (secure 0 0 0) 0x7f4df803e9d0 con 0x7f4dfc1430e0 2024-04-20T11:32:05.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.348+0000 7f4df6000640 1 --2- 172.21.15.152:0/2393483958 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f4ddc06cc50 0x7f4ddc06f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:05.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.348+0000 7f4df6000640 1 -- 172.21.15.152:0/2393483958 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5709+0+0 (secure 0 0 0) 0x7f4df80aed60 con 0x7f4dfc1430e0 2024-04-20T11:32:05.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.348+0000 7f4e01e00640 1 --2- 172.21.15.152:0/2393483958 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f4ddc06cc50 0x7f4ddc06f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:05.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.349+0000 7f4e01e00640 1 --2- 172.21.15.152:0/2393483958 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f4ddc06cc50 0x7f4ddc06f110 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f4df0006fd0 tx=0x7f4df0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:05.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.353+0000 7f4df6000640 1 -- 172.21.15.152:0/2393483958 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f4df8032e20 con 0x7f4dfc1430e0 2024-04-20T11:32:05.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.565+0000 7f4e03200640 1 -- 172.21.15.152:0/2393483958 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f4dfc06cc40 con 0x7f4dfc1430e0 2024-04-20T11:32:05.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.566+0000 7f4df6000640 1 -- 172.21.15.152:0/2393483958 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v71) ==== 80+0+0 (secure 0 0 0) 0x7f4df807cdd0 con 0x7f4dfc1430e0 2024-04-20T11:32:05.566 INFO:teuthology.orchestra.run.smithi152.stderr:listed 0 entries 2024-04-20T11:32:05.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.568+0000 7f4e03200640 1 -- 172.21.15.152:0/2393483958 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f4ddc06cc50 msgr2=0x7f4ddc06f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:05.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.568+0000 7f4e03200640 1 --2- 172.21.15.152:0/2393483958 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f4ddc06cc50 0x7f4ddc06f110 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f4df0006fd0 tx=0x7f4df0008040 comp rx=0 tx=0).stop 2024-04-20T11:32:05.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.568+0000 7f4e03200640 1 -- 172.21.15.152:0/2393483958 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4dfc1430e0 msgr2=0x7f4dfc072da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:05.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.568+0000 7f4e03200640 1 --2- 172.21.15.152:0/2393483958 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4dfc1430e0 0x7f4dfc072da0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f4df8009eb0 tx=0x7f4df8004a40 comp rx=0 tx=0).stop 2024-04-20T11:32:05.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.568+0000 7f4e03200640 1 -- 172.21.15.152:0/2393483958 shutdown_connections 2024-04-20T11:32:05.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.568+0000 7f4e03200640 1 --2- 172.21.15.152:0/2393483958 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f4ddc06cc50 0x7f4ddc06f110 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:05.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.568+0000 7f4e03200640 1 --2- 172.21.15.152:0/2393483958 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4dfc1430e0 0x7f4dfc072da0 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:05.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.568+0000 7f4e03200640 1 --2- 172.21.15.152:0/2393483958 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4dfc0b4870 0x7f4dfc157870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:05.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.568+0000 7f4e03200640 1 --2- 172.21.15.152:0/2393483958 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4dfc073b60 0x7f4dfc15a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:05.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.568+0000 7f4e03200640 1 -- 172.21.15.152:0/2393483958 >> 172.21.15.152:0/2393483958 conn(0x7f4dfc0d0260 msgr2=0x7f4dfc15bad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:05.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.569+0000 7f4e03200640 1 -- 172.21.15.152:0/2393483958 shutdown_connections 2024-04-20T11:32:05.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.569+0000 7f4e03200640 1 -- 172.21.15.152:0/2393483958 wait complete. 2024-04-20T11:32:05.587 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-20T11:32:05.587 DEBUG:teuthology.orchestra.run.smithi152:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:32:05.632 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:32:05.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.871+0000 7f886a200640 1 Processor -- start 2024-04-20T11:32:05.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.871+0000 7f886a200640 1 -- start start 2024-04-20T11:32:05.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.873+0000 7f886a200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f886415d1e0 0x7f8864160b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:05.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.873+0000 7f886a200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f886415d5c0 0x7f88640cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:05.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.873+0000 7f886a200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88640ce8c0 con 0x7f8864071cd0 2024-04-20T11:32:05.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.873+0000 7f886a200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f88641600b0 con 0x7f886415d1e0 2024-04-20T11:32:05.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.873+0000 7f886a200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f8864160390 con 0x7f886415d5c0 2024-04-20T11:32:05.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.873+0000 7f8868e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f886415d1e0 0x7f8864160b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:05.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.873+0000 7f8868e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f886415d1e0 0x7f8864160b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36670/0 (socket says 172.21.15.152:36670) 2024-04-20T11:32:05.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.873+0000 7f8868e00640 1 -- 172.21.15.152:0/1850845155 learned_addr learned my addr 172.21.15.152:0/1850845155 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:32:05.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.873+0000 7f8868e00640 1 -- 172.21.15.152:0/1850845155 >> v2:172.21.15.175:3300/0 conn(0x7f886415d5c0 msgr2=0x7f88640cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:05.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.873+0000 7f8868e00640 1 --2- 172.21.15.152:0/1850845155 >> v2:172.21.15.175:3300/0 conn(0x7f886415d5c0 0x7f88640cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:05.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.873+0000 7f8868e00640 1 -- 172.21.15.152:0/1850845155 >> v1:172.21.15.175:6789/0 conn(0x7f8864071cd0 legacy=0x7f8864072120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:05.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.873+0000 7f8868e00640 1 -- 172.21.15.152:0/1850845155 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8864160670 con 0x7f886415d1e0 2024-04-20T11:32:05.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.874+0000 7f8868e00640 1 --2- 172.21.15.152:0/1850845155 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f886415d1e0 0x7f8864160b90 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f8858009870 tx=0x7f885802b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0f57f87473bf660 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:05.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.875+0000 7f8863400640 1 -- 172.21.15.152:0/1850845155 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8858035020 con 0x7f886415d1e0 2024-04-20T11:32:05.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.875+0000 7f8863400640 1 -- 172.21.15.152:0/1850845155 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f885802d040 con 0x7f886415d1e0 2024-04-20T11:32:05.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.875+0000 7f8863400640 1 -- 172.21.15.152:0/1850845155 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8858031a80 con 0x7f886415d1e0 2024-04-20T11:32:05.877 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.875+0000 7f886a200640 1 -- 172.21.15.152:0/1850845155 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f886415d1e0 msgr2=0x7f8864160b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:05.877 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.875+0000 7f886a200640 1 --2- 172.21.15.152:0/1850845155 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f886415d1e0 0x7f8864160b90 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f8858009870 tx=0x7f885802b900 comp rx=0 tx=0).stop 2024-04-20T11:32:05.877 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.876+0000 7f886a200640 1 -- 172.21.15.152:0/1850845155 shutdown_connections 2024-04-20T11:32:05.877 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.876+0000 7f886a200640 1 --2- 172.21.15.152:0/1850845155 >> v2:172.21.15.175:3300/0 conn(0x7f886415d5c0 0x7f88640cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:05.877 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.876+0000 7f886a200640 1 --2- 172.21.15.152:0/1850845155 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f886415d1e0 0x7f8864160b90 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:05.877 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.876+0000 7f886a200640 1 -- 172.21.15.152:0/1850845155 >> 172.21.15.152:0/1850845155 conn(0x7f8864067ae0 msgr2=0x7f886406b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:05.877 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.876+0000 7f886a200640 1 -- 172.21.15.152:0/1850845155 shutdown_connections 2024-04-20T11:32:05.877 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.876+0000 7f886a200640 1 -- 172.21.15.152:0/1850845155 wait complete. 2024-04-20T11:32:05.878 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.877+0000 7f886a200640 1 Processor -- start 2024-04-20T11:32:05.878 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.877+0000 7f886a200640 1 -- start start 2024-04-20T11:32:05.879 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.878+0000 7f886a200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8864071cd0 0x7f8864144b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:05.879 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.878+0000 7f886a200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f886415d1e0 0x7f8864157050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:05.879 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.878+0000 7f886a200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f886415d5c0 0x7f8864141790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:05.879 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.878+0000 7f886a200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f88640ce6a0 con 0x7f886415d1e0 2024-04-20T11:32:05.879 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.878+0000 7f886a200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f88640ce520 con 0x7f886415d5c0 2024-04-20T11:32:05.879 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.878+0000 7f886a200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f88640ccd70 con 0x7f8864071cd0 2024-04-20T11:32:05.879 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.879+0000 7f8863e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f886415d1e0 0x7f8864157050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:05.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.879+0000 7f8863e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f886415d1e0 0x7f8864157050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36672/0 (socket says 172.21.15.152:36672) 2024-04-20T11:32:05.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.879+0000 7f8863e00640 1 -- 172.21.15.152:0/3028113550 learned_addr learned my addr 172.21.15.152:0/3028113550 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:32:05.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.879+0000 7f8863e00640 1 -- 172.21.15.152:0/3028113550 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8864071cd0 msgr2=0x7f8864144b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:05.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.879+0000 7f8863e00640 1 --2- 172.21.15.152:0/3028113550 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8864071cd0 0x7f8864144b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:05.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.879+0000 7f8863e00640 1 -- 172.21.15.152:0/3028113550 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f886415d5c0 msgr2=0x7f8864141790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:05.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.879+0000 7f8863e00640 1 --2- 172.21.15.152:0/3028113550 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f886415d5c0 0x7f8864141790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:05.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.879+0000 7f8863e00640 1 -- 172.21.15.152:0/3028113550 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8864157590 con 0x7f886415d1e0 2024-04-20T11:32:05.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.879+0000 7f8863e00640 1 --2- 172.21.15.152:0/3028113550 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f886415d1e0 0x7f8864157050 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f885000b440 tx=0x7f885000b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:05.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.880+0000 7f8860c00640 1 -- 172.21.15.152:0/3028113550 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8850015020 con 0x7f886415d1e0 2024-04-20T11:32:05.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.880+0000 7f8860c00640 1 -- 172.21.15.152:0/3028113550 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8850011890 con 0x7f886415d1e0 2024-04-20T11:32:05.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.880+0000 7f8860c00640 1 -- 172.21.15.152:0/3028113550 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8850011c80 con 0x7f886415d1e0 2024-04-20T11:32:05.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.880+0000 7f886a200640 1 -- 172.21.15.152:0/3028113550 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88640b35c0 con 0x7f886415d1e0 2024-04-20T11:32:05.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.880+0000 7f886a200640 1 -- 172.21.15.152:0/3028113550 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88640b3860 con 0x7f886415d1e0 2024-04-20T11:32:05.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.881+0000 7f886a200640 1 -- 172.21.15.152:0/3028113550 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88640cfa10 con 0x7f886415d1e0 2024-04-20T11:32:05.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.882+0000 7f8860c00640 1 -- 172.21.15.152:0/3028113550 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 11) ==== 91346+0+0 (secure 0 0 0) 0x7f8850018020 con 0x7f886415d1e0 2024-04-20T11:32:05.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.883+0000 7f8860c00640 1 --2- 172.21.15.152:0/3028113550 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f884406cc50 0x7f884406f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:05.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.883+0000 7f8860c00640 1 -- 172.21.15.152:0/3028113550 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5709+0+0 (secure 0 0 0) 0x7f885008c190 con 0x7f886415d1e0 2024-04-20T11:32:05.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.883+0000 7f8868e00640 1 --2- 172.21.15.152:0/3028113550 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f884406cc50 0x7f884406f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:05.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.884+0000 7f8868e00640 1 --2- 172.21.15.152:0/3028113550 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f884406cc50 0x7f884406f110 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f8858036000 tx=0x7f8858030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:05.888 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:05.888+0000 7f8860c00640 1 -- 172.21.15.152:0/3028113550 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f88640cfa10 con 0x7f886415d1e0 2024-04-20T11:32:06.073 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.073+0000 7f886a200640 1 -- 172.21.15.152:0/3028113550 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8864075e60 con 0x7f886415d1e0 2024-04-20T11:32:06.075 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.074+0000 7f8860c00640 1 -- 172.21.15.152:0/3028113550 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) ==== 74+0+16728 (secure 0 0 0) 0x7f885001f020 con 0x7f886415d1e0 2024-04-20T11:32:06.075 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:32:06.076 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":71,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:31:51.301720+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:31:48.024528+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-20T11:31:48.725648+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:32:06.078 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.077+0000 7f886a200640 1 -- 172.21.15.152:0/3028113550 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f884406cc50 msgr2=0x7f884406f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:06.078 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.077+0000 7f886a200640 1 --2- 172.21.15.152:0/3028113550 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f884406cc50 0x7f884406f110 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f8858036000 tx=0x7f8858030000 comp rx=0 tx=0).stop 2024-04-20T11:32:06.078 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.077+0000 7f886a200640 1 -- 172.21.15.152:0/3028113550 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f886415d1e0 msgr2=0x7f8864157050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:06.078 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.077+0000 7f886a200640 1 --2- 172.21.15.152:0/3028113550 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f886415d1e0 0x7f8864157050 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f885000b440 tx=0x7f885000b900 comp rx=0 tx=0).stop 2024-04-20T11:32:06.078 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.077+0000 7f886a200640 1 -- 172.21.15.152:0/3028113550 shutdown_connections 2024-04-20T11:32:06.078 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.077+0000 7f886a200640 1 --2- 172.21.15.152:0/3028113550 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f884406cc50 0x7f884406f110 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:06.078 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.077+0000 7f886a200640 1 --2- 172.21.15.152:0/3028113550 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f886415d5c0 0x7f8864141790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:06.078 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.077+0000 7f886a200640 1 --2- 172.21.15.152:0/3028113550 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f886415d1e0 0x7f8864157050 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:06.079 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.077+0000 7f886a200640 1 --2- 172.21.15.152:0/3028113550 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8864071cd0 0x7f8864144b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:06.079 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.077+0000 7f886a200640 1 -- 172.21.15.152:0/3028113550 >> 172.21.15.152:0/3028113550 conn(0x7f8864067ae0 msgr2=0x7f886415aee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:06.079 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.077+0000 7f886a200640 1 -- 172.21.15.152:0/3028113550 shutdown_connections 2024-04-20T11:32:06.079 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.077+0000 7f886a200640 1 -- 172.21.15.152:0/3028113550 wait complete. 2024-04-20T11:32:06.097 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T11:26:47.246571+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T11:31:48.024528+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '70', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T11:31:48.725648+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '71', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T11:32:06.097 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T11:32:06.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.293+0000 7fc121a00640 1 Processor -- start 2024-04-20T11:32:06.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.293+0000 7fc121a00640 1 -- start start 2024-04-20T11:32:06.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.295+0000 7fc121a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fc11c0afba0 0x7fc11c162440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:06.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.295+0000 7fc121a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc11c0b3fa0 0x7fc11c0ae110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:06.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.295+0000 7fc121a00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc11c0adc30 con 0x7fc11c15e5d0 2024-04-20T11:32:06.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.295+0000 7fc121a00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fc11c162a70 con 0x7fc11c0afba0 2024-04-20T11:32:06.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.295+0000 7fc121a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc11c162d50 con 0x7fc11c0b3fa0 2024-04-20T11:32:06.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.295+0000 7fc11a000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fc11c0afba0 0x7fc11c162440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:06.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.296+0000 7fc11aa00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fc11c15e5d0 0x7fc11c0af490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:36606/0 (socket says 172.21.15.152:36606) 2024-04-20T11:32:06.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.296+0000 7fc11aa00640 1 -- 172.21.15.152:0/15485776 learned_addr learned my addr 172.21.15.152:0/15485776 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:32:06.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.296+0000 7fc11b400640 1 --2- 172.21.15.152:0/15485776 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc11c0b3fa0 0x7fc11c0ae110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:06.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.296+0000 7fc11a000640 1 -- 172.21.15.152:0/15485776 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc11c0b3fa0 msgr2=0x7fc11c0ae110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:06.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.296+0000 7fc11a000640 1 --2- 172.21.15.152:0/15485776 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc11c0b3fa0 0x7fc11c0ae110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:06.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.296+0000 7fc11a000640 1 -- 172.21.15.152:0/15485776 >> v1:172.21.15.152:6789/0 conn(0x7fc11c15e5d0 legacy=0x7fc11c0af490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:06.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.296+0000 7fc11a000640 1 -- 172.21.15.152:0/15485776 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc11c163030 con 0x7fc11c0afba0 2024-04-20T11:32:06.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.296+0000 7fc119600640 1 -- 172.21.15.152:0/15485776 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1571332306 0 0) 0x7fc11c0adc30 con 0x7fc11c15e5d0 2024-04-20T11:32:06.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.297+0000 7fc11a000640 1 --2- 172.21.15.152:0/15485776 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc11c0afba0 0x7fc11c162440 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fc104009d90 tx=0x7fc10402bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5d594415a9e2dbc server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:06.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.297+0000 7fc119600640 1 -- 172.21.15.152:0/15485776 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc104039070 con 0x7fc11c0afba0 2024-04-20T11:32:06.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.298+0000 7fc119600640 1 -- 172.21.15.152:0/15485776 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc10402cc50 con 0x7fc11c0afba0 2024-04-20T11:32:06.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.298+0000 7fc119600640 1 -- 172.21.15.152:0/15485776 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc104034440 con 0x7fc11c0afba0 2024-04-20T11:32:06.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.298+0000 7fc121a00640 1 -- 172.21.15.152:0/15485776 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc11c0afba0 msgr2=0x7fc11c162440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:06.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.298+0000 7fc121a00640 1 --2- 172.21.15.152:0/15485776 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc11c0afba0 0x7fc11c162440 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fc104009d90 tx=0x7fc10402bea0 comp rx=0 tx=0).stop 2024-04-20T11:32:06.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.298+0000 7fc121a00640 1 -- 172.21.15.152:0/15485776 shutdown_connections 2024-04-20T11:32:06.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.298+0000 7fc121a00640 1 --2- 172.21.15.152:0/15485776 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc11c0b3fa0 0x7fc11c0ae110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:06.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.298+0000 7fc121a00640 1 --2- 172.21.15.152:0/15485776 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc11c0afba0 0x7fc11c162440 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:06.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.298+0000 7fc121a00640 1 -- 172.21.15.152:0/15485776 >> 172.21.15.152:0/15485776 conn(0x7fc11c06b980 msgr2=0x7fc11c06bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:06.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.298+0000 7fc121a00640 1 -- 172.21.15.152:0/15485776 shutdown_connections 2024-04-20T11:32:06.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.298+0000 7fc121a00640 1 -- 172.21.15.152:0/15485776 wait complete. 2024-04-20T11:32:06.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.300+0000 7fc121a00640 1 Processor -- start 2024-04-20T11:32:06.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.300+0000 7fc121a00640 1 -- start start 2024-04-20T11:32:06.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.301+0000 7fc121a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc11c0afba0 0x7fc11c148750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:06.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.301+0000 7fc121a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc11c0b3fa0 0x7fc11c15ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:06.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.301+0000 7fc121a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc11c15e5d0 0x7fc11c1451f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:06.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.301+0000 7fc121a00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fc11c06e0a0 con 0x7fc11c0afba0 2024-04-20T11:32:06.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.301+0000 7fc121a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc11c163310 con 0x7fc11c15e5d0 2024-04-20T11:32:06.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.301+0000 7fc121a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc11c06e220 con 0x7fc11c0b3fa0 2024-04-20T11:32:06.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.301+0000 7fc11aa00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc11c0afba0 0x7fc11c148750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:06.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.302+0000 7fc11aa00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc11c0afba0 0x7fc11c148750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36698/0 (socket says 172.21.15.152:36698) 2024-04-20T11:32:06.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.302+0000 7fc11aa00640 1 -- 172.21.15.152:0/398241399 learned_addr learned my addr 172.21.15.152:0/398241399 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:32:06.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.302+0000 7fc11a000640 1 --2- 172.21.15.152:0/398241399 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc11c0b3fa0 0x7fc11c15ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:06.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.302+0000 7fc11b400640 1 --2- 172.21.15.152:0/398241399 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc11c15e5d0 0x7fc11c1451f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:06.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.302+0000 7fc11aa00640 1 -- 172.21.15.152:0/398241399 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc11c0b3fa0 msgr2=0x7fc11c15ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:06.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.302+0000 7fc11aa00640 1 --2- 172.21.15.152:0/398241399 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc11c0b3fa0 0x7fc11c15ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:06.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.302+0000 7fc11aa00640 1 -- 172.21.15.152:0/398241399 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc11c15e5d0 msgr2=0x7fc11c1451f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:06.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.302+0000 7fc11aa00640 1 --2- 172.21.15.152:0/398241399 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc11c15e5d0 0x7fc11c1451f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:06.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.302+0000 7fc11aa00640 1 -- 172.21.15.152:0/398241399 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc11c15f1e0 con 0x7fc11c0afba0 2024-04-20T11:32:06.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.302+0000 7fc11aa00640 1 --2- 172.21.15.152:0/398241399 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc11c0afba0 0x7fc11c148750 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fc10c00c460 tx=0x7fc10c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:06.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.303+0000 7fc112a00640 1 -- 172.21.15.152:0/398241399 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc10c018070 con 0x7fc11c0afba0 2024-04-20T11:32:06.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.303+0000 7fc112a00640 1 -- 172.21.15.152:0/398241399 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc10c007500 con 0x7fc11c0afba0 2024-04-20T11:32:06.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.303+0000 7fc112a00640 1 -- 172.21.15.152:0/398241399 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc10c004a20 con 0x7fc11c0afba0 2024-04-20T11:32:06.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.303+0000 7fc121a00640 1 -- 172.21.15.152:0/398241399 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc11c160640 con 0x7fc11c0afba0 2024-04-20T11:32:06.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.303+0000 7fc121a00640 1 -- 172.21.15.152:0/398241399 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc11c160bc0 con 0x7fc11c0afba0 2024-04-20T11:32:06.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.304+0000 7fc121a00640 1 -- 172.21.15.152:0/398241399 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc11c0689f0 con 0x7fc11c0afba0 2024-04-20T11:32:06.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.305+0000 7fc112a00640 1 -- 172.21.15.152:0/398241399 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 11) ==== 91346+0+0 (secure 0 0 0) 0x7fc10c004bc0 con 0x7fc11c0afba0 2024-04-20T11:32:06.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.306+0000 7fc112a00640 1 --2- 172.21.15.152:0/398241399 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc0f806cc50 0x7fc0f806f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:06.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.306+0000 7fc112a00640 1 -- 172.21.15.152:0/398241399 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5709+0+0 (secure 0 0 0) 0x7fc10c08b420 con 0x7fc11c0afba0 2024-04-20T11:32:06.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.306+0000 7fc11a000640 1 --2- 172.21.15.152:0/398241399 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc0f806cc50 0x7fc0f806f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:06.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.307+0000 7fc11a000640 1 --2- 172.21.15.152:0/398241399 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc0f806cc50 0x7fc0f806f110 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fc104036000 tx=0x7fc104030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:06.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.311+0000 7fc112a00640 1 -- 172.21.15.152:0/398241399 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fc11c15f1e0 con 0x7fc11c0afba0 2024-04-20T11:32:06.502 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.501+0000 7fc121a00640 1 -- 172.21.15.152:0/398241399 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc11c159e30 con 0x7fc11c0afba0 2024-04-20T11:32:06.502 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.502+0000 7fc112a00640 1 -- 172.21.15.152:0/398241399 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v71) ==== 93+0+10 (secure 0 0 0) 0x7fc10c059400 con 0x7fc11c0afba0 2024-04-20T11:32:06.503 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 1 2024-04-20T11:32:06.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.505+0000 7fc121a00640 1 -- 172.21.15.152:0/398241399 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc0f806cc50 msgr2=0x7fc0f806f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:06.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.505+0000 7fc121a00640 1 --2- 172.21.15.152:0/398241399 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc0f806cc50 0x7fc0f806f110 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fc104036000 tx=0x7fc104030000 comp rx=0 tx=0).stop 2024-04-20T11:32:06.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.505+0000 7fc121a00640 1 -- 172.21.15.152:0/398241399 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc11c0afba0 msgr2=0x7fc11c148750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:06.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.505+0000 7fc121a00640 1 --2- 172.21.15.152:0/398241399 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc11c0afba0 0x7fc11c148750 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fc10c00c460 tx=0x7fc10c00c920 comp rx=0 tx=0).stop 2024-04-20T11:32:06.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.505+0000 7fc121a00640 1 -- 172.21.15.152:0/398241399 shutdown_connections 2024-04-20T11:32:06.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.505+0000 7fc121a00640 1 --2- 172.21.15.152:0/398241399 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fc0f806cc50 0x7fc0f806f110 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:06.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.505+0000 7fc121a00640 1 --2- 172.21.15.152:0/398241399 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc11c15e5d0 0x7fc11c1451f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:06.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.505+0000 7fc121a00640 1 --2- 172.21.15.152:0/398241399 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc11c0b3fa0 0x7fc11c15ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:06.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.505+0000 7fc121a00640 1 --2- 172.21.15.152:0/398241399 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc11c0afba0 0x7fc11c148750 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:06.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.505+0000 7fc121a00640 1 -- 172.21.15.152:0/398241399 >> 172.21.15.152:0/398241399 conn(0x7fc11c06b980 msgr2=0x7fc11c158d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:06.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.505+0000 7fc121a00640 1 -- 172.21.15.152:0/398241399 shutdown_connections 2024-04-20T11:32:06.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.505+0000 7fc121a00640 1 -- 172.21.15.152:0/398241399 wait complete. 2024-04-20T11:32:06.524 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T11:32:06.715 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.714+0000 7f95a8a00640 1 Processor -- start 2024-04-20T11:32:06.715 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.714+0000 7f95a8a00640 1 -- start start 2024-04-20T11:32:06.717 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.716+0000 7f95a8a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f95a415eb50 0x7f95a40d0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:06.717 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.716+0000 7f95a8a00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f95a4163230 0x7f95a40d2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:06.717 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.716+0000 7f95a8a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f95a40d1440 0x7f95a40d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:06.717 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.716+0000 7f95a8a00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f95a40d19c0 con 0x7f95a415eb50 2024-04-20T11:32:06.717 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.716+0000 7f95a8a00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f95a4162750 con 0x7f95a4163230 2024-04-20T11:32:06.717 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.716+0000 7f95a8a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f95a4162a30 con 0x7f95a40d1440 2024-04-20T11:32:06.717 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.716+0000 7f95a2800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f95a415eb50 0x7f95a40d0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:06.717 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.716+0000 7f95a2800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f95a415eb50 0x7f95a40d0f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36706/0 (socket says 172.21.15.152:36706) 2024-04-20T11:32:06.717 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.716+0000 7f95a2800640 1 -- 172.21.15.152:0/3566995971 learned_addr learned my addr 172.21.15.152:0/3566995971 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:32:06.718 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.717+0000 7f95a1e00640 1 --2- 172.21.15.152:0/3566995971 >> v2:172.21.15.175:3300/0 conn(0x7f95a4163230 0x7f95a40d2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:06.718 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.717+0000 7f95a3200640 1 --2- 172.21.15.152:0/3566995971 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f95a40d1440 0x7f95a40d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:06.718 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.717+0000 7f95a2800640 1 -- 172.21.15.152:0/3566995971 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f95a40d1440 msgr2=0x7f95a40d33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:06.719 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.717+0000 7f95a2800640 1 --2- 172.21.15.152:0/3566995971 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f95a40d1440 0x7f95a40d33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:06.719 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.717+0000 7f95a2800640 1 -- 172.21.15.152:0/3566995971 >> v2:172.21.15.175:3300/0 conn(0x7f95a4163230 msgr2=0x7f95a40d2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:06.719 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.717+0000 7f95a2800640 1 --2- 172.21.15.152:0/3566995971 >> v2:172.21.15.175:3300/0 conn(0x7f95a4163230 0x7f95a40d2e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:06.719 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.717+0000 7f95a2800640 1 -- 172.21.15.152:0/3566995971 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95a4162d10 con 0x7f95a415eb50 2024-04-20T11:32:06.719 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.718+0000 7f95a2800640 1 --2- 172.21.15.152:0/3566995971 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f95a415eb50 0x7f95a40d0f00 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f95a408df30 tx=0x7f958c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10b2902db365486e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:06.720 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.719+0000 7f95a1400640 1 -- 172.21.15.152:0/3566995971 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f958c02e070 con 0x7f95a415eb50 2024-04-20T11:32:06.720 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.719+0000 7f95a1400640 1 -- 172.21.15.152:0/3566995971 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f958c037070 con 0x7f95a415eb50 2024-04-20T11:32:06.720 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.719+0000 7f95a1400640 1 -- 172.21.15.152:0/3566995971 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f958c031a00 con 0x7f95a415eb50 2024-04-20T11:32:06.720 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.719+0000 7f95a8a00640 1 -- 172.21.15.152:0/3566995971 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f95a415eb50 msgr2=0x7f95a40d0f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:06.720 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.719+0000 7f95a8a00640 1 --2- 172.21.15.152:0/3566995971 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f95a415eb50 0x7f95a40d0f00 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f95a408df30 tx=0x7f958c02b880 comp rx=0 tx=0).stop 2024-04-20T11:32:06.720 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.719+0000 7f95a8a00640 1 -- 172.21.15.152:0/3566995971 shutdown_connections 2024-04-20T11:32:06.721 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.719+0000 7f95a8a00640 1 --2- 172.21.15.152:0/3566995971 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f95a40d1440 0x7f95a40d33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:06.721 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.719+0000 7f95a8a00640 1 --2- 172.21.15.152:0/3566995971 >> v2:172.21.15.175:3300/0 conn(0x7f95a4163230 0x7f95a40d2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:06.721 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.719+0000 7f95a8a00640 1 --2- 172.21.15.152:0/3566995971 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f95a415eb50 0x7f95a40d0f00 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:06.721 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.719+0000 7f95a8a00640 1 -- 172.21.15.152:0/3566995971 >> 172.21.15.152:0/3566995971 conn(0x7f95a408bfd0 msgr2=0x7f95a40d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:06.721 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.719+0000 7f95a8a00640 1 -- 172.21.15.152:0/3566995971 shutdown_connections 2024-04-20T11:32:06.721 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.719+0000 7f95a8a00640 1 -- 172.21.15.152:0/3566995971 wait complete. 2024-04-20T11:32:06.721 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.721+0000 7f95a8a00640 1 Processor -- start 2024-04-20T11:32:06.721 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.721+0000 7f95a8a00640 1 -- start start 2024-04-20T11:32:06.725 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.722+0000 7f95a8a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f95a40d1440 0x7f95a40b6800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:06.726 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.722+0000 7f95a8a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f95a415eb50 0x7f95a4161e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:06.726 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.722+0000 7f95a8a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f95a4163230 0x7f95a41577d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:06.726 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.722+0000 7f95a8a00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f95a415f0b0 con 0x7f95a415eb50 2024-04-20T11:32:06.726 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.722+0000 7f95a8a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f95a415ef30 con 0x7f95a4163230 2024-04-20T11:32:06.726 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.722+0000 7f95a8a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f95a415f230 con 0x7f95a40d1440 2024-04-20T11:32:06.726 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.723+0000 7f95a1e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f95a415eb50 0x7f95a4161e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:06.726 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.723+0000 7f95a1e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f95a415eb50 0x7f95a4161e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36722/0 (socket says 172.21.15.152:36722) 2024-04-20T11:32:06.726 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.723+0000 7f95a1e00640 1 -- 172.21.15.152:0/4012737556 learned_addr learned my addr 172.21.15.152:0/4012737556 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:32:06.726 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.723+0000 7f95a2800640 1 --2- 172.21.15.152:0/4012737556 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f95a40d1440 0x7f95a40b6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:06.726 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.723+0000 7f95a3200640 1 --2- 172.21.15.152:0/4012737556 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f95a4163230 0x7f95a41577d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:06.727 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.723+0000 7f95a1e00640 1 -- 172.21.15.152:0/4012737556 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f95a40d1440 msgr2=0x7f95a40b6800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:06.727 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.723+0000 7f95a1e00640 1 --2- 172.21.15.152:0/4012737556 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f95a40d1440 0x7f95a40b6800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:06.727 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.723+0000 7f95a1e00640 1 -- 172.21.15.152:0/4012737556 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f95a4163230 msgr2=0x7f95a41577d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:06.727 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.723+0000 7f95a1e00640 1 --2- 172.21.15.152:0/4012737556 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f95a4163230 0x7f95a41577d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:06.727 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.723+0000 7f95a1e00640 1 -- 172.21.15.152:0/4012737556 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95a40b62b0 con 0x7f95a415eb50 2024-04-20T11:32:06.727 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.723+0000 7f95a3200640 1 --2- 172.21.15.152:0/4012737556 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f95a4163230 0x7f95a41577d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:32:06.727 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.723+0000 7f95a1e00640 1 --2- 172.21.15.152:0/4012737556 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f95a415eb50 0x7f95a4161e70 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f9594007f20 tx=0x7f959400e570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:06.727 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.724+0000 7f959aa00640 1 -- 172.21.15.152:0/4012737556 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9594014620 con 0x7f95a415eb50 2024-04-20T11:32:06.727 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.724+0000 7f959aa00640 1 -- 172.21.15.152:0/4012737556 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9594019070 con 0x7f95a415eb50 2024-04-20T11:32:06.727 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.724+0000 7f959aa00640 1 -- 172.21.15.152:0/4012737556 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9594014a00 con 0x7f95a415eb50 2024-04-20T11:32:06.727 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.724+0000 7f95a8a00640 1 -- 172.21.15.152:0/4012737556 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95a415aff0 con 0x7f95a415eb50 2024-04-20T11:32:06.727 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.724+0000 7f95a8a00640 1 -- 172.21.15.152:0/4012737556 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95a415b570 con 0x7f95a415eb50 2024-04-20T11:32:06.727 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.725+0000 7f95a8a00640 1 -- 172.21.15.152:0/4012737556 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95a4075e60 con 0x7f95a415eb50 2024-04-20T11:32:06.728 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.726+0000 7f959aa00640 1 -- 172.21.15.152:0/4012737556 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 11) ==== 91346+0+0 (secure 0 0 0) 0x7f9594014ba0 con 0x7f95a415eb50 2024-04-20T11:32:06.728 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.727+0000 7f959aa00640 1 --2- 172.21.15.152:0/4012737556 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f958006cc50 0x7f958006f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:06.728 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.727+0000 7f959aa00640 1 -- 172.21.15.152:0/4012737556 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5709+0+0 (secure 0 0 0) 0x7f959408c920 con 0x7f95a415eb50 2024-04-20T11:32:06.728 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.727+0000 7f95a2800640 1 --2- 172.21.15.152:0/4012737556 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f958006cc50 0x7f958006f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:06.728 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.728+0000 7f95a2800640 1 --2- 172.21.15.152:0/4012737556 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f958006cc50 0x7f958006f110 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f958c036000 tx=0x7f958c03d040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:06.731 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.731+0000 7f959aa00640 1 -- 172.21.15.152:0/4012737556 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f959405a990 con 0x7f95a415eb50 2024-04-20T11:32:06.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.915+0000 7f95a8a00640 1 -- 172.21.15.152:0/4012737556 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f95a408c680 con 0x7f95a415eb50 2024-04-20T11:32:06.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.916+0000 7f959aa00640 1 -- 172.21.15.152:0/4012737556 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v71) ==== 104+0+11 (secure 0 0 0) 0x7f959405a290 con 0x7f95a415eb50 2024-04-20T11:32:06.917 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 32 2024-04-20T11:32:06.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.918+0000 7f95a8a00640 1 -- 172.21.15.152:0/4012737556 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f958006cc50 msgr2=0x7f958006f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:06.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.918+0000 7f95a8a00640 1 --2- 172.21.15.152:0/4012737556 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f958006cc50 0x7f958006f110 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f958c036000 tx=0x7f958c03d040 comp rx=0 tx=0).stop 2024-04-20T11:32:06.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.918+0000 7f95a8a00640 1 -- 172.21.15.152:0/4012737556 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f95a415eb50 msgr2=0x7f95a4161e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:06.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.918+0000 7f95a8a00640 1 --2- 172.21.15.152:0/4012737556 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f95a415eb50 0x7f95a4161e70 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f9594007f20 tx=0x7f959400e570 comp rx=0 tx=0).stop 2024-04-20T11:32:06.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.918+0000 7f95a8a00640 1 -- 172.21.15.152:0/4012737556 shutdown_connections 2024-04-20T11:32:06.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.918+0000 7f95a8a00640 1 --2- 172.21.15.152:0/4012737556 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f958006cc50 0x7f958006f110 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:06.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.918+0000 7f95a8a00640 1 --2- 172.21.15.152:0/4012737556 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f95a4163230 0x7f95a41577d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:06.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.918+0000 7f95a8a00640 1 --2- 172.21.15.152:0/4012737556 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f95a415eb50 0x7f95a4161e70 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:06.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.918+0000 7f95a8a00640 1 --2- 172.21.15.152:0/4012737556 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f95a40d1440 0x7f95a40b6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:06.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.918+0000 7f95a8a00640 1 -- 172.21.15.152:0/4012737556 >> 172.21.15.152:0/4012737556 conn(0x7f95a408bfd0 msgr2=0x7f95a40cd660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:06.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.919+0000 7f95a8a00640 1 -- 172.21.15.152:0/4012737556 shutdown_connections 2024-04-20T11:32:06.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:06.919+0000 7f95a8a00640 1 -- 172.21.15.152:0/4012737556 wait complete. 2024-04-20T11:32:06.936 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T11:32:07.131 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.130+0000 7fb778400640 1 Processor -- start 2024-04-20T11:32:07.131 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.131+0000 7fb778400640 1 -- start start 2024-04-20T11:32:07.133 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.133+0000 7fb778400640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fb770162bc0 0x7fb770167240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:07.133 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.133+0000 7fb778400640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb770167780 con 0x7fb7700b2880 2024-04-20T11:32:07.134 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.133+0000 7fb778400640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb77006fc30 con 0x7fb77006f720 2024-04-20T11:32:07.134 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.133+0000 7fb778400640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fb7700ccb40 con 0x7fb770162bc0 2024-04-20T11:32:07.134 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.133+0000 7fb776400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fb7700b2880 0x7fb77006e010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:36614/0 (socket says 172.21.15.152:36614) 2024-04-20T11:32:07.134 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.133+0000 7fb776400640 1 -- 172.21.15.152:0/2631620191 learned_addr learned my addr 172.21.15.152:0/2631620191 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:32:07.136 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.133+0000 7fb775a00640 1 --2- 172.21.15.152:0/2631620191 >> v2:172.21.15.175:3300/0 conn(0x7fb770162bc0 0x7fb770167240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:07.136 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.134+0000 7fb76fe00640 1 -- 172.21.15.152:0/2631620191 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1409799418 0 0) 0x7fb768001110 con 0x7fb7700b2880 2024-04-20T11:32:07.136 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.134+0000 7fb76fe00640 1 -- 172.21.15.152:0/2631620191 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb758003520 con 0x7fb7700b2880 2024-04-20T11:32:07.136 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.134+0000 7fb76fe00640 1 -- 172.21.15.152:0/2631620191 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 394873625 0 0) 0x7fb758003520 con 0x7fb7700b2880 2024-04-20T11:32:07.136 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.134+0000 7fb76fe00640 1 -- 172.21.15.152:0/2631620191 >> v2:172.21.15.175:3300/0 conn(0x7fb770162bc0 msgr2=0x7fb770167240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:07.136 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.134+0000 7fb76fe00640 1 --2- 172.21.15.152:0/2631620191 >> v2:172.21.15.175:3300/0 conn(0x7fb770162bc0 0x7fb770167240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:07.136 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.134+0000 7fb76fe00640 1 -- 172.21.15.152:0/2631620191 >> v1:172.21.15.175:6789/0 conn(0x7fb77006f720 legacy=0x7fb7700b2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:07.136 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.134+0000 7fb76fe00640 1 -- 172.21.15.152:0/2631620191 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7700b4090 con 0x7fb7700b2880 2024-04-20T11:32:07.136 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.134+0000 7fb76fe00640 1 -- 172.21.15.152:0/2631620191 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7fb768001840 con 0x7fb7700b2880 2024-04-20T11:32:07.137 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.134+0000 7fb76fe00640 1 -- 172.21.15.152:0/2631620191 >> v1:172.21.15.152:6789/0 conn(0x7fb7700b2880 legacy=0x7fb77006e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:07.137 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.134+0000 7fb76fe00640 1 --2- 172.21.15.152:0/2631620191 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7580039b0 0x7fb758003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:07.137 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.134+0000 7fb76fe00640 1 --2- 172.21.15.152:0/2631620191 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb758004800 0x7fb758004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:07.137 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.134+0000 7fb76fe00640 1 --2- 172.21.15.152:0/2631620191 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb7580051c0 0x7fb75800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:07.137 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.134+0000 7fb76fe00640 1 -- 172.21.15.152:0/2631620191 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fb7700ccb40 con 0x7fb758004800 2024-04-20T11:32:07.137 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.134+0000 7fb76fe00640 1 -- 172.21.15.152:0/2631620191 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb75800c060 con 0x7fb7580039b0 2024-04-20T11:32:07.137 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.134+0000 7fb76fe00640 1 -- 172.21.15.152:0/2631620191 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb75800c260 con 0x7fb7580051c0 2024-04-20T11:32:07.137 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.135+0000 7fb775000640 1 --2- 172.21.15.152:0/2631620191 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb758004800 0x7fb758004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:07.137 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.135+0000 7fb776400640 1 --2- 172.21.15.152:0/2631620191 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb7580051c0 0x7fb75800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:07.137 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.135+0000 7fb775a00640 1 --2- 172.21.15.152:0/2631620191 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7580039b0 0x7fb758003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:07.137 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.135+0000 7fb775000640 1 -- 172.21.15.152:0/2631620191 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb7580051c0 msgr2=0x7fb75800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:07.137 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.135+0000 7fb775000640 1 --2- 172.21.15.152:0/2631620191 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb7580051c0 0x7fb75800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:07.137 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.135+0000 7fb775000640 1 -- 172.21.15.152:0/2631620191 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7580039b0 msgr2=0x7fb758003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:07.137 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.135+0000 7fb775000640 1 --2- 172.21.15.152:0/2631620191 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7580039b0 0x7fb758003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:07.138 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.135+0000 7fb775000640 1 -- 172.21.15.152:0/2631620191 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb758003520 con 0x7fb758004800 2024-04-20T11:32:07.138 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.135+0000 7fb776400640 1 --2- 172.21.15.152:0/2631620191 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb7580051c0 0x7fb75800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:32:07.138 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.135+0000 7fb776400640 1 -- 172.21.15.152:0/2631620191 reap_dead start 2024-04-20T11:32:07.139 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.137+0000 7fb775000640 1 --2- 172.21.15.152:0/2631620191 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb758004800 0x7fb758004c80 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fb75c001720 tx=0x7fb758005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fd870a31634ac7e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:07.139 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.137+0000 7fb76fe00640 1 -- 172.21.15.152:0/2631620191 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb75c001370 con 0x7fb758004800 2024-04-20T11:32:07.139 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.137+0000 7fb76fe00640 1 -- 172.21.15.152:0/2631620191 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb75c0052d0 con 0x7fb758004800 2024-04-20T11:32:07.139 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.137+0000 7fb76fe00640 1 -- 172.21.15.152:0/2631620191 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb75c003830 con 0x7fb758004800 2024-04-20T11:32:07.139 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.137+0000 7fb778400640 1 -- 172.21.15.152:0/2631620191 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb758004800 msgr2=0x7fb758004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:07.139 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.137+0000 7fb778400640 1 --2- 172.21.15.152:0/2631620191 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb758004800 0x7fb758004c80 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fb75c001720 tx=0x7fb758005830 comp rx=0 tx=0).stop 2024-04-20T11:32:07.139 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.138+0000 7fb778400640 1 -- 172.21.15.152:0/2631620191 shutdown_connections 2024-04-20T11:32:07.139 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.138+0000 7fb778400640 1 --2- 172.21.15.152:0/2631620191 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb758004800 0x7fb758004c80 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:07.139 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.138+0000 7fb778400640 1 -- 172.21.15.152:0/2631620191 >> 172.21.15.152:0/2631620191 conn(0x7fb77006b980 msgr2=0x7fb77006bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:07.139 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.138+0000 7fb778400640 1 -- 172.21.15.152:0/2631620191 shutdown_connections 2024-04-20T11:32:07.139 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.138+0000 7fb778400640 1 -- 172.21.15.152:0/2631620191 wait complete. 2024-04-20T11:32:07.140 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.139+0000 7fb778400640 1 Processor -- start 2024-04-20T11:32:07.140 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.139+0000 7fb778400640 1 -- start start 2024-04-20T11:32:07.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.140+0000 7fb778400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb758004800 0x7fb77015aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:07.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.140+0000 7fb778400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb77013c900 0x7fb7701616e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:07.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.140+0000 7fb778400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb77015fcd0 0x7fb770156700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:07.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.140+0000 7fb778400640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fb7700b4650 con 0x7fb758004800 2024-04-20T11:32:07.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.140+0000 7fb778400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb7700b3610 con 0x7fb77015fcd0 2024-04-20T11:32:07.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.140+0000 7fb778400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb770162fa0 con 0x7fb77013c900 2024-04-20T11:32:07.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.141+0000 7fb775a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb758004800 0x7fb77015aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:07.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.141+0000 7fb775a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb758004800 0x7fb77015aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36744/0 (socket says 172.21.15.152:36744) 2024-04-20T11:32:07.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.141+0000 7fb775a00640 1 -- 172.21.15.152:0/8270692 learned_addr learned my addr 172.21.15.152:0/8270692 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:32:07.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.141+0000 7fb776400640 1 --2- 172.21.15.152:0/8270692 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb77015fcd0 0x7fb770156700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:07.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.141+0000 7fb775000640 1 --2- 172.21.15.152:0/8270692 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb77013c900 0x7fb7701616e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:07.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.141+0000 7fb775a00640 1 -- 172.21.15.152:0/8270692 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb77013c900 msgr2=0x7fb7701616e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:07.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.141+0000 7fb775a00640 1 --2- 172.21.15.152:0/8270692 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb77013c900 0x7fb7701616e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:07.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.141+0000 7fb775a00640 1 -- 172.21.15.152:0/8270692 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb77015fcd0 msgr2=0x7fb770156700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:07.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.141+0000 7fb775a00640 1 --2- 172.21.15.152:0/8270692 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb77015fcd0 0x7fb770156700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:07.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.141+0000 7fb775a00640 1 -- 172.21.15.152:0/8270692 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb77015f510 con 0x7fb758004800 2024-04-20T11:32:07.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.141+0000 7fb775a00640 1 --2- 172.21.15.152:0/8270692 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb758004800 0x7fb77015aa40 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fb764011e30 tx=0x7fb76400f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:07.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.142+0000 7fb76d600640 1 -- 172.21.15.152:0/8270692 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb764015630 con 0x7fb758004800 2024-04-20T11:32:07.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.142+0000 7fb76d600640 1 -- 172.21.15.152:0/8270692 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7640157d0 con 0x7fb758004800 2024-04-20T11:32:07.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.142+0000 7fb76d600640 1 -- 172.21.15.152:0/8270692 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb764015ba0 con 0x7fb758004800 2024-04-20T11:32:07.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.142+0000 7fb778400640 1 -- 172.21.15.152:0/8270692 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb770155c60 con 0x7fb758004800 2024-04-20T11:32:07.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.142+0000 7fb778400640 1 -- 172.21.15.152:0/8270692 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb77015f7b0 con 0x7fb758004800 2024-04-20T11:32:07.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.143+0000 7fb778400640 1 -- 172.21.15.152:0/8270692 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7700689f0 con 0x7fb758004800 2024-04-20T11:32:07.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.144+0000 7fb76d600640 1 -- 172.21.15.152:0/8270692 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 11) ==== 91346+0+0 (secure 0 0 0) 0x7fb76401d400 con 0x7fb758004800 2024-04-20T11:32:07.145 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.145+0000 7fb76d600640 1 --2- 172.21.15.152:0/8270692 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fb75006cc50 0x7fb75006f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:07.145 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.145+0000 7fb76d600640 1 -- 172.21.15.152:0/8270692 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5709+0+0 (secure 0 0 0) 0x7fb76408d370 con 0x7fb758004800 2024-04-20T11:32:07.146 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.145+0000 7fb775000640 1 --2- 172.21.15.152:0/8270692 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fb75006cc50 0x7fb75006f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:07.146 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.146+0000 7fb775000640 1 --2- 172.21.15.152:0/8270692 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fb75006cc50 0x7fb75006f110 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fb75c033000 tx=0x7fb75c009030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:07.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.150+0000 7fb76d600640 1 -- 172.21.15.152:0/8270692 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fb764019030 con 0x7fb758004800 2024-04-20T11:32:07.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.349+0000 7fb778400640 1 -- 172.21.15.152:0/8270692 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb77006e8b0 con 0x7fb758004800 2024-04-20T11:32:07.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.350+0000 7fb76d600640 1 -- 172.21.15.152:0/8270692 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v71) ==== 100+0+11 (secure 0 0 0) 0x7fb764010040 con 0x7fb758004800 2024-04-20T11:32:07.351 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 64 2024-04-20T11:32:07.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.353+0000 7fb778400640 1 -- 172.21.15.152:0/8270692 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fb75006cc50 msgr2=0x7fb75006f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:07.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.353+0000 7fb778400640 1 --2- 172.21.15.152:0/8270692 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fb75006cc50 0x7fb75006f110 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fb75c033000 tx=0x7fb75c009030 comp rx=0 tx=0).stop 2024-04-20T11:32:07.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.353+0000 7fb778400640 1 -- 172.21.15.152:0/8270692 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb758004800 msgr2=0x7fb77015aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:07.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.353+0000 7fb778400640 1 --2- 172.21.15.152:0/8270692 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb758004800 0x7fb77015aa40 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fb764011e30 tx=0x7fb76400f580 comp rx=0 tx=0).stop 2024-04-20T11:32:07.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.353+0000 7fb778400640 1 -- 172.21.15.152:0/8270692 shutdown_connections 2024-04-20T11:32:07.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.353+0000 7fb778400640 1 --2- 172.21.15.152:0/8270692 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fb75006cc50 0x7fb75006f110 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:07.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.353+0000 7fb778400640 1 --2- 172.21.15.152:0/8270692 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb77015fcd0 0x7fb770156700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:07.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.353+0000 7fb778400640 1 --2- 172.21.15.152:0/8270692 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb77013c900 0x7fb7701616e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:07.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.353+0000 7fb778400640 1 --2- 172.21.15.152:0/8270692 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb758004800 0x7fb77015aa40 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:07.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.353+0000 7fb778400640 1 -- 172.21.15.152:0/8270692 >> 172.21.15.152:0/8270692 conn(0x7fb77006b980 msgr2=0x7fb7700b1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:07.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.354+0000 7fb778400640 1 -- 172.21.15.152:0/8270692 shutdown_connections 2024-04-20T11:32:07.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.354+0000 7fb778400640 1 -- 172.21.15.152:0/8270692 wait complete. 2024-04-20T11:32:07.372 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-20T11:32:07.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.563+0000 7f4417800640 1 Processor -- start 2024-04-20T11:32:07.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.564+0000 7f4417800640 1 -- start start 2024-04-20T11:32:07.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.565+0000 7f4417800640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f441006fa50 0x7f44100b3470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:07.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.565+0000 7f4417800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f441015fd60 0x7f44100b39b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:07.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.565+0000 7f4417800640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44100b4130 con 0x7f441015b2b0 2024-04-20T11:32:07.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.565+0000 7f4417800640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f44100b3ef0 con 0x7f441006fa50 2024-04-20T11:32:07.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.565+0000 7f4417800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f441015f8d0 con 0x7f441015fd60 2024-04-20T11:32:07.567 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.566+0000 7f4407e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f441006fa50 0x7f44100b3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:07.567 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.566+0000 7f4415800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f441015fd60 0x7f44100b39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:07.567 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.566+0000 7f4407e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f441006fa50 0x7f44100b3470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.152:46114/0 (socket says 172.21.15.152:46114) 2024-04-20T11:32:07.567 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.566+0000 7f4407e00640 1 -- 172.21.15.152:0/2151818323 learned_addr learned my addr 172.21.15.152:0/2151818323 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:32:07.567 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.567+0000 7f4407e00640 1 -- 172.21.15.152:0/2151818323 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f441015fd60 msgr2=0x7f44100b39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:07.567 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.567+0000 7f4407e00640 1 --2- 172.21.15.152:0/2151818323 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f441015fd60 0x7f44100b39b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:07.567 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.567+0000 7f4407e00640 1 -- 172.21.15.152:0/2151818323 >> v1:172.21.15.175:6789/0 conn(0x7f441015b2b0 legacy=0x7f441006f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:07.567 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.567+0000 7f4407e00640 1 -- 172.21.15.152:0/2151818323 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f441015f280 con 0x7f441006fa50 2024-04-20T11:32:07.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.568+0000 7f4407e00640 1 --2- 172.21.15.152:0/2151818323 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f441006fa50 0x7f44100b3470 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f43f8009870 tx=0x7f43f802b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1619d17299a47aeb server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:07.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.569+0000 7f440fe00640 1 -- 172.21.15.152:0/2151818323 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43f8034070 con 0x7f441006fa50 2024-04-20T11:32:07.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.569+0000 7f440fe00640 1 -- 172.21.15.152:0/2151818323 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43f8030840 con 0x7f441006fa50 2024-04-20T11:32:07.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.569+0000 7f440fe00640 1 -- 172.21.15.152:0/2151818323 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43f8030c10 con 0x7f441006fa50 2024-04-20T11:32:07.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.569+0000 7f4417800640 1 -- 172.21.15.152:0/2151818323 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f441006fa50 msgr2=0x7f44100b3470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:07.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.569+0000 7f4417800640 1 --2- 172.21.15.152:0/2151818323 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f441006fa50 0x7f44100b3470 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f43f8009870 tx=0x7f43f802b980 comp rx=0 tx=0).stop 2024-04-20T11:32:07.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.569+0000 7f4417800640 1 -- 172.21.15.152:0/2151818323 shutdown_connections 2024-04-20T11:32:07.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.569+0000 7f4417800640 1 --2- 172.21.15.152:0/2151818323 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f441015fd60 0x7f44100b39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:07.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.569+0000 7f4417800640 1 --2- 172.21.15.152:0/2151818323 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f441006fa50 0x7f44100b3470 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:07.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.569+0000 7f4417800640 1 -- 172.21.15.152:0/2151818323 >> 172.21.15.152:0/2151818323 conn(0x7f4410067cf0 msgr2=0x7f441006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:07.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.569+0000 7f4417800640 1 -- 172.21.15.152:0/2151818323 shutdown_connections 2024-04-20T11:32:07.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.569+0000 7f4417800640 1 -- 172.21.15.152:0/2151818323 wait complete. 2024-04-20T11:32:07.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.571+0000 7f4417800640 1 Processor -- start 2024-04-20T11:32:07.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.571+0000 7f4417800640 1 -- start start 2024-04-20T11:32:07.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.572+0000 7f4417800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f441006fa50 0x7f4410148690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:07.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.572+0000 7f4417800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f441015b2b0 0x7f4410144e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:07.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.572+0000 7f4417800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f441015fd60 0x7f4410145390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:07.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.572+0000 7f4417800640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f441015b810 con 0x7f441015fd60 2024-04-20T11:32:07.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.572+0000 7f4417800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f441015b690 con 0x7f441006fa50 2024-04-20T11:32:07.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.572+0000 7f4417800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f441015b990 con 0x7f441015b2b0 2024-04-20T11:32:07.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.572+0000 7f4415800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f441015fd60 0x7f4410145390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:07.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.573+0000 7f4415800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f441015fd60 0x7f4410145390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36754/0 (socket says 172.21.15.152:36754) 2024-04-20T11:32:07.574 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.573+0000 7f4415800640 1 -- 172.21.15.152:0/3778057508 learned_addr learned my addr 172.21.15.152:0/3778057508 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:32:07.574 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.573+0000 7f4414e00640 1 --2- 172.21.15.152:0/3778057508 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f441006fa50 0x7f4410148690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:07.574 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.573+0000 7f4407e00640 1 --2- 172.21.15.152:0/3778057508 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f441015b2b0 0x7f4410144e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:07.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.573+0000 7f4415800640 1 -- 172.21.15.152:0/3778057508 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f441015b2b0 msgr2=0x7f4410144e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:07.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.573+0000 7f4415800640 1 --2- 172.21.15.152:0/3778057508 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f441015b2b0 0x7f4410144e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:07.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.573+0000 7f4415800640 1 -- 172.21.15.152:0/3778057508 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f441006fa50 msgr2=0x7f4410148690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:07.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.573+0000 7f4415800640 1 --2- 172.21.15.152:0/3778057508 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f441006fa50 0x7f4410148690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:07.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.573+0000 7f4415800640 1 -- 172.21.15.152:0/3778057508 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f441015daa0 con 0x7f441015fd60 2024-04-20T11:32:07.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.573+0000 7f4414e00640 1 --2- 172.21.15.152:0/3778057508 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f441006fa50 0x7f4410148690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:32:07.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.573+0000 7f4415800640 1 --2- 172.21.15.152:0/3778057508 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f441015fd60 0x7f4410145390 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f440800bde0 tx=0x7f440800e490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:07.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.574+0000 7f440d600640 1 -- 172.21.15.152:0/3778057508 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f440801a070 con 0x7f441015fd60 2024-04-20T11:32:07.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.574+0000 7f440d600640 1 -- 172.21.15.152:0/3778057508 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f440800ed20 con 0x7f441015fd60 2024-04-20T11:32:07.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.574+0000 7f440d600640 1 -- 172.21.15.152:0/3778057508 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f440800b3e0 con 0x7f441015fd60 2024-04-20T11:32:07.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.574+0000 7f4417800640 1 -- 172.21.15.152:0/3778057508 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f441015dd80 con 0x7f441015fd60 2024-04-20T11:32:07.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.574+0000 7f4417800640 1 -- 172.21.15.152:0/3778057508 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f441015e340 con 0x7f441015fd60 2024-04-20T11:32:07.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.575+0000 7f4417800640 1 -- 172.21.15.152:0/3778057508 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44100689f0 con 0x7f441015fd60 2024-04-20T11:32:07.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.576+0000 7f440d600640 1 -- 172.21.15.152:0/3778057508 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 11) ==== 91346+0+0 (secure 0 0 0) 0x7f440801d070 con 0x7f441015fd60 2024-04-20T11:32:07.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.577+0000 7f440d600640 1 --2- 172.21.15.152:0/3778057508 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f43ec06cc50 0x7f43ec06f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:07.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.577+0000 7f440d600640 1 -- 172.21.15.152:0/3778057508 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5709+0+0 (secure 0 0 0) 0x7f440808c3e0 con 0x7f441015fd60 2024-04-20T11:32:07.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.577+0000 7f4414e00640 1 --2- 172.21.15.152:0/3778057508 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f43ec06cc50 0x7f43ec06f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:07.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.578+0000 7f4414e00640 1 --2- 172.21.15.152:0/3778057508 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f43ec06cc50 0x7f43ec06f110 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f4400001680 tx=0x7f4400009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:07.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.581+0000 7f440d600640 1 -- 172.21.15.152:0/3778057508 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f440805a2b0 con 0x7f441015fd60 2024-04-20T11:32:07.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.796+0000 7f4417800640 1 -- 172.21.15.152:0/3778057508 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f441006eef0 con 0x7f441015fd60 2024-04-20T11:32:07.797 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.797+0000 7f440d600640 1 -- 172.21.15.152:0/3778057508 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v71) ==== 80+0+0 (secure 0 0 0) 0x7f4408059bb0 con 0x7f441015fd60 2024-04-20T11:32:07.797 INFO:teuthology.orchestra.run.smithi152.stderr:listed 0 entries 2024-04-20T11:32:07.800 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.799+0000 7f4417800640 1 -- 172.21.15.152:0/3778057508 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f43ec06cc50 msgr2=0x7f43ec06f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:07.800 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.799+0000 7f4417800640 1 --2- 172.21.15.152:0/3778057508 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f43ec06cc50 0x7f43ec06f110 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f4400001680 tx=0x7f4400009040 comp rx=0 tx=0).stop 2024-04-20T11:32:07.800 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.799+0000 7f4417800640 1 -- 172.21.15.152:0/3778057508 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f441015fd60 msgr2=0x7f4410145390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:07.801 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.799+0000 7f4417800640 1 --2- 172.21.15.152:0/3778057508 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f441015fd60 0x7f4410145390 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f440800bde0 tx=0x7f440800e490 comp rx=0 tx=0).stop 2024-04-20T11:32:07.801 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.800+0000 7f4417800640 1 -- 172.21.15.152:0/3778057508 shutdown_connections 2024-04-20T11:32:07.801 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.800+0000 7f4417800640 1 --2- 172.21.15.152:0/3778057508 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f43ec06cc50 0x7f43ec06f110 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:07.801 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.800+0000 7f4417800640 1 --2- 172.21.15.152:0/3778057508 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f441015fd60 0x7f4410145390 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:07.801 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.800+0000 7f4417800640 1 --2- 172.21.15.152:0/3778057508 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f441015b2b0 0x7f4410144e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:07.801 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.800+0000 7f4417800640 1 --2- 172.21.15.152:0/3778057508 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f441006fa50 0x7f4410148690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:07.801 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.800+0000 7f4417800640 1 -- 172.21.15.152:0/3778057508 >> 172.21.15.152:0/3778057508 conn(0x7f4410067cf0 msgr2=0x7f44100d0980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:07.801 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.800+0000 7f4417800640 1 -- 172.21.15.152:0/3778057508 shutdown_connections 2024-04-20T11:32:07.801 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:07.800+0000 7f4417800640 1 -- 172.21.15.152:0/3778057508 wait complete. 2024-04-20T11:32:07.819 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi152.front.sepia.ceph.com... 2024-04-20T11:32:07.819 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:32:07.819 DEBUG:teuthology.orchestra.run.smithi152:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-20T11:32:07.887 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi152.stderr:ceph-fuse[47498]: fuse finished with error 0 and tester_r 0 2024-04-20T11:32:07.891 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T11:32:13.993 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi152.front.sepia.ceph.com 2024-04-20T11:32:13.994 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:32:13.994 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-20T11:32:14.051 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T11:32:14.051 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:32:14.109 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:32:14.319 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.319+0000 7fa9be600640 1 Processor -- start 2024-04-20T11:32:14.320 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.319+0000 7fa9be600640 1 -- start start 2024-04-20T11:32:14.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.321+0000 7fa9be600640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fa9b80d1810 0x7fa9b80d1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:14.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.321+0000 7fa9be600640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9b80b80b0 con 0x7fa9b815eb80 2024-04-20T11:32:14.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.321+0000 7fa9be600640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9b80b8330 con 0x7fa9b80ba520 2024-04-20T11:32:14.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.321+0000 7fa9be600640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fa9b80d2180 con 0x7fa9b80d1810 2024-04-20T11:32:14.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.321+0000 7fa9b7400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fa9b815eb80 0x7fa9b80b79a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:36624/0 (socket says 172.21.15.152:36624) 2024-04-20T11:32:14.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.321+0000 7fa9b7400640 1 -- 172.21.15.152:0/4118845548 learned_addr learned my addr 172.21.15.152:0/4118845548 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:32:14.323 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.322+0000 7fa9b7e00640 1 --2- 172.21.15.152:0/4118845548 >> v2:172.21.15.175:3300/0 conn(0x7fa9b80d1810 0x7fa9b80d1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:14.323 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.322+0000 7fa9b6000640 1 -- 172.21.15.152:0/4118845548 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3508808838 0 0) 0x7fa9a8001110 con 0x7fa9b815eb80 2024-04-20T11:32:14.324 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.322+0000 7fa9b6000640 1 -- 172.21.15.152:0/4118845548 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa99c003520 con 0x7fa9b815eb80 2024-04-20T11:32:14.324 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.322+0000 7fa9b6000640 1 -- 172.21.15.152:0/4118845548 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2961422411 0 0) 0x7fa99c003520 con 0x7fa9b815eb80 2024-04-20T11:32:14.324 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.322+0000 7fa9b6000640 1 -- 172.21.15.152:0/4118845548 >> v2:172.21.15.175:3300/0 conn(0x7fa9b80d1810 msgr2=0x7fa9b80d1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:14.324 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.322+0000 7fa9b6000640 1 --2- 172.21.15.152:0/4118845548 >> v2:172.21.15.175:3300/0 conn(0x7fa9b80d1810 0x7fa9b80d1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:14.324 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.322+0000 7fa9b6000640 1 -- 172.21.15.152:0/4118845548 >> v1:172.21.15.175:6789/0 conn(0x7fa9b80ba520 legacy=0x7fa9b80d28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:14.324 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.323+0000 7fa9b6000640 1 -- 172.21.15.152:0/4118845548 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9b80b8610 con 0x7fa9b815eb80 2024-04-20T11:32:14.324 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.323+0000 7fa9b6000640 1 -- 172.21.15.152:0/4118845548 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7fa9a8001840 con 0x7fa9b815eb80 2024-04-20T11:32:14.324 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.323+0000 7fa9b6000640 1 -- 172.21.15.152:0/4118845548 >> v1:172.21.15.152:6789/0 conn(0x7fa9b815eb80 legacy=0x7fa9b80b79a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:14.324 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.323+0000 7fa9b6000640 1 --2- 172.21.15.152:0/4118845548 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa99c003a70 0x7fa99c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:14.324 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.323+0000 7fa9b6000640 1 --2- 172.21.15.152:0/4118845548 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa99c004800 0x7fa99c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:14.324 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.323+0000 7fa9b6000640 1 --2- 172.21.15.152:0/4118845548 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa99c0053c0 0x7fa99c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:14.324 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.323+0000 7fa9b6000640 1 -- 172.21.15.152:0/4118845548 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b80d2180 con 0x7fa99c004800 2024-04-20T11:32:14.324 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.323+0000 7fa9b6000640 1 -- 172.21.15.152:0/4118845548 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa99c00c250 con 0x7fa99c003a70 2024-04-20T11:32:14.324 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.323+0000 7fa9b6000640 1 -- 172.21.15.152:0/4118845548 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa99c00c450 con 0x7fa99c0053c0 2024-04-20T11:32:14.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.323+0000 7fa9b6a00640 1 --2- 172.21.15.152:0/4118845548 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa99c004800 0x7fa99c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:14.326 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.323+0000 7fa9b6a00640 1 -- 172.21.15.152:0/4118845548 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa99c0053c0 msgr2=0x7fa99c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:14.326 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.323+0000 7fa9b6a00640 1 --2- 172.21.15.152:0/4118845548 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa99c0053c0 0x7fa99c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:14.326 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.323+0000 7fa9b6a00640 1 -- 172.21.15.152:0/4118845548 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa99c003a70 msgr2=0x7fa99c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:14.326 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.323+0000 7fa9b6a00640 1 --2- 172.21.15.152:0/4118845548 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa99c003a70 0x7fa99c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:14.326 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.323+0000 7fa9b6a00640 1 -- 172.21.15.152:0/4118845548 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa99c003520 con 0x7fa99c004800 2024-04-20T11:32:14.326 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.324+0000 7fa9b7e00640 1 -- 172.21.15.152:0/4118845548 reap_dead start 2024-04-20T11:32:14.326 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.325+0000 7fa9b6a00640 1 --2- 172.21.15.152:0/4118845548 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa99c004800 0x7fa99c004c80 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fa9a4003db0 tx=0x7fa99c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=befec0a29239e939 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:14.327 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.325+0000 7fa9b6000640 1 -- 172.21.15.152:0/4118845548 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a4037070 con 0x7fa99c004800 2024-04-20T11:32:14.327 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.325+0000 7fa9b6000640 1 -- 172.21.15.152:0/4118845548 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9a402be50 con 0x7fa99c004800 2024-04-20T11:32:14.327 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.325+0000 7fa9b6000640 1 -- 172.21.15.152:0/4118845548 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a4003980 con 0x7fa99c004800 2024-04-20T11:32:14.327 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.325+0000 7fa9be600640 1 -- 172.21.15.152:0/4118845548 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa99c004800 msgr2=0x7fa99c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:14.327 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.325+0000 7fa9be600640 1 --2- 172.21.15.152:0/4118845548 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa99c004800 0x7fa99c004c80 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fa9a4003db0 tx=0x7fa99c005830 comp rx=0 tx=0).stop 2024-04-20T11:32:14.327 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.326+0000 7fa9be600640 1 -- 172.21.15.152:0/4118845548 shutdown_connections 2024-04-20T11:32:14.327 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.326+0000 7fa9be600640 1 --2- 172.21.15.152:0/4118845548 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa99c004800 0x7fa99c004c80 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:14.327 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.326+0000 7fa9be600640 1 -- 172.21.15.152:0/4118845548 >> 172.21.15.152:0/4118845548 conn(0x7fa9b8067ae0 msgr2=0x7fa9b8067ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:14.327 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.326+0000 7fa9be600640 1 -- 172.21.15.152:0/4118845548 shutdown_connections 2024-04-20T11:32:14.327 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.326+0000 7fa9be600640 1 -- 172.21.15.152:0/4118845548 wait complete. 2024-04-20T11:32:14.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.327+0000 7fa9be600640 1 Processor -- start 2024-04-20T11:32:14.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.327+0000 7fa9be600640 1 -- start start 2024-04-20T11:32:14.329 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.328+0000 7fa9be600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa99c004800 0x7fa9b815b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:14.329 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.328+0000 7fa9be600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9b8154d30 0x7fa9b8157990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:14.329 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.328+0000 7fa9be600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa9b8152a70 0x7fa9b8152ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:14.329 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.328+0000 7fa9be600640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b80d1c40 con 0x7fa9b8152a70 2024-04-20T11:32:14.329 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.328+0000 7fa9be600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b80b8bd0 con 0x7fa9b8154d30 2024-04-20T11:32:14.329 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.328+0000 7fa9be600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa9b80d1dc0 con 0x7fa99c004800 2024-04-20T11:32:14.329 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.329+0000 7fa9b7e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa9b8152a70 0x7fa9b8152ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:14.330 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.329+0000 7fa9b7e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa9b8152a70 0x7fa9b8152ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36776/0 (socket says 172.21.15.152:36776) 2024-04-20T11:32:14.330 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.329+0000 7fa9b7e00640 1 -- 172.21.15.152:0/2365028530 learned_addr learned my addr 172.21.15.152:0/2365028530 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:32:14.331 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.329+0000 7fa9b7400640 1 --2- 172.21.15.152:0/2365028530 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa99c004800 0x7fa9b815b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:14.331 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.329+0000 7fa9b7e00640 1 -- 172.21.15.152:0/2365028530 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa99c004800 msgr2=0x7fa9b815b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:14.331 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.329+0000 7fa9b7e00640 1 --2- 172.21.15.152:0/2365028530 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa99c004800 0x7fa9b815b220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:14.331 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.329+0000 7fa9b7e00640 1 -- 172.21.15.152:0/2365028530 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9b8154d30 msgr2=0x7fa9b8157990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:14.331 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.329+0000 7fa9b7e00640 1 --2- 172.21.15.152:0/2365028530 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9b8154d30 0x7fa9b8157990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:14.331 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.329+0000 7fa9b7e00640 1 -- 172.21.15.152:0/2365028530 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9b81558c0 con 0x7fa9b8152a70 2024-04-20T11:32:14.331 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.330+0000 7fa9b7e00640 1 --2- 172.21.15.152:0/2365028530 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa9b8152a70 0x7fa9b8152ef0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fa9b80b6820 tx=0x7fa9b000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:14.332 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.330+0000 7fa9af400640 1 -- 172.21.15.152:0/2365028530 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b000ed90 con 0x7fa9b8152a70 2024-04-20T11:32:14.332 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.330+0000 7fa9af400640 1 -- 172.21.15.152:0/2365028530 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9b0019070 con 0x7fa9b8152a70 2024-04-20T11:32:14.332 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.330+0000 7fa9af400640 1 -- 172.21.15.152:0/2365028530 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b0014660 con 0x7fa9b8152a70 2024-04-20T11:32:14.332 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.330+0000 7fa9be600640 1 -- 172.21.15.152:0/2365028530 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9b8155ba0 con 0x7fa9b8152a70 2024-04-20T11:32:14.332 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.331+0000 7fa9be600640 1 -- 172.21.15.152:0/2365028530 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9b81551b0 con 0x7fa9b8152a70 2024-04-20T11:32:14.332 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.332+0000 7fa9be600640 1 -- 172.21.15.152:0/2365028530 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9b8073be0 con 0x7fa9b8152a70 2024-04-20T11:32:14.333 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.333+0000 7fa9af400640 1 -- 172.21.15.152:0/2365028530 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 11) ==== 91346+0+0 (secure 0 0 0) 0x7fa9b0014800 con 0x7fa9b8152a70 2024-04-20T11:32:14.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.334+0000 7fa9af400640 1 --2- 172.21.15.152:0/2365028530 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fa99806cc50 0x7fa99806f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:14.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.334+0000 7fa9af400640 1 -- 172.21.15.152:0/2365028530 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5709+0+0 (secure 0 0 0) 0x7fa9b008c390 con 0x7fa9b8152a70 2024-04-20T11:32:14.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.334+0000 7fa9b7400640 1 --2- 172.21.15.152:0/2365028530 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fa99806cc50 0x7fa99806f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:14.335 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.335+0000 7fa9b7400640 1 --2- 172.21.15.152:0/2365028530 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fa99806cc50 0x7fa99806f110 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fa9a80079b0 tx=0x7fa9a8009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:14.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.339+0000 7fa9af400640 1 -- 172.21.15.152:0/2365028530 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fa9b8073be0 con 0x7fa9b8152a70 2024-04-20T11:32:14.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.543+0000 7fa9be600640 1 -- 172.21.15.152:0/2365028530 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa9b80689f0 con 0x7fa9b8152a70 2024-04-20T11:32:14.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.546+0000 7fa9af400640 1 -- 172.21.15.152:0/2365028530 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) ==== 74+0+16728 (secure 0 0 0) 0x7fa9b0023020 con 0x7fa9b8152a70 2024-04-20T11:32:14.547 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:32:14.547 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":71,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:31:51.301720+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:31:48.024528+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-20T11:31:48.725648+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:32:14.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.549+0000 7fa9be600640 1 -- 172.21.15.152:0/2365028530 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fa99806cc50 msgr2=0x7fa99806f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:14.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.549+0000 7fa9be600640 1 --2- 172.21.15.152:0/2365028530 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fa99806cc50 0x7fa99806f110 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fa9a80079b0 tx=0x7fa9a8009040 comp rx=0 tx=0).stop 2024-04-20T11:32:14.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.549+0000 7fa9be600640 1 -- 172.21.15.152:0/2365028530 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa9b8152a70 msgr2=0x7fa9b8152ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:14.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.549+0000 7fa9be600640 1 --2- 172.21.15.152:0/2365028530 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa9b8152a70 0x7fa9b8152ef0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fa9b80b6820 tx=0x7fa9b000e4b0 comp rx=0 tx=0).stop 2024-04-20T11:32:14.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.549+0000 7fa9be600640 1 -- 172.21.15.152:0/2365028530 shutdown_connections 2024-04-20T11:32:14.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.549+0000 7fa9be600640 1 --2- 172.21.15.152:0/2365028530 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fa99806cc50 0x7fa99806f110 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:14.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.549+0000 7fa9be600640 1 --2- 172.21.15.152:0/2365028530 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa9b8152a70 0x7fa9b8152ef0 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:14.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.549+0000 7fa9be600640 1 --2- 172.21.15.152:0/2365028530 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9b8154d30 0x7fa9b8157990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:14.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.549+0000 7fa9be600640 1 --2- 172.21.15.152:0/2365028530 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa99c004800 0x7fa9b815b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:14.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.549+0000 7fa9be600640 1 -- 172.21.15.152:0/2365028530 >> 172.21.15.152:0/2365028530 conn(0x7fa9b8067ae0 msgr2=0x7fa9b80b9b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:14.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.549+0000 7fa9be600640 1 -- 172.21.15.152:0/2365028530 shutdown_connections 2024-04-20T11:32:14.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.549+0000 7fa9be600640 1 -- 172.21.15.152:0/2365028530 wait complete. 2024-04-20T11:32:14.569 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T11:26:47.246571+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T11:31:48.024528+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '70', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T11:31:48.725648+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '71', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T11:32:14.569 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T11:32:14.770 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.769+0000 7f9e45800640 1 Processor -- start 2024-04-20T11:32:14.770 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.769+0000 7f9e45800640 1 -- start start 2024-04-20T11:32:14.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.771+0000 7f9e45800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9e400ba430 0x7f9e400b7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:14.776 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.771+0000 7f9e45800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f9e400b7cc0 0x7f9e400d1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:14.776 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.771+0000 7f9e45800640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e4015ec70 con 0x7f9e4015e890 2024-04-20T11:32:14.776 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.771+0000 7f9e45800640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f9e400b8e20 con 0x7f9e400b7cc0 2024-04-20T11:32:14.776 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.771+0000 7f9e45800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f9e400d2ef0 con 0x7f9e400ba430 2024-04-20T11:32:14.776 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.771+0000 7f9e3ea00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f9e4015e890 0x7f9e400d27e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:36636/0 (socket says 172.21.15.152:36636) 2024-04-20T11:32:14.776 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.771+0000 7f9e3ea00640 1 -- 172.21.15.152:0/694947887 learned_addr learned my addr 172.21.15.152:0/694947887 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:32:14.776 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.771+0000 7f9e3f400640 1 --2- 172.21.15.152:0/694947887 >> v2:172.21.15.152:3300/0 conn(0x7f9e400b7cc0 0x7f9e400d1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:14.776 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.772+0000 7f9e3e000640 1 --2- 172.21.15.152:0/694947887 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9e400ba430 0x7f9e400b7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:14.776 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.772+0000 7f9e3d600640 1 -- 172.21.15.152:0/694947887 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3254833594 0 0) 0x7f9e4015ec70 con 0x7f9e4015e890 2024-04-20T11:32:14.776 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.772+0000 7f9e3d600640 1 -- 172.21.15.152:0/694947887 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9e20003520 con 0x7f9e4015e890 2024-04-20T11:32:14.776 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.772+0000 7f9e3f400640 1 -- 172.21.15.152:0/694947887 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9e400ba430 msgr2=0x7f9e400b7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:14.776 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.772+0000 7f9e3f400640 1 --2- 172.21.15.152:0/694947887 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9e400ba430 0x7f9e400b7780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:14.777 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.772+0000 7f9e3f400640 1 -- 172.21.15.152:0/694947887 >> v1:172.21.15.152:6789/0 conn(0x7f9e4015e890 legacy=0x7f9e400d27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:14.777 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.772+0000 7f9e3f400640 1 -- 172.21.15.152:0/694947887 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e400b8210 con 0x7f9e400b7cc0 2024-04-20T11:32:14.777 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.773+0000 7f9e3f400640 1 --2- 172.21.15.152:0/694947887 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9e400b7cc0 0x7f9e400d1790 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f9e380097a0 tx=0x7f9e3802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dd48d71c3d34dbd server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:14.777 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.773+0000 7f9e3d600640 1 -- 172.21.15.152:0/694947887 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e38034070 con 0x7f9e400b7cc0 2024-04-20T11:32:14.777 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.773+0000 7f9e3d600640 1 -- 172.21.15.152:0/694947887 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e38030610 con 0x7f9e400b7cc0 2024-04-20T11:32:14.777 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.773+0000 7f9e3d600640 1 -- 172.21.15.152:0/694947887 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e380309e0 con 0x7f9e400b7cc0 2024-04-20T11:32:14.777 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.774+0000 7f9e45800640 1 -- 172.21.15.152:0/694947887 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9e400b7cc0 msgr2=0x7f9e400d1790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:14.777 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.774+0000 7f9e45800640 1 --2- 172.21.15.152:0/694947887 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9e400b7cc0 0x7f9e400d1790 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f9e380097a0 tx=0x7f9e3802b960 comp rx=0 tx=0).stop 2024-04-20T11:32:14.777 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.774+0000 7f9e45800640 1 -- 172.21.15.152:0/694947887 shutdown_connections 2024-04-20T11:32:14.777 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.774+0000 7f9e45800640 1 --2- 172.21.15.152:0/694947887 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9e400ba430 0x7f9e400b7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:14.777 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.774+0000 7f9e45800640 1 --2- 172.21.15.152:0/694947887 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9e400b7cc0 0x7f9e400d1790 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:14.777 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.774+0000 7f9e45800640 1 -- 172.21.15.152:0/694947887 >> 172.21.15.152:0/694947887 conn(0x7f9e40067ae0 msgr2=0x7f9e40067ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:14.777 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.774+0000 7f9e45800640 1 -- 172.21.15.152:0/694947887 shutdown_connections 2024-04-20T11:32:14.778 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.774+0000 7f9e45800640 1 -- 172.21.15.152:0/694947887 wait complete. 2024-04-20T11:32:14.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.776+0000 7f9e45800640 1 Processor -- start 2024-04-20T11:32:14.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.776+0000 7f9e45800640 1 -- start start 2024-04-20T11:32:14.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.777+0000 7f9e45800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9e400b7cc0 0x7f9e400c8140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:14.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.777+0000 7f9e45800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9e400ba430 0x7f9e400ce350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:14.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.777+0000 7f9e45800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e4015e890 0x7f9e400ca6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:14.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.777+0000 7f9e45800640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f9e400b8ab0 con 0x7f9e400b7cc0 2024-04-20T11:32:14.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.777+0000 7f9e45800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9e400b87d0 con 0x7f9e4015e890 2024-04-20T11:32:14.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.777+0000 7f9e45800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f9e400d1cd0 con 0x7f9e400ba430 2024-04-20T11:32:14.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.778+0000 7f9e3ea00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9e400b7cc0 0x7f9e400c8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:14.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.778+0000 7f9e3ea00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9e400b7cc0 0x7f9e400c8140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36790/0 (socket says 172.21.15.152:36790) 2024-04-20T11:32:14.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.778+0000 7f9e3ea00640 1 -- 172.21.15.152:0/2941294788 learned_addr learned my addr 172.21.15.152:0/2941294788 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:32:14.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.778+0000 7f9e3e000640 1 --2- 172.21.15.152:0/2941294788 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9e400ba430 0x7f9e400ce350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:14.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.778+0000 7f9e3f400640 1 --2- 172.21.15.152:0/2941294788 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e4015e890 0x7f9e400ca6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:14.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.778+0000 7f9e3ea00640 1 -- 172.21.15.152:0/2941294788 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9e400ba430 msgr2=0x7f9e400ce350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:14.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.778+0000 7f9e3ea00640 1 --2- 172.21.15.152:0/2941294788 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9e400ba430 0x7f9e400ce350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:14.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.778+0000 7f9e3ea00640 1 -- 172.21.15.152:0/2941294788 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e4015e890 msgr2=0x7f9e400ca6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:14.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.778+0000 7f9e3ea00640 1 --2- 172.21.15.152:0/2941294788 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e4015e890 0x7f9e400ca6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:14.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.778+0000 7f9e3ea00640 1 -- 172.21.15.152:0/2941294788 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e400c8800 con 0x7f9e400b7cc0 2024-04-20T11:32:14.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.778+0000 7f9e3e000640 1 --2- 172.21.15.152:0/2941294788 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9e400ba430 0x7f9e400ce350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:32:14.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.779+0000 7f9e3ea00640 1 --2- 172.21.15.152:0/2941294788 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9e400b7cc0 0x7f9e400c8140 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f9e3000c390 tx=0x7f9e3000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:14.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.779+0000 7f9e36a00640 1 -- 172.21.15.152:0/2941294788 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e30017070 con 0x7f9e400b7cc0 2024-04-20T11:32:14.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.779+0000 7f9e36a00640 1 -- 172.21.15.152:0/2941294788 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e30005ce0 con 0x7f9e400b7cc0 2024-04-20T11:32:14.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.779+0000 7f9e36a00640 1 -- 172.21.15.152:0/2941294788 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e300072f0 con 0x7f9e400b7cc0 2024-04-20T11:32:14.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.779+0000 7f9e45800640 1 -- 172.21.15.152:0/2941294788 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e400c8ae0 con 0x7f9e400b7cc0 2024-04-20T11:32:14.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.779+0000 7f9e45800640 1 -- 172.21.15.152:0/2941294788 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e400c90a0 con 0x7f9e400b7cc0 2024-04-20T11:32:14.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.780+0000 7f9e45800640 1 -- 172.21.15.152:0/2941294788 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e400689f0 con 0x7f9e400b7cc0 2024-04-20T11:32:14.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.781+0000 7f9e36a00640 1 -- 172.21.15.152:0/2941294788 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 11) ==== 91346+0+0 (secure 0 0 0) 0x7f9e3001a070 con 0x7f9e400b7cc0 2024-04-20T11:32:14.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.782+0000 7f9e36a00640 1 --2- 172.21.15.152:0/2941294788 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f9e1c06cc50 0x7f9e1c06f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:14.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.782+0000 7f9e36a00640 1 -- 172.21.15.152:0/2941294788 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5709+0+0 (secure 0 0 0) 0x7f9e3008b4e0 con 0x7f9e400b7cc0 2024-04-20T11:32:14.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.782+0000 7f9e3e000640 1 --2- 172.21.15.152:0/2941294788 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f9e1c06cc50 0x7f9e1c06f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:14.784 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.783+0000 7f9e3e000640 1 --2- 172.21.15.152:0/2941294788 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f9e1c06cc50 0x7f9e1c06f110 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f9e400cdac0 tx=0x7f9e280039d0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:14.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.786+0000 7f9e36a00640 1 -- 172.21.15.152:0/2941294788 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f9e30059500 con 0x7f9e400b7cc0 2024-04-20T11:32:14.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.991+0000 7f9e45800640 1 -- 172.21.15.152:0/2941294788 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9e400b4cd0 con 0x7f9e400b7cc0 2024-04-20T11:32:14.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.992+0000 7f9e36a00640 1 -- 172.21.15.152:0/2941294788 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v71) ==== 93+0+10 (secure 0 0 0) 0x7f9e30058e00 con 0x7f9e400b7cc0 2024-04-20T11:32:14.992 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 1 2024-04-20T11:32:14.995 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.994+0000 7f9e45800640 1 -- 172.21.15.152:0/2941294788 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f9e1c06cc50 msgr2=0x7f9e1c06f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:14.995 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.994+0000 7f9e45800640 1 --2- 172.21.15.152:0/2941294788 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f9e1c06cc50 0x7f9e1c06f110 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f9e400cdac0 tx=0x7f9e280039d0 comp rx=0 tx=0).stop 2024-04-20T11:32:14.995 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.994+0000 7f9e45800640 1 -- 172.21.15.152:0/2941294788 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9e400b7cc0 msgr2=0x7f9e400c8140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:14.995 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.994+0000 7f9e45800640 1 --2- 172.21.15.152:0/2941294788 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9e400b7cc0 0x7f9e400c8140 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f9e3000c390 tx=0x7f9e3000c850 comp rx=0 tx=0).stop 2024-04-20T11:32:14.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.994+0000 7f9e45800640 1 -- 172.21.15.152:0/2941294788 shutdown_connections 2024-04-20T11:32:14.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.994+0000 7f9e45800640 1 --2- 172.21.15.152:0/2941294788 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f9e1c06cc50 0x7f9e1c06f110 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:14.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.994+0000 7f9e45800640 1 --2- 172.21.15.152:0/2941294788 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e4015e890 0x7f9e400ca6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:14.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.994+0000 7f9e45800640 1 --2- 172.21.15.152:0/2941294788 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9e400ba430 0x7f9e400ce350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:14.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.994+0000 7f9e45800640 1 --2- 172.21.15.152:0/2941294788 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9e400b7cc0 0x7f9e400c8140 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:14.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.994+0000 7f9e45800640 1 -- 172.21.15.152:0/2941294788 >> 172.21.15.152:0/2941294788 conn(0x7f9e40067ae0 msgr2=0x7f9e400b5470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:14.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.995+0000 7f9e45800640 1 -- 172.21.15.152:0/2941294788 shutdown_connections 2024-04-20T11:32:14.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:14.995+0000 7f9e45800640 1 -- 172.21.15.152:0/2941294788 wait complete. 2024-04-20T11:32:15.015 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T11:32:15.208 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.207+0000 7f8b12600640 1 Processor -- start 2024-04-20T11:32:15.208 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.207+0000 7f8b12600640 1 -- start start 2024-04-20T11:32:15.210 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.209+0000 7f8b12600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8b0c071cd0 0x7f8b0c0720b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:15.210 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.209+0000 7f8b12600640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f8b0c160220 0x7f8b0c160680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:15.210 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.209+0000 7f8b12600640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b0c0725f0 con 0x7f8b0c06eb30 2024-04-20T11:32:15.210 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.209+0000 7f8b12600640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f8b0c13f8e0 con 0x7f8b0c160220 2024-04-20T11:32:15.211 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.209+0000 7f8b12600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8b0c160f90 con 0x7f8b0c071cd0 2024-04-20T11:32:15.211 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.210+0000 7f8b11200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8b0c071cd0 0x7f8b0c0720b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:15.211 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.210+0000 7f8b11200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8b0c071cd0 0x7f8b0c0720b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.152:46664/0 (socket says 172.21.15.152:46664) 2024-04-20T11:32:15.211 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.210+0000 7f8b0be00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f8b0c160220 0x7f8b0c160680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:15.211 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.210+0000 7f8b11200640 1 -- 172.21.15.152:0/98240929 learned_addr learned my addr 172.21.15.152:0/98240929 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:32:15.212 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.211+0000 7f8b0be00640 1 -- 172.21.15.152:0/98240929 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8b0c071cd0 msgr2=0x7f8b0c0720b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:15.212 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.211+0000 7f8b0be00640 1 --2- 172.21.15.152:0/98240929 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8b0c071cd0 0x7f8b0c0720b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:15.212 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.211+0000 7f8b0be00640 1 -- 172.21.15.152:0/98240929 >> v1:172.21.15.175:6789/0 conn(0x7f8b0c06eb30 legacy=0x7f8b0c06efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:15.212 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.211+0000 7f8b0be00640 1 -- 172.21.15.152:0/98240929 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b0c06dd90 con 0x7f8b0c160220 2024-04-20T11:32:15.212 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.211+0000 7f8b11200640 1 --2- 172.21.15.152:0/98240929 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8b0c071cd0 0x7f8b0c0720b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:32:15.214 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.212+0000 7f8b0be00640 1 --2- 172.21.15.152:0/98240929 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b0c160220 0x7f8b0c160680 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f8af80097a0 tx=0x7f8af802b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39690882d7b83d0e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:15.214 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.212+0000 7f8b0b400640 1 -- 172.21.15.152:0/98240929 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8af8031780 con 0x7f8b0c160220 2024-04-20T11:32:15.214 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.212+0000 7f8b0b400640 1 -- 172.21.15.152:0/98240929 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8af8037070 con 0x7f8b0c160220 2024-04-20T11:32:15.214 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.212+0000 7f8b0b400640 1 -- 172.21.15.152:0/98240929 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8af8031b40 con 0x7f8b0c160220 2024-04-20T11:32:15.214 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.213+0000 7f8b12600640 1 -- 172.21.15.152:0/98240929 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b0c160220 msgr2=0x7f8b0c160680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:15.214 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.213+0000 7f8b12600640 1 --2- 172.21.15.152:0/98240929 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b0c160220 0x7f8b0c160680 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f8af80097a0 tx=0x7f8af802b820 comp rx=0 tx=0).stop 2024-04-20T11:32:15.214 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.213+0000 7f8b12600640 1 -- 172.21.15.152:0/98240929 shutdown_connections 2024-04-20T11:32:15.215 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.213+0000 7f8b12600640 1 --2- 172.21.15.152:0/98240929 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b0c160220 0x7f8b0c160680 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:15.215 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.213+0000 7f8b12600640 1 --2- 172.21.15.152:0/98240929 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8b0c071cd0 0x7f8b0c0720b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:15.215 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.213+0000 7f8b12600640 1 -- 172.21.15.152:0/98240929 >> 172.21.15.152:0/98240929 conn(0x7f8b0c067ae0 msgr2=0x7f8b0c06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:15.215 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.213+0000 7f8b12600640 1 -- 172.21.15.152:0/98240929 shutdown_connections 2024-04-20T11:32:15.215 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.213+0000 7f8b12600640 1 -- 172.21.15.152:0/98240929 wait complete. 2024-04-20T11:32:15.215 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.215+0000 7f8b12600640 1 Processor -- start 2024-04-20T11:32:15.216 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.215+0000 7f8b12600640 1 -- start start 2024-04-20T11:32:15.217 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.216+0000 7f8b12600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b0c06eb30 0x7f8b0c0b3490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:15.217 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.216+0000 7f8b12600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8b0c071cd0 0x7f8b0c156830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:15.217 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.216+0000 7f8b12600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8b0c160220 0x7f8b0c0b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:15.217 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.216+0000 7f8b12600640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f8b0c06e630 con 0x7f8b0c071cd0 2024-04-20T11:32:15.217 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.216+0000 7f8b12600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8b0c06e350 con 0x7f8b0c06eb30 2024-04-20T11:32:15.217 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.216+0000 7f8b12600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8b0c13fb90 con 0x7f8b0c160220 2024-04-20T11:32:15.217 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.216+0000 7f8b0be00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8b0c071cd0 0x7f8b0c156830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:15.217 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.216+0000 7f8b0be00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8b0c071cd0 0x7f8b0c156830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36796/0 (socket says 172.21.15.152:36796) 2024-04-20T11:32:15.217 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.216+0000 7f8b0be00640 1 -- 172.21.15.152:0/2167533600 learned_addr learned my addr 172.21.15.152:0/2167533600 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:32:15.219 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.217+0000 7f8b11200640 1 --2- 172.21.15.152:0/2167533600 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b0c06eb30 0x7f8b0c0b3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:15.219 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.217+0000 7f8b11c00640 1 --2- 172.21.15.152:0/2167533600 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8b0c160220 0x7f8b0c0b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:15.219 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.217+0000 7f8b0be00640 1 -- 172.21.15.152:0/2167533600 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8b0c160220 msgr2=0x7f8b0c0b4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:15.219 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.217+0000 7f8b0be00640 1 --2- 172.21.15.152:0/2167533600 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8b0c160220 0x7f8b0c0b4d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:15.219 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.217+0000 7f8b0be00640 1 -- 172.21.15.152:0/2167533600 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b0c06eb30 msgr2=0x7f8b0c0b3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:15.219 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.217+0000 7f8b0be00640 1 --2- 172.21.15.152:0/2167533600 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b0c06eb30 0x7f8b0c0b3490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:15.219 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.217+0000 7f8b0be00640 1 -- 172.21.15.152:0/2167533600 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b0c1f7690 con 0x7f8b0c071cd0 2024-04-20T11:32:15.219 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.217+0000 7f8b0be00640 1 --2- 172.21.15.152:0/2167533600 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8b0c071cd0 0x7f8b0c156830 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f8af80098c0 tx=0x7f8af80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:15.219 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.218+0000 7f8b08c00640 1 -- 172.21.15.152:0/2167533600 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8af8032610 con 0x7f8b0c071cd0 2024-04-20T11:32:15.220 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.218+0000 7f8b08c00640 1 -- 172.21.15.152:0/2167533600 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8af80327d0 con 0x7f8b0c071cd0 2024-04-20T11:32:15.220 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.218+0000 7f8b08c00640 1 -- 172.21.15.152:0/2167533600 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8af8032bc0 con 0x7f8b0c071cd0 2024-04-20T11:32:15.220 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.218+0000 7f8b12600640 1 -- 172.21.15.152:0/2167533600 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b0c1f7970 con 0x7f8b0c071cd0 2024-04-20T11:32:15.220 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.218+0000 7f8b12600640 1 -- 172.21.15.152:0/2167533600 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b0c1f7f30 con 0x7f8b0c071cd0 2024-04-20T11:32:15.220 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.219+0000 7f8b12600640 1 -- 172.21.15.152:0/2167533600 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b0c075e60 con 0x7f8b0c071cd0 2024-04-20T11:32:15.220 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.220+0000 7f8b08c00640 1 -- 172.21.15.152:0/2167533600 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 11) ==== 91346+0+0 (secure 0 0 0) 0x7f8af804c060 con 0x7f8b0c071cd0 2024-04-20T11:32:15.221 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.221+0000 7f8b08c00640 1 --2- 172.21.15.152:0/2167533600 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f8aec06cc50 0x7f8aec06f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:15.221 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.221+0000 7f8b08c00640 1 -- 172.21.15.152:0/2167533600 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5709+0+0 (secure 0 0 0) 0x7f8af80adaf0 con 0x7f8b0c071cd0 2024-04-20T11:32:15.222 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.221+0000 7f8b11200640 1 --2- 172.21.15.152:0/2167533600 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f8aec06cc50 0x7f8aec06f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:15.222 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.222+0000 7f8b11200640 1 --2- 172.21.15.152:0/2167533600 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f8aec06cc50 0x7f8aec06f110 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f8b00009b70 tx=0x7f8b00008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:15.225 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.225+0000 7f8b08c00640 1 -- 172.21.15.152:0/2167533600 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f8af8040070 con 0x7f8b0c071cd0 2024-04-20T11:32:15.423 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.422+0000 7f8b12600640 1 -- 172.21.15.152:0/2167533600 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f8b0c06ccb0 con 0x7f8b0c071cd0 2024-04-20T11:32:15.423 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.423+0000 7f8b08c00640 1 -- 172.21.15.152:0/2167533600 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v71) ==== 104+0+11 (secure 0 0 0) 0x7f8af802f050 con 0x7f8b0c071cd0 2024-04-20T11:32:15.424 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 32 2024-04-20T11:32:15.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.425+0000 7f8b12600640 1 -- 172.21.15.152:0/2167533600 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f8aec06cc50 msgr2=0x7f8aec06f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:15.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.426+0000 7f8b12600640 1 --2- 172.21.15.152:0/2167533600 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f8aec06cc50 0x7f8aec06f110 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f8b00009b70 tx=0x7f8b00008040 comp rx=0 tx=0).stop 2024-04-20T11:32:15.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.426+0000 7f8b12600640 1 -- 172.21.15.152:0/2167533600 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8b0c071cd0 msgr2=0x7f8b0c156830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:15.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.426+0000 7f8b12600640 1 --2- 172.21.15.152:0/2167533600 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8b0c071cd0 0x7f8b0c156830 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f8af80098c0 tx=0x7f8af80046c0 comp rx=0 tx=0).stop 2024-04-20T11:32:15.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.426+0000 7f8b12600640 1 -- 172.21.15.152:0/2167533600 shutdown_connections 2024-04-20T11:32:15.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.426+0000 7f8b12600640 1 --2- 172.21.15.152:0/2167533600 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f8aec06cc50 0x7f8aec06f110 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:15.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.426+0000 7f8b12600640 1 --2- 172.21.15.152:0/2167533600 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8b0c160220 0x7f8b0c0b4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:15.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.426+0000 7f8b12600640 1 --2- 172.21.15.152:0/2167533600 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8b0c071cd0 0x7f8b0c156830 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:15.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.426+0000 7f8b12600640 1 --2- 172.21.15.152:0/2167533600 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b0c06eb30 0x7f8b0c0b3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:15.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.426+0000 7f8b12600640 1 -- 172.21.15.152:0/2167533600 >> 172.21.15.152:0/2167533600 conn(0x7f8b0c067ae0 msgr2=0x7f8b0c15d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:15.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.426+0000 7f8b12600640 1 -- 172.21.15.152:0/2167533600 shutdown_connections 2024-04-20T11:32:15.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.426+0000 7f8b12600640 1 -- 172.21.15.152:0/2167533600 wait complete. 2024-04-20T11:32:15.446 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T11:32:15.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.639+0000 7f185ae00640 1 Processor -- start 2024-04-20T11:32:15.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.639+0000 7f185ae00640 1 -- start start 2024-04-20T11:32:15.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.640+0000 7f185ae00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f185415ca40 0x7f185415cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:15.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.640+0000 7f185ae00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f185415d5c0 con 0x7f1854156c00 2024-04-20T11:32:15.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.640+0000 7f185ae00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f185415d8a0 con 0x7f185406bda0 2024-04-20T11:32:15.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.640+0000 7f185ae00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1854160290 con 0x7f185415ca40 2024-04-20T11:32:15.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.641+0000 7f1859a00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f1854156c00 0x7f1854160a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:36640/0 (socket says 172.21.15.152:36640) 2024-04-20T11:32:15.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.641+0000 7f1859a00640 1 -- 172.21.15.152:0/257428763 learned_addr learned my addr 172.21.15.152:0/257428763 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:32:15.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.641+0000 7f1853e00640 1 -- 172.21.15.152:0/257428763 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1440358694 0 0) 0x7f185415d5c0 con 0x7f1854156c00 2024-04-20T11:32:15.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.641+0000 7f1853e00640 1 -- 172.21.15.152:0/257428763 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f183c003520 con 0x7f1854156c00 2024-04-20T11:32:15.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.641+0000 7f1859000640 1 --2- 172.21.15.152:0/257428763 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f185415ca40 0x7f185415cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:15.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.642+0000 7f1853e00640 1 -- 172.21.15.152:0/257428763 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2474972921 0 0) 0x7f183c003520 con 0x7f1854156c00 2024-04-20T11:32:15.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.642+0000 7f1853e00640 1 -- 172.21.15.152:0/257428763 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f185415ca40 msgr2=0x7f185415cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:15.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.642+0000 7f1853e00640 1 --2- 172.21.15.152:0/257428763 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f185415ca40 0x7f185415cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:15.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.642+0000 7f1853e00640 1 -- 172.21.15.152:0/257428763 >> v1:172.21.15.175:6789/0 conn(0x7f185406bda0 legacy=0x7f18540b30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:15.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.642+0000 7f1853e00640 1 -- 172.21.15.152:0/257428763 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18540ad2e0 con 0x7f1854156c00 2024-04-20T11:32:15.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.642+0000 7f1853e00640 1 -- 172.21.15.152:0/257428763 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f1848001670 con 0x7f1854156c00 2024-04-20T11:32:15.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.642+0000 7f1853e00640 1 -- 172.21.15.152:0/257428763 >> v1:172.21.15.152:6789/0 conn(0x7f1854156c00 legacy=0x7f1854160a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:15.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.642+0000 7f1853e00640 1 --2- 172.21.15.152:0/257428763 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f183c0039e0 0x7f183c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:15.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.642+0000 7f1853e00640 1 --2- 172.21.15.152:0/257428763 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f185415ca40 0x7f183c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:15.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.642+0000 7f1853e00640 1 --2- 172.21.15.152:0/257428763 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f183c004d40 0x7f183c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:15.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.642+0000 7f1853e00640 1 -- 172.21.15.152:0/257428763 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f1854160290 con 0x7f183c0039e0 2024-04-20T11:32:15.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.642+0000 7f1853e00640 1 -- 172.21.15.152:0/257428763 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f183c00bb50 con 0x7f183c004d40 2024-04-20T11:32:15.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.642+0000 7f1853e00640 1 -- 172.21.15.152:0/257428763 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f183c00bd20 con 0x7f185415ca40 2024-04-20T11:32:15.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.642+0000 7f1859a00640 1 --2- 172.21.15.152:0/257428763 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f183c0039e0 0x7f183c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:15.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.642+0000 7f1859a00640 1 -- 172.21.15.152:0/257428763 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f185415ca40 msgr2=0x7f183c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:15.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.642+0000 7f1859a00640 1 --2- 172.21.15.152:0/257428763 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f185415ca40 0x7f183c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:15.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.642+0000 7f1859a00640 1 -- 172.21.15.152:0/257428763 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f183c004d40 msgr2=0x7f183c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:15.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.642+0000 7f1859a00640 1 --2- 172.21.15.152:0/257428763 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f183c004d40 0x7f183c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:15.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.642+0000 7f1859a00640 1 -- 172.21.15.152:0/257428763 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f183c003520 con 0x7f183c0039e0 2024-04-20T11:32:15.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.644+0000 7f1859a00640 1 --2- 172.21.15.152:0/257428763 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f183c0039e0 0x7f183c003e00 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f184800c240 tx=0x7f183c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=486c10f53beb96b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:15.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.644+0000 7f1853e00640 1 -- 172.21.15.152:0/257428763 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1848007d30 con 0x7f183c0039e0 2024-04-20T11:32:15.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.644+0000 7f1853e00640 1 -- 172.21.15.152:0/257428763 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1848030070 con 0x7f183c0039e0 2024-04-20T11:32:15.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.644+0000 7f1853e00640 1 -- 172.21.15.152:0/257428763 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f184802cd40 con 0x7f183c0039e0 2024-04-20T11:32:15.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.645+0000 7f185ae00640 1 -- 172.21.15.152:0/257428763 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f183c0039e0 msgr2=0x7f183c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:15.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.645+0000 7f185ae00640 1 --2- 172.21.15.152:0/257428763 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f183c0039e0 0x7f183c003e00 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f184800c240 tx=0x7f183c005830 comp rx=0 tx=0).stop 2024-04-20T11:32:15.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.645+0000 7f185a400640 1 -- 172.21.15.152:0/257428763 reap_dead start 2024-04-20T11:32:15.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.645+0000 7f185ae00640 1 -- 172.21.15.152:0/257428763 shutdown_connections 2024-04-20T11:32:15.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.645+0000 7f185ae00640 1 -- 172.21.15.152:0/257428763 >> 172.21.15.152:0/257428763 conn(0x7f18540b21e0 msgr2=0x7f18540b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:15.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.645+0000 7f185ae00640 1 -- 172.21.15.152:0/257428763 shutdown_connections 2024-04-20T11:32:15.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.645+0000 7f185ae00640 1 -- 172.21.15.152:0/257428763 wait complete. 2024-04-20T11:32:15.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.646+0000 7f185ae00640 1 Processor -- start 2024-04-20T11:32:15.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.647+0000 7f185ae00640 1 -- start start 2024-04-20T11:32:15.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.647+0000 7f185ae00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f183c0039e0 0x7f18540990a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:15.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.647+0000 7f185ae00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18541518c0 0x7f18541f5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:15.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.648+0000 7f185ae00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1854151d40 0x7f1854154e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:15.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.648+0000 7f185ae00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f185415d2d0 con 0x7f183c0039e0 2024-04-20T11:32:15.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.648+0000 7f185ae00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f185415d150 con 0x7f18541518c0 2024-04-20T11:32:15.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.648+0000 7f185ae00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f18540b37d0 con 0x7f1854151d40 2024-04-20T11:32:15.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.648+0000 7f1859a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f183c0039e0 0x7f18540990a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:15.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.648+0000 7f1859a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f183c0039e0 0x7f18540990a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36818/0 (socket says 172.21.15.152:36818) 2024-04-20T11:32:15.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.648+0000 7f1859a00640 1 -- 172.21.15.152:0/1879589190 learned_addr learned my addr 172.21.15.152:0/1879589190 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:32:15.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.648+0000 7f1859a00640 1 -- 172.21.15.152:0/1879589190 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1854151d40 msgr2=0x7f1854154e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:15.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.648+0000 7f1859a00640 1 --2- 172.21.15.152:0/1879589190 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1854151d40 0x7f1854154e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:15.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.648+0000 7f1859a00640 1 -- 172.21.15.152:0/1879589190 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18541518c0 msgr2=0x7f18541f5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:15.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.648+0000 7f1859a00640 1 --2- 172.21.15.152:0/1879589190 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18541518c0 0x7f18541f5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:15.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.648+0000 7f1859a00640 1 -- 172.21.15.152:0/1879589190 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18541f6450 con 0x7f183c0039e0 2024-04-20T11:32:15.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.649+0000 7f1859a00640 1 --2- 172.21.15.152:0/1879589190 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f183c0039e0 0x7f18540990a0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f1848001970 tx=0x7f1848001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:15.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.649+0000 7f1851600640 1 -- 172.21.15.152:0/1879589190 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1848004bf0 con 0x7f183c0039e0 2024-04-20T11:32:15.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.649+0000 7f1851600640 1 -- 172.21.15.152:0/1879589190 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1848004d90 con 0x7f183c0039e0 2024-04-20T11:32:15.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.649+0000 7f185ae00640 1 -- 172.21.15.152:0/1879589190 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18541f6730 con 0x7f183c0039e0 2024-04-20T11:32:15.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.649+0000 7f1851600640 1 -- 172.21.15.152:0/1879589190 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1848007270 con 0x7f183c0039e0 2024-04-20T11:32:15.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.649+0000 7f185ae00640 1 -- 172.21.15.152:0/1879589190 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18540995e0 con 0x7f183c0039e0 2024-04-20T11:32:15.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.650+0000 7f185ae00640 1 -- 172.21.15.152:0/1879589190 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1854075e60 con 0x7f183c0039e0 2024-04-20T11:32:15.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.651+0000 7f1851600640 1 -- 172.21.15.152:0/1879589190 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 11) ==== 91346+0+0 (secure 0 0 0) 0x7f1848048020 con 0x7f183c0039e0 2024-04-20T11:32:15.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.652+0000 7f1851600640 1 --2- 172.21.15.152:0/1879589190 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f183406cc80 0x7f183406f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:15.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.652+0000 7f1851600640 1 -- 172.21.15.152:0/1879589190 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5709+0+0 (secure 0 0 0) 0x7f18480ad4c0 con 0x7f183c0039e0 2024-04-20T11:32:15.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.652+0000 7f1859000640 1 --2- 172.21.15.152:0/1879589190 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f183406cc80 0x7f183406f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:15.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.653+0000 7f1859000640 1 --2- 172.21.15.152:0/1879589190 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f183406cc80 0x7f183406f140 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f1840006ce0 tx=0x7f1840008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:15.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.657+0000 7f1851600640 1 -- 172.21.15.152:0/1879589190 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f184807b530 con 0x7f183c0039e0 2024-04-20T11:32:15.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.838+0000 7f185ae00640 1 -- 172.21.15.152:0/1879589190 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f185406f390 con 0x7f183c0039e0 2024-04-20T11:32:15.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.840+0000 7f1851600640 1 -- 172.21.15.152:0/1879589190 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v71) ==== 100+0+11 (secure 0 0 0) 0x7f184807ae30 con 0x7f183c0039e0 2024-04-20T11:32:15.840 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 64 2024-04-20T11:32:15.843 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.842+0000 7f185ae00640 1 -- 172.21.15.152:0/1879589190 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f183406cc80 msgr2=0x7f183406f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:15.843 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.842+0000 7f185ae00640 1 --2- 172.21.15.152:0/1879589190 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f183406cc80 0x7f183406f140 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f1840006ce0 tx=0x7f1840008040 comp rx=0 tx=0).stop 2024-04-20T11:32:15.843 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.842+0000 7f185ae00640 1 -- 172.21.15.152:0/1879589190 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f183c0039e0 msgr2=0x7f18540990a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:15.844 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.842+0000 7f185ae00640 1 --2- 172.21.15.152:0/1879589190 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f183c0039e0 0x7f18540990a0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f1848001970 tx=0x7f1848001a40 comp rx=0 tx=0).stop 2024-04-20T11:32:15.844 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.842+0000 7f185ae00640 1 -- 172.21.15.152:0/1879589190 shutdown_connections 2024-04-20T11:32:15.844 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.842+0000 7f185ae00640 1 --2- 172.21.15.152:0/1879589190 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f183406cc80 0x7f183406f140 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:15.844 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.842+0000 7f185ae00640 1 --2- 172.21.15.152:0/1879589190 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1854151d40 0x7f1854154e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:15.844 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.842+0000 7f185ae00640 1 --2- 172.21.15.152:0/1879589190 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18541518c0 0x7f18541f5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:15.844 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.842+0000 7f185ae00640 1 --2- 172.21.15.152:0/1879589190 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f183c0039e0 0x7f18540990a0 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:15.844 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.842+0000 7f185ae00640 1 -- 172.21.15.152:0/1879589190 >> 172.21.15.152:0/1879589190 conn(0x7f18540b21e0 msgr2=0x7f18540add10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:15.844 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.843+0000 7f185ae00640 1 -- 172.21.15.152:0/1879589190 shutdown_connections 2024-04-20T11:32:15.844 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:15.843+0000 7f185ae00640 1 -- 172.21.15.152:0/1879589190 wait complete. 2024-04-20T11:32:15.861 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-20T11:32:16.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.050+0000 7f2a3c400640 1 Processor -- start 2024-04-20T11:32:16.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.050+0000 7f2a3c400640 1 -- start start 2024-04-20T11:32:16.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.052+0000 7f2a3c400640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f2a34145f00 0x7f2a340cd0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:16.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.052+0000 7f2a3c400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f2a340cd610 0x7f2a3406e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:16.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.052+0000 7f2a3c400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a3406e900 0x7f2a341601c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:16.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.052+0000 7f2a3c400640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f2a340cda80 con 0x7f2a340cd610 2024-04-20T11:32:16.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.052+0000 7f2a3c400640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f2a340cf9f0 con 0x7f2a34145f00 2024-04-20T11:32:16.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.052+0000 7f2a3c400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2a3406f110 con 0x7f2a3406e900 2024-04-20T11:32:16.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.052+0000 7f2a39000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f2a340cd610 0x7f2a3406e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:16.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.052+0000 7f2a39000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f2a340cd610 0x7f2a3406e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:58400/0 (socket says 172.21.15.152:58400) 2024-04-20T11:32:16.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.052+0000 7f2a39000640 1 -- 172.21.15.152:0/4078322424 learned_addr learned my addr 172.21.15.152:0/4078322424 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:32:16.054 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.052+0000 7f2a39000640 1 -- 172.21.15.152:0/4078322424 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a3406e900 msgr2=0x7f2a341601c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:16.054 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.052+0000 7f2a3a400640 1 --2- 172.21.15.152:0/4078322424 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a3406e900 0x7f2a341601c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:16.054 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.052+0000 7f2a39a00640 1 --2- 172.21.15.152:0/4078322424 >> v2:172.21.15.175:3300/0 conn(0x7f2a34145f00 0x7f2a340cd0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:16.054 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.052+0000 7f2a39000640 1 --2- 172.21.15.152:0/4078322424 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a3406e900 0x7f2a341601c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:16.054 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.052+0000 7f2a39000640 1 -- 172.21.15.152:0/4078322424 >> v2:172.21.15.175:3300/0 conn(0x7f2a34145f00 msgr2=0x7f2a340cd0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:16.054 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.052+0000 7f2a39000640 1 --2- 172.21.15.152:0/4078322424 >> v2:172.21.15.175:3300/0 conn(0x7f2a34145f00 0x7f2a340cd0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:16.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.052+0000 7f2a39000640 1 -- 172.21.15.152:0/4078322424 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a3406f3f0 con 0x7f2a340cd610 2024-04-20T11:32:16.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.052+0000 7f2a3a400640 1 --2- 172.21.15.152:0/4078322424 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a3406e900 0x7f2a341601c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:32:16.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.052+0000 7f2a39a00640 1 --2- 172.21.15.152:0/4078322424 >> v2:172.21.15.175:3300/0 conn(0x7f2a34145f00 0x7f2a340cd0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:32:16.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.054+0000 7f2a39000640 1 --2- 172.21.15.152:0/4078322424 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2a340cd610 0x7f2a3406e3c0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f2a3406c840 tx=0x7f2a2002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5843af6c26569c00 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:16.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.054+0000 7f2a33e00640 1 -- 172.21.15.152:0/4078322424 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a20035020 con 0x7f2a340cd610 2024-04-20T11:32:16.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.054+0000 7f2a33e00640 1 -- 172.21.15.152:0/4078322424 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a20031870 con 0x7f2a340cd610 2024-04-20T11:32:16.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.054+0000 7f2a33e00640 1 -- 172.21.15.152:0/4078322424 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a20031c40 con 0x7f2a340cd610 2024-04-20T11:32:16.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.054+0000 7f2a3c400640 1 -- 172.21.15.152:0/4078322424 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2a340cd610 msgr2=0x7f2a3406e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:16.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.054+0000 7f2a3c400640 1 --2- 172.21.15.152:0/4078322424 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2a340cd610 0x7f2a3406e3c0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f2a3406c840 tx=0x7f2a2002b910 comp rx=0 tx=0).stop 2024-04-20T11:32:16.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.055+0000 7f2a3c400640 1 -- 172.21.15.152:0/4078322424 shutdown_connections 2024-04-20T11:32:16.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.055+0000 7f2a3c400640 1 --2- 172.21.15.152:0/4078322424 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a3406e900 0x7f2a341601c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:16.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.055+0000 7f2a3c400640 1 --2- 172.21.15.152:0/4078322424 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2a340cd610 0x7f2a3406e3c0 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:16.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.055+0000 7f2a3c400640 1 --2- 172.21.15.152:0/4078322424 >> v2:172.21.15.175:3300/0 conn(0x7f2a34145f00 0x7f2a340cd0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:16.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.055+0000 7f2a3c400640 1 -- 172.21.15.152:0/4078322424 >> 172.21.15.152:0/4078322424 conn(0x7f2a34067cf0 msgr2=0x7f2a3406b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:16.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.055+0000 7f2a3c400640 1 -- 172.21.15.152:0/4078322424 shutdown_connections 2024-04-20T11:32:16.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.055+0000 7f2a3c400640 1 -- 172.21.15.152:0/4078322424 wait complete. 2024-04-20T11:32:16.057 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.056+0000 7f2a3c400640 1 Processor -- start 2024-04-20T11:32:16.057 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.057+0000 7f2a3c400640 1 -- start start 2024-04-20T11:32:16.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.057+0000 7f2a3c400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2a3406e900 0x7f2a341f4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:16.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.057+0000 7f2a3c400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a340cd610 0x7f2a341f7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:16.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.058+0000 7f2a3c400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a34145f00 0x7f2a3413cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:16.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.058+0000 7f2a3c400640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f2a341462e0 con 0x7f2a3406e900 2024-04-20T11:32:16.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.058+0000 7f2a3c400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2a340cf680 con 0x7f2a34145f00 2024-04-20T11:32:16.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.058+0000 7f2a3c400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2a3406ed90 con 0x7f2a340cd610 2024-04-20T11:32:16.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.058+0000 7f2a39a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2a3406e900 0x7f2a341f4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:16.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.058+0000 7f2a39a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2a3406e900 0x7f2a341f4dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:58416/0 (socket says 172.21.15.152:58416) 2024-04-20T11:32:16.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.058+0000 7f2a39a00640 1 -- 172.21.15.152:0/1540550267 learned_addr learned my addr 172.21.15.152:0/1540550267 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:32:16.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.058+0000 7f2a39a00640 1 -- 172.21.15.152:0/1540550267 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a340cd610 msgr2=0x7f2a341f7bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:16.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.058+0000 7f2a39a00640 1 --2- 172.21.15.152:0/1540550267 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a340cd610 0x7f2a341f7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:16.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.058+0000 7f2a39a00640 1 -- 172.21.15.152:0/1540550267 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a34145f00 msgr2=0x7f2a3413cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:16.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.058+0000 7f2a39a00640 1 --2- 172.21.15.152:0/1540550267 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a34145f00 0x7f2a3413cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:16.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.058+0000 7f2a39a00640 1 -- 172.21.15.152:0/1540550267 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a341f5310 con 0x7f2a3406e900 2024-04-20T11:32:16.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.058+0000 7f2a39a00640 1 --2- 172.21.15.152:0/1540550267 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2a3406e900 0x7f2a341f4dd0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f2a2800c7f0 tx=0x7f2a2800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:16.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.059+0000 7f2a31600640 1 -- 172.21.15.152:0/1540550267 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a28015070 con 0x7f2a3406e900 2024-04-20T11:32:16.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.059+0000 7f2a31600640 1 -- 172.21.15.152:0/1540550267 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a28007820 con 0x7f2a3406e900 2024-04-20T11:32:16.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.059+0000 7f2a31600640 1 -- 172.21.15.152:0/1540550267 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a28007bd0 con 0x7f2a3406e900 2024-04-20T11:32:16.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.059+0000 7f2a3c400640 1 -- 172.21.15.152:0/1540550267 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a3413a680 con 0x7f2a3406e900 2024-04-20T11:32:16.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.059+0000 7f2a3c400640 1 -- 172.21.15.152:0/1540550267 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a340b38f0 con 0x7f2a3406e900 2024-04-20T11:32:16.061 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.060+0000 7f2a3c400640 1 -- 172.21.15.152:0/1540550267 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a3413a960 con 0x7f2a3406e900 2024-04-20T11:32:16.061 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.061+0000 7f2a31600640 1 -- 172.21.15.152:0/1540550267 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 11) ==== 91346+0+0 (secure 0 0 0) 0x7f2a28018070 con 0x7f2a3406e900 2024-04-20T11:32:16.062 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.062+0000 7f2a31600640 1 --2- 172.21.15.152:0/1540550267 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f2a1406cc50 0x7f2a1406f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:16.063 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.062+0000 7f2a31600640 1 -- 172.21.15.152:0/1540550267 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5709+0+0 (secure 0 0 0) 0x7f2a2808c260 con 0x7f2a3406e900 2024-04-20T11:32:16.063 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.062+0000 7f2a39000640 1 --2- 172.21.15.152:0/1540550267 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f2a1406cc50 0x7f2a1406f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:16.063 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.063+0000 7f2a39000640 1 --2- 172.21.15.152:0/1540550267 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f2a1406cc50 0x7f2a1406f110 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f2a20036000 tx=0x7f2a20030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:16.067 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.067+0000 7f2a31600640 1 -- 172.21.15.152:0/1540550267 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f2a3413a960 con 0x7f2a3406e900 2024-04-20T11:32:16.263 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.263+0000 7f2a3c400640 1 -- 172.21.15.152:0/1540550267 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f2a340b3d10 con 0x7f2a3406e900 2024-04-20T11:32:16.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.264+0000 7f2a31600640 1 -- 172.21.15.152:0/1540550267 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v71) ==== 80+0+0 (secure 0 0 0) 0x7f2a3413a680 con 0x7f2a3406e900 2024-04-20T11:32:16.264 INFO:teuthology.orchestra.run.smithi152.stderr:listed 0 entries 2024-04-20T11:32:16.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.266+0000 7f2a3c400640 1 -- 172.21.15.152:0/1540550267 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f2a1406cc50 msgr2=0x7f2a1406f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:16.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.266+0000 7f2a3c400640 1 --2- 172.21.15.152:0/1540550267 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f2a1406cc50 0x7f2a1406f110 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f2a20036000 tx=0x7f2a20030000 comp rx=0 tx=0).stop 2024-04-20T11:32:16.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.266+0000 7f2a3c400640 1 -- 172.21.15.152:0/1540550267 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2a3406e900 msgr2=0x7f2a341f4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:16.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.266+0000 7f2a3c400640 1 --2- 172.21.15.152:0/1540550267 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2a3406e900 0x7f2a341f4dd0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f2a2800c7f0 tx=0x7f2a2800ccb0 comp rx=0 tx=0).stop 2024-04-20T11:32:16.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.266+0000 7f2a3c400640 1 -- 172.21.15.152:0/1540550267 shutdown_connections 2024-04-20T11:32:16.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.266+0000 7f2a3c400640 1 --2- 172.21.15.152:0/1540550267 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f2a1406cc50 0x7f2a1406f110 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:16.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.266+0000 7f2a3c400640 1 --2- 172.21.15.152:0/1540550267 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a34145f00 0x7f2a3413cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:16.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.266+0000 7f2a3c400640 1 --2- 172.21.15.152:0/1540550267 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a340cd610 0x7f2a341f7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:16.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.266+0000 7f2a3c400640 1 --2- 172.21.15.152:0/1540550267 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2a3406e900 0x7f2a341f4dd0 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:16.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.266+0000 7f2a3c400640 1 -- 172.21.15.152:0/1540550267 >> 172.21.15.152:0/1540550267 conn(0x7f2a34067cf0 msgr2=0x7f2a3415d7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:16.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.267+0000 7f2a3c400640 1 -- 172.21.15.152:0/1540550267 shutdown_connections 2024-04-20T11:32:16.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.267+0000 7f2a3c400640 1 -- 172.21.15.152:0/1540550267 wait complete. 2024-04-20T11:32:16.285 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T11:32:16.285 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:32:16.344 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:32:16.541 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.541+0000 7fa147e00640 1 Processor -- start 2024-04-20T11:32:16.542 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.541+0000 7fa147e00640 1 -- start start 2024-04-20T11:32:16.543 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.542+0000 7fa147e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa14806f510 0x7fa148140dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:16.543 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.542+0000 7fa147e00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1480cf0a0 con 0x7fa148141310 2024-04-20T11:32:16.543 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.542+0000 7fa147e00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1480d00e0 con 0x7fa14813e530 2024-04-20T11:32:16.543 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.542+0000 7fa147e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa148141b00 con 0x7fa14806f510 2024-04-20T11:32:16.543 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.543+0000 7fa147400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fa148141310 0x7fa148071cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:50266/0 (socket says 172.21.15.152:50266) 2024-04-20T11:32:16.543 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.543+0000 7fa147400640 1 -- 172.21.15.152:0/3818527617 learned_addr learned my addr 172.21.15.152:0/3818527617 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:32:16.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.543+0000 7fa13fe00640 1 --2- 172.21.15.152:0/3818527617 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa14806f510 0x7fa148140dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:16.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.543+0000 7fa146000640 1 -- 172.21.15.152:0/3818527617 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2765731639 0 0) 0x7fa140001490 con 0x7fa148141310 2024-04-20T11:32:16.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.543+0000 7fa146000640 1 -- 172.21.15.152:0/3818527617 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa12c005730 con 0x7fa148141310 2024-04-20T11:32:16.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.544+0000 7fa146000640 1 -- 172.21.15.152:0/3818527617 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3838895948 0 0) 0x7fa12c005730 con 0x7fa148141310 2024-04-20T11:32:16.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.544+0000 7fa146000640 1 -- 172.21.15.152:0/3818527617 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa14806f510 msgr2=0x7fa148140dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:16.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.544+0000 7fa146000640 1 --2- 172.21.15.152:0/3818527617 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa14806f510 0x7fa148140dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:16.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.544+0000 7fa146000640 1 -- 172.21.15.152:0/3818527617 >> v1:172.21.15.175:6789/0 conn(0x7fa14813e530 legacy=0x7fa14806ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:16.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.544+0000 7fa146000640 1 -- 172.21.15.152:0/3818527617 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1480cf320 con 0x7fa148141310 2024-04-20T11:32:16.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.544+0000 7fa146000640 1 -- 172.21.15.152:0/3818527617 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7fa140001bc0 con 0x7fa148141310 2024-04-20T11:32:16.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.544+0000 7fa146000640 1 -- 172.21.15.152:0/3818527617 >> v1:172.21.15.152:6789/0 conn(0x7fa148141310 legacy=0x7fa148071cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:16.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.544+0000 7fa146000640 1 --2- 172.21.15.152:0/3818527617 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa14806f510 0x7fa12c005bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:16.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.544+0000 7fa146000640 1 --2- 172.21.15.152:0/3818527617 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa12c0061c0 0x7fa12c006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:16.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.544+0000 7fa146000640 1 --2- 172.21.15.152:0/3818527617 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa12c006f50 0x7fa12c0073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:16.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.544+0000 7fa146000640 1 -- 172.21.15.152:0/3818527617 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fa12c00dce0 con 0x7fa12c006f50 2024-04-20T11:32:16.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.544+0000 7fa146000640 1 -- 172.21.15.152:0/3818527617 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa12c00de90 con 0x7fa12c0061c0 2024-04-20T11:32:16.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.544+0000 7fa146000640 1 -- 172.21.15.152:0/3818527617 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa12c00e090 con 0x7fa14806f510 2024-04-20T11:32:16.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.544+0000 7fa147400640 1 --2- 172.21.15.152:0/3818527617 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa12c006f50 0x7fa12c0073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:16.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.544+0000 7fa146a00640 1 --2- 172.21.15.152:0/3818527617 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa14806f510 0x7fa12c005bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:16.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.545+0000 7fa13fe00640 1 --2- 172.21.15.152:0/3818527617 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa12c0061c0 0x7fa12c006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:16.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.545+0000 7fa147400640 1 -- 172.21.15.152:0/3818527617 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa14806f510 msgr2=0x7fa12c005bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:16.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.545+0000 7fa147400640 1 --2- 172.21.15.152:0/3818527617 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa14806f510 0x7fa12c005bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:16.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.545+0000 7fa147400640 1 -- 172.21.15.152:0/3818527617 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa12c0061c0 msgr2=0x7fa12c006a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:16.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.545+0000 7fa147400640 1 --2- 172.21.15.152:0/3818527617 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa12c0061c0 0x7fa12c006a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:16.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.545+0000 7fa147400640 1 -- 172.21.15.152:0/3818527617 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa12c005730 con 0x7fa12c006f50 2024-04-20T11:32:16.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.545+0000 7fa13fe00640 1 --2- 172.21.15.152:0/3818527617 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa12c0061c0 0x7fa12c006a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:32:16.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.546+0000 7fa147400640 1 --2- 172.21.15.152:0/3818527617 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa12c006f50 0x7fa12c0073d0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fa14000a950 tx=0x7fa12c007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cd76cc9d2f298b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:16.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.546+0000 7fa146000640 1 -- 172.21.15.152:0/3818527617 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa14000a570 con 0x7fa12c006f50 2024-04-20T11:32:16.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.546+0000 7fa146000640 1 -- 172.21.15.152:0/3818527617 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa140007310 con 0x7fa12c006f50 2024-04-20T11:32:16.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.546+0000 7fa146000640 1 -- 172.21.15.152:0/3818527617 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa140007740 con 0x7fa12c006f50 2024-04-20T11:32:16.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.547+0000 7fa147e00640 1 -- 172.21.15.152:0/3818527617 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa12c006f50 msgr2=0x7fa12c0073d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:16.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.547+0000 7fa147e00640 1 --2- 172.21.15.152:0/3818527617 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa12c006f50 0x7fa12c0073d0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fa14000a950 tx=0x7fa12c007a40 comp rx=0 tx=0).stop 2024-04-20T11:32:16.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.547+0000 7fa147400640 1 -- 172.21.15.152:0/3818527617 reap_dead start 2024-04-20T11:32:16.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.547+0000 7fa147e00640 1 -- 172.21.15.152:0/3818527617 shutdown_connections 2024-04-20T11:32:16.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.547+0000 7fa147e00640 1 -- 172.21.15.152:0/3818527617 >> 172.21.15.152:0/3818527617 conn(0x7fa148067ae0 msgr2=0x7fa14806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:16.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.547+0000 7fa147e00640 1 -- 172.21.15.152:0/3818527617 shutdown_connections 2024-04-20T11:32:16.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.547+0000 7fa147e00640 1 -- 172.21.15.152:0/3818527617 wait complete. 2024-04-20T11:32:16.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.549+0000 7fa147e00640 1 Processor -- start 2024-04-20T11:32:16.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.549+0000 7fa147e00640 1 -- start start 2024-04-20T11:32:16.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.550+0000 7fa147e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa14813fbd0 0x7fa148158530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:16.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.550+0000 7fa147e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa148158a70 0x7fa1481f3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:16.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.550+0000 7fa147e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa14813ffb0 0x7fa1481f4510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:16.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.550+0000 7fa147e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fa1480cf8e0 con 0x7fa14813ffb0 2024-04-20T11:32:16.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.550+0000 7fa147e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa1480cfc50 con 0x7fa14813fbd0 2024-04-20T11:32:16.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.550+0000 7fa147e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa1481417c0 con 0x7fa148158a70 2024-04-20T11:32:16.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.550+0000 7fa147400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa14813ffb0 0x7fa1481f4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:16.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.550+0000 7fa147400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa14813ffb0 0x7fa1481f4510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:58448/0 (socket says 172.21.15.152:58448) 2024-04-20T11:32:16.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.550+0000 7fa147400640 1 -- 172.21.15.152:0/615583864 learned_addr learned my addr 172.21.15.152:0/615583864 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:32:16.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.550+0000 7fa13fe00640 1 --2- 172.21.15.152:0/615583864 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa148158a70 0x7fa1481f3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:16.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.550+0000 7fa146a00640 1 --2- 172.21.15.152:0/615583864 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa14813fbd0 0x7fa148158530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:16.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.550+0000 7fa147400640 1 -- 172.21.15.152:0/615583864 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa148158a70 msgr2=0x7fa1481f3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:16.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.551+0000 7fa147400640 1 --2- 172.21.15.152:0/615583864 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa148158a70 0x7fa1481f3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:16.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.551+0000 7fa147400640 1 -- 172.21.15.152:0/615583864 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa14813fbd0 msgr2=0x7fa148158530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:16.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.551+0000 7fa147400640 1 --2- 172.21.15.152:0/615583864 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa14813fbd0 0x7fa148158530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:16.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.551+0000 7fa147400640 1 -- 172.21.15.152:0/615583864 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1481f4a50 con 0x7fa14813ffb0 2024-04-20T11:32:16.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.551+0000 7fa147400640 1 --2- 172.21.15.152:0/615583864 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa14813ffb0 0x7fa1481f4510 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fa14002cd60 tx=0x7fa14815b9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:16.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.551+0000 7fa13f400640 1 -- 172.21.15.152:0/615583864 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa140045070 con 0x7fa14813ffb0 2024-04-20T11:32:16.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.551+0000 7fa13f400640 1 -- 172.21.15.152:0/615583864 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa14003a040 con 0x7fa14813ffb0 2024-04-20T11:32:16.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.551+0000 7fa13f400640 1 -- 172.21.15.152:0/615583864 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa14000a570 con 0x7fa14813ffb0 2024-04-20T11:32:16.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.551+0000 7fa147e00640 1 -- 172.21.15.152:0/615583864 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa148156260 con 0x7fa14813ffb0 2024-04-20T11:32:16.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.551+0000 7fa147e00640 1 -- 172.21.15.152:0/615583864 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa148156820 con 0x7fa14813ffb0 2024-04-20T11:32:16.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.553+0000 7fa147e00640 1 -- 172.21.15.152:0/615583864 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa148075e60 con 0x7fa14813ffb0 2024-04-20T11:32:16.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.554+0000 7fa13f400640 1 -- 172.21.15.152:0/615583864 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 11) ==== 91346+0+0 (secure 0 0 0) 0x7fa140048070 con 0x7fa14813ffb0 2024-04-20T11:32:16.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.554+0000 7fa13f400640 1 --2- 172.21.15.152:0/615583864 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fa12406cc80 0x7fa12406f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:16.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.554+0000 7fa13f400640 1 -- 172.21.15.152:0/615583864 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5709+0+0 (secure 0 0 0) 0x7fa1400adc20 con 0x7fa14813ffb0 2024-04-20T11:32:16.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.555+0000 7fa146a00640 1 --2- 172.21.15.152:0/615583864 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fa12406cc80 0x7fa12406f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:16.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.555+0000 7fa146a00640 1 --2- 172.21.15.152:0/615583864 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fa12406cc80 0x7fa12406f140 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fa138007fd0 tx=0x7fa138009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:16.559 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.559+0000 7fa13f400640 1 -- 172.21.15.152:0/615583864 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fa1480cf0a0 con 0x7fa14813ffb0 2024-04-20T11:32:16.746 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.745+0000 7fa147e00640 1 -- 172.21.15.152:0/615583864 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa1480b3490 con 0x7fa14813ffb0 2024-04-20T11:32:16.748 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.747+0000 7fa13f400640 1 -- 172.21.15.152:0/615583864 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) ==== 74+0+16728 (secure 0 0 0) 0x7fa14007bb40 con 0x7fa14813ffb0 2024-04-20T11:32:16.748 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:32:16.749 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":71,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:31:51.301720+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:31:48.024528+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-20T11:31:48.725648+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:32:16.751 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.750+0000 7fa147e00640 1 -- 172.21.15.152:0/615583864 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fa12406cc80 msgr2=0x7fa12406f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:16.751 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.750+0000 7fa147e00640 1 --2- 172.21.15.152:0/615583864 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fa12406cc80 0x7fa12406f140 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fa138007fd0 tx=0x7fa138009040 comp rx=0 tx=0).stop 2024-04-20T11:32:16.751 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.750+0000 7fa147e00640 1 -- 172.21.15.152:0/615583864 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa14813ffb0 msgr2=0x7fa1481f4510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:16.751 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.750+0000 7fa147e00640 1 --2- 172.21.15.152:0/615583864 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa14813ffb0 0x7fa1481f4510 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fa14002cd60 tx=0x7fa14815b9e0 comp rx=0 tx=0).stop 2024-04-20T11:32:16.752 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.751+0000 7fa147e00640 1 -- 172.21.15.152:0/615583864 shutdown_connections 2024-04-20T11:32:16.752 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.751+0000 7fa147e00640 1 --2- 172.21.15.152:0/615583864 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fa12406cc80 0x7fa12406f140 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:16.752 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.751+0000 7fa147e00640 1 --2- 172.21.15.152:0/615583864 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa14813ffb0 0x7fa1481f4510 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:16.752 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.751+0000 7fa147e00640 1 --2- 172.21.15.152:0/615583864 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa148158a70 0x7fa1481f3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:16.752 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.751+0000 7fa147e00640 1 --2- 172.21.15.152:0/615583864 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa14813fbd0 0x7fa148158530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:16.752 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.751+0000 7fa147e00640 1 -- 172.21.15.152:0/615583864 >> 172.21.15.152:0/615583864 conn(0x7fa148067ae0 msgr2=0x7fa148159570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:16.752 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.751+0000 7fa147e00640 1 -- 172.21.15.152:0/615583864 shutdown_connections 2024-04-20T11:32:16.752 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.751+0000 7fa147e00640 1 -- 172.21.15.152:0/615583864 wait complete. 2024-04-20T11:32:16.770 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T11:26:47.246571+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T11:31:48.024528+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '70', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T11:31:48.725648+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '71', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T11:32:16.770 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T11:32:16.964 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.963+0000 7f01cd200640 1 Processor -- start 2024-04-20T11:32:16.964 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.963+0000 7f01cd200640 1 -- start start 2024-04-20T11:32:16.966 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.965+0000 7f01cd200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f01c815ca40 0x7f01c815cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:16.966 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.965+0000 7f01cd200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01c806bda0 0x7f01c806c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:16.966 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.965+0000 7f01cd200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01c815d5c0 con 0x7f01c8156c00 2024-04-20T11:32:16.966 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.965+0000 7f01cd200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f01c815d8a0 con 0x7f01c815ca40 2024-04-20T11:32:16.966 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.965+0000 7f01cd200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f01c8160290 con 0x7f01c806bda0 2024-04-20T11:32:16.966 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.965+0000 7f01c6a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f01c815ca40 0x7f01c815cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:16.966 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.965+0000 7f01c6a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f01c815ca40 0x7f01c815cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:58460/0 (socket says 172.21.15.152:58460) 2024-04-20T11:32:16.966 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.965+0000 7f01c6a00640 1 -- 172.21.15.152:0/3766466938 learned_addr learned my addr 172.21.15.152:0/3766466938 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:32:16.967 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.966+0000 7f01c7e00640 1 --2- 172.21.15.152:0/3766466938 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01c806bda0 0x7f01c806c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:16.967 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.966+0000 7f01c6a00640 1 -- 172.21.15.152:0/3766466938 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01c806bda0 msgr2=0x7f01c806c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:16.967 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.966+0000 7f01c6a00640 1 --2- 172.21.15.152:0/3766466938 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01c806bda0 0x7f01c806c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:16.967 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.966+0000 7f01c6a00640 1 -- 172.21.15.152:0/3766466938 >> v1:172.21.15.175:6789/0 conn(0x7f01c8156c00 legacy=0x7f01c8160a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:16.967 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.966+0000 7f01c6a00640 1 -- 172.21.15.152:0/3766466938 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01c80ad2e0 con 0x7f01c815ca40 2024-04-20T11:32:16.968 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.966+0000 7f01c7e00640 1 --2- 172.21.15.152:0/3766466938 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01c806bda0 0x7f01c806c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:32:16.968 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.967+0000 7f01c6a00640 1 --2- 172.21.15.152:0/3766466938 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f01c815ca40 0x7f01c815cec0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f01b4009870 tx=0x7f01b402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5eb3f2a17561ba5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:16.969 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.967+0000 7f01c6000640 1 -- 172.21.15.152:0/3766466938 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b4035020 con 0x7f01c815ca40 2024-04-20T11:32:16.969 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.967+0000 7f01c6000640 1 -- 172.21.15.152:0/3766466938 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01b4031640 con 0x7f01c815ca40 2024-04-20T11:32:16.969 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.967+0000 7f01c6000640 1 -- 172.21.15.152:0/3766466938 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b402d040 con 0x7f01c815ca40 2024-04-20T11:32:16.969 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.968+0000 7f01cd200640 1 -- 172.21.15.152:0/3766466938 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f01c815ca40 msgr2=0x7f01c815cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:16.969 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.968+0000 7f01cd200640 1 --2- 172.21.15.152:0/3766466938 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f01c815ca40 0x7f01c815cec0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f01b4009870 tx=0x7f01b402b8f0 comp rx=0 tx=0).stop 2024-04-20T11:32:16.969 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.968+0000 7f01cd200640 1 -- 172.21.15.152:0/3766466938 shutdown_connections 2024-04-20T11:32:16.969 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.968+0000 7f01cd200640 1 --2- 172.21.15.152:0/3766466938 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01c806bda0 0x7f01c806c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:16.969 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.968+0000 7f01cd200640 1 --2- 172.21.15.152:0/3766466938 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f01c815ca40 0x7f01c815cec0 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:16.969 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.968+0000 7f01cd200640 1 -- 172.21.15.152:0/3766466938 >> 172.21.15.152:0/3766466938 conn(0x7f01c80b21e0 msgr2=0x7f01c80b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:16.969 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.968+0000 7f01cd200640 1 -- 172.21.15.152:0/3766466938 shutdown_connections 2024-04-20T11:32:16.970 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.968+0000 7f01cd200640 1 -- 172.21.15.152:0/3766466938 wait complete. 2024-04-20T11:32:16.970 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.970+0000 7f01cd200640 1 Processor -- start 2024-04-20T11:32:16.970 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.970+0000 7f01cd200640 1 -- start start 2024-04-20T11:32:16.972 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.970+0000 7f01cd200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01c806bda0 0x7f01c81545f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:16.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.971+0000 7f01cd200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f01c8156c00 0x7f01c8152350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:16.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.971+0000 7f01cd200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01c815ca40 0x7f01c814e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:16.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.971+0000 7f01cd200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f01c80b3490 con 0x7f01c8156c00 2024-04-20T11:32:16.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.971+0000 7f01cd200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f01c80b3310 con 0x7f01c815ca40 2024-04-20T11:32:16.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.971+0000 7f01cd200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f01c80b3610 con 0x7f01c806bda0 2024-04-20T11:32:16.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.971+0000 7f01c6a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f01c8156c00 0x7f01c8152350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:16.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.971+0000 7f01c6a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f01c8156c00 0x7f01c8152350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:58474/0 (socket says 172.21.15.152:58474) 2024-04-20T11:32:16.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.971+0000 7f01c6a00640 1 -- 172.21.15.152:0/2644546878 learned_addr learned my addr 172.21.15.152:0/2644546878 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:32:16.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.971+0000 7f01c7400640 1 --2- 172.21.15.152:0/2644546878 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01c806bda0 0x7f01c81545f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:16.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.971+0000 7f01c6a00640 1 -- 172.21.15.152:0/2644546878 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01c806bda0 msgr2=0x7f01c81545f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:16.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.971+0000 7f01c6a00640 1 --2- 172.21.15.152:0/2644546878 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01c806bda0 0x7f01c81545f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:16.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.971+0000 7f01c6a00640 1 -- 172.21.15.152:0/2644546878 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01c815ca40 msgr2=0x7f01c814e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:16.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.971+0000 7f01c7e00640 1 --2- 172.21.15.152:0/2644546878 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01c815ca40 0x7f01c814e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:16.974 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.971+0000 7f01c6a00640 1 --2- 172.21.15.152:0/2644546878 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01c815ca40 0x7f01c814e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:16.974 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.971+0000 7f01c6a00640 1 -- 172.21.15.152:0/2644546878 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01c814f950 con 0x7f01c8156c00 2024-04-20T11:32:16.974 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.972+0000 7f01c6a00640 1 --2- 172.21.15.152:0/2644546878 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f01c8156c00 0x7f01c8152350 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f01b4000c00 tx=0x7f01b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:16.974 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.972+0000 7f01bf400640 1 -- 172.21.15.152:0/2644546878 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b4040070 con 0x7f01c8156c00 2024-04-20T11:32:16.974 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.972+0000 7f01bf400640 1 -- 172.21.15.152:0/2644546878 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01b4039c10 con 0x7f01c8156c00 2024-04-20T11:32:16.974 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.972+0000 7f01bf400640 1 -- 172.21.15.152:0/2644546878 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b4035040 con 0x7f01c8156c00 2024-04-20T11:32:16.974 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.972+0000 7f01cd200640 1 -- 172.21.15.152:0/2644546878 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01c814fee0 con 0x7f01c8156c00 2024-04-20T11:32:16.974 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.972+0000 7f01cd200640 1 -- 172.21.15.152:0/2644546878 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01c8152db0 con 0x7f01c8156c00 2024-04-20T11:32:16.975 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.973+0000 7f01cd200640 1 -- 172.21.15.152:0/2644546878 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01c8075e60 con 0x7f01c8156c00 2024-04-20T11:32:16.975 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.974+0000 7f01bf400640 1 -- 172.21.15.152:0/2644546878 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 11) ==== 91346+0+0 (secure 0 0 0) 0x7f01b404b060 con 0x7f01c8156c00 2024-04-20T11:32:16.975 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.975+0000 7f01bf400640 1 --2- 172.21.15.152:0/2644546878 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f01a806cc50 0x7f01a806f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:16.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.975+0000 7f01bf400640 1 -- 172.21.15.152:0/2644546878 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5709+0+0 (secure 0 0 0) 0x7f01b40adab0 con 0x7f01c8156c00 2024-04-20T11:32:16.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.975+0000 7f01c7400640 1 --2- 172.21.15.152:0/2644546878 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f01a806cc50 0x7f01a806f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:16.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.976+0000 7f01c7400640 1 --2- 172.21.15.152:0/2644546878 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f01a806cc50 0x7f01a806f110 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f01b8003d90 tx=0x7f01b8009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:16.980 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:16.979+0000 7f01bf400640 1 -- 172.21.15.152:0/2644546878 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f01b4031610 con 0x7f01c8156c00 2024-04-20T11:32:17.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.167+0000 7f01cd200640 1 -- 172.21.15.152:0/2644546878 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f01c806f390 con 0x7f01c8156c00 2024-04-20T11:32:17.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.168+0000 7f01bf400640 1 -- 172.21.15.152:0/2644546878 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v71) ==== 93+0+10 (secure 0 0 0) 0x7f01b407bb60 con 0x7f01c8156c00 2024-04-20T11:32:17.168 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 1 2024-04-20T11:32:17.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.170+0000 7f01cd200640 1 -- 172.21.15.152:0/2644546878 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f01a806cc50 msgr2=0x7f01a806f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:17.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.170+0000 7f01cd200640 1 --2- 172.21.15.152:0/2644546878 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f01a806cc50 0x7f01a806f110 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f01b8003d90 tx=0x7f01b8009040 comp rx=0 tx=0).stop 2024-04-20T11:32:17.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.170+0000 7f01cd200640 1 -- 172.21.15.152:0/2644546878 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f01c8156c00 msgr2=0x7f01c8152350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:17.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.170+0000 7f01cd200640 1 --2- 172.21.15.152:0/2644546878 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f01c8156c00 0x7f01c8152350 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f01b4000c00 tx=0x7f01b40046c0 comp rx=0 tx=0).stop 2024-04-20T11:32:17.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.170+0000 7f01cd200640 1 -- 172.21.15.152:0/2644546878 shutdown_connections 2024-04-20T11:32:17.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.170+0000 7f01cd200640 1 --2- 172.21.15.152:0/2644546878 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f01a806cc50 0x7f01a806f110 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:17.172 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.170+0000 7f01cd200640 1 --2- 172.21.15.152:0/2644546878 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01c815ca40 0x7f01c814e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:17.172 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.170+0000 7f01cd200640 1 --2- 172.21.15.152:0/2644546878 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f01c8156c00 0x7f01c8152350 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:17.172 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.170+0000 7f01cd200640 1 --2- 172.21.15.152:0/2644546878 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01c806bda0 0x7f01c81545f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:17.172 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.170+0000 7f01cd200640 1 -- 172.21.15.152:0/2644546878 >> 172.21.15.152:0/2644546878 conn(0x7f01c80b21e0 msgr2=0x7f01c815a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:17.172 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.170+0000 7f01cd200640 1 -- 172.21.15.152:0/2644546878 shutdown_connections 2024-04-20T11:32:17.172 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.170+0000 7f01cd200640 1 -- 172.21.15.152:0/2644546878 wait complete. 2024-04-20T11:32:17.190 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T11:32:17.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.391+0000 7ff8cb200640 1 Processor -- start 2024-04-20T11:32:17.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.391+0000 7ff8cb200640 1 -- start start 2024-04-20T11:32:17.393 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.393+0000 7ff8cb200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ff8cc1480b0 0x7ff8cc15ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:17.394 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.393+0000 7ff8cb200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7ff8cc15f1f0 0x7ff8cc06f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:17.394 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.393+0000 7ff8cb200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8cc15f7e0 con 0x7ff8cc147cd0 2024-04-20T11:32:17.394 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.393+0000 7ff8cb200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7ff8cc06ee10 con 0x7ff8cc15f1f0 2024-04-20T11:32:17.394 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.393+0000 7ff8cb200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7ff8cc15fac0 con 0x7ff8cc1480b0 2024-04-20T11:32:17.395 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.394+0000 7ff8ca800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7ff8cc15f1f0 0x7ff8cc06f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:17.395 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.394+0000 7ff8ca800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7ff8cc15f1f0 0x7ff8cc06f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:58482/0 (socket says 172.21.15.152:58482) 2024-04-20T11:32:17.395 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.394+0000 7ff8ca800640 1 -- 172.21.15.152:0/309711030 learned_addr learned my addr 172.21.15.152:0/309711030 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:32:17.395 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.394+0000 7ff8c3e00640 1 --2- 172.21.15.152:0/309711030 >> v2:172.21.15.175:3300/0 conn(0x7ff8cc1480b0 0x7ff8cc15ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:17.395 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.394+0000 7ff8ca800640 1 -- 172.21.15.152:0/309711030 >> v2:172.21.15.175:3300/0 conn(0x7ff8cc1480b0 msgr2=0x7ff8cc15ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:17.395 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.394+0000 7ff8ca800640 1 --2- 172.21.15.152:0/309711030 >> v2:172.21.15.175:3300/0 conn(0x7ff8cc1480b0 0x7ff8cc15ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:17.395 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.394+0000 7ff8ca800640 1 -- 172.21.15.152:0/309711030 >> v1:172.21.15.175:6789/0 conn(0x7ff8cc147cd0 legacy=0x7ff8cc06e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:17.395 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.394+0000 7ff8ca800640 1 -- 172.21.15.152:0/309711030 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8cc06df00 con 0x7ff8cc15f1f0 2024-04-20T11:32:17.396 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.395+0000 7ff8ca800640 1 --2- 172.21.15.152:0/309711030 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff8cc15f1f0 0x7ff8cc06f1c0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc06fd60 tx=0x7ff8c402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d6d7e38a93f1e52 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:17.396 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.396+0000 7ff8c9400640 1 -- 172.21.15.152:0/309711030 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8c402ca80 con 0x7ff8cc15f1f0 2024-04-20T11:32:17.396 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.396+0000 7ff8c9400640 1 -- 172.21.15.152:0/309711030 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8c402cc20 con 0x7ff8cc15f1f0 2024-04-20T11:32:17.396 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.396+0000 7ff8c9400640 1 -- 172.21.15.152:0/309711030 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8c40343f0 con 0x7ff8cc15f1f0 2024-04-20T11:32:17.396 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.396+0000 7ff8cb200640 1 -- 172.21.15.152:0/309711030 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff8cc15f1f0 msgr2=0x7ff8cc06f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:17.397 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.396+0000 7ff8cb200640 1 --2- 172.21.15.152:0/309711030 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff8cc15f1f0 0x7ff8cc06f1c0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc06fd60 tx=0x7ff8c402bee0 comp rx=0 tx=0).stop 2024-04-20T11:32:17.397 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.396+0000 7ff8cb200640 1 -- 172.21.15.152:0/309711030 shutdown_connections 2024-04-20T11:32:17.397 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.396+0000 7ff8cb200640 1 --2- 172.21.15.152:0/309711030 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff8cc15f1f0 0x7ff8cc06f1c0 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:17.397 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.396+0000 7ff8cb200640 1 --2- 172.21.15.152:0/309711030 >> v2:172.21.15.175:3300/0 conn(0x7ff8cc1480b0 0x7ff8cc15ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:17.397 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.396+0000 7ff8cb200640 1 -- 172.21.15.152:0/309711030 >> 172.21.15.152:0/309711030 conn(0x7ff8cc067ae0 msgr2=0x7ff8cc06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:17.397 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.396+0000 7ff8cb200640 1 -- 172.21.15.152:0/309711030 shutdown_connections 2024-04-20T11:32:17.397 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.396+0000 7ff8cb200640 1 -- 172.21.15.152:0/309711030 wait complete. 2024-04-20T11:32:17.398 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.398+0000 7ff8cb200640 1 Processor -- start 2024-04-20T11:32:17.398 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.398+0000 7ff8cb200640 1 -- start start 2024-04-20T11:32:17.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.399+0000 7ff8cb200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff8cc147cd0 0x7ff8cc15bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:17.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.399+0000 7ff8cb200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff8cc1480b0 0x7ff8cc0b08d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:17.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.399+0000 7ff8cb200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff8cc15f1f0 0x7ff8cc0b0e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:17.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.399+0000 7ff8cb200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7ff8cc06f700 con 0x7ff8cc147cd0 2024-04-20T11:32:17.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.399+0000 7ff8cb200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff8cc071ed0 con 0x7ff8cc1480b0 2024-04-20T11:32:17.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.399+0000 7ff8cb200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff8cc06f880 con 0x7ff8cc15f1f0 2024-04-20T11:32:17.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.399+0000 7ff8c9e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff8cc147cd0 0x7ff8cc15bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:17.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.399+0000 7ff8c9e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff8cc147cd0 0x7ff8cc15bf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:58498/0 (socket says 172.21.15.152:58498) 2024-04-20T11:32:17.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.399+0000 7ff8c9e00640 1 -- 172.21.15.152:0/460486823 learned_addr learned my addr 172.21.15.152:0/460486823 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:32:17.401 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.400+0000 7ff8c9e00640 1 -- 172.21.15.152:0/460486823 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff8cc15f1f0 msgr2=0x7ff8cc0b0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:17.402 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.400+0000 7ff8c9e00640 1 --2- 172.21.15.152:0/460486823 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff8cc15f1f0 0x7ff8cc0b0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:17.402 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.400+0000 7ff8c9e00640 1 -- 172.21.15.152:0/460486823 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff8cc1480b0 msgr2=0x7ff8cc0b08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:17.402 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.400+0000 7ff8c3e00640 1 --2- 172.21.15.152:0/460486823 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff8cc1480b0 0x7ff8cc0b08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:17.402 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.400+0000 7ff8c9e00640 1 --2- 172.21.15.152:0/460486823 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff8cc1480b0 0x7ff8cc0b08d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:17.402 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.400+0000 7ff8c9e00640 1 -- 172.21.15.152:0/460486823 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8cc0af790 con 0x7ff8cc147cd0 2024-04-20T11:32:17.402 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.400+0000 7ff8c9e00640 1 --2- 172.21.15.152:0/460486823 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff8cc147cd0 0x7ff8cc15bf50 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc00c460 tx=0x7ff8bc00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:17.402 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.400+0000 7ff8c2000640 1 -- 172.21.15.152:0/460486823 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8bc016020 con 0x7ff8cc147cd0 2024-04-20T11:32:17.402 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.400+0000 7ff8c2000640 1 -- 172.21.15.152:0/460486823 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8bc012870 con 0x7ff8cc147cd0 2024-04-20T11:32:17.402 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.400+0000 7ff8c2000640 1 -- 172.21.15.152:0/460486823 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8bc012c60 con 0x7ff8cc147cd0 2024-04-20T11:32:17.402 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.400+0000 7ff8cb200640 1 -- 172.21.15.152:0/460486823 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8cc0afa70 con 0x7ff8cc147cd0 2024-04-20T11:32:17.402 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.401+0000 7ff8cb200640 1 -- 172.21.15.152:0/460486823 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8cc0b0030 con 0x7ff8cc147cd0 2024-04-20T11:32:17.403 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.402+0000 7ff8cb200640 1 -- 172.21.15.152:0/460486823 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8cc075e60 con 0x7ff8cc147cd0 2024-04-20T11:32:17.403 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.403+0000 7ff8c2000640 1 -- 172.21.15.152:0/460486823 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 11) ==== 91346+0+0 (secure 0 0 0) 0x7ff8bc019020 con 0x7ff8cc147cd0 2024-04-20T11:32:17.404 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.404+0000 7ff8c2000640 1 --2- 172.21.15.152:0/460486823 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff8a806cc50 0x7ff8a806f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:17.404 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.404+0000 7ff8c2000640 1 -- 172.21.15.152:0/460486823 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5709+0+0 (secure 0 0 0) 0x7ff8bc08bde0 con 0x7ff8cc147cd0 2024-04-20T11:32:17.404 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.404+0000 7ff8c3e00640 1 --2- 172.21.15.152:0/460486823 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff8a806cc50 0x7ff8a806f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:17.405 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.405+0000 7ff8c3e00640 1 --2- 172.21.15.152:0/460486823 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff8a806cc50 0x7ff8a806f110 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc1473c0 tx=0x7ff8b4008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:17.408 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.408+0000 7ff8c2000640 1 -- 172.21.15.152:0/460486823 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ff8bc020030 con 0x7ff8cc147cd0 2024-04-20T11:32:17.607 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.606+0000 7ff8cb200640 1 -- 172.21.15.152:0/460486823 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff8cc0b3120 con 0x7ff8cc147cd0 2024-04-20T11:32:17.607 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.607+0000 7ff8c2000640 1 -- 172.21.15.152:0/460486823 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v71) ==== 104+0+11 (secure 0 0 0) 0x7ff8bc019310 con 0x7ff8cc147cd0 2024-04-20T11:32:17.607 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 32 2024-04-20T11:32:17.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.609+0000 7ff8cb200640 1 -- 172.21.15.152:0/460486823 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff8a806cc50 msgr2=0x7ff8a806f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:17.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.609+0000 7ff8cb200640 1 --2- 172.21.15.152:0/460486823 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff8a806cc50 0x7ff8a806f110 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc1473c0 tx=0x7ff8b4008040 comp rx=0 tx=0).stop 2024-04-20T11:32:17.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.609+0000 7ff8cb200640 1 -- 172.21.15.152:0/460486823 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff8cc147cd0 msgr2=0x7ff8cc15bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:17.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.609+0000 7ff8cb200640 1 --2- 172.21.15.152:0/460486823 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff8cc147cd0 0x7ff8cc15bf50 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc00c460 tx=0x7ff8bc00c920 comp rx=0 tx=0).stop 2024-04-20T11:32:17.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.609+0000 7ff8cb200640 1 -- 172.21.15.152:0/460486823 shutdown_connections 2024-04-20T11:32:17.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.609+0000 7ff8cb200640 1 --2- 172.21.15.152:0/460486823 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7ff8a806cc50 0x7ff8a806f110 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:17.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.609+0000 7ff8cb200640 1 --2- 172.21.15.152:0/460486823 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff8cc15f1f0 0x7ff8cc0b0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:17.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.609+0000 7ff8cb200640 1 --2- 172.21.15.152:0/460486823 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff8cc1480b0 0x7ff8cc0b08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:17.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.609+0000 7ff8cb200640 1 --2- 172.21.15.152:0/460486823 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff8cc147cd0 0x7ff8cc15bf50 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:17.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.609+0000 7ff8cb200640 1 -- 172.21.15.152:0/460486823 >> 172.21.15.152:0/460486823 conn(0x7ff8cc067ae0 msgr2=0x7ff8cc15e400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:17.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.610+0000 7ff8cb200640 1 -- 172.21.15.152:0/460486823 shutdown_connections 2024-04-20T11:32:17.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.610+0000 7ff8cb200640 1 -- 172.21.15.152:0/460486823 wait complete. 2024-04-20T11:32:17.630 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T11:32:17.826 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.825+0000 7f81bee00640 1 Processor -- start 2024-04-20T11:32:17.826 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.825+0000 7f81bee00640 1 -- start start 2024-04-20T11:32:17.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.826+0000 7f81bee00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f81b8146000 0x7f81b80b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:17.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.826+0000 7f81bee00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f81b80b2170 0x7f81b80b2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:17.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.826+0000 7f81bee00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81b81463e0 con 0x7f81b80cd1b0 2024-04-20T11:32:17.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.826+0000 7f81bee00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f81b80cdcc0 con 0x7f81b8146000 2024-04-20T11:32:17.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.826+0000 7f81bee00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f81b80b4750 con 0x7f81b80b2170 2024-04-20T11:32:17.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.827+0000 7f81b7e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f81b8146000 0x7f81b80b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:17.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.827+0000 7f81bce00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f81b80cd1b0 0x7f81b80cd5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:50268/0 (socket says 172.21.15.152:50268) 2024-04-20T11:32:17.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.827+0000 7f81bce00640 1 -- 172.21.15.152:0/2325592410 learned_addr learned my addr 172.21.15.152:0/2325592410 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:32:17.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.827+0000 7f81b7e00640 1 -- 172.21.15.152:0/2325592410 >> v2:172.21.15.175:3300/0 conn(0x7f81b80b2170 msgr2=0x7f81b80b2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:17.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.827+0000 7f81b7e00640 1 --2- 172.21.15.152:0/2325592410 >> v2:172.21.15.175:3300/0 conn(0x7f81b80b2170 0x7f81b80b2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:17.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.827+0000 7f81b7e00640 1 -- 172.21.15.152:0/2325592410 >> v1:172.21.15.152:6789/0 conn(0x7f81b80cd1b0 legacy=0x7f81b80cd5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:17.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.827+0000 7f81b6a00640 1 -- 172.21.15.152:0/2325592410 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2505847687 0 0) 0x7f81b81463e0 con 0x7f81b80cd1b0 2024-04-20T11:32:17.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.827+0000 7f81b7e00640 1 -- 172.21.15.152:0/2325592410 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81b80b3e20 con 0x7f81b8146000 2024-04-20T11:32:17.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.829+0000 7f81b7e00640 1 --2- 172.21.15.152:0/2325592410 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f81b8146000 0x7f81b80b4be0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f81b80b3280 tx=0x7f81a802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=160d8087c8ad1796 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:17.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.829+0000 7f81b6a00640 1 -- 172.21.15.152:0/2325592410 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a80315a0 con 0x7f81b8146000 2024-04-20T11:32:17.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.829+0000 7f81b6a00640 1 -- 172.21.15.152:0/2325592410 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81a8037070 con 0x7f81b8146000 2024-04-20T11:32:17.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.829+0000 7f81b6a00640 1 -- 172.21.15.152:0/2325592410 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a8031960 con 0x7f81b8146000 2024-04-20T11:32:17.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.829+0000 7f81bee00640 1 -- 172.21.15.152:0/2325592410 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f81b8146000 msgr2=0x7f81b80b4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:17.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.830+0000 7f81bee00640 1 --2- 172.21.15.152:0/2325592410 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f81b8146000 0x7f81b80b4be0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f81b80b3280 tx=0x7f81a802b840 comp rx=0 tx=0).stop 2024-04-20T11:32:17.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.830+0000 7f81bee00640 1 -- 172.21.15.152:0/2325592410 shutdown_connections 2024-04-20T11:32:17.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.830+0000 7f81bee00640 1 --2- 172.21.15.152:0/2325592410 >> v2:172.21.15.175:3300/0 conn(0x7f81b80b2170 0x7f81b80b2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:17.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.830+0000 7f81bee00640 1 --2- 172.21.15.152:0/2325592410 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f81b8146000 0x7f81b80b4be0 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:17.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.830+0000 7f81bee00640 1 -- 172.21.15.152:0/2325592410 >> 172.21.15.152:0/2325592410 conn(0x7f81b806b980 msgr2=0x7f81b806bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:17.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.830+0000 7f81bee00640 1 -- 172.21.15.152:0/2325592410 shutdown_connections 2024-04-20T11:32:17.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.830+0000 7f81bee00640 1 -- 172.21.15.152:0/2325592410 wait complete. 2024-04-20T11:32:17.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.831+0000 7f81bee00640 1 Processor -- start 2024-04-20T11:32:17.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.832+0000 7f81bee00640 1 -- start start 2024-04-20T11:32:17.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.832+0000 7f81bee00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f81b80b2170 0x7f81b806dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:17.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.832+0000 7f81bee00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f81b80cd1b0 0x7f81b806e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:17.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.832+0000 7f81bee00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f81b8146000 0x7f81b81f7c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:17.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.833+0000 7f81bee00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f81b815ccf0 con 0x7f81b8146000 2024-04-20T11:32:17.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.833+0000 7f81bee00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f81b80b43e0 con 0x7f81b80cd1b0 2024-04-20T11:32:17.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.833+0000 7f81bee00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f81b815ce70 con 0x7f81b80b2170 2024-04-20T11:32:17.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.833+0000 7f81bce00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f81b8146000 0x7f81b81f7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:17.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.833+0000 7f81bce00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f81b8146000 0x7f81b81f7c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:58524/0 (socket says 172.21.15.152:58524) 2024-04-20T11:32:17.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.833+0000 7f81bce00640 1 -- 172.21.15.152:0/4119504956 learned_addr learned my addr 172.21.15.152:0/4119504956 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:32:17.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.833+0000 7f81b7400640 1 --2- 172.21.15.152:0/4119504956 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f81b80cd1b0 0x7f81b806e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:17.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.833+0000 7f81b7e00640 1 --2- 172.21.15.152:0/4119504956 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f81b80b2170 0x7f81b806dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:17.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.833+0000 7f81bce00640 1 -- 172.21.15.152:0/4119504956 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f81b80b2170 msgr2=0x7f81b806dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:17.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.833+0000 7f81bce00640 1 --2- 172.21.15.152:0/4119504956 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f81b80b2170 0x7f81b806dc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:17.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.833+0000 7f81bce00640 1 -- 172.21.15.152:0/4119504956 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f81b80cd1b0 msgr2=0x7f81b806e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:17.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.833+0000 7f81bce00640 1 --2- 172.21.15.152:0/4119504956 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f81b80cd1b0 0x7f81b806e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:17.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.833+0000 7f81bce00640 1 -- 172.21.15.152:0/4119504956 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81b813d7d0 con 0x7f81b8146000 2024-04-20T11:32:17.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.833+0000 7f81b7400640 1 --2- 172.21.15.152:0/4119504956 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f81b80cd1b0 0x7f81b806e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:32:17.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.834+0000 7f81bce00640 1 --2- 172.21.15.152:0/4119504956 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f81b8146000 0x7f81b81f7c10 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f81b000c7d0 tx=0x7f81b000cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:17.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.834+0000 7f81afe00640 1 -- 172.21.15.152:0/4119504956 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b0018070 con 0x7f81b8146000 2024-04-20T11:32:17.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.834+0000 7f81afe00640 1 -- 172.21.15.152:0/4119504956 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81b000a640 con 0x7f81b8146000 2024-04-20T11:32:17.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.834+0000 7f81afe00640 1 -- 172.21.15.152:0/4119504956 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b000aa70 con 0x7f81b8146000 2024-04-20T11:32:17.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.834+0000 7f81bee00640 1 -- 172.21.15.152:0/4119504956 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81b806e7c0 con 0x7f81b8146000 2024-04-20T11:32:17.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.834+0000 7f81bee00640 1 -- 172.21.15.152:0/4119504956 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81b806ed80 con 0x7f81b8146000 2024-04-20T11:32:17.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.835+0000 7f81bee00640 1 -- 172.21.15.152:0/4119504956 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81b80689f0 con 0x7f81b8146000 2024-04-20T11:32:17.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.836+0000 7f81afe00640 1 -- 172.21.15.152:0/4119504956 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 11) ==== 91346+0+0 (secure 0 0 0) 0x7f81b001b070 con 0x7f81b8146000 2024-04-20T11:32:17.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.837+0000 7f81afe00640 1 --2- 172.21.15.152:0/4119504956 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f819806cbc0 0x7f819806f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:17.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.837+0000 7f81afe00640 1 -- 172.21.15.152:0/4119504956 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5709+0+0 (secure 0 0 0) 0x7f81b008b730 con 0x7f81b8146000 2024-04-20T11:32:17.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.837+0000 7f81b7e00640 1 --2- 172.21.15.152:0/4119504956 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f819806cbc0 0x7f819806f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:17.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.838+0000 7f81b7e00640 1 --2- 172.21.15.152:0/4119504956 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f819806cbc0 0x7f819806f080 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f81a8036000 tx=0x7f81a8008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:17.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:17.841+0000 7f81afe00640 1 -- 172.21.15.152:0/4119504956 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f81b00596c0 con 0x7f81b8146000 2024-04-20T11:32:18.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.031+0000 7f81bee00640 1 -- 172.21.15.152:0/4119504956 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f81b80d0e00 con 0x7f81b8146000 2024-04-20T11:32:18.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.031+0000 7f81afe00640 1 -- 172.21.15.152:0/4119504956 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v71) ==== 100+0+11 (secure 0 0 0) 0x7f81b0058fc0 con 0x7f81b8146000 2024-04-20T11:32:18.032 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 64 2024-04-20T11:32:18.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.034+0000 7f81bee00640 1 -- 172.21.15.152:0/4119504956 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f819806cbc0 msgr2=0x7f819806f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:18.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.034+0000 7f81bee00640 1 --2- 172.21.15.152:0/4119504956 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f819806cbc0 0x7f819806f080 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f81a8036000 tx=0x7f81a8008030 comp rx=0 tx=0).stop 2024-04-20T11:32:18.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.034+0000 7f81bee00640 1 -- 172.21.15.152:0/4119504956 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f81b8146000 msgr2=0x7f81b81f7c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:18.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.034+0000 7f81bee00640 1 --2- 172.21.15.152:0/4119504956 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f81b8146000 0x7f81b81f7c10 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f81b000c7d0 tx=0x7f81b000cc90 comp rx=0 tx=0).stop 2024-04-20T11:32:18.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.034+0000 7f81bee00640 1 -- 172.21.15.152:0/4119504956 shutdown_connections 2024-04-20T11:32:18.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.034+0000 7f81bee00640 1 --2- 172.21.15.152:0/4119504956 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f819806cbc0 0x7f819806f080 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:18.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.034+0000 7f81bee00640 1 --2- 172.21.15.152:0/4119504956 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f81b8146000 0x7f81b81f7c10 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:18.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.034+0000 7f81bee00640 1 --2- 172.21.15.152:0/4119504956 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f81b80cd1b0 0x7f81b806e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:18.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.034+0000 7f81bee00640 1 --2- 172.21.15.152:0/4119504956 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f81b80b2170 0x7f81b806dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:18.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.034+0000 7f81bee00640 1 -- 172.21.15.152:0/4119504956 >> 172.21.15.152:0/4119504956 conn(0x7f81b806b980 msgr2=0x7f81b806bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:18.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.034+0000 7f81bee00640 1 -- 172.21.15.152:0/4119504956 shutdown_connections 2024-04-20T11:32:18.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.034+0000 7f81bee00640 1 -- 172.21.15.152:0/4119504956 wait complete. 2024-04-20T11:32:18.053 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-20T11:32:18.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.244+0000 7f23c6000640 1 Processor -- start 2024-04-20T11:32:18.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.244+0000 7f23c6000640 1 -- start start 2024-04-20T11:32:18.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.246+0000 7f23c6000640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f23c015c7f0 0x7f23c0072b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:18.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.246+0000 7f23c6000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f23c00b4870 0x7f23c00ad2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:18.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.246+0000 7f23c6000640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23c00ce390 con 0x7f23c0073b60 2024-04-20T11:32:18.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.246+0000 7f23c6000640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f23c0073ff0 con 0x7f23c015c7f0 2024-04-20T11:32:18.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.246+0000 7f23c6000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f23c00ce140 con 0x7f23c00b4870 2024-04-20T11:32:18.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.246+0000 7f23bfe00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f23c0073b60 0x7f23c00b4160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:50276/0 (socket says 172.21.15.152:50276) 2024-04-20T11:32:18.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.246+0000 7f23bfe00640 1 -- 172.21.15.152:0/2920126352 learned_addr learned my addr 172.21.15.152:0/2920126352 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:32:18.249 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.246+0000 7f23c4c00640 1 --2- 172.21.15.152:0/2920126352 >> v2:172.21.15.175:3300/0 conn(0x7f23c015c7f0 0x7f23c0072b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:18.249 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.246+0000 7f23c5600640 1 --2- 172.21.15.152:0/2920126352 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f23c00b4870 0x7f23c00ad2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:18.249 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.247+0000 7f23bf400640 1 -- 172.21.15.152:0/2920126352 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2398101530 0 0) 0x7f23c00ce390 con 0x7f23c0073b60 2024-04-20T11:32:18.249 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.247+0000 7f23bf400640 1 -- 172.21.15.152:0/2920126352 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f23a8003520 con 0x7f23c0073b60 2024-04-20T11:32:18.249 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.247+0000 7f23bf400640 1 -- 172.21.15.152:0/2920126352 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3862748462 0 0) 0x7f23a8003520 con 0x7f23c0073b60 2024-04-20T11:32:18.249 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.247+0000 7f23bf400640 1 -- 172.21.15.152:0/2920126352 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f23c00b4870 msgr2=0x7f23c00ad2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:18.249 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.247+0000 7f23bf400640 1 --2- 172.21.15.152:0/2920126352 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f23c00b4870 0x7f23c00ad2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:18.249 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.247+0000 7f23bf400640 1 -- 172.21.15.152:0/2920126352 >> v2:172.21.15.175:3300/0 conn(0x7f23c015c7f0 msgr2=0x7f23c0072b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:18.249 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.247+0000 7f23bf400640 1 --2- 172.21.15.152:0/2920126352 >> v2:172.21.15.175:3300/0 conn(0x7f23c015c7f0 0x7f23c0072b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:18.249 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.247+0000 7f23bf400640 1 -- 172.21.15.152:0/2920126352 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23c00ce670 con 0x7f23c0073b60 2024-04-20T11:32:18.249 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.247+0000 7f23bf400640 1 -- 172.21.15.152:0/2920126352 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f23ac001670 con 0x7f23c0073b60 2024-04-20T11:32:18.249 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.247+0000 7f23bf400640 1 -- 172.21.15.152:0/2920126352 >> v1:172.21.15.152:6789/0 conn(0x7f23c0073b60 legacy=0x7f23c00b4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:18.250 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.247+0000 7f23bf400640 1 --2- 172.21.15.152:0/2920126352 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f23a80039b0 0x7f23a8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:18.250 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.247+0000 7f23bf400640 1 --2- 172.21.15.152:0/2920126352 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f23c00b4870 0x7f23a8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:18.250 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.247+0000 7f23bf400640 1 --2- 172.21.15.152:0/2920126352 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f23a8004d40 0x7f23a80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:18.250 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.247+0000 7f23bf400640 1 -- 172.21.15.152:0/2920126352 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f23c0073ff0 con 0x7f23a8004d40 2024-04-20T11:32:18.250 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.247+0000 7f23bf400640 1 -- 172.21.15.152:0/2920126352 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f23c00ce140 con 0x7f23a80039b0 2024-04-20T11:32:18.250 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.247+0000 7f23bf400640 1 -- 172.21.15.152:0/2920126352 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f23a800b9f0 con 0x7f23c00b4870 2024-04-20T11:32:18.250 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.247+0000 7f23c5600640 1 --2- 172.21.15.152:0/2920126352 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f23a8004d40 0x7f23a80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:18.250 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.248+0000 7f23c5600640 1 -- 172.21.15.152:0/2920126352 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f23c00b4870 msgr2=0x7f23a8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:18.250 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.248+0000 7f23c5600640 1 --2- 172.21.15.152:0/2920126352 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f23c00b4870 0x7f23a8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:18.250 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.248+0000 7f23c4c00640 1 --2- 172.21.15.152:0/2920126352 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f23a80039b0 0x7f23a8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:18.250 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.248+0000 7f23c5600640 1 -- 172.21.15.152:0/2920126352 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f23a80039b0 msgr2=0x7f23a8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:18.250 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.248+0000 7f23c5600640 1 --2- 172.21.15.152:0/2920126352 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f23a80039b0 0x7f23a8003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:18.250 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.248+0000 7f23c5600640 1 -- 172.21.15.152:0/2920126352 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23a8003520 con 0x7f23a8004d40 2024-04-20T11:32:18.252 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.249+0000 7f23c5600640 1 --2- 172.21.15.152:0/2920126352 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f23a8004d40 0x7f23a80051c0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f23b8009fc0 tx=0x7f23a8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97d69e4fb70f1a6b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:18.252 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.249+0000 7f23bf400640 1 -- 172.21.15.152:0/2920126352 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23b8039070 con 0x7f23a8004d40 2024-04-20T11:32:18.252 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.249+0000 7f23bf400640 1 -- 172.21.15.152:0/2920126352 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23b802cc60 con 0x7f23a8004d40 2024-04-20T11:32:18.252 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.249+0000 7f23bf400640 1 -- 172.21.15.152:0/2920126352 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23b8009c10 con 0x7f23a8004d40 2024-04-20T11:32:18.252 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.250+0000 7f23c6000640 1 -- 172.21.15.152:0/2920126352 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f23a8004d40 msgr2=0x7f23a80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:18.252 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.250+0000 7f23c6000640 1 --2- 172.21.15.152:0/2920126352 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f23a8004d40 0x7f23a80051c0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f23b8009fc0 tx=0x7f23a8005830 comp rx=0 tx=0).stop 2024-04-20T11:32:18.252 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.250+0000 7f23c5600640 1 -- 172.21.15.152:0/2920126352 reap_dead start 2024-04-20T11:32:18.252 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.250+0000 7f23c6000640 1 -- 172.21.15.152:0/2920126352 shutdown_connections 2024-04-20T11:32:18.252 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.250+0000 7f23c6000640 1 -- 172.21.15.152:0/2920126352 >> 172.21.15.152:0/2920126352 conn(0x7f23c00d0260 msgr2=0x7f23c00d0670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:18.252 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.250+0000 7f23c6000640 1 -- 172.21.15.152:0/2920126352 shutdown_connections 2024-04-20T11:32:18.252 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.250+0000 7f23c6000640 1 -- 172.21.15.152:0/2920126352 wait complete. 2024-04-20T11:32:18.252 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.252+0000 7f23c6000640 1 Processor -- start 2024-04-20T11:32:18.254 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.252+0000 7f23c6000640 1 -- start start 2024-04-20T11:32:18.255 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.253+0000 7f23c6000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f23a8004d40 0x7f23c00ad8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:18.255 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.253+0000 7f23c6000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f23c0099560 0x7f23c00999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:18.255 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.253+0000 7f23c6000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f23c01551b0 0x7f23c0155640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:18.255 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.253+0000 7f23c6000640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f23c00cef10 con 0x7f23c01551b0 2024-04-20T11:32:18.255 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.253+0000 7f23c6000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f23c00cec30 con 0x7f23c0099560 2024-04-20T11:32:18.255 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.253+0000 7f23c6000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f23c00ad2b0 con 0x7f23a8004d40 2024-04-20T11:32:18.255 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.253+0000 7f23c5600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f23c01551b0 0x7f23c0155640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:18.255 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.253+0000 7f23c5600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f23c01551b0 0x7f23c0155640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:58556/0 (socket says 172.21.15.152:58556) 2024-04-20T11:32:18.255 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.253+0000 7f23c5600640 1 -- 172.21.15.152:0/17876592 learned_addr learned my addr 172.21.15.152:0/17876592 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:32:18.255 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.253+0000 7f23c5600640 1 -- 172.21.15.152:0/17876592 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f23a8004d40 msgr2=0x7f23c00ad8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:18.255 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.253+0000 7f23c5600640 1 --2- 172.21.15.152:0/17876592 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f23a8004d40 0x7f23c00ad8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:18.255 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.253+0000 7f23c5600640 1 -- 172.21.15.152:0/17876592 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f23c0099560 msgr2=0x7f23c00999e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:18.255 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.253+0000 7f23c5600640 1 --2- 172.21.15.152:0/17876592 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f23c0099560 0x7f23c00999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:18.255 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.253+0000 7f23c5600640 1 -- 172.21.15.152:0/17876592 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23c006da80 con 0x7f23c01551b0 2024-04-20T11:32:18.256 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.254+0000 7f23c5600640 1 --2- 172.21.15.152:0/17876592 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f23c01551b0 0x7f23c0155640 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f23b802c6d0 tx=0x7f23b8009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:18.256 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.254+0000 7f23bcc00640 1 -- 172.21.15.152:0/17876592 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23b8039030 con 0x7f23c01551b0 2024-04-20T11:32:18.256 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.254+0000 7f23bcc00640 1 -- 172.21.15.152:0/17876592 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23b803a040 con 0x7f23c01551b0 2024-04-20T11:32:18.256 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.254+0000 7f23c6000640 1 -- 172.21.15.152:0/17876592 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23c006dd60 con 0x7f23c01551b0 2024-04-20T11:32:18.256 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.254+0000 7f23bcc00640 1 -- 172.21.15.152:0/17876592 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23b8003990 con 0x7f23c01551b0 2024-04-20T11:32:18.256 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.254+0000 7f23c6000640 1 -- 172.21.15.152:0/17876592 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23c0155f70 con 0x7f23c01551b0 2024-04-20T11:32:18.257 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.255+0000 7f23c6000640 1 -- 172.21.15.152:0/17876592 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23c0075e60 con 0x7f23c01551b0 2024-04-20T11:32:18.257 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.256+0000 7f23bcc00640 1 -- 172.21.15.152:0/17876592 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 11) ==== 91346+0+0 (secure 0 0 0) 0x7f23b8045030 con 0x7f23c01551b0 2024-04-20T11:32:18.257 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.257+0000 7f23bcc00640 1 --2- 172.21.15.152:0/17876592 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f23a006cc80 0x7f23a006f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:18.257 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.257+0000 7f23bcc00640 1 -- 172.21.15.152:0/17876592 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5709+0+0 (secure 0 0 0) 0x7f23b80ae000 con 0x7f23c01551b0 2024-04-20T11:32:18.258 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.257+0000 7f23c4c00640 1 --2- 172.21.15.152:0/17876592 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f23a006cc80 0x7f23a006f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:18.258 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.258+0000 7f23c4c00640 1 --2- 172.21.15.152:0/17876592 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f23a006cc80 0x7f23a006f140 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f23b40099d0 tx=0x7f23b4008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:18.262 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.261+0000 7f23bcc00640 1 -- 172.21.15.152:0/17876592 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f23b804c020 con 0x7f23c01551b0 2024-04-20T11:32:18.473 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.472+0000 7f23c6000640 1 -- 172.21.15.152:0/17876592 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f23c0158c20 con 0x7f23c01551b0 2024-04-20T11:32:18.473 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.473+0000 7f23bcc00640 1 -- 172.21.15.152:0/17876592 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v71) ==== 80+0+0 (secure 0 0 0) 0x7f23b807c070 con 0x7f23c01551b0 2024-04-20T11:32:18.473 INFO:teuthology.orchestra.run.smithi152.stderr:listed 0 entries 2024-04-20T11:32:18.476 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.475+0000 7f23c6000640 1 -- 172.21.15.152:0/17876592 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f23a006cc80 msgr2=0x7f23a006f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:18.476 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.475+0000 7f23c6000640 1 --2- 172.21.15.152:0/17876592 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f23a006cc80 0x7f23a006f140 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f23b40099d0 tx=0x7f23b4008040 comp rx=0 tx=0).stop 2024-04-20T11:32:18.476 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.475+0000 7f23c6000640 1 -- 172.21.15.152:0/17876592 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f23c01551b0 msgr2=0x7f23c0155640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:18.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.475+0000 7f23c6000640 1 --2- 172.21.15.152:0/17876592 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f23c01551b0 0x7f23c0155640 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f23b802c6d0 tx=0x7f23b8009750 comp rx=0 tx=0).stop 2024-04-20T11:32:18.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.476+0000 7f23c6000640 1 -- 172.21.15.152:0/17876592 shutdown_connections 2024-04-20T11:32:18.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.476+0000 7f23c6000640 1 --2- 172.21.15.152:0/17876592 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f23a006cc80 0x7f23a006f140 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:18.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.476+0000 7f23c6000640 1 --2- 172.21.15.152:0/17876592 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f23c01551b0 0x7f23c0155640 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:18.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.476+0000 7f23c6000640 1 --2- 172.21.15.152:0/17876592 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f23c0099560 0x7f23c00999e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:18.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.476+0000 7f23c6000640 1 --2- 172.21.15.152:0/17876592 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f23a8004d40 0x7f23c00ad8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:18.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.476+0000 7f23c6000640 1 -- 172.21.15.152:0/17876592 >> 172.21.15.152:0/17876592 conn(0x7f23c00d0260 msgr2=0x7f23c00aeca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:18.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.476+0000 7f23c6000640 1 -- 172.21.15.152:0/17876592 shutdown_connections 2024-04-20T11:32:18.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.476+0000 7f23c6000640 1 -- 172.21.15.152:0/17876592 wait complete. 2024-04-20T11:32:18.495 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi175.front.sepia.ceph.com... 2024-04-20T11:32:18.496 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:32:18.496 DEBUG:teuthology.orchestra.run.smithi175:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-04-20T11:32:18.579 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi175.stderr:ceph-fuse[32167]: fuse finished with error 0 and tester_r 0 2024-04-20T11:32:18.584 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T11:32:18.642 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi175.front.sepia.ceph.com 2024-04-20T11:32:18.642 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:32:18.642 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-20T11:32:18.696 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:32:18.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.907+0000 7f51d7e00640 1 Processor -- start 2024-04-20T11:32:18.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.907+0000 7f51d7e00640 1 -- start start 2024-04-20T11:32:18.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.909+0000 7f51d7e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f51d813e530 0x7f51d806ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:18.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.909+0000 7f51d7e00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51d80cf0a0 con 0x7f51d81414e0 2024-04-20T11:32:18.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.909+0000 7f51d7e00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51d80d00e0 con 0x7f51d806f340 2024-04-20T11:32:18.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.909+0000 7f51d7e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f51d8141b00 con 0x7f51d813e530 2024-04-20T11:32:18.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.909+0000 7f51d7400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f51d81414e0 0x7f51d8071cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:50280/0 (socket says 172.21.15.152:50280) 2024-04-20T11:32:18.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.909+0000 7f51d7400640 1 -- 172.21.15.152:0/3455705988 learned_addr learned my addr 172.21.15.152:0/3455705988 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:32:18.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.909+0000 7f51d6a00640 1 --2- 172.21.15.152:0/3455705988 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f51d813e530 0x7f51d806ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:18.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.910+0000 7f51d6000640 1 -- 172.21.15.152:0/3455705988 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2518672017 0 0) 0x7f51d0001490 con 0x7f51d81414e0 2024-04-20T11:32:18.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.910+0000 7f51d6000640 1 -- 172.21.15.152:0/3455705988 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f51bc005730 con 0x7f51d81414e0 2024-04-20T11:32:18.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.910+0000 7f51d6000640 1 -- 172.21.15.152:0/3455705988 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 579336603 0 0) 0x7f51bc005730 con 0x7f51d81414e0 2024-04-20T11:32:18.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.910+0000 7f51d6000640 1 -- 172.21.15.152:0/3455705988 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f51d813e530 msgr2=0x7f51d806ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:18.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.910+0000 7f51d6000640 1 --2- 172.21.15.152:0/3455705988 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f51d813e530 0x7f51d806ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:18.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.910+0000 7f51d6000640 1 -- 172.21.15.152:0/3455705988 >> v1:172.21.15.175:6789/0 conn(0x7f51d806f340 legacy=0x7f51d8140dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:18.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.910+0000 7f51d6000640 1 -- 172.21.15.152:0/3455705988 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51d80cf320 con 0x7f51d81414e0 2024-04-20T11:32:18.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.910+0000 7f51d6000640 1 -- 172.21.15.152:0/3455705988 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f51d0001bc0 con 0x7f51d81414e0 2024-04-20T11:32:18.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.910+0000 7f51d6000640 1 -- 172.21.15.152:0/3455705988 >> v1:172.21.15.152:6789/0 conn(0x7f51d81414e0 legacy=0x7f51d8071cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:18.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.910+0000 7f51d6000640 1 --2- 172.21.15.152:0/3455705988 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51bc005bf0 0x7f51bc006010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:18.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.910+0000 7f51d6000640 1 --2- 172.21.15.152:0/3455705988 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f51d813e530 0x7f51bc006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:18.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.910+0000 7f51d6000640 1 --2- 172.21.15.152:0/3455705988 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f51bc006f50 0x7f51bc0073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:18.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.910+0000 7f51d6000640 1 -- 172.21.15.152:0/3455705988 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f51bc00dc80 con 0x7f51bc006f50 2024-04-20T11:32:18.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.910+0000 7f51d6000640 1 -- 172.21.15.152:0/3455705988 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f51bc00de00 con 0x7f51bc005bf0 2024-04-20T11:32:18.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.910+0000 7f51d6000640 1 -- 172.21.15.152:0/3455705988 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f51bc00e000 con 0x7f51d813e530 2024-04-20T11:32:18.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.911+0000 7f51d7400640 1 --2- 172.21.15.152:0/3455705988 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f51bc006f50 0x7f51bc0073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:18.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.911+0000 7f51d6a00640 1 --2- 172.21.15.152:0/3455705988 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51bc005bf0 0x7f51bc006010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:18.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.911+0000 7f51cfe00640 1 --2- 172.21.15.152:0/3455705988 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f51d813e530 0x7f51bc006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:18.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.911+0000 7f51d7400640 1 -- 172.21.15.152:0/3455705988 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f51d813e530 msgr2=0x7f51bc006a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:18.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.911+0000 7f51d7400640 1 --2- 172.21.15.152:0/3455705988 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f51d813e530 0x7f51bc006a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:18.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.911+0000 7f51d7400640 1 -- 172.21.15.152:0/3455705988 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51bc005bf0 msgr2=0x7f51bc006010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:18.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.911+0000 7f51d7400640 1 --2- 172.21.15.152:0/3455705988 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51bc005bf0 0x7f51bc006010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:18.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.911+0000 7f51d7400640 1 -- 172.21.15.152:0/3455705988 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51bc005730 con 0x7f51bc006f50 2024-04-20T11:32:18.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.912+0000 7f51d7400640 1 --2- 172.21.15.152:0/3455705988 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f51bc006f50 0x7f51bc0073d0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f51d000a9f0 tx=0x7f51bc007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27f71da9019cba25 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:18.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.912+0000 7f51d6000640 1 -- 172.21.15.152:0/3455705988 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51d000a640 con 0x7f51bc006f50 2024-04-20T11:32:18.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.912+0000 7f51d6000640 1 -- 172.21.15.152:0/3455705988 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51d0030070 con 0x7f51bc006f50 2024-04-20T11:32:18.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.912+0000 7f51d6000640 1 -- 172.21.15.152:0/3455705988 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51d0034870 con 0x7f51bc006f50 2024-04-20T11:32:18.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.913+0000 7f51d7e00640 1 -- 172.21.15.152:0/3455705988 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f51bc006f50 msgr2=0x7f51bc0073d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:18.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.913+0000 7f51d7e00640 1 --2- 172.21.15.152:0/3455705988 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f51bc006f50 0x7f51bc0073d0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f51d000a9f0 tx=0x7f51bc007a40 comp rx=0 tx=0).stop 2024-04-20T11:32:18.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.913+0000 7f51d7400640 1 -- 172.21.15.152:0/3455705988 reap_dead start 2024-04-20T11:32:18.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.913+0000 7f51d7e00640 1 -- 172.21.15.152:0/3455705988 shutdown_connections 2024-04-20T11:32:18.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.913+0000 7f51d7e00640 1 -- 172.21.15.152:0/3455705988 >> 172.21.15.152:0/3455705988 conn(0x7f51d8067ae0 msgr2=0x7f51d806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:18.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.913+0000 7f51d7e00640 1 -- 172.21.15.152:0/3455705988 shutdown_connections 2024-04-20T11:32:18.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.913+0000 7f51d7e00640 1 -- 172.21.15.152:0/3455705988 wait complete. 2024-04-20T11:32:18.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.915+0000 7f51d7e00640 1 Processor -- start 2024-04-20T11:32:18.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.915+0000 7f51d7e00640 1 -- start start 2024-04-20T11:32:18.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.916+0000 7f51d7e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51d813fbd0 0x7f51d8157cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:18.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.916+0000 7f51d7e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f51d80b2c40 0x7f51d80b0d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:18.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.916+0000 7f51d7e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f51d80b12b0 0x7f51d8152b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:18.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.916+0000 7f51d7e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f51d80cf8e0 con 0x7f51d80b2c40 2024-04-20T11:32:18.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.916+0000 7f51d7e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f51d80cfc50 con 0x7f51d813fbd0 2024-04-20T11:32:18.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.916+0000 7f51d7e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f51d80723e0 con 0x7f51d80b12b0 2024-04-20T11:32:18.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.916+0000 7f51cfe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f51d80b2c40 0x7f51d80b0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:18.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.916+0000 7f51cfe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f51d80b2c40 0x7f51d80b0d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:58574/0 (socket says 172.21.15.152:58574) 2024-04-20T11:32:18.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.917+0000 7f51cfe00640 1 -- 172.21.15.152:0/2171481781 learned_addr learned my addr 172.21.15.152:0/2171481781 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:32:18.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.917+0000 7f51d7400640 1 --2- 172.21.15.152:0/2171481781 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f51d80b12b0 0x7f51d8152b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:18.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.917+0000 7f51d6a00640 1 --2- 172.21.15.152:0/2171481781 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51d813fbd0 0x7f51d8157cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:18.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.917+0000 7f51cfe00640 1 -- 172.21.15.152:0/2171481781 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f51d80b12b0 msgr2=0x7f51d8152b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:18.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.917+0000 7f51cfe00640 1 --2- 172.21.15.152:0/2171481781 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f51d80b12b0 0x7f51d8152b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:18.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.917+0000 7f51cfe00640 1 -- 172.21.15.152:0/2171481781 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51d813fbd0 msgr2=0x7f51d8157cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:18.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.917+0000 7f51cfe00640 1 --2- 172.21.15.152:0/2171481781 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51d813fbd0 0x7f51d8157cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:18.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.917+0000 7f51cfe00640 1 -- 172.21.15.152:0/2171481781 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51d81585f0 con 0x7f51d80b2c40 2024-04-20T11:32:18.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.917+0000 7f51d7400640 1 --2- 172.21.15.152:0/2171481781 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f51d80b12b0 0x7f51d8152b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:32:18.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.917+0000 7f51cfe00640 1 --2- 172.21.15.152:0/2171481781 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f51d80b2c40 0x7f51d80b0d70 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f51c000d5a0 tx=0x7f51c000da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:18.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.917+0000 7f51cea00640 1 -- 172.21.15.152:0/2171481781 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51c0005480 con 0x7f51d80b2c40 2024-04-20T11:32:18.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.918+0000 7f51cea00640 1 -- 172.21.15.152:0/2171481781 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51c0017070 con 0x7f51d80b2c40 2024-04-20T11:32:18.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.918+0000 7f51cea00640 1 -- 172.21.15.152:0/2171481781 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51c0003810 con 0x7f51d80b2c40 2024-04-20T11:32:18.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.918+0000 7f51d7e00640 1 -- 172.21.15.152:0/2171481781 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51d80affb0 con 0x7f51d80b2c40 2024-04-20T11:32:18.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.918+0000 7f51d7e00640 1 -- 172.21.15.152:0/2171481781 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51d80b0570 con 0x7f51d80b2c40 2024-04-20T11:32:18.922 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.919+0000 7f51d7e00640 1 -- 172.21.15.152:0/2171481781 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51d8075e60 con 0x7f51d80b2c40 2024-04-20T11:32:18.922 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.920+0000 7f51cea00640 1 -- 172.21.15.152:0/2171481781 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 11) ==== 91346+0+0 (secure 0 0 0) 0x7f51c00039b0 con 0x7f51d80b2c40 2024-04-20T11:32:18.922 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.921+0000 7f51cea00640 1 --2- 172.21.15.152:0/2171481781 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f51b406cc50 0x7f51b406f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:18.922 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.921+0000 7f51cea00640 1 -- 172.21.15.152:0/2171481781 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5709+0+0 (secure 0 0 0) 0x7f51c008c090 con 0x7f51d80b2c40 2024-04-20T11:32:18.922 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.921+0000 7f51d6a00640 1 --2- 172.21.15.152:0/2171481781 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f51b406cc50 0x7f51b406f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:18.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.922+0000 7f51d6a00640 1 --2- 172.21.15.152:0/2171481781 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f51b406cc50 0x7f51b406f110 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f51d8159110 tx=0x7f51c8008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:18.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:18.926+0000 7f51cea00640 1 -- 172.21.15.152:0/2171481781 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f51c0059f60 con 0x7f51d80b2c40 2024-04-20T11:32:19.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:19.170+0000 7f51d7e00640 1 -- 172.21.15.152:0/2171481781 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f51d80b3490 con 0x7f51d80b2c40 2024-04-20T11:32:19.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:19.171+0000 7f51cea00640 1 -- 172.21.15.152:0/2171481781 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+5210 (secure 0 0 0) 0x7f51c001ae40 con 0x7f51d80b2c40 2024-04-20T11:32:19.172 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:32:19.172 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5234,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6836/359391273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6835","nonce":359391273},{"type":"v1","addr":"172.21.15.152:6836","nonce":359391273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":5237,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/4269836436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":4269836436},{"type":"v1","addr":"172.21.15.152:6837","nonce":4269836436}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":5611,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6834/4153485754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":4153485754},{"type":"v1","addr":"172.21.15.175:6834","nonce":4153485754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T11:31:49.738765+0000","modified":"2024-04-20T11:31:52.857955+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5614},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5614":{"gid":5614,"name":"b","rank":0,"incarnation":39,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6835/2057501014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6833","nonce":2057501014},{"type":"v1","addr":"172.21.15.175:6835","nonce":2057501014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5614,"qdb_cluster":[5614]},"id":5}]} 2024-04-20T11:32:19.172 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 44 2024-04-20T11:32:19.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:19.174+0000 7f51d7e00640 1 -- 172.21.15.152:0/2171481781 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f51b406cc50 msgr2=0x7f51b406f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:19.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:19.174+0000 7f51d7e00640 1 --2- 172.21.15.152:0/2171481781 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f51b406cc50 0x7f51b406f110 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f51d8159110 tx=0x7f51c8008040 comp rx=0 tx=0).stop 2024-04-20T11:32:19.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:19.174+0000 7f51d7e00640 1 -- 172.21.15.152:0/2171481781 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f51d80b2c40 msgr2=0x7f51d80b0d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:19.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:19.174+0000 7f51d7e00640 1 --2- 172.21.15.152:0/2171481781 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f51d80b2c40 0x7f51d80b0d70 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f51c000d5a0 tx=0x7f51c000da60 comp rx=0 tx=0).stop 2024-04-20T11:32:19.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:19.174+0000 7f51d7e00640 1 -- 172.21.15.152:0/2171481781 shutdown_connections 2024-04-20T11:32:19.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:19.174+0000 7f51d7e00640 1 --2- 172.21.15.152:0/2171481781 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f51b406cc50 0x7f51b406f110 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:19.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:19.174+0000 7f51d7e00640 1 --2- 172.21.15.152:0/2171481781 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f51d80b12b0 0x7f51d8152b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:19.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:19.174+0000 7f51d7e00640 1 --2- 172.21.15.152:0/2171481781 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f51d80b2c40 0x7f51d80b0d70 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:19.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:19.174+0000 7f51d7e00640 1 --2- 172.21.15.152:0/2171481781 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51d813fbd0 0x7f51d8157cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:19.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:19.174+0000 7f51d7e00640 1 -- 172.21.15.152:0/2171481781 >> 172.21.15.152:0/2171481781 conn(0x7f51d8067ae0 msgr2=0x7f51d8071cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:19.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:19.174+0000 7f51d7e00640 1 -- 172.21.15.152:0/2171481781 shutdown_connections 2024-04-20T11:32:19.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:19.174+0000 7f51d7e00640 1 -- 172.21.15.152:0/2171481781 wait complete. 2024-04-20T11:32:19.193 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json session ls 2024-04-20T11:32:19.425 INFO:teuthology.orchestra.run.smithi175.stdout:[] 2024-04-20T11:32:19.425 DEBUG:tasks.cephfs.filesystem:_json_asok output [] 2024-04-20T11:32:19.426 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol16310 2024-04-20T11:32:19.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:19.641+0000 7f0e23c00640 1 Processor -- start 2024-04-20T11:32:19.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:19.641+0000 7f0e23c00640 1 -- start start 2024-04-20T11:32:19.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:19.643+0000 7f0e23c00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f0e1c0b2880 0x7f0e1c0b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:19.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:19.643+0000 7f0e23c00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e1c1463e0 con 0x7f0e1c146000 2024-04-20T11:32:19.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:19.643+0000 7f0e23c00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e1c0b3e20 con 0x7f0e1c0cd1b0 2024-04-20T11:32:19.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:19.643+0000 7f0e23c00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f0e1c0b4750 con 0x7f0e1c0b2880 2024-04-20T11:32:19.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:19.643+0000 7f0e21200640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f0e1c146000 0x7f0e1c0b2170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:50282/0 (socket says 172.21.15.152:50282) 2024-04-20T11:32:19.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:19.643+0000 7f0e21200640 1 -- 172.21.15.152:0/835452266 learned_addr learned my addr 172.21.15.152:0/835452266 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:32:19.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:19.643+0000 7f0e1be00640 1 --2- 172.21.15.152:0/835452266 >> v2:172.21.15.152:3300/0 conn(0x7f0e1c0b2880 0x7f0e1c0b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:19.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:19.643+0000 7f0e1b400640 1 -- 172.21.15.152:0/835452266 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 565936917 0 0) 0x7f0e1c1463e0 con 0x7f0e1c146000 2024-04-20T11:32:19.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:19.644+0000 7f0e1b400640 1 -- 172.21.15.152:0/835452266 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0e00003520 con 0x7f0e1c146000 2024-04-20T11:32:19.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:19.644+0000 7f0e1be00640 1 -- 172.21.15.152:0/835452266 >> v1:172.21.15.175:6789/0 conn(0x7f0e1c0cd1b0 legacy=0x7f0e1c0cd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:19.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:19.644+0000 7f0e1be00640 1 -- 172.21.15.152:0/835452266 >> v1:172.21.15.152:6789/0 conn(0x7f0e1c146000 legacy=0x7f0e1c0b2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:19.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:19.644+0000 7f0e1be00640 1 -- 172.21.15.152:0/835452266 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e1c0b4100 con 0x7f0e1c0b2880 2024-04-20T11:32:19.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:19.645+0000 7f0e1be00640 1 --2- 172.21.15.152:0/835452266 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0e1c0b2880 0x7f0e1c0b4be0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f0e080097a0 tx=0x7f0e0802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8909014db8476102 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:19.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:19.645+0000 7f0e1b400640 1 -- 172.21.15.152:0/835452266 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e080315a0 con 0x7f0e1c0b2880 2024-04-20T11:32:19.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:19.645+0000 7f0e1b400640 1 -- 172.21.15.152:0/835452266 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e08037070 con 0x7f0e1c0b2880 2024-04-20T11:32:19.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:19.645+0000 7f0e1b400640 1 -- 172.21.15.152:0/835452266 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e08031940 con 0x7f0e1c0b2880 2024-04-20T11:32:19.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:19.646+0000 7f0e23c00640 1 -- 172.21.15.152:0/835452266 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0e1c0b2880 msgr2=0x7f0e1c0b4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:19.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:19.646+0000 7f0e23c00640 1 --2- 172.21.15.152:0/835452266 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0e1c0b2880 0x7f0e1c0b4be0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f0e080097a0 tx=0x7f0e0802b840 comp rx=0 tx=0).stop 2024-04-20T11:32:19.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:19.646+0000 7f0e23c00640 1 -- 172.21.15.152:0/835452266 shutdown_connections 2024-04-20T11:32:19.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:19.646+0000 7f0e23c00640 1 --2- 172.21.15.152:0/835452266 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0e1c0b2880 0x7f0e1c0b4be0 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:19.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:19.646+0000 7f0e23c00640 1 -- 172.21.15.152:0/835452266 >> 172.21.15.152:0/835452266 conn(0x7f0e1c06b980 msgr2=0x7f0e1c06bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:19.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:19.646+0000 7f0e23c00640 1 -- 172.21.15.152:0/835452266 shutdown_connections 2024-04-20T11:32:19.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:19.646+0000 7f0e23c00640 1 -- 172.21.15.152:0/835452266 wait complete. 2024-04-20T11:32:19.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:19.648+0000 7f0e23c00640 1 Processor -- start 2024-04-20T11:32:19.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:19.648+0000 7f0e23c00640 1 -- start start 2024-04-20T11:32:19.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:19.649+0000 7f0e23c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e1c0b2880 0x7f0e1c0c5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:19.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:19.649+0000 7f0e23c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0e1c0cd1b0 0x7f0e1c136500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:19.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:19.649+0000 7f0e23c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0e1c146000 0x7f0e1c136a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:19.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:19.649+0000 7f0e23c00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f0e1c15ce70 con 0x7f0e1c0cd1b0 2024-04-20T11:32:19.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:19.649+0000 7f0e23c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0e1c15ccf0 con 0x7f0e1c0b2880 2024-04-20T11:32:19.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:19.649+0000 7f0e23c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0e1c15cff0 con 0x7f0e1c146000 2024-04-20T11:32:19.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:19.649+0000 7f0e1be00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0e1c0cd1b0 0x7f0e1c136500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:19.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:19.649+0000 7f0e1be00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0e1c0cd1b0 0x7f0e1c136500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:58594/0 (socket says 172.21.15.152:58594) 2024-04-20T11:32:19.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:19.649+0000 7f0e1be00640 1 -- 172.21.15.152:0/177641069 learned_addr learned my addr 172.21.15.152:0/177641069 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:32:19.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:19.650+0000 7f0e21200640 1 --2- 172.21.15.152:0/177641069 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e1c0b2880 0x7f0e1c0c5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:19.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:19.650+0000 7f0e1be00640 1 -- 172.21.15.152:0/177641069 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0e1c146000 msgr2=0x7f0e1c136a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:19.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:19.650+0000 7f0e21c00640 1 --2- 172.21.15.152:0/177641069 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0e1c146000 0x7f0e1c136a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:19.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:19.650+0000 7f0e1be00640 1 --2- 172.21.15.152:0/177641069 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0e1c146000 0x7f0e1c136a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:19.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:19.650+0000 7f0e1be00640 1 -- 172.21.15.152:0/177641069 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e1c0b2880 msgr2=0x7f0e1c0c5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:19.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:19.650+0000 7f0e1be00640 1 --2- 172.21.15.152:0/177641069 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e1c0b2880 0x7f0e1c0c5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:19.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:19.650+0000 7f0e1be00640 1 -- 172.21.15.152:0/177641069 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e1c0aea60 con 0x7f0e1c0cd1b0 2024-04-20T11:32:19.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:19.650+0000 7f0e21200640 1 --2- 172.21.15.152:0/177641069 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e1c0b2880 0x7f0e1c0c5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:32:19.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:19.650+0000 7f0e21c00640 1 --2- 172.21.15.152:0/177641069 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0e1c146000 0x7f0e1c136a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:32:19.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:19.650+0000 7f0e1be00640 1 --2- 172.21.15.152:0/177641069 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0e1c0cd1b0 0x7f0e1c136500 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f0e08000c00 tx=0x7f0e080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:19.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:19.650+0000 7f0e18c00640 1 -- 172.21.15.152:0/177641069 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e08032630 con 0x7f0e1c0cd1b0 2024-04-20T11:32:19.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:19.650+0000 7f0e18c00640 1 -- 172.21.15.152:0/177641069 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e080327d0 con 0x7f0e1c0cd1b0 2024-04-20T11:32:19.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:19.650+0000 7f0e18c00640 1 -- 172.21.15.152:0/177641069 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e08032bc0 con 0x7f0e1c0cd1b0 2024-04-20T11:32:19.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:19.650+0000 7f0e23c00640 1 -- 172.21.15.152:0/177641069 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e1c0aed40 con 0x7f0e1c0cd1b0 2024-04-20T11:32:19.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:19.651+0000 7f0e23c00640 1 -- 172.21.15.152:0/177641069 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e1c06dfc0 con 0x7f0e1c0cd1b0 2024-04-20T11:32:19.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:19.652+0000 7f0e23c00640 1 -- 172.21.15.152:0/177641069 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e1c0af260 con 0x7f0e1c0cd1b0 2024-04-20T11:32:19.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:19.653+0000 7f0e18c00640 1 -- 172.21.15.152:0/177641069 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 11) ==== 91346+0+0 (secure 0 0 0) 0x7f0e0803b3f0 con 0x7f0e1c0cd1b0 2024-04-20T11:32:19.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:19.653+0000 7f0e18c00640 1 --2- 172.21.15.152:0/177641069 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f0dfc06cc50 0x7f0dfc06f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:19.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:19.654+0000 7f0e18c00640 1 -- 172.21.15.152:0/177641069 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5709+0+0 (secure 0 0 0) 0x7f0e0802f030 con 0x7f0e1c0cd1b0 2024-04-20T11:32:19.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:19.654+0000 7f0e21200640 1 --2- 172.21.15.152:0/177641069 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f0dfc06cc50 0x7f0dfc06f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:19.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:19.654+0000 7f0e21200640 1 --2- 172.21.15.152:0/177641069 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f0dfc06cc50 0x7f0dfc06f110 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c005880 tx=0x7f0e0c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:19.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:19.659+0000 7f0e18c00640 1 -- 172.21.15.152:0/177641069 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f0e08040070 con 0x7f0e1c0cd1b0 2024-04-20T11:32:19.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:19.888+0000 7f0e23c00640 1 -- 172.21.15.152:0/177641069 --> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvol16310", "target": ["mon-mgr", ""]}) -- 0x7f0e1c145460 con 0x7f0dfc06cc50 2024-04-20T11:32:19.899 INFO:tasks.ceph.mgr.y.smithi152.stderr:2024-04-20T11:32:19.898+0000 7fd165e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:32:19.899 INFO:tasks.ceph.mgr.y.smithi152.stderr:2024-04-20T11:32:19.898+0000 7fd165e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:32:19.899 INFO:tasks.ceph.mgr.y.smithi152.stderr:2024-04-20T11:32:19.898+0000 7fd165e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:32:19.899 INFO:tasks.ceph.mgr.y.smithi152.stderr:2024-04-20T11:32:19.898+0000 7fd165e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:32:19.899 INFO:tasks.ceph.mgr.y.smithi152.stderr:2024-04-20T11:32:19.898+0000 7fd165e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:32:19.969 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:19.968+0000 7f0e18c00640 1 -- 172.21.15.152:0/177641069 <== mgr.4113 v2:172.21.15.152:6832/2285110240 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0e1c145460 con 0x7f0dfc06cc50 2024-04-20T11:32:19.972 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:19.971+0000 7f0e23c00640 1 -- 172.21.15.152:0/177641069 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f0dfc06cc50 msgr2=0x7f0dfc06f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:19.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:19.971+0000 7f0e23c00640 1 --2- 172.21.15.152:0/177641069 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f0dfc06cc50 0x7f0dfc06f110 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c005880 tx=0x7f0e0c009040 comp rx=0 tx=0).stop 2024-04-20T11:32:19.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:19.971+0000 7f0e23c00640 1 -- 172.21.15.152:0/177641069 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0e1c0cd1b0 msgr2=0x7f0e1c136500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:19.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:19.971+0000 7f0e23c00640 1 --2- 172.21.15.152:0/177641069 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0e1c0cd1b0 0x7f0e1c136500 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f0e08000c00 tx=0x7f0e080046c0 comp rx=0 tx=0).stop 2024-04-20T11:32:19.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:19.972+0000 7f0e23c00640 1 -- 172.21.15.152:0/177641069 shutdown_connections 2024-04-20T11:32:19.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:19.972+0000 7f0e23c00640 1 --2- 172.21.15.152:0/177641069 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f0dfc06cc50 0x7f0dfc06f110 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:19.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:19.972+0000 7f0e23c00640 1 --2- 172.21.15.152:0/177641069 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0e1c146000 0x7f0e1c136a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:19.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:19.972+0000 7f0e23c00640 1 --2- 172.21.15.152:0/177641069 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0e1c0cd1b0 0x7f0e1c136500 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:19.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:19.972+0000 7f0e23c00640 1 --2- 172.21.15.152:0/177641069 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e1c0b2880 0x7f0e1c0c5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:19.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:19.972+0000 7f0e23c00640 1 -- 172.21.15.152:0/177641069 >> 172.21.15.152:0/177641069 conn(0x7f0e1c06b980 msgr2=0x7f0e1c0afec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:19.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:19.972+0000 7f0e23c00640 1 -- 172.21.15.152:0/177641069 shutdown_connections 2024-04-20T11:32:19.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:19.972+0000 7f0e23c00640 1 -- 172.21.15.152:0/177641069 wait complete. 2024-04-20T11:32:19.993 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:32:20.187 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.186+0000 7fae2ba00640 1 Processor -- start 2024-04-20T11:32:20.187 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.187+0000 7fae2ba00640 1 -- start start 2024-04-20T11:32:20.188 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.188+0000 7fae2ba00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fae240b8070 0x7fae2409aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:20.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.188+0000 7fae2ba00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae24161400 con 0x7fae240d13a0 2024-04-20T11:32:20.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.188+0000 7fae2ba00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae240cd630 con 0x7fae2409b670 2024-04-20T11:32:20.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.188+0000 7fae2ba00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fae241603f0 con 0x7fae240b8070 2024-04-20T11:32:20.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.188+0000 7fae1be00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fae240d13a0 0x7fae241605e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:50286/0 (socket says 172.21.15.152:50286) 2024-04-20T11:32:20.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.188+0000 7fae1be00640 1 -- 172.21.15.152:0/3984422623 learned_addr learned my addr 172.21.15.152:0/3984422623 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:32:20.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.189+0000 7fae29000640 1 --2- 172.21.15.152:0/3984422623 >> v2:172.21.15.175:3300/0 conn(0x7fae240b8070 0x7fae2409aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:20.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.189+0000 7fae23e00640 1 -- 172.21.15.152:0/3984422623 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1801544973 0 0) 0x7fae24161400 con 0x7fae240d13a0 2024-04-20T11:32:20.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.189+0000 7fae23e00640 1 -- 172.21.15.152:0/3984422623 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fae08003520 con 0x7fae240d13a0 2024-04-20T11:32:20.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.189+0000 7fae23e00640 1 -- 172.21.15.152:0/3984422623 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1791207859 0 0) 0x7fae08003520 con 0x7fae240d13a0 2024-04-20T11:32:20.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.189+0000 7fae23e00640 1 -- 172.21.15.152:0/3984422623 >> v2:172.21.15.175:3300/0 conn(0x7fae240b8070 msgr2=0x7fae2409aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:20.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.189+0000 7fae23e00640 1 --2- 172.21.15.152:0/3984422623 >> v2:172.21.15.175:3300/0 conn(0x7fae240b8070 0x7fae2409aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.189+0000 7fae23e00640 1 -- 172.21.15.152:0/3984422623 >> v1:172.21.15.175:6789/0 conn(0x7fae2409b670 legacy=0x7fae24160cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:20.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.189+0000 7fae23e00640 1 -- 172.21.15.152:0/3984422623 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae240cd910 con 0x7fae240d13a0 2024-04-20T11:32:20.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.189+0000 7fae23e00640 1 -- 172.21.15.152:0/3984422623 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7fae0c001670 con 0x7fae240d13a0 2024-04-20T11:32:20.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.189+0000 7fae23e00640 1 -- 172.21.15.152:0/3984422623 >> v1:172.21.15.152:6789/0 conn(0x7fae240d13a0 legacy=0x7fae241605e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:20.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.189+0000 7fae23e00640 1 --2- 172.21.15.152:0/3984422623 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae08003a70 0x7fae08003e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:20.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.189+0000 7fae23e00640 1 --2- 172.21.15.152:0/3984422623 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fae08004800 0x7fae08004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:20.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.189+0000 7fae23e00640 1 --2- 172.21.15.152:0/3984422623 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fae080052e0 0x7fae0800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:20.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.189+0000 7fae23e00640 1 -- 172.21.15.152:0/3984422623 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fae241603f0 con 0x7fae08004800 2024-04-20T11:32:20.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.189+0000 7fae23e00640 1 -- 172.21.15.152:0/3984422623 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fae0800c1e0 con 0x7fae08003a70 2024-04-20T11:32:20.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.189+0000 7fae23e00640 1 -- 172.21.15.152:0/3984422623 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fae0800c3e0 con 0x7fae080052e0 2024-04-20T11:32:20.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.190+0000 7fae1be00640 1 --2- 172.21.15.152:0/3984422623 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fae08004800 0x7fae08004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:20.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.190+0000 7fae29000640 1 --2- 172.21.15.152:0/3984422623 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae08003a70 0x7fae08003e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:20.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.190+0000 7fae1be00640 1 -- 172.21.15.152:0/3984422623 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fae080052e0 msgr2=0x7fae0800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:20.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.190+0000 7fae1be00640 1 --2- 172.21.15.152:0/3984422623 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fae080052e0 0x7fae0800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.190+0000 7fae1be00640 1 -- 172.21.15.152:0/3984422623 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae08003a70 msgr2=0x7fae08003e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:20.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.190+0000 7fae1be00640 1 --2- 172.21.15.152:0/3984422623 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae08003a70 0x7fae08003e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.190+0000 7fae1be00640 1 -- 172.21.15.152:0/3984422623 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae08003520 con 0x7fae08004800 2024-04-20T11:32:20.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.190+0000 7fae29a00640 1 -- 172.21.15.152:0/3984422623 reap_dead start 2024-04-20T11:32:20.194 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.191+0000 7fae1be00640 1 --2- 172.21.15.152:0/3984422623 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fae08004800 0x7fae08004c80 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fae0c00c160 tx=0x7fae08005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f37624f1d52e438 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:20.194 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.191+0000 7fae23e00640 1 -- 172.21.15.152:0/3984422623 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae0c007c60 con 0x7fae08004800 2024-04-20T11:32:20.194 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.192+0000 7fae23e00640 1 -- 172.21.15.152:0/3984422623 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae0c030070 con 0x7fae08004800 2024-04-20T11:32:20.194 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.192+0000 7fae23e00640 1 -- 172.21.15.152:0/3984422623 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae0c005d90 con 0x7fae08004800 2024-04-20T11:32:20.195 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.192+0000 7fae2ba00640 1 -- 172.21.15.152:0/3984422623 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fae08004800 msgr2=0x7fae08004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:20.195 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.192+0000 7fae2ba00640 1 --2- 172.21.15.152:0/3984422623 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fae08004800 0x7fae08004c80 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fae0c00c160 tx=0x7fae08005830 comp rx=0 tx=0).stop 2024-04-20T11:32:20.195 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.192+0000 7fae2ba00640 1 -- 172.21.15.152:0/3984422623 shutdown_connections 2024-04-20T11:32:20.195 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.192+0000 7fae2ba00640 1 --2- 172.21.15.152:0/3984422623 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fae08004800 0x7fae08004c80 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.195 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.192+0000 7fae2ba00640 1 -- 172.21.15.152:0/3984422623 >> 172.21.15.152:0/3984422623 conn(0x7fae240b40c0 msgr2=0x7fae240b44d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:20.195 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.192+0000 7fae2ba00640 1 -- 172.21.15.152:0/3984422623 shutdown_connections 2024-04-20T11:32:20.195 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.192+0000 7fae2ba00640 1 -- 172.21.15.152:0/3984422623 wait complete. 2024-04-20T11:32:20.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.194+0000 7fae2ba00640 1 Processor -- start 2024-04-20T11:32:20.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.194+0000 7fae2ba00640 1 -- start start 2024-04-20T11:32:20.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.195+0000 7fae2ba00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae08004800 0x7fae2414ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:20.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.195+0000 7fae2ba00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fae241e7720 0x7fae2414b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:20.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.195+0000 7fae2ba00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fae241e7ba0 0x7fae2414b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:20.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.195+0000 7fae2ba00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fae240ce1b0 con 0x7fae241e7ba0 2024-04-20T11:32:20.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.195+0000 7fae2ba00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fae240cded0 con 0x7fae08004800 2024-04-20T11:32:20.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.195+0000 7fae2ba00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fae240ce6d0 con 0x7fae241e7720 2024-04-20T11:32:20.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.195+0000 7fae29a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fae241e7ba0 0x7fae2414b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:20.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.196+0000 7fae29a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fae241e7ba0 0x7fae2414b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:58634/0 (socket says 172.21.15.152:58634) 2024-04-20T11:32:20.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.196+0000 7fae29a00640 1 -- 172.21.15.152:0/3749142551 learned_addr learned my addr 172.21.15.152:0/3749142551 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:32:20.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.196+0000 7fae29000640 1 --2- 172.21.15.152:0/3749142551 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae08004800 0x7fae2414ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:20.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.196+0000 7fae1be00640 1 --2- 172.21.15.152:0/3749142551 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fae241e7720 0x7fae2414b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:20.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.196+0000 7fae29a00640 1 -- 172.21.15.152:0/3749142551 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fae241e7720 msgr2=0x7fae2414b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:20.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.196+0000 7fae29a00640 1 --2- 172.21.15.152:0/3749142551 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fae241e7720 0x7fae2414b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.196+0000 7fae29a00640 1 -- 172.21.15.152:0/3749142551 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae08004800 msgr2=0x7fae2414ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:20.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.196+0000 7fae29a00640 1 --2- 172.21.15.152:0/3749142551 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae08004800 0x7fae2414ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.196+0000 7fae29a00640 1 -- 172.21.15.152:0/3749142551 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae2414b150 con 0x7fae241e7ba0 2024-04-20T11:32:20.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.196+0000 7fae29a00640 1 --2- 172.21.15.152:0/3749142551 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fae241e7ba0 0x7fae2414b980 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fae240d22c0 tx=0x7fae1c00c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:20.199 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.197+0000 7fae21600640 1 -- 172.21.15.152:0/3749142551 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae1c0074e0 con 0x7fae241e7ba0 2024-04-20T11:32:20.199 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.197+0000 7fae21600640 1 -- 172.21.15.152:0/3749142551 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae1c004020 con 0x7fae241e7ba0 2024-04-20T11:32:20.199 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.197+0000 7fae21600640 1 -- 172.21.15.152:0/3749142551 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae1c004410 con 0x7fae241e7ba0 2024-04-20T11:32:20.199 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.197+0000 7fae2ba00640 1 -- 172.21.15.152:0/3749142551 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae241e63e0 con 0x7fae241e7ba0 2024-04-20T11:32:20.199 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.197+0000 7fae2ba00640 1 -- 172.21.15.152:0/3749142551 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae241e6960 con 0x7fae241e7ba0 2024-04-20T11:32:20.199 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.198+0000 7fae2ba00640 1 -- 172.21.15.152:0/3749142551 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae240689f0 con 0x7fae241e7ba0 2024-04-20T11:32:20.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.199+0000 7fae21600640 1 -- 172.21.15.152:0/3749142551 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 11) ==== 91346+0+0 (secure 0 0 0) 0x7fae1c0057c0 con 0x7fae241e7ba0 2024-04-20T11:32:20.201 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.201+0000 7fae21600640 1 --2- 172.21.15.152:0/3749142551 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fae0006cc50 0x7fae0006f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:20.201 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.201+0000 7fae21600640 1 -- 172.21.15.152:0/3749142551 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5709+0+0 (secure 0 0 0) 0x7fae1c08b250 con 0x7fae241e7ba0 2024-04-20T11:32:20.201 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.201+0000 7fae29000640 1 --2- 172.21.15.152:0/3749142551 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fae0006cc50 0x7fae0006f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:20.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.202+0000 7fae29000640 1 --2- 172.21.15.152:0/3749142551 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fae0006cc50 0x7fae0006f110 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fae14003fd0 tx=0x7fae1400b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:20.206 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.205+0000 7fae21600640 1 -- 172.21.15.152:0/3749142551 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fae1c059230 con 0x7fae241e7ba0 2024-04-20T11:32:20.445 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.445+0000 7fae2ba00640 1 -- 172.21.15.152:0/3749142551 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fae2406fda0 con 0x7fae241e7ba0 2024-04-20T11:32:20.447 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.446+0000 7fae21600640 1 -- 172.21.15.152:0/3749142551 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+5210 (secure 0 0 0) 0x7fae1c058b30 con 0x7fae241e7ba0 2024-04-20T11:32:20.447 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:32:20.447 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5234,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6836/359391273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6835","nonce":359391273},{"type":"v1","addr":"172.21.15.152:6836","nonce":359391273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":5237,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/4269836436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":4269836436},{"type":"v1","addr":"172.21.15.152:6837","nonce":4269836436}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":5611,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6834/4153485754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":4153485754},{"type":"v1","addr":"172.21.15.175:6834","nonce":4153485754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T11:31:49.738765+0000","modified":"2024-04-20T11:31:52.857955+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5614},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5614":{"gid":5614,"name":"b","rank":0,"incarnation":39,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6835/2057501014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6833","nonce":2057501014},{"type":"v1","addr":"172.21.15.175:6835","nonce":2057501014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5614,"qdb_cluster":[5614]},"id":5}]} 2024-04-20T11:32:20.447 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 44 2024-04-20T11:32:20.450 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.449+0000 7fae2ba00640 1 -- 172.21.15.152:0/3749142551 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fae0006cc50 msgr2=0x7fae0006f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:20.450 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.449+0000 7fae2ba00640 1 --2- 172.21.15.152:0/3749142551 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fae0006cc50 0x7fae0006f110 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fae14003fd0 tx=0x7fae1400b040 comp rx=0 tx=0).stop 2024-04-20T11:32:20.450 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.449+0000 7fae2ba00640 1 -- 172.21.15.152:0/3749142551 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fae241e7ba0 msgr2=0x7fae2414b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:20.450 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.449+0000 7fae2ba00640 1 --2- 172.21.15.152:0/3749142551 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fae241e7ba0 0x7fae2414b980 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fae240d22c0 tx=0x7fae1c00c820 comp rx=0 tx=0).stop 2024-04-20T11:32:20.450 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.449+0000 7fae2ba00640 1 -- 172.21.15.152:0/3749142551 shutdown_connections 2024-04-20T11:32:20.450 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.449+0000 7fae2ba00640 1 --2- 172.21.15.152:0/3749142551 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fae0006cc50 0x7fae0006f110 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.450 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.449+0000 7fae2ba00640 1 --2- 172.21.15.152:0/3749142551 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fae241e7ba0 0x7fae2414b980 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.450 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.449+0000 7fae2ba00640 1 --2- 172.21.15.152:0/3749142551 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fae241e7720 0x7fae2414b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.450 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.449+0000 7fae2ba00640 1 --2- 172.21.15.152:0/3749142551 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae08004800 0x7fae2414ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.450 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.449+0000 7fae2ba00640 1 -- 172.21.15.152:0/3749142551 >> 172.21.15.152:0/3749142551 conn(0x7fae240b40c0 msgr2=0x7fae24070030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:20.450 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.449+0000 7fae2ba00640 1 -- 172.21.15.152:0/3749142551 shutdown_connections 2024-04-20T11:32:20.450 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.449+0000 7fae2ba00640 1 -- 172.21.15.152:0/3749142551 wait complete. 2024-04-20T11:32:20.469 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json session ls 2024-04-20T11:32:20.702 INFO:teuthology.orchestra.run.smithi175.stdout:[{"id":7119,"entity":{"name":{"type":"client","num":7119},"addr":{"type":"any","addr":"172.21.15.152:0","nonce":3082578161}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":14,"uptime":0.78298779500000004,"requests_in_flight":2,"num_completed_requests":0,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":21.961476919788549,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.7119 172.21.15.152:0/3082578161","completed_requests":[],"prealloc_inos":[{"start":"0x10000000005","length":996}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"a8c84f5997950fd7d008204839253291d52e6eda","ceph_version":"ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)","entity_id":"y","hostname":"smithi152","pid":"22712","root":"/"}}] 2024-04-20T11:32:20.703 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "a8c84f5997950fd7d008204839253291d52e6eda", "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi152", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "22712", "root": "/" }, "completed_requests": [], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.152:0", "nonce": 3082578161, "type": "any" }, "name": { "num": 7119, "type": "client" } }, "id": 7119, "inst": "client.7119 172.21.15.152:0/3082578161", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 0, "num_leases": 1, "prealloc_inos": [ { "length": 996, "start": "0x10000000005" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 14, "requests_in_flight": 2, "session_cache_liveness": { "halflife": 300, "value": 21.96147691978855 }, "state": "open", "uptime": 0.782987795 } ] 2024-04-20T11:32:20.705 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-04-20T11:32:20.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.910+0000 7f6f74c00640 1 Processor -- start 2024-04-20T11:32:20.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.910+0000 7f6f74c00640 1 -- start start 2024-04-20T11:32:20.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.911+0000 7f6f74c00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f6f700cd0d0 0x7f6f700cd580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:20.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.911+0000 7f6f74c00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f70146340 con 0x7f6f70071fe0 2024-04-20T11:32:20.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.911+0000 7f6f74c00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f70072d50 con 0x7f6f70145f60 2024-04-20T11:32:20.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.911+0000 7f6f74c00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f6f7006e9f0 con 0x7f6f700cd0d0 2024-04-20T11:32:20.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.912+0000 7f6f6e800640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f6f70071fe0 0x7f6f70072470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:50302/0 (socket says 172.21.15.152:50302) 2024-04-20T11:32:20.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.912+0000 7f6f6e800640 1 -- 172.21.15.152:0/686387930 learned_addr learned my addr 172.21.15.152:0/686387930 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:32:20.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.912+0000 7f6f65400640 1 --2- 172.21.15.152:0/686387930 >> v2:172.21.15.175:3300/0 conn(0x7f6f700cd0d0 0x7f6f700cd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:20.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.912+0000 7f6f6d400640 1 -- 172.21.15.152:0/686387930 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1551884678 0 0) 0x7f6f70146340 con 0x7f6f70071fe0 2024-04-20T11:32:20.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.912+0000 7f6f6d400640 1 -- 172.21.15.152:0/686387930 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6f54003520 con 0x7f6f70071fe0 2024-04-20T11:32:20.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.913+0000 7f6f6d400640 1 -- 172.21.15.152:0/686387930 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1335565571 0 0) 0x7f6f54003520 con 0x7f6f70071fe0 2024-04-20T11:32:20.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.913+0000 7f6f6d400640 1 -- 172.21.15.152:0/686387930 >> v2:172.21.15.175:3300/0 conn(0x7f6f700cd0d0 msgr2=0x7f6f700cd580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:20.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.913+0000 7f6f6d400640 1 --2- 172.21.15.152:0/686387930 >> v2:172.21.15.175:3300/0 conn(0x7f6f700cd0d0 0x7f6f700cd580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.913+0000 7f6f6d400640 1 -- 172.21.15.152:0/686387930 >> v1:172.21.15.175:6789/0 conn(0x7f6f70145f60 legacy=0x7f6f7006ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:20.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.913+0000 7f6f6d400640 1 -- 172.21.15.152:0/686387930 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f7006e0c0 con 0x7f6f70071fe0 2024-04-20T11:32:20.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.913+0000 7f6f6d400640 1 -- 172.21.15.152:0/686387930 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f6f680019f0 con 0x7f6f70071fe0 2024-04-20T11:32:20.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.913+0000 7f6f6d400640 1 -- 172.21.15.152:0/686387930 >> v1:172.21.15.152:6789/0 conn(0x7f6f70071fe0 legacy=0x7f6f70072470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:20.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.913+0000 7f6f6d400640 1 --2- 172.21.15.152:0/686387930 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6f54003a20 0x7f6f54003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:20.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.913+0000 7f6f6d400640 1 --2- 172.21.15.152:0/686387930 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6f54004800 0x7f6f54004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:20.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.913+0000 7f6f6d400640 1 --2- 172.21.15.152:0/686387930 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6f540051c0 0x7f6f5400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:20.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.913+0000 7f6f6d400640 1 -- 172.21.15.152:0/686387930 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f6f7006e9f0 con 0x7f6f540051c0 2024-04-20T11:32:20.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.913+0000 7f6f6d400640 1 -- 172.21.15.152:0/686387930 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6f5400c0f0 con 0x7f6f54004800 2024-04-20T11:32:20.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.913+0000 7f6f6d400640 1 -- 172.21.15.152:0/686387930 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6f5400c2f0 con 0x7f6f54003a20 2024-04-20T11:32:20.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.913+0000 7f6f6e800640 1 --2- 172.21.15.152:0/686387930 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6f540051c0 0x7f6f5400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:20.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.913+0000 7f6f65400640 1 --2- 172.21.15.152:0/686387930 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6f54004800 0x7f6f54004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:20.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.913+0000 7f6f6de00640 1 --2- 172.21.15.152:0/686387930 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6f54003a20 0x7f6f54003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:20.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.914+0000 7f6f6e800640 1 -- 172.21.15.152:0/686387930 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6f54003a20 msgr2=0x7f6f54003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:20.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.914+0000 7f6f6e800640 1 --2- 172.21.15.152:0/686387930 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6f54003a20 0x7f6f54003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.914+0000 7f6f6e800640 1 -- 172.21.15.152:0/686387930 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6f54004800 msgr2=0x7f6f54004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:20.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.914+0000 7f6f6e800640 1 --2- 172.21.15.152:0/686387930 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6f54004800 0x7f6f54004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.914+0000 7f6f6e800640 1 -- 172.21.15.152:0/686387930 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f54003520 con 0x7f6f540051c0 2024-04-20T11:32:20.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.914+0000 7f6f6de00640 1 --2- 172.21.15.152:0/686387930 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6f54003a20 0x7f6f54003e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:32:20.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.914+0000 7f6f6e800640 1 -- 172.21.15.152:0/686387930 reap_dead start 2024-04-20T11:32:20.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.915+0000 7f6f6e800640 1 --2- 172.21.15.152:0/686387930 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6f540051c0 0x7f6f5400b9c0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f6f6800a840 tx=0x7f6f54005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=306e4753ee614f42 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:20.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.915+0000 7f6f6d400640 1 -- 172.21.15.152:0/686387930 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f6800a460 con 0x7f6f540051c0 2024-04-20T11:32:20.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.915+0000 7f6f6d400640 1 -- 172.21.15.152:0/686387930 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f68034650 con 0x7f6f540051c0 2024-04-20T11:32:20.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.915+0000 7f6f6d400640 1 -- 172.21.15.152:0/686387930 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f68034a40 con 0x7f6f540051c0 2024-04-20T11:32:20.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.915+0000 7f6f74c00640 1 -- 172.21.15.152:0/686387930 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6f540051c0 msgr2=0x7f6f5400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:20.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.915+0000 7f6f74c00640 1 --2- 172.21.15.152:0/686387930 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6f540051c0 0x7f6f5400b9c0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f6f6800a840 tx=0x7f6f54005830 comp rx=0 tx=0).stop 2024-04-20T11:32:20.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.915+0000 7f6f74c00640 1 -- 172.21.15.152:0/686387930 shutdown_connections 2024-04-20T11:32:20.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.915+0000 7f6f74c00640 1 --2- 172.21.15.152:0/686387930 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6f540051c0 0x7f6f5400b9c0 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.916+0000 7f6f74c00640 1 -- 172.21.15.152:0/686387930 >> 172.21.15.152:0/686387930 conn(0x7f6f70067cf0 msgr2=0x7f6f7006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:20.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.916+0000 7f6f74c00640 1 -- 172.21.15.152:0/686387930 shutdown_connections 2024-04-20T11:32:20.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.916+0000 7f6f74c00640 1 -- 172.21.15.152:0/686387930 wait complete. 2024-04-20T11:32:20.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.917+0000 7f6f74c00640 1 Processor -- start 2024-04-20T11:32:20.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.917+0000 7f6f74c00640 1 -- start start 2024-04-20T11:32:20.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.918+0000 7f6f74c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6f540051c0 0x7f6f70159900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:20.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.918+0000 7f6f74c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6f7015fcf0 0x7f6f70160130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:20.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.918+0000 7f6f74c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6f700ad0f0 0x7f6f700ad570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:20.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.918+0000 7f6f74c00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f6f700cd580 con 0x7f6f540051c0 2024-04-20T11:32:20.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.918+0000 7f6f74c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6f7006e680 con 0x7f6f7015fcf0 2024-04-20T11:32:20.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.918+0000 7f6f74c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6f700cd700 con 0x7f6f700ad0f0 2024-04-20T11:32:20.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.918+0000 7f6f6de00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6f540051c0 0x7f6f70159900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:20.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.919+0000 7f6f6de00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6f540051c0 0x7f6f70159900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:58656/0 (socket says 172.21.15.152:58656) 2024-04-20T11:32:20.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.919+0000 7f6f6de00640 1 -- 172.21.15.152:0/134333627 learned_addr learned my addr 172.21.15.152:0/134333627 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:32:20.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.919+0000 7f6f65400640 1 --2- 172.21.15.152:0/134333627 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6f7015fcf0 0x7f6f70160130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:20.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.919+0000 7f6f6e800640 1 --2- 172.21.15.152:0/134333627 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6f700ad0f0 0x7f6f700ad570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:20.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.919+0000 7f6f6de00640 1 -- 172.21.15.152:0/134333627 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6f700ad0f0 msgr2=0x7f6f700ad570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:20.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.919+0000 7f6f6de00640 1 --2- 172.21.15.152:0/134333627 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6f700ad0f0 0x7f6f700ad570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.919+0000 7f6f6de00640 1 -- 172.21.15.152:0/134333627 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6f7015fcf0 msgr2=0x7f6f70160130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:20.922 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.919+0000 7f6f6de00640 1 --2- 172.21.15.152:0/134333627 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6f7015fcf0 0x7f6f70160130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.922 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.919+0000 7f6f6de00640 1 -- 172.21.15.152:0/134333627 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f70135ee0 con 0x7f6f540051c0 2024-04-20T11:32:20.922 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.919+0000 7f6f6e800640 1 --2- 172.21.15.152:0/134333627 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6f700ad0f0 0x7f6f700ad570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:32:20.922 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.919+0000 7f6f65400640 1 --2- 172.21.15.152:0/134333627 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6f7015fcf0 0x7f6f70160130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:32:20.922 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.919+0000 7f6f6de00640 1 --2- 172.21.15.152:0/134333627 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6f540051c0 0x7f6f70159900 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f6f6000d770 tx=0x7f6f6000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:20.922 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.920+0000 7f6f66a00640 1 -- 172.21.15.152:0/134333627 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f60005270 con 0x7f6f540051c0 2024-04-20T11:32:20.922 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.920+0000 7f6f66a00640 1 -- 172.21.15.152:0/134333627 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f60017070 con 0x7f6f540051c0 2024-04-20T11:32:20.922 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.920+0000 7f6f66a00640 1 -- 172.21.15.152:0/134333627 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f6000b650 con 0x7f6f540051c0 2024-04-20T11:32:20.922 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.920+0000 7f6f74c00640 1 -- 172.21.15.152:0/134333627 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f701361c0 con 0x7f6f540051c0 2024-04-20T11:32:20.922 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.920+0000 7f6f74c00640 1 -- 172.21.15.152:0/134333627 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f70136780 con 0x7f6f540051c0 2024-04-20T11:32:20.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.921+0000 7f6f74c00640 1 -- 172.21.15.152:0/134333627 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f700689f0 con 0x7f6f540051c0 2024-04-20T11:32:20.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.922+0000 7f6f66a00640 1 -- 172.21.15.152:0/134333627 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 11) ==== 91346+0+0 (secure 0 0 0) 0x7f6f6000b7f0 con 0x7f6f540051c0 2024-04-20T11:32:20.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.923+0000 7f6f66a00640 1 --2- 172.21.15.152:0/134333627 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f6f4c06cc80 0x7f6f4c06f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:20.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.923+0000 7f6f66a00640 1 -- 172.21.15.152:0/134333627 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5709+0+0 (secure 0 0 0) 0x7f6f6008bfc0 con 0x7f6f540051c0 2024-04-20T11:32:20.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.923+0000 7f6f65400640 1 --2- 172.21.15.152:0/134333627 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f6f4c06cc80 0x7f6f4c06f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:20.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.924+0000 7f6f65400640 1 --2- 172.21.15.152:0/134333627 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f6f4c06cc80 0x7f6f4c06f140 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f6f70144d60 tx=0x7f6f58008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:20.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:20.927+0000 7f6f66a00640 1 -- 172.21.15.152:0/134333627 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f6f60059f20 con 0x7f6f540051c0 2024-04-20T11:32:21.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:21.180+0000 7f6f74c00640 1 -- 172.21.15.152:0/134333627 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f6f7006cd90 con 0x7f6f540051c0 2024-04-20T11:32:21.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:21.190+0000 7f6f66a00640 1 -- 172.21.15.152:0/134333627 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v11) ==== 81+0+475277 (secure 0 0 0) 0x7f6f60001230 con 0x7f6f540051c0 2024-04-20T11:32:21.198 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:32:21.198 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-04-20T11:32:21.198 INFO:teuthology.orchestra.run.smithi152.stdout: "epoch": 11, 2024-04-20T11:32:21.198 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.198 INFO:teuthology.orchestra.run.smithi152.stdout: "active_gid": 4113, 2024-04-20T11:32:21.198 INFO:teuthology.orchestra.run.smithi152.stdout: "active_name": "y", 2024-04-20T11:32:21.198 INFO:teuthology.orchestra.run.smithi152.stdout: "active_addrs": { 2024-04-20T11:32:21.198 INFO:teuthology.orchestra.run.smithi152.stdout: "addrvec": [ 2024-04-20T11:32:21.198 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:32:21.198 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "v2", 2024-04-20T11:32:21.198 INFO:teuthology.orchestra.run.smithi152.stdout: "addr": "172.21.15.152:6832", 2024-04-20T11:32:21.199 INFO:teuthology.orchestra.run.smithi152.stdout: "nonce": 2285110240 2024-04-20T11:32:21.199 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.199 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:32:21.199 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "v1", 2024-04-20T11:32:21.199 INFO:teuthology.orchestra.run.smithi152.stdout: "addr": "172.21.15.152:6833", 2024-04-20T11:32:21.199 INFO:teuthology.orchestra.run.smithi152.stdout: "nonce": 2285110240 2024-04-20T11:32:21.199 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.199 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-04-20T11:32:21.199 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.199 INFO:teuthology.orchestra.run.smithi152.stdout: "active_addr": "172.21.15.152:6833/2285110240", 2024-04-20T11:32:21.199 INFO:teuthology.orchestra.run.smithi152.stdout: "active_change": "2024-04-20T11:26:39.191554+0000", 2024-04-20T11:32:21.199 INFO:teuthology.orchestra.run.smithi152.stdout: "active_mgr_features": 4540701547738038271, 2024-04-20T11:32:21.199 INFO:teuthology.orchestra.run.smithi152.stdout: "available": true, 2024-04-20T11:32:21.199 INFO:teuthology.orchestra.run.smithi152.stdout: "standbys": [ 2024-04-20T11:32:21.199 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:32:21.200 INFO:teuthology.orchestra.run.smithi152.stdout: "gid": 4109, 2024-04-20T11:32:21.200 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "x", 2024-04-20T11:32:21.200 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr_features": 4540701547738038271, 2024-04-20T11:32:21.200 INFO:teuthology.orchestra.run.smithi152.stdout: "available_modules": [ 2024-04-20T11:32:21.200 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:32:21.200 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "alerts", 2024-04-20T11:32:21.200 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-04-20T11:32:21.200 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-04-20T11:32:21.200 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-04-20T11:32:21.200 INFO:teuthology.orchestra.run.smithi152.stdout: "interval": { 2024-04-20T11:32:21.200 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "interval", 2024-04-20T11:32:21.200 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "secs", 2024-04-20T11:32:21.200 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.200 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.201 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "60", 2024-04-20T11:32:21.201 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.201 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.201 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.201 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "How frequently to reexamine health status", 2024-04-20T11:32:21.201 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.201 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.201 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.201 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.201 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-04-20T11:32:21.201 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-04-20T11:32:21.201 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.201 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.202 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.202 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.202 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.202 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.202 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.202 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.202 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.202 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.202 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.202 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.202 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.202 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.202 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.203 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.203 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.203 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.203 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.203 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-04-20T11:32:21.203 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-04-20T11:32:21.203 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.203 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.203 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.203 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.203 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.203 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.203 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.203 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.204 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.204 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.204 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.204 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.204 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-04-20T11:32:21.204 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:21.204 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.204 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.204 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.204 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-04-20T11:32:21.204 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.204 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.204 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.204 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.205 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.205 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.205 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.205 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.205 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.205 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.205 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.205 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.205 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.205 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.205 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.205 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-04-20T11:32:21.205 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-04-20T11:32:21.205 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.206 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.206 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.206 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.206 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.206 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.206 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.206 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.206 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.206 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.206 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.206 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.206 INFO:teuthology.orchestra.run.smithi152.stdout: "smtp_destination": { 2024-04-20T11:32:21.206 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "smtp_destination", 2024-04-20T11:32:21.206 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.207 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.207 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.207 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.207 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.207 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.207 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.207 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Email address to send alerts to", 2024-04-20T11:32:21.207 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.207 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.207 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.207 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.207 INFO:teuthology.orchestra.run.smithi152.stdout: "smtp_from_name": { 2024-04-20T11:32:21.207 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "smtp_from_name", 2024-04-20T11:32:21.207 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.208 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.208 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.208 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "Ceph", 2024-04-20T11:32:21.208 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.208 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.208 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.208 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Email From: name", 2024-04-20T11:32:21.208 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.208 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.208 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.208 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.208 INFO:teuthology.orchestra.run.smithi152.stdout: "smtp_host": { 2024-04-20T11:32:21.208 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "smtp_host", 2024-04-20T11:32:21.208 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.208 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.209 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.209 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.209 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.209 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.209 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.209 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "SMTP server", 2024-04-20T11:32:21.209 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.209 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.209 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.209 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.209 INFO:teuthology.orchestra.run.smithi152.stdout: "smtp_password": { 2024-04-20T11:32:21.209 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "smtp_password", 2024-04-20T11:32:21.209 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.210 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.210 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.210 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.210 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.210 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.210 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.210 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Password to authenticate with", 2024-04-20T11:32:21.210 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.210 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.210 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.210 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.210 INFO:teuthology.orchestra.run.smithi152.stdout: "smtp_port": { 2024-04-20T11:32:21.210 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "smtp_port", 2024-04-20T11:32:21.211 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-04-20T11:32:21.211 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.211 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.211 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "465", 2024-04-20T11:32:21.211 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.211 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.211 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.211 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "SMTP port", 2024-04-20T11:32:21.211 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.211 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.211 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.211 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.211 INFO:teuthology.orchestra.run.smithi152.stdout: "smtp_sender": { 2024-04-20T11:32:21.212 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "smtp_sender", 2024-04-20T11:32:21.212 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.212 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.212 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.212 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.212 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.212 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.212 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.212 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "SMTP envelope sender", 2024-04-20T11:32:21.212 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.212 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.212 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.213 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.213 INFO:teuthology.orchestra.run.smithi152.stdout: "smtp_ssl": { 2024-04-20T11:32:21.213 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "smtp_ssl", 2024-04-20T11:32:21.213 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.213 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.213 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.213 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-04-20T11:32:21.213 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.213 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.213 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.213 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-20T11:32:21.213 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.213 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.213 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.214 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.214 INFO:teuthology.orchestra.run.smithi152.stdout: "smtp_user": { 2024-04-20T11:32:21.214 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "smtp_user", 2024-04-20T11:32:21.214 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.214 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.214 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.214 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.214 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.214 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.214 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.214 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "User to authenticate as", 2024-04-20T11:32:21.214 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.214 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.215 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.215 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.215 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.215 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.215 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:32:21.215 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "balancer", 2024-04-20T11:32:21.215 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-04-20T11:32:21.215 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-04-20T11:32:21.215 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-04-20T11:32:21.215 INFO:teuthology.orchestra.run.smithi152.stdout: "active": { 2024-04-20T11:32:21.215 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "active", 2024-04-20T11:32:21.215 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.215 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.215 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.216 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-04-20T11:32:21.216 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.216 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.216 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.216 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "automatically balance PGs across cluster", 2024-04-20T11:32:21.216 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.216 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.216 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.216 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.216 INFO:teuthology.orchestra.run.smithi152.stdout: "begin_time": { 2024-04-20T11:32:21.216 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "begin_time", 2024-04-20T11:32:21.216 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.216 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.216 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.217 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "0000", 2024-04-20T11:32:21.217 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.217 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.217 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.217 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "beginning time of day to automatically balance", 2024-04-20T11:32:21.217 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-20T11:32:21.217 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.217 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.217 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.217 INFO:teuthology.orchestra.run.smithi152.stdout: "begin_weekday": { 2024-04-20T11:32:21.217 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "begin_weekday", 2024-04-20T11:32:21.217 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "uint", 2024-04-20T11:32:21.217 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.217 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.218 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "0", 2024-04-20T11:32:21.218 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "0", 2024-04-20T11:32:21.218 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "6", 2024-04-20T11:32:21.218 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.218 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-20T11:32:21.218 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-20T11:32:21.218 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.218 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.218 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.218 INFO:teuthology.orchestra.run.smithi152.stdout: "crush_compat_max_iterations": { 2024-04-20T11:32:21.218 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "crush_compat_max_iterations", 2024-04-20T11:32:21.218 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "uint", 2024-04-20T11:32:21.218 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.219 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.219 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "25", 2024-04-20T11:32:21.219 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "1", 2024-04-20T11:32:21.219 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "250", 2024-04-20T11:32:21.219 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.219 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-20T11:32:21.219 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.219 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.219 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.219 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.219 INFO:teuthology.orchestra.run.smithi152.stdout: "crush_compat_metrics": { 2024-04-20T11:32:21.219 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "crush_compat_metrics", 2024-04-20T11:32:21.219 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.219 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.220 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.220 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "pgs,objects,bytes", 2024-04-20T11:32:21.220 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.220 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.220 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.220 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-20T11:32:21.220 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-20T11:32:21.220 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.220 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.220 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.220 INFO:teuthology.orchestra.run.smithi152.stdout: "crush_compat_step": { 2024-04-20T11:32:21.220 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "crush_compat_step", 2024-04-20T11:32:21.221 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "float", 2024-04-20T11:32:21.221 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.221 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.221 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "0.5", 2024-04-20T11:32:21.221 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "0.001", 2024-04-20T11:32:21.221 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "0.999", 2024-04-20T11:32:21.221 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.221 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "aggressiveness of optimization", 2024-04-20T11:32:21.221 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-20T11:32:21.221 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.221 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.221 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.222 INFO:teuthology.orchestra.run.smithi152.stdout: "end_time": { 2024-04-20T11:32:21.222 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "end_time", 2024-04-20T11:32:21.222 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.222 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.222 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.222 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "2359", 2024-04-20T11:32:21.222 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.222 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.222 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.222 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "ending time of day to automatically balance", 2024-04-20T11:32:21.222 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-20T11:32:21.222 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.222 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.222 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.223 INFO:teuthology.orchestra.run.smithi152.stdout: "end_weekday": { 2024-04-20T11:32:21.223 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "end_weekday", 2024-04-20T11:32:21.223 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "uint", 2024-04-20T11:32:21.223 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.223 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.223 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "0", 2024-04-20T11:32:21.223 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "0", 2024-04-20T11:32:21.223 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "6", 2024-04-20T11:32:21.223 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.223 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-20T11:32:21.223 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-20T11:32:21.223 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.223 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.223 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.224 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-04-20T11:32:21.224 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-04-20T11:32:21.224 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.224 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.224 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.224 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.224 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.224 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.224 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.224 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.224 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.224 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.224 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.224 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.225 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.225 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.225 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.225 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.225 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.225 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.225 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.225 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-04-20T11:32:21.225 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-04-20T11:32:21.225 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.225 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.225 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.225 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.226 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.226 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.226 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.226 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.226 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.226 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.226 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.226 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.226 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-04-20T11:32:21.226 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:21.226 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.226 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.226 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.226 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-04-20T11:32:21.227 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.227 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.227 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.227 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.227 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.227 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.227 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.227 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.227 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.227 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.227 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.227 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.227 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.227 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.228 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.228 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-04-20T11:32:21.228 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-04-20T11:32:21.228 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.228 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.228 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.228 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.228 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.228 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.228 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.228 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.228 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.229 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.229 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.229 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.229 INFO:teuthology.orchestra.run.smithi152.stdout: "min_score": { 2024-04-20T11:32:21.229 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "min_score", 2024-04-20T11:32:21.229 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "float", 2024-04-20T11:32:21.229 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.229 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.229 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "0", 2024-04-20T11:32:21.229 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.229 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.229 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.229 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-20T11:32:21.229 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.229 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.230 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.230 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.230 INFO:teuthology.orchestra.run.smithi152.stdout: "mode": { 2024-04-20T11:32:21.230 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "mode", 2024-04-20T11:32:21.230 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.230 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.230 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.230 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "upmap", 2024-04-20T11:32:21.230 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.230 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.230 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.230 INFO:teuthology.orchestra.run.smithi152.stdout: "crush-compat", 2024-04-20T11:32:21.230 INFO:teuthology.orchestra.run.smithi152.stdout: "none", 2024-04-20T11:32:21.231 INFO:teuthology.orchestra.run.smithi152.stdout: "read", 2024-04-20T11:32:21.231 INFO:teuthology.orchestra.run.smithi152.stdout: "upmap", 2024-04-20T11:32:21.231 INFO:teuthology.orchestra.run.smithi152.stdout: "upmap-read" 2024-04-20T11:32:21.231 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.231 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Balancer mode", 2024-04-20T11:32:21.231 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.231 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.231 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.231 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.231 INFO:teuthology.orchestra.run.smithi152.stdout: "pool_ids": { 2024-04-20T11:32:21.231 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "pool_ids", 2024-04-20T11:32:21.231 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.232 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.232 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.232 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.232 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.232 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.232 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.232 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-20T11:32:21.232 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.232 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.232 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.232 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.232 INFO:teuthology.orchestra.run.smithi152.stdout: "sleep_interval": { 2024-04-20T11:32:21.232 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "sleep_interval", 2024-04-20T11:32:21.233 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "secs", 2024-04-20T11:32:21.233 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.233 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.233 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "60", 2024-04-20T11:32:21.233 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.233 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.233 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.233 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-20T11:32:21.233 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.233 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.233 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.233 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.233 INFO:teuthology.orchestra.run.smithi152.stdout: "upmap_max_deviation": { 2024-04-20T11:32:21.233 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "upmap_max_deviation", 2024-04-20T11:32:21.234 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-04-20T11:32:21.234 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.234 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.234 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "5", 2024-04-20T11:32:21.234 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "1", 2024-04-20T11:32:21.234 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.234 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.234 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-20T11:32:21.234 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-20T11:32:21.234 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.234 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.234 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.234 INFO:teuthology.orchestra.run.smithi152.stdout: "upmap_max_optimizations": { 2024-04-20T11:32:21.235 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "upmap_max_optimizations", 2024-04-20T11:32:21.235 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "uint", 2024-04-20T11:32:21.235 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.235 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.235 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "10", 2024-04-20T11:32:21.235 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.235 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.235 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.235 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-20T11:32:21.235 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.235 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.235 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.235 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.235 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.236 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.236 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:32:21.236 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "cephadm", 2024-04-20T11:32:21.236 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-04-20T11:32:21.236 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-04-20T11:32:21.236 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-04-20T11:32:21.236 INFO:teuthology.orchestra.run.smithi152.stdout: "agent_down_multiplier": { 2024-04-20T11:32:21.236 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "agent_down_multiplier", 2024-04-20T11:32:21.236 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "float", 2024-04-20T11:32:21.236 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.236 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.236 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "3.0", 2024-04-20T11:32:21.236 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.237 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.237 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.237 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-20T11:32:21.237 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.237 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.237 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.237 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.237 INFO:teuthology.orchestra.run.smithi152.stdout: "agent_refresh_rate": { 2024-04-20T11:32:21.237 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "agent_refresh_rate", 2024-04-20T11:32:21.237 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "secs", 2024-04-20T11:32:21.237 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.237 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.237 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "20", 2024-04-20T11:32:21.237 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.238 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.238 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.238 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-20T11:32:21.238 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.238 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.238 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.238 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.238 INFO:teuthology.orchestra.run.smithi152.stdout: "agent_starting_port": { 2024-04-20T11:32:21.238 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "agent_starting_port", 2024-04-20T11:32:21.238 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-04-20T11:32:21.238 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.238 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.238 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "4721", 2024-04-20T11:32:21.238 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.239 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.239 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.239 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-20T11:32:21.239 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.239 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.239 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.239 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.239 INFO:teuthology.orchestra.run.smithi152.stdout: "allow_ptrace": { 2024-04-20T11:32:21.239 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "allow_ptrace", 2024-04-20T11:32:21.239 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.239 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.239 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.240 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.240 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.240 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.240 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.240 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-20T11:32:21.240 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-20T11:32:21.240 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.240 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.240 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.240 INFO:teuthology.orchestra.run.smithi152.stdout: "autotune_interval": { 2024-04-20T11:32:21.240 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "autotune_interval", 2024-04-20T11:32:21.240 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "secs", 2024-04-20T11:32:21.240 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.240 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.241 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "600", 2024-04-20T11:32:21.241 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.241 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.241 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.241 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-20T11:32:21.241 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.241 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.241 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.241 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.241 INFO:teuthology.orchestra.run.smithi152.stdout: "autotune_memory_target_ratio": { 2024-04-20T11:32:21.241 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "autotune_memory_target_ratio", 2024-04-20T11:32:21.241 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "float", 2024-04-20T11:32:21.241 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.242 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.242 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "0.7", 2024-04-20T11:32:21.242 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.242 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.242 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.242 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-20T11:32:21.242 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.242 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.242 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.242 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.242 INFO:teuthology.orchestra.run.smithi152.stdout: "cephadm_log_destination": { 2024-04-20T11:32:21.242 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "cephadm_log_destination", 2024-04-20T11:32:21.242 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.243 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.243 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.243 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.243 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.243 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.243 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.243 INFO:teuthology.orchestra.run.smithi152.stdout: "file", 2024-04-20T11:32:21.243 INFO:teuthology.orchestra.run.smithi152.stdout: "file,syslog", 2024-04-20T11:32:21.243 INFO:teuthology.orchestra.run.smithi152.stdout: "syslog" 2024-04-20T11:32:21.243 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.243 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-20T11:32:21.243 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.243 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.243 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.243 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.244 INFO:teuthology.orchestra.run.smithi152.stdout: "cgroups_split": { 2024-04-20T11:32:21.244 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "cgroups_split", 2024-04-20T11:32:21.244 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.244 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.244 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.244 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-04-20T11:32:21.244 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.244 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.244 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.244 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-20T11:32:21.244 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.244 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.244 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.244 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.245 INFO:teuthology.orchestra.run.smithi152.stdout: "config_checks_enabled": { 2024-04-20T11:32:21.245 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "config_checks_enabled", 2024-04-20T11:32:21.245 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.245 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.245 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.245 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.245 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.245 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.245 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.245 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-20T11:32:21.245 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.245 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.245 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.245 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.246 INFO:teuthology.orchestra.run.smithi152.stdout: "config_dashboard": { 2024-04-20T11:32:21.246 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "config_dashboard", 2024-04-20T11:32:21.246 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.246 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.246 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.246 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-04-20T11:32:21.246 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.246 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.246 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.246 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-20T11:32:21.246 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.246 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.247 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.247 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.247 INFO:teuthology.orchestra.run.smithi152.stdout: "container_image_alertmanager": { 2024-04-20T11:32:21.247 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "container_image_alertmanager", 2024-04-20T11:32:21.247 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.247 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.247 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.247 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-20T11:32:21.247 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.247 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.247 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.247 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Prometheus container image", 2024-04-20T11:32:21.247 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.247 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.248 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.248 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.248 INFO:teuthology.orchestra.run.smithi152.stdout: "container_image_base": { 2024-04-20T11:32:21.248 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "container_image_base", 2024-04-20T11:32:21.248 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.248 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.248 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.248 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-20T11:32:21.248 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.248 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.248 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.248 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Container image name, without the tag", 2024-04-20T11:32:21.248 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.248 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.249 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.249 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.249 INFO:teuthology.orchestra.run.smithi152.stdout: "container_image_elasticsearch": { 2024-04-20T11:32:21.249 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "container_image_elasticsearch", 2024-04-20T11:32:21.249 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.249 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.249 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.249 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-20T11:32:21.249 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.249 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.249 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.249 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "elasticsearch container image", 2024-04-20T11:32:21.249 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.249 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.249 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.250 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.250 INFO:teuthology.orchestra.run.smithi152.stdout: "container_image_grafana": { 2024-04-20T11:32:21.250 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "container_image_grafana", 2024-04-20T11:32:21.250 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.250 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.250 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.250 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-20T11:32:21.250 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.250 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.250 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.250 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Prometheus container image", 2024-04-20T11:32:21.250 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.251 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.251 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.251 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.251 INFO:teuthology.orchestra.run.smithi152.stdout: "container_image_haproxy": { 2024-04-20T11:32:21.251 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "container_image_haproxy", 2024-04-20T11:32:21.251 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.251 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.251 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.251 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-20T11:32:21.251 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.251 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.251 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.251 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "HAproxy container image", 2024-04-20T11:32:21.252 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.252 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.252 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.252 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.252 INFO:teuthology.orchestra.run.smithi152.stdout: "container_image_jaeger_agent": { 2024-04-20T11:32:21.252 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "container_image_jaeger_agent", 2024-04-20T11:32:21.252 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.252 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.252 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.252 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-20T11:32:21.252 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.252 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.252 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.253 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Jaeger agent container image", 2024-04-20T11:32:21.253 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.253 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.253 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.253 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.253 INFO:teuthology.orchestra.run.smithi152.stdout: "container_image_jaeger_collector": { 2024-04-20T11:32:21.253 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "container_image_jaeger_collector", 2024-04-20T11:32:21.253 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.253 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.253 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.253 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-20T11:32:21.253 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.253 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.254 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.254 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Jaeger collector container image", 2024-04-20T11:32:21.254 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.254 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.254 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.254 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.254 INFO:teuthology.orchestra.run.smithi152.stdout: "container_image_jaeger_query": { 2024-04-20T11:32:21.254 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "container_image_jaeger_query", 2024-04-20T11:32:21.254 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.254 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.254 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.254 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-20T11:32:21.254 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.254 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.254 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.255 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Jaeger query container image", 2024-04-20T11:32:21.255 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.255 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.255 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.255 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.255 INFO:teuthology.orchestra.run.smithi152.stdout: "container_image_keepalived": { 2024-04-20T11:32:21.255 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "container_image_keepalived", 2024-04-20T11:32:21.255 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.255 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.255 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.255 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-20T11:32:21.255 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.255 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.255 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.256 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Keepalived container image", 2024-04-20T11:32:21.256 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.256 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.256 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.256 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.256 INFO:teuthology.orchestra.run.smithi152.stdout: "container_image_loki": { 2024-04-20T11:32:21.256 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "container_image_loki", 2024-04-20T11:32:21.256 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.256 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.256 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.256 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-20T11:32:21.256 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.256 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.256 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.257 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Loki container image", 2024-04-20T11:32:21.257 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.257 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.257 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.257 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.257 INFO:teuthology.orchestra.run.smithi152.stdout: "container_image_node_exporter": { 2024-04-20T11:32:21.257 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "container_image_node_exporter", 2024-04-20T11:32:21.257 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.257 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.257 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.257 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-20T11:32:21.257 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.257 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.258 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.258 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Prometheus container image", 2024-04-20T11:32:21.258 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.258 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.258 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.258 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.258 INFO:teuthology.orchestra.run.smithi152.stdout: "container_image_nvmeof": { 2024-04-20T11:32:21.258 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "container_image_nvmeof", 2024-04-20T11:32:21.258 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.258 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.258 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.258 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-20T11:32:21.258 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.259 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.259 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.259 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Nvme-of container image", 2024-04-20T11:32:21.259 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.259 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.259 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.259 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.259 INFO:teuthology.orchestra.run.smithi152.stdout: "container_image_prometheus": { 2024-04-20T11:32:21.259 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "container_image_prometheus", 2024-04-20T11:32:21.259 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.259 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.259 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.259 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-20T11:32:21.259 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.260 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.260 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.260 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Prometheus container image", 2024-04-20T11:32:21.260 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.260 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.260 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.260 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.260 INFO:teuthology.orchestra.run.smithi152.stdout: "container_image_promtail": { 2024-04-20T11:32:21.260 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "container_image_promtail", 2024-04-20T11:32:21.260 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.260 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.260 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.260 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-20T11:32:21.261 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.261 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.261 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.261 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Promtail container image", 2024-04-20T11:32:21.261 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.261 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.261 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.261 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.261 INFO:teuthology.orchestra.run.smithi152.stdout: "container_image_samba": { 2024-04-20T11:32:21.261 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "container_image_samba", 2024-04-20T11:32:21.261 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.261 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.261 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.261 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-20T11:32:21.261 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.262 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.262 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.262 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Samba/SMB container image", 2024-04-20T11:32:21.262 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.262 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.262 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.262 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.262 INFO:teuthology.orchestra.run.smithi152.stdout: "container_image_snmp_gateway": { 2024-04-20T11:32:21.262 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "container_image_snmp_gateway", 2024-04-20T11:32:21.262 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.262 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.262 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.262 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-20T11:32:21.262 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.263 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.263 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.263 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "SNMP Gateway container image", 2024-04-20T11:32:21.263 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.263 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.263 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.263 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.263 INFO:teuthology.orchestra.run.smithi152.stdout: "container_init": { 2024-04-20T11:32:21.263 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "container_init", 2024-04-20T11:32:21.263 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.263 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.263 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.263 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-04-20T11:32:21.263 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.264 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.264 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.264 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Run podman/docker with `--init`", 2024-04-20T11:32:21.264 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.264 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.264 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.264 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.264 INFO:teuthology.orchestra.run.smithi152.stdout: "daemon_cache_timeout": { 2024-04-20T11:32:21.264 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "daemon_cache_timeout", 2024-04-20T11:32:21.264 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "secs", 2024-04-20T11:32:21.264 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.264 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.264 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "600", 2024-04-20T11:32:21.265 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.265 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.265 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.265 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-20T11:32:21.265 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.265 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.265 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.265 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.265 INFO:teuthology.orchestra.run.smithi152.stdout: "default_cephadm_command_timeout": { 2024-04-20T11:32:21.265 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "default_cephadm_command_timeout", 2024-04-20T11:32:21.265 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-04-20T11:32:21.265 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.265 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.266 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "900", 2024-04-20T11:32:21.266 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.266 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.266 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.266 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-20T11:32:21.266 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.266 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.266 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.266 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.266 INFO:teuthology.orchestra.run.smithi152.stdout: "default_registry": { 2024-04-20T11:32:21.266 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "default_registry", 2024-04-20T11:32:21.266 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.266 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.266 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.267 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "docker.io", 2024-04-20T11:32:21.267 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.267 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.267 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.267 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-20T11:32:21.267 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.267 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.267 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.267 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.267 INFO:teuthology.orchestra.run.smithi152.stdout: "device_cache_timeout": { 2024-04-20T11:32:21.267 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "device_cache_timeout", 2024-04-20T11:32:21.267 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "secs", 2024-04-20T11:32:21.267 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.267 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.268 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "1800", 2024-04-20T11:32:21.268 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.268 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.268 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.268 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "seconds to cache device inventory", 2024-04-20T11:32:21.268 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.268 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.268 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.268 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.268 INFO:teuthology.orchestra.run.smithi152.stdout: "device_enhanced_scan": { 2024-04-20T11:32:21.268 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "device_enhanced_scan", 2024-04-20T11:32:21.268 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.268 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.268 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.269 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.269 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.269 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.269 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.269 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-20T11:32:21.269 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.269 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.269 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.269 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.269 INFO:teuthology.orchestra.run.smithi152.stdout: "facts_cache_timeout": { 2024-04-20T11:32:21.269 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "facts_cache_timeout", 2024-04-20T11:32:21.269 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "secs", 2024-04-20T11:32:21.269 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.269 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.269 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "60", 2024-04-20T11:32:21.270 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.270 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.270 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.270 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "seconds to cache host facts data", 2024-04-20T11:32:21.270 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.270 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.270 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.270 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.270 INFO:teuthology.orchestra.run.smithi152.stdout: "grafana_dashboards_path": { 2024-04-20T11:32:21.270 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "grafana_dashboards_path", 2024-04-20T11:32:21.270 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.270 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.270 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.271 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-20T11:32:21.271 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.271 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.271 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.271 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-20T11:32:21.271 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.271 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.271 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.271 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.271 INFO:teuthology.orchestra.run.smithi152.stdout: "host_check_interval": { 2024-04-20T11:32:21.271 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "host_check_interval", 2024-04-20T11:32:21.271 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "secs", 2024-04-20T11:32:21.271 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.272 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.272 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "600", 2024-04-20T11:32:21.272 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.272 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.272 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.272 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "how frequently to perform a host check", 2024-04-20T11:32:21.272 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.272 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.272 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.272 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.272 INFO:teuthology.orchestra.run.smithi152.stdout: "hw_monitoring": { 2024-04-20T11:32:21.272 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "hw_monitoring", 2024-04-20T11:32:21.272 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.272 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.272 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.273 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.273 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.273 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.273 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.273 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-20T11:32:21.273 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.273 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.273 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.273 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.273 INFO:teuthology.orchestra.run.smithi152.stdout: "inventory_list_all": { 2024-04-20T11:32:21.273 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "inventory_list_all", 2024-04-20T11:32:21.273 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.273 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.273 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.274 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.274 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.274 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.274 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.274 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-20T11:32:21.274 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.274 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.274 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.274 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.274 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-04-20T11:32:21.274 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-04-20T11:32:21.274 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.274 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.274 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.275 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.275 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.275 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.275 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.275 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.275 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.275 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.275 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.275 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.275 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.275 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.275 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.275 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.276 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.276 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.276 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.276 INFO:teuthology.orchestra.run.smithi152.stdout: "log_refresh_metadata": { 2024-04-20T11:32:21.276 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_refresh_metadata", 2024-04-20T11:32:21.276 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.276 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.276 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.276 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.276 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.276 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.276 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.276 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-20T11:32:21.276 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.277 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.277 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.277 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.277 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-04-20T11:32:21.277 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-04-20T11:32:21.277 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.277 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.277 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.277 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-04-20T11:32:21.277 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.277 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.277 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.277 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-20T11:32:21.277 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.278 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.278 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.278 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.278 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-04-20T11:32:21.278 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:21.278 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.278 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.278 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.278 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-04-20T11:32:21.278 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.278 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.278 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.278 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.278 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.279 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.279 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.279 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.279 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.279 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.279 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.279 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.279 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.279 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.279 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.279 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-04-20T11:32:21.279 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-04-20T11:32:21.279 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.279 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.280 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.280 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.280 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.280 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.280 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.280 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.280 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.280 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.280 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.280 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.280 INFO:teuthology.orchestra.run.smithi152.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-20T11:32:21.280 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-20T11:32:21.280 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.280 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.280 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.281 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.281 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.281 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.281 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.281 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-20T11:32:21.281 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.281 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.281 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.281 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.281 INFO:teuthology.orchestra.run.smithi152.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-20T11:32:21.281 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-20T11:32:21.281 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.281 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.281 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.282 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "*", 2024-04-20T11:32:21.282 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.282 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.282 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.282 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-20T11:32:21.282 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.282 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.282 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.282 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.282 INFO:teuthology.orchestra.run.smithi152.stdout: "max_count_per_host": { 2024-04-20T11:32:21.282 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "max_count_per_host", 2024-04-20T11:32:21.282 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-04-20T11:32:21.282 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.282 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.283 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "10", 2024-04-20T11:32:21.283 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.283 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.283 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.283 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "max number of daemons per service per host", 2024-04-20T11:32:21.283 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.283 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.283 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.283 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.283 INFO:teuthology.orchestra.run.smithi152.stdout: "max_osd_draining_count": { 2024-04-20T11:32:21.283 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "max_osd_draining_count", 2024-04-20T11:32:21.283 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-04-20T11:32:21.283 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.283 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.284 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "10", 2024-04-20T11:32:21.284 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.284 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.284 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.284 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-20T11:32:21.284 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.284 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.284 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.284 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.284 INFO:teuthology.orchestra.run.smithi152.stdout: "migration_current": { 2024-04-20T11:32:21.284 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "migration_current", 2024-04-20T11:32:21.284 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-04-20T11:32:21.284 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.284 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.284 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.285 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.285 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.285 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.285 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "internal - do not modify", 2024-04-20T11:32:21.285 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.285 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.285 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.285 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.285 INFO:teuthology.orchestra.run.smithi152.stdout: "mode": { 2024-04-20T11:32:21.285 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "mode", 2024-04-20T11:32:21.285 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.285 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.285 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.285 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "root", 2024-04-20T11:32:21.286 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.286 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.286 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.286 INFO:teuthology.orchestra.run.smithi152.stdout: "cephadm-package", 2024-04-20T11:32:21.286 INFO:teuthology.orchestra.run.smithi152.stdout: "root" 2024-04-20T11:32:21.286 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.286 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "mode for remote execution of cephadm", 2024-04-20T11:32:21.286 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.286 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.286 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.286 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.286 INFO:teuthology.orchestra.run.smithi152.stdout: "oob_default_addr": { 2024-04-20T11:32:21.286 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "oob_default_addr", 2024-04-20T11:32:21.287 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.287 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.287 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.287 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "169.254.1.1", 2024-04-20T11:32:21.287 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.287 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.287 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.287 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-20T11:32:21.287 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.287 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.287 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.287 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.287 INFO:teuthology.orchestra.run.smithi152.stdout: "prometheus_alerts_path": { 2024-04-20T11:32:21.287 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "prometheus_alerts_path", 2024-04-20T11:32:21.288 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.288 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.288 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.288 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-20T11:32:21.288 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.288 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.288 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.288 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-20T11:32:21.288 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.288 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.288 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.288 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.288 INFO:teuthology.orchestra.run.smithi152.stdout: "registry_insecure": { 2024-04-20T11:32:21.289 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "registry_insecure", 2024-04-20T11:32:21.289 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.289 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.289 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.289 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.289 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.289 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.289 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.289 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-20T11:32:21.289 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.290 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.290 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.290 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.290 INFO:teuthology.orchestra.run.smithi152.stdout: "registry_password": { 2024-04-20T11:32:21.290 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "registry_password", 2024-04-20T11:32:21.290 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.290 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.290 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.290 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.290 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.290 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.290 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.290 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-20T11:32:21.291 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.291 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.291 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.291 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.291 INFO:teuthology.orchestra.run.smithi152.stdout: "registry_url": { 2024-04-20T11:32:21.291 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "registry_url", 2024-04-20T11:32:21.291 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.291 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.291 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.291 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.291 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.291 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.291 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.291 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-20T11:32:21.292 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.292 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.292 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.292 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.292 INFO:teuthology.orchestra.run.smithi152.stdout: "registry_username": { 2024-04-20T11:32:21.292 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "registry_username", 2024-04-20T11:32:21.292 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.292 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.292 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.292 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.292 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.292 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.292 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.293 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-20T11:32:21.293 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.293 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.293 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.293 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.293 INFO:teuthology.orchestra.run.smithi152.stdout: "secure_monitoring_stack": { 2024-04-20T11:32:21.293 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "secure_monitoring_stack", 2024-04-20T11:32:21.293 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.293 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.293 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.293 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.293 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.293 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.293 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.294 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-20T11:32:21.294 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.294 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.294 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.294 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.294 INFO:teuthology.orchestra.run.smithi152.stdout: "service_discovery_port": { 2024-04-20T11:32:21.294 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "service_discovery_port", 2024-04-20T11:32:21.294 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-04-20T11:32:21.294 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.294 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.294 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "8765", 2024-04-20T11:32:21.294 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.294 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.294 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.295 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "cephadm service discovery port", 2024-04-20T11:32:21.295 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.295 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.295 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.295 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.295 INFO:teuthology.orchestra.run.smithi152.stdout: "ssh_config_file": { 2024-04-20T11:32:21.295 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "ssh_config_file", 2024-04-20T11:32:21.295 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.295 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.295 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.295 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.295 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.295 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.295 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.296 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-20T11:32:21.296 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.296 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.296 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.296 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.296 INFO:teuthology.orchestra.run.smithi152.stdout: "use_agent": { 2024-04-20T11:32:21.296 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "use_agent", 2024-04-20T11:32:21.296 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.296 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.296 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.296 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.296 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.296 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.296 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.297 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-20T11:32:21.297 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.297 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.297 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.297 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.297 INFO:teuthology.orchestra.run.smithi152.stdout: "use_repo_digest": { 2024-04-20T11:32:21.297 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "use_repo_digest", 2024-04-20T11:32:21.297 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.297 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.297 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.297 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-04-20T11:32:21.297 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.297 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.297 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.297 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-20T11:32:21.298 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.298 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.298 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.298 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.298 INFO:teuthology.orchestra.run.smithi152.stdout: "warn_on_failed_host_check": { 2024-04-20T11:32:21.298 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "warn_on_failed_host_check", 2024-04-20T11:32:21.298 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.298 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.298 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.298 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-04-20T11:32:21.298 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.298 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.298 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.298 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "raise a health warning if the host check fails", 2024-04-20T11:32:21.299 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.299 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.299 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.299 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.299 INFO:teuthology.orchestra.run.smithi152.stdout: "warn_on_stray_daemons": { 2024-04-20T11:32:21.299 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "warn_on_stray_daemons", 2024-04-20T11:32:21.299 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.299 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.299 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.299 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-04-20T11:32:21.299 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.299 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.299 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.299 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-20T11:32:21.300 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.300 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.300 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.300 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.300 INFO:teuthology.orchestra.run.smithi152.stdout: "warn_on_stray_hosts": { 2024-04-20T11:32:21.300 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "warn_on_stray_hosts", 2024-04-20T11:32:21.300 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.300 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.300 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.300 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-04-20T11:32:21.300 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.300 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.300 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.300 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-20T11:32:21.301 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.301 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.301 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.301 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.301 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.301 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.301 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:32:21.301 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "crash", 2024-04-20T11:32:21.301 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-04-20T11:32:21.301 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-04-20T11:32:21.302 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-04-20T11:32:21.302 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-04-20T11:32:21.302 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-04-20T11:32:21.302 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.302 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.302 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.302 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.302 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.302 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.302 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.302 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.302 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.302 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.303 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.303 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.303 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.303 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.303 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.303 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.303 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.303 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.303 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.303 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-04-20T11:32:21.303 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-04-20T11:32:21.303 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.303 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.303 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.304 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.304 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.304 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.304 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.304 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.304 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.304 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.304 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.304 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.304 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-04-20T11:32:21.304 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:21.304 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.304 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.304 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.304 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-04-20T11:32:21.304 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.305 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.305 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.305 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.305 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.305 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.305 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.305 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.305 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.305 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.305 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.305 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.305 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.305 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.306 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.306 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-04-20T11:32:21.306 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-04-20T11:32:21.306 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.306 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.306 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.306 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.306 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.306 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.306 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.306 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.306 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.306 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.306 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.307 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.307 INFO:teuthology.orchestra.run.smithi152.stdout: "retain_interval": { 2024-04-20T11:32:21.307 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "retain_interval", 2024-04-20T11:32:21.307 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "secs", 2024-04-20T11:32:21.307 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.307 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.307 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "31536000", 2024-04-20T11:32:21.307 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.307 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.307 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.307 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-20T11:32:21.307 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.307 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.307 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.307 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.308 INFO:teuthology.orchestra.run.smithi152.stdout: "warn_recent_interval": { 2024-04-20T11:32:21.308 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "warn_recent_interval", 2024-04-20T11:32:21.308 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "secs", 2024-04-20T11:32:21.308 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.308 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.308 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "1209600", 2024-04-20T11:32:21.308 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.308 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.308 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.308 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-20T11:32:21.308 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.308 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.308 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.309 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.309 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.309 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.309 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:32:21.309 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "dashboard", 2024-04-20T11:32:21.309 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-04-20T11:32:21.309 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-04-20T11:32:21.309 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-04-20T11:32:21.309 INFO:teuthology.orchestra.run.smithi152.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-20T11:32:21.309 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-20T11:32:21.309 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-04-20T11:32:21.309 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.309 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.309 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "10", 2024-04-20T11:32:21.310 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.310 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.310 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.310 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.310 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.310 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.310 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.310 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.310 INFO:teuthology.orchestra.run.smithi152.stdout: "ALERTMANAGER_API_HOST": { 2024-04-20T11:32:21.310 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-20T11:32:21.310 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.310 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.310 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.311 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.311 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.311 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.311 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.311 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.311 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.311 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.311 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.311 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.311 INFO:teuthology.orchestra.run.smithi152.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-20T11:32:21.311 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-20T11:32:21.311 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.311 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.311 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.312 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-04-20T11:32:21.312 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.312 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.312 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.312 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.312 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.312 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.312 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.312 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.312 INFO:teuthology.orchestra.run.smithi152.stdout: "AUDIT_API_ENABLED": { 2024-04-20T11:32:21.312 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "AUDIT_API_ENABLED", 2024-04-20T11:32:21.312 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.312 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.312 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.313 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.313 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.313 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.313 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.313 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.313 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.313 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.313 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.313 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.313 INFO:teuthology.orchestra.run.smithi152.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-20T11:32:21.313 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-20T11:32:21.313 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.313 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.313 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.314 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-04-20T11:32:21.314 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.314 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.314 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.314 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.314 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.314 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.314 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.314 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.314 INFO:teuthology.orchestra.run.smithi152.stdout: "ENABLE_BROWSABLE_API": { 2024-04-20T11:32:21.314 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-20T11:32:21.314 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.314 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.314 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.315 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-04-20T11:32:21.315 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.315 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.315 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.315 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.315 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.315 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.315 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.315 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.315 INFO:teuthology.orchestra.run.smithi152.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-20T11:32:21.315 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-20T11:32:21.315 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.316 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.316 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.316 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-04-20T11:32:21.316 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.316 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.316 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.316 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.316 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.316 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.316 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.316 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.316 INFO:teuthology.orchestra.run.smithi152.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-20T11:32:21.316 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-20T11:32:21.316 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.316 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.317 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.317 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-04-20T11:32:21.317 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.317 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.317 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.317 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.317 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.317 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.317 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.317 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.317 INFO:teuthology.orchestra.run.smithi152.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-20T11:32:21.317 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-20T11:32:21.317 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.317 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.318 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.318 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-04-20T11:32:21.318 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.318 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.318 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.318 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.318 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.318 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.318 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.318 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.318 INFO:teuthology.orchestra.run.smithi152.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-20T11:32:21.318 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-20T11:32:21.318 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.319 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.319 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.319 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-04-20T11:32:21.319 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.319 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.319 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.319 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.319 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.319 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.319 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.319 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.319 INFO:teuthology.orchestra.run.smithi152.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-20T11:32:21.319 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-20T11:32:21.319 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.320 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.320 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.320 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-04-20T11:32:21.320 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.320 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.320 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.320 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.320 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.320 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.320 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.320 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.320 INFO:teuthology.orchestra.run.smithi152.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-20T11:32:21.320 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-20T11:32:21.320 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.320 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.321 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.321 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-04-20T11:32:21.321 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.321 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.321 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.321 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.321 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.321 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.321 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.321 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.322 INFO:teuthology.orchestra.run.smithi152.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-20T11:32:21.322 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-20T11:32:21.322 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.322 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.322 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.322 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-04-20T11:32:21.322 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.322 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.322 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.322 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.322 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.322 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.322 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.322 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.323 INFO:teuthology.orchestra.run.smithi152.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-20T11:32:21.323 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-20T11:32:21.323 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.323 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.323 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.323 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.323 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.323 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.323 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.323 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.323 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.323 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.323 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.323 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.324 INFO:teuthology.orchestra.run.smithi152.stdout: "GRAFANA_API_PASSWORD": { 2024-04-20T11:32:21.324 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-20T11:32:21.324 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.324 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.324 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.324 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "admin", 2024-04-20T11:32:21.324 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.324 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.324 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.324 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.324 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.324 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.324 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.324 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.324 INFO:teuthology.orchestra.run.smithi152.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-20T11:32:21.325 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-20T11:32:21.325 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.325 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.325 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.325 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-04-20T11:32:21.325 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.325 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.325 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.325 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.325 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.325 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.325 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.325 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.325 INFO:teuthology.orchestra.run.smithi152.stdout: "GRAFANA_API_URL": { 2024-04-20T11:32:21.326 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "GRAFANA_API_URL", 2024-04-20T11:32:21.326 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.326 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.326 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.326 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.326 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.326 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.326 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.326 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.326 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.326 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.326 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.326 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.326 INFO:teuthology.orchestra.run.smithi152.stdout: "GRAFANA_API_USERNAME": { 2024-04-20T11:32:21.327 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-20T11:32:21.327 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.327 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.327 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.327 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "admin", 2024-04-20T11:32:21.327 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.327 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.327 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.327 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.327 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.327 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.327 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.327 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.327 INFO:teuthology.orchestra.run.smithi152.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-20T11:32:21.328 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-20T11:32:21.328 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.328 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.328 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.328 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.328 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.328 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.328 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.328 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.328 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.329 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.329 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.329 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.329 INFO:teuthology.orchestra.run.smithi152.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-20T11:32:21.329 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-20T11:32:21.329 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.329 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.329 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.329 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.329 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.329 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.329 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.329 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.330 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.330 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.330 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.330 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.330 INFO:teuthology.orchestra.run.smithi152.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-20T11:32:21.330 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-20T11:32:21.330 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.330 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.330 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.330 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-04-20T11:32:21.330 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.330 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.330 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.331 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.331 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.331 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.331 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.331 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.331 INFO:teuthology.orchestra.run.smithi152.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-20T11:32:21.331 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-20T11:32:21.331 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.331 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.331 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.331 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.331 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.331 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.332 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.332 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.332 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.332 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.332 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.332 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.332 INFO:teuthology.orchestra.run.smithi152.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-20T11:32:21.332 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-20T11:32:21.332 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.332 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.332 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.332 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "[]", 2024-04-20T11:32:21.332 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.332 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.333 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.333 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.333 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.333 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.333 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.333 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.333 INFO:teuthology.orchestra.run.smithi152.stdout: "MULTICLUSTER_CONFIG": { 2024-04-20T11:32:21.333 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-20T11:32:21.333 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.333 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.333 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.333 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "{}", 2024-04-20T11:32:21.333 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.333 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.334 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.334 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.334 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.334 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.334 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.334 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.334 INFO:teuthology.orchestra.run.smithi152.stdout: "PROMETHEUS_API_HOST": { 2024-04-20T11:32:21.334 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-20T11:32:21.334 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.334 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.334 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.334 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.334 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.334 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.335 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.335 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.335 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.335 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.335 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.335 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.335 INFO:teuthology.orchestra.run.smithi152.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-20T11:32:21.335 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-20T11:32:21.335 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.335 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.335 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.335 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-04-20T11:32:21.335 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.335 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.336 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.336 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.336 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.336 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.336 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.336 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.336 INFO:teuthology.orchestra.run.smithi152.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-20T11:32:21.336 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-20T11:32:21.336 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.336 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.336 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.336 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.336 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.336 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.337 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.337 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.337 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.337 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.337 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.337 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.337 INFO:teuthology.orchestra.run.smithi152.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-20T11:32:21.337 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-20T11:32:21.337 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.337 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.337 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.337 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.337 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.337 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.338 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.338 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.338 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.338 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.338 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.338 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.338 INFO:teuthology.orchestra.run.smithi152.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-20T11:32:21.338 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-20T11:32:21.338 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.338 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.338 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.338 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-04-20T11:32:21.338 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.338 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.339 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.339 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.339 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.339 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.339 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.339 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.339 INFO:teuthology.orchestra.run.smithi152.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-20T11:32:21.339 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-20T11:32:21.339 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.339 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.339 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.339 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-04-20T11:32:21.339 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.339 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.339 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.340 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.340 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.340 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.340 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.340 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.340 INFO:teuthology.orchestra.run.smithi152.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-20T11:32:21.340 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-20T11:32:21.340 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.340 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.340 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.340 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.340 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.340 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.340 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.341 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.341 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.341 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.341 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.341 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.341 INFO:teuthology.orchestra.run.smithi152.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-20T11:32:21.341 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-20T11:32:21.341 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.341 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.341 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.341 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.341 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.341 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.341 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.341 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.342 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.342 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.342 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.342 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.342 INFO:teuthology.orchestra.run.smithi152.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-20T11:32:21.342 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-20T11:32:21.342 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.342 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.342 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.342 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.342 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.342 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.343 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.343 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.343 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.343 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.343 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.343 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.343 INFO:teuthology.orchestra.run.smithi152.stdout: "PWD_POLICY_ENABLED": { 2024-04-20T11:32:21.343 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-20T11:32:21.343 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.343 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.343 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.343 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-04-20T11:32:21.343 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.343 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.343 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.344 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.344 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.344 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.344 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.344 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.344 INFO:teuthology.orchestra.run.smithi152.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-20T11:32:21.344 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-20T11:32:21.344 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.344 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.344 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.344 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-20T11:32:21.344 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.344 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.344 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.345 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.345 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.345 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.345 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.345 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.345 INFO:teuthology.orchestra.run.smithi152.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-20T11:32:21.345 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-20T11:32:21.345 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-04-20T11:32:21.345 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.345 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.345 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "10", 2024-04-20T11:32:21.345 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.345 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.345 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.345 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.346 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.346 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.346 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.346 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.346 INFO:teuthology.orchestra.run.smithi152.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-20T11:32:21.346 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-20T11:32:21.346 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-04-20T11:32:21.346 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.346 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.346 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "8", 2024-04-20T11:32:21.346 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.346 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.346 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.346 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.347 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.347 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.347 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.347 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.347 INFO:teuthology.orchestra.run.smithi152.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-20T11:32:21.347 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-20T11:32:21.347 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-04-20T11:32:21.347 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.347 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.347 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "45", 2024-04-20T11:32:21.347 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.347 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.347 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.347 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.347 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.348 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.348 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.348 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.348 INFO:teuthology.orchestra.run.smithi152.stdout: "RGW_API_ACCESS_KEY": { 2024-04-20T11:32:21.348 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-20T11:32:21.348 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.348 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.348 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.348 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.348 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.348 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.348 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.348 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.348 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.349 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.349 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.349 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.349 INFO:teuthology.orchestra.run.smithi152.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-20T11:32:21.349 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-20T11:32:21.349 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.349 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.349 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.349 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "admin", 2024-04-20T11:32:21.349 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.349 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.349 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.349 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.349 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.350 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.350 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.350 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.350 INFO:teuthology.orchestra.run.smithi152.stdout: "RGW_API_SECRET_KEY": { 2024-04-20T11:32:21.350 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-20T11:32:21.350 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.350 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.350 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.350 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.350 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.350 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.350 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.350 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.350 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.351 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.351 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.351 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.351 INFO:teuthology.orchestra.run.smithi152.stdout: "RGW_API_SSL_VERIFY": { 2024-04-20T11:32:21.351 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-20T11:32:21.351 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.351 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.351 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.351 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-04-20T11:32:21.351 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.351 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.351 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.351 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.351 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.352 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.352 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.352 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.352 INFO:teuthology.orchestra.run.smithi152.stdout: "UNSAFE_TLS_v1_2": { 2024-04-20T11:32:21.352 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-20T11:32:21.352 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.352 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.352 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.352 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.352 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.352 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.352 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.352 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.352 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.352 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.353 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.353 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.353 INFO:teuthology.orchestra.run.smithi152.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-20T11:32:21.353 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-20T11:32:21.353 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-04-20T11:32:21.353 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.353 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.353 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "0", 2024-04-20T11:32:21.353 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.353 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.353 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.353 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.353 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.354 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.354 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.354 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.354 INFO:teuthology.orchestra.run.smithi152.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-20T11:32:21.354 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-20T11:32:21.354 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-04-20T11:32:21.354 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.354 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.354 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "10", 2024-04-20T11:32:21.354 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.354 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.354 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.354 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.354 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.355 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.355 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.355 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.355 INFO:teuthology.orchestra.run.smithi152.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-20T11:32:21.355 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-20T11:32:21.355 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-04-20T11:32:21.355 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.355 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.355 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "5", 2024-04-20T11:32:21.355 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.355 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.355 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.355 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.356 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.356 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.356 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.356 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.356 INFO:teuthology.orchestra.run.smithi152.stdout: "cross_origin_url": { 2024-04-20T11:32:21.356 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "cross_origin_url", 2024-04-20T11:32:21.356 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.356 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.356 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.356 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.356 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.356 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.356 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.356 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.356 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.357 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.357 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.357 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.357 INFO:teuthology.orchestra.run.smithi152.stdout: "crt_file": { 2024-04-20T11:32:21.357 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "crt_file", 2024-04-20T11:32:21.357 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.357 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.357 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.357 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.357 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.357 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.357 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.357 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.358 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.358 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.358 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.358 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.358 INFO:teuthology.orchestra.run.smithi152.stdout: "debug": { 2024-04-20T11:32:21.358 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "debug", 2024-04-20T11:32:21.358 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.358 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.358 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.358 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.358 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.358 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.358 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.358 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Enable/disable debug options", 2024-04-20T11:32:21.359 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.359 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.359 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.359 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.359 INFO:teuthology.orchestra.run.smithi152.stdout: "jwt_token_ttl": { 2024-04-20T11:32:21.359 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "jwt_token_ttl", 2024-04-20T11:32:21.359 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-04-20T11:32:21.359 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.359 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.359 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "28800", 2024-04-20T11:32:21.359 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.359 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.359 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.359 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.359 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.360 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.360 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.360 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.360 INFO:teuthology.orchestra.run.smithi152.stdout: "key_file": { 2024-04-20T11:32:21.360 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "key_file", 2024-04-20T11:32:21.360 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.360 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.360 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.360 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.360 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.360 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.360 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.360 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.360 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.361 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.361 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.361 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.361 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-04-20T11:32:21.361 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-04-20T11:32:21.361 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.361 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.361 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.361 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.361 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.361 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.361 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.361 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.361 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.362 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.362 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.362 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.362 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.362 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.362 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.362 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.362 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.362 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.362 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.362 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-04-20T11:32:21.362 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-04-20T11:32:21.362 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.362 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.363 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.363 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.363 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.363 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.363 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.363 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.363 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.363 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.363 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.363 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.363 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-04-20T11:32:21.363 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:21.363 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.363 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.364 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.364 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-04-20T11:32:21.364 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.364 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.364 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.364 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.364 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.364 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.364 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.364 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.364 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.364 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.364 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.364 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.365 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.365 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.365 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.365 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-04-20T11:32:21.365 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-04-20T11:32:21.365 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.365 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.365 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.365 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.365 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.365 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.365 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.365 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.365 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.366 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.366 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.366 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.366 INFO:teuthology.orchestra.run.smithi152.stdout: "motd": { 2024-04-20T11:32:21.366 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "motd", 2024-04-20T11:32:21.366 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.366 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.366 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.366 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.366 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.366 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.366 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.366 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "The message of the day", 2024-04-20T11:32:21.366 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.367 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.367 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.367 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.367 INFO:teuthology.orchestra.run.smithi152.stdout: "redirect_resolve_ip_addr": { 2024-04-20T11:32:21.367 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "redirect_resolve_ip_addr", 2024-04-20T11:32:21.367 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.367 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.367 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.367 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.367 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.367 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.367 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.367 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.368 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.368 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.368 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.368 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.368 INFO:teuthology.orchestra.run.smithi152.stdout: "server_addr": { 2024-04-20T11:32:21.368 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "server_addr", 2024-04-20T11:32:21.368 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.368 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.368 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.368 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "::", 2024-04-20T11:32:21.368 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.368 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.368 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.368 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.369 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.369 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.369 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.369 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.369 INFO:teuthology.orchestra.run.smithi152.stdout: "server_port": { 2024-04-20T11:32:21.369 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "server_port", 2024-04-20T11:32:21.369 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-04-20T11:32:21.369 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.369 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.369 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "8080", 2024-04-20T11:32:21.369 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.369 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.369 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.370 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.370 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.370 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.370 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.370 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.370 INFO:teuthology.orchestra.run.smithi152.stdout: "ssl": { 2024-04-20T11:32:21.370 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "ssl", 2024-04-20T11:32:21.370 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.370 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.370 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.370 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-04-20T11:32:21.370 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.370 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.370 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.371 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.371 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.371 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.371 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.371 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.371 INFO:teuthology.orchestra.run.smithi152.stdout: "ssl_server_port": { 2024-04-20T11:32:21.371 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "ssl_server_port", 2024-04-20T11:32:21.371 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-04-20T11:32:21.371 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.371 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.371 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "8443", 2024-04-20T11:32:21.371 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.371 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.371 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.371 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.372 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.372 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.372 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.372 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.372 INFO:teuthology.orchestra.run.smithi152.stdout: "standby_behaviour": { 2024-04-20T11:32:21.372 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "standby_behaviour", 2024-04-20T11:32:21.372 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.372 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.372 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.372 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "redirect", 2024-04-20T11:32:21.372 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.372 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.372 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.372 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.373 INFO:teuthology.orchestra.run.smithi152.stdout: "redirect" 2024-04-20T11:32:21.373 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.373 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.373 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.373 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.373 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.373 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.373 INFO:teuthology.orchestra.run.smithi152.stdout: "standby_error_status_code": { 2024-04-20T11:32:21.373 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "standby_error_status_code", 2024-04-20T11:32:21.373 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-04-20T11:32:21.373 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.373 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.373 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "500", 2024-04-20T11:32:21.373 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "400", 2024-04-20T11:32:21.374 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "599", 2024-04-20T11:32:21.374 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.374 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.374 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.374 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.374 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.374 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.374 INFO:teuthology.orchestra.run.smithi152.stdout: "url_prefix": { 2024-04-20T11:32:21.374 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "url_prefix", 2024-04-20T11:32:21.374 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.374 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.374 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.374 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.374 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.375 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.375 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.375 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.375 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.375 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.375 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.375 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.375 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.375 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.375 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:32:21.375 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "devicehealth", 2024-04-20T11:32:21.375 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-04-20T11:32:21.375 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-04-20T11:32:21.375 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-04-20T11:32:21.375 INFO:teuthology.orchestra.run.smithi152.stdout: "enable_monitoring": { 2024-04-20T11:32:21.376 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "enable_monitoring", 2024-04-20T11:32:21.376 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.376 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.376 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.376 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-04-20T11:32:21.376 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.376 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.376 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.376 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "monitor device health metrics", 2024-04-20T11:32:21.376 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.376 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.376 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.376 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.376 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-04-20T11:32:21.377 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-04-20T11:32:21.377 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.377 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.377 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.377 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.377 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.377 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.377 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.377 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.377 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.377 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.377 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.377 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.377 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.378 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.378 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.378 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.378 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.378 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.378 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.378 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-04-20T11:32:21.378 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-04-20T11:32:21.378 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.378 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.378 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.378 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.378 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.378 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.379 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.379 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.379 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.379 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.379 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.379 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.379 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-04-20T11:32:21.379 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:21.379 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.379 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.379 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.379 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-04-20T11:32:21.379 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.379 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.380 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.380 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.380 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.380 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.380 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.380 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.380 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.380 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.380 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.380 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.380 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.380 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.380 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.380 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-04-20T11:32:21.381 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-04-20T11:32:21.381 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.381 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.381 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.381 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.381 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.381 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.381 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.381 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.381 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.381 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.381 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.381 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.381 INFO:teuthology.orchestra.run.smithi152.stdout: "mark_out_threshold": { 2024-04-20T11:32:21.381 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "mark_out_threshold", 2024-04-20T11:32:21.382 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "secs", 2024-04-20T11:32:21.382 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.382 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.382 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "2419200", 2024-04-20T11:32:21.382 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.382 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.382 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.382 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-20T11:32:21.382 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.382 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.382 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.382 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.382 INFO:teuthology.orchestra.run.smithi152.stdout: "pool_name": { 2024-04-20T11:32:21.383 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "pool_name", 2024-04-20T11:32:21.383 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.383 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.383 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.383 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "device_health_metrics", 2024-04-20T11:32:21.383 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.383 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.383 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.383 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-20T11:32:21.383 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.383 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.383 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.383 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.383 INFO:teuthology.orchestra.run.smithi152.stdout: "retention_period": { 2024-04-20T11:32:21.383 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "retention_period", 2024-04-20T11:32:21.384 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "secs", 2024-04-20T11:32:21.384 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.384 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.384 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "15552000", 2024-04-20T11:32:21.384 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.384 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.384 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.384 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "how long to retain device health metrics", 2024-04-20T11:32:21.384 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.384 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.384 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.384 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.384 INFO:teuthology.orchestra.run.smithi152.stdout: "scrape_frequency": { 2024-04-20T11:32:21.384 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "scrape_frequency", 2024-04-20T11:32:21.384 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "secs", 2024-04-20T11:32:21.385 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.385 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.385 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "86400", 2024-04-20T11:32:21.385 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.385 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.385 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.385 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-20T11:32:21.385 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.385 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.385 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.385 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.385 INFO:teuthology.orchestra.run.smithi152.stdout: "self_heal": { 2024-04-20T11:32:21.385 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "self_heal", 2024-04-20T11:32:21.385 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.386 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.386 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.386 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-04-20T11:32:21.386 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.386 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.386 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.386 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-20T11:32:21.386 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.386 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.386 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.386 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.386 INFO:teuthology.orchestra.run.smithi152.stdout: "sleep_interval": { 2024-04-20T11:32:21.386 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "sleep_interval", 2024-04-20T11:32:21.386 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "secs", 2024-04-20T11:32:21.387 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.387 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.387 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "600", 2024-04-20T11:32:21.387 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.387 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.387 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.387 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "how frequently to wake up and check device health", 2024-04-20T11:32:21.387 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.387 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.387 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.387 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.387 INFO:teuthology.orchestra.run.smithi152.stdout: "warn_threshold": { 2024-04-20T11:32:21.387 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "warn_threshold", 2024-04-20T11:32:21.387 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "secs", 2024-04-20T11:32:21.387 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.388 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.388 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "7257600", 2024-04-20T11:32:21.388 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.388 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.388 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.388 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-20T11:32:21.388 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.388 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.388 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.388 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.388 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.388 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.388 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:32:21.389 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "diskprediction_local", 2024-04-20T11:32:21.389 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-04-20T11:32:21.389 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-04-20T11:32:21.389 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-04-20T11:32:21.389 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-04-20T11:32:21.389 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-04-20T11:32:21.389 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.389 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.389 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.389 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.389 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.389 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.389 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.389 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.390 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.390 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.390 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.390 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.390 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.390 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.390 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.390 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.390 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.390 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.390 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.390 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-04-20T11:32:21.390 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-04-20T11:32:21.390 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.391 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.391 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.391 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.391 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.391 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.391 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.391 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.391 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.391 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.391 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.391 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.391 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-04-20T11:32:21.391 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:21.391 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.391 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.392 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.392 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-04-20T11:32:21.392 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.392 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.392 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.392 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.392 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.392 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.392 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.392 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.392 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.392 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.392 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.392 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.393 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.393 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.393 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.393 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-04-20T11:32:21.393 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-04-20T11:32:21.393 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.393 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.393 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.393 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.393 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.393 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.393 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.393 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.393 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.393 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.394 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.394 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.394 INFO:teuthology.orchestra.run.smithi152.stdout: "predict_interval": { 2024-04-20T11:32:21.394 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "predict_interval", 2024-04-20T11:32:21.394 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.394 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.394 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.394 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "86400", 2024-04-20T11:32:21.394 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.394 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.394 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.394 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.394 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.394 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.395 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.395 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.395 INFO:teuthology.orchestra.run.smithi152.stdout: "predictor_model": { 2024-04-20T11:32:21.395 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "predictor_model", 2024-04-20T11:32:21.395 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.395 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.395 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.395 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "prophetstor", 2024-04-20T11:32:21.395 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.395 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.395 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.395 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.395 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.396 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.396 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.396 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.396 INFO:teuthology.orchestra.run.smithi152.stdout: "sleep_interval": { 2024-04-20T11:32:21.396 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "sleep_interval", 2024-04-20T11:32:21.396 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.396 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.396 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.396 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "600", 2024-04-20T11:32:21.396 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.396 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.396 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.396 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.396 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.397 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.397 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.397 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.397 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.397 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.397 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:32:21.397 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "influx", 2024-04-20T11:32:21.397 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": false, 2024-04-20T11:32:21.397 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "influxdb python module not found", 2024-04-20T11:32:21.397 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-04-20T11:32:21.397 INFO:teuthology.orchestra.run.smithi152.stdout: "batch_size": { 2024-04-20T11:32:21.397 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "batch_size", 2024-04-20T11:32:21.397 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-04-20T11:32:21.397 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.398 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.398 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "5000", 2024-04-20T11:32:21.398 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.398 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.398 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.398 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-20T11:32:21.398 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.398 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.398 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.398 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.398 INFO:teuthology.orchestra.run.smithi152.stdout: "database": { 2024-04-20T11:32:21.398 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "database", 2024-04-20T11:32:21.398 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.399 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.399 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.399 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "ceph", 2024-04-20T11:32:21.399 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.399 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.399 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.399 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-20T11:32:21.399 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.399 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.399 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.399 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.399 INFO:teuthology.orchestra.run.smithi152.stdout: "hostname": { 2024-04-20T11:32:21.399 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "hostname", 2024-04-20T11:32:21.399 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.400 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.400 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.400 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.400 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.400 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.400 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.400 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "InfluxDB server hostname", 2024-04-20T11:32:21.400 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.400 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.400 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.400 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.400 INFO:teuthology.orchestra.run.smithi152.stdout: "interval": { 2024-04-20T11:32:21.400 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "interval", 2024-04-20T11:32:21.400 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "secs", 2024-04-20T11:32:21.400 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.401 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.401 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "30", 2024-04-20T11:32:21.401 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "5", 2024-04-20T11:32:21.401 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.401 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.401 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-20T11:32:21.401 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.401 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.401 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.401 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.401 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-04-20T11:32:21.401 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-04-20T11:32:21.401 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.401 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.402 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.402 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.402 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.402 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.402 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.402 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.402 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.402 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.402 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.402 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.402 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.402 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.402 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.402 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.403 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.403 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.403 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.403 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-04-20T11:32:21.403 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-04-20T11:32:21.403 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.403 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.403 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.403 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.403 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.403 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.403 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.403 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.403 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.404 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.404 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.404 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.404 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-04-20T11:32:21.404 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:21.404 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.404 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.404 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.404 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-04-20T11:32:21.404 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.404 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.404 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.404 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.404 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.405 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.405 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.405 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.405 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.405 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.405 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.405 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.405 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.405 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.405 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.405 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-04-20T11:32:21.405 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-04-20T11:32:21.405 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.405 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.405 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.406 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.406 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.406 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.406 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.406 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.406 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.406 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.406 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.406 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.406 INFO:teuthology.orchestra.run.smithi152.stdout: "password": { 2024-04-20T11:32:21.406 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "password", 2024-04-20T11:32:21.406 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.406 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.406 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.407 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.407 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.407 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.407 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.407 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "password of InfluxDB server user", 2024-04-20T11:32:21.407 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.407 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.407 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.407 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.407 INFO:teuthology.orchestra.run.smithi152.stdout: "port": { 2024-04-20T11:32:21.407 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "port", 2024-04-20T11:32:21.407 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-04-20T11:32:21.407 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.408 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.408 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "8086", 2024-04-20T11:32:21.408 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.408 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.408 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.408 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "InfluxDB server port", 2024-04-20T11:32:21.408 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.408 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.408 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.408 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.408 INFO:teuthology.orchestra.run.smithi152.stdout: "ssl": { 2024-04-20T11:32:21.408 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "ssl", 2024-04-20T11:32:21.408 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.408 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.409 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.409 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "false", 2024-04-20T11:32:21.409 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.409 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.409 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.409 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-20T11:32:21.409 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.409 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.409 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.409 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.409 INFO:teuthology.orchestra.run.smithi152.stdout: "threads": { 2024-04-20T11:32:21.409 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "threads", 2024-04-20T11:32:21.409 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-04-20T11:32:21.410 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.410 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.410 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "5", 2024-04-20T11:32:21.410 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "1", 2024-04-20T11:32:21.410 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "32", 2024-04-20T11:32:21.410 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.410 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-20T11:32:21.410 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.410 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.410 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.410 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.410 INFO:teuthology.orchestra.run.smithi152.stdout: "username": { 2024-04-20T11:32:21.410 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "username", 2024-04-20T11:32:21.410 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.411 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.411 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.411 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.411 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.411 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.411 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.411 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "username of InfluxDB server user", 2024-04-20T11:32:21.411 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.411 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.411 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.411 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.411 INFO:teuthology.orchestra.run.smithi152.stdout: "verify_ssl": { 2024-04-20T11:32:21.411 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "verify_ssl", 2024-04-20T11:32:21.411 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.412 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.412 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.412 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "true", 2024-04-20T11:32:21.412 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.412 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.412 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.412 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-20T11:32:21.412 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.412 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.412 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.412 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.412 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.412 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.413 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:32:21.413 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "insights", 2024-04-20T11:32:21.413 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-04-20T11:32:21.413 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-04-20T11:32:21.413 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-04-20T11:32:21.413 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-04-20T11:32:21.413 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-04-20T11:32:21.413 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.413 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.413 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.413 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.413 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.413 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.413 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.413 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.414 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.414 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.414 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.414 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.414 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.414 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.414 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.414 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.414 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.414 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.414 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.414 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-04-20T11:32:21.414 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-04-20T11:32:21.414 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.415 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.415 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.415 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.415 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.415 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.415 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.415 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.415 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.415 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.415 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.415 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.415 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-04-20T11:32:21.415 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:21.415 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.415 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.416 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.416 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-04-20T11:32:21.416 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.416 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.416 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.416 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.416 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.416 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.416 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.416 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.416 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.416 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.416 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.417 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.417 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.417 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.417 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.417 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-04-20T11:32:21.417 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-04-20T11:32:21.417 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.417 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.417 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.417 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.417 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.417 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.417 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.417 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.418 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.418 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.418 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.418 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.418 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.418 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.418 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:32:21.418 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "iostat", 2024-04-20T11:32:21.418 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-04-20T11:32:21.418 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-04-20T11:32:21.418 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-04-20T11:32:21.418 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-04-20T11:32:21.418 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-04-20T11:32:21.418 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.419 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.419 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.419 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.419 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.419 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.419 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.419 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.419 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.419 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.419 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.419 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.419 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.419 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.419 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.420 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.420 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.420 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.420 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.420 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-04-20T11:32:21.420 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-04-20T11:32:21.420 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.420 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.420 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.420 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.420 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.420 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.420 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.420 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.421 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.421 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.421 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.421 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.421 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-04-20T11:32:21.421 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:21.421 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.421 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.421 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.421 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-04-20T11:32:21.421 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.421 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.421 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.421 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.421 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.422 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.422 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.422 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.422 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.422 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.422 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.422 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.422 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.422 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.422 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.422 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-04-20T11:32:21.422 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-04-20T11:32:21.422 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.423 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.423 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.423 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.423 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.423 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.423 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.423 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.423 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.423 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.423 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.423 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.423 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.423 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.423 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:32:21.424 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "localpool", 2024-04-20T11:32:21.424 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-04-20T11:32:21.424 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-04-20T11:32:21.424 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-04-20T11:32:21.424 INFO:teuthology.orchestra.run.smithi152.stdout: "failure_domain": { 2024-04-20T11:32:21.424 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "failure_domain", 2024-04-20T11:32:21.424 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.424 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.424 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.424 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "host", 2024-04-20T11:32:21.424 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.424 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.424 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.424 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "failure domain for any created local pool", 2024-04-20T11:32:21.424 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-20T11:32:21.425 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.425 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.425 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.425 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-04-20T11:32:21.425 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-04-20T11:32:21.425 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.425 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.425 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.425 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.425 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.425 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.425 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.425 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.425 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.425 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.426 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.426 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.426 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.426 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.426 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.426 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.426 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.426 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.426 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.426 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-04-20T11:32:21.426 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-04-20T11:32:21.426 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.426 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.427 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.427 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.427 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.427 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.427 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.427 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.427 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.427 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.427 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.427 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.427 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-04-20T11:32:21.427 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:21.427 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.427 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.427 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.428 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-04-20T11:32:21.428 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.428 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.428 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.428 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.428 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.428 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.428 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.428 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.428 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.428 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.428 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.428 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.428 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.429 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.429 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.429 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-04-20T11:32:21.429 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-04-20T11:32:21.429 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.429 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.429 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.429 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.429 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.429 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.429 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.429 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.429 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.429 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.430 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.430 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.430 INFO:teuthology.orchestra.run.smithi152.stdout: "min_size": { 2024-04-20T11:32:21.430 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "min_size", 2024-04-20T11:32:21.430 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-04-20T11:32:21.430 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.430 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.430 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.430 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.430 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.430 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.430 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "default min_size for any created local pool", 2024-04-20T11:32:21.430 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-20T11:32:21.430 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.431 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.431 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.431 INFO:teuthology.orchestra.run.smithi152.stdout: "num_rep": { 2024-04-20T11:32:21.431 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "num_rep", 2024-04-20T11:32:21.431 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-04-20T11:32:21.431 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.431 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.431 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "3", 2024-04-20T11:32:21.431 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.431 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.431 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.431 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "default replica count for any created local pool", 2024-04-20T11:32:21.431 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.432 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.432 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.432 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.432 INFO:teuthology.orchestra.run.smithi152.stdout: "pg_num": { 2024-04-20T11:32:21.432 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "pg_num", 2024-04-20T11:32:21.432 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-04-20T11:32:21.432 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.432 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.432 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "128", 2024-04-20T11:32:21.432 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.432 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.432 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.432 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "default pg_num for any created local pool", 2024-04-20T11:32:21.433 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.433 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.433 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.433 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.433 INFO:teuthology.orchestra.run.smithi152.stdout: "prefix": { 2024-04-20T11:32:21.433 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "prefix", 2024-04-20T11:32:21.433 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.433 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.433 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.433 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.433 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.433 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.433 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.433 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "name prefix for any created local pool", 2024-04-20T11:32:21.433 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.434 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.434 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.434 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.434 INFO:teuthology.orchestra.run.smithi152.stdout: "subtree": { 2024-04-20T11:32:21.434 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "subtree", 2024-04-20T11:32:21.434 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.434 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.434 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.434 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "rack", 2024-04-20T11:32:21.434 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.434 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.434 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.434 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-20T11:32:21.435 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-20T11:32:21.435 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.435 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.435 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.435 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.435 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.435 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:32:21.435 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "mds_autoscaler", 2024-04-20T11:32:21.435 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-04-20T11:32:21.435 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-04-20T11:32:21.435 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-04-20T11:32:21.435 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-04-20T11:32:21.435 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-04-20T11:32:21.435 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.436 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.436 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.436 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.436 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.436 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.436 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.436 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.436 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.436 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.436 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.436 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.436 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.436 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.436 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.437 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.437 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.437 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.437 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.437 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-04-20T11:32:21.437 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-04-20T11:32:21.437 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.437 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.437 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.437 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.437 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.437 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.437 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.437 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.438 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.438 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.438 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.438 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.438 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-04-20T11:32:21.438 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:21.438 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.438 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.438 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.438 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-04-20T11:32:21.438 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.438 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.438 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.438 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.439 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.439 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.439 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.439 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.439 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.439 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.439 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.439 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.439 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.439 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.439 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.439 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-04-20T11:32:21.439 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-04-20T11:32:21.439 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.440 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.440 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.440 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.440 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.440 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.440 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.440 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.440 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.440 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.440 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.440 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.440 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.440 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.440 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:32:21.440 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "mirroring", 2024-04-20T11:32:21.441 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-04-20T11:32:21.441 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-04-20T11:32:21.441 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-04-20T11:32:21.441 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-04-20T11:32:21.441 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-04-20T11:32:21.441 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.441 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.441 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.441 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.441 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.441 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.441 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.441 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.442 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.442 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.442 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.442 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.442 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.442 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.442 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.442 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.442 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.442 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.442 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.442 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-04-20T11:32:21.442 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-04-20T11:32:21.443 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.443 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.443 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.443 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.443 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.443 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.443 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.443 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.443 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.443 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.443 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.443 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.443 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-04-20T11:32:21.443 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:21.444 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.444 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.444 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.444 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-04-20T11:32:21.444 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.444 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.444 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.444 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.444 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.444 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.444 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.444 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.444 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.444 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.445 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.445 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.445 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.445 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.445 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.445 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-04-20T11:32:21.445 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-04-20T11:32:21.445 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.445 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.445 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.445 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.445 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.445 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.445 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.446 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.446 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.446 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.446 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.446 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.446 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.446 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.446 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:32:21.446 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "nfs", 2024-04-20T11:32:21.446 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-04-20T11:32:21.446 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-04-20T11:32:21.446 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-04-20T11:32:21.446 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-04-20T11:32:21.446 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-04-20T11:32:21.447 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.447 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.447 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.447 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.447 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.447 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.447 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.447 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.447 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.447 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.447 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.447 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.447 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.447 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.448 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.448 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.448 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.448 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.448 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.448 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-04-20T11:32:21.448 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-04-20T11:32:21.448 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.448 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.448 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.448 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.448 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.448 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.448 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.449 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.449 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.449 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.449 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.449 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.449 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-04-20T11:32:21.449 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:21.449 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.449 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.449 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.449 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-04-20T11:32:21.449 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.449 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.449 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.449 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.450 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.450 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.450 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.450 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.450 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.450 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.450 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.450 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.450 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.450 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.450 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.450 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-04-20T11:32:21.450 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-04-20T11:32:21.451 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.451 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.451 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.451 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.451 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.451 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.451 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.451 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.451 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.451 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.451 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.451 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.451 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.451 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.451 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:32:21.452 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "orchestrator", 2024-04-20T11:32:21.452 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-04-20T11:32:21.452 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-04-20T11:32:21.452 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-04-20T11:32:21.452 INFO:teuthology.orchestra.run.smithi152.stdout: "fail_fs": { 2024-04-20T11:32:21.452 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "fail_fs", 2024-04-20T11:32:21.452 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.452 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.452 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.452 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.452 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.452 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.452 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.452 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-20T11:32:21.453 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.453 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.453 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.453 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.453 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-04-20T11:32:21.453 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-04-20T11:32:21.453 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.453 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.453 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.453 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.453 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.453 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.453 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.453 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.453 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.454 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.454 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.454 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.454 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.454 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.454 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.454 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.454 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.454 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.454 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.454 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-04-20T11:32:21.454 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-04-20T11:32:21.454 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.454 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.455 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.455 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.455 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.455 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.455 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.455 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.455 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.455 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.455 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.455 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.455 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-04-20T11:32:21.455 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:21.455 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.455 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.456 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.456 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-04-20T11:32:21.456 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.456 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.456 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.456 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.456 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.456 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.456 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.456 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.456 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.456 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.456 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.456 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.456 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.457 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.457 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.457 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-04-20T11:32:21.457 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-04-20T11:32:21.457 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.457 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.457 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.457 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.457 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.457 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.457 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.457 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.457 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.457 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.458 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.458 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.458 INFO:teuthology.orchestra.run.smithi152.stdout: "orchestrator": { 2024-04-20T11:32:21.458 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "orchestrator", 2024-04-20T11:32:21.458 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.458 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.458 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.458 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.458 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.458 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.458 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.458 INFO:teuthology.orchestra.run.smithi152.stdout: "cephadm", 2024-04-20T11:32:21.458 INFO:teuthology.orchestra.run.smithi152.stdout: "rook", 2024-04-20T11:32:21.458 INFO:teuthology.orchestra.run.smithi152.stdout: "test_orchestrator" 2024-04-20T11:32:21.459 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.459 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Orchestrator backend", 2024-04-20T11:32:21.459 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.459 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.459 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.459 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.459 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.459 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.459 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:32:21.459 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "osd_perf_query", 2024-04-20T11:32:21.459 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-04-20T11:32:21.459 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-04-20T11:32:21.459 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-04-20T11:32:21.459 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-04-20T11:32:21.459 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-04-20T11:32:21.460 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.460 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.460 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.460 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.460 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.460 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.460 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.460 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.460 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.460 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.460 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.460 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.460 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.460 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.461 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.461 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.461 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.461 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.461 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.461 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-04-20T11:32:21.461 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-04-20T11:32:21.461 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.461 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.461 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.461 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.461 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.461 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.461 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.462 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.462 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.462 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.462 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.462 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.462 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-04-20T11:32:21.462 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:21.462 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.462 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.462 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.462 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-04-20T11:32:21.462 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.462 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.462 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.463 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.463 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.463 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.463 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.463 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.463 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.463 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.463 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.463 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.463 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.463 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.463 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.463 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-04-20T11:32:21.463 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-04-20T11:32:21.464 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.464 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.464 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.464 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.464 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.464 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.464 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.464 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.464 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.464 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.464 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.464 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.464 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.464 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.465 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:32:21.465 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "osd_support", 2024-04-20T11:32:21.465 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-04-20T11:32:21.465 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-04-20T11:32:21.465 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-04-20T11:32:21.465 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-04-20T11:32:21.465 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-04-20T11:32:21.465 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.465 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.465 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.465 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.465 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.465 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.465 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.466 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.466 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.466 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.466 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.466 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.466 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.466 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.466 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.466 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.466 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.466 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.466 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.466 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-04-20T11:32:21.466 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-04-20T11:32:21.467 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.467 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.467 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.467 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.467 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.467 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.467 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.467 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.467 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.467 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.467 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.467 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.467 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-04-20T11:32:21.467 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:21.468 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.468 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.468 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.468 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-04-20T11:32:21.468 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.468 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.468 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.468 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.468 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.468 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.468 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.468 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.468 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.468 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.468 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.469 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.469 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.469 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.469 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.469 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-04-20T11:32:21.469 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-04-20T11:32:21.469 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.469 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.469 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.469 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.469 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.469 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.469 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.469 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.469 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.470 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.470 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.470 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.470 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.470 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.470 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:32:21.470 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "pg_autoscaler", 2024-04-20T11:32:21.470 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-04-20T11:32:21.470 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-04-20T11:32:21.470 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-04-20T11:32:21.470 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-04-20T11:32:21.470 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-04-20T11:32:21.470 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.470 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.471 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.471 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.471 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.471 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.471 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.471 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.471 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.471 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.471 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.471 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.471 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.471 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.471 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.471 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.471 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.472 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.472 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.472 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-04-20T11:32:21.472 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-04-20T11:32:21.472 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.472 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.472 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.472 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.472 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.472 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.472 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.472 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.472 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.472 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.473 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.473 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.473 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-04-20T11:32:21.473 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:21.473 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.473 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.473 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.473 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-04-20T11:32:21.473 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.473 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.473 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.473 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.473 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.473 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.473 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.473 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.474 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.474 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.474 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.474 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.474 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.474 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.474 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.474 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-04-20T11:32:21.474 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-04-20T11:32:21.474 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.474 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.474 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.474 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.474 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.475 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.475 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.475 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.475 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.475 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.475 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.475 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.475 INFO:teuthology.orchestra.run.smithi152.stdout: "sleep_interval": { 2024-04-20T11:32:21.475 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "sleep_interval", 2024-04-20T11:32:21.475 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "secs", 2024-04-20T11:32:21.475 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.475 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.475 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "60", 2024-04-20T11:32:21.475 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.476 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.476 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.476 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.476 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.476 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.476 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.476 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.476 INFO:teuthology.orchestra.run.smithi152.stdout: "threshold": { 2024-04-20T11:32:21.476 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "threshold", 2024-04-20T11:32:21.476 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "float", 2024-04-20T11:32:21.476 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.476 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.476 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "3.0", 2024-04-20T11:32:21.476 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "1.0", 2024-04-20T11:32:21.477 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.477 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.477 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "scaling threshold", 2024-04-20T11:32:21.477 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-20T11:32:21.477 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.477 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.477 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.477 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.477 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.477 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:32:21.477 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "progress", 2024-04-20T11:32:21.477 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-04-20T11:32:21.477 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-04-20T11:32:21.478 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-04-20T11:32:21.478 INFO:teuthology.orchestra.run.smithi152.stdout: "allow_pg_recovery_event": { 2024-04-20T11:32:21.478 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "allow_pg_recovery_event", 2024-04-20T11:32:21.478 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.478 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.478 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.478 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.478 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.478 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.478 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.478 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-20T11:32:21.478 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.478 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.478 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.479 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.479 INFO:teuthology.orchestra.run.smithi152.stdout: "enabled": { 2024-04-20T11:32:21.479 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "enabled", 2024-04-20T11:32:21.479 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.479 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.479 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.479 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-04-20T11:32:21.479 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.479 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.479 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.479 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.479 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.479 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.479 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.480 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.480 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-04-20T11:32:21.480 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-04-20T11:32:21.480 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.480 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.480 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.480 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.480 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.480 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.480 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.480 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.480 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.480 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.480 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.480 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.481 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.481 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.481 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.481 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.481 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.481 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.481 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.481 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-04-20T11:32:21.481 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-04-20T11:32:21.481 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.481 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.481 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.481 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.481 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.481 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.482 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.482 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.482 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.482 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.482 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.482 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.482 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-04-20T11:32:21.482 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:21.482 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.482 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.482 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.482 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-04-20T11:32:21.482 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.482 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.483 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.483 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.483 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.483 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.483 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.483 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.483 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.483 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.483 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.483 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.483 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.483 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.483 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.483 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-04-20T11:32:21.484 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-04-20T11:32:21.484 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.484 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.484 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.484 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.484 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.484 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.484 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.484 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.484 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.484 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.484 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.484 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.484 INFO:teuthology.orchestra.run.smithi152.stdout: "max_completed_events": { 2024-04-20T11:32:21.484 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "max_completed_events", 2024-04-20T11:32:21.485 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-04-20T11:32:21.485 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.485 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.485 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "50", 2024-04-20T11:32:21.485 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.485 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.485 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.485 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "number of past completed events to remember", 2024-04-20T11:32:21.485 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.485 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.485 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.485 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.485 INFO:teuthology.orchestra.run.smithi152.stdout: "sleep_interval": { 2024-04-20T11:32:21.485 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "sleep_interval", 2024-04-20T11:32:21.486 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "secs", 2024-04-20T11:32:21.486 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.486 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.486 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "5", 2024-04-20T11:32:21.486 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.486 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.486 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.486 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "how long the module is going to sleep", 2024-04-20T11:32:21.486 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.486 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.486 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.486 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.486 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.486 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.487 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:32:21.487 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "prometheus", 2024-04-20T11:32:21.487 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-04-20T11:32:21.487 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-04-20T11:32:21.487 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-04-20T11:32:21.487 INFO:teuthology.orchestra.run.smithi152.stdout: "cache": { 2024-04-20T11:32:21.487 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "cache", 2024-04-20T11:32:21.487 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.487 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.487 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.487 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-04-20T11:32:21.487 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.487 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.487 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.487 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.488 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.488 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.488 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.488 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.488 INFO:teuthology.orchestra.run.smithi152.stdout: "exclude_perf_counters": { 2024-04-20T11:32:21.488 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "exclude_perf_counters", 2024-04-20T11:32:21.488 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.488 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.488 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.488 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-04-20T11:32:21.488 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.488 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.488 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.488 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-20T11:32:21.488 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-20T11:32:21.489 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.489 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.489 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.489 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-04-20T11:32:21.489 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-04-20T11:32:21.489 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.489 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.489 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.489 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.489 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.489 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.489 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.489 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.489 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.490 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.490 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.490 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.490 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.490 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.490 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.490 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.490 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.490 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.490 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.490 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-04-20T11:32:21.490 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-04-20T11:32:21.490 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.491 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.491 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.491 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.491 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.491 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.491 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.491 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.491 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.491 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.491 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.491 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.491 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-04-20T11:32:21.491 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:21.491 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.491 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.492 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.492 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-04-20T11:32:21.492 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.492 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.492 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.492 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.492 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.492 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.492 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.492 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.492 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.492 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.492 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.492 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.493 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.493 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.493 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.493 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-04-20T11:32:21.493 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-04-20T11:32:21.493 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.493 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.493 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.493 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.493 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.493 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.493 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.493 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.493 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.493 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.494 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.494 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.494 INFO:teuthology.orchestra.run.smithi152.stdout: "rbd_stats_pools": { 2024-04-20T11:32:21.494 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "rbd_stats_pools", 2024-04-20T11:32:21.494 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.494 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.494 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.494 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.494 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.494 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.494 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.494 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.494 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.494 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.495 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.495 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.495 INFO:teuthology.orchestra.run.smithi152.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-20T11:32:21.495 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-20T11:32:21.495 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-04-20T11:32:21.495 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.495 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.495 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "300", 2024-04-20T11:32:21.495 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.495 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.495 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.495 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.495 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.495 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.496 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.496 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.496 INFO:teuthology.orchestra.run.smithi152.stdout: "scrape_interval": { 2024-04-20T11:32:21.496 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "scrape_interval", 2024-04-20T11:32:21.496 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "float", 2024-04-20T11:32:21.496 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.496 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.496 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "15.0", 2024-04-20T11:32:21.496 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.496 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.496 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.496 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.496 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.496 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.497 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.497 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.497 INFO:teuthology.orchestra.run.smithi152.stdout: "server_addr": { 2024-04-20T11:32:21.497 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "server_addr", 2024-04-20T11:32:21.497 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.497 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.497 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.497 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "::", 2024-04-20T11:32:21.497 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.497 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.497 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.497 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-20T11:32:21.497 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.497 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.498 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.498 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.498 INFO:teuthology.orchestra.run.smithi152.stdout: "server_port": { 2024-04-20T11:32:21.498 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "server_port", 2024-04-20T11:32:21.498 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-04-20T11:32:21.498 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.498 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.498 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "9283", 2024-04-20T11:32:21.498 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.498 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.498 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.498 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-20T11:32:21.498 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.499 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.499 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.499 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.499 INFO:teuthology.orchestra.run.smithi152.stdout: "stale_cache_strategy": { 2024-04-20T11:32:21.499 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "stale_cache_strategy", 2024-04-20T11:32:21.499 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.499 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.499 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.499 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "log", 2024-04-20T11:32:21.499 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.499 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.499 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.499 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.499 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.500 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.500 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.500 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.500 INFO:teuthology.orchestra.run.smithi152.stdout: "standby_behaviour": { 2024-04-20T11:32:21.500 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "standby_behaviour", 2024-04-20T11:32:21.500 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.500 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.500 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.500 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "default", 2024-04-20T11:32:21.500 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.500 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.500 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.500 INFO:teuthology.orchestra.run.smithi152.stdout: "default", 2024-04-20T11:32:21.500 INFO:teuthology.orchestra.run.smithi152.stdout: "error" 2024-04-20T11:32:21.501 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.501 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.501 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.501 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.501 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.501 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.501 INFO:teuthology.orchestra.run.smithi152.stdout: "standby_error_status_code": { 2024-04-20T11:32:21.501 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "standby_error_status_code", 2024-04-20T11:32:21.501 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-04-20T11:32:21.501 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.501 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.501 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "500", 2024-04-20T11:32:21.501 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "400", 2024-04-20T11:32:21.501 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "599", 2024-04-20T11:32:21.501 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.502 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.502 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.502 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.502 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.502 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.502 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.502 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.502 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:32:21.502 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "rbd_support", 2024-04-20T11:32:21.502 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-04-20T11:32:21.502 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-04-20T11:32:21.502 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-04-20T11:32:21.502 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-04-20T11:32:21.503 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-04-20T11:32:21.503 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.503 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.503 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.503 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.503 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.503 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.503 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.503 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.503 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.503 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.503 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.503 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.504 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.504 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.504 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.504 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.504 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.504 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.504 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.504 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-04-20T11:32:21.504 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-04-20T11:32:21.504 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.504 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.504 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.505 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.505 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.505 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.505 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.505 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.505 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.505 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.505 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.505 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.505 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-04-20T11:32:21.505 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:21.505 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.505 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.506 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.506 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-04-20T11:32:21.506 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.506 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.506 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.506 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.506 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.506 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.506 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.506 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.506 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.506 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.506 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.507 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.507 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.507 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.507 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.507 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-04-20T11:32:21.507 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-04-20T11:32:21.507 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.507 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.507 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.507 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.507 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.507 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.507 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.507 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.508 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.508 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.508 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.508 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.508 INFO:teuthology.orchestra.run.smithi152.stdout: "max_concurrent_snap_create": { 2024-04-20T11:32:21.508 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "max_concurrent_snap_create", 2024-04-20T11:32:21.508 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-04-20T11:32:21.508 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.508 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.508 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "10", 2024-04-20T11:32:21.508 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.508 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.508 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.508 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.509 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.509 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.509 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.509 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.509 INFO:teuthology.orchestra.run.smithi152.stdout: "mirror_snapshot_schedule": { 2024-04-20T11:32:21.509 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "mirror_snapshot_schedule", 2024-04-20T11:32:21.509 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.509 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.509 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.509 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.509 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.509 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.509 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.509 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.510 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.510 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.510 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.510 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.510 INFO:teuthology.orchestra.run.smithi152.stdout: "trash_purge_schedule": { 2024-04-20T11:32:21.510 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "trash_purge_schedule", 2024-04-20T11:32:21.510 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.510 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.510 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.510 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.510 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.510 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.510 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.511 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.511 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.511 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.511 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.511 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.511 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.511 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.511 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:32:21.511 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "restful", 2024-04-20T11:32:21.511 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-04-20T11:32:21.511 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-04-20T11:32:21.511 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-04-20T11:32:21.511 INFO:teuthology.orchestra.run.smithi152.stdout: "enable_auth": { 2024-04-20T11:32:21.511 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "enable_auth", 2024-04-20T11:32:21.512 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.512 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.512 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.512 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-04-20T11:32:21.512 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.512 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.512 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.512 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.512 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.512 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.512 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.512 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.512 INFO:teuthology.orchestra.run.smithi152.stdout: "key_file": { 2024-04-20T11:32:21.513 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "key_file", 2024-04-20T11:32:21.513 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.513 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.513 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.513 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.513 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.513 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.513 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.513 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.513 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.513 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.513 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.513 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.513 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-04-20T11:32:21.514 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-04-20T11:32:21.514 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.514 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.514 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.514 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.514 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.514 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.514 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.514 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.514 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.514 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.514 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.514 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.515 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.515 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.515 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.515 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.515 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.515 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.515 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.515 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-04-20T11:32:21.515 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-04-20T11:32:21.515 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.515 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.515 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.515 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.515 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.516 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.516 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.516 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.516 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.516 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.516 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.516 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.516 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-04-20T11:32:21.516 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:21.516 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.516 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.516 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.516 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-04-20T11:32:21.516 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.517 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.517 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.517 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.517 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.517 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.517 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.517 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.517 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.517 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.517 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.517 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.517 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.517 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.517 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.518 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-04-20T11:32:21.518 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-04-20T11:32:21.518 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.518 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.518 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.518 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.518 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.518 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.518 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.518 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.518 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.518 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.518 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.519 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.519 INFO:teuthology.orchestra.run.smithi152.stdout: "server_addr": { 2024-04-20T11:32:21.519 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "server_addr", 2024-04-20T11:32:21.519 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.519 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.519 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.519 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.519 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.519 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.519 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.519 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.519 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.521 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.521 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.521 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.521 INFO:teuthology.orchestra.run.smithi152.stdout: "server_port": { 2024-04-20T11:32:21.521 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "server_port", 2024-04-20T11:32:21.521 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.521 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.521 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.521 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.521 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.521 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.522 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.522 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.522 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.522 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.522 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.522 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.522 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.522 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.522 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:32:21.522 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "rgw", 2024-04-20T11:32:21.522 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-04-20T11:32:21.522 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-04-20T11:32:21.522 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-04-20T11:32:21.522 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-04-20T11:32:21.523 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-04-20T11:32:21.523 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.523 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.523 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.523 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.523 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.523 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.523 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.523 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.523 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.523 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.523 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.523 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.523 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.523 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.524 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.524 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.524 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.524 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.524 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.524 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-04-20T11:32:21.524 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-04-20T11:32:21.524 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.524 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.524 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.524 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.524 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.524 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.524 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.525 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.525 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.525 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.525 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.525 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.525 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-04-20T11:32:21.525 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:21.525 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.525 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.525 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.525 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-04-20T11:32:21.525 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.525 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.525 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.526 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.526 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.526 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.526 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.526 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.526 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.526 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.526 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.526 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.526 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.526 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.526 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.526 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-04-20T11:32:21.526 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-04-20T11:32:21.527 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.527 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.527 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.527 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.527 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.527 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.527 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.527 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.527 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.527 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.527 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.527 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.527 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.527 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.528 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:32:21.528 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "rook", 2024-04-20T11:32:21.528 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-04-20T11:32:21.528 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-04-20T11:32:21.528 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-04-20T11:32:21.528 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-04-20T11:32:21.528 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-04-20T11:32:21.528 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.528 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.528 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.528 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.528 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.528 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.528 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.529 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.529 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.529 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.529 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.529 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.529 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.529 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.529 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.529 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.529 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.529 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.529 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.529 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-04-20T11:32:21.530 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-04-20T11:32:21.530 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.530 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.530 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.530 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.530 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.530 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.530 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.530 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.530 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.530 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.530 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.530 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.530 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-04-20T11:32:21.531 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:21.531 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.531 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.531 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.531 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-04-20T11:32:21.531 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.531 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.531 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.531 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.531 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.531 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.531 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.531 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.532 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.532 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.532 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.532 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.532 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.532 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.532 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.532 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-04-20T11:32:21.532 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-04-20T11:32:21.532 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.532 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.532 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.532 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.532 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.533 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.533 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.533 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.533 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.533 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.533 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.533 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.533 INFO:teuthology.orchestra.run.smithi152.stdout: "storage_class": { 2024-04-20T11:32:21.533 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "storage_class", 2024-04-20T11:32:21.533 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.533 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.533 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.533 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "local", 2024-04-20T11:32:21.534 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.534 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.534 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.534 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-20T11:32:21.534 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.534 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.534 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.534 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.534 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.534 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.534 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:32:21.534 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "selftest", 2024-04-20T11:32:21.534 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-04-20T11:32:21.535 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-04-20T11:32:21.535 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-04-20T11:32:21.535 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-04-20T11:32:21.535 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-04-20T11:32:21.535 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.535 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.535 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.535 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.535 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.535 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.535 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.535 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.535 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.535 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.536 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.536 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.536 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.536 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.536 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.536 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.536 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.536 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.536 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.536 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-04-20T11:32:21.536 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-04-20T11:32:21.536 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.536 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.536 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.536 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.537 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.537 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.537 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.537 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.537 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.537 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.537 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.537 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.537 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-04-20T11:32:21.537 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:21.537 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.537 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.537 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.537 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-04-20T11:32:21.537 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.538 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.538 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.538 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.538 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.538 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.538 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.538 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.538 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.538 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.538 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.538 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.538 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.538 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.538 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.539 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-04-20T11:32:21.539 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-04-20T11:32:21.539 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.539 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.539 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.539 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.539 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.539 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.539 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.539 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.539 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.539 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.539 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.539 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.539 INFO:teuthology.orchestra.run.smithi152.stdout: "roption1": { 2024-04-20T11:32:21.539 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "roption1", 2024-04-20T11:32:21.540 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.540 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.540 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.540 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.540 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.540 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.540 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.540 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.540 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.540 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.540 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.540 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.540 INFO:teuthology.orchestra.run.smithi152.stdout: "roption2": { 2024-04-20T11:32:21.540 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "roption2", 2024-04-20T11:32:21.541 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.541 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.541 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.541 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "xyz", 2024-04-20T11:32:21.541 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.541 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.541 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.541 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.541 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.541 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.541 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.541 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.541 INFO:teuthology.orchestra.run.smithi152.stdout: "rwoption1": { 2024-04-20T11:32:21.541 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "rwoption1", 2024-04-20T11:32:21.541 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.542 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.542 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.542 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.542 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.542 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.542 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.542 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.542 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.542 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.542 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.542 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.542 INFO:teuthology.orchestra.run.smithi152.stdout: "rwoption2": { 2024-04-20T11:32:21.542 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "rwoption2", 2024-04-20T11:32:21.543 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-04-20T11:32:21.543 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.543 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.543 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.543 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.543 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.543 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.543 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.543 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.543 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.543 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.543 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.543 INFO:teuthology.orchestra.run.smithi152.stdout: "rwoption3": { 2024-04-20T11:32:21.543 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "rwoption3", 2024-04-20T11:32:21.544 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "float", 2024-04-20T11:32:21.544 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.544 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.544 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.544 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.544 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.544 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.544 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.544 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.544 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.544 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.544 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.544 INFO:teuthology.orchestra.run.smithi152.stdout: "rwoption4": { 2024-04-20T11:32:21.544 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "rwoption4", 2024-04-20T11:32:21.545 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.545 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.545 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.545 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.545 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.545 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.545 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.545 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.545 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.545 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.545 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.545 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.545 INFO:teuthology.orchestra.run.smithi152.stdout: "rwoption5": { 2024-04-20T11:32:21.546 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "rwoption5", 2024-04-20T11:32:21.546 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.546 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.546 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.546 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.546 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.546 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.546 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.546 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.546 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.546 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.546 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.546 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.547 INFO:teuthology.orchestra.run.smithi152.stdout: "rwoption6": { 2024-04-20T11:32:21.547 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "rwoption6", 2024-04-20T11:32:21.547 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.547 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.547 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.547 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-04-20T11:32:21.547 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.547 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.547 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.547 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.547 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.547 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.547 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.548 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.548 INFO:teuthology.orchestra.run.smithi152.stdout: "rwoption7": { 2024-04-20T11:32:21.548 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "rwoption7", 2024-04-20T11:32:21.548 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-04-20T11:32:21.548 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.548 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.548 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.548 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "1", 2024-04-20T11:32:21.548 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "42", 2024-04-20T11:32:21.548 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.548 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.548 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.548 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.549 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.549 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.549 INFO:teuthology.orchestra.run.smithi152.stdout: "testkey": { 2024-04-20T11:32:21.549 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "testkey", 2024-04-20T11:32:21.549 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.549 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.549 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.549 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.549 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.549 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.549 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.549 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.549 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.549 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.549 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.550 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.550 INFO:teuthology.orchestra.run.smithi152.stdout: "testlkey": { 2024-04-20T11:32:21.550 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "testlkey", 2024-04-20T11:32:21.550 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.550 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.550 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.550 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.550 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.550 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.550 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.550 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.550 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.550 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.550 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.551 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.551 INFO:teuthology.orchestra.run.smithi152.stdout: "testnewline": { 2024-04-20T11:32:21.551 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "testnewline", 2024-04-20T11:32:21.551 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.551 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.551 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.551 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.551 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.551 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.551 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.551 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.551 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.551 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.552 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.552 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.552 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.552 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.552 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:32:21.552 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "snap_schedule", 2024-04-20T11:32:21.552 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-04-20T11:32:21.552 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-04-20T11:32:21.552 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-04-20T11:32:21.552 INFO:teuthology.orchestra.run.smithi152.stdout: "allow_m_granularity": { 2024-04-20T11:32:21.552 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "allow_m_granularity", 2024-04-20T11:32:21.552 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.552 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.552 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.552 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.553 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.553 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.553 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.553 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "allow minute scheduled snapshots", 2024-04-20T11:32:21.553 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.553 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.553 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.553 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.553 INFO:teuthology.orchestra.run.smithi152.stdout: "dump_on_update": { 2024-04-20T11:32:21.553 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "dump_on_update", 2024-04-20T11:32:21.553 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.553 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.553 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.553 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.554 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.554 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.554 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.554 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "dump database to debug log on update", 2024-04-20T11:32:21.554 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.554 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.554 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.554 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.554 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-04-20T11:32:21.554 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-04-20T11:32:21.554 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.554 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.554 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.555 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.555 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.555 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.555 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.555 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.555 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.555 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.555 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.555 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.555 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.555 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.555 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.555 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.555 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.556 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.556 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.556 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-04-20T11:32:21.556 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-04-20T11:32:21.556 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.556 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.556 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.556 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.556 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.556 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.556 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.556 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.556 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.556 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.556 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.557 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.557 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-04-20T11:32:21.557 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:21.557 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.557 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.557 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.557 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-04-20T11:32:21.557 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.557 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.557 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.557 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.557 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.557 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.557 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.558 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.558 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.558 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.558 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.558 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.558 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.558 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.558 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.558 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-04-20T11:32:21.558 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-04-20T11:32:21.558 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.558 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.558 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.558 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.558 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.559 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.559 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.559 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.559 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.559 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.559 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.559 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.559 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.559 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.559 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:32:21.559 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "stats", 2024-04-20T11:32:21.559 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-04-20T11:32:21.559 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-04-20T11:32:21.559 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-04-20T11:32:21.560 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-04-20T11:32:21.560 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-04-20T11:32:21.560 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.560 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.560 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.560 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.560 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.560 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.560 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.560 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.560 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.560 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.560 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.560 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.560 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.561 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.561 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.561 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.561 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.561 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.561 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.561 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-04-20T11:32:21.561 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-04-20T11:32:21.561 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.561 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.561 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.561 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.561 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.561 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.562 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.562 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.562 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.562 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.562 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.562 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.562 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-04-20T11:32:21.562 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:21.562 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.562 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.562 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.562 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-04-20T11:32:21.562 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.562 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.563 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.563 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.563 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.563 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.563 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.563 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.563 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.563 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.563 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.563 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.563 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.563 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.563 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.563 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-04-20T11:32:21.563 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-04-20T11:32:21.564 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.564 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.564 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.564 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.564 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.564 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.564 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.564 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.564 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.564 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.564 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.564 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.564 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.564 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.565 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:32:21.565 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "status", 2024-04-20T11:32:21.565 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-04-20T11:32:21.565 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-04-20T11:32:21.565 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-04-20T11:32:21.565 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-04-20T11:32:21.565 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-04-20T11:32:21.565 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.565 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.565 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.565 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.565 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.565 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.565 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.565 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.566 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.566 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.566 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.566 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.566 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.566 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.566 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.566 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.566 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.566 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.566 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.566 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-04-20T11:32:21.566 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-04-20T11:32:21.566 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.567 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.567 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.567 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.567 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.567 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.567 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.567 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.567 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.567 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.567 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.567 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.567 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-04-20T11:32:21.567 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:21.567 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.568 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.568 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.568 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-04-20T11:32:21.568 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.568 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.568 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.568 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.568 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.568 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.568 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.568 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.568 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.568 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.568 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.568 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.569 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.569 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.569 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.569 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-04-20T11:32:21.569 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-04-20T11:32:21.569 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.569 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.569 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.569 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.569 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.569 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.569 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.569 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.569 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.570 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.570 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.570 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.570 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.570 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.570 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:32:21.570 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "telegraf", 2024-04-20T11:32:21.570 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-04-20T11:32:21.570 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-04-20T11:32:21.570 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-04-20T11:32:21.570 INFO:teuthology.orchestra.run.smithi152.stdout: "address": { 2024-04-20T11:32:21.570 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "address", 2024-04-20T11:32:21.570 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.570 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.570 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.571 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-20T11:32:21.571 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.571 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.571 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.571 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.571 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.571 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.571 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.571 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.571 INFO:teuthology.orchestra.run.smithi152.stdout: "interval": { 2024-04-20T11:32:21.571 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "interval", 2024-04-20T11:32:21.571 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "secs", 2024-04-20T11:32:21.571 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.571 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.572 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "15", 2024-04-20T11:32:21.572 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.572 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.572 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.572 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.572 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.572 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.572 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.572 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.572 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-04-20T11:32:21.572 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-04-20T11:32:21.572 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.572 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.572 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.573 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.573 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.573 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.573 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.573 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.573 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.573 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.573 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.573 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.573 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.573 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.573 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.573 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.573 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.573 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.574 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.574 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-04-20T11:32:21.574 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-04-20T11:32:21.574 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.574 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.574 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.574 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.574 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.574 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.574 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.574 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.574 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.574 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.574 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.575 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.575 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-04-20T11:32:21.575 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:21.575 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.575 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.575 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.575 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-04-20T11:32:21.575 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.575 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.575 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.575 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.575 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.575 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.576 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.576 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.576 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.576 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.576 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.576 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.576 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.576 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.576 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.576 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-04-20T11:32:21.576 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-04-20T11:32:21.576 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.576 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.576 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.577 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.577 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.577 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.577 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.577 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.577 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.577 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.577 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.577 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.577 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.577 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.577 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:32:21.577 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "telemetry", 2024-04-20T11:32:21.577 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-04-20T11:32:21.578 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-04-20T11:32:21.578 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-04-20T11:32:21.578 INFO:teuthology.orchestra.run.smithi152.stdout: "channel_basic": { 2024-04-20T11:32:21.578 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "channel_basic", 2024-04-20T11:32:21.578 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.578 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.578 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.578 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-04-20T11:32:21.578 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.578 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.578 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.578 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-20T11:32:21.578 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.578 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.579 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.579 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.579 INFO:teuthology.orchestra.run.smithi152.stdout: "channel_crash": { 2024-04-20T11:32:21.579 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "channel_crash", 2024-04-20T11:32:21.579 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.579 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.579 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.579 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-04-20T11:32:21.579 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.579 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.579 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.579 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-20T11:32:21.579 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.579 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.580 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.580 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.580 INFO:teuthology.orchestra.run.smithi152.stdout: "channel_device": { 2024-04-20T11:32:21.580 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "channel_device", 2024-04-20T11:32:21.580 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.580 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.580 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.580 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-04-20T11:32:21.580 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.580 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.580 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.580 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-20T11:32:21.580 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.580 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.581 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.581 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.581 INFO:teuthology.orchestra.run.smithi152.stdout: "channel_ident": { 2024-04-20T11:32:21.581 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "channel_ident", 2024-04-20T11:32:21.581 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.581 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.581 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.581 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.581 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.581 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.581 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.581 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-20T11:32:21.581 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.581 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.581 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.582 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.582 INFO:teuthology.orchestra.run.smithi152.stdout: "channel_perf": { 2024-04-20T11:32:21.582 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "channel_perf", 2024-04-20T11:32:21.582 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.582 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.582 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.582 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.582 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.582 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.582 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.582 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-20T11:32:21.582 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.582 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.583 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.583 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.583 INFO:teuthology.orchestra.run.smithi152.stdout: "contact": { 2024-04-20T11:32:21.583 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "contact", 2024-04-20T11:32:21.583 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.583 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.583 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.583 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.583 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.583 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.583 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.583 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.583 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.583 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.584 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.584 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.584 INFO:teuthology.orchestra.run.smithi152.stdout: "description": { 2024-04-20T11:32:21.584 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "description", 2024-04-20T11:32:21.584 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.584 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.584 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.584 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.584 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.584 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.584 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.584 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.584 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.584 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.585 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.585 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.585 INFO:teuthology.orchestra.run.smithi152.stdout: "device_url": { 2024-04-20T11:32:21.585 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "device_url", 2024-04-20T11:32:21.585 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.585 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.585 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.585 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-20T11:32:21.585 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.585 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.585 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.585 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.585 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.585 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.586 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.586 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.586 INFO:teuthology.orchestra.run.smithi152.stdout: "enabled": { 2024-04-20T11:32:21.586 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "enabled", 2024-04-20T11:32:21.586 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.586 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.586 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.586 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.586 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.586 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.586 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.586 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.586 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.586 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.587 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.587 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.587 INFO:teuthology.orchestra.run.smithi152.stdout: "interval": { 2024-04-20T11:32:21.587 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "interval", 2024-04-20T11:32:21.587 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-04-20T11:32:21.587 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.587 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.587 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "24", 2024-04-20T11:32:21.587 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "8", 2024-04-20T11:32:21.587 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.587 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.587 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.587 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.587 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.587 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.588 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.588 INFO:teuthology.orchestra.run.smithi152.stdout: "last_opt_revision": { 2024-04-20T11:32:21.588 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "last_opt_revision", 2024-04-20T11:32:21.588 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-04-20T11:32:21.588 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.588 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.588 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "1", 2024-04-20T11:32:21.588 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.588 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.588 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.588 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.588 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.588 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.589 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.589 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.589 INFO:teuthology.orchestra.run.smithi152.stdout: "leaderboard": { 2024-04-20T11:32:21.589 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "leaderboard", 2024-04-20T11:32:21.589 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.589 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.589 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.589 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.589 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.589 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.589 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.589 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.589 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.589 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.589 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.590 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.590 INFO:teuthology.orchestra.run.smithi152.stdout: "leaderboard_description": { 2024-04-20T11:32:21.590 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "leaderboard_description", 2024-04-20T11:32:21.590 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.590 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.590 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.590 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.590 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.590 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.590 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.590 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.590 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.590 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.591 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.591 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.591 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-04-20T11:32:21.591 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-04-20T11:32:21.591 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.591 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.591 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.591 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.591 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.591 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.591 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.591 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.591 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.591 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.591 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.592 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.592 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.592 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.592 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.592 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.592 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.592 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.592 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.592 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-04-20T11:32:21.592 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-04-20T11:32:21.592 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.592 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.592 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.592 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.592 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.593 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.593 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.593 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.593 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.593 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.593 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.593 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.593 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-04-20T11:32:21.593 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:21.593 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.593 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.593 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.593 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-04-20T11:32:21.593 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.593 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.594 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.594 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.594 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.594 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.594 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.594 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.594 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.594 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.594 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.594 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.594 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.594 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.594 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.594 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-04-20T11:32:21.595 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-04-20T11:32:21.595 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.595 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.595 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.595 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.595 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.595 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.595 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.595 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.595 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.595 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.595 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.595 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.595 INFO:teuthology.orchestra.run.smithi152.stdout: "organization": { 2024-04-20T11:32:21.596 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "organization", 2024-04-20T11:32:21.596 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.596 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.596 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.596 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.596 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.596 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.596 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.596 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.596 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.596 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.596 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.596 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.596 INFO:teuthology.orchestra.run.smithi152.stdout: "proxy": { 2024-04-20T11:32:21.596 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "proxy", 2024-04-20T11:32:21.597 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.597 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.597 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.597 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.597 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.597 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.597 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.597 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.597 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.597 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.597 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.597 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.597 INFO:teuthology.orchestra.run.smithi152.stdout: "url": { 2024-04-20T11:32:21.597 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "url", 2024-04-20T11:32:21.598 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.598 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.598 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.598 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-20T11:32:21.598 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.598 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.598 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.598 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.598 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.598 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.598 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.598 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.598 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.598 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.598 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:32:21.599 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "test_orchestrator", 2024-04-20T11:32:21.599 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-04-20T11:32:21.599 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-04-20T11:32:21.599 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-04-20T11:32:21.599 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-04-20T11:32:21.599 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-04-20T11:32:21.599 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.599 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.599 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.599 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.599 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.599 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.599 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.599 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.600 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.600 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.600 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.600 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.600 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.600 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.600 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.600 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.600 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.600 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.600 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.600 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-04-20T11:32:21.600 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-04-20T11:32:21.600 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.600 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.601 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.601 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.601 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.601 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.601 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.601 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.601 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.601 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.601 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.601 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.601 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-04-20T11:32:21.601 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:21.601 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.601 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.602 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.602 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-04-20T11:32:21.602 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.602 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.602 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.602 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.602 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.602 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.602 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.602 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.602 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.602 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.602 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.603 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.603 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.603 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.603 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.603 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-04-20T11:32:21.603 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-04-20T11:32:21.603 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.603 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.603 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.603 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.603 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.603 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.603 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.603 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.604 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.604 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.604 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.604 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.604 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.604 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.604 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:32:21.604 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "volumes", 2024-04-20T11:32:21.604 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-04-20T11:32:21.604 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-04-20T11:32:21.604 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-04-20T11:32:21.604 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-04-20T11:32:21.604 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-04-20T11:32:21.604 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.605 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.605 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.605 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.605 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.605 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.605 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.605 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.605 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.605 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.605 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.605 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.605 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.605 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.605 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.606 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.606 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.606 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.606 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.606 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-04-20T11:32:21.606 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-04-20T11:32:21.606 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.606 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.606 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.606 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.606 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.606 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.606 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.606 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.607 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.607 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.607 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.607 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.607 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-04-20T11:32:21.607 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:21.607 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.607 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.607 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.607 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-04-20T11:32:21.607 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.607 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.607 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.607 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.608 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.608 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.608 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.608 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.608 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.608 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.608 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.608 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.608 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.608 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.608 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.608 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-04-20T11:32:21.608 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-04-20T11:32:21.608 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.608 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.609 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.609 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.609 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.609 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.609 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.609 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.609 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.609 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.609 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.609 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.609 INFO:teuthology.orchestra.run.smithi152.stdout: "max_concurrent_clones": { 2024-04-20T11:32:21.609 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "max_concurrent_clones", 2024-04-20T11:32:21.609 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-04-20T11:32:21.609 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.610 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.610 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "4", 2024-04-20T11:32:21.610 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.610 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.610 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.610 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-20T11:32:21.610 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.610 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.610 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.610 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.610 INFO:teuthology.orchestra.run.smithi152.stdout: "periodic_async_work": { 2024-04-20T11:32:21.610 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "periodic_async_work", 2024-04-20T11:32:21.610 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.610 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.611 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.611 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.611 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.611 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.611 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.611 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Periodically check for async work", 2024-04-20T11:32:21.611 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.611 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.611 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.611 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.611 INFO:teuthology.orchestra.run.smithi152.stdout: "snapshot_clone_delay": { 2024-04-20T11:32:21.611 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "snapshot_clone_delay", 2024-04-20T11:32:21.611 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-04-20T11:32:21.611 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.612 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.612 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "0", 2024-04-20T11:32:21.612 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.612 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.612 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.612 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-20T11:32:21.612 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.612 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.612 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.612 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.612 INFO:teuthology.orchestra.run.smithi152.stdout: "snapshot_clone_no_wait": { 2024-04-20T11:32:21.612 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "snapshot_clone_no_wait", 2024-04-20T11:32:21.612 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.612 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.613 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.613 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-04-20T11:32:21.613 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.613 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.613 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.613 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-20T11:32:21.613 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.613 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.613 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.613 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.613 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.613 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.613 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:32:21.613 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "zabbix", 2024-04-20T11:32:21.613 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-04-20T11:32:21.614 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-04-20T11:32:21.614 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-04-20T11:32:21.614 INFO:teuthology.orchestra.run.smithi152.stdout: "discovery_interval": { 2024-04-20T11:32:21.614 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "discovery_interval", 2024-04-20T11:32:21.614 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "uint", 2024-04-20T11:32:21.614 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.614 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.614 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "100", 2024-04-20T11:32:21.614 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.614 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.614 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.614 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.614 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.614 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.614 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.615 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.615 INFO:teuthology.orchestra.run.smithi152.stdout: "identifier": { 2024-04-20T11:32:21.615 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "identifier", 2024-04-20T11:32:21.615 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.615 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.615 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.615 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.615 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.615 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.615 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.615 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.615 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.615 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.616 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.616 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.616 INFO:teuthology.orchestra.run.smithi152.stdout: "interval": { 2024-04-20T11:32:21.616 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "interval", 2024-04-20T11:32:21.616 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "secs", 2024-04-20T11:32:21.616 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.616 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.616 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "60", 2024-04-20T11:32:21.616 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.616 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.616 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.616 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.616 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.616 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.616 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.617 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.617 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-04-20T11:32:21.617 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-04-20T11:32:21.617 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.617 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.617 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.617 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.617 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.617 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.617 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.617 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.617 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.617 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.617 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.617 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.618 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.618 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.618 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.618 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.618 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.618 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.618 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.618 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-04-20T11:32:21.618 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-04-20T11:32:21.618 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.618 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.618 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.618 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.618 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.619 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.619 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.619 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.619 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.619 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.619 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.619 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.619 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-04-20T11:32:21.619 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:21.619 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.619 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.619 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.619 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-04-20T11:32:21.619 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.619 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.620 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.620 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.620 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.620 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.620 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.620 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.620 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.620 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.620 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.620 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.620 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.620 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.620 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.620 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-04-20T11:32:21.621 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-04-20T11:32:21.621 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.621 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.621 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.621 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.621 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.621 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.621 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.621 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.621 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.621 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.621 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.621 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.621 INFO:teuthology.orchestra.run.smithi152.stdout: "zabbix_host": { 2024-04-20T11:32:21.621 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "zabbix_host", 2024-04-20T11:32:21.622 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.622 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.622 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.622 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.622 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.622 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.622 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.622 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.622 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.622 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.622 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.622 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.622 INFO:teuthology.orchestra.run.smithi152.stdout: "zabbix_port": { 2024-04-20T11:32:21.622 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "zabbix_port", 2024-04-20T11:32:21.623 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-04-20T11:32:21.623 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.623 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.623 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "10051", 2024-04-20T11:32:21.623 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.623 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.623 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.623 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.623 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.623 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.623 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.623 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.623 INFO:teuthology.orchestra.run.smithi152.stdout: "zabbix_sender": { 2024-04-20T11:32:21.623 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "zabbix_sender", 2024-04-20T11:32:21.623 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.624 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.624 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.624 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-20T11:32:21.624 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.624 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.624 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.624 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.624 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.624 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.624 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.624 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.624 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.624 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.624 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-04-20T11:32:21.624 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.625 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.625 INFO:teuthology.orchestra.run.smithi152.stdout: "modules": [ 2024-04-20T11:32:21.625 INFO:teuthology.orchestra.run.smithi152.stdout: "iostat", 2024-04-20T11:32:21.625 INFO:teuthology.orchestra.run.smithi152.stdout: "nfs", 2024-04-20T11:32:21.625 INFO:teuthology.orchestra.run.smithi152.stdout: "restful" 2024-04-20T11:32:21.625 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.625 INFO:teuthology.orchestra.run.smithi152.stdout: "available_modules": [ 2024-04-20T11:32:21.625 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:32:21.625 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "alerts", 2024-04-20T11:32:21.625 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-04-20T11:32:21.625 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-04-20T11:32:21.625 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-04-20T11:32:21.625 INFO:teuthology.orchestra.run.smithi152.stdout: "interval": { 2024-04-20T11:32:21.625 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "interval", 2024-04-20T11:32:21.626 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "secs", 2024-04-20T11:32:21.626 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.626 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.626 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "60", 2024-04-20T11:32:21.626 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.626 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.626 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.626 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "How frequently to reexamine health status", 2024-04-20T11:32:21.626 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.626 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.626 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.626 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.626 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-04-20T11:32:21.626 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-04-20T11:32:21.627 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.627 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.627 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.627 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.627 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.627 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.627 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.627 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.627 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.627 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.627 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.627 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.627 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.627 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.627 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.628 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.628 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.628 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.628 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.628 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-04-20T11:32:21.628 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-04-20T11:32:21.628 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.628 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.628 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.628 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.628 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.628 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.628 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.628 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.629 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.629 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.629 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.629 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.629 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-04-20T11:32:21.629 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:21.629 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.629 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.629 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.629 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-04-20T11:32:21.629 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.629 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.629 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.629 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.629 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.630 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.630 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.630 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.630 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.630 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.630 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.630 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.630 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.630 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.630 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.630 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-04-20T11:32:21.630 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-04-20T11:32:21.630 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.630 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.631 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.631 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.631 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.631 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.631 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.631 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.631 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.631 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.631 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.631 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.631 INFO:teuthology.orchestra.run.smithi152.stdout: "smtp_destination": { 2024-04-20T11:32:21.631 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "smtp_destination", 2024-04-20T11:32:21.631 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.631 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.632 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.632 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.632 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.632 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.632 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.632 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Email address to send alerts to", 2024-04-20T11:32:21.632 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.632 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.632 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.632 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.632 INFO:teuthology.orchestra.run.smithi152.stdout: "smtp_from_name": { 2024-04-20T11:32:21.632 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "smtp_from_name", 2024-04-20T11:32:21.632 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.632 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.632 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.633 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "Ceph", 2024-04-20T11:32:21.633 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.633 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.633 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.633 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Email From: name", 2024-04-20T11:32:21.633 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.633 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.633 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.633 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.633 INFO:teuthology.orchestra.run.smithi152.stdout: "smtp_host": { 2024-04-20T11:32:21.633 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "smtp_host", 2024-04-20T11:32:21.633 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.633 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.633 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.634 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.634 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.634 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.634 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.634 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "SMTP server", 2024-04-20T11:32:21.634 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.634 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.634 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.634 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.634 INFO:teuthology.orchestra.run.smithi152.stdout: "smtp_password": { 2024-04-20T11:32:21.634 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "smtp_password", 2024-04-20T11:32:21.634 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.634 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.634 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.635 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.635 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.635 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.635 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.636 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Password to authenticate with", 2024-04-20T11:32:21.636 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.636 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.636 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.636 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.636 INFO:teuthology.orchestra.run.smithi152.stdout: "smtp_port": { 2024-04-20T11:32:21.636 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "smtp_port", 2024-04-20T11:32:21.636 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-04-20T11:32:21.636 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.636 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.636 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "465", 2024-04-20T11:32:21.636 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.636 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.636 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.637 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "SMTP port", 2024-04-20T11:32:21.637 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.637 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.637 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.637 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.637 INFO:teuthology.orchestra.run.smithi152.stdout: "smtp_sender": { 2024-04-20T11:32:21.637 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "smtp_sender", 2024-04-20T11:32:21.637 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.637 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.637 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.637 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.637 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.637 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.637 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.638 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "SMTP envelope sender", 2024-04-20T11:32:21.638 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.638 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.638 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.638 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.638 INFO:teuthology.orchestra.run.smithi152.stdout: "smtp_ssl": { 2024-04-20T11:32:21.638 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "smtp_ssl", 2024-04-20T11:32:21.638 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.638 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.638 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.638 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-04-20T11:32:21.638 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.638 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.638 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.639 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-20T11:32:21.639 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.639 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.639 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.639 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.640 INFO:teuthology.orchestra.run.smithi152.stdout: "smtp_user": { 2024-04-20T11:32:21.640 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "smtp_user", 2024-04-20T11:32:21.640 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.640 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.640 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.640 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.640 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.640 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.640 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.640 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "User to authenticate as", 2024-04-20T11:32:21.640 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.640 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.640 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.640 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.641 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.641 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.641 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:32:21.641 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "balancer", 2024-04-20T11:32:21.641 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-04-20T11:32:21.641 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-04-20T11:32:21.641 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-04-20T11:32:21.641 INFO:teuthology.orchestra.run.smithi152.stdout: "active": { 2024-04-20T11:32:21.641 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "active", 2024-04-20T11:32:21.641 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.641 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.641 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.641 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-04-20T11:32:21.641 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.641 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.642 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.642 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "automatically balance PGs across cluster", 2024-04-20T11:32:21.642 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.642 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.642 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.642 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.642 INFO:teuthology.orchestra.run.smithi152.stdout: "begin_time": { 2024-04-20T11:32:21.643 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "begin_time", 2024-04-20T11:32:21.643 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.643 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.643 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.643 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "0000", 2024-04-20T11:32:21.643 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.643 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.643 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.643 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "beginning time of day to automatically balance", 2024-04-20T11:32:21.643 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-20T11:32:21.643 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.643 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.644 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.644 INFO:teuthology.orchestra.run.smithi152.stdout: "begin_weekday": { 2024-04-20T11:32:21.644 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "begin_weekday", 2024-04-20T11:32:21.644 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "uint", 2024-04-20T11:32:21.644 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.644 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.644 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "0", 2024-04-20T11:32:21.644 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "0", 2024-04-20T11:32:21.644 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "6", 2024-04-20T11:32:21.644 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.644 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-20T11:32:21.644 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-20T11:32:21.644 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.644 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.645 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.645 INFO:teuthology.orchestra.run.smithi152.stdout: "crush_compat_max_iterations": { 2024-04-20T11:32:21.645 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "crush_compat_max_iterations", 2024-04-20T11:32:21.645 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "uint", 2024-04-20T11:32:21.645 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.645 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.645 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "25", 2024-04-20T11:32:21.645 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "1", 2024-04-20T11:32:21.645 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "250", 2024-04-20T11:32:21.645 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.645 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-20T11:32:21.645 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.645 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.645 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.645 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.646 INFO:teuthology.orchestra.run.smithi152.stdout: "crush_compat_metrics": { 2024-04-20T11:32:21.646 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "crush_compat_metrics", 2024-04-20T11:32:21.646 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.646 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.646 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.646 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "pgs,objects,bytes", 2024-04-20T11:32:21.646 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.646 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.646 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.646 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-20T11:32:21.646 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-20T11:32:21.646 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.646 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.647 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.647 INFO:teuthology.orchestra.run.smithi152.stdout: "crush_compat_step": { 2024-04-20T11:32:21.647 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "crush_compat_step", 2024-04-20T11:32:21.647 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "float", 2024-04-20T11:32:21.647 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.647 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.647 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "0.5", 2024-04-20T11:32:21.647 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "0.001", 2024-04-20T11:32:21.647 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "0.999", 2024-04-20T11:32:21.647 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.647 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "aggressiveness of optimization", 2024-04-20T11:32:21.647 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-20T11:32:21.647 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.647 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.648 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.648 INFO:teuthology.orchestra.run.smithi152.stdout: "end_time": { 2024-04-20T11:32:21.648 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "end_time", 2024-04-20T11:32:21.648 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.648 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.648 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.648 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "2359", 2024-04-20T11:32:21.648 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.648 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.648 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.648 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "ending time of day to automatically balance", 2024-04-20T11:32:21.648 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-20T11:32:21.648 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.649 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.649 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.649 INFO:teuthology.orchestra.run.smithi152.stdout: "end_weekday": { 2024-04-20T11:32:21.649 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "end_weekday", 2024-04-20T11:32:21.649 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "uint", 2024-04-20T11:32:21.649 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.649 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.649 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "0", 2024-04-20T11:32:21.649 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "0", 2024-04-20T11:32:21.649 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "6", 2024-04-20T11:32:21.649 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.649 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-20T11:32:21.649 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-20T11:32:21.649 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.649 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.650 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.650 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-04-20T11:32:21.650 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-04-20T11:32:21.650 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.650 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.650 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.650 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.650 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.650 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.650 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.650 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.650 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.650 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.651 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.651 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.651 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.651 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.651 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.651 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.651 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.651 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.651 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.651 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-04-20T11:32:21.651 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-04-20T11:32:21.651 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.651 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.651 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.651 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.652 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.652 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.652 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.652 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.652 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.652 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.652 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.652 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.652 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-04-20T11:32:21.652 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:21.652 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.652 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.652 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.652 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-04-20T11:32:21.653 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.653 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.653 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.653 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.653 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.653 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.653 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.653 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.653 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.653 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.653 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.653 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.653 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.653 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.654 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.654 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-04-20T11:32:21.654 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-04-20T11:32:21.654 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.654 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.654 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.654 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.654 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.654 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.654 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.654 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.654 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.654 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.654 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.655 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.655 INFO:teuthology.orchestra.run.smithi152.stdout: "min_score": { 2024-04-20T11:32:21.655 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "min_score", 2024-04-20T11:32:21.655 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "float", 2024-04-20T11:32:21.655 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.655 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.655 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "0", 2024-04-20T11:32:21.655 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.655 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.655 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.655 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-20T11:32:21.655 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.655 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.655 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.656 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.656 INFO:teuthology.orchestra.run.smithi152.stdout: "mode": { 2024-04-20T11:32:21.656 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "mode", 2024-04-20T11:32:21.656 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.656 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.656 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.656 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "upmap", 2024-04-20T11:32:21.656 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.656 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.656 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.656 INFO:teuthology.orchestra.run.smithi152.stdout: "crush-compat", 2024-04-20T11:32:21.656 INFO:teuthology.orchestra.run.smithi152.stdout: "none", 2024-04-20T11:32:21.656 INFO:teuthology.orchestra.run.smithi152.stdout: "read", 2024-04-20T11:32:21.656 INFO:teuthology.orchestra.run.smithi152.stdout: "upmap", 2024-04-20T11:32:21.657 INFO:teuthology.orchestra.run.smithi152.stdout: "upmap-read" 2024-04-20T11:32:21.657 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.657 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Balancer mode", 2024-04-20T11:32:21.657 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.657 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.657 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.657 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.657 INFO:teuthology.orchestra.run.smithi152.stdout: "pool_ids": { 2024-04-20T11:32:21.657 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "pool_ids", 2024-04-20T11:32:21.657 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.657 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.657 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.657 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.657 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.658 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.658 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.658 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-20T11:32:21.658 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.658 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.658 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.658 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.658 INFO:teuthology.orchestra.run.smithi152.stdout: "sleep_interval": { 2024-04-20T11:32:21.658 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "sleep_interval", 2024-04-20T11:32:21.658 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "secs", 2024-04-20T11:32:21.658 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.658 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.658 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "60", 2024-04-20T11:32:21.658 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.659 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.659 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.659 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-20T11:32:21.659 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.659 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.659 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.659 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.659 INFO:teuthology.orchestra.run.smithi152.stdout: "upmap_max_deviation": { 2024-04-20T11:32:21.659 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "upmap_max_deviation", 2024-04-20T11:32:21.659 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-04-20T11:32:21.659 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.659 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.659 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "5", 2024-04-20T11:32:21.659 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "1", 2024-04-20T11:32:21.660 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.660 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.660 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-20T11:32:21.660 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-20T11:32:21.660 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.660 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.660 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.660 INFO:teuthology.orchestra.run.smithi152.stdout: "upmap_max_optimizations": { 2024-04-20T11:32:21.660 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "upmap_max_optimizations", 2024-04-20T11:32:21.660 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "uint", 2024-04-20T11:32:21.660 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.660 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.660 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "10", 2024-04-20T11:32:21.661 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.661 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.661 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.661 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-20T11:32:21.661 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.661 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.661 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.661 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.661 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.661 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.661 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:32:21.661 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "cephadm", 2024-04-20T11:32:21.661 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-04-20T11:32:21.661 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-04-20T11:32:21.662 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-04-20T11:32:21.662 INFO:teuthology.orchestra.run.smithi152.stdout: "agent_down_multiplier": { 2024-04-20T11:32:21.662 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "agent_down_multiplier", 2024-04-20T11:32:21.662 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "float", 2024-04-20T11:32:21.662 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.662 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.662 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "3.0", 2024-04-20T11:32:21.662 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.662 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.662 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.662 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-20T11:32:21.662 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.662 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.662 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.663 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.663 INFO:teuthology.orchestra.run.smithi152.stdout: "agent_refresh_rate": { 2024-04-20T11:32:21.663 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "agent_refresh_rate", 2024-04-20T11:32:21.663 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "secs", 2024-04-20T11:32:21.663 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.663 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.663 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "20", 2024-04-20T11:32:21.663 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.663 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.663 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.663 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-20T11:32:21.663 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.663 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.664 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.664 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.664 INFO:teuthology.orchestra.run.smithi152.stdout: "agent_starting_port": { 2024-04-20T11:32:21.664 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "agent_starting_port", 2024-04-20T11:32:21.664 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-04-20T11:32:21.664 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.664 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.664 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "4721", 2024-04-20T11:32:21.664 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.664 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.664 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.664 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-20T11:32:21.664 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.664 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.665 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.665 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.665 INFO:teuthology.orchestra.run.smithi152.stdout: "allow_ptrace": { 2024-04-20T11:32:21.665 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "allow_ptrace", 2024-04-20T11:32:21.665 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.665 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.665 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.665 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.665 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.665 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.665 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.665 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-20T11:32:21.665 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-20T11:32:21.665 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.665 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.666 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.666 INFO:teuthology.orchestra.run.smithi152.stdout: "autotune_interval": { 2024-04-20T11:32:21.666 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "autotune_interval", 2024-04-20T11:32:21.666 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "secs", 2024-04-20T11:32:21.666 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.666 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.666 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "600", 2024-04-20T11:32:21.666 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.666 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.666 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.666 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-20T11:32:21.666 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.666 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.667 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.667 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.667 INFO:teuthology.orchestra.run.smithi152.stdout: "autotune_memory_target_ratio": { 2024-04-20T11:32:21.667 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "autotune_memory_target_ratio", 2024-04-20T11:32:21.667 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "float", 2024-04-20T11:32:21.667 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.667 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.667 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "0.7", 2024-04-20T11:32:21.667 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.667 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.667 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.667 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-20T11:32:21.667 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.667 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.667 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.668 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.668 INFO:teuthology.orchestra.run.smithi152.stdout: "cephadm_log_destination": { 2024-04-20T11:32:21.668 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "cephadm_log_destination", 2024-04-20T11:32:21.668 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.668 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.668 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.668 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.668 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.668 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.668 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.668 INFO:teuthology.orchestra.run.smithi152.stdout: "file", 2024-04-20T11:32:21.668 INFO:teuthology.orchestra.run.smithi152.stdout: "file,syslog", 2024-04-20T11:32:21.668 INFO:teuthology.orchestra.run.smithi152.stdout: "syslog" 2024-04-20T11:32:21.668 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.669 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-20T11:32:21.669 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.669 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.669 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.669 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.669 INFO:teuthology.orchestra.run.smithi152.stdout: "cgroups_split": { 2024-04-20T11:32:21.669 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "cgroups_split", 2024-04-20T11:32:21.669 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.669 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.669 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.669 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-04-20T11:32:21.669 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.669 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.669 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.670 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-20T11:32:21.670 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.670 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.670 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.670 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.670 INFO:teuthology.orchestra.run.smithi152.stdout: "config_checks_enabled": { 2024-04-20T11:32:21.670 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "config_checks_enabled", 2024-04-20T11:32:21.670 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.670 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.670 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.670 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.670 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.670 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.671 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.671 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-20T11:32:21.671 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.671 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.671 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.671 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.671 INFO:teuthology.orchestra.run.smithi152.stdout: "config_dashboard": { 2024-04-20T11:32:21.671 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "config_dashboard", 2024-04-20T11:32:21.671 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.671 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.671 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.671 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-04-20T11:32:21.671 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.671 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.671 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.672 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-20T11:32:21.672 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.672 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.672 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.672 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.672 INFO:teuthology.orchestra.run.smithi152.stdout: "container_image_alertmanager": { 2024-04-20T11:32:21.672 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "container_image_alertmanager", 2024-04-20T11:32:21.672 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.672 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.672 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.672 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-20T11:32:21.672 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.672 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.672 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.673 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Prometheus container image", 2024-04-20T11:32:21.673 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.673 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.673 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.673 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.673 INFO:teuthology.orchestra.run.smithi152.stdout: "container_image_base": { 2024-04-20T11:32:21.673 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "container_image_base", 2024-04-20T11:32:21.673 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.673 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.673 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.673 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-20T11:32:21.673 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.673 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.673 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.674 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Container image name, without the tag", 2024-04-20T11:32:21.674 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.674 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.674 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.674 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.674 INFO:teuthology.orchestra.run.smithi152.stdout: "container_image_elasticsearch": { 2024-04-20T11:32:21.674 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "container_image_elasticsearch", 2024-04-20T11:32:21.674 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.674 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.674 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.674 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-20T11:32:21.674 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.674 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.675 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.675 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "elasticsearch container image", 2024-04-20T11:32:21.675 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.675 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.675 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.675 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.675 INFO:teuthology.orchestra.run.smithi152.stdout: "container_image_grafana": { 2024-04-20T11:32:21.675 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "container_image_grafana", 2024-04-20T11:32:21.675 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.675 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.675 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.675 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-20T11:32:21.675 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.676 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.676 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.676 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Prometheus container image", 2024-04-20T11:32:21.676 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.676 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.676 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.676 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.676 INFO:teuthology.orchestra.run.smithi152.stdout: "container_image_haproxy": { 2024-04-20T11:32:21.676 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "container_image_haproxy", 2024-04-20T11:32:21.676 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.676 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.676 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.676 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-20T11:32:21.676 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.677 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.677 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.677 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "HAproxy container image", 2024-04-20T11:32:21.677 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.677 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.677 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.677 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.677 INFO:teuthology.orchestra.run.smithi152.stdout: "container_image_jaeger_agent": { 2024-04-20T11:32:21.677 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "container_image_jaeger_agent", 2024-04-20T11:32:21.677 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.677 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.677 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.677 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-20T11:32:21.677 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.678 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.678 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.678 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Jaeger agent container image", 2024-04-20T11:32:21.678 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.678 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.678 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.678 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.678 INFO:teuthology.orchestra.run.smithi152.stdout: "container_image_jaeger_collector": { 2024-04-20T11:32:21.678 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "container_image_jaeger_collector", 2024-04-20T11:32:21.678 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.678 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.678 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.678 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-20T11:32:21.679 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.679 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.679 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.679 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Jaeger collector container image", 2024-04-20T11:32:21.679 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.679 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.679 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.679 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.679 INFO:teuthology.orchestra.run.smithi152.stdout: "container_image_jaeger_query": { 2024-04-20T11:32:21.679 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "container_image_jaeger_query", 2024-04-20T11:32:21.679 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.679 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.679 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.680 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-20T11:32:21.680 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.680 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.680 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.680 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Jaeger query container image", 2024-04-20T11:32:21.680 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.680 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.680 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.680 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.680 INFO:teuthology.orchestra.run.smithi152.stdout: "container_image_keepalived": { 2024-04-20T11:32:21.680 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "container_image_keepalived", 2024-04-20T11:32:21.680 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.680 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.680 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.681 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-20T11:32:21.681 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.681 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.681 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.681 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Keepalived container image", 2024-04-20T11:32:21.681 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.681 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.681 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.681 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.681 INFO:teuthology.orchestra.run.smithi152.stdout: "container_image_loki": { 2024-04-20T11:32:21.681 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "container_image_loki", 2024-04-20T11:32:21.681 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.681 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.681 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.682 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-20T11:32:21.682 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.682 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.682 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.682 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Loki container image", 2024-04-20T11:32:21.682 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.682 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.682 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.682 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.682 INFO:teuthology.orchestra.run.smithi152.stdout: "container_image_node_exporter": { 2024-04-20T11:32:21.682 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "container_image_node_exporter", 2024-04-20T11:32:21.682 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.682 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.683 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.683 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-20T11:32:21.683 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.683 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.683 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.683 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Prometheus container image", 2024-04-20T11:32:21.683 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.683 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.683 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.683 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.683 INFO:teuthology.orchestra.run.smithi152.stdout: "container_image_nvmeof": { 2024-04-20T11:32:21.683 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "container_image_nvmeof", 2024-04-20T11:32:21.683 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.683 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.683 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.684 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-20T11:32:21.684 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.684 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.684 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.684 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Nvme-of container image", 2024-04-20T11:32:21.684 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.684 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.684 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.684 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.684 INFO:teuthology.orchestra.run.smithi152.stdout: "container_image_prometheus": { 2024-04-20T11:32:21.684 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "container_image_prometheus", 2024-04-20T11:32:21.684 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.684 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.685 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.685 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-20T11:32:21.685 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.685 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.685 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.685 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Prometheus container image", 2024-04-20T11:32:21.685 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.685 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.685 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.685 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.685 INFO:teuthology.orchestra.run.smithi152.stdout: "container_image_promtail": { 2024-04-20T11:32:21.685 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "container_image_promtail", 2024-04-20T11:32:21.685 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.685 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.685 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.686 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-20T11:32:21.686 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.686 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.686 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.686 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Promtail container image", 2024-04-20T11:32:21.686 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.686 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.686 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.686 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.686 INFO:teuthology.orchestra.run.smithi152.stdout: "container_image_samba": { 2024-04-20T11:32:21.686 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "container_image_samba", 2024-04-20T11:32:21.686 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.686 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.686 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.687 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-20T11:32:21.687 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.687 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.687 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.687 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Samba/SMB container image", 2024-04-20T11:32:21.687 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.687 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.687 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.687 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.687 INFO:teuthology.orchestra.run.smithi152.stdout: "container_image_snmp_gateway": { 2024-04-20T11:32:21.687 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "container_image_snmp_gateway", 2024-04-20T11:32:21.687 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.687 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.687 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.688 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-20T11:32:21.688 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.688 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.688 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.688 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "SNMP Gateway container image", 2024-04-20T11:32:21.688 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.688 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.688 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.688 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.688 INFO:teuthology.orchestra.run.smithi152.stdout: "container_init": { 2024-04-20T11:32:21.688 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "container_init", 2024-04-20T11:32:21.688 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.688 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.688 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.688 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-04-20T11:32:21.689 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.689 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.689 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.689 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Run podman/docker with `--init`", 2024-04-20T11:32:21.689 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.689 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.689 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.689 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.689 INFO:teuthology.orchestra.run.smithi152.stdout: "daemon_cache_timeout": { 2024-04-20T11:32:21.689 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "daemon_cache_timeout", 2024-04-20T11:32:21.689 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "secs", 2024-04-20T11:32:21.689 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.689 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.689 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "600", 2024-04-20T11:32:21.690 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.690 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.690 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.690 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-20T11:32:21.690 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.690 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.690 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.690 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.690 INFO:teuthology.orchestra.run.smithi152.stdout: "default_cephadm_command_timeout": { 2024-04-20T11:32:21.690 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "default_cephadm_command_timeout", 2024-04-20T11:32:21.690 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-04-20T11:32:21.690 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.690 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.690 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "900", 2024-04-20T11:32:21.691 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.691 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.691 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.691 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-20T11:32:21.691 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.691 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.691 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.691 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.691 INFO:teuthology.orchestra.run.smithi152.stdout: "default_registry": { 2024-04-20T11:32:21.691 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "default_registry", 2024-04-20T11:32:21.691 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.691 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.691 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.691 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "docker.io", 2024-04-20T11:32:21.692 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.692 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.692 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.692 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-20T11:32:21.692 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.692 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.692 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.692 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.692 INFO:teuthology.orchestra.run.smithi152.stdout: "device_cache_timeout": { 2024-04-20T11:32:21.692 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "device_cache_timeout", 2024-04-20T11:32:21.692 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "secs", 2024-04-20T11:32:21.692 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.692 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.693 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "1800", 2024-04-20T11:32:21.693 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.693 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.693 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.693 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "seconds to cache device inventory", 2024-04-20T11:32:21.693 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.693 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.693 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.693 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.693 INFO:teuthology.orchestra.run.smithi152.stdout: "device_enhanced_scan": { 2024-04-20T11:32:21.693 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "device_enhanced_scan", 2024-04-20T11:32:21.693 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.693 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.693 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.694 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.694 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.694 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.694 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.694 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-20T11:32:21.694 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.694 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.694 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.694 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.694 INFO:teuthology.orchestra.run.smithi152.stdout: "facts_cache_timeout": { 2024-04-20T11:32:21.694 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "facts_cache_timeout", 2024-04-20T11:32:21.694 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "secs", 2024-04-20T11:32:21.694 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.694 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.695 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "60", 2024-04-20T11:32:21.695 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.695 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.695 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.695 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "seconds to cache host facts data", 2024-04-20T11:32:21.695 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.695 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.695 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.695 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.695 INFO:teuthology.orchestra.run.smithi152.stdout: "grafana_dashboards_path": { 2024-04-20T11:32:21.695 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "grafana_dashboards_path", 2024-04-20T11:32:21.695 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.695 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.695 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.696 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-20T11:32:21.696 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.696 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.696 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.696 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-20T11:32:21.696 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.696 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.696 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.696 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.696 INFO:teuthology.orchestra.run.smithi152.stdout: "host_check_interval": { 2024-04-20T11:32:21.696 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "host_check_interval", 2024-04-20T11:32:21.696 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "secs", 2024-04-20T11:32:21.696 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.696 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.696 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "600", 2024-04-20T11:32:21.697 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.697 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.697 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.697 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "how frequently to perform a host check", 2024-04-20T11:32:21.697 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.697 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.697 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.697 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.697 INFO:teuthology.orchestra.run.smithi152.stdout: "hw_monitoring": { 2024-04-20T11:32:21.697 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "hw_monitoring", 2024-04-20T11:32:21.697 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.697 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.697 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.697 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.697 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.698 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.698 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.698 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-20T11:32:21.698 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.698 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.698 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.698 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.698 INFO:teuthology.orchestra.run.smithi152.stdout: "inventory_list_all": { 2024-04-20T11:32:21.698 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "inventory_list_all", 2024-04-20T11:32:21.698 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.698 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.698 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.698 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.698 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.699 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.699 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.699 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-20T11:32:21.699 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.699 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.699 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.699 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.699 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-04-20T11:32:21.699 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-04-20T11:32:21.699 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.699 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.699 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.699 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.699 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.700 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.700 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.700 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.700 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.700 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.700 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.700 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.700 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.700 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.700 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.700 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.700 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.700 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.700 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.700 INFO:teuthology.orchestra.run.smithi152.stdout: "log_refresh_metadata": { 2024-04-20T11:32:21.701 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_refresh_metadata", 2024-04-20T11:32:21.701 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.701 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.701 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.701 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.701 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.701 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.701 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.701 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-20T11:32:21.701 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.701 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.701 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.701 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.701 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-04-20T11:32:21.702 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-04-20T11:32:21.702 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.702 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.702 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.702 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-04-20T11:32:21.702 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.702 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.702 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.702 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-20T11:32:21.702 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.702 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.702 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.702 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.702 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-04-20T11:32:21.703 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:21.703 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.703 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.703 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.703 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-04-20T11:32:21.703 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.703 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.703 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.703 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.703 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.703 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.703 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.703 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.703 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.704 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.704 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.704 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.704 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.704 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.704 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.704 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-04-20T11:32:21.704 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-04-20T11:32:21.704 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.704 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.704 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.704 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.704 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.704 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.704 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.705 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.705 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.705 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.705 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.705 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.705 INFO:teuthology.orchestra.run.smithi152.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-20T11:32:21.705 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-20T11:32:21.705 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.705 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.705 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.705 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.705 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.705 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.705 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.706 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-20T11:32:21.706 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.706 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.706 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.706 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.706 INFO:teuthology.orchestra.run.smithi152.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-20T11:32:21.706 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-20T11:32:21.706 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.706 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.706 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.706 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "*", 2024-04-20T11:32:21.706 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.706 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.706 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.707 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-20T11:32:21.707 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.707 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.707 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.707 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.707 INFO:teuthology.orchestra.run.smithi152.stdout: "max_count_per_host": { 2024-04-20T11:32:21.707 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "max_count_per_host", 2024-04-20T11:32:21.707 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-04-20T11:32:21.707 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.707 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.707 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "10", 2024-04-20T11:32:21.707 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.707 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.708 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.708 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "max number of daemons per service per host", 2024-04-20T11:32:21.708 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.708 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.708 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.708 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.708 INFO:teuthology.orchestra.run.smithi152.stdout: "max_osd_draining_count": { 2024-04-20T11:32:21.708 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "max_osd_draining_count", 2024-04-20T11:32:21.708 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-04-20T11:32:21.708 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.708 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.708 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "10", 2024-04-20T11:32:21.708 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.708 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.708 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.709 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-20T11:32:21.709 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.709 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.709 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.709 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.709 INFO:teuthology.orchestra.run.smithi152.stdout: "migration_current": { 2024-04-20T11:32:21.709 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "migration_current", 2024-04-20T11:32:21.709 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-04-20T11:32:21.709 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.709 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.709 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.709 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.709 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.710 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.710 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "internal - do not modify", 2024-04-20T11:32:21.710 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.710 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.710 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.710 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.710 INFO:teuthology.orchestra.run.smithi152.stdout: "mode": { 2024-04-20T11:32:21.710 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "mode", 2024-04-20T11:32:21.710 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.710 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.710 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.710 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "root", 2024-04-20T11:32:21.710 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.710 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.711 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.711 INFO:teuthology.orchestra.run.smithi152.stdout: "cephadm-package", 2024-04-20T11:32:21.711 INFO:teuthology.orchestra.run.smithi152.stdout: "root" 2024-04-20T11:32:21.711 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.711 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "mode for remote execution of cephadm", 2024-04-20T11:32:21.711 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.711 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.711 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.711 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.711 INFO:teuthology.orchestra.run.smithi152.stdout: "oob_default_addr": { 2024-04-20T11:32:21.711 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "oob_default_addr", 2024-04-20T11:32:21.711 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.711 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.711 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.712 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "169.254.1.1", 2024-04-20T11:32:21.712 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.712 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.712 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.712 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-20T11:32:21.712 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.712 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.712 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.712 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.712 INFO:teuthology.orchestra.run.smithi152.stdout: "prometheus_alerts_path": { 2024-04-20T11:32:21.712 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "prometheus_alerts_path", 2024-04-20T11:32:21.712 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.712 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.712 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.712 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-20T11:32:21.713 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.713 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.713 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.713 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-20T11:32:21.713 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.713 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.713 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.713 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.713 INFO:teuthology.orchestra.run.smithi152.stdout: "registry_insecure": { 2024-04-20T11:32:21.713 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "registry_insecure", 2024-04-20T11:32:21.713 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.713 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.713 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.713 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.714 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.714 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.714 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.714 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-20T11:32:21.714 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.714 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.714 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.714 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.714 INFO:teuthology.orchestra.run.smithi152.stdout: "registry_password": { 2024-04-20T11:32:21.714 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "registry_password", 2024-04-20T11:32:21.714 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.714 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.714 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.714 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.715 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.715 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.715 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.715 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-20T11:32:21.715 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.715 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.715 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.715 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.715 INFO:teuthology.orchestra.run.smithi152.stdout: "registry_url": { 2024-04-20T11:32:21.715 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "registry_url", 2024-04-20T11:32:21.715 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.715 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.715 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.715 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.716 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.716 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.716 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.716 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-20T11:32:21.716 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.716 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.716 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.716 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.716 INFO:teuthology.orchestra.run.smithi152.stdout: "registry_username": { 2024-04-20T11:32:21.716 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "registry_username", 2024-04-20T11:32:21.716 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.716 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.716 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.716 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.716 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.717 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.717 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.717 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-20T11:32:21.717 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.717 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.717 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.717 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.717 INFO:teuthology.orchestra.run.smithi152.stdout: "secure_monitoring_stack": { 2024-04-20T11:32:21.717 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "secure_monitoring_stack", 2024-04-20T11:32:21.717 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.717 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.717 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.717 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.717 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.718 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.718 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.718 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-20T11:32:21.718 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.718 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.718 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.718 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.718 INFO:teuthology.orchestra.run.smithi152.stdout: "service_discovery_port": { 2024-04-20T11:32:21.718 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "service_discovery_port", 2024-04-20T11:32:21.718 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-04-20T11:32:21.718 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.718 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.718 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "8765", 2024-04-20T11:32:21.719 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.719 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.719 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.719 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "cephadm service discovery port", 2024-04-20T11:32:21.719 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.719 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.719 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.719 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.719 INFO:teuthology.orchestra.run.smithi152.stdout: "ssh_config_file": { 2024-04-20T11:32:21.719 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "ssh_config_file", 2024-04-20T11:32:21.719 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.719 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.719 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.719 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.719 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.720 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.720 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.720 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-20T11:32:21.720 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.720 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.720 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.720 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.720 INFO:teuthology.orchestra.run.smithi152.stdout: "use_agent": { 2024-04-20T11:32:21.720 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "use_agent", 2024-04-20T11:32:21.720 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.720 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.720 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.720 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.720 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.720 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.721 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.721 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-20T11:32:21.721 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.721 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.721 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.721 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.721 INFO:teuthology.orchestra.run.smithi152.stdout: "use_repo_digest": { 2024-04-20T11:32:21.721 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "use_repo_digest", 2024-04-20T11:32:21.721 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.721 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.721 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.721 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-04-20T11:32:21.721 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.721 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.722 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.722 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-20T11:32:21.722 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.722 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.722 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.722 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.722 INFO:teuthology.orchestra.run.smithi152.stdout: "warn_on_failed_host_check": { 2024-04-20T11:32:21.722 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "warn_on_failed_host_check", 2024-04-20T11:32:21.722 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.722 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.722 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.722 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-04-20T11:32:21.722 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.723 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.723 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.723 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "raise a health warning if the host check fails", 2024-04-20T11:32:21.723 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.723 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.723 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.723 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.723 INFO:teuthology.orchestra.run.smithi152.stdout: "warn_on_stray_daemons": { 2024-04-20T11:32:21.723 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "warn_on_stray_daemons", 2024-04-20T11:32:21.723 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.723 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.723 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.723 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-04-20T11:32:21.724 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.724 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.724 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.724 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-20T11:32:21.724 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.724 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.724 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.724 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.724 INFO:teuthology.orchestra.run.smithi152.stdout: "warn_on_stray_hosts": { 2024-04-20T11:32:21.724 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "warn_on_stray_hosts", 2024-04-20T11:32:21.724 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.724 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.724 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.724 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-04-20T11:32:21.725 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.725 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.725 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.725 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-20T11:32:21.725 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.725 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.725 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.725 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.725 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.725 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.725 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:32:21.725 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "crash", 2024-04-20T11:32:21.725 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-04-20T11:32:21.725 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-04-20T11:32:21.726 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-04-20T11:32:21.726 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-04-20T11:32:21.726 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-04-20T11:32:21.726 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.726 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.726 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.726 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.726 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.726 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.726 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.726 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.726 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.726 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.726 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.727 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.727 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.727 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.727 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.727 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.727 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.727 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.727 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.727 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-04-20T11:32:21.727 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-04-20T11:32:21.727 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.727 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.727 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.727 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.728 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.728 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.729 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.729 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.729 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.729 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.729 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.729 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.729 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-04-20T11:32:21.729 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:21.729 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.729 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.729 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.730 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-04-20T11:32:21.730 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.730 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.730 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.730 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.730 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.730 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.730 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.730 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.730 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.730 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.730 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.730 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.731 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.731 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.731 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.731 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-04-20T11:32:21.731 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-04-20T11:32:21.731 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.731 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.731 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.731 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.731 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.731 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.731 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.731 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.731 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.731 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.732 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.732 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.732 INFO:teuthology.orchestra.run.smithi152.stdout: "retain_interval": { 2024-04-20T11:32:21.732 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "retain_interval", 2024-04-20T11:32:21.732 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "secs", 2024-04-20T11:32:21.732 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.732 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.732 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "31536000", 2024-04-20T11:32:21.732 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.732 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.732 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.732 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-20T11:32:21.732 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.733 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.733 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.733 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.733 INFO:teuthology.orchestra.run.smithi152.stdout: "warn_recent_interval": { 2024-04-20T11:32:21.733 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "warn_recent_interval", 2024-04-20T11:32:21.733 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "secs", 2024-04-20T11:32:21.733 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.733 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.733 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "1209600", 2024-04-20T11:32:21.733 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.733 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.733 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.733 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-20T11:32:21.733 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.733 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.734 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.734 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.734 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.734 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.734 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:32:21.734 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "dashboard", 2024-04-20T11:32:21.734 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-04-20T11:32:21.734 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-04-20T11:32:21.734 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-04-20T11:32:21.734 INFO:teuthology.orchestra.run.smithi152.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-20T11:32:21.734 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-20T11:32:21.734 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-04-20T11:32:21.734 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.735 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.735 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "10", 2024-04-20T11:32:21.735 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.735 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.735 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.735 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.735 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.735 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.735 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.735 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.735 INFO:teuthology.orchestra.run.smithi152.stdout: "ALERTMANAGER_API_HOST": { 2024-04-20T11:32:21.735 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-20T11:32:21.735 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.735 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.736 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.736 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.736 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.736 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.736 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.736 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.736 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.736 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.736 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.736 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.736 INFO:teuthology.orchestra.run.smithi152.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-20T11:32:21.736 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-20T11:32:21.736 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.736 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.736 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.737 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-04-20T11:32:21.737 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.737 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.737 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.737 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.737 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.737 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.737 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.737 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.737 INFO:teuthology.orchestra.run.smithi152.stdout: "AUDIT_API_ENABLED": { 2024-04-20T11:32:21.737 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "AUDIT_API_ENABLED", 2024-04-20T11:32:21.737 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.737 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.737 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.738 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.738 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.738 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.738 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.738 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.738 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.738 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.738 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.738 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.738 INFO:teuthology.orchestra.run.smithi152.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-20T11:32:21.738 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-20T11:32:21.738 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.738 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.738 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.739 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-04-20T11:32:21.739 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.739 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.739 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.739 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.739 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.739 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.739 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.739 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.739 INFO:teuthology.orchestra.run.smithi152.stdout: "ENABLE_BROWSABLE_API": { 2024-04-20T11:32:21.739 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-20T11:32:21.739 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.739 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.740 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.740 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-04-20T11:32:21.740 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.740 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.740 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.740 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.740 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.740 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.740 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.740 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.740 INFO:teuthology.orchestra.run.smithi152.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-20T11:32:21.740 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-20T11:32:21.740 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.740 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.741 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.741 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-04-20T11:32:21.741 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.741 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.741 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.741 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.741 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.741 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.741 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.741 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.741 INFO:teuthology.orchestra.run.smithi152.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-20T11:32:21.741 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-20T11:32:21.741 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.741 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.742 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.742 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-04-20T11:32:21.742 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.742 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.742 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.742 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.742 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.742 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.742 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.742 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.742 INFO:teuthology.orchestra.run.smithi152.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-20T11:32:21.742 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-20T11:32:21.742 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.742 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.743 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.743 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-04-20T11:32:21.743 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.743 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.743 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.743 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.743 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.743 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.743 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.743 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.743 INFO:teuthology.orchestra.run.smithi152.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-20T11:32:21.743 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-20T11:32:21.743 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.744 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.744 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.744 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-04-20T11:32:21.744 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.744 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.744 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.744 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.744 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.744 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.744 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.744 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.744 INFO:teuthology.orchestra.run.smithi152.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-20T11:32:21.744 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-20T11:32:21.744 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.745 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.745 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.745 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-04-20T11:32:21.745 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.745 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.745 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.745 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.745 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.745 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.745 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.745 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.745 INFO:teuthology.orchestra.run.smithi152.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-20T11:32:21.745 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-20T11:32:21.745 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.746 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.746 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.746 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-04-20T11:32:21.746 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.746 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.746 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.746 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.746 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.746 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.746 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.746 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.746 INFO:teuthology.orchestra.run.smithi152.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-20T11:32:21.746 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-20T11:32:21.746 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.747 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.747 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.747 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-04-20T11:32:21.747 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.747 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.747 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.747 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.747 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.747 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.747 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.747 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.747 INFO:teuthology.orchestra.run.smithi152.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-20T11:32:21.747 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-20T11:32:21.747 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.747 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.748 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.748 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.748 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.748 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.748 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.748 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.748 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.748 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.748 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.748 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.748 INFO:teuthology.orchestra.run.smithi152.stdout: "GRAFANA_API_PASSWORD": { 2024-04-20T11:32:21.748 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-20T11:32:21.748 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.748 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.749 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.749 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "admin", 2024-04-20T11:32:21.749 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.749 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.749 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.749 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.749 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.749 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.749 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.749 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.749 INFO:teuthology.orchestra.run.smithi152.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-20T11:32:21.749 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-20T11:32:21.749 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.749 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.749 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.750 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-04-20T11:32:21.750 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.750 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.750 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.750 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.750 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.750 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.750 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.750 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.750 INFO:teuthology.orchestra.run.smithi152.stdout: "GRAFANA_API_URL": { 2024-04-20T11:32:21.750 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "GRAFANA_API_URL", 2024-04-20T11:32:21.750 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.750 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.751 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.751 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.751 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.751 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.751 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.751 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.751 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.751 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.751 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.751 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.751 INFO:teuthology.orchestra.run.smithi152.stdout: "GRAFANA_API_USERNAME": { 2024-04-20T11:32:21.751 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-20T11:32:21.751 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.751 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.751 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.752 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "admin", 2024-04-20T11:32:21.752 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.752 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.752 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.752 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.752 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.752 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.752 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.752 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.752 INFO:teuthology.orchestra.run.smithi152.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-20T11:32:21.752 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-20T11:32:21.752 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.752 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.752 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.753 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.753 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.753 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.753 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.753 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.753 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.753 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.753 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.753 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.753 INFO:teuthology.orchestra.run.smithi152.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-20T11:32:21.753 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-20T11:32:21.753 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.753 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.753 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.754 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.754 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.754 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.754 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.754 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.754 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.754 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.754 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.754 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.754 INFO:teuthology.orchestra.run.smithi152.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-20T11:32:21.754 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-20T11:32:21.754 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.754 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.754 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.755 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-04-20T11:32:21.755 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.755 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.755 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.755 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.755 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.755 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.755 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.755 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.755 INFO:teuthology.orchestra.run.smithi152.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-20T11:32:21.755 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-20T11:32:21.755 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.755 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.756 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.756 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.756 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.756 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.756 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.756 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.756 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.756 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.756 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.756 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.756 INFO:teuthology.orchestra.run.smithi152.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-20T11:32:21.756 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-20T11:32:21.756 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.756 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.757 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.757 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "[]", 2024-04-20T11:32:21.757 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.757 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.757 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.757 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.757 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.757 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.757 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.757 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.757 INFO:teuthology.orchestra.run.smithi152.stdout: "MULTICLUSTER_CONFIG": { 2024-04-20T11:32:21.757 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-20T11:32:21.757 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.757 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.758 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.758 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "{}", 2024-04-20T11:32:21.758 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.758 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.758 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.758 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.758 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.758 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.758 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.758 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.758 INFO:teuthology.orchestra.run.smithi152.stdout: "PROMETHEUS_API_HOST": { 2024-04-20T11:32:21.758 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-20T11:32:21.758 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.758 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.759 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.759 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.759 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.759 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.759 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.759 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.759 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.759 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.759 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.759 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.759 INFO:teuthology.orchestra.run.smithi152.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-20T11:32:21.759 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-20T11:32:21.759 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.759 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.759 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.760 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-04-20T11:32:21.760 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.760 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.760 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.760 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.760 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.760 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.760 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.760 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.760 INFO:teuthology.orchestra.run.smithi152.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-20T11:32:21.760 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-20T11:32:21.760 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.760 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.760 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.761 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.761 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.761 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.761 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.761 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.761 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.761 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.761 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.761 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.761 INFO:teuthology.orchestra.run.smithi152.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-20T11:32:21.761 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-20T11:32:21.761 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.761 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.761 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.762 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.762 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.762 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.762 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.762 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.762 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.762 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.762 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.762 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.762 INFO:teuthology.orchestra.run.smithi152.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-20T11:32:21.762 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-20T11:32:21.762 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.762 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.762 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.763 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-04-20T11:32:21.763 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.763 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.763 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.763 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.763 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.763 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.763 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.763 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.763 INFO:teuthology.orchestra.run.smithi152.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-20T11:32:21.763 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-20T11:32:21.763 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.763 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.763 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.764 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-04-20T11:32:21.764 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.764 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.764 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.764 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.764 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.764 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.764 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.764 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.764 INFO:teuthology.orchestra.run.smithi152.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-20T11:32:21.764 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-20T11:32:21.764 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.764 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.764 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.764 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.765 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.765 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.765 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.765 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.765 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.765 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.765 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.765 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.765 INFO:teuthology.orchestra.run.smithi152.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-20T11:32:21.765 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-20T11:32:21.765 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.765 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.765 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.765 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.766 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.766 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.766 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.766 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.766 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.766 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.766 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.766 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.766 INFO:teuthology.orchestra.run.smithi152.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-20T11:32:21.766 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-20T11:32:21.766 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.766 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.766 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.766 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.767 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.767 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.767 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.767 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.767 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.767 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.767 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.767 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.767 INFO:teuthology.orchestra.run.smithi152.stdout: "PWD_POLICY_ENABLED": { 2024-04-20T11:32:21.767 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-20T11:32:21.767 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.767 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.767 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.767 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-04-20T11:32:21.767 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.768 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.768 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.768 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.768 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.768 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.768 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.768 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.768 INFO:teuthology.orchestra.run.smithi152.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-20T11:32:21.768 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-20T11:32:21.768 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.768 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.768 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.768 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-20T11:32:21.768 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.769 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.769 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.769 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.769 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.769 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.769 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.769 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.769 INFO:teuthology.orchestra.run.smithi152.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-20T11:32:21.769 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-20T11:32:21.769 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-04-20T11:32:21.769 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.769 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.769 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "10", 2024-04-20T11:32:21.769 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.769 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.770 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.770 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.770 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.770 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.770 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.770 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.770 INFO:teuthology.orchestra.run.smithi152.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-20T11:32:21.770 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-20T11:32:21.770 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-04-20T11:32:21.770 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.770 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.770 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "8", 2024-04-20T11:32:21.770 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.771 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.771 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.771 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.771 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.771 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.771 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.771 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.771 INFO:teuthology.orchestra.run.smithi152.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-20T11:32:21.771 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-20T11:32:21.771 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-04-20T11:32:21.771 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.771 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.771 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "45", 2024-04-20T11:32:21.771 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.771 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.772 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.772 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.772 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.772 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.772 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.772 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.772 INFO:teuthology.orchestra.run.smithi152.stdout: "RGW_API_ACCESS_KEY": { 2024-04-20T11:32:21.772 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-20T11:32:21.772 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.772 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.772 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.772 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.772 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.772 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.773 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.773 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.773 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.773 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.773 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.773 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.773 INFO:teuthology.orchestra.run.smithi152.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-20T11:32:21.773 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-20T11:32:21.773 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.773 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.773 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.773 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "admin", 2024-04-20T11:32:21.773 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.773 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.774 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.774 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.774 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.774 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.774 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.774 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.774 INFO:teuthology.orchestra.run.smithi152.stdout: "RGW_API_SECRET_KEY": { 2024-04-20T11:32:21.774 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-20T11:32:21.774 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.774 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.774 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.774 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.774 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.774 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.775 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.775 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.775 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.775 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.775 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.775 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.775 INFO:teuthology.orchestra.run.smithi152.stdout: "RGW_API_SSL_VERIFY": { 2024-04-20T11:32:21.775 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-20T11:32:21.775 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.775 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.775 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.775 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-04-20T11:32:21.775 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.775 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.775 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.776 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.776 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.776 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.776 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.776 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.776 INFO:teuthology.orchestra.run.smithi152.stdout: "UNSAFE_TLS_v1_2": { 2024-04-20T11:32:21.776 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-20T11:32:21.776 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.776 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.776 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.776 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.776 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.776 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.776 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.777 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.777 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.777 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.777 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.777 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.777 INFO:teuthology.orchestra.run.smithi152.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-20T11:32:21.777 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-20T11:32:21.777 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-04-20T11:32:21.777 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.777 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.777 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "0", 2024-04-20T11:32:21.777 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.777 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.777 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.777 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.778 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.778 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.778 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.778 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.778 INFO:teuthology.orchestra.run.smithi152.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-20T11:32:21.778 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-20T11:32:21.778 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-04-20T11:32:21.778 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.778 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.778 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "10", 2024-04-20T11:32:21.778 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.778 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.778 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.779 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.779 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.779 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.779 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.779 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.779 INFO:teuthology.orchestra.run.smithi152.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-20T11:32:21.779 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-20T11:32:21.779 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-04-20T11:32:21.779 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.779 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.779 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "5", 2024-04-20T11:32:21.779 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.779 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.779 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.780 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.780 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.780 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.780 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.780 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.780 INFO:teuthology.orchestra.run.smithi152.stdout: "cross_origin_url": { 2024-04-20T11:32:21.780 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "cross_origin_url", 2024-04-20T11:32:21.780 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.780 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.780 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.780 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.780 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.780 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.780 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.781 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.781 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.781 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.781 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.781 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.781 INFO:teuthology.orchestra.run.smithi152.stdout: "crt_file": { 2024-04-20T11:32:21.781 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "crt_file", 2024-04-20T11:32:21.781 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.781 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.781 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.781 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.781 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.781 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.781 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.781 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.782 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.782 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.782 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.782 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.782 INFO:teuthology.orchestra.run.smithi152.stdout: "debug": { 2024-04-20T11:32:21.782 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "debug", 2024-04-20T11:32:21.782 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.782 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.782 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.782 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.782 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.782 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.782 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.782 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Enable/disable debug options", 2024-04-20T11:32:21.783 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.783 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.783 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.783 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.783 INFO:teuthology.orchestra.run.smithi152.stdout: "jwt_token_ttl": { 2024-04-20T11:32:21.783 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "jwt_token_ttl", 2024-04-20T11:32:21.783 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-04-20T11:32:21.783 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.783 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.783 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "28800", 2024-04-20T11:32:21.783 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.783 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.783 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.783 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.784 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.784 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.784 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.784 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.784 INFO:teuthology.orchestra.run.smithi152.stdout: "key_file": { 2024-04-20T11:32:21.784 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "key_file", 2024-04-20T11:32:21.784 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.784 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.784 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.784 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.784 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.784 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.784 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.784 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.784 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.785 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.785 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.785 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.785 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-04-20T11:32:21.785 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-04-20T11:32:21.785 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.785 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.785 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.785 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.785 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.785 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.785 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.785 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.785 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.786 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.786 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.786 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.786 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.786 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.786 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.786 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.786 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.786 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.786 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.786 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-04-20T11:32:21.786 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-04-20T11:32:21.786 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.786 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.787 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.787 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.787 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.787 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.787 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.787 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.787 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.787 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.787 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.787 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.787 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-04-20T11:32:21.787 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:21.787 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.788 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.788 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.788 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-04-20T11:32:21.788 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.788 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.788 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.788 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.788 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.788 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.788 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.788 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.788 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.788 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.788 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.789 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.789 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.789 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.789 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.789 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-04-20T11:32:21.789 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-04-20T11:32:21.789 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.789 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.789 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.789 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.789 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.789 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.789 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.789 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.789 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.790 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.790 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.790 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.790 INFO:teuthology.orchestra.run.smithi152.stdout: "motd": { 2024-04-20T11:32:21.790 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "motd", 2024-04-20T11:32:21.790 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.790 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.790 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.790 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.790 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.790 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.790 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.790 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "The message of the day", 2024-04-20T11:32:21.790 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.791 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.791 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.791 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.791 INFO:teuthology.orchestra.run.smithi152.stdout: "redirect_resolve_ip_addr": { 2024-04-20T11:32:21.791 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "redirect_resolve_ip_addr", 2024-04-20T11:32:21.791 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.791 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.791 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.791 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.791 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.791 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.791 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.791 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.791 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.791 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.792 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.792 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.792 INFO:teuthology.orchestra.run.smithi152.stdout: "server_addr": { 2024-04-20T11:32:21.792 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "server_addr", 2024-04-20T11:32:21.792 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.792 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.792 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.792 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "::", 2024-04-20T11:32:21.792 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.792 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.792 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.792 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.792 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.792 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.792 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.793 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.793 INFO:teuthology.orchestra.run.smithi152.stdout: "server_port": { 2024-04-20T11:32:21.793 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "server_port", 2024-04-20T11:32:21.793 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-04-20T11:32:21.793 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.793 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.793 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "8080", 2024-04-20T11:32:21.793 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.793 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.793 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.793 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.793 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.793 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.793 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.794 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.794 INFO:teuthology.orchestra.run.smithi152.stdout: "ssl": { 2024-04-20T11:32:21.794 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "ssl", 2024-04-20T11:32:21.794 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.794 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.794 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.794 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-04-20T11:32:21.794 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.794 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.794 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.794 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.794 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.794 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.794 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.794 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.795 INFO:teuthology.orchestra.run.smithi152.stdout: "ssl_server_port": { 2024-04-20T11:32:21.795 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "ssl_server_port", 2024-04-20T11:32:21.795 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-04-20T11:32:21.795 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.795 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.795 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "8443", 2024-04-20T11:32:21.795 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.795 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.795 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.795 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.795 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.795 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.795 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.795 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.796 INFO:teuthology.orchestra.run.smithi152.stdout: "standby_behaviour": { 2024-04-20T11:32:21.796 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "standby_behaviour", 2024-04-20T11:32:21.796 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.796 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.796 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.796 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "redirect", 2024-04-20T11:32:21.796 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.796 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.796 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.796 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.796 INFO:teuthology.orchestra.run.smithi152.stdout: "redirect" 2024-04-20T11:32:21.796 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.796 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.796 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.796 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.797 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.797 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.797 INFO:teuthology.orchestra.run.smithi152.stdout: "standby_error_status_code": { 2024-04-20T11:32:21.797 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "standby_error_status_code", 2024-04-20T11:32:21.797 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-04-20T11:32:21.797 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.797 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.797 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "500", 2024-04-20T11:32:21.797 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "400", 2024-04-20T11:32:21.797 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "599", 2024-04-20T11:32:21.797 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.797 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.797 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.797 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.797 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.798 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.798 INFO:teuthology.orchestra.run.smithi152.stdout: "url_prefix": { 2024-04-20T11:32:21.798 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "url_prefix", 2024-04-20T11:32:21.798 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.798 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.798 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.798 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.798 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.798 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.798 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.798 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.798 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.798 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.798 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.798 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.799 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.799 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.799 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:32:21.799 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "devicehealth", 2024-04-20T11:32:21.799 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-04-20T11:32:21.799 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-04-20T11:32:21.799 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-04-20T11:32:21.799 INFO:teuthology.orchestra.run.smithi152.stdout: "enable_monitoring": { 2024-04-20T11:32:21.799 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "enable_monitoring", 2024-04-20T11:32:21.799 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.799 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.799 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.799 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-04-20T11:32:21.799 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.800 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.800 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.800 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "monitor device health metrics", 2024-04-20T11:32:21.800 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.800 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.800 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.800 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.800 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-04-20T11:32:21.800 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-04-20T11:32:21.800 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.800 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.800 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.800 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.800 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.800 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.801 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.801 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.801 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.801 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.801 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.801 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.801 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.801 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.801 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.801 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.801 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.801 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.801 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.801 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-04-20T11:32:21.802 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-04-20T11:32:21.802 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.802 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.802 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.802 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.802 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.802 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.802 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.802 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.802 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.802 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.802 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.802 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.802 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-04-20T11:32:21.802 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:21.803 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.803 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.803 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.803 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-04-20T11:32:21.803 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.803 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.803 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.803 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.803 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.803 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.803 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.803 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.803 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.804 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.804 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.804 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.804 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.804 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.804 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.804 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-04-20T11:32:21.804 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-04-20T11:32:21.804 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.804 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.804 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.804 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.804 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.804 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.804 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.805 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.805 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.805 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.805 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.805 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.805 INFO:teuthology.orchestra.run.smithi152.stdout: "mark_out_threshold": { 2024-04-20T11:32:21.805 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "mark_out_threshold", 2024-04-20T11:32:21.805 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "secs", 2024-04-20T11:32:21.805 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.805 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.805 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "2419200", 2024-04-20T11:32:21.805 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.805 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.805 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.806 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-20T11:32:21.806 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.806 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.806 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.806 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.806 INFO:teuthology.orchestra.run.smithi152.stdout: "pool_name": { 2024-04-20T11:32:21.806 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "pool_name", 2024-04-20T11:32:21.806 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.806 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.806 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.806 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "device_health_metrics", 2024-04-20T11:32:21.806 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.806 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.806 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.806 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-20T11:32:21.807 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.807 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.807 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.807 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.807 INFO:teuthology.orchestra.run.smithi152.stdout: "retention_period": { 2024-04-20T11:32:21.807 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "retention_period", 2024-04-20T11:32:21.807 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "secs", 2024-04-20T11:32:21.807 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.807 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.807 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "15552000", 2024-04-20T11:32:21.807 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.807 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.807 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.807 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "how long to retain device health metrics", 2024-04-20T11:32:21.808 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.808 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.808 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.808 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.808 INFO:teuthology.orchestra.run.smithi152.stdout: "scrape_frequency": { 2024-04-20T11:32:21.808 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "scrape_frequency", 2024-04-20T11:32:21.808 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "secs", 2024-04-20T11:32:21.808 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.808 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.808 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "86400", 2024-04-20T11:32:21.808 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.808 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.808 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.808 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-20T11:32:21.808 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.809 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.809 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.809 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.809 INFO:teuthology.orchestra.run.smithi152.stdout: "self_heal": { 2024-04-20T11:32:21.809 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "self_heal", 2024-04-20T11:32:21.809 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.809 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.809 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.809 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-04-20T11:32:21.809 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.809 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.809 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.809 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-20T11:32:21.809 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.809 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.810 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.810 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.810 INFO:teuthology.orchestra.run.smithi152.stdout: "sleep_interval": { 2024-04-20T11:32:21.810 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "sleep_interval", 2024-04-20T11:32:21.810 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "secs", 2024-04-20T11:32:21.810 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.810 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.810 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "600", 2024-04-20T11:32:21.810 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.810 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.810 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.810 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "how frequently to wake up and check device health", 2024-04-20T11:32:21.810 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.810 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.811 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.811 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.811 INFO:teuthology.orchestra.run.smithi152.stdout: "warn_threshold": { 2024-04-20T11:32:21.811 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "warn_threshold", 2024-04-20T11:32:21.811 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "secs", 2024-04-20T11:32:21.811 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.811 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.811 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "7257600", 2024-04-20T11:32:21.811 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.811 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.811 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.811 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-20T11:32:21.811 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.811 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.811 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.812 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.812 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.812 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.812 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:32:21.812 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "diskprediction_local", 2024-04-20T11:32:21.812 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-04-20T11:32:21.812 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-04-20T11:32:21.812 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-04-20T11:32:21.812 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-04-20T11:32:21.812 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-04-20T11:32:21.812 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.812 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.812 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.812 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.812 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.813 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.813 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.813 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.813 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.813 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.813 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.813 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.813 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.813 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.813 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.813 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.813 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.813 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.813 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.813 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-04-20T11:32:21.814 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-04-20T11:32:21.814 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.814 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.814 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.814 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.814 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.814 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.814 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.814 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.814 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.814 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.814 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.814 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.814 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-04-20T11:32:21.815 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:21.815 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.815 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.815 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.815 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-04-20T11:32:21.815 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.815 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.815 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.815 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.815 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.815 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.815 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.815 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.815 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.815 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.816 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.816 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.816 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.816 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.816 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.816 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-04-20T11:32:21.816 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-04-20T11:32:21.816 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.816 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.816 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.816 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.816 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.816 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.816 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.816 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.817 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.817 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.817 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.817 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.817 INFO:teuthology.orchestra.run.smithi152.stdout: "predict_interval": { 2024-04-20T11:32:21.817 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "predict_interval", 2024-04-20T11:32:21.817 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.817 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.817 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.817 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "86400", 2024-04-20T11:32:21.817 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.817 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.817 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.817 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.817 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.818 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.818 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.818 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.818 INFO:teuthology.orchestra.run.smithi152.stdout: "predictor_model": { 2024-04-20T11:32:21.818 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "predictor_model", 2024-04-20T11:32:21.818 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.818 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.818 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.818 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "prophetstor", 2024-04-20T11:32:21.818 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.818 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.818 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.818 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.818 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.819 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.819 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.819 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.819 INFO:teuthology.orchestra.run.smithi152.stdout: "sleep_interval": { 2024-04-20T11:32:21.819 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "sleep_interval", 2024-04-20T11:32:21.819 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.819 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.819 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.819 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "600", 2024-04-20T11:32:21.819 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.819 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.819 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.819 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.819 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.820 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.820 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.820 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.820 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.820 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.820 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:32:21.820 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "influx", 2024-04-20T11:32:21.820 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": false, 2024-04-20T11:32:21.820 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "influxdb python module not found", 2024-04-20T11:32:21.820 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-04-20T11:32:21.820 INFO:teuthology.orchestra.run.smithi152.stdout: "batch_size": { 2024-04-20T11:32:21.820 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "batch_size", 2024-04-20T11:32:21.820 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-04-20T11:32:21.820 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.821 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.821 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "5000", 2024-04-20T11:32:21.821 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.821 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.821 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.821 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-20T11:32:21.821 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.821 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.821 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.821 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.821 INFO:teuthology.orchestra.run.smithi152.stdout: "database": { 2024-04-20T11:32:21.821 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "database", 2024-04-20T11:32:21.821 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.821 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.821 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.822 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "ceph", 2024-04-20T11:32:21.822 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.822 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.822 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.822 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-20T11:32:21.822 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.822 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.822 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.822 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.822 INFO:teuthology.orchestra.run.smithi152.stdout: "hostname": { 2024-04-20T11:32:21.822 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "hostname", 2024-04-20T11:32:21.822 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.822 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.822 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.823 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.823 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.823 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.823 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.823 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "InfluxDB server hostname", 2024-04-20T11:32:21.823 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.823 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.823 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.823 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.823 INFO:teuthology.orchestra.run.smithi152.stdout: "interval": { 2024-04-20T11:32:21.823 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "interval", 2024-04-20T11:32:21.823 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "secs", 2024-04-20T11:32:21.823 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.823 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.824 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "30", 2024-04-20T11:32:21.824 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "5", 2024-04-20T11:32:21.824 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.824 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.824 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-20T11:32:21.824 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.824 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.824 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.824 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.824 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-04-20T11:32:21.824 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-04-20T11:32:21.824 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.824 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.824 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.824 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.825 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.825 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.825 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.825 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.825 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.825 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.825 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.825 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.825 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.825 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.825 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.825 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.825 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.825 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.826 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.826 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-04-20T11:32:21.826 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-04-20T11:32:21.826 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.826 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.826 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.826 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.826 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.826 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.826 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.826 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.826 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.826 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.826 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.827 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.827 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-04-20T11:32:21.827 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:21.827 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.827 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.827 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.827 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-04-20T11:32:21.827 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.827 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.827 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.827 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.827 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.827 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.827 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.827 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.828 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.828 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.828 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.828 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.828 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.828 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.828 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.828 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-04-20T11:32:21.828 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-04-20T11:32:21.828 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.828 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.828 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.828 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.828 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.829 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.829 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.829 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.829 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.829 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.829 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.829 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.829 INFO:teuthology.orchestra.run.smithi152.stdout: "password": { 2024-04-20T11:32:21.829 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "password", 2024-04-20T11:32:21.829 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.829 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.829 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.829 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.829 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.829 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.830 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.830 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "password of InfluxDB server user", 2024-04-20T11:32:21.830 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.830 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.830 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.830 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.830 INFO:teuthology.orchestra.run.smithi152.stdout: "port": { 2024-04-20T11:32:21.830 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "port", 2024-04-20T11:32:21.830 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-04-20T11:32:21.830 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.830 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.830 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "8086", 2024-04-20T11:32:21.830 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.830 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.831 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.831 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "InfluxDB server port", 2024-04-20T11:32:21.831 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.831 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.831 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.831 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.831 INFO:teuthology.orchestra.run.smithi152.stdout: "ssl": { 2024-04-20T11:32:21.831 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "ssl", 2024-04-20T11:32:21.831 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.831 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.831 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.831 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "false", 2024-04-20T11:32:21.831 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.831 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.832 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.832 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-20T11:32:21.832 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.832 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.832 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.832 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.832 INFO:teuthology.orchestra.run.smithi152.stdout: "threads": { 2024-04-20T11:32:21.832 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "threads", 2024-04-20T11:32:21.832 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-04-20T11:32:21.832 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.832 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.832 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "5", 2024-04-20T11:32:21.832 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "1", 2024-04-20T11:32:21.832 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "32", 2024-04-20T11:32:21.832 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.833 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-20T11:32:21.833 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.833 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.833 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.833 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.833 INFO:teuthology.orchestra.run.smithi152.stdout: "username": { 2024-04-20T11:32:21.833 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "username", 2024-04-20T11:32:21.833 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.833 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.833 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.833 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.833 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.833 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.833 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.834 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "username of InfluxDB server user", 2024-04-20T11:32:21.834 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.834 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.834 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.834 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.834 INFO:teuthology.orchestra.run.smithi152.stdout: "verify_ssl": { 2024-04-20T11:32:21.834 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "verify_ssl", 2024-04-20T11:32:21.834 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.834 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.834 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.834 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "true", 2024-04-20T11:32:21.834 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.834 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.835 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.835 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-20T11:32:21.835 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.835 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.835 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.835 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.835 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.835 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.835 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:32:21.835 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "insights", 2024-04-20T11:32:21.835 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-04-20T11:32:21.835 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-04-20T11:32:21.835 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-04-20T11:32:21.835 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-04-20T11:32:21.835 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-04-20T11:32:21.836 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.836 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.836 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.836 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.836 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.836 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.836 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.836 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.836 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.836 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.836 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.836 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.836 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.836 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.837 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.837 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.837 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.837 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.837 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.837 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-04-20T11:32:21.837 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-04-20T11:32:21.837 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.837 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.837 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.837 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.837 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.837 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.837 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.837 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.838 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.838 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.838 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.838 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.838 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-04-20T11:32:21.838 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:21.838 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.838 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.838 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.838 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-04-20T11:32:21.838 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.838 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.838 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.838 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.839 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.839 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.839 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.839 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.839 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.839 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.839 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.839 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.839 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.839 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.839 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.839 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-04-20T11:32:21.839 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-04-20T11:32:21.839 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.839 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.840 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.840 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.840 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.840 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.840 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.840 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.840 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.840 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.840 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.840 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.840 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.840 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.840 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:32:21.840 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "iostat", 2024-04-20T11:32:21.841 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-04-20T11:32:21.841 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-04-20T11:32:21.841 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-04-20T11:32:21.841 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-04-20T11:32:21.841 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-04-20T11:32:21.841 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.841 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.841 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.841 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.841 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.841 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.841 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.841 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.841 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.842 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.842 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.842 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.842 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.842 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.842 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.842 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.842 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.842 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.842 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.842 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-04-20T11:32:21.842 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-04-20T11:32:21.842 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.842 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.842 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.843 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.843 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.843 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.843 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.843 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.843 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.843 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.843 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.843 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.843 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-04-20T11:32:21.843 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:21.843 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.843 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.843 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.844 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-04-20T11:32:21.844 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.844 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.844 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.844 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.844 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.844 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.844 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.844 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.844 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.844 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.844 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.844 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.844 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.844 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.845 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.845 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-04-20T11:32:21.845 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-04-20T11:32:21.845 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.845 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.845 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.845 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.845 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.845 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.845 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.845 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.845 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.845 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.845 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.846 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.846 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.846 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.846 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:32:21.846 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "localpool", 2024-04-20T11:32:21.846 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-04-20T11:32:21.846 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-04-20T11:32:21.846 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-04-20T11:32:21.846 INFO:teuthology.orchestra.run.smithi152.stdout: "failure_domain": { 2024-04-20T11:32:21.846 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "failure_domain", 2024-04-20T11:32:21.846 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.846 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.846 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.846 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "host", 2024-04-20T11:32:21.846 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.847 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.847 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.847 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "failure domain for any created local pool", 2024-04-20T11:32:21.847 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-20T11:32:21.847 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.847 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.847 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.847 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-04-20T11:32:21.847 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-04-20T11:32:21.847 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.847 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.847 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.847 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.847 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.848 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.848 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.848 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.848 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.848 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.848 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.848 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.848 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.848 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.848 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.848 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.848 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.848 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.848 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.848 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-04-20T11:32:21.849 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-04-20T11:32:21.849 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.849 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.849 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.849 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.849 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.849 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.849 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.849 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.849 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.849 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.849 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.849 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.849 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-04-20T11:32:21.849 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:21.850 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.850 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.850 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.850 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-04-20T11:32:21.850 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.850 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.850 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.850 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.850 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.850 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.850 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.850 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.850 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.850 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.851 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.851 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.851 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.851 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.851 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.851 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-04-20T11:32:21.851 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-04-20T11:32:21.851 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.851 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.851 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.851 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.851 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.851 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.852 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.852 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.852 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.852 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.852 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.852 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.852 INFO:teuthology.orchestra.run.smithi152.stdout: "min_size": { 2024-04-20T11:32:21.852 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "min_size", 2024-04-20T11:32:21.852 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-04-20T11:32:21.852 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.852 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.852 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.852 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.852 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.853 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.853 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "default min_size for any created local pool", 2024-04-20T11:32:21.853 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-20T11:32:21.853 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.853 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.853 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.853 INFO:teuthology.orchestra.run.smithi152.stdout: "num_rep": { 2024-04-20T11:32:21.853 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "num_rep", 2024-04-20T11:32:21.853 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-04-20T11:32:21.853 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.853 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.853 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "3", 2024-04-20T11:32:21.853 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.853 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.853 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.854 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "default replica count for any created local pool", 2024-04-20T11:32:21.854 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.854 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.854 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.854 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.854 INFO:teuthology.orchestra.run.smithi152.stdout: "pg_num": { 2024-04-20T11:32:21.854 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "pg_num", 2024-04-20T11:32:21.854 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-04-20T11:32:21.854 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.854 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.854 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "128", 2024-04-20T11:32:21.854 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.854 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.854 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.855 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "default pg_num for any created local pool", 2024-04-20T11:32:21.855 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.855 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.855 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.855 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.855 INFO:teuthology.orchestra.run.smithi152.stdout: "prefix": { 2024-04-20T11:32:21.855 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "prefix", 2024-04-20T11:32:21.855 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.855 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.855 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.855 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.855 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.855 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.855 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.855 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "name prefix for any created local pool", 2024-04-20T11:32:21.856 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.856 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.856 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.856 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.856 INFO:teuthology.orchestra.run.smithi152.stdout: "subtree": { 2024-04-20T11:32:21.856 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "subtree", 2024-04-20T11:32:21.856 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.856 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.856 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.856 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "rack", 2024-04-20T11:32:21.856 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.856 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.856 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.856 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-20T11:32:21.857 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-20T11:32:21.857 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.857 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.857 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.857 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.857 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.857 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:32:21.857 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "mds_autoscaler", 2024-04-20T11:32:21.857 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-04-20T11:32:21.857 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-04-20T11:32:21.857 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-04-20T11:32:21.857 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-04-20T11:32:21.857 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-04-20T11:32:21.857 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.857 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.858 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.858 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.858 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.858 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.858 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.858 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.858 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.858 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.858 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.858 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.858 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.858 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.858 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.858 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.859 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.859 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.859 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.859 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-04-20T11:32:21.859 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-04-20T11:32:21.859 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.859 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.859 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.859 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.859 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.859 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.859 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.859 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.859 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.860 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.860 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.860 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.860 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-04-20T11:32:21.860 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:21.860 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.860 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.860 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.860 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-04-20T11:32:21.860 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.860 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.860 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.860 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.860 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.861 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.861 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.861 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.861 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.861 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.861 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.861 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.861 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.861 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.861 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.861 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-04-20T11:32:21.861 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-04-20T11:32:21.861 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.861 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.861 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.862 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.862 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.862 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.862 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.862 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.862 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.862 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.862 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.862 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.862 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.862 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.862 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:32:21.862 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "mirroring", 2024-04-20T11:32:21.862 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-04-20T11:32:21.863 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-04-20T11:32:21.863 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-04-20T11:32:21.863 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-04-20T11:32:21.863 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-04-20T11:32:21.863 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.863 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.863 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.863 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.863 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.863 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.863 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.863 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.863 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.863 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.864 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.864 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.864 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.864 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.864 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.864 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.864 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.864 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.864 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.864 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-04-20T11:32:21.864 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-04-20T11:32:21.864 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.864 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.864 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.865 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.865 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.865 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.865 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.865 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.865 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.865 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.865 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.865 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.865 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-04-20T11:32:21.865 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:21.865 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.865 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.865 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.865 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-04-20T11:32:21.866 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.866 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.866 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.866 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.866 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.866 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.866 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.866 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.866 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.866 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.866 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.866 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.866 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.867 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.867 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.867 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-04-20T11:32:21.867 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-04-20T11:32:21.867 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.867 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.867 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.867 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.867 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.867 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.867 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.867 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.867 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.868 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.868 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.868 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.868 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.868 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.868 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:32:21.868 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "nfs", 2024-04-20T11:32:21.868 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-04-20T11:32:21.868 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-04-20T11:32:21.868 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-04-20T11:32:21.868 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-04-20T11:32:21.868 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-04-20T11:32:21.868 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.869 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.869 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.869 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.869 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.869 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.869 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.869 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.869 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.869 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.869 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.869 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.869 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.869 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.869 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.870 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.870 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.870 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.870 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.870 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-04-20T11:32:21.870 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-04-20T11:32:21.870 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.870 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.870 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.870 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.870 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.870 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.870 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.870 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.871 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.871 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.871 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.871 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.871 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-04-20T11:32:21.871 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:21.871 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.871 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.871 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.871 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-04-20T11:32:21.871 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.871 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.871 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.872 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.872 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.872 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.872 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.872 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.872 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.872 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.872 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.872 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.872 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.872 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.872 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.872 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-04-20T11:32:21.872 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-04-20T11:32:21.872 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.873 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.873 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.873 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.873 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.873 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.873 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.873 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.873 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.873 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.873 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.873 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.873 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.873 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.873 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:32:21.874 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "orchestrator", 2024-04-20T11:32:21.874 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-04-20T11:32:21.874 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-04-20T11:32:21.874 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-04-20T11:32:21.874 INFO:teuthology.orchestra.run.smithi152.stdout: "fail_fs": { 2024-04-20T11:32:21.874 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "fail_fs", 2024-04-20T11:32:21.874 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.874 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.874 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.874 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.874 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.874 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.874 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.874 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-20T11:32:21.875 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.875 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.875 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.875 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.875 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-04-20T11:32:21.875 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-04-20T11:32:21.875 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.875 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.875 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.875 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.875 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.875 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.875 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.876 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.876 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.876 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.876 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.876 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.876 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.876 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.876 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.876 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.876 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.876 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.876 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.876 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-04-20T11:32:21.876 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-04-20T11:32:21.877 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.877 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.877 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.877 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.877 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.877 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.877 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.877 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.877 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.877 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.877 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.877 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.877 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-04-20T11:32:21.877 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:21.878 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.878 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.878 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.878 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-04-20T11:32:21.878 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.878 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.878 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.878 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.878 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.878 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.878 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.878 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.878 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.878 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.879 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.879 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.879 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.879 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.879 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.879 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-04-20T11:32:21.879 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-04-20T11:32:21.879 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.879 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.879 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.879 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.879 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.879 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.879 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.880 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.880 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.880 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.880 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.880 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.880 INFO:teuthology.orchestra.run.smithi152.stdout: "orchestrator": { 2024-04-20T11:32:21.880 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "orchestrator", 2024-04-20T11:32:21.880 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.880 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.880 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.880 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.880 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.880 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.880 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.880 INFO:teuthology.orchestra.run.smithi152.stdout: "cephadm", 2024-04-20T11:32:21.881 INFO:teuthology.orchestra.run.smithi152.stdout: "rook", 2024-04-20T11:32:21.881 INFO:teuthology.orchestra.run.smithi152.stdout: "test_orchestrator" 2024-04-20T11:32:21.881 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.881 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Orchestrator backend", 2024-04-20T11:32:21.881 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.881 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.881 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.881 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.881 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.881 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.881 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:32:21.881 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "osd_perf_query", 2024-04-20T11:32:21.881 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-04-20T11:32:21.881 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-04-20T11:32:21.882 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-04-20T11:32:21.882 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-04-20T11:32:21.882 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-04-20T11:32:21.882 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.882 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.882 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.882 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.882 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.882 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.882 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.882 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.882 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.882 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.882 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.883 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.883 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.883 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.883 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.883 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.883 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.883 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.883 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.883 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-04-20T11:32:21.883 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-04-20T11:32:21.883 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.883 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.883 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.883 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.884 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.884 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.884 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.884 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.884 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.884 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.884 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.884 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.884 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-04-20T11:32:21.884 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:21.884 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.884 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.884 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.884 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-04-20T11:32:21.884 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.885 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.885 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.885 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.885 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.885 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.885 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.885 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.885 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.885 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.885 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.885 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.885 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.885 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.886 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.886 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-04-20T11:32:21.886 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-04-20T11:32:21.886 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.886 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.886 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.886 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.886 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.886 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.886 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.886 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.886 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.886 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.886 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.887 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.887 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.887 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.887 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:32:21.887 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "osd_support", 2024-04-20T11:32:21.887 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-04-20T11:32:21.887 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-04-20T11:32:21.887 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-04-20T11:32:21.887 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-04-20T11:32:21.887 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-04-20T11:32:21.887 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.887 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.887 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.887 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.888 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.888 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.888 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.888 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.888 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.888 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.888 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.888 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.888 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.888 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.888 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.888 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.888 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.888 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.888 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.889 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-04-20T11:32:21.889 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-04-20T11:32:21.889 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.889 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.889 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.889 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.889 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.889 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.889 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.889 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.889 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.889 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.889 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.889 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.890 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-04-20T11:32:21.890 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:21.890 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.890 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.890 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.890 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-04-20T11:32:21.890 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.890 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.890 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.890 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.890 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.890 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.890 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.890 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.891 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.891 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.891 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.891 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.891 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.891 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.891 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.891 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-04-20T11:32:21.891 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-04-20T11:32:21.891 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.891 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.891 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.891 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.891 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.891 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.892 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.892 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.892 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.892 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.892 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.892 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.892 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.892 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.892 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:32:21.892 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "pg_autoscaler", 2024-04-20T11:32:21.892 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-04-20T11:32:21.892 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-04-20T11:32:21.892 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-04-20T11:32:21.892 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-04-20T11:32:21.893 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-04-20T11:32:21.893 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.893 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.893 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.893 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.893 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.893 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.893 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.893 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.893 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.893 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.893 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.893 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.893 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.893 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.894 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.894 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.894 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.894 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.894 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.894 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-04-20T11:32:21.894 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-04-20T11:32:21.894 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.894 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.894 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.894 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.894 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.894 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.895 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.895 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.895 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.895 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.895 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.895 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.895 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-04-20T11:32:21.895 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:21.895 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.895 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.895 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.895 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-04-20T11:32:21.895 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.895 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.896 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.896 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.896 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.896 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.896 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.896 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.896 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.896 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.896 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.896 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.896 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.896 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.896 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.896 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-04-20T11:32:21.896 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-04-20T11:32:21.897 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.897 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.897 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.897 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.897 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.897 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.897 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.897 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.897 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.897 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.897 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.897 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.897 INFO:teuthology.orchestra.run.smithi152.stdout: "sleep_interval": { 2024-04-20T11:32:21.897 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "sleep_interval", 2024-04-20T11:32:21.898 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "secs", 2024-04-20T11:32:21.898 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.898 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.898 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "60", 2024-04-20T11:32:21.898 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.898 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.898 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.898 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.898 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.898 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.898 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.898 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.898 INFO:teuthology.orchestra.run.smithi152.stdout: "threshold": { 2024-04-20T11:32:21.898 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "threshold", 2024-04-20T11:32:21.898 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "float", 2024-04-20T11:32:21.899 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.899 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.899 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "3.0", 2024-04-20T11:32:21.899 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "1.0", 2024-04-20T11:32:21.899 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.899 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.899 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "scaling threshold", 2024-04-20T11:32:21.899 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-20T11:32:21.899 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.899 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.899 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.899 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.899 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.899 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:32:21.900 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "progress", 2024-04-20T11:32:21.900 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-04-20T11:32:21.900 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-04-20T11:32:21.900 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-04-20T11:32:21.900 INFO:teuthology.orchestra.run.smithi152.stdout: "allow_pg_recovery_event": { 2024-04-20T11:32:21.900 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "allow_pg_recovery_event", 2024-04-20T11:32:21.900 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.900 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.900 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.900 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.900 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.900 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.900 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.900 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-20T11:32:21.900 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.901 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.901 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.901 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.901 INFO:teuthology.orchestra.run.smithi152.stdout: "enabled": { 2024-04-20T11:32:21.901 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "enabled", 2024-04-20T11:32:21.901 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.901 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.901 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.901 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-04-20T11:32:21.901 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.901 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.901 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.901 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.901 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.901 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.902 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.902 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.902 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-04-20T11:32:21.902 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-04-20T11:32:21.902 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.902 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.902 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.902 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.902 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.902 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.903 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.903 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.903 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.903 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.903 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.903 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.903 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.903 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.903 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.903 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.903 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.903 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.903 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.903 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-04-20T11:32:21.904 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-04-20T11:32:21.904 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.904 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.904 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.904 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.904 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.904 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.904 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.904 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.904 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.904 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.904 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.904 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.904 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-04-20T11:32:21.905 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:21.905 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.905 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.905 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.905 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-04-20T11:32:21.905 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.905 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.905 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.905 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.905 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.905 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.905 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.905 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.905 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.906 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.906 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.906 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.906 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.906 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.906 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.906 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-04-20T11:32:21.906 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-04-20T11:32:21.906 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.906 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.906 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.906 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.906 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.906 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.907 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.907 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.907 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.907 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.907 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.907 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.907 INFO:teuthology.orchestra.run.smithi152.stdout: "max_completed_events": { 2024-04-20T11:32:21.907 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "max_completed_events", 2024-04-20T11:32:21.907 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-04-20T11:32:21.907 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.907 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.907 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "50", 2024-04-20T11:32:21.907 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.907 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.908 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.908 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "number of past completed events to remember", 2024-04-20T11:32:21.908 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.908 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.908 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.908 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.908 INFO:teuthology.orchestra.run.smithi152.stdout: "sleep_interval": { 2024-04-20T11:32:21.908 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "sleep_interval", 2024-04-20T11:32:21.908 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "secs", 2024-04-20T11:32:21.908 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.908 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.908 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "5", 2024-04-20T11:32:21.908 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.908 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.909 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.909 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "how long the module is going to sleep", 2024-04-20T11:32:21.909 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.909 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.909 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.909 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.909 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.909 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.909 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:32:21.909 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "prometheus", 2024-04-20T11:32:21.909 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-04-20T11:32:21.909 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-04-20T11:32:21.909 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-04-20T11:32:21.909 INFO:teuthology.orchestra.run.smithi152.stdout: "cache": { 2024-04-20T11:32:21.910 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "cache", 2024-04-20T11:32:21.910 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.910 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.910 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.910 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-04-20T11:32:21.910 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.910 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.910 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.910 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.910 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.910 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.910 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.910 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.911 INFO:teuthology.orchestra.run.smithi152.stdout: "exclude_perf_counters": { 2024-04-20T11:32:21.911 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "exclude_perf_counters", 2024-04-20T11:32:21.911 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.911 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.911 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.911 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-04-20T11:32:21.911 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.911 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.911 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.911 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-20T11:32:21.911 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-20T11:32:21.911 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.911 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.911 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.912 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-04-20T11:32:21.912 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-04-20T11:32:21.912 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.912 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.912 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.912 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.912 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.912 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.912 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.912 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.912 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.912 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.912 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.912 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.912 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.913 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.913 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.913 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.913 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.913 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.913 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.913 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-04-20T11:32:21.913 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-04-20T11:32:21.913 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.913 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.913 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.913 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.913 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.913 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.914 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.914 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.914 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.914 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.914 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.914 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.914 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-04-20T11:32:21.914 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:21.914 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.914 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.914 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.914 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-04-20T11:32:21.914 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.914 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.914 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.915 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.915 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.915 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.915 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.915 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.915 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.915 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.915 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.915 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.915 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.915 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.915 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.915 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-04-20T11:32:21.915 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-04-20T11:32:21.916 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.916 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.916 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.916 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.916 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.916 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.916 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.916 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.916 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.916 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.916 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.916 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.916 INFO:teuthology.orchestra.run.smithi152.stdout: "rbd_stats_pools": { 2024-04-20T11:32:21.916 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "rbd_stats_pools", 2024-04-20T11:32:21.917 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.917 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.917 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.917 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.917 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.917 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.917 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.917 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.917 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.917 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.917 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.917 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.917 INFO:teuthology.orchestra.run.smithi152.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-20T11:32:21.917 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-20T11:32:21.917 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-04-20T11:32:21.918 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.918 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.918 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "300", 2024-04-20T11:32:21.918 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.918 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.918 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.918 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.918 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.918 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.918 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.918 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.918 INFO:teuthology.orchestra.run.smithi152.stdout: "scrape_interval": { 2024-04-20T11:32:21.918 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "scrape_interval", 2024-04-20T11:32:21.919 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "float", 2024-04-20T11:32:21.919 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.919 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.919 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "15.0", 2024-04-20T11:32:21.919 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.919 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.919 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.919 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.919 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.919 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.919 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.919 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.919 INFO:teuthology.orchestra.run.smithi152.stdout: "server_addr": { 2024-04-20T11:32:21.919 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "server_addr", 2024-04-20T11:32:21.920 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.920 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.920 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.920 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "::", 2024-04-20T11:32:21.920 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.920 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.920 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.920 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-20T11:32:21.920 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.920 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.920 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.920 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.920 INFO:teuthology.orchestra.run.smithi152.stdout: "server_port": { 2024-04-20T11:32:21.920 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "server_port", 2024-04-20T11:32:21.920 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-04-20T11:32:21.921 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.921 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.921 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "9283", 2024-04-20T11:32:21.921 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.921 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.921 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.921 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-20T11:32:21.921 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.921 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.921 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.921 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.921 INFO:teuthology.orchestra.run.smithi152.stdout: "stale_cache_strategy": { 2024-04-20T11:32:21.921 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "stale_cache_strategy", 2024-04-20T11:32:21.921 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.921 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.922 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.922 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "log", 2024-04-20T11:32:21.922 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.922 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.922 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.922 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.922 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.922 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.922 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.922 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.922 INFO:teuthology.orchestra.run.smithi152.stdout: "standby_behaviour": { 2024-04-20T11:32:21.922 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "standby_behaviour", 2024-04-20T11:32:21.922 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.922 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.923 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.923 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "default", 2024-04-20T11:32:21.923 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.923 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.923 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.923 INFO:teuthology.orchestra.run.smithi152.stdout: "default", 2024-04-20T11:32:21.923 INFO:teuthology.orchestra.run.smithi152.stdout: "error" 2024-04-20T11:32:21.923 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.923 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.923 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.923 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.923 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.923 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.923 INFO:teuthology.orchestra.run.smithi152.stdout: "standby_error_status_code": { 2024-04-20T11:32:21.924 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "standby_error_status_code", 2024-04-20T11:32:21.924 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-04-20T11:32:21.924 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.924 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.924 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "500", 2024-04-20T11:32:21.924 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "400", 2024-04-20T11:32:21.924 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "599", 2024-04-20T11:32:21.924 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.924 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.924 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.924 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.924 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.924 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.924 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.925 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.925 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:32:21.925 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "rbd_support", 2024-04-20T11:32:21.925 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-04-20T11:32:21.925 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-04-20T11:32:21.925 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-04-20T11:32:21.925 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-04-20T11:32:21.925 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-04-20T11:32:21.925 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.925 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.925 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.925 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.925 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.925 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.925 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.926 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.926 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.926 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.926 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.926 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.926 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.926 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.926 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.926 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.926 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.926 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.926 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.926 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-04-20T11:32:21.926 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-04-20T11:32:21.927 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.927 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.927 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.927 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.927 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.927 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.927 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.927 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.927 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.927 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.927 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.927 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.927 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-04-20T11:32:21.927 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:21.928 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.928 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.928 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.928 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-04-20T11:32:21.928 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.928 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.928 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.928 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.928 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.928 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.928 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.928 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.928 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.928 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.928 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.929 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.929 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.929 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.929 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.929 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-04-20T11:32:21.929 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-04-20T11:32:21.929 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.929 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.929 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.929 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.929 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.929 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.929 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.929 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.930 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.930 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.930 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.930 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.930 INFO:teuthology.orchestra.run.smithi152.stdout: "max_concurrent_snap_create": { 2024-04-20T11:32:21.930 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "max_concurrent_snap_create", 2024-04-20T11:32:21.930 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-04-20T11:32:21.930 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.930 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.930 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "10", 2024-04-20T11:32:21.930 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.930 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.930 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.930 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.931 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.931 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.931 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.931 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.931 INFO:teuthology.orchestra.run.smithi152.stdout: "mirror_snapshot_schedule": { 2024-04-20T11:32:21.931 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "mirror_snapshot_schedule", 2024-04-20T11:32:21.931 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.931 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.931 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.931 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.931 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.931 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.931 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.931 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.932 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.932 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.932 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.932 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.932 INFO:teuthology.orchestra.run.smithi152.stdout: "trash_purge_schedule": { 2024-04-20T11:32:21.932 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "trash_purge_schedule", 2024-04-20T11:32:21.932 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.932 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.932 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.932 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.932 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.932 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.932 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.932 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.933 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.933 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.933 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.933 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.933 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.933 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.933 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:32:21.933 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "restful", 2024-04-20T11:32:21.933 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-04-20T11:32:21.933 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-04-20T11:32:21.933 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-04-20T11:32:21.933 INFO:teuthology.orchestra.run.smithi152.stdout: "enable_auth": { 2024-04-20T11:32:21.933 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "enable_auth", 2024-04-20T11:32:21.933 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.933 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.934 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.934 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-04-20T11:32:21.934 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.934 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.934 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.934 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.934 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.934 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.934 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.934 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.934 INFO:teuthology.orchestra.run.smithi152.stdout: "key_file": { 2024-04-20T11:32:21.934 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "key_file", 2024-04-20T11:32:21.934 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.934 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.935 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.935 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.935 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.935 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.935 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.935 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.935 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.935 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.935 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.935 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.935 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-04-20T11:32:21.935 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-04-20T11:32:21.935 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.936 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.936 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.936 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.936 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.936 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.936 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.936 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.936 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.936 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.936 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.936 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.936 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.936 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.936 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.937 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.937 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.937 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.937 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.937 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-04-20T11:32:21.937 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-04-20T11:32:21.937 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.937 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.937 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.937 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.937 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.937 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.937 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.937 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.938 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.938 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.938 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.938 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.938 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-04-20T11:32:21.938 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:21.938 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.938 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.938 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.938 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-04-20T11:32:21.938 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.938 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.938 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.938 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.939 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.939 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.939 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.939 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.939 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.939 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.939 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.939 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.939 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.939 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.939 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.939 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-04-20T11:32:21.939 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-04-20T11:32:21.939 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.940 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.940 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.940 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.940 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.940 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.940 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.940 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.940 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.940 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.940 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.940 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.940 INFO:teuthology.orchestra.run.smithi152.stdout: "server_addr": { 2024-04-20T11:32:21.940 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "server_addr", 2024-04-20T11:32:21.940 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.941 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.941 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.941 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.941 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.941 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.941 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.941 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.941 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.941 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.941 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.941 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.941 INFO:teuthology.orchestra.run.smithi152.stdout: "server_port": { 2024-04-20T11:32:21.941 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "server_port", 2024-04-20T11:32:21.941 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.941 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.942 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.942 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.942 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.942 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.942 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.942 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.942 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.942 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.942 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.942 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.942 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.942 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.942 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:32:21.943 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "rgw", 2024-04-20T11:32:21.943 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-04-20T11:32:21.943 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-04-20T11:32:21.943 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-04-20T11:32:21.943 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-04-20T11:32:21.943 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-04-20T11:32:21.943 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.943 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.943 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.943 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.943 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.943 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.943 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.943 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.944 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.944 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.944 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.944 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.944 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.944 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.944 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.944 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.944 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.944 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.944 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.944 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-04-20T11:32:21.944 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-04-20T11:32:21.944 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.945 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.945 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.945 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.945 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.945 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.945 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.945 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.945 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.945 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.945 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.945 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.945 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-04-20T11:32:21.945 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:21.945 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.945 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.946 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.946 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-04-20T11:32:21.946 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.946 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.946 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.946 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.946 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.946 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.946 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.946 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.946 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.946 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.946 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.947 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.947 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.947 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.947 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.947 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-04-20T11:32:21.947 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-04-20T11:32:21.947 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.947 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.947 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.947 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.947 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.947 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.947 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.947 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.948 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.948 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.948 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.948 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.948 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.948 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.948 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:32:21.948 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "rook", 2024-04-20T11:32:21.948 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-04-20T11:32:21.948 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-04-20T11:32:21.948 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-04-20T11:32:21.948 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-04-20T11:32:21.948 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-04-20T11:32:21.948 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.949 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.949 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.949 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.949 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.949 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.949 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.949 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.949 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.949 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.949 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.949 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.949 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.949 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.949 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.949 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.950 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.950 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.950 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.950 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-04-20T11:32:21.950 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-04-20T11:32:21.950 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.950 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.950 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.950 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.950 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.950 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.950 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.950 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.951 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.951 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.951 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.951 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.951 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-04-20T11:32:21.951 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:21.951 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.951 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.951 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.951 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-04-20T11:32:21.951 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.951 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.951 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.951 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.952 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.952 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.952 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.952 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.952 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.952 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.952 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.952 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.952 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.952 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.952 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.952 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-04-20T11:32:21.952 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-04-20T11:32:21.952 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.952 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.953 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.953 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.953 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.953 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.953 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.953 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.953 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.953 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.953 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.953 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.953 INFO:teuthology.orchestra.run.smithi152.stdout: "storage_class": { 2024-04-20T11:32:21.953 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "storage_class", 2024-04-20T11:32:21.953 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.953 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.954 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.954 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "local", 2024-04-20T11:32:21.954 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.954 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.954 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.954 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-20T11:32:21.954 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.954 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.954 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.954 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.954 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.954 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.954 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:32:21.954 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "selftest", 2024-04-20T11:32:21.955 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-04-20T11:32:21.955 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-04-20T11:32:21.955 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-04-20T11:32:21.955 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-04-20T11:32:21.955 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-04-20T11:32:21.955 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.955 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.955 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.955 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.955 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.955 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.955 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.955 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.955 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.956 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.956 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.956 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.956 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.956 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.956 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.956 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.956 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.956 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.956 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.956 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-04-20T11:32:21.956 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-04-20T11:32:21.956 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.956 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.956 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.957 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.957 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.957 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.957 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.957 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.957 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.957 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.957 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.957 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.957 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-04-20T11:32:21.957 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:21.957 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.957 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.957 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.957 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-04-20T11:32:21.958 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.958 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.958 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.958 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.958 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.958 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.958 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.958 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.958 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.958 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.958 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.958 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.958 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.958 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.959 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.959 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-04-20T11:32:21.959 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-04-20T11:32:21.959 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.959 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.959 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.959 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.959 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.959 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.959 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.959 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.959 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.959 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.959 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.959 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.960 INFO:teuthology.orchestra.run.smithi152.stdout: "roption1": { 2024-04-20T11:32:21.960 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "roption1", 2024-04-20T11:32:21.960 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.960 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.960 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.960 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.960 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.960 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.960 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.960 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.960 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.960 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.960 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.960 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.961 INFO:teuthology.orchestra.run.smithi152.stdout: "roption2": { 2024-04-20T11:32:21.961 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "roption2", 2024-04-20T11:32:21.961 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.961 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.961 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.961 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "xyz", 2024-04-20T11:32:21.961 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.961 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.961 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.961 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.961 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.961 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.961 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.961 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.961 INFO:teuthology.orchestra.run.smithi152.stdout: "rwoption1": { 2024-04-20T11:32:21.962 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "rwoption1", 2024-04-20T11:32:21.962 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.962 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.962 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.962 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.962 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.962 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.962 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.962 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.962 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.962 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.962 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.962 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.962 INFO:teuthology.orchestra.run.smithi152.stdout: "rwoption2": { 2024-04-20T11:32:21.963 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "rwoption2", 2024-04-20T11:32:21.963 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-04-20T11:32:21.963 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.963 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.963 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.963 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.963 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.963 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.963 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.963 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.963 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.963 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.963 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.963 INFO:teuthology.orchestra.run.smithi152.stdout: "rwoption3": { 2024-04-20T11:32:21.964 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "rwoption3", 2024-04-20T11:32:21.964 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "float", 2024-04-20T11:32:21.964 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.964 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.964 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.964 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.964 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.964 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.964 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.964 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.964 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.964 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.964 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.964 INFO:teuthology.orchestra.run.smithi152.stdout: "rwoption4": { 2024-04-20T11:32:21.964 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "rwoption4", 2024-04-20T11:32:21.965 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.965 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.965 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.965 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.965 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.965 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.965 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.965 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.965 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.965 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.965 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.965 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.965 INFO:teuthology.orchestra.run.smithi152.stdout: "rwoption5": { 2024-04-20T11:32:21.965 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "rwoption5", 2024-04-20T11:32:21.965 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.966 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.966 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.966 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.966 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.966 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.966 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.966 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.966 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.966 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.966 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.966 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.966 INFO:teuthology.orchestra.run.smithi152.stdout: "rwoption6": { 2024-04-20T11:32:21.966 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "rwoption6", 2024-04-20T11:32:21.966 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.967 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.967 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.967 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-04-20T11:32:21.967 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.967 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.967 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.967 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.967 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.967 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.967 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.967 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.967 INFO:teuthology.orchestra.run.smithi152.stdout: "rwoption7": { 2024-04-20T11:32:21.967 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "rwoption7", 2024-04-20T11:32:21.967 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-04-20T11:32:21.967 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.968 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.968 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.968 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "1", 2024-04-20T11:32:21.968 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "42", 2024-04-20T11:32:21.968 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.968 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.968 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.968 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.968 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.968 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.968 INFO:teuthology.orchestra.run.smithi152.stdout: "testkey": { 2024-04-20T11:32:21.968 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "testkey", 2024-04-20T11:32:21.968 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.968 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.969 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.969 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.969 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.969 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.969 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.969 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.969 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.969 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.969 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.969 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.969 INFO:teuthology.orchestra.run.smithi152.stdout: "testlkey": { 2024-04-20T11:32:21.969 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "testlkey", 2024-04-20T11:32:21.969 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.969 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.970 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.970 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.970 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.970 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.970 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.970 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.970 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.970 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.970 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.970 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.970 INFO:teuthology.orchestra.run.smithi152.stdout: "testnewline": { 2024-04-20T11:32:21.970 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "testnewline", 2024-04-20T11:32:21.970 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.970 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.971 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.971 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.971 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.971 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.971 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.971 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.971 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.971 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.971 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.971 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.971 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.971 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.971 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:32:21.971 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "snap_schedule", 2024-04-20T11:32:21.972 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-04-20T11:32:21.972 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-04-20T11:32:21.972 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-04-20T11:32:21.972 INFO:teuthology.orchestra.run.smithi152.stdout: "allow_m_granularity": { 2024-04-20T11:32:21.972 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "allow_m_granularity", 2024-04-20T11:32:21.972 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.972 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.972 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.972 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.972 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.972 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.972 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.972 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "allow minute scheduled snapshots", 2024-04-20T11:32:21.972 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.973 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.973 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.973 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.973 INFO:teuthology.orchestra.run.smithi152.stdout: "dump_on_update": { 2024-04-20T11:32:21.973 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "dump_on_update", 2024-04-20T11:32:21.973 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.973 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.973 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.973 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.973 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.973 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.973 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.973 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "dump database to debug log on update", 2024-04-20T11:32:21.973 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.974 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.974 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.974 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.974 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-04-20T11:32:21.974 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-04-20T11:32:21.974 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.974 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.974 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.974 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.974 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.974 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.974 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.974 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.974 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.974 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.975 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.975 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.975 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.975 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.975 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.975 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.975 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.975 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.975 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.975 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-04-20T11:32:21.975 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-04-20T11:32:21.975 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.976 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.976 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.976 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.976 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.976 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.976 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.976 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.976 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.976 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.976 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.976 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.976 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-04-20T11:32:21.976 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:21.976 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.977 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.977 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.977 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-04-20T11:32:21.977 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.977 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.977 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.977 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.977 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.977 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.977 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.977 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.977 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.977 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.977 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.977 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.978 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.978 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.978 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.978 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-04-20T11:32:21.978 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-04-20T11:32:21.978 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.978 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.978 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.978 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.978 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.978 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.978 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.978 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.978 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.979 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.979 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.979 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.979 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.979 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.979 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:32:21.979 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "stats", 2024-04-20T11:32:21.979 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-04-20T11:32:21.979 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-04-20T11:32:21.979 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-04-20T11:32:21.979 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-04-20T11:32:21.979 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-04-20T11:32:21.979 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.979 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.980 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.980 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.980 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.980 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.980 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.980 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.980 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.980 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.980 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.980 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.980 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.980 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.980 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.980 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.981 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.981 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.981 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.981 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-04-20T11:32:21.981 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-04-20T11:32:21.981 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.981 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.981 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.981 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.981 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.981 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.981 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.981 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.981 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.981 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.982 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.982 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.982 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-04-20T11:32:21.982 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:21.982 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.982 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.982 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.982 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-04-20T11:32:21.982 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.982 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.982 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.982 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.982 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.983 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.983 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.983 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.983 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.983 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.983 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.983 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.983 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.983 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.983 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.983 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-04-20T11:32:21.983 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-04-20T11:32:21.983 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.983 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.983 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.984 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.984 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.984 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.984 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.984 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.984 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.984 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.984 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.984 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.984 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.984 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.984 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:32:21.984 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "status", 2024-04-20T11:32:21.984 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-04-20T11:32:21.985 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-04-20T11:32:21.985 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-04-20T11:32:21.985 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-04-20T11:32:21.985 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-04-20T11:32:21.985 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.985 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.985 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.985 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.985 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.985 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.985 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.985 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.985 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.985 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.985 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.986 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.986 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.986 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.986 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.986 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.986 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.986 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.986 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.986 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-04-20T11:32:21.986 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-04-20T11:32:21.986 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.986 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.986 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.987 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.987 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.987 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.987 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.987 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.987 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.987 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.987 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.987 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.987 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-04-20T11:32:21.987 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:21.987 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.987 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.987 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.988 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-04-20T11:32:21.988 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.988 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.988 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.988 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.988 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.988 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.988 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.988 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.988 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.988 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.988 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.988 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.988 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.989 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.989 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.989 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-04-20T11:32:21.989 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-04-20T11:32:21.989 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.989 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.989 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.989 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.989 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.989 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.989 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.989 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.989 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.989 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.989 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.990 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.990 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.990 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.990 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:32:21.990 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "telegraf", 2024-04-20T11:32:21.990 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-04-20T11:32:21.990 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-04-20T11:32:21.990 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-04-20T11:32:21.990 INFO:teuthology.orchestra.run.smithi152.stdout: "address": { 2024-04-20T11:32:21.990 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "address", 2024-04-20T11:32:21.990 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.990 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.990 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.990 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-20T11:32:21.991 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.991 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.991 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.991 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.991 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.991 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.991 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.991 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.991 INFO:teuthology.orchestra.run.smithi152.stdout: "interval": { 2024-04-20T11:32:21.991 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "interval", 2024-04-20T11:32:21.991 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "secs", 2024-04-20T11:32:21.991 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.991 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.991 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "15", 2024-04-20T11:32:21.991 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.992 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.992 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.992 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.992 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.992 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.992 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.992 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.992 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-04-20T11:32:21.992 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-04-20T11:32:21.992 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.992 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.992 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.992 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:21.992 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.993 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.993 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.993 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.993 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.993 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.993 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.993 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.993 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.993 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.993 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.993 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.993 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.993 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.993 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.993 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-04-20T11:32:21.994 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-04-20T11:32:21.994 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.994 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.994 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.994 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.994 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.994 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.994 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.994 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.994 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.994 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.994 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.994 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.994 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-04-20T11:32:21.995 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:21.995 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:21.995 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.995 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.995 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-04-20T11:32:21.995 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.995 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.995 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:21.995 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:21.995 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:21.995 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:21.995 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:21.995 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:21.995 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:21.996 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:21.996 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.996 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.996 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.996 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.996 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.996 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-04-20T11:32:21.996 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-04-20T11:32:21.996 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.996 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.996 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:21.996 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:21.996 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.996 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.996 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.997 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:21.997 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.997 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.997 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.997 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.997 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:21.997 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.997 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:32:21.997 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "telemetry", 2024-04-20T11:32:21.997 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-04-20T11:32:21.997 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-04-20T11:32:21.997 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-04-20T11:32:21.997 INFO:teuthology.orchestra.run.smithi152.stdout: "channel_basic": { 2024-04-20T11:32:21.997 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "channel_basic", 2024-04-20T11:32:21.997 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.998 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.998 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.998 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-04-20T11:32:21.998 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.998 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.998 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.998 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-20T11:32:21.998 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.998 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.998 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.998 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.998 INFO:teuthology.orchestra.run.smithi152.stdout: "channel_crash": { 2024-04-20T11:32:21.998 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "channel_crash", 2024-04-20T11:32:21.998 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.999 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:21.999 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:21.999 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-04-20T11:32:21.999 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:21.999 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:21.999 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:21.999 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-20T11:32:21.999 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:21.999 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:21.999 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:21.999 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:21.999 INFO:teuthology.orchestra.run.smithi152.stdout: "channel_device": { 2024-04-20T11:32:21.999 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "channel_device", 2024-04-20T11:32:21.999 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:21.999 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:22.000 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:22.000 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-04-20T11:32:22.000 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:22.000 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:22.000 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:22.000 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-20T11:32:22.000 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:22.000 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:22.000 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:22.000 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:22.000 INFO:teuthology.orchestra.run.smithi152.stdout: "channel_ident": { 2024-04-20T11:32:22.000 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "channel_ident", 2024-04-20T11:32:22.000 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:22.000 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:22.001 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:22.001 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:22.001 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:22.001 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:22.001 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:22.001 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-20T11:32:22.001 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:22.001 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:22.001 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:22.001 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:22.001 INFO:teuthology.orchestra.run.smithi152.stdout: "channel_perf": { 2024-04-20T11:32:22.001 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "channel_perf", 2024-04-20T11:32:22.001 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:22.001 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:22.001 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:22.002 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:22.002 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:22.002 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:22.002 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:22.002 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-20T11:32:22.002 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:22.002 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:22.002 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:22.002 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:22.002 INFO:teuthology.orchestra.run.smithi152.stdout: "contact": { 2024-04-20T11:32:22.002 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "contact", 2024-04-20T11:32:22.002 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:22.002 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:22.003 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:22.003 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:22.003 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:22.003 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:22.003 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:22.003 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:22.003 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:22.003 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:22.003 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:22.003 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:22.003 INFO:teuthology.orchestra.run.smithi152.stdout: "description": { 2024-04-20T11:32:22.003 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "description", 2024-04-20T11:32:22.003 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:22.003 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:22.003 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:22.004 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:22.004 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:22.004 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:22.004 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:22.004 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:22.004 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:22.004 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:22.004 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:22.004 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:22.004 INFO:teuthology.orchestra.run.smithi152.stdout: "device_url": { 2024-04-20T11:32:22.004 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "device_url", 2024-04-20T11:32:22.004 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:22.004 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:22.004 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:22.005 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-20T11:32:22.005 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:22.005 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:22.005 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:22.005 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:22.005 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:22.005 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:22.005 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:22.005 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:22.005 INFO:teuthology.orchestra.run.smithi152.stdout: "enabled": { 2024-04-20T11:32:22.005 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "enabled", 2024-04-20T11:32:22.005 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:22.005 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:22.005 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:22.005 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:22.006 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:22.006 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:22.006 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:22.006 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:22.006 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:22.006 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:22.006 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:22.006 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:22.006 INFO:teuthology.orchestra.run.smithi152.stdout: "interval": { 2024-04-20T11:32:22.006 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "interval", 2024-04-20T11:32:22.006 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-04-20T11:32:22.006 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:22.006 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:22.007 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "24", 2024-04-20T11:32:22.007 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "8", 2024-04-20T11:32:22.007 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:22.007 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:22.007 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:22.007 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:22.007 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:22.007 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:22.007 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:22.007 INFO:teuthology.orchestra.run.smithi152.stdout: "last_opt_revision": { 2024-04-20T11:32:22.007 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "last_opt_revision", 2024-04-20T11:32:22.007 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-04-20T11:32:22.007 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:22.007 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:22.007 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "1", 2024-04-20T11:32:22.008 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:22.008 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:22.008 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:22.008 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:22.008 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:22.008 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:22.008 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:22.008 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:22.008 INFO:teuthology.orchestra.run.smithi152.stdout: "leaderboard": { 2024-04-20T11:32:22.008 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "leaderboard", 2024-04-20T11:32:22.008 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:22.008 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:22.008 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:22.008 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:22.009 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:22.009 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:22.009 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:22.009 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:22.009 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:22.009 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:22.009 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:22.009 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:22.009 INFO:teuthology.orchestra.run.smithi152.stdout: "leaderboard_description": { 2024-04-20T11:32:22.009 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "leaderboard_description", 2024-04-20T11:32:22.009 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:22.009 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:22.009 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:22.009 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:22.009 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:22.010 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:22.010 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:22.010 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:22.010 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:22.010 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:22.010 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:22.010 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:22.010 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-04-20T11:32:22.010 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-04-20T11:32:22.010 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:22.010 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:22.010 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:22.010 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:22.010 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:22.011 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:22.011 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:22.011 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:22.011 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:22.011 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:22.011 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:22.011 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:22.011 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:22.011 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:22.011 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:22.011 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:22.011 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:22.011 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:22.011 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:22.012 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-04-20T11:32:22.012 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-04-20T11:32:22.012 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:22.012 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:22.012 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:22.012 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:22.012 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:22.012 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:22.012 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:22.012 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:22.012 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:22.012 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:22.012 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:22.012 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:22.013 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-04-20T11:32:22.013 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:22.013 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:22.013 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:22.013 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:22.013 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-04-20T11:32:22.013 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:22.013 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:22.013 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:22.013 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:22.013 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:22.013 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:22.013 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:22.013 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:22.013 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:22.014 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:22.014 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:22.014 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:22.014 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:22.014 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:22.014 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:22.014 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-04-20T11:32:22.014 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-04-20T11:32:22.014 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:22.014 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:22.014 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:22.014 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:22.014 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:22.014 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:22.015 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:22.015 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:22.015 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:22.015 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:22.015 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:22.015 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:22.015 INFO:teuthology.orchestra.run.smithi152.stdout: "organization": { 2024-04-20T11:32:22.015 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "organization", 2024-04-20T11:32:22.015 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:22.015 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:22.015 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:22.015 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:22.015 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:22.015 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:22.016 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:22.016 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:22.016 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:22.016 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:22.016 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:22.016 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:22.016 INFO:teuthology.orchestra.run.smithi152.stdout: "proxy": { 2024-04-20T11:32:22.016 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "proxy", 2024-04-20T11:32:22.016 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:22.016 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:22.016 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:22.016 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:22.016 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:22.016 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:22.016 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:22.017 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:22.017 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:22.017 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:22.017 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:22.017 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:22.017 INFO:teuthology.orchestra.run.smithi152.stdout: "url": { 2024-04-20T11:32:22.017 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "url", 2024-04-20T11:32:22.017 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:22.017 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:22.017 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:22.017 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-20T11:32:22.017 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:22.017 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:22.017 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:22.018 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:22.018 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:22.018 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:22.018 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:22.018 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:22.018 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:22.018 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:22.018 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:32:22.018 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "test_orchestrator", 2024-04-20T11:32:22.018 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-04-20T11:32:22.018 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-04-20T11:32:22.018 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-04-20T11:32:22.018 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-04-20T11:32:22.018 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-04-20T11:32:22.019 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:22.019 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:22.019 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:22.019 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:22.019 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:22.019 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:22.019 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:22.019 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:22.019 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:22.019 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:22.019 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:22.019 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:22.019 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:22.019 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:22.020 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:22.020 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:22.020 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:22.020 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:22.020 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:22.020 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-04-20T11:32:22.020 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-04-20T11:32:22.020 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:22.020 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:22.020 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:22.020 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:22.020 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:22.020 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:22.020 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:22.020 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:22.021 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:22.021 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:22.021 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:22.021 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:22.021 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-04-20T11:32:22.021 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:22.021 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:22.021 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:22.021 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:22.021 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-04-20T11:32:22.021 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:22.021 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:22.021 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:22.021 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:22.022 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:22.022 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:22.022 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:22.022 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:22.022 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:22.022 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:22.022 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:22.022 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:22.022 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:22.022 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:22.022 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:22.022 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-04-20T11:32:22.022 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-04-20T11:32:22.022 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:22.023 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:22.023 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:22.023 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:22.023 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:22.023 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:22.023 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:22.023 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:22.023 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:22.023 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:22.023 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:22.023 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:22.023 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:22.023 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:22.023 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:32:22.023 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "volumes", 2024-04-20T11:32:22.024 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-04-20T11:32:22.024 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-04-20T11:32:22.024 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-04-20T11:32:22.024 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-04-20T11:32:22.024 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-04-20T11:32:22.024 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:22.024 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:22.024 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:22.024 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:22.024 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:22.024 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:22.024 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:22.024 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:22.024 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:22.025 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:22.025 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:22.025 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:22.025 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:22.025 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:22.025 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:22.025 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:22.025 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:22.025 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:22.025 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:22.025 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-04-20T11:32:22.025 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-04-20T11:32:22.025 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:22.025 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:22.026 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:22.026 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:22.026 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:22.026 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:22.026 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:22.026 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:22.026 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:22.026 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:22.026 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:22.026 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:22.026 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-04-20T11:32:22.026 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:22.026 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:22.027 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:22.027 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:22.027 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-04-20T11:32:22.027 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:22.027 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:22.027 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:22.027 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:22.027 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:22.027 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:22.027 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:22.027 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:22.027 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:22.027 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:22.027 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:22.028 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:22.028 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:22.028 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:22.028 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:22.028 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-04-20T11:32:22.028 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-04-20T11:32:22.028 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:22.028 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:22.028 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:22.028 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:22.028 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:22.028 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:22.028 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:22.028 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:22.029 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:22.029 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:22.029 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:22.029 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:22.029 INFO:teuthology.orchestra.run.smithi152.stdout: "max_concurrent_clones": { 2024-04-20T11:32:22.029 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "max_concurrent_clones", 2024-04-20T11:32:22.029 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-04-20T11:32:22.029 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:22.029 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:22.029 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "4", 2024-04-20T11:32:22.029 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:22.029 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:22.029 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:22.029 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-20T11:32:22.029 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:22.029 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:22.030 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:22.030 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:22.030 INFO:teuthology.orchestra.run.smithi152.stdout: "periodic_async_work": { 2024-04-20T11:32:22.030 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "periodic_async_work", 2024-04-20T11:32:22.030 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:22.030 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:22.030 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:22.030 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:22.030 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:22.030 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:22.030 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:22.030 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Periodically check for async work", 2024-04-20T11:32:22.030 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:22.031 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:22.031 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:22.031 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:22.031 INFO:teuthology.orchestra.run.smithi152.stdout: "snapshot_clone_delay": { 2024-04-20T11:32:22.031 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "snapshot_clone_delay", 2024-04-20T11:32:22.031 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-04-20T11:32:22.031 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:22.031 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:22.031 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "0", 2024-04-20T11:32:22.031 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:22.031 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:22.031 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:22.031 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-20T11:32:22.031 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:22.031 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:22.032 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:22.032 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:22.032 INFO:teuthology.orchestra.run.smithi152.stdout: "snapshot_clone_no_wait": { 2024-04-20T11:32:22.032 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "snapshot_clone_no_wait", 2024-04-20T11:32:22.032 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:22.032 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:22.032 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:22.032 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-04-20T11:32:22.032 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:22.032 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:22.032 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:22.032 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-20T11:32:22.032 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:22.033 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:22.033 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:22.033 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:22.033 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:22.033 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:22.033 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:32:22.033 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "zabbix", 2024-04-20T11:32:22.033 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-04-20T11:32:22.033 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-04-20T11:32:22.033 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-04-20T11:32:22.033 INFO:teuthology.orchestra.run.smithi152.stdout: "discovery_interval": { 2024-04-20T11:32:22.033 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "discovery_interval", 2024-04-20T11:32:22.033 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "uint", 2024-04-20T11:32:22.033 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:22.034 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:22.034 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "100", 2024-04-20T11:32:22.034 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:22.034 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:22.034 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:22.034 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:22.034 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:22.034 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:22.034 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:22.034 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:22.034 INFO:teuthology.orchestra.run.smithi152.stdout: "identifier": { 2024-04-20T11:32:22.034 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "identifier", 2024-04-20T11:32:22.034 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:22.034 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:22.034 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:22.035 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:22.035 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:22.035 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:22.035 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:22.035 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:22.035 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:22.035 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:22.035 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:22.035 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:22.035 INFO:teuthology.orchestra.run.smithi152.stdout: "interval": { 2024-04-20T11:32:22.035 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "interval", 2024-04-20T11:32:22.035 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "secs", 2024-04-20T11:32:22.035 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:22.035 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:22.036 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "60", 2024-04-20T11:32:22.036 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:22.036 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:22.036 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:22.036 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:22.036 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:22.036 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:22.036 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:22.036 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:22.036 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-04-20T11:32:22.036 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-04-20T11:32:22.036 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:22.036 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:22.036 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:22.037 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:22.037 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:22.037 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:22.037 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:22.037 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:22.037 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:22.037 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:22.037 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:22.037 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:22.037 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:22.037 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:22.037 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:22.037 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:22.037 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:22.037 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:22.038 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:22.038 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-04-20T11:32:22.038 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-04-20T11:32:22.038 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:22.038 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:22.038 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:22.038 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:22.038 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:22.038 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:22.038 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:22.038 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:22.038 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:22.038 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:22.038 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:22.039 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:22.039 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-04-20T11:32:22.039 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:22.039 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:22.039 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:22.039 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:22.039 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-04-20T11:32:22.039 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:22.039 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:22.039 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-04-20T11:32:22.039 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-04-20T11:32:22.039 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-04-20T11:32:22.039 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-04-20T11:32:22.039 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-04-20T11:32:22.040 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-04-20T11:32:22.040 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-04-20T11:32:22.040 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:22.040 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:22.040 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:22.040 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:22.040 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:22.040 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:22.040 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-04-20T11:32:22.040 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-04-20T11:32:22.040 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-04-20T11:32:22.040 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:22.040 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-04-20T11:32:22.040 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-04-20T11:32:22.041 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:22.041 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:22.041 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:22.041 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:22.041 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:22.041 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:22.041 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:22.041 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:22.041 INFO:teuthology.orchestra.run.smithi152.stdout: "zabbix_host": { 2024-04-20T11:32:22.041 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "zabbix_host", 2024-04-20T11:32:22.041 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:22.041 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:22.041 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:22.041 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-04-20T11:32:22.041 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:22.042 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:22.042 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:22.042 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:22.042 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:22.042 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:22.042 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:22.042 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:22.042 INFO:teuthology.orchestra.run.smithi152.stdout: "zabbix_port": { 2024-04-20T11:32:22.042 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "zabbix_port", 2024-04-20T11:32:22.042 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-04-20T11:32:22.042 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:22.042 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:22.042 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "10051", 2024-04-20T11:32:22.042 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:22.043 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:22.043 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:22.043 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:22.043 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:22.043 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:22.043 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:22.043 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:22.043 INFO:teuthology.orchestra.run.smithi152.stdout: "zabbix_sender": { 2024-04-20T11:32:22.043 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "zabbix_sender", 2024-04-20T11:32:22.043 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-04-20T11:32:22.043 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-04-20T11:32:22.043 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-04-20T11:32:22.043 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-20T11:32:22.043 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-04-20T11:32:22.044 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-04-20T11:32:22.044 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-04-20T11:32:22.044 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-04-20T11:32:22.044 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-04-20T11:32:22.044 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-04-20T11:32:22.044 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-04-20T11:32:22.044 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:22.044 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:22.044 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:22.044 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:22.044 INFO:teuthology.orchestra.run.smithi152.stdout: "services": {}, 2024-04-20T11:32:22.044 INFO:teuthology.orchestra.run.smithi152.stdout: "always_on_modules": { 2024-04-20T11:32:22.044 INFO:teuthology.orchestra.run.smithi152.stdout: "octopus": [ 2024-04-20T11:32:22.044 INFO:teuthology.orchestra.run.smithi152.stdout: "balancer", 2024-04-20T11:32:22.044 INFO:teuthology.orchestra.run.smithi152.stdout: "crash", 2024-04-20T11:32:22.045 INFO:teuthology.orchestra.run.smithi152.stdout: "devicehealth", 2024-04-20T11:32:22.045 INFO:teuthology.orchestra.run.smithi152.stdout: "orchestrator", 2024-04-20T11:32:22.045 INFO:teuthology.orchestra.run.smithi152.stdout: "pg_autoscaler", 2024-04-20T11:32:22.045 INFO:teuthology.orchestra.run.smithi152.stdout: "progress", 2024-04-20T11:32:22.045 INFO:teuthology.orchestra.run.smithi152.stdout: "rbd_support", 2024-04-20T11:32:22.045 INFO:teuthology.orchestra.run.smithi152.stdout: "status", 2024-04-20T11:32:22.045 INFO:teuthology.orchestra.run.smithi152.stdout: "telemetry", 2024-04-20T11:32:22.045 INFO:teuthology.orchestra.run.smithi152.stdout: "volumes" 2024-04-20T11:32:22.045 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:22.045 INFO:teuthology.orchestra.run.smithi152.stdout: "pacific": [ 2024-04-20T11:32:22.045 INFO:teuthology.orchestra.run.smithi152.stdout: "balancer", 2024-04-20T11:32:22.045 INFO:teuthology.orchestra.run.smithi152.stdout: "crash", 2024-04-20T11:32:22.045 INFO:teuthology.orchestra.run.smithi152.stdout: "devicehealth", 2024-04-20T11:32:22.045 INFO:teuthology.orchestra.run.smithi152.stdout: "orchestrator", 2024-04-20T11:32:22.046 INFO:teuthology.orchestra.run.smithi152.stdout: "pg_autoscaler", 2024-04-20T11:32:22.046 INFO:teuthology.orchestra.run.smithi152.stdout: "progress", 2024-04-20T11:32:22.046 INFO:teuthology.orchestra.run.smithi152.stdout: "rbd_support", 2024-04-20T11:32:22.046 INFO:teuthology.orchestra.run.smithi152.stdout: "status", 2024-04-20T11:32:22.046 INFO:teuthology.orchestra.run.smithi152.stdout: "telemetry", 2024-04-20T11:32:22.046 INFO:teuthology.orchestra.run.smithi152.stdout: "volumes" 2024-04-20T11:32:22.046 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:22.046 INFO:teuthology.orchestra.run.smithi152.stdout: "quincy": [ 2024-04-20T11:32:22.046 INFO:teuthology.orchestra.run.smithi152.stdout: "balancer", 2024-04-20T11:32:22.046 INFO:teuthology.orchestra.run.smithi152.stdout: "crash", 2024-04-20T11:32:22.046 INFO:teuthology.orchestra.run.smithi152.stdout: "devicehealth", 2024-04-20T11:32:22.046 INFO:teuthology.orchestra.run.smithi152.stdout: "orchestrator", 2024-04-20T11:32:22.046 INFO:teuthology.orchestra.run.smithi152.stdout: "pg_autoscaler", 2024-04-20T11:32:22.046 INFO:teuthology.orchestra.run.smithi152.stdout: "progress", 2024-04-20T11:32:22.047 INFO:teuthology.orchestra.run.smithi152.stdout: "rbd_support", 2024-04-20T11:32:22.047 INFO:teuthology.orchestra.run.smithi152.stdout: "status", 2024-04-20T11:32:22.047 INFO:teuthology.orchestra.run.smithi152.stdout: "telemetry", 2024-04-20T11:32:22.047 INFO:teuthology.orchestra.run.smithi152.stdout: "volumes" 2024-04-20T11:32:22.047 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:22.047 INFO:teuthology.orchestra.run.smithi152.stdout: "reef": [ 2024-04-20T11:32:22.047 INFO:teuthology.orchestra.run.smithi152.stdout: "balancer", 2024-04-20T11:32:22.047 INFO:teuthology.orchestra.run.smithi152.stdout: "crash", 2024-04-20T11:32:22.047 INFO:teuthology.orchestra.run.smithi152.stdout: "devicehealth", 2024-04-20T11:32:22.047 INFO:teuthology.orchestra.run.smithi152.stdout: "orchestrator", 2024-04-20T11:32:22.047 INFO:teuthology.orchestra.run.smithi152.stdout: "pg_autoscaler", 2024-04-20T11:32:22.047 INFO:teuthology.orchestra.run.smithi152.stdout: "progress", 2024-04-20T11:32:22.047 INFO:teuthology.orchestra.run.smithi152.stdout: "rbd_support", 2024-04-20T11:32:22.047 INFO:teuthology.orchestra.run.smithi152.stdout: "status", 2024-04-20T11:32:22.048 INFO:teuthology.orchestra.run.smithi152.stdout: "telemetry", 2024-04-20T11:32:22.048 INFO:teuthology.orchestra.run.smithi152.stdout: "volumes" 2024-04-20T11:32:22.048 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:32:22.048 INFO:teuthology.orchestra.run.smithi152.stdout: "squid": [ 2024-04-20T11:32:22.048 INFO:teuthology.orchestra.run.smithi152.stdout: "balancer", 2024-04-20T11:32:22.048 INFO:teuthology.orchestra.run.smithi152.stdout: "crash", 2024-04-20T11:32:22.048 INFO:teuthology.orchestra.run.smithi152.stdout: "devicehealth", 2024-04-20T11:32:22.048 INFO:teuthology.orchestra.run.smithi152.stdout: "orchestrator", 2024-04-20T11:32:22.048 INFO:teuthology.orchestra.run.smithi152.stdout: "pg_autoscaler", 2024-04-20T11:32:22.048 INFO:teuthology.orchestra.run.smithi152.stdout: "progress", 2024-04-20T11:32:22.048 INFO:teuthology.orchestra.run.smithi152.stdout: "rbd_support", 2024-04-20T11:32:22.048 INFO:teuthology.orchestra.run.smithi152.stdout: "status", 2024-04-20T11:32:22.048 INFO:teuthology.orchestra.run.smithi152.stdout: "telemetry", 2024-04-20T11:32:22.048 INFO:teuthology.orchestra.run.smithi152.stdout: "volumes" 2024-04-20T11:32:22.049 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-04-20T11:32:22.049 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:22.049 INFO:teuthology.orchestra.run.smithi152.stdout: "last_failure_osd_epoch": 0, 2024-04-20T11:32:22.049 INFO:teuthology.orchestra.run.smithi152.stdout: "active_clients": [ 2024-04-20T11:32:22.049 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:32:22.049 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "devicehealth", 2024-04-20T11:32:22.049 INFO:teuthology.orchestra.run.smithi152.stdout: "addrvec": [ 2024-04-20T11:32:22.049 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:32:22.049 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "v2", 2024-04-20T11:32:22.049 INFO:teuthology.orchestra.run.smithi152.stdout: "addr": "172.21.15.152:0", 2024-04-20T11:32:22.049 INFO:teuthology.orchestra.run.smithi152.stdout: "nonce": 2510881178 2024-04-20T11:32:22.049 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:22.049 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-04-20T11:32:22.049 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:22.049 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:32:22.050 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "libcephsqlite", 2024-04-20T11:32:22.050 INFO:teuthology.orchestra.run.smithi152.stdout: "addrvec": [ 2024-04-20T11:32:22.050 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:32:22.050 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "v2", 2024-04-20T11:32:22.050 INFO:teuthology.orchestra.run.smithi152.stdout: "addr": "172.21.15.152:0", 2024-04-20T11:32:22.050 INFO:teuthology.orchestra.run.smithi152.stdout: "nonce": 2828322453 2024-04-20T11:32:22.050 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:22.050 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-04-20T11:32:22.050 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:22.050 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:32:22.050 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "rbd_support", 2024-04-20T11:32:22.050 INFO:teuthology.orchestra.run.smithi152.stdout: "addrvec": [ 2024-04-20T11:32:22.050 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:32:22.051 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "v2", 2024-04-20T11:32:22.051 INFO:teuthology.orchestra.run.smithi152.stdout: "addr": "172.21.15.152:0", 2024-04-20T11:32:22.051 INFO:teuthology.orchestra.run.smithi152.stdout: "nonce": 698943652 2024-04-20T11:32:22.051 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:22.051 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-04-20T11:32:22.051 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:22.051 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:32:22.051 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "volumes", 2024-04-20T11:32:22.051 INFO:teuthology.orchestra.run.smithi152.stdout: "addrvec": [ 2024-04-20T11:32:22.051 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:32:22.051 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "v2", 2024-04-20T11:32:22.051 INFO:teuthology.orchestra.run.smithi152.stdout: "addr": "172.21.15.152:0", 2024-04-20T11:32:22.051 INFO:teuthology.orchestra.run.smithi152.stdout: "nonce": 2184739958 2024-04-20T11:32:22.051 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:22.051 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-04-20T11:32:22.052 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:22.052 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:32:22.052 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "volumes", 2024-04-20T11:32:22.052 INFO:teuthology.orchestra.run.smithi152.stdout: "addrvec": [ 2024-04-20T11:32:22.052 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:32:22.052 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "v2", 2024-04-20T11:32:22.052 INFO:teuthology.orchestra.run.smithi152.stdout: "addr": "172.21.15.152:0", 2024-04-20T11:32:22.052 INFO:teuthology.orchestra.run.smithi152.stdout: "nonce": 3263977099 2024-04-20T11:32:22.052 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:22.052 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-04-20T11:32:22.052 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:22.052 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:32:22.052 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "volumes", 2024-04-20T11:32:22.052 INFO:teuthology.orchestra.run.smithi152.stdout: "addrvec": [ 2024-04-20T11:32:22.053 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:32:22.053 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "v2", 2024-04-20T11:32:22.053 INFO:teuthology.orchestra.run.smithi152.stdout: "addr": "172.21.15.152:0", 2024-04-20T11:32:22.053 INFO:teuthology.orchestra.run.smithi152.stdout: "nonce": 3593059376 2024-04-20T11:32:22.053 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:22.053 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-04-20T11:32:22.053 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:22.053 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:32:22.053 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "volumes", 2024-04-20T11:32:22.053 INFO:teuthology.orchestra.run.smithi152.stdout: "addrvec": [ 2024-04-20T11:32:22.053 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:32:22.053 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "v2", 2024-04-20T11:32:22.053 INFO:teuthology.orchestra.run.smithi152.stdout: "addr": "172.21.15.152:0", 2024-04-20T11:32:22.053 INFO:teuthology.orchestra.run.smithi152.stdout: "nonce": 1524004838 2024-04-20T11:32:22.053 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:22.054 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-04-20T11:32:22.054 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:22.054 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:32:22.054 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "volumes", 2024-04-20T11:32:22.054 INFO:teuthology.orchestra.run.smithi152.stdout: "addrvec": [ 2024-04-20T11:32:22.054 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:32:22.054 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "v2", 2024-04-20T11:32:22.054 INFO:teuthology.orchestra.run.smithi152.stdout: "addr": "172.21.15.152:0", 2024-04-20T11:32:22.054 INFO:teuthology.orchestra.run.smithi152.stdout: "nonce": 2405473735 2024-04-20T11:32:22.054 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:22.054 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-04-20T11:32:22.054 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:22.054 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:32:22.054 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "volumes", 2024-04-20T11:32:22.055 INFO:teuthology.orchestra.run.smithi152.stdout: "addrvec": [ 2024-04-20T11:32:22.055 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:32:22.055 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "v2", 2024-04-20T11:32:22.055 INFO:teuthology.orchestra.run.smithi152.stdout: "addr": "172.21.15.152:0", 2024-04-20T11:32:22.055 INFO:teuthology.orchestra.run.smithi152.stdout: "nonce": 142101408 2024-04-20T11:32:22.055 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:22.055 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-04-20T11:32:22.055 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:22.055 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:32:22.055 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "volumes", 2024-04-20T11:32:22.055 INFO:teuthology.orchestra.run.smithi152.stdout: "addrvec": [ 2024-04-20T11:32:22.055 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:32:22.055 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "v2", 2024-04-20T11:32:22.055 INFO:teuthology.orchestra.run.smithi152.stdout: "addr": "172.21.15.152:0", 2024-04-20T11:32:22.056 INFO:teuthology.orchestra.run.smithi152.stdout: "nonce": 3331653721 2024-04-20T11:32:22.056 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:22.056 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-04-20T11:32:22.056 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:22.056 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-04-20T11:32:22.056 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-04-20T11:32:22.057 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:21.196+0000 7f6f74c00640 1 -- 172.21.15.152:0/134333627 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f6f4c06cc80 msgr2=0x7f6f4c06f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:22.057 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:21.197+0000 7f6f74c00640 1 --2- 172.21.15.152:0/134333627 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f6f4c06cc80 0x7f6f4c06f140 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f6f70144d60 tx=0x7f6f58008040 comp rx=0 tx=0).stop 2024-04-20T11:32:22.057 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:21.197+0000 7f6f74c00640 1 -- 172.21.15.152:0/134333627 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6f540051c0 msgr2=0x7f6f70159900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:22.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:21.197+0000 7f6f74c00640 1 --2- 172.21.15.152:0/134333627 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6f540051c0 0x7f6f70159900 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f6f6000d770 tx=0x7f6f6000dc30 comp rx=0 tx=0).stop 2024-04-20T11:32:22.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:21.197+0000 7f6f74c00640 1 -- 172.21.15.152:0/134333627 shutdown_connections 2024-04-20T11:32:22.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:21.197+0000 7f6f74c00640 1 --2- 172.21.15.152:0/134333627 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7f6f4c06cc80 0x7f6f4c06f140 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:22.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:21.197+0000 7f6f74c00640 1 --2- 172.21.15.152:0/134333627 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6f700ad0f0 0x7f6f700ad570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:22.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:21.197+0000 7f6f74c00640 1 --2- 172.21.15.152:0/134333627 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6f7015fcf0 0x7f6f70160130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:22.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:21.197+0000 7f6f74c00640 1 --2- 172.21.15.152:0/134333627 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6f540051c0 0x7f6f70159900 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:22.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:21.197+0000 7f6f74c00640 1 -- 172.21.15.152:0/134333627 >> 172.21.15.152:0/134333627 conn(0x7f6f70067cf0 msgr2=0x7f6f7015c170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:22.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:21.197+0000 7f6f74c00640 1 -- 172.21.15.152:0/134333627 shutdown_connections 2024-04-20T11:32:22.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:21.197+0000 7f6f74c00640 1 -- 172.21.15.152:0/134333627 wait complete. 2024-04-20T11:32:22.062 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr fail y 2024-04-20T11:32:22.265 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.264+0000 7fae16800640 1 Processor -- start 2024-04-20T11:32:22.265 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.265+0000 7fae16800640 1 -- start start 2024-04-20T11:32:22.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.266+0000 7fae16800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fae10162c40 0x7fae100cf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:22.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.266+0000 7fae16800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fae1006f6a0 0x7fae100ce140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:22.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.266+0000 7fae16800640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae10159420 con 0x7fae10159040 2024-04-20T11:32:22.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.266+0000 7fae16800640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fae100ce680 con 0x7fae1006f6a0 2024-04-20T11:32:22.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.266+0000 7fae16800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fae1006fad0 con 0x7fae10162c40 2024-04-20T11:32:22.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.267+0000 7fae15e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fae1006f6a0 0x7fae100ce140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:22.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.267+0000 7fae15e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fae1006f6a0 0x7fae100ce140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:58672/0 (socket says 172.21.15.152:58672) 2024-04-20T11:32:22.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.267+0000 7fae15e00640 1 -- 172.21.15.152:0/945117820 learned_addr learned my addr 172.21.15.152:0/945117820 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:32:22.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.267+0000 7fae15e00640 1 -- 172.21.15.152:0/945117820 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fae10162c40 msgr2=0x7fae100cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:22.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.267+0000 7fae14a00640 1 --2- 172.21.15.152:0/945117820 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fae10162c40 0x7fae100cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:22.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.267+0000 7fae15e00640 1 --2- 172.21.15.152:0/945117820 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fae10162c40 0x7fae100cf950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:22.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.267+0000 7fae15e00640 1 -- 172.21.15.152:0/945117820 >> v1:172.21.15.175:6789/0 conn(0x7fae10159040 legacy=0x7fae10162530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:22.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.267+0000 7fae15e00640 1 -- 172.21.15.152:0/945117820 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae100ce8b0 con 0x7fae1006f6a0 2024-04-20T11:32:22.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.268+0000 7fae15e00640 1 --2- 172.21.15.152:0/945117820 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fae1006f6a0 0x7fae100ce140 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fae08009e50 tx=0x7fae0802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5cb6e9fa66d66dd server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:22.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.269+0000 7fae0fe00640 1 -- 172.21.15.152:0/945117820 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae0802ca70 con 0x7fae1006f6a0 2024-04-20T11:32:22.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.269+0000 7fae0fe00640 1 -- 172.21.15.152:0/945117820 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae0802cc10 con 0x7fae1006f6a0 2024-04-20T11:32:22.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.269+0000 7fae0fe00640 1 -- 172.21.15.152:0/945117820 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae080343f0 con 0x7fae1006f6a0 2024-04-20T11:32:22.270 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.269+0000 7fae16800640 1 -- 172.21.15.152:0/945117820 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fae1006f6a0 msgr2=0x7fae100ce140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:22.270 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.269+0000 7fae16800640 1 --2- 172.21.15.152:0/945117820 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fae1006f6a0 0x7fae100ce140 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fae08009e50 tx=0x7fae0802bed0 comp rx=0 tx=0).stop 2024-04-20T11:32:22.270 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.269+0000 7fae16800640 1 -- 172.21.15.152:0/945117820 shutdown_connections 2024-04-20T11:32:22.271 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.269+0000 7fae16800640 1 --2- 172.21.15.152:0/945117820 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fae1006f6a0 0x7fae100ce140 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:22.271 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.269+0000 7fae16800640 1 --2- 172.21.15.152:0/945117820 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fae10162c40 0x7fae100cf950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:22.271 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.269+0000 7fae16800640 1 -- 172.21.15.152:0/945117820 >> 172.21.15.152:0/945117820 conn(0x7fae100d0260 msgr2=0x7fae100d0670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:22.271 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.270+0000 7fae16800640 1 -- 172.21.15.152:0/945117820 shutdown_connections 2024-04-20T11:32:22.271 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.270+0000 7fae16800640 1 -- 172.21.15.152:0/945117820 wait complete. 2024-04-20T11:32:22.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.271+0000 7fae16800640 1 Processor -- start 2024-04-20T11:32:22.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.271+0000 7fae16800640 1 -- start start 2024-04-20T11:32:22.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.272+0000 7fae16800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae1006f6a0 0x7fae100b43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:22.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.272+0000 7fae16800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fae10159040 0x7fae10156c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:22.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.272+0000 7fae16800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fae10162c40 0x7fae101465d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:22.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.272+0000 7fae16800640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fae100cf150 con 0x7fae10162c40 2024-04-20T11:32:22.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.272+0000 7fae16800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fae100cee70 con 0x7fae1006f6a0 2024-04-20T11:32:22.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.272+0000 7fae16800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fae100cf430 con 0x7fae10159040 2024-04-20T11:32:22.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.273+0000 7fae15e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fae10162c40 0x7fae101465d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:22.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.273+0000 7fae15e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fae10162c40 0x7fae101465d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:58684/0 (socket says 172.21.15.152:58684) 2024-04-20T11:32:22.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.273+0000 7fae15e00640 1 -- 172.21.15.152:0/3852174197 learned_addr learned my addr 172.21.15.152:0/3852174197 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:32:22.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.273+0000 7fae15400640 1 --2- 172.21.15.152:0/3852174197 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae1006f6a0 0x7fae100b43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:22.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.273+0000 7fae14a00640 1 --2- 172.21.15.152:0/3852174197 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fae10159040 0x7fae10156c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:22.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.273+0000 7fae15e00640 1 -- 172.21.15.152:0/3852174197 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fae10159040 msgr2=0x7fae10156c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:22.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.273+0000 7fae15e00640 1 --2- 172.21.15.152:0/3852174197 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fae10159040 0x7fae10156c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:22.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.273+0000 7fae15e00640 1 -- 172.21.15.152:0/3852174197 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae1006f6a0 msgr2=0x7fae100b43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:22.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.273+0000 7fae15e00640 1 --2- 172.21.15.152:0/3852174197 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae1006f6a0 0x7fae100b43d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:22.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.273+0000 7fae15e00640 1 -- 172.21.15.152:0/3852174197 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae101574c0 con 0x7fae10162c40 2024-04-20T11:32:22.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.273+0000 7fae15e00640 1 --2- 172.21.15.152:0/3852174197 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fae10162c40 0x7fae101465d0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fae08036040 tx=0x7fae08004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:22.276 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.274+0000 7fae0d600640 1 -- 172.21.15.152:0/3852174197 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae08043020 con 0x7fae10162c40 2024-04-20T11:32:22.276 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.274+0000 7fae0d600640 1 -- 172.21.15.152:0/3852174197 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae0803e450 con 0x7fae10162c40 2024-04-20T11:32:22.276 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.274+0000 7fae0d600640 1 -- 172.21.15.152:0/3852174197 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae0803e840 con 0x7fae10162c40 2024-04-20T11:32:22.276 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.274+0000 7fae16800640 1 -- 172.21.15.152:0/3852174197 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae101577a0 con 0x7fae10162c40 2024-04-20T11:32:22.276 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.274+0000 7fae16800640 1 -- 172.21.15.152:0/3852174197 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae10157d60 con 0x7fae10162c40 2024-04-20T11:32:22.276 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.275+0000 7fae16800640 1 -- 172.21.15.152:0/3852174197 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae10075e60 con 0x7fae10162c40 2024-04-20T11:32:22.276 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.276+0000 7fae0d600640 1 -- 172.21.15.152:0/3852174197 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 12) ==== 91397+0+0 (secure 0 0 0) 0x7fae08005d10 con 0x7fae10162c40 2024-04-20T11:32:22.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.277+0000 7fae0d600640 1 --2- 172.21.15.152:0/3852174197 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fadf006cd10 0x7fadf006f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:22.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.278+0000 7fae0d600640 1 -- 172.21.15.152:0/3852174197 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5709+0+0 (secure 0 0 0) 0x7fae080ae6a0 con 0x7fae10162c40 2024-04-20T11:32:22.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.278+0000 7fae15400640 1 --2- 172.21.15.152:0/3852174197 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fadf006cd10 0x7fadf006f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:22.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.278+0000 7fae15400640 1 --2- 172.21.15.152:0/3852174197 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fadf006cd10 0x7fadf006f1d0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fae04006fd0 tx=0x7fae04008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:22.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.282+0000 7fae0d600640 1 -- 172.21.15.152:0/3852174197 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fae0804d020 con 0x7fae10162c40 2024-04-20T11:32:22.447 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:32:22.455 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:32:22.540 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.540+0000 7fae16800640 1 -- 172.21.15.152:0/3852174197 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "mgr fail", "who": "y"} v 0) -- 0x7fae100cd990 con 0x7fae10162c40 2024-04-20T11:32:22.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.545+0000 7fae0d600640 1 -- 172.21.15.152:0/3852174197 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "mgr fail", "who": "y"}]=0 v13) ==== 68+0+0 (secure 0 0 0) 0x7fae10159420 con 0x7fae10162c40 2024-04-20T11:32:22.550 INFO:tasks.ceph.mgr.y.smithi152.stderr:2024-04-20T11:32:22.548+0000 7fd1e9000640 -1 mgr handle_mgr_map I was active but no longer am 2024-04-20T11:32:22.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.550+0000 7fae16800640 1 -- 172.21.15.152:0/3852174197 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fadf006cd10 msgr2=0x7fadf006f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:22.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.550+0000 7fae16800640 1 --2- 172.21.15.152:0/3852174197 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fadf006cd10 0x7fadf006f1d0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fae04006fd0 tx=0x7fae04008040 comp rx=0 tx=0).stop 2024-04-20T11:32:22.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.550+0000 7fae16800640 1 -- 172.21.15.152:0/3852174197 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fae10162c40 msgr2=0x7fae101465d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:22.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.550+0000 7fae16800640 1 --2- 172.21.15.152:0/3852174197 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fae10162c40 0x7fae101465d0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fae08036040 tx=0x7fae08004a40 comp rx=0 tx=0).stop 2024-04-20T11:32:22.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.550+0000 7fae16800640 1 -- 172.21.15.152:0/3852174197 shutdown_connections 2024-04-20T11:32:22.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.550+0000 7fae16800640 1 --2- 172.21.15.152:0/3852174197 >> [v2:172.21.15.152:6832/2285110240,v1:172.21.15.152:6833/2285110240] conn(0x7fadf006cd10 0x7fadf006f1d0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:22.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.550+0000 7fae16800640 1 --2- 172.21.15.152:0/3852174197 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fae10162c40 0x7fae101465d0 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:22.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.550+0000 7fae16800640 1 --2- 172.21.15.152:0/3852174197 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fae10159040 0x7fae10156c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:22.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.550+0000 7fae16800640 1 --2- 172.21.15.152:0/3852174197 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae1006f6a0 0x7fae100b43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:22.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.550+0000 7fae16800640 1 -- 172.21.15.152:0/3852174197 >> 172.21.15.152:0/3852174197 conn(0x7fae100d0260 msgr2=0x7fae100ad2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:22.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.552+0000 7fae16800640 1 -- 172.21.15.152:0/3852174197 shutdown_connections 2024-04-20T11:32:22.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.552+0000 7fae16800640 1 -- 172.21.15.152:0/3852174197 wait complete. 2024-04-20T11:32:22.583 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:32:22.679 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:32:22.679+0000 7f8a64400640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:32:22.679 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:32:22.679+0000 7f8a64400640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:32:22.680 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:32:22.679+0000 7f8a64400640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:32:22.680 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:32:22.679+0000 7f8a64400640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:32:22.680 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:32:22.679+0000 7f8a64400640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:32:22.680 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:32:22.680+0000 7f8a60800640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:32:22.681 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:32:22.681+0000 7f8a60800640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:32:22.681 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:32:22.681+0000 7f8a60800640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:32:22.681 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:32:22.681+0000 7f8a60800640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:32:22.681 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:32:22.681+0000 7f8a60800640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:32:22.773 INFO:tasks.ceph.mgr.y.smithi152.stderr:2024-04-20T11:32:22.773+0000 7fe15ce4b040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-20T11:32:22.794 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.794+0000 7ff2fb200640 1 Processor -- start 2024-04-20T11:32:22.794 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.794+0000 7ff2fb200640 1 -- start start 2024-04-20T11:32:22.795 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.795+0000 7ff2fb200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ff2fc15fcc0 0x7ff2fc0d4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:22.795 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.795+0000 7ff2fb200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff2fc1603d0 0x7ff2fc0b5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:22.795 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.795+0000 7ff2fb200640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2fc0d3350 con 0x7ff2fc0b5480 2024-04-20T11:32:22.795 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.795+0000 7ff2fb200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7ff2fc0d3630 con 0x7ff2fc15fcc0 2024-04-20T11:32:22.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.795+0000 7ff2fb200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff2fc0d3910 con 0x7ff2fc1603d0 2024-04-20T11:32:22.798 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.795+0000 7ff2fa800640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7ff2fc0b5480 0x7ff2fc1624d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:50312/0 (socket says 172.21.15.152:50312) 2024-04-20T11:32:22.798 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.795+0000 7ff2fa800640 1 -- 172.21.15.152:0/1439206964 learned_addr learned my addr 172.21.15.152:0/1439206964 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:32:22.798 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.796+0000 7ff2f3e00640 1 --2- 172.21.15.152:0/1439206964 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff2fc1603d0 0x7ff2fc0b5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:22.798 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.796+0000 7ff2f9e00640 1 --2- 172.21.15.152:0/1439206964 >> v2:172.21.15.175:3300/0 conn(0x7ff2fc15fcc0 0x7ff2fc0d4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:22.798 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.796+0000 7ff2f9400640 1 -- 172.21.15.152:0/1439206964 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 34887124 0 0) 0x7ff2fc0d3350 con 0x7ff2fc0b5480 2024-04-20T11:32:22.798 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.796+0000 7ff2f9400640 1 -- 172.21.15.152:0/1439206964 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff2e0003520 con 0x7ff2fc0b5480 2024-04-20T11:32:22.798 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.796+0000 7ff2f9400640 1 -- 172.21.15.152:0/1439206964 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1720119905 0 0) 0x7ff2e0003520 con 0x7ff2fc0b5480 2024-04-20T11:32:22.798 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.796+0000 7ff2f9400640 1 -- 172.21.15.152:0/1439206964 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff2fc1603d0 msgr2=0x7ff2fc0b5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:22.798 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.796+0000 7ff2f9400640 1 --2- 172.21.15.152:0/1439206964 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff2fc1603d0 0x7ff2fc0b5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:22.798 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.796+0000 7ff2f9400640 1 -- 172.21.15.152:0/1439206964 >> v2:172.21.15.175:3300/0 conn(0x7ff2fc15fcc0 msgr2=0x7ff2fc0d4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:22.798 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.796+0000 7ff2f9400640 1 --2- 172.21.15.152:0/1439206964 >> v2:172.21.15.175:3300/0 conn(0x7ff2fc15fcc0 0x7ff2fc0d4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:22.798 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.796+0000 7ff2f9400640 1 -- 172.21.15.152:0/1439206964 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2fc0ced20 con 0x7ff2fc0b5480 2024-04-20T11:32:22.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.796+0000 7ff2f9400640 1 -- 172.21.15.152:0/1439206964 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7ff2f40019f0 con 0x7ff2fc0b5480 2024-04-20T11:32:22.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.796+0000 7ff2f9400640 1 -- 172.21.15.152:0/1439206964 >> v1:172.21.15.152:6789/0 conn(0x7ff2fc0b5480 legacy=0x7ff2fc1624d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:22.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.796+0000 7ff2f9400640 1 --2- 172.21.15.152:0/1439206964 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff2fc1603d0 0x7ff2e00039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:22.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.796+0000 7ff2f9400640 1 --2- 172.21.15.152:0/1439206964 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff2e0003f30 0x7ff2e0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:22.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.796+0000 7ff2f9400640 1 --2- 172.21.15.152:0/1439206964 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff2e0004d40 0x7ff2e00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:22.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.797+0000 7ff2f9400640 1 -- 172.21.15.152:0/1439206964 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7ff2fc0d3630 con 0x7ff2e0004d40 2024-04-20T11:32:22.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.797+0000 7ff2f9400640 1 -- 172.21.15.152:0/1439206964 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff2fc0d3910 con 0x7ff2e0003f30 2024-04-20T11:32:22.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.797+0000 7ff2f9400640 1 -- 172.21.15.152:0/1439206964 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff2e000b9c0 con 0x7ff2fc1603d0 2024-04-20T11:32:22.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.797+0000 7ff2fa800640 1 --2- 172.21.15.152:0/1439206964 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff2e0004d40 0x7ff2e00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:22.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.797+0000 7ff2f9e00640 1 --2- 172.21.15.152:0/1439206964 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff2fc1603d0 0x7ff2e00039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:22.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.797+0000 7ff2fa800640 1 -- 172.21.15.152:0/1439206964 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff2fc1603d0 msgr2=0x7ff2e00039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:22.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.797+0000 7ff2fa800640 1 --2- 172.21.15.152:0/1439206964 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff2fc1603d0 0x7ff2e00039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:22.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.797+0000 7ff2fa800640 1 -- 172.21.15.152:0/1439206964 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff2e0003f30 msgr2=0x7ff2e0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:22.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.797+0000 7ff2f3e00640 1 --2- 172.21.15.152:0/1439206964 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff2e0003f30 0x7ff2e0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:22.800 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.797+0000 7ff2fa800640 1 --2- 172.21.15.152:0/1439206964 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff2e0003f30 0x7ff2e0004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:22.800 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.797+0000 7ff2fa800640 1 -- 172.21.15.152:0/1439206964 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2e0003520 con 0x7ff2e0004d40 2024-04-20T11:32:22.801 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.798+0000 7ff2fa800640 1 --2- 172.21.15.152:0/1439206964 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff2e0004d40 0x7ff2e00051c0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7ff2f400a870 tx=0x7ff2e0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e712d2a46d3ed036 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:22.801 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.799+0000 7ff2f9400640 1 -- 172.21.15.152:0/1439206964 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f400a4c0 con 0x7ff2e0004d40 2024-04-20T11:32:22.801 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.799+0000 7ff2f9400640 1 -- 172.21.15.152:0/1439206964 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2f4034630 con 0x7ff2e0004d40 2024-04-20T11:32:22.801 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.799+0000 7ff2f9400640 1 -- 172.21.15.152:0/1439206964 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f4034a20 con 0x7ff2e0004d40 2024-04-20T11:32:22.801 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.799+0000 7ff2fb200640 1 -- 172.21.15.152:0/1439206964 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff2e0004d40 msgr2=0x7ff2e00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:22.801 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.799+0000 7ff2fb200640 1 --2- 172.21.15.152:0/1439206964 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff2e0004d40 0x7ff2e00051c0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7ff2f400a870 tx=0x7ff2e0005830 comp rx=0 tx=0).stop 2024-04-20T11:32:22.801 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.799+0000 7ff2fa800640 1 -- 172.21.15.152:0/1439206964 reap_dead start 2024-04-20T11:32:22.801 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.799+0000 7ff2fb200640 1 -- 172.21.15.152:0/1439206964 shutdown_connections 2024-04-20T11:32:22.801 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.799+0000 7ff2fb200640 1 -- 172.21.15.152:0/1439206964 >> 172.21.15.152:0/1439206964 conn(0x7ff2fc08bfd0 msgr2=0x7ff2fc0d4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:22.801 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.799+0000 7ff2fb200640 1 -- 172.21.15.152:0/1439206964 shutdown_connections 2024-04-20T11:32:22.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.799+0000 7ff2fb200640 1 -- 172.21.15.152:0/1439206964 wait complete. 2024-04-20T11:32:22.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.801+0000 7ff2fb200640 1 Processor -- start 2024-04-20T11:32:22.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.801+0000 7ff2fb200640 1 -- start start 2024-04-20T11:32:22.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.802+0000 7ff2fb200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff2e0004d40 0x7ff2fc0b90b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:22.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.802+0000 7ff2fb200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff2fc0ca390 0x7ff2fc1589a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:22.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.802+0000 7ff2fb200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff2fc0c90a0 0x7ff2fc0c9520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:22.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.802+0000 7ff2fb200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7ff2fc162d60 con 0x7ff2fc0ca390 2024-04-20T11:32:22.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.802+0000 7ff2fb200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff2fc162be0 con 0x7ff2e0004d40 2024-04-20T11:32:22.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.802+0000 7ff2fb200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff2fc162ee0 con 0x7ff2fc0c90a0 2024-04-20T11:32:22.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.803+0000 7ff2f3e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff2fc0ca390 0x7ff2fc1589a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:22.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.803+0000 7ff2f3e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff2fc0ca390 0x7ff2fc1589a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:58724/0 (socket says 172.21.15.152:58724) 2024-04-20T11:32:22.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.803+0000 7ff2f3e00640 1 -- 172.21.15.152:0/2434090905 learned_addr learned my addr 172.21.15.152:0/2434090905 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:32:22.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.803+0000 7ff2fa800640 1 --2- 172.21.15.152:0/2434090905 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff2fc0c90a0 0x7ff2fc0c9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:22.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.803+0000 7ff2f9e00640 1 --2- 172.21.15.152:0/2434090905 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff2e0004d40 0x7ff2fc0b90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:22.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.803+0000 7ff2f3e00640 1 -- 172.21.15.152:0/2434090905 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff2fc0c90a0 msgr2=0x7ff2fc0c9520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:22.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.803+0000 7ff2f3e00640 1 --2- 172.21.15.152:0/2434090905 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff2fc0c90a0 0x7ff2fc0c9520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:22.805 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.803+0000 7ff2f3e00640 1 -- 172.21.15.152:0/2434090905 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff2e0004d40 msgr2=0x7ff2fc0b90b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:22.805 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.803+0000 7ff2f3e00640 1 --2- 172.21.15.152:0/2434090905 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff2e0004d40 0x7ff2fc0b90b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:22.805 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.803+0000 7ff2f3e00640 1 -- 172.21.15.152:0/2434090905 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2fc0ca810 con 0x7ff2fc0ca390 2024-04-20T11:32:22.805 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.803+0000 7ff2f9e00640 1 --2- 172.21.15.152:0/2434090905 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff2e0004d40 0x7ff2fc0b90b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:32:22.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.804+0000 7ff2f3e00640 1 --2- 172.21.15.152:0/2434090905 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff2fc0ca390 0x7ff2fc1589a0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7ff2e4010890 tx=0x7ff2e4010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:22.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.804+0000 7ff2f2000640 1 -- 172.21.15.152:0/2434090905 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e400f040 con 0x7ff2fc0ca390 2024-04-20T11:32:22.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.804+0000 7ff2f2000640 1 -- 172.21.15.152:0/2434090905 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2e4005b90 con 0x7ff2fc0ca390 2024-04-20T11:32:22.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.804+0000 7ff2f2000640 1 -- 172.21.15.152:0/2434090905 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e400e960 con 0x7ff2fc0ca390 2024-04-20T11:32:22.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.804+0000 7ff2fb200640 1 -- 172.21.15.152:0/2434090905 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2fc0b9770 con 0x7ff2fc0ca390 2024-04-20T11:32:22.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.804+0000 7ff2fb200640 1 -- 172.21.15.152:0/2434090905 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2fc0b9d30 con 0x7ff2fc0ca390 2024-04-20T11:32:22.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.805+0000 7ff2fb200640 1 -- 172.21.15.152:0/2434090905 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2fc075e60 con 0x7ff2fc0ca390 2024-04-20T11:32:22.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.806+0000 7ff2f2000640 1 -- 172.21.15.152:0/2434090905 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 13) ==== 45545+0+0 (secure 0 0 0) 0x7ff2e400eb00 con 0x7ff2fc0ca390 2024-04-20T11:32:22.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.806+0000 7ff2f2000640 1 -- 172.21.15.152:0/2434090905 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 6268+0+0 (secure 0 0 0) 0x7ff2e404bf80 con 0x7ff2fc0ca390 2024-04-20T11:32:22.812 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:22.811+0000 7ff2f2000640 1 -- 172.21.15.152:0/2434090905 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ff2fc0ca810 con 0x7ff2fc0ca390 2024-04-20T11:32:22.913 INFO:tasks.ceph.mgr.y.smithi152.stderr:2024-04-20T11:32:22.910+0000 7fe15ce4b040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-20T11:32:22.966 INFO:tasks.ceph.mgr.y.smithi152.stderr:2024-04-20T11:32:22.965+0000 7fe15ce4b040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-20T11:32:23.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:23.045+0000 7ff2fb200640 1 -- 172.21.15.152:0/2434090905 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff2fc0b6ce0 con 0x7ff2fc0ca390 2024-04-20T11:32:23.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:23.046+0000 7ff2f2000640 1 -- 172.21.15.152:0/2434090905 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+5210 (secure 0 0 0) 0x7ff2e4018030 con 0x7ff2fc0ca390 2024-04-20T11:32:23.047 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:32:23.047 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5234,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6836/359391273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6835","nonce":359391273},{"type":"v1","addr":"172.21.15.152:6836","nonce":359391273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":5237,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/4269836436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":4269836436},{"type":"v1","addr":"172.21.15.152:6837","nonce":4269836436}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":5611,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6834/4153485754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":4153485754},{"type":"v1","addr":"172.21.15.175:6834","nonce":4153485754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T11:31:49.738765+0000","modified":"2024-04-20T11:31:52.857955+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5614},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5614":{"gid":5614,"name":"b","rank":0,"incarnation":39,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6835/2057501014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6833","nonce":2057501014},{"type":"v1","addr":"172.21.15.175:6835","nonce":2057501014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5614,"qdb_cluster":[5614]},"id":5}]} 2024-04-20T11:32:23.047 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 44 2024-04-20T11:32:23.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:23.049+0000 7ff2fb200640 1 -- 172.21.15.152:0/2434090905 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff2fc0ca390 msgr2=0x7ff2fc1589a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:23.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:23.049+0000 7ff2fb200640 1 --2- 172.21.15.152:0/2434090905 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff2fc0ca390 0x7ff2fc1589a0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7ff2e4010890 tx=0x7ff2e4010d50 comp rx=0 tx=0).stop 2024-04-20T11:32:23.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:23.049+0000 7ff2fb200640 1 -- 172.21.15.152:0/2434090905 shutdown_connections 2024-04-20T11:32:23.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:23.049+0000 7ff2fb200640 1 --2- 172.21.15.152:0/2434090905 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff2fc0c90a0 0x7ff2fc0c9520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:23.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:23.049+0000 7ff2fb200640 1 --2- 172.21.15.152:0/2434090905 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff2fc0ca390 0x7ff2fc1589a0 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:23.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:23.049+0000 7ff2fb200640 1 --2- 172.21.15.152:0/2434090905 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff2e0004d40 0x7ff2fc0b90b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:23.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:23.049+0000 7ff2fb200640 1 -- 172.21.15.152:0/2434090905 >> 172.21.15.152:0/2434090905 conn(0x7ff2fc08bfd0 msgr2=0x7ff2fc0b5d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:23.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:23.049+0000 7ff2fb200640 1 -- 172.21.15.152:0/2434090905 shutdown_connections 2024-04-20T11:32:23.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:23.049+0000 7ff2fb200640 1 -- 172.21.15.152:0/2434090905 wait complete. 2024-04-20T11:32:23.067 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json session ls 2024-04-20T11:32:23.208 INFO:tasks.ceph.mgr.y.smithi152.stderr:2024-04-20T11:32:23.207+0000 7fe15ce4b040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-20T11:32:23.258 INFO:tasks.ceph.mgr.y.smithi152.stderr:2024-04-20T11:32:23.257+0000 7fe15ce4b040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-20T11:32:23.300 INFO:teuthology.orchestra.run.smithi175.stdout:[{"id":5387,"entity":{"name":{"type":"client","num":5387},"addr":{"type":"any","addr":"172.21.15.175:0","nonce":3369420122}},"state":"open","num_leases":0,"num_caps":2,"request_load_avg":2,"uptime":0.59899066300000003,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":1.9972639844689619,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":2,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.5387 172.21.15.175:0/3369420122","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"a8c84f5997950fd7d008204839253291d52e6eda","ceph_version":"ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)","entity_id":"x","hostname":"smithi175","pid":"22088","root":"/"}},{"id":5797,"entity":{"name":{"type":"client","num":5797},"addr":{"type":"any","addr":"172.21.15.175:0","nonce":1731099426}},"state":"open","num_leases":0,"num_caps":2,"request_load_avg":2,"uptime":0.60199061700000001,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":1.9972547553117019,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":2,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.5797 172.21.15.175:0/1731099426","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"a8c84f5997950fd7d008204839253291d52e6eda","ceph_version":"ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)","entity_id":"x","hostname":"smithi175","pid":"22088","root":"/"}}] 2024-04-20T11:32:23.301 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "a8c84f5997950fd7d008204839253291d52e6eda", "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi175", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "22088", "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "172.21.15.175:0", "nonce": 3369420122, "type": "any" }, "name": { "num": 5387, "type": "client" } }, "id": 5387, "inst": "client.5387 172.21.15.175:0/3369420122", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 2, "num_caps": 2, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 2, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 1.997263984468962 }, "state": "open", "uptime": 0.598990663 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "a8c84f5997950fd7d008204839253291d52e6eda", "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi175", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "22088", "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "172.21.15.175:0", "nonce": 1731099426, "type": "any" }, "name": { "num": 5797, "type": "client" } }, "id": 5797, "inst": "client.5797 172.21.15.175:0/1731099426", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 2, "num_caps": 2, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 2, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 1.997254755311702 }, "state": "open", "uptime": 0.601990617 } ] 2024-04-20T11:32:23.301 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-20T11:32:23.301 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-20T11:32:23.301 DEBUG:teuthology.orchestra.run.smithi152:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-20T11:32:23.410 DEBUG:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-20T11:32:23.516 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-20T11:32:23.516 INFO:tasks.cephfs.mount:Terminating background process 2024-04-20T11:32:23.516 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-20T11:32:23.517 DEBUG:teuthology.orchestra.run.smithi152:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:32:23.569 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-20T11:32:23.569 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi152.front.sepia.ceph.com 2024-04-20T11:32:23.569 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:32:23.569 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-20T11:32:23.612 INFO:teuthology.orchestra.run.smithi152.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-20T11:32:23.613 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T11:32:23.613 INFO:tasks.cephfs.mount:Terminating background process 2024-04-20T11:32:23.614 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T11:32:23.614 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:32:23.669 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-20T11:32:23.669 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi175.front.sepia.ceph.com 2024-04-20T11:32:23.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:32:23.669 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-20T11:32:23.717 INFO:teuthology.orchestra.run.smithi175.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-04-20T11:32:23.718 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T11:32:23.718 INFO:tasks.cephfs.mount:Terminating background process 2024-04-20T11:32:23.718 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-20T11:32:23.718 DEBUG:teuthology.orchestra.run.smithi152:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:32:23.762 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-20T11:32:23.762 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi152.front.sepia.ceph.com 2024-04-20T11:32:23.762 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:32:23.762 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-20T11:32:23.848 INFO:teuthology.orchestra.run.smithi152.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-20T11:32:23.848 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T11:32:23.849 INFO:tasks.cephfs.mount:Terminating background process 2024-04-20T11:32:23.849 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T11:32:23.849 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:32:23.904 INFO:tasks.ceph.mgr.y.smithi152.stderr:2024-04-20T11:32:23.904+0000 7fe15ce4b040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-20T11:32:23.907 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-20T11:32:23.908 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi175.front.sepia.ceph.com 2024-04-20T11:32:23.908 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:32:23.908 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-20T11:32:23.958 INFO:teuthology.orchestra.run.smithi175.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-20T11:32:23.959 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T11:32:23.959 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:32:24.007 INFO:tasks.ceph.mgr.y.smithi152.stderr:2024-04-20T11:32:24.007+0000 7fe15ce4b040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-20T11:32:24.060 INFO:tasks.ceph.mgr.y.smithi152.stderr:2024-04-20T11:32:24.060+0000 7fe15ce4b040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-20T11:32:24.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.152+0000 7f84dbe00640 1 Processor -- start 2024-04-20T11:32:24.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.152+0000 7f84dbe00640 1 -- start start 2024-04-20T11:32:24.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.154+0000 7f84dbe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f84dc15fcc0 0x7f84dc0d4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:24.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.154+0000 7f84dbe00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f84dc1603d0 0x7f84dc0b5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:24.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.154+0000 7f84dbe00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84dc0d3350 con 0x7f84dc0b5480 2024-04-20T11:32:24.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.154+0000 7f84dbe00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f84dc0d3630 con 0x7f84dc1603d0 2024-04-20T11:32:24.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.154+0000 7f84dbe00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f84dc0d3910 con 0x7f84dc15fcc0 2024-04-20T11:32:24.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.154+0000 7f84d2000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f84dc1603d0 0x7f84dc0b5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:24.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.154+0000 7f84d2000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f84dc1603d0 0x7f84dc0b5eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:58730/0 (socket says 172.21.15.152:58730) 2024-04-20T11:32:24.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.154+0000 7f84d2000640 1 -- 172.21.15.152:0/2104022847 learned_addr learned my addr 172.21.15.152:0/2104022847 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:32:24.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.155+0000 7f84daa00640 1 --2- 172.21.15.152:0/2104022847 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f84dc15fcc0 0x7f84dc0d4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:24.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.155+0000 7f84d2000640 1 -- 172.21.15.152:0/2104022847 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f84dc15fcc0 msgr2=0x7f84dc0d4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:24.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.155+0000 7f84d2000640 1 --2- 172.21.15.152:0/2104022847 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f84dc15fcc0 0x7f84dc0d4e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:24.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.155+0000 7f84d2000640 1 -- 172.21.15.152:0/2104022847 >> v1:172.21.15.175:6789/0 conn(0x7f84dc0b5480 legacy=0x7f84dc1624d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:24.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.155+0000 7f84d2000640 1 -- 172.21.15.152:0/2104022847 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84dc0ced20 con 0x7f84dc1603d0 2024-04-20T11:32:24.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.156+0000 7f84d2000640 1 --2- 172.21.15.152:0/2104022847 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f84dc1603d0 0x7f84dc0b5eb0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f84c4009850 tx=0x7f84c402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f784db4808a10ad7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:24.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.156+0000 7f84da000640 1 -- 172.21.15.152:0/2104022847 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84c4004230 con 0x7f84dc1603d0 2024-04-20T11:32:24.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.156+0000 7f84da000640 1 -- 172.21.15.152:0/2104022847 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84c40043d0 con 0x7f84dc1603d0 2024-04-20T11:32:24.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.156+0000 7f84da000640 1 -- 172.21.15.152:0/2104022847 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84c4031830 con 0x7f84dc1603d0 2024-04-20T11:32:24.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.157+0000 7f84dbe00640 1 -- 172.21.15.152:0/2104022847 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f84dc1603d0 msgr2=0x7f84dc0b5eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:24.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.157+0000 7f84dbe00640 1 --2- 172.21.15.152:0/2104022847 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f84dc1603d0 0x7f84dc0b5eb0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f84c4009850 tx=0x7f84c402b8d0 comp rx=0 tx=0).stop 2024-04-20T11:32:24.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.157+0000 7f84dbe00640 1 -- 172.21.15.152:0/2104022847 shutdown_connections 2024-04-20T11:32:24.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.157+0000 7f84dbe00640 1 --2- 172.21.15.152:0/2104022847 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f84dc1603d0 0x7f84dc0b5eb0 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:24.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.157+0000 7f84dbe00640 1 --2- 172.21.15.152:0/2104022847 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f84dc15fcc0 0x7f84dc0d4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:24.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.157+0000 7f84dbe00640 1 -- 172.21.15.152:0/2104022847 >> 172.21.15.152:0/2104022847 conn(0x7f84dc08bfd0 msgr2=0x7f84dc0d4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:24.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.157+0000 7f84dbe00640 1 -- 172.21.15.152:0/2104022847 shutdown_connections 2024-04-20T11:32:24.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.157+0000 7f84dbe00640 1 -- 172.21.15.152:0/2104022847 wait complete. 2024-04-20T11:32:24.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.160+0000 7f84dbe00640 1 Processor -- start 2024-04-20T11:32:24.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.160+0000 7f84dbe00640 1 -- start start 2024-04-20T11:32:24.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.161+0000 7f84dbe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f84dc0b5480 0x7f84dc0b7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:24.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.161+0000 7f84dbe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f84dc15fcc0 0x7f84dc1583b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:24.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.161+0000 7f84dbe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f84dc1603d0 0x7f84dc0d1400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:24.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.161+0000 7f84dbe00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f84dc0b6570 con 0x7f84dc1603d0 2024-04-20T11:32:24.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.161+0000 7f84dbe00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f84dc0b63f0 con 0x7f84dc15fcc0 2024-04-20T11:32:24.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.161+0000 7f84dbe00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f84dc0b66f0 con 0x7f84dc0b5480 2024-04-20T11:32:24.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.162+0000 7f84db400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f84dc1603d0 0x7f84dc0d1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:24.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.162+0000 7f84db400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f84dc1603d0 0x7f84dc0d1400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:58732/0 (socket says 172.21.15.152:58732) 2024-04-20T11:32:24.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.162+0000 7f84db400640 1 -- 172.21.15.152:0/1459245574 learned_addr learned my addr 172.21.15.152:0/1459245574 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:32:24.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.162+0000 7f84daa00640 1 --2- 172.21.15.152:0/1459245574 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f84dc0b5480 0x7f84dc0b7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:24.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.162+0000 7f84db400640 1 -- 172.21.15.152:0/1459245574 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f84dc0b5480 msgr2=0x7f84dc0b7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:24.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.162+0000 7f84db400640 1 --2- 172.21.15.152:0/1459245574 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f84dc0b5480 0x7f84dc0b7090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:24.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.162+0000 7f84d2000640 1 --2- 172.21.15.152:0/1459245574 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f84dc15fcc0 0x7f84dc1583b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:24.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.162+0000 7f84db400640 1 -- 172.21.15.152:0/1459245574 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f84dc15fcc0 msgr2=0x7f84dc1583b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:24.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.162+0000 7f84db400640 1 --2- 172.21.15.152:0/1459245574 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f84dc15fcc0 0x7f84dc1583b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:24.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.162+0000 7f84db400640 1 -- 172.21.15.152:0/1459245574 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84dc1e8400 con 0x7f84dc1603d0 2024-04-20T11:32:24.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.163+0000 7f84db400640 1 --2- 172.21.15.152:0/1459245574 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f84dc1603d0 0x7f84dc0d1400 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f84d400b410 tx=0x7f84d400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:24.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.163+0000 7f84d3e00640 1 -- 172.21.15.152:0/1459245574 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d4015030 con 0x7f84dc1603d0 2024-04-20T11:32:24.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.163+0000 7f84d3e00640 1 -- 172.21.15.152:0/1459245574 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84d4004020 con 0x7f84dc1603d0 2024-04-20T11:32:24.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.163+0000 7f84d3e00640 1 -- 172.21.15.152:0/1459245574 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d4004410 con 0x7f84dc1603d0 2024-04-20T11:32:24.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.163+0000 7f84dbe00640 1 -- 172.21.15.152:0/1459245574 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84dc1e86e0 con 0x7f84dc1603d0 2024-04-20T11:32:24.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.163+0000 7f84dbe00640 1 -- 172.21.15.152:0/1459245574 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84dc1e8ca0 con 0x7f84dc1603d0 2024-04-20T11:32:24.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.164+0000 7f84d3e00640 1 -- 172.21.15.152:0/1459245574 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 14) ==== 45880+0+0 (secure 0 0 0) 0x7f84d4018070 con 0x7f84dc1603d0 2024-04-20T11:32:24.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.164+0000 7f84dbe00640 1 -- 172.21.15.152:0/1459245574 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84dc075e60 con 0x7f84dc1603d0 2024-04-20T11:32:24.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.165+0000 7f84d3e00640 1 --2- 172.21.15.152:0/1459245574 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f84b8038760 0x7f84b803ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:24.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.165+0000 7f84d3e00640 1 -- 172.21.15.152:0/1459245574 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 6268+0+0 (secure 0 0 0) 0x7f84d404bd80 con 0x7f84dc1603d0 2024-04-20T11:32:24.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.165+0000 7f84daa00640 1 --2- 172.21.15.152:0/1459245574 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f84b8038760 0x7f84b803ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:24.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.166+0000 7f84daa00640 1 --2- 172.21.15.152:0/1459245574 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f84b8038760 0x7f84b803ac20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f84cc008ef0 tx=0x7f84cc008440 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:24.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.170+0000 7f84d3e00640 1 -- 172.21.15.152:0/1459245574 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f84d40197b0 con 0x7f84dc1603d0 2024-04-20T11:32:24.261 INFO:tasks.ceph.mgr.y.smithi152.stderr:2024-04-20T11:32:24.260+0000 7fe15ce4b040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-20T11:32:24.315 INFO:tasks.ceph.mgr.y.smithi152.stderr:2024-04-20T11:32:24.315+0000 7fe15ce4b040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-20T11:32:24.374 INFO:tasks.ceph.mgr.y.smithi152.stderr:2024-04-20T11:32:24.374+0000 7fe15ce4b040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-20T11:32:24.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.415+0000 7f84dbe00640 1 -- 172.21.15.152:0/1459245574 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f84dc0b6ce0 con 0x7f84dc1603d0 2024-04-20T11:32:24.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.416+0000 7f84d3e00640 1 -- 172.21.15.152:0/1459245574 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+5210 (secure 0 0 0) 0x7f84d4019a10 con 0x7f84dc1603d0 2024-04-20T11:32:24.417 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:32:24.417 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5234,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6836/359391273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6835","nonce":359391273},{"type":"v1","addr":"172.21.15.152:6836","nonce":359391273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":5237,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/4269836436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":4269836436},{"type":"v1","addr":"172.21.15.152:6837","nonce":4269836436}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":5611,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6834/4153485754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":4153485754},{"type":"v1","addr":"172.21.15.175:6834","nonce":4153485754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T11:31:49.738765+0000","modified":"2024-04-20T11:31:52.857955+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5614},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5614":{"gid":5614,"name":"b","rank":0,"incarnation":39,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6835/2057501014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6833","nonce":2057501014},{"type":"v1","addr":"172.21.15.175:6835","nonce":2057501014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5614,"qdb_cluster":[5614]},"id":5}]} 2024-04-20T11:32:24.417 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 44 2024-04-20T11:32:24.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.420+0000 7f84dbe00640 1 -- 172.21.15.152:0/1459245574 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f84b8038760 msgr2=0x7f84b803ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:24.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.420+0000 7f84dbe00640 1 --2- 172.21.15.152:0/1459245574 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f84b8038760 0x7f84b803ac20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f84cc008ef0 tx=0x7f84cc008440 comp rx=0 tx=0).stop 2024-04-20T11:32:24.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.420+0000 7f84dbe00640 1 -- 172.21.15.152:0/1459245574 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f84dc1603d0 msgr2=0x7f84dc0d1400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:24.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.420+0000 7f84dbe00640 1 --2- 172.21.15.152:0/1459245574 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f84dc1603d0 0x7f84dc0d1400 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f84d400b410 tx=0x7f84d400b8d0 comp rx=0 tx=0).stop 2024-04-20T11:32:24.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.420+0000 7f84dbe00640 1 -- 172.21.15.152:0/1459245574 shutdown_connections 2024-04-20T11:32:24.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.420+0000 7f84dbe00640 1 --2- 172.21.15.152:0/1459245574 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f84b8038760 0x7f84b803ac20 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:24.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.420+0000 7f84dbe00640 1 --2- 172.21.15.152:0/1459245574 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f84dc1603d0 0x7f84dc0d1400 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:24.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.420+0000 7f84dbe00640 1 --2- 172.21.15.152:0/1459245574 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f84dc15fcc0 0x7f84dc1583b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:24.422 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.420+0000 7f84dbe00640 1 --2- 172.21.15.152:0/1459245574 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f84dc0b5480 0x7f84dc0b7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:24.422 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.420+0000 7f84dbe00640 1 -- 172.21.15.152:0/1459245574 >> 172.21.15.152:0/1459245574 conn(0x7f84dc08bfd0 msgr2=0x7f84dc06cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:24.422 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.420+0000 7f84dbe00640 1 -- 172.21.15.152:0/1459245574 shutdown_connections 2024-04-20T11:32:24.422 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.420+0000 7f84dbe00640 1 -- 172.21.15.152:0/1459245574 wait complete. 2024-04-20T11:32:24.438 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:32:24.507 INFO:tasks.ceph.mgr.y.smithi152.stderr:2024-04-20T11:32:24.507+0000 7fe15ce4b040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-20T11:32:24.674 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.673+0000 7fb920e00640 1 Processor -- start 2024-04-20T11:32:24.674 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.673+0000 7fb920e00640 1 -- start start 2024-04-20T11:32:24.675 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.675+0000 7fb920e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb91c15eb80 0x7fb91c0b79a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:24.675 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.675+0000 7fb920e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fb91c0d1810 0x7fb91c0d1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:24.675 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.675+0000 7fb920e00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb91c0b8330 con 0x7fb91c0ba520 2024-04-20T11:32:24.675 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.675+0000 7fb920e00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fb91c0b8610 con 0x7fb91c0d1810 2024-04-20T11:32:24.675 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.675+0000 7fb920e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb91c0b88f0 con 0x7fb91c15eb80 2024-04-20T11:32:24.676 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.675+0000 7fb919e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb91c15eb80 0x7fb91c0b79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:24.676 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.675+0000 7fb919e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb91c15eb80 0x7fb91c0b79a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.152:43628/0 (socket says 172.21.15.152:43628) 2024-04-20T11:32:24.676 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.675+0000 7fb919e00640 1 -- 172.21.15.152:0/1608432722 learned_addr learned my addr 172.21.15.152:0/1608432722 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:32:24.676 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.675+0000 7fb91a800640 1 --2- 172.21.15.152:0/1608432722 >> v2:172.21.15.175:3300/0 conn(0x7fb91c0d1810 0x7fb91c0d1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:24.677 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.676+0000 7fb919e00640 1 -- 172.21.15.152:0/1608432722 >> v2:172.21.15.175:3300/0 conn(0x7fb91c0d1810 msgr2=0x7fb91c0d1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:24.677 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.676+0000 7fb919e00640 1 --2- 172.21.15.152:0/1608432722 >> v2:172.21.15.175:3300/0 conn(0x7fb91c0d1810 0x7fb91c0d1ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:24.677 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.676+0000 7fb919e00640 1 -- 172.21.15.152:0/1608432722 >> v1:172.21.15.175:6789/0 conn(0x7fb91c0ba520 legacy=0x7fb91c0d28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:24.677 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.676+0000 7fb919e00640 1 -- 172.21.15.152:0/1608432722 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb91c0b8bd0 con 0x7fb91c15eb80 2024-04-20T11:32:24.677 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.676+0000 7fb91a800640 1 --2- 172.21.15.152:0/1608432722 >> v2:172.21.15.175:3300/0 conn(0x7fb91c0d1810 0x7fb91c0d1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:32:24.678 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.678+0000 7fb919e00640 1 --2- 172.21.15.152:0/1608432722 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb91c15eb80 0x7fb91c0b79a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb90c009870 tx=0x7fb90c02b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a496d37441d0318c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:24.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.678+0000 7fb918a00640 1 -- 172.21.15.152:0/1608432722 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb90c035020 con 0x7fb91c15eb80 2024-04-20T11:32:24.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.678+0000 7fb918a00640 1 -- 172.21.15.152:0/1608432722 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb90c031640 con 0x7fb91c15eb80 2024-04-20T11:32:24.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.678+0000 7fb918a00640 1 -- 172.21.15.152:0/1608432722 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb90c02d040 con 0x7fb91c15eb80 2024-04-20T11:32:24.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.679+0000 7fb920e00640 1 -- 172.21.15.152:0/1608432722 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb91c15eb80 msgr2=0x7fb91c0b79a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:24.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.679+0000 7fb920e00640 1 --2- 172.21.15.152:0/1608432722 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb91c15eb80 0x7fb91c0b79a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb90c009870 tx=0x7fb90c02b8f0 comp rx=0 tx=0).stop 2024-04-20T11:32:24.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.679+0000 7fb920e00640 1 -- 172.21.15.152:0/1608432722 shutdown_connections 2024-04-20T11:32:24.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.679+0000 7fb920e00640 1 --2- 172.21.15.152:0/1608432722 >> v2:172.21.15.175:3300/0 conn(0x7fb91c0d1810 0x7fb91c0d1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:24.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.679+0000 7fb920e00640 1 --2- 172.21.15.152:0/1608432722 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb91c15eb80 0x7fb91c0b79a0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:24.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.679+0000 7fb920e00640 1 -- 172.21.15.152:0/1608432722 >> 172.21.15.152:0/1608432722 conn(0x7fb91c067ae0 msgr2=0x7fb91c067ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:24.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.679+0000 7fb920e00640 1 -- 172.21.15.152:0/1608432722 shutdown_connections 2024-04-20T11:32:24.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.680+0000 7fb920e00640 1 -- 172.21.15.152:0/1608432722 wait complete. 2024-04-20T11:32:24.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.682+0000 7fb920e00640 1 Processor -- start 2024-04-20T11:32:24.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.682+0000 7fb920e00640 1 -- start start 2024-04-20T11:32:24.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.683+0000 7fb920e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb91c0ba520 0x7fb91c1609c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:24.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.683+0000 7fb920e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb91c0d1810 0x7fb91c1f4220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:24.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.683+0000 7fb920e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb91c15eb80 0x7fb91c1f4760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:24.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.683+0000 7fb920e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fb91c0b8060 con 0x7fb91c0ba520 2024-04-20T11:32:24.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.683+0000 7fb920e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb91c0b7ee0 con 0x7fb91c15eb80 2024-04-20T11:32:24.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.683+0000 7fb920e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb91c0b93d0 con 0x7fb91c0d1810 2024-04-20T11:32:24.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.683+0000 7fb919e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb91c0ba520 0x7fb91c1609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:24.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.683+0000 7fb919e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb91c0ba520 0x7fb91c1609c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:58744/0 (socket says 172.21.15.152:58744) 2024-04-20T11:32:24.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.683+0000 7fb919e00640 1 -- 172.21.15.152:0/4013707803 learned_addr learned my addr 172.21.15.152:0/4013707803 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:32:24.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.684+0000 7fb919400640 1 --2- 172.21.15.152:0/4013707803 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb91c0d1810 0x7fb91c1f4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:24.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.684+0000 7fb91a800640 1 --2- 172.21.15.152:0/4013707803 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb91c15eb80 0x7fb91c1f4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:24.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.684+0000 7fb919e00640 1 -- 172.21.15.152:0/4013707803 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb91c0d1810 msgr2=0x7fb91c1f4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:24.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.684+0000 7fb919e00640 1 --2- 172.21.15.152:0/4013707803 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb91c0d1810 0x7fb91c1f4220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:24.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.684+0000 7fb919e00640 1 -- 172.21.15.152:0/4013707803 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb91c15eb80 msgr2=0x7fb91c1f4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:24.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.684+0000 7fb919e00640 1 --2- 172.21.15.152:0/4013707803 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb91c15eb80 0x7fb91c1f4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:24.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.684+0000 7fb919e00640 1 -- 172.21.15.152:0/4013707803 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb91c1614f0 con 0x7fb91c0ba520 2024-04-20T11:32:24.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.684+0000 7fb919e00640 1 --2- 172.21.15.152:0/4013707803 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb91c0ba520 0x7fb91c1609c0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7fb90c000c00 tx=0x7fb90c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:24.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.685+0000 7fb912000640 1 -- 172.21.15.152:0/4013707803 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb90c040070 con 0x7fb91c0ba520 2024-04-20T11:32:24.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.685+0000 7fb912000640 1 -- 172.21.15.152:0/4013707803 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb90c0399f0 con 0x7fb91c0ba520 2024-04-20T11:32:24.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.685+0000 7fb912000640 1 -- 172.21.15.152:0/4013707803 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb90c035040 con 0x7fb91c0ba520 2024-04-20T11:32:24.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.685+0000 7fb920e00640 1 -- 172.21.15.152:0/4013707803 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb91c1617d0 con 0x7fb91c0ba520 2024-04-20T11:32:24.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.685+0000 7fb920e00640 1 -- 172.21.15.152:0/4013707803 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb91c160f00 con 0x7fb91c0ba520 2024-04-20T11:32:24.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.686+0000 7fb912000640 1 -- 172.21.15.152:0/4013707803 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 14) ==== 45880+0+0 (secure 0 0 0) 0x7fb90c04b060 con 0x7fb91c0ba520 2024-04-20T11:32:24.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.686+0000 7fb920e00640 1 -- 172.21.15.152:0/4013707803 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb91c0689f0 con 0x7fb91c0ba520 2024-04-20T11:32:24.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.687+0000 7fb912000640 1 --2- 172.21.15.152:0/4013707803 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb8fc038760 0x7fb8fc03ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:24.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.687+0000 7fb912000640 1 -- 172.21.15.152:0/4013707803 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 6268+0+0 (secure 0 0 0) 0x7fb90c06d9c0 con 0x7fb91c0ba520 2024-04-20T11:32:24.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.688+0000 7fb919400640 1 --2- 172.21.15.152:0/4013707803 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb8fc038760 0x7fb8fc03ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:24.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.689+0000 7fb919400640 1 --2- 172.21.15.152:0/4013707803 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb8fc038760 0x7fb8fc03ac20 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fb91c0d22f0 tx=0x7fb908009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:24.692 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.692+0000 7fb912000640 1 -- 172.21.15.152:0/4013707803 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fb90c03ac70 con 0x7fb91c0ba520 2024-04-20T11:32:24.894 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.894+0000 7fb920e00640 1 -- 172.21.15.152:0/4013707803 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb91c0d5660 con 0x7fb91c0ba520 2024-04-20T11:32:24.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.895+0000 7fb912000640 1 -- 172.21.15.152:0/4013707803 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) ==== 74+0+17550 (secure 0 0 0) 0x7fb90c04bdf0 con 0x7fb91c0ba520 2024-04-20T11:32:24.896 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:32:24.896 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":72,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:32:22.542221+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:31:48.024528+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-20T11:31:48.725648+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.152:0/3082578161":"2024-04-21T11:32:22.542176+0000","172.21.15.152:0/1524004838":"2024-04-21T11:32:22.542176+0000","172.21.15.152:6833/2285110240":"2024-04-21T11:32:22.542176+0000","172.21.15.152:6832/2285110240":"2024-04-21T11:32:22.542176+0000","172.21.15.152:0/698943652":"2024-04-21T11:32:22.542176+0000","172.21.15.152:0/142101408":"2024-04-21T11:32:22.542176+0000","172.21.15.152:0/2510881178":"2024-04-21T11:32:22.542176+0000","172.21.15.152:0/3263977099":"2024-04-21T11:32:22.542176+0000","172.21.15.152:0/2184739958":"2024-04-21T11:32:22.542176+0000","172.21.15.152:0/3331653721":"2024-04-21T11:32:22.542176+0000","172.21.15.152:0/2405473735":"2024-04-21T11:32:22.542176+0000","172.21.15.152:0/2828322453":"2024-04-21T11:32:22.542176+0000","172.21.15.152:0/3593059376":"2024-04-21T11:32:22.542176+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:32:24.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.898+0000 7fb920e00640 1 -- 172.21.15.152:0/4013707803 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb8fc038760 msgr2=0x7fb8fc03ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:24.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.898+0000 7fb920e00640 1 --2- 172.21.15.152:0/4013707803 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb8fc038760 0x7fb8fc03ac20 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fb91c0d22f0 tx=0x7fb908009040 comp rx=0 tx=0).stop 2024-04-20T11:32:24.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.898+0000 7fb920e00640 1 -- 172.21.15.152:0/4013707803 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb91c0ba520 msgr2=0x7fb91c1609c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:24.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.898+0000 7fb920e00640 1 --2- 172.21.15.152:0/4013707803 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb91c0ba520 0x7fb91c1609c0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7fb90c000c00 tx=0x7fb90c0046c0 comp rx=0 tx=0).stop 2024-04-20T11:32:24.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.898+0000 7fb920e00640 1 -- 172.21.15.152:0/4013707803 shutdown_connections 2024-04-20T11:32:24.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.898+0000 7fb920e00640 1 --2- 172.21.15.152:0/4013707803 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb8fc038760 0x7fb8fc03ac20 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:24.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.898+0000 7fb920e00640 1 --2- 172.21.15.152:0/4013707803 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb91c15eb80 0x7fb91c1f4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:24.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.898+0000 7fb920e00640 1 --2- 172.21.15.152:0/4013707803 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb91c0d1810 0x7fb91c1f4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:24.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.898+0000 7fb920e00640 1 --2- 172.21.15.152:0/4013707803 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb91c0ba520 0x7fb91c1609c0 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:24.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.898+0000 7fb920e00640 1 -- 172.21.15.152:0/4013707803 >> 172.21.15.152:0/4013707803 conn(0x7fb91c067ae0 msgr2=0x7fb91c0d3260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:24.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.898+0000 7fb920e00640 1 -- 172.21.15.152:0/4013707803 shutdown_connections 2024-04-20T11:32:24.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:24.898+0000 7fb920e00640 1 -- 172.21.15.152:0/4013707803 wait complete. 2024-04-20T11:32:24.918 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T11:26:47.246571+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T11:31:48.024528+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '70', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T11:31:48.725648+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '71', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T11:32:24.918 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T11:32:24.919 INFO:tasks.ceph.mgr.y.smithi152.stderr:2024-04-20T11:32:24.918+0000 7fe15ce4b040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-20T11:32:25.110 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.109+0000 7ff45fe00640 1 Processor -- start 2024-04-20T11:32:25.110 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.109+0000 7ff45fe00640 1 -- start start 2024-04-20T11:32:25.112 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.110+0000 7ff45fe00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ff4601622c0 0x7ff4600cfe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:25.112 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.110+0000 7ff45fe00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7ff4601626a0 0x7ff4600d31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:25.112 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.110+0000 7ff45fe00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4600ced60 con 0x7ff4600d3720 2024-04-20T11:32:25.112 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.110+0000 7ff45fe00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7ff460163170 con 0x7ff4601626a0 2024-04-20T11:32:25.112 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.110+0000 7ff45fe00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7ff4600cf970 con 0x7ff4601622c0 2024-04-20T11:32:25.112 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.111+0000 7ff457e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7ff4601626a0 0x7ff4600d31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:25.112 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.111+0000 7ff45f400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7ff4600d3720 0x7ff4601600a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:50318/0 (socket says 172.21.15.152:50318) 2024-04-20T11:32:25.112 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.111+0000 7ff45f400640 1 -- 172.21.15.152:0/1888558404 learned_addr learned my addr 172.21.15.152:0/1888558404 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:32:25.112 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.111+0000 7ff457e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7ff4601626a0 0x7ff4600d31e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:58758/0 (socket says 172.21.15.152:58758) 2024-04-20T11:32:25.113 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.111+0000 7ff45ea00640 1 --2- 172.21.15.152:0/1888558404 >> v2:172.21.15.175:3300/0 conn(0x7ff4601622c0 0x7ff4600cfe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:25.113 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.111+0000 7ff457e00640 1 -- 172.21.15.152:0/1888558404 >> v2:172.21.15.175:3300/0 conn(0x7ff4601622c0 msgr2=0x7ff4600cfe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:25.113 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.111+0000 7ff457e00640 1 --2- 172.21.15.152:0/1888558404 >> v2:172.21.15.175:3300/0 conn(0x7ff4601622c0 0x7ff4600cfe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:25.113 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.111+0000 7ff457e00640 1 -- 172.21.15.152:0/1888558404 >> v1:172.21.15.152:6789/0 conn(0x7ff4600d3720 legacy=0x7ff4601600a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:25.113 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.111+0000 7ff457e00640 1 -- 172.21.15.152:0/1888558404 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4600cf040 con 0x7ff4601626a0 2024-04-20T11:32:25.113 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.111+0000 7ff45ea00640 1 --2- 172.21.15.152:0/1888558404 >> v2:172.21.15.175:3300/0 conn(0x7ff4601622c0 0x7ff4600cfe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:32:25.113 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.111+0000 7ff45e000640 1 -- 172.21.15.152:0/1888558404 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1369961906 0 0) 0x7ff4600ced60 con 0x7ff4600d3720 2024-04-20T11:32:25.114 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.113+0000 7ff457e00640 1 --2- 172.21.15.152:0/1888558404 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff4601626a0 0x7ff4600d31e0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7ff46008de20 tx=0x7ff44802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b92b1cf9964cd7bf server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:25.114 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.113+0000 7ff45e000640 1 -- 172.21.15.152:0/1888558404 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff448035020 con 0x7ff4601626a0 2024-04-20T11:32:25.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.113+0000 7ff45e000640 1 -- 172.21.15.152:0/1888558404 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff448031830 con 0x7ff4601626a0 2024-04-20T11:32:25.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.113+0000 7ff45e000640 1 -- 172.21.15.152:0/1888558404 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff448031c00 con 0x7ff4601626a0 2024-04-20T11:32:25.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.113+0000 7ff45fe00640 1 -- 172.21.15.152:0/1888558404 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff4601626a0 msgr2=0x7ff4600d31e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:25.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.113+0000 7ff45fe00640 1 --2- 172.21.15.152:0/1888558404 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff4601626a0 0x7ff4600d31e0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7ff46008de20 tx=0x7ff44802b8d0 comp rx=0 tx=0).stop 2024-04-20T11:32:25.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.114+0000 7ff45fe00640 1 -- 172.21.15.152:0/1888558404 shutdown_connections 2024-04-20T11:32:25.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.114+0000 7ff45fe00640 1 --2- 172.21.15.152:0/1888558404 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff4601626a0 0x7ff4600d31e0 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:25.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.114+0000 7ff45fe00640 1 --2- 172.21.15.152:0/1888558404 >> v2:172.21.15.175:3300/0 conn(0x7ff4601622c0 0x7ff4600cfe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:25.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.114+0000 7ff45fe00640 1 -- 172.21.15.152:0/1888558404 >> 172.21.15.152:0/1888558404 conn(0x7ff46008bfd0 msgr2=0x7ff4600d4570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:25.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.114+0000 7ff45fe00640 1 -- 172.21.15.152:0/1888558404 shutdown_connections 2024-04-20T11:32:25.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.114+0000 7ff45fe00640 1 -- 172.21.15.152:0/1888558404 wait complete. 2024-04-20T11:32:25.116 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.115+0000 7ff45fe00640 1 Processor -- start 2024-04-20T11:32:25.116 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.116+0000 7ff45fe00640 1 -- start start 2024-04-20T11:32:25.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.116+0000 7ff45fe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4600d3720 0x7ff460161480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:25.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.116+0000 7ff45fe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff4601622c0 0x7ff4600ca270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:25.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.117+0000 7ff45fe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4601626a0 0x7ff4600c8470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:25.118 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.117+0000 7ff45fe00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7ff460162b20 con 0x7ff4601622c0 2024-04-20T11:32:25.118 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.117+0000 7ff45fe00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff4600cf600 con 0x7ff4600d3720 2024-04-20T11:32:25.118 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.117+0000 7ff45fe00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff46015f9c0 con 0x7ff4601626a0 2024-04-20T11:32:25.118 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.117+0000 7ff457e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff4601622c0 0x7ff4600ca270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:25.118 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.117+0000 7ff457e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff4601622c0 0x7ff4600ca270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:58772/0 (socket says 172.21.15.152:58772) 2024-04-20T11:32:25.118 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.117+0000 7ff457e00640 1 -- 172.21.15.152:0/3779749930 learned_addr learned my addr 172.21.15.152:0/3779749930 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:32:25.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.117+0000 7ff457e00640 1 -- 172.21.15.152:0/3779749930 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4601626a0 msgr2=0x7ff4600c8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:25.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.117+0000 7ff457e00640 1 --2- 172.21.15.152:0/3779749930 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4601626a0 0x7ff4600c8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:25.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.117+0000 7ff45ea00640 1 --2- 172.21.15.152:0/3779749930 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4600d3720 0x7ff460161480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:25.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.117+0000 7ff457e00640 1 -- 172.21.15.152:0/3779749930 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4600d3720 msgr2=0x7ff460161480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:25.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.117+0000 7ff457e00640 1 --2- 172.21.15.152:0/3779749930 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4600d3720 0x7ff460161480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:25.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.117+0000 7ff457e00640 1 -- 172.21.15.152:0/3779749930 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4600cad50 con 0x7ff4601622c0 2024-04-20T11:32:25.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.118+0000 7ff457e00640 1 --2- 172.21.15.152:0/3779749930 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff4601622c0 0x7ff4600ca270 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7ff448009880 tx=0x7ff4480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:25.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.118+0000 7ff456a00640 1 -- 172.21.15.152:0/3779749930 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff448041070 con 0x7ff4601622c0 2024-04-20T11:32:25.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.118+0000 7ff456a00640 1 -- 172.21.15.152:0/3779749930 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff448005bb0 con 0x7ff4601622c0 2024-04-20T11:32:25.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.118+0000 7ff456a00640 1 -- 172.21.15.152:0/3779749930 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff448035040 con 0x7ff4601622c0 2024-04-20T11:32:25.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.118+0000 7ff45fe00640 1 -- 172.21.15.152:0/3779749930 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4600cb030 con 0x7ff4601622c0 2024-04-20T11:32:25.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.118+0000 7ff45fe00640 1 -- 172.21.15.152:0/3779749930 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4600cc140 con 0x7ff4601622c0 2024-04-20T11:32:25.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.119+0000 7ff456a00640 1 -- 172.21.15.152:0/3779749930 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 14) ==== 45880+0+0 (secure 0 0 0) 0x7ff448038020 con 0x7ff4601622c0 2024-04-20T11:32:25.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.119+0000 7ff45fe00640 1 -- 172.21.15.152:0/3779749930 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff460075e60 con 0x7ff4601622c0 2024-04-20T11:32:25.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.120+0000 7ff456a00640 1 --2- 172.21.15.152:0/3779749930 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7ff43c038790 0x7ff43c03ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:25.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.120+0000 7ff456a00640 1 -- 172.21.15.152:0/3779749930 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 6268+0+0 (secure 0 0 0) 0x7ff44806dc00 con 0x7ff4601622c0 2024-04-20T11:32:25.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.120+0000 7ff45ea00640 1 --2- 172.21.15.152:0/3779749930 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7ff43c038790 0x7ff43c03ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:25.122 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.122+0000 7ff45ea00640 1 --2- 172.21.15.152:0/3779749930 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7ff43c038790 0x7ff43c03ac50 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff450008850 tx=0x7ff450008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:25.125 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.125+0000 7ff456a00640 1 -- 172.21.15.152:0/3779749930 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ff44803bd10 con 0x7ff4601622c0 2024-04-20T11:32:25.177 INFO:tasks.ceph.mgr.y.smithi152.stderr:2024-04-20T11:32:25.176+0000 7fe15ce4b040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-20T11:32:25.236 INFO:tasks.ceph.mgr.y.smithi152.stderr:2024-04-20T11:32:25.235+0000 7fe15ce4b040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-20T11:32:25.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.321+0000 7ff45fe00640 1 -- 172.21.15.152:0/3779749930 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff4600b6d40 con 0x7ff4601622c0 2024-04-20T11:32:25.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.322+0000 7ff456a00640 1 -- 172.21.15.152:0/3779749930 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v72) ==== 93+0+10 (secure 0 0 0) 0x7ff448006030 con 0x7ff4601622c0 2024-04-20T11:32:25.322 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 1 2024-04-20T11:32:25.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.324+0000 7ff45fe00640 1 -- 172.21.15.152:0/3779749930 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7ff43c038790 msgr2=0x7ff43c03ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:25.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.324+0000 7ff45fe00640 1 --2- 172.21.15.152:0/3779749930 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7ff43c038790 0x7ff43c03ac50 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff450008850 tx=0x7ff450008040 comp rx=0 tx=0).stop 2024-04-20T11:32:25.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.324+0000 7ff45fe00640 1 -- 172.21.15.152:0/3779749930 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff4601622c0 msgr2=0x7ff4600ca270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:25.326 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.324+0000 7ff45fe00640 1 --2- 172.21.15.152:0/3779749930 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff4601622c0 0x7ff4600ca270 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7ff448009880 tx=0x7ff4480046c0 comp rx=0 tx=0).stop 2024-04-20T11:32:25.326 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.325+0000 7ff45fe00640 1 -- 172.21.15.152:0/3779749930 shutdown_connections 2024-04-20T11:32:25.326 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.325+0000 7ff45fe00640 1 --2- 172.21.15.152:0/3779749930 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7ff43c038790 0x7ff43c03ac50 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:25.326 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.325+0000 7ff45fe00640 1 --2- 172.21.15.152:0/3779749930 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4601626a0 0x7ff4600c8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:25.326 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.325+0000 7ff45fe00640 1 --2- 172.21.15.152:0/3779749930 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff4601622c0 0x7ff4600ca270 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:25.326 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.325+0000 7ff45fe00640 1 --2- 172.21.15.152:0/3779749930 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4600d3720 0x7ff460161480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:25.326 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.325+0000 7ff45fe00640 1 -- 172.21.15.152:0/3779749930 >> 172.21.15.152:0/3779749930 conn(0x7ff46008bfd0 msgr2=0x7ff4600d26e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:25.326 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.325+0000 7ff45fe00640 1 -- 172.21.15.152:0/3779749930 shutdown_connections 2024-04-20T11:32:25.326 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.325+0000 7ff45fe00640 1 -- 172.21.15.152:0/3779749930 wait complete. 2024-04-20T11:32:25.344 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T11:32:25.370 INFO:tasks.ceph.mgr.y.smithi152.stderr:2024-04-20T11:32:25.369+0000 7fe15ce4b040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-20T11:32:25.537 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.536+0000 7f062be00640 1 Processor -- start 2024-04-20T11:32:25.537 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.536+0000 7f062be00640 1 -- start start 2024-04-20T11:32:25.539 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.538+0000 7f062be00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f062c0d0920 0x7f062c0d0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:25.539 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.538+0000 7f062be00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f062c0d1240 0x7f062c09b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:25.539 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.538+0000 7f062be00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f062c06cce0 con 0x7f062c09b990 2024-04-20T11:32:25.539 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.538+0000 7f062be00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f062c06ca90 con 0x7f062c0d0920 2024-04-20T11:32:25.540 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.538+0000 7f062be00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f062c06d8f0 con 0x7f062c0d1240 2024-04-20T11:32:25.540 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.538+0000 7f062aa00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f062c0d0920 0x7f062c0d0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:25.540 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.538+0000 7f062aa00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f062c0d0920 0x7f062c0d0d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:58786/0 (socket says 172.21.15.152:58786) 2024-04-20T11:32:25.540 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.538+0000 7f062aa00640 1 -- 172.21.15.152:0/3768474687 learned_addr learned my addr 172.21.15.152:0/3768474687 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:32:25.540 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.539+0000 7f0623e00640 1 --2- 172.21.15.152:0/3768474687 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f062c0d1240 0x7f062c09b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:25.540 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.539+0000 7f062aa00640 1 -- 172.21.15.152:0/3768474687 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f062c0d1240 msgr2=0x7f062c09b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:25.540 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.539+0000 7f062aa00640 1 --2- 172.21.15.152:0/3768474687 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f062c0d1240 0x7f062c09b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:25.540 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.539+0000 7f062aa00640 1 -- 172.21.15.152:0/3768474687 >> v1:172.21.15.175:6789/0 conn(0x7f062c09b990 legacy=0x7f062c0cc9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:25.540 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.539+0000 7f062aa00640 1 -- 172.21.15.152:0/3768474687 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f062c06cfc0 con 0x7f062c0d0920 2024-04-20T11:32:25.540 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.539+0000 7f0623e00640 1 --2- 172.21.15.152:0/3768474687 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f062c0d1240 0x7f062c09b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:32:25.541 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.540+0000 7f062aa00640 1 --2- 172.21.15.152:0/3768474687 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f062c0d0920 0x7f062c0d0d00 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f061c009870 tx=0x7f061c02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb28d91900e95965 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:25.541 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.540+0000 7f062a000640 1 -- 172.21.15.152:0/3768474687 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f061c0317c0 con 0x7f062c0d0920 2024-04-20T11:32:25.541 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.540+0000 7f062a000640 1 -- 172.21.15.152:0/3768474687 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f061c037070 con 0x7f062c0d0920 2024-04-20T11:32:25.541 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.540+0000 7f062a000640 1 -- 172.21.15.152:0/3768474687 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f061c031b80 con 0x7f062c0d0920 2024-04-20T11:32:25.542 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.541+0000 7f062be00640 1 -- 172.21.15.152:0/3768474687 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f062c0d0920 msgr2=0x7f062c0d0d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:25.542 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.541+0000 7f062be00640 1 --2- 172.21.15.152:0/3768474687 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f062c0d0920 0x7f062c0d0d00 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f061c009870 tx=0x7f061c02b860 comp rx=0 tx=0).stop 2024-04-20T11:32:25.542 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.541+0000 7f062be00640 1 -- 172.21.15.152:0/3768474687 shutdown_connections 2024-04-20T11:32:25.542 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.541+0000 7f062be00640 1 --2- 172.21.15.152:0/3768474687 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f062c0d1240 0x7f062c09b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:25.542 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.541+0000 7f062be00640 1 --2- 172.21.15.152:0/3768474687 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f062c0d0920 0x7f062c0d0d00 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:25.542 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.541+0000 7f062be00640 1 -- 172.21.15.152:0/3768474687 >> 172.21.15.152:0/3768474687 conn(0x7f062c0b3e60 msgr2=0x7f062c0b4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:25.542 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.541+0000 7f062be00640 1 -- 172.21.15.152:0/3768474687 shutdown_connections 2024-04-20T11:32:25.542 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.541+0000 7f062be00640 1 -- 172.21.15.152:0/3768474687 wait complete. 2024-04-20T11:32:25.543 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.543+0000 7f062be00640 1 Processor -- start 2024-04-20T11:32:25.543 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.543+0000 7f062be00640 1 -- start start 2024-04-20T11:32:25.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.544+0000 7f062be00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f062c09b990 0x7f062c1536e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:25.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.544+0000 7f062be00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f062c0d0920 0x7f062c1503a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:25.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.544+0000 7f062be00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f062c0d1240 0x7f062c1606c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:25.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.544+0000 7f062be00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f062c06dd80 con 0x7f062c09b990 2024-04-20T11:32:25.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.544+0000 7f062be00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f062c06d580 con 0x7f062c0d1240 2024-04-20T11:32:25.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.544+0000 7f062be00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f062c153c20 con 0x7f062c0d0920 2024-04-20T11:32:25.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.544+0000 7f062aa00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f062c09b990 0x7f062c1536e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:25.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.544+0000 7f062aa00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f062c09b990 0x7f062c1536e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:58800/0 (socket says 172.21.15.152:58800) 2024-04-20T11:32:25.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.544+0000 7f062aa00640 1 -- 172.21.15.152:0/1547008924 learned_addr learned my addr 172.21.15.152:0/1547008924 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:32:25.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.544+0000 7f062aa00640 1 -- 172.21.15.152:0/1547008924 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f062c0d0920 msgr2=0x7f062c1503a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:25.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.544+0000 7f0623e00640 1 --2- 172.21.15.152:0/1547008924 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f062c0d0920 0x7f062c1503a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:25.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.545+0000 7f062aa00640 1 --2- 172.21.15.152:0/1547008924 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f062c0d0920 0x7f062c1503a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:25.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.545+0000 7f062aa00640 1 -- 172.21.15.152:0/1547008924 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f062c0d1240 msgr2=0x7f062c1606c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:25.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.545+0000 7f062aa00640 1 --2- 172.21.15.152:0/1547008924 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f062c0d1240 0x7f062c1606c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:25.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.545+0000 7f062aa00640 1 -- 172.21.15.152:0/1547008924 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f062c0c7ed0 con 0x7f062c09b990 2024-04-20T11:32:25.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.545+0000 7f062aa00640 1 --2- 172.21.15.152:0/1547008924 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f062c09b990 0x7f062c1536e0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f061c000c00 tx=0x7f061c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:25.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.545+0000 7f0623400640 1 -- 172.21.15.152:0/1547008924 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f061c032860 con 0x7f062c09b990 2024-04-20T11:32:25.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.545+0000 7f0623400640 1 -- 172.21.15.152:0/1547008924 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f061c032a20 con 0x7f062c09b990 2024-04-20T11:32:25.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.545+0000 7f0623400640 1 -- 172.21.15.152:0/1547008924 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f061c032e10 con 0x7f062c09b990 2024-04-20T11:32:25.548 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.545+0000 7f062be00640 1 -- 172.21.15.152:0/1547008924 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f062c0c81b0 con 0x7f062c09b990 2024-04-20T11:32:25.548 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.545+0000 7f062be00640 1 -- 172.21.15.152:0/1547008924 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f062c0c8770 con 0x7f062c09b990 2024-04-20T11:32:25.548 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.547+0000 7f062be00640 1 -- 172.21.15.152:0/1547008924 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f062c075e60 con 0x7f062c09b990 2024-04-20T11:32:25.548 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.547+0000 7f0623400640 1 -- 172.21.15.152:0/1547008924 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 14) ==== 45880+0+0 (secure 0 0 0) 0x7f061c04c060 con 0x7f062c09b990 2024-04-20T11:32:25.548 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.547+0000 7f0623400640 1 --2- 172.21.15.152:0/1547008924 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f0608038760 0x7f060803ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:25.548 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.547+0000 7f0623400640 1 -- 172.21.15.152:0/1547008924 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 6268+0+0 (secure 0 0 0) 0x7f061c06db60 con 0x7f062c09b990 2024-04-20T11:32:25.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.548+0000 7f0623e00640 1 --2- 172.21.15.152:0/1547008924 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f0608038760 0x7f060803ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:25.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.549+0000 7f0623e00640 1 --2- 172.21.15.152:0/1547008924 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f0608038760 0x7f060803ac20 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f062c0cc4b0 tx=0x7f0614008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:25.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.552+0000 7f0623400640 1 -- 172.21.15.152:0/1547008924 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f061c040070 con 0x7f062c09b990 2024-04-20T11:32:25.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.573+0000 7f0623400640 1 -- 172.21.15.152:0/1547008924 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mgrmap(e 15) ==== 45936+0+0 (secure 0 0 0) 0x7f061c03a380 con 0x7f062c09b990 2024-04-20T11:32:25.694 INFO:tasks.ceph.mgr.y.smithi152.stderr:2024-04-20T11:32:25.693+0000 7fe15ce4b040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-20T11:32:25.746 INFO:tasks.ceph.mgr.y.smithi152.stderr:2024-04-20T11:32:25.745+0000 7fe15ce4b040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-20T11:32:25.752 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.752+0000 7f062be00640 1 -- 172.21.15.152:0/1547008924 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f062c0704e0 con 0x7f062c09b990 2024-04-20T11:32:25.753 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.753+0000 7f0623400640 1 -- 172.21.15.152:0/1547008924 <== mon.0 v2:172.21.15.152:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v72) ==== 104+0+11 (secure 0 0 0) 0x7f061c04c350 con 0x7f062c09b990 2024-04-20T11:32:25.754 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 32 2024-04-20T11:32:25.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.755+0000 7f062be00640 1 -- 172.21.15.152:0/1547008924 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f0608038760 msgr2=0x7f060803ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:25.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.755+0000 7f062be00640 1 --2- 172.21.15.152:0/1547008924 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f0608038760 0x7f060803ac20 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f062c0cc4b0 tx=0x7f0614008040 comp rx=0 tx=0).stop 2024-04-20T11:32:25.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.755+0000 7f062be00640 1 -- 172.21.15.152:0/1547008924 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f062c09b990 msgr2=0x7f062c1536e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:25.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.755+0000 7f062be00640 1 --2- 172.21.15.152:0/1547008924 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f062c09b990 0x7f062c1536e0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f061c000c00 tx=0x7f061c0046c0 comp rx=0 tx=0).stop 2024-04-20T11:32:25.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.755+0000 7f062be00640 1 -- 172.21.15.152:0/1547008924 shutdown_connections 2024-04-20T11:32:25.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.755+0000 7f062be00640 1 --2- 172.21.15.152:0/1547008924 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f0608038760 0x7f060803ac20 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:25.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.755+0000 7f062be00640 1 --2- 172.21.15.152:0/1547008924 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f062c0d1240 0x7f062c1606c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:25.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.755+0000 7f062be00640 1 --2- 172.21.15.152:0/1547008924 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f062c0d0920 0x7f062c1503a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:25.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.755+0000 7f062be00640 1 --2- 172.21.15.152:0/1547008924 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f062c09b990 0x7f062c1536e0 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:25.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.755+0000 7f062be00640 1 -- 172.21.15.152:0/1547008924 >> 172.21.15.152:0/1547008924 conn(0x7f062c0b3e60 msgr2=0x7f062c0b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:25.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.755+0000 7f062be00640 1 -- 172.21.15.152:0/1547008924 shutdown_connections 2024-04-20T11:32:25.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:25.756+0000 7f062be00640 1 -- 172.21.15.152:0/1547008924 wait complete. 2024-04-20T11:32:25.774 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T11:32:25.924 INFO:tasks.ceph.mgr.y.smithi152.stderr:2024-04-20T11:32:25.923+0000 7fe15ce4b040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-20T11:32:26.010 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.009+0000 7fdc06e00640 1 Processor -- start 2024-04-20T11:32:26.010 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.010+0000 7fdc06e00640 1 -- start start 2024-04-20T11:32:26.011 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.011+0000 7fdc06e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdc000d46a0 0x7fdc000d4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:26.012 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.011+0000 7fdc06e00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc000d19a0 con 0x7fdc0015e9e0 2024-04-20T11:32:26.012 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.011+0000 7fdc06e00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc000d1c80 con 0x7fdc000d0d90 2024-04-20T11:32:26.012 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.011+0000 7fdc06e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fdc0006da30 con 0x7fdc000d46a0 2024-04-20T11:32:26.013 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.011+0000 7fdbffe00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fdc0015e9e0 0x7fdc0006d320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:33634/0 (socket says 172.21.15.152:33634) 2024-04-20T11:32:26.013 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.011+0000 7fdbffe00640 1 -- 172.21.15.152:0/2640669039 learned_addr learned my addr 172.21.15.152:0/2640669039 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:32:26.013 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.012+0000 7fdbf7400640 1 --2- 172.21.15.152:0/2640669039 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdc000d46a0 0x7fdc000d4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:26.013 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.012+0000 7fdbff400640 1 -- 172.21.15.152:0/2640669039 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3384488946 0 0) 0x7fdc000d19a0 con 0x7fdc0015e9e0 2024-04-20T11:32:26.013 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.012+0000 7fdbff400640 1 -- 172.21.15.152:0/2640669039 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdbe4003520 con 0x7fdc0015e9e0 2024-04-20T11:32:26.013 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.012+0000 7fdbff400640 1 -- 172.21.15.152:0/2640669039 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4240856710 0 0) 0x7fdbe4003520 con 0x7fdc0015e9e0 2024-04-20T11:32:26.014 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.012+0000 7fdbff400640 1 -- 172.21.15.152:0/2640669039 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdc000d46a0 msgr2=0x7fdc000d4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:26.014 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.012+0000 7fdbff400640 1 --2- 172.21.15.152:0/2640669039 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdc000d46a0 0x7fdc000d4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:26.014 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.012+0000 7fdbff400640 1 -- 172.21.15.152:0/2640669039 >> v1:172.21.15.175:6789/0 conn(0x7fdc000d0d90 legacy=0x7fdc000cfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:26.014 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.012+0000 7fdbff400640 1 -- 172.21.15.152:0/2640669039 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc000d1f60 con 0x7fdc0015e9e0 2024-04-20T11:32:26.014 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.012+0000 7fdbff400640 1 -- 172.21.15.152:0/2640669039 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7fdbf00019f0 con 0x7fdc0015e9e0 2024-04-20T11:32:26.014 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.012+0000 7fdbff400640 1 -- 172.21.15.152:0/2640669039 >> v1:172.21.15.152:6789/0 conn(0x7fdc0015e9e0 legacy=0x7fdc0006d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:26.014 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.012+0000 7fdbff400640 1 --2- 172.21.15.152:0/2640669039 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdbe40039e0 0x7fdbe4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:26.014 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.013+0000 7fdbff400640 1 --2- 172.21.15.152:0/2640669039 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdbe40043d0 0x7fdbe40048a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:26.014 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.013+0000 7fdbff400640 1 --2- 172.21.15.152:0/2640669039 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdc000d46a0 0x7fdbe4005100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:26.014 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.013+0000 7fdbff400640 1 -- 172.21.15.152:0/2640669039 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fdc0006da30 con 0x7fdbe40043d0 2024-04-20T11:32:26.014 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.013+0000 7fdbff400640 1 -- 172.21.15.152:0/2640669039 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe400b9f0 con 0x7fdbe40039e0 2024-04-20T11:32:26.014 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.013+0000 7fdbff400640 1 -- 172.21.15.152:0/2640669039 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fdbe400bbf0 con 0x7fdc000d46a0 2024-04-20T11:32:26.014 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.013+0000 7fdbf7400640 1 --2- 172.21.15.152:0/2640669039 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdbe40043d0 0x7fdbe40048a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:26.016 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.013+0000 7fdc04e00640 1 --2- 172.21.15.152:0/2640669039 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdc000d46a0 0x7fdbe4005100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:26.016 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.013+0000 7fdbffe00640 1 --2- 172.21.15.152:0/2640669039 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdbe40039e0 0x7fdbe4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:26.016 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.013+0000 7fdbf7400640 1 -- 172.21.15.152:0/2640669039 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdc000d46a0 msgr2=0x7fdbe4005100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:26.016 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.013+0000 7fdbf7400640 1 --2- 172.21.15.152:0/2640669039 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdc000d46a0 0x7fdbe4005100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:26.016 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.013+0000 7fdbf7400640 1 -- 172.21.15.152:0/2640669039 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdbe40039e0 msgr2=0x7fdbe4003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:26.016 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.013+0000 7fdbf7400640 1 --2- 172.21.15.152:0/2640669039 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdbe40039e0 0x7fdbe4003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:26.016 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.013+0000 7fdbf7400640 1 -- 172.21.15.152:0/2640669039 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc000d1f60 con 0x7fdbe40043d0 2024-04-20T11:32:26.016 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.013+0000 7fdbffe00640 1 --2- 172.21.15.152:0/2640669039 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdbe40039e0 0x7fdbe4003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:32:26.016 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.015+0000 7fdbf7400640 1 --2- 172.21.15.152:0/2640669039 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdbe40043d0 0x7fdbe40048a0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fdbe8009ac0 tx=0x7fdbe4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d69ca57e31fafeb2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:26.016 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.015+0000 7fdbff400640 1 -- 172.21.15.152:0/2640669039 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe8039070 con 0x7fdbe40043d0 2024-04-20T11:32:26.016 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.015+0000 7fdbff400640 1 -- 172.21.15.152:0/2640669039 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbe802c6b0 con 0x7fdbe40043d0 2024-04-20T11:32:26.016 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.015+0000 7fdbff400640 1 -- 172.21.15.152:0/2640669039 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe8009690 con 0x7fdbe40043d0 2024-04-20T11:32:26.017 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.016+0000 7fdc06e00640 1 -- 172.21.15.152:0/2640669039 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdbe40043d0 msgr2=0x7fdbe40048a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:26.017 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.016+0000 7fdc06e00640 1 --2- 172.21.15.152:0/2640669039 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdbe40043d0 0x7fdbe40048a0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fdbe8009ac0 tx=0x7fdbe4005830 comp rx=0 tx=0).stop 2024-04-20T11:32:26.017 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.016+0000 7fdc04e00640 1 -- 172.21.15.152:0/2640669039 reap_dead start 2024-04-20T11:32:26.017 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.016+0000 7fdc06e00640 1 -- 172.21.15.152:0/2640669039 shutdown_connections 2024-04-20T11:32:26.018 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.016+0000 7fdc06e00640 1 -- 172.21.15.152:0/2640669039 >> 172.21.15.152:0/2640669039 conn(0x7fdc0009dab0 msgr2=0x7fdc0008bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:26.018 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.016+0000 7fdc06e00640 1 -- 172.21.15.152:0/2640669039 shutdown_connections 2024-04-20T11:32:26.018 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.016+0000 7fdc06e00640 1 -- 172.21.15.152:0/2640669039 wait complete. 2024-04-20T11:32:26.019 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.019+0000 7fdc06e00640 1 Processor -- start 2024-04-20T11:32:26.019 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.019+0000 7fdc06e00640 1 -- start start 2024-04-20T11:32:26.022 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.021+0000 7fdc06e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdbe40043d0 0x7fdc0015fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:26.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.021+0000 7fdc06e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdc001616a0 0x7fdc000b7e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:26.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.021+0000 7fdc06e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdc001e8fd0 0x7fdc001e9460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:26.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.021+0000 7fdc06e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fdc0015edc0 con 0x7fdc001e8fd0 2024-04-20T11:32:26.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.021+0000 7fdc06e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdc000cf880 con 0x7fdbe40043d0 2024-04-20T11:32:26.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.021+0000 7fdc06e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fdc0015ef40 con 0x7fdc001616a0 2024-04-20T11:32:26.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.021+0000 7fdc04e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdc001e8fd0 0x7fdc001e9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:26.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.021+0000 7fdc04e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdc001e8fd0 0x7fdc001e9460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:60066/0 (socket says 172.21.15.152:60066) 2024-04-20T11:32:26.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.021+0000 7fdc04e00640 1 -- 172.21.15.152:0/3725468611 learned_addr learned my addr 172.21.15.152:0/3725468611 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:32:26.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.021+0000 7fdbf7400640 1 --2- 172.21.15.152:0/3725468611 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdc001616a0 0x7fdc000b7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:26.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.021+0000 7fdbffe00640 1 --2- 172.21.15.152:0/3725468611 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdbe40043d0 0x7fdc0015fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:26.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.021+0000 7fdc04e00640 1 -- 172.21.15.152:0/3725468611 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdc001616a0 msgr2=0x7fdc000b7e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:26.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.021+0000 7fdc04e00640 1 --2- 172.21.15.152:0/3725468611 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdc001616a0 0x7fdc000b7e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:26.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.021+0000 7fdc04e00640 1 -- 172.21.15.152:0/3725468611 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdbe40043d0 msgr2=0x7fdc0015fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:26.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.021+0000 7fdc04e00640 1 --2- 172.21.15.152:0/3725468611 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdbe40043d0 0x7fdc0015fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:26.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.021+0000 7fdc04e00640 1 -- 172.21.15.152:0/3725468611 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc000b83b0 con 0x7fdc001e8fd0 2024-04-20T11:32:26.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.021+0000 7fdbffe00640 1 --2- 172.21.15.152:0/3725468611 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdbe40043d0 0x7fdc0015fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:32:26.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.022+0000 7fdc04e00640 1 --2- 172.21.15.152:0/3725468611 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdc001e8fd0 0x7fdc001e9460 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fdbf8000c00 tx=0x7fdbf800e8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:26.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.022+0000 7fdbfcc00640 1 -- 172.21.15.152:0/3725468611 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbf8019070 con 0x7fdc001e8fd0 2024-04-20T11:32:26.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.022+0000 7fdbfcc00640 1 -- 172.21.15.152:0/3725468611 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbf80052e0 con 0x7fdc001e8fd0 2024-04-20T11:32:26.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.022+0000 7fdbfcc00640 1 -- 172.21.15.152:0/3725468611 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbf8003830 con 0x7fdc001e8fd0 2024-04-20T11:32:26.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.022+0000 7fdc06e00640 1 -- 172.21.15.152:0/3725468611 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc0015c1a0 con 0x7fdc001e8fd0 2024-04-20T11:32:26.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.022+0000 7fdc06e00640 1 -- 172.21.15.152:0/3725468611 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc001608e0 con 0x7fdc001e8fd0 2024-04-20T11:32:26.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.024+0000 7fdbfcc00640 1 -- 172.21.15.152:0/3725468611 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 15) ==== 45936+0+0 (secure 0 0 0) 0x7fdbf801c070 con 0x7fdc001e8fd0 2024-04-20T11:32:26.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.024+0000 7fdc06e00640 1 -- 172.21.15.152:0/3725468611 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc000689f0 con 0x7fdc001e8fd0 2024-04-20T11:32:26.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.024+0000 7fdbfcc00640 1 --2- 172.21.15.152:0/3725468611 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fdbdc038820 0x7fdbdc03ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:26.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.024+0000 7fdbfcc00640 1 -- 172.21.15.152:0/3725468611 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 6268+0+0 (secure 0 0 0) 0x7fdbf804b5c0 con 0x7fdc001e8fd0 2024-04-20T11:32:26.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.025+0000 7fdbffe00640 1 --2- 172.21.15.152:0/3725468611 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fdbdc038820 0x7fdbdc03ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:26.026 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.026+0000 7fdbffe00640 1 --2- 172.21.15.152:0/3725468611 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fdbdc038820 0x7fdbdc03ace0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fdbf0001560 tx=0x7fdbf0009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:26.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.029+0000 7fdbfcc00640 1 -- 172.21.15.152:0/3725468611 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fdbf8012c60 con 0x7fdc001e8fd0 2024-04-20T11:32:26.117 INFO:tasks.ceph.mgr.y.smithi152.stderr:2024-04-20T11:32:26.117+0000 7fe15ce4b040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-20T11:32:26.181 INFO:tasks.ceph.mgr.y.smithi152.stderr:2024-04-20T11:32:26.180+0000 7fe15ce4b040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-20T11:32:26.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.232+0000 7fdc06e00640 1 -- 172.21.15.152:0/3725468611 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fdc00074d80 con 0x7fdc001e8fd0 2024-04-20T11:32:26.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.232+0000 7fdbfcc00640 1 -- 172.21.15.152:0/3725468611 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v72) ==== 100+0+11 (secure 0 0 0) 0x7fdbf8013760 con 0x7fdc001e8fd0 2024-04-20T11:32:26.233 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 64 2024-04-20T11:32:26.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.235+0000 7fdc06e00640 1 -- 172.21.15.152:0/3725468611 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fdbdc038820 msgr2=0x7fdbdc03ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:26.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.235+0000 7fdc06e00640 1 --2- 172.21.15.152:0/3725468611 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fdbdc038820 0x7fdbdc03ace0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fdbf0001560 tx=0x7fdbf0009040 comp rx=0 tx=0).stop 2024-04-20T11:32:26.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.235+0000 7fdc06e00640 1 -- 172.21.15.152:0/3725468611 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdc001e8fd0 msgr2=0x7fdc001e9460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:26.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.235+0000 7fdc06e00640 1 --2- 172.21.15.152:0/3725468611 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdc001e8fd0 0x7fdc001e9460 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fdbf8000c00 tx=0x7fdbf800e8a0 comp rx=0 tx=0).stop 2024-04-20T11:32:26.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.235+0000 7fdc06e00640 1 -- 172.21.15.152:0/3725468611 shutdown_connections 2024-04-20T11:32:26.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.235+0000 7fdc06e00640 1 --2- 172.21.15.152:0/3725468611 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fdbdc038820 0x7fdbdc03ace0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:26.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.235+0000 7fdc06e00640 1 --2- 172.21.15.152:0/3725468611 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdc001e8fd0 0x7fdc001e9460 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:26.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.235+0000 7fdc06e00640 1 --2- 172.21.15.152:0/3725468611 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdc001616a0 0x7fdc000b7e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:26.237 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.235+0000 7fdc06e00640 1 --2- 172.21.15.152:0/3725468611 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdbe40043d0 0x7fdc0015fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:26.237 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.235+0000 7fdc06e00640 1 -- 172.21.15.152:0/3725468611 >> 172.21.15.152:0/3725468611 conn(0x7fdc0009dab0 msgr2=0x7fdc00162d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:26.237 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.236+0000 7fdc06e00640 1 -- 172.21.15.152:0/3725468611 shutdown_connections 2024-04-20T11:32:26.237 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.236+0000 7fdc06e00640 1 -- 172.21.15.152:0/3725468611 wait complete. 2024-04-20T11:32:26.251 INFO:tasks.ceph.mgr.y.smithi152.stderr:2024-04-20T11:32:26.251+0000 7fe15ce4b040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-20T11:32:26.256 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:32:26.307 INFO:tasks.ceph.mgr.y.smithi152.stderr:2024-04-20T11:32:26.306+0000 7fe15ce4b040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-20T11:32:26.443 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.442+0000 7f3ef3000640 1 Processor -- start 2024-04-20T11:32:26.443 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.442+0000 7f3ef3000640 1 -- start start 2024-04-20T11:32:26.445 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.444+0000 7f3ef3000640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f3eec0b4660 0x7f3eec06bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:26.445 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.444+0000 7f3ef3000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f3eec1601b0 0x7f3eec160640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:26.445 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.444+0000 7f3ef3000640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3eec06c2a0 con 0x7f3eec06b980 2024-04-20T11:32:26.445 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.444+0000 7f3ef3000640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f3eec0b4aa0 con 0x7f3eec1601b0 2024-04-20T11:32:26.446 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.444+0000 7f3ef3000640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f3eec06f5d0 con 0x7f3eec0b4660 2024-04-20T11:32:26.446 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.444+0000 7f3ef1c00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f3eec06b980 0x7f3eec06ee10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:33650/0 (socket says 172.21.15.152:33650) 2024-04-20T11:32:26.446 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.444+0000 7f3ef1c00640 1 -- 172.21.15.152:0/945786640 learned_addr learned my addr 172.21.15.152:0/945786640 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:32:26.446 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.444+0000 7f3ef2600640 1 --2- 172.21.15.152:0/945786640 >> v2:172.21.15.152:3300/0 conn(0x7f3eec1601b0 0x7f3eec160640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:26.446 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.445+0000 7f3ef1200640 1 --2- 172.21.15.152:0/945786640 >> v2:172.21.15.175:3300/0 conn(0x7f3eec0b4660 0x7f3eec06bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:26.446 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.445+0000 7f3ef2600640 1 -- 172.21.15.152:0/945786640 >> v2:172.21.15.175:3300/0 conn(0x7f3eec0b4660 msgr2=0x7f3eec06bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:26.446 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.445+0000 7f3ef2600640 1 --2- 172.21.15.152:0/945786640 >> v2:172.21.15.175:3300/0 conn(0x7f3eec0b4660 0x7f3eec06bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:26.446 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.445+0000 7f3ef2600640 1 -- 172.21.15.152:0/945786640 >> v1:172.21.15.152:6789/0 conn(0x7f3eec06b980 legacy=0x7f3eec06ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:26.446 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.445+0000 7f3ef2600640 1 -- 172.21.15.152:0/945786640 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3eec06f8b0 con 0x7f3eec1601b0 2024-04-20T11:32:26.446 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.445+0000 7f3eebe00640 1 -- 172.21.15.152:0/945786640 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2464405961 0 0) 0x7f3eec06c2a0 con 0x7f3eec06b980 2024-04-20T11:32:26.448 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.446+0000 7f3ef2600640 1 --2- 172.21.15.152:0/945786640 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3eec1601b0 0x7f3eec160640 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f3eec0b4060 tx=0x7f3ee402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6d978b743a5e65b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:26.448 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.446+0000 7f3eebe00640 1 -- 172.21.15.152:0/945786640 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee4039070 con 0x7f3eec1601b0 2024-04-20T11:32:26.448 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.446+0000 7f3eebe00640 1 -- 172.21.15.152:0/945786640 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ee402cc80 con 0x7f3eec1601b0 2024-04-20T11:32:26.448 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.446+0000 7f3eebe00640 1 -- 172.21.15.152:0/945786640 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee4034660 con 0x7f3eec1601b0 2024-04-20T11:32:26.448 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.447+0000 7f3ef3000640 1 -- 172.21.15.152:0/945786640 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3eec1601b0 msgr2=0x7f3eec160640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:26.448 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.447+0000 7f3ef3000640 1 --2- 172.21.15.152:0/945786640 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3eec1601b0 0x7f3eec160640 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f3eec0b4060 tx=0x7f3ee402bed0 comp rx=0 tx=0).stop 2024-04-20T11:32:26.448 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.447+0000 7f3ef3000640 1 -- 172.21.15.152:0/945786640 shutdown_connections 2024-04-20T11:32:26.448 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.447+0000 7f3ef3000640 1 --2- 172.21.15.152:0/945786640 >> v2:172.21.15.175:3300/0 conn(0x7f3eec0b4660 0x7f3eec06bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:26.448 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.447+0000 7f3ef3000640 1 --2- 172.21.15.152:0/945786640 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3eec1601b0 0x7f3eec160640 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:26.448 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.447+0000 7f3ef3000640 1 -- 172.21.15.152:0/945786640 >> 172.21.15.152:0/945786640 conn(0x7f3eec0732b0 msgr2=0x7f3eec09aa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:26.449 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.447+0000 7f3ef3000640 1 -- 172.21.15.152:0/945786640 shutdown_connections 2024-04-20T11:32:26.449 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.447+0000 7f3ef3000640 1 -- 172.21.15.152:0/945786640 wait complete. 2024-04-20T11:32:26.449 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.448+0000 7f3ef3000640 1 Processor -- start 2024-04-20T11:32:26.449 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.449+0000 7f3ef3000640 1 -- start start 2024-04-20T11:32:26.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.450+0000 7f3ef3000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3eec06b980 0x7f3eec09c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:26.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.450+0000 7f3ef3000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3eec0b4660 0x7f3eec136ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:26.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.450+0000 7f3ef3000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3eec1601b0 0x7f3eec1370e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:26.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.450+0000 7f3ef3000640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f3eec070150 con 0x7f3eec1601b0 2024-04-20T11:32:26.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.450+0000 7f3ef3000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3eec06fe70 con 0x7f3eec06b980 2024-04-20T11:32:26.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.450+0000 7f3ef3000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f3eec162000 con 0x7f3eec0b4660 2024-04-20T11:32:26.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.450+0000 7f3ef2600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3eec1601b0 0x7f3eec1370e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:26.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.450+0000 7f3ef2600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3eec1601b0 0x7f3eec1370e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:60084/0 (socket says 172.21.15.152:60084) 2024-04-20T11:32:26.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.450+0000 7f3ef2600640 1 -- 172.21.15.152:0/2339366851 learned_addr learned my addr 172.21.15.152:0/2339366851 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:32:26.452 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.450+0000 7f3ef2600640 1 -- 172.21.15.152:0/2339366851 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3eec0b4660 msgr2=0x7f3eec136ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:26.452 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.450+0000 7f3ef2600640 1 --2- 172.21.15.152:0/2339366851 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3eec0b4660 0x7f3eec136ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:26.452 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.450+0000 7f3ef2600640 1 -- 172.21.15.152:0/2339366851 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3eec06b980 msgr2=0x7f3eec09c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:26.452 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.450+0000 7f3ef2600640 1 --2- 172.21.15.152:0/2339366851 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3eec06b980 0x7f3eec09c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:26.452 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.450+0000 7f3ef2600640 1 -- 172.21.15.152:0/2339366851 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3eec09d150 con 0x7f3eec1601b0 2024-04-20T11:32:26.452 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.451+0000 7f3ef2600640 1 --2- 172.21.15.152:0/2339366851 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3eec1601b0 0x7f3eec1370e0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f3ee402c4e0 tx=0x7f3ee4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:26.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.451+0000 7f3ee9600640 1 -- 172.21.15.152:0/2339366851 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee4044030 con 0x7f3eec1601b0 2024-04-20T11:32:26.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.451+0000 7f3ee9600640 1 -- 172.21.15.152:0/2339366851 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ee402ca80 con 0x7f3eec1601b0 2024-04-20T11:32:26.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.451+0000 7f3ee9600640 1 -- 172.21.15.152:0/2339366851 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee4039040 con 0x7f3eec1601b0 2024-04-20T11:32:26.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.451+0000 7f3ef3000640 1 -- 172.21.15.152:0/2339366851 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3eec09d430 con 0x7f3eec1601b0 2024-04-20T11:32:26.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.451+0000 7f3ef3000640 1 -- 172.21.15.152:0/2339366851 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3eec09d9f0 con 0x7f3eec1601b0 2024-04-20T11:32:26.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.452+0000 7f3ef3000640 1 -- 172.21.15.152:0/2339366851 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3eec075e60 con 0x7f3eec1601b0 2024-04-20T11:32:26.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.452+0000 7f3ee9600640 1 -- 172.21.15.152:0/2339366851 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 15) ==== 45936+0+0 (secure 0 0 0) 0x7f3ee403f470 con 0x7f3eec1601b0 2024-04-20T11:32:26.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.453+0000 7f3ee9600640 1 --2- 172.21.15.152:0/2339366851 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f3ecc0387f0 0x7f3ecc03acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:26.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.453+0000 7f3ee9600640 1 -- 172.21.15.152:0/2339366851 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 6268+0+0 (secure 0 0 0) 0x7f3ee406ecc0 con 0x7f3eec1601b0 2024-04-20T11:32:26.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.453+0000 7f3ef1c00640 1 --2- 172.21.15.152:0/2339366851 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f3ecc0387f0 0x7f3ecc03acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:26.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.455+0000 7f3ef1c00640 1 --2- 172.21.15.152:0/2339366851 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f3ecc0387f0 0x7f3ecc03acb0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3ee0007fd0 tx=0x7f3ee0007920 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:26.462 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.462+0000 7f3ee9600640 1 -- 172.21.15.152:0/2339366851 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f3ee4048ce0 con 0x7f3eec1601b0 2024-04-20T11:32:26.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.685+0000 7f3ef3000640 1 -- 172.21.15.152:0/2339366851 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3eec15f5b0 con 0x7f3eec1601b0 2024-04-20T11:32:26.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.686+0000 7f3ee9600640 1 -- 172.21.15.152:0/2339366851 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+5210 (secure 0 0 0) 0x7f3ee4049460 con 0x7f3eec1601b0 2024-04-20T11:32:26.687 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:32:26.687 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5234,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6836/359391273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6835","nonce":359391273},{"type":"v1","addr":"172.21.15.152:6836","nonce":359391273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":5237,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/4269836436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":4269836436},{"type":"v1","addr":"172.21.15.152:6837","nonce":4269836436}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":5611,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6834/4153485754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":4153485754},{"type":"v1","addr":"172.21.15.175:6834","nonce":4153485754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T11:31:49.738765+0000","modified":"2024-04-20T11:31:52.857955+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5614},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5614":{"gid":5614,"name":"b","rank":0,"incarnation":39,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6835/2057501014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6833","nonce":2057501014},{"type":"v1","addr":"172.21.15.175:6835","nonce":2057501014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5614,"qdb_cluster":[5614]},"id":5}]} 2024-04-20T11:32:26.687 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 44 2024-04-20T11:32:26.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.689+0000 7f3ef3000640 1 -- 172.21.15.152:0/2339366851 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f3ecc0387f0 msgr2=0x7f3ecc03acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:26.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.689+0000 7f3ef3000640 1 --2- 172.21.15.152:0/2339366851 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f3ecc0387f0 0x7f3ecc03acb0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3ee0007fd0 tx=0x7f3ee0007920 comp rx=0 tx=0).stop 2024-04-20T11:32:26.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.689+0000 7f3ef3000640 1 -- 172.21.15.152:0/2339366851 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3eec1601b0 msgr2=0x7f3eec1370e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:26.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.689+0000 7f3ef3000640 1 --2- 172.21.15.152:0/2339366851 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3eec1601b0 0x7f3eec1370e0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f3ee402c4e0 tx=0x7f3ee4004a40 comp rx=0 tx=0).stop 2024-04-20T11:32:26.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.689+0000 7f3ef3000640 1 -- 172.21.15.152:0/2339366851 shutdown_connections 2024-04-20T11:32:26.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.689+0000 7f3ef3000640 1 --2- 172.21.15.152:0/2339366851 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f3ecc0387f0 0x7f3ecc03acb0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:26.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.689+0000 7f3ef3000640 1 --2- 172.21.15.152:0/2339366851 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3eec1601b0 0x7f3eec1370e0 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:26.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.689+0000 7f3ef3000640 1 --2- 172.21.15.152:0/2339366851 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3eec0b4660 0x7f3eec136ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:26.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.689+0000 7f3ef3000640 1 --2- 172.21.15.152:0/2339366851 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3eec06b980 0x7f3eec09c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:26.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.689+0000 7f3ef3000640 1 -- 172.21.15.152:0/2339366851 >> 172.21.15.152:0/2339366851 conn(0x7f3eec0732b0 msgr2=0x7f3eec0b1130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:26.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.689+0000 7f3ef3000640 1 -- 172.21.15.152:0/2339366851 shutdown_connections 2024-04-20T11:32:26.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.689+0000 7f3ef3000640 1 -- 172.21.15.152:0/2339366851 wait complete. 2024-04-20T11:32:26.707 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:32:26.774 INFO:tasks.ceph.mgr.y.smithi152.stderr:2024-04-20T11:32:26.773+0000 7fe15ce4b040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-20T11:32:26.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.898+0000 7f6d53a00640 1 Processor -- start 2024-04-20T11:32:26.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.898+0000 7f6d53a00640 1 -- start start 2024-04-20T11:32:26.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.899+0000 7f6d53a00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f6d4c0d12d0 0x7f6d4c09b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:26.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.900+0000 7f6d53a00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d4c09b710 con 0x7f6d4c0b4be0 2024-04-20T11:32:26.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.900+0000 7f6d53a00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d4c06d050 con 0x7f6d4c15edc0 2024-04-20T11:32:26.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.900+0000 7f6d53a00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f6d4c15f1a0 con 0x7f6d4c0d12d0 2024-04-20T11:32:26.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.900+0000 7f6d51a00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f6d4c0b4be0 0x7f6d4c0ccc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:33654/0 (socket says 172.21.15.152:33654) 2024-04-20T11:32:26.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.900+0000 7f6d51a00640 1 -- 172.21.15.152:0/2482120786 learned_addr learned my addr 172.21.15.152:0/2482120786 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:32:26.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.900+0000 7f6d4be00640 1 --2- 172.21.15.152:0/2482120786 >> v2:172.21.15.175:3300/0 conn(0x7f6d4c0d12d0 0x7f6d4c09b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:26.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.900+0000 7f6d4b400640 1 -- 172.21.15.152:0/2482120786 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1258876848 0 0) 0x7f6d4c09b710 con 0x7f6d4c0b4be0 2024-04-20T11:32:26.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.900+0000 7f6d4b400640 1 -- 172.21.15.152:0/2482120786 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6d34003520 con 0x7f6d4c0b4be0 2024-04-20T11:32:26.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.901+0000 7f6d4b400640 1 -- 172.21.15.152:0/2482120786 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1484680577 0 0) 0x7f6d34003520 con 0x7f6d4c0b4be0 2024-04-20T11:32:26.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.901+0000 7f6d4b400640 1 -- 172.21.15.152:0/2482120786 >> v2:172.21.15.175:3300/0 conn(0x7f6d4c0d12d0 msgr2=0x7f6d4c09b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:26.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.901+0000 7f6d4b400640 1 --2- 172.21.15.152:0/2482120786 >> v2:172.21.15.175:3300/0 conn(0x7f6d4c0d12d0 0x7f6d4c09b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:26.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.901+0000 7f6d4b400640 1 -- 172.21.15.152:0/2482120786 >> v1:172.21.15.175:6789/0 conn(0x7f6d4c15edc0 legacy=0x7f6d4c0d0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:26.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.901+0000 7f6d4b400640 1 -- 172.21.15.152:0/2482120786 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d4c06d2f0 con 0x7f6d4c0b4be0 2024-04-20T11:32:26.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.901+0000 7f6d4b400640 1 -- 172.21.15.152:0/2482120786 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f6d440019f0 con 0x7f6d4c0b4be0 2024-04-20T11:32:26.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.901+0000 7f6d4b400640 1 -- 172.21.15.152:0/2482120786 >> v1:172.21.15.152:6789/0 conn(0x7f6d4c0b4be0 legacy=0x7f6d4c0ccc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:26.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.901+0000 7f6d4b400640 1 --2- 172.21.15.152:0/2482120786 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6d34003a70 0x7f6d34003e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:26.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.901+0000 7f6d4b400640 1 --2- 172.21.15.152:0/2482120786 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6d34004800 0x7f6d34004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:26.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.901+0000 7f6d4b400640 1 --2- 172.21.15.152:0/2482120786 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d34005210 0x7f6d3400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:26.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.901+0000 7f6d4b400640 1 -- 172.21.15.152:0/2482120786 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f6d4c15f1a0 con 0x7f6d34003a70 2024-04-20T11:32:26.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.901+0000 7f6d4b400640 1 -- 172.21.15.152:0/2482120786 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6d3400c140 con 0x7f6d34005210 2024-04-20T11:32:26.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.901+0000 7f6d4b400640 1 -- 172.21.15.152:0/2482120786 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6d3400c340 con 0x7f6d34004800 2024-04-20T11:32:26.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.901+0000 7f6d51000640 1 --2- 172.21.15.152:0/2482120786 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6d34003a70 0x7f6d34003e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:26.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.902+0000 7f6d51a00640 1 --2- 172.21.15.152:0/2482120786 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d34005210 0x7f6d3400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:26.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.902+0000 7f6d4be00640 1 --2- 172.21.15.152:0/2482120786 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6d34004800 0x7f6d34004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:26.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.902+0000 7f6d51000640 1 -- 172.21.15.152:0/2482120786 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6d34004800 msgr2=0x7f6d34004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:26.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.902+0000 7f6d51000640 1 --2- 172.21.15.152:0/2482120786 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6d34004800 0x7f6d34004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:26.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.902+0000 7f6d51000640 1 -- 172.21.15.152:0/2482120786 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d34005210 msgr2=0x7f6d3400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:26.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.902+0000 7f6d51000640 1 --2- 172.21.15.152:0/2482120786 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d34005210 0x7f6d3400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:26.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.902+0000 7f6d51000640 1 -- 172.21.15.152:0/2482120786 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d34003520 con 0x7f6d34003a70 2024-04-20T11:32:26.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.902+0000 7f6d51a00640 1 -- 172.21.15.152:0/2482120786 reap_dead start 2024-04-20T11:32:26.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.903+0000 7f6d51000640 1 --2- 172.21.15.152:0/2482120786 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6d34003a70 0x7f6d34003e90 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f6d40001750 tx=0x7f6d34005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2baf7c56ed5b2642 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:26.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.903+0000 7f6d4b400640 1 -- 172.21.15.152:0/2482120786 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d40001370 con 0x7f6d34003a70 2024-04-20T11:32:26.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.903+0000 7f6d4b400640 1 -- 172.21.15.152:0/2482120786 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d400052d0 con 0x7f6d34003a70 2024-04-20T11:32:26.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.903+0000 7f6d4b400640 1 -- 172.21.15.152:0/2482120786 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d40003830 con 0x7f6d34003a70 2024-04-20T11:32:26.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.904+0000 7f6d53a00640 1 -- 172.21.15.152:0/2482120786 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6d34003a70 msgr2=0x7f6d34003e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:26.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.904+0000 7f6d53a00640 1 --2- 172.21.15.152:0/2482120786 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6d34003a70 0x7f6d34003e90 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f6d40001750 tx=0x7f6d34005830 comp rx=0 tx=0).stop 2024-04-20T11:32:26.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.904+0000 7f6d53a00640 1 -- 172.21.15.152:0/2482120786 shutdown_connections 2024-04-20T11:32:26.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.904+0000 7f6d53a00640 1 --2- 172.21.15.152:0/2482120786 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6d34003a70 0x7f6d34003e90 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:26.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.904+0000 7f6d53a00640 1 -- 172.21.15.152:0/2482120786 >> 172.21.15.152:0/2482120786 conn(0x7f6d4c0b40c0 msgr2=0x7f6d4c0b44d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:26.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.904+0000 7f6d53a00640 1 -- 172.21.15.152:0/2482120786 shutdown_connections 2024-04-20T11:32:26.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.904+0000 7f6d53a00640 1 -- 172.21.15.152:0/2482120786 wait complete. 2024-04-20T11:32:26.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.906+0000 7f6d53a00640 1 Processor -- start 2024-04-20T11:32:26.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.906+0000 7f6d53a00640 1 -- start start 2024-04-20T11:32:26.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.907+0000 7f6d53a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6d34003a70 0x7f6d4c158680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:26.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.907+0000 7f6d53a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d4c0cdd80 0x7f6d4c0ce200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:26.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.907+0000 7f6d53a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6d4c0cb5d0 0x7f6d4c0cba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:26.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.907+0000 7f6d53a00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f6d4c06db90 con 0x7f6d34003a70 2024-04-20T11:32:26.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.907+0000 7f6d53a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6d4c06d8b0 con 0x7f6d4c0cdd80 2024-04-20T11:32:26.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.907+0000 7f6d53a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6d4c06de70 con 0x7f6d4c0cb5d0 2024-04-20T11:32:26.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.907+0000 7f6d51000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6d34003a70 0x7f6d4c158680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:26.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.907+0000 7f6d51000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6d34003a70 0x7f6d4c158680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:60102/0 (socket says 172.21.15.152:60102) 2024-04-20T11:32:26.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.907+0000 7f6d51000640 1 -- 172.21.15.152:0/3648332460 learned_addr learned my addr 172.21.15.152:0/3648332460 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:32:26.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.907+0000 7f6d51a00640 1 --2- 172.21.15.152:0/3648332460 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6d4c0cb5d0 0x7f6d4c0cba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:26.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.907+0000 7f6d4be00640 1 --2- 172.21.15.152:0/3648332460 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d4c0cdd80 0x7f6d4c0ce200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:26.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.907+0000 7f6d51000640 1 -- 172.21.15.152:0/3648332460 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6d4c0cb5d0 msgr2=0x7f6d4c0cba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:26.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.907+0000 7f6d51000640 1 --2- 172.21.15.152:0/3648332460 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6d4c0cb5d0 0x7f6d4c0cba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:26.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.907+0000 7f6d51000640 1 -- 172.21.15.152:0/3648332460 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d4c0cdd80 msgr2=0x7f6d4c0ce200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:26.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.907+0000 7f6d51000640 1 --2- 172.21.15.152:0/3648332460 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d4c0cdd80 0x7f6d4c0ce200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:26.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.907+0000 7f6d51000640 1 -- 172.21.15.152:0/3648332460 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d4c1f8ca0 con 0x7f6d34003a70 2024-04-20T11:32:26.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.908+0000 7f6d51000640 1 --2- 172.21.15.152:0/3648332460 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6d34003a70 0x7f6d4c158680 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f6d40005040 tx=0x7f6d40001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:26.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.908+0000 7f6d48c00640 1 -- 172.21.15.152:0/3648332460 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d40041070 con 0x7f6d34003a70 2024-04-20T11:32:26.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.908+0000 7f6d48c00640 1 -- 172.21.15.152:0/3648332460 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d40005480 con 0x7f6d34003a70 2024-04-20T11:32:26.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.908+0000 7f6d48c00640 1 -- 172.21.15.152:0/3648332460 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d40007a30 con 0x7f6d34003a70 2024-04-20T11:32:26.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.908+0000 7f6d53a00640 1 -- 172.21.15.152:0/3648332460 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d4c1f7ee0 con 0x7f6d34003a70 2024-04-20T11:32:26.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.908+0000 7f6d53a00640 1 -- 172.21.15.152:0/3648332460 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d4c1f84a0 con 0x7f6d34003a70 2024-04-20T11:32:26.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.909+0000 7f6d53a00640 1 -- 172.21.15.152:0/3648332460 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d4c0689f0 con 0x7f6d34003a70 2024-04-20T11:32:26.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.910+0000 7f6d48c00640 1 -- 172.21.15.152:0/3648332460 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 15) ==== 45936+0+0 (secure 0 0 0) 0x7f6d40044070 con 0x7f6d34003a70 2024-04-20T11:32:26.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.910+0000 7f6d48c00640 1 --2- 172.21.15.152:0/3648332460 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f6d2c0387f0 0x7f6d2c03acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:26.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.910+0000 7f6d48c00640 1 -- 172.21.15.152:0/3648332460 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 6268+0+0 (secure 0 0 0) 0x7f6d4006df50 con 0x7f6d34003a70 2024-04-20T11:32:26.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.911+0000 7f6d4be00640 1 --2- 172.21.15.152:0/3648332460 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f6d2c0387f0 0x7f6d2c03acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:26.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.912+0000 7f6d4be00640 1 --2- 172.21.15.152:0/3648332460 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f6d2c0387f0 0x7f6d2c03acb0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c159a80 tx=0x7f6d38008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:26.914 INFO:tasks.ceph.mgr.y.smithi152.stderr:2024-04-20T11:32:26.914+0000 7fe15ce4b040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-20T11:32:26.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:26.915+0000 7f6d48c00640 1 -- 172.21.15.152:0/3648332460 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f6d400036e0 con 0x7f6d34003a70 2024-04-20T11:32:27.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.117+0000 7f6d53a00640 1 -- 172.21.15.152:0/3648332460 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6d4c1597d0 con 0x7f6d34003a70 2024-04-20T11:32:27.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.119+0000 7f6d48c00640 1 -- 172.21.15.152:0/3648332460 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) ==== 74+0+17550 (secure 0 0 0) 0x7f6d400313a0 con 0x7f6d34003a70 2024-04-20T11:32:27.120 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:32:27.120 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":72,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:32:22.542221+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:31:48.024528+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-20T11:31:48.725648+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.152:0/3082578161":"2024-04-21T11:32:22.542176+0000","172.21.15.152:0/1524004838":"2024-04-21T11:32:22.542176+0000","172.21.15.152:6833/2285110240":"2024-04-21T11:32:22.542176+0000","172.21.15.152:6832/2285110240":"2024-04-21T11:32:22.542176+0000","172.21.15.152:0/698943652":"2024-04-21T11:32:22.542176+0000","172.21.15.152:0/142101408":"2024-04-21T11:32:22.542176+0000","172.21.15.152:0/2510881178":"2024-04-21T11:32:22.542176+0000","172.21.15.152:0/3263977099":"2024-04-21T11:32:22.542176+0000","172.21.15.152:0/2184739958":"2024-04-21T11:32:22.542176+0000","172.21.15.152:0/3331653721":"2024-04-21T11:32:22.542176+0000","172.21.15.152:0/2405473735":"2024-04-21T11:32:22.542176+0000","172.21.15.152:0/2828322453":"2024-04-21T11:32:22.542176+0000","172.21.15.152:0/3593059376":"2024-04-21T11:32:22.542176+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:32:27.123 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.122+0000 7f6d53a00640 1 -- 172.21.15.152:0/3648332460 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f6d2c0387f0 msgr2=0x7f6d2c03acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:27.123 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.122+0000 7f6d53a00640 1 --2- 172.21.15.152:0/3648332460 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f6d2c0387f0 0x7f6d2c03acb0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c159a80 tx=0x7f6d38008040 comp rx=0 tx=0).stop 2024-04-20T11:32:27.123 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.122+0000 7f6d53a00640 1 -- 172.21.15.152:0/3648332460 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6d34003a70 msgr2=0x7f6d4c158680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:27.123 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.122+0000 7f6d53a00640 1 --2- 172.21.15.152:0/3648332460 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6d34003a70 0x7f6d4c158680 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f6d40005040 tx=0x7f6d40001120 comp rx=0 tx=0).stop 2024-04-20T11:32:27.123 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.122+0000 7f6d53a00640 1 -- 172.21.15.152:0/3648332460 shutdown_connections 2024-04-20T11:32:27.123 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.122+0000 7f6d53a00640 1 --2- 172.21.15.152:0/3648332460 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f6d2c0387f0 0x7f6d2c03acb0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:27.123 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.122+0000 7f6d53a00640 1 --2- 172.21.15.152:0/3648332460 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6d4c0cb5d0 0x7f6d4c0cba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:27.123 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.122+0000 7f6d53a00640 1 --2- 172.21.15.152:0/3648332460 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d4c0cdd80 0x7f6d4c0ce200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:27.123 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.122+0000 7f6d53a00640 1 --2- 172.21.15.152:0/3648332460 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6d34003a70 0x7f6d4c158680 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:27.123 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.122+0000 7f6d53a00640 1 -- 172.21.15.152:0/3648332460 >> 172.21.15.152:0/3648332460 conn(0x7f6d4c0b40c0 msgr2=0x7f6d4c0d04d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:27.123 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.122+0000 7f6d53a00640 1 -- 172.21.15.152:0/3648332460 shutdown_connections 2024-04-20T11:32:27.123 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.122+0000 7f6d53a00640 1 -- 172.21.15.152:0/3648332460 wait complete. 2024-04-20T11:32:27.142 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-20T11:32:27.142 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:32:27.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.344+0000 7ff279400640 1 Processor -- start 2024-04-20T11:32:27.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.344+0000 7ff279400640 1 -- start start 2024-04-20T11:32:27.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.346+0000 7ff279400640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ff27406f510 0x7ff274140dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:27.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.346+0000 7ff279400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7ff274141310 0x7ff274071cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:27.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.346+0000 7ff279400640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2740cf0a0 con 0x7ff27413e530 2024-04-20T11:32:27.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.346+0000 7ff279400640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7ff274141b00 con 0x7ff274141310 2024-04-20T11:32:27.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.346+0000 7ff279400640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7ff2740cfc50 con 0x7ff27406f510 2024-04-20T11:32:27.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.346+0000 7ff278a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7ff274141310 0x7ff274071cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:27.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.346+0000 7ff273e00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7ff27413e530 0x7ff27406ee00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:33658/0 (socket says 172.21.15.152:33658) 2024-04-20T11:32:27.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.346+0000 7ff273e00640 1 -- 172.21.15.152:0/3955736214 learned_addr learned my addr 172.21.15.152:0/3955736214 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:32:27.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.346+0000 7ff278a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7ff274141310 0x7ff274071cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:60118/0 (socket says 172.21.15.152:60118) 2024-04-20T11:32:27.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.346+0000 7ff273400640 1 --2- 172.21.15.152:0/3955736214 >> v2:172.21.15.175:3300/0 conn(0x7ff27406f510 0x7ff274140dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:27.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.347+0000 7ff278a00640 1 -- 172.21.15.152:0/3955736214 >> v2:172.21.15.175:3300/0 conn(0x7ff27406f510 msgr2=0x7ff274140dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:27.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.347+0000 7ff278a00640 1 --2- 172.21.15.152:0/3955736214 >> v2:172.21.15.175:3300/0 conn(0x7ff27406f510 0x7ff274140dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:27.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.347+0000 7ff278a00640 1 -- 172.21.15.152:0/3955736214 >> v1:172.21.15.152:6789/0 conn(0x7ff27413e530 legacy=0x7ff27406ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:27.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.347+0000 7ff278a00640 1 -- 172.21.15.152:0/3955736214 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2740d00e0 con 0x7ff274141310 2024-04-20T11:32:27.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.347+0000 7ff272a00640 1 -- 172.21.15.152:0/3955736214 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2951324091 0 0) 0x7ff264001110 con 0x7ff27413e530 2024-04-20T11:32:27.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.348+0000 7ff278a00640 1 --2- 172.21.15.152:0/3955736214 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff274141310 0x7ff274071cd0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7ff27406feb0 tx=0x7ff26c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1c0abe6d9ab573c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:27.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.348+0000 7ff272a00640 1 -- 172.21.15.152:0/3955736214 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff26c039070 con 0x7ff274141310 2024-04-20T11:32:27.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.348+0000 7ff272a00640 1 -- 172.21.15.152:0/3955736214 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff26c02cc80 con 0x7ff274141310 2024-04-20T11:32:27.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.348+0000 7ff272a00640 1 -- 172.21.15.152:0/3955736214 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff26c034660 con 0x7ff274141310 2024-04-20T11:32:27.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.349+0000 7ff279400640 1 -- 172.21.15.152:0/3955736214 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff274141310 msgr2=0x7ff274071cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:27.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.349+0000 7ff279400640 1 --2- 172.21.15.152:0/3955736214 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff274141310 0x7ff274071cd0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7ff27406feb0 tx=0x7ff26c02bed0 comp rx=0 tx=0).stop 2024-04-20T11:32:27.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.349+0000 7ff279400640 1 -- 172.21.15.152:0/3955736214 shutdown_connections 2024-04-20T11:32:27.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.349+0000 7ff279400640 1 --2- 172.21.15.152:0/3955736214 >> v2:172.21.15.175:3300/0 conn(0x7ff27406f510 0x7ff274140dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:27.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.349+0000 7ff279400640 1 --2- 172.21.15.152:0/3955736214 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff274141310 0x7ff274071cd0 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:27.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.349+0000 7ff279400640 1 -- 172.21.15.152:0/3955736214 >> 172.21.15.152:0/3955736214 conn(0x7ff274067ae0 msgr2=0x7ff27406b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:27.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.349+0000 7ff279400640 1 -- 172.21.15.152:0/3955736214 shutdown_connections 2024-04-20T11:32:27.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.349+0000 7ff279400640 1 -- 172.21.15.152:0/3955736214 wait complete. 2024-04-20T11:32:27.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.351+0000 7ff279400640 1 Processor -- start 2024-04-20T11:32:27.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.351+0000 7ff279400640 1 -- start start 2024-04-20T11:32:27.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.352+0000 7ff279400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff27406f510 0x7ff27413dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:27.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.352+0000 7ff279400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff27413e530 0x7ff2740ae500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:27.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.352+0000 7ff279400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff274141310 0x7ff2740aea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:27.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.352+0000 7ff279400640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7ff2740cf8e0 con 0x7ff27406f510 2024-04-20T11:32:27.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.352+0000 7ff279400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff2740cf600 con 0x7ff27413e530 2024-04-20T11:32:27.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.352+0000 7ff279400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff2741417a0 con 0x7ff274141310 2024-04-20T11:32:27.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.352+0000 7ff273e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff27406f510 0x7ff27413dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:27.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.352+0000 7ff273e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff27406f510 0x7ff27413dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:60124/0 (socket says 172.21.15.152:60124) 2024-04-20T11:32:27.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.352+0000 7ff273e00640 1 -- 172.21.15.152:0/1021363882 learned_addr learned my addr 172.21.15.152:0/1021363882 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:32:27.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.353+0000 7ff273e00640 1 -- 172.21.15.152:0/1021363882 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff274141310 msgr2=0x7ff2740aea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:27.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.353+0000 7ff273e00640 1 --2- 172.21.15.152:0/1021363882 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff274141310 0x7ff2740aea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:27.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.353+0000 7ff273400640 1 --2- 172.21.15.152:0/1021363882 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff27413e530 0x7ff2740ae500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:27.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.353+0000 7ff273e00640 1 -- 172.21.15.152:0/1021363882 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff27413e530 msgr2=0x7ff2740ae500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:27.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.353+0000 7ff273e00640 1 --2- 172.21.15.152:0/1021363882 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff27413e530 0x7ff2740ae500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:27.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.353+0000 7ff273e00640 1 -- 172.21.15.152:0/1021363882 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff274158090 con 0x7ff27406f510 2024-04-20T11:32:27.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.353+0000 7ff273400640 1 --2- 172.21.15.152:0/1021363882 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff27413e530 0x7ff2740ae500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:32:27.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.353+0000 7ff273e00640 1 --2- 172.21.15.152:0/1021363882 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff27406f510 0x7ff27413dc10 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7ff26400c780 tx=0x7ff26400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:27.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.353+0000 7ff26be00640 1 -- 172.21.15.152:0/1021363882 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff264018070 con 0x7ff27406f510 2024-04-20T11:32:27.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.353+0000 7ff26be00640 1 -- 172.21.15.152:0/1021363882 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2640072d0 con 0x7ff27406f510 2024-04-20T11:32:27.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.353+0000 7ff279400640 1 -- 172.21.15.152:0/1021363882 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2740b0ae0 con 0x7ff27406f510 2024-04-20T11:32:27.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.353+0000 7ff26be00640 1 -- 172.21.15.152:0/1021363882 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2640047e0 con 0x7ff27406f510 2024-04-20T11:32:27.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.353+0000 7ff279400640 1 -- 172.21.15.152:0/1021363882 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2740b10a0 con 0x7ff27406f510 2024-04-20T11:32:27.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.354+0000 7ff26be00640 1 -- 172.21.15.152:0/1021363882 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 15) ==== 45936+0+0 (secure 0 0 0) 0x7ff26401b070 con 0x7ff27406f510 2024-04-20T11:32:27.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.354+0000 7ff279400640 1 -- 172.21.15.152:0/1021363882 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff274075e60 con 0x7ff27406f510 2024-04-20T11:32:27.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.355+0000 7ff26be00640 1 --2- 172.21.15.152:0/1021363882 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7ff2540387f0 0x7ff25403acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:27.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.355+0000 7ff26be00640 1 -- 172.21.15.152:0/1021363882 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 6268+0+0 (secure 0 0 0) 0x7ff26404b640 con 0x7ff27406f510 2024-04-20T11:32:27.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.355+0000 7ff273400640 1 --2- 172.21.15.152:0/1021363882 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7ff2540387f0 0x7ff25403acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:27.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.357+0000 7ff273400640 1 --2- 172.21.15.152:0/1021363882 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7ff2540387f0 0x7ff25403acb0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff274157cf0 tx=0x7ff260008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:27.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.360+0000 7ff26be00640 1 -- 172.21.15.152:0/1021363882 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ff2740cf0a0 con 0x7ff27406f510 2024-04-20T11:32:27.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.580+0000 7ff279400640 1 -- 172.21.15.152:0/1021363882 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff2740add70 con 0x7ff27406f510 2024-04-20T11:32:27.583 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.582+0000 7ff26be00640 1 -- 172.21.15.152:0/1021363882 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+5210 (secure 0 0 0) 0x7ff264010300 con 0x7ff27406f510 2024-04-20T11:32:27.584 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:32:27.584 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5234,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6836/359391273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6835","nonce":359391273},{"type":"v1","addr":"172.21.15.152:6836","nonce":359391273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":5237,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/4269836436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":4269836436},{"type":"v1","addr":"172.21.15.152:6837","nonce":4269836436}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":5611,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6834/4153485754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":4153485754},{"type":"v1","addr":"172.21.15.175:6834","nonce":4153485754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T11:31:49.738765+0000","modified":"2024-04-20T11:31:52.857955+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5614},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5614":{"gid":5614,"name":"b","rank":0,"incarnation":39,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6835/2057501014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6833","nonce":2057501014},{"type":"v1","addr":"172.21.15.175:6835","nonce":2057501014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5614,"qdb_cluster":[5614]},"id":5}]} 2024-04-20T11:32:27.584 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 44 2024-04-20T11:32:27.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.585+0000 7ff279400640 1 -- 172.21.15.152:0/1021363882 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7ff2540387f0 msgr2=0x7ff25403acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:27.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.585+0000 7ff279400640 1 --2- 172.21.15.152:0/1021363882 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7ff2540387f0 0x7ff25403acb0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff274157cf0 tx=0x7ff260008040 comp rx=0 tx=0).stop 2024-04-20T11:32:27.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.585+0000 7ff279400640 1 -- 172.21.15.152:0/1021363882 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff27406f510 msgr2=0x7ff27413dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:27.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.585+0000 7ff279400640 1 --2- 172.21.15.152:0/1021363882 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff27406f510 0x7ff27413dc10 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7ff26400c780 tx=0x7ff26400cc40 comp rx=0 tx=0).stop 2024-04-20T11:32:27.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.585+0000 7ff279400640 1 -- 172.21.15.152:0/1021363882 shutdown_connections 2024-04-20T11:32:27.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.585+0000 7ff279400640 1 --2- 172.21.15.152:0/1021363882 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7ff2540387f0 0x7ff25403acb0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:27.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.586+0000 7ff279400640 1 --2- 172.21.15.152:0/1021363882 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff274141310 0x7ff2740aea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:27.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.586+0000 7ff279400640 1 --2- 172.21.15.152:0/1021363882 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff27413e530 0x7ff2740ae500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:27.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.586+0000 7ff279400640 1 --2- 172.21.15.152:0/1021363882 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff27406f510 0x7ff27413dc10 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:27.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.586+0000 7ff279400640 1 -- 172.21.15.152:0/1021363882 >> 172.21.15.152:0/1021363882 conn(0x7ff274067ae0 msgr2=0x7ff27406e970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:27.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.586+0000 7ff279400640 1 -- 172.21.15.152:0/1021363882 shutdown_connections 2024-04-20T11:32:27.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.586+0000 7ff279400640 1 -- 172.21.15.152:0/1021363882 wait complete. 2024-04-20T11:32:27.607 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:32:27.794 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.794+0000 7fa6b4a00640 1 Processor -- start 2024-04-20T11:32:27.795 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.794+0000 7fa6b4a00640 1 -- start start 2024-04-20T11:32:27.797 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.796+0000 7fa6b4a00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fa6b015d1e0 0x7fa6b0160b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:27.797 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.796+0000 7fa6b4a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fa6b0071cd0 0x7fa6b0072120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:27.797 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.796+0000 7fa6b4a00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6b0072660 con 0x7fa6b015d5c0 2024-04-20T11:32:27.797 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.796+0000 7fa6b4a00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fa6b00ce8c0 con 0x7fa6b0071cd0 2024-04-20T11:32:27.797 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.796+0000 7fa6b4a00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fa6b01600b0 con 0x7fa6b015d1e0 2024-04-20T11:32:27.797 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.796+0000 7fa6afe00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fa6b0071cd0 0x7fa6b0072120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:27.797 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.796+0000 7fa6afe00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fa6b0071cd0 0x7fa6b0072120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:60126/0 (socket says 172.21.15.152:60126) 2024-04-20T11:32:27.797 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.796+0000 7fa6aea00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fa6b015d5c0 0x7fa6b00cdfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:33662/0 (socket says 172.21.15.152:33662) 2024-04-20T11:32:27.797 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.796+0000 7fa6afe00640 1 -- 172.21.15.152:0/3054267523 learned_addr learned my addr 172.21.15.152:0/3054267523 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:32:27.798 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.796+0000 7fa6af400640 1 --2- 172.21.15.152:0/3054267523 >> v2:172.21.15.175:3300/0 conn(0x7fa6b015d1e0 0x7fa6b0160b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:27.798 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.796+0000 7fa6afe00640 1 -- 172.21.15.152:0/3054267523 >> v2:172.21.15.175:3300/0 conn(0x7fa6b015d1e0 msgr2=0x7fa6b0160b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:27.798 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.796+0000 7fa6afe00640 1 --2- 172.21.15.152:0/3054267523 >> v2:172.21.15.175:3300/0 conn(0x7fa6b015d1e0 0x7fa6b0160b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:27.798 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.796+0000 7fa6afe00640 1 -- 172.21.15.152:0/3054267523 >> v1:172.21.15.152:6789/0 conn(0x7fa6b015d5c0 legacy=0x7fa6b00cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:27.798 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.796+0000 7fa6afe00640 1 -- 172.21.15.152:0/3054267523 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6b0160390 con 0x7fa6b0071cd0 2024-04-20T11:32:27.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.798+0000 7fa6afe00640 1 --2- 172.21.15.152:0/3054267523 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa6b0071cd0 0x7fa6b0072120 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fa6a8009e50 tx=0x7fa6a802bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e82681e2cfa25ad server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:27.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.798+0000 7fa6ae000640 1 -- 172.21.15.152:0/3054267523 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a802cb10 con 0x7fa6b0071cd0 2024-04-20T11:32:27.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.798+0000 7fa6ae000640 1 -- 172.21.15.152:0/3054267523 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6a802ccb0 con 0x7fa6b0071cd0 2024-04-20T11:32:27.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.798+0000 7fa6ae000640 1 -- 172.21.15.152:0/3054267523 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a802ce60 con 0x7fa6b0071cd0 2024-04-20T11:32:27.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.798+0000 7fa6b4a00640 1 -- 172.21.15.152:0/3054267523 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa6b0071cd0 msgr2=0x7fa6b0072120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:27.800 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.798+0000 7fa6b4a00640 1 --2- 172.21.15.152:0/3054267523 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa6b0071cd0 0x7fa6b0072120 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fa6a8009e50 tx=0x7fa6a802bf70 comp rx=0 tx=0).stop 2024-04-20T11:32:27.800 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.799+0000 7fa6b4a00640 1 -- 172.21.15.152:0/3054267523 shutdown_connections 2024-04-20T11:32:27.800 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.799+0000 7fa6b4a00640 1 --2- 172.21.15.152:0/3054267523 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa6b0071cd0 0x7fa6b0072120 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:27.800 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.799+0000 7fa6b4a00640 1 --2- 172.21.15.152:0/3054267523 >> v2:172.21.15.175:3300/0 conn(0x7fa6b015d1e0 0x7fa6b0160b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:27.800 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.799+0000 7fa6b4a00640 1 -- 172.21.15.152:0/3054267523 >> 172.21.15.152:0/3054267523 conn(0x7fa6b0067ae0 msgr2=0x7fa6b006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:27.800 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.799+0000 7fa6b4a00640 1 -- 172.21.15.152:0/3054267523 shutdown_connections 2024-04-20T11:32:27.800 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.799+0000 7fa6b4a00640 1 -- 172.21.15.152:0/3054267523 wait complete. 2024-04-20T11:32:27.801 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.800+0000 7fa6b4a00640 1 Processor -- start 2024-04-20T11:32:27.801 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.801+0000 7fa6b4a00640 1 -- start start 2024-04-20T11:32:27.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.801+0000 7fa6b4a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa6b0071cd0 0x7fa6b0156060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:27.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.802+0000 7fa6b4a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa6b015d1e0 0x7fa6b01f5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:27.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.802+0000 7fa6b4a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6b015d5c0 0x7fa6b01f5f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:27.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.802+0000 7fa6b4a00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b00ccd70 con 0x7fa6b0071cd0 2024-04-20T11:32:27.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.802+0000 7fa6b4a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b00ceba0 con 0x7fa6b015d5c0 2024-04-20T11:32:27.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.802+0000 7fa6b4a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa6b00ccef0 con 0x7fa6b015d1e0 2024-04-20T11:32:27.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.802+0000 7fa6af400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa6b0071cd0 0x7fa6b0156060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:27.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.802+0000 7fa6af400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa6b0071cd0 0x7fa6b0156060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:60142/0 (socket says 172.21.15.152:60142) 2024-04-20T11:32:27.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.802+0000 7fa6af400640 1 -- 172.21.15.152:0/3262940548 learned_addr learned my addr 172.21.15.152:0/3262940548 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:32:27.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.802+0000 7fa6af400640 1 -- 172.21.15.152:0/3262940548 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa6b015d1e0 msgr2=0x7fa6b01f5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:27.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.802+0000 7fa6af400640 1 --2- 172.21.15.152:0/3262940548 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa6b015d1e0 0x7fa6b01f5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:27.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.802+0000 7fa6af400640 1 -- 172.21.15.152:0/3262940548 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6b015d5c0 msgr2=0x7fa6b01f5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:27.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.802+0000 7fa6af400640 1 --2- 172.21.15.152:0/3262940548 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6b015d5c0 0x7fa6b01f5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:27.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.802+0000 7fa6af400640 1 -- 172.21.15.152:0/3262940548 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6b0157da0 con 0x7fa6b0071cd0 2024-04-20T11:32:27.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.802+0000 7fa6af400640 1 --2- 172.21.15.152:0/3262940548 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa6b0071cd0 0x7fa6b0156060 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fa6a000c7f0 tx=0x7fa6a000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:27.805 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.803+0000 7fa6a7400640 1 -- 172.21.15.152:0/3262940548 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a0015070 con 0x7fa6b0071cd0 2024-04-20T11:32:27.805 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.803+0000 7fa6a7400640 1 -- 172.21.15.152:0/3262940548 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6a0004240 con 0x7fa6b0071cd0 2024-04-20T11:32:27.805 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.803+0000 7fa6a7400640 1 -- 172.21.15.152:0/3262940548 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a0007620 con 0x7fa6b0071cd0 2024-04-20T11:32:27.805 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.803+0000 7fa6b4a00640 1 -- 172.21.15.152:0/3262940548 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6b0158070 con 0x7fa6b0071cd0 2024-04-20T11:32:27.805 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.803+0000 7fa6b4a00640 1 -- 172.21.15.152:0/3262940548 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6b01e8870 con 0x7fa6b0071cd0 2024-04-20T11:32:27.805 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.804+0000 7fa6b4a00640 1 -- 172.21.15.152:0/3262940548 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6b0075e60 con 0x7fa6b0071cd0 2024-04-20T11:32:27.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.805+0000 7fa6a7400640 1 -- 172.21.15.152:0/3262940548 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 91142+0+0 (secure 0 0 0) 0x7fa6a00079d0 con 0x7fa6b0071cd0 2024-04-20T11:32:27.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.806+0000 7fa6a7400640 1 --2- 172.21.15.152:0/3262940548 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fa69006ca10 0x7fa69006eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:27.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.806+0000 7fa6a7400640 1 -- 172.21.15.152:0/3262940548 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 6268+0+0 (secure 0 0 0) 0x7fa6a008c560 con 0x7fa6b0071cd0 2024-04-20T11:32:27.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.806+0000 7fa6aea00640 1 --2- 172.21.15.152:0/3262940548 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fa69006ca10 0x7fa69006eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:27.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.808+0000 7fa6aea00640 1 --2- 172.21.15.152:0/3262940548 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fa69006ca10 0x7fa69006eed0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fa69c000f60 tx=0x7fa69c007a00 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:27.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:27.810+0000 7fa6a7400640 1 -- 172.21.15.152:0/3262940548 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fa6b0158070 con 0x7fa6b0071cd0 2024-04-20T11:32:28.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.049+0000 7fa6b4a00640 1 -- 172.21.15.152:0/3262940548 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa6b0072c90 con 0x7fa6b0071cd0 2024-04-20T11:32:28.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.050+0000 7fa6a7400640 1 -- 172.21.15.152:0/3262940548 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+5210 (secure 0 0 0) 0x7fa6a005a4a0 con 0x7fa6b0071cd0 2024-04-20T11:32:28.050 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:32:28.051 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5234,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6836/359391273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6835","nonce":359391273},{"type":"v1","addr":"172.21.15.152:6836","nonce":359391273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":5237,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/4269836436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":4269836436},{"type":"v1","addr":"172.21.15.152:6837","nonce":4269836436}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":5611,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6834/4153485754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":4153485754},{"type":"v1","addr":"172.21.15.175:6834","nonce":4153485754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T11:31:49.738765+0000","modified":"2024-04-20T11:31:52.857955+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5614},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5614":{"gid":5614,"name":"b","rank":0,"incarnation":39,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6835/2057501014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6833","nonce":2057501014},{"type":"v1","addr":"172.21.15.175:6835","nonce":2057501014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5614,"qdb_cluster":[5614]},"id":5}]} 2024-04-20T11:32:28.051 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 44 2024-04-20T11:32:28.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.052+0000 7fa6b4a00640 1 -- 172.21.15.152:0/3262940548 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fa69006ca10 msgr2=0x7fa69006eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:28.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.052+0000 7fa6b4a00640 1 --2- 172.21.15.152:0/3262940548 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fa69006ca10 0x7fa69006eed0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fa69c000f60 tx=0x7fa69c007a00 comp rx=0 tx=0).stop 2024-04-20T11:32:28.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.052+0000 7fa6b4a00640 1 -- 172.21.15.152:0/3262940548 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa6b0071cd0 msgr2=0x7fa6b0156060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:28.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.052+0000 7fa6b4a00640 1 --2- 172.21.15.152:0/3262940548 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa6b0071cd0 0x7fa6b0156060 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fa6a000c7f0 tx=0x7fa6a000ccb0 comp rx=0 tx=0).stop 2024-04-20T11:32:28.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.052+0000 7fa6b4a00640 1 -- 172.21.15.152:0/3262940548 shutdown_connections 2024-04-20T11:32:28.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.052+0000 7fa6b4a00640 1 --2- 172.21.15.152:0/3262940548 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fa69006ca10 0x7fa69006eed0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:28.054 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.052+0000 7fa6b4a00640 1 --2- 172.21.15.152:0/3262940548 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6b015d5c0 0x7fa6b01f5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:28.054 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.052+0000 7fa6b4a00640 1 --2- 172.21.15.152:0/3262940548 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa6b015d1e0 0x7fa6b01f5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:28.054 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.052+0000 7fa6b4a00640 1 --2- 172.21.15.152:0/3262940548 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa6b0071cd0 0x7fa6b0156060 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:28.054 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.052+0000 7fa6b4a00640 1 -- 172.21.15.152:0/3262940548 >> 172.21.15.152:0/3262940548 conn(0x7fa6b0067ae0 msgr2=0x7fa6b00b1060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:28.054 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.053+0000 7fa6b4a00640 1 -- 172.21.15.152:0/3262940548 shutdown_connections 2024-04-20T11:32:28.054 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.053+0000 7fa6b4a00640 1 -- 172.21.15.152:0/3262940548 wait complete. 2024-04-20T11:32:28.074 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:32:28.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.263+0000 7f5fca400640 1 Processor -- start 2024-04-20T11:32:28.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.264+0000 7f5fca400640 1 -- start start 2024-04-20T11:32:28.266 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.265+0000 7f5fca400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f5fc415d1e0 0x7f5fc4160b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:28.266 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.265+0000 7f5fca400640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5fc40ce8c0 con 0x7f5fc415d5c0 2024-04-20T11:32:28.266 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.265+0000 7f5fca400640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5fc41600b0 con 0x7f5fc4071cd0 2024-04-20T11:32:28.266 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.265+0000 7f5fca400640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f5fc4160390 con 0x7f5fc415d1e0 2024-04-20T11:32:28.266 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.265+0000 7f5fc9000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f5fc415d1e0 0x7f5fc4160b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:28.266 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.265+0000 7f5fc3e00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f5fc415d5c0 0x7f5fc40cdfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:33676/0 (socket says 172.21.15.152:33676) 2024-04-20T11:32:28.266 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.265+0000 7f5fc3e00640 1 -- 172.21.15.152:0/2240535080 learned_addr learned my addr 172.21.15.152:0/2240535080 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:32:28.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.266+0000 7f5fc9000640 1 -- 172.21.15.152:0/2240535080 >> v1:172.21.15.175:6789/0 conn(0x7f5fc4071cd0 legacy=0x7f5fc4072120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:28.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.266+0000 7f5fc9000640 1 -- 172.21.15.152:0/2240535080 >> v1:172.21.15.152:6789/0 conn(0x7f5fc415d5c0 legacy=0x7f5fc40cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:28.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.266+0000 7f5fc9000640 1 -- 172.21.15.152:0/2240535080 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fc4160670 con 0x7f5fc415d1e0 2024-04-20T11:32:28.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.267+0000 7f5fc9000640 1 --2- 172.21.15.152:0/2240535080 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5fc415d1e0 0x7f5fc4160b90 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f5fb80097d0 tx=0x7f5fb802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c4bba7b8e52d187 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:28.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.267+0000 7f5fc3400640 1 -- 172.21.15.152:0/2240535080 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fb8035020 con 0x7f5fc415d1e0 2024-04-20T11:32:28.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.267+0000 7f5fc3400640 1 -- 172.21.15.152:0/2240535080 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fb8031830 con 0x7f5fc415d1e0 2024-04-20T11:32:28.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.267+0000 7f5fc3400640 1 -- 172.21.15.152:0/2240535080 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fb8031c00 con 0x7f5fc415d1e0 2024-04-20T11:32:28.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.268+0000 7f5fca400640 1 -- 172.21.15.152:0/2240535080 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5fc415d1e0 msgr2=0x7f5fc4160b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:28.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.268+0000 7f5fca400640 1 --2- 172.21.15.152:0/2240535080 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5fc415d1e0 0x7f5fc4160b90 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f5fb80097d0 tx=0x7f5fb802b8d0 comp rx=0 tx=0).stop 2024-04-20T11:32:28.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.268+0000 7f5fca400640 1 -- 172.21.15.152:0/2240535080 shutdown_connections 2024-04-20T11:32:28.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.268+0000 7f5fca400640 1 --2- 172.21.15.152:0/2240535080 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5fc415d1e0 0x7f5fc4160b90 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:28.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.268+0000 7f5fca400640 1 -- 172.21.15.152:0/2240535080 >> 172.21.15.152:0/2240535080 conn(0x7f5fc4067ae0 msgr2=0x7f5fc406b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:28.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.268+0000 7f5fca400640 1 -- 172.21.15.152:0/2240535080 shutdown_connections 2024-04-20T11:32:28.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.268+0000 7f5fca400640 1 -- 172.21.15.152:0/2240535080 wait complete. 2024-04-20T11:32:28.270 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.270+0000 7f5fca400640 1 Processor -- start 2024-04-20T11:32:28.270 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.270+0000 7f5fca400640 1 -- start start 2024-04-20T11:32:28.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.271+0000 7f5fca400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5fc4071cd0 0x7f5fc4155710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:28.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.271+0000 7f5fca400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5fc415d1e0 0x7f5fc414a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:28.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.271+0000 7f5fca400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5fc415d5c0 0x7f5fc414ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:28.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.271+0000 7f5fca400640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f5fc40ccef0 con 0x7f5fc415d5c0 2024-04-20T11:32:28.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.271+0000 7f5fca400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5fc40ccd70 con 0x7f5fc415d1e0 2024-04-20T11:32:28.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.271+0000 7f5fca400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5fc40cd070 con 0x7f5fc4071cd0 2024-04-20T11:32:28.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.271+0000 7f5fc9a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5fc415d5c0 0x7f5fc414ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:28.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.271+0000 7f5fc9a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5fc415d5c0 0x7f5fc414ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:60162/0 (socket says 172.21.15.152:60162) 2024-04-20T11:32:28.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.271+0000 7f5fc9a00640 1 -- 172.21.15.152:0/932437492 learned_addr learned my addr 172.21.15.152:0/932437492 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:32:28.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.271+0000 7f5fc9a00640 1 -- 172.21.15.152:0/932437492 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5fc4071cd0 msgr2=0x7f5fc4155710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:28.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.271+0000 7f5fc9a00640 1 --2- 172.21.15.152:0/932437492 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5fc4071cd0 0x7f5fc4155710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:28.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.271+0000 7f5fc9a00640 1 -- 172.21.15.152:0/932437492 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5fc415d1e0 msgr2=0x7f5fc414a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:28.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.271+0000 7f5fc9a00640 1 --2- 172.21.15.152:0/932437492 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5fc415d1e0 0x7f5fc414a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:28.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.271+0000 7f5fc9a00640 1 -- 172.21.15.152:0/932437492 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fc415dad0 con 0x7f5fc415d5c0 2024-04-20T11:32:28.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.272+0000 7f5fc9a00640 1 --2- 172.21.15.152:0/932437492 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5fc415d5c0 0x7f5fc414ae10 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc00c780 tx=0x7f5fbc00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:28.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.272+0000 7f5fc0c00640 1 -- 172.21.15.152:0/932437492 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fbc004b60 con 0x7f5fc415d5c0 2024-04-20T11:32:28.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.272+0000 7f5fc0c00640 1 -- 172.21.15.152:0/932437492 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fbc004d00 con 0x7f5fc415d5c0 2024-04-20T11:32:28.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.272+0000 7f5fc0c00640 1 -- 172.21.15.152:0/932437492 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fbc00aa90 con 0x7f5fc415d5c0 2024-04-20T11:32:28.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.272+0000 7f5fca400640 1 -- 172.21.15.152:0/932437492 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fc415ddb0 con 0x7f5fc415d5c0 2024-04-20T11:32:28.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.272+0000 7f5fca400640 1 -- 172.21.15.152:0/932437492 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fc415e370 con 0x7f5fc415d5c0 2024-04-20T11:32:28.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.273+0000 7f5fca400640 1 -- 172.21.15.152:0/932437492 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fc4075e60 con 0x7f5fc415d5c0 2024-04-20T11:32:28.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.274+0000 7f5fc0c00640 1 -- 172.21.15.152:0/932437492 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 91142+0+0 (secure 0 0 0) 0x7f5fbc00ac30 con 0x7f5fc415d5c0 2024-04-20T11:32:28.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.275+0000 7f5fc0c00640 1 --2- 172.21.15.152:0/932437492 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f5fa406ca10 0x7f5fa406eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:28.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.275+0000 7f5fc0c00640 1 -- 172.21.15.152:0/932437492 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 6268+0+0 (secure 0 0 0) 0x7f5fbc08c1b0 con 0x7f5fc415d5c0 2024-04-20T11:32:28.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.275+0000 7f5fc9000640 1 --2- 172.21.15.152:0/932437492 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f5fa406ca10 0x7f5fa406eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:28.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.277+0000 7f5fc9000640 1 --2- 172.21.15.152:0/932437492 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f5fa406ca10 0x7f5fa406eed0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f5fb8036000 tx=0x7f5fb8030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:28.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.279+0000 7f5fc0c00640 1 -- 172.21.15.152:0/932437492 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f5fbc020070 con 0x7f5fc415d5c0 2024-04-20T11:32:28.465 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.465+0000 7f5fca400640 1 -- 172.21.15.152:0/932437492 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5fc4072c90 con 0x7f5fc415d5c0 2024-04-20T11:32:28.467 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.466+0000 7f5fc0c00640 1 -- 172.21.15.152:0/932437492 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) ==== 74+0+17550 (secure 0 0 0) 0x7f5fbc05a330 con 0x7f5fc415d5c0 2024-04-20T11:32:28.467 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:32:28.468 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":72,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:32:22.542221+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:31:48.024528+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-20T11:31:48.725648+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.152:0/3082578161":"2024-04-21T11:32:22.542176+0000","172.21.15.152:0/1524004838":"2024-04-21T11:32:22.542176+0000","172.21.15.152:6833/2285110240":"2024-04-21T11:32:22.542176+0000","172.21.15.152:6832/2285110240":"2024-04-21T11:32:22.542176+0000","172.21.15.152:0/698943652":"2024-04-21T11:32:22.542176+0000","172.21.15.152:0/142101408":"2024-04-21T11:32:22.542176+0000","172.21.15.152:0/2510881178":"2024-04-21T11:32:22.542176+0000","172.21.15.152:0/3263977099":"2024-04-21T11:32:22.542176+0000","172.21.15.152:0/2184739958":"2024-04-21T11:32:22.542176+0000","172.21.15.152:0/3331653721":"2024-04-21T11:32:22.542176+0000","172.21.15.152:0/2405473735":"2024-04-21T11:32:22.542176+0000","172.21.15.152:0/2828322453":"2024-04-21T11:32:22.542176+0000","172.21.15.152:0/3593059376":"2024-04-21T11:32:22.542176+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:32:28.469 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.469+0000 7f5fca400640 1 -- 172.21.15.152:0/932437492 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f5fa406ca10 msgr2=0x7f5fa406eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:28.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.469+0000 7f5fca400640 1 --2- 172.21.15.152:0/932437492 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f5fa406ca10 0x7f5fa406eed0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f5fb8036000 tx=0x7f5fb8030000 comp rx=0 tx=0).stop 2024-04-20T11:32:28.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.469+0000 7f5fca400640 1 -- 172.21.15.152:0/932437492 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5fc415d5c0 msgr2=0x7f5fc414ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:28.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.469+0000 7f5fca400640 1 --2- 172.21.15.152:0/932437492 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5fc415d5c0 0x7f5fc414ae10 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc00c780 tx=0x7f5fbc00cc40 comp rx=0 tx=0).stop 2024-04-20T11:32:28.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.469+0000 7f5fca400640 1 -- 172.21.15.152:0/932437492 shutdown_connections 2024-04-20T11:32:28.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.469+0000 7f5fca400640 1 --2- 172.21.15.152:0/932437492 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f5fa406ca10 0x7f5fa406eed0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:28.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.469+0000 7f5fca400640 1 --2- 172.21.15.152:0/932437492 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5fc415d5c0 0x7f5fc414ae10 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:28.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.469+0000 7f5fca400640 1 --2- 172.21.15.152:0/932437492 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5fc415d1e0 0x7f5fc414a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:28.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.469+0000 7f5fca400640 1 --2- 172.21.15.152:0/932437492 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5fc4071cd0 0x7f5fc4155710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:28.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.469+0000 7f5fca400640 1 -- 172.21.15.152:0/932437492 >> 172.21.15.152:0/932437492 conn(0x7f5fc4067ae0 msgr2=0x7f5fc40cd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:28.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.469+0000 7f5fca400640 1 -- 172.21.15.152:0/932437492 shutdown_connections 2024-04-20T11:32:28.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.469+0000 7f5fca400640 1 -- 172.21.15.152:0/932437492 wait complete. 2024-04-20T11:32:28.489 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-20T11:32:28.692 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.691+0000 7f965f200640 1 Processor -- start 2024-04-20T11:32:28.692 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.692+0000 7f965f200640 1 -- start start 2024-04-20T11:32:28.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.693+0000 7f965f200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f96600d4ec0 0x7f96600d0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:28.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.693+0000 7f965f200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f96600d2b80 0x7f96600d3010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:28.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.693+0000 7f965f200640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96600d26a0 con 0x7f9660161d30 2024-04-20T11:32:28.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.693+0000 7f965f200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f96600d0540 con 0x7f96600d4ec0 2024-04-20T11:32:28.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.693+0000 7f965f200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f96600cfb70 con 0x7f96600d2b80 2024-04-20T11:32:28.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.693+0000 7f965de00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f9660161d30 0x7f96600d15e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:33680/0 (socket says 172.21.15.152:33680) 2024-04-20T11:32:28.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.694+0000 7f965de00640 1 -- 172.21.15.152:0/1396189526 learned_addr learned my addr 172.21.15.152:0/1396189526 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:32:28.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.694+0000 7f965d400640 1 --2- 172.21.15.152:0/1396189526 >> v2:172.21.15.175:3300/0 conn(0x7f96600d4ec0 0x7f96600d0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:28.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.694+0000 7f965e800640 1 --2- 172.21.15.152:0/1396189526 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f96600d2b80 0x7f96600d3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:28.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.694+0000 7f965ca00640 1 -- 172.21.15.152:0/1396189526 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1514666315 0 0) 0x7f96600d26a0 con 0x7f9660161d30 2024-04-20T11:32:28.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.694+0000 7f965ca00640 1 -- 172.21.15.152:0/1396189526 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9644003520 con 0x7f9660161d30 2024-04-20T11:32:28.697 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.695+0000 7f965ca00640 1 -- 172.21.15.152:0/1396189526 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1756582747 0 0) 0x7f9644003520 con 0x7f9660161d30 2024-04-20T11:32:28.697 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.695+0000 7f965ca00640 1 -- 172.21.15.152:0/1396189526 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f96600d2b80 msgr2=0x7f96600d3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:28.697 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.695+0000 7f965ca00640 1 --2- 172.21.15.152:0/1396189526 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f96600d2b80 0x7f96600d3010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:28.697 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.695+0000 7f965ca00640 1 -- 172.21.15.152:0/1396189526 >> v2:172.21.15.175:3300/0 conn(0x7f96600d4ec0 msgr2=0x7f96600d0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:28.698 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.695+0000 7f965ca00640 1 --2- 172.21.15.152:0/1396189526 >> v2:172.21.15.175:3300/0 conn(0x7f96600d4ec0 0x7f96600d0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:28.698 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.695+0000 7f965ca00640 1 -- 172.21.15.152:0/1396189526 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96600d53c0 con 0x7f9660161d30 2024-04-20T11:32:28.698 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.695+0000 7f965ca00640 1 -- 172.21.15.152:0/1396189526 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f9650001670 con 0x7f9660161d30 2024-04-20T11:32:28.698 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.695+0000 7f965ca00640 1 -- 172.21.15.152:0/1396189526 >> v1:172.21.15.152:6789/0 conn(0x7f9660161d30 legacy=0x7f96600d15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:28.698 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.695+0000 7f965ca00640 1 --2- 172.21.15.152:0/1396189526 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f96440039b0 0x7f9644003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:28.698 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.695+0000 7f965ca00640 1 --2- 172.21.15.152:0/1396189526 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9644004800 0x7f9644004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:28.698 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.695+0000 7f965ca00640 1 --2- 172.21.15.152:0/1396189526 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f96600d2b80 0x7f96440051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:28.698 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.695+0000 7f965ca00640 1 -- 172.21.15.152:0/1396189526 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f96600d0540 con 0x7f96440039b0 2024-04-20T11:32:28.698 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.695+0000 7f965ca00640 1 -- 172.21.15.152:0/1396189526 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f96600cfb70 con 0x7f9644004800 2024-04-20T11:32:28.698 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.695+0000 7f965ca00640 1 -- 172.21.15.152:0/1396189526 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f964400b9c0 con 0x7f96600d2b80 2024-04-20T11:32:28.698 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.695+0000 7f965de00640 1 --2- 172.21.15.152:0/1396189526 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f96440039b0 0x7f9644003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:28.698 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.695+0000 7f965e800640 1 --2- 172.21.15.152:0/1396189526 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f96600d2b80 0x7f96440051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:28.698 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.695+0000 7f965d400640 1 --2- 172.21.15.152:0/1396189526 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9644004800 0x7f9644004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:28.699 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.696+0000 7f965de00640 1 -- 172.21.15.152:0/1396189526 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f96600d2b80 msgr2=0x7f96440051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:28.699 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.696+0000 7f965de00640 1 --2- 172.21.15.152:0/1396189526 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f96600d2b80 0x7f96440051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:28.699 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.696+0000 7f965de00640 1 -- 172.21.15.152:0/1396189526 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9644004800 msgr2=0x7f9644004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:28.699 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.696+0000 7f965de00640 1 --2- 172.21.15.152:0/1396189526 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9644004800 0x7f9644004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:28.699 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.696+0000 7f965de00640 1 -- 172.21.15.152:0/1396189526 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9644003520 con 0x7f96440039b0 2024-04-20T11:32:28.699 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.696+0000 7f965d400640 1 --2- 172.21.15.152:0/1396189526 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9644004800 0x7f9644004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:32:28.699 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.697+0000 7f965de00640 1 --2- 172.21.15.152:0/1396189526 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f96440039b0 0x7f9644003dd0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f965000c290 tx=0x7f9644005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a282ca0267301e76 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:28.699 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.697+0000 7f965ca00640 1 -- 172.21.15.152:0/1396189526 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9650007d50 con 0x7f96440039b0 2024-04-20T11:32:28.699 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.697+0000 7f965ca00640 1 -- 172.21.15.152:0/1396189526 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9650030070 con 0x7f96440039b0 2024-04-20T11:32:28.699 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.697+0000 7f965ca00640 1 -- 172.21.15.152:0/1396189526 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f965002cd80 con 0x7f96440039b0 2024-04-20T11:32:28.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.698+0000 7f965f200640 1 -- 172.21.15.152:0/1396189526 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f96440039b0 msgr2=0x7f9644003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:28.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.698+0000 7f965f200640 1 --2- 172.21.15.152:0/1396189526 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f96440039b0 0x7f9644003dd0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f965000c290 tx=0x7f9644005830 comp rx=0 tx=0).stop 2024-04-20T11:32:28.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.698+0000 7f965e800640 1 -- 172.21.15.152:0/1396189526 reap_dead start 2024-04-20T11:32:28.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.698+0000 7f965f200640 1 -- 172.21.15.152:0/1396189526 shutdown_connections 2024-04-20T11:32:28.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.698+0000 7f965f200640 1 -- 172.21.15.152:0/1396189526 >> 172.21.15.152:0/1396189526 conn(0x7f966008bfd0 msgr2=0x7f96600d45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:28.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.698+0000 7f965f200640 1 -- 172.21.15.152:0/1396189526 shutdown_connections 2024-04-20T11:32:28.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.698+0000 7f965f200640 1 -- 172.21.15.152:0/1396189526 wait complete. 2024-04-20T11:32:28.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.699+0000 7f965f200640 1 Processor -- start 2024-04-20T11:32:28.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.700+0000 7f965f200640 1 -- start start 2024-04-20T11:32:28.703 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.701+0000 7f965f200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f96440039b0 0x7f966015bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:28.703 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.701+0000 7f965f200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f966015c2a0 0x7f96601f7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:28.703 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.701+0000 7f965f200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f96601f8160 0x7f96600ccc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:28.703 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.701+0000 7f965f200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f96600cf1b0 con 0x7f96601f8160 2024-04-20T11:32:28.703 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.701+0000 7f965f200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f96600cf800 con 0x7f96440039b0 2024-04-20T11:32:28.703 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.701+0000 7f965f200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f9660162110 con 0x7f966015c2a0 2024-04-20T11:32:28.704 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.701+0000 7f965e800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f96601f8160 0x7f96600ccc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:28.704 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.701+0000 7f965e800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f96601f8160 0x7f96600ccc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:60176/0 (socket says 172.21.15.152:60176) 2024-04-20T11:32:28.704 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.701+0000 7f965e800640 1 -- 172.21.15.152:0/273482213 learned_addr learned my addr 172.21.15.152:0/273482213 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:32:28.704 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.701+0000 7f965de00640 1 --2- 172.21.15.152:0/273482213 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f96440039b0 0x7f966015bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:28.704 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.701+0000 7f965d400640 1 --2- 172.21.15.152:0/273482213 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f966015c2a0 0x7f96601f7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:28.704 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.701+0000 7f965e800640 1 -- 172.21.15.152:0/273482213 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f966015c2a0 msgr2=0x7f96601f7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:28.704 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.701+0000 7f965e800640 1 --2- 172.21.15.152:0/273482213 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f966015c2a0 0x7f96601f7c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:28.704 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.701+0000 7f965e800640 1 -- 172.21.15.152:0/273482213 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f96440039b0 msgr2=0x7f966015bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:28.704 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.701+0000 7f965e800640 1 --2- 172.21.15.152:0/273482213 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f96440039b0 0x7f966015bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:28.704 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.701+0000 7f965e800640 1 -- 172.21.15.152:0/273482213 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96600cd940 con 0x7f96601f8160 2024-04-20T11:32:28.704 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.701+0000 7f965de00640 1 --2- 172.21.15.152:0/273482213 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f96440039b0 0x7f966015bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:32:28.704 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.702+0000 7f965e800640 1 --2- 172.21.15.152:0/273482213 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f96601f8160 0x7f96600ccc80 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f96600d2120 tx=0x7f96580115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:28.705 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.702+0000 7f9656000640 1 -- 172.21.15.152:0/273482213 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9658011e10 con 0x7f96601f8160 2024-04-20T11:32:28.705 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.702+0000 7f9656000640 1 -- 172.21.15.152:0/273482213 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f965801b070 con 0x7f96601f8160 2024-04-20T11:32:28.705 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.702+0000 7f9656000640 1 -- 172.21.15.152:0/273482213 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9658003990 con 0x7f96601f8160 2024-04-20T11:32:28.705 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.702+0000 7f965f200640 1 -- 172.21.15.152:0/273482213 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96600cdc20 con 0x7f96601f8160 2024-04-20T11:32:28.705 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.702+0000 7f965f200640 1 -- 172.21.15.152:0/273482213 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96600ce1e0 con 0x7f96601f8160 2024-04-20T11:32:28.706 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.703+0000 7f965f200640 1 -- 172.21.15.152:0/273482213 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96600ce5c0 con 0x7f96601f8160 2024-04-20T11:32:28.706 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.704+0000 7f9656000640 1 -- 172.21.15.152:0/273482213 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 91142+0+0 (secure 0 0 0) 0x7f9658005d10 con 0x7f96601f8160 2024-04-20T11:32:28.706 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.705+0000 7f9656000640 1 --2- 172.21.15.152:0/273482213 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f964006ca40 0x7f964006ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:28.706 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.705+0000 7f9656000640 1 -- 172.21.15.152:0/273482213 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 6268+0+0 (secure 0 0 0) 0x7f965808c3b0 con 0x7f96601f8160 2024-04-20T11:32:28.706 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.706+0000 7f965de00640 1 --2- 172.21.15.152:0/273482213 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f964006ca40 0x7f964006ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:28.707 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.707+0000 7f965de00640 1 --2- 172.21.15.152:0/273482213 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f964006ca40 0x7f964006ef00 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f9650034000 tx=0x7f9650007000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:28.710 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.709+0000 7f9656000640 1 -- 172.21.15.152:0/273482213 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f965801d310 con 0x7f96601f8160 2024-04-20T11:32:28.951 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.950+0000 7f965f200640 1 -- 172.21.15.152:0/273482213 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f9660075e60 con 0x7f96601f8160 2024-04-20T11:32:28.952 INFO:tasks.ceph.mon.a.smithi152.stderr:2024-04-20T11:32:28.952+0000 7f8229800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-20T11:32:28.957 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.956+0000 7f9656000640 1 -- 172.21.15.152:0/273482213 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v45) ==== 161+0+0 (secure 0 0 0) 0x7f965805a380 con 0x7f96601f8160 2024-04-20T11:32:28.957 INFO:teuthology.orchestra.run.smithi152.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-20T11:32:28.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.959+0000 7f965f200640 1 -- 172.21.15.152:0/273482213 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f964006ca40 msgr2=0x7f964006ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:28.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.959+0000 7f965f200640 1 --2- 172.21.15.152:0/273482213 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f964006ca40 0x7f964006ef00 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f9650034000 tx=0x7f9650007000 comp rx=0 tx=0).stop 2024-04-20T11:32:28.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.959+0000 7f965f200640 1 -- 172.21.15.152:0/273482213 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f96601f8160 msgr2=0x7f96600ccc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:28.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.959+0000 7f965f200640 1 --2- 172.21.15.152:0/273482213 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f96601f8160 0x7f96600ccc80 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f96600d2120 tx=0x7f96580115a0 comp rx=0 tx=0).stop 2024-04-20T11:32:28.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.959+0000 7f965f200640 1 -- 172.21.15.152:0/273482213 shutdown_connections 2024-04-20T11:32:28.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.959+0000 7f965f200640 1 --2- 172.21.15.152:0/273482213 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f964006ca40 0x7f964006ef00 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:28.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.959+0000 7f965f200640 1 --2- 172.21.15.152:0/273482213 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f96601f8160 0x7f96600ccc80 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:28.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.959+0000 7f965f200640 1 --2- 172.21.15.152:0/273482213 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f966015c2a0 0x7f96601f7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:28.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.959+0000 7f965f200640 1 --2- 172.21.15.152:0/273482213 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f96440039b0 0x7f966015bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:28.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.959+0000 7f965f200640 1 -- 172.21.15.152:0/273482213 >> 172.21.15.152:0/273482213 conn(0x7f966008bfd0 msgr2=0x7f9660160fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:28.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.959+0000 7f965f200640 1 -- 172.21.15.152:0/273482213 shutdown_connections 2024-04-20T11:32:28.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:28.959+0000 7f965f200640 1 -- 172.21.15.152:0/273482213 wait complete. 2024-04-20T11:32:28.981 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-20T11:32:28.998 INFO:tasks.ceph.mds.b.smithi175.stderr: -7014> 2024-04-20T11:31:56.252+0000 7f1661600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-20T11:32:29.090 INFO:tasks.ceph.mds.b.smithi175.stdout:starting mds.b at 2024-04-20T11:32:29.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:29.185+0000 7f2670a00640 1 Processor -- start 2024-04-20T11:32:29.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:29.185+0000 7f2670a00640 1 -- start start 2024-04-20T11:32:29.187 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:29.186+0000 7f2670a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f266c160220 0x7f266c1606b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:29.187 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:29.186+0000 7f2670a00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f266c06fb40 con 0x7f266c141450 2024-04-20T11:32:29.187 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:29.186+0000 7f2670a00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f266c160f90 con 0x7f266c071cd0 2024-04-20T11:32:29.187 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:29.186+0000 7f2670a00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f266c13f8e0 con 0x7f266c160220 2024-04-20T11:32:29.187 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:29.187+0000 7f2661e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f266c160220 0x7f266c1606b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:29.187 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:29.187+0000 7f266b200640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f266c141450 0x7f266c1418e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:33690/0 (socket says 172.21.15.152:33690) 2024-04-20T11:32:29.187 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:29.187+0000 7f266b200640 1 -- 172.21.15.152:0/264402500 learned_addr learned my addr 172.21.15.152:0/264402500 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:32:29.188 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:29.187+0000 7f2661e00640 1 -- 172.21.15.152:0/264402500 >> v1:172.21.15.175:6789/0 conn(0x7f266c071cd0 legacy=0x7f266c0720b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:29.188 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:29.187+0000 7f2661e00640 1 -- 172.21.15.152:0/264402500 >> v1:172.21.15.152:6789/0 conn(0x7f266c141450 legacy=0x7f266c1418e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:29.188 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:29.187+0000 7f2661e00640 1 -- 172.21.15.152:0/264402500 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f266c06dd90 con 0x7f266c160220 2024-04-20T11:32:29.188 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:29.187+0000 7f2669e00640 1 -- 172.21.15.152:0/264402500 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2673471940 0 0) 0x7f266c06fb40 con 0x7f266c141450 2024-04-20T11:32:29.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:29.188+0000 7f2661e00640 1 --2- 172.21.15.152:0/264402500 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f266c160220 0x7f266c1606b0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f26540097d0 tx=0x7f265402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ff30bca3b4a7adf server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:29.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:29.189+0000 7f2669e00640 1 -- 172.21.15.152:0/264402500 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26540315a0 con 0x7f266c160220 2024-04-20T11:32:29.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:29.189+0000 7f2669e00640 1 -- 172.21.15.152:0/264402500 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2654037070 con 0x7f266c160220 2024-04-20T11:32:29.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:29.189+0000 7f2669e00640 1 -- 172.21.15.152:0/264402500 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2654031960 con 0x7f266c160220 2024-04-20T11:32:29.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:29.189+0000 7f2670a00640 1 -- 172.21.15.152:0/264402500 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f266c160220 msgr2=0x7f266c1606b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:29.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:29.189+0000 7f2670a00640 1 --2- 172.21.15.152:0/264402500 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f266c160220 0x7f266c1606b0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f26540097d0 tx=0x7f265402b840 comp rx=0 tx=0).stop 2024-04-20T11:32:29.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:29.189+0000 7f2670a00640 1 -- 172.21.15.152:0/264402500 shutdown_connections 2024-04-20T11:32:29.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:29.189+0000 7f2670a00640 1 --2- 172.21.15.152:0/264402500 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f266c160220 0x7f266c1606b0 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:29.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:29.189+0000 7f2670a00640 1 -- 172.21.15.152:0/264402500 >> 172.21.15.152:0/264402500 conn(0x7f266c067ae0 msgr2=0x7f266c06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:29.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:29.190+0000 7f2670a00640 1 -- 172.21.15.152:0/264402500 shutdown_connections 2024-04-20T11:32:29.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:29.190+0000 7f2670a00640 1 -- 172.21.15.152:0/264402500 wait complete. 2024-04-20T11:32:29.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:29.191+0000 7f2670a00640 1 Processor -- start 2024-04-20T11:32:29.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:29.191+0000 7f2670a00640 1 -- start start 2024-04-20T11:32:29.193 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:29.192+0000 7f2670a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f266c071cd0 0x7f266c13cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:29.193 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:29.192+0000 7f2670a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f266c141450 0x7f266c15b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:29.193 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:29.192+0000 7f2670a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f266c160220 0x7f266c0aed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:29.193 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:29.192+0000 7f2670a00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f266c06e630 con 0x7f266c141450 2024-04-20T11:32:29.193 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:29.192+0000 7f2670a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f266c06e350 con 0x7f266c071cd0 2024-04-20T11:32:29.193 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:29.192+0000 7f2670a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f266c160bf0 con 0x7f266c160220 2024-04-20T11:32:29.194 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:29.193+0000 7f2661e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f266c141450 0x7f266c15b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:29.194 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:29.193+0000 7f2661e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f266c141450 0x7f266c15b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:60182/0 (socket says 172.21.15.152:60182) 2024-04-20T11:32:29.194 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:29.193+0000 7f2661e00640 1 -- 172.21.15.152:0/911255919 learned_addr learned my addr 172.21.15.152:0/911255919 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:32:29.195 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:29.193+0000 7f2661e00640 1 -- 172.21.15.152:0/911255919 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f266c160220 msgr2=0x7f266c0aed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:29.195 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:29.193+0000 7f2661e00640 1 --2- 172.21.15.152:0/911255919 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f266c160220 0x7f266c0aed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:29.195 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:29.193+0000 7f2661e00640 1 -- 172.21.15.152:0/911255919 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f266c071cd0 msgr2=0x7f266c13cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:29.195 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:29.193+0000 7f266a800640 1 --2- 172.21.15.152:0/911255919 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f266c071cd0 0x7f266c13cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:29.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:29.193+0000 7f2661e00640 1 --2- 172.21.15.152:0/911255919 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f266c071cd0 0x7f266c13cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:29.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:29.193+0000 7f2661e00640 1 -- 172.21.15.152:0/911255919 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f266c0ad990 con 0x7f266c141450 2024-04-20T11:32:29.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:29.193+0000 7f2661e00640 1 --2- 172.21.15.152:0/911255919 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f266c141450 0x7f266c15b8f0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f26540097d0 tx=0x7f26540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:29.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:29.194+0000 7f2663400640 1 -- 172.21.15.152:0/911255919 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2654032840 con 0x7f266c141450 2024-04-20T11:32:29.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:29.194+0000 7f2663400640 1 -- 172.21.15.152:0/911255919 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26540329e0 con 0x7f266c141450 2024-04-20T11:32:29.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:29.194+0000 7f2670a00640 1 -- 172.21.15.152:0/911255919 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f266c0adc70 con 0x7f266c141450 2024-04-20T11:32:29.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:29.194+0000 7f2663400640 1 -- 172.21.15.152:0/911255919 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2654032dd0 con 0x7f266c141450 2024-04-20T11:32:29.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:29.194+0000 7f2670a00640 1 -- 172.21.15.152:0/911255919 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f266c0ae230 con 0x7f266c141450 2024-04-20T11:32:29.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:29.195+0000 7f2670a00640 1 -- 172.21.15.152:0/911255919 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f266c075e60 con 0x7f266c141450 2024-04-20T11:32:29.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:29.196+0000 7f2663400640 1 -- 172.21.15.152:0/911255919 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 91142+0+0 (secure 0 0 0) 0x7f265404d3f0 con 0x7f266c141450 2024-04-20T11:32:29.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:29.197+0000 7f2663400640 1 --2- 172.21.15.152:0/911255919 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f264806ca10 0x7f264806eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:29.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:29.197+0000 7f2663400640 1 -- 172.21.15.152:0/911255919 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 6354+0+0 (secure 0 0 0) 0x7f265402f030 con 0x7f266c141450 2024-04-20T11:32:29.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:29.198+0000 7f266a800640 1 --2- 172.21.15.152:0/911255919 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f264806ca10 0x7f264806eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:29.199 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:29.199+0000 7f266a800640 1 --2- 172.21.15.152:0/911255919 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f264806ca10 0x7f264806eed0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f265c003d90 tx=0x7f265c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:29.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:29.202+0000 7f2663400640 1 -- 172.21.15.152:0/911255919 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f2654040070 con 0x7f266c141450 2024-04-20T11:32:29.445 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:29.444+0000 7f2670a00640 1 -- 172.21.15.152:0/911255919 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f266c15fa00 con 0x7f266c141450 2024-04-20T11:32:29.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:29.960+0000 7f2663400640 1 -- 172.21.15.152:0/911255919 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v46) ==== 104+0+0 (secure 0 0 0) 0x7f265407bc60 con 0x7f266c141450 2024-04-20T11:32:29.964 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:29.963+0000 7f2670a00640 1 -- 172.21.15.152:0/911255919 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f264806ca10 msgr2=0x7f264806eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:29.964 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:29.963+0000 7f2670a00640 1 --2- 172.21.15.152:0/911255919 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f264806ca10 0x7f264806eed0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f265c003d90 tx=0x7f265c009040 comp rx=0 tx=0).stop 2024-04-20T11:32:29.964 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:29.963+0000 7f2670a00640 1 -- 172.21.15.152:0/911255919 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f266c141450 msgr2=0x7f266c15b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:29.964 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:29.963+0000 7f2670a00640 1 --2- 172.21.15.152:0/911255919 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f266c141450 0x7f266c15b8f0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f26540097d0 tx=0x7f26540046c0 comp rx=0 tx=0).stop 2024-04-20T11:32:29.964 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:29.963+0000 7f2670a00640 1 -- 172.21.15.152:0/911255919 shutdown_connections 2024-04-20T11:32:29.964 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:29.963+0000 7f2670a00640 1 --2- 172.21.15.152:0/911255919 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f264806ca10 0x7f264806eed0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:29.965 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:29.963+0000 7f2670a00640 1 --2- 172.21.15.152:0/911255919 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f266c160220 0x7f266c0aed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:29.965 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:29.963+0000 7f2670a00640 1 --2- 172.21.15.152:0/911255919 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f266c141450 0x7f266c15b8f0 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:29.965 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:29.963+0000 7f2670a00640 1 --2- 172.21.15.152:0/911255919 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f266c071cd0 0x7f266c13cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:29.965 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:29.963+0000 7f2670a00640 1 -- 172.21.15.152:0/911255919 >> 172.21.15.152:0/911255919 conn(0x7f266c067ae0 msgr2=0x7f266c15d010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:29.965 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:29.964+0000 7f2670a00640 1 -- 172.21.15.152:0/911255919 shutdown_connections 2024-04-20T11:32:29.965 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:29.964+0000 7f2670a00640 1 -- 172.21.15.152:0/911255919 wait complete. 2024-04-20T11:32:29.986 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-20T11:32:30.193 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:30.192+0000 7f0dbde00640 1 Processor -- start 2024-04-20T11:32:30.193 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:30.192+0000 7f0dbde00640 1 -- start start 2024-04-20T11:32:30.194 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:30.194+0000 7f0dbde00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0db815b5c0 0x7f0db8071f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:30.194 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:30.194+0000 7f0dbde00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f0db80d05c0 0x7f0db80d0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:30.194 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:30.194+0000 7f0dbde00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0db815b9a0 con 0x7f0db80cebd0 2024-04-20T11:32:30.195 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:30.194+0000 7f0dbde00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f0db8072610 con 0x7f0db80d05c0 2024-04-20T11:32:30.195 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:30.194+0000 7f0dbde00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0db80728f0 con 0x7f0db815b5c0 2024-04-20T11:32:30.195 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:30.194+0000 7f0dbd400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f0db80d05c0 0x7f0db80d0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:30.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:30.194+0000 7f0dbd400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f0db80d05c0 0x7f0db80d0a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:60198/0 (socket says 172.21.15.152:60198) 2024-04-20T11:32:30.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:30.194+0000 7f0dbd400640 1 -- 172.21.15.152:0/3615741833 learned_addr learned my addr 172.21.15.152:0/3615741833 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:32:30.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:30.195+0000 7f0dbca00640 1 --2- 172.21.15.152:0/3615741833 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0db815b5c0 0x7f0db8071f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:30.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:30.195+0000 7f0dbd400640 1 -- 172.21.15.152:0/3615741833 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0db815b5c0 msgr2=0x7f0db8071f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:30.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:30.195+0000 7f0dbd400640 1 --2- 172.21.15.152:0/3615741833 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0db815b5c0 0x7f0db8071f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:30.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:30.195+0000 7f0dbd400640 1 -- 172.21.15.152:0/3615741833 >> v1:172.21.15.175:6789/0 conn(0x7f0db80cebd0 legacy=0x7f0db80b3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:30.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:30.195+0000 7f0dbd400640 1 -- 172.21.15.152:0/3615741833 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0db80b3500 con 0x7f0db80d05c0 2024-04-20T11:32:30.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:30.195+0000 7f0dbca00640 1 --2- 172.21.15.152:0/3615741833 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0db815b5c0 0x7f0db8071f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:32:30.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:30.196+0000 7f0dbd400640 1 --2- 172.21.15.152:0/3615741833 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0db80d05c0 0x7f0db80d0a50 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f0db0009870 tx=0x7f0db002db00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5136af43757f8dea server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:30.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:30.196+0000 7f0db7e00640 1 -- 172.21.15.152:0/3615741833 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db002f8c0 con 0x7f0db80d05c0 2024-04-20T11:32:30.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:30.196+0000 7f0db7e00640 1 -- 172.21.15.152:0/3615741833 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0db002fa60 con 0x7f0db80d05c0 2024-04-20T11:32:30.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:30.196+0000 7f0db7e00640 1 -- 172.21.15.152:0/3615741833 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db002fe30 con 0x7f0db80d05c0 2024-04-20T11:32:30.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:30.197+0000 7f0dbde00640 1 -- 172.21.15.152:0/3615741833 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0db80d05c0 msgr2=0x7f0db80d0a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:30.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:30.197+0000 7f0dbde00640 1 --2- 172.21.15.152:0/3615741833 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0db80d05c0 0x7f0db80d0a50 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f0db0009870 tx=0x7f0db002db00 comp rx=0 tx=0).stop 2024-04-20T11:32:30.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:30.197+0000 7f0dbde00640 1 -- 172.21.15.152:0/3615741833 shutdown_connections 2024-04-20T11:32:30.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:30.197+0000 7f0dbde00640 1 --2- 172.21.15.152:0/3615741833 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0db80d05c0 0x7f0db80d0a50 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:30.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:30.197+0000 7f0dbde00640 1 --2- 172.21.15.152:0/3615741833 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0db815b5c0 0x7f0db8071f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:30.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:30.197+0000 7f0dbde00640 1 -- 172.21.15.152:0/3615741833 >> 172.21.15.152:0/3615741833 conn(0x7f0db806bda0 msgr2=0x7f0db806c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:30.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:30.197+0000 7f0dbde00640 1 -- 172.21.15.152:0/3615741833 shutdown_connections 2024-04-20T11:32:30.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:30.197+0000 7f0dbde00640 1 -- 172.21.15.152:0/3615741833 wait complete. 2024-04-20T11:32:30.199 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:30.199+0000 7f0dbde00640 1 Processor -- start 2024-04-20T11:32:30.199 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:30.199+0000 7f0dbde00640 1 -- start start 2024-04-20T11:32:30.201 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:30.200+0000 7f0dbde00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0db80cebd0 0x7f0db80af2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:30.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:30.200+0000 7f0dbde00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0db80d05c0 0x7f0db8155700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:30.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:30.200+0000 7f0dbde00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0db815b5c0 0x7f0db81e47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:30.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:30.200+0000 7f0dbde00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f0db80b4410 con 0x7f0db80d05c0 2024-04-20T11:32:30.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:30.200+0000 7f0dbde00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0db80cf030 con 0x7f0db80cebd0 2024-04-20T11:32:30.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:30.200+0000 7f0dbde00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0db806db10 con 0x7f0db815b5c0 2024-04-20T11:32:30.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:30.200+0000 7f0dafe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0db80d05c0 0x7f0db8155700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:30.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:30.200+0000 7f0dafe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0db80d05c0 0x7f0db8155700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:60210/0 (socket says 172.21.15.152:60210) 2024-04-20T11:32:30.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:30.200+0000 7f0dafe00640 1 -- 172.21.15.152:0/877865051 learned_addr learned my addr 172.21.15.152:0/877865051 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:32:30.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:30.201+0000 7f0dbd400640 1 --2- 172.21.15.152:0/877865051 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0db815b5c0 0x7f0db81e47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:30.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:30.201+0000 7f0dbca00640 1 --2- 172.21.15.152:0/877865051 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0db80cebd0 0x7f0db80af2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:30.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:30.201+0000 7f0dafe00640 1 -- 172.21.15.152:0/877865051 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0db815b5c0 msgr2=0x7f0db81e47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:30.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:30.201+0000 7f0dafe00640 1 --2- 172.21.15.152:0/877865051 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0db815b5c0 0x7f0db81e47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:30.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:30.201+0000 7f0dafe00640 1 -- 172.21.15.152:0/877865051 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0db80cebd0 msgr2=0x7f0db80af2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:30.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:30.201+0000 7f0dafe00640 1 --2- 172.21.15.152:0/877865051 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0db80cebd0 0x7f0db80af2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:30.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:30.201+0000 7f0dafe00640 1 -- 172.21.15.152:0/877865051 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0db80af8d0 con 0x7f0db80d05c0 2024-04-20T11:32:30.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:30.201+0000 7f0dbca00640 1 --2- 172.21.15.152:0/877865051 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0db80cebd0 0x7f0db80af2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:32:30.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:30.201+0000 7f0dbd400640 1 --2- 172.21.15.152:0/877865051 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0db815b5c0 0x7f0db81e47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:32:30.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:30.201+0000 7f0dafe00640 1 --2- 172.21.15.152:0/877865051 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0db80d05c0 0x7f0db8155700 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f0da000c400 tx=0x7f0da000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:30.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:30.201+0000 7f0db5600640 1 -- 172.21.15.152:0/877865051 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da00052d0 con 0x7f0db80d05c0 2024-04-20T11:32:30.204 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:30.201+0000 7f0db5600640 1 -- 172.21.15.152:0/877865051 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0da0005490 con 0x7f0db80d05c0 2024-04-20T11:32:30.204 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:30.202+0000 7f0db5600640 1 -- 172.21.15.152:0/877865051 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da0007260 con 0x7f0db80d05c0 2024-04-20T11:32:30.204 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:30.202+0000 7f0dbde00640 1 -- 172.21.15.152:0/877865051 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0db80afbb0 con 0x7f0db80d05c0 2024-04-20T11:32:30.204 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:30.202+0000 7f0dbde00640 1 -- 172.21.15.152:0/877865051 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0db80b0170 con 0x7f0db80d05c0 2024-04-20T11:32:30.204 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:30.203+0000 7f0dbde00640 1 -- 172.21.15.152:0/877865051 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0db80689f0 con 0x7f0db80d05c0 2024-04-20T11:32:30.204 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:30.204+0000 7f0db5600640 1 -- 172.21.15.152:0/877865051 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 91142+0+0 (secure 0 0 0) 0x7f0da0007400 con 0x7f0db80d05c0 2024-04-20T11:32:30.205 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:30.205+0000 7f0db5600640 1 --2- 172.21.15.152:0/877865051 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f0d9406ca10 0x7f0d9406eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:30.205 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:30.205+0000 7f0db5600640 1 -- 172.21.15.152:0/877865051 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 6354+0+0 (secure 0 0 0) 0x7f0da008b4d0 con 0x7f0db80d05c0 2024-04-20T11:32:30.206 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:30.205+0000 7f0dbca00640 1 --2- 172.21.15.152:0/877865051 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f0d9406ca10 0x7f0d9406eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:30.207 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:30.207+0000 7f0dbca00640 1 --2- 172.21.15.152:0/877865051 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f0d9406ca10 0x7f0d9406eed0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0da8009540 tx=0x7f0da800b040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:30.210 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:30.209+0000 7f0db5600640 1 -- 172.21.15.152:0/877865051 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f0da0059550 con 0x7f0db80d05c0 2024-04-20T11:32:30.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:30.416+0000 7f0dbde00640 1 -- 172.21.15.152:0/877865051 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f0db80cfe30 con 0x7f0db80d05c0 2024-04-20T11:32:30.965 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:30.964+0000 7f0db5600640 1 -- 172.21.15.152:0/877865051 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v74) ==== 181+0+0 (secure 0 0 0) 0x7f0da0058e50 con 0x7f0db80d05c0 2024-04-20T11:32:30.965 INFO:teuthology.orchestra.run.smithi152.stderr:pool 'cephfs_metadata' removed 2024-04-20T11:32:30.969 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:30.968+0000 7f0dbde00640 1 -- 172.21.15.152:0/877865051 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f0d9406ca10 msgr2=0x7f0d9406eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:30.969 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:30.968+0000 7f0dbde00640 1 --2- 172.21.15.152:0/877865051 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f0d9406ca10 0x7f0d9406eed0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0da8009540 tx=0x7f0da800b040 comp rx=0 tx=0).stop 2024-04-20T11:32:30.969 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:30.968+0000 7f0dbde00640 1 -- 172.21.15.152:0/877865051 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0db80d05c0 msgr2=0x7f0db8155700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:30.969 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:30.968+0000 7f0dbde00640 1 --2- 172.21.15.152:0/877865051 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0db80d05c0 0x7f0db8155700 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f0da000c400 tx=0x7f0da000c8c0 comp rx=0 tx=0).stop 2024-04-20T11:32:30.969 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:30.968+0000 7f0dbde00640 1 -- 172.21.15.152:0/877865051 shutdown_connections 2024-04-20T11:32:30.969 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:30.968+0000 7f0dbde00640 1 --2- 172.21.15.152:0/877865051 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f0d9406ca10 0x7f0d9406eed0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:30.969 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:30.968+0000 7f0dbde00640 1 --2- 172.21.15.152:0/877865051 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0db815b5c0 0x7f0db81e47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:30.969 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:30.968+0000 7f0dbde00640 1 --2- 172.21.15.152:0/877865051 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0db80d05c0 0x7f0db8155700 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:30.969 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:30.968+0000 7f0dbde00640 1 --2- 172.21.15.152:0/877865051 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0db80cebd0 0x7f0db80af2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:30.969 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:30.968+0000 7f0dbde00640 1 -- 172.21.15.152:0/877865051 >> 172.21.15.152:0/877865051 conn(0x7f0db806bda0 msgr2=0x7f0db8072d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:30.970 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:30.968+0000 7f0dbde00640 1 -- 172.21.15.152:0/877865051 shutdown_connections 2024-04-20T11:32:30.970 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:30.968+0000 7f0dbde00640 1 -- 172.21.15.152:0/877865051 wait complete. 2024-04-20T11:32:30.994 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-20T11:32:31.194 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:31.193+0000 7f0136a00640 1 Processor -- start 2024-04-20T11:32:31.194 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:31.193+0000 7f0136a00640 1 -- start start 2024-04-20T11:32:31.195 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:31.194+0000 7f0136a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f013015e6c0 0x7f013006dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:31.195 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:31.195+0000 7f0136a00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f013006f550 0x7f01300d05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:31.195 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:31.195+0000 7f0136a00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f013006ec10 con 0x7f01300d0b30 2024-04-20T11:32:31.195 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:31.195+0000 7f0136a00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f01300b50f0 con 0x7f013006f550 2024-04-20T11:32:31.195 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:31.195+0000 7f0136a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f013006e370 con 0x7f013015e6c0 2024-04-20T11:32:31.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:31.195+0000 7f0135600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f013015e6c0 0x7f013006dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:31.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:31.195+0000 7f0134c00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f013006f550 0x7f01300d05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:31.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:31.195+0000 7f0135600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f013015e6c0 0x7f013006dc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.152:56684/0 (socket says 172.21.15.152:56684) 2024-04-20T11:32:31.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:31.195+0000 7f0135600640 1 -- 172.21.15.152:0/4185634669 learned_addr learned my addr 172.21.15.152:0/4185634669 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:32:31.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:31.196+0000 7f0135600640 1 -- 172.21.15.152:0/4185634669 >> v2:172.21.15.175:3300/0 conn(0x7f013006f550 msgr2=0x7f01300d05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:31.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:31.196+0000 7f0135600640 1 --2- 172.21.15.152:0/4185634669 >> v2:172.21.15.175:3300/0 conn(0x7f013006f550 0x7f01300d05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:31.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:31.196+0000 7f0135600640 1 -- 172.21.15.152:0/4185634669 >> v1:172.21.15.175:6789/0 conn(0x7f01300d0b30 legacy=0x7f01300b36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:31.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:31.196+0000 7f0135600640 1 -- 172.21.15.152:0/4185634669 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f013006e650 con 0x7f013015e6c0 2024-04-20T11:32:31.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:31.198+0000 7f0135600640 1 --2- 172.21.15.152:0/4185634669 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f013015e6c0 0x7f013006dc70 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0124009870 tx=0x7f012402b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29a00aaeedbb5195 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:31.199 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:31.199+0000 7f012fe00640 1 -- 172.21.15.152:0/4185634669 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0124034070 con 0x7f013015e6c0 2024-04-20T11:32:31.199 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:31.199+0000 7f012fe00640 1 -- 172.21.15.152:0/4185634669 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0124030840 con 0x7f013015e6c0 2024-04-20T11:32:31.199 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:31.199+0000 7f012fe00640 1 -- 172.21.15.152:0/4185634669 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0124030c10 con 0x7f013015e6c0 2024-04-20T11:32:31.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:31.200+0000 7f0136a00640 1 -- 172.21.15.152:0/4185634669 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f013015e6c0 msgr2=0x7f013006dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:31.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:31.200+0000 7f0136a00640 1 --2- 172.21.15.152:0/4185634669 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f013015e6c0 0x7f013006dc70 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0124009870 tx=0x7f012402b980 comp rx=0 tx=0).stop 2024-04-20T11:32:31.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:31.200+0000 7f0136a00640 1 -- 172.21.15.152:0/4185634669 shutdown_connections 2024-04-20T11:32:31.201 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:31.200+0000 7f0136a00640 1 --2- 172.21.15.152:0/4185634669 >> v2:172.21.15.175:3300/0 conn(0x7f013006f550 0x7f01300d05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:31.201 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:31.200+0000 7f0136a00640 1 --2- 172.21.15.152:0/4185634669 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f013015e6c0 0x7f013006dc70 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:31.201 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:31.200+0000 7f0136a00640 1 -- 172.21.15.152:0/4185634669 >> 172.21.15.152:0/4185634669 conn(0x7f01300b21e0 msgr2=0x7f01300b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:31.201 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:31.200+0000 7f0136a00640 1 -- 172.21.15.152:0/4185634669 shutdown_connections 2024-04-20T11:32:31.201 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:31.200+0000 7f0136a00640 1 -- 172.21.15.152:0/4185634669 wait complete. 2024-04-20T11:32:31.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:31.202+0000 7f0136a00640 1 Processor -- start 2024-04-20T11:32:31.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:31.202+0000 7f0136a00640 1 -- start start 2024-04-20T11:32:31.205 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:31.203+0000 7f0136a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f013006f550 0x7f0130149eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:31.205 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:31.203+0000 7f0136a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f01300d0b30 0x7f013009cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:31.205 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:31.203+0000 7f0136a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f013015e6c0 0x7f013014b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:31.205 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:31.203+0000 7f0136a00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f013015eaa0 con 0x7f01300d0b30 2024-04-20T11:32:31.205 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:31.203+0000 7f0136a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f01300ad140 con 0x7f013006f550 2024-04-20T11:32:31.205 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:31.203+0000 7f0136a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f01300ae670 con 0x7f013015e6c0 2024-04-20T11:32:31.205 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:31.204+0000 7f0134c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f01300d0b30 0x7f013009cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:31.205 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:31.204+0000 7f0134c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f01300d0b30 0x7f013009cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:60214/0 (socket says 172.21.15.152:60214) 2024-04-20T11:32:31.205 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:31.204+0000 7f0134c00640 1 -- 172.21.15.152:0/3421333133 learned_addr learned my addr 172.21.15.152:0/3421333133 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:32:31.205 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:31.204+0000 7f0136000640 1 --2- 172.21.15.152:0/3421333133 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f013015e6c0 0x7f013014b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:31.205 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:31.204+0000 7f0135600640 1 --2- 172.21.15.152:0/3421333133 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f013006f550 0x7f0130149eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:31.205 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:31.204+0000 7f0134c00640 1 -- 172.21.15.152:0/3421333133 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f013015e6c0 msgr2=0x7f013014b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:31.206 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:31.204+0000 7f0134c00640 1 --2- 172.21.15.152:0/3421333133 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f013015e6c0 0x7f013014b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:31.206 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:31.204+0000 7f0134c00640 1 -- 172.21.15.152:0/3421333133 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f013006f550 msgr2=0x7f0130149eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:31.206 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:31.204+0000 7f0134c00640 1 --2- 172.21.15.152:0/3421333133 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f013006f550 0x7f0130149eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:31.206 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:31.204+0000 7f0134c00640 1 -- 172.21.15.152:0/3421333133 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f013014a490 con 0x7f01300d0b30 2024-04-20T11:32:31.207 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:31.205+0000 7f0134c00640 1 --2- 172.21.15.152:0/3421333133 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f01300d0b30 0x7f013009cf40 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f011c00e9b0 tx=0x7f011c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:31.207 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:31.205+0000 7f012d600640 1 -- 172.21.15.152:0/3421333133 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f011c017070 con 0x7f01300d0b30 2024-04-20T11:32:31.207 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:31.205+0000 7f012d600640 1 -- 172.21.15.152:0/3421333133 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f011c00c9a0 con 0x7f01300d0b30 2024-04-20T11:32:31.207 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:31.205+0000 7f012d600640 1 -- 172.21.15.152:0/3421333133 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f011c00cd90 con 0x7f01300d0b30 2024-04-20T11:32:31.207 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:31.205+0000 7f0136a00640 1 -- 172.21.15.152:0/3421333133 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f013014a770 con 0x7f01300d0b30 2024-04-20T11:32:31.207 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:31.205+0000 7f0136a00640 1 -- 172.21.15.152:0/3421333133 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f013014b010 con 0x7f01300d0b30 2024-04-20T11:32:31.207 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:31.206+0000 7f0136a00640 1 -- 172.21.15.152:0/3421333133 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0130075e60 con 0x7f01300d0b30 2024-04-20T11:32:31.208 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:31.207+0000 7f012d600640 1 -- 172.21.15.152:0/3421333133 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 91142+0+0 (secure 0 0 0) 0x7f011c01a070 con 0x7f01300d0b30 2024-04-20T11:32:31.209 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:31.208+0000 7f012d600640 1 --2- 172.21.15.152:0/3421333133 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f011006ca10 0x7f011006eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:31.209 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:31.209+0000 7f012d600640 1 -- 172.21.15.152:0/3421333133 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5916+0+0 (secure 0 0 0) 0x7f011c08c1b0 con 0x7f01300d0b30 2024-04-20T11:32:31.209 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:31.209+0000 7f0135600640 1 --2- 172.21.15.152:0/3421333133 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f011006ca10 0x7f011006eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:31.211 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:31.210+0000 7f0135600640 1 --2- 172.21.15.152:0/3421333133 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f011006ca10 0x7f011006eed0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0124036000 tx=0x7f012403b040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:31.213 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:31.213+0000 7f012d600640 1 -- 172.21.15.152:0/3421333133 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f011c05a3f0 con 0x7f01300d0b30 2024-04-20T11:32:31.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:31.414+0000 7f0136a00640 1 -- 172.21.15.152:0/3421333133 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f013014a940 con 0x7f01300d0b30 2024-04-20T11:32:31.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:31.975+0000 7f012d600640 1 -- 172.21.15.152:0/3421333133 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v75) ==== 169+0+0 (secure 0 0 0) 0x7f011c059cf0 con 0x7f01300d0b30 2024-04-20T11:32:31.976 INFO:teuthology.orchestra.run.smithi152.stderr:pool 'cephfs_data' removed 2024-04-20T11:32:31.980 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:31.979+0000 7f0136a00640 1 -- 172.21.15.152:0/3421333133 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f011006ca10 msgr2=0x7f011006eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:31.980 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:31.979+0000 7f0136a00640 1 --2- 172.21.15.152:0/3421333133 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f011006ca10 0x7f011006eed0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0124036000 tx=0x7f012403b040 comp rx=0 tx=0).stop 2024-04-20T11:32:31.980 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:31.979+0000 7f0136a00640 1 -- 172.21.15.152:0/3421333133 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f01300d0b30 msgr2=0x7f013009cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:31.980 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:31.979+0000 7f0136a00640 1 --2- 172.21.15.152:0/3421333133 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f01300d0b30 0x7f013009cf40 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f011c00e9b0 tx=0x7f011c00ee70 comp rx=0 tx=0).stop 2024-04-20T11:32:31.980 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:31.979+0000 7f0136a00640 1 -- 172.21.15.152:0/3421333133 shutdown_connections 2024-04-20T11:32:31.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:31.979+0000 7f0136a00640 1 --2- 172.21.15.152:0/3421333133 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f011006ca10 0x7f011006eed0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:31.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:31.979+0000 7f0136a00640 1 --2- 172.21.15.152:0/3421333133 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f013015e6c0 0x7f013014b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:31.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:31.979+0000 7f0136a00640 1 --2- 172.21.15.152:0/3421333133 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f01300d0b30 0x7f013009cf40 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:31.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:31.979+0000 7f0136a00640 1 --2- 172.21.15.152:0/3421333133 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f013006f550 0x7f0130149eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:31.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:31.980+0000 7f0136a00640 1 -- 172.21.15.152:0/3421333133 >> 172.21.15.152:0/3421333133 conn(0x7f01300b21e0 msgr2=0x7f0130160c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:31.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:31.980+0000 7f0136a00640 1 -- 172.21.15.152:0/3421333133 shutdown_connections 2024-04-20T11:32:31.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:31.980+0000 7f0136a00640 1 -- 172.21.15.152:0/3421333133 wait complete. 2024-04-20T11:32:32.012 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-20T11:32:32.193 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.192+0000 7f51e6400640 1 Processor -- start 2024-04-20T11:32:32.193 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.192+0000 7f51e6400640 1 -- start start 2024-04-20T11:32:32.195 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.194+0000 7f51e6400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f51e015eb50 0x7f51e00d0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:32.195 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.194+0000 7f51e6400640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51e00d19c0 con 0x7f51e00d35a0 2024-04-20T11:32:32.195 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.194+0000 7f51e6400640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51e0162750 con 0x7f51e0163230 2024-04-20T11:32:32.195 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.194+0000 7f51e6400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f51e0162a30 con 0x7f51e015eb50 2024-04-20T11:32:32.195 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.194+0000 7f51e5a00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f51e00d35a0 0x7f51e00cf470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:33700/0 (socket says 172.21.15.152:33700) 2024-04-20T11:32:32.195 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.194+0000 7f51e5a00640 1 -- 172.21.15.152:0/4030630569 learned_addr learned my addr 172.21.15.152:0/4030630569 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:32:32.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.195+0000 7f51e5000640 1 --2- 172.21.15.152:0/4030630569 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f51e015eb50 0x7f51e00d0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:32.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.195+0000 7f51df400640 1 -- 172.21.15.152:0/4030630569 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2409501585 0 0) 0x7f51e00d19c0 con 0x7f51e00d35a0 2024-04-20T11:32:32.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.195+0000 7f51df400640 1 -- 172.21.15.152:0/4030630569 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f51c8003520 con 0x7f51e00d35a0 2024-04-20T11:32:32.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.195+0000 7f51df400640 1 -- 172.21.15.152:0/4030630569 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2015434950 0 0) 0x7f51c8003520 con 0x7f51e00d35a0 2024-04-20T11:32:32.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.195+0000 7f51df400640 1 -- 172.21.15.152:0/4030630569 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f51e015eb50 msgr2=0x7f51e00d0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:32.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.195+0000 7f51df400640 1 --2- 172.21.15.152:0/4030630569 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f51e015eb50 0x7f51e00d0f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:32.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.195+0000 7f51df400640 1 -- 172.21.15.152:0/4030630569 >> v1:172.21.15.175:6789/0 conn(0x7f51e0163230 legacy=0x7f51e00d2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:32.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.195+0000 7f51df400640 1 -- 172.21.15.152:0/4030630569 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51e0162d10 con 0x7f51e00d35a0 2024-04-20T11:32:32.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.195+0000 7f51df400640 1 -- 172.21.15.152:0/4030630569 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f51d80019f0 con 0x7f51e00d35a0 2024-04-20T11:32:32.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.195+0000 7f51df400640 1 -- 172.21.15.152:0/4030630569 >> v1:172.21.15.152:6789/0 conn(0x7f51e00d35a0 legacy=0x7f51e00cf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:32.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.195+0000 7f51df400640 1 --2- 172.21.15.152:0/4030630569 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f51e015eb50 0x7f51c80039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:32.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.195+0000 7f51df400640 1 --2- 172.21.15.152:0/4030630569 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f51c8003f60 0x7f51c8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:32.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.195+0000 7f51df400640 1 --2- 172.21.15.152:0/4030630569 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51c8004d40 0x7f51c80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:32.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.195+0000 7f51df400640 1 -- 172.21.15.152:0/4030630569 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f51e0162a30 con 0x7f51c8003f60 2024-04-20T11:32:32.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.195+0000 7f51df400640 1 -- 172.21.15.152:0/4030630569 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f51c800bb50 con 0x7f51c8004d40 2024-04-20T11:32:32.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.195+0000 7f51df400640 1 -- 172.21.15.152:0/4030630569 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f51c800bd50 con 0x7f51e015eb50 2024-04-20T11:32:32.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.196+0000 7f51dfe00640 1 --2- 172.21.15.152:0/4030630569 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f51c8003f60 0x7f51c8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:32.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.196+0000 7f51e5000640 1 --2- 172.21.15.152:0/4030630569 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f51e015eb50 0x7f51c80039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:32.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.196+0000 7f51e5a00640 1 --2- 172.21.15.152:0/4030630569 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51c8004d40 0x7f51c80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:32.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.196+0000 7f51dfe00640 1 -- 172.21.15.152:0/4030630569 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f51e015eb50 msgr2=0x7f51c80039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:32.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.196+0000 7f51dfe00640 1 --2- 172.21.15.152:0/4030630569 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f51e015eb50 0x7f51c80039e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:32.199 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.196+0000 7f51dfe00640 1 -- 172.21.15.152:0/4030630569 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51c8004d40 msgr2=0x7f51c80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:32.199 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.196+0000 7f51dfe00640 1 --2- 172.21.15.152:0/4030630569 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51c8004d40 0x7f51c80051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:32.199 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.196+0000 7f51dfe00640 1 -- 172.21.15.152:0/4030630569 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51c8003520 con 0x7f51c8003f60 2024-04-20T11:32:32.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.197+0000 7f51dfe00640 1 --2- 172.21.15.152:0/4030630569 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f51c8003f60 0x7f51c8004800 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f51cc001770 tx=0x7f51c8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c3d34a01ba35e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:32.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.197+0000 7f51df400640 1 -- 172.21.15.152:0/4030630569 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51cc0013c0 con 0x7f51c8003f60 2024-04-20T11:32:32.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.197+0000 7f51df400640 1 -- 172.21.15.152:0/4030630569 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51cc0052d0 con 0x7f51c8003f60 2024-04-20T11:32:32.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.198+0000 7f51df400640 1 -- 172.21.15.152:0/4030630569 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51cc003640 con 0x7f51c8003f60 2024-04-20T11:32:32.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.198+0000 7f51e6400640 1 -- 172.21.15.152:0/4030630569 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f51c8003f60 msgr2=0x7f51c8004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:32.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.198+0000 7f51e6400640 1 --2- 172.21.15.152:0/4030630569 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f51c8003f60 0x7f51c8004800 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f51cc001770 tx=0x7f51c8005830 comp rx=0 tx=0).stop 2024-04-20T11:32:32.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.198+0000 7f51e5a00640 1 -- 172.21.15.152:0/4030630569 reap_dead start 2024-04-20T11:32:32.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.198+0000 7f51e6400640 1 -- 172.21.15.152:0/4030630569 shutdown_connections 2024-04-20T11:32:32.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.198+0000 7f51e6400640 1 -- 172.21.15.152:0/4030630569 >> 172.21.15.152:0/4030630569 conn(0x7f51e008bfd0 msgr2=0x7f51e00d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:32.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.198+0000 7f51e6400640 1 -- 172.21.15.152:0/4030630569 shutdown_connections 2024-04-20T11:32:32.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.198+0000 7f51e6400640 1 -- 172.21.15.152:0/4030630569 wait complete. 2024-04-20T11:32:32.201 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.200+0000 7f51e6400640 1 Processor -- start 2024-04-20T11:32:32.201 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.200+0000 7f51e6400640 1 -- start start 2024-04-20T11:32:32.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.201+0000 7f51e6400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f51c8003f60 0x7f51e014da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:32.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.201+0000 7f51e6400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51e00b70e0 0x7f51e01e8db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:32.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.201+0000 7f51e6400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f51e00b43b0 0x7f51e00b4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:32.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.201+0000 7f51e6400640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f51e00d1080 con 0x7f51e00b43b0 2024-04-20T11:32:32.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.201+0000 7f51e6400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f51e00d0f00 con 0x7f51e00b70e0 2024-04-20T11:32:32.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.201+0000 7f51e6400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f51e00d1200 con 0x7f51c8003f60 2024-04-20T11:32:32.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.201+0000 7f51e5a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f51e00b43b0 0x7f51e00b4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:32.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.201+0000 7f51e5a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f51e00b43b0 0x7f51e00b4840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:60232/0 (socket says 172.21.15.152:60232) 2024-04-20T11:32:32.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.201+0000 7f51e5a00640 1 -- 172.21.15.152:0/1894000842 learned_addr learned my addr 172.21.15.152:0/1894000842 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:32:32.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.202+0000 7f51e5000640 1 --2- 172.21.15.152:0/1894000842 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f51c8003f60 0x7f51e014da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:32.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.202+0000 7f51dfe00640 1 --2- 172.21.15.152:0/1894000842 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51e00b70e0 0x7f51e01e8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:32.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.202+0000 7f51e5a00640 1 -- 172.21.15.152:0/1894000842 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f51c8003f60 msgr2=0x7f51e014da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:32.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.202+0000 7f51e5a00640 1 --2- 172.21.15.152:0/1894000842 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f51c8003f60 0x7f51e014da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:32.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.202+0000 7f51e5a00640 1 -- 172.21.15.152:0/1894000842 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51e00b70e0 msgr2=0x7f51e01e8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:32.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.202+0000 7f51e5a00640 1 --2- 172.21.15.152:0/1894000842 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51e00b70e0 0x7f51e01e8db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:32.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.202+0000 7f51e5a00640 1 -- 172.21.15.152:0/1894000842 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51e00b75c0 con 0x7f51e00b43b0 2024-04-20T11:32:32.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.202+0000 7f51e5a00640 1 --2- 172.21.15.152:0/1894000842 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f51e00b43b0 0x7f51e00b4840 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f51d8000f80 tx=0x7f51d800b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:32.204 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.202+0000 7f51dcc00640 1 -- 172.21.15.152:0/1894000842 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51d800de60 con 0x7f51e00b43b0 2024-04-20T11:32:32.204 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.202+0000 7f51dcc00640 1 -- 172.21.15.152:0/1894000842 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51d800bb20 con 0x7f51e00b43b0 2024-04-20T11:32:32.204 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.202+0000 7f51e6400640 1 -- 172.21.15.152:0/1894000842 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51e00b78a0 con 0x7f51e00b43b0 2024-04-20T11:32:32.204 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.202+0000 7f51dcc00640 1 -- 172.21.15.152:0/1894000842 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51d8007070 con 0x7f51e00b43b0 2024-04-20T11:32:32.205 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.203+0000 7f51e6400640 1 -- 172.21.15.152:0/1894000842 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51e00b7e60 con 0x7f51e00b43b0 2024-04-20T11:32:32.205 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.204+0000 7f51e6400640 1 -- 172.21.15.152:0/1894000842 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51e0075e60 con 0x7f51e00b43b0 2024-04-20T11:32:32.205 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.205+0000 7f51dcc00640 1 -- 172.21.15.152:0/1894000842 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 91142+0+0 (secure 0 0 0) 0x7f51d801b060 con 0x7f51e00b43b0 2024-04-20T11:32:32.206 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.206+0000 7f51dcc00640 1 --2- 172.21.15.152:0/1894000842 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f51c006ca40 0x7f51c006ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:32.206 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.206+0000 7f51dcc00640 1 -- 172.21.15.152:0/1894000842 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5502+0+0 (secure 0 0 0) 0x7f51d808bc50 con 0x7f51e00b43b0 2024-04-20T11:32:32.207 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.207+0000 7f51e5000640 1 --2- 172.21.15.152:0/1894000842 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f51c006ca40 0x7f51c006ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:32.208 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.208+0000 7f51e5000640 1 --2- 172.21.15.152:0/1894000842 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f51c006ca40 0x7f51c006ef00 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f51d4009a10 tx=0x7f51d4008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:32.211 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.210+0000 7f51dcc00640 1 -- 172.21.15.152:0/1894000842 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f51d8059e90 con 0x7f51e00b43b0 2024-04-20T11:32:32.412 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.412+0000 7f51e6400640 1 -- 172.21.15.152:0/1894000842 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f51e00cad50 con 0x7f51e00b43b0 2024-04-20T11:32:32.418 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.417+0000 7f51dcc00640 1 -- 172.21.15.152:0/1894000842 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) ==== 104+0+0 (secure 0 0 0) 0x7f51d8059790 con 0x7f51e00b43b0 2024-04-20T11:32:32.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.420+0000 7f51e6400640 1 -- 172.21.15.152:0/1894000842 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f51c006ca40 msgr2=0x7f51c006ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:32.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.420+0000 7f51e6400640 1 --2- 172.21.15.152:0/1894000842 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f51c006ca40 0x7f51c006ef00 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f51d4009a10 tx=0x7f51d4008040 comp rx=0 tx=0).stop 2024-04-20T11:32:32.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.420+0000 7f51e6400640 1 -- 172.21.15.152:0/1894000842 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f51e00b43b0 msgr2=0x7f51e00b4840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:32.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.420+0000 7f51e6400640 1 --2- 172.21.15.152:0/1894000842 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f51e00b43b0 0x7f51e00b4840 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f51d8000f80 tx=0x7f51d800b4a0 comp rx=0 tx=0).stop 2024-04-20T11:32:32.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.420+0000 7f51e6400640 1 -- 172.21.15.152:0/1894000842 shutdown_connections 2024-04-20T11:32:32.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.420+0000 7f51e6400640 1 --2- 172.21.15.152:0/1894000842 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f51c006ca40 0x7f51c006ef00 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:32.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.420+0000 7f51e6400640 1 --2- 172.21.15.152:0/1894000842 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f51e00b43b0 0x7f51e00b4840 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:32.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.420+0000 7f51e6400640 1 --2- 172.21.15.152:0/1894000842 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51e00b70e0 0x7f51e01e8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:32.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.420+0000 7f51e6400640 1 --2- 172.21.15.152:0/1894000842 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f51c8003f60 0x7f51e014da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:32.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.420+0000 7f51e6400640 1 -- 172.21.15.152:0/1894000842 >> 172.21.15.152:0/1894000842 conn(0x7f51e008bfd0 msgr2=0x7f51e00cf430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:32.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.420+0000 7f51e6400640 1 -- 172.21.15.152:0/1894000842 shutdown_connections 2024-04-20T11:32:32.422 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.420+0000 7f51e6400640 1 -- 172.21.15.152:0/1894000842 wait complete. 2024-04-20T11:32:32.440 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_mgr_eviction' 2024-04-20T11:32:32.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.640+0000 7f080c000640 1 Processor -- start 2024-04-20T11:32:32.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.641+0000 7f080c000640 1 -- start start 2024-04-20T11:32:32.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.642+0000 7f080c000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f080415edc0 0x7f080409b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:32.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.642+0000 7f080c000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f08040b4be0 0x7f080406e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:32.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.642+0000 7f080c000640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f080409b710 con 0x7f08040d0bc0 2024-04-20T11:32:32.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.642+0000 7f080c000640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f080406d050 con 0x7f080415edc0 2024-04-20T11:32:32.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.642+0000 7f080c000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f080415f1a0 con 0x7f08040b4be0 2024-04-20T11:32:32.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.642+0000 7f0809600640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f080415edc0 0x7f080409b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:32.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.642+0000 7f080a000640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f08040d0bc0 0x7f08040ccc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:33706/0 (socket says 172.21.15.152:33706) 2024-04-20T11:32:32.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.643+0000 7f080a000640 1 -- 172.21.15.152:0/708872429 learned_addr learned my addr 172.21.15.152:0/708872429 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:32:32.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.643+0000 7f0808c00640 1 --2- 172.21.15.152:0/708872429 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f08040b4be0 0x7f080406e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:32.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.643+0000 7f0809600640 1 -- 172.21.15.152:0/708872429 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f08040b4be0 msgr2=0x7f080406e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:32.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.643+0000 7f0809600640 1 --2- 172.21.15.152:0/708872429 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f08040b4be0 0x7f080406e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:32.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.643+0000 7f0809600640 1 -- 172.21.15.152:0/708872429 >> v1:172.21.15.152:6789/0 conn(0x7f08040d0bc0 legacy=0x7f08040ccc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:32.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.643+0000 7f0809600640 1 -- 172.21.15.152:0/708872429 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f080406d2f0 con 0x7f080415edc0 2024-04-20T11:32:32.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.643+0000 7f0803e00640 1 -- 172.21.15.152:0/708872429 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2427286134 0 0) 0x7f080409b710 con 0x7f08040d0bc0 2024-04-20T11:32:32.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.645+0000 7f0809600640 1 --2- 172.21.15.152:0/708872429 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f080415edc0 0x7f080409b910 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f07f80097a0 tx=0x7f07f802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fadab2566c7fb13 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:32.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.645+0000 7f0803e00640 1 -- 172.21.15.152:0/708872429 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07f8035020 con 0x7f080415edc0 2024-04-20T11:32:32.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.645+0000 7f0803e00640 1 -- 172.21.15.152:0/708872429 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07f8031830 con 0x7f080415edc0 2024-04-20T11:32:32.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.645+0000 7f0803e00640 1 -- 172.21.15.152:0/708872429 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07f8031c00 con 0x7f080415edc0 2024-04-20T11:32:32.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.646+0000 7f080c000640 1 -- 172.21.15.152:0/708872429 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f080415edc0 msgr2=0x7f080409b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:32.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.646+0000 7f080c000640 1 --2- 172.21.15.152:0/708872429 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f080415edc0 0x7f080409b910 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f07f80097a0 tx=0x7f07f802b8d0 comp rx=0 tx=0).stop 2024-04-20T11:32:32.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.647+0000 7f080c000640 1 -- 172.21.15.152:0/708872429 shutdown_connections 2024-04-20T11:32:32.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.647+0000 7f080c000640 1 --2- 172.21.15.152:0/708872429 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f08040b4be0 0x7f080406e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:32.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.647+0000 7f080c000640 1 --2- 172.21.15.152:0/708872429 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f080415edc0 0x7f080409b910 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:32.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.647+0000 7f080c000640 1 -- 172.21.15.152:0/708872429 >> 172.21.15.152:0/708872429 conn(0x7f08040b40c0 msgr2=0x7f08040b44d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:32.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.647+0000 7f080c000640 1 -- 172.21.15.152:0/708872429 shutdown_connections 2024-04-20T11:32:32.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.647+0000 7f080c000640 1 -- 172.21.15.152:0/708872429 wait complete. 2024-04-20T11:32:32.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.649+0000 7f080c000640 1 Processor -- start 2024-04-20T11:32:32.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.651+0000 7f080c000640 1 -- start start 2024-04-20T11:32:32.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.652+0000 7f080c000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f08040b4be0 0x7f080415fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:32.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.653+0000 7f080c000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f08040d0bc0 0x7f0804158ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:32.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.653+0000 7f080c000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f080415edc0 0x7f08040c7c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:32.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.653+0000 7f080c000640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f080406db90 con 0x7f08040b4be0 2024-04-20T11:32:32.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.653+0000 7f080c000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f080406d8b0 con 0x7f08040d0bc0 2024-04-20T11:32:32.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.653+0000 7f080c000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f080406de70 con 0x7f080415edc0 2024-04-20T11:32:32.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.653+0000 7f0809600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f08040b4be0 0x7f080415fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:32.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.653+0000 7f0809600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f08040b4be0 0x7f080415fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:60252/0 (socket says 172.21.15.152:60252) 2024-04-20T11:32:32.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.653+0000 7f0809600640 1 -- 172.21.15.152:0/2763492828 learned_addr learned my addr 172.21.15.152:0/2763492828 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:32:32.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.655+0000 7f0808c00640 1 --2- 172.21.15.152:0/2763492828 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f08040d0bc0 0x7f0804158ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:32.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.655+0000 7f0809600640 1 -- 172.21.15.152:0/2763492828 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f080415edc0 msgr2=0x7f08040c7c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:32.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.655+0000 7f0809600640 1 --2- 172.21.15.152:0/2763492828 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f080415edc0 0x7f08040c7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:32.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.655+0000 7f0809600640 1 -- 172.21.15.152:0/2763492828 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f08040d0bc0 msgr2=0x7f0804158ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:32.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.655+0000 7f0809600640 1 --2- 172.21.15.152:0/2763492828 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f08040d0bc0 0x7f0804158ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:32.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.655+0000 7f0809600640 1 -- 172.21.15.152:0/2763492828 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0804160460 con 0x7f08040b4be0 2024-04-20T11:32:32.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.656+0000 7f0809600640 1 --2- 172.21.15.152:0/2763492828 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f08040b4be0 0x7f080415fd60 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f07f80098c0 tx=0x7f07f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:32.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.657+0000 7f0801600640 1 -- 172.21.15.152:0/2763492828 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07f8041070 con 0x7f08040b4be0 2024-04-20T11:32:32.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.657+0000 7f0801600640 1 -- 172.21.15.152:0/2763492828 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07f803b920 con 0x7f08040b4be0 2024-04-20T11:32:32.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.657+0000 7f0801600640 1 -- 172.21.15.152:0/2763492828 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07f8035040 con 0x7f08040b4be0 2024-04-20T11:32:32.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.657+0000 7f080c000640 1 -- 172.21.15.152:0/2763492828 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0804160740 con 0x7f08040b4be0 2024-04-20T11:32:32.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.657+0000 7f080c000640 1 -- 172.21.15.152:0/2763492828 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0804160d00 con 0x7f08040b4be0 2024-04-20T11:32:32.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.659+0000 7f080c000640 1 -- 172.21.15.152:0/2763492828 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08040689f0 con 0x7f08040b4be0 2024-04-20T11:32:32.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.662+0000 7f0801600640 1 -- 172.21.15.152:0/2763492828 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 91142+0+0 (secure 0 0 0) 0x7f07f804c060 con 0x7f08040b4be0 2024-04-20T11:32:32.664 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.663+0000 7f0801600640 1 --2- 172.21.15.152:0/2763492828 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f07e406ca60 0x7f07e406ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:32.664 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.663+0000 7f0801600640 1 -- 172.21.15.152:0/2763492828 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5502+0+0 (secure 0 0 0) 0x7f07f80ad7e0 con 0x7f08040b4be0 2024-04-20T11:32:32.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.665+0000 7f0808c00640 1 --2- 172.21.15.152:0/2763492828 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f07e406ca60 0x7f07e406ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:32.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.667+0000 7f0808c00640 1 --2- 172.21.15.152:0/2763492828 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f07e406ca60 0x7f07e406ef20 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f07f0006fd0 tx=0x7f07f0008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:32.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.667+0000 7f0801600640 1 -- 172.21.15.152:0/2763492828 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f07f8031800 con 0x7f08040b4be0 2024-04-20T11:32:32.945 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.944+0000 7f080c000640 1 -- 172.21.15.152:0/2763492828 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_mgr_eviction"]} v 0) -- 0x7f08040b8550 con 0x7f08040b4be0 2024-04-20T11:32:32.979 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.978+0000 7f0801600640 1 -- 172.21.15.152:0/2763492828 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_mgr_eviction"]}]=0 v300) ==== 131+0+0 (secure 0 0 0) 0x7f07f807bd40 con 0x7f08040b4be0 2024-04-20T11:32:32.982 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.981+0000 7f080c000640 1 -- 172.21.15.152:0/2763492828 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f07e406ca60 msgr2=0x7f07e406ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:32.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.981+0000 7f080c000640 1 --2- 172.21.15.152:0/2763492828 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f07e406ca60 0x7f07e406ef20 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f07f0006fd0 tx=0x7f07f0008040 comp rx=0 tx=0).stop 2024-04-20T11:32:32.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.981+0000 7f080c000640 1 -- 172.21.15.152:0/2763492828 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f08040b4be0 msgr2=0x7f080415fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:32.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.981+0000 7f080c000640 1 --2- 172.21.15.152:0/2763492828 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f08040b4be0 0x7f080415fd60 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f07f80098c0 tx=0x7f07f80046c0 comp rx=0 tx=0).stop 2024-04-20T11:32:32.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.981+0000 7f080c000640 1 -- 172.21.15.152:0/2763492828 shutdown_connections 2024-04-20T11:32:32.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.981+0000 7f080c000640 1 --2- 172.21.15.152:0/2763492828 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f07e406ca60 0x7f07e406ef20 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:32.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.981+0000 7f080c000640 1 --2- 172.21.15.152:0/2763492828 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f080415edc0 0x7f08040c7c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:32.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.981+0000 7f080c000640 1 --2- 172.21.15.152:0/2763492828 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f08040d0bc0 0x7f0804158ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:32.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.981+0000 7f080c000640 1 --2- 172.21.15.152:0/2763492828 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f08040b4be0 0x7f080415fd60 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:32.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.981+0000 7f080c000640 1 -- 172.21.15.152:0/2763492828 >> 172.21.15.152:0/2763492828 conn(0x7f08040b40c0 msgr2=0x7f08040cee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:32.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.982+0000 7f080c000640 1 -- 172.21.15.152:0/2763492828 shutdown_connections 2024-04-20T11:32:32.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:32.982+0000 7f080c000640 1 -- 172.21.15.152:0/2763492828 wait complete. 2024-04-20T11:32:33.002 INFO:tasks.cephfs_test_runner:test_mgr_eviction (tasks.cephfs.test_volumes.TestMisc) ... ok 2024-04-20T11:32:33.002 INFO:tasks.cephfs_test_runner:Starting test: test_names_can_only_be_goodchars (tasks.cephfs.test_volumes.TestMisc) Test the creating vols, subvols subvolgroups fails when their names uses 2024-04-20T11:32:33.003 INFO:tasks.cephfs_test_runner:test_names_can_only_be_goodchars (tasks.cephfs.test_volumes.TestMisc) 2024-04-20T11:32:33.003 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:32:33.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.203+0000 7fb47e800640 1 Processor -- start 2024-04-20T11:32:33.204 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.203+0000 7fb47e800640 1 -- start start 2024-04-20T11:32:33.206 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.205+0000 7fb47e800640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fb4780cc9b0 0x7fb4780b43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:33.206 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.205+0000 7fb47e800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb4780ccd90 0x7fb4780b4930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:33.206 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.205+0000 7fb47e800640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb47815b890 con 0x7fb47815b320 2024-04-20T11:32:33.206 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.205+0000 7fb47e800640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fb47815bb70 con 0x7fb4780cc9b0 2024-04-20T11:32:33.206 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.205+0000 7fb47e800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb47815be50 con 0x7fb4780ccd90 2024-04-20T11:32:33.207 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.206+0000 7fb46ea00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb4780ccd90 0x7fb4780b4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:33.207 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.206+0000 7fb477400640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fb4780cc9b0 0x7fb4780b43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:33.207 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.206+0000 7fb46ea00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb4780ccd90 0x7fb4780b4930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.152:56742/0 (socket says 172.21.15.152:56742) 2024-04-20T11:32:33.207 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.206+0000 7fb46ea00640 1 -- 172.21.15.152:0/2677353443 learned_addr learned my addr 172.21.15.152:0/2677353443 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:32:33.207 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.206+0000 7fb477400640 1 -- 172.21.15.152:0/2677353443 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb4780ccd90 msgr2=0x7fb4780b4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:33.207 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.206+0000 7fb477400640 1 --2- 172.21.15.152:0/2677353443 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb4780ccd90 0x7fb4780b4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:33.207 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.206+0000 7fb477400640 1 -- 172.21.15.152:0/2677353443 >> v1:172.21.15.175:6789/0 conn(0x7fb47815b320 legacy=0x7fb47815e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:33.207 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.206+0000 7fb477400640 1 -- 172.21.15.152:0/2677353443 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb47815c130 con 0x7fb4780cc9b0 2024-04-20T11:32:33.208 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.208+0000 7fb477400640 1 --2- 172.21.15.152:0/2677353443 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4780cc9b0 0x7fb4780b43f0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fb46800a940 tx=0x7fb46802cd80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56d5e7f08f867cb0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:33.209 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.208+0000 7fb476a00640 1 -- 172.21.15.152:0/2677353443 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb46803a070 con 0x7fb4780cc9b0 2024-04-20T11:32:33.209 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.208+0000 7fb476a00640 1 -- 172.21.15.152:0/2677353443 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb46802db30 con 0x7fb4780cc9b0 2024-04-20T11:32:33.209 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.208+0000 7fb476a00640 1 -- 172.21.15.152:0/2677353443 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb46800b040 con 0x7fb4780cc9b0 2024-04-20T11:32:33.210 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.209+0000 7fb47e800640 1 -- 172.21.15.152:0/2677353443 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4780cc9b0 msgr2=0x7fb4780b43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:33.210 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.209+0000 7fb47e800640 1 --2- 172.21.15.152:0/2677353443 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4780cc9b0 0x7fb4780b43f0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fb46800a940 tx=0x7fb46802cd80 comp rx=0 tx=0).stop 2024-04-20T11:32:33.210 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.209+0000 7fb47e800640 1 -- 172.21.15.152:0/2677353443 shutdown_connections 2024-04-20T11:32:33.210 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.209+0000 7fb47e800640 1 --2- 172.21.15.152:0/2677353443 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb4780ccd90 0x7fb4780b4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:33.210 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.209+0000 7fb47e800640 1 --2- 172.21.15.152:0/2677353443 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4780cc9b0 0x7fb4780b43f0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:33.210 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.209+0000 7fb47e800640 1 -- 172.21.15.152:0/2677353443 >> 172.21.15.152:0/2677353443 conn(0x7fb478067cf0 msgr2=0x7fb47806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:33.210 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.209+0000 7fb47e800640 1 -- 172.21.15.152:0/2677353443 shutdown_connections 2024-04-20T11:32:33.210 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.209+0000 7fb47e800640 1 -- 172.21.15.152:0/2677353443 wait complete. 2024-04-20T11:32:33.211 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.211+0000 7fb47e800640 1 Processor -- start 2024-04-20T11:32:33.212 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.211+0000 7fb47e800640 1 -- start start 2024-04-20T11:32:33.212 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.212+0000 7fb47e800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb4780cc9b0 0x7fb4781e4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:33.213 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.212+0000 7fb47e800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb47815b320 0x7fb478151610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:33.213 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.212+0000 7fb47e800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb4781f83a0 0x7fb4781e8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:33.213 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.212+0000 7fb47e800640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fb47806f3f0 con 0x7fb4781f83a0 2024-04-20T11:32:33.213 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.212+0000 7fb47e800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb47806f270 con 0x7fb47815b320 2024-04-20T11:32:33.213 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.212+0000 7fb47e800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb47806f570 con 0x7fb4780cc9b0 2024-04-20T11:32:33.214 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.213+0000 7fb477e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb4781f83a0 0x7fb4781e8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:33.214 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.213+0000 7fb477e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb4781f83a0 0x7fb4781e8ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:60256/0 (socket says 172.21.15.152:60256) 2024-04-20T11:32:33.214 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.213+0000 7fb477e00640 1 -- 172.21.15.152:0/21180262 learned_addr learned my addr 172.21.15.152:0/21180262 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:32:33.215 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.213+0000 7fb477400640 1 --2- 172.21.15.152:0/21180262 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb4780cc9b0 0x7fb4781e4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:33.215 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.213+0000 7fb46ea00640 1 --2- 172.21.15.152:0/21180262 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb47815b320 0x7fb478151610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:33.215 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.213+0000 7fb477e00640 1 -- 172.21.15.152:0/21180262 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb4780cc9b0 msgr2=0x7fb4781e4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:33.215 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.213+0000 7fb477e00640 1 --2- 172.21.15.152:0/21180262 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb4780cc9b0 0x7fb4781e4ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:33.215 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.213+0000 7fb477e00640 1 -- 172.21.15.152:0/21180262 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb47815b320 msgr2=0x7fb478151610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:33.215 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.213+0000 7fb477e00640 1 --2- 172.21.15.152:0/21180262 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb47815b320 0x7fb478151610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:33.215 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.213+0000 7fb477e00640 1 -- 172.21.15.152:0/21180262 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb478151b50 con 0x7fb4781f83a0 2024-04-20T11:32:33.215 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.213+0000 7fb477e00640 1 --2- 172.21.15.152:0/21180262 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb4781f83a0 0x7fb4781e8ec0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fb47000c7a0 tx=0x7fb47000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:33.215 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.214+0000 7fb46fe00640 1 -- 172.21.15.152:0/21180262 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb470003b70 con 0x7fb4781f83a0 2024-04-20T11:32:33.216 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.214+0000 7fb46fe00640 1 -- 172.21.15.152:0/21180262 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb470003d30 con 0x7fb4781f83a0 2024-04-20T11:32:33.216 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.214+0000 7fb46fe00640 1 -- 172.21.15.152:0/21180262 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb47000aa90 con 0x7fb4781f83a0 2024-04-20T11:32:33.216 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.214+0000 7fb47e800640 1 -- 172.21.15.152:0/21180262 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4781e5180 con 0x7fb4781f83a0 2024-04-20T11:32:33.216 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.214+0000 7fb47e800640 1 -- 172.21.15.152:0/21180262 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4781f7ba0 con 0x7fb4781f83a0 2024-04-20T11:32:33.216 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.215+0000 7fb47e800640 1 -- 172.21.15.152:0/21180262 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4780689f0 con 0x7fb4781f83a0 2024-04-20T11:32:33.216 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.216+0000 7fb46fe00640 1 -- 172.21.15.152:0/21180262 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 91142+0+0 (secure 0 0 0) 0x7fb47000ac30 con 0x7fb4781f83a0 2024-04-20T11:32:33.217 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.217+0000 7fb46fe00640 1 --2- 172.21.15.152:0/21180262 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb45406ca10 0x7fb45406eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:33.217 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.217+0000 7fb46fe00640 1 -- 172.21.15.152:0/21180262 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5502+0+0 (secure 0 0 0) 0x7fb47008bb00 con 0x7fb4781f83a0 2024-04-20T11:32:33.218 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.218+0000 7fb477400640 1 --2- 172.21.15.152:0/21180262 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb45406ca10 0x7fb45406eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:33.219 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.219+0000 7fb477400640 1 --2- 172.21.15.152:0/21180262 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb45406ca10 0x7fb45406eed0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb468037000 tx=0x7fb46800c040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:33.222 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.222+0000 7fb46fe00640 1 -- 172.21.15.152:0/21180262 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fb470059ef0 con 0x7fb4781f83a0 2024-04-20T11:32:33.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.425+0000 7fb47e800640 1 -- 172.21.15.152:0/21180262 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb4781e8c30 con 0x7fb4781f83a0 2024-04-20T11:32:33.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.427+0000 7fb46fe00640 1 -- 172.21.15.152:0/21180262 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) ==== 74+0+13906 (secure 0 0 0) 0x7fb4700597f0 con 0x7fb4781f83a0 2024-04-20T11:32:33.428 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:32:33.429 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":75,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:32:31.965800+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.175:6833/2057501014":"2024-04-21T11:32:28.953296+0000","172.21.15.152:0/3593059376":"2024-04-21T11:32:22.542176+0000","172.21.15.152:0/2828322453":"2024-04-21T11:32:22.542176+0000","172.21.15.152:0/3331653721":"2024-04-21T11:32:22.542176+0000","172.21.15.152:0/2184739958":"2024-04-21T11:32:22.542176+0000","172.21.15.152:0/3263977099":"2024-04-21T11:32:22.542176+0000","172.21.15.152:0/2510881178":"2024-04-21T11:32:22.542176+0000","172.21.15.152:0/142101408":"2024-04-21T11:32:22.542176+0000","172.21.15.152:0/698943652":"2024-04-21T11:32:22.542176+0000","172.21.15.152:0/2405473735":"2024-04-21T11:32:22.542176+0000","172.21.15.152:6832/2285110240":"2024-04-21T11:32:22.542176+0000","172.21.15.152:6833/2285110240":"2024-04-21T11:32:22.542176+0000","172.21.15.152:0/1524004838":"2024-04-21T11:32:22.542176+0000","172.21.15.175:6835/2057501014":"2024-04-21T11:32:28.953296+0000","172.21.15.152:0/3082578161":"2024-04-21T11:32:22.542176+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:32:33.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.430+0000 7fb47e800640 1 -- 172.21.15.152:0/21180262 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb45406ca10 msgr2=0x7fb45406eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:33.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.430+0000 7fb47e800640 1 --2- 172.21.15.152:0/21180262 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb45406ca10 0x7fb45406eed0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb468037000 tx=0x7fb46800c040 comp rx=0 tx=0).stop 2024-04-20T11:32:33.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.430+0000 7fb47e800640 1 -- 172.21.15.152:0/21180262 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb4781f83a0 msgr2=0x7fb4781e8ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:33.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.430+0000 7fb47e800640 1 --2- 172.21.15.152:0/21180262 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb4781f83a0 0x7fb4781e8ec0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fb47000c7a0 tx=0x7fb47000cc60 comp rx=0 tx=0).stop 2024-04-20T11:32:33.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.430+0000 7fb47e800640 1 -- 172.21.15.152:0/21180262 shutdown_connections 2024-04-20T11:32:33.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.430+0000 7fb47e800640 1 --2- 172.21.15.152:0/21180262 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb45406ca10 0x7fb45406eed0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:33.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.430+0000 7fb47e800640 1 --2- 172.21.15.152:0/21180262 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb4781f83a0 0x7fb4781e8ec0 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:33.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.430+0000 7fb47e800640 1 --2- 172.21.15.152:0/21180262 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb47815b320 0x7fb478151610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:33.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.430+0000 7fb47e800640 1 --2- 172.21.15.152:0/21180262 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb4780cc9b0 0x7fb4781e4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:33.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.430+0000 7fb47e800640 1 -- 172.21.15.152:0/21180262 >> 172.21.15.152:0/21180262 conn(0x7fb478067cf0 msgr2=0x7fb4780b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:33.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.430+0000 7fb47e800640 1 -- 172.21.15.152:0/21180262 shutdown_connections 2024-04-20T11:32:33.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.430+0000 7fb47e800640 1 -- 172.21.15.152:0/21180262 wait complete. 2024-04-20T11:32:33.450 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T11:26:47.246571+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T11:32:33.451 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T11:32:33.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.656+0000 7ff343a00640 1 Processor -- start 2024-04-20T11:32:33.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.656+0000 7ff343a00640 1 -- start start 2024-04-20T11:32:33.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.658+0000 7ff343a00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ff33c162bc0 0x7ff33c167240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:33.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.658+0000 7ff343a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7ff33c0b2880 0x7ff33c06e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:33.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.658+0000 7ff343a00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff33c167780 con 0x7ff33c06f720 2024-04-20T11:32:33.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.658+0000 7ff343a00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7ff33c0ccb40 con 0x7ff33c0b2880 2024-04-20T11:32:33.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.658+0000 7ff343a00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7ff33c0b3610 con 0x7ff33c162bc0 2024-04-20T11:32:33.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.658+0000 7ff341a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7ff33c0b2880 0x7ff33c06e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:33.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.658+0000 7ff341a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7ff33c0b2880 0x7ff33c06e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:60262/0 (socket says 172.21.15.152:60262) 2024-04-20T11:32:33.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.659+0000 7ff341a00640 1 -- 172.21.15.152:0/3277963919 learned_addr learned my addr 172.21.15.152:0/3277963919 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:32:33.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.659+0000 7ff341000640 1 --2- 172.21.15.152:0/3277963919 >> v2:172.21.15.175:3300/0 conn(0x7ff33c162bc0 0x7ff33c167240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:33.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.659+0000 7ff341a00640 1 -- 172.21.15.152:0/3277963919 >> v2:172.21.15.175:3300/0 conn(0x7ff33c162bc0 msgr2=0x7ff33c167240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:33.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.659+0000 7ff341a00640 1 --2- 172.21.15.152:0/3277963919 >> v2:172.21.15.175:3300/0 conn(0x7ff33c162bc0 0x7ff33c167240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:33.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.659+0000 7ff341a00640 1 -- 172.21.15.152:0/3277963919 >> v1:172.21.15.175:6789/0 conn(0x7ff33c06f720 legacy=0x7ff33c0b2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:33.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.659+0000 7ff341a00640 1 -- 172.21.15.152:0/3277963919 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff33c06fc30 con 0x7ff33c0b2880 2024-04-20T11:32:33.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.659+0000 7ff341000640 1 --2- 172.21.15.152:0/3277963919 >> v2:172.21.15.175:3300/0 conn(0x7ff33c162bc0 0x7ff33c167240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:32:33.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.660+0000 7ff341a00640 1 --2- 172.21.15.152:0/3277963919 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff33c0b2880 0x7ff33c06e010 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7ff33c0700a0 tx=0x7ff33402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68b3327d76a4846b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:33.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.660+0000 7ff33b400640 1 -- 172.21.15.152:0/3277963919 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff33402ca80 con 0x7ff33c0b2880 2024-04-20T11:32:33.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.661+0000 7ff33b400640 1 -- 172.21.15.152:0/3277963919 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff33402cc20 con 0x7ff33c0b2880 2024-04-20T11:32:33.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.661+0000 7ff33b400640 1 -- 172.21.15.152:0/3277963919 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3340343f0 con 0x7ff33c0b2880 2024-04-20T11:32:33.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.661+0000 7ff343a00640 1 -- 172.21.15.152:0/3277963919 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff33c0b2880 msgr2=0x7ff33c06e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:33.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.661+0000 7ff343a00640 1 --2- 172.21.15.152:0/3277963919 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff33c0b2880 0x7ff33c06e010 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7ff33c0700a0 tx=0x7ff33402bee0 comp rx=0 tx=0).stop 2024-04-20T11:32:33.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.661+0000 7ff343a00640 1 -- 172.21.15.152:0/3277963919 shutdown_connections 2024-04-20T11:32:33.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.661+0000 7ff343a00640 1 --2- 172.21.15.152:0/3277963919 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff33c0b2880 0x7ff33c06e010 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:33.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.661+0000 7ff343a00640 1 --2- 172.21.15.152:0/3277963919 >> v2:172.21.15.175:3300/0 conn(0x7ff33c162bc0 0x7ff33c167240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:33.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.661+0000 7ff343a00640 1 -- 172.21.15.152:0/3277963919 >> 172.21.15.152:0/3277963919 conn(0x7ff33c06b980 msgr2=0x7ff33c06bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:33.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.661+0000 7ff343a00640 1 -- 172.21.15.152:0/3277963919 shutdown_connections 2024-04-20T11:32:33.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.661+0000 7ff343a00640 1 -- 172.21.15.152:0/3277963919 wait complete. 2024-04-20T11:32:33.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.663+0000 7ff343a00640 1 Processor -- start 2024-04-20T11:32:33.664 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.663+0000 7ff343a00640 1 -- start start 2024-04-20T11:32:33.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.664+0000 7ff343a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff33c06f720 0x7ff33c14bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:33.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.664+0000 7ff343a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff33c0b2880 0x7ff33c0a94a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:33.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.664+0000 7ff343a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff33c162bc0 0x7ff33c0a99e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:33.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.664+0000 7ff343a00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7ff33c0b4650 con 0x7ff33c162bc0 2024-04-20T11:32:33.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.664+0000 7ff343a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff33c0b4370 con 0x7ff33c0b2880 2024-04-20T11:32:33.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.664+0000 7ff343a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff33c162fa0 con 0x7ff33c06f720 2024-04-20T11:32:33.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.664+0000 7ff341a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff33c162bc0 0x7ff33c0a99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:33.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.664+0000 7ff341a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff33c162bc0 0x7ff33c0a99e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:60268/0 (socket says 172.21.15.152:60268) 2024-04-20T11:32:33.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.665+0000 7ff341a00640 1 -- 172.21.15.152:0/3395631832 learned_addr learned my addr 172.21.15.152:0/3395631832 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:32:33.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.665+0000 7ff341000640 1 --2- 172.21.15.152:0/3395631832 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff33c06f720 0x7ff33c14bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:33.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.665+0000 7ff341a00640 1 -- 172.21.15.152:0/3395631832 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff33c06f720 msgr2=0x7ff33c14bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:33.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.665+0000 7ff341a00640 1 --2- 172.21.15.152:0/3395631832 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff33c06f720 0x7ff33c14bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:33.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.665+0000 7ff341a00640 1 -- 172.21.15.152:0/3395631832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff33c0b2880 msgr2=0x7ff33c0a94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:33.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.665+0000 7ff33be00640 1 --2- 172.21.15.152:0/3395631832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff33c0b2880 0x7ff33c0a94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:33.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.665+0000 7ff341a00640 1 --2- 172.21.15.152:0/3395631832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff33c0b2880 0x7ff33c0a94a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:33.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.665+0000 7ff341a00640 1 -- 172.21.15.152:0/3395631832 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff33c14f230 con 0x7ff33c162bc0 2024-04-20T11:32:33.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.665+0000 7ff341a00640 1 --2- 172.21.15.152:0/3395631832 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff33c162bc0 0x7ff33c0a99e0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7ff33402c4f0 tx=0x7ff334004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:33.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.666+0000 7ff338c00640 1 -- 172.21.15.152:0/3395631832 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff334043020 con 0x7ff33c162bc0 2024-04-20T11:32:33.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.666+0000 7ff338c00640 1 -- 172.21.15.152:0/3395631832 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff33403e450 con 0x7ff33c162bc0 2024-04-20T11:32:33.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.666+0000 7ff343a00640 1 -- 172.21.15.152:0/3395631832 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff33c14c380 con 0x7ff33c162bc0 2024-04-20T11:32:33.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.666+0000 7ff338c00640 1 -- 172.21.15.152:0/3395631832 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff33403e820 con 0x7ff33c162bc0 2024-04-20T11:32:33.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.666+0000 7ff343a00640 1 -- 172.21.15.152:0/3395631832 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff33c14c940 con 0x7ff33c162bc0 2024-04-20T11:32:33.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.667+0000 7ff343a00640 1 -- 172.21.15.152:0/3395631832 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff33c0689f0 con 0x7ff33c162bc0 2024-04-20T11:32:33.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.668+0000 7ff338c00640 1 -- 172.21.15.152:0/3395631832 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 91142+0+0 (secure 0 0 0) 0x7ff334046020 con 0x7ff33c162bc0 2024-04-20T11:32:33.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.669+0000 7ff338c00640 1 --2- 172.21.15.152:0/3395631832 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7ff31806ca10 0x7ff31806eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:33.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.669+0000 7ff338c00640 1 -- 172.21.15.152:0/3395631832 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5502+0+0 (secure 0 0 0) 0x7ff3340ae970 con 0x7ff33c162bc0 2024-04-20T11:32:33.670 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.669+0000 7ff341000640 1 --2- 172.21.15.152:0/3395631832 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7ff31806ca10 0x7ff31806eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:33.671 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.671+0000 7ff341000640 1 --2- 172.21.15.152:0/3395631832 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7ff31806ca10 0x7ff31806eed0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff32c00b840 tx=0x7ff32c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:33.674 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.673+0000 7ff338c00640 1 -- 172.21.15.152:0/3395631832 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ff33404d020 con 0x7ff33c162bc0 2024-04-20T11:32:33.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.870+0000 7ff343a00640 1 -- 172.21.15.152:0/3395631832 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff33c0b0a40 con 0x7ff33c162bc0 2024-04-20T11:32:33.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.871+0000 7ff338c00640 1 -- 172.21.15.152:0/3395631832 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v75) ==== 93+0+10 (secure 0 0 0) 0x7ff33c167780 con 0x7ff33c162bc0 2024-04-20T11:32:33.871 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 1 2024-04-20T11:32:33.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.873+0000 7ff343a00640 1 -- 172.21.15.152:0/3395631832 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7ff31806ca10 msgr2=0x7ff31806eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:33.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.873+0000 7ff343a00640 1 --2- 172.21.15.152:0/3395631832 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7ff31806ca10 0x7ff31806eed0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff32c00b840 tx=0x7ff32c008040 comp rx=0 tx=0).stop 2024-04-20T11:32:33.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.873+0000 7ff343a00640 1 -- 172.21.15.152:0/3395631832 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff33c162bc0 msgr2=0x7ff33c0a99e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:33.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.873+0000 7ff343a00640 1 --2- 172.21.15.152:0/3395631832 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff33c162bc0 0x7ff33c0a99e0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7ff33402c4f0 tx=0x7ff334004a40 comp rx=0 tx=0).stop 2024-04-20T11:32:33.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.874+0000 7ff343a00640 1 -- 172.21.15.152:0/3395631832 shutdown_connections 2024-04-20T11:32:33.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.874+0000 7ff343a00640 1 --2- 172.21.15.152:0/3395631832 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7ff31806ca10 0x7ff31806eed0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:33.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.874+0000 7ff343a00640 1 --2- 172.21.15.152:0/3395631832 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff33c162bc0 0x7ff33c0a99e0 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:33.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.874+0000 7ff343a00640 1 --2- 172.21.15.152:0/3395631832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff33c0b2880 0x7ff33c0a94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:33.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.874+0000 7ff343a00640 1 --2- 172.21.15.152:0/3395631832 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff33c06f720 0x7ff33c14bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:33.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.874+0000 7ff343a00640 1 -- 172.21.15.152:0/3395631832 >> 172.21.15.152:0/3395631832 conn(0x7ff33c06b980 msgr2=0x7ff33c0b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:33.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.874+0000 7ff343a00640 1 -- 172.21.15.152:0/3395631832 shutdown_connections 2024-04-20T11:32:33.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:33.874+0000 7ff343a00640 1 -- 172.21.15.152:0/3395631832 wait complete. 2024-04-20T11:32:33.895 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_names_can_only_be_goodchars' 2024-04-20T11:32:34.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:34.098+0000 7f795cc00640 1 Processor -- start 2024-04-20T11:32:34.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:34.099+0000 7f795cc00640 1 -- start start 2024-04-20T11:32:34.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:34.100+0000 7f795cc00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f79581603d0 0x7f79580b5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:34.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:34.100+0000 7f795cc00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f79580b5480 0x7f79580b63f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:34.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:34.100+0000 7f795cc00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79580d3350 con 0x7f795815fcc0 2024-04-20T11:32:34.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:34.100+0000 7f795cc00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f79580d3630 con 0x7f79580b5480 2024-04-20T11:32:34.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:34.100+0000 7f795cc00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f79580d3910 con 0x7f79581603d0 2024-04-20T11:32:34.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:34.100+0000 7f7957e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f79580b5480 0x7f79580b63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:34.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:34.101+0000 7f7957e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f79580b5480 0x7f79580b63f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:60280/0 (socket says 172.21.15.152:60280) 2024-04-20T11:32:34.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:34.101+0000 7f7957e00640 1 -- 172.21.15.152:0/1149336128 learned_addr learned my addr 172.21.15.152:0/1149336128 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:32:34.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:34.101+0000 7f7956a00640 1 --2- 172.21.15.152:0/1149336128 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f79581603d0 0x7f79580b5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:34.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:34.101+0000 7f7957e00640 1 -- 172.21.15.152:0/1149336128 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f79581603d0 msgr2=0x7f79580b5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:34.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:34.101+0000 7f7957e00640 1 --2- 172.21.15.152:0/1149336128 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f79581603d0 0x7f79580b5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:34.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:34.101+0000 7f7957e00640 1 -- 172.21.15.152:0/1149336128 >> v1:172.21.15.175:6789/0 conn(0x7f795815fcc0 legacy=0x7f79580d4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:34.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:34.101+0000 7f7957e00640 1 -- 172.21.15.152:0/1149336128 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79580ced20 con 0x7f79580b5480 2024-04-20T11:32:34.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:34.103+0000 7f7957e00640 1 --2- 172.21.15.152:0/1149336128 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f79580b5480 0x7f79580b63f0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f7950009870 tx=0x7f795002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=170f726e1205b71a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:34.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:34.103+0000 7f7956000640 1 -- 172.21.15.152:0/1149336128 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7950034070 con 0x7f79580b5480 2024-04-20T11:32:34.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:34.103+0000 7f7956000640 1 -- 172.21.15.152:0/1149336128 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7950030840 con 0x7f79580b5480 2024-04-20T11:32:34.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:34.103+0000 7f7956000640 1 -- 172.21.15.152:0/1149336128 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7950030c10 con 0x7f79580b5480 2024-04-20T11:32:34.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:34.103+0000 7f795cc00640 1 -- 172.21.15.152:0/1149336128 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f79580b5480 msgr2=0x7f79580b63f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:34.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:34.103+0000 7f795cc00640 1 --2- 172.21.15.152:0/1149336128 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f79580b5480 0x7f79580b63f0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f7950009870 tx=0x7f795002b980 comp rx=0 tx=0).stop 2024-04-20T11:32:34.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:34.104+0000 7f795cc00640 1 -- 172.21.15.152:0/1149336128 shutdown_connections 2024-04-20T11:32:34.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:34.104+0000 7f795cc00640 1 --2- 172.21.15.152:0/1149336128 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f79580b5480 0x7f79580b63f0 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:34.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:34.104+0000 7f795cc00640 1 --2- 172.21.15.152:0/1149336128 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f79581603d0 0x7f79580b5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:34.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:34.104+0000 7f795cc00640 1 -- 172.21.15.152:0/1149336128 >> 172.21.15.152:0/1149336128 conn(0x7f795808bfd0 msgr2=0x7f79580d4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:34.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:34.104+0000 7f795cc00640 1 -- 172.21.15.152:0/1149336128 shutdown_connections 2024-04-20T11:32:34.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:34.104+0000 7f795cc00640 1 -- 172.21.15.152:0/1149336128 wait complete. 2024-04-20T11:32:34.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:34.105+0000 7f795cc00640 1 Processor -- start 2024-04-20T11:32:34.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:34.106+0000 7f795cc00640 1 -- start start 2024-04-20T11:32:34.108 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:34.106+0000 7f795cc00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f79580b5480 0x7f7958149ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:34.108 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:34.106+0000 7f795cc00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f795815fcc0 0x7f795806dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:34.108 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:34.106+0000 7f795cc00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f79581603d0 0x7f795806e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:34.108 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:34.106+0000 7f795cc00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f79580cf5d0 con 0x7f79580b5480 2024-04-20T11:32:34.108 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:34.106+0000 7f795cc00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f79580cf450 con 0x7f795815fcc0 2024-04-20T11:32:34.108 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:34.106+0000 7f795cc00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f79580cf750 con 0x7f79581603d0 2024-04-20T11:32:34.108 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:34.107+0000 7f7957400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f79580b5480 0x7f7958149ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:34.108 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:34.107+0000 7f7957400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f79580b5480 0x7f7958149ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:60292/0 (socket says 172.21.15.152:60292) 2024-04-20T11:32:34.108 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:34.107+0000 7f7957400640 1 -- 172.21.15.152:0/3188933253 learned_addr learned my addr 172.21.15.152:0/3188933253 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:32:34.108 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:34.107+0000 7f7957e00640 1 --2- 172.21.15.152:0/3188933253 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f79581603d0 0x7f795806e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:34.108 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:34.107+0000 7f7957400640 1 -- 172.21.15.152:0/3188933253 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f79581603d0 msgr2=0x7f795806e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:34.108 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:34.107+0000 7f7957400640 1 --2- 172.21.15.152:0/3188933253 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f79581603d0 0x7f795806e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:34.109 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:34.107+0000 7f7956a00640 1 --2- 172.21.15.152:0/3188933253 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f795815fcc0 0x7f795806dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:34.109 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:34.107+0000 7f7957400640 1 -- 172.21.15.152:0/3188933253 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f795815fcc0 msgr2=0x7f795806dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:34.109 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:34.107+0000 7f7957400640 1 --2- 172.21.15.152:0/3188933253 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f795815fcc0 0x7f795806dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:34.109 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:34.107+0000 7f7957400640 1 -- 172.21.15.152:0/3188933253 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7958149ff0 con 0x7f79580b5480 2024-04-20T11:32:34.110 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:34.108+0000 7f7957400640 1 --2- 172.21.15.152:0/3188933253 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f79580b5480 0x7f7958149ab0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f794800c790 tx=0x7f794800cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:34.110 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:34.108+0000 7f794f400640 1 -- 172.21.15.152:0/3188933253 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7948016020 con 0x7f79580b5480 2024-04-20T11:32:34.110 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:34.108+0000 7f794f400640 1 -- 172.21.15.152:0/3188933253 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7948006d50 con 0x7f79580b5480 2024-04-20T11:32:34.110 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:34.108+0000 7f794f400640 1 -- 172.21.15.152:0/3188933253 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7948003bb0 con 0x7f79580b5480 2024-04-20T11:32:34.110 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:34.108+0000 7f795cc00640 1 -- 172.21.15.152:0/3188933253 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7958147260 con 0x7f79580b5480 2024-04-20T11:32:34.110 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:34.108+0000 7f795cc00640 1 -- 172.21.15.152:0/3188933253 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7958147820 con 0x7f79580b5480 2024-04-20T11:32:34.110 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:34.109+0000 7f795cc00640 1 -- 172.21.15.152:0/3188933253 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7958075e60 con 0x7f79580b5480 2024-04-20T11:32:34.111 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:34.110+0000 7f794f400640 1 -- 172.21.15.152:0/3188933253 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 91142+0+0 (secure 0 0 0) 0x7f794800a850 con 0x7f79580b5480 2024-04-20T11:32:34.112 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:34.111+0000 7f794f400640 1 --2- 172.21.15.152:0/3188933253 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f793406ca10 0x7f793406eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:34.112 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:34.111+0000 7f794f400640 1 -- 172.21.15.152:0/3188933253 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5502+0+0 (secure 0 0 0) 0x7f794808b990 con 0x7f79580b5480 2024-04-20T11:32:34.112 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:34.112+0000 7f7956a00640 1 --2- 172.21.15.152:0/3188933253 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f793406ca10 0x7f793406eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:34.113 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:34.113+0000 7f7956a00640 1 --2- 172.21.15.152:0/3188933253 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f793406ca10 0x7f793406eed0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f79581eb290 tx=0x7f7940008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:34.116 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:34.116+0000 7f794f400640 1 -- 172.21.15.152:0/3188933253 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f7948020020 con 0x7f79580b5480 2024-04-20T11:32:34.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:34.387+0000 7f795cc00640 1 -- 172.21.15.152:0/3188933253 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_names_can_only_be_goodchars"]} v 0) -- 0x7f795815cfe0 con 0x7f79580b5480 2024-04-20T11:32:34.986 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:34.985+0000 7f794f400640 1 -- 172.21.15.152:0/3188933253 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_names_can_only_be_goodchars"]}]=0 v302) ==== 149+0+0 (secure 0 0 0) 0x7f7948019310 con 0x7f79580b5480 2024-04-20T11:32:34.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:34.988+0000 7f795cc00640 1 -- 172.21.15.152:0/3188933253 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f793406ca10 msgr2=0x7f793406eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:34.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:34.988+0000 7f795cc00640 1 --2- 172.21.15.152:0/3188933253 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f793406ca10 0x7f793406eed0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f79581eb290 tx=0x7f7940008040 comp rx=0 tx=0).stop 2024-04-20T11:32:34.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:34.989+0000 7f795cc00640 1 -- 172.21.15.152:0/3188933253 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f79580b5480 msgr2=0x7f7958149ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:34.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:34.989+0000 7f795cc00640 1 --2- 172.21.15.152:0/3188933253 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f79580b5480 0x7f7958149ab0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f794800c790 tx=0x7f794800cc50 comp rx=0 tx=0).stop 2024-04-20T11:32:34.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:34.989+0000 7f795cc00640 1 -- 172.21.15.152:0/3188933253 shutdown_connections 2024-04-20T11:32:34.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:34.989+0000 7f795cc00640 1 --2- 172.21.15.152:0/3188933253 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f793406ca10 0x7f793406eed0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:34.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:34.989+0000 7f795cc00640 1 --2- 172.21.15.152:0/3188933253 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f79581603d0 0x7f795806e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:34.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:34.989+0000 7f795cc00640 1 --2- 172.21.15.152:0/3188933253 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f795815fcc0 0x7f795806dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:34.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:34.989+0000 7f795cc00640 1 --2- 172.21.15.152:0/3188933253 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f79580b5480 0x7f7958149ab0 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:34.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:34.989+0000 7f795cc00640 1 -- 172.21.15.152:0/3188933253 >> 172.21.15.152:0/3188933253 conn(0x7f795808bfd0 msgr2=0x7f79580d4530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:34.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:34.989+0000 7f795cc00640 1 -- 172.21.15.152:0/3188933253 shutdown_connections 2024-04-20T11:32:34.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:34.989+0000 7f795cc00640 1 -- 172.21.15.152:0/3188933253 wait complete. 2024-04-20T11:32:35.013 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-20T11:32:35.221 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:35.220+0000 7f7089000640 1 Processor -- start 2024-04-20T11:32:35.221 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:35.220+0000 7f7089000640 1 -- start start 2024-04-20T11:32:35.223 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:35.222+0000 7f7089000640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f70840cd8c0 0x7f70840b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:35.223 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:35.222+0000 7f7089000640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70841463e0 con 0x7f70840b2880 2024-04-20T11:32:35.223 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:35.222+0000 7f7089000640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70840b3e20 con 0x7f7084146000 2024-04-20T11:32:35.223 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:35.222+0000 7f7089000640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f70840b4750 con 0x7f70840cd8c0 2024-04-20T11:32:35.223 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:35.222+0000 7f7081400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f70840b2880 0x7f70840cd1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:33722/0 (socket says 172.21.15.152:33722) 2024-04-20T11:32:35.223 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:35.222+0000 7f7081400640 1 -- 172.21.15.152:0/1477335422 learned_addr learned my addr 172.21.15.152:0/1477335422 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:32:35.225 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:35.223+0000 7f7082800640 1 --2- 172.21.15.152:0/1477335422 >> v2:172.21.15.175:3300/0 conn(0x7f70840cd8c0 0x7f70840b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:35.225 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:35.223+0000 7f7080a00640 1 -- 172.21.15.152:0/1477335422 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 417516072 0 0) 0x7f70841463e0 con 0x7f70840b2880 2024-04-20T11:32:35.225 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:35.223+0000 7f7080a00640 1 -- 172.21.15.152:0/1477335422 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7068003520 con 0x7f70840b2880 2024-04-20T11:32:35.225 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:35.223+0000 7f7080a00640 1 -- 172.21.15.152:0/1477335422 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2629937072 0 0) 0x7f7068003520 con 0x7f70840b2880 2024-04-20T11:32:35.225 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:35.223+0000 7f7080a00640 1 -- 172.21.15.152:0/1477335422 >> v2:172.21.15.175:3300/0 conn(0x7f70840cd8c0 msgr2=0x7f70840b4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:35.225 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:35.223+0000 7f7080a00640 1 --2- 172.21.15.152:0/1477335422 >> v2:172.21.15.175:3300/0 conn(0x7f70840cd8c0 0x7f70840b4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:35.225 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:35.223+0000 7f7080a00640 1 -- 172.21.15.152:0/1477335422 >> v1:172.21.15.175:6789/0 conn(0x7f7084146000 legacy=0x7f70840b2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:35.225 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:35.223+0000 7f7080a00640 1 -- 172.21.15.152:0/1477335422 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70840b4100 con 0x7f70840b2880 2024-04-20T11:32:35.225 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:35.223+0000 7f7080a00640 1 -- 172.21.15.152:0/1477335422 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f706c001670 con 0x7f70840b2880 2024-04-20T11:32:35.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:35.223+0000 7f7080a00640 1 -- 172.21.15.152:0/1477335422 >> v1:172.21.15.152:6789/0 conn(0x7f70840b2880 legacy=0x7f70840cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:35.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:35.223+0000 7f7080a00640 1 --2- 172.21.15.152:0/1477335422 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f70680039b0 0x7f7068003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:35.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:35.224+0000 7f7080a00640 1 --2- 172.21.15.152:0/1477335422 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7068004800 0x7f7068004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:35.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:35.224+0000 7f7080a00640 1 --2- 172.21.15.152:0/1477335422 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f70680051c0 0x7f706800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:35.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:35.224+0000 7f7080a00640 1 -- 172.21.15.152:0/1477335422 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f70840b4750 con 0x7f70680039b0 2024-04-20T11:32:35.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:35.224+0000 7f7080a00640 1 -- 172.21.15.152:0/1477335422 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f706800c160 con 0x7f7068004800 2024-04-20T11:32:35.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:35.224+0000 7f7080a00640 1 -- 172.21.15.152:0/1477335422 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f706800c360 con 0x7f70680051c0 2024-04-20T11:32:35.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:35.224+0000 7f7081e00640 1 --2- 172.21.15.152:0/1477335422 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f70680039b0 0x7f7068003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:35.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:35.224+0000 7f7081e00640 1 -- 172.21.15.152:0/1477335422 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f70680051c0 msgr2=0x7f706800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:35.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:35.224+0000 7f7081e00640 1 --2- 172.21.15.152:0/1477335422 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f70680051c0 0x7f706800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:35.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:35.224+0000 7f7081e00640 1 -- 172.21.15.152:0/1477335422 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7068004800 msgr2=0x7f7068004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:35.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:35.224+0000 7f7081e00640 1 --2- 172.21.15.152:0/1477335422 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7068004800 0x7f7068004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:35.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:35.224+0000 7f7081e00640 1 -- 172.21.15.152:0/1477335422 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7068003520 con 0x7f70680039b0 2024-04-20T11:32:35.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:35.224+0000 7f7082800640 1 -- 172.21.15.152:0/1477335422 reap_dead start 2024-04-20T11:32:35.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:35.225+0000 7f7081e00640 1 --2- 172.21.15.152:0/1477335422 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f70680039b0 0x7f7068003dd0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f7074001750 tx=0x7f7068005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec8b6120b30179aa server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:35.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:35.226+0000 7f7080a00640 1 -- 172.21.15.152:0/1477335422 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7074001370 con 0x7f70680039b0 2024-04-20T11:32:35.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:35.226+0000 7f7080a00640 1 -- 172.21.15.152:0/1477335422 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70740052d0 con 0x7f70680039b0 2024-04-20T11:32:35.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:35.226+0000 7f7080a00640 1 -- 172.21.15.152:0/1477335422 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7074003830 con 0x7f70680039b0 2024-04-20T11:32:35.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:35.226+0000 7f7089000640 1 -- 172.21.15.152:0/1477335422 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f70680039b0 msgr2=0x7f7068003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:35.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:35.226+0000 7f7089000640 1 --2- 172.21.15.152:0/1477335422 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f70680039b0 0x7f7068003dd0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f7074001750 tx=0x7f7068005830 comp rx=0 tx=0).stop 2024-04-20T11:32:35.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:35.226+0000 7f7089000640 1 -- 172.21.15.152:0/1477335422 shutdown_connections 2024-04-20T11:32:35.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:35.226+0000 7f7089000640 1 --2- 172.21.15.152:0/1477335422 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f70680039b0 0x7f7068003dd0 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:35.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:35.226+0000 7f7089000640 1 -- 172.21.15.152:0/1477335422 >> 172.21.15.152:0/1477335422 conn(0x7f708406b980 msgr2=0x7f708406bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:35.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:35.226+0000 7f7089000640 1 -- 172.21.15.152:0/1477335422 shutdown_connections 2024-04-20T11:32:35.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:35.226+0000 7f7089000640 1 -- 172.21.15.152:0/1477335422 wait complete. 2024-04-20T11:32:35.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:35.228+0000 7f7089000640 1 Processor -- start 2024-04-20T11:32:35.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:35.228+0000 7f7089000640 1 -- start start 2024-04-20T11:32:35.230 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:35.229+0000 7f7089000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f70680039b0 0x7f70841f5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:35.230 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:35.229+0000 7f7089000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f70840cb350 0x7f708412f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:35.230 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:35.229+0000 7f7089000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f70840cbd70 0x7f70840cc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:35.230 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:35.229+0000 7f7089000640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f70840b4d60 con 0x7f70680039b0 2024-04-20T11:32:35.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:35.229+0000 7f7089000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f70840b4be0 con 0x7f70840cb350 2024-04-20T11:32:35.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:35.229+0000 7f7089000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f70840b4ee0 con 0x7f70840cbd70 2024-04-20T11:32:35.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:35.229+0000 7f7081e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f70680039b0 0x7f70841f5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:35.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:35.229+0000 7f7081e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f70680039b0 0x7f70841f5370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:60310/0 (socket says 172.21.15.152:60310) 2024-04-20T11:32:35.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:35.229+0000 7f7081e00640 1 -- 172.21.15.152:0/2617324807 learned_addr learned my addr 172.21.15.152:0/2617324807 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:32:35.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:35.230+0000 7f7081e00640 1 -- 172.21.15.152:0/2617324807 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f70840cbd70 msgr2=0x7f70840cc1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:35.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:35.230+0000 7f7081e00640 1 --2- 172.21.15.152:0/2617324807 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f70840cbd70 0x7f70840cc1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:35.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:35.230+0000 7f7081e00640 1 -- 172.21.15.152:0/2617324807 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f70840cb350 msgr2=0x7f708412f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:35.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:35.230+0000 7f7081e00640 1 --2- 172.21.15.152:0/2617324807 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f70840cb350 0x7f708412f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:35.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:35.230+0000 7f7081e00640 1 -- 172.21.15.152:0/2617324807 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70841f58b0 con 0x7f70680039b0 2024-04-20T11:32:35.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:35.230+0000 7f7081e00640 1 --2- 172.21.15.152:0/2617324807 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f70680039b0 0x7f70841f5370 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f7074000c00 tx=0x7f7074001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:35.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:35.230+0000 7f707a000640 1 -- 172.21.15.152:0/2617324807 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7074041020 con 0x7f70680039b0 2024-04-20T11:32:35.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:35.230+0000 7f707a000640 1 -- 172.21.15.152:0/2617324807 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7074005480 con 0x7f70680039b0 2024-04-20T11:32:35.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:35.230+0000 7f7089000640 1 -- 172.21.15.152:0/2617324807 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f708412fe00 con 0x7f70680039b0 2024-04-20T11:32:35.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:35.230+0000 7f707a000640 1 -- 172.21.15.152:0/2617324807 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7074001370 con 0x7f70680039b0 2024-04-20T11:32:35.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:35.231+0000 7f7089000640 1 -- 172.21.15.152:0/2617324807 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70840c9be0 con 0x7f70680039b0 2024-04-20T11:32:35.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:35.232+0000 7f7089000640 1 -- 172.21.15.152:0/2617324807 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7084069100 con 0x7f70680039b0 2024-04-20T11:32:35.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:35.232+0000 7f707a000640 1 -- 172.21.15.152:0/2617324807 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 91142+0+0 (secure 0 0 0) 0x7f7074044020 con 0x7f70680039b0 2024-04-20T11:32:35.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:35.233+0000 7f707a000640 1 --2- 172.21.15.152:0/2617324807 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f706006ca10 0x7f706006eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:35.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:35.233+0000 7f707a000640 1 -- 172.21.15.152:0/2617324807 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5502+0+0 (secure 0 0 0) 0x7f70740ad810 con 0x7f70680039b0 2024-04-20T11:32:35.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:35.234+0000 7f7081400640 1 --2- 172.21.15.152:0/2617324807 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f706006ca10 0x7f706006eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:35.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:35.235+0000 7f7081400640 1 --2- 172.21.15.152:0/2617324807 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f706006ca10 0x7f706006eed0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f706c000fd0 tx=0x7f706c0014c0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:35.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:35.238+0000 7f707a000640 1 -- 172.21.15.152:0/2617324807 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f7084069100 con 0x7f70680039b0 2024-04-20T11:32:35.438 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:35.437+0000 7f7089000640 1 -- 172.21.15.152:0/2617324807 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f70840689f0 con 0x7f70680039b0 2024-04-20T11:32:35.444 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:35.444+0000 7f707a000640 1 -- 172.21.15.152:0/2617324807 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) ==== 122+0+0 (secure 0 0 0) 0x7f707404b030 con 0x7f70680039b0 2024-04-20T11:32:35.448 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:35.447+0000 7f7089000640 1 -- 172.21.15.152:0/2617324807 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f706006ca10 msgr2=0x7f706006eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:35.448 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:35.447+0000 7f7089000640 1 --2- 172.21.15.152:0/2617324807 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f706006ca10 0x7f706006eed0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f706c000fd0 tx=0x7f706c0014c0 comp rx=0 tx=0).stop 2024-04-20T11:32:35.448 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:35.447+0000 7f7089000640 1 -- 172.21.15.152:0/2617324807 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f70680039b0 msgr2=0x7f70841f5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:35.448 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:35.447+0000 7f7089000640 1 --2- 172.21.15.152:0/2617324807 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f70680039b0 0x7f70841f5370 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f7074000c00 tx=0x7f7074001120 comp rx=0 tx=0).stop 2024-04-20T11:32:35.448 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:35.447+0000 7f7089000640 1 -- 172.21.15.152:0/2617324807 shutdown_connections 2024-04-20T11:32:35.448 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:35.447+0000 7f7089000640 1 --2- 172.21.15.152:0/2617324807 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f706006ca10 0x7f706006eed0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:35.448 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:35.447+0000 7f7089000640 1 --2- 172.21.15.152:0/2617324807 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f70840cbd70 0x7f70840cc1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:35.448 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:35.447+0000 7f7089000640 1 --2- 172.21.15.152:0/2617324807 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f70840cb350 0x7f708412f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:35.448 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:35.447+0000 7f7089000640 1 --2- 172.21.15.152:0/2617324807 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f70680039b0 0x7f70841f5370 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:35.448 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:35.447+0000 7f7089000640 1 -- 172.21.15.152:0/2617324807 >> 172.21.15.152:0/2617324807 conn(0x7f708406b980 msgr2=0x7f708415d240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:35.448 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:35.447+0000 7f7089000640 1 -- 172.21.15.152:0/2617324807 shutdown_connections 2024-04-20T11:32:35.448 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:35.447+0000 7f7089000640 1 -- 172.21.15.152:0/2617324807 wait complete. 2024-04-20T11:32:35.465 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-20T11:32:35.465 DEBUG:teuthology.orchestra.run.smithi152:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-20T11:32:35.559 DEBUG:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-20T11:32:35.675 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-20T11:32:35.675 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-20T11:32:35.675 DEBUG:teuthology.orchestra.run.smithi152:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:32:35.732 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-20T11:32:35.732 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T11:32:35.732 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:32:35.789 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-20T11:32:35.789 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-20T11:32:35.789 DEBUG:teuthology.orchestra.run.smithi152:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:32:35.847 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-20T11:32:35.847 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T11:32:35.847 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:32:35.908 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-20T11:32:35.908 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-20T11:32:35.909 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:32:36.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.120+0000 7f2171e00640 1 Processor -- start 2024-04-20T11:32:36.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.120+0000 7f2171e00640 1 -- start start 2024-04-20T11:32:36.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.122+0000 7f2171e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f216c15d1e0 0x7f216c160b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:36.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.122+0000 7f2171e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f216c15d5c0 0x7f216c0cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:36.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.122+0000 7f2171e00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f216c0ce8c0 con 0x7f216c071cd0 2024-04-20T11:32:36.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.122+0000 7f2171e00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f216c1600b0 con 0x7f216c15d1e0 2024-04-20T11:32:36.122 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.122+0000 7f2171e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f216c160390 con 0x7f216c15d5c0 2024-04-20T11:32:36.122 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.122+0000 7f2171400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f216c071cd0 0x7f216c0ccd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:48340/0 (socket says 172.21.15.152:48340) 2024-04-20T11:32:36.122 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.122+0000 7f2171400640 1 -- 172.21.15.152:0/1566424518 learned_addr learned my addr 172.21.15.152:0/1566424518 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:32:36.123 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.122+0000 7f216b400640 1 -- 172.21.15.152:0/1566424518 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4121984593 0 0) 0x7f216c0ce8c0 con 0x7f216c071cd0 2024-04-20T11:32:36.124 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.122+0000 7f2170a00640 1 --2- 172.21.15.152:0/1566424518 >> v2:172.21.15.175:3300/0 conn(0x7f216c15d1e0 0x7f216c160b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:36.124 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.122+0000 7f216b400640 1 -- 172.21.15.152:0/1566424518 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2154003520 con 0x7f216c071cd0 2024-04-20T11:32:36.124 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.122+0000 7f216be00640 1 --2- 172.21.15.152:0/1566424518 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f216c15d5c0 0x7f216c0cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:36.124 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.122+0000 7f216b400640 1 -- 172.21.15.152:0/1566424518 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4195226054 0 0) 0x7f2154003520 con 0x7f216c071cd0 2024-04-20T11:32:36.124 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.122+0000 7f216b400640 1 -- 172.21.15.152:0/1566424518 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f216c15d5c0 msgr2=0x7f216c0cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:36.124 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.122+0000 7f216b400640 1 --2- 172.21.15.152:0/1566424518 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f216c15d5c0 0x7f216c0cdfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:36.124 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.122+0000 7f216b400640 1 -- 172.21.15.152:0/1566424518 >> v2:172.21.15.175:3300/0 conn(0x7f216c15d1e0 msgr2=0x7f216c160b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:36.124 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.122+0000 7f216b400640 1 --2- 172.21.15.152:0/1566424518 >> v2:172.21.15.175:3300/0 conn(0x7f216c15d1e0 0x7f216c160b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:36.124 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.122+0000 7f216b400640 1 -- 172.21.15.152:0/1566424518 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f216c160670 con 0x7f216c071cd0 2024-04-20T11:32:36.124 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.122+0000 7f216b400640 1 -- 172.21.15.152:0/1566424518 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f21640019f0 con 0x7f216c071cd0 2024-04-20T11:32:36.124 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.122+0000 7f216be00640 1 --2- 172.21.15.152:0/1566424518 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f216c15d5c0 0x7f216c0cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:32:36.124 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.122+0000 7f216b400640 1 -- 172.21.15.152:0/1566424518 >> v1:172.21.15.152:6789/0 conn(0x7f216c071cd0 legacy=0x7f216c0ccd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:36.124 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.122+0000 7f216b400640 1 --2- 172.21.15.152:0/1566424518 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f21540039b0 0x7f2154003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:36.124 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.122+0000 7f216b400640 1 --2- 172.21.15.152:0/1566424518 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f21540043a0 0x7f2154004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:36.125 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.122+0000 7f216b400640 1 --2- 172.21.15.152:0/1566424518 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2154004fa0 0x7f215400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:36.125 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.122+0000 7f216b400640 1 -- 172.21.15.152:0/1566424518 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f216c1600b0 con 0x7f21540043a0 2024-04-20T11:32:36.125 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.122+0000 7f216b400640 1 -- 172.21.15.152:0/1566424518 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f216c160390 con 0x7f2154004fa0 2024-04-20T11:32:36.125 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.122+0000 7f216b400640 1 -- 172.21.15.152:0/1566424518 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f215400bf30 con 0x7f21540039b0 2024-04-20T11:32:36.125 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.122+0000 7f216be00640 1 --2- 172.21.15.152:0/1566424518 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f21540043a0 0x7f2154004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:36.125 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.123+0000 7f2170a00640 1 --2- 172.21.15.152:0/1566424518 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f21540039b0 0x7f2154003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:36.125 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.123+0000 7f216be00640 1 -- 172.21.15.152:0/1566424518 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f21540039b0 msgr2=0x7f2154003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:36.125 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.123+0000 7f216be00640 1 --2- 172.21.15.152:0/1566424518 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f21540039b0 0x7f2154003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:36.125 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.123+0000 7f216be00640 1 -- 172.21.15.152:0/1566424518 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2154004fa0 msgr2=0x7f215400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:36.125 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.123+0000 7f216be00640 1 --2- 172.21.15.152:0/1566424518 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2154004fa0 0x7f215400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:36.125 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.123+0000 7f216be00640 1 -- 172.21.15.152:0/1566424518 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2154003520 con 0x7f21540043a0 2024-04-20T11:32:36.126 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.124+0000 7f216be00640 1 --2- 172.21.15.152:0/1566424518 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f21540043a0 0x7f2154004880 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f215c006fd0 tx=0x7f2154005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ee0f78bebb69092 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:36.127 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.124+0000 7f216b400640 1 -- 172.21.15.152:0/1566424518 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f215c02f050 con 0x7f21540043a0 2024-04-20T11:32:36.127 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.124+0000 7f216b400640 1 -- 172.21.15.152:0/1566424518 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f215c033630 con 0x7f21540043a0 2024-04-20T11:32:36.127 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.124+0000 7f216b400640 1 -- 172.21.15.152:0/1566424518 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f215c00f040 con 0x7f21540043a0 2024-04-20T11:32:36.127 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.125+0000 7f2171e00640 1 -- 172.21.15.152:0/1566424518 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f21540043a0 msgr2=0x7f2154004880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:36.127 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.125+0000 7f2171e00640 1 --2- 172.21.15.152:0/1566424518 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f21540043a0 0x7f2154004880 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f215c006fd0 tx=0x7f2154005830 comp rx=0 tx=0).stop 2024-04-20T11:32:36.127 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.125+0000 7f2171400640 1 -- 172.21.15.152:0/1566424518 reap_dead start 2024-04-20T11:32:36.127 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.125+0000 7f2171e00640 1 -- 172.21.15.152:0/1566424518 shutdown_connections 2024-04-20T11:32:36.127 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.125+0000 7f2171e00640 1 -- 172.21.15.152:0/1566424518 >> 172.21.15.152:0/1566424518 conn(0x7f216c067ae0 msgr2=0x7f216c06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:36.127 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.125+0000 7f2171e00640 1 -- 172.21.15.152:0/1566424518 shutdown_connections 2024-04-20T11:32:36.127 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.125+0000 7f2171e00640 1 -- 172.21.15.152:0/1566424518 wait complete. 2024-04-20T11:32:36.127 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.126+0000 7f2171e00640 1 Processor -- start 2024-04-20T11:32:36.129 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.127+0000 7f2171e00640 1 -- start start 2024-04-20T11:32:36.129 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.127+0000 7f2171e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f21540043a0 0x7f216c0b0a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:36.129 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.127+0000 7f2171e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f216c0b0f50 0x7f216c157e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:36.129 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.127+0000 7f2171e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f216c158380 0x7f216c0b43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:36.130 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.127+0000 7f2171e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f216c0722e0 con 0x7f216c158380 2024-04-20T11:32:36.130 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.127+0000 7f2171e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f216c072160 con 0x7f216c0b0f50 2024-04-20T11:32:36.130 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.128+0000 7f2171e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f216c072460 con 0x7f21540043a0 2024-04-20T11:32:36.130 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.128+0000 7f2171400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f216c158380 0x7f216c0b43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:36.130 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.128+0000 7f2171400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f216c158380 0x7f216c0b43d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:42848/0 (socket says 172.21.15.152:42848) 2024-04-20T11:32:36.130 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.128+0000 7f2171400640 1 -- 172.21.15.152:0/228447123 learned_addr learned my addr 172.21.15.152:0/228447123 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:32:36.130 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.128+0000 7f2170a00640 1 --2- 172.21.15.152:0/228447123 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f21540043a0 0x7f216c0b0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:36.130 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.128+0000 7f2171400640 1 -- 172.21.15.152:0/228447123 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f21540043a0 msgr2=0x7f216c0b0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:36.130 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.128+0000 7f216be00640 1 --2- 172.21.15.152:0/228447123 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f216c0b0f50 0x7f216c157e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:36.130 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.128+0000 7f2171400640 1 --2- 172.21.15.152:0/228447123 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f21540043a0 0x7f216c0b0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:36.130 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.128+0000 7f2171400640 1 -- 172.21.15.152:0/228447123 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f216c0b0f50 msgr2=0x7f216c157e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:36.130 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.128+0000 7f2171400640 1 --2- 172.21.15.152:0/228447123 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f216c0b0f50 0x7f216c157e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:36.130 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.128+0000 7f2171400640 1 -- 172.21.15.152:0/228447123 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f216c158800 con 0x7f216c158380 2024-04-20T11:32:36.130 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.128+0000 7f2170a00640 1 --2- 172.21.15.152:0/228447123 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f21540043a0 0x7f216c0b0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:32:36.131 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.129+0000 7f2171400640 1 --2- 172.21.15.152:0/228447123 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f216c158380 0x7f216c0b43d0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f216c06fe80 tx=0x7f216400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:36.131 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.129+0000 7f2168c00640 1 -- 172.21.15.152:0/228447123 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f216400ce70 con 0x7f216c158380 2024-04-20T11:32:36.131 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.129+0000 7f2168c00640 1 -- 172.21.15.152:0/228447123 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f216400a610 con 0x7f216c158380 2024-04-20T11:32:36.131 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.129+0000 7f2168c00640 1 -- 172.21.15.152:0/228447123 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f216400aa00 con 0x7f216c158380 2024-04-20T11:32:36.131 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.129+0000 7f2171e00640 1 -- 172.21.15.152:0/228447123 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f216c156da0 con 0x7f216c158380 2024-04-20T11:32:36.131 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.129+0000 7f2171e00640 1 -- 172.21.15.152:0/228447123 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f216c157360 con 0x7f216c158380 2024-04-20T11:32:36.131 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.130+0000 7f2171e00640 1 -- 172.21.15.152:0/228447123 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f216c075e60 con 0x7f216c158380 2024-04-20T11:32:36.131 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.131+0000 7f2168c00640 1 -- 172.21.15.152:0/228447123 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 91142+0+0 (secure 0 0 0) 0x7f2164011070 con 0x7f216c158380 2024-04-20T11:32:36.132 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.132+0000 7f2168c00640 1 --2- 172.21.15.152:0/228447123 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f215006ca10 0x7f215006eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:36.133 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.132+0000 7f2168c00640 1 -- 172.21.15.152:0/228447123 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5502+0+0 (secure 0 0 0) 0x7f216408b690 con 0x7f216c158380 2024-04-20T11:32:36.133 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.133+0000 7f2170a00640 1 --2- 172.21.15.152:0/228447123 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f215006ca10 0x7f215006eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:36.134 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.134+0000 7f2170a00640 1 --2- 172.21.15.152:0/228447123 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f215006ca10 0x7f215006eed0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2160009a60 tx=0x7f2160008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:36.137 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.137+0000 7f2168c00640 1 -- 172.21.15.152:0/228447123 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f216401a060 con 0x7f216c158380 2024-04-20T11:32:36.377 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.376+0000 7f2171e00640 1 -- 172.21.15.152:0/228447123 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f216c072c30 con 0x7f216c158380 2024-04-20T11:32:36.378 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.378+0000 7f2168c00640 1 -- 172.21.15.152:0/228447123 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+3878 (secure 0 0 0) 0x7f21640599e0 con 0x7f216c158380 2024-04-20T11:32:36.378 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:32:36.378 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":46,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5234,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6836/359391273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6835","nonce":359391273},{"type":"v1","addr":"172.21.15.152:6836","nonce":359391273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":5237,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/4269836436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":4269836436},{"type":"v1","addr":"172.21.15.152:6837","nonce":4269836436}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":5420,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/3199836495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6833","nonce":3199836495},{"type":"v1","addr":"172.21.15.175:6835","nonce":3199836495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46},{"gid":5611,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6834/4153485754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":4153485754},{"type":"v1","addr":"172.21.15.175:6834","nonce":4153485754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[]} 2024-04-20T11:32:36.379 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 46 2024-04-20T11:32:36.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.380+0000 7f2171e00640 1 -- 172.21.15.152:0/228447123 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f215006ca10 msgr2=0x7f215006eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:36.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.380+0000 7f2171e00640 1 --2- 172.21.15.152:0/228447123 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f215006ca10 0x7f215006eed0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2160009a60 tx=0x7f2160008040 comp rx=0 tx=0).stop 2024-04-20T11:32:36.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.380+0000 7f2171e00640 1 -- 172.21.15.152:0/228447123 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f216c158380 msgr2=0x7f216c0b43d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:36.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.380+0000 7f2171e00640 1 --2- 172.21.15.152:0/228447123 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f216c158380 0x7f216c0b43d0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f216c06fe80 tx=0x7f216400cc20 comp rx=0 tx=0).stop 2024-04-20T11:32:36.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.380+0000 7f2171e00640 1 -- 172.21.15.152:0/228447123 shutdown_connections 2024-04-20T11:32:36.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.380+0000 7f2171e00640 1 --2- 172.21.15.152:0/228447123 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f215006ca10 0x7f215006eed0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:36.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.380+0000 7f2171e00640 1 --2- 172.21.15.152:0/228447123 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f216c158380 0x7f216c0b43d0 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:36.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.380+0000 7f2171e00640 1 --2- 172.21.15.152:0/228447123 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f216c0b0f50 0x7f216c157e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:36.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.380+0000 7f2171e00640 1 --2- 172.21.15.152:0/228447123 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f21540043a0 0x7f216c0b0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:36.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.380+0000 7f2171e00640 1 -- 172.21.15.152:0/228447123 >> 172.21.15.152:0/228447123 conn(0x7f216c067ae0 msgr2=0x7f216c0cd600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:36.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.380+0000 7f2171e00640 1 -- 172.21.15.152:0/228447123 shutdown_connections 2024-04-20T11:32:36.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.380+0000 7f2171e00640 1 -- 172.21.15.152:0/228447123 wait complete. 2024-04-20T11:32:36.399 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-20T11:32:36.399 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-20T11:32:36.399 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-20T11:32:36.400 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T11:32:36.400 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-20T11:32:36.400 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-20T11:32:36.400 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-20T11:32:36.400 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T11:32:36.400 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-20T11:32:36.400 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-20T11:32:36.401 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-20T11:32:36.401 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T11:32:36.401 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-20T11:32:36.401 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-20T11:32:36.401 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-20T11:32:36.401 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T11:32:36.418 INFO:tasks.ceph.mds.c:Stopped 2024-04-20T11:32:36.418 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-20T11:32:36.425 INFO:tasks.ceph.mds.a:Stopped 2024-04-20T11:32:36.425 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-20T11:32:36.426 INFO:tasks.ceph.mds.b:Stopped 2024-04-20T11:32:36.426 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-20T11:32:36.431 INFO:tasks.ceph.mds.d:Stopped 2024-04-20T11:32:36.432 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-20T11:32:36.470 INFO:tasks.ceph.mds.c:Started 2024-04-20T11:32:36.470 DEBUG:teuthology.parallel:result is None 2024-04-20T11:32:36.476 INFO:tasks.ceph.mds.a:Started 2024-04-20T11:32:36.476 DEBUG:teuthology.parallel:result is None 2024-04-20T11:32:36.478 INFO:tasks.ceph.mds.b:Started 2024-04-20T11:32:36.479 DEBUG:teuthology.parallel:result is None 2024-04-20T11:32:36.485 INFO:tasks.ceph.mds.d:Started 2024-04-20T11:32:36.485 DEBUG:teuthology.parallel:result is None 2024-04-20T11:32:36.485 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-20T11:32:36.616 INFO:tasks.ceph.mds.c.smithi152.stdout:starting mds.c at 2024-04-20T11:32:36.618 INFO:tasks.ceph.mds.a.smithi152.stdout:starting mds.a at 2024-04-20T11:32:36.627 INFO:tasks.ceph.mds.b.smithi175.stdout:starting mds.b at 2024-04-20T11:32:36.641 INFO:tasks.ceph.mds.d.smithi175.stdout:starting mds.d at 2024-04-20T11:32:36.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.661+0000 7f0b24a00640 1 Processor -- start 2024-04-20T11:32:36.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.662+0000 7f0b24a00640 1 -- start start 2024-04-20T11:32:36.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.663+0000 7f0b24a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b20145f00 0x7f0b200cd0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:36.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.663+0000 7f0b24a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f0b200cd610 0x7f0b2006e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:36.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.663+0000 7f0b24a00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b2006f110 con 0x7f0b2006e900 2024-04-20T11:32:36.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.663+0000 7f0b24a00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f0b200cf9f0 con 0x7f0b200cd610 2024-04-20T11:32:36.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.663+0000 7f0b24a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0b200cda50 con 0x7f0b20145f00 2024-04-20T11:32:36.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.663+0000 7f0b15400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f0b200cd610 0x7f0b2006e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:36.664 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.663+0000 7f0b15400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f0b200cd610 0x7f0b2006e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:42888/0 (socket says 172.21.15.152:42888) 2024-04-20T11:32:36.664 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.663+0000 7f0b15400640 1 -- 172.21.15.152:0/581133322 learned_addr learned my addr 172.21.15.152:0/581133322 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:32:36.664 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.663+0000 7f0b1de00640 1 --2- 172.21.15.152:0/581133322 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b20145f00 0x7f0b200cd0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:36.664 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.664+0000 7f0b15400640 1 -- 172.21.15.152:0/581133322 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b20145f00 msgr2=0x7f0b200cd0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:36.664 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.664+0000 7f0b15400640 1 --2- 172.21.15.152:0/581133322 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b20145f00 0x7f0b200cd0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:36.664 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.664+0000 7f0b15400640 1 -- 172.21.15.152:0/581133322 >> v1:172.21.15.175:6789/0 conn(0x7f0b2006e900 legacy=0x7f0b201601c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:36.664 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.664+0000 7f0b15400640 1 -- 172.21.15.152:0/581133322 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b2006f3f0 con 0x7f0b200cd610 2024-04-20T11:32:36.664 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.665+0000 7f0b15400640 1 --2- 172.21.15.152:0/581133322 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0b200cd610 0x7f0b2006e3c0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f0b10009870 tx=0x7f0b1002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1aac4303dc5518c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:36.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.665+0000 7f0b1d400640 1 -- 172.21.15.152:0/581133322 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b10034070 con 0x7f0b200cd610 2024-04-20T11:32:36.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.665+0000 7f0b1d400640 1 -- 172.21.15.152:0/581133322 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b10030840 con 0x7f0b200cd610 2024-04-20T11:32:36.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.665+0000 7f0b1d400640 1 -- 172.21.15.152:0/581133322 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b10030c10 con 0x7f0b200cd610 2024-04-20T11:32:36.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.666+0000 7f0b24a00640 1 -- 172.21.15.152:0/581133322 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0b200cd610 msgr2=0x7f0b2006e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:36.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.666+0000 7f0b24a00640 1 --2- 172.21.15.152:0/581133322 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0b200cd610 0x7f0b2006e3c0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f0b10009870 tx=0x7f0b1002b980 comp rx=0 tx=0).stop 2024-04-20T11:32:36.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.666+0000 7f0b24a00640 1 -- 172.21.15.152:0/581133322 shutdown_connections 2024-04-20T11:32:36.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.666+0000 7f0b24a00640 1 --2- 172.21.15.152:0/581133322 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0b200cd610 0x7f0b2006e3c0 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:36.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.666+0000 7f0b24a00640 1 --2- 172.21.15.152:0/581133322 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b20145f00 0x7f0b200cd0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:36.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.666+0000 7f0b24a00640 1 -- 172.21.15.152:0/581133322 >> 172.21.15.152:0/581133322 conn(0x7f0b20067cf0 msgr2=0x7f0b2006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:36.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.666+0000 7f0b24a00640 1 -- 172.21.15.152:0/581133322 shutdown_connections 2024-04-20T11:32:36.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.666+0000 7f0b24a00640 1 -- 172.21.15.152:0/581133322 wait complete. 2024-04-20T11:32:36.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.667+0000 7f0b24a00640 1 Processor -- start 2024-04-20T11:32:36.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.668+0000 7f0b24a00640 1 -- start start 2024-04-20T11:32:36.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.668+0000 7f0b24a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b2006e900 0x7f0b2013d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:36.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.668+0000 7f0b24a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0b200cd610 0x7f0b2013d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:36.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.668+0000 7f0b24a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b20145f00 0x7f0b20151510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:36.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.668+0000 7f0b24a00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f0b201462e0 con 0x7f0b200cd610 2024-04-20T11:32:36.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.668+0000 7f0b24a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0b200cf680 con 0x7f0b2006e900 2024-04-20T11:32:36.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.668+0000 7f0b24a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0b2006ed90 con 0x7f0b20145f00 2024-04-20T11:32:36.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.669+0000 7f0b15400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0b200cd610 0x7f0b2013d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:36.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.669+0000 7f0b1de00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b2006e900 0x7f0b2013d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:36.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.669+0000 7f0b15400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0b200cd610 0x7f0b2013d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:42900/0 (socket says 172.21.15.152:42900) 2024-04-20T11:32:36.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.669+0000 7f0b15400640 1 -- 172.21.15.152:0/1495553683 learned_addr learned my addr 172.21.15.152:0/1495553683 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:32:36.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.669+0000 7f0b1e800640 1 --2- 172.21.15.152:0/1495553683 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b20145f00 0x7f0b20151510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:36.670 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.669+0000 7f0b15400640 1 -- 172.21.15.152:0/1495553683 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b20145f00 msgr2=0x7f0b20151510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:36.670 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.669+0000 7f0b15400640 1 --2- 172.21.15.152:0/1495553683 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b20145f00 0x7f0b20151510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:36.670 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.669+0000 7f0b15400640 1 -- 172.21.15.152:0/1495553683 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b2006e900 msgr2=0x7f0b2013d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:36.670 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.669+0000 7f0b15400640 1 --2- 172.21.15.152:0/1495553683 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b2006e900 0x7f0b2013d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:36.670 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.669+0000 7f0b15400640 1 -- 172.21.15.152:0/1495553683 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b20139560 con 0x7f0b200cd610 2024-04-20T11:32:36.670 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.669+0000 7f0b1de00640 1 --2- 172.21.15.152:0/1495553683 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b2006e900 0x7f0b2013d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:32:36.671 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.670+0000 7f0b15400640 1 --2- 172.21.15.152:0/1495553683 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0b200cd610 0x7f0b2013d6a0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f0b10000c00 tx=0x7f0b10004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:36.671 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.670+0000 7f0b16a00640 1 -- 172.21.15.152:0/1495553683 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b10048070 con 0x7f0b200cd610 2024-04-20T11:32:36.671 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.670+0000 7f0b16a00640 1 -- 172.21.15.152:0/1495553683 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b10030640 con 0x7f0b200cd610 2024-04-20T11:32:36.671 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.670+0000 7f0b16a00640 1 -- 172.21.15.152:0/1495553683 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b10034040 con 0x7f0b200cd610 2024-04-20T11:32:36.671 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.670+0000 7f0b24a00640 1 -- 172.21.15.152:0/1495553683 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b20139840 con 0x7f0b200cd610 2024-04-20T11:32:36.671 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.670+0000 7f0b24a00640 1 -- 172.21.15.152:0/1495553683 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b20139e00 con 0x7f0b200cd610 2024-04-20T11:32:36.671 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.671+0000 7f0b24a00640 1 -- 172.21.15.152:0/1495553683 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b200689f0 con 0x7f0b200cd610 2024-04-20T11:32:36.672 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.672+0000 7f0b16a00640 1 -- 172.21.15.152:0/1495553683 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 91142+0+0 (secure 0 0 0) 0x7f0b10037070 con 0x7f0b200cd610 2024-04-20T11:32:36.672 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.673+0000 7f0b16a00640 1 --2- 172.21.15.152:0/1495553683 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f0afc06ca10 0x7f0afc06eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:36.672 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.673+0000 7f0b16a00640 1 -- 172.21.15.152:0/1495553683 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5502+0+0 (secure 0 0 0) 0x7f0b100ad9c0 con 0x7f0b200cd610 2024-04-20T11:32:36.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.674+0000 7f0b1de00640 1 --2- 172.21.15.152:0/1495553683 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f0afc06ca10 0x7f0afc06eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:36.674 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.674+0000 7f0b1de00640 1 --2- 172.21.15.152:0/1495553683 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f0afc06ca10 0x7f0afc06eed0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f0b08006fd0 tx=0x7f0b08008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:36.677 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.677+0000 7f0b16a00640 1 -- 172.21.15.152:0/1495553683 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f0b1007b5b0 con 0x7f0b200cd610 2024-04-20T11:32:36.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:36.890+0000 7f0b24a00640 1 -- 172.21.15.152:0/1495553683 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f0b2015d380 con 0x7f0b200cd610 2024-04-20T11:32:37.009 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.010+0000 7f0b16a00640 1 -- 172.21.15.152:0/1495553683 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v77) ==== 97+0+0 (secure 0 0 0) 0x7f0b1007f8b0 con 0x7f0b200cd610 2024-04-20T11:32:37.010 INFO:teuthology.orchestra.run.smithi152.stderr: removed all blocklist entries 2024-04-20T11:32:37.013 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.013+0000 7f0b24a00640 1 -- 172.21.15.152:0/1495553683 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f0afc06ca10 msgr2=0x7f0afc06eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:37.013 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.013+0000 7f0b24a00640 1 --2- 172.21.15.152:0/1495553683 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f0afc06ca10 0x7f0afc06eed0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f0b08006fd0 tx=0x7f0b08008040 comp rx=0 tx=0).stop 2024-04-20T11:32:37.013 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.013+0000 7f0b24a00640 1 -- 172.21.15.152:0/1495553683 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0b200cd610 msgr2=0x7f0b2013d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:37.013 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.013+0000 7f0b24a00640 1 --2- 172.21.15.152:0/1495553683 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0b200cd610 0x7f0b2013d6a0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f0b10000c00 tx=0x7f0b10004730 comp rx=0 tx=0).stop 2024-04-20T11:32:37.014 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.014+0000 7f0b24a00640 1 -- 172.21.15.152:0/1495553683 shutdown_connections 2024-04-20T11:32:37.014 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.014+0000 7f0b24a00640 1 --2- 172.21.15.152:0/1495553683 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f0afc06ca10 0x7f0afc06eed0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:37.014 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.014+0000 7f0b24a00640 1 --2- 172.21.15.152:0/1495553683 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b20145f00 0x7f0b20151510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:37.014 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.014+0000 7f0b24a00640 1 --2- 172.21.15.152:0/1495553683 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0b200cd610 0x7f0b2013d6a0 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:37.014 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.014+0000 7f0b24a00640 1 --2- 172.21.15.152:0/1495553683 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b2006e900 0x7f0b2013d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:37.014 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.014+0000 7f0b24a00640 1 -- 172.21.15.152:0/1495553683 >> 172.21.15.152:0/1495553683 conn(0x7f0b20067cf0 msgr2=0x7f0b2015ff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:37.014 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.014+0000 7f0b24a00640 1 -- 172.21.15.152:0/1495553683 shutdown_connections 2024-04-20T11:32:37.014 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.015+0000 7f0b24a00640 1 -- 172.21.15.152:0/1495553683 wait complete. 2024-04-20T11:32:37.036 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-20T11:32:37.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.229+0000 7f64ec800640 1 Processor -- start 2024-04-20T11:32:37.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.229+0000 7f64ec800640 1 -- start start 2024-04-20T11:32:37.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.231+0000 7f64ec800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f64e40726f0 0x7f64e40cd580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:37.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.231+0000 7f64ec800640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64e4146340 con 0x7f64e40cd0d0 2024-04-20T11:32:37.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.231+0000 7f64ec800640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64e4072d50 con 0x7f64e4145f60 2024-04-20T11:32:37.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.231+0000 7f64ec800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f64e406e9f0 con 0x7f64e40726f0 2024-04-20T11:32:37.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.232+0000 7f64e9400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f64e40cd0d0 0x7f64e4071fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:48350/0 (socket says 172.21.15.152:48350) 2024-04-20T11:32:37.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.232+0000 7f64e9400640 1 -- 172.21.15.152:0/2369503277 learned_addr learned my addr 172.21.15.152:0/2369503277 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:32:37.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.232+0000 7f64ea800640 1 --2- 172.21.15.152:0/2369503277 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f64e40726f0 0x7f64e40cd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:37.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.232+0000 7f64e8a00640 1 -- 172.21.15.152:0/2369503277 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2630911816 0 0) 0x7f64e4146340 con 0x7f64e40cd0d0 2024-04-20T11:32:37.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.232+0000 7f64e8a00640 1 -- 172.21.15.152:0/2369503277 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f64cc003520 con 0x7f64e40cd0d0 2024-04-20T11:32:37.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.232+0000 7f64e8a00640 1 -- 172.21.15.152:0/2369503277 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2852140115 0 0) 0x7f64cc003520 con 0x7f64e40cd0d0 2024-04-20T11:32:37.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.232+0000 7f64e8a00640 1 -- 172.21.15.152:0/2369503277 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f64e40726f0 msgr2=0x7f64e40cd580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:37.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.232+0000 7f64e8a00640 1 --2- 172.21.15.152:0/2369503277 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f64e40726f0 0x7f64e40cd580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:37.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.232+0000 7f64e8a00640 1 -- 172.21.15.152:0/2369503277 >> v1:172.21.15.175:6789/0 conn(0x7f64e4145f60 legacy=0x7f64e406ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:37.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.232+0000 7f64e8a00640 1 -- 172.21.15.152:0/2369503277 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64e406e0c0 con 0x7f64e40cd0d0 2024-04-20T11:32:37.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.232+0000 7f64e8a00640 1 -- 172.21.15.152:0/2369503277 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f64d4001690 con 0x7f64e40cd0d0 2024-04-20T11:32:37.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.232+0000 7f64e8a00640 1 -- 172.21.15.152:0/2369503277 >> v1:172.21.15.152:6789/0 conn(0x7f64e40cd0d0 legacy=0x7f64e4071fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:37.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.233+0000 7f64e8a00640 1 --2- 172.21.15.152:0/2369503277 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f64cc003a50 0x7f64cc003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:37.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.233+0000 7f64e8a00640 1 --2- 172.21.15.152:0/2369503277 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f64cc004800 0x7f64cc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:37.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.233+0000 7f64e8a00640 1 --2- 172.21.15.152:0/2369503277 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f64e40726f0 0x7f64cc005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:37.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.233+0000 7f64e8a00640 1 -- 172.21.15.152:0/2369503277 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f64e406e9f0 con 0x7f64cc004800 2024-04-20T11:32:37.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.233+0000 7f64e8a00640 1 -- 172.21.15.152:0/2369503277 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f64cc00bbb0 con 0x7f64cc003a50 2024-04-20T11:32:37.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.233+0000 7f64e8a00640 1 -- 172.21.15.152:0/2369503277 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f64cc00bdb0 con 0x7f64e40726f0 2024-04-20T11:32:37.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.233+0000 7f64e9400640 1 --2- 172.21.15.152:0/2369503277 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f64cc004800 0x7f64cc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:37.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.233+0000 7f64e9e00640 1 --2- 172.21.15.152:0/2369503277 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f64cc003a50 0x7f64cc003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:37.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.233+0000 7f64e9400640 1 -- 172.21.15.152:0/2369503277 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f64e40726f0 msgr2=0x7f64cc005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:37.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.233+0000 7f64e9400640 1 --2- 172.21.15.152:0/2369503277 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f64e40726f0 0x7f64cc005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:37.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.233+0000 7f64e9400640 1 -- 172.21.15.152:0/2369503277 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f64cc003a50 msgr2=0x7f64cc003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:37.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.233+0000 7f64e9400640 1 --2- 172.21.15.152:0/2369503277 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f64cc003a50 0x7f64cc003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:37.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.233+0000 7f64e9400640 1 -- 172.21.15.152:0/2369503277 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64cc003520 con 0x7f64cc004800 2024-04-20T11:32:37.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.233+0000 7f64e9e00640 1 --2- 172.21.15.152:0/2369503277 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f64cc003a50 0x7f64cc003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:32:37.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.235+0000 7f64e9400640 1 --2- 172.21.15.152:0/2369503277 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f64cc004800 0x7f64cc004c80 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f64e40cce90 tx=0x7f64cc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c46a0b2d0a8fbf server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:37.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.235+0000 7f64e8a00640 1 -- 172.21.15.152:0/2369503277 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64d4007c80 con 0x7f64cc004800 2024-04-20T11:32:37.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.235+0000 7f64e8a00640 1 -- 172.21.15.152:0/2369503277 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64d4030070 con 0x7f64cc004800 2024-04-20T11:32:37.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.235+0000 7f64e8a00640 1 -- 172.21.15.152:0/2369503277 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64d4005cd0 con 0x7f64cc004800 2024-04-20T11:32:37.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.235+0000 7f64ec800640 1 -- 172.21.15.152:0/2369503277 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f64cc004800 msgr2=0x7f64cc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:37.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.235+0000 7f64ec800640 1 --2- 172.21.15.152:0/2369503277 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f64cc004800 0x7f64cc004c80 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f64e40cce90 tx=0x7f64cc005830 comp rx=0 tx=0).stop 2024-04-20T11:32:37.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.236+0000 7f64ea800640 1 -- 172.21.15.152:0/2369503277 reap_dead start 2024-04-20T11:32:37.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.236+0000 7f64ec800640 1 -- 172.21.15.152:0/2369503277 shutdown_connections 2024-04-20T11:32:37.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.236+0000 7f64ec800640 1 -- 172.21.15.152:0/2369503277 >> 172.21.15.152:0/2369503277 conn(0x7f64e4067cf0 msgr2=0x7f64e406b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:37.237 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.236+0000 7f64ec800640 1 -- 172.21.15.152:0/2369503277 shutdown_connections 2024-04-20T11:32:37.237 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.236+0000 7f64ec800640 1 -- 172.21.15.152:0/2369503277 wait complete. 2024-04-20T11:32:37.237 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.238+0000 7f64ec800640 1 Processor -- start 2024-04-20T11:32:37.237 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.238+0000 7f64ec800640 1 -- start start 2024-04-20T11:32:37.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.239+0000 7f64ec800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f64cc004800 0x7f64e4141660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:37.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.239+0000 7f64ec800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f64e40c6240 0x7f64e40d0260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:37.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.239+0000 7f64ec800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f64e40d07a0 0x7f64e40c70a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:37.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.239+0000 7f64ec800640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f64e40cd580 con 0x7f64e40d07a0 2024-04-20T11:32:37.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.239+0000 7f64ec800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f64e406e680 con 0x7f64cc004800 2024-04-20T11:32:37.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.239+0000 7f64ec800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f64e40cd700 con 0x7f64e40c6240 2024-04-20T11:32:37.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.240+0000 7f64ea800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f64e40d07a0 0x7f64e40c70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:37.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.240+0000 7f64ea800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f64e40d07a0 0x7f64e40c70a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:42928/0 (socket says 172.21.15.152:42928) 2024-04-20T11:32:37.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.240+0000 7f64ea800640 1 -- 172.21.15.152:0/1129877331 learned_addr learned my addr 172.21.15.152:0/1129877331 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:32:37.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.240+0000 7f64e9400640 1 --2- 172.21.15.152:0/1129877331 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f64e40c6240 0x7f64e40d0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:37.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.240+0000 7f64ea800640 1 -- 172.21.15.152:0/1129877331 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f64e40c6240 msgr2=0x7f64e40d0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:37.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.240+0000 7f64ea800640 1 --2- 172.21.15.152:0/1129877331 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f64e40c6240 0x7f64e40d0260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:37.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.240+0000 7f64e9e00640 1 --2- 172.21.15.152:0/1129877331 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f64cc004800 0x7f64e4141660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:37.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.240+0000 7f64ea800640 1 -- 172.21.15.152:0/1129877331 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f64cc004800 msgr2=0x7f64e4141660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:37.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.240+0000 7f64ea800640 1 --2- 172.21.15.152:0/1129877331 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f64cc004800 0x7f64e4141660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:37.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.240+0000 7f64ea800640 1 -- 172.21.15.152:0/1129877331 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64e40b4f60 con 0x7f64e40d07a0 2024-04-20T11:32:37.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.241+0000 7f64ea800640 1 --2- 172.21.15.152:0/1129877331 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f64e40d07a0 0x7f64e40c70a0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f64e40b5810 tx=0x7f64e000b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:37.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.241+0000 7f64de000640 1 -- 172.21.15.152:0/1129877331 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e000bae0 con 0x7f64e40d07a0 2024-04-20T11:32:37.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.241+0000 7f64de000640 1 -- 172.21.15.152:0/1129877331 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64e001a070 con 0x7f64e40d07a0 2024-04-20T11:32:37.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.241+0000 7f64de000640 1 -- 172.21.15.152:0/1129877331 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e0014400 con 0x7f64e40d07a0 2024-04-20T11:32:37.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.241+0000 7f64ec800640 1 -- 172.21.15.152:0/1129877331 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64e40c5480 con 0x7f64e40d07a0 2024-04-20T11:32:37.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.241+0000 7f64ec800640 1 -- 172.21.15.152:0/1129877331 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64e40c5a40 con 0x7f64e40d07a0 2024-04-20T11:32:37.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.242+0000 7f64ec800640 1 -- 172.21.15.152:0/1129877331 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64e40689f0 con 0x7f64e40d07a0 2024-04-20T11:32:37.243 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.243+0000 7f64de000640 1 -- 172.21.15.152:0/1129877331 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 91142+0+0 (secure 0 0 0) 0x7f64e0005d10 con 0x7f64e40d07a0 2024-04-20T11:32:37.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.244+0000 7f64de000640 1 --2- 172.21.15.152:0/1129877331 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f64c806ca10 0x7f64c806eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:37.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.244+0000 7f64de000640 1 -- 172.21.15.152:0/1129877331 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 4857+0+0 (secure 0 0 0) 0x7f64e008bbc0 con 0x7f64e40d07a0 2024-04-20T11:32:37.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.245+0000 7f64e9e00640 1 --2- 172.21.15.152:0/1129877331 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f64c806ca10 0x7f64c806eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:37.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.246+0000 7f64e9e00640 1 --2- 172.21.15.152:0/1129877331 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f64c806ca10 0x7f64c806eed0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f64d8001630 tx=0x7f64d8009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:37.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.248+0000 7f64de000640 1 -- 172.21.15.152:0/1129877331 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f64e00122c0 con 0x7f64e40d07a0 2024-04-20T11:32:37.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.501+0000 7f64ec800640 1 -- 172.21.15.152:0/1129877331 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f64e406f840 con 0x7f64e40d07a0 2024-04-20T11:32:37.502 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.502+0000 7f64de000640 1 -- 172.21.15.152:0/1129877331 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) ==== 80+0+5304 (secure 0 0 0) 0x7f64e005a080 con 0x7f64e40d07a0 2024-04-20T11:32:37.502 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:32:37.502 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-04-20T11:32:37.502 INFO:teuthology.orchestra.run.smithi152.stdout: "auth_dump": [ 2024-04-20T11:32:37.502 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:32:37.502 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "mds.a", 2024-04-20T11:32:37.502 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBCpiNmyFRJIBAAj2aIMXdb65nqRgKFDBLLzA==", 2024-04-20T11:32:37.502 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:32:37.503 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow", 2024-04-20T11:32:37.503 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow *", 2024-04-20T11:32:37.503 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow *", 2024-04-20T11:32:37.503 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:32:37.503 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:37.503 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:37.503 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:32:37.503 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "mds.b", 2024-04-20T11:32:37.503 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBCpiNmVPytOhAAzQkvNOJaNy+MeKETWhYLyg==", 2024-04-20T11:32:37.503 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:32:37.503 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow", 2024-04-20T11:32:37.503 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow *", 2024-04-20T11:32:37.503 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow *", 2024-04-20T11:32:37.503 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:32:37.503 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:37.504 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:37.504 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:32:37.504 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "mds.c", 2024-04-20T11:32:37.504 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBCpiNm5ViKLRAAg7VBQG4hFTLYUpJMpBAaVA==", 2024-04-20T11:32:37.504 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:32:37.504 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow", 2024-04-20T11:32:37.504 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow *", 2024-04-20T11:32:37.504 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow *", 2024-04-20T11:32:37.504 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:32:37.504 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:37.504 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:37.504 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:32:37.504 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "mds.d", 2024-04-20T11:32:37.504 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBDpiNmLWhaCxAA2e4qd/7+TKenJlWICIsvGg==", 2024-04-20T11:32:37.505 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:32:37.505 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow", 2024-04-20T11:32:37.505 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow *", 2024-04-20T11:32:37.505 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow *", 2024-04-20T11:32:37.505 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:32:37.505 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:37.505 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:37.505 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:32:37.505 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "osd.0", 2024-04-20T11:32:37.505 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBHpiNmpuhyDBAAsQXX9DNvWlo0UYh53lb+FA==", 2024-04-20T11:32:37.505 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:32:37.505 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow profile osd", 2024-04-20T11:32:37.505 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile osd", 2024-04-20T11:32:37.505 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:32:37.506 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:37.506 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:37.506 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:32:37.506 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "osd.1", 2024-04-20T11:32:37.506 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBKpiNmxBiLJhAAHon1adC8G2jbYoDcWdE9jA==", 2024-04-20T11:32:37.506 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:32:37.506 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow profile osd", 2024-04-20T11:32:37.506 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile osd", 2024-04-20T11:32:37.506 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:32:37.506 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:37.506 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:37.506 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:32:37.506 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "osd.2", 2024-04-20T11:32:37.507 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBNpiNmdn0zMhAAqwIO2kKahdSzTVh8/p3FCg==", 2024-04-20T11:32:37.507 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:32:37.507 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow profile osd", 2024-04-20T11:32:37.507 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile osd", 2024-04-20T11:32:37.507 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:32:37.507 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:37.507 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:37.507 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:32:37.507 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "osd.3", 2024-04-20T11:32:37.507 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBRpiNmTPCYAhAAHFNHX/xKn21jEFHDkHMjEw==", 2024-04-20T11:32:37.507 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:32:37.507 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow profile osd", 2024-04-20T11:32:37.507 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile osd", 2024-04-20T11:32:37.507 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:32:37.507 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:37.508 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:37.508 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:32:37.508 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "osd.4", 2024-04-20T11:32:37.508 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBXpiNmM8cVFxAAJl9Hu+MvlkEiMglutI89eg==", 2024-04-20T11:32:37.508 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:32:37.508 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow profile osd", 2024-04-20T11:32:37.508 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile osd", 2024-04-20T11:32:37.508 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:32:37.508 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:37.508 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:37.508 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:32:37.508 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "osd.5", 2024-04-20T11:32:37.508 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBapiNmnGxJMRAA90P0T+VsTA8LCTcexTVr7w==", 2024-04-20T11:32:37.508 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:32:37.509 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow profile osd", 2024-04-20T11:32:37.509 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile osd", 2024-04-20T11:32:37.509 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:32:37.509 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:37.509 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:37.509 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:32:37.509 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "osd.6", 2024-04-20T11:32:37.509 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBepiNmyLXWDxAAhtHD5h478mLovE5IejAFng==", 2024-04-20T11:32:37.509 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:32:37.509 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow profile osd", 2024-04-20T11:32:37.509 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile osd", 2024-04-20T11:32:37.509 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:32:37.509 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:37.509 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:37.510 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:32:37.510 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "osd.7", 2024-04-20T11:32:37.510 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBhpiNmB3jwHBAAqco9om6r2N/GqAV6rVKiaA==", 2024-04-20T11:32:37.510 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:32:37.510 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow profile osd", 2024-04-20T11:32:37.510 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile osd", 2024-04-20T11:32:37.510 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:32:37.510 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:37.510 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:37.510 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:32:37.510 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "client.0", 2024-04-20T11:32:37.510 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBDpiNmtsBWFhAArq9A7r33CcwgMmOU2pAGPw==", 2024-04-20T11:32:37.510 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:32:37.510 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow", 2024-04-20T11:32:37.511 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow r", 2024-04-20T11:32:37.511 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-20T11:32:37.511 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:37.511 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:37.511 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:32:37.511 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "client.1", 2024-04-20T11:32:37.511 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBDpiNm6aNnJxAAB431GSGytZ0IKWMh4DMalw==", 2024-04-20T11:32:37.511 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:32:37.511 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow", 2024-04-20T11:32:37.511 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow r", 2024-04-20T11:32:37.511 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow rw", 2024-04-20T11:32:37.511 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow rwx" 2024-04-20T11:32:37.511 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:37.511 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:37.512 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:32:37.512 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "client.2", 2024-04-20T11:32:37.512 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBDpiNm5QLtHRAAa2kHTDVdgSh/B4XGZaKa5Q==", 2024-04-20T11:32:37.512 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:32:37.512 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow", 2024-04-20T11:32:37.512 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow r", 2024-04-20T11:32:37.512 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow rw", 2024-04-20T11:32:37.512 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow rwx" 2024-04-20T11:32:37.512 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:37.512 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:37.512 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:32:37.512 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "client.3", 2024-04-20T11:32:37.512 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBDpiNmw95SMBAAdodzTuPp4WdKSEf+jI8Uuw==", 2024-04-20T11:32:37.512 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:32:37.513 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow", 2024-04-20T11:32:37.513 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow r", 2024-04-20T11:32:37.513 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow rw", 2024-04-20T11:32:37.513 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow rwx" 2024-04-20T11:32:37.513 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:37.513 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:37.513 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:32:37.513 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "client.admin", 2024-04-20T11:32:37.513 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBBpiNm3xz6HhAA82+N+trDW/AmBr3q8l4ECw==", 2024-04-20T11:32:37.513 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:32:37.513 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow *", 2024-04-20T11:32:37.513 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow *", 2024-04-20T11:32:37.513 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow *", 2024-04-20T11:32:37.513 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:32:37.513 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:37.514 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:37.514 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:32:37.514 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "mgr.x", 2024-04-20T11:32:37.514 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBCpiNmuhalFxAA9dTl/zSbUovCCZAanAipww==", 2024-04-20T11:32:37.514 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:32:37.514 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow *", 2024-04-20T11:32:37.514 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile mgr", 2024-04-20T11:32:37.514 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:32:37.514 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:37.514 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:37.514 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:32:37.514 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "mgr.y", 2024-04-20T11:32:37.514 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBCpiNmP1+5DRAAs6bVAEOr0i6FdPmSEyo/vQ==", 2024-04-20T11:32:37.514 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:32:37.515 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow *", 2024-04-20T11:32:37.515 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile mgr", 2024-04-20T11:32:37.515 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:32:37.515 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:37.515 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:37.515 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-04-20T11:32:37.515 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-04-20T11:32:37.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.504+0000 7f64ec800640 1 -- 172.21.15.152:0/1129877331 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f64c806ca10 msgr2=0x7f64c806eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:37.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.504+0000 7f64ec800640 1 --2- 172.21.15.152:0/1129877331 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f64c806ca10 0x7f64c806eed0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f64d8001630 tx=0x7f64d8009040 comp rx=0 tx=0).stop 2024-04-20T11:32:37.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.504+0000 7f64ec800640 1 -- 172.21.15.152:0/1129877331 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f64e40d07a0 msgr2=0x7f64e40c70a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:37.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.504+0000 7f64ec800640 1 --2- 172.21.15.152:0/1129877331 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f64e40d07a0 0x7f64e40c70a0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f64e40b5810 tx=0x7f64e000b480 comp rx=0 tx=0).stop 2024-04-20T11:32:37.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.505+0000 7f64ec800640 1 -- 172.21.15.152:0/1129877331 shutdown_connections 2024-04-20T11:32:37.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.505+0000 7f64ec800640 1 --2- 172.21.15.152:0/1129877331 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f64c806ca10 0x7f64c806eed0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:37.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.505+0000 7f64ec800640 1 --2- 172.21.15.152:0/1129877331 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f64e40d07a0 0x7f64e40c70a0 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:37.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.505+0000 7f64ec800640 1 --2- 172.21.15.152:0/1129877331 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f64e40c6240 0x7f64e40d0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:37.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.505+0000 7f64ec800640 1 --2- 172.21.15.152:0/1129877331 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f64cc004800 0x7f64e4141660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:37.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.505+0000 7f64ec800640 1 -- 172.21.15.152:0/1129877331 >> 172.21.15.152:0/1129877331 conn(0x7f64e4067cf0 msgr2=0x7f64e406b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:37.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.505+0000 7f64ec800640 1 -- 172.21.15.152:0/1129877331 shutdown_connections 2024-04-20T11:32:37.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.505+0000 7f64ec800640 1 -- 172.21.15.152:0/1129877331 wait complete. 2024-04-20T11:32:37.521 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:32:37.711 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.711+0000 7fec83a00640 1 Processor -- start 2024-04-20T11:32:37.711 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.711+0000 7fec83a00640 1 -- start start 2024-04-20T11:32:37.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.713+0000 7fec83a00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fec7c0cd0d0 0x7fec7c0cd580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:37.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.713+0000 7fec83a00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec7c146340 con 0x7fec7c145f60 2024-04-20T11:32:37.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.713+0000 7fec83a00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec7c072d50 con 0x7fec7c071fe0 2024-04-20T11:32:37.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.713+0000 7fec83a00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fec7c06e9f0 con 0x7fec7c0cd0d0 2024-04-20T11:32:37.714 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.713+0000 7fec81000640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fec7c145f60 0x7fec7c06ee80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:48366/0 (socket says 172.21.15.152:48366) 2024-04-20T11:32:37.715 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.713+0000 7fec81000640 1 -- 172.21.15.152:0/1165364761 learned_addr learned my addr 172.21.15.152:0/1165364761 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:32:37.715 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.714+0000 7fec7be00640 1 --2- 172.21.15.152:0/1165364761 >> v2:172.21.15.175:3300/0 conn(0x7fec7c0cd0d0 0x7fec7c0cd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:37.715 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.714+0000 7fec7b400640 1 -- 172.21.15.152:0/1165364761 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1309149064 0 0) 0x7fec7c146340 con 0x7fec7c145f60 2024-04-20T11:32:37.715 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.714+0000 7fec7b400640 1 -- 172.21.15.152:0/1165364761 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fec64003520 con 0x7fec7c145f60 2024-04-20T11:32:37.715 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.714+0000 7fec7b400640 1 -- 172.21.15.152:0/1165364761 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 873386310 0 0) 0x7fec64003520 con 0x7fec7c145f60 2024-04-20T11:32:37.715 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.714+0000 7fec7b400640 1 -- 172.21.15.152:0/1165364761 >> v2:172.21.15.175:3300/0 conn(0x7fec7c0cd0d0 msgr2=0x7fec7c0cd580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:37.715 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.714+0000 7fec7b400640 1 --2- 172.21.15.152:0/1165364761 >> v2:172.21.15.175:3300/0 conn(0x7fec7c0cd0d0 0x7fec7c0cd580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:37.715 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.714+0000 7fec7b400640 1 -- 172.21.15.152:0/1165364761 >> v1:172.21.15.175:6789/0 conn(0x7fec7c071fe0 legacy=0x7fec7c072470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:37.715 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.714+0000 7fec7b400640 1 -- 172.21.15.152:0/1165364761 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec7c06e0c0 con 0x7fec7c145f60 2024-04-20T11:32:37.715 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.714+0000 7fec7b400640 1 -- 172.21.15.152:0/1165364761 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7fec70001670 con 0x7fec7c145f60 2024-04-20T11:32:37.715 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.714+0000 7fec7b400640 1 -- 172.21.15.152:0/1165364761 >> v1:172.21.15.152:6789/0 conn(0x7fec7c145f60 legacy=0x7fec7c06ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:37.715 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.714+0000 7fec7b400640 1 --2- 172.21.15.152:0/1165364761 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fec64003a70 0x7fec64003e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:37.715 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.714+0000 7fec7b400640 1 --2- 172.21.15.152:0/1165364761 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fec64004800 0x7fec64004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:37.716 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.714+0000 7fec7b400640 1 --2- 172.21.15.152:0/1165364761 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fec64005210 0x7fec6400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:37.716 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.714+0000 7fec7b400640 1 -- 172.21.15.152:0/1165364761 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fec7c06e9f0 con 0x7fec64005210 2024-04-20T11:32:37.716 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.714+0000 7fec7b400640 1 -- 172.21.15.152:0/1165364761 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fec6400c140 con 0x7fec64003a70 2024-04-20T11:32:37.716 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.714+0000 7fec7b400640 1 -- 172.21.15.152:0/1165364761 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fec6400c340 con 0x7fec64004800 2024-04-20T11:32:37.716 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.715+0000 7fec81a00640 1 --2- 172.21.15.152:0/1165364761 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fec64005210 0x7fec6400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:37.716 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.715+0000 7fec81000640 1 --2- 172.21.15.152:0/1165364761 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fec64003a70 0x7fec64003e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:37.716 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.715+0000 7fec81a00640 1 -- 172.21.15.152:0/1165364761 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fec64004800 msgr2=0x7fec64004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:37.716 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.715+0000 7fec81a00640 1 --2- 172.21.15.152:0/1165364761 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fec64004800 0x7fec64004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:37.716 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.715+0000 7fec81a00640 1 -- 172.21.15.152:0/1165364761 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fec64003a70 msgr2=0x7fec64003e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:37.716 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.715+0000 7fec81a00640 1 --2- 172.21.15.152:0/1165364761 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fec64003a70 0x7fec64003e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:37.716 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.715+0000 7fec81a00640 1 -- 172.21.15.152:0/1165364761 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec64003520 con 0x7fec64005210 2024-04-20T11:32:37.717 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.716+0000 7fec81a00640 1 -- 172.21.15.152:0/1165364761 reap_dead start 2024-04-20T11:32:37.717 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.716+0000 7fec81a00640 1 --2- 172.21.15.152:0/1165364761 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fec64005210 0x7fec6400b9c0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fec74001e50 tx=0x7fec64005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a403491a975e6696 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:37.718 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.716+0000 7fec7b400640 1 -- 172.21.15.152:0/1165364761 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec74001a70 con 0x7fec64005210 2024-04-20T11:32:37.718 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.716+0000 7fec7b400640 1 -- 172.21.15.152:0/1165364761 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec74003bb0 con 0x7fec64005210 2024-04-20T11:32:37.718 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.716+0000 7fec7b400640 1 -- 172.21.15.152:0/1165364761 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec740050d0 con 0x7fec64005210 2024-04-20T11:32:37.718 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.717+0000 7fec83a00640 1 -- 172.21.15.152:0/1165364761 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fec64005210 msgr2=0x7fec6400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:37.718 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.717+0000 7fec83a00640 1 --2- 172.21.15.152:0/1165364761 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fec64005210 0x7fec6400b9c0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fec74001e50 tx=0x7fec64005830 comp rx=0 tx=0).stop 2024-04-20T11:32:37.718 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.717+0000 7fec83a00640 1 -- 172.21.15.152:0/1165364761 shutdown_connections 2024-04-20T11:32:37.718 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.717+0000 7fec83a00640 1 --2- 172.21.15.152:0/1165364761 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fec64005210 0x7fec6400b9c0 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:37.718 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.717+0000 7fec83a00640 1 -- 172.21.15.152:0/1165364761 >> 172.21.15.152:0/1165364761 conn(0x7fec7c067cf0 msgr2=0x7fec7c06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:37.718 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.717+0000 7fec83a00640 1 -- 172.21.15.152:0/1165364761 shutdown_connections 2024-04-20T11:32:37.718 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.717+0000 7fec83a00640 1 -- 172.21.15.152:0/1165364761 wait complete. 2024-04-20T11:32:37.721 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.719+0000 7fec83a00640 1 Processor -- start 2024-04-20T11:32:37.721 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.719+0000 7fec83a00640 1 -- start start 2024-04-20T11:32:37.721 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.720+0000 7fec83a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fec64005210 0x7fec7c141540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:37.721 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.720+0000 7fec83a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fec7c13df30 0x7fec7c13d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:37.721 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.720+0000 7fec83a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fec7c13fc70 0x7fec7c143b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:37.721 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.720+0000 7fec83a00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fec7c0cd580 con 0x7fec64005210 2024-04-20T11:32:37.721 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.720+0000 7fec83a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fec7c06e680 con 0x7fec7c13df30 2024-04-20T11:32:37.721 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.720+0000 7fec83a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fec7c0cd700 con 0x7fec7c13fc70 2024-04-20T11:32:37.721 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.720+0000 7fec81000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fec64005210 0x7fec7c141540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:37.722 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.720+0000 7fec81000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fec64005210 0x7fec7c141540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:42948/0 (socket says 172.21.15.152:42948) 2024-04-20T11:32:37.722 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.720+0000 7fec81000640 1 -- 172.21.15.152:0/2955072931 learned_addr learned my addr 172.21.15.152:0/2955072931 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:32:37.722 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.720+0000 7fec81000640 1 -- 172.21.15.152:0/2955072931 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fec7c13fc70 msgr2=0x7fec7c143b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:37.722 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.720+0000 7fec81000640 1 --2- 172.21.15.152:0/2955072931 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fec7c13fc70 0x7fec7c143b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:37.722 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.720+0000 7fec81000640 1 -- 172.21.15.152:0/2955072931 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fec7c13df30 msgr2=0x7fec7c13d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:37.722 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.720+0000 7fec81000640 1 --2- 172.21.15.152:0/2955072931 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fec7c13df30 0x7fec7c13d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:37.722 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.720+0000 7fec81000640 1 -- 172.21.15.152:0/2955072931 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec7c1404a0 con 0x7fec64005210 2024-04-20T11:32:37.722 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.721+0000 7fec81000640 1 --2- 172.21.15.152:0/2955072931 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fec64005210 0x7fec7c141540 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fec70001050 tx=0x7fec7000ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:37.722 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.721+0000 7fec78c00640 1 -- 172.21.15.152:0/2955072931 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec7000c660 con 0x7fec64005210 2024-04-20T11:32:37.722 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.721+0000 7fec78c00640 1 -- 172.21.15.152:0/2955072931 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec70018070 con 0x7fec64005210 2024-04-20T11:32:37.722 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.721+0000 7fec78c00640 1 -- 172.21.15.152:0/2955072931 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec7000ca40 con 0x7fec64005210 2024-04-20T11:32:37.722 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.721+0000 7fec83a00640 1 -- 172.21.15.152:0/2955072931 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec7c140780 con 0x7fec64005210 2024-04-20T11:32:37.722 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.721+0000 7fec83a00640 1 -- 172.21.15.152:0/2955072931 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec7c140d40 con 0x7fec64005210 2024-04-20T11:32:37.723 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.722+0000 7fec83a00640 1 -- 172.21.15.152:0/2955072931 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec7c0689f0 con 0x7fec64005210 2024-04-20T11:32:37.723 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.723+0000 7fec78c00640 1 -- 172.21.15.152:0/2955072931 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 91142+0+0 (secure 0 0 0) 0x7fec7000cbe0 con 0x7fec64005210 2024-04-20T11:32:37.723 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.724+0000 7fec78c00640 1 --2- 172.21.15.152:0/2955072931 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fec6006ca40 0x7fec6006ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:37.723 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.724+0000 7fec78c00640 1 -- 172.21.15.152:0/2955072931 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 4857+0+0 (secure 0 0 0) 0x7fec7008b6e0 con 0x7fec64005210 2024-04-20T11:32:37.724 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.725+0000 7fec7be00640 1 --2- 172.21.15.152:0/2955072931 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fec6006ca40 0x7fec6006ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:37.725 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.726+0000 7fec7be00640 1 --2- 172.21.15.152:0/2955072931 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fec6006ca40 0x7fec6006ef00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fec6c006a90 tx=0x7fec6c006a20 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:37.727 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.728+0000 7fec78c00640 1 -- 172.21.15.152:0/2955072931 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fec70059d40 con 0x7fec64005210 2024-04-20T11:32:37.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.930+0000 7fec83a00640 1 -- 172.21.15.152:0/2955072931 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fec7c0b3710 con 0x7fec64005210 2024-04-20T11:32:37.931 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.932+0000 7fec78c00640 1 -- 172.21.15.152:0/2955072931 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+12952 (secure 0 0 0) 0x7fec70059640 con 0x7fec64005210 2024-04-20T11:32:37.932 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:32:37.932 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":79,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:32:37.021911+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:32:37.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.934+0000 7fec83a00640 1 -- 172.21.15.152:0/2955072931 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fec6006ca40 msgr2=0x7fec6006ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:37.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.934+0000 7fec83a00640 1 --2- 172.21.15.152:0/2955072931 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fec6006ca40 0x7fec6006ef00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fec6c006a90 tx=0x7fec6c006a20 comp rx=0 tx=0).stop 2024-04-20T11:32:37.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.934+0000 7fec83a00640 1 -- 172.21.15.152:0/2955072931 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fec64005210 msgr2=0x7fec7c141540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:37.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.934+0000 7fec83a00640 1 --2- 172.21.15.152:0/2955072931 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fec64005210 0x7fec7c141540 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fec70001050 tx=0x7fec7000ec10 comp rx=0 tx=0).stop 2024-04-20T11:32:37.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.934+0000 7fec83a00640 1 -- 172.21.15.152:0/2955072931 shutdown_connections 2024-04-20T11:32:37.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.934+0000 7fec83a00640 1 --2- 172.21.15.152:0/2955072931 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fec6006ca40 0x7fec6006ef00 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:37.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.934+0000 7fec83a00640 1 --2- 172.21.15.152:0/2955072931 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fec7c13fc70 0x7fec7c143b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:37.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.934+0000 7fec83a00640 1 --2- 172.21.15.152:0/2955072931 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fec7c13df30 0x7fec7c13d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:37.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.934+0000 7fec83a00640 1 --2- 172.21.15.152:0/2955072931 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fec64005210 0x7fec7c141540 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:37.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.934+0000 7fec83a00640 1 -- 172.21.15.152:0/2955072931 >> 172.21.15.152:0/2955072931 conn(0x7fec7c067cf0 msgr2=0x7fec7c15a9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:37.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.935+0000 7fec83a00640 1 -- 172.21.15.152:0/2955072931 shutdown_connections 2024-04-20T11:32:37.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:37.935+0000 7fec83a00640 1 -- 172.21.15.152:0/2955072931 wait complete. 2024-04-20T11:32:37.952 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T11:26:47.246571+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T11:32:37.952 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T11:32:38.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.149+0000 7f3428600640 1 Processor -- start 2024-04-20T11:32:38.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.149+0000 7f3428600640 1 -- start start 2024-04-20T11:32:38.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.150+0000 7f3428600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f342015b510 0x7f342015fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:38.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.150+0000 7f3428600640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f34200b2880 0x7f34200b3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:38.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.150+0000 7f3428600640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f342015f1c0 con 0x7f34200cfc40 2024-04-20T11:32:38.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.150+0000 7f3428600640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f342006f420 con 0x7f34200b2880 2024-04-20T11:32:38.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.150+0000 7f3428600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f34200b4430 con 0x7f342015b510 2024-04-20T11:32:38.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.151+0000 7f3425200640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f34200cfc40 0x7f34200b2170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:48374/0 (socket says 172.21.15.152:48374) 2024-04-20T11:32:38.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.151+0000 7f3425200640 1 -- 172.21.15.152:0/1979003973 learned_addr learned my addr 172.21.15.152:0/1979003973 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:32:38.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.151+0000 7f3426600640 1 --2- 172.21.15.152:0/1979003973 >> v2:172.21.15.152:3300/0 conn(0x7f34200b2880 0x7f34200b3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:38.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.151+0000 7f3425c00640 1 --2- 172.21.15.152:0/1979003973 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f342015b510 0x7f342015fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:38.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.151+0000 7f341fe00640 1 -- 172.21.15.152:0/1979003973 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3915236364 0 0) 0x7f342015f1c0 con 0x7f34200cfc40 2024-04-20T11:32:38.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.151+0000 7f341fe00640 1 -- 172.21.15.152:0/1979003973 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3408003520 con 0x7f34200cfc40 2024-04-20T11:32:38.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.151+0000 7f3426600640 1 -- 172.21.15.152:0/1979003973 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f342015b510 msgr2=0x7f342015fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:38.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.151+0000 7f3426600640 1 --2- 172.21.15.152:0/1979003973 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f342015b510 0x7f342015fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:38.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.151+0000 7f3426600640 1 -- 172.21.15.152:0/1979003973 >> v1:172.21.15.152:6789/0 conn(0x7f34200cfc40 legacy=0x7f34200b2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:38.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.151+0000 7f3426600640 1 -- 172.21.15.152:0/1979003973 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f342006f670 con 0x7f34200b2880 2024-04-20T11:32:38.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.153+0000 7f3426600640 1 --2- 172.21.15.152:0/1979003973 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f34200b2880 0x7f34200b3ef0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f3418009d90 tx=0x7f341802be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a2d131de1d1230 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:38.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.153+0000 7f341fe00640 1 -- 172.21.15.152:0/1979003973 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3418039070 con 0x7f34200b2880 2024-04-20T11:32:38.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.153+0000 7f341fe00640 1 -- 172.21.15.152:0/1979003973 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f341802cbc0 con 0x7f34200b2880 2024-04-20T11:32:38.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.153+0000 7f341fe00640 1 -- 172.21.15.152:0/1979003973 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3418034440 con 0x7f34200b2880 2024-04-20T11:32:38.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.153+0000 7f3428600640 1 -- 172.21.15.152:0/1979003973 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f34200b2880 msgr2=0x7f34200b3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:38.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.153+0000 7f3428600640 1 --2- 172.21.15.152:0/1979003973 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f34200b2880 0x7f34200b3ef0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f3418009d90 tx=0x7f341802be10 comp rx=0 tx=0).stop 2024-04-20T11:32:38.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.153+0000 7f3428600640 1 -- 172.21.15.152:0/1979003973 shutdown_connections 2024-04-20T11:32:38.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.153+0000 7f3428600640 1 --2- 172.21.15.152:0/1979003973 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f34200b2880 0x7f34200b3ef0 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:38.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.153+0000 7f3428600640 1 --2- 172.21.15.152:0/1979003973 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f342015b510 0x7f342015fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:38.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.153+0000 7f3428600640 1 -- 172.21.15.152:0/1979003973 >> 172.21.15.152:0/1979003973 conn(0x7f342006b980 msgr2=0x7f342006bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:38.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.154+0000 7f3428600640 1 -- 172.21.15.152:0/1979003973 shutdown_connections 2024-04-20T11:32:38.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.154+0000 7f3428600640 1 -- 172.21.15.152:0/1979003973 wait complete. 2024-04-20T11:32:38.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.155+0000 7f3428600640 1 Processor -- start 2024-04-20T11:32:38.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.155+0000 7f3428600640 1 -- start start 2024-04-20T11:32:38.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.156+0000 7f3428600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f34200b2880 0x7f3420147e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:38.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.156+0000 7f3428600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f34200cfc40 0x7f34200b1070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:38.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.156+0000 7f3428600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f342015b510 0x7f34200b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:38.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.156+0000 7f3428600640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f342015b8f0 con 0x7f342015b510 2024-04-20T11:32:38.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.156+0000 7f3428600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f342015f710 con 0x7f34200cfc40 2024-04-20T11:32:38.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.156+0000 7f3428600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f342006f950 con 0x7f34200b2880 2024-04-20T11:32:38.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.157+0000 7f3426600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f342015b510 0x7f34200b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:38.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.157+0000 7f3426600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f342015b510 0x7f34200b15b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:42974/0 (socket says 172.21.15.152:42974) 2024-04-20T11:32:38.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.157+0000 7f3426600640 1 -- 172.21.15.152:0/140534184 learned_addr learned my addr 172.21.15.152:0/140534184 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:32:38.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.157+0000 7f3425c00640 1 --2- 172.21.15.152:0/140534184 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f34200b2880 0x7f3420147e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:38.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.157+0000 7f3426600640 1 -- 172.21.15.152:0/140534184 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f34200b2880 msgr2=0x7f3420147e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:38.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.157+0000 7f3426600640 1 --2- 172.21.15.152:0/140534184 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f34200b2880 0x7f3420147e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:38.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.157+0000 7f3425200640 1 --2- 172.21.15.152:0/140534184 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f34200cfc40 0x7f34200b1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:38.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.157+0000 7f3426600640 1 -- 172.21.15.152:0/140534184 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f34200cfc40 msgr2=0x7f34200b1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:38.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.157+0000 7f3426600640 1 --2- 172.21.15.152:0/140534184 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f34200cfc40 0x7f34200b1070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:38.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.157+0000 7f3426600640 1 -- 172.21.15.152:0/140534184 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34201f7ea0 con 0x7f342015b510 2024-04-20T11:32:38.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.157+0000 7f3425200640 1 --2- 172.21.15.152:0/140534184 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f34200cfc40 0x7f34200b1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:32:38.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.158+0000 7f3426600640 1 --2- 172.21.15.152:0/140534184 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f342015b510 0x7f34200b15b0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f3418009eb0 tx=0x7f3418004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:38.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.158+0000 7f341d600640 1 -- 172.21.15.152:0/140534184 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3418039050 con 0x7f342015b510 2024-04-20T11:32:38.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.158+0000 7f341d600640 1 -- 172.21.15.152:0/140534184 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3418045070 con 0x7f342015b510 2024-04-20T11:32:38.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.158+0000 7f341d600640 1 -- 172.21.15.152:0/140534184 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f341803e830 con 0x7f342015b510 2024-04-20T11:32:38.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.158+0000 7f3428600640 1 -- 172.21.15.152:0/140534184 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34201483d0 con 0x7f342015b510 2024-04-20T11:32:38.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.158+0000 7f3428600640 1 -- 172.21.15.152:0/140534184 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34201f8420 con 0x7f342015b510 2024-04-20T11:32:38.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.159+0000 7f3428600640 1 -- 172.21.15.152:0/140534184 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34200689f0 con 0x7f342015b510 2024-04-20T11:32:38.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.160+0000 7f341d600640 1 -- 172.21.15.152:0/140534184 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 91142+0+0 (secure 0 0 0) 0x7f341803e9d0 con 0x7f342015b510 2024-04-20T11:32:38.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.161+0000 7f341d600640 1 --2- 172.21.15.152:0/140534184 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f340006ca10 0x7f340006eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:38.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.161+0000 7f341d600640 1 -- 172.21.15.152:0/140534184 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 4857+0+0 (secure 0 0 0) 0x7f34180ae760 con 0x7f342015b510 2024-04-20T11:32:38.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.162+0000 7f3425c00640 1 --2- 172.21.15.152:0/140534184 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f340006ca10 0x7f340006eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:38.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.163+0000 7f3425c00640 1 --2- 172.21.15.152:0/140534184 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f340006ca10 0x7f340006eed0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3414006fd0 tx=0x7f3414008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:38.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.165+0000 7f341d600640 1 -- 172.21.15.152:0/140534184 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f341807cdc0 con 0x7f342015b510 2024-04-20T11:32:38.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.364+0000 7f3428600640 1 -- 172.21.15.152:0/140534184 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f342015ab20 con 0x7f342015b510 2024-04-20T11:32:38.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.365+0000 7f341d600640 1 -- 172.21.15.152:0/140534184 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) ==== 93+0+10 (secure 0 0 0) 0x7f341807c6c0 con 0x7f342015b510 2024-04-20T11:32:38.365 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 1 2024-04-20T11:32:38.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.368+0000 7f3428600640 1 -- 172.21.15.152:0/140534184 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f340006ca10 msgr2=0x7f340006eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:38.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.368+0000 7f3428600640 1 --2- 172.21.15.152:0/140534184 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f340006ca10 0x7f340006eed0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3414006fd0 tx=0x7f3414008040 comp rx=0 tx=0).stop 2024-04-20T11:32:38.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.368+0000 7f3428600640 1 -- 172.21.15.152:0/140534184 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f342015b510 msgr2=0x7f34200b15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:38.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.368+0000 7f3428600640 1 --2- 172.21.15.152:0/140534184 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f342015b510 0x7f34200b15b0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f3418009eb0 tx=0x7f3418004a40 comp rx=0 tx=0).stop 2024-04-20T11:32:38.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.368+0000 7f3428600640 1 -- 172.21.15.152:0/140534184 shutdown_connections 2024-04-20T11:32:38.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.368+0000 7f3428600640 1 --2- 172.21.15.152:0/140534184 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f340006ca10 0x7f340006eed0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:38.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.368+0000 7f3428600640 1 --2- 172.21.15.152:0/140534184 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f342015b510 0x7f34200b15b0 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:38.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.368+0000 7f3428600640 1 --2- 172.21.15.152:0/140534184 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f34200cfc40 0x7f34200b1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:38.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.368+0000 7f3428600640 1 --2- 172.21.15.152:0/140534184 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f34200b2880 0x7f3420147e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:38.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.368+0000 7f3428600640 1 -- 172.21.15.152:0/140534184 >> 172.21.15.152:0/140534184 conn(0x7f342006b980 msgr2=0x7f34200cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:38.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.368+0000 7f3428600640 1 -- 172.21.15.152:0/140534184 shutdown_connections 2024-04-20T11:32:38.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.368+0000 7f3428600640 1 -- 172.21.15.152:0/140534184 wait complete. 2024-04-20T11:32:38.387 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-20T11:32:38.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.590+0000 7f75af400640 1 Processor -- start 2024-04-20T11:32:38.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.590+0000 7f75af400640 1 -- start start 2024-04-20T11:32:38.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.591+0000 7f75af400640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f75a815eb80 0x7f75a80b79a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:38.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.591+0000 7f75af400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f75a80d1810 0x7f75a80d1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:38.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.591+0000 7f75af400640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75a80b8330 con 0x7f75a80ba520 2024-04-20T11:32:38.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.591+0000 7f75af400640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f75a80d2180 con 0x7f75a80d1810 2024-04-20T11:32:38.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.591+0000 7f75af400640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f75a80b8610 con 0x7f75a815eb80 2024-04-20T11:32:38.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.592+0000 7f75ad400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f75a80d1810 0x7f75a80d1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:38.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.592+0000 7f75ad400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f75a80d1810 0x7f75a80d1c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:42984/0 (socket says 172.21.15.152:42984) 2024-04-20T11:32:38.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.592+0000 7f75ad400640 1 -- 172.21.15.152:0/680710029 learned_addr learned my addr 172.21.15.152:0/680710029 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:32:38.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.592+0000 7f75aca00640 1 --2- 172.21.15.152:0/680710029 >> v2:172.21.15.175:3300/0 conn(0x7f75a815eb80 0x7f75a80b79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:38.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.592+0000 7f75ad400640 1 -- 172.21.15.152:0/680710029 >> v2:172.21.15.175:3300/0 conn(0x7f75a815eb80 msgr2=0x7f75a80b79a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:38.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.592+0000 7f75ad400640 1 --2- 172.21.15.152:0/680710029 >> v2:172.21.15.175:3300/0 conn(0x7f75a815eb80 0x7f75a80b79a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:38.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.592+0000 7f75ad400640 1 -- 172.21.15.152:0/680710029 >> v1:172.21.15.175:6789/0 conn(0x7f75a80ba520 legacy=0x7f75a80d28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:38.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.592+0000 7f75ad400640 1 -- 172.21.15.152:0/680710029 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75a80b88f0 con 0x7f75a80d1810 2024-04-20T11:32:38.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.592+0000 7f75aca00640 1 --2- 172.21.15.152:0/680710029 >> v2:172.21.15.175:3300/0 conn(0x7f75a815eb80 0x7f75a80b79a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:32:38.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.593+0000 7f75ad400640 1 --2- 172.21.15.152:0/680710029 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f75a80d1810 0x7f75a80d1c40 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f75a80b6b00 tx=0x7f75a002be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e81973fa1dff3e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:38.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.594+0000 7f75a7400640 1 -- 172.21.15.152:0/680710029 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a002c9f0 con 0x7f75a80d1810 2024-04-20T11:32:38.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.594+0000 7f75a7400640 1 -- 172.21.15.152:0/680710029 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75a002cb90 con 0x7f75a80d1810 2024-04-20T11:32:38.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.594+0000 7f75a7400640 1 -- 172.21.15.152:0/680710029 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a00343f0 con 0x7f75a80d1810 2024-04-20T11:32:38.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.594+0000 7f75af400640 1 -- 172.21.15.152:0/680710029 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f75a80d1810 msgr2=0x7f75a80d1c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:38.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.594+0000 7f75af400640 1 --2- 172.21.15.152:0/680710029 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f75a80d1810 0x7f75a80d1c40 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f75a80b6b00 tx=0x7f75a002be50 comp rx=0 tx=0).stop 2024-04-20T11:32:38.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.594+0000 7f75af400640 1 -- 172.21.15.152:0/680710029 shutdown_connections 2024-04-20T11:32:38.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.594+0000 7f75af400640 1 --2- 172.21.15.152:0/680710029 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f75a80d1810 0x7f75a80d1c40 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:38.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.594+0000 7f75af400640 1 --2- 172.21.15.152:0/680710029 >> v2:172.21.15.175:3300/0 conn(0x7f75a815eb80 0x7f75a80b79a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:38.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.594+0000 7f75af400640 1 -- 172.21.15.152:0/680710029 >> 172.21.15.152:0/680710029 conn(0x7f75a8067ae0 msgr2=0x7f75a8067ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:38.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.594+0000 7f75af400640 1 -- 172.21.15.152:0/680710029 shutdown_connections 2024-04-20T11:32:38.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.594+0000 7f75af400640 1 -- 172.21.15.152:0/680710029 wait complete. 2024-04-20T11:32:38.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.596+0000 7f75af400640 1 Processor -- start 2024-04-20T11:32:38.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.596+0000 7f75af400640 1 -- start start 2024-04-20T11:32:38.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.597+0000 7f75af400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f75a80ba520 0x7f75a815b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:38.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.597+0000 7f75af400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75a80d1810 0x7f75a8158340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:38.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.597+0000 7f75af400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f75a815eb80 0x7f75a80ce130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:38.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.597+0000 7f75af400640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f75a80b7ee0 con 0x7f75a80ba520 2024-04-20T11:32:38.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.597+0000 7f75af400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f75a80d3000 con 0x7f75a80d1810 2024-04-20T11:32:38.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.597+0000 7f75af400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f75a80b8060 con 0x7f75a815eb80 2024-04-20T11:32:38.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.597+0000 7f75aca00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f75a80ba520 0x7f75a815b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:38.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.598+0000 7f75aca00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f75a80ba520 0x7f75a815b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:42994/0 (socket says 172.21.15.152:42994) 2024-04-20T11:32:38.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.598+0000 7f75aca00640 1 -- 172.21.15.152:0/1062356203 learned_addr learned my addr 172.21.15.152:0/1062356203 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:32:38.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.598+0000 7f75a7e00640 1 --2- 172.21.15.152:0/1062356203 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75a80d1810 0x7f75a8158340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:38.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.598+0000 7f75aca00640 1 -- 172.21.15.152:0/1062356203 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f75a815eb80 msgr2=0x7f75a80ce130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:38.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.598+0000 7f75ad400640 1 --2- 172.21.15.152:0/1062356203 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f75a815eb80 0x7f75a80ce130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:38.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.598+0000 7f75aca00640 1 --2- 172.21.15.152:0/1062356203 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f75a815eb80 0x7f75a80ce130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:38.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.598+0000 7f75aca00640 1 -- 172.21.15.152:0/1062356203 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75a80d1810 msgr2=0x7f75a8158340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:38.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.598+0000 7f75aca00640 1 --2- 172.21.15.152:0/1062356203 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75a80d1810 0x7f75a8158340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:38.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.598+0000 7f75aca00640 1 -- 172.21.15.152:0/1062356203 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75a8158880 con 0x7f75a80ba520 2024-04-20T11:32:38.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.598+0000 7f75aca00640 1 --2- 172.21.15.152:0/1062356203 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f75a80ba520 0x7f75a815b2f0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f759800c820 tx=0x7f759800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:38.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.598+0000 7f75a4c00640 1 -- 172.21.15.152:0/1062356203 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7598015070 con 0x7f75a80ba520 2024-04-20T11:32:38.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.599+0000 7f75a4c00640 1 -- 172.21.15.152:0/1062356203 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7598007a30 con 0x7f75a80ba520 2024-04-20T11:32:38.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.599+0000 7f75a4c00640 1 -- 172.21.15.152:0/1062356203 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7598007de0 con 0x7f75a80ba520 2024-04-20T11:32:38.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.599+0000 7f75af400640 1 -- 172.21.15.152:0/1062356203 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75a8160470 con 0x7f75a80ba520 2024-04-20T11:32:38.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.599+0000 7f75af400640 1 -- 172.21.15.152:0/1062356203 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75a8160a30 con 0x7f75a80ba520 2024-04-20T11:32:38.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.600+0000 7f75af400640 1 -- 172.21.15.152:0/1062356203 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75a80689f0 con 0x7f75a80ba520 2024-04-20T11:32:38.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.601+0000 7f75a4c00640 1 -- 172.21.15.152:0/1062356203 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 91142+0+0 (secure 0 0 0) 0x7f7598018070 con 0x7f75a80ba520 2024-04-20T11:32:38.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.602+0000 7f75a4c00640 1 --2- 172.21.15.152:0/1062356203 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f758806ca10 0x7f758806eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:38.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.602+0000 7f75a4c00640 1 -- 172.21.15.152:0/1062356203 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 4857+0+0 (secure 0 0 0) 0x7f759808b940 con 0x7f75a80ba520 2024-04-20T11:32:38.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.603+0000 7f75a7e00640 1 --2- 172.21.15.152:0/1062356203 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f758806ca10 0x7f758806eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:38.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.604+0000 7f75a7e00640 1 --2- 172.21.15.152:0/1062356203 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f758806ca10 0x7f758806eed0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f75a8147e70 tx=0x7f7594008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:38.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.606+0000 7f75a4c00640 1 -- 172.21.15.152:0/1062356203 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f75a8158880 con 0x7f75a80ba520 2024-04-20T11:32:38.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.807+0000 7f75af400640 1 -- 172.21.15.152:0/1062356203 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f75a80d5660 con 0x7f75a80ba520 2024-04-20T11:32:38.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.808+0000 7f75a4c00640 1 -- 172.21.15.152:0/1062356203 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v79) ==== 84+0+65 (secure 0 0 0) 0x7f7598059f30 con 0x7f75a80ba520 2024-04-20T11:32:38.808 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:32:38.808 INFO:teuthology.orchestra.run.smithi152.stdout:[ 2024-04-20T11:32:38.808 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:32:38.808 INFO:teuthology.orchestra.run.smithi152.stdout: "poolnum": 1, 2024-04-20T11:32:38.808 INFO:teuthology.orchestra.run.smithi152.stdout: "poolname": ".mgr" 2024-04-20T11:32:38.808 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:38.808 INFO:teuthology.orchestra.run.smithi152.stdout:] 2024-04-20T11:32:38.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.811+0000 7f75af400640 1 -- 172.21.15.152:0/1062356203 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f758806ca10 msgr2=0x7f758806eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:38.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.811+0000 7f75af400640 1 --2- 172.21.15.152:0/1062356203 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f758806ca10 0x7f758806eed0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f75a8147e70 tx=0x7f7594008040 comp rx=0 tx=0).stop 2024-04-20T11:32:38.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.811+0000 7f75af400640 1 -- 172.21.15.152:0/1062356203 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f75a80ba520 msgr2=0x7f75a815b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:38.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.811+0000 7f75af400640 1 --2- 172.21.15.152:0/1062356203 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f75a80ba520 0x7f75a815b2f0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f759800c820 tx=0x7f759800cce0 comp rx=0 tx=0).stop 2024-04-20T11:32:38.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.811+0000 7f75af400640 1 -- 172.21.15.152:0/1062356203 shutdown_connections 2024-04-20T11:32:38.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.811+0000 7f75af400640 1 --2- 172.21.15.152:0/1062356203 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f758806ca10 0x7f758806eed0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:38.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.811+0000 7f75af400640 1 --2- 172.21.15.152:0/1062356203 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f75a815eb80 0x7f75a80ce130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:38.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.811+0000 7f75af400640 1 --2- 172.21.15.152:0/1062356203 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75a80d1810 0x7f75a8158340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:38.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.811+0000 7f75af400640 1 --2- 172.21.15.152:0/1062356203 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f75a80ba520 0x7f75a815b2f0 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:38.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.811+0000 7f75af400640 1 -- 172.21.15.152:0/1062356203 >> 172.21.15.152:0/1062356203 conn(0x7f75a8067ae0 msgr2=0x7f75a80c83a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:38.812 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.811+0000 7f75af400640 1 -- 172.21.15.152:0/1062356203 shutdown_connections 2024-04-20T11:32:38.812 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:38.811+0000 7f75af400640 1 -- 172.21.15.152:0/1062356203 wait complete. 2024-04-20T11:32:38.830 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-20T11:32:38.830 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-20T11:32:39.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:39.037+0000 7f4d93e00640 1 Processor -- start 2024-04-20T11:32:39.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:39.037+0000 7f4d93e00640 1 -- start start 2024-04-20T11:32:39.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:39.039+0000 7f4d93e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f4d940ae180 0x7f4d940ae600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:39.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:39.039+0000 7f4d93e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4d94071f20 0x7f4d940723b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:39.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:39.039+0000 7f4d93e00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d940aeef0 con 0x7f4d940d05f0 2024-04-20T11:32:39.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:39.039+0000 7f4d93e00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f4d940af1d0 con 0x7f4d940ae180 2024-04-20T11:32:39.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:39.039+0000 7f4d93e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4d940af4b0 con 0x7f4d94071f20 2024-04-20T11:32:39.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:39.039+0000 7f4d92a00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f4d940d05f0 0x7f4d940d09d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:48390/0 (socket says 172.21.15.152:48390) 2024-04-20T11:32:39.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:39.039+0000 7f4d92a00640 1 -- 172.21.15.152:0/1643922626 learned_addr learned my addr 172.21.15.152:0/1643922626 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:32:39.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:39.039+0000 7f4d91600640 1 -- 172.21.15.152:0/1643922626 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4067910014 0 0) 0x7f4d940aeef0 con 0x7f4d940d05f0 2024-04-20T11:32:39.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:39.039+0000 7f4d92000640 1 --2- 172.21.15.152:0/1643922626 >> v2:172.21.15.175:3300/0 conn(0x7f4d940ae180 0x7f4d940ae600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:39.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:39.039+0000 7f4d93400640 1 --2- 172.21.15.152:0/1643922626 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4d94071f20 0x7f4d940723b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:39.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:39.039+0000 7f4d91600640 1 -- 172.21.15.152:0/1643922626 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4d78003520 con 0x7f4d940d05f0 2024-04-20T11:32:39.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:39.040+0000 7f4d91600640 1 -- 172.21.15.152:0/1643922626 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1374405968 0 0) 0x7f4d78003520 con 0x7f4d940d05f0 2024-04-20T11:32:39.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:39.040+0000 7f4d91600640 1 -- 172.21.15.152:0/1643922626 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4d94071f20 msgr2=0x7f4d940723b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:39.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:39.040+0000 7f4d91600640 1 --2- 172.21.15.152:0/1643922626 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4d94071f20 0x7f4d940723b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:39.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:39.040+0000 7f4d91600640 1 -- 172.21.15.152:0/1643922626 >> v2:172.21.15.175:3300/0 conn(0x7f4d940ae180 msgr2=0x7f4d940ae600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:39.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:39.040+0000 7f4d91600640 1 --2- 172.21.15.152:0/1643922626 >> v2:172.21.15.175:3300/0 conn(0x7f4d940ae180 0x7f4d940ae600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:39.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:39.040+0000 7f4d91600640 1 -- 172.21.15.152:0/1643922626 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d940af790 con 0x7f4d940d05f0 2024-04-20T11:32:39.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:39.040+0000 7f4d91600640 1 -- 172.21.15.152:0/1643922626 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f4d84001670 con 0x7f4d940d05f0 2024-04-20T11:32:39.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:39.040+0000 7f4d93400640 1 --2- 172.21.15.152:0/1643922626 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4d94071f20 0x7f4d940723b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:32:39.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:39.040+0000 7f4d91600640 1 -- 172.21.15.152:0/1643922626 >> v1:172.21.15.152:6789/0 conn(0x7f4d940d05f0 legacy=0x7f4d940d09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:39.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:39.040+0000 7f4d91600640 1 --2- 172.21.15.152:0/1643922626 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d780039b0 0x7f4d78003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:39.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:39.040+0000 7f4d91600640 1 --2- 172.21.15.152:0/1643922626 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4d94071f20 0x7f4d78004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:39.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:39.040+0000 7f4d91600640 1 --2- 172.21.15.152:0/1643922626 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d78004d40 0x7f4d780051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:39.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:39.040+0000 7f4d91600640 1 -- 172.21.15.152:0/1643922626 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f4d940af1d0 con 0x7f4d78004d40 2024-04-20T11:32:39.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:39.040+0000 7f4d91600640 1 -- 172.21.15.152:0/1643922626 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4d940af4b0 con 0x7f4d780039b0 2024-04-20T11:32:39.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:39.040+0000 7f4d91600640 1 -- 172.21.15.152:0/1643922626 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4d7800dbd0 con 0x7f4d94071f20 2024-04-20T11:32:39.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:39.040+0000 7f4d93400640 1 --2- 172.21.15.152:0/1643922626 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d78004d40 0x7f4d780051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:39.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:39.041+0000 7f4d92000640 1 --2- 172.21.15.152:0/1643922626 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4d94071f20 0x7f4d78004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:39.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:39.041+0000 7f4d92a00640 1 --2- 172.21.15.152:0/1643922626 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d780039b0 0x7f4d78003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:39.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:39.041+0000 7f4d93400640 1 -- 172.21.15.152:0/1643922626 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4d94071f20 msgr2=0x7f4d78004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:39.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:39.041+0000 7f4d93400640 1 --2- 172.21.15.152:0/1643922626 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4d94071f20 0x7f4d78004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:39.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:39.041+0000 7f4d93400640 1 -- 172.21.15.152:0/1643922626 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d780039b0 msgr2=0x7f4d78003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:39.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:39.041+0000 7f4d93400640 1 --2- 172.21.15.152:0/1643922626 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d780039b0 0x7f4d78003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:39.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:39.041+0000 7f4d93400640 1 -- 172.21.15.152:0/1643922626 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d78003520 con 0x7f4d78004d40 2024-04-20T11:32:39.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:39.041+0000 7f4d92a00640 1 --2- 172.21.15.152:0/1643922626 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d780039b0 0x7f4d78003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:32:39.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:39.042+0000 7f4d93400640 1 --2- 172.21.15.152:0/1643922626 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d78004d40 0x7f4d780051c0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c0073e0 tx=0x7f4d78005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ac0f164974ce3b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:39.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:39.042+0000 7f4d91600640 1 -- 172.21.15.152:0/1643922626 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d8c039070 con 0x7f4d78004d40 2024-04-20T11:32:39.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:39.042+0000 7f4d91600640 1 -- 172.21.15.152:0/1643922626 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d8c02cd30 con 0x7f4d78004d40 2024-04-20T11:32:39.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:39.042+0000 7f4d91600640 1 -- 172.21.15.152:0/1643922626 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d8c009c80 con 0x7f4d78004d40 2024-04-20T11:32:39.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:39.043+0000 7f4d93e00640 1 -- 172.21.15.152:0/1643922626 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d78004d40 msgr2=0x7f4d780051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:39.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:39.043+0000 7f4d93e00640 1 --2- 172.21.15.152:0/1643922626 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d78004d40 0x7f4d780051c0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c0073e0 tx=0x7f4d78005830 comp rx=0 tx=0).stop 2024-04-20T11:32:39.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:39.043+0000 7f4d93400640 1 -- 172.21.15.152:0/1643922626 reap_dead start 2024-04-20T11:32:39.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:39.043+0000 7f4d93e00640 1 -- 172.21.15.152:0/1643922626 shutdown_connections 2024-04-20T11:32:39.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:39.043+0000 7f4d93e00640 1 -- 172.21.15.152:0/1643922626 >> 172.21.15.152:0/1643922626 conn(0x7f4d940b21e0 msgr2=0x7f4d940b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:39.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:39.043+0000 7f4d93e00640 1 -- 172.21.15.152:0/1643922626 shutdown_connections 2024-04-20T11:32:39.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:39.043+0000 7f4d93e00640 1 -- 172.21.15.152:0/1643922626 wait complete. 2024-04-20T11:32:39.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:39.045+0000 7f4d93e00640 1 Processor -- start 2024-04-20T11:32:39.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:39.045+0000 7f4d93e00640 1 -- start start 2024-04-20T11:32:39.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:39.046+0000 7f4d93e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4d78004d40 0x7f4d94157c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:39.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:39.046+0000 7f4d93e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d940cf4b0 0x7f4d94154ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:39.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:39.046+0000 7f4d93e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d94158b80 0x7f4d9415ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:39.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:39.046+0000 7f4d93e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f4d94155ca0 con 0x7f4d94158b80 2024-04-20T11:32:39.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:39.046+0000 7f4d93e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4d940aeb40 con 0x7f4d940cf4b0 2024-04-20T11:32:39.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:39.046+0000 7f4d93e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4d941553e0 con 0x7f4d78004d40 2024-04-20T11:32:39.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:39.046+0000 7f4d93400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d94158b80 0x7f4d9415ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:39.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:39.046+0000 7f4d93400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d94158b80 0x7f4d9415ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43022/0 (socket says 172.21.15.152:43022) 2024-04-20T11:32:39.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:39.046+0000 7f4d93400640 1 -- 172.21.15.152:0/1093046486 learned_addr learned my addr 172.21.15.152:0/1093046486 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:32:39.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:39.046+0000 7f4d92a00640 1 --2- 172.21.15.152:0/1093046486 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4d78004d40 0x7f4d94157c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:39.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:39.047+0000 7f4d92000640 1 --2- 172.21.15.152:0/1093046486 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d940cf4b0 0x7f4d94154ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:39.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:39.047+0000 7f4d93400640 1 -- 172.21.15.152:0/1093046486 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4d78004d40 msgr2=0x7f4d94157c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:39.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:39.047+0000 7f4d93400640 1 --2- 172.21.15.152:0/1093046486 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4d78004d40 0x7f4d94157c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:39.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:39.047+0000 7f4d93400640 1 -- 172.21.15.152:0/1093046486 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d940cf4b0 msgr2=0x7f4d94154ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:39.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:39.047+0000 7f4d93400640 1 --2- 172.21.15.152:0/1093046486 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d940cf4b0 0x7f4d94154ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:39.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:39.047+0000 7f4d93400640 1 -- 172.21.15.152:0/1093046486 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d94159460 con 0x7f4d94158b80 2024-04-20T11:32:39.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:39.047+0000 7f4d93400640 1 --2- 172.21.15.152:0/1093046486 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d94158b80 0x7f4d9415ac00 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c037040 tx=0x7f4d8c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:39.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:39.047+0000 7f4d8aa00640 1 -- 172.21.15.152:0/1093046486 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d8c039030 con 0x7f4d94158b80 2024-04-20T11:32:39.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:39.047+0000 7f4d8aa00640 1 -- 172.21.15.152:0/1093046486 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d8c03a040 con 0x7f4d94158b80 2024-04-20T11:32:39.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:39.047+0000 7f4d93e00640 1 -- 172.21.15.152:0/1093046486 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d94157160 con 0x7f4d94158b80 2024-04-20T11:32:39.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:39.047+0000 7f4d8aa00640 1 -- 172.21.15.152:0/1093046486 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d8c009c80 con 0x7f4d94158b80 2024-04-20T11:32:39.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:39.048+0000 7f4d93e00640 1 -- 172.21.15.152:0/1093046486 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d94159000 con 0x7f4d94158b80 2024-04-20T11:32:39.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:39.049+0000 7f4d93e00640 1 -- 172.21.15.152:0/1093046486 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d940b1980 con 0x7f4d94158b80 2024-04-20T11:32:39.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:39.050+0000 7f4d8aa00640 1 -- 172.21.15.152:0/1093046486 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 91142+0+0 (secure 0 0 0) 0x7f4d8c045020 con 0x7f4d94158b80 2024-04-20T11:32:39.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:39.050+0000 7f4d8aa00640 1 --2- 172.21.15.152:0/1093046486 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4d7406ca10 0x7f4d7406eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:39.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:39.051+0000 7f4d8aa00640 1 -- 172.21.15.152:0/1093046486 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 4857+0+0 (secure 0 0 0) 0x7f4d8c0ad240 con 0x7f4d94158b80 2024-04-20T11:32:39.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:39.051+0000 7f4d92a00640 1 --2- 172.21.15.152:0/1093046486 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4d7406ca10 0x7f4d7406eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:39.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:39.052+0000 7f4d92a00640 1 --2- 172.21.15.152:0/1093046486 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4d7406ca10 0x7f4d7406eed0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4d84000c00 tx=0x7f4d84009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:39.054 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:39.055+0000 7f4d8aa00640 1 -- 172.21.15.152:0/1093046486 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f4d940b1980 con 0x7f4d94158b80 2024-04-20T11:32:39.254 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:39.254+0000 7f4d93e00640 1 -- 172.21.15.152:0/1093046486 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f4d94075e60 con 0x7f4d94158b80 2024-04-20T11:32:40.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:40.029+0000 7f4d8aa00640 1 -- 172.21.15.152:0/1093046486 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v80) ==== 138+0+0 (secure 0 0 0) 0x7f4d8c07b8a0 con 0x7f4d94158b80 2024-04-20T11:32:40.029 INFO:teuthology.orchestra.run.smithi152.stderr:pool 'cephfs_metadata' created 2024-04-20T11:32:40.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:40.032+0000 7f4d93e00640 1 -- 172.21.15.152:0/1093046486 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4d7406ca10 msgr2=0x7f4d7406eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:40.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:40.032+0000 7f4d93e00640 1 --2- 172.21.15.152:0/1093046486 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4d7406ca10 0x7f4d7406eed0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4d84000c00 tx=0x7f4d84009040 comp rx=0 tx=0).stop 2024-04-20T11:32:40.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:40.032+0000 7f4d93e00640 1 -- 172.21.15.152:0/1093046486 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d94158b80 msgr2=0x7f4d9415ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:40.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:40.032+0000 7f4d93e00640 1 --2- 172.21.15.152:0/1093046486 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d94158b80 0x7f4d9415ac00 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c037040 tx=0x7f4d8c004ab0 comp rx=0 tx=0).stop 2024-04-20T11:32:40.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:40.033+0000 7f4d93e00640 1 -- 172.21.15.152:0/1093046486 shutdown_connections 2024-04-20T11:32:40.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:40.033+0000 7f4d93e00640 1 --2- 172.21.15.152:0/1093046486 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4d7406ca10 0x7f4d7406eed0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:40.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:40.033+0000 7f4d93e00640 1 --2- 172.21.15.152:0/1093046486 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d94158b80 0x7f4d9415ac00 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:40.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:40.033+0000 7f4d93e00640 1 --2- 172.21.15.152:0/1093046486 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d940cf4b0 0x7f4d94154ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:40.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:40.033+0000 7f4d93e00640 1 --2- 172.21.15.152:0/1093046486 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4d78004d40 0x7f4d94157c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:40.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:40.033+0000 7f4d93e00640 1 -- 172.21.15.152:0/1093046486 >> 172.21.15.152:0/1093046486 conn(0x7f4d940b21e0 msgr2=0x7f4d9406dc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:40.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:40.033+0000 7f4d93e00640 1 -- 172.21.15.152:0/1093046486 shutdown_connections 2024-04-20T11:32:40.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:40.033+0000 7f4d93e00640 1 -- 172.21.15.152:0/1093046486 wait complete. 2024-04-20T11:32:40.061 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-20T11:32:40.262 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:40.262+0000 7fef2ca00640 1 Processor -- start 2024-04-20T11:32:40.262 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:40.263+0000 7fef2ca00640 1 -- start start 2024-04-20T11:32:40.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:40.264+0000 7fef2ca00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fef28161c20 0x7fef280d5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:40.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:40.264+0000 7fef2ca00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef280cfcb0 0x7fef280d2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:40.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:40.264+0000 7fef2ca00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fef280d33d0 0x7fef280d0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:40.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:40.264+0000 7fef2ca00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fef280d2160 con 0x7fef28161c20 2024-04-20T11:32:40.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:40.264+0000 7fef2ca00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fef280d2a00 con 0x7fef280d33d0 2024-04-20T11:32:40.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:40.264+0000 7fef2ca00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fef280d0400 con 0x7fef280cfcb0 2024-04-20T11:32:40.265 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:40.264+0000 7fef27400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fef28161c20 0x7fef280d5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:40.265 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:40.264+0000 7fef27400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fef28161c20 0x7fef280d5060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43030/0 (socket says 172.21.15.152:43030) 2024-04-20T11:32:40.265 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:40.264+0000 7fef27400640 1 -- 172.21.15.152:0/83814063 learned_addr learned my addr 172.21.15.152:0/83814063 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:32:40.265 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:40.264+0000 7fef26a00640 1 --2- 172.21.15.152:0/83814063 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef280cfcb0 0x7fef280d2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:40.265 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:40.265+0000 7fef27e00640 1 --2- 172.21.15.152:0/83814063 >> v2:172.21.15.175:3300/0 conn(0x7fef280d33d0 0x7fef280d0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:40.265 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:40.265+0000 7fef27400640 1 -- 172.21.15.152:0/83814063 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef280cfcb0 msgr2=0x7fef280d2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:40.265 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:40.265+0000 7fef27400640 1 --2- 172.21.15.152:0/83814063 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef280cfcb0 0x7fef280d2e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:40.265 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:40.265+0000 7fef27400640 1 -- 172.21.15.152:0/83814063 >> v2:172.21.15.175:3300/0 conn(0x7fef280d33d0 msgr2=0x7fef280d0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:40.265 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:40.265+0000 7fef27400640 1 --2- 172.21.15.152:0/83814063 >> v2:172.21.15.175:3300/0 conn(0x7fef280d33d0 0x7fef280d0f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:40.265 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:40.265+0000 7fef27400640 1 -- 172.21.15.152:0/83814063 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef280d23b0 con 0x7fef28161c20 2024-04-20T11:32:40.266 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:40.266+0000 7fef27400640 1 --2- 172.21.15.152:0/83814063 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef28161c20 0x7fef280d5060 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7fef2808e280 tx=0x7fef1802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da31d1d090c7c812 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:40.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:40.266+0000 7fef26000640 1 -- 172.21.15.152:0/83814063 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef1802c9e0 con 0x7fef28161c20 2024-04-20T11:32:40.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:40.266+0000 7fef26000640 1 -- 172.21.15.152:0/83814063 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef1802cb80 con 0x7fef28161c20 2024-04-20T11:32:40.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:40.266+0000 7fef26000640 1 -- 172.21.15.152:0/83814063 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef180343f0 con 0x7fef28161c20 2024-04-20T11:32:40.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:40.267+0000 7fef2ca00640 1 -- 172.21.15.152:0/83814063 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef28161c20 msgr2=0x7fef280d5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:40.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:40.267+0000 7fef2ca00640 1 --2- 172.21.15.152:0/83814063 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef28161c20 0x7fef280d5060 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7fef2808e280 tx=0x7fef1802be40 comp rx=0 tx=0).stop 2024-04-20T11:32:40.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:40.267+0000 7fef2ca00640 1 -- 172.21.15.152:0/83814063 shutdown_connections 2024-04-20T11:32:40.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:40.267+0000 7fef2ca00640 1 --2- 172.21.15.152:0/83814063 >> v2:172.21.15.175:3300/0 conn(0x7fef280d33d0 0x7fef280d0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:40.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:40.267+0000 7fef2ca00640 1 --2- 172.21.15.152:0/83814063 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef280cfcb0 0x7fef280d2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:40.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:40.267+0000 7fef2ca00640 1 --2- 172.21.15.152:0/83814063 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef28161c20 0x7fef280d5060 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:40.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:40.267+0000 7fef2ca00640 1 -- 172.21.15.152:0/83814063 >> 172.21.15.152:0/83814063 conn(0x7fef2808bfd0 msgr2=0x7fef280d4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:40.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:40.267+0000 7fef2ca00640 1 -- 172.21.15.152:0/83814063 shutdown_connections 2024-04-20T11:32:40.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:40.267+0000 7fef2ca00640 1 -- 172.21.15.152:0/83814063 wait complete. 2024-04-20T11:32:40.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:40.269+0000 7fef2ca00640 1 Processor -- start 2024-04-20T11:32:40.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:40.269+0000 7fef2ca00640 1 -- start start 2024-04-20T11:32:40.271 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:40.270+0000 7fef2ca00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef280cfcb0 0x7fef280a1e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:40.271 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:40.270+0000 7fef2ca00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef280d33d0 0x7fef280cc070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:40.271 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:40.270+0000 7fef2ca00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef28161c20 0x7fef2815da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:40.271 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:40.270+0000 7fef2ca00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fef28162000 con 0x7fef280d33d0 2024-04-20T11:32:40.271 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:40.270+0000 7fef2ca00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fef280d00f0 con 0x7fef28161c20 2024-04-20T11:32:40.271 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:40.270+0000 7fef2ca00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fef28162180 con 0x7fef280cfcb0 2024-04-20T11:32:40.271 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:40.270+0000 7fef26a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef280d33d0 0x7fef280cc070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:40.271 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:40.270+0000 7fef26a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef280d33d0 0x7fef280cc070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43046/0 (socket says 172.21.15.152:43046) 2024-04-20T11:32:40.271 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:40.270+0000 7fef26a00640 1 -- 172.21.15.152:0/2790790737 learned_addr learned my addr 172.21.15.152:0/2790790737 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:32:40.271 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:40.271+0000 7fef27400640 1 --2- 172.21.15.152:0/2790790737 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef280cfcb0 0x7fef280a1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:40.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:40.271+0000 7fef27e00640 1 --2- 172.21.15.152:0/2790790737 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef28161c20 0x7fef2815da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:40.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:40.271+0000 7fef26a00640 1 -- 172.21.15.152:0/2790790737 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef280cfcb0 msgr2=0x7fef280a1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:40.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:40.271+0000 7fef26a00640 1 --2- 172.21.15.152:0/2790790737 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef280cfcb0 0x7fef280a1e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:40.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:40.271+0000 7fef26a00640 1 -- 172.21.15.152:0/2790790737 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef28161c20 msgr2=0x7fef2815da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:40.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:40.271+0000 7fef26a00640 1 --2- 172.21.15.152:0/2790790737 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef28161c20 0x7fef2815da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:40.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:40.271+0000 7fef26a00640 1 -- 172.21.15.152:0/2790790737 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef280a0d90 con 0x7fef280d33d0 2024-04-20T11:32:40.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:40.271+0000 7fef26a00640 1 --2- 172.21.15.152:0/2790790737 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef280d33d0 0x7fef280cc070 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fef1000d830 tx=0x7fef1000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:40.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:40.271+0000 7fef1f400640 1 -- 172.21.15.152:0/2790790737 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef10018030 con 0x7fef280d33d0 2024-04-20T11:32:40.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:40.271+0000 7fef1f400640 1 -- 172.21.15.152:0/2790790737 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef1000be10 con 0x7fef280d33d0 2024-04-20T11:32:40.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:40.271+0000 7fef2ca00640 1 -- 172.21.15.152:0/2790790737 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef280a1070 con 0x7fef280d33d0 2024-04-20T11:32:40.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:40.271+0000 7fef1f400640 1 -- 172.21.15.152:0/2790790737 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef100098b0 con 0x7fef280d33d0 2024-04-20T11:32:40.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:40.271+0000 7fef2ca00640 1 -- 172.21.15.152:0/2790790737 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef280a1630 con 0x7fef280d33d0 2024-04-20T11:32:40.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:40.273+0000 7fef2ca00640 1 -- 172.21.15.152:0/2790790737 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef28075e60 con 0x7fef280d33d0 2024-04-20T11:32:40.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:40.274+0000 7fef1f400640 1 -- 172.21.15.152:0/2790790737 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 91142+0+0 (secure 0 0 0) 0x7fef1001b070 con 0x7fef280d33d0 2024-04-20T11:32:40.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:40.275+0000 7fef1f400640 1 --2- 172.21.15.152:0/2790790737 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fef0406ca10 0x7fef0406eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:40.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:40.275+0000 7fef1f400640 1 -- 172.21.15.152:0/2790790737 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5227+0+0 (secure 0 0 0) 0x7fef1008bcf0 con 0x7fef280d33d0 2024-04-20T11:32:40.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:40.276+0000 7fef27400640 1 --2- 172.21.15.152:0/2790790737 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fef0406ca10 0x7fef0406eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:40.276 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:40.277+0000 7fef27400640 1 --2- 172.21.15.152:0/2790790737 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fef0406ca10 0x7fef0406eed0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fef18035000 tx=0x7fef18030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:40.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:40.280+0000 7fef1f400640 1 -- 172.21.15.152:0/2790790737 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fef10090070 con 0x7fef280d33d0 2024-04-20T11:32:40.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:40.484+0000 7fef2ca00640 1 -- 172.21.15.152:0/2790790737 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fef280cf910 con 0x7fef280d33d0 2024-04-20T11:32:41.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:41.035+0000 7fef1f400640 1 -- 172.21.15.152:0/2790790737 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v81) ==== 170+0+0 (secure 0 0 0) 0x7fef1005a1f0 con 0x7fef280d33d0 2024-04-20T11:32:41.035 INFO:teuthology.orchestra.run.smithi152.stderr:pool 'cephfs_data' created 2024-04-20T11:32:41.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:41.039+0000 7fef2ca00640 1 -- 172.21.15.152:0/2790790737 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fef0406ca10 msgr2=0x7fef0406eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:41.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:41.039+0000 7fef2ca00640 1 --2- 172.21.15.152:0/2790790737 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fef0406ca10 0x7fef0406eed0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fef18035000 tx=0x7fef18030000 comp rx=0 tx=0).stop 2024-04-20T11:32:41.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:41.039+0000 7fef2ca00640 1 -- 172.21.15.152:0/2790790737 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef280d33d0 msgr2=0x7fef280cc070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:41.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:41.039+0000 7fef2ca00640 1 --2- 172.21.15.152:0/2790790737 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef280d33d0 0x7fef280cc070 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fef1000d830 tx=0x7fef1000dcf0 comp rx=0 tx=0).stop 2024-04-20T11:32:41.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:41.039+0000 7fef2ca00640 1 -- 172.21.15.152:0/2790790737 shutdown_connections 2024-04-20T11:32:41.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:41.039+0000 7fef2ca00640 1 --2- 172.21.15.152:0/2790790737 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fef0406ca10 0x7fef0406eed0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:41.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:41.039+0000 7fef2ca00640 1 --2- 172.21.15.152:0/2790790737 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef28161c20 0x7fef2815da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:41.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:41.039+0000 7fef2ca00640 1 --2- 172.21.15.152:0/2790790737 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef280d33d0 0x7fef280cc070 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:41.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:41.039+0000 7fef2ca00640 1 --2- 172.21.15.152:0/2790790737 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef280cfcb0 0x7fef280a1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:41.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:41.039+0000 7fef2ca00640 1 -- 172.21.15.152:0/2790790737 >> 172.21.15.152:0/2790790737 conn(0x7fef2808bfd0 msgr2=0x7fef280b8f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:41.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:41.040+0000 7fef2ca00640 1 -- 172.21.15.152:0/2790790737 shutdown_connections 2024-04-20T11:32:41.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:41.040+0000 7fef2ca00640 1 -- 172.21.15.152:0/2790790737 wait complete. 2024-04-20T11:32:41.075 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-20T11:32:41.253 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:41.254+0000 7fa321800640 1 Processor -- start 2024-04-20T11:32:41.253 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:41.254+0000 7fa321800640 1 -- start start 2024-04-20T11:32:41.255 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:41.255+0000 7fa321800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa31c071cd0 0x7fa31c0720b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:41.255 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:41.255+0000 7fa321800640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa31c06fb40 con 0x7fa31c160220 2024-04-20T11:32:41.255 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:41.255+0000 7fa321800640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa31c0725f0 con 0x7fa31c141450 2024-04-20T11:32:41.255 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:41.255+0000 7fa321800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa31c13f8e0 con 0x7fa31c071cd0 2024-04-20T11:32:41.257 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:41.256+0000 7fa313400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fa31c160220 0x7fa31c160680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:48406/0 (socket says 172.21.15.152:48406) 2024-04-20T11:32:41.257 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:41.256+0000 7fa313400640 1 -- 172.21.15.152:0/1864242197 learned_addr learned my addr 172.21.15.152:0/1864242197 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:32:41.257 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:41.256+0000 7fa31be00640 1 --2- 172.21.15.152:0/1864242197 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa31c071cd0 0x7fa31c0720b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:41.257 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:41.256+0000 7fa31b400640 1 -- 172.21.15.152:0/1864242197 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4155085068 0 0) 0x7fa31c06fb40 con 0x7fa31c160220 2024-04-20T11:32:41.257 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:41.256+0000 7fa31b400640 1 -- 172.21.15.152:0/1864242197 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa300003520 con 0x7fa31c160220 2024-04-20T11:32:41.257 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:41.257+0000 7fa31b400640 1 -- 172.21.15.152:0/1864242197 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2110437214 0 0) 0x7fa300003520 con 0x7fa31c160220 2024-04-20T11:32:41.257 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:41.257+0000 7fa31b400640 1 -- 172.21.15.152:0/1864242197 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa31c071cd0 msgr2=0x7fa31c0720b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:41.257 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:41.257+0000 7fa31b400640 1 --2- 172.21.15.152:0/1864242197 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa31c071cd0 0x7fa31c0720b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:41.257 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:41.257+0000 7fa31b400640 1 -- 172.21.15.152:0/1864242197 >> v1:172.21.15.175:6789/0 conn(0x7fa31c141450 legacy=0x7fa31c1418e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:41.257 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:41.257+0000 7fa31b400640 1 -- 172.21.15.152:0/1864242197 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa31c160f90 con 0x7fa31c160220 2024-04-20T11:32:41.257 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:41.257+0000 7fa31b400640 1 -- 172.21.15.152:0/1864242197 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7fa304001670 con 0x7fa31c160220 2024-04-20T11:32:41.257 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:41.257+0000 7fa31b400640 1 -- 172.21.15.152:0/1864242197 >> v1:172.21.15.152:6789/0 conn(0x7fa31c160220 legacy=0x7fa31c160680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:41.257 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:41.257+0000 7fa31b400640 1 --2- 172.21.15.152:0/1864242197 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa3000039e0 0x7fa300003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:41.258 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:41.257+0000 7fa31b400640 1 --2- 172.21.15.152:0/1864242197 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa31c071cd0 0x7fa300004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:41.258 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:41.257+0000 7fa31b400640 1 --2- 172.21.15.152:0/1864242197 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa300004d40 0x7fa3000051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:41.258 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:41.257+0000 7fa31b400640 1 -- 172.21.15.152:0/1864242197 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fa31c13f8e0 con 0x7fa3000039e0 2024-04-20T11:32:41.258 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:41.257+0000 7fa31b400640 1 -- 172.21.15.152:0/1864242197 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa30000baf0 con 0x7fa300004d40 2024-04-20T11:32:41.258 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:41.257+0000 7fa31b400640 1 -- 172.21.15.152:0/1864242197 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa30000bcf0 con 0x7fa31c071cd0 2024-04-20T11:32:41.258 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:41.257+0000 7fa31be00640 1 --2- 172.21.15.152:0/1864242197 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa3000039e0 0x7fa300003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:41.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:41.258+0000 7fa313400640 1 --2- 172.21.15.152:0/1864242197 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa31c071cd0 0x7fa300004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:41.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:41.258+0000 7fa320e00640 1 --2- 172.21.15.152:0/1864242197 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa300004d40 0x7fa3000051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:41.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:41.258+0000 7fa31be00640 1 -- 172.21.15.152:0/1864242197 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa31c071cd0 msgr2=0x7fa300004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:41.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:41.258+0000 7fa31be00640 1 --2- 172.21.15.152:0/1864242197 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa31c071cd0 0x7fa300004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:41.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:41.258+0000 7fa31be00640 1 -- 172.21.15.152:0/1864242197 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa300004d40 msgr2=0x7fa3000051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:41.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:41.258+0000 7fa31be00640 1 --2- 172.21.15.152:0/1864242197 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa300004d40 0x7fa3000051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:41.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:41.258+0000 7fa31be00640 1 -- 172.21.15.152:0/1864242197 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa300003520 con 0x7fa3000039e0 2024-04-20T11:32:41.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:41.258+0000 7fa320e00640 1 --2- 172.21.15.152:0/1864242197 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa300004d40 0x7fa3000051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:32:41.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:41.259+0000 7fa31be00640 1 --2- 172.21.15.152:0/1864242197 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa3000039e0 0x7fa300003e00 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fa30c009ac0 tx=0x7fa300005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=872979464b0e366b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:41.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:41.259+0000 7fa31b400640 1 -- 172.21.15.152:0/1864242197 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa30c039070 con 0x7fa3000039e0 2024-04-20T11:32:41.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:41.259+0000 7fa31b400640 1 -- 172.21.15.152:0/1864242197 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa30c02c600 con 0x7fa3000039e0 2024-04-20T11:32:41.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:41.259+0000 7fa31b400640 1 -- 172.21.15.152:0/1864242197 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa30c009690 con 0x7fa3000039e0 2024-04-20T11:32:41.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:41.260+0000 7fa321800640 1 -- 172.21.15.152:0/1864242197 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa3000039e0 msgr2=0x7fa300003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:41.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:41.260+0000 7fa321800640 1 --2- 172.21.15.152:0/1864242197 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa3000039e0 0x7fa300003e00 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fa30c009ac0 tx=0x7fa300005830 comp rx=0 tx=0).stop 2024-04-20T11:32:41.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:41.260+0000 7fa320e00640 1 -- 172.21.15.152:0/1864242197 reap_dead start 2024-04-20T11:32:41.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:41.260+0000 7fa321800640 1 -- 172.21.15.152:0/1864242197 shutdown_connections 2024-04-20T11:32:41.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:41.260+0000 7fa321800640 1 -- 172.21.15.152:0/1864242197 >> 172.21.15.152:0/1864242197 conn(0x7fa31c067ae0 msgr2=0x7fa31c06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:41.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:41.260+0000 7fa321800640 1 -- 172.21.15.152:0/1864242197 shutdown_connections 2024-04-20T11:32:41.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:41.260+0000 7fa321800640 1 -- 172.21.15.152:0/1864242197 wait complete. 2024-04-20T11:32:41.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:41.262+0000 7fa321800640 1 Processor -- start 2024-04-20T11:32:41.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:41.262+0000 7fa321800640 1 -- start start 2024-04-20T11:32:41.263 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:41.263+0000 7fa321800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3000039e0 0x7fa31c0aa7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:41.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:41.263+0000 7fa321800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa31c15b5b0 0x7fa31c15ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:41.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:41.263+0000 7fa321800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa31c0b3b60 0x7fa31c0b4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:41.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:41.263+0000 7fa321800640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fa31c06e350 con 0x7fa31c0b3b60 2024-04-20T11:32:41.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:41.263+0000 7fa321800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa31c06e070 con 0x7fa3000039e0 2024-04-20T11:32:41.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:41.263+0000 7fa321800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa31c06e630 con 0x7fa31c15b5b0 2024-04-20T11:32:41.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:41.263+0000 7fa320e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa31c0b3b60 0x7fa31c0b4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:41.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:41.263+0000 7fa320e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa31c0b3b60 0x7fa31c0b4010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43054/0 (socket says 172.21.15.152:43054) 2024-04-20T11:32:41.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:41.263+0000 7fa320e00640 1 -- 172.21.15.152:0/2356537900 learned_addr learned my addr 172.21.15.152:0/2356537900 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:32:41.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:41.264+0000 7fa313400640 1 --2- 172.21.15.152:0/2356537900 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa31c15b5b0 0x7fa31c15ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:41.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:41.264+0000 7fa31be00640 1 --2- 172.21.15.152:0/2356537900 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3000039e0 0x7fa31c0aa7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:41.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:41.264+0000 7fa320e00640 1 -- 172.21.15.152:0/2356537900 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa31c15b5b0 msgr2=0x7fa31c15ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:41.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:41.264+0000 7fa320e00640 1 --2- 172.21.15.152:0/2356537900 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa31c15b5b0 0x7fa31c15ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:41.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:41.264+0000 7fa320e00640 1 -- 172.21.15.152:0/2356537900 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3000039e0 msgr2=0x7fa31c0aa7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:41.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:41.264+0000 7fa320e00640 1 --2- 172.21.15.152:0/2356537900 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3000039e0 0x7fa31c0aa7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:41.265 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:41.264+0000 7fa320e00640 1 -- 172.21.15.152:0/2356537900 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa31c0aad30 con 0x7fa31c0b3b60 2024-04-20T11:32:41.265 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:41.264+0000 7fa320e00640 1 --2- 172.21.15.152:0/2356537900 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa31c0b3b60 0x7fa31c0b4010 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fa31c0b59c0 tx=0x7fa31400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:41.265 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:41.264+0000 7fa318c00640 1 -- 172.21.15.152:0/2356537900 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa314006d50 con 0x7fa31c0b3b60 2024-04-20T11:32:41.265 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:41.265+0000 7fa318c00640 1 -- 172.21.15.152:0/2356537900 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa314003970 con 0x7fa31c0b3b60 2024-04-20T11:32:41.266 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:41.265+0000 7fa321800640 1 -- 172.21.15.152:0/2356537900 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa31c0b5450 con 0x7fa31c0b3b60 2024-04-20T11:32:41.266 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:41.265+0000 7fa318c00640 1 -- 172.21.15.152:0/2356537900 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa314003d60 con 0x7fa31c0b3b60 2024-04-20T11:32:41.266 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:41.265+0000 7fa321800640 1 -- 172.21.15.152:0/2356537900 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa31c0b4bb0 con 0x7fa31c0b3b60 2024-04-20T11:32:41.266 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:41.266+0000 7fa321800640 1 -- 172.21.15.152:0/2356537900 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa31c15bf70 con 0x7fa31c0b3b60 2024-04-20T11:32:41.266 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:41.267+0000 7fa318c00640 1 -- 172.21.15.152:0/2356537900 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 91142+0+0 (secure 0 0 0) 0x7fa31401b020 con 0x7fa31c0b3b60 2024-04-20T11:32:41.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:41.268+0000 7fa318c00640 1 --2- 172.21.15.152:0/2356537900 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fa2f806ca40 0x7fa2f806ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:41.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:41.268+0000 7fa318c00640 1 -- 172.21.15.152:0/2356537900 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5609+0+0 (secure 0 0 0) 0x7fa31408b990 con 0x7fa31c0b3b60 2024-04-20T11:32:41.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:41.269+0000 7fa31be00640 1 --2- 172.21.15.152:0/2356537900 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fa2f806ca40 0x7fa2f806ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:41.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:41.270+0000 7fa31be00640 1 --2- 172.21.15.152:0/2356537900 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fa2f806ca40 0x7fa2f806ef00 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa30c036000 tx=0x7fa30c030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:41.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:41.273+0000 7fa318c00640 1 -- 172.21.15.152:0/2356537900 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fa314059bf0 con 0x7fa31c0b3b60 2024-04-20T11:32:41.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:41.507+0000 7fa321800640 1 -- 172.21.15.152:0/2356537900 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fa31c075e60 con 0x7fa31c0b3b60 2024-04-20T11:32:42.091 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.091+0000 7fa318c00640 1 -- 172.21.15.152:0/2356537900 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 12 and data pool 13 v49) ==== 174+0+0 (secure 0 0 0) 0x7fa3140594f0 con 0x7fa31c0b3b60 2024-04-20T11:32:42.095 INFO:teuthology.orchestra.run.smithi152.stderr:new fs with metadata pool 12 and data pool 13 2024-04-20T11:32:42.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.104+0000 7fa321800640 1 -- 172.21.15.152:0/2356537900 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fa2f806ca40 msgr2=0x7fa2f806ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:42.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.104+0000 7fa321800640 1 --2- 172.21.15.152:0/2356537900 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fa2f806ca40 0x7fa2f806ef00 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa30c036000 tx=0x7fa30c030000 comp rx=0 tx=0).stop 2024-04-20T11:32:42.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.104+0000 7fa321800640 1 -- 172.21.15.152:0/2356537900 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa31c0b3b60 msgr2=0x7fa31c0b4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:42.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.104+0000 7fa321800640 1 --2- 172.21.15.152:0/2356537900 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa31c0b3b60 0x7fa31c0b4010 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fa31c0b59c0 tx=0x7fa31400dc00 comp rx=0 tx=0).stop 2024-04-20T11:32:42.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.106+0000 7fa321800640 1 -- 172.21.15.152:0/2356537900 shutdown_connections 2024-04-20T11:32:42.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.106+0000 7fa321800640 1 --2- 172.21.15.152:0/2356537900 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fa2f806ca40 0x7fa2f806ef00 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:42.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.106+0000 7fa321800640 1 --2- 172.21.15.152:0/2356537900 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa31c0b3b60 0x7fa31c0b4010 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:42.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.106+0000 7fa321800640 1 --2- 172.21.15.152:0/2356537900 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa31c15b5b0 0x7fa31c15ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:42.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.106+0000 7fa321800640 1 --2- 172.21.15.152:0/2356537900 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3000039e0 0x7fa31c0aa7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:42.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.106+0000 7fa321800640 1 -- 172.21.15.152:0/2356537900 >> 172.21.15.152:0/2356537900 conn(0x7fa31c067ae0 msgr2=0x7fa31c0afd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:42.109 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.109+0000 7fa321800640 1 -- 172.21.15.152:0/2356537900 shutdown_connections 2024-04-20T11:32:42.109 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.110+0000 7fa321800640 1 -- 172.21.15.152:0/2356537900 wait complete. 2024-04-20T11:32:42.139 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:32:42.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.311+0000 7f6fcb200640 1 Processor -- start 2024-04-20T11:32:42.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.312+0000 7f6fcb200640 1 -- start start 2024-04-20T11:32:42.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.313+0000 7f6fcb200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f6fcc1414e0 0x7f6fcc166360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:42.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.313+0000 7f6fcb200640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6fcc0cf0a0 con 0x7f6fcc06f510 2024-04-20T11:32:42.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.313+0000 7f6fcb200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6fcc0d00e0 con 0x7f6fcc13e530 2024-04-20T11:32:42.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.313+0000 7f6fcb200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f6fcc141b00 con 0x7f6fcc1414e0 2024-04-20T11:32:42.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.313+0000 7f6fc9400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f6fcc06f510 0x7f6fcc140dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:48412/0 (socket says 172.21.15.152:48412) 2024-04-20T11:32:42.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.313+0000 7f6fc9400640 1 -- 172.21.15.152:0/72506942 learned_addr learned my addr 172.21.15.152:0/72506942 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:32:42.315 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.314+0000 7f6fc8a00640 1 -- 172.21.15.152:0/72506942 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 549888201 0 0) 0x7f6fb4001110 con 0x7f6fcc06f510 2024-04-20T11:32:42.315 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.314+0000 7f6fc8a00640 1 -- 172.21.15.152:0/72506942 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6fac003520 con 0x7f6fcc06f510 2024-04-20T11:32:42.315 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.314+0000 7f6fca800640 1 --2- 172.21.15.152:0/72506942 >> v2:172.21.15.175:3300/0 conn(0x7f6fcc1414e0 0x7f6fcc166360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:42.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.314+0000 7f6fc8a00640 1 -- 172.21.15.152:0/72506942 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 708709700 0 0) 0x7f6fac003520 con 0x7f6fcc06f510 2024-04-20T11:32:42.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.314+0000 7f6fc8a00640 1 -- 172.21.15.152:0/72506942 >> v2:172.21.15.175:3300/0 conn(0x7f6fcc1414e0 msgr2=0x7f6fcc166360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:42.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.314+0000 7f6fc8a00640 1 --2- 172.21.15.152:0/72506942 >> v2:172.21.15.175:3300/0 conn(0x7f6fcc1414e0 0x7f6fcc166360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:42.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.314+0000 7f6fc8a00640 1 -- 172.21.15.152:0/72506942 >> v1:172.21.15.175:6789/0 conn(0x7f6fcc13e530 legacy=0x7f6fcc06ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:42.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.314+0000 7f6fc8a00640 1 -- 172.21.15.152:0/72506942 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fcc0cf320 con 0x7f6fcc06f510 2024-04-20T11:32:42.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.314+0000 7f6fc8a00640 1 -- 172.21.15.152:0/72506942 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f6fb4001840 con 0x7f6fcc06f510 2024-04-20T11:32:42.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.314+0000 7f6fc8a00640 1 -- 172.21.15.152:0/72506942 >> v1:172.21.15.152:6789/0 conn(0x7f6fcc06f510 legacy=0x7f6fcc140dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:42.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.314+0000 7f6fc8a00640 1 --2- 172.21.15.152:0/72506942 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6fac0039b0 0x7f6fac003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:42.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.314+0000 7f6fc8a00640 1 --2- 172.21.15.152:0/72506942 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6fac004800 0x7f6fac004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:42.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.315+0000 7f6fc8a00640 1 --2- 172.21.15.152:0/72506942 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6fac0051c0 0x7f6fac00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:42.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.315+0000 7f6fc8a00640 1 -- 172.21.15.152:0/72506942 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f6fac00c0e0 con 0x7f6fac0039b0 2024-04-20T11:32:42.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.315+0000 7f6fc8a00640 1 -- 172.21.15.152:0/72506942 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6fac00c290 con 0x7f6fac004800 2024-04-20T11:32:42.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.315+0000 7f6fc8a00640 1 -- 172.21.15.152:0/72506942 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6fac00c490 con 0x7f6fac0051c0 2024-04-20T11:32:42.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.315+0000 7f6fc9e00640 1 --2- 172.21.15.152:0/72506942 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6fac0039b0 0x7f6fac003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:42.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.315+0000 7f6fc9400640 1 --2- 172.21.15.152:0/72506942 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6fac004800 0x7f6fac004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:42.317 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.315+0000 7f6fc9e00640 1 -- 172.21.15.152:0/72506942 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6fac0051c0 msgr2=0x7f6fac00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:42.317 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.315+0000 7f6fc9e00640 1 --2- 172.21.15.152:0/72506942 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6fac0051c0 0x7f6fac00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:42.317 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.315+0000 7f6fc9e00640 1 -- 172.21.15.152:0/72506942 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6fac004800 msgr2=0x7f6fac004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:42.317 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.315+0000 7f6fc9e00640 1 --2- 172.21.15.152:0/72506942 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6fac004800 0x7f6fac004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:42.317 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.315+0000 7f6fc9e00640 1 -- 172.21.15.152:0/72506942 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fac003520 con 0x7f6fac0039b0 2024-04-20T11:32:42.317 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.315+0000 7f6fc9400640 1 --2- 172.21.15.152:0/72506942 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6fac004800 0x7f6fac004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:32:42.317 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.315+0000 7f6fca800640 1 -- 172.21.15.152:0/72506942 reap_dead start 2024-04-20T11:32:42.318 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.316+0000 7f6fc9e00640 1 --2- 172.21.15.152:0/72506942 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6fac0039b0 0x7f6fac003dd0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc001750 tx=0x7f6fac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc2a0405e4381087 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:42.318 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.317+0000 7f6fc8a00640 1 -- 172.21.15.152:0/72506942 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fbc001370 con 0x7f6fac0039b0 2024-04-20T11:32:42.318 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.317+0000 7f6fc8a00640 1 -- 172.21.15.152:0/72506942 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fbc0052d0 con 0x7f6fac0039b0 2024-04-20T11:32:42.319 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.317+0000 7f6fc8a00640 1 -- 172.21.15.152:0/72506942 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fbc003830 con 0x7f6fac0039b0 2024-04-20T11:32:42.319 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.317+0000 7f6fcb200640 1 -- 172.21.15.152:0/72506942 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6fac0039b0 msgr2=0x7f6fac003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:42.319 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.317+0000 7f6fcb200640 1 --2- 172.21.15.152:0/72506942 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6fac0039b0 0x7f6fac003dd0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc001750 tx=0x7f6fac005830 comp rx=0 tx=0).stop 2024-04-20T11:32:42.319 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.317+0000 7f6fcb200640 1 -- 172.21.15.152:0/72506942 shutdown_connections 2024-04-20T11:32:42.319 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.317+0000 7f6fcb200640 1 --2- 172.21.15.152:0/72506942 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6fac0039b0 0x7f6fac003dd0 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:42.319 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.317+0000 7f6fcb200640 1 -- 172.21.15.152:0/72506942 >> 172.21.15.152:0/72506942 conn(0x7f6fcc067ae0 msgr2=0x7f6fcc06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:42.319 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.317+0000 7f6fcb200640 1 -- 172.21.15.152:0/72506942 shutdown_connections 2024-04-20T11:32:42.319 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.317+0000 7f6fcb200640 1 -- 172.21.15.152:0/72506942 wait complete. 2024-04-20T11:32:42.319 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.319+0000 7f6fcb200640 1 Processor -- start 2024-04-20T11:32:42.319 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.319+0000 7f6fcb200640 1 -- start start 2024-04-20T11:32:42.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.320+0000 7f6fcb200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6fac0039b0 0x7f6fcc15a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:42.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.320+0000 7f6fcb200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6fcc1583f0 0x7f6fcc13d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:42.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.320+0000 7f6fcb200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6fcc13ddd0 0x7f6fcc1668a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:42.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.320+0000 7f6fcb200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f6fcc0cf8e0 con 0x7f6fcc13ddd0 2024-04-20T11:32:42.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.320+0000 7f6fcb200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6fcc0cfc50 con 0x7f6fcc1583f0 2024-04-20T11:32:42.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.320+0000 7f6fcb200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6fcc140dd0 con 0x7f6fac0039b0 2024-04-20T11:32:42.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.320+0000 7f6fca800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6fcc13ddd0 0x7f6fcc1668a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:42.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.320+0000 7f6fca800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6fcc13ddd0 0x7f6fcc1668a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43076/0 (socket says 172.21.15.152:43076) 2024-04-20T11:32:42.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.320+0000 7f6fca800640 1 -- 172.21.15.152:0/3683435614 learned_addr learned my addr 172.21.15.152:0/3683435614 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:32:42.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.321+0000 7f6fc9400640 1 --2- 172.21.15.152:0/3683435614 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6fcc1583f0 0x7f6fcc13d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:42.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.321+0000 7f6fc9e00640 1 --2- 172.21.15.152:0/3683435614 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6fac0039b0 0x7f6fcc15a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:42.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.321+0000 7f6fca800640 1 -- 172.21.15.152:0/3683435614 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6fac0039b0 msgr2=0x7f6fcc15a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:42.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.321+0000 7f6fca800640 1 --2- 172.21.15.152:0/3683435614 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6fac0039b0 0x7f6fcc15a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:42.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.321+0000 7f6fca800640 1 -- 172.21.15.152:0/3683435614 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6fcc1583f0 msgr2=0x7f6fcc13d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:42.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.321+0000 7f6fca800640 1 --2- 172.21.15.152:0/3683435614 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6fcc1583f0 0x7f6fcc13d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:42.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.321+0000 7f6fca800640 1 -- 172.21.15.152:0/3683435614 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fcc0ae360 con 0x7f6fcc13ddd0 2024-04-20T11:32:42.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.321+0000 7f6fc9e00640 1 --2- 172.21.15.152:0/3683435614 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6fac0039b0 0x7f6fcc15a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:32:42.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.321+0000 7f6fca800640 1 --2- 172.21.15.152:0/3683435614 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6fcc13ddd0 0x7f6fcc1668a0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc159520 tx=0x7f6fc400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:42.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.321+0000 7f6fc2000640 1 -- 172.21.15.152:0/3683435614 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fc400ed90 con 0x7f6fcc13ddd0 2024-04-20T11:32:42.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.321+0000 7f6fc2000640 1 -- 172.21.15.152:0/3683435614 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fc4019070 con 0x7f6fcc13ddd0 2024-04-20T11:32:42.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.321+0000 7f6fc2000640 1 -- 172.21.15.152:0/3683435614 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fc4014660 con 0x7f6fcc13ddd0 2024-04-20T11:32:42.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.321+0000 7f6fcb200640 1 -- 172.21.15.152:0/3683435614 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fcc0ae640 con 0x7f6fcc13ddd0 2024-04-20T11:32:42.323 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.322+0000 7f6fcb200640 1 -- 172.21.15.152:0/3683435614 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fcc0aec00 con 0x7f6fcc13ddd0 2024-04-20T11:32:42.323 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.323+0000 7f6fcb200640 1 -- 172.21.15.152:0/3683435614 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fcc0af0e0 con 0x7f6fcc13ddd0 2024-04-20T11:32:42.323 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.324+0000 7f6fc2000640 1 -- 172.21.15.152:0/3683435614 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 91142+0+0 (secure 0 0 0) 0x7f6fc4014800 con 0x7f6fcc13ddd0 2024-04-20T11:32:42.324 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.325+0000 7f6fc2000640 1 --2- 172.21.15.152:0/3683435614 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f6fa806ca10 0x7f6fa806eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:42.324 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.325+0000 7f6fc2000640 1 -- 172.21.15.152:0/3683435614 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7f6fc408c0e0 con 0x7f6fcc13ddd0 2024-04-20T11:32:42.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.325+0000 7f6fc9e00640 1 --2- 172.21.15.152:0/3683435614 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f6fa806ca10 0x7f6fa806eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:42.326 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.326+0000 7f6fc9e00640 1 --2- 172.21.15.152:0/3683435614 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f6fa806ca10 0x7f6fa806eed0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc033000 tx=0x7f6fbc009030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:42.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.329+0000 7f6fc2000640 1 -- 172.21.15.152:0/3683435614 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f6fcc0af0e0 con 0x7f6fcc13ddd0 2024-04-20T11:32:42.527 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.527+0000 7f6fcb200640 1 -- 172.21.15.152:0/3683435614 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6fcc0b3490 con 0x7f6fcc13ddd0 2024-04-20T11:32:42.528 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.529+0000 7f6fc2000640 1 -- 172.21.15.152:0/3683435614 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+16733 (secure 0 0 0) 0x7f6fc405a360 con 0x7f6fcc13ddd0 2024-04-20T11:32:42.528 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:32:42.529 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":82,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:32:41.508738+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:32:39.268411+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-20T11:32:40.497900+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:32:42.531 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.531+0000 7f6fcb200640 1 -- 172.21.15.152:0/3683435614 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f6fa806ca10 msgr2=0x7f6fa806eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:42.531 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.531+0000 7f6fcb200640 1 --2- 172.21.15.152:0/3683435614 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f6fa806ca10 0x7f6fa806eed0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc033000 tx=0x7f6fbc009030 comp rx=0 tx=0).stop 2024-04-20T11:32:42.531 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.531+0000 7f6fcb200640 1 -- 172.21.15.152:0/3683435614 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6fcc13ddd0 msgr2=0x7f6fcc1668a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:42.531 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.531+0000 7f6fcb200640 1 --2- 172.21.15.152:0/3683435614 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6fcc13ddd0 0x7f6fcc1668a0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc159520 tx=0x7f6fc400e4b0 comp rx=0 tx=0).stop 2024-04-20T11:32:42.532 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.532+0000 7f6fcb200640 1 -- 172.21.15.152:0/3683435614 shutdown_connections 2024-04-20T11:32:42.532 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.532+0000 7f6fcb200640 1 --2- 172.21.15.152:0/3683435614 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f6fa806ca10 0x7f6fa806eed0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:42.532 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.532+0000 7f6fcb200640 1 --2- 172.21.15.152:0/3683435614 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6fcc13ddd0 0x7f6fcc1668a0 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:42.532 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.532+0000 7f6fcb200640 1 --2- 172.21.15.152:0/3683435614 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6fcc1583f0 0x7f6fcc13d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:42.532 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.532+0000 7f6fcb200640 1 --2- 172.21.15.152:0/3683435614 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6fac0039b0 0x7f6fcc15a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:42.532 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.532+0000 7f6fcb200640 1 -- 172.21.15.152:0/3683435614 >> 172.21.15.152:0/3683435614 conn(0x7f6fcc067ae0 msgr2=0x7f6fcc0b4a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:42.532 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.532+0000 7f6fcb200640 1 -- 172.21.15.152:0/3683435614 shutdown_connections 2024-04-20T11:32:42.532 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.532+0000 7f6fcb200640 1 -- 172.21.15.152:0/3683435614 wait complete. 2024-04-20T11:32:42.549 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:32:42.740 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.741+0000 7fce14c00640 1 Processor -- start 2024-04-20T11:32:42.741 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.741+0000 7fce14c00640 1 -- start start 2024-04-20T11:32:42.742 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.742+0000 7fce14c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fce10145f60 0x7fce1006ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:42.742 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.742+0000 7fce14c00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fce100cd0d0 0x7fce100cd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:42.742 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.743+0000 7fce14c00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce10146340 con 0x7fce10071fe0 2024-04-20T11:32:42.742 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.743+0000 7fce14c00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fce1006e9f0 con 0x7fce100cd0d0 2024-04-20T11:32:42.742 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.743+0000 7fce14c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fce1006f3c0 con 0x7fce10145f60 2024-04-20T11:32:42.743 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.743+0000 7fce0de00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fce10145f60 0x7fce1006ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:42.743 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.743+0000 7fce0de00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fce10145f60 0x7fce1006ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.152:60710/0 (socket says 172.21.15.152:60710) 2024-04-20T11:32:42.743 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.743+0000 7fce0de00640 1 -- 172.21.15.152:0/1118124099 learned_addr learned my addr 172.21.15.152:0/1118124099 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:32:42.743 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.743+0000 7fce05400640 1 --2- 172.21.15.152:0/1118124099 >> v2:172.21.15.175:3300/0 conn(0x7fce100cd0d0 0x7fce100cd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:42.744 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.744+0000 7fce0de00640 1 -- 172.21.15.152:0/1118124099 >> v2:172.21.15.175:3300/0 conn(0x7fce100cd0d0 msgr2=0x7fce100cd550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:42.744 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.744+0000 7fce0de00640 1 --2- 172.21.15.152:0/1118124099 >> v2:172.21.15.175:3300/0 conn(0x7fce100cd0d0 0x7fce100cd550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:42.744 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.744+0000 7fce0de00640 1 -- 172.21.15.152:0/1118124099 >> v1:172.21.15.175:6789/0 conn(0x7fce10071fe0 legacy=0x7fce10072470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:42.744 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.744+0000 7fce0de00640 1 -- 172.21.15.152:0/1118124099 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce10072d50 con 0x7fce10145f60 2024-04-20T11:32:42.745 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.746+0000 7fce0de00640 1 --2- 172.21.15.152:0/1118124099 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fce10145f60 0x7fce1006ee80 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fce1006eb70 tx=0x7fce0002b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1db157b6d5266002 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:42.746 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.746+0000 7fce0d400640 1 -- 172.21.15.152:0/1118124099 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce00035020 con 0x7fce10145f60 2024-04-20T11:32:42.746 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.746+0000 7fce0d400640 1 -- 172.21.15.152:0/1118124099 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce00031640 con 0x7fce10145f60 2024-04-20T11:32:42.746 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.746+0000 7fce0d400640 1 -- 172.21.15.152:0/1118124099 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce0002d040 con 0x7fce10145f60 2024-04-20T11:32:42.747 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.747+0000 7fce14c00640 1 -- 172.21.15.152:0/1118124099 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fce10145f60 msgr2=0x7fce1006ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:42.747 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.747+0000 7fce14c00640 1 --2- 172.21.15.152:0/1118124099 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fce10145f60 0x7fce1006ee80 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fce1006eb70 tx=0x7fce0002b8f0 comp rx=0 tx=0).stop 2024-04-20T11:32:42.747 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.747+0000 7fce14c00640 1 -- 172.21.15.152:0/1118124099 shutdown_connections 2024-04-20T11:32:42.747 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.747+0000 7fce14c00640 1 --2- 172.21.15.152:0/1118124099 >> v2:172.21.15.175:3300/0 conn(0x7fce100cd0d0 0x7fce100cd550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:42.747 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.747+0000 7fce14c00640 1 --2- 172.21.15.152:0/1118124099 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fce10145f60 0x7fce1006ee80 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:42.747 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.747+0000 7fce14c00640 1 -- 172.21.15.152:0/1118124099 >> 172.21.15.152:0/1118124099 conn(0x7fce10067cf0 msgr2=0x7fce1006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:42.747 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.747+0000 7fce14c00640 1 -- 172.21.15.152:0/1118124099 shutdown_connections 2024-04-20T11:32:42.747 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.747+0000 7fce14c00640 1 -- 172.21.15.152:0/1118124099 wait complete. 2024-04-20T11:32:42.748 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.749+0000 7fce14c00640 1 Processor -- start 2024-04-20T11:32:42.748 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.749+0000 7fce14c00640 1 -- start start 2024-04-20T11:32:42.749 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.750+0000 7fce14c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fce10071fe0 0x7fce10159fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:42.750 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.750+0000 7fce14c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fce100cd0d0 0x7fce10142280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:42.750 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.750+0000 7fce14c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce10145f60 0x7fce101f2370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:42.750 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.750+0000 7fce14c00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fce1006e680 con 0x7fce10071fe0 2024-04-20T11:32:42.750 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.750+0000 7fce14c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fce1006e3a0 con 0x7fce10145f60 2024-04-20T11:32:42.750 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.750+0000 7fce14c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fce100cf290 con 0x7fce100cd0d0 2024-04-20T11:32:42.750 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.750+0000 7fce0de00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fce10071fe0 0x7fce10159fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:42.750 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.750+0000 7fce0de00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fce10071fe0 0x7fce10159fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43084/0 (socket says 172.21.15.152:43084) 2024-04-20T11:32:42.750 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.750+0000 7fce0de00640 1 -- 172.21.15.152:0/1695844160 learned_addr learned my addr 172.21.15.152:0/1695844160 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:32:42.751 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.751+0000 7fce05400640 1 --2- 172.21.15.152:0/1695844160 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fce100cd0d0 0x7fce10142280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:42.751 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.751+0000 7fce0e800640 1 --2- 172.21.15.152:0/1695844160 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce10145f60 0x7fce101f2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:42.751 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.751+0000 7fce0de00640 1 -- 172.21.15.152:0/1695844160 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fce100cd0d0 msgr2=0x7fce10142280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:42.752 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.751+0000 7fce0de00640 1 --2- 172.21.15.152:0/1695844160 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fce100cd0d0 0x7fce10142280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:42.752 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.751+0000 7fce0de00640 1 -- 172.21.15.152:0/1695844160 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce10145f60 msgr2=0x7fce101f2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:42.752 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.751+0000 7fce0de00640 1 --2- 172.21.15.152:0/1695844160 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce10145f60 0x7fce101f2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:42.752 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.751+0000 7fce0de00640 1 -- 172.21.15.152:0/1695844160 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce101e59b0 con 0x7fce10071fe0 2024-04-20T11:32:42.752 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.751+0000 7fce0e800640 1 --2- 172.21.15.152:0/1695844160 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce10145f60 0x7fce101f2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:32:42.752 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.751+0000 7fce0de00640 1 --2- 172.21.15.152:0/1695844160 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fce10071fe0 0x7fce10159fa0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fce00000c00 tx=0x7fce000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:42.752 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.751+0000 7fce06a00640 1 -- 172.21.15.152:0/1695844160 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce00040070 con 0x7fce10071fe0 2024-04-20T11:32:42.752 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.751+0000 7fce06a00640 1 -- 172.21.15.152:0/1695844160 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce000399f0 con 0x7fce10071fe0 2024-04-20T11:32:42.752 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.751+0000 7fce06a00640 1 -- 172.21.15.152:0/1695844160 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce00035040 con 0x7fce10071fe0 2024-04-20T11:32:42.752 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.751+0000 7fce14c00640 1 -- 172.21.15.152:0/1695844160 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce1015a4e0 con 0x7fce10071fe0 2024-04-20T11:32:42.752 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.752+0000 7fce14c00640 1 -- 172.21.15.152:0/1695844160 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce101e5f30 con 0x7fce10071fe0 2024-04-20T11:32:42.753 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.753+0000 7fce14c00640 1 -- 172.21.15.152:0/1695844160 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce100689f0 con 0x7fce10071fe0 2024-04-20T11:32:42.753 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.754+0000 7fce06a00640 1 -- 172.21.15.152:0/1695844160 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 91142+0+0 (secure 0 0 0) 0x7fce0004b060 con 0x7fce10071fe0 2024-04-20T11:32:42.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.754+0000 7fce06a00640 1 --2- 172.21.15.152:0/1695844160 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fcdec06ca40 0x7fcdec06ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:42.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.755+0000 7fce06a00640 1 -- 172.21.15.152:0/1695844160 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7fce000ad5b0 con 0x7fce10071fe0 2024-04-20T11:32:42.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.755+0000 7fce05400640 1 --2- 172.21.15.152:0/1695844160 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fcdec06ca40 0x7fcdec06ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:42.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.756+0000 7fce05400640 1 --2- 172.21.15.152:0/1695844160 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fcdec06ca40 0x7fcdec06ef00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fce100cda40 tx=0x7fcdf8008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:42.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.759+0000 7fce06a00640 1 -- 172.21.15.152:0/1695844160 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fce0007b8d0 con 0x7fce10071fe0 2024-04-20T11:32:42.945 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.945+0000 7fce14c00640 1 -- 172.21.15.152:0/1695844160 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fce100cd6b0 con 0x7fce10071fe0 2024-04-20T11:32:42.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.948+0000 7fce06a00640 1 -- 172.21.15.152:0/1695844160 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+16733 (secure 0 0 0) 0x7fce0007b1d0 con 0x7fce10071fe0 2024-04-20T11:32:42.948 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:32:42.948 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":82,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:32:41.508738+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:32:39.268411+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-20T11:32:40.497900+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:32:42.950 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.950+0000 7fce14c00640 1 -- 172.21.15.152:0/1695844160 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fcdec06ca40 msgr2=0x7fcdec06ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:42.950 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.950+0000 7fce14c00640 1 --2- 172.21.15.152:0/1695844160 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fcdec06ca40 0x7fcdec06ef00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fce100cda40 tx=0x7fcdf8008040 comp rx=0 tx=0).stop 2024-04-20T11:32:42.950 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.950+0000 7fce14c00640 1 -- 172.21.15.152:0/1695844160 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fce10071fe0 msgr2=0x7fce10159fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:42.950 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.950+0000 7fce14c00640 1 --2- 172.21.15.152:0/1695844160 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fce10071fe0 0x7fce10159fa0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fce00000c00 tx=0x7fce000046c0 comp rx=0 tx=0).stop 2024-04-20T11:32:42.950 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.950+0000 7fce14c00640 1 -- 172.21.15.152:0/1695844160 shutdown_connections 2024-04-20T11:32:42.950 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.950+0000 7fce14c00640 1 --2- 172.21.15.152:0/1695844160 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fcdec06ca40 0x7fcdec06ef00 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:42.950 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.950+0000 7fce14c00640 1 --2- 172.21.15.152:0/1695844160 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce10145f60 0x7fce101f2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:42.950 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.950+0000 7fce14c00640 1 --2- 172.21.15.152:0/1695844160 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fce100cd0d0 0x7fce10142280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:42.950 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.950+0000 7fce14c00640 1 --2- 172.21.15.152:0/1695844160 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fce10071fe0 0x7fce10159fa0 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:42.950 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.950+0000 7fce14c00640 1 -- 172.21.15.152:0/1695844160 >> 172.21.15.152:0/1695844160 conn(0x7fce10067cf0 msgr2=0x7fce1006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:42.951 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.950+0000 7fce14c00640 1 -- 172.21.15.152:0/1695844160 shutdown_connections 2024-04-20T11:32:42.951 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:42.950+0000 7fce14c00640 1 -- 172.21.15.152:0/1695844160 wait complete. 2024-04-20T11:32:42.967 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-20T11:32:43.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:43.162+0000 7f53f6c00640 1 Processor -- start 2024-04-20T11:32:43.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:43.162+0000 7f53f6c00640 1 -- start start 2024-04-20T11:32:43.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:43.163+0000 7f53f6c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f53f0146000 0x7f53f00b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:43.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:43.163+0000 7f53f6c00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f53f00b2170 0x7f53f00b25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:43.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:43.163+0000 7f53f6c00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f53f00cd1b0 0x7f53f00cd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:43.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:43.163+0000 7f53f6c00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f53f00b4750 con 0x7f53f00b2170 2024-04-20T11:32:43.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:43.163+0000 7f53f6c00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f53f01463e0 con 0x7f53f00cd1b0 2024-04-20T11:32:43.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:43.163+0000 7f53f6c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f53f00b3e20 con 0x7f53f0146000 2024-04-20T11:32:43.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:43.164+0000 7f53e7e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f53f00b2170 0x7f53f00b25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:43.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:43.164+0000 7f53e7e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f53f00b2170 0x7f53f00b25f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43088/0 (socket says 172.21.15.152:43088) 2024-04-20T11:32:43.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:43.164+0000 7f53e7e00640 1 -- 172.21.15.152:0/226601472 learned_addr learned my addr 172.21.15.152:0/226601472 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:32:43.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:43.164+0000 7f53f4c00640 1 --2- 172.21.15.152:0/226601472 >> v2:172.21.15.175:3300/0 conn(0x7f53f00cd1b0 0x7f53f00cd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:43.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:43.164+0000 7f53efe00640 1 --2- 172.21.15.152:0/226601472 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f53f0146000 0x7f53f00b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:43.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:43.164+0000 7f53e7e00640 1 -- 172.21.15.152:0/226601472 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f53f0146000 msgr2=0x7f53f00b4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:43.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:43.164+0000 7f53e7e00640 1 --2- 172.21.15.152:0/226601472 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f53f0146000 0x7f53f00b4be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:43.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:43.164+0000 7f53e7e00640 1 -- 172.21.15.152:0/226601472 >> v2:172.21.15.175:3300/0 conn(0x7f53f00cd1b0 msgr2=0x7f53f00cd5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:43.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:43.164+0000 7f53e7e00640 1 --2- 172.21.15.152:0/226601472 >> v2:172.21.15.175:3300/0 conn(0x7f53f00cd1b0 0x7f53f00cd5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:43.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:43.164+0000 7f53e7e00640 1 -- 172.21.15.152:0/226601472 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53f00b4100 con 0x7f53f00b2170 2024-04-20T11:32:43.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:43.165+0000 7f53e7e00640 1 --2- 172.21.15.152:0/226601472 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f53f00b2170 0x7f53f00b25f0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f53f00b3280 tx=0x7f53d802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31abddc8ce2cad5a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:43.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:43.165+0000 7f53ef400640 1 -- 172.21.15.152:0/226601472 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d802e070 con 0x7f53f00b2170 2024-04-20T11:32:43.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:43.165+0000 7f53ef400640 1 -- 172.21.15.152:0/226601472 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53d8037070 con 0x7f53f00b2170 2024-04-20T11:32:43.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:43.165+0000 7f53ef400640 1 -- 172.21.15.152:0/226601472 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d8031a00 con 0x7f53f00b2170 2024-04-20T11:32:43.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:43.166+0000 7f53f6c00640 1 -- 172.21.15.152:0/226601472 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f53f00b2170 msgr2=0x7f53f00b25f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:43.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:43.166+0000 7f53f6c00640 1 --2- 172.21.15.152:0/226601472 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f53f00b2170 0x7f53f00b25f0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f53f00b3280 tx=0x7f53d802b880 comp rx=0 tx=0).stop 2024-04-20T11:32:43.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:43.166+0000 7f53f6c00640 1 -- 172.21.15.152:0/226601472 shutdown_connections 2024-04-20T11:32:43.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:43.166+0000 7f53f6c00640 1 --2- 172.21.15.152:0/226601472 >> v2:172.21.15.175:3300/0 conn(0x7f53f00cd1b0 0x7f53f00cd5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:43.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:43.166+0000 7f53f6c00640 1 --2- 172.21.15.152:0/226601472 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f53f00b2170 0x7f53f00b25f0 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:43.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:43.166+0000 7f53f6c00640 1 --2- 172.21.15.152:0/226601472 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f53f0146000 0x7f53f00b4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:43.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:43.166+0000 7f53f6c00640 1 -- 172.21.15.152:0/226601472 >> 172.21.15.152:0/226601472 conn(0x7f53f006b980 msgr2=0x7f53f006bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:43.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:43.166+0000 7f53f6c00640 1 -- 172.21.15.152:0/226601472 shutdown_connections 2024-04-20T11:32:43.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:43.166+0000 7f53f6c00640 1 -- 172.21.15.152:0/226601472 wait complete. 2024-04-20T11:32:43.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:43.168+0000 7f53f6c00640 1 Processor -- start 2024-04-20T11:32:43.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:43.168+0000 7f53f6c00640 1 -- start start 2024-04-20T11:32:43.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:43.169+0000 7f53f6c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f53f00b2170 0x7f53f00ad8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:43.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:43.169+0000 7f53f6c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f53f00cd1b0 0x7f53f013d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:43.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:43.169+0000 7f53f6c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f53f0146000 0x7f53f006ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:43.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:43.169+0000 7f53f6c00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f53f00cdc80 con 0x7f53f00cd1b0 2024-04-20T11:32:43.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:43.169+0000 7f53f6c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f53f00cdb00 con 0x7f53f00b2170 2024-04-20T11:32:43.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:43.169+0000 7f53f6c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f53f015ccf0 con 0x7f53f0146000 2024-04-20T11:32:43.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:43.169+0000 7f53e7e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f53f00cd1b0 0x7f53f013d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:43.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:43.169+0000 7f53e7e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f53f00cd1b0 0x7f53f013d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43100/0 (socket says 172.21.15.152:43100) 2024-04-20T11:32:43.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:43.169+0000 7f53e7e00640 1 -- 172.21.15.152:0/3646864858 learned_addr learned my addr 172.21.15.152:0/3646864858 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:32:43.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:43.170+0000 7f53efe00640 1 --2- 172.21.15.152:0/3646864858 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f53f00b2170 0x7f53f00ad8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:43.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:43.170+0000 7f53f4c00640 1 --2- 172.21.15.152:0/3646864858 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f53f0146000 0x7f53f006ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:43.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:43.170+0000 7f53e7e00640 1 -- 172.21.15.152:0/3646864858 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f53f0146000 msgr2=0x7f53f006ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:43.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:43.170+0000 7f53e7e00640 1 --2- 172.21.15.152:0/3646864858 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f53f0146000 0x7f53f006ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:43.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:43.170+0000 7f53e7e00640 1 -- 172.21.15.152:0/3646864858 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f53f00b2170 msgr2=0x7f53f00ad8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:43.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:43.170+0000 7f53e7e00640 1 --2- 172.21.15.152:0/3646864858 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f53f00b2170 0x7f53f00ad8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:43.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:43.170+0000 7f53e7e00640 1 -- 172.21.15.152:0/3646864858 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53f015c060 con 0x7f53f00cd1b0 2024-04-20T11:32:43.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:43.170+0000 7f53efe00640 1 --2- 172.21.15.152:0/3646864858 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f53f00b2170 0x7f53f00ad8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:32:43.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:43.170+0000 7f53e7e00640 1 --2- 172.21.15.152:0/3646864858 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f53f00cd1b0 0x7f53f013d890 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f53d802b8c0 tx=0x7f53d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:43.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:43.170+0000 7f53ecc00640 1 -- 172.21.15.152:0/3646864858 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d8032670 con 0x7f53f00cd1b0 2024-04-20T11:32:43.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:43.170+0000 7f53ecc00640 1 -- 172.21.15.152:0/3646864858 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53d8032850 con 0x7f53f00cd1b0 2024-04-20T11:32:43.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:43.170+0000 7f53ecc00640 1 -- 172.21.15.152:0/3646864858 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d8032c40 con 0x7f53f00cd1b0 2024-04-20T11:32:43.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:43.170+0000 7f53f6c00640 1 -- 172.21.15.152:0/3646864858 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53f006f840 con 0x7f53f00cd1b0 2024-04-20T11:32:43.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:43.170+0000 7f53f6c00640 1 -- 172.21.15.152:0/3646864858 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53f013c060 con 0x7f53f00cd1b0 2024-04-20T11:32:43.172 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:43.171+0000 7f53f6c00640 1 -- 172.21.15.152:0/3646864858 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53f00689f0 con 0x7f53f00cd1b0 2024-04-20T11:32:43.172 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:43.172+0000 7f53ecc00640 1 -- 172.21.15.152:0/3646864858 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 91142+0+0 (secure 0 0 0) 0x7f53d80453f0 con 0x7f53f00cd1b0 2024-04-20T11:32:43.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:43.173+0000 7f53ecc00640 1 --2- 172.21.15.152:0/3646864858 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f53cc06ca10 0x7f53cc06eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:43.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:43.173+0000 7f53ecc00640 1 -- 172.21.15.152:0/3646864858 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7f53d80adc10 con 0x7f53f00cd1b0 2024-04-20T11:32:43.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:43.174+0000 7f53efe00640 1 --2- 172.21.15.152:0/3646864858 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f53cc06ca10 0x7f53cc06eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:43.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:43.175+0000 7f53efe00640 1 --2- 172.21.15.152:0/3646864858 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f53cc06ca10 0x7f53cc06eed0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f53e000b800 tx=0x7f53e0008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:43.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:43.178+0000 7f53ecc00640 1 -- 172.21.15.152:0/3646864858 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f53d8040070 con 0x7f53f00cd1b0 2024-04-20T11:32:43.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:43.416+0000 7f53f6c00640 1 -- 172.21.15.152:0/3646864858 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f53f00b03b0 con 0x7f53f00cd1b0 2024-04-20T11:32:44.111 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:44.111+0000 7f53ecc00640 1 -- 172.21.15.152:0/3646864858 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v52) ==== 118+0+0 (secure 0 0 0) 0x7f53d807bd80 con 0x7f53f00cd1b0 2024-04-20T11:32:44.114 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:44.114+0000 7f53f6c00640 1 -- 172.21.15.152:0/3646864858 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f53cc06ca10 msgr2=0x7f53cc06eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:44.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:44.114+0000 7f53f6c00640 1 --2- 172.21.15.152:0/3646864858 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f53cc06ca10 0x7f53cc06eed0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f53e000b800 tx=0x7f53e0008040 comp rx=0 tx=0).stop 2024-04-20T11:32:44.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:44.114+0000 7f53f6c00640 1 -- 172.21.15.152:0/3646864858 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f53f00cd1b0 msgr2=0x7f53f013d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:44.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:44.114+0000 7f53f6c00640 1 --2- 172.21.15.152:0/3646864858 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f53f00cd1b0 0x7f53f013d890 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f53d802b8c0 tx=0x7f53d80046c0 comp rx=0 tx=0).stop 2024-04-20T11:32:44.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:44.115+0000 7f53f6c00640 1 -- 172.21.15.152:0/3646864858 shutdown_connections 2024-04-20T11:32:44.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:44.115+0000 7f53f6c00640 1 --2- 172.21.15.152:0/3646864858 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f53cc06ca10 0x7f53cc06eed0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:44.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:44.115+0000 7f53f6c00640 1 --2- 172.21.15.152:0/3646864858 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f53f0146000 0x7f53f006ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:44.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:44.115+0000 7f53f6c00640 1 --2- 172.21.15.152:0/3646864858 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f53f00cd1b0 0x7f53f013d890 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:44.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:44.115+0000 7f53f6c00640 1 --2- 172.21.15.152:0/3646864858 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f53f00b2170 0x7f53f00ad8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:44.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:44.115+0000 7f53f6c00640 1 -- 172.21.15.152:0/3646864858 >> 172.21.15.152:0/3646864858 conn(0x7f53f006b980 msgr2=0x7f53f0072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:44.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:44.115+0000 7f53f6c00640 1 -- 172.21.15.152:0/3646864858 shutdown_connections 2024-04-20T11:32:44.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:44.115+0000 7f53f6c00640 1 -- 172.21.15.152:0/3646864858 wait complete. 2024-04-20T11:32:44.137 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-20T11:32:44.137 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-20T11:32:44.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:44.342+0000 7f76fea00640 1 Processor -- start 2024-04-20T11:32:44.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:44.342+0000 7f76fea00640 1 -- start start 2024-04-20T11:32:44.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:44.343+0000 7f76fea00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f76f815b510 0x7f76f815fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:44.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:44.343+0000 7f76fea00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f76f80b2880 0x7f76f80b3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:44.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:44.343+0000 7f76fea00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76f815f1c0 con 0x7f76f80cfc40 2024-04-20T11:32:44.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:44.343+0000 7f76fea00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f76f806f420 con 0x7f76f80b2880 2024-04-20T11:32:44.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:44.343+0000 7f76fea00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f76f80b4430 con 0x7f76f815b510 2024-04-20T11:32:44.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:44.344+0000 7f76ef400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f76f80cfc40 0x7f76f80b2170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:48424/0 (socket says 172.21.15.152:48424) 2024-04-20T11:32:44.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:44.344+0000 7f76ef400640 1 -- 172.21.15.152:0/1834931034 learned_addr learned my addr 172.21.15.152:0/1834931034 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:32:44.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:44.344+0000 7f76fca00640 1 --2- 172.21.15.152:0/1834931034 >> v2:172.21.15.152:3300/0 conn(0x7f76f80b2880 0x7f76f80b3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:44.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:44.344+0000 7f76f7400640 1 -- 172.21.15.152:0/1834931034 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2106316048 0 0) 0x7f76f815f1c0 con 0x7f76f80cfc40 2024-04-20T11:32:44.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:44.344+0000 7f76f7400640 1 -- 172.21.15.152:0/1834931034 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f76dc003520 con 0x7f76f80cfc40 2024-04-20T11:32:44.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:44.344+0000 7f76f7e00640 1 --2- 172.21.15.152:0/1834931034 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f76f815b510 0x7f76f815fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:44.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:44.345+0000 7f76fca00640 1 -- 172.21.15.152:0/1834931034 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f76f815b510 msgr2=0x7f76f815fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:44.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:44.345+0000 7f76fca00640 1 --2- 172.21.15.152:0/1834931034 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f76f815b510 0x7f76f815fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:44.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:44.345+0000 7f76fca00640 1 -- 172.21.15.152:0/1834931034 >> v1:172.21.15.152:6789/0 conn(0x7f76f80cfc40 legacy=0x7f76f80b2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:44.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:44.345+0000 7f76fca00640 1 -- 172.21.15.152:0/1834931034 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76f806f670 con 0x7f76f80b2880 2024-04-20T11:32:44.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:44.345+0000 7f76f7e00640 1 --2- 172.21.15.152:0/1834931034 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f76f815b510 0x7f76f815fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:32:44.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:44.346+0000 7f76fca00640 1 --2- 172.21.15.152:0/1834931034 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f76f80b2880 0x7f76f80b3ef0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f76f0009d90 tx=0x7f76f002be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b282fe30a1f30fd1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:44.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:44.346+0000 7f76f7400640 1 -- 172.21.15.152:0/1834931034 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76f0039070 con 0x7f76f80b2880 2024-04-20T11:32:44.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:44.346+0000 7f76f7400640 1 -- 172.21.15.152:0/1834931034 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76f002cbc0 con 0x7f76f80b2880 2024-04-20T11:32:44.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:44.346+0000 7f76f7400640 1 -- 172.21.15.152:0/1834931034 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76f0034440 con 0x7f76f80b2880 2024-04-20T11:32:44.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:44.346+0000 7f76fea00640 1 -- 172.21.15.152:0/1834931034 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f76f80b2880 msgr2=0x7f76f80b3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:44.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:44.346+0000 7f76fea00640 1 --2- 172.21.15.152:0/1834931034 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f76f80b2880 0x7f76f80b3ef0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f76f0009d90 tx=0x7f76f002be10 comp rx=0 tx=0).stop 2024-04-20T11:32:44.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:44.347+0000 7f76fea00640 1 -- 172.21.15.152:0/1834931034 shutdown_connections 2024-04-20T11:32:44.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:44.347+0000 7f76fea00640 1 --2- 172.21.15.152:0/1834931034 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f76f80b2880 0x7f76f80b3ef0 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:44.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:44.347+0000 7f76fea00640 1 --2- 172.21.15.152:0/1834931034 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f76f815b510 0x7f76f815fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:44.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:44.347+0000 7f76fea00640 1 -- 172.21.15.152:0/1834931034 >> 172.21.15.152:0/1834931034 conn(0x7f76f806b980 msgr2=0x7f76f806bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:44.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:44.347+0000 7f76fea00640 1 -- 172.21.15.152:0/1834931034 shutdown_connections 2024-04-20T11:32:44.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:44.347+0000 7f76fea00640 1 -- 172.21.15.152:0/1834931034 wait complete. 2024-04-20T11:32:44.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:44.349+0000 7f76fea00640 1 Processor -- start 2024-04-20T11:32:44.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:44.349+0000 7f76fea00640 1 -- start start 2024-04-20T11:32:44.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:44.350+0000 7f76fea00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f76f80b2880 0x7f76f813f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:44.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:44.350+0000 7f76fea00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f76f80cfc40 0x7f76f8143060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:44.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:44.350+0000 7f76fea00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f76f815b510 0x7f76f80ced30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:44.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:44.350+0000 7f76fea00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f76f815b8f0 con 0x7f76f80b2880 2024-04-20T11:32:44.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:44.350+0000 7f76fea00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f76f815f710 con 0x7f76f815b510 2024-04-20T11:32:44.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:44.350+0000 7f76fea00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f76f806f950 con 0x7f76f80cfc40 2024-04-20T11:32:44.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:44.350+0000 7f76f7e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f76f80b2880 0x7f76f813f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:44.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:44.350+0000 7f76f7e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f76f80b2880 0x7f76f813f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43122/0 (socket says 172.21.15.152:43122) 2024-04-20T11:32:44.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:44.350+0000 7f76f7e00640 1 -- 172.21.15.152:0/3902312020 learned_addr learned my addr 172.21.15.152:0/3902312020 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:32:44.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:44.350+0000 7f76ef400640 1 --2- 172.21.15.152:0/3902312020 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f76f80cfc40 0x7f76f8143060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:44.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:44.350+0000 7f76f7e00640 1 -- 172.21.15.152:0/3902312020 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f76f80cfc40 msgr2=0x7f76f8143060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:44.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:44.350+0000 7f76f7e00640 1 --2- 172.21.15.152:0/3902312020 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f76f80cfc40 0x7f76f8143060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:44.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:44.350+0000 7f76f7e00640 1 -- 172.21.15.152:0/3902312020 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f76f815b510 msgr2=0x7f76f80ced30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:44.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:44.350+0000 7f76f7e00640 1 --2- 172.21.15.152:0/3902312020 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f76f815b510 0x7f76f80ced30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:44.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:44.350+0000 7f76f7e00640 1 -- 172.21.15.152:0/3902312020 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76f814b450 con 0x7f76f80b2880 2024-04-20T11:32:44.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:44.350+0000 7f76ef400640 1 --2- 172.21.15.152:0/3902312020 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f76f80cfc40 0x7f76f8143060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:32:44.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:44.351+0000 7f76f7e00640 1 --2- 172.21.15.152:0/3902312020 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f76f80b2880 0x7f76f813f350 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f76e800c7f0 tx=0x7f76e800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:44.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:44.351+0000 7f76f4c00640 1 -- 172.21.15.152:0/3902312020 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e8015070 con 0x7f76f80b2880 2024-04-20T11:32:44.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:44.351+0000 7f76f4c00640 1 -- 172.21.15.152:0/3902312020 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76e8007820 con 0x7f76f80b2880 2024-04-20T11:32:44.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:44.351+0000 7f76f4c00640 1 -- 172.21.15.152:0/3902312020 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e8007bd0 con 0x7f76f80b2880 2024-04-20T11:32:44.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:44.351+0000 7f76fea00640 1 -- 172.21.15.152:0/3902312020 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76f8143b60 con 0x7f76f80b2880 2024-04-20T11:32:44.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:44.351+0000 7f76fea00640 1 -- 172.21.15.152:0/3902312020 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76f8144120 con 0x7f76f80b2880 2024-04-20T11:32:44.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:44.352+0000 7f76fea00640 1 -- 172.21.15.152:0/3902312020 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76f80689f0 con 0x7f76f80b2880 2024-04-20T11:32:44.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:44.353+0000 7f76f4c00640 1 -- 172.21.15.152:0/3902312020 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 91142+0+0 (secure 0 0 0) 0x7f76e8018070 con 0x7f76f80b2880 2024-04-20T11:32:44.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:44.354+0000 7f76f4c00640 1 --2- 172.21.15.152:0/3902312020 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f76d406ca10 0x7f76d406eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:44.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:44.354+0000 7f76f4c00640 1 -- 172.21.15.152:0/3902312020 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7f76e808bf90 con 0x7f76f80b2880 2024-04-20T11:32:44.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:44.355+0000 7f76ef400640 1 --2- 172.21.15.152:0/3902312020 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f76d406ca10 0x7f76d406eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:44.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:44.356+0000 7f76ef400640 1 --2- 172.21.15.152:0/3902312020 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f76d406ca10 0x7f76d406eed0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f76f81f5960 tx=0x7f76e0009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:44.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:44.359+0000 7f76f4c00640 1 -- 172.21.15.152:0/3902312020 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f76e805a100 con 0x7f76f80b2880 2024-04-20T11:32:44.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:44.599+0000 7f76fea00640 1 -- 172.21.15.152:0/3902312020 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f76f81588c0 con 0x7f76f80b2880 2024-04-20T11:32:45.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.121+0000 7f76f4c00640 1 -- 172.21.15.152:0/3902312020 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v53) ==== 122+0+0 (secure 0 0 0) 0x7f76e8059a00 con 0x7f76f80b2880 2024-04-20T11:32:45.125 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.125+0000 7f76fea00640 1 -- 172.21.15.152:0/3902312020 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f76d406ca10 msgr2=0x7f76d406eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:45.125 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.125+0000 7f76fea00640 1 --2- 172.21.15.152:0/3902312020 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f76d406ca10 0x7f76d406eed0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f76f81f5960 tx=0x7f76e0009040 comp rx=0 tx=0).stop 2024-04-20T11:32:45.125 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.125+0000 7f76fea00640 1 -- 172.21.15.152:0/3902312020 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f76f80b2880 msgr2=0x7f76f813f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:45.125 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.125+0000 7f76fea00640 1 --2- 172.21.15.152:0/3902312020 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f76f80b2880 0x7f76f813f350 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f76e800c7f0 tx=0x7f76e800ccb0 comp rx=0 tx=0).stop 2024-04-20T11:32:45.125 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.125+0000 7f76fea00640 1 -- 172.21.15.152:0/3902312020 shutdown_connections 2024-04-20T11:32:45.125 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.125+0000 7f76fea00640 1 --2- 172.21.15.152:0/3902312020 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f76d406ca10 0x7f76d406eed0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:45.125 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.125+0000 7f76fea00640 1 --2- 172.21.15.152:0/3902312020 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f76f815b510 0x7f76f80ced30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:45.125 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.125+0000 7f76fea00640 1 --2- 172.21.15.152:0/3902312020 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f76f80cfc40 0x7f76f8143060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:45.125 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.125+0000 7f76fea00640 1 --2- 172.21.15.152:0/3902312020 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f76f80b2880 0x7f76f813f350 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:45.126 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.125+0000 7f76fea00640 1 -- 172.21.15.152:0/3902312020 >> 172.21.15.152:0/3902312020 conn(0x7f76f806b980 msgr2=0x7f76f8150d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:45.126 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.125+0000 7f76fea00640 1 -- 172.21.15.152:0/3902312020 shutdown_connections 2024-04-20T11:32:45.126 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.125+0000 7f76fea00640 1 -- 172.21.15.152:0/3902312020 wait complete. 2024-04-20T11:32:45.146 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-20T11:32:45.147 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:32:45.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.354+0000 7f8aeee00640 1 Processor -- start 2024-04-20T11:32:45.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.354+0000 7f8aeee00640 1 -- start start 2024-04-20T11:32:45.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.355+0000 7f8aeee00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f8ae815d1e0 0x7f8ae8160b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:45.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.355+0000 7f8aeee00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ae815d5c0 0x7f8ae80cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:45.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.355+0000 7f8aeee00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f8ae8071cd0 0x7f8ae8072160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:45.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.355+0000 7f8aeee00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f8ae80726a0 con 0x7f8ae815d1e0 2024-04-20T11:32:45.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.355+0000 7f8aeee00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f8ae80ce8c0 con 0x7f8ae8071cd0 2024-04-20T11:32:45.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.355+0000 7f8aeee00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8ae81600b0 con 0x7f8ae815d5c0 2024-04-20T11:32:45.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.356+0000 7f8aeda00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f8ae815d1e0 0x7f8ae8160b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:45.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.356+0000 7f8aeda00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f8ae815d1e0 0x7f8ae8160b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43138/0 (socket says 172.21.15.152:43138) 2024-04-20T11:32:45.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.356+0000 7f8aeda00640 1 -- 172.21.15.152:0/3395274127 learned_addr learned my addr 172.21.15.152:0/3395274127 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:32:45.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.356+0000 7f8aee400640 1 --2- 172.21.15.152:0/3395274127 >> v2:172.21.15.175:3300/0 conn(0x7f8ae8071cd0 0x7f8ae8072160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:45.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.356+0000 7f8aed000640 1 --2- 172.21.15.152:0/3395274127 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ae815d5c0 0x7f8ae80cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:45.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.356+0000 7f8aeda00640 1 -- 172.21.15.152:0/3395274127 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ae815d5c0 msgr2=0x7f8ae80cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:45.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.356+0000 7f8aeda00640 1 --2- 172.21.15.152:0/3395274127 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ae815d5c0 0x7f8ae80cdfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:45.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.356+0000 7f8aeda00640 1 -- 172.21.15.152:0/3395274127 >> v2:172.21.15.175:3300/0 conn(0x7f8ae8071cd0 msgr2=0x7f8ae8072160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:45.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.356+0000 7f8aeda00640 1 --2- 172.21.15.152:0/3395274127 >> v2:172.21.15.175:3300/0 conn(0x7f8ae8071cd0 0x7f8ae8072160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:45.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.356+0000 7f8aeda00640 1 -- 172.21.15.152:0/3395274127 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ae8160390 con 0x7f8ae815d1e0 2024-04-20T11:32:45.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.356+0000 7f8aed000640 1 --2- 172.21.15.152:0/3395274127 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ae815d5c0 0x7f8ae80cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:32:45.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.358+0000 7f8aeda00640 1 --2- 172.21.15.152:0/3395274127 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8ae815d1e0 0x7f8ae8160b90 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f8ae806c4f0 tx=0x7f8adc02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2e7c1a8af19e87f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:45.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.358+0000 7f8ae7e00640 1 -- 172.21.15.152:0/3395274127 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8adc035020 con 0x7f8ae815d1e0 2024-04-20T11:32:45.359 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.358+0000 7f8ae7e00640 1 -- 172.21.15.152:0/3395274127 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8adc031870 con 0x7f8ae815d1e0 2024-04-20T11:32:45.359 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.358+0000 7f8ae7e00640 1 -- 172.21.15.152:0/3395274127 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8adc031c40 con 0x7f8ae815d1e0 2024-04-20T11:32:45.359 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.359+0000 7f8aeee00640 1 -- 172.21.15.152:0/3395274127 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8ae815d1e0 msgr2=0x7f8ae8160b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:45.359 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.359+0000 7f8aeee00640 1 --2- 172.21.15.152:0/3395274127 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8ae815d1e0 0x7f8ae8160b90 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f8ae806c4f0 tx=0x7f8adc02b910 comp rx=0 tx=0).stop 2024-04-20T11:32:45.359 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.359+0000 7f8aeee00640 1 -- 172.21.15.152:0/3395274127 shutdown_connections 2024-04-20T11:32:45.359 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.359+0000 7f8aeee00640 1 --2- 172.21.15.152:0/3395274127 >> v2:172.21.15.175:3300/0 conn(0x7f8ae8071cd0 0x7f8ae8072160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:45.359 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.359+0000 7f8aeee00640 1 --2- 172.21.15.152:0/3395274127 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ae815d5c0 0x7f8ae80cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:45.359 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.359+0000 7f8aeee00640 1 --2- 172.21.15.152:0/3395274127 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8ae815d1e0 0x7f8ae8160b90 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:45.359 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.359+0000 7f8aeee00640 1 -- 172.21.15.152:0/3395274127 >> 172.21.15.152:0/3395274127 conn(0x7f8ae8067ae0 msgr2=0x7f8ae806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:45.359 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.359+0000 7f8aeee00640 1 -- 172.21.15.152:0/3395274127 shutdown_connections 2024-04-20T11:32:45.359 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.359+0000 7f8aeee00640 1 -- 172.21.15.152:0/3395274127 wait complete. 2024-04-20T11:32:45.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.360+0000 7f8aeee00640 1 Processor -- start 2024-04-20T11:32:45.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.361+0000 7f8aeee00640 1 -- start start 2024-04-20T11:32:45.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.362+0000 7f8aeee00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ae8071cd0 0x7f8ae8150df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:45.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.362+0000 7f8aeee00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ae815d1e0 0x7f8ae8151e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:45.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.362+0000 7f8aeee00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8ae815d5c0 0x7f8ae81537c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:45.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.362+0000 7f8aeee00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f8ae80ce520 con 0x7f8ae815d5c0 2024-04-20T11:32:45.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.362+0000 7f8aeee00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8ae80ceba0 con 0x7f8ae815d1e0 2024-04-20T11:32:45.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.362+0000 7f8aeee00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8ae80ce6a0 con 0x7f8ae8071cd0 2024-04-20T11:32:45.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.362+0000 7f8aee400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8ae815d5c0 0x7f8ae81537c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:45.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.362+0000 7f8aee400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8ae815d5c0 0x7f8ae81537c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43140/0 (socket says 172.21.15.152:43140) 2024-04-20T11:32:45.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.362+0000 7f8aee400640 1 -- 172.21.15.152:0/2970094254 learned_addr learned my addr 172.21.15.152:0/2970094254 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:32:45.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.362+0000 7f8aed000640 1 --2- 172.21.15.152:0/2970094254 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ae815d1e0 0x7f8ae8151e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:45.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.362+0000 7f8aeda00640 1 --2- 172.21.15.152:0/2970094254 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ae8071cd0 0x7f8ae8150df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:45.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.362+0000 7f8aee400640 1 -- 172.21.15.152:0/2970094254 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ae8071cd0 msgr2=0x7f8ae8150df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:45.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.362+0000 7f8aee400640 1 --2- 172.21.15.152:0/2970094254 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ae8071cd0 0x7f8ae8150df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:45.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.362+0000 7f8aee400640 1 -- 172.21.15.152:0/2970094254 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ae815d1e0 msgr2=0x7f8ae8151e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:45.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.362+0000 7f8aee400640 1 --2- 172.21.15.152:0/2970094254 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ae815d1e0 0x7f8ae8151e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:45.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.362+0000 7f8aee400640 1 -- 172.21.15.152:0/2970094254 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ae8151330 con 0x7f8ae815d5c0 2024-04-20T11:32:45.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.362+0000 7f8aed000640 1 --2- 172.21.15.152:0/2970094254 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ae815d1e0 0x7f8ae8151e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:32:45.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.363+0000 7f8aee400640 1 --2- 172.21.15.152:0/2970094254 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8ae815d5c0 0x7f8ae81537c0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f8ae0007e60 tx=0x7f8ae000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:45.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.363+0000 7f8ae5600640 1 -- 172.21.15.152:0/2970094254 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae0014620 con 0x7f8ae815d5c0 2024-04-20T11:32:45.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.363+0000 7f8ae5600640 1 -- 172.21.15.152:0/2970094254 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ae0019070 con 0x7f8ae815d5c0 2024-04-20T11:32:45.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.363+0000 7f8ae5600640 1 -- 172.21.15.152:0/2970094254 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae0014a20 con 0x7f8ae815d5c0 2024-04-20T11:32:45.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.363+0000 7f8aeee00640 1 -- 172.21.15.152:0/2970094254 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ae8152720 con 0x7f8ae815d5c0 2024-04-20T11:32:45.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.363+0000 7f8aeee00640 1 -- 172.21.15.152:0/2970094254 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ae8152ce0 con 0x7f8ae815d5c0 2024-04-20T11:32:45.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.364+0000 7f8aeee00640 1 -- 172.21.15.152:0/2970094254 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ae8075e60 con 0x7f8ae815d5c0 2024-04-20T11:32:45.366 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.365+0000 7f8ae5600640 1 -- 172.21.15.152:0/2970094254 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 91142+0+0 (secure 0 0 0) 0x7f8ae0014bc0 con 0x7f8ae815d5c0 2024-04-20T11:32:45.366 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.366+0000 7f8ae5600640 1 --2- 172.21.15.152:0/2970094254 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f8acc06ca10 0x7f8acc06eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:45.366 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.366+0000 7f8ae5600640 1 -- 172.21.15.152:0/2970094254 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7f8ae008c5c0 con 0x7f8ae815d5c0 2024-04-20T11:32:45.366 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.367+0000 7f8aeda00640 1 --2- 172.21.15.152:0/2970094254 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f8acc06ca10 0x7f8acc06eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:45.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.368+0000 7f8aeda00640 1 --2- 172.21.15.152:0/2970094254 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f8acc06ca10 0x7f8acc06eed0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8adc036000 tx=0x7f8adc03c040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:45.370 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.371+0000 7f8ae5600640 1 -- 172.21.15.152:0/2970094254 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f8ae005a840 con 0x7f8ae815d5c0 2024-04-20T11:32:45.608 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.609+0000 7f8aeee00640 1 -- 172.21.15.152:0/2970094254 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8ae815fe70 con 0x7f8ae815d5c0 2024-04-20T11:32:45.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.610+0000 7f8ae5600640 1 -- 172.21.15.152:0/2970094254 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) ==== 94+0+5210 (secure 0 0 0) 0x7f8ae005a140 con 0x7f8ae815d5c0 2024-04-20T11:32:45.610 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:32:45.610 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":53,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5465,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/3844264695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":3844264695},{"type":"v1","addr":"172.21.15.152:6835","nonce":3844264695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":5468,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/706098839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":706098839},{"type":"v1","addr":"172.21.15.175:6833","nonce":706098839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":5899,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/2632275131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":2632275131},{"type":"v1","addr":"172.21.15.175:6835","nonce":2632275131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T11:32:41.508684+0000","modified":"2024-04-20T11:32:44.600495+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7332},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7332":{"gid":7332,"name":"c","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6833/3577228528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":3577228528},{"type":"v1","addr":"172.21.15.152:6833","nonce":3577228528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7332,"qdb_cluster":[7332]},"id":6}]} 2024-04-20T11:32:45.610 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 53 2024-04-20T11:32:45.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.613+0000 7f8aeee00640 1 -- 172.21.15.152:0/2970094254 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f8acc06ca10 msgr2=0x7f8acc06eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:45.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.613+0000 7f8aeee00640 1 --2- 172.21.15.152:0/2970094254 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f8acc06ca10 0x7f8acc06eed0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8adc036000 tx=0x7f8adc03c040 comp rx=0 tx=0).stop 2024-04-20T11:32:45.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.613+0000 7f8aeee00640 1 -- 172.21.15.152:0/2970094254 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8ae815d5c0 msgr2=0x7f8ae81537c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:45.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.613+0000 7f8aeee00640 1 --2- 172.21.15.152:0/2970094254 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8ae815d5c0 0x7f8ae81537c0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f8ae0007e60 tx=0x7f8ae000e4b0 comp rx=0 tx=0).stop 2024-04-20T11:32:45.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.613+0000 7f8aeee00640 1 -- 172.21.15.152:0/2970094254 shutdown_connections 2024-04-20T11:32:45.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.613+0000 7f8aeee00640 1 --2- 172.21.15.152:0/2970094254 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f8acc06ca10 0x7f8acc06eed0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:45.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.613+0000 7f8aeee00640 1 --2- 172.21.15.152:0/2970094254 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8ae815d5c0 0x7f8ae81537c0 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:45.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.613+0000 7f8aeee00640 1 --2- 172.21.15.152:0/2970094254 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ae815d1e0 0x7f8ae8151e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:45.614 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.613+0000 7f8aeee00640 1 --2- 172.21.15.152:0/2970094254 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ae8071cd0 0x7f8ae8150df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:45.614 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.613+0000 7f8aeee00640 1 -- 172.21.15.152:0/2970094254 >> 172.21.15.152:0/2970094254 conn(0x7f8ae8067ae0 msgr2=0x7f8ae815a620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:45.614 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.613+0000 7f8aeee00640 1 -- 172.21.15.152:0/2970094254 shutdown_connections 2024-04-20T11:32:45.614 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.613+0000 7f8aeee00640 1 -- 172.21.15.152:0/2970094254 wait complete. 2024-04-20T11:32:45.633 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:32:45.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.834+0000 7fe0b1c00640 1 Processor -- start 2024-04-20T11:32:45.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.834+0000 7fe0b1c00640 1 -- start start 2024-04-20T11:32:45.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.835+0000 7fe0b1c00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fe0ac15d1e0 0x7fe0ac160b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:45.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.836+0000 7fe0b1c00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0ac0ce8c0 con 0x7fe0ac071cd0 2024-04-20T11:32:45.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.836+0000 7fe0b1c00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0ac1600b0 con 0x7fe0ac15d5c0 2024-04-20T11:32:45.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.836+0000 7fe0b1c00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fe0ac160390 con 0x7fe0ac15d1e0 2024-04-20T11:32:45.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.836+0000 7fe0b1200640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fe0ac071cd0 0x7fe0ac072120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:41126/0 (socket says 172.21.15.152:41126) 2024-04-20T11:32:45.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.836+0000 7fe0b1200640 1 -- 172.21.15.152:0/1814701532 learned_addr learned my addr 172.21.15.152:0/1814701532 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:32:45.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.836+0000 7fe0abe00640 1 --2- 172.21.15.152:0/1814701532 >> v2:172.21.15.175:3300/0 conn(0x7fe0ac15d1e0 0x7fe0ac160b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:45.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.836+0000 7fe0aaa00640 1 -- 172.21.15.152:0/1814701532 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2997763670 0 0) 0x7fe0ac0ce8c0 con 0x7fe0ac071cd0 2024-04-20T11:32:45.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.837+0000 7fe0aaa00640 1 -- 172.21.15.152:0/1814701532 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe090003520 con 0x7fe0ac071cd0 2024-04-20T11:32:45.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.837+0000 7fe0aaa00640 1 -- 172.21.15.152:0/1814701532 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2898404200 0 0) 0x7fe090003520 con 0x7fe0ac071cd0 2024-04-20T11:32:45.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.837+0000 7fe0aaa00640 1 -- 172.21.15.152:0/1814701532 >> v2:172.21.15.175:3300/0 conn(0x7fe0ac15d1e0 msgr2=0x7fe0ac160b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:45.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.837+0000 7fe0aaa00640 1 --2- 172.21.15.152:0/1814701532 >> v2:172.21.15.175:3300/0 conn(0x7fe0ac15d1e0 0x7fe0ac160b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:45.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.837+0000 7fe0aaa00640 1 -- 172.21.15.152:0/1814701532 >> v1:172.21.15.175:6789/0 conn(0x7fe0ac15d5c0 legacy=0x7fe0ac0cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:45.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.837+0000 7fe0aaa00640 1 -- 172.21.15.152:0/1814701532 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0ac160670 con 0x7fe0ac071cd0 2024-04-20T11:32:45.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.837+0000 7fe0aaa00640 1 -- 172.21.15.152:0/1814701532 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7fe0a40019f0 con 0x7fe0ac071cd0 2024-04-20T11:32:45.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.837+0000 7fe0aaa00640 1 -- 172.21.15.152:0/1814701532 >> v1:172.21.15.152:6789/0 conn(0x7fe0ac071cd0 legacy=0x7fe0ac072120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:45.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.837+0000 7fe0aaa00640 1 --2- 172.21.15.152:0/1814701532 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe090003a00 0x7fe090003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:45.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.837+0000 7fe0aaa00640 1 --2- 172.21.15.152:0/1814701532 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe090004800 0x7fe090004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:45.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.837+0000 7fe0aaa00640 1 --2- 172.21.15.152:0/1814701532 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0900051c0 0x7fe09000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:45.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.837+0000 7fe0aaa00640 1 -- 172.21.15.152:0/1814701532 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fe0ac160390 con 0x7fe090003a00 2024-04-20T11:32:45.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.837+0000 7fe0aaa00640 1 -- 172.21.15.152:0/1814701532 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe09000c0c0 con 0x7fe0900051c0 2024-04-20T11:32:45.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.837+0000 7fe0aaa00640 1 -- 172.21.15.152:0/1814701532 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe09000c2c0 con 0x7fe090004800 2024-04-20T11:32:45.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.837+0000 7fe0abe00640 1 --2- 172.21.15.152:0/1814701532 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe090003a00 0x7fe090003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:45.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.838+0000 7fe0abe00640 1 -- 172.21.15.152:0/1814701532 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe090004800 msgr2=0x7fe090004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:45.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.838+0000 7fe0ab400640 1 --2- 172.21.15.152:0/1814701532 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe090004800 0x7fe090004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:45.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.838+0000 7fe0b1200640 1 --2- 172.21.15.152:0/1814701532 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0900051c0 0x7fe09000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:45.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.838+0000 7fe0abe00640 1 --2- 172.21.15.152:0/1814701532 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe090004800 0x7fe090004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:45.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.838+0000 7fe0abe00640 1 -- 172.21.15.152:0/1814701532 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0900051c0 msgr2=0x7fe09000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:45.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.838+0000 7fe0abe00640 1 --2- 172.21.15.152:0/1814701532 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0900051c0 0x7fe09000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:45.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.838+0000 7fe0abe00640 1 -- 172.21.15.152:0/1814701532 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe090003520 con 0x7fe090003a00 2024-04-20T11:32:45.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.838+0000 7fe0b1200640 1 --2- 172.21.15.152:0/1814701532 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0900051c0 0x7fe09000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:32:45.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.838+0000 7fe0b1200640 1 -- 172.21.15.152:0/1814701532 reap_dead start 2024-04-20T11:32:45.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.839+0000 7fe0abe00640 1 --2- 172.21.15.152:0/1814701532 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe090003a00 0x7fe090003e20 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7fe09c009ac0 tx=0x7fe090005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ede3c058cf46eae1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:45.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.839+0000 7fe0aaa00640 1 -- 172.21.15.152:0/1814701532 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe09c039070 con 0x7fe090003a00 2024-04-20T11:32:45.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.839+0000 7fe0aaa00640 1 -- 172.21.15.152:0/1814701532 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe09c02c6b0 con 0x7fe090003a00 2024-04-20T11:32:45.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.839+0000 7fe0aaa00640 1 -- 172.21.15.152:0/1814701532 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe09c009690 con 0x7fe090003a00 2024-04-20T11:32:45.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.840+0000 7fe0b1c00640 1 -- 172.21.15.152:0/1814701532 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe090003a00 msgr2=0x7fe090003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:45.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.840+0000 7fe0b1c00640 1 --2- 172.21.15.152:0/1814701532 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe090003a00 0x7fe090003e20 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7fe09c009ac0 tx=0x7fe090005830 comp rx=0 tx=0).stop 2024-04-20T11:32:45.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.840+0000 7fe0b1c00640 1 -- 172.21.15.152:0/1814701532 shutdown_connections 2024-04-20T11:32:45.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.840+0000 7fe0b1c00640 1 --2- 172.21.15.152:0/1814701532 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe090003a00 0x7fe090003e20 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:45.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.840+0000 7fe0b1c00640 1 -- 172.21.15.152:0/1814701532 >> 172.21.15.152:0/1814701532 conn(0x7fe0ac067ae0 msgr2=0x7fe0ac06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:45.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.840+0000 7fe0b1c00640 1 -- 172.21.15.152:0/1814701532 shutdown_connections 2024-04-20T11:32:45.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.840+0000 7fe0b1c00640 1 -- 172.21.15.152:0/1814701532 wait complete. 2024-04-20T11:32:45.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.842+0000 7fe0b1c00640 1 Processor -- start 2024-04-20T11:32:45.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.842+0000 7fe0b1c00640 1 -- start start 2024-04-20T11:32:45.843 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.843+0000 7fe0b1c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe090003a00 0x7fe0ac1593f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:45.843 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.843+0000 7fe0b1c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0ac155d30 0x7fe0ac1427e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:45.843 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.843+0000 7fe0b1c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe0ac1561b0 0x7fe0ac142d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:45.843 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.843+0000 7fe0b1c00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fe0ac160d10 con 0x7fe0ac1561b0 2024-04-20T11:32:45.843 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.843+0000 7fe0b1c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe0ac160b90 con 0x7fe0ac155d30 2024-04-20T11:32:45.843 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.843+0000 7fe0b1c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe0ac160e90 con 0x7fe090003a00 2024-04-20T11:32:45.843 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.843+0000 7fe0b1200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe0ac1561b0 0x7fe0ac142d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:45.843 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.843+0000 7fe0b1200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe0ac1561b0 0x7fe0ac142d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:44898/0 (socket says 172.21.15.152:44898) 2024-04-20T11:32:45.843 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.843+0000 7fe0b1200640 1 -- 172.21.15.152:0/994331878 learned_addr learned my addr 172.21.15.152:0/994331878 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:32:45.844 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.844+0000 7fe0ab400640 1 --2- 172.21.15.152:0/994331878 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0ac155d30 0x7fe0ac1427e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:45.844 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.844+0000 7fe0abe00640 1 --2- 172.21.15.152:0/994331878 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe090003a00 0x7fe0ac1593f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:45.844 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.844+0000 7fe0b1200640 1 -- 172.21.15.152:0/994331878 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe090003a00 msgr2=0x7fe0ac1593f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:45.845 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.844+0000 7fe0b1200640 1 --2- 172.21.15.152:0/994331878 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe090003a00 0x7fe0ac1593f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:45.845 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.844+0000 7fe0b1200640 1 -- 172.21.15.152:0/994331878 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0ac155d30 msgr2=0x7fe0ac1427e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:45.845 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.844+0000 7fe0b1200640 1 --2- 172.21.15.152:0/994331878 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0ac155d30 0x7fe0ac1427e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:45.845 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.844+0000 7fe0b1200640 1 -- 172.21.15.152:0/994331878 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0ac1e6b60 con 0x7fe0ac1561b0 2024-04-20T11:32:45.845 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.844+0000 7fe0b1200640 1 --2- 172.21.15.152:0/994331878 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe0ac1561b0 0x7fe0ac142d20 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac15a560 tx=0x7fe0a400df70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:45.845 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.844+0000 7fe0a3e00640 1 -- 172.21.15.152:0/994331878 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a400b860 con 0x7fe0ac1561b0 2024-04-20T11:32:45.845 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.844+0000 7fe0a3e00640 1 -- 172.21.15.152:0/994331878 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0a400ba20 con 0x7fe0ac1561b0 2024-04-20T11:32:45.845 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.844+0000 7fe0a3e00640 1 -- 172.21.15.152:0/994331878 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a400be10 con 0x7fe0ac1561b0 2024-04-20T11:32:45.845 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.844+0000 7fe0b1c00640 1 -- 172.21.15.152:0/994331878 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0ac158910 con 0x7fe0ac1561b0 2024-04-20T11:32:45.845 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.845+0000 7fe0b1c00640 1 -- 172.21.15.152:0/994331878 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0ac158ed0 con 0x7fe0ac1561b0 2024-04-20T11:32:45.846 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.846+0000 7fe0b1c00640 1 -- 172.21.15.152:0/994331878 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0ac1590d0 con 0x7fe0ac1561b0 2024-04-20T11:32:45.846 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.847+0000 7fe0a3e00640 1 -- 172.21.15.152:0/994331878 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 91142+0+0 (secure 0 0 0) 0x7fe0a401b020 con 0x7fe0ac1561b0 2024-04-20T11:32:45.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.848+0000 7fe0a3e00640 1 --2- 172.21.15.152:0/994331878 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe08c06ca10 0x7fe08c06eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:45.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.848+0000 7fe0a3e00640 1 -- 172.21.15.152:0/994331878 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7fe0a408bea0 con 0x7fe0ac1561b0 2024-04-20T11:32:45.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.848+0000 7fe0abe00640 1 --2- 172.21.15.152:0/994331878 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe08c06ca10 0x7fe08c06eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:45.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.849+0000 7fe0abe00640 1 --2- 172.21.15.152:0/994331878 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe08c06ca10 0x7fe08c06eed0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe09c036000 tx=0x7fe09c030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:45.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:45.852+0000 7fe0a3e00640 1 -- 172.21.15.152:0/994331878 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe0ac1590d0 con 0x7fe0ac1561b0 2024-04-20T11:32:46.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.048+0000 7fe0b1c00640 1 -- 172.21.15.152:0/994331878 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe0ac075e60 con 0x7fe0ac1561b0 2024-04-20T11:32:46.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.050+0000 7fe0a3e00640 1 -- 172.21.15.152:0/994331878 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+16720 (secure 0 0 0) 0x7fe0a405a0a0 con 0x7fe0ac1561b0 2024-04-20T11:32:46.050 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:32:46.050 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":83,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:32:43.089139+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:32:39.268411+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-20T11:32:40.497900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:32:46.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.052+0000 7fe0b1c00640 1 -- 172.21.15.152:0/994331878 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe08c06ca10 msgr2=0x7fe08c06eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:46.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.052+0000 7fe0b1c00640 1 --2- 172.21.15.152:0/994331878 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe08c06ca10 0x7fe08c06eed0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe09c036000 tx=0x7fe09c030000 comp rx=0 tx=0).stop 2024-04-20T11:32:46.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.052+0000 7fe0b1c00640 1 -- 172.21.15.152:0/994331878 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe0ac1561b0 msgr2=0x7fe0ac142d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:46.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.052+0000 7fe0b1c00640 1 --2- 172.21.15.152:0/994331878 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe0ac1561b0 0x7fe0ac142d20 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac15a560 tx=0x7fe0a400df70 comp rx=0 tx=0).stop 2024-04-20T11:32:46.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.052+0000 7fe0b1c00640 1 -- 172.21.15.152:0/994331878 shutdown_connections 2024-04-20T11:32:46.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.053+0000 7fe0b1c00640 1 --2- 172.21.15.152:0/994331878 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe08c06ca10 0x7fe08c06eed0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:46.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.053+0000 7fe0b1c00640 1 --2- 172.21.15.152:0/994331878 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe0ac1561b0 0x7fe0ac142d20 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:46.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.053+0000 7fe0b1c00640 1 --2- 172.21.15.152:0/994331878 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0ac155d30 0x7fe0ac1427e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:46.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.053+0000 7fe0b1c00640 1 --2- 172.21.15.152:0/994331878 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe090003a00 0x7fe0ac1593f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:46.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.053+0000 7fe0b1c00640 1 -- 172.21.15.152:0/994331878 >> 172.21.15.152:0/994331878 conn(0x7fe0ac067ae0 msgr2=0x7fe0ac15f990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:46.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.053+0000 7fe0b1c00640 1 -- 172.21.15.152:0/994331878 shutdown_connections 2024-04-20T11:32:46.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.053+0000 7fe0b1c00640 1 -- 172.21.15.152:0/994331878 wait complete. 2024-04-20T11:32:46.070 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-20T11:32:46.070 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T11:32:46.265 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.266+0000 7f9c48a00640 1 Processor -- start 2024-04-20T11:32:46.265 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.266+0000 7f9c48a00640 1 -- start start 2024-04-20T11:32:46.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.267+0000 7f9c48a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f9c440b3cd0 0x7f9c44071cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:46.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.267+0000 7f9c48a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9c44072210 0x7f9c44072680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:46.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.267+0000 7f9c48a00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c4415b340 con 0x7f9c44151ab0 2024-04-20T11:32:46.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.267+0000 7f9c48a00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f9c4415c100 con 0x7f9c440b3cd0 2024-04-20T11:32:46.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.267+0000 7f9c48a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f9c4415bc70 con 0x7f9c44072210 2024-04-20T11:32:46.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.268+0000 7f9c3aa00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f9c440b3cd0 0x7f9c44071cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:46.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.268+0000 7f9c3aa00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f9c440b3cd0 0x7f9c44071cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:44906/0 (socket says 172.21.15.152:44906) 2024-04-20T11:32:46.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.268+0000 7f9c43400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f9c44151ab0 0x7f9c440b35c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:41136/0 (socket says 172.21.15.152:41136) 2024-04-20T11:32:46.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.268+0000 7f9c3aa00640 1 -- 172.21.15.152:0/897802645 learned_addr learned my addr 172.21.15.152:0/897802645 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:32:46.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.268+0000 7f9c3aa00640 1 -- 172.21.15.152:0/897802645 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9c44072210 msgr2=0x7f9c44072680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:46.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.268+0000 7f9c3aa00640 1 --2- 172.21.15.152:0/897802645 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9c44072210 0x7f9c44072680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:46.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.268+0000 7f9c3aa00640 1 -- 172.21.15.152:0/897802645 >> v1:172.21.15.152:6789/0 conn(0x7f9c44151ab0 legacy=0x7f9c440b35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:46.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.268+0000 7f9c3aa00640 1 -- 172.21.15.152:0/897802645 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c4415b620 con 0x7f9c440b3cd0 2024-04-20T11:32:46.270 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.269+0000 7f9c3aa00640 1 --2- 172.21.15.152:0/897802645 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9c440b3cd0 0x7f9c44071cd0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c009870 tx=0x7f9c2c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60322281f627e8d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:46.270 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.269+0000 7f9c42a00640 1 -- 172.21.15.152:0/897802645 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c2c034070 con 0x7f9c440b3cd0 2024-04-20T11:32:46.270 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.269+0000 7f9c42a00640 1 -- 172.21.15.152:0/897802645 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c2c030840 con 0x7f9c440b3cd0 2024-04-20T11:32:46.270 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.269+0000 7f9c42a00640 1 -- 172.21.15.152:0/897802645 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c2c030c10 con 0x7f9c440b3cd0 2024-04-20T11:32:46.270 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.270+0000 7f9c48a00640 1 -- 172.21.15.152:0/897802645 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9c440b3cd0 msgr2=0x7f9c44071cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:46.270 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.270+0000 7f9c48a00640 1 --2- 172.21.15.152:0/897802645 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9c440b3cd0 0x7f9c44071cd0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c009870 tx=0x7f9c2c02b980 comp rx=0 tx=0).stop 2024-04-20T11:32:46.270 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.270+0000 7f9c48a00640 1 -- 172.21.15.152:0/897802645 shutdown_connections 2024-04-20T11:32:46.270 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.270+0000 7f9c48a00640 1 --2- 172.21.15.152:0/897802645 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9c44072210 0x7f9c44072680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:46.270 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.270+0000 7f9c48a00640 1 --2- 172.21.15.152:0/897802645 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9c440b3cd0 0x7f9c44071cd0 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:46.271 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.270+0000 7f9c48a00640 1 -- 172.21.15.152:0/897802645 >> 172.21.15.152:0/897802645 conn(0x7f9c44067ae0 msgr2=0x7f9c4406b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:46.271 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.270+0000 7f9c48a00640 1 -- 172.21.15.152:0/897802645 shutdown_connections 2024-04-20T11:32:46.271 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.270+0000 7f9c48a00640 1 -- 172.21.15.152:0/897802645 wait complete. 2024-04-20T11:32:46.271 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.272+0000 7f9c48a00640 1 Processor -- start 2024-04-20T11:32:46.271 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.272+0000 7f9c48a00640 1 -- start start 2024-04-20T11:32:46.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.273+0000 7f9c48a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9c44072210 0x7f9c44146f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:46.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.273+0000 7f9c48a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9c440b3cd0 0x7f9c4414eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:46.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.273+0000 7f9c48a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c44151ab0 0x7f9c44154760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:46.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.273+0000 7f9c48a00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f9c440ce930 con 0x7f9c440b3cd0 2024-04-20T11:32:46.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.273+0000 7f9c48a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9c440ce7b0 con 0x7f9c44151ab0 2024-04-20T11:32:46.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.273+0000 7f9c48a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f9c44151e90 con 0x7f9c44072210 2024-04-20T11:32:46.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.273+0000 7f9c3aa00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9c440b3cd0 0x7f9c4414eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:46.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.273+0000 7f9c3aa00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9c440b3cd0 0x7f9c4414eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:44922/0 (socket says 172.21.15.152:44922) 2024-04-20T11:32:46.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.273+0000 7f9c3aa00640 1 -- 172.21.15.152:0/3336203427 learned_addr learned my addr 172.21.15.152:0/3336203427 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:32:46.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.273+0000 7f9c3aa00640 1 -- 172.21.15.152:0/3336203427 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9c44072210 msgr2=0x7f9c44146f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:46.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.273+0000 7f9c3aa00640 1 --2- 172.21.15.152:0/3336203427 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9c44072210 0x7f9c44146f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:46.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.273+0000 7f9c3aa00640 1 -- 172.21.15.152:0/3336203427 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c44151ab0 msgr2=0x7f9c44154760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:46.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.273+0000 7f9c3aa00640 1 --2- 172.21.15.152:0/3336203427 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c44151ab0 0x7f9c44154760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:46.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.273+0000 7f9c3aa00640 1 -- 172.21.15.152:0/3336203427 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c4414b690 con 0x7f9c440b3cd0 2024-04-20T11:32:46.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.274+0000 7f9c3aa00640 1 --2- 172.21.15.152:0/3336203427 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9c440b3cd0 0x7f9c4414eb70 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c009870 tx=0x7f9c2c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:46.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.274+0000 7f9c3be00640 1 -- 172.21.15.152:0/3336203427 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c2c048070 con 0x7f9c440b3cd0 2024-04-20T11:32:46.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.274+0000 7f9c3be00640 1 -- 172.21.15.152:0/3336203427 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c2c030640 con 0x7f9c440b3cd0 2024-04-20T11:32:46.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.274+0000 7f9c48a00640 1 -- 172.21.15.152:0/3336203427 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c4414f4e0 con 0x7f9c440b3cd0 2024-04-20T11:32:46.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.274+0000 7f9c3be00640 1 -- 172.21.15.152:0/3336203427 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c2c034040 con 0x7f9c440b3cd0 2024-04-20T11:32:46.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.274+0000 7f9c48a00640 1 -- 172.21.15.152:0/3336203427 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c4414faa0 con 0x7f9c440b3cd0 2024-04-20T11:32:46.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.275+0000 7f9c48a00640 1 -- 172.21.15.152:0/3336203427 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c44075e60 con 0x7f9c440b3cd0 2024-04-20T11:32:46.276 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.276+0000 7f9c3be00640 1 -- 172.21.15.152:0/3336203427 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 91142+0+0 (secure 0 0 0) 0x7f9c2c04b070 con 0x7f9c440b3cd0 2024-04-20T11:32:46.276 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.277+0000 7f9c3be00640 1 --2- 172.21.15.152:0/3336203427 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f9c2006ca10 0x7f9c2006eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:46.276 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.277+0000 7f9c3be00640 1 -- 172.21.15.152:0/3336203427 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7f9c2c0ade70 con 0x7f9c440b3cd0 2024-04-20T11:32:46.277 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.278+0000 7f9c43400640 1 --2- 172.21.15.152:0/3336203427 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f9c2006ca10 0x7f9c2006eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:46.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.279+0000 7f9c43400640 1 --2- 172.21.15.152:0/3336203427 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f9c2006ca10 0x7f9c2006eed0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9c34000f60 tx=0x7f9c34007a00 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:46.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.281+0000 7f9c3be00640 1 -- 172.21.15.152:0/3336203427 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f9c2c07c110 con 0x7f9c440b3cd0 2024-04-20T11:32:46.462 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.463+0000 7f9c48a00640 1 -- 172.21.15.152:0/3336203427 --> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9c4406be70 con 0x7f9c2006ca10 2024-04-20T11:32:46.485 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.486+0000 7f9c3be00640 1 -- 172.21.15.152:0/3336203427 <== mgr.4109 v2:172.21.15.175:6836/4021250441 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252079 (secure 0 0 0) 0x7f9c4406be70 con 0x7f9c2006ca10 2024-04-20T11:32:46.489 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:32:46.490 INFO:teuthology.orchestra.run.smithi152.stderr:dumped all 2024-04-20T11:32:46.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.490+0000 7f9c48a00640 1 -- 172.21.15.152:0/3336203427 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f9c2006ca10 msgr2=0x7f9c2006eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:46.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.490+0000 7f9c48a00640 1 --2- 172.21.15.152:0/3336203427 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f9c2006ca10 0x7f9c2006eed0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9c34000f60 tx=0x7f9c34007a00 comp rx=0 tx=0).stop 2024-04-20T11:32:46.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.490+0000 7f9c48a00640 1 -- 172.21.15.152:0/3336203427 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9c440b3cd0 msgr2=0x7f9c4414eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:46.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.490+0000 7f9c48a00640 1 --2- 172.21.15.152:0/3336203427 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9c440b3cd0 0x7f9c4414eb70 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c009870 tx=0x7f9c2c004730 comp rx=0 tx=0).stop 2024-04-20T11:32:46.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.490+0000 7f9c48a00640 1 -- 172.21.15.152:0/3336203427 shutdown_connections 2024-04-20T11:32:46.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.490+0000 7f9c48a00640 1 --2- 172.21.15.152:0/3336203427 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f9c2006ca10 0x7f9c2006eed0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:46.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.490+0000 7f9c48a00640 1 --2- 172.21.15.152:0/3336203427 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c44151ab0 0x7f9c44154760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:46.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.490+0000 7f9c48a00640 1 --2- 172.21.15.152:0/3336203427 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9c440b3cd0 0x7f9c4414eb70 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:46.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.490+0000 7f9c48a00640 1 --2- 172.21.15.152:0/3336203427 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9c44072210 0x7f9c44146f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:46.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.490+0000 7f9c48a00640 1 -- 172.21.15.152:0/3336203427 >> 172.21.15.152:0/3336203427 conn(0x7f9c44067ae0 msgr2=0x7f9c440b4880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:46.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.491+0000 7f9c48a00640 1 -- 172.21.15.152:0/3336203427 shutdown_connections 2024-04-20T11:32:46.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.491+0000 7f9c48a00640 1 -- 172.21.15.152:0/3336203427 wait complete. 2024-04-20T11:32:46.508 INFO:teuthology.orchestra.run.smithi152.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2024-04-20T11:32:44.601647+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":224,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":167,"ondisk_log_size":167,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":123,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":258596,"kb_used_data":9132,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754716124,"statfs":{"total":773094113280,"available":772829310976,"internally_reserved":0,"allocated":9351168,"data_stored":4657504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12746,"internal_metadata":255315510},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.568220"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.408527+0000","last_change":"2024-04-20T11:32:42.111460+0000","last_active":"2024-04-20T11:32:43.408527+0000","last_peered":"2024-04-20T11:32:43.408527+0000","last_clean":"2024-04-20T11:32:43.408527+0000","last_became_active":"2024-04-20T11:32:42.111130+0000","last_became_peered":"2024-04-20T11:32:42.111130+0000","last_unstale":"2024-04-20T11:32:43.408527+0000","last_undegraded":"2024-04-20T11:32:43.408527+0000","last_fullsized":"2024-04-20T11:32:43.408527+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:26:31.889820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.110198+0000","last_change":"2024-04-20T11:32:42.121976+0000","last_active":"2024-04-20T11:32:43.110198+0000","last_peered":"2024-04-20T11:32:43.110198+0000","last_clean":"2024-04-20T11:32:43.110198+0000","last_became_active":"2024-04-20T11:32:42.121709+0000","last_became_peered":"2024-04-20T11:32:42.121709+0000","last_unstale":"2024-04-20T11:32:43.110198+0000","last_undegraded":"2024-04-20T11:32:43.110198+0000","last_fullsized":"2024-04-20T11:32:43.110198+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:03:38.796454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.107969+0000","last_change":"2024-04-20T11:32:42.110285+0000","last_active":"2024-04-20T11:32:43.107969+0000","last_peered":"2024-04-20T11:32:43.107969+0000","last_clean":"2024-04-20T11:32:43.107969+0000","last_became_active":"2024-04-20T11:32:42.109780+0000","last_became_peered":"2024-04-20T11:32:42.109780+0000","last_unstale":"2024-04-20T11:32:43.107969+0000","last_undegraded":"2024-04-20T11:32:43.107969+0000","last_fullsized":"2024-04-20T11:32:43.107969+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:28:47.958615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.404221+0000","last_change":"2024-04-20T11:32:42.112937+0000","last_active":"2024-04-20T11:32:43.404221+0000","last_peered":"2024-04-20T11:32:43.404221+0000","last_clean":"2024-04-20T11:32:43.404221+0000","last_became_active":"2024-04-20T11:32:42.112649+0000","last_became_peered":"2024-04-20T11:32:42.112649+0000","last_unstale":"2024-04-20T11:32:43.404221+0000","last_undegraded":"2024-04-20T11:32:43.404221+0000","last_fullsized":"2024-04-20T11:32:43.404221+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:49:56.916658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.406817+0000","last_change":"2024-04-20T11:32:42.105433+0000","last_active":"2024-04-20T11:32:43.406817+0000","last_peered":"2024-04-20T11:32:43.406817+0000","last_clean":"2024-04-20T11:32:43.406817+0000","last_became_active":"2024-04-20T11:32:42.104343+0000","last_became_peered":"2024-04-20T11:32:42.104343+0000","last_unstale":"2024-04-20T11:32:43.406817+0000","last_undegraded":"2024-04-20T11:32:43.406817+0000","last_fullsized":"2024-04-20T11:32:43.406817+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:12:08.390088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.405557+0000","last_change":"2024-04-20T11:32:42.107285+0000","last_active":"2024-04-20T11:32:43.405557+0000","last_peered":"2024-04-20T11:32:43.405557+0000","last_clean":"2024-04-20T11:32:43.405557+0000","last_became_active":"2024-04-20T11:32:42.106940+0000","last_became_peered":"2024-04-20T11:32:42.106940+0000","last_unstale":"2024-04-20T11:32:43.405557+0000","last_undegraded":"2024-04-20T11:32:43.405557+0000","last_fullsized":"2024-04-20T11:32:43.405557+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:03:28.197615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.403718+0000","last_change":"2024-04-20T11:32:42.106822+0000","last_active":"2024-04-20T11:32:43.403718+0000","last_peered":"2024-04-20T11:32:43.403718+0000","last_clean":"2024-04-20T11:32:43.403718+0000","last_became_active":"2024-04-20T11:32:42.106443+0000","last_became_peered":"2024-04-20T11:32:42.106443+0000","last_unstale":"2024-04-20T11:32:43.403718+0000","last_undegraded":"2024-04-20T11:32:43.403718+0000","last_fullsized":"2024-04-20T11:32:43.403718+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:14:53.766323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.404649+0000","last_change":"2024-04-20T11:32:42.093736+0000","last_active":"2024-04-20T11:32:43.404649+0000","last_peered":"2024-04-20T11:32:43.404649+0000","last_clean":"2024-04-20T11:32:43.404649+0000","last_became_active":"2024-04-20T11:32:42.093205+0000","last_became_peered":"2024-04-20T11:32:42.093205+0000","last_unstale":"2024-04-20T11:32:43.404649+0000","last_undegraded":"2024-04-20T11:32:43.404649+0000","last_fullsized":"2024-04-20T11:32:43.404649+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:08:25.164456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.406778+0000","last_change":"2024-04-20T11:32:42.089757+0000","last_active":"2024-04-20T11:32:43.406778+0000","last_peered":"2024-04-20T11:32:43.406778+0000","last_clean":"2024-04-20T11:32:43.406778+0000","last_became_active":"2024-04-20T11:32:42.089064+0000","last_became_peered":"2024-04-20T11:32:42.089064+0000","last_unstale":"2024-04-20T11:32:43.406778+0000","last_undegraded":"2024-04-20T11:32:43.406778+0000","last_fullsized":"2024-04-20T11:32:43.406778+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:07:47.911322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.408442+0000","last_change":"2024-04-20T11:32:42.107648+0000","last_active":"2024-04-20T11:32:43.408442+0000","last_peered":"2024-04-20T11:32:43.408442+0000","last_clean":"2024-04-20T11:32:43.408442+0000","last_became_active":"2024-04-20T11:32:42.107358+0000","last_became_peered":"2024-04-20T11:32:42.107358+0000","last_unstale":"2024-04-20T11:32:43.408442+0000","last_undegraded":"2024-04-20T11:32:43.408442+0000","last_fullsized":"2024-04-20T11:32:43.408442+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:34:34.592302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.110232+0000","last_change":"2024-04-20T11:32:42.118815+0000","last_active":"2024-04-20T11:32:43.110232+0000","last_peered":"2024-04-20T11:32:43.110232+0000","last_clean":"2024-04-20T11:32:43.110232+0000","last_became_active":"2024-04-20T11:32:42.114455+0000","last_became_peered":"2024-04-20T11:32:42.114455+0000","last_unstale":"2024-04-20T11:32:43.110232+0000","last_undegraded":"2024-04-20T11:32:43.110232+0000","last_fullsized":"2024-04-20T11:32:43.110232+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:37:27.505970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.406797+0000","last_change":"2024-04-20T11:32:42.111744+0000","last_active":"2024-04-20T11:32:43.406797+0000","last_peered":"2024-04-20T11:32:43.406797+0000","last_clean":"2024-04-20T11:32:43.406797+0000","last_became_active":"2024-04-20T11:32:42.110870+0000","last_became_peered":"2024-04-20T11:32:42.110870+0000","last_unstale":"2024-04-20T11:32:43.406797+0000","last_undegraded":"2024-04-20T11:32:43.406797+0000","last_fullsized":"2024-04-20T11:32:43.406797+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:07:53.171385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.107789+0000","last_change":"2024-04-20T11:32:42.109002+0000","last_active":"2024-04-20T11:32:43.107789+0000","last_peered":"2024-04-20T11:32:43.107789+0000","last_clean":"2024-04-20T11:32:43.107789+0000","last_became_active":"2024-04-20T11:32:42.106196+0000","last_became_peered":"2024-04-20T11:32:42.106196+0000","last_unstale":"2024-04-20T11:32:43.107789+0000","last_undegraded":"2024-04-20T11:32:43.107789+0000","last_fullsized":"2024-04-20T11:32:43.107789+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:26:31.889820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.404219+0000","last_change":"2024-04-20T11:32:42.110011+0000","last_active":"2024-04-20T11:32:43.404219+0000","last_peered":"2024-04-20T11:32:43.404219+0000","last_clean":"2024-04-20T11:32:43.404219+0000","last_became_active":"2024-04-20T11:32:42.109446+0000","last_became_peered":"2024-04-20T11:32:42.109446+0000","last_unstale":"2024-04-20T11:32:43.404219+0000","last_undegraded":"2024-04-20T11:32:43.404219+0000","last_fullsized":"2024-04-20T11:32:43.404219+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:46:32.134841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.105824+0000","last_change":"2024-04-20T11:32:42.107863+0000","last_active":"2024-04-20T11:32:43.105824+0000","last_peered":"2024-04-20T11:32:43.105824+0000","last_clean":"2024-04-20T11:32:43.105824+0000","last_became_active":"2024-04-20T11:32:42.107566+0000","last_became_peered":"2024-04-20T11:32:42.107566+0000","last_unstale":"2024-04-20T11:32:43.105824+0000","last_undegraded":"2024-04-20T11:32:43.105824+0000","last_fullsized":"2024-04-20T11:32:43.105824+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:30:03.622345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.404651+0000","last_change":"2024-04-20T11:32:42.109665+0000","last_active":"2024-04-20T11:32:43.404651+0000","last_peered":"2024-04-20T11:32:43.404651+0000","last_clean":"2024-04-20T11:32:43.404651+0000","last_became_active":"2024-04-20T11:32:42.108737+0000","last_became_peered":"2024-04-20T11:32:42.108737+0000","last_unstale":"2024-04-20T11:32:43.404651+0000","last_undegraded":"2024-04-20T11:32:43.404651+0000","last_fullsized":"2024-04-20T11:32:43.404651+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:54:47.268039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.407577+0000","last_change":"2024-04-20T11:32:42.111566+0000","last_active":"2024-04-20T11:32:43.407577+0000","last_peered":"2024-04-20T11:32:43.407577+0000","last_clean":"2024-04-20T11:32:43.407577+0000","last_became_active":"2024-04-20T11:32:42.111211+0000","last_became_peered":"2024-04-20T11:32:42.111211+0000","last_unstale":"2024-04-20T11:32:43.407577+0000","last_undegraded":"2024-04-20T11:32:43.407577+0000","last_fullsized":"2024-04-20T11:32:43.407577+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:34:34.592302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.404219+0000","last_change":"2024-04-20T11:32:42.109629+0000","last_active":"2024-04-20T11:32:43.404219+0000","last_peered":"2024-04-20T11:32:43.404219+0000","last_clean":"2024-04-20T11:32:43.404219+0000","last_became_active":"2024-04-20T11:32:42.109280+0000","last_became_peered":"2024-04-20T11:32:42.109280+0000","last_unstale":"2024-04-20T11:32:43.404219+0000","last_undegraded":"2024-04-20T11:32:43.404219+0000","last_fullsized":"2024-04-20T11:32:43.404219+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:41:17.239137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.105070+0000","last_change":"2024-04-20T11:32:42.101285+0000","last_active":"2024-04-20T11:32:43.105070+0000","last_peered":"2024-04-20T11:32:43.105070+0000","last_clean":"2024-04-20T11:32:43.105070+0000","last_became_active":"2024-04-20T11:32:42.100964+0000","last_became_peered":"2024-04-20T11:32:42.100964+0000","last_unstale":"2024-04-20T11:32:43.105070+0000","last_undegraded":"2024-04-20T11:32:43.105070+0000","last_fullsized":"2024-04-20T11:32:43.105070+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:02:58.915650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.404429+0000","last_change":"2024-04-20T11:32:42.100419+0000","last_active":"2024-04-20T11:32:43.404429+0000","last_peered":"2024-04-20T11:32:43.404429+0000","last_clean":"2024-04-20T11:32:43.404429+0000","last_became_active":"2024-04-20T11:32:42.099478+0000","last_became_peered":"2024-04-20T11:32:42.099478+0000","last_unstale":"2024-04-20T11:32:43.404429+0000","last_undegraded":"2024-04-20T11:32:43.404429+0000","last_fullsized":"2024-04-20T11:32:43.404429+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:04:18.751528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.404618+0000","last_change":"2024-04-20T11:32:42.108113+0000","last_active":"2024-04-20T11:32:43.404618+0000","last_peered":"2024-04-20T11:32:43.404618+0000","last_clean":"2024-04-20T11:32:43.404618+0000","last_became_active":"2024-04-20T11:32:42.107828+0000","last_became_peered":"2024-04-20T11:32:42.107828+0000","last_unstale":"2024-04-20T11:32:43.404618+0000","last_undegraded":"2024-04-20T11:32:43.404618+0000","last_fullsized":"2024-04-20T11:32:43.404618+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:39:35.303398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.109628+0000","last_change":"2024-04-20T11:32:42.103936+0000","last_active":"2024-04-20T11:32:43.109628+0000","last_peered":"2024-04-20T11:32:43.109628+0000","last_clean":"2024-04-20T11:32:43.109628+0000","last_became_active":"2024-04-20T11:32:42.103624+0000","last_became_peered":"2024-04-20T11:32:42.103624+0000","last_unstale":"2024-04-20T11:32:43.109628+0000","last_undegraded":"2024-04-20T11:32:43.109628+0000","last_fullsized":"2024-04-20T11:32:43.109628+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:41:06.829303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.406710+0000","last_change":"2024-04-20T11:32:42.105882+0000","last_active":"2024-04-20T11:32:43.406710+0000","last_peered":"2024-04-20T11:32:43.406710+0000","last_clean":"2024-04-20T11:32:43.406710+0000","last_became_active":"2024-04-20T11:32:42.105159+0000","last_became_peered":"2024-04-20T11:32:42.105159+0000","last_unstale":"2024-04-20T11:32:43.406710+0000","last_undegraded":"2024-04-20T11:32:43.406710+0000","last_fullsized":"2024-04-20T11:32:43.406710+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:18:37.443421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.403484+0000","last_change":"2024-04-20T11:32:42.105945+0000","last_active":"2024-04-20T11:32:43.403484+0000","last_peered":"2024-04-20T11:32:43.403484+0000","last_clean":"2024-04-20T11:32:43.403484+0000","last_became_active":"2024-04-20T11:32:42.104810+0000","last_became_peered":"2024-04-20T11:32:42.104810+0000","last_unstale":"2024-04-20T11:32:43.403484+0000","last_undegraded":"2024-04-20T11:32:43.403484+0000","last_fullsized":"2024-04-20T11:32:43.403484+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:29:31.317890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.110439+0000","last_change":"2024-04-20T11:32:42.111085+0000","last_active":"2024-04-20T11:32:43.110439+0000","last_peered":"2024-04-20T11:32:43.110439+0000","last_clean":"2024-04-20T11:32:43.110439+0000","last_became_active":"2024-04-20T11:32:42.110114+0000","last_became_peered":"2024-04-20T11:32:42.110114+0000","last_unstale":"2024-04-20T11:32:43.110439+0000","last_undegraded":"2024-04-20T11:32:43.110439+0000","last_fullsized":"2024-04-20T11:32:43.110439+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:56:21.525634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.107628+0000","last_change":"2024-04-20T11:32:42.124187+0000","last_active":"2024-04-20T11:32:43.107628+0000","last_peered":"2024-04-20T11:32:43.107628+0000","last_clean":"2024-04-20T11:32:43.107628+0000","last_became_active":"2024-04-20T11:32:42.123702+0000","last_became_peered":"2024-04-20T11:32:42.123702+0000","last_unstale":"2024-04-20T11:32:43.107628+0000","last_undegraded":"2024-04-20T11:32:43.107628+0000","last_fullsized":"2024-04-20T11:32:43.107628+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:31:20.713207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.404937+0000","last_change":"2024-04-20T11:32:42.108747+0000","last_active":"2024-04-20T11:32:43.404937+0000","last_peered":"2024-04-20T11:32:43.404937+0000","last_clean":"2024-04-20T11:32:43.404937+0000","last_became_active":"2024-04-20T11:32:42.108422+0000","last_became_peered":"2024-04-20T11:32:42.108422+0000","last_unstale":"2024-04-20T11:32:43.404937+0000","last_undegraded":"2024-04-20T11:32:43.404937+0000","last_fullsized":"2024-04-20T11:32:43.404937+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:50:54.488207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.407230+0000","last_change":"2024-04-20T11:32:42.104636+0000","last_active":"2024-04-20T11:32:43.407230+0000","last_peered":"2024-04-20T11:32:43.407230+0000","last_clean":"2024-04-20T11:32:43.407230+0000","last_became_active":"2024-04-20T11:32:42.103041+0000","last_became_peered":"2024-04-20T11:32:42.103041+0000","last_unstale":"2024-04-20T11:32:43.407230+0000","last_undegraded":"2024-04-20T11:32:43.407230+0000","last_fullsized":"2024-04-20T11:32:43.407230+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:02:58.915650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.109878+0000","last_change":"2024-04-20T11:32:42.105874+0000","last_active":"2024-04-20T11:32:43.109878+0000","last_peered":"2024-04-20T11:32:43.109878+0000","last_clean":"2024-04-20T11:32:43.109878+0000","last_became_active":"2024-04-20T11:32:42.104810+0000","last_became_peered":"2024-04-20T11:32:42.104810+0000","last_unstale":"2024-04-20T11:32:43.109878+0000","last_undegraded":"2024-04-20T11:32:43.109878+0000","last_fullsized":"2024-04-20T11:32:43.109878+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:30:08.274640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.109761+0000","last_change":"2024-04-20T11:32:42.129362+0000","last_active":"2024-04-20T11:32:43.109761+0000","last_peered":"2024-04-20T11:32:43.109761+0000","last_clean":"2024-04-20T11:32:43.109761+0000","last_became_active":"2024-04-20T11:32:42.128561+0000","last_became_peered":"2024-04-20T11:32:42.128561+0000","last_unstale":"2024-04-20T11:32:43.109761+0000","last_undegraded":"2024-04-20T11:32:43.109761+0000","last_fullsized":"2024-04-20T11:32:43.109761+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:27:40.289322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.406213+0000","last_change":"2024-04-20T11:32:42.092356+0000","last_active":"2024-04-20T11:32:43.406213+0000","last_peered":"2024-04-20T11:32:43.406213+0000","last_clean":"2024-04-20T11:32:43.406213+0000","last_became_active":"2024-04-20T11:32:42.091575+0000","last_became_peered":"2024-04-20T11:32:42.091575+0000","last_unstale":"2024-04-20T11:32:43.406213+0000","last_undegraded":"2024-04-20T11:32:43.406213+0000","last_fullsized":"2024-04-20T11:32:43.406213+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:02:58.915650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.404413+0000","last_change":"2024-04-20T11:32:42.106251+0000","last_active":"2024-04-20T11:32:43.404413+0000","last_peered":"2024-04-20T11:32:43.404413+0000","last_clean":"2024-04-20T11:32:43.404413+0000","last_became_active":"2024-04-20T11:32:42.105562+0000","last_became_peered":"2024-04-20T11:32:42.105562+0000","last_unstale":"2024-04-20T11:32:43.404413+0000","last_undegraded":"2024-04-20T11:32:43.404413+0000","last_fullsized":"2024-04-20T11:32:43.404413+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:54:47.268039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.407804+0000","last_change":"2024-04-20T11:32:42.109796+0000","last_active":"2024-04-20T11:32:43.407804+0000","last_peered":"2024-04-20T11:32:43.407804+0000","last_clean":"2024-04-20T11:32:43.407804+0000","last_became_active":"2024-04-20T11:32:42.109507+0000","last_became_peered":"2024-04-20T11:32:42.109507+0000","last_unstale":"2024-04-20T11:32:43.407804+0000","last_undegraded":"2024-04-20T11:32:43.407804+0000","last_fullsized":"2024-04-20T11:32:43.407804+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:31:20.713207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"82'4","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.409236+0000","last_change":"2024-04-20T11:32:41.080530+0000","last_active":"2024-04-20T11:32:43.409236+0000","last_peered":"2024-04-20T11:32:43.409236+0000","last_clean":"2024-04-20T11:32:43.409236+0000","last_became_active":"2024-04-20T11:32:41.078998+0000","last_became_peered":"2024-04-20T11:32:41.078998+0000","last_unstale":"2024-04-20T11:32:43.409236+0000","last_undegraded":"2024-04-20T11:32:43.409236+0000","last_fullsized":"2024-04-20T11:32:43.409236+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_clean_scrub_stamp":"2024-04-20T11:32:40.024004+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:03:37.791227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.408821+0000","last_change":"2024-04-20T11:32:42.104032+0000","last_active":"2024-04-20T11:32:43.408821+0000","last_peered":"2024-04-20T11:32:43.408821+0000","last_clean":"2024-04-20T11:32:43.408821+0000","last_became_active":"2024-04-20T11:32:42.103737+0000","last_became_peered":"2024-04-20T11:32:42.103737+0000","last_unstale":"2024-04-20T11:32:43.408821+0000","last_undegraded":"2024-04-20T11:32:43.408821+0000","last_fullsized":"2024-04-20T11:32:43.408821+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:31:20.713207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"82'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.408483+0000","last_change":"2024-04-20T11:32:41.073052+0000","last_active":"2024-04-20T11:32:43.408483+0000","last_peered":"2024-04-20T11:32:43.408483+0000","last_clean":"2024-04-20T11:32:43.408483+0000","last_became_active":"2024-04-20T11:32:41.072725+0000","last_became_peered":"2024-04-20T11:32:41.072725+0000","last_unstale":"2024-04-20T11:32:43.408483+0000","last_undegraded":"2024-04-20T11:32:43.408483+0000","last_fullsized":"2024-04-20T11:32:43.408483+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_clean_scrub_stamp":"2024-04-20T11:32:40.024004+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:14:42.465963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.407625+0000","last_change":"2024-04-20T11:32:42.113909+0000","last_active":"2024-04-20T11:32:43.407625+0000","last_peered":"2024-04-20T11:32:43.407625+0000","last_clean":"2024-04-20T11:32:43.407625+0000","last_became_active":"2024-04-20T11:32:42.113534+0000","last_became_peered":"2024-04-20T11:32:42.113534+0000","last_unstale":"2024-04-20T11:32:43.407625+0000","last_undegraded":"2024-04-20T11:32:43.407625+0000","last_fullsized":"2024-04-20T11:32:43.407625+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:12:08.390088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"82'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.105034+0000","last_change":"2024-04-20T11:32:41.061032+0000","last_active":"2024-04-20T11:32:43.105034+0000","last_peered":"2024-04-20T11:32:43.105034+0000","last_clean":"2024-04-20T11:32:43.105034+0000","last_became_active":"2024-04-20T11:32:41.060708+0000","last_became_peered":"2024-04-20T11:32:41.060708+0000","last_unstale":"2024-04-20T11:32:43.105034+0000","last_undegraded":"2024-04-20T11:32:43.105034+0000","last_fullsized":"2024-04-20T11:32:43.105034+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_clean_scrub_stamp":"2024-04-20T11:32:40.024004+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:07:46.906095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.107496+0000","last_change":"2024-04-20T11:32:42.113220+0000","last_active":"2024-04-20T11:32:43.107496+0000","last_peered":"2024-04-20T11:32:43.107496+0000","last_clean":"2024-04-20T11:32:43.107496+0000","last_became_active":"2024-04-20T11:32:42.112875+0000","last_became_peered":"2024-04-20T11:32:42.112875+0000","last_unstale":"2024-04-20T11:32:43.107496+0000","last_undegraded":"2024-04-20T11:32:43.107496+0000","last_fullsized":"2024-04-20T11:32:43.107496+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:34:34.592302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"82'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.104797+0000","last_change":"2024-04-20T11:32:41.067513+0000","last_active":"2024-04-20T11:32:43.104797+0000","last_peered":"2024-04-20T11:32:43.104797+0000","last_clean":"2024-04-20T11:32:43.104797+0000","last_became_active":"2024-04-20T11:32:41.066766+0000","last_became_peered":"2024-04-20T11:32:41.066766+0000","last_unstale":"2024-04-20T11:32:43.104797+0000","last_undegraded":"2024-04-20T11:32:43.104797+0000","last_fullsized":"2024-04-20T11:32:43.104797+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_clean_scrub_stamp":"2024-04-20T11:32:40.024004+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:50:31.018668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.109935+0000","last_change":"2024-04-20T11:32:42.127546+0000","last_active":"2024-04-20T11:32:43.109935+0000","last_peered":"2024-04-20T11:32:43.109935+0000","last_clean":"2024-04-20T11:32:43.109935+0000","last_became_active":"2024-04-20T11:32:42.127264+0000","last_became_peered":"2024-04-20T11:32:42.127264+0000","last_unstale":"2024-04-20T11:32:43.109935+0000","last_undegraded":"2024-04-20T11:32:43.109935+0000","last_fullsized":"2024-04-20T11:32:43.109935+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:02:32.140626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"82'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.406498+0000","last_change":"2024-04-20T11:32:41.077903+0000","last_active":"2024-04-20T11:32:43.406498+0000","last_peered":"2024-04-20T11:32:43.406498+0000","last_clean":"2024-04-20T11:32:43.406498+0000","last_became_active":"2024-04-20T11:32:41.077582+0000","last_became_peered":"2024-04-20T11:32:41.077582+0000","last_unstale":"2024-04-20T11:32:43.406498+0000","last_undegraded":"2024-04-20T11:32:43.406498+0000","last_fullsized":"2024-04-20T11:32:43.406498+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_clean_scrub_stamp":"2024-04-20T11:32:40.024004+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:56:20.520407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.405978+0000","last_change":"2024-04-20T11:32:42.097219+0000","last_active":"2024-04-20T11:32:43.405978+0000","last_peered":"2024-04-20T11:32:43.405978+0000","last_clean":"2024-04-20T11:32:43.405978+0000","last_became_active":"2024-04-20T11:32:42.096671+0000","last_became_peered":"2024-04-20T11:32:42.096671+0000","last_unstale":"2024-04-20T11:32:43.405978+0000","last_undegraded":"2024-04-20T11:32:43.405978+0000","last_fullsized":"2024-04-20T11:32:43.405978+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:07:53.171385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"82'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.404958+0000","last_change":"2024-04-20T11:32:41.073595+0000","last_active":"2024-04-20T11:32:43.404958+0000","last_peered":"2024-04-20T11:32:43.404958+0000","last_clean":"2024-04-20T11:32:43.404958+0000","last_became_active":"2024-04-20T11:32:41.072939+0000","last_became_peered":"2024-04-20T11:32:41.072939+0000","last_unstale":"2024-04-20T11:32:43.404958+0000","last_undegraded":"2024-04-20T11:32:43.404958+0000","last_fullsized":"2024-04-20T11:32:43.404958+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_clean_scrub_stamp":"2024-04-20T11:32:40.024004+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:37:26.500743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.107785+0000","last_change":"2024-04-20T11:32:42.104094+0000","last_active":"2024-04-20T11:32:43.107785+0000","last_peered":"2024-04-20T11:32:43.107785+0000","last_clean":"2024-04-20T11:32:43.107785+0000","last_became_active":"2024-04-20T11:32:42.103798+0000","last_became_peered":"2024-04-20T11:32:42.103798+0000","last_unstale":"2024-04-20T11:32:43.107785+0000","last_undegraded":"2024-04-20T11:32:43.107785+0000","last_fullsized":"2024-04-20T11:32:43.107785+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:57:39.307299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.104363+0000","last_change":"2024-04-20T11:32:41.066204+0000","last_active":"2024-04-20T11:32:43.104363+0000","last_peered":"2024-04-20T11:32:43.104363+0000","last_clean":"2024-04-20T11:32:43.104363+0000","last_became_active":"2024-04-20T11:32:41.065792+0000","last_became_peered":"2024-04-20T11:32:41.065792+0000","last_unstale":"2024-04-20T11:32:43.104363+0000","last_undegraded":"2024-04-20T11:32:43.104363+0000","last_fullsized":"2024-04-20T11:32:43.104363+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_clean_scrub_stamp":"2024-04-20T11:32:40.024004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:08:24.159228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.105627+0000","last_change":"2024-04-20T11:32:42.106147+0000","last_active":"2024-04-20T11:32:43.105627+0000","last_peered":"2024-04-20T11:32:43.105627+0000","last_clean":"2024-04-20T11:32:43.105627+0000","last_became_active":"2024-04-20T11:32:42.104108+0000","last_became_peered":"2024-04-20T11:32:42.104108+0000","last_unstale":"2024-04-20T11:32:43.105627+0000","last_undegraded":"2024-04-20T11:32:43.105627+0000","last_fullsized":"2024-04-20T11:32:43.105627+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:41:30.083314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.109524+0000","last_change":"2024-04-20T11:32:41.074941+0000","last_active":"2024-04-20T11:32:43.109524+0000","last_peered":"2024-04-20T11:32:43.109524+0000","last_clean":"2024-04-20T11:32:43.109524+0000","last_became_active":"2024-04-20T11:32:41.074655+0000","last_became_peered":"2024-04-20T11:32:41.074655+0000","last_unstale":"2024-04-20T11:32:43.109524+0000","last_undegraded":"2024-04-20T11:32:43.109524+0000","last_fullsized":"2024-04-20T11:32:43.109524+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_clean_scrub_stamp":"2024-04-20T11:32:40.024004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:46:31.129613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.106308+0000","last_change":"2024-04-20T11:32:42.106249+0000","last_active":"2024-04-20T11:32:43.106308+0000","last_peered":"2024-04-20T11:32:43.106308+0000","last_clean":"2024-04-20T11:32:43.106308+0000","last_became_active":"2024-04-20T11:32:42.105953+0000","last_became_peered":"2024-04-20T11:32:42.105953+0000","last_unstale":"2024-04-20T11:32:43.106308+0000","last_undegraded":"2024-04-20T11:32:43.106308+0000","last_fullsized":"2024-04-20T11:32:43.106308+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:19:40.411555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.409421+0000","last_change":"2024-04-20T11:32:41.077901+0000","last_active":"2024-04-20T11:32:43.409421+0000","last_peered":"2024-04-20T11:32:43.409421+0000","last_clean":"2024-04-20T11:32:43.409421+0000","last_became_active":"2024-04-20T11:32:41.077332+0000","last_became_peered":"2024-04-20T11:32:41.077332+0000","last_unstale":"2024-04-20T11:32:43.409421+0000","last_undegraded":"2024-04-20T11:32:43.409421+0000","last_fullsized":"2024-04-20T11:32:43.409421+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_clean_scrub_stamp":"2024-04-20T11:32:40.024004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:56:20.520407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.407013+0000","last_change":"2024-04-20T11:32:42.096170+0000","last_active":"2024-04-20T11:32:43.407013+0000","last_peered":"2024-04-20T11:32:43.407013+0000","last_clean":"2024-04-20T11:32:43.407013+0000","last_became_active":"2024-04-20T11:32:42.095392+0000","last_became_peered":"2024-04-20T11:32:42.095392+0000","last_unstale":"2024-04-20T11:32:43.407013+0000","last_undegraded":"2024-04-20T11:32:43.407013+0000","last_fullsized":"2024-04-20T11:32:43.407013+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:35:05.919911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.408383+0000","last_change":"2024-04-20T11:32:41.080580+0000","last_active":"2024-04-20T11:32:43.408383+0000","last_peered":"2024-04-20T11:32:43.408383+0000","last_clean":"2024-04-20T11:32:43.408383+0000","last_became_active":"2024-04-20T11:32:41.080248+0000","last_became_peered":"2024-04-20T11:32:41.080248+0000","last_unstale":"2024-04-20T11:32:43.408383+0000","last_undegraded":"2024-04-20T11:32:43.408383+0000","last_fullsized":"2024-04-20T11:32:43.408383+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_clean_scrub_stamp":"2024-04-20T11:32:40.024004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:38:47.685872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.104814+0000","last_change":"2024-04-20T11:32:42.109503+0000","last_active":"2024-04-20T11:32:43.104814+0000","last_peered":"2024-04-20T11:32:43.104814+0000","last_clean":"2024-04-20T11:32:43.104814+0000","last_became_active":"2024-04-20T11:32:42.108144+0000","last_became_peered":"2024-04-20T11:32:42.108144+0000","last_unstale":"2024-04-20T11:32:43.104814+0000","last_undegraded":"2024-04-20T11:32:43.104814+0000","last_fullsized":"2024-04-20T11:32:43.104814+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:07:53.171385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"82'3","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.405229+0000","last_change":"2024-04-20T11:32:41.080065+0000","last_active":"2024-04-20T11:32:43.405229+0000","last_peered":"2024-04-20T11:32:43.405229+0000","last_clean":"2024-04-20T11:32:43.405229+0000","last_became_active":"2024-04-20T11:32:41.079784+0000","last_became_peered":"2024-04-20T11:32:41.079784+0000","last_unstale":"2024-04-20T11:32:43.405229+0000","last_undegraded":"2024-04-20T11:32:43.405229+0000","last_fullsized":"2024-04-20T11:32:43.405229+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_clean_scrub_stamp":"2024-04-20T11:32:40.024004+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:37:26.500743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.405387+0000","last_change":"2024-04-20T11:32:42.100953+0000","last_active":"2024-04-20T11:32:43.405387+0000","last_peered":"2024-04-20T11:32:43.405387+0000","last_clean":"2024-04-20T11:32:43.405387+0000","last_became_active":"2024-04-20T11:32:42.099940+0000","last_became_peered":"2024-04-20T11:32:42.099940+0000","last_unstale":"2024-04-20T11:32:43.405387+0000","last_undegraded":"2024-04-20T11:32:43.405387+0000","last_fullsized":"2024-04-20T11:32:43.405387+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:18:37.443421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"82'6","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.405747+0000","last_change":"2024-04-20T11:32:41.056012+0000","last_active":"2024-04-20T11:32:43.405747+0000","last_peered":"2024-04-20T11:32:43.405747+0000","last_clean":"2024-04-20T11:32:43.405747+0000","last_became_active":"2024-04-20T11:32:41.054375+0000","last_became_peered":"2024-04-20T11:32:41.054375+0000","last_unstale":"2024-04-20T11:32:43.405747+0000","last_undegraded":"2024-04-20T11:32:43.405747+0000","last_fullsized":"2024-04-20T11:32:43.405747+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_clean_scrub_stamp":"2024-04-20T11:32:40.024004+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:02:31.135399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.105204+0000","last_change":"2024-04-20T11:32:42.091011+0000","last_active":"2024-04-20T11:32:43.105204+0000","last_peered":"2024-04-20T11:32:43.105204+0000","last_clean":"2024-04-20T11:32:43.105204+0000","last_became_active":"2024-04-20T11:32:42.090401+0000","last_became_peered":"2024-04-20T11:32:42.090401+0000","last_unstale":"2024-04-20T11:32:43.105204+0000","last_undegraded":"2024-04-20T11:32:43.105204+0000","last_fullsized":"2024-04-20T11:32:43.105204+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:22:54.913135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"82'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.109995+0000","last_change":"2024-04-20T11:32:41.068858+0000","last_active":"2024-04-20T11:32:43.109995+0000","last_peered":"2024-04-20T11:32:43.109995+0000","last_clean":"2024-04-20T11:32:43.109995+0000","last_became_active":"2024-04-20T11:32:41.068627+0000","last_became_peered":"2024-04-20T11:32:41.068627+0000","last_unstale":"2024-04-20T11:32:43.109995+0000","last_undegraded":"2024-04-20T11:32:43.109995+0000","last_fullsized":"2024-04-20T11:32:43.109995+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_clean_scrub_stamp":"2024-04-20T11:32:40.024004+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:41:16.233910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.108007+0000","last_change":"2024-04-20T11:32:42.105192+0000","last_active":"2024-04-20T11:32:43.108007+0000","last_peered":"2024-04-20T11:32:43.108007+0000","last_clean":"2024-04-20T11:32:43.108007+0000","last_became_active":"2024-04-20T11:32:42.104854+0000","last_became_peered":"2024-04-20T11:32:42.104854+0000","last_unstale":"2024-04-20T11:32:43.108007+0000","last_undegraded":"2024-04-20T11:32:43.108007+0000","last_fullsized":"2024-04-20T11:32:43.108007+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:27:30.418673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.105893+0000","last_change":"2024-04-20T11:32:41.063060+0000","last_active":"2024-04-20T11:32:43.105893+0000","last_peered":"2024-04-20T11:32:43.105893+0000","last_clean":"2024-04-20T11:32:43.105893+0000","last_became_active":"2024-04-20T11:32:41.062056+0000","last_became_peered":"2024-04-20T11:32:41.062056+0000","last_unstale":"2024-04-20T11:32:43.105893+0000","last_undegraded":"2024-04-20T11:32:43.105893+0000","last_fullsized":"2024-04-20T11:32:43.105893+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_clean_scrub_stamp":"2024-04-20T11:32:40.024004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:09:31.855425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.109406+0000","last_change":"2024-04-20T11:32:42.127563+0000","last_active":"2024-04-20T11:32:43.109406+0000","last_peered":"2024-04-20T11:32:43.109406+0000","last_clean":"2024-04-20T11:32:43.109406+0000","last_became_active":"2024-04-20T11:32:42.127289+0000","last_became_peered":"2024-04-20T11:32:42.127289+0000","last_unstale":"2024-04-20T11:32:43.109406+0000","last_undegraded":"2024-04-20T11:32:43.109406+0000","last_fullsized":"2024-04-20T11:32:43.109406+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:47:04.528964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.405647+0000","last_change":"2024-04-20T11:32:41.076233+0000","last_active":"2024-04-20T11:32:43.405647+0000","last_peered":"2024-04-20T11:32:43.405647+0000","last_clean":"2024-04-20T11:32:43.405647+0000","last_became_active":"2024-04-20T11:32:41.075768+0000","last_became_peered":"2024-04-20T11:32:41.075768+0000","last_unstale":"2024-04-20T11:32:43.405647+0000","last_undegraded":"2024-04-20T11:32:43.405647+0000","last_fullsized":"2024-04-20T11:32:43.405647+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_clean_scrub_stamp":"2024-04-20T11:32:40.024004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:03:37.791227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.109696+0000","last_change":"2024-04-20T11:32:42.120788+0000","last_active":"2024-04-20T11:32:43.109696+0000","last_peered":"2024-04-20T11:32:43.109696+0000","last_clean":"2024-04-20T11:32:43.109696+0000","last_became_active":"2024-04-20T11:32:42.120507+0000","last_became_peered":"2024-04-20T11:32:42.120507+0000","last_unstale":"2024-04-20T11:32:43.109696+0000","last_undegraded":"2024-04-20T11:32:43.109696+0000","last_fullsized":"2024-04-20T11:32:43.109696+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:55:45.744229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"82'8","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.104322+0000","last_change":"2024-04-20T11:32:41.072752+0000","last_active":"2024-04-20T11:32:43.104322+0000","last_peered":"2024-04-20T11:32:43.104322+0000","last_clean":"2024-04-20T11:32:43.104322+0000","last_became_active":"2024-04-20T11:32:41.072495+0000","last_became_peered":"2024-04-20T11:32:41.072495+0000","last_unstale":"2024-04-20T11:32:43.104322+0000","last_undegraded":"2024-04-20T11:32:43.104322+0000","last_fullsized":"2024-04-20T11:32:43.104322+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_clean_scrub_stamp":"2024-04-20T11:32:40.024004+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:54:50.982481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.107959+0000","last_change":"2024-04-20T11:32:42.101821+0000","last_active":"2024-04-20T11:32:43.107959+0000","last_peered":"2024-04-20T11:32:43.107959+0000","last_clean":"2024-04-20T11:32:43.107959+0000","last_became_active":"2024-04-20T11:32:42.101485+0000","last_became_peered":"2024-04-20T11:32:42.101485+0000","last_unstale":"2024-04-20T11:32:43.107959+0000","last_undegraded":"2024-04-20T11:32:43.107959+0000","last_fullsized":"2024-04-20T11:32:43.107959+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:21:31.149890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"82'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.409539+0000","last_change":"2024-04-20T11:32:41.074739+0000","last_active":"2024-04-20T11:32:43.409539+0000","last_peered":"2024-04-20T11:32:43.409539+0000","last_clean":"2024-04-20T11:32:43.409539+0000","last_became_active":"2024-04-20T11:32:41.074479+0000","last_became_peered":"2024-04-20T11:32:41.074479+0000","last_unstale":"2024-04-20T11:32:43.409539+0000","last_undegraded":"2024-04-20T11:32:43.409539+0000","last_fullsized":"2024-04-20T11:32:43.409539+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_clean_scrub_stamp":"2024-04-20T11:32:40.024004+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:35:37.812862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.409241+0000","last_change":"2024-04-20T11:32:42.105637+0000","last_active":"2024-04-20T11:32:43.409241+0000","last_peered":"2024-04-20T11:32:43.409241+0000","last_clean":"2024-04-20T11:32:43.409241+0000","last_became_active":"2024-04-20T11:32:42.103463+0000","last_became_peered":"2024-04-20T11:32:42.103463+0000","last_unstale":"2024-04-20T11:32:43.409241+0000","last_undegraded":"2024-04-20T11:32:43.409241+0000","last_fullsized":"2024-04-20T11:32:43.409241+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:57:39.307299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"82'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.104824+0000","last_change":"2024-04-20T11:32:41.080416+0000","last_active":"2024-04-20T11:32:43.104824+0000","last_peered":"2024-04-20T11:32:43.104824+0000","last_clean":"2024-04-20T11:32:43.104824+0000","last_became_active":"2024-04-20T11:32:41.080175+0000","last_became_peered":"2024-04-20T11:32:41.080175+0000","last_unstale":"2024-04-20T11:32:43.104824+0000","last_undegraded":"2024-04-20T11:32:43.104824+0000","last_fullsized":"2024-04-20T11:32:43.104824+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_clean_scrub_stamp":"2024-04-20T11:32:40.024004+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:35:04.914684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.407736+0000","last_change":"2024-04-20T11:32:42.106196+0000","last_active":"2024-04-20T11:32:43.407736+0000","last_peered":"2024-04-20T11:32:43.407736+0000","last_clean":"2024-04-20T11:32:43.407736+0000","last_became_active":"2024-04-20T11:32:42.105887+0000","last_became_peered":"2024-04-20T11:32:42.105887+0000","last_unstale":"2024-04-20T11:32:43.407736+0000","last_undegraded":"2024-04-20T11:32:43.407736+0000","last_fullsized":"2024-04-20T11:32:43.407736+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:26:31.889820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.104912+0000","last_change":"2024-04-20T11:32:41.067520+0000","last_active":"2024-04-20T11:32:43.104912+0000","last_peered":"2024-04-20T11:32:43.104912+0000","last_clean":"2024-04-20T11:32:43.104912+0000","last_became_active":"2024-04-20T11:32:41.067072+0000","last_became_peered":"2024-04-20T11:32:41.067072+0000","last_unstale":"2024-04-20T11:32:43.104912+0000","last_undegraded":"2024-04-20T11:32:43.104912+0000","last_fullsized":"2024-04-20T11:32:43.104912+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_clean_scrub_stamp":"2024-04-20T11:32:40.024004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:38:22.319723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.404210+0000","last_change":"2024-04-20T11:32:42.112898+0000","last_active":"2024-04-20T11:32:43.404210+0000","last_peered":"2024-04-20T11:32:43.404210+0000","last_clean":"2024-04-20T11:32:43.404210+0000","last_became_active":"2024-04-20T11:32:42.112532+0000","last_became_peered":"2024-04-20T11:32:42.112532+0000","last_unstale":"2024-04-20T11:32:43.404210+0000","last_undegraded":"2024-04-20T11:32:43.404210+0000","last_fullsized":"2024-04-20T11:32:43.404210+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:02:39.189706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"82'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.105981+0000","last_change":"2024-04-20T11:32:41.046547+0000","last_active":"2024-04-20T11:32:43.105981+0000","last_peered":"2024-04-20T11:32:43.105981+0000","last_clean":"2024-04-20T11:32:43.105981+0000","last_became_active":"2024-04-20T11:32:41.045500+0000","last_became_peered":"2024-04-20T11:32:41.045500+0000","last_unstale":"2024-04-20T11:32:43.105981+0000","last_undegraded":"2024-04-20T11:32:43.105981+0000","last_fullsized":"2024-04-20T11:32:43.105981+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_clean_scrub_stamp":"2024-04-20T11:32:40.024004+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:33:33.430126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.405128+0000","last_change":"2024-04-20T11:32:42.104740+0000","last_active":"2024-04-20T11:32:43.405128+0000","last_peered":"2024-04-20T11:32:43.405128+0000","last_clean":"2024-04-20T11:32:43.405128+0000","last_became_active":"2024-04-20T11:32:42.102706+0000","last_became_peered":"2024-04-20T11:32:42.102706+0000","last_unstale":"2024-04-20T11:32:43.405128+0000","last_undegraded":"2024-04-20T11:32:43.405128+0000","last_fullsized":"2024-04-20T11:32:43.405128+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:50:32.023896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.404456+0000","last_change":"2024-04-20T11:32:41.066182+0000","last_active":"2024-04-20T11:32:43.404456+0000","last_peered":"2024-04-20T11:32:43.404456+0000","last_clean":"2024-04-20T11:32:43.404456+0000","last_became_active":"2024-04-20T11:32:41.065918+0000","last_became_peered":"2024-04-20T11:32:41.065918+0000","last_unstale":"2024-04-20T11:32:43.404456+0000","last_undegraded":"2024-04-20T11:32:43.404456+0000","last_fullsized":"2024-04-20T11:32:43.404456+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_clean_scrub_stamp":"2024-04-20T11:32:40.024004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:13:15.784362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.403348+0000","last_change":"2024-04-20T11:32:42.090868+0000","last_active":"2024-04-20T11:32:43.403348+0000","last_peered":"2024-04-20T11:32:43.403348+0000","last_clean":"2024-04-20T11:32:43.403348+0000","last_became_active":"2024-04-20T11:32:42.089885+0000","last_became_peered":"2024-04-20T11:32:42.089885+0000","last_unstale":"2024-04-20T11:32:43.403348+0000","last_undegraded":"2024-04-20T11:32:43.403348+0000","last_fullsized":"2024-04-20T11:32:43.403348+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:30:33.256438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.403688+0000","last_change":"2024-04-20T11:32:41.073570+0000","last_active":"2024-04-20T11:32:43.403688+0000","last_peered":"2024-04-20T11:32:43.403688+0000","last_clean":"2024-04-20T11:32:43.403688+0000","last_became_active":"2024-04-20T11:32:41.073282+0000","last_became_peered":"2024-04-20T11:32:41.073282+0000","last_unstale":"2024-04-20T11:32:43.403688+0000","last_undegraded":"2024-04-20T11:32:43.403688+0000","last_fullsized":"2024-04-20T11:32:43.403688+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_clean_scrub_stamp":"2024-04-20T11:32:40.024004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:21:41.469361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.404439+0000","last_change":"2024-04-20T11:32:42.081070+0000","last_active":"2024-04-20T11:32:43.404439+0000","last_peered":"2024-04-20T11:32:43.404439+0000","last_clean":"2024-04-20T11:32:43.404439+0000","last_became_active":"2024-04-20T11:32:42.080184+0000","last_became_peered":"2024-04-20T11:32:42.080184+0000","last_unstale":"2024-04-20T11:32:43.404439+0000","last_undegraded":"2024-04-20T11:32:43.404439+0000","last_fullsized":"2024-04-20T11:32:43.404439+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:54:51.987709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.105358+0000","last_change":"2024-04-20T11:32:41.046250+0000","last_active":"2024-04-20T11:32:43.105358+0000","last_peered":"2024-04-20T11:32:43.105358+0000","last_clean":"2024-04-20T11:32:43.105358+0000","last_became_active":"2024-04-20T11:32:41.045906+0000","last_became_peered":"2024-04-20T11:32:41.045906+0000","last_unstale":"2024-04-20T11:32:43.105358+0000","last_undegraded":"2024-04-20T11:32:43.105358+0000","last_fullsized":"2024-04-20T11:32:43.105358+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_clean_scrub_stamp":"2024-04-20T11:32:40.024004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:52:27.911885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.408212+0000","last_change":"2024-04-20T11:32:42.086957+0000","last_active":"2024-04-20T11:32:43.408212+0000","last_peered":"2024-04-20T11:32:43.408212+0000","last_clean":"2024-04-20T11:32:43.408212+0000","last_became_active":"2024-04-20T11:32:42.086249+0000","last_became_peered":"2024-04-20T11:32:42.086249+0000","last_unstale":"2024-04-20T11:32:43.408212+0000","last_undegraded":"2024-04-20T11:32:43.408212+0000","last_fullsized":"2024-04-20T11:32:43.408212+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:57:39.307299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"82'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.110474+0000","last_change":"2024-04-20T11:32:41.071485+0000","last_active":"2024-04-20T11:32:43.110474+0000","last_peered":"2024-04-20T11:32:43.110474+0000","last_clean":"2024-04-20T11:32:43.110474+0000","last_became_active":"2024-04-20T11:32:41.071233+0000","last_became_peered":"2024-04-20T11:32:41.071233+0000","last_unstale":"2024-04-20T11:32:43.110474+0000","last_undegraded":"2024-04-20T11:32:43.110474+0000","last_fullsized":"2024-04-20T11:32:43.110474+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_clean_scrub_stamp":"2024-04-20T11:32:40.024004+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:49:55.911430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.108011+0000","last_change":"2024-04-20T11:32:42.110626+0000","last_active":"2024-04-20T11:32:43.108011+0000","last_peered":"2024-04-20T11:32:43.108011+0000","last_clean":"2024-04-20T11:32:43.108011+0000","last_became_active":"2024-04-20T11:32:42.110297+0000","last_became_peered":"2024-04-20T11:32:42.110297+0000","last_unstale":"2024-04-20T11:32:43.108011+0000","last_undegraded":"2024-04-20T11:32:43.108011+0000","last_fullsized":"2024-04-20T11:32:43.108011+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:52:28.917113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"83'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.112522+0000","last_change":"2024-04-20T11:32:41.070454+0000","last_active":"2024-04-20T11:32:43.112522+0000","last_peered":"2024-04-20T11:32:43.112522+0000","last_clean":"2024-04-20T11:32:43.112522+0000","last_became_active":"2024-04-20T11:32:41.070199+0000","last_became_peered":"2024-04-20T11:32:41.070199+0000","last_unstale":"2024-04-20T11:32:43.112522+0000","last_undegraded":"2024-04-20T11:32:43.112522+0000","last_fullsized":"2024-04-20T11:32:43.112522+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_clean_scrub_stamp":"2024-04-20T11:32:40.024004+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:02:38.184479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.407927+0000","last_change":"2024-04-20T11:32:42.104955+0000","last_active":"2024-04-20T11:32:43.407927+0000","last_peered":"2024-04-20T11:32:43.407927+0000","last_clean":"2024-04-20T11:32:43.407927+0000","last_became_active":"2024-04-20T11:32:42.104676+0000","last_became_peered":"2024-04-20T11:32:42.104676+0000","last_unstale":"2024-04-20T11:32:43.407927+0000","last_undegraded":"2024-04-20T11:32:43.407927+0000","last_fullsized":"2024-04-20T11:32:43.407927+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:22:54.913135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.404946+0000","last_change":"2024-04-20T11:32:41.078591+0000","last_active":"2024-04-20T11:32:43.404946+0000","last_peered":"2024-04-20T11:32:43.404946+0000","last_clean":"2024-04-20T11:32:43.404946+0000","last_became_active":"2024-04-20T11:32:41.078296+0000","last_became_peered":"2024-04-20T11:32:41.078296+0000","last_unstale":"2024-04-20T11:32:43.404946+0000","last_undegraded":"2024-04-20T11:32:43.404946+0000","last_fullsized":"2024-04-20T11:32:43.404946+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_clean_scrub_stamp":"2024-04-20T11:32:40.024004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:02:31.135399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.407332+0000","last_change":"2024-04-20T11:32:42.098341+0000","last_active":"2024-04-20T11:32:43.407332+0000","last_peered":"2024-04-20T11:32:43.407332+0000","last_clean":"2024-04-20T11:32:43.407332+0000","last_became_active":"2024-04-20T11:32:42.097858+0000","last_became_peered":"2024-04-20T11:32:42.097858+0000","last_unstale":"2024-04-20T11:32:43.407332+0000","last_undegraded":"2024-04-20T11:32:43.407332+0000","last_fullsized":"2024-04-20T11:32:43.407332+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:22:54.913135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.407890+0000","last_change":"2024-04-20T11:32:41.077912+0000","last_active":"2024-04-20T11:32:43.407890+0000","last_peered":"2024-04-20T11:32:43.407890+0000","last_clean":"2024-04-20T11:32:43.407890+0000","last_became_active":"2024-04-20T11:32:41.077465+0000","last_became_peered":"2024-04-20T11:32:41.077465+0000","last_unstale":"2024-04-20T11:32:43.407890+0000","last_undegraded":"2024-04-20T11:32:43.407890+0000","last_fullsized":"2024-04-20T11:32:43.407890+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_clean_scrub_stamp":"2024-04-20T11:32:40.024004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:14:42.465963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.405966+0000","last_change":"2024-04-20T11:32:42.104767+0000","last_active":"2024-04-20T11:32:43.405966+0000","last_peered":"2024-04-20T11:32:43.405966+0000","last_clean":"2024-04-20T11:32:43.405966+0000","last_became_active":"2024-04-20T11:32:42.104440+0000","last_became_peered":"2024-04-20T11:32:42.104440+0000","last_unstale":"2024-04-20T11:32:43.405966+0000","last_undegraded":"2024-04-20T11:32:43.405966+0000","last_fullsized":"2024-04-20T11:32:43.405966+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:38:23.324951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.104809+0000","last_change":"2024-04-20T11:32:41.071601+0000","last_active":"2024-04-20T11:32:43.104809+0000","last_peered":"2024-04-20T11:32:43.104809+0000","last_clean":"2024-04-20T11:32:43.104809+0000","last_became_active":"2024-04-20T11:32:41.071360+0000","last_became_peered":"2024-04-20T11:32:41.071360+0000","last_unstale":"2024-04-20T11:32:43.104809+0000","last_undegraded":"2024-04-20T11:32:43.104809+0000","last_fullsized":"2024-04-20T11:32:43.104809+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_clean_scrub_stamp":"2024-04-20T11:32:40.024004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:03:27.192387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.105165+0000","last_change":"2024-04-20T11:32:42.089814+0000","last_active":"2024-04-20T11:32:43.105165+0000","last_peered":"2024-04-20T11:32:43.105165+0000","last_clean":"2024-04-20T11:32:43.105165+0000","last_became_active":"2024-04-20T11:32:42.089312+0000","last_became_peered":"2024-04-20T11:32:42.089312+0000","last_unstale":"2024-04-20T11:32:43.105165+0000","last_undegraded":"2024-04-20T11:32:43.105165+0000","last_fullsized":"2024-04-20T11:32:43.105165+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:12:08.390088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.405234+0000","last_change":"2024-04-20T11:32:41.073930+0000","last_active":"2024-04-20T11:32:43.405234+0000","last_peered":"2024-04-20T11:32:43.405234+0000","last_clean":"2024-04-20T11:32:43.405234+0000","last_became_active":"2024-04-20T11:32:41.073268+0000","last_became_peered":"2024-04-20T11:32:41.073268+0000","last_unstale":"2024-04-20T11:32:43.405234+0000","last_undegraded":"2024-04-20T11:32:43.405234+0000","last_fullsized":"2024-04-20T11:32:43.405234+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_clean_scrub_stamp":"2024-04-20T11:32:40.024004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:03:37.791227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.107074+0000","last_change":"2024-04-20T11:32:42.113272+0000","last_active":"2024-04-20T11:32:43.107074+0000","last_peered":"2024-04-20T11:32:43.107074+0000","last_clean":"2024-04-20T11:32:43.107074+0000","last_became_active":"2024-04-20T11:32:42.112914+0000","last_became_peered":"2024-04-20T11:32:42.112914+0000","last_unstale":"2024-04-20T11:32:43.107074+0000","last_undegraded":"2024-04-20T11:32:43.107074+0000","last_fullsized":"2024-04-20T11:32:43.107074+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:22:54.913135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.406271+0000","last_change":"2024-04-20T11:32:41.072370+0000","last_active":"2024-04-20T11:32:43.406271+0000","last_peered":"2024-04-20T11:32:43.406271+0000","last_clean":"2024-04-20T11:32:43.406271+0000","last_became_active":"2024-04-20T11:32:41.071963+0000","last_became_peered":"2024-04-20T11:32:41.071963+0000","last_unstale":"2024-04-20T11:32:43.406271+0000","last_undegraded":"2024-04-20T11:32:43.406271+0000","last_fullsized":"2024-04-20T11:32:43.406271+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_clean_scrub_stamp":"2024-04-20T11:32:40.024004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:30:07.269413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.405846+0000","last_change":"2024-04-20T11:32:42.102348+0000","last_active":"2024-04-20T11:32:43.405846+0000","last_peered":"2024-04-20T11:32:43.405846+0000","last_clean":"2024-04-20T11:32:43.405846+0000","last_became_active":"2024-04-20T11:32:42.101914+0000","last_became_peered":"2024-04-20T11:32:42.101914+0000","last_unstale":"2024-04-20T11:32:43.405846+0000","last_undegraded":"2024-04-20T11:32:43.405846+0000","last_fullsized":"2024-04-20T11:32:43.405846+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:50:54.488207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.405034+0000","last_change":"2024-04-20T11:32:41.074508+0000","last_active":"2024-04-20T11:32:43.405034+0000","last_peered":"2024-04-20T11:32:43.405034+0000","last_clean":"2024-04-20T11:32:43.405034+0000","last_became_active":"2024-04-20T11:32:41.074194+0000","last_became_peered":"2024-04-20T11:32:41.074194+0000","last_unstale":"2024-04-20T11:32:43.405034+0000","last_undegraded":"2024-04-20T11:32:43.405034+0000","last_fullsized":"2024-04-20T11:32:43.405034+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_clean_scrub_stamp":"2024-04-20T11:32:40.024004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:02:31.135399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.110424+0000","last_change":"2024-04-20T11:32:42.107164+0000","last_active":"2024-04-20T11:32:43.110424+0000","last_peered":"2024-04-20T11:32:43.110424+0000","last_clean":"2024-04-20T11:32:43.110424+0000","last_became_active":"2024-04-20T11:32:42.106906+0000","last_became_peered":"2024-04-20T11:32:42.106906+0000","last_unstale":"2024-04-20T11:32:43.110424+0000","last_undegraded":"2024-04-20T11:32:43.110424+0000","last_fullsized":"2024-04-20T11:32:43.110424+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:14:43.471190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"82'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.108004+0000","last_change":"2024-04-20T11:32:41.079526+0000","last_active":"2024-04-20T11:32:43.108004+0000","last_peered":"2024-04-20T11:32:43.108004+0000","last_clean":"2024-04-20T11:32:43.108004+0000","last_became_active":"2024-04-20T11:32:41.079233+0000","last_became_peered":"2024-04-20T11:32:41.079233+0000","last_unstale":"2024-04-20T11:32:43.108004+0000","last_undegraded":"2024-04-20T11:32:43.108004+0000","last_fullsized":"2024-04-20T11:32:43.108004+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_clean_scrub_stamp":"2024-04-20T11:32:40.024004+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:38:22.319723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"72'128","reported_seq":322,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.107228+0000","last_change":"2024-04-20T11:26:48.661897+0000","last_active":"2024-04-20T11:32:43.107228+0000","last_peered":"2024-04-20T11:32:43.107228+0000","last_clean":"2024-04-20T11:32:43.107228+0000","last_became_active":"2024-04-20T11:26:48.656535+0000","last_became_peered":"2024-04-20T11:26:48.656535+0000","last_unstale":"2024-04-20T11:32:43.107228+0000","last_undegraded":"2024-04-20T11:32:43.107228+0000","last_fullsized":"2024-04-20T11:32:43.107228+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:47.543004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:47.543004+0000","last_clean_scrub_stamp":"2024-04-20T11:26:47.543004+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:58:35.527105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574925,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32760,"kb_used_data":1580,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339080,"statfs":{"total":96636764160,"available":96603217920,"internally_reserved":0,"allocated":1617920,"data_stored":1028369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574922,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32176,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339664,"statfs":{"total":96636764160,"available":96603815936,"internally_reserved":0,"allocated":1019904,"data_stored":432325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574924,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32188,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339652,"statfs":{"total":96636764160,"available":96603803648,"internally_reserved":0,"allocated":1024000,"data_stored":433265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574922,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32172,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339668,"statfs":{"total":96636764160,"available":96603820032,"internally_reserved":0,"allocated":1007616,"data_stored":431631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574923,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32184,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339656,"statfs":{"total":96636764160,"available":96603807744,"internally_reserved":0,"allocated":1019904,"data_stored":433243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574924,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32168,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339672,"statfs":{"total":96636764160,"available":96603824128,"internally_reserved":0,"allocated":1011712,"data_stored":432213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574922,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32188,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339652,"statfs":{"total":96636764160,"available":96603803648,"internally_reserved":0,"allocated":1032192,"data_stored":438125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574923,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32760,"kb_used_data":1580,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339080,"statfs":{"total":96636764160,"available":96603217920,"internally_reserved":0,"allocated":1617920,"data_stored":1028333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T11:32:46.513 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T11:32:46.751 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.751+0000 7f24eb200640 1 Processor -- start 2024-04-20T11:32:46.751 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.751+0000 7f24eb200640 1 -- start start 2024-04-20T11:32:46.752 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.752+0000 7f24eb200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f24e415fcc0 0x7f24e40d4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:46.752 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.752+0000 7f24eb200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f24e41603d0 0x7f24e40b5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:46.752 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.753+0000 7f24eb200640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24e40d3350 con 0x7f24e40b5480 2024-04-20T11:32:46.752 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.753+0000 7f24eb200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f24e40d3630 con 0x7f24e415fcc0 2024-04-20T11:32:46.752 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.753+0000 7f24eb200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f24e40d3910 con 0x7f24e41603d0 2024-04-20T11:32:46.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.753+0000 7f24ea800640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f24e40b5480 0x7f24e41624d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:41146/0 (socket says 172.21.15.152:41146) 2024-04-20T11:32:46.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.753+0000 7f24ea800640 1 -- 172.21.15.152:0/2936257377 learned_addr learned my addr 172.21.15.152:0/2936257377 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:32:46.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.753+0000 7f24e9400640 1 --2- 172.21.15.152:0/2936257377 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f24e41603d0 0x7f24e40b5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:46.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.753+0000 7f24e9e00640 1 --2- 172.21.15.152:0/2936257377 >> v2:172.21.15.175:3300/0 conn(0x7f24e415fcc0 0x7f24e40d4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:46.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.753+0000 7f24e8a00640 1 -- 172.21.15.152:0/2936257377 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3843331750 0 0) 0x7f24e40d3350 con 0x7f24e40b5480 2024-04-20T11:32:46.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.753+0000 7f24e8a00640 1 -- 172.21.15.152:0/2936257377 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f24cc003520 con 0x7f24e40b5480 2024-04-20T11:32:46.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.754+0000 7f24e8a00640 1 -- 172.21.15.152:0/2936257377 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1749082160 0 0) 0x7f24cc003520 con 0x7f24e40b5480 2024-04-20T11:32:46.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.754+0000 7f24e8a00640 1 -- 172.21.15.152:0/2936257377 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f24e41603d0 msgr2=0x7f24e40b5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:46.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.754+0000 7f24e8a00640 1 --2- 172.21.15.152:0/2936257377 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f24e41603d0 0x7f24e40b5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:46.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.754+0000 7f24e8a00640 1 -- 172.21.15.152:0/2936257377 >> v2:172.21.15.175:3300/0 conn(0x7f24e415fcc0 msgr2=0x7f24e40d4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:46.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.754+0000 7f24e8a00640 1 --2- 172.21.15.152:0/2936257377 >> v2:172.21.15.175:3300/0 conn(0x7f24e415fcc0 0x7f24e40d4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:46.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.754+0000 7f24e8a00640 1 -- 172.21.15.152:0/2936257377 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24e40ced20 con 0x7f24e40b5480 2024-04-20T11:32:46.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.754+0000 7f24e8a00640 1 -- 172.21.15.152:0/2936257377 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f24e00019f0 con 0x7f24e40b5480 2024-04-20T11:32:46.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.754+0000 7f24e8a00640 1 -- 172.21.15.152:0/2936257377 >> v1:172.21.15.152:6789/0 conn(0x7f24e40b5480 legacy=0x7f24e41624d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:46.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.754+0000 7f24e8a00640 1 --2- 172.21.15.152:0/2936257377 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f24e41603d0 0x7f24cc0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:46.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.754+0000 7f24e8a00640 1 --2- 172.21.15.152:0/2936257377 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f24cc003f30 0x7f24cc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:46.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.754+0000 7f24e8a00640 1 --2- 172.21.15.152:0/2936257377 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f24cc004d40 0x7f24cc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:46.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.754+0000 7f24e8a00640 1 -- 172.21.15.152:0/2936257377 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f24e40d3630 con 0x7f24cc003f30 2024-04-20T11:32:46.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.754+0000 7f24e8a00640 1 -- 172.21.15.152:0/2936257377 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f24e40d3910 con 0x7f24cc004d40 2024-04-20T11:32:46.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.754+0000 7f24e8a00640 1 -- 172.21.15.152:0/2936257377 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f24cc00b9c0 con 0x7f24e41603d0 2024-04-20T11:32:46.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.754+0000 7f24e9400640 1 --2- 172.21.15.152:0/2936257377 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f24cc003f30 0x7f24cc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:46.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.755+0000 7f24e9e00640 1 --2- 172.21.15.152:0/2936257377 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f24e41603d0 0x7f24cc0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:46.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.755+0000 7f24ea800640 1 --2- 172.21.15.152:0/2936257377 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f24cc004d40 0x7f24cc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:46.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.755+0000 7f24e9400640 1 -- 172.21.15.152:0/2936257377 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f24e41603d0 msgr2=0x7f24cc0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:46.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.755+0000 7f24e9400640 1 --2- 172.21.15.152:0/2936257377 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f24e41603d0 0x7f24cc0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:46.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.755+0000 7f24e9400640 1 -- 172.21.15.152:0/2936257377 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f24cc004d40 msgr2=0x7f24cc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:46.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.755+0000 7f24e9400640 1 --2- 172.21.15.152:0/2936257377 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f24cc004d40 0x7f24cc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:46.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.755+0000 7f24e9400640 1 -- 172.21.15.152:0/2936257377 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24cc003520 con 0x7f24cc003f30 2024-04-20T11:32:46.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.756+0000 7f24e9400640 1 --2- 172.21.15.152:0/2936257377 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f24cc003f30 0x7f24cc004800 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f24d4009a80 tx=0x7f24cc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=815174da105aed8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:46.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.756+0000 7f24e8a00640 1 -- 172.21.15.152:0/2936257377 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d4009680 con 0x7f24cc003f30 2024-04-20T11:32:46.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.756+0000 7f24e8a00640 1 -- 172.21.15.152:0/2936257377 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24d4005990 con 0x7f24cc003f30 2024-04-20T11:32:46.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.756+0000 7f24e8a00640 1 -- 172.21.15.152:0/2936257377 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d4005d80 con 0x7f24cc003f30 2024-04-20T11:32:46.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.757+0000 7f24eb200640 1 -- 172.21.15.152:0/2936257377 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f24cc003f30 msgr2=0x7f24cc004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:46.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.757+0000 7f24eb200640 1 --2- 172.21.15.152:0/2936257377 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f24cc003f30 0x7f24cc004800 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f24d4009a80 tx=0x7f24cc005830 comp rx=0 tx=0).stop 2024-04-20T11:32:46.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.757+0000 7f24ea800640 1 -- 172.21.15.152:0/2936257377 reap_dead start 2024-04-20T11:32:46.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.757+0000 7f24eb200640 1 -- 172.21.15.152:0/2936257377 shutdown_connections 2024-04-20T11:32:46.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.757+0000 7f24eb200640 1 -- 172.21.15.152:0/2936257377 >> 172.21.15.152:0/2936257377 conn(0x7f24e408bfd0 msgr2=0x7f24e40d4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:46.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.757+0000 7f24eb200640 1 -- 172.21.15.152:0/2936257377 shutdown_connections 2024-04-20T11:32:46.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.757+0000 7f24eb200640 1 -- 172.21.15.152:0/2936257377 wait complete. 2024-04-20T11:32:46.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.759+0000 7f24eb200640 1 Processor -- start 2024-04-20T11:32:46.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.759+0000 7f24eb200640 1 -- start start 2024-04-20T11:32:46.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.760+0000 7f24eb200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f24cc003f30 0x7f24e40c90d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:46.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.760+0000 7f24eb200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f24e40ca800 0x7f24e415e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:46.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.760+0000 7f24eb200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f24e40b6f00 0x7f24e40b9540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:46.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.760+0000 7f24eb200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f24e4162d60 con 0x7f24e40ca800 2024-04-20T11:32:46.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.760+0000 7f24eb200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f24e4162be0 con 0x7f24e40b6f00 2024-04-20T11:32:46.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.760+0000 7f24eb200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f24e4162ee0 con 0x7f24cc003f30 2024-04-20T11:32:46.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.760+0000 7f24e9400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f24e40ca800 0x7f24e415e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:46.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.760+0000 7f24e9400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f24e40ca800 0x7f24e415e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:44942/0 (socket says 172.21.15.152:44942) 2024-04-20T11:32:46.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.760+0000 7f24e9400640 1 -- 172.21.15.152:0/2327807001 learned_addr learned my addr 172.21.15.152:0/2327807001 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:32:46.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.760+0000 7f24e9e00640 1 --2- 172.21.15.152:0/2327807001 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f24cc003f30 0x7f24e40c90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:46.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.760+0000 7f24ea800640 1 --2- 172.21.15.152:0/2327807001 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f24e40b6f00 0x7f24e40b9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:46.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.760+0000 7f24e9400640 1 -- 172.21.15.152:0/2327807001 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f24cc003f30 msgr2=0x7f24e40c90d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:46.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.761+0000 7f24e9400640 1 --2- 172.21.15.152:0/2327807001 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f24cc003f30 0x7f24e40c90d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:46.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.761+0000 7f24e9400640 1 -- 172.21.15.152:0/2327807001 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f24e40b6f00 msgr2=0x7f24e40b9540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:46.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.761+0000 7f24e9400640 1 --2- 172.21.15.152:0/2327807001 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f24e40b6f00 0x7f24e40b9540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:46.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.761+0000 7f24e9400640 1 -- 172.21.15.152:0/2327807001 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24e40ba2c0 con 0x7f24e40ca800 2024-04-20T11:32:46.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.761+0000 7f24e9400640 1 --2- 172.21.15.152:0/2327807001 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f24e40ca800 0x7f24e415e260 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f24d4009ba0 tx=0x7f24d4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:46.762 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.761+0000 7f24de000640 1 -- 172.21.15.152:0/2327807001 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d4043070 con 0x7f24e40ca800 2024-04-20T11:32:46.762 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.761+0000 7f24de000640 1 -- 172.21.15.152:0/2327807001 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24d402ccd0 con 0x7f24e40ca800 2024-04-20T11:32:46.762 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.761+0000 7f24de000640 1 -- 172.21.15.152:0/2327807001 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d4006270 con 0x7f24e40ca800 2024-04-20T11:32:46.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.761+0000 7f24eb200640 1 -- 172.21.15.152:0/2327807001 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24e40c9760 con 0x7f24e40ca800 2024-04-20T11:32:46.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.761+0000 7f24eb200640 1 -- 172.21.15.152:0/2327807001 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24e40c9d20 con 0x7f24e40ca800 2024-04-20T11:32:46.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.763+0000 7f24eb200640 1 -- 172.21.15.152:0/2327807001 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24e4075e60 con 0x7f24e40ca800 2024-04-20T11:32:46.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.763+0000 7f24de000640 1 -- 172.21.15.152:0/2327807001 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 91142+0+0 (secure 0 0 0) 0x7f24d4046070 con 0x7f24e40ca800 2024-04-20T11:32:46.764 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.764+0000 7f24de000640 1 --2- 172.21.15.152:0/2327807001 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f24c806ca10 0x7f24c806eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:46.764 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.764+0000 7f24de000640 1 -- 172.21.15.152:0/2327807001 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7f24d40adf10 con 0x7f24e40ca800 2024-04-20T11:32:46.764 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.765+0000 7f24e9e00640 1 --2- 172.21.15.152:0/2327807001 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f24c806ca10 0x7f24c806eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:46.765 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.766+0000 7f24e9e00640 1 --2- 172.21.15.152:0/2327807001 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f24c806ca10 0x7f24c806eed0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f24e40b6a70 tx=0x7f24d8008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:46.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.769+0000 7f24de000640 1 -- 172.21.15.152:0/2327807001 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f24d407c340 con 0x7f24e40ca800 2024-04-20T11:32:46.951 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.951+0000 7f24eb200640 1 -- 172.21.15.152:0/2327807001 --> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f24e40b6ce0 con 0x7f24c806ca10 2024-04-20T11:32:46.971 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.972+0000 7f24de000640 1 -- 172.21.15.152:0/2327807001 <== mgr.4109 v2:172.21.15.175:6836/4021250441 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252079 (secure 0 0 0) 0x7f24e40b6ce0 con 0x7f24c806ca10 2024-04-20T11:32:46.975 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:32:46.976 INFO:teuthology.orchestra.run.smithi152.stderr:dumped all 2024-04-20T11:32:46.978 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.978+0000 7f24eb200640 1 -- 172.21.15.152:0/2327807001 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f24c806ca10 msgr2=0x7f24c806eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:46.978 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.978+0000 7f24eb200640 1 --2- 172.21.15.152:0/2327807001 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f24c806ca10 0x7f24c806eed0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f24e40b6a70 tx=0x7f24d8008040 comp rx=0 tx=0).stop 2024-04-20T11:32:46.978 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.978+0000 7f24eb200640 1 -- 172.21.15.152:0/2327807001 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f24e40ca800 msgr2=0x7f24e415e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:46.978 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.978+0000 7f24eb200640 1 --2- 172.21.15.152:0/2327807001 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f24e40ca800 0x7f24e415e260 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f24d4009ba0 tx=0x7f24d4004730 comp rx=0 tx=0).stop 2024-04-20T11:32:46.978 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.978+0000 7f24eb200640 1 -- 172.21.15.152:0/2327807001 shutdown_connections 2024-04-20T11:32:46.978 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.978+0000 7f24eb200640 1 --2- 172.21.15.152:0/2327807001 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f24c806ca10 0x7f24c806eed0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:46.978 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.978+0000 7f24eb200640 1 --2- 172.21.15.152:0/2327807001 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f24e40b6f00 0x7f24e40b9540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:46.978 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.978+0000 7f24eb200640 1 --2- 172.21.15.152:0/2327807001 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f24e40ca800 0x7f24e415e260 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:46.978 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.978+0000 7f24eb200640 1 --2- 172.21.15.152:0/2327807001 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f24cc003f30 0x7f24e40c90d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:46.978 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.978+0000 7f24eb200640 1 -- 172.21.15.152:0/2327807001 >> 172.21.15.152:0/2327807001 conn(0x7f24e408bfd0 msgr2=0x7f24e40cd630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:46.978 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.978+0000 7f24eb200640 1 -- 172.21.15.152:0/2327807001 shutdown_connections 2024-04-20T11:32:46.978 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:46.978+0000 7f24eb200640 1 -- 172.21.15.152:0/2327807001 wait complete. 2024-04-20T11:32:46.996 INFO:teuthology.orchestra.run.smithi152.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2024-04-20T11:32:46.602576+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":224,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":167,"ondisk_log_size":167,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":123,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":258596,"kb_used_data":9132,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754716124,"statfs":{"total":773094113280,"available":772829310976,"internally_reserved":0,"allocated":9351168,"data_stored":4657504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12746,"internal_metadata":255315510},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002396"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.408527+0000","last_change":"2024-04-20T11:32:42.111460+0000","last_active":"2024-04-20T11:32:43.408527+0000","last_peered":"2024-04-20T11:32:43.408527+0000","last_clean":"2024-04-20T11:32:43.408527+0000","last_became_active":"2024-04-20T11:32:42.111130+0000","last_became_peered":"2024-04-20T11:32:42.111130+0000","last_unstale":"2024-04-20T11:32:43.408527+0000","last_undegraded":"2024-04-20T11:32:43.408527+0000","last_fullsized":"2024-04-20T11:32:43.408527+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:26:31.889820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.110198+0000","last_change":"2024-04-20T11:32:42.121976+0000","last_active":"2024-04-20T11:32:43.110198+0000","last_peered":"2024-04-20T11:32:43.110198+0000","last_clean":"2024-04-20T11:32:43.110198+0000","last_became_active":"2024-04-20T11:32:42.121709+0000","last_became_peered":"2024-04-20T11:32:42.121709+0000","last_unstale":"2024-04-20T11:32:43.110198+0000","last_undegraded":"2024-04-20T11:32:43.110198+0000","last_fullsized":"2024-04-20T11:32:43.110198+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:03:38.796454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.107969+0000","last_change":"2024-04-20T11:32:42.110285+0000","last_active":"2024-04-20T11:32:43.107969+0000","last_peered":"2024-04-20T11:32:43.107969+0000","last_clean":"2024-04-20T11:32:43.107969+0000","last_became_active":"2024-04-20T11:32:42.109780+0000","last_became_peered":"2024-04-20T11:32:42.109780+0000","last_unstale":"2024-04-20T11:32:43.107969+0000","last_undegraded":"2024-04-20T11:32:43.107969+0000","last_fullsized":"2024-04-20T11:32:43.107969+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:28:47.958615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.404221+0000","last_change":"2024-04-20T11:32:42.112937+0000","last_active":"2024-04-20T11:32:43.404221+0000","last_peered":"2024-04-20T11:32:43.404221+0000","last_clean":"2024-04-20T11:32:43.404221+0000","last_became_active":"2024-04-20T11:32:42.112649+0000","last_became_peered":"2024-04-20T11:32:42.112649+0000","last_unstale":"2024-04-20T11:32:43.404221+0000","last_undegraded":"2024-04-20T11:32:43.404221+0000","last_fullsized":"2024-04-20T11:32:43.404221+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:49:56.916658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.406817+0000","last_change":"2024-04-20T11:32:42.105433+0000","last_active":"2024-04-20T11:32:43.406817+0000","last_peered":"2024-04-20T11:32:43.406817+0000","last_clean":"2024-04-20T11:32:43.406817+0000","last_became_active":"2024-04-20T11:32:42.104343+0000","last_became_peered":"2024-04-20T11:32:42.104343+0000","last_unstale":"2024-04-20T11:32:43.406817+0000","last_undegraded":"2024-04-20T11:32:43.406817+0000","last_fullsized":"2024-04-20T11:32:43.406817+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:12:08.390088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.405557+0000","last_change":"2024-04-20T11:32:42.107285+0000","last_active":"2024-04-20T11:32:43.405557+0000","last_peered":"2024-04-20T11:32:43.405557+0000","last_clean":"2024-04-20T11:32:43.405557+0000","last_became_active":"2024-04-20T11:32:42.106940+0000","last_became_peered":"2024-04-20T11:32:42.106940+0000","last_unstale":"2024-04-20T11:32:43.405557+0000","last_undegraded":"2024-04-20T11:32:43.405557+0000","last_fullsized":"2024-04-20T11:32:43.405557+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:03:28.197615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.403718+0000","last_change":"2024-04-20T11:32:42.106822+0000","last_active":"2024-04-20T11:32:43.403718+0000","last_peered":"2024-04-20T11:32:43.403718+0000","last_clean":"2024-04-20T11:32:43.403718+0000","last_became_active":"2024-04-20T11:32:42.106443+0000","last_became_peered":"2024-04-20T11:32:42.106443+0000","last_unstale":"2024-04-20T11:32:43.403718+0000","last_undegraded":"2024-04-20T11:32:43.403718+0000","last_fullsized":"2024-04-20T11:32:43.403718+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:14:53.766323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.404649+0000","last_change":"2024-04-20T11:32:42.093736+0000","last_active":"2024-04-20T11:32:43.404649+0000","last_peered":"2024-04-20T11:32:43.404649+0000","last_clean":"2024-04-20T11:32:43.404649+0000","last_became_active":"2024-04-20T11:32:42.093205+0000","last_became_peered":"2024-04-20T11:32:42.093205+0000","last_unstale":"2024-04-20T11:32:43.404649+0000","last_undegraded":"2024-04-20T11:32:43.404649+0000","last_fullsized":"2024-04-20T11:32:43.404649+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:08:25.164456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.406778+0000","last_change":"2024-04-20T11:32:42.089757+0000","last_active":"2024-04-20T11:32:43.406778+0000","last_peered":"2024-04-20T11:32:43.406778+0000","last_clean":"2024-04-20T11:32:43.406778+0000","last_became_active":"2024-04-20T11:32:42.089064+0000","last_became_peered":"2024-04-20T11:32:42.089064+0000","last_unstale":"2024-04-20T11:32:43.406778+0000","last_undegraded":"2024-04-20T11:32:43.406778+0000","last_fullsized":"2024-04-20T11:32:43.406778+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:07:47.911322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.408442+0000","last_change":"2024-04-20T11:32:42.107648+0000","last_active":"2024-04-20T11:32:43.408442+0000","last_peered":"2024-04-20T11:32:43.408442+0000","last_clean":"2024-04-20T11:32:43.408442+0000","last_became_active":"2024-04-20T11:32:42.107358+0000","last_became_peered":"2024-04-20T11:32:42.107358+0000","last_unstale":"2024-04-20T11:32:43.408442+0000","last_undegraded":"2024-04-20T11:32:43.408442+0000","last_fullsized":"2024-04-20T11:32:43.408442+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:34:34.592302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.110232+0000","last_change":"2024-04-20T11:32:42.118815+0000","last_active":"2024-04-20T11:32:43.110232+0000","last_peered":"2024-04-20T11:32:43.110232+0000","last_clean":"2024-04-20T11:32:43.110232+0000","last_became_active":"2024-04-20T11:32:42.114455+0000","last_became_peered":"2024-04-20T11:32:42.114455+0000","last_unstale":"2024-04-20T11:32:43.110232+0000","last_undegraded":"2024-04-20T11:32:43.110232+0000","last_fullsized":"2024-04-20T11:32:43.110232+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:37:27.505970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.406797+0000","last_change":"2024-04-20T11:32:42.111744+0000","last_active":"2024-04-20T11:32:43.406797+0000","last_peered":"2024-04-20T11:32:43.406797+0000","last_clean":"2024-04-20T11:32:43.406797+0000","last_became_active":"2024-04-20T11:32:42.110870+0000","last_became_peered":"2024-04-20T11:32:42.110870+0000","last_unstale":"2024-04-20T11:32:43.406797+0000","last_undegraded":"2024-04-20T11:32:43.406797+0000","last_fullsized":"2024-04-20T11:32:43.406797+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:07:53.171385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.107789+0000","last_change":"2024-04-20T11:32:42.109002+0000","last_active":"2024-04-20T11:32:43.107789+0000","last_peered":"2024-04-20T11:32:43.107789+0000","last_clean":"2024-04-20T11:32:43.107789+0000","last_became_active":"2024-04-20T11:32:42.106196+0000","last_became_peered":"2024-04-20T11:32:42.106196+0000","last_unstale":"2024-04-20T11:32:43.107789+0000","last_undegraded":"2024-04-20T11:32:43.107789+0000","last_fullsized":"2024-04-20T11:32:43.107789+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:26:31.889820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.404219+0000","last_change":"2024-04-20T11:32:42.110011+0000","last_active":"2024-04-20T11:32:43.404219+0000","last_peered":"2024-04-20T11:32:43.404219+0000","last_clean":"2024-04-20T11:32:43.404219+0000","last_became_active":"2024-04-20T11:32:42.109446+0000","last_became_peered":"2024-04-20T11:32:42.109446+0000","last_unstale":"2024-04-20T11:32:43.404219+0000","last_undegraded":"2024-04-20T11:32:43.404219+0000","last_fullsized":"2024-04-20T11:32:43.404219+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:46:32.134841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.105824+0000","last_change":"2024-04-20T11:32:42.107863+0000","last_active":"2024-04-20T11:32:43.105824+0000","last_peered":"2024-04-20T11:32:43.105824+0000","last_clean":"2024-04-20T11:32:43.105824+0000","last_became_active":"2024-04-20T11:32:42.107566+0000","last_became_peered":"2024-04-20T11:32:42.107566+0000","last_unstale":"2024-04-20T11:32:43.105824+0000","last_undegraded":"2024-04-20T11:32:43.105824+0000","last_fullsized":"2024-04-20T11:32:43.105824+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:30:03.622345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.404651+0000","last_change":"2024-04-20T11:32:42.109665+0000","last_active":"2024-04-20T11:32:43.404651+0000","last_peered":"2024-04-20T11:32:43.404651+0000","last_clean":"2024-04-20T11:32:43.404651+0000","last_became_active":"2024-04-20T11:32:42.108737+0000","last_became_peered":"2024-04-20T11:32:42.108737+0000","last_unstale":"2024-04-20T11:32:43.404651+0000","last_undegraded":"2024-04-20T11:32:43.404651+0000","last_fullsized":"2024-04-20T11:32:43.404651+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:54:47.268039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.407577+0000","last_change":"2024-04-20T11:32:42.111566+0000","last_active":"2024-04-20T11:32:43.407577+0000","last_peered":"2024-04-20T11:32:43.407577+0000","last_clean":"2024-04-20T11:32:43.407577+0000","last_became_active":"2024-04-20T11:32:42.111211+0000","last_became_peered":"2024-04-20T11:32:42.111211+0000","last_unstale":"2024-04-20T11:32:43.407577+0000","last_undegraded":"2024-04-20T11:32:43.407577+0000","last_fullsized":"2024-04-20T11:32:43.407577+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:34:34.592302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.404219+0000","last_change":"2024-04-20T11:32:42.109629+0000","last_active":"2024-04-20T11:32:43.404219+0000","last_peered":"2024-04-20T11:32:43.404219+0000","last_clean":"2024-04-20T11:32:43.404219+0000","last_became_active":"2024-04-20T11:32:42.109280+0000","last_became_peered":"2024-04-20T11:32:42.109280+0000","last_unstale":"2024-04-20T11:32:43.404219+0000","last_undegraded":"2024-04-20T11:32:43.404219+0000","last_fullsized":"2024-04-20T11:32:43.404219+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:41:17.239137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.105070+0000","last_change":"2024-04-20T11:32:42.101285+0000","last_active":"2024-04-20T11:32:43.105070+0000","last_peered":"2024-04-20T11:32:43.105070+0000","last_clean":"2024-04-20T11:32:43.105070+0000","last_became_active":"2024-04-20T11:32:42.100964+0000","last_became_peered":"2024-04-20T11:32:42.100964+0000","last_unstale":"2024-04-20T11:32:43.105070+0000","last_undegraded":"2024-04-20T11:32:43.105070+0000","last_fullsized":"2024-04-20T11:32:43.105070+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:02:58.915650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.404429+0000","last_change":"2024-04-20T11:32:42.100419+0000","last_active":"2024-04-20T11:32:43.404429+0000","last_peered":"2024-04-20T11:32:43.404429+0000","last_clean":"2024-04-20T11:32:43.404429+0000","last_became_active":"2024-04-20T11:32:42.099478+0000","last_became_peered":"2024-04-20T11:32:42.099478+0000","last_unstale":"2024-04-20T11:32:43.404429+0000","last_undegraded":"2024-04-20T11:32:43.404429+0000","last_fullsized":"2024-04-20T11:32:43.404429+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:04:18.751528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.404618+0000","last_change":"2024-04-20T11:32:42.108113+0000","last_active":"2024-04-20T11:32:43.404618+0000","last_peered":"2024-04-20T11:32:43.404618+0000","last_clean":"2024-04-20T11:32:43.404618+0000","last_became_active":"2024-04-20T11:32:42.107828+0000","last_became_peered":"2024-04-20T11:32:42.107828+0000","last_unstale":"2024-04-20T11:32:43.404618+0000","last_undegraded":"2024-04-20T11:32:43.404618+0000","last_fullsized":"2024-04-20T11:32:43.404618+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:39:35.303398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.109628+0000","last_change":"2024-04-20T11:32:42.103936+0000","last_active":"2024-04-20T11:32:43.109628+0000","last_peered":"2024-04-20T11:32:43.109628+0000","last_clean":"2024-04-20T11:32:43.109628+0000","last_became_active":"2024-04-20T11:32:42.103624+0000","last_became_peered":"2024-04-20T11:32:42.103624+0000","last_unstale":"2024-04-20T11:32:43.109628+0000","last_undegraded":"2024-04-20T11:32:43.109628+0000","last_fullsized":"2024-04-20T11:32:43.109628+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:41:06.829303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.406710+0000","last_change":"2024-04-20T11:32:42.105882+0000","last_active":"2024-04-20T11:32:43.406710+0000","last_peered":"2024-04-20T11:32:43.406710+0000","last_clean":"2024-04-20T11:32:43.406710+0000","last_became_active":"2024-04-20T11:32:42.105159+0000","last_became_peered":"2024-04-20T11:32:42.105159+0000","last_unstale":"2024-04-20T11:32:43.406710+0000","last_undegraded":"2024-04-20T11:32:43.406710+0000","last_fullsized":"2024-04-20T11:32:43.406710+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:18:37.443421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.403484+0000","last_change":"2024-04-20T11:32:42.105945+0000","last_active":"2024-04-20T11:32:43.403484+0000","last_peered":"2024-04-20T11:32:43.403484+0000","last_clean":"2024-04-20T11:32:43.403484+0000","last_became_active":"2024-04-20T11:32:42.104810+0000","last_became_peered":"2024-04-20T11:32:42.104810+0000","last_unstale":"2024-04-20T11:32:43.403484+0000","last_undegraded":"2024-04-20T11:32:43.403484+0000","last_fullsized":"2024-04-20T11:32:43.403484+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:29:31.317890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.110439+0000","last_change":"2024-04-20T11:32:42.111085+0000","last_active":"2024-04-20T11:32:43.110439+0000","last_peered":"2024-04-20T11:32:43.110439+0000","last_clean":"2024-04-20T11:32:43.110439+0000","last_became_active":"2024-04-20T11:32:42.110114+0000","last_became_peered":"2024-04-20T11:32:42.110114+0000","last_unstale":"2024-04-20T11:32:43.110439+0000","last_undegraded":"2024-04-20T11:32:43.110439+0000","last_fullsized":"2024-04-20T11:32:43.110439+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:56:21.525634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.107628+0000","last_change":"2024-04-20T11:32:42.124187+0000","last_active":"2024-04-20T11:32:43.107628+0000","last_peered":"2024-04-20T11:32:43.107628+0000","last_clean":"2024-04-20T11:32:43.107628+0000","last_became_active":"2024-04-20T11:32:42.123702+0000","last_became_peered":"2024-04-20T11:32:42.123702+0000","last_unstale":"2024-04-20T11:32:43.107628+0000","last_undegraded":"2024-04-20T11:32:43.107628+0000","last_fullsized":"2024-04-20T11:32:43.107628+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:31:20.713207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.404937+0000","last_change":"2024-04-20T11:32:42.108747+0000","last_active":"2024-04-20T11:32:43.404937+0000","last_peered":"2024-04-20T11:32:43.404937+0000","last_clean":"2024-04-20T11:32:43.404937+0000","last_became_active":"2024-04-20T11:32:42.108422+0000","last_became_peered":"2024-04-20T11:32:42.108422+0000","last_unstale":"2024-04-20T11:32:43.404937+0000","last_undegraded":"2024-04-20T11:32:43.404937+0000","last_fullsized":"2024-04-20T11:32:43.404937+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:50:54.488207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.407230+0000","last_change":"2024-04-20T11:32:42.104636+0000","last_active":"2024-04-20T11:32:43.407230+0000","last_peered":"2024-04-20T11:32:43.407230+0000","last_clean":"2024-04-20T11:32:43.407230+0000","last_became_active":"2024-04-20T11:32:42.103041+0000","last_became_peered":"2024-04-20T11:32:42.103041+0000","last_unstale":"2024-04-20T11:32:43.407230+0000","last_undegraded":"2024-04-20T11:32:43.407230+0000","last_fullsized":"2024-04-20T11:32:43.407230+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:02:58.915650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.109878+0000","last_change":"2024-04-20T11:32:42.105874+0000","last_active":"2024-04-20T11:32:43.109878+0000","last_peered":"2024-04-20T11:32:43.109878+0000","last_clean":"2024-04-20T11:32:43.109878+0000","last_became_active":"2024-04-20T11:32:42.104810+0000","last_became_peered":"2024-04-20T11:32:42.104810+0000","last_unstale":"2024-04-20T11:32:43.109878+0000","last_undegraded":"2024-04-20T11:32:43.109878+0000","last_fullsized":"2024-04-20T11:32:43.109878+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:30:08.274640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.109761+0000","last_change":"2024-04-20T11:32:42.129362+0000","last_active":"2024-04-20T11:32:43.109761+0000","last_peered":"2024-04-20T11:32:43.109761+0000","last_clean":"2024-04-20T11:32:43.109761+0000","last_became_active":"2024-04-20T11:32:42.128561+0000","last_became_peered":"2024-04-20T11:32:42.128561+0000","last_unstale":"2024-04-20T11:32:43.109761+0000","last_undegraded":"2024-04-20T11:32:43.109761+0000","last_fullsized":"2024-04-20T11:32:43.109761+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:27:40.289322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.406213+0000","last_change":"2024-04-20T11:32:42.092356+0000","last_active":"2024-04-20T11:32:43.406213+0000","last_peered":"2024-04-20T11:32:43.406213+0000","last_clean":"2024-04-20T11:32:43.406213+0000","last_became_active":"2024-04-20T11:32:42.091575+0000","last_became_peered":"2024-04-20T11:32:42.091575+0000","last_unstale":"2024-04-20T11:32:43.406213+0000","last_undegraded":"2024-04-20T11:32:43.406213+0000","last_fullsized":"2024-04-20T11:32:43.406213+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:02:58.915650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.404413+0000","last_change":"2024-04-20T11:32:42.106251+0000","last_active":"2024-04-20T11:32:43.404413+0000","last_peered":"2024-04-20T11:32:43.404413+0000","last_clean":"2024-04-20T11:32:43.404413+0000","last_became_active":"2024-04-20T11:32:42.105562+0000","last_became_peered":"2024-04-20T11:32:42.105562+0000","last_unstale":"2024-04-20T11:32:43.404413+0000","last_undegraded":"2024-04-20T11:32:43.404413+0000","last_fullsized":"2024-04-20T11:32:43.404413+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:54:47.268039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.407804+0000","last_change":"2024-04-20T11:32:42.109796+0000","last_active":"2024-04-20T11:32:43.407804+0000","last_peered":"2024-04-20T11:32:43.407804+0000","last_clean":"2024-04-20T11:32:43.407804+0000","last_became_active":"2024-04-20T11:32:42.109507+0000","last_became_peered":"2024-04-20T11:32:42.109507+0000","last_unstale":"2024-04-20T11:32:43.407804+0000","last_undegraded":"2024-04-20T11:32:43.407804+0000","last_fullsized":"2024-04-20T11:32:43.407804+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:31:20.713207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"82'4","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.409236+0000","last_change":"2024-04-20T11:32:41.080530+0000","last_active":"2024-04-20T11:32:43.409236+0000","last_peered":"2024-04-20T11:32:43.409236+0000","last_clean":"2024-04-20T11:32:43.409236+0000","last_became_active":"2024-04-20T11:32:41.078998+0000","last_became_peered":"2024-04-20T11:32:41.078998+0000","last_unstale":"2024-04-20T11:32:43.409236+0000","last_undegraded":"2024-04-20T11:32:43.409236+0000","last_fullsized":"2024-04-20T11:32:43.409236+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_clean_scrub_stamp":"2024-04-20T11:32:40.024004+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:03:37.791227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.408821+0000","last_change":"2024-04-20T11:32:42.104032+0000","last_active":"2024-04-20T11:32:43.408821+0000","last_peered":"2024-04-20T11:32:43.408821+0000","last_clean":"2024-04-20T11:32:43.408821+0000","last_became_active":"2024-04-20T11:32:42.103737+0000","last_became_peered":"2024-04-20T11:32:42.103737+0000","last_unstale":"2024-04-20T11:32:43.408821+0000","last_undegraded":"2024-04-20T11:32:43.408821+0000","last_fullsized":"2024-04-20T11:32:43.408821+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:31:20.713207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"82'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.408483+0000","last_change":"2024-04-20T11:32:41.073052+0000","last_active":"2024-04-20T11:32:43.408483+0000","last_peered":"2024-04-20T11:32:43.408483+0000","last_clean":"2024-04-20T11:32:43.408483+0000","last_became_active":"2024-04-20T11:32:41.072725+0000","last_became_peered":"2024-04-20T11:32:41.072725+0000","last_unstale":"2024-04-20T11:32:43.408483+0000","last_undegraded":"2024-04-20T11:32:43.408483+0000","last_fullsized":"2024-04-20T11:32:43.408483+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_clean_scrub_stamp":"2024-04-20T11:32:40.024004+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:14:42.465963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.407625+0000","last_change":"2024-04-20T11:32:42.113909+0000","last_active":"2024-04-20T11:32:43.407625+0000","last_peered":"2024-04-20T11:32:43.407625+0000","last_clean":"2024-04-20T11:32:43.407625+0000","last_became_active":"2024-04-20T11:32:42.113534+0000","last_became_peered":"2024-04-20T11:32:42.113534+0000","last_unstale":"2024-04-20T11:32:43.407625+0000","last_undegraded":"2024-04-20T11:32:43.407625+0000","last_fullsized":"2024-04-20T11:32:43.407625+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:12:08.390088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"82'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.105034+0000","last_change":"2024-04-20T11:32:41.061032+0000","last_active":"2024-04-20T11:32:43.105034+0000","last_peered":"2024-04-20T11:32:43.105034+0000","last_clean":"2024-04-20T11:32:43.105034+0000","last_became_active":"2024-04-20T11:32:41.060708+0000","last_became_peered":"2024-04-20T11:32:41.060708+0000","last_unstale":"2024-04-20T11:32:43.105034+0000","last_undegraded":"2024-04-20T11:32:43.105034+0000","last_fullsized":"2024-04-20T11:32:43.105034+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_clean_scrub_stamp":"2024-04-20T11:32:40.024004+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:07:46.906095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.107496+0000","last_change":"2024-04-20T11:32:42.113220+0000","last_active":"2024-04-20T11:32:43.107496+0000","last_peered":"2024-04-20T11:32:43.107496+0000","last_clean":"2024-04-20T11:32:43.107496+0000","last_became_active":"2024-04-20T11:32:42.112875+0000","last_became_peered":"2024-04-20T11:32:42.112875+0000","last_unstale":"2024-04-20T11:32:43.107496+0000","last_undegraded":"2024-04-20T11:32:43.107496+0000","last_fullsized":"2024-04-20T11:32:43.107496+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:34:34.592302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"82'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.104797+0000","last_change":"2024-04-20T11:32:41.067513+0000","last_active":"2024-04-20T11:32:43.104797+0000","last_peered":"2024-04-20T11:32:43.104797+0000","last_clean":"2024-04-20T11:32:43.104797+0000","last_became_active":"2024-04-20T11:32:41.066766+0000","last_became_peered":"2024-04-20T11:32:41.066766+0000","last_unstale":"2024-04-20T11:32:43.104797+0000","last_undegraded":"2024-04-20T11:32:43.104797+0000","last_fullsized":"2024-04-20T11:32:43.104797+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_clean_scrub_stamp":"2024-04-20T11:32:40.024004+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:50:31.018668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.109935+0000","last_change":"2024-04-20T11:32:42.127546+0000","last_active":"2024-04-20T11:32:43.109935+0000","last_peered":"2024-04-20T11:32:43.109935+0000","last_clean":"2024-04-20T11:32:43.109935+0000","last_became_active":"2024-04-20T11:32:42.127264+0000","last_became_peered":"2024-04-20T11:32:42.127264+0000","last_unstale":"2024-04-20T11:32:43.109935+0000","last_undegraded":"2024-04-20T11:32:43.109935+0000","last_fullsized":"2024-04-20T11:32:43.109935+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:02:32.140626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"82'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.406498+0000","last_change":"2024-04-20T11:32:41.077903+0000","last_active":"2024-04-20T11:32:43.406498+0000","last_peered":"2024-04-20T11:32:43.406498+0000","last_clean":"2024-04-20T11:32:43.406498+0000","last_became_active":"2024-04-20T11:32:41.077582+0000","last_became_peered":"2024-04-20T11:32:41.077582+0000","last_unstale":"2024-04-20T11:32:43.406498+0000","last_undegraded":"2024-04-20T11:32:43.406498+0000","last_fullsized":"2024-04-20T11:32:43.406498+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_clean_scrub_stamp":"2024-04-20T11:32:40.024004+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:56:20.520407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.405978+0000","last_change":"2024-04-20T11:32:42.097219+0000","last_active":"2024-04-20T11:32:43.405978+0000","last_peered":"2024-04-20T11:32:43.405978+0000","last_clean":"2024-04-20T11:32:43.405978+0000","last_became_active":"2024-04-20T11:32:42.096671+0000","last_became_peered":"2024-04-20T11:32:42.096671+0000","last_unstale":"2024-04-20T11:32:43.405978+0000","last_undegraded":"2024-04-20T11:32:43.405978+0000","last_fullsized":"2024-04-20T11:32:43.405978+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:07:53.171385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"82'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.404958+0000","last_change":"2024-04-20T11:32:41.073595+0000","last_active":"2024-04-20T11:32:43.404958+0000","last_peered":"2024-04-20T11:32:43.404958+0000","last_clean":"2024-04-20T11:32:43.404958+0000","last_became_active":"2024-04-20T11:32:41.072939+0000","last_became_peered":"2024-04-20T11:32:41.072939+0000","last_unstale":"2024-04-20T11:32:43.404958+0000","last_undegraded":"2024-04-20T11:32:43.404958+0000","last_fullsized":"2024-04-20T11:32:43.404958+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_clean_scrub_stamp":"2024-04-20T11:32:40.024004+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:37:26.500743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.107785+0000","last_change":"2024-04-20T11:32:42.104094+0000","last_active":"2024-04-20T11:32:43.107785+0000","last_peered":"2024-04-20T11:32:43.107785+0000","last_clean":"2024-04-20T11:32:43.107785+0000","last_became_active":"2024-04-20T11:32:42.103798+0000","last_became_peered":"2024-04-20T11:32:42.103798+0000","last_unstale":"2024-04-20T11:32:43.107785+0000","last_undegraded":"2024-04-20T11:32:43.107785+0000","last_fullsized":"2024-04-20T11:32:43.107785+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:57:39.307299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.104363+0000","last_change":"2024-04-20T11:32:41.066204+0000","last_active":"2024-04-20T11:32:43.104363+0000","last_peered":"2024-04-20T11:32:43.104363+0000","last_clean":"2024-04-20T11:32:43.104363+0000","last_became_active":"2024-04-20T11:32:41.065792+0000","last_became_peered":"2024-04-20T11:32:41.065792+0000","last_unstale":"2024-04-20T11:32:43.104363+0000","last_undegraded":"2024-04-20T11:32:43.104363+0000","last_fullsized":"2024-04-20T11:32:43.104363+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_clean_scrub_stamp":"2024-04-20T11:32:40.024004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:08:24.159228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.105627+0000","last_change":"2024-04-20T11:32:42.106147+0000","last_active":"2024-04-20T11:32:43.105627+0000","last_peered":"2024-04-20T11:32:43.105627+0000","last_clean":"2024-04-20T11:32:43.105627+0000","last_became_active":"2024-04-20T11:32:42.104108+0000","last_became_peered":"2024-04-20T11:32:42.104108+0000","last_unstale":"2024-04-20T11:32:43.105627+0000","last_undegraded":"2024-04-20T11:32:43.105627+0000","last_fullsized":"2024-04-20T11:32:43.105627+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:41:30.083314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.109524+0000","last_change":"2024-04-20T11:32:41.074941+0000","last_active":"2024-04-20T11:32:43.109524+0000","last_peered":"2024-04-20T11:32:43.109524+0000","last_clean":"2024-04-20T11:32:43.109524+0000","last_became_active":"2024-04-20T11:32:41.074655+0000","last_became_peered":"2024-04-20T11:32:41.074655+0000","last_unstale":"2024-04-20T11:32:43.109524+0000","last_undegraded":"2024-04-20T11:32:43.109524+0000","last_fullsized":"2024-04-20T11:32:43.109524+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_clean_scrub_stamp":"2024-04-20T11:32:40.024004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:46:31.129613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.106308+0000","last_change":"2024-04-20T11:32:42.106249+0000","last_active":"2024-04-20T11:32:43.106308+0000","last_peered":"2024-04-20T11:32:43.106308+0000","last_clean":"2024-04-20T11:32:43.106308+0000","last_became_active":"2024-04-20T11:32:42.105953+0000","last_became_peered":"2024-04-20T11:32:42.105953+0000","last_unstale":"2024-04-20T11:32:43.106308+0000","last_undegraded":"2024-04-20T11:32:43.106308+0000","last_fullsized":"2024-04-20T11:32:43.106308+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:19:40.411555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.409421+0000","last_change":"2024-04-20T11:32:41.077901+0000","last_active":"2024-04-20T11:32:43.409421+0000","last_peered":"2024-04-20T11:32:43.409421+0000","last_clean":"2024-04-20T11:32:43.409421+0000","last_became_active":"2024-04-20T11:32:41.077332+0000","last_became_peered":"2024-04-20T11:32:41.077332+0000","last_unstale":"2024-04-20T11:32:43.409421+0000","last_undegraded":"2024-04-20T11:32:43.409421+0000","last_fullsized":"2024-04-20T11:32:43.409421+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_clean_scrub_stamp":"2024-04-20T11:32:40.024004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:56:20.520407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.407013+0000","last_change":"2024-04-20T11:32:42.096170+0000","last_active":"2024-04-20T11:32:43.407013+0000","last_peered":"2024-04-20T11:32:43.407013+0000","last_clean":"2024-04-20T11:32:43.407013+0000","last_became_active":"2024-04-20T11:32:42.095392+0000","last_became_peered":"2024-04-20T11:32:42.095392+0000","last_unstale":"2024-04-20T11:32:43.407013+0000","last_undegraded":"2024-04-20T11:32:43.407013+0000","last_fullsized":"2024-04-20T11:32:43.407013+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:35:05.919911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.408383+0000","last_change":"2024-04-20T11:32:41.080580+0000","last_active":"2024-04-20T11:32:43.408383+0000","last_peered":"2024-04-20T11:32:43.408383+0000","last_clean":"2024-04-20T11:32:43.408383+0000","last_became_active":"2024-04-20T11:32:41.080248+0000","last_became_peered":"2024-04-20T11:32:41.080248+0000","last_unstale":"2024-04-20T11:32:43.408383+0000","last_undegraded":"2024-04-20T11:32:43.408383+0000","last_fullsized":"2024-04-20T11:32:43.408383+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_clean_scrub_stamp":"2024-04-20T11:32:40.024004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:38:47.685872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.104814+0000","last_change":"2024-04-20T11:32:42.109503+0000","last_active":"2024-04-20T11:32:43.104814+0000","last_peered":"2024-04-20T11:32:43.104814+0000","last_clean":"2024-04-20T11:32:43.104814+0000","last_became_active":"2024-04-20T11:32:42.108144+0000","last_became_peered":"2024-04-20T11:32:42.108144+0000","last_unstale":"2024-04-20T11:32:43.104814+0000","last_undegraded":"2024-04-20T11:32:43.104814+0000","last_fullsized":"2024-04-20T11:32:43.104814+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:07:53.171385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"82'3","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.405229+0000","last_change":"2024-04-20T11:32:41.080065+0000","last_active":"2024-04-20T11:32:43.405229+0000","last_peered":"2024-04-20T11:32:43.405229+0000","last_clean":"2024-04-20T11:32:43.405229+0000","last_became_active":"2024-04-20T11:32:41.079784+0000","last_became_peered":"2024-04-20T11:32:41.079784+0000","last_unstale":"2024-04-20T11:32:43.405229+0000","last_undegraded":"2024-04-20T11:32:43.405229+0000","last_fullsized":"2024-04-20T11:32:43.405229+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_clean_scrub_stamp":"2024-04-20T11:32:40.024004+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:37:26.500743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.405387+0000","last_change":"2024-04-20T11:32:42.100953+0000","last_active":"2024-04-20T11:32:43.405387+0000","last_peered":"2024-04-20T11:32:43.405387+0000","last_clean":"2024-04-20T11:32:43.405387+0000","last_became_active":"2024-04-20T11:32:42.099940+0000","last_became_peered":"2024-04-20T11:32:42.099940+0000","last_unstale":"2024-04-20T11:32:43.405387+0000","last_undegraded":"2024-04-20T11:32:43.405387+0000","last_fullsized":"2024-04-20T11:32:43.405387+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:18:37.443421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"82'6","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.405747+0000","last_change":"2024-04-20T11:32:41.056012+0000","last_active":"2024-04-20T11:32:43.405747+0000","last_peered":"2024-04-20T11:32:43.405747+0000","last_clean":"2024-04-20T11:32:43.405747+0000","last_became_active":"2024-04-20T11:32:41.054375+0000","last_became_peered":"2024-04-20T11:32:41.054375+0000","last_unstale":"2024-04-20T11:32:43.405747+0000","last_undegraded":"2024-04-20T11:32:43.405747+0000","last_fullsized":"2024-04-20T11:32:43.405747+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_clean_scrub_stamp":"2024-04-20T11:32:40.024004+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:02:31.135399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.105204+0000","last_change":"2024-04-20T11:32:42.091011+0000","last_active":"2024-04-20T11:32:43.105204+0000","last_peered":"2024-04-20T11:32:43.105204+0000","last_clean":"2024-04-20T11:32:43.105204+0000","last_became_active":"2024-04-20T11:32:42.090401+0000","last_became_peered":"2024-04-20T11:32:42.090401+0000","last_unstale":"2024-04-20T11:32:43.105204+0000","last_undegraded":"2024-04-20T11:32:43.105204+0000","last_fullsized":"2024-04-20T11:32:43.105204+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:22:54.913135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"82'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.109995+0000","last_change":"2024-04-20T11:32:41.068858+0000","last_active":"2024-04-20T11:32:43.109995+0000","last_peered":"2024-04-20T11:32:43.109995+0000","last_clean":"2024-04-20T11:32:43.109995+0000","last_became_active":"2024-04-20T11:32:41.068627+0000","last_became_peered":"2024-04-20T11:32:41.068627+0000","last_unstale":"2024-04-20T11:32:43.109995+0000","last_undegraded":"2024-04-20T11:32:43.109995+0000","last_fullsized":"2024-04-20T11:32:43.109995+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_clean_scrub_stamp":"2024-04-20T11:32:40.024004+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:41:16.233910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.108007+0000","last_change":"2024-04-20T11:32:42.105192+0000","last_active":"2024-04-20T11:32:43.108007+0000","last_peered":"2024-04-20T11:32:43.108007+0000","last_clean":"2024-04-20T11:32:43.108007+0000","last_became_active":"2024-04-20T11:32:42.104854+0000","last_became_peered":"2024-04-20T11:32:42.104854+0000","last_unstale":"2024-04-20T11:32:43.108007+0000","last_undegraded":"2024-04-20T11:32:43.108007+0000","last_fullsized":"2024-04-20T11:32:43.108007+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:27:30.418673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.105893+0000","last_change":"2024-04-20T11:32:41.063060+0000","last_active":"2024-04-20T11:32:43.105893+0000","last_peered":"2024-04-20T11:32:43.105893+0000","last_clean":"2024-04-20T11:32:43.105893+0000","last_became_active":"2024-04-20T11:32:41.062056+0000","last_became_peered":"2024-04-20T11:32:41.062056+0000","last_unstale":"2024-04-20T11:32:43.105893+0000","last_undegraded":"2024-04-20T11:32:43.105893+0000","last_fullsized":"2024-04-20T11:32:43.105893+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_clean_scrub_stamp":"2024-04-20T11:32:40.024004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:09:31.855425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.109406+0000","last_change":"2024-04-20T11:32:42.127563+0000","last_active":"2024-04-20T11:32:43.109406+0000","last_peered":"2024-04-20T11:32:43.109406+0000","last_clean":"2024-04-20T11:32:43.109406+0000","last_became_active":"2024-04-20T11:32:42.127289+0000","last_became_peered":"2024-04-20T11:32:42.127289+0000","last_unstale":"2024-04-20T11:32:43.109406+0000","last_undegraded":"2024-04-20T11:32:43.109406+0000","last_fullsized":"2024-04-20T11:32:43.109406+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:47:04.528964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.405647+0000","last_change":"2024-04-20T11:32:41.076233+0000","last_active":"2024-04-20T11:32:43.405647+0000","last_peered":"2024-04-20T11:32:43.405647+0000","last_clean":"2024-04-20T11:32:43.405647+0000","last_became_active":"2024-04-20T11:32:41.075768+0000","last_became_peered":"2024-04-20T11:32:41.075768+0000","last_unstale":"2024-04-20T11:32:43.405647+0000","last_undegraded":"2024-04-20T11:32:43.405647+0000","last_fullsized":"2024-04-20T11:32:43.405647+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_clean_scrub_stamp":"2024-04-20T11:32:40.024004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:03:37.791227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.109696+0000","last_change":"2024-04-20T11:32:42.120788+0000","last_active":"2024-04-20T11:32:43.109696+0000","last_peered":"2024-04-20T11:32:43.109696+0000","last_clean":"2024-04-20T11:32:43.109696+0000","last_became_active":"2024-04-20T11:32:42.120507+0000","last_became_peered":"2024-04-20T11:32:42.120507+0000","last_unstale":"2024-04-20T11:32:43.109696+0000","last_undegraded":"2024-04-20T11:32:43.109696+0000","last_fullsized":"2024-04-20T11:32:43.109696+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:55:45.744229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"82'8","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.104322+0000","last_change":"2024-04-20T11:32:41.072752+0000","last_active":"2024-04-20T11:32:43.104322+0000","last_peered":"2024-04-20T11:32:43.104322+0000","last_clean":"2024-04-20T11:32:43.104322+0000","last_became_active":"2024-04-20T11:32:41.072495+0000","last_became_peered":"2024-04-20T11:32:41.072495+0000","last_unstale":"2024-04-20T11:32:43.104322+0000","last_undegraded":"2024-04-20T11:32:43.104322+0000","last_fullsized":"2024-04-20T11:32:43.104322+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_clean_scrub_stamp":"2024-04-20T11:32:40.024004+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:54:50.982481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.107959+0000","last_change":"2024-04-20T11:32:42.101821+0000","last_active":"2024-04-20T11:32:43.107959+0000","last_peered":"2024-04-20T11:32:43.107959+0000","last_clean":"2024-04-20T11:32:43.107959+0000","last_became_active":"2024-04-20T11:32:42.101485+0000","last_became_peered":"2024-04-20T11:32:42.101485+0000","last_unstale":"2024-04-20T11:32:43.107959+0000","last_undegraded":"2024-04-20T11:32:43.107959+0000","last_fullsized":"2024-04-20T11:32:43.107959+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:21:31.149890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"82'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.409539+0000","last_change":"2024-04-20T11:32:41.074739+0000","last_active":"2024-04-20T11:32:43.409539+0000","last_peered":"2024-04-20T11:32:43.409539+0000","last_clean":"2024-04-20T11:32:43.409539+0000","last_became_active":"2024-04-20T11:32:41.074479+0000","last_became_peered":"2024-04-20T11:32:41.074479+0000","last_unstale":"2024-04-20T11:32:43.409539+0000","last_undegraded":"2024-04-20T11:32:43.409539+0000","last_fullsized":"2024-04-20T11:32:43.409539+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_clean_scrub_stamp":"2024-04-20T11:32:40.024004+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:35:37.812862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.409241+0000","last_change":"2024-04-20T11:32:42.105637+0000","last_active":"2024-04-20T11:32:43.409241+0000","last_peered":"2024-04-20T11:32:43.409241+0000","last_clean":"2024-04-20T11:32:43.409241+0000","last_became_active":"2024-04-20T11:32:42.103463+0000","last_became_peered":"2024-04-20T11:32:42.103463+0000","last_unstale":"2024-04-20T11:32:43.409241+0000","last_undegraded":"2024-04-20T11:32:43.409241+0000","last_fullsized":"2024-04-20T11:32:43.409241+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:57:39.307299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"82'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.104824+0000","last_change":"2024-04-20T11:32:41.080416+0000","last_active":"2024-04-20T11:32:43.104824+0000","last_peered":"2024-04-20T11:32:43.104824+0000","last_clean":"2024-04-20T11:32:43.104824+0000","last_became_active":"2024-04-20T11:32:41.080175+0000","last_became_peered":"2024-04-20T11:32:41.080175+0000","last_unstale":"2024-04-20T11:32:43.104824+0000","last_undegraded":"2024-04-20T11:32:43.104824+0000","last_fullsized":"2024-04-20T11:32:43.104824+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_clean_scrub_stamp":"2024-04-20T11:32:40.024004+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:35:04.914684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.407736+0000","last_change":"2024-04-20T11:32:42.106196+0000","last_active":"2024-04-20T11:32:43.407736+0000","last_peered":"2024-04-20T11:32:43.407736+0000","last_clean":"2024-04-20T11:32:43.407736+0000","last_became_active":"2024-04-20T11:32:42.105887+0000","last_became_peered":"2024-04-20T11:32:42.105887+0000","last_unstale":"2024-04-20T11:32:43.407736+0000","last_undegraded":"2024-04-20T11:32:43.407736+0000","last_fullsized":"2024-04-20T11:32:43.407736+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:26:31.889820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.104912+0000","last_change":"2024-04-20T11:32:41.067520+0000","last_active":"2024-04-20T11:32:43.104912+0000","last_peered":"2024-04-20T11:32:43.104912+0000","last_clean":"2024-04-20T11:32:43.104912+0000","last_became_active":"2024-04-20T11:32:41.067072+0000","last_became_peered":"2024-04-20T11:32:41.067072+0000","last_unstale":"2024-04-20T11:32:43.104912+0000","last_undegraded":"2024-04-20T11:32:43.104912+0000","last_fullsized":"2024-04-20T11:32:43.104912+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_clean_scrub_stamp":"2024-04-20T11:32:40.024004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:38:22.319723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.404210+0000","last_change":"2024-04-20T11:32:42.112898+0000","last_active":"2024-04-20T11:32:43.404210+0000","last_peered":"2024-04-20T11:32:43.404210+0000","last_clean":"2024-04-20T11:32:43.404210+0000","last_became_active":"2024-04-20T11:32:42.112532+0000","last_became_peered":"2024-04-20T11:32:42.112532+0000","last_unstale":"2024-04-20T11:32:43.404210+0000","last_undegraded":"2024-04-20T11:32:43.404210+0000","last_fullsized":"2024-04-20T11:32:43.404210+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:02:39.189706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"82'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.105981+0000","last_change":"2024-04-20T11:32:41.046547+0000","last_active":"2024-04-20T11:32:43.105981+0000","last_peered":"2024-04-20T11:32:43.105981+0000","last_clean":"2024-04-20T11:32:43.105981+0000","last_became_active":"2024-04-20T11:32:41.045500+0000","last_became_peered":"2024-04-20T11:32:41.045500+0000","last_unstale":"2024-04-20T11:32:43.105981+0000","last_undegraded":"2024-04-20T11:32:43.105981+0000","last_fullsized":"2024-04-20T11:32:43.105981+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_clean_scrub_stamp":"2024-04-20T11:32:40.024004+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:33:33.430126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.405128+0000","last_change":"2024-04-20T11:32:42.104740+0000","last_active":"2024-04-20T11:32:43.405128+0000","last_peered":"2024-04-20T11:32:43.405128+0000","last_clean":"2024-04-20T11:32:43.405128+0000","last_became_active":"2024-04-20T11:32:42.102706+0000","last_became_peered":"2024-04-20T11:32:42.102706+0000","last_unstale":"2024-04-20T11:32:43.405128+0000","last_undegraded":"2024-04-20T11:32:43.405128+0000","last_fullsized":"2024-04-20T11:32:43.405128+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:50:32.023896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.404456+0000","last_change":"2024-04-20T11:32:41.066182+0000","last_active":"2024-04-20T11:32:43.404456+0000","last_peered":"2024-04-20T11:32:43.404456+0000","last_clean":"2024-04-20T11:32:43.404456+0000","last_became_active":"2024-04-20T11:32:41.065918+0000","last_became_peered":"2024-04-20T11:32:41.065918+0000","last_unstale":"2024-04-20T11:32:43.404456+0000","last_undegraded":"2024-04-20T11:32:43.404456+0000","last_fullsized":"2024-04-20T11:32:43.404456+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_clean_scrub_stamp":"2024-04-20T11:32:40.024004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:13:15.784362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.403348+0000","last_change":"2024-04-20T11:32:42.090868+0000","last_active":"2024-04-20T11:32:43.403348+0000","last_peered":"2024-04-20T11:32:43.403348+0000","last_clean":"2024-04-20T11:32:43.403348+0000","last_became_active":"2024-04-20T11:32:42.089885+0000","last_became_peered":"2024-04-20T11:32:42.089885+0000","last_unstale":"2024-04-20T11:32:43.403348+0000","last_undegraded":"2024-04-20T11:32:43.403348+0000","last_fullsized":"2024-04-20T11:32:43.403348+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:30:33.256438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.403688+0000","last_change":"2024-04-20T11:32:41.073570+0000","last_active":"2024-04-20T11:32:43.403688+0000","last_peered":"2024-04-20T11:32:43.403688+0000","last_clean":"2024-04-20T11:32:43.403688+0000","last_became_active":"2024-04-20T11:32:41.073282+0000","last_became_peered":"2024-04-20T11:32:41.073282+0000","last_unstale":"2024-04-20T11:32:43.403688+0000","last_undegraded":"2024-04-20T11:32:43.403688+0000","last_fullsized":"2024-04-20T11:32:43.403688+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_clean_scrub_stamp":"2024-04-20T11:32:40.024004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:21:41.469361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.404439+0000","last_change":"2024-04-20T11:32:42.081070+0000","last_active":"2024-04-20T11:32:43.404439+0000","last_peered":"2024-04-20T11:32:43.404439+0000","last_clean":"2024-04-20T11:32:43.404439+0000","last_became_active":"2024-04-20T11:32:42.080184+0000","last_became_peered":"2024-04-20T11:32:42.080184+0000","last_unstale":"2024-04-20T11:32:43.404439+0000","last_undegraded":"2024-04-20T11:32:43.404439+0000","last_fullsized":"2024-04-20T11:32:43.404439+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:54:51.987709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.105358+0000","last_change":"2024-04-20T11:32:41.046250+0000","last_active":"2024-04-20T11:32:43.105358+0000","last_peered":"2024-04-20T11:32:43.105358+0000","last_clean":"2024-04-20T11:32:43.105358+0000","last_became_active":"2024-04-20T11:32:41.045906+0000","last_became_peered":"2024-04-20T11:32:41.045906+0000","last_unstale":"2024-04-20T11:32:43.105358+0000","last_undegraded":"2024-04-20T11:32:43.105358+0000","last_fullsized":"2024-04-20T11:32:43.105358+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_clean_scrub_stamp":"2024-04-20T11:32:40.024004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:52:27.911885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.408212+0000","last_change":"2024-04-20T11:32:42.086957+0000","last_active":"2024-04-20T11:32:43.408212+0000","last_peered":"2024-04-20T11:32:43.408212+0000","last_clean":"2024-04-20T11:32:43.408212+0000","last_became_active":"2024-04-20T11:32:42.086249+0000","last_became_peered":"2024-04-20T11:32:42.086249+0000","last_unstale":"2024-04-20T11:32:43.408212+0000","last_undegraded":"2024-04-20T11:32:43.408212+0000","last_fullsized":"2024-04-20T11:32:43.408212+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:57:39.307299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"82'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.110474+0000","last_change":"2024-04-20T11:32:41.071485+0000","last_active":"2024-04-20T11:32:43.110474+0000","last_peered":"2024-04-20T11:32:43.110474+0000","last_clean":"2024-04-20T11:32:43.110474+0000","last_became_active":"2024-04-20T11:32:41.071233+0000","last_became_peered":"2024-04-20T11:32:41.071233+0000","last_unstale":"2024-04-20T11:32:43.110474+0000","last_undegraded":"2024-04-20T11:32:43.110474+0000","last_fullsized":"2024-04-20T11:32:43.110474+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_clean_scrub_stamp":"2024-04-20T11:32:40.024004+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:49:55.911430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.108011+0000","last_change":"2024-04-20T11:32:42.110626+0000","last_active":"2024-04-20T11:32:43.108011+0000","last_peered":"2024-04-20T11:32:43.108011+0000","last_clean":"2024-04-20T11:32:43.108011+0000","last_became_active":"2024-04-20T11:32:42.110297+0000","last_became_peered":"2024-04-20T11:32:42.110297+0000","last_unstale":"2024-04-20T11:32:43.108011+0000","last_undegraded":"2024-04-20T11:32:43.108011+0000","last_fullsized":"2024-04-20T11:32:43.108011+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:52:28.917113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"83'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.112522+0000","last_change":"2024-04-20T11:32:41.070454+0000","last_active":"2024-04-20T11:32:43.112522+0000","last_peered":"2024-04-20T11:32:43.112522+0000","last_clean":"2024-04-20T11:32:43.112522+0000","last_became_active":"2024-04-20T11:32:41.070199+0000","last_became_peered":"2024-04-20T11:32:41.070199+0000","last_unstale":"2024-04-20T11:32:43.112522+0000","last_undegraded":"2024-04-20T11:32:43.112522+0000","last_fullsized":"2024-04-20T11:32:43.112522+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_clean_scrub_stamp":"2024-04-20T11:32:40.024004+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:02:38.184479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.407927+0000","last_change":"2024-04-20T11:32:42.104955+0000","last_active":"2024-04-20T11:32:43.407927+0000","last_peered":"2024-04-20T11:32:43.407927+0000","last_clean":"2024-04-20T11:32:43.407927+0000","last_became_active":"2024-04-20T11:32:42.104676+0000","last_became_peered":"2024-04-20T11:32:42.104676+0000","last_unstale":"2024-04-20T11:32:43.407927+0000","last_undegraded":"2024-04-20T11:32:43.407927+0000","last_fullsized":"2024-04-20T11:32:43.407927+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:22:54.913135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.404946+0000","last_change":"2024-04-20T11:32:41.078591+0000","last_active":"2024-04-20T11:32:43.404946+0000","last_peered":"2024-04-20T11:32:43.404946+0000","last_clean":"2024-04-20T11:32:43.404946+0000","last_became_active":"2024-04-20T11:32:41.078296+0000","last_became_peered":"2024-04-20T11:32:41.078296+0000","last_unstale":"2024-04-20T11:32:43.404946+0000","last_undegraded":"2024-04-20T11:32:43.404946+0000","last_fullsized":"2024-04-20T11:32:43.404946+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_clean_scrub_stamp":"2024-04-20T11:32:40.024004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:02:31.135399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.407332+0000","last_change":"2024-04-20T11:32:42.098341+0000","last_active":"2024-04-20T11:32:43.407332+0000","last_peered":"2024-04-20T11:32:43.407332+0000","last_clean":"2024-04-20T11:32:43.407332+0000","last_became_active":"2024-04-20T11:32:42.097858+0000","last_became_peered":"2024-04-20T11:32:42.097858+0000","last_unstale":"2024-04-20T11:32:43.407332+0000","last_undegraded":"2024-04-20T11:32:43.407332+0000","last_fullsized":"2024-04-20T11:32:43.407332+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:22:54.913135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.407890+0000","last_change":"2024-04-20T11:32:41.077912+0000","last_active":"2024-04-20T11:32:43.407890+0000","last_peered":"2024-04-20T11:32:43.407890+0000","last_clean":"2024-04-20T11:32:43.407890+0000","last_became_active":"2024-04-20T11:32:41.077465+0000","last_became_peered":"2024-04-20T11:32:41.077465+0000","last_unstale":"2024-04-20T11:32:43.407890+0000","last_undegraded":"2024-04-20T11:32:43.407890+0000","last_fullsized":"2024-04-20T11:32:43.407890+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_clean_scrub_stamp":"2024-04-20T11:32:40.024004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:14:42.465963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.405966+0000","last_change":"2024-04-20T11:32:42.104767+0000","last_active":"2024-04-20T11:32:43.405966+0000","last_peered":"2024-04-20T11:32:43.405966+0000","last_clean":"2024-04-20T11:32:43.405966+0000","last_became_active":"2024-04-20T11:32:42.104440+0000","last_became_peered":"2024-04-20T11:32:42.104440+0000","last_unstale":"2024-04-20T11:32:43.405966+0000","last_undegraded":"2024-04-20T11:32:43.405966+0000","last_fullsized":"2024-04-20T11:32:43.405966+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:38:23.324951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.104809+0000","last_change":"2024-04-20T11:32:41.071601+0000","last_active":"2024-04-20T11:32:43.104809+0000","last_peered":"2024-04-20T11:32:43.104809+0000","last_clean":"2024-04-20T11:32:43.104809+0000","last_became_active":"2024-04-20T11:32:41.071360+0000","last_became_peered":"2024-04-20T11:32:41.071360+0000","last_unstale":"2024-04-20T11:32:43.104809+0000","last_undegraded":"2024-04-20T11:32:43.104809+0000","last_fullsized":"2024-04-20T11:32:43.104809+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_clean_scrub_stamp":"2024-04-20T11:32:40.024004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:03:27.192387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.105165+0000","last_change":"2024-04-20T11:32:42.089814+0000","last_active":"2024-04-20T11:32:43.105165+0000","last_peered":"2024-04-20T11:32:43.105165+0000","last_clean":"2024-04-20T11:32:43.105165+0000","last_became_active":"2024-04-20T11:32:42.089312+0000","last_became_peered":"2024-04-20T11:32:42.089312+0000","last_unstale":"2024-04-20T11:32:43.105165+0000","last_undegraded":"2024-04-20T11:32:43.105165+0000","last_fullsized":"2024-04-20T11:32:43.105165+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:12:08.390088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.405234+0000","last_change":"2024-04-20T11:32:41.073930+0000","last_active":"2024-04-20T11:32:43.405234+0000","last_peered":"2024-04-20T11:32:43.405234+0000","last_clean":"2024-04-20T11:32:43.405234+0000","last_became_active":"2024-04-20T11:32:41.073268+0000","last_became_peered":"2024-04-20T11:32:41.073268+0000","last_unstale":"2024-04-20T11:32:43.405234+0000","last_undegraded":"2024-04-20T11:32:43.405234+0000","last_fullsized":"2024-04-20T11:32:43.405234+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_clean_scrub_stamp":"2024-04-20T11:32:40.024004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:03:37.791227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.107074+0000","last_change":"2024-04-20T11:32:42.113272+0000","last_active":"2024-04-20T11:32:43.107074+0000","last_peered":"2024-04-20T11:32:43.107074+0000","last_clean":"2024-04-20T11:32:43.107074+0000","last_became_active":"2024-04-20T11:32:42.112914+0000","last_became_peered":"2024-04-20T11:32:42.112914+0000","last_unstale":"2024-04-20T11:32:43.107074+0000","last_undegraded":"2024-04-20T11:32:43.107074+0000","last_fullsized":"2024-04-20T11:32:43.107074+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:22:54.913135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.406271+0000","last_change":"2024-04-20T11:32:41.072370+0000","last_active":"2024-04-20T11:32:43.406271+0000","last_peered":"2024-04-20T11:32:43.406271+0000","last_clean":"2024-04-20T11:32:43.406271+0000","last_became_active":"2024-04-20T11:32:41.071963+0000","last_became_peered":"2024-04-20T11:32:41.071963+0000","last_unstale":"2024-04-20T11:32:43.406271+0000","last_undegraded":"2024-04-20T11:32:43.406271+0000","last_fullsized":"2024-04-20T11:32:43.406271+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_clean_scrub_stamp":"2024-04-20T11:32:40.024004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:30:07.269413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.405846+0000","last_change":"2024-04-20T11:32:42.102348+0000","last_active":"2024-04-20T11:32:43.405846+0000","last_peered":"2024-04-20T11:32:43.405846+0000","last_clean":"2024-04-20T11:32:43.405846+0000","last_became_active":"2024-04-20T11:32:42.101914+0000","last_became_peered":"2024-04-20T11:32:42.101914+0000","last_unstale":"2024-04-20T11:32:43.405846+0000","last_undegraded":"2024-04-20T11:32:43.405846+0000","last_fullsized":"2024-04-20T11:32:43.405846+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:50:54.488207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.405034+0000","last_change":"2024-04-20T11:32:41.074508+0000","last_active":"2024-04-20T11:32:43.405034+0000","last_peered":"2024-04-20T11:32:43.405034+0000","last_clean":"2024-04-20T11:32:43.405034+0000","last_became_active":"2024-04-20T11:32:41.074194+0000","last_became_peered":"2024-04-20T11:32:41.074194+0000","last_unstale":"2024-04-20T11:32:43.405034+0000","last_undegraded":"2024-04-20T11:32:43.405034+0000","last_fullsized":"2024-04-20T11:32:43.405034+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_clean_scrub_stamp":"2024-04-20T11:32:40.024004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:02:31.135399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.110424+0000","last_change":"2024-04-20T11:32:42.107164+0000","last_active":"2024-04-20T11:32:43.110424+0000","last_peered":"2024-04-20T11:32:43.110424+0000","last_clean":"2024-04-20T11:32:43.110424+0000","last_became_active":"2024-04-20T11:32:42.106906+0000","last_became_peered":"2024-04-20T11:32:42.106906+0000","last_unstale":"2024-04-20T11:32:43.110424+0000","last_undegraded":"2024-04-20T11:32:43.110424+0000","last_fullsized":"2024-04-20T11:32:43.110424+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:41.029232+0000","last_clean_scrub_stamp":"2024-04-20T11:32:41.029232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:14:43.471190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"82'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.108004+0000","last_change":"2024-04-20T11:32:41.079526+0000","last_active":"2024-04-20T11:32:43.108004+0000","last_peered":"2024-04-20T11:32:43.108004+0000","last_clean":"2024-04-20T11:32:43.108004+0000","last_became_active":"2024-04-20T11:32:41.079233+0000","last_became_peered":"2024-04-20T11:32:41.079233+0000","last_unstale":"2024-04-20T11:32:43.108004+0000","last_undegraded":"2024-04-20T11:32:43.108004+0000","last_fullsized":"2024-04-20T11:32:43.108004+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:32:40.024004+0000","last_clean_scrub_stamp":"2024-04-20T11:32:40.024004+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:38:22.319723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"72'128","reported_seq":322,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-20T11:32:43.107228+0000","last_change":"2024-04-20T11:26:48.661897+0000","last_active":"2024-04-20T11:32:43.107228+0000","last_peered":"2024-04-20T11:32:43.107228+0000","last_clean":"2024-04-20T11:32:43.107228+0000","last_became_active":"2024-04-20T11:26:48.656535+0000","last_became_peered":"2024-04-20T11:26:48.656535+0000","last_unstale":"2024-04-20T11:32:43.107228+0000","last_undegraded":"2024-04-20T11:32:43.107228+0000","last_fullsized":"2024-04-20T11:32:43.107228+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:47.543004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:47.543004+0000","last_clean_scrub_stamp":"2024-04-20T11:26:47.543004+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:58:35.527105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574925,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32760,"kb_used_data":1580,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339080,"statfs":{"total":96636764160,"available":96603217920,"internally_reserved":0,"allocated":1617920,"data_stored":1028369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574922,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32176,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339664,"statfs":{"total":96636764160,"available":96603815936,"internally_reserved":0,"allocated":1019904,"data_stored":432325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574924,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32188,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339652,"statfs":{"total":96636764160,"available":96603803648,"internally_reserved":0,"allocated":1024000,"data_stored":433265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574922,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32172,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339668,"statfs":{"total":96636764160,"available":96603820032,"internally_reserved":0,"allocated":1007616,"data_stored":431631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574923,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32184,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339656,"statfs":{"total":96636764160,"available":96603807744,"internally_reserved":0,"allocated":1019904,"data_stored":433243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574924,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32168,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339672,"statfs":{"total":96636764160,"available":96603824128,"internally_reserved":0,"allocated":1011712,"data_stored":432213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574922,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32188,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339652,"statfs":{"total":96636764160,"available":96603803648,"internally_reserved":0,"allocated":1032192,"data_stored":438125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574923,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32760,"kb_used_data":1580,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339080,"statfs":{"total":96636764160,"available":96603217920,"internally_reserved":0,"allocated":1617920,"data_stored":1028333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T11:32:46.999 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-20T11:32:47.000 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-20T11:32:47.017 INFO:tasks.ceph.mds.c.smithi152.stderr:2024-04-20T11:32:47.018+0000 7fd55ba00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-20T11:32:47.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.226+0000 7f4176600640 1 Processor -- start 2024-04-20T11:32:47.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.226+0000 7f4176600640 1 -- start start 2024-04-20T11:32:47.227 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.228+0000 7f4176600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f41700d0b80 0x7f4170160d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:47.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.228+0000 7f4176600640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4170161290 con 0x7f41700cd450 2024-04-20T11:32:47.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.228+0000 7f4176600640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4170072af0 con 0x7f41700ccff0 2024-04-20T11:32:47.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.228+0000 7f4176600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4170145830 con 0x7f41700d0b80 2024-04-20T11:32:47.230 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.229+0000 7f416fe00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f41700cd450 0x7f41700723e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:41150/0 (socket says 172.21.15.152:41150) 2024-04-20T11:32:47.230 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.229+0000 7f416fe00640 1 -- 172.21.15.152:0/3495956903 learned_addr learned my addr 172.21.15.152:0/3495956903 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:32:47.230 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.229+0000 7f416f400640 1 --2- 172.21.15.152:0/3495956903 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f41700d0b80 0x7f4170160d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:47.230 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.229+0000 7f416ea00640 1 -- 172.21.15.152:0/3495956903 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4272191344 0 0) 0x7f4170161290 con 0x7f41700cd450 2024-04-20T11:32:47.230 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.229+0000 7f416ea00640 1 -- 172.21.15.152:0/3495956903 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4154003520 con 0x7f41700cd450 2024-04-20T11:32:47.230 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.229+0000 7f416ea00640 1 -- 172.21.15.152:0/3495956903 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3416077915 0 0) 0x7f4154003520 con 0x7f41700cd450 2024-04-20T11:32:47.230 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.229+0000 7f416ea00640 1 -- 172.21.15.152:0/3495956903 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f41700d0b80 msgr2=0x7f4170160d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:47.230 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.229+0000 7f416ea00640 1 --2- 172.21.15.152:0/3495956903 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f41700d0b80 0x7f4170160d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:47.230 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.229+0000 7f416ea00640 1 -- 172.21.15.152:0/3495956903 >> v1:172.21.15.175:6789/0 conn(0x7f41700ccff0 legacy=0x7f4170071cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:47.230 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.229+0000 7f416ea00640 1 -- 172.21.15.152:0/3495956903 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f417006f500 con 0x7f41700cd450 2024-04-20T11:32:47.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.229+0000 7f416ea00640 1 -- 172.21.15.152:0/3495956903 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f41680019f0 con 0x7f41700cd450 2024-04-20T11:32:47.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.229+0000 7f416ea00640 1 -- 172.21.15.152:0/3495956903 >> v1:172.21.15.152:6789/0 conn(0x7f41700cd450 legacy=0x7f41700723e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:47.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.230+0000 7f416ea00640 1 --2- 172.21.15.152:0/3495956903 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41540039e0 0x7f4154003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:47.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.230+0000 7f416ea00640 1 --2- 172.21.15.152:0/3495956903 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f41700d0b80 0x7f4154004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:47.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.230+0000 7f416ea00640 1 --2- 172.21.15.152:0/3495956903 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4154004d40 0x7f41540051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:47.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.230+0000 7f416ea00640 1 -- 172.21.15.152:0/3495956903 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f4170145830 con 0x7f4154004d40 2024-04-20T11:32:47.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.230+0000 7f416ea00640 1 -- 172.21.15.152:0/3495956903 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f415400ba70 con 0x7f41540039e0 2024-04-20T11:32:47.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.230+0000 7f416ea00640 1 -- 172.21.15.152:0/3495956903 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f415400bc70 con 0x7f41700d0b80 2024-04-20T11:32:47.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.230+0000 7f416fe00640 1 --2- 172.21.15.152:0/3495956903 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4154004d40 0x7f41540051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:47.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.230+0000 7f416f400640 1 --2- 172.21.15.152:0/3495956903 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41540039e0 0x7f4154003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:47.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.230+0000 7f4167e00640 1 --2- 172.21.15.152:0/3495956903 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f41700d0b80 0x7f4154004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:47.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.230+0000 7f416fe00640 1 -- 172.21.15.152:0/3495956903 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f41700d0b80 msgr2=0x7f4154004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:47.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.230+0000 7f416fe00640 1 --2- 172.21.15.152:0/3495956903 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f41700d0b80 0x7f4154004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:47.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.230+0000 7f416fe00640 1 -- 172.21.15.152:0/3495956903 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41540039e0 msgr2=0x7f4154003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:47.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.230+0000 7f416fe00640 1 --2- 172.21.15.152:0/3495956903 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41540039e0 0x7f4154003e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:47.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.230+0000 7f416fe00640 1 -- 172.21.15.152:0/3495956903 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4154003520 con 0x7f4154004d40 2024-04-20T11:32:47.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.231+0000 7f416fe00640 1 --2- 172.21.15.152:0/3495956903 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4154004d40 0x7f41540051c0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f41680013d0 tx=0x7f4154005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c079e5da03f5695 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:47.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.231+0000 7f416ea00640 1 -- 172.21.15.152:0/3495956903 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f416800a4a0 con 0x7f4154004d40 2024-04-20T11:32:47.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.232+0000 7f416ea00640 1 -- 172.21.15.152:0/3495956903 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4168034650 con 0x7f4154004d40 2024-04-20T11:32:47.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.232+0000 7f416ea00640 1 -- 172.21.15.152:0/3495956903 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4168034a40 con 0x7f4154004d40 2024-04-20T11:32:47.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.232+0000 7f4176600640 1 -- 172.21.15.152:0/3495956903 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4154004d40 msgr2=0x7f41540051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:47.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.232+0000 7f4176600640 1 --2- 172.21.15.152:0/3495956903 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4154004d40 0x7f41540051c0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f41680013d0 tx=0x7f4154005830 comp rx=0 tx=0).stop 2024-04-20T11:32:47.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.232+0000 7f416fe00640 1 -- 172.21.15.152:0/3495956903 reap_dead start 2024-04-20T11:32:47.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.232+0000 7f4176600640 1 -- 172.21.15.152:0/3495956903 shutdown_connections 2024-04-20T11:32:47.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.232+0000 7f4176600640 1 -- 172.21.15.152:0/3495956903 >> 172.21.15.152:0/3495956903 conn(0x7f417006b980 msgr2=0x7f417006bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:47.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.232+0000 7f4176600640 1 -- 172.21.15.152:0/3495956903 shutdown_connections 2024-04-20T11:32:47.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.232+0000 7f4176600640 1 -- 172.21.15.152:0/3495956903 wait complete. 2024-04-20T11:32:47.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.234+0000 7f4176600640 1 Processor -- start 2024-04-20T11:32:47.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.234+0000 7f4176600640 1 -- start start 2024-04-20T11:32:47.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.235+0000 7f4176600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4154004d40 0x7f417015c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:47.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.235+0000 7f4176600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4170132ff0 0x7f41701f8570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:47.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.235+0000 7f4176600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f41700ae2a0 0x7f41700ae720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:47.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.235+0000 7f4176600640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f417006e740 con 0x7f41700ae2a0 2024-04-20T11:32:47.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.235+0000 7f4176600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f417006f070 con 0x7f4170132ff0 2024-04-20T11:32:47.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.235+0000 7f4176600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f417006ea20 con 0x7f4154004d40 2024-04-20T11:32:47.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.235+0000 7f416fe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f41700ae2a0 0x7f41700ae720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:47.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.235+0000 7f416fe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f41700ae2a0 0x7f41700ae720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:44964/0 (socket says 172.21.15.152:44964) 2024-04-20T11:32:47.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.235+0000 7f416fe00640 1 -- 172.21.15.152:0/1058260646 learned_addr learned my addr 172.21.15.152:0/1058260646 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:32:47.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.236+0000 7f416f400640 1 --2- 172.21.15.152:0/1058260646 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4154004d40 0x7f417015c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:47.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.236+0000 7f4167e00640 1 --2- 172.21.15.152:0/1058260646 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4170132ff0 0x7f41701f8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:47.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.236+0000 7f416fe00640 1 -- 172.21.15.152:0/1058260646 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4154004d40 msgr2=0x7f417015c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:47.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.236+0000 7f416fe00640 1 --2- 172.21.15.152:0/1058260646 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4154004d40 0x7f417015c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:47.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.236+0000 7f416fe00640 1 -- 172.21.15.152:0/1058260646 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4170132ff0 msgr2=0x7f41701f8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:47.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.236+0000 7f416fe00640 1 --2- 172.21.15.152:0/1058260646 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4170132ff0 0x7f41701f8570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:47.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.236+0000 7f416fe00640 1 -- 172.21.15.152:0/1058260646 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f417015f360 con 0x7f41700ae2a0 2024-04-20T11:32:47.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.236+0000 7f4167e00640 1 --2- 172.21.15.152:0/1058260646 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4170132ff0 0x7f41701f8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:32:47.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.236+0000 7f416fe00640 1 --2- 172.21.15.152:0/1058260646 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f41700ae2a0 0x7f41700ae720 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f416802cc20 tx=0x7f4168001360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:47.237 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.236+0000 7f4167400640 1 -- 172.21.15.152:0/1058260646 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4168002000 con 0x7f41700ae2a0 2024-04-20T11:32:47.237 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.236+0000 7f4167400640 1 -- 172.21.15.152:0/1058260646 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4168044070 con 0x7f41700ae2a0 2024-04-20T11:32:47.237 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.236+0000 7f4167400640 1 -- 172.21.15.152:0/1058260646 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4168034a20 con 0x7f41700ae2a0 2024-04-20T11:32:47.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.236+0000 7f4176600640 1 -- 172.21.15.152:0/1058260646 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41701f8ab0 con 0x7f41700ae2a0 2024-04-20T11:32:47.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.237+0000 7f4176600640 1 -- 172.21.15.152:0/1058260646 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f417015f8c0 con 0x7f41700ae2a0 2024-04-20T11:32:47.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.238+0000 7f4176600640 1 -- 172.21.15.152:0/1058260646 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41700689f0 con 0x7f41700ae2a0 2024-04-20T11:32:47.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.239+0000 7f4167400640 1 -- 172.21.15.152:0/1058260646 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 91142+0+0 (secure 0 0 0) 0x7f4168034bc0 con 0x7f41700ae2a0 2024-04-20T11:32:47.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.240+0000 7f4167400640 1 --2- 172.21.15.152:0/1058260646 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f414c06ca10 0x7f414c06eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:47.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.240+0000 7f4167400640 1 -- 172.21.15.152:0/1058260646 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7f41680ae3a0 con 0x7f41700ae2a0 2024-04-20T11:32:47.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.241+0000 7f416f400640 1 --2- 172.21.15.152:0/1058260646 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f414c06ca10 0x7f414c06eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:47.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.242+0000 7f416f400640 1 --2- 172.21.15.152:0/1058260646 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f414c06ca10 0x7f414c06eed0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4160009a10 tx=0x7f4160008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:47.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.245+0000 7f4167400640 1 -- 172.21.15.152:0/1058260646 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f417015f360 con 0x7f41700ae2a0 2024-04-20T11:32:47.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.518+0000 7f4176600640 1 -- 172.21.15.152:0/1058260646 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f41700d1050 con 0x7f41700ae2a0 2024-04-20T11:32:47.522 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.523+0000 7f4167400640 1 -- 172.21.15.152:0/1058260646 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) ==== 190+0+147 (secure 0 0 0) 0x7f4168032380 con 0x7f41700ae2a0 2024-04-20T11:32:47.522 INFO:teuthology.orchestra.run.smithi152.stdout:[client.0] 2024-04-20T11:32:47.523 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQBDpiNmtsBWFhAArq9A7r33CcwgMmOU2pAGPw== 2024-04-20T11:32:47.523 INFO:teuthology.orchestra.run.smithi152.stdout: caps mds = "allow" 2024-04-20T11:32:47.523 INFO:teuthology.orchestra.run.smithi152.stdout: caps mon = "allow r" 2024-04-20T11:32:47.523 INFO:teuthology.orchestra.run.smithi152.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-20T11:32:47.523 INFO:teuthology.orchestra.run.smithi152.stderr:updated caps for client.0 2024-04-20T11:32:47.526 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.526+0000 7f4176600640 1 -- 172.21.15.152:0/1058260646 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f414c06ca10 msgr2=0x7f414c06eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:47.526 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.526+0000 7f4176600640 1 --2- 172.21.15.152:0/1058260646 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f414c06ca10 0x7f414c06eed0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4160009a10 tx=0x7f4160008040 comp rx=0 tx=0).stop 2024-04-20T11:32:47.526 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.526+0000 7f4176600640 1 -- 172.21.15.152:0/1058260646 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f41700ae2a0 msgr2=0x7f41700ae720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:47.526 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.526+0000 7f4176600640 1 --2- 172.21.15.152:0/1058260646 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f41700ae2a0 0x7f41700ae720 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f416802cc20 tx=0x7f4168001360 comp rx=0 tx=0).stop 2024-04-20T11:32:47.526 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.526+0000 7f4176600640 1 -- 172.21.15.152:0/1058260646 shutdown_connections 2024-04-20T11:32:47.526 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.526+0000 7f4176600640 1 --2- 172.21.15.152:0/1058260646 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f414c06ca10 0x7f414c06eed0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:47.526 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.526+0000 7f4176600640 1 --2- 172.21.15.152:0/1058260646 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f41700ae2a0 0x7f41700ae720 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:47.526 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.526+0000 7f4176600640 1 --2- 172.21.15.152:0/1058260646 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4170132ff0 0x7f41701f8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:47.526 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.526+0000 7f4176600640 1 --2- 172.21.15.152:0/1058260646 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4154004d40 0x7f417015c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:47.527 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.526+0000 7f4176600640 1 -- 172.21.15.152:0/1058260646 >> 172.21.15.152:0/1058260646 conn(0x7f417006b980 msgr2=0x7f41700b1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:47.527 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.526+0000 7f4176600640 1 -- 172.21.15.152:0/1058260646 shutdown_connections 2024-04-20T11:32:47.527 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.526+0000 7f4176600640 1 -- 172.21.15.152:0/1058260646 wait complete. 2024-04-20T11:32:47.546 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:32:47.747 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.748+0000 7fbc61c00640 1 Processor -- start 2024-04-20T11:32:47.748 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.748+0000 7fbc61c00640 1 -- start start 2024-04-20T11:32:47.749 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.750+0000 7fbc61c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbc5c0d1350 0x7fbc5c0d1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:47.749 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.750+0000 7fbc61c00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fbc5c15b9a0 0x7fbc5c0b43e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:47.749 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.750+0000 7fbc61c00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc5c0ae8f0 con 0x7fbc5c0b4920 2024-04-20T11:32:47.749 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.750+0000 7fbc61c00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fbc5c0d1c70 con 0x7fbc5c15b9a0 2024-04-20T11:32:47.750 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.750+0000 7fbc61c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fbc5c0b25c0 con 0x7fbc5c0d1350 2024-04-20T11:32:47.750 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.750+0000 7fbc5b400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fbc5c0b4920 0x7fbc5c0b0d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:41156/0 (socket says 172.21.15.152:41156) 2024-04-20T11:32:47.750 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.750+0000 7fbc5b400640 1 -- 172.21.15.152:0/3627085278 learned_addr learned my addr 172.21.15.152:0/3627085278 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:32:47.751 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.750+0000 7fbc5a000640 1 --2- 172.21.15.152:0/3627085278 >> v2:172.21.15.175:3300/0 conn(0x7fbc5c15b9a0 0x7fbc5c0b43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:47.751 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.750+0000 7fbc5aa00640 1 --2- 172.21.15.152:0/3627085278 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbc5c0d1350 0x7fbc5c0d1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:47.751 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.751+0000 7fbc59600640 1 -- 172.21.15.152:0/3627085278 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3221980232 0 0) 0x7fbc5c0ae8f0 con 0x7fbc5c0b4920 2024-04-20T11:32:47.751 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.751+0000 7fbc59600640 1 -- 172.21.15.152:0/3627085278 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbc40005730 con 0x7fbc5c0b4920 2024-04-20T11:32:47.751 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.751+0000 7fbc5a000640 1 -- 172.21.15.152:0/3627085278 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbc5c0d1350 msgr2=0x7fbc5c0d1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:47.751 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.751+0000 7fbc5a000640 1 --2- 172.21.15.152:0/3627085278 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbc5c0d1350 0x7fbc5c0d1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:47.751 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.751+0000 7fbc59600640 1 -- 172.21.15.152:0/3627085278 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3580923430 0 0) 0x7fbc40005730 con 0x7fbc5c0b4920 2024-04-20T11:32:47.751 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.751+0000 7fbc5a000640 1 -- 172.21.15.152:0/3627085278 >> v1:172.21.15.152:6789/0 conn(0x7fbc5c0b4920 legacy=0x7fbc5c0b0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:47.751 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.751+0000 7fbc5a000640 1 -- 172.21.15.152:0/3627085278 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc5c0b28a0 con 0x7fbc5c15b9a0 2024-04-20T11:32:47.752 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.752+0000 7fbc5a000640 1 --2- 172.21.15.152:0/3627085278 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbc5c15b9a0 0x7fbc5c0b43e0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c0097a0 tx=0x7fbc4c02b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=458680c7ecc8dfba server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:47.752 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.753+0000 7fbc59600640 1 -- 172.21.15.152:0/3627085278 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc4c037070 con 0x7fbc5c15b9a0 2024-04-20T11:32:47.752 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.753+0000 7fbc59600640 1 -- 172.21.15.152:0/3627085278 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc4c030510 con 0x7fbc5c15b9a0 2024-04-20T11:32:47.752 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.753+0000 7fbc59600640 1 -- 172.21.15.152:0/3627085278 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc44001670 con 0x7fbc5c15b9a0 2024-04-20T11:32:47.753 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.753+0000 7fbc61c00640 1 -- 172.21.15.152:0/3627085278 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbc5c15b9a0 msgr2=0x7fbc5c0b43e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:47.753 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.753+0000 7fbc61c00640 1 --2- 172.21.15.152:0/3627085278 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbc5c15b9a0 0x7fbc5c0b43e0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c0097a0 tx=0x7fbc4c02b790 comp rx=0 tx=0).stop 2024-04-20T11:32:47.753 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.754+0000 7fbc61c00640 1 -- 172.21.15.152:0/3627085278 shutdown_connections 2024-04-20T11:32:47.753 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.754+0000 7fbc61c00640 1 --2- 172.21.15.152:0/3627085278 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbc5c15b9a0 0x7fbc5c0b43e0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:47.753 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.754+0000 7fbc61c00640 1 --2- 172.21.15.152:0/3627085278 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbc5c0d1350 0x7fbc5c0d1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:47.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.754+0000 7fbc61c00640 1 -- 172.21.15.152:0/3627085278 >> 172.21.15.152:0/3627085278 conn(0x7fbc5c0b6870 msgr2=0x7fbc5c0b6c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:47.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.754+0000 7fbc61c00640 1 -- 172.21.15.152:0/3627085278 shutdown_connections 2024-04-20T11:32:47.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.754+0000 7fbc61c00640 1 -- 172.21.15.152:0/3627085278 wait complete. 2024-04-20T11:32:47.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.755+0000 7fbc61c00640 1 Processor -- start 2024-04-20T11:32:47.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.756+0000 7fbc61c00640 1 -- start start 2024-04-20T11:32:47.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.757+0000 7fbc61c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc5c0b4920 0x7fbc5c142430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:47.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.757+0000 7fbc61c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbc5c0d1350 0x7fbc5c15d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:47.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.757+0000 7fbc61c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbc5c15b9a0 0x7fbc5c15d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:47.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.757+0000 7fbc61c00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fbc5c0b21e0 con 0x7fbc5c0b4920 2024-04-20T11:32:47.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.757+0000 7fbc61c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbc5c0aeb90 con 0x7fbc5c15b9a0 2024-04-20T11:32:47.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.757+0000 7fbc61c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fbc5c0b2360 con 0x7fbc5c0d1350 2024-04-20T11:32:47.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.757+0000 7fbc5aa00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc5c0b4920 0x7fbc5c142430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:47.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.757+0000 7fbc5aa00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc5c0b4920 0x7fbc5c142430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:44972/0 (socket says 172.21.15.152:44972) 2024-04-20T11:32:47.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.757+0000 7fbc5aa00640 1 -- 172.21.15.152:0/1198231857 learned_addr learned my addr 172.21.15.152:0/1198231857 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:32:47.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.757+0000 7fbc5a000640 1 --2- 172.21.15.152:0/1198231857 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbc5c0d1350 0x7fbc5c15d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:47.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.757+0000 7fbc5aa00640 1 -- 172.21.15.152:0/1198231857 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbc5c0d1350 msgr2=0x7fbc5c15d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:47.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.757+0000 7fbc5b400640 1 --2- 172.21.15.152:0/1198231857 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbc5c15b9a0 0x7fbc5c15d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:47.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.757+0000 7fbc5aa00640 1 --2- 172.21.15.152:0/1198231857 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbc5c0d1350 0x7fbc5c15d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:47.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.757+0000 7fbc5aa00640 1 -- 172.21.15.152:0/1198231857 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbc5c15b9a0 msgr2=0x7fbc5c15d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:47.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.757+0000 7fbc5aa00640 1 --2- 172.21.15.152:0/1198231857 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbc5c15b9a0 0x7fbc5c15d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:47.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.757+0000 7fbc5aa00640 1 -- 172.21.15.152:0/1198231857 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc5c144d60 con 0x7fbc5c0b4920 2024-04-20T11:32:47.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.757+0000 7fbc5b400640 1 --2- 172.21.15.152:0/1198231857 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbc5c15b9a0 0x7fbc5c15d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:32:47.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.758+0000 7fbc5aa00640 1 --2- 172.21.15.152:0/1198231857 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc5c0b4920 0x7fbc5c142430 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7fbc54009db0 tx=0x7fbc5400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:47.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.758+0000 7fbc52a00640 1 -- 172.21.15.152:0/1198231857 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc54018020 con 0x7fbc5c0b4920 2024-04-20T11:32:47.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.758+0000 7fbc52a00640 1 -- 172.21.15.152:0/1198231857 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc54014480 con 0x7fbc5c0b4920 2024-04-20T11:32:47.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.758+0000 7fbc61c00640 1 -- 172.21.15.152:0/1198231857 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc5c145000 con 0x7fbc5c0b4920 2024-04-20T11:32:47.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.758+0000 7fbc52a00640 1 -- 172.21.15.152:0/1198231857 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc54014850 con 0x7fbc5c0b4920 2024-04-20T11:32:47.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.758+0000 7fbc61c00640 1 -- 172.21.15.152:0/1198231857 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc5c1455c0 con 0x7fbc5c0b4920 2024-04-20T11:32:47.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.759+0000 7fbc61c00640 1 -- 172.21.15.152:0/1198231857 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc5c0738f0 con 0x7fbc5c0b4920 2024-04-20T11:32:47.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.760+0000 7fbc52a00640 1 -- 172.21.15.152:0/1198231857 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 91142+0+0 (secure 0 0 0) 0x7fbc54005d10 con 0x7fbc5c0b4920 2024-04-20T11:32:47.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.761+0000 7fbc52a00640 1 --2- 172.21.15.152:0/1198231857 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fbc3806ca10 0x7fbc3806eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:47.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.761+0000 7fbc52a00640 1 -- 172.21.15.152:0/1198231857 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7fbc5408c900 con 0x7fbc5c0b4920 2024-04-20T11:32:47.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.762+0000 7fbc5a000640 1 --2- 172.21.15.152:0/1198231857 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fbc3806ca10 0x7fbc3806eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:47.762 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.763+0000 7fbc5a000640 1 --2- 172.21.15.152:0/1198231857 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fbc3806ca10 0x7fbc3806eed0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c034000 tx=0x7fbc4c02f000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:47.765 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:47.766+0000 7fbc52a00640 1 -- 172.21.15.152:0/1198231857 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fbc5c0738f0 con 0x7fbc5c0b4920 2024-04-20T11:32:48.010 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.011+0000 7fbc61c00640 1 -- 172.21.15.152:0/1198231857 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbc5c075e60 con 0x7fbc5c0b4920 2024-04-20T11:32:48.012 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.012+0000 7fbc52a00640 1 -- 172.21.15.152:0/1198231857 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) ==== 94+0+5210 (secure 0 0 0) 0x7fbc5408c070 con 0x7fbc5c0b4920 2024-04-20T11:32:48.012 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:32:48.012 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":53,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5465,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/3844264695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":3844264695},{"type":"v1","addr":"172.21.15.152:6835","nonce":3844264695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":5468,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/706098839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":706098839},{"type":"v1","addr":"172.21.15.175:6833","nonce":706098839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":5899,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/2632275131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":2632275131},{"type":"v1","addr":"172.21.15.175:6835","nonce":2632275131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T11:32:41.508684+0000","modified":"2024-04-20T11:32:44.600495+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7332},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7332":{"gid":7332,"name":"c","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6833/3577228528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":3577228528},{"type":"v1","addr":"172.21.15.152:6833","nonce":3577228528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7332,"qdb_cluster":[7332]},"id":6}]} 2024-04-20T11:32:48.012 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 53 2024-04-20T11:32:48.015 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.015+0000 7fbc61c00640 1 -- 172.21.15.152:0/1198231857 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fbc3806ca10 msgr2=0x7fbc3806eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:48.015 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.015+0000 7fbc61c00640 1 --2- 172.21.15.152:0/1198231857 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fbc3806ca10 0x7fbc3806eed0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c034000 tx=0x7fbc4c02f000 comp rx=0 tx=0).stop 2024-04-20T11:32:48.015 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.015+0000 7fbc61c00640 1 -- 172.21.15.152:0/1198231857 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc5c0b4920 msgr2=0x7fbc5c142430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:48.015 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.015+0000 7fbc61c00640 1 --2- 172.21.15.152:0/1198231857 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc5c0b4920 0x7fbc5c142430 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7fbc54009db0 tx=0x7fbc5400c490 comp rx=0 tx=0).stop 2024-04-20T11:32:48.015 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.015+0000 7fbc61c00640 1 -- 172.21.15.152:0/1198231857 shutdown_connections 2024-04-20T11:32:48.015 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.015+0000 7fbc61c00640 1 --2- 172.21.15.152:0/1198231857 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fbc3806ca10 0x7fbc3806eed0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:48.015 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.015+0000 7fbc61c00640 1 --2- 172.21.15.152:0/1198231857 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbc5c15b9a0 0x7fbc5c15d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:48.015 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.015+0000 7fbc61c00640 1 --2- 172.21.15.152:0/1198231857 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbc5c0d1350 0x7fbc5c15d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:48.015 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.015+0000 7fbc61c00640 1 --2- 172.21.15.152:0/1198231857 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc5c0b4920 0x7fbc5c142430 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:48.015 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.015+0000 7fbc61c00640 1 -- 172.21.15.152:0/1198231857 >> 172.21.15.152:0/1198231857 conn(0x7fbc5c0b6870 msgr2=0x7fbc5c074c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:48.015 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.015+0000 7fbc61c00640 1 -- 172.21.15.152:0/1198231857 shutdown_connections 2024-04-20T11:32:48.015 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.015+0000 7fbc61c00640 1 -- 172.21.15.152:0/1198231857 wait complete. 2024-04-20T11:32:48.439 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 53, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-20T11:32:41.508684+0000', 'modified': '2024-04-20T11:32:44.600495+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'snap_rstat': True, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7332}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7332': {'gid': 7332, 'name': 'c', 'rank': 0, 'incarnation': 49, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.152:6833/3577228528', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6832', 'nonce': 3577228528}, {'type': 'v1', 'addr': '172.21.15.152:6833', 'nonce': 3577228528}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7332, 'qdb_cluster': [7332]} 2024-04-20T11:32:48.440 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-20T11:32:48.440 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-20T11:32:48.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.655+0000 7fe4d3e00640 1 Processor -- start 2024-04-20T11:32:48.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.656+0000 7fe4d3e00640 1 -- start start 2024-04-20T11:32:48.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.657+0000 7fe4d3e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe4cc0d0700 0x7fe4cc06f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:48.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.657+0000 7fe4d3e00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4cc0b3a50 con 0x7fe4cc162b40 2024-04-20T11:32:48.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.657+0000 7fe4d3e00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4cc06e300 con 0x7fe4cc0b3470 2024-04-20T11:32:48.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.657+0000 7fe4d3e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe4cc06e0b0 con 0x7fe4cc0d0700 2024-04-20T11:32:48.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.658+0000 7fe4d1400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fe4cc162b40 0x7fe4cc06ede0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:41166/0 (socket says 172.21.15.152:41166) 2024-04-20T11:32:48.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.658+0000 7fe4d1400640 1 -- 172.21.15.152:0/203066034 learned_addr learned my addr 172.21.15.152:0/203066034 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:32:48.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.658+0000 7fe4d1e00640 1 --2- 172.21.15.152:0/203066034 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe4cc0d0700 0x7fe4cc06f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:48.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.658+0000 7fe4cbe00640 1 -- 172.21.15.152:0/203066034 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2181737662 0 0) 0x7fe4cc0b3a50 con 0x7fe4cc162b40 2024-04-20T11:32:48.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.658+0000 7fe4cbe00640 1 -- 172.21.15.152:0/203066034 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe4b4003520 con 0x7fe4cc162b40 2024-04-20T11:32:48.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.659+0000 7fe4cbe00640 1 -- 172.21.15.152:0/203066034 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2149563319 0 0) 0x7fe4b4003520 con 0x7fe4cc162b40 2024-04-20T11:32:48.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.659+0000 7fe4cbe00640 1 -- 172.21.15.152:0/203066034 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe4cc0d0700 msgr2=0x7fe4cc06f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:48.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.659+0000 7fe4cbe00640 1 --2- 172.21.15.152:0/203066034 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe4cc0d0700 0x7fe4cc06f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:48.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.659+0000 7fe4cbe00640 1 -- 172.21.15.152:0/203066034 >> v1:172.21.15.175:6789/0 conn(0x7fe4cc0b3470 legacy=0x7fe4cc0cfff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:48.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.659+0000 7fe4cbe00640 1 -- 172.21.15.152:0/203066034 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4cc06e5e0 con 0x7fe4cc162b40 2024-04-20T11:32:48.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.659+0000 7fe4cbe00640 1 -- 172.21.15.152:0/203066034 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7fe4c0001670 con 0x7fe4cc162b40 2024-04-20T11:32:48.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.659+0000 7fe4cbe00640 1 -- 172.21.15.152:0/203066034 >> v1:172.21.15.152:6789/0 conn(0x7fe4cc162b40 legacy=0x7fe4cc06ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:48.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.659+0000 7fe4cbe00640 1 --2- 172.21.15.152:0/203066034 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe4cc0d0700 0x7fe4b40039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:48.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.659+0000 7fe4cbe00640 1 --2- 172.21.15.152:0/203066034 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe4b4003f20 0x7fe4b4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:48.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.659+0000 7fe4cbe00640 1 --2- 172.21.15.152:0/203066034 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe4b4004d40 0x7fe4b40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:48.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.659+0000 7fe4cbe00640 1 -- 172.21.15.152:0/203066034 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fe4cc06e0b0 con 0x7fe4b4003f20 2024-04-20T11:32:48.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.659+0000 7fe4cbe00640 1 -- 172.21.15.152:0/203066034 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b400bb00 con 0x7fe4b4004d40 2024-04-20T11:32:48.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.659+0000 7fe4cbe00640 1 -- 172.21.15.152:0/203066034 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe4b400bd00 con 0x7fe4cc0d0700 2024-04-20T11:32:48.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.659+0000 7fe4d0a00640 1 --2- 172.21.15.152:0/203066034 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe4b4003f20 0x7fe4b4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:48.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.659+0000 7fe4d1400640 1 --2- 172.21.15.152:0/203066034 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe4cc0d0700 0x7fe4b40039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:48.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.660+0000 7fe4d1e00640 1 --2- 172.21.15.152:0/203066034 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe4b4004d40 0x7fe4b40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:48.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.660+0000 7fe4d0a00640 1 -- 172.21.15.152:0/203066034 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe4cc0d0700 msgr2=0x7fe4b40039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:48.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.660+0000 7fe4d0a00640 1 --2- 172.21.15.152:0/203066034 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe4cc0d0700 0x7fe4b40039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:48.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.660+0000 7fe4d0a00640 1 -- 172.21.15.152:0/203066034 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe4b4004d40 msgr2=0x7fe4b40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:48.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.660+0000 7fe4d0a00640 1 --2- 172.21.15.152:0/203066034 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe4b4004d40 0x7fe4b40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:48.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.660+0000 7fe4d0a00640 1 -- 172.21.15.152:0/203066034 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4b4003520 con 0x7fe4b4003f20 2024-04-20T11:32:48.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.661+0000 7fe4d0a00640 1 --2- 172.21.15.152:0/203066034 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe4b4003f20 0x7fe4b4004800 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fe4b8001860 tx=0x7fe4b4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74eecb77ff26f8e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:48.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.661+0000 7fe4cbe00640 1 -- 172.21.15.152:0/203066034 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b8038070 con 0x7fe4b4003f20 2024-04-20T11:32:48.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.661+0000 7fe4cbe00640 1 -- 172.21.15.152:0/203066034 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4b8033410 con 0x7fe4b4003f20 2024-04-20T11:32:48.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.661+0000 7fe4cbe00640 1 -- 172.21.15.152:0/203066034 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b8001430 con 0x7fe4b4003f20 2024-04-20T11:32:48.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.662+0000 7fe4d3e00640 1 -- 172.21.15.152:0/203066034 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe4b4003f20 msgr2=0x7fe4b4004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:48.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.662+0000 7fe4d3e00640 1 --2- 172.21.15.152:0/203066034 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe4b4003f20 0x7fe4b4004800 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fe4b8001860 tx=0x7fe4b4005830 comp rx=0 tx=0).stop 2024-04-20T11:32:48.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.662+0000 7fe4d1e00640 1 -- 172.21.15.152:0/203066034 reap_dead start 2024-04-20T11:32:48.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.662+0000 7fe4d3e00640 1 -- 172.21.15.152:0/203066034 shutdown_connections 2024-04-20T11:32:48.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.662+0000 7fe4d3e00640 1 -- 172.21.15.152:0/203066034 >> 172.21.15.152:0/203066034 conn(0x7fe4cc067cf0 msgr2=0x7fe4cc06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:48.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.662+0000 7fe4d3e00640 1 -- 172.21.15.152:0/203066034 shutdown_connections 2024-04-20T11:32:48.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.662+0000 7fe4d3e00640 1 -- 172.21.15.152:0/203066034 wait complete. 2024-04-20T11:32:48.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.664+0000 7fe4d3e00640 1 Processor -- start 2024-04-20T11:32:48.664 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.664+0000 7fe4d3e00640 1 -- start start 2024-04-20T11:32:48.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.665+0000 7fe4d3e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe4b4003f20 0x7fe4cc155590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:48.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.665+0000 7fe4d3e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe4cc165990 0x7fe4cc1f8880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:48.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.665+0000 7fe4d3e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe4cc165e10 0x7fe4cc1f8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:48.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.665+0000 7fe4d3e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fe4cc06e950 con 0x7fe4b4003f20 2024-04-20T11:32:48.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.665+0000 7fe4d3e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe4cc06f810 con 0x7fe4cc165e10 2024-04-20T11:32:48.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.665+0000 7fe4d3e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe4cc06f990 con 0x7fe4cc165990 2024-04-20T11:32:48.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.665+0000 7fe4d1400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe4b4003f20 0x7fe4cc155590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:48.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.665+0000 7fe4d1400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe4b4003f20 0x7fe4cc155590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:44994/0 (socket says 172.21.15.152:44994) 2024-04-20T11:32:48.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.665+0000 7fe4d1400640 1 -- 172.21.15.152:0/3476246143 learned_addr learned my addr 172.21.15.152:0/3476246143 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:32:48.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.665+0000 7fe4d1400640 1 -- 172.21.15.152:0/3476246143 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe4cc165990 msgr2=0x7fe4cc1f8880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:48.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.665+0000 7fe4d1400640 1 --2- 172.21.15.152:0/3476246143 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe4cc165990 0x7fe4cc1f8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:48.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.665+0000 7fe4d1400640 1 -- 172.21.15.152:0/3476246143 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe4cc165e10 msgr2=0x7fe4cc1f8dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:48.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.665+0000 7fe4d1400640 1 --2- 172.21.15.152:0/3476246143 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe4cc165e10 0x7fe4cc1f8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:48.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.665+0000 7fe4d1400640 1 -- 172.21.15.152:0/3476246143 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4cc1eb630 con 0x7fe4b4003f20 2024-04-20T11:32:48.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.666+0000 7fe4d1400640 1 --2- 172.21.15.152:0/3476246143 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe4b4003f20 0x7fe4cc155590 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fe4c000e790 tx=0x7fe4c000ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:48.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.666+0000 7fe4c9600640 1 -- 172.21.15.152:0/3476246143 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c000c5e0 con 0x7fe4b4003f20 2024-04-20T11:32:48.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.666+0000 7fe4c9600640 1 -- 172.21.15.152:0/3476246143 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4c0018070 con 0x7fe4b4003f20 2024-04-20T11:32:48.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.666+0000 7fe4c9600640 1 -- 172.21.15.152:0/3476246143 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c000c9c0 con 0x7fe4b4003f20 2024-04-20T11:32:48.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.666+0000 7fe4d3e00640 1 -- 172.21.15.152:0/3476246143 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4cc1eb910 con 0x7fe4b4003f20 2024-04-20T11:32:48.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.666+0000 7fe4d3e00640 1 -- 172.21.15.152:0/3476246143 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4cc1ebed0 con 0x7fe4b4003f20 2024-04-20T11:32:48.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.668+0000 7fe4c9600640 1 -- 172.21.15.152:0/3476246143 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 91142+0+0 (secure 0 0 0) 0x7fe4c000cb60 con 0x7fe4b4003f20 2024-04-20T11:32:48.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.669+0000 7fe4c9600640 1 --2- 172.21.15.152:0/3476246143 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe4ac06ca10 0x7fe4ac06eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:48.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.669+0000 7fe4c9600640 1 -- 172.21.15.152:0/3476246143 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7fe4c008b9a0 con 0x7fe4b4003f20 2024-04-20T11:32:48.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.670+0000 7fe4d0a00640 1 --2- 172.21.15.152:0/3476246143 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe4ac06ca10 0x7fe4ac06eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:48.670 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.670+0000 7fe4d6baf740 1 Processor -- start 2024-04-20T11:32:48.670 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.670+0000 7fe4d6baf740 1 -- start start 2024-04-20T11:32:48.672 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.671+0000 7fe4d0a00640 1 --2- 172.21.15.152:0/3476246143 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe4ac06ca10 0x7fe4ac06eed0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe4b8035000 tx=0x7fe4b8030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:48.672 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.671+0000 7fe4d6baf740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55745f751460 0x55745f708a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:48.672 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.671+0000 7fe4d6baf740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55745f7074e0 0x55745f708fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:48.672 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.671+0000 7fe4d6baf740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55745f703930 0x55745f733770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:48.672 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.671+0000 7fe4d6baf740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55745f66f9d0 con 0x55745f703930 2024-04-20T11:32:48.672 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.671+0000 7fe4d6baf740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55745f616270 con 0x55745f751460 2024-04-20T11:32:48.672 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.671+0000 7fe4d6baf740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x55745f66b990 con 0x55745f7074e0 2024-04-20T11:32:48.672 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.671+0000 7fe4d1e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55745f703930 0x55745f733770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:48.672 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.671+0000 7fe4d1e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55745f703930 0x55745f733770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:44996/0 (socket says 172.21.15.152:44996) 2024-04-20T11:32:48.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.672+0000 7fe4d1e00640 1 -- 172.21.15.152:0/1456289177 learned_addr learned my addr 172.21.15.152:0/1456289177 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:32:48.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.672+0000 7fe4d1400640 1 --2- 172.21.15.152:0/1456289177 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55745f751460 0x55745f708a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:48.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.672+0000 7fe4d0a00640 1 --2- 172.21.15.152:0/1456289177 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55745f7074e0 0x55745f708fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:48.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.672+0000 7fe4d1e00640 1 -- 172.21.15.152:0/1456289177 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55745f7074e0 msgr2=0x55745f708fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:48.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.672+0000 7fe4d1e00640 1 --2- 172.21.15.152:0/1456289177 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55745f7074e0 0x55745f708fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:48.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.672+0000 7fe4d1e00640 1 -- 172.21.15.152:0/1456289177 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55745f751460 msgr2=0x55745f708a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:48.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.672+0000 7fe4d1e00640 1 --2- 172.21.15.152:0/1456289177 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55745f751460 0x55745f708a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:48.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.672+0000 7fe4d1e00640 1 -- 172.21.15.152:0/1456289177 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55745f3822d0 con 0x55745f703930 2024-04-20T11:32:48.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.672+0000 7fe4d1e00640 1 --2- 172.21.15.152:0/1456289177 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55745f703930 0x55745f733770 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc0cfa50 tx=0x7fe4c4007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=618fd10f4b8ec7a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:48.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.673+0000 7fe4bdc00640 1 -- 172.21.15.152:0/1456289177 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c4003ba0 con 0x55745f703930 2024-04-20T11:32:48.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.673+0000 7fe4bdc00640 1 -- 172.21.15.152:0/1456289177 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4c401a070 con 0x55745f703930 2024-04-20T11:32:48.674 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.673+0000 7fe4bdc00640 1 -- 172.21.15.152:0/1456289177 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c4003dc0 con 0x55745f703930 2024-04-20T11:32:48.677 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.673+0000 7fe4d6baf740 1 -- 172.21.15.152:0/1456289177 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55745f703930 msgr2=0x55745f733770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:48.677 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.673+0000 7fe4d6baf740 1 --2- 172.21.15.152:0/1456289177 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55745f703930 0x55745f733770 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc0cfa50 tx=0x7fe4c4007700 comp rx=0 tx=0).stop 2024-04-20T11:32:48.677 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.673+0000 7fe4d6baf740 1 -- 172.21.15.152:0/1456289177 shutdown_connections 2024-04-20T11:32:48.677 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.673+0000 7fe4d6baf740 1 --2- 172.21.15.152:0/1456289177 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55745f703930 0x55745f733770 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:48.678 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.673+0000 7fe4d6baf740 1 --2- 172.21.15.152:0/1456289177 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55745f7074e0 0x55745f708fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:48.678 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.673+0000 7fe4d6baf740 1 --2- 172.21.15.152:0/1456289177 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55745f751460 0x55745f708a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:48.678 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.673+0000 7fe4d6baf740 1 -- 172.21.15.152:0/1456289177 >> 172.21.15.152:0/1456289177 conn(0x55745f705bf0 msgr2=0x55745f74e770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:48.678 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.673+0000 7fe4d6baf740 1 -- 172.21.15.152:0/1456289177 shutdown_connections 2024-04-20T11:32:48.678 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.673+0000 7fe4d6baf740 1 -- 172.21.15.152:0/1456289177 wait complete. 2024-04-20T11:32:48.678 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.674+0000 7fe4d6baf740 1 -- start start 2024-04-20T11:32:48.678 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.674+0000 7fe4d6baf740 1 Processor -- start 2024-04-20T11:32:48.678 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.675+0000 7fe4d6baf740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55745f7074e0 0x55745f731ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:48.678 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.675+0000 7fe4d6baf740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55745f751460 0x55745f732410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:48.678 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.675+0000 7fe4d6baf740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55745f72b780 0x55745f752e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:48.678 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.675+0000 7fe4d6baf740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55745f7079f0 con 0x55745f72b780 2024-04-20T11:32:48.678 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.675+0000 7fe4d6baf740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55745f6295e0 con 0x55745f751460 2024-04-20T11:32:48.678 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.675+0000 7fe4d6baf740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x55745f707cd0 con 0x55745f7074e0 2024-04-20T11:32:48.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.676+0000 7fe4d6baf740 10 client.0 _ll_register_callbacks cb 0x55745f381ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:32:48.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.676+0000 7fe4d1e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55745f72b780 0x55745f752e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:48.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.676+0000 7fe4d1e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55745f72b780 0x55745f752e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:45012/0 (socket says 172.21.15.152:45012) 2024-04-20T11:32:48.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.676+0000 7fe4d1e00640 1 -- 172.21.15.152:0/4285898204 learned_addr learned my addr 172.21.15.152:0/4285898204 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:32:48.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.676+0000 7fe4d1400640 1 --2- 172.21.15.152:0/4285898204 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55745f7074e0 0x55745f731ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:48.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.676+0000 7fe4d0a00640 1 --2- 172.21.15.152:0/4285898204 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55745f751460 0x55745f732410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:48.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.676+0000 7fe4d1e00640 1 -- 172.21.15.152:0/4285898204 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55745f7074e0 msgr2=0x55745f731ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:48.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.676+0000 7fe4d1e00640 1 --2- 172.21.15.152:0/4285898204 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55745f7074e0 0x55745f731ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:48.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.676+0000 7fe4d1e00640 1 -- 172.21.15.152:0/4285898204 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55745f751460 msgr2=0x55745f732410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:48.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.676+0000 7fe4d1e00640 1 --2- 172.21.15.152:0/4285898204 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55745f751460 0x55745f732410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:48.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.676+0000 7fe4d1e00640 1 -- 172.21.15.152:0/4285898204 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55745f44c6b0 con 0x55745f72b780 2024-04-20T11:32:48.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.677+0000 7fe4d1e00640 1 --2- 172.21.15.152:0/4285898204 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55745f72b780 0x55745f752e40 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc0b5b30 tx=0x7fe4c400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82edc03a8219d241 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:48.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.677+0000 7fe4b2000640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-04-20T11:32:48.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.677+0000 7fe4b2000640 1 -- 172.21.15.152:0/4285898204 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c4023070 con 0x55745f72b780 2024-04-20T11:32:48.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.677+0000 7fe4d6baf740 1 -- 172.21.15.152:0/4285898204 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4c401a070 con 0x55745f72b780 2024-04-20T11:32:48.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.677+0000 7fe4b2000640 1 -- 172.21.15.152:0/4285898204 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7fe4c4028660 con 0x55745f72b780 2024-04-20T11:32:48.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.677+0000 7fe4b2000640 1 -- 172.21.15.152:0/4285898204 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7fe4c4030050 con 0x55745f72b780 2024-04-20T11:32:48.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.677+0000 7fe4d6baf740 10 client.7470 fetch_fsmap learned FSMap version 53 2024-04-20T11:32:48.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.677+0000 7fe4d6baf740 1 -- 172.21.15.152:0/4285898204 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) -- 0x55745f755130 con 0x55745f72b780 2024-04-20T11:32:48.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.678+0000 7fe4b2000640 1 -- 172.21.15.152:0/4285898204 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4c401a070 con 0x55745f72b780 2024-04-20T11:32:48.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.678+0000 7fe4b2000640 1 -- 172.21.15.152:0/4285898204 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 53) ==== 3495+0+0 (secure 0 0 0) 0x7fe4c4033020 con 0x55745f72b780 2024-04-20T11:32:48.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.678+0000 7fe4d6baf740 10 client.7470 fetch_fsmap finished waiting for FSMap version 53 2024-04-20T11:32:48.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.678+0000 7fe4d6baf740 10 client.7470 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-20T11:32:48.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.678+0000 7fe4d6baf740 20 client.7470 populate_metadata read hostname 'smithi152' 2024-04-20T11:32:48.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.678+0000 7fe4d6baf740 1 --2- 172.21.15.152:0/4285898204 >> [v2:172.21.15.152:6832/3577228528,v1:172.21.15.152:6833/3577228528] conn(0x55745f753550 0x55745f756440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:48.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.678+0000 7fe4d6baf740 4 client.7470 mds_command: new command op to 7332 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:32:48.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.678+0000 7fe4d6baf740 1 -- 172.21.15.152:0/4285898204 --> [v2:172.21.15.152:6832/3577228528,v1:172.21.15.152:6833/3577228528] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55745f67d3f0 con 0x55745f753550 2024-04-20T11:32:48.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.678+0000 7fe4d1400640 1 --2- 172.21.15.152:0/4285898204 >> [v2:172.21.15.152:6832/3577228528,v1:172.21.15.152:6833/3577228528] conn(0x55745f753550 0x55745f756440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:48.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.679+0000 7fe4d1400640 1 --2- 172.21.15.152:0/4285898204 >> [v2:172.21.15.152:6832/3577228528,v1:172.21.15.152:6833/3577228528] conn(0x55745f753550 0x55745f756440 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9be80b02ceb82355 server_cookie=368b4ec98d969dfe in_seq=0 out_seq=0 2024-04-20T11:32:48.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.679+0000 7fe4b2000640 10 client.7470 ms_handle_connect on v2:172.21.15.152:6832/3577228528 2024-04-20T11:32:48.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.679+0000 7fe4b2000640 1 -- 172.21.15.152:0/4285898204 <== mds.0 v2:172.21.15.152:6832/3577228528 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55745f67d3f0 con 0x55745f753550 2024-04-20T11:32:48.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.679+0000 7fe4b2000640 10 client.7470 handle_command_reply: tid=0 2024-04-20T11:32:48.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.679+0000 7fe4d6baf740 1 client.7470 shutdown 2024-04-20T11:32:48.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.680+0000 7fe4d6baf740 1 -- 172.21.15.152:0/4285898204 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55745f72b780 msgr2=0x55745f752e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:48.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.680+0000 7fe4d6baf740 1 --2- 172.21.15.152:0/4285898204 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55745f72b780 0x55745f752e40 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc0b5b30 tx=0x7fe4c400b040 comp rx=0 tx=0).stop 2024-04-20T11:32:48.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.680+0000 7fe4d6baf740 1 -- 172.21.15.152:0/4285898204 shutdown_connections 2024-04-20T11:32:48.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.680+0000 7fe4d6baf740 1 --2- 172.21.15.152:0/4285898204 >> [v2:172.21.15.152:6832/3577228528,v1:172.21.15.152:6833/3577228528] conn(0x55745f753550 0x55745f756440 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:48.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.680+0000 7fe4d6baf740 1 --2- 172.21.15.152:0/4285898204 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55745f72b780 0x55745f752e40 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:48.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.680+0000 7fe4d6baf740 1 --2- 172.21.15.152:0/4285898204 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55745f751460 0x55745f732410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:48.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.680+0000 7fe4d6baf740 1 --2- 172.21.15.152:0/4285898204 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55745f7074e0 0x55745f731ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:48.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.680+0000 7fe4b2000640 0 client.7470 ms_handle_reset on v2:172.21.15.152:6832/3577228528 2024-04-20T11:32:48.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.680+0000 7fe4d6baf740 1 -- 172.21.15.152:0/4285898204 >> 172.21.15.152:0/4285898204 conn(0x55745f703930 msgr2=0x55745f7095a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:48.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.680+0000 7fe4d6baf740 1 -- 172.21.15.152:0/4285898204 shutdown_connections 2024-04-20T11:32:48.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.680+0000 7fe4d6baf740 1 -- 172.21.15.152:0/4285898204 wait complete. 2024-04-20T11:32:48.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.680+0000 7fe4d6baf740 20 client.7470 trim_cache size 0 max 16384 2024-04-20T11:32:48.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.700+0000 7fe4d6baf740 1 Processor -- start 2024-04-20T11:32:48.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.700+0000 7fe4d6baf740 1 -- start start 2024-04-20T11:32:48.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.701+0000 7fe4d6baf740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55745f7074e0 0x55745f752c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:48.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.701+0000 7fe4d6baf740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55745f72b780 0x55745f753e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:48.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.701+0000 7fe4d6baf740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55745f751460 0x55745f754390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:48.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.701+0000 7fe4d6baf740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fe4cc204710 con 0x55745f7074e0 2024-04-20T11:32:48.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.701+0000 7fe4d6baf740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe4cc163cd0 con 0x55745f751460 2024-04-20T11:32:48.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.701+0000 7fe4d6baf740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe4cc2040f0 con 0x55745f72b780 2024-04-20T11:32:48.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.701+0000 7fe4d1400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55745f7074e0 0x55745f752c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:48.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.701+0000 7fe4d1400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55745f7074e0 0x55745f752c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:45024/0 (socket says 172.21.15.152:45024) 2024-04-20T11:32:48.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.701+0000 7fe4d1400640 1 -- 172.21.15.152:0/609223220 learned_addr learned my addr 172.21.15.152:0/609223220 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:32:48.703 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.702+0000 7fe4d0a00640 1 --2- 172.21.15.152:0/609223220 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55745f72b780 0x55745f753e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:48.703 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.702+0000 7fe4d1e00640 1 --2- 172.21.15.152:0/609223220 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55745f751460 0x55745f754390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:48.703 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.702+0000 7fe4d1400640 1 -- 172.21.15.152:0/609223220 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55745f72b780 msgr2=0x55745f753e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:48.703 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.702+0000 7fe4d1400640 1 --2- 172.21.15.152:0/609223220 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55745f72b780 0x55745f753e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:48.703 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.702+0000 7fe4d1400640 1 -- 172.21.15.152:0/609223220 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55745f751460 msgr2=0x55745f754390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:48.703 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.702+0000 7fe4d1400640 1 --2- 172.21.15.152:0/609223220 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55745f751460 0x55745f754390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:48.703 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.702+0000 7fe4d1400640 1 -- 172.21.15.152:0/609223220 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55745f7531b0 con 0x55745f7074e0 2024-04-20T11:32:48.703 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.702+0000 7fe4d1e00640 1 --2- 172.21.15.152:0/609223220 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55745f751460 0x55745f754390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:32:48.703 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.702+0000 7fe4d1400640 1 --2- 172.21.15.152:0/609223220 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55745f7074e0 0x55745f752c70 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7fe4c00592d0 tx=0x7fe4c0015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce458ba5369330c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:48.704 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.702+0000 7fe4bd200640 1 -- 172.21.15.152:0/609223220 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c0055c40 con 0x55745f7074e0 2024-04-20T11:32:48.704 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.702+0000 7fe4bd200640 1 -- 172.21.15.152:0/609223220 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4c0076070 con 0x55745f7074e0 2024-04-20T11:32:48.704 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.702+0000 7fe4bd200640 1 -- 172.21.15.152:0/609223220 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c0055de0 con 0x55745f7074e0 2024-04-20T11:32:48.704 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.703+0000 7fe4d6baf740 1 -- 172.21.15.152:0/609223220 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55745f7074e0 msgr2=0x55745f752c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:48.704 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.703+0000 7fe4d6baf740 1 --2- 172.21.15.152:0/609223220 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55745f7074e0 0x55745f752c70 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7fe4c00592d0 tx=0x7fe4c0015040 comp rx=0 tx=0).stop 2024-04-20T11:32:48.704 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.703+0000 7fe4d6baf740 1 -- 172.21.15.152:0/609223220 shutdown_connections 2024-04-20T11:32:48.704 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.703+0000 7fe4d6baf740 1 --2- 172.21.15.152:0/609223220 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55745f751460 0x55745f754390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:48.704 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.703+0000 7fe4d6baf740 1 --2- 172.21.15.152:0/609223220 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55745f72b780 0x55745f753e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:48.704 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.703+0000 7fe4d6baf740 1 --2- 172.21.15.152:0/609223220 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55745f7074e0 0x55745f752c70 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:48.704 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.703+0000 7fe4d6baf740 1 -- 172.21.15.152:0/609223220 >> 172.21.15.152:0/609223220 conn(0x55745f703930 msgr2=0x55745f7089c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:48.704 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.703+0000 7fe4d6baf740 1 -- 172.21.15.152:0/609223220 shutdown_connections 2024-04-20T11:32:48.704 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.703+0000 7fe4d6baf740 1 -- 172.21.15.152:0/609223220 wait complete. 2024-04-20T11:32:48.707 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.704+0000 7fe4d6baf740 1 -- start start 2024-04-20T11:32:48.707 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.704+0000 7fe4d6baf740 1 Processor -- start 2024-04-20T11:32:48.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.705+0000 7fe4d6baf740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55745f7074e0 0x55745f7c86b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:48.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.705+0000 7fe4d6baf740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55745f72b780 0x55745f751840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:48.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.705+0000 7fe4d6baf740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55745f751460 0x55745f751d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:48.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.705+0000 7fe4d6baf740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55745f7533d0 con 0x55745f751460 2024-04-20T11:32:48.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.705+0000 7fe4d6baf740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55745f755470 con 0x55745f7074e0 2024-04-20T11:32:48.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.705+0000 7fe4d6baf740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x55745f7539f0 con 0x55745f72b780 2024-04-20T11:32:48.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.705+0000 7fe4d6baf740 10 client.0 _ll_register_callbacks cb 0x7fe4c4009e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:32:48.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.705+0000 7fe4d1e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55745f751460 0x55745f751d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:48.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.705+0000 7fe4d1e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55745f751460 0x55745f751d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:45040/0 (socket says 172.21.15.152:45040) 2024-04-20T11:32:48.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.705+0000 7fe4d1e00640 1 -- 172.21.15.152:0/1131313367 learned_addr learned my addr 172.21.15.152:0/1131313367 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:32:48.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.706+0000 7fe4d1400640 1 --2- 172.21.15.152:0/1131313367 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55745f7074e0 0x55745f7c86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:48.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.706+0000 7fe4d0a00640 1 --2- 172.21.15.152:0/1131313367 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55745f72b780 0x55745f751840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:48.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.706+0000 7fe4d1e00640 1 -- 172.21.15.152:0/1131313367 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55745f72b780 msgr2=0x55745f751840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:48.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.706+0000 7fe4d1e00640 1 --2- 172.21.15.152:0/1131313367 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55745f72b780 0x55745f751840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:48.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.706+0000 7fe4d1e00640 1 -- 172.21.15.152:0/1131313367 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55745f7074e0 msgr2=0x55745f7c86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:48.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.706+0000 7fe4d1e00640 1 --2- 172.21.15.152:0/1131313367 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55745f7074e0 0x55745f7c86b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:48.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.706+0000 7fe4d1e00640 1 -- 172.21.15.152:0/1131313367 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55745f7c8bf0 con 0x55745f751460 2024-04-20T11:32:48.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.706+0000 7fe4d1e00640 1 --2- 172.21.15.152:0/1131313367 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55745f751460 0x55745f751d80 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x55745f72ff40 tx=0x7fe4c4009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67fef03d84a29d32 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:48.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.706+0000 7fe4b2000640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-04-20T11:32:48.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.706+0000 7fe4b2000640 1 -- 172.21.15.152:0/1131313367 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c4023070 con 0x55745f751460 2024-04-20T11:32:48.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.707+0000 7fe4b2000640 1 -- 172.21.15.152:0/1131313367 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7fe4c4028ca0 con 0x55745f751460 2024-04-20T11:32:48.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.707+0000 7fe4d6baf740 1 -- 172.21.15.152:0/1131313367 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4c0076070 con 0x55745f751460 2024-04-20T11:32:48.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.707+0000 7fe4b2000640 1 -- 172.21.15.152:0/1131313367 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7fe4c402d070 con 0x55745f751460 2024-04-20T11:32:48.710 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.707+0000 7fe4d6baf740 10 client.7476 fetch_fsmap learned FSMap version 53 2024-04-20T11:32:48.710 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.707+0000 7fe4d6baf740 1 -- 172.21.15.152:0/1131313367 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) -- 0x55745f7522c0 con 0x55745f751460 2024-04-20T11:32:48.710 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.707+0000 7fe4b2000640 1 -- 172.21.15.152:0/1131313367 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4c0076070 con 0x55745f751460 2024-04-20T11:32:48.710 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.707+0000 7fe4b2000640 1 -- 172.21.15.152:0/1131313367 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 53) ==== 3495+0+0 (secure 0 0 0) 0x7fe4c4046080 con 0x55745f751460 2024-04-20T11:32:48.710 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.707+0000 7fe4d6baf740 10 client.7476 fetch_fsmap finished waiting for FSMap version 53 2024-04-20T11:32:48.710 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.707+0000 7fe4d6baf740 10 client.7476 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-20T11:32:48.710 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.707+0000 7fe4d6baf740 20 client.7476 populate_metadata read hostname 'smithi152' 2024-04-20T11:32:48.710 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.707+0000 7fe4d6baf740 1 --2- 172.21.15.152:0/1131313367 >> [v2:172.21.15.152:6832/3577228528,v1:172.21.15.152:6833/3577228528] conn(0x55745f725dd0 0x55745f74c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:48.710 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.707+0000 7fe4d6baf740 4 client.7476 mds_command: new command op to 7332 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-20T11:32:48.710 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.707+0000 7fe4d6baf740 1 -- 172.21.15.152:0/1131313367 --> [v2:172.21.15.152:6832/3577228528,v1:172.21.15.152:6833/3577228528] -- command(tid 0: {"prefix": "status"}) -- 0x55745f677260 con 0x55745f725dd0 2024-04-20T11:32:48.710 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.708+0000 7fe4d1400640 1 --2- 172.21.15.152:0/1131313367 >> [v2:172.21.15.152:6832/3577228528,v1:172.21.15.152:6833/3577228528] conn(0x55745f725dd0 0x55745f74c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:48.711 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.708+0000 7fe4d1400640 1 --2- 172.21.15.152:0/1131313367 >> [v2:172.21.15.152:6832/3577228528,v1:172.21.15.152:6833/3577228528] conn(0x55745f725dd0 0x55745f74c240 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c52bc1cd62d546fe server_cookie=c3e275b101b2f195 in_seq=0 out_seq=0 2024-04-20T11:32:48.711 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.708+0000 7fe4b2000640 10 client.7476 ms_handle_connect on v2:172.21.15.152:6832/3577228528 2024-04-20T11:32:48.711 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.708+0000 7fe4b2000640 1 -- 172.21.15.152:0/1131313367 <== mds.0 v2:172.21.15.152:6832/3577228528 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55745f677260 con 0x55745f725dd0 2024-04-20T11:32:48.711 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.708+0000 7fe4b2000640 10 client.7476 handle_command_reply: tid=0 2024-04-20T11:32:48.711 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.708+0000 7fe4d6baf740 1 client.7476 shutdown 2024-04-20T11:32:48.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.708+0000 7fe4d6baf740 1 -- 172.21.15.152:0/1131313367 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55745f751460 msgr2=0x55745f751d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:48.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.708+0000 7fe4d6baf740 1 --2- 172.21.15.152:0/1131313367 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55745f751460 0x55745f751d80 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x55745f72ff40 tx=0x7fe4c4009750 comp rx=0 tx=0).stop 2024-04-20T11:32:48.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.709+0000 7fe4d6baf740 1 -- 172.21.15.152:0/1131313367 shutdown_connections 2024-04-20T11:32:48.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.709+0000 7fe4d6baf740 1 --2- 172.21.15.152:0/1131313367 >> [v2:172.21.15.152:6832/3577228528,v1:172.21.15.152:6833/3577228528] conn(0x55745f725dd0 0x55745f74c240 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:48.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.709+0000 7fe4d6baf740 1 --2- 172.21.15.152:0/1131313367 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55745f751460 0x55745f751d80 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:48.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.709+0000 7fe4d6baf740 1 --2- 172.21.15.152:0/1131313367 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55745f72b780 0x55745f751840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:48.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.709+0000 7fe4d6baf740 1 --2- 172.21.15.152:0/1131313367 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55745f7074e0 0x55745f7c86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:48.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.709+0000 7fe4d6baf740 1 -- 172.21.15.152:0/1131313367 >> 172.21.15.152:0/1131313367 conn(0x55745f703930 msgr2=0x55745f7089c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:48.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.709+0000 7fe4b2000640 0 client.7476 ms_handle_reset on v2:172.21.15.152:6832/3577228528 2024-04-20T11:32:48.713 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.709+0000 7fe4d6baf740 1 -- 172.21.15.152:0/1131313367 shutdown_connections 2024-04-20T11:32:48.713 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.709+0000 7fe4d6baf740 1 -- 172.21.15.152:0/1131313367 wait complete. 2024-04-20T11:32:48.713 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.709+0000 7fe4d6baf740 20 client.7476 trim_cache size 0 max 16384 2024-04-20T11:32:48.713 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-04-20T11:32:48.713 INFO:teuthology.orchestra.run.smithi152.stdout: "cluster_fsid": "c8a72131-0d4f-42c7-892f-3f3af4d2bd3d", 2024-04-20T11:32:48.713 INFO:teuthology.orchestra.run.smithi152.stdout: "whoami": 0, 2024-04-20T11:32:48.713 INFO:teuthology.orchestra.run.smithi152.stdout: "id": 7332, 2024-04-20T11:32:48.713 INFO:teuthology.orchestra.run.smithi152.stdout: "want_state": "up:active", 2024-04-20T11:32:48.713 INFO:teuthology.orchestra.run.smithi152.stdout: "state": "up:active", 2024-04-20T11:32:48.714 INFO:teuthology.orchestra.run.smithi152.stdout: "fs_name": "cephfs", 2024-04-20T11:32:48.714 INFO:teuthology.orchestra.run.smithi152.stdout: "rank_uptime": 6.6152194849999999, 2024-04-20T11:32:48.714 INFO:teuthology.orchestra.run.smithi152.stdout: "mdsmap_epoch": 53, 2024-04-20T11:32:48.714 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_epoch": 83, 2024-04-20T11:32:48.714 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_epoch_barrier": 83, 2024-04-20T11:32:48.714 INFO:teuthology.orchestra.run.smithi152.stdout: "uptime": 12.088105709000001 2024-04-20T11:32:48.714 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-04-20T11:32:48.715 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.710+0000 7fe4d3e00640 1 -- 172.21.15.152:0/3476246143 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe4ac06ca10 msgr2=0x7fe4ac06eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:48.715 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.710+0000 7fe4d3e00640 1 --2- 172.21.15.152:0/3476246143 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe4ac06ca10 0x7fe4ac06eed0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe4b8035000 tx=0x7fe4b8030000 comp rx=0 tx=0).stop 2024-04-20T11:32:48.715 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.710+0000 7fe4d3e00640 1 -- 172.21.15.152:0/3476246143 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe4b4003f20 msgr2=0x7fe4cc155590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:48.715 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.710+0000 7fe4d3e00640 1 --2- 172.21.15.152:0/3476246143 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe4b4003f20 0x7fe4cc155590 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fe4c000e790 tx=0x7fe4c000ec50 comp rx=0 tx=0).stop 2024-04-20T11:32:48.716 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.710+0000 7fe4d3e00640 1 -- 172.21.15.152:0/3476246143 shutdown_connections 2024-04-20T11:32:48.716 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.710+0000 7fe4d3e00640 1 --2- 172.21.15.152:0/3476246143 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe4ac06ca10 0x7fe4ac06eed0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:48.716 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.710+0000 7fe4d3e00640 1 --2- 172.21.15.152:0/3476246143 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe4cc165e10 0x7fe4cc1f8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:48.716 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.710+0000 7fe4d3e00640 1 --2- 172.21.15.152:0/3476246143 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe4cc165990 0x7fe4cc1f8880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:48.716 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.710+0000 7fe4d3e00640 1 --2- 172.21.15.152:0/3476246143 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe4b4003f20 0x7fe4cc155590 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:48.716 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.710+0000 7fe4d3e00640 1 -- 172.21.15.152:0/3476246143 >> 172.21.15.152:0/3476246143 conn(0x7fe4cc067cf0 msgr2=0x7fe4cc1615c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:48.716 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.710+0000 7fe4d3e00640 1 -- 172.21.15.152:0/3476246143 shutdown_connections 2024-04-20T11:32:48.716 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.710+0000 7fe4d3e00640 1 -- 172.21.15.152:0/3476246143 wait complete. 2024-04-20T11:32:48.728 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-20T11:32:48.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.914+0000 7f3a33200640 1 Processor -- start 2024-04-20T11:32:48.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.914+0000 7f3a33200640 1 -- start start 2024-04-20T11:32:48.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.916+0000 7f3a33200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f3a3406f510 0x7f3a34140dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:48.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.916+0000 7f3a33200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3a34141310 0x7f3a34071cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:48.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.916+0000 7f3a33200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a340cf0a0 con 0x7f3a3413e530 2024-04-20T11:32:48.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.916+0000 7f3a33200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f3a34141b00 con 0x7f3a3406f510 2024-04-20T11:32:48.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.916+0000 7f3a33200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f3a340cfc50 con 0x7f3a34141310 2024-04-20T11:32:48.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.917+0000 7f3a31400640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f3a3406f510 0x7f3a34140dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:48.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.917+0000 7f3a32800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3a34141310 0x7f3a34071cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:48.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.917+0000 7f3a31400640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f3a3406f510 0x7f3a34140dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.152:48810/0 (socket says 172.21.15.152:48810) 2024-04-20T11:32:48.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.917+0000 7f3a31400640 1 -- 172.21.15.152:0/3197786178 learned_addr learned my addr 172.21.15.152:0/3197786178 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:32:48.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.917+0000 7f3a31400640 1 -- 172.21.15.152:0/3197786178 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3a34141310 msgr2=0x7f3a34071cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:48.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.917+0000 7f3a31400640 1 --2- 172.21.15.152:0/3197786178 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3a34141310 0x7f3a34071cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:48.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.917+0000 7f3a31400640 1 -- 172.21.15.152:0/3197786178 >> v1:172.21.15.175:6789/0 conn(0x7f3a3413e530 legacy=0x7f3a3406ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:48.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.917+0000 7f3a31400640 1 -- 172.21.15.152:0/3197786178 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a340d00e0 con 0x7f3a3406f510 2024-04-20T11:32:48.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.919+0000 7f3a31400640 1 --2- 172.21.15.152:0/3197786178 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3a3406f510 0x7f3a34140dd0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f3a20009870 tx=0x7f3a2002b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e065b5f0e9846154 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:48.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.919+0000 7f3a30a00640 1 -- 172.21.15.152:0/3197786178 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a20034070 con 0x7f3a3406f510 2024-04-20T11:32:48.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.919+0000 7f3a30a00640 1 -- 172.21.15.152:0/3197786178 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a20030840 con 0x7f3a3406f510 2024-04-20T11:32:48.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.919+0000 7f3a30a00640 1 -- 172.21.15.152:0/3197786178 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a20030c10 con 0x7f3a3406f510 2024-04-20T11:32:48.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.920+0000 7f3a33200640 1 -- 172.21.15.152:0/3197786178 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3a3406f510 msgr2=0x7f3a34140dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:48.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.920+0000 7f3a33200640 1 --2- 172.21.15.152:0/3197786178 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3a3406f510 0x7f3a34140dd0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f3a20009870 tx=0x7f3a2002b980 comp rx=0 tx=0).stop 2024-04-20T11:32:48.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.920+0000 7f3a33200640 1 -- 172.21.15.152:0/3197786178 shutdown_connections 2024-04-20T11:32:48.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.920+0000 7f3a33200640 1 --2- 172.21.15.152:0/3197786178 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3a34141310 0x7f3a34071cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:48.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.920+0000 7f3a33200640 1 --2- 172.21.15.152:0/3197786178 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3a3406f510 0x7f3a34140dd0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:48.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.920+0000 7f3a33200640 1 -- 172.21.15.152:0/3197786178 >> 172.21.15.152:0/3197786178 conn(0x7f3a34067ae0 msgr2=0x7f3a3406b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:48.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.920+0000 7f3a33200640 1 -- 172.21.15.152:0/3197786178 shutdown_connections 2024-04-20T11:32:48.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.920+0000 7f3a33200640 1 -- 172.21.15.152:0/3197786178 wait complete. 2024-04-20T11:32:48.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.922+0000 7f3a33200640 1 Processor -- start 2024-04-20T11:32:48.922 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.922+0000 7f3a33200640 1 -- start start 2024-04-20T11:32:48.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.923+0000 7f3a33200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3a3406f510 0x7f3a340affe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:48.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.923+0000 7f3a33200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3a3413e530 0x7f3a341e48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:48.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.924+0000 7f3a33200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3a34141310 0x7f3a3413d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:48.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.924+0000 7f3a33200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f3a340cf8e0 con 0x7f3a3406f510 2024-04-20T11:32:48.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.924+0000 7f3a33200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3a340cf600 con 0x7f3a34141310 2024-04-20T11:32:48.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.924+0000 7f3a33200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f3a341417c0 con 0x7f3a3413e530 2024-04-20T11:32:48.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.924+0000 7f3a31e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3a3406f510 0x7f3a340affe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:48.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.924+0000 7f3a31e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3a3406f510 0x7f3a340affe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:45042/0 (socket says 172.21.15.152:45042) 2024-04-20T11:32:48.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.924+0000 7f3a31e00640 1 -- 172.21.15.152:0/3827805114 learned_addr learned my addr 172.21.15.152:0/3827805114 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:32:48.925 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.924+0000 7f3a31400640 1 --2- 172.21.15.152:0/3827805114 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3a3413e530 0x7f3a341e48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:48.925 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.924+0000 7f3a32800640 1 --2- 172.21.15.152:0/3827805114 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3a34141310 0x7f3a3413d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:48.925 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.924+0000 7f3a31e00640 1 -- 172.21.15.152:0/3827805114 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3a3413e530 msgr2=0x7f3a341e48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:48.925 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.924+0000 7f3a31e00640 1 --2- 172.21.15.152:0/3827805114 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3a3413e530 0x7f3a341e48f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:48.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.924+0000 7f3a31e00640 1 -- 172.21.15.152:0/3827805114 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3a34141310 msgr2=0x7f3a3413d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:48.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.924+0000 7f3a31e00640 1 --2- 172.21.15.152:0/3827805114 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3a34141310 0x7f3a3413d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:48.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.924+0000 7f3a31e00640 1 -- 172.21.15.152:0/3827805114 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a340b0650 con 0x7f3a3406f510 2024-04-20T11:32:48.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.924+0000 7f3a31400640 1 --2- 172.21.15.152:0/3827805114 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3a3413e530 0x7f3a341e48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:32:48.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.924+0000 7f3a32800640 1 --2- 172.21.15.152:0/3827805114 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3a34141310 0x7f3a3413d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:32:48.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.925+0000 7f3a31e00640 1 --2- 172.21.15.152:0/3827805114 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3a3406f510 0x7f3a340affe0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f3a2400c450 tx=0x7f3a2400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:48.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.925+0000 7f3a2a000640 1 -- 172.21.15.152:0/3827805114 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a240052d0 con 0x7f3a3406f510 2024-04-20T11:32:48.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.925+0000 7f3a2a000640 1 -- 172.21.15.152:0/3827805114 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a24005470 con 0x7f3a3406f510 2024-04-20T11:32:48.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.925+0000 7f3a33200640 1 -- 172.21.15.152:0/3827805114 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a340b0930 con 0x7f3a3406f510 2024-04-20T11:32:48.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.925+0000 7f3a2a000640 1 -- 172.21.15.152:0/3827805114 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a24012870 con 0x7f3a3406f510 2024-04-20T11:32:48.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.925+0000 7f3a33200640 1 -- 172.21.15.152:0/3827805114 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a340b0ef0 con 0x7f3a3406f510 2024-04-20T11:32:48.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.926+0000 7f3a33200640 1 -- 172.21.15.152:0/3827805114 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a34075e60 con 0x7f3a3406f510 2024-04-20T11:32:48.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.928+0000 7f3a2a000640 1 -- 172.21.15.152:0/3827805114 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 91142+0+0 (secure 0 0 0) 0x7f3a24012a10 con 0x7f3a3406f510 2024-04-20T11:32:48.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.928+0000 7f3a2a000640 1 --2- 172.21.15.152:0/3827805114 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f3a1406ca10 0x7f3a1406eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:48.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.929+0000 7f3a2a000640 1 -- 172.21.15.152:0/3827805114 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7f3a2408b8f0 con 0x7f3a3406f510 2024-04-20T11:32:48.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.929+0000 7f3a31400640 1 --2- 172.21.15.152:0/3827805114 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f3a1406ca10 0x7f3a1406eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:48.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.930+0000 7f3a31400640 1 --2- 172.21.15.152:0/3827805114 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f3a1406ca10 0x7f3a1406eed0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3a20036000 tx=0x7f3a2003b040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:48.932 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:48.933+0000 7f3a2a000640 1 -- 172.21.15.152:0/3827805114 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f3a340cf0a0 con 0x7f3a3406f510 2024-04-20T11:32:49.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:49.175+0000 7f3a33200640 1 -- 172.21.15.152:0/3827805114 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f3a340b3490 con 0x7f3a3406f510 2024-04-20T11:32:49.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:49.176+0000 7f3a2a000640 1 -- 172.21.15.152:0/3827805114 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v53) ==== 53+0+73 (secure 0 0 0) 0x7f3a24059c00 con 0x7f3a3406f510 2024-04-20T11:32:49.176 INFO:teuthology.orchestra.run.smithi152.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-20T11:32:49.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:49.179+0000 7f3a33200640 1 -- 172.21.15.152:0/3827805114 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f3a1406ca10 msgr2=0x7f3a1406eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:49.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:49.179+0000 7f3a33200640 1 --2- 172.21.15.152:0/3827805114 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f3a1406ca10 0x7f3a1406eed0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3a20036000 tx=0x7f3a2003b040 comp rx=0 tx=0).stop 2024-04-20T11:32:49.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:49.179+0000 7f3a33200640 1 -- 172.21.15.152:0/3827805114 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3a3406f510 msgr2=0x7f3a340affe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:49.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:49.179+0000 7f3a33200640 1 --2- 172.21.15.152:0/3827805114 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3a3406f510 0x7f3a340affe0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f3a2400c450 tx=0x7f3a2400c910 comp rx=0 tx=0).stop 2024-04-20T11:32:49.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:49.179+0000 7f3a33200640 1 -- 172.21.15.152:0/3827805114 shutdown_connections 2024-04-20T11:32:49.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:49.179+0000 7f3a33200640 1 --2- 172.21.15.152:0/3827805114 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f3a1406ca10 0x7f3a1406eed0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:49.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:49.179+0000 7f3a33200640 1 --2- 172.21.15.152:0/3827805114 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3a34141310 0x7f3a3413d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:49.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:49.179+0000 7f3a33200640 1 --2- 172.21.15.152:0/3827805114 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3a3413e530 0x7f3a341e48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:49.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:49.179+0000 7f3a33200640 1 --2- 172.21.15.152:0/3827805114 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3a3406f510 0x7f3a340affe0 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:49.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:49.179+0000 7f3a33200640 1 -- 172.21.15.152:0/3827805114 >> 172.21.15.152:0/3827805114 conn(0x7f3a34067ae0 msgr2=0x7f3a34152340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:49.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:49.179+0000 7f3a33200640 1 -- 172.21.15.152:0/3827805114 shutdown_connections 2024-04-20T11:32:49.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:49.179+0000 7f3a33200640 1 -- 172.21.15.152:0/3827805114 wait complete. 2024-04-20T11:32:49.198 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-20T11:32:49.198 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-20T11:32:49.198 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi152.front.sepia.ceph.com 2024-04-20T11:32:49.198 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-20T11:32:49.198 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-20T11:32:49.198 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-20T11:32:49.198 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-20T11:32:49.198 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-20T11:32:49.198 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-20T11:32:49.199 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:32:49.199 DEBUG:teuthology.orchestra.run.smithi152:> ip addr 2024-04-20T11:32:49.252 INFO:teuthology.orchestra.run.smithi152.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-20T11:32:49.252 INFO:teuthology.orchestra.run.smithi152.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-20T11:32:49.253 INFO:teuthology.orchestra.run.smithi152.stdout: inet 127.0.0.1/8 scope host lo 2024-04-20T11:32:49.253 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:32:49.253 INFO:teuthology.orchestra.run.smithi152.stdout: inet6 ::1/128 scope host 2024-04-20T11:32:49.253 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:32:49.253 INFO:teuthology.orchestra.run.smithi152.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T11:32:49.253 INFO:teuthology.orchestra.run.smithi152.stdout: link/ether 0c:c4:7a:da:6c:44 brd ff:ff:ff:ff:ff:ff 2024-04-20T11:32:49.253 INFO:teuthology.orchestra.run.smithi152.stdout: altname enp7s0f0 2024-04-20T11:32:49.253 INFO:teuthology.orchestra.run.smithi152.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T11:32:49.253 INFO:teuthology.orchestra.run.smithi152.stdout: link/ether 0c:c4:7a:da:6c:45 brd ff:ff:ff:ff:ff:ff 2024-04-20T11:32:49.253 INFO:teuthology.orchestra.run.smithi152.stdout: altname enp7s0f1 2024-04-20T11:32:49.253 INFO:teuthology.orchestra.run.smithi152.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T11:32:49.253 INFO:teuthology.orchestra.run.smithi152.stdout: link/ether 0c:c4:7a:88:6d:e2 brd ff:ff:ff:ff:ff:ff 2024-04-20T11:32:49.254 INFO:teuthology.orchestra.run.smithi152.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-20T11:32:49.254 INFO:teuthology.orchestra.run.smithi152.stdout: link/ether 0c:c4:7a:88:6d:e3 brd ff:ff:ff:ff:ff:ff 2024-04-20T11:32:49.254 INFO:teuthology.orchestra.run.smithi152.stdout: inet 172.21.15.152/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-20T11:32:49.254 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft 42437sec preferred_lft 42437sec 2024-04-20T11:32:49.254 INFO:teuthology.orchestra.run.smithi152.stdout: inet6 fe80::ec4:7aff:fe88:6de3/64 scope link 2024-04-20T11:32:49.254 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:32:49.254 INFO:teuthology.orchestra.run.smithi152.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-20T11:32:49.254 INFO:teuthology.orchestra.run.smithi152.stdout: link/ether 06:3e:9a:d3:49:1e brd ff:ff:ff:ff:ff:ff 2024-04-20T11:32:49.254 INFO:teuthology.orchestra.run.smithi152.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-20T11:32:49.254 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:32:49.254 INFO:teuthology.orchestra.run.smithi152.stdout: inet6 fe80::a019:d8ff:fe70:1185/64 scope link 2024-04-20T11:32:49.254 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:32:49.254 INFO:teuthology.orchestra.run.smithi152.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-20T11:32:49.255 INFO:teuthology.orchestra.run.smithi152.stdout: link/ether 16:e5:08:78:20:47 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T11:32:49.255 INFO:teuthology.orchestra.run.smithi152.stdout: inet6 fe80::14e5:8ff:fe78:2047/64 scope link 2024-04-20T11:32:49.255 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:32:49.255 INFO:teuthology.orchestra.run.smithi152.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-20T11:32:49.255 INFO:teuthology.orchestra.run.smithi152.stdout: link/ether 06:3e:9a:d3:49:1e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T11:32:49.255 INFO:teuthology.orchestra.run.smithi152.stdout: inet6 fe80::43e:9aff:fed3:491e/64 scope link 2024-04-20T11:32:49.255 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:32:49.256 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:32:49.256 DEBUG:teuthology.orchestra.run.smithi152:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-20T11:32:49.330 INFO:teuthology.orchestra.run.smithi152.stdout:1 2024-04-20T11:32:49.335 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:32:49.335 DEBUG:teuthology.orchestra.run.smithi152:> route 2024-04-20T11:32:49.387 INFO:teuthology.orchestra.run.smithi152.stdout:Kernel IP routing table 2024-04-20T11:32:49.388 INFO:teuthology.orchestra.run.smithi152.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-20T11:32:49.388 INFO:teuthology.orchestra.run.smithi152.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-20T11:32:49.388 INFO:teuthology.orchestra.run.smithi152.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-20T11:32:49.388 INFO:teuthology.orchestra.run.smithi152.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-20T11:32:49.389 DEBUG:teuthology.orchestra.run.smithi152:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T11:32:49.389 DEBUG:teuthology.orchestra.run.smithi152:> set -e 2024-04-20T11:32:49.389 DEBUG:teuthology.orchestra.run.smithi152:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-20T11:32:49.389 DEBUG:teuthology.orchestra.run.smithi152:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-20T11:32:49.389 DEBUG:teuthology.orchestra.run.smithi152:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-20T11:32:49.389 DEBUG:teuthology.orchestra.run.smithi152:> ') 2024-04-20T11:32:49.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T11:32:49.585 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T11:32:49.594 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:32:49.594 DEBUG:teuthology.orchestra.run.smithi152:> ip netns list 2024-04-20T11:32:49.639 INFO:teuthology.orchestra.run.smithi152.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-20T11:32:49.639 INFO:teuthology.orchestra.run.smithi152.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-20T11:32:49.639 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-20T11:32:49.639 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:32:49.639 DEBUG:teuthology.orchestra.run.smithi152:> sudo ip link set brx.0 up 2024-04-20T11:32:49.763 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-20T11:32:49.763 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T11:32:49.763 DEBUG:teuthology.orchestra.run.smithi152:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-20T11:32:49.814 INFO:teuthology.orchestra.run.smithi152.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-20T11:32:49.814 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T11:32:49.814 DEBUG:teuthology.orchestra.run.smithi152:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-20T11:32:49.861 DEBUG:teuthology.orchestra.run.smithi152:> sudo modprobe fuse 2024-04-20T11:32:49.933 DEBUG:teuthology.orchestra.run.smithi152:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T11:32:49.984 INFO:teuthology.orchestra.run.smithi152.stdout:/proc 2024-04-20T11:32:49.984 INFO:teuthology.orchestra.run.smithi152.stdout:/sys 2024-04-20T11:32:49.984 INFO:teuthology.orchestra.run.smithi152.stdout:/dev 2024-04-20T11:32:49.984 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/security 2024-04-20T11:32:49.984 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/shm 2024-04-20T11:32:49.984 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/pts 2024-04-20T11:32:49.984 INFO:teuthology.orchestra.run.smithi152.stdout:/run 2024-04-20T11:32:49.984 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/cgroup 2024-04-20T11:32:49.984 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/pstore 2024-04-20T11:32:49.984 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/bpf 2024-04-20T11:32:49.984 INFO:teuthology.orchestra.run.smithi152.stdout:/ 2024-04-20T11:32:49.984 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/selinux 2024-04-20T11:32:49.985 INFO:teuthology.orchestra.run.smithi152.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T11:32:49.985 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/mqueue 2024-04-20T11:32:49.985 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/hugepages 2024-04-20T11:32:49.985 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/debug 2024-04-20T11:32:49.985 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/tracing 2024-04-20T11:32:49.985 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/fuse/connections 2024-04-20T11:32:49.985 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/config 2024-04-20T11:32:49.985 INFO:teuthology.orchestra.run.smithi152.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T11:32:49.985 INFO:teuthology.orchestra.run.smithi152.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T11:32:49.985 INFO:teuthology.orchestra.run.smithi152.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T11:32:49.985 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T11:32:49.985 INFO:teuthology.orchestra.run.smithi152.stdout:/run/user/1000 2024-04-20T11:32:49.985 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph 2024-04-20T11:32:49.986 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph/osd/ceph-0 2024-04-20T11:32:49.986 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph/osd/ceph-1 2024-04-20T11:32:49.986 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph/osd/ceph-2 2024-04-20T11:32:49.986 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph/osd/ceph-3 2024-04-20T11:32:49.986 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns 2024-04-20T11:32:49.986 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T11:32:49.986 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T11:32:49.986 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T11:32:49.986 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T11:32:49.987 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:32:49.987 DEBUG:teuthology.orchestra.run.smithi152:> ls /sys/fs/fuse/connections 2024-04-20T11:32:50.036 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-20T11:32:50.036 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-20T11:32:50.043 DEBUG:teuthology.orchestra.run.smithi152:> sudo modprobe fuse 2024-04-20T11:32:50.114 DEBUG:teuthology.orchestra.run.smithi152:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T11:32:50.158 INFO:teuthology.orchestra.run.smithi152.stdout:/proc 2024-04-20T11:32:50.158 INFO:teuthology.orchestra.run.smithi152.stdout:/sys 2024-04-20T11:32:50.159 INFO:teuthology.orchestra.run.smithi152.stdout:/dev 2024-04-20T11:32:50.159 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/security 2024-04-20T11:32:50.159 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/shm 2024-04-20T11:32:50.159 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/pts 2024-04-20T11:32:50.159 INFO:teuthology.orchestra.run.smithi152.stdout:/run 2024-04-20T11:32:50.159 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/cgroup 2024-04-20T11:32:50.159 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/pstore 2024-04-20T11:32:50.159 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/bpf 2024-04-20T11:32:50.159 INFO:teuthology.orchestra.run.smithi152.stdout:/ 2024-04-20T11:32:50.159 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/selinux 2024-04-20T11:32:50.159 INFO:teuthology.orchestra.run.smithi152.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T11:32:50.159 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/mqueue 2024-04-20T11:32:50.159 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/hugepages 2024-04-20T11:32:50.160 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/debug 2024-04-20T11:32:50.160 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/tracing 2024-04-20T11:32:50.160 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/fuse/connections 2024-04-20T11:32:50.160 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/config 2024-04-20T11:32:50.160 INFO:teuthology.orchestra.run.smithi152.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T11:32:50.160 INFO:teuthology.orchestra.run.smithi152.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T11:32:50.160 INFO:teuthology.orchestra.run.smithi152.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T11:32:50.160 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T11:32:50.160 INFO:teuthology.orchestra.run.smithi152.stdout:/run/user/1000 2024-04-20T11:32:50.160 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph 2024-04-20T11:32:50.160 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph/osd/ceph-0 2024-04-20T11:32:50.160 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph/osd/ceph-1 2024-04-20T11:32:50.160 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph/osd/ceph-2 2024-04-20T11:32:50.160 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph/osd/ceph-3 2024-04-20T11:32:50.160 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns 2024-04-20T11:32:50.161 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T11:32:50.161 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T11:32:50.161 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T11:32:50.161 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T11:32:50.162 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:32:50.162 DEBUG:teuthology.orchestra.run.smithi152:> ls /sys/fs/fuse/connections 2024-04-20T11:32:50.219 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi152.stderr:2024-04-20T11:32:50.219+0000 7f4b17046480 -1 init, newargv = 0x55a912d452a0 newargc=15 2024-04-20T11:32:50.219 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi152.stderr:ceph-fuse[51248]: starting ceph client 2024-04-20T11:32:50.229 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi152.stderr:ceph-fuse[51248]: starting fuse 2024-04-20T11:32:50.245 INFO:teuthology.orchestra.run.smithi152.stdout:42 2024-04-20T11:32:50.245 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-04-20T11:32:50.245 DEBUG:teuthology.orchestra.run.smithi152:> sudo stdin-killer -- python3 -c ' 2024-04-20T11:32:50.245 DEBUG:teuthology.orchestra.run.smithi152:> import glob 2024-04-20T11:32:50.245 DEBUG:teuthology.orchestra.run.smithi152:> import re 2024-04-20T11:32:50.245 DEBUG:teuthology.orchestra.run.smithi152:> import os 2024-04-20T11:32:50.245 DEBUG:teuthology.orchestra.run.smithi152:> import subprocess 2024-04-20T11:32:50.246 DEBUG:teuthology.orchestra.run.smithi152:> 2024-04-20T11:32:50.246 DEBUG:teuthology.orchestra.run.smithi152:> def _find_admin_socket(client_name): 2024-04-20T11:32:50.246 DEBUG:teuthology.orchestra.run.smithi152:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-20T11:32:50.246 DEBUG:teuthology.orchestra.run.smithi152:> files = glob.glob(asok_path) 2024-04-20T11:32:50.246 DEBUG:teuthology.orchestra.run.smithi152:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-20T11:32:50.246 DEBUG:teuthology.orchestra.run.smithi152:> 2024-04-20T11:32:50.246 DEBUG:teuthology.orchestra.run.smithi152:> # Given a non-glob path, it better be there 2024-04-20T11:32:50.246 DEBUG:teuthology.orchestra.run.smithi152:> if "*" not in asok_path: 2024-04-20T11:32:50.246 DEBUG:teuthology.orchestra.run.smithi152:> assert(len(files) == 1) 2024-04-20T11:32:50.246 DEBUG:teuthology.orchestra.run.smithi152:> return files[0] 2024-04-20T11:32:50.246 DEBUG:teuthology.orchestra.run.smithi152:> 2024-04-20T11:32:50.246 DEBUG:teuthology.orchestra.run.smithi152:> for f in files: 2024-04-20T11:32:50.246 DEBUG:teuthology.orchestra.run.smithi152:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-20T11:32:50.246 DEBUG:teuthology.orchestra.run.smithi152:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-20T11:32:50.246 DEBUG:teuthology.orchestra.run.smithi152:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-20T11:32:50.247 DEBUG:teuthology.orchestra.run.smithi152:> contents = proc_f.read() 2024-04-20T11:32:50.247 DEBUG:teuthology.orchestra.run.smithi152:> if mountpoint in contents: 2024-04-20T11:32:50.247 DEBUG:teuthology.orchestra.run.smithi152:> return f 2024-04-20T11:32:50.247 DEBUG:teuthology.orchestra.run.smithi152:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-20T11:32:50.247 DEBUG:teuthology.orchestra.run.smithi152:> 2024-04-20T11:32:50.247 DEBUG:teuthology.orchestra.run.smithi152:> print(_find_admin_socket("client.0")) 2024-04-20T11:32:50.247 DEBUG:teuthology.orchestra.run.smithi152:> ' 2024-04-20T11:32:50.433 INFO:teuthology.orchestra.run.smithi152.stdout:/var/run/ceph/ceph-client.0.51248.asok 2024-04-20T11:32:50.437 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T11:32:50.449 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.51248.asok 2024-04-20T11:32:50.450 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:32:50.450 DEBUG:teuthology.orchestra.run.smithi152:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.51248.asok status 2024-04-20T11:32:50.669 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-04-20T11:32:50.669 INFO:teuthology.orchestra.run.smithi152.stdout: "metadata": { 2024-04-20T11:32:50.669 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_sha1": "a8c84f5997950fd7d008204839253291d52e6eda", 2024-04-20T11:32:50.669 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", 2024-04-20T11:32:50.669 INFO:teuthology.orchestra.run.smithi152.stdout: "entity_id": "0", 2024-04-20T11:32:50.669 INFO:teuthology.orchestra.run.smithi152.stdout: "hostname": "smithi152", 2024-04-20T11:32:50.669 INFO:teuthology.orchestra.run.smithi152.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-20T11:32:50.669 INFO:teuthology.orchestra.run.smithi152.stdout: "pid": "51248", 2024-04-20T11:32:50.669 INFO:teuthology.orchestra.run.smithi152.stdout: "root": "/" 2024-04-20T11:32:50.670 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:50.670 INFO:teuthology.orchestra.run.smithi152.stdout: "dentry_count": 0, 2024-04-20T11:32:50.670 INFO:teuthology.orchestra.run.smithi152.stdout: "dentry_pinned_count": 0, 2024-04-20T11:32:50.670 INFO:teuthology.orchestra.run.smithi152.stdout: "id": 7488, 2024-04-20T11:32:50.670 INFO:teuthology.orchestra.run.smithi152.stdout: "inst": { 2024-04-20T11:32:50.670 INFO:teuthology.orchestra.run.smithi152.stdout: "name": { 2024-04-20T11:32:50.670 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "client", 2024-04-20T11:32:50.670 INFO:teuthology.orchestra.run.smithi152.stdout: "num": 7488 2024-04-20T11:32:50.670 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:50.670 INFO:teuthology.orchestra.run.smithi152.stdout: "addr": { 2024-04-20T11:32:50.670 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "v1", 2024-04-20T11:32:50.670 INFO:teuthology.orchestra.run.smithi152.stdout: "addr": "192.168.0.1:0", 2024-04-20T11:32:50.671 INFO:teuthology.orchestra.run.smithi152.stdout: "nonce": 2520780637 2024-04-20T11:32:50.671 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:50.671 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:50.671 INFO:teuthology.orchestra.run.smithi152.stdout: "addr": { 2024-04-20T11:32:50.671 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "v1", 2024-04-20T11:32:50.671 INFO:teuthology.orchestra.run.smithi152.stdout: "addr": "192.168.0.1:0", 2024-04-20T11:32:50.671 INFO:teuthology.orchestra.run.smithi152.stdout: "nonce": 2520780637 2024-04-20T11:32:50.671 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:32:50.671 INFO:teuthology.orchestra.run.smithi152.stdout: "inst_str": "client.7488 192.168.0.1:0/2520780637", 2024-04-20T11:32:50.671 INFO:teuthology.orchestra.run.smithi152.stdout: "addr_str": "192.168.0.1:0/2520780637", 2024-04-20T11:32:50.671 INFO:teuthology.orchestra.run.smithi152.stdout: "inode_count": 1, 2024-04-20T11:32:50.671 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_epoch": 53, 2024-04-20T11:32:50.671 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_epoch": 83, 2024-04-20T11:32:50.671 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_epoch_barrier": 0, 2024-04-20T11:32:50.672 INFO:teuthology.orchestra.run.smithi152.stdout: "blocklisted": false, 2024-04-20T11:32:50.672 INFO:teuthology.orchestra.run.smithi152.stdout: "fs_name": "cephfs" 2024-04-20T11:32:50.672 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-04-20T11:32:50.684 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:32:50.684 DEBUG:teuthology.orchestra.run.smithi152:> stat --file-system '--printf=%T 2024-04-20T11:32:50.684 DEBUG:teuthology.orchestra.run.smithi152:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-20T11:32:50.735 INFO:teuthology.orchestra.run.smithi152.stdout:fuseblk 2024-04-20T11:32:50.735 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-20T11:32:50.735 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:32:50.735 DEBUG:teuthology.orchestra.run.smithi152:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-20T11:32:50.814 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-20T11:32:51.015 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:51.016+0000 7f51f1000640 1 Processor -- start 2024-04-20T11:32:51.016 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:51.016+0000 7f51f1000640 1 -- start start 2024-04-20T11:32:51.017 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:51.017+0000 7f51f1000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f51ec0723e0 0x7f51ec15d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:51.017 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:51.017+0000 7f51f1000640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51ec0ce8c0 con 0x7f51ec160b90 2024-04-20T11:32:51.017 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:51.017+0000 7f51f1000640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51ec1600b0 con 0x7f51ec15d1e0 2024-04-20T11:32:51.017 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:51.017+0000 7f51f1000640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f51ec160390 con 0x7f51ec0723e0 2024-04-20T11:32:51.017 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:51.018+0000 7f51eaa00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f51ec160b90 0x7f51ec071cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:41174/0 (socket says 172.21.15.152:41174) 2024-04-20T11:32:51.017 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:51.018+0000 7f51eaa00640 1 -- 172.21.15.152:0/2735583134 learned_addr learned my addr 172.21.15.152:0/2735583134 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:32:51.018 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:51.018+0000 7f51ebe00640 1 --2- 172.21.15.152:0/2735583134 >> v2:172.21.15.152:3300/0 conn(0x7f51ec0723e0 0x7f51ec15d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:51.019 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:51.018+0000 7f51ea000640 1 -- 172.21.15.152:0/2735583134 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2110461336 0 0) 0x7f51ec0ce8c0 con 0x7f51ec160b90 2024-04-20T11:32:51.019 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:51.018+0000 7f51ea000640 1 -- 172.21.15.152:0/2735583134 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f51d0003520 con 0x7f51ec160b90 2024-04-20T11:32:51.019 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:51.018+0000 7f51ebe00640 1 -- 172.21.15.152:0/2735583134 >> v1:172.21.15.175:6789/0 conn(0x7f51ec15d1e0 legacy=0x7f51ec0cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:51.019 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:51.018+0000 7f51ebe00640 1 -- 172.21.15.152:0/2735583134 >> v1:172.21.15.152:6789/0 conn(0x7f51ec160b90 legacy=0x7f51ec071cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:51.019 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:51.018+0000 7f51ebe00640 1 -- 172.21.15.152:0/2735583134 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51ec160670 con 0x7f51ec0723e0 2024-04-20T11:32:51.019 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:51.019+0000 7f51ebe00640 1 --2- 172.21.15.152:0/2735583134 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f51ec0723e0 0x7f51ec15d5c0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f51e4009dc0 tx=0x7f51e402bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eca70c58639e9ff7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:51.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:51.020+0000 7f51ea000640 1 -- 172.21.15.152:0/2735583134 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51e4039070 con 0x7f51ec0723e0 2024-04-20T11:32:51.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:51.020+0000 7f51ea000640 1 -- 172.21.15.152:0/2735583134 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51e402cce0 con 0x7f51ec0723e0 2024-04-20T11:32:51.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:51.020+0000 7f51ea000640 1 -- 172.21.15.152:0/2735583134 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51e4034660 con 0x7f51ec0723e0 2024-04-20T11:32:51.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:51.020+0000 7f51f1000640 1 -- 172.21.15.152:0/2735583134 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f51ec0723e0 msgr2=0x7f51ec15d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:51.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:51.020+0000 7f51f1000640 1 --2- 172.21.15.152:0/2735583134 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f51ec0723e0 0x7f51ec15d5c0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f51e4009dc0 tx=0x7f51e402bf30 comp rx=0 tx=0).stop 2024-04-20T11:32:51.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:51.020+0000 7f51f1000640 1 -- 172.21.15.152:0/2735583134 shutdown_connections 2024-04-20T11:32:51.021 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:51.020+0000 7f51f1000640 1 --2- 172.21.15.152:0/2735583134 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f51ec0723e0 0x7f51ec15d5c0 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:51.021 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:51.020+0000 7f51f1000640 1 -- 172.21.15.152:0/2735583134 >> 172.21.15.152:0/2735583134 conn(0x7f51ec067ae0 msgr2=0x7f51ec06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:51.021 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:51.020+0000 7f51f1000640 1 -- 172.21.15.152:0/2735583134 shutdown_connections 2024-04-20T11:32:51.021 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:51.020+0000 7f51f1000640 1 -- 172.21.15.152:0/2735583134 wait complete. 2024-04-20T11:32:51.021 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:51.022+0000 7f51f1000640 1 Processor -- start 2024-04-20T11:32:51.022 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:51.022+0000 7f51f1000640 1 -- start start 2024-04-20T11:32:51.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:51.023+0000 7f51f1000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f51ec0723e0 0x7f51ec1e60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:51.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:51.023+0000 7f51f1000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f51ec15d1e0 0x7f51ec147c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:51.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:51.023+0000 7f51f1000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51ec160b90 0x7f51ec1593b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:51.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:51.023+0000 7f51f1000640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f51ec0ccef0 con 0x7f51ec0723e0 2024-04-20T11:32:51.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:51.023+0000 7f51f1000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f51ec0ccd70 con 0x7f51ec160b90 2024-04-20T11:32:51.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:51.023+0000 7f51f1000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f51ec0cd070 con 0x7f51ec15d1e0 2024-04-20T11:32:51.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:51.023+0000 7f51eb400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f51ec0723e0 0x7f51ec1e60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:51.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:51.024+0000 7f51eb400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f51ec0723e0 0x7f51ec1e60f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:45072/0 (socket says 172.21.15.152:45072) 2024-04-20T11:32:51.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:51.024+0000 7f51eb400640 1 -- 172.21.15.152:0/2197496232 learned_addr learned my addr 172.21.15.152:0/2197496232 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:32:51.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:51.024+0000 7f51ebe00640 1 --2- 172.21.15.152:0/2197496232 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51ec160b90 0x7f51ec1593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:51.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:51.024+0000 7f51eaa00640 1 --2- 172.21.15.152:0/2197496232 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f51ec15d1e0 0x7f51ec147c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:51.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:51.024+0000 7f51eb400640 1 -- 172.21.15.152:0/2197496232 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f51ec15d1e0 msgr2=0x7f51ec147c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:51.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:51.024+0000 7f51eb400640 1 --2- 172.21.15.152:0/2197496232 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f51ec15d1e0 0x7f51ec147c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:51.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:51.024+0000 7f51eb400640 1 -- 172.21.15.152:0/2197496232 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51ec160b90 msgr2=0x7f51ec1593b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:51.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:51.024+0000 7f51eb400640 1 --2- 172.21.15.152:0/2197496232 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51ec160b90 0x7f51ec1593b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:51.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:51.024+0000 7f51eb400640 1 -- 172.21.15.152:0/2197496232 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51ec148180 con 0x7f51ec0723e0 2024-04-20T11:32:51.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:51.024+0000 7f51eb400640 1 --2- 172.21.15.152:0/2197496232 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f51ec0723e0 0x7f51ec1e60f0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f51d800c480 tx=0x7f51d800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:51.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:51.025+0000 7f51e3400640 1 -- 172.21.15.152:0/2197496232 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51d80052d0 con 0x7f51ec0723e0 2024-04-20T11:32:51.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:51.025+0000 7f51e3400640 1 -- 172.21.15.152:0/2197496232 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51d8005470 con 0x7f51ec0723e0 2024-04-20T11:32:51.026 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:51.025+0000 7f51e3400640 1 -- 172.21.15.152:0/2197496232 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51d8012870 con 0x7f51ec0723e0 2024-04-20T11:32:51.026 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:51.025+0000 7f51f1000640 1 -- 172.21.15.152:0/2197496232 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51ec0b45d0 con 0x7f51ec0723e0 2024-04-20T11:32:51.026 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:51.025+0000 7f51f1000640 1 -- 172.21.15.152:0/2197496232 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51ec0b52e0 con 0x7f51ec0723e0 2024-04-20T11:32:51.026 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:51.026+0000 7f51f1000640 1 -- 172.21.15.152:0/2197496232 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51ec0b3ea0 con 0x7f51ec0723e0 2024-04-20T11:32:51.026 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:51.027+0000 7f51e3400640 1 -- 172.21.15.152:0/2197496232 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 91142+0+0 (secure 0 0 0) 0x7f51d8012a10 con 0x7f51ec0723e0 2024-04-20T11:32:51.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:51.028+0000 7f51e3400640 1 --2- 172.21.15.152:0/2197496232 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f51cc06ca40 0x7f51cc06ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:51.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:51.028+0000 7f51e3400640 1 -- 172.21.15.152:0/2197496232 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7f51d808ba80 con 0x7f51ec0723e0 2024-04-20T11:32:51.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:51.029+0000 7f51eaa00640 1 --2- 172.21.15.152:0/2197496232 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f51cc06ca40 0x7f51cc06ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:51.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:51.030+0000 7f51eaa00640 1 --2- 172.21.15.152:0/2197496232 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f51cc06ca40 0x7f51cc06ef00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f51ec0af110 tx=0x7f51dc009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:51.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:51.033+0000 7f51e3400640 1 -- 172.21.15.152:0/2197496232 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f51d808de20 con 0x7f51ec0723e0 2024-04-20T11:32:51.276 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:51.277+0000 7f51f1000640 1 -- 172.21.15.152:0/2197496232 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f51ec075e60 con 0x7f51ec0723e0 2024-04-20T11:32:51.277 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:51.277+0000 7f51e3400640 1 -- 172.21.15.152:0/2197496232 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v53) ==== 53+0+73 (secure 0 0 0) 0x7f51ec148180 con 0x7f51ec0723e0 2024-04-20T11:32:51.277 INFO:teuthology.orchestra.run.smithi152.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-20T11:32:51.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:51.280+0000 7f51f1000640 1 -- 172.21.15.152:0/2197496232 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f51cc06ca40 msgr2=0x7f51cc06ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:51.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:51.280+0000 7f51f1000640 1 --2- 172.21.15.152:0/2197496232 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f51cc06ca40 0x7f51cc06ef00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f51ec0af110 tx=0x7f51dc009040 comp rx=0 tx=0).stop 2024-04-20T11:32:51.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:51.280+0000 7f51f1000640 1 -- 172.21.15.152:0/2197496232 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f51ec0723e0 msgr2=0x7f51ec1e60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:51.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:51.280+0000 7f51f1000640 1 --2- 172.21.15.152:0/2197496232 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f51ec0723e0 0x7f51ec1e60f0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f51d800c480 tx=0x7f51d800c940 comp rx=0 tx=0).stop 2024-04-20T11:32:51.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:51.281+0000 7f51f1000640 1 -- 172.21.15.152:0/2197496232 shutdown_connections 2024-04-20T11:32:51.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:51.281+0000 7f51f1000640 1 --2- 172.21.15.152:0/2197496232 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f51cc06ca40 0x7f51cc06ef00 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:51.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:51.281+0000 7f51f1000640 1 --2- 172.21.15.152:0/2197496232 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51ec160b90 0x7f51ec1593b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:51.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:51.281+0000 7f51f1000640 1 --2- 172.21.15.152:0/2197496232 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f51ec15d1e0 0x7f51ec147c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:51.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:51.281+0000 7f51f1000640 1 --2- 172.21.15.152:0/2197496232 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f51ec0723e0 0x7f51ec1e60f0 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:51.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:51.281+0000 7f51f1000640 1 -- 172.21.15.152:0/2197496232 >> 172.21.15.152:0/2197496232 conn(0x7f51ec067ae0 msgr2=0x7f51ec06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:51.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:51.281+0000 7f51f1000640 1 -- 172.21.15.152:0/2197496232 shutdown_connections 2024-04-20T11:32:51.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:51.281+0000 7f51f1000640 1 -- 172.21.15.152:0/2197496232 wait complete. 2024-04-20T11:32:51.300 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-20T11:32:51.300 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-20T11:32:51.300 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi175.front.sepia.ceph.com 2024-04-20T11:32:51.300 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-20T11:32:51.300 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-20T11:32:51.300 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-20T11:32:51.300 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-20T11:32:51.301 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-20T11:32:51.301 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-20T11:32:51.301 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:32:51.301 DEBUG:teuthology.orchestra.run.smithi175:> ip addr 2024-04-20T11:32:51.360 INFO:teuthology.orchestra.run.smithi175.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-20T11:32:51.360 INFO:teuthology.orchestra.run.smithi175.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-20T11:32:51.360 INFO:teuthology.orchestra.run.smithi175.stdout: inet 127.0.0.1/8 scope host lo 2024-04-20T11:32:51.360 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:32:51.360 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 ::1/128 scope host 2024-04-20T11:32:51.360 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:32:51.361 INFO:teuthology.orchestra.run.smithi175.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T11:32:51.361 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:d9:39:ee brd ff:ff:ff:ff:ff:ff 2024-04-20T11:32:51.361 INFO:teuthology.orchestra.run.smithi175.stdout: altname enp7s0f0 2024-04-20T11:32:51.361 INFO:teuthology.orchestra.run.smithi175.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T11:32:51.361 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:d9:39:ef brd ff:ff:ff:ff:ff:ff 2024-04-20T11:32:51.361 INFO:teuthology.orchestra.run.smithi175.stdout: altname enp7s0f1 2024-04-20T11:32:51.361 INFO:teuthology.orchestra.run.smithi175.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T11:32:51.361 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:88:71:f0 brd ff:ff:ff:ff:ff:ff 2024-04-20T11:32:51.361 INFO:teuthology.orchestra.run.smithi175.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-20T11:32:51.361 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:88:71:f1 brd ff:ff:ff:ff:ff:ff 2024-04-20T11:32:51.361 INFO:teuthology.orchestra.run.smithi175.stdout: inet 172.21.15.175/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-20T11:32:51.361 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft 42410sec preferred_lft 42410sec 2024-04-20T11:32:51.361 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::ec4:7aff:fe88:71f1/64 scope link 2024-04-20T11:32:51.362 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:32:51.362 INFO:teuthology.orchestra.run.smithi175.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-20T11:32:51.362 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 02:d6:c6:31:26:70 brd ff:ff:ff:ff:ff:ff 2024-04-20T11:32:51.362 INFO:teuthology.orchestra.run.smithi175.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-20T11:32:51.362 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:32:51.362 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::5091:d5ff:fee7:54b3/64 scope link 2024-04-20T11:32:51.362 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:32:51.362 INFO:teuthology.orchestra.run.smithi175.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-20T11:32:51.362 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 02:d6:c6:31:26:70 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-20T11:32:51.362 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::d6:c6ff:fe31:2670/64 scope link 2024-04-20T11:32:51.362 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:32:51.362 INFO:teuthology.orchestra.run.smithi175.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-20T11:32:51.362 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 9e:14:f3:e2:18:1e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-20T11:32:51.362 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::9c14:f3ff:fee2:181e/64 scope link 2024-04-20T11:32:51.363 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:32:51.363 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:32:51.364 DEBUG:teuthology.orchestra.run.smithi175:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-20T11:32:51.447 INFO:teuthology.orchestra.run.smithi175.stdout:1 2024-04-20T11:32:51.451 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:32:51.451 DEBUG:teuthology.orchestra.run.smithi175:> route 2024-04-20T11:32:51.502 INFO:teuthology.orchestra.run.smithi175.stdout:Kernel IP routing table 2024-04-20T11:32:51.502 INFO:teuthology.orchestra.run.smithi175.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-20T11:32:51.502 INFO:teuthology.orchestra.run.smithi175.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-20T11:32:51.502 INFO:teuthology.orchestra.run.smithi175.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-20T11:32:51.502 INFO:teuthology.orchestra.run.smithi175.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-20T11:32:51.503 DEBUG:teuthology.orchestra.run.smithi175:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T11:32:51.503 DEBUG:teuthology.orchestra.run.smithi175:> set -e 2024-04-20T11:32:51.503 DEBUG:teuthology.orchestra.run.smithi175:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-20T11:32:51.503 DEBUG:teuthology.orchestra.run.smithi175:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-20T11:32:51.503 DEBUG:teuthology.orchestra.run.smithi175:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-20T11:32:51.503 DEBUG:teuthology.orchestra.run.smithi175:> ') 2024-04-20T11:32:51.603 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:32:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T11:32:51.697 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:32:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T11:32:51.706 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:32:51.707 DEBUG:teuthology.orchestra.run.smithi175:> ip netns list 2024-04-20T11:32:51.755 INFO:teuthology.orchestra.run.smithi175.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-20T11:32:51.755 INFO:teuthology.orchestra.run.smithi175.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-20T11:32:51.756 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-20T11:32:51.756 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:32:51.756 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip link set brx.0 up 2024-04-20T11:32:51.835 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-04-20T11:32:51.835 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T11:32:51.835 DEBUG:teuthology.orchestra.run.smithi175:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-20T11:32:51.888 INFO:teuthology.orchestra.run.smithi175.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-20T11:32:51.888 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T11:32:51.889 DEBUG:teuthology.orchestra.run.smithi175:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-20T11:32:51.940 DEBUG:teuthology.orchestra.run.smithi175:> sudo modprobe fuse 2024-04-20T11:32:52.021 DEBUG:teuthology.orchestra.run.smithi175:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T11:32:52.073 INFO:teuthology.orchestra.run.smithi175.stdout:/proc 2024-04-20T11:32:52.073 INFO:teuthology.orchestra.run.smithi175.stdout:/sys 2024-04-20T11:32:52.073 INFO:teuthology.orchestra.run.smithi175.stdout:/dev 2024-04-20T11:32:52.073 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/security 2024-04-20T11:32:52.073 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/shm 2024-04-20T11:32:52.073 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/pts 2024-04-20T11:32:52.073 INFO:teuthology.orchestra.run.smithi175.stdout:/run 2024-04-20T11:32:52.073 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/cgroup 2024-04-20T11:32:52.073 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/pstore 2024-04-20T11:32:52.073 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/bpf 2024-04-20T11:32:52.073 INFO:teuthology.orchestra.run.smithi175.stdout:/ 2024-04-20T11:32:52.074 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/selinux 2024-04-20T11:32:52.074 INFO:teuthology.orchestra.run.smithi175.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T11:32:52.074 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/mqueue 2024-04-20T11:32:52.074 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/debug 2024-04-20T11:32:52.074 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/hugepages 2024-04-20T11:32:52.074 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/tracing 2024-04-20T11:32:52.074 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/fuse/connections 2024-04-20T11:32:52.074 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/config 2024-04-20T11:32:52.074 INFO:teuthology.orchestra.run.smithi175.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T11:32:52.074 INFO:teuthology.orchestra.run.smithi175.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T11:32:52.074 INFO:teuthology.orchestra.run.smithi175.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T11:32:52.074 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T11:32:52.074 INFO:teuthology.orchestra.run.smithi175.stdout:/run/user/1000 2024-04-20T11:32:52.074 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph 2024-04-20T11:32:52.074 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-4 2024-04-20T11:32:52.074 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-5 2024-04-20T11:32:52.075 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-6 2024-04-20T11:32:52.075 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-7 2024-04-20T11:32:52.075 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns 2024-04-20T11:32:52.075 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-20T11:32:52.075 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-20T11:32:52.075 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-20T11:32:52.075 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-20T11:32:52.076 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:32:52.076 DEBUG:teuthology.orchestra.run.smithi175:> ls /sys/fs/fuse/connections 2024-04-20T11:32:52.126 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-20T11:32:52.127 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-04-20T11:32:52.133 DEBUG:teuthology.orchestra.run.smithi175:> sudo modprobe fuse 2024-04-20T11:32:52.205 DEBUG:teuthology.orchestra.run.smithi175:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T11:32:52.248 INFO:teuthology.orchestra.run.smithi175.stdout:/proc 2024-04-20T11:32:52.248 INFO:teuthology.orchestra.run.smithi175.stdout:/sys 2024-04-20T11:32:52.248 INFO:teuthology.orchestra.run.smithi175.stdout:/dev 2024-04-20T11:32:52.248 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/security 2024-04-20T11:32:52.248 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/shm 2024-04-20T11:32:52.248 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/pts 2024-04-20T11:32:52.248 INFO:teuthology.orchestra.run.smithi175.stdout:/run 2024-04-20T11:32:52.248 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/cgroup 2024-04-20T11:32:52.248 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/pstore 2024-04-20T11:32:52.248 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/bpf 2024-04-20T11:32:52.249 INFO:teuthology.orchestra.run.smithi175.stdout:/ 2024-04-20T11:32:52.249 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/selinux 2024-04-20T11:32:52.249 INFO:teuthology.orchestra.run.smithi175.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T11:32:52.249 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/mqueue 2024-04-20T11:32:52.249 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/debug 2024-04-20T11:32:52.249 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/hugepages 2024-04-20T11:32:52.249 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/tracing 2024-04-20T11:32:52.249 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/fuse/connections 2024-04-20T11:32:52.249 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/config 2024-04-20T11:32:52.249 INFO:teuthology.orchestra.run.smithi175.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T11:32:52.249 INFO:teuthology.orchestra.run.smithi175.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T11:32:52.249 INFO:teuthology.orchestra.run.smithi175.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T11:32:52.249 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T11:32:52.250 INFO:teuthology.orchestra.run.smithi175.stdout:/run/user/1000 2024-04-20T11:32:52.250 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph 2024-04-20T11:32:52.250 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-4 2024-04-20T11:32:52.250 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-5 2024-04-20T11:32:52.250 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-6 2024-04-20T11:32:52.250 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-7 2024-04-20T11:32:52.250 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns 2024-04-20T11:32:52.250 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-20T11:32:52.250 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-20T11:32:52.250 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-20T11:32:52.250 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-20T11:32:52.251 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:32:52.251 DEBUG:teuthology.orchestra.run.smithi175:> ls /sys/fs/fuse/connections 2024-04-20T11:32:52.304 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi175.stderr:2024-04-20T11:32:52.306+0000 7f4a9800f480 -1 init, newargv = 0x5605f9719fd0 newargc=15 2024-04-20T11:32:52.305 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi175.stderr:ceph-fuse[33043]: starting ceph client 2024-04-20T11:32:52.315 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi175.stderr:ceph-fuse[33043]: starting fuse 2024-04-20T11:32:52.334 INFO:teuthology.orchestra.run.smithi175.stdout:42 2024-04-20T11:32:52.335 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-04-20T11:32:52.335 DEBUG:teuthology.orchestra.run.smithi175:> sudo stdin-killer -- python3 -c ' 2024-04-20T11:32:52.335 DEBUG:teuthology.orchestra.run.smithi175:> import glob 2024-04-20T11:32:52.335 DEBUG:teuthology.orchestra.run.smithi175:> import re 2024-04-20T11:32:52.335 DEBUG:teuthology.orchestra.run.smithi175:> import os 2024-04-20T11:32:52.335 DEBUG:teuthology.orchestra.run.smithi175:> import subprocess 2024-04-20T11:32:52.335 DEBUG:teuthology.orchestra.run.smithi175:> 2024-04-20T11:32:52.335 DEBUG:teuthology.orchestra.run.smithi175:> def _find_admin_socket(client_name): 2024-04-20T11:32:52.336 DEBUG:teuthology.orchestra.run.smithi175:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-04-20T11:32:52.336 DEBUG:teuthology.orchestra.run.smithi175:> files = glob.glob(asok_path) 2024-04-20T11:32:52.336 DEBUG:teuthology.orchestra.run.smithi175:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-04-20T11:32:52.336 DEBUG:teuthology.orchestra.run.smithi175:> 2024-04-20T11:32:52.336 DEBUG:teuthology.orchestra.run.smithi175:> # Given a non-glob path, it better be there 2024-04-20T11:32:52.336 DEBUG:teuthology.orchestra.run.smithi175:> if "*" not in asok_path: 2024-04-20T11:32:52.336 DEBUG:teuthology.orchestra.run.smithi175:> assert(len(files) == 1) 2024-04-20T11:32:52.336 DEBUG:teuthology.orchestra.run.smithi175:> return files[0] 2024-04-20T11:32:52.336 DEBUG:teuthology.orchestra.run.smithi175:> 2024-04-20T11:32:52.336 DEBUG:teuthology.orchestra.run.smithi175:> for f in files: 2024-04-20T11:32:52.336 DEBUG:teuthology.orchestra.run.smithi175:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-20T11:32:52.336 DEBUG:teuthology.orchestra.run.smithi175:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-20T11:32:52.336 DEBUG:teuthology.orchestra.run.smithi175:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-20T11:32:52.336 DEBUG:teuthology.orchestra.run.smithi175:> contents = proc_f.read() 2024-04-20T11:32:52.336 DEBUG:teuthology.orchestra.run.smithi175:> if mountpoint in contents: 2024-04-20T11:32:52.337 DEBUG:teuthology.orchestra.run.smithi175:> return f 2024-04-20T11:32:52.337 DEBUG:teuthology.orchestra.run.smithi175:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-20T11:32:52.337 DEBUG:teuthology.orchestra.run.smithi175:> 2024-04-20T11:32:52.337 DEBUG:teuthology.orchestra.run.smithi175:> print(_find_admin_socket("client.1")) 2024-04-20T11:32:52.337 DEBUG:teuthology.orchestra.run.smithi175:> ' 2024-04-20T11:32:52.524 INFO:teuthology.orchestra.run.smithi175.stdout:/var/run/ceph/ceph-client.1.33043.asok 2024-04-20T11:32:52.528 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:32:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T11:32:52.541 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.33043.asok 2024-04-20T11:32:52.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:32:52.541 DEBUG:teuthology.orchestra.run.smithi175:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.33043.asok status 2024-04-20T11:32:52.602 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:32:52.610 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:32:52.768 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-04-20T11:32:52.768 INFO:teuthology.orchestra.run.smithi175.stdout: "metadata": { 2024-04-20T11:32:52.768 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_sha1": "a8c84f5997950fd7d008204839253291d52e6eda", 2024-04-20T11:32:52.768 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", 2024-04-20T11:32:52.768 INFO:teuthology.orchestra.run.smithi175.stdout: "entity_id": "1", 2024-04-20T11:32:52.768 INFO:teuthology.orchestra.run.smithi175.stdout: "hostname": "smithi175", 2024-04-20T11:32:52.769 INFO:teuthology.orchestra.run.smithi175.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-04-20T11:32:52.769 INFO:teuthology.orchestra.run.smithi175.stdout: "pid": "33043", 2024-04-20T11:32:52.769 INFO:teuthology.orchestra.run.smithi175.stdout: "root": "/" 2024-04-20T11:32:52.769 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-20T11:32:52.769 INFO:teuthology.orchestra.run.smithi175.stdout: "dentry_count": 0, 2024-04-20T11:32:52.769 INFO:teuthology.orchestra.run.smithi175.stdout: "dentry_pinned_count": 0, 2024-04-20T11:32:52.769 INFO:teuthology.orchestra.run.smithi175.stdout: "id": 5995, 2024-04-20T11:32:52.769 INFO:teuthology.orchestra.run.smithi175.stdout: "inst": { 2024-04-20T11:32:52.769 INFO:teuthology.orchestra.run.smithi175.stdout: "name": { 2024-04-20T11:32:52.769 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "client", 2024-04-20T11:32:52.769 INFO:teuthology.orchestra.run.smithi175.stdout: "num": 5995 2024-04-20T11:32:52.769 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-20T11:32:52.769 INFO:teuthology.orchestra.run.smithi175.stdout: "addr": { 2024-04-20T11:32:52.769 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "v1", 2024-04-20T11:32:52.770 INFO:teuthology.orchestra.run.smithi175.stdout: "addr": "192.168.0.1:0", 2024-04-20T11:32:52.770 INFO:teuthology.orchestra.run.smithi175.stdout: "nonce": 171246516 2024-04-20T11:32:52.770 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-04-20T11:32:52.770 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-20T11:32:52.770 INFO:teuthology.orchestra.run.smithi175.stdout: "addr": { 2024-04-20T11:32:52.770 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "v1", 2024-04-20T11:32:52.770 INFO:teuthology.orchestra.run.smithi175.stdout: "addr": "192.168.0.1:0", 2024-04-20T11:32:52.770 INFO:teuthology.orchestra.run.smithi175.stdout: "nonce": 171246516 2024-04-20T11:32:52.770 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-20T11:32:52.770 INFO:teuthology.orchestra.run.smithi175.stdout: "inst_str": "client.5995 192.168.0.1:0/171246516", 2024-04-20T11:32:52.770 INFO:teuthology.orchestra.run.smithi175.stdout: "addr_str": "192.168.0.1:0/171246516", 2024-04-20T11:32:52.770 INFO:teuthology.orchestra.run.smithi175.stdout: "inode_count": 1, 2024-04-20T11:32:52.770 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_epoch": 53, 2024-04-20T11:32:52.771 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_epoch": 83, 2024-04-20T11:32:52.771 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_epoch_barrier": 0, 2024-04-20T11:32:52.771 INFO:teuthology.orchestra.run.smithi175.stdout: "blocklisted": false, 2024-04-20T11:32:52.771 INFO:teuthology.orchestra.run.smithi175.stdout: "fs_name": "cephfs" 2024-04-20T11:32:52.771 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-04-20T11:32:52.783 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:32:52.783 DEBUG:teuthology.orchestra.run.smithi175:> stat --file-system '--printf=%T 2024-04-20T11:32:52.783 DEBUG:teuthology.orchestra.run.smithi175:> ' -- /home/ubuntu/cephtest/mnt.1 2024-04-20T11:32:52.879 INFO:teuthology.orchestra.run.smithi175.stdout:fuseblk 2024-04-20T11:32:52.879 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-04-20T11:32:52.879 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:32:52.879 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-20T11:32:52.960 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-04-20T11:32:53.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:53.178+0000 7fc27b600640 1 Processor -- start 2024-04-20T11:32:53.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:53.179+0000 7fc27b600640 1 -- start start 2024-04-20T11:32:53.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:53.180+0000 7fc27b600640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fc27415a450 0x7fc2740cd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:53.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:53.180+0000 7fc27b600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc2740ce1e0 0x7fc27415cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:53.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:53.180+0000 7fc27b600640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2740cdcf0 con 0x7fc27415d1b0 2024-04-20T11:32:53.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:53.180+0000 7fc27b600640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fc2740cd320 con 0x7fc27415a450 2024-04-20T11:32:53.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:53.180+0000 7fc27b600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc27415d710 con 0x7fc2740ce1e0 2024-04-20T11:32:53.181 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:53.181+0000 7fc279600640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fc27415d1b0 0x7fc274071db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:41188/0 (socket says 172.21.15.152:41188) 2024-04-20T11:32:53.181 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:53.181+0000 7fc279600640 1 -- 172.21.15.152:0/1143919099 learned_addr learned my addr 172.21.15.152:0/1143919099 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:32:53.184 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:53.181+0000 7fc278c00640 1 --2- 172.21.15.152:0/1143919099 >> v2:172.21.15.175:3300/0 conn(0x7fc27415a450 0x7fc2740cd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:53.184 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:53.181+0000 7fc26be00640 1 --2- 172.21.15.152:0/1143919099 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc2740ce1e0 0x7fc27415cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:53.184 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:53.181+0000 7fc273e00640 1 -- 172.21.15.152:0/1143919099 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3388620672 0 0) 0x7fc26c001490 con 0x7fc27415d1b0 2024-04-20T11:32:53.184 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:53.181+0000 7fc273e00640 1 -- 172.21.15.152:0/1143919099 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc258003520 con 0x7fc27415d1b0 2024-04-20T11:32:53.184 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:53.181+0000 7fc273e00640 1 -- 172.21.15.152:0/1143919099 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 344832782 0 0) 0x7fc258003520 con 0x7fc27415d1b0 2024-04-20T11:32:53.184 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:53.181+0000 7fc273e00640 1 -- 172.21.15.152:0/1143919099 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc2740ce1e0 msgr2=0x7fc27415cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:53.184 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:53.181+0000 7fc273e00640 1 --2- 172.21.15.152:0/1143919099 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc2740ce1e0 0x7fc27415cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:53.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:53.181+0000 7fc273e00640 1 -- 172.21.15.152:0/1143919099 >> v2:172.21.15.175:3300/0 conn(0x7fc27415a450 msgr2=0x7fc2740cd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:53.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:53.181+0000 7fc273e00640 1 --2- 172.21.15.152:0/1143919099 >> v2:172.21.15.175:3300/0 conn(0x7fc27415a450 0x7fc2740cd7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:53.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:53.181+0000 7fc273e00640 1 -- 172.21.15.152:0/1143919099 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc27415d9f0 con 0x7fc27415d1b0 2024-04-20T11:32:53.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:53.181+0000 7fc273e00640 1 -- 172.21.15.152:0/1143919099 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7fc26c001bc0 con 0x7fc27415d1b0 2024-04-20T11:32:53.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:53.182+0000 7fc273e00640 1 -- 172.21.15.152:0/1143919099 >> v1:172.21.15.152:6789/0 conn(0x7fc27415d1b0 legacy=0x7fc274071db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:53.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:53.182+0000 7fc273e00640 1 --2- 172.21.15.152:0/1143919099 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc2740ce1e0 0x7fc258003a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:53.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:53.182+0000 7fc273e00640 1 --2- 172.21.15.152:0/1143919099 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc258003ff0 0x7fc258004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:53.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:53.182+0000 7fc273e00640 1 --2- 172.21.15.152:0/1143919099 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc258004d40 0x7fc2580051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:53.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:53.182+0000 7fc273e00640 1 -- 172.21.15.152:0/1143919099 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fc2740cd320 con 0x7fc258004d40 2024-04-20T11:32:53.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:53.182+0000 7fc273e00640 1 -- 172.21.15.152:0/1143919099 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc27415d710 con 0x7fc258003ff0 2024-04-20T11:32:53.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:53.182+0000 7fc273e00640 1 -- 172.21.15.152:0/1143919099 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc25800fee0 con 0x7fc2740ce1e0 2024-04-20T11:32:53.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:53.182+0000 7fc279600640 1 --2- 172.21.15.152:0/1143919099 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc258004d40 0x7fc2580051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:53.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:53.182+0000 7fc278c00640 1 --2- 172.21.15.152:0/1143919099 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc2740ce1e0 0x7fc258003a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:53.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:53.182+0000 7fc279600640 1 -- 172.21.15.152:0/1143919099 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc2740ce1e0 msgr2=0x7fc258003a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:53.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:53.182+0000 7fc279600640 1 --2- 172.21.15.152:0/1143919099 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc2740ce1e0 0x7fc258003a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:53.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:53.182+0000 7fc279600640 1 -- 172.21.15.152:0/1143919099 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc258003ff0 msgr2=0x7fc258004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:53.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:53.182+0000 7fc26be00640 1 --2- 172.21.15.152:0/1143919099 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc258003ff0 0x7fc258004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:53.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:53.182+0000 7fc279600640 1 --2- 172.21.15.152:0/1143919099 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc258003ff0 0x7fc258004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:53.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:53.182+0000 7fc279600640 1 -- 172.21.15.152:0/1143919099 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc258003520 con 0x7fc258004d40 2024-04-20T11:32:53.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:53.183+0000 7fc279600640 1 --2- 172.21.15.152:0/1143919099 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc258004d40 0x7fc2580051c0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7fc26c00a970 tx=0x7fc258005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97a5cee52ecfa5f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:53.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:53.184+0000 7fc273e00640 1 -- 172.21.15.152:0/1143919099 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc26c00a5c0 con 0x7fc258004d40 2024-04-20T11:32:53.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:53.184+0000 7fc273e00640 1 -- 172.21.15.152:0/1143919099 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc26c007290 con 0x7fc258004d40 2024-04-20T11:32:53.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:53.184+0000 7fc273e00640 1 -- 172.21.15.152:0/1143919099 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc26c0076c0 con 0x7fc258004d40 2024-04-20T11:32:53.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:53.184+0000 7fc27b600640 1 -- 172.21.15.152:0/1143919099 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc258004d40 msgr2=0x7fc2580051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:53.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:53.184+0000 7fc27b600640 1 --2- 172.21.15.152:0/1143919099 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc258004d40 0x7fc2580051c0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7fc26c00a970 tx=0x7fc258005830 comp rx=0 tx=0).stop 2024-04-20T11:32:53.187 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:53.184+0000 7fc279600640 1 -- 172.21.15.152:0/1143919099 reap_dead start 2024-04-20T11:32:53.187 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:53.184+0000 7fc27b600640 1 -- 172.21.15.152:0/1143919099 shutdown_connections 2024-04-20T11:32:53.187 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:53.184+0000 7fc27b600640 1 -- 172.21.15.152:0/1143919099 >> 172.21.15.152:0/1143919099 conn(0x7fc274067cf0 msgr2=0x7fc27406b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:53.187 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:53.185+0000 7fc27b600640 1 -- 172.21.15.152:0/1143919099 shutdown_connections 2024-04-20T11:32:53.187 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:53.185+0000 7fc27b600640 1 -- 172.21.15.152:0/1143919099 wait complete. 2024-04-20T11:32:53.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:53.186+0000 7fc27b600640 1 Processor -- start 2024-04-20T11:32:53.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:53.186+0000 7fc27b600640 1 -- start start 2024-04-20T11:32:53.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:53.187+0000 7fc27b600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc258004d40 0x7fc27413c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:53.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:53.187+0000 7fc27b600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc2740b2cc0 0x7fc274158cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:53.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:53.187+0000 7fc27b600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc27413ab40 0x7fc274142690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:53.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:53.187+0000 7fc27b600640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fc274072640 con 0x7fc258004d40 2024-04-20T11:32:53.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:53.187+0000 7fc27b600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc2740724c0 con 0x7fc2740b2cc0 2024-04-20T11:32:53.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:53.187+0000 7fc27b600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc27415cc70 con 0x7fc27413ab40 2024-04-20T11:32:53.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:53.188+0000 7fc278c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc258004d40 0x7fc27413c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:53.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:53.188+0000 7fc278c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc258004d40 0x7fc27413c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:45094/0 (socket says 172.21.15.152:45094) 2024-04-20T11:32:53.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:53.188+0000 7fc278c00640 1 -- 172.21.15.152:0/3884570062 learned_addr learned my addr 172.21.15.152:0/3884570062 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:32:53.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:53.188+0000 7fc279600640 1 --2- 172.21.15.152:0/3884570062 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc27413ab40 0x7fc274142690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:53.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:53.188+0000 7fc26be00640 1 --2- 172.21.15.152:0/3884570062 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc2740b2cc0 0x7fc274158cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:53.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:53.188+0000 7fc278c00640 1 -- 172.21.15.152:0/3884570062 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc27413ab40 msgr2=0x7fc274142690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:53.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:53.188+0000 7fc278c00640 1 --2- 172.21.15.152:0/3884570062 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc27413ab40 0x7fc274142690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:53.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:53.188+0000 7fc278c00640 1 -- 172.21.15.152:0/3884570062 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc2740b2cc0 msgr2=0x7fc274158cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:53.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:53.188+0000 7fc278c00640 1 --2- 172.21.15.152:0/3884570062 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc2740b2cc0 0x7fc274158cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:53.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:53.188+0000 7fc278c00640 1 -- 172.21.15.152:0/3884570062 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc27413b420 con 0x7fc258004d40 2024-04-20T11:32:53.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:53.189+0000 7fc278c00640 1 --2- 172.21.15.152:0/3884570062 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc258004d40 0x7fc27413c4c0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7fc264011700 tx=0x7fc264011bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:53.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:53.189+0000 7fc271600640 1 -- 172.21.15.152:0/3884570062 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc26400f9c0 con 0x7fc258004d40 2024-04-20T11:32:53.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:53.189+0000 7fc271600640 1 -- 172.21.15.152:0/3884570062 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc26400fb60 con 0x7fc258004d40 2024-04-20T11:32:53.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:53.189+0000 7fc271600640 1 -- 172.21.15.152:0/3884570062 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc264015460 con 0x7fc258004d40 2024-04-20T11:32:53.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:53.189+0000 7fc27b600640 1 -- 172.21.15.152:0/3884570062 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc27413b700 con 0x7fc258004d40 2024-04-20T11:32:53.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:53.189+0000 7fc27b600640 1 -- 172.21.15.152:0/3884570062 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc27413bcc0 con 0x7fc258004d40 2024-04-20T11:32:53.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:53.190+0000 7fc27b600640 1 -- 172.21.15.152:0/3884570062 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc274068a60 con 0x7fc258004d40 2024-04-20T11:32:53.193 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:53.191+0000 7fc271600640 1 -- 172.21.15.152:0/3884570062 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 91142+0+0 (secure 0 0 0) 0x7fc264015600 con 0x7fc258004d40 2024-04-20T11:32:53.193 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:53.192+0000 7fc271600640 1 --2- 172.21.15.152:0/3884570062 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fc25006ca40 0x7fc25006ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:53.193 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:53.192+0000 7fc271600640 1 -- 172.21.15.152:0/3884570062 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7fc26408c7d0 con 0x7fc258004d40 2024-04-20T11:32:53.193 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:53.193+0000 7fc26be00640 1 --2- 172.21.15.152:0/3884570062 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fc25006ca40 0x7fc25006ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:53.193 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:53.194+0000 7fc26be00640 1 --2- 172.21.15.152:0/3884570062 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fc25006ca40 0x7fc25006ef00 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc27415fd80 tx=0x7fc25c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:53.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:53.197+0000 7fc271600640 1 -- 172.21.15.152:0/3884570062 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fc264019030 con 0x7fc258004d40 2024-04-20T11:32:53.436 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:53.437+0000 7fc27b600640 1 -- 172.21.15.152:0/3884570062 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7fc274072f10 con 0x7fc258004d40 2024-04-20T11:32:54.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.149+0000 7fc271600640 1 -- 172.21.15.152:0/3884570062 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v54) ==== 136+0+0 (secure 0 0 0) 0x7fc26405a9c0 con 0x7fc258004d40 2024-04-20T11:32:54.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.152+0000 7fc27b600640 1 -- 172.21.15.152:0/3884570062 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fc25006ca40 msgr2=0x7fc25006ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:54.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.152+0000 7fc27b600640 1 --2- 172.21.15.152:0/3884570062 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fc25006ca40 0x7fc25006ef00 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc27415fd80 tx=0x7fc25c008040 comp rx=0 tx=0).stop 2024-04-20T11:32:54.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.152+0000 7fc27b600640 1 -- 172.21.15.152:0/3884570062 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc258004d40 msgr2=0x7fc27413c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:54.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.152+0000 7fc27b600640 1 --2- 172.21.15.152:0/3884570062 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc258004d40 0x7fc27413c4c0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7fc264011700 tx=0x7fc264011bc0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.152+0000 7fc27b600640 1 -- 172.21.15.152:0/3884570062 shutdown_connections 2024-04-20T11:32:54.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.152+0000 7fc27b600640 1 --2- 172.21.15.152:0/3884570062 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fc25006ca40 0x7fc25006ef00 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.152+0000 7fc27b600640 1 --2- 172.21.15.152:0/3884570062 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc27413ab40 0x7fc274142690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.152+0000 7fc27b600640 1 --2- 172.21.15.152:0/3884570062 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc2740b2cc0 0x7fc274158cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.152+0000 7fc27b600640 1 --2- 172.21.15.152:0/3884570062 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc258004d40 0x7fc27413c4c0 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.152+0000 7fc27b600640 1 -- 172.21.15.152:0/3884570062 >> 172.21.15.152:0/3884570062 conn(0x7fc274067cf0 msgr2=0x7fc2740ce5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:54.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.153+0000 7fc27b600640 1 -- 172.21.15.152:0/3884570062 shutdown_connections 2024-04-20T11:32:54.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.153+0000 7fc27b600640 1 -- 172.21.15.152:0/3884570062 wait complete. 2024-04-20T11:32:54.174 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-04-20T11:32:54.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.372+0000 7f7dc8600640 1 Processor -- start 2024-04-20T11:32:54.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.373+0000 7f7dc8600640 1 -- start start 2024-04-20T11:32:54.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.374+0000 7f7dc8600640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f7dc00ce9a0 0x7f7dc00d0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:54.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.374+0000 7f7dc8600640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f7dc006d880 0x7f7dc015e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:54.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.374+0000 7f7dc8600640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7dc00d0e10 con 0x7f7dc009b710 2024-04-20T11:32:54.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.374+0000 7f7dc8600640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f7dc00d1a20 con 0x7f7dc006d880 2024-04-20T11:32:54.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.374+0000 7f7dc8600640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f7dc00d10f0 con 0x7f7dc00ce9a0 2024-04-20T11:32:54.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.374+0000 7f7dc6600640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f7dc006d880 0x7f7dc015e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:54.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.374+0000 7f7dc5200640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f7dc009b710 0x7f7dc006d170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:41196/0 (socket says 172.21.15.152:41196) 2024-04-20T11:32:54.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.374+0000 7f7dc5200640 1 -- 172.21.15.152:0/1967924527 learned_addr learned my addr 172.21.15.152:0/1967924527 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:32:54.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.375+0000 7f7dc5c00640 1 --2- 172.21.15.152:0/1967924527 >> v2:172.21.15.175:3300/0 conn(0x7f7dc00ce9a0 0x7f7dc00d0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:54.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.375+0000 7f7dc6600640 1 -- 172.21.15.152:0/1967924527 >> v2:172.21.15.175:3300/0 conn(0x7f7dc00ce9a0 msgr2=0x7f7dc00d0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:54.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.375+0000 7f7dc6600640 1 --2- 172.21.15.152:0/1967924527 >> v2:172.21.15.175:3300/0 conn(0x7f7dc00ce9a0 0x7f7dc00d0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.375+0000 7f7dbfe00640 1 -- 172.21.15.152:0/1967924527 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3023531330 0 0) 0x7f7dc00d0e10 con 0x7f7dc009b710 2024-04-20T11:32:54.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.375+0000 7f7dc6600640 1 -- 172.21.15.152:0/1967924527 >> v1:172.21.15.152:6789/0 conn(0x7f7dc009b710 legacy=0x7f7dc006d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:54.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.375+0000 7f7dc6600640 1 -- 172.21.15.152:0/1967924527 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dc00d13d0 con 0x7f7dc006d880 2024-04-20T11:32:54.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.376+0000 7f7dc6600640 1 --2- 172.21.15.152:0/1967924527 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7dc006d880 0x7f7dc015e1c0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f7dc006c3b0 tx=0x7f7db802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1891ed6c46ad81b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:54.377 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.376+0000 7f7dbfe00640 1 -- 172.21.15.152:0/1967924527 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db80315a0 con 0x7f7dc006d880 2024-04-20T11:32:54.377 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.377+0000 7f7dbfe00640 1 -- 172.21.15.152:0/1967924527 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7db8037070 con 0x7f7dc006d880 2024-04-20T11:32:54.377 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.377+0000 7f7dbfe00640 1 -- 172.21.15.152:0/1967924527 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db8031960 con 0x7f7dc006d880 2024-04-20T11:32:54.377 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.377+0000 7f7dc8600640 1 -- 172.21.15.152:0/1967924527 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7dc006d880 msgr2=0x7f7dc015e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:54.377 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.377+0000 7f7dc8600640 1 --2- 172.21.15.152:0/1967924527 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7dc006d880 0x7f7dc015e1c0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f7dc006c3b0 tx=0x7f7db802b840 comp rx=0 tx=0).stop 2024-04-20T11:32:54.377 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.377+0000 7f7dc8600640 1 -- 172.21.15.152:0/1967924527 shutdown_connections 2024-04-20T11:32:54.377 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.377+0000 7f7dc8600640 1 --2- 172.21.15.152:0/1967924527 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7dc006d880 0x7f7dc015e1c0 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.378 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.377+0000 7f7dc8600640 1 --2- 172.21.15.152:0/1967924527 >> v2:172.21.15.175:3300/0 conn(0x7f7dc00ce9a0 0x7f7dc00d0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.378 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.377+0000 7f7dc8600640 1 -- 172.21.15.152:0/1967924527 >> 172.21.15.152:0/1967924527 conn(0x7f7dc00b4040 msgr2=0x7f7dc00b4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:54.378 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.377+0000 7f7dc8600640 1 -- 172.21.15.152:0/1967924527 shutdown_connections 2024-04-20T11:32:54.378 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.377+0000 7f7dc8600640 1 -- 172.21.15.152:0/1967924527 wait complete. 2024-04-20T11:32:54.378 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.379+0000 7f7dc8600640 1 Processor -- start 2024-04-20T11:32:54.378 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.379+0000 7f7dc8600640 1 -- start start 2024-04-20T11:32:54.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.380+0000 7f7dc8600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7dc006d880 0x7f7dc0150ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:54.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.380+0000 7f7dc8600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7dc009b710 0x7f7dc00ca670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:54.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.380+0000 7f7dc8600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7dc00ce9a0 0x7f7dc00c6080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:54.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.380+0000 7f7dc8600640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f7dc009bb50 con 0x7f7dc00ce9a0 2024-04-20T11:32:54.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.380+0000 7f7dc8600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7dc00ced80 con 0x7f7dc006d880 2024-04-20T11:32:54.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.380+0000 7f7dc8600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7dc009bcd0 con 0x7f7dc009b710 2024-04-20T11:32:54.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.380+0000 7f7dc6600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7dc00ce9a0 0x7f7dc00c6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:54.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.380+0000 7f7dc6600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7dc00ce9a0 0x7f7dc00c6080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:45126/0 (socket says 172.21.15.152:45126) 2024-04-20T11:32:54.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.380+0000 7f7dc6600640 1 -- 172.21.15.152:0/3700737405 learned_addr learned my addr 172.21.15.152:0/3700737405 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:32:54.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.381+0000 7f7dc5200640 1 --2- 172.21.15.152:0/3700737405 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7dc009b710 0x7f7dc00ca670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:54.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.381+0000 7f7dc6600640 1 -- 172.21.15.152:0/3700737405 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7dc009b710 msgr2=0x7f7dc00ca670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:54.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.381+0000 7f7dc5c00640 1 --2- 172.21.15.152:0/3700737405 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7dc006d880 0x7f7dc0150ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:54.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.381+0000 7f7dc6600640 1 --2- 172.21.15.152:0/3700737405 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7dc009b710 0x7f7dc00ca670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.381+0000 7f7dc6600640 1 -- 172.21.15.152:0/3700737405 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7dc006d880 msgr2=0x7f7dc0150ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:54.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.381+0000 7f7dc6600640 1 --2- 172.21.15.152:0/3700737405 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7dc006d880 0x7f7dc0150ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.381+0000 7f7dc6600640 1 -- 172.21.15.152:0/3700737405 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dc01f7f30 con 0x7f7dc00ce9a0 2024-04-20T11:32:54.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.381+0000 7f7dc6600640 1 --2- 172.21.15.152:0/3700737405 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7dc00ce9a0 0x7f7dc00c6080 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f7db8009890 tx=0x7f7db80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:54.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.381+0000 7f7dbd600640 1 -- 172.21.15.152:0/3700737405 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db8004230 con 0x7f7dc00ce9a0 2024-04-20T11:32:54.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.381+0000 7f7dbd600640 1 -- 172.21.15.152:0/3700737405 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7db80043f0 con 0x7f7dc00ce9a0 2024-04-20T11:32:54.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.381+0000 7f7dbd600640 1 -- 172.21.15.152:0/3700737405 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db8003850 con 0x7f7dc00ce9a0 2024-04-20T11:32:54.383 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.381+0000 7f7dc8600640 1 -- 172.21.15.152:0/3700737405 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dc01f8210 con 0x7f7dc00ce9a0 2024-04-20T11:32:54.383 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.382+0000 7f7dc8600640 1 -- 172.21.15.152:0/3700737405 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dc01f87d0 con 0x7f7dc00ce9a0 2024-04-20T11:32:54.383 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.383+0000 7f7dc8600640 1 -- 172.21.15.152:0/3700737405 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7dc00689f0 con 0x7f7dc00ce9a0 2024-04-20T11:32:54.383 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.384+0000 7f7dbd600640 1 -- 172.21.15.152:0/3700737405 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 91142+0+0 (secure 0 0 0) 0x7f7db80039f0 con 0x7f7dc00ce9a0 2024-04-20T11:32:54.384 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.385+0000 7f7dbd600640 1 --2- 172.21.15.152:0/3700737405 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f7da006ca10 0x7f7da006eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:54.384 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.385+0000 7f7dbd600640 1 -- 172.21.15.152:0/3700737405 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7f7db802f030 con 0x7f7dc00ce9a0 2024-04-20T11:32:54.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.385+0000 7f7dc5c00640 1 --2- 172.21.15.152:0/3700737405 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f7da006ca10 0x7f7da006eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:54.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.387+0000 7f7dc5c00640 1 --2- 172.21.15.152:0/3700737405 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f7da006ca10 0x7f7da006eed0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7db0006fd0 tx=0x7f7db0008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:54.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.390+0000 7f7dbd600640 1 -- 172.21.15.152:0/3700737405 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f7db8041070 con 0x7f7dc00ce9a0 2024-04-20T11:32:54.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.613+0000 7f7dc8600640 1 -- 172.21.15.152:0/3700737405 --> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f7dc00b4ca0 con 0x7f7da006ca10 2024-04-20T11:32:54.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.618+0000 7f7dbd600640 1 -- 172.21.15.152:0/3700737405 <== mgr.4109 v2:172.21.15.175:6836/4021250441 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f7dc00b4ca0 con 0x7f7da006ca10 2024-04-20T11:32:54.617 INFO:teuthology.orchestra.run.smithi152.stdout:[ 2024-04-20T11:32:54.618 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:32:54.618 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "cephfs" 2024-04-20T11:32:54.618 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:32:54.618 INFO:teuthology.orchestra.run.smithi152.stdout:] 2024-04-20T11:32:54.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.620+0000 7f7dc8600640 1 -- 172.21.15.152:0/3700737405 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f7da006ca10 msgr2=0x7f7da006eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:54.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.620+0000 7f7dc8600640 1 --2- 172.21.15.152:0/3700737405 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f7da006ca10 0x7f7da006eed0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7db0006fd0 tx=0x7f7db0008040 comp rx=0 tx=0).stop 2024-04-20T11:32:54.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.620+0000 7f7dc8600640 1 -- 172.21.15.152:0/3700737405 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7dc00ce9a0 msgr2=0x7f7dc00c6080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:54.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.620+0000 7f7dc8600640 1 --2- 172.21.15.152:0/3700737405 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7dc00ce9a0 0x7f7dc00c6080 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f7db8009890 tx=0x7f7db80046c0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.620+0000 7f7dc8600640 1 -- 172.21.15.152:0/3700737405 shutdown_connections 2024-04-20T11:32:54.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.620+0000 7f7dc8600640 1 --2- 172.21.15.152:0/3700737405 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f7da006ca10 0x7f7da006eed0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.620+0000 7f7dc8600640 1 --2- 172.21.15.152:0/3700737405 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7dc00ce9a0 0x7f7dc00c6080 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.620+0000 7f7dc8600640 1 --2- 172.21.15.152:0/3700737405 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7dc009b710 0x7f7dc00ca670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.620+0000 7f7dc8600640 1 --2- 172.21.15.152:0/3700737405 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7dc006d880 0x7f7dc0150ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.620+0000 7f7dc8600640 1 -- 172.21.15.152:0/3700737405 >> 172.21.15.152:0/3700737405 conn(0x7f7dc00b4040 msgr2=0x7f7dc00b12b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:54.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.620+0000 7f7dc8600640 1 -- 172.21.15.152:0/3700737405 shutdown_connections 2024-04-20T11:32:54.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.620+0000 7f7dc8600640 1 -- 172.21.15.152:0/3700737405 wait complete. 2024-04-20T11:32:54.638 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-20T11:32:54.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.843+0000 7fabbf600640 1 Processor -- start 2024-04-20T11:32:54.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.843+0000 7fabbf600640 1 -- start start 2024-04-20T11:32:54.846 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.845+0000 7fabbf600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fabb8146000 0x7fabb80b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:54.846 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.845+0000 7fabbf600640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fabb80cd8c0 0x7fabb80b25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:54.846 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.845+0000 7fabbf600640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fabb81463e0 con 0x7fabb80b2170 2024-04-20T11:32:54.846 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.845+0000 7fabbf600640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fabb80b4750 con 0x7fabb80cd8c0 2024-04-20T11:32:54.846 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.845+0000 7fabbf600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fabb80b3e20 con 0x7fabb8146000 2024-04-20T11:32:54.846 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.845+0000 7fabbd600640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fabb80cd8c0 0x7fabb80b25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:54.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.845+0000 7fabbd600640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fabb80cd8c0 0x7fabb80b25f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:45136/0 (socket says 172.21.15.152:45136) 2024-04-20T11:32:54.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.845+0000 7fabbd600640 1 -- 172.21.15.152:0/2206562322 learned_addr learned my addr 172.21.15.152:0/2206562322 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:32:54.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.845+0000 7fabbcc00640 1 --2- 172.21.15.152:0/2206562322 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fabb8146000 0x7fabb80b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:54.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.846+0000 7fabbd600640 1 -- 172.21.15.152:0/2206562322 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fabb8146000 msgr2=0x7fabb80b4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:54.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.846+0000 7fabbd600640 1 --2- 172.21.15.152:0/2206562322 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fabb8146000 0x7fabb80b4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.846+0000 7fabbd600640 1 -- 172.21.15.152:0/2206562322 >> v1:172.21.15.175:6789/0 conn(0x7fabb80b2170 legacy=0x7fabb80cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:54.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.846+0000 7fabbd600640 1 -- 172.21.15.152:0/2206562322 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabb80b4100 con 0x7fabb80cd8c0 2024-04-20T11:32:54.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.847+0000 7fabbd600640 1 --2- 172.21.15.152:0/2206562322 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fabb80cd8c0 0x7fabb80b25f0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7fabb0009e50 tx=0x7fabb002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7089e61518c759f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:54.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.847+0000 7fabb7400640 1 -- 172.21.15.152:0/2206562322 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb002ca80 con 0x7fabb80cd8c0 2024-04-20T11:32:54.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.847+0000 7fabb7400640 1 -- 172.21.15.152:0/2206562322 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fabb002cc20 con 0x7fabb80cd8c0 2024-04-20T11:32:54.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.847+0000 7fabb7400640 1 -- 172.21.15.152:0/2206562322 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb00343f0 con 0x7fabb80cd8c0 2024-04-20T11:32:54.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.848+0000 7fabbf600640 1 -- 172.21.15.152:0/2206562322 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fabb80cd8c0 msgr2=0x7fabb80b25f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:54.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.848+0000 7fabbf600640 1 --2- 172.21.15.152:0/2206562322 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fabb80cd8c0 0x7fabb80b25f0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7fabb0009e50 tx=0x7fabb002bee0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.848+0000 7fabbf600640 1 -- 172.21.15.152:0/2206562322 shutdown_connections 2024-04-20T11:32:54.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.848+0000 7fabbf600640 1 --2- 172.21.15.152:0/2206562322 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fabb80cd8c0 0x7fabb80b25f0 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.848+0000 7fabbf600640 1 --2- 172.21.15.152:0/2206562322 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fabb8146000 0x7fabb80b4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.848+0000 7fabbf600640 1 -- 172.21.15.152:0/2206562322 >> 172.21.15.152:0/2206562322 conn(0x7fabb806b980 msgr2=0x7fabb806bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:54.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.848+0000 7fabbf600640 1 -- 172.21.15.152:0/2206562322 shutdown_connections 2024-04-20T11:32:54.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.848+0000 7fabbf600640 1 -- 172.21.15.152:0/2206562322 wait complete. 2024-04-20T11:32:54.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.850+0000 7fabbf600640 1 Processor -- start 2024-04-20T11:32:54.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.850+0000 7fabbf600640 1 -- start start 2024-04-20T11:32:54.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.851+0000 7fabbf600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fabb80b2170 0x7fabb815e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:54.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.851+0000 7fabbf600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fabb80cd8c0 0x7fabb806f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:54.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.851+0000 7fabbf600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fabb8146000 0x7fabb806f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:54.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.851+0000 7fabbf600640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fabb815ce70 con 0x7fabb80cd8c0 2024-04-20T11:32:54.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.851+0000 7fabbf600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fabb815ccf0 con 0x7fabb8146000 2024-04-20T11:32:54.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.851+0000 7fabbf600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fabb815cff0 con 0x7fabb80b2170 2024-04-20T11:32:54.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.851+0000 7fabb7e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fabb80cd8c0 0x7fabb806f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:54.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.851+0000 7fabb7e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fabb80cd8c0 0x7fabb806f2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:45146/0 (socket says 172.21.15.152:45146) 2024-04-20T11:32:54.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.851+0000 7fabb7e00640 1 -- 172.21.15.152:0/2307137414 learned_addr learned my addr 172.21.15.152:0/2307137414 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:32:54.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.851+0000 7fabbcc00640 1 --2- 172.21.15.152:0/2307137414 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fabb80b2170 0x7fabb815e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:54.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.851+0000 7fabbd600640 1 --2- 172.21.15.152:0/2307137414 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fabb8146000 0x7fabb806f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:54.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.852+0000 7fabb7e00640 1 -- 172.21.15.152:0/2307137414 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fabb80b2170 msgr2=0x7fabb815e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:54.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.852+0000 7fabb7e00640 1 --2- 172.21.15.152:0/2307137414 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fabb80b2170 0x7fabb815e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.852+0000 7fabb7e00640 1 -- 172.21.15.152:0/2307137414 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fabb8146000 msgr2=0x7fabb806f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:54.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.852+0000 7fabb7e00640 1 --2- 172.21.15.152:0/2307137414 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fabb8146000 0x7fabb806f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.852+0000 7fabb7e00640 1 -- 172.21.15.152:0/2307137414 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabb806ed90 con 0x7fabb80cd8c0 2024-04-20T11:32:54.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.852+0000 7fabbcc00640 1 --2- 172.21.15.152:0/2307137414 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fabb80b2170 0x7fabb815e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:32:54.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.852+0000 7fabb7e00640 1 --2- 172.21.15.152:0/2307137414 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fabb80cd8c0 0x7fabb806f2e0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7faba400c480 tx=0x7faba400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:54.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.852+0000 7fabb4c00640 1 -- 172.21.15.152:0/2307137414 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba40052d0 con 0x7fabb80cd8c0 2024-04-20T11:32:54.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.852+0000 7fabb4c00640 1 -- 172.21.15.152:0/2307137414 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faba4005470 con 0x7fabb80cd8c0 2024-04-20T11:32:54.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.852+0000 7fabbf600640 1 -- 172.21.15.152:0/2307137414 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabb80af450 con 0x7fabb80cd8c0 2024-04-20T11:32:54.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.852+0000 7fabb4c00640 1 -- 172.21.15.152:0/2307137414 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba4012870 con 0x7fabb80cd8c0 2024-04-20T11:32:54.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.852+0000 7fabbf600640 1 -- 172.21.15.152:0/2307137414 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabb806e2b0 con 0x7fabb80cd8c0 2024-04-20T11:32:54.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.853+0000 7fabbf600640 1 -- 172.21.15.152:0/2307137414 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabb806dd50 con 0x7fabb80cd8c0 2024-04-20T11:32:54.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.855+0000 7fabb4c00640 1 -- 172.21.15.152:0/2307137414 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 91142+0+0 (secure 0 0 0) 0x7faba4012a10 con 0x7fabb80cd8c0 2024-04-20T11:32:54.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.855+0000 7fabb4c00640 1 --2- 172.21.15.152:0/2307137414 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fab9806ca40 0x7fab9806ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:54.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.856+0000 7fabb4c00640 1 -- 172.21.15.152:0/2307137414 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7faba408b9f0 con 0x7fabb80cd8c0 2024-04-20T11:32:54.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.856+0000 7fabbcc00640 1 --2- 172.21.15.152:0/2307137414 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fab9806ca40 0x7fab9806ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:54.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.857+0000 7fabbcc00640 1 --2- 172.21.15.152:0/2307137414 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fab9806ca40 0x7fab9806ef00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fabac006fd0 tx=0x7fabac008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:54.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:54.860+0000 7fabb4c00640 1 -- 172.21.15.152:0/2307137414 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fabb806dd50 con 0x7fabb80cd8c0 2024-04-20T11:32:55.054 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.054+0000 7fabbf600640 1 -- 172.21.15.152:0/2307137414 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fabb80689f0 con 0x7fabb80cd8c0 2024-04-20T11:32:55.054 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.055+0000 7fabb4c00640 1 -- 172.21.15.152:0/2307137414 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) ==== 122+0+0 (secure 0 0 0) 0x7faba4059d00 con 0x7fabb80cd8c0 2024-04-20T11:32:55.057 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.057+0000 7fabbf600640 1 -- 172.21.15.152:0/2307137414 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fab9806ca40 msgr2=0x7fab9806ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:55.057 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.057+0000 7fabbf600640 1 --2- 172.21.15.152:0/2307137414 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fab9806ca40 0x7fab9806ef00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fabac006fd0 tx=0x7fabac008040 comp rx=0 tx=0).stop 2024-04-20T11:32:55.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.057+0000 7fabbf600640 1 -- 172.21.15.152:0/2307137414 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fabb80cd8c0 msgr2=0x7fabb806f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:55.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.057+0000 7fabbf600640 1 --2- 172.21.15.152:0/2307137414 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fabb80cd8c0 0x7fabb806f2e0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7faba400c480 tx=0x7faba400c940 comp rx=0 tx=0).stop 2024-04-20T11:32:55.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.058+0000 7fabbf600640 1 -- 172.21.15.152:0/2307137414 shutdown_connections 2024-04-20T11:32:55.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.058+0000 7fabbf600640 1 --2- 172.21.15.152:0/2307137414 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fab9806ca40 0x7fab9806ef00 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:55.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.058+0000 7fabbf600640 1 --2- 172.21.15.152:0/2307137414 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fabb8146000 0x7fabb806f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:55.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.058+0000 7fabbf600640 1 --2- 172.21.15.152:0/2307137414 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fabb80cd8c0 0x7fabb806f2e0 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:55.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.058+0000 7fabbf600640 1 --2- 172.21.15.152:0/2307137414 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fabb80b2170 0x7fabb815e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:55.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.058+0000 7fabbf600640 1 -- 172.21.15.152:0/2307137414 >> 172.21.15.152:0/2307137414 conn(0x7fabb806b980 msgr2=0x7fabb80b4be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:55.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.058+0000 7fabbf600640 1 -- 172.21.15.152:0/2307137414 shutdown_connections 2024-04-20T11:32:55.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.058+0000 7fabbf600640 1 -- 172.21.15.152:0/2307137414 wait complete. 2024-04-20T11:32:55.076 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume create 'abcd@#' 2024-04-20T11:32:55.271 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.271+0000 7f664d400640 1 Processor -- start 2024-04-20T11:32:55.271 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.272+0000 7f664d400640 1 -- start start 2024-04-20T11:32:55.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.273+0000 7f664d400640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f6648147cd0 0x7f664806e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:55.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.273+0000 7f664d400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f66481480b0 0x7f664815ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:55.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.273+0000 7f664d400640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f664815f7e0 con 0x7f664815f1f0 2024-04-20T11:32:55.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.273+0000 7f664d400640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f664815fac0 con 0x7f66481480b0 2024-04-20T11:32:55.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.273+0000 7f664d400640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f664806df00 con 0x7f6648147cd0 2024-04-20T11:32:55.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.274+0000 7f664ca00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f664815f1f0 0x7f664806f1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:41202/0 (socket says 172.21.15.152:41202) 2024-04-20T11:32:55.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.274+0000 7f663f400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f66481480b0 0x7f664815ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:55.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.274+0000 7f664ca00640 1 -- 172.21.15.152:0/2798403760 learned_addr learned my addr 172.21.15.152:0/2798403760 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:32:55.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.274+0000 7f6647e00640 1 --2- 172.21.15.152:0/2798403760 >> v2:172.21.15.175:3300/0 conn(0x7f6648147cd0 0x7f664806e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:55.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.274+0000 7f663f400640 1 -- 172.21.15.152:0/2798403760 >> v2:172.21.15.175:3300/0 conn(0x7f6648147cd0 msgr2=0x7f664806e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:55.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.274+0000 7f663f400640 1 --2- 172.21.15.152:0/2798403760 >> v2:172.21.15.175:3300/0 conn(0x7f6648147cd0 0x7f664806e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:55.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.274+0000 7f663f400640 1 -- 172.21.15.152:0/2798403760 >> v1:172.21.15.152:6789/0 conn(0x7f664815f1f0 legacy=0x7f664806f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:55.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.274+0000 7f663f400640 1 -- 172.21.15.152:0/2798403760 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f664806e1e0 con 0x7f66481480b0 2024-04-20T11:32:55.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.274+0000 7f6647400640 1 -- 172.21.15.152:0/2798403760 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 715451760 0 0) 0x7f664815f7e0 con 0x7f664815f1f0 2024-04-20T11:32:55.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.275+0000 7f663f400640 1 --2- 172.21.15.152:0/2798403760 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f66481480b0 0x7f664815ecb0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f664806ffc0 tx=0x7f663002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1facc4b49099dd77 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:55.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.276+0000 7f6647400640 1 -- 172.21.15.152:0/2798403760 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6630035020 con 0x7f66481480b0 2024-04-20T11:32:55.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.276+0000 7f6647400640 1 -- 172.21.15.152:0/2798403760 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6630031830 con 0x7f66481480b0 2024-04-20T11:32:55.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.276+0000 7f6647400640 1 -- 172.21.15.152:0/2798403760 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6630031c00 con 0x7f66481480b0 2024-04-20T11:32:55.276 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.276+0000 7f664d400640 1 -- 172.21.15.152:0/2798403760 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f66481480b0 msgr2=0x7f664815ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:55.276 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.276+0000 7f664d400640 1 --2- 172.21.15.152:0/2798403760 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f66481480b0 0x7f664815ecb0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f664806ffc0 tx=0x7f663002b8d0 comp rx=0 tx=0).stop 2024-04-20T11:32:55.276 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.276+0000 7f664d400640 1 -- 172.21.15.152:0/2798403760 shutdown_connections 2024-04-20T11:32:55.276 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.276+0000 7f664d400640 1 --2- 172.21.15.152:0/2798403760 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f66481480b0 0x7f664815ecb0 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:55.277 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.276+0000 7f664d400640 1 --2- 172.21.15.152:0/2798403760 >> v2:172.21.15.175:3300/0 conn(0x7f6648147cd0 0x7f664806e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:55.277 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.276+0000 7f664d400640 1 -- 172.21.15.152:0/2798403760 >> 172.21.15.152:0/2798403760 conn(0x7f6648067ae0 msgr2=0x7f664806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:55.277 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.276+0000 7f664d400640 1 -- 172.21.15.152:0/2798403760 shutdown_connections 2024-04-20T11:32:55.277 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.276+0000 7f664d400640 1 -- 172.21.15.152:0/2798403760 wait complete. 2024-04-20T11:32:55.277 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.278+0000 7f664d400640 1 Processor -- start 2024-04-20T11:32:55.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.278+0000 7f664d400640 1 -- start start 2024-04-20T11:32:55.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.279+0000 7f664d400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6648147cd0 0x7f664815c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:55.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.279+0000 7f664d400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f66481480b0 0x7f66481e9690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:55.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.279+0000 7f664d400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f664815f1f0 0x7f66481e9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:55.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.279+0000 7f664d400640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f664806edc0 con 0x7f664815f1f0 2024-04-20T11:32:55.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.279+0000 7f664d400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f664806ec40 con 0x7f6648147cd0 2024-04-20T11:32:55.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.279+0000 7f664d400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f664806f8d0 con 0x7f66481480b0 2024-04-20T11:32:55.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.280+0000 7f664ca00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f664815f1f0 0x7f66481e9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:55.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.280+0000 7f664ca00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f664815f1f0 0x7f66481e9bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:45166/0 (socket says 172.21.15.152:45166) 2024-04-20T11:32:55.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.280+0000 7f664ca00640 1 -- 172.21.15.152:0/944412780 learned_addr learned my addr 172.21.15.152:0/944412780 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:32:55.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.280+0000 7f664ca00640 1 -- 172.21.15.152:0/944412780 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f66481480b0 msgr2=0x7f66481e9690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:55.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.280+0000 7f664ca00640 1 --2- 172.21.15.152:0/944412780 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f66481480b0 0x7f66481e9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:55.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.280+0000 7f664ca00640 1 -- 172.21.15.152:0/944412780 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6648147cd0 msgr2=0x7f664815c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:55.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.280+0000 7f664ca00640 1 --2- 172.21.15.152:0/944412780 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6648147cd0 0x7f664815c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:55.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.280+0000 7f664ca00640 1 -- 172.21.15.152:0/944412780 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66481e62d0 con 0x7f664815f1f0 2024-04-20T11:32:55.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.280+0000 7f664ca00640 1 --2- 172.21.15.152:0/944412780 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f664815f1f0 0x7f66481e9bd0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f664000c730 tx=0x7f664000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:55.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.281+0000 7f6644c00640 1 -- 172.21.15.152:0/944412780 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6640018070 con 0x7f664815f1f0 2024-04-20T11:32:55.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.281+0000 7f6644c00640 1 -- 172.21.15.152:0/944412780 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f664000ce40 con 0x7f664815f1f0 2024-04-20T11:32:55.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.281+0000 7f6644c00640 1 -- 172.21.15.152:0/944412780 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f664000a860 con 0x7f664815f1f0 2024-04-20T11:32:55.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.281+0000 7f664d400640 1 -- 172.21.15.152:0/944412780 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66481e7010 con 0x7f664815f1f0 2024-04-20T11:32:55.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.281+0000 7f664d400640 1 -- 172.21.15.152:0/944412780 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f664815c540 con 0x7f664815f1f0 2024-04-20T11:32:55.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.282+0000 7f664d400640 1 -- 172.21.15.152:0/944412780 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6648075e60 con 0x7f664815f1f0 2024-04-20T11:32:55.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.283+0000 7f6644c00640 1 -- 172.21.15.152:0/944412780 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 91142+0+0 (secure 0 0 0) 0x7f664000aa00 con 0x7f664815f1f0 2024-04-20T11:32:55.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.284+0000 7f6644c00640 1 --2- 172.21.15.152:0/944412780 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f662406ca10 0x7f662406eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:55.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.285+0000 7f6644c00640 1 -- 172.21.15.152:0/944412780 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7f664008b300 con 0x7f664815f1f0 2024-04-20T11:32:55.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.285+0000 7f6647e00640 1 --2- 172.21.15.152:0/944412780 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f662406ca10 0x7f662406eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:55.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.286+0000 7f6647e00640 1 --2- 172.21.15.152:0/944412780 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f662406ca10 0x7f662406eed0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6638006fd0 tx=0x7f6638008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:55.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.289+0000 7f6644c00640 1 -- 172.21.15.152:0/944412780 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f6640059610 con 0x7f664815f1f0 2024-04-20T11:32:55.520 INFO:teuthology.orchestra.run.smithi152.stderr:Invalid command: invalid chars @# in abcd@# 2024-04-20T11:32:55.520 INFO:teuthology.orchestra.run.smithi152.stderr:fs volume create [] : Create a CephFS volume 2024-04-20T11:32:55.520 INFO:teuthology.orchestra.run.smithi152.stderr:Error EINVAL: invalid command 2024-04-20T11:32:55.529 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-20T11:32:55.530 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume create testvol 2024-04-20T11:32:55.732 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.733+0000 7fa6ce000640 1 Processor -- start 2024-04-20T11:32:55.733 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.733+0000 7fa6ce000640 1 -- start start 2024-04-20T11:32:55.735 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.735+0000 7fa6ce000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa6c8071cd0 0x7fa6c80720b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:55.735 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.735+0000 7fa6ce000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fa6c8141450 0x7fa6c81418e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:55.735 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.735+0000 7fa6ce000640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6c806fb40 con 0x7fa6c8160220 2024-04-20T11:32:55.735 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.735+0000 7fa6ce000640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fa6c813f8e0 con 0x7fa6c8141450 2024-04-20T11:32:55.735 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.735+0000 7fa6ce000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa6c80725f0 con 0x7fa6c8071cd0 2024-04-20T11:32:55.735 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.735+0000 7fa6cd600640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fa6c8141450 0x7fa6c81418e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:55.735 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.735+0000 7fa6cd600640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fa6c8141450 0x7fa6c81418e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:45174/0 (socket says 172.21.15.152:45174) 2024-04-20T11:32:55.735 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.735+0000 7fa6cd600640 1 -- 172.21.15.152:0/1543072357 learned_addr learned my addr 172.21.15.152:0/1543072357 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:32:55.735 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.735+0000 7fa6ccc00640 1 --2- 172.21.15.152:0/1543072357 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa6c8071cd0 0x7fa6c80720b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:55.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.736+0000 7fa6cd600640 1 -- 172.21.15.152:0/1543072357 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa6c8071cd0 msgr2=0x7fa6c80720b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:55.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.736+0000 7fa6cd600640 1 --2- 172.21.15.152:0/1543072357 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa6c8071cd0 0x7fa6c80720b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:55.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.736+0000 7fa6cd600640 1 -- 172.21.15.152:0/1543072357 >> v1:172.21.15.175:6789/0 conn(0x7fa6c8160220 legacy=0x7fa6c8160680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:55.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.736+0000 7fa6cd600640 1 -- 172.21.15.152:0/1543072357 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6c8160f90 con 0x7fa6c8141450 2024-04-20T11:32:55.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.737+0000 7fa6cd600640 1 --2- 172.21.15.152:0/1543072357 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa6c8141450 0x7fa6c81418e0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fa6c0009e50 tx=0x7fa6c002be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e99d7fcd9ff0dbab server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:55.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.737+0000 7fa6c7400640 1 -- 172.21.15.152:0/1543072357 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c002c9f0 con 0x7fa6c8141450 2024-04-20T11:32:55.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.737+0000 7fa6c7400640 1 -- 172.21.15.152:0/1543072357 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6c002cb90 con 0x7fa6c8141450 2024-04-20T11:32:55.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.737+0000 7fa6c7400640 1 -- 172.21.15.152:0/1543072357 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c00343f0 con 0x7fa6c8141450 2024-04-20T11:32:55.738 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.738+0000 7fa6ce000640 1 -- 172.21.15.152:0/1543072357 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa6c8141450 msgr2=0x7fa6c81418e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:55.738 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.738+0000 7fa6ce000640 1 --2- 172.21.15.152:0/1543072357 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa6c8141450 0x7fa6c81418e0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fa6c0009e50 tx=0x7fa6c002be50 comp rx=0 tx=0).stop 2024-04-20T11:32:55.738 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.738+0000 7fa6ce000640 1 -- 172.21.15.152:0/1543072357 shutdown_connections 2024-04-20T11:32:55.738 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.738+0000 7fa6ce000640 1 --2- 172.21.15.152:0/1543072357 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa6c8141450 0x7fa6c81418e0 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:55.738 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.738+0000 7fa6ce000640 1 --2- 172.21.15.152:0/1543072357 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa6c8071cd0 0x7fa6c80720b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:55.738 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.738+0000 7fa6ce000640 1 -- 172.21.15.152:0/1543072357 >> 172.21.15.152:0/1543072357 conn(0x7fa6c8067ae0 msgr2=0x7fa6c806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:55.738 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.738+0000 7fa6ce000640 1 -- 172.21.15.152:0/1543072357 shutdown_connections 2024-04-20T11:32:55.738 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.738+0000 7fa6ce000640 1 -- 172.21.15.152:0/1543072357 wait complete. 2024-04-20T11:32:55.739 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.740+0000 7fa6ce000640 1 Processor -- start 2024-04-20T11:32:55.739 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.740+0000 7fa6ce000640 1 -- start start 2024-04-20T11:32:55.740 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.741+0000 7fa6ce000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa6c8071cd0 0x7fa6c8155bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:55.740 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.741+0000 7fa6ce000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6c8141450 0x7fa6c80b1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:55.740 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.741+0000 7fa6ce000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa6c8160220 0x7fa6c80b16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:55.740 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.741+0000 7fa6ce000640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c806e350 con 0x7fa6c8160220 2024-04-20T11:32:55.740 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.741+0000 7fa6ce000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c806e070 con 0x7fa6c8141450 2024-04-20T11:32:55.741 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.741+0000 7fa6ce000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa6c806e630 con 0x7fa6c8071cd0 2024-04-20T11:32:55.742 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.741+0000 7fa6cd600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa6c8160220 0x7fa6c80b16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:55.742 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.741+0000 7fa6cd600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa6c8160220 0x7fa6c80b16c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:45186/0 (socket says 172.21.15.152:45186) 2024-04-20T11:32:55.742 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.741+0000 7fa6cd600640 1 -- 172.21.15.152:0/364161253 learned_addr learned my addr 172.21.15.152:0/364161253 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:32:55.742 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.741+0000 7fa6ccc00640 1 --2- 172.21.15.152:0/364161253 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa6c8071cd0 0x7fa6c8155bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:55.742 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.742+0000 7fa6c7e00640 1 --2- 172.21.15.152:0/364161253 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6c8141450 0x7fa6c80b1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:55.743 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.742+0000 7fa6cd600640 1 -- 172.21.15.152:0/364161253 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa6c8071cd0 msgr2=0x7fa6c8155bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:55.743 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.742+0000 7fa6cd600640 1 --2- 172.21.15.152:0/364161253 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa6c8071cd0 0x7fa6c8155bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:55.743 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.742+0000 7fa6cd600640 1 -- 172.21.15.152:0/364161253 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6c8141450 msgr2=0x7fa6c80b1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:55.743 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.742+0000 7fa6cd600640 1 --2- 172.21.15.152:0/364161253 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6c8141450 0x7fa6c80b1180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:55.743 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.742+0000 7fa6cd600640 1 -- 172.21.15.152:0/364161253 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6c80aa450 con 0x7fa6c8160220 2024-04-20T11:32:55.743 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.742+0000 7fa6cd600640 1 --2- 172.21.15.152:0/364161253 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa6c8160220 0x7fa6c80b16c0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7fa6c0036040 tx=0x7fa6c0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:55.743 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.742+0000 7fa6c4c00640 1 -- 172.21.15.152:0/364161253 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c0043020 con 0x7fa6c8160220 2024-04-20T11:32:55.743 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.742+0000 7fa6c4c00640 1 -- 172.21.15.152:0/364161253 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6c003e450 con 0x7fa6c8160220 2024-04-20T11:32:55.743 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.743+0000 7fa6ce000640 1 -- 172.21.15.152:0/364161253 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6c80a96b0 con 0x7fa6c8160220 2024-04-20T11:32:55.743 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.743+0000 7fa6c4c00640 1 -- 172.21.15.152:0/364161253 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c0009510 con 0x7fa6c8160220 2024-04-20T11:32:55.743 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.743+0000 7fa6ce000640 1 -- 172.21.15.152:0/364161253 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6c80a9c50 con 0x7fa6c8160220 2024-04-20T11:32:55.743 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.744+0000 7fa6ce000640 1 -- 172.21.15.152:0/364161253 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6c8075e60 con 0x7fa6c8160220 2024-04-20T11:32:55.744 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.745+0000 7fa6c4c00640 1 -- 172.21.15.152:0/364161253 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 91142+0+0 (secure 0 0 0) 0x7fa6c0046020 con 0x7fa6c8160220 2024-04-20T11:32:55.745 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.745+0000 7fa6c4c00640 1 --2- 172.21.15.152:0/364161253 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fa6a806ca10 0x7fa6a806eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:55.745 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.746+0000 7fa6c4c00640 1 -- 172.21.15.152:0/364161253 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7fa6c00ae610 con 0x7fa6c8160220 2024-04-20T11:32:55.745 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.746+0000 7fa6ccc00640 1 --2- 172.21.15.152:0/364161253 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fa6a806ca10 0x7fa6a806eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:55.747 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.747+0000 7fa6ccc00640 1 --2- 172.21.15.152:0/364161253 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fa6a806ca10 0x7fa6a806eed0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc006fd0 tx=0x7fa6bc008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:55.749 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.750+0000 7fa6c4c00640 1 -- 172.21.15.152:0/364161253 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fa6c004d030 con 0x7fa6c8160220 2024-04-20T11:32:55.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:55.958+0000 7fa6ce000640 1 -- 172.21.15.152:0/364161253 --> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] -- mgr_command(tid 0: {"prefix": "fs volume create", "name": "testvol", "target": ["mon-mgr", ""]}) -- 0x7fa6c8072d30 con 0x7fa6a806ca10 2024-04-20T11:32:58.201 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.201+0000 7fa6c4c00640 1 -- 172.21.15.152:0/364161253 <== mgr.4109 v2:172.21.15.175:6836/4021250441 1 ==== mgr_command_reply(tid 0: 0 Volume created successfully (no MDS daemons created)) ==== 60+0+0 (secure 0 0 0) 0x7fa6c8072d30 con 0x7fa6a806ca10 2024-04-20T11:32:58.201 INFO:teuthology.orchestra.run.smithi152.stderr:Volume created successfully (no MDS daemons created) 2024-04-20T11:32:58.205 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.205+0000 7fa6ce000640 1 -- 172.21.15.152:0/364161253 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fa6a806ca10 msgr2=0x7fa6a806eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:58.205 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.205+0000 7fa6ce000640 1 --2- 172.21.15.152:0/364161253 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fa6a806ca10 0x7fa6a806eed0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc006fd0 tx=0x7fa6bc008040 comp rx=0 tx=0).stop 2024-04-20T11:32:58.205 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.205+0000 7fa6ce000640 1 -- 172.21.15.152:0/364161253 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa6c8160220 msgr2=0x7fa6c80b16c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:58.205 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.205+0000 7fa6ce000640 1 --2- 172.21.15.152:0/364161253 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa6c8160220 0x7fa6c80b16c0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7fa6c0036040 tx=0x7fa6c0004a40 comp rx=0 tx=0).stop 2024-04-20T11:32:58.205 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.205+0000 7fa6ce000640 1 -- 172.21.15.152:0/364161253 shutdown_connections 2024-04-20T11:32:58.205 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.205+0000 7fa6ce000640 1 --2- 172.21.15.152:0/364161253 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fa6a806ca10 0x7fa6a806eed0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:58.205 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.205+0000 7fa6ce000640 1 --2- 172.21.15.152:0/364161253 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa6c8160220 0x7fa6c80b16c0 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:58.205 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.205+0000 7fa6ce000640 1 --2- 172.21.15.152:0/364161253 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6c8141450 0x7fa6c80b1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:58.205 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.205+0000 7fa6ce000640 1 --2- 172.21.15.152:0/364161253 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa6c8071cd0 0x7fa6c8155bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:58.205 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.205+0000 7fa6ce000640 1 -- 172.21.15.152:0/364161253 >> 172.21.15.152:0/364161253 conn(0x7fa6c8067ae0 msgr2=0x7fa6c813edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:58.205 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.205+0000 7fa6ce000640 1 -- 172.21.15.152:0/364161253 shutdown_connections 2024-04-20T11:32:58.205 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.205+0000 7fa6ce000640 1 -- 172.21.15.152:0/364161253 wait complete. 2024-04-20T11:32:58.239 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create testvol 'abcd@#' 2024-04-20T11:32:58.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.425+0000 7f4f1dc00640 1 Processor -- start 2024-04-20T11:32:58.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.425+0000 7f4f1dc00640 1 -- start start 2024-04-20T11:32:58.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.427+0000 7f4f1dc00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f4f18146000 0x7f4f180b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:58.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.427+0000 7f4f1dc00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f181463e0 con 0x7f4f180cd8c0 2024-04-20T11:32:58.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.427+0000 7f4f1dc00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f180b3e20 con 0x7f4f180b2170 2024-04-20T11:32:58.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.427+0000 7f4f1dc00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f4f180cdca0 con 0x7f4f18146000 2024-04-20T11:32:58.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.427+0000 7f4f17400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f4f180cd8c0 0x7f4f1815ccf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:33552/0 (socket says 172.21.15.152:33552) 2024-04-20T11:32:58.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.427+0000 7f4f16a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f4f18146000 0x7f4f180b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:58.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.427+0000 7f4f17400640 1 -- 172.21.15.152:0/3537668691 learned_addr learned my addr 172.21.15.152:0/3537668691 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:32:58.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.428+0000 7f4f16a00640 1 -- 172.21.15.152:0/3537668691 >> v1:172.21.15.175:6789/0 conn(0x7f4f180b2170 legacy=0x7f4f180cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:58.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.428+0000 7f4f16a00640 1 -- 172.21.15.152:0/3537668691 >> v1:172.21.15.152:6789/0 conn(0x7f4f180cd8c0 legacy=0x7f4f1815ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:58.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.428+0000 7f4f16a00640 1 -- 172.21.15.152:0/3537668691 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f180b4100 con 0x7f4f18146000 2024-04-20T11:32:58.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.429+0000 7f4f16a00640 1 --2- 172.21.15.152:0/3537668691 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4f18146000 0x7f4f180b4be0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f4f080097a0 tx=0x7f4f0802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41b64fc6c986ece1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:58.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.429+0000 7f4f16000640 1 -- 172.21.15.152:0/3537668691 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f080315a0 con 0x7f4f18146000 2024-04-20T11:32:58.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.429+0000 7f4f16000640 1 -- 172.21.15.152:0/3537668691 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f08037070 con 0x7f4f18146000 2024-04-20T11:32:58.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.429+0000 7f4f16000640 1 -- 172.21.15.152:0/3537668691 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f08031960 con 0x7f4f18146000 2024-04-20T11:32:58.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.430+0000 7f4f1dc00640 1 -- 172.21.15.152:0/3537668691 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4f18146000 msgr2=0x7f4f180b4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:58.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.430+0000 7f4f1dc00640 1 --2- 172.21.15.152:0/3537668691 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4f18146000 0x7f4f180b4be0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f4f080097a0 tx=0x7f4f0802b840 comp rx=0 tx=0).stop 2024-04-20T11:32:58.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.430+0000 7f4f1dc00640 1 -- 172.21.15.152:0/3537668691 shutdown_connections 2024-04-20T11:32:58.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.430+0000 7f4f1dc00640 1 --2- 172.21.15.152:0/3537668691 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4f18146000 0x7f4f180b4be0 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:58.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.430+0000 7f4f1dc00640 1 -- 172.21.15.152:0/3537668691 >> 172.21.15.152:0/3537668691 conn(0x7f4f1806b980 msgr2=0x7f4f1806bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:58.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.430+0000 7f4f1dc00640 1 -- 172.21.15.152:0/3537668691 shutdown_connections 2024-04-20T11:32:58.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.430+0000 7f4f1dc00640 1 -- 172.21.15.152:0/3537668691 wait complete. 2024-04-20T11:32:58.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.432+0000 7f4f1dc00640 1 Processor -- start 2024-04-20T11:32:58.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.432+0000 7f4f1dc00640 1 -- start start 2024-04-20T11:32:58.433 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.433+0000 7f4f1dc00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4f180b2170 0x7f4f18140180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:58.433 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.433+0000 7f4f1dc00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4f180cd8c0 0x7f4f181f4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:58.433 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.433+0000 7f4f1dc00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4f18146000 0x7f4f1812d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:58.433 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.433+0000 7f4f1dc00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f4f180b2620 con 0x7f4f180cd8c0 2024-04-20T11:32:58.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.433+0000 7f4f1dc00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4f180b4750 con 0x7f4f180b2170 2024-04-20T11:32:58.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.433+0000 7f4f1dc00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4f180b27a0 con 0x7f4f18146000 2024-04-20T11:32:58.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.434+0000 7f4f0e000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4f180cd8c0 0x7f4f181f4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:58.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.434+0000 7f4f0e000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4f180cd8c0 0x7f4f181f4ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:32956/0 (socket says 172.21.15.152:32956) 2024-04-20T11:32:58.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.434+0000 7f4f0e000640 1 -- 172.21.15.152:0/260826955 learned_addr learned my addr 172.21.15.152:0/260826955 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:32:58.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.434+0000 7f4f16a00640 1 --2- 172.21.15.152:0/260826955 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4f180b2170 0x7f4f18140180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:58.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.434+0000 7f4f17400640 1 --2- 172.21.15.152:0/260826955 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4f18146000 0x7f4f1812d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:58.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.434+0000 7f4f0e000640 1 -- 172.21.15.152:0/260826955 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4f18146000 msgr2=0x7f4f1812d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:58.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.434+0000 7f4f0e000640 1 --2- 172.21.15.152:0/260826955 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4f18146000 0x7f4f1812d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:58.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.434+0000 7f4f0e000640 1 -- 172.21.15.152:0/260826955 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4f180b2170 msgr2=0x7f4f18140180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:58.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.434+0000 7f4f0e000640 1 --2- 172.21.15.152:0/260826955 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4f180b2170 0x7f4f18140180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:58.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.434+0000 7f4f0e000640 1 -- 172.21.15.152:0/260826955 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f181f5b30 con 0x7f4f180cd8c0 2024-04-20T11:32:58.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.434+0000 7f4f0e000640 1 --2- 172.21.15.152:0/260826955 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4f180cd8c0 0x7f4f181f4ba0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f4f0000c430 tx=0x7f4f0000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:58.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.435+0000 7f4f0f400640 1 -- 172.21.15.152:0/260826955 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f00011790 con 0x7f4f180cd8c0 2024-04-20T11:32:58.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.435+0000 7f4f0f400640 1 -- 172.21.15.152:0/260826955 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f00017070 con 0x7f4f180cd8c0 2024-04-20T11:32:58.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.435+0000 7f4f0f400640 1 -- 172.21.15.152:0/260826955 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f00011b70 con 0x7f4f180cd8c0 2024-04-20T11:32:58.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.435+0000 7f4f1dc00640 1 -- 172.21.15.152:0/260826955 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f181f5570 con 0x7f4f180cd8c0 2024-04-20T11:32:58.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.435+0000 7f4f1dc00640 1 -- 172.21.15.152:0/260826955 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f1806dfd0 con 0x7f4f180cd8c0 2024-04-20T11:32:58.436 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.436+0000 7f4f1dc00640 1 -- 172.21.15.152:0/260826955 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f1806e1d0 con 0x7f4f180cd8c0 2024-04-20T11:32:58.436 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.437+0000 7f4f0f400640 1 -- 172.21.15.152:0/260826955 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 91142+0+0 (secure 0 0 0) 0x7f4f00011d10 con 0x7f4f180cd8c0 2024-04-20T11:32:58.437 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.438+0000 7f4f0f400640 1 --2- 172.21.15.152:0/260826955 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4ef406ca10 0x7f4ef406eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:58.437 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.438+0000 7f4f0f400640 1 -- 172.21.15.152:0/260826955 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6543+0+0 (secure 0 0 0) 0x7f4f0008bef0 con 0x7f4f180cd8c0 2024-04-20T11:32:58.438 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.439+0000 7f4f16a00640 1 --2- 172.21.15.152:0/260826955 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4ef406ca10 0x7f4ef406eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:58.439 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.440+0000 7f4f16a00640 1 --2- 172.21.15.152:0/260826955 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4ef406ca10 0x7f4ef406eed0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4f08036000 tx=0x7f4f0803d040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:58.442 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.443+0000 7f4f0f400640 1 -- 172.21.15.152:0/260826955 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f4f1806e1d0 con 0x7f4f180cd8c0 2024-04-20T11:32:58.668 INFO:teuthology.orchestra.run.smithi152.stderr:Invalid command: invalid chars @# in abcd@# 2024-04-20T11:32:58.668 INFO:teuthology.orchestra.run.smithi152.stderr:fs subvolumegroup create [] [] [] [] [] : Create a CephFS subvolume group in a volume, and optionally, with a specific data pool layout, and a specific numeric mode 2024-04-20T11:32:58.668 INFO:teuthology.orchestra.run.smithi152.stderr:Error EINVAL: invalid command 2024-04-20T11:32:58.677 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-20T11:32:58.678 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create testvol 'abcd@#' 2024-04-20T11:32:58.873 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.873+0000 7fd880600640 1 Processor -- start 2024-04-20T11:32:58.873 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.874+0000 7fd880600640 1 -- start start 2024-04-20T11:32:58.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.875+0000 7fd880600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd87815e890 0x7fd8780d27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:58.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.875+0000 7fd880600640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fd8780d2d20 0x7fd8780b7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:58.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.875+0000 7fd880600640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd87815ec70 con 0x7fd8780b7cc0 2024-04-20T11:32:58.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.875+0000 7fd880600640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fd8780b8e20 con 0x7fd8780d2d20 2024-04-20T11:32:58.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.875+0000 7fd880600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd8780b8210 con 0x7fd87815e890 2024-04-20T11:32:58.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.875+0000 7fd87d200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fd8780d2d20 0x7fd8780b7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:58.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.875+0000 7fd87e600640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fd8780b7cc0 0x7fd8780d1790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:33554/0 (socket says 172.21.15.152:33554) 2024-04-20T11:32:58.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.875+0000 7fd87e600640 1 -- 172.21.15.152:0/620461863 learned_addr learned my addr 172.21.15.152:0/620461863 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:32:58.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.875+0000 7fd87dc00640 1 --2- 172.21.15.152:0/620461863 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd87815e890 0x7fd8780d27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:58.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.876+0000 7fd87d200640 1 -- 172.21.15.152:0/620461863 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd87815e890 msgr2=0x7fd8780d27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:58.877 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.876+0000 7fd87d200640 1 --2- 172.21.15.152:0/620461863 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd87815e890 0x7fd8780d27e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:58.877 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.876+0000 7fd87d200640 1 -- 172.21.15.152:0/620461863 >> v1:172.21.15.152:6789/0 conn(0x7fd8780b7cc0 legacy=0x7fd8780d1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:58.877 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.876+0000 7fd87d200640 1 -- 172.21.15.152:0/620461863 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8780b84f0 con 0x7fd8780d2d20 2024-04-20T11:32:58.877 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.876+0000 7fd877e00640 1 -- 172.21.15.152:0/620461863 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2578439751 0 0) 0x7fd87815ec70 con 0x7fd8780b7cc0 2024-04-20T11:32:58.877 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.876+0000 7fd87dc00640 1 --2- 172.21.15.152:0/620461863 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd87815e890 0x7fd8780d27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:32:58.878 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.877+0000 7fd87d200640 1 --2- 172.21.15.152:0/620461863 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd8780d2d20 0x7fd8780b7780 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7fd8640097a0 tx=0x7fd86402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc90d5bfbf422a2d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:58.878 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.877+0000 7fd877e00640 1 -- 172.21.15.152:0/620461863 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd864034070 con 0x7fd8780d2d20 2024-04-20T11:32:58.878 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.877+0000 7fd877e00640 1 -- 172.21.15.152:0/620461863 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd864030610 con 0x7fd8780d2d20 2024-04-20T11:32:58.879 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.877+0000 7fd877e00640 1 -- 172.21.15.152:0/620461863 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8640309e0 con 0x7fd8780d2d20 2024-04-20T11:32:58.879 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.878+0000 7fd880600640 1 -- 172.21.15.152:0/620461863 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd8780d2d20 msgr2=0x7fd8780b7780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:58.879 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.878+0000 7fd880600640 1 --2- 172.21.15.152:0/620461863 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd8780d2d20 0x7fd8780b7780 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7fd8640097a0 tx=0x7fd86402b960 comp rx=0 tx=0).stop 2024-04-20T11:32:58.879 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.878+0000 7fd880600640 1 -- 172.21.15.152:0/620461863 shutdown_connections 2024-04-20T11:32:58.879 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.878+0000 7fd880600640 1 --2- 172.21.15.152:0/620461863 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd8780d2d20 0x7fd8780b7780 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:58.879 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.878+0000 7fd880600640 1 --2- 172.21.15.152:0/620461863 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd87815e890 0x7fd8780d27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:58.879 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.878+0000 7fd880600640 1 -- 172.21.15.152:0/620461863 >> 172.21.15.152:0/620461863 conn(0x7fd878067ae0 msgr2=0x7fd878067ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:58.879 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.878+0000 7fd880600640 1 -- 172.21.15.152:0/620461863 shutdown_connections 2024-04-20T11:32:58.879 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.878+0000 7fd880600640 1 -- 172.21.15.152:0/620461863 wait complete. 2024-04-20T11:32:58.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.880+0000 7fd880600640 1 Processor -- start 2024-04-20T11:32:58.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.880+0000 7fd880600640 1 -- start start 2024-04-20T11:32:58.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.881+0000 7fd880600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd8780b7cc0 0x7fd8781f76c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:58.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.881+0000 7fd880600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd87815e890 0x7fd87815c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:58.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.881+0000 7fd880600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd8780c7cb0 0x7fd878159680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:58.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.881+0000 7fd880600640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fd8780d1ea0 con 0x7fd8780c7cb0 2024-04-20T11:32:58.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.881+0000 7fd880600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd8780b8ab0 con 0x7fd87815e890 2024-04-20T11:32:58.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.881+0000 7fd880600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd8780d2020 con 0x7fd8780b7cc0 2024-04-20T11:32:58.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.881+0000 7fd87e600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd8780c7cb0 0x7fd878159680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:58.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.881+0000 7fd87e600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd8780c7cb0 0x7fd878159680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:32964/0 (socket says 172.21.15.152:32964) 2024-04-20T11:32:58.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.881+0000 7fd87e600640 1 -- 172.21.15.152:0/2986365754 learned_addr learned my addr 172.21.15.152:0/2986365754 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:32:58.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.881+0000 7fd87dc00640 1 --2- 172.21.15.152:0/2986365754 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd8780b7cc0 0x7fd8781f76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:58.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.881+0000 7fd87d200640 1 --2- 172.21.15.152:0/2986365754 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd87815e890 0x7fd87815c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:58.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.881+0000 7fd87e600640 1 -- 172.21.15.152:0/2986365754 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd8780b7cc0 msgr2=0x7fd8781f76c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:58.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.881+0000 7fd87e600640 1 --2- 172.21.15.152:0/2986365754 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd8780b7cc0 0x7fd8781f76c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:58.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.881+0000 7fd87e600640 1 -- 172.21.15.152:0/2986365754 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd87815e890 msgr2=0x7fd87815c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:58.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.881+0000 7fd87e600640 1 --2- 172.21.15.152:0/2986365754 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd87815e890 0x7fd87815c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:58.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.881+0000 7fd87e600640 1 -- 172.21.15.152:0/2986365754 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd87815f010 con 0x7fd8780c7cb0 2024-04-20T11:32:58.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.881+0000 7fd87d200640 1 --2- 172.21.15.152:0/2986365754 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd87815e890 0x7fd87815c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:32:58.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.882+0000 7fd87e600640 1 --2- 172.21.15.152:0/2986365754 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd8780c7cb0 0x7fd878159680 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7fd87000c450 tx=0x7fd87000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:58.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.882+0000 7fd875600640 1 -- 172.21.15.152:0/2986365754 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8700074e0 con 0x7fd8780c7cb0 2024-04-20T11:32:58.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.882+0000 7fd875600640 1 -- 172.21.15.152:0/2986365754 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8700047e0 con 0x7fd8780c7cb0 2024-04-20T11:32:58.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.882+0000 7fd880600640 1 -- 172.21.15.152:0/2986365754 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd87815f2f0 con 0x7fd8780c7cb0 2024-04-20T11:32:58.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.882+0000 7fd875600640 1 -- 172.21.15.152:0/2986365754 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd870004bd0 con 0x7fd8780c7cb0 2024-04-20T11:32:58.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.882+0000 7fd880600640 1 -- 172.21.15.152:0/2986365754 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd87815fb90 con 0x7fd8780c7cb0 2024-04-20T11:32:58.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.883+0000 7fd880600640 1 -- 172.21.15.152:0/2986365754 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd87815f4c0 con 0x7fd8780c7cb0 2024-04-20T11:32:58.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.885+0000 7fd875600640 1 -- 172.21.15.152:0/2986365754 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 91142+0+0 (secure 0 0 0) 0x7fd870011070 con 0x7fd8780c7cb0 2024-04-20T11:32:58.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.885+0000 7fd875600640 1 --2- 172.21.15.152:0/2986365754 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd85806ca10 0x7fd85806eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:58.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.886+0000 7fd875600640 1 -- 172.21.15.152:0/2986365754 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6543+0+0 (secure 0 0 0) 0x7fd87008bc80 con 0x7fd8780c7cb0 2024-04-20T11:32:58.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.886+0000 7fd87dc00640 1 --2- 172.21.15.152:0/2986365754 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd85806ca10 0x7fd85806eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:58.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.887+0000 7fd87dc00640 1 --2- 172.21.15.152:0/2986365754 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd85806ca10 0x7fd85806eed0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd868008ee0 tx=0x7fd868008430 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:58.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:58.890+0000 7fd875600640 1 -- 172.21.15.152:0/2986365754 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fd87001a2b0 con 0x7fd8780c7cb0 2024-04-20T11:32:59.118 INFO:teuthology.orchestra.run.smithi152.stderr:Invalid command: invalid chars @# in abcd@# 2024-04-20T11:32:59.119 INFO:teuthology.orchestra.run.smithi152.stderr:fs subvolume create [] [] [] [] [] [] [--namespace-isolated] : Create a CephFS subvolume in a volume, and optionally, with a specific size (in bytes), a specific data pool layout, a specific mode, in a specific subvolume group and in separate RADOS namespace 2024-04-20T11:32:59.119 INFO:teuthology.orchestra.run.smithi152.stderr:Error EINVAL: invalid command 2024-04-20T11:32:59.130 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-20T11:32:59.131 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rm testvol --yes-i-really-mean-it 2024-04-20T11:32:59.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:59.337+0000 7fbb01200640 1 Processor -- start 2024-04-20T11:32:59.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:59.337+0000 7fbb01200640 1 -- start start 2024-04-20T11:32:59.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:59.339+0000 7fbb01200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fbafc06f1c0 0x7fbafc06f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:59.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:59.339+0000 7fbb01200640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbafc15f7e0 con 0x7fbafc147cd0 2024-04-20T11:32:59.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:59.339+0000 7fbb01200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbafc15fac0 con 0x7fbafc1480b0 2024-04-20T11:32:59.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:59.339+0000 7fbb01200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fbafc06fb70 con 0x7fbafc06f1c0 2024-04-20T11:32:59.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:59.339+0000 7fbafb400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fbafc147cd0 0x7fbafc06e700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:33562/0 (socket says 172.21.15.152:33562) 2024-04-20T11:32:59.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:59.339+0000 7fbafb400640 1 -- 172.21.15.152:0/3450983058 learned_addr learned my addr 172.21.15.152:0/3450983058 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:32:59.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:59.339+0000 7fbafbe00640 1 --2- 172.21.15.152:0/3450983058 >> v2:172.21.15.152:3300/0 conn(0x7fbafc06f1c0 0x7fbafc06f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:59.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:59.339+0000 7fbafa000640 1 -- 172.21.15.152:0/3450983058 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3275733109 0 0) 0x7fbafc15f7e0 con 0x7fbafc147cd0 2024-04-20T11:32:59.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:59.339+0000 7fbafa000640 1 -- 172.21.15.152:0/3450983058 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbae0003520 con 0x7fbafc147cd0 2024-04-20T11:32:59.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:59.340+0000 7fbafbe00640 1 -- 172.21.15.152:0/3450983058 >> v1:172.21.15.175:6789/0 conn(0x7fbafc1480b0 legacy=0x7fbafc15ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:59.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:59.340+0000 7fbafbe00640 1 -- 172.21.15.152:0/3450983058 >> v1:172.21.15.152:6789/0 conn(0x7fbafc147cd0 legacy=0x7fbafc06e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:59.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:59.340+0000 7fbafbe00640 1 -- 172.21.15.152:0/3450983058 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbafc06df00 con 0x7fbafc06f1c0 2024-04-20T11:32:59.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:59.341+0000 7fbafbe00640 1 --2- 172.21.15.152:0/3450983058 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbafc06f1c0 0x7fbafc06f630 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7fbaf4009d90 tx=0x7fbaf402bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e635dd609bde1dab server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:59.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:59.341+0000 7fbafa000640 1 -- 172.21.15.152:0/3450983058 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf4039070 con 0x7fbafc06f1c0 2024-04-20T11:32:59.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:59.341+0000 7fbafa000640 1 -- 172.21.15.152:0/3450983058 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbaf402cd00 con 0x7fbafc06f1c0 2024-04-20T11:32:59.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:59.341+0000 7fbafa000640 1 -- 172.21.15.152:0/3450983058 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf4034640 con 0x7fbafc06f1c0 2024-04-20T11:32:59.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:59.342+0000 7fbb01200640 1 -- 172.21.15.152:0/3450983058 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbafc06f1c0 msgr2=0x7fbafc06f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:59.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:59.342+0000 7fbb01200640 1 --2- 172.21.15.152:0/3450983058 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbafc06f1c0 0x7fbafc06f630 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7fbaf4009d90 tx=0x7fbaf402bf50 comp rx=0 tx=0).stop 2024-04-20T11:32:59.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:59.342+0000 7fbb01200640 1 -- 172.21.15.152:0/3450983058 shutdown_connections 2024-04-20T11:32:59.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:59.342+0000 7fbb01200640 1 --2- 172.21.15.152:0/3450983058 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbafc06f1c0 0x7fbafc06f630 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:59.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:59.342+0000 7fbb01200640 1 -- 172.21.15.152:0/3450983058 >> 172.21.15.152:0/3450983058 conn(0x7fbafc067ae0 msgr2=0x7fbafc06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:59.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:59.342+0000 7fbb01200640 1 -- 172.21.15.152:0/3450983058 shutdown_connections 2024-04-20T11:32:59.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:59.342+0000 7fbb01200640 1 -- 172.21.15.152:0/3450983058 wait complete. 2024-04-20T11:32:59.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:59.344+0000 7fbb01200640 1 Processor -- start 2024-04-20T11:32:59.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:59.344+0000 7fbb01200640 1 -- start start 2024-04-20T11:32:59.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:59.345+0000 7fbb01200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbafc06f1c0 0x7fbafc13d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:59.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:59.345+0000 7fbb01200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbafc147cd0 0x7fbafc1f6d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:59.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:59.345+0000 7fbb01200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbafc1480b0 0x7fbafc144a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:59.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:59.345+0000 7fbb01200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fbafc071ed0 con 0x7fbafc1480b0 2024-04-20T11:32:59.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:59.345+0000 7fbb01200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbafc06ee10 con 0x7fbafc147cd0 2024-04-20T11:32:59.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:59.345+0000 7fbb01200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fbafc15f3c0 con 0x7fbafc06f1c0 2024-04-20T11:32:59.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:59.345+0000 7fbafbe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbafc1480b0 0x7fbafc144a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:59.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:59.345+0000 7fbafbe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbafc1480b0 0x7fbafc144a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:32986/0 (socket says 172.21.15.152:32986) 2024-04-20T11:32:59.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:59.345+0000 7fbafbe00640 1 -- 172.21.15.152:0/1827973589 learned_addr learned my addr 172.21.15.152:0/1827973589 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:32:59.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:59.345+0000 7fbafbe00640 1 -- 172.21.15.152:0/1827973589 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbafc06f1c0 msgr2=0x7fbafc13d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:59.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:59.345+0000 7fbafbe00640 1 --2- 172.21.15.152:0/1827973589 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbafc06f1c0 0x7fbafc13d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:59.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:59.345+0000 7fbafbe00640 1 -- 172.21.15.152:0/1827973589 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbafc147cd0 msgr2=0x7fbafc1f6d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:59.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:59.345+0000 7fbafbe00640 1 --2- 172.21.15.152:0/1827973589 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbafc147cd0 0x7fbafc1f6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:59.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:59.345+0000 7fbafbe00640 1 -- 172.21.15.152:0/1827973589 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbafc1f5cc0 con 0x7fbafc1480b0 2024-04-20T11:32:59.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:59.346+0000 7fbafbe00640 1 --2- 172.21.15.152:0/1827973589 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbafc1480b0 0x7fbafc144a90 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7fbaf4009d90 tx=0x7fbaf4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:59.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:59.346+0000 7fbaf3400640 1 -- 172.21.15.152:0/1827973589 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf4044020 con 0x7fbafc1480b0 2024-04-20T11:32:59.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:59.346+0000 7fbaf3400640 1 -- 172.21.15.152:0/1827973589 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbaf403a040 con 0x7fbafc1480b0 2024-04-20T11:32:59.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:59.346+0000 7fbb01200640 1 -- 172.21.15.152:0/1827973589 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbafc1f5fa0 con 0x7fbafc1480b0 2024-04-20T11:32:59.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:59.346+0000 7fbaf3400640 1 -- 172.21.15.152:0/1827973589 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf4039050 con 0x7fbafc1480b0 2024-04-20T11:32:59.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:59.346+0000 7fbb01200640 1 -- 172.21.15.152:0/1827973589 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbafc1f6560 con 0x7fbafc1480b0 2024-04-20T11:32:59.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:59.347+0000 7fbb01200640 1 -- 172.21.15.152:0/1827973589 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbafc0cdcf0 con 0x7fbafc1480b0 2024-04-20T11:32:59.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:59.348+0000 7fbaf3400640 1 -- 172.21.15.152:0/1827973589 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 91142+0+0 (secure 0 0 0) 0x7fbaf4003bd0 con 0x7fbafc1480b0 2024-04-20T11:32:59.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:59.349+0000 7fbaf3400640 1 --2- 172.21.15.152:0/1827973589 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fbadc06ca10 0x7fbadc06eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:59.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:59.349+0000 7fbaf3400640 1 -- 172.21.15.152:0/1827973589 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 6543+0+0 (secure 0 0 0) 0x7fbaf40ae6b0 con 0x7fbafc1480b0 2024-04-20T11:32:59.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:59.350+0000 7fbafb400640 1 --2- 172.21.15.152:0/1827973589 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fbadc06ca10 0x7fbadc06eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:59.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:59.351+0000 7fbafb400640 1 --2- 172.21.15.152:0/1827973589 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fbadc06ca10 0x7fbadc06eed0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fbaec001610 tx=0x7fbaec007930 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:59.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:59.354+0000 7fbaf3400640 1 -- 172.21.15.152:0/1827973589 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fbaf404e060 con 0x7fbafc1480b0 2024-04-20T11:32:59.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:32:59.560+0000 7fbb01200640 1 -- 172.21.15.152:0/1827973589 --> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] -- mgr_command(tid 0: {"prefix": "fs volume rm", "vol_name": "testvol", "yes-i-really-mean-it": "--yes-i-really-mean-it", "target": ["mon-mgr", ""]}) -- 0x7fbafc075e60 con 0x7fbadc06ca10 2024-04-20T11:32:59.571 INFO:tasks.ceph.mon.a.smithi152.stderr:2024-04-20T11:32:59.571+0000 7f8229800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-20T11:32:59.653 INFO:tasks.ceph.mds.d.smithi175.stdout:starting mds.d at 2024-04-20T11:33:02.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:02.617+0000 7fbaf3400640 1 -- 172.21.15.152:0/1827973589 <== mgr.4109 v2:172.21.15.175:6836/4021250441 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+78 (secure 0 0 0) 0x7fbafc075e60 con 0x7fbadc06ca10 2024-04-20T11:33:02.618 INFO:teuthology.orchestra.run.smithi152.stdout:metadata pool: cephfs.testvol.meta data pool: ['cephfs.testvol.data'] removed 2024-04-20T11:33:02.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:02.622+0000 7fbb01200640 1 -- 172.21.15.152:0/1827973589 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fbadc06ca10 msgr2=0x7fbadc06eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:02.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:02.622+0000 7fbb01200640 1 --2- 172.21.15.152:0/1827973589 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fbadc06ca10 0x7fbadc06eed0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fbaec001610 tx=0x7fbaec007930 comp rx=0 tx=0).stop 2024-04-20T11:33:02.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:02.622+0000 7fbb01200640 1 -- 172.21.15.152:0/1827973589 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbafc1480b0 msgr2=0x7fbafc144a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:02.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:02.622+0000 7fbb01200640 1 --2- 172.21.15.152:0/1827973589 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbafc1480b0 0x7fbafc144a90 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7fbaf4009d90 tx=0x7fbaf4004a40 comp rx=0 tx=0).stop 2024-04-20T11:33:02.623 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:02.622+0000 7fbb01200640 1 -- 172.21.15.152:0/1827973589 shutdown_connections 2024-04-20T11:33:02.623 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:02.622+0000 7fbb01200640 1 --2- 172.21.15.152:0/1827973589 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fbadc06ca10 0x7fbadc06eed0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:02.623 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:02.622+0000 7fbb01200640 1 --2- 172.21.15.152:0/1827973589 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbafc1480b0 0x7fbafc144a90 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:02.623 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:02.622+0000 7fbb01200640 1 --2- 172.21.15.152:0/1827973589 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbafc147cd0 0x7fbafc1f6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:02.623 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:02.622+0000 7fbb01200640 1 --2- 172.21.15.152:0/1827973589 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbafc06f1c0 0x7fbafc13d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:02.623 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:02.622+0000 7fbb01200640 1 -- 172.21.15.152:0/1827973589 >> 172.21.15.152:0/1827973589 conn(0x7fbafc067ae0 msgr2=0x7fbafc0cea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:02.623 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:02.623+0000 7fbb01200640 1 -- 172.21.15.152:0/1827973589 shutdown_connections 2024-04-20T11:33:02.623 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:02.623+0000 7fbb01200640 1 -- 172.21.15.152:0/1827973589 wait complete. 2024-04-20T11:33:02.644 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-20T11:33:02.644 DEBUG:teuthology.orchestra.run.smithi152:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-20T11:33:02.731 DEBUG:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-20T11:33:02.856 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-20T11:33:02.856 INFO:tasks.cephfs.mount:Terminating background process 2024-04-20T11:33:02.856 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-20T11:33:02.856 DEBUG:teuthology.orchestra.run.smithi152:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:33:02.914 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:33:03.127 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.127+0000 7f68d2e00640 1 Processor -- start 2024-04-20T11:33:03.127 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.127+0000 7f68d2e00640 1 -- start start 2024-04-20T11:33:03.128 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.129+0000 7f68d2e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f68cc160b90 0x7f68cc15d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:03.128 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.129+0000 7f68d2e00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68cc0ce8c0 con 0x7f68cc15d1e0 2024-04-20T11:33:03.128 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.129+0000 7f68d2e00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68cc1600b0 con 0x7f68cc071cd0 2024-04-20T11:33:03.129 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.129+0000 7f68d2e00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f68cc160390 con 0x7f68cc160b90 2024-04-20T11:33:03.129 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.129+0000 7f68d1000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f68cc160b90 0x7f68cc15d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:03.129 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.129+0000 7f68d1a00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f68cc15d1e0 0x7f68cc0cdfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:33572/0 (socket says 172.21.15.152:33572) 2024-04-20T11:33:03.129 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.129+0000 7f68d1a00640 1 -- 172.21.15.152:0/1829737623 learned_addr learned my addr 172.21.15.152:0/1829737623 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:33:03.130 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.130+0000 7f68d1000640 1 -- 172.21.15.152:0/1829737623 >> v1:172.21.15.175:6789/0 conn(0x7f68cc071cd0 legacy=0x7f68cc072120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:03.130 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.130+0000 7f68d1000640 1 -- 172.21.15.152:0/1829737623 >> v1:172.21.15.152:6789/0 conn(0x7f68cc15d1e0 legacy=0x7f68cc0cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:03.130 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.130+0000 7f68cbe00640 1 -- 172.21.15.152:0/1829737623 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3197826010 0 0) 0x7f68cc0ce8c0 con 0x7f68cc15d1e0 2024-04-20T11:33:03.130 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.130+0000 7f68d1000640 1 -- 172.21.15.152:0/1829737623 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68cc160670 con 0x7f68cc160b90 2024-04-20T11:33:03.130 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.131+0000 7f68d1000640 1 --2- 172.21.15.152:0/1829737623 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f68cc160b90 0x7f68cc15d5c0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f68b8009dc0 tx=0x7f68b802be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4bfe0f1cf965715 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:03.131 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.131+0000 7f68cbe00640 1 -- 172.21.15.152:0/1829737623 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b8039070 con 0x7f68cc160b90 2024-04-20T11:33:03.131 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.131+0000 7f68cbe00640 1 -- 172.21.15.152:0/1829737623 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68b802cc40 con 0x7f68cc160b90 2024-04-20T11:33:03.131 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.131+0000 7f68cbe00640 1 -- 172.21.15.152:0/1829737623 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b8034440 con 0x7f68cc160b90 2024-04-20T11:33:03.132 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.132+0000 7f68d2e00640 1 -- 172.21.15.152:0/1829737623 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f68cc160b90 msgr2=0x7f68cc15d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:03.132 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.132+0000 7f68d2e00640 1 --2- 172.21.15.152:0/1829737623 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f68cc160b90 0x7f68cc15d5c0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f68b8009dc0 tx=0x7f68b802be90 comp rx=0 tx=0).stop 2024-04-20T11:33:03.132 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.132+0000 7f68d2e00640 1 -- 172.21.15.152:0/1829737623 shutdown_connections 2024-04-20T11:33:03.132 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.132+0000 7f68d2e00640 1 --2- 172.21.15.152:0/1829737623 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f68cc160b90 0x7f68cc15d5c0 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:03.132 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.132+0000 7f68d2e00640 1 -- 172.21.15.152:0/1829737623 >> 172.21.15.152:0/1829737623 conn(0x7f68cc067ae0 msgr2=0x7f68cc06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:03.132 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.132+0000 7f68d2e00640 1 -- 172.21.15.152:0/1829737623 shutdown_connections 2024-04-20T11:33:03.132 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.132+0000 7f68d2e00640 1 -- 172.21.15.152:0/1829737623 wait complete. 2024-04-20T11:33:03.133 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.134+0000 7f68d2e00640 1 Processor -- start 2024-04-20T11:33:03.133 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.134+0000 7f68d2e00640 1 -- start start 2024-04-20T11:33:03.135 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.135+0000 7f68d2e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f68cc071cd0 0x7f68cc1f1a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:03.135 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.135+0000 7f68d2e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f68cc15d1e0 0x7f68cc1580a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:03.135 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.135+0000 7f68d2e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68cc160b90 0x7f68cc1585e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:03.135 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.135+0000 7f68d2e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f68cc0ccef0 con 0x7f68cc15d1e0 2024-04-20T11:33:03.135 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.135+0000 7f68d2e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f68cc0ccd70 con 0x7f68cc160b90 2024-04-20T11:33:03.135 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.135+0000 7f68d2e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f68cc0cd070 con 0x7f68cc071cd0 2024-04-20T11:33:03.135 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.135+0000 7f68d1000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f68cc15d1e0 0x7f68cc1580a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:03.135 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.135+0000 7f68d1000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f68cc15d1e0 0x7f68cc1580a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:33006/0 (socket says 172.21.15.152:33006) 2024-04-20T11:33:03.135 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.135+0000 7f68d1000640 1 -- 172.21.15.152:0/1343588843 learned_addr learned my addr 172.21.15.152:0/1343588843 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:33:03.136 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.136+0000 7f68d1a00640 1 --2- 172.21.15.152:0/1343588843 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f68cc071cd0 0x7f68cc1f1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:03.136 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.136+0000 7f68d2400640 1 --2- 172.21.15.152:0/1343588843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68cc160b90 0x7f68cc1585e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:03.136 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.136+0000 7f68d1000640 1 -- 172.21.15.152:0/1343588843 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f68cc071cd0 msgr2=0x7f68cc1f1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:03.137 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.136+0000 7f68d1000640 1 --2- 172.21.15.152:0/1343588843 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f68cc071cd0 0x7f68cc1f1a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:03.137 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.136+0000 7f68d1000640 1 -- 172.21.15.152:0/1343588843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68cc160b90 msgr2=0x7f68cc1585e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:03.137 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.136+0000 7f68d1000640 1 --2- 172.21.15.152:0/1343588843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68cc160b90 0x7f68cc1585e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:03.137 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.136+0000 7f68d1000640 1 -- 172.21.15.152:0/1343588843 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68cc0a9e00 con 0x7f68cc15d1e0 2024-04-20T11:33:03.137 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.136+0000 7f68d1000640 1 --2- 172.21.15.152:0/1343588843 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f68cc15d1e0 0x7f68cc1580a0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f68b8037040 tx=0x7f68b8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:03.137 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.137+0000 7f68c9600640 1 -- 172.21.15.152:0/1343588843 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b8044020 con 0x7f68cc15d1e0 2024-04-20T11:33:03.137 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.137+0000 7f68c9600640 1 -- 172.21.15.152:0/1343588843 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68b803f470 con 0x7f68cc15d1e0 2024-04-20T11:33:03.137 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.137+0000 7f68c9600640 1 -- 172.21.15.152:0/1343588843 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b8039040 con 0x7f68cc15d1e0 2024-04-20T11:33:03.137 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.137+0000 7f68d2e00640 1 -- 172.21.15.152:0/1343588843 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68cc1e7090 con 0x7f68cc15d1e0 2024-04-20T11:33:03.137 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.137+0000 7f68d2e00640 1 -- 172.21.15.152:0/1343588843 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68cc1561a0 con 0x7f68cc15d1e0 2024-04-20T11:33:03.138 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.138+0000 7f68d2e00640 1 -- 172.21.15.152:0/1343588843 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68cc075e60 con 0x7f68cc15d1e0 2024-04-20T11:33:03.138 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.139+0000 7f68c9600640 1 -- 172.21.15.152:0/1343588843 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 91142+0+0 (secure 0 0 0) 0x7f68b8047020 con 0x7f68cc15d1e0 2024-04-20T11:33:03.139 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.140+0000 7f68c9600640 1 --2- 172.21.15.152:0/1343588843 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f68ac06ca10 0x7f68ac06eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:03.139 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.140+0000 7f68c9600640 1 -- 172.21.15.152:0/1343588843 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5795+0+0 (secure 0 0 0) 0x7f68b80ae750 con 0x7f68cc15d1e0 2024-04-20T11:33:03.140 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.141+0000 7f68d1a00640 1 --2- 172.21.15.152:0/1343588843 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f68ac06ca10 0x7f68ac06eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:03.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.142+0000 7f68d1a00640 1 --2- 172.21.15.152:0/1343588843 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f68ac06ca10 0x7f68ac06eed0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f68bc000f60 tx=0x7f68bc007b50 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:03.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.144+0000 7f68c9600640 1 -- 172.21.15.152:0/1343588843 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f68b804e030 con 0x7f68cc15d1e0 2024-04-20T11:33:03.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.349+0000 7f68d2e00640 1 -- 172.21.15.152:0/1343588843 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f68cc1e73b0 con 0x7f68cc15d1e0 2024-04-20T11:33:03.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.350+0000 7f68c9600640 1 -- 172.21.15.152:0/1343588843 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+16851 (secure 0 0 0) 0x7f68b807cb60 con 0x7f68cc15d1e0 2024-04-20T11:33:03.350 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:33:03.351 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":91,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:33:02.588760+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:32:39.268411+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-20T11:32:40.497900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.175:6835/2632275131":"2024-04-21T11:32:59.571722+0000","172.21.15.175:6834/2632275131":"2024-04-21T11:32:59.571722+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:33:03.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.353+0000 7f68d2e00640 1 -- 172.21.15.152:0/1343588843 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f68ac06ca10 msgr2=0x7f68ac06eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:03.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.353+0000 7f68d2e00640 1 --2- 172.21.15.152:0/1343588843 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f68ac06ca10 0x7f68ac06eed0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f68bc000f60 tx=0x7f68bc007b50 comp rx=0 tx=0).stop 2024-04-20T11:33:03.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.353+0000 7f68d2e00640 1 -- 172.21.15.152:0/1343588843 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f68cc15d1e0 msgr2=0x7f68cc1580a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:03.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.353+0000 7f68d2e00640 1 --2- 172.21.15.152:0/1343588843 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f68cc15d1e0 0x7f68cc1580a0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f68b8037040 tx=0x7f68b8004a40 comp rx=0 tx=0).stop 2024-04-20T11:33:03.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.353+0000 7f68d2e00640 1 -- 172.21.15.152:0/1343588843 shutdown_connections 2024-04-20T11:33:03.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.353+0000 7f68d2e00640 1 --2- 172.21.15.152:0/1343588843 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f68ac06ca10 0x7f68ac06eed0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:03.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.353+0000 7f68d2e00640 1 --2- 172.21.15.152:0/1343588843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68cc160b90 0x7f68cc1585e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:03.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.353+0000 7f68d2e00640 1 --2- 172.21.15.152:0/1343588843 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f68cc15d1e0 0x7f68cc1580a0 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:03.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.353+0000 7f68d2e00640 1 --2- 172.21.15.152:0/1343588843 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f68cc071cd0 0x7f68cc1f1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:03.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.353+0000 7f68d2e00640 1 -- 172.21.15.152:0/1343588843 >> 172.21.15.152:0/1343588843 conn(0x7f68cc067ae0 msgr2=0x7f68cc06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:03.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.353+0000 7f68d2e00640 1 -- 172.21.15.152:0/1343588843 shutdown_connections 2024-04-20T11:33:03.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.354+0000 7f68d2e00640 1 -- 172.21.15.152:0/1343588843 wait complete. 2024-04-20T11:33:03.372 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T11:26:47.246571+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T11:32:39.268411+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T11:32:40.497900+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T11:33:03.373 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T11:33:03.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.579+0000 7fa3e6600640 1 Processor -- start 2024-04-20T11:33:03.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.579+0000 7fa3e6600640 1 -- start start 2024-04-20T11:33:03.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.581+0000 7fa3e6600640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fa3e0146000 0x7fa3e00b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:03.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.581+0000 7fa3e6600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa3e00b2170 0x7fa3e00b25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:03.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.581+0000 7fa3e6600640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3e01463e0 con 0x7fa3e00cd1b0 2024-04-20T11:33:03.582 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.581+0000 7fa3e6600640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fa3e00b4750 con 0x7fa3e0146000 2024-04-20T11:33:03.582 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.581+0000 7fa3e6600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa3e00b3e20 con 0x7fa3e00b2170 2024-04-20T11:33:03.582 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.581+0000 7fa3dfe00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fa3e00cd1b0 0x7fa3e00cd5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:33586/0 (socket says 172.21.15.152:33586) 2024-04-20T11:33:03.582 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.581+0000 7fa3df400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fa3e0146000 0x7fa3e00b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:03.582 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.581+0000 7fa3dfe00640 1 -- 172.21.15.152:0/588465927 learned_addr learned my addr 172.21.15.152:0/588465927 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:33:03.582 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.581+0000 7fa3dea00640 1 --2- 172.21.15.152:0/588465927 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa3e00b2170 0x7fa3e00b25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:03.582 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.582+0000 7fa3df400640 1 -- 172.21.15.152:0/588465927 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa3e00b2170 msgr2=0x7fa3e00b25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:03.582 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.582+0000 7fa3df400640 1 --2- 172.21.15.152:0/588465927 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa3e00b2170 0x7fa3e00b25f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:03.582 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.582+0000 7fa3df400640 1 -- 172.21.15.152:0/588465927 >> v1:172.21.15.152:6789/0 conn(0x7fa3e00cd1b0 legacy=0x7fa3e00cd5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:03.582 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.582+0000 7fa3df400640 1 -- 172.21.15.152:0/588465927 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3e00b4100 con 0x7fa3e0146000 2024-04-20T11:33:03.582 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.582+0000 7fa3de000640 1 -- 172.21.15.152:0/588465927 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 237615182 0 0) 0x7fa3e01463e0 con 0x7fa3e00cd1b0 2024-04-20T11:33:03.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.583+0000 7fa3df400640 1 --2- 172.21.15.152:0/588465927 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa3e0146000 0x7fa3e00b4be0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fa3d0009d90 tx=0x7fa3d002be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61d7f8b9ccb67eba server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:03.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.583+0000 7fa3de000640 1 -- 172.21.15.152:0/588465927 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d0039070 con 0x7fa3e0146000 2024-04-20T11:33:03.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.583+0000 7fa3de000640 1 -- 172.21.15.152:0/588465927 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3d002cbc0 con 0x7fa3e0146000 2024-04-20T11:33:03.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.583+0000 7fa3de000640 1 -- 172.21.15.152:0/588465927 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d0034440 con 0x7fa3e0146000 2024-04-20T11:33:03.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.584+0000 7fa3e6600640 1 -- 172.21.15.152:0/588465927 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa3e0146000 msgr2=0x7fa3e00b4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:03.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.584+0000 7fa3e6600640 1 --2- 172.21.15.152:0/588465927 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa3e0146000 0x7fa3e00b4be0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fa3d0009d90 tx=0x7fa3d002be10 comp rx=0 tx=0).stop 2024-04-20T11:33:03.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.584+0000 7fa3e6600640 1 -- 172.21.15.152:0/588465927 shutdown_connections 2024-04-20T11:33:03.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.584+0000 7fa3e6600640 1 --2- 172.21.15.152:0/588465927 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa3e00b2170 0x7fa3e00b25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:03.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.584+0000 7fa3e6600640 1 --2- 172.21.15.152:0/588465927 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa3e0146000 0x7fa3e00b4be0 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:03.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.584+0000 7fa3e6600640 1 -- 172.21.15.152:0/588465927 >> 172.21.15.152:0/588465927 conn(0x7fa3e006b980 msgr2=0x7fa3e006bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:03.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.584+0000 7fa3e6600640 1 -- 172.21.15.152:0/588465927 shutdown_connections 2024-04-20T11:33:03.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.584+0000 7fa3e6600640 1 -- 172.21.15.152:0/588465927 wait complete. 2024-04-20T11:33:03.585 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.586+0000 7fa3e6600640 1 Processor -- start 2024-04-20T11:33:03.585 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.586+0000 7fa3e6600640 1 -- start start 2024-04-20T11:33:03.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.587+0000 7fa3e6600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa3e00b2170 0x7fa3e01f1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:03.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.587+0000 7fa3e6600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa3e00cd1b0 0x7fa3e01f5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:03.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.587+0000 7fa3e6600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3e0146000 0x7fa3e00ae810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:03.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.587+0000 7fa3e6600640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e015ce70 con 0x7fa3e00cd1b0 2024-04-20T11:33:03.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.587+0000 7fa3e6600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e015ccf0 con 0x7fa3e0146000 2024-04-20T11:33:03.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.587+0000 7fa3e6600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa3e015cff0 con 0x7fa3e00b2170 2024-04-20T11:33:03.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.587+0000 7fa3dea00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa3e00cd1b0 0x7fa3e01f5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:03.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.587+0000 7fa3dea00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa3e00cd1b0 0x7fa3e01f5350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:33022/0 (socket says 172.21.15.152:33022) 2024-04-20T11:33:03.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.587+0000 7fa3dea00640 1 -- 172.21.15.152:0/225094817 learned_addr learned my addr 172.21.15.152:0/225094817 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:33:03.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.587+0000 7fa3df400640 1 --2- 172.21.15.152:0/225094817 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa3e00b2170 0x7fa3e01f1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:03.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.588+0000 7fa3dfe00640 1 --2- 172.21.15.152:0/225094817 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3e0146000 0x7fa3e00ae810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:03.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.588+0000 7fa3dea00640 1 -- 172.21.15.152:0/225094817 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa3e00b2170 msgr2=0x7fa3e01f1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:03.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.588+0000 7fa3dea00640 1 --2- 172.21.15.152:0/225094817 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa3e00b2170 0x7fa3e01f1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:03.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.588+0000 7fa3dea00640 1 -- 172.21.15.152:0/225094817 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3e0146000 msgr2=0x7fa3e00ae810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:03.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.588+0000 7fa3dea00640 1 --2- 172.21.15.152:0/225094817 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3e0146000 0x7fa3e00ae810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:03.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.588+0000 7fa3dea00640 1 -- 172.21.15.152:0/225094817 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3e013efe0 con 0x7fa3e00cd1b0 2024-04-20T11:33:03.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.588+0000 7fa3df400640 1 --2- 172.21.15.152:0/225094817 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa3e00b2170 0x7fa3e01f1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:33:03.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.588+0000 7fa3dea00640 1 --2- 172.21.15.152:0/225094817 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa3e00cd1b0 0x7fa3e01f5350 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fa3c800d830 tx=0x7fa3c800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:03.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.588+0000 7fa3d7400640 1 -- 172.21.15.152:0/225094817 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c800b6c0 con 0x7fa3e00cd1b0 2024-04-20T11:33:03.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.588+0000 7fa3d7400640 1 -- 172.21.15.152:0/225094817 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3c8019070 con 0x7fa3e00cd1b0 2024-04-20T11:33:03.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.588+0000 7fa3d7400640 1 -- 172.21.15.152:0/225094817 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c800ba80 con 0x7fa3e00cd1b0 2024-04-20T11:33:03.590 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.588+0000 7fa3e6600640 1 -- 172.21.15.152:0/225094817 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3e01f5890 con 0x7fa3e00cd1b0 2024-04-20T11:33:03.590 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.589+0000 7fa3e6600640 1 -- 172.21.15.152:0/225094817 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3e013f560 con 0x7fa3e00cd1b0 2024-04-20T11:33:03.590 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.590+0000 7fa3e6600640 1 -- 172.21.15.152:0/225094817 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3e00689f0 con 0x7fa3e00cd1b0 2024-04-20T11:33:03.590 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.591+0000 7fa3d7400640 1 -- 172.21.15.152:0/225094817 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 91142+0+0 (secure 0 0 0) 0x7fa3c801e460 con 0x7fa3e00cd1b0 2024-04-20T11:33:03.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.591+0000 7fa3d7400640 1 --2- 172.21.15.152:0/225094817 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fa3bc06ca40 0x7fa3bc06ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:03.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.592+0000 7fa3d7400640 1 -- 172.21.15.152:0/225094817 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5795+0+0 (secure 0 0 0) 0x7fa3c808c480 con 0x7fa3e00cd1b0 2024-04-20T11:33:03.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.592+0000 7fa3df400640 1 --2- 172.21.15.152:0/225094817 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fa3bc06ca40 0x7fa3bc06ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:03.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.593+0000 7fa3df400640 1 --2- 172.21.15.152:0/225094817 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fa3bc06ca40 0x7fa3bc06ef00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa3d0036000 tx=0x7fa3d0030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:03.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.596+0000 7fa3d7400640 1 -- 172.21.15.152:0/225094817 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fa3c805a6a0 con 0x7fa3e00cd1b0 2024-04-20T11:33:03.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.802+0000 7fa3e6600640 1 -- 172.21.15.152:0/225094817 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa3e0145500 con 0x7fa3e00cd1b0 2024-04-20T11:33:03.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.803+0000 7fa3d7400640 1 -- 172.21.15.152:0/225094817 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) ==== 93+0+10 (secure 0 0 0) 0x7fa3c8059fa0 con 0x7fa3e00cd1b0 2024-04-20T11:33:03.803 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 1 2024-04-20T11:33:03.805 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.805+0000 7fa3e6600640 1 -- 172.21.15.152:0/225094817 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fa3bc06ca40 msgr2=0x7fa3bc06ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:03.805 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.805+0000 7fa3e6600640 1 --2- 172.21.15.152:0/225094817 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fa3bc06ca40 0x7fa3bc06ef00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa3d0036000 tx=0x7fa3d0030000 comp rx=0 tx=0).stop 2024-04-20T11:33:03.805 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.805+0000 7fa3e6600640 1 -- 172.21.15.152:0/225094817 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa3e00cd1b0 msgr2=0x7fa3e01f5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:03.805 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.805+0000 7fa3e6600640 1 --2- 172.21.15.152:0/225094817 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa3e00cd1b0 0x7fa3e01f5350 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fa3c800d830 tx=0x7fa3c800dcf0 comp rx=0 tx=0).stop 2024-04-20T11:33:03.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.806+0000 7fa3e6600640 1 -- 172.21.15.152:0/225094817 shutdown_connections 2024-04-20T11:33:03.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.806+0000 7fa3e6600640 1 --2- 172.21.15.152:0/225094817 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fa3bc06ca40 0x7fa3bc06ef00 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:03.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.806+0000 7fa3e6600640 1 --2- 172.21.15.152:0/225094817 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3e0146000 0x7fa3e00ae810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:03.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.806+0000 7fa3e6600640 1 --2- 172.21.15.152:0/225094817 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa3e00cd1b0 0x7fa3e01f5350 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:03.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.806+0000 7fa3e6600640 1 --2- 172.21.15.152:0/225094817 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa3e00b2170 0x7fa3e01f1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:03.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.806+0000 7fa3e6600640 1 -- 172.21.15.152:0/225094817 >> 172.21.15.152:0/225094817 conn(0x7fa3e006b980 msgr2=0x7fa3e006f0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:03.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.806+0000 7fa3e6600640 1 -- 172.21.15.152:0/225094817 shutdown_connections 2024-04-20T11:33:03.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:03.806+0000 7fa3e6600640 1 -- 172.21.15.152:0/225094817 wait complete. 2024-04-20T11:33:03.824 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T11:33:04.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.025+0000 7fdab8e00640 1 Processor -- start 2024-04-20T11:33:04.026 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.026+0000 7fdab8e00640 1 -- start start 2024-04-20T11:33:04.026 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.027+0000 7fdab8e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fdab415eb50 0x7fdab40d0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:04.026 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.027+0000 7fdab8e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdab40d35a0 0x7fdab40cf470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:04.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.027+0000 7fdab8e00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdab40d19c0 con 0x7fdab4163230 2024-04-20T11:33:04.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.027+0000 7fdab8e00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fdab4162750 con 0x7fdab415eb50 2024-04-20T11:33:04.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.027+0000 7fdab8e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fdab4162a30 con 0x7fdab40d35a0 2024-04-20T11:33:04.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.027+0000 7fdab2a00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fdab4163230 0x7fdab40d2e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:33590/0 (socket says 172.21.15.152:33590) 2024-04-20T11:33:04.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.027+0000 7fdab2a00640 1 -- 172.21.15.152:0/2570977297 learned_addr learned my addr 172.21.15.152:0/2570977297 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:33:04.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.028+0000 7fdab3400640 1 --2- 172.21.15.152:0/2570977297 >> v2:172.21.15.175:3300/0 conn(0x7fdab415eb50 0x7fdab40d0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:04.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.028+0000 7fdab3e00640 1 --2- 172.21.15.152:0/2570977297 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdab40d35a0 0x7fdab40cf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:04.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.028+0000 7fdab2000640 1 -- 172.21.15.152:0/2570977297 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 223685515 0 0) 0x7fdab40d19c0 con 0x7fdab4163230 2024-04-20T11:33:04.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.028+0000 7fdab2000640 1 -- 172.21.15.152:0/2570977297 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fda98003520 con 0x7fdab4163230 2024-04-20T11:33:04.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.028+0000 7fdab2000640 1 -- 172.21.15.152:0/2570977297 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 497497651 0 0) 0x7fda98003520 con 0x7fdab4163230 2024-04-20T11:33:04.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.028+0000 7fdab2000640 1 -- 172.21.15.152:0/2570977297 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdab40d35a0 msgr2=0x7fdab40cf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:04.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.028+0000 7fdab2000640 1 --2- 172.21.15.152:0/2570977297 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdab40d35a0 0x7fdab40cf470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:04.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.028+0000 7fdab2000640 1 -- 172.21.15.152:0/2570977297 >> v2:172.21.15.175:3300/0 conn(0x7fdab415eb50 msgr2=0x7fdab40d0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:04.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.028+0000 7fdab2000640 1 --2- 172.21.15.152:0/2570977297 >> v2:172.21.15.175:3300/0 conn(0x7fdab415eb50 0x7fdab40d0f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:04.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.028+0000 7fdab2000640 1 -- 172.21.15.152:0/2570977297 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdab4162d10 con 0x7fdab4163230 2024-04-20T11:33:04.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.028+0000 7fdab2000640 1 -- 172.21.15.152:0/2570977297 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7fda9c0017f0 con 0x7fdab4163230 2024-04-20T11:33:04.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.028+0000 7fdab2000640 1 -- 172.21.15.152:0/2570977297 >> v1:172.21.15.152:6789/0 conn(0x7fdab4163230 legacy=0x7fdab40d2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:04.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.028+0000 7fdab2000640 1 --2- 172.21.15.152:0/2570977297 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fda980039b0 0x7fda98003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:04.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.028+0000 7fdab2000640 1 --2- 172.21.15.152:0/2570977297 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdab40d35a0 0x7fda98004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:04.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.028+0000 7fdab2000640 1 --2- 172.21.15.152:0/2570977297 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda98004d40 0x7fda980051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:04.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.028+0000 7fdab2000640 1 -- 172.21.15.152:0/2570977297 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fdab4162750 con 0x7fda980039b0 2024-04-20T11:33:04.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.028+0000 7fdab2000640 1 -- 172.21.15.152:0/2570977297 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdab4162a30 con 0x7fda98004d40 2024-04-20T11:33:04.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.028+0000 7fdab2000640 1 -- 172.21.15.152:0/2570977297 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fda9800b9f0 con 0x7fdab40d35a0 2024-04-20T11:33:04.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.029+0000 7fdab3400640 1 --2- 172.21.15.152:0/2570977297 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fda980039b0 0x7fda98003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:04.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.029+0000 7fdab2a00640 1 --2- 172.21.15.152:0/2570977297 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdab40d35a0 0x7fda98004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:04.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.029+0000 7fdab3400640 1 -- 172.21.15.152:0/2570977297 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdab40d35a0 msgr2=0x7fda98004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:04.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.029+0000 7fdab3400640 1 --2- 172.21.15.152:0/2570977297 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdab40d35a0 0x7fda98004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:04.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.029+0000 7fdab3400640 1 -- 172.21.15.152:0/2570977297 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda98004d40 msgr2=0x7fda980051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:04.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.029+0000 7fdab3400640 1 --2- 172.21.15.152:0/2570977297 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda98004d40 0x7fda980051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:04.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.029+0000 7fdab3400640 1 -- 172.21.15.152:0/2570977297 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda98003520 con 0x7fda980039b0 2024-04-20T11:33:04.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.030+0000 7fdab3400640 1 --2- 172.21.15.152:0/2570977297 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fda980039b0 0x7fda98003dd0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fdaa4009ac0 tx=0x7fda98005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73a5dcc64ded23c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:04.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.030+0000 7fdab2000640 1 -- 172.21.15.152:0/2570977297 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa4039070 con 0x7fda980039b0 2024-04-20T11:33:04.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.031+0000 7fdab2000640 1 -- 172.21.15.152:0/2570977297 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdaa402c710 con 0x7fda980039b0 2024-04-20T11:33:04.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.031+0000 7fdab2000640 1 -- 172.21.15.152:0/2570977297 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa4009690 con 0x7fda980039b0 2024-04-20T11:33:04.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.031+0000 7fdab8e00640 1 -- 172.21.15.152:0/2570977297 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fda980039b0 msgr2=0x7fda98003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:04.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.031+0000 7fdab8e00640 1 --2- 172.21.15.152:0/2570977297 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fda980039b0 0x7fda98003dd0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fdaa4009ac0 tx=0x7fda98005830 comp rx=0 tx=0).stop 2024-04-20T11:33:04.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.031+0000 7fdab3e00640 1 -- 172.21.15.152:0/2570977297 reap_dead start 2024-04-20T11:33:04.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.031+0000 7fdab8e00640 1 -- 172.21.15.152:0/2570977297 shutdown_connections 2024-04-20T11:33:04.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.031+0000 7fdab8e00640 1 -- 172.21.15.152:0/2570977297 >> 172.21.15.152:0/2570977297 conn(0x7fdab408bfd0 msgr2=0x7fdab40d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:04.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.031+0000 7fdab8e00640 1 -- 172.21.15.152:0/2570977297 shutdown_connections 2024-04-20T11:33:04.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.031+0000 7fdab8e00640 1 -- 172.21.15.152:0/2570977297 wait complete. 2024-04-20T11:33:04.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.033+0000 7fdab8e00640 1 Processor -- start 2024-04-20T11:33:04.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.033+0000 7fdab8e00640 1 -- start start 2024-04-20T11:33:04.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.034+0000 7fdab8e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda980039b0 0x7fdab4157ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:04.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.034+0000 7fdab8e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdab40b9010 0x7fdab40b9490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:04.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.034+0000 7fdab8e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdab41595d0 0x7fdab4159a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:04.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.034+0000 7fdab8e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fdab40d15c0 con 0x7fdab41595d0 2024-04-20T11:33:04.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.034+0000 7fdab8e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdab40d1440 con 0x7fda980039b0 2024-04-20T11:33:04.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.034+0000 7fdab8e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fdab40d1740 con 0x7fdab40b9010 2024-04-20T11:33:04.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.034+0000 7fdab3e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdab41595d0 0x7fdab4159a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:04.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.034+0000 7fdab3e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdab41595d0 0x7fdab4159a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:33040/0 (socket says 172.21.15.152:33040) 2024-04-20T11:33:04.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.034+0000 7fdab3e00640 1 -- 172.21.15.152:0/1946826263 learned_addr learned my addr 172.21.15.152:0/1946826263 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:33:04.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.035+0000 7fdab2a00640 1 --2- 172.21.15.152:0/1946826263 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdab40b9010 0x7fdab40b9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:04.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.035+0000 7fdab3400640 1 --2- 172.21.15.152:0/1946826263 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda980039b0 0x7fdab4157ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:04.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.035+0000 7fdab3e00640 1 -- 172.21.15.152:0/1946826263 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdab40b9010 msgr2=0x7fdab40b9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:04.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.035+0000 7fdab3e00640 1 --2- 172.21.15.152:0/1946826263 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdab40b9010 0x7fdab40b9490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:04.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.035+0000 7fdab3e00640 1 -- 172.21.15.152:0/1946826263 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda980039b0 msgr2=0x7fdab4157ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:04.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.035+0000 7fdab3e00640 1 --2- 172.21.15.152:0/1946826263 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda980039b0 0x7fdab4157ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:04.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.035+0000 7fdab3e00640 1 -- 172.21.15.152:0/1946826263 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdab40b5860 con 0x7fdab41595d0 2024-04-20T11:33:04.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.035+0000 7fdab2a00640 1 --2- 172.21.15.152:0/1946826263 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdab40b9010 0x7fdab40b9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:33:04.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.035+0000 7fdab3e00640 1 --2- 172.21.15.152:0/1946826263 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdab41595d0 0x7fdab4159a50 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fdab40b3a30 tx=0x7fdaac00b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:04.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.035+0000 7fdaab400640 1 -- 172.21.15.152:0/1946826263 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaac018020 con 0x7fdab41595d0 2024-04-20T11:33:04.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.036+0000 7fdaab400640 1 -- 172.21.15.152:0/1946826263 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdaac00bce0 con 0x7fdab41595d0 2024-04-20T11:33:04.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.036+0000 7fdaab400640 1 -- 172.21.15.152:0/1946826263 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaac014640 con 0x7fdab41595d0 2024-04-20T11:33:04.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.036+0000 7fdab8e00640 1 -- 172.21.15.152:0/1946826263 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdab40b4ce0 con 0x7fdab41595d0 2024-04-20T11:33:04.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.036+0000 7fdab8e00640 1 -- 172.21.15.152:0/1946826263 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdab40b52a0 con 0x7fdab41595d0 2024-04-20T11:33:04.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.037+0000 7fdab8e00640 1 -- 172.21.15.152:0/1946826263 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdab4075e60 con 0x7fdab41595d0 2024-04-20T11:33:04.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.038+0000 7fdaab400640 1 -- 172.21.15.152:0/1946826263 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 91142+0+0 (secure 0 0 0) 0x7fdaac005d10 con 0x7fdab41595d0 2024-04-20T11:33:04.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.039+0000 7fdaab400640 1 --2- 172.21.15.152:0/1946826263 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fda9006ca10 0x7fda9006eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:04.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.039+0000 7fdaab400640 1 -- 172.21.15.152:0/1946826263 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5795+0+0 (secure 0 0 0) 0x7fdaac08c600 con 0x7fdab41595d0 2024-04-20T11:33:04.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.039+0000 7fdab3400640 1 --2- 172.21.15.152:0/1946826263 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fda9006ca10 0x7fda9006eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:04.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.041+0000 7fdab3400640 1 --2- 172.21.15.152:0/1946826263 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fda9006ca10 0x7fda9006eed0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fdaa4036000 tx=0x7fdaa4030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:04.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.044+0000 7fdaab400640 1 -- 172.21.15.152:0/1946826263 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fdaac022030 con 0x7fdab41595d0 2024-04-20T11:33:04.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.241+0000 7fdab8e00640 1 -- 172.21.15.152:0/1946826263 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fdab408c680 con 0x7fdab41595d0 2024-04-20T11:33:04.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.242+0000 7fdaab400640 1 -- 172.21.15.152:0/1946826263 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) ==== 104+0+11 (secure 0 0 0) 0x7fdaac05a7a0 con 0x7fdab41595d0 2024-04-20T11:33:04.242 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 32 2024-04-20T11:33:04.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.245+0000 7fdab8e00640 1 -- 172.21.15.152:0/1946826263 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fda9006ca10 msgr2=0x7fda9006eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:04.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.245+0000 7fdab8e00640 1 --2- 172.21.15.152:0/1946826263 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fda9006ca10 0x7fda9006eed0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fdaa4036000 tx=0x7fdaa4030000 comp rx=0 tx=0).stop 2024-04-20T11:33:04.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.245+0000 7fdab8e00640 1 -- 172.21.15.152:0/1946826263 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdab41595d0 msgr2=0x7fdab4159a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:04.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.245+0000 7fdab8e00640 1 --2- 172.21.15.152:0/1946826263 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdab41595d0 0x7fdab4159a50 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fdab40b3a30 tx=0x7fdaac00b480 comp rx=0 tx=0).stop 2024-04-20T11:33:04.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.245+0000 7fdab8e00640 1 -- 172.21.15.152:0/1946826263 shutdown_connections 2024-04-20T11:33:04.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.245+0000 7fdab8e00640 1 --2- 172.21.15.152:0/1946826263 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fda9006ca10 0x7fda9006eed0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:04.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.245+0000 7fdab8e00640 1 --2- 172.21.15.152:0/1946826263 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdab41595d0 0x7fdab4159a50 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:04.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.245+0000 7fdab8e00640 1 --2- 172.21.15.152:0/1946826263 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdab40b9010 0x7fdab40b9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:04.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.245+0000 7fdab8e00640 1 --2- 172.21.15.152:0/1946826263 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda980039b0 0x7fdab4157ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:04.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.245+0000 7fdab8e00640 1 -- 172.21.15.152:0/1946826263 >> 172.21.15.152:0/1946826263 conn(0x7fdab408bfd0 msgr2=0x7fdab40b1130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:04.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.245+0000 7fdab8e00640 1 -- 172.21.15.152:0/1946826263 shutdown_connections 2024-04-20T11:33:04.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.245+0000 7fdab8e00640 1 -- 172.21.15.152:0/1946826263 wait complete. 2024-04-20T11:33:04.262 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T11:33:04.467 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.467+0000 7fb435800640 1 Processor -- start 2024-04-20T11:33:04.467 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.468+0000 7fb435800640 1 -- start start 2024-04-20T11:33:04.469 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.469+0000 7fb435800640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fb430162bd0 0x7fb43006f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:04.469 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.469+0000 7fb435800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fb43006c4b0 0x7fb43016c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:04.469 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.469+0000 7fb435800640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4300b46a0 con 0x7fb4300b4890 2024-04-20T11:33:04.469 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.469+0000 7fb435800640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fb43006e870 con 0x7fb43006c4b0 2024-04-20T11:33:04.469 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.469+0000 7fb435800640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fb43006dc60 con 0x7fb430162bd0 2024-04-20T11:33:04.469 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.469+0000 7fb434e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fb43006c4b0 0x7fb43016c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:04.469 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.469+0000 7fb427e00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fb4300b4890 0x7fb43006bda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:33604/0 (socket says 172.21.15.152:33604) 2024-04-20T11:33:04.469 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.469+0000 7fb427e00640 1 -- 172.21.15.152:0/3970279438 learned_addr learned my addr 172.21.15.152:0/3970279438 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:33:04.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.470+0000 7fb42fe00640 1 --2- 172.21.15.152:0/3970279438 >> v2:172.21.15.175:3300/0 conn(0x7fb430162bd0 0x7fb43006f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:04.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.470+0000 7fb434e00640 1 -- 172.21.15.152:0/3970279438 >> v2:172.21.15.175:3300/0 conn(0x7fb430162bd0 msgr2=0x7fb43006f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:04.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.470+0000 7fb434e00640 1 --2- 172.21.15.152:0/3970279438 >> v2:172.21.15.175:3300/0 conn(0x7fb430162bd0 0x7fb43006f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:04.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.470+0000 7fb434e00640 1 -- 172.21.15.152:0/3970279438 >> v1:172.21.15.152:6789/0 conn(0x7fb4300b4890 legacy=0x7fb43006bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:04.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.470+0000 7fb434e00640 1 -- 172.21.15.152:0/3970279438 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb43006df40 con 0x7fb43006c4b0 2024-04-20T11:33:04.471 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.471+0000 7fb434e00640 1 --2- 172.21.15.152:0/3970279438 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb43006c4b0 0x7fb43016c7e0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7fb4300b2d00 tx=0x7fb42802bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6ca656c286b43ad server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:04.471 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.471+0000 7fb42f400640 1 -- 172.21.15.152:0/3970279438 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb428039070 con 0x7fb43006c4b0 2024-04-20T11:33:04.471 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.471+0000 7fb42f400640 1 -- 172.21.15.152:0/3970279438 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb42802cc70 con 0x7fb43006c4b0 2024-04-20T11:33:04.471 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.471+0000 7fb42f400640 1 -- 172.21.15.152:0/3970279438 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb428034660 con 0x7fb43006c4b0 2024-04-20T11:33:04.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.472+0000 7fb435800640 1 -- 172.21.15.152:0/3970279438 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb43006c4b0 msgr2=0x7fb43016c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:04.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.472+0000 7fb435800640 1 --2- 172.21.15.152:0/3970279438 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb43006c4b0 0x7fb43016c7e0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7fb4300b2d00 tx=0x7fb42802bec0 comp rx=0 tx=0).stop 2024-04-20T11:33:04.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.472+0000 7fb435800640 1 -- 172.21.15.152:0/3970279438 shutdown_connections 2024-04-20T11:33:04.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.472+0000 7fb435800640 1 --2- 172.21.15.152:0/3970279438 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb43006c4b0 0x7fb43016c7e0 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:04.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.472+0000 7fb435800640 1 --2- 172.21.15.152:0/3970279438 >> v2:172.21.15.175:3300/0 conn(0x7fb430162bd0 0x7fb43006f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:04.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.472+0000 7fb435800640 1 -- 172.21.15.152:0/3970279438 >> 172.21.15.152:0/3970279438 conn(0x7fb4300b21e0 msgr2=0x7fb4300b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:04.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.472+0000 7fb435800640 1 -- 172.21.15.152:0/3970279438 shutdown_connections 2024-04-20T11:33:04.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.472+0000 7fb435800640 1 -- 172.21.15.152:0/3970279438 wait complete. 2024-04-20T11:33:04.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.474+0000 7fb435800640 1 Processor -- start 2024-04-20T11:33:04.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.475+0000 7fb435800640 1 -- start start 2024-04-20T11:33:04.476 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.476+0000 7fb435800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb43006c4b0 0x7fb43014d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:04.476 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.476+0000 7fb435800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4300b4890 0x7fb43015cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:04.476 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.476+0000 7fb435800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb430162bd0 0x7fb43015d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:04.476 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.476+0000 7fb435800640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fb430162fb0 con 0x7fb43006c4b0 2024-04-20T11:33:04.476 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.476+0000 7fb435800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb43006e500 con 0x7fb4300b4890 2024-04-20T11:33:04.476 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.476+0000 7fb435800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb43006f950 con 0x7fb430162bd0 2024-04-20T11:33:04.476 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.476+0000 7fb42fe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb43006c4b0 0x7fb43014d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:04.476 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.476+0000 7fb42fe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb43006c4b0 0x7fb43014d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:33060/0 (socket says 172.21.15.152:33060) 2024-04-20T11:33:04.476 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.476+0000 7fb42fe00640 1 -- 172.21.15.152:0/1856757949 learned_addr learned my addr 172.21.15.152:0/1856757949 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:33:04.476 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.476+0000 7fb427e00640 1 --2- 172.21.15.152:0/1856757949 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4300b4890 0x7fb43015cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:04.476 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.476+0000 7fb42fe00640 1 -- 172.21.15.152:0/1856757949 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb430162bd0 msgr2=0x7fb43015d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:04.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.476+0000 7fb42fe00640 1 --2- 172.21.15.152:0/1856757949 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb430162bd0 0x7fb43015d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:04.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.476+0000 7fb42fe00640 1 -- 172.21.15.152:0/1856757949 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4300b4890 msgr2=0x7fb43015cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:04.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.476+0000 7fb42fe00640 1 --2- 172.21.15.152:0/1856757949 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4300b4890 0x7fb43015cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:04.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.476+0000 7fb42fe00640 1 -- 172.21.15.152:0/1856757949 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb43015eb20 con 0x7fb43006c4b0 2024-04-20T11:33:04.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.476+0000 7fb427e00640 1 --2- 172.21.15.152:0/1856757949 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4300b4890 0x7fb43015cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:33:04.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.477+0000 7fb42fe00640 1 --2- 172.21.15.152:0/1856757949 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb43006c4b0 0x7fb43014d630 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7fb42000d620 tx=0x7fb42000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:04.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.477+0000 7fb42cc00640 1 -- 172.21.15.152:0/1856757949 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb42000b620 con 0x7fb43006c4b0 2024-04-20T11:33:04.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.477+0000 7fb42cc00640 1 -- 172.21.15.152:0/1856757949 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb420019070 con 0x7fb43006c4b0 2024-04-20T11:33:04.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.477+0000 7fb42cc00640 1 -- 172.21.15.152:0/1856757949 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb42000ba00 con 0x7fb43006c4b0 2024-04-20T11:33:04.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.477+0000 7fb435800640 1 -- 172.21.15.152:0/1856757949 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb43015ee00 con 0x7fb43006c4b0 2024-04-20T11:33:04.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.477+0000 7fb435800640 1 -- 172.21.15.152:0/1856757949 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb43015f3c0 con 0x7fb43006c4b0 2024-04-20T11:33:04.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.479+0000 7fb435800640 1 -- 172.21.15.152:0/1856757949 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb430075e60 con 0x7fb43006c4b0 2024-04-20T11:33:04.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.480+0000 7fb42cc00640 1 -- 172.21.15.152:0/1856757949 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 91142+0+0 (secure 0 0 0) 0x7fb42000bba0 con 0x7fb43006c4b0 2024-04-20T11:33:04.480 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.481+0000 7fb42cc00640 1 --2- 172.21.15.152:0/1856757949 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb40c06ca10 0x7fb40c06eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:04.480 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.481+0000 7fb42cc00640 1 -- 172.21.15.152:0/1856757949 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5795+0+0 (secure 0 0 0) 0x7fb42008c130 con 0x7fb43006c4b0 2024-04-20T11:33:04.480 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.481+0000 7fb427e00640 1 --2- 172.21.15.152:0/1856757949 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb40c06ca10 0x7fb40c06eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:04.482 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.482+0000 7fb427e00640 1 --2- 172.21.15.152:0/1856757949 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb40c06ca10 0x7fb40c06eed0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb4301fc400 tx=0x7fb418009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:04.484 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.485+0000 7fb42cc00640 1 -- 172.21.15.152:0/1856757949 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fb42005a3e0 con 0x7fb43006c4b0 2024-04-20T11:33:04.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.688+0000 7fb435800640 1 -- 172.21.15.152:0/1856757949 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb430072950 con 0x7fb43006c4b0 2024-04-20T11:33:04.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.689+0000 7fb42cc00640 1 -- 172.21.15.152:0/1856757949 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) ==== 100+0+11 (secure 0 0 0) 0x7fb420059ce0 con 0x7fb43006c4b0 2024-04-20T11:33:04.689 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 64 2024-04-20T11:33:04.692 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.692+0000 7fb435800640 1 -- 172.21.15.152:0/1856757949 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb40c06ca10 msgr2=0x7fb40c06eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:04.692 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.692+0000 7fb435800640 1 --2- 172.21.15.152:0/1856757949 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb40c06ca10 0x7fb40c06eed0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb4301fc400 tx=0x7fb418009040 comp rx=0 tx=0).stop 2024-04-20T11:33:04.692 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.692+0000 7fb435800640 1 -- 172.21.15.152:0/1856757949 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb43006c4b0 msgr2=0x7fb43014d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:04.692 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.692+0000 7fb435800640 1 --2- 172.21.15.152:0/1856757949 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb43006c4b0 0x7fb43014d630 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7fb42000d620 tx=0x7fb42000dae0 comp rx=0 tx=0).stop 2024-04-20T11:33:04.692 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.692+0000 7fb435800640 1 -- 172.21.15.152:0/1856757949 shutdown_connections 2024-04-20T11:33:04.692 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.692+0000 7fb435800640 1 --2- 172.21.15.152:0/1856757949 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb40c06ca10 0x7fb40c06eed0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:04.692 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.692+0000 7fb435800640 1 --2- 172.21.15.152:0/1856757949 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb430162bd0 0x7fb43015d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:04.692 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.692+0000 7fb435800640 1 --2- 172.21.15.152:0/1856757949 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4300b4890 0x7fb43015cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:04.692 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.692+0000 7fb435800640 1 --2- 172.21.15.152:0/1856757949 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb43006c4b0 0x7fb43014d630 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:04.692 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.692+0000 7fb435800640 1 -- 172.21.15.152:0/1856757949 >> 172.21.15.152:0/1856757949 conn(0x7fb4300b21e0 msgr2=0x7fb430071d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:04.692 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.692+0000 7fb435800640 1 -- 172.21.15.152:0/1856757949 shutdown_connections 2024-04-20T11:33:04.692 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.692+0000 7fb435800640 1 -- 172.21.15.152:0/1856757949 wait complete. 2024-04-20T11:33:04.710 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-20T11:33:04.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.918+0000 7f1cc2400640 1 Processor -- start 2024-04-20T11:33:04.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.918+0000 7f1cc2400640 1 -- start start 2024-04-20T11:33:04.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.920+0000 7f1cc2400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1cbc145f00 0x7f1cbc0cd0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:04.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.920+0000 7f1cc2400640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f1cbc0cd610 0x7f1cbc06e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:04.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.920+0000 7f1cc2400640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1cbc06f110 con 0x7f1cbc06e900 2024-04-20T11:33:04.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.920+0000 7f1cc2400640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f1cbc0cf9f0 con 0x7f1cbc0cd610 2024-04-20T11:33:04.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.920+0000 7f1cc2400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1cbc0cda50 con 0x7f1cbc145f00 2024-04-20T11:33:04.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.921+0000 7f1cbb400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1cbc145f00 0x7f1cbc0cd0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:04.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.921+0000 7f1cbbe00640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7f1cbc06e900 0x7f1cbc1601c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.152:51222/0 (socket says 172.21.15.152:51222) 2024-04-20T11:33:04.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.921+0000 7f1cb3e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f1cbc0cd610 0x7f1cbc06e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:04.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.921+0000 7f1cbb400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1cbc145f00 0x7f1cbc0cd0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.152:53494/0 (socket says 172.21.15.152:53494) 2024-04-20T11:33:04.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.921+0000 7f1cbbe00640 1 -- 172.21.15.152:0/2428253657 learned_addr learned my addr 172.21.15.152:0/2428253657 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:33:04.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.922+0000 7f1cbb400640 1 -- 172.21.15.152:0/2428253657 >> v2:172.21.15.175:3300/0 conn(0x7f1cbc0cd610 msgr2=0x7f1cbc06e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:04.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.922+0000 7f1cbb400640 1 --2- 172.21.15.152:0/2428253657 >> v2:172.21.15.175:3300/0 conn(0x7f1cbc0cd610 0x7f1cbc06e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:04.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.922+0000 7f1cbb400640 1 -- 172.21.15.152:0/2428253657 >> v1:172.21.15.175:6789/0 conn(0x7f1cbc06e900 legacy=0x7f1cbc1601c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:04.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.922+0000 7f1cbb400640 1 -- 172.21.15.152:0/2428253657 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cbc06f3f0 con 0x7f1cbc145f00 2024-04-20T11:33:04.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.923+0000 7f1cbb400640 1 --2- 172.21.15.152:0/2428253657 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1cbc145f00 0x7f1cbc0cd0d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1cac0097a0 tx=0x7f1cac02b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b60483caece005 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:04.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.924+0000 7f1cbaa00640 1 -- 172.21.15.152:0/2428253657 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cac035020 con 0x7f1cbc145f00 2024-04-20T11:33:04.925 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.924+0000 7f1cbaa00640 1 -- 172.21.15.152:0/2428253657 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cac031810 con 0x7f1cbc145f00 2024-04-20T11:33:04.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.924+0000 7f1cbaa00640 1 -- 172.21.15.152:0/2428253657 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cac031be0 con 0x7f1cbc145f00 2024-04-20T11:33:04.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.925+0000 7f1cc2400640 1 -- 172.21.15.152:0/2428253657 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1cbc145f00 msgr2=0x7f1cbc0cd0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:04.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.925+0000 7f1cc2400640 1 --2- 172.21.15.152:0/2428253657 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1cbc145f00 0x7f1cbc0cd0d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1cac0097a0 tx=0x7f1cac02b8b0 comp rx=0 tx=0).stop 2024-04-20T11:33:04.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.925+0000 7f1cc2400640 1 -- 172.21.15.152:0/2428253657 shutdown_connections 2024-04-20T11:33:04.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.925+0000 7f1cc2400640 1 --2- 172.21.15.152:0/2428253657 >> v2:172.21.15.175:3300/0 conn(0x7f1cbc0cd610 0x7f1cbc06e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:04.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.925+0000 7f1cc2400640 1 --2- 172.21.15.152:0/2428253657 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1cbc145f00 0x7f1cbc0cd0d0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:04.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.925+0000 7f1cc2400640 1 -- 172.21.15.152:0/2428253657 >> 172.21.15.152:0/2428253657 conn(0x7f1cbc067cf0 msgr2=0x7f1cbc06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:04.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.925+0000 7f1cc2400640 1 -- 172.21.15.152:0/2428253657 shutdown_connections 2024-04-20T11:33:04.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.925+0000 7f1cc2400640 1 -- 172.21.15.152:0/2428253657 wait complete. 2024-04-20T11:33:04.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.927+0000 7f1cc2400640 1 Processor -- start 2024-04-20T11:33:04.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.928+0000 7f1cc2400640 1 -- start start 2024-04-20T11:33:04.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.929+0000 7f1cc2400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1cbc06e900 0x7f1cbc143c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:04.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.929+0000 7f1cc2400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1cbc0cd610 0x7f1cbc1f87e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:04.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.929+0000 7f1cc2400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1cbc145f00 0x7f1cbc13fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:04.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.929+0000 7f1cc2400640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f1cbc1462e0 con 0x7f1cbc0cd610 2024-04-20T11:33:04.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.929+0000 7f1cc2400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1cbc0cf680 con 0x7f1cbc06e900 2024-04-20T11:33:04.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.929+0000 7f1cc2400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1cbc06ed90 con 0x7f1cbc145f00 2024-04-20T11:33:04.931 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.929+0000 7f1cb3e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1cbc0cd610 0x7f1cbc1f87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:04.931 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.929+0000 7f1cb3e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1cbc0cd610 0x7f1cbc1f87e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:33062/0 (socket says 172.21.15.152:33062) 2024-04-20T11:33:04.931 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.929+0000 7f1cb3e00640 1 -- 172.21.15.152:0/2867150026 learned_addr learned my addr 172.21.15.152:0/2867150026 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:33:04.931 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.929+0000 7f1cbb400640 1 --2- 172.21.15.152:0/2867150026 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1cbc06e900 0x7f1cbc143c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:04.931 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.929+0000 7f1cbbe00640 1 --2- 172.21.15.152:0/2867150026 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1cbc145f00 0x7f1cbc13fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:04.931 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.930+0000 7f1cb3e00640 1 -- 172.21.15.152:0/2867150026 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1cbc145f00 msgr2=0x7f1cbc13fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:04.931 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.930+0000 7f1cb3e00640 1 --2- 172.21.15.152:0/2867150026 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1cbc145f00 0x7f1cbc13fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:04.931 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.930+0000 7f1cb3e00640 1 -- 172.21.15.152:0/2867150026 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1cbc06e900 msgr2=0x7f1cbc143c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:04.931 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.930+0000 7f1cb3e00640 1 --2- 172.21.15.152:0/2867150026 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1cbc06e900 0x7f1cbc143c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:04.931 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.930+0000 7f1cb3e00640 1 -- 172.21.15.152:0/2867150026 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cbc144b70 con 0x7f1cbc0cd610 2024-04-20T11:33:04.931 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.930+0000 7f1cb3e00640 1 --2- 172.21.15.152:0/2867150026 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1cbc0cd610 0x7f1cbc1f87e0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f1ca400e9b0 tx=0x7f1ca400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:04.931 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.930+0000 7f1cb3400640 1 -- 172.21.15.152:0/2867150026 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca4017070 con 0x7f1cbc0cd610 2024-04-20T11:33:04.931 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.930+0000 7f1cb3400640 1 -- 172.21.15.152:0/2867150026 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ca400c9a0 con 0x7f1cbc0cd610 2024-04-20T11:33:04.931 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.930+0000 7f1cc2400640 1 -- 172.21.15.152:0/2867150026 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cbc1442d0 con 0x7f1cbc0cd610 2024-04-20T11:33:04.932 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.930+0000 7f1cb3400640 1 -- 172.21.15.152:0/2867150026 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca400cd90 con 0x7f1cbc0cd610 2024-04-20T11:33:04.932 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.930+0000 7f1cc2400640 1 -- 172.21.15.152:0/2867150026 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cbc0af400 con 0x7f1cbc0cd610 2024-04-20T11:33:04.932 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.931+0000 7f1cc2400640 1 -- 172.21.15.152:0/2867150026 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cbc0689f0 con 0x7f1cbc0cd610 2024-04-20T11:33:04.932 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.933+0000 7f1cb3400640 1 -- 172.21.15.152:0/2867150026 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 91142+0+0 (secure 0 0 0) 0x7f1ca401a070 con 0x7f1cbc0cd610 2024-04-20T11:33:04.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.934+0000 7f1cb3400640 1 --2- 172.21.15.152:0/2867150026 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f1c9806ca10 0x7f1c9806eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:04.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.934+0000 7f1cb3400640 1 -- 172.21.15.152:0/2867150026 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5795+0+0 (secure 0 0 0) 0x7f1ca408bdd0 con 0x7f1cbc0cd610 2024-04-20T11:33:04.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.934+0000 7f1cbb400640 1 --2- 172.21.15.152:0/2867150026 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f1c9806ca10 0x7f1c9806eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:04.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.935+0000 7f1cbb400640 1 --2- 172.21.15.152:0/2867150026 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f1c9806ca10 0x7f1c9806eed0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1cac036000 tx=0x7f1cac03c040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:04.938 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:04.939+0000 7f1cb3400640 1 -- 172.21.15.152:0/2867150026 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f1ca4059ff0 con 0x7f1cbc0cd610 2024-04-20T11:33:05.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:05.153+0000 7f1cc2400640 1 -- 172.21.15.152:0/2867150026 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f1cbc0d0aa0 con 0x7f1cbc0cd610 2024-04-20T11:33:05.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:05.154+0000 7f1cb3400640 1 -- 172.21.15.152:0/2867150026 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v91) ==== 80+0+124 (secure 0 0 0) 0x7f1ca40598f0 con 0x7f1cbc0cd610 2024-04-20T11:33:05.153 INFO:teuthology.orchestra.run.smithi152.stdout:172.21.15.175:6835/2632275131 2024-04-21T11:32:59.571722+0000 2024-04-20T11:33:05.153 INFO:teuthology.orchestra.run.smithi152.stdout:172.21.15.175:6834/2632275131 2024-04-21T11:32:59.571722+0000 2024-04-20T11:33:05.154 INFO:teuthology.orchestra.run.smithi152.stderr:listed 2 entries 2024-04-20T11:33:05.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:05.156+0000 7f1cc2400640 1 -- 172.21.15.152:0/2867150026 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f1c9806ca10 msgr2=0x7f1c9806eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:05.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:05.156+0000 7f1cc2400640 1 --2- 172.21.15.152:0/2867150026 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f1c9806ca10 0x7f1c9806eed0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1cac036000 tx=0x7f1cac03c040 comp rx=0 tx=0).stop 2024-04-20T11:33:05.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:05.156+0000 7f1cc2400640 1 -- 172.21.15.152:0/2867150026 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1cbc0cd610 msgr2=0x7f1cbc1f87e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:05.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:05.156+0000 7f1cc2400640 1 --2- 172.21.15.152:0/2867150026 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1cbc0cd610 0x7f1cbc1f87e0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f1ca400e9b0 tx=0x7f1ca400ee70 comp rx=0 tx=0).stop 2024-04-20T11:33:05.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:05.157+0000 7f1cc2400640 1 -- 172.21.15.152:0/2867150026 shutdown_connections 2024-04-20T11:33:05.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:05.157+0000 7f1cc2400640 1 --2- 172.21.15.152:0/2867150026 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f1c9806ca10 0x7f1c9806eed0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:05.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:05.157+0000 7f1cc2400640 1 --2- 172.21.15.152:0/2867150026 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1cbc145f00 0x7f1cbc13fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:05.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:05.157+0000 7f1cc2400640 1 --2- 172.21.15.152:0/2867150026 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1cbc0cd610 0x7f1cbc1f87e0 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:05.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:05.157+0000 7f1cc2400640 1 --2- 172.21.15.152:0/2867150026 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1cbc06e900 0x7f1cbc143c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:05.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:05.157+0000 7f1cc2400640 1 -- 172.21.15.152:0/2867150026 >> 172.21.15.152:0/2867150026 conn(0x7f1cbc067cf0 msgr2=0x7f1cbc0b0dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:05.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:05.157+0000 7f1cc2400640 1 -- 172.21.15.152:0/2867150026 shutdown_connections 2024-04-20T11:33:05.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:05.157+0000 7f1cc2400640 1 -- 172.21.15.152:0/2867150026 wait complete. 2024-04-20T11:33:05.177 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi152.front.sepia.ceph.com... 2024-04-20T11:33:05.177 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:05.177 DEBUG:teuthology.orchestra.run.smithi152:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-20T11:33:05.254 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi152.stderr:ceph-fuse[51248]: fuse finished with error 0 and tester_r 0 2024-04-20T11:33:05.259 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi152.front.sepia.ceph.com 2024-04-20T11:33:05.260 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:05.260 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-20T11:33:05.435 INFO:tasks.cephfs.mount:Terminating background process 2024-04-20T11:33:05.435 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T11:33:05.435 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:33:05.491 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:33:05.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:05.702+0000 7f5705a00640 1 Processor -- start 2024-04-20T11:33:05.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:05.702+0000 7f5705a00640 1 -- start start 2024-04-20T11:33:05.703 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:05.704+0000 7f5705a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f57000726f0 0x7f57000cd580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:05.703 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:05.704+0000 7f5705a00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5700146340 con 0x7f57000cd0d0 2024-04-20T11:33:05.703 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:05.704+0000 7f5705a00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5700072d50 con 0x7f5700145f60 2024-04-20T11:33:05.704 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:05.704+0000 7f5705a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f570006e9f0 con 0x7f57000726f0 2024-04-20T11:33:05.704 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:05.704+0000 7f56fe000640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f57000cd0d0 0x7f5700071fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:33606/0 (socket says 172.21.15.152:33606) 2024-04-20T11:33:05.704 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:05.704+0000 7f56fe000640 1 -- 172.21.15.152:0/2640985889 learned_addr learned my addr 172.21.15.152:0/2640985889 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:33:05.706 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:05.705+0000 7f56ff400640 1 --2- 172.21.15.152:0/2640985889 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f57000726f0 0x7f57000cd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:05.706 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:05.705+0000 7f56fd600640 1 -- 172.21.15.152:0/2640985889 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1480351682 0 0) 0x7f5700146340 con 0x7f57000cd0d0 2024-04-20T11:33:05.706 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:05.705+0000 7f56fd600640 1 -- 172.21.15.152:0/2640985889 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f56e4003520 con 0x7f57000cd0d0 2024-04-20T11:33:05.706 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:05.705+0000 7f56fd600640 1 -- 172.21.15.152:0/2640985889 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3780046865 0 0) 0x7f56e4003520 con 0x7f57000cd0d0 2024-04-20T11:33:05.706 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:05.705+0000 7f56fd600640 1 -- 172.21.15.152:0/2640985889 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f57000726f0 msgr2=0x7f57000cd580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:05.706 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:05.705+0000 7f56fd600640 1 --2- 172.21.15.152:0/2640985889 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f57000726f0 0x7f57000cd580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:05.706 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:05.705+0000 7f56fd600640 1 -- 172.21.15.152:0/2640985889 >> v1:172.21.15.175:6789/0 conn(0x7f5700145f60 legacy=0x7f570006ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:05.706 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:05.705+0000 7f56fd600640 1 -- 172.21.15.152:0/2640985889 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f570006e0c0 con 0x7f57000cd0d0 2024-04-20T11:33:05.706 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:05.705+0000 7f56fd600640 1 -- 172.21.15.152:0/2640985889 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f56ec001690 con 0x7f57000cd0d0 2024-04-20T11:33:05.706 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:05.705+0000 7f56fd600640 1 -- 172.21.15.152:0/2640985889 >> v1:172.21.15.152:6789/0 conn(0x7f57000cd0d0 legacy=0x7f5700071fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:05.706 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:05.705+0000 7f56fd600640 1 --2- 172.21.15.152:0/2640985889 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f57000726f0 0x7f56e4003a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:05.706 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:05.705+0000 7f56fd600640 1 --2- 172.21.15.152:0/2640985889 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f56e4003fd0 0x7f56e4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:05.707 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:05.705+0000 7f56fd600640 1 --2- 172.21.15.152:0/2640985889 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f56e4004d40 0x7f56e40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:05.707 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:05.705+0000 7f56fd600640 1 -- 172.21.15.152:0/2640985889 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f570006e9f0 con 0x7f56e4003fd0 2024-04-20T11:33:05.707 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:05.705+0000 7f56fd600640 1 -- 172.21.15.152:0/2640985889 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f56e400bb20 con 0x7f56e4004d40 2024-04-20T11:33:05.707 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:05.705+0000 7f56fd600640 1 -- 172.21.15.152:0/2640985889 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f56e400bd20 con 0x7f57000726f0 2024-04-20T11:33:05.707 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:05.705+0000 7f56fe000640 1 --2- 172.21.15.152:0/2640985889 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f56e4003fd0 0x7f56e4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:05.707 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:05.706+0000 7f56fea00640 1 --2- 172.21.15.152:0/2640985889 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f57000726f0 0x7f56e4003a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:05.707 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:05.706+0000 7f56ff400640 1 --2- 172.21.15.152:0/2640985889 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f56e4004d40 0x7f56e40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:05.707 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:05.706+0000 7f56fe000640 1 -- 172.21.15.152:0/2640985889 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f57000726f0 msgr2=0x7f56e4003a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:05.707 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:05.706+0000 7f56fe000640 1 --2- 172.21.15.152:0/2640985889 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f57000726f0 0x7f56e4003a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:05.707 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:05.706+0000 7f56fe000640 1 -- 172.21.15.152:0/2640985889 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f56e4004d40 msgr2=0x7f56e40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:05.707 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:05.706+0000 7f56fe000640 1 --2- 172.21.15.152:0/2640985889 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f56e4004d40 0x7f56e40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:05.707 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:05.706+0000 7f56fe000640 1 -- 172.21.15.152:0/2640985889 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56e4003520 con 0x7f56e4003fd0 2024-04-20T11:33:05.707 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:05.706+0000 7f56ff400640 1 --2- 172.21.15.152:0/2640985889 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f56e4004d40 0x7f56e40051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:33:05.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:05.707+0000 7f56fe000640 1 --2- 172.21.15.152:0/2640985889 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f56e4003fd0 0x7f56e4004800 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f56ec007fd0 tx=0x7f56e4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fefedf60a1ff0cbd server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:05.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:05.707+0000 7f56fd600640 1 -- 172.21.15.152:0/2640985889 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56ec007d50 con 0x7f56e4003fd0 2024-04-20T11:33:05.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:05.707+0000 7f56fd600640 1 -- 172.21.15.152:0/2640985889 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56ec030070 con 0x7f56e4003fd0 2024-04-20T11:33:05.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:05.707+0000 7f56fd600640 1 -- 172.21.15.152:0/2640985889 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56ec005d30 con 0x7f56e4003fd0 2024-04-20T11:33:05.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:05.708+0000 7f5705a00640 1 -- 172.21.15.152:0/2640985889 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f56e4003fd0 msgr2=0x7f56e4004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:05.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:05.708+0000 7f5705a00640 1 --2- 172.21.15.152:0/2640985889 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f56e4003fd0 0x7f56e4004800 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f56ec007fd0 tx=0x7f56e4005830 comp rx=0 tx=0).stop 2024-04-20T11:33:05.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:05.708+0000 7f56ff400640 1 -- 172.21.15.152:0/2640985889 reap_dead start 2024-04-20T11:33:05.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:05.708+0000 7f5705a00640 1 -- 172.21.15.152:0/2640985889 shutdown_connections 2024-04-20T11:33:05.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:05.708+0000 7f5705a00640 1 -- 172.21.15.152:0/2640985889 >> 172.21.15.152:0/2640985889 conn(0x7f5700067cf0 msgr2=0x7f570006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:05.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:05.708+0000 7f5705a00640 1 -- 172.21.15.152:0/2640985889 shutdown_connections 2024-04-20T11:33:05.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:05.708+0000 7f5705a00640 1 -- 172.21.15.152:0/2640985889 wait complete. 2024-04-20T11:33:05.710 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:05.710+0000 7f5705a00640 1 Processor -- start 2024-04-20T11:33:05.710 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:05.710+0000 7f5705a00640 1 -- start start 2024-04-20T11:33:05.711 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:05.711+0000 7f5705a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f56e4003fd0 0x7f57000aa1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:05.711 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:05.711+0000 7f5705a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f57001f2b40 0x7f57000cbb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:05.711 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:05.711+0000 7f5705a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f57000acba0 0x7f57000ad050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:05.711 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:05.711+0000 7f5705a00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f57000cd580 con 0x7f56e4003fd0 2024-04-20T11:33:05.711 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:05.711+0000 7f5705a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f570006e680 con 0x7f57000acba0 2024-04-20T11:33:05.711 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:05.711+0000 7f5705a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f57000cd700 con 0x7f57001f2b40 2024-04-20T11:33:05.711 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:05.711+0000 7f56fea00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f56e4003fd0 0x7f57000aa1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:05.711 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:05.711+0000 7f56fea00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f56e4003fd0 0x7f57000aa1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:33086/0 (socket says 172.21.15.152:33086) 2024-04-20T11:33:05.711 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:05.711+0000 7f56fea00640 1 -- 172.21.15.152:0/651901511 learned_addr learned my addr 172.21.15.152:0/651901511 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:33:05.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:05.712+0000 7f56fea00640 1 -- 172.21.15.152:0/651901511 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f57001f2b40 msgr2=0x7f57000cbb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:05.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:05.712+0000 7f56fea00640 1 --2- 172.21.15.152:0/651901511 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f57001f2b40 0x7f57000cbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:05.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:05.712+0000 7f56fea00640 1 -- 172.21.15.152:0/651901511 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f57000acba0 msgr2=0x7f57000ad050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:05.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:05.712+0000 7f56fea00640 1 --2- 172.21.15.152:0/651901511 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f57000acba0 0x7f57000ad050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:05.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:05.712+0000 7f56fea00640 1 -- 172.21.15.152:0/651901511 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57000aaad0 con 0x7f56e4003fd0 2024-04-20T11:33:05.713 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:05.712+0000 7f56fea00640 1 --2- 172.21.15.152:0/651901511 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f56e4003fd0 0x7f57000aa1f0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f56f000e550 tx=0x7f56f000ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:05.713 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:05.712+0000 7f56f6a00640 1 -- 172.21.15.152:0/651901511 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56f00052e0 con 0x7f56e4003fd0 2024-04-20T11:33:05.713 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:05.712+0000 7f56f6a00640 1 -- 172.21.15.152:0/651901511 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56f0018070 con 0x7f56e4003fd0 2024-04-20T11:33:05.713 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:05.712+0000 7f5705a00640 1 -- 172.21.15.152:0/651901511 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57000aadb0 con 0x7f56e4003fd0 2024-04-20T11:33:05.713 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:05.712+0000 7f56f6a00640 1 -- 172.21.15.152:0/651901511 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56f0003810 con 0x7f56e4003fd0 2024-04-20T11:33:05.713 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:05.712+0000 7f5705a00640 1 -- 172.21.15.152:0/651901511 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57000ab370 con 0x7f56e4003fd0 2024-04-20T11:33:05.713 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:05.713+0000 7f5705a00640 1 -- 172.21.15.152:0/651901511 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57000689f0 con 0x7f56e4003fd0 2024-04-20T11:33:05.714 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:05.714+0000 7f56f6a00640 1 -- 172.21.15.152:0/651901511 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 91142+0+0 (secure 0 0 0) 0x7f56f00039b0 con 0x7f56e4003fd0 2024-04-20T11:33:05.715 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:05.715+0000 7f56f6a00640 1 --2- 172.21.15.152:0/651901511 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f56e006ca10 0x7f56e006eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:05.715 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:05.715+0000 7f56f6a00640 1 -- 172.21.15.152:0/651901511 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5795+0+0 (secure 0 0 0) 0x7f56f008ba50 con 0x7f56e4003fd0 2024-04-20T11:33:05.715 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:05.716+0000 7f56fe000640 1 --2- 172.21.15.152:0/651901511 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f56e006ca10 0x7f56e006eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:05.717 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:05.717+0000 7f56fe000640 1 --2- 172.21.15.152:0/651901511 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f56e006ca10 0x7f56e006eed0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f56ec034000 tx=0x7f56ec007000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:05.719 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:05.720+0000 7f56f6a00640 1 -- 172.21.15.152:0/651901511 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f56f0059c70 con 0x7f56e4003fd0 2024-04-20T11:33:05.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:05.922+0000 7f5705a00640 1 -- 172.21.15.152:0/651901511 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f57000008d0 con 0x7f56e4003fd0 2024-04-20T11:33:05.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:05.924+0000 7f56f6a00640 1 -- 172.21.15.152:0/651901511 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+16851 (secure 0 0 0) 0x7f56f0059570 con 0x7f56e4003fd0 2024-04-20T11:33:05.924 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:33:05.924 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":91,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:33:02.588760+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:32:39.268411+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-20T11:32:40.497900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.175:6835/2632275131":"2024-04-21T11:32:59.571722+0000","172.21.15.175:6834/2632275131":"2024-04-21T11:32:59.571722+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:33:05.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:05.927+0000 7f5705a00640 1 -- 172.21.15.152:0/651901511 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f56e006ca10 msgr2=0x7f56e006eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:05.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:05.927+0000 7f5705a00640 1 --2- 172.21.15.152:0/651901511 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f56e006ca10 0x7f56e006eed0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f56ec034000 tx=0x7f56ec007000 comp rx=0 tx=0).stop 2024-04-20T11:33:05.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:05.927+0000 7f5705a00640 1 -- 172.21.15.152:0/651901511 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f56e4003fd0 msgr2=0x7f57000aa1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:05.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:05.927+0000 7f5705a00640 1 --2- 172.21.15.152:0/651901511 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f56e4003fd0 0x7f57000aa1f0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f56f000e550 tx=0x7f56f000ea10 comp rx=0 tx=0).stop 2024-04-20T11:33:05.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:05.927+0000 7f5705a00640 1 -- 172.21.15.152:0/651901511 shutdown_connections 2024-04-20T11:33:05.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:05.927+0000 7f5705a00640 1 --2- 172.21.15.152:0/651901511 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f56e006ca10 0x7f56e006eed0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:05.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:05.927+0000 7f5705a00640 1 --2- 172.21.15.152:0/651901511 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f57000acba0 0x7f57000ad050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:05.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:05.927+0000 7f5705a00640 1 --2- 172.21.15.152:0/651901511 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f57001f2b40 0x7f57000cbb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:05.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:05.927+0000 7f5705a00640 1 --2- 172.21.15.152:0/651901511 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f56e4003fd0 0x7f57000aa1f0 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:05.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:05.927+0000 7f5705a00640 1 -- 172.21.15.152:0/651901511 >> 172.21.15.152:0/651901511 conn(0x7f5700067cf0 msgr2=0x7f570015a8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:05.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:05.928+0000 7f5705a00640 1 -- 172.21.15.152:0/651901511 shutdown_connections 2024-04-20T11:33:05.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:05.928+0000 7f5705a00640 1 -- 172.21.15.152:0/651901511 wait complete. 2024-04-20T11:33:05.947 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T11:26:47.246571+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T11:32:39.268411+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T11:32:40.497900+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T11:33:05.948 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T11:33:06.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.158+0000 7ff464200640 1 Processor -- start 2024-04-20T11:33:06.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.158+0000 7ff464200640 1 -- start start 2024-04-20T11:33:06.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.159+0000 7ff464200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff45c0cd8c0 0x7ff45c0b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:06.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.159+0000 7ff464200640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff45c1463e0 con 0x7ff45c146000 2024-04-20T11:33:06.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.159+0000 7ff464200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff45c0b3e20 con 0x7ff45c0b2880 2024-04-20T11:33:06.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.159+0000 7ff464200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff45c0b4750 con 0x7ff45c0cd8c0 2024-04-20T11:33:06.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.160+0000 7ff461800640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7ff45c146000 0x7ff45c0b2170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:58170/0 (socket says 172.21.15.152:58170) 2024-04-20T11:33:06.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.160+0000 7ff461800640 1 -- 172.21.15.152:0/1929264106 learned_addr learned my addr 172.21.15.152:0/1929264106 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:33:06.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.160+0000 7ff462200640 1 --2- 172.21.15.152:0/1929264106 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff45c0cd8c0 0x7ff45c0b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:06.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.160+0000 7ff45be00640 1 -- 172.21.15.152:0/1929264106 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 107629234 0 0) 0x7ff45c1463e0 con 0x7ff45c146000 2024-04-20T11:33:06.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.160+0000 7ff45be00640 1 -- 172.21.15.152:0/1929264106 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff444003520 con 0x7ff45c146000 2024-04-20T11:33:06.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.161+0000 7ff45be00640 1 -- 172.21.15.152:0/1929264106 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1539271105 0 0) 0x7ff444003520 con 0x7ff45c146000 2024-04-20T11:33:06.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.161+0000 7ff45be00640 1 -- 172.21.15.152:0/1929264106 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff45c0cd8c0 msgr2=0x7ff45c0b4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:06.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.161+0000 7ff45be00640 1 --2- 172.21.15.152:0/1929264106 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff45c0cd8c0 0x7ff45c0b4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:06.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.161+0000 7ff45be00640 1 -- 172.21.15.152:0/1929264106 >> v1:172.21.15.175:6789/0 conn(0x7ff45c0b2880 legacy=0x7ff45c0cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:06.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.161+0000 7ff45be00640 1 -- 172.21.15.152:0/1929264106 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff45c0b4100 con 0x7ff45c146000 2024-04-20T11:33:06.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.161+0000 7ff45be00640 1 -- 172.21.15.152:0/1929264106 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7ff450001690 con 0x7ff45c146000 2024-04-20T11:33:06.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.161+0000 7ff45be00640 1 -- 172.21.15.152:0/1929264106 >> v1:172.21.15.152:6789/0 conn(0x7ff45c146000 legacy=0x7ff45c0b2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:06.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.161+0000 7ff45be00640 1 --2- 172.21.15.152:0/1929264106 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff4440039e0 0x7ff444003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:06.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.161+0000 7ff45be00640 1 --2- 172.21.15.152:0/1929264106 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff444004800 0x7ff444004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:06.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.161+0000 7ff45be00640 1 --2- 172.21.15.152:0/1929264106 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff45c0cd8c0 0x7ff444005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:06.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.161+0000 7ff45be00640 1 -- 172.21.15.152:0/1929264106 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7ff45c0b4750 con 0x7ff4440039e0 2024-04-20T11:33:06.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.161+0000 7ff45be00640 1 -- 172.21.15.152:0/1929264106 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff44400bbb0 con 0x7ff444004800 2024-04-20T11:33:06.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.161+0000 7ff45be00640 1 -- 172.21.15.152:0/1929264106 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff44400bdb0 con 0x7ff45c0cd8c0 2024-04-20T11:33:06.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.161+0000 7ff461800640 1 --2- 172.21.15.152:0/1929264106 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff4440039e0 0x7ff444003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:06.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.162+0000 7ff462200640 1 --2- 172.21.15.152:0/1929264106 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff45c0cd8c0 0x7ff444005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:06.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.162+0000 7ff460e00640 1 --2- 172.21.15.152:0/1929264106 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff444004800 0x7ff444004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:06.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.162+0000 7ff461800640 1 -- 172.21.15.152:0/1929264106 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff45c0cd8c0 msgr2=0x7ff444005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:06.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.162+0000 7ff461800640 1 --2- 172.21.15.152:0/1929264106 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff45c0cd8c0 0x7ff444005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:06.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.162+0000 7ff461800640 1 -- 172.21.15.152:0/1929264106 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff444004800 msgr2=0x7ff444004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:06.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.162+0000 7ff461800640 1 --2- 172.21.15.152:0/1929264106 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff444004800 0x7ff444004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:06.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.162+0000 7ff461800640 1 -- 172.21.15.152:0/1929264106 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff444003520 con 0x7ff4440039e0 2024-04-20T11:33:06.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.162+0000 7ff462200640 1 --2- 172.21.15.152:0/1929264106 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff45c0cd8c0 0x7ff444005220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:33:06.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.162+0000 7ff460e00640 1 --2- 172.21.15.152:0/1929264106 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff444004800 0x7ff444004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:33:06.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.163+0000 7ff461800640 1 --2- 172.21.15.152:0/1929264106 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff4440039e0 0x7ff444003e20 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7ff45000c210 tx=0x7ff444005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40fefb194433a97e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:06.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.163+0000 7ff45be00640 1 -- 172.21.15.152:0/1929264106 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff450007d10 con 0x7ff4440039e0 2024-04-20T11:33:06.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.163+0000 7ff45be00640 1 -- 172.21.15.152:0/1929264106 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff450030070 con 0x7ff4440039e0 2024-04-20T11:33:06.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.163+0000 7ff45be00640 1 -- 172.21.15.152:0/1929264106 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff450005d30 con 0x7ff4440039e0 2024-04-20T11:33:06.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.164+0000 7ff464200640 1 -- 172.21.15.152:0/1929264106 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff4440039e0 msgr2=0x7ff444003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:06.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.164+0000 7ff464200640 1 --2- 172.21.15.152:0/1929264106 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff4440039e0 0x7ff444003e20 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7ff45000c210 tx=0x7ff444005830 comp rx=0 tx=0).stop 2024-04-20T11:33:06.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.164+0000 7ff462200640 1 -- 172.21.15.152:0/1929264106 reap_dead start 2024-04-20T11:33:06.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.164+0000 7ff464200640 1 -- 172.21.15.152:0/1929264106 shutdown_connections 2024-04-20T11:33:06.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.164+0000 7ff464200640 1 -- 172.21.15.152:0/1929264106 >> 172.21.15.152:0/1929264106 conn(0x7ff45c06b980 msgr2=0x7ff45c06bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:06.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.164+0000 7ff464200640 1 -- 172.21.15.152:0/1929264106 shutdown_connections 2024-04-20T11:33:06.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.164+0000 7ff464200640 1 -- 172.21.15.152:0/1929264106 wait complete. 2024-04-20T11:33:06.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.166+0000 7ff464200640 1 Processor -- start 2024-04-20T11:33:06.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.166+0000 7ff464200640 1 -- start start 2024-04-20T11:33:06.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.167+0000 7ff464200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4440039e0 0x7ff45c1f3190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:06.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.167+0000 7ff464200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff45c15bcf0 0x7ff45c141870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:06.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.167+0000 7ff464200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff45c13fe10 0x7ff45c140290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:06.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.167+0000 7ff464200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7ff45c0b4d60 con 0x7ff45c13fe10 2024-04-20T11:33:06.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.167+0000 7ff464200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff45c0b4be0 con 0x7ff4440039e0 2024-04-20T11:33:06.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.167+0000 7ff464200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff45c0b4ee0 con 0x7ff45c15bcf0 2024-04-20T11:33:06.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.167+0000 7ff462200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff45c13fe10 0x7ff45c140290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:06.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.167+0000 7ff462200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff45c13fe10 0x7ff45c140290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36398/0 (socket says 172.21.15.152:36398) 2024-04-20T11:33:06.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.167+0000 7ff462200640 1 -- 172.21.15.152:0/4036862964 learned_addr learned my addr 172.21.15.152:0/4036862964 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:33:06.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.167+0000 7ff460e00640 1 --2- 172.21.15.152:0/4036862964 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff45c15bcf0 0x7ff45c141870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:06.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.167+0000 7ff461800640 1 --2- 172.21.15.152:0/4036862964 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4440039e0 0x7ff45c1f3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:06.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.167+0000 7ff462200640 1 -- 172.21.15.152:0/4036862964 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff45c15bcf0 msgr2=0x7ff45c141870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:06.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.168+0000 7ff462200640 1 --2- 172.21.15.152:0/4036862964 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff45c15bcf0 0x7ff45c141870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:06.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.168+0000 7ff462200640 1 -- 172.21.15.152:0/4036862964 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4440039e0 msgr2=0x7ff45c1f3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:06.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.168+0000 7ff462200640 1 --2- 172.21.15.152:0/4036862964 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4440039e0 0x7ff45c1f3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:06.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.168+0000 7ff462200640 1 -- 172.21.15.152:0/4036862964 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff45c1422f0 con 0x7ff45c13fe10 2024-04-20T11:33:06.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.168+0000 7ff460e00640 1 --2- 172.21.15.152:0/4036862964 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff45c15bcf0 0x7ff45c141870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:33:06.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.168+0000 7ff461800640 1 --2- 172.21.15.152:0/4036862964 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4440039e0 0x7ff45c1f3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:33:06.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.168+0000 7ff462200640 1 --2- 172.21.15.152:0/4036862964 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff45c13fe10 0x7ff45c140290 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7ff45c0700a0 tx=0x7ff45400f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:06.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.168+0000 7ff459600640 1 -- 172.21.15.152:0/4036862964 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff45401c070 con 0x7ff45c13fe10 2024-04-20T11:33:06.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.168+0000 7ff459600640 1 -- 172.21.15.152:0/4036862964 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff454017400 con 0x7ff45c13fe10 2024-04-20T11:33:06.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.168+0000 7ff459600640 1 -- 172.21.15.152:0/4036862964 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4540177d0 con 0x7ff45c13fe10 2024-04-20T11:33:06.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.168+0000 7ff464200640 1 -- 172.21.15.152:0/4036862964 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff45c142010 con 0x7ff45c13fe10 2024-04-20T11:33:06.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.168+0000 7ff464200640 1 -- 172.21.15.152:0/4036862964 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff45c13f610 con 0x7ff45c13fe10 2024-04-20T11:33:06.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.169+0000 7ff464200640 1 -- 172.21.15.152:0/4036862964 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff45c0689f0 con 0x7ff45c13fe10 2024-04-20T11:33:06.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.170+0000 7ff459600640 1 -- 172.21.15.152:0/4036862964 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 91142+0+0 (secure 0 0 0) 0x7ff454005d10 con 0x7ff45c13fe10 2024-04-20T11:33:06.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.171+0000 7ff459600640 1 --2- 172.21.15.152:0/4036862964 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7ff43c06ca40 0x7ff43c06ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:06.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.172+0000 7ff459600640 1 -- 172.21.15.152:0/4036862964 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5795+0+0 (secure 0 0 0) 0x7ff45408bfb0 con 0x7ff45c13fe10 2024-04-20T11:33:06.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.172+0000 7ff461800640 1 --2- 172.21.15.152:0/4036862964 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7ff43c06ca40 0x7ff43c06ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:06.172 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.173+0000 7ff461800640 1 --2- 172.21.15.152:0/4036862964 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7ff43c06ca40 0x7ff43c06ef00 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff450034000 tx=0x7ff450007000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:06.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.176+0000 7ff459600640 1 -- 172.21.15.152:0/4036862964 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ff454012070 con 0x7ff45c13fe10 2024-04-20T11:33:06.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.375+0000 7ff464200640 1 -- 172.21.15.152:0/4036862964 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff45c141650 con 0x7ff45c13fe10 2024-04-20T11:33:06.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.376+0000 7ff459600640 1 -- 172.21.15.152:0/4036862964 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) ==== 93+0+10 (secure 0 0 0) 0x7ff45405a150 con 0x7ff45c13fe10 2024-04-20T11:33:06.375 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 1 2024-04-20T11:33:06.378 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.379+0000 7ff464200640 1 -- 172.21.15.152:0/4036862964 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7ff43c06ca40 msgr2=0x7ff43c06ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:06.378 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.379+0000 7ff464200640 1 --2- 172.21.15.152:0/4036862964 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7ff43c06ca40 0x7ff43c06ef00 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff450034000 tx=0x7ff450007000 comp rx=0 tx=0).stop 2024-04-20T11:33:06.378 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.379+0000 7ff464200640 1 -- 172.21.15.152:0/4036862964 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff45c13fe10 msgr2=0x7ff45c140290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:06.378 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.379+0000 7ff464200640 1 --2- 172.21.15.152:0/4036862964 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff45c13fe10 0x7ff45c140290 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7ff45c0700a0 tx=0x7ff45400f4a0 comp rx=0 tx=0).stop 2024-04-20T11:33:06.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.379+0000 7ff464200640 1 -- 172.21.15.152:0/4036862964 shutdown_connections 2024-04-20T11:33:06.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.379+0000 7ff464200640 1 --2- 172.21.15.152:0/4036862964 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7ff43c06ca40 0x7ff43c06ef00 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:06.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.379+0000 7ff464200640 1 --2- 172.21.15.152:0/4036862964 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff45c13fe10 0x7ff45c140290 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:06.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.379+0000 7ff464200640 1 --2- 172.21.15.152:0/4036862964 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff45c15bcf0 0x7ff45c141870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:06.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.379+0000 7ff464200640 1 --2- 172.21.15.152:0/4036862964 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4440039e0 0x7ff45c1f3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:06.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.379+0000 7ff464200640 1 -- 172.21.15.152:0/4036862964 >> 172.21.15.152:0/4036862964 conn(0x7ff45c06b980 msgr2=0x7ff45c143d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:06.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.379+0000 7ff464200640 1 -- 172.21.15.152:0/4036862964 shutdown_connections 2024-04-20T11:33:06.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.379+0000 7ff464200640 1 -- 172.21.15.152:0/4036862964 wait complete. 2024-04-20T11:33:06.399 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T11:33:06.607 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.608+0000 7f8981e00640 1 Processor -- start 2024-04-20T11:33:06.608 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.608+0000 7f8981e00640 1 -- start start 2024-04-20T11:33:06.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.609+0000 7f8981e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f897c0b2880 0x7f897c0b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:06.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.610+0000 7f8981e00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f897c1463e0 con 0x7f897c146000 2024-04-20T11:33:06.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.610+0000 7f8981e00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f897c0b3e20 con 0x7f897c0cd1b0 2024-04-20T11:33:06.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.610+0000 7f8981e00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f897c0b4750 con 0x7f897c0b2880 2024-04-20T11:33:06.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.610+0000 7f897aa00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f897c146000 0x7f897c0b2170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:58182/0 (socket says 172.21.15.152:58182) 2024-04-20T11:33:06.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.610+0000 7f897aa00640 1 -- 172.21.15.152:0/1044858741 learned_addr learned my addr 172.21.15.152:0/1044858741 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:33:06.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.610+0000 7f8972000640 1 --2- 172.21.15.152:0/1044858741 >> v2:172.21.15.152:3300/0 conn(0x7f897c0b2880 0x7f897c0b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:06.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.610+0000 7f8972000640 1 -- 172.21.15.152:0/1044858741 >> v1:172.21.15.175:6789/0 conn(0x7f897c0cd1b0 legacy=0x7f897c0cd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:06.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.610+0000 7f8972000640 1 -- 172.21.15.152:0/1044858741 >> v1:172.21.15.152:6789/0 conn(0x7f897c146000 legacy=0x7f897c0b2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:06.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.610+0000 7f8972000640 1 -- 172.21.15.152:0/1044858741 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f897c0b4100 con 0x7f897c0b2880 2024-04-20T11:33:06.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.612+0000 7f8972000640 1 --2- 172.21.15.152:0/1044858741 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f897c0b2880 0x7f897c0b4be0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f89640097a0 tx=0x7f896402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8d4dc3dda3e2eb4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:06.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.612+0000 7f897a000640 1 -- 172.21.15.152:0/1044858741 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89640315a0 con 0x7f897c0b2880 2024-04-20T11:33:06.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.612+0000 7f897a000640 1 -- 172.21.15.152:0/1044858741 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8964037070 con 0x7f897c0b2880 2024-04-20T11:33:06.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.612+0000 7f897a000640 1 -- 172.21.15.152:0/1044858741 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8964031960 con 0x7f897c0b2880 2024-04-20T11:33:06.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.613+0000 7f8981e00640 1 -- 172.21.15.152:0/1044858741 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f897c0b2880 msgr2=0x7f897c0b4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:06.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.613+0000 7f8981e00640 1 --2- 172.21.15.152:0/1044858741 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f897c0b2880 0x7f897c0b4be0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f89640097a0 tx=0x7f896402b840 comp rx=0 tx=0).stop 2024-04-20T11:33:06.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.613+0000 7f8981e00640 1 -- 172.21.15.152:0/1044858741 shutdown_connections 2024-04-20T11:33:06.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.613+0000 7f8981e00640 1 --2- 172.21.15.152:0/1044858741 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f897c0b2880 0x7f897c0b4be0 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:06.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.613+0000 7f8981e00640 1 -- 172.21.15.152:0/1044858741 >> 172.21.15.152:0/1044858741 conn(0x7f897c06b980 msgr2=0x7f897c06bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:06.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.613+0000 7f8981e00640 1 -- 172.21.15.152:0/1044858741 shutdown_connections 2024-04-20T11:33:06.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.613+0000 7f8981e00640 1 -- 172.21.15.152:0/1044858741 wait complete. 2024-04-20T11:33:06.614 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.615+0000 7f8981e00640 1 Processor -- start 2024-04-20T11:33:06.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.615+0000 7f8981e00640 1 -- start start 2024-04-20T11:33:06.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.617+0000 7f8981e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f897c0b2880 0x7f897c15e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:06.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.617+0000 7f8981e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f897c0cd1b0 0x7f897c0ab240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:06.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.617+0000 7f8981e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f897c146000 0x7f897c1347a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:06.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.617+0000 7f8981e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f897c15ce70 con 0x7f897c146000 2024-04-20T11:33:06.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.617+0000 7f8981e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f897c15ccf0 con 0x7f897c0cd1b0 2024-04-20T11:33:06.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.617+0000 7f8981e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f897c15cff0 con 0x7f897c0b2880 2024-04-20T11:33:06.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.617+0000 7f897b400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f897c146000 0x7f897c1347a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:06.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.617+0000 7f897b400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f897c146000 0x7f897c1347a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36420/0 (socket says 172.21.15.152:36420) 2024-04-20T11:33:06.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.617+0000 7f897b400640 1 -- 172.21.15.152:0/280012821 learned_addr learned my addr 172.21.15.152:0/280012821 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:33:06.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.617+0000 7f897aa00640 1 --2- 172.21.15.152:0/280012821 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f897c0b2880 0x7f897c15e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:06.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.617+0000 7f8972000640 1 --2- 172.21.15.152:0/280012821 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f897c0cd1b0 0x7f897c0ab240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:06.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.617+0000 7f897b400640 1 -- 172.21.15.152:0/280012821 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f897c0b2880 msgr2=0x7f897c15e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:06.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.617+0000 7f897b400640 1 --2- 172.21.15.152:0/280012821 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f897c0b2880 0x7f897c15e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:06.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.617+0000 7f897b400640 1 -- 172.21.15.152:0/280012821 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f897c0cd1b0 msgr2=0x7f897c0ab240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:06.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.617+0000 7f897b400640 1 --2- 172.21.15.152:0/280012821 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f897c0cd1b0 0x7f897c0ab240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:06.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.617+0000 7f897b400640 1 -- 172.21.15.152:0/280012821 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f897c140360 con 0x7f897c146000 2024-04-20T11:33:06.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.618+0000 7f897b400640 1 --2- 172.21.15.152:0/280012821 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f897c146000 0x7f897c1347a0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f897400c410 tx=0x7f897400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:06.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.618+0000 7f8973e00640 1 -- 172.21.15.152:0/280012821 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8974011790 con 0x7f897c146000 2024-04-20T11:33:06.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.618+0000 7f8973e00640 1 -- 172.21.15.152:0/280012821 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8974017070 con 0x7f897c146000 2024-04-20T11:33:06.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.618+0000 7f8973e00640 1 -- 172.21.15.152:0/280012821 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8974011bb0 con 0x7f897c146000 2024-04-20T11:33:06.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.618+0000 7f8981e00640 1 -- 172.21.15.152:0/280012821 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f897c15e880 con 0x7f897c146000 2024-04-20T11:33:06.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.618+0000 7f8981e00640 1 -- 172.21.15.152:0/280012821 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f897c15ee40 con 0x7f897c146000 2024-04-20T11:33:06.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.620+0000 7f8981e00640 1 -- 172.21.15.152:0/280012821 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f897c0689f0 con 0x7f897c146000 2024-04-20T11:33:06.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.621+0000 7f8973e00640 1 -- 172.21.15.152:0/280012821 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 91142+0+0 (secure 0 0 0) 0x7f897401c460 con 0x7f897c146000 2024-04-20T11:33:06.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.621+0000 7f8973e00640 1 --2- 172.21.15.152:0/280012821 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f895806ca10 0x7f895806eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:06.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.622+0000 7f8973e00640 1 -- 172.21.15.152:0/280012821 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5795+0+0 (secure 0 0 0) 0x7f897408b930 con 0x7f897c146000 2024-04-20T11:33:06.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.622+0000 7f897aa00640 1 --2- 172.21.15.152:0/280012821 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f895806ca10 0x7f895806eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:06.623 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.623+0000 7f897aa00640 1 --2- 172.21.15.152:0/280012821 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f895806ca10 0x7f895806eed0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f896c0012e0 tx=0x7f896c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:06.625 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.626+0000 7f8973e00640 1 -- 172.21.15.152:0/280012821 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f8974021020 con 0x7f897c146000 2024-04-20T11:33:06.826 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.826+0000 7f8981e00640 1 -- 172.21.15.152:0/280012821 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f897c0b4df0 con 0x7f897c146000 2024-04-20T11:33:06.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.827+0000 7f8973e00640 1 -- 172.21.15.152:0/280012821 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) ==== 104+0+11 (secure 0 0 0) 0x7f8974059be0 con 0x7f897c146000 2024-04-20T11:33:06.827 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 32 2024-04-20T11:33:06.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.829+0000 7f8981e00640 1 -- 172.21.15.152:0/280012821 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f895806ca10 msgr2=0x7f895806eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:06.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.829+0000 7f8981e00640 1 --2- 172.21.15.152:0/280012821 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f895806ca10 0x7f895806eed0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f896c0012e0 tx=0x7f896c009040 comp rx=0 tx=0).stop 2024-04-20T11:33:06.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.829+0000 7f8981e00640 1 -- 172.21.15.152:0/280012821 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f897c146000 msgr2=0x7f897c1347a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:06.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.829+0000 7f8981e00640 1 --2- 172.21.15.152:0/280012821 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f897c146000 0x7f897c1347a0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f897400c410 tx=0x7f897400c8d0 comp rx=0 tx=0).stop 2024-04-20T11:33:06.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.830+0000 7f8981e00640 1 -- 172.21.15.152:0/280012821 shutdown_connections 2024-04-20T11:33:06.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.830+0000 7f8981e00640 1 --2- 172.21.15.152:0/280012821 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f895806ca10 0x7f895806eed0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:06.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.830+0000 7f8981e00640 1 --2- 172.21.15.152:0/280012821 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f897c146000 0x7f897c1347a0 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:06.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.830+0000 7f8981e00640 1 --2- 172.21.15.152:0/280012821 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f897c0cd1b0 0x7f897c0ab240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:06.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.830+0000 7f8981e00640 1 --2- 172.21.15.152:0/280012821 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f897c0b2880 0x7f897c15e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:06.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.830+0000 7f8981e00640 1 -- 172.21.15.152:0/280012821 >> 172.21.15.152:0/280012821 conn(0x7f897c06b980 msgr2=0x7f897c071db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:06.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.830+0000 7f8981e00640 1 -- 172.21.15.152:0/280012821 shutdown_connections 2024-04-20T11:33:06.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:06.830+0000 7f8981e00640 1 -- 172.21.15.152:0/280012821 wait complete. 2024-04-20T11:33:06.850 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T11:33:07.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.055+0000 7f5979800640 1 Processor -- start 2024-04-20T11:33:07.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.055+0000 7f5979800640 1 -- start start 2024-04-20T11:33:07.057 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.057+0000 7f5979800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f5974146000 0x7f59740b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:07.057 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.057+0000 7f5979800640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59741463e0 con 0x7f59740cd8c0 2024-04-20T11:33:07.057 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.057+0000 7f5979800640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59740b3e20 con 0x7f59740b2170 2024-04-20T11:33:07.057 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.057+0000 7f5979800640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f59740cdca0 con 0x7f5974146000 2024-04-20T11:33:07.057 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.057+0000 7f5972a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f5974146000 0x7f59740b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:07.057 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.057+0000 7f5972a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f5974146000 0x7f59740b4be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36434/0 (socket says 172.21.15.152:36434) 2024-04-20T11:33:07.057 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.057+0000 7f5972a00640 1 -- 172.21.15.152:0/3513966864 learned_addr learned my addr 172.21.15.152:0/3513966864 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:33:07.057 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.057+0000 7f5973400640 1 --1- 172.21.15.152:0/3513966864 >> v1:172.21.15.152:6789/0 conn(0x7f59740cd8c0 0x7f597415ccf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:58184/0 (socket says 172.21.15.152:58184) 2024-04-20T11:33:07.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.058+0000 7f5972a00640 1 -- 172.21.15.152:0/3513966864 >> v1:172.21.15.175:6789/0 conn(0x7f59740b2170 legacy=0x7f59740cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:07.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.058+0000 7f5972a00640 1 -- 172.21.15.152:0/3513966864 >> v1:172.21.15.152:6789/0 conn(0x7f59740cd8c0 legacy=0x7f597415ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:07.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.058+0000 7f5972a00640 1 -- 172.21.15.152:0/3513966864 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59740b4100 con 0x7f5974146000 2024-04-20T11:33:07.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.059+0000 7f5972a00640 1 --2- 172.21.15.152:0/3513966864 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5974146000 0x7f59740b4be0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f5964009e50 tx=0x7f596402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e48dfdc139e8d95d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:07.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.059+0000 7f5972000640 1 -- 172.21.15.152:0/3513966864 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f596402ca80 con 0x7f5974146000 2024-04-20T11:33:07.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.059+0000 7f5972000640 1 -- 172.21.15.152:0/3513966864 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f596402cc20 con 0x7f5974146000 2024-04-20T11:33:07.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.059+0000 7f5972000640 1 -- 172.21.15.152:0/3513966864 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59640343f0 con 0x7f5974146000 2024-04-20T11:33:07.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.060+0000 7f5979800640 1 -- 172.21.15.152:0/3513966864 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5974146000 msgr2=0x7f59740b4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:07.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.060+0000 7f5979800640 1 --2- 172.21.15.152:0/3513966864 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5974146000 0x7f59740b4be0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f5964009e50 tx=0x7f596402bee0 comp rx=0 tx=0).stop 2024-04-20T11:33:07.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.060+0000 7f5979800640 1 -- 172.21.15.152:0/3513966864 shutdown_connections 2024-04-20T11:33:07.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.060+0000 7f5979800640 1 --2- 172.21.15.152:0/3513966864 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5974146000 0x7f59740b4be0 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:07.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.060+0000 7f5979800640 1 -- 172.21.15.152:0/3513966864 >> 172.21.15.152:0/3513966864 conn(0x7f597406b980 msgr2=0x7f597406bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:07.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.060+0000 7f5979800640 1 -- 172.21.15.152:0/3513966864 shutdown_connections 2024-04-20T11:33:07.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.060+0000 7f5979800640 1 -- 172.21.15.152:0/3513966864 wait complete. 2024-04-20T11:33:07.062 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.063+0000 7f5979800640 1 Processor -- start 2024-04-20T11:33:07.062 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.063+0000 7f5979800640 1 -- start start 2024-04-20T11:33:07.064 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.064+0000 7f5979800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f59740b2170 0x7f59741350b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:07.064 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.064+0000 7f5979800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59740cd8c0 0x7f59740af570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:07.064 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.064+0000 7f5979800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5974146000 0x7f5974143360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:07.064 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.064+0000 7f5979800640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f59740b2620 con 0x7f59740b2170 2024-04-20T11:33:07.064 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.064+0000 7f5979800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f59740b4750 con 0x7f59740cd8c0 2024-04-20T11:33:07.064 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.064+0000 7f5979800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f59740b27a0 con 0x7f5974146000 2024-04-20T11:33:07.064 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.064+0000 7f5972a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f59740b2170 0x7f59741350b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:07.064 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.064+0000 7f5972a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f59740b2170 0x7f59741350b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36448/0 (socket says 172.21.15.152:36448) 2024-04-20T11:33:07.064 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.064+0000 7f5972a00640 1 -- 172.21.15.152:0/308662039 learned_addr learned my addr 172.21.15.152:0/308662039 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:33:07.065 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.065+0000 7f596a000640 1 --2- 172.21.15.152:0/308662039 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59740cd8c0 0x7f59740af570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:07.066 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.065+0000 7f5973400640 1 --2- 172.21.15.152:0/308662039 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5974146000 0x7f5974143360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:07.066 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.065+0000 7f5972a00640 1 -- 172.21.15.152:0/308662039 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5974146000 msgr2=0x7f5974143360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:07.066 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.065+0000 7f5972a00640 1 --2- 172.21.15.152:0/308662039 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5974146000 0x7f5974143360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:07.066 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.065+0000 7f5972a00640 1 -- 172.21.15.152:0/308662039 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59740cd8c0 msgr2=0x7f59740af570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:07.066 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.065+0000 7f5972a00640 1 --2- 172.21.15.152:0/308662039 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59740cd8c0 0x7f59740af570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:07.066 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.065+0000 7f5972a00640 1 -- 172.21.15.152:0/308662039 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59740b0060 con 0x7f59740b2170 2024-04-20T11:33:07.066 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.065+0000 7f5972a00640 1 --2- 172.21.15.152:0/308662039 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f59740b2170 0x7f59741350b0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f5964009e50 tx=0x7f5964004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:07.066 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.066+0000 7f596b400640 1 -- 172.21.15.152:0/308662039 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5964043020 con 0x7f59740b2170 2024-04-20T11:33:07.066 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.066+0000 7f596b400640 1 -- 172.21.15.152:0/308662039 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f596403e450 con 0x7f59740b2170 2024-04-20T11:33:07.066 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.066+0000 7f596b400640 1 -- 172.21.15.152:0/308662039 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5964009510 con 0x7f59740b2170 2024-04-20T11:33:07.066 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.066+0000 7f5979800640 1 -- 172.21.15.152:0/308662039 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59740b0340 con 0x7f59740b2170 2024-04-20T11:33:07.066 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.066+0000 7f5979800640 1 -- 172.21.15.152:0/308662039 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59740b0900 con 0x7f59740b2170 2024-04-20T11:33:07.067 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.067+0000 7f5979800640 1 -- 172.21.15.152:0/308662039 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59740689f0 con 0x7f59740b2170 2024-04-20T11:33:07.067 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.068+0000 7f596b400640 1 -- 172.21.15.152:0/308662039 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 91142+0+0 (secure 0 0 0) 0x7f5964046020 con 0x7f59740b2170 2024-04-20T11:33:07.068 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.069+0000 7f596b400640 1 --2- 172.21.15.152:0/308662039 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f595006ca40 0x7f595006ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:07.069 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.069+0000 7f596b400640 1 -- 172.21.15.152:0/308662039 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5795+0+0 (secure 0 0 0) 0x7f59640ae700 con 0x7f59740b2170 2024-04-20T11:33:07.069 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.070+0000 7f596a000640 1 --2- 172.21.15.152:0/308662039 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f595006ca40 0x7f595006ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:07.070 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.071+0000 7f596a000640 1 --2- 172.21.15.152:0/308662039 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f595006ca40 0x7f595006ef00 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f59740af4a0 tx=0x7f595c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:07.073 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.074+0000 7f596b400640 1 -- 172.21.15.152:0/308662039 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f596404d030 con 0x7f59740b2170 2024-04-20T11:33:07.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.275+0000 7f5979800640 1 -- 172.21.15.152:0/308662039 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f59740721c0 con 0x7f59740b2170 2024-04-20T11:33:07.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.276+0000 7f596b400640 1 -- 172.21.15.152:0/308662039 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) ==== 100+0+11 (secure 0 0 0) 0x7f596407c9b0 con 0x7f59740b2170 2024-04-20T11:33:07.276 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 64 2024-04-20T11:33:07.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.278+0000 7f5979800640 1 -- 172.21.15.152:0/308662039 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f595006ca40 msgr2=0x7f595006ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:07.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.278+0000 7f5979800640 1 --2- 172.21.15.152:0/308662039 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f595006ca40 0x7f595006ef00 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f59740af4a0 tx=0x7f595c009040 comp rx=0 tx=0).stop 2024-04-20T11:33:07.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.278+0000 7f5979800640 1 -- 172.21.15.152:0/308662039 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f59740b2170 msgr2=0x7f59741350b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:07.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.278+0000 7f5979800640 1 --2- 172.21.15.152:0/308662039 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f59740b2170 0x7f59741350b0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f5964009e50 tx=0x7f5964004a40 comp rx=0 tx=0).stop 2024-04-20T11:33:07.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.279+0000 7f5979800640 1 -- 172.21.15.152:0/308662039 shutdown_connections 2024-04-20T11:33:07.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.279+0000 7f5979800640 1 --2- 172.21.15.152:0/308662039 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f595006ca40 0x7f595006ef00 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:07.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.279+0000 7f5979800640 1 --2- 172.21.15.152:0/308662039 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5974146000 0x7f5974143360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:07.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.279+0000 7f5979800640 1 --2- 172.21.15.152:0/308662039 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59740cd8c0 0x7f59740af570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:07.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.279+0000 7f5979800640 1 --2- 172.21.15.152:0/308662039 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f59740b2170 0x7f59741350b0 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:07.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.279+0000 7f5979800640 1 -- 172.21.15.152:0/308662039 >> 172.21.15.152:0/308662039 conn(0x7f597406b980 msgr2=0x7f5974145240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:07.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.279+0000 7f5979800640 1 -- 172.21.15.152:0/308662039 shutdown_connections 2024-04-20T11:33:07.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.279+0000 7f5979800640 1 -- 172.21.15.152:0/308662039 wait complete. 2024-04-20T11:33:07.298 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-20T11:33:07.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.497+0000 7f00ddc00640 1 Processor -- start 2024-04-20T11:33:07.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.497+0000 7f00ddc00640 1 -- start start 2024-04-20T11:33:07.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.499+0000 7f00ddc00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f00d8161d30 0x7f00d80d0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:07.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.499+0000 7f00ddc00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f00d80d4ec0 0x7f00d80d15e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:07.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.499+0000 7f00ddc00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f00d80d2b80 0x7f00d80d3010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:07.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.499+0000 7f00ddc00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f00d80d0540 con 0x7f00d8161d30 2024-04-20T11:33:07.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.499+0000 7f00ddc00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f00d80cfb70 con 0x7f00d80d2b80 2024-04-20T11:33:07.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.499+0000 7f00ddc00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f00d80d1b20 con 0x7f00d80d4ec0 2024-04-20T11:33:07.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.499+0000 7f00d7e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f00d8161d30 0x7f00d80d0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:07.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.499+0000 7f00d7e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f00d8161d30 0x7f00d80d0000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36450/0 (socket says 172.21.15.152:36450) 2024-04-20T11:33:07.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.499+0000 7f00d7e00640 1 -- 172.21.15.152:0/2299853303 learned_addr learned my addr 172.21.15.152:0/2299853303 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:33:07.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.500+0000 7f00d7400640 1 --2- 172.21.15.152:0/2299853303 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f00d80d4ec0 0x7f00d80d15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:07.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.500+0000 7f00dd200640 1 --2- 172.21.15.152:0/2299853303 >> v2:172.21.15.175:3300/0 conn(0x7f00d80d2b80 0x7f00d80d3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:07.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.500+0000 7f00d7e00640 1 -- 172.21.15.152:0/2299853303 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f00d80d4ec0 msgr2=0x7f00d80d15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:07.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.500+0000 7f00d7e00640 1 --2- 172.21.15.152:0/2299853303 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f00d80d4ec0 0x7f00d80d15e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:07.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.500+0000 7f00d7e00640 1 -- 172.21.15.152:0/2299853303 >> v2:172.21.15.175:3300/0 conn(0x7f00d80d2b80 msgr2=0x7f00d80d3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:07.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.500+0000 7f00d7e00640 1 --2- 172.21.15.152:0/2299853303 >> v2:172.21.15.175:3300/0 conn(0x7f00d80d2b80 0x7f00d80d3010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:07.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.500+0000 7f00d7e00640 1 -- 172.21.15.152:0/2299853303 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00d80d26a0 con 0x7f00d8161d30 2024-04-20T11:33:07.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.501+0000 7f00d7e00640 1 --2- 172.21.15.152:0/2299853303 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f00d8161d30 0x7f00d80d0000 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f00d80d20d0 tx=0x7f00c802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e433adfe68fd01a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:07.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.501+0000 7f00d6a00640 1 -- 172.21.15.152:0/2299853303 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c8035020 con 0x7f00d8161d30 2024-04-20T11:33:07.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.501+0000 7f00d6a00640 1 -- 172.21.15.152:0/2299853303 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00c8031870 con 0x7f00d8161d30 2024-04-20T11:33:07.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.501+0000 7f00d6a00640 1 -- 172.21.15.152:0/2299853303 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c8031c40 con 0x7f00d8161d30 2024-04-20T11:33:07.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.502+0000 7f00ddc00640 1 -- 172.21.15.152:0/2299853303 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f00d8161d30 msgr2=0x7f00d80d0000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:07.502 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.502+0000 7f00ddc00640 1 --2- 172.21.15.152:0/2299853303 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f00d8161d30 0x7f00d80d0000 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f00d80d20d0 tx=0x7f00c802b910 comp rx=0 tx=0).stop 2024-04-20T11:33:07.502 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.502+0000 7f00ddc00640 1 -- 172.21.15.152:0/2299853303 shutdown_connections 2024-04-20T11:33:07.502 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.502+0000 7f00ddc00640 1 --2- 172.21.15.152:0/2299853303 >> v2:172.21.15.175:3300/0 conn(0x7f00d80d2b80 0x7f00d80d3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:07.502 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.502+0000 7f00ddc00640 1 --2- 172.21.15.152:0/2299853303 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f00d80d4ec0 0x7f00d80d15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:07.502 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.502+0000 7f00ddc00640 1 --2- 172.21.15.152:0/2299853303 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f00d8161d30 0x7f00d80d0000 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:07.502 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.502+0000 7f00ddc00640 1 -- 172.21.15.152:0/2299853303 >> 172.21.15.152:0/2299853303 conn(0x7f00d808bfd0 msgr2=0x7f00d80d45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:07.503 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.502+0000 7f00ddc00640 1 -- 172.21.15.152:0/2299853303 shutdown_connections 2024-04-20T11:33:07.503 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.502+0000 7f00ddc00640 1 -- 172.21.15.152:0/2299853303 wait complete. 2024-04-20T11:33:07.503 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.504+0000 7f00ddc00640 1 Processor -- start 2024-04-20T11:33:07.503 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.504+0000 7f00ddc00640 1 -- start start 2024-04-20T11:33:07.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.505+0000 7f00ddc00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f00d80d2b80 0x7f00d8158920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:07.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.505+0000 7f00ddc00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f00d80d4ec0 0x7f00d8159520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:07.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.505+0000 7f00ddc00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00d8161d30 0x7f00d81579f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:07.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.505+0000 7f00ddc00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f00d80cf800 con 0x7f00d80d2b80 2024-04-20T11:33:07.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.505+0000 7f00ddc00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f00d80cf520 con 0x7f00d8161d30 2024-04-20T11:33:07.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.505+0000 7f00ddc00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f00d80cf1b0 con 0x7f00d80d4ec0 2024-04-20T11:33:07.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.505+0000 7f00d7e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f00d80d2b80 0x7f00d8158920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:07.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.505+0000 7f00d7e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f00d80d2b80 0x7f00d8158920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36458/0 (socket says 172.21.15.152:36458) 2024-04-20T11:33:07.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.505+0000 7f00d7e00640 1 -- 172.21.15.152:0/1605910631 learned_addr learned my addr 172.21.15.152:0/1605910631 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:33:07.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.506+0000 7f00d7400640 1 --2- 172.21.15.152:0/1605910631 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f00d80d4ec0 0x7f00d8159520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:07.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.506+0000 7f00dd200640 1 --2- 172.21.15.152:0/1605910631 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00d8161d30 0x7f00d81579f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:07.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.506+0000 7f00d7e00640 1 -- 172.21.15.152:0/1605910631 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f00d80d4ec0 msgr2=0x7f00d8159520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:07.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.506+0000 7f00d7e00640 1 --2- 172.21.15.152:0/1605910631 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f00d80d4ec0 0x7f00d8159520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:07.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.506+0000 7f00d7e00640 1 -- 172.21.15.152:0/1605910631 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00d8161d30 msgr2=0x7f00d81579f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:07.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.506+0000 7f00d7e00640 1 --2- 172.21.15.152:0/1605910631 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00d8161d30 0x7f00d81579f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:07.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.506+0000 7f00d7e00640 1 -- 172.21.15.152:0/1605910631 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00d8158e60 con 0x7f00d80d2b80 2024-04-20T11:33:07.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.506+0000 7f00d7400640 1 --2- 172.21.15.152:0/1605910631 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f00d80d4ec0 0x7f00d8159520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:33:07.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.506+0000 7f00d7e00640 1 --2- 172.21.15.152:0/1605910631 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f00d80d2b80 0x7f00d8158920 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f00c802b950 tx=0x7f00c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:07.508 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.507+0000 7f00cfe00640 1 -- 172.21.15.152:0/1605910631 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c803ad60 con 0x7f00d80d2b80 2024-04-20T11:33:07.508 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.507+0000 7f00cfe00640 1 -- 172.21.15.152:0/1605910631 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00c80443f0 con 0x7f00d80d2b80 2024-04-20T11:33:07.508 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.507+0000 7f00cfe00640 1 -- 172.21.15.152:0/1605910631 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c80447c0 con 0x7f00d80d2b80 2024-04-20T11:33:07.508 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.507+0000 7f00ddc00640 1 -- 172.21.15.152:0/1605910631 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00d815c020 con 0x7f00d80d2b80 2024-04-20T11:33:07.508 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.507+0000 7f00ddc00640 1 -- 172.21.15.152:0/1605910631 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00d815c5e0 con 0x7f00d80d2b80 2024-04-20T11:33:07.508 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.508+0000 7f00ddc00640 1 -- 172.21.15.152:0/1605910631 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00d8075e60 con 0x7f00d80d2b80 2024-04-20T11:33:07.508 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.509+0000 7f00cfe00640 1 -- 172.21.15.152:0/1605910631 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 91142+0+0 (secure 0 0 0) 0x7f00c8038020 con 0x7f00d80d2b80 2024-04-20T11:33:07.510 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.510+0000 7f00cfe00640 1 --2- 172.21.15.152:0/1605910631 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f00b806ca10 0x7f00b806eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:07.510 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.510+0000 7f00cfe00640 1 -- 172.21.15.152:0/1605910631 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5795+0+0 (secure 0 0 0) 0x7f00c80ae040 con 0x7f00d80d2b80 2024-04-20T11:33:07.510 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.511+0000 7f00d7400640 1 --2- 172.21.15.152:0/1605910631 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f00b806ca10 0x7f00b806eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:07.511 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.512+0000 7f00d7400640 1 --2- 172.21.15.152:0/1605910631 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f00b806ca10 0x7f00b806eed0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f00d80ce780 tx=0x7f00c4008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:07.514 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.515+0000 7f00cfe00640 1 -- 172.21.15.152:0/1605910631 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f00c803f070 con 0x7f00d80d2b80 2024-04-20T11:33:07.725 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.725+0000 7f00ddc00640 1 -- 172.21.15.152:0/1605910631 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f00d80b2be0 con 0x7f00d80d2b80 2024-04-20T11:33:07.725 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.726+0000 7f00cfe00640 1 -- 172.21.15.152:0/1605910631 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v91) ==== 80+0+124 (secure 0 0 0) 0x7f00c807c150 con 0x7f00d80d2b80 2024-04-20T11:33:07.726 INFO:teuthology.orchestra.run.smithi152.stdout:172.21.15.175:6835/2632275131 2024-04-21T11:32:59.571722+0000 2024-04-20T11:33:07.726 INFO:teuthology.orchestra.run.smithi152.stdout:172.21.15.175:6834/2632275131 2024-04-21T11:32:59.571722+0000 2024-04-20T11:33:07.726 INFO:teuthology.orchestra.run.smithi152.stderr:listed 2 entries 2024-04-20T11:33:07.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.729+0000 7f00ddc00640 1 -- 172.21.15.152:0/1605910631 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f00b806ca10 msgr2=0x7f00b806eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:07.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.729+0000 7f00ddc00640 1 --2- 172.21.15.152:0/1605910631 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f00b806ca10 0x7f00b806eed0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f00d80ce780 tx=0x7f00c4008040 comp rx=0 tx=0).stop 2024-04-20T11:33:07.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.729+0000 7f00ddc00640 1 -- 172.21.15.152:0/1605910631 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f00d80d2b80 msgr2=0x7f00d8158920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:07.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.729+0000 7f00ddc00640 1 --2- 172.21.15.152:0/1605910631 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f00d80d2b80 0x7f00d8158920 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f00c802b950 tx=0x7f00c80046c0 comp rx=0 tx=0).stop 2024-04-20T11:33:07.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.729+0000 7f00ddc00640 1 -- 172.21.15.152:0/1605910631 shutdown_connections 2024-04-20T11:33:07.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.729+0000 7f00ddc00640 1 --2- 172.21.15.152:0/1605910631 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f00b806ca10 0x7f00b806eed0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:07.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.729+0000 7f00ddc00640 1 --2- 172.21.15.152:0/1605910631 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00d8161d30 0x7f00d81579f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:07.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.729+0000 7f00ddc00640 1 --2- 172.21.15.152:0/1605910631 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f00d80d4ec0 0x7f00d8159520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:07.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.729+0000 7f00ddc00640 1 --2- 172.21.15.152:0/1605910631 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f00d80d2b80 0x7f00d8158920 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:07.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.729+0000 7f00ddc00640 1 -- 172.21.15.152:0/1605910631 >> 172.21.15.152:0/1605910631 conn(0x7f00d808bfd0 msgr2=0x7f00d8160fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:07.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.729+0000 7f00ddc00640 1 -- 172.21.15.152:0/1605910631 shutdown_connections 2024-04-20T11:33:07.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:07.729+0000 7f00ddc00640 1 -- 172.21.15.152:0/1605910631 wait complete. 2024-04-20T11:33:07.746 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi175.front.sepia.ceph.com... 2024-04-20T11:33:07.747 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:07.747 DEBUG:teuthology.orchestra.run.smithi175:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-04-20T11:33:07.831 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi175.stderr:ceph-fuse[33043]: fuse finished with error 0 and tester_r 0 2024-04-20T11:33:07.838 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi175.front.sepia.ceph.com 2024-04-20T11:33:07.838 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:07.838 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-20T11:33:07.913 INFO:tasks.cephfs.mount:Terminating background process 2024-04-20T11:33:07.913 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-20T11:33:07.913 DEBUG:teuthology.orchestra.run.smithi152:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:33:07.967 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-20T11:33:07.968 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi152.front.sepia.ceph.com 2024-04-20T11:33:07.968 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:07.968 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-20T11:33:08.018 INFO:teuthology.orchestra.run.smithi152.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-20T11:33:08.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T11:33:08.019 INFO:tasks.cephfs.mount:Terminating background process 2024-04-20T11:33:08.019 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T11:33:08.019 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:33:08.077 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-20T11:33:08.077 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi175.front.sepia.ceph.com 2024-04-20T11:33:08.077 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:08.077 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-20T11:33:08.128 INFO:teuthology.orchestra.run.smithi175.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-20T11:33:08.129 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T11:33:08.129 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:33:08.331 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.332+0000 7fb4d4a00640 1 Processor -- start 2024-04-20T11:33:08.332 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.332+0000 7fb4d4a00640 1 -- start start 2024-04-20T11:33:08.333 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.334+0000 7fb4d4a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb4d006f510 0x7fb4d0140dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:08.333 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.334+0000 7fb4d4a00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fb4d0141310 0x7fb4d0071cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:08.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.334+0000 7fb4d4a00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4d00cf0a0 con 0x7fb4d013e530 2024-04-20T11:33:08.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.334+0000 7fb4d4a00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fb4d0141b00 con 0x7fb4d0141310 2024-04-20T11:33:08.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.334+0000 7fb4d4a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb4d00cfc50 con 0x7fb4d006f510 2024-04-20T11:33:08.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.334+0000 7fb4cf400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fb4d013e530 0x7fb4d006ee00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:58198/0 (socket says 172.21.15.152:58198) 2024-04-20T11:33:08.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.334+0000 7fb4cf400640 1 -- 172.21.15.152:0/2417926195 learned_addr learned my addr 172.21.15.152:0/2417926195 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:33:08.336 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.335+0000 7fb4c6a00640 1 --2- 172.21.15.152:0/2417926195 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb4d006f510 0x7fb4d0140dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:08.336 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.335+0000 7fb4cea00640 1 -- 172.21.15.152:0/2417926195 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4235913657 0 0) 0x7fb4c0001490 con 0x7fb4d013e530 2024-04-20T11:33:08.336 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.335+0000 7fb4cea00640 1 -- 172.21.15.152:0/2417926195 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb4b4003520 con 0x7fb4d013e530 2024-04-20T11:33:08.336 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.335+0000 7fb4cfe00640 1 --2- 172.21.15.152:0/2417926195 >> v2:172.21.15.175:3300/0 conn(0x7fb4d0141310 0x7fb4d0071cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:08.336 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.335+0000 7fb4cea00640 1 -- 172.21.15.152:0/2417926195 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2845049382 0 0) 0x7fb4b4003520 con 0x7fb4d013e530 2024-04-20T11:33:08.336 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.335+0000 7fb4cea00640 1 -- 172.21.15.152:0/2417926195 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb4d006f510 msgr2=0x7fb4d0140dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:08.336 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.335+0000 7fb4cea00640 1 --2- 172.21.15.152:0/2417926195 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb4d006f510 0x7fb4d0140dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:08.336 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.335+0000 7fb4cea00640 1 -- 172.21.15.152:0/2417926195 >> v2:172.21.15.175:3300/0 conn(0x7fb4d0141310 msgr2=0x7fb4d0071cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:08.336 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.335+0000 7fb4cea00640 1 --2- 172.21.15.152:0/2417926195 >> v2:172.21.15.175:3300/0 conn(0x7fb4d0141310 0x7fb4d0071cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:08.336 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.335+0000 7fb4cea00640 1 -- 172.21.15.152:0/2417926195 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4d00d00e0 con 0x7fb4d013e530 2024-04-20T11:33:08.336 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.335+0000 7fb4cea00640 1 -- 172.21.15.152:0/2417926195 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7fb4c0001bc0 con 0x7fb4d013e530 2024-04-20T11:33:08.336 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.335+0000 7fb4cea00640 1 -- 172.21.15.152:0/2417926195 >> v1:172.21.15.152:6789/0 conn(0x7fb4d013e530 legacy=0x7fb4d006ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:08.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.335+0000 7fb4cea00640 1 --2- 172.21.15.152:0/2417926195 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4b40039b0 0x7fb4b4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:08.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.335+0000 7fb4cea00640 1 --2- 172.21.15.152:0/2417926195 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb4d006f510 0x7fb4b4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:08.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.335+0000 7fb4cea00640 1 --2- 172.21.15.152:0/2417926195 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb4b4004d40 0x7fb4b40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:08.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.335+0000 7fb4cea00640 1 -- 172.21.15.152:0/2417926195 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d00cfc50 con 0x7fb4b4004d40 2024-04-20T11:33:08.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.335+0000 7fb4cea00640 1 -- 172.21.15.152:0/2417926195 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b400dbd0 con 0x7fb4b40039b0 2024-04-20T11:33:08.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.335+0000 7fb4cea00640 1 -- 172.21.15.152:0/2417926195 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb4b400dd50 con 0x7fb4d006f510 2024-04-20T11:33:08.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.336+0000 7fb4cfe00640 1 --2- 172.21.15.152:0/2417926195 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb4b4004d40 0x7fb4b40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:08.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.336+0000 7fb4c6a00640 1 --2- 172.21.15.152:0/2417926195 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb4d006f510 0x7fb4b4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:08.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.336+0000 7fb4cf400640 1 --2- 172.21.15.152:0/2417926195 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4b40039b0 0x7fb4b4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:08.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.336+0000 7fb4cfe00640 1 -- 172.21.15.152:0/2417926195 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb4d006f510 msgr2=0x7fb4b4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:08.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.336+0000 7fb4cfe00640 1 --2- 172.21.15.152:0/2417926195 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb4d006f510 0x7fb4b4004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:08.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.336+0000 7fb4cfe00640 1 -- 172.21.15.152:0/2417926195 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4b40039b0 msgr2=0x7fb4b4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:08.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.336+0000 7fb4cfe00640 1 --2- 172.21.15.152:0/2417926195 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4b40039b0 0x7fb4b4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:08.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.336+0000 7fb4cfe00640 1 -- 172.21.15.152:0/2417926195 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4b4003520 con 0x7fb4b4004d40 2024-04-20T11:33:08.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.336+0000 7fb4cf400640 1 --2- 172.21.15.152:0/2417926195 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4b40039b0 0x7fb4b4003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:33:08.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.337+0000 7fb4cfe00640 1 --2- 172.21.15.152:0/2417926195 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb4b4004d40 0x7fb4b40051c0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7fb4c8009ac0 tx=0x7fb4b4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d10e5406c32745f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:08.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.337+0000 7fb4cea00640 1 -- 172.21.15.152:0/2417926195 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c8039070 con 0x7fb4b4004d40 2024-04-20T11:33:08.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.337+0000 7fb4cea00640 1 -- 172.21.15.152:0/2417926195 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4c802c710 con 0x7fb4b4004d40 2024-04-20T11:33:08.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.337+0000 7fb4cea00640 1 -- 172.21.15.152:0/2417926195 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c8009690 con 0x7fb4b4004d40 2024-04-20T11:33:08.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.338+0000 7fb4d4a00640 1 -- 172.21.15.152:0/2417926195 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb4b4004d40 msgr2=0x7fb4b40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:08.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.338+0000 7fb4d4a00640 1 --2- 172.21.15.152:0/2417926195 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb4b4004d40 0x7fb4b40051c0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7fb4c8009ac0 tx=0x7fb4b4005830 comp rx=0 tx=0).stop 2024-04-20T11:33:08.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.338+0000 7fb4cfe00640 1 -- 172.21.15.152:0/2417926195 reap_dead start 2024-04-20T11:33:08.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.338+0000 7fb4d4a00640 1 -- 172.21.15.152:0/2417926195 shutdown_connections 2024-04-20T11:33:08.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.338+0000 7fb4d4a00640 1 -- 172.21.15.152:0/2417926195 >> 172.21.15.152:0/2417926195 conn(0x7fb4d0067ae0 msgr2=0x7fb4d006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:08.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.338+0000 7fb4d4a00640 1 -- 172.21.15.152:0/2417926195 shutdown_connections 2024-04-20T11:33:08.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.338+0000 7fb4d4a00640 1 -- 172.21.15.152:0/2417926195 wait complete. 2024-04-20T11:33:08.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.340+0000 7fb4d4a00640 1 Processor -- start 2024-04-20T11:33:08.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.340+0000 7fb4d4a00640 1 -- start start 2024-04-20T11:33:08.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.341+0000 7fb4d4a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb4b4004d40 0x7fb4d013bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:08.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.341+0000 7fb4d4a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4d013d950 0x7fb4d013c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:08.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.341+0000 7fb4d4a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb4d01f3810 0x7fb4d01f3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:08.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.341+0000 7fb4d4a00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d00cf8e0 con 0x7fb4b4004d40 2024-04-20T11:33:08.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.341+0000 7fb4d4a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d00cf600 con 0x7fb4d013d950 2024-04-20T11:33:08.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.341+0000 7fb4d4a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb4d01417c0 con 0x7fb4d01f3810 2024-04-20T11:33:08.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.341+0000 7fb4cf400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb4b4004d40 0x7fb4d013bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:08.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.341+0000 7fb4cf400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb4b4004d40 0x7fb4d013bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36484/0 (socket says 172.21.15.152:36484) 2024-04-20T11:33:08.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.341+0000 7fb4cf400640 1 -- 172.21.15.152:0/2374204438 learned_addr learned my addr 172.21.15.152:0/2374204438 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:33:08.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.341+0000 7fb4cfe00640 1 --2- 172.21.15.152:0/2374204438 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb4d01f3810 0x7fb4d01f3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:08.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.342+0000 7fb4c6a00640 1 --2- 172.21.15.152:0/2374204438 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4d013d950 0x7fb4d013c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:08.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.342+0000 7fb4cf400640 1 -- 172.21.15.152:0/2374204438 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb4d01f3810 msgr2=0x7fb4d01f3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:08.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.342+0000 7fb4cf400640 1 --2- 172.21.15.152:0/2374204438 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb4d01f3810 0x7fb4d01f3c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:08.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.342+0000 7fb4cf400640 1 -- 172.21.15.152:0/2374204438 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4d013d950 msgr2=0x7fb4d013c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:08.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.342+0000 7fb4cf400640 1 --2- 172.21.15.152:0/2374204438 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4d013d950 0x7fb4d013c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:08.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.342+0000 7fb4cf400640 1 -- 172.21.15.152:0/2374204438 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4d013c8b0 con 0x7fb4b4004d40 2024-04-20T11:33:08.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.342+0000 7fb4c6a00640 1 --2- 172.21.15.152:0/2374204438 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4d013d950 0x7fb4d013c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:33:08.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.342+0000 7fb4cf400640 1 --2- 172.21.15.152:0/2374204438 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb4b4004d40 0x7fb4d013bcf0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7fb4c000dfd0 tx=0x7fb4c000b560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:08.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.342+0000 7fb4c7e00640 1 -- 172.21.15.152:0/2374204438 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c0019070 con 0x7fb4b4004d40 2024-04-20T11:33:08.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.342+0000 7fb4c7e00640 1 -- 172.21.15.152:0/2374204438 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4c0004d60 con 0x7fb4b4004d40 2024-04-20T11:33:08.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.342+0000 7fb4d4a00640 1 -- 172.21.15.152:0/2374204438 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4d013cb90 con 0x7fb4b4004d40 2024-04-20T11:33:08.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.342+0000 7fb4c7e00640 1 -- 172.21.15.152:0/2374204438 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c00072d0 con 0x7fb4b4004d40 2024-04-20T11:33:08.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.342+0000 7fb4d4a00640 1 -- 172.21.15.152:0/2374204438 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4d013d150 con 0x7fb4b4004d40 2024-04-20T11:33:08.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.343+0000 7fb4d4a00640 1 -- 172.21.15.152:0/2374204438 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4d0075e60 con 0x7fb4b4004d40 2024-04-20T11:33:08.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.344+0000 7fb4c7e00640 1 -- 172.21.15.152:0/2374204438 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 91142+0+0 (secure 0 0 0) 0x7fb4c001c070 con 0x7fb4b4004d40 2024-04-20T11:33:08.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.345+0000 7fb4c7e00640 1 --2- 172.21.15.152:0/2374204438 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb4ac06ca10 0x7fb4ac06eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:08.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.345+0000 7fb4c7e00640 1 -- 172.21.15.152:0/2374204438 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5795+0+0 (secure 0 0 0) 0x7fb4c008c200 con 0x7fb4b4004d40 2024-04-20T11:33:08.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.346+0000 7fb4c6a00640 1 --2- 172.21.15.152:0/2374204438 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb4ac06ca10 0x7fb4ac06eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:08.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.347+0000 7fb4c6a00640 1 --2- 172.21.15.152:0/2374204438 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb4ac06ca10 0x7fb4ac06eed0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb4d013dd50 tx=0x7fb4b8008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:08.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.350+0000 7fb4c7e00640 1 -- 172.21.15.152:0/2374204438 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fb4d00cf0a0 con 0x7fb4b4004d40 2024-04-20T11:33:08.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.592+0000 7fb4d4a00640 1 -- 172.21.15.152:0/2374204438 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb4d00b3490 con 0x7fb4b4004d40 2024-04-20T11:33:08.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.593+0000 7fb4c7e00640 1 -- 172.21.15.152:0/2374204438 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) ==== 94+0+5210 (secure 0 0 0) 0x7fb4c005a310 con 0x7fb4b4004d40 2024-04-20T11:33:08.594 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:33:08.594 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":58,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5465,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/3844264695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":3844264695},{"type":"v1","addr":"172.21.15.152:6835","nonce":3844264695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":5468,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/706098839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":706098839},{"type":"v1","addr":"172.21.15.175:6833","nonce":706098839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":6016,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/3381264900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3381264900},{"type":"v1","addr":"172.21.15.175:6835","nonce":3381264900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T11:32:41.508684+0000","modified":"2024-04-20T11:32:44.600495+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7332},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7332":{"gid":7332,"name":"c","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6833/3577228528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":3577228528},{"type":"v1","addr":"172.21.15.152:6833","nonce":3577228528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7332,"qdb_cluster":[7332]},"id":6}]} 2024-04-20T11:33:08.594 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 58 2024-04-20T11:33:08.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.596+0000 7fb4d4a00640 1 -- 172.21.15.152:0/2374204438 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb4ac06ca10 msgr2=0x7fb4ac06eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:08.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.597+0000 7fb4d4a00640 1 --2- 172.21.15.152:0/2374204438 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb4ac06ca10 0x7fb4ac06eed0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb4d013dd50 tx=0x7fb4b8008040 comp rx=0 tx=0).stop 2024-04-20T11:33:08.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.597+0000 7fb4d4a00640 1 -- 172.21.15.152:0/2374204438 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb4b4004d40 msgr2=0x7fb4d013bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:08.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.597+0000 7fb4d4a00640 1 --2- 172.21.15.152:0/2374204438 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb4b4004d40 0x7fb4d013bcf0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7fb4c000dfd0 tx=0x7fb4c000b560 comp rx=0 tx=0).stop 2024-04-20T11:33:08.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.597+0000 7fb4d4a00640 1 -- 172.21.15.152:0/2374204438 shutdown_connections 2024-04-20T11:33:08.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.597+0000 7fb4d4a00640 1 --2- 172.21.15.152:0/2374204438 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb4ac06ca10 0x7fb4ac06eed0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:08.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.597+0000 7fb4d4a00640 1 --2- 172.21.15.152:0/2374204438 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb4d01f3810 0x7fb4d01f3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:08.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.597+0000 7fb4d4a00640 1 --2- 172.21.15.152:0/2374204438 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4d013d950 0x7fb4d013c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:08.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.597+0000 7fb4d4a00640 1 --2- 172.21.15.152:0/2374204438 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb4b4004d40 0x7fb4d013bcf0 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:08.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.597+0000 7fb4d4a00640 1 -- 172.21.15.152:0/2374204438 >> 172.21.15.152:0/2374204438 conn(0x7fb4d0067ae0 msgr2=0x7fb4d0072420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:08.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.597+0000 7fb4d4a00640 1 -- 172.21.15.152:0/2374204438 shutdown_connections 2024-04-20T11:33:08.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.597+0000 7fb4d4a00640 1 -- 172.21.15.152:0/2374204438 wait complete. 2024-04-20T11:33:08.615 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:33:08.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.814+0000 7fbf80c00640 1 Processor -- start 2024-04-20T11:33:08.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.814+0000 7fbf80c00640 1 -- start start 2024-04-20T11:33:08.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.816+0000 7fbf80c00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fbf7c145f60 0x7fbf7c06ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:08.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.816+0000 7fbf80c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbf7c0cd0d0 0x7fbf7c0cd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:08.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.816+0000 7fbf80c00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf7c146340 con 0x7fbf7c071fe0 2024-04-20T11:33:08.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.816+0000 7fbf80c00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fbf7c06e9f0 con 0x7fbf7c145f60 2024-04-20T11:33:08.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.816+0000 7fbf80c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fbf7c06f3c0 con 0x7fbf7c0cd0d0 2024-04-20T11:33:08.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.817+0000 7fbf79400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbf7c0cd0d0 0x7fbf7c0cd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:08.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.817+0000 7fbf79e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fbf7c145f60 0x7fbf7c06ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:08.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.817+0000 7fbf79e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fbf7c145f60 0x7fbf7c06ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.152:57900/0 (socket says 172.21.15.152:57900) 2024-04-20T11:33:08.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.817+0000 7fbf79e00640 1 -- 172.21.15.152:0/2898329118 learned_addr learned my addr 172.21.15.152:0/2898329118 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:33:08.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.817+0000 7fbf79400640 1 -- 172.21.15.152:0/2898329118 >> v2:172.21.15.175:3300/0 conn(0x7fbf7c145f60 msgr2=0x7fbf7c06ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:08.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.817+0000 7fbf79400640 1 --2- 172.21.15.152:0/2898329118 >> v2:172.21.15.175:3300/0 conn(0x7fbf7c145f60 0x7fbf7c06ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:08.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.817+0000 7fbf79400640 1 -- 172.21.15.152:0/2898329118 >> v1:172.21.15.175:6789/0 conn(0x7fbf7c071fe0 legacy=0x7fbf7c072470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:08.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.818+0000 7fbf79400640 1 -- 172.21.15.152:0/2898329118 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf7c072d50 con 0x7fbf7c0cd0d0 2024-04-20T11:33:08.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.819+0000 7fbf79400640 1 --2- 172.21.15.152:0/2898329118 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbf7c0cd0d0 0x7fbf7c0cd550 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c06eb70 tx=0x7fbf6802b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=693620e01c79d4ea server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:08.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.819+0000 7fbf78a00640 1 -- 172.21.15.152:0/2898329118 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf68035020 con 0x7fbf7c0cd0d0 2024-04-20T11:33:08.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.819+0000 7fbf78a00640 1 -- 172.21.15.152:0/2898329118 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf68031810 con 0x7fbf7c0cd0d0 2024-04-20T11:33:08.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.819+0000 7fbf78a00640 1 -- 172.21.15.152:0/2898329118 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf68031be0 con 0x7fbf7c0cd0d0 2024-04-20T11:33:08.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.820+0000 7fbf80c00640 1 -- 172.21.15.152:0/2898329118 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbf7c0cd0d0 msgr2=0x7fbf7c0cd550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:08.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.820+0000 7fbf80c00640 1 --2- 172.21.15.152:0/2898329118 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbf7c0cd0d0 0x7fbf7c0cd550 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c06eb70 tx=0x7fbf6802b8b0 comp rx=0 tx=0).stop 2024-04-20T11:33:08.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.820+0000 7fbf80c00640 1 -- 172.21.15.152:0/2898329118 shutdown_connections 2024-04-20T11:33:08.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.820+0000 7fbf80c00640 1 --2- 172.21.15.152:0/2898329118 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbf7c0cd0d0 0x7fbf7c0cd550 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:08.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.820+0000 7fbf80c00640 1 --2- 172.21.15.152:0/2898329118 >> v2:172.21.15.175:3300/0 conn(0x7fbf7c145f60 0x7fbf7c06ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:08.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.820+0000 7fbf80c00640 1 -- 172.21.15.152:0/2898329118 >> 172.21.15.152:0/2898329118 conn(0x7fbf7c067cf0 msgr2=0x7fbf7c06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:08.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.820+0000 7fbf80c00640 1 -- 172.21.15.152:0/2898329118 shutdown_connections 2024-04-20T11:33:08.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.820+0000 7fbf80c00640 1 -- 172.21.15.152:0/2898329118 wait complete. 2024-04-20T11:33:08.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.822+0000 7fbf80c00640 1 Processor -- start 2024-04-20T11:33:08.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.822+0000 7fbf80c00640 1 -- start start 2024-04-20T11:33:08.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.823+0000 7fbf80c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbf7c071fe0 0x7fbf7c13f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:08.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.823+0000 7fbf80c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbf7c0cd0d0 0x7fbf7c141bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:08.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.823+0000 7fbf80c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbf7c145f60 0x7fbf7c142e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:08.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.823+0000 7fbf80c00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fbf7c06e680 con 0x7fbf7c0cd0d0 2024-04-20T11:33:08.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.823+0000 7fbf80c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbf7c06e3a0 con 0x7fbf7c145f60 2024-04-20T11:33:08.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.823+0000 7fbf80c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fbf7c0cf290 con 0x7fbf7c071fe0 2024-04-20T11:33:08.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.823+0000 7fbf79400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbf7c0cd0d0 0x7fbf7c141bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:08.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.823+0000 7fbf79400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbf7c0cd0d0 0x7fbf7c141bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36486/0 (socket says 172.21.15.152:36486) 2024-04-20T11:33:08.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.823+0000 7fbf79400640 1 -- 172.21.15.152:0/4174603691 learned_addr learned my addr 172.21.15.152:0/4174603691 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:33:08.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.824+0000 7fbf7a800640 1 --2- 172.21.15.152:0/4174603691 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbf7c145f60 0x7fbf7c142e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:08.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.824+0000 7fbf79e00640 1 --2- 172.21.15.152:0/4174603691 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbf7c071fe0 0x7fbf7c13f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:08.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.824+0000 7fbf79400640 1 -- 172.21.15.152:0/4174603691 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbf7c071fe0 msgr2=0x7fbf7c13f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:08.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.824+0000 7fbf79400640 1 --2- 172.21.15.152:0/4174603691 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbf7c071fe0 0x7fbf7c13f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:08.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.824+0000 7fbf79400640 1 -- 172.21.15.152:0/4174603691 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbf7c145f60 msgr2=0x7fbf7c142e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:08.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.824+0000 7fbf79400640 1 --2- 172.21.15.152:0/4174603691 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbf7c145f60 0x7fbf7c142e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:08.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.824+0000 7fbf79400640 1 -- 172.21.15.152:0/4174603691 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf7c140b10 con 0x7fbf7c0cd0d0 2024-04-20T11:33:08.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.824+0000 7fbf79400640 1 --2- 172.21.15.152:0/4174603691 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbf7c0cd0d0 0x7fbf7c141bb0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7fbf6802bec0 tx=0x7fbf680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:08.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.824+0000 7fbf72000640 1 -- 172.21.15.152:0/4174603691 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf68035040 con 0x7fbf7c0cd0d0 2024-04-20T11:33:08.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.824+0000 7fbf72000640 1 -- 172.21.15.152:0/4174603691 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf6803ad00 con 0x7fbf7c0cd0d0 2024-04-20T11:33:08.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.825+0000 7fbf80c00640 1 -- 172.21.15.152:0/4174603691 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf7c140df0 con 0x7fbf7c0cd0d0 2024-04-20T11:33:08.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.825+0000 7fbf72000640 1 -- 172.21.15.152:0/4174603691 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf68045650 con 0x7fbf7c0cd0d0 2024-04-20T11:33:08.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.825+0000 7fbf80c00640 1 -- 172.21.15.152:0/4174603691 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf7c1413b0 con 0x7fbf7c0cd0d0 2024-04-20T11:33:08.826 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.826+0000 7fbf80c00640 1 -- 172.21.15.152:0/4174603691 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf7c0689f0 con 0x7fbf7c0cd0d0 2024-04-20T11:33:08.826 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.827+0000 7fbf72000640 1 -- 172.21.15.152:0/4174603691 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 91142+0+0 (secure 0 0 0) 0x7fbf68038020 con 0x7fbf7c0cd0d0 2024-04-20T11:33:08.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.828+0000 7fbf72000640 1 --2- 172.21.15.152:0/4174603691 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fbf5c06ca10 0x7fbf5c06eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:08.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.828+0000 7fbf72000640 1 -- 172.21.15.152:0/4174603691 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5795+0+0 (secure 0 0 0) 0x7fbf680adb40 con 0x7fbf7c0cd0d0 2024-04-20T11:33:08.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.829+0000 7fbf79e00640 1 --2- 172.21.15.152:0/4174603691 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fbf5c06ca10 0x7fbf5c06eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:08.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.830+0000 7fbf79e00640 1 --2- 172.21.15.152:0/4174603691 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fbf5c06ca10 0x7fbf5c06eed0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c009a00 tx=0x7fbf6c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:08.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:08.832+0000 7fbf72000640 1 -- 172.21.15.152:0/4174603691 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fbf6807be00 con 0x7fbf7c0cd0d0 2024-04-20T11:33:09.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.030+0000 7fbf80c00640 1 -- 172.21.15.152:0/4174603691 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbf7c06bbd0 con 0x7fbf7c0cd0d0 2024-04-20T11:33:09.145 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.033+0000 7fbf72000640 1 -- 172.21.15.152:0/4174603691 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+16851 (secure 0 0 0) 0x7fbf6807b700 con 0x7fbf7c0cd0d0 2024-04-20T11:33:09.146 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:33:09.146 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":91,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:33:02.588760+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:32:39.268411+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-20T11:32:40.497900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.175:6835/2632275131":"2024-04-21T11:32:59.571722+0000","172.21.15.175:6834/2632275131":"2024-04-21T11:32:59.571722+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:33:09.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.036+0000 7fbf80c00640 1 -- 172.21.15.152:0/4174603691 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fbf5c06ca10 msgr2=0x7fbf5c06eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:09.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.036+0000 7fbf80c00640 1 --2- 172.21.15.152:0/4174603691 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fbf5c06ca10 0x7fbf5c06eed0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c009a00 tx=0x7fbf6c008040 comp rx=0 tx=0).stop 2024-04-20T11:33:09.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.036+0000 7fbf80c00640 1 -- 172.21.15.152:0/4174603691 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbf7c0cd0d0 msgr2=0x7fbf7c141bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:09.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.036+0000 7fbf80c00640 1 --2- 172.21.15.152:0/4174603691 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbf7c0cd0d0 0x7fbf7c141bb0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7fbf6802bec0 tx=0x7fbf680046c0 comp rx=0 tx=0).stop 2024-04-20T11:33:09.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.036+0000 7fbf80c00640 1 -- 172.21.15.152:0/4174603691 shutdown_connections 2024-04-20T11:33:09.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.036+0000 7fbf80c00640 1 --2- 172.21.15.152:0/4174603691 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fbf5c06ca10 0x7fbf5c06eed0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:09.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.036+0000 7fbf80c00640 1 --2- 172.21.15.152:0/4174603691 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbf7c145f60 0x7fbf7c142e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:09.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.036+0000 7fbf80c00640 1 --2- 172.21.15.152:0/4174603691 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbf7c0cd0d0 0x7fbf7c141bb0 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:09.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.036+0000 7fbf80c00640 1 --2- 172.21.15.152:0/4174603691 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbf7c071fe0 0x7fbf7c13f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:09.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.037+0000 7fbf80c00640 1 -- 172.21.15.152:0/4174603691 >> 172.21.15.152:0/4174603691 conn(0x7fbf7c067cf0 msgr2=0x7fbf7c0cdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:09.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.037+0000 7fbf80c00640 1 -- 172.21.15.152:0/4174603691 shutdown_connections 2024-04-20T11:33:09.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.037+0000 7fbf80c00640 1 -- 172.21.15.152:0/4174603691 wait complete. 2024-04-20T11:33:09.148 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T11:26:47.246571+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T11:32:39.268411+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T11:32:40.497900+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T11:33:09.149 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T11:33:09.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.356+0000 7f8fb6200640 1 Processor -- start 2024-04-20T11:33:09.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.357+0000 7f8fb6200640 1 -- start start 2024-04-20T11:33:09.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.358+0000 7f8fb6200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8fb00b2880 0x7f8fb00b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:09.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.358+0000 7f8fb6200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f8fb00cd1b0 0x7f8fb00cd600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:09.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.358+0000 7f8fb6200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8fb01463e0 con 0x7f8fb0146000 2024-04-20T11:33:09.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.358+0000 7f8fb6200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f8fb00b4750 con 0x7f8fb00cd1b0 2024-04-20T11:33:09.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.358+0000 7f8fb6200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8fb00b3e20 con 0x7f8fb00b2880 2024-04-20T11:33:09.359 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.359+0000 7f8fa6a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8fb00b2880 0x7f8fb00b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:09.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.359+0000 7f8fa6a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8fb00b2880 0x7f8fb00b4be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.152:38286/0 (socket says 172.21.15.152:38286) 2024-04-20T11:33:09.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.359+0000 7f8faf400640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7f8fb0146000 0x7f8fb00b2170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.152:41464/0 (socket says 172.21.15.152:41464) 2024-04-20T11:33:09.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.359+0000 7f8fa6a00640 1 -- 172.21.15.152:0/1034996854 learned_addr learned my addr 172.21.15.152:0/1034996854 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:33:09.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.359+0000 7f8fafe00640 1 --2- 172.21.15.152:0/1034996854 >> v2:172.21.15.175:3300/0 conn(0x7f8fb00cd1b0 0x7f8fb00cd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:09.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.360+0000 7f8fa6a00640 1 -- 172.21.15.152:0/1034996854 >> v2:172.21.15.175:3300/0 conn(0x7f8fb00cd1b0 msgr2=0x7f8fb00cd600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:09.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.360+0000 7f8fa6a00640 1 --2- 172.21.15.152:0/1034996854 >> v2:172.21.15.175:3300/0 conn(0x7f8fb00cd1b0 0x7f8fb00cd600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:09.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.360+0000 7f8fa6a00640 1 -- 172.21.15.152:0/1034996854 >> v1:172.21.15.175:6789/0 conn(0x7f8fb0146000 legacy=0x7f8fb00b2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:09.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.360+0000 7f8fa6a00640 1 -- 172.21.15.152:0/1034996854 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fb00b4100 con 0x7f8fb00b2880 2024-04-20T11:33:09.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.361+0000 7f8fa6a00640 1 --2- 172.21.15.152:0/1034996854 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8fb00b2880 0x7f8fb00b4be0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8f98009870 tx=0x7f8f9802b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93ab5783dd523b2c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:09.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.361+0000 7f8faea00640 1 -- 172.21.15.152:0/1034996854 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f98035020 con 0x7f8fb00b2880 2024-04-20T11:33:09.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.361+0000 7f8faea00640 1 -- 172.21.15.152:0/1034996854 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f98031640 con 0x7f8fb00b2880 2024-04-20T11:33:09.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.362+0000 7f8faea00640 1 -- 172.21.15.152:0/1034996854 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f9802d040 con 0x7f8fb00b2880 2024-04-20T11:33:09.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.362+0000 7f8fb6200640 1 -- 172.21.15.152:0/1034996854 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8fb00b2880 msgr2=0x7f8fb00b4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:09.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.362+0000 7f8fb6200640 1 --2- 172.21.15.152:0/1034996854 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8fb00b2880 0x7f8fb00b4be0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8f98009870 tx=0x7f8f9802b8f0 comp rx=0 tx=0).stop 2024-04-20T11:33:09.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.362+0000 7f8fb6200640 1 -- 172.21.15.152:0/1034996854 shutdown_connections 2024-04-20T11:33:09.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.362+0000 7f8fb6200640 1 --2- 172.21.15.152:0/1034996854 >> v2:172.21.15.175:3300/0 conn(0x7f8fb00cd1b0 0x7f8fb00cd600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:09.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.362+0000 7f8fb6200640 1 --2- 172.21.15.152:0/1034996854 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8fb00b2880 0x7f8fb00b4be0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:09.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.362+0000 7f8fb6200640 1 -- 172.21.15.152:0/1034996854 >> 172.21.15.152:0/1034996854 conn(0x7f8fb006b980 msgr2=0x7f8fb006bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:09.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.362+0000 7f8fb6200640 1 -- 172.21.15.152:0/1034996854 shutdown_connections 2024-04-20T11:33:09.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.362+0000 7f8fb6200640 1 -- 172.21.15.152:0/1034996854 wait complete. 2024-04-20T11:33:09.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.364+0000 7f8fb6200640 1 Processor -- start 2024-04-20T11:33:09.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.364+0000 7f8fb6200640 1 -- start start 2024-04-20T11:33:09.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.365+0000 7f8fb6200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8fb00b2880 0x7f8fb01e64c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:09.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.365+0000 7f8fb6200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8fb00cd1b0 0x7f8fb01446f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:09.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.365+0000 7f8fb6200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8fb0146000 0x7f8fb00b12e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:09.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.365+0000 7f8fb6200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb00cdcc0 con 0x7f8fb00cd1b0 2024-04-20T11:33:09.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.365+0000 7f8fb6200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb00cdb40 con 0x7f8fb00b2880 2024-04-20T11:33:09.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.365+0000 7f8fb6200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8fb015ccf0 con 0x7f8fb0146000 2024-04-20T11:33:09.366 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.366+0000 7f8faf400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8fb00b2880 0x7f8fb01e64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:09.366 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.366+0000 7f8fafe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8fb0146000 0x7f8fb00b12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:09.366 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.366+0000 7f8fa6a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8fb00cd1b0 0x7f8fb01446f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:09.366 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.366+0000 7f8faf400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8fb00b2880 0x7f8fb01e64c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.152:57924/0 (socket says 172.21.15.152:57924) 2024-04-20T11:33:09.366 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.366+0000 7f8faf400640 1 -- 172.21.15.152:0/55527967 learned_addr learned my addr 172.21.15.152:0/55527967 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:33:09.366 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.366+0000 7f8fa6a00640 1 -- 172.21.15.152:0/55527967 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8fb0146000 msgr2=0x7f8fb00b12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:09.366 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.366+0000 7f8fa6a00640 1 --2- 172.21.15.152:0/55527967 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8fb0146000 0x7f8fb00b12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:09.366 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.366+0000 7f8fa6a00640 1 -- 172.21.15.152:0/55527967 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8fb00b2880 msgr2=0x7f8fb01e64c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:09.366 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.366+0000 7f8fa6a00640 1 --2- 172.21.15.152:0/55527967 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8fb00b2880 0x7f8fb01e64c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:09.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.366+0000 7f8fa6a00640 1 -- 172.21.15.152:0/55527967 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fb01e5420 con 0x7f8fb00cd1b0 2024-04-20T11:33:09.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.367+0000 7f8fa6a00640 1 --2- 172.21.15.152:0/55527967 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8fb00cd1b0 0x7f8fb01446f0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f8f98033000 tx=0x7f8f980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:09.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.367+0000 7f8fa7e00640 1 -- 172.21.15.152:0/55527967 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f98040070 con 0x7f8fb00cd1b0 2024-04-20T11:33:09.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.367+0000 7f8fa7e00640 1 -- 172.21.15.152:0/55527967 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f980399f0 con 0x7f8fb00cd1b0 2024-04-20T11:33:09.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.367+0000 7f8fa7e00640 1 -- 172.21.15.152:0/55527967 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f98035040 con 0x7f8fb00cd1b0 2024-04-20T11:33:09.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.367+0000 7f8fb6200640 1 -- 172.21.15.152:0/55527967 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fb01e5700 con 0x7f8fb00cd1b0 2024-04-20T11:33:09.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.367+0000 7f8fb6200640 1 -- 172.21.15.152:0/55527967 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fb01e5cc0 con 0x7f8fb00cd1b0 2024-04-20T11:33:09.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.368+0000 7f8fb6200640 1 -- 172.21.15.152:0/55527967 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fb00689f0 con 0x7f8fb00cd1b0 2024-04-20T11:33:09.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.369+0000 7f8fa7e00640 1 -- 172.21.15.152:0/55527967 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 91142+0+0 (secure 0 0 0) 0x7f8f9804b060 con 0x7f8fb00cd1b0 2024-04-20T11:33:09.369 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.370+0000 7f8fa7e00640 1 --2- 172.21.15.152:0/55527967 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f8f8c06ca10 0x7f8f8c06eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:09.369 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.370+0000 7f8fa7e00640 1 -- 172.21.15.152:0/55527967 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5795+0+0 (secure 0 0 0) 0x7f8f980ad5f0 con 0x7f8fb00cd1b0 2024-04-20T11:33:09.370 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.371+0000 7f8faf400640 1 --2- 172.21.15.152:0/55527967 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f8f8c06ca10 0x7f8f8c06eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:09.371 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.372+0000 7f8faf400640 1 --2- 172.21.15.152:0/55527967 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f8f8c06ca10 0x7f8f8c06eed0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8fa0007fd0 tx=0x7f8fa00016a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:09.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.375+0000 7f8fa7e00640 1 -- 172.21.15.152:0/55527967 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f8f9807b250 con 0x7f8fb00cd1b0 2024-04-20T11:33:09.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.573+0000 7f8fb6200640 1 -- 172.21.15.152:0/55527967 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8fb00af690 con 0x7f8fb00cd1b0 2024-04-20T11:33:09.574 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.574+0000 7f8fa7e00640 1 -- 172.21.15.152:0/55527967 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) ==== 93+0+10 (secure 0 0 0) 0x7f8f9807f550 con 0x7f8fb00cd1b0 2024-04-20T11:33:09.574 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 1 2024-04-20T11:33:09.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.576+0000 7f8fb6200640 1 -- 172.21.15.152:0/55527967 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f8f8c06ca10 msgr2=0x7f8f8c06eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:09.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.576+0000 7f8fb6200640 1 --2- 172.21.15.152:0/55527967 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f8f8c06ca10 0x7f8f8c06eed0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8fa0007fd0 tx=0x7f8fa00016a0 comp rx=0 tx=0).stop 2024-04-20T11:33:09.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.576+0000 7f8fb6200640 1 -- 172.21.15.152:0/55527967 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8fb00cd1b0 msgr2=0x7f8fb01446f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:09.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.576+0000 7f8fb6200640 1 --2- 172.21.15.152:0/55527967 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8fb00cd1b0 0x7f8fb01446f0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f8f98033000 tx=0x7f8f980046c0 comp rx=0 tx=0).stop 2024-04-20T11:33:09.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.576+0000 7f8fb6200640 1 -- 172.21.15.152:0/55527967 shutdown_connections 2024-04-20T11:33:09.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.576+0000 7f8fb6200640 1 --2- 172.21.15.152:0/55527967 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f8f8c06ca10 0x7f8f8c06eed0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:09.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.576+0000 7f8fb6200640 1 --2- 172.21.15.152:0/55527967 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8fb0146000 0x7f8fb00b12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:09.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.576+0000 7f8fb6200640 1 --2- 172.21.15.152:0/55527967 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8fb00cd1b0 0x7f8fb01446f0 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:09.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.576+0000 7f8fb6200640 1 --2- 172.21.15.152:0/55527967 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8fb00b2880 0x7f8fb01e64c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:09.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.576+0000 7f8fb6200640 1 -- 172.21.15.152:0/55527967 >> 172.21.15.152:0/55527967 conn(0x7f8fb006b980 msgr2=0x7f8fb00ae9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:09.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.577+0000 7f8fb6200640 1 -- 172.21.15.152:0/55527967 shutdown_connections 2024-04-20T11:33:09.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.577+0000 7f8fb6200640 1 -- 172.21.15.152:0/55527967 wait complete. 2024-04-20T11:33:09.594 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T11:33:09.790 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.791+0000 7fd75f200640 1 Processor -- start 2024-04-20T11:33:09.791 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.791+0000 7fd75f200640 1 -- start start 2024-04-20T11:33:09.792 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.793+0000 7fd75f200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fd760167110 0x7fd760071cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:09.792 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.793+0000 7fd75f200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fd760072210 0x7fd76006e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:09.792 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.793+0000 7fd75f200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd760072610 con 0x7fd760162ac0 2024-04-20T11:33:09.792 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.793+0000 7fd75f200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fd7600b42f0 con 0x7fd760072210 2024-04-20T11:33:09.793 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.793+0000 7fd75f200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fd760167570 con 0x7fd760167110 2024-04-20T11:33:09.793 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.793+0000 7fd75e800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fd760072210 0x7fd76006e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:09.793 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.793+0000 7fd75e800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fd760072210 0x7fd76006e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36504/0 (socket says 172.21.15.152:36504) 2024-04-20T11:33:09.793 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.793+0000 7fd75e800640 1 -- 172.21.15.152:0/1037624681 learned_addr learned my addr 172.21.15.152:0/1037624681 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:33:09.794 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.793+0000 7fd75d400640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fd760167110 0x7fd760071cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:09.794 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.794+0000 7fd75e800640 1 -- 172.21.15.152:0/1037624681 >> v2:172.21.15.175:3300/0 conn(0x7fd760167110 msgr2=0x7fd760071cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:09.794 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.794+0000 7fd75e800640 1 --2- 172.21.15.152:0/1037624681 >> v2:172.21.15.175:3300/0 conn(0x7fd760167110 0x7fd760071cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:09.794 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.794+0000 7fd75e800640 1 -- 172.21.15.152:0/1037624681 >> v1:172.21.15.175:6789/0 conn(0x7fd760162ac0 legacy=0x7fd7600b46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:09.794 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.794+0000 7fd75e800640 1 -- 172.21.15.152:0/1037624681 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7600b3770 con 0x7fd760072210 2024-04-20T11:33:09.794 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.795+0000 7fd75e800640 1 --2- 172.21.15.152:0/1037624681 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd760072210 0x7fd76006e060 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7fd760072e20 tx=0x7fd75802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=257c4e67f20e8bd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:09.795 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.796+0000 7fd75ca00640 1 -- 172.21.15.152:0/1037624681 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd75802ca80 con 0x7fd760072210 2024-04-20T11:33:09.795 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.796+0000 7fd75ca00640 1 -- 172.21.15.152:0/1037624681 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd75802cc20 con 0x7fd760072210 2024-04-20T11:33:09.795 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.796+0000 7fd75ca00640 1 -- 172.21.15.152:0/1037624681 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7580343f0 con 0x7fd760072210 2024-04-20T11:33:09.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.796+0000 7fd75f200640 1 -- 172.21.15.152:0/1037624681 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd760072210 msgr2=0x7fd76006e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:09.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.796+0000 7fd75f200640 1 --2- 172.21.15.152:0/1037624681 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd760072210 0x7fd76006e060 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7fd760072e20 tx=0x7fd75802bee0 comp rx=0 tx=0).stop 2024-04-20T11:33:09.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.796+0000 7fd75f200640 1 -- 172.21.15.152:0/1037624681 shutdown_connections 2024-04-20T11:33:09.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.796+0000 7fd75f200640 1 --2- 172.21.15.152:0/1037624681 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd760072210 0x7fd76006e060 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:09.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.796+0000 7fd75f200640 1 --2- 172.21.15.152:0/1037624681 >> v2:172.21.15.175:3300/0 conn(0x7fd760167110 0x7fd760071cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:09.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.796+0000 7fd75f200640 1 -- 172.21.15.152:0/1037624681 >> 172.21.15.152:0/1037624681 conn(0x7fd760067ae0 msgr2=0x7fd76006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:09.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.797+0000 7fd75f200640 1 -- 172.21.15.152:0/1037624681 shutdown_connections 2024-04-20T11:33:09.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.797+0000 7fd75f200640 1 -- 172.21.15.152:0/1037624681 wait complete. 2024-04-20T11:33:09.798 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.798+0000 7fd75f200640 1 Processor -- start 2024-04-20T11:33:09.798 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.799+0000 7fd75f200640 1 -- start start 2024-04-20T11:33:09.800 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.799+0000 7fd75f200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd760072210 0x7fd76014f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:09.800 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.800+0000 7fd75f200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd760162ac0 0x7fd76014e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:09.800 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.800+0000 7fd75f200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd760167110 0x7fd7601654d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:09.800 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.800+0000 7fd75f200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fd7600b4010 con 0x7fd760072210 2024-04-20T11:33:09.800 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.800+0000 7fd75f200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd7600b3d30 con 0x7fd760167110 2024-04-20T11:33:09.800 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.800+0000 7fd75f200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd76006f530 con 0x7fd760162ac0 2024-04-20T11:33:09.800 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.800+0000 7fd75de00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd760072210 0x7fd76014f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:09.800 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.800+0000 7fd75de00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd760072210 0x7fd76014f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36510/0 (socket says 172.21.15.152:36510) 2024-04-20T11:33:09.800 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.800+0000 7fd75de00640 1 -- 172.21.15.152:0/1822076216 learned_addr learned my addr 172.21.15.152:0/1822076216 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:33:09.800 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.800+0000 7fd75e800640 1 --2- 172.21.15.152:0/1822076216 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd760167110 0x7fd7601654d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:09.800 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.800+0000 7fd75de00640 1 -- 172.21.15.152:0/1822076216 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd760162ac0 msgr2=0x7fd76014e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:09.800 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.800+0000 7fd75de00640 1 --2- 172.21.15.152:0/1822076216 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd760162ac0 0x7fd76014e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:09.801 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.800+0000 7fd75de00640 1 -- 172.21.15.152:0/1822076216 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd760167110 msgr2=0x7fd7601654d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:09.801 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.800+0000 7fd75de00640 1 --2- 172.21.15.152:0/1822076216 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd760167110 0x7fd7601654d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:09.801 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.800+0000 7fd75de00640 1 -- 172.21.15.152:0/1822076216 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd76014ec90 con 0x7fd760072210 2024-04-20T11:33:09.801 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.800+0000 7fd75e800640 1 --2- 172.21.15.152:0/1822076216 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd760167110 0x7fd7601654d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:33:09.801 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.801+0000 7fd75de00640 1 --2- 172.21.15.152:0/1822076216 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd760072210 0x7fd76014f1d0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7fd75000c400 tx=0x7fd75000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:09.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.801+0000 7fd756000640 1 -- 172.21.15.152:0/1822076216 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7500052d0 con 0x7fd760072210 2024-04-20T11:33:09.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.801+0000 7fd756000640 1 -- 172.21.15.152:0/1822076216 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd750005490 con 0x7fd760072210 2024-04-20T11:33:09.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.801+0000 7fd756000640 1 -- 172.21.15.152:0/1822076216 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd750007260 con 0x7fd760072210 2024-04-20T11:33:09.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.801+0000 7fd75f200640 1 -- 172.21.15.152:0/1822076216 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7600a95d0 con 0x7fd760072210 2024-04-20T11:33:09.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.801+0000 7fd75f200640 1 -- 172.21.15.152:0/1822076216 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7600a9b90 con 0x7fd760072210 2024-04-20T11:33:09.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.803+0000 7fd75f200640 1 -- 172.21.15.152:0/1822076216 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd760075e60 con 0x7fd760072210 2024-04-20T11:33:09.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.803+0000 7fd756000640 1 -- 172.21.15.152:0/1822076216 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 91142+0+0 (secure 0 0 0) 0x7fd750007400 con 0x7fd760072210 2024-04-20T11:33:09.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.804+0000 7fd756000640 1 --2- 172.21.15.152:0/1822076216 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd74006ca40 0x7fd74006ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:09.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.804+0000 7fd756000640 1 -- 172.21.15.152:0/1822076216 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5795+0+0 (secure 0 0 0) 0x7fd75008b270 con 0x7fd760072210 2024-04-20T11:33:09.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.805+0000 7fd75d400640 1 --2- 172.21.15.152:0/1822076216 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd74006ca40 0x7fd74006ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:09.805 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.806+0000 7fd75d400640 1 --2- 172.21.15.152:0/1822076216 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd74006ca40 0x7fd74006ef00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd74c006a90 tx=0x7fd74c006a20 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:09.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:09.809+0000 7fd756000640 1 -- 172.21.15.152:0/1822076216 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fd750059520 con 0x7fd760072210 2024-04-20T11:33:10.009 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.009+0000 7fd75f200640 1 -- 172.21.15.152:0/1822076216 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd76006ccb0 con 0x7fd760072210 2024-04-20T11:33:10.010 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.010+0000 7fd756000640 1 -- 172.21.15.152:0/1822076216 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) ==== 104+0+11 (secure 0 0 0) 0x7fd750058e20 con 0x7fd760072210 2024-04-20T11:33:10.010 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 32 2024-04-20T11:33:10.013 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.013+0000 7fd75f200640 1 -- 172.21.15.152:0/1822076216 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd74006ca40 msgr2=0x7fd74006ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:10.013 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.013+0000 7fd75f200640 1 --2- 172.21.15.152:0/1822076216 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd74006ca40 0x7fd74006ef00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd74c006a90 tx=0x7fd74c006a20 comp rx=0 tx=0).stop 2024-04-20T11:33:10.013 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.013+0000 7fd75f200640 1 -- 172.21.15.152:0/1822076216 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd760072210 msgr2=0x7fd76014f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:10.013 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.013+0000 7fd75f200640 1 --2- 172.21.15.152:0/1822076216 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd760072210 0x7fd76014f1d0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7fd75000c400 tx=0x7fd75000c8c0 comp rx=0 tx=0).stop 2024-04-20T11:33:10.013 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.013+0000 7fd75f200640 1 -- 172.21.15.152:0/1822076216 shutdown_connections 2024-04-20T11:33:10.013 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.013+0000 7fd75f200640 1 --2- 172.21.15.152:0/1822076216 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd74006ca40 0x7fd74006ef00 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:10.013 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.013+0000 7fd75f200640 1 --2- 172.21.15.152:0/1822076216 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd760167110 0x7fd7601654d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:10.013 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.013+0000 7fd75f200640 1 --2- 172.21.15.152:0/1822076216 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd760162ac0 0x7fd76014e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:10.013 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.013+0000 7fd75f200640 1 --2- 172.21.15.152:0/1822076216 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd760072210 0x7fd76014f1d0 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:10.013 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.013+0000 7fd75f200640 1 -- 172.21.15.152:0/1822076216 >> 172.21.15.152:0/1822076216 conn(0x7fd760067ae0 msgr2=0x7fd760160820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:10.014 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.013+0000 7fd75f200640 1 -- 172.21.15.152:0/1822076216 shutdown_connections 2024-04-20T11:33:10.014 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.013+0000 7fd75f200640 1 -- 172.21.15.152:0/1822076216 wait complete. 2024-04-20T11:33:10.032 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T11:33:10.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.236+0000 7f15d3200640 1 Processor -- start 2024-04-20T11:33:10.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.236+0000 7f15d3200640 1 -- start start 2024-04-20T11:33:10.237 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.238+0000 7f15d3200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f15d415ca40 0x7f15d415cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:10.237 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.238+0000 7f15d3200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f15d406bda0 0x7f15d406c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:10.237 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.238+0000 7f15d3200640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15d415d5c0 con 0x7f15d4156c00 2024-04-20T11:33:10.237 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.238+0000 7f15d3200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f15d406c750 con 0x7f15d406bda0 2024-04-20T11:33:10.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.238+0000 7f15d3200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f15d415d8a0 con 0x7f15d415ca40 2024-04-20T11:33:10.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.238+0000 7f15d2800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f15d406bda0 0x7f15d406c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:10.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.238+0000 7f15d1e00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f15d4156c00 0x7f15d4160a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:58204/0 (socket says 172.21.15.152:58204) 2024-04-20T11:33:10.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.238+0000 7f15d1e00640 1 -- 172.21.15.152:0/185783854 learned_addr learned my addr 172.21.15.152:0/185783854 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:33:10.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.239+0000 7f15cbe00640 1 --2- 172.21.15.152:0/185783854 >> v2:172.21.15.175:3300/0 conn(0x7f15d415ca40 0x7f15d415cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:10.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.239+0000 7f15d2800640 1 -- 172.21.15.152:0/185783854 >> v2:172.21.15.175:3300/0 conn(0x7f15d415ca40 msgr2=0x7f15d415cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:10.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.239+0000 7f15d2800640 1 --2- 172.21.15.152:0/185783854 >> v2:172.21.15.175:3300/0 conn(0x7f15d415ca40 0x7f15d415cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:10.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.239+0000 7f15d2800640 1 -- 172.21.15.152:0/185783854 >> v1:172.21.15.152:6789/0 conn(0x7f15d4156c00 legacy=0x7f15d4160a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:10.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.239+0000 7f15d2800640 1 -- 172.21.15.152:0/185783854 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15d4160290 con 0x7f15d406bda0 2024-04-20T11:33:10.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.240+0000 7f15d2800640 1 --2- 172.21.15.152:0/185783854 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f15d406bda0 0x7f15d406c210 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f15d40b2e20 tx=0x7f15cc02bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc702870dd205225 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:10.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.240+0000 7f15d1400640 1 -- 172.21.15.152:0/185783854 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15cc039070 con 0x7f15d406bda0 2024-04-20T11:33:10.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.240+0000 7f15d1400640 1 -- 172.21.15.152:0/185783854 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15cc02cc70 con 0x7f15d406bda0 2024-04-20T11:33:10.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.240+0000 7f15d1400640 1 -- 172.21.15.152:0/185783854 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15cc034660 con 0x7f15d406bda0 2024-04-20T11:33:10.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.241+0000 7f15d3200640 1 -- 172.21.15.152:0/185783854 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f15d406bda0 msgr2=0x7f15d406c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:10.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.241+0000 7f15d3200640 1 --2- 172.21.15.152:0/185783854 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f15d406bda0 0x7f15d406c210 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f15d40b2e20 tx=0x7f15cc02bec0 comp rx=0 tx=0).stop 2024-04-20T11:33:10.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.241+0000 7f15d3200640 1 -- 172.21.15.152:0/185783854 shutdown_connections 2024-04-20T11:33:10.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.241+0000 7f15d3200640 1 --2- 172.21.15.152:0/185783854 >> v2:172.21.15.175:3300/0 conn(0x7f15d415ca40 0x7f15d415cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:10.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.241+0000 7f15d3200640 1 --2- 172.21.15.152:0/185783854 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f15d406bda0 0x7f15d406c210 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:10.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.241+0000 7f15d3200640 1 -- 172.21.15.152:0/185783854 >> 172.21.15.152:0/185783854 conn(0x7f15d40b21e0 msgr2=0x7f15d40b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:10.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.241+0000 7f15d3200640 1 -- 172.21.15.152:0/185783854 shutdown_connections 2024-04-20T11:33:10.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.241+0000 7f15d3200640 1 -- 172.21.15.152:0/185783854 wait complete. 2024-04-20T11:33:10.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.243+0000 7f15d3200640 1 Processor -- start 2024-04-20T11:33:10.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.243+0000 7f15d3200640 1 -- start start 2024-04-20T11:33:10.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.244+0000 7f15d3200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f15d406bda0 0x7f15d409abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:10.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.244+0000 7f15d3200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15d4156c00 0x7f15d415eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:10.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.244+0000 7f15d3200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f15d415ca40 0x7f15d409d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:10.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.244+0000 7f15d3200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f15d40b3240 con 0x7f15d406bda0 2024-04-20T11:33:10.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.244+0000 7f15d3200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f15d40b30c0 con 0x7f15d4156c00 2024-04-20T11:33:10.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.244+0000 7f15d3200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f15d40b33c0 con 0x7f15d415ca40 2024-04-20T11:33:10.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.244+0000 7f15d1e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f15d406bda0 0x7f15d409abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:10.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.244+0000 7f15d1e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f15d406bda0 0x7f15d409abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36526/0 (socket says 172.21.15.152:36526) 2024-04-20T11:33:10.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.244+0000 7f15d1e00640 1 -- 172.21.15.152:0/224840882 learned_addr learned my addr 172.21.15.152:0/224840882 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:33:10.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.244+0000 7f15d1e00640 1 -- 172.21.15.152:0/224840882 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f15d415ca40 msgr2=0x7f15d409d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:10.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.244+0000 7f15d1e00640 1 --2- 172.21.15.152:0/224840882 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f15d415ca40 0x7f15d409d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:10.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.245+0000 7f15d1e00640 1 -- 172.21.15.152:0/224840882 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15d4156c00 msgr2=0x7f15d415eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:10.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.245+0000 7f15d1e00640 1 --2- 172.21.15.152:0/224840882 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15d4156c00 0x7f15d415eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:10.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.245+0000 7f15d1e00640 1 -- 172.21.15.152:0/224840882 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15d409cc30 con 0x7f15d406bda0 2024-04-20T11:33:10.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.245+0000 7f15d1e00640 1 --2- 172.21.15.152:0/224840882 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f15d406bda0 0x7f15d409abc0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f15c400c430 tx=0x7f15c400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:10.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.245+0000 7f15ca000640 1 -- 172.21.15.152:0/224840882 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c4017070 con 0x7f15d406bda0 2024-04-20T11:33:10.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.245+0000 7f15ca000640 1 -- 172.21.15.152:0/224840882 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15c4005ce0 con 0x7f15d406bda0 2024-04-20T11:33:10.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.245+0000 7f15ca000640 1 -- 172.21.15.152:0/224840882 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c40072f0 con 0x7f15d406bda0 2024-04-20T11:33:10.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.245+0000 7f15d3200640 1 -- 172.21.15.152:0/224840882 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15d409cf10 con 0x7f15d406bda0 2024-04-20T11:33:10.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.245+0000 7f15d3200640 1 -- 172.21.15.152:0/224840882 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15d409d4d0 con 0x7f15d406bda0 2024-04-20T11:33:10.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.246+0000 7f15d3200640 1 -- 172.21.15.152:0/224840882 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15d4075e60 con 0x7f15d406bda0 2024-04-20T11:33:10.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.247+0000 7f15ca000640 1 -- 172.21.15.152:0/224840882 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 91142+0+0 (secure 0 0 0) 0x7f15c401a070 con 0x7f15d406bda0 2024-04-20T11:33:10.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.248+0000 7f15ca000640 1 --2- 172.21.15.152:0/224840882 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f15b006ca10 0x7f15b006eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:10.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.248+0000 7f15ca000640 1 -- 172.21.15.152:0/224840882 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5795+0+0 (secure 0 0 0) 0x7f15c408b160 con 0x7f15d406bda0 2024-04-20T11:33:10.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.249+0000 7f15cbe00640 1 --2- 172.21.15.152:0/224840882 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f15b006ca10 0x7f15b006eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:10.249 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.250+0000 7f15cbe00640 1 --2- 172.21.15.152:0/224840882 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f15b006ca10 0x7f15b006eed0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f15bc006a90 tx=0x7f15bc006a20 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:10.252 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.252+0000 7f15ca000640 1 -- 172.21.15.152:0/224840882 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f15c4059410 con 0x7f15d406bda0 2024-04-20T11:33:10.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.451+0000 7f15d3200640 1 -- 172.21.15.152:0/224840882 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f15d406f390 con 0x7f15d406bda0 2024-04-20T11:33:10.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.452+0000 7f15ca000640 1 -- 172.21.15.152:0/224840882 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) ==== 100+0+11 (secure 0 0 0) 0x7f15c4058d10 con 0x7f15d406bda0 2024-04-20T11:33:10.452 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 64 2024-04-20T11:33:10.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.455+0000 7f15d3200640 1 -- 172.21.15.152:0/224840882 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f15b006ca10 msgr2=0x7f15b006eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:10.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.455+0000 7f15d3200640 1 --2- 172.21.15.152:0/224840882 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f15b006ca10 0x7f15b006eed0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f15bc006a90 tx=0x7f15bc006a20 comp rx=0 tx=0).stop 2024-04-20T11:33:10.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.455+0000 7f15d3200640 1 -- 172.21.15.152:0/224840882 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f15d406bda0 msgr2=0x7f15d409abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:10.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.455+0000 7f15d3200640 1 --2- 172.21.15.152:0/224840882 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f15d406bda0 0x7f15d409abc0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f15c400c430 tx=0x7f15c400c8f0 comp rx=0 tx=0).stop 2024-04-20T11:33:10.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.455+0000 7f15d3200640 1 -- 172.21.15.152:0/224840882 shutdown_connections 2024-04-20T11:33:10.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.455+0000 7f15d3200640 1 --2- 172.21.15.152:0/224840882 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f15b006ca10 0x7f15b006eed0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:10.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.455+0000 7f15d3200640 1 --2- 172.21.15.152:0/224840882 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f15d415ca40 0x7f15d409d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:10.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.455+0000 7f15d3200640 1 --2- 172.21.15.152:0/224840882 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15d4156c00 0x7f15d415eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:10.456 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.455+0000 7f15d3200640 1 --2- 172.21.15.152:0/224840882 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f15d406bda0 0x7f15d409abc0 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:10.456 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.455+0000 7f15d3200640 1 -- 172.21.15.152:0/224840882 >> 172.21.15.152:0/224840882 conn(0x7f15d40b21e0 msgr2=0x7f15d40735c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:10.456 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.455+0000 7f15d3200640 1 -- 172.21.15.152:0/224840882 shutdown_connections 2024-04-20T11:33:10.456 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.455+0000 7f15d3200640 1 -- 172.21.15.152:0/224840882 wait complete. 2024-04-20T11:33:10.474 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:33:10.678 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.679+0000 7f569ce00640 1 Processor -- start 2024-04-20T11:33:10.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.679+0000 7f569ce00640 1 -- start start 2024-04-20T11:33:10.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.680+0000 7f569ce00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f569815c560 0x7f56980b3df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:10.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.681+0000 7f569ce00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f569815fd20 con 0x7f56980d0810 2024-04-20T11:33:10.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.681+0000 7f569ce00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f569815ef60 con 0x7f56980b4330 2024-04-20T11:33:10.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.681+0000 7f569ce00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f569815f240 con 0x7f569815c560 2024-04-20T11:33:10.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.681+0000 7f5696800640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f56980d0810 0x7f56980ada90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:58212/0 (socket says 172.21.15.152:58212) 2024-04-20T11:33:10.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.681+0000 7f5696800640 1 -- 172.21.15.152:0/3352029920 learned_addr learned my addr 172.21.15.152:0/3352029920 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:33:10.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.681+0000 7f5695e00640 1 --2- 172.21.15.152:0/3352029920 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f569815c560 0x7f56980b3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:10.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.681+0000 7f5695400640 1 -- 172.21.15.152:0/3352029920 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2125826314 0 0) 0x7f569815fd20 con 0x7f56980d0810 2024-04-20T11:33:10.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.681+0000 7f5695400640 1 -- 172.21.15.152:0/3352029920 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5678003520 con 0x7f56980d0810 2024-04-20T11:33:10.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.682+0000 7f5695400640 1 -- 172.21.15.152:0/3352029920 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1254054880 0 0) 0x7f5678003520 con 0x7f56980d0810 2024-04-20T11:33:10.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.682+0000 7f5695400640 1 -- 172.21.15.152:0/3352029920 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f569815c560 msgr2=0x7f56980b3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:10.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.682+0000 7f5695400640 1 --2- 172.21.15.152:0/3352029920 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f569815c560 0x7f56980b3df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:10.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.682+0000 7f5695400640 1 -- 172.21.15.152:0/3352029920 >> v1:172.21.15.175:6789/0 conn(0x7f56980b4330 legacy=0x7f56980d0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:10.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.682+0000 7f5695400640 1 -- 172.21.15.152:0/3352029920 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f569815f520 con 0x7f56980d0810 2024-04-20T11:33:10.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.682+0000 7f5695400640 1 -- 172.21.15.152:0/3352029920 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f56900019f0 con 0x7f56980d0810 2024-04-20T11:33:10.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.682+0000 7f5695400640 1 -- 172.21.15.152:0/3352029920 >> v1:172.21.15.152:6789/0 conn(0x7f56980d0810 legacy=0x7f56980ada90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:10.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.682+0000 7f5695400640 1 --2- 172.21.15.152:0/3352029920 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5678003a50 0x7f5678003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:10.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.682+0000 7f5695400640 1 --2- 172.21.15.152:0/3352029920 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f569815c560 0x7f5678004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:10.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.682+0000 7f5695400640 1 --2- 172.21.15.152:0/3352029920 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5678004d40 0x7f56780051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:10.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.682+0000 7f5695400640 1 -- 172.21.15.152:0/3352029920 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f569815f240 con 0x7f5678004d40 2024-04-20T11:33:10.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.682+0000 7f5695400640 1 -- 172.21.15.152:0/3352029920 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f567800bba0 con 0x7f5678003a50 2024-04-20T11:33:10.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.682+0000 7f5695400640 1 -- 172.21.15.152:0/3352029920 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f567800bda0 con 0x7f569815c560 2024-04-20T11:33:10.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.682+0000 7f5696800640 1 --2- 172.21.15.152:0/3352029920 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5678004d40 0x7f56780051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:10.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.683+0000 7f5695e00640 1 --2- 172.21.15.152:0/3352029920 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5678003a50 0x7f5678003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:10.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.683+0000 7f568d400640 1 --2- 172.21.15.152:0/3352029920 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f569815c560 0x7f5678004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:10.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.683+0000 7f5696800640 1 -- 172.21.15.152:0/3352029920 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f569815c560 msgr2=0x7f5678004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:10.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.683+0000 7f5696800640 1 --2- 172.21.15.152:0/3352029920 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f569815c560 0x7f5678004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:10.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.683+0000 7f5696800640 1 -- 172.21.15.152:0/3352029920 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5678003a50 msgr2=0x7f5678003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:10.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.683+0000 7f5696800640 1 --2- 172.21.15.152:0/3352029920 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5678003a50 0x7f5678003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:10.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.683+0000 7f5696800640 1 -- 172.21.15.152:0/3352029920 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5678003520 con 0x7f5678004d40 2024-04-20T11:33:10.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.684+0000 7f5696800640 1 --2- 172.21.15.152:0/3352029920 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5678004d40 0x7f56780051c0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f569000a790 tx=0x7f5678005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daf820c9448407d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:10.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.684+0000 7f5695400640 1 -- 172.21.15.152:0/3352029920 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f569000a3b0 con 0x7f5678004d40 2024-04-20T11:33:10.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.684+0000 7f5695400640 1 -- 172.21.15.152:0/3352029920 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5690034650 con 0x7f5678004d40 2024-04-20T11:33:10.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.684+0000 7f5695400640 1 -- 172.21.15.152:0/3352029920 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5690034a40 con 0x7f5678004d40 2024-04-20T11:33:10.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.685+0000 7f569ce00640 1 -- 172.21.15.152:0/3352029920 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5678004d40 msgr2=0x7f56780051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:10.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.685+0000 7f569ce00640 1 --2- 172.21.15.152:0/3352029920 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5678004d40 0x7f56780051c0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f569000a790 tx=0x7f5678005830 comp rx=0 tx=0).stop 2024-04-20T11:33:10.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.685+0000 7f5696800640 1 -- 172.21.15.152:0/3352029920 reap_dead start 2024-04-20T11:33:10.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.685+0000 7f569ce00640 1 -- 172.21.15.152:0/3352029920 shutdown_connections 2024-04-20T11:33:10.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.685+0000 7f569ce00640 1 -- 172.21.15.152:0/3352029920 >> 172.21.15.152:0/3352029920 conn(0x7f5698067cf0 msgr2=0x7f569806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:10.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.685+0000 7f569ce00640 1 -- 172.21.15.152:0/3352029920 shutdown_connections 2024-04-20T11:33:10.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.685+0000 7f569ce00640 1 -- 172.21.15.152:0/3352029920 wait complete. 2024-04-20T11:33:10.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.687+0000 7f569ce00640 1 Processor -- start 2024-04-20T11:33:10.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.687+0000 7f569ce00640 1 -- start start 2024-04-20T11:33:10.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.688+0000 7f569ce00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5678004d40 0x7f569814f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:10.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.688+0000 7f569ce00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f569815dba0 0x7f56981509a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:10.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.688+0000 7f569ce00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5698150ee0 0x7f5698155d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:10.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.688+0000 7f569ce00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f56980b3df0 con 0x7f5698150ee0 2024-04-20T11:33:10.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.688+0000 7f569ce00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f56980d0dc0 con 0x7f569815dba0 2024-04-20T11:33:10.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.688+0000 7f569ce00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f56980b3f70 con 0x7f5678004d40 2024-04-20T11:33:10.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.688+0000 7f5696800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5698150ee0 0x7f5698155d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:10.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.688+0000 7f5696800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5698150ee0 0x7f5698155d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36538/0 (socket says 172.21.15.152:36538) 2024-04-20T11:33:10.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.688+0000 7f5696800640 1 -- 172.21.15.152:0/3049867854 learned_addr learned my addr 172.21.15.152:0/3049867854 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:33:10.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.688+0000 7f5695e00640 1 --2- 172.21.15.152:0/3049867854 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5678004d40 0x7f569814f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:10.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.689+0000 7f5696800640 1 -- 172.21.15.152:0/3049867854 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5678004d40 msgr2=0x7f569814f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:10.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.689+0000 7f5696800640 1 --2- 172.21.15.152:0/3049867854 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5678004d40 0x7f569814f1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:10.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.689+0000 7f5696800640 1 -- 172.21.15.152:0/3049867854 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f569815dba0 msgr2=0x7f56981509a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:10.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.689+0000 7f5696800640 1 --2- 172.21.15.152:0/3049867854 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f569815dba0 0x7f56981509a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:10.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.689+0000 7f5696800640 1 -- 172.21.15.152:0/3049867854 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f569815cb20 con 0x7f5698150ee0 2024-04-20T11:33:10.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.689+0000 7f5696800640 1 --2- 172.21.15.152:0/3049867854 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5698150ee0 0x7f5698155d60 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f569002cba0 tx=0x7f569815f6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:10.691 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.689+0000 7f568ea00640 1 -- 172.21.15.152:0/3049867854 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5690001250 con 0x7f5698150ee0 2024-04-20T11:33:10.691 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.689+0000 7f568ea00640 1 -- 172.21.15.152:0/3049867854 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5690044070 con 0x7f5698150ee0 2024-04-20T11:33:10.691 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.689+0000 7f568ea00640 1 -- 172.21.15.152:0/3049867854 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5690034800 con 0x7f5698150ee0 2024-04-20T11:33:10.691 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.689+0000 7f569ce00640 1 -- 172.21.15.152:0/3049867854 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f569815cde0 con 0x7f5698150ee0 2024-04-20T11:33:10.691 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.689+0000 7f569ce00640 1 -- 172.21.15.152:0/3049867854 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f569815d3a0 con 0x7f5698150ee0 2024-04-20T11:33:10.691 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.691+0000 7f569ce00640 1 -- 172.21.15.152:0/3049867854 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56980689f0 con 0x7f5698150ee0 2024-04-20T11:33:10.691 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.691+0000 7f568ea00640 1 -- 172.21.15.152:0/3049867854 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 91142+0+0 (secure 0 0 0) 0x7f56900349a0 con 0x7f5698150ee0 2024-04-20T11:33:10.692 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.692+0000 7f568ea00640 1 --2- 172.21.15.152:0/3049867854 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f567406cbb0 0x7f567406f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:10.692 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.692+0000 7f568ea00640 1 -- 172.21.15.152:0/3049867854 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5795+0+0 (secure 0 0 0) 0x7f56900ae290 con 0x7f5698150ee0 2024-04-20T11:33:10.692 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.693+0000 7f5695e00640 1 --2- 172.21.15.152:0/3049867854 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f567406cbb0 0x7f567406f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:10.693 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.694+0000 7f5695e00640 1 --2- 172.21.15.152:0/3049867854 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f567406cbb0 0x7f567406f070 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5688009950 tx=0x7f5688008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:10.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.697+0000 7f568ea00640 1 -- 172.21.15.152:0/3049867854 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f569007c550 con 0x7f5698150ee0 2024-04-20T11:33:10.938 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.939+0000 7f569ce00640 1 -- 172.21.15.152:0/3049867854 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5698159680 con 0x7f5698150ee0 2024-04-20T11:33:10.939 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.940+0000 7f568ea00640 1 -- 172.21.15.152:0/3049867854 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) ==== 94+0+5210 (secure 0 0 0) 0x7f569007be50 con 0x7f5698150ee0 2024-04-20T11:33:10.940 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:33:10.940 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":58,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5465,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/3844264695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":3844264695},{"type":"v1","addr":"172.21.15.152:6835","nonce":3844264695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":5468,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/706098839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":706098839},{"type":"v1","addr":"172.21.15.175:6833","nonce":706098839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":6016,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/3381264900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3381264900},{"type":"v1","addr":"172.21.15.175:6835","nonce":3381264900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T11:32:41.508684+0000","modified":"2024-04-20T11:32:44.600495+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7332},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7332":{"gid":7332,"name":"c","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6833/3577228528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":3577228528},{"type":"v1","addr":"172.21.15.152:6833","nonce":3577228528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7332,"qdb_cluster":[7332]},"id":6}]} 2024-04-20T11:33:10.940 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 58 2024-04-20T11:33:10.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.943+0000 7f569ce00640 1 -- 172.21.15.152:0/3049867854 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f567406cbb0 msgr2=0x7f567406f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:10.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.943+0000 7f569ce00640 1 --2- 172.21.15.152:0/3049867854 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f567406cbb0 0x7f567406f070 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5688009950 tx=0x7f5688008040 comp rx=0 tx=0).stop 2024-04-20T11:33:10.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.943+0000 7f569ce00640 1 -- 172.21.15.152:0/3049867854 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5698150ee0 msgr2=0x7f5698155d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:10.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.943+0000 7f569ce00640 1 --2- 172.21.15.152:0/3049867854 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5698150ee0 0x7f5698155d60 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f569002cba0 tx=0x7f569815f6f0 comp rx=0 tx=0).stop 2024-04-20T11:33:10.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.943+0000 7f569ce00640 1 -- 172.21.15.152:0/3049867854 shutdown_connections 2024-04-20T11:33:10.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.943+0000 7f569ce00640 1 --2- 172.21.15.152:0/3049867854 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f567406cbb0 0x7f567406f070 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:10.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.943+0000 7f569ce00640 1 --2- 172.21.15.152:0/3049867854 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5698150ee0 0x7f5698155d60 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:10.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.943+0000 7f569ce00640 1 --2- 172.21.15.152:0/3049867854 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f569815dba0 0x7f56981509a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:10.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.943+0000 7f569ce00640 1 --2- 172.21.15.152:0/3049867854 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5678004d40 0x7f569814f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:10.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.943+0000 7f569ce00640 1 -- 172.21.15.152:0/3049867854 >> 172.21.15.152:0/3049867854 conn(0x7f5698067cf0 msgr2=0x7f569806ed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:10.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.943+0000 7f569ce00640 1 -- 172.21.15.152:0/3049867854 shutdown_connections 2024-04-20T11:33:10.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:10.943+0000 7f569ce00640 1 -- 172.21.15.152:0/3049867854 wait complete. 2024-04-20T11:33:10.963 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:33:11.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.170+0000 7fc8cb200640 1 Processor -- start 2024-04-20T11:33:11.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.170+0000 7fc8cb200640 1 -- start start 2024-04-20T11:33:11.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.172+0000 7fc8cb200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fc8c40cd0d0 0x7fc8c40cd580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:11.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.172+0000 7fc8cb200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fc8c4071fe0 0x7fc8c4072470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:11.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.172+0000 7fc8cb200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8c4146340 con 0x7fc8c4145f60 2024-04-20T11:33:11.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.172+0000 7fc8cb200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fc8c406e9f0 con 0x7fc8c40cd0d0 2024-04-20T11:33:11.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.172+0000 7fc8cb200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fc8c4072d50 con 0x7fc8c4071fe0 2024-04-20T11:33:11.172 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.172+0000 7fc8c3400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fc8c40cd0d0 0x7fc8c40cd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:11.172 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.172+0000 7fc8c3400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fc8c40cd0d0 0x7fc8c40cd580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36540/0 (socket says 172.21.15.152:36540) 2024-04-20T11:33:11.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.172+0000 7fc8c3400640 1 -- 172.21.15.152:0/1094324740 learned_addr learned my addr 172.21.15.152:0/1094324740 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:33:11.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.172+0000 7fc8c9200640 1 --2- 172.21.15.152:0/1094324740 >> v2:172.21.15.175:3300/0 conn(0x7fc8c4071fe0 0x7fc8c4072470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:11.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.173+0000 7fc8c3400640 1 -- 172.21.15.152:0/1094324740 >> v2:172.21.15.175:3300/0 conn(0x7fc8c4071fe0 msgr2=0x7fc8c4072470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:11.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.173+0000 7fc8c3400640 1 --2- 172.21.15.152:0/1094324740 >> v2:172.21.15.175:3300/0 conn(0x7fc8c4071fe0 0x7fc8c4072470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:11.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.173+0000 7fc8c3400640 1 -- 172.21.15.152:0/1094324740 >> v1:172.21.15.175:6789/0 conn(0x7fc8c4145f60 legacy=0x7fc8c406ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:11.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.173+0000 7fc8c3400640 1 -- 172.21.15.152:0/1094324740 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8c406e0c0 con 0x7fc8c40cd0d0 2024-04-20T11:33:11.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.174+0000 7fc8c3400640 1 --2- 172.21.15.152:0/1094324740 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc8c40cd0d0 0x7fc8c40cd580 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fc8c406eb70 tx=0x7fc8ac02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb8783900d828bc4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:11.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.174+0000 7fc8c2a00640 1 -- 172.21.15.152:0/1094324740 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8ac035020 con 0x7fc8c40cd0d0 2024-04-20T11:33:11.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.174+0000 7fc8c2a00640 1 -- 172.21.15.152:0/1094324740 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8ac031870 con 0x7fc8c40cd0d0 2024-04-20T11:33:11.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.174+0000 7fc8c2a00640 1 -- 172.21.15.152:0/1094324740 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8ac031c40 con 0x7fc8c40cd0d0 2024-04-20T11:33:11.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.175+0000 7fc8cb200640 1 -- 172.21.15.152:0/1094324740 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc8c40cd0d0 msgr2=0x7fc8c40cd580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:11.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.175+0000 7fc8cb200640 1 --2- 172.21.15.152:0/1094324740 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc8c40cd0d0 0x7fc8c40cd580 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fc8c406eb70 tx=0x7fc8ac02b910 comp rx=0 tx=0).stop 2024-04-20T11:33:11.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.175+0000 7fc8cb200640 1 -- 172.21.15.152:0/1094324740 shutdown_connections 2024-04-20T11:33:11.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.175+0000 7fc8cb200640 1 --2- 172.21.15.152:0/1094324740 >> v2:172.21.15.175:3300/0 conn(0x7fc8c4071fe0 0x7fc8c4072470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:11.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.175+0000 7fc8cb200640 1 --2- 172.21.15.152:0/1094324740 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc8c40cd0d0 0x7fc8c40cd580 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:11.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.175+0000 7fc8cb200640 1 -- 172.21.15.152:0/1094324740 >> 172.21.15.152:0/1094324740 conn(0x7fc8c4067cf0 msgr2=0x7fc8c406b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:11.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.175+0000 7fc8cb200640 1 -- 172.21.15.152:0/1094324740 shutdown_connections 2024-04-20T11:33:11.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.175+0000 7fc8cb200640 1 -- 172.21.15.152:0/1094324740 wait complete. 2024-04-20T11:33:11.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.177+0000 7fc8cb200640 1 Processor -- start 2024-04-20T11:33:11.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.177+0000 7fc8cb200640 1 -- start start 2024-04-20T11:33:11.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.178+0000 7fc8cb200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc8c4071fe0 0x7fc8c40cfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:11.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.178+0000 7fc8cb200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc8c40cd0d0 0x7fc8c41e5a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:11.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.178+0000 7fc8cb200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8c4145f60 0x7fc8c413ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:11.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.178+0000 7fc8cb200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c40729b0 con 0x7fc8c4071fe0 2024-04-20T11:33:11.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.178+0000 7fc8cb200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c406e680 con 0x7fc8c4145f60 2024-04-20T11:33:11.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.178+0000 7fc8cb200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc8c4072b30 con 0x7fc8c40cd0d0 2024-04-20T11:33:11.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.178+0000 7fc8c3e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc8c4071fe0 0x7fc8c40cfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:11.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.178+0000 7fc8c3e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc8c4071fe0 0x7fc8c40cfcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36544/0 (socket says 172.21.15.152:36544) 2024-04-20T11:33:11.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.178+0000 7fc8c3e00640 1 -- 172.21.15.152:0/2047084636 learned_addr learned my addr 172.21.15.152:0/2047084636 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:33:11.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.179+0000 7fc8c9200640 1 --2- 172.21.15.152:0/2047084636 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8c4145f60 0x7fc8c413ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:11.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.179+0000 7fc8c3e00640 1 -- 172.21.15.152:0/2047084636 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc8c40cd0d0 msgr2=0x7fc8c41e5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:11.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.179+0000 7fc8c3400640 1 --2- 172.21.15.152:0/2047084636 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc8c40cd0d0 0x7fc8c41e5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:11.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.179+0000 7fc8c3e00640 1 --2- 172.21.15.152:0/2047084636 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc8c40cd0d0 0x7fc8c41e5a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:11.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.179+0000 7fc8c3e00640 1 -- 172.21.15.152:0/2047084636 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8c4145f60 msgr2=0x7fc8c413ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:11.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.179+0000 7fc8c3e00640 1 --2- 172.21.15.152:0/2047084636 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8c4145f60 0x7fc8c413ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:11.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.179+0000 7fc8c3e00640 1 -- 172.21.15.152:0/2047084636 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8c41e64b0 con 0x7fc8c4071fe0 2024-04-20T11:33:11.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.179+0000 7fc8c9200640 1 --2- 172.21.15.152:0/2047084636 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8c4145f60 0x7fc8c413ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:33:11.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.179+0000 7fc8c3400640 1 --2- 172.21.15.152:0/2047084636 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc8c40cd0d0 0x7fc8c41e5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:33:11.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.179+0000 7fc8c3e00640 1 --2- 172.21.15.152:0/2047084636 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc8c4071fe0 0x7fc8c40cfcd0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7fc8b400b420 tx=0x7fc8b400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:11.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.179+0000 7fc8bbe00640 1 -- 172.21.15.152:0/2047084636 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b4011830 con 0x7fc8c4071fe0 2024-04-20T11:33:11.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.179+0000 7fc8bbe00640 1 -- 172.21.15.152:0/2047084636 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8b4016070 con 0x7fc8c4071fe0 2024-04-20T11:33:11.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.179+0000 7fc8cb200640 1 -- 172.21.15.152:0/2047084636 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8c41f57e0 con 0x7fc8c4071fe0 2024-04-20T11:33:11.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.179+0000 7fc8bbe00640 1 -- 172.21.15.152:0/2047084636 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b4011c10 con 0x7fc8c4071fe0 2024-04-20T11:33:11.181 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.180+0000 7fc8cb200640 1 -- 172.21.15.152:0/2047084636 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8c41f5da0 con 0x7fc8c4071fe0 2024-04-20T11:33:11.181 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.181+0000 7fc8cb200640 1 -- 172.21.15.152:0/2047084636 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8c40cee20 con 0x7fc8c4071fe0 2024-04-20T11:33:11.181 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.182+0000 7fc8bbe00640 1 -- 172.21.15.152:0/2047084636 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 91142+0+0 (secure 0 0 0) 0x7fc8b401b460 con 0x7fc8c4071fe0 2024-04-20T11:33:11.182 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.182+0000 7fc8bbe00640 1 --2- 172.21.15.152:0/2047084636 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fc8a006ca10 0x7fc8a006eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:11.182 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.183+0000 7fc8bbe00640 1 -- 172.21.15.152:0/2047084636 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5795+0+0 (secure 0 0 0) 0x7fc8b408b870 con 0x7fc8c4071fe0 2024-04-20T11:33:11.182 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.183+0000 7fc8c3400640 1 --2- 172.21.15.152:0/2047084636 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fc8a006ca10 0x7fc8a006eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:11.184 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.184+0000 7fc8c3400640 1 --2- 172.21.15.152:0/2047084636 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fc8a006ca10 0x7fc8a006eed0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc8ac036000 tx=0x7fc8ac008030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:11.187 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.187+0000 7fc8bbe00640 1 -- 172.21.15.152:0/2047084636 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fc8c40cee20 con 0x7fc8c4071fe0 2024-04-20T11:33:11.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.387+0000 7fc8cb200640 1 -- 172.21.15.152:0/2047084636 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc8c406bbd0 con 0x7fc8c4071fe0 2024-04-20T11:33:11.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.389+0000 7fc8bbe00640 1 -- 172.21.15.152:0/2047084636 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+16851 (secure 0 0 0) 0x7fc8b4020020 con 0x7fc8c4071fe0 2024-04-20T11:33:11.389 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:33:11.389 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":91,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:33:02.588760+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:32:39.268411+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-20T11:32:40.497900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.175:6835/2632275131":"2024-04-21T11:32:59.571722+0000","172.21.15.175:6834/2632275131":"2024-04-21T11:32:59.571722+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:33:11.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.392+0000 7fc8cb200640 1 -- 172.21.15.152:0/2047084636 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fc8a006ca10 msgr2=0x7fc8a006eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:11.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.392+0000 7fc8cb200640 1 --2- 172.21.15.152:0/2047084636 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fc8a006ca10 0x7fc8a006eed0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc8ac036000 tx=0x7fc8ac008030 comp rx=0 tx=0).stop 2024-04-20T11:33:11.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.392+0000 7fc8cb200640 1 -- 172.21.15.152:0/2047084636 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc8c4071fe0 msgr2=0x7fc8c40cfcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:11.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.392+0000 7fc8cb200640 1 --2- 172.21.15.152:0/2047084636 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc8c4071fe0 0x7fc8c40cfcd0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7fc8b400b420 tx=0x7fc8b400b8e0 comp rx=0 tx=0).stop 2024-04-20T11:33:11.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.392+0000 7fc8cb200640 1 -- 172.21.15.152:0/2047084636 shutdown_connections 2024-04-20T11:33:11.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.392+0000 7fc8cb200640 1 --2- 172.21.15.152:0/2047084636 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fc8a006ca10 0x7fc8a006eed0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:11.393 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.392+0000 7fc8cb200640 1 --2- 172.21.15.152:0/2047084636 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8c4145f60 0x7fc8c413ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:11.393 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.392+0000 7fc8cb200640 1 --2- 172.21.15.152:0/2047084636 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc8c40cd0d0 0x7fc8c41e5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:11.393 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.392+0000 7fc8cb200640 1 --2- 172.21.15.152:0/2047084636 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc8c4071fe0 0x7fc8c40cfcd0 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:11.393 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.392+0000 7fc8cb200640 1 -- 172.21.15.152:0/2047084636 >> 172.21.15.152:0/2047084636 conn(0x7fc8c4067cf0 msgr2=0x7fc8c415bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:11.393 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.392+0000 7fc8cb200640 1 -- 172.21.15.152:0/2047084636 shutdown_connections 2024-04-20T11:33:11.393 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.393+0000 7fc8cb200640 1 -- 172.21.15.152:0/2047084636 wait complete. 2024-04-20T11:33:11.412 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-20T11:33:11.412 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:33:11.614 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.614+0000 7fa906600640 1 Processor -- start 2024-04-20T11:33:11.614 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.614+0000 7fa906600640 1 -- start start 2024-04-20T11:33:11.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.616+0000 7fa906600640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fa90015c560 0x7fa9000b3df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:11.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.616+0000 7fa906600640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fa9000b4330 0x7fa9000d0100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:11.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.616+0000 7fa906600640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa90015fd20 con 0x7fa9000d0640 2024-04-20T11:33:11.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.616+0000 7fa906600640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fa90015ef60 con 0x7fa90015c560 2024-04-20T11:33:11.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.616+0000 7fa906600640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fa90015f240 con 0x7fa9000b4330 2024-04-20T11:33:11.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.616+0000 7fa8ffe00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fa9000d0640 0x7fa900071db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:58218/0 (socket says 172.21.15.152:58218) 2024-04-20T11:33:11.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.616+0000 7fa8ffe00640 1 -- 172.21.15.152:0/537682887 learned_addr learned my addr 172.21.15.152:0/537682887 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:33:11.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.616+0000 7fa8ff400640 1 --2- 172.21.15.152:0/537682887 >> v2:172.21.15.152:3300/0 conn(0x7fa90015c560 0x7fa9000b3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:11.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.617+0000 7fa8fea00640 1 --2- 172.21.15.152:0/537682887 >> v2:172.21.15.175:3300/0 conn(0x7fa9000b4330 0x7fa9000d0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:11.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.617+0000 7fa8fe000640 1 -- 172.21.15.152:0/537682887 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1429751348 0 0) 0x7fa90015fd20 con 0x7fa9000d0640 2024-04-20T11:33:11.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.617+0000 7fa8fe000640 1 -- 172.21.15.152:0/537682887 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa8e4003520 con 0x7fa9000d0640 2024-04-20T11:33:11.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.617+0000 7fa8ff400640 1 -- 172.21.15.152:0/537682887 >> v2:172.21.15.175:3300/0 conn(0x7fa9000b4330 msgr2=0x7fa9000d0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:11.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.617+0000 7fa8ff400640 1 --2- 172.21.15.152:0/537682887 >> v2:172.21.15.175:3300/0 conn(0x7fa9000b4330 0x7fa9000d0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:11.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.617+0000 7fa8ff400640 1 -- 172.21.15.152:0/537682887 >> v1:172.21.15.152:6789/0 conn(0x7fa9000d0640 legacy=0x7fa900071db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:11.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.617+0000 7fa8ff400640 1 -- 172.21.15.152:0/537682887 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa90015f520 con 0x7fa90015c560 2024-04-20T11:33:11.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.617+0000 7fa8fea00640 1 --2- 172.21.15.152:0/537682887 >> v2:172.21.15.175:3300/0 conn(0x7fa9000b4330 0x7fa9000d0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:33:11.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.618+0000 7fa8ff400640 1 --2- 172.21.15.152:0/537682887 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa90015c560 0x7fa9000b3df0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7fa9000ccdf0 tx=0x7fa8f002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29df019538466251 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:11.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.618+0000 7fa8fe000640 1 -- 172.21.15.152:0/537682887 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8f00315a0 con 0x7fa90015c560 2024-04-20T11:33:11.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.619+0000 7fa8fe000640 1 -- 172.21.15.152:0/537682887 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8f0037070 con 0x7fa90015c560 2024-04-20T11:33:11.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.619+0000 7fa8fe000640 1 -- 172.21.15.152:0/537682887 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8f0031940 con 0x7fa90015c560 2024-04-20T11:33:11.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.619+0000 7fa906600640 1 -- 172.21.15.152:0/537682887 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa90015c560 msgr2=0x7fa9000b3df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:11.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.619+0000 7fa906600640 1 --2- 172.21.15.152:0/537682887 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa90015c560 0x7fa9000b3df0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7fa9000ccdf0 tx=0x7fa8f002b840 comp rx=0 tx=0).stop 2024-04-20T11:33:11.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.619+0000 7fa906600640 1 -- 172.21.15.152:0/537682887 shutdown_connections 2024-04-20T11:33:11.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.619+0000 7fa906600640 1 --2- 172.21.15.152:0/537682887 >> v2:172.21.15.175:3300/0 conn(0x7fa9000b4330 0x7fa9000d0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:11.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.619+0000 7fa906600640 1 --2- 172.21.15.152:0/537682887 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa90015c560 0x7fa9000b3df0 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:11.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.619+0000 7fa906600640 1 -- 172.21.15.152:0/537682887 >> 172.21.15.152:0/537682887 conn(0x7fa900067cf0 msgr2=0x7fa90006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:11.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.619+0000 7fa906600640 1 -- 172.21.15.152:0/537682887 shutdown_connections 2024-04-20T11:33:11.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.619+0000 7fa906600640 1 -- 172.21.15.152:0/537682887 wait complete. 2024-04-20T11:33:11.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.621+0000 7fa906600640 1 Processor -- start 2024-04-20T11:33:11.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.621+0000 7fa906600640 1 -- start start 2024-04-20T11:33:11.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.622+0000 7fa906600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa9000b4330 0x7fa9000c53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:11.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.622+0000 7fa906600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa9000d0640 0x7fa90014b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:11.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.622+0000 7fa906600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa90015c560 0x7fa9001487f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:11.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.622+0000 7fa906600640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fa9000d0ab0 con 0x7fa9000d0640 2024-04-20T11:33:11.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.622+0000 7fa906600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa9000d0dc0 con 0x7fa90015c560 2024-04-20T11:33:11.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.622+0000 7fa906600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa9000aeaa0 con 0x7fa9000b4330 2024-04-20T11:33:11.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.622+0000 7fa8fea00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa9000d0640 0x7fa90014b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:11.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.623+0000 7fa8fea00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa9000d0640 0x7fa90014b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36570/0 (socket says 172.21.15.152:36570) 2024-04-20T11:33:11.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.623+0000 7fa8fea00640 1 -- 172.21.15.152:0/826384928 learned_addr learned my addr 172.21.15.152:0/826384928 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:33:11.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.623+0000 7fa8ff400640 1 --2- 172.21.15.152:0/826384928 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa9000b4330 0x7fa9000c53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:11.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.623+0000 7fa8ffe00640 1 --2- 172.21.15.152:0/826384928 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa90015c560 0x7fa9001487f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:11.625 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.623+0000 7fa8fea00640 1 -- 172.21.15.152:0/826384928 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa9000b4330 msgr2=0x7fa9000c53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:11.625 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.623+0000 7fa8fea00640 1 --2- 172.21.15.152:0/826384928 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa9000b4330 0x7fa9000c53e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:11.625 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.623+0000 7fa8fea00640 1 -- 172.21.15.152:0/826384928 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa90015c560 msgr2=0x7fa9001487f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:11.625 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.623+0000 7fa8fea00640 1 --2- 172.21.15.152:0/826384928 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa90015c560 0x7fa9001487f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:11.625 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.623+0000 7fa8fea00640 1 -- 172.21.15.152:0/826384928 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa90014b8b0 con 0x7fa9000d0640 2024-04-20T11:33:11.625 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.623+0000 7fa8fea00640 1 --2- 172.21.15.152:0/826384928 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa9000d0640 0x7fa90014b190 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec00d7d0 tx=0x7fa8ec00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:11.625 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.624+0000 7fa8f7400640 1 -- 172.21.15.152:0/826384928 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8ec004030 con 0x7fa9000d0640 2024-04-20T11:33:11.625 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.624+0000 7fa8f7400640 1 -- 172.21.15.152:0/826384928 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8ec0041d0 con 0x7fa9000d0640 2024-04-20T11:33:11.625 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.624+0000 7fa8f7400640 1 -- 172.21.15.152:0/826384928 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8ec0096a0 con 0x7fa9000d0640 2024-04-20T11:33:11.625 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.624+0000 7fa906600640 1 -- 172.21.15.152:0/826384928 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa90014bb90 con 0x7fa9000d0640 2024-04-20T11:33:11.625 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.624+0000 7fa906600640 1 -- 172.21.15.152:0/826384928 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa900148d30 con 0x7fa9000d0640 2024-04-20T11:33:11.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.625+0000 7fa906600640 1 -- 172.21.15.152:0/826384928 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa90006beb0 con 0x7fa9000d0640 2024-04-20T11:33:11.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.626+0000 7fa8f7400640 1 -- 172.21.15.152:0/826384928 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 91142+0+0 (secure 0 0 0) 0x7fa8ec009840 con 0x7fa9000d0640 2024-04-20T11:33:11.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.627+0000 7fa8f7400640 1 --2- 172.21.15.152:0/826384928 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fa8e006ca40 0x7fa8e006ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:11.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.627+0000 7fa8f7400640 1 -- 172.21.15.152:0/826384928 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5795+0+0 (secure 0 0 0) 0x7fa8ec08bf90 con 0x7fa9000d0640 2024-04-20T11:33:11.627 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.627+0000 7fa8ff400640 1 --2- 172.21.15.152:0/826384928 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fa8e006ca40 0x7fa8e006ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:11.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.629+0000 7fa8ff400640 1 --2- 172.21.15.152:0/826384928 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fa8e006ca40 0x7fa8e006ef00 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa8f0036000 tx=0x7fa8f0008030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:11.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.631+0000 7fa8f7400640 1 -- 172.21.15.152:0/826384928 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fa90006beb0 con 0x7fa9000d0640 2024-04-20T11:33:11.877 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.877+0000 7fa906600640 1 -- 172.21.15.152:0/826384928 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa9000689f0 con 0x7fa9000d0640 2024-04-20T11:33:11.878 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.878+0000 7fa8f7400640 1 -- 172.21.15.152:0/826384928 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) ==== 94+0+5210 (secure 0 0 0) 0x7fa8ec05a250 con 0x7fa9000d0640 2024-04-20T11:33:11.878 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:33:11.878 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":58,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5465,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/3844264695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":3844264695},{"type":"v1","addr":"172.21.15.152:6835","nonce":3844264695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":5468,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/706098839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":706098839},{"type":"v1","addr":"172.21.15.175:6833","nonce":706098839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":6016,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/3381264900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3381264900},{"type":"v1","addr":"172.21.15.175:6835","nonce":3381264900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T11:32:41.508684+0000","modified":"2024-04-20T11:32:44.600495+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7332},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7332":{"gid":7332,"name":"c","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6833/3577228528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":3577228528},{"type":"v1","addr":"172.21.15.152:6833","nonce":3577228528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7332,"qdb_cluster":[7332]},"id":6}]} 2024-04-20T11:33:11.878 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 58 2024-04-20T11:33:11.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.881+0000 7fa906600640 1 -- 172.21.15.152:0/826384928 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fa8e006ca40 msgr2=0x7fa8e006ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:11.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.881+0000 7fa906600640 1 --2- 172.21.15.152:0/826384928 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fa8e006ca40 0x7fa8e006ef00 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa8f0036000 tx=0x7fa8f0008030 comp rx=0 tx=0).stop 2024-04-20T11:33:11.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.881+0000 7fa906600640 1 -- 172.21.15.152:0/826384928 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa9000d0640 msgr2=0x7fa90014b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:11.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.881+0000 7fa906600640 1 --2- 172.21.15.152:0/826384928 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa9000d0640 0x7fa90014b190 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec00d7d0 tx=0x7fa8ec00dc90 comp rx=0 tx=0).stop 2024-04-20T11:33:11.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.881+0000 7fa906600640 1 -- 172.21.15.152:0/826384928 shutdown_connections 2024-04-20T11:33:11.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.881+0000 7fa906600640 1 --2- 172.21.15.152:0/826384928 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fa8e006ca40 0x7fa8e006ef00 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:11.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.881+0000 7fa906600640 1 --2- 172.21.15.152:0/826384928 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa90015c560 0x7fa9001487f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:11.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.881+0000 7fa906600640 1 --2- 172.21.15.152:0/826384928 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa9000d0640 0x7fa90014b190 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:11.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.881+0000 7fa906600640 1 --2- 172.21.15.152:0/826384928 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa9000b4330 0x7fa9000c53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:11.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.881+0000 7fa906600640 1 -- 172.21.15.152:0/826384928 >> 172.21.15.152:0/826384928 conn(0x7fa900067cf0 msgr2=0x7fa900159450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:11.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.882+0000 7fa906600640 1 -- 172.21.15.152:0/826384928 shutdown_connections 2024-04-20T11:33:11.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:11.882+0000 7fa906600640 1 -- 172.21.15.152:0/826384928 wait complete. 2024-04-20T11:33:11.900 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:33:12.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.105+0000 7f381c600640 1 Processor -- start 2024-04-20T11:33:12.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.106+0000 7f381c600640 1 -- start start 2024-04-20T11:33:12.107 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.108+0000 7f381c600640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f38140b4500 0x7f38140d0100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:12.108 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.108+0000 7f381c600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f38140d0640 0x7f38140ada90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:12.108 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.108+0000 7f381c600640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f381415fd20 con 0x7f381415c560 2024-04-20T11:33:12.108 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.108+0000 7f381c600640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f381415ef60 con 0x7f38140b4500 2024-04-20T11:33:12.108 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.108+0000 7f381c600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f381415f240 con 0x7f38140d0640 2024-04-20T11:33:12.108 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.108+0000 7f3819c00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f381415c560 0x7f38140b3df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:58230/0 (socket says 172.21.15.152:58230) 2024-04-20T11:33:12.108 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.108+0000 7f3819c00640 1 -- 172.21.15.152:0/3908812166 learned_addr learned my addr 172.21.15.152:0/3908812166 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:33:12.108 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.108+0000 7f3819200640 1 --2- 172.21.15.152:0/3908812166 >> v2:172.21.15.152:3300/0 conn(0x7f38140b4500 0x7f38140d0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:12.109 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.109+0000 7f381a600640 1 --2- 172.21.15.152:0/3908812166 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f38140d0640 0x7f38140ada90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:12.109 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.109+0000 7f3813e00640 1 -- 172.21.15.152:0/3908812166 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1746680510 0 0) 0x7f381415fd20 con 0x7f381415c560 2024-04-20T11:33:12.109 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.109+0000 7f3819200640 1 -- 172.21.15.152:0/3908812166 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f38140d0640 msgr2=0x7f38140ada90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:12.109 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.109+0000 7f3819200640 1 --2- 172.21.15.152:0/3908812166 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f38140d0640 0x7f38140ada90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:12.109 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.109+0000 7f3819200640 1 -- 172.21.15.152:0/3908812166 >> v1:172.21.15.152:6789/0 conn(0x7f381415c560 legacy=0x7f38140b3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:12.109 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.109+0000 7f3819200640 1 -- 172.21.15.152:0/3908812166 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f381415f520 con 0x7f38140b4500 2024-04-20T11:33:12.111 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.110+0000 7f3819200640 1 --2- 172.21.15.152:0/3908812166 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f38140b4500 0x7f38140d0100 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f38000097a0 tx=0x7f380002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=452b59b92d75789 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:12.111 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.110+0000 7f3813e00640 1 -- 172.21.15.152:0/3908812166 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38000315a0 con 0x7f38140b4500 2024-04-20T11:33:12.111 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.110+0000 7f3813e00640 1 -- 172.21.15.152:0/3908812166 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3800037070 con 0x7f38140b4500 2024-04-20T11:33:12.111 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.110+0000 7f3813e00640 1 -- 172.21.15.152:0/3908812166 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3800031960 con 0x7f38140b4500 2024-04-20T11:33:12.111 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.111+0000 7f381c600640 1 -- 172.21.15.152:0/3908812166 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f38140b4500 msgr2=0x7f38140d0100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:12.111 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.111+0000 7f381c600640 1 --2- 172.21.15.152:0/3908812166 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f38140b4500 0x7f38140d0100 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f38000097a0 tx=0x7f380002b840 comp rx=0 tx=0).stop 2024-04-20T11:33:12.111 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.111+0000 7f381c600640 1 -- 172.21.15.152:0/3908812166 shutdown_connections 2024-04-20T11:33:12.112 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.111+0000 7f381c600640 1 --2- 172.21.15.152:0/3908812166 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f38140d0640 0x7f38140ada90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:12.112 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.111+0000 7f381c600640 1 --2- 172.21.15.152:0/3908812166 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f38140b4500 0x7f38140d0100 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:12.112 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.111+0000 7f381c600640 1 -- 172.21.15.152:0/3908812166 >> 172.21.15.152:0/3908812166 conn(0x7f3814067cf0 msgr2=0x7f381406b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:12.112 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.111+0000 7f381c600640 1 -- 172.21.15.152:0/3908812166 shutdown_connections 2024-04-20T11:33:12.112 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.111+0000 7f381c600640 1 -- 172.21.15.152:0/3908812166 wait complete. 2024-04-20T11:33:12.113 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.113+0000 7f381c600640 1 Processor -- start 2024-04-20T11:33:12.113 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.113+0000 7f381c600640 1 -- start start 2024-04-20T11:33:12.114 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.114+0000 7f381c600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38140b4500 0x7f38140b2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:12.114 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.114+0000 7f381c600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f38140d0640 0x7f38140cce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:12.114 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.114+0000 7f381c600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f381415c560 0x7f38140ce640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:12.114 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.114+0000 7f381c600640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f38140d0ad0 con 0x7f381415c560 2024-04-20T11:33:12.114 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.114+0000 7f381c600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f38140d0dc0 con 0x7f38140b4500 2024-04-20T11:33:12.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.114+0000 7f381c600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f38140adfd0 con 0x7f38140d0640 2024-04-20T11:33:12.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.114+0000 7f381a600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f381415c560 0x7f38140ce640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:12.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.114+0000 7f381a600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f381415c560 0x7f38140ce640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36586/0 (socket says 172.21.15.152:36586) 2024-04-20T11:33:12.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.114+0000 7f381a600640 1 -- 172.21.15.152:0/145759205 learned_addr learned my addr 172.21.15.152:0/145759205 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:33:12.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.115+0000 7f3819c00640 1 --2- 172.21.15.152:0/145759205 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38140b4500 0x7f38140b2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:12.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.115+0000 7f3819200640 1 --2- 172.21.15.152:0/145759205 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f38140d0640 0x7f38140cce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:12.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.115+0000 7f381a600640 1 -- 172.21.15.152:0/145759205 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f38140d0640 msgr2=0x7f38140cce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:12.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.115+0000 7f381a600640 1 --2- 172.21.15.152:0/145759205 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f38140d0640 0x7f38140cce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:12.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.115+0000 7f381a600640 1 -- 172.21.15.152:0/145759205 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38140b4500 msgr2=0x7f38140b2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:12.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.115+0000 7f381a600640 1 --2- 172.21.15.152:0/145759205 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38140b4500 0x7f38140b2410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:12.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.115+0000 7f381a600640 1 -- 172.21.15.152:0/145759205 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38140ceb80 con 0x7f381415c560 2024-04-20T11:33:12.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.115+0000 7f3819200640 1 --2- 172.21.15.152:0/145759205 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f38140d0640 0x7f38140cce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:33:12.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.115+0000 7f381a600640 1 --2- 172.21.15.152:0/145759205 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f381415c560 0x7f38140ce640 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f380c007e40 tx=0x7f380c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:12.116 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.115+0000 7f3811600640 1 -- 172.21.15.152:0/145759205 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f380c014620 con 0x7f381415c560 2024-04-20T11:33:12.116 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.115+0000 7f3811600640 1 -- 172.21.15.152:0/145759205 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f380c019070 con 0x7f381415c560 2024-04-20T11:33:12.116 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.115+0000 7f3811600640 1 -- 172.21.15.152:0/145759205 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f380c014a20 con 0x7f381415c560 2024-04-20T11:33:12.116 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.115+0000 7f381c600640 1 -- 172.21.15.152:0/145759205 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38140cd5a0 con 0x7f381415c560 2024-04-20T11:33:12.116 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.115+0000 7f381c600640 1 -- 172.21.15.152:0/145759205 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38140cdb60 con 0x7f381415c560 2024-04-20T11:33:12.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.116+0000 7f381c600640 1 -- 172.21.15.152:0/145759205 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38140689f0 con 0x7f381415c560 2024-04-20T11:33:12.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.118+0000 7f3811600640 1 -- 172.21.15.152:0/145759205 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 91142+0+0 (secure 0 0 0) 0x7f380c014bc0 con 0x7f381415c560 2024-04-20T11:33:12.118 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.119+0000 7f3811600640 1 --2- 172.21.15.152:0/145759205 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f37f406ca10 0x7f37f406eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:12.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.119+0000 7f3811600640 1 -- 172.21.15.152:0/145759205 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5795+0+0 (secure 0 0 0) 0x7f380c08c7a0 con 0x7f381415c560 2024-04-20T11:33:12.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.120+0000 7f3819c00640 1 --2- 172.21.15.152:0/145759205 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f37f406ca10 0x7f37f406eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:12.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.121+0000 7f3819c00640 1 --2- 172.21.15.152:0/145759205 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f37f406ca10 0x7f37f406eed0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3808000f60 tx=0x7f3808007c00 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:12.123 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.124+0000 7f3811600640 1 -- 172.21.15.152:0/145759205 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f380c05a930 con 0x7f381415c560 2024-04-20T11:33:12.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.358+0000 7f381c600640 1 -- 172.21.15.152:0/145759205 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f381406df40 con 0x7f381415c560 2024-04-20T11:33:12.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.360+0000 7f3811600640 1 -- 172.21.15.152:0/145759205 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) ==== 94+0+5210 (secure 0 0 0) 0x7f380c05a230 con 0x7f381415c560 2024-04-20T11:33:12.360 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:33:12.360 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":58,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5465,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/3844264695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":3844264695},{"type":"v1","addr":"172.21.15.152:6835","nonce":3844264695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":5468,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/706098839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":706098839},{"type":"v1","addr":"172.21.15.175:6833","nonce":706098839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":6016,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/3381264900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3381264900},{"type":"v1","addr":"172.21.15.175:6835","nonce":3381264900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T11:32:41.508684+0000","modified":"2024-04-20T11:32:44.600495+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7332},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7332":{"gid":7332,"name":"c","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6833/3577228528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":3577228528},{"type":"v1","addr":"172.21.15.152:6833","nonce":3577228528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7332,"qdb_cluster":[7332]},"id":6}]} 2024-04-20T11:33:12.360 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 58 2024-04-20T11:33:12.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.362+0000 7f381c600640 1 -- 172.21.15.152:0/145759205 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f37f406ca10 msgr2=0x7f37f406eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:12.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.362+0000 7f381c600640 1 --2- 172.21.15.152:0/145759205 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f37f406ca10 0x7f37f406eed0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3808000f60 tx=0x7f3808007c00 comp rx=0 tx=0).stop 2024-04-20T11:33:12.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.362+0000 7f381c600640 1 -- 172.21.15.152:0/145759205 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f381415c560 msgr2=0x7f38140ce640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:12.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.362+0000 7f381c600640 1 --2- 172.21.15.152:0/145759205 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f381415c560 0x7f38140ce640 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f380c007e40 tx=0x7f380c00e4b0 comp rx=0 tx=0).stop 2024-04-20T11:33:12.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.363+0000 7f381c600640 1 -- 172.21.15.152:0/145759205 shutdown_connections 2024-04-20T11:33:12.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.363+0000 7f381c600640 1 --2- 172.21.15.152:0/145759205 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f37f406ca10 0x7f37f406eed0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:12.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.363+0000 7f381c600640 1 --2- 172.21.15.152:0/145759205 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f381415c560 0x7f38140ce640 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:12.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.363+0000 7f381c600640 1 --2- 172.21.15.152:0/145759205 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f38140d0640 0x7f38140cce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:12.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.363+0000 7f381c600640 1 --2- 172.21.15.152:0/145759205 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38140b4500 0x7f38140b2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:12.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.363+0000 7f381c600640 1 -- 172.21.15.152:0/145759205 >> 172.21.15.152:0/145759205 conn(0x7f3814067cf0 msgr2=0x7f38140cc660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:12.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.363+0000 7f381c600640 1 -- 172.21.15.152:0/145759205 shutdown_connections 2024-04-20T11:33:12.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.363+0000 7f381c600640 1 -- 172.21.15.152:0/145759205 wait complete. 2024-04-20T11:33:12.382 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:33:12.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.573+0000 7f5cd1200640 1 Processor -- start 2024-04-20T11:33:12.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.573+0000 7f5cd1200640 1 -- start start 2024-04-20T11:33:12.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.575+0000 7f5cd1200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f5ccc0b4500 0x7f5ccc0d0100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:12.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.575+0000 7f5cd1200640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ccc15fd20 con 0x7f5ccc15c560 2024-04-20T11:33:12.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.575+0000 7f5cd1200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ccc15ef60 con 0x7f5ccc0d0640 2024-04-20T11:33:12.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.575+0000 7f5cd1200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f5ccc15f240 con 0x7f5ccc0b4500 2024-04-20T11:33:12.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.575+0000 7f5cc3e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f5ccc0b4500 0x7f5ccc0d0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:12.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.575+0000 7f5cc9e00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f5ccc15c560 0x7f5ccc0b3df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:58242/0 (socket says 172.21.15.152:58242) 2024-04-20T11:33:12.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.575+0000 7f5cc9e00640 1 -- 172.21.15.152:0/2906175403 learned_addr learned my addr 172.21.15.152:0/2906175403 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:33:12.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.576+0000 7f5cc3e00640 1 -- 172.21.15.152:0/2906175403 >> v1:172.21.15.175:6789/0 conn(0x7f5ccc0d0640 legacy=0x7f5ccc0ada90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:12.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.576+0000 7f5cc3e00640 1 -- 172.21.15.152:0/2906175403 >> v1:172.21.15.152:6789/0 conn(0x7f5ccc15c560 legacy=0x7f5ccc0b3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:12.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.576+0000 7f5cc3e00640 1 -- 172.21.15.152:0/2906175403 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ccc15f520 con 0x7f5ccc0b4500 2024-04-20T11:33:12.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.576+0000 7f5cc9400640 1 -- 172.21.15.152:0/2906175403 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1729822079 0 0) 0x7f5ccc15fd20 con 0x7f5ccc15c560 2024-04-20T11:33:12.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.577+0000 7f5cc3e00640 1 --2- 172.21.15.152:0/2906175403 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5ccc0b4500 0x7f5ccc0d0100 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f5cb40097a0 tx=0x7f5cb402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f9847c0cebd4552 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:12.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.577+0000 7f5cc9400640 1 -- 172.21.15.152:0/2906175403 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb40315a0 con 0x7f5ccc0b4500 2024-04-20T11:33:12.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.577+0000 7f5cc9400640 1 -- 172.21.15.152:0/2906175403 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5cb4037070 con 0x7f5ccc0b4500 2024-04-20T11:33:12.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.577+0000 7f5cc9400640 1 -- 172.21.15.152:0/2906175403 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb4031960 con 0x7f5ccc0b4500 2024-04-20T11:33:12.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.578+0000 7f5cd1200640 1 -- 172.21.15.152:0/2906175403 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5ccc0b4500 msgr2=0x7f5ccc0d0100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:12.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.578+0000 7f5cd1200640 1 --2- 172.21.15.152:0/2906175403 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5ccc0b4500 0x7f5ccc0d0100 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f5cb40097a0 tx=0x7f5cb402b840 comp rx=0 tx=0).stop 2024-04-20T11:33:12.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.578+0000 7f5cd1200640 1 -- 172.21.15.152:0/2906175403 shutdown_connections 2024-04-20T11:33:12.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.578+0000 7f5cd1200640 1 --2- 172.21.15.152:0/2906175403 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5ccc0b4500 0x7f5ccc0d0100 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:12.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.578+0000 7f5cd1200640 1 -- 172.21.15.152:0/2906175403 >> 172.21.15.152:0/2906175403 conn(0x7f5ccc067cf0 msgr2=0x7f5ccc06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:12.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.578+0000 7f5cd1200640 1 -- 172.21.15.152:0/2906175403 shutdown_connections 2024-04-20T11:33:12.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.578+0000 7f5cd1200640 1 -- 172.21.15.152:0/2906175403 wait complete. 2024-04-20T11:33:12.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.580+0000 7f5cd1200640 1 Processor -- start 2024-04-20T11:33:12.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.580+0000 7f5cd1200640 1 -- start start 2024-04-20T11:33:12.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.581+0000 7f5cd1200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ccc0b4500 0x7f5ccc14f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:12.582 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.581+0000 7f5cd1200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ccc0d0640 0x7f5ccc152b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:12.582 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.581+0000 7f5cd1200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5ccc15c560 0x7f5ccc15af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:12.582 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.581+0000 7f5cd1200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f5ccc0d0ad0 con 0x7f5ccc15c560 2024-04-20T11:33:12.582 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.581+0000 7f5cd1200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5ccc0d0dc0 con 0x7f5ccc0b4500 2024-04-20T11:33:12.582 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.581+0000 7f5cd1200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5ccc0ae1a0 con 0x7f5ccc0d0640 2024-04-20T11:33:12.582 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.581+0000 7f5cca800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5ccc15c560 0x7f5ccc15af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:12.582 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.581+0000 7f5cca800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5ccc15c560 0x7f5ccc15af70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36598/0 (socket says 172.21.15.152:36598) 2024-04-20T11:33:12.582 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.581+0000 7f5cca800640 1 -- 172.21.15.152:0/2636862727 learned_addr learned my addr 172.21.15.152:0/2636862727 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:33:12.583 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.581+0000 7f5cc9e00640 1 --2- 172.21.15.152:0/2636862727 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ccc0b4500 0x7f5ccc14f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:12.583 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.582+0000 7f5cca800640 1 -- 172.21.15.152:0/2636862727 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ccc0d0640 msgr2=0x7f5ccc152b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:12.583 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.582+0000 7f5cca800640 1 --2- 172.21.15.152:0/2636862727 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ccc0d0640 0x7f5ccc152b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:12.583 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.582+0000 7f5cca800640 1 -- 172.21.15.152:0/2636862727 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ccc0b4500 msgr2=0x7f5ccc14f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:12.583 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.582+0000 7f5cca800640 1 --2- 172.21.15.152:0/2636862727 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ccc0b4500 0x7f5ccc14f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:12.583 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.582+0000 7f5cca800640 1 -- 172.21.15.152:0/2636862727 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ccc156ba0 con 0x7f5ccc15c560 2024-04-20T11:33:12.583 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.582+0000 7f5cca800640 1 --2- 172.21.15.152:0/2636862727 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5ccc15c560 0x7f5ccc15af70 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f5cc400c7b0 tx=0x7f5cc400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:12.583 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.582+0000 7f5cc2000640 1 -- 172.21.15.152:0/2636862727 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc4004b60 con 0x7f5ccc15c560 2024-04-20T11:33:12.583 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.582+0000 7f5cc2000640 1 -- 172.21.15.152:0/2636862727 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5cc4004d00 con 0x7f5ccc15c560 2024-04-20T11:33:12.583 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.582+0000 7f5cc2000640 1 -- 172.21.15.152:0/2636862727 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc400aa90 con 0x7f5ccc15c560 2024-04-20T11:33:12.583 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.582+0000 7f5cd1200640 1 -- 172.21.15.152:0/2636862727 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ccc156e10 con 0x7f5ccc15c560 2024-04-20T11:33:12.583 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.582+0000 7f5cd1200640 1 -- 172.21.15.152:0/2636862727 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ccc1573d0 con 0x7f5ccc15c560 2024-04-20T11:33:12.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.583+0000 7f5cd1200640 1 -- 172.21.15.152:0/2636862727 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ccc0689f0 con 0x7f5ccc15c560 2024-04-20T11:33:12.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.584+0000 7f5cc2000640 1 -- 172.21.15.152:0/2636862727 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 91142+0+0 (secure 0 0 0) 0x7f5cc400ac30 con 0x7f5ccc15c560 2024-04-20T11:33:12.585 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.585+0000 7f5cc2000640 1 --2- 172.21.15.152:0/2636862727 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f5ca806ca10 0x7f5ca806eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:12.585 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.585+0000 7f5cc2000640 1 -- 172.21.15.152:0/2636862727 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5795+0+0 (secure 0 0 0) 0x7f5cc408bff0 con 0x7f5ccc15c560 2024-04-20T11:33:12.585 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.586+0000 7f5cc9e00640 1 --2- 172.21.15.152:0/2636862727 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f5ca806ca10 0x7f5ca806eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:12.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.587+0000 7f5cc9e00640 1 --2- 172.21.15.152:0/2636862727 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f5ca806ca10 0x7f5ca806eed0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc000f60 tx=0x7f5cbc007c00 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:12.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.590+0000 7f5cc2000640 1 -- 172.21.15.152:0/2636862727 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f5cc4020070 con 0x7f5ccc15c560 2024-04-20T11:33:12.792 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.793+0000 7f5cd1200640 1 -- 172.21.15.152:0/2636862727 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5ccc0af420 con 0x7f5ccc15c560 2024-04-20T11:33:12.794 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.795+0000 7f5cc2000640 1 -- 172.21.15.152:0/2636862727 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+16851 (secure 0 0 0) 0x7f5cc405a300 con 0x7f5ccc15c560 2024-04-20T11:33:12.795 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:33:12.795 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":91,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:33:02.588760+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:32:39.268411+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-20T11:32:40.497900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.175:6835/2632275131":"2024-04-21T11:32:59.571722+0000","172.21.15.175:6834/2632275131":"2024-04-21T11:32:59.571722+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:33:12.797 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.798+0000 7f5cd1200640 1 -- 172.21.15.152:0/2636862727 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f5ca806ca10 msgr2=0x7f5ca806eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:12.798 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.798+0000 7f5cd1200640 1 --2- 172.21.15.152:0/2636862727 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f5ca806ca10 0x7f5ca806eed0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc000f60 tx=0x7f5cbc007c00 comp rx=0 tx=0).stop 2024-04-20T11:33:12.798 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.798+0000 7f5cd1200640 1 -- 172.21.15.152:0/2636862727 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5ccc15c560 msgr2=0x7f5ccc15af70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:12.798 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.798+0000 7f5cd1200640 1 --2- 172.21.15.152:0/2636862727 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5ccc15c560 0x7f5ccc15af70 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f5cc400c7b0 tx=0x7f5cc400cc70 comp rx=0 tx=0).stop 2024-04-20T11:33:12.798 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.798+0000 7f5cd1200640 1 -- 172.21.15.152:0/2636862727 shutdown_connections 2024-04-20T11:33:12.798 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.798+0000 7f5cd1200640 1 --2- 172.21.15.152:0/2636862727 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f5ca806ca10 0x7f5ca806eed0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:12.798 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.798+0000 7f5cd1200640 1 --2- 172.21.15.152:0/2636862727 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5ccc15c560 0x7f5ccc15af70 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:12.798 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.798+0000 7f5cd1200640 1 --2- 172.21.15.152:0/2636862727 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ccc0d0640 0x7f5ccc152b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:12.798 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.798+0000 7f5cd1200640 1 --2- 172.21.15.152:0/2636862727 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ccc0b4500 0x7f5ccc14f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:12.798 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.798+0000 7f5cd1200640 1 -- 172.21.15.152:0/2636862727 >> 172.21.15.152:0/2636862727 conn(0x7f5ccc067cf0 msgr2=0x7f5ccc0ae5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:12.798 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.798+0000 7f5cd1200640 1 -- 172.21.15.152:0/2636862727 shutdown_connections 2024-04-20T11:33:12.798 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:12.798+0000 7f5cd1200640 1 -- 172.21.15.152:0/2636862727 wait complete. 2024-04-20T11:33:12.817 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-20T11:33:13.018 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.018+0000 7fd2e7200640 1 Processor -- start 2024-04-20T11:33:13.018 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.019+0000 7fd2e7200640 1 -- start start 2024-04-20T11:33:13.019 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.020+0000 7fd2e7200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd2e8161d30 0x7fd2e80d0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:13.019 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.020+0000 7fd2e7200640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2e80d26a0 con 0x7fd2e80d2b80 2024-04-20T11:33:13.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.020+0000 7fd2e7200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2e80d53c0 con 0x7fd2e80d4ec0 2024-04-20T11:33:13.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.020+0000 7fd2e7200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd2e80d0540 con 0x7fd2e8161d30 2024-04-20T11:33:13.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.020+0000 7fd2e6800640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fd2e80d2b80 0x7fd2e806c830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:58244/0 (socket says 172.21.15.152:58244) 2024-04-20T11:33:13.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.020+0000 7fd2e6800640 1 -- 172.21.15.152:0/4200334656 learned_addr learned my addr 172.21.15.152:0/4200334656 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:33:13.022 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.021+0000 7fd2e5e00640 1 --2- 172.21.15.152:0/4200334656 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd2e8161d30 0x7fd2e80d0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:13.022 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.021+0000 7fd2e5400640 1 -- 172.21.15.152:0/4200334656 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 66270312 0 0) 0x7fd2e80d26a0 con 0x7fd2e80d2b80 2024-04-20T11:33:13.022 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.021+0000 7fd2e5400640 1 -- 172.21.15.152:0/4200334656 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd2cc003520 con 0x7fd2e80d2b80 2024-04-20T11:33:13.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.021+0000 7fd2e5400640 1 -- 172.21.15.152:0/4200334656 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3581718943 0 0) 0x7fd2cc003520 con 0x7fd2e80d2b80 2024-04-20T11:33:13.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.021+0000 7fd2e5400640 1 -- 172.21.15.152:0/4200334656 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd2e8161d30 msgr2=0x7fd2e80d0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:13.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.021+0000 7fd2e5400640 1 --2- 172.21.15.152:0/4200334656 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd2e8161d30 0x7fd2e80d0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:13.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.021+0000 7fd2e5400640 1 -- 172.21.15.152:0/4200334656 >> v1:172.21.15.175:6789/0 conn(0x7fd2e80d4ec0 legacy=0x7fd2e80d15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:13.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.021+0000 7fd2e5400640 1 -- 172.21.15.152:0/4200334656 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2e80cf520 con 0x7fd2e80d2b80 2024-04-20T11:33:13.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.021+0000 7fd2e5400640 1 -- 172.21.15.152:0/4200334656 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7fd2e00019f0 con 0x7fd2e80d2b80 2024-04-20T11:33:13.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.021+0000 7fd2e5400640 1 -- 172.21.15.152:0/4200334656 >> v1:172.21.15.152:6789/0 conn(0x7fd2e80d2b80 legacy=0x7fd2e806c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:13.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.021+0000 7fd2e5400640 1 --2- 172.21.15.152:0/4200334656 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd2e8161d30 0x7fd2cc0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:13.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.021+0000 7fd2e5400640 1 --2- 172.21.15.152:0/4200334656 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2cc003f60 0x7fd2cc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:13.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.021+0000 7fd2e5400640 1 --2- 172.21.15.152:0/4200334656 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd2cc004d40 0x7fd2cc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:13.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.021+0000 7fd2e5400640 1 -- 172.21.15.152:0/4200334656 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fd2e80d0540 con 0x7fd2cc004d40 2024-04-20T11:33:13.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.021+0000 7fd2e5400640 1 -- 172.21.15.152:0/4200334656 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd2cc00baa0 con 0x7fd2cc003f60 2024-04-20T11:33:13.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.021+0000 7fd2e5400640 1 -- 172.21.15.152:0/4200334656 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd2cc00bca0 con 0x7fd2e8161d30 2024-04-20T11:33:13.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.022+0000 7fd2e6800640 1 --2- 172.21.15.152:0/4200334656 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd2cc004d40 0x7fd2cc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:13.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.022+0000 7fd2e5e00640 1 --2- 172.21.15.152:0/4200334656 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd2e8161d30 0x7fd2cc0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:13.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.022+0000 7fd2dd400640 1 --2- 172.21.15.152:0/4200334656 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2cc003f60 0x7fd2cc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:13.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.022+0000 7fd2e6800640 1 -- 172.21.15.152:0/4200334656 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd2e8161d30 msgr2=0x7fd2cc0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:13.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.022+0000 7fd2e6800640 1 --2- 172.21.15.152:0/4200334656 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd2e8161d30 0x7fd2cc0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:13.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.022+0000 7fd2e6800640 1 -- 172.21.15.152:0/4200334656 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2cc003f60 msgr2=0x7fd2cc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:13.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.022+0000 7fd2e6800640 1 --2- 172.21.15.152:0/4200334656 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2cc003f60 0x7fd2cc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:13.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.022+0000 7fd2e6800640 1 -- 172.21.15.152:0/4200334656 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2cc003520 con 0x7fd2cc004d40 2024-04-20T11:33:13.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.022+0000 7fd2dd400640 1 --2- 172.21.15.152:0/4200334656 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2cc003f60 0x7fd2cc004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:33:13.026 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.023+0000 7fd2e6800640 1 --2- 172.21.15.152:0/4200334656 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd2cc004d40 0x7fd2cc0051c0 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7fd2e000a860 tx=0x7fd2cc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8083a228820bf5d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:13.026 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.024+0000 7fd2e5400640 1 -- 172.21.15.152:0/4200334656 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2e000a480 con 0x7fd2cc004d40 2024-04-20T11:33:13.026 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.024+0000 7fd2e5400640 1 -- 172.21.15.152:0/4200334656 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2e0034630 con 0x7fd2cc004d40 2024-04-20T11:33:13.026 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.024+0000 7fd2e5400640 1 -- 172.21.15.152:0/4200334656 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2e0034a80 con 0x7fd2cc004d40 2024-04-20T11:33:13.026 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.024+0000 7fd2e7200640 1 -- 172.21.15.152:0/4200334656 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd2cc004d40 msgr2=0x7fd2cc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:13.026 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.024+0000 7fd2e7200640 1 --2- 172.21.15.152:0/4200334656 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd2cc004d40 0x7fd2cc0051c0 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7fd2e000a860 tx=0x7fd2cc005830 comp rx=0 tx=0).stop 2024-04-20T11:33:13.026 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.024+0000 7fd2e6800640 1 -- 172.21.15.152:0/4200334656 reap_dead start 2024-04-20T11:33:13.026 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.024+0000 7fd2e7200640 1 -- 172.21.15.152:0/4200334656 shutdown_connections 2024-04-20T11:33:13.026 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.024+0000 7fd2e7200640 1 -- 172.21.15.152:0/4200334656 >> 172.21.15.152:0/4200334656 conn(0x7fd2e808bfd0 msgr2=0x7fd2e80d45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:13.026 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.024+0000 7fd2e7200640 1 -- 172.21.15.152:0/4200334656 shutdown_connections 2024-04-20T11:33:13.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.024+0000 7fd2e7200640 1 -- 172.21.15.152:0/4200334656 wait complete. 2024-04-20T11:33:13.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.026+0000 7fd2e7200640 1 Processor -- start 2024-04-20T11:33:13.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.026+0000 7fd2e7200640 1 -- start start 2024-04-20T11:33:13.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.027+0000 7fd2e7200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd2cc004d40 0x7fd2e80b5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:13.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.027+0000 7fd2e7200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2e80b7ef0 0x7fd2e80b6980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:13.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.027+0000 7fd2e7200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd2e81588c0 0x7fd2e815f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:13.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.027+0000 7fd2e7200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fd2e80cf1b0 con 0x7fd2cc004d40 2024-04-20T11:33:13.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.027+0000 7fd2e7200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd2e80cfb70 con 0x7fd2e80b7ef0 2024-04-20T11:33:13.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.027+0000 7fd2e7200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd2e8162110 con 0x7fd2e81588c0 2024-04-20T11:33:13.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.027+0000 7fd2e5e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd2cc004d40 0x7fd2e80b5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:13.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.027+0000 7fd2e5e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd2cc004d40 0x7fd2e80b5aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36610/0 (socket says 172.21.15.152:36610) 2024-04-20T11:33:13.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.027+0000 7fd2e5e00640 1 -- 172.21.15.152:0/2257913200 learned_addr learned my addr 172.21.15.152:0/2257913200 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:33:13.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.028+0000 7fd2e5e00640 1 -- 172.21.15.152:0/2257913200 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd2e81588c0 msgr2=0x7fd2e815f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:13.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.028+0000 7fd2e5e00640 1 --2- 172.21.15.152:0/2257913200 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd2e81588c0 0x7fd2e815f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:13.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.028+0000 7fd2e5e00640 1 -- 172.21.15.152:0/2257913200 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2e80b7ef0 msgr2=0x7fd2e80b6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:13.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.028+0000 7fd2dd400640 1 --2- 172.21.15.152:0/2257913200 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2e80b7ef0 0x7fd2e80b6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:13.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.028+0000 7fd2e5e00640 1 --2- 172.21.15.152:0/2257913200 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2e80b7ef0 0x7fd2e80b6980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:13.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.028+0000 7fd2e5e00640 1 -- 172.21.15.152:0/2257913200 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2e80b6ec0 con 0x7fd2cc004d40 2024-04-20T11:33:13.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.028+0000 7fd2e5e00640 1 --2- 172.21.15.152:0/2257913200 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd2cc004d40 0x7fd2e80b5aa0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7fd2d80115b0 tx=0x7fd2d8011a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:13.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.028+0000 7fd2dea00640 1 -- 172.21.15.152:0/2257913200 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2d8015870 con 0x7fd2cc004d40 2024-04-20T11:33:13.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.028+0000 7fd2dea00640 1 -- 172.21.15.152:0/2257913200 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2d8015a10 con 0x7fd2cc004d40 2024-04-20T11:33:13.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.028+0000 7fd2dea00640 1 -- 172.21.15.152:0/2257913200 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2d8015dc0 con 0x7fd2cc004d40 2024-04-20T11:33:13.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.028+0000 7fd2e7200640 1 -- 172.21.15.152:0/2257913200 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2e80b8370 con 0x7fd2cc004d40 2024-04-20T11:33:13.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.028+0000 7fd2e7200640 1 -- 172.21.15.152:0/2257913200 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2e80b7410 con 0x7fd2cc004d40 2024-04-20T11:33:13.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.030+0000 7fd2e7200640 1 -- 172.21.15.152:0/2257913200 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2e8075e60 con 0x7fd2cc004d40 2024-04-20T11:33:13.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.031+0000 7fd2dea00640 1 -- 172.21.15.152:0/2257913200 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 91142+0+0 (secure 0 0 0) 0x7fd2d801d400 con 0x7fd2cc004d40 2024-04-20T11:33:13.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.031+0000 7fd2dea00640 1 --2- 172.21.15.152:0/2257913200 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd2c406ca10 0x7fd2c406eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:13.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.032+0000 7fd2dea00640 1 -- 172.21.15.152:0/2257913200 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5795+0+0 (secure 0 0 0) 0x7fd2d808c2f0 con 0x7fd2cc004d40 2024-04-20T11:33:13.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.032+0000 7fd2dd400640 1 --2- 172.21.15.152:0/2257913200 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd2c406ca10 0x7fd2c406eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:13.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.033+0000 7fd2dd400640 1 --2- 172.21.15.152:0/2257913200 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd2c406ca10 0x7fd2c406eed0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fd2e80b5810 tx=0x7fd2d0009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:13.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.036+0000 7fd2dea00640 1 -- 172.21.15.152:0/2257913200 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fd2e80b6ec0 con 0x7fd2cc004d40 2024-04-20T11:33:13.277 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.277+0000 7fd2e7200640 1 -- 172.21.15.152:0/2257913200 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fd2e80cea60 con 0x7fd2cc004d40 2024-04-20T11:33:13.277 INFO:tasks.ceph.mon.a.smithi152.stderr:2024-04-20T11:33:13.278+0000 7f8229800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-20T11:33:13.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.282+0000 7fd2dea00640 1 -- 172.21.15.152:0/2257913200 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v59) ==== 161+0+0 (secure 0 0 0) 0x7fd2d8019030 con 0x7fd2cc004d40 2024-04-20T11:33:13.281 INFO:teuthology.orchestra.run.smithi152.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-20T11:33:13.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.286+0000 7fd2e7200640 1 -- 172.21.15.152:0/2257913200 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd2c406ca10 msgr2=0x7fd2c406eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:13.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.286+0000 7fd2e7200640 1 --2- 172.21.15.152:0/2257913200 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd2c406ca10 0x7fd2c406eed0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fd2e80b5810 tx=0x7fd2d0009040 comp rx=0 tx=0).stop 2024-04-20T11:33:13.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.286+0000 7fd2e7200640 1 -- 172.21.15.152:0/2257913200 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd2cc004d40 msgr2=0x7fd2e80b5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:13.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.286+0000 7fd2e7200640 1 --2- 172.21.15.152:0/2257913200 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd2cc004d40 0x7fd2e80b5aa0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7fd2d80115b0 tx=0x7fd2d8011a70 comp rx=0 tx=0).stop 2024-04-20T11:33:13.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.287+0000 7fd2e7200640 1 -- 172.21.15.152:0/2257913200 shutdown_connections 2024-04-20T11:33:13.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.287+0000 7fd2e7200640 1 --2- 172.21.15.152:0/2257913200 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd2c406ca10 0x7fd2c406eed0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:13.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.287+0000 7fd2e7200640 1 --2- 172.21.15.152:0/2257913200 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd2e81588c0 0x7fd2e815f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:13.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.287+0000 7fd2e7200640 1 --2- 172.21.15.152:0/2257913200 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2e80b7ef0 0x7fd2e80b6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:13.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.287+0000 7fd2e7200640 1 --2- 172.21.15.152:0/2257913200 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd2cc004d40 0x7fd2e80b5aa0 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:13.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.287+0000 7fd2e7200640 1 -- 172.21.15.152:0/2257913200 >> 172.21.15.152:0/2257913200 conn(0x7fd2e808bfd0 msgr2=0x7fd2e80c8230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:13.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.287+0000 7fd2e7200640 1 -- 172.21.15.152:0/2257913200 shutdown_connections 2024-04-20T11:33:13.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.287+0000 7fd2e7200640 1 -- 172.21.15.152:0/2257913200 wait complete. 2024-04-20T11:33:13.291 INFO:tasks.ceph.mds.c.smithi152.stderr: -1703> 2024-04-20T11:32:47.018+0000 7fd55ba00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-20T11:33:13.306 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-20T11:33:13.338 INFO:tasks.ceph.mds.c.smithi152.stdout:starting mds.c at 2024-04-20T11:33:13.485 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.485+0000 7f8692000640 1 Processor -- start 2024-04-20T11:33:13.485 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.485+0000 7f8692000640 1 -- start start 2024-04-20T11:33:13.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.487+0000 7f8692000640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f868c160220 0x7f868c1606b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:13.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.487+0000 7f8692000640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f868c06fb40 con 0x7f868c141450 2024-04-20T11:33:13.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.487+0000 7f8692000640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f868c160f90 con 0x7f868c071cd0 2024-04-20T11:33:13.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.487+0000 7f8692000640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f868c13f8e0 con 0x7f868c160220 2024-04-20T11:33:13.489 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.487+0000 7f8691600640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f868c141450 0x7f868c1418e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:58264/0 (socket says 172.21.15.152:58264) 2024-04-20T11:33:13.489 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.487+0000 7f8691600640 1 -- 172.21.15.152:0/1739667679 learned_addr learned my addr 172.21.15.152:0/1739667679 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:33:13.489 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.487+0000 7f868be00640 1 --2- 172.21.15.152:0/1739667679 >> v2:172.21.15.175:3300/0 conn(0x7f868c160220 0x7f868c1606b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:13.489 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.488+0000 7f868b400640 1 -- 172.21.15.152:0/1739667679 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1043188925 0 0) 0x7f868c06fb40 con 0x7f868c141450 2024-04-20T11:33:13.489 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.488+0000 7f868b400640 1 -- 172.21.15.152:0/1739667679 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8674003520 con 0x7f868c141450 2024-04-20T11:33:13.489 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.488+0000 7f868b400640 1 -- 172.21.15.152:0/1739667679 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 777699219 0 0) 0x7f8674003520 con 0x7f868c141450 2024-04-20T11:33:13.489 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.488+0000 7f868b400640 1 -- 172.21.15.152:0/1739667679 >> v2:172.21.15.175:3300/0 conn(0x7f868c160220 msgr2=0x7f868c1606b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:13.489 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.488+0000 7f868b400640 1 --2- 172.21.15.152:0/1739667679 >> v2:172.21.15.175:3300/0 conn(0x7f868c160220 0x7f868c1606b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:13.489 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.488+0000 7f868b400640 1 -- 172.21.15.152:0/1739667679 >> v1:172.21.15.175:6789/0 conn(0x7f868c071cd0 legacy=0x7f868c0720b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:13.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.488+0000 7f868b400640 1 -- 172.21.15.152:0/1739667679 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f868c06dd90 con 0x7f868c141450 2024-04-20T11:33:13.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.488+0000 7f868b400640 1 -- 172.21.15.152:0/1739667679 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f86840019f0 con 0x7f868c141450 2024-04-20T11:33:13.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.488+0000 7f868b400640 1 -- 172.21.15.152:0/1739667679 >> v1:172.21.15.152:6789/0 conn(0x7f868c141450 legacy=0x7f868c1418e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:13.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.488+0000 7f868b400640 1 --2- 172.21.15.152:0/1739667679 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f86740039e0 0x7f8674003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:13.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.488+0000 7f868b400640 1 --2- 172.21.15.152:0/1739667679 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8674004800 0x7f8674004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:13.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.488+0000 7f868b400640 1 --2- 172.21.15.152:0/1739667679 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8674005210 0x7f867400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:13.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.488+0000 7f868b400640 1 -- 172.21.15.152:0/1739667679 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f868c13f8e0 con 0x7f8674004800 2024-04-20T11:33:13.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.488+0000 7f868b400640 1 -- 172.21.15.152:0/1739667679 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f867400c140 con 0x7f8674005210 2024-04-20T11:33:13.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.488+0000 7f868b400640 1 -- 172.21.15.152:0/1739667679 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f867400c340 con 0x7f86740039e0 2024-04-20T11:33:13.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.489+0000 7f868be00640 1 --2- 172.21.15.152:0/1739667679 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8674004800 0x7f8674004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:13.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.489+0000 7f8690c00640 1 --2- 172.21.15.152:0/1739667679 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f86740039e0 0x7f8674003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:13.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.489+0000 7f8691600640 1 --2- 172.21.15.152:0/1739667679 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8674005210 0x7f867400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:13.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.489+0000 7f868be00640 1 -- 172.21.15.152:0/1739667679 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f86740039e0 msgr2=0x7f8674003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:13.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.489+0000 7f868be00640 1 --2- 172.21.15.152:0/1739667679 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f86740039e0 0x7f8674003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:13.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.489+0000 7f868be00640 1 -- 172.21.15.152:0/1739667679 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8674005210 msgr2=0x7f867400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:13.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.489+0000 7f868be00640 1 --2- 172.21.15.152:0/1739667679 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8674005210 0x7f867400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:13.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.489+0000 7f868be00640 1 -- 172.21.15.152:0/1739667679 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8674003520 con 0x7f8674004800 2024-04-20T11:33:13.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.489+0000 7f8691600640 1 --2- 172.21.15.152:0/1739667679 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8674005210 0x7f867400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:33:13.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.489+0000 7f8691600640 1 -- 172.21.15.152:0/1739667679 reap_dead start 2024-04-20T11:33:13.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.490+0000 7f868be00640 1 --2- 172.21.15.152:0/1739667679 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8674004800 0x7f8674004c80 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f8678009ac0 tx=0x7f8674005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d867afa99cebff7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:13.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.491+0000 7f868b400640 1 -- 172.21.15.152:0/1739667679 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8678039070 con 0x7f8674004800 2024-04-20T11:33:13.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.491+0000 7f868b400640 1 -- 172.21.15.152:0/1739667679 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f867802c6b0 con 0x7f8674004800 2024-04-20T11:33:13.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.491+0000 7f868b400640 1 -- 172.21.15.152:0/1739667679 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8678009690 con 0x7f8674004800 2024-04-20T11:33:13.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.491+0000 7f8692000640 1 -- 172.21.15.152:0/1739667679 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8674004800 msgr2=0x7f8674004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:13.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.491+0000 7f8692000640 1 --2- 172.21.15.152:0/1739667679 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8674004800 0x7f8674004c80 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f8678009ac0 tx=0x7f8674005830 comp rx=0 tx=0).stop 2024-04-20T11:33:13.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.491+0000 7f8692000640 1 -- 172.21.15.152:0/1739667679 shutdown_connections 2024-04-20T11:33:13.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.491+0000 7f8692000640 1 --2- 172.21.15.152:0/1739667679 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8674004800 0x7f8674004c80 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:13.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.491+0000 7f8692000640 1 -- 172.21.15.152:0/1739667679 >> 172.21.15.152:0/1739667679 conn(0x7f868c067ae0 msgr2=0x7f868c06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:13.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.491+0000 7f8692000640 1 -- 172.21.15.152:0/1739667679 shutdown_connections 2024-04-20T11:33:13.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.491+0000 7f8692000640 1 -- 172.21.15.152:0/1739667679 wait complete. 2024-04-20T11:33:13.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.493+0000 7f8692000640 1 Processor -- start 2024-04-20T11:33:13.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.494+0000 7f8692000640 1 -- start start 2024-04-20T11:33:13.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.494+0000 7f8692000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8674004800 0x7f868c151d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:13.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.495+0000 7f8692000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f868c15e5a0 0x7f868c152c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:13.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.495+0000 7f8692000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f868c150d70 0x7f868c1511d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:13.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.495+0000 7f8692000640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f868c06e630 con 0x7f8674004800 2024-04-20T11:33:13.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.495+0000 7f8692000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f868c06e350 con 0x7f868c15e5a0 2024-04-20T11:33:13.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.495+0000 7f8692000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f868c13fb90 con 0x7f868c150d70 2024-04-20T11:33:13.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.495+0000 7f8690c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8674004800 0x7f868c151d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:13.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.495+0000 7f8690c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8674004800 0x7f868c151d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36636/0 (socket says 172.21.15.152:36636) 2024-04-20T11:33:13.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.495+0000 7f8690c00640 1 -- 172.21.15.152:0/3370365446 learned_addr learned my addr 172.21.15.152:0/3370365446 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:33:13.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.495+0000 7f8690c00640 1 -- 172.21.15.152:0/3370365446 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f868c150d70 msgr2=0x7f868c1511d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:13.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.495+0000 7f8690c00640 1 --2- 172.21.15.152:0/3370365446 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f868c150d70 0x7f868c1511d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:13.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.495+0000 7f8690c00640 1 -- 172.21.15.152:0/3370365446 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f868c15e5a0 msgr2=0x7f868c152c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:13.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.495+0000 7f8690c00640 1 --2- 172.21.15.152:0/3370365446 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f868c15e5a0 0x7f868c152c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:13.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.495+0000 7f8690c00640 1 -- 172.21.15.152:0/3370365446 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f868c15ed10 con 0x7f8674004800 2024-04-20T11:33:13.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.495+0000 7f8690c00640 1 --2- 172.21.15.152:0/3370365446 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8674004800 0x7f868c151d80 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f868000e4c0 tx=0x7f868000e980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:13.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.496+0000 7f8688c00640 1 -- 172.21.15.152:0/3370365446 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86800052e0 con 0x7f8674004800 2024-04-20T11:33:13.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.496+0000 7f8688c00640 1 -- 172.21.15.152:0/3370365446 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8680018070 con 0x7f8674004800 2024-04-20T11:33:13.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.496+0000 7f8688c00640 1 -- 172.21.15.152:0/3370365446 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8680003830 con 0x7f8674004800 2024-04-20T11:33:13.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.496+0000 7f8692000640 1 -- 172.21.15.152:0/3370365446 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f868c15eff0 con 0x7f8674004800 2024-04-20T11:33:13.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.496+0000 7f8692000640 1 -- 172.21.15.152:0/3370365446 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f868c0b3c50 con 0x7f8674004800 2024-04-20T11:33:13.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.497+0000 7f8692000640 1 -- 172.21.15.152:0/3370365446 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f868c075e60 con 0x7f8674004800 2024-04-20T11:33:13.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.498+0000 7f8688c00640 1 -- 172.21.15.152:0/3370365446 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 91142+0+0 (secure 0 0 0) 0x7f86800039d0 con 0x7f8674004800 2024-04-20T11:33:13.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.499+0000 7f8688c00640 1 --2- 172.21.15.152:0/3370365446 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f866c06ca10 0x7f866c06eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:13.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.499+0000 7f8688c00640 1 -- 172.21.15.152:0/3370365446 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5881+0+0 (secure 0 0 0) 0x7f868008ba90 con 0x7f8674004800 2024-04-20T11:33:13.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.500+0000 7f868be00640 1 --2- 172.21.15.152:0/3370365446 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f866c06ca10 0x7f866c06eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:13.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.501+0000 7f868be00640 1 --2- 172.21.15.152:0/3370365446 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f866c06ca10 0x7f866c06eed0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8678036000 tx=0x7f867802e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:13.503 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.503+0000 7f8688c00640 1 -- 172.21.15.152:0/3370365446 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f8680059c60 con 0x7f8674004800 2024-04-20T11:33:13.746 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:13.747+0000 7f8692000640 1 -- 172.21.15.152:0/3370365446 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f868c15cb10 con 0x7f8674004800 2024-04-20T11:33:14.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:14.285+0000 7f8688c00640 1 -- 172.21.15.152:0/3370365446 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v60) ==== 104+0+0 (secure 0 0 0) 0x7f8680059560 con 0x7f8674004800 2024-04-20T11:33:14.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:14.289+0000 7f8692000640 1 -- 172.21.15.152:0/3370365446 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f866c06ca10 msgr2=0x7f866c06eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:14.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:14.289+0000 7f8692000640 1 --2- 172.21.15.152:0/3370365446 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f866c06ca10 0x7f866c06eed0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8678036000 tx=0x7f867802e000 comp rx=0 tx=0).stop 2024-04-20T11:33:14.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:14.289+0000 7f8692000640 1 -- 172.21.15.152:0/3370365446 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8674004800 msgr2=0x7f868c151d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:14.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:14.289+0000 7f8692000640 1 --2- 172.21.15.152:0/3370365446 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8674004800 0x7f868c151d80 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f868000e4c0 tx=0x7f868000e980 comp rx=0 tx=0).stop 2024-04-20T11:33:14.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:14.289+0000 7f8692000640 1 -- 172.21.15.152:0/3370365446 shutdown_connections 2024-04-20T11:33:14.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:14.289+0000 7f8692000640 1 --2- 172.21.15.152:0/3370365446 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f866c06ca10 0x7f866c06eed0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:14.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:14.289+0000 7f8692000640 1 --2- 172.21.15.152:0/3370365446 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f868c150d70 0x7f868c1511d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:14.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:14.289+0000 7f8692000640 1 --2- 172.21.15.152:0/3370365446 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f868c15e5a0 0x7f868c152c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:14.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:14.289+0000 7f8692000640 1 --2- 172.21.15.152:0/3370365446 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8674004800 0x7f868c151d80 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:14.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:14.289+0000 7f8692000640 1 -- 172.21.15.152:0/3370365446 >> 172.21.15.152:0/3370365446 conn(0x7f868c067ae0 msgr2=0x7f868c06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:14.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:14.289+0000 7f8692000640 1 -- 172.21.15.152:0/3370365446 shutdown_connections 2024-04-20T11:33:14.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:14.289+0000 7f8692000640 1 -- 172.21.15.152:0/3370365446 wait complete. 2024-04-20T11:33:14.312 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-20T11:33:14.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:14.564+0000 7f841d600640 1 Processor -- start 2024-04-20T11:33:14.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:14.565+0000 7f841d600640 1 -- start start 2024-04-20T11:33:14.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:14.566+0000 7f841d600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f841815e900 0x7f84180d3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:14.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:14.566+0000 7f841d600640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f8418162610 0x7f84180b7af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:14.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:14.566+0000 7f841d600640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8418162b10 con 0x7f84180d11d0 2024-04-20T11:33:14.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:14.566+0000 7f841d600640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f84180d4130 con 0x7f8418162610 2024-04-20T11:33:14.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:14.566+0000 7f841d600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8418162df0 con 0x7f841815e900 2024-04-20T11:33:14.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:14.566+0000 7f8416000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f8418162610 0x7f84180b7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:14.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:14.566+0000 7f8416000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f8418162610 0x7f84180b7af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36644/0 (socket says 172.21.15.152:36644) 2024-04-20T11:33:14.567 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:14.566+0000 7f8416000640 1 -- 172.21.15.152:0/522105613 learned_addr learned my addr 172.21.15.152:0/522105613 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:33:14.567 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:14.566+0000 7f8416a00640 1 --2- 172.21.15.152:0/522105613 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f841815e900 0x7f84180d3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:14.567 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:14.566+0000 7f8416000640 1 -- 172.21.15.152:0/522105613 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f841815e900 msgr2=0x7f84180d3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:14.567 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:14.567+0000 7f8416000640 1 --2- 172.21.15.152:0/522105613 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f841815e900 0x7f84180d3bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:14.567 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:14.567+0000 7f8416000640 1 -- 172.21.15.152:0/522105613 >> v1:172.21.15.175:6789/0 conn(0x7f84180d11d0 legacy=0x7f84180d2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:14.567 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:14.567+0000 7f8416000640 1 -- 172.21.15.152:0/522105613 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84181630d0 con 0x7f8418162610 2024-04-20T11:33:14.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:14.568+0000 7f8416000640 1 --2- 172.21.15.152:0/522105613 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8418162610 0x7f84180b7af0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f8400009870 tx=0x7f840002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b2516c8b3628322 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:14.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:14.568+0000 7f8415600640 1 -- 172.21.15.152:0/522105613 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8400035020 con 0x7f8418162610 2024-04-20T11:33:14.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:14.568+0000 7f8415600640 1 -- 172.21.15.152:0/522105613 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8400031640 con 0x7f8418162610 2024-04-20T11:33:14.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:14.568+0000 7f8415600640 1 -- 172.21.15.152:0/522105613 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f840002d040 con 0x7f8418162610 2024-04-20T11:33:14.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:14.568+0000 7f841d600640 1 -- 172.21.15.152:0/522105613 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8418162610 msgr2=0x7f84180b7af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:14.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:14.568+0000 7f841d600640 1 --2- 172.21.15.152:0/522105613 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8418162610 0x7f84180b7af0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f8400009870 tx=0x7f840002b8f0 comp rx=0 tx=0).stop 2024-04-20T11:33:14.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:14.569+0000 7f841d600640 1 -- 172.21.15.152:0/522105613 shutdown_connections 2024-04-20T11:33:14.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:14.569+0000 7f841d600640 1 --2- 172.21.15.152:0/522105613 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8418162610 0x7f84180b7af0 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:14.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:14.569+0000 7f841d600640 1 --2- 172.21.15.152:0/522105613 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f841815e900 0x7f84180d3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:14.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:14.569+0000 7f841d600640 1 -- 172.21.15.152:0/522105613 >> 172.21.15.152:0/522105613 conn(0x7f84180676c0 msgr2=0x7f8418067ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:14.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:14.569+0000 7f841d600640 1 -- 172.21.15.152:0/522105613 shutdown_connections 2024-04-20T11:33:14.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:14.569+0000 7f841d600640 1 -- 172.21.15.152:0/522105613 wait complete. 2024-04-20T11:33:14.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:14.570+0000 7f841d600640 1 Processor -- start 2024-04-20T11:33:14.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:14.570+0000 7f841d600640 1 -- start start 2024-04-20T11:33:14.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:14.571+0000 7f841d600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f84180d11d0 0x7f84181615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:14.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:14.571+0000 7f841d600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f841815e900 0x7f8418155aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:14.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:14.571+0000 7f841d600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8418162610 0x7f841815b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:14.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:14.571+0000 7f841d600640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f84180b8030 con 0x7f8418162610 2024-04-20T11:33:14.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:14.571+0000 7f841d600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f84180d4560 con 0x7f84180d11d0 2024-04-20T11:33:14.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:14.571+0000 7f841d600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f84180b81b0 con 0x7f841815e900 2024-04-20T11:33:14.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:14.572+0000 7f8417400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8418162610 0x7f841815b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:14.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:14.572+0000 7f8417400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8418162610 0x7f841815b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36660/0 (socket says 172.21.15.152:36660) 2024-04-20T11:33:14.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:14.572+0000 7f8417400640 1 -- 172.21.15.152:0/2903882835 learned_addr learned my addr 172.21.15.152:0/2903882835 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:33:14.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:14.572+0000 7f8416000640 1 --2- 172.21.15.152:0/2903882835 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f841815e900 0x7f8418155aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:14.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:14.572+0000 7f8416a00640 1 --2- 172.21.15.152:0/2903882835 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f84180d11d0 0x7f84181615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:14.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:14.572+0000 7f8417400640 1 -- 172.21.15.152:0/2903882835 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f841815e900 msgr2=0x7f8418155aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:14.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:14.572+0000 7f8417400640 1 --2- 172.21.15.152:0/2903882835 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f841815e900 0x7f8418155aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:14.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:14.572+0000 7f8417400640 1 -- 172.21.15.152:0/2903882835 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f84180d11d0 msgr2=0x7f84181615e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:14.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:14.572+0000 7f8417400640 1 --2- 172.21.15.152:0/2903882835 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f84180d11d0 0x7f84181615e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:14.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:14.572+0000 7f8417400640 1 -- 172.21.15.152:0/2903882835 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f841815f790 con 0x7f8418162610 2024-04-20T11:33:14.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:14.572+0000 7f8416a00640 1 --2- 172.21.15.152:0/2903882835 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f84180d11d0 0x7f84181615e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:33:14.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:14.573+0000 7f8417400640 1 --2- 172.21.15.152:0/2903882835 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8418162610 0x7f841815b3d0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f841000b3e0 tx=0x7f841000b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:14.574 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:14.573+0000 7f840ea00640 1 -- 172.21.15.152:0/2903882835 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8410015020 con 0x7f8418162610 2024-04-20T11:33:14.574 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:14.573+0000 7f840ea00640 1 -- 172.21.15.152:0/2903882835 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8410011830 con 0x7f8418162610 2024-04-20T11:33:14.574 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:14.573+0000 7f840ea00640 1 -- 172.21.15.152:0/2903882835 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8410011c60 con 0x7f8418162610 2024-04-20T11:33:14.574 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:14.573+0000 7f841d600640 1 -- 172.21.15.152:0/2903882835 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84180c79c0 con 0x7f8418162610 2024-04-20T11:33:14.574 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:14.573+0000 7f841d600640 1 -- 172.21.15.152:0/2903882835 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84180c7f80 con 0x7f8418162610 2024-04-20T11:33:14.574 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:14.574+0000 7f841d600640 1 -- 172.21.15.152:0/2903882835 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8418068a60 con 0x7f8418162610 2024-04-20T11:33:14.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:14.575+0000 7f840ea00640 1 -- 172.21.15.152:0/2903882835 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 91142+0+0 (secure 0 0 0) 0x7f8410018020 con 0x7f8418162610 2024-04-20T11:33:14.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:14.576+0000 7f840ea00640 1 --2- 172.21.15.152:0/2903882835 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f83f406ca10 0x7f83f406eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:14.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:14.576+0000 7f840ea00640 1 -- 172.21.15.152:0/2903882835 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5881+0+0 (secure 0 0 0) 0x7f841008b850 con 0x7f8418162610 2024-04-20T11:33:14.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:14.577+0000 7f8416a00640 1 --2- 172.21.15.152:0/2903882835 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f83f406ca10 0x7f83f406eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:14.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:14.578+0000 7f8416a00640 1 --2- 172.21.15.152:0/2903882835 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f83f406ca10 0x7f83f406eed0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8408007f50 tx=0x7f84080074a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:14.580 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:14.581+0000 7f840ea00640 1 -- 172.21.15.152:0/2903882835 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f841001f030 con 0x7f8418162610 2024-04-20T11:33:14.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:14.782+0000 7f841d600640 1 -- 172.21.15.152:0/2903882835 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f84180d2670 con 0x7f8418162610 2024-04-20T11:33:15.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:15.695+0000 7f840ea00640 1 -- 172.21.15.152:0/2903882835 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v93) ==== 181+0+0 (secure 0 0 0) 0x7f84100172c0 con 0x7f8418162610 2024-04-20T11:33:15.696 INFO:teuthology.orchestra.run.smithi152.stderr:pool 'cephfs_metadata' removed 2024-04-20T11:33:15.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:15.702+0000 7f841d600640 1 -- 172.21.15.152:0/2903882835 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f83f406ca10 msgr2=0x7f83f406eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:15.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:15.702+0000 7f841d600640 1 --2- 172.21.15.152:0/2903882835 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f83f406ca10 0x7f83f406eed0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8408007f50 tx=0x7f84080074a0 comp rx=0 tx=0).stop 2024-04-20T11:33:15.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:15.702+0000 7f841d600640 1 -- 172.21.15.152:0/2903882835 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8418162610 msgr2=0x7f841815b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:15.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:15.702+0000 7f841d600640 1 --2- 172.21.15.152:0/2903882835 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8418162610 0x7f841815b3d0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f841000b3e0 tx=0x7f841000b8a0 comp rx=0 tx=0).stop 2024-04-20T11:33:15.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:15.702+0000 7f841d600640 1 -- 172.21.15.152:0/2903882835 shutdown_connections 2024-04-20T11:33:15.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:15.702+0000 7f841d600640 1 --2- 172.21.15.152:0/2903882835 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f83f406ca10 0x7f83f406eed0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:15.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:15.702+0000 7f841d600640 1 --2- 172.21.15.152:0/2903882835 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8418162610 0x7f841815b3d0 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:15.703 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:15.702+0000 7f841d600640 1 --2- 172.21.15.152:0/2903882835 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f841815e900 0x7f8418155aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:15.703 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:15.702+0000 7f841d600640 1 --2- 172.21.15.152:0/2903882835 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f84180d11d0 0x7f84181615e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:15.703 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:15.702+0000 7f841d600640 1 -- 172.21.15.152:0/2903882835 >> 172.21.15.152:0/2903882835 conn(0x7f84180676c0 msgr2=0x7f84180cb950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:15.703 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:15.702+0000 7f841d600640 1 -- 172.21.15.152:0/2903882835 shutdown_connections 2024-04-20T11:33:15.703 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:15.702+0000 7f841d600640 1 -- 172.21.15.152:0/2903882835 wait complete. 2024-04-20T11:33:15.730 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-20T11:33:15.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:15.924+0000 7f2176000640 1 Processor -- start 2024-04-20T11:33:15.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:15.924+0000 7f2176000640 1 -- start start 2024-04-20T11:33:15.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:15.926+0000 7f2176000640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f217015e870 0x7f21700d3370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:15.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:15.926+0000 7f2176000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f21700d0fa0 0x7f21700d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:15.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:15.926+0000 7f2176000640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f217015ec50 con 0x7f21700cff90 2024-04-20T11:33:15.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:15.926+0000 7f2176000640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f21700d1960 con 0x7f217015e870 2024-04-20T11:33:15.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:15.926+0000 7f2176000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f21700d2ee0 con 0x7f21700d0fa0 2024-04-20T11:33:15.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:15.927+0000 7f2174c00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f217015e870 0x7f21700d3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:15.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:15.927+0000 7f2174c00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f217015e870 0x7f21700d3370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.152:50504/0 (socket says 172.21.15.152:50504) 2024-04-20T11:33:15.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:15.927+0000 7f2174c00640 1 -- 172.21.15.152:0/827527639 learned_addr learned my addr 172.21.15.152:0/827527639 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:33:15.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:15.927+0000 7f216fe00640 1 --2- 172.21.15.152:0/827527639 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f21700d0fa0 0x7f21700d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:15.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:15.927+0000 7f2174c00640 1 -- 172.21.15.152:0/827527639 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f21700d0fa0 msgr2=0x7f21700d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:15.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:15.927+0000 7f2174c00640 1 --2- 172.21.15.152:0/827527639 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f21700d0fa0 0x7f21700d1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:15.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:15.927+0000 7f2174c00640 1 -- 172.21.15.152:0/827527639 >> v1:172.21.15.175:6789/0 conn(0x7f21700cff90 legacy=0x7f21700d0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:15.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:15.927+0000 7f2174c00640 1 -- 172.21.15.152:0/827527639 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21700d38b0 con 0x7f217015e870 2024-04-20T11:33:15.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:15.929+0000 7f2174c00640 1 --2- 172.21.15.152:0/827527639 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f217015e870 0x7f21700d3370 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f2164009870 tx=0x7f216402b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c7106188ecbf3a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:15.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:15.929+0000 7f216f400640 1 -- 172.21.15.152:0/827527639 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2164035020 con 0x7f217015e870 2024-04-20T11:33:15.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:15.929+0000 7f216f400640 1 -- 172.21.15.152:0/827527639 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2164031640 con 0x7f217015e870 2024-04-20T11:33:15.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:15.929+0000 7f216f400640 1 -- 172.21.15.152:0/827527639 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f216402d040 con 0x7f217015e870 2024-04-20T11:33:15.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:15.930+0000 7f2176000640 1 -- 172.21.15.152:0/827527639 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f217015e870 msgr2=0x7f21700d3370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:15.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:15.930+0000 7f2176000640 1 --2- 172.21.15.152:0/827527639 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f217015e870 0x7f21700d3370 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f2164009870 tx=0x7f216402b8f0 comp rx=0 tx=0).stop 2024-04-20T11:33:15.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:15.930+0000 7f2176000640 1 -- 172.21.15.152:0/827527639 shutdown_connections 2024-04-20T11:33:15.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:15.930+0000 7f2176000640 1 --2- 172.21.15.152:0/827527639 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f21700d0fa0 0x7f21700d1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:15.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:15.930+0000 7f2176000640 1 --2- 172.21.15.152:0/827527639 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f217015e870 0x7f21700d3370 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:15.931 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:15.930+0000 7f2176000640 1 -- 172.21.15.152:0/827527639 >> 172.21.15.152:0/827527639 conn(0x7f217008bfd0 msgr2=0x7f21700d45d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:15.931 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:15.930+0000 7f2176000640 1 -- 172.21.15.152:0/827527639 shutdown_connections 2024-04-20T11:33:15.931 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:15.930+0000 7f2176000640 1 -- 172.21.15.152:0/827527639 wait complete. 2024-04-20T11:33:15.931 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:15.932+0000 7f2176000640 1 Processor -- start 2024-04-20T11:33:15.931 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:15.932+0000 7f2176000640 1 -- start start 2024-04-20T11:33:15.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:15.933+0000 7f2176000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f21700cff90 0x7f21700b7160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:15.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:15.933+0000 7f2176000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f21700d0fa0 0x7f2170157cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:15.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:15.933+0000 7f2176000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f217015e870 0x7f21700b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:15.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:15.933+0000 7f2176000640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f21700d2890 con 0x7f217015e870 2024-04-20T11:33:15.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:15.933+0000 7f2176000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f21700d25b0 con 0x7f21700cff90 2024-04-20T11:33:15.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:15.933+0000 7f2176000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f21700d2b70 con 0x7f21700d0fa0 2024-04-20T11:33:15.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:15.933+0000 7f2175600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f217015e870 0x7f21700b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:15.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:15.933+0000 7f2175600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f217015e870 0x7f21700b5a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:37408/0 (socket says 172.21.15.152:37408) 2024-04-20T11:33:15.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:15.933+0000 7f2175600640 1 -- 172.21.15.152:0/2201330043 learned_addr learned my addr 172.21.15.152:0/2201330043 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:33:15.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:15.934+0000 7f2174c00640 1 --2- 172.21.15.152:0/2201330043 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f21700cff90 0x7f21700b7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:15.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:15.934+0000 7f216fe00640 1 --2- 172.21.15.152:0/2201330043 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f21700d0fa0 0x7f2170157cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:15.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:15.934+0000 7f2175600640 1 -- 172.21.15.152:0/2201330043 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f21700d0fa0 msgr2=0x7f2170157cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:15.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:15.934+0000 7f2175600640 1 --2- 172.21.15.152:0/2201330043 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f21700d0fa0 0x7f2170157cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:15.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:15.934+0000 7f2175600640 1 -- 172.21.15.152:0/2201330043 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f21700cff90 msgr2=0x7f21700b7160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:15.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:15.934+0000 7f2175600640 1 --2- 172.21.15.152:0/2201330043 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f21700cff90 0x7f21700b7160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:15.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:15.934+0000 7f2175600640 1 -- 172.21.15.152:0/2201330043 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21700ca280 con 0x7f217015e870 2024-04-20T11:33:15.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:15.934+0000 7f2174c00640 1 --2- 172.21.15.152:0/2201330043 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f21700cff90 0x7f21700b7160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:33:15.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:15.934+0000 7f2175600640 1 --2- 172.21.15.152:0/2201330043 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f217015e870 0x7f21700b5a30 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f216800c730 tx=0x7f216800cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:15.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:15.934+0000 7f216cc00640 1 -- 172.21.15.152:0/2201330043 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2168016020 con 0x7f217015e870 2024-04-20T11:33:15.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:15.935+0000 7f216cc00640 1 -- 172.21.15.152:0/2201330043 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2168006d50 con 0x7f217015e870 2024-04-20T11:33:15.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:15.935+0000 7f2176000640 1 -- 172.21.15.152:0/2201330043 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21700ca560 con 0x7f217015e870 2024-04-20T11:33:15.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:15.935+0000 7f216cc00640 1 -- 172.21.15.152:0/2201330043 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2168003bb0 con 0x7f217015e870 2024-04-20T11:33:15.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:15.935+0000 7f2176000640 1 -- 172.21.15.152:0/2201330043 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21700b5230 con 0x7f217015e870 2024-04-20T11:33:15.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:15.936+0000 7f2176000640 1 -- 172.21.15.152:0/2201330043 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2170075e60 con 0x7f217015e870 2024-04-20T11:33:15.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:15.937+0000 7f216cc00640 1 -- 172.21.15.152:0/2201330043 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 91142+0+0 (secure 0 0 0) 0x7f21680050d0 con 0x7f217015e870 2024-04-20T11:33:15.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:15.938+0000 7f216cc00640 1 --2- 172.21.15.152:0/2201330043 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f215006ca10 0x7f215006eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:15.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:15.938+0000 7f216cc00640 1 -- 172.21.15.152:0/2201330043 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5443+0+0 (secure 0 0 0) 0x7f216808b800 con 0x7f217015e870 2024-04-20T11:33:15.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:15.938+0000 7f2174c00640 1 --2- 172.21.15.152:0/2201330043 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f215006ca10 0x7f215006eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:15.939 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:15.939+0000 7f2174c00640 1 --2- 172.21.15.152:0/2201330043 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f215006ca10 0x7f215006eed0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2164036000 tx=0x7f216403c040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:15.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:15.942+0000 7f216cc00640 1 -- 172.21.15.152:0/2201330043 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f2168020020 con 0x7f217015e870 2024-04-20T11:33:16.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:16.142+0000 7f2176000640 1 -- 172.21.15.152:0/2201330043 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f21700ba550 con 0x7f217015e870 2024-04-20T11:33:16.707 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:16.707+0000 7f216cc00640 1 -- 172.21.15.152:0/2201330043 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v94) ==== 169+0+0 (secure 0 0 0) 0x7f216808b0d0 con 0x7f217015e870 2024-04-20T11:33:16.707 INFO:teuthology.orchestra.run.smithi152.stderr:pool 'cephfs_data' removed 2024-04-20T11:33:16.711 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:16.711+0000 7f2176000640 1 -- 172.21.15.152:0/2201330043 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f215006ca10 msgr2=0x7f215006eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:16.711 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:16.711+0000 7f2176000640 1 --2- 172.21.15.152:0/2201330043 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f215006ca10 0x7f215006eed0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2164036000 tx=0x7f216403c040 comp rx=0 tx=0).stop 2024-04-20T11:33:16.711 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:16.711+0000 7f2176000640 1 -- 172.21.15.152:0/2201330043 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f217015e870 msgr2=0x7f21700b5a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:16.711 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:16.711+0000 7f2176000640 1 --2- 172.21.15.152:0/2201330043 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f217015e870 0x7f21700b5a30 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f216800c730 tx=0x7f216800cb30 comp rx=0 tx=0).stop 2024-04-20T11:33:16.711 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:16.711+0000 7f2176000640 1 -- 172.21.15.152:0/2201330043 shutdown_connections 2024-04-20T11:33:16.711 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:16.711+0000 7f2176000640 1 --2- 172.21.15.152:0/2201330043 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f215006ca10 0x7f215006eed0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:16.711 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:16.711+0000 7f2176000640 1 --2- 172.21.15.152:0/2201330043 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f217015e870 0x7f21700b5a30 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:16.711 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:16.711+0000 7f2176000640 1 --2- 172.21.15.152:0/2201330043 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f21700d0fa0 0x7f2170157cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:16.711 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:16.711+0000 7f2176000640 1 --2- 172.21.15.152:0/2201330043 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f21700cff90 0x7f21700b7160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:16.711 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:16.711+0000 7f2176000640 1 -- 172.21.15.152:0/2201330043 >> 172.21.15.152:0/2201330043 conn(0x7f217008bfd0 msgr2=0x7f21700b28c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:16.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:16.711+0000 7f2176000640 1 -- 172.21.15.152:0/2201330043 shutdown_connections 2024-04-20T11:33:16.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:16.711+0000 7f2176000640 1 -- 172.21.15.152:0/2201330043 wait complete. 2024-04-20T11:33:16.740 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-20T11:33:16.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:16.928+0000 7fce47e00640 1 Processor -- start 2024-04-20T11:33:16.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:16.928+0000 7fce47e00640 1 -- start start 2024-04-20T11:33:16.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:16.929+0000 7fce47e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fce48071cd0 0x7fce480720b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:16.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:16.929+0000 7fce47e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fce48160220 0x7fce48160680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:16.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:16.929+0000 7fce47e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fce4806eb30 0x7fce4806efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:16.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:16.929+0000 7fce47e00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fce4813f8e0 con 0x7fce4806eb30 2024-04-20T11:33:16.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:16.929+0000 7fce47e00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fce480725f0 con 0x7fce48160220 2024-04-20T11:33:16.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:16.929+0000 7fce47e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fce48160f90 con 0x7fce48071cd0 2024-04-20T11:33:16.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:16.930+0000 7fce47400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fce4806eb30 0x7fce4806efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:16.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:16.930+0000 7fce47400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fce4806eb30 0x7fce4806efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:37410/0 (socket says 172.21.15.152:37410) 2024-04-20T11:33:16.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:16.930+0000 7fce47400640 1 -- 172.21.15.152:0/20677284 learned_addr learned my addr 172.21.15.152:0/20677284 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:33:16.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:16.930+0000 7fce3fe00640 1 --2- 172.21.15.152:0/20677284 >> v2:172.21.15.175:3300/0 conn(0x7fce48160220 0x7fce48160680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:16.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:16.930+0000 7fce46a00640 1 --2- 172.21.15.152:0/20677284 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fce48071cd0 0x7fce480720b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:16.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:16.930+0000 7fce47400640 1 -- 172.21.15.152:0/20677284 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fce48071cd0 msgr2=0x7fce480720b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:16.931 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:16.930+0000 7fce47400640 1 --2- 172.21.15.152:0/20677284 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fce48071cd0 0x7fce480720b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:16.931 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:16.930+0000 7fce47400640 1 -- 172.21.15.152:0/20677284 >> v2:172.21.15.175:3300/0 conn(0x7fce48160220 msgr2=0x7fce48160680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:16.931 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:16.930+0000 7fce47400640 1 --2- 172.21.15.152:0/20677284 >> v2:172.21.15.175:3300/0 conn(0x7fce48160220 0x7fce48160680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:16.931 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:16.930+0000 7fce47400640 1 -- 172.21.15.152:0/20677284 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce4806dd90 con 0x7fce4806eb30 2024-04-20T11:33:16.931 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:16.932+0000 7fce47400640 1 --2- 172.21.15.152:0/20677284 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fce4806eb30 0x7fce4806efc0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7fce480b58d0 tx=0x7fce4002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc72004dcf1ecf09 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:16.931 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:16.932+0000 7fce46000640 1 -- 172.21.15.152:0/20677284 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce4002c9e0 con 0x7fce4806eb30 2024-04-20T11:33:16.932 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:16.932+0000 7fce46000640 1 -- 172.21.15.152:0/20677284 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce4002cb80 con 0x7fce4806eb30 2024-04-20T11:33:16.932 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:16.932+0000 7fce46000640 1 -- 172.21.15.152:0/20677284 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce400343f0 con 0x7fce4806eb30 2024-04-20T11:33:16.932 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:16.932+0000 7fce47e00640 1 -- 172.21.15.152:0/20677284 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fce4806eb30 msgr2=0x7fce4806efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:16.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:16.932+0000 7fce47e00640 1 --2- 172.21.15.152:0/20677284 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fce4806eb30 0x7fce4806efc0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7fce480b58d0 tx=0x7fce4002be40 comp rx=0 tx=0).stop 2024-04-20T11:33:16.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:16.933+0000 7fce47e00640 1 -- 172.21.15.152:0/20677284 shutdown_connections 2024-04-20T11:33:16.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:16.933+0000 7fce47e00640 1 --2- 172.21.15.152:0/20677284 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fce4806eb30 0x7fce4806efc0 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:16.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:16.933+0000 7fce47e00640 1 --2- 172.21.15.152:0/20677284 >> v2:172.21.15.175:3300/0 conn(0x7fce48160220 0x7fce48160680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:16.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:16.933+0000 7fce47e00640 1 --2- 172.21.15.152:0/20677284 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fce48071cd0 0x7fce480720b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:16.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:16.933+0000 7fce47e00640 1 -- 172.21.15.152:0/20677284 >> 172.21.15.152:0/20677284 conn(0x7fce48067ae0 msgr2=0x7fce4806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:16.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:16.933+0000 7fce47e00640 1 -- 172.21.15.152:0/20677284 shutdown_connections 2024-04-20T11:33:16.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:16.933+0000 7fce47e00640 1 -- 172.21.15.152:0/20677284 wait complete. 2024-04-20T11:33:16.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:16.934+0000 7fce47e00640 1 Processor -- start 2024-04-20T11:33:16.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:16.935+0000 7fce47e00640 1 -- start start 2024-04-20T11:33:16.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:16.935+0000 7fce47e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce4806eb30 0x7fce4815f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:16.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:16.935+0000 7fce47e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fce48071cd0 0x7fce4815d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:16.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:16.935+0000 7fce47e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fce48160220 0x7fce4813e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:16.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:16.935+0000 7fce47e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fce4806e630 con 0x7fce48071cd0 2024-04-20T11:33:16.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:16.935+0000 7fce47e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fce4806e350 con 0x7fce4806eb30 2024-04-20T11:33:16.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:16.935+0000 7fce47e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fce4813fb90 con 0x7fce48160220 2024-04-20T11:33:16.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:16.936+0000 7fce3fe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fce48071cd0 0x7fce4815d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:16.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:16.936+0000 7fce3fe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fce48071cd0 0x7fce4815d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:37412/0 (socket says 172.21.15.152:37412) 2024-04-20T11:33:16.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:16.936+0000 7fce3fe00640 1 -- 172.21.15.152:0/703789581 learned_addr learned my addr 172.21.15.152:0/703789581 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:33:16.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:16.936+0000 7fce46a00640 1 --2- 172.21.15.152:0/703789581 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce4806eb30 0x7fce4815f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:16.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:16.936+0000 7fce47400640 1 --2- 172.21.15.152:0/703789581 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fce48160220 0x7fce4813e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:16.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:16.936+0000 7fce3fe00640 1 -- 172.21.15.152:0/703789581 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fce48160220 msgr2=0x7fce4813e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:16.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:16.936+0000 7fce3fe00640 1 --2- 172.21.15.152:0/703789581 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fce48160220 0x7fce4813e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:16.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:16.936+0000 7fce3fe00640 1 -- 172.21.15.152:0/703789581 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce4806eb30 msgr2=0x7fce4815f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:16.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:16.936+0000 7fce3fe00640 1 --2- 172.21.15.152:0/703789581 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce4806eb30 0x7fce4815f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:16.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:16.936+0000 7fce3fe00640 1 -- 172.21.15.152:0/703789581 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce481e7b20 con 0x7fce48071cd0 2024-04-20T11:33:16.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:16.937+0000 7fce3fe00640 1 --2- 172.21.15.152:0/703789581 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fce48071cd0 0x7fce4815d930 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7fce3000d830 tx=0x7fce3000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:16.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:16.937+0000 7fce3ea00640 1 -- 172.21.15.152:0/703789581 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce3000b660 con 0x7fce48071cd0 2024-04-20T11:33:16.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:16.937+0000 7fce3ea00640 1 -- 172.21.15.152:0/703789581 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce30019070 con 0x7fce48071cd0 2024-04-20T11:33:16.938 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:16.937+0000 7fce47e00640 1 -- 172.21.15.152:0/703789581 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce481e7e00 con 0x7fce48071cd0 2024-04-20T11:33:16.938 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:16.937+0000 7fce3ea00640 1 -- 172.21.15.152:0/703789581 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce3000ba20 con 0x7fce48071cd0 2024-04-20T11:33:16.938 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:16.937+0000 7fce47e00640 1 -- 172.21.15.152:0/703789581 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce481e83c0 con 0x7fce48071cd0 2024-04-20T11:33:16.938 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:16.938+0000 7fce47e00640 1 -- 172.21.15.152:0/703789581 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce48075e60 con 0x7fce48071cd0 2024-04-20T11:33:16.939 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:16.939+0000 7fce3ea00640 1 -- 172.21.15.152:0/703789581 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 91142+0+0 (secure 0 0 0) 0x7fce3001e460 con 0x7fce48071cd0 2024-04-20T11:33:16.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:16.940+0000 7fce3ea00640 1 --2- 172.21.15.152:0/703789581 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fce2406ca10 0x7fce2406eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:16.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:16.940+0000 7fce3ea00640 1 -- 172.21.15.152:0/703789581 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5029+0+0 (secure 0 0 0) 0x7fce3008c0c0 con 0x7fce48071cd0 2024-04-20T11:33:16.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:16.941+0000 7fce46a00640 1 --2- 172.21.15.152:0/703789581 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fce2406ca10 0x7fce2406eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:16.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:16.942+0000 7fce46a00640 1 --2- 172.21.15.152:0/703789581 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fce2406ca10 0x7fce2406eed0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fce38006fd0 tx=0x7fce38008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:16.944 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:16.945+0000 7fce3ea00640 1 -- 172.21.15.152:0/703789581 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fce3005a5f0 con 0x7fce48071cd0 2024-04-20T11:33:17.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.154+0000 7fce47e00640 1 -- 172.21.15.152:0/703789581 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fce48072d30 con 0x7fce48071cd0 2024-04-20T11:33:17.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.161+0000 7fce3ea00640 1 -- 172.21.15.152:0/703789581 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v19) ==== 104+0+0 (secure 0 0 0) 0x7fce30059ef0 con 0x7fce48071cd0 2024-04-20T11:33:17.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.165+0000 7fce47e00640 1 -- 172.21.15.152:0/703789581 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fce2406ca10 msgr2=0x7fce2406eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:17.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.165+0000 7fce47e00640 1 --2- 172.21.15.152:0/703789581 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fce2406ca10 0x7fce2406eed0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fce38006fd0 tx=0x7fce38008040 comp rx=0 tx=0).stop 2024-04-20T11:33:17.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.165+0000 7fce47e00640 1 -- 172.21.15.152:0/703789581 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fce48071cd0 msgr2=0x7fce4815d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:17.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.165+0000 7fce47e00640 1 --2- 172.21.15.152:0/703789581 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fce48071cd0 0x7fce4815d930 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7fce3000d830 tx=0x7fce3000dcf0 comp rx=0 tx=0).stop 2024-04-20T11:33:17.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.165+0000 7fce47e00640 1 -- 172.21.15.152:0/703789581 shutdown_connections 2024-04-20T11:33:17.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.165+0000 7fce47e00640 1 --2- 172.21.15.152:0/703789581 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fce2406ca10 0x7fce2406eed0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:17.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.165+0000 7fce47e00640 1 --2- 172.21.15.152:0/703789581 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fce48160220 0x7fce4813e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:17.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.165+0000 7fce47e00640 1 --2- 172.21.15.152:0/703789581 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fce48071cd0 0x7fce4815d930 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:17.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.165+0000 7fce47e00640 1 --2- 172.21.15.152:0/703789581 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce4806eb30 0x7fce4815f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:17.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.165+0000 7fce47e00640 1 -- 172.21.15.152:0/703789581 >> 172.21.15.152:0/703789581 conn(0x7fce48067ae0 msgr2=0x7fce48153160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:17.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.165+0000 7fce47e00640 1 -- 172.21.15.152:0/703789581 shutdown_connections 2024-04-20T11:33:17.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.165+0000 7fce47e00640 1 -- 172.21.15.152:0/703789581 wait complete. 2024-04-20T11:33:17.186 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_names_can_only_be_goodchars' 2024-04-20T11:33:17.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.386+0000 7f3458200640 1 Processor -- start 2024-04-20T11:33:17.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.386+0000 7f3458200640 1 -- start start 2024-04-20T11:33:17.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.388+0000 7f3458200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f345015eae0 0x7f34500d2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:17.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.388+0000 7f3458200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f34500d1f60 0x7f34500b7a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:17.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.388+0000 7f3458200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34500d3190 con 0x7f34500ba210 2024-04-20T11:33:17.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.388+0000 7f3458200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f345015eec0 con 0x7f34500d1f60 2024-04-20T11:33:17.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.388+0000 7f3458200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f34500d3410 con 0x7f345015eae0 2024-04-20T11:33:17.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.389+0000 7f3456200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f34500d1f60 0x7f34500b7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:17.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.389+0000 7f3455800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f345015eae0 0x7f34500d2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:17.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.389+0000 7f3456200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f34500d1f60 0x7f34500b7a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.152:50544/0 (socket says 172.21.15.152:50544) 2024-04-20T11:33:17.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.389+0000 7f3456200640 1 -- 172.21.15.152:0/1278267132 learned_addr learned my addr 172.21.15.152:0/1278267132 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:33:17.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.390+0000 7f3456200640 1 -- 172.21.15.152:0/1278267132 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f345015eae0 msgr2=0x7f34500d2c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:17.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.390+0000 7f3456200640 1 --2- 172.21.15.152:0/1278267132 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f345015eae0 0x7f34500d2c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:17.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.390+0000 7f3456200640 1 -- 172.21.15.152:0/1278267132 >> v1:172.21.15.175:6789/0 conn(0x7f34500ba210 legacy=0x7f34500d1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:17.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.390+0000 7f3456200640 1 -- 172.21.15.152:0/1278267132 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34500d36f0 con 0x7f34500d1f60 2024-04-20T11:33:17.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.390+0000 7f3455800640 1 --2- 172.21.15.152:0/1278267132 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f345015eae0 0x7f34500d2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:33:17.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.390+0000 7f344fe00640 1 -- 172.21.15.152:0/1278267132 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1901697391 0 0) 0x7f3440001490 con 0x7f34500ba210 2024-04-20T11:33:17.391 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.391+0000 7f3456200640 1 --2- 172.21.15.152:0/1278267132 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f34500d1f60 0x7f34500b7a20 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f3448009870 tx=0x7f344802b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5399420b9753028 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:17.391 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.392+0000 7f344fe00640 1 -- 172.21.15.152:0/1278267132 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3448034070 con 0x7f34500d1f60 2024-04-20T11:33:17.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.392+0000 7f344fe00640 1 -- 172.21.15.152:0/1278267132 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3448030840 con 0x7f34500d1f60 2024-04-20T11:33:17.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.392+0000 7f344fe00640 1 -- 172.21.15.152:0/1278267132 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3448030c10 con 0x7f34500d1f60 2024-04-20T11:33:17.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.393+0000 7f3458200640 1 -- 172.21.15.152:0/1278267132 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f34500d1f60 msgr2=0x7f34500b7a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:17.393 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.393+0000 7f3458200640 1 --2- 172.21.15.152:0/1278267132 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f34500d1f60 0x7f34500b7a20 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f3448009870 tx=0x7f344802b980 comp rx=0 tx=0).stop 2024-04-20T11:33:17.393 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.393+0000 7f3458200640 1 -- 172.21.15.152:0/1278267132 shutdown_connections 2024-04-20T11:33:17.393 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.393+0000 7f3458200640 1 --2- 172.21.15.152:0/1278267132 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f34500d1f60 0x7f34500b7a20 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:17.393 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.393+0000 7f3458200640 1 --2- 172.21.15.152:0/1278267132 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f345015eae0 0x7f34500d2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:17.393 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.393+0000 7f3458200640 1 -- 172.21.15.152:0/1278267132 >> 172.21.15.152:0/1278267132 conn(0x7f345006d260 msgr2=0x7f345006d670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:17.393 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.393+0000 7f3458200640 1 -- 172.21.15.152:0/1278267132 shutdown_connections 2024-04-20T11:33:17.393 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.393+0000 7f3458200640 1 -- 172.21.15.152:0/1278267132 wait complete. 2024-04-20T11:33:17.394 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.395+0000 7f3458200640 1 Processor -- start 2024-04-20T11:33:17.395 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.395+0000 7f3458200640 1 -- start start 2024-04-20T11:33:17.396 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.396+0000 7f3458200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f34500ba210 0x7f345014bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:17.396 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.396+0000 7f3458200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f34500d1f60 0x7f345015f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:17.396 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.396+0000 7f3458200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f345015eae0 0x7f345015d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:17.396 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.396+0000 7f3458200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f34500d3f90 con 0x7f34500d1f60 2024-04-20T11:33:17.396 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.396+0000 7f3458200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f34500d3cb0 con 0x7f34500ba210 2024-04-20T11:33:17.396 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.396+0000 7f3458200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f34500d44b0 con 0x7f345015eae0 2024-04-20T11:33:17.398 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.397+0000 7f3454e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f34500d1f60 0x7f345015f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:17.398 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.397+0000 7f3454e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f34500d1f60 0x7f345015f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:37424/0 (socket says 172.21.15.152:37424) 2024-04-20T11:33:17.398 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.397+0000 7f3454e00640 1 -- 172.21.15.152:0/1224340235 learned_addr learned my addr 172.21.15.152:0/1224340235 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:33:17.398 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.397+0000 7f3456200640 1 --2- 172.21.15.152:0/1224340235 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f345015eae0 0x7f345015d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:17.398 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.397+0000 7f3455800640 1 --2- 172.21.15.152:0/1224340235 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f34500ba210 0x7f345014bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:17.399 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.397+0000 7f3454e00640 1 -- 172.21.15.152:0/1224340235 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f345015eae0 msgr2=0x7f345015d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:17.399 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.397+0000 7f3454e00640 1 --2- 172.21.15.152:0/1224340235 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f345015eae0 0x7f345015d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:17.399 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.398+0000 7f3454e00640 1 -- 172.21.15.152:0/1224340235 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f34500ba210 msgr2=0x7f345014bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:17.399 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.398+0000 7f3454e00640 1 --2- 172.21.15.152:0/1224340235 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f34500ba210 0x7f345014bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:17.399 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.398+0000 7f3454e00640 1 -- 172.21.15.152:0/1224340235 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f345014c4f0 con 0x7f34500d1f60 2024-04-20T11:33:17.399 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.398+0000 7f3454e00640 1 --2- 172.21.15.152:0/1224340235 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f34500d1f60 0x7f345015f050 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f344000c8c0 tx=0x7f344000cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:17.399 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.398+0000 7f344d600640 1 -- 172.21.15.152:0/1224340235 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3440017070 con 0x7f34500d1f60 2024-04-20T11:33:17.399 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.398+0000 7f344d600640 1 -- 172.21.15.152:0/1224340235 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3440003de0 con 0x7f34500d1f60 2024-04-20T11:33:17.399 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.398+0000 7f3458200640 1 -- 172.21.15.152:0/1224340235 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f345015f590 con 0x7f34500d1f60 2024-04-20T11:33:17.399 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.398+0000 7f344d600640 1 -- 172.21.15.152:0/1224340235 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34400052b0 con 0x7f34500d1f60 2024-04-20T11:33:17.399 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.398+0000 7f3458200640 1 -- 172.21.15.152:0/1224340235 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f345015fb10 con 0x7f34500d1f60 2024-04-20T11:33:17.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.399+0000 7f3458200640 1 -- 172.21.15.152:0/1224340235 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34500689f0 con 0x7f34500d1f60 2024-04-20T11:33:17.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.400+0000 7f344d600640 1 -- 172.21.15.152:0/1224340235 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 91142+0+0 (secure 0 0 0) 0x7f344001a070 con 0x7f34500d1f60 2024-04-20T11:33:17.401 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.401+0000 7f344d600640 1 --2- 172.21.15.152:0/1224340235 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f343406ca10 0x7f343406eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:17.401 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.401+0000 7f344d600640 1 -- 172.21.15.152:0/1224340235 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5029+0+0 (secure 0 0 0) 0x7f344008b150 con 0x7f34500d1f60 2024-04-20T11:33:17.401 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.402+0000 7f3455800640 1 --2- 172.21.15.152:0/1224340235 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f343406ca10 0x7f343406eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:17.402 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.403+0000 7f3455800640 1 --2- 172.21.15.152:0/1224340235 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f343406ca10 0x7f343406eed0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3444009a30 tx=0x7f3444008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:17.405 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.406+0000 7f344d600640 1 -- 172.21.15.152:0/1224340235 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f34500d3190 con 0x7f34500d1f60 2024-04-20T11:33:17.664 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.664+0000 7f3458200640 1 -- 172.21.15.152:0/1224340235 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_names_can_only_be_goodchars"]} v 0) -- 0x7f345006d9c0 con 0x7f34500d1f60 2024-04-20T11:33:17.710 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.710+0000 7f344d600640 1 -- 172.21.15.152:0/1224340235 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_names_can_only_be_goodchars"]}]=0 v340) ==== 146+0+0 (secure 0 0 0) 0x7f3440059020 con 0x7f34500d1f60 2024-04-20T11:33:17.713 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.713+0000 7f3458200640 1 -- 172.21.15.152:0/1224340235 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f343406ca10 msgr2=0x7f343406eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:17.714 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.713+0000 7f3458200640 1 --2- 172.21.15.152:0/1224340235 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f343406ca10 0x7f343406eed0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3444009a30 tx=0x7f3444008040 comp rx=0 tx=0).stop 2024-04-20T11:33:17.714 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.713+0000 7f3458200640 1 -- 172.21.15.152:0/1224340235 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f34500d1f60 msgr2=0x7f345015f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:17.714 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.713+0000 7f3458200640 1 --2- 172.21.15.152:0/1224340235 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f34500d1f60 0x7f345015f050 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f344000c8c0 tx=0x7f344000cd80 comp rx=0 tx=0).stop 2024-04-20T11:33:17.714 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.714+0000 7f3458200640 1 -- 172.21.15.152:0/1224340235 shutdown_connections 2024-04-20T11:33:17.714 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.714+0000 7f3458200640 1 --2- 172.21.15.152:0/1224340235 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f343406ca10 0x7f343406eed0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:17.714 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.714+0000 7f3458200640 1 --2- 172.21.15.152:0/1224340235 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f345015eae0 0x7f345015d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:17.714 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.714+0000 7f3458200640 1 --2- 172.21.15.152:0/1224340235 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f34500d1f60 0x7f345015f050 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:17.714 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.714+0000 7f3458200640 1 --2- 172.21.15.152:0/1224340235 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f34500ba210 0x7f345014bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:17.714 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.714+0000 7f3458200640 1 -- 172.21.15.152:0/1224340235 >> 172.21.15.152:0/1224340235 conn(0x7f345006d260 msgr2=0x7f34500d50a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:17.714 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.714+0000 7f3458200640 1 -- 172.21.15.152:0/1224340235 shutdown_connections 2024-04-20T11:33:17.714 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.714+0000 7f3458200640 1 -- 172.21.15.152:0/1224340235 wait complete. 2024-04-20T11:33:17.732 INFO:tasks.cephfs_test_runner:Test the creating vols, subvols subvolgroups fails when their names uses ... ok 2024-04-20T11:33:17.732 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_no_upgrade_v1_sanity (tasks.cephfs.test_volumes.TestMisc) poor man's upgrade test -- theme continues... 2024-04-20T11:33:17.732 INFO:tasks.cephfs_test_runner:test_subvolume_no_upgrade_v1_sanity (tasks.cephfs.test_volumes.TestMisc) 2024-04-20T11:33:17.733 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:33:17.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.924+0000 7fdd27e00640 1 Processor -- start 2024-04-20T11:33:17.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.924+0000 7fdd27e00640 1 -- start start 2024-04-20T11:33:17.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.926+0000 7fdd27e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd2815d1e0 0x7fdd28160b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:17.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.926+0000 7fdd27e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fdd2815d5c0 0x7fdd280cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:17.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.926+0000 7fdd27e00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd280ce8c0 con 0x7fdd28071cd0 2024-04-20T11:33:17.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.926+0000 7fdd27e00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fdd281600b0 con 0x7fdd2815d5c0 2024-04-20T11:33:17.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.926+0000 7fdd27e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fdd28160390 con 0x7fdd2815d1e0 2024-04-20T11:33:17.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.926+0000 7fdd26000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fdd2815d5c0 0x7fdd280cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:17.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.926+0000 7fdd26000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fdd2815d5c0 0x7fdd280cdfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:37432/0 (socket says 172.21.15.152:37432) 2024-04-20T11:33:17.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.926+0000 7fdd26000640 1 -- 172.21.15.152:0/205067609 learned_addr learned my addr 172.21.15.152:0/205067609 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:33:17.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.927+0000 7fdd26a00640 1 --2- 172.21.15.152:0/205067609 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd2815d1e0 0x7fdd28160b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:17.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.927+0000 7fdd26000640 1 -- 172.21.15.152:0/205067609 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd2815d1e0 msgr2=0x7fdd28160b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:17.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.927+0000 7fdd26000640 1 --2- 172.21.15.152:0/205067609 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd2815d1e0 0x7fdd28160b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:17.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.927+0000 7fdd26000640 1 -- 172.21.15.152:0/205067609 >> v1:172.21.15.175:6789/0 conn(0x7fdd28071cd0 legacy=0x7fdd280ccd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:17.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.927+0000 7fdd26000640 1 -- 172.21.15.152:0/205067609 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd28160670 con 0x7fdd2815d5c0 2024-04-20T11:33:17.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.928+0000 7fdd26000640 1 --2- 172.21.15.152:0/205067609 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdd2815d5c0 0x7fdd280cdfe0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7fdd14009870 tx=0x7fdd1402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34a5bf6388d9f690 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:17.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.928+0000 7fdd25600640 1 -- 172.21.15.152:0/205067609 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd14035020 con 0x7fdd2815d5c0 2024-04-20T11:33:17.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.928+0000 7fdd25600640 1 -- 172.21.15.152:0/205067609 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd14031640 con 0x7fdd2815d5c0 2024-04-20T11:33:17.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.928+0000 7fdd25600640 1 -- 172.21.15.152:0/205067609 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd1402d040 con 0x7fdd2815d5c0 2024-04-20T11:33:17.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.929+0000 7fdd27e00640 1 -- 172.21.15.152:0/205067609 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdd2815d5c0 msgr2=0x7fdd280cdfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:17.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.929+0000 7fdd27e00640 1 --2- 172.21.15.152:0/205067609 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdd2815d5c0 0x7fdd280cdfe0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7fdd14009870 tx=0x7fdd1402b8f0 comp rx=0 tx=0).stop 2024-04-20T11:33:17.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.929+0000 7fdd27e00640 1 -- 172.21.15.152:0/205067609 shutdown_connections 2024-04-20T11:33:17.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.929+0000 7fdd27e00640 1 --2- 172.21.15.152:0/205067609 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdd2815d5c0 0x7fdd280cdfe0 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:17.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.929+0000 7fdd27e00640 1 --2- 172.21.15.152:0/205067609 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd2815d1e0 0x7fdd28160b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:17.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.929+0000 7fdd27e00640 1 -- 172.21.15.152:0/205067609 >> 172.21.15.152:0/205067609 conn(0x7fdd28067ae0 msgr2=0x7fdd2806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:17.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.929+0000 7fdd27e00640 1 -- 172.21.15.152:0/205067609 shutdown_connections 2024-04-20T11:33:17.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.929+0000 7fdd27e00640 1 -- 172.21.15.152:0/205067609 wait complete. 2024-04-20T11:33:17.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.931+0000 7fdd27e00640 1 Processor -- start 2024-04-20T11:33:17.931 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.931+0000 7fdd27e00640 1 -- start start 2024-04-20T11:33:17.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.932+0000 7fdd27e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd28071cd0 0x7fdd28159b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:17.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.932+0000 7fdd27e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdd2815d1e0 0x7fdd2814ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:17.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.932+0000 7fdd27e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd2815d5c0 0x7fdd281f44f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:17.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.932+0000 7fdd27e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fdd280ce6a0 con 0x7fdd2815d1e0 2024-04-20T11:33:17.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.932+0000 7fdd27e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdd280ce520 con 0x7fdd2815d5c0 2024-04-20T11:33:17.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.932+0000 7fdd27e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fdd28072160 con 0x7fdd28071cd0 2024-04-20T11:33:17.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.932+0000 7fdd26000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdd2815d1e0 0x7fdd2814ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:17.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.932+0000 7fdd26000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdd2815d1e0 0x7fdd2814ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:37436/0 (socket says 172.21.15.152:37436) 2024-04-20T11:33:17.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.932+0000 7fdd26000640 1 -- 172.21.15.152:0/31363525 learned_addr learned my addr 172.21.15.152:0/31363525 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:33:17.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.933+0000 7fdd26000640 1 -- 172.21.15.152:0/31363525 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd28071cd0 msgr2=0x7fdd28159b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:17.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.933+0000 7fdd26000640 1 --2- 172.21.15.152:0/31363525 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd28071cd0 0x7fdd28159b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:17.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.933+0000 7fdd26000640 1 -- 172.21.15.152:0/31363525 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd2815d5c0 msgr2=0x7fdd281f44f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:17.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.933+0000 7fdd26000640 1 --2- 172.21.15.152:0/31363525 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd2815d5c0 0x7fdd281f44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:17.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.933+0000 7fdd26000640 1 -- 172.21.15.152:0/31363525 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd280ae3f0 con 0x7fdd2815d1e0 2024-04-20T11:33:17.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.933+0000 7fdd26000640 1 --2- 172.21.15.152:0/31363525 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdd2815d1e0 0x7fdd2814ac70 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7fdd1402b930 tx=0x7fdd140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:17.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.933+0000 7fdd1ea00640 1 -- 172.21.15.152:0/31363525 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd1403aa60 con 0x7fdd2815d1e0 2024-04-20T11:33:17.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.933+0000 7fdd1ea00640 1 -- 172.21.15.152:0/31363525 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd1403ac20 con 0x7fdd2815d1e0 2024-04-20T11:33:17.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.933+0000 7fdd1ea00640 1 -- 172.21.15.152:0/31363525 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd140443f0 con 0x7fdd2815d1e0 2024-04-20T11:33:17.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.933+0000 7fdd27e00640 1 -- 172.21.15.152:0/31363525 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd280ae6d0 con 0x7fdd2815d1e0 2024-04-20T11:33:17.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.934+0000 7fdd27e00640 1 -- 172.21.15.152:0/31363525 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd280aec90 con 0x7fdd2815d1e0 2024-04-20T11:33:17.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.935+0000 7fdd27e00640 1 -- 172.21.15.152:0/31363525 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd28075e60 con 0x7fdd2815d1e0 2024-04-20T11:33:17.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.936+0000 7fdd1ea00640 1 -- 172.21.15.152:0/31363525 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 91142+0+0 (secure 0 0 0) 0x7fdd1404b060 con 0x7fdd2815d1e0 2024-04-20T11:33:17.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.937+0000 7fdd1ea00640 1 --2- 172.21.15.152:0/31363525 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fdd0806ca10 0x7fdd0806eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:17.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.937+0000 7fdd1ea00640 1 -- 172.21.15.152:0/31363525 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5029+0+0 (secure 0 0 0) 0x7fdd140ad8a0 con 0x7fdd2815d1e0 2024-04-20T11:33:17.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.938+0000 7fdd26a00640 1 --2- 172.21.15.152:0/31363525 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fdd0806ca10 0x7fdd0806eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:17.938 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.939+0000 7fdd26a00640 1 --2- 172.21.15.152:0/31363525 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fdd0806ca10 0x7fdd0806eed0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fdd180098f0 tx=0x7fdd18008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:17.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:17.942+0000 7fdd1ea00640 1 -- 172.21.15.152:0/31363525 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fdd1403f070 con 0x7fdd2815d1e0 2024-04-20T11:33:18.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.144+0000 7fdd27e00640 1 -- 172.21.15.152:0/31363525 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdd280cd9d0 con 0x7fdd2815d1e0 2024-04-20T11:33:18.145 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.145+0000 7fdd1ea00640 1 -- 172.21.15.152:0/31363525 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+13215 (secure 0 0 0) 0x7fdd1402f300 con 0x7fdd2815d1e0 2024-04-20T11:33:18.145 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:33:18.145 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":94,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:33:16.697058+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.152:6833/3577228528":"2024-04-21T11:33:13.279021+0000","172.21.15.152:6832/3577228528":"2024-04-21T11:33:13.279021+0000","172.21.15.175:6835/2632275131":"2024-04-21T11:32:59.571722+0000","172.21.15.175:6834/2632275131":"2024-04-21T11:32:59.571722+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:33:18.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.149+0000 7fdd27e00640 1 -- 172.21.15.152:0/31363525 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fdd0806ca10 msgr2=0x7fdd0806eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:18.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.149+0000 7fdd27e00640 1 --2- 172.21.15.152:0/31363525 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fdd0806ca10 0x7fdd0806eed0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fdd180098f0 tx=0x7fdd18008040 comp rx=0 tx=0).stop 2024-04-20T11:33:18.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.149+0000 7fdd27e00640 1 -- 172.21.15.152:0/31363525 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdd2815d1e0 msgr2=0x7fdd2814ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:18.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.149+0000 7fdd27e00640 1 --2- 172.21.15.152:0/31363525 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdd2815d1e0 0x7fdd2814ac70 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7fdd1402b930 tx=0x7fdd140046c0 comp rx=0 tx=0).stop 2024-04-20T11:33:18.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.149+0000 7fdd27e00640 1 -- 172.21.15.152:0/31363525 shutdown_connections 2024-04-20T11:33:18.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.149+0000 7fdd27e00640 1 --2- 172.21.15.152:0/31363525 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fdd0806ca10 0x7fdd0806eed0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:18.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.149+0000 7fdd27e00640 1 --2- 172.21.15.152:0/31363525 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd2815d5c0 0x7fdd281f44f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:18.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.149+0000 7fdd27e00640 1 --2- 172.21.15.152:0/31363525 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdd2815d1e0 0x7fdd2814ac70 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:18.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.149+0000 7fdd27e00640 1 --2- 172.21.15.152:0/31363525 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd28071cd0 0x7fdd28159b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:18.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.149+0000 7fdd27e00640 1 -- 172.21.15.152:0/31363525 >> 172.21.15.152:0/31363525 conn(0x7fdd28067ae0 msgr2=0x7fdd2806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:18.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.149+0000 7fdd27e00640 1 -- 172.21.15.152:0/31363525 shutdown_connections 2024-04-20T11:33:18.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.149+0000 7fdd27e00640 1 -- 172.21.15.152:0/31363525 wait complete. 2024-04-20T11:33:18.168 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T11:26:47.246571+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T11:33:18.168 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T11:33:18.371 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.371+0000 7fd353200640 1 Processor -- start 2024-04-20T11:33:18.371 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.371+0000 7fd353200640 1 -- start start 2024-04-20T11:33:18.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.372+0000 7fd353200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fd35415c590 0x7fd35406bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:18.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.372+0000 7fd353200640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd354152830 con 0x7fd354152450 2024-04-20T11:33:18.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.372+0000 7fd353200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3540b39c0 con 0x7fd35406c2e0 2024-04-20T11:33:18.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.372+0000 7fd353200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fd35406c750 con 0x7fd35415c590 2024-04-20T11:33:18.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.372+0000 7fd351e00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fd354152450 0x7fd3540b30c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:55148/0 (socket says 172.21.15.152:55148) 2024-04-20T11:33:18.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.373+0000 7fd351e00640 1 -- 172.21.15.152:0/1096434334 learned_addr learned my addr 172.21.15.152:0/1096434334 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:33:18.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.373+0000 7fd350a00640 1 -- 172.21.15.152:0/1096434334 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3153142554 0 0) 0x7fd354152830 con 0x7fd354152450 2024-04-20T11:33:18.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.373+0000 7fd350a00640 1 -- 172.21.15.152:0/1096434334 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd338003520 con 0x7fd354152450 2024-04-20T11:33:18.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.373+0000 7fd351400640 1 --2- 172.21.15.152:0/1096434334 >> v2:172.21.15.175:3300/0 conn(0x7fd35415c590 0x7fd35406bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:18.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.373+0000 7fd350a00640 1 -- 172.21.15.152:0/1096434334 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3966188251 0 0) 0x7fd338003520 con 0x7fd354152450 2024-04-20T11:33:18.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.373+0000 7fd350a00640 1 -- 172.21.15.152:0/1096434334 >> v2:172.21.15.175:3300/0 conn(0x7fd35415c590 msgr2=0x7fd35406bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:18.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.373+0000 7fd350a00640 1 --2- 172.21.15.152:0/1096434334 >> v2:172.21.15.175:3300/0 conn(0x7fd35415c590 0x7fd35406bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:18.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.373+0000 7fd350a00640 1 -- 172.21.15.152:0/1096434334 >> v1:172.21.15.175:6789/0 conn(0x7fd35406c2e0 legacy=0x7fd3540ae470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:18.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.373+0000 7fd350a00640 1 -- 172.21.15.152:0/1096434334 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3540b3ca0 con 0x7fd354152450 2024-04-20T11:33:18.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.373+0000 7fd350a00640 1 -- 172.21.15.152:0/1096434334 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7fd344001670 con 0x7fd354152450 2024-04-20T11:33:18.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.373+0000 7fd350a00640 1 -- 172.21.15.152:0/1096434334 >> v1:172.21.15.152:6789/0 conn(0x7fd354152450 legacy=0x7fd3540b30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:18.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.373+0000 7fd350a00640 1 --2- 172.21.15.152:0/1096434334 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd3380039b0 0x7fd338003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:18.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.373+0000 7fd350a00640 1 --2- 172.21.15.152:0/1096434334 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd338004800 0x7fd338004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:18.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.373+0000 7fd350a00640 1 --2- 172.21.15.152:0/1096434334 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd3380051c0 0x7fd33800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:18.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.374+0000 7fd350a00640 1 -- 172.21.15.152:0/1096434334 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fd35406c750 con 0x7fd3380051c0 2024-04-20T11:33:18.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.374+0000 7fd350a00640 1 -- 172.21.15.152:0/1096434334 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd33800c0f0 con 0x7fd338004800 2024-04-20T11:33:18.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.374+0000 7fd350a00640 1 -- 172.21.15.152:0/1096434334 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd33800c2f0 con 0x7fd3380039b0 2024-04-20T11:33:18.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.374+0000 7fd352800640 1 --2- 172.21.15.152:0/1096434334 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd3380051c0 0x7fd33800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:18.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.374+0000 7fd351400640 1 --2- 172.21.15.152:0/1096434334 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd338004800 0x7fd338004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:18.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.374+0000 7fd352800640 1 -- 172.21.15.152:0/1096434334 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd3380039b0 msgr2=0x7fd338003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:18.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.374+0000 7fd352800640 1 --2- 172.21.15.152:0/1096434334 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd3380039b0 0x7fd338003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:18.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.374+0000 7fd352800640 1 -- 172.21.15.152:0/1096434334 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd338004800 msgr2=0x7fd338004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:18.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.374+0000 7fd352800640 1 --2- 172.21.15.152:0/1096434334 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd338004800 0x7fd338004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:18.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.374+0000 7fd352800640 1 -- 172.21.15.152:0/1096434334 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd338003520 con 0x7fd3380051c0 2024-04-20T11:33:18.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.375+0000 7fd352800640 1 -- 172.21.15.152:0/1096434334 reap_dead start 2024-04-20T11:33:18.377 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.375+0000 7fd352800640 1 --2- 172.21.15.152:0/1096434334 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd3380051c0 0x7fd33800b9c0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fd34c004480 tx=0x7fd338005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84dce9007318ef79 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:18.378 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.375+0000 7fd350a00640 1 -- 172.21.15.152:0/1096434334 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd34c034070 con 0x7fd3380051c0 2024-04-20T11:33:18.378 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.376+0000 7fd350a00640 1 -- 172.21.15.152:0/1096434334 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd34c006b50 con 0x7fd3380051c0 2024-04-20T11:33:18.378 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.376+0000 7fd350a00640 1 -- 172.21.15.152:0/1096434334 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd34c004080 con 0x7fd3380051c0 2024-04-20T11:33:18.378 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.376+0000 7fd353200640 1 -- 172.21.15.152:0/1096434334 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd3380051c0 msgr2=0x7fd33800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:18.378 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.376+0000 7fd353200640 1 --2- 172.21.15.152:0/1096434334 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd3380051c0 0x7fd33800b9c0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fd34c004480 tx=0x7fd338005830 comp rx=0 tx=0).stop 2024-04-20T11:33:18.378 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.376+0000 7fd353200640 1 -- 172.21.15.152:0/1096434334 shutdown_connections 2024-04-20T11:33:18.378 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.376+0000 7fd353200640 1 --2- 172.21.15.152:0/1096434334 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd3380051c0 0x7fd33800b9c0 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:18.378 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.376+0000 7fd353200640 1 -- 172.21.15.152:0/1096434334 >> 172.21.15.152:0/1096434334 conn(0x7fd3540b21e0 msgr2=0x7fd3540b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:18.378 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.376+0000 7fd353200640 1 -- 172.21.15.152:0/1096434334 shutdown_connections 2024-04-20T11:33:18.378 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.376+0000 7fd353200640 1 -- 172.21.15.152:0/1096434334 wait complete. 2024-04-20T11:33:18.378 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.378+0000 7fd353200640 1 Processor -- start 2024-04-20T11:33:18.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.378+0000 7fd353200640 1 -- start start 2024-04-20T11:33:18.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.379+0000 7fd353200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd3380051c0 0x7fd35406de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:18.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.379+0000 7fd353200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3540cca50 0x7fd3540cced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:18.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.379+0000 7fd353200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd35415d9c0 0x7fd35415de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:18.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.379+0000 7fd353200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fd3540adf00 con 0x7fd35415d9c0 2024-04-20T11:33:18.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.379+0000 7fd353200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd3540b4260 con 0x7fd3540cca50 2024-04-20T11:33:18.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.379+0000 7fd353200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd35406bda0 con 0x7fd3380051c0 2024-04-20T11:33:18.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.379+0000 7fd352800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd35415d9c0 0x7fd35415de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:18.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.379+0000 7fd352800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd35415d9c0 0x7fd35415de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:37450/0 (socket says 172.21.15.152:37450) 2024-04-20T11:33:18.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.379+0000 7fd352800640 1 -- 172.21.15.152:0/3498878288 learned_addr learned my addr 172.21.15.152:0/3498878288 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:33:18.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.380+0000 7fd351e00640 1 --2- 172.21.15.152:0/3498878288 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd3380051c0 0x7fd35406de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:18.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.380+0000 7fd351400640 1 --2- 172.21.15.152:0/3498878288 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3540cca50 0x7fd3540cced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:18.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.380+0000 7fd352800640 1 -- 172.21.15.152:0/3498878288 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd3380051c0 msgr2=0x7fd35406de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:18.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.380+0000 7fd352800640 1 --2- 172.21.15.152:0/3498878288 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd3380051c0 0x7fd35406de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:18.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.380+0000 7fd352800640 1 -- 172.21.15.152:0/3498878288 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3540cca50 msgr2=0x7fd3540cced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:18.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.380+0000 7fd352800640 1 --2- 172.21.15.152:0/3498878288 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3540cca50 0x7fd3540cced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:18.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.380+0000 7fd352800640 1 -- 172.21.15.152:0/3498878288 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3540ce7b0 con 0x7fd35415d9c0 2024-04-20T11:33:18.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.380+0000 7fd352800640 1 --2- 172.21.15.152:0/3498878288 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd35415d9c0 0x7fd35415de40 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7fd34c0045a0 tx=0x7fd34c003a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:18.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.380+0000 7fd34a000640 1 -- 172.21.15.152:0/3498878288 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd34c034040 con 0x7fd35415d9c0 2024-04-20T11:33:18.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.380+0000 7fd34a000640 1 -- 172.21.15.152:0/3498878288 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd34c038600 con 0x7fd35415d9c0 2024-04-20T11:33:18.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.380+0000 7fd34a000640 1 -- 172.21.15.152:0/3498878288 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd34c007df0 con 0x7fd35415d9c0 2024-04-20T11:33:18.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.380+0000 7fd353200640 1 -- 172.21.15.152:0/3498878288 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3540cd710 con 0x7fd35415d9c0 2024-04-20T11:33:18.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.380+0000 7fd353200640 1 -- 172.21.15.152:0/3498878288 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3540cdcd0 con 0x7fd35415d9c0 2024-04-20T11:33:18.383 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.381+0000 7fd353200640 1 -- 172.21.15.152:0/3498878288 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd354075e60 con 0x7fd35415d9c0 2024-04-20T11:33:18.383 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.382+0000 7fd34a000640 1 -- 172.21.15.152:0/3498878288 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 91142+0+0 (secure 0 0 0) 0x7fd34c0389f0 con 0x7fd35415d9c0 2024-04-20T11:33:18.383 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.383+0000 7fd34a000640 1 --2- 172.21.15.152:0/3498878288 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd33406ca10 0x7fd33406eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:18.383 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.383+0000 7fd34a000640 1 -- 172.21.15.152:0/3498878288 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5029+0+0 (secure 0 0 0) 0x7fd34c0ac590 con 0x7fd35415d9c0 2024-04-20T11:33:18.383 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.384+0000 7fd351e00640 1 --2- 172.21.15.152:0/3498878288 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd33406ca10 0x7fd33406eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:18.384 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.385+0000 7fd351e00640 1 --2- 172.21.15.152:0/3498878288 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd33406ca10 0x7fd33406eed0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd3440079b0 tx=0x7fd344009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:18.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.387+0000 7fd34a000640 1 -- 172.21.15.152:0/3498878288 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fd34c049020 con 0x7fd35415d9c0 2024-04-20T11:33:18.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.573+0000 7fd353200640 1 -- 172.21.15.152:0/3498878288 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd35406f770 con 0x7fd35415d9c0 2024-04-20T11:33:18.574 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.574+0000 7fd34a000640 1 -- 172.21.15.152:0/3498878288 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7fd34c07a9a0 con 0x7fd35415d9c0 2024-04-20T11:33:18.574 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 1 2024-04-20T11:33:18.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.577+0000 7fd353200640 1 -- 172.21.15.152:0/3498878288 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd33406ca10 msgr2=0x7fd33406eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:18.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.577+0000 7fd353200640 1 --2- 172.21.15.152:0/3498878288 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd33406ca10 0x7fd33406eed0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd3440079b0 tx=0x7fd344009040 comp rx=0 tx=0).stop 2024-04-20T11:33:18.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.577+0000 7fd353200640 1 -- 172.21.15.152:0/3498878288 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd35415d9c0 msgr2=0x7fd35415de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:18.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.577+0000 7fd353200640 1 --2- 172.21.15.152:0/3498878288 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd35415d9c0 0x7fd35415de40 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7fd34c0045a0 tx=0x7fd34c003a70 comp rx=0 tx=0).stop 2024-04-20T11:33:18.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.577+0000 7fd353200640 1 -- 172.21.15.152:0/3498878288 shutdown_connections 2024-04-20T11:33:18.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.577+0000 7fd353200640 1 --2- 172.21.15.152:0/3498878288 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd33406ca10 0x7fd33406eed0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:18.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.577+0000 7fd353200640 1 --2- 172.21.15.152:0/3498878288 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd35415d9c0 0x7fd35415de40 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:18.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.577+0000 7fd353200640 1 --2- 172.21.15.152:0/3498878288 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3540cca50 0x7fd3540cced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:18.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.577+0000 7fd353200640 1 --2- 172.21.15.152:0/3498878288 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd3380051c0 0x7fd35406de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:18.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.577+0000 7fd353200640 1 -- 172.21.15.152:0/3498878288 >> 172.21.15.152:0/3498878288 conn(0x7fd3540b21e0 msgr2=0x7fd3540b48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:18.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.577+0000 7fd353200640 1 -- 172.21.15.152:0/3498878288 shutdown_connections 2024-04-20T11:33:18.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.577+0000 7fd353200640 1 -- 172.21.15.152:0/3498878288 wait complete. 2024-04-20T11:33:18.595 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_sanity' 2024-04-20T11:33:18.792 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.792+0000 7f7f9f200640 1 Processor -- start 2024-04-20T11:33:18.793 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.792+0000 7f7f9f200640 1 -- start start 2024-04-20T11:33:18.795 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.794+0000 7f7f9f200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f7fa0160b90 0x7f7fa015d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:18.795 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.794+0000 7f7f9f200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7fa0071cd0 0x7f7fa0072160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:18.795 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.794+0000 7f7f9f200640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7fa00ce8c0 con 0x7f7fa015d1e0 2024-04-20T11:33:18.795 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.794+0000 7f7f9f200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f7fa00726a0 con 0x7f7fa0160b90 2024-04-20T11:33:18.795 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.794+0000 7f7f9f200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7fa01600b0 con 0x7f7fa0071cd0 2024-04-20T11:33:18.795 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.795+0000 7f7f9d400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f7fa0160b90 0x7f7fa015d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:18.795 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.795+0000 7f7f9de00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f7fa015d1e0 0x7f7fa00cdfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:55164/0 (socket says 172.21.15.152:55164) 2024-04-20T11:33:18.795 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.795+0000 7f7f9d400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f7fa0160b90 0x7f7fa015d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:37456/0 (socket says 172.21.15.152:37456) 2024-04-20T11:33:18.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.795+0000 7f7f9de00640 1 -- 172.21.15.152:0/4066447456 learned_addr learned my addr 172.21.15.152:0/4066447456 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:33:18.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.795+0000 7f7f9d400640 1 -- 172.21.15.152:0/4066447456 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7fa0071cd0 msgr2=0x7f7fa0072160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:18.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.795+0000 7f7f9d400640 1 --2- 172.21.15.152:0/4066447456 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7fa0071cd0 0x7f7fa0072160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:18.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.795+0000 7f7f9d400640 1 -- 172.21.15.152:0/4066447456 >> v1:172.21.15.152:6789/0 conn(0x7f7fa015d1e0 legacy=0x7f7fa00cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:18.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.795+0000 7f7f9d400640 1 -- 172.21.15.152:0/4066447456 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fa0160390 con 0x7f7fa0160b90 2024-04-20T11:33:18.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.795+0000 7f7f9ca00640 1 -- 172.21.15.152:0/4066447456 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1344779040 0 0) 0x7f7fa00ce8c0 con 0x7f7fa015d1e0 2024-04-20T11:33:18.797 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.797+0000 7f7f9d400640 1 --2- 172.21.15.152:0/4066447456 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7fa0160b90 0x7f7fa015d5c0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f7f90009dc0 tx=0x7f7f9002be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f6d18e72949745f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:18.797 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.797+0000 7f7f9ca00640 1 -- 172.21.15.152:0/4066447456 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f90039070 con 0x7f7fa0160b90 2024-04-20T11:33:18.797 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.797+0000 7f7f9ca00640 1 -- 172.21.15.152:0/4066447456 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f9002cc40 con 0x7f7fa0160b90 2024-04-20T11:33:18.797 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.797+0000 7f7f9ca00640 1 -- 172.21.15.152:0/4066447456 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f90034440 con 0x7f7fa0160b90 2024-04-20T11:33:18.797 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.797+0000 7f7f9f200640 1 -- 172.21.15.152:0/4066447456 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7fa0160b90 msgr2=0x7f7fa015d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:18.797 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.797+0000 7f7f9f200640 1 --2- 172.21.15.152:0/4066447456 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7fa0160b90 0x7f7fa015d5c0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f7f90009dc0 tx=0x7f7f9002be90 comp rx=0 tx=0).stop 2024-04-20T11:33:18.798 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.798+0000 7f7f9f200640 1 -- 172.21.15.152:0/4066447456 shutdown_connections 2024-04-20T11:33:18.798 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.798+0000 7f7f9f200640 1 --2- 172.21.15.152:0/4066447456 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7fa0071cd0 0x7f7fa0072160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:18.798 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.798+0000 7f7f9f200640 1 --2- 172.21.15.152:0/4066447456 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7fa0160b90 0x7f7fa015d5c0 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:18.798 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.798+0000 7f7f9f200640 1 -- 172.21.15.152:0/4066447456 >> 172.21.15.152:0/4066447456 conn(0x7f7fa0067ae0 msgr2=0x7f7fa006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:18.798 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.798+0000 7f7f9f200640 1 -- 172.21.15.152:0/4066447456 shutdown_connections 2024-04-20T11:33:18.798 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.798+0000 7f7f9f200640 1 -- 172.21.15.152:0/4066447456 wait complete. 2024-04-20T11:33:18.798 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.799+0000 7f7f9f200640 1 Processor -- start 2024-04-20T11:33:18.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.799+0000 7f7f9f200640 1 -- start start 2024-04-20T11:33:18.801 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.800+0000 7f7f9f200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7fa0071cd0 0x7f7fa0159a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:18.801 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.800+0000 7f7f9f200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7fa015d1e0 0x7f7fa0144320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:18.801 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.800+0000 7f7f9f200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7fa0160b90 0x7f7fa00b30c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:18.801 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.800+0000 7f7f9f200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f7fa00ccd70 con 0x7f7fa0160b90 2024-04-20T11:33:18.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.800+0000 7f7f9f200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7fa00ceba0 con 0x7f7fa015d1e0 2024-04-20T11:33:18.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.800+0000 7f7f9f200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7fa00ccef0 con 0x7f7fa0071cd0 2024-04-20T11:33:18.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.801+0000 7f7f9e800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7fa0160b90 0x7f7fa00b30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:18.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.801+0000 7f7f9e800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7fa0160b90 0x7f7fa00b30c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:37462/0 (socket says 172.21.15.152:37462) 2024-04-20T11:33:18.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.801+0000 7f7f9e800640 1 -- 172.21.15.152:0/3964197627 learned_addr learned my addr 172.21.15.152:0/3964197627 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:33:18.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.801+0000 7f7f9de00640 1 --2- 172.21.15.152:0/3964197627 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7fa0071cd0 0x7f7fa0159a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:18.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.801+0000 7f7f9e800640 1 -- 172.21.15.152:0/3964197627 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7fa0071cd0 msgr2=0x7f7fa0159a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:18.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.801+0000 7f7f9e800640 1 --2- 172.21.15.152:0/3964197627 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7fa0071cd0 0x7f7fa0159a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:18.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.801+0000 7f7f9e800640 1 -- 172.21.15.152:0/3964197627 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7fa015d1e0 msgr2=0x7f7fa0144320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:18.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.801+0000 7f7f9e800640 1 --2- 172.21.15.152:0/3964197627 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7fa015d1e0 0x7f7fa0144320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:18.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.801+0000 7f7f9e800640 1 -- 172.21.15.152:0/3964197627 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fa0153900 con 0x7f7fa0160b90 2024-04-20T11:33:18.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.801+0000 7f7f9e800640 1 --2- 172.21.15.152:0/3964197627 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7fa0160b90 0x7f7fa00b30c0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f7f9800b450 tx=0x7f7f9800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:18.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.802+0000 7f7f96000640 1 -- 172.21.15.152:0/3964197627 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f98015020 con 0x7f7fa0160b90 2024-04-20T11:33:18.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.802+0000 7f7f96000640 1 -- 172.21.15.152:0/3964197627 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f98011890 con 0x7f7fa0160b90 2024-04-20T11:33:18.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.802+0000 7f7f9f200640 1 -- 172.21.15.152:0/3964197627 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fa0144cc0 con 0x7f7fa0160b90 2024-04-20T11:33:18.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.802+0000 7f7f96000640 1 -- 172.21.15.152:0/3964197627 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f98011c60 con 0x7f7fa0160b90 2024-04-20T11:33:18.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.802+0000 7f7f9f200640 1 -- 172.21.15.152:0/3964197627 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fa0155580 con 0x7f7fa0160b90 2024-04-20T11:33:18.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.803+0000 7f7f9f200640 1 -- 172.21.15.152:0/3964197627 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fa0075e60 con 0x7f7fa0160b90 2024-04-20T11:33:18.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.804+0000 7f7f96000640 1 -- 172.21.15.152:0/3964197627 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 91142+0+0 (secure 0 0 0) 0x7f7f98018020 con 0x7f7fa0160b90 2024-04-20T11:33:18.805 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.805+0000 7f7f96000640 1 --2- 172.21.15.152:0/3964197627 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f7f7c06ca10 0x7f7f7c06eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:18.805 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.805+0000 7f7f96000640 1 -- 172.21.15.152:0/3964197627 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5029+0+0 (secure 0 0 0) 0x7f7f9808bc60 con 0x7f7fa0160b90 2024-04-20T11:33:18.805 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.806+0000 7f7f9de00640 1 --2- 172.21.15.152:0/3964197627 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f7f7c06ca10 0x7f7f7c06eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:18.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.807+0000 7f7f9de00640 1 --2- 172.21.15.152:0/3964197627 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f7f7c06ca10 0x7f7f7c06eed0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7f88007fd0 tx=0x7f7f88009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:18.809 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:18.810+0000 7f7f96000640 1 -- 172.21.15.152:0/3964197627 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f7f9801f020 con 0x7f7fa0160b90 2024-04-20T11:33:19.091 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:19.091+0000 7f7f9f200640 1 -- 172.21.15.152:0/3964197627 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_sanity"]} v 0) -- 0x7f7fa00aec80 con 0x7f7fa0160b90 2024-04-20T11:33:19.723 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:19.723+0000 7f7f96000640 1 -- 172.21.15.152:0/3964197627 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_sanity"]}]=0 v342) ==== 152+0+0 (secure 0 0 0) 0x7f7f9800d040 con 0x7f7fa0160b90 2024-04-20T11:33:19.726 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:19.727+0000 7f7f9f200640 1 -- 172.21.15.152:0/3964197627 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f7f7c06ca10 msgr2=0x7f7f7c06eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:19.727 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:19.727+0000 7f7f9f200640 1 --2- 172.21.15.152:0/3964197627 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f7f7c06ca10 0x7f7f7c06eed0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7f88007fd0 tx=0x7f7f88009040 comp rx=0 tx=0).stop 2024-04-20T11:33:19.727 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:19.727+0000 7f7f9f200640 1 -- 172.21.15.152:0/3964197627 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7fa0160b90 msgr2=0x7f7fa00b30c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:19.727 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:19.727+0000 7f7f9f200640 1 --2- 172.21.15.152:0/3964197627 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7fa0160b90 0x7f7fa00b30c0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f7f9800b450 tx=0x7f7f9800b910 comp rx=0 tx=0).stop 2024-04-20T11:33:19.727 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:19.727+0000 7f7f9f200640 1 -- 172.21.15.152:0/3964197627 shutdown_connections 2024-04-20T11:33:19.727 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:19.727+0000 7f7f9f200640 1 --2- 172.21.15.152:0/3964197627 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f7f7c06ca10 0x7f7f7c06eed0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:19.727 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:19.727+0000 7f7f9f200640 1 --2- 172.21.15.152:0/3964197627 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7fa0160b90 0x7f7fa00b30c0 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:19.727 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:19.727+0000 7f7f9f200640 1 --2- 172.21.15.152:0/3964197627 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7fa015d1e0 0x7f7fa0144320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:19.727 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:19.727+0000 7f7f9f200640 1 --2- 172.21.15.152:0/3964197627 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7fa0071cd0 0x7f7fa0159a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:19.727 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:19.727+0000 7f7f9f200640 1 -- 172.21.15.152:0/3964197627 >> 172.21.15.152:0/3964197627 conn(0x7f7fa0067ae0 msgr2=0x7f7fa006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:19.727 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:19.727+0000 7f7f9f200640 1 -- 172.21.15.152:0/3964197627 shutdown_connections 2024-04-20T11:33:19.727 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:19.727+0000 7f7f9f200640 1 -- 172.21.15.152:0/3964197627 wait complete. 2024-04-20T11:33:19.750 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-20T11:33:19.955 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:19.956+0000 7fdc8de00640 1 Processor -- start 2024-04-20T11:33:19.956 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:19.956+0000 7fdc8de00640 1 -- start start 2024-04-20T11:33:19.957 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:19.958+0000 7fdc8de00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fdc880cfc40 0x7fdc8815fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:19.957 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:19.958+0000 7fdc8de00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc8815f1c0 con 0x7fdc880b2880 2024-04-20T11:33:19.957 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:19.958+0000 7fdc8de00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc8806f670 con 0x7fdc8815b510 2024-04-20T11:33:19.957 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:19.958+0000 7fdc8de00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fdc8806f420 con 0x7fdc880cfc40 2024-04-20T11:33:19.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:19.958+0000 7fdc87400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fdc880b2880 0x7fdc880b4800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:55174/0 (socket says 172.21.15.152:55174) 2024-04-20T11:33:19.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:19.958+0000 7fdc87400640 1 -- 172.21.15.152:0/329193286 learned_addr learned my addr 172.21.15.152:0/329193286 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:33:19.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:19.958+0000 7fdc86000640 1 --2- 172.21.15.152:0/329193286 >> v2:172.21.15.152:3300/0 conn(0x7fdc880cfc40 0x7fdc8815fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:19.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:19.958+0000 7fdc85600640 1 -- 172.21.15.152:0/329193286 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1991871096 0 0) 0x7fdc8815f1c0 con 0x7fdc880b2880 2024-04-20T11:33:19.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:19.959+0000 7fdc86000640 1 -- 172.21.15.152:0/329193286 >> v1:172.21.15.175:6789/0 conn(0x7fdc8815b510 legacy=0x7fdc880b2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:19.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:19.959+0000 7fdc86000640 1 -- 172.21.15.152:0/329193286 >> v1:172.21.15.152:6789/0 conn(0x7fdc880b2880 legacy=0x7fdc880b4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:19.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:19.959+0000 7fdc86000640 1 -- 172.21.15.152:0/329193286 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc8815f430 con 0x7fdc880cfc40 2024-04-20T11:33:19.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:19.960+0000 7fdc86000640 1 --2- 172.21.15.152:0/329193286 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdc880cfc40 0x7fdc8815fc30 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7fdc740097a0 tx=0x7fdc7402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0f5aa7b7d43d09e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:19.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:19.960+0000 7fdc85600640 1 -- 172.21.15.152:0/329193286 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc740315a0 con 0x7fdc880cfc40 2024-04-20T11:33:19.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:19.960+0000 7fdc85600640 1 -- 172.21.15.152:0/329193286 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc74037070 con 0x7fdc880cfc40 2024-04-20T11:33:19.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:19.960+0000 7fdc85600640 1 -- 172.21.15.152:0/329193286 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc74031960 con 0x7fdc880cfc40 2024-04-20T11:33:19.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:19.961+0000 7fdc8de00640 1 -- 172.21.15.152:0/329193286 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdc880cfc40 msgr2=0x7fdc8815fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:19.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:19.961+0000 7fdc8de00640 1 --2- 172.21.15.152:0/329193286 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdc880cfc40 0x7fdc8815fc30 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7fdc740097a0 tx=0x7fdc7402b840 comp rx=0 tx=0).stop 2024-04-20T11:33:19.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:19.961+0000 7fdc8de00640 1 -- 172.21.15.152:0/329193286 shutdown_connections 2024-04-20T11:33:19.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:19.961+0000 7fdc8de00640 1 --2- 172.21.15.152:0/329193286 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdc880cfc40 0x7fdc8815fc30 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:19.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:19.961+0000 7fdc8de00640 1 -- 172.21.15.152:0/329193286 >> 172.21.15.152:0/329193286 conn(0x7fdc8806b980 msgr2=0x7fdc8806bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:19.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:19.961+0000 7fdc8de00640 1 -- 172.21.15.152:0/329193286 shutdown_connections 2024-04-20T11:33:19.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:19.961+0000 7fdc8de00640 1 -- 172.21.15.152:0/329193286 wait complete. 2024-04-20T11:33:19.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:19.963+0000 7fdc8de00640 1 Processor -- start 2024-04-20T11:33:19.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:19.963+0000 7fdc8de00640 1 -- start start 2024-04-20T11:33:19.965 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:19.964+0000 7fdc8de00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdc880b2880 0x7fdc88153d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:19.965 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:19.964+0000 7fdc8de00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc880cfc40 0x7fdc881f4c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:19.965 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:19.964+0000 7fdc8de00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdc8815b510 0x7fdc881f3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:19.965 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:19.964+0000 7fdc8de00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fdc8806f950 con 0x7fdc880b2880 2024-04-20T11:33:19.965 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:19.964+0000 7fdc8de00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdc8815b8f0 con 0x7fdc880cfc40 2024-04-20T11:33:19.965 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:19.964+0000 7fdc8de00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fdc880b3ef0 con 0x7fdc8815b510 2024-04-20T11:33:19.965 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:19.964+0000 7fdc86a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdc880b2880 0x7fdc88153d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:19.965 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:19.964+0000 7fdc86a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdc880b2880 0x7fdc88153d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:37484/0 (socket says 172.21.15.152:37484) 2024-04-20T11:33:19.965 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:19.964+0000 7fdc86a00640 1 -- 172.21.15.152:0/856501935 learned_addr learned my addr 172.21.15.152:0/856501935 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:33:19.965 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:19.964+0000 7fdc87400640 1 --2- 172.21.15.152:0/856501935 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdc8815b510 0x7fdc881f3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:19.965 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:19.964+0000 7fdc86000640 1 --2- 172.21.15.152:0/856501935 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc880cfc40 0x7fdc881f4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:19.966 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:19.964+0000 7fdc86a00640 1 -- 172.21.15.152:0/856501935 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdc8815b510 msgr2=0x7fdc881f3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:19.966 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:19.964+0000 7fdc86a00640 1 --2- 172.21.15.152:0/856501935 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdc8815b510 0x7fdc881f3dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:19.966 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:19.964+0000 7fdc86a00640 1 -- 172.21.15.152:0/856501935 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc880cfc40 msgr2=0x7fdc881f4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:19.966 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:19.964+0000 7fdc86a00640 1 --2- 172.21.15.152:0/856501935 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc880cfc40 0x7fdc881f4c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:19.966 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:19.965+0000 7fdc86a00640 1 -- 172.21.15.152:0/856501935 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc880b0020 con 0x7fdc880b2880 2024-04-20T11:33:19.966 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:19.965+0000 7fdc86a00640 1 --2- 172.21.15.152:0/856501935 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdc880b2880 0x7fdc88153d90 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7fdc7800b3f0 tx=0x7fdc7800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:19.966 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:19.965+0000 7fdc7f400640 1 -- 172.21.15.152:0/856501935 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc78015020 con 0x7fdc880b2880 2024-04-20T11:33:19.966 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:19.965+0000 7fdc7f400640 1 -- 172.21.15.152:0/856501935 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc78011830 con 0x7fdc880b2880 2024-04-20T11:33:19.966 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:19.965+0000 7fdc7f400640 1 -- 172.21.15.152:0/856501935 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc78011c20 con 0x7fdc880b2880 2024-04-20T11:33:19.966 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:19.965+0000 7fdc8de00640 1 -- 172.21.15.152:0/856501935 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc880ae830 con 0x7fdc880b2880 2024-04-20T11:33:19.966 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:19.965+0000 7fdc8de00640 1 -- 172.21.15.152:0/856501935 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc880af0d0 con 0x7fdc880b2880 2024-04-20T11:33:19.967 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:19.966+0000 7fdc8de00640 1 -- 172.21.15.152:0/856501935 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc880689f0 con 0x7fdc880b2880 2024-04-20T11:33:19.967 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:19.968+0000 7fdc7f400640 1 -- 172.21.15.152:0/856501935 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 91142+0+0 (secure 0 0 0) 0x7fdc78018020 con 0x7fdc880b2880 2024-04-20T11:33:19.968 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:19.969+0000 7fdc7f400640 1 --2- 172.21.15.152:0/856501935 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fdc6806ca10 0x7fdc6806eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:19.968 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:19.969+0000 7fdc7f400640 1 -- 172.21.15.152:0/856501935 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5029+0+0 (secure 0 0 0) 0x7fdc7808b670 con 0x7fdc880b2880 2024-04-20T11:33:19.968 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:19.969+0000 7fdc86000640 1 --2- 172.21.15.152:0/856501935 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fdc6806ca10 0x7fdc6806eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:19.970 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:19.970+0000 7fdc86000640 1 --2- 172.21.15.152:0/856501935 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fdc6806ca10 0x7fdc6806eed0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fdc74036000 tx=0x7fdc74008030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:19.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:19.974+0000 7fdc7f400640 1 -- 172.21.15.152:0/856501935 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fdc7801f030 con 0x7fdc880b2880 2024-04-20T11:33:20.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:20.167+0000 7fdc8de00640 1 -- 172.21.15.152:0/856501935 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fdc8806ecf0 con 0x7fdc880b2880 2024-04-20T11:33:20.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:20.174+0000 7fdc7f400640 1 -- 172.21.15.152:0/856501935 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v20)=0 v20) ==== 122+0+0 (secure 0 0 0) 0x7fdc780172c0 con 0x7fdc880b2880 2024-04-20T11:33:20.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:20.177+0000 7fdc8de00640 1 -- 172.21.15.152:0/856501935 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fdc6806ca10 msgr2=0x7fdc6806eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:20.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:20.177+0000 7fdc8de00640 1 --2- 172.21.15.152:0/856501935 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fdc6806ca10 0x7fdc6806eed0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fdc74036000 tx=0x7fdc74008030 comp rx=0 tx=0).stop 2024-04-20T11:33:20.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:20.177+0000 7fdc8de00640 1 -- 172.21.15.152:0/856501935 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdc880b2880 msgr2=0x7fdc88153d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:20.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:20.177+0000 7fdc8de00640 1 --2- 172.21.15.152:0/856501935 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdc880b2880 0x7fdc88153d90 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7fdc7800b3f0 tx=0x7fdc7800b8b0 comp rx=0 tx=0).stop 2024-04-20T11:33:20.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:20.177+0000 7fdc8de00640 1 -- 172.21.15.152:0/856501935 shutdown_connections 2024-04-20T11:33:20.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:20.177+0000 7fdc8de00640 1 --2- 172.21.15.152:0/856501935 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fdc6806ca10 0x7fdc6806eed0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:20.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:20.177+0000 7fdc8de00640 1 --2- 172.21.15.152:0/856501935 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdc8815b510 0x7fdc881f3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:20.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:20.177+0000 7fdc8de00640 1 --2- 172.21.15.152:0/856501935 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc880cfc40 0x7fdc881f4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:20.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:20.177+0000 7fdc8de00640 1 --2- 172.21.15.152:0/856501935 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdc880b2880 0x7fdc88153d90 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:20.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:20.177+0000 7fdc8de00640 1 -- 172.21.15.152:0/856501935 >> 172.21.15.152:0/856501935 conn(0x7fdc8806b980 msgr2=0x7fdc880cf460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:20.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:20.178+0000 7fdc8de00640 1 -- 172.21.15.152:0/856501935 shutdown_connections 2024-04-20T11:33:20.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:20.178+0000 7fdc8de00640 1 -- 172.21.15.152:0/856501935 wait complete. 2024-04-20T11:33:20.195 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-20T11:33:20.195 DEBUG:teuthology.orchestra.run.smithi152:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-20T11:33:20.292 DEBUG:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-20T11:33:20.411 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-20T11:33:20.411 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-20T11:33:20.411 DEBUG:teuthology.orchestra.run.smithi152:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:33:20.467 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-20T11:33:20.467 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T11:33:20.467 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:33:20.527 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-20T11:33:20.527 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-20T11:33:20.527 DEBUG:teuthology.orchestra.run.smithi152:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:33:20.585 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-20T11:33:20.585 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T11:33:20.585 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:33:20.643 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-20T11:33:20.643 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-20T11:33:20.644 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:33:20.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:20.851+0000 7f2740c00640 1 Processor -- start 2024-04-20T11:33:20.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:20.851+0000 7f2740c00640 1 -- start start 2024-04-20T11:33:20.852 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:20.852+0000 7f2740c00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f273c0723e0 0x7f273c15d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:20.852 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:20.852+0000 7f2740c00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f273c0ce8c0 con 0x7f273c160b90 2024-04-20T11:33:20.852 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:20.852+0000 7f2740c00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f273c1600b0 con 0x7f273c15d1e0 2024-04-20T11:33:20.852 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:20.852+0000 7f2740c00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f273c160390 con 0x7f273c0723e0 2024-04-20T11:33:20.852 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:20.853+0000 7f2733e00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f273c160b90 0x7f273c071cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:55180/0 (socket says 172.21.15.152:55180) 2024-04-20T11:33:20.852 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:20.853+0000 7f2733e00640 1 -- 172.21.15.152:0/1589499729 learned_addr learned my addr 172.21.15.152:0/1589499729 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:33:20.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:20.853+0000 7f273be00640 1 --2- 172.21.15.152:0/1589499729 >> v2:172.21.15.175:3300/0 conn(0x7f273c0723e0 0x7f273c15d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:20.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:20.853+0000 7f273aa00640 1 -- 172.21.15.152:0/1589499729 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4070704442 0 0) 0x7f273c0ce8c0 con 0x7f273c160b90 2024-04-20T11:33:20.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:20.853+0000 7f273aa00640 1 -- 172.21.15.152:0/1589499729 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2720003520 con 0x7f273c160b90 2024-04-20T11:33:20.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:20.853+0000 7f273aa00640 1 -- 172.21.15.152:0/1589499729 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 980462073 0 0) 0x7f2720003520 con 0x7f273c160b90 2024-04-20T11:33:20.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:20.853+0000 7f273aa00640 1 -- 172.21.15.152:0/1589499729 >> v2:172.21.15.175:3300/0 conn(0x7f273c0723e0 msgr2=0x7f273c15d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:20.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:20.853+0000 7f273aa00640 1 --2- 172.21.15.152:0/1589499729 >> v2:172.21.15.175:3300/0 conn(0x7f273c0723e0 0x7f273c15d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:20.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:20.853+0000 7f273aa00640 1 -- 172.21.15.152:0/1589499729 >> v1:172.21.15.175:6789/0 conn(0x7f273c15d1e0 legacy=0x7f273c0cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:20.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:20.853+0000 7f273aa00640 1 -- 172.21.15.152:0/1589499729 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f273c160670 con 0x7f273c160b90 2024-04-20T11:33:20.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:20.853+0000 7f273aa00640 1 -- 172.21.15.152:0/1589499729 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f2724001670 con 0x7f273c160b90 2024-04-20T11:33:20.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:20.853+0000 7f273aa00640 1 -- 172.21.15.152:0/1589499729 >> v1:172.21.15.152:6789/0 conn(0x7f273c160b90 legacy=0x7f273c071cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:20.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:20.854+0000 7f273aa00640 1 --2- 172.21.15.152:0/1589499729 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27200039b0 0x7f2720003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:20.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:20.854+0000 7f273aa00640 1 --2- 172.21.15.152:0/1589499729 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2720004800 0x7f2720004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:20.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:20.854+0000 7f273aa00640 1 --2- 172.21.15.152:0/1589499729 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f27200051c0 0x7f272000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:20.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:20.854+0000 7f273aa00640 1 -- 172.21.15.152:0/1589499729 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f273c160390 con 0x7f2720004800 2024-04-20T11:33:20.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:20.854+0000 7f273aa00640 1 -- 172.21.15.152:0/1589499729 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f272000c0f0 con 0x7f27200039b0 2024-04-20T11:33:20.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:20.854+0000 7f273aa00640 1 -- 172.21.15.152:0/1589499729 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f272000c2f0 con 0x7f27200051c0 2024-04-20T11:33:20.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:20.854+0000 7f2733e00640 1 --2- 172.21.15.152:0/1589499729 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2720004800 0x7f2720004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:20.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:20.854+0000 7f2733e00640 1 -- 172.21.15.152:0/1589499729 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f27200051c0 msgr2=0x7f272000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:20.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:20.854+0000 7f2733e00640 1 --2- 172.21.15.152:0/1589499729 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f27200051c0 0x7f272000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:20.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:20.854+0000 7f2733e00640 1 -- 172.21.15.152:0/1589499729 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27200039b0 msgr2=0x7f2720003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:20.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:20.854+0000 7f2733e00640 1 --2- 172.21.15.152:0/1589499729 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27200039b0 0x7f2720003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:20.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:20.854+0000 7f2733e00640 1 -- 172.21.15.152:0/1589499729 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2720003520 con 0x7f2720004800 2024-04-20T11:33:20.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:20.854+0000 7f273be00640 1 -- 172.21.15.152:0/1589499729 reap_dead start 2024-04-20T11:33:20.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:20.855+0000 7f2733e00640 1 --2- 172.21.15.152:0/1589499729 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2720004800 0x7f2720004c80 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f272400c210 tx=0x7f2720005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3add2d5a32252b57 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:20.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:20.856+0000 7f273aa00640 1 -- 172.21.15.152:0/1589499729 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2724007d30 con 0x7f2720004800 2024-04-20T11:33:20.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:20.856+0000 7f273aa00640 1 -- 172.21.15.152:0/1589499729 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2724030070 con 0x7f2720004800 2024-04-20T11:33:20.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:20.856+0000 7f273aa00640 1 -- 172.21.15.152:0/1589499729 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2724005d70 con 0x7f2720004800 2024-04-20T11:33:20.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:20.856+0000 7f2740c00640 1 -- 172.21.15.152:0/1589499729 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2720004800 msgr2=0x7f2720004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:20.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:20.856+0000 7f2740c00640 1 --2- 172.21.15.152:0/1589499729 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2720004800 0x7f2720004c80 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f272400c210 tx=0x7f2720005830 comp rx=0 tx=0).stop 2024-04-20T11:33:20.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:20.856+0000 7f2740c00640 1 -- 172.21.15.152:0/1589499729 shutdown_connections 2024-04-20T11:33:20.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:20.856+0000 7f2740c00640 1 --2- 172.21.15.152:0/1589499729 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2720004800 0x7f2720004c80 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:20.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:20.856+0000 7f2740c00640 1 -- 172.21.15.152:0/1589499729 >> 172.21.15.152:0/1589499729 conn(0x7f273c067ae0 msgr2=0x7f273c06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:20.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:20.856+0000 7f2740c00640 1 -- 172.21.15.152:0/1589499729 shutdown_connections 2024-04-20T11:33:20.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:20.856+0000 7f2740c00640 1 -- 172.21.15.152:0/1589499729 wait complete. 2024-04-20T11:33:20.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:20.858+0000 7f2740c00640 1 Processor -- start 2024-04-20T11:33:20.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:20.858+0000 7f2740c00640 1 -- start start 2024-04-20T11:33:20.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:20.859+0000 7f2740c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2720004800 0x7f273c157050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:20.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:20.859+0000 7f2740c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f273c13ea80 0x7f273c0b2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:20.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:20.859+0000 7f2740c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f273c0b25b0 0x7f273c151e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:20.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:20.859+0000 7f2740c00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f273c15d740 con 0x7f2720004800 2024-04-20T11:33:20.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:20.859+0000 7f2740c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f273c15d5c0 con 0x7f273c0b25b0 2024-04-20T11:33:20.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:20.859+0000 7f2740c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f273c15d8c0 con 0x7f273c13ea80 2024-04-20T11:33:20.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:20.859+0000 7f273b400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2720004800 0x7f273c157050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:20.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:20.859+0000 7f273b400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2720004800 0x7f273c157050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:37504/0 (socket says 172.21.15.152:37504) 2024-04-20T11:33:20.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:20.859+0000 7f273b400640 1 -- 172.21.15.152:0/604491937 learned_addr learned my addr 172.21.15.152:0/604491937 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:33:20.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:20.860+0000 7f273b400640 1 -- 172.21.15.152:0/604491937 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f273c13ea80 msgr2=0x7f273c0b2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:20.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:20.860+0000 7f273b400640 1 --2- 172.21.15.152:0/604491937 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f273c13ea80 0x7f273c0b2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:20.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:20.860+0000 7f273b400640 1 -- 172.21.15.152:0/604491937 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f273c0b25b0 msgr2=0x7f273c151e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:20.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:20.860+0000 7f273b400640 1 --2- 172.21.15.152:0/604491937 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f273c0b25b0 0x7f273c151e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:20.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:20.860+0000 7f273b400640 1 -- 172.21.15.152:0/604491937 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f273c153870 con 0x7f2720004800 2024-04-20T11:33:20.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:20.860+0000 7f273b400640 1 --2- 172.21.15.152:0/604491937 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2720004800 0x7f273c157050 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f272c00c3f0 tx=0x7f272c00c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:20.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:20.860+0000 7f2733400640 1 -- 172.21.15.152:0/604491937 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f272c011790 con 0x7f2720004800 2024-04-20T11:33:20.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:20.860+0000 7f2733400640 1 -- 172.21.15.152:0/604491937 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f272c017070 con 0x7f2720004800 2024-04-20T11:33:20.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:20.860+0000 7f2733400640 1 -- 172.21.15.152:0/604491937 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f272c011b70 con 0x7f2720004800 2024-04-20T11:33:20.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:20.860+0000 7f2740c00640 1 -- 172.21.15.152:0/604491937 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f273c153b50 con 0x7f2720004800 2024-04-20T11:33:20.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:20.861+0000 7f2740c00640 1 -- 172.21.15.152:0/604491937 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f273c159780 con 0x7f2720004800 2024-04-20T11:33:20.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:20.862+0000 7f2740c00640 1 -- 172.21.15.152:0/604491937 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f273c075e60 con 0x7f2720004800 2024-04-20T11:33:20.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:20.863+0000 7f2733400640 1 -- 172.21.15.152:0/604491937 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 91142+0+0 (secure 0 0 0) 0x7f272c011d10 con 0x7f2720004800 2024-04-20T11:33:20.863 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:20.864+0000 7f2733400640 1 --2- 172.21.15.152:0/604491937 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f271806c980 0x7f271806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:20.863 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:20.864+0000 7f2733400640 1 -- 172.21.15.152:0/604491937 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5029+0+0 (secure 0 0 0) 0x7f272c08b7c0 con 0x7f2720004800 2024-04-20T11:33:20.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:20.864+0000 7f2733e00640 1 --2- 172.21.15.152:0/604491937 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f271806c980 0x7f271806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:20.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:20.866+0000 7f2733e00640 1 --2- 172.21.15.152:0/604491937 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f271806c980 0x7f271806ee40 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2724034000 tx=0x7f2724007000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:20.867 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:20.868+0000 7f2733400640 1 -- 172.21.15.152:0/604491937 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f272c059e40 con 0x7f2720004800 2024-04-20T11:33:21.090 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:21.090+0000 7f2740c00640 1 -- 172.21.15.152:0/604491937 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f273c072a10 con 0x7f2720004800 2024-04-20T11:33:21.091 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:21.091+0000 7f2733400640 1 -- 172.21.15.152:0/604491937 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) ==== 94+0+3878 (secure 0 0 0) 0x7f272c059740 con 0x7f2720004800 2024-04-20T11:33:21.091 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:33:21.091 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":60,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5465,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/3844264695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":3844264695},{"type":"v1","addr":"172.21.15.152:6835","nonce":3844264695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":5468,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/706098839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":706098839},{"type":"v1","addr":"172.21.15.175:6833","nonce":706098839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":6016,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/3381264900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3381264900},{"type":"v1","addr":"172.21.15.175:6835","nonce":3381264900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58},{"gid":7722,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/3530200519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":3530200519},{"type":"v1","addr":"172.21.15.152:6833","nonce":3530200519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60}],"filesystems":[]} 2024-04-20T11:33:21.091 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 60 2024-04-20T11:33:21.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:21.094+0000 7f2740c00640 1 -- 172.21.15.152:0/604491937 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f271806c980 msgr2=0x7f271806ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:21.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:21.094+0000 7f2740c00640 1 --2- 172.21.15.152:0/604491937 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f271806c980 0x7f271806ee40 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2724034000 tx=0x7f2724007000 comp rx=0 tx=0).stop 2024-04-20T11:33:21.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:21.094+0000 7f2740c00640 1 -- 172.21.15.152:0/604491937 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2720004800 msgr2=0x7f273c157050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:21.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:21.094+0000 7f2740c00640 1 --2- 172.21.15.152:0/604491937 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2720004800 0x7f273c157050 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f272c00c3f0 tx=0x7f272c00c7f0 comp rx=0 tx=0).stop 2024-04-20T11:33:21.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:21.094+0000 7f2740c00640 1 -- 172.21.15.152:0/604491937 shutdown_connections 2024-04-20T11:33:21.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:21.094+0000 7f2740c00640 1 --2- 172.21.15.152:0/604491937 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f271806c980 0x7f271806ee40 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:21.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:21.094+0000 7f2740c00640 1 --2- 172.21.15.152:0/604491937 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f273c0b25b0 0x7f273c151e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:21.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:21.094+0000 7f2740c00640 1 --2- 172.21.15.152:0/604491937 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f273c13ea80 0x7f273c0b2070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:21.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:21.094+0000 7f2740c00640 1 --2- 172.21.15.152:0/604491937 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2720004800 0x7f273c157050 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:21.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:21.094+0000 7f2740c00640 1 -- 172.21.15.152:0/604491937 >> 172.21.15.152:0/604491937 conn(0x7f273c067ae0 msgr2=0x7f273c06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:21.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:21.094+0000 7f2740c00640 1 -- 172.21.15.152:0/604491937 shutdown_connections 2024-04-20T11:33:21.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:21.094+0000 7f2740c00640 1 -- 172.21.15.152:0/604491937 wait complete. 2024-04-20T11:33:21.112 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-20T11:33:21.112 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-20T11:33:21.112 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-20T11:33:21.113 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T11:33:21.113 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-20T11:33:21.113 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-20T11:33:21.113 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-20T11:33:21.113 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T11:33:21.113 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-20T11:33:21.113 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-20T11:33:21.114 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-20T11:33:21.114 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T11:33:21.114 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-20T11:33:21.114 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-20T11:33:21.114 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-20T11:33:21.114 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T11:33:21.130 INFO:tasks.ceph.mds.c:Stopped 2024-04-20T11:33:21.130 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-20T11:33:21.132 INFO:tasks.ceph.mds.a:Stopped 2024-04-20T11:33:21.132 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-20T11:33:21.137 INFO:tasks.ceph.mds.d:Stopped 2024-04-20T11:33:21.137 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-20T11:33:21.138 INFO:tasks.ceph.mds.b:Stopped 2024-04-20T11:33:21.139 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-20T11:33:21.179 INFO:tasks.ceph.mds.c:Started 2024-04-20T11:33:21.179 DEBUG:teuthology.parallel:result is None 2024-04-20T11:33:21.185 INFO:tasks.ceph.mds.a:Started 2024-04-20T11:33:21.186 DEBUG:teuthology.parallel:result is None 2024-04-20T11:33:21.186 INFO:tasks.ceph.mds.d:Started 2024-04-20T11:33:21.187 DEBUG:teuthology.parallel:result is None 2024-04-20T11:33:21.193 INFO:tasks.ceph.mds.b:Started 2024-04-20T11:33:21.193 DEBUG:teuthology.parallel:result is None 2024-04-20T11:33:21.193 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-20T11:33:21.321 INFO:tasks.ceph.mds.c.smithi152.stdout:starting mds.c at 2024-04-20T11:33:21.322 INFO:tasks.ceph.mds.a.smithi152.stdout:starting mds.a at 2024-04-20T11:33:21.339 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2024-04-20T11:33:21.342 INFO:tasks.ceph.mds.d.smithi175.stdout:starting mds.d at 2024-04-20T11:33:21.345 INFO:tasks.ceph.mds.b.smithi175.stdout:starting mds.b at 2024-04-20T11:33:21.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:21.358+0000 7f6e08000640 1 Processor -- start 2024-04-20T11:33:21.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:21.358+0000 7f6e08000640 1 -- start start 2024-04-20T11:33:21.359 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:21.359+0000 7f6e08000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e00162b40 0x7f6e000b3470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:21.359 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:21.359+0000 7f6e08000640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f6e000cfff0 0x7f6e000d0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:21.359 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:21.359+0000 7f6e08000640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e000b3a50 con 0x7f6e0006f810 2024-04-20T11:33:21.359 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:21.359+0000 7f6e08000640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f6e000d09a0 con 0x7f6e000cfff0 2024-04-20T11:33:21.359 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:21.359+0000 7f6e08000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6e0006fc70 con 0x7f6e00162b40 2024-04-20T11:33:21.359 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:21.359+0000 7f6e04c00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f6e0006f810 0x7f6e0006ede0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:55196/0 (socket says 172.21.15.152:55196) 2024-04-20T11:33:21.359 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:21.359+0000 7f6e04c00640 1 -- 172.21.15.152:0/2780952693 learned_addr learned my addr 172.21.15.152:0/2780952693 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:33:21.359 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:21.359+0000 7f6e05600640 1 --2- 172.21.15.152:0/2780952693 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e00162b40 0x7f6e000b3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:21.359 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:21.359+0000 7f6e06000640 1 --2- 172.21.15.152:0/2780952693 >> v2:172.21.15.175:3300/0 conn(0x7f6e000cfff0 0x7f6e000d0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:21.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:21.360+0000 7f6dffe00640 1 -- 172.21.15.152:0/2780952693 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1523559937 0 0) 0x7f6e000b3a50 con 0x7f6e0006f810 2024-04-20T11:33:21.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:21.360+0000 7f6dffe00640 1 -- 172.21.15.152:0/2780952693 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6de8003520 con 0x7f6e0006f810 2024-04-20T11:33:21.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:21.360+0000 7f6e05600640 1 -- 172.21.15.152:0/2780952693 >> v2:172.21.15.175:3300/0 conn(0x7f6e000cfff0 msgr2=0x7f6e000d0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:21.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:21.360+0000 7f6e05600640 1 --2- 172.21.15.152:0/2780952693 >> v2:172.21.15.175:3300/0 conn(0x7f6e000cfff0 0x7f6e000d0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:21.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:21.360+0000 7f6e05600640 1 -- 172.21.15.152:0/2780952693 >> v1:172.21.15.152:6789/0 conn(0x7f6e0006f810 legacy=0x7f6e0006ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:21.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:21.360+0000 7f6e05600640 1 -- 172.21.15.152:0/2780952693 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e0006e300 con 0x7f6e00162b40 2024-04-20T11:33:21.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:21.360+0000 7f6e06000640 1 --2- 172.21.15.152:0/2780952693 >> v2:172.21.15.175:3300/0 conn(0x7f6e000cfff0 0x7f6e000d0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:33:21.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:21.361+0000 7f6e05600640 1 --2- 172.21.15.152:0/2780952693 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e00162b40 0x7f6e000b3470 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6df40097a0 tx=0x7f6df402b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b33c1a5d10dffaec server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:21.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:21.361+0000 7f6dffe00640 1 -- 172.21.15.152:0/2780952693 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df40315a0 con 0x7f6e00162b40 2024-04-20T11:33:21.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:21.361+0000 7f6dffe00640 1 -- 172.21.15.152:0/2780952693 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6df4037070 con 0x7f6e00162b40 2024-04-20T11:33:21.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:21.361+0000 7f6dffe00640 1 -- 172.21.15.152:0/2780952693 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df4031960 con 0x7f6e00162b40 2024-04-20T11:33:21.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:21.362+0000 7f6e08000640 1 -- 172.21.15.152:0/2780952693 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e00162b40 msgr2=0x7f6e000b3470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:21.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:21.362+0000 7f6e08000640 1 --2- 172.21.15.152:0/2780952693 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e00162b40 0x7f6e000b3470 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6df40097a0 tx=0x7f6df402b840 comp rx=0 tx=0).stop 2024-04-20T11:33:21.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:21.362+0000 7f6e08000640 1 -- 172.21.15.152:0/2780952693 shutdown_connections 2024-04-20T11:33:21.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:21.362+0000 7f6e08000640 1 --2- 172.21.15.152:0/2780952693 >> v2:172.21.15.175:3300/0 conn(0x7f6e000cfff0 0x7f6e000d0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:21.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:21.362+0000 7f6e08000640 1 --2- 172.21.15.152:0/2780952693 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e00162b40 0x7f6e000b3470 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:21.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:21.362+0000 7f6e08000640 1 -- 172.21.15.152:0/2780952693 >> 172.21.15.152:0/2780952693 conn(0x7f6e00067cf0 msgr2=0x7f6e0006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:21.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:21.362+0000 7f6e08000640 1 -- 172.21.15.152:0/2780952693 shutdown_connections 2024-04-20T11:33:21.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:21.362+0000 7f6e08000640 1 -- 172.21.15.152:0/2780952693 wait complete. 2024-04-20T11:33:21.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:21.364+0000 7f6e08000640 1 Processor -- start 2024-04-20T11:33:21.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:21.364+0000 7f6e08000640 1 -- start start 2024-04-20T11:33:21.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:21.365+0000 7f6e08000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e0006f810 0x7f6e0014a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:21.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:21.365+0000 7f6e08000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e000cfff0 0x7f6e0015b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:21.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:21.365+0000 7f6e08000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6e00162b40 0x7f6e0015c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:21.366 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:21.365+0000 7f6e08000640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f6e0006e950 con 0x7f6e00162b40 2024-04-20T11:33:21.366 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:21.365+0000 7f6e08000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6e0006e0b0 con 0x7f6e000cfff0 2024-04-20T11:33:21.366 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:21.365+0000 7f6e08000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6e00162f20 con 0x7f6e0006f810 2024-04-20T11:33:21.366 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:21.365+0000 7f6e06000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6e00162b40 0x7f6e0015c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:21.366 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:21.365+0000 7f6e06000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6e00162b40 0x7f6e0015c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:37562/0 (socket says 172.21.15.152:37562) 2024-04-20T11:33:21.366 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:21.365+0000 7f6e06000640 1 -- 172.21.15.152:0/197997316 learned_addr learned my addr 172.21.15.152:0/197997316 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:33:21.366 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:21.365+0000 7f6e05600640 1 --2- 172.21.15.152:0/197997316 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e0006f810 0x7f6e0014a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:21.366 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:21.365+0000 7f6e04c00640 1 --2- 172.21.15.152:0/197997316 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e000cfff0 0x7f6e0015b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:21.366 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:21.365+0000 7f6e06000640 1 -- 172.21.15.152:0/197997316 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e0006f810 msgr2=0x7f6e0014a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:21.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:21.365+0000 7f6e06000640 1 --2- 172.21.15.152:0/197997316 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e0006f810 0x7f6e0014a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:21.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:21.365+0000 7f6e06000640 1 -- 172.21.15.152:0/197997316 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e000cfff0 msgr2=0x7f6e0015b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:21.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:21.365+0000 7f6e06000640 1 --2- 172.21.15.152:0/197997316 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e000cfff0 0x7f6e0015b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:21.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:21.365+0000 7f6e06000640 1 -- 172.21.15.152:0/197997316 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e0015bea0 con 0x7f6e00162b40 2024-04-20T11:33:21.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:21.366+0000 7f6e06000640 1 --2- 172.21.15.152:0/197997316 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6e00162b40 0x7f6e0015c6c0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f6df800efc0 tx=0x7f6df8009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:21.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:21.366+0000 7f6dfd600640 1 -- 172.21.15.152:0/197997316 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df8018030 con 0x7f6e00162b40 2024-04-20T11:33:21.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:21.366+0000 7f6dfd600640 1 -- 172.21.15.152:0/197997316 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6df800cde0 con 0x7f6e00162b40 2024-04-20T11:33:21.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:21.366+0000 7f6dfd600640 1 -- 172.21.15.152:0/197997316 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df8003bb0 con 0x7f6e00162b40 2024-04-20T11:33:21.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:21.366+0000 7f6e08000640 1 -- 172.21.15.152:0/197997316 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e001633a0 con 0x7f6e00162b40 2024-04-20T11:33:21.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:21.367+0000 7f6e08000640 1 -- 172.21.15.152:0/197997316 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e00165b60 con 0x7f6e00162b40 2024-04-20T11:33:21.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:21.368+0000 7f6e08000640 1 -- 172.21.15.152:0/197997316 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e000689f0 con 0x7f6e00162b40 2024-04-20T11:33:21.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:21.369+0000 7f6dfd600640 1 -- 172.21.15.152:0/197997316 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 91142+0+0 (secure 0 0 0) 0x7f6df801b070 con 0x7f6e00162b40 2024-04-20T11:33:21.369 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:21.369+0000 7f6dfd600640 1 --2- 172.21.15.152:0/197997316 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f6de006ca10 0x7f6de006eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:21.369 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:21.369+0000 7f6dfd600640 1 -- 172.21.15.152:0/197997316 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5029+0+0 (secure 0 0 0) 0x7f6df808c4c0 con 0x7f6e00162b40 2024-04-20T11:33:21.369 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:21.370+0000 7f6e05600640 1 --2- 172.21.15.152:0/197997316 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f6de006ca10 0x7f6de006eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:21.370 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:21.370+0000 7f6e05600640 1 --2- 172.21.15.152:0/197997316 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f6de006ca10 0x7f6de006eed0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6df4036000 tx=0x7f6df403d040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:21.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:21.374+0000 7f6dfd600640 1 -- 172.21.15.152:0/197997316 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f6df8010040 con 0x7f6e00162b40 2024-04-20T11:33:21.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:21.571+0000 7f6e08000640 1 -- 172.21.15.152:0/197997316 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f6e00069100 con 0x7f6e00162b40 2024-04-20T11:33:22.736 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:33:22.744 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:33:22.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:22.762+0000 7f6dfd600640 1 -- 172.21.15.152:0/197997316 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v99) ==== 97+0+0 (secure 0 0 0) 0x7f6df8012340 con 0x7f6e00162b40 2024-04-20T11:33:22.762 INFO:teuthology.orchestra.run.smithi152.stderr: removed all blocklist entries 2024-04-20T11:33:22.766 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:22.766+0000 7f6e08000640 1 -- 172.21.15.152:0/197997316 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f6de006ca10 msgr2=0x7f6de006eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:22.767 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:22.766+0000 7f6e08000640 1 --2- 172.21.15.152:0/197997316 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f6de006ca10 0x7f6de006eed0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6df4036000 tx=0x7f6df403d040 comp rx=0 tx=0).stop 2024-04-20T11:33:22.767 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:22.766+0000 7f6e08000640 1 -- 172.21.15.152:0/197997316 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6e00162b40 msgr2=0x7f6e0015c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:22.767 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:22.766+0000 7f6e08000640 1 --2- 172.21.15.152:0/197997316 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6e00162b40 0x7f6e0015c6c0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f6df800efc0 tx=0x7f6df8009f90 comp rx=0 tx=0).stop 2024-04-20T11:33:22.767 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:22.767+0000 7f6e08000640 1 -- 172.21.15.152:0/197997316 shutdown_connections 2024-04-20T11:33:22.767 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:22.767+0000 7f6e08000640 1 --2- 172.21.15.152:0/197997316 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f6de006ca10 0x7f6de006eed0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:22.767 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:22.767+0000 7f6e08000640 1 --2- 172.21.15.152:0/197997316 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6e00162b40 0x7f6e0015c6c0 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:22.767 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:22.767+0000 7f6e08000640 1 --2- 172.21.15.152:0/197997316 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e000cfff0 0x7f6e0015b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:22.767 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:22.767+0000 7f6e08000640 1 --2- 172.21.15.152:0/197997316 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e0006f810 0x7f6e0014a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:22.767 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:22.767+0000 7f6e08000640 1 -- 172.21.15.152:0/197997316 >> 172.21.15.152:0/197997316 conn(0x7f6e00067cf0 msgr2=0x7f6e000722e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:22.767 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:22.767+0000 7f6e08000640 1 -- 172.21.15.152:0/197997316 shutdown_connections 2024-04-20T11:33:22.767 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:22.767+0000 7f6e08000640 1 -- 172.21.15.152:0/197997316 wait complete. 2024-04-20T11:33:22.789 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-20T11:33:23.012 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.013+0000 7f52b7200640 1 Processor -- start 2024-04-20T11:33:23.013 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.013+0000 7f52b7200640 1 -- start start 2024-04-20T11:33:23.014 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.014+0000 7f52b7200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f52b815fca0 0x7f52b80cf740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:23.014 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.014+0000 7f52b7200640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52b80cfe20 con 0x7f52b806f510 2024-04-20T11:33:23.014 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.014+0000 7f52b7200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52b815f1e0 con 0x7f52b8144d40 2024-04-20T11:33:23.014 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.014+0000 7f52b7200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f52b815f4c0 con 0x7f52b815fca0 2024-04-20T11:33:23.014 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.015+0000 7f52ad400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f52b806f510 0x7f52b80cf030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:55206/0 (socket says 172.21.15.152:55206) 2024-04-20T11:33:23.014 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.015+0000 7f52ad400640 1 -- 172.21.15.152:0/2116346508 learned_addr learned my addr 172.21.15.152:0/2116346508 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:33:23.017 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.015+0000 7f52b6800640 1 --2- 172.21.15.152:0/2116346508 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f52b815fca0 0x7f52b80cf740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:23.017 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.015+0000 7f52b5400640 1 -- 172.21.15.152:0/2116346508 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1334329259 0 0) 0x7f52b80cfe20 con 0x7f52b806f510 2024-04-20T11:33:23.017 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.015+0000 7f52b5400640 1 -- 172.21.15.152:0/2116346508 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f529c003520 con 0x7f52b806f510 2024-04-20T11:33:23.017 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.016+0000 7f52b5400640 1 -- 172.21.15.152:0/2116346508 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3583211044 0 0) 0x7f529c003520 con 0x7f52b806f510 2024-04-20T11:33:23.017 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.016+0000 7f52b5400640 1 -- 172.21.15.152:0/2116346508 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f52b815fca0 msgr2=0x7f52b80cf740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:23.017 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.016+0000 7f52b5400640 1 --2- 172.21.15.152:0/2116346508 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f52b815fca0 0x7f52b80cf740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:23.017 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.016+0000 7f52b5400640 1 -- 172.21.15.152:0/2116346508 >> v1:172.21.15.175:6789/0 conn(0x7f52b8144d40 legacy=0x7f52b806ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:23.017 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.016+0000 7f52b5400640 1 -- 172.21.15.152:0/2116346508 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52b8145120 con 0x7f52b806f510 2024-04-20T11:33:23.017 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.016+0000 7f52b5400640 1 -- 172.21.15.152:0/2116346508 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f52a0001a10 con 0x7f52b806f510 2024-04-20T11:33:23.018 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.016+0000 7f52b5400640 1 -- 172.21.15.152:0/2116346508 >> v1:172.21.15.152:6789/0 conn(0x7f52b806f510 legacy=0x7f52b80cf030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:23.018 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.016+0000 7f52b5400640 1 --2- 172.21.15.152:0/2116346508 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f52b815fca0 0x7f529c0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:23.018 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.016+0000 7f52b5400640 1 --2- 172.21.15.152:0/2116346508 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f529c003f60 0x7f529c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:23.018 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.016+0000 7f52b5400640 1 --2- 172.21.15.152:0/2116346508 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f529c004d40 0x7f529c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:23.018 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.016+0000 7f52b5400640 1 -- 172.21.15.152:0/2116346508 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f52b815f4c0 con 0x7f529c004d40 2024-04-20T11:33:23.018 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.016+0000 7f52b5400640 1 -- 172.21.15.152:0/2116346508 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f529c00bb50 con 0x7f529c003f60 2024-04-20T11:33:23.018 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.016+0000 7f52b5400640 1 -- 172.21.15.152:0/2116346508 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f529c00df60 con 0x7f52b815fca0 2024-04-20T11:33:23.018 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.016+0000 7f52b6800640 1 --2- 172.21.15.152:0/2116346508 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f529c004d40 0x7f529c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:23.018 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.016+0000 7f52ad400640 1 --2- 172.21.15.152:0/2116346508 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f529c003f60 0x7f529c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:23.018 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.017+0000 7f52b5e00640 1 --2- 172.21.15.152:0/2116346508 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f52b815fca0 0x7f529c0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:23.018 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.017+0000 7f52b6800640 1 -- 172.21.15.152:0/2116346508 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f52b815fca0 msgr2=0x7f529c0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:23.018 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.017+0000 7f52b6800640 1 --2- 172.21.15.152:0/2116346508 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f52b815fca0 0x7f529c0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:23.018 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.017+0000 7f52b6800640 1 -- 172.21.15.152:0/2116346508 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f529c003f60 msgr2=0x7f529c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:23.019 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.017+0000 7f52b6800640 1 --2- 172.21.15.152:0/2116346508 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f529c003f60 0x7f529c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:23.019 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.017+0000 7f52b6800640 1 -- 172.21.15.152:0/2116346508 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f529c003520 con 0x7f529c004d40 2024-04-20T11:33:23.019 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.017+0000 7f52b5e00640 1 --2- 172.21.15.152:0/2116346508 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f52b815fca0 0x7f529c0039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:33:23.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.018+0000 7f52b6800640 1 --2- 172.21.15.152:0/2116346508 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f529c004d40 0x7f529c0051c0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f52b0009ac0 tx=0x7f529c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dff89b68f5671d94 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:23.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.018+0000 7f52b5400640 1 -- 172.21.15.152:0/2116346508 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b0039070 con 0x7f529c004d40 2024-04-20T11:33:23.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.018+0000 7f52b5400640 1 -- 172.21.15.152:0/2116346508 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52b002c6b0 con 0x7f529c004d40 2024-04-20T11:33:23.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.018+0000 7f52b5400640 1 -- 172.21.15.152:0/2116346508 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b0009690 con 0x7f529c004d40 2024-04-20T11:33:23.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.019+0000 7f52b7200640 1 -- 172.21.15.152:0/2116346508 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f529c004d40 msgr2=0x7f529c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:23.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.019+0000 7f52b7200640 1 --2- 172.21.15.152:0/2116346508 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f529c004d40 0x7f529c0051c0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f52b0009ac0 tx=0x7f529c005830 comp rx=0 tx=0).stop 2024-04-20T11:33:23.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.019+0000 7f52b6800640 1 -- 172.21.15.152:0/2116346508 reap_dead start 2024-04-20T11:33:23.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.019+0000 7f52b7200640 1 -- 172.21.15.152:0/2116346508 shutdown_connections 2024-04-20T11:33:23.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.019+0000 7f52b7200640 1 -- 172.21.15.152:0/2116346508 >> 172.21.15.152:0/2116346508 conn(0x7f52b8067ae0 msgr2=0x7f52b806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:23.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.019+0000 7f52b7200640 1 -- 172.21.15.152:0/2116346508 shutdown_connections 2024-04-20T11:33:23.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.019+0000 7f52b7200640 1 -- 172.21.15.152:0/2116346508 wait complete. 2024-04-20T11:33:23.021 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.021+0000 7f52b7200640 1 Processor -- start 2024-04-20T11:33:23.021 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.021+0000 7f52b7200640 1 -- start start 2024-04-20T11:33:23.022 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.022+0000 7f52b7200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f529c004d40 0x7f52b80b0330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:23.022 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.022+0000 7f52b7200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f52b80b46c0 0x7f52b80b4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:23.022 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.022+0000 7f52b7200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f52b80b24f0 0x7f52b80b29a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:23.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.022+0000 7f52b7200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f52b80cfad0 con 0x7f52b80b46c0 2024-04-20T11:33:23.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.022+0000 7f52b7200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f52b80cf950 con 0x7f52b80b24f0 2024-04-20T11:33:23.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.022+0000 7f52b7200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f52b806dcf0 con 0x7f529c004d40 2024-04-20T11:33:23.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.022+0000 7f52ad400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f52b80b46c0 0x7f52b80b4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:23.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.022+0000 7f52ad400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f52b80b46c0 0x7f52b80b4b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:37588/0 (socket says 172.21.15.152:37588) 2024-04-20T11:33:23.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.022+0000 7f52ad400640 1 -- 172.21.15.152:0/3692799221 learned_addr learned my addr 172.21.15.152:0/3692799221 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:33:23.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.023+0000 7f52b5e00640 1 --2- 172.21.15.152:0/3692799221 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f529c004d40 0x7f52b80b0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:23.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.023+0000 7f52b6800640 1 --2- 172.21.15.152:0/3692799221 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f52b80b24f0 0x7f52b80b29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:23.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.023+0000 7f52ad400640 1 -- 172.21.15.152:0/3692799221 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f529c004d40 msgr2=0x7f52b80b0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:23.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.023+0000 7f52ad400640 1 --2- 172.21.15.152:0/3692799221 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f529c004d40 0x7f52b80b0330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:23.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.023+0000 7f52ad400640 1 -- 172.21.15.152:0/3692799221 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f52b80b24f0 msgr2=0x7f52b80b29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:23.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.023+0000 7f52ad400640 1 --2- 172.21.15.152:0/3692799221 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f52b80b24f0 0x7f52b80b29a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:23.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.023+0000 7f52ad400640 1 -- 172.21.15.152:0/3692799221 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52b81f5200 con 0x7f52b80b46c0 2024-04-20T11:33:23.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.023+0000 7f52b5e00640 1 --2- 172.21.15.152:0/3692799221 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f529c004d40 0x7f52b80b0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:33:23.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.023+0000 7f52ad400640 1 --2- 172.21.15.152:0/3692799221 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f52b80b46c0 0x7f52b80b4b40 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f52a0007a70 tx=0x7f52a000e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:23.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.023+0000 7f52aea00640 1 -- 172.21.15.152:0/3692799221 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a0013420 con 0x7f52b80b46c0 2024-04-20T11:33:23.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.023+0000 7f52aea00640 1 -- 172.21.15.152:0/3692799221 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52a0018070 con 0x7f52b80b46c0 2024-04-20T11:33:23.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.023+0000 7f52aea00640 1 -- 172.21.15.152:0/3692799221 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a0013820 con 0x7f52b80b46c0 2024-04-20T11:33:23.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.024+0000 7f52b7200640 1 -- 172.21.15.152:0/3692799221 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52b81f54e0 con 0x7f52b80b46c0 2024-04-20T11:33:23.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.024+0000 7f52b7200640 1 -- 172.21.15.152:0/3692799221 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52b81f5aa0 con 0x7f52b80b46c0 2024-04-20T11:33:23.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.025+0000 7f52b7200640 1 -- 172.21.15.152:0/3692799221 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52b8075e60 con 0x7f52b80b46c0 2024-04-20T11:33:23.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.026+0000 7f52aea00640 1 -- 172.21.15.152:0/3692799221 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 91142+0+0 (secure 0 0 0) 0x7f52a00139c0 con 0x7f52b80b46c0 2024-04-20T11:33:23.026 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.027+0000 7f52aea00640 1 --2- 172.21.15.152:0/3692799221 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f529406ca10 0x7f529406eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:23.026 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.027+0000 7f52aea00640 1 -- 172.21.15.152:0/3692799221 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 4857+0+0 (secure 0 0 0) 0x7f52a008c190 con 0x7f52b80b46c0 2024-04-20T11:33:23.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.027+0000 7f52b5e00640 1 --2- 172.21.15.152:0/3692799221 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f529406ca10 0x7f529406eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:23.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.029+0000 7f52b5e00640 1 --2- 172.21.15.152:0/3692799221 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f529406ca10 0x7f529406eed0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f52a8001770 tx=0x7f52a8009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:23.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.031+0000 7f52aea00640 1 -- 172.21.15.152:0/3692799221 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f52a005a110 con 0x7f52b80b46c0 2024-04-20T11:33:23.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.301+0000 7f52b7200640 1 -- 172.21.15.152:0/3692799221 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f52b806ecc0 con 0x7f52b80b46c0 2024-04-20T11:33:23.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.302+0000 7f52aea00640 1 -- 172.21.15.152:0/3692799221 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) ==== 80+0+5304 (secure 0 0 0) 0x7f52a005e410 con 0x7f52b80b46c0 2024-04-20T11:33:23.302 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:33:23.302 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-04-20T11:33:23.302 INFO:teuthology.orchestra.run.smithi152.stdout: "auth_dump": [ 2024-04-20T11:33:23.303 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:33:23.303 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "mds.a", 2024-04-20T11:33:23.303 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBCpiNmyFRJIBAAj2aIMXdb65nqRgKFDBLLzA==", 2024-04-20T11:33:23.303 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:33:23.303 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow", 2024-04-20T11:33:23.303 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow *", 2024-04-20T11:33:23.303 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow *", 2024-04-20T11:33:23.303 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:33:23.303 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:33:23.303 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:33:23.303 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:33:23.303 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "mds.b", 2024-04-20T11:33:23.303 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBCpiNmVPytOhAAzQkvNOJaNy+MeKETWhYLyg==", 2024-04-20T11:33:23.303 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:33:23.304 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow", 2024-04-20T11:33:23.304 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow *", 2024-04-20T11:33:23.304 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow *", 2024-04-20T11:33:23.304 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:33:23.304 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:33:23.304 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:33:23.304 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:33:23.304 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "mds.c", 2024-04-20T11:33:23.304 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBCpiNm5ViKLRAAg7VBQG4hFTLYUpJMpBAaVA==", 2024-04-20T11:33:23.304 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:33:23.304 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow", 2024-04-20T11:33:23.304 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow *", 2024-04-20T11:33:23.304 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow *", 2024-04-20T11:33:23.304 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:33:23.305 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:33:23.305 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:33:23.305 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:33:23.305 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "mds.d", 2024-04-20T11:33:23.305 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBDpiNmLWhaCxAA2e4qd/7+TKenJlWICIsvGg==", 2024-04-20T11:33:23.305 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:33:23.305 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow", 2024-04-20T11:33:23.305 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow *", 2024-04-20T11:33:23.305 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow *", 2024-04-20T11:33:23.305 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:33:23.305 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:33:23.305 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:33:23.305 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:33:23.305 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "osd.0", 2024-04-20T11:33:23.306 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBHpiNmpuhyDBAAsQXX9DNvWlo0UYh53lb+FA==", 2024-04-20T11:33:23.306 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:33:23.306 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow profile osd", 2024-04-20T11:33:23.306 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile osd", 2024-04-20T11:33:23.306 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:33:23.306 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:33:23.306 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:33:23.306 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:33:23.307 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "osd.1", 2024-04-20T11:33:23.307 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBKpiNmxBiLJhAAHon1adC8G2jbYoDcWdE9jA==", 2024-04-20T11:33:23.307 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:33:23.307 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow profile osd", 2024-04-20T11:33:23.307 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile osd", 2024-04-20T11:33:23.308 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:33:23.308 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:33:23.308 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:33:23.308 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:33:23.308 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "osd.2", 2024-04-20T11:33:23.308 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBNpiNmdn0zMhAAqwIO2kKahdSzTVh8/p3FCg==", 2024-04-20T11:33:23.308 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:33:23.308 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow profile osd", 2024-04-20T11:33:23.308 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile osd", 2024-04-20T11:33:23.308 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:33:23.309 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:33:23.309 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:33:23.309 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:33:23.309 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "osd.3", 2024-04-20T11:33:23.309 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBRpiNmTPCYAhAAHFNHX/xKn21jEFHDkHMjEw==", 2024-04-20T11:33:23.309 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:33:23.310 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow profile osd", 2024-04-20T11:33:23.310 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile osd", 2024-04-20T11:33:23.310 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:33:23.310 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:33:23.310 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:33:23.310 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:33:23.310 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "osd.4", 2024-04-20T11:33:23.310 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBXpiNmM8cVFxAAJl9Hu+MvlkEiMglutI89eg==", 2024-04-20T11:33:23.310 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:33:23.310 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow profile osd", 2024-04-20T11:33:23.311 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile osd", 2024-04-20T11:33:23.311 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:33:23.311 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:33:23.311 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:33:23.311 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:33:23.311 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "osd.5", 2024-04-20T11:33:23.311 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBapiNmnGxJMRAA90P0T+VsTA8LCTcexTVr7w==", 2024-04-20T11:33:23.311 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:33:23.311 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow profile osd", 2024-04-20T11:33:23.311 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile osd", 2024-04-20T11:33:23.311 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:33:23.311 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:33:23.311 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:33:23.311 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:33:23.311 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "osd.6", 2024-04-20T11:33:23.312 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBepiNmyLXWDxAAhtHD5h478mLovE5IejAFng==", 2024-04-20T11:33:23.312 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:33:23.312 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow profile osd", 2024-04-20T11:33:23.312 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile osd", 2024-04-20T11:33:23.312 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:33:23.312 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:33:23.312 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:33:23.312 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:33:23.312 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "osd.7", 2024-04-20T11:33:23.312 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBhpiNmB3jwHBAAqco9om6r2N/GqAV6rVKiaA==", 2024-04-20T11:33:23.312 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:33:23.312 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow profile osd", 2024-04-20T11:33:23.312 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile osd", 2024-04-20T11:33:23.312 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:33:23.313 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:33:23.313 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:33:23.313 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:33:23.313 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "client.0", 2024-04-20T11:33:23.313 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBDpiNmtsBWFhAArq9A7r33CcwgMmOU2pAGPw==", 2024-04-20T11:33:23.313 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:33:23.313 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow", 2024-04-20T11:33:23.313 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow r", 2024-04-20T11:33:23.313 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-20T11:33:23.313 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:33:23.314 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:33:23.314 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:33:23.314 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "client.1", 2024-04-20T11:33:23.314 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBDpiNm6aNnJxAAB431GSGytZ0IKWMh4DMalw==", 2024-04-20T11:33:23.314 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:33:23.314 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow", 2024-04-20T11:33:23.314 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow r", 2024-04-20T11:33:23.314 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow rw", 2024-04-20T11:33:23.314 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow rwx" 2024-04-20T11:33:23.314 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:33:23.314 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:33:23.314 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:33:23.314 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "client.2", 2024-04-20T11:33:23.314 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBDpiNm5QLtHRAAa2kHTDVdgSh/B4XGZaKa5Q==", 2024-04-20T11:33:23.315 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:33:23.315 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow", 2024-04-20T11:33:23.315 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow r", 2024-04-20T11:33:23.315 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow rw", 2024-04-20T11:33:23.315 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow rwx" 2024-04-20T11:33:23.315 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:33:23.315 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:33:23.315 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:33:23.315 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "client.3", 2024-04-20T11:33:23.315 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBDpiNmw95SMBAAdodzTuPp4WdKSEf+jI8Uuw==", 2024-04-20T11:33:23.315 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:33:23.315 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow", 2024-04-20T11:33:23.315 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow r", 2024-04-20T11:33:23.315 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow rw", 2024-04-20T11:33:23.315 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow rwx" 2024-04-20T11:33:23.316 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:33:23.316 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:33:23.316 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:33:23.316 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "client.admin", 2024-04-20T11:33:23.316 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBBpiNm3xz6HhAA82+N+trDW/AmBr3q8l4ECw==", 2024-04-20T11:33:23.316 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:33:23.316 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow *", 2024-04-20T11:33:23.316 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow *", 2024-04-20T11:33:23.316 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow *", 2024-04-20T11:33:23.316 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:33:23.316 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:33:23.316 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:33:23.316 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:33:23.316 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "mgr.x", 2024-04-20T11:33:23.316 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBCpiNmuhalFxAA9dTl/zSbUovCCZAanAipww==", 2024-04-20T11:33:23.317 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:33:23.317 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow *", 2024-04-20T11:33:23.317 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile mgr", 2024-04-20T11:33:23.317 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:33:23.317 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:33:23.317 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:33:23.317 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:33:23.317 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "mgr.y", 2024-04-20T11:33:23.317 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBCpiNmP1+5DRAAs6bVAEOr0i6FdPmSEyo/vQ==", 2024-04-20T11:33:23.317 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:33:23.317 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow *", 2024-04-20T11:33:23.317 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile mgr", 2024-04-20T11:33:23.317 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:33:23.317 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:33:23.318 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:33:23.318 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-04-20T11:33:23.318 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-04-20T11:33:23.318 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.305+0000 7f52b7200640 1 -- 172.21.15.152:0/3692799221 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f529406ca10 msgr2=0x7f529406eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:23.319 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.305+0000 7f52b7200640 1 --2- 172.21.15.152:0/3692799221 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f529406ca10 0x7f529406eed0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f52a8001770 tx=0x7f52a8009040 comp rx=0 tx=0).stop 2024-04-20T11:33:23.319 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.305+0000 7f52b7200640 1 -- 172.21.15.152:0/3692799221 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f52b80b46c0 msgr2=0x7f52b80b4b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:23.319 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.305+0000 7f52b7200640 1 --2- 172.21.15.152:0/3692799221 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f52b80b46c0 0x7f52b80b4b40 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f52a0007a70 tx=0x7f52a000e530 comp rx=0 tx=0).stop 2024-04-20T11:33:23.319 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.305+0000 7f52b7200640 1 -- 172.21.15.152:0/3692799221 shutdown_connections 2024-04-20T11:33:23.319 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.305+0000 7f52b7200640 1 --2- 172.21.15.152:0/3692799221 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f529406ca10 0x7f529406eed0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:23.319 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.305+0000 7f52b7200640 1 --2- 172.21.15.152:0/3692799221 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f52b80b24f0 0x7f52b80b29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:23.319 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.305+0000 7f52b7200640 1 --2- 172.21.15.152:0/3692799221 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f52b80b46c0 0x7f52b80b4b40 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:23.319 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.305+0000 7f52b7200640 1 --2- 172.21.15.152:0/3692799221 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f529c004d40 0x7f52b80b0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:23.319 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.305+0000 7f52b7200640 1 -- 172.21.15.152:0/3692799221 >> 172.21.15.152:0/3692799221 conn(0x7f52b8067ae0 msgr2=0x7f52b806e080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:23.320 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.305+0000 7f52b7200640 1 -- 172.21.15.152:0/3692799221 shutdown_connections 2024-04-20T11:33:23.320 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.305+0000 7f52b7200640 1 -- 172.21.15.152:0/3692799221 wait complete. 2024-04-20T11:33:23.323 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:33:23.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.520+0000 7f6dbfe00640 1 Processor -- start 2024-04-20T11:33:23.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.520+0000 7f6dbfe00640 1 -- start start 2024-04-20T11:33:23.522 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.522+0000 7f6dbfe00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f6dc00723e0 0x7f6dc00afa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:23.522 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.522+0000 7f6dbfe00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f6dc006e010 0x7f6dc006e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:23.522 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.522+0000 7f6dbfe00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6dc00aec90 con 0x7f6dc0147410 2024-04-20T11:33:23.522 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.522+0000 7f6dbfe00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f6dc00aef70 con 0x7f6dc00723e0 2024-04-20T11:33:23.522 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.522+0000 7f6dbfe00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f6dc00af250 con 0x7f6dc006e010 2024-04-20T11:33:23.522 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.522+0000 7f6db6000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f6dc00723e0 0x7f6dc00afa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:23.522 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.522+0000 7f6dbea00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f6dc0147410 0x7f6dc0071cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:55214/0 (socket says 172.21.15.152:55214) 2024-04-20T11:33:23.522 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.522+0000 7f6dbea00640 1 -- 172.21.15.152:0/3070590200 learned_addr learned my addr 172.21.15.152:0/3070590200 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:33:23.523 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.523+0000 7f6dbf400640 1 --2- 172.21.15.152:0/3070590200 >> v2:172.21.15.175:3300/0 conn(0x7f6dc006e010 0x7f6dc006e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:23.523 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.523+0000 7f6db6000640 1 -- 172.21.15.152:0/3070590200 >> v2:172.21.15.175:3300/0 conn(0x7f6dc006e010 msgr2=0x7f6dc006e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:23.523 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.523+0000 7f6db6000640 1 --2- 172.21.15.152:0/3070590200 >> v2:172.21.15.175:3300/0 conn(0x7f6dc006e010 0x7f6dc006e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:23.523 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.523+0000 7f6db6000640 1 -- 172.21.15.152:0/3070590200 >> v1:172.21.15.152:6789/0 conn(0x7f6dc0147410 legacy=0x7f6dc0071cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:23.523 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.523+0000 7f6db6000640 1 -- 172.21.15.152:0/3070590200 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dc00af530 con 0x7f6dc00723e0 2024-04-20T11:33:23.523 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.523+0000 7f6dbe000640 1 -- 172.21.15.152:0/3070590200 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 906889122 0 0) 0x7f6dc00aec90 con 0x7f6dc0147410 2024-04-20T11:33:23.524 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.524+0000 7f6db6000640 1 --2- 172.21.15.152:0/3070590200 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6dc00723e0 0x7f6dc00afa50 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f6dc00700c0 tx=0x7f6da802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4db92c39392949cc server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:23.524 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.524+0000 7f6dbe000640 1 -- 172.21.15.152:0/3070590200 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da8035020 con 0x7f6dc00723e0 2024-04-20T11:33:23.524 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.524+0000 7f6dbe000640 1 -- 172.21.15.152:0/3070590200 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6da8031830 con 0x7f6dc00723e0 2024-04-20T11:33:23.524 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.525+0000 7f6dbe000640 1 -- 172.21.15.152:0/3070590200 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da8031c00 con 0x7f6dc00723e0 2024-04-20T11:33:23.525 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.525+0000 7f6dbfe00640 1 -- 172.21.15.152:0/3070590200 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6dc00723e0 msgr2=0x7f6dc00afa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:23.525 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.525+0000 7f6dbfe00640 1 --2- 172.21.15.152:0/3070590200 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6dc00723e0 0x7f6dc00afa50 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f6dc00700c0 tx=0x7f6da802b8d0 comp rx=0 tx=0).stop 2024-04-20T11:33:23.525 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.525+0000 7f6dbfe00640 1 -- 172.21.15.152:0/3070590200 shutdown_connections 2024-04-20T11:33:23.525 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.525+0000 7f6dbfe00640 1 --2- 172.21.15.152:0/3070590200 >> v2:172.21.15.175:3300/0 conn(0x7f6dc006e010 0x7f6dc006e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:23.525 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.525+0000 7f6dbfe00640 1 --2- 172.21.15.152:0/3070590200 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6dc00723e0 0x7f6dc00afa50 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:23.525 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.525+0000 7f6dbfe00640 1 -- 172.21.15.152:0/3070590200 >> 172.21.15.152:0/3070590200 conn(0x7f6dc0067ae0 msgr2=0x7f6dc006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:23.525 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.525+0000 7f6dbfe00640 1 -- 172.21.15.152:0/3070590200 shutdown_connections 2024-04-20T11:33:23.525 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.525+0000 7f6dbfe00640 1 -- 172.21.15.152:0/3070590200 wait complete. 2024-04-20T11:33:23.526 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.527+0000 7f6dbfe00640 1 Processor -- start 2024-04-20T11:33:23.526 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.527+0000 7f6dbfe00640 1 -- start start 2024-04-20T11:33:23.528 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.528+0000 7f6dbfe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6dc006e010 0x7f6dc013e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:23.528 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.528+0000 7f6dbfe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6dc00723e0 0x7f6dc0142fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:23.528 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.528+0000 7f6dbfe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6dc0147410 0x7f6dc0143520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:23.528 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.528+0000 7f6dbfe00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc006eb40 con 0x7f6dc0147410 2024-04-20T11:33:23.529 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.528+0000 7f6dbfe00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc006e9c0 con 0x7f6dc006e010 2024-04-20T11:33:23.529 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.528+0000 7f6dbfe00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6dc00b3390 con 0x7f6dc00723e0 2024-04-20T11:33:23.529 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.528+0000 7f6dbf400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6dc0147410 0x7f6dc0143520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:23.529 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.528+0000 7f6dbf400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6dc0147410 0x7f6dc0143520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:37610/0 (socket says 172.21.15.152:37610) 2024-04-20T11:33:23.529 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.528+0000 7f6dbf400640 1 -- 172.21.15.152:0/4055954239 learned_addr learned my addr 172.21.15.152:0/4055954239 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:33:23.529 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.529+0000 7f6dbea00640 1 --2- 172.21.15.152:0/4055954239 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6dc006e010 0x7f6dc013e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:23.529 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.529+0000 7f6db6000640 1 --2- 172.21.15.152:0/4055954239 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6dc00723e0 0x7f6dc0142fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:23.529 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.529+0000 7f6dbf400640 1 -- 172.21.15.152:0/4055954239 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6dc00723e0 msgr2=0x7f6dc0142fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:23.529 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.529+0000 7f6dbf400640 1 --2- 172.21.15.152:0/4055954239 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6dc00723e0 0x7f6dc0142fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:23.529 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.529+0000 7f6dbf400640 1 -- 172.21.15.152:0/4055954239 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6dc006e010 msgr2=0x7f6dc013e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:23.529 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.529+0000 7f6dbf400640 1 --2- 172.21.15.152:0/4055954239 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6dc006e010 0x7f6dc013e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:23.529 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.529+0000 7f6dbf400640 1 -- 172.21.15.152:0/4055954239 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dc01415b0 con 0x7f6dc0147410 2024-04-20T11:33:23.530 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.529+0000 7f6dbf400640 1 --2- 172.21.15.152:0/4055954239 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6dc0147410 0x7f6dc0143520 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f6db800d800 tx=0x7f6db800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:23.530 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.529+0000 7f6db7400640 1 -- 172.21.15.152:0/4055954239 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6db800b6c0 con 0x7f6dc0147410 2024-04-20T11:33:23.530 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.530+0000 7f6db7400640 1 -- 172.21.15.152:0/4055954239 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6db8019070 con 0x7f6dc0147410 2024-04-20T11:33:23.530 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.530+0000 7f6db7400640 1 -- 172.21.15.152:0/4055954239 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6db800bac0 con 0x7f6dc0147410 2024-04-20T11:33:23.531 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.530+0000 7f6dbfe00640 1 -- 172.21.15.152:0/4055954239 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6dc0141890 con 0x7f6dc0147410 2024-04-20T11:33:23.531 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.530+0000 7f6dbfe00640 1 -- 172.21.15.152:0/4055954239 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6dc0141e50 con 0x7f6dc0147410 2024-04-20T11:33:23.531 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.531+0000 7f6dbfe00640 1 -- 172.21.15.152:0/4055954239 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6dc0075e60 con 0x7f6dc0147410 2024-04-20T11:33:23.531 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.532+0000 7f6db7400640 1 -- 172.21.15.152:0/4055954239 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 91142+0+0 (secure 0 0 0) 0x7f6db800bc60 con 0x7f6dc0147410 2024-04-20T11:33:23.532 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.533+0000 7f6db7400640 1 --2- 172.21.15.152:0/4055954239 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f6d9c06ca40 0x7f6d9c06ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:23.532 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.533+0000 7f6db7400640 1 -- 172.21.15.152:0/4055954239 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 4857+0+0 (secure 0 0 0) 0x7f6db808bdf0 con 0x7f6dc0147410 2024-04-20T11:33:23.533 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.533+0000 7f6dbea00640 1 --2- 172.21.15.152:0/4055954239 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f6d9c06ca40 0x7f6d9c06ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:23.534 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.535+0000 7f6dbea00640 1 --2- 172.21.15.152:0/4055954239 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f6d9c06ca40 0x7f6d9c06ef00 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6db0007ed0 tx=0x7f6db0007e60 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:23.537 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.537+0000 7f6db7400640 1 -- 172.21.15.152:0/4055954239 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f6db805a450 con 0x7f6dc0147410 2024-04-20T11:33:23.741 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.741+0000 7f6dbfe00640 1 -- 172.21.15.152:0/4055954239 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6dc01421c0 con 0x7f6dc0147410 2024-04-20T11:33:23.742 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.743+0000 7f6db7400640 1 -- 172.21.15.152:0/4055954239 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) ==== 74+0+12952 (secure 0 0 0) 0x7f6db8059d50 con 0x7f6dc0147410 2024-04-20T11:33:23.743 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:33:23.743 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":99,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:33:22.759621+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:33:23.745 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.746+0000 7f6dbfe00640 1 -- 172.21.15.152:0/4055954239 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f6d9c06ca40 msgr2=0x7f6d9c06ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:23.745 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.746+0000 7f6dbfe00640 1 --2- 172.21.15.152:0/4055954239 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f6d9c06ca40 0x7f6d9c06ef00 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6db0007ed0 tx=0x7f6db0007e60 comp rx=0 tx=0).stop 2024-04-20T11:33:23.745 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.746+0000 7f6dbfe00640 1 -- 172.21.15.152:0/4055954239 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6dc0147410 msgr2=0x7f6dc0143520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:23.746 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.746+0000 7f6dbfe00640 1 --2- 172.21.15.152:0/4055954239 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6dc0147410 0x7f6dc0143520 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f6db800d800 tx=0x7f6db800dcc0 comp rx=0 tx=0).stop 2024-04-20T11:33:23.746 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.746+0000 7f6dbfe00640 1 -- 172.21.15.152:0/4055954239 shutdown_connections 2024-04-20T11:33:23.746 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.746+0000 7f6dbfe00640 1 --2- 172.21.15.152:0/4055954239 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f6d9c06ca40 0x7f6d9c06ef00 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:23.746 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.746+0000 7f6dbfe00640 1 --2- 172.21.15.152:0/4055954239 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6dc0147410 0x7f6dc0143520 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:23.746 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.746+0000 7f6dbfe00640 1 --2- 172.21.15.152:0/4055954239 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6dc00723e0 0x7f6dc0142fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:23.746 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.746+0000 7f6dbfe00640 1 --2- 172.21.15.152:0/4055954239 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6dc006e010 0x7f6dc013e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:23.747 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.746+0000 7f6dbfe00640 1 -- 172.21.15.152:0/4055954239 >> 172.21.15.152:0/4055954239 conn(0x7f6dc0067ae0 msgr2=0x7f6dc01477f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:23.747 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.746+0000 7f6dbfe00640 1 -- 172.21.15.152:0/4055954239 shutdown_connections 2024-04-20T11:33:23.747 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.746+0000 7f6dbfe00640 1 -- 172.21.15.152:0/4055954239 wait complete. 2024-04-20T11:33:23.764 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T11:26:47.246571+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T11:33:23.765 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T11:33:23.963 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.963+0000 7fe4f1600640 1 Processor -- start 2024-04-20T11:33:23.963 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.963+0000 7fe4f1600640 1 -- start start 2024-04-20T11:33:23.965 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.965+0000 7fe4f1600640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fe4ec0cd8c0 0x7fe4ec0b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:23.965 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.965+0000 7fe4f1600640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4ec1463e0 con 0x7fe4ec0b2880 2024-04-20T11:33:23.965 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.965+0000 7fe4f1600640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4ec0b3e20 con 0x7fe4ec146000 2024-04-20T11:33:23.965 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.965+0000 7fe4f1600640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fe4ec0b4750 con 0x7fe4ec0cd8c0 2024-04-20T11:33:23.965 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.965+0000 7fe4eb400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fe4ec0cd8c0 0x7fe4ec0b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:23.965 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.965+0000 7fe4eb400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fe4ec0cd8c0 0x7fe4ec0b4be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:37624/0 (socket says 172.21.15.152:37624) 2024-04-20T11:33:23.965 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.965+0000 7fe4eb400640 1 -- 172.21.15.152:0/1437168938 learned_addr learned my addr 172.21.15.152:0/1437168938 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:33:23.965 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.965+0000 7fe4ea000640 1 --1- 172.21.15.152:0/1437168938 >> v1:172.21.15.152:6789/0 conn(0x7fe4ec0b2880 0x7fe4ec0cd1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:55230/0 (socket says 172.21.15.152:55230) 2024-04-20T11:33:23.965 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.965+0000 7fe4eb400640 1 -- 172.21.15.152:0/1437168938 >> v1:172.21.15.175:6789/0 conn(0x7fe4ec146000 legacy=0x7fe4ec0b2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:23.965 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.965+0000 7fe4eb400640 1 -- 172.21.15.152:0/1437168938 >> v1:172.21.15.152:6789/0 conn(0x7fe4ec0b2880 legacy=0x7fe4ec0cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:23.965 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.965+0000 7fe4eb400640 1 -- 172.21.15.152:0/1437168938 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4ec0b4100 con 0x7fe4ec0cd8c0 2024-04-20T11:33:23.966 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.967+0000 7fe4eb400640 1 --2- 172.21.15.152:0/1437168938 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe4ec0cd8c0 0x7fe4ec0b4be0 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7fe4e4009e50 tx=0x7fe4e402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=766b311b15200cd server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:23.966 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.967+0000 7fe4e9600640 1 -- 172.21.15.152:0/1437168938 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e402ca80 con 0x7fe4ec0cd8c0 2024-04-20T11:33:23.966 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.967+0000 7fe4e9600640 1 -- 172.21.15.152:0/1437168938 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4e402cc20 con 0x7fe4ec0cd8c0 2024-04-20T11:33:23.967 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.967+0000 7fe4e9600640 1 -- 172.21.15.152:0/1437168938 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e40343f0 con 0x7fe4ec0cd8c0 2024-04-20T11:33:23.967 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.967+0000 7fe4f1600640 1 -- 172.21.15.152:0/1437168938 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe4ec0cd8c0 msgr2=0x7fe4ec0b4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:23.967 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.967+0000 7fe4f1600640 1 --2- 172.21.15.152:0/1437168938 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe4ec0cd8c0 0x7fe4ec0b4be0 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7fe4e4009e50 tx=0x7fe4e402bee0 comp rx=0 tx=0).stop 2024-04-20T11:33:23.967 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.968+0000 7fe4f1600640 1 -- 172.21.15.152:0/1437168938 shutdown_connections 2024-04-20T11:33:23.968 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.968+0000 7fe4f1600640 1 --2- 172.21.15.152:0/1437168938 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe4ec0cd8c0 0x7fe4ec0b4be0 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:23.968 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.968+0000 7fe4f1600640 1 -- 172.21.15.152:0/1437168938 >> 172.21.15.152:0/1437168938 conn(0x7fe4ec06b980 msgr2=0x7fe4ec06bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:23.968 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.968+0000 7fe4f1600640 1 -- 172.21.15.152:0/1437168938 shutdown_connections 2024-04-20T11:33:23.968 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.968+0000 7fe4f1600640 1 -- 172.21.15.152:0/1437168938 wait complete. 2024-04-20T11:33:23.969 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.969+0000 7fe4f1600640 1 Processor -- start 2024-04-20T11:33:23.969 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.970+0000 7fe4f1600640 1 -- start start 2024-04-20T11:33:23.971 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.970+0000 7fe4f1600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe4ec0b2880 0x7fe4ec13f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:23.971 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.970+0000 7fe4f1600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe4ec0cd8c0 0x7fe4ec1f45a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:23.971 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.970+0000 7fe4f1600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe4ec146000 0x7fe4ec1442b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:23.971 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.970+0000 7fe4f1600640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fe4ec15ce70 con 0x7fe4ec0cd8c0 2024-04-20T11:33:23.971 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.970+0000 7fe4f1600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe4ec15ccf0 con 0x7fe4ec0b2880 2024-04-20T11:33:23.971 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.970+0000 7fe4f1600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe4ec15cff0 con 0x7fe4ec146000 2024-04-20T11:33:23.971 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.971+0000 7fe4ea000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe4ec0cd8c0 0x7fe4ec1f45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:23.971 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.971+0000 7fe4ea000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe4ec0cd8c0 0x7fe4ec1f45a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:37640/0 (socket says 172.21.15.152:37640) 2024-04-20T11:33:23.971 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.971+0000 7fe4ea000640 1 -- 172.21.15.152:0/1863391373 learned_addr learned my addr 172.21.15.152:0/1863391373 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:33:23.971 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.971+0000 7fe4ea000640 1 -- 172.21.15.152:0/1863391373 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe4ec146000 msgr2=0x7fe4ec1442b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:23.971 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.971+0000 7fe4ea000640 1 --2- 172.21.15.152:0/1863391373 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe4ec146000 0x7fe4ec1442b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:23.972 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.971+0000 7fe4eaa00640 1 --2- 172.21.15.152:0/1863391373 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe4ec0b2880 0x7fe4ec13f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:23.972 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.971+0000 7fe4ea000640 1 -- 172.21.15.152:0/1863391373 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe4ec0b2880 msgr2=0x7fe4ec13f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:23.972 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.971+0000 7fe4ea000640 1 --2- 172.21.15.152:0/1863391373 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe4ec0b2880 0x7fe4ec13f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:23.972 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.971+0000 7fe4ea000640 1 -- 172.21.15.152:0/1863391373 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4ec1f5060 con 0x7fe4ec0cd8c0 2024-04-20T11:33:23.972 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.971+0000 7fe4eaa00640 1 --2- 172.21.15.152:0/1863391373 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe4ec0b2880 0x7fe4ec13f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:33:23.972 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.971+0000 7fe4ea000640 1 --2- 172.21.15.152:0/1863391373 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe4ec0cd8c0 0x7fe4ec1f45a0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7fe4d800c3d0 tx=0x7fe4d800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:23.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.972+0000 7fe4e2a00640 1 -- 172.21.15.152:0/1863391373 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d80074e0 con 0x7fe4ec0cd8c0 2024-04-20T11:33:23.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.972+0000 7fe4e2a00640 1 -- 172.21.15.152:0/1863391373 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4d8007680 con 0x7fe4ec0cd8c0 2024-04-20T11:33:23.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.972+0000 7fe4f1600640 1 -- 172.21.15.152:0/1863391373 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4ec1f5340 con 0x7fe4ec0cd8c0 2024-04-20T11:33:23.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.972+0000 7fe4e2a00640 1 -- 172.21.15.152:0/1863391373 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d8004a00 con 0x7fe4ec0cd8c0 2024-04-20T11:33:23.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.972+0000 7fe4f1600640 1 -- 172.21.15.152:0/1863391373 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4ec0b1780 con 0x7fe4ec0cd8c0 2024-04-20T11:33:23.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.973+0000 7fe4f1600640 1 -- 172.21.15.152:0/1863391373 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4ec0689f0 con 0x7fe4ec0cd8c0 2024-04-20T11:33:23.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.974+0000 7fe4e2a00640 1 -- 172.21.15.152:0/1863391373 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 91142+0+0 (secure 0 0 0) 0x7fe4d801a020 con 0x7fe4ec0cd8c0 2024-04-20T11:33:23.974 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.975+0000 7fe4e2a00640 1 --2- 172.21.15.152:0/1863391373 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe4cc06ca10 0x7fe4cc06eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:23.974 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.975+0000 7fe4e2a00640 1 -- 172.21.15.152:0/1863391373 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 4857+0+0 (secure 0 0 0) 0x7fe4d808b210 con 0x7fe4ec0cd8c0 2024-04-20T11:33:23.975 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.976+0000 7fe4eaa00640 1 --2- 172.21.15.152:0/1863391373 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe4cc06ca10 0x7fe4cc06eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:23.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.977+0000 7fe4eaa00640 1 --2- 172.21.15.152:0/1863391373 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe4cc06ca10 0x7fe4cc06eed0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec06fc90 tx=0x7fe4dc008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:23.978 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:23.979+0000 7fe4e2a00640 1 -- 172.21.15.152:0/1863391373 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe4d8010300 con 0x7fe4ec0cd8c0 2024-04-20T11:33:24.182 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.182+0000 7fe4f1600640 1 -- 172.21.15.152:0/1863391373 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe4ec13f7d0 con 0x7fe4ec0cd8c0 2024-04-20T11:33:24.182 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.183+0000 7fe4e2a00640 1 -- 172.21.15.152:0/1863391373 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v99) ==== 93+0+10 (secure 0 0 0) 0x7fe4d80597e0 con 0x7fe4ec0cd8c0 2024-04-20T11:33:24.183 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 1 2024-04-20T11:33:24.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.185+0000 7fe4f1600640 1 -- 172.21.15.152:0/1863391373 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe4cc06ca10 msgr2=0x7fe4cc06eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:24.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.185+0000 7fe4f1600640 1 --2- 172.21.15.152:0/1863391373 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe4cc06ca10 0x7fe4cc06eed0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec06fc90 tx=0x7fe4dc008040 comp rx=0 tx=0).stop 2024-04-20T11:33:24.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.185+0000 7fe4f1600640 1 -- 172.21.15.152:0/1863391373 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe4ec0cd8c0 msgr2=0x7fe4ec1f45a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:24.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.185+0000 7fe4f1600640 1 --2- 172.21.15.152:0/1863391373 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe4ec0cd8c0 0x7fe4ec1f45a0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7fe4d800c3d0 tx=0x7fe4d800c890 comp rx=0 tx=0).stop 2024-04-20T11:33:24.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.186+0000 7fe4f1600640 1 -- 172.21.15.152:0/1863391373 shutdown_connections 2024-04-20T11:33:24.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.186+0000 7fe4f1600640 1 --2- 172.21.15.152:0/1863391373 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe4cc06ca10 0x7fe4cc06eed0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:24.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.186+0000 7fe4f1600640 1 --2- 172.21.15.152:0/1863391373 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe4ec146000 0x7fe4ec1442b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:24.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.186+0000 7fe4f1600640 1 --2- 172.21.15.152:0/1863391373 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe4ec0cd8c0 0x7fe4ec1f45a0 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:24.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.186+0000 7fe4f1600640 1 --2- 172.21.15.152:0/1863391373 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe4ec0b2880 0x7fe4ec13f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:24.187 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.186+0000 7fe4f1600640 1 -- 172.21.15.152:0/1863391373 >> 172.21.15.152:0/1863391373 conn(0x7fe4ec06b980 msgr2=0x7fe4ec0ae9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:24.187 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.186+0000 7fe4f1600640 1 -- 172.21.15.152:0/1863391373 shutdown_connections 2024-04-20T11:33:24.187 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.186+0000 7fe4f1600640 1 -- 172.21.15.152:0/1863391373 wait complete. 2024-04-20T11:33:24.204 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-20T11:33:24.404 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.405+0000 7efd54a00640 1 Processor -- start 2024-04-20T11:33:24.405 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.405+0000 7efd54a00640 1 -- start start 2024-04-20T11:33:24.406 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.406+0000 7efd54a00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7efd5015b5e0 0x7efd5006eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:24.406 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.406+0000 7efd54a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efd500b4c50 0x7efd500b50d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:24.406 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.406+0000 7efd54a00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd500b5770 con 0x7efd5006f340 2024-04-20T11:33:24.406 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.406+0000 7efd54a00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7efd5006de10 con 0x7efd5015b5e0 2024-04-20T11:33:24.406 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.406+0000 7efd54a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7efd5006e060 con 0x7efd500b4c50 2024-04-20T11:33:24.408 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.407+0000 7efd4e800640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7efd5006f340 0x7efd5015cd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:55232/0 (socket says 172.21.15.152:55232) 2024-04-20T11:33:24.408 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.407+0000 7efd4e800640 1 -- 172.21.15.152:0/2724228070 learned_addr learned my addr 172.21.15.152:0/2724228070 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:33:24.408 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.407+0000 7efd4de00640 1 --2- 172.21.15.152:0/2724228070 >> v2:172.21.15.175:3300/0 conn(0x7efd5015b5e0 0x7efd5006eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:24.408 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.407+0000 7efd45400640 1 --2- 172.21.15.152:0/2724228070 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efd500b4c50 0x7efd500b50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:24.408 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.407+0000 7efd4d400640 1 -- 172.21.15.152:0/2724228070 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3165617916 0 0) 0x7efd500b5770 con 0x7efd5006f340 2024-04-20T11:33:24.409 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.407+0000 7efd4d400640 1 -- 172.21.15.152:0/2724228070 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efd34005730 con 0x7efd5006f340 2024-04-20T11:33:24.409 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.407+0000 7efd4d400640 1 -- 172.21.15.152:0/2724228070 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1224710680 0 0) 0x7efd34005730 con 0x7efd5006f340 2024-04-20T11:33:24.409 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.407+0000 7efd4d400640 1 -- 172.21.15.152:0/2724228070 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efd500b4c50 msgr2=0x7efd500b50d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:24.409 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.407+0000 7efd4d400640 1 --2- 172.21.15.152:0/2724228070 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efd500b4c50 0x7efd500b50d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:24.409 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.408+0000 7efd4d400640 1 -- 172.21.15.152:0/2724228070 >> v2:172.21.15.175:3300/0 conn(0x7efd5015b5e0 msgr2=0x7efd5006eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:24.409 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.408+0000 7efd4d400640 1 --2- 172.21.15.152:0/2724228070 >> v2:172.21.15.175:3300/0 conn(0x7efd5015b5e0 0x7efd5006eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:24.409 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.408+0000 7efd4d400640 1 -- 172.21.15.152:0/2724228070 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd5006e340 con 0x7efd5006f340 2024-04-20T11:33:24.409 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.408+0000 7efd4d400640 1 -- 172.21.15.152:0/2724228070 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7efd480019f0 con 0x7efd5006f340 2024-04-20T11:33:24.409 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.408+0000 7efd4d400640 1 -- 172.21.15.152:0/2724228070 >> v1:172.21.15.152:6789/0 conn(0x7efd5006f340 legacy=0x7efd5015cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:24.409 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.408+0000 7efd4d400640 1 --2- 172.21.15.152:0/2724228070 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efd500b4c50 0x7efd34005c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:24.409 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.408+0000 7efd4d400640 1 --2- 172.21.15.152:0/2724228070 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efd34006170 0x7efd34006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:24.409 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.408+0000 7efd4d400640 1 --2- 172.21.15.152:0/2724228070 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efd34006f50 0x7efd340073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:24.409 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.408+0000 7efd4d400640 1 -- 172.21.15.152:0/2724228070 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7efd5006de10 con 0x7efd34006170 2024-04-20T11:33:24.409 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.408+0000 7efd4d400640 1 -- 172.21.15.152:0/2724228070 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7efd5006e060 con 0x7efd34006f50 2024-04-20T11:33:24.410 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.408+0000 7efd4d400640 1 -- 172.21.15.152:0/2724228070 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7efd3400ffd0 con 0x7efd500b4c50 2024-04-20T11:33:24.410 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.408+0000 7efd45400640 1 --2- 172.21.15.152:0/2724228070 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efd34006170 0x7efd34006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:24.410 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.408+0000 7efd45400640 1 -- 172.21.15.152:0/2724228070 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efd500b4c50 msgr2=0x7efd34005c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:24.410 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.408+0000 7efd45400640 1 --2- 172.21.15.152:0/2724228070 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efd500b4c50 0x7efd34005c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:24.410 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.408+0000 7efd45400640 1 -- 172.21.15.152:0/2724228070 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efd34006f50 msgr2=0x7efd340073d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:24.410 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.408+0000 7efd4e800640 1 --2- 172.21.15.152:0/2724228070 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efd34006f50 0x7efd340073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:24.410 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.408+0000 7efd45400640 1 --2- 172.21.15.152:0/2724228070 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efd34006f50 0x7efd340073d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:24.410 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.408+0000 7efd45400640 1 -- 172.21.15.152:0/2724228070 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd34005730 con 0x7efd34006170 2024-04-20T11:33:24.410 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.408+0000 7efd4e800640 1 --2- 172.21.15.152:0/2724228070 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efd34006f50 0x7efd340073d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:33:24.411 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.410+0000 7efd45400640 1 --2- 172.21.15.152:0/2724228070 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efd34006170 0x7efd34006a10 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7efd38009af0 tx=0x7efd34007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e595ddc40ddba7e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:24.412 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.410+0000 7efd4d400640 1 -- 172.21.15.152:0/2724228070 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd38039070 con 0x7efd34006170 2024-04-20T11:33:24.412 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.410+0000 7efd4d400640 1 -- 172.21.15.152:0/2724228070 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd3802c710 con 0x7efd34006170 2024-04-20T11:33:24.412 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.410+0000 7efd4d400640 1 -- 172.21.15.152:0/2724228070 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd38009690 con 0x7efd34006170 2024-04-20T11:33:24.412 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.410+0000 7efd54a00640 1 -- 172.21.15.152:0/2724228070 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efd34006170 msgr2=0x7efd34006a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:24.412 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.410+0000 7efd54a00640 1 --2- 172.21.15.152:0/2724228070 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efd34006170 0x7efd34006a10 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7efd38009af0 tx=0x7efd34007a40 comp rx=0 tx=0).stop 2024-04-20T11:33:24.412 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.410+0000 7efd4e800640 1 -- 172.21.15.152:0/2724228070 reap_dead start 2024-04-20T11:33:24.412 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.410+0000 7efd54a00640 1 -- 172.21.15.152:0/2724228070 shutdown_connections 2024-04-20T11:33:24.412 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.410+0000 7efd54a00640 1 -- 172.21.15.152:0/2724228070 >> 172.21.15.152:0/2724228070 conn(0x7efd50067cf0 msgr2=0x7efd5006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:24.412 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.411+0000 7efd54a00640 1 -- 172.21.15.152:0/2724228070 shutdown_connections 2024-04-20T11:33:24.412 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.411+0000 7efd54a00640 1 -- 172.21.15.152:0/2724228070 wait complete. 2024-04-20T11:33:24.412 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.412+0000 7efd54a00640 1 Processor -- start 2024-04-20T11:33:24.412 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.412+0000 7efd54a00640 1 -- start start 2024-04-20T11:33:24.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.413+0000 7efd54a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efd34006170 0x7efd50147250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:24.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.413+0000 7efd54a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efd5015d660 0x7efd5015dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:24.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.413+0000 7efd54a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efd501f36e0 0x7efd501f3b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:24.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.413+0000 7efd54a00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7efd500b42d0 con 0x7efd5015d660 2024-04-20T11:33:24.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.413+0000 7efd54a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7efd5015c9d0 con 0x7efd501f36e0 2024-04-20T11:33:24.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.413+0000 7efd54a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7efd500b50d0 con 0x7efd34006170 2024-04-20T11:33:24.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.414+0000 7efd45400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efd5015d660 0x7efd5015dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:24.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.414+0000 7efd45400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efd5015d660 0x7efd5015dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:37662/0 (socket says 172.21.15.152:37662) 2024-04-20T11:33:24.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.414+0000 7efd45400640 1 -- 172.21.15.152:0/2166919043 learned_addr learned my addr 172.21.15.152:0/2166919043 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:33:24.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.414+0000 7efd4de00640 1 --2- 172.21.15.152:0/2166919043 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efd34006170 0x7efd50147250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:24.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.414+0000 7efd45400640 1 -- 172.21.15.152:0/2166919043 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efd34006170 msgr2=0x7efd50147250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:24.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.414+0000 7efd45400640 1 --2- 172.21.15.152:0/2166919043 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efd34006170 0x7efd50147250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:24.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.414+0000 7efd45400640 1 -- 172.21.15.152:0/2166919043 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efd501f36e0 msgr2=0x7efd501f3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:24.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.414+0000 7efd45400640 1 --2- 172.21.15.152:0/2166919043 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efd501f36e0 0x7efd501f3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:24.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.414+0000 7efd45400640 1 -- 172.21.15.152:0/2166919043 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd50147790 con 0x7efd5015d660 2024-04-20T11:33:24.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.414+0000 7efd45400640 1 --2- 172.21.15.152:0/2166919043 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efd5015d660 0x7efd5015dae0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7efd38009af0 tx=0x7efd38004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:24.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.415+0000 7efd46a00640 1 -- 172.21.15.152:0/2166919043 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd38042020 con 0x7efd5015d660 2024-04-20T11:33:24.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.415+0000 7efd46a00640 1 -- 172.21.15.152:0/2166919043 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd3803a040 con 0x7efd5015d660 2024-04-20T11:33:24.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.415+0000 7efd46a00640 1 -- 172.21.15.152:0/2166919043 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd38039070 con 0x7efd5015d660 2024-04-20T11:33:24.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.415+0000 7efd54a00640 1 -- 172.21.15.152:0/2166919043 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd501f4450 con 0x7efd5015d660 2024-04-20T11:33:24.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.415+0000 7efd54a00640 1 -- 172.21.15.152:0/2166919043 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd50147d50 con 0x7efd5015d660 2024-04-20T11:33:24.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.416+0000 7efd54a00640 1 -- 172.21.15.152:0/2166919043 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd5015e020 con 0x7efd5015d660 2024-04-20T11:33:24.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.417+0000 7efd46a00640 1 -- 172.21.15.152:0/2166919043 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 91142+0+0 (secure 0 0 0) 0x7efd38045020 con 0x7efd5015d660 2024-04-20T11:33:24.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.418+0000 7efd46a00640 1 --2- 172.21.15.152:0/2166919043 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7efd2c06ca10 0x7efd2c06eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:24.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.418+0000 7efd46a00640 1 -- 172.21.15.152:0/2166919043 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 4857+0+0 (secure 0 0 0) 0x7efd380ad150 con 0x7efd5015d660 2024-04-20T11:33:24.418 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.419+0000 7efd4de00640 1 --2- 172.21.15.152:0/2166919043 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7efd2c06ca10 0x7efd2c06eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:24.419 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.420+0000 7efd4de00640 1 --2- 172.21.15.152:0/2166919043 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7efd2c06ca10 0x7efd2c06eed0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7efd400099f0 tx=0x7efd40008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:24.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.422+0000 7efd46a00640 1 -- 172.21.15.152:0/2166919043 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7efd3804c030 con 0x7efd5015d660 2024-04-20T11:33:24.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.622+0000 7efd54a00640 1 -- 172.21.15.152:0/2166919043 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7efd500689f0 con 0x7efd5015d660 2024-04-20T11:33:24.623 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.623+0000 7efd46a00640 1 -- 172.21.15.152:0/2166919043 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v99) ==== 84+0+65 (secure 0 0 0) 0x7efd50147790 con 0x7efd5015d660 2024-04-20T11:33:24.623 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:33:24.623 INFO:teuthology.orchestra.run.smithi152.stdout:[ 2024-04-20T11:33:24.623 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:33:24.623 INFO:teuthology.orchestra.run.smithi152.stdout: "poolnum": 1, 2024-04-20T11:33:24.623 INFO:teuthology.orchestra.run.smithi152.stdout: "poolname": ".mgr" 2024-04-20T11:33:24.624 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:33:24.624 INFO:teuthology.orchestra.run.smithi152.stdout:] 2024-04-20T11:33:24.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.625+0000 7efd54a00640 1 -- 172.21.15.152:0/2166919043 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7efd2c06ca10 msgr2=0x7efd2c06eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:24.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.625+0000 7efd54a00640 1 --2- 172.21.15.152:0/2166919043 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7efd2c06ca10 0x7efd2c06eed0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7efd400099f0 tx=0x7efd40008040 comp rx=0 tx=0).stop 2024-04-20T11:33:24.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.625+0000 7efd54a00640 1 -- 172.21.15.152:0/2166919043 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efd5015d660 msgr2=0x7efd5015dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:24.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.625+0000 7efd54a00640 1 --2- 172.21.15.152:0/2166919043 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efd5015d660 0x7efd5015dae0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7efd38009af0 tx=0x7efd38004730 comp rx=0 tx=0).stop 2024-04-20T11:33:24.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.625+0000 7efd54a00640 1 -- 172.21.15.152:0/2166919043 shutdown_connections 2024-04-20T11:33:24.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.625+0000 7efd54a00640 1 --2- 172.21.15.152:0/2166919043 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7efd2c06ca10 0x7efd2c06eed0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:24.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.625+0000 7efd54a00640 1 --2- 172.21.15.152:0/2166919043 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efd501f36e0 0x7efd501f3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:24.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.625+0000 7efd54a00640 1 --2- 172.21.15.152:0/2166919043 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efd5015d660 0x7efd5015dae0 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:24.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.625+0000 7efd54a00640 1 --2- 172.21.15.152:0/2166919043 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efd34006170 0x7efd50147250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:24.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.626+0000 7efd54a00640 1 -- 172.21.15.152:0/2166919043 >> 172.21.15.152:0/2166919043 conn(0x7efd50067cf0 msgr2=0x7efd5015e5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:24.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.626+0000 7efd54a00640 1 -- 172.21.15.152:0/2166919043 shutdown_connections 2024-04-20T11:33:24.627 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.626+0000 7efd54a00640 1 -- 172.21.15.152:0/2166919043 wait complete. 2024-04-20T11:33:24.644 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-20T11:33:24.644 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-20T11:33:24.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.847+0000 7f41c2000640 1 Processor -- start 2024-04-20T11:33:24.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.847+0000 7f41c2000640 1 -- start start 2024-04-20T11:33:24.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.849+0000 7f41c2000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f41bc0d05f0 0x7f41bc0d0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:24.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.849+0000 7f41c2000640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41bc15d8d0 con 0x7f41bc1569d0 2024-04-20T11:33:24.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.849+0000 7f41c2000640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41bc15d310 con 0x7f41bc0ad760 2024-04-20T11:33:24.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.849+0000 7f41c2000640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f41bc0d0f80 con 0x7f41bc0d05f0 2024-04-20T11:33:24.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.849+0000 7f41c0c00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f41bc1569d0 0x7f41bc06e510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:55234/0 (socket says 172.21.15.152:55234) 2024-04-20T11:33:24.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.849+0000 7f41c0c00640 1 -- 172.21.15.152:0/2094760938 learned_addr learned my addr 172.21.15.152:0/2094760938 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:33:24.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.849+0000 7f41c1600640 1 --2- 172.21.15.152:0/2094760938 >> v2:172.21.15.152:3300/0 conn(0x7f41bc0d05f0 0x7f41bc0d0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:24.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.849+0000 7f41bb400640 1 -- 172.21.15.152:0/2094760938 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1469722641 0 0) 0x7f41bc15d8d0 con 0x7f41bc1569d0 2024-04-20T11:33:24.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.849+0000 7f41bb400640 1 -- 172.21.15.152:0/2094760938 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f41a4003520 con 0x7f41bc1569d0 2024-04-20T11:33:24.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.850+0000 7f41c1600640 1 -- 172.21.15.152:0/2094760938 >> v1:172.21.15.175:6789/0 conn(0x7f41bc0ad760 legacy=0x7f41bc15ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:24.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.850+0000 7f41c1600640 1 -- 172.21.15.152:0/2094760938 >> v1:172.21.15.152:6789/0 conn(0x7f41bc1569d0 legacy=0x7f41bc06e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:24.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.850+0000 7f41c1600640 1 -- 172.21.15.152:0/2094760938 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41bc15d5f0 con 0x7f41bc0d05f0 2024-04-20T11:33:24.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.851+0000 7f41c1600640 1 --2- 172.21.15.152:0/2094760938 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f41bc0d05f0 0x7f41bc0d0a40 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f41b4009d90 tx=0x7f41b402bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6005113dd94b1f03 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:24.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.851+0000 7f41bb400640 1 -- 172.21.15.152:0/2094760938 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41b4039070 con 0x7f41bc0d05f0 2024-04-20T11:33:24.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.851+0000 7f41bb400640 1 -- 172.21.15.152:0/2094760938 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41b402cd00 con 0x7f41bc0d05f0 2024-04-20T11:33:24.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.851+0000 7f41bb400640 1 -- 172.21.15.152:0/2094760938 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41b4034640 con 0x7f41bc0d05f0 2024-04-20T11:33:24.852 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.852+0000 7f41c2000640 1 -- 172.21.15.152:0/2094760938 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f41bc0d05f0 msgr2=0x7f41bc0d0a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:24.852 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.852+0000 7f41c2000640 1 --2- 172.21.15.152:0/2094760938 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f41bc0d05f0 0x7f41bc0d0a40 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f41b4009d90 tx=0x7f41b402bf50 comp rx=0 tx=0).stop 2024-04-20T11:33:24.852 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.852+0000 7f41c2000640 1 -- 172.21.15.152:0/2094760938 shutdown_connections 2024-04-20T11:33:24.852 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.852+0000 7f41c2000640 1 --2- 172.21.15.152:0/2094760938 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f41bc0d05f0 0x7f41bc0d0a40 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:24.852 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.852+0000 7f41c2000640 1 -- 172.21.15.152:0/2094760938 >> 172.21.15.152:0/2094760938 conn(0x7f41bc0b21e0 msgr2=0x7f41bc0b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:24.852 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.852+0000 7f41c2000640 1 -- 172.21.15.152:0/2094760938 shutdown_connections 2024-04-20T11:33:24.852 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.852+0000 7f41c2000640 1 -- 172.21.15.152:0/2094760938 wait complete. 2024-04-20T11:33:24.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.854+0000 7f41c2000640 1 Processor -- start 2024-04-20T11:33:24.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.854+0000 7f41c2000640 1 -- start start 2024-04-20T11:33:24.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.855+0000 7f41c2000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f41bc0ad760 0x7f41bc1f7560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:24.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.855+0000 7f41c2000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f41bc0d05f0 0x7f41bc159160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:24.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.855+0000 7f41c2000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41bc1569d0 0x7f41bc1596a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:24.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.855+0000 7f41c2000640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f41bc156db0 con 0x7f41bc0ad760 2024-04-20T11:33:24.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.855+0000 7f41c2000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f41bc06dff0 con 0x7f41bc1569d0 2024-04-20T11:33:24.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.855+0000 7f41c2000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f41bc0b30c0 con 0x7f41bc0d05f0 2024-04-20T11:33:24.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.855+0000 7f41c0c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f41bc0ad760 0x7f41bc1f7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:24.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.855+0000 7f41c0c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f41bc0ad760 0x7f41bc1f7560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:37686/0 (socket says 172.21.15.152:37686) 2024-04-20T11:33:24.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.855+0000 7f41c0c00640 1 -- 172.21.15.152:0/4081476312 learned_addr learned my addr 172.21.15.152:0/4081476312 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:33:24.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.855+0000 7f41c0c00640 1 -- 172.21.15.152:0/4081476312 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f41bc0d05f0 msgr2=0x7f41bc159160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:24.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.855+0000 7f41c0c00640 1 --2- 172.21.15.152:0/4081476312 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f41bc0d05f0 0x7f41bc159160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:24.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.855+0000 7f41c0c00640 1 -- 172.21.15.152:0/4081476312 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41bc1569d0 msgr2=0x7f41bc1596a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:24.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.855+0000 7f41c0c00640 1 --2- 172.21.15.152:0/4081476312 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41bc1569d0 0x7f41bc1596a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:24.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.855+0000 7f41c0c00640 1 -- 172.21.15.152:0/4081476312 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41bc154370 con 0x7f41bc0ad760 2024-04-20T11:33:24.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.856+0000 7f41c0c00640 1 --2- 172.21.15.152:0/4081476312 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f41bc0ad760 0x7f41bc1f7560 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f41b000c440 tx=0x7f41b000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:24.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.856+0000 7f41b8c00640 1 -- 172.21.15.152:0/4081476312 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41b0017070 con 0x7f41bc0ad760 2024-04-20T11:33:24.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.856+0000 7f41b8c00640 1 -- 172.21.15.152:0/4081476312 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41b0005ce0 con 0x7f41bc0ad760 2024-04-20T11:33:24.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.856+0000 7f41b8c00640 1 -- 172.21.15.152:0/4081476312 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41b00072b0 con 0x7f41bc0ad760 2024-04-20T11:33:24.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.856+0000 7f41c2000640 1 -- 172.21.15.152:0/4081476312 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41bc155340 con 0x7f41bc0ad760 2024-04-20T11:33:24.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.856+0000 7f41c2000640 1 -- 172.21.15.152:0/4081476312 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41bc159d70 con 0x7f41bc0ad760 2024-04-20T11:33:24.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.857+0000 7f41c2000640 1 -- 172.21.15.152:0/4081476312 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41bc075e60 con 0x7f41bc0ad760 2024-04-20T11:33:24.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.858+0000 7f41b8c00640 1 -- 172.21.15.152:0/4081476312 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 91142+0+0 (secure 0 0 0) 0x7f41b001a070 con 0x7f41bc0ad760 2024-04-20T11:33:24.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.859+0000 7f41b8c00640 1 --2- 172.21.15.152:0/4081476312 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f419c06ca10 0x7f419c06eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:24.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.859+0000 7f41b8c00640 1 -- 172.21.15.152:0/4081476312 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 4857+0+0 (secure 0 0 0) 0x7f41b008acd0 con 0x7f41bc0ad760 2024-04-20T11:33:24.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.860+0000 7f41bbe00640 1 --2- 172.21.15.152:0/4081476312 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f419c06ca10 0x7f419c06eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:24.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.861+0000 7f41bbe00640 1 --2- 172.21.15.152:0/4081476312 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f419c06ca10 0x7f419c06eed0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f41a8000f60 tx=0x7f41a80079b0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:24.863 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:24.864+0000 7f41b8c00640 1 -- 172.21.15.152:0/4081476312 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f41b0059470 con 0x7f41bc0ad760 2024-04-20T11:33:25.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:25.046+0000 7f41c2000640 1 -- 172.21.15.152:0/4081476312 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f41bc156540 con 0x7f41bc0ad760 2024-04-20T11:33:25.791 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:25.791+0000 7f41b8c00640 1 -- 172.21.15.152:0/4081476312 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v100) ==== 138+0+0 (secure 0 0 0) 0x7f41b0058d70 con 0x7f41bc0ad760 2024-04-20T11:33:25.791 INFO:teuthology.orchestra.run.smithi152.stderr:pool 'cephfs_metadata' created 2024-04-20T11:33:25.795 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:25.795+0000 7f41c2000640 1 -- 172.21.15.152:0/4081476312 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f419c06ca10 msgr2=0x7f419c06eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:25.795 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:25.795+0000 7f41c2000640 1 --2- 172.21.15.152:0/4081476312 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f419c06ca10 0x7f419c06eed0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f41a8000f60 tx=0x7f41a80079b0 comp rx=0 tx=0).stop 2024-04-20T11:33:25.795 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:25.795+0000 7f41c2000640 1 -- 172.21.15.152:0/4081476312 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f41bc0ad760 msgr2=0x7f41bc1f7560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:25.795 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:25.795+0000 7f41c2000640 1 --2- 172.21.15.152:0/4081476312 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f41bc0ad760 0x7f41bc1f7560 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f41b000c440 tx=0x7f41b000c900 comp rx=0 tx=0).stop 2024-04-20T11:33:25.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:25.796+0000 7f41c2000640 1 -- 172.21.15.152:0/4081476312 shutdown_connections 2024-04-20T11:33:25.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:25.796+0000 7f41c2000640 1 --2- 172.21.15.152:0/4081476312 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f419c06ca10 0x7f419c06eed0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:25.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:25.796+0000 7f41c2000640 1 --2- 172.21.15.152:0/4081476312 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41bc1569d0 0x7f41bc1596a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:25.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:25.796+0000 7f41c2000640 1 --2- 172.21.15.152:0/4081476312 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f41bc0d05f0 0x7f41bc159160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:25.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:25.796+0000 7f41c2000640 1 --2- 172.21.15.152:0/4081476312 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f41bc0ad760 0x7f41bc1f7560 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:25.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:25.796+0000 7f41c2000640 1 -- 172.21.15.152:0/4081476312 >> 172.21.15.152:0/4081476312 conn(0x7f41bc0b21e0 msgr2=0x7f41bc0b25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:25.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:25.796+0000 7f41c2000640 1 -- 172.21.15.152:0/4081476312 shutdown_connections 2024-04-20T11:33:25.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:25.796+0000 7f41c2000640 1 -- 172.21.15.152:0/4081476312 wait complete. 2024-04-20T11:33:25.823 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-20T11:33:26.015 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:26.016+0000 7fcc0b000640 1 Processor -- start 2024-04-20T11:33:26.016 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:26.016+0000 7fcc0b000640 1 -- start start 2024-04-20T11:33:26.017 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:26.017+0000 7fcc0b000640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fcc040b4fb0 0x7fcc0415fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:26.017 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:26.018+0000 7fcc0b000640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc04160080 con 0x7fcc040b5390 2024-04-20T11:33:26.017 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:26.018+0000 7fcc0b000640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc0406ec70 con 0x7fcc0406dc60 2024-04-20T11:33:26.017 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:26.018+0000 7fcc0b000640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fcc0406fad0 con 0x7fcc040b4fb0 2024-04-20T11:33:26.017 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:26.018+0000 7fcc09000640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fcc040b5390 0x7fcc040b3d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:55066/0 (socket says 172.21.15.152:55066) 2024-04-20T11:33:26.018 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:26.018+0000 7fcc09000640 1 -- 172.21.15.152:0/3332864407 learned_addr learned my addr 172.21.15.152:0/3332864407 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:33:26.019 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:26.018+0000 7fcc03e00640 1 --2- 172.21.15.152:0/3332864407 >> v2:172.21.15.175:3300/0 conn(0x7fcc040b4fb0 0x7fcc0415fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:26.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:26.018+0000 7fcc02a00640 1 -- 172.21.15.152:0/3332864407 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4085094641 0 0) 0x7fcc04160080 con 0x7fcc040b5390 2024-04-20T11:33:26.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:26.018+0000 7fcc02a00640 1 -- 172.21.15.152:0/3332864407 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcbe8005730 con 0x7fcc040b5390 2024-04-20T11:33:26.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:26.019+0000 7fcc02a00640 1 -- 172.21.15.152:0/3332864407 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2232818944 0 0) 0x7fcbe8005730 con 0x7fcc040b5390 2024-04-20T11:33:26.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:26.019+0000 7fcc02a00640 1 -- 172.21.15.152:0/3332864407 >> v2:172.21.15.175:3300/0 conn(0x7fcc040b4fb0 msgr2=0x7fcc0415fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:26.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:26.019+0000 7fcc02a00640 1 --2- 172.21.15.152:0/3332864407 >> v2:172.21.15.175:3300/0 conn(0x7fcc040b4fb0 0x7fcc0415fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:26.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:26.019+0000 7fcc02a00640 1 -- 172.21.15.152:0/3332864407 >> v1:172.21.15.175:6789/0 conn(0x7fcc0406dc60 legacy=0x7fcc040b3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:26.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:26.019+0000 7fcc02a00640 1 -- 172.21.15.152:0/3332864407 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc0406e0f0 con 0x7fcc040b5390 2024-04-20T11:33:26.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:26.019+0000 7fcc02a00640 1 -- 172.21.15.152:0/3332864407 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7fcbfc001a10 con 0x7fcc040b5390 2024-04-20T11:33:26.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:26.019+0000 7fcc02a00640 1 -- 172.21.15.152:0/3332864407 >> v1:172.21.15.152:6789/0 conn(0x7fcc040b5390 legacy=0x7fcc040b3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:26.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:26.019+0000 7fcc02a00640 1 --2- 172.21.15.152:0/3332864407 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcbe8005bc0 0x7fcbe8005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:26.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:26.019+0000 7fcc02a00640 1 --2- 172.21.15.152:0/3332864407 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcbe8006a10 0x7fcbe8006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:26.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:26.019+0000 7fcc02a00640 1 --2- 172.21.15.152:0/3332864407 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcbe80073d0 0x7fcbe800dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:26.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:26.019+0000 7fcc02a00640 1 -- 172.21.15.152:0/3332864407 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fcc0406fad0 con 0x7fcbe8005bc0 2024-04-20T11:33:26.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:26.019+0000 7fcc02a00640 1 -- 172.21.15.152:0/3332864407 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcbe800e370 con 0x7fcbe80073d0 2024-04-20T11:33:26.021 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:26.019+0000 7fcc02a00640 1 -- 172.21.15.152:0/3332864407 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fcbe800e570 con 0x7fcbe8006a10 2024-04-20T11:33:26.021 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:26.019+0000 7fcc03e00640 1 --2- 172.21.15.152:0/3332864407 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcbe8005bc0 0x7fcbe8005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:26.021 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:26.020+0000 7fcc03400640 1 --2- 172.21.15.152:0/3332864407 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcbe8006a10 0x7fcbe8006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:26.021 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:26.020+0000 7fcc03e00640 1 -- 172.21.15.152:0/3332864407 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcbe8006a10 msgr2=0x7fcbe8006e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:26.021 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:26.020+0000 7fcc03e00640 1 --2- 172.21.15.152:0/3332864407 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcbe8006a10 0x7fcbe8006e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:26.021 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:26.020+0000 7fcc03e00640 1 -- 172.21.15.152:0/3332864407 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcbe80073d0 msgr2=0x7fcbe800dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:26.021 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:26.020+0000 7fcc09000640 1 --2- 172.21.15.152:0/3332864407 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcbe80073d0 0x7fcbe800dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:26.021 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:26.020+0000 7fcc03e00640 1 --2- 172.21.15.152:0/3332864407 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcbe80073d0 0x7fcbe800dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:26.021 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:26.020+0000 7fcc03e00640 1 -- 172.21.15.152:0/3332864407 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbe8005730 con 0x7fcbe8005bc0 2024-04-20T11:33:26.021 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:26.020+0000 7fcc09000640 1 -- 172.21.15.152:0/3332864407 reap_dead start 2024-04-20T11:33:26.022 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:26.021+0000 7fcc03e00640 1 --2- 172.21.15.152:0/3332864407 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcbe8005bc0 0x7fcbe8005fe0 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fcbf4009ac0 tx=0x7fcbe8007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ef1be36a6398f7b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:26.022 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:26.021+0000 7fcc02a00640 1 -- 172.21.15.152:0/3332864407 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf4039070 con 0x7fcbe8005bc0 2024-04-20T11:33:26.022 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:26.021+0000 7fcc02a00640 1 -- 172.21.15.152:0/3332864407 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbf402c6b0 con 0x7fcbe8005bc0 2024-04-20T11:33:26.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:26.021+0000 7fcc02a00640 1 -- 172.21.15.152:0/3332864407 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf4009690 con 0x7fcbe8005bc0 2024-04-20T11:33:26.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:26.022+0000 7fcc0b000640 1 -- 172.21.15.152:0/3332864407 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcbe8005bc0 msgr2=0x7fcbe8005fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:26.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:26.022+0000 7fcc0b000640 1 --2- 172.21.15.152:0/3332864407 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcbe8005bc0 0x7fcbe8005fe0 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fcbf4009ac0 tx=0x7fcbe8007a40 comp rx=0 tx=0).stop 2024-04-20T11:33:26.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:26.022+0000 7fcc0b000640 1 -- 172.21.15.152:0/3332864407 shutdown_connections 2024-04-20T11:33:26.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:26.022+0000 7fcc0b000640 1 --2- 172.21.15.152:0/3332864407 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcbe8005bc0 0x7fcbe8005fe0 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:26.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:26.022+0000 7fcc0b000640 1 -- 172.21.15.152:0/3332864407 >> 172.21.15.152:0/3332864407 conn(0x7fcc0406b980 msgr2=0x7fcc0406bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:26.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:26.022+0000 7fcc0b000640 1 -- 172.21.15.152:0/3332864407 shutdown_connections 2024-04-20T11:33:26.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:26.022+0000 7fcc0b000640 1 -- 172.21.15.152:0/3332864407 wait complete. 2024-04-20T11:33:26.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:26.024+0000 7fcc0b000640 1 Processor -- start 2024-04-20T11:33:26.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:26.024+0000 7fcc0b000640 1 -- start start 2024-04-20T11:33:26.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:26.025+0000 7fcc0b000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcbe8005bc0 0x7fcc041e4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:26.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:26.025+0000 7fcc0b000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcc04145e60 0x7fcc0415be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:26.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:26.025+0000 7fcc0b000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcc04159b50 0x7fcc04159fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:26.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:26.025+0000 7fcc0b000640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fcc0406e990 con 0x7fcc04145e60 2024-04-20T11:33:26.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:26.025+0000 7fcc0b000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcc0406e6b0 con 0x7fcbe8005bc0 2024-04-20T11:33:26.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:26.025+0000 7fcc0b000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fcc0415fb40 con 0x7fcc04159b50 2024-04-20T11:33:26.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:26.025+0000 7fcc03400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcc04145e60 0x7fcc0415be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:26.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:26.025+0000 7fcc03400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcc04145e60 0x7fcc0415be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54148/0 (socket says 172.21.15.152:54148) 2024-04-20T11:33:26.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:26.025+0000 7fcc03400640 1 -- 172.21.15.152:0/1754016094 learned_addr learned my addr 172.21.15.152:0/1754016094 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:33:26.026 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:26.026+0000 7fcc09000640 1 --2- 172.21.15.152:0/1754016094 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcc04159b50 0x7fcc04159fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:26.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:26.026+0000 7fcc03e00640 1 --2- 172.21.15.152:0/1754016094 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcbe8005bc0 0x7fcc041e4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:26.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:26.026+0000 7fcc03400640 1 -- 172.21.15.152:0/1754016094 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcc04159b50 msgr2=0x7fcc04159fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:26.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:26.026+0000 7fcc03400640 1 --2- 172.21.15.152:0/1754016094 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcc04159b50 0x7fcc04159fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:26.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:26.026+0000 7fcc03400640 1 -- 172.21.15.152:0/1754016094 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcbe8005bc0 msgr2=0x7fcc041e4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:26.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:26.026+0000 7fcc03400640 1 --2- 172.21.15.152:0/1754016094 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcbe8005bc0 0x7fcc041e4f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:26.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:26.026+0000 7fcc03400640 1 -- 172.21.15.152:0/1754016094 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc0415c380 con 0x7fcc04145e60 2024-04-20T11:33:26.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:26.026+0000 7fcc03e00640 1 --2- 172.21.15.152:0/1754016094 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcbe8005bc0 0x7fcc041e4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:33:26.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:26.026+0000 7fcc03400640 1 --2- 172.21.15.152:0/1754016094 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcc04145e60 0x7fcc0415be40 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fcbec00e490 tx=0x7fcbec00e950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:26.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:26.026+0000 7fcbfbe00640 1 -- 172.21.15.152:0/1754016094 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbec0052e0 con 0x7fcc04145e60 2024-04-20T11:33:26.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:26.026+0000 7fcbfbe00640 1 -- 172.21.15.152:0/1754016094 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbec018070 con 0x7fcc04145e60 2024-04-20T11:33:26.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:26.026+0000 7fcbfbe00640 1 -- 172.21.15.152:0/1754016094 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbec003830 con 0x7fcc04145e60 2024-04-20T11:33:26.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:26.026+0000 7fcc0b000640 1 -- 172.21.15.152:0/1754016094 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc041e5600 con 0x7fcc04145e60 2024-04-20T11:33:26.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:26.027+0000 7fcc0b000640 1 -- 172.21.15.152:0/1754016094 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc041e5bc0 con 0x7fcc04145e60 2024-04-20T11:33:26.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:26.028+0000 7fcc0b000640 1 -- 172.21.15.152:0/1754016094 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc040689f0 con 0x7fcc04145e60 2024-04-20T11:33:26.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:26.029+0000 7fcbfbe00640 1 -- 172.21.15.152:0/1754016094 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 91142+0+0 (secure 0 0 0) 0x7fcbec0039d0 con 0x7fcc04145e60 2024-04-20T11:33:26.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:26.030+0000 7fcbfbe00640 1 --2- 172.21.15.152:0/1754016094 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fcbe006ca10 0x7fcbe006eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:26.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:26.030+0000 7fcbfbe00640 1 -- 172.21.15.152:0/1754016094 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5227+0+0 (secure 0 0 0) 0x7fcbec08b710 con 0x7fcc04145e60 2024-04-20T11:33:26.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:26.031+0000 7fcc03e00640 1 --2- 172.21.15.152:0/1754016094 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fcbe006ca10 0x7fcbe006eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:26.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:26.032+0000 7fcc03e00640 1 --2- 172.21.15.152:0/1754016094 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fcbe006ca10 0x7fcbe006eed0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fcbf4036000 tx=0x7fcbf4030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:26.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:26.035+0000 7fcbfbe00640 1 -- 172.21.15.152:0/1754016094 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fcbec059c00 con 0x7fcc04145e60 2024-04-20T11:33:26.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:26.242+0000 7fcc0b000640 1 -- 172.21.15.152:0/1754016094 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fcc0406ee90 con 0x7fcc04145e60 2024-04-20T11:33:26.809 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:26.808+0000 7fcbfbe00640 1 -- 172.21.15.152:0/1754016094 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v101) ==== 170+0+0 (secure 0 0 0) 0x7fcbec059500 con 0x7fcc04145e60 2024-04-20T11:33:26.810 INFO:teuthology.orchestra.run.smithi152.stderr:pool 'cephfs_data' created 2024-04-20T11:33:26.812 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:26.812+0000 7fcc0b000640 1 -- 172.21.15.152:0/1754016094 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fcbe006ca10 msgr2=0x7fcbe006eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:26.812 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:26.812+0000 7fcc0b000640 1 --2- 172.21.15.152:0/1754016094 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fcbe006ca10 0x7fcbe006eed0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fcbf4036000 tx=0x7fcbf4030000 comp rx=0 tx=0).stop 2024-04-20T11:33:26.812 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:26.812+0000 7fcc0b000640 1 -- 172.21.15.152:0/1754016094 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcc04145e60 msgr2=0x7fcc0415be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:26.812 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:26.812+0000 7fcc0b000640 1 --2- 172.21.15.152:0/1754016094 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcc04145e60 0x7fcc0415be40 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fcbec00e490 tx=0x7fcbec00e950 comp rx=0 tx=0).stop 2024-04-20T11:33:26.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:26.813+0000 7fcc0b000640 1 -- 172.21.15.152:0/1754016094 shutdown_connections 2024-04-20T11:33:26.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:26.813+0000 7fcc0b000640 1 --2- 172.21.15.152:0/1754016094 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fcbe006ca10 0x7fcbe006eed0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:26.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:26.813+0000 7fcc0b000640 1 --2- 172.21.15.152:0/1754016094 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcc04159b50 0x7fcc04159fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:26.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:26.813+0000 7fcc0b000640 1 --2- 172.21.15.152:0/1754016094 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcc04145e60 0x7fcc0415be40 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:26.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:26.813+0000 7fcc0b000640 1 --2- 172.21.15.152:0/1754016094 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcbe8005bc0 0x7fcc041e4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:26.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:26.813+0000 7fcc0b000640 1 -- 172.21.15.152:0/1754016094 >> 172.21.15.152:0/1754016094 conn(0x7fcc0406b980 msgr2=0x7fcc0415f3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:26.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:26.813+0000 7fcc0b000640 1 -- 172.21.15.152:0/1754016094 shutdown_connections 2024-04-20T11:33:26.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:26.813+0000 7fcc0b000640 1 -- 172.21.15.152:0/1754016094 wait complete. 2024-04-20T11:33:26.849 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-20T11:33:27.026 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:27.027+0000 7f6e18600640 1 Processor -- start 2024-04-20T11:33:27.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:27.027+0000 7f6e18600640 1 -- start start 2024-04-20T11:33:27.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:27.028+0000 7f6e18600640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f6e10146000 0x7f6e100b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:27.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:27.029+0000 7f6e18600640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e101463e0 con 0x7f6e100cd8c0 2024-04-20T11:33:27.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:27.029+0000 7f6e18600640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e100b3e20 con 0x7f6e100b2170 2024-04-20T11:33:27.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:27.029+0000 7f6e18600640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f6e100cdca0 con 0x7f6e10146000 2024-04-20T11:33:27.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:27.029+0000 7f6e16600640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f6e100cd8c0 0x7f6e1015ccf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:55072/0 (socket says 172.21.15.152:55072) 2024-04-20T11:33:27.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:27.029+0000 7f6e16600640 1 -- 172.21.15.152:0/3018526753 learned_addr learned my addr 172.21.15.152:0/3018526753 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:33:27.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:27.029+0000 7f6e0fe00640 1 -- 172.21.15.152:0/3018526753 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2037433049 0 0) 0x7f6e101463e0 con 0x7f6e100cd8c0 2024-04-20T11:33:27.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:27.029+0000 7f6e0fe00640 1 -- 172.21.15.152:0/3018526753 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6df8003520 con 0x7f6e100cd8c0 2024-04-20T11:33:27.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:27.029+0000 7f6e15c00640 1 --2- 172.21.15.152:0/3018526753 >> v2:172.21.15.175:3300/0 conn(0x7f6e10146000 0x7f6e100b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:27.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:27.030+0000 7f6e0fe00640 1 -- 172.21.15.152:0/3018526753 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3367032735 0 0) 0x7f6df8003520 con 0x7f6e100cd8c0 2024-04-20T11:33:27.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:27.030+0000 7f6e0fe00640 1 -- 172.21.15.152:0/3018526753 >> v2:172.21.15.175:3300/0 conn(0x7f6e10146000 msgr2=0x7f6e100b4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:27.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:27.030+0000 7f6e0fe00640 1 --2- 172.21.15.152:0/3018526753 >> v2:172.21.15.175:3300/0 conn(0x7f6e10146000 0x7f6e100b4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:27.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:27.030+0000 7f6e0fe00640 1 -- 172.21.15.152:0/3018526753 >> v1:172.21.15.175:6789/0 conn(0x7f6e100b2170 legacy=0x7f6e100cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:27.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:27.030+0000 7f6e0fe00640 1 -- 172.21.15.152:0/3018526753 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e100b4100 con 0x7f6e100cd8c0 2024-04-20T11:33:27.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:27.030+0000 7f6e0fe00640 1 -- 172.21.15.152:0/3018526753 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f6e080019f0 con 0x7f6e100cd8c0 2024-04-20T11:33:27.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:27.030+0000 7f6e0fe00640 1 -- 172.21.15.152:0/3018526753 >> v1:172.21.15.152:6789/0 conn(0x7f6e100cd8c0 legacy=0x7f6e1015ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:27.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:27.030+0000 7f6e0fe00640 1 --2- 172.21.15.152:0/3018526753 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6df80039b0 0x7f6df8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:27.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:27.030+0000 7f6e0fe00640 1 --2- 172.21.15.152:0/3018526753 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6df8004800 0x7f6df8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:27.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:27.030+0000 7f6e0fe00640 1 --2- 172.21.15.152:0/3018526753 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6df80052e0 0x7f6df800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:27.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:27.030+0000 7f6e0fe00640 1 -- 172.21.15.152:0/3018526753 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f6e100cdca0 con 0x7f6df80052e0 2024-04-20T11:33:27.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:27.030+0000 7f6e0fe00640 1 -- 172.21.15.152:0/3018526753 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6df800c1b0 con 0x7f6df80039b0 2024-04-20T11:33:27.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:27.030+0000 7f6e0fe00640 1 -- 172.21.15.152:0/3018526753 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6df800c3b0 con 0x7f6df8004800 2024-04-20T11:33:27.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:27.030+0000 7f6e16600640 1 --2- 172.21.15.152:0/3018526753 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6df80052e0 0x7f6df800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:27.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:27.031+0000 7f6e15c00640 1 --2- 172.21.15.152:0/3018526753 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6df80039b0 0x7f6df8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:27.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:27.031+0000 7f6e15200640 1 --2- 172.21.15.152:0/3018526753 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6df8004800 0x7f6df8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:27.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:27.031+0000 7f6e16600640 1 -- 172.21.15.152:0/3018526753 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6df8004800 msgr2=0x7f6df8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:27.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:27.031+0000 7f6e16600640 1 --2- 172.21.15.152:0/3018526753 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6df8004800 0x7f6df8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:27.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:27.031+0000 7f6e16600640 1 -- 172.21.15.152:0/3018526753 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6df80039b0 msgr2=0x7f6df8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:27.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:27.031+0000 7f6e16600640 1 --2- 172.21.15.152:0/3018526753 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6df80039b0 0x7f6df8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:27.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:27.031+0000 7f6e16600640 1 -- 172.21.15.152:0/3018526753 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6df8003520 con 0x7f6df80052e0 2024-04-20T11:33:27.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:27.031+0000 7f6e15c00640 1 --2- 172.21.15.152:0/3018526753 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6df80039b0 0x7f6df8003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:33:27.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:27.032+0000 7f6e16600640 1 -- 172.21.15.152:0/3018526753 reap_dead start 2024-04-20T11:33:27.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:27.032+0000 7f6e16600640 1 --2- 172.21.15.152:0/3018526753 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6df80052e0 0x7f6df800b9c0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f6e0800a810 tx=0x7f6df8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4deb5283f140bcae server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:27.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:27.032+0000 7f6e0fe00640 1 -- 172.21.15.152:0/3018526753 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e0800a460 con 0x7f6df80052e0 2024-04-20T11:33:27.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:27.032+0000 7f6e0fe00640 1 -- 172.21.15.152:0/3018526753 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e08034650 con 0x7f6df80052e0 2024-04-20T11:33:27.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:27.032+0000 7f6e0fe00640 1 -- 172.21.15.152:0/3018526753 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e08034a40 con 0x7f6df80052e0 2024-04-20T11:33:27.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:27.033+0000 7f6e18600640 1 -- 172.21.15.152:0/3018526753 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6df80052e0 msgr2=0x7f6df800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:27.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:27.033+0000 7f6e18600640 1 --2- 172.21.15.152:0/3018526753 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6df80052e0 0x7f6df800b9c0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f6e0800a810 tx=0x7f6df8005830 comp rx=0 tx=0).stop 2024-04-20T11:33:27.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:27.033+0000 7f6e18600640 1 -- 172.21.15.152:0/3018526753 shutdown_connections 2024-04-20T11:33:27.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:27.033+0000 7f6e18600640 1 --2- 172.21.15.152:0/3018526753 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6df80052e0 0x7f6df800b9c0 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:27.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:27.033+0000 7f6e18600640 1 -- 172.21.15.152:0/3018526753 >> 172.21.15.152:0/3018526753 conn(0x7f6e1006b980 msgr2=0x7f6e1006bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:27.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:27.033+0000 7f6e18600640 1 -- 172.21.15.152:0/3018526753 shutdown_connections 2024-04-20T11:33:27.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:27.033+0000 7f6e18600640 1 -- 172.21.15.152:0/3018526753 wait complete. 2024-04-20T11:33:27.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:27.035+0000 7f6e18600640 1 Processor -- start 2024-04-20T11:33:27.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:27.035+0000 7f6e18600640 1 -- start start 2024-04-20T11:33:27.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:27.036+0000 7f6e18600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6df80052e0 0x7f6e1013f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:27.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:27.036+0000 7f6e18600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e1006eb90 0x7f6e1006f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:27.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:27.036+0000 7f6e18600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e100a9580 0x7f6e100a9a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:27.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:27.036+0000 7f6e18600640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f6e100b4be0 con 0x7f6df80052e0 2024-04-20T11:33:27.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:27.036+0000 7f6e18600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6e100b4750 con 0x7f6e100a9580 2024-04-20T11:33:27.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:27.036+0000 7f6e18600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6e100b4d60 con 0x7f6e1006eb90 2024-04-20T11:33:27.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:27.036+0000 7f6e15c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6df80052e0 0x7f6e1013f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:27.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:27.036+0000 7f6e15c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6df80052e0 0x7f6e1013f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54160/0 (socket says 172.21.15.152:54160) 2024-04-20T11:33:27.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:27.036+0000 7f6e15c00640 1 -- 172.21.15.152:0/1981833013 learned_addr learned my addr 172.21.15.152:0/1981833013 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:33:27.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:27.037+0000 7f6e15200640 1 --2- 172.21.15.152:0/1981833013 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e1006eb90 0x7f6e1006f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:27.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:27.037+0000 7f6e16600640 1 --2- 172.21.15.152:0/1981833013 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e100a9580 0x7f6e100a9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:27.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:27.037+0000 7f6e15c00640 1 -- 172.21.15.152:0/1981833013 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e1006eb90 msgr2=0x7f6e1006f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:27.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:27.037+0000 7f6e15c00640 1 --2- 172.21.15.152:0/1981833013 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e1006eb90 0x7f6e1006f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:27.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:27.037+0000 7f6e15c00640 1 -- 172.21.15.152:0/1981833013 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e100a9580 msgr2=0x7f6e100a9a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:27.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:27.037+0000 7f6e15c00640 1 --2- 172.21.15.152:0/1981833013 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e100a9580 0x7f6e100a9a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:27.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:27.037+0000 7f6e15c00640 1 -- 172.21.15.152:0/1981833013 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e100aa2f0 con 0x7f6df80052e0 2024-04-20T11:33:27.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:27.037+0000 7f6e15c00640 1 --2- 172.21.15.152:0/1981833013 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6df80052e0 0x7f6e1013f6e0 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f6e0400f580 tx=0x7f6e0400fa40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:27.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:27.038+0000 7f6e0d600640 1 -- 172.21.15.152:0/1981833013 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e04013810 con 0x7f6df80052e0 2024-04-20T11:33:27.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:27.038+0000 7f6e0d600640 1 -- 172.21.15.152:0/1981833013 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e0401a070 con 0x7f6df80052e0 2024-04-20T11:33:27.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:27.038+0000 7f6e0d600640 1 -- 172.21.15.152:0/1981833013 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e04013bb0 con 0x7f6df80052e0 2024-04-20T11:33:27.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:27.038+0000 7f6e18600640 1 -- 172.21.15.152:0/1981833013 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e10140760 con 0x7f6df80052e0 2024-04-20T11:33:27.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:27.038+0000 7f6e18600640 1 -- 172.21.15.152:0/1981833013 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e10140d20 con 0x7f6df80052e0 2024-04-20T11:33:27.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:27.039+0000 7f6e18600640 1 -- 172.21.15.152:0/1981833013 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e100689f0 con 0x7f6df80052e0 2024-04-20T11:33:27.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:27.041+0000 7f6e0d600640 1 -- 172.21.15.152:0/1981833013 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 91142+0+0 (secure 0 0 0) 0x7f6e04012070 con 0x7f6df80052e0 2024-04-20T11:33:27.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:27.041+0000 7f6e0d600640 1 --2- 172.21.15.152:0/1981833013 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f6df006ca10 0x7f6df006eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:27.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:27.042+0000 7f6e0d600640 1 -- 172.21.15.152:0/1981833013 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5609+0+0 (secure 0 0 0) 0x7f6e0408c7d0 con 0x7f6df80052e0 2024-04-20T11:33:27.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:27.042+0000 7f6e15200640 1 --2- 172.21.15.152:0/1981833013 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f6df006ca10 0x7f6df006eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:27.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:27.044+0000 7f6e15200640 1 --2- 172.21.15.152:0/1981833013 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f6df006ca10 0x7f6df006eed0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc007fd0 tx=0x7f6dfc009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:27.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:27.045+0000 7f6e0d600640 1 -- 172.21.15.152:0/1981833013 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f6e0401d360 con 0x7f6df80052e0 2024-04-20T11:33:27.276 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:27.276+0000 7f6e18600640 1 -- 172.21.15.152:0/1981833013 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f6e10072130 con 0x7f6df80052e0 2024-04-20T11:33:27.812 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:27.811+0000 7f6e0d600640 1 -- 172.21.15.152:0/1981833013 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 16 and data pool 17 v62) ==== 174+0+0 (secure 0 0 0) 0x7f6e0405ab40 con 0x7f6df80052e0 2024-04-20T11:33:27.812 INFO:teuthology.orchestra.run.smithi152.stderr:new fs with metadata pool 16 and data pool 17 2024-04-20T11:33:27.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:27.817+0000 7f6e18600640 1 -- 172.21.15.152:0/1981833013 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f6df006ca10 msgr2=0x7f6df006eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:27.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:27.817+0000 7f6e18600640 1 --2- 172.21.15.152:0/1981833013 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f6df006ca10 0x7f6df006eed0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc007fd0 tx=0x7f6dfc009040 comp rx=0 tx=0).stop 2024-04-20T11:33:27.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:27.817+0000 7f6e18600640 1 -- 172.21.15.152:0/1981833013 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6df80052e0 msgr2=0x7f6e1013f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:27.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:27.817+0000 7f6e18600640 1 --2- 172.21.15.152:0/1981833013 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6df80052e0 0x7f6e1013f6e0 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f6e0400f580 tx=0x7f6e0400fa40 comp rx=0 tx=0).stop 2024-04-20T11:33:27.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:27.818+0000 7f6e18600640 1 -- 172.21.15.152:0/1981833013 shutdown_connections 2024-04-20T11:33:27.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:27.818+0000 7f6e18600640 1 --2- 172.21.15.152:0/1981833013 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f6df006ca10 0x7f6df006eed0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:27.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:27.818+0000 7f6e18600640 1 --2- 172.21.15.152:0/1981833013 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e100a9580 0x7f6e100a9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:27.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:27.818+0000 7f6e18600640 1 --2- 172.21.15.152:0/1981833013 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e1006eb90 0x7f6e1006f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:27.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:27.818+0000 7f6e18600640 1 --2- 172.21.15.152:0/1981833013 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6df80052e0 0x7f6e1013f6e0 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:27.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:27.818+0000 7f6e18600640 1 -- 172.21.15.152:0/1981833013 >> 172.21.15.152:0/1981833013 conn(0x7f6e1006b980 msgr2=0x7f6e100b1a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:27.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:27.820+0000 7f6e18600640 1 -- 172.21.15.152:0/1981833013 shutdown_connections 2024-04-20T11:33:27.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:27.820+0000 7f6e18600640 1 -- 172.21.15.152:0/1981833013 wait complete. 2024-04-20T11:33:27.862 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:33:28.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.030+0000 7f5caf200640 1 Processor -- start 2024-04-20T11:33:28.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.030+0000 7f5caf200640 1 -- start start 2024-04-20T11:33:28.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.032+0000 7f5caf200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5cb015d1e0 0x7f5cb0160b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:28.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.032+0000 7f5caf200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f5cb0071cd0 0x7f5cb0072160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:28.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.032+0000 7f5caf200640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5cb00ce8c0 con 0x7f5cb015d5c0 2024-04-20T11:33:28.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.032+0000 7f5caf200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f5cb00726a0 con 0x7f5cb0071cd0 2024-04-20T11:33:28.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.032+0000 7f5caf200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5cb01600b0 con 0x7f5cb015d1e0 2024-04-20T11:33:28.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.032+0000 7f5cad400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f5cb015d5c0 0x7f5cb00cdfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:55078/0 (socket says 172.21.15.152:55078) 2024-04-20T11:33:28.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.032+0000 7f5cad400640 1 -- 172.21.15.152:0/1311113517 learned_addr learned my addr 172.21.15.152:0/1311113517 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:33:28.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.032+0000 7f5cae800640 1 --2- 172.21.15.152:0/1311113517 >> v2:172.21.15.175:3300/0 conn(0x7f5cb0071cd0 0x7f5cb0072160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:28.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.032+0000 7f5cade00640 1 --2- 172.21.15.152:0/1311113517 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5cb015d1e0 0x7f5cb0160b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:28.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.032+0000 7f5caca00640 1 -- 172.21.15.152:0/1311113517 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3411797410 0 0) 0x7f5cb00ce8c0 con 0x7f5cb015d5c0 2024-04-20T11:33:28.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.033+0000 7f5caca00640 1 -- 172.21.15.152:0/1311113517 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5c94003520 con 0x7f5cb015d5c0 2024-04-20T11:33:28.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.033+0000 7f5caca00640 1 -- 172.21.15.152:0/1311113517 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1569909774 0 0) 0x7f5c94003520 con 0x7f5cb015d5c0 2024-04-20T11:33:28.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.033+0000 7f5caca00640 1 -- 172.21.15.152:0/1311113517 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5cb015d1e0 msgr2=0x7f5cb0160b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:28.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.033+0000 7f5caca00640 1 --2- 172.21.15.152:0/1311113517 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5cb015d1e0 0x7f5cb0160b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:28.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.033+0000 7f5caca00640 1 -- 172.21.15.152:0/1311113517 >> v2:172.21.15.175:3300/0 conn(0x7f5cb0071cd0 msgr2=0x7f5cb0072160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:28.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.033+0000 7f5caca00640 1 --2- 172.21.15.152:0/1311113517 >> v2:172.21.15.175:3300/0 conn(0x7f5cb0071cd0 0x7f5cb0072160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:28.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.033+0000 7f5caca00640 1 -- 172.21.15.152:0/1311113517 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cb0160390 con 0x7f5cb015d5c0 2024-04-20T11:33:28.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.033+0000 7f5caca00640 1 -- 172.21.15.152:0/1311113517 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f5ca0001670 con 0x7f5cb015d5c0 2024-04-20T11:33:28.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.033+0000 7f5caca00640 1 -- 172.21.15.152:0/1311113517 >> v1:172.21.15.152:6789/0 conn(0x7f5cb015d5c0 legacy=0x7f5cb00cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:28.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.033+0000 7f5caca00640 1 --2- 172.21.15.152:0/1311113517 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5cb015d1e0 0x7f5c940039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:28.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.033+0000 7f5caca00640 1 --2- 172.21.15.152:0/1311113517 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5c94003f80 0x7f5c94004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:28.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.033+0000 7f5caca00640 1 --2- 172.21.15.152:0/1311113517 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5c94004d40 0x7f5c940051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:28.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.033+0000 7f5caca00640 1 -- 172.21.15.152:0/1311113517 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f5cb00726a0 con 0x7f5c94004d40 2024-04-20T11:33:28.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.033+0000 7f5caca00640 1 -- 172.21.15.152:0/1311113517 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5cb01600b0 con 0x7f5c94003f80 2024-04-20T11:33:28.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.033+0000 7f5caca00640 1 -- 172.21.15.152:0/1311113517 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5c9400b9c0 con 0x7f5cb015d1e0 2024-04-20T11:33:28.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.034+0000 7f5cae800640 1 --2- 172.21.15.152:0/1311113517 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5c94004d40 0x7f5c940051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:28.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.034+0000 7f5cad400640 1 --2- 172.21.15.152:0/1311113517 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5c94003f80 0x7f5c94004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:28.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.034+0000 7f5cae800640 1 -- 172.21.15.152:0/1311113517 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5cb015d1e0 msgr2=0x7f5c940039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:28.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.034+0000 7f5cae800640 1 --2- 172.21.15.152:0/1311113517 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5cb015d1e0 0x7f5c940039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:28.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.034+0000 7f5cae800640 1 -- 172.21.15.152:0/1311113517 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5c94003f80 msgr2=0x7f5c94004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:28.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.034+0000 7f5cae800640 1 --2- 172.21.15.152:0/1311113517 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5c94003f80 0x7f5c94004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:28.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.034+0000 7f5cae800640 1 -- 172.21.15.152:0/1311113517 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c94003520 con 0x7f5c94004d40 2024-04-20T11:33:28.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.034+0000 7f5cad400640 1 --2- 172.21.15.152:0/1311113517 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5c94003f80 0x7f5c94004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:33:28.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.035+0000 7f5cae800640 1 --2- 172.21.15.152:0/1311113517 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5c94004d40 0x7f5c940051c0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f5ca800e170 tx=0x7f5c94005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0911ffc8d54daea server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:28.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.035+0000 7f5caca00640 1 -- 172.21.15.152:0/1311113517 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca8039070 con 0x7f5c94004d40 2024-04-20T11:33:28.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.035+0000 7f5caca00640 1 -- 172.21.15.152:0/1311113517 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ca802cc20 con 0x7f5c94004d40 2024-04-20T11:33:28.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.035+0000 7f5caca00640 1 -- 172.21.15.152:0/1311113517 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca8009c10 con 0x7f5c94004d40 2024-04-20T11:33:28.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.036+0000 7f5caf200640 1 -- 172.21.15.152:0/1311113517 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5c94004d40 msgr2=0x7f5c940051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:28.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.036+0000 7f5caf200640 1 --2- 172.21.15.152:0/1311113517 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5c94004d40 0x7f5c940051c0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f5ca800e170 tx=0x7f5c94005830 comp rx=0 tx=0).stop 2024-04-20T11:33:28.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.036+0000 7f5cae800640 1 -- 172.21.15.152:0/1311113517 reap_dead start 2024-04-20T11:33:28.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.036+0000 7f5caf200640 1 -- 172.21.15.152:0/1311113517 shutdown_connections 2024-04-20T11:33:28.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.036+0000 7f5caf200640 1 -- 172.21.15.152:0/1311113517 >> 172.21.15.152:0/1311113517 conn(0x7f5cb0067ae0 msgr2=0x7f5cb006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:28.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.036+0000 7f5caf200640 1 -- 172.21.15.152:0/1311113517 shutdown_connections 2024-04-20T11:33:28.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.036+0000 7f5caf200640 1 -- 172.21.15.152:0/1311113517 wait complete. 2024-04-20T11:33:28.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.038+0000 7f5caf200640 1 Processor -- start 2024-04-20T11:33:28.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.038+0000 7f5caf200640 1 -- start start 2024-04-20T11:33:28.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.039+0000 7f5caf200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5c94004d40 0x7f5cb0147b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:28.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.039+0000 7f5caf200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5cb015bd70 0x7f5cb00ad980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:28.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.039+0000 7f5caf200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5cb00b2070 0x7f5cb00b24f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:28.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.039+0000 7f5caf200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f5cb0160b90 con 0x7f5cb00b2070 2024-04-20T11:33:28.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.039+0000 7f5caf200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5cb00ceba0 con 0x7f5c94004d40 2024-04-20T11:33:28.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.039+0000 7f5caf200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5cb0160d10 con 0x7f5cb015bd70 2024-04-20T11:33:28.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.039+0000 7f5cae800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5cb00b2070 0x7f5cb00b24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:28.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.039+0000 7f5cae800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5cb00b2070 0x7f5cb00b24f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54178/0 (socket says 172.21.15.152:54178) 2024-04-20T11:33:28.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.039+0000 7f5cae800640 1 -- 172.21.15.152:0/2009695803 learned_addr learned my addr 172.21.15.152:0/2009695803 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:33:28.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.040+0000 7f5cade00640 1 --2- 172.21.15.152:0/2009695803 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5c94004d40 0x7f5cb0147b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:28.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.040+0000 7f5cad400640 1 --2- 172.21.15.152:0/2009695803 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5cb015bd70 0x7f5cb00ad980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:28.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.040+0000 7f5cae800640 1 -- 172.21.15.152:0/2009695803 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5cb015bd70 msgr2=0x7f5cb00ad980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:28.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.040+0000 7f5cae800640 1 --2- 172.21.15.152:0/2009695803 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5cb015bd70 0x7f5cb00ad980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:28.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.040+0000 7f5cae800640 1 -- 172.21.15.152:0/2009695803 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5c94004d40 msgr2=0x7f5cb0147b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:28.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.040+0000 7f5cae800640 1 --2- 172.21.15.152:0/2009695803 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5c94004d40 0x7f5cb0147b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:28.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.040+0000 7f5cae800640 1 -- 172.21.15.152:0/2009695803 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cb00ae2b0 con 0x7f5cb00b2070 2024-04-20T11:33:28.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.040+0000 7f5cae800640 1 --2- 172.21.15.152:0/2009695803 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5cb00b2070 0x7f5cb00b24f0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f5ca800e170 tx=0x7f5ca8009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:28.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.040+0000 7f5ca6000640 1 -- 172.21.15.152:0/2009695803 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca8039070 con 0x7f5cb00b2070 2024-04-20T11:33:28.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.040+0000 7f5ca6000640 1 -- 172.21.15.152:0/2009695803 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ca803a040 con 0x7f5cb00b2070 2024-04-20T11:33:28.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.040+0000 7f5ca6000640 1 -- 172.21.15.152:0/2009695803 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca8009c10 con 0x7f5cb00b2070 2024-04-20T11:33:28.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.040+0000 7f5caf200640 1 -- 172.21.15.152:0/2009695803 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cb00ae590 con 0x7f5cb00b2070 2024-04-20T11:33:28.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.041+0000 7f5caf200640 1 -- 172.21.15.152:0/2009695803 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cb00aeb50 con 0x7f5cb00b2070 2024-04-20T11:33:28.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.042+0000 7f5caf200640 1 -- 172.21.15.152:0/2009695803 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cb006ef10 con 0x7f5cb00b2070 2024-04-20T11:33:28.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.043+0000 7f5ca6000640 1 -- 172.21.15.152:0/2009695803 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 91142+0+0 (secure 0 0 0) 0x7f5ca8005d10 con 0x7f5cb00b2070 2024-04-20T11:33:28.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.044+0000 7f5ca6000640 1 --2- 172.21.15.152:0/2009695803 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f5c8c06ca10 0x7f5c8c06eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:28.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.044+0000 7f5ca6000640 1 -- 172.21.15.152:0/2009695803 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f5ca80ad840 con 0x7f5cb00b2070 2024-04-20T11:33:28.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.044+0000 7f5cade00640 1 --2- 172.21.15.152:0/2009695803 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f5c8c06ca10 0x7f5c8c06eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:28.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.046+0000 7f5cade00640 1 --2- 172.21.15.152:0/2009695803 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f5c8c06ca10 0x7f5c8c06eed0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5c98004770 tx=0x7f5c98008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:28.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.050+0000 7f5ca6000640 1 -- 172.21.15.152:0/2009695803 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f5cb006ef10 con 0x7f5cb00b2070 2024-04-20T11:33:28.050 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2024-04-20T11:33:28.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.247+0000 7f5caf200640 1 -- 172.21.15.152:0/2009695803 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5cb0075e60 con 0x7f5cb00b2070 2024-04-20T11:33:28.250 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.249+0000 7f5ca6000640 1 -- 172.21.15.152:0/2009695803 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) ==== 74+0+16700 (secure 0 0 0) 0x7f5ca807bb50 con 0x7f5cb00b2070 2024-04-20T11:33:28.251 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:33:28.251 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":102,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:33:27.277929+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:33:25.062365+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-20T11:33:26.260945+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:33:28.252 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.252+0000 7f5caf200640 1 -- 172.21.15.152:0/2009695803 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f5c8c06ca10 msgr2=0x7f5c8c06eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:28.253 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.252+0000 7f5caf200640 1 --2- 172.21.15.152:0/2009695803 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f5c8c06ca10 0x7f5c8c06eed0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5c98004770 tx=0x7f5c98008040 comp rx=0 tx=0).stop 2024-04-20T11:33:28.253 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.252+0000 7f5caf200640 1 -- 172.21.15.152:0/2009695803 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5cb00b2070 msgr2=0x7f5cb00b24f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:28.253 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.252+0000 7f5caf200640 1 --2- 172.21.15.152:0/2009695803 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5cb00b2070 0x7f5cb00b24f0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f5ca800e170 tx=0x7f5ca8009750 comp rx=0 tx=0).stop 2024-04-20T11:33:28.253 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.252+0000 7f5caf200640 1 -- 172.21.15.152:0/2009695803 shutdown_connections 2024-04-20T11:33:28.253 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.252+0000 7f5caf200640 1 --2- 172.21.15.152:0/2009695803 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f5c8c06ca10 0x7f5c8c06eed0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:28.253 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.252+0000 7f5caf200640 1 --2- 172.21.15.152:0/2009695803 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5cb00b2070 0x7f5cb00b24f0 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:28.253 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.252+0000 7f5caf200640 1 --2- 172.21.15.152:0/2009695803 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5cb015bd70 0x7f5cb00ad980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:28.253 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.252+0000 7f5caf200640 1 --2- 172.21.15.152:0/2009695803 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5c94004d40 0x7f5cb0147b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:28.253 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.252+0000 7f5caf200640 1 -- 172.21.15.152:0/2009695803 >> 172.21.15.152:0/2009695803 conn(0x7f5cb0067ae0 msgr2=0x7f5cb006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:28.253 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.252+0000 7f5caf200640 1 -- 172.21.15.152:0/2009695803 shutdown_connections 2024-04-20T11:33:28.253 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.252+0000 7f5caf200640 1 -- 172.21.15.152:0/2009695803 wait complete. 2024-04-20T11:33:28.271 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:33:28.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.453+0000 7fd648e00640 1 Processor -- start 2024-04-20T11:33:28.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.454+0000 7fd648e00640 1 -- start start 2024-04-20T11:33:28.456 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.455+0000 7fd648e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd64415d1e0 0x7fd644160b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:28.456 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.455+0000 7fd648e00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6440ce8c0 con 0x7fd644071cd0 2024-04-20T11:33:28.456 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.455+0000 7fd648e00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6441600b0 con 0x7fd64415d5c0 2024-04-20T11:33:28.456 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.455+0000 7fd648e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd644160390 con 0x7fd64415d1e0 2024-04-20T11:33:28.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.456+0000 7fd63be00640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7fd64415d5c0 0x7fd6440cdfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.152:53094/0 (socket says 172.21.15.152:53094) 2024-04-20T11:33:28.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.456+0000 7fd63be00640 1 -- 172.21.15.152:0/512647041 learned_addr learned my addr 172.21.15.152:0/512647041 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:33:28.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.456+0000 7fd643400640 1 --2- 172.21.15.152:0/512647041 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd64415d1e0 0x7fd644160b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:28.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.457+0000 7fd642a00640 1 -- 172.21.15.152:0/512647041 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 62449026 0 0) 0x7fd6440ce8c0 con 0x7fd644071cd0 2024-04-20T11:33:28.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.457+0000 7fd642a00640 1 -- 172.21.15.152:0/512647041 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd628003520 con 0x7fd644071cd0 2024-04-20T11:33:28.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.457+0000 7fd642a00640 1 -- 172.21.15.152:0/512647041 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3235295333 0 0) 0x7fd6441600b0 con 0x7fd64415d5c0 2024-04-20T11:33:28.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.457+0000 7fd642a00640 1 -- 172.21.15.152:0/512647041 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd6440ce8c0 con 0x7fd64415d5c0 2024-04-20T11:33:28.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.457+0000 7fd642a00640 1 -- 172.21.15.152:0/512647041 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3456114118 0 0) 0x7fd628003520 con 0x7fd644071cd0 2024-04-20T11:33:28.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.457+0000 7fd642a00640 1 -- 172.21.15.152:0/512647041 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd64415d1e0 msgr2=0x7fd644160b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:28.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.457+0000 7fd642a00640 1 --2- 172.21.15.152:0/512647041 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd64415d1e0 0x7fd644160b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:28.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.457+0000 7fd642a00640 1 -- 172.21.15.152:0/512647041 >> v1:172.21.15.175:6789/0 conn(0x7fd64415d5c0 legacy=0x7fd6440cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:28.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.457+0000 7fd642a00640 1 -- 172.21.15.152:0/512647041 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd644160670 con 0x7fd644071cd0 2024-04-20T11:33:28.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.457+0000 7fd642a00640 1 -- 172.21.15.152:0/512647041 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7fd63c0019f0 con 0x7fd644071cd0 2024-04-20T11:33:28.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.457+0000 7fd642a00640 1 -- 172.21.15.152:0/512647041 >> v1:172.21.15.152:6789/0 conn(0x7fd644071cd0 legacy=0x7fd6440ccd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:28.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.457+0000 7fd642a00640 1 --2- 172.21.15.152:0/512647041 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd628003b70 0x7fd628003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:28.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.457+0000 7fd642a00640 1 --2- 172.21.15.152:0/512647041 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd628004800 0x7fd628004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:28.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.457+0000 7fd642a00640 1 --2- 172.21.15.152:0/512647041 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd64415d1e0 0x7fd6280051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:28.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.457+0000 7fd642a00640 1 -- 172.21.15.152:0/512647041 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fd644160390 con 0x7fd628004800 2024-04-20T11:33:28.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.457+0000 7fd642a00640 1 -- 172.21.15.152:0/512647041 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd62800ba70 con 0x7fd628003b70 2024-04-20T11:33:28.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.457+0000 7fd642a00640 1 -- 172.21.15.152:0/512647041 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd62800bc40 con 0x7fd64415d1e0 2024-04-20T11:33:28.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.457+0000 7fd63be00640 1 --2- 172.21.15.152:0/512647041 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd628004800 0x7fd628004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:28.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.458+0000 7fd643400640 1 --2- 172.21.15.152:0/512647041 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd628003b70 0x7fd628003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:28.461 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.458+0000 7fd643e00640 1 --2- 172.21.15.152:0/512647041 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd64415d1e0 0x7fd6280051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:28.461 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.458+0000 7fd63be00640 1 -- 172.21.15.152:0/512647041 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd64415d1e0 msgr2=0x7fd6280051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:28.461 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.458+0000 7fd63be00640 1 --2- 172.21.15.152:0/512647041 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd64415d1e0 0x7fd6280051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:28.461 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.458+0000 7fd63be00640 1 -- 172.21.15.152:0/512647041 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd628003b70 msgr2=0x7fd628003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:28.461 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.458+0000 7fd63be00640 1 --2- 172.21.15.152:0/512647041 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd628003b70 0x7fd628003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:28.461 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.458+0000 7fd63be00640 1 -- 172.21.15.152:0/512647041 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd628003520 con 0x7fd628004800 2024-04-20T11:33:28.461 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.459+0000 7fd63be00640 1 --2- 172.21.15.152:0/512647041 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd628004800 0x7fd628004c80 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7fd62c0019c0 tx=0x7fd628005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7065353fc782e009 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:28.461 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.459+0000 7fd642a00640 1 -- 172.21.15.152:0/512647041 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd62c037070 con 0x7fd628004800 2024-04-20T11:33:28.461 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.459+0000 7fd642a00640 1 -- 172.21.15.152:0/512647041 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd62c0048a0 con 0x7fd628004800 2024-04-20T11:33:28.461 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.459+0000 7fd642a00640 1 -- 172.21.15.152:0/512647041 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd62c0015e0 con 0x7fd628004800 2024-04-20T11:33:28.461 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.460+0000 7fd648e00640 1 -- 172.21.15.152:0/512647041 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd628004800 msgr2=0x7fd628004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:28.461 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.460+0000 7fd648e00640 1 --2- 172.21.15.152:0/512647041 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd628004800 0x7fd628004c80 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7fd62c0019c0 tx=0x7fd628005830 comp rx=0 tx=0).stop 2024-04-20T11:33:28.461 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.460+0000 7fd643e00640 1 -- 172.21.15.152:0/512647041 reap_dead start 2024-04-20T11:33:28.462 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.460+0000 7fd648e00640 1 -- 172.21.15.152:0/512647041 shutdown_connections 2024-04-20T11:33:28.462 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.460+0000 7fd648e00640 1 -- 172.21.15.152:0/512647041 >> 172.21.15.152:0/512647041 conn(0x7fd644067ae0 msgr2=0x7fd64406b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:28.462 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.460+0000 7fd648e00640 1 -- 172.21.15.152:0/512647041 shutdown_connections 2024-04-20T11:33:28.462 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.460+0000 7fd648e00640 1 -- 172.21.15.152:0/512647041 wait complete. 2024-04-20T11:33:28.462 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.462+0000 7fd648e00640 1 Processor -- start 2024-04-20T11:33:28.462 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.462+0000 7fd648e00640 1 -- start start 2024-04-20T11:33:28.465 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.463+0000 7fd648e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd628004800 0x7fd6440ae3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:28.465 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.463+0000 7fd648e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd644143760 0x7fd644143be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:28.465 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.463+0000 7fd648e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd6440b3910 0x7fd6440b3d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:28.465 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.463+0000 7fd648e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fd644160d10 con 0x7fd644143760 2024-04-20T11:33:28.465 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.463+0000 7fd648e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd644160b90 con 0x7fd628004800 2024-04-20T11:33:28.465 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.463+0000 7fd648e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd644160e90 con 0x7fd6440b3910 2024-04-20T11:33:28.465 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.463+0000 7fd63be00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd644143760 0x7fd644143be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:28.465 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.463+0000 7fd63be00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd644143760 0x7fd644143be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54202/0 (socket says 172.21.15.152:54202) 2024-04-20T11:33:28.465 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.463+0000 7fd63be00640 1 -- 172.21.15.152:0/591582067 learned_addr learned my addr 172.21.15.152:0/591582067 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:33:28.465 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.463+0000 7fd643e00640 1 --2- 172.21.15.152:0/591582067 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd6440b3910 0x7fd6440b3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:28.465 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.464+0000 7fd63be00640 1 -- 172.21.15.152:0/591582067 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd6440b3910 msgr2=0x7fd6440b3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:28.465 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.464+0000 7fd63be00640 1 --2- 172.21.15.152:0/591582067 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd6440b3910 0x7fd6440b3d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:28.465 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.464+0000 7fd643400640 1 --2- 172.21.15.152:0/591582067 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd628004800 0x7fd6440ae3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:28.465 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.464+0000 7fd63be00640 1 -- 172.21.15.152:0/591582067 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd628004800 msgr2=0x7fd6440ae3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:28.466 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.464+0000 7fd63be00640 1 --2- 172.21.15.152:0/591582067 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd628004800 0x7fd6440ae3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:28.466 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.464+0000 7fd63be00640 1 -- 172.21.15.152:0/591582067 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd644144120 con 0x7fd644143760 2024-04-20T11:33:28.466 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.464+0000 7fd643400640 1 --2- 172.21.15.152:0/591582067 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd628004800 0x7fd6440ae3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:33:28.466 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.464+0000 7fd63be00640 1 --2- 172.21.15.152:0/591582067 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd644143760 0x7fd644143be0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7fd62c035040 tx=0x7fd62c0012d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:28.466 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.464+0000 7fd63b400640 1 -- 172.21.15.152:0/591582067 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd62c037040 con 0x7fd644143760 2024-04-20T11:33:28.466 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.464+0000 7fd63b400640 1 -- 172.21.15.152:0/591582067 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd62c007760 con 0x7fd644143760 2024-04-20T11:33:28.466 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.464+0000 7fd63b400640 1 -- 172.21.15.152:0/591582067 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd62c0015e0 con 0x7fd644143760 2024-04-20T11:33:28.466 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.464+0000 7fd648e00640 1 -- 172.21.15.152:0/591582067 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6440b45d0 con 0x7fd644143760 2024-04-20T11:33:28.466 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.464+0000 7fd648e00640 1 -- 172.21.15.152:0/591582067 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6440b4b90 con 0x7fd644143760 2024-04-20T11:33:28.467 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.465+0000 7fd648e00640 1 -- 172.21.15.152:0/591582067 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd644075e60 con 0x7fd644143760 2024-04-20T11:33:28.467 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.466+0000 7fd63b400640 1 -- 172.21.15.152:0/591582067 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 91142+0+0 (secure 0 0 0) 0x7fd62c031650 con 0x7fd644143760 2024-04-20T11:33:28.468 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.467+0000 7fd63b400640 1 --2- 172.21.15.152:0/591582067 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd62006ca40 0x7fd62006ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:28.468 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.467+0000 7fd63b400640 1 -- 172.21.15.152:0/591582067 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7fd62c0ad4a0 con 0x7fd644143760 2024-04-20T11:33:28.468 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.468+0000 7fd643400640 1 --2- 172.21.15.152:0/591582067 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd62006ca40 0x7fd62006ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:28.469 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.469+0000 7fd643400640 1 --2- 172.21.15.152:0/591582067 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd62006ca40 0x7fd62006ef00 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd6440b3660 tx=0x7fd634008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:28.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.471+0000 7fd63b400640 1 -- 172.21.15.152:0/591582067 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fd62c07b720 con 0x7fd644143760 2024-04-20T11:33:28.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.658+0000 7fd648e00640 1 -- 172.21.15.152:0/591582067 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd64406f900 con 0x7fd644143760 2024-04-20T11:33:28.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.661+0000 7fd63b400640 1 -- 172.21.15.152:0/591582067 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) ==== 74+0+16700 (secure 0 0 0) 0x7fd62c07b020 con 0x7fd644143760 2024-04-20T11:33:28.662 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:33:28.663 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":102,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:33:27.277929+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:33:25.062365+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-20T11:33:26.260945+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:33:28.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.664+0000 7fd648e00640 1 -- 172.21.15.152:0/591582067 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd62006ca40 msgr2=0x7fd62006ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:28.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.664+0000 7fd648e00640 1 --2- 172.21.15.152:0/591582067 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd62006ca40 0x7fd62006ef00 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd6440b3660 tx=0x7fd634008040 comp rx=0 tx=0).stop 2024-04-20T11:33:28.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.664+0000 7fd648e00640 1 -- 172.21.15.152:0/591582067 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd644143760 msgr2=0x7fd644143be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:28.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.664+0000 7fd648e00640 1 --2- 172.21.15.152:0/591582067 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd644143760 0x7fd644143be0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7fd62c035040 tx=0x7fd62c0012d0 comp rx=0 tx=0).stop 2024-04-20T11:33:28.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.664+0000 7fd648e00640 1 -- 172.21.15.152:0/591582067 shutdown_connections 2024-04-20T11:33:28.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.664+0000 7fd648e00640 1 --2- 172.21.15.152:0/591582067 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd62006ca40 0x7fd62006ef00 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:28.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.664+0000 7fd648e00640 1 --2- 172.21.15.152:0/591582067 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd6440b3910 0x7fd6440b3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:28.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.664+0000 7fd648e00640 1 --2- 172.21.15.152:0/591582067 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd644143760 0x7fd644143be0 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:28.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.664+0000 7fd648e00640 1 --2- 172.21.15.152:0/591582067 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd628004800 0x7fd6440ae3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:28.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.664+0000 7fd648e00640 1 -- 172.21.15.152:0/591582067 >> 172.21.15.152:0/591582067 conn(0x7fd644067ae0 msgr2=0x7fd64415a740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:28.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.664+0000 7fd648e00640 1 -- 172.21.15.152:0/591582067 shutdown_connections 2024-04-20T11:33:28.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.664+0000 7fd648e00640 1 -- 172.21.15.152:0/591582067 wait complete. 2024-04-20T11:33:28.684 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-20T11:33:28.888 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.888+0000 7fec55000640 1 Processor -- start 2024-04-20T11:33:28.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.888+0000 7fec55000640 1 -- start start 2024-04-20T11:33:28.890 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.889+0000 7fec55000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fec500d46a0 0x7fec500d4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:28.890 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.889+0000 7fec55000640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec500d19a0 con 0x7fec5015e9e0 2024-04-20T11:33:28.890 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.889+0000 7fec55000640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec500d1c80 con 0x7fec500d0d90 2024-04-20T11:33:28.890 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.889+0000 7fec55000640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fec5006da30 con 0x7fec500d46a0 2024-04-20T11:33:28.890 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.889+0000 7fec45400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fec500d46a0 0x7fec500d4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:28.890 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.890+0000 7fec45400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fec500d46a0 0x7fec500d4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54208/0 (socket says 172.21.15.152:54208) 2024-04-20T11:33:28.890 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.890+0000 7fec45400640 1 -- 172.21.15.152:0/3565679258 learned_addr learned my addr 172.21.15.152:0/3565679258 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:33:28.891 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.890+0000 7fec4de00640 1 --1- 172.21.15.152:0/3565679258 >> v1:172.21.15.152:6789/0 conn(0x7fec5015e9e0 0x7fec5006d320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:55096/0 (socket says 172.21.15.152:55096) 2024-04-20T11:33:28.891 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.890+0000 7fec45400640 1 -- 172.21.15.152:0/3565679258 >> v1:172.21.15.175:6789/0 conn(0x7fec500d0d90 legacy=0x7fec500cfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:28.891 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.890+0000 7fec45400640 1 -- 172.21.15.152:0/3565679258 >> v1:172.21.15.152:6789/0 conn(0x7fec5015e9e0 legacy=0x7fec5006d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:28.891 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.890+0000 7fec45400640 1 -- 172.21.15.152:0/3565679258 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec500d1f60 con 0x7fec500d46a0 2024-04-20T11:33:28.891 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.890+0000 7fec4d400640 1 -- 172.21.15.152:0/3565679258 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3498461272 0 0) 0x7fec500d19a0 con 0x7fec5015e9e0 2024-04-20T11:33:28.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.891+0000 7fec45400640 1 --2- 172.21.15.152:0/3565679258 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fec500d46a0 0x7fec500d4b00 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7fec38009870 tx=0x7fec3802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f374964d08a0a51e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:28.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.891+0000 7fec4d400640 1 -- 172.21.15.152:0/3565679258 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec38034070 con 0x7fec500d46a0 2024-04-20T11:33:28.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.891+0000 7fec4d400640 1 -- 172.21.15.152:0/3565679258 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec38030860 con 0x7fec500d46a0 2024-04-20T11:33:28.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.891+0000 7fec4d400640 1 -- 172.21.15.152:0/3565679258 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec38030c30 con 0x7fec500d46a0 2024-04-20T11:33:28.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.892+0000 7fec55000640 1 -- 172.21.15.152:0/3565679258 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fec500d46a0 msgr2=0x7fec500d4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:28.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.892+0000 7fec55000640 1 --2- 172.21.15.152:0/3565679258 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fec500d46a0 0x7fec500d4b00 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7fec38009870 tx=0x7fec3802b9a0 comp rx=0 tx=0).stop 2024-04-20T11:33:28.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.892+0000 7fec55000640 1 -- 172.21.15.152:0/3565679258 shutdown_connections 2024-04-20T11:33:28.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.892+0000 7fec55000640 1 --2- 172.21.15.152:0/3565679258 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fec500d46a0 0x7fec500d4b00 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:28.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.892+0000 7fec55000640 1 -- 172.21.15.152:0/3565679258 >> 172.21.15.152:0/3565679258 conn(0x7fec5009dab0 msgr2=0x7fec5008bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:28.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.892+0000 7fec55000640 1 -- 172.21.15.152:0/3565679258 shutdown_connections 2024-04-20T11:33:28.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.892+0000 7fec55000640 1 -- 172.21.15.152:0/3565679258 wait complete. 2024-04-20T11:33:28.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.894+0000 7fec55000640 1 Processor -- start 2024-04-20T11:33:28.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.894+0000 7fec55000640 1 -- start start 2024-04-20T11:33:28.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.895+0000 7fec55000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fec500d0d90 0x7fec5015f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:28.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.895+0000 7fec55000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fec500d46a0 0x7fec5014d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:28.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.895+0000 7fec55000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fec5015e9e0 0x7fec5014dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:28.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.895+0000 7fec55000640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fec500cf880 con 0x7fec500d46a0 2024-04-20T11:33:28.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.895+0000 7fec55000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fec500d5040 con 0x7fec5015e9e0 2024-04-20T11:33:28.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.895+0000 7fec55000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fec5015edc0 con 0x7fec500d0d90 2024-04-20T11:33:28.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.895+0000 7fec45400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fec500d46a0 0x7fec5014d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:28.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.895+0000 7fec45400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fec500d46a0 0x7fec5014d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54218/0 (socket says 172.21.15.152:54218) 2024-04-20T11:33:28.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.895+0000 7fec45400640 1 -- 172.21.15.152:0/3546492883 learned_addr learned my addr 172.21.15.152:0/3546492883 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:33:28.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.896+0000 7fec45400640 1 -- 172.21.15.152:0/3546492883 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fec500d0d90 msgr2=0x7fec5015f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:28.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.896+0000 7fec4de00640 1 --2- 172.21.15.152:0/3546492883 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fec500d0d90 0x7fec5015f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:28.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.896+0000 7fec4e800640 1 --2- 172.21.15.152:0/3546492883 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fec5015e9e0 0x7fec5014dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:28.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.896+0000 7fec45400640 1 --2- 172.21.15.152:0/3546492883 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fec500d0d90 0x7fec5015f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:28.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.896+0000 7fec45400640 1 -- 172.21.15.152:0/3546492883 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fec5015e9e0 msgr2=0x7fec5014dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:28.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.896+0000 7fec45400640 1 --2- 172.21.15.152:0/3546492883 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fec5015e9e0 0x7fec5014dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:28.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.896+0000 7fec45400640 1 -- 172.21.15.152:0/3546492883 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec50161f00 con 0x7fec500d46a0 2024-04-20T11:33:28.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.896+0000 7fec45400640 1 --2- 172.21.15.152:0/3546492883 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fec500d46a0 0x7fec5014d960 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7fec38009870 tx=0x7fec38004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:28.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.896+0000 7fec46a00640 1 -- 172.21.15.152:0/3546492883 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec38048070 con 0x7fec500d46a0 2024-04-20T11:33:28.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.896+0000 7fec46a00640 1 -- 172.21.15.152:0/3546492883 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec38030660 con 0x7fec500d46a0 2024-04-20T11:33:28.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.896+0000 7fec46a00640 1 -- 172.21.15.152:0/3546492883 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec38034040 con 0x7fec500d46a0 2024-04-20T11:33:28.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.896+0000 7fec55000640 1 -- 172.21.15.152:0/3546492883 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec50161660 con 0x7fec500d46a0 2024-04-20T11:33:28.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.896+0000 7fec55000640 1 -- 172.21.15.152:0/3546492883 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec50161c20 con 0x7fec500d46a0 2024-04-20T11:33:28.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.897+0000 7fec55000640 1 -- 172.21.15.152:0/3546492883 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec500689f0 con 0x7fec500d46a0 2024-04-20T11:33:28.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.898+0000 7fec46a00640 1 -- 172.21.15.152:0/3546492883 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 91142+0+0 (secure 0 0 0) 0x7fec38037070 con 0x7fec500d46a0 2024-04-20T11:33:28.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.899+0000 7fec46a00640 1 --2- 172.21.15.152:0/3546492883 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fec2c06ca10 0x7fec2c06eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:28.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.899+0000 7fec46a00640 1 -- 172.21.15.152:0/3546492883 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7fec380adde0 con 0x7fec500d46a0 2024-04-20T11:33:28.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.900+0000 7fec4de00640 1 --2- 172.21.15.152:0/3546492883 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fec2c06ca10 0x7fec2c06eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:28.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.901+0000 7fec4de00640 1 --2- 172.21.15.152:0/3546492883 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fec2c06ca10 0x7fec2c06eed0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fec40000f60 tx=0x7fec40009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:28.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:28.904+0000 7fec46a00640 1 -- 172.21.15.152:0/3546492883 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fec3807c070 con 0x7fec500d46a0 2024-04-20T11:33:29.132 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:29.132+0000 7fec55000640 1 -- 172.21.15.152:0/3546492883 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fec500cc930 con 0x7fec500d46a0 2024-04-20T11:33:29.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:29.853+0000 7fec46a00640 1 -- 172.21.15.152:0/3546492883 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v65) ==== 118+0+0 (secure 0 0 0) 0x7fec3807b970 con 0x7fec500d46a0 2024-04-20T11:33:29.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:29.856+0000 7fec55000640 1 -- 172.21.15.152:0/3546492883 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fec2c06ca10 msgr2=0x7fec2c06eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:29.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:29.856+0000 7fec55000640 1 --2- 172.21.15.152:0/3546492883 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fec2c06ca10 0x7fec2c06eed0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fec40000f60 tx=0x7fec40009040 comp rx=0 tx=0).stop 2024-04-20T11:33:29.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:29.856+0000 7fec55000640 1 -- 172.21.15.152:0/3546492883 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fec500d46a0 msgr2=0x7fec5014d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:29.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:29.856+0000 7fec55000640 1 --2- 172.21.15.152:0/3546492883 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fec500d46a0 0x7fec5014d960 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7fec38009870 tx=0x7fec38004730 comp rx=0 tx=0).stop 2024-04-20T11:33:29.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:29.856+0000 7fec55000640 1 -- 172.21.15.152:0/3546492883 shutdown_connections 2024-04-20T11:33:29.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:29.856+0000 7fec55000640 1 --2- 172.21.15.152:0/3546492883 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fec2c06ca10 0x7fec2c06eed0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:29.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:29.856+0000 7fec55000640 1 --2- 172.21.15.152:0/3546492883 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fec5015e9e0 0x7fec5014dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:29.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:29.856+0000 7fec55000640 1 --2- 172.21.15.152:0/3546492883 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fec500d46a0 0x7fec5014d960 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:29.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:29.856+0000 7fec55000640 1 --2- 172.21.15.152:0/3546492883 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fec500d0d90 0x7fec5015f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:29.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:29.856+0000 7fec55000640 1 -- 172.21.15.152:0/3546492883 >> 172.21.15.152:0/3546492883 conn(0x7fec5009dab0 msgr2=0x7fec500b4a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:29.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:29.856+0000 7fec55000640 1 -- 172.21.15.152:0/3546492883 shutdown_connections 2024-04-20T11:33:29.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:29.857+0000 7fec55000640 1 -- 172.21.15.152:0/3546492883 wait complete. 2024-04-20T11:33:29.878 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-20T11:33:29.878 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-20T11:33:30.092 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:30.087+0000 7f4b35000640 1 Processor -- start 2024-04-20T11:33:30.092 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:30.087+0000 7f4b35000640 1 -- start start 2024-04-20T11:33:30.092 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:30.089+0000 7f4b35000640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f4b300d35a0 0x7f4b301698d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:30.092 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:30.089+0000 7f4b35000640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b300d19c0 con 0x7f4b30163230 2024-04-20T11:33:30.092 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:30.089+0000 7f4b35000640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b30162750 con 0x7f4b3015eb50 2024-04-20T11:33:30.092 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:30.089+0000 7f4b35000640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f4b30162a30 con 0x7f4b300d35a0 2024-04-20T11:33:30.092 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:30.089+0000 7f4b27e00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f4b30163230 0x7f4b300d2e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:55112/0 (socket says 172.21.15.152:55112) 2024-04-20T11:33:30.092 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:30.089+0000 7f4b27e00640 1 -- 172.21.15.152:0/3338523035 learned_addr learned my addr 172.21.15.152:0/3338523035 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:33:30.092 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:30.090+0000 7f4b2ea00640 1 -- 172.21.15.152:0/3338523035 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3182257818 0 0) 0x7f4b300d19c0 con 0x7f4b30163230 2024-04-20T11:33:30.092 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:30.090+0000 7f4b2ea00640 1 -- 172.21.15.152:0/3338523035 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4b14003520 con 0x7f4b30163230 2024-04-20T11:33:30.092 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:30.090+0000 7f4b2fe00640 1 --2- 172.21.15.152:0/3338523035 >> v2:172.21.15.175:3300/0 conn(0x7f4b300d35a0 0x7f4b301698d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:30.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:30.090+0000 7f4b2ea00640 1 -- 172.21.15.152:0/3338523035 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 429180128 0 0) 0x7f4b14003520 con 0x7f4b30163230 2024-04-20T11:33:30.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:30.090+0000 7f4b2ea00640 1 -- 172.21.15.152:0/3338523035 >> v2:172.21.15.175:3300/0 conn(0x7f4b300d35a0 msgr2=0x7f4b301698d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:30.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:30.090+0000 7f4b2ea00640 1 --2- 172.21.15.152:0/3338523035 >> v2:172.21.15.175:3300/0 conn(0x7f4b300d35a0 0x7f4b301698d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:30.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:30.090+0000 7f4b2ea00640 1 -- 172.21.15.152:0/3338523035 >> v1:172.21.15.175:6789/0 conn(0x7f4b3015eb50 legacy=0x7f4b300d0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:30.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:30.090+0000 7f4b2ea00640 1 -- 172.21.15.152:0/3338523035 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b30162d10 con 0x7f4b30163230 2024-04-20T11:33:30.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:30.090+0000 7f4b2ea00640 1 -- 172.21.15.152:0/3338523035 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f4b180017f0 con 0x7f4b30163230 2024-04-20T11:33:30.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:30.090+0000 7f4b2ea00640 1 -- 172.21.15.152:0/3338523035 >> v1:172.21.15.152:6789/0 conn(0x7f4b30163230 legacy=0x7f4b300d2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:30.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:30.090+0000 7f4b2ea00640 1 --2- 172.21.15.152:0/3338523035 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4b140039b0 0x7f4b14003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:30.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:30.090+0000 7f4b2ea00640 1 --2- 172.21.15.152:0/3338523035 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b14004800 0x7f4b14004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:30.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:30.090+0000 7f4b2ea00640 1 --2- 172.21.15.152:0/3338523035 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4b140052e0 0x7f4b1400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:30.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:30.090+0000 7f4b2ea00640 1 -- 172.21.15.152:0/3338523035 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f4b30162a30 con 0x7f4b14004800 2024-04-20T11:33:30.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:30.090+0000 7f4b2ea00640 1 -- 172.21.15.152:0/3338523035 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4b1400c190 con 0x7f4b140039b0 2024-04-20T11:33:30.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:30.090+0000 7f4b2ea00640 1 -- 172.21.15.152:0/3338523035 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4b1400c390 con 0x7f4b140052e0 2024-04-20T11:33:30.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:30.090+0000 7f4b27e00640 1 --2- 172.21.15.152:0/3338523035 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b14004800 0x7f4b14004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:30.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:30.091+0000 7f4b2f400640 1 --2- 172.21.15.152:0/3338523035 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4b140039b0 0x7f4b14003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:30.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:30.091+0000 7f4b27e00640 1 -- 172.21.15.152:0/3338523035 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4b140052e0 msgr2=0x7f4b1400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:30.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:30.091+0000 7f4b2fe00640 1 --2- 172.21.15.152:0/3338523035 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4b140052e0 0x7f4b1400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:30.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:30.091+0000 7f4b27e00640 1 --2- 172.21.15.152:0/3338523035 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4b140052e0 0x7f4b1400b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:30.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:30.091+0000 7f4b27e00640 1 -- 172.21.15.152:0/3338523035 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4b140039b0 msgr2=0x7f4b14003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:30.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:30.091+0000 7f4b27e00640 1 --2- 172.21.15.152:0/3338523035 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4b140039b0 0x7f4b14003db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:30.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:30.091+0000 7f4b27e00640 1 -- 172.21.15.152:0/3338523035 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b14003520 con 0x7f4b14004800 2024-04-20T11:33:30.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:30.091+0000 7f4b2f400640 1 --2- 172.21.15.152:0/3338523035 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4b140039b0 0x7f4b14003db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:33:30.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:30.091+0000 7f4b2fe00640 1 -- 172.21.15.152:0/3338523035 reap_dead start 2024-04-20T11:33:30.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:30.092+0000 7f4b27e00640 1 --2- 172.21.15.152:0/3338523035 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b14004800 0x7f4b14004c80 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f4b1800c190 tx=0x7f4b14005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d41b22a532a01eb server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:30.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:30.092+0000 7f4b2ea00640 1 -- 172.21.15.152:0/3338523035 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b18007d00 con 0x7f4b14004800 2024-04-20T11:33:30.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:30.092+0000 7f4b2ea00640 1 -- 172.21.15.152:0/3338523035 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b18030070 con 0x7f4b14004800 2024-04-20T11:33:30.096 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:30.092+0000 7f4b2ea00640 1 -- 172.21.15.152:0/3338523035 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b180074d0 con 0x7f4b14004800 2024-04-20T11:33:30.096 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:30.093+0000 7f4b35000640 1 -- 172.21.15.152:0/3338523035 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b14004800 msgr2=0x7f4b14004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:30.096 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:30.093+0000 7f4b35000640 1 --2- 172.21.15.152:0/3338523035 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b14004800 0x7f4b14004c80 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f4b1800c190 tx=0x7f4b14005830 comp rx=0 tx=0).stop 2024-04-20T11:33:30.096 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:30.093+0000 7f4b35000640 1 -- 172.21.15.152:0/3338523035 shutdown_connections 2024-04-20T11:33:30.096 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:30.093+0000 7f4b35000640 1 --2- 172.21.15.152:0/3338523035 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b14004800 0x7f4b14004c80 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:30.096 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:30.093+0000 7f4b35000640 1 -- 172.21.15.152:0/3338523035 >> 172.21.15.152:0/3338523035 conn(0x7f4b3008bfd0 msgr2=0x7f4b300d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:30.096 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:30.093+0000 7f4b35000640 1 -- 172.21.15.152:0/3338523035 shutdown_connections 2024-04-20T11:33:30.096 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:30.093+0000 7f4b35000640 1 -- 172.21.15.152:0/3338523035 wait complete. 2024-04-20T11:33:30.096 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:30.095+0000 7f4b35000640 1 Processor -- start 2024-04-20T11:33:30.096 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:30.095+0000 7f4b35000640 1 -- start start 2024-04-20T11:33:30.097 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:30.096+0000 7f4b35000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4b14004800 0x7f4b3014d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:30.097 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:30.096+0000 7f4b35000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4b300b8f70 0x7f4b300b93f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:30.098 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:30.096+0000 7f4b35000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b300c8a80 0x7f4b300c8f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:30.098 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:30.096+0000 7f4b35000640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f4b300d3010 con 0x7f4b300c8a80 2024-04-20T11:33:30.098 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:30.096+0000 7f4b35000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4b300d2e90 con 0x7f4b14004800 2024-04-20T11:33:30.098 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:30.096+0000 7f4b35000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4b300d3190 con 0x7f4b300b8f70 2024-04-20T11:33:30.098 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:30.096+0000 7f4b2fe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b300c8a80 0x7f4b300c8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:30.098 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:30.096+0000 7f4b2fe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b300c8a80 0x7f4b300c8f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54244/0 (socket says 172.21.15.152:54244) 2024-04-20T11:33:30.098 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:30.096+0000 7f4b2fe00640 1 -- 172.21.15.152:0/2539690099 learned_addr learned my addr 172.21.15.152:0/2539690099 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:33:30.098 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:30.097+0000 7f4b27e00640 1 --2- 172.21.15.152:0/2539690099 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4b300b8f70 0x7f4b300b93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:30.098 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:30.097+0000 7f4b2f400640 1 --2- 172.21.15.152:0/2539690099 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4b14004800 0x7f4b3014d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:30.098 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:30.097+0000 7f4b2fe00640 1 -- 172.21.15.152:0/2539690099 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4b300b8f70 msgr2=0x7f4b300b93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:30.098 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:30.097+0000 7f4b2fe00640 1 --2- 172.21.15.152:0/2539690099 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4b300b8f70 0x7f4b300b93f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:30.098 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:30.097+0000 7f4b2fe00640 1 -- 172.21.15.152:0/2539690099 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4b14004800 msgr2=0x7f4b3014d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:30.098 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:30.097+0000 7f4b2fe00640 1 --2- 172.21.15.152:0/2539690099 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4b14004800 0x7f4b3014d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:30.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:30.097+0000 7f4b2fe00640 1 -- 172.21.15.152:0/2539690099 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b300c9840 con 0x7f4b300c8a80 2024-04-20T11:33:30.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:30.097+0000 7f4b2f400640 1 --2- 172.21.15.152:0/2539690099 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4b14004800 0x7f4b3014d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:33:30.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:30.097+0000 7f4b2fe00640 1 --2- 172.21.15.152:0/2539690099 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b300c8a80 0x7f4b300c8f00 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f4b300d2010 tx=0x7f4b28009dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:30.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:30.098+0000 7f4b27400640 1 -- 172.21.15.152:0/2539690099 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b28019070 con 0x7f4b300c8a80 2024-04-20T11:33:30.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:30.098+0000 7f4b27400640 1 -- 172.21.15.152:0/2539690099 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b28007e00 con 0x7f4b300c8a80 2024-04-20T11:33:30.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:30.098+0000 7f4b27400640 1 -- 172.21.15.152:0/2539690099 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b28014640 con 0x7f4b300c8a80 2024-04-20T11:33:30.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:30.098+0000 7f4b35000640 1 -- 172.21.15.152:0/2539690099 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b300c9b20 con 0x7f4b300c8a80 2024-04-20T11:33:30.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:30.098+0000 7f4b35000640 1 -- 172.21.15.152:0/2539690099 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b300b4620 con 0x7f4b300c8a80 2024-04-20T11:33:30.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:30.099+0000 7f4b35000640 1 -- 172.21.15.152:0/2539690099 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b300b4820 con 0x7f4b300c8a80 2024-04-20T11:33:30.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:30.100+0000 7f4b27400640 1 -- 172.21.15.152:0/2539690099 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 91142+0+0 (secure 0 0 0) 0x7f4b2801c070 con 0x7f4b300c8a80 2024-04-20T11:33:30.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:30.101+0000 7f4b27400640 1 --2- 172.21.15.152:0/2539690099 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4b0c06ca40 0x7f4b0c06ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:30.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:30.101+0000 7f4b27400640 1 -- 172.21.15.152:0/2539690099 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7f4b2808b8e0 con 0x7f4b300c8a80 2024-04-20T11:33:30.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:30.102+0000 7f4b2f400640 1 --2- 172.21.15.152:0/2539690099 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4b0c06ca40 0x7f4b0c06ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:30.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:30.103+0000 7f4b2f400640 1 --2- 172.21.15.152:0/2539690099 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4b0c06ca40 0x7f4b0c06ef00 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f4b20007fd0 tx=0x7f4b20009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:30.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:30.106+0000 7f4b27400640 1 -- 172.21.15.152:0/2539690099 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f4b300b4820 con 0x7f4b300c8a80 2024-04-20T11:33:30.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:30.340+0000 7f4b35000640 1 -- 172.21.15.152:0/2539690099 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f4b300d0b70 con 0x7f4b300c8a80 2024-04-20T11:33:30.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:30.856+0000 7f4b27400640 1 -- 172.21.15.152:0/2539690099 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v66) ==== 122+0+0 (secure 0 0 0) 0x7f4b28059c00 con 0x7f4b300c8a80 2024-04-20T11:33:30.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:30.860+0000 7f4b35000640 1 -- 172.21.15.152:0/2539690099 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4b0c06ca40 msgr2=0x7f4b0c06ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:30.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:30.860+0000 7f4b35000640 1 --2- 172.21.15.152:0/2539690099 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4b0c06ca40 0x7f4b0c06ef00 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f4b20007fd0 tx=0x7f4b20009040 comp rx=0 tx=0).stop 2024-04-20T11:33:30.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:30.860+0000 7f4b35000640 1 -- 172.21.15.152:0/2539690099 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b300c8a80 msgr2=0x7f4b300c8f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:30.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:30.860+0000 7f4b35000640 1 --2- 172.21.15.152:0/2539690099 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b300c8a80 0x7f4b300c8f00 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f4b300d2010 tx=0x7f4b28009dc0 comp rx=0 tx=0).stop 2024-04-20T11:33:30.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:30.860+0000 7f4b35000640 1 -- 172.21.15.152:0/2539690099 shutdown_connections 2024-04-20T11:33:30.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:30.860+0000 7f4b35000640 1 --2- 172.21.15.152:0/2539690099 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4b0c06ca40 0x7f4b0c06ef00 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:30.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:30.860+0000 7f4b35000640 1 --2- 172.21.15.152:0/2539690099 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b300c8a80 0x7f4b300c8f00 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:30.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:30.860+0000 7f4b35000640 1 --2- 172.21.15.152:0/2539690099 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4b300b8f70 0x7f4b300b93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:30.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:30.860+0000 7f4b35000640 1 --2- 172.21.15.152:0/2539690099 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4b14004800 0x7f4b3014d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:30.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:30.860+0000 7f4b35000640 1 -- 172.21.15.152:0/2539690099 >> 172.21.15.152:0/2539690099 conn(0x7f4b3008bfd0 msgr2=0x7f4b300b1130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:30.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:30.860+0000 7f4b35000640 1 -- 172.21.15.152:0/2539690099 shutdown_connections 2024-04-20T11:33:30.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:30.860+0000 7f4b35000640 1 -- 172.21.15.152:0/2539690099 wait complete. 2024-04-20T11:33:30.881 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-20T11:33:30.881 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:33:31.073 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.073+0000 7fd39a400640 1 Processor -- start 2024-04-20T11:33:31.074 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.073+0000 7fd39a400640 1 -- start start 2024-04-20T11:33:31.075 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.075+0000 7fd39a400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd39413e530 0x7fd39406ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:31.075 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.075+0000 7fd39a400640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3940cf0a0 con 0x7fd39406f340 2024-04-20T11:33:31.075 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.075+0000 7fd39a400640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3940d00e0 con 0x7fd3941414e0 2024-04-20T11:33:31.075 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.075+0000 7fd39a400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd394141b00 con 0x7fd39413e530 2024-04-20T11:33:31.076 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.075+0000 7fd393e00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fd39406f340 0x7fd394140dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:55128/0 (socket says 172.21.15.152:55128) 2024-04-20T11:33:31.076 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.075+0000 7fd393e00640 1 -- 172.21.15.152:0/1428647040 learned_addr learned my addr 172.21.15.152:0/1428647040 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:33:31.078 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.075+0000 7fd399000640 1 --2- 172.21.15.152:0/1428647040 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd39413e530 0x7fd39406ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:31.078 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.075+0000 7fd393400640 1 -- 172.21.15.152:0/1428647040 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 134870492 0 0) 0x7fd380001110 con 0x7fd39406f340 2024-04-20T11:33:31.078 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.075+0000 7fd393400640 1 -- 172.21.15.152:0/1428647040 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd37c005730 con 0x7fd39406f340 2024-04-20T11:33:31.078 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.076+0000 7fd393400640 1 -- 172.21.15.152:0/1428647040 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2846217516 0 0) 0x7fd37c005730 con 0x7fd39406f340 2024-04-20T11:33:31.078 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.076+0000 7fd393400640 1 -- 172.21.15.152:0/1428647040 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd39413e530 msgr2=0x7fd39406ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:31.078 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.076+0000 7fd393400640 1 --2- 172.21.15.152:0/1428647040 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd39413e530 0x7fd39406ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:31.078 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.076+0000 7fd393400640 1 -- 172.21.15.152:0/1428647040 >> v1:172.21.15.175:6789/0 conn(0x7fd3941414e0 legacy=0x7fd394071cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:31.078 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.076+0000 7fd393400640 1 -- 172.21.15.152:0/1428647040 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3940cf320 con 0x7fd39406f340 2024-04-20T11:33:31.078 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.076+0000 7fd393400640 1 -- 172.21.15.152:0/1428647040 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7fd380001840 con 0x7fd39406f340 2024-04-20T11:33:31.078 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.076+0000 7fd393400640 1 -- 172.21.15.152:0/1428647040 >> v1:172.21.15.152:6789/0 conn(0x7fd39406f340 legacy=0x7fd394140dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:31.078 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.076+0000 7fd393400640 1 --2- 172.21.15.152:0/1428647040 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd37c005bf0 0x7fd37c006010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:31.078 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.076+0000 7fd393400640 1 --2- 172.21.15.152:0/1428647040 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd37c006a10 0x7fd37c006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:31.079 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.076+0000 7fd393400640 1 --2- 172.21.15.152:0/1428647040 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd37c0073d0 0x7fd37c00dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:31.079 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.076+0000 7fd393400640 1 -- 172.21.15.152:0/1428647040 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fd37c00e240 con 0x7fd37c005bf0 2024-04-20T11:33:31.079 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.076+0000 7fd393400640 1 -- 172.21.15.152:0/1428647040 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd37c00e3f0 con 0x7fd37c006a10 2024-04-20T11:33:31.079 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.076+0000 7fd393400640 1 -- 172.21.15.152:0/1428647040 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd37c00e5f0 con 0x7fd37c0073d0 2024-04-20T11:33:31.079 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.076+0000 7fd399000640 1 --2- 172.21.15.152:0/1428647040 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd39413e530 0x7fd39406ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:33:31.079 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.076+0000 7fd399000640 1 --2- 172.21.15.152:0/1428647040 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd37c005bf0 0x7fd37c006010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:31.079 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.077+0000 7fd393e00640 1 --2- 172.21.15.152:0/1428647040 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd37c006a10 0x7fd37c006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:31.079 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.077+0000 7fd399000640 1 -- 172.21.15.152:0/1428647040 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd37c0073d0 msgr2=0x7fd37c00dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:31.079 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.077+0000 7fd399000640 1 --2- 172.21.15.152:0/1428647040 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd37c0073d0 0x7fd37c00dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:31.079 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.077+0000 7fd399000640 1 -- 172.21.15.152:0/1428647040 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd37c006a10 msgr2=0x7fd37c006e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:31.079 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.077+0000 7fd399000640 1 --2- 172.21.15.152:0/1428647040 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd37c006a10 0x7fd37c006e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:31.079 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.077+0000 7fd399000640 1 -- 172.21.15.152:0/1428647040 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd37c005730 con 0x7fd37c005bf0 2024-04-20T11:33:31.079 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.077+0000 7fd393e00640 1 --2- 172.21.15.152:0/1428647040 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd37c006a10 0x7fd37c006e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:33:31.081 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.078+0000 7fd399000640 1 --2- 172.21.15.152:0/1428647040 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd37c005bf0 0x7fd37c006010 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7fd388006fd0 tx=0x7fd37c007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ea6a8e8449d44ce server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:31.081 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.078+0000 7fd393400640 1 -- 172.21.15.152:0/1428647040 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd38800f040 con 0x7fd37c005bf0 2024-04-20T11:33:31.081 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.078+0000 7fd393400640 1 -- 172.21.15.152:0/1428647040 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd388033600 con 0x7fd37c005bf0 2024-04-20T11:33:31.081 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.078+0000 7fd393400640 1 -- 172.21.15.152:0/1428647040 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3880339d0 con 0x7fd37c005bf0 2024-04-20T11:33:31.081 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.079+0000 7fd39a400640 1 -- 172.21.15.152:0/1428647040 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd37c005bf0 msgr2=0x7fd37c006010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:31.081 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.079+0000 7fd39a400640 1 --2- 172.21.15.152:0/1428647040 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd37c005bf0 0x7fd37c006010 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7fd388006fd0 tx=0x7fd37c007a40 comp rx=0 tx=0).stop 2024-04-20T11:33:31.081 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.079+0000 7fd399a00640 1 -- 172.21.15.152:0/1428647040 reap_dead start 2024-04-20T11:33:31.081 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.079+0000 7fd39a400640 1 -- 172.21.15.152:0/1428647040 shutdown_connections 2024-04-20T11:33:31.081 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.079+0000 7fd39a400640 1 -- 172.21.15.152:0/1428647040 >> 172.21.15.152:0/1428647040 conn(0x7fd394067ae0 msgr2=0x7fd39406b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:31.081 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.079+0000 7fd39a400640 1 -- 172.21.15.152:0/1428647040 shutdown_connections 2024-04-20T11:33:31.082 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.079+0000 7fd39a400640 1 -- 172.21.15.152:0/1428647040 wait complete. 2024-04-20T11:33:31.082 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.080+0000 7fd39a400640 1 Processor -- start 2024-04-20T11:33:31.082 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.081+0000 7fd39a400640 1 -- start start 2024-04-20T11:33:31.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.081+0000 7fd39a400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd37c005bf0 0x7fd394133d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:31.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.081+0000 7fd39a400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd39415a950 0x7fd3941321b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:31.084 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.082+0000 7fd39a400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd3941326f0 0x7fd3940b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:31.084 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.082+0000 7fd39a400640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fd3940cf8e0 con 0x7fd39415a950 2024-04-20T11:33:31.084 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.082+0000 7fd39a400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd3940cfc50 con 0x7fd37c005bf0 2024-04-20T11:33:31.084 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.082+0000 7fd39a400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd3940723e0 con 0x7fd3941326f0 2024-04-20T11:33:31.084 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.082+0000 7fd393e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd39415a950 0x7fd3941321b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:31.084 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.082+0000 7fd393e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd39415a950 0x7fd3941321b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54248/0 (socket says 172.21.15.152:54248) 2024-04-20T11:33:31.084 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.082+0000 7fd393e00640 1 -- 172.21.15.152:0/2746913971 learned_addr learned my addr 172.21.15.152:0/2746913971 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:33:31.084 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.082+0000 7fd393e00640 1 -- 172.21.15.152:0/2746913971 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd3941326f0 msgr2=0x7fd3940b2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:31.084 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.082+0000 7fd393e00640 1 --2- 172.21.15.152:0/2746913971 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd3941326f0 0x7fd3940b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:31.085 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.082+0000 7fd393e00640 1 -- 172.21.15.152:0/2746913971 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd37c005bf0 msgr2=0x7fd394133d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:31.085 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.082+0000 7fd393e00640 1 --2- 172.21.15.152:0/2746913971 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd37c005bf0 0x7fd394133d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:31.085 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.082+0000 7fd393e00640 1 -- 172.21.15.152:0/2746913971 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd394132cd0 con 0x7fd39415a950 2024-04-20T11:33:31.085 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.082+0000 7fd393e00640 1 --2- 172.21.15.152:0/2746913971 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd39415a950 0x7fd3941321b0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7fd3800013d0 tx=0x7fd38000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:31.085 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.083+0000 7fd390c00640 1 -- 172.21.15.152:0/2746913971 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd380019070 con 0x7fd39415a950 2024-04-20T11:33:31.085 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.083+0000 7fd390c00640 1 -- 172.21.15.152:0/2746913971 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3800074e0 con 0x7fd39415a950 2024-04-20T11:33:31.085 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.083+0000 7fd390c00640 1 -- 172.21.15.152:0/2746913971 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3800049e0 con 0x7fd39415a950 2024-04-20T11:33:31.085 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.083+0000 7fd39a400640 1 -- 172.21.15.152:0/2746913971 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd394132fb0 con 0x7fd39415a950 2024-04-20T11:33:31.085 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.083+0000 7fd39a400640 1 -- 172.21.15.152:0/2746913971 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd394133570 con 0x7fd39415a950 2024-04-20T11:33:31.086 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.084+0000 7fd39a400640 1 -- 172.21.15.152:0/2746913971 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd394075e60 con 0x7fd39415a950 2024-04-20T11:33:31.086 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.085+0000 7fd390c00640 1 -- 172.21.15.152:0/2746913971 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 91142+0+0 (secure 0 0 0) 0x7fd380004b80 con 0x7fd39415a950 2024-04-20T11:33:31.086 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.086+0000 7fd390c00640 1 --2- 172.21.15.152:0/2746913971 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd37406ca10 0x7fd37406eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:31.086 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.086+0000 7fd390c00640 1 -- 172.21.15.152:0/2746913971 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7fd38008b8a0 con 0x7fd39415a950 2024-04-20T11:33:31.086 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.086+0000 7fd399000640 1 --2- 172.21.15.152:0/2746913971 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd37406ca10 0x7fd37406eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:31.088 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.087+0000 7fd399000640 1 --2- 172.21.15.152:0/2746913971 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd37406ca10 0x7fd37406eed0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd388036000 tx=0x7fd388035040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:31.090 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.090+0000 7fd390c00640 1 -- 172.21.15.152:0/2746913971 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fd3940cf0a0 con 0x7fd39415a950 2024-04-20T11:33:31.333 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.333+0000 7fd39a400640 1 -- 172.21.15.152:0/2746913971 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd3940b3490 con 0x7fd39415a950 2024-04-20T11:33:31.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.334+0000 7fd390c00640 1 -- 172.21.15.152:0/2746913971 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) ==== 94+0+5210 (secure 0 0 0) 0x7fd380059aa0 con 0x7fd39415a950 2024-04-20T11:33:31.335 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:33:31.335 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":66,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5648,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6834/387198089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":387198089},{"type":"v1","addr":"172.21.15.152:6834","nonce":387198089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":6154,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/1674013715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1674013715},{"type":"v1","addr":"172.21.15.175:6833","nonce":1674013715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":6157,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/3898238490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3898238490},{"type":"v1","addr":"172.21.15.175:6835","nonce":3898238490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T11:33:27.277882+0000","modified":"2024-04-20T11:33:30.342429+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7794},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7794":{"gid":7794,"name":"a","rank":0,"incarnation":62,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6835/3582312712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6833","nonce":3582312712},{"type":"v1","addr":"172.21.15.152:6835","nonce":3582312712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7794,"qdb_cluster":[7794]},"id":8}]} 2024-04-20T11:33:31.335 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 66 2024-04-20T11:33:31.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.337+0000 7fd39a400640 1 -- 172.21.15.152:0/2746913971 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd37406ca10 msgr2=0x7fd37406eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:31.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.337+0000 7fd39a400640 1 --2- 172.21.15.152:0/2746913971 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd37406ca10 0x7fd37406eed0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd388036000 tx=0x7fd388035040 comp rx=0 tx=0).stop 2024-04-20T11:33:31.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.337+0000 7fd39a400640 1 -- 172.21.15.152:0/2746913971 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd39415a950 msgr2=0x7fd3941321b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:31.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.337+0000 7fd39a400640 1 --2- 172.21.15.152:0/2746913971 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd39415a950 0x7fd3941321b0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7fd3800013d0 tx=0x7fd38000dd40 comp rx=0 tx=0).stop 2024-04-20T11:33:31.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.337+0000 7fd39a400640 1 -- 172.21.15.152:0/2746913971 shutdown_connections 2024-04-20T11:33:31.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.337+0000 7fd39a400640 1 --2- 172.21.15.152:0/2746913971 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd37406ca10 0x7fd37406eed0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:31.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.337+0000 7fd39a400640 1 --2- 172.21.15.152:0/2746913971 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd3941326f0 0x7fd3940b2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:31.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.337+0000 7fd39a400640 1 --2- 172.21.15.152:0/2746913971 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd39415a950 0x7fd3941321b0 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:31.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.337+0000 7fd39a400640 1 --2- 172.21.15.152:0/2746913971 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd37c005bf0 0x7fd394133d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:31.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.337+0000 7fd39a400640 1 -- 172.21.15.152:0/2746913971 >> 172.21.15.152:0/2746913971 conn(0x7fd394067ae0 msgr2=0x7fd39415d540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:31.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.337+0000 7fd39a400640 1 -- 172.21.15.152:0/2746913971 shutdown_connections 2024-04-20T11:33:31.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.337+0000 7fd39a400640 1 -- 172.21.15.152:0/2746913971 wait complete. 2024-04-20T11:33:31.357 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:33:31.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.557+0000 7ff450c00640 1 Processor -- start 2024-04-20T11:33:31.558 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.557+0000 7ff450c00640 1 -- start start 2024-04-20T11:33:31.559 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.559+0000 7ff450c00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ff44c147410 0x7ff44c0afa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:31.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.559+0000 7ff450c00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7ff44c06e720 0x7ff44c072130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:31.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.559+0000 7ff450c00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff44c0aec90 con 0x7ff44c071cd0 2024-04-20T11:33:31.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.559+0000 7ff450c00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7ff44c072670 con 0x7ff44c06e720 2024-04-20T11:33:31.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.559+0000 7ff450c00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7ff44c0aef70 con 0x7ff44c147410 2024-04-20T11:33:31.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.559+0000 7ff44be00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7ff44c06e720 0x7ff44c072130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:31.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.559+0000 7ff44aa00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7ff44c071cd0 0x7ff44c06e010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:55144/0 (socket says 172.21.15.152:55144) 2024-04-20T11:33:31.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.559+0000 7ff44aa00640 1 -- 172.21.15.152:0/492525921 learned_addr learned my addr 172.21.15.152:0/492525921 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:33:31.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.559+0000 7ff44b400640 1 --2- 172.21.15.152:0/492525921 >> v2:172.21.15.175:3300/0 conn(0x7ff44c147410 0x7ff44c0afa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:31.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.560+0000 7ff44be00640 1 -- 172.21.15.152:0/492525921 >> v2:172.21.15.175:3300/0 conn(0x7ff44c147410 msgr2=0x7ff44c0afa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:31.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.560+0000 7ff44be00640 1 --2- 172.21.15.152:0/492525921 >> v2:172.21.15.175:3300/0 conn(0x7ff44c147410 0x7ff44c0afa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:31.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.560+0000 7ff44be00640 1 -- 172.21.15.152:0/492525921 >> v1:172.21.15.152:6789/0 conn(0x7ff44c071cd0 legacy=0x7ff44c06e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:31.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.560+0000 7ff44a000640 1 -- 172.21.15.152:0/492525921 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2871334372 0 0) 0x7ff44c0aec90 con 0x7ff44c071cd0 2024-04-20T11:33:31.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.560+0000 7ff44be00640 1 -- 172.21.15.152:0/492525921 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff44c0af250 con 0x7ff44c06e720 2024-04-20T11:33:31.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.560+0000 7ff44b400640 1 --2- 172.21.15.152:0/492525921 >> v2:172.21.15.175:3300/0 conn(0x7ff44c147410 0x7ff44c0afa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:33:31.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.561+0000 7ff44be00640 1 --2- 172.21.15.152:0/492525921 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff44c06e720 0x7ff44c072130 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7ff44c0b30e0 tx=0x7ff44402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31f9691612df58eb server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:31.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.561+0000 7ff44a000640 1 -- 172.21.15.152:0/492525921 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff444039070 con 0x7ff44c06e720 2024-04-20T11:33:31.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.561+0000 7ff44a000640 1 -- 172.21.15.152:0/492525921 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff44402cc80 con 0x7ff44c06e720 2024-04-20T11:33:31.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.561+0000 7ff44a000640 1 -- 172.21.15.152:0/492525921 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff444034660 con 0x7ff44c06e720 2024-04-20T11:33:31.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.562+0000 7ff450c00640 1 -- 172.21.15.152:0/492525921 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff44c06e720 msgr2=0x7ff44c072130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:31.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.562+0000 7ff450c00640 1 --2- 172.21.15.152:0/492525921 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff44c06e720 0x7ff44c072130 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7ff44c0b30e0 tx=0x7ff44402bed0 comp rx=0 tx=0).stop 2024-04-20T11:33:31.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.562+0000 7ff450c00640 1 -- 172.21.15.152:0/492525921 shutdown_connections 2024-04-20T11:33:31.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.562+0000 7ff450c00640 1 --2- 172.21.15.152:0/492525921 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff44c06e720 0x7ff44c072130 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:31.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.562+0000 7ff450c00640 1 --2- 172.21.15.152:0/492525921 >> v2:172.21.15.175:3300/0 conn(0x7ff44c147410 0x7ff44c0afa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:31.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.562+0000 7ff450c00640 1 -- 172.21.15.152:0/492525921 >> 172.21.15.152:0/492525921 conn(0x7ff44c067ae0 msgr2=0x7ff44c06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:31.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.562+0000 7ff450c00640 1 -- 172.21.15.152:0/492525921 shutdown_connections 2024-04-20T11:33:31.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.562+0000 7ff450c00640 1 -- 172.21.15.152:0/492525921 wait complete. 2024-04-20T11:33:31.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.564+0000 7ff450c00640 1 Processor -- start 2024-04-20T11:33:31.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.564+0000 7ff450c00640 1 -- start start 2024-04-20T11:33:31.567 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.565+0000 7ff450c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff44c06e720 0x7ff44c0af8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:31.567 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.565+0000 7ff450c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff44c071cd0 0x7ff44c143010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:31.567 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.565+0000 7ff450c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff44c147410 0x7ff44c0b1310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:31.567 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.565+0000 7ff450c00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7ff44c0b3390 con 0x7ff44c147410 2024-04-20T11:33:31.567 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.565+0000 7ff450c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff44c0ae980 con 0x7ff44c071cd0 2024-04-20T11:33:31.567 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.565+0000 7ff450c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff44c0b3510 con 0x7ff44c06e720 2024-04-20T11:33:31.567 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.565+0000 7ff44be00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff44c147410 0x7ff44c0b1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:31.567 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.565+0000 7ff44be00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff44c147410 0x7ff44c0b1310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54258/0 (socket says 172.21.15.152:54258) 2024-04-20T11:33:31.567 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.565+0000 7ff44be00640 1 -- 172.21.15.152:0/1488160106 learned_addr learned my addr 172.21.15.152:0/1488160106 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:33:31.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.565+0000 7ff44aa00640 1 --2- 172.21.15.152:0/1488160106 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff44c071cd0 0x7ff44c143010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:31.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.565+0000 7ff44b400640 1 --2- 172.21.15.152:0/1488160106 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff44c06e720 0x7ff44c0af8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:31.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.565+0000 7ff44be00640 1 -- 172.21.15.152:0/1488160106 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff44c06e720 msgr2=0x7ff44c0af8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:31.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.565+0000 7ff44be00640 1 --2- 172.21.15.152:0/1488160106 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff44c06e720 0x7ff44c0af8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:31.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.565+0000 7ff44be00640 1 -- 172.21.15.152:0/1488160106 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff44c071cd0 msgr2=0x7ff44c143010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:31.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.565+0000 7ff44be00640 1 --2- 172.21.15.152:0/1488160106 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff44c071cd0 0x7ff44c143010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:31.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.565+0000 7ff44be00640 1 -- 172.21.15.152:0/1488160106 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff44c1600a0 con 0x7ff44c147410 2024-04-20T11:33:31.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.565+0000 7ff44b400640 1 --2- 172.21.15.152:0/1488160106 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff44c06e720 0x7ff44c0af8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:33:31.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.566+0000 7ff44be00640 1 --2- 172.21.15.152:0/1488160106 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff44c147410 0x7ff44c0b1310 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7ff444009e80 tx=0x7ff444004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:31.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.566+0000 7ff443400640 1 -- 172.21.15.152:0/1488160106 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff444039050 con 0x7ff44c147410 2024-04-20T11:33:31.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.566+0000 7ff443400640 1 -- 172.21.15.152:0/1488160106 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff444045070 con 0x7ff44c147410 2024-04-20T11:33:31.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.566+0000 7ff443400640 1 -- 172.21.15.152:0/1488160106 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff44403e830 con 0x7ff44c147410 2024-04-20T11:33:31.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.566+0000 7ff450c00640 1 -- 172.21.15.152:0/1488160106 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff44c0aff30 con 0x7ff44c147410 2024-04-20T11:33:31.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.566+0000 7ff450c00640 1 -- 172.21.15.152:0/1488160106 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff44c0b04f0 con 0x7ff44c147410 2024-04-20T11:33:31.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.568+0000 7ff450c00640 1 -- 172.21.15.152:0/1488160106 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff44c075e60 con 0x7ff44c147410 2024-04-20T11:33:31.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.569+0000 7ff443400640 1 -- 172.21.15.152:0/1488160106 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 91142+0+0 (secure 0 0 0) 0x7ff44403e9d0 con 0x7ff44c147410 2024-04-20T11:33:31.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.570+0000 7ff443400640 1 --2- 172.21.15.152:0/1488160106 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7ff42c06ca10 0x7ff42c06eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:31.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.570+0000 7ff443400640 1 -- 172.21.15.152:0/1488160106 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7ff4440aeb10 con 0x7ff44c147410 2024-04-20T11:33:31.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.571+0000 7ff44b400640 1 --2- 172.21.15.152:0/1488160106 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7ff42c06ca10 0x7ff42c06eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:31.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.572+0000 7ff44b400640 1 --2- 172.21.15.152:0/1488160106 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7ff42c06ca10 0x7ff42c06eed0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff43c006fd0 tx=0x7ff43c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:31.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.574+0000 7ff443400640 1 -- 172.21.15.152:0/1488160106 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ff44407ce20 con 0x7ff44c147410 2024-04-20T11:33:31.777 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.777+0000 7ff450c00640 1 -- 172.21.15.152:0/1488160106 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff44c06bc60 con 0x7ff44c147410 2024-04-20T11:33:31.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.780+0000 7ff443400640 1 -- 172.21.15.152:0/1488160106 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) ==== 74+0+16687 (secure 0 0 0) 0x7ff44407c720 con 0x7ff44c147410 2024-04-20T11:33:31.781 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:33:31.781 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":103,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:33:28.828485+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:33:25.062365+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-20T11:33:26.260945+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:33:31.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.782+0000 7ff450c00640 1 -- 172.21.15.152:0/1488160106 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7ff42c06ca10 msgr2=0x7ff42c06eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:31.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.782+0000 7ff450c00640 1 --2- 172.21.15.152:0/1488160106 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7ff42c06ca10 0x7ff42c06eed0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff43c006fd0 tx=0x7ff43c008040 comp rx=0 tx=0).stop 2024-04-20T11:33:31.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.782+0000 7ff450c00640 1 -- 172.21.15.152:0/1488160106 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff44c147410 msgr2=0x7ff44c0b1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:31.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.782+0000 7ff450c00640 1 --2- 172.21.15.152:0/1488160106 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff44c147410 0x7ff44c0b1310 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7ff444009e80 tx=0x7ff444004a40 comp rx=0 tx=0).stop 2024-04-20T11:33:31.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.782+0000 7ff450c00640 1 -- 172.21.15.152:0/1488160106 shutdown_connections 2024-04-20T11:33:31.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.782+0000 7ff450c00640 1 --2- 172.21.15.152:0/1488160106 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7ff42c06ca10 0x7ff42c06eed0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:31.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.782+0000 7ff450c00640 1 --2- 172.21.15.152:0/1488160106 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff44c147410 0x7ff44c0b1310 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:31.784 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.782+0000 7ff450c00640 1 --2- 172.21.15.152:0/1488160106 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff44c071cd0 0x7ff44c143010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:31.784 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.782+0000 7ff450c00640 1 --2- 172.21.15.152:0/1488160106 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff44c06e720 0x7ff44c0af8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:31.784 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.782+0000 7ff450c00640 1 -- 172.21.15.152:0/1488160106 >> 172.21.15.152:0/1488160106 conn(0x7ff44c067ae0 msgr2=0x7ff44c140bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:31.784 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.783+0000 7ff450c00640 1 -- 172.21.15.152:0/1488160106 shutdown_connections 2024-04-20T11:33:31.784 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.783+0000 7ff450c00640 1 -- 172.21.15.152:0/1488160106 wait complete. 2024-04-20T11:33:31.803 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-20T11:33:31.803 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T11:33:31.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.993+0000 7f498de00640 1 Processor -- start 2024-04-20T11:33:31.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.993+0000 7f498de00640 1 -- start start 2024-04-20T11:33:31.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.994+0000 7f498de00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f498815d1e0 0x7f4988160b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:31.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.995+0000 7f498de00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49880ce8c0 con 0x7f498815d5c0 2024-04-20T11:33:31.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.995+0000 7f498de00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49881600b0 con 0x7f4988071cd0 2024-04-20T11:33:31.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.995+0000 7f498de00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4988160390 con 0x7f498815d1e0 2024-04-20T11:33:31.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.995+0000 7f4987e00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f498815d5c0 0x7f49880cdfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:55156/0 (socket says 172.21.15.152:55156) 2024-04-20T11:33:31.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.995+0000 7f4987e00640 1 -- 172.21.15.152:0/2660822482 learned_addr learned my addr 172.21.15.152:0/2660822482 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:33:31.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.995+0000 7f498ca00640 1 --2- 172.21.15.152:0/2660822482 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f498815d1e0 0x7f4988160b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:31.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.995+0000 7f4987400640 1 -- 172.21.15.152:0/2660822482 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3697163475 0 0) 0x7f49880ce8c0 con 0x7f498815d5c0 2024-04-20T11:33:31.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.996+0000 7f4987400640 1 -- 172.21.15.152:0/2660822482 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4970003520 con 0x7f498815d5c0 2024-04-20T11:33:31.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.996+0000 7f4987400640 1 -- 172.21.15.152:0/2660822482 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2834359364 0 0) 0x7f4970003520 con 0x7f498815d5c0 2024-04-20T11:33:31.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.996+0000 7f4987400640 1 -- 172.21.15.152:0/2660822482 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f498815d1e0 msgr2=0x7f4988160b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:32.000 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.996+0000 7f4987400640 1 --2- 172.21.15.152:0/2660822482 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f498815d1e0 0x7f4988160b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:32.000 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.996+0000 7f4987400640 1 -- 172.21.15.152:0/2660822482 >> v1:172.21.15.175:6789/0 conn(0x7f4988071cd0 legacy=0x7f49880ccd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:32.000 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.996+0000 7f4987400640 1 -- 172.21.15.152:0/2660822482 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4988160670 con 0x7f498815d5c0 2024-04-20T11:33:32.000 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.996+0000 7f4987400640 1 -- 172.21.15.152:0/2660822482 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f4974001670 con 0x7f498815d5c0 2024-04-20T11:33:32.000 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.996+0000 7f4987400640 1 -- 172.21.15.152:0/2660822482 >> v1:172.21.15.152:6789/0 conn(0x7f498815d5c0 legacy=0x7f49880cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:32.000 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.996+0000 7f4987400640 1 --2- 172.21.15.152:0/2660822482 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49700039e0 0x7f4970003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:32.000 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.996+0000 7f4987400640 1 --2- 172.21.15.152:0/2660822482 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4970004800 0x7f4970004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:32.001 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.996+0000 7f4987400640 1 --2- 172.21.15.152:0/2660822482 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f498815d1e0 0x7f49700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:32.001 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.996+0000 7f4987400640 1 -- 172.21.15.152:0/2660822482 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f4988160390 con 0x7f4970004800 2024-04-20T11:33:32.001 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.996+0000 7f4987400640 1 -- 172.21.15.152:0/2660822482 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f497000ba70 con 0x7f49700039e0 2024-04-20T11:33:32.001 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.996+0000 7f4987400640 1 -- 172.21.15.152:0/2660822482 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f497000bc70 con 0x7f498815d1e0 2024-04-20T11:33:32.001 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.996+0000 7f4987e00640 1 --2- 172.21.15.152:0/2660822482 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4970004800 0x7f4970004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:32.001 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.997+0000 7f498ca00640 1 --2- 172.21.15.152:0/2660822482 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49700039e0 0x7f4970003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:32.001 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.997+0000 7f498d400640 1 --2- 172.21.15.152:0/2660822482 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f498815d1e0 0x7f49700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:32.001 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.997+0000 7f4987e00640 1 -- 172.21.15.152:0/2660822482 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f498815d1e0 msgr2=0x7f49700051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:32.001 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.997+0000 7f4987e00640 1 --2- 172.21.15.152:0/2660822482 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f498815d1e0 0x7f49700051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:32.002 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.997+0000 7f4987e00640 1 -- 172.21.15.152:0/2660822482 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49700039e0 msgr2=0x7f4970003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:32.002 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.997+0000 7f4987e00640 1 --2- 172.21.15.152:0/2660822482 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49700039e0 0x7f4970003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:32.002 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.997+0000 7f4987e00640 1 -- 172.21.15.152:0/2660822482 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4970003520 con 0x7f4970004800 2024-04-20T11:33:32.002 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.998+0000 7f4987e00640 1 --2- 172.21.15.152:0/2660822482 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4970004800 0x7f4970004c80 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f497400c210 tx=0x7f4970005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18f39b42dede859d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:32.005 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.998+0000 7f4987400640 1 -- 172.21.15.152:0/2660822482 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4974007d10 con 0x7f4970004800 2024-04-20T11:33:32.005 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.998+0000 7f4987400640 1 -- 172.21.15.152:0/2660822482 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4974030070 con 0x7f4970004800 2024-04-20T11:33:32.005 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.998+0000 7f4987400640 1 -- 172.21.15.152:0/2660822482 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4974005d30 con 0x7f4970004800 2024-04-20T11:33:32.005 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.999+0000 7f498de00640 1 -- 172.21.15.152:0/2660822482 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4970004800 msgr2=0x7f4970004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:32.005 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.999+0000 7f498de00640 1 --2- 172.21.15.152:0/2660822482 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4970004800 0x7f4970004c80 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f497400c210 tx=0x7f4970005830 comp rx=0 tx=0).stop 2024-04-20T11:33:32.005 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.999+0000 7f498d400640 1 -- 172.21.15.152:0/2660822482 reap_dead start 2024-04-20T11:33:32.005 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.999+0000 7f498de00640 1 -- 172.21.15.152:0/2660822482 shutdown_connections 2024-04-20T11:33:32.006 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.999+0000 7f498de00640 1 -- 172.21.15.152:0/2660822482 >> 172.21.15.152:0/2660822482 conn(0x7f4988067ae0 msgr2=0x7f498806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:32.006 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.999+0000 7f498de00640 1 -- 172.21.15.152:0/2660822482 shutdown_connections 2024-04-20T11:33:32.006 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:31.999+0000 7f498de00640 1 -- 172.21.15.152:0/2660822482 wait complete. 2024-04-20T11:33:32.006 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.001+0000 7f498de00640 1 Processor -- start 2024-04-20T11:33:32.006 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.001+0000 7f498de00640 1 -- start start 2024-04-20T11:33:32.006 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.001+0000 7f498de00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4970004800 0x7f49881f4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:32.006 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.002+0000 7f498de00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f49880b2810 0x7f49881f7920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:32.006 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.002+0000 7f498de00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f49881f7e60 0x7f49880ae970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:32.006 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.002+0000 7f498de00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f4988160d10 con 0x7f49881f7e60 2024-04-20T11:33:32.006 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.002+0000 7f498de00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4988160b90 con 0x7f4970004800 2024-04-20T11:33:32.006 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.002+0000 7f498de00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4988160e90 con 0x7f49880b2810 2024-04-20T11:33:32.007 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.002+0000 7f498d400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f49881f7e60 0x7f49880ae970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:32.007 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.002+0000 7f498d400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f49881f7e60 0x7f49880ae970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54286/0 (socket says 172.21.15.152:54286) 2024-04-20T11:33:32.007 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.002+0000 7f498d400640 1 -- 172.21.15.152:0/324815726 learned_addr learned my addr 172.21.15.152:0/324815726 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:33:32.007 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.002+0000 7f498ca00640 1 --2- 172.21.15.152:0/324815726 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4970004800 0x7f49881f4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:32.007 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.002+0000 7f498d400640 1 -- 172.21.15.152:0/324815726 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f49880b2810 msgr2=0x7f49881f7920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:32.007 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.002+0000 7f498d400640 1 --2- 172.21.15.152:0/324815726 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f49880b2810 0x7f49881f7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:32.007 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.002+0000 7f498d400640 1 -- 172.21.15.152:0/324815726 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4970004800 msgr2=0x7f49881f4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:32.007 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.002+0000 7f498d400640 1 --2- 172.21.15.152:0/324815726 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4970004800 0x7f49881f4e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:32.007 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.002+0000 7f498d400640 1 -- 172.21.15.152:0/324815726 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49881f83a0 con 0x7f49881f7e60 2024-04-20T11:33:32.007 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.002+0000 7f498ca00640 1 --2- 172.21.15.152:0/324815726 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4970004800 0x7f49881f4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:33:32.007 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.003+0000 7f498d400640 1 --2- 172.21.15.152:0/324815726 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f49881f7e60 0x7f49880ae970 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f49880b5900 tx=0x7f498000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:32.007 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.003+0000 7f4984c00640 1 -- 172.21.15.152:0/324815726 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4980018070 con 0x7f49881f7e60 2024-04-20T11:33:32.007 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.003+0000 7f4984c00640 1 -- 172.21.15.152:0/324815726 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4980003b70 con 0x7f49881f7e60 2024-04-20T11:33:32.007 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.003+0000 7f4984c00640 1 -- 172.21.15.152:0/324815726 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4980005070 con 0x7f49881f7e60 2024-04-20T11:33:32.007 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.003+0000 7f498de00640 1 -- 172.21.15.152:0/324815726 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49881f8680 con 0x7f49881f7e60 2024-04-20T11:33:32.008 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.003+0000 7f498de00640 1 -- 172.21.15.152:0/324815726 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49881f4330 con 0x7f49881f7e60 2024-04-20T11:33:32.008 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.004+0000 7f498de00640 1 -- 172.21.15.152:0/324815726 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4988075e60 con 0x7f49881f7e60 2024-04-20T11:33:32.009 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.005+0000 7f4984c00640 1 -- 172.21.15.152:0/324815726 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 91142+0+0 (secure 0 0 0) 0x7f498001b070 con 0x7f49881f7e60 2024-04-20T11:33:32.009 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.006+0000 7f4984c00640 1 --2- 172.21.15.152:0/324815726 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f496806ca10 0x7f496806eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:32.009 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.006+0000 7f4984c00640 1 -- 172.21.15.152:0/324815726 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7f498008b630 con 0x7f49881f7e60 2024-04-20T11:33:32.009 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.007+0000 7f498ca00640 1 --2- 172.21.15.152:0/324815726 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f496806ca10 0x7f496806eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:32.009 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.008+0000 7f498ca00640 1 --2- 172.21.15.152:0/324815726 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f496806ca10 0x7f496806eed0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f49880b30b0 tx=0x7f497c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:32.011 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.010+0000 7f4984c00640 1 -- 172.21.15.152:0/324815726 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f49800598d0 con 0x7f49881f7e60 2024-04-20T11:33:32.193 INFO:tasks.ceph.mds.a.smithi152.stderr:2024-04-20T11:33:32.192+0000 7f970ea00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-20T11:33:32.199 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.199+0000 7f498de00640 1 -- 172.21.15.152:0/324815726 --> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f498815dbf0 con 0x7f496806ca10 2024-04-20T11:33:32.220 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.219+0000 7f4984c00640 1 -- 172.21.15.152:0/324815726 <== mgr.4109 v2:172.21.15.175:6836/4021250441 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252409 (secure 0 0 0) 0x7f498815dbf0 con 0x7f496806ca10 2024-04-20T11:33:32.224 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:33:32.225 INFO:teuthology.orchestra.run.smithi152.stderr:dumped all 2024-04-20T11:33:32.227 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.225+0000 7f498de00640 1 -- 172.21.15.152:0/324815726 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f496806ca10 msgr2=0x7f496806eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:32.227 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.225+0000 7f498de00640 1 --2- 172.21.15.152:0/324815726 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f496806ca10 0x7f496806eed0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f49880b30b0 tx=0x7f497c008040 comp rx=0 tx=0).stop 2024-04-20T11:33:32.227 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.225+0000 7f498de00640 1 -- 172.21.15.152:0/324815726 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f49881f7e60 msgr2=0x7f49880ae970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:32.227 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.225+0000 7f498de00640 1 --2- 172.21.15.152:0/324815726 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f49881f7e60 0x7f49880ae970 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f49880b5900 tx=0x7f498000dc00 comp rx=0 tx=0).stop 2024-04-20T11:33:32.227 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.225+0000 7f498de00640 1 -- 172.21.15.152:0/324815726 shutdown_connections 2024-04-20T11:33:32.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.225+0000 7f498de00640 1 --2- 172.21.15.152:0/324815726 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f496806ca10 0x7f496806eed0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:32.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.225+0000 7f498de00640 1 --2- 172.21.15.152:0/324815726 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f49881f7e60 0x7f49880ae970 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:32.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.225+0000 7f498de00640 1 --2- 172.21.15.152:0/324815726 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f49880b2810 0x7f49881f7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:32.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.225+0000 7f498de00640 1 --2- 172.21.15.152:0/324815726 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4970004800 0x7f49881f4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:32.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.226+0000 7f498de00640 1 -- 172.21.15.152:0/324815726 >> 172.21.15.152:0/324815726 conn(0x7f4988067ae0 msgr2=0x7f498815e500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:32.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.226+0000 7f498de00640 1 -- 172.21.15.152:0/324815726 shutdown_connections 2024-04-20T11:33:32.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.226+0000 7f498de00640 1 -- 172.21.15.152:0/324815726 wait complete. 2024-04-20T11:33:32.243 INFO:teuthology.orchestra.run.smithi152.stdout:{"pg_ready":true,"pg_map":{"version":69,"stamp":"2024-04-20T11:33:30.619389+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":223,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":166,"ondisk_log_size":166,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":45,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":260572,"kb_used_data":11104,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754714148,"statfs":{"total":773094113280,"available":772827287552,"internally_reserved":0,"allocated":11370496,"data_stored":5622890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12746,"internal_metadata":255315510},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":39,"apply_latency_ms":39,"commit_latency_ns":39000000,"apply_latency_ns":39000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.823283"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.862146+0000","last_change":"2024-04-20T11:33:27.848229+0000","last_active":"2024-04-20T11:33:27.862146+0000","last_peered":"2024-04-20T11:33:27.862146+0000","last_clean":"2024-04-20T11:33:27.862146+0000","last_became_active":"2024-04-20T11:33:27.847963+0000","last_became_peered":"2024-04-20T11:33:27.847963+0000","last_unstale":"2024-04-20T11:33:27.862146+0000","last_undegraded":"2024-04-20T11:33:27.862146+0000","last_fullsized":"2024-04-20T11:33:27.862146+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:05:47.568084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.856218+0000","last_change":"2024-04-20T11:33:27.847032+0000","last_active":"2024-04-20T11:33:27.856218+0000","last_peered":"2024-04-20T11:33:27.856218+0000","last_clean":"2024-04-20T11:33:27.856218+0000","last_became_active":"2024-04-20T11:33:27.846705+0000","last_became_peered":"2024-04-20T11:33:27.846705+0000","last_unstale":"2024-04-20T11:33:27.856218+0000","last_undegraded":"2024-04-20T11:33:27.856218+0000","last_fullsized":"2024-04-20T11:33:27.856218+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:36:13.415680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.859815+0000","last_change":"2024-04-20T11:33:27.854525+0000","last_active":"2024-04-20T11:33:27.859815+0000","last_peered":"2024-04-20T11:33:27.859815+0000","last_clean":"2024-04-20T11:33:27.859815+0000","last_became_active":"2024-04-20T11:33:27.854268+0000","last_became_peered":"2024-04-20T11:33:27.854268+0000","last_unstale":"2024-04-20T11:33:27.859815+0000","last_undegraded":"2024-04-20T11:33:27.859815+0000","last_fullsized":"2024-04-20T11:33:27.859815+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:45:02.469763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.862711+0000","last_change":"2024-04-20T11:33:27.851841+0000","last_active":"2024-04-20T11:33:27.862711+0000","last_peered":"2024-04-20T11:33:27.862711+0000","last_clean":"2024-04-20T11:33:27.862711+0000","last_became_active":"2024-04-20T11:33:27.850730+0000","last_became_peered":"2024-04-20T11:33:27.850730+0000","last_unstale":"2024-04-20T11:33:27.862711+0000","last_undegraded":"2024-04-20T11:33:27.862711+0000","last_fullsized":"2024-04-20T11:33:27.862711+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:57:53.334243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.837530+0000","last_change":"2024-04-20T11:33:27.834120+0000","last_active":"2024-04-20T11:33:27.837530+0000","last_peered":"2024-04-20T11:33:27.837530+0000","last_clean":"2024-04-20T11:33:27.837530+0000","last_became_active":"2024-04-20T11:33:27.833828+0000","last_became_peered":"2024-04-20T11:33:27.833828+0000","last_unstale":"2024-04-20T11:33:27.837530+0000","last_undegraded":"2024-04-20T11:33:27.837530+0000","last_fullsized":"2024-04-20T11:33:27.837530+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:04:26.805899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.845791+0000","last_change":"2024-04-20T11:33:27.840307+0000","last_active":"2024-04-20T11:33:27.845791+0000","last_peered":"2024-04-20T11:33:27.845791+0000","last_clean":"2024-04-20T11:33:27.845791+0000","last_became_active":"2024-04-20T11:33:27.837352+0000","last_became_peered":"2024-04-20T11:33:27.837352+0000","last_unstale":"2024-04-20T11:33:27.845791+0000","last_undegraded":"2024-04-20T11:33:27.845791+0000","last_fullsized":"2024-04-20T11:33:27.845791+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:35:13.974282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.855696+0000","last_change":"2024-04-20T11:33:27.850691+0000","last_active":"2024-04-20T11:33:27.855696+0000","last_peered":"2024-04-20T11:33:27.855696+0000","last_clean":"2024-04-20T11:33:27.855696+0000","last_became_active":"2024-04-20T11:33:27.850069+0000","last_became_peered":"2024-04-20T11:33:27.850069+0000","last_unstale":"2024-04-20T11:33:27.855696+0000","last_undegraded":"2024-04-20T11:33:27.855696+0000","last_fullsized":"2024-04-20T11:33:27.855696+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:44:07.022876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.860336+0000","last_change":"2024-04-20T11:33:27.851290+0000","last_active":"2024-04-20T11:33:27.860336+0000","last_peered":"2024-04-20T11:33:27.860336+0000","last_clean":"2024-04-20T11:33:27.860336+0000","last_became_active":"2024-04-20T11:33:27.850925+0000","last_became_peered":"2024-04-20T11:33:27.850925+0000","last_unstale":"2024-04-20T11:33:27.860336+0000","last_undegraded":"2024-04-20T11:33:27.860336+0000","last_fullsized":"2024-04-20T11:33:27.860336+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:56:36.336723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.860575+0000","last_change":"2024-04-20T11:33:27.846731+0000","last_active":"2024-04-20T11:33:27.860575+0000","last_peered":"2024-04-20T11:33:27.860575+0000","last_clean":"2024-04-20T11:33:27.860575+0000","last_became_active":"2024-04-20T11:33:27.845681+0000","last_became_peered":"2024-04-20T11:33:27.845681+0000","last_unstale":"2024-04-20T11:33:27.860575+0000","last_undegraded":"2024-04-20T11:33:27.860575+0000","last_fullsized":"2024-04-20T11:33:27.860575+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:51:22.920527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.838286+0000","last_change":"2024-04-20T11:33:27.834331+0000","last_active":"2024-04-20T11:33:27.838286+0000","last_peered":"2024-04-20T11:33:27.838286+0000","last_clean":"2024-04-20T11:33:27.838286+0000","last_became_active":"2024-04-20T11:33:27.834031+0000","last_became_peered":"2024-04-20T11:33:27.834031+0000","last_unstale":"2024-04-20T11:33:27.838286+0000","last_undegraded":"2024-04-20T11:33:27.838286+0000","last_fullsized":"2024-04-20T11:33:27.838286+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:55:22.848139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.861081+0000","last_change":"2024-04-20T11:33:27.854573+0000","last_active":"2024-04-20T11:33:27.861081+0000","last_peered":"2024-04-20T11:33:27.861081+0000","last_clean":"2024-04-20T11:33:27.861081+0000","last_became_active":"2024-04-20T11:33:27.852917+0000","last_became_peered":"2024-04-20T11:33:27.852917+0000","last_unstale":"2024-04-20T11:33:27.861081+0000","last_undegraded":"2024-04-20T11:33:27.861081+0000","last_fullsized":"2024-04-20T11:33:27.861081+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:17:49.663226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.847796+0000","last_change":"2024-04-20T11:33:27.838357+0000","last_active":"2024-04-20T11:33:27.847796+0000","last_peered":"2024-04-20T11:33:27.847796+0000","last_clean":"2024-04-20T11:33:27.847796+0000","last_became_active":"2024-04-20T11:33:27.838086+0000","last_became_peered":"2024-04-20T11:33:27.838086+0000","last_unstale":"2024-04-20T11:33:27.847796+0000","last_undegraded":"2024-04-20T11:33:27.847796+0000","last_fullsized":"2024-04-20T11:33:27.847796+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:46:18.445720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.862610+0000","last_change":"2024-04-20T11:33:27.860845+0000","last_active":"2024-04-20T11:33:27.862610+0000","last_peered":"2024-04-20T11:33:27.862610+0000","last_clean":"2024-04-20T11:33:27.862610+0000","last_became_active":"2024-04-20T11:33:27.860513+0000","last_became_peered":"2024-04-20T11:33:27.860513+0000","last_unstale":"2024-04-20T11:33:27.862610+0000","last_undegraded":"2024-04-20T11:33:27.862610+0000","last_fullsized":"2024-04-20T11:33:27.862610+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:59:32.524896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.858279+0000","last_change":"2024-04-20T11:33:27.853742+0000","last_active":"2024-04-20T11:33:27.858279+0000","last_peered":"2024-04-20T11:33:27.858279+0000","last_clean":"2024-04-20T11:33:27.858279+0000","last_became_active":"2024-04-20T11:33:27.853318+0000","last_became_peered":"2024-04-20T11:33:27.853318+0000","last_unstale":"2024-04-20T11:33:27.858279+0000","last_undegraded":"2024-04-20T11:33:27.858279+0000","last_fullsized":"2024-04-20T11:33:27.858279+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:00:27.531704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.846211+0000","last_change":"2024-04-20T11:33:27.841695+0000","last_active":"2024-04-20T11:33:27.846211+0000","last_peered":"2024-04-20T11:33:27.846211+0000","last_clean":"2024-04-20T11:33:27.846211+0000","last_became_active":"2024-04-20T11:33:27.841378+0000","last_became_peered":"2024-04-20T11:33:27.841378+0000","last_unstale":"2024-04-20T11:33:27.846211+0000","last_undegraded":"2024-04-20T11:33:27.846211+0000","last_fullsized":"2024-04-20T11:33:27.846211+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:13:24.052397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.862929+0000","last_change":"2024-04-20T11:33:27.857091+0000","last_active":"2024-04-20T11:33:27.862929+0000","last_peered":"2024-04-20T11:33:27.862929+0000","last_clean":"2024-04-20T11:33:27.862929+0000","last_became_active":"2024-04-20T11:33:27.856225+0000","last_became_peered":"2024-04-20T11:33:27.856225+0000","last_unstale":"2024-04-20T11:33:27.862929+0000","last_undegraded":"2024-04-20T11:33:27.862929+0000","last_fullsized":"2024-04-20T11:33:27.862929+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:27:38.288230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.860144+0000","last_change":"2024-04-20T11:33:27.851992+0000","last_active":"2024-04-20T11:33:27.860144+0000","last_peered":"2024-04-20T11:33:27.860144+0000","last_clean":"2024-04-20T11:33:27.860144+0000","last_became_active":"2024-04-20T11:33:27.850820+0000","last_became_peered":"2024-04-20T11:33:27.850820+0000","last_unstale":"2024-04-20T11:33:27.860144+0000","last_undegraded":"2024-04-20T11:33:27.860144+0000","last_fullsized":"2024-04-20T11:33:27.860144+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:54:42.147929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.860957+0000","last_change":"2024-04-20T11:33:27.858685+0000","last_active":"2024-04-20T11:33:27.860957+0000","last_peered":"2024-04-20T11:33:27.860957+0000","last_clean":"2024-04-20T11:33:27.860957+0000","last_became_active":"2024-04-20T11:33:27.857526+0000","last_became_peered":"2024-04-20T11:33:27.857526+0000","last_unstale":"2024-04-20T11:33:27.860957+0000","last_undegraded":"2024-04-20T11:33:27.860957+0000","last_fullsized":"2024-04-20T11:33:27.860957+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:59:32.524896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.862552+0000","last_change":"2024-04-20T11:33:27.859200+0000","last_active":"2024-04-20T11:33:27.862552+0000","last_peered":"2024-04-20T11:33:27.862552+0000","last_clean":"2024-04-20T11:33:27.862552+0000","last_became_active":"2024-04-20T11:33:27.858370+0000","last_became_peered":"2024-04-20T11:33:27.858370+0000","last_unstale":"2024-04-20T11:33:27.862552+0000","last_undegraded":"2024-04-20T11:33:27.862552+0000","last_fullsized":"2024-04-20T11:33:27.862552+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:54:42.147929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.860302+0000","last_change":"2024-04-20T11:33:27.852521+0000","last_active":"2024-04-20T11:33:27.860302+0000","last_peered":"2024-04-20T11:33:27.860302+0000","last_clean":"2024-04-20T11:33:27.860302+0000","last_became_active":"2024-04-20T11:33:27.850203+0000","last_became_peered":"2024-04-20T11:33:27.850203+0000","last_unstale":"2024-04-20T11:33:27.860302+0000","last_undegraded":"2024-04-20T11:33:27.860302+0000","last_fullsized":"2024-04-20T11:33:27.860302+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:30:56.853136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.856275+0000","last_change":"2024-04-20T11:33:27.847915+0000","last_active":"2024-04-20T11:33:27.856275+0000","last_peered":"2024-04-20T11:33:27.856275+0000","last_clean":"2024-04-20T11:33:27.856275+0000","last_became_active":"2024-04-20T11:33:27.844237+0000","last_became_peered":"2024-04-20T11:33:27.844237+0000","last_unstale":"2024-04-20T11:33:27.856275+0000","last_undegraded":"2024-04-20T11:33:27.856275+0000","last_fullsized":"2024-04-20T11:33:27.856275+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:00:27.531704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.861815+0000","last_change":"2024-04-20T11:33:27.853886+0000","last_active":"2024-04-20T11:33:27.861815+0000","last_peered":"2024-04-20T11:33:27.861815+0000","last_clean":"2024-04-20T11:33:27.861815+0000","last_became_active":"2024-04-20T11:33:27.853619+0000","last_became_peered":"2024-04-20T11:33:27.853619+0000","last_unstale":"2024-04-20T11:33:27.861815+0000","last_undegraded":"2024-04-20T11:33:27.861815+0000","last_fullsized":"2024-04-20T11:33:27.861815+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:39:29.783954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.860322+0000","last_change":"2024-04-20T11:33:27.854677+0000","last_active":"2024-04-20T11:33:27.860322+0000","last_peered":"2024-04-20T11:33:27.860322+0000","last_clean":"2024-04-20T11:33:27.860322+0000","last_became_active":"2024-04-20T11:33:27.854384+0000","last_became_peered":"2024-04-20T11:33:27.854384+0000","last_unstale":"2024-04-20T11:33:27.860322+0000","last_undegraded":"2024-04-20T11:33:27.860322+0000","last_fullsized":"2024-04-20T11:33:27.860322+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:09:45.877194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.862262+0000","last_change":"2024-04-20T11:33:27.857929+0000","last_active":"2024-04-20T11:33:27.862262+0000","last_peered":"2024-04-20T11:33:27.862262+0000","last_clean":"2024-04-20T11:33:27.862262+0000","last_became_active":"2024-04-20T11:33:27.857251+0000","last_became_peered":"2024-04-20T11:33:27.857251+0000","last_unstale":"2024-04-20T11:33:27.862262+0000","last_undegraded":"2024-04-20T11:33:27.862262+0000","last_fullsized":"2024-04-20T11:33:27.862262+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:01:11.471097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.860175+0000","last_change":"2024-04-20T11:33:27.852262+0000","last_active":"2024-04-20T11:33:27.860175+0000","last_peered":"2024-04-20T11:33:27.860175+0000","last_clean":"2024-04-20T11:33:27.860175+0000","last_became_active":"2024-04-20T11:33:27.851428+0000","last_became_peered":"2024-04-20T11:33:27.851428+0000","last_unstale":"2024-04-20T11:33:27.860175+0000","last_undegraded":"2024-04-20T11:33:27.860175+0000","last_fullsized":"2024-04-20T11:33:27.860175+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:16:02.482303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.846937+0000","last_change":"2024-04-20T11:33:27.844435+0000","last_active":"2024-04-20T11:33:27.846937+0000","last_peered":"2024-04-20T11:33:27.846937+0000","last_clean":"2024-04-20T11:33:27.846937+0000","last_became_active":"2024-04-20T11:33:27.844167+0000","last_became_peered":"2024-04-20T11:33:27.844167+0000","last_unstale":"2024-04-20T11:33:27.846937+0000","last_undegraded":"2024-04-20T11:33:27.846937+0000","last_fullsized":"2024-04-20T11:33:27.846937+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:06:03.077656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.845913+0000","last_change":"2024-04-20T11:33:27.838940+0000","last_active":"2024-04-20T11:33:27.845913+0000","last_peered":"2024-04-20T11:33:27.845913+0000","last_clean":"2024-04-20T11:33:27.845913+0000","last_became_active":"2024-04-20T11:33:27.838688+0000","last_became_peered":"2024-04-20T11:33:27.838688+0000","last_unstale":"2024-04-20T11:33:27.845913+0000","last_undegraded":"2024-04-20T11:33:27.845913+0000","last_fullsized":"2024-04-20T11:33:27.845913+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:38:17.170326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.862714+0000","last_change":"2024-04-20T11:33:27.859975+0000","last_active":"2024-04-20T11:33:27.862714+0000","last_peered":"2024-04-20T11:33:27.862714+0000","last_clean":"2024-04-20T11:33:27.862714+0000","last_became_active":"2024-04-20T11:33:27.858467+0000","last_became_peered":"2024-04-20T11:33:27.858467+0000","last_unstale":"2024-04-20T11:33:27.862714+0000","last_undegraded":"2024-04-20T11:33:27.862714+0000","last_fullsized":"2024-04-20T11:33:27.862714+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:44:27.265387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.860178+0000","last_change":"2024-04-20T11:33:27.853566+0000","last_active":"2024-04-20T11:33:27.860178+0000","last_peered":"2024-04-20T11:33:27.860178+0000","last_clean":"2024-04-20T11:33:27.860178+0000","last_became_active":"2024-04-20T11:33:27.853248+0000","last_became_peered":"2024-04-20T11:33:27.853248+0000","last_unstale":"2024-04-20T11:33:27.860178+0000","last_undegraded":"2024-04-20T11:33:27.860178+0000","last_fullsized":"2024-04-20T11:33:27.860178+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:59:32.524896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.843266+0000","last_change":"2024-04-20T11:33:27.836889+0000","last_active":"2024-04-20T11:33:27.843266+0000","last_peered":"2024-04-20T11:33:27.843266+0000","last_clean":"2024-04-20T11:33:27.843266+0000","last_became_active":"2024-04-20T11:33:27.834717+0000","last_became_peered":"2024-04-20T11:33:27.834717+0000","last_unstale":"2024-04-20T11:33:27.843266+0000","last_undegraded":"2024-04-20T11:33:27.843266+0000","last_fullsized":"2024-04-20T11:33:27.843266+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:50:16.162308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.860951+0000","last_change":"2024-04-20T11:33:27.847138+0000","last_active":"2024-04-20T11:33:27.860951+0000","last_peered":"2024-04-20T11:33:27.860951+0000","last_clean":"2024-04-20T11:33:27.860951+0000","last_became_active":"2024-04-20T11:33:27.844179+0000","last_became_peered":"2024-04-20T11:33:27.844179+0000","last_unstale":"2024-04-20T11:33:27.860951+0000","last_undegraded":"2024-04-20T11:33:27.860951+0000","last_fullsized":"2024-04-20T11:33:27.860951+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:05:33.427852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.846141+0000","last_change":"2024-04-20T11:33:27.841674+0000","last_active":"2024-04-20T11:33:27.846141+0000","last_peered":"2024-04-20T11:33:27.846141+0000","last_clean":"2024-04-20T11:33:27.846141+0000","last_became_active":"2024-04-20T11:33:27.841372+0000","last_became_peered":"2024-04-20T11:33:27.841372+0000","last_unstale":"2024-04-20T11:33:27.846141+0000","last_undegraded":"2024-04-20T11:33:27.846141+0000","last_fullsized":"2024-04-20T11:33:27.846141+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:12:15.817377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.847372+0000","last_change":"2024-04-20T11:33:27.843808+0000","last_active":"2024-04-20T11:33:27.847372+0000","last_peered":"2024-04-20T11:33:27.847372+0000","last_clean":"2024-04-20T11:33:27.847372+0000","last_became_active":"2024-04-20T11:33:27.843352+0000","last_became_peered":"2024-04-20T11:33:27.843352+0000","last_unstale":"2024-04-20T11:33:27.847372+0000","last_undegraded":"2024-04-20T11:33:27.847372+0000","last_fullsized":"2024-04-20T11:33:27.847372+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:37:38.864942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"102'4","reported_seq":16,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.854954+0000","last_change":"2024-04-20T11:33:26.837604+0000","last_active":"2024-04-20T11:33:27.854954+0000","last_peered":"2024-04-20T11:33:27.854954+0000","last_clean":"2024-04-20T11:33:27.854954+0000","last_became_active":"2024-04-20T11:33:26.835341+0000","last_became_peered":"2024-04-20T11:33:26.835341+0000","last_unstale":"2024-04-20T11:33:27.854954+0000","last_undegraded":"2024-04-20T11:33:27.854954+0000","last_fullsized":"2024-04-20T11:33:27.854954+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_clean_scrub_stamp":"2024-04-20T11:33:25.777424+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:10:41.417443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.859517+0000","last_change":"2024-04-20T11:33:27.855661+0000","last_active":"2024-04-20T11:33:27.859517+0000","last_peered":"2024-04-20T11:33:27.859517+0000","last_clean":"2024-04-20T11:33:27.859517+0000","last_became_active":"2024-04-20T11:33:27.853947+0000","last_became_peered":"2024-04-20T11:33:27.853947+0000","last_unstale":"2024-04-20T11:33:27.859517+0000","last_undegraded":"2024-04-20T11:33:27.859517+0000","last_fullsized":"2024-04-20T11:33:27.859517+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:30:56.853136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"102'1","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.852630+0000","last_change":"2024-04-20T11:33:26.842539+0000","last_active":"2024-04-20T11:33:27.852630+0000","last_peered":"2024-04-20T11:33:27.852630+0000","last_clean":"2024-04-20T11:33:27.852630+0000","last_became_active":"2024-04-20T11:33:26.842047+0000","last_became_peered":"2024-04-20T11:33:26.842047+0000","last_unstale":"2024-04-20T11:33:27.852630+0000","last_undegraded":"2024-04-20T11:33:27.852630+0000","last_fullsized":"2024-04-20T11:33:27.852630+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_clean_scrub_stamp":"2024-04-20T11:33:25.777424+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:52:44.015481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.842982+0000","last_change":"2024-04-20T11:33:27.838080+0000","last_active":"2024-04-20T11:33:27.842982+0000","last_peered":"2024-04-20T11:33:27.842982+0000","last_clean":"2024-04-20T11:33:27.842982+0000","last_became_active":"2024-04-20T11:33:27.837809+0000","last_became_peered":"2024-04-20T11:33:27.837809+0000","last_unstale":"2024-04-20T11:33:27.842982+0000","last_undegraded":"2024-04-20T11:33:27.842982+0000","last_fullsized":"2024-04-20T11:33:27.842982+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:06:33.521677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"102'2","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.862516+0000","last_change":"2024-04-20T11:33:26.839092+0000","last_active":"2024-04-20T11:33:27.862516+0000","last_peered":"2024-04-20T11:33:27.862516+0000","last_clean":"2024-04-20T11:33:27.862516+0000","last_became_active":"2024-04-20T11:33:26.838837+0000","last_became_peered":"2024-04-20T11:33:26.838837+0000","last_unstale":"2024-04-20T11:33:27.862516+0000","last_undegraded":"2024-04-20T11:33:27.862516+0000","last_fullsized":"2024-04-20T11:33:27.862516+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_clean_scrub_stamp":"2024-04-20T11:33:25.777424+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:20:25.159747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.862582+0000","last_change":"2024-04-20T11:33:27.859609+0000","last_active":"2024-04-20T11:33:27.862582+0000","last_peered":"2024-04-20T11:33:27.862582+0000","last_clean":"2024-04-20T11:33:27.862582+0000","last_became_active":"2024-04-20T11:33:27.858815+0000","last_became_peered":"2024-04-20T11:33:27.858815+0000","last_unstale":"2024-04-20T11:33:27.862582+0000","last_undegraded":"2024-04-20T11:33:27.862582+0000","last_fullsized":"2024-04-20T11:33:27.862582+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:16:02.482303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"102'2","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.861497+0000","last_change":"2024-04-20T11:33:26.839790+0000","last_active":"2024-04-20T11:33:27.861497+0000","last_peered":"2024-04-20T11:33:27.861497+0000","last_clean":"2024-04-20T11:33:27.861497+0000","last_became_active":"2024-04-20T11:33:26.839487+0000","last_became_peered":"2024-04-20T11:33:26.839487+0000","last_unstale":"2024-04-20T11:33:27.861497+0000","last_undegraded":"2024-04-20T11:33:27.861497+0000","last_fullsized":"2024-04-20T11:33:27.861497+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_clean_scrub_stamp":"2024-04-20T11:33:25.777424+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:16:25.270692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.836802+0000","last_change":"2024-04-20T11:33:27.832787+0000","last_active":"2024-04-20T11:33:27.836802+0000","last_peered":"2024-04-20T11:33:27.836802+0000","last_clean":"2024-04-20T11:33:27.836802+0000","last_became_active":"2024-04-20T11:33:27.832499+0000","last_became_peered":"2024-04-20T11:33:27.832499+0000","last_unstale":"2024-04-20T11:33:27.836802+0000","last_undegraded":"2024-04-20T11:33:27.836802+0000","last_fullsized":"2024-04-20T11:33:27.836802+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:37:10.682345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"102'1","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.855177+0000","last_change":"2024-04-20T11:33:26.838736+0000","last_active":"2024-04-20T11:33:27.855177+0000","last_peered":"2024-04-20T11:33:27.855177+0000","last_clean":"2024-04-20T11:33:27.855177+0000","last_became_active":"2024-04-20T11:33:26.838418+0000","last_became_peered":"2024-04-20T11:33:26.838418+0000","last_unstale":"2024-04-20T11:33:27.855177+0000","last_undegraded":"2024-04-20T11:33:27.855177+0000","last_fullsized":"2024-04-20T11:33:27.855177+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_clean_scrub_stamp":"2024-04-20T11:33:25.777424+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:35:29.275289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.860093+0000","last_change":"2024-04-20T11:33:27.850112+0000","last_active":"2024-04-20T11:33:27.860093+0000","last_peered":"2024-04-20T11:33:27.860093+0000","last_clean":"2024-04-20T11:33:27.860093+0000","last_became_active":"2024-04-20T11:33:27.844949+0000","last_became_peered":"2024-04-20T11:33:27.844949+0000","last_unstale":"2024-04-20T11:33:27.860093+0000","last_undegraded":"2024-04-20T11:33:27.860093+0000","last_fullsized":"2024-04-20T11:33:27.860093+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:45:02.469763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"102'1","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.852243+0000","last_change":"2024-04-20T11:33:26.845804+0000","last_active":"2024-04-20T11:33:27.852243+0000","last_peered":"2024-04-20T11:33:27.852243+0000","last_clean":"2024-04-20T11:33:27.852243+0000","last_became_active":"2024-04-20T11:33:26.845502+0000","last_became_peered":"2024-04-20T11:33:26.845502+0000","last_unstale":"2024-04-20T11:33:27.852243+0000","last_undegraded":"2024-04-20T11:33:27.852243+0000","last_fullsized":"2024-04-20T11:33:27.852243+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_clean_scrub_stamp":"2024-04-20T11:33:25.777424+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:57:49.108902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.849349+0000","last_change":"2024-04-20T11:33:27.842041+0000","last_active":"2024-04-20T11:33:27.849349+0000","last_peered":"2024-04-20T11:33:27.849349+0000","last_clean":"2024-04-20T11:33:27.849349+0000","last_became_active":"2024-04-20T11:33:27.841777+0000","last_became_peered":"2024-04-20T11:33:27.841777+0000","last_unstale":"2024-04-20T11:33:27.849349+0000","last_undegraded":"2024-04-20T11:33:27.849349+0000","last_fullsized":"2024-04-20T11:33:27.849349+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:31:47.662413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.834706+0000","last_change":"2024-04-20T11:33:26.840150+0000","last_active":"2024-04-20T11:33:27.834706+0000","last_peered":"2024-04-20T11:33:27.834706+0000","last_clean":"2024-04-20T11:33:27.834706+0000","last_became_active":"2024-04-20T11:33:26.839811+0000","last_became_peered":"2024-04-20T11:33:26.839811+0000","last_unstale":"2024-04-20T11:33:27.834706+0000","last_undegraded":"2024-04-20T11:33:27.834706+0000","last_fullsized":"2024-04-20T11:33:27.834706+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_clean_scrub_stamp":"2024-04-20T11:33:25.777424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:24:42.256401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.855898+0000","last_change":"2024-04-20T11:33:27.850922+0000","last_active":"2024-04-20T11:33:27.855898+0000","last_peered":"2024-04-20T11:33:27.855898+0000","last_clean":"2024-04-20T11:33:27.855898+0000","last_became_active":"2024-04-20T11:33:27.850423+0000","last_became_peered":"2024-04-20T11:33:27.850423+0000","last_unstale":"2024-04-20T11:33:27.855898+0000","last_undegraded":"2024-04-20T11:33:27.855898+0000","last_fullsized":"2024-04-20T11:33:27.855898+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:59:36.001217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.818334+0000","last_change":"2024-04-20T11:33:26.841194+0000","last_active":"2024-04-20T11:33:27.818334+0000","last_peered":"2024-04-20T11:33:27.818334+0000","last_clean":"2024-04-20T11:33:27.818334+0000","last_became_active":"2024-04-20T11:33:26.840389+0000","last_became_peered":"2024-04-20T11:33:26.840389+0000","last_unstale":"2024-04-20T11:33:27.818334+0000","last_undegraded":"2024-04-20T11:33:27.818334+0000","last_fullsized":"2024-04-20T11:33:27.818334+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_clean_scrub_stamp":"2024-04-20T11:33:25.777424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:39:28.768039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.849577+0000","last_change":"2024-04-20T11:33:27.845411+0000","last_active":"2024-04-20T11:33:27.849577+0000","last_peered":"2024-04-20T11:33:27.849577+0000","last_clean":"2024-04-20T11:33:27.849577+0000","last_became_active":"2024-04-20T11:33:27.845095+0000","last_became_peered":"2024-04-20T11:33:27.845095+0000","last_unstale":"2024-04-20T11:33:27.849577+0000","last_undegraded":"2024-04-20T11:33:27.849577+0000","last_fullsized":"2024-04-20T11:33:27.849577+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:00:56.317262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.825537+0000","last_change":"2024-04-20T11:33:26.842136+0000","last_active":"2024-04-20T11:33:27.825537+0000","last_peered":"2024-04-20T11:33:27.825537+0000","last_clean":"2024-04-20T11:33:27.825537+0000","last_became_active":"2024-04-20T11:33:26.841835+0000","last_became_peered":"2024-04-20T11:33:26.841835+0000","last_unstale":"2024-04-20T11:33:27.825537+0000","last_undegraded":"2024-04-20T11:33:27.825537+0000","last_fullsized":"2024-04-20T11:33:27.825537+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_clean_scrub_stamp":"2024-04-20T11:33:25.777424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:30:48.370537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.857524+0000","last_change":"2024-04-20T11:33:27.848104+0000","last_active":"2024-04-20T11:33:27.857524+0000","last_peered":"2024-04-20T11:33:27.857524+0000","last_clean":"2024-04-20T11:33:27.857524+0000","last_became_active":"2024-04-20T11:33:27.847552+0000","last_became_peered":"2024-04-20T11:33:27.847552+0000","last_unstale":"2024-04-20T11:33:27.857524+0000","last_undegraded":"2024-04-20T11:33:27.857524+0000","last_fullsized":"2024-04-20T11:33:27.857524+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:45:02.469763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.821147+0000","last_change":"2024-04-20T11:33:26.847147+0000","last_active":"2024-04-20T11:33:27.821147+0000","last_peered":"2024-04-20T11:33:27.821147+0000","last_clean":"2024-04-20T11:33:27.821147+0000","last_became_active":"2024-04-20T11:33:26.846404+0000","last_became_peered":"2024-04-20T11:33:26.846404+0000","last_unstale":"2024-04-20T11:33:27.821147+0000","last_undegraded":"2024-04-20T11:33:27.821147+0000","last_fullsized":"2024-04-20T11:33:27.821147+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_clean_scrub_stamp":"2024-04-20T11:33:25.777424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:45:01.453847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.862694+0000","last_change":"2024-04-20T11:33:27.859724+0000","last_active":"2024-04-20T11:33:27.862694+0000","last_peered":"2024-04-20T11:33:27.862694+0000","last_clean":"2024-04-20T11:33:27.862694+0000","last_became_active":"2024-04-20T11:33:27.858031+0000","last_became_peered":"2024-04-20T11:33:27.858031+0000","last_unstale":"2024-04-20T11:33:27.862694+0000","last_undegraded":"2024-04-20T11:33:27.862694+0000","last_fullsized":"2024-04-20T11:33:27.862694+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:41:33.281799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"102'3","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.863451+0000","last_change":"2024-04-20T11:33:26.841652+0000","last_active":"2024-04-20T11:33:27.863451+0000","last_peered":"2024-04-20T11:33:27.863451+0000","last_clean":"2024-04-20T11:33:27.863451+0000","last_became_active":"2024-04-20T11:33:26.841374+0000","last_became_peered":"2024-04-20T11:33:26.841374+0000","last_unstale":"2024-04-20T11:33:27.863451+0000","last_undegraded":"2024-04-20T11:33:27.863451+0000","last_fullsized":"2024-04-20T11:33:27.863451+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_clean_scrub_stamp":"2024-04-20T11:33:25.777424+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:41:32.265884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.855155+0000","last_change":"2024-04-20T11:33:27.849250+0000","last_active":"2024-04-20T11:33:27.855155+0000","last_peered":"2024-04-20T11:33:27.855155+0000","last_clean":"2024-04-20T11:33:27.855155+0000","last_became_active":"2024-04-20T11:33:27.848115+0000","last_became_peered":"2024-04-20T11:33:27.848115+0000","last_unstale":"2024-04-20T11:33:27.855155+0000","last_undegraded":"2024-04-20T11:33:27.855155+0000","last_fullsized":"2024-04-20T11:33:27.855155+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:28:03.773387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"102'6","reported_seq":18,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.865750+0000","last_change":"2024-04-20T11:33:26.842851+0000","last_active":"2024-04-20T11:33:27.865750+0000","last_peered":"2024-04-20T11:33:27.865750+0000","last_clean":"2024-04-20T11:33:27.865750+0000","last_became_active":"2024-04-20T11:33:26.840238+0000","last_became_peered":"2024-04-20T11:33:26.840238+0000","last_unstale":"2024-04-20T11:33:27.865750+0000","last_undegraded":"2024-04-20T11:33:27.865750+0000","last_fullsized":"2024-04-20T11:33:27.865750+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_clean_scrub_stamp":"2024-04-20T11:33:25.777424+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:18:47.905902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.847921+0000","last_change":"2024-04-20T11:33:27.843812+0000","last_active":"2024-04-20T11:33:27.847921+0000","last_peered":"2024-04-20T11:33:27.847921+0000","last_clean":"2024-04-20T11:33:27.847921+0000","last_became_active":"2024-04-20T11:33:27.843532+0000","last_became_peered":"2024-04-20T11:33:27.843532+0000","last_unstale":"2024-04-20T11:33:27.847921+0000","last_undegraded":"2024-04-20T11:33:27.847921+0000","last_fullsized":"2024-04-20T11:33:27.847921+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:59:32.524896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"102'2","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.855598+0000","last_change":"2024-04-20T11:33:26.836014+0000","last_active":"2024-04-20T11:33:27.855598+0000","last_peered":"2024-04-20T11:33:27.855598+0000","last_clean":"2024-04-20T11:33:27.855598+0000","last_became_active":"2024-04-20T11:33:26.835034+0000","last_became_peered":"2024-04-20T11:33:26.835034+0000","last_unstale":"2024-04-20T11:33:27.855598+0000","last_undegraded":"2024-04-20T11:33:27.855598+0000","last_fullsized":"2024-04-20T11:33:27.855598+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_clean_scrub_stamp":"2024-04-20T11:33:25.777424+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:30:48.370537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.862201+0000","last_change":"2024-04-20T11:33:27.857053+0000","last_active":"2024-04-20T11:33:27.862201+0000","last_peered":"2024-04-20T11:33:27.862201+0000","last_clean":"2024-04-20T11:33:27.862201+0000","last_became_active":"2024-04-20T11:33:27.856384+0000","last_became_peered":"2024-04-20T11:33:27.856384+0000","last_unstale":"2024-04-20T11:33:27.862201+0000","last_undegraded":"2024-04-20T11:33:27.862201+0000","last_fullsized":"2024-04-20T11:33:27.862201+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:22:01.734829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.846746+0000","last_change":"2024-04-20T11:33:26.847363+0000","last_active":"2024-04-20T11:33:27.846746+0000","last_peered":"2024-04-20T11:33:27.846746+0000","last_clean":"2024-04-20T11:33:27.846746+0000","last_became_active":"2024-04-20T11:33:26.846996+0000","last_became_peered":"2024-04-20T11:33:26.846996+0000","last_unstale":"2024-04-20T11:33:27.846746+0000","last_undegraded":"2024-04-20T11:33:27.846746+0000","last_fullsized":"2024-04-20T11:33:27.846746+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_clean_scrub_stamp":"2024-04-20T11:33:25.777424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:07:41.749690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.855827+0000","last_change":"2024-04-20T11:33:27.848977+0000","last_active":"2024-04-20T11:33:27.855827+0000","last_peered":"2024-04-20T11:33:27.855827+0000","last_clean":"2024-04-20T11:33:27.855827+0000","last_became_active":"2024-04-20T11:33:27.848708+0000","last_became_peered":"2024-04-20T11:33:27.848708+0000","last_unstale":"2024-04-20T11:33:27.855827+0000","last_undegraded":"2024-04-20T11:33:27.855827+0000","last_fullsized":"2024-04-20T11:33:27.855827+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:17:49.663226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"102'8","reported_seq":20,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.864373+0000","last_change":"2024-04-20T11:33:26.845361+0000","last_active":"2024-04-20T11:33:27.864373+0000","last_peered":"2024-04-20T11:33:27.864373+0000","last_clean":"2024-04-20T11:33:27.864373+0000","last_became_active":"2024-04-20T11:33:26.845092+0000","last_became_peered":"2024-04-20T11:33:26.845092+0000","last_unstale":"2024-04-20T11:33:27.864373+0000","last_undegraded":"2024-04-20T11:33:27.864373+0000","last_fullsized":"2024-04-20T11:33:27.864373+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_clean_scrub_stamp":"2024-04-20T11:33:25.777424+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:10:17.608844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.847118+0000","last_change":"2024-04-20T11:33:27.839471+0000","last_active":"2024-04-20T11:33:27.847118+0000","last_peered":"2024-04-20T11:33:27.847118+0000","last_clean":"2024-04-20T11:33:27.847118+0000","last_became_active":"2024-04-20T11:33:27.837621+0000","last_became_peered":"2024-04-20T11:33:27.837621+0000","last_unstale":"2024-04-20T11:33:27.847118+0000","last_undegraded":"2024-04-20T11:33:27.847118+0000","last_fullsized":"2024-04-20T11:33:27.847118+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:06:33.521677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"102'1","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.862825+0000","last_change":"2024-04-20T11:33:26.848198+0000","last_active":"2024-04-20T11:33:27.862825+0000","last_peered":"2024-04-20T11:33:27.862825+0000","last_clean":"2024-04-20T11:33:27.862825+0000","last_became_active":"2024-04-20T11:33:26.847929+0000","last_became_peered":"2024-04-20T11:33:26.847929+0000","last_unstale":"2024-04-20T11:33:27.862825+0000","last_undegraded":"2024-04-20T11:33:27.862825+0000","last_fullsized":"2024-04-20T11:33:27.862825+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_clean_scrub_stamp":"2024-04-20T11:33:25.777424+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:38:32.463275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.842029+0000","last_change":"2024-04-20T11:33:27.836301+0000","last_active":"2024-04-20T11:33:27.842029+0000","last_peered":"2024-04-20T11:33:27.842029+0000","last_clean":"2024-04-20T11:33:27.842029+0000","last_became_active":"2024-04-20T11:33:27.835267+0000","last_became_peered":"2024-04-20T11:33:27.835267+0000","last_unstale":"2024-04-20T11:33:27.842029+0000","last_undegraded":"2024-04-20T11:33:27.842029+0000","last_fullsized":"2024-04-20T11:33:27.842029+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:04:26.805899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"102'2","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.861540+0000","last_change":"2024-04-20T11:33:26.836478+0000","last_active":"2024-04-20T11:33:27.861540+0000","last_peered":"2024-04-20T11:33:27.861540+0000","last_clean":"2024-04-20T11:33:27.861540+0000","last_became_active":"2024-04-20T11:33:26.836224+0000","last_became_peered":"2024-04-20T11:33:26.836224+0000","last_unstale":"2024-04-20T11:33:27.861540+0000","last_undegraded":"2024-04-20T11:33:27.861540+0000","last_fullsized":"2024-04-20T11:33:27.861540+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_clean_scrub_stamp":"2024-04-20T11:33:25.777424+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:42:14.831506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.858132+0000","last_change":"2024-04-20T11:33:27.853164+0000","last_active":"2024-04-20T11:33:27.858132+0000","last_peered":"2024-04-20T11:33:27.858132+0000","last_clean":"2024-04-20T11:33:27.858132+0000","last_became_active":"2024-04-20T11:33:27.852899+0000","last_became_peered":"2024-04-20T11:33:27.852899+0000","last_unstale":"2024-04-20T11:33:27.858132+0000","last_undegraded":"2024-04-20T11:33:27.858132+0000","last_fullsized":"2024-04-20T11:33:27.858132+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:36:13.415680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"102'1","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.862047+0000","last_change":"2024-04-20T11:33:26.845248+0000","last_active":"2024-04-20T11:33:27.862047+0000","last_peered":"2024-04-20T11:33:27.862047+0000","last_clean":"2024-04-20T11:33:27.862047+0000","last_became_active":"2024-04-20T11:33:26.844912+0000","last_became_peered":"2024-04-20T11:33:26.844912+0000","last_unstale":"2024-04-20T11:33:27.862047+0000","last_undegraded":"2024-04-20T11:33:27.862047+0000","last_fullsized":"2024-04-20T11:33:27.862047+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_clean_scrub_stamp":"2024-04-20T11:33:25.777424+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:57:49.108902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.859792+0000","last_change":"2024-04-20T11:33:27.852749+0000","last_active":"2024-04-20T11:33:27.859792+0000","last_peered":"2024-04-20T11:33:27.859792+0000","last_clean":"2024-04-20T11:33:27.859792+0000","last_became_active":"2024-04-20T11:33:27.852450+0000","last_became_peered":"2024-04-20T11:33:27.852450+0000","last_unstale":"2024-04-20T11:33:27.859792+0000","last_undegraded":"2024-04-20T11:33:27.859792+0000","last_fullsized":"2024-04-20T11:33:27.859792+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:24:43.272317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.823018+0000","last_change":"2024-04-20T11:33:26.847931+0000","last_active":"2024-04-20T11:33:27.823018+0000","last_peered":"2024-04-20T11:33:27.823018+0000","last_clean":"2024-04-20T11:33:27.823018+0000","last_became_active":"2024-04-20T11:33:26.847527+0000","last_became_peered":"2024-04-20T11:33:26.847527+0000","last_unstale":"2024-04-20T11:33:27.823018+0000","last_undegraded":"2024-04-20T11:33:27.823018+0000","last_fullsized":"2024-04-20T11:33:27.823018+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_clean_scrub_stamp":"2024-04-20T11:33:25.777424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:10:47.785537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.858308+0000","last_change":"2024-04-20T11:33:27.854355+0000","last_active":"2024-04-20T11:33:27.858308+0000","last_peered":"2024-04-20T11:33:27.858308+0000","last_clean":"2024-04-20T11:33:27.858308+0000","last_became_active":"2024-04-20T11:33:27.854088+0000","last_became_peered":"2024-04-20T11:33:27.854088+0000","last_unstale":"2024-04-20T11:33:27.858308+0000","last_undegraded":"2024-04-20T11:33:27.858308+0000","last_fullsized":"2024-04-20T11:33:27.858308+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:07:42.765605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"102'2","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.863187+0000","last_change":"2024-04-20T11:33:26.831020+0000","last_active":"2024-04-20T11:33:27.863187+0000","last_peered":"2024-04-20T11:33:27.863187+0000","last_clean":"2024-04-20T11:33:27.863187+0000","last_became_active":"2024-04-20T11:33:26.829433+0000","last_became_peered":"2024-04-20T11:33:26.829433+0000","last_unstale":"2024-04-20T11:33:27.863187+0000","last_undegraded":"2024-04-20T11:33:27.863187+0000","last_fullsized":"2024-04-20T11:33:27.863187+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_clean_scrub_stamp":"2024-04-20T11:33:25.777424+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:20:25.159747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.863647+0000","last_change":"2024-04-20T11:33:27.852435+0000","last_active":"2024-04-20T11:33:27.863647+0000","last_peered":"2024-04-20T11:33:27.863647+0000","last_clean":"2024-04-20T11:33:27.863647+0000","last_became_active":"2024-04-20T11:33:27.852220+0000","last_became_peered":"2024-04-20T11:33:27.852220+0000","last_unstale":"2024-04-20T11:33:27.863647+0000","last_undegraded":"2024-04-20T11:33:27.863647+0000","last_fullsized":"2024-04-20T11:33:27.863647+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:57:04.047082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.824270+0000","last_change":"2024-04-20T11:33:26.841407+0000","last_active":"2024-04-20T11:33:27.824270+0000","last_peered":"2024-04-20T11:33:27.824270+0000","last_clean":"2024-04-20T11:33:27.824270+0000","last_became_active":"2024-04-20T11:33:26.841109+0000","last_became_peered":"2024-04-20T11:33:26.841109+0000","last_unstale":"2024-04-20T11:33:27.824270+0000","last_undegraded":"2024-04-20T11:33:27.824270+0000","last_fullsized":"2024-04-20T11:33:27.824270+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_clean_scrub_stamp":"2024-04-20T11:33:25.777424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:37:16.972405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.861467+0000","last_change":"2024-04-20T11:33:27.858903+0000","last_active":"2024-04-20T11:33:27.861467+0000","last_peered":"2024-04-20T11:33:27.861467+0000","last_clean":"2024-04-20T11:33:27.861467+0000","last_became_active":"2024-04-20T11:33:27.858436+0000","last_became_peered":"2024-04-20T11:33:27.858436+0000","last_unstale":"2024-04-20T11:33:27.861467+0000","last_undegraded":"2024-04-20T11:33:27.861467+0000","last_fullsized":"2024-04-20T11:33:27.861467+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:57:50.124818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.825801+0000","last_change":"2024-04-20T11:33:26.830948+0000","last_active":"2024-04-20T11:33:27.825801+0000","last_peered":"2024-04-20T11:33:27.825801+0000","last_clean":"2024-04-20T11:33:27.825801+0000","last_became_active":"2024-04-20T11:33:26.830668+0000","last_became_peered":"2024-04-20T11:33:26.830668+0000","last_unstale":"2024-04-20T11:33:27.825801+0000","last_undegraded":"2024-04-20T11:33:27.825801+0000","last_fullsized":"2024-04-20T11:33:27.825801+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_clean_scrub_stamp":"2024-04-20T11:33:25.777424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:42:14.831506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.838096+0000","last_change":"2024-04-20T11:33:27.833175+0000","last_active":"2024-04-20T11:33:27.838096+0000","last_peered":"2024-04-20T11:33:27.838096+0000","last_clean":"2024-04-20T11:33:27.838096+0000","last_became_active":"2024-04-20T11:33:27.831308+0000","last_became_peered":"2024-04-20T11:33:27.831308+0000","last_unstale":"2024-04-20T11:33:27.838096+0000","last_undegraded":"2024-04-20T11:33:27.838096+0000","last_fullsized":"2024-04-20T11:33:27.838096+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:30:56.853136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.818484+0000","last_change":"2024-04-20T11:33:26.847111+0000","last_active":"2024-04-20T11:33:27.818484+0000","last_peered":"2024-04-20T11:33:27.818484+0000","last_clean":"2024-04-20T11:33:27.818484+0000","last_became_active":"2024-04-20T11:33:26.846496+0000","last_became_peered":"2024-04-20T11:33:26.846496+0000","last_unstale":"2024-04-20T11:33:27.818484+0000","last_undegraded":"2024-04-20T11:33:27.818484+0000","last_fullsized":"2024-04-20T11:33:27.818484+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_clean_scrub_stamp":"2024-04-20T11:33:25.777424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:22:00.718913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.851849+0000","last_change":"2024-04-20T11:33:27.844566+0000","last_active":"2024-04-20T11:33:27.851849+0000","last_peered":"2024-04-20T11:33:27.851849+0000","last_clean":"2024-04-20T11:33:27.851849+0000","last_became_active":"2024-04-20T11:33:27.844306+0000","last_became_peered":"2024-04-20T11:33:27.844306+0000","last_unstale":"2024-04-20T11:33:27.851849+0000","last_undegraded":"2024-04-20T11:33:27.851849+0000","last_fullsized":"2024-04-20T11:33:27.851849+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:00:27.531704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"102'2","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.863547+0000","last_change":"2024-04-20T11:33:26.845533+0000","last_active":"2024-04-20T11:33:27.863547+0000","last_peered":"2024-04-20T11:33:27.863547+0000","last_clean":"2024-04-20T11:33:27.863547+0000","last_became_active":"2024-04-20T11:33:26.843541+0000","last_became_peered":"2024-04-20T11:33:26.843541+0000","last_unstale":"2024-04-20T11:33:27.863547+0000","last_undegraded":"2024-04-20T11:33:27.863547+0000","last_fullsized":"2024-04-20T11:33:27.863547+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_clean_scrub_stamp":"2024-04-20T11:33:25.777424+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:16:01.466387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.862612+0000","last_change":"2024-04-20T11:33:27.850391+0000","last_active":"2024-04-20T11:33:27.862612+0000","last_peered":"2024-04-20T11:33:27.862612+0000","last_clean":"2024-04-20T11:33:27.862612+0000","last_became_active":"2024-04-20T11:33:27.850033+0000","last_became_peered":"2024-04-20T11:33:27.850033+0000","last_unstale":"2024-04-20T11:33:27.862612+0000","last_undegraded":"2024-04-20T11:33:27.862612+0000","last_fullsized":"2024-04-20T11:33:27.862612+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:24:51.665909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.849913+0000","last_change":"2024-04-20T11:33:26.844369+0000","last_active":"2024-04-20T11:33:27.849913+0000","last_peered":"2024-04-20T11:33:27.849913+0000","last_clean":"2024-04-20T11:33:27.849913+0000","last_became_active":"2024-04-20T11:33:26.843997+0000","last_became_peered":"2024-04-20T11:33:26.843997+0000","last_unstale":"2024-04-20T11:33:27.849913+0000","last_undegraded":"2024-04-20T11:33:27.849913+0000","last_fullsized":"2024-04-20T11:33:27.849913+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_clean_scrub_stamp":"2024-04-20T11:33:25.777424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:54:41.132013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.862010+0000","last_change":"2024-04-20T11:33:27.847700+0000","last_active":"2024-04-20T11:33:27.862010+0000","last_peered":"2024-04-20T11:33:27.862010+0000","last_clean":"2024-04-20T11:33:27.862010+0000","last_became_active":"2024-04-20T11:33:27.847424+0000","last_became_peered":"2024-04-20T11:33:27.847424+0000","last_unstale":"2024-04-20T11:33:27.862010+0000","last_undegraded":"2024-04-20T11:33:27.862010+0000","last_fullsized":"2024-04-20T11:33:27.862010+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:34:43.790859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.822542+0000","last_change":"2024-04-20T11:33:26.847908+0000","last_active":"2024-04-20T11:33:27.822542+0000","last_peered":"2024-04-20T11:33:27.822542+0000","last_clean":"2024-04-20T11:33:27.822542+0000","last_became_active":"2024-04-20T11:33:26.847511+0000","last_became_peered":"2024-04-20T11:33:26.847511+0000","last_unstale":"2024-04-20T11:33:27.822542+0000","last_undegraded":"2024-04-20T11:33:27.822542+0000","last_fullsized":"2024-04-20T11:33:27.822542+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_clean_scrub_stamp":"2024-04-20T11:33:25.777424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:44:26.249472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.857195+0000","last_change":"2024-04-20T11:33:27.849915+0000","last_active":"2024-04-20T11:33:27.857195+0000","last_peered":"2024-04-20T11:33:27.857195+0000","last_clean":"2024-04-20T11:33:27.857195+0000","last_became_active":"2024-04-20T11:33:27.849274+0000","last_became_peered":"2024-04-20T11:33:27.849274+0000","last_unstale":"2024-04-20T11:33:27.857195+0000","last_undegraded":"2024-04-20T11:33:27.857195+0000","last_fullsized":"2024-04-20T11:33:27.857195+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:16:02.482303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.826735+0000","last_change":"2024-04-20T11:33:26.841918+0000","last_active":"2024-04-20T11:33:27.826735+0000","last_peered":"2024-04-20T11:33:27.826735+0000","last_clean":"2024-04-20T11:33:27.826735+0000","last_became_active":"2024-04-20T11:33:26.841359+0000","last_became_peered":"2024-04-20T11:33:26.841359+0000","last_unstale":"2024-04-20T11:33:27.826735+0000","last_undegraded":"2024-04-20T11:33:27.826735+0000","last_fullsized":"2024-04-20T11:33:27.826735+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_clean_scrub_stamp":"2024-04-20T11:33:25.777424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:54:41.132013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.860257+0000","last_change":"2024-04-20T11:33:27.851597+0000","last_active":"2024-04-20T11:33:27.860257+0000","last_peered":"2024-04-20T11:33:27.860257+0000","last_clean":"2024-04-20T11:33:27.860257+0000","last_became_active":"2024-04-20T11:33:27.851275+0000","last_became_peered":"2024-04-20T11:33:27.851275+0000","last_unstale":"2024-04-20T11:33:27.860257+0000","last_undegraded":"2024-04-20T11:33:27.860257+0000","last_fullsized":"2024-04-20T11:33:27.860257+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:57:50.124818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.822588+0000","last_change":"2024-04-20T11:33:26.842657+0000","last_active":"2024-04-20T11:33:27.822588+0000","last_peered":"2024-04-20T11:33:27.822588+0000","last_clean":"2024-04-20T11:33:27.822588+0000","last_became_active":"2024-04-20T11:33:26.842018+0000","last_became_peered":"2024-04-20T11:33:26.842018+0000","last_unstale":"2024-04-20T11:33:27.822588+0000","last_undegraded":"2024-04-20T11:33:27.822588+0000","last_fullsized":"2024-04-20T11:33:27.822588+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_clean_scrub_stamp":"2024-04-20T11:33:25.777424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:55:51.307982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.856299+0000","last_change":"2024-04-20T11:33:27.848164+0000","last_active":"2024-04-20T11:33:27.856299+0000","last_peered":"2024-04-20T11:33:27.856299+0000","last_clean":"2024-04-20T11:33:27.856299+0000","last_became_active":"2024-04-20T11:33:27.844626+0000","last_became_peered":"2024-04-20T11:33:27.844626+0000","last_unstale":"2024-04-20T11:33:27.856299+0000","last_undegraded":"2024-04-20T11:33:27.856299+0000","last_fullsized":"2024-04-20T11:33:27.856299+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:07:42.765605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.834470+0000","last_change":"2024-04-20T11:33:26.826111+0000","last_active":"2024-04-20T11:33:27.834470+0000","last_peered":"2024-04-20T11:33:27.834470+0000","last_clean":"2024-04-20T11:33:27.834470+0000","last_became_active":"2024-04-20T11:33:26.825658+0000","last_became_peered":"2024-04-20T11:33:26.825658+0000","last_unstale":"2024-04-20T11:33:27.834470+0000","last_undegraded":"2024-04-20T11:33:27.834470+0000","last_fullsized":"2024-04-20T11:33:27.834470+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_clean_scrub_stamp":"2024-04-20T11:33:25.777424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:56:35.320808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.862579+0000","last_change":"2024-04-20T11:33:27.860457+0000","last_active":"2024-04-20T11:33:27.862579+0000","last_peered":"2024-04-20T11:33:27.862579+0000","last_clean":"2024-04-20T11:33:27.862579+0000","last_became_active":"2024-04-20T11:33:27.858749+0000","last_became_peered":"2024-04-20T11:33:27.858749+0000","last_unstale":"2024-04-20T11:33:27.862579+0000","last_undegraded":"2024-04-20T11:33:27.862579+0000","last_fullsized":"2024-04-20T11:33:27.862579+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:30:56.853136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.824069+0000","last_change":"2024-04-20T11:33:26.838440+0000","last_active":"2024-04-20T11:33:27.824069+0000","last_peered":"2024-04-20T11:33:27.824069+0000","last_clean":"2024-04-20T11:33:27.824069+0000","last_became_active":"2024-04-20T11:33:26.838163+0000","last_became_peered":"2024-04-20T11:33:26.838163+0000","last_unstale":"2024-04-20T11:33:27.824069+0000","last_undegraded":"2024-04-20T11:33:27.824069+0000","last_fullsized":"2024-04-20T11:33:27.824069+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_clean_scrub_stamp":"2024-04-20T11:33:25.777424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:16:54.161688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.860737+0000","last_change":"2024-04-20T11:33:27.844461+0000","last_active":"2024-04-20T11:33:27.860737+0000","last_peered":"2024-04-20T11:33:27.860737+0000","last_clean":"2024-04-20T11:33:27.860737+0000","last_became_active":"2024-04-20T11:33:27.843813+0000","last_became_peered":"2024-04-20T11:33:27.843813+0000","last_unstale":"2024-04-20T11:33:27.860737+0000","last_undegraded":"2024-04-20T11:33:27.860737+0000","last_fullsized":"2024-04-20T11:33:27.860737+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:12:27.653618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.818651+0000","last_change":"2024-04-20T11:33:26.844681+0000","last_active":"2024-04-20T11:33:27.818651+0000","last_peered":"2024-04-20T11:33:27.818651+0000","last_clean":"2024-04-20T11:33:27.818651+0000","last_became_active":"2024-04-20T11:33:26.844405+0000","last_became_peered":"2024-04-20T11:33:26.844405+0000","last_unstale":"2024-04-20T11:33:27.818651+0000","last_undegraded":"2024-04-20T11:33:27.818651+0000","last_fullsized":"2024-04-20T11:33:27.818651+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_clean_scrub_stamp":"2024-04-20T11:33:25.777424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:01:10.455182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.861811+0000","last_change":"2024-04-20T11:33:27.852427+0000","last_active":"2024-04-20T11:33:27.861811+0000","last_peered":"2024-04-20T11:33:27.861811+0000","last_clean":"2024-04-20T11:33:27.861811+0000","last_became_active":"2024-04-20T11:33:27.852091+0000","last_became_peered":"2024-04-20T11:33:27.852091+0000","last_unstale":"2024-04-20T11:33:27.861811+0000","last_undegraded":"2024-04-20T11:33:27.861811+0000","last_fullsized":"2024-04-20T11:33:27.861811+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:09:45.877194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.824242+0000","last_change":"2024-04-20T11:33:26.848097+0000","last_active":"2024-04-20T11:33:27.824242+0000","last_peered":"2024-04-20T11:33:27.824242+0000","last_clean":"2024-04-20T11:33:27.824242+0000","last_became_active":"2024-04-20T11:33:26.847781+0000","last_became_peered":"2024-04-20T11:33:26.847781+0000","last_unstale":"2024-04-20T11:33:27.824242+0000","last_undegraded":"2024-04-20T11:33:27.824242+0000","last_fullsized":"2024-04-20T11:33:27.824242+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_clean_scrub_stamp":"2024-04-20T11:33:25.777424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:26:09.194401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"72'128","reported_seq":356,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.843920+0000","last_change":"2024-04-20T11:26:48.661897+0000","last_active":"2024-04-20T11:33:27.843920+0000","last_peered":"2024-04-20T11:33:27.843920+0000","last_clean":"2024-04-20T11:33:27.843920+0000","last_became_active":"2024-04-20T11:26:48.656535+0000","last_became_peered":"2024-04-20T11:26:48.656535+0000","last_unstale":"2024-04-20T11:33:27.843920+0000","last_undegraded":"2024-04-20T11:33:27.843920+0000","last_fullsized":"2024-04-20T11:33:27.843920+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:47.543004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:47.543004+0000","last_clean_scrub_stamp":"2024-04-20T11:26:47.543004+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:58:35.527105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574934,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32992,"kb_used_data":1808,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338848,"statfs":{"total":96636764160,"available":96602980352,"internally_reserved":0,"allocated":1851392,"data_stored":1140106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574931,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32424,"kb_used_data":1244,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339416,"statfs":{"total":96636764160,"available":96603561984,"internally_reserved":0,"allocated":1273856,"data_stored":556583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574933,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32412,"kb_used_data":1228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339428,"statfs":{"total":96636764160,"available":96603574272,"internally_reserved":0,"allocated":1257472,"data_stored":549738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574931,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32416,"kb_used_data":1240,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339424,"statfs":{"total":96636764160,"available":96603570176,"internally_reserved":0,"allocated":1269760,"data_stored":556513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574932,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32444,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339396,"statfs":{"total":96636764160,"available":96603541504,"internally_reserved":0,"allocated":1286144,"data_stored":558195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574933,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32420,"kb_used_data":1240,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339420,"statfs":{"total":96636764160,"available":96603566080,"internally_reserved":0,"allocated":1269760,"data_stored":556513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574931,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32436,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339404,"statfs":{"total":96636764160,"available":96603549696,"internally_reserved":0,"allocated":1277952,"data_stored":556637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574932,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33028,"kb_used_data":1840,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338812,"statfs":{"total":96636764160,"available":96602943488,"internally_reserved":0,"allocated":1884160,"data_stored":1148605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T11:33:32.248 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T11:33:32.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.482+0000 7f1c6da00640 1 Processor -- start 2024-04-20T11:33:32.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.483+0000 7f1c6da00640 1 -- start start 2024-04-20T11:33:32.484 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.484+0000 7f1c6da00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1c6815fcc0 0x7f1c680d4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:32.485 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.484+0000 7f1c6da00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f1c680b65c0 0x7f1c680cf240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:32.485 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.484+0000 7f1c6da00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c680d3350 con 0x7f1c681603d0 2024-04-20T11:33:32.485 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.484+0000 7f1c6da00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f1c680d3630 con 0x7f1c680b65c0 2024-04-20T11:33:32.485 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.484+0000 7f1c6da00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1c680d3910 con 0x7f1c6815fcc0 2024-04-20T11:33:32.485 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.485+0000 7f1c67e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1c6815fcc0 0x7f1c680d4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:32.485 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.485+0000 7f1c67e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1c6815fcc0 0x7f1c680d4e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.152:56862/0 (socket says 172.21.15.152:56862) 2024-04-20T11:33:32.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.485+0000 7f1c67e00640 1 -- 172.21.15.152:0/173906476 learned_addr learned my addr 172.21.15.152:0/173906476 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:33:32.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.485+0000 7f1c6d000640 1 --2- 172.21.15.152:0/173906476 >> v2:172.21.15.175:3300/0 conn(0x7f1c680b65c0 0x7f1c680cf240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:32.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.485+0000 7f1c67400640 1 --1- 172.21.15.152:0/173906476 >> v1:172.21.15.175:6789/0 conn(0x7f1c681603d0 0x7f1c680b5eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.152:53144/0 (socket says 172.21.15.152:53144) 2024-04-20T11:33:32.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.486+0000 7f1c6d000640 1 -- 172.21.15.152:0/173906476 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1c6815fcc0 msgr2=0x7f1c680d4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:32.487 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.486+0000 7f1c6d000640 1 --2- 172.21.15.152:0/173906476 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1c6815fcc0 0x7f1c680d4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:32.487 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.486+0000 7f1c6d000640 1 -- 172.21.15.152:0/173906476 >> v1:172.21.15.175:6789/0 conn(0x7f1c681603d0 legacy=0x7f1c680b5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:32.487 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.486+0000 7f1c6d000640 1 -- 172.21.15.152:0/173906476 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c680ced20 con 0x7f1c680b65c0 2024-04-20T11:33:32.487 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.486+0000 7f1c67e00640 1 --2- 172.21.15.152:0/173906476 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1c6815fcc0 0x7f1c680d4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:33:32.487 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.487+0000 7f1c6d000640 1 --2- 172.21.15.152:0/173906476 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c680b65c0 0x7f1c680cf240 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f1c60009d90 tx=0x7f1c6002beb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6faa52b6ef9148af server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:32.488 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.488+0000 7f1c66a00640 1 -- 172.21.15.152:0/173906476 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c60039070 con 0x7f1c680b65c0 2024-04-20T11:33:32.488 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.488+0000 7f1c66a00640 1 -- 172.21.15.152:0/173906476 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c6002cc60 con 0x7f1c680b65c0 2024-04-20T11:33:32.489 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.488+0000 7f1c66a00640 1 -- 172.21.15.152:0/173906476 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c60034660 con 0x7f1c680b65c0 2024-04-20T11:33:32.489 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.488+0000 7f1c6da00640 1 -- 172.21.15.152:0/173906476 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c680b65c0 msgr2=0x7f1c680cf240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:32.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.489+0000 7f1c6da00640 1 --2- 172.21.15.152:0/173906476 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c680b65c0 0x7f1c680cf240 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f1c60009d90 tx=0x7f1c6002beb0 comp rx=0 tx=0).stop 2024-04-20T11:33:32.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.489+0000 7f1c6da00640 1 -- 172.21.15.152:0/173906476 shutdown_connections 2024-04-20T11:33:32.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.489+0000 7f1c6da00640 1 --2- 172.21.15.152:0/173906476 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c680b65c0 0x7f1c680cf240 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:32.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.489+0000 7f1c6da00640 1 --2- 172.21.15.152:0/173906476 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1c6815fcc0 0x7f1c680d4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:32.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.489+0000 7f1c6da00640 1 -- 172.21.15.152:0/173906476 >> 172.21.15.152:0/173906476 conn(0x7f1c6808bfd0 msgr2=0x7f1c680d4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:32.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.489+0000 7f1c6da00640 1 -- 172.21.15.152:0/173906476 shutdown_connections 2024-04-20T11:33:32.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.489+0000 7f1c6da00640 1 -- 172.21.15.152:0/173906476 wait complete. 2024-04-20T11:33:32.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.490+0000 7f1c6da00640 1 Processor -- start 2024-04-20T11:33:32.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.491+0000 7f1c6da00640 1 -- start start 2024-04-20T11:33:32.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.491+0000 7f1c6da00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1c680b65c0 0x7f1c681f4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:32.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.492+0000 7f1c6da00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1c6815fcc0 0x7f1c680cc960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:32.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.492+0000 7f1c6da00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c681603d0 0x7f1c6815c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:32.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.492+0000 7f1c6da00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f1c680b5600 con 0x7f1c680b65c0 2024-04-20T11:33:32.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.492+0000 7f1c6da00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1c680b5480 con 0x7f1c681603d0 2024-04-20T11:33:32.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.492+0000 7f1c6da00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1c680cf780 con 0x7f1c6815fcc0 2024-04-20T11:33:32.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.492+0000 7f1c67e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1c680b65c0 0x7f1c681f4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:32.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.492+0000 7f1c67e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1c680b65c0 0x7f1c681f4d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54302/0 (socket says 172.21.15.152:54302) 2024-04-20T11:33:32.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.492+0000 7f1c67e00640 1 -- 172.21.15.152:0/4277144416 learned_addr learned my addr 172.21.15.152:0/4277144416 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:33:32.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.492+0000 7f1c67e00640 1 -- 172.21.15.152:0/4277144416 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1c6815fcc0 msgr2=0x7f1c680cc960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:32.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.492+0000 7f1c67e00640 1 --2- 172.21.15.152:0/4277144416 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1c6815fcc0 0x7f1c680cc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:32.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.492+0000 7f1c67e00640 1 -- 172.21.15.152:0/4277144416 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c681603d0 msgr2=0x7f1c6815c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:32.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.492+0000 7f1c67e00640 1 --2- 172.21.15.152:0/4277144416 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c681603d0 0x7f1c6815c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:32.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.492+0000 7f1c67e00640 1 -- 172.21.15.152:0/4277144416 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c681f5290 con 0x7f1c680b65c0 2024-04-20T11:33:32.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.492+0000 7f1c67e00640 1 --2- 172.21.15.152:0/4277144416 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1c680b65c0 0x7f1c681f4d50 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f1c5800e9d0 tx=0x7f1c5800ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:32.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.493+0000 7f1c5fe00640 1 -- 172.21.15.152:0/4277144416 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c58017070 con 0x7f1c680b65c0 2024-04-20T11:33:32.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.493+0000 7f1c5fe00640 1 -- 172.21.15.152:0/4277144416 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c5800ca60 con 0x7f1c680b65c0 2024-04-20T11:33:32.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.493+0000 7f1c6da00640 1 -- 172.21.15.152:0/4277144416 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c680cb620 con 0x7f1c680b65c0 2024-04-20T11:33:32.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.493+0000 7f1c5fe00640 1 -- 172.21.15.152:0/4277144416 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c5800ce10 con 0x7f1c680b65c0 2024-04-20T11:33:32.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.493+0000 7f1c6da00640 1 -- 172.21.15.152:0/4277144416 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c680cbe80 con 0x7f1c680b65c0 2024-04-20T11:33:32.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.494+0000 7f1c6da00640 1 -- 172.21.15.152:0/4277144416 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c68075e60 con 0x7f1c680b65c0 2024-04-20T11:33:32.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.495+0000 7f1c5fe00640 1 -- 172.21.15.152:0/4277144416 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 91142+0+0 (secure 0 0 0) 0x7f1c5801a070 con 0x7f1c680b65c0 2024-04-20T11:33:32.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.496+0000 7f1c5fe00640 1 --2- 172.21.15.152:0/4277144416 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f1c4406ca10 0x7f1c4406eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:32.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.496+0000 7f1c5fe00640 1 -- 172.21.15.152:0/4277144416 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7f1c5808bd30 con 0x7f1c680b65c0 2024-04-20T11:33:32.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.497+0000 7f1c67400640 1 --2- 172.21.15.152:0/4277144416 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f1c4406ca10 0x7f1c4406eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:32.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.498+0000 7f1c67400640 1 --2- 172.21.15.152:0/4277144416 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f1c4406ca10 0x7f1c4406eed0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1c50000fb0 tx=0x7f1c500079f0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:32.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.500+0000 7f1c5fe00640 1 -- 172.21.15.152:0/4277144416 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f1c681f5290 con 0x7f1c680b65c0 2024-04-20T11:33:32.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.683+0000 7f1c6da00640 1 -- 172.21.15.152:0/4277144416 --> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1c680ba4d0 con 0x7f1c4406ca10 2024-04-20T11:33:32.704 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.703+0000 7f1c5fe00640 1 -- 172.21.15.152:0/4277144416 <== mgr.4109 v2:172.21.15.175:6836/4021250441 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252409 (secure 0 0 0) 0x7f1c680ba4d0 con 0x7f1c4406ca10 2024-04-20T11:33:32.707 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:33:32.708 INFO:teuthology.orchestra.run.smithi152.stderr:dumped all 2024-04-20T11:33:32.710 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.708+0000 7f1c6da00640 1 -- 172.21.15.152:0/4277144416 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f1c4406ca10 msgr2=0x7f1c4406eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:32.710 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.708+0000 7f1c6da00640 1 --2- 172.21.15.152:0/4277144416 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f1c4406ca10 0x7f1c4406eed0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1c50000fb0 tx=0x7f1c500079f0 comp rx=0 tx=0).stop 2024-04-20T11:33:32.710 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.708+0000 7f1c6da00640 1 -- 172.21.15.152:0/4277144416 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1c680b65c0 msgr2=0x7f1c681f4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:32.710 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.708+0000 7f1c6da00640 1 --2- 172.21.15.152:0/4277144416 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1c680b65c0 0x7f1c681f4d50 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f1c5800e9d0 tx=0x7f1c5800ee90 comp rx=0 tx=0).stop 2024-04-20T11:33:32.710 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.708+0000 7f1c6da00640 1 -- 172.21.15.152:0/4277144416 shutdown_connections 2024-04-20T11:33:32.710 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.708+0000 7f1c6da00640 1 --2- 172.21.15.152:0/4277144416 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f1c4406ca10 0x7f1c4406eed0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:32.710 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.708+0000 7f1c6da00640 1 --2- 172.21.15.152:0/4277144416 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c681603d0 0x7f1c6815c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:32.710 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.708+0000 7f1c6da00640 1 --2- 172.21.15.152:0/4277144416 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1c6815fcc0 0x7f1c680cc960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:32.710 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.708+0000 7f1c6da00640 1 --2- 172.21.15.152:0/4277144416 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1c680b65c0 0x7f1c681f4d50 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:32.710 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.708+0000 7f1c6da00640 1 -- 172.21.15.152:0/4277144416 >> 172.21.15.152:0/4277144416 conn(0x7f1c6808bfd0 msgr2=0x7f1c680b5c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:32.711 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.708+0000 7f1c6da00640 1 -- 172.21.15.152:0/4277144416 shutdown_connections 2024-04-20T11:33:32.711 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.708+0000 7f1c6da00640 1 -- 172.21.15.152:0/4277144416 wait complete. 2024-04-20T11:33:32.725 INFO:teuthology.orchestra.run.smithi152.stdout:{"pg_ready":true,"pg_map":{"version":70,"stamp":"2024-04-20T11:33:32.619965+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":223,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":166,"ondisk_log_size":166,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":45,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":260572,"kb_used_data":11104,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754714148,"statfs":{"total":773094113280,"available":772827287552,"internally_reserved":0,"allocated":11370496,"data_stored":5622890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12746,"internal_metadata":255315510},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":39,"apply_latency_ms":39,"commit_latency_ns":39000000,"apply_latency_ns":39000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002105"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.862146+0000","last_change":"2024-04-20T11:33:27.848229+0000","last_active":"2024-04-20T11:33:27.862146+0000","last_peered":"2024-04-20T11:33:27.862146+0000","last_clean":"2024-04-20T11:33:27.862146+0000","last_became_active":"2024-04-20T11:33:27.847963+0000","last_became_peered":"2024-04-20T11:33:27.847963+0000","last_unstale":"2024-04-20T11:33:27.862146+0000","last_undegraded":"2024-04-20T11:33:27.862146+0000","last_fullsized":"2024-04-20T11:33:27.862146+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:05:47.568084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.856218+0000","last_change":"2024-04-20T11:33:27.847032+0000","last_active":"2024-04-20T11:33:27.856218+0000","last_peered":"2024-04-20T11:33:27.856218+0000","last_clean":"2024-04-20T11:33:27.856218+0000","last_became_active":"2024-04-20T11:33:27.846705+0000","last_became_peered":"2024-04-20T11:33:27.846705+0000","last_unstale":"2024-04-20T11:33:27.856218+0000","last_undegraded":"2024-04-20T11:33:27.856218+0000","last_fullsized":"2024-04-20T11:33:27.856218+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:36:13.415680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.859815+0000","last_change":"2024-04-20T11:33:27.854525+0000","last_active":"2024-04-20T11:33:27.859815+0000","last_peered":"2024-04-20T11:33:27.859815+0000","last_clean":"2024-04-20T11:33:27.859815+0000","last_became_active":"2024-04-20T11:33:27.854268+0000","last_became_peered":"2024-04-20T11:33:27.854268+0000","last_unstale":"2024-04-20T11:33:27.859815+0000","last_undegraded":"2024-04-20T11:33:27.859815+0000","last_fullsized":"2024-04-20T11:33:27.859815+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:45:02.469763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.862711+0000","last_change":"2024-04-20T11:33:27.851841+0000","last_active":"2024-04-20T11:33:27.862711+0000","last_peered":"2024-04-20T11:33:27.862711+0000","last_clean":"2024-04-20T11:33:27.862711+0000","last_became_active":"2024-04-20T11:33:27.850730+0000","last_became_peered":"2024-04-20T11:33:27.850730+0000","last_unstale":"2024-04-20T11:33:27.862711+0000","last_undegraded":"2024-04-20T11:33:27.862711+0000","last_fullsized":"2024-04-20T11:33:27.862711+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:57:53.334243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.837530+0000","last_change":"2024-04-20T11:33:27.834120+0000","last_active":"2024-04-20T11:33:27.837530+0000","last_peered":"2024-04-20T11:33:27.837530+0000","last_clean":"2024-04-20T11:33:27.837530+0000","last_became_active":"2024-04-20T11:33:27.833828+0000","last_became_peered":"2024-04-20T11:33:27.833828+0000","last_unstale":"2024-04-20T11:33:27.837530+0000","last_undegraded":"2024-04-20T11:33:27.837530+0000","last_fullsized":"2024-04-20T11:33:27.837530+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:04:26.805899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.845791+0000","last_change":"2024-04-20T11:33:27.840307+0000","last_active":"2024-04-20T11:33:27.845791+0000","last_peered":"2024-04-20T11:33:27.845791+0000","last_clean":"2024-04-20T11:33:27.845791+0000","last_became_active":"2024-04-20T11:33:27.837352+0000","last_became_peered":"2024-04-20T11:33:27.837352+0000","last_unstale":"2024-04-20T11:33:27.845791+0000","last_undegraded":"2024-04-20T11:33:27.845791+0000","last_fullsized":"2024-04-20T11:33:27.845791+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:35:13.974282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.855696+0000","last_change":"2024-04-20T11:33:27.850691+0000","last_active":"2024-04-20T11:33:27.855696+0000","last_peered":"2024-04-20T11:33:27.855696+0000","last_clean":"2024-04-20T11:33:27.855696+0000","last_became_active":"2024-04-20T11:33:27.850069+0000","last_became_peered":"2024-04-20T11:33:27.850069+0000","last_unstale":"2024-04-20T11:33:27.855696+0000","last_undegraded":"2024-04-20T11:33:27.855696+0000","last_fullsized":"2024-04-20T11:33:27.855696+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:44:07.022876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.860336+0000","last_change":"2024-04-20T11:33:27.851290+0000","last_active":"2024-04-20T11:33:27.860336+0000","last_peered":"2024-04-20T11:33:27.860336+0000","last_clean":"2024-04-20T11:33:27.860336+0000","last_became_active":"2024-04-20T11:33:27.850925+0000","last_became_peered":"2024-04-20T11:33:27.850925+0000","last_unstale":"2024-04-20T11:33:27.860336+0000","last_undegraded":"2024-04-20T11:33:27.860336+0000","last_fullsized":"2024-04-20T11:33:27.860336+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:56:36.336723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.860575+0000","last_change":"2024-04-20T11:33:27.846731+0000","last_active":"2024-04-20T11:33:27.860575+0000","last_peered":"2024-04-20T11:33:27.860575+0000","last_clean":"2024-04-20T11:33:27.860575+0000","last_became_active":"2024-04-20T11:33:27.845681+0000","last_became_peered":"2024-04-20T11:33:27.845681+0000","last_unstale":"2024-04-20T11:33:27.860575+0000","last_undegraded":"2024-04-20T11:33:27.860575+0000","last_fullsized":"2024-04-20T11:33:27.860575+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:51:22.920527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.838286+0000","last_change":"2024-04-20T11:33:27.834331+0000","last_active":"2024-04-20T11:33:27.838286+0000","last_peered":"2024-04-20T11:33:27.838286+0000","last_clean":"2024-04-20T11:33:27.838286+0000","last_became_active":"2024-04-20T11:33:27.834031+0000","last_became_peered":"2024-04-20T11:33:27.834031+0000","last_unstale":"2024-04-20T11:33:27.838286+0000","last_undegraded":"2024-04-20T11:33:27.838286+0000","last_fullsized":"2024-04-20T11:33:27.838286+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:55:22.848139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.861081+0000","last_change":"2024-04-20T11:33:27.854573+0000","last_active":"2024-04-20T11:33:27.861081+0000","last_peered":"2024-04-20T11:33:27.861081+0000","last_clean":"2024-04-20T11:33:27.861081+0000","last_became_active":"2024-04-20T11:33:27.852917+0000","last_became_peered":"2024-04-20T11:33:27.852917+0000","last_unstale":"2024-04-20T11:33:27.861081+0000","last_undegraded":"2024-04-20T11:33:27.861081+0000","last_fullsized":"2024-04-20T11:33:27.861081+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:17:49.663226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.847796+0000","last_change":"2024-04-20T11:33:27.838357+0000","last_active":"2024-04-20T11:33:27.847796+0000","last_peered":"2024-04-20T11:33:27.847796+0000","last_clean":"2024-04-20T11:33:27.847796+0000","last_became_active":"2024-04-20T11:33:27.838086+0000","last_became_peered":"2024-04-20T11:33:27.838086+0000","last_unstale":"2024-04-20T11:33:27.847796+0000","last_undegraded":"2024-04-20T11:33:27.847796+0000","last_fullsized":"2024-04-20T11:33:27.847796+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:46:18.445720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.862610+0000","last_change":"2024-04-20T11:33:27.860845+0000","last_active":"2024-04-20T11:33:27.862610+0000","last_peered":"2024-04-20T11:33:27.862610+0000","last_clean":"2024-04-20T11:33:27.862610+0000","last_became_active":"2024-04-20T11:33:27.860513+0000","last_became_peered":"2024-04-20T11:33:27.860513+0000","last_unstale":"2024-04-20T11:33:27.862610+0000","last_undegraded":"2024-04-20T11:33:27.862610+0000","last_fullsized":"2024-04-20T11:33:27.862610+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:59:32.524896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.858279+0000","last_change":"2024-04-20T11:33:27.853742+0000","last_active":"2024-04-20T11:33:27.858279+0000","last_peered":"2024-04-20T11:33:27.858279+0000","last_clean":"2024-04-20T11:33:27.858279+0000","last_became_active":"2024-04-20T11:33:27.853318+0000","last_became_peered":"2024-04-20T11:33:27.853318+0000","last_unstale":"2024-04-20T11:33:27.858279+0000","last_undegraded":"2024-04-20T11:33:27.858279+0000","last_fullsized":"2024-04-20T11:33:27.858279+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:00:27.531704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.846211+0000","last_change":"2024-04-20T11:33:27.841695+0000","last_active":"2024-04-20T11:33:27.846211+0000","last_peered":"2024-04-20T11:33:27.846211+0000","last_clean":"2024-04-20T11:33:27.846211+0000","last_became_active":"2024-04-20T11:33:27.841378+0000","last_became_peered":"2024-04-20T11:33:27.841378+0000","last_unstale":"2024-04-20T11:33:27.846211+0000","last_undegraded":"2024-04-20T11:33:27.846211+0000","last_fullsized":"2024-04-20T11:33:27.846211+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:13:24.052397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.862929+0000","last_change":"2024-04-20T11:33:27.857091+0000","last_active":"2024-04-20T11:33:27.862929+0000","last_peered":"2024-04-20T11:33:27.862929+0000","last_clean":"2024-04-20T11:33:27.862929+0000","last_became_active":"2024-04-20T11:33:27.856225+0000","last_became_peered":"2024-04-20T11:33:27.856225+0000","last_unstale":"2024-04-20T11:33:27.862929+0000","last_undegraded":"2024-04-20T11:33:27.862929+0000","last_fullsized":"2024-04-20T11:33:27.862929+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:27:38.288230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.860144+0000","last_change":"2024-04-20T11:33:27.851992+0000","last_active":"2024-04-20T11:33:27.860144+0000","last_peered":"2024-04-20T11:33:27.860144+0000","last_clean":"2024-04-20T11:33:27.860144+0000","last_became_active":"2024-04-20T11:33:27.850820+0000","last_became_peered":"2024-04-20T11:33:27.850820+0000","last_unstale":"2024-04-20T11:33:27.860144+0000","last_undegraded":"2024-04-20T11:33:27.860144+0000","last_fullsized":"2024-04-20T11:33:27.860144+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:54:42.147929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.860957+0000","last_change":"2024-04-20T11:33:27.858685+0000","last_active":"2024-04-20T11:33:27.860957+0000","last_peered":"2024-04-20T11:33:27.860957+0000","last_clean":"2024-04-20T11:33:27.860957+0000","last_became_active":"2024-04-20T11:33:27.857526+0000","last_became_peered":"2024-04-20T11:33:27.857526+0000","last_unstale":"2024-04-20T11:33:27.860957+0000","last_undegraded":"2024-04-20T11:33:27.860957+0000","last_fullsized":"2024-04-20T11:33:27.860957+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:59:32.524896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.862552+0000","last_change":"2024-04-20T11:33:27.859200+0000","last_active":"2024-04-20T11:33:27.862552+0000","last_peered":"2024-04-20T11:33:27.862552+0000","last_clean":"2024-04-20T11:33:27.862552+0000","last_became_active":"2024-04-20T11:33:27.858370+0000","last_became_peered":"2024-04-20T11:33:27.858370+0000","last_unstale":"2024-04-20T11:33:27.862552+0000","last_undegraded":"2024-04-20T11:33:27.862552+0000","last_fullsized":"2024-04-20T11:33:27.862552+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:54:42.147929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.860302+0000","last_change":"2024-04-20T11:33:27.852521+0000","last_active":"2024-04-20T11:33:27.860302+0000","last_peered":"2024-04-20T11:33:27.860302+0000","last_clean":"2024-04-20T11:33:27.860302+0000","last_became_active":"2024-04-20T11:33:27.850203+0000","last_became_peered":"2024-04-20T11:33:27.850203+0000","last_unstale":"2024-04-20T11:33:27.860302+0000","last_undegraded":"2024-04-20T11:33:27.860302+0000","last_fullsized":"2024-04-20T11:33:27.860302+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:30:56.853136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.856275+0000","last_change":"2024-04-20T11:33:27.847915+0000","last_active":"2024-04-20T11:33:27.856275+0000","last_peered":"2024-04-20T11:33:27.856275+0000","last_clean":"2024-04-20T11:33:27.856275+0000","last_became_active":"2024-04-20T11:33:27.844237+0000","last_became_peered":"2024-04-20T11:33:27.844237+0000","last_unstale":"2024-04-20T11:33:27.856275+0000","last_undegraded":"2024-04-20T11:33:27.856275+0000","last_fullsized":"2024-04-20T11:33:27.856275+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:00:27.531704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.861815+0000","last_change":"2024-04-20T11:33:27.853886+0000","last_active":"2024-04-20T11:33:27.861815+0000","last_peered":"2024-04-20T11:33:27.861815+0000","last_clean":"2024-04-20T11:33:27.861815+0000","last_became_active":"2024-04-20T11:33:27.853619+0000","last_became_peered":"2024-04-20T11:33:27.853619+0000","last_unstale":"2024-04-20T11:33:27.861815+0000","last_undegraded":"2024-04-20T11:33:27.861815+0000","last_fullsized":"2024-04-20T11:33:27.861815+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:39:29.783954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.860322+0000","last_change":"2024-04-20T11:33:27.854677+0000","last_active":"2024-04-20T11:33:27.860322+0000","last_peered":"2024-04-20T11:33:27.860322+0000","last_clean":"2024-04-20T11:33:27.860322+0000","last_became_active":"2024-04-20T11:33:27.854384+0000","last_became_peered":"2024-04-20T11:33:27.854384+0000","last_unstale":"2024-04-20T11:33:27.860322+0000","last_undegraded":"2024-04-20T11:33:27.860322+0000","last_fullsized":"2024-04-20T11:33:27.860322+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:09:45.877194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.862262+0000","last_change":"2024-04-20T11:33:27.857929+0000","last_active":"2024-04-20T11:33:27.862262+0000","last_peered":"2024-04-20T11:33:27.862262+0000","last_clean":"2024-04-20T11:33:27.862262+0000","last_became_active":"2024-04-20T11:33:27.857251+0000","last_became_peered":"2024-04-20T11:33:27.857251+0000","last_unstale":"2024-04-20T11:33:27.862262+0000","last_undegraded":"2024-04-20T11:33:27.862262+0000","last_fullsized":"2024-04-20T11:33:27.862262+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:01:11.471097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.860175+0000","last_change":"2024-04-20T11:33:27.852262+0000","last_active":"2024-04-20T11:33:27.860175+0000","last_peered":"2024-04-20T11:33:27.860175+0000","last_clean":"2024-04-20T11:33:27.860175+0000","last_became_active":"2024-04-20T11:33:27.851428+0000","last_became_peered":"2024-04-20T11:33:27.851428+0000","last_unstale":"2024-04-20T11:33:27.860175+0000","last_undegraded":"2024-04-20T11:33:27.860175+0000","last_fullsized":"2024-04-20T11:33:27.860175+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:16:02.482303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.846937+0000","last_change":"2024-04-20T11:33:27.844435+0000","last_active":"2024-04-20T11:33:27.846937+0000","last_peered":"2024-04-20T11:33:27.846937+0000","last_clean":"2024-04-20T11:33:27.846937+0000","last_became_active":"2024-04-20T11:33:27.844167+0000","last_became_peered":"2024-04-20T11:33:27.844167+0000","last_unstale":"2024-04-20T11:33:27.846937+0000","last_undegraded":"2024-04-20T11:33:27.846937+0000","last_fullsized":"2024-04-20T11:33:27.846937+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:06:03.077656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.845913+0000","last_change":"2024-04-20T11:33:27.838940+0000","last_active":"2024-04-20T11:33:27.845913+0000","last_peered":"2024-04-20T11:33:27.845913+0000","last_clean":"2024-04-20T11:33:27.845913+0000","last_became_active":"2024-04-20T11:33:27.838688+0000","last_became_peered":"2024-04-20T11:33:27.838688+0000","last_unstale":"2024-04-20T11:33:27.845913+0000","last_undegraded":"2024-04-20T11:33:27.845913+0000","last_fullsized":"2024-04-20T11:33:27.845913+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:38:17.170326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.862714+0000","last_change":"2024-04-20T11:33:27.859975+0000","last_active":"2024-04-20T11:33:27.862714+0000","last_peered":"2024-04-20T11:33:27.862714+0000","last_clean":"2024-04-20T11:33:27.862714+0000","last_became_active":"2024-04-20T11:33:27.858467+0000","last_became_peered":"2024-04-20T11:33:27.858467+0000","last_unstale":"2024-04-20T11:33:27.862714+0000","last_undegraded":"2024-04-20T11:33:27.862714+0000","last_fullsized":"2024-04-20T11:33:27.862714+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:44:27.265387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.860178+0000","last_change":"2024-04-20T11:33:27.853566+0000","last_active":"2024-04-20T11:33:27.860178+0000","last_peered":"2024-04-20T11:33:27.860178+0000","last_clean":"2024-04-20T11:33:27.860178+0000","last_became_active":"2024-04-20T11:33:27.853248+0000","last_became_peered":"2024-04-20T11:33:27.853248+0000","last_unstale":"2024-04-20T11:33:27.860178+0000","last_undegraded":"2024-04-20T11:33:27.860178+0000","last_fullsized":"2024-04-20T11:33:27.860178+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:59:32.524896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.843266+0000","last_change":"2024-04-20T11:33:27.836889+0000","last_active":"2024-04-20T11:33:27.843266+0000","last_peered":"2024-04-20T11:33:27.843266+0000","last_clean":"2024-04-20T11:33:27.843266+0000","last_became_active":"2024-04-20T11:33:27.834717+0000","last_became_peered":"2024-04-20T11:33:27.834717+0000","last_unstale":"2024-04-20T11:33:27.843266+0000","last_undegraded":"2024-04-20T11:33:27.843266+0000","last_fullsized":"2024-04-20T11:33:27.843266+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:50:16.162308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.860951+0000","last_change":"2024-04-20T11:33:27.847138+0000","last_active":"2024-04-20T11:33:27.860951+0000","last_peered":"2024-04-20T11:33:27.860951+0000","last_clean":"2024-04-20T11:33:27.860951+0000","last_became_active":"2024-04-20T11:33:27.844179+0000","last_became_peered":"2024-04-20T11:33:27.844179+0000","last_unstale":"2024-04-20T11:33:27.860951+0000","last_undegraded":"2024-04-20T11:33:27.860951+0000","last_fullsized":"2024-04-20T11:33:27.860951+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:05:33.427852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.846141+0000","last_change":"2024-04-20T11:33:27.841674+0000","last_active":"2024-04-20T11:33:27.846141+0000","last_peered":"2024-04-20T11:33:27.846141+0000","last_clean":"2024-04-20T11:33:27.846141+0000","last_became_active":"2024-04-20T11:33:27.841372+0000","last_became_peered":"2024-04-20T11:33:27.841372+0000","last_unstale":"2024-04-20T11:33:27.846141+0000","last_undegraded":"2024-04-20T11:33:27.846141+0000","last_fullsized":"2024-04-20T11:33:27.846141+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:12:15.817377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.847372+0000","last_change":"2024-04-20T11:33:27.843808+0000","last_active":"2024-04-20T11:33:27.847372+0000","last_peered":"2024-04-20T11:33:27.847372+0000","last_clean":"2024-04-20T11:33:27.847372+0000","last_became_active":"2024-04-20T11:33:27.843352+0000","last_became_peered":"2024-04-20T11:33:27.843352+0000","last_unstale":"2024-04-20T11:33:27.847372+0000","last_undegraded":"2024-04-20T11:33:27.847372+0000","last_fullsized":"2024-04-20T11:33:27.847372+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:37:38.864942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"102'4","reported_seq":16,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.854954+0000","last_change":"2024-04-20T11:33:26.837604+0000","last_active":"2024-04-20T11:33:27.854954+0000","last_peered":"2024-04-20T11:33:27.854954+0000","last_clean":"2024-04-20T11:33:27.854954+0000","last_became_active":"2024-04-20T11:33:26.835341+0000","last_became_peered":"2024-04-20T11:33:26.835341+0000","last_unstale":"2024-04-20T11:33:27.854954+0000","last_undegraded":"2024-04-20T11:33:27.854954+0000","last_fullsized":"2024-04-20T11:33:27.854954+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_clean_scrub_stamp":"2024-04-20T11:33:25.777424+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:10:41.417443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.859517+0000","last_change":"2024-04-20T11:33:27.855661+0000","last_active":"2024-04-20T11:33:27.859517+0000","last_peered":"2024-04-20T11:33:27.859517+0000","last_clean":"2024-04-20T11:33:27.859517+0000","last_became_active":"2024-04-20T11:33:27.853947+0000","last_became_peered":"2024-04-20T11:33:27.853947+0000","last_unstale":"2024-04-20T11:33:27.859517+0000","last_undegraded":"2024-04-20T11:33:27.859517+0000","last_fullsized":"2024-04-20T11:33:27.859517+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:30:56.853136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"102'1","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.852630+0000","last_change":"2024-04-20T11:33:26.842539+0000","last_active":"2024-04-20T11:33:27.852630+0000","last_peered":"2024-04-20T11:33:27.852630+0000","last_clean":"2024-04-20T11:33:27.852630+0000","last_became_active":"2024-04-20T11:33:26.842047+0000","last_became_peered":"2024-04-20T11:33:26.842047+0000","last_unstale":"2024-04-20T11:33:27.852630+0000","last_undegraded":"2024-04-20T11:33:27.852630+0000","last_fullsized":"2024-04-20T11:33:27.852630+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_clean_scrub_stamp":"2024-04-20T11:33:25.777424+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:52:44.015481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.842982+0000","last_change":"2024-04-20T11:33:27.838080+0000","last_active":"2024-04-20T11:33:27.842982+0000","last_peered":"2024-04-20T11:33:27.842982+0000","last_clean":"2024-04-20T11:33:27.842982+0000","last_became_active":"2024-04-20T11:33:27.837809+0000","last_became_peered":"2024-04-20T11:33:27.837809+0000","last_unstale":"2024-04-20T11:33:27.842982+0000","last_undegraded":"2024-04-20T11:33:27.842982+0000","last_fullsized":"2024-04-20T11:33:27.842982+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:06:33.521677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"102'2","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.862516+0000","last_change":"2024-04-20T11:33:26.839092+0000","last_active":"2024-04-20T11:33:27.862516+0000","last_peered":"2024-04-20T11:33:27.862516+0000","last_clean":"2024-04-20T11:33:27.862516+0000","last_became_active":"2024-04-20T11:33:26.838837+0000","last_became_peered":"2024-04-20T11:33:26.838837+0000","last_unstale":"2024-04-20T11:33:27.862516+0000","last_undegraded":"2024-04-20T11:33:27.862516+0000","last_fullsized":"2024-04-20T11:33:27.862516+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_clean_scrub_stamp":"2024-04-20T11:33:25.777424+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:20:25.159747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.862582+0000","last_change":"2024-04-20T11:33:27.859609+0000","last_active":"2024-04-20T11:33:27.862582+0000","last_peered":"2024-04-20T11:33:27.862582+0000","last_clean":"2024-04-20T11:33:27.862582+0000","last_became_active":"2024-04-20T11:33:27.858815+0000","last_became_peered":"2024-04-20T11:33:27.858815+0000","last_unstale":"2024-04-20T11:33:27.862582+0000","last_undegraded":"2024-04-20T11:33:27.862582+0000","last_fullsized":"2024-04-20T11:33:27.862582+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:16:02.482303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"102'2","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.861497+0000","last_change":"2024-04-20T11:33:26.839790+0000","last_active":"2024-04-20T11:33:27.861497+0000","last_peered":"2024-04-20T11:33:27.861497+0000","last_clean":"2024-04-20T11:33:27.861497+0000","last_became_active":"2024-04-20T11:33:26.839487+0000","last_became_peered":"2024-04-20T11:33:26.839487+0000","last_unstale":"2024-04-20T11:33:27.861497+0000","last_undegraded":"2024-04-20T11:33:27.861497+0000","last_fullsized":"2024-04-20T11:33:27.861497+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_clean_scrub_stamp":"2024-04-20T11:33:25.777424+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:16:25.270692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.836802+0000","last_change":"2024-04-20T11:33:27.832787+0000","last_active":"2024-04-20T11:33:27.836802+0000","last_peered":"2024-04-20T11:33:27.836802+0000","last_clean":"2024-04-20T11:33:27.836802+0000","last_became_active":"2024-04-20T11:33:27.832499+0000","last_became_peered":"2024-04-20T11:33:27.832499+0000","last_unstale":"2024-04-20T11:33:27.836802+0000","last_undegraded":"2024-04-20T11:33:27.836802+0000","last_fullsized":"2024-04-20T11:33:27.836802+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:37:10.682345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"102'1","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.855177+0000","last_change":"2024-04-20T11:33:26.838736+0000","last_active":"2024-04-20T11:33:27.855177+0000","last_peered":"2024-04-20T11:33:27.855177+0000","last_clean":"2024-04-20T11:33:27.855177+0000","last_became_active":"2024-04-20T11:33:26.838418+0000","last_became_peered":"2024-04-20T11:33:26.838418+0000","last_unstale":"2024-04-20T11:33:27.855177+0000","last_undegraded":"2024-04-20T11:33:27.855177+0000","last_fullsized":"2024-04-20T11:33:27.855177+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_clean_scrub_stamp":"2024-04-20T11:33:25.777424+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:35:29.275289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.860093+0000","last_change":"2024-04-20T11:33:27.850112+0000","last_active":"2024-04-20T11:33:27.860093+0000","last_peered":"2024-04-20T11:33:27.860093+0000","last_clean":"2024-04-20T11:33:27.860093+0000","last_became_active":"2024-04-20T11:33:27.844949+0000","last_became_peered":"2024-04-20T11:33:27.844949+0000","last_unstale":"2024-04-20T11:33:27.860093+0000","last_undegraded":"2024-04-20T11:33:27.860093+0000","last_fullsized":"2024-04-20T11:33:27.860093+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:45:02.469763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"102'1","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.852243+0000","last_change":"2024-04-20T11:33:26.845804+0000","last_active":"2024-04-20T11:33:27.852243+0000","last_peered":"2024-04-20T11:33:27.852243+0000","last_clean":"2024-04-20T11:33:27.852243+0000","last_became_active":"2024-04-20T11:33:26.845502+0000","last_became_peered":"2024-04-20T11:33:26.845502+0000","last_unstale":"2024-04-20T11:33:27.852243+0000","last_undegraded":"2024-04-20T11:33:27.852243+0000","last_fullsized":"2024-04-20T11:33:27.852243+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_clean_scrub_stamp":"2024-04-20T11:33:25.777424+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:57:49.108902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.849349+0000","last_change":"2024-04-20T11:33:27.842041+0000","last_active":"2024-04-20T11:33:27.849349+0000","last_peered":"2024-04-20T11:33:27.849349+0000","last_clean":"2024-04-20T11:33:27.849349+0000","last_became_active":"2024-04-20T11:33:27.841777+0000","last_became_peered":"2024-04-20T11:33:27.841777+0000","last_unstale":"2024-04-20T11:33:27.849349+0000","last_undegraded":"2024-04-20T11:33:27.849349+0000","last_fullsized":"2024-04-20T11:33:27.849349+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:31:47.662413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.834706+0000","last_change":"2024-04-20T11:33:26.840150+0000","last_active":"2024-04-20T11:33:27.834706+0000","last_peered":"2024-04-20T11:33:27.834706+0000","last_clean":"2024-04-20T11:33:27.834706+0000","last_became_active":"2024-04-20T11:33:26.839811+0000","last_became_peered":"2024-04-20T11:33:26.839811+0000","last_unstale":"2024-04-20T11:33:27.834706+0000","last_undegraded":"2024-04-20T11:33:27.834706+0000","last_fullsized":"2024-04-20T11:33:27.834706+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_clean_scrub_stamp":"2024-04-20T11:33:25.777424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:24:42.256401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.855898+0000","last_change":"2024-04-20T11:33:27.850922+0000","last_active":"2024-04-20T11:33:27.855898+0000","last_peered":"2024-04-20T11:33:27.855898+0000","last_clean":"2024-04-20T11:33:27.855898+0000","last_became_active":"2024-04-20T11:33:27.850423+0000","last_became_peered":"2024-04-20T11:33:27.850423+0000","last_unstale":"2024-04-20T11:33:27.855898+0000","last_undegraded":"2024-04-20T11:33:27.855898+0000","last_fullsized":"2024-04-20T11:33:27.855898+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:59:36.001217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.818334+0000","last_change":"2024-04-20T11:33:26.841194+0000","last_active":"2024-04-20T11:33:27.818334+0000","last_peered":"2024-04-20T11:33:27.818334+0000","last_clean":"2024-04-20T11:33:27.818334+0000","last_became_active":"2024-04-20T11:33:26.840389+0000","last_became_peered":"2024-04-20T11:33:26.840389+0000","last_unstale":"2024-04-20T11:33:27.818334+0000","last_undegraded":"2024-04-20T11:33:27.818334+0000","last_fullsized":"2024-04-20T11:33:27.818334+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_clean_scrub_stamp":"2024-04-20T11:33:25.777424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:39:28.768039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.849577+0000","last_change":"2024-04-20T11:33:27.845411+0000","last_active":"2024-04-20T11:33:27.849577+0000","last_peered":"2024-04-20T11:33:27.849577+0000","last_clean":"2024-04-20T11:33:27.849577+0000","last_became_active":"2024-04-20T11:33:27.845095+0000","last_became_peered":"2024-04-20T11:33:27.845095+0000","last_unstale":"2024-04-20T11:33:27.849577+0000","last_undegraded":"2024-04-20T11:33:27.849577+0000","last_fullsized":"2024-04-20T11:33:27.849577+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:00:56.317262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.825537+0000","last_change":"2024-04-20T11:33:26.842136+0000","last_active":"2024-04-20T11:33:27.825537+0000","last_peered":"2024-04-20T11:33:27.825537+0000","last_clean":"2024-04-20T11:33:27.825537+0000","last_became_active":"2024-04-20T11:33:26.841835+0000","last_became_peered":"2024-04-20T11:33:26.841835+0000","last_unstale":"2024-04-20T11:33:27.825537+0000","last_undegraded":"2024-04-20T11:33:27.825537+0000","last_fullsized":"2024-04-20T11:33:27.825537+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_clean_scrub_stamp":"2024-04-20T11:33:25.777424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:30:48.370537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.857524+0000","last_change":"2024-04-20T11:33:27.848104+0000","last_active":"2024-04-20T11:33:27.857524+0000","last_peered":"2024-04-20T11:33:27.857524+0000","last_clean":"2024-04-20T11:33:27.857524+0000","last_became_active":"2024-04-20T11:33:27.847552+0000","last_became_peered":"2024-04-20T11:33:27.847552+0000","last_unstale":"2024-04-20T11:33:27.857524+0000","last_undegraded":"2024-04-20T11:33:27.857524+0000","last_fullsized":"2024-04-20T11:33:27.857524+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:45:02.469763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.821147+0000","last_change":"2024-04-20T11:33:26.847147+0000","last_active":"2024-04-20T11:33:27.821147+0000","last_peered":"2024-04-20T11:33:27.821147+0000","last_clean":"2024-04-20T11:33:27.821147+0000","last_became_active":"2024-04-20T11:33:26.846404+0000","last_became_peered":"2024-04-20T11:33:26.846404+0000","last_unstale":"2024-04-20T11:33:27.821147+0000","last_undegraded":"2024-04-20T11:33:27.821147+0000","last_fullsized":"2024-04-20T11:33:27.821147+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_clean_scrub_stamp":"2024-04-20T11:33:25.777424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:45:01.453847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.862694+0000","last_change":"2024-04-20T11:33:27.859724+0000","last_active":"2024-04-20T11:33:27.862694+0000","last_peered":"2024-04-20T11:33:27.862694+0000","last_clean":"2024-04-20T11:33:27.862694+0000","last_became_active":"2024-04-20T11:33:27.858031+0000","last_became_peered":"2024-04-20T11:33:27.858031+0000","last_unstale":"2024-04-20T11:33:27.862694+0000","last_undegraded":"2024-04-20T11:33:27.862694+0000","last_fullsized":"2024-04-20T11:33:27.862694+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:41:33.281799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"102'3","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.863451+0000","last_change":"2024-04-20T11:33:26.841652+0000","last_active":"2024-04-20T11:33:27.863451+0000","last_peered":"2024-04-20T11:33:27.863451+0000","last_clean":"2024-04-20T11:33:27.863451+0000","last_became_active":"2024-04-20T11:33:26.841374+0000","last_became_peered":"2024-04-20T11:33:26.841374+0000","last_unstale":"2024-04-20T11:33:27.863451+0000","last_undegraded":"2024-04-20T11:33:27.863451+0000","last_fullsized":"2024-04-20T11:33:27.863451+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_clean_scrub_stamp":"2024-04-20T11:33:25.777424+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:41:32.265884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.855155+0000","last_change":"2024-04-20T11:33:27.849250+0000","last_active":"2024-04-20T11:33:27.855155+0000","last_peered":"2024-04-20T11:33:27.855155+0000","last_clean":"2024-04-20T11:33:27.855155+0000","last_became_active":"2024-04-20T11:33:27.848115+0000","last_became_peered":"2024-04-20T11:33:27.848115+0000","last_unstale":"2024-04-20T11:33:27.855155+0000","last_undegraded":"2024-04-20T11:33:27.855155+0000","last_fullsized":"2024-04-20T11:33:27.855155+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:28:03.773387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"102'6","reported_seq":18,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.865750+0000","last_change":"2024-04-20T11:33:26.842851+0000","last_active":"2024-04-20T11:33:27.865750+0000","last_peered":"2024-04-20T11:33:27.865750+0000","last_clean":"2024-04-20T11:33:27.865750+0000","last_became_active":"2024-04-20T11:33:26.840238+0000","last_became_peered":"2024-04-20T11:33:26.840238+0000","last_unstale":"2024-04-20T11:33:27.865750+0000","last_undegraded":"2024-04-20T11:33:27.865750+0000","last_fullsized":"2024-04-20T11:33:27.865750+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_clean_scrub_stamp":"2024-04-20T11:33:25.777424+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:18:47.905902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.847921+0000","last_change":"2024-04-20T11:33:27.843812+0000","last_active":"2024-04-20T11:33:27.847921+0000","last_peered":"2024-04-20T11:33:27.847921+0000","last_clean":"2024-04-20T11:33:27.847921+0000","last_became_active":"2024-04-20T11:33:27.843532+0000","last_became_peered":"2024-04-20T11:33:27.843532+0000","last_unstale":"2024-04-20T11:33:27.847921+0000","last_undegraded":"2024-04-20T11:33:27.847921+0000","last_fullsized":"2024-04-20T11:33:27.847921+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:59:32.524896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"102'2","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.855598+0000","last_change":"2024-04-20T11:33:26.836014+0000","last_active":"2024-04-20T11:33:27.855598+0000","last_peered":"2024-04-20T11:33:27.855598+0000","last_clean":"2024-04-20T11:33:27.855598+0000","last_became_active":"2024-04-20T11:33:26.835034+0000","last_became_peered":"2024-04-20T11:33:26.835034+0000","last_unstale":"2024-04-20T11:33:27.855598+0000","last_undegraded":"2024-04-20T11:33:27.855598+0000","last_fullsized":"2024-04-20T11:33:27.855598+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_clean_scrub_stamp":"2024-04-20T11:33:25.777424+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:30:48.370537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.862201+0000","last_change":"2024-04-20T11:33:27.857053+0000","last_active":"2024-04-20T11:33:27.862201+0000","last_peered":"2024-04-20T11:33:27.862201+0000","last_clean":"2024-04-20T11:33:27.862201+0000","last_became_active":"2024-04-20T11:33:27.856384+0000","last_became_peered":"2024-04-20T11:33:27.856384+0000","last_unstale":"2024-04-20T11:33:27.862201+0000","last_undegraded":"2024-04-20T11:33:27.862201+0000","last_fullsized":"2024-04-20T11:33:27.862201+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:22:01.734829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.846746+0000","last_change":"2024-04-20T11:33:26.847363+0000","last_active":"2024-04-20T11:33:27.846746+0000","last_peered":"2024-04-20T11:33:27.846746+0000","last_clean":"2024-04-20T11:33:27.846746+0000","last_became_active":"2024-04-20T11:33:26.846996+0000","last_became_peered":"2024-04-20T11:33:26.846996+0000","last_unstale":"2024-04-20T11:33:27.846746+0000","last_undegraded":"2024-04-20T11:33:27.846746+0000","last_fullsized":"2024-04-20T11:33:27.846746+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_clean_scrub_stamp":"2024-04-20T11:33:25.777424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:07:41.749690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.855827+0000","last_change":"2024-04-20T11:33:27.848977+0000","last_active":"2024-04-20T11:33:27.855827+0000","last_peered":"2024-04-20T11:33:27.855827+0000","last_clean":"2024-04-20T11:33:27.855827+0000","last_became_active":"2024-04-20T11:33:27.848708+0000","last_became_peered":"2024-04-20T11:33:27.848708+0000","last_unstale":"2024-04-20T11:33:27.855827+0000","last_undegraded":"2024-04-20T11:33:27.855827+0000","last_fullsized":"2024-04-20T11:33:27.855827+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:17:49.663226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"102'8","reported_seq":20,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.864373+0000","last_change":"2024-04-20T11:33:26.845361+0000","last_active":"2024-04-20T11:33:27.864373+0000","last_peered":"2024-04-20T11:33:27.864373+0000","last_clean":"2024-04-20T11:33:27.864373+0000","last_became_active":"2024-04-20T11:33:26.845092+0000","last_became_peered":"2024-04-20T11:33:26.845092+0000","last_unstale":"2024-04-20T11:33:27.864373+0000","last_undegraded":"2024-04-20T11:33:27.864373+0000","last_fullsized":"2024-04-20T11:33:27.864373+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_clean_scrub_stamp":"2024-04-20T11:33:25.777424+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:10:17.608844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.847118+0000","last_change":"2024-04-20T11:33:27.839471+0000","last_active":"2024-04-20T11:33:27.847118+0000","last_peered":"2024-04-20T11:33:27.847118+0000","last_clean":"2024-04-20T11:33:27.847118+0000","last_became_active":"2024-04-20T11:33:27.837621+0000","last_became_peered":"2024-04-20T11:33:27.837621+0000","last_unstale":"2024-04-20T11:33:27.847118+0000","last_undegraded":"2024-04-20T11:33:27.847118+0000","last_fullsized":"2024-04-20T11:33:27.847118+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:06:33.521677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"102'1","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.862825+0000","last_change":"2024-04-20T11:33:26.848198+0000","last_active":"2024-04-20T11:33:27.862825+0000","last_peered":"2024-04-20T11:33:27.862825+0000","last_clean":"2024-04-20T11:33:27.862825+0000","last_became_active":"2024-04-20T11:33:26.847929+0000","last_became_peered":"2024-04-20T11:33:26.847929+0000","last_unstale":"2024-04-20T11:33:27.862825+0000","last_undegraded":"2024-04-20T11:33:27.862825+0000","last_fullsized":"2024-04-20T11:33:27.862825+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_clean_scrub_stamp":"2024-04-20T11:33:25.777424+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:38:32.463275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.842029+0000","last_change":"2024-04-20T11:33:27.836301+0000","last_active":"2024-04-20T11:33:27.842029+0000","last_peered":"2024-04-20T11:33:27.842029+0000","last_clean":"2024-04-20T11:33:27.842029+0000","last_became_active":"2024-04-20T11:33:27.835267+0000","last_became_peered":"2024-04-20T11:33:27.835267+0000","last_unstale":"2024-04-20T11:33:27.842029+0000","last_undegraded":"2024-04-20T11:33:27.842029+0000","last_fullsized":"2024-04-20T11:33:27.842029+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:04:26.805899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"102'2","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.861540+0000","last_change":"2024-04-20T11:33:26.836478+0000","last_active":"2024-04-20T11:33:27.861540+0000","last_peered":"2024-04-20T11:33:27.861540+0000","last_clean":"2024-04-20T11:33:27.861540+0000","last_became_active":"2024-04-20T11:33:26.836224+0000","last_became_peered":"2024-04-20T11:33:26.836224+0000","last_unstale":"2024-04-20T11:33:27.861540+0000","last_undegraded":"2024-04-20T11:33:27.861540+0000","last_fullsized":"2024-04-20T11:33:27.861540+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_clean_scrub_stamp":"2024-04-20T11:33:25.777424+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:42:14.831506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.858132+0000","last_change":"2024-04-20T11:33:27.853164+0000","last_active":"2024-04-20T11:33:27.858132+0000","last_peered":"2024-04-20T11:33:27.858132+0000","last_clean":"2024-04-20T11:33:27.858132+0000","last_became_active":"2024-04-20T11:33:27.852899+0000","last_became_peered":"2024-04-20T11:33:27.852899+0000","last_unstale":"2024-04-20T11:33:27.858132+0000","last_undegraded":"2024-04-20T11:33:27.858132+0000","last_fullsized":"2024-04-20T11:33:27.858132+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:36:13.415680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"102'1","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.862047+0000","last_change":"2024-04-20T11:33:26.845248+0000","last_active":"2024-04-20T11:33:27.862047+0000","last_peered":"2024-04-20T11:33:27.862047+0000","last_clean":"2024-04-20T11:33:27.862047+0000","last_became_active":"2024-04-20T11:33:26.844912+0000","last_became_peered":"2024-04-20T11:33:26.844912+0000","last_unstale":"2024-04-20T11:33:27.862047+0000","last_undegraded":"2024-04-20T11:33:27.862047+0000","last_fullsized":"2024-04-20T11:33:27.862047+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_clean_scrub_stamp":"2024-04-20T11:33:25.777424+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:57:49.108902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.859792+0000","last_change":"2024-04-20T11:33:27.852749+0000","last_active":"2024-04-20T11:33:27.859792+0000","last_peered":"2024-04-20T11:33:27.859792+0000","last_clean":"2024-04-20T11:33:27.859792+0000","last_became_active":"2024-04-20T11:33:27.852450+0000","last_became_peered":"2024-04-20T11:33:27.852450+0000","last_unstale":"2024-04-20T11:33:27.859792+0000","last_undegraded":"2024-04-20T11:33:27.859792+0000","last_fullsized":"2024-04-20T11:33:27.859792+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:24:43.272317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.823018+0000","last_change":"2024-04-20T11:33:26.847931+0000","last_active":"2024-04-20T11:33:27.823018+0000","last_peered":"2024-04-20T11:33:27.823018+0000","last_clean":"2024-04-20T11:33:27.823018+0000","last_became_active":"2024-04-20T11:33:26.847527+0000","last_became_peered":"2024-04-20T11:33:26.847527+0000","last_unstale":"2024-04-20T11:33:27.823018+0000","last_undegraded":"2024-04-20T11:33:27.823018+0000","last_fullsized":"2024-04-20T11:33:27.823018+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_clean_scrub_stamp":"2024-04-20T11:33:25.777424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:10:47.785537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.858308+0000","last_change":"2024-04-20T11:33:27.854355+0000","last_active":"2024-04-20T11:33:27.858308+0000","last_peered":"2024-04-20T11:33:27.858308+0000","last_clean":"2024-04-20T11:33:27.858308+0000","last_became_active":"2024-04-20T11:33:27.854088+0000","last_became_peered":"2024-04-20T11:33:27.854088+0000","last_unstale":"2024-04-20T11:33:27.858308+0000","last_undegraded":"2024-04-20T11:33:27.858308+0000","last_fullsized":"2024-04-20T11:33:27.858308+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:07:42.765605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"102'2","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.863187+0000","last_change":"2024-04-20T11:33:26.831020+0000","last_active":"2024-04-20T11:33:27.863187+0000","last_peered":"2024-04-20T11:33:27.863187+0000","last_clean":"2024-04-20T11:33:27.863187+0000","last_became_active":"2024-04-20T11:33:26.829433+0000","last_became_peered":"2024-04-20T11:33:26.829433+0000","last_unstale":"2024-04-20T11:33:27.863187+0000","last_undegraded":"2024-04-20T11:33:27.863187+0000","last_fullsized":"2024-04-20T11:33:27.863187+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_clean_scrub_stamp":"2024-04-20T11:33:25.777424+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:20:25.159747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.863647+0000","last_change":"2024-04-20T11:33:27.852435+0000","last_active":"2024-04-20T11:33:27.863647+0000","last_peered":"2024-04-20T11:33:27.863647+0000","last_clean":"2024-04-20T11:33:27.863647+0000","last_became_active":"2024-04-20T11:33:27.852220+0000","last_became_peered":"2024-04-20T11:33:27.852220+0000","last_unstale":"2024-04-20T11:33:27.863647+0000","last_undegraded":"2024-04-20T11:33:27.863647+0000","last_fullsized":"2024-04-20T11:33:27.863647+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:57:04.047082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.824270+0000","last_change":"2024-04-20T11:33:26.841407+0000","last_active":"2024-04-20T11:33:27.824270+0000","last_peered":"2024-04-20T11:33:27.824270+0000","last_clean":"2024-04-20T11:33:27.824270+0000","last_became_active":"2024-04-20T11:33:26.841109+0000","last_became_peered":"2024-04-20T11:33:26.841109+0000","last_unstale":"2024-04-20T11:33:27.824270+0000","last_undegraded":"2024-04-20T11:33:27.824270+0000","last_fullsized":"2024-04-20T11:33:27.824270+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_clean_scrub_stamp":"2024-04-20T11:33:25.777424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:37:16.972405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.861467+0000","last_change":"2024-04-20T11:33:27.858903+0000","last_active":"2024-04-20T11:33:27.861467+0000","last_peered":"2024-04-20T11:33:27.861467+0000","last_clean":"2024-04-20T11:33:27.861467+0000","last_became_active":"2024-04-20T11:33:27.858436+0000","last_became_peered":"2024-04-20T11:33:27.858436+0000","last_unstale":"2024-04-20T11:33:27.861467+0000","last_undegraded":"2024-04-20T11:33:27.861467+0000","last_fullsized":"2024-04-20T11:33:27.861467+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:57:50.124818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.825801+0000","last_change":"2024-04-20T11:33:26.830948+0000","last_active":"2024-04-20T11:33:27.825801+0000","last_peered":"2024-04-20T11:33:27.825801+0000","last_clean":"2024-04-20T11:33:27.825801+0000","last_became_active":"2024-04-20T11:33:26.830668+0000","last_became_peered":"2024-04-20T11:33:26.830668+0000","last_unstale":"2024-04-20T11:33:27.825801+0000","last_undegraded":"2024-04-20T11:33:27.825801+0000","last_fullsized":"2024-04-20T11:33:27.825801+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_clean_scrub_stamp":"2024-04-20T11:33:25.777424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:42:14.831506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.838096+0000","last_change":"2024-04-20T11:33:27.833175+0000","last_active":"2024-04-20T11:33:27.838096+0000","last_peered":"2024-04-20T11:33:27.838096+0000","last_clean":"2024-04-20T11:33:27.838096+0000","last_became_active":"2024-04-20T11:33:27.831308+0000","last_became_peered":"2024-04-20T11:33:27.831308+0000","last_unstale":"2024-04-20T11:33:27.838096+0000","last_undegraded":"2024-04-20T11:33:27.838096+0000","last_fullsized":"2024-04-20T11:33:27.838096+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:30:56.853136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.818484+0000","last_change":"2024-04-20T11:33:26.847111+0000","last_active":"2024-04-20T11:33:27.818484+0000","last_peered":"2024-04-20T11:33:27.818484+0000","last_clean":"2024-04-20T11:33:27.818484+0000","last_became_active":"2024-04-20T11:33:26.846496+0000","last_became_peered":"2024-04-20T11:33:26.846496+0000","last_unstale":"2024-04-20T11:33:27.818484+0000","last_undegraded":"2024-04-20T11:33:27.818484+0000","last_fullsized":"2024-04-20T11:33:27.818484+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_clean_scrub_stamp":"2024-04-20T11:33:25.777424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:22:00.718913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.851849+0000","last_change":"2024-04-20T11:33:27.844566+0000","last_active":"2024-04-20T11:33:27.851849+0000","last_peered":"2024-04-20T11:33:27.851849+0000","last_clean":"2024-04-20T11:33:27.851849+0000","last_became_active":"2024-04-20T11:33:27.844306+0000","last_became_peered":"2024-04-20T11:33:27.844306+0000","last_unstale":"2024-04-20T11:33:27.851849+0000","last_undegraded":"2024-04-20T11:33:27.851849+0000","last_fullsized":"2024-04-20T11:33:27.851849+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:00:27.531704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"102'2","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.863547+0000","last_change":"2024-04-20T11:33:26.845533+0000","last_active":"2024-04-20T11:33:27.863547+0000","last_peered":"2024-04-20T11:33:27.863547+0000","last_clean":"2024-04-20T11:33:27.863547+0000","last_became_active":"2024-04-20T11:33:26.843541+0000","last_became_peered":"2024-04-20T11:33:26.843541+0000","last_unstale":"2024-04-20T11:33:27.863547+0000","last_undegraded":"2024-04-20T11:33:27.863547+0000","last_fullsized":"2024-04-20T11:33:27.863547+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_clean_scrub_stamp":"2024-04-20T11:33:25.777424+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:16:01.466387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.862612+0000","last_change":"2024-04-20T11:33:27.850391+0000","last_active":"2024-04-20T11:33:27.862612+0000","last_peered":"2024-04-20T11:33:27.862612+0000","last_clean":"2024-04-20T11:33:27.862612+0000","last_became_active":"2024-04-20T11:33:27.850033+0000","last_became_peered":"2024-04-20T11:33:27.850033+0000","last_unstale":"2024-04-20T11:33:27.862612+0000","last_undegraded":"2024-04-20T11:33:27.862612+0000","last_fullsized":"2024-04-20T11:33:27.862612+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:24:51.665909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.849913+0000","last_change":"2024-04-20T11:33:26.844369+0000","last_active":"2024-04-20T11:33:27.849913+0000","last_peered":"2024-04-20T11:33:27.849913+0000","last_clean":"2024-04-20T11:33:27.849913+0000","last_became_active":"2024-04-20T11:33:26.843997+0000","last_became_peered":"2024-04-20T11:33:26.843997+0000","last_unstale":"2024-04-20T11:33:27.849913+0000","last_undegraded":"2024-04-20T11:33:27.849913+0000","last_fullsized":"2024-04-20T11:33:27.849913+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_clean_scrub_stamp":"2024-04-20T11:33:25.777424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:54:41.132013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.862010+0000","last_change":"2024-04-20T11:33:27.847700+0000","last_active":"2024-04-20T11:33:27.862010+0000","last_peered":"2024-04-20T11:33:27.862010+0000","last_clean":"2024-04-20T11:33:27.862010+0000","last_became_active":"2024-04-20T11:33:27.847424+0000","last_became_peered":"2024-04-20T11:33:27.847424+0000","last_unstale":"2024-04-20T11:33:27.862010+0000","last_undegraded":"2024-04-20T11:33:27.862010+0000","last_fullsized":"2024-04-20T11:33:27.862010+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:34:43.790859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.822542+0000","last_change":"2024-04-20T11:33:26.847908+0000","last_active":"2024-04-20T11:33:27.822542+0000","last_peered":"2024-04-20T11:33:27.822542+0000","last_clean":"2024-04-20T11:33:27.822542+0000","last_became_active":"2024-04-20T11:33:26.847511+0000","last_became_peered":"2024-04-20T11:33:26.847511+0000","last_unstale":"2024-04-20T11:33:27.822542+0000","last_undegraded":"2024-04-20T11:33:27.822542+0000","last_fullsized":"2024-04-20T11:33:27.822542+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_clean_scrub_stamp":"2024-04-20T11:33:25.777424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:44:26.249472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.857195+0000","last_change":"2024-04-20T11:33:27.849915+0000","last_active":"2024-04-20T11:33:27.857195+0000","last_peered":"2024-04-20T11:33:27.857195+0000","last_clean":"2024-04-20T11:33:27.857195+0000","last_became_active":"2024-04-20T11:33:27.849274+0000","last_became_peered":"2024-04-20T11:33:27.849274+0000","last_unstale":"2024-04-20T11:33:27.857195+0000","last_undegraded":"2024-04-20T11:33:27.857195+0000","last_fullsized":"2024-04-20T11:33:27.857195+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:16:02.482303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.826735+0000","last_change":"2024-04-20T11:33:26.841918+0000","last_active":"2024-04-20T11:33:27.826735+0000","last_peered":"2024-04-20T11:33:27.826735+0000","last_clean":"2024-04-20T11:33:27.826735+0000","last_became_active":"2024-04-20T11:33:26.841359+0000","last_became_peered":"2024-04-20T11:33:26.841359+0000","last_unstale":"2024-04-20T11:33:27.826735+0000","last_undegraded":"2024-04-20T11:33:27.826735+0000","last_fullsized":"2024-04-20T11:33:27.826735+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_clean_scrub_stamp":"2024-04-20T11:33:25.777424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:54:41.132013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.860257+0000","last_change":"2024-04-20T11:33:27.851597+0000","last_active":"2024-04-20T11:33:27.860257+0000","last_peered":"2024-04-20T11:33:27.860257+0000","last_clean":"2024-04-20T11:33:27.860257+0000","last_became_active":"2024-04-20T11:33:27.851275+0000","last_became_peered":"2024-04-20T11:33:27.851275+0000","last_unstale":"2024-04-20T11:33:27.860257+0000","last_undegraded":"2024-04-20T11:33:27.860257+0000","last_fullsized":"2024-04-20T11:33:27.860257+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:57:50.124818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.822588+0000","last_change":"2024-04-20T11:33:26.842657+0000","last_active":"2024-04-20T11:33:27.822588+0000","last_peered":"2024-04-20T11:33:27.822588+0000","last_clean":"2024-04-20T11:33:27.822588+0000","last_became_active":"2024-04-20T11:33:26.842018+0000","last_became_peered":"2024-04-20T11:33:26.842018+0000","last_unstale":"2024-04-20T11:33:27.822588+0000","last_undegraded":"2024-04-20T11:33:27.822588+0000","last_fullsized":"2024-04-20T11:33:27.822588+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_clean_scrub_stamp":"2024-04-20T11:33:25.777424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:55:51.307982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.856299+0000","last_change":"2024-04-20T11:33:27.848164+0000","last_active":"2024-04-20T11:33:27.856299+0000","last_peered":"2024-04-20T11:33:27.856299+0000","last_clean":"2024-04-20T11:33:27.856299+0000","last_became_active":"2024-04-20T11:33:27.844626+0000","last_became_peered":"2024-04-20T11:33:27.844626+0000","last_unstale":"2024-04-20T11:33:27.856299+0000","last_undegraded":"2024-04-20T11:33:27.856299+0000","last_fullsized":"2024-04-20T11:33:27.856299+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:07:42.765605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.834470+0000","last_change":"2024-04-20T11:33:26.826111+0000","last_active":"2024-04-20T11:33:27.834470+0000","last_peered":"2024-04-20T11:33:27.834470+0000","last_clean":"2024-04-20T11:33:27.834470+0000","last_became_active":"2024-04-20T11:33:26.825658+0000","last_became_peered":"2024-04-20T11:33:26.825658+0000","last_unstale":"2024-04-20T11:33:27.834470+0000","last_undegraded":"2024-04-20T11:33:27.834470+0000","last_fullsized":"2024-04-20T11:33:27.834470+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_clean_scrub_stamp":"2024-04-20T11:33:25.777424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:56:35.320808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.862579+0000","last_change":"2024-04-20T11:33:27.860457+0000","last_active":"2024-04-20T11:33:27.862579+0000","last_peered":"2024-04-20T11:33:27.862579+0000","last_clean":"2024-04-20T11:33:27.862579+0000","last_became_active":"2024-04-20T11:33:27.858749+0000","last_became_peered":"2024-04-20T11:33:27.858749+0000","last_unstale":"2024-04-20T11:33:27.862579+0000","last_undegraded":"2024-04-20T11:33:27.862579+0000","last_fullsized":"2024-04-20T11:33:27.862579+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:30:56.853136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.824069+0000","last_change":"2024-04-20T11:33:26.838440+0000","last_active":"2024-04-20T11:33:27.824069+0000","last_peered":"2024-04-20T11:33:27.824069+0000","last_clean":"2024-04-20T11:33:27.824069+0000","last_became_active":"2024-04-20T11:33:26.838163+0000","last_became_peered":"2024-04-20T11:33:26.838163+0000","last_unstale":"2024-04-20T11:33:27.824069+0000","last_undegraded":"2024-04-20T11:33:27.824069+0000","last_fullsized":"2024-04-20T11:33:27.824069+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_clean_scrub_stamp":"2024-04-20T11:33:25.777424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:16:54.161688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.860737+0000","last_change":"2024-04-20T11:33:27.844461+0000","last_active":"2024-04-20T11:33:27.860737+0000","last_peered":"2024-04-20T11:33:27.860737+0000","last_clean":"2024-04-20T11:33:27.860737+0000","last_became_active":"2024-04-20T11:33:27.843813+0000","last_became_peered":"2024-04-20T11:33:27.843813+0000","last_unstale":"2024-04-20T11:33:27.860737+0000","last_undegraded":"2024-04-20T11:33:27.860737+0000","last_fullsized":"2024-04-20T11:33:27.860737+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:12:27.653618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.818651+0000","last_change":"2024-04-20T11:33:26.844681+0000","last_active":"2024-04-20T11:33:27.818651+0000","last_peered":"2024-04-20T11:33:27.818651+0000","last_clean":"2024-04-20T11:33:27.818651+0000","last_became_active":"2024-04-20T11:33:26.844405+0000","last_became_peered":"2024-04-20T11:33:26.844405+0000","last_unstale":"2024-04-20T11:33:27.818651+0000","last_undegraded":"2024-04-20T11:33:27.818651+0000","last_fullsized":"2024-04-20T11:33:27.818651+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_clean_scrub_stamp":"2024-04-20T11:33:25.777424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:01:10.455182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.861811+0000","last_change":"2024-04-20T11:33:27.852427+0000","last_active":"2024-04-20T11:33:27.861811+0000","last_peered":"2024-04-20T11:33:27.861811+0000","last_clean":"2024-04-20T11:33:27.861811+0000","last_became_active":"2024-04-20T11:33:27.852091+0000","last_became_peered":"2024-04-20T11:33:27.852091+0000","last_unstale":"2024-04-20T11:33:27.861811+0000","last_undegraded":"2024-04-20T11:33:27.861811+0000","last_fullsized":"2024-04-20T11:33:27.861811+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:26.793339+0000","last_clean_scrub_stamp":"2024-04-20T11:33:26.793339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:09:45.877194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.824242+0000","last_change":"2024-04-20T11:33:26.848097+0000","last_active":"2024-04-20T11:33:27.824242+0000","last_peered":"2024-04-20T11:33:27.824242+0000","last_clean":"2024-04-20T11:33:27.824242+0000","last_became_active":"2024-04-20T11:33:26.847781+0000","last_became_peered":"2024-04-20T11:33:26.847781+0000","last_unstale":"2024-04-20T11:33:27.824242+0000","last_undegraded":"2024-04-20T11:33:27.824242+0000","last_fullsized":"2024-04-20T11:33:27.824242+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:33:25.777424+0000","last_clean_scrub_stamp":"2024-04-20T11:33:25.777424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:26:09.194401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"72'128","reported_seq":356,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-20T11:33:27.843920+0000","last_change":"2024-04-20T11:26:48.661897+0000","last_active":"2024-04-20T11:33:27.843920+0000","last_peered":"2024-04-20T11:33:27.843920+0000","last_clean":"2024-04-20T11:33:27.843920+0000","last_became_active":"2024-04-20T11:26:48.656535+0000","last_became_peered":"2024-04-20T11:26:48.656535+0000","last_unstale":"2024-04-20T11:33:27.843920+0000","last_undegraded":"2024-04-20T11:33:27.843920+0000","last_fullsized":"2024-04-20T11:33:27.843920+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:47.543004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:47.543004+0000","last_clean_scrub_stamp":"2024-04-20T11:26:47.543004+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:58:35.527105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574934,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32992,"kb_used_data":1808,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338848,"statfs":{"total":96636764160,"available":96602980352,"internally_reserved":0,"allocated":1851392,"data_stored":1140106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574931,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32424,"kb_used_data":1244,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339416,"statfs":{"total":96636764160,"available":96603561984,"internally_reserved":0,"allocated":1273856,"data_stored":556583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574933,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32412,"kb_used_data":1228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339428,"statfs":{"total":96636764160,"available":96603574272,"internally_reserved":0,"allocated":1257472,"data_stored":549738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574931,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32416,"kb_used_data":1240,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339424,"statfs":{"total":96636764160,"available":96603570176,"internally_reserved":0,"allocated":1269760,"data_stored":556513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574932,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32444,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339396,"statfs":{"total":96636764160,"available":96603541504,"internally_reserved":0,"allocated":1286144,"data_stored":558195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574933,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32420,"kb_used_data":1240,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339420,"statfs":{"total":96636764160,"available":96603566080,"internally_reserved":0,"allocated":1269760,"data_stored":556513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574931,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32436,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339404,"statfs":{"total":96636764160,"available":96603549696,"internally_reserved":0,"allocated":1277952,"data_stored":556637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574932,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33028,"kb_used_data":1840,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338812,"statfs":{"total":96636764160,"available":96602943488,"internally_reserved":0,"allocated":1884160,"data_stored":1148605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T11:33:32.729 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-20T11:33:32.730 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-20T11:33:32.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.972+0000 7f97f0c00640 1 Processor -- start 2024-04-20T11:33:32.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.973+0000 7f97f0c00640 1 -- start start 2024-04-20T11:33:32.975 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.974+0000 7f97f0c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f97ec159040 0x7f97ec0cf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:32.975 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.974+0000 7f97f0c00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f97ec06f6a0 0x7f97ec0ce140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:32.975 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.974+0000 7f97f0c00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97ec159420 con 0x7f97ec162530 2024-04-20T11:33:32.975 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.974+0000 7f97f0c00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f97ec0ce680 con 0x7f97ec06f6a0 2024-04-20T11:33:32.975 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.974+0000 7f97f0c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f97ec06fb00 con 0x7f97ec159040 2024-04-20T11:33:32.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.975+0000 7f97eaa00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f97ec06f6a0 0x7f97ec0ce140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:32.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.975+0000 7f97eb400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f97ec159040 0x7f97ec0cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:32.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.975+0000 7f97eaa00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f97ec06f6a0 0x7f97ec0ce140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.152:48042/0 (socket says 172.21.15.152:48042) 2024-04-20T11:33:32.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.975+0000 7f97ebe00640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7f97ec162530 0x7f97ec0b32d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.152:53156/0 (socket says 172.21.15.152:53156) 2024-04-20T11:33:32.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.975+0000 7f97eaa00640 1 -- 172.21.15.152:0/3720550519 learned_addr learned my addr 172.21.15.152:0/3720550519 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:33:32.977 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.976+0000 7f97eaa00640 1 -- 172.21.15.152:0/3720550519 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f97ec159040 msgr2=0x7f97ec0cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:32.977 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.976+0000 7f97eaa00640 1 --2- 172.21.15.152:0/3720550519 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f97ec159040 0x7f97ec0cf950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:32.977 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.976+0000 7f97eaa00640 1 -- 172.21.15.152:0/3720550519 >> v1:172.21.15.175:6789/0 conn(0x7f97ec162530 legacy=0x7f97ec0b32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:32.977 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.976+0000 7f97eaa00640 1 -- 172.21.15.152:0/3720550519 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97ec0ce8b0 con 0x7f97ec06f6a0 2024-04-20T11:33:32.977 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.977+0000 7f97eaa00640 1 --2- 172.21.15.152:0/3720550519 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f97ec06f6a0 0x7f97ec0ce140 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f97d4009870 tx=0x7f97d402b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2194d190dc76bd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:32.978 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.978+0000 7f97ea000640 1 -- 172.21.15.152:0/3720550519 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d4034070 con 0x7f97ec06f6a0 2024-04-20T11:33:32.978 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.978+0000 7f97ea000640 1 -- 172.21.15.152:0/3720550519 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97d4030840 con 0x7f97ec06f6a0 2024-04-20T11:33:32.978 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.978+0000 7f97ea000640 1 -- 172.21.15.152:0/3720550519 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d4030c10 con 0x7f97ec06f6a0 2024-04-20T11:33:32.979 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.978+0000 7f97f0c00640 1 -- 172.21.15.152:0/3720550519 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f97ec06f6a0 msgr2=0x7f97ec0ce140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:32.979 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.978+0000 7f97f0c00640 1 --2- 172.21.15.152:0/3720550519 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f97ec06f6a0 0x7f97ec0ce140 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f97d4009870 tx=0x7f97d402b980 comp rx=0 tx=0).stop 2024-04-20T11:33:32.979 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.978+0000 7f97f0c00640 1 -- 172.21.15.152:0/3720550519 shutdown_connections 2024-04-20T11:33:32.979 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.978+0000 7f97f0c00640 1 --2- 172.21.15.152:0/3720550519 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f97ec06f6a0 0x7f97ec0ce140 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:32.980 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.978+0000 7f97f0c00640 1 --2- 172.21.15.152:0/3720550519 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f97ec159040 0x7f97ec0cf950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:32.980 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.978+0000 7f97f0c00640 1 -- 172.21.15.152:0/3720550519 >> 172.21.15.152:0/3720550519 conn(0x7f97ec0d0260 msgr2=0x7f97ec0d0670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:32.980 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.979+0000 7f97f0c00640 1 -- 172.21.15.152:0/3720550519 shutdown_connections 2024-04-20T11:33:32.980 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.979+0000 7f97f0c00640 1 -- 172.21.15.152:0/3720550519 wait complete. 2024-04-20T11:33:32.980 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.980+0000 7f97f0c00640 1 Processor -- start 2024-04-20T11:33:32.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.980+0000 7f97f0c00640 1 -- start start 2024-04-20T11:33:32.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.981+0000 7f97f0c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f97ec06f6a0 0x7f97ec072ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:32.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.981+0000 7f97f0c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f97ec159040 0x7f97ec09a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:32.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.981+0000 7f97f0c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f97ec162530 0x7f97ec09ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:32.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.981+0000 7f97f0c00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f97ec0cf150 con 0x7f97ec06f6a0 2024-04-20T11:33:32.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.981+0000 7f97f0c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f97ec0cee70 con 0x7f97ec159040 2024-04-20T11:33:32.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.981+0000 7f97f0c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f97ec0cf430 con 0x7f97ec162530 2024-04-20T11:33:32.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.982+0000 7f97eb400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f97ec06f6a0 0x7f97ec072ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:32.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.982+0000 7f97eb400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f97ec06f6a0 0x7f97ec072ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54304/0 (socket says 172.21.15.152:54304) 2024-04-20T11:33:32.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.982+0000 7f97eb400640 1 -- 172.21.15.152:0/826946844 learned_addr learned my addr 172.21.15.152:0/826946844 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:33:32.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.982+0000 7f97eaa00640 1 --2- 172.21.15.152:0/826946844 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f97ec159040 0x7f97ec09a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:32.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.982+0000 7f97ebe00640 1 --2- 172.21.15.152:0/826946844 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f97ec162530 0x7f97ec09ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:32.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.982+0000 7f97eb400640 1 -- 172.21.15.152:0/826946844 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f97ec162530 msgr2=0x7f97ec09ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:32.984 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.982+0000 7f97eb400640 1 --2- 172.21.15.152:0/826946844 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f97ec162530 0x7f97ec09ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:32.984 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.982+0000 7f97eb400640 1 -- 172.21.15.152:0/826946844 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f97ec159040 msgr2=0x7f97ec09a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:32.984 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.982+0000 7f97eb400640 1 --2- 172.21.15.152:0/826946844 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f97ec159040 0x7f97ec09a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:32.984 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.982+0000 7f97eb400640 1 -- 172.21.15.152:0/826946844 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97ec09b240 con 0x7f97ec06f6a0 2024-04-20T11:33:32.984 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.982+0000 7f97eaa00640 1 --2- 172.21.15.152:0/826946844 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f97ec159040 0x7f97ec09a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:33:32.984 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.982+0000 7f97eb400640 1 --2- 172.21.15.152:0/826946844 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f97ec06f6a0 0x7f97ec072ea0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f97dc009d10 tx=0x7f97dc00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:32.985 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.983+0000 7f97e3400640 1 -- 172.21.15.152:0/826946844 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97dc018020 con 0x7f97ec06f6a0 2024-04-20T11:33:32.985 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.983+0000 7f97e3400640 1 -- 172.21.15.152:0/826946844 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97dc014480 con 0x7f97ec06f6a0 2024-04-20T11:33:32.985 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.983+0000 7f97e3400640 1 -- 172.21.15.152:0/826946844 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97dc014830 con 0x7f97ec06f6a0 2024-04-20T11:33:32.985 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.983+0000 7f97f0c00640 1 -- 172.21.15.152:0/826946844 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97ec09b520 con 0x7f97ec06f6a0 2024-04-20T11:33:32.985 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.983+0000 7f97f0c00640 1 -- 172.21.15.152:0/826946844 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97ec09bae0 con 0x7f97ec06f6a0 2024-04-20T11:33:32.986 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.984+0000 7f97f0c00640 1 -- 172.21.15.152:0/826946844 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97ec075e60 con 0x7f97ec06f6a0 2024-04-20T11:33:32.986 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.985+0000 7f97e3400640 1 -- 172.21.15.152:0/826946844 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 91142+0+0 (secure 0 0 0) 0x7f97dc005d10 con 0x7f97ec06f6a0 2024-04-20T11:33:32.986 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.986+0000 7f97e3400640 1 --2- 172.21.15.152:0/826946844 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f97c806ca10 0x7f97c806eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:32.986 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.986+0000 7f97e3400640 1 -- 172.21.15.152:0/826946844 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7f97dc08c940 con 0x7f97ec06f6a0 2024-04-20T11:33:32.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.987+0000 7f97eaa00640 1 --2- 172.21.15.152:0/826946844 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f97c806ca10 0x7f97c806eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:32.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.988+0000 7f97eaa00640 1 --2- 172.21.15.152:0/826946844 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f97c806ca10 0x7f97c806eed0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f97d4036000 tx=0x7f97d403b040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:32.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:32.991+0000 7f97e3400640 1 -- 172.21.15.152:0/826946844 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f97dc022070 con 0x7f97ec06f6a0 2024-04-20T11:33:33.256 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:33.255+0000 7f97f0c00640 1 -- 172.21.15.152:0/826946844 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f97ec14fc40 con 0x7f97ec06f6a0 2024-04-20T11:33:33.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:33.260+0000 7f97e3400640 1 -- 172.21.15.152:0/826946844 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v16) ==== 190+0+147 (secure 0 0 0) 0x7f97dc010040 con 0x7f97ec06f6a0 2024-04-20T11:33:33.261 INFO:teuthology.orchestra.run.smithi152.stdout:[client.0] 2024-04-20T11:33:33.261 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQBDpiNmtsBWFhAArq9A7r33CcwgMmOU2pAGPw== 2024-04-20T11:33:33.261 INFO:teuthology.orchestra.run.smithi152.stdout: caps mds = "allow" 2024-04-20T11:33:33.261 INFO:teuthology.orchestra.run.smithi152.stdout: caps mon = "allow r" 2024-04-20T11:33:33.261 INFO:teuthology.orchestra.run.smithi152.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-20T11:33:33.261 INFO:teuthology.orchestra.run.smithi152.stderr:updated caps for client.0 2024-04-20T11:33:33.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:33.262+0000 7f97f0c00640 1 -- 172.21.15.152:0/826946844 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f97c806ca10 msgr2=0x7f97c806eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:33.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:33.262+0000 7f97f0c00640 1 --2- 172.21.15.152:0/826946844 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f97c806ca10 0x7f97c806eed0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f97d4036000 tx=0x7f97d403b040 comp rx=0 tx=0).stop 2024-04-20T11:33:33.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:33.262+0000 7f97f0c00640 1 -- 172.21.15.152:0/826946844 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f97ec06f6a0 msgr2=0x7f97ec072ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:33.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:33.262+0000 7f97f0c00640 1 --2- 172.21.15.152:0/826946844 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f97ec06f6a0 0x7f97ec072ea0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f97dc009d10 tx=0x7f97dc00c490 comp rx=0 tx=0).stop 2024-04-20T11:33:33.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:33.263+0000 7f97f0c00640 1 -- 172.21.15.152:0/826946844 shutdown_connections 2024-04-20T11:33:33.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:33.263+0000 7f97f0c00640 1 --2- 172.21.15.152:0/826946844 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f97c806ca10 0x7f97c806eed0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:33.265 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:33.263+0000 7f97f0c00640 1 --2- 172.21.15.152:0/826946844 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f97ec162530 0x7f97ec09ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:33.265 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:33.263+0000 7f97f0c00640 1 --2- 172.21.15.152:0/826946844 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f97ec159040 0x7f97ec09a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:33.265 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:33.263+0000 7f97f0c00640 1 --2- 172.21.15.152:0/826946844 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f97ec06f6a0 0x7f97ec072ea0 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:33.265 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:33.263+0000 7f97f0c00640 1 -- 172.21.15.152:0/826946844 >> 172.21.15.152:0/826946844 conn(0x7f97ec0d0260 msgr2=0x7f97ec0add00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:33.265 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:33.263+0000 7f97f0c00640 1 -- 172.21.15.152:0/826946844 shutdown_connections 2024-04-20T11:33:33.265 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:33.263+0000 7f97f0c00640 1 -- 172.21.15.152:0/826946844 wait complete. 2024-04-20T11:33:33.282 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:33:33.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:33.472+0000 7f6063600640 1 Processor -- start 2024-04-20T11:33:33.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:33.472+0000 7f6063600640 1 -- start start 2024-04-20T11:33:33.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:33.474+0000 7f6063600640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f605c0cd0d0 0x7f605c0cd580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:33.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:33.474+0000 7f6063600640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f605c071fe0 0x7f605c072470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:33.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:33.474+0000 7f6063600640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f605c146340 con 0x7f605c145f60 2024-04-20T11:33:33.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:33.474+0000 7f6063600640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f605c06e9f0 con 0x7f605c071fe0 2024-04-20T11:33:33.476 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:33.474+0000 7f6063600640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f605c072d50 con 0x7f605c0cd0d0 2024-04-20T11:33:33.476 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:33.474+0000 7f6061600640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f605c071fe0 0x7f605c072470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:33.476 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:33.474+0000 7f6061600640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f605c071fe0 0x7f605c072470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54306/0 (socket says 172.21.15.152:54306) 2024-04-20T11:33:33.476 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:33.474+0000 7f6061600640 1 -- 172.21.15.152:0/2664389649 learned_addr learned my addr 172.21.15.152:0/2664389649 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:33:33.476 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:33.474+0000 7f6060c00640 1 --1- 172.21.15.152:0/2664389649 >> v1:172.21.15.152:6789/0 conn(0x7f605c145f60 0x7f605c06ee80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:55170/0 (socket says 172.21.15.152:55170) 2024-04-20T11:33:33.476 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:33.474+0000 7f605be00640 1 --2- 172.21.15.152:0/2664389649 >> v2:172.21.15.175:3300/0 conn(0x7f605c0cd0d0 0x7f605c0cd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:33.476 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:33.475+0000 7f6061600640 1 -- 172.21.15.152:0/2664389649 >> v2:172.21.15.175:3300/0 conn(0x7f605c0cd0d0 msgr2=0x7f605c0cd580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:33.476 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:33.475+0000 7f6061600640 1 --2- 172.21.15.152:0/2664389649 >> v2:172.21.15.175:3300/0 conn(0x7f605c0cd0d0 0x7f605c0cd580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:33.476 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:33.475+0000 7f6061600640 1 -- 172.21.15.152:0/2664389649 >> v1:172.21.15.152:6789/0 conn(0x7f605c145f60 legacy=0x7f605c06ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:33.476 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:33.475+0000 7f6061600640 1 -- 172.21.15.152:0/2664389649 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f605c06e0c0 con 0x7f605c071fe0 2024-04-20T11:33:33.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:33.476+0000 7f6061600640 1 --2- 172.21.15.152:0/2664389649 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f605c071fe0 0x7f605c072470 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f605c06eb70 tx=0x7f605402bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c30d0cf80f6c0040 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:33.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:33.476+0000 7f605b400640 1 -- 172.21.15.152:0/2664389649 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f605402cb00 con 0x7f605c071fe0 2024-04-20T11:33:33.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:33.476+0000 7f605b400640 1 -- 172.21.15.152:0/2664389649 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f605402cca0 con 0x7f605c071fe0 2024-04-20T11:33:33.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:33.476+0000 7f605b400640 1 -- 172.21.15.152:0/2664389649 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6054034610 con 0x7f605c071fe0 2024-04-20T11:33:33.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:33.477+0000 7f6063600640 1 -- 172.21.15.152:0/2664389649 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f605c071fe0 msgr2=0x7f605c072470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:33.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:33.477+0000 7f6063600640 1 --2- 172.21.15.152:0/2664389649 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f605c071fe0 0x7f605c072470 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f605c06eb70 tx=0x7f605402bf60 comp rx=0 tx=0).stop 2024-04-20T11:33:33.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:33.477+0000 7f6063600640 1 -- 172.21.15.152:0/2664389649 shutdown_connections 2024-04-20T11:33:33.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:33.477+0000 7f6063600640 1 --2- 172.21.15.152:0/2664389649 >> v2:172.21.15.175:3300/0 conn(0x7f605c0cd0d0 0x7f605c0cd580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:33.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:33.477+0000 7f6063600640 1 --2- 172.21.15.152:0/2664389649 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f605c071fe0 0x7f605c072470 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:33.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:33.477+0000 7f6063600640 1 -- 172.21.15.152:0/2664389649 >> 172.21.15.152:0/2664389649 conn(0x7f605c067cf0 msgr2=0x7f605c06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:33.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:33.477+0000 7f6063600640 1 -- 172.21.15.152:0/2664389649 shutdown_connections 2024-04-20T11:33:33.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:33.477+0000 7f6063600640 1 -- 172.21.15.152:0/2664389649 wait complete. 2024-04-20T11:33:33.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:33.479+0000 7f6063600640 1 Processor -- start 2024-04-20T11:33:33.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:33.479+0000 7f6063600640 1 -- start start 2024-04-20T11:33:33.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:33.480+0000 7f6063600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f605c071fe0 0x7f605c158f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:33.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:33.480+0000 7f6063600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f605c0cd0d0 0x7f605c1437b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:33.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:33.480+0000 7f6063600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f605c145f60 0x7f605c0d0110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:33.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:33.480+0000 7f6063600640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f605c0729b0 con 0x7f605c0cd0d0 2024-04-20T11:33:33.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:33.480+0000 7f6063600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f605c06e680 con 0x7f605c071fe0 2024-04-20T11:33:33.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:33.480+0000 7f6063600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f605c072b30 con 0x7f605c145f60 2024-04-20T11:33:33.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:33.480+0000 7f605be00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f605c0cd0d0 0x7f605c1437b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:33.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:33.480+0000 7f605be00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f605c0cd0d0 0x7f605c1437b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54322/0 (socket says 172.21.15.152:54322) 2024-04-20T11:33:33.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:33.480+0000 7f605be00640 1 -- 172.21.15.152:0/3816905558 learned_addr learned my addr 172.21.15.152:0/3816905558 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:33:33.482 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:33.480+0000 7f605be00640 1 -- 172.21.15.152:0/3816905558 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f605c145f60 msgr2=0x7f605c0d0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:33.482 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:33.480+0000 7f605be00640 1 --2- 172.21.15.152:0/3816905558 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f605c145f60 0x7f605c0d0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:33.482 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:33.480+0000 7f605be00640 1 -- 172.21.15.152:0/3816905558 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f605c071fe0 msgr2=0x7f605c158f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:33.482 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:33.480+0000 7f605be00640 1 --2- 172.21.15.152:0/3816905558 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f605c071fe0 0x7f605c158f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:33.482 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:33.480+0000 7f605be00640 1 -- 172.21.15.152:0/3816905558 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f605c144170 con 0x7f605c0cd0d0 2024-04-20T11:33:33.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:33.481+0000 7f605be00640 1 --2- 172.21.15.152:0/3816905558 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f605c0cd0d0 0x7f605c1437b0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f604800d830 tx=0x7f604800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:33.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:33.481+0000 7f6058c00640 1 -- 172.21.15.152:0/3816905558 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f604800b6c0 con 0x7f605c0cd0d0 2024-04-20T11:33:33.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:33.481+0000 7f6058c00640 1 -- 172.21.15.152:0/3816905558 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6048019070 con 0x7f605c0cd0d0 2024-04-20T11:33:33.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:33.481+0000 7f6058c00640 1 -- 172.21.15.152:0/3816905558 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f604800ba80 con 0x7f605c0cd0d0 2024-04-20T11:33:33.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:33.481+0000 7f6063600640 1 -- 172.21.15.152:0/3816905558 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f605c144450 con 0x7f605c0cd0d0 2024-04-20T11:33:33.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:33.481+0000 7f6063600640 1 -- 172.21.15.152:0/3816905558 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f605c144a10 con 0x7f605c0cd0d0 2024-04-20T11:33:33.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:33.483+0000 7f6063600640 1 -- 172.21.15.152:0/3816905558 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f605c0689f0 con 0x7f605c0cd0d0 2024-04-20T11:33:33.484 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:33.483+0000 7f6058c00640 1 -- 172.21.15.152:0/3816905558 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 91142+0+0 (secure 0 0 0) 0x7f604801e460 con 0x7f605c0cd0d0 2024-04-20T11:33:33.485 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:33.484+0000 7f6058c00640 1 --2- 172.21.15.152:0/3816905558 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f603c06ca10 0x7f603c06eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:33.485 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:33.484+0000 7f6058c00640 1 -- 172.21.15.152:0/3816905558 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7f604808c3b0 con 0x7f605c0cd0d0 2024-04-20T11:33:33.485 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:33.485+0000 7f6060c00640 1 --2- 172.21.15.152:0/3816905558 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f603c06ca10 0x7f603c06eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:33.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:33.486+0000 7f6060c00640 1 --2- 172.21.15.152:0/3816905558 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f603c06ca10 0x7f603c06eed0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6050000f60 tx=0x7f6050007a00 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:33.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:33.491+0000 7f6058c00640 1 -- 172.21.15.152:0/3816905558 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f604805a6d0 con 0x7f605c0cd0d0 2024-04-20T11:33:33.731 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:33.730+0000 7f6063600640 1 -- 172.21.15.152:0/3816905558 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f605c1594b0 con 0x7f605c0cd0d0 2024-04-20T11:33:33.731 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:33.731+0000 7f6058c00640 1 -- 172.21.15.152:0/3816905558 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) ==== 94+0+5210 (secure 0 0 0) 0x7f6048059fd0 con 0x7f605c0cd0d0 2024-04-20T11:33:33.732 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:33:33.732 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":66,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5648,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6834/387198089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":387198089},{"type":"v1","addr":"172.21.15.152:6834","nonce":387198089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":6154,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/1674013715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1674013715},{"type":"v1","addr":"172.21.15.175:6833","nonce":1674013715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":6157,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/3898238490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3898238490},{"type":"v1","addr":"172.21.15.175:6835","nonce":3898238490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T11:33:27.277882+0000","modified":"2024-04-20T11:33:30.342429+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7794},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7794":{"gid":7794,"name":"a","rank":0,"incarnation":62,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6835/3582312712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6833","nonce":3582312712},{"type":"v1","addr":"172.21.15.152:6835","nonce":3582312712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7794,"qdb_cluster":[7794]},"id":8}]} 2024-04-20T11:33:33.732 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 66 2024-04-20T11:33:33.735 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:33.734+0000 7f6063600640 1 -- 172.21.15.152:0/3816905558 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f603c06ca10 msgr2=0x7f603c06eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:33.735 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:33.734+0000 7f6063600640 1 --2- 172.21.15.152:0/3816905558 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f603c06ca10 0x7f603c06eed0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6050000f60 tx=0x7f6050007a00 comp rx=0 tx=0).stop 2024-04-20T11:33:33.735 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:33.734+0000 7f6063600640 1 -- 172.21.15.152:0/3816905558 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f605c0cd0d0 msgr2=0x7f605c1437b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:33.735 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:33.734+0000 7f6063600640 1 --2- 172.21.15.152:0/3816905558 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f605c0cd0d0 0x7f605c1437b0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f604800d830 tx=0x7f604800dcf0 comp rx=0 tx=0).stop 2024-04-20T11:33:33.735 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:33.734+0000 7f6063600640 1 -- 172.21.15.152:0/3816905558 shutdown_connections 2024-04-20T11:33:33.735 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:33.734+0000 7f6063600640 1 --2- 172.21.15.152:0/3816905558 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f603c06ca10 0x7f603c06eed0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:33.735 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:33.734+0000 7f6063600640 1 --2- 172.21.15.152:0/3816905558 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f605c145f60 0x7f605c0d0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:33.735 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:33.734+0000 7f6063600640 1 --2- 172.21.15.152:0/3816905558 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f605c0cd0d0 0x7f605c1437b0 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:33.735 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:33.734+0000 7f6063600640 1 --2- 172.21.15.152:0/3816905558 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f605c071fe0 0x7f605c158f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:33.735 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:33.734+0000 7f6063600640 1 -- 172.21.15.152:0/3816905558 >> 172.21.15.152:0/3816905558 conn(0x7f605c067cf0 msgr2=0x7f605c15a890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:33.735 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:33.734+0000 7f6063600640 1 -- 172.21.15.152:0/3816905558 shutdown_connections 2024-04-20T11:33:33.735 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:33.734+0000 7f6063600640 1 -- 172.21.15.152:0/3816905558 wait complete. 2024-04-20T11:33:34.158 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 66, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-20T11:33:27.277882+0000', 'modified': '2024-04-20T11:33:30.342429+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'snap_rstat': True, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7794}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7794': {'gid': 7794, 'name': 'a', 'rank': 0, 'incarnation': 62, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.152:6835/3582312712', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6833', 'nonce': 3582312712}, {'type': 'v1', 'addr': '172.21.15.152:6835', 'nonce': 3582312712}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7794, 'qdb_cluster': [7794]} 2024-04-20T11:33:34.158 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-20T11:33:34.158 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-20T11:33:34.370 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.369+0000 7f5deb200640 1 Processor -- start 2024-04-20T11:33:34.370 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.370+0000 7f5deb200640 1 -- start start 2024-04-20T11:33:34.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.371+0000 7f5deb200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f5dec0b4870 0x7f5dec072b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:34.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.371+0000 7f5deb200640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5dec073f60 con 0x7f5dec15c7f0 2024-04-20T11:33:34.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.371+0000 7f5deb200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5dec0ce390 con 0x7f5dec073b60 2024-04-20T11:33:34.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.371+0000 7f5deb200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f5dec0ce140 con 0x7f5dec0b4870 2024-04-20T11:33:34.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.372+0000 7f5de1400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f5dec0b4870 0x7f5dec072b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:34.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.372+0000 7f5de9e00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f5dec15c7f0 0x7f5dec0b4160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:55172/0 (socket says 172.21.15.152:55172) 2024-04-20T11:33:34.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.372+0000 7f5de9e00640 1 -- 172.21.15.152:0/3404854343 learned_addr learned my addr 172.21.15.152:0/3404854343 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:33:34.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.372+0000 7f5de1400640 1 -- 172.21.15.152:0/3404854343 >> v1:172.21.15.175:6789/0 conn(0x7f5dec073b60 legacy=0x7f5dec0cf430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:34.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.372+0000 7f5de1400640 1 -- 172.21.15.152:0/3404854343 >> v1:172.21.15.152:6789/0 conn(0x7f5dec15c7f0 legacy=0x7f5dec0b4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:34.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.372+0000 7f5de1400640 1 -- 172.21.15.152:0/3404854343 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dec0ce670 con 0x7f5dec0b4870 2024-04-20T11:33:34.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.373+0000 7f5de1400640 1 --2- 172.21.15.152:0/3404854343 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5dec0b4870 0x7f5dec072b70 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f5dd40097a0 tx=0x7f5dd402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9497fa3f38ff38d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:34.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.373+0000 7f5de9400640 1 -- 172.21.15.152:0/3404854343 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd40315a0 con 0x7f5dec0b4870 2024-04-20T11:33:34.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.374+0000 7f5de9400640 1 -- 172.21.15.152:0/3404854343 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5dd4037070 con 0x7f5dec0b4870 2024-04-20T11:33:34.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.374+0000 7f5de9400640 1 -- 172.21.15.152:0/3404854343 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd4031960 con 0x7f5dec0b4870 2024-04-20T11:33:34.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.374+0000 7f5deb200640 1 -- 172.21.15.152:0/3404854343 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5dec0b4870 msgr2=0x7f5dec072b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:34.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.374+0000 7f5deb200640 1 --2- 172.21.15.152:0/3404854343 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5dec0b4870 0x7f5dec072b70 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f5dd40097a0 tx=0x7f5dd402b840 comp rx=0 tx=0).stop 2024-04-20T11:33:34.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.374+0000 7f5deb200640 1 -- 172.21.15.152:0/3404854343 shutdown_connections 2024-04-20T11:33:34.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.374+0000 7f5deb200640 1 --2- 172.21.15.152:0/3404854343 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5dec0b4870 0x7f5dec072b70 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:34.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.374+0000 7f5deb200640 1 -- 172.21.15.152:0/3404854343 >> 172.21.15.152:0/3404854343 conn(0x7f5dec0d0260 msgr2=0x7f5dec0d0670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:34.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.374+0000 7f5deb200640 1 -- 172.21.15.152:0/3404854343 shutdown_connections 2024-04-20T11:33:34.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.375+0000 7f5deb200640 1 -- 172.21.15.152:0/3404854343 wait complete. 2024-04-20T11:33:34.377 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.377+0000 7f5deb200640 1 Processor -- start 2024-04-20T11:33:34.377 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.377+0000 7f5deb200640 1 -- start start 2024-04-20T11:33:34.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.378+0000 7f5deb200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5dec073b60 0x7f5dec156be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:34.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.378+0000 7f5deb200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5dec0b4870 0x7f5dec0b1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:34.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.378+0000 7f5deb200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5dec1f7ae0 0x7f5dec1f7f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:34.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.378+0000 7f5deb200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f5dec0cef10 con 0x7f5dec0b4870 2024-04-20T11:33:34.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.378+0000 7f5deb200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5dec0cec30 con 0x7f5dec1f7ae0 2024-04-20T11:33:34.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.378+0000 7f5deb200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5dec0ad2b0 con 0x7f5dec073b60 2024-04-20T11:33:34.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.379+0000 7f5de1400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5dec0b4870 0x7f5dec0b1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:34.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.379+0000 7f5de1400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5dec0b4870 0x7f5dec0b1590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54340/0 (socket says 172.21.15.152:54340) 2024-04-20T11:33:34.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.379+0000 7f5de1400640 1 -- 172.21.15.152:0/1682046809 learned_addr learned my addr 172.21.15.152:0/1682046809 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:33:34.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.379+0000 7f5de9e00640 1 --2- 172.21.15.152:0/1682046809 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5dec073b60 0x7f5dec156be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:34.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.379+0000 7f5de1400640 1 -- 172.21.15.152:0/1682046809 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5dec073b60 msgr2=0x7f5dec156be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:34.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.379+0000 7f5de1400640 1 --2- 172.21.15.152:0/1682046809 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5dec073b60 0x7f5dec156be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:34.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.379+0000 7f5de1400640 1 -- 172.21.15.152:0/1682046809 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5dec1f7ae0 msgr2=0x7f5dec1f7f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:34.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.379+0000 7f5de1400640 1 --2- 172.21.15.152:0/1682046809 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5dec1f7ae0 0x7f5dec1f7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:34.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.379+0000 7f5de1400640 1 -- 172.21.15.152:0/1682046809 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dec1f8530 con 0x7f5dec0b4870 2024-04-20T11:33:34.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.379+0000 7f5de1400640 1 --2- 172.21.15.152:0/1682046809 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5dec0b4870 0x7f5dec0b1590 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f5dd4000c00 tx=0x7f5dd40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:34.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.380+0000 7f5de2a00640 1 -- 172.21.15.152:0/1682046809 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd4032840 con 0x7f5dec0b4870 2024-04-20T11:33:34.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.380+0000 7f5de2a00640 1 -- 172.21.15.152:0/1682046809 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5dd40329e0 con 0x7f5dec0b4870 2024-04-20T11:33:34.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.380+0000 7f5de2a00640 1 -- 172.21.15.152:0/1682046809 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd4032dd0 con 0x7f5dec0b4870 2024-04-20T11:33:34.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.380+0000 7f5deb200640 1 -- 172.21.15.152:0/1682046809 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dec1f8810 con 0x7f5dec0b4870 2024-04-20T11:33:34.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.380+0000 7f5deb200640 1 -- 172.21.15.152:0/1682046809 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dec1566e0 con 0x7f5dec0b4870 2024-04-20T11:33:34.383 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.382+0000 7f5de2a00640 1 -- 172.21.15.152:0/1682046809 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 91142+0+0 (secure 0 0 0) 0x7f5dd404d3f0 con 0x7f5dec0b4870 2024-04-20T11:33:34.384 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.383+0000 7f5de2a00640 1 --2- 172.21.15.152:0/1682046809 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f5dc806ca10 0x7f5dc806eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:34.384 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.383+0000 7f5de2a00640 1 -- 172.21.15.152:0/1682046809 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7f5dd402f030 con 0x7f5dec0b4870 2024-04-20T11:33:34.384 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.384+0000 7f5de9e00640 1 --2- 172.21.15.152:0/1682046809 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f5dc806ca10 0x7f5dc806eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:34.384 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.384+0000 7f5df383d740 1 Processor -- start 2024-04-20T11:33:34.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.384+0000 7f5df383d740 1 -- start start 2024-04-20T11:33:34.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.385+0000 7f5df383d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561fffead670 0x561fffe64ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:34.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.385+0000 7f5df383d740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x561fffe636f0 0x561fffe651e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:34.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.385+0000 7f5df383d740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561fffe5fb90 0x561fffe8f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:34.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.385+0000 7f5df383d740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x561fffdcb9d0 con 0x561fffe5fb90 2024-04-20T11:33:34.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.385+0000 7f5df383d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x561fffd72270 con 0x561fffead670 2024-04-20T11:33:34.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.385+0000 7f5df383d740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x561fffdc7990 con 0x561fffe636f0 2024-04-20T11:33:34.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.385+0000 7f5de9e00640 1 --2- 172.21.15.152:0/1682046809 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f5dc806ca10 0x7f5dc806eed0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc0012e0 tx=0x7f5ddc009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:34.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.385+0000 7f5dea800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561fffe5fb90 0x561fffe8f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:34.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.385+0000 7f5dea800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561fffe5fb90 0x561fffe8f980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54352/0 (socket says 172.21.15.152:54352) 2024-04-20T11:33:34.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.385+0000 7f5dea800640 1 -- 172.21.15.152:0/2513196152 learned_addr learned my addr 172.21.15.152:0/2513196152 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:33:34.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.386+0000 7f5de1400640 1 --2- 172.21.15.152:0/2513196152 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x561fffe636f0 0x561fffe651e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:34.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.386+0000 7f5de9e00640 1 --2- 172.21.15.152:0/2513196152 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561fffead670 0x561fffe64ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:34.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.386+0000 7f5dea800640 1 -- 172.21.15.152:0/2513196152 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x561fffe636f0 msgr2=0x561fffe651e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:34.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.386+0000 7f5dea800640 1 --2- 172.21.15.152:0/2513196152 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x561fffe636f0 0x561fffe651e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:34.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.386+0000 7f5dea800640 1 -- 172.21.15.152:0/2513196152 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561fffead670 msgr2=0x561fffe64ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:34.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.386+0000 7f5dea800640 1 --2- 172.21.15.152:0/2513196152 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561fffead670 0x561fffe64ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:34.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.386+0000 7f5dea800640 1 -- 172.21.15.152:0/2513196152 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fffade2d0 con 0x561fffe5fb90 2024-04-20T11:33:34.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.386+0000 7f5dea800640 1 --2- 172.21.15.152:0/2513196152 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561fffe5fb90 0x561fffe8f980 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f5de4003d40 tx=0x7f5de4004200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5132b8d26d3e7e3c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:34.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.386+0000 7f5dda600640 1 -- 172.21.15.152:0/2513196152 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de4017070 con 0x561fffe5fb90 2024-04-20T11:33:34.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.386+0000 7f5dda600640 1 -- 172.21.15.152:0/2513196152 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5de4005480 con 0x561fffe5fb90 2024-04-20T11:33:34.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.386+0000 7f5dda600640 1 -- 172.21.15.152:0/2513196152 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de400c8e0 con 0x561fffe5fb90 2024-04-20T11:33:34.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.387+0000 7f5df383d740 1 -- 172.21.15.152:0/2513196152 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561fffe5fb90 msgr2=0x561fffe8f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:34.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.387+0000 7f5df383d740 1 --2- 172.21.15.152:0/2513196152 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561fffe5fb90 0x561fffe8f980 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f5de4003d40 tx=0x7f5de4004200 comp rx=0 tx=0).stop 2024-04-20T11:33:34.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.387+0000 7f5df383d740 1 -- 172.21.15.152:0/2513196152 shutdown_connections 2024-04-20T11:33:34.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.387+0000 7f5df383d740 1 --2- 172.21.15.152:0/2513196152 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561fffe5fb90 0x561fffe8f980 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:34.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.387+0000 7f5df383d740 1 --2- 172.21.15.152:0/2513196152 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x561fffe636f0 0x561fffe651e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:34.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.387+0000 7f5df383d740 1 --2- 172.21.15.152:0/2513196152 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561fffead670 0x561fffe64ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:34.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.387+0000 7f5df383d740 1 -- 172.21.15.152:0/2513196152 >> 172.21.15.152:0/2513196152 conn(0x561fffe61e00 msgr2=0x561fffeaa980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:34.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.387+0000 7f5df383d740 1 -- 172.21.15.152:0/2513196152 shutdown_connections 2024-04-20T11:33:34.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.387+0000 7f5df383d740 1 -- 172.21.15.152:0/2513196152 wait complete. 2024-04-20T11:33:34.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.388+0000 7f5df383d740 1 -- start start 2024-04-20T11:33:34.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.388+0000 7f5df383d740 1 Processor -- start 2024-04-20T11:33:34.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.389+0000 7f5df383d740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561fffe636f0 0x561fffe8d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:34.393 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.389+0000 7f5df383d740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x561fffead670 0x561fffe8e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:34.393 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.389+0000 7f5df383d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561fffe7a220 0x561fffe8ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:34.393 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.389+0000 7f5df383d740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x561fffe63c00 con 0x561fffe636f0 2024-04-20T11:33:34.393 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.389+0000 7f5df383d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x561fffd855e0 con 0x561fffe7a220 2024-04-20T11:33:34.393 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.389+0000 7f5df383d740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x561fffe63ee0 con 0x561fffead670 2024-04-20T11:33:34.393 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.389+0000 7f5df383d740 10 client.0 _ll_register_callbacks cb 0x561fffaddac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:33:34.393 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.389+0000 7f5de9e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561fffe636f0 0x561fffe8d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:34.393 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.390+0000 7f5de9e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561fffe636f0 0x561fffe8d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54362/0 (socket says 172.21.15.152:54362) 2024-04-20T11:33:34.393 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.390+0000 7f5de9e00640 1 -- 172.21.15.152:0/4089800445 learned_addr learned my addr 172.21.15.152:0/4089800445 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:33:34.393 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.390+0000 7f5de9e00640 1 -- 172.21.15.152:0/4089800445 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x561fffead670 msgr2=0x561fffe8e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:34.393 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.390+0000 7f5de9e00640 1 --2- 172.21.15.152:0/4089800445 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x561fffead670 0x561fffe8e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:34.393 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.390+0000 7f5de9e00640 1 -- 172.21.15.152:0/4089800445 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561fffe7a220 msgr2=0x561fffe8ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:34.393 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.390+0000 7f5de9e00640 1 --2- 172.21.15.152:0/4089800445 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561fffe7a220 0x561fffe8ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:34.393 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.390+0000 7f5de9e00640 1 -- 172.21.15.152:0/4089800445 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561fffba86b0 con 0x561fffe636f0 2024-04-20T11:33:34.394 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.390+0000 7f5de9e00640 1 --2- 172.21.15.152:0/4089800445 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561fffe636f0 0x561fffe8d860 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc007480 tx=0x7f5ddc000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f048ff52cfe5a10 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:34.394 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.390+0000 7f5dcea00640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-04-20T11:33:34.394 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.390+0000 7f5dcea00640 1 -- 172.21.15.152:0/4089800445 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ddc013630 con 0x561fffe636f0 2024-04-20T11:33:34.394 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.390+0000 7f5dcea00640 1 -- 172.21.15.152:0/4089800445 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7f5ddc021a80 con 0x561fffe636f0 2024-04-20T11:33:34.394 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.390+0000 7f5df383d740 1 -- 172.21.15.152:0/4089800445 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5de4005480 con 0x561fffe636f0 2024-04-20T11:33:34.394 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.391+0000 7f5dcea00640 1 -- 172.21.15.152:0/4089800445 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f5ddc021c80 con 0x561fffe636f0 2024-04-20T11:33:34.394 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.391+0000 7f5df383d740 10 client.7944 fetch_fsmap learned FSMap version 66 2024-04-20T11:33:34.394 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.391+0000 7f5df383d740 1 -- 172.21.15.152:0/4089800445 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=2+}) -- 0x561fffe8dda0 con 0x561fffe636f0 2024-04-20T11:33:34.394 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.391+0000 7f5dcea00640 1 -- 172.21.15.152:0/4089800445 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5de4005480 con 0x561fffe636f0 2024-04-20T11:33:34.394 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.391+0000 7f5dcea00640 1 -- 172.21.15.152:0/4089800445 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 66) ==== 3495+0+0 (secure 0 0 0) 0x7f5ddc021450 con 0x561fffe636f0 2024-04-20T11:33:34.394 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.391+0000 7f5df383d740 10 client.7944 fetch_fsmap finished waiting for FSMap version 66 2024-04-20T11:33:34.394 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.391+0000 7f5df383d740 10 client.7944 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-20T11:33:34.394 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.391+0000 7f5df383d740 20 client.7944 populate_metadata read hostname 'smithi152' 2024-04-20T11:33:34.394 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.391+0000 7f5df383d740 1 --2- 172.21.15.152:0/4089800445 >> [v2:172.21.15.152:6833/3582312712,v1:172.21.15.152:6835/3582312712] conn(0x561fffeb0030 0x561fffeca220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:34.395 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.391+0000 7f5df383d740 4 client.7944 mds_command: new command op to 7794 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:33:34.395 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.391+0000 7f5df383d740 1 -- 172.21.15.152:0/4089800445 --> [v2:172.21.15.152:6833/3582312712,v1:172.21.15.152:6835/3582312712] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561fffdd93f0 con 0x561fffeb0030 2024-04-20T11:33:34.395 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.392+0000 7f5de1400640 1 --2- 172.21.15.152:0/4089800445 >> [v2:172.21.15.152:6833/3582312712,v1:172.21.15.152:6835/3582312712] conn(0x561fffeb0030 0x561fffeca220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:34.395 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.392+0000 7f5de1400640 1 --2- 172.21.15.152:0/4089800445 >> [v2:172.21.15.152:6833/3582312712,v1:172.21.15.152:6835/3582312712] conn(0x561fffeb0030 0x561fffeca220 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93f3c62d02e2b350 server_cookie=4d55d848487eb2bc in_seq=0 out_seq=0 2024-04-20T11:33:34.395 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.392+0000 7f5dcea00640 10 client.7944 ms_handle_connect on v2:172.21.15.152:6833/3582312712 2024-04-20T11:33:34.396 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.393+0000 7f5dcea00640 1 -- 172.21.15.152:0/4089800445 <== mds.0 v2:172.21.15.152:6833/3582312712 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561fffdd93f0 con 0x561fffeb0030 2024-04-20T11:33:34.396 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.393+0000 7f5dcea00640 10 client.7944 handle_command_reply: tid=0 2024-04-20T11:33:34.396 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.393+0000 7f5df383d740 1 client.7944 shutdown 2024-04-20T11:33:34.396 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.393+0000 7f5df383d740 1 -- 172.21.15.152:0/4089800445 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561fffe636f0 msgr2=0x561fffe8d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:34.396 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.393+0000 7f5df383d740 1 --2- 172.21.15.152:0/4089800445 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561fffe636f0 0x561fffe8d860 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc007480 tx=0x7f5ddc000f40 comp rx=0 tx=0).stop 2024-04-20T11:33:34.396 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.393+0000 7f5df383d740 1 -- 172.21.15.152:0/4089800445 shutdown_connections 2024-04-20T11:33:34.397 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.393+0000 7f5df383d740 1 --2- 172.21.15.152:0/4089800445 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561fffe7a220 0x561fffe8ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:34.397 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.393+0000 7f5df383d740 1 --2- 172.21.15.152:0/4089800445 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x561fffead670 0x561fffe8e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:34.397 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.393+0000 7f5df383d740 1 --2- 172.21.15.152:0/4089800445 >> [v2:172.21.15.152:6833/3582312712,v1:172.21.15.152:6835/3582312712] conn(0x561fffeb0030 0x561fffeca220 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:34.397 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.393+0000 7f5df383d740 1 --2- 172.21.15.152:0/4089800445 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561fffe636f0 0x561fffe8d860 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:34.397 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.393+0000 7f5dcea00640 0 client.7944 ms_handle_reset on v2:172.21.15.152:6833/3582312712 2024-04-20T11:33:34.397 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.393+0000 7f5df383d740 1 -- 172.21.15.152:0/4089800445 >> 172.21.15.152:0/4089800445 conn(0x561fffe5fb90 msgr2=0x561fffeaa780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:34.397 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.394+0000 7f5df383d740 1 -- 172.21.15.152:0/4089800445 shutdown_connections 2024-04-20T11:33:34.397 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.394+0000 7f5df383d740 1 -- 172.21.15.152:0/4089800445 wait complete. 2024-04-20T11:33:34.397 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.394+0000 7f5df383d740 20 client.7944 trim_cache size 0 max 16384 2024-04-20T11:33:34.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.413+0000 7f5df383d740 1 Processor -- start 2024-04-20T11:33:34.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.413+0000 7f5df383d740 1 -- start start 2024-04-20T11:33:34.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.414+0000 7f5df383d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561fffe636f0 0x561fffea9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:34.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.414+0000 7f5df383d740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561fffe7a220 0x561fffecfa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:34.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.414+0000 7f5df383d740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x561fffead670 0x561fffe5eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:34.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.414+0000 7f5df383d740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f5dec200d70 con 0x561fffe7a220 2024-04-20T11:33:34.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.414+0000 7f5df383d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5dec1ffe50 con 0x561fffe636f0 2024-04-20T11:33:34.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.414+0000 7f5df383d740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5dec0ad8f0 con 0x561fffead670 2024-04-20T11:33:34.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.415+0000 7f5de1400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561fffe7a220 0x561fffecfa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:34.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.415+0000 7f5de1400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561fffe7a220 0x561fffecfa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54368/0 (socket says 172.21.15.152:54368) 2024-04-20T11:33:34.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.415+0000 7f5de1400640 1 -- 172.21.15.152:0/4011349377 learned_addr learned my addr 172.21.15.152:0/4011349377 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:33:34.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.415+0000 7f5de1400640 1 -- 172.21.15.152:0/4011349377 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x561fffead670 msgr2=0x561fffe5eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:34.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.415+0000 7f5de1400640 1 --2- 172.21.15.152:0/4011349377 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x561fffead670 0x561fffe5eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:34.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.415+0000 7f5de1400640 1 -- 172.21.15.152:0/4011349377 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561fffe636f0 msgr2=0x561fffea9ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:34.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.415+0000 7f5de1400640 1 --2- 172.21.15.152:0/4011349377 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561fffe636f0 0x561fffea9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:34.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.415+0000 7f5de1400640 1 -- 172.21.15.152:0/4011349377 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fffe8dbd0 con 0x561fffe7a220 2024-04-20T11:33:34.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.415+0000 7f5de1400640 1 --2- 172.21.15.152:0/4011349377 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561fffe7a220 0x561fffecfa30 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f5dd407b4c0 tx=0x7f5dd407af50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0628e2a0b6978 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:34.418 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.416+0000 7f5dd9c00640 1 -- 172.21.15.152:0/4011349377 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd4098070 con 0x561fffe7a220 2024-04-20T11:33:34.418 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.416+0000 7f5dd9c00640 1 -- 172.21.15.152:0/4011349377 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5dd4037040 con 0x561fffe7a220 2024-04-20T11:33:34.418 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.416+0000 7f5dd9c00640 1 -- 172.21.15.152:0/4011349377 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd4093070 con 0x561fffe7a220 2024-04-20T11:33:34.418 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.416+0000 7f5df383d740 1 -- 172.21.15.152:0/4011349377 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561fffe7a220 msgr2=0x561fffecfa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:34.418 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.416+0000 7f5df383d740 1 --2- 172.21.15.152:0/4011349377 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561fffe7a220 0x561fffecfa30 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f5dd407b4c0 tx=0x7f5dd407af50 comp rx=0 tx=0).stop 2024-04-20T11:33:34.418 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.416+0000 7f5df383d740 1 -- 172.21.15.152:0/4011349377 shutdown_connections 2024-04-20T11:33:34.418 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.416+0000 7f5df383d740 1 --2- 172.21.15.152:0/4011349377 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x561fffead670 0x561fffe5eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:34.418 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.416+0000 7f5df383d740 1 --2- 172.21.15.152:0/4011349377 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561fffe7a220 0x561fffecfa30 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:34.418 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.416+0000 7f5df383d740 1 --2- 172.21.15.152:0/4011349377 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561fffe636f0 0x561fffea9ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:34.418 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.416+0000 7f5df383d740 1 -- 172.21.15.152:0/4011349377 >> 172.21.15.152:0/4011349377 conn(0x561fffe5fb90 msgr2=0x561fffeb1da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:34.419 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.416+0000 7f5df383d740 1 -- 172.21.15.152:0/4011349377 shutdown_connections 2024-04-20T11:33:34.419 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.416+0000 7f5df383d740 1 -- 172.21.15.152:0/4011349377 wait complete. 2024-04-20T11:33:34.419 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.417+0000 7f5df383d740 1 -- start start 2024-04-20T11:33:34.419 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.417+0000 7f5df383d740 1 Processor -- start 2024-04-20T11:33:34.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.418+0000 7f5df383d740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x561fffe636f0 0x561fffeac570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:34.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.418+0000 7f5df383d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561fffe7a220 0x561fffebdef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:34.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.418+0000 7f5df383d740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561fffead670 0x561fffebe430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:34.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.418+0000 7f5df383d740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x561fffeb17e0 con 0x561fffead670 2024-04-20T11:33:34.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.418+0000 7f5df383d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5dec0adf40 con 0x561fffe7a220 2024-04-20T11:33:34.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.418+0000 7f5df383d740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x561fffeb2e90 con 0x561fffe636f0 2024-04-20T11:33:34.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.418+0000 7f5df383d740 10 client.0 _ll_register_callbacks cb 0x7f5ddc001cd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:33:34.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.419+0000 7f5dea800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561fffead670 0x561fffebe430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:34.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.419+0000 7f5dea800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561fffead670 0x561fffebe430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54378/0 (socket says 172.21.15.152:54378) 2024-04-20T11:33:34.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.419+0000 7f5dea800640 1 -- 172.21.15.152:0/1722387486 learned_addr learned my addr 172.21.15.152:0/1722387486 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:33:34.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.419+0000 7f5de1400640 1 --2- 172.21.15.152:0/1722387486 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561fffe7a220 0x561fffebdef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:34.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.419+0000 7f5de9e00640 1 --2- 172.21.15.152:0/1722387486 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x561fffe636f0 0x561fffeac570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:34.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.419+0000 7f5dea800640 1 -- 172.21.15.152:0/1722387486 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x561fffe636f0 msgr2=0x561fffeac570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:34.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.419+0000 7f5dea800640 1 --2- 172.21.15.152:0/1722387486 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x561fffe636f0 0x561fffeac570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:34.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.419+0000 7f5dea800640 1 -- 172.21.15.152:0/1722387486 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561fffe7a220 msgr2=0x561fffebdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:34.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.419+0000 7f5dea800640 1 --2- 172.21.15.152:0/1722387486 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561fffe7a220 0x561fffebdef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:34.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.419+0000 7f5dea800640 1 -- 172.21.15.152:0/1722387486 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561fffe60fb0 con 0x561fffead670 2024-04-20T11:33:34.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.419+0000 7f5dea800640 1 --2- 172.21.15.152:0/1722387486 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561fffead670 0x561fffebe430 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f5de4012000 tx=0x7f5de4007c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=908ba9b14038cb9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:34.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.419+0000 7f5dcea00640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-04-20T11:33:34.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.420+0000 7f5dcea00640 1 -- 172.21.15.152:0/1722387486 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de4021020 con 0x561fffead670 2024-04-20T11:33:34.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.420+0000 7f5df383d740 1 -- 172.21.15.152:0/1722387486 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5dd4037040 con 0x561fffead670 2024-04-20T11:33:34.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.420+0000 7f5dcea00640 1 -- 172.21.15.152:0/1722387486 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7f5de40074d0 con 0x561fffead670 2024-04-20T11:33:34.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.420+0000 7f5dcea00640 1 -- 172.21.15.152:0/1722387486 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f5de402b050 con 0x561fffead670 2024-04-20T11:33:34.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.420+0000 7f5df383d740 10 client.7950 fetch_fsmap learned FSMap version 66 2024-04-20T11:33:34.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.420+0000 7f5df383d740 1 -- 172.21.15.152:0/1722387486 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=2+}) -- 0x561fffeaa9e0 con 0x561fffead670 2024-04-20T11:33:34.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.420+0000 7f5dcea00640 1 -- 172.21.15.152:0/1722387486 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5dd4037040 con 0x561fffead670 2024-04-20T11:33:34.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.420+0000 7f5dcea00640 1 -- 172.21.15.152:0/1722387486 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 66) ==== 3495+0+0 (secure 0 0 0) 0x7f5de402e020 con 0x561fffead670 2024-04-20T11:33:34.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.420+0000 7f5df383d740 10 client.7950 fetch_fsmap finished waiting for FSMap version 66 2024-04-20T11:33:34.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.420+0000 7f5df383d740 10 client.7950 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-20T11:33:34.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.420+0000 7f5df383d740 20 client.7950 populate_metadata read hostname 'smithi152' 2024-04-20T11:33:34.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.420+0000 7f5df383d740 1 --2- 172.21.15.152:0/1722387486 >> [v2:172.21.15.152:6833/3582312712,v1:172.21.15.152:6835/3582312712] conn(0x561ffff2df30 0x561fffe8df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:34.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.420+0000 7f5df383d740 4 client.7950 mds_command: new command op to 7794 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-20T11:33:34.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.420+0000 7f5df383d740 1 -- 172.21.15.152:0/1722387486 --> [v2:172.21.15.152:6833/3582312712,v1:172.21.15.152:6835/3582312712] -- command(tid 0: {"prefix": "status"}) -- 0x561fffe77700 con 0x561ffff2df30 2024-04-20T11:33:34.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.421+0000 7f5de9e00640 1 --2- 172.21.15.152:0/1722387486 >> [v2:172.21.15.152:6833/3582312712,v1:172.21.15.152:6835/3582312712] conn(0x561ffff2df30 0x561fffe8df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:34.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.421+0000 7f5de9e00640 1 --2- 172.21.15.152:0/1722387486 >> [v2:172.21.15.152:6833/3582312712,v1:172.21.15.152:6835/3582312712] conn(0x561ffff2df30 0x561fffe8df70 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=924d2541445fab0c server_cookie=1a926037158c51ce in_seq=0 out_seq=0 2024-04-20T11:33:34.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.421+0000 7f5dcea00640 10 client.7950 ms_handle_connect on v2:172.21.15.152:6833/3582312712 2024-04-20T11:33:34.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.421+0000 7f5dcea00640 1 -- 172.21.15.152:0/1722387486 <== mds.0 v2:172.21.15.152:6833/3582312712 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x561fffe77700 con 0x561ffff2df30 2024-04-20T11:33:34.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.421+0000 7f5dcea00640 10 client.7950 handle_command_reply: tid=0 2024-04-20T11:33:34.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.422+0000 7f5df383d740 1 client.7950 shutdown 2024-04-20T11:33:34.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.422+0000 7f5df383d740 1 -- 172.21.15.152:0/1722387486 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561fffead670 msgr2=0x561fffebe430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:34.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.422+0000 7f5df383d740 1 --2- 172.21.15.152:0/1722387486 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561fffead670 0x561fffebe430 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f5de4012000 tx=0x7f5de4007c10 comp rx=0 tx=0).stop 2024-04-20T11:33:34.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.422+0000 7f5df383d740 1 -- 172.21.15.152:0/1722387486 shutdown_connections 2024-04-20T11:33:34.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.422+0000 7f5df383d740 1 --2- 172.21.15.152:0/1722387486 >> [v2:172.21.15.152:6833/3582312712,v1:172.21.15.152:6835/3582312712] conn(0x561ffff2df30 0x561fffe8df70 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:34.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.422+0000 7f5df383d740 1 --2- 172.21.15.152:0/1722387486 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561fffead670 0x561fffebe430 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:34.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.422+0000 7f5df383d740 1 --2- 172.21.15.152:0/1722387486 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561fffe7a220 0x561fffebdef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:34.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.422+0000 7f5dcea00640 0 client.7950 ms_handle_reset on v2:172.21.15.152:6833/3582312712 2024-04-20T11:33:34.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.422+0000 7f5df383d740 1 --2- 172.21.15.152:0/1722387486 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x561fffe636f0 0x561fffeac570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:34.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.422+0000 7f5df383d740 1 -- 172.21.15.152:0/1722387486 >> 172.21.15.152:0/1722387486 conn(0x561fffe5fb90 msgr2=0x561fffe64030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:34.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.422+0000 7f5df383d740 1 -- 172.21.15.152:0/1722387486 shutdown_connections 2024-04-20T11:33:34.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.422+0000 7f5df383d740 1 -- 172.21.15.152:0/1722387486 wait complete. 2024-04-20T11:33:34.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.422+0000 7f5df383d740 20 client.7950 trim_cache size 0 max 16384 2024-04-20T11:33:34.430 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-04-20T11:33:34.430 INFO:teuthology.orchestra.run.smithi152.stdout: "cluster_fsid": "c8a72131-0d4f-42c7-892f-3f3af4d2bd3d", 2024-04-20T11:33:34.430 INFO:teuthology.orchestra.run.smithi152.stdout: "whoami": 0, 2024-04-20T11:33:34.430 INFO:teuthology.orchestra.run.smithi152.stdout: "id": 7794, 2024-04-20T11:33:34.430 INFO:teuthology.orchestra.run.smithi152.stdout: "want_state": "up:active", 2024-04-20T11:33:34.430 INFO:teuthology.orchestra.run.smithi152.stdout: "state": "up:active", 2024-04-20T11:33:34.431 INFO:teuthology.orchestra.run.smithi152.stdout: "fs_name": "cephfs", 2024-04-20T11:33:34.431 INFO:teuthology.orchestra.run.smithi152.stdout: "rank_uptime": 6.6087603699999997, 2024-04-20T11:33:34.431 INFO:teuthology.orchestra.run.smithi152.stdout: "mdsmap_epoch": 66, 2024-04-20T11:33:34.431 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_epoch": 103, 2024-04-20T11:33:34.431 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_epoch_barrier": 103, 2024-04-20T11:33:34.431 INFO:teuthology.orchestra.run.smithi152.stdout: "uptime": 13.095253934 2024-04-20T11:33:34.431 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-04-20T11:33:34.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.423+0000 7f5deb200640 1 -- 172.21.15.152:0/1682046809 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f5dc806ca10 msgr2=0x7f5dc806eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:34.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.423+0000 7f5deb200640 1 --2- 172.21.15.152:0/1682046809 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f5dc806ca10 0x7f5dc806eed0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc0012e0 tx=0x7f5ddc009040 comp rx=0 tx=0).stop 2024-04-20T11:33:34.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.423+0000 7f5deb200640 1 -- 172.21.15.152:0/1682046809 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5dec0b4870 msgr2=0x7f5dec0b1590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:34.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.423+0000 7f5deb200640 1 --2- 172.21.15.152:0/1682046809 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5dec0b4870 0x7f5dec0b1590 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f5dd4000c00 tx=0x7f5dd40046c0 comp rx=0 tx=0).stop 2024-04-20T11:33:34.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.423+0000 7f5deb200640 1 -- 172.21.15.152:0/1682046809 shutdown_connections 2024-04-20T11:33:34.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.423+0000 7f5deb200640 1 --2- 172.21.15.152:0/1682046809 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f5dc806ca10 0x7f5dc806eed0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:34.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.423+0000 7f5deb200640 1 --2- 172.21.15.152:0/1682046809 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5dec1f7ae0 0x7f5dec1f7f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:34.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.423+0000 7f5deb200640 1 --2- 172.21.15.152:0/1682046809 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5dec0b4870 0x7f5dec0b1590 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:34.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.423+0000 7f5deb200640 1 --2- 172.21.15.152:0/1682046809 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5dec073b60 0x7f5dec156be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:34.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.423+0000 7f5deb200640 1 -- 172.21.15.152:0/1682046809 >> 172.21.15.152:0/1682046809 conn(0x7f5dec0d0260 msgr2=0x7f5dec14f5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:34.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.423+0000 7f5deb200640 1 -- 172.21.15.152:0/1682046809 shutdown_connections 2024-04-20T11:33:34.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.423+0000 7f5deb200640 1 -- 172.21.15.152:0/1682046809 wait complete. 2024-04-20T11:33:34.441 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-20T11:33:34.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.639+0000 7fd553200640 1 Processor -- start 2024-04-20T11:33:34.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.639+0000 7fd553200640 1 -- start start 2024-04-20T11:33:34.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.641+0000 7fd553200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fd554071cd0 0x7fd5540720b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:34.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.641+0000 7fd553200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fd55406ea20 0x7fd5540b4a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:34.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.641+0000 7fd553200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5540725f0 con 0x7fd5540b4f90 2024-04-20T11:33:34.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.641+0000 7fd553200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fd55406dcf0 con 0x7fd554071cd0 2024-04-20T11:33:34.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.641+0000 7fd553200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fd5540b5510 con 0x7fd55406ea20 2024-04-20T11:33:34.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.641+0000 7fd551e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fd554071cd0 0x7fd5540720b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:34.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.641+0000 7fd551e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fd554071cd0 0x7fd5540720b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54380/0 (socket says 172.21.15.152:54380) 2024-04-20T11:33:34.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.641+0000 7fd551e00640 1 -- 172.21.15.152:0/880061236 learned_addr learned my addr 172.21.15.152:0/880061236 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:33:34.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.642+0000 7fd551e00640 1 -- 172.21.15.152:0/880061236 >> v2:172.21.15.175:3300/0 conn(0x7fd55406ea20 msgr2=0x7fd5540b4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:34.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.642+0000 7fd551400640 1 --2- 172.21.15.152:0/880061236 >> v2:172.21.15.175:3300/0 conn(0x7fd55406ea20 0x7fd5540b4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:34.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.642+0000 7fd551e00640 1 --2- 172.21.15.152:0/880061236 >> v2:172.21.15.175:3300/0 conn(0x7fd55406ea20 0x7fd5540b4a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:34.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.642+0000 7fd551e00640 1 -- 172.21.15.152:0/880061236 >> v1:172.21.15.175:6789/0 conn(0x7fd5540b4f90 legacy=0x7fd5540cdeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:34.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.642+0000 7fd551e00640 1 -- 172.21.15.152:0/880061236 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd55406df40 con 0x7fd554071cd0 2024-04-20T11:33:34.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.642+0000 7fd551400640 1 --2- 172.21.15.152:0/880061236 >> v2:172.21.15.175:3300/0 conn(0x7fd55406ea20 0x7fd5540b4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:33:34.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.643+0000 7fd551e00640 1 --2- 172.21.15.152:0/880061236 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd554071cd0 0x7fd5540720b0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7fd5541641d0 tx=0x7fd54402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=209b204a4e5dd822 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:34.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.644+0000 7fd550a00640 1 -- 172.21.15.152:0/880061236 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd544035020 con 0x7fd554071cd0 2024-04-20T11:33:34.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.644+0000 7fd550a00640 1 -- 172.21.15.152:0/880061236 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd544031870 con 0x7fd554071cd0 2024-04-20T11:33:34.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.644+0000 7fd550a00640 1 -- 172.21.15.152:0/880061236 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd544031c40 con 0x7fd554071cd0 2024-04-20T11:33:34.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.644+0000 7fd553200640 1 -- 172.21.15.152:0/880061236 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd554071cd0 msgr2=0x7fd5540720b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:34.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.644+0000 7fd553200640 1 --2- 172.21.15.152:0/880061236 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd554071cd0 0x7fd5540720b0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7fd5541641d0 tx=0x7fd54402b910 comp rx=0 tx=0).stop 2024-04-20T11:33:34.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.645+0000 7fd553200640 1 -- 172.21.15.152:0/880061236 shutdown_connections 2024-04-20T11:33:34.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.645+0000 7fd553200640 1 --2- 172.21.15.152:0/880061236 >> v2:172.21.15.175:3300/0 conn(0x7fd55406ea20 0x7fd5540b4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:34.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.645+0000 7fd553200640 1 --2- 172.21.15.152:0/880061236 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd554071cd0 0x7fd5540720b0 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:34.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.645+0000 7fd553200640 1 -- 172.21.15.152:0/880061236 >> 172.21.15.152:0/880061236 conn(0x7fd554067ae0 msgr2=0x7fd55406b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:34.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.645+0000 7fd553200640 1 -- 172.21.15.152:0/880061236 shutdown_connections 2024-04-20T11:33:34.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.645+0000 7fd553200640 1 -- 172.21.15.152:0/880061236 wait complete. 2024-04-20T11:33:34.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.647+0000 7fd553200640 1 Processor -- start 2024-04-20T11:33:34.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.647+0000 7fd553200640 1 -- start start 2024-04-20T11:33:34.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.648+0000 7fd553200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd55406ea20 0x7fd55414a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:34.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.648+0000 7fd553200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd554071cd0 0x7fd554147550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:34.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.648+0000 7fd553200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd5540b4f90 0x7fd55415a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:34.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.648+0000 7fd553200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fd554163e60 con 0x7fd5540b4f90 2024-04-20T11:33:34.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.648+0000 7fd553200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd55406e500 con 0x7fd554071cd0 2024-04-20T11:33:34.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.648+0000 7fd553200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd5540b4180 con 0x7fd55406ea20 2024-04-20T11:33:34.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.649+0000 7fd552800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd5540b4f90 0x7fd55415a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:34.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.649+0000 7fd552800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd5540b4f90 0x7fd55415a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54386/0 (socket says 172.21.15.152:54386) 2024-04-20T11:33:34.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.649+0000 7fd552800640 1 -- 172.21.15.152:0/1003907016 learned_addr learned my addr 172.21.15.152:0/1003907016 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:33:34.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.649+0000 7fd552800640 1 -- 172.21.15.152:0/1003907016 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd55406ea20 msgr2=0x7fd55414a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:34.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.649+0000 7fd552800640 1 --2- 172.21.15.152:0/1003907016 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd55406ea20 0x7fd55414a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:34.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.649+0000 7fd552800640 1 -- 172.21.15.152:0/1003907016 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd554071cd0 msgr2=0x7fd554147550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:34.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.649+0000 7fd551400640 1 --2- 172.21.15.152:0/1003907016 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd554071cd0 0x7fd554147550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:34.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.649+0000 7fd552800640 1 --2- 172.21.15.152:0/1003907016 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd554071cd0 0x7fd554147550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:34.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.649+0000 7fd552800640 1 -- 172.21.15.152:0/1003907016 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd55414ad80 con 0x7fd5540b4f90 2024-04-20T11:33:34.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.649+0000 7fd552800640 1 --2- 172.21.15.152:0/1003907016 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd5540b4f90 0x7fd55415a330 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7fd54c00b410 tx=0x7fd54c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:34.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.650+0000 7fd54a000640 1 -- 172.21.15.152:0/1003907016 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd54c015030 con 0x7fd5540b4f90 2024-04-20T11:33:34.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.650+0000 7fd54a000640 1 -- 172.21.15.152:0/1003907016 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd54c004020 con 0x7fd5540b4f90 2024-04-20T11:33:34.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.650+0000 7fd553200640 1 -- 172.21.15.152:0/1003907016 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd554147a90 con 0x7fd5540b4f90 2024-04-20T11:33:34.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.650+0000 7fd54a000640 1 -- 172.21.15.152:0/1003907016 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd54c004410 con 0x7fd5540b4f90 2024-04-20T11:33:34.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.650+0000 7fd553200640 1 -- 172.21.15.152:0/1003907016 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd554148420 con 0x7fd5540b4f90 2024-04-20T11:33:34.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.651+0000 7fd553200640 1 -- 172.21.15.152:0/1003907016 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd55406b980 con 0x7fd5540b4f90 2024-04-20T11:33:34.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.653+0000 7fd54a000640 1 -- 172.21.15.152:0/1003907016 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 91142+0+0 (secure 0 0 0) 0x7fd54c018070 con 0x7fd5540b4f90 2024-04-20T11:33:34.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.654+0000 7fd54a000640 1 --2- 172.21.15.152:0/1003907016 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd53006ca10 0x7fd53006eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:34.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.654+0000 7fd54a000640 1 -- 172.21.15.152:0/1003907016 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7fd54c08b820 con 0x7fd5540b4f90 2024-04-20T11:33:34.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.655+0000 7fd551e00640 1 --2- 172.21.15.152:0/1003907016 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd53006ca10 0x7fd53006eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:34.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.656+0000 7fd551e00640 1 --2- 172.21.15.152:0/1003907016 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd53006ca10 0x7fd53006eed0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd544036000 tx=0x7fd544030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:34.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.660+0000 7fd54a000640 1 -- 172.21.15.152:0/1003907016 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fd55406b980 con 0x7fd5540b4f90 2024-04-20T11:33:34.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.909+0000 7fd553200640 1 -- 172.21.15.152:0/1003907016 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fd554162330 con 0x7fd5540b4f90 2024-04-20T11:33:34.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.910+0000 7fd54a000640 1 -- 172.21.15.152:0/1003907016 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v66) ==== 53+0+73 (secure 0 0 0) 0x7fd55414ad80 con 0x7fd5540b4f90 2024-04-20T11:33:34.911 INFO:teuthology.orchestra.run.smithi152.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-20T11:33:34.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.912+0000 7fd553200640 1 -- 172.21.15.152:0/1003907016 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd53006ca10 msgr2=0x7fd53006eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:34.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.912+0000 7fd553200640 1 --2- 172.21.15.152:0/1003907016 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd53006ca10 0x7fd53006eed0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd544036000 tx=0x7fd544030000 comp rx=0 tx=0).stop 2024-04-20T11:33:34.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.912+0000 7fd553200640 1 -- 172.21.15.152:0/1003907016 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd5540b4f90 msgr2=0x7fd55415a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:34.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.912+0000 7fd553200640 1 --2- 172.21.15.152:0/1003907016 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd5540b4f90 0x7fd55415a330 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7fd54c00b410 tx=0x7fd54c00b8d0 comp rx=0 tx=0).stop 2024-04-20T11:33:34.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.912+0000 7fd553200640 1 -- 172.21.15.152:0/1003907016 shutdown_connections 2024-04-20T11:33:34.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.912+0000 7fd553200640 1 --2- 172.21.15.152:0/1003907016 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd53006ca10 0x7fd53006eed0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:34.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.912+0000 7fd553200640 1 --2- 172.21.15.152:0/1003907016 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd5540b4f90 0x7fd55415a330 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:34.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.912+0000 7fd553200640 1 --2- 172.21.15.152:0/1003907016 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd554071cd0 0x7fd554147550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:34.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.912+0000 7fd553200640 1 --2- 172.21.15.152:0/1003907016 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd55406ea20 0x7fd55414a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:34.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.912+0000 7fd553200640 1 -- 172.21.15.152:0/1003907016 >> 172.21.15.152:0/1003907016 conn(0x7fd554067ae0 msgr2=0x7fd554160780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:34.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.912+0000 7fd553200640 1 -- 172.21.15.152:0/1003907016 shutdown_connections 2024-04-20T11:33:34.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:34.913+0000 7fd553200640 1 -- 172.21.15.152:0/1003907016 wait complete. 2024-04-20T11:33:34.931 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-20T11:33:34.932 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-20T11:33:34.932 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi152.front.sepia.ceph.com 2024-04-20T11:33:34.932 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-20T11:33:34.932 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-20T11:33:34.932 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-20T11:33:34.932 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-20T11:33:34.932 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-20T11:33:34.932 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-20T11:33:34.932 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:34.932 DEBUG:teuthology.orchestra.run.smithi152:> ip addr 2024-04-20T11:33:34.985 INFO:teuthology.orchestra.run.smithi152.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-20T11:33:34.985 INFO:teuthology.orchestra.run.smithi152.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-20T11:33:34.986 INFO:teuthology.orchestra.run.smithi152.stdout: inet 127.0.0.1/8 scope host lo 2024-04-20T11:33:34.986 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:33:34.986 INFO:teuthology.orchestra.run.smithi152.stdout: inet6 ::1/128 scope host 2024-04-20T11:33:34.986 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:33:34.986 INFO:teuthology.orchestra.run.smithi152.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T11:33:34.986 INFO:teuthology.orchestra.run.smithi152.stdout: link/ether 0c:c4:7a:da:6c:44 brd ff:ff:ff:ff:ff:ff 2024-04-20T11:33:34.986 INFO:teuthology.orchestra.run.smithi152.stdout: altname enp7s0f0 2024-04-20T11:33:34.986 INFO:teuthology.orchestra.run.smithi152.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T11:33:34.986 INFO:teuthology.orchestra.run.smithi152.stdout: link/ether 0c:c4:7a:da:6c:45 brd ff:ff:ff:ff:ff:ff 2024-04-20T11:33:34.986 INFO:teuthology.orchestra.run.smithi152.stdout: altname enp7s0f1 2024-04-20T11:33:34.986 INFO:teuthology.orchestra.run.smithi152.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T11:33:34.986 INFO:teuthology.orchestra.run.smithi152.stdout: link/ether 0c:c4:7a:88:6d:e2 brd ff:ff:ff:ff:ff:ff 2024-04-20T11:33:34.986 INFO:teuthology.orchestra.run.smithi152.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-20T11:33:34.986 INFO:teuthology.orchestra.run.smithi152.stdout: link/ether 0c:c4:7a:88:6d:e3 brd ff:ff:ff:ff:ff:ff 2024-04-20T11:33:34.987 INFO:teuthology.orchestra.run.smithi152.stdout: inet 172.21.15.152/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-20T11:33:34.987 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft 42391sec preferred_lft 42391sec 2024-04-20T11:33:34.987 INFO:teuthology.orchestra.run.smithi152.stdout: inet6 fe80::ec4:7aff:fe88:6de3/64 scope link 2024-04-20T11:33:34.987 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:33:34.987 INFO:teuthology.orchestra.run.smithi152.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-20T11:33:34.987 INFO:teuthology.orchestra.run.smithi152.stdout: link/ether 06:3e:9a:d3:49:1e brd ff:ff:ff:ff:ff:ff 2024-04-20T11:33:34.987 INFO:teuthology.orchestra.run.smithi152.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-20T11:33:34.987 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:33:34.987 INFO:teuthology.orchestra.run.smithi152.stdout: inet6 fe80::a019:d8ff:fe70:1185/64 scope link 2024-04-20T11:33:34.987 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:33:34.987 INFO:teuthology.orchestra.run.smithi152.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-20T11:33:34.987 INFO:teuthology.orchestra.run.smithi152.stdout: link/ether 16:e5:08:78:20:47 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T11:33:34.987 INFO:teuthology.orchestra.run.smithi152.stdout: inet6 fe80::14e5:8ff:fe78:2047/64 scope link 2024-04-20T11:33:34.987 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:33:34.988 INFO:teuthology.orchestra.run.smithi152.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-20T11:33:34.988 INFO:teuthology.orchestra.run.smithi152.stdout: link/ether 06:3e:9a:d3:49:1e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T11:33:34.988 INFO:teuthology.orchestra.run.smithi152.stdout: inet6 fe80::43e:9aff:fed3:491e/64 scope link 2024-04-20T11:33:34.988 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:33:34.989 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:34.989 DEBUG:teuthology.orchestra.run.smithi152:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-20T11:33:35.064 INFO:teuthology.orchestra.run.smithi152.stdout:1 2024-04-20T11:33:35.068 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:35.068 DEBUG:teuthology.orchestra.run.smithi152:> route 2024-04-20T11:33:35.119 INFO:teuthology.orchestra.run.smithi152.stdout:Kernel IP routing table 2024-04-20T11:33:35.119 INFO:teuthology.orchestra.run.smithi152.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-20T11:33:35.120 INFO:teuthology.orchestra.run.smithi152.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-20T11:33:35.120 INFO:teuthology.orchestra.run.smithi152.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-20T11:33:35.120 INFO:teuthology.orchestra.run.smithi152.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-20T11:33:35.120 DEBUG:teuthology.orchestra.run.smithi152:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T11:33:35.120 DEBUG:teuthology.orchestra.run.smithi152:> set -e 2024-04-20T11:33:35.120 DEBUG:teuthology.orchestra.run.smithi152:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-20T11:33:35.120 DEBUG:teuthology.orchestra.run.smithi152:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-20T11:33:35.120 DEBUG:teuthology.orchestra.run.smithi152:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-20T11:33:35.120 DEBUG:teuthology.orchestra.run.smithi152:> ') 2024-04-20T11:33:35.221 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:35 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T11:33:35.318 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T11:33:35.328 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:35.328 DEBUG:teuthology.orchestra.run.smithi152:> ip netns list 2024-04-20T11:33:35.375 INFO:teuthology.orchestra.run.smithi152.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-20T11:33:35.375 INFO:teuthology.orchestra.run.smithi152.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-20T11:33:35.375 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-20T11:33:35.375 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:35.375 DEBUG:teuthology.orchestra.run.smithi152:> sudo ip link set brx.0 up 2024-04-20T11:33:35.506 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-20T11:33:35.506 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T11:33:35.507 DEBUG:teuthology.orchestra.run.smithi152:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-20T11:33:35.560 INFO:teuthology.orchestra.run.smithi152.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-20T11:33:35.560 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T11:33:35.560 DEBUG:teuthology.orchestra.run.smithi152:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-20T11:33:35.611 DEBUG:teuthology.orchestra.run.smithi152:> sudo modprobe fuse 2024-04-20T11:33:35.693 DEBUG:teuthology.orchestra.run.smithi152:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T11:33:35.745 INFO:teuthology.orchestra.run.smithi152.stdout:/proc 2024-04-20T11:33:35.745 INFO:teuthology.orchestra.run.smithi152.stdout:/sys 2024-04-20T11:33:35.746 INFO:teuthology.orchestra.run.smithi152.stdout:/dev 2024-04-20T11:33:35.746 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/security 2024-04-20T11:33:35.746 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/shm 2024-04-20T11:33:35.746 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/pts 2024-04-20T11:33:35.746 INFO:teuthology.orchestra.run.smithi152.stdout:/run 2024-04-20T11:33:35.746 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/cgroup 2024-04-20T11:33:35.746 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/pstore 2024-04-20T11:33:35.746 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/bpf 2024-04-20T11:33:35.746 INFO:teuthology.orchestra.run.smithi152.stdout:/ 2024-04-20T11:33:35.746 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/selinux 2024-04-20T11:33:35.746 INFO:teuthology.orchestra.run.smithi152.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T11:33:35.746 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/mqueue 2024-04-20T11:33:35.747 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/hugepages 2024-04-20T11:33:35.747 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/debug 2024-04-20T11:33:35.747 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/tracing 2024-04-20T11:33:35.747 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/fuse/connections 2024-04-20T11:33:35.747 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/config 2024-04-20T11:33:35.747 INFO:teuthology.orchestra.run.smithi152.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T11:33:35.747 INFO:teuthology.orchestra.run.smithi152.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T11:33:35.747 INFO:teuthology.orchestra.run.smithi152.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T11:33:35.747 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T11:33:35.747 INFO:teuthology.orchestra.run.smithi152.stdout:/run/user/1000 2024-04-20T11:33:35.747 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph 2024-04-20T11:33:35.747 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph/osd/ceph-0 2024-04-20T11:33:35.747 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph/osd/ceph-1 2024-04-20T11:33:35.748 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph/osd/ceph-2 2024-04-20T11:33:35.748 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph/osd/ceph-3 2024-04-20T11:33:35.748 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns 2024-04-20T11:33:35.748 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T11:33:35.748 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T11:33:35.748 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T11:33:35.748 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T11:33:35.749 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:35.749 DEBUG:teuthology.orchestra.run.smithi152:> ls /sys/fs/fuse/connections 2024-04-20T11:33:35.800 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-20T11:33:35.800 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-20T11:33:35.804 DEBUG:teuthology.orchestra.run.smithi152:> sudo modprobe fuse 2024-04-20T11:33:35.876 DEBUG:teuthology.orchestra.run.smithi152:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T11:33:35.917 INFO:teuthology.orchestra.run.smithi152.stdout:/proc 2024-04-20T11:33:35.917 INFO:teuthology.orchestra.run.smithi152.stdout:/sys 2024-04-20T11:33:35.917 INFO:teuthology.orchestra.run.smithi152.stdout:/dev 2024-04-20T11:33:35.917 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/security 2024-04-20T11:33:35.917 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/shm 2024-04-20T11:33:35.917 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/pts 2024-04-20T11:33:35.917 INFO:teuthology.orchestra.run.smithi152.stdout:/run 2024-04-20T11:33:35.918 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/cgroup 2024-04-20T11:33:35.918 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/pstore 2024-04-20T11:33:35.918 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/bpf 2024-04-20T11:33:35.918 INFO:teuthology.orchestra.run.smithi152.stdout:/ 2024-04-20T11:33:35.918 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/selinux 2024-04-20T11:33:35.918 INFO:teuthology.orchestra.run.smithi152.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T11:33:35.918 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/mqueue 2024-04-20T11:33:35.918 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/hugepages 2024-04-20T11:33:35.918 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/debug 2024-04-20T11:33:35.918 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/tracing 2024-04-20T11:33:35.918 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/fuse/connections 2024-04-20T11:33:35.918 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/config 2024-04-20T11:33:35.918 INFO:teuthology.orchestra.run.smithi152.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T11:33:35.918 INFO:teuthology.orchestra.run.smithi152.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T11:33:35.919 INFO:teuthology.orchestra.run.smithi152.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T11:33:35.919 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T11:33:35.919 INFO:teuthology.orchestra.run.smithi152.stdout:/run/user/1000 2024-04-20T11:33:35.919 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph 2024-04-20T11:33:35.919 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph/osd/ceph-0 2024-04-20T11:33:35.919 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph/osd/ceph-1 2024-04-20T11:33:35.919 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph/osd/ceph-2 2024-04-20T11:33:35.919 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph/osd/ceph-3 2024-04-20T11:33:35.919 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns 2024-04-20T11:33:35.919 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T11:33:35.919 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T11:33:35.919 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T11:33:35.919 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T11:33:35.920 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:35.920 DEBUG:teuthology.orchestra.run.smithi152:> ls /sys/fs/fuse/connections 2024-04-20T11:33:35.976 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi152.stderr:2024-04-20T11:33:35.976+0000 7f5bb5c37480 -1 init, newargv = 0x55fcc3dea510 newargc=15 2024-04-20T11:33:35.977 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi152.stderr:ceph-fuse[54476]: starting ceph client 2024-04-20T11:33:35.984 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi152.stderr:ceph-fuse[54476]: starting fuse 2024-04-20T11:33:36.005 INFO:teuthology.orchestra.run.smithi152.stdout:42 2024-04-20T11:33:36.005 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-04-20T11:33:36.005 DEBUG:teuthology.orchestra.run.smithi152:> sudo stdin-killer -- python3 -c ' 2024-04-20T11:33:36.005 DEBUG:teuthology.orchestra.run.smithi152:> import glob 2024-04-20T11:33:36.005 DEBUG:teuthology.orchestra.run.smithi152:> import re 2024-04-20T11:33:36.005 DEBUG:teuthology.orchestra.run.smithi152:> import os 2024-04-20T11:33:36.005 DEBUG:teuthology.orchestra.run.smithi152:> import subprocess 2024-04-20T11:33:36.005 DEBUG:teuthology.orchestra.run.smithi152:> 2024-04-20T11:33:36.005 DEBUG:teuthology.orchestra.run.smithi152:> def _find_admin_socket(client_name): 2024-04-20T11:33:36.006 DEBUG:teuthology.orchestra.run.smithi152:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-20T11:33:36.006 DEBUG:teuthology.orchestra.run.smithi152:> files = glob.glob(asok_path) 2024-04-20T11:33:36.006 DEBUG:teuthology.orchestra.run.smithi152:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-20T11:33:36.006 DEBUG:teuthology.orchestra.run.smithi152:> 2024-04-20T11:33:36.006 DEBUG:teuthology.orchestra.run.smithi152:> # Given a non-glob path, it better be there 2024-04-20T11:33:36.006 DEBUG:teuthology.orchestra.run.smithi152:> if "*" not in asok_path: 2024-04-20T11:33:36.006 DEBUG:teuthology.orchestra.run.smithi152:> assert(len(files) == 1) 2024-04-20T11:33:36.006 DEBUG:teuthology.orchestra.run.smithi152:> return files[0] 2024-04-20T11:33:36.006 DEBUG:teuthology.orchestra.run.smithi152:> 2024-04-20T11:33:36.006 DEBUG:teuthology.orchestra.run.smithi152:> for f in files: 2024-04-20T11:33:36.006 DEBUG:teuthology.orchestra.run.smithi152:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-20T11:33:36.006 DEBUG:teuthology.orchestra.run.smithi152:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-20T11:33:36.006 DEBUG:teuthology.orchestra.run.smithi152:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-20T11:33:36.006 DEBUG:teuthology.orchestra.run.smithi152:> contents = proc_f.read() 2024-04-20T11:33:36.006 DEBUG:teuthology.orchestra.run.smithi152:> if mountpoint in contents: 2024-04-20T11:33:36.007 DEBUG:teuthology.orchestra.run.smithi152:> return f 2024-04-20T11:33:36.007 DEBUG:teuthology.orchestra.run.smithi152:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-20T11:33:36.007 DEBUG:teuthology.orchestra.run.smithi152:> 2024-04-20T11:33:36.007 DEBUG:teuthology.orchestra.run.smithi152:> print(_find_admin_socket("client.0")) 2024-04-20T11:33:36.007 DEBUG:teuthology.orchestra.run.smithi152:> ' 2024-04-20T11:33:36.202 INFO:teuthology.orchestra.run.smithi152.stdout:/var/run/ceph/ceph-client.0.54476.asok 2024-04-20T11:33:36.207 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T11:33:36.222 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.54476.asok 2024-04-20T11:33:36.222 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:36.223 DEBUG:teuthology.orchestra.run.smithi152:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.54476.asok status 2024-04-20T11:33:36.451 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-04-20T11:33:36.451 INFO:teuthology.orchestra.run.smithi152.stdout: "metadata": { 2024-04-20T11:33:36.451 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_sha1": "a8c84f5997950fd7d008204839253291d52e6eda", 2024-04-20T11:33:36.451 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", 2024-04-20T11:33:36.451 INFO:teuthology.orchestra.run.smithi152.stdout: "entity_id": "0", 2024-04-20T11:33:36.451 INFO:teuthology.orchestra.run.smithi152.stdout: "hostname": "smithi152", 2024-04-20T11:33:36.451 INFO:teuthology.orchestra.run.smithi152.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-20T11:33:36.451 INFO:teuthology.orchestra.run.smithi152.stdout: "pid": "54476", 2024-04-20T11:33:36.451 INFO:teuthology.orchestra.run.smithi152.stdout: "root": "/" 2024-04-20T11:33:36.452 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:33:36.452 INFO:teuthology.orchestra.run.smithi152.stdout: "dentry_count": 0, 2024-04-20T11:33:36.452 INFO:teuthology.orchestra.run.smithi152.stdout: "dentry_pinned_count": 0, 2024-04-20T11:33:36.452 INFO:teuthology.orchestra.run.smithi152.stdout: "id": 7962, 2024-04-20T11:33:36.452 INFO:teuthology.orchestra.run.smithi152.stdout: "inst": { 2024-04-20T11:33:36.452 INFO:teuthology.orchestra.run.smithi152.stdout: "name": { 2024-04-20T11:33:36.452 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "client", 2024-04-20T11:33:36.452 INFO:teuthology.orchestra.run.smithi152.stdout: "num": 7962 2024-04-20T11:33:36.452 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:33:36.452 INFO:teuthology.orchestra.run.smithi152.stdout: "addr": { 2024-04-20T11:33:36.452 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "v1", 2024-04-20T11:33:36.452 INFO:teuthology.orchestra.run.smithi152.stdout: "addr": "192.168.0.1:0", 2024-04-20T11:33:36.452 INFO:teuthology.orchestra.run.smithi152.stdout: "nonce": 854624273 2024-04-20T11:33:36.453 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:33:36.453 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:33:36.453 INFO:teuthology.orchestra.run.smithi152.stdout: "addr": { 2024-04-20T11:33:36.453 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "v1", 2024-04-20T11:33:36.453 INFO:teuthology.orchestra.run.smithi152.stdout: "addr": "192.168.0.1:0", 2024-04-20T11:33:36.453 INFO:teuthology.orchestra.run.smithi152.stdout: "nonce": 854624273 2024-04-20T11:33:36.453 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:33:36.453 INFO:teuthology.orchestra.run.smithi152.stdout: "inst_str": "client.7962 192.168.0.1:0/854624273", 2024-04-20T11:33:36.453 INFO:teuthology.orchestra.run.smithi152.stdout: "addr_str": "192.168.0.1:0/854624273", 2024-04-20T11:33:36.453 INFO:teuthology.orchestra.run.smithi152.stdout: "inode_count": 1, 2024-04-20T11:33:36.453 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_epoch": 66, 2024-04-20T11:33:36.453 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_epoch": 103, 2024-04-20T11:33:36.453 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_epoch_barrier": 0, 2024-04-20T11:33:36.454 INFO:teuthology.orchestra.run.smithi152.stdout: "blocklisted": false, 2024-04-20T11:33:36.454 INFO:teuthology.orchestra.run.smithi152.stdout: "fs_name": "cephfs" 2024-04-20T11:33:36.454 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-04-20T11:33:36.465 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:36.466 DEBUG:teuthology.orchestra.run.smithi152:> stat --file-system '--printf=%T 2024-04-20T11:33:36.466 DEBUG:teuthology.orchestra.run.smithi152:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-20T11:33:36.518 INFO:teuthology.orchestra.run.smithi152.stdout:fuseblk 2024-04-20T11:33:36.519 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-20T11:33:36.519 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:36.519 DEBUG:teuthology.orchestra.run.smithi152:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-20T11:33:36.602 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-20T11:33:36.809 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:36.809+0000 7f27a5c00640 1 Processor -- start 2024-04-20T11:33:36.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:36.809+0000 7f27a5c00640 1 -- start start 2024-04-20T11:33:36.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:36.810+0000 7f27a5c00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f27a0145f60 0x7f27a006ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:36.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:36.810+0000 7f27a5c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f27a00726f0 0x7f27a00cd580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:36.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:36.811+0000 7f27a5c00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27a0146340 con 0x7f27a00cd0d0 2024-04-20T11:33:36.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:36.811+0000 7f27a5c00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f27a006e9f0 con 0x7f27a0145f60 2024-04-20T11:33:36.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:36.811+0000 7f27a5c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f27a006f3c0 con 0x7f27a00726f0 2024-04-20T11:33:36.812 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:36.811+0000 7f2797e00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f27a00cd0d0 0x7f27a0071fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:58046/0 (socket says 172.21.15.152:58046) 2024-04-20T11:33:36.812 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:36.811+0000 7f2797e00640 1 -- 172.21.15.152:0/4052891005 learned_addr learned my addr 172.21.15.152:0/4052891005 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:33:36.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:36.811+0000 7f279f400640 1 --2- 172.21.15.152:0/4052891005 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f27a00726f0 0x7f27a00cd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:36.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:36.811+0000 7f279ea00640 1 --2- 172.21.15.152:0/4052891005 >> v2:172.21.15.175:3300/0 conn(0x7f27a0145f60 0x7f27a006ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:36.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:36.811+0000 7f279e000640 1 -- 172.21.15.152:0/4052891005 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3244207477 0 0) 0x7f27a0146340 con 0x7f27a00cd0d0 2024-04-20T11:33:36.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:36.812+0000 7f279e000640 1 -- 172.21.15.152:0/4052891005 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2784003520 con 0x7f27a00cd0d0 2024-04-20T11:33:36.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:36.812+0000 7f279e000640 1 -- 172.21.15.152:0/4052891005 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3660534698 0 0) 0x7f2784003520 con 0x7f27a00cd0d0 2024-04-20T11:33:36.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:36.812+0000 7f279e000640 1 -- 172.21.15.152:0/4052891005 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f27a00726f0 msgr2=0x7f27a00cd580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:36.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:36.812+0000 7f279e000640 1 --2- 172.21.15.152:0/4052891005 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f27a00726f0 0x7f27a00cd580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:36.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:36.812+0000 7f279e000640 1 -- 172.21.15.152:0/4052891005 >> v2:172.21.15.175:3300/0 conn(0x7f27a0145f60 msgr2=0x7f27a006ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:36.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:36.812+0000 7f279e000640 1 --2- 172.21.15.152:0/4052891005 >> v2:172.21.15.175:3300/0 conn(0x7f27a0145f60 0x7f27a006ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:36.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:36.812+0000 7f279e000640 1 -- 172.21.15.152:0/4052891005 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27a0072d50 con 0x7f27a00cd0d0 2024-04-20T11:33:36.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:36.812+0000 7f279e000640 1 -- 172.21.15.152:0/4052891005 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f2788001690 con 0x7f27a00cd0d0 2024-04-20T11:33:36.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:36.812+0000 7f279e000640 1 -- 172.21.15.152:0/4052891005 >> v1:172.21.15.152:6789/0 conn(0x7f27a00cd0d0 legacy=0x7f27a0071fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:36.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:36.812+0000 7f279e000640 1 --2- 172.21.15.152:0/4052891005 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2784003a20 0x7f2784003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:36.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:36.812+0000 7f279e000640 1 --2- 172.21.15.152:0/4052891005 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f27a00726f0 0x7f2784004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:36.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:36.812+0000 7f279e000640 1 --2- 172.21.15.152:0/4052891005 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2784004d40 0x7f27840051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:36.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:36.812+0000 7f279e000640 1 -- 172.21.15.152:0/4052891005 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f27a006e9f0 con 0x7f2784004d40 2024-04-20T11:33:36.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:36.812+0000 7f279e000640 1 -- 172.21.15.152:0/4052891005 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f27a006f3c0 con 0x7f2784003a20 2024-04-20T11:33:36.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:36.812+0000 7f279e000640 1 -- 172.21.15.152:0/4052891005 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f278400b9c0 con 0x7f27a00726f0 2024-04-20T11:33:36.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:36.812+0000 7f279f400640 1 --2- 172.21.15.152:0/4052891005 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2784004d40 0x7f27840051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:36.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:36.813+0000 7f279ea00640 1 --2- 172.21.15.152:0/4052891005 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2784003a20 0x7f2784003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:36.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:36.813+0000 7f2797e00640 1 --2- 172.21.15.152:0/4052891005 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f27a00726f0 0x7f2784004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:36.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:36.813+0000 7f279f400640 1 -- 172.21.15.152:0/4052891005 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f27a00726f0 msgr2=0x7f2784004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:36.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:36.813+0000 7f279f400640 1 --2- 172.21.15.152:0/4052891005 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f27a00726f0 0x7f2784004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:36.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:36.813+0000 7f279f400640 1 -- 172.21.15.152:0/4052891005 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2784003a20 msgr2=0x7f2784003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:36.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:36.813+0000 7f279f400640 1 --2- 172.21.15.152:0/4052891005 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2784003a20 0x7f2784003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:36.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:36.813+0000 7f279f400640 1 -- 172.21.15.152:0/4052891005 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2784003520 con 0x7f2784004d40 2024-04-20T11:33:36.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:36.813+0000 7f279ea00640 1 --2- 172.21.15.152:0/4052891005 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2784003a20 0x7f2784003e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:33:36.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:36.814+0000 7f279f400640 1 --2- 172.21.15.152:0/4052891005 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2784004d40 0x7f27840051c0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f2798009fc0 tx=0x7f2784005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86cbfe8eda3cd680 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:36.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:36.814+0000 7f279e000640 1 -- 172.21.15.152:0/4052891005 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2798039070 con 0x7f2784004d40 2024-04-20T11:33:36.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:36.814+0000 7f279e000640 1 -- 172.21.15.152:0/4052891005 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f279802cc20 con 0x7f2784004d40 2024-04-20T11:33:36.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:36.814+0000 7f279e000640 1 -- 172.21.15.152:0/4052891005 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2798009c10 con 0x7f2784004d40 2024-04-20T11:33:36.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:36.815+0000 7f27a5c00640 1 -- 172.21.15.152:0/4052891005 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2784004d40 msgr2=0x7f27840051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:36.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:36.815+0000 7f27a5c00640 1 --2- 172.21.15.152:0/4052891005 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2784004d40 0x7f27840051c0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f2798009fc0 tx=0x7f2784005830 comp rx=0 tx=0).stop 2024-04-20T11:33:36.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:36.815+0000 7f279f400640 1 -- 172.21.15.152:0/4052891005 reap_dead start 2024-04-20T11:33:36.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:36.815+0000 7f27a5c00640 1 -- 172.21.15.152:0/4052891005 shutdown_connections 2024-04-20T11:33:36.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:36.815+0000 7f27a5c00640 1 -- 172.21.15.152:0/4052891005 >> 172.21.15.152:0/4052891005 conn(0x7f27a0067cf0 msgr2=0x7f27a006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:36.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:36.815+0000 7f27a5c00640 1 -- 172.21.15.152:0/4052891005 shutdown_connections 2024-04-20T11:33:36.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:36.815+0000 7f27a5c00640 1 -- 172.21.15.152:0/4052891005 wait complete. 2024-04-20T11:33:36.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:36.817+0000 7f27a5c00640 1 Processor -- start 2024-04-20T11:33:36.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:36.817+0000 7f27a5c00640 1 -- start start 2024-04-20T11:33:36.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:36.818+0000 7f27a5c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2784004d40 0x7f27a013ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:36.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:36.818+0000 7f27a5c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f27a00af050 0x7f27a00af4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:36.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:36.818+0000 7f27a5c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f27a00aa470 0x7f27a00aa8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:36.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:36.818+0000 7f27a5c00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f27a006e680 con 0x7f27a00aa470 2024-04-20T11:33:36.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:36.818+0000 7f27a5c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f27a006e3a0 con 0x7f2784004d40 2024-04-20T11:33:36.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:36.818+0000 7f27a5c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f27a00cd580 con 0x7f27a00af050 2024-04-20T11:33:36.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:36.818+0000 7f279f400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f27a00aa470 0x7f27a00aa8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:36.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:36.818+0000 7f279f400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f27a00aa470 0x7f27a00aa8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:55438/0 (socket says 172.21.15.152:55438) 2024-04-20T11:33:36.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:36.818+0000 7f279f400640 1 -- 172.21.15.152:0/2979764678 learned_addr learned my addr 172.21.15.152:0/2979764678 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:33:36.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:36.818+0000 7f279ea00640 1 --2- 172.21.15.152:0/2979764678 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2784004d40 0x7f27a013ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:36.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:36.819+0000 7f2797e00640 1 --2- 172.21.15.152:0/2979764678 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f27a00af050 0x7f27a00af4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:36.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:36.819+0000 7f279f400640 1 -- 172.21.15.152:0/2979764678 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f27a00af050 msgr2=0x7f27a00af4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:36.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:36.819+0000 7f279f400640 1 --2- 172.21.15.152:0/2979764678 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f27a00af050 0x7f27a00af4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:36.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:36.819+0000 7f279f400640 1 -- 172.21.15.152:0/2979764678 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2784004d40 msgr2=0x7f27a013ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:36.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:36.819+0000 7f279f400640 1 --2- 172.21.15.152:0/2979764678 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2784004d40 0x7f27a013ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:36.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:36.819+0000 7f279f400640 1 -- 172.21.15.152:0/2979764678 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27a00ab1a0 con 0x7f27a00aa470 2024-04-20T11:33:36.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:36.819+0000 7f279f400640 1 --2- 172.21.15.152:0/2979764678 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f27a00aa470 0x7f27a00aa8f0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f279802c690 tx=0x7f2798009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:36.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:36.819+0000 7f2796a00640 1 -- 172.21.15.152:0/2979764678 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2798042020 con 0x7f27a00aa470 2024-04-20T11:33:36.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:36.819+0000 7f2796a00640 1 -- 172.21.15.152:0/2979764678 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f279803a040 con 0x7f27a00aa470 2024-04-20T11:33:36.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:36.819+0000 7f2796a00640 1 -- 172.21.15.152:0/2979764678 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2798039070 con 0x7f27a00aa470 2024-04-20T11:33:36.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:36.819+0000 7f27a5c00640 1 -- 172.21.15.152:0/2979764678 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27a00ac940 con 0x7f27a00aa470 2024-04-20T11:33:36.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:36.820+0000 7f27a5c00640 1 -- 172.21.15.152:0/2979764678 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27a013b140 con 0x7f27a00aa470 2024-04-20T11:33:36.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:36.821+0000 7f27a5c00640 1 -- 172.21.15.152:0/2979764678 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27a00689f0 con 0x7f27a00aa470 2024-04-20T11:33:36.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:36.822+0000 7f2796a00640 1 -- 172.21.15.152:0/2979764678 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 91142+0+0 (secure 0 0 0) 0x7f2798045020 con 0x7f27a00aa470 2024-04-20T11:33:36.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:36.823+0000 7f2796a00640 1 --2- 172.21.15.152:0/2979764678 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f277c06ca10 0x7f277c06eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:36.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:36.823+0000 7f2796a00640 1 -- 172.21.15.152:0/2979764678 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7f27980addb0 con 0x7f27a00aa470 2024-04-20T11:33:36.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:36.823+0000 7f279ea00640 1 --2- 172.21.15.152:0/2979764678 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f277c06ca10 0x7f277c06eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:36.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:36.824+0000 7f279ea00640 1 --2- 172.21.15.152:0/2979764678 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f277c06ca10 0x7f277c06eed0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2790009a90 tx=0x7f2790008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:36.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:36.827+0000 7f2796a00640 1 -- 172.21.15.152:0/2979764678 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f27980b2070 con 0x7f27a00aa470 2024-04-20T11:33:37.075 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:37.074+0000 7f27a5c00640 1 -- 172.21.15.152:0/2979764678 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f27a006bb20 con 0x7f27a00aa470 2024-04-20T11:33:37.075 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:37.074+0000 7f2796a00640 1 -- 172.21.15.152:0/2979764678 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v66) ==== 53+0+73 (secure 0 0 0) 0x7f279807c0c0 con 0x7f27a00aa470 2024-04-20T11:33:37.075 INFO:teuthology.orchestra.run.smithi152.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-20T11:33:37.078 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:37.077+0000 7f27a5c00640 1 -- 172.21.15.152:0/2979764678 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f277c06ca10 msgr2=0x7f277c06eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:37.078 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:37.077+0000 7f27a5c00640 1 --2- 172.21.15.152:0/2979764678 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f277c06ca10 0x7f277c06eed0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2790009a90 tx=0x7f2790008040 comp rx=0 tx=0).stop 2024-04-20T11:33:37.078 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:37.077+0000 7f27a5c00640 1 -- 172.21.15.152:0/2979764678 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f27a00aa470 msgr2=0x7f27a00aa8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:37.078 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:37.077+0000 7f27a5c00640 1 --2- 172.21.15.152:0/2979764678 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f27a00aa470 0x7f27a00aa8f0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f279802c690 tx=0x7f2798009750 comp rx=0 tx=0).stop 2024-04-20T11:33:37.078 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:37.077+0000 7f27a5c00640 1 -- 172.21.15.152:0/2979764678 shutdown_connections 2024-04-20T11:33:37.078 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:37.077+0000 7f27a5c00640 1 --2- 172.21.15.152:0/2979764678 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f277c06ca10 0x7f277c06eed0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:37.078 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:37.077+0000 7f27a5c00640 1 --2- 172.21.15.152:0/2979764678 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f27a00aa470 0x7f27a00aa8f0 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:37.079 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:37.077+0000 7f27a5c00640 1 --2- 172.21.15.152:0/2979764678 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f27a00af050 0x7f27a00af4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:37.079 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:37.077+0000 7f27a5c00640 1 --2- 172.21.15.152:0/2979764678 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2784004d40 0x7f27a013ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:37.079 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:37.077+0000 7f27a5c00640 1 -- 172.21.15.152:0/2979764678 >> 172.21.15.152:0/2979764678 conn(0x7f27a0067cf0 msgr2=0x7f27a015a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:37.079 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:37.077+0000 7f27a5c00640 1 -- 172.21.15.152:0/2979764678 shutdown_connections 2024-04-20T11:33:37.079 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:37.077+0000 7f27a5c00640 1 -- 172.21.15.152:0/2979764678 wait complete. 2024-04-20T11:33:37.097 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-20T11:33:37.098 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-20T11:33:37.098 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi175.front.sepia.ceph.com 2024-04-20T11:33:37.098 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-20T11:33:37.098 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-20T11:33:37.098 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-20T11:33:37.098 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-20T11:33:37.098 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-20T11:33:37.098 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-20T11:33:37.098 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:37.099 DEBUG:teuthology.orchestra.run.smithi175:> ip addr 2024-04-20T11:33:37.157 INFO:teuthology.orchestra.run.smithi175.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-20T11:33:37.157 INFO:teuthology.orchestra.run.smithi175.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-20T11:33:37.157 INFO:teuthology.orchestra.run.smithi175.stdout: inet 127.0.0.1/8 scope host lo 2024-04-20T11:33:37.158 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:33:37.158 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 ::1/128 scope host 2024-04-20T11:33:37.158 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:33:37.158 INFO:teuthology.orchestra.run.smithi175.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T11:33:37.158 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:d9:39:ee brd ff:ff:ff:ff:ff:ff 2024-04-20T11:33:37.158 INFO:teuthology.orchestra.run.smithi175.stdout: altname enp7s0f0 2024-04-20T11:33:37.158 INFO:teuthology.orchestra.run.smithi175.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T11:33:37.158 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:d9:39:ef brd ff:ff:ff:ff:ff:ff 2024-04-20T11:33:37.158 INFO:teuthology.orchestra.run.smithi175.stdout: altname enp7s0f1 2024-04-20T11:33:37.158 INFO:teuthology.orchestra.run.smithi175.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T11:33:37.159 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:88:71:f0 brd ff:ff:ff:ff:ff:ff 2024-04-20T11:33:37.159 INFO:teuthology.orchestra.run.smithi175.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-20T11:33:37.159 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:88:71:f1 brd ff:ff:ff:ff:ff:ff 2024-04-20T11:33:37.159 INFO:teuthology.orchestra.run.smithi175.stdout: inet 172.21.15.175/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-20T11:33:37.159 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft 42364sec preferred_lft 42364sec 2024-04-20T11:33:37.159 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::ec4:7aff:fe88:71f1/64 scope link 2024-04-20T11:33:37.159 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:33:37.159 INFO:teuthology.orchestra.run.smithi175.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-20T11:33:37.159 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 02:d6:c6:31:26:70 brd ff:ff:ff:ff:ff:ff 2024-04-20T11:33:37.159 INFO:teuthology.orchestra.run.smithi175.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-20T11:33:37.159 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:33:37.159 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::5091:d5ff:fee7:54b3/64 scope link 2024-04-20T11:33:37.159 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:33:37.159 INFO:teuthology.orchestra.run.smithi175.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-20T11:33:37.160 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 02:d6:c6:31:26:70 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-20T11:33:37.160 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::d6:c6ff:fe31:2670/64 scope link 2024-04-20T11:33:37.160 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:33:37.160 INFO:teuthology.orchestra.run.smithi175.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-20T11:33:37.160 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 9e:14:f3:e2:18:1e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-20T11:33:37.160 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::9c14:f3ff:fee2:181e/64 scope link 2024-04-20T11:33:37.160 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:33:37.160 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:37.160 DEBUG:teuthology.orchestra.run.smithi175:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-20T11:33:37.239 INFO:teuthology.orchestra.run.smithi175.stdout:1 2024-04-20T11:33:37.243 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:37.244 DEBUG:teuthology.orchestra.run.smithi175:> route 2024-04-20T11:33:37.297 INFO:teuthology.orchestra.run.smithi175.stdout:Kernel IP routing table 2024-04-20T11:33:37.297 INFO:teuthology.orchestra.run.smithi175.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-20T11:33:37.297 INFO:teuthology.orchestra.run.smithi175.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-20T11:33:37.297 INFO:teuthology.orchestra.run.smithi175.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-20T11:33:37.297 INFO:teuthology.orchestra.run.smithi175.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-20T11:33:37.298 DEBUG:teuthology.orchestra.run.smithi175:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T11:33:37.298 DEBUG:teuthology.orchestra.run.smithi175:> set -e 2024-04-20T11:33:37.298 DEBUG:teuthology.orchestra.run.smithi175:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-20T11:33:37.298 DEBUG:teuthology.orchestra.run.smithi175:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-20T11:33:37.298 DEBUG:teuthology.orchestra.run.smithi175:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-20T11:33:37.298 DEBUG:teuthology.orchestra.run.smithi175:> ') 2024-04-20T11:33:37.400 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:33:37 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T11:33:37.492 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:33:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T11:33:37.501 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:37.501 DEBUG:teuthology.orchestra.run.smithi175:> ip netns list 2024-04-20T11:33:37.545 INFO:teuthology.orchestra.run.smithi175.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-20T11:33:37.545 INFO:teuthology.orchestra.run.smithi175.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-20T11:33:37.546 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-20T11:33:37.546 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:37.546 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip link set brx.0 up 2024-04-20T11:33:37.674 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-04-20T11:33:37.675 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T11:33:37.675 DEBUG:teuthology.orchestra.run.smithi175:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-20T11:33:37.730 INFO:teuthology.orchestra.run.smithi175.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-20T11:33:37.731 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T11:33:37.731 DEBUG:teuthology.orchestra.run.smithi175:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-20T11:33:37.779 DEBUG:teuthology.orchestra.run.smithi175:> sudo modprobe fuse 2024-04-20T11:33:37.858 DEBUG:teuthology.orchestra.run.smithi175:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T11:33:37.913 INFO:teuthology.orchestra.run.smithi175.stdout:/proc 2024-04-20T11:33:37.913 INFO:teuthology.orchestra.run.smithi175.stdout:/sys 2024-04-20T11:33:37.913 INFO:teuthology.orchestra.run.smithi175.stdout:/dev 2024-04-20T11:33:37.913 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/security 2024-04-20T11:33:37.913 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/shm 2024-04-20T11:33:37.913 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/pts 2024-04-20T11:33:37.914 INFO:teuthology.orchestra.run.smithi175.stdout:/run 2024-04-20T11:33:37.914 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/cgroup 2024-04-20T11:33:37.914 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/pstore 2024-04-20T11:33:37.914 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/bpf 2024-04-20T11:33:37.914 INFO:teuthology.orchestra.run.smithi175.stdout:/ 2024-04-20T11:33:37.914 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/selinux 2024-04-20T11:33:37.914 INFO:teuthology.orchestra.run.smithi175.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T11:33:37.914 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/mqueue 2024-04-20T11:33:37.914 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/debug 2024-04-20T11:33:37.914 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/hugepages 2024-04-20T11:33:37.914 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/tracing 2024-04-20T11:33:37.914 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/fuse/connections 2024-04-20T11:33:37.914 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/config 2024-04-20T11:33:37.915 INFO:teuthology.orchestra.run.smithi175.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T11:33:37.915 INFO:teuthology.orchestra.run.smithi175.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T11:33:37.915 INFO:teuthology.orchestra.run.smithi175.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T11:33:37.915 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T11:33:37.915 INFO:teuthology.orchestra.run.smithi175.stdout:/run/user/1000 2024-04-20T11:33:37.915 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph 2024-04-20T11:33:37.915 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-4 2024-04-20T11:33:37.915 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-5 2024-04-20T11:33:37.915 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-6 2024-04-20T11:33:37.915 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-7 2024-04-20T11:33:37.915 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns 2024-04-20T11:33:37.915 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-20T11:33:37.915 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-20T11:33:37.915 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-20T11:33:37.916 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-20T11:33:37.916 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:37.917 DEBUG:teuthology.orchestra.run.smithi175:> ls /sys/fs/fuse/connections 2024-04-20T11:33:37.968 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-20T11:33:37.968 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-04-20T11:33:37.973 DEBUG:teuthology.orchestra.run.smithi175:> sudo modprobe fuse 2024-04-20T11:33:38.040 DEBUG:teuthology.orchestra.run.smithi175:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T11:33:38.080 INFO:teuthology.orchestra.run.smithi175.stdout:/proc 2024-04-20T11:33:38.080 INFO:teuthology.orchestra.run.smithi175.stdout:/sys 2024-04-20T11:33:38.081 INFO:teuthology.orchestra.run.smithi175.stdout:/dev 2024-04-20T11:33:38.081 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/security 2024-04-20T11:33:38.081 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/shm 2024-04-20T11:33:38.081 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/pts 2024-04-20T11:33:38.081 INFO:teuthology.orchestra.run.smithi175.stdout:/run 2024-04-20T11:33:38.081 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/cgroup 2024-04-20T11:33:38.082 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/pstore 2024-04-20T11:33:38.082 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/bpf 2024-04-20T11:33:38.082 INFO:teuthology.orchestra.run.smithi175.stdout:/ 2024-04-20T11:33:38.082 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/selinux 2024-04-20T11:33:38.082 INFO:teuthology.orchestra.run.smithi175.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T11:33:38.082 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/mqueue 2024-04-20T11:33:38.082 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/debug 2024-04-20T11:33:38.083 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/hugepages 2024-04-20T11:33:38.083 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/tracing 2024-04-20T11:33:38.083 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/fuse/connections 2024-04-20T11:33:38.083 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/config 2024-04-20T11:33:38.083 INFO:teuthology.orchestra.run.smithi175.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T11:33:38.083 INFO:teuthology.orchestra.run.smithi175.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T11:33:38.083 INFO:teuthology.orchestra.run.smithi175.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T11:33:38.083 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T11:33:38.083 INFO:teuthology.orchestra.run.smithi175.stdout:/run/user/1000 2024-04-20T11:33:38.083 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph 2024-04-20T11:33:38.083 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-4 2024-04-20T11:33:38.083 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-5 2024-04-20T11:33:38.083 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-6 2024-04-20T11:33:38.084 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-7 2024-04-20T11:33:38.084 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns 2024-04-20T11:33:38.084 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-20T11:33:38.084 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-20T11:33:38.084 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-20T11:33:38.084 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-20T11:33:38.085 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:38.085 DEBUG:teuthology.orchestra.run.smithi175:> ls /sys/fs/fuse/connections 2024-04-20T11:33:38.141 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi175.stderr:2024-04-20T11:33:38.141+0000 7f08fac3a480 -1 init, newargv = 0x56371b5464d0 newargc=15 2024-04-20T11:33:38.142 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi175.stderr:ceph-fuse[33724]: starting ceph client 2024-04-20T11:33:38.154 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi175.stderr:ceph-fuse[33724]: starting fuse 2024-04-20T11:33:38.170 INFO:teuthology.orchestra.run.smithi175.stdout:42 2024-04-20T11:33:38.170 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-04-20T11:33:38.170 DEBUG:teuthology.orchestra.run.smithi175:> sudo stdin-killer -- python3 -c ' 2024-04-20T11:33:38.171 DEBUG:teuthology.orchestra.run.smithi175:> import glob 2024-04-20T11:33:38.171 DEBUG:teuthology.orchestra.run.smithi175:> import re 2024-04-20T11:33:38.171 DEBUG:teuthology.orchestra.run.smithi175:> import os 2024-04-20T11:33:38.171 DEBUG:teuthology.orchestra.run.smithi175:> import subprocess 2024-04-20T11:33:38.171 DEBUG:teuthology.orchestra.run.smithi175:> 2024-04-20T11:33:38.171 DEBUG:teuthology.orchestra.run.smithi175:> def _find_admin_socket(client_name): 2024-04-20T11:33:38.171 DEBUG:teuthology.orchestra.run.smithi175:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-04-20T11:33:38.171 DEBUG:teuthology.orchestra.run.smithi175:> files = glob.glob(asok_path) 2024-04-20T11:33:38.171 DEBUG:teuthology.orchestra.run.smithi175:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-04-20T11:33:38.171 DEBUG:teuthology.orchestra.run.smithi175:> 2024-04-20T11:33:38.171 DEBUG:teuthology.orchestra.run.smithi175:> # Given a non-glob path, it better be there 2024-04-20T11:33:38.171 DEBUG:teuthology.orchestra.run.smithi175:> if "*" not in asok_path: 2024-04-20T11:33:38.171 DEBUG:teuthology.orchestra.run.smithi175:> assert(len(files) == 1) 2024-04-20T11:33:38.171 DEBUG:teuthology.orchestra.run.smithi175:> return files[0] 2024-04-20T11:33:38.171 DEBUG:teuthology.orchestra.run.smithi175:> 2024-04-20T11:33:38.172 DEBUG:teuthology.orchestra.run.smithi175:> for f in files: 2024-04-20T11:33:38.172 DEBUG:teuthology.orchestra.run.smithi175:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-20T11:33:38.172 DEBUG:teuthology.orchestra.run.smithi175:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-20T11:33:38.172 DEBUG:teuthology.orchestra.run.smithi175:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-20T11:33:38.172 DEBUG:teuthology.orchestra.run.smithi175:> contents = proc_f.read() 2024-04-20T11:33:38.172 DEBUG:teuthology.orchestra.run.smithi175:> if mountpoint in contents: 2024-04-20T11:33:38.172 DEBUG:teuthology.orchestra.run.smithi175:> return f 2024-04-20T11:33:38.172 DEBUG:teuthology.orchestra.run.smithi175:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-20T11:33:38.172 DEBUG:teuthology.orchestra.run.smithi175:> 2024-04-20T11:33:38.172 DEBUG:teuthology.orchestra.run.smithi175:> print(_find_admin_socket("client.1")) 2024-04-20T11:33:38.172 DEBUG:teuthology.orchestra.run.smithi175:> ' 2024-04-20T11:33:38.365 INFO:teuthology.orchestra.run.smithi175.stdout:/var/run/ceph/ceph-client.1.33724.asok 2024-04-20T11:33:38.370 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:33:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T11:33:38.381 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.33724.asok 2024-04-20T11:33:38.382 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:38.382 DEBUG:teuthology.orchestra.run.smithi175:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.33724.asok status 2024-04-20T11:33:38.608 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-04-20T11:33:38.608 INFO:teuthology.orchestra.run.smithi175.stdout: "metadata": { 2024-04-20T11:33:38.608 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_sha1": "a8c84f5997950fd7d008204839253291d52e6eda", 2024-04-20T11:33:38.608 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", 2024-04-20T11:33:38.608 INFO:teuthology.orchestra.run.smithi175.stdout: "entity_id": "1", 2024-04-20T11:33:38.608 INFO:teuthology.orchestra.run.smithi175.stdout: "hostname": "smithi175", 2024-04-20T11:33:38.608 INFO:teuthology.orchestra.run.smithi175.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-04-20T11:33:38.609 INFO:teuthology.orchestra.run.smithi175.stdout: "pid": "33724", 2024-04-20T11:33:38.609 INFO:teuthology.orchestra.run.smithi175.stdout: "root": "/" 2024-04-20T11:33:38.609 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-20T11:33:38.609 INFO:teuthology.orchestra.run.smithi175.stdout: "dentry_count": 0, 2024-04-20T11:33:38.609 INFO:teuthology.orchestra.run.smithi175.stdout: "dentry_pinned_count": 0, 2024-04-20T11:33:38.609 INFO:teuthology.orchestra.run.smithi175.stdout: "id": 5717, 2024-04-20T11:33:38.609 INFO:teuthology.orchestra.run.smithi175.stdout: "inst": { 2024-04-20T11:33:38.609 INFO:teuthology.orchestra.run.smithi175.stdout: "name": { 2024-04-20T11:33:38.609 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "client", 2024-04-20T11:33:38.609 INFO:teuthology.orchestra.run.smithi175.stdout: "num": 5717 2024-04-20T11:33:38.609 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-20T11:33:38.609 INFO:teuthology.orchestra.run.smithi175.stdout: "addr": { 2024-04-20T11:33:38.610 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "v1", 2024-04-20T11:33:38.610 INFO:teuthology.orchestra.run.smithi175.stdout: "addr": "192.168.0.1:0", 2024-04-20T11:33:38.610 INFO:teuthology.orchestra.run.smithi175.stdout: "nonce": 166488912 2024-04-20T11:33:38.610 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-04-20T11:33:38.610 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-20T11:33:38.610 INFO:teuthology.orchestra.run.smithi175.stdout: "addr": { 2024-04-20T11:33:38.610 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "v1", 2024-04-20T11:33:38.610 INFO:teuthology.orchestra.run.smithi175.stdout: "addr": "192.168.0.1:0", 2024-04-20T11:33:38.610 INFO:teuthology.orchestra.run.smithi175.stdout: "nonce": 166488912 2024-04-20T11:33:38.610 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-20T11:33:38.610 INFO:teuthology.orchestra.run.smithi175.stdout: "inst_str": "client.5717 192.168.0.1:0/166488912", 2024-04-20T11:33:38.610 INFO:teuthology.orchestra.run.smithi175.stdout: "addr_str": "192.168.0.1:0/166488912", 2024-04-20T11:33:38.610 INFO:teuthology.orchestra.run.smithi175.stdout: "inode_count": 1, 2024-04-20T11:33:38.610 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_epoch": 66, 2024-04-20T11:33:38.611 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_epoch": 103, 2024-04-20T11:33:38.611 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_epoch_barrier": 0, 2024-04-20T11:33:38.611 INFO:teuthology.orchestra.run.smithi175.stdout: "blocklisted": false, 2024-04-20T11:33:38.611 INFO:teuthology.orchestra.run.smithi175.stdout: "fs_name": "cephfs" 2024-04-20T11:33:38.611 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-04-20T11:33:38.623 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:38.623 DEBUG:teuthology.orchestra.run.smithi175:> stat --file-system '--printf=%T 2024-04-20T11:33:38.623 DEBUG:teuthology.orchestra.run.smithi175:> ' -- /home/ubuntu/cephtest/mnt.1 2024-04-20T11:33:38.678 INFO:teuthology.orchestra.run.smithi175.stdout:fuseblk 2024-04-20T11:33:38.678 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-04-20T11:33:38.679 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:38.679 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-20T11:33:38.766 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-04-20T11:33:38.982 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:38.981+0000 7f5bd6400640 1 Processor -- start 2024-04-20T11:33:38.982 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:38.981+0000 7f5bd6400640 1 -- start start 2024-04-20T11:33:38.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:38.983+0000 7f5bd6400640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f5bd0073790 0x7f5bd00ccbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:38.984 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:38.983+0000 7f5bd6400640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5bd0073c70 con 0x7f5bd00cd0f0 2024-04-20T11:33:38.984 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:38.983+0000 7f5bd6400640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5bd0071d10 con 0x7f5bd015ae50 2024-04-20T11:33:38.984 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:38.983+0000 7f5bd6400640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f5bd015b230 con 0x7f5bd0073790 2024-04-20T11:33:38.984 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:38.983+0000 7f5bd5a00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f5bd00cd0f0 0x7f5bd00b3430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:58058/0 (socket says 172.21.15.152:58058) 2024-04-20T11:33:38.984 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:38.983+0000 7f5bd5a00640 1 -- 172.21.15.152:0/1463129294 learned_addr learned my addr 172.21.15.152:0/1463129294 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:33:38.986 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:38.983+0000 7f5bcf400640 1 -- 172.21.15.152:0/1463129294 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 297006794 0 0) 0x7f5bd0073c70 con 0x7f5bd00cd0f0 2024-04-20T11:33:38.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:38.983+0000 7f5bcf400640 1 -- 172.21.15.152:0/1463129294 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5bb8003520 con 0x7f5bd00cd0f0 2024-04-20T11:33:38.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:38.983+0000 7f5bcfe00640 1 --2- 172.21.15.152:0/1463129294 >> v2:172.21.15.175:3300/0 conn(0x7f5bd0073790 0x7f5bd00ccbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:38.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:38.984+0000 7f5bcf400640 1 -- 172.21.15.152:0/1463129294 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1693074472 0 0) 0x7f5bb8003520 con 0x7f5bd00cd0f0 2024-04-20T11:33:38.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:38.984+0000 7f5bcf400640 1 -- 172.21.15.152:0/1463129294 >> v2:172.21.15.175:3300/0 conn(0x7f5bd0073790 msgr2=0x7f5bd00ccbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:38.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:38.984+0000 7f5bcf400640 1 --2- 172.21.15.152:0/1463129294 >> v2:172.21.15.175:3300/0 conn(0x7f5bd0073790 0x7f5bd00ccbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:38.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:38.984+0000 7f5bcf400640 1 -- 172.21.15.152:0/1463129294 >> v1:172.21.15.175:6789/0 conn(0x7f5bd015ae50 legacy=0x7f5bd015f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:38.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:38.984+0000 7f5bcf400640 1 -- 172.21.15.152:0/1463129294 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bd0071ff0 con 0x7f5bd00cd0f0 2024-04-20T11:33:38.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:38.984+0000 7f5bcf400640 1 -- 172.21.15.152:0/1463129294 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f5bc80019f0 con 0x7f5bd00cd0f0 2024-04-20T11:33:38.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:38.984+0000 7f5bcf400640 1 -- 172.21.15.152:0/1463129294 >> v1:172.21.15.152:6789/0 conn(0x7f5bd00cd0f0 legacy=0x7f5bd00b3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:38.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:38.984+0000 7f5bcf400640 1 --2- 172.21.15.152:0/1463129294 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5bb80039b0 0x7f5bb8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:38.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:38.984+0000 7f5bcf400640 1 --2- 172.21.15.152:0/1463129294 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5bb8004800 0x7f5bb8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:38.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:38.984+0000 7f5bcf400640 1 --2- 172.21.15.152:0/1463129294 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5bb80052e0 0x7f5bb800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:38.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:38.984+0000 7f5bcf400640 1 -- 172.21.15.152:0/1463129294 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f5bd015b230 con 0x7f5bb8004800 2024-04-20T11:33:38.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:38.984+0000 7f5bcf400640 1 -- 172.21.15.152:0/1463129294 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb800c1b0 con 0x7f5bb80052e0 2024-04-20T11:33:38.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:38.984+0000 7f5bcf400640 1 -- 172.21.15.152:0/1463129294 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5bb800c3b0 con 0x7f5bb80039b0 2024-04-20T11:33:38.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:38.984+0000 7f5bcfe00640 1 --2- 172.21.15.152:0/1463129294 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5bb8004800 0x7f5bb8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:38.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:38.985+0000 7f5bd5a00640 1 --2- 172.21.15.152:0/1463129294 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5bb80052e0 0x7f5bb800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:38.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:38.985+0000 7f5bcfe00640 1 -- 172.21.15.152:0/1463129294 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5bb80039b0 msgr2=0x7f5bb8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:38.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:38.985+0000 7f5bd5000640 1 --2- 172.21.15.152:0/1463129294 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5bb80039b0 0x7f5bb8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:38.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:38.985+0000 7f5bcfe00640 1 --2- 172.21.15.152:0/1463129294 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5bb80039b0 0x7f5bb8003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:38.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:38.985+0000 7f5bcfe00640 1 -- 172.21.15.152:0/1463129294 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5bb80052e0 msgr2=0x7f5bb800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:38.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:38.985+0000 7f5bcfe00640 1 --2- 172.21.15.152:0/1463129294 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5bb80052e0 0x7f5bb800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:38.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:38.985+0000 7f5bcfe00640 1 -- 172.21.15.152:0/1463129294 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bb8003520 con 0x7f5bb8004800 2024-04-20T11:33:38.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:38.985+0000 7f5bd5a00640 1 -- 172.21.15.152:0/1463129294 reap_dead start 2024-04-20T11:33:38.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:38.986+0000 7f5bcfe00640 1 --2- 172.21.15.152:0/1463129294 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5bb8004800 0x7f5bb8004c80 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc00ba00 tx=0x7f5bb8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7eb957cf79845ab6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:38.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:38.986+0000 7f5bcf400640 1 -- 172.21.15.152:0/1463129294 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bbc036020 con 0x7f5bb8004800 2024-04-20T11:33:38.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:38.986+0000 7f5bcf400640 1 -- 172.21.15.152:0/1463129294 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bbc032620 con 0x7f5bb8004800 2024-04-20T11:33:38.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:38.986+0000 7f5bcf400640 1 -- 172.21.15.152:0/1463129294 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bbc00b600 con 0x7f5bb8004800 2024-04-20T11:33:38.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:38.987+0000 7f5bd6400640 1 -- 172.21.15.152:0/1463129294 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5bb8004800 msgr2=0x7f5bb8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:38.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:38.987+0000 7f5bd6400640 1 --2- 172.21.15.152:0/1463129294 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5bb8004800 0x7f5bb8004c80 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc00ba00 tx=0x7f5bb8005830 comp rx=0 tx=0).stop 2024-04-20T11:33:38.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:38.987+0000 7f5bd6400640 1 -- 172.21.15.152:0/1463129294 shutdown_connections 2024-04-20T11:33:38.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:38.987+0000 7f5bd6400640 1 --2- 172.21.15.152:0/1463129294 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5bb8004800 0x7f5bb8004c80 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:38.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:38.987+0000 7f5bd6400640 1 -- 172.21.15.152:0/1463129294 >> 172.21.15.152:0/1463129294 conn(0x7f5bd006bda0 msgr2=0x7f5bd006c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:38.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:38.987+0000 7f5bd6400640 1 -- 172.21.15.152:0/1463129294 shutdown_connections 2024-04-20T11:33:38.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:38.987+0000 7f5bd6400640 1 -- 172.21.15.152:0/1463129294 wait complete. 2024-04-20T11:33:38.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:38.989+0000 7f5bd6400640 1 Processor -- start 2024-04-20T11:33:38.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:38.989+0000 7f5bd6400640 1 -- start start 2024-04-20T11:33:38.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:38.990+0000 7f5bd6400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5bb8004800 0x7f5bd01e4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:38.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:38.990+0000 7f5bd6400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5bd015b3b0 0x7f5bd01f7960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:38.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:38.990+0000 7f5bd6400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5bd015b830 0x7f5bd01f7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:38.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:38.990+0000 7f5bd6400640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f5bd0072360 con 0x7f5bd015b830 2024-04-20T11:33:38.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:38.990+0000 7f5bd6400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5bd00cd560 con 0x7f5bb8004800 2024-04-20T11:33:38.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:38.990+0000 7f5bd6400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5bd00ccbb0 con 0x7f5bd015b3b0 2024-04-20T11:33:38.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:38.990+0000 7f5bd5a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5bd015b830 0x7f5bd01f7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:38.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:38.990+0000 7f5bd5a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5bd015b830 0x7f5bd01f7ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:55452/0 (socket says 172.21.15.152:55452) 2024-04-20T11:33:38.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:38.990+0000 7f5bd5a00640 1 -- 172.21.15.152:0/203261158 learned_addr learned my addr 172.21.15.152:0/203261158 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:33:38.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:38.991+0000 7f5bcfe00640 1 --2- 172.21.15.152:0/203261158 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5bd015b3b0 0x7f5bd01f7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:38.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:38.991+0000 7f5bd5000640 1 --2- 172.21.15.152:0/203261158 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5bb8004800 0x7f5bd01e4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:38.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:38.991+0000 7f5bd5a00640 1 -- 172.21.15.152:0/203261158 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5bd015b3b0 msgr2=0x7f5bd01f7960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:38.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:38.991+0000 7f5bd5a00640 1 --2- 172.21.15.152:0/203261158 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5bd015b3b0 0x7f5bd01f7960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:38.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:38.991+0000 7f5bd5a00640 1 -- 172.21.15.152:0/203261158 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5bb8004800 msgr2=0x7f5bd01e4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:38.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:38.991+0000 7f5bd5a00640 1 --2- 172.21.15.152:0/203261158 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5bb8004800 0x7f5bd01e4bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:38.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:38.991+0000 7f5bd5a00640 1 -- 172.21.15.152:0/203261158 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bd015a680 con 0x7f5bd015b830 2024-04-20T11:33:38.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:38.991+0000 7f5bcfe00640 1 --2- 172.21.15.152:0/203261158 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5bd015b3b0 0x7f5bd01f7960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:33:38.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:38.991+0000 7f5bd5a00640 1 --2- 172.21.15.152:0/203261158 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5bd015b830 0x7f5bd01f7ea0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f5bc8000f80 tx=0x7f5bc800e160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:38.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:38.991+0000 7f5bccc00640 1 -- 172.21.15.152:0/203261158 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bc800eb10 con 0x7f5bd015b830 2024-04-20T11:33:38.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:38.991+0000 7f5bccc00640 1 -- 172.21.15.152:0/203261158 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bc8018070 con 0x7f5bd015b830 2024-04-20T11:33:38.995 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:38.991+0000 7f5bd6400640 1 -- 172.21.15.152:0/203261158 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bd01e5110 con 0x7f5bd015b830 2024-04-20T11:33:38.995 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:38.991+0000 7f5bccc00640 1 -- 172.21.15.152:0/203261158 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bc8013420 con 0x7f5bd015b830 2024-04-20T11:33:38.995 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:38.991+0000 7f5bd6400640 1 -- 172.21.15.152:0/203261158 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bd01e5670 con 0x7f5bd015b830 2024-04-20T11:33:38.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:38.993+0000 7f5bd6400640 1 -- 172.21.15.152:0/203261158 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bd00689f0 con 0x7f5bd015b830 2024-04-20T11:33:38.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:38.994+0000 7f5bccc00640 1 -- 172.21.15.152:0/203261158 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 91142+0+0 (secure 0 0 0) 0x7f5bc80135c0 con 0x7f5bd015b830 2024-04-20T11:33:38.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:38.994+0000 7f5bccc00640 1 --2- 172.21.15.152:0/203261158 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f5bb006ca40 0x7f5bb006ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:38.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:38.995+0000 7f5bccc00640 1 -- 172.21.15.152:0/203261158 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7f5bc808bff0 con 0x7f5bd015b830 2024-04-20T11:33:38.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:38.995+0000 7f5bd5000640 1 --2- 172.21.15.152:0/203261158 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f5bb006ca40 0x7f5bb006ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:38.997 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:38.997+0000 7f5bd5000640 1 --2- 172.21.15.152:0/203261158 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f5bb006ca40 0x7f5bb006ef00 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5bd015f340 tx=0x7f5bc4009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:38.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:38.999+0000 7f5bccc00640 1 -- 172.21.15.152:0/203261158 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f5bd01e5110 con 0x7f5bd015b830 2024-04-20T11:33:39.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:39.235+0000 7f5bd6400640 1 -- 172.21.15.152:0/203261158 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f5bd00cf550 con 0x7f5bd015b830 2024-04-20T11:33:39.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:39.898+0000 7f5bccc00640 1 -- 172.21.15.152:0/203261158 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v67) ==== 136+0+0 (secure 0 0 0) 0x7f5bc805a290 con 0x7f5bd015b830 2024-04-20T11:33:39.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:39.901+0000 7f5bd6400640 1 -- 172.21.15.152:0/203261158 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f5bb006ca40 msgr2=0x7f5bb006ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:39.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:39.902+0000 7f5bd6400640 1 --2- 172.21.15.152:0/203261158 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f5bb006ca40 0x7f5bb006ef00 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5bd015f340 tx=0x7f5bc4009040 comp rx=0 tx=0).stop 2024-04-20T11:33:39.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:39.902+0000 7f5bd6400640 1 -- 172.21.15.152:0/203261158 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5bd015b830 msgr2=0x7f5bd01f7ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:39.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:39.902+0000 7f5bd6400640 1 --2- 172.21.15.152:0/203261158 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5bd015b830 0x7f5bd01f7ea0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f5bc8000f80 tx=0x7f5bc800e160 comp rx=0 tx=0).stop 2024-04-20T11:33:39.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:39.902+0000 7f5bd6400640 1 -- 172.21.15.152:0/203261158 shutdown_connections 2024-04-20T11:33:39.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:39.902+0000 7f5bd6400640 1 --2- 172.21.15.152:0/203261158 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f5bb006ca40 0x7f5bb006ef00 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:39.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:39.902+0000 7f5bd6400640 1 --2- 172.21.15.152:0/203261158 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5bd015b830 0x7f5bd01f7ea0 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:39.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:39.902+0000 7f5bd6400640 1 --2- 172.21.15.152:0/203261158 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5bd015b3b0 0x7f5bd01f7960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:39.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:39.902+0000 7f5bd6400640 1 --2- 172.21.15.152:0/203261158 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5bb8004800 0x7f5bd01e4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:39.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:39.902+0000 7f5bd6400640 1 -- 172.21.15.152:0/203261158 >> 172.21.15.152:0/203261158 conn(0x7f5bd006bda0 msgr2=0x7f5bd00afcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:39.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:39.902+0000 7f5bd6400640 1 -- 172.21.15.152:0/203261158 shutdown_connections 2024-04-20T11:33:39.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:39.902+0000 7f5bd6400640 1 -- 172.21.15.152:0/203261158 wait complete. 2024-04-20T11:33:39.927 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-04-20T11:33:40.131 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.130+0000 7fd017200640 1 Processor -- start 2024-04-20T11:33:40.131 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.131+0000 7fd017200640 1 -- start start 2024-04-20T11:33:40.133 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.132+0000 7fd017200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd0101588f0 0x7fd0100b4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:40.133 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.132+0000 7fd017200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fd01015b330 0x7fd0100b49a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:40.133 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.132+0000 7fd017200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fd0100b3d60 0x7fd01015e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:40.134 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.132+0000 7fd017200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fd010158cd0 con 0x7fd01015b330 2024-04-20T11:33:40.134 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.132+0000 7fd017200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fd01015b7c0 con 0x7fd0100b3d60 2024-04-20T11:33:40.134 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.132+0000 7fd017200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd01015baa0 con 0x7fd0101588f0 2024-04-20T11:33:40.134 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.132+0000 7fd007400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fd01015b330 0x7fd0100b49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:40.134 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.132+0000 7fd007400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fd01015b330 0x7fd0100b49a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:55464/0 (socket says 172.21.15.152:55464) 2024-04-20T11:33:40.134 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.132+0000 7fd007400640 1 -- 172.21.15.152:0/2439865339 learned_addr learned my addr 172.21.15.152:0/2439865339 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:33:40.134 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.133+0000 7fd007400640 1 -- 172.21.15.152:0/2439865339 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd0101588f0 msgr2=0x7fd0100b4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:40.134 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.133+0000 7fd015200640 1 --2- 172.21.15.152:0/2439865339 >> v2:172.21.15.175:3300/0 conn(0x7fd0100b3d60 0x7fd01015e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:40.134 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.133+0000 7fd00fe00640 1 --2- 172.21.15.152:0/2439865339 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd0101588f0 0x7fd0100b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:40.134 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.133+0000 7fd007400640 1 --2- 172.21.15.152:0/2439865339 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd0101588f0 0x7fd0100b4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:40.134 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.133+0000 7fd007400640 1 -- 172.21.15.152:0/2439865339 >> v2:172.21.15.175:3300/0 conn(0x7fd0100b3d60 msgr2=0x7fd01015e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:40.134 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.133+0000 7fd007400640 1 --2- 172.21.15.152:0/2439865339 >> v2:172.21.15.175:3300/0 conn(0x7fd0100b3d60 0x7fd01015e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:40.134 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.133+0000 7fd007400640 1 -- 172.21.15.152:0/2439865339 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd01015bd80 con 0x7fd01015b330 2024-04-20T11:33:40.135 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.133+0000 7fd015200640 1 --2- 172.21.15.152:0/2439865339 >> v2:172.21.15.175:3300/0 conn(0x7fd0100b3d60 0x7fd01015e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:33:40.136 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.134+0000 7fd007400640 1 --2- 172.21.15.152:0/2439865339 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd01015b330 0x7fd0100b49a0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7fd010070190 tx=0x7fcff802b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=707e75a93ac7886 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:40.136 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.134+0000 7fd00f400640 1 -- 172.21.15.152:0/2439865339 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff8035020 con 0x7fd01015b330 2024-04-20T11:33:40.136 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.134+0000 7fd00f400640 1 -- 172.21.15.152:0/2439865339 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcff802d040 con 0x7fd01015b330 2024-04-20T11:33:40.136 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.134+0000 7fd00f400640 1 -- 172.21.15.152:0/2439865339 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff8031a80 con 0x7fd01015b330 2024-04-20T11:33:40.136 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.135+0000 7fd017200640 1 -- 172.21.15.152:0/2439865339 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd01015b330 msgr2=0x7fd0100b49a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:40.136 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.135+0000 7fd017200640 1 --2- 172.21.15.152:0/2439865339 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd01015b330 0x7fd0100b49a0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7fd010070190 tx=0x7fcff802b900 comp rx=0 tx=0).stop 2024-04-20T11:33:40.136 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.135+0000 7fd017200640 1 -- 172.21.15.152:0/2439865339 shutdown_connections 2024-04-20T11:33:40.136 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.135+0000 7fd017200640 1 --2- 172.21.15.152:0/2439865339 >> v2:172.21.15.175:3300/0 conn(0x7fd0100b3d60 0x7fd01015e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:40.136 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.135+0000 7fd017200640 1 --2- 172.21.15.152:0/2439865339 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd01015b330 0x7fd0100b49a0 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:40.136 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.135+0000 7fd017200640 1 --2- 172.21.15.152:0/2439865339 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd0101588f0 0x7fd0100b4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:40.137 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.135+0000 7fd017200640 1 -- 172.21.15.152:0/2439865339 >> 172.21.15.152:0/2439865339 conn(0x7fd010067cf0 msgr2=0x7fd01006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:40.137 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.135+0000 7fd017200640 1 -- 172.21.15.152:0/2439865339 shutdown_connections 2024-04-20T11:33:40.137 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.135+0000 7fd017200640 1 -- 172.21.15.152:0/2439865339 wait complete. 2024-04-20T11:33:40.137 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.137+0000 7fd017200640 1 Processor -- start 2024-04-20T11:33:40.137 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.137+0000 7fd017200640 1 -- start start 2024-04-20T11:33:40.139 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.137+0000 7fd017200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd0100b3d60 0x7fd0101f1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:40.139 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.138+0000 7fd017200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd0101588f0 0x7fd010152650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:40.140 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.138+0000 7fd017200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd01014eb80 0x7fd0101543b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:40.140 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.138+0000 7fd017200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fd01006f460 con 0x7fd0100b3d60 2024-04-20T11:33:40.140 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.138+0000 7fd017200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd01006f2e0 con 0x7fd0101588f0 2024-04-20T11:33:40.140 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.138+0000 7fd017200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd010072750 con 0x7fd01014eb80 2024-04-20T11:33:40.140 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.138+0000 7fd00fe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd0100b3d60 0x7fd0101f1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:40.140 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.138+0000 7fd00fe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd0100b3d60 0x7fd0101f1fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:55480/0 (socket says 172.21.15.152:55480) 2024-04-20T11:33:40.140 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.138+0000 7fd00fe00640 1 -- 172.21.15.152:0/2836338430 learned_addr learned my addr 172.21.15.152:0/2836338430 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:33:40.140 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.138+0000 7fd015200640 1 --2- 172.21.15.152:0/2836338430 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd01014eb80 0x7fd0101543b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:40.140 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.138+0000 7fd00fe00640 1 -- 172.21.15.152:0/2836338430 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd01014eb80 msgr2=0x7fd0101543b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:40.140 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.138+0000 7fd007400640 1 --2- 172.21.15.152:0/2836338430 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd0101588f0 0x7fd010152650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:40.140 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.138+0000 7fd00fe00640 1 --2- 172.21.15.152:0/2836338430 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd01014eb80 0x7fd0101543b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:40.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.138+0000 7fd00fe00640 1 -- 172.21.15.152:0/2836338430 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd0101588f0 msgr2=0x7fd010152650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:40.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.138+0000 7fd00fe00640 1 --2- 172.21.15.152:0/2836338430 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd0101588f0 0x7fd010152650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:40.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.138+0000 7fd00fe00640 1 -- 172.21.15.152:0/2836338430 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd01014f040 con 0x7fd0100b3d60 2024-04-20T11:33:40.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.139+0000 7fd00fe00640 1 --2- 172.21.15.152:0/2836338430 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd0100b3d60 0x7fd0101f1fa0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7fd00000c7f0 tx=0x7fd00000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:40.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.139+0000 7fd00cc00640 1 -- 172.21.15.152:0/2836338430 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd000015070 con 0x7fd0100b3d60 2024-04-20T11:33:40.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.139+0000 7fd00cc00640 1 -- 172.21.15.152:0/2836338430 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd000004240 con 0x7fd0100b3d60 2024-04-20T11:33:40.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.139+0000 7fd00cc00640 1 -- 172.21.15.152:0/2836338430 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd000007620 con 0x7fd0100b3d60 2024-04-20T11:33:40.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.139+0000 7fd017200640 1 -- 172.21.15.152:0/2836338430 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd01014f320 con 0x7fd0100b3d60 2024-04-20T11:33:40.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.139+0000 7fd017200640 1 -- 172.21.15.152:0/2836338430 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd01014f8e0 con 0x7fd0100b3d60 2024-04-20T11:33:40.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.140+0000 7fd017200640 1 -- 172.21.15.152:0/2836338430 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd01014fd20 con 0x7fd0100b3d60 2024-04-20T11:33:40.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.141+0000 7fd00cc00640 1 -- 172.21.15.152:0/2836338430 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 91142+0+0 (secure 0 0 0) 0x7fd0000079d0 con 0x7fd0100b3d60 2024-04-20T11:33:40.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.142+0000 7fd00cc00640 1 --2- 172.21.15.152:0/2836338430 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fcfec06ca40 0x7fcfec06ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:40.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.142+0000 7fd00cc00640 1 -- 172.21.15.152:0/2836338430 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7fd00008bea0 con 0x7fd0100b3d60 2024-04-20T11:33:40.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.143+0000 7fd007400640 1 --2- 172.21.15.152:0/2836338430 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fcfec06ca40 0x7fcfec06ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:40.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.144+0000 7fd007400640 1 --2- 172.21.15.152:0/2836338430 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fcfec06ca40 0x7fcfec06ef00 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fcff8036000 tx=0x7fcff8008030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:40.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.147+0000 7fd00cc00640 1 -- 172.21.15.152:0/2836338430 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fd00005a090 con 0x7fd0100b3d60 2024-04-20T11:33:40.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.371+0000 7fd017200640 1 -- 172.21.15.152:0/2836338430 --> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7fd0100689f0 con 0x7fcfec06ca40 2024-04-20T11:33:40.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.375+0000 7fd00cc00640 1 -- 172.21.15.152:0/2836338430 <== mgr.4109 v2:172.21.15.175:6836/4021250441 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7fd0100689f0 con 0x7fcfec06ca40 2024-04-20T11:33:40.375 INFO:teuthology.orchestra.run.smithi152.stdout:[ 2024-04-20T11:33:40.375 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:33:40.376 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "cephfs" 2024-04-20T11:33:40.376 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:33:40.376 INFO:teuthology.orchestra.run.smithi152.stdout:] 2024-04-20T11:33:40.378 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.377+0000 7fd017200640 1 -- 172.21.15.152:0/2836338430 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fcfec06ca40 msgr2=0x7fcfec06ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:40.378 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.377+0000 7fd017200640 1 --2- 172.21.15.152:0/2836338430 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fcfec06ca40 0x7fcfec06ef00 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fcff8036000 tx=0x7fcff8008030 comp rx=0 tx=0).stop 2024-04-20T11:33:40.378 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.377+0000 7fd017200640 1 -- 172.21.15.152:0/2836338430 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd0100b3d60 msgr2=0x7fd0101f1fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:40.378 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.377+0000 7fd017200640 1 --2- 172.21.15.152:0/2836338430 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd0100b3d60 0x7fd0101f1fa0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7fd00000c7f0 tx=0x7fd00000ccb0 comp rx=0 tx=0).stop 2024-04-20T11:33:40.378 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.377+0000 7fd017200640 1 -- 172.21.15.152:0/2836338430 shutdown_connections 2024-04-20T11:33:40.378 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.377+0000 7fd017200640 1 --2- 172.21.15.152:0/2836338430 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fcfec06ca40 0x7fcfec06ef00 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:40.378 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.377+0000 7fd017200640 1 --2- 172.21.15.152:0/2836338430 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd01014eb80 0x7fd0101543b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:40.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.377+0000 7fd017200640 1 --2- 172.21.15.152:0/2836338430 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd0101588f0 0x7fd010152650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:40.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.377+0000 7fd017200640 1 --2- 172.21.15.152:0/2836338430 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd0100b3d60 0x7fd0101f1fa0 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:40.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.377+0000 7fd017200640 1 -- 172.21.15.152:0/2836338430 >> 172.21.15.152:0/2836338430 conn(0x7fd010067cf0 msgr2=0x7fd010157d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:40.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.377+0000 7fd017200640 1 -- 172.21.15.152:0/2836338430 shutdown_connections 2024-04-20T11:33:40.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.378+0000 7fd017200640 1 -- 172.21.15.152:0/2836338430 wait complete. 2024-04-20T11:33:40.398 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-20T11:33:40.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.602+0000 7f6c26c00640 1 Processor -- start 2024-04-20T11:33:40.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.603+0000 7f6c26c00640 1 -- start start 2024-04-20T11:33:40.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.604+0000 7f6c26c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6c20071cd0 0x7f6c200720b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:40.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.604+0000 7f6c26c00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f6c20160220 0x7f6c20160680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:40.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.604+0000 7f6c26c00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f6c2006eb30 0x7f6c2006efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:40.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.604+0000 7f6c26c00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f6c2013f8e0 con 0x7f6c20160220 2024-04-20T11:33:40.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.604+0000 7f6c26c00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f6c200725f0 con 0x7f6c2006eb30 2024-04-20T11:33:40.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.604+0000 7f6c26c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6c20160f90 con 0x7f6c20071cd0 2024-04-20T11:33:40.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.605+0000 7f6c24e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f6c20160220 0x7f6c20160680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:40.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.605+0000 7f6c24e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f6c20160220 0x7f6c20160680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:55488/0 (socket says 172.21.15.152:55488) 2024-04-20T11:33:40.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.605+0000 7f6c24e00640 1 -- 172.21.15.152:0/2547753849 learned_addr learned my addr 172.21.15.152:0/2547753849 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:33:40.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.605+0000 7f6c25800640 1 --2- 172.21.15.152:0/2547753849 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6c20071cd0 0x7f6c200720b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:40.607 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.605+0000 7f6c24e00640 1 -- 172.21.15.152:0/2547753849 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6c20071cd0 msgr2=0x7f6c200720b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:40.607 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.605+0000 7f6c26200640 1 --2- 172.21.15.152:0/2547753849 >> v2:172.21.15.175:3300/0 conn(0x7f6c2006eb30 0x7f6c2006efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:40.607 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.605+0000 7f6c24e00640 1 --2- 172.21.15.152:0/2547753849 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6c20071cd0 0x7f6c200720b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:40.607 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.605+0000 7f6c24e00640 1 -- 172.21.15.152:0/2547753849 >> v2:172.21.15.175:3300/0 conn(0x7f6c2006eb30 msgr2=0x7f6c2006efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:40.607 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.605+0000 7f6c24e00640 1 --2- 172.21.15.152:0/2547753849 >> v2:172.21.15.175:3300/0 conn(0x7f6c2006eb30 0x7f6c2006efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:40.607 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.605+0000 7f6c24e00640 1 -- 172.21.15.152:0/2547753849 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c2006dd90 con 0x7f6c20160220 2024-04-20T11:33:40.607 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.605+0000 7f6c26200640 1 --2- 172.21.15.152:0/2547753849 >> v2:172.21.15.175:3300/0 conn(0x7f6c2006eb30 0x7f6c2006efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:33:40.607 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.606+0000 7f6c24e00640 1 --2- 172.21.15.152:0/2547753849 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6c20160220 0x7f6c20160680 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f6c200cfa50 tx=0x7f6c0c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a172ff3e83e14f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:40.608 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.607+0000 7f6c1fe00640 1 -- 172.21.15.152:0/2547753849 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c0c02e070 con 0x7f6c20160220 2024-04-20T11:33:40.608 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.607+0000 7f6c1fe00640 1 -- 172.21.15.152:0/2547753849 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c0c037070 con 0x7f6c20160220 2024-04-20T11:33:40.608 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.607+0000 7f6c1fe00640 1 -- 172.21.15.152:0/2547753849 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c0c031a00 con 0x7f6c20160220 2024-04-20T11:33:40.608 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.607+0000 7f6c26c00640 1 -- 172.21.15.152:0/2547753849 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6c20160220 msgr2=0x7f6c20160680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:40.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.607+0000 7f6c26c00640 1 --2- 172.21.15.152:0/2547753849 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6c20160220 0x7f6c20160680 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f6c200cfa50 tx=0x7f6c0c02b880 comp rx=0 tx=0).stop 2024-04-20T11:33:40.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.607+0000 7f6c26c00640 1 -- 172.21.15.152:0/2547753849 shutdown_connections 2024-04-20T11:33:40.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.607+0000 7f6c26c00640 1 --2- 172.21.15.152:0/2547753849 >> v2:172.21.15.175:3300/0 conn(0x7f6c2006eb30 0x7f6c2006efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:40.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.607+0000 7f6c26c00640 1 --2- 172.21.15.152:0/2547753849 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6c20160220 0x7f6c20160680 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:40.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.607+0000 7f6c26c00640 1 --2- 172.21.15.152:0/2547753849 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6c20071cd0 0x7f6c200720b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:40.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.607+0000 7f6c26c00640 1 -- 172.21.15.152:0/2547753849 >> 172.21.15.152:0/2547753849 conn(0x7f6c20067ae0 msgr2=0x7f6c2006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:40.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.607+0000 7f6c26c00640 1 -- 172.21.15.152:0/2547753849 shutdown_connections 2024-04-20T11:33:40.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.608+0000 7f6c26c00640 1 -- 172.21.15.152:0/2547753849 wait complete. 2024-04-20T11:33:40.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.609+0000 7f6c26c00640 1 Processor -- start 2024-04-20T11:33:40.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.609+0000 7f6c26c00640 1 -- start start 2024-04-20T11:33:40.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.610+0000 7f6c26c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6c2006eb30 0x7f6c2014fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:40.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.610+0000 7f6c26c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6c20071cd0 0x7f6c20135590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:40.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.610+0000 7f6c26c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6c20160220 0x7f6c20135ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:40.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.610+0000 7f6c26c00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f6c2006e630 con 0x7f6c20160220 2024-04-20T11:33:40.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.610+0000 7f6c26c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6c2006e350 con 0x7f6c2006eb30 2024-04-20T11:33:40.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.610+0000 7f6c26c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6c2013fb90 con 0x7f6c20071cd0 2024-04-20T11:33:40.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.611+0000 7f6c26200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6c20160220 0x7f6c20135ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:40.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.611+0000 7f6c26200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6c20160220 0x7f6c20135ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:55496/0 (socket says 172.21.15.152:55496) 2024-04-20T11:33:40.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.611+0000 7f6c26200640 1 -- 172.21.15.152:0/3608771265 learned_addr learned my addr 172.21.15.152:0/3608771265 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:33:40.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.611+0000 7f6c25800640 1 --2- 172.21.15.152:0/3608771265 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6c2006eb30 0x7f6c2014fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:40.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.611+0000 7f6c24e00640 1 --2- 172.21.15.152:0/3608771265 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6c20071cd0 0x7f6c20135590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:40.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.611+0000 7f6c26200640 1 -- 172.21.15.152:0/3608771265 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6c20071cd0 msgr2=0x7f6c20135590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:40.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.611+0000 7f6c26200640 1 --2- 172.21.15.152:0/3608771265 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6c20071cd0 0x7f6c20135590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:40.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.611+0000 7f6c26200640 1 -- 172.21.15.152:0/3608771265 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6c2006eb30 msgr2=0x7f6c2014fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:40.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.611+0000 7f6c26200640 1 --2- 172.21.15.152:0/3608771265 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6c2006eb30 0x7f6c2014fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:40.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.611+0000 7f6c26200640 1 -- 172.21.15.152:0/3608771265 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c20134910 con 0x7f6c20160220 2024-04-20T11:33:40.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.611+0000 7f6c25800640 1 --2- 172.21.15.152:0/3608771265 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6c2006eb30 0x7f6c2014fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:33:40.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.611+0000 7f6c26200640 1 --2- 172.21.15.152:0/3608771265 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6c20160220 0x7f6c20135ad0 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f6c1800cd80 tx=0x7f6c1800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:40.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.612+0000 7f6c1d600640 1 -- 172.21.15.152:0/3608771265 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c1800f680 con 0x7f6c20160220 2024-04-20T11:33:40.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.612+0000 7f6c1d600640 1 -- 172.21.15.152:0/3608771265 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c1800f840 con 0x7f6c20160220 2024-04-20T11:33:40.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.612+0000 7f6c1d600640 1 -- 172.21.15.152:0/3608771265 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c1800fc30 con 0x7f6c20160220 2024-04-20T11:33:40.614 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.612+0000 7f6c26c00640 1 -- 172.21.15.152:0/3608771265 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c20137d90 con 0x7f6c20160220 2024-04-20T11:33:40.614 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.612+0000 7f6c26c00640 1 -- 172.21.15.152:0/3608771265 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c20138350 con 0x7f6c20160220 2024-04-20T11:33:40.614 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.613+0000 7f6c26c00640 1 -- 172.21.15.152:0/3608771265 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c200cfb20 con 0x7f6c20160220 2024-04-20T11:33:40.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.614+0000 7f6c1d600640 1 -- 172.21.15.152:0/3608771265 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 91142+0+0 (secure 0 0 0) 0x7f6c1800e3e0 con 0x7f6c20160220 2024-04-20T11:33:40.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.615+0000 7f6c1d600640 1 --2- 172.21.15.152:0/3608771265 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f6c0006ca10 0x7f6c0006eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:40.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.615+0000 7f6c1d600640 1 -- 172.21.15.152:0/3608771265 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7f6c1808cb80 con 0x7f6c20160220 2024-04-20T11:33:40.616 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.615+0000 7f6c25800640 1 --2- 172.21.15.152:0/3608771265 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f6c0006ca10 0x7f6c0006eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:40.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.616+0000 7f6c25800640 1 --2- 172.21.15.152:0/3608771265 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f6c0006ca10 0x7f6c0006eed0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6c1400a950 tx=0x7f6c14008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:40.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.619+0000 7f6c1d600640 1 -- 172.21.15.152:0/3608771265 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f6c200cfb20 con 0x7f6c20160220 2024-04-20T11:33:40.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.818+0000 7f6c26c00640 1 -- 172.21.15.152:0/3608771265 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f6c20075e60 con 0x7f6c20160220 2024-04-20T11:33:40.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.819+0000 7f6c1d600640 1 -- 172.21.15.152:0/3608771265 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v20)=0 v20) ==== 122+0+0 (secure 0 0 0) 0x7f6c18014030 con 0x7f6c20160220 2024-04-20T11:33:40.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.821+0000 7f6c26c00640 1 -- 172.21.15.152:0/3608771265 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f6c0006ca10 msgr2=0x7f6c0006eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:40.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.821+0000 7f6c26c00640 1 --2- 172.21.15.152:0/3608771265 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f6c0006ca10 0x7f6c0006eed0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6c1400a950 tx=0x7f6c14008040 comp rx=0 tx=0).stop 2024-04-20T11:33:40.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.821+0000 7f6c26c00640 1 -- 172.21.15.152:0/3608771265 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6c20160220 msgr2=0x7f6c20135ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:40.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.821+0000 7f6c26c00640 1 --2- 172.21.15.152:0/3608771265 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6c20160220 0x7f6c20135ad0 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f6c1800cd80 tx=0x7f6c1800a650 comp rx=0 tx=0).stop 2024-04-20T11:33:40.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.822+0000 7f6c26c00640 1 -- 172.21.15.152:0/3608771265 shutdown_connections 2024-04-20T11:33:40.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.822+0000 7f6c26c00640 1 --2- 172.21.15.152:0/3608771265 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f6c0006ca10 0x7f6c0006eed0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:40.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.822+0000 7f6c26c00640 1 --2- 172.21.15.152:0/3608771265 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6c20160220 0x7f6c20135ad0 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:40.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.822+0000 7f6c26c00640 1 --2- 172.21.15.152:0/3608771265 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6c20071cd0 0x7f6c20135590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:40.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.822+0000 7f6c26c00640 1 --2- 172.21.15.152:0/3608771265 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6c2006eb30 0x7f6c2014fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:40.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.822+0000 7f6c26c00640 1 -- 172.21.15.152:0/3608771265 >> 172.21.15.152:0/3608771265 conn(0x7f6c20067ae0 msgr2=0x7f6c2013ec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:40.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.822+0000 7f6c26c00640 1 -- 172.21.15.152:0/3608771265 shutdown_connections 2024-04-20T11:33:40.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:40.822+0000 7f6c26c00640 1 -- 172.21.15.152:0/3608771265 wait complete. 2024-04-20T11:33:40.843 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:40.843 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/_nogroup/subvol48180/bf9c0317-2e4a-4eaf-b73e-875a6eb33282) 2024-04-20T11:33:40.947 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:40.948 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/_nogroup/subvol48180/bf9c0317-2e4a-4eaf-b73e-875a6eb33282/.snap/fake) 2024-04-20T11:33:41.050 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:41.051 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2024-04-20T11:33:41.108 INFO:teuthology.orchestra.run.smithi152.stdout:cephfs_data 2024-04-20T11:33:41.108 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:41.108 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data volumes/_nogroup/subvol48180/bf9c0317-2e4a-4eaf-b73e-875a6eb33282) 2024-04-20T11:33:41.199 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-20T11:33:41.199 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/home/ubuntu/cephtest/mnt.0/volumes/_nogroup/subvol48180/.meta 2024-04-20T11:33:41.293 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol48180 2024-04-20T11:33:41.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:41.550+0000 7f6180600640 1 Processor -- start 2024-04-20T11:33:41.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:41.550+0000 7f6180600640 1 -- start start 2024-04-20T11:33:41.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:41.552+0000 7f6180600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f617815e890 0x7f61780d27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:41.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:41.552+0000 7f6180600640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f61780ba430 0x7f61780d1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:41.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:41.552+0000 7f6180600640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f617815ec70 con 0x7f61780d2d20 2024-04-20T11:33:41.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:41.552+0000 7f6180600640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f61780b8e20 con 0x7f61780ba430 2024-04-20T11:33:41.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:41.552+0000 7f6180600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f61780b8210 con 0x7f617815e890 2024-04-20T11:33:41.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:41.552+0000 7f617dc00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f617815e890 0x7f61780d27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:41.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:41.553+0000 7f617e600640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f61780ba430 0x7f61780d1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:41.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:41.553+0000 7f617d200640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7f61780d2d20 0x7f61780b7780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.152:43954/0 (socket says 172.21.15.152:43954) 2024-04-20T11:33:41.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:41.553+0000 7f617d200640 1 -- 172.21.15.152:0/2036593763 learned_addr learned my addr 172.21.15.152:0/2036593763 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:33:41.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:41.553+0000 7f617e600640 1 --2- 172.21.15.152:0/2036593763 >> v2:172.21.15.175:3300/0 conn(0x7f61780ba430 0x7f61780d1790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.152:54300/0 (socket says 172.21.15.152:54300) 2024-04-20T11:33:41.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:41.554+0000 7f617dc00640 1 -- 172.21.15.152:0/2036593763 >> v2:172.21.15.175:3300/0 conn(0x7f61780ba430 msgr2=0x7f61780d1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:41.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:41.554+0000 7f617dc00640 1 --2- 172.21.15.152:0/2036593763 >> v2:172.21.15.175:3300/0 conn(0x7f61780ba430 0x7f61780d1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:41.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:41.554+0000 7f617dc00640 1 -- 172.21.15.152:0/2036593763 >> v1:172.21.15.175:6789/0 conn(0x7f61780d2d20 legacy=0x7f61780b7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:41.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:41.554+0000 7f617dc00640 1 -- 172.21.15.152:0/2036593763 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61780b84f0 con 0x7f617815e890 2024-04-20T11:33:41.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:41.554+0000 7f617e600640 1 --2- 172.21.15.152:0/2036593763 >> v2:172.21.15.175:3300/0 conn(0x7f61780ba430 0x7f61780d1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:33:41.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:41.555+0000 7f617dc00640 1 --2- 172.21.15.152:0/2036593763 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f617815e890 0x7f61780d27e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f616c0097a0 tx=0x7f616c02b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e6aaeb5623dc1d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:41.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:41.556+0000 7f6177e00640 1 -- 172.21.15.152:0/2036593763 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f616c034070 con 0x7f617815e890 2024-04-20T11:33:41.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:41.556+0000 7f6177e00640 1 -- 172.21.15.152:0/2036593763 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f616c030610 con 0x7f617815e890 2024-04-20T11:33:41.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:41.556+0000 7f6177e00640 1 -- 172.21.15.152:0/2036593763 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f616c0309e0 con 0x7f617815e890 2024-04-20T11:33:41.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:41.556+0000 7f6180600640 1 -- 172.21.15.152:0/2036593763 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f617815e890 msgr2=0x7f61780d27e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:41.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:41.556+0000 7f6180600640 1 --2- 172.21.15.152:0/2036593763 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f617815e890 0x7f61780d27e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f616c0097a0 tx=0x7f616c02b960 comp rx=0 tx=0).stop 2024-04-20T11:33:41.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:41.556+0000 7f6180600640 1 -- 172.21.15.152:0/2036593763 shutdown_connections 2024-04-20T11:33:41.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:41.556+0000 7f6180600640 1 --2- 172.21.15.152:0/2036593763 >> v2:172.21.15.175:3300/0 conn(0x7f61780ba430 0x7f61780d1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:41.558 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:41.556+0000 7f6180600640 1 --2- 172.21.15.152:0/2036593763 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f617815e890 0x7f61780d27e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:41.558 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:41.556+0000 7f6180600640 1 -- 172.21.15.152:0/2036593763 >> 172.21.15.152:0/2036593763 conn(0x7f6178067ae0 msgr2=0x7f6178067ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:41.558 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:41.557+0000 7f6180600640 1 -- 172.21.15.152:0/2036593763 shutdown_connections 2024-04-20T11:33:41.558 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:41.557+0000 7f6180600640 1 -- 172.21.15.152:0/2036593763 wait complete. 2024-04-20T11:33:41.559 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:41.558+0000 7f6180600640 1 Processor -- start 2024-04-20T11:33:41.559 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:41.558+0000 7f6180600640 1 -- start start 2024-04-20T11:33:41.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:41.559+0000 7f6180600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f61780ba430 0x7f6178161110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:41.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:41.559+0000 7f6180600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f617815e890 0x7f6178161650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:41.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:41.559+0000 7f6180600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6178161b90 0x7f6178146d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:41.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:41.559+0000 7f6180600640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f61780b7e90 con 0x7f617815e890 2024-04-20T11:33:41.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:41.559+0000 7f6180600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f61780b8ab0 con 0x7f61780ba430 2024-04-20T11:33:41.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:41.559+0000 7f6180600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f61780d1cd0 con 0x7f6178161b90 2024-04-20T11:33:41.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:41.560+0000 7f617d200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f617815e890 0x7f6178161650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:41.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:41.560+0000 7f617d200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f617815e890 0x7f6178161650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:55512/0 (socket says 172.21.15.152:55512) 2024-04-20T11:33:41.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:41.560+0000 7f617d200640 1 -- 172.21.15.152:0/4210784864 learned_addr learned my addr 172.21.15.152:0/4210784864 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:33:41.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:41.560+0000 7f617e600640 1 --2- 172.21.15.152:0/4210784864 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6178161b90 0x7f6178146d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:41.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:41.560+0000 7f617d200640 1 -- 172.21.15.152:0/4210784864 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6178161b90 msgr2=0x7f6178146d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:41.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:41.560+0000 7f617dc00640 1 --2- 172.21.15.152:0/4210784864 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f61780ba430 0x7f6178161110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:41.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:41.560+0000 7f617d200640 1 --2- 172.21.15.152:0/4210784864 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6178161b90 0x7f6178146d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:41.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:41.560+0000 7f617d200640 1 -- 172.21.15.152:0/4210784864 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f61780ba430 msgr2=0x7f6178161110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:41.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:41.560+0000 7f617d200640 1 --2- 172.21.15.152:0/4210784864 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f61780ba430 0x7f6178161110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:41.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:41.560+0000 7f617d200640 1 -- 172.21.15.152:0/4210784864 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61780cbd00 con 0x7f617815e890 2024-04-20T11:33:41.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:41.561+0000 7f617d200640 1 --2- 172.21.15.152:0/4210784864 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f617815e890 0x7f6178161650 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f616800cb40 tx=0x7f616800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:41.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:41.561+0000 7f6175600640 1 -- 172.21.15.152:0/4210784864 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6168017070 con 0x7f617815e890 2024-04-20T11:33:41.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:41.561+0000 7f6175600640 1 -- 172.21.15.152:0/4210784864 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f616800aca0 con 0x7f617815e890 2024-04-20T11:33:41.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:41.561+0000 7f6175600640 1 -- 172.21.15.152:0/4210784864 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6168012620 con 0x7f617815e890 2024-04-20T11:33:41.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:41.561+0000 7f6180600640 1 -- 172.21.15.152:0/4210784864 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61780c9d50 con 0x7f617815e890 2024-04-20T11:33:41.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:41.561+0000 7f6180600640 1 -- 172.21.15.152:0/4210784864 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61780ca310 con 0x7f617815e890 2024-04-20T11:33:41.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:41.562+0000 7f6180600640 1 -- 172.21.15.152:0/4210784864 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61780689f0 con 0x7f617815e890 2024-04-20T11:33:41.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:41.563+0000 7f6175600640 1 -- 172.21.15.152:0/4210784864 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 91142+0+0 (secure 0 0 0) 0x7f616801a070 con 0x7f617815e890 2024-04-20T11:33:41.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:41.565+0000 7f6175600640 1 --2- 172.21.15.152:0/4210784864 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f615c06ca10 0x7f615c06eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:41.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:41.565+0000 7f6175600640 1 -- 172.21.15.152:0/4210784864 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7f616808bac0 con 0x7f617815e890 2024-04-20T11:33:41.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:41.565+0000 7f617dc00640 1 --2- 172.21.15.152:0/4210784864 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f615c06ca10 0x7f615c06eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:41.567 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:41.566+0000 7f617dc00640 1 --2- 172.21.15.152:0/4210784864 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f615c06ca10 0x7f615c06eed0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f616c036000 tx=0x7f616c008030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:41.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:41.569+0000 7f6175600640 1 -- 172.21.15.152:0/4210784864 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f6168059dd0 con 0x7f617815e890 2024-04-20T11:33:41.797 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:41.796+0000 7f6180600640 1 -- 172.21.15.152:0/4210784864 --> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol48180", "target": ["mon-mgr", ""]}) -- 0x7f61780cf420 con 0x7f615c06ca10 2024-04-20T11:33:41.810 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:33:41.810+0000 7f8a5cc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:33:41.810 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:33:41.810+0000 7f8a5cc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:33:41.810 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:33:41.810+0000 7f8a5cc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:33:41.811 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:33:41.810+0000 7f8a5cc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:33:41.811 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:33:41.810+0000 7f8a5cc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:33:41.878 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:41.874+0000 7f6175600640 1 -- 172.21.15.152:0/4210784864 <== mgr.4109 v2:172.21.15.175:6836/4021250441 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+67 (secure 0 0 0) 0x7f61780cf420 con 0x7f615c06ca10 2024-04-20T11:33:41.879 INFO:teuthology.orchestra.run.smithi152.stdout:/volumes/_nogroup/subvol48180/bf9c0317-2e4a-4eaf-b73e-875a6eb33282 2024-04-20T11:33:41.879 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:41.877+0000 7f6180600640 1 -- 172.21.15.152:0/4210784864 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f615c06ca10 msgr2=0x7f615c06eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:41.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:41.877+0000 7f6180600640 1 --2- 172.21.15.152:0/4210784864 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f615c06ca10 0x7f615c06eed0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f616c036000 tx=0x7f616c008030 comp rx=0 tx=0).stop 2024-04-20T11:33:41.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:41.877+0000 7f6180600640 1 -- 172.21.15.152:0/4210784864 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f617815e890 msgr2=0x7f6178161650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:41.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:41.877+0000 7f6180600640 1 --2- 172.21.15.152:0/4210784864 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f617815e890 0x7f6178161650 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f616800cb40 tx=0x7f616800a060 comp rx=0 tx=0).stop 2024-04-20T11:33:41.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:41.877+0000 7f6180600640 1 -- 172.21.15.152:0/4210784864 shutdown_connections 2024-04-20T11:33:41.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:41.877+0000 7f6180600640 1 --2- 172.21.15.152:0/4210784864 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f615c06ca10 0x7f615c06eed0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:41.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:41.877+0000 7f6180600640 1 --2- 172.21.15.152:0/4210784864 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6178161b90 0x7f6178146d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:41.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:41.877+0000 7f6180600640 1 --2- 172.21.15.152:0/4210784864 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f617815e890 0x7f6178161650 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:41.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:41.877+0000 7f6180600640 1 --2- 172.21.15.152:0/4210784864 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f61780ba430 0x7f6178161110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:41.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:41.877+0000 7f6180600640 1 -- 172.21.15.152:0/4210784864 >> 172.21.15.152:0/4210784864 conn(0x7f6178067ae0 msgr2=0x7f6178162c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:41.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:41.877+0000 7f6180600640 1 -- 172.21.15.152:0/4210784864 shutdown_connections 2024-04-20T11:33:41.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:41.877+0000 7f6180600640 1 -- 172.21.15.152:0/4210784864 wait complete. 2024-04-20T11:33:41.900 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs 2024-04-20T11:33:42.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.105+0000 7f1d77000640 1 Processor -- start 2024-04-20T11:33:42.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.105+0000 7f1d77000640 1 -- start start 2024-04-20T11:33:42.107 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.107+0000 7f1d77000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f1d7015b510 0x7f1d7015fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:42.108 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.107+0000 7f1d77000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1d700cfc40 0x7f1d700b3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:42.108 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.107+0000 7f1d77000640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d7015f1c0 con 0x7f1d700b2170 2024-04-20T11:33:42.108 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.107+0000 7f1d77000640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f1d7006f420 con 0x7f1d7015b510 2024-04-20T11:33:42.108 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.107+0000 7f1d77000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1d700b4430 con 0x7f1d700cfc40 2024-04-20T11:33:42.109 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.107+0000 7f1d75000640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f1d700b2170 0x7f1d700b4800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:58074/0 (socket says 172.21.15.152:58074) 2024-04-20T11:33:42.109 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.108+0000 7f1d75000640 1 -- 172.21.15.152:0/3611710591 learned_addr learned my addr 172.21.15.152:0/3611710591 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:33:42.109 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.107+0000 7f1d6fe00640 1 --2- 172.21.15.152:0/3611710591 >> v2:172.21.15.152:3300/0 conn(0x7f1d7015b510 0x7f1d7015fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:42.109 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.108+0000 7f1d67400640 1 --2- 172.21.15.152:0/3611710591 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1d700cfc40 0x7f1d700b3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:42.109 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.108+0000 7f1d6fe00640 1 -- 172.21.15.152:0/3611710591 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1d700cfc40 msgr2=0x7f1d700b3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:42.109 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.108+0000 7f1d6fe00640 1 --2- 172.21.15.152:0/3611710591 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1d700cfc40 0x7f1d700b3ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:42.109 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.108+0000 7f1d6fe00640 1 -- 172.21.15.152:0/3611710591 >> v1:172.21.15.152:6789/0 conn(0x7f1d700b2170 legacy=0x7f1d700b4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:42.109 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.108+0000 7f1d6fe00640 1 -- 172.21.15.152:0/3611710591 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d7006f670 con 0x7f1d7015b510 2024-04-20T11:33:42.109 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.108+0000 7f1d6f400640 1 -- 172.21.15.152:0/3611710591 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 582738124 0 0) 0x7f1d7015f1c0 con 0x7f1d700b2170 2024-04-20T11:33:42.110 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.110+0000 7f1d6fe00640 1 --2- 172.21.15.152:0/3611710591 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d7015b510 0x7f1d7015fc30 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f1d600097a0 tx=0x7f1d6002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d29793e870856b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:42.110 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.110+0000 7f1d6f400640 1 -- 172.21.15.152:0/3611710591 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d60035020 con 0x7f1d7015b510 2024-04-20T11:33:42.111 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.110+0000 7f1d6f400640 1 -- 172.21.15.152:0/3611710591 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d60031830 con 0x7f1d7015b510 2024-04-20T11:33:42.111 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.110+0000 7f1d6f400640 1 -- 172.21.15.152:0/3611710591 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d60031c00 con 0x7f1d7015b510 2024-04-20T11:33:42.111 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.111+0000 7f1d77000640 1 -- 172.21.15.152:0/3611710591 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d7015b510 msgr2=0x7f1d7015fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:42.112 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.111+0000 7f1d77000640 1 --2- 172.21.15.152:0/3611710591 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d7015b510 0x7f1d7015fc30 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f1d600097a0 tx=0x7f1d6002b8d0 comp rx=0 tx=0).stop 2024-04-20T11:33:42.112 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.111+0000 7f1d77000640 1 -- 172.21.15.152:0/3611710591 shutdown_connections 2024-04-20T11:33:42.112 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.111+0000 7f1d77000640 1 --2- 172.21.15.152:0/3611710591 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1d700cfc40 0x7f1d700b3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:42.112 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.111+0000 7f1d77000640 1 --2- 172.21.15.152:0/3611710591 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d7015b510 0x7f1d7015fc30 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:42.112 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.111+0000 7f1d77000640 1 -- 172.21.15.152:0/3611710591 >> 172.21.15.152:0/3611710591 conn(0x7f1d7006b980 msgr2=0x7f1d7006bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:42.112 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.111+0000 7f1d77000640 1 -- 172.21.15.152:0/3611710591 shutdown_connections 2024-04-20T11:33:42.112 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.111+0000 7f1d77000640 1 -- 172.21.15.152:0/3611710591 wait complete. 2024-04-20T11:33:42.113 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.113+0000 7f1d77000640 1 Processor -- start 2024-04-20T11:33:42.113 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.113+0000 7f1d77000640 1 -- start start 2024-04-20T11:33:42.114 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.114+0000 7f1d77000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d700b2170 0x7f1d7014a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:42.114 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.114+0000 7f1d77000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1d700cfc40 0x7f1d7015ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:42.114 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.114+0000 7f1d77000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d7015b510 0x7f1d7014fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:42.114 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.114+0000 7f1d77000640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f1d7015b8f0 con 0x7f1d700b2170 2024-04-20T11:33:42.114 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.114+0000 7f1d77000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1d7015f710 con 0x7f1d7015b510 2024-04-20T11:33:42.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.114+0000 7f1d77000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1d7006f950 con 0x7f1d700cfc40 2024-04-20T11:33:42.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.114+0000 7f1d6fe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d700b2170 0x7f1d7014a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:42.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.114+0000 7f1d6fe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d700b2170 0x7f1d7014a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:55526/0 (socket says 172.21.15.152:55526) 2024-04-20T11:33:42.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.114+0000 7f1d6fe00640 1 -- 172.21.15.152:0/3694756384 learned_addr learned my addr 172.21.15.152:0/3694756384 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:33:42.116 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.114+0000 7f1d67400640 1 --2- 172.21.15.152:0/3694756384 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1d700cfc40 0x7f1d7015ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:42.116 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.114+0000 7f1d6fe00640 1 -- 172.21.15.152:0/3694756384 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1d700cfc40 msgr2=0x7f1d7015ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:42.116 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.114+0000 7f1d6fe00640 1 --2- 172.21.15.152:0/3694756384 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1d700cfc40 0x7f1d7015ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:42.116 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.114+0000 7f1d6fe00640 1 -- 172.21.15.152:0/3694756384 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d7015b510 msgr2=0x7f1d7014fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:42.116 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.114+0000 7f1d6fe00640 1 --2- 172.21.15.152:0/3694756384 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d7015b510 0x7f1d7014fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:42.116 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.114+0000 7f1d6fe00640 1 -- 172.21.15.152:0/3694756384 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d700b0150 con 0x7f1d700b2170 2024-04-20T11:33:42.116 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.114+0000 7f1d67400640 1 --2- 172.21.15.152:0/3694756384 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1d700cfc40 0x7f1d7015ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:33:42.116 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.115+0000 7f1d6fe00640 1 --2- 172.21.15.152:0/3694756384 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d700b2170 0x7f1d7014a520 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f1d60000c00 tx=0x7f1d600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:42.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.115+0000 7f1d6cc00640 1 -- 172.21.15.152:0/3694756384 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d60035040 con 0x7f1d700b2170 2024-04-20T11:33:42.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.115+0000 7f1d6cc00640 1 -- 172.21.15.152:0/3694756384 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d60004250 con 0x7f1d700b2170 2024-04-20T11:33:42.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.115+0000 7f1d6cc00640 1 -- 172.21.15.152:0/3694756384 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d6003a700 con 0x7f1d700b2170 2024-04-20T11:33:42.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.115+0000 7f1d77000640 1 -- 172.21.15.152:0/3694756384 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d700b0430 con 0x7f1d700b2170 2024-04-20T11:33:42.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.115+0000 7f1d77000640 1 -- 172.21.15.152:0/3694756384 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d70147160 con 0x7f1d700b2170 2024-04-20T11:33:42.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.117+0000 7f1d77000640 1 -- 172.21.15.152:0/3694756384 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d700689f0 con 0x7f1d700b2170 2024-04-20T11:33:42.118 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.118+0000 7f1d6cc00640 1 -- 172.21.15.152:0/3694756384 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 91142+0+0 (secure 0 0 0) 0x7f1d60038020 con 0x7f1d700b2170 2024-04-20T11:33:42.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.118+0000 7f1d6cc00640 1 --2- 172.21.15.152:0/3694756384 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f1d4c06ca10 0x7f1d4c06eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:42.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.119+0000 7f1d6cc00640 1 -- 172.21.15.152:0/3694756384 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7f1d600ada80 con 0x7f1d700b2170 2024-04-20T11:33:42.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.119+0000 7f1d67400640 1 --2- 172.21.15.152:0/3694756384 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f1d4c06ca10 0x7f1d4c06eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:42.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.120+0000 7f1d67400640 1 --2- 172.21.15.152:0/3694756384 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f1d4c06ca10 0x7f1d4c06eed0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f1d7014ad60 tx=0x7f1d58008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:42.124 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.123+0000 7f1d6cc00640 1 -- 172.21.15.152:0/3694756384 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f1d6007bdd0 con 0x7f1d700b2170 2024-04-20T11:33:42.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.351+0000 7f1d77000640 1 -- 172.21.15.152:0/3694756384 --> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""]}) -- 0x7f1d701507e0 con 0x7f1d4c06ca10 2024-04-20T11:33:42.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.361+0000 7f1d6cc00640 1 -- 172.21.15.152:0/3694756384 <== mgr.4109 v2:172.21.15.175:6836/4021250441 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+46 (secure 0 0 0) 0x7f1d701507e0 con 0x7f1d4c06ca10 2024-04-20T11:33:42.362 INFO:teuthology.orchestra.run.smithi152.stdout:[ 2024-04-20T11:33:42.362 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:33:42.362 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "subvol48180" 2024-04-20T11:33:42.362 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:33:42.362 INFO:teuthology.orchestra.run.smithi152.stdout:] 2024-04-20T11:33:42.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.363+0000 7f1d77000640 1 -- 172.21.15.152:0/3694756384 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f1d4c06ca10 msgr2=0x7f1d4c06eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:42.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.363+0000 7f1d77000640 1 --2- 172.21.15.152:0/3694756384 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f1d4c06ca10 0x7f1d4c06eed0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f1d7014ad60 tx=0x7f1d58008040 comp rx=0 tx=0).stop 2024-04-20T11:33:42.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.363+0000 7f1d77000640 1 -- 172.21.15.152:0/3694756384 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d700b2170 msgr2=0x7f1d7014a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:42.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.363+0000 7f1d77000640 1 --2- 172.21.15.152:0/3694756384 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d700b2170 0x7f1d7014a520 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f1d60000c00 tx=0x7f1d600046c0 comp rx=0 tx=0).stop 2024-04-20T11:33:42.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.364+0000 7f1d77000640 1 -- 172.21.15.152:0/3694756384 shutdown_connections 2024-04-20T11:33:42.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.364+0000 7f1d77000640 1 --2- 172.21.15.152:0/3694756384 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f1d4c06ca10 0x7f1d4c06eed0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:42.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.364+0000 7f1d77000640 1 --2- 172.21.15.152:0/3694756384 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d7015b510 0x7f1d7014fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:42.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.364+0000 7f1d77000640 1 --2- 172.21.15.152:0/3694756384 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1d700cfc40 0x7f1d7015ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:42.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.364+0000 7f1d77000640 1 --2- 172.21.15.152:0/3694756384 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d700b2170 0x7f1d7014a520 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:42.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.364+0000 7f1d77000640 1 -- 172.21.15.152:0/3694756384 >> 172.21.15.152:0/3694756384 conn(0x7f1d7006b980 msgr2=0x7f1d70154850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:42.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.364+0000 7f1d77000640 1 -- 172.21.15.152:0/3694756384 shutdown_connections 2024-04-20T11:33:42.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.364+0000 7f1d77000640 1 -- 172.21.15.152:0/3694756384 wait complete. 2024-04-20T11:33:42.382 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume info cephfs subvol48180 2024-04-20T11:33:42.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.588+0000 7fdab4a00640 1 Processor -- start 2024-04-20T11:33:42.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.588+0000 7fdab4a00640 1 -- start start 2024-04-20T11:33:42.590 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.590+0000 7fdab4a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdab00cd8c0 0x7fdab00b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:42.590 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.590+0000 7fdab4a00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdab01463e0 con 0x7fdab00b2880 2024-04-20T11:33:42.590 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.590+0000 7fdab4a00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdab00b3e20 con 0x7fdab0146000 2024-04-20T11:33:42.590 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.590+0000 7fdab4a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fdab00b4750 con 0x7fdab00cd8c0 2024-04-20T11:33:42.590 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.590+0000 7fdaad400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fdab00b2880 0x7fdab00cd1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:58088/0 (socket says 172.21.15.152:58088) 2024-04-20T11:33:42.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.590+0000 7fdaad400640 1 -- 172.21.15.152:0/3580458684 learned_addr learned my addr 172.21.15.152:0/3580458684 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:33:42.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.590+0000 7fdaae800640 1 --2- 172.21.15.152:0/3580458684 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdab00cd8c0 0x7fdab00b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:42.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.590+0000 7fdaaca00640 1 -- 172.21.15.152:0/3580458684 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3626005757 0 0) 0x7fdab01463e0 con 0x7fdab00b2880 2024-04-20T11:33:42.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.591+0000 7fdaaca00640 1 -- 172.21.15.152:0/3580458684 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fda94003520 con 0x7fdab00b2880 2024-04-20T11:33:42.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.591+0000 7fdaaca00640 1 -- 172.21.15.152:0/3580458684 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3199225134 0 0) 0x7fda94003520 con 0x7fdab00b2880 2024-04-20T11:33:42.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.591+0000 7fdaaca00640 1 -- 172.21.15.152:0/3580458684 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdab00cd8c0 msgr2=0x7fdab00b4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:42.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.591+0000 7fdaaca00640 1 --2- 172.21.15.152:0/3580458684 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdab00cd8c0 0x7fdab00b4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:42.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.591+0000 7fdaaca00640 1 -- 172.21.15.152:0/3580458684 >> v1:172.21.15.175:6789/0 conn(0x7fdab0146000 legacy=0x7fdab00b2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:42.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.591+0000 7fdaaca00640 1 -- 172.21.15.152:0/3580458684 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdab00b4100 con 0x7fdab00b2880 2024-04-20T11:33:42.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.591+0000 7fdaaca00640 1 -- 172.21.15.152:0/3580458684 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7fda9c001690 con 0x7fdab00b2880 2024-04-20T11:33:42.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.591+0000 7fdaaca00640 1 -- 172.21.15.152:0/3580458684 >> v1:172.21.15.152:6789/0 conn(0x7fdab00b2880 legacy=0x7fdab00cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:42.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.591+0000 7fdaaca00640 1 --2- 172.21.15.152:0/3580458684 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fda940039e0 0x7fda94003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:42.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.591+0000 7fdaaca00640 1 --2- 172.21.15.152:0/3580458684 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdab00cd8c0 0x7fda94004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:42.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.591+0000 7fdaaca00640 1 --2- 172.21.15.152:0/3580458684 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda94004d40 0x7fda940051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:42.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.591+0000 7fdaaca00640 1 -- 172.21.15.152:0/3580458684 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fdab00b4750 con 0x7fda940039e0 2024-04-20T11:33:42.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.591+0000 7fdaaca00640 1 -- 172.21.15.152:0/3580458684 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fda9400bb80 con 0x7fda94004d40 2024-04-20T11:33:42.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.591+0000 7fdaaca00640 1 -- 172.21.15.152:0/3580458684 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fda9400bd80 con 0x7fdab00cd8c0 2024-04-20T11:33:42.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.591+0000 7fdaade00640 1 --2- 172.21.15.152:0/3580458684 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fda940039e0 0x7fda94003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:42.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.592+0000 7fdaad400640 1 --2- 172.21.15.152:0/3580458684 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdab00cd8c0 0x7fda94004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:42.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.592+0000 7fdaae800640 1 --2- 172.21.15.152:0/3580458684 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda94004d40 0x7fda940051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:42.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.592+0000 7fdaade00640 1 -- 172.21.15.152:0/3580458684 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdab00cd8c0 msgr2=0x7fda94004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:42.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.592+0000 7fdaade00640 1 --2- 172.21.15.152:0/3580458684 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdab00cd8c0 0x7fda94004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:42.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.592+0000 7fdaade00640 1 -- 172.21.15.152:0/3580458684 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda94004d40 msgr2=0x7fda940051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:42.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.592+0000 7fdaade00640 1 --2- 172.21.15.152:0/3580458684 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda94004d40 0x7fda940051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:42.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.592+0000 7fdaade00640 1 -- 172.21.15.152:0/3580458684 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda94003520 con 0x7fda940039e0 2024-04-20T11:33:42.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.593+0000 7fdaade00640 1 --2- 172.21.15.152:0/3580458684 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fda940039e0 0x7fda94003e20 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7fdaa0001750 tx=0x7fda94005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd4f3b6cc453ca7a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:42.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.593+0000 7fdaaca00640 1 -- 172.21.15.152:0/3580458684 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa0001370 con 0x7fda940039e0 2024-04-20T11:33:42.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.593+0000 7fdaaca00640 1 -- 172.21.15.152:0/3580458684 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdaa00052d0 con 0x7fda940039e0 2024-04-20T11:33:42.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.593+0000 7fdaaca00640 1 -- 172.21.15.152:0/3580458684 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa0003830 con 0x7fda940039e0 2024-04-20T11:33:42.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.594+0000 7fdab4a00640 1 -- 172.21.15.152:0/3580458684 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fda940039e0 msgr2=0x7fda94003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:42.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.594+0000 7fdab4a00640 1 --2- 172.21.15.152:0/3580458684 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fda940039e0 0x7fda94003e20 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7fdaa0001750 tx=0x7fda94005830 comp rx=0 tx=0).stop 2024-04-20T11:33:42.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.594+0000 7fdaae800640 1 -- 172.21.15.152:0/3580458684 reap_dead start 2024-04-20T11:33:42.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.594+0000 7fdab4a00640 1 -- 172.21.15.152:0/3580458684 shutdown_connections 2024-04-20T11:33:42.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.594+0000 7fdab4a00640 1 -- 172.21.15.152:0/3580458684 >> 172.21.15.152:0/3580458684 conn(0x7fdab006b980 msgr2=0x7fdab006bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:42.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.594+0000 7fdab4a00640 1 -- 172.21.15.152:0/3580458684 shutdown_connections 2024-04-20T11:33:42.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.594+0000 7fdab4a00640 1 -- 172.21.15.152:0/3580458684 wait complete. 2024-04-20T11:33:42.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.596+0000 7fdab4a00640 1 Processor -- start 2024-04-20T11:33:42.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.596+0000 7fdab4a00640 1 -- start start 2024-04-20T11:33:42.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.597+0000 7fdab4a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fda940039e0 0x7fdab013e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:42.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.597+0000 7fdab4a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdab013fc00 0x7fdab0143600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:42.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.597+0000 7fdab4a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdab0143b40 0x7fdab015e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:42.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.597+0000 7fdab4a00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fdab00b4d60 con 0x7fda940039e0 2024-04-20T11:33:42.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.597+0000 7fdab4a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdab00b4be0 con 0x7fdab0143b40 2024-04-20T11:33:42.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.597+0000 7fdab4a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fdab00b4ee0 con 0x7fdab013fc00 2024-04-20T11:33:42.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.597+0000 7fdaade00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fda940039e0 0x7fdab013e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:42.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.598+0000 7fdaade00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fda940039e0 0x7fdab013e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:55542/0 (socket says 172.21.15.152:55542) 2024-04-20T11:33:42.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.598+0000 7fdaade00640 1 -- 172.21.15.152:0/3237124092 learned_addr learned my addr 172.21.15.152:0/3237124092 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:33:42.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.598+0000 7fdaad400640 1 --2- 172.21.15.152:0/3237124092 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdab013fc00 0x7fdab0143600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:42.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.598+0000 7fdaae800640 1 --2- 172.21.15.152:0/3237124092 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdab0143b40 0x7fdab015e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:42.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.598+0000 7fdaade00640 1 -- 172.21.15.152:0/3237124092 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdab013fc00 msgr2=0x7fdab0143600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:42.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.598+0000 7fdaade00640 1 --2- 172.21.15.152:0/3237124092 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdab013fc00 0x7fdab0143600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:42.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.598+0000 7fdaade00640 1 -- 172.21.15.152:0/3237124092 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdab0143b40 msgr2=0x7fdab015e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:42.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.598+0000 7fdaade00640 1 --2- 172.21.15.152:0/3237124092 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdab0143b40 0x7fdab015e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:42.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.598+0000 7fdaade00640 1 -- 172.21.15.152:0/3237124092 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdab013e9a0 con 0x7fda940039e0 2024-04-20T11:33:42.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.598+0000 7fdaad400640 1 --2- 172.21.15.152:0/3237124092 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdab013fc00 0x7fdab0143600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:33:42.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.598+0000 7fdaade00640 1 --2- 172.21.15.152:0/3237124092 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fda940039e0 0x7fdab013e460 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7fdaa0008fd0 tx=0x7fdaa0001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:42.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.599+0000 7fdaa6000640 1 -- 172.21.15.152:0/3237124092 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa0041070 con 0x7fda940039e0 2024-04-20T11:33:42.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.599+0000 7fdaa6000640 1 -- 172.21.15.152:0/3237124092 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdaa0005480 con 0x7fda940039e0 2024-04-20T11:33:42.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.599+0000 7fdaa6000640 1 -- 172.21.15.152:0/3237124092 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa0001370 con 0x7fda940039e0 2024-04-20T11:33:42.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.599+0000 7fdab4a00640 1 -- 172.21.15.152:0/3237124092 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdab013ee40 con 0x7fda940039e0 2024-04-20T11:33:42.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.599+0000 7fdab4a00640 1 -- 172.21.15.152:0/3237124092 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdab013f400 con 0x7fda940039e0 2024-04-20T11:33:42.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.600+0000 7fdab4a00640 1 -- 172.21.15.152:0/3237124092 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdab00cec70 con 0x7fda940039e0 2024-04-20T11:33:42.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.601+0000 7fdaa6000640 1 -- 172.21.15.152:0/3237124092 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 91142+0+0 (secure 0 0 0) 0x7fdaa0044070 con 0x7fda940039e0 2024-04-20T11:33:42.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.602+0000 7fdaa6000640 1 --2- 172.21.15.152:0/3237124092 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fda9006ca70 0x7fda9006ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:42.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.602+0000 7fdaa6000640 1 -- 172.21.15.152:0/3237124092 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7fdaa00ad8f0 con 0x7fda940039e0 2024-04-20T11:33:42.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.603+0000 7fdaad400640 1 --2- 172.21.15.152:0/3237124092 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fda9006ca70 0x7fda9006ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:42.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.604+0000 7fdaad400640 1 --2- 172.21.15.152:0/3237124092 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fda9006ca70 0x7fda9006ef30 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fda9c001eb0 tx=0x7fda9c001170 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:42.607 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.607+0000 7fdaa6000640 1 -- 172.21.15.152:0/3237124092 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fdab00cec70 con 0x7fda940039e0 2024-04-20T11:33:42.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.837+0000 7fdab4a00640 1 -- 172.21.15.152:0/3237124092 --> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] -- mgr_command(tid 0: {"prefix": "fs subvolume info", "vol_name": "cephfs", "sub_name": "subvol48180", "target": ["mon-mgr", ""]}) -- 0x7fdab013ebf0 con 0x7fda9006ca70 2024-04-20T11:33:42.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.869+0000 7fdaa6000640 1 -- 172.21.15.152:0/3237124092 <== mgr.4109 v2:172.21.15.175:6836/4021250441 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+687 (secure 0 0 0) 0x7fdab013ebf0 con 0x7fda9006ca70 2024-04-20T11:33:42.870 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-04-20T11:33:42.870 INFO:teuthology.orchestra.run.smithi152.stdout: "atime": "2024-04-20 11:33:40", 2024-04-20T11:33:42.870 INFO:teuthology.orchestra.run.smithi152.stdout: "bytes_pcent": "undefined", 2024-04-20T11:33:42.870 INFO:teuthology.orchestra.run.smithi152.stdout: "bytes_quota": "infinite", 2024-04-20T11:33:42.871 INFO:teuthology.orchestra.run.smithi152.stdout: "bytes_used": 0, 2024-04-20T11:33:42.871 INFO:teuthology.orchestra.run.smithi152.stdout: "created_at": "2024-04-20 11:33:40", 2024-04-20T11:33:42.871 INFO:teuthology.orchestra.run.smithi152.stdout: "ctime": "2024-04-20 11:33:41", 2024-04-20T11:33:42.871 INFO:teuthology.orchestra.run.smithi152.stdout: "data_pool": "cephfs_data", 2024-04-20T11:33:42.871 INFO:teuthology.orchestra.run.smithi152.stdout: "features": [ 2024-04-20T11:33:42.871 INFO:teuthology.orchestra.run.smithi152.stdout: "snapshot-clone", 2024-04-20T11:33:42.871 INFO:teuthology.orchestra.run.smithi152.stdout: "snapshot-autoprotect" 2024-04-20T11:33:42.871 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:33:42.871 INFO:teuthology.orchestra.run.smithi152.stdout: "flavor": 1, 2024-04-20T11:33:42.871 INFO:teuthology.orchestra.run.smithi152.stdout: "gid": 0, 2024-04-20T11:33:42.871 INFO:teuthology.orchestra.run.smithi152.stdout: "mode": 16877, 2024-04-20T11:33:42.871 INFO:teuthology.orchestra.run.smithi152.stdout: "mon_addrs": [ 2024-04-20T11:33:42.871 INFO:teuthology.orchestra.run.smithi152.stdout: "172.21.15.152:6789", 2024-04-20T11:33:42.872 INFO:teuthology.orchestra.run.smithi152.stdout: "172.21.15.175:6789", 2024-04-20T11:33:42.872 INFO:teuthology.orchestra.run.smithi152.stdout: "172.21.15.175:6790" 2024-04-20T11:33:42.872 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:33:42.872 INFO:teuthology.orchestra.run.smithi152.stdout: "mtime": "2024-04-20 11:33:40", 2024-04-20T11:33:42.872 INFO:teuthology.orchestra.run.smithi152.stdout: "path": "/volumes/_nogroup/subvol48180/bf9c0317-2e4a-4eaf-b73e-875a6eb33282", 2024-04-20T11:33:42.872 INFO:teuthology.orchestra.run.smithi152.stdout: "pool_namespace": "", 2024-04-20T11:33:42.872 INFO:teuthology.orchestra.run.smithi152.stdout: "state": "complete", 2024-04-20T11:33:42.872 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "subvolume", 2024-04-20T11:33:42.872 INFO:teuthology.orchestra.run.smithi152.stdout: "uid": 0 2024-04-20T11:33:42.872 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-04-20T11:33:42.873 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.872+0000 7fdab4a00640 1 -- 172.21.15.152:0/3237124092 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fda9006ca70 msgr2=0x7fda9006ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:42.873 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.872+0000 7fdab4a00640 1 --2- 172.21.15.152:0/3237124092 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fda9006ca70 0x7fda9006ef30 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fda9c001eb0 tx=0x7fda9c001170 comp rx=0 tx=0).stop 2024-04-20T11:33:42.873 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.872+0000 7fdab4a00640 1 -- 172.21.15.152:0/3237124092 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fda940039e0 msgr2=0x7fdab013e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:42.873 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.872+0000 7fdab4a00640 1 --2- 172.21.15.152:0/3237124092 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fda940039e0 0x7fdab013e460 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7fdaa0008fd0 tx=0x7fdaa0001120 comp rx=0 tx=0).stop 2024-04-20T11:33:42.873 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.872+0000 7fdab4a00640 1 -- 172.21.15.152:0/3237124092 shutdown_connections 2024-04-20T11:33:42.873 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.872+0000 7fdab4a00640 1 --2- 172.21.15.152:0/3237124092 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fda9006ca70 0x7fda9006ef30 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:42.873 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.872+0000 7fdab4a00640 1 --2- 172.21.15.152:0/3237124092 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdab0143b40 0x7fdab015e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:42.873 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.872+0000 7fdab4a00640 1 --2- 172.21.15.152:0/3237124092 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdab013fc00 0x7fdab0143600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:42.873 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.872+0000 7fdab4a00640 1 --2- 172.21.15.152:0/3237124092 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fda940039e0 0x7fdab013e460 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:42.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.872+0000 7fdab4a00640 1 -- 172.21.15.152:0/3237124092 >> 172.21.15.152:0/3237124092 conn(0x7fdab006b980 msgr2=0x7fdab00ad8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:42.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.873+0000 7fdab4a00640 1 -- 172.21.15.152:0/3237124092 shutdown_connections 2024-04-20T11:33:42.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:42.873+0000 7fdab4a00640 1 -- 172.21.15.152:0/3237124092 wait complete. 2024-04-20T11:33:42.892 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume resize cephfs subvol48180 10485760 2024-04-20T11:33:43.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.100+0000 7f2416a00640 1 Processor -- start 2024-04-20T11:33:43.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.100+0000 7f2416a00640 1 -- start start 2024-04-20T11:33:43.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.101+0000 7f2416a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f24100cf220 0x7f24100b4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:43.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.101+0000 7f2416a00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f2410144d70 0x7f24100b4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:43.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.101+0000 7f2416a00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24100cf6c0 con 0x7f24100cd8e0 2024-04-20T11:33:43.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.101+0000 7f2416a00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f24100d0240 con 0x7f24100cf220 2024-04-20T11:33:43.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.101+0000 7f2416a00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f24100cf9a0 con 0x7f2410144d70 2024-04-20T11:33:43.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.102+0000 7f2407400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f24100cf220 0x7f24100b4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:43.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.102+0000 7f2407400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f24100cf220 0x7f24100b4680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:55552/0 (socket says 172.21.15.152:55552) 2024-04-20T11:33:43.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.102+0000 7f2407400640 1 -- 172.21.15.152:0/751421163 learned_addr learned my addr 172.21.15.152:0/751421163 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:33:43.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.102+0000 7f2414a00640 1 --2- 172.21.15.152:0/751421163 >> v2:172.21.15.175:3300/0 conn(0x7f2410144d70 0x7f24100b4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:43.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.102+0000 7f2407400640 1 -- 172.21.15.152:0/751421163 >> v2:172.21.15.175:3300/0 conn(0x7f2410144d70 msgr2=0x7f24100b4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:43.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.102+0000 7f2407400640 1 --2- 172.21.15.152:0/751421163 >> v2:172.21.15.175:3300/0 conn(0x7f2410144d70 0x7f24100b4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:43.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.102+0000 7f2407400640 1 -- 172.21.15.152:0/751421163 >> v1:172.21.15.175:6789/0 conn(0x7f24100cd8e0 legacy=0x7f241006e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:43.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.102+0000 7f2407400640 1 -- 172.21.15.152:0/751421163 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24100cfc80 con 0x7f24100cf220 2024-04-20T11:33:43.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.104+0000 7f2407400640 1 --2- 172.21.15.152:0/751421163 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f24100cf220 0x7f24100b4680 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f241006e220 tx=0x7f23f802e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c49eeb30a84d27b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:43.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.104+0000 7f240f400640 1 -- 172.21.15.152:0/751421163 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f803b070 con 0x7f24100cf220 2024-04-20T11:33:43.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.104+0000 7f240f400640 1 -- 172.21.15.152:0/751421163 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23f8036440 con 0x7f24100cf220 2024-04-20T11:33:43.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.104+0000 7f240f400640 1 -- 172.21.15.152:0/751421163 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f8036810 con 0x7f24100cf220 2024-04-20T11:33:43.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.105+0000 7f2416a00640 1 -- 172.21.15.152:0/751421163 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f24100cf220 msgr2=0x7f24100b4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:43.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.105+0000 7f2416a00640 1 --2- 172.21.15.152:0/751421163 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f24100cf220 0x7f24100b4680 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f241006e220 tx=0x7f23f802e0e0 comp rx=0 tx=0).stop 2024-04-20T11:33:43.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.105+0000 7f2416a00640 1 -- 172.21.15.152:0/751421163 shutdown_connections 2024-04-20T11:33:43.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.105+0000 7f2416a00640 1 --2- 172.21.15.152:0/751421163 >> v2:172.21.15.175:3300/0 conn(0x7f2410144d70 0x7f24100b4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:43.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.105+0000 7f2416a00640 1 --2- 172.21.15.152:0/751421163 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f24100cf220 0x7f24100b4680 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:43.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.105+0000 7f2416a00640 1 -- 172.21.15.152:0/751421163 >> 172.21.15.152:0/751421163 conn(0x7f241006b980 msgr2=0x7f241006bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:43.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.105+0000 7f2416a00640 1 -- 172.21.15.152:0/751421163 shutdown_connections 2024-04-20T11:33:43.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.105+0000 7f2416a00640 1 -- 172.21.15.152:0/751421163 wait complete. 2024-04-20T11:33:43.107 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.107+0000 7f2416a00640 1 Processor -- start 2024-04-20T11:33:43.107 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.107+0000 7f2416a00640 1 -- start start 2024-04-20T11:33:43.108 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.108+0000 7f2416a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f24100cd8e0 0x7f24100c9f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:43.108 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.108+0000 7f2416a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f24100cf220 0x7f2410141c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:43.108 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.108+0000 7f2416a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2410144d70 0x7f24101405a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:43.109 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.108+0000 7f2416a00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f24101447b0 con 0x7f24100cf220 2024-04-20T11:33:43.109 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.108+0000 7f2416a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f24100cdcc0 con 0x7f2410144d70 2024-04-20T11:33:43.109 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.108+0000 7f2416a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2410144930 con 0x7f24100cd8e0 2024-04-20T11:33:43.109 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.108+0000 7f2407400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f24100cf220 0x7f2410141c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:43.109 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.108+0000 7f2407400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f24100cf220 0x7f2410141c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:55568/0 (socket says 172.21.15.152:55568) 2024-04-20T11:33:43.109 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.108+0000 7f2407400640 1 -- 172.21.15.152:0/3042400998 learned_addr learned my addr 172.21.15.152:0/3042400998 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:33:43.110 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.108+0000 7f2414a00640 1 --2- 172.21.15.152:0/3042400998 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2410144d70 0x7f24101405a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:43.110 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.108+0000 7f240fe00640 1 --2- 172.21.15.152:0/3042400998 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f24100cd8e0 0x7f24100c9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:43.110 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.108+0000 7f2407400640 1 -- 172.21.15.152:0/3042400998 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f24100cd8e0 msgr2=0x7f24100c9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:43.111 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.108+0000 7f2407400640 1 --2- 172.21.15.152:0/3042400998 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f24100cd8e0 0x7f24100c9f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:43.111 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.108+0000 7f2407400640 1 -- 172.21.15.152:0/3042400998 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2410144d70 msgr2=0x7f24101405a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:43.111 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.108+0000 7f2407400640 1 --2- 172.21.15.152:0/3042400998 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2410144d70 0x7f24101405a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:43.111 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.109+0000 7f2407400640 1 -- 172.21.15.152:0/3042400998 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2410140ae0 con 0x7f24100cf220 2024-04-20T11:33:43.111 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.109+0000 7f2407400640 1 --2- 172.21.15.152:0/3042400998 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f24100cf220 0x7f2410141c50 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f23f802e120 tx=0x7f23f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:43.111 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.109+0000 7f240cc00640 1 -- 172.21.15.152:0/3042400998 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f8046070 con 0x7f24100cf220 2024-04-20T11:33:43.111 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.109+0000 7f240cc00640 1 -- 172.21.15.152:0/3042400998 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23f803d050 con 0x7f24100cf220 2024-04-20T11:33:43.111 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.109+0000 7f240cc00640 1 -- 172.21.15.152:0/3042400998 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f803b030 con 0x7f24100cf220 2024-04-20T11:33:43.111 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.109+0000 7f2416a00640 1 -- 172.21.15.152:0/3042400998 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24100ca590 con 0x7f24100cf220 2024-04-20T11:33:43.111 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.109+0000 7f2416a00640 1 -- 172.21.15.152:0/3042400998 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24100cab50 con 0x7f24100cf220 2024-04-20T11:33:43.112 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.111+0000 7f2416a00640 1 -- 172.21.15.152:0/3042400998 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2410068a60 con 0x7f24100cf220 2024-04-20T11:33:43.112 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.112+0000 7f240cc00640 1 -- 172.21.15.152:0/3042400998 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 91142+0+0 (secure 0 0 0) 0x7f23f8034910 con 0x7f24100cf220 2024-04-20T11:33:43.113 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.112+0000 7f240cc00640 1 --2- 172.21.15.152:0/3042400998 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f23ec06ca10 0x7f23ec06eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:43.113 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.113+0000 7f240cc00640 1 -- 172.21.15.152:0/3042400998 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7f23f80ad8a0 con 0x7f24100cf220 2024-04-20T11:33:43.113 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.113+0000 7f240fe00640 1 --2- 172.21.15.152:0/3042400998 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f23ec06ca10 0x7f23ec06eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:43.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.114+0000 7f240fe00640 1 --2- 172.21.15.152:0/3042400998 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f23ec06ca10 0x7f23ec06eed0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2400001500 tx=0x7f2400009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:43.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.117+0000 7f240cc00640 1 -- 172.21.15.152:0/3042400998 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f23f807bbc0 con 0x7f24100cf220 2024-04-20T11:33:43.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.355+0000 7f2416a00640 1 -- 172.21.15.152:0/3042400998 --> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] -- mgr_command(tid 0: {"prefix": "fs subvolume resize", "vol_name": "cephfs", "sub_name": "subvol48180", "new_size": "10485760", "target": ["mon-mgr", ""]}) -- 0x7f241015e770 con 0x7f23ec06ca10 2024-04-20T11:33:43.393 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.392+0000 7f240cc00640 1 -- 172.21.15.152:0/3042400998 <== mgr.4109 v2:172.21.15.175:6836/4021250441 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+128 (secure 0 0 0) 0x7f241015e770 con 0x7f23ec06ca10 2024-04-20T11:33:43.393 INFO:teuthology.orchestra.run.smithi152.stdout:[ 2024-04-20T11:33:43.393 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:33:43.393 INFO:teuthology.orchestra.run.smithi152.stdout: "bytes_used": 0 2024-04-20T11:33:43.394 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:33:43.394 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:33:43.394 INFO:teuthology.orchestra.run.smithi152.stdout: "bytes_quota": 10485760 2024-04-20T11:33:43.394 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:33:43.394 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:33:43.394 INFO:teuthology.orchestra.run.smithi152.stdout: "bytes_pcent": "0.00" 2024-04-20T11:33:43.394 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:33:43.394 INFO:teuthology.orchestra.run.smithi152.stdout:] 2024-04-20T11:33:43.396 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.395+0000 7f2416a00640 1 -- 172.21.15.152:0/3042400998 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f23ec06ca10 msgr2=0x7f23ec06eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:43.396 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.395+0000 7f2416a00640 1 --2- 172.21.15.152:0/3042400998 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f23ec06ca10 0x7f23ec06eed0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2400001500 tx=0x7f2400009040 comp rx=0 tx=0).stop 2024-04-20T11:33:43.397 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.395+0000 7f2416a00640 1 -- 172.21.15.152:0/3042400998 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f24100cf220 msgr2=0x7f2410141c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:43.397 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.395+0000 7f2416a00640 1 --2- 172.21.15.152:0/3042400998 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f24100cf220 0x7f2410141c50 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f23f802e120 tx=0x7f23f80046c0 comp rx=0 tx=0).stop 2024-04-20T11:33:43.397 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.395+0000 7f2416a00640 1 -- 172.21.15.152:0/3042400998 shutdown_connections 2024-04-20T11:33:43.397 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.395+0000 7f2416a00640 1 --2- 172.21.15.152:0/3042400998 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f23ec06ca10 0x7f23ec06eed0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:43.397 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.395+0000 7f2416a00640 1 --2- 172.21.15.152:0/3042400998 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2410144d70 0x7f24101405a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:43.397 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.395+0000 7f2416a00640 1 --2- 172.21.15.152:0/3042400998 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f24100cf220 0x7f2410141c50 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:43.397 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.395+0000 7f2416a00640 1 --2- 172.21.15.152:0/3042400998 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f24100cd8e0 0x7f24100c9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:43.397 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.395+0000 7f2416a00640 1 -- 172.21.15.152:0/3042400998 >> 172.21.15.152:0/3042400998 conn(0x7f241006b980 msgr2=0x7f241015d880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:43.397 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.395+0000 7f2416a00640 1 -- 172.21.15.152:0/3042400998 shutdown_connections 2024-04-20T11:33:43.397 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.396+0000 7f2416a00640 1 -- 172.21.15.152:0/3042400998 wait complete. 2024-04-20T11:33:43.415 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume info cephfs subvol48180 2024-04-20T11:33:43.614 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.614+0000 7f9e7be00640 1 Processor -- start 2024-04-20T11:33:43.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.614+0000 7f9e7be00640 1 -- start start 2024-04-20T11:33:43.616 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.615+0000 7f9e7be00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9e7c15d6d0 0x7f9e7c15db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:43.616 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.615+0000 7f9e7be00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f9e7c1601f0 0x7f9e7c160680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:43.616 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.615+0000 7f9e7be00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e7c15e270 con 0x7f9e7c1460b0 2024-04-20T11:33:43.616 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.616+0000 7f9e7be00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f9e7c15e550 con 0x7f9e7c1601f0 2024-04-20T11:33:43.616 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.616+0000 7f9e7be00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f9e7c15e830 con 0x7f9e7c15d6d0 2024-04-20T11:33:43.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.616+0000 7f9e7a000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9e7c15d6d0 0x7f9e7c15db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:43.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.616+0000 7f9e7a000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9e7c15d6d0 0x7f9e7c15db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.152:47558/0 (socket says 172.21.15.152:47558) 2024-04-20T11:33:43.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.616+0000 7f9e7a000640 1 -- 172.21.15.152:0/3601054327 learned_addr learned my addr 172.21.15.152:0/3601054327 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:33:43.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.617+0000 7f9e7b400640 1 --2- 172.21.15.152:0/3601054327 >> v2:172.21.15.175:3300/0 conn(0x7f9e7c1601f0 0x7f9e7c160680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:43.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.617+0000 7f9e7a000640 1 -- 172.21.15.152:0/3601054327 >> v2:172.21.15.175:3300/0 conn(0x7f9e7c1601f0 msgr2=0x7f9e7c160680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:43.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.617+0000 7f9e79600640 1 -- 172.21.15.152:0/3601054327 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1826273406 0 0) 0x7f9e7c15e270 con 0x7f9e7c1460b0 2024-04-20T11:33:43.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.617+0000 7f9e7a000640 1 --2- 172.21.15.152:0/3601054327 >> v2:172.21.15.175:3300/0 conn(0x7f9e7c1601f0 0x7f9e7c160680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:43.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.617+0000 7f9e7a000640 1 -- 172.21.15.152:0/3601054327 >> v1:172.21.15.175:6789/0 conn(0x7f9e7c1460b0 legacy=0x7f9e7c0af930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:43.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.617+0000 7f9e7a000640 1 -- 172.21.15.152:0/3601054327 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e7c15eb10 con 0x7f9e7c15d6d0 2024-04-20T11:33:43.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.619+0000 7f9e7a000640 1 --2- 172.21.15.152:0/3601054327 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9e7c15d6d0 0x7f9e7c15db30 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c009e50 tx=0x7f9e6c02bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8365fa1aaa782d7c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:43.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.619+0000 7f9e79600640 1 -- 172.21.15.152:0/3601054327 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e6c02cb00 con 0x7f9e7c15d6d0 2024-04-20T11:33:43.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.619+0000 7f9e79600640 1 -- 172.21.15.152:0/3601054327 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e6c02cca0 con 0x7f9e7c15d6d0 2024-04-20T11:33:43.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.619+0000 7f9e79600640 1 -- 172.21.15.152:0/3601054327 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e6c034610 con 0x7f9e7c15d6d0 2024-04-20T11:33:43.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.620+0000 7f9e7be00640 1 -- 172.21.15.152:0/3601054327 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9e7c15d6d0 msgr2=0x7f9e7c15db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:43.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.620+0000 7f9e7be00640 1 --2- 172.21.15.152:0/3601054327 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9e7c15d6d0 0x7f9e7c15db30 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c009e50 tx=0x7f9e6c02bf60 comp rx=0 tx=0).stop 2024-04-20T11:33:43.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.620+0000 7f9e7be00640 1 -- 172.21.15.152:0/3601054327 shutdown_connections 2024-04-20T11:33:43.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.620+0000 7f9e7be00640 1 --2- 172.21.15.152:0/3601054327 >> v2:172.21.15.175:3300/0 conn(0x7f9e7c1601f0 0x7f9e7c160680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:43.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.620+0000 7f9e7be00640 1 --2- 172.21.15.152:0/3601054327 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9e7c15d6d0 0x7f9e7c15db30 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:43.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.620+0000 7f9e7be00640 1 -- 172.21.15.152:0/3601054327 >> 172.21.15.152:0/3601054327 conn(0x7f9e7c0b6790 msgr2=0x7f9e7c0b6ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:43.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.620+0000 7f9e7be00640 1 -- 172.21.15.152:0/3601054327 shutdown_connections 2024-04-20T11:33:43.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.620+0000 7f9e7be00640 1 -- 172.21.15.152:0/3601054327 wait complete. 2024-04-20T11:33:43.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.622+0000 7f9e7be00640 1 Processor -- start 2024-04-20T11:33:43.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.622+0000 7f9e7be00640 1 -- start start 2024-04-20T11:33:43.623 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.623+0000 7f9e7be00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9e7c1460b0 0x7f9e7c0cc3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:43.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.623+0000 7f9e7be00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9e7c15d6d0 0x7f9e7c0cb4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:43.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.623+0000 7f9e7be00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e7c1601f0 0x7f9e7c13dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:43.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.623+0000 7f9e7be00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f9e7c15f310 con 0x7f9e7c15d6d0 2024-04-20T11:33:43.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.623+0000 7f9e7be00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9e7c160fd0 con 0x7f9e7c1601f0 2024-04-20T11:33:43.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.623+0000 7f9e7be00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f9e7c15f490 con 0x7f9e7c1460b0 2024-04-20T11:33:43.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.623+0000 7f9e7a000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9e7c15d6d0 0x7f9e7c0cb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:43.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.623+0000 7f9e7a000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9e7c15d6d0 0x7f9e7c0cb4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:55578/0 (socket says 172.21.15.152:55578) 2024-04-20T11:33:43.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.623+0000 7f9e7a000640 1 -- 172.21.15.152:0/1076734956 learned_addr learned my addr 172.21.15.152:0/1076734956 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:33:43.625 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.623+0000 7f9e7aa00640 1 --2- 172.21.15.152:0/1076734956 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9e7c1460b0 0x7f9e7c0cc3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:43.625 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.624+0000 7f9e7a000640 1 -- 172.21.15.152:0/1076734956 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9e7c1460b0 msgr2=0x7f9e7c0cc3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:43.625 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.624+0000 7f9e7a000640 1 --2- 172.21.15.152:0/1076734956 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9e7c1460b0 0x7f9e7c0cc3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:43.625 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.624+0000 7f9e7a000640 1 -- 172.21.15.152:0/1076734956 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e7c1601f0 msgr2=0x7f9e7c13dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:43.625 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.624+0000 7f9e7a000640 1 --2- 172.21.15.152:0/1076734956 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e7c1601f0 0x7f9e7c13dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:43.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.624+0000 7f9e7a000640 1 -- 172.21.15.152:0/1076734956 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e7c1458e0 con 0x7f9e7c15d6d0 2024-04-20T11:33:43.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.624+0000 7f9e7a000640 1 --2- 172.21.15.152:0/1076734956 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9e7c15d6d0 0x7f9e7c0cb4e0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c036040 tx=0x7f9e6c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:43.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.624+0000 7f9e72a00640 1 -- 172.21.15.152:0/1076734956 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e6c043020 con 0x7f9e7c15d6d0 2024-04-20T11:33:43.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.624+0000 7f9e72a00640 1 -- 172.21.15.152:0/1076734956 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e6c02ce50 con 0x7f9e7c15d6d0 2024-04-20T11:33:43.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.624+0000 7f9e72a00640 1 -- 172.21.15.152:0/1076734956 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e6c009510 con 0x7f9e7c15d6d0 2024-04-20T11:33:43.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.624+0000 7f9e7be00640 1 -- 172.21.15.152:0/1076734956 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e7c0ca720 con 0x7f9e7c15d6d0 2024-04-20T11:33:43.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.625+0000 7f9e7be00640 1 -- 172.21.15.152:0/1076734956 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e7c0cace0 con 0x7f9e7c15d6d0 2024-04-20T11:33:43.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.626+0000 7f9e7be00640 1 -- 172.21.15.152:0/1076734956 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e7c0cb1d0 con 0x7f9e7c15d6d0 2024-04-20T11:33:43.627 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.627+0000 7f9e72a00640 1 -- 172.21.15.152:0/1076734956 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 91142+0+0 (secure 0 0 0) 0x7f9e6c046020 con 0x7f9e7c15d6d0 2024-04-20T11:33:43.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.628+0000 7f9e72a00640 1 --2- 172.21.15.152:0/1076734956 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f9e5806ca40 0x7f9e5806ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:43.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.628+0000 7f9e72a00640 1 -- 172.21.15.152:0/1076734956 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7f9e6c0ae3e0 con 0x7f9e7c15d6d0 2024-04-20T11:33:43.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.628+0000 7f9e7aa00640 1 --2- 172.21.15.152:0/1076734956 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f9e5806ca40 0x7f9e5806ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:43.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.630+0000 7f9e7aa00640 1 --2- 172.21.15.152:0/1076734956 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f9e5806ca40 0x7f9e5806ef00 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9e64000f60 tx=0x7f9e64009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:43.632 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.632+0000 7f9e72a00640 1 -- 172.21.15.152:0/1076734956 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f9e6c04d030 con 0x7f9e7c15d6d0 2024-04-20T11:33:43.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.874+0000 7f9e7be00640 1 -- 172.21.15.152:0/1076734956 --> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] -- mgr_command(tid 0: {"prefix": "fs subvolume info", "vol_name": "cephfs", "sub_name": "subvol48180", "target": ["mon-mgr", ""]}) -- 0x7f9e7c0b39f0 con 0x7f9e5806ca40 2024-04-20T11:33:43.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.913+0000 7f9e72a00640 1 -- 172.21.15.152:0/1076734956 <== mgr.4109 v2:172.21.15.175:6836/4021250441 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+680 (secure 0 0 0) 0x7f9e7c0b39f0 con 0x7f9e5806ca40 2024-04-20T11:33:43.914 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-04-20T11:33:43.915 INFO:teuthology.orchestra.run.smithi152.stdout: "atime": "2024-04-20 11:33:40", 2024-04-20T11:33:43.915 INFO:teuthology.orchestra.run.smithi152.stdout: "bytes_pcent": "0.00", 2024-04-20T11:33:43.915 INFO:teuthology.orchestra.run.smithi152.stdout: "bytes_quota": 10485760, 2024-04-20T11:33:43.915 INFO:teuthology.orchestra.run.smithi152.stdout: "bytes_used": 0, 2024-04-20T11:33:43.915 INFO:teuthology.orchestra.run.smithi152.stdout: "created_at": "2024-04-20 11:33:40", 2024-04-20T11:33:43.915 INFO:teuthology.orchestra.run.smithi152.stdout: "ctime": "2024-04-20 11:33:43", 2024-04-20T11:33:43.915 INFO:teuthology.orchestra.run.smithi152.stdout: "data_pool": "cephfs_data", 2024-04-20T11:33:43.916 INFO:teuthology.orchestra.run.smithi152.stdout: "features": [ 2024-04-20T11:33:43.916 INFO:teuthology.orchestra.run.smithi152.stdout: "snapshot-clone", 2024-04-20T11:33:43.916 INFO:teuthology.orchestra.run.smithi152.stdout: "snapshot-autoprotect" 2024-04-20T11:33:43.916 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:33:43.916 INFO:teuthology.orchestra.run.smithi152.stdout: "flavor": 1, 2024-04-20T11:33:43.917 INFO:teuthology.orchestra.run.smithi152.stdout: "gid": 0, 2024-04-20T11:33:43.917 INFO:teuthology.orchestra.run.smithi152.stdout: "mode": 16877, 2024-04-20T11:33:43.917 INFO:teuthology.orchestra.run.smithi152.stdout: "mon_addrs": [ 2024-04-20T11:33:43.917 INFO:teuthology.orchestra.run.smithi152.stdout: "172.21.15.152:6789", 2024-04-20T11:33:43.917 INFO:teuthology.orchestra.run.smithi152.stdout: "172.21.15.175:6789", 2024-04-20T11:33:43.917 INFO:teuthology.orchestra.run.smithi152.stdout: "172.21.15.175:6790" 2024-04-20T11:33:43.917 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:33:43.918 INFO:teuthology.orchestra.run.smithi152.stdout: "mtime": "2024-04-20 11:33:40", 2024-04-20T11:33:43.918 INFO:teuthology.orchestra.run.smithi152.stdout: "path": "/volumes/_nogroup/subvol48180/bf9c0317-2e4a-4eaf-b73e-875a6eb33282", 2024-04-20T11:33:43.918 INFO:teuthology.orchestra.run.smithi152.stdout: "pool_namespace": "", 2024-04-20T11:33:43.918 INFO:teuthology.orchestra.run.smithi152.stdout: "state": "complete", 2024-04-20T11:33:43.918 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "subvolume", 2024-04-20T11:33:43.918 INFO:teuthology.orchestra.run.smithi152.stdout: "uid": 0 2024-04-20T11:33:43.919 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-04-20T11:33:43.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.916+0000 7f9e7be00640 1 -- 172.21.15.152:0/1076734956 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f9e5806ca40 msgr2=0x7f9e5806ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:43.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.916+0000 7f9e7be00640 1 --2- 172.21.15.152:0/1076734956 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f9e5806ca40 0x7f9e5806ef00 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9e64000f60 tx=0x7f9e64009040 comp rx=0 tx=0).stop 2024-04-20T11:33:43.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.916+0000 7f9e7be00640 1 -- 172.21.15.152:0/1076734956 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9e7c15d6d0 msgr2=0x7f9e7c0cb4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:43.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.916+0000 7f9e7be00640 1 --2- 172.21.15.152:0/1076734956 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9e7c15d6d0 0x7f9e7c0cb4e0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c036040 tx=0x7f9e6c004a40 comp rx=0 tx=0).stop 2024-04-20T11:33:43.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.916+0000 7f9e7be00640 1 -- 172.21.15.152:0/1076734956 shutdown_connections 2024-04-20T11:33:43.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.916+0000 7f9e7be00640 1 --2- 172.21.15.152:0/1076734956 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f9e5806ca40 0x7f9e5806ef00 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:43.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.916+0000 7f9e7be00640 1 --2- 172.21.15.152:0/1076734956 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e7c1601f0 0x7f9e7c13dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:43.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.916+0000 7f9e7be00640 1 --2- 172.21.15.152:0/1076734956 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9e7c15d6d0 0x7f9e7c0cb4e0 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:43.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.916+0000 7f9e7be00640 1 --2- 172.21.15.152:0/1076734956 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9e7c1460b0 0x7f9e7c0cc3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:43.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.916+0000 7f9e7be00640 1 -- 172.21.15.152:0/1076734956 >> 172.21.15.152:0/1076734956 conn(0x7f9e7c0b6790 msgr2=0x7f9e7c0702c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:43.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.916+0000 7f9e7be00640 1 -- 172.21.15.152:0/1076734956 shutdown_connections 2024-04-20T11:33:43.922 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:43.916+0000 7f9e7be00640 1 -- 172.21.15.152:0/1076734956 wait complete. 2024-04-20T11:33:43.937 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol48180 --mode 777 --uid 1000 --gid 1000 2024-04-20T11:33:44.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.142+0000 7f6751a00640 1 Processor -- start 2024-04-20T11:33:44.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.142+0000 7f6751a00640 1 -- start start 2024-04-20T11:33:44.145 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.144+0000 7f6751a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f674c144d60 0x7f674c0cc400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:44.145 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.144+0000 7f6751a00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f674c145140 con 0x7f674c06f510 2024-04-20T11:33:44.145 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.144+0000 7f6751a00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f674c0726b0 con 0x7f674c071cd0 2024-04-20T11:33:44.145 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.144+0000 7f6751a00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f674c072990 con 0x7f674c144d60 2024-04-20T11:33:44.146 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.145+0000 7f674be00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f674c144d60 0x7f674c0cc400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:44.146 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.145+0000 7f6751000640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f674c06f510 0x7f674c0b4500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:58104/0 (socket says 172.21.15.152:58104) 2024-04-20T11:33:44.146 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.145+0000 7f6751000640 1 -- 172.21.15.152:0/1233395560 learned_addr learned my addr 172.21.15.152:0/1233395560 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:33:44.146 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.145+0000 7f674be00640 1 -- 172.21.15.152:0/1233395560 >> v1:172.21.15.175:6789/0 conn(0x7f674c071cd0 legacy=0x7f674c06ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:44.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.145+0000 7f674be00640 1 -- 172.21.15.152:0/1233395560 >> v1:172.21.15.152:6789/0 conn(0x7f674c06f510 legacy=0x7f674c0b4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:44.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.145+0000 7f674be00640 1 -- 172.21.15.152:0/1233395560 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f674c0b3d00 con 0x7f674c144d60 2024-04-20T11:33:44.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.146+0000 7f674be00640 1 --2- 172.21.15.152:0/1233395560 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f674c144d60 0x7f674c0cc400 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f673c0097a0 tx=0x7f673c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fe8d3ed3dc0e212 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:44.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.147+0000 7f674aa00640 1 -- 172.21.15.152:0/1233395560 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f673c0315a0 con 0x7f674c144d60 2024-04-20T11:33:44.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.147+0000 7f674aa00640 1 -- 172.21.15.152:0/1233395560 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f673c037070 con 0x7f674c144d60 2024-04-20T11:33:44.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.147+0000 7f674aa00640 1 -- 172.21.15.152:0/1233395560 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f673c031960 con 0x7f674c144d60 2024-04-20T11:33:44.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.147+0000 7f6751a00640 1 -- 172.21.15.152:0/1233395560 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f674c144d60 msgr2=0x7f674c0cc400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:44.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.147+0000 7f6751a00640 1 --2- 172.21.15.152:0/1233395560 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f674c144d60 0x7f674c0cc400 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f673c0097a0 tx=0x7f673c02b840 comp rx=0 tx=0).stop 2024-04-20T11:33:44.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.148+0000 7f6751a00640 1 -- 172.21.15.152:0/1233395560 shutdown_connections 2024-04-20T11:33:44.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.148+0000 7f6751a00640 1 --2- 172.21.15.152:0/1233395560 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f674c144d60 0x7f674c0cc400 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:44.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.148+0000 7f6751a00640 1 -- 172.21.15.152:0/1233395560 >> 172.21.15.152:0/1233395560 conn(0x7f674c067ae0 msgr2=0x7f674c06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:44.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.148+0000 7f6751a00640 1 -- 172.21.15.152:0/1233395560 shutdown_connections 2024-04-20T11:33:44.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.148+0000 7f6751a00640 1 -- 172.21.15.152:0/1233395560 wait complete. 2024-04-20T11:33:44.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.149+0000 7f6751a00640 1 Processor -- start 2024-04-20T11:33:44.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.149+0000 7f6751a00640 1 -- start start 2024-04-20T11:33:44.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.150+0000 7f6751a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f674c06f510 0x7f674c139bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:44.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.150+0000 7f6751a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f674c071cd0 0x7f674c1f8530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:44.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.150+0000 7f6751a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f674c144d60 0x7f674c1f8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:44.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.150+0000 7f6751a00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f674c0afbf0 con 0x7f674c071cd0 2024-04-20T11:33:44.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.150+0000 7f6751a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f674c0cc940 con 0x7f674c06f510 2024-04-20T11:33:44.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.150+0000 7f6751a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f674c0afd70 con 0x7f674c144d60 2024-04-20T11:33:44.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.151+0000 7f674b400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f674c071cd0 0x7f674c1f8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:44.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.151+0000 7f674b400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f674c071cd0 0x7f674c1f8530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:55592/0 (socket says 172.21.15.152:55592) 2024-04-20T11:33:44.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.151+0000 7f674b400640 1 -- 172.21.15.152:0/2664828507 learned_addr learned my addr 172.21.15.152:0/2664828507 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:33:44.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.151+0000 7f674be00640 1 --2- 172.21.15.152:0/2664828507 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f674c06f510 0x7f674c139bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:44.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.151+0000 7f674b400640 1 -- 172.21.15.152:0/2664828507 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f674c144d60 msgr2=0x7f674c1f8a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:44.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.151+0000 7f674b400640 1 --2- 172.21.15.152:0/2664828507 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f674c144d60 0x7f674c1f8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:44.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.151+0000 7f674b400640 1 -- 172.21.15.152:0/2664828507 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f674c06f510 msgr2=0x7f674c139bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:44.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.151+0000 7f674b400640 1 --2- 172.21.15.152:0/2664828507 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f674c06f510 0x7f674c139bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:44.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.151+0000 7f674b400640 1 -- 172.21.15.152:0/2664828507 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f674c15bc50 con 0x7f674c071cd0 2024-04-20T11:33:44.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.151+0000 7f674be00640 1 --2- 172.21.15.152:0/2664828507 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f674c06f510 0x7f674c139bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:33:44.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.152+0000 7f674b400640 1 --2- 172.21.15.152:0/2664828507 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f674c071cd0 0x7f674c1f8530 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f673800c430 tx=0x7f673800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:44.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.152+0000 7f6743e00640 1 -- 172.21.15.152:0/2664828507 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6738011790 con 0x7f674c071cd0 2024-04-20T11:33:44.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.152+0000 7f6743e00640 1 -- 172.21.15.152:0/2664828507 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6738017070 con 0x7f674c071cd0 2024-04-20T11:33:44.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.152+0000 7f6743e00640 1 -- 172.21.15.152:0/2664828507 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6738011bb0 con 0x7f674c071cd0 2024-04-20T11:33:44.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.152+0000 7f6751a00640 1 -- 172.21.15.152:0/2664828507 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f674c15bf30 con 0x7f674c071cd0 2024-04-20T11:33:44.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.152+0000 7f6751a00640 1 -- 172.21.15.152:0/2664828507 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f674c15c4f0 con 0x7f674c071cd0 2024-04-20T11:33:44.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.153+0000 7f6751a00640 1 -- 172.21.15.152:0/2664828507 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f674c15c6f0 con 0x7f674c071cd0 2024-04-20T11:33:44.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.154+0000 7f6743e00640 1 -- 172.21.15.152:0/2664828507 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 17) ==== 91193+0+0 (secure 0 0 0) 0x7f673801c460 con 0x7f674c071cd0 2024-04-20T11:33:44.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.155+0000 7f6743e00640 1 --2- 172.21.15.152:0/2664828507 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f672c06caa0 0x7f672c06ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:44.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.155+0000 7f6743e00640 1 -- 172.21.15.152:0/2664828507 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7f673808bda0 con 0x7f674c071cd0 2024-04-20T11:33:44.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.156+0000 7f674be00640 1 --2- 172.21.15.152:0/2664828507 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f672c06caa0 0x7f672c06ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:44.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.157+0000 7f674be00640 1 --2- 172.21.15.152:0/2664828507 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f672c06caa0 0x7f672c06ef60 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f673c036000 tx=0x7f673c008030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:44.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.160+0000 7f6743e00640 1 -- 172.21.15.152:0/2664828507 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f674c15c6f0 con 0x7f674c071cd0 2024-04-20T11:33:44.383 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.382+0000 7f6751a00640 1 -- 172.21.15.152:0/2664828507 --> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvol48180", "mode": "777", "uid": 1000, "gid": 1000, "target": ["mon-mgr", ""]}) -- 0x7f674c075e40 con 0x7f672c06caa0 2024-04-20T11:33:44.422 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.422+0000 7f6743e00640 1 -- 172.21.15.152:0/2664828507 <== mgr.4109 v2:172.21.15.175:6836/4021250441 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f674c075e40 con 0x7f672c06caa0 2024-04-20T11:33:44.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.424+0000 7f6751a00640 1 -- 172.21.15.152:0/2664828507 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f672c06caa0 msgr2=0x7f672c06ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:44.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.424+0000 7f6751a00640 1 --2- 172.21.15.152:0/2664828507 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f672c06caa0 0x7f672c06ef60 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f673c036000 tx=0x7f673c008030 comp rx=0 tx=0).stop 2024-04-20T11:33:44.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.424+0000 7f6751a00640 1 -- 172.21.15.152:0/2664828507 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f674c071cd0 msgr2=0x7f674c1f8530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:44.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.424+0000 7f6751a00640 1 --2- 172.21.15.152:0/2664828507 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f674c071cd0 0x7f674c1f8530 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f673800c430 tx=0x7f673800c8f0 comp rx=0 tx=0).stop 2024-04-20T11:33:44.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.424+0000 7f6751a00640 1 -- 172.21.15.152:0/2664828507 shutdown_connections 2024-04-20T11:33:44.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.424+0000 7f6751a00640 1 --2- 172.21.15.152:0/2664828507 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f672c06caa0 0x7f672c06ef60 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:44.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.424+0000 7f6751a00640 1 --2- 172.21.15.152:0/2664828507 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f674c144d60 0x7f674c1f8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:44.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.424+0000 7f6751a00640 1 --2- 172.21.15.152:0/2664828507 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f674c071cd0 0x7f674c1f8530 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:44.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.424+0000 7f6751a00640 1 --2- 172.21.15.152:0/2664828507 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f674c06f510 0x7f674c139bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:44.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.424+0000 7f6751a00640 1 -- 172.21.15.152:0/2664828507 >> 172.21.15.152:0/2664828507 conn(0x7f674c067ae0 msgr2=0x7f674c0b02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:44.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.425+0000 7f6751a00640 1 -- 172.21.15.152:0/2664828507 shutdown_connections 2024-04-20T11:33:44.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.425+0000 7f6751a00640 1 -- 172.21.15.152:0/2664828507 wait complete. 2024-04-20T11:33:44.443 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol48180 2024-04-20T11:33:44.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.642+0000 7f5dae600640 1 Processor -- start 2024-04-20T11:33:44.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.642+0000 7f5dae600640 1 -- start start 2024-04-20T11:33:44.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.643+0000 7f5dae600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5da8163230 0x7f5da80d2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:44.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.643+0000 7f5dae600640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f5da80d33d0 0x7f5da80cf470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:44.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.643+0000 7f5dae600640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5da80d19c0 con 0x7f5da815eb50 2024-04-20T11:33:44.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.643+0000 7f5dae600640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f5da8162750 con 0x7f5da80d33d0 2024-04-20T11:33:44.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.643+0000 7f5dae600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5da8162a30 con 0x7f5da8163230 2024-04-20T11:33:44.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.643+0000 7f5dad200640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f5da815eb50 0x7f5da80d0f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:58112/0 (socket says 172.21.15.152:58112) 2024-04-20T11:33:44.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.643+0000 7f5dad200640 1 -- 172.21.15.152:0/3970068301 learned_addr learned my addr 172.21.15.152:0/3970068301 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:33:44.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.644+0000 7f5da7400640 1 -- 172.21.15.152:0/3970068301 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3929208525 0 0) 0x7f5da80d19c0 con 0x7f5da815eb50 2024-04-20T11:33:44.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.644+0000 7f5da7400640 1 -- 172.21.15.152:0/3970068301 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5d88003520 con 0x7f5da815eb50 2024-04-20T11:33:44.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.644+0000 7f5dadc00640 1 --2- 172.21.15.152:0/3970068301 >> v2:172.21.15.175:3300/0 conn(0x7f5da80d33d0 0x7f5da80cf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:44.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.644+0000 7f5da7e00640 1 --2- 172.21.15.152:0/3970068301 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5da8163230 0x7f5da80d2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:44.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.644+0000 7f5da7400640 1 -- 172.21.15.152:0/3970068301 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2165196980 0 0) 0x7f5d88003520 con 0x7f5da815eb50 2024-04-20T11:33:44.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.644+0000 7f5da7400640 1 -- 172.21.15.152:0/3970068301 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5da8163230 msgr2=0x7f5da80d2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:44.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.644+0000 7f5da7400640 1 --2- 172.21.15.152:0/3970068301 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5da8163230 0x7f5da80d2e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:44.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.644+0000 7f5da7400640 1 -- 172.21.15.152:0/3970068301 >> v2:172.21.15.175:3300/0 conn(0x7f5da80d33d0 msgr2=0x7f5da80cf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:44.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.644+0000 7f5da7400640 1 --2- 172.21.15.152:0/3970068301 >> v2:172.21.15.175:3300/0 conn(0x7f5da80d33d0 0x7f5da80cf470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:44.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.644+0000 7f5da7400640 1 -- 172.21.15.152:0/3970068301 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5da8162d10 con 0x7f5da815eb50 2024-04-20T11:33:44.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.644+0000 7f5da7400640 1 -- 172.21.15.152:0/3970068301 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f5d98001670 con 0x7f5da815eb50 2024-04-20T11:33:44.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.644+0000 7f5da7400640 1 -- 172.21.15.152:0/3970068301 >> v1:172.21.15.152:6789/0 conn(0x7f5da815eb50 legacy=0x7f5da80d0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:44.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.644+0000 7f5da7400640 1 --2- 172.21.15.152:0/3970068301 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d880039b0 0x7f5d88003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:44.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.644+0000 7f5da7400640 1 --2- 172.21.15.152:0/3970068301 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5da8163230 0x7f5d88004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:44.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.644+0000 7f5da7400640 1 --2- 172.21.15.152:0/3970068301 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5d88004d40 0x7f5d880051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:44.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.644+0000 7f5da7400640 1 -- 172.21.15.152:0/3970068301 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f5da8162750 con 0x7f5d88004d40 2024-04-20T11:33:44.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.644+0000 7f5da7400640 1 -- 172.21.15.152:0/3970068301 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5da8162a30 con 0x7f5d880039b0 2024-04-20T11:33:44.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.644+0000 7f5da7400640 1 -- 172.21.15.152:0/3970068301 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5d8800bb20 con 0x7f5da8163230 2024-04-20T11:33:44.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.645+0000 7f5dadc00640 1 --2- 172.21.15.152:0/3970068301 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5d88004d40 0x7f5d880051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:44.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.645+0000 7f5da7e00640 1 --2- 172.21.15.152:0/3970068301 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5da8163230 0x7f5d88004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:44.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.645+0000 7f5dad200640 1 --2- 172.21.15.152:0/3970068301 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d880039b0 0x7f5d88003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:44.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.645+0000 7f5dadc00640 1 -- 172.21.15.152:0/3970068301 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5da8163230 msgr2=0x7f5d88004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:44.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.645+0000 7f5dadc00640 1 --2- 172.21.15.152:0/3970068301 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5da8163230 0x7f5d88004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:44.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.645+0000 7f5dadc00640 1 -- 172.21.15.152:0/3970068301 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d880039b0 msgr2=0x7f5d88003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:44.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.645+0000 7f5dadc00640 1 --2- 172.21.15.152:0/3970068301 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d880039b0 0x7f5d88003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:44.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.645+0000 7f5dadc00640 1 -- 172.21.15.152:0/3970068301 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d88003520 con 0x7f5d88004d40 2024-04-20T11:33:44.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.646+0000 7f5dadc00640 1 --2- 172.21.15.152:0/3970068301 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5d88004d40 0x7f5d880051c0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f5da000ba30 tx=0x7f5d88005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b51e947a00d4203d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:44.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.646+0000 7f5da7400640 1 -- 172.21.15.152:0/3970068301 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5da0036020 con 0x7f5d88004d40 2024-04-20T11:33:44.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.646+0000 7f5da7400640 1 -- 172.21.15.152:0/3970068301 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5da0032680 con 0x7f5d88004d40 2024-04-20T11:33:44.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.646+0000 7f5da7400640 1 -- 172.21.15.152:0/3970068301 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5da000b600 con 0x7f5d88004d40 2024-04-20T11:33:44.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.647+0000 7f5dae600640 1 -- 172.21.15.152:0/3970068301 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5d88004d40 msgr2=0x7f5d880051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:44.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.647+0000 7f5dae600640 1 --2- 172.21.15.152:0/3970068301 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5d88004d40 0x7f5d880051c0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f5da000ba30 tx=0x7f5d88005830 comp rx=0 tx=0).stop 2024-04-20T11:33:44.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.647+0000 7f5dadc00640 1 -- 172.21.15.152:0/3970068301 reap_dead start 2024-04-20T11:33:44.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.647+0000 7f5dae600640 1 -- 172.21.15.152:0/3970068301 shutdown_connections 2024-04-20T11:33:44.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.647+0000 7f5dae600640 1 -- 172.21.15.152:0/3970068301 >> 172.21.15.152:0/3970068301 conn(0x7f5da808bfd0 msgr2=0x7f5da80d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:44.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.647+0000 7f5dae600640 1 -- 172.21.15.152:0/3970068301 shutdown_connections 2024-04-20T11:33:44.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.647+0000 7f5dae600640 1 -- 172.21.15.152:0/3970068301 wait complete. 2024-04-20T11:33:44.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.649+0000 7f5dae600640 1 Processor -- start 2024-04-20T11:33:44.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.649+0000 7f5dae600640 1 -- start start 2024-04-20T11:33:44.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.650+0000 7f5dae600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5da814d940 0x7f5da815c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:44.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.650+0000 7f5dae600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5da814dd20 0x7f5da815cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:44.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.650+0000 7f5dae600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5da80b3f30 0x7f5da80b43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:44.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.650+0000 7f5dae600640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f5da80d1610 con 0x7f5da80b3f30 2024-04-20T11:33:44.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.650+0000 7f5dae600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5da80d2e90 con 0x7f5da814dd20 2024-04-20T11:33:44.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.650+0000 7f5dae600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5da80b4920 con 0x7f5da814d940 2024-04-20T11:33:44.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.650+0000 7f5dadc00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5da80b3f30 0x7f5da80b43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:44.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.650+0000 7f5dadc00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5da80b3f30 0x7f5da80b43e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:55600/0 (socket says 172.21.15.152:55600) 2024-04-20T11:33:44.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.650+0000 7f5dadc00640 1 -- 172.21.15.152:0/2579030553 learned_addr learned my addr 172.21.15.152:0/2579030553 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:33:44.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.651+0000 7f5dad200640 1 --2- 172.21.15.152:0/2579030553 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5da814d940 0x7f5da815c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:44.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.651+0000 7f5dadc00640 1 -- 172.21.15.152:0/2579030553 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5da814d940 msgr2=0x7f5da815c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:44.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.651+0000 7f5da7e00640 1 --2- 172.21.15.152:0/2579030553 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5da814dd20 0x7f5da815cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:44.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.651+0000 7f5dadc00640 1 --2- 172.21.15.152:0/2579030553 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5da814d940 0x7f5da815c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:44.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.651+0000 7f5dadc00640 1 -- 172.21.15.152:0/2579030553 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5da814dd20 msgr2=0x7f5da815cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:44.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.651+0000 7f5dadc00640 1 --2- 172.21.15.152:0/2579030553 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5da814dd20 0x7f5da815cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:44.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.651+0000 7f5dadc00640 1 -- 172.21.15.152:0/2579030553 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5da80b4bd0 con 0x7f5da80b3f30 2024-04-20T11:33:44.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.651+0000 7f5da7e00640 1 --2- 172.21.15.152:0/2579030553 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5da814dd20 0x7f5da815cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:33:44.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.651+0000 7f5dadc00640 1 --2- 172.21.15.152:0/2579030553 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5da80b3f30 0x7f5da80b43e0 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f5da0030680 tx=0x7f5da0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:44.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.651+0000 7f5da4c00640 1 -- 172.21.15.152:0/2579030553 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5da0036040 con 0x7f5da80b3f30 2024-04-20T11:33:44.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.651+0000 7f5da4c00640 1 -- 172.21.15.152:0/2579030553 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5da0040070 con 0x7f5da80b3f30 2024-04-20T11:33:44.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.651+0000 7f5dae600640 1 -- 172.21.15.152:0/2579030553 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5da80b4eb0 con 0x7f5da80b3f30 2024-04-20T11:33:44.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.651+0000 7f5da4c00640 1 -- 172.21.15.152:0/2579030553 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5da000b600 con 0x7f5da80b3f30 2024-04-20T11:33:44.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.651+0000 7f5dae600640 1 -- 172.21.15.152:0/2579030553 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5da815f7b0 con 0x7f5da80b3f30 2024-04-20T11:33:44.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.653+0000 7f5dae600640 1 -- 172.21.15.152:0/2579030553 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5da815f9b0 con 0x7f5da80b3f30 2024-04-20T11:33:44.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.654+0000 7f5da4c00640 1 -- 172.21.15.152:0/2579030553 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 17) ==== 91193+0+0 (secure 0 0 0) 0x7f5da0039020 con 0x7f5da80b3f30 2024-04-20T11:33:44.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.654+0000 7f5da4c00640 1 --2- 172.21.15.152:0/2579030553 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f5d8406cad0 0x7f5d8406ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:44.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.655+0000 7f5da4c00640 1 -- 172.21.15.152:0/2579030553 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7f5da00ae1a0 con 0x7f5da80b3f30 2024-04-20T11:33:44.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.655+0000 7f5dad200640 1 --2- 172.21.15.152:0/2579030553 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f5d8406cad0 0x7f5d8406ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:44.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.656+0000 7f5dad200640 1 --2- 172.21.15.152:0/2579030553 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f5d8406cad0 0x7f5d8406ef90 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5d98007990 tx=0x7f5d98009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:44.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.659+0000 7f5da4c00640 1 -- 172.21.15.152:0/2579030553 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f5da815f9b0 con 0x7f5da80b3f30 2024-04-20T11:33:44.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.887+0000 7f5dae600640 1 -- 172.21.15.152:0/2579030553 --> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol48180", "target": ["mon-mgr", ""]}) -- 0x7f5da8075e60 con 0x7f5d8406cad0 2024-04-20T11:33:44.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.933+0000 7f5da4c00640 1 -- 172.21.15.152:0/2579030553 <== mgr.4109 v2:172.21.15.175:6836/4021250441 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+67 (secure 0 0 0) 0x7f5da8075e60 con 0x7f5d8406cad0 2024-04-20T11:33:44.935 INFO:teuthology.orchestra.run.smithi152.stdout:/volumes/_nogroup/subvol48180/bf9c0317-2e4a-4eaf-b73e-875a6eb33282 2024-04-20T11:33:44.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.936+0000 7f5dae600640 1 -- 172.21.15.152:0/2579030553 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f5d8406cad0 msgr2=0x7f5d8406ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:44.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.936+0000 7f5dae600640 1 --2- 172.21.15.152:0/2579030553 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f5d8406cad0 0x7f5d8406ef90 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5d98007990 tx=0x7f5d98009040 comp rx=0 tx=0).stop 2024-04-20T11:33:44.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.936+0000 7f5dae600640 1 -- 172.21.15.152:0/2579030553 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5da80b3f30 msgr2=0x7f5da80b43e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:44.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.936+0000 7f5dae600640 1 --2- 172.21.15.152:0/2579030553 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5da80b3f30 0x7f5da80b43e0 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f5da0030680 tx=0x7f5da0004730 comp rx=0 tx=0).stop 2024-04-20T11:33:44.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.936+0000 7f5dae600640 1 -- 172.21.15.152:0/2579030553 shutdown_connections 2024-04-20T11:33:44.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.936+0000 7f5dae600640 1 --2- 172.21.15.152:0/2579030553 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f5d8406cad0 0x7f5d8406ef90 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:44.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.936+0000 7f5dae600640 1 --2- 172.21.15.152:0/2579030553 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5da80b3f30 0x7f5da80b43e0 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:44.938 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.936+0000 7f5dae600640 1 --2- 172.21.15.152:0/2579030553 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5da814dd20 0x7f5da815cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:44.938 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.936+0000 7f5dae600640 1 --2- 172.21.15.152:0/2579030553 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5da814d940 0x7f5da815c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:44.938 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.936+0000 7f5dae600640 1 -- 172.21.15.152:0/2579030553 >> 172.21.15.152:0/2579030553 conn(0x7f5da808bfd0 msgr2=0x7f5da80b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:44.938 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.937+0000 7f5dae600640 1 -- 172.21.15.152:0/2579030553 shutdown_connections 2024-04-20T11:33:44.938 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:44.937+0000 7f5dae600640 1 -- 172.21.15.152:0/2579030553 wait complete. 2024-04-20T11:33:44.956 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvol48180 with 8 files each 1MB size under directory volumes/_nogroup/subvol48180/bf9c0317-2e4a-4eaf-b73e-875a6eb33282 2024-04-20T11:33:44.956 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-20T11:33:44.956 DEBUG:teuthology.orchestra.run.smithi152:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:33:45.005 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:45.005 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol48180/bf9c0317-2e4a-4eaf-b73e-875a6eb33282/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0) 2024-04-20T11:33:45.152 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records in 2024-04-20T11:33:45.152 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records out 2024-04-20T11:33:45.153 INFO:teuthology.orchestra.run.smithi152.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0367468 s, 28.5 MB/s 2024-04-20T11:33:45.154 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-20T11:33:45.154 DEBUG:teuthology.orchestra.run.smithi152:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:33:45.210 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:45.210 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol48180/bf9c0317-2e4a-4eaf-b73e-875a6eb33282/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0) 2024-04-20T11:33:45.303 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records in 2024-04-20T11:33:45.303 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records out 2024-04-20T11:33:45.303 INFO:teuthology.orchestra.run.smithi152.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0304192 s, 34.5 MB/s 2024-04-20T11:33:45.304 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-20T11:33:45.305 DEBUG:teuthology.orchestra.run.smithi152:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:33:45.360 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:45.361 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol48180/bf9c0317-2e4a-4eaf-b73e-875a6eb33282/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0) 2024-04-20T11:33:45.461 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records in 2024-04-20T11:33:45.461 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records out 2024-04-20T11:33:45.461 INFO:teuthology.orchestra.run.smithi152.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0324207 s, 32.3 MB/s 2024-04-20T11:33:45.463 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-20T11:33:45.463 DEBUG:teuthology.orchestra.run.smithi152:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:33:45.518 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:45.518 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol48180/bf9c0317-2e4a-4eaf-b73e-875a6eb33282/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0) 2024-04-20T11:33:45.606 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records in 2024-04-20T11:33:45.606 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records out 2024-04-20T11:33:45.607 INFO:teuthology.orchestra.run.smithi152.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0289446 s, 36.2 MB/s 2024-04-20T11:33:45.608 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-20T11:33:45.608 DEBUG:teuthology.orchestra.run.smithi152:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:33:45.664 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:45.664 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol48180/bf9c0317-2e4a-4eaf-b73e-875a6eb33282/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0) 2024-04-20T11:33:45.753 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records in 2024-04-20T11:33:45.754 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records out 2024-04-20T11:33:45.754 INFO:teuthology.orchestra.run.smithi152.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0264392 s, 39.7 MB/s 2024-04-20T11:33:45.755 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-20T11:33:45.755 DEBUG:teuthology.orchestra.run.smithi152:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:33:45.807 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:45.808 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol48180/bf9c0317-2e4a-4eaf-b73e-875a6eb33282/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0) 2024-04-20T11:33:45.893 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records in 2024-04-20T11:33:45.894 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records out 2024-04-20T11:33:45.894 INFO:teuthology.orchestra.run.smithi152.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0307295 s, 34.1 MB/s 2024-04-20T11:33:45.895 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-20T11:33:45.895 DEBUG:teuthology.orchestra.run.smithi152:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:33:45.951 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:45.952 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol48180/bf9c0317-2e4a-4eaf-b73e-875a6eb33282/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0) 2024-04-20T11:33:46.040 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records in 2024-04-20T11:33:46.040 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records out 2024-04-20T11:33:46.041 INFO:teuthology.orchestra.run.smithi152.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0273365 s, 38.4 MB/s 2024-04-20T11:33:46.042 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-20T11:33:46.042 DEBUG:teuthology.orchestra.run.smithi152:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:33:46.099 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:46.099 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol48180/bf9c0317-2e4a-4eaf-b73e-875a6eb33282/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0) 2024-04-20T11:33:46.199 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records in 2024-04-20T11:33:46.199 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records out 2024-04-20T11:33:46.199 INFO:teuthology.orchestra.run.smithi152.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0398762 s, 26.3 MB/s 2024-04-20T11:33:46.201 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs subvol48180 subvol_snap14510 2024-04-20T11:33:46.418 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.417+0000 7f2496e00640 1 Processor -- start 2024-04-20T11:33:46.418 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.418+0000 7f2496e00640 1 -- start start 2024-04-20T11:33:46.420 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.419+0000 7f2496e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f24900cd210 0x7f24900cea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:46.420 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.419+0000 7f2496e00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f249015eb70 con 0x7f249015adc0 2024-04-20T11:33:46.420 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.419+0000 7f2496e00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f249015ee50 con 0x7f2490072dc0 2024-04-20T11:33:46.420 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.419+0000 7f2496e00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f249015f130 con 0x7f24900cd210 2024-04-20T11:33:46.420 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.420+0000 7f248fe00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f249015adc0 0x7f24900726b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:41738/0 (socket says 172.21.15.152:41738) 2024-04-20T11:33:46.420 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.420+0000 7f248fe00640 1 -- 172.21.15.152:0/12329729 learned_addr learned my addr 172.21.15.152:0/12329729 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:33:46.420 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.420+0000 7f2494e00640 1 --2- 172.21.15.152:0/12329729 >> v2:172.21.15.152:3300/0 conn(0x7f24900cd210 0x7f24900cea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:46.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.420+0000 7f248f400640 1 -- 172.21.15.152:0/12329729 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3030133277 0 0) 0x7f249015eb70 con 0x7f249015adc0 2024-04-20T11:33:46.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.420+0000 7f248f400640 1 -- 172.21.15.152:0/12329729 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2474003440 con 0x7f249015adc0 2024-04-20T11:33:46.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.420+0000 7f2494e00640 1 -- 172.21.15.152:0/12329729 >> v1:172.21.15.175:6789/0 conn(0x7f2490072dc0 legacy=0x7f24900b3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:46.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.420+0000 7f2494e00640 1 -- 172.21.15.152:0/12329729 >> v1:172.21.15.152:6789/0 conn(0x7f249015adc0 legacy=0x7f24900726b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:46.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.420+0000 7f2494e00640 1 -- 172.21.15.152:0/12329729 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f249015f410 con 0x7f24900cd210 2024-04-20T11:33:46.422 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.421+0000 7f2494e00640 1 --2- 172.21.15.152:0/12329729 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f24900cd210 0x7f24900cea20 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f2488009d90 tx=0x7f248802bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6c2e4590f325562 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:46.422 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.422+0000 7f248f400640 1 -- 172.21.15.152:0/12329729 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2488039070 con 0x7f24900cd210 2024-04-20T11:33:46.422 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.422+0000 7f248f400640 1 -- 172.21.15.152:0/12329729 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f248802cd00 con 0x7f24900cd210 2024-04-20T11:33:46.422 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.422+0000 7f248f400640 1 -- 172.21.15.152:0/12329729 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2488034660 con 0x7f24900cd210 2024-04-20T11:33:46.423 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.422+0000 7f2496e00640 1 -- 172.21.15.152:0/12329729 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f24900cd210 msgr2=0x7f24900cea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:46.423 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.422+0000 7f2496e00640 1 --2- 172.21.15.152:0/12329729 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f24900cd210 0x7f24900cea20 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f2488009d90 tx=0x7f248802bf50 comp rx=0 tx=0).stop 2024-04-20T11:33:46.423 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.423+0000 7f2496e00640 1 -- 172.21.15.152:0/12329729 shutdown_connections 2024-04-20T11:33:46.423 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.423+0000 7f2496e00640 1 --2- 172.21.15.152:0/12329729 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f24900cd210 0x7f24900cea20 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:46.423 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.423+0000 7f2496e00640 1 -- 172.21.15.152:0/12329729 >> 172.21.15.152:0/12329729 conn(0x7f249006b980 msgr2=0x7f249006bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:46.423 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.423+0000 7f2496e00640 1 -- 172.21.15.152:0/12329729 shutdown_connections 2024-04-20T11:33:46.423 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.423+0000 7f2496e00640 1 -- 172.21.15.152:0/12329729 wait complete. 2024-04-20T11:33:46.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.424+0000 7f2496e00640 1 Processor -- start 2024-04-20T11:33:46.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.424+0000 7f2496e00640 1 -- start start 2024-04-20T11:33:46.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.425+0000 7f2496e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2490072dc0 0x7f2490156310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:46.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.425+0000 7f2496e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f24900cd210 0x7f2490156850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:46.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.425+0000 7f2496e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f249015adc0 0x7f2490140780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:46.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.425+0000 7f2496e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f249006fbb0 con 0x7f24900cd210 2024-04-20T11:33:46.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.425+0000 7f2496e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f249015f6f0 con 0x7f2490072dc0 2024-04-20T11:33:46.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.425+0000 7f2496e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f249006fd30 con 0x7f249015adc0 2024-04-20T11:33:46.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.426+0000 7f2487e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f24900cd210 0x7f2490156850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:46.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.426+0000 7f2487e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f24900cd210 0x7f2490156850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:55328/0 (socket says 172.21.15.152:55328) 2024-04-20T11:33:46.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.426+0000 7f2487e00640 1 -- 172.21.15.152:0/114191479 learned_addr learned my addr 172.21.15.152:0/114191479 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:33:46.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.426+0000 7f248fe00640 1 --2- 172.21.15.152:0/114191479 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2490072dc0 0x7f2490156310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:46.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.426+0000 7f2487e00640 1 -- 172.21.15.152:0/114191479 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f249015adc0 msgr2=0x7f2490140780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:46.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.426+0000 7f2494e00640 1 --2- 172.21.15.152:0/114191479 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f249015adc0 0x7f2490140780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:46.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.426+0000 7f2487e00640 1 --2- 172.21.15.152:0/114191479 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f249015adc0 0x7f2490140780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:46.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.426+0000 7f2487e00640 1 -- 172.21.15.152:0/114191479 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2490072dc0 msgr2=0x7f2490156310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:46.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.426+0000 7f2487e00640 1 --2- 172.21.15.152:0/114191479 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2490072dc0 0x7f2490156310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:46.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.426+0000 7f2487e00640 1 -- 172.21.15.152:0/114191479 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2490142590 con 0x7f24900cd210 2024-04-20T11:33:46.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.427+0000 7f2487e00640 1 --2- 172.21.15.152:0/114191479 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f24900cd210 0x7f2490156850 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f247800c400 tx=0x7f247800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:46.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.427+0000 7f248cc00640 1 -- 172.21.15.152:0/114191479 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24780052d0 con 0x7f24900cd210 2024-04-20T11:33:46.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.427+0000 7f248cc00640 1 -- 172.21.15.152:0/114191479 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2478005490 con 0x7f24900cd210 2024-04-20T11:33:46.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.427+0000 7f248cc00640 1 -- 172.21.15.152:0/114191479 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2478007260 con 0x7f24900cd210 2024-04-20T11:33:46.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.427+0000 7f2496e00640 1 -- 172.21.15.152:0/114191479 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24901414f0 con 0x7f24900cd210 2024-04-20T11:33:46.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.427+0000 7f2496e00640 1 -- 172.21.15.152:0/114191479 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2490141ab0 con 0x7f24900cd210 2024-04-20T11:33:46.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.428+0000 7f2496e00640 1 -- 172.21.15.152:0/114191479 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2490068a60 con 0x7f24900cd210 2024-04-20T11:33:46.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.429+0000 7f248cc00640 1 -- 172.21.15.152:0/114191479 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 17) ==== 91193+0+0 (secure 0 0 0) 0x7f2478007400 con 0x7f24900cd210 2024-04-20T11:33:46.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.431+0000 7f248cc00640 1 --2- 172.21.15.152:0/114191479 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f246c06cad0 0x7f246c06ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:46.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.431+0000 7f248cc00640 1 -- 172.21.15.152:0/114191479 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7f247808b380 con 0x7f24900cd210 2024-04-20T11:33:46.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.431+0000 7f248fe00640 1 --2- 172.21.15.152:0/114191479 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f246c06cad0 0x7f246c06ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:46.433 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.432+0000 7f248fe00640 1 --2- 172.21.15.152:0/114191479 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f246c06cad0 0x7f246c06ef90 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f24800011e0 tx=0x7f2480009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:46.436 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.436+0000 7f248cc00640 1 -- 172.21.15.152:0/114191479 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f2478059520 con 0x7f24900cd210 2024-04-20T11:33:46.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.669+0000 7f2496e00640 1 -- 172.21.15.152:0/114191479 --> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "subvol48180", "snap_name": "subvol_snap14510", "target": ["mon-mgr", ""]}) -- 0x7f2490074ce0 con 0x7f246c06cad0 2024-04-20T11:33:46.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.708+0000 7f248cc00640 1 -- 172.21.15.152:0/114191479 <== mgr.4109 v2:172.21.15.175:6836/4021250441 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f2490074ce0 con 0x7f246c06cad0 2024-04-20T11:33:46.713 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.712+0000 7f2496e00640 1 -- 172.21.15.152:0/114191479 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f246c06cad0 msgr2=0x7f246c06ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:46.713 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.712+0000 7f2496e00640 1 --2- 172.21.15.152:0/114191479 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f246c06cad0 0x7f246c06ef90 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f24800011e0 tx=0x7f2480009040 comp rx=0 tx=0).stop 2024-04-20T11:33:46.714 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.712+0000 7f2496e00640 1 -- 172.21.15.152:0/114191479 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f24900cd210 msgr2=0x7f2490156850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:46.714 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.712+0000 7f2496e00640 1 --2- 172.21.15.152:0/114191479 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f24900cd210 0x7f2490156850 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f247800c400 tx=0x7f247800c8c0 comp rx=0 tx=0).stop 2024-04-20T11:33:46.714 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.712+0000 7f2496e00640 1 -- 172.21.15.152:0/114191479 shutdown_connections 2024-04-20T11:33:46.714 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.712+0000 7f2496e00640 1 --2- 172.21.15.152:0/114191479 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f246c06cad0 0x7f246c06ef90 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:46.714 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.712+0000 7f2496e00640 1 --2- 172.21.15.152:0/114191479 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f249015adc0 0x7f2490140780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:46.714 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.712+0000 7f2496e00640 1 --2- 172.21.15.152:0/114191479 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f24900cd210 0x7f2490156850 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:46.714 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.712+0000 7f2496e00640 1 --2- 172.21.15.152:0/114191479 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2490072dc0 0x7f2490156310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:46.714 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.712+0000 7f2496e00640 1 -- 172.21.15.152:0/114191479 >> 172.21.15.152:0/114191479 conn(0x7f249006b980 msgr2=0x7f2490150ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:46.714 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.713+0000 7f2496e00640 1 -- 172.21.15.152:0/114191479 shutdown_connections 2024-04-20T11:33:46.714 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.713+0000 7f2496e00640 1 -- 172.21.15.152:0/114191479 wait complete. 2024-04-20T11:33:46.734 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs subvol48180 subvol_snap14510 subvol_clone89010 2024-04-20T11:33:46.931 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.930+0000 7f2e8d400640 1 Processor -- start 2024-04-20T11:33:46.931 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.931+0000 7f2e8d400640 1 -- start start 2024-04-20T11:33:46.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.932+0000 7f2e8d400640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f2e88163230 0x7f2e880d2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:46.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.932+0000 7f2e8d400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2e880d33d0 0x7f2e880cf470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:46.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.932+0000 7f2e8d400640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e880d19c0 con 0x7f2e8815eb50 2024-04-20T11:33:46.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.932+0000 7f2e8d400640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f2e88162750 con 0x7f2e88163230 2024-04-20T11:33:46.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.932+0000 7f2e8d400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2e88162a30 con 0x7f2e880d33d0 2024-04-20T11:33:46.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.932+0000 7f2e87e00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f2e8815eb50 0x7f2e880d0f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:41748/0 (socket says 172.21.15.152:41748) 2024-04-20T11:33:46.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.932+0000 7f2e87e00640 1 -- 172.21.15.152:0/748489302 learned_addr learned my addr 172.21.15.152:0/748489302 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:33:46.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.933+0000 7f2e87400640 1 -- 172.21.15.152:0/748489302 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2894102484 0 0) 0x7f2e880d19c0 con 0x7f2e8815eb50 2024-04-20T11:33:46.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.933+0000 7f2e87400640 1 -- 172.21.15.152:0/748489302 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2e6c003520 con 0x7f2e8815eb50 2024-04-20T11:33:46.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.933+0000 7f2e7f400640 1 --2- 172.21.15.152:0/748489302 >> v2:172.21.15.175:3300/0 conn(0x7f2e88163230 0x7f2e880d2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:46.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.933+0000 7f2e8ca00640 1 --2- 172.21.15.152:0/748489302 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2e880d33d0 0x7f2e880cf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:46.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.933+0000 7f2e87400640 1 -- 172.21.15.152:0/748489302 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1856621410 0 0) 0x7f2e6c003520 con 0x7f2e8815eb50 2024-04-20T11:33:46.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.933+0000 7f2e87400640 1 -- 172.21.15.152:0/748489302 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2e880d33d0 msgr2=0x7f2e880cf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:46.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.933+0000 7f2e87400640 1 --2- 172.21.15.152:0/748489302 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2e880d33d0 0x7f2e880cf470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:46.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.933+0000 7f2e87400640 1 -- 172.21.15.152:0/748489302 >> v2:172.21.15.175:3300/0 conn(0x7f2e88163230 msgr2=0x7f2e880d2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:46.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.933+0000 7f2e87400640 1 --2- 172.21.15.152:0/748489302 >> v2:172.21.15.175:3300/0 conn(0x7f2e88163230 0x7f2e880d2e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:46.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.933+0000 7f2e87400640 1 -- 172.21.15.152:0/748489302 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e88162d10 con 0x7f2e8815eb50 2024-04-20T11:33:46.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.933+0000 7f2e87400640 1 -- 172.21.15.152:0/748489302 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f2e78001670 con 0x7f2e8815eb50 2024-04-20T11:33:46.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.933+0000 7f2e87400640 1 -- 172.21.15.152:0/748489302 >> v1:172.21.15.152:6789/0 conn(0x7f2e8815eb50 legacy=0x7f2e880d0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:46.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.933+0000 7f2e87400640 1 --2- 172.21.15.152:0/748489302 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2e6c0039b0 0x7f2e6c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:46.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.933+0000 7f2e87400640 1 --2- 172.21.15.152:0/748489302 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2e6c004800 0x7f2e6c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:46.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.933+0000 7f2e87400640 1 --2- 172.21.15.152:0/748489302 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2e880d33d0 0x7f2e6c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:46.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.933+0000 7f2e87400640 1 -- 172.21.15.152:0/748489302 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f2e88162750 con 0x7f2e6c0039b0 2024-04-20T11:33:46.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.933+0000 7f2e87400640 1 -- 172.21.15.152:0/748489302 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2e88162a30 con 0x7f2e6c004800 2024-04-20T11:33:46.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.933+0000 7f2e87400640 1 -- 172.21.15.152:0/748489302 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2e6c00bb50 con 0x7f2e880d33d0 2024-04-20T11:33:46.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.933+0000 7f2e87e00640 1 --2- 172.21.15.152:0/748489302 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2e6c0039b0 0x7f2e6c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:46.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.934+0000 7f2e7f400640 1 --2- 172.21.15.152:0/748489302 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2e6c004800 0x7f2e6c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:46.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.934+0000 7f2e8ca00640 1 --2- 172.21.15.152:0/748489302 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2e880d33d0 0x7f2e6c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:46.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.934+0000 7f2e87e00640 1 -- 172.21.15.152:0/748489302 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2e880d33d0 msgr2=0x7f2e6c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:46.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.934+0000 7f2e87e00640 1 --2- 172.21.15.152:0/748489302 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2e880d33d0 0x7f2e6c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:46.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.934+0000 7f2e87e00640 1 -- 172.21.15.152:0/748489302 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2e6c004800 msgr2=0x7f2e6c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:46.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.934+0000 7f2e87e00640 1 --2- 172.21.15.152:0/748489302 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2e6c004800 0x7f2e6c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:46.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.934+0000 7f2e87e00640 1 -- 172.21.15.152:0/748489302 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e6c003520 con 0x7f2e6c0039b0 2024-04-20T11:33:46.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.934+0000 7f2e8ca00640 1 --2- 172.21.15.152:0/748489302 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2e880d33d0 0x7f2e6c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:33:46.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.935+0000 7f2e87e00640 1 --2- 172.21.15.152:0/748489302 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2e6c0039b0 0x7f2e6c003dd0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f2e78007fd0 tx=0x7f2e6c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d74b7d7fdd2755e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:46.938 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.935+0000 7f2e87400640 1 -- 172.21.15.152:0/748489302 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e78007d50 con 0x7f2e6c0039b0 2024-04-20T11:33:46.938 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.935+0000 7f2e87400640 1 -- 172.21.15.152:0/748489302 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e78030070 con 0x7f2e6c0039b0 2024-04-20T11:33:46.938 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.935+0000 7f2e87400640 1 -- 172.21.15.152:0/748489302 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e78005cd0 con 0x7f2e6c0039b0 2024-04-20T11:33:46.939 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.936+0000 7f2e8d400640 1 -- 172.21.15.152:0/748489302 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2e6c0039b0 msgr2=0x7f2e6c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:46.939 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.936+0000 7f2e8d400640 1 --2- 172.21.15.152:0/748489302 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2e6c0039b0 0x7f2e6c003dd0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f2e78007fd0 tx=0x7f2e6c005830 comp rx=0 tx=0).stop 2024-04-20T11:33:46.939 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.936+0000 7f2e8ca00640 1 -- 172.21.15.152:0/748489302 reap_dead start 2024-04-20T11:33:46.939 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.936+0000 7f2e8d400640 1 -- 172.21.15.152:0/748489302 shutdown_connections 2024-04-20T11:33:46.939 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.936+0000 7f2e8d400640 1 -- 172.21.15.152:0/748489302 >> 172.21.15.152:0/748489302 conn(0x7f2e8808bfd0 msgr2=0x7f2e880d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:46.939 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.936+0000 7f2e8d400640 1 -- 172.21.15.152:0/748489302 shutdown_connections 2024-04-20T11:33:46.939 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.936+0000 7f2e8d400640 1 -- 172.21.15.152:0/748489302 wait complete. 2024-04-20T11:33:46.939 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.938+0000 7f2e8d400640 1 Processor -- start 2024-04-20T11:33:46.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.938+0000 7f2e8d400640 1 -- start start 2024-04-20T11:33:46.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.939+0000 7f2e8d400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2e6c0039b0 0x7f2e880b7130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:46.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.939+0000 7f2e8d400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2e880b3f30 0x7f2e880ca220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:46.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.939+0000 7f2e8d400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2e88159fb0 0x7f2e880b9550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:46.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.939+0000 7f2e8d400640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f2e880d1610 con 0x7f2e88159fb0 2024-04-20T11:33:46.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.939+0000 7f2e8d400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2e880cf470 con 0x7f2e880b3f30 2024-04-20T11:33:46.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.939+0000 7f2e8d400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2e880b7940 con 0x7f2e6c0039b0 2024-04-20T11:33:46.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.939+0000 7f2e8ca00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2e88159fb0 0x7f2e880b9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:46.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.939+0000 7f2e8ca00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2e88159fb0 0x7f2e880b9550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:55348/0 (socket says 172.21.15.152:55348) 2024-04-20T11:33:46.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.939+0000 7f2e8ca00640 1 -- 172.21.15.152:0/3047749419 learned_addr learned my addr 172.21.15.152:0/3047749419 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:33:46.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.940+0000 7f2e87e00640 1 --2- 172.21.15.152:0/3047749419 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2e6c0039b0 0x7f2e880b7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:46.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.940+0000 7f2e7f400640 1 --2- 172.21.15.152:0/3047749419 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2e880b3f30 0x7f2e880ca220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:46.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.940+0000 7f2e8ca00640 1 -- 172.21.15.152:0/3047749419 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2e6c0039b0 msgr2=0x7f2e880b7130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:46.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.940+0000 7f2e8ca00640 1 --2- 172.21.15.152:0/3047749419 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2e6c0039b0 0x7f2e880b7130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:46.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.940+0000 7f2e8ca00640 1 -- 172.21.15.152:0/3047749419 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2e880b3f30 msgr2=0x7f2e880ca220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:46.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.940+0000 7f2e8ca00640 1 --2- 172.21.15.152:0/3047749419 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2e880b3f30 0x7f2e880ca220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:46.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.940+0000 7f2e8ca00640 1 -- 172.21.15.152:0/3047749419 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e880b7b90 con 0x7f2e88159fb0 2024-04-20T11:33:46.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.940+0000 7f2e87e00640 1 --2- 172.21.15.152:0/3047749419 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2e6c0039b0 0x7f2e880b7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:33:46.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.940+0000 7f2e7f400640 1 --2- 172.21.15.152:0/3047749419 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2e880b3f30 0x7f2e880ca220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:33:46.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.940+0000 7f2e8ca00640 1 --2- 172.21.15.152:0/3047749419 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2e88159fb0 0x7f2e880b9550 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f2e880d4550 tx=0x7f2e8000f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:46.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.940+0000 7f2e84c00640 1 -- 172.21.15.152:0/3047749419 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e80018020 con 0x7f2e88159fb0 2024-04-20T11:33:46.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.940+0000 7f2e84c00640 1 -- 172.21.15.152:0/3047749419 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e8000fe40 con 0x7f2e88159fb0 2024-04-20T11:33:46.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.940+0000 7f2e84c00640 1 -- 172.21.15.152:0/3047749419 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e80014630 con 0x7f2e88159fb0 2024-04-20T11:33:46.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.940+0000 7f2e8d400640 1 -- 172.21.15.152:0/3047749419 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e880b7e70 con 0x7f2e88159fb0 2024-04-20T11:33:46.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.940+0000 7f2e8d400640 1 -- 172.21.15.152:0/3047749419 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e880b9a90 con 0x7f2e88159fb0 2024-04-20T11:33:46.944 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.942+0000 7f2e8d400640 1 -- 172.21.15.152:0/3047749419 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e88075e60 con 0x7f2e88159fb0 2024-04-20T11:33:46.944 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.943+0000 7f2e84c00640 1 -- 172.21.15.152:0/3047749419 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 17) ==== 91193+0+0 (secure 0 0 0) 0x7f2e8001b020 con 0x7f2e88159fb0 2024-04-20T11:33:46.944 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.944+0000 7f2e84c00640 1 --2- 172.21.15.152:0/3047749419 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f2e6406caa0 0x7f2e6406ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:46.944 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.944+0000 7f2e84c00640 1 -- 172.21.15.152:0/3047749419 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7f2e8008c450 con 0x7f2e88159fb0 2024-04-20T11:33:46.945 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.944+0000 7f2e87e00640 1 --2- 172.21.15.152:0/3047749419 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f2e6406caa0 0x7f2e6406ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:46.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.946+0000 7f2e87e00640 1 --2- 172.21.15.152:0/3047749419 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f2e6406caa0 0x7f2e6406ef60 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2e78034000 tx=0x7f2e78007000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:46.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:46.949+0000 7f2e84c00640 1 -- 172.21.15.152:0/3047749419 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f2e80022030 con 0x7f2e88159fb0 2024-04-20T11:33:47.187 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:47.186+0000 7f2e8d400640 1 -- 172.21.15.152:0/3047749419 --> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "subvol48180", "snap_name": "subvol_snap14510", "target_sub_name": "subvol_clone89010", "target": ["mon-mgr", ""]}) -- 0x7f2e880ca760 con 0x7f2e6406caa0 2024-04-20T11:33:47.383 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:47.382+0000 7f2e84c00640 1 -- 172.21.15.152:0/3047749419 <== mgr.4109 v2:172.21.15.175:6836/4021250441 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f2e7802c8d0 con 0x7f2e6406caa0 2024-04-20T11:33:47.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:47.386+0000 7f2e8d400640 1 -- 172.21.15.152:0/3047749419 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f2e6406caa0 msgr2=0x7f2e6406ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:47.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:47.386+0000 7f2e8d400640 1 --2- 172.21.15.152:0/3047749419 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f2e6406caa0 0x7f2e6406ef60 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2e78034000 tx=0x7f2e78007000 comp rx=0 tx=0).stop 2024-04-20T11:33:47.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:47.386+0000 7f2e8d400640 1 -- 172.21.15.152:0/3047749419 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2e88159fb0 msgr2=0x7f2e880b9550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:47.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:47.386+0000 7f2e8d400640 1 --2- 172.21.15.152:0/3047749419 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2e88159fb0 0x7f2e880b9550 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f2e880d4550 tx=0x7f2e8000f520 comp rx=0 tx=0).stop 2024-04-20T11:33:47.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:47.386+0000 7f2e8d400640 1 -- 172.21.15.152:0/3047749419 shutdown_connections 2024-04-20T11:33:47.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:47.386+0000 7f2e8d400640 1 --2- 172.21.15.152:0/3047749419 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f2e6406caa0 0x7f2e6406ef60 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:47.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:47.386+0000 7f2e8d400640 1 --2- 172.21.15.152:0/3047749419 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2e88159fb0 0x7f2e880b9550 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:47.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:47.386+0000 7f2e8d400640 1 --2- 172.21.15.152:0/3047749419 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2e880b3f30 0x7f2e880ca220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:47.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:47.386+0000 7f2e8d400640 1 --2- 172.21.15.152:0/3047749419 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2e6c0039b0 0x7f2e880b7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:47.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:47.386+0000 7f2e8d400640 1 -- 172.21.15.152:0/3047749419 >> 172.21.15.152:0/3047749419 conn(0x7f2e8808bfd0 msgr2=0x7f2e880b2860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:47.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:47.386+0000 7f2e8d400640 1 -- 172.21.15.152:0/3047749419 shutdown_connections 2024-04-20T11:33:47.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:47.387+0000 7f2e8d400640 1 -- 172.21.15.152:0/3047749419 wait complete. 2024-04-20T11:33:47.393 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:33:47.393+0000 7f8a63a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:33:47.393 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:33:47.393+0000 7f8a63a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:33:47.393 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:33:47.393+0000 7f8a63a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:33:47.393 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:33:47.393+0000 7f8a63a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:33:47.393 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:33:47.393+0000 7f8a63a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:33:47.409 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone89010 2024-04-20T11:33:47.425 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:33:47.426+0000 7f8a62600640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:33:47.425 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:33:47.426+0000 7f8a62600640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:33:47.426 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:33:47.426+0000 7f8a62600640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:33:47.426 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:33:47.426+0000 7f8a62600640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:33:47.426 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:33:47.426+0000 7f8a62600640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:33:47.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:47.614+0000 7fda00a00640 1 Processor -- start 2024-04-20T11:33:47.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:47.614+0000 7fda00a00640 1 -- start start 2024-04-20T11:33:47.616 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:47.616+0000 7fda00a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fd9fc0cf5b0 0x7fd9fc0d4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:47.616 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:47.616+0000 7fda00a00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9fc0d2b70 con 0x7fd9fc160230 2024-04-20T11:33:47.616 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:47.616+0000 7fda00a00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9fc0d2e50 con 0x7fd9fc0ceeb0 2024-04-20T11:33:47.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:47.616+0000 7fda00a00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fd9fc0d5420 con 0x7fd9fc0cf5b0 2024-04-20T11:33:47.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:47.616+0000 7fd9fb400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fd9fc0cf5b0 0x7fd9fc0d4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:47.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:47.616+0000 7fd9fbe00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fd9fc160230 0x7fd9fc1606e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:41760/0 (socket says 172.21.15.152:41760) 2024-04-20T11:33:47.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:47.616+0000 7fd9fbe00640 1 -- 172.21.15.152:0/1626222363 learned_addr learned my addr 172.21.15.152:0/1626222363 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:33:47.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:47.617+0000 7fd9fb400640 1 -- 172.21.15.152:0/1626222363 >> v1:172.21.15.175:6789/0 conn(0x7fd9fc0ceeb0 legacy=0x7fd9fc0cf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:47.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:47.617+0000 7fd9fb400640 1 -- 172.21.15.152:0/1626222363 >> v1:172.21.15.152:6789/0 conn(0x7fd9fc160230 legacy=0x7fd9fc1606e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:47.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:47.617+0000 7fd9fb400640 1 -- 172.21.15.152:0/1626222363 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9fc0d3130 con 0x7fd9fc0cf5b0 2024-04-20T11:33:47.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:47.617+0000 7fd9fa000640 1 -- 172.21.15.152:0/1626222363 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2487174870 0 0) 0x7fd9fc0d2b70 con 0x7fd9fc160230 2024-04-20T11:33:47.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:47.618+0000 7fd9fb400640 1 --2- 172.21.15.152:0/1626222363 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd9fc0cf5b0 0x7fd9fc0d4ee0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec0097a0 tx=0x7fd9ec02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62bca84d94a9fc6d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:47.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:47.619+0000 7fd9fa000640 1 -- 172.21.15.152:0/1626222363 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9ec034070 con 0x7fd9fc0cf5b0 2024-04-20T11:33:47.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:47.619+0000 7fd9fa000640 1 -- 172.21.15.152:0/1626222363 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9ec030610 con 0x7fd9fc0cf5b0 2024-04-20T11:33:47.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:47.619+0000 7fd9fa000640 1 -- 172.21.15.152:0/1626222363 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9ec0309e0 con 0x7fd9fc0cf5b0 2024-04-20T11:33:47.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:47.619+0000 7fda00a00640 1 -- 172.21.15.152:0/1626222363 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd9fc0cf5b0 msgr2=0x7fd9fc0d4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:47.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:47.619+0000 7fda00a00640 1 --2- 172.21.15.152:0/1626222363 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd9fc0cf5b0 0x7fd9fc0d4ee0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec0097a0 tx=0x7fd9ec02b960 comp rx=0 tx=0).stop 2024-04-20T11:33:47.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:47.620+0000 7fda00a00640 1 -- 172.21.15.152:0/1626222363 shutdown_connections 2024-04-20T11:33:47.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:47.620+0000 7fda00a00640 1 --2- 172.21.15.152:0/1626222363 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd9fc0cf5b0 0x7fd9fc0d4ee0 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:47.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:47.620+0000 7fda00a00640 1 -- 172.21.15.152:0/1626222363 >> 172.21.15.152:0/1626222363 conn(0x7fd9fc08bfd0 msgr2=0x7fd9fc0d4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:47.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:47.620+0000 7fda00a00640 1 -- 172.21.15.152:0/1626222363 shutdown_connections 2024-04-20T11:33:47.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:47.620+0000 7fda00a00640 1 -- 172.21.15.152:0/1626222363 wait complete. 2024-04-20T11:33:47.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:47.621+0000 7fda00a00640 1 Processor -- start 2024-04-20T11:33:47.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:47.622+0000 7fda00a00640 1 -- start start 2024-04-20T11:33:47.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:47.622+0000 7fda00a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd9fc0ceeb0 0x7fd9fc0cd510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:47.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:47.622+0000 7fda00a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd9fc0cf5b0 0x7fd9fc0ce070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:47.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:47.622+0000 7fda00a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd9fc160230 0x7fd9fc1625a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:47.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:47.623+0000 7fda00a00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fd9fc0d3c10 con 0x7fd9fc160230 2024-04-20T11:33:47.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:47.623+0000 7fda00a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd9fc0d36f0 con 0x7fd9fc0cf5b0 2024-04-20T11:33:47.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:47.623+0000 7fda00a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd9fc0d2790 con 0x7fd9fc0ceeb0 2024-04-20T11:33:47.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:47.623+0000 7fd9fbe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd9fc160230 0x7fd9fc1625a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:47.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:47.623+0000 7fd9fbe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd9fc160230 0x7fd9fc1625a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:55364/0 (socket says 172.21.15.152:55364) 2024-04-20T11:33:47.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:47.623+0000 7fd9fbe00640 1 -- 172.21.15.152:0/2242343670 learned_addr learned my addr 172.21.15.152:0/2242343670 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:33:47.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:47.623+0000 7fd9fb400640 1 --2- 172.21.15.152:0/2242343670 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd9fc0ceeb0 0x7fd9fc0cd510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:47.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:47.623+0000 7fd9fbe00640 1 -- 172.21.15.152:0/2242343670 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd9fc0ceeb0 msgr2=0x7fd9fc0cd510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:47.625 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:47.623+0000 7fd9fbe00640 1 --2- 172.21.15.152:0/2242343670 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd9fc0ceeb0 0x7fd9fc0cd510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:47.625 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:47.623+0000 7fd9faa00640 1 --2- 172.21.15.152:0/2242343670 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd9fc0cf5b0 0x7fd9fc0ce070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:47.625 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:47.623+0000 7fd9fbe00640 1 -- 172.21.15.152:0/2242343670 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd9fc0cf5b0 msgr2=0x7fd9fc0ce070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:47.625 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:47.623+0000 7fd9fbe00640 1 --2- 172.21.15.152:0/2242343670 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd9fc0cf5b0 0x7fd9fc0ce070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:47.625 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:47.623+0000 7fd9fbe00640 1 -- 172.21.15.152:0/2242343670 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9fc1f9100 con 0x7fd9fc160230 2024-04-20T11:33:47.625 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:47.623+0000 7fd9fb400640 1 --2- 172.21.15.152:0/2242343670 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd9fc0ceeb0 0x7fd9fc0cd510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:33:47.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:47.624+0000 7fd9fbe00640 1 --2- 172.21.15.152:0/2242343670 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd9fc160230 0x7fd9fc1625a0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7fd9f400c7a0 tx=0x7fd9f400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:47.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:47.624+0000 7fd9f3400640 1 -- 172.21.15.152:0/2242343670 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f4018070 con 0x7fd9fc160230 2024-04-20T11:33:47.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:47.624+0000 7fd9f3400640 1 -- 172.21.15.152:0/2242343670 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9f400a640 con 0x7fd9fc160230 2024-04-20T11:33:47.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:47.624+0000 7fda00a00640 1 -- 172.21.15.152:0/2242343670 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9fc1f8580 con 0x7fd9fc160230 2024-04-20T11:33:47.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:47.624+0000 7fd9f3400640 1 -- 172.21.15.152:0/2242343670 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f400aab0 con 0x7fd9fc160230 2024-04-20T11:33:47.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:47.624+0000 7fda00a00640 1 -- 172.21.15.152:0/2242343670 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9fc1f8e20 con 0x7fd9fc160230 2024-04-20T11:33:47.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:47.625+0000 7fda00a00640 1 -- 172.21.15.152:0/2242343670 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9fc075e60 con 0x7fd9fc160230 2024-04-20T11:33:47.627 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:47.626+0000 7fd9f3400640 1 -- 172.21.15.152:0/2242343670 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 17) ==== 91193+0+0 (secure 0 0 0) 0x7fd9f401b070 con 0x7fd9fc160230 2024-04-20T11:33:47.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:47.627+0000 7fd9f3400640 1 --2- 172.21.15.152:0/2242343670 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd9dc06caa0 0x7fd9dc06ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:47.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:47.627+0000 7fd9f3400640 1 -- 172.21.15.152:0/2242343670 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7fd9f408b4c0 con 0x7fd9fc160230 2024-04-20T11:33:47.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:47.628+0000 7fd9fb400640 1 --2- 172.21.15.152:0/2242343670 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd9dc06caa0 0x7fd9dc06ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:47.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:47.629+0000 7fd9fb400640 1 --2- 172.21.15.152:0/2242343670 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd9dc06caa0 0x7fd9dc06ef60 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec036000 tx=0x7fd9ec008030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:47.632 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:47.632+0000 7fd9f3400640 1 -- 172.21.15.152:0/2242343670 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fd9f4059670 con 0x7fd9fc160230 2024-04-20T11:33:47.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:47.896+0000 7fda00a00640 1 -- 172.21.15.152:0/2242343670 --> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone89010", "target": ["mon-mgr", ""]}) -- 0x7fd9fc0cf390 con 0x7fd9dc06caa0 2024-04-20T11:33:47.925 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:47.925+0000 7fd9f3400640 1 -- 172.21.15.152:0/2242343670 <== mgr.4109 v2:172.21.15.175:6836/4021250441 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+169 (secure 0 0 0) 0x7fd9fc0cf390 con 0x7fd9dc06caa0 2024-04-20T11:33:47.925 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-04-20T11:33:47.926 INFO:teuthology.orchestra.run.smithi152.stdout: "status": { 2024-04-20T11:33:47.926 INFO:teuthology.orchestra.run.smithi152.stdout: "state": "in-progress", 2024-04-20T11:33:47.926 INFO:teuthology.orchestra.run.smithi152.stdout: "source": { 2024-04-20T11:33:47.926 INFO:teuthology.orchestra.run.smithi152.stdout: "volume": "cephfs", 2024-04-20T11:33:47.926 INFO:teuthology.orchestra.run.smithi152.stdout: "subvolume": "subvol48180", 2024-04-20T11:33:47.926 INFO:teuthology.orchestra.run.smithi152.stdout: "snapshot": "subvol_snap14510" 2024-04-20T11:33:47.926 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:33:47.926 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:33:47.926 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-04-20T11:33:47.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:47.928+0000 7fda00a00640 1 -- 172.21.15.152:0/2242343670 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd9dc06caa0 msgr2=0x7fd9dc06ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:47.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:47.928+0000 7fda00a00640 1 --2- 172.21.15.152:0/2242343670 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd9dc06caa0 0x7fd9dc06ef60 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec036000 tx=0x7fd9ec008030 comp rx=0 tx=0).stop 2024-04-20T11:33:47.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:47.928+0000 7fda00a00640 1 -- 172.21.15.152:0/2242343670 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd9fc160230 msgr2=0x7fd9fc1625a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:47.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:47.928+0000 7fda00a00640 1 --2- 172.21.15.152:0/2242343670 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd9fc160230 0x7fd9fc1625a0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7fd9f400c7a0 tx=0x7fd9f400cc60 comp rx=0 tx=0).stop 2024-04-20T11:33:47.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:47.928+0000 7fda00a00640 1 -- 172.21.15.152:0/2242343670 shutdown_connections 2024-04-20T11:33:47.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:47.928+0000 7fda00a00640 1 --2- 172.21.15.152:0/2242343670 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd9dc06caa0 0x7fd9dc06ef60 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:47.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:47.928+0000 7fda00a00640 1 --2- 172.21.15.152:0/2242343670 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd9fc160230 0x7fd9fc1625a0 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:47.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:47.928+0000 7fda00a00640 1 --2- 172.21.15.152:0/2242343670 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd9fc0cf5b0 0x7fd9fc0ce070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:47.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:47.928+0000 7fda00a00640 1 --2- 172.21.15.152:0/2242343670 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd9fc0ceeb0 0x7fd9fc0cd510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:47.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:47.928+0000 7fda00a00640 1 -- 172.21.15.152:0/2242343670 >> 172.21.15.152:0/2242343670 conn(0x7fd9fc08bfd0 msgr2=0x7fd9fc06df90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:47.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:47.928+0000 7fda00a00640 1 -- 172.21.15.152:0/2242343670 shutdown_connections 2024-04-20T11:33:47.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:47.928+0000 7fda00a00640 1 -- 172.21.15.152:0/2242343670 wait complete. 2024-04-20T11:33:48.951 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone89010 2024-04-20T11:33:49.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:49.165+0000 7f179e400640 1 Processor -- start 2024-04-20T11:33:49.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:49.165+0000 7f179e400640 1 -- start start 2024-04-20T11:33:49.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:49.167+0000 7f179e400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f179815ac50 0x7f17980ad5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:49.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:49.167+0000 7f179e400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f17980b4480 0x7f17980b48e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:49.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:49.167+0000 7f179e400640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17980b4f40 con 0x7f17980725c0 2024-04-20T11:33:49.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:49.167+0000 7f179e400640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f179806dd40 con 0x7f17980b4480 2024-04-20T11:33:49.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:49.167+0000 7f179e400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f179806e020 con 0x7f179815ac50 2024-04-20T11:33:49.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:49.167+0000 7f178fe00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f17980b4480 0x7f17980b48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:49.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:49.167+0000 7f179da00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f17980725c0 0x7f179806e820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:41768/0 (socket says 172.21.15.152:41768) 2024-04-20T11:33:49.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:49.167+0000 7f179da00640 1 -- 172.21.15.152:0/1440549809 learned_addr learned my addr 172.21.15.152:0/1440549809 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:33:49.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:49.167+0000 7f179d000640 1 --2- 172.21.15.152:0/1440549809 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f179815ac50 0x7f17980ad5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:49.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:49.168+0000 7f178fe00640 1 -- 172.21.15.152:0/1440549809 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f179815ac50 msgr2=0x7f17980ad5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:49.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:49.168+0000 7f178fe00640 1 --2- 172.21.15.152:0/1440549809 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f179815ac50 0x7f17980ad5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:49.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:49.168+0000 7f178fe00640 1 -- 172.21.15.152:0/1440549809 >> v1:172.21.15.152:6789/0 conn(0x7f17980725c0 legacy=0x7f179806e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:49.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:49.168+0000 7f178fe00640 1 -- 172.21.15.152:0/1440549809 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f179806e300 con 0x7f17980b4480 2024-04-20T11:33:49.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:49.169+0000 7f178fe00640 1 --2- 172.21.15.152:0/1440549809 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f17980b4480 0x7f17980b48e0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f17800097a0 tx=0x7f178002c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77f4dd73fa1c22d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:49.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:49.169+0000 7f1797e00640 1 -- 172.21.15.152:0/1440549809 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1780030800 con 0x7f17980b4480 2024-04-20T11:33:49.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:49.169+0000 7f1797e00640 1 -- 172.21.15.152:0/1440549809 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17800309a0 con 0x7f17980b4480 2024-04-20T11:33:49.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:49.169+0000 7f1797e00640 1 -- 172.21.15.152:0/1440549809 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1780030d70 con 0x7f17980b4480 2024-04-20T11:33:49.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:49.170+0000 7f179e400640 1 -- 172.21.15.152:0/1440549809 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f17980b4480 msgr2=0x7f17980b48e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:49.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:49.170+0000 7f179e400640 1 --2- 172.21.15.152:0/1440549809 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f17980b4480 0x7f17980b48e0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f17800097a0 tx=0x7f178002c9e0 comp rx=0 tx=0).stop 2024-04-20T11:33:49.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:49.170+0000 7f179e400640 1 -- 172.21.15.152:0/1440549809 shutdown_connections 2024-04-20T11:33:49.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:49.170+0000 7f179e400640 1 --2- 172.21.15.152:0/1440549809 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f17980b4480 0x7f17980b48e0 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:49.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:49.170+0000 7f179e400640 1 --2- 172.21.15.152:0/1440549809 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f179815ac50 0x7f17980ad5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:49.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:49.170+0000 7f179e400640 1 -- 172.21.15.152:0/1440549809 >> 172.21.15.152:0/1440549809 conn(0x7f17980b21e0 msgr2=0x7f17980b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:49.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:49.170+0000 7f179e400640 1 -- 172.21.15.152:0/1440549809 shutdown_connections 2024-04-20T11:33:49.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:49.170+0000 7f179e400640 1 -- 172.21.15.152:0/1440549809 wait complete. 2024-04-20T11:33:49.172 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:49.172+0000 7f179e400640 1 Processor -- start 2024-04-20T11:33:49.172 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:49.172+0000 7f179e400640 1 -- start start 2024-04-20T11:33:49.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:49.173+0000 7f179e400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f17980725c0 0x7f17980921c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:49.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:49.173+0000 7f179e400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f17980b4480 0x7f17980c68a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:49.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:49.173+0000 7f179e400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f179815ac50 0x7f17980907b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:49.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:49.173+0000 7f179e400640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f1798072a50 con 0x7f17980b4480 2024-04-20T11:33:49.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:49.173+0000 7f179e400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f179815b030 con 0x7f179815ac50 2024-04-20T11:33:49.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:49.173+0000 7f179e400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1798072bd0 con 0x7f17980725c0 2024-04-20T11:33:49.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:49.173+0000 7f178fe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f17980b4480 0x7f17980c68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:49.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:49.173+0000 7f178fe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f17980b4480 0x7f17980c68a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:55386/0 (socket says 172.21.15.152:55386) 2024-04-20T11:33:49.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:49.173+0000 7f178fe00640 1 -- 172.21.15.152:0/120748573 learned_addr learned my addr 172.21.15.152:0/120748573 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:33:49.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:49.173+0000 7f179d000640 1 --2- 172.21.15.152:0/120748573 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f17980725c0 0x7f17980921c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:49.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:49.173+0000 7f178fe00640 1 -- 172.21.15.152:0/120748573 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f17980725c0 msgr2=0x7f17980921c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:49.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:49.173+0000 7f178fe00640 1 --2- 172.21.15.152:0/120748573 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f17980725c0 0x7f17980921c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:49.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:49.174+0000 7f178fe00640 1 -- 172.21.15.152:0/120748573 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f179815ac50 msgr2=0x7f17980907b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:49.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:49.174+0000 7f178fe00640 1 --2- 172.21.15.152:0/120748573 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f179815ac50 0x7f17980907b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:49.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:49.174+0000 7f178fe00640 1 -- 172.21.15.152:0/120748573 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1798091120 con 0x7f17980b4480 2024-04-20T11:33:49.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:49.174+0000 7f178fe00640 1 --2- 172.21.15.152:0/120748573 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f17980b4480 0x7f17980c68a0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f178002cff0 tx=0x7f17800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:49.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:49.174+0000 7f1795600640 1 -- 172.21.15.152:0/120748573 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1780038a80 con 0x7f17980b4480 2024-04-20T11:33:49.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:49.174+0000 7f1795600640 1 -- 172.21.15.152:0/120748573 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1780038c40 con 0x7f17980b4480 2024-04-20T11:33:49.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:49.174+0000 7f1795600640 1 -- 172.21.15.152:0/120748573 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17800413f0 con 0x7f17980b4480 2024-04-20T11:33:49.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:49.174+0000 7f179e400640 1 -- 172.21.15.152:0/120748573 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1798091400 con 0x7f17980b4480 2024-04-20T11:33:49.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:49.174+0000 7f179e400640 1 -- 172.21.15.152:0/120748573 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17980919c0 con 0x7f17980b4480 2024-04-20T11:33:49.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:49.176+0000 7f179e400640 1 -- 172.21.15.152:0/120748573 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1798075e60 con 0x7f17980b4480 2024-04-20T11:33:49.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:49.176+0000 7f1795600640 1 -- 172.21.15.152:0/120748573 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 18) ==== 91295+0+0 (secure 0 0 0) 0x7f1780041590 con 0x7f17980b4480 2024-04-20T11:33:49.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:49.177+0000 7f1795600640 1 --2- 172.21.15.152:0/120748573 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f177406cbc0 0x7f177406f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:49.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:49.177+0000 7f1795600640 1 -- 172.21.15.152:0/120748573 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7f17800ae2f0 con 0x7f17980b4480 2024-04-20T11:33:49.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:49.178+0000 7f179d000640 1 --2- 172.21.15.152:0/120748573 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f177406cbc0 0x7f177406f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:49.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:49.179+0000 7f179d000640 1 --2- 172.21.15.152:0/120748573 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f177406cbc0 0x7f177406f080 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1788006fd0 tx=0x7f1788008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:49.182 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:49.182+0000 7f1795600640 1 -- 172.21.15.152:0/120748573 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f1780034030 con 0x7f17980b4480 2024-04-20T11:33:49.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:49.421+0000 7f179e400640 1 -- 172.21.15.152:0/120748573 --> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone89010", "target": ["mon-mgr", ""]}) -- 0x7f1798157b60 con 0x7f177406cbc0 2024-04-20T11:33:51.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:51.354+0000 7f1795600640 1 -- 172.21.15.152:0/120748573 <== mgr.4109 v2:172.21.15.175:6836/4021250441 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+46 (secure 0 0 0) 0x7f1798157b60 con 0x7f177406cbc0 2024-04-20T11:33:51.355 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-04-20T11:33:51.355 INFO:teuthology.orchestra.run.smithi152.stdout: "status": { 2024-04-20T11:33:51.355 INFO:teuthology.orchestra.run.smithi152.stdout: "state": "complete" 2024-04-20T11:33:51.355 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:33:51.355 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-04-20T11:33:51.359 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:51.358+0000 7f179e400640 1 -- 172.21.15.152:0/120748573 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f177406cbc0 msgr2=0x7f177406f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:51.359 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:51.358+0000 7f179e400640 1 --2- 172.21.15.152:0/120748573 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f177406cbc0 0x7f177406f080 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1788006fd0 tx=0x7f1788008040 comp rx=0 tx=0).stop 2024-04-20T11:33:51.359 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:51.358+0000 7f179e400640 1 -- 172.21.15.152:0/120748573 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f17980b4480 msgr2=0x7f17980c68a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:51.359 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:51.358+0000 7f179e400640 1 --2- 172.21.15.152:0/120748573 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f17980b4480 0x7f17980c68a0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f178002cff0 tx=0x7f17800046c0 comp rx=0 tx=0).stop 2024-04-20T11:33:51.359 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:51.358+0000 7f179e400640 1 -- 172.21.15.152:0/120748573 shutdown_connections 2024-04-20T11:33:51.359 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:51.358+0000 7f179e400640 1 --2- 172.21.15.152:0/120748573 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f177406cbc0 0x7f177406f080 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:51.359 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:51.358+0000 7f179e400640 1 --2- 172.21.15.152:0/120748573 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f179815ac50 0x7f17980907b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:51.359 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:51.358+0000 7f179e400640 1 --2- 172.21.15.152:0/120748573 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f17980b4480 0x7f17980c68a0 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:51.359 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:51.358+0000 7f179e400640 1 --2- 172.21.15.152:0/120748573 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f17980725c0 0x7f17980921c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:51.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:51.358+0000 7f179e400640 1 -- 172.21.15.152:0/120748573 >> 172.21.15.152:0/120748573 conn(0x7f17980b21e0 msgr2=0x7f179809d9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:51.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:51.358+0000 7f179e400640 1 -- 172.21.15.152:0/120748573 shutdown_connections 2024-04-20T11:33:51.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:51.359+0000 7f179e400640 1 -- 172.21.15.152:0/120748573 wait complete. 2024-04-20T11:33:51.383 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:51.383 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_nogroup/subvol_clone89010/.meta) 2024-04-20T11:33:51.466 INFO:teuthology.orchestra.run.smithi152.stdout:[GLOBAL] 2024-04-20T11:33:51.466 INFO:teuthology.orchestra.run.smithi152.stdout:version = 2 2024-04-20T11:33:51.466 INFO:teuthology.orchestra.run.smithi152.stdout:type = clone 2024-04-20T11:33:51.466 INFO:teuthology.orchestra.run.smithi152.stdout:path = /volumes/_nogroup/subvol_clone89010/0630167e-6147-477c-b1c7-aaaa3ec12631 2024-04-20T11:33:51.467 INFO:teuthology.orchestra.run.smithi152.stdout:state = complete 2024-04-20T11:33:51.467 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:33:51.470 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol48180 2024-04-20T11:33:51.672 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:51.672+0000 7f18a6200640 1 Processor -- start 2024-04-20T11:33:51.672 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:51.672+0000 7f18a6200640 1 -- start start 2024-04-20T11:33:51.674 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:51.673+0000 7f18a6200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f18a0146000 0x7f18a00b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:51.674 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:51.673+0000 7f18a6200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f18a00b2170 0x7f18a00b2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:51.674 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:51.673+0000 7f18a6200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18a01463e0 con 0x7f18a00cd1b0 2024-04-20T11:33:51.674 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:51.673+0000 7f18a6200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f18a00cdcc0 con 0x7f18a00b2170 2024-04-20T11:33:51.674 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:51.673+0000 7f18a6200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f18a00b4750 con 0x7f18a0146000 2024-04-20T11:33:51.674 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:51.674+0000 7f1896a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f18a00b2170 0x7f18a00b2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:51.674 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:51.674+0000 7f1896a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f18a00b2170 0x7f18a00b2620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:55390/0 (socket says 172.21.15.152:55390) 2024-04-20T11:33:51.675 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:51.674+0000 7f1896a00640 1 -- 172.21.15.152:0/2614049085 learned_addr learned my addr 172.21.15.152:0/2614049085 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:33:51.675 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:51.674+0000 7f189f400640 1 --2- 172.21.15.152:0/2614049085 >> v2:172.21.15.175:3300/0 conn(0x7f18a0146000 0x7f18a00b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:51.675 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:51.674+0000 7f1896a00640 1 -- 172.21.15.152:0/2614049085 >> v2:172.21.15.175:3300/0 conn(0x7f18a0146000 msgr2=0x7f18a00b4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:51.675 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:51.674+0000 7f1896a00640 1 --2- 172.21.15.152:0/2614049085 >> v2:172.21.15.175:3300/0 conn(0x7f18a0146000 0x7f18a00b4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:51.676 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:51.674+0000 7f1896a00640 1 -- 172.21.15.152:0/2614049085 >> v1:172.21.15.175:6789/0 conn(0x7f18a00cd1b0 legacy=0x7f18a00cd5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:51.676 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:51.674+0000 7f1896a00640 1 -- 172.21.15.152:0/2614049085 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18a00b3e20 con 0x7f18a00b2170 2024-04-20T11:33:51.676 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:51.674+0000 7f189f400640 1 --2- 172.21.15.152:0/2614049085 >> v2:172.21.15.175:3300/0 conn(0x7f18a0146000 0x7f18a00b4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:33:51.676 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:51.675+0000 7f1896a00640 1 --2- 172.21.15.152:0/2614049085 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f18a00b2170 0x7f18a00b2620 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f18a0070190 tx=0x7f188802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dad872cb8552c24b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:51.677 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:51.676+0000 7f189ea00640 1 -- 172.21.15.152:0/2614049085 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f188802e070 con 0x7f18a00b2170 2024-04-20T11:33:51.677 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:51.676+0000 7f189ea00640 1 -- 172.21.15.152:0/2614049085 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1888037070 con 0x7f18a00b2170 2024-04-20T11:33:51.677 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:51.676+0000 7f189ea00640 1 -- 172.21.15.152:0/2614049085 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1888031a00 con 0x7f18a00b2170 2024-04-20T11:33:51.677 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:51.676+0000 7f18a6200640 1 -- 172.21.15.152:0/2614049085 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f18a00b2170 msgr2=0x7f18a00b2620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:51.677 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:51.676+0000 7f18a6200640 1 --2- 172.21.15.152:0/2614049085 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f18a00b2170 0x7f18a00b2620 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f18a0070190 tx=0x7f188802b880 comp rx=0 tx=0).stop 2024-04-20T11:33:51.678 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:51.676+0000 7f18a6200640 1 -- 172.21.15.152:0/2614049085 shutdown_connections 2024-04-20T11:33:51.678 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:51.676+0000 7f18a6200640 1 --2- 172.21.15.152:0/2614049085 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f18a00b2170 0x7f18a00b2620 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:51.678 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:51.676+0000 7f18a6200640 1 --2- 172.21.15.152:0/2614049085 >> v2:172.21.15.175:3300/0 conn(0x7f18a0146000 0x7f18a00b4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:51.678 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:51.676+0000 7f18a6200640 1 -- 172.21.15.152:0/2614049085 >> 172.21.15.152:0/2614049085 conn(0x7f18a006b980 msgr2=0x7f18a006bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:51.678 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:51.677+0000 7f18a6200640 1 -- 172.21.15.152:0/2614049085 shutdown_connections 2024-04-20T11:33:51.678 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:51.677+0000 7f18a6200640 1 -- 172.21.15.152:0/2614049085 wait complete. 2024-04-20T11:33:51.678 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:51.678+0000 7f18a6200640 1 Processor -- start 2024-04-20T11:33:51.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:51.678+0000 7f18a6200640 1 -- start start 2024-04-20T11:33:51.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:51.679+0000 7f18a6200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18a00b2170 0x7f18a0144120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:51.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:51.679+0000 7f18a6200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f18a00cd1b0 0x7f18a015bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:51.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:51.679+0000 7f18a6200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f18a0146000 0x7f18a015c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:51.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:51.679+0000 7f18a6200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f18a015ccf0 con 0x7f18a00cd1b0 2024-04-20T11:33:51.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:51.679+0000 7f18a6200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f18a00b43e0 con 0x7f18a00b2170 2024-04-20T11:33:51.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:51.679+0000 7f18a6200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f18a015ce70 con 0x7f18a0146000 2024-04-20T11:33:51.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:51.680+0000 7f1896a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f18a00cd1b0 0x7f18a015bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:51.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:51.680+0000 7f1896a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f18a00cd1b0 0x7f18a015bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:55392/0 (socket says 172.21.15.152:55392) 2024-04-20T11:33:51.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:51.680+0000 7f1896a00640 1 -- 172.21.15.152:0/1655660438 learned_addr learned my addr 172.21.15.152:0/1655660438 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:33:51.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:51.680+0000 7f189f400640 1 --2- 172.21.15.152:0/1655660438 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18a00b2170 0x7f18a0144120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:51.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:51.680+0000 7f189fe00640 1 --2- 172.21.15.152:0/1655660438 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f18a0146000 0x7f18a015c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:51.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:51.680+0000 7f1896a00640 1 -- 172.21.15.152:0/1655660438 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f18a0146000 msgr2=0x7f18a015c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:51.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:51.680+0000 7f1896a00640 1 --2- 172.21.15.152:0/1655660438 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f18a0146000 0x7f18a015c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:51.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:51.680+0000 7f1896a00640 1 -- 172.21.15.152:0/1655660438 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18a00b2170 msgr2=0x7f18a0144120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:51.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:51.680+0000 7f1896a00640 1 --2- 172.21.15.152:0/1655660438 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18a00b2170 0x7f18a0144120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:51.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:51.680+0000 7f1896a00640 1 -- 172.21.15.152:0/1655660438 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18a0140d00 con 0x7f18a00cd1b0 2024-04-20T11:33:51.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:51.680+0000 7f189fe00640 1 --2- 172.21.15.152:0/1655660438 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f18a0146000 0x7f18a015c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:33:51.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:51.680+0000 7f189f400640 1 --2- 172.21.15.152:0/1655660438 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18a00b2170 0x7f18a0144120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:33:51.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:51.681+0000 7f1896a00640 1 --2- 172.21.15.152:0/1655660438 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f18a00cd1b0 0x7f18a015bc50 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f1888000c00 tx=0x7f18880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:51.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:51.681+0000 7f1897e00640 1 -- 172.21.15.152:0/1655660438 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1888032880 con 0x7f18a00cd1b0 2024-04-20T11:33:51.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:51.681+0000 7f1897e00640 1 -- 172.21.15.152:0/1655660438 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1888032a20 con 0x7f18a00cd1b0 2024-04-20T11:33:51.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:51.681+0000 7f1897e00640 1 -- 172.21.15.152:0/1655660438 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1888032e10 con 0x7f18a00cd1b0 2024-04-20T11:33:51.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:51.681+0000 7f18a6200640 1 -- 172.21.15.152:0/1655660438 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18a013ff50 con 0x7f18a00cd1b0 2024-04-20T11:33:51.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:51.681+0000 7f18a6200640 1 -- 172.21.15.152:0/1655660438 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18a0140510 con 0x7f18a00cd1b0 2024-04-20T11:33:51.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:51.682+0000 7f18a6200640 1 -- 172.21.15.152:0/1655660438 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18a00689f0 con 0x7f18a00cd1b0 2024-04-20T11:33:51.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:51.683+0000 7f1897e00640 1 -- 172.21.15.152:0/1655660438 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 18) ==== 91295+0+0 (secure 0 0 0) 0x7f18880453f0 con 0x7f18a00cd1b0 2024-04-20T11:33:51.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:51.684+0000 7f1897e00640 1 --2- 172.21.15.152:0/1655660438 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f187c06cbc0 0x7f187c06f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:51.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:51.684+0000 7f1897e00640 1 -- 172.21.15.152:0/1655660438 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7f18880ae1b0 con 0x7f18a00cd1b0 2024-04-20T11:33:51.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:51.685+0000 7f189f400640 1 --2- 172.21.15.152:0/1655660438 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f187c06cbc0 0x7f187c06f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:51.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:51.686+0000 7f189f400640 1 --2- 172.21.15.152:0/1655660438 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f187c06cbc0 0x7f187c06f080 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1890006fd0 tx=0x7f1890008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:51.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:51.689+0000 7f1897e00640 1 -- 172.21.15.152:0/1655660438 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f1888040070 con 0x7f18a00cd1b0 2024-04-20T11:33:51.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:51.929+0000 7f18a6200640 1 -- 172.21.15.152:0/1655660438 --> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol48180", "target": ["mon-mgr", ""]}) -- 0x7f18a015c9f0 con 0x7f187c06cbc0 2024-04-20T11:33:51.974 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:51.972+0000 7f1897e00640 1 -- 172.21.15.152:0/1655660438 <== mgr.4109 v2:172.21.15.175:6836/4021250441 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+67 (secure 0 0 0) 0x7f18a015c9f0 con 0x7f187c06cbc0 2024-04-20T11:33:51.975 INFO:teuthology.orchestra.run.smithi152.stdout:/volumes/_nogroup/subvol48180/bf9c0317-2e4a-4eaf-b73e-875a6eb33282 2024-04-20T11:33:51.975 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:51.975+0000 7f18a6200640 1 -- 172.21.15.152:0/1655660438 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f187c06cbc0 msgr2=0x7f187c06f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:51.975 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:51.975+0000 7f18a6200640 1 --2- 172.21.15.152:0/1655660438 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f187c06cbc0 0x7f187c06f080 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1890006fd0 tx=0x7f1890008040 comp rx=0 tx=0).stop 2024-04-20T11:33:51.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:51.975+0000 7f18a6200640 1 -- 172.21.15.152:0/1655660438 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f18a00cd1b0 msgr2=0x7f18a015bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:51.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:51.975+0000 7f18a6200640 1 --2- 172.21.15.152:0/1655660438 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f18a00cd1b0 0x7f18a015bc50 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f1888000c00 tx=0x7f18880046c0 comp rx=0 tx=0).stop 2024-04-20T11:33:51.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:51.975+0000 7f18a6200640 1 -- 172.21.15.152:0/1655660438 shutdown_connections 2024-04-20T11:33:51.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:51.975+0000 7f18a6200640 1 --2- 172.21.15.152:0/1655660438 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f187c06cbc0 0x7f187c06f080 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:51.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:51.975+0000 7f18a6200640 1 --2- 172.21.15.152:0/1655660438 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f18a0146000 0x7f18a015c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:51.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:51.975+0000 7f18a6200640 1 --2- 172.21.15.152:0/1655660438 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f18a00cd1b0 0x7f18a015bc50 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:51.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:51.975+0000 7f18a6200640 1 --2- 172.21.15.152:0/1655660438 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18a00b2170 0x7f18a0144120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:51.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:51.975+0000 7f18a6200640 1 -- 172.21.15.152:0/1655660438 >> 172.21.15.152:0/1655660438 conn(0x7f18a006b980 msgr2=0x7f18a00b02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:51.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:51.975+0000 7f18a6200640 1 -- 172.21.15.152:0/1655660438 shutdown_connections 2024-04-20T11:33:51.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:51.975+0000 7f18a6200640 1 -- 172.21.15.152:0/1655660438 wait complete. 2024-04-20T11:33:51.995 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol_clone89010 2024-04-20T11:33:52.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.199+0000 7fb5cba00640 1 Processor -- start 2024-04-20T11:33:52.201 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.200+0000 7fb5cba00640 1 -- start start 2024-04-20T11:33:52.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.201+0000 7fb5cba00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fb5c40cf5d0 0x7fb5c40cf9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:52.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.201+0000 7fb5cba00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fb5c40d29a0 0x7fb5c40b62a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:52.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.201+0000 7fb5cba00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5c40cfef0 con 0x7fb5c40d2e30 2024-04-20T11:33:52.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.201+0000 7fb5cba00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fb5c40d3f90 con 0x7fb5c40cf5d0 2024-04-20T11:33:52.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.201+0000 7fb5cba00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fb5c40d3380 con 0x7fb5c40d29a0 2024-04-20T11:33:52.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.201+0000 7fb5c9000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fb5c40cf5d0 0x7fb5c40cf9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:52.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.201+0000 7fb5c9000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fb5c40cf5d0 0x7fb5c40cf9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:55398/0 (socket says 172.21.15.152:55398) 2024-04-20T11:33:52.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.201+0000 7fb5c9000640 1 -- 172.21.15.152:0/2564389866 learned_addr learned my addr 172.21.15.152:0/2564389866 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:33:52.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.202+0000 7fb5c3e00640 1 --2- 172.21.15.152:0/2564389866 >> v2:172.21.15.175:3300/0 conn(0x7fb5c40d29a0 0x7fb5c40b62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:52.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.202+0000 7fb5c9000640 1 -- 172.21.15.152:0/2564389866 >> v2:172.21.15.175:3300/0 conn(0x7fb5c40d29a0 msgr2=0x7fb5c40b62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:52.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.202+0000 7fb5c9000640 1 --2- 172.21.15.152:0/2564389866 >> v2:172.21.15.175:3300/0 conn(0x7fb5c40d29a0 0x7fb5c40b62a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:52.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.202+0000 7fb5c9000640 1 -- 172.21.15.152:0/2564389866 >> v1:172.21.15.175:6789/0 conn(0x7fb5c40d2e30 legacy=0x7fb5c40b67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:52.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.202+0000 7fb5c9000640 1 -- 172.21.15.152:0/2564389866 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5c40d3660 con 0x7fb5c40cf5d0 2024-04-20T11:33:52.204 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.203+0000 7fb5c9000640 1 --2- 172.21.15.152:0/2564389866 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb5c40cf5d0 0x7fb5c40cf9b0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7fb5c40b73b0 tx=0x7fb5b402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dca608483e3ef147 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:52.204 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.203+0000 7fb5c3400640 1 -- 172.21.15.152:0/2564389866 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b402ca80 con 0x7fb5c40cf5d0 2024-04-20T11:33:52.204 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.203+0000 7fb5c3400640 1 -- 172.21.15.152:0/2564389866 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5b402cc20 con 0x7fb5c40cf5d0 2024-04-20T11:33:52.204 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.203+0000 7fb5c3400640 1 -- 172.21.15.152:0/2564389866 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b40343f0 con 0x7fb5c40cf5d0 2024-04-20T11:33:52.205 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.204+0000 7fb5cba00640 1 -- 172.21.15.152:0/2564389866 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb5c40cf5d0 msgr2=0x7fb5c40cf9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:52.205 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.204+0000 7fb5cba00640 1 --2- 172.21.15.152:0/2564389866 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb5c40cf5d0 0x7fb5c40cf9b0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7fb5c40b73b0 tx=0x7fb5b402bee0 comp rx=0 tx=0).stop 2024-04-20T11:33:52.205 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.204+0000 7fb5cba00640 1 -- 172.21.15.152:0/2564389866 shutdown_connections 2024-04-20T11:33:52.205 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.204+0000 7fb5cba00640 1 --2- 172.21.15.152:0/2564389866 >> v2:172.21.15.175:3300/0 conn(0x7fb5c40d29a0 0x7fb5c40b62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:52.205 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.204+0000 7fb5cba00640 1 --2- 172.21.15.152:0/2564389866 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb5c40cf5d0 0x7fb5c40cf9b0 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:52.205 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.204+0000 7fb5cba00640 1 -- 172.21.15.152:0/2564389866 >> 172.21.15.152:0/2564389866 conn(0x7fb5c406cc00 msgr2=0x7fb5c406d010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:52.205 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.204+0000 7fb5cba00640 1 -- 172.21.15.152:0/2564389866 shutdown_connections 2024-04-20T11:33:52.205 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.204+0000 7fb5cba00640 1 -- 172.21.15.152:0/2564389866 wait complete. 2024-04-20T11:33:52.207 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.206+0000 7fb5cba00640 1 Processor -- start 2024-04-20T11:33:52.207 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.207+0000 7fb5cba00640 1 -- start start 2024-04-20T11:33:52.209 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.208+0000 7fb5cba00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb5c40cf5d0 0x7fb5c41e7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:52.209 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.208+0000 7fb5cba00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5c40d29a0 0x7fb5c40c9da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:52.209 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.208+0000 7fb5cba00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb5c40d2e30 0x7fb5c40ca2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:52.209 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.208+0000 7fb5cba00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fb5c40b8960 con 0x7fb5c40cf5d0 2024-04-20T11:33:52.209 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.208+0000 7fb5cba00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb5c40d3c20 con 0x7fb5c40d29a0 2024-04-20T11:33:52.209 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.208+0000 7fb5cba00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb5c40d4420 con 0x7fb5c40d2e30 2024-04-20T11:33:52.209 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.208+0000 7fb5c9000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb5c40cf5d0 0x7fb5c41e7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:52.209 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.208+0000 7fb5c9000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb5c40cf5d0 0x7fb5c41e7300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:55406/0 (socket says 172.21.15.152:55406) 2024-04-20T11:33:52.210 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.208+0000 7fb5c9000640 1 -- 172.21.15.152:0/808645420 learned_addr learned my addr 172.21.15.152:0/808645420 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:33:52.210 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.208+0000 7fb5c9000640 1 -- 172.21.15.152:0/808645420 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb5c40d2e30 msgr2=0x7fb5c40ca2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:52.210 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.208+0000 7fb5c9000640 1 --2- 172.21.15.152:0/808645420 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb5c40d2e30 0x7fb5c40ca2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:52.210 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.208+0000 7fb5c9000640 1 -- 172.21.15.152:0/808645420 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5c40d29a0 msgr2=0x7fb5c40c9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:52.210 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.208+0000 7fb5c3e00640 1 --2- 172.21.15.152:0/808645420 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5c40d29a0 0x7fb5c40c9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:52.210 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.208+0000 7fb5c9000640 1 --2- 172.21.15.152:0/808645420 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5c40d29a0 0x7fb5c40c9da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:52.210 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.208+0000 7fb5c9000640 1 -- 172.21.15.152:0/808645420 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5c40b6aa0 con 0x7fb5c40cf5d0 2024-04-20T11:33:52.211 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.209+0000 7fb5c9000640 1 --2- 172.21.15.152:0/808645420 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb5c40cf5d0 0x7fb5c41e7300 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7fb5b4009f40 tx=0x7fb5b4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:52.211 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.209+0000 7fb5c0c00640 1 -- 172.21.15.152:0/808645420 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b4043020 con 0x7fb5c40cf5d0 2024-04-20T11:33:52.211 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.209+0000 7fb5c0c00640 1 -- 172.21.15.152:0/808645420 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5b403e450 con 0x7fb5c40cf5d0 2024-04-20T11:33:52.211 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.209+0000 7fb5c0c00640 1 -- 172.21.15.152:0/808645420 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b403e840 con 0x7fb5c40cf5d0 2024-04-20T11:33:52.212 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.209+0000 7fb5cba00640 1 -- 172.21.15.152:0/808645420 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5c40c92c0 con 0x7fb5c40cf5d0 2024-04-20T11:33:52.212 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.209+0000 7fb5cba00640 1 -- 172.21.15.152:0/808645420 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5c415da70 con 0x7fb5c40cf5d0 2024-04-20T11:33:52.212 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.211+0000 7fb5cba00640 1 -- 172.21.15.152:0/808645420 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5c40689f0 con 0x7fb5c40cf5d0 2024-04-20T11:33:52.212 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.212+0000 7fb5c0c00640 1 -- 172.21.15.152:0/808645420 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 18) ==== 91295+0+0 (secure 0 0 0) 0x7fb5b4046020 con 0x7fb5c40cf5d0 2024-04-20T11:33:52.213 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.212+0000 7fb5c0c00640 1 --2- 172.21.15.152:0/808645420 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb5a006cbc0 0x7fb5a006f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:52.213 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.213+0000 7fb5c0c00640 1 -- 172.21.15.152:0/808645420 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7fb5b40ae950 con 0x7fb5c40cf5d0 2024-04-20T11:33:52.213 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.213+0000 7fb5c3e00640 1 --2- 172.21.15.152:0/808645420 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb5a006cbc0 0x7fb5a006f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:52.215 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.214+0000 7fb5c3e00640 1 --2- 172.21.15.152:0/808645420 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb5a006cbc0 0x7fb5a006f080 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb5c40b6fd0 tx=0x7fb5ac008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:52.218 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.217+0000 7fb5c0c00640 1 -- 172.21.15.152:0/808645420 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fb5b404d030 con 0x7fb5c40cf5d0 2024-04-20T11:33:52.449 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.448+0000 7fb5cba00640 1 -- 172.21.15.152:0/808645420 --> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol_clone89010", "target": ["mon-mgr", ""]}) -- 0x7fb5c40b6d80 con 0x7fb5a006cbc0 2024-04-20T11:33:52.462 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.462+0000 7fb5c0c00640 1 -- 172.21.15.152:0/808645420 <== mgr.4109 v2:172.21.15.175:6836/4021250441 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+73 (secure 0 0 0) 0x7fb5c40b6d80 con 0x7fb5a006cbc0 2024-04-20T11:33:52.462 INFO:teuthology.orchestra.run.smithi152.stdout:/volumes/_nogroup/subvol_clone89010/0630167e-6147-477c-b1c7-aaaa3ec12631 2024-04-20T11:33:52.465 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.464+0000 7fb5cba00640 1 -- 172.21.15.152:0/808645420 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb5a006cbc0 msgr2=0x7fb5a006f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:52.465 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.464+0000 7fb5cba00640 1 --2- 172.21.15.152:0/808645420 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb5a006cbc0 0x7fb5a006f080 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb5c40b6fd0 tx=0x7fb5ac008040 comp rx=0 tx=0).stop 2024-04-20T11:33:52.465 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.464+0000 7fb5cba00640 1 -- 172.21.15.152:0/808645420 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb5c40cf5d0 msgr2=0x7fb5c41e7300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:52.465 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.464+0000 7fb5cba00640 1 --2- 172.21.15.152:0/808645420 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb5c40cf5d0 0x7fb5c41e7300 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7fb5b4009f40 tx=0x7fb5b4004a40 comp rx=0 tx=0).stop 2024-04-20T11:33:52.466 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.464+0000 7fb5cba00640 1 -- 172.21.15.152:0/808645420 shutdown_connections 2024-04-20T11:33:52.466 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.464+0000 7fb5cba00640 1 --2- 172.21.15.152:0/808645420 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb5a006cbc0 0x7fb5a006f080 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:52.466 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.464+0000 7fb5cba00640 1 --2- 172.21.15.152:0/808645420 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb5c40d2e30 0x7fb5c40ca2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:52.466 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.464+0000 7fb5cba00640 1 --2- 172.21.15.152:0/808645420 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5c40d29a0 0x7fb5c40c9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:52.466 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.464+0000 7fb5cba00640 1 --2- 172.21.15.152:0/808645420 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb5c40cf5d0 0x7fb5c41e7300 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:52.466 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.464+0000 7fb5cba00640 1 -- 172.21.15.152:0/808645420 >> 172.21.15.152:0/808645420 conn(0x7fb5c406cc00 msgr2=0x7fb5c41609d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:52.466 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.465+0000 7fb5cba00640 1 -- 172.21.15.152:0/808645420 shutdown_connections 2024-04-20T11:33:52.466 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.465+0000 7fb5cba00640 1 -- 172.21.15.152:0/808645420 wait complete. 2024-04-20T11:33:52.485 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:52.485 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.rentries volumes/_nogroup/subvol48180/bf9c0317-2e4a-4eaf-b73e-875a6eb33282/.snap/subvol_snap14510) 2024-04-20T11:33:52.548 INFO:teuthology.orchestra.run.smithi152.stdout:9 2024-04-20T11:33:52.548 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:52.548 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.rentries volumes/_nogroup/subvol_clone89010/0630167e-6147-477c-b1c7-aaaa3ec12631) 2024-04-20T11:33:52.605 INFO:teuthology.orchestra.run.smithi152.stdout:9 2024-04-20T11:33:52.605 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume info cephfs subvol_clone89010 2024-04-20T11:33:52.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.803+0000 7fe4c7e00640 1 Processor -- start 2024-04-20T11:33:52.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.804+0000 7fe4c7e00640 1 -- start start 2024-04-20T11:33:52.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.805+0000 7fe4c7e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fe4c00b2880 0x7fe4c00b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:52.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.805+0000 7fe4c7e00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4c01463e0 con 0x7fe4c00cd1b0 2024-04-20T11:33:52.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.805+0000 7fe4c7e00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4c00b3e20 con 0x7fe4c0146000 2024-04-20T11:33:52.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.805+0000 7fe4c7e00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fe4c00b4750 con 0x7fe4c00b2880 2024-04-20T11:33:52.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.805+0000 7fe4c4a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fe4c00b2880 0x7fe4c00b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:52.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.805+0000 7fe4c5e00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fe4c00cd1b0 0x7fe4c00cd5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:41770/0 (socket says 172.21.15.152:41770) 2024-04-20T11:33:52.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.805+0000 7fe4c5e00640 1 -- 172.21.15.152:0/2542638731 learned_addr learned my addr 172.21.15.152:0/2542638731 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:33:52.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.806+0000 7fe4c4a00640 1 -- 172.21.15.152:0/2542638731 >> v1:172.21.15.175:6789/0 conn(0x7fe4c0146000 legacy=0x7fe4c00b2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:52.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.806+0000 7fe4c4a00640 1 -- 172.21.15.152:0/2542638731 >> v1:172.21.15.152:6789/0 conn(0x7fe4c00cd1b0 legacy=0x7fe4c00cd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:52.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.806+0000 7fe4c4a00640 1 -- 172.21.15.152:0/2542638731 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4c00b4100 con 0x7fe4c00b2880 2024-04-20T11:33:52.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.807+0000 7fe4c4a00640 1 --2- 172.21.15.152:0/2542638731 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe4c00b2880 0x7fe4c00b4be0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac0097a0 tx=0x7fe4ac02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34b7ef92dcc432de server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:52.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.807+0000 7fe4bfe00640 1 -- 172.21.15.152:0/2542638731 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4ac0315a0 con 0x7fe4c00b2880 2024-04-20T11:33:52.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.807+0000 7fe4bfe00640 1 -- 172.21.15.152:0/2542638731 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4ac037070 con 0x7fe4c00b2880 2024-04-20T11:33:52.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.807+0000 7fe4bfe00640 1 -- 172.21.15.152:0/2542638731 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4ac031960 con 0x7fe4c00b2880 2024-04-20T11:33:52.809 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.808+0000 7fe4c7e00640 1 -- 172.21.15.152:0/2542638731 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe4c00b2880 msgr2=0x7fe4c00b4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:52.809 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.808+0000 7fe4c7e00640 1 --2- 172.21.15.152:0/2542638731 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe4c00b2880 0x7fe4c00b4be0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac0097a0 tx=0x7fe4ac02b840 comp rx=0 tx=0).stop 2024-04-20T11:33:52.809 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.808+0000 7fe4c7e00640 1 -- 172.21.15.152:0/2542638731 shutdown_connections 2024-04-20T11:33:52.809 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.808+0000 7fe4c7e00640 1 --2- 172.21.15.152:0/2542638731 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe4c00b2880 0x7fe4c00b4be0 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:52.809 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.808+0000 7fe4c7e00640 1 -- 172.21.15.152:0/2542638731 >> 172.21.15.152:0/2542638731 conn(0x7fe4c006b980 msgr2=0x7fe4c006bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:52.809 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.808+0000 7fe4c7e00640 1 -- 172.21.15.152:0/2542638731 shutdown_connections 2024-04-20T11:33:52.809 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.808+0000 7fe4c7e00640 1 -- 172.21.15.152:0/2542638731 wait complete. 2024-04-20T11:33:52.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.810+0000 7fe4c7e00640 1 Processor -- start 2024-04-20T11:33:52.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.810+0000 7fe4c7e00640 1 -- start start 2024-04-20T11:33:52.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.811+0000 7fe4c7e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe4c00b2880 0x7fe4c015c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:52.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.811+0000 7fe4c7e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe4c00cd1b0 0x7fe4c00ab0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:52.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.811+0000 7fe4c7e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe4c0146000 0x7fe4c015e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:52.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.811+0000 7fe4c7e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c015ce70 con 0x7fe4c0146000 2024-04-20T11:33:52.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.811+0000 7fe4c7e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c015ccf0 con 0x7fe4c00b2880 2024-04-20T11:33:52.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.811+0000 7fe4c7e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe4c015cff0 con 0x7fe4c00cd1b0 2024-04-20T11:33:52.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.811+0000 7fe4c5e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe4c0146000 0x7fe4c015e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:52.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.811+0000 7fe4c5e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe4c0146000 0x7fe4c015e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:55422/0 (socket says 172.21.15.152:55422) 2024-04-20T11:33:52.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.811+0000 7fe4c5e00640 1 -- 172.21.15.152:0/539376767 learned_addr learned my addr 172.21.15.152:0/539376767 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:33:52.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.812+0000 7fe4c5400640 1 --2- 172.21.15.152:0/539376767 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe4c00b2880 0x7fe4c015c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:52.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.812+0000 7fe4c4a00640 1 --2- 172.21.15.152:0/539376767 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe4c00cd1b0 0x7fe4c00ab0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:52.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.812+0000 7fe4c5e00640 1 -- 172.21.15.152:0/539376767 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe4c00cd1b0 msgr2=0x7fe4c00ab0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:52.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.812+0000 7fe4c5e00640 1 --2- 172.21.15.152:0/539376767 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe4c00cd1b0 0x7fe4c00ab0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:52.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.812+0000 7fe4c5e00640 1 -- 172.21.15.152:0/539376767 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe4c00b2880 msgr2=0x7fe4c015c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:52.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.812+0000 7fe4c5e00640 1 --2- 172.21.15.152:0/539376767 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe4c00b2880 0x7fe4c015c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:52.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.812+0000 7fe4c5e00640 1 -- 172.21.15.152:0/539376767 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4c0143c00 con 0x7fe4c0146000 2024-04-20T11:33:52.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.812+0000 7fe4c5400640 1 --2- 172.21.15.152:0/539376767 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe4c00b2880 0x7fe4c015c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:33:52.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.812+0000 7fe4c5e00640 1 --2- 172.21.15.152:0/539376767 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe4c0146000 0x7fe4c015e3d0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7fe4b800c420 tx=0x7fe4b800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:52.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.812+0000 7fe4bd600640 1 -- 172.21.15.152:0/539376767 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b8017070 con 0x7fe4c0146000 2024-04-20T11:33:52.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.812+0000 7fe4bd600640 1 -- 172.21.15.152:0/539376767 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4b8005ce0 con 0x7fe4c0146000 2024-04-20T11:33:52.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.812+0000 7fe4c7e00640 1 -- 172.21.15.152:0/539376767 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4c0143ee0 con 0x7fe4c0146000 2024-04-20T11:33:52.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.812+0000 7fe4bd600640 1 -- 172.21.15.152:0/539376767 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b80072d0 con 0x7fe4c0146000 2024-04-20T11:33:52.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.813+0000 7fe4c7e00640 1 -- 172.21.15.152:0/539376767 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4c01444a0 con 0x7fe4c0146000 2024-04-20T11:33:52.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.814+0000 7fe4c7e00640 1 -- 172.21.15.152:0/539376767 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4c00689f0 con 0x7fe4c0146000 2024-04-20T11:33:52.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.815+0000 7fe4bd600640 1 -- 172.21.15.152:0/539376767 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 18) ==== 91295+0+0 (secure 0 0 0) 0x7fe4b801a070 con 0x7fe4c0146000 2024-04-20T11:33:52.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.815+0000 7fe4bd600640 1 --2- 172.21.15.152:0/539376767 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe4a006cbc0 0x7fe4a006f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:52.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.816+0000 7fe4bd600640 1 -- 172.21.15.152:0/539376767 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7fe4b808b3c0 con 0x7fe4c0146000 2024-04-20T11:33:52.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.816+0000 7fe4c5400640 1 --2- 172.21.15.152:0/539376767 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe4a006cbc0 0x7fe4a006f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:52.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.817+0000 7fe4c5400640 1 --2- 172.21.15.152:0/539376767 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe4a006cbc0 0x7fe4a006f080 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe4b4007e90 tx=0x7fe4b40073a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:52.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:52.820+0000 7fe4bd600640 1 -- 172.21.15.152:0/539376767 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe4b8058d30 con 0x7fe4c0146000 2024-04-20T11:33:53.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:53.053+0000 7fe4c7e00640 1 -- 172.21.15.152:0/539376767 --> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] -- mgr_command(tid 0: {"prefix": "fs subvolume info", "vol_name": "cephfs", "sub_name": "subvol_clone89010", "target": ["mon-mgr", ""]}) -- 0x7fe4c00d0500 con 0x7fe4a006cbc0 2024-04-20T11:33:53.074 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-04-20T11:33:53.075 INFO:teuthology.orchestra.run.smithi152.stdout: "atime": "2024-04-20 11:33:40", 2024-04-20T11:33:53.075 INFO:teuthology.orchestra.run.smithi152.stdout: "bytes_pcent": "80.00", 2024-04-20T11:33:53.075 INFO:teuthology.orchestra.run.smithi152.stdout: "bytes_quota": 10485760, 2024-04-20T11:33:53.075 INFO:teuthology.orchestra.run.smithi152.stdout: "bytes_used": 8388608, 2024-04-20T11:33:53.075 INFO:teuthology.orchestra.run.smithi152.stdout: "created_at": "2024-04-20 11:33:47", 2024-04-20T11:33:53.075 INFO:teuthology.orchestra.run.smithi152.stdout: "ctime": "2024-04-20 11:33:47", 2024-04-20T11:33:53.075 INFO:teuthology.orchestra.run.smithi152.stdout: "data_pool": "cephfs_data", 2024-04-20T11:33:53.075 INFO:teuthology.orchestra.run.smithi152.stdout: "features": [ 2024-04-20T11:33:53.075 INFO:teuthology.orchestra.run.smithi152.stdout: "snapshot-clone", 2024-04-20T11:33:53.075 INFO:teuthology.orchestra.run.smithi152.stdout: "snapshot-autoprotect", 2024-04-20T11:33:53.075 INFO:teuthology.orchestra.run.smithi152.stdout: "snapshot-retention" 2024-04-20T11:33:53.075 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:33:53.075 INFO:teuthology.orchestra.run.smithi152.stdout: "flavor": 2, 2024-04-20T11:33:53.075 INFO:teuthology.orchestra.run.smithi152.stdout: "gid": 1000, 2024-04-20T11:33:53.076 INFO:teuthology.orchestra.run.smithi152.stdout: "mode": 16895, 2024-04-20T11:33:53.076 INFO:teuthology.orchestra.run.smithi152.stdout: "mon_addrs": [ 2024-04-20T11:33:53.076 INFO:teuthology.orchestra.run.smithi152.stdout: "172.21.15.152:6789", 2024-04-20T11:33:53.076 INFO:teuthology.orchestra.run.smithi152.stdout: "172.21.15.175:6789", 2024-04-20T11:33:53.076 INFO:teuthology.orchestra.run.smithi152.stdout: "172.21.15.175:6790" 2024-04-20T11:33:53.076 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-04-20T11:33:53.076 INFO:teuthology.orchestra.run.smithi152.stdout: "mtime": "2024-04-20 11:33:46", 2024-04-20T11:33:53.076 INFO:teuthology.orchestra.run.smithi152.stdout: "path": "/volumes/_nogroup/subvol_clone89010/0630167e-6147-477c-b1c7-aaaa3ec12631", 2024-04-20T11:33:53.076 INFO:teuthology.orchestra.run.smithi152.stdout: "pool_namespace": "", 2024-04-20T11:33:53.076 INFO:teuthology.orchestra.run.smithi152.stdout: "state": "complete", 2024-04-20T11:33:53.076 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "clone", 2024-04-20T11:33:53.076 INFO:teuthology.orchestra.run.smithi152.stdout: "uid": 1000 2024-04-20T11:33:53.076 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-04-20T11:33:53.077 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:53.074+0000 7fe4bd600640 1 -- 172.21.15.152:0/539376767 <== mgr.4109 v2:172.21.15.175:6836/4021250441 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+725 (secure 0 0 0) 0x7fe4c00d0500 con 0x7fe4a006cbc0 2024-04-20T11:33:53.077 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:53.076+0000 7fe4c7e00640 1 -- 172.21.15.152:0/539376767 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe4a006cbc0 msgr2=0x7fe4a006f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:53.077 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:53.076+0000 7fe4c7e00640 1 --2- 172.21.15.152:0/539376767 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe4a006cbc0 0x7fe4a006f080 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe4b4007e90 tx=0x7fe4b40073a0 comp rx=0 tx=0).stop 2024-04-20T11:33:53.077 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:53.076+0000 7fe4c7e00640 1 -- 172.21.15.152:0/539376767 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe4c0146000 msgr2=0x7fe4c015e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:53.077 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:53.076+0000 7fe4c7e00640 1 --2- 172.21.15.152:0/539376767 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe4c0146000 0x7fe4c015e3d0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7fe4b800c420 tx=0x7fe4b800c8e0 comp rx=0 tx=0).stop 2024-04-20T11:33:53.077 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:53.077+0000 7fe4c7e00640 1 -- 172.21.15.152:0/539376767 shutdown_connections 2024-04-20T11:33:53.078 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:53.077+0000 7fe4c7e00640 1 --2- 172.21.15.152:0/539376767 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe4a006cbc0 0x7fe4a006f080 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:53.078 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:53.077+0000 7fe4c7e00640 1 --2- 172.21.15.152:0/539376767 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe4c0146000 0x7fe4c015e3d0 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:53.078 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:53.077+0000 7fe4c7e00640 1 --2- 172.21.15.152:0/539376767 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe4c00cd1b0 0x7fe4c00ab0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:53.078 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:53.077+0000 7fe4c7e00640 1 --2- 172.21.15.152:0/539376767 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe4c00b2880 0x7fe4c015c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:53.078 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:53.077+0000 7fe4c7e00640 1 -- 172.21.15.152:0/539376767 >> 172.21.15.152:0/539376767 conn(0x7fe4c006b980 msgr2=0x7fe4c00b19a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:53.078 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:53.077+0000 7fe4c7e00640 1 -- 172.21.15.152:0/539376767 shutdown_connections 2024-04-20T11:33:53.078 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:53.077+0000 7fe4c7e00640 1 -- 172.21.15.152:0/539376767 wait complete. 2024-04-20T11:33:53.079 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:33:53.085 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:33:53.096 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:53.096 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.quota.max_bytes volumes/_nogroup/subvol48180/bf9c0317-2e4a-4eaf-b73e-875a6eb33282/.snap/subvol_snap14510) 2024-04-20T11:33:53.195 INFO:teuthology.orchestra.run.smithi152.stdout:10485760 2024-04-20T11:33:53.195 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:53.196 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool volumes/_nogroup/subvol48180/bf9c0317-2e4a-4eaf-b73e-875a6eb33282/.snap/subvol_snap14510) 2024-04-20T11:33:53.304 INFO:teuthology.orchestra.run.smithi152.stdout:cephfs_data 2024-04-20T11:33:53.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:53.304 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool_namespace volumes/_nogroup/subvol48180/bf9c0317-2e4a-4eaf-b73e-875a6eb33282/.snap/subvol_snap14510) 2024-04-20T11:33:53.374 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:53.374 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec find volumes/_nogroup/subvol48180/bf9c0317-2e4a-4eaf-b73e-875a6eb33282/.snap/subvol_snap14510) 2024-04-20T11:33:53.443 INFO:teuthology.orchestra.run.smithi152.stdout:volumes/_nogroup/subvol48180/bf9c0317-2e4a-4eaf-b73e-875a6eb33282/.snap/subvol_snap14510 2024-04-20T11:33:53.443 INFO:teuthology.orchestra.run.smithi152.stdout:volumes/_nogroup/subvol48180/bf9c0317-2e4a-4eaf-b73e-875a6eb33282/.snap/subvol_snap14510/subvolume_file.2 2024-04-20T11:33:53.443 INFO:teuthology.orchestra.run.smithi152.stdout:volumes/_nogroup/subvol48180/bf9c0317-2e4a-4eaf-b73e-875a6eb33282/.snap/subvol_snap14510/subvolume_file.5 2024-04-20T11:33:53.444 INFO:teuthology.orchestra.run.smithi152.stdout:volumes/_nogroup/subvol48180/bf9c0317-2e4a-4eaf-b73e-875a6eb33282/.snap/subvol_snap14510/subvolume_file.4 2024-04-20T11:33:53.444 INFO:teuthology.orchestra.run.smithi152.stdout:volumes/_nogroup/subvol48180/bf9c0317-2e4a-4eaf-b73e-875a6eb33282/.snap/subvol_snap14510/subvolume_file.7 2024-04-20T11:33:53.444 INFO:teuthology.orchestra.run.smithi152.stdout:volumes/_nogroup/subvol48180/bf9c0317-2e4a-4eaf-b73e-875a6eb33282/.snap/subvol_snap14510/subvolume_file.6 2024-04-20T11:33:53.444 INFO:teuthology.orchestra.run.smithi152.stdout:volumes/_nogroup/subvol48180/bf9c0317-2e4a-4eaf-b73e-875a6eb33282/.snap/subvol_snap14510/subvolume_file.0 2024-04-20T11:33:53.444 INFO:teuthology.orchestra.run.smithi152.stdout:volumes/_nogroup/subvol48180/bf9c0317-2e4a-4eaf-b73e-875a6eb33282/.snap/subvol_snap14510/subvolume_file.3 2024-04-20T11:33:53.444 INFO:teuthology.orchestra.run.smithi152.stdout:volumes/_nogroup/subvol48180/bf9c0317-2e4a-4eaf-b73e-875a6eb33282/.snap/subvol_snap14510/subvolume_file.1 2024-04-20T11:33:53.445 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:53.445 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol48180/bf9c0317-2e4a-4eaf-b73e-875a6eb33282/.snap/subvol_snap14510) 2024-04-20T11:33:53.502 INFO:teuthology.orchestra.run.smithi152.stdout:41ff 2024-04-20T11:33:53.504 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:53.504 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone89010/0630167e-6147-477c-b1c7-aaaa3ec12631/) 2024-04-20T11:33:53.558 INFO:teuthology.orchestra.run.smithi152.stdout:41ff 2024-04-20T11:33:53.559 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:53.559 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol48180/bf9c0317-2e4a-4eaf-b73e-875a6eb33282/.snap/subvol_snap14510) 2024-04-20T11:33:53.614 INFO:teuthology.orchestra.run.smithi152.stdout:1000 2024-04-20T11:33:53.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:53.615 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone89010/0630167e-6147-477c-b1c7-aaaa3ec12631/) 2024-04-20T11:33:53.667 INFO:teuthology.orchestra.run.smithi152.stdout:1000 2024-04-20T11:33:53.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:53.669 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol48180/bf9c0317-2e4a-4eaf-b73e-875a6eb33282/.snap/subvol_snap14510) 2024-04-20T11:33:53.722 INFO:teuthology.orchestra.run.smithi152.stdout:1000 2024-04-20T11:33:53.723 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:53.723 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone89010/0630167e-6147-477c-b1c7-aaaa3ec12631/) 2024-04-20T11:33:53.777 INFO:teuthology.orchestra.run.smithi152.stdout:1000 2024-04-20T11:33:53.778 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:53.779 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol48180/bf9c0317-2e4a-4eaf-b73e-875a6eb33282/.snap/subvol_snap14510) 2024-04-20T11:33:53.836 INFO:teuthology.orchestra.run.smithi152.stdout:1713612826 2024-04-20T11:33:53.837 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:53.837 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone89010/0630167e-6147-477c-b1c7-aaaa3ec12631/) 2024-04-20T11:33:53.890 INFO:teuthology.orchestra.run.smithi152.stdout:1713612826 2024-04-20T11:33:53.891 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:53.892 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol48180/bf9c0317-2e4a-4eaf-b73e-875a6eb33282/.snap/subvol_snap14510/subvolume_file.2) 2024-04-20T11:33:53.948 INFO:teuthology.orchestra.run.smithi152.stdout:81a4 2024-04-20T11:33:53.950 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:53.950 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone89010/0630167e-6147-477c-b1c7-aaaa3ec12631/subvolume_file.2) 2024-04-20T11:33:54.005 INFO:teuthology.orchestra.run.smithi152.stdout:81a4 2024-04-20T11:33:54.007 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:54.007 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol48180/bf9c0317-2e4a-4eaf-b73e-875a6eb33282/.snap/subvol_snap14510/subvolume_file.2) 2024-04-20T11:33:54.063 INFO:teuthology.orchestra.run.smithi152.stdout:1000 2024-04-20T11:33:54.065 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:54.065 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone89010/0630167e-6147-477c-b1c7-aaaa3ec12631/subvolume_file.2) 2024-04-20T11:33:54.114 INFO:teuthology.orchestra.run.smithi152.stdout:1000 2024-04-20T11:33:54.115 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:54.115 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol48180/bf9c0317-2e4a-4eaf-b73e-875a6eb33282/.snap/subvol_snap14510/subvolume_file.2) 2024-04-20T11:33:54.167 INFO:teuthology.orchestra.run.smithi152.stdout:1316 2024-04-20T11:33:54.169 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:54.169 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone89010/0630167e-6147-477c-b1c7-aaaa3ec12631/subvolume_file.2) 2024-04-20T11:33:54.222 INFO:teuthology.orchestra.run.smithi152.stdout:1316 2024-04-20T11:33:54.222 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:54.222 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol48180/bf9c0317-2e4a-4eaf-b73e-875a6eb33282/.snap/subvol_snap14510/subvolume_file.2) 2024-04-20T11:33:54.273 INFO:teuthology.orchestra.run.smithi152.stdout:1713612825 2024-04-20T11:33:54.274 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:54.274 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone89010/0630167e-6147-477c-b1c7-aaaa3ec12631/subvolume_file.2) 2024-04-20T11:33:54.322 INFO:teuthology.orchestra.run.smithi152.stdout:1713612825 2024-04-20T11:33:54.324 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:54.324 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol48180/bf9c0317-2e4a-4eaf-b73e-875a6eb33282/.snap/subvol_snap14510/subvolume_file.5) 2024-04-20T11:33:54.380 INFO:teuthology.orchestra.run.smithi152.stdout:81a4 2024-04-20T11:33:54.382 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:54.382 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone89010/0630167e-6147-477c-b1c7-aaaa3ec12631/subvolume_file.5) 2024-04-20T11:33:54.438 INFO:teuthology.orchestra.run.smithi152.stdout:81a4 2024-04-20T11:33:54.439 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:54.440 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol48180/bf9c0317-2e4a-4eaf-b73e-875a6eb33282/.snap/subvol_snap14510/subvolume_file.5) 2024-04-20T11:33:54.493 INFO:teuthology.orchestra.run.smithi152.stdout:1000 2024-04-20T11:33:54.494 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:54.494 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone89010/0630167e-6147-477c-b1c7-aaaa3ec12631/subvolume_file.5) 2024-04-20T11:33:54.544 INFO:teuthology.orchestra.run.smithi152.stdout:1000 2024-04-20T11:33:54.545 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:54.545 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol48180/bf9c0317-2e4a-4eaf-b73e-875a6eb33282/.snap/subvol_snap14510/subvolume_file.5) 2024-04-20T11:33:54.598 INFO:teuthology.orchestra.run.smithi152.stdout:1316 2024-04-20T11:33:54.599 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:54.599 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone89010/0630167e-6147-477c-b1c7-aaaa3ec12631/subvolume_file.5) 2024-04-20T11:33:54.654 INFO:teuthology.orchestra.run.smithi152.stdout:1316 2024-04-20T11:33:54.655 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:54.655 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol48180/bf9c0317-2e4a-4eaf-b73e-875a6eb33282/.snap/subvol_snap14510/subvolume_file.5) 2024-04-20T11:33:54.712 INFO:teuthology.orchestra.run.smithi152.stdout:1713612825 2024-04-20T11:33:54.713 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:54.713 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone89010/0630167e-6147-477c-b1c7-aaaa3ec12631/subvolume_file.5) 2024-04-20T11:33:54.765 INFO:teuthology.orchestra.run.smithi152.stdout:1713612825 2024-04-20T11:33:54.766 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:54.766 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol48180/bf9c0317-2e4a-4eaf-b73e-875a6eb33282/.snap/subvol_snap14510/subvolume_file.4) 2024-04-20T11:33:54.821 INFO:teuthology.orchestra.run.smithi152.stdout:81a4 2024-04-20T11:33:54.823 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:54.823 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone89010/0630167e-6147-477c-b1c7-aaaa3ec12631/subvolume_file.4) 2024-04-20T11:33:54.877 INFO:teuthology.orchestra.run.smithi152.stdout:81a4 2024-04-20T11:33:54.879 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:54.879 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol48180/bf9c0317-2e4a-4eaf-b73e-875a6eb33282/.snap/subvol_snap14510/subvolume_file.4) 2024-04-20T11:33:54.934 INFO:teuthology.orchestra.run.smithi152.stdout:1000 2024-04-20T11:33:54.935 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:54.935 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone89010/0630167e-6147-477c-b1c7-aaaa3ec12631/subvolume_file.4) 2024-04-20T11:33:54.981 INFO:teuthology.orchestra.run.smithi152.stdout:1000 2024-04-20T11:33:54.983 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:54.983 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol48180/bf9c0317-2e4a-4eaf-b73e-875a6eb33282/.snap/subvol_snap14510/subvolume_file.4) 2024-04-20T11:33:55.034 INFO:teuthology.orchestra.run.smithi152.stdout:1316 2024-04-20T11:33:55.037 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:55.037 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone89010/0630167e-6147-477c-b1c7-aaaa3ec12631/subvolume_file.4) 2024-04-20T11:33:55.087 INFO:teuthology.orchestra.run.smithi152.stdout:1316 2024-04-20T11:33:55.088 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:55.088 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol48180/bf9c0317-2e4a-4eaf-b73e-875a6eb33282/.snap/subvol_snap14510/subvolume_file.4) 2024-04-20T11:33:55.135 INFO:teuthology.orchestra.run.smithi152.stdout:1713612825 2024-04-20T11:33:55.136 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:55.137 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone89010/0630167e-6147-477c-b1c7-aaaa3ec12631/subvolume_file.4) 2024-04-20T11:33:55.187 INFO:teuthology.orchestra.run.smithi152.stdout:1713612825 2024-04-20T11:33:55.188 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:55.188 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol48180/bf9c0317-2e4a-4eaf-b73e-875a6eb33282/.snap/subvol_snap14510/subvolume_file.7) 2024-04-20T11:33:55.243 INFO:teuthology.orchestra.run.smithi152.stdout:81a4 2024-04-20T11:33:55.244 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:55.245 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone89010/0630167e-6147-477c-b1c7-aaaa3ec12631/subvolume_file.7) 2024-04-20T11:33:55.300 INFO:teuthology.orchestra.run.smithi152.stdout:81a4 2024-04-20T11:33:55.301 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:55.301 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol48180/bf9c0317-2e4a-4eaf-b73e-875a6eb33282/.snap/subvol_snap14510/subvolume_file.7) 2024-04-20T11:33:55.353 INFO:teuthology.orchestra.run.smithi152.stdout:1000 2024-04-20T11:33:55.355 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:55.355 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone89010/0630167e-6147-477c-b1c7-aaaa3ec12631/subvolume_file.7) 2024-04-20T11:33:55.407 INFO:teuthology.orchestra.run.smithi152.stdout:1000 2024-04-20T11:33:55.408 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:55.408 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol48180/bf9c0317-2e4a-4eaf-b73e-875a6eb33282/.snap/subvol_snap14510/subvolume_file.7) 2024-04-20T11:33:55.463 INFO:teuthology.orchestra.run.smithi152.stdout:1316 2024-04-20T11:33:55.464 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:55.464 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone89010/0630167e-6147-477c-b1c7-aaaa3ec12631/subvolume_file.7) 2024-04-20T11:33:55.517 INFO:teuthology.orchestra.run.smithi152.stdout:1316 2024-04-20T11:33:55.518 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:55.518 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol48180/bf9c0317-2e4a-4eaf-b73e-875a6eb33282/.snap/subvol_snap14510/subvolume_file.7) 2024-04-20T11:33:55.576 INFO:teuthology.orchestra.run.smithi152.stdout:1713612826 2024-04-20T11:33:55.577 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:55.578 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone89010/0630167e-6147-477c-b1c7-aaaa3ec12631/subvolume_file.7) 2024-04-20T11:33:55.630 INFO:teuthology.orchestra.run.smithi152.stdout:1713612826 2024-04-20T11:33:55.631 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:55.632 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol48180/bf9c0317-2e4a-4eaf-b73e-875a6eb33282/.snap/subvol_snap14510/subvolume_file.6) 2024-04-20T11:33:55.688 INFO:teuthology.orchestra.run.smithi152.stdout:81a4 2024-04-20T11:33:55.690 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:55.690 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone89010/0630167e-6147-477c-b1c7-aaaa3ec12631/subvolume_file.6) 2024-04-20T11:33:55.749 INFO:teuthology.orchestra.run.smithi152.stdout:81a4 2024-04-20T11:33:55.750 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:55.750 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol48180/bf9c0317-2e4a-4eaf-b73e-875a6eb33282/.snap/subvol_snap14510/subvolume_file.6) 2024-04-20T11:33:55.807 INFO:teuthology.orchestra.run.smithi152.stdout:1000 2024-04-20T11:33:55.808 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:55.808 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone89010/0630167e-6147-477c-b1c7-aaaa3ec12631/subvolume_file.6) 2024-04-20T11:33:55.861 INFO:teuthology.orchestra.run.smithi152.stdout:1000 2024-04-20T11:33:55.862 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:55.863 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol48180/bf9c0317-2e4a-4eaf-b73e-875a6eb33282/.snap/subvol_snap14510/subvolume_file.6) 2024-04-20T11:33:55.923 INFO:teuthology.orchestra.run.smithi152.stdout:1316 2024-04-20T11:33:55.925 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:55.925 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone89010/0630167e-6147-477c-b1c7-aaaa3ec12631/subvolume_file.6) 2024-04-20T11:33:55.980 INFO:teuthology.orchestra.run.smithi152.stdout:1316 2024-04-20T11:33:55.981 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:55.982 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol48180/bf9c0317-2e4a-4eaf-b73e-875a6eb33282/.snap/subvol_snap14510/subvolume_file.6) 2024-04-20T11:33:56.038 INFO:teuthology.orchestra.run.smithi152.stdout:1713612826 2024-04-20T11:33:56.040 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:56.040 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone89010/0630167e-6147-477c-b1c7-aaaa3ec12631/subvolume_file.6) 2024-04-20T11:33:56.092 INFO:teuthology.orchestra.run.smithi152.stdout:1713612826 2024-04-20T11:33:56.093 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:56.093 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol48180/bf9c0317-2e4a-4eaf-b73e-875a6eb33282/.snap/subvol_snap14510/subvolume_file.0) 2024-04-20T11:33:56.145 INFO:teuthology.orchestra.run.smithi152.stdout:81a4 2024-04-20T11:33:56.147 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:56.147 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone89010/0630167e-6147-477c-b1c7-aaaa3ec12631/subvolume_file.0) 2024-04-20T11:33:56.199 INFO:teuthology.orchestra.run.smithi152.stdout:81a4 2024-04-20T11:33:56.200 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:56.200 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol48180/bf9c0317-2e4a-4eaf-b73e-875a6eb33282/.snap/subvol_snap14510/subvolume_file.0) 2024-04-20T11:33:56.256 INFO:teuthology.orchestra.run.smithi152.stdout:1000 2024-04-20T11:33:56.258 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:56.258 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone89010/0630167e-6147-477c-b1c7-aaaa3ec12631/subvolume_file.0) 2024-04-20T11:33:56.315 INFO:teuthology.orchestra.run.smithi152.stdout:1000 2024-04-20T11:33:56.316 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:56.316 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol48180/bf9c0317-2e4a-4eaf-b73e-875a6eb33282/.snap/subvol_snap14510/subvolume_file.0) 2024-04-20T11:33:56.373 INFO:teuthology.orchestra.run.smithi152.stdout:1316 2024-04-20T11:33:56.375 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:56.375 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone89010/0630167e-6147-477c-b1c7-aaaa3ec12631/subvolume_file.0) 2024-04-20T11:33:56.426 INFO:teuthology.orchestra.run.smithi152.stdout:1316 2024-04-20T11:33:56.427 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:56.427 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol48180/bf9c0317-2e4a-4eaf-b73e-875a6eb33282/.snap/subvol_snap14510/subvolume_file.0) 2024-04-20T11:33:56.483 INFO:teuthology.orchestra.run.smithi152.stdout:1713612825 2024-04-20T11:33:56.485 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:56.485 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone89010/0630167e-6147-477c-b1c7-aaaa3ec12631/subvolume_file.0) 2024-04-20T11:33:56.538 INFO:teuthology.orchestra.run.smithi152.stdout:1713612825 2024-04-20T11:33:56.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:56.540 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol48180/bf9c0317-2e4a-4eaf-b73e-875a6eb33282/.snap/subvol_snap14510/subvolume_file.3) 2024-04-20T11:33:56.593 INFO:teuthology.orchestra.run.smithi152.stdout:81a4 2024-04-20T11:33:56.594 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:56.594 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone89010/0630167e-6147-477c-b1c7-aaaa3ec12631/subvolume_file.3) 2024-04-20T11:33:56.649 INFO:teuthology.orchestra.run.smithi152.stdout:81a4 2024-04-20T11:33:56.650 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:56.650 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol48180/bf9c0317-2e4a-4eaf-b73e-875a6eb33282/.snap/subvol_snap14510/subvolume_file.3) 2024-04-20T11:33:56.707 INFO:teuthology.orchestra.run.smithi152.stdout:1000 2024-04-20T11:33:56.708 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:56.708 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone89010/0630167e-6147-477c-b1c7-aaaa3ec12631/subvolume_file.3) 2024-04-20T11:33:56.764 INFO:teuthology.orchestra.run.smithi152.stdout:1000 2024-04-20T11:33:56.766 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:56.766 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol48180/bf9c0317-2e4a-4eaf-b73e-875a6eb33282/.snap/subvol_snap14510/subvolume_file.3) 2024-04-20T11:33:56.825 INFO:teuthology.orchestra.run.smithi152.stdout:1316 2024-04-20T11:33:56.826 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:56.827 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone89010/0630167e-6147-477c-b1c7-aaaa3ec12631/subvolume_file.3) 2024-04-20T11:33:56.884 INFO:teuthology.orchestra.run.smithi152.stdout:1316 2024-04-20T11:33:56.885 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:56.885 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol48180/bf9c0317-2e4a-4eaf-b73e-875a6eb33282/.snap/subvol_snap14510/subvolume_file.3) 2024-04-20T11:33:56.938 INFO:teuthology.orchestra.run.smithi152.stdout:1713612825 2024-04-20T11:33:56.939 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:56.939 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone89010/0630167e-6147-477c-b1c7-aaaa3ec12631/subvolume_file.3) 2024-04-20T11:33:56.991 INFO:teuthology.orchestra.run.smithi152.stdout:1713612825 2024-04-20T11:33:56.992 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:56.992 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol48180/bf9c0317-2e4a-4eaf-b73e-875a6eb33282/.snap/subvol_snap14510/subvolume_file.1) 2024-04-20T11:33:57.052 INFO:teuthology.orchestra.run.smithi152.stdout:81a4 2024-04-20T11:33:57.053 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:57.053 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone89010/0630167e-6147-477c-b1c7-aaaa3ec12631/subvolume_file.1) 2024-04-20T11:33:57.109 INFO:teuthology.orchestra.run.smithi152.stdout:81a4 2024-04-20T11:33:57.110 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:57.110 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol48180/bf9c0317-2e4a-4eaf-b73e-875a6eb33282/.snap/subvol_snap14510/subvolume_file.1) 2024-04-20T11:33:57.162 INFO:teuthology.orchestra.run.smithi152.stdout:1000 2024-04-20T11:33:57.163 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:57.163 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone89010/0630167e-6147-477c-b1c7-aaaa3ec12631/subvolume_file.1) 2024-04-20T11:33:57.217 INFO:teuthology.orchestra.run.smithi152.stdout:1000 2024-04-20T11:33:57.218 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:57.218 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol48180/bf9c0317-2e4a-4eaf-b73e-875a6eb33282/.snap/subvol_snap14510/subvolume_file.1) 2024-04-20T11:33:57.273 INFO:teuthology.orchestra.run.smithi152.stdout:1316 2024-04-20T11:33:57.274 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:57.274 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone89010/0630167e-6147-477c-b1c7-aaaa3ec12631/subvolume_file.1) 2024-04-20T11:33:57.326 INFO:teuthology.orchestra.run.smithi152.stdout:1316 2024-04-20T11:33:57.327 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:57.327 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol48180/bf9c0317-2e4a-4eaf-b73e-875a6eb33282/.snap/subvol_snap14510/subvolume_file.1) 2024-04-20T11:33:57.382 INFO:teuthology.orchestra.run.smithi152.stdout:1713612825 2024-04-20T11:33:57.384 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:57.384 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone89010/0630167e-6147-477c-b1c7-aaaa3ec12631/subvolume_file.1) 2024-04-20T11:33:57.438 INFO:teuthology.orchestra.run.smithi152.stdout:1713612825 2024-04-20T11:33:57.440 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs subvol48180 fake subvol_clone68251 2024-04-20T11:33:57.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:57.650+0000 7f3b27000640 1 Processor -- start 2024-04-20T11:33:57.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:57.650+0000 7f3b27000640 1 -- start start 2024-04-20T11:33:57.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:57.651+0000 7f3b27000640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f3b20160220 0x7f3b201606b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:57.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:57.652+0000 7f3b27000640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b2006fb40 con 0x7f3b20071cd0 2024-04-20T11:33:57.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:57.652+0000 7f3b27000640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b20160f90 con 0x7f3b20141450 2024-04-20T11:33:57.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:57.652+0000 7f3b27000640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f3b2013f8e0 con 0x7f3b20160220 2024-04-20T11:33:57.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:57.652+0000 7f3b25c00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f3b20071cd0 0x7f3b200720b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:42682/0 (socket says 172.21.15.152:42682) 2024-04-20T11:33:57.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:57.652+0000 7f3b25c00640 1 -- 172.21.15.152:0/387284575 learned_addr learned my addr 172.21.15.152:0/387284575 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:33:57.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:57.652+0000 7f3b1fe00640 1 -- 172.21.15.152:0/387284575 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1760373809 0 0) 0x7f3b2006fb40 con 0x7f3b20071cd0 2024-04-20T11:33:57.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:57.652+0000 7f3b25200640 1 --2- 172.21.15.152:0/387284575 >> v2:172.21.15.175:3300/0 conn(0x7f3b20160220 0x7f3b201606b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:57.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:57.652+0000 7f3b1fe00640 1 -- 172.21.15.152:0/387284575 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3b08003520 con 0x7f3b20071cd0 2024-04-20T11:33:57.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:57.653+0000 7f3b1fe00640 1 -- 172.21.15.152:0/387284575 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2293412346 0 0) 0x7f3b08003520 con 0x7f3b20071cd0 2024-04-20T11:33:57.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:57.653+0000 7f3b1fe00640 1 -- 172.21.15.152:0/387284575 >> v2:172.21.15.175:3300/0 conn(0x7f3b20160220 msgr2=0x7f3b201606b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:57.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:57.653+0000 7f3b1fe00640 1 --2- 172.21.15.152:0/387284575 >> v2:172.21.15.175:3300/0 conn(0x7f3b20160220 0x7f3b201606b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:57.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:57.653+0000 7f3b1fe00640 1 -- 172.21.15.152:0/387284575 >> v1:172.21.15.175:6789/0 conn(0x7f3b20141450 legacy=0x7f3b201418e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:57.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:57.653+0000 7f3b1fe00640 1 -- 172.21.15.152:0/387284575 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b2006dd90 con 0x7f3b20071cd0 2024-04-20T11:33:57.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:57.653+0000 7f3b1fe00640 1 -- 172.21.15.152:0/387284575 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f3b14001670 con 0x7f3b20071cd0 2024-04-20T11:33:57.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:57.653+0000 7f3b1fe00640 1 -- 172.21.15.152:0/387284575 >> v1:172.21.15.152:6789/0 conn(0x7f3b20071cd0 legacy=0x7f3b200720b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:57.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:57.653+0000 7f3b1fe00640 1 --2- 172.21.15.152:0/387284575 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3b080039e0 0x7f3b08003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:57.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:57.653+0000 7f3b1fe00640 1 --2- 172.21.15.152:0/387284575 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3b08004800 0x7f3b08004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:57.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:57.653+0000 7f3b1fe00640 1 --2- 172.21.15.152:0/387284575 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3b08005290 0x7f3b0800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:57.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:57.653+0000 7f3b1fe00640 1 -- 172.21.15.152:0/387284575 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f3b2013f8e0 con 0x7f3b08005290 2024-04-20T11:33:57.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:57.653+0000 7f3b1fe00640 1 -- 172.21.15.152:0/387284575 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3b0800c1e0 con 0x7f3b080039e0 2024-04-20T11:33:57.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:57.653+0000 7f3b1fe00640 1 -- 172.21.15.152:0/387284575 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f3b0800c3e0 con 0x7f3b08004800 2024-04-20T11:33:57.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:57.653+0000 7f3b26600640 1 --2- 172.21.15.152:0/387284575 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3b08005290 0x7f3b0800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:57.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:57.653+0000 7f3b25c00640 1 --2- 172.21.15.152:0/387284575 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3b080039e0 0x7f3b08003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:57.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:57.654+0000 7f3b25200640 1 --2- 172.21.15.152:0/387284575 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3b08004800 0x7f3b08004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:57.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:57.654+0000 7f3b26600640 1 -- 172.21.15.152:0/387284575 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3b08004800 msgr2=0x7f3b08004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:57.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:57.654+0000 7f3b26600640 1 --2- 172.21.15.152:0/387284575 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3b08004800 0x7f3b08004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:57.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:57.654+0000 7f3b26600640 1 -- 172.21.15.152:0/387284575 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3b080039e0 msgr2=0x7f3b08003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:57.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:57.654+0000 7f3b26600640 1 --2- 172.21.15.152:0/387284575 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3b080039e0 0x7f3b08003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:57.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:57.654+0000 7f3b26600640 1 -- 172.21.15.152:0/387284575 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b08003520 con 0x7f3b08005290 2024-04-20T11:33:57.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:57.655+0000 7f3b26600640 1 -- 172.21.15.152:0/387284575 reap_dead start 2024-04-20T11:33:57.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:57.655+0000 7f3b26600640 1 --2- 172.21.15.152:0/387284575 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3b08005290 0x7f3b0800b9c0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f3b18001e50 tx=0x7f3b08005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5b261246d491d54 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:57.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:57.655+0000 7f3b1fe00640 1 -- 172.21.15.152:0/387284575 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b18001a70 con 0x7f3b08005290 2024-04-20T11:33:57.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:57.655+0000 7f3b1fe00640 1 -- 172.21.15.152:0/387284575 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b18003bb0 con 0x7f3b08005290 2024-04-20T11:33:57.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:57.655+0000 7f3b1fe00640 1 -- 172.21.15.152:0/387284575 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b180050d0 con 0x7f3b08005290 2024-04-20T11:33:57.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:57.656+0000 7f3b27000640 1 -- 172.21.15.152:0/387284575 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3b08005290 msgr2=0x7f3b0800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:57.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:57.656+0000 7f3b27000640 1 --2- 172.21.15.152:0/387284575 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3b08005290 0x7f3b0800b9c0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f3b18001e50 tx=0x7f3b08005830 comp rx=0 tx=0).stop 2024-04-20T11:33:57.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:57.656+0000 7f3b27000640 1 -- 172.21.15.152:0/387284575 shutdown_connections 2024-04-20T11:33:57.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:57.656+0000 7f3b27000640 1 --2- 172.21.15.152:0/387284575 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3b08005290 0x7f3b0800b9c0 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:57.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:57.656+0000 7f3b27000640 1 -- 172.21.15.152:0/387284575 >> 172.21.15.152:0/387284575 conn(0x7f3b20067ae0 msgr2=0x7f3b2006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:57.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:57.656+0000 7f3b27000640 1 -- 172.21.15.152:0/387284575 shutdown_connections 2024-04-20T11:33:57.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:57.656+0000 7f3b27000640 1 -- 172.21.15.152:0/387284575 wait complete. 2024-04-20T11:33:57.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:57.658+0000 7f3b27000640 1 Processor -- start 2024-04-20T11:33:57.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:57.658+0000 7f3b27000640 1 -- start start 2024-04-20T11:33:57.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:57.659+0000 7f3b27000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3b08005290 0x7f3b2012f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:57.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:57.659+0000 7f3b27000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3b20158660 0x7f3b2013e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:57.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:57.659+0000 7f3b27000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3b2015b0c0 0x7f3b2015b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:57.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:57.659+0000 7f3b27000640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f3b2006e630 con 0x7f3b2015b0c0 2024-04-20T11:33:57.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:57.659+0000 7f3b27000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3b2006e350 con 0x7f3b20158660 2024-04-20T11:33:57.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:57.659+0000 7f3b27000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f3b2013fb90 con 0x7f3b08005290 2024-04-20T11:33:57.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:57.659+0000 7f3b26600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3b2015b0c0 0x7f3b2015b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:57.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:57.659+0000 7f3b26600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3b2015b0c0 0x7f3b2015b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:47092/0 (socket says 172.21.15.152:47092) 2024-04-20T11:33:57.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:57.659+0000 7f3b26600640 1 -- 172.21.15.152:0/1685354307 learned_addr learned my addr 172.21.15.152:0/1685354307 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:33:57.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:57.659+0000 7f3b25c00640 1 --2- 172.21.15.152:0/1685354307 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3b08005290 0x7f3b2012f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:57.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:57.659+0000 7f3b25200640 1 --2- 172.21.15.152:0/1685354307 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3b20158660 0x7f3b2013e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:57.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:57.659+0000 7f3b26600640 1 -- 172.21.15.152:0/1685354307 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3b08005290 msgr2=0x7f3b2012f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:57.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:57.659+0000 7f3b26600640 1 --2- 172.21.15.152:0/1685354307 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3b08005290 0x7f3b2012f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:57.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:57.659+0000 7f3b26600640 1 -- 172.21.15.152:0/1685354307 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3b20158660 msgr2=0x7f3b2013e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:57.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:57.659+0000 7f3b26600640 1 --2- 172.21.15.152:0/1685354307 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3b20158660 0x7f3b2013e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:57.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:57.659+0000 7f3b26600640 1 -- 172.21.15.152:0/1685354307 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b20130910 con 0x7f3b2015b0c0 2024-04-20T11:33:57.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:57.659+0000 7f3b25c00640 1 --2- 172.21.15.152:0/1685354307 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3b08005290 0x7f3b2012f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:33:57.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:57.660+0000 7f3b26600640 1 --2- 172.21.15.152:0/1685354307 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3b2015b0c0 0x7f3b2015b540 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f3b18001f40 tx=0x7f3b180014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:57.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:57.660+0000 7f3b1d600640 1 -- 172.21.15.152:0/1685354307 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b18006d60 con 0x7f3b2015b0c0 2024-04-20T11:33:57.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:57.660+0000 7f3b1d600640 1 -- 172.21.15.152:0/1685354307 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b18039480 con 0x7f3b2015b0c0 2024-04-20T11:33:57.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:57.660+0000 7f3b1d600640 1 -- 172.21.15.152:0/1685354307 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b18039850 con 0x7f3b2015b0c0 2024-04-20T11:33:57.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:57.660+0000 7f3b27000640 1 -- 172.21.15.152:0/1685354307 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b20130070 con 0x7f3b2015b0c0 2024-04-20T11:33:57.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:57.660+0000 7f3b27000640 1 -- 172.21.15.152:0/1685354307 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b20130630 con 0x7f3b2015b0c0 2024-04-20T11:33:57.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:57.661+0000 7f3b27000640 1 -- 172.21.15.152:0/1685354307 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b2015ba80 con 0x7f3b2015b0c0 2024-04-20T11:33:57.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:57.662+0000 7f3b1d600640 1 -- 172.21.15.152:0/1685354307 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 18) ==== 91295+0+0 (secure 0 0 0) 0x7f3b180399f0 con 0x7f3b2015b0c0 2024-04-20T11:33:57.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:57.663+0000 7f3b1d600640 1 --2- 172.21.15.152:0/1685354307 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f3b0006cbf0 0x7f3b0006f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:57.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:57.663+0000 7f3b1d600640 1 -- 172.21.15.152:0/1685354307 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7f3b180ae050 con 0x7f3b2015b0c0 2024-04-20T11:33:57.664 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:57.664+0000 7f3b25c00640 1 --2- 172.21.15.152:0/1685354307 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f3b0006cbf0 0x7f3b0006f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:57.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:57.665+0000 7f3b25c00640 1 --2- 172.21.15.152:0/1685354307 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f3b0006cbf0 0x7f3b0006f0b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3b14001eb0 tx=0x7f3b14000ef0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:57.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:57.667+0000 7f3b1d600640 1 -- 172.21.15.152:0/1685354307 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f3b18033030 con 0x7f3b2015b0c0 2024-04-20T11:33:57.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:57.905+0000 7f3b27000640 1 -- 172.21.15.152:0/1685354307 --> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "subvol48180", "snap_name": "fake", "target_sub_name": "subvol_clone68251", "target": ["mon-mgr", ""]}) -- 0x7f3b20075e60 con 0x7f3b0006cbf0 2024-04-20T11:33:58.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:58.105+0000 7f3b1d600640 1 -- 172.21.15.152:0/1685354307 <== mgr.4109 v2:172.21.15.175:6836/4021250441 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3b20075e60 con 0x7f3b0006cbf0 2024-04-20T11:33:58.110 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:58.109+0000 7f3b27000640 1 -- 172.21.15.152:0/1685354307 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f3b0006cbf0 msgr2=0x7f3b0006f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:58.110 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:58.109+0000 7f3b27000640 1 --2- 172.21.15.152:0/1685354307 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f3b0006cbf0 0x7f3b0006f0b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3b14001eb0 tx=0x7f3b14000ef0 comp rx=0 tx=0).stop 2024-04-20T11:33:58.110 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:58.109+0000 7f3b27000640 1 -- 172.21.15.152:0/1685354307 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3b2015b0c0 msgr2=0x7f3b2015b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:58.110 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:58.109+0000 7f3b27000640 1 --2- 172.21.15.152:0/1685354307 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3b2015b0c0 0x7f3b2015b540 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f3b18001f40 tx=0x7f3b180014a0 comp rx=0 tx=0).stop 2024-04-20T11:33:58.110 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:58.109+0000 7f3b27000640 1 -- 172.21.15.152:0/1685354307 shutdown_connections 2024-04-20T11:33:58.110 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:58.109+0000 7f3b27000640 1 --2- 172.21.15.152:0/1685354307 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f3b0006cbf0 0x7f3b0006f0b0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:58.110 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:58.109+0000 7f3b27000640 1 --2- 172.21.15.152:0/1685354307 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3b2015b0c0 0x7f3b2015b540 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:58.110 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:58.109+0000 7f3b27000640 1 --2- 172.21.15.152:0/1685354307 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3b20158660 0x7f3b2013e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:58.110 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:58.109+0000 7f3b27000640 1 --2- 172.21.15.152:0/1685354307 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3b08005290 0x7f3b2012f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:58.110 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:58.109+0000 7f3b27000640 1 -- 172.21.15.152:0/1685354307 >> 172.21.15.152:0/1685354307 conn(0x7f3b20067ae0 msgr2=0x7f3b2013fe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:58.110 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:58.109+0000 7f3b27000640 1 -- 172.21.15.152:0/1685354307 shutdown_connections 2024-04-20T11:33:58.111 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:58.109+0000 7f3b27000640 1 -- 172.21.15.152:0/1685354307 wait complete. 2024-04-20T11:33:58.134 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone68251 2024-04-20T11:33:58.333 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:58.333+0000 7fb452600640 1 Processor -- start 2024-04-20T11:33:58.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:58.333+0000 7fb452600640 1 -- start start 2024-04-20T11:33:58.335 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:58.334+0000 7fb452600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb44c0b2880 0x7fb44c0b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:58.335 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:58.334+0000 7fb452600640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb44c1463e0 con 0x7fb44c0cd1b0 2024-04-20T11:33:58.335 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:58.334+0000 7fb452600640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb44c0b3e20 con 0x7fb44c146000 2024-04-20T11:33:58.335 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:58.334+0000 7fb452600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb44c0b4750 con 0x7fb44c0b2880 2024-04-20T11:33:58.336 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:58.335+0000 7fb44be00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fb44c0cd1b0 0x7fb44c0cd600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:42684/0 (socket says 172.21.15.152:42684) 2024-04-20T11:33:58.336 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:58.335+0000 7fb44be00640 1 -- 172.21.15.152:0/173721258 learned_addr learned my addr 172.21.15.152:0/173721258 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:33:58.336 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:58.335+0000 7fb443e00640 1 --2- 172.21.15.152:0/173721258 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb44c0b2880 0x7fb44c0b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:58.336 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:58.335+0000 7fb44aa00640 1 -- 172.21.15.152:0/173721258 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1559108488 0 0) 0x7fb44c1463e0 con 0x7fb44c0cd1b0 2024-04-20T11:33:58.336 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:58.335+0000 7fb44aa00640 1 -- 172.21.15.152:0/173721258 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb42c003520 con 0x7fb44c0cd1b0 2024-04-20T11:33:58.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:58.336+0000 7fb44aa00640 1 -- 172.21.15.152:0/173721258 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1687774592 0 0) 0x7fb42c003520 con 0x7fb44c0cd1b0 2024-04-20T11:33:58.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:58.336+0000 7fb44aa00640 1 -- 172.21.15.152:0/173721258 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb44c0b2880 msgr2=0x7fb44c0b4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:58.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:58.336+0000 7fb44aa00640 1 --2- 172.21.15.152:0/173721258 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb44c0b2880 0x7fb44c0b4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:58.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:58.336+0000 7fb44aa00640 1 -- 172.21.15.152:0/173721258 >> v1:172.21.15.175:6789/0 conn(0x7fb44c146000 legacy=0x7fb44c0b2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:58.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:58.336+0000 7fb44aa00640 1 -- 172.21.15.152:0/173721258 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb44c0b4100 con 0x7fb44c0cd1b0 2024-04-20T11:33:58.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:58.336+0000 7fb44aa00640 1 -- 172.21.15.152:0/173721258 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7fb4440019f0 con 0x7fb44c0cd1b0 2024-04-20T11:33:58.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:58.336+0000 7fb44aa00640 1 -- 172.21.15.152:0/173721258 >> v1:172.21.15.152:6789/0 conn(0x7fb44c0cd1b0 legacy=0x7fb44c0cd600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:58.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:58.336+0000 7fb44aa00640 1 --2- 172.21.15.152:0/173721258 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb44c0b2880 0x7fb42c0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:58.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:58.336+0000 7fb44aa00640 1 --2- 172.21.15.152:0/173721258 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb42c003fb0 0x7fb42c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:58.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:58.336+0000 7fb44aa00640 1 --2- 172.21.15.152:0/173721258 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb42c004d40 0x7fb42c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:58.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:58.336+0000 7fb44aa00640 1 -- 172.21.15.152:0/173721258 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fb44c0b4750 con 0x7fb42c003fb0 2024-04-20T11:33:58.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:58.336+0000 7fb44aa00640 1 -- 172.21.15.152:0/173721258 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb42c00bbb0 con 0x7fb42c004d40 2024-04-20T11:33:58.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:58.336+0000 7fb44aa00640 1 -- 172.21.15.152:0/173721258 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb42c00bdb0 con 0x7fb44c0b2880 2024-04-20T11:33:58.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:58.336+0000 7fb443e00640 1 --2- 172.21.15.152:0/173721258 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb42c003fb0 0x7fb42c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:58.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:58.336+0000 7fb44b400640 1 --2- 172.21.15.152:0/173721258 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb44c0b2880 0x7fb42c0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:58.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:58.337+0000 7fb44be00640 1 --2- 172.21.15.152:0/173721258 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb42c004d40 0x7fb42c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:58.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:58.337+0000 7fb443e00640 1 -- 172.21.15.152:0/173721258 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb44c0b2880 msgr2=0x7fb42c0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:58.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:58.337+0000 7fb443e00640 1 --2- 172.21.15.152:0/173721258 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb44c0b2880 0x7fb42c0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:58.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:58.337+0000 7fb443e00640 1 -- 172.21.15.152:0/173721258 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb42c004d40 msgr2=0x7fb42c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:33:58.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:58.337+0000 7fb443e00640 1 --2- 172.21.15.152:0/173721258 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb42c004d40 0x7fb42c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:58.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:58.337+0000 7fb443e00640 1 -- 172.21.15.152:0/173721258 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb42c003520 con 0x7fb42c003fb0 2024-04-20T11:33:58.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:58.338+0000 7fb443e00640 1 --2- 172.21.15.152:0/173721258 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb42c003fb0 0x7fb42c004800 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7fb434009ac0 tx=0x7fb42c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75703d14e1b527c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:58.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:58.338+0000 7fb44aa00640 1 -- 172.21.15.152:0/173721258 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb434039070 con 0x7fb42c003fb0 2024-04-20T11:33:58.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:58.338+0000 7fb44aa00640 1 -- 172.21.15.152:0/173721258 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb43402c6b0 con 0x7fb42c003fb0 2024-04-20T11:33:58.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:58.338+0000 7fb44aa00640 1 -- 172.21.15.152:0/173721258 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb434009690 con 0x7fb42c003fb0 2024-04-20T11:33:58.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:58.339+0000 7fb452600640 1 -- 172.21.15.152:0/173721258 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb42c003fb0 msgr2=0x7fb42c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:58.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:58.339+0000 7fb452600640 1 --2- 172.21.15.152:0/173721258 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb42c003fb0 0x7fb42c004800 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7fb434009ac0 tx=0x7fb42c005830 comp rx=0 tx=0).stop 2024-04-20T11:33:58.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:58.339+0000 7fb44be00640 1 -- 172.21.15.152:0/173721258 reap_dead start 2024-04-20T11:33:58.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:58.339+0000 7fb452600640 1 -- 172.21.15.152:0/173721258 shutdown_connections 2024-04-20T11:33:58.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:58.339+0000 7fb452600640 1 -- 172.21.15.152:0/173721258 >> 172.21.15.152:0/173721258 conn(0x7fb44c06b980 msgr2=0x7fb44c06bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:58.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:58.339+0000 7fb452600640 1 -- 172.21.15.152:0/173721258 shutdown_connections 2024-04-20T11:33:58.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:58.339+0000 7fb452600640 1 -- 172.21.15.152:0/173721258 wait complete. 2024-04-20T11:33:58.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:58.341+0000 7fb452600640 1 Processor -- start 2024-04-20T11:33:58.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:58.341+0000 7fb452600640 1 -- start start 2024-04-20T11:33:58.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:58.342+0000 7fb452600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb42c003fb0 0x7fb44c13d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:58.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:58.342+0000 7fb452600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb44c0c9c30 0x7fb44c13d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:58.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:58.342+0000 7fb452600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb44c13deb0 0x7fb44c13f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:58.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:58.342+0000 7fb452600640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fb44c0b4d60 con 0x7fb42c003fb0 2024-04-20T11:33:58.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:58.342+0000 7fb452600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb44c0b4be0 con 0x7fb44c13deb0 2024-04-20T11:33:58.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:58.342+0000 7fb452600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb44c0b4ee0 con 0x7fb44c0c9c30 2024-04-20T11:33:58.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:58.342+0000 7fb44b400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb42c003fb0 0x7fb44c13d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:58.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:58.342+0000 7fb44b400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb42c003fb0 0x7fb44c13d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:47114/0 (socket says 172.21.15.152:47114) 2024-04-20T11:33:58.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:58.342+0000 7fb44b400640 1 -- 172.21.15.152:0/789438993 learned_addr learned my addr 172.21.15.152:0/789438993 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:33:58.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:58.342+0000 7fb44b400640 1 -- 172.21.15.152:0/789438993 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb44c0c9c30 msgr2=0x7fb44c13d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:58.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:58.342+0000 7fb44b400640 1 --2- 172.21.15.152:0/789438993 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb44c0c9c30 0x7fb44c13d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:58.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:58.342+0000 7fb44b400640 1 -- 172.21.15.152:0/789438993 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb44c13deb0 msgr2=0x7fb44c13f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:58.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:58.342+0000 7fb44b400640 1 --2- 172.21.15.152:0/789438993 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb44c13deb0 0x7fb44c13f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:58.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:58.342+0000 7fb44b400640 1 -- 172.21.15.152:0/789438993 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb44c13d630 con 0x7fb42c003fb0 2024-04-20T11:33:58.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:58.343+0000 7fb44b400640 1 --2- 172.21.15.152:0/789438993 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb42c003fb0 0x7fb44c13d0f0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7fb43c00e580 tx=0x7fb43c00ea40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:58.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:58.343+0000 7fb443400640 1 -- 172.21.15.152:0/789438993 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb43c0052e0 con 0x7fb42c003fb0 2024-04-20T11:33:58.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:58.343+0000 7fb443400640 1 -- 172.21.15.152:0/789438993 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb43c018070 con 0x7fb42c003fb0 2024-04-20T11:33:58.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:58.343+0000 7fb452600640 1 -- 172.21.15.152:0/789438993 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb44c0cb440 con 0x7fb42c003fb0 2024-04-20T11:33:58.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:58.343+0000 7fb443400640 1 -- 172.21.15.152:0/789438993 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb43c003810 con 0x7fb42c003fb0 2024-04-20T11:33:58.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:58.343+0000 7fb452600640 1 -- 172.21.15.152:0/789438993 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb44c0cba00 con 0x7fb42c003fb0 2024-04-20T11:33:58.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:58.344+0000 7fb452600640 1 -- 172.21.15.152:0/789438993 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb44c13e2d0 con 0x7fb42c003fb0 2024-04-20T11:33:58.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:58.345+0000 7fb443400640 1 -- 172.21.15.152:0/789438993 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 18) ==== 91295+0+0 (secure 0 0 0) 0x7fb43c0039b0 con 0x7fb42c003fb0 2024-04-20T11:33:58.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:58.346+0000 7fb443400640 1 --2- 172.21.15.152:0/789438993 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb42806cbf0 0x7fb42806f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:58.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:58.346+0000 7fb443400640 1 -- 172.21.15.152:0/789438993 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7fb43c08bcd0 con 0x7fb42c003fb0 2024-04-20T11:33:58.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:58.347+0000 7fb443e00640 1 --2- 172.21.15.152:0/789438993 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb42806cbf0 0x7fb42806f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:58.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:58.348+0000 7fb443e00640 1 --2- 172.21.15.152:0/789438993 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb42806cbf0 0x7fb42806f0b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb434036000 tx=0x7fb43402e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:58.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:58.350+0000 7fb443400640 1 -- 172.21.15.152:0/789438993 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fb43c059d80 con 0x7fb42c003fb0 2024-04-20T11:33:58.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:33:58.587+0000 7fb452600640 1 -- 172.21.15.152:0/789438993 --> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone68251", "target": ["mon-mgr", ""]}) -- 0x7fb44c15d650 con 0x7fb42806cbf0 2024-04-20T11:34:01.370 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:01.370+0000 7fb443400640 1 -- 172.21.15.152:0/789438993 <== mgr.4109 v2:172.21.15.175:6836/4021250441 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+153 (secure 0 0 0) 0x7fb44c15d650 con 0x7fb42806cbf0 2024-04-20T11:34:01.370 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-04-20T11:34:01.370 INFO:teuthology.orchestra.run.smithi152.stdout: "status": { 2024-04-20T11:34:01.370 INFO:teuthology.orchestra.run.smithi152.stdout: "state": "pending", 2024-04-20T11:34:01.370 INFO:teuthology.orchestra.run.smithi152.stdout: "source": { 2024-04-20T11:34:01.370 INFO:teuthology.orchestra.run.smithi152.stdout: "volume": "cephfs", 2024-04-20T11:34:01.370 INFO:teuthology.orchestra.run.smithi152.stdout: "subvolume": "subvol48180", 2024-04-20T11:34:01.370 INFO:teuthology.orchestra.run.smithi152.stdout: "snapshot": "fake" 2024-04-20T11:34:01.371 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:34:01.371 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:34:01.371 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-04-20T11:34:01.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:01.373+0000 7fb452600640 1 -- 172.21.15.152:0/789438993 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb42806cbf0 msgr2=0x7fb42806f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:01.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:01.373+0000 7fb452600640 1 --2- 172.21.15.152:0/789438993 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb42806cbf0 0x7fb42806f0b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb434036000 tx=0x7fb43402e000 comp rx=0 tx=0).stop 2024-04-20T11:34:01.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:01.373+0000 7fb452600640 1 -- 172.21.15.152:0/789438993 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb42c003fb0 msgr2=0x7fb44c13d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:01.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:01.373+0000 7fb452600640 1 --2- 172.21.15.152:0/789438993 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb42c003fb0 0x7fb44c13d0f0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7fb43c00e580 tx=0x7fb43c00ea40 comp rx=0 tx=0).stop 2024-04-20T11:34:01.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:01.373+0000 7fb452600640 1 -- 172.21.15.152:0/789438993 shutdown_connections 2024-04-20T11:34:01.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:01.373+0000 7fb452600640 1 --2- 172.21.15.152:0/789438993 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb42806cbf0 0x7fb42806f0b0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:01.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:01.373+0000 7fb452600640 1 --2- 172.21.15.152:0/789438993 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb44c13deb0 0x7fb44c13f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:01.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:01.373+0000 7fb452600640 1 --2- 172.21.15.152:0/789438993 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb44c0c9c30 0x7fb44c13d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:01.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:01.373+0000 7fb452600640 1 --2- 172.21.15.152:0/789438993 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb42c003fb0 0x7fb44c13d0f0 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:01.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:01.373+0000 7fb452600640 1 -- 172.21.15.152:0/789438993 >> 172.21.15.152:0/789438993 conn(0x7fb44c06b980 msgr2=0x7fb44c0d0430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:01.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:01.374+0000 7fb452600640 1 -- 172.21.15.152:0/789438993 shutdown_connections 2024-04-20T11:34:01.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:01.374+0000 7fb452600640 1 -- 172.21.15.152:0/789438993 wait complete. 2024-04-20T11:34:02.402 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone68251 2024-04-20T11:34:02.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:02.619+0000 7fd8c9c00640 1 Processor -- start 2024-04-20T11:34:02.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:02.619+0000 7fd8c9c00640 1 -- start start 2024-04-20T11:34:02.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:02.621+0000 7fd8c9c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd8c406c4b0 0x7fd8c406dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:02.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:02.621+0000 7fd8c9c00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8c40b5070 con 0x7fd8c415ea20 2024-04-20T11:34:02.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:02.621+0000 7fd8c9c00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8c406e1a0 con 0x7fd8c415e640 2024-04-20T11:34:02.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:02.621+0000 7fd8c9c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd8c406e480 con 0x7fd8c406c4b0 2024-04-20T11:34:02.623 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:02.621+0000 7fd8bbe00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fd8c415ea20 0x7fd8c40ad5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:42698/0 (socket says 172.21.15.152:42698) 2024-04-20T11:34:02.623 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:02.621+0000 7fd8bbe00640 1 -- 172.21.15.152:0/980610241 learned_addr learned my addr 172.21.15.152:0/980610241 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:34:02.623 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:02.621+0000 7fd8c9200640 1 --2- 172.21.15.152:0/980610241 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd8c406c4b0 0x7fd8c406dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:02.623 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:02.621+0000 7fd8c3400640 1 -- 172.21.15.152:0/980610241 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 385691702 0 0) 0x7fd8c40b5070 con 0x7fd8c415ea20 2024-04-20T11:34:02.623 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:02.621+0000 7fd8c3400640 1 -- 172.21.15.152:0/980610241 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd8a8003520 con 0x7fd8c415ea20 2024-04-20T11:34:02.623 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:02.622+0000 7fd8c3400640 1 -- 172.21.15.152:0/980610241 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4125406394 0 0) 0x7fd8a8003520 con 0x7fd8c415ea20 2024-04-20T11:34:02.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:02.622+0000 7fd8c3400640 1 -- 172.21.15.152:0/980610241 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd8c406c4b0 msgr2=0x7fd8c406dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:02.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:02.622+0000 7fd8c3400640 1 --2- 172.21.15.152:0/980610241 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd8c406c4b0 0x7fd8c406dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:02.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:02.622+0000 7fd8c3400640 1 -- 172.21.15.152:0/980610241 >> v1:172.21.15.175:6789/0 conn(0x7fd8c415e640 legacy=0x7fd8c406bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:02.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:02.622+0000 7fd8c3400640 1 -- 172.21.15.152:0/980610241 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8c406e760 con 0x7fd8c415ea20 2024-04-20T11:34:02.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:02.622+0000 7fd8c3400640 1 -- 172.21.15.152:0/980610241 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7fd8ac001670 con 0x7fd8c415ea20 2024-04-20T11:34:02.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:02.622+0000 7fd8c3400640 1 -- 172.21.15.152:0/980610241 >> v1:172.21.15.152:6789/0 conn(0x7fd8c415ea20 legacy=0x7fd8c40ad5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:02.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:02.622+0000 7fd8c3400640 1 --2- 172.21.15.152:0/980610241 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd8a80039e0 0x7fd8a8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:02.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:02.622+0000 7fd8c3400640 1 --2- 172.21.15.152:0/980610241 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd8a8004800 0x7fd8a8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:02.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:02.622+0000 7fd8c3400640 1 --2- 172.21.15.152:0/980610241 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd8c406c4b0 0x7fd8a80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:02.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:02.622+0000 7fd8c3400640 1 -- 172.21.15.152:0/980610241 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c406e480 con 0x7fd8a8004800 2024-04-20T11:34:02.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:02.622+0000 7fd8c3400640 1 -- 172.21.15.152:0/980610241 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd8a800baa0 con 0x7fd8a80039e0 2024-04-20T11:34:02.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:02.622+0000 7fd8c3400640 1 -- 172.21.15.152:0/980610241 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd8a800bca0 con 0x7fd8c406c4b0 2024-04-20T11:34:02.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:02.622+0000 7fd8bbe00640 1 --2- 172.21.15.152:0/980610241 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd8a8004800 0x7fd8a8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:02.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:02.622+0000 7fd8c3e00640 1 --2- 172.21.15.152:0/980610241 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd8a80039e0 0x7fd8a8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:02.625 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:02.622+0000 7fd8c9200640 1 --2- 172.21.15.152:0/980610241 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd8c406c4b0 0x7fd8a80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:02.625 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:02.623+0000 7fd8bbe00640 1 -- 172.21.15.152:0/980610241 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd8c406c4b0 msgr2=0x7fd8a80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:02.625 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:02.623+0000 7fd8bbe00640 1 --2- 172.21.15.152:0/980610241 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd8c406c4b0 0x7fd8a80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:02.625 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:02.623+0000 7fd8bbe00640 1 -- 172.21.15.152:0/980610241 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd8a80039e0 msgr2=0x7fd8a8003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:02.625 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:02.623+0000 7fd8bbe00640 1 --2- 172.21.15.152:0/980610241 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd8a80039e0 0x7fd8a8003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:02.625 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:02.623+0000 7fd8bbe00640 1 -- 172.21.15.152:0/980610241 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8a8003520 con 0x7fd8a8004800 2024-04-20T11:34:02.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:02.624+0000 7fd8bbe00640 1 --2- 172.21.15.152:0/980610241 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd8a8004800 0x7fd8a8004c80 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7fd8c40afb70 tx=0x7fd8a8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45f7be427ffa5517 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:02.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:02.624+0000 7fd8c3400640 1 -- 172.21.15.152:0/980610241 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8ac007cf0 con 0x7fd8a8004800 2024-04-20T11:34:02.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:02.624+0000 7fd8c3400640 1 -- 172.21.15.152:0/980610241 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8ac030070 con 0x7fd8a8004800 2024-04-20T11:34:02.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:02.624+0000 7fd8c3400640 1 -- 172.21.15.152:0/980610241 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8ac005d30 con 0x7fd8a8004800 2024-04-20T11:34:02.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:02.625+0000 7fd8c9c00640 1 -- 172.21.15.152:0/980610241 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd8a8004800 msgr2=0x7fd8a8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:02.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:02.625+0000 7fd8c9c00640 1 --2- 172.21.15.152:0/980610241 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd8a8004800 0x7fd8a8004c80 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7fd8c40afb70 tx=0x7fd8a8005830 comp rx=0 tx=0).stop 2024-04-20T11:34:02.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:02.625+0000 7fd8c9200640 1 -- 172.21.15.152:0/980610241 reap_dead start 2024-04-20T11:34:02.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:02.625+0000 7fd8c9c00640 1 -- 172.21.15.152:0/980610241 shutdown_connections 2024-04-20T11:34:02.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:02.625+0000 7fd8c9c00640 1 -- 172.21.15.152:0/980610241 >> 172.21.15.152:0/980610241 conn(0x7fd8c40b21e0 msgr2=0x7fd8c40b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:02.627 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:02.625+0000 7fd8c9c00640 1 -- 172.21.15.152:0/980610241 shutdown_connections 2024-04-20T11:34:02.627 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:02.625+0000 7fd8c9c00640 1 -- 172.21.15.152:0/980610241 wait complete. 2024-04-20T11:34:02.627 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:02.627+0000 7fd8c9c00640 1 Processor -- start 2024-04-20T11:34:02.627 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:02.627+0000 7fd8c9c00640 1 -- start start 2024-04-20T11:34:02.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:02.628+0000 7fd8c9c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd8a8004800 0x7fd8c4156790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:02.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:02.628+0000 7fd8c9c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd8c414bc30 0x7fd8c414b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:02.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:02.628+0000 7fd8c9c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd8c414b5b0 0x7fd8c415fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:02.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:02.628+0000 7fd8c9c00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c406f410 con 0x7fd8a8004800 2024-04-20T11:34:02.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:02.628+0000 7fd8c9c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c40ad0a0 con 0x7fd8c414b5b0 2024-04-20T11:34:02.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:02.628+0000 7fd8c9c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd8c406f590 con 0x7fd8c414bc30 2024-04-20T11:34:02.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:02.628+0000 7fd8c3e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd8a8004800 0x7fd8c4156790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:02.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:02.628+0000 7fd8c3e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd8a8004800 0x7fd8c4156790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:47132/0 (socket says 172.21.15.152:47132) 2024-04-20T11:34:02.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:02.628+0000 7fd8c3e00640 1 -- 172.21.15.152:0/1165011946 learned_addr learned my addr 172.21.15.152:0/1165011946 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:02.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:02.628+0000 7fd8bbe00640 1 --2- 172.21.15.152:0/1165011946 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd8c414bc30 0x7fd8c414b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:02.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:02.628+0000 7fd8c9200640 1 --2- 172.21.15.152:0/1165011946 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd8c414b5b0 0x7fd8c415fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:02.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:02.629+0000 7fd8c3e00640 1 -- 172.21.15.152:0/1165011946 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd8c414bc30 msgr2=0x7fd8c414b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:02.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:02.629+0000 7fd8c3e00640 1 --2- 172.21.15.152:0/1165011946 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd8c414bc30 0x7fd8c414b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:02.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:02.629+0000 7fd8c3e00640 1 -- 172.21.15.152:0/1165011946 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd8c414b5b0 msgr2=0x7fd8c415fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:02.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:02.629+0000 7fd8c3e00640 1 --2- 172.21.15.152:0/1165011946 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd8c414b5b0 0x7fd8c415fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:02.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:02.629+0000 7fd8c3e00640 1 -- 172.21.15.152:0/1165011946 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8c4159c40 con 0x7fd8a8004800 2024-04-20T11:34:02.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:02.629+0000 7fd8c9200640 1 --2- 172.21.15.152:0/1165011946 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd8c414b5b0 0x7fd8c415fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:34:02.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:02.629+0000 7fd8c3e00640 1 --2- 172.21.15.152:0/1165011946 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd8a8004800 0x7fd8c4156790 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7fd8b400e500 tx=0x7fd8b400e9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:02.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:02.629+0000 7fd8c0c00640 1 -- 172.21.15.152:0/1165011946 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8b40052e0 con 0x7fd8a8004800 2024-04-20T11:34:02.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:02.629+0000 7fd8c0c00640 1 -- 172.21.15.152:0/1165011946 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8b4018070 con 0x7fd8a8004800 2024-04-20T11:34:02.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:02.629+0000 7fd8c0c00640 1 -- 172.21.15.152:0/1165011946 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8b4003810 con 0x7fd8a8004800 2024-04-20T11:34:02.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:02.629+0000 7fd8c9c00640 1 -- 172.21.15.152:0/1165011946 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8c414ba30 con 0x7fd8a8004800 2024-04-20T11:34:02.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:02.629+0000 7fd8c9c00640 1 -- 172.21.15.152:0/1165011946 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8c415a1a0 con 0x7fd8a8004800 2024-04-20T11:34:02.632 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:02.631+0000 7fd8c9c00640 1 -- 172.21.15.152:0/1165011946 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8c4075e60 con 0x7fd8a8004800 2024-04-20T11:34:02.632 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:02.632+0000 7fd8c0c00640 1 -- 172.21.15.152:0/1165011946 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 18) ==== 91295+0+0 (secure 0 0 0) 0x7fd8b40039b0 con 0x7fd8a8004800 2024-04-20T11:34:02.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:02.632+0000 7fd8c0c00640 1 --2- 172.21.15.152:0/1165011946 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd8a006cbc0 0x7fd8a006f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:02.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:02.633+0000 7fd8c0c00640 1 -- 172.21.15.152:0/1165011946 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7fd8b408bcc0 con 0x7fd8a8004800 2024-04-20T11:34:02.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:02.633+0000 7fd8bbe00640 1 --2- 172.21.15.152:0/1165011946 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd8a006cbc0 0x7fd8a006f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:02.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:02.634+0000 7fd8bbe00640 1 --2- 172.21.15.152:0/1165011946 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd8a006cbc0 0x7fd8a006f080 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac034000 tx=0x7fd8ac007000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:02.638 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:02.637+0000 7fd8c0c00640 1 -- 172.21.15.152:0/1165011946 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fd8b4092080 con 0x7fd8a8004800 2024-04-20T11:34:02.878 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:02.878+0000 7fd8c9c00640 1 -- 172.21.15.152:0/1165011946 --> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone68251", "target": ["mon-mgr", ""]}) -- 0x7fd8c40cfa80 con 0x7fd8a006cbc0 2024-04-20T11:34:06.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:06.353+0000 7fd8c0c00640 1 -- 172.21.15.152:0/1165011946 <== mgr.4109 v2:172.21.15.175:6836/4021250441 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+46 (secure 0 0 0) 0x7fd8c40cfa80 con 0x7fd8a006cbc0 2024-04-20T11:34:06.353 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-04-20T11:34:06.353 INFO:teuthology.orchestra.run.smithi152.stdout: "status": { 2024-04-20T11:34:06.354 INFO:teuthology.orchestra.run.smithi152.stdout: "state": "complete" 2024-04-20T11:34:06.354 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:34:06.354 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-04-20T11:34:06.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:06.357+0000 7fd8c9c00640 1 -- 172.21.15.152:0/1165011946 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd8a006cbc0 msgr2=0x7fd8a006f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:06.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:06.357+0000 7fd8c9c00640 1 --2- 172.21.15.152:0/1165011946 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd8a006cbc0 0x7fd8a006f080 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac034000 tx=0x7fd8ac007000 comp rx=0 tx=0).stop 2024-04-20T11:34:06.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:06.357+0000 7fd8c9c00640 1 -- 172.21.15.152:0/1165011946 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd8a8004800 msgr2=0x7fd8c4156790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:06.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:06.357+0000 7fd8c9c00640 1 --2- 172.21.15.152:0/1165011946 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd8a8004800 0x7fd8c4156790 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7fd8b400e500 tx=0x7fd8b400e9c0 comp rx=0 tx=0).stop 2024-04-20T11:34:06.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:06.357+0000 7fd8c9c00640 1 -- 172.21.15.152:0/1165011946 shutdown_connections 2024-04-20T11:34:06.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:06.358+0000 7fd8c9c00640 1 --2- 172.21.15.152:0/1165011946 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd8a006cbc0 0x7fd8a006f080 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:06.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:06.358+0000 7fd8c9c00640 1 --2- 172.21.15.152:0/1165011946 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd8c414b5b0 0x7fd8c415fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:06.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:06.358+0000 7fd8c9c00640 1 --2- 172.21.15.152:0/1165011946 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd8c414bc30 0x7fd8c414b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:06.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:06.358+0000 7fd8c9c00640 1 --2- 172.21.15.152:0/1165011946 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd8a8004800 0x7fd8c4156790 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:06.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:06.358+0000 7fd8c9c00640 1 -- 172.21.15.152:0/1165011946 >> 172.21.15.152:0/1165011946 conn(0x7fd8c40b21e0 msgr2=0x7fd8c40ad4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:06.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:06.358+0000 7fd8c9c00640 1 -- 172.21.15.152:0/1165011946 shutdown_connections 2024-04-20T11:34:06.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:06.358+0000 7fd8c9c00640 1 -- 172.21.15.152:0/1165011946 wait complete. 2024-04-20T11:34:06.378 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:34:06.379 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_nogroup/subvol_clone68251/.meta) 2024-04-20T11:34:06.457 INFO:teuthology.orchestra.run.smithi152.stdout:[GLOBAL] 2024-04-20T11:34:06.457 INFO:teuthology.orchestra.run.smithi152.stdout:version = 2 2024-04-20T11:34:06.457 INFO:teuthology.orchestra.run.smithi152.stdout:type = clone 2024-04-20T11:34:06.457 INFO:teuthology.orchestra.run.smithi152.stdout:path = /volumes/_nogroup/subvol_clone68251/32c9c364-ff2a-477d-8e47-3b41e360f2eb 2024-04-20T11:34:06.457 INFO:teuthology.orchestra.run.smithi152.stdout:state = complete 2024-04-20T11:34:06.457 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:34:06.461 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot info cephfs subvol48180 subvol_snap14510 2024-04-20T11:34:06.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:06.669+0000 7f945be00640 1 Processor -- start 2024-04-20T11:34:06.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:06.669+0000 7f945be00640 1 -- start start 2024-04-20T11:34:06.670 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:06.670+0000 7f945be00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f9454071e20 0x7f9454072200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:06.670 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:06.670+0000 7f945be00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f94540b3470 0x7f94540b5100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:06.670 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:06.671+0000 7f945be00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9454163130 con 0x7f94541635b0 2024-04-20T11:34:06.670 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:06.671+0000 7f945be00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f94541676b0 con 0x7f94540b3470 2024-04-20T11:34:06.670 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:06.671+0000 7f945be00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f94540b4e70 con 0x7f9454071e20 2024-04-20T11:34:06.671 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:06.671+0000 7f9458a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f94540b3470 0x7f94540b5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:06.671 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:06.671+0000 7f9459e00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f94541635b0 0x7f9454163a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:34988/0 (socket says 172.21.15.152:34988) 2024-04-20T11:34:06.671 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:06.671+0000 7f9459e00640 1 -- 172.21.15.152:0/324765883 learned_addr learned my addr 172.21.15.152:0/324765883 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:34:06.671 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:06.671+0000 7f9459400640 1 --2- 172.21.15.152:0/324765883 >> v2:172.21.15.175:3300/0 conn(0x7f9454071e20 0x7f9454072200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:06.671 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:06.671+0000 7f9458a00640 1 -- 172.21.15.152:0/324765883 >> v2:172.21.15.175:3300/0 conn(0x7f9454071e20 msgr2=0x7f9454072200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:06.671 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:06.671+0000 7f9458a00640 1 --2- 172.21.15.152:0/324765883 >> v2:172.21.15.175:3300/0 conn(0x7f9454071e20 0x7f9454072200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:06.671 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:06.671+0000 7f9458a00640 1 -- 172.21.15.152:0/324765883 >> v1:172.21.15.152:6789/0 conn(0x7f94541635b0 legacy=0x7f9454163a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:06.671 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:06.672+0000 7f9458a00640 1 -- 172.21.15.152:0/324765883 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9454072740 con 0x7f94540b3470 2024-04-20T11:34:06.672 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:06.673+0000 7f9458a00640 1 --2- 172.21.15.152:0/324765883 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f94540b3470 0x7f94540b5100 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f9454073150 tx=0x7f944002b680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d18c38315352e03 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:06.672 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:06.673+0000 7f9453e00640 1 -- 172.21.15.152:0/324765883 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9440037070 con 0x7f94540b3470 2024-04-20T11:34:06.672 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:06.673+0000 7f9453e00640 1 -- 172.21.15.152:0/324765883 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9440030710 con 0x7f94540b3470 2024-04-20T11:34:06.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:06.673+0000 7f9453e00640 1 -- 172.21.15.152:0/324765883 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9440030ae0 con 0x7f94540b3470 2024-04-20T11:34:06.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:06.674+0000 7f945be00640 1 -- 172.21.15.152:0/324765883 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f94540b3470 msgr2=0x7f94540b5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:06.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:06.674+0000 7f945be00640 1 --2- 172.21.15.152:0/324765883 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f94540b3470 0x7f94540b5100 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f9454073150 tx=0x7f944002b680 comp rx=0 tx=0).stop 2024-04-20T11:34:06.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:06.674+0000 7f945be00640 1 -- 172.21.15.152:0/324765883 shutdown_connections 2024-04-20T11:34:06.674 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:06.674+0000 7f945be00640 1 --2- 172.21.15.152:0/324765883 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f94540b3470 0x7f94540b5100 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:06.674 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:06.674+0000 7f945be00640 1 --2- 172.21.15.152:0/324765883 >> v2:172.21.15.175:3300/0 conn(0x7f9454071e20 0x7f9454072200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:06.674 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:06.674+0000 7f945be00640 1 -- 172.21.15.152:0/324765883 >> 172.21.15.152:0/324765883 conn(0x7f945406b980 msgr2=0x7f945406bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:06.674 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:06.674+0000 7f945be00640 1 -- 172.21.15.152:0/324765883 shutdown_connections 2024-04-20T11:34:06.674 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:06.674+0000 7f945be00640 1 -- 172.21.15.152:0/324765883 wait complete. 2024-04-20T11:34:06.675 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:06.676+0000 7f945be00640 1 Processor -- start 2024-04-20T11:34:06.675 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:06.676+0000 7f945be00640 1 -- start start 2024-04-20T11:34:06.678 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:06.677+0000 7f945be00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9454071e20 0x7f9454143df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:06.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:06.677+0000 7f945be00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94540b3470 0x7f945415c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:06.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:06.677+0000 7f945be00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f94541635b0 0x7f9454146520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:06.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:06.677+0000 7f945be00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f9454166e10 con 0x7f94541635b0 2024-04-20T11:34:06.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:06.677+0000 7f945be00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9454164130 con 0x7f9454071e20 2024-04-20T11:34:06.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:06.677+0000 7f945be00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f94541670f0 con 0x7f94540b3470 2024-04-20T11:34:06.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:06.677+0000 7f9459e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f94541635b0 0x7f9454146520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:06.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:06.677+0000 7f9459e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f94541635b0 0x7f9454146520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:53254/0 (socket says 172.21.15.152:53254) 2024-04-20T11:34:06.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:06.678+0000 7f9459e00640 1 -- 172.21.15.152:0/2327217198 learned_addr learned my addr 172.21.15.152:0/2327217198 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:06.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:06.678+0000 7f9458a00640 1 --2- 172.21.15.152:0/2327217198 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94540b3470 0x7f945415c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:06.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:06.678+0000 7f9459400640 1 --2- 172.21.15.152:0/2327217198 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9454071e20 0x7f9454143df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:06.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:06.678+0000 7f9459e00640 1 -- 172.21.15.152:0/2327217198 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94540b3470 msgr2=0x7f945415c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:06.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:06.678+0000 7f9459e00640 1 --2- 172.21.15.152:0/2327217198 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94540b3470 0x7f945415c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:06.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:06.678+0000 7f9459e00640 1 -- 172.21.15.152:0/2327217198 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9454071e20 msgr2=0x7f9454143df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:06.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:06.678+0000 7f9459e00640 1 --2- 172.21.15.152:0/2327217198 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9454071e20 0x7f9454143df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:06.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:06.678+0000 7f9459e00640 1 -- 172.21.15.152:0/2327217198 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f945415ce80 con 0x7f94541635b0 2024-04-20T11:34:06.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:06.678+0000 7f9459e00640 1 --2- 172.21.15.152:0/2327217198 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f94541635b0 0x7f9454146520 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f944c00caf0 tx=0x7f944c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:06.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:06.679+0000 7f9451600640 1 -- 172.21.15.152:0/2327217198 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f944c018070 con 0x7f94541635b0 2024-04-20T11:34:06.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:06.679+0000 7f9451600640 1 -- 172.21.15.152:0/2327217198 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f944c00a890 con 0x7f94541635b0 2024-04-20T11:34:06.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:06.679+0000 7f9451600640 1 -- 172.21.15.152:0/2327217198 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f944c00ac80 con 0x7f94541635b0 2024-04-20T11:34:06.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:06.679+0000 7f945be00640 1 -- 172.21.15.152:0/2327217198 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f945415d160 con 0x7f94541635b0 2024-04-20T11:34:06.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:06.679+0000 7f945be00640 1 -- 172.21.15.152:0/2327217198 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f945415d720 con 0x7f94541635b0 2024-04-20T11:34:06.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:06.680+0000 7f945be00640 1 -- 172.21.15.152:0/2327217198 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9454068a60 con 0x7f94541635b0 2024-04-20T11:34:06.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:06.681+0000 7f9451600640 1 -- 172.21.15.152:0/2327217198 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 18) ==== 91295+0+0 (secure 0 0 0) 0x7f944c01b070 con 0x7f94541635b0 2024-04-20T11:34:06.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:06.682+0000 7f9451600640 1 --2- 172.21.15.152:0/2327217198 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f943406cbc0 0x7f943406f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:06.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:06.682+0000 7f9451600640 1 -- 172.21.15.152:0/2327217198 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7f944c08b9b0 con 0x7f94541635b0 2024-04-20T11:34:06.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:06.683+0000 7f9459400640 1 --2- 172.21.15.152:0/2327217198 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f943406cbc0 0x7f943406f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:06.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:06.684+0000 7f9459400640 1 --2- 172.21.15.152:0/2327217198 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f943406cbc0 0x7f943406f080 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f944800a970 tx=0x7f9448008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:06.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:06.687+0000 7f9451600640 1 -- 172.21.15.152:0/2327217198 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f9454163130 con 0x7f94541635b0 2024-04-20T11:34:06.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:06.927+0000 7f945be00640 1 -- 172.21.15.152:0/2327217198 --> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot info", "vol_name": "cephfs", "sub_name": "subvol48180", "snap_name": "subvol_snap14510", "target": ["mon-mgr", ""]}) -- 0x7f94541ed1b0 con 0x7f943406cbc0 2024-04-20T11:34:06.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:06.973+0000 7f9451600640 1 -- 172.21.15.152:0/2327217198 <== mgr.4109 v2:172.21.15.175:6836/4021250441 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+115 (secure 0 0 0) 0x7f94541ed1b0 con 0x7f943406cbc0 2024-04-20T11:34:06.973 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-04-20T11:34:06.973 INFO:teuthology.orchestra.run.smithi152.stdout: "created_at": "2024-04-20 11:33:46.699334", 2024-04-20T11:34:06.973 INFO:teuthology.orchestra.run.smithi152.stdout: "data_pool": "cephfs_data", 2024-04-20T11:34:06.973 INFO:teuthology.orchestra.run.smithi152.stdout: "has_pending_clones": "no" 2024-04-20T11:34:06.974 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-04-20T11:34:06.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:06.976+0000 7f945be00640 1 -- 172.21.15.152:0/2327217198 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f943406cbc0 msgr2=0x7f943406f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:06.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:06.976+0000 7f945be00640 1 --2- 172.21.15.152:0/2327217198 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f943406cbc0 0x7f943406f080 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f944800a970 tx=0x7f9448008040 comp rx=0 tx=0).stop 2024-04-20T11:34:06.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:06.976+0000 7f945be00640 1 -- 172.21.15.152:0/2327217198 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f94541635b0 msgr2=0x7f9454146520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:06.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:06.976+0000 7f945be00640 1 --2- 172.21.15.152:0/2327217198 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f94541635b0 0x7f9454146520 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f944c00caf0 tx=0x7f944c00a060 comp rx=0 tx=0).stop 2024-04-20T11:34:06.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:06.976+0000 7f945be00640 1 -- 172.21.15.152:0/2327217198 shutdown_connections 2024-04-20T11:34:06.977 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:06.976+0000 7f945be00640 1 --2- 172.21.15.152:0/2327217198 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f943406cbc0 0x7f943406f080 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:06.977 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:06.976+0000 7f945be00640 1 --2- 172.21.15.152:0/2327217198 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f94541635b0 0x7f9454146520 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:06.977 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:06.976+0000 7f945be00640 1 --2- 172.21.15.152:0/2327217198 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94540b3470 0x7f945415c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:06.977 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:06.976+0000 7f945be00640 1 --2- 172.21.15.152:0/2327217198 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9454071e20 0x7f9454143df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:06.977 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:06.976+0000 7f945be00640 1 -- 172.21.15.152:0/2327217198 >> 172.21.15.152:0/2327217198 conn(0x7f945406b980 msgr2=0x7f94540b38a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:06.977 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:06.977+0000 7f945be00640 1 -- 172.21.15.152:0/2327217198 shutdown_connections 2024-04-20T11:34:06.977 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:06.977+0000 7f945be00640 1 -- 172.21.15.152:0/2327217198 wait complete. 2024-04-20T11:34:06.997 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls cephfs subvol48180 2024-04-20T11:34:07.199 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.200+0000 7f5bc1c00640 1 Processor -- start 2024-04-20T11:34:07.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.200+0000 7f5bc1c00640 1 -- start start 2024-04-20T11:34:07.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.201+0000 7f5bc1c00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f5bbc0b4ac0 0x7f5bbc0b3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:07.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.201+0000 7f5bc1c00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f5bbc0d05c0 0x7f5bbc0d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:07.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.201+0000 7f5bc1c00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5bbc15da60 con 0x7f5bbc15cae0 2024-04-20T11:34:07.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.201+0000 7f5bc1c00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f5bbc0d0f60 con 0x7f5bbc0b4ac0 2024-04-20T11:34:07.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.201+0000 7f5bc1c00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f5bbc0b3740 con 0x7f5bbc0d05c0 2024-04-20T11:34:07.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.202+0000 7f5bc1200640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f5bbc15cae0 0x7f5bbc15cf70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:35000/0 (socket says 172.21.15.152:35000) 2024-04-20T11:34:07.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.202+0000 7f5bc1200640 1 -- 172.21.15.152:0/493733175 learned_addr learned my addr 172.21.15.152:0/493733175 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:34:07.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.202+0000 7f5bbbe00640 1 --2- 172.21.15.152:0/493733175 >> v2:172.21.15.152:3300/0 conn(0x7f5bbc0b4ac0 0x7f5bbc0b3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:07.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.202+0000 7f5bbb400640 1 --2- 172.21.15.152:0/493733175 >> v2:172.21.15.175:3300/0 conn(0x7f5bbc0d05c0 0x7f5bbc0d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:07.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.202+0000 7f5bbaa00640 1 -- 172.21.15.152:0/493733175 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3553117682 0 0) 0x7f5bbc15da60 con 0x7f5bbc15cae0 2024-04-20T11:34:07.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.202+0000 7f5bbaa00640 1 -- 172.21.15.152:0/493733175 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5ba0003520 con 0x7f5bbc15cae0 2024-04-20T11:34:07.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.202+0000 7f5bbbe00640 1 -- 172.21.15.152:0/493733175 >> v2:172.21.15.175:3300/0 conn(0x7f5bbc0d05c0 msgr2=0x7f5bbc0d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:07.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.202+0000 7f5bbbe00640 1 --2- 172.21.15.152:0/493733175 >> v2:172.21.15.175:3300/0 conn(0x7f5bbc0d05c0 0x7f5bbc0d0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:07.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.202+0000 7f5bbbe00640 1 -- 172.21.15.152:0/493733175 >> v1:172.21.15.152:6789/0 conn(0x7f5bbc15cae0 legacy=0x7f5bbc15cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:07.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.202+0000 7f5bbbe00640 1 -- 172.21.15.152:0/493733175 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bbc15d780 con 0x7f5bbc0b4ac0 2024-04-20T11:34:07.204 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.204+0000 7f5bbbe00640 1 --2- 172.21.15.152:0/493733175 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5bbc0b4ac0 0x7f5bbc0b3bd0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc0ce770 tx=0x7f5bac02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d70d5d3680934255 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:07.204 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.204+0000 7f5bbaa00640 1 -- 172.21.15.152:0/493733175 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bac035020 con 0x7f5bbc0b4ac0 2024-04-20T11:34:07.204 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.204+0000 7f5bbaa00640 1 -- 172.21.15.152:0/493733175 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bac031830 con 0x7f5bbc0b4ac0 2024-04-20T11:34:07.204 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.204+0000 7f5bbaa00640 1 -- 172.21.15.152:0/493733175 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bac031c00 con 0x7f5bbc0b4ac0 2024-04-20T11:34:07.205 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.204+0000 7f5bc1c00640 1 -- 172.21.15.152:0/493733175 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5bbc0b4ac0 msgr2=0x7f5bbc0b3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:07.205 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.204+0000 7f5bc1c00640 1 --2- 172.21.15.152:0/493733175 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5bbc0b4ac0 0x7f5bbc0b3bd0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc0ce770 tx=0x7f5bac02b8d0 comp rx=0 tx=0).stop 2024-04-20T11:34:07.205 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.205+0000 7f5bc1c00640 1 -- 172.21.15.152:0/493733175 shutdown_connections 2024-04-20T11:34:07.205 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.205+0000 7f5bc1c00640 1 --2- 172.21.15.152:0/493733175 >> v2:172.21.15.175:3300/0 conn(0x7f5bbc0d05c0 0x7f5bbc0d0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:07.205 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.205+0000 7f5bc1c00640 1 --2- 172.21.15.152:0/493733175 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5bbc0b4ac0 0x7f5bbc0b3bd0 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:07.205 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.205+0000 7f5bc1c00640 1 -- 172.21.15.152:0/493733175 >> 172.21.15.152:0/493733175 conn(0x7f5bbc06bda0 msgr2=0x7f5bbc06c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:07.205 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.205+0000 7f5bc1c00640 1 -- 172.21.15.152:0/493733175 shutdown_connections 2024-04-20T11:34:07.205 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.205+0000 7f5bc1c00640 1 -- 172.21.15.152:0/493733175 wait complete. 2024-04-20T11:34:07.206 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.206+0000 7f5bc1c00640 1 Processor -- start 2024-04-20T11:34:07.206 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.207+0000 7f5bc1c00640 1 -- start start 2024-04-20T11:34:07.208 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.208+0000 7f5bc1c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5bbc0b4ac0 0x7f5bbc147b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:07.208 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.208+0000 7f5bc1c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5bbc0d05c0 0x7f5bbc06f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:07.208 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.208+0000 7f5bc1c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5bbc15cae0 0x7f5bbc06f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:07.208 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.208+0000 7f5bc1c00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f5bbc0b4ea0 con 0x7f5bbc15cae0 2024-04-20T11:34:07.208 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.208+0000 7f5bc1c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5bbc0b33d0 con 0x7f5bbc0b4ac0 2024-04-20T11:34:07.208 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.208+0000 7f5bc1c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5bbc0ccb80 con 0x7f5bbc0d05c0 2024-04-20T11:34:07.208 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.208+0000 7f5bc1200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5bbc15cae0 0x7f5bbc06f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:07.208 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.208+0000 7f5bc1200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5bbc15cae0 0x7f5bbc06f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:53270/0 (socket says 172.21.15.152:53270) 2024-04-20T11:34:07.208 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.208+0000 7f5bc1200640 1 -- 172.21.15.152:0/3109485498 learned_addr learned my addr 172.21.15.152:0/3109485498 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:07.208 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.208+0000 7f5bbbe00640 1 --2- 172.21.15.152:0/3109485498 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5bbc0b4ac0 0x7f5bbc147b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:07.209 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.208+0000 7f5bbb400640 1 --2- 172.21.15.152:0/3109485498 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5bbc0d05c0 0x7f5bbc06f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:07.209 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.208+0000 7f5bc1200640 1 -- 172.21.15.152:0/3109485498 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5bbc0d05c0 msgr2=0x7f5bbc06f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:07.209 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.208+0000 7f5bc1200640 1 --2- 172.21.15.152:0/3109485498 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5bbc0d05c0 0x7f5bbc06f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:07.209 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.208+0000 7f5bc1200640 1 -- 172.21.15.152:0/3109485498 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5bbc0b4ac0 msgr2=0x7f5bbc147b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:07.209 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.208+0000 7f5bc1200640 1 --2- 172.21.15.152:0/3109485498 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5bbc0b4ac0 0x7f5bbc147b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:07.209 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.208+0000 7f5bc1200640 1 -- 172.21.15.152:0/3109485498 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bbc1480d0 con 0x7f5bbc15cae0 2024-04-20T11:34:07.209 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.209+0000 7f5bc1200640 1 --2- 172.21.15.152:0/3109485498 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5bbc15cae0 0x7f5bbc06f6d0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f5bb400caa0 tx=0x7f5bb400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:07.210 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.209+0000 7f5bb3e00640 1 -- 172.21.15.152:0/3109485498 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb4017070 con 0x7f5bbc15cae0 2024-04-20T11:34:07.210 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.209+0000 7f5bb3e00640 1 -- 172.21.15.152:0/3109485498 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bb400ac00 con 0x7f5bbc15cae0 2024-04-20T11:34:07.210 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.209+0000 7f5bb3e00640 1 -- 172.21.15.152:0/3109485498 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb4012400 con 0x7f5bbc15cae0 2024-04-20T11:34:07.210 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.209+0000 7f5bc1c00640 1 -- 172.21.15.152:0/3109485498 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bbc14b300 con 0x7f5bbc15cae0 2024-04-20T11:34:07.210 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.209+0000 7f5bc1c00640 1 -- 172.21.15.152:0/3109485498 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bbc148730 con 0x7f5bbc15cae0 2024-04-20T11:34:07.210 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.210+0000 7f5bc1c00640 1 -- 172.21.15.152:0/3109485498 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bbc0681f0 con 0x7f5bbc15cae0 2024-04-20T11:34:07.210 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.211+0000 7f5bb3e00640 1 -- 172.21.15.152:0/3109485498 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 18) ==== 91295+0+0 (secure 0 0 0) 0x7f5bb401a070 con 0x7f5bbc15cae0 2024-04-20T11:34:07.211 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.212+0000 7f5bb3e00640 1 --2- 172.21.15.152:0/3109485498 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f5b9806cbc0 0x7f5b9806f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:07.212 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.212+0000 7f5bb3e00640 1 -- 172.21.15.152:0/3109485498 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7f5bb408be40 con 0x7f5bbc15cae0 2024-04-20T11:34:07.212 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.213+0000 7f5bbbe00640 1 --2- 172.21.15.152:0/3109485498 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f5b9806cbc0 0x7f5b9806f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:07.213 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.214+0000 7f5bbbe00640 1 --2- 172.21.15.152:0/3109485498 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f5b9806cbc0 0x7f5b9806f080 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f5bac036000 tx=0x7f5bac008030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:07.216 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.217+0000 7f5bb3e00640 1 -- 172.21.15.152:0/3109485498 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f5bbc0681f0 con 0x7f5bbc15cae0 2024-04-20T11:34:07.447 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.448+0000 7f5bc1c00640 1 -- 172.21.15.152:0/3109485498 --> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "cephfs", "sub_name": "subvol48180", "target": ["mon-mgr", ""]}) -- 0x7f5bbc0689f0 con 0x7f5b9806cbc0 2024-04-20T11:34:07.487 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.488+0000 7f5bb3e00640 1 -- 172.21.15.152:0/3109485498 <== mgr.4109 v2:172.21.15.175:6836/4021250441 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+87 (secure 0 0 0) 0x7f5bbc0689f0 con 0x7f5b9806cbc0 2024-04-20T11:34:07.487 INFO:teuthology.orchestra.run.smithi152.stdout:[ 2024-04-20T11:34:07.487 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:34:07.488 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "fake" 2024-04-20T11:34:07.488 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:34:07.488 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:34:07.488 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "subvol_snap14510" 2024-04-20T11:34:07.488 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:34:07.488 INFO:teuthology.orchestra.run.smithi152.stdout:] 2024-04-20T11:34:07.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.490+0000 7f5bc1c00640 1 -- 172.21.15.152:0/3109485498 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f5b9806cbc0 msgr2=0x7f5b9806f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:07.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.491+0000 7f5bc1c00640 1 --2- 172.21.15.152:0/3109485498 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f5b9806cbc0 0x7f5b9806f080 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f5bac036000 tx=0x7f5bac008030 comp rx=0 tx=0).stop 2024-04-20T11:34:07.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.491+0000 7f5bc1c00640 1 -- 172.21.15.152:0/3109485498 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5bbc15cae0 msgr2=0x7f5bbc06f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:07.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.491+0000 7f5bc1c00640 1 --2- 172.21.15.152:0/3109485498 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5bbc15cae0 0x7f5bbc06f6d0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f5bb400caa0 tx=0x7f5bb400cf60 comp rx=0 tx=0).stop 2024-04-20T11:34:07.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.491+0000 7f5bc1c00640 1 -- 172.21.15.152:0/3109485498 shutdown_connections 2024-04-20T11:34:07.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.491+0000 7f5bc1c00640 1 --2- 172.21.15.152:0/3109485498 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f5b9806cbc0 0x7f5b9806f080 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:07.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.491+0000 7f5bc1c00640 1 --2- 172.21.15.152:0/3109485498 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5bbc15cae0 0x7f5bbc06f6d0 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:07.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.491+0000 7f5bc1c00640 1 --2- 172.21.15.152:0/3109485498 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5bbc0d05c0 0x7f5bbc06f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:07.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.491+0000 7f5bc1c00640 1 --2- 172.21.15.152:0/3109485498 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5bbc0b4ac0 0x7f5bbc147b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:07.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.491+0000 7f5bc1c00640 1 -- 172.21.15.152:0/3109485498 >> 172.21.15.152:0/3109485498 conn(0x7f5bbc06bda0 msgr2=0x7f5bbc0cf190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:07.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.491+0000 7f5bc1c00640 1 -- 172.21.15.152:0/3109485498 shutdown_connections 2024-04-20T11:34:07.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.491+0000 7f5bc1c00640 1 -- 172.21.15.152:0/3109485498 wait complete. 2024-04-20T11:34:07.511 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm cephfs subvol48180 subvol_snap14510 2024-04-20T11:34:07.713 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.713+0000 7f04dfe00640 1 Processor -- start 2024-04-20T11:34:07.713 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.714+0000 7f04dfe00640 1 -- start start 2024-04-20T11:34:07.714 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.715+0000 7f04dfe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f04e015e870 0x7f04e00d3370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:07.714 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.715+0000 7f04dfe00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f04e00d0fa0 0x7f04e00d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:07.715 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.715+0000 7f04dfe00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04e015ec50 con 0x7f04e00cff90 2024-04-20T11:34:07.715 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.715+0000 7f04dfe00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f04e00d1960 con 0x7f04e00d0fa0 2024-04-20T11:34:07.715 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.715+0000 7f04dfe00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f04e00d2ee0 con 0x7f04e015e870 2024-04-20T11:34:07.716 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.716+0000 7f04dea00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f04e015e870 0x7f04e00d3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:07.716 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.716+0000 7f04d6000640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f04e00d0fa0 0x7f04e00d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:07.716 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.716+0000 7f04dea00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f04e015e870 0x7f04e00d3370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.152:42072/0 (socket says 172.21.15.152:42072) 2024-04-20T11:34:07.716 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.716+0000 7f04dea00640 1 -- 172.21.15.152:0/2104708296 learned_addr learned my addr 172.21.15.152:0/2104708296 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:07.716 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.716+0000 7f04dea00640 1 -- 172.21.15.152:0/2104708296 >> v2:172.21.15.175:3300/0 conn(0x7f04e00d0fa0 msgr2=0x7f04e00d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:07.716 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.716+0000 7f04dea00640 1 --2- 172.21.15.152:0/2104708296 >> v2:172.21.15.175:3300/0 conn(0x7f04e00d0fa0 0x7f04e00d1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:07.716 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.716+0000 7f04dea00640 1 -- 172.21.15.152:0/2104708296 >> v1:172.21.15.175:6789/0 conn(0x7f04e00cff90 legacy=0x7f04e00d0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:07.716 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.716+0000 7f04dea00640 1 -- 172.21.15.152:0/2104708296 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04e00d38b0 con 0x7f04e015e870 2024-04-20T11:34:07.717 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.718+0000 7f04dea00640 1 --2- 172.21.15.152:0/2104708296 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f04e015e870 0x7f04e00d3370 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f04d0009870 tx=0x7f04d002b900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34e1a739c1cdbe4f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:07.718 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.718+0000 7f04de000640 1 -- 172.21.15.152:0/2104708296 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d0035020 con 0x7f04e015e870 2024-04-20T11:34:07.718 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.718+0000 7f04de000640 1 -- 172.21.15.152:0/2104708296 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04d002d040 con 0x7f04e015e870 2024-04-20T11:34:07.718 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.718+0000 7f04de000640 1 -- 172.21.15.152:0/2104708296 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d0031a80 con 0x7f04e015e870 2024-04-20T11:34:07.719 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.719+0000 7f04dfe00640 1 -- 172.21.15.152:0/2104708296 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f04e015e870 msgr2=0x7f04e00d3370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:07.719 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.719+0000 7f04dfe00640 1 --2- 172.21.15.152:0/2104708296 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f04e015e870 0x7f04e00d3370 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f04d0009870 tx=0x7f04d002b900 comp rx=0 tx=0).stop 2024-04-20T11:34:07.719 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.719+0000 7f04dfe00640 1 -- 172.21.15.152:0/2104708296 shutdown_connections 2024-04-20T11:34:07.719 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.719+0000 7f04dfe00640 1 --2- 172.21.15.152:0/2104708296 >> v2:172.21.15.175:3300/0 conn(0x7f04e00d0fa0 0x7f04e00d1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:07.719 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.719+0000 7f04dfe00640 1 --2- 172.21.15.152:0/2104708296 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f04e015e870 0x7f04e00d3370 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:07.719 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.719+0000 7f04dfe00640 1 -- 172.21.15.152:0/2104708296 >> 172.21.15.152:0/2104708296 conn(0x7f04e008bfd0 msgr2=0x7f04e00d45d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:07.719 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.719+0000 7f04dfe00640 1 -- 172.21.15.152:0/2104708296 shutdown_connections 2024-04-20T11:34:07.720 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.719+0000 7f04dfe00640 1 -- 172.21.15.152:0/2104708296 wait complete. 2024-04-20T11:34:07.720 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.721+0000 7f04dfe00640 1 Processor -- start 2024-04-20T11:34:07.720 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.721+0000 7f04dfe00640 1 -- start start 2024-04-20T11:34:07.723 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.722+0000 7f04dfe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f04e00cff90 0x7f04e00b4510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:07.723 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.722+0000 7f04dfe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f04e00d0fa0 0x7f04e00ca540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:07.723 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.722+0000 7f04dfe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f04e015e870 0x7f04e015d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:07.723 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.722+0000 7f04dfe00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f04e00d2890 con 0x7f04e00d0fa0 2024-04-20T11:34:07.723 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.722+0000 7f04dfe00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f04e00d25b0 con 0x7f04e00cff90 2024-04-20T11:34:07.724 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.722+0000 7f04dfe00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f04e00d2b70 con 0x7f04e015e870 2024-04-20T11:34:07.724 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.722+0000 7f04d6000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f04e00d0fa0 0x7f04e00ca540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:07.724 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.722+0000 7f04d6000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f04e00d0fa0 0x7f04e00ca540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:53274/0 (socket says 172.21.15.152:53274) 2024-04-20T11:34:07.724 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.722+0000 7f04d6000640 1 -- 172.21.15.152:0/1699861161 learned_addr learned my addr 172.21.15.152:0/1699861161 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:07.724 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.722+0000 7f04dea00640 1 --2- 172.21.15.152:0/1699861161 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f04e00cff90 0x7f04e00b4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:07.724 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.723+0000 7f04df400640 1 --2- 172.21.15.152:0/1699861161 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f04e015e870 0x7f04e015d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:07.724 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.723+0000 7f04d6000640 1 -- 172.21.15.152:0/1699861161 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f04e015e870 msgr2=0x7f04e015d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:07.724 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.723+0000 7f04d6000640 1 --2- 172.21.15.152:0/1699861161 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f04e015e870 0x7f04e015d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:07.725 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.723+0000 7f04d6000640 1 -- 172.21.15.152:0/1699861161 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f04e00cff90 msgr2=0x7f04e00b4510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:07.725 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.723+0000 7f04d6000640 1 --2- 172.21.15.152:0/1699861161 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f04e00cff90 0x7f04e00b4510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:07.725 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.723+0000 7f04d6000640 1 -- 172.21.15.152:0/1699861161 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04e00b4df0 con 0x7f04e00d0fa0 2024-04-20T11:34:07.725 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.723+0000 7f04d6000640 1 --2- 172.21.15.152:0/1699861161 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f04e00d0fa0 0x7f04e00ca540 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f04c800e8a0 tx=0x7f04c800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:07.725 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.723+0000 7f04d7400640 1 -- 172.21.15.152:0/1699861161 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c8017070 con 0x7f04e00d0fa0 2024-04-20T11:34:07.725 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.723+0000 7f04d7400640 1 -- 172.21.15.152:0/1699861161 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04c800c940 con 0x7f04e00d0fa0 2024-04-20T11:34:07.725 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.723+0000 7f04d7400640 1 -- 172.21.15.152:0/1699861161 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c800ccf0 con 0x7f04e00d0fa0 2024-04-20T11:34:07.725 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.723+0000 7f04dfe00640 1 -- 172.21.15.152:0/1699861161 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04e00b50d0 con 0x7f04e00d0fa0 2024-04-20T11:34:07.725 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.724+0000 7f04dfe00640 1 -- 172.21.15.152:0/1699861161 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04e00b5690 con 0x7f04e00d0fa0 2024-04-20T11:34:07.726 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.725+0000 7f04dfe00640 1 -- 172.21.15.152:0/1699861161 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04e00693b0 con 0x7f04e00d0fa0 2024-04-20T11:34:07.726 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.726+0000 7f04d7400640 1 -- 172.21.15.152:0/1699861161 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 18) ==== 91295+0+0 (secure 0 0 0) 0x7f04c801a070 con 0x7f04e00d0fa0 2024-04-20T11:34:07.727 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.727+0000 7f04d7400640 1 --2- 172.21.15.152:0/1699861161 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f04bc06cbc0 0x7f04bc06f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:07.727 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.727+0000 7f04d7400640 1 -- 172.21.15.152:0/1699861161 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7f04c808c400 con 0x7f04e00d0fa0 2024-04-20T11:34:07.727 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.728+0000 7f04dea00640 1 --2- 172.21.15.152:0/1699861161 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f04bc06cbc0 0x7f04bc06f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:07.728 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.729+0000 7f04dea00640 1 --2- 172.21.15.152:0/1699861161 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f04bc06cbc0 0x7f04bc06f080 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f04d0036000 tx=0x7f04d003c040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:07.732 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.732+0000 7f04d7400640 1 -- 172.21.15.152:0/1699861161 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f04e00693b0 con 0x7f04e00d0fa0 2024-04-20T11:34:07.979 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:07.979+0000 7f04dfe00640 1 -- 172.21.15.152:0/1699861161 --> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "vol_name": "cephfs", "sub_name": "subvol48180", "snap_name": "subvol_snap14510", "target": ["mon-mgr", ""]}) -- 0x7f04e0075e60 con 0x7f04bc06cbc0 2024-04-20T11:34:08.054 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:08.055+0000 7f04d7400640 1 -- 172.21.15.152:0/1699861161 <== mgr.4109 v2:172.21.15.175:6836/4021250441 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f04e0075e60 con 0x7f04bc06cbc0 2024-04-20T11:34:08.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:08.058+0000 7f04dfe00640 1 -- 172.21.15.152:0/1699861161 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f04bc06cbc0 msgr2=0x7f04bc06f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:08.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:08.058+0000 7f04dfe00640 1 --2- 172.21.15.152:0/1699861161 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f04bc06cbc0 0x7f04bc06f080 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f04d0036000 tx=0x7f04d003c040 comp rx=0 tx=0).stop 2024-04-20T11:34:08.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:08.058+0000 7f04dfe00640 1 -- 172.21.15.152:0/1699861161 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f04e00d0fa0 msgr2=0x7f04e00ca540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:08.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:08.058+0000 7f04dfe00640 1 --2- 172.21.15.152:0/1699861161 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f04e00d0fa0 0x7f04e00ca540 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f04c800e8a0 tx=0x7f04c800ed60 comp rx=0 tx=0).stop 2024-04-20T11:34:08.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:08.058+0000 7f04dfe00640 1 -- 172.21.15.152:0/1699861161 shutdown_connections 2024-04-20T11:34:08.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:08.058+0000 7f04dfe00640 1 --2- 172.21.15.152:0/1699861161 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f04bc06cbc0 0x7f04bc06f080 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:08.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:08.058+0000 7f04dfe00640 1 --2- 172.21.15.152:0/1699861161 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f04e015e870 0x7f04e015d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:08.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:08.058+0000 7f04dfe00640 1 --2- 172.21.15.152:0/1699861161 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f04e00d0fa0 0x7f04e00ca540 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:08.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:08.058+0000 7f04dfe00640 1 --2- 172.21.15.152:0/1699861161 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f04e00cff90 0x7f04e00b4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:08.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:08.058+0000 7f04dfe00640 1 -- 172.21.15.152:0/1699861161 >> 172.21.15.152:0/1699861161 conn(0x7f04e008bfd0 msgr2=0x7f04e00b2790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:08.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:08.059+0000 7f04dfe00640 1 -- 172.21.15.152:0/1699861161 shutdown_connections 2024-04-20T11:34:08.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:08.059+0000 7f04dfe00640 1 -- 172.21.15.152:0/1699861161 wait complete. 2024-04-20T11:34:08.080 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm cephfs subvol48180 fake 2024-04-20T11:34:08.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:08.283+0000 7f1dbcc00640 1 Processor -- start 2024-04-20T11:34:08.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:08.283+0000 7f1dbcc00640 1 -- start start 2024-04-20T11:34:08.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:08.285+0000 7f1dbcc00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1db8144fa0 0x7f1db80b3470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:08.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:08.285+0000 7f1dbcc00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f1db8072530 0x7f1db806e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:08.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:08.285+0000 7f1dbcc00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1db80b3a50 con 0x7f1db8145620 2024-04-20T11:34:08.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:08.285+0000 7f1dbcc00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f1db806ee20 con 0x7f1db8072530 2024-04-20T11:34:08.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:08.285+0000 7f1dbcc00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1db8145a80 con 0x7f1db8144fa0 2024-04-20T11:34:08.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:08.285+0000 7f1db5400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f1db8145620 0x7f1db8071e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:35008/0 (socket says 172.21.15.152:35008) 2024-04-20T11:34:08.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:08.285+0000 7f1db5400640 1 -- 172.21.15.152:0/1090282437 learned_addr learned my addr 172.21.15.152:0/1090282437 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:34:08.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:08.286+0000 7f1db5e00640 1 --2- 172.21.15.152:0/1090282437 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1db8144fa0 0x7f1db80b3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:08.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:08.286+0000 7f1db6800640 1 --2- 172.21.15.152:0/1090282437 >> v2:172.21.15.175:3300/0 conn(0x7f1db8072530 0x7f1db806e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:08.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:08.286+0000 7f1db4a00640 1 -- 172.21.15.152:0/1090282437 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1522417941 0 0) 0x7f1db80b3a50 con 0x7f1db8145620 2024-04-20T11:34:08.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:08.286+0000 7f1db4a00640 1 -- 172.21.15.152:0/1090282437 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1d9c003520 con 0x7f1db8145620 2024-04-20T11:34:08.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:08.286+0000 7f1db4a00640 1 -- 172.21.15.152:0/1090282437 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 191844668 0 0) 0x7f1d9c003520 con 0x7f1db8145620 2024-04-20T11:34:08.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:08.286+0000 7f1db4a00640 1 -- 172.21.15.152:0/1090282437 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1db8144fa0 msgr2=0x7f1db80b3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:08.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:08.286+0000 7f1db4a00640 1 --2- 172.21.15.152:0/1090282437 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1db8144fa0 0x7f1db80b3470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:08.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:08.286+0000 7f1db4a00640 1 -- 172.21.15.152:0/1090282437 >> v2:172.21.15.175:3300/0 conn(0x7f1db8072530 msgr2=0x7f1db806e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:08.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:08.286+0000 7f1db4a00640 1 --2- 172.21.15.152:0/1090282437 >> v2:172.21.15.175:3300/0 conn(0x7f1db8072530 0x7f1db806e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:08.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:08.286+0000 7f1db4a00640 1 -- 172.21.15.152:0/1090282437 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1db80b3d30 con 0x7f1db8145620 2024-04-20T11:34:08.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:08.286+0000 7f1db4a00640 1 -- 172.21.15.152:0/1090282437 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f1da4001670 con 0x7f1db8145620 2024-04-20T11:34:08.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:08.286+0000 7f1db4a00640 1 -- 172.21.15.152:0/1090282437 >> v1:172.21.15.152:6789/0 conn(0x7f1db8145620 legacy=0x7f1db8071e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:08.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:08.286+0000 7f1db4a00640 1 --2- 172.21.15.152:0/1090282437 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1db8144fa0 0x7f1d9c003a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:08.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:08.286+0000 7f1db4a00640 1 --2- 172.21.15.152:0/1090282437 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d9c003fa0 0x7f1d9c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:08.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:08.286+0000 7f1db4a00640 1 --2- 172.21.15.152:0/1090282437 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d9c004d40 0x7f1d9c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:08.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:08.286+0000 7f1db4a00640 1 -- 172.21.15.152:0/1090282437 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f1db806ee20 con 0x7f1d9c003fa0 2024-04-20T11:34:08.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:08.286+0000 7f1db4a00640 1 -- 172.21.15.152:0/1090282437 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1db8145a80 con 0x7f1d9c004d40 2024-04-20T11:34:08.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:08.286+0000 7f1db4a00640 1 -- 172.21.15.152:0/1090282437 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1d9c00bb20 con 0x7f1db8144fa0 2024-04-20T11:34:08.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:08.287+0000 7f1db5400640 1 --2- 172.21.15.152:0/1090282437 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d9c003fa0 0x7f1d9c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:08.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:08.287+0000 7f1db5e00640 1 --2- 172.21.15.152:0/1090282437 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1db8144fa0 0x7f1d9c003a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:08.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:08.287+0000 7f1db5400640 1 -- 172.21.15.152:0/1090282437 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1db8144fa0 msgr2=0x7f1d9c003a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:08.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:08.287+0000 7f1db5400640 1 --2- 172.21.15.152:0/1090282437 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1db8144fa0 0x7f1d9c003a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:08.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:08.287+0000 7f1db6800640 1 --2- 172.21.15.152:0/1090282437 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d9c004d40 0x7f1d9c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:08.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:08.287+0000 7f1db5400640 1 -- 172.21.15.152:0/1090282437 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d9c004d40 msgr2=0x7f1d9c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:08.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:08.287+0000 7f1db5400640 1 --2- 172.21.15.152:0/1090282437 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d9c004d40 0x7f1d9c0051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:08.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:08.287+0000 7f1db5400640 1 -- 172.21.15.152:0/1090282437 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d9c003520 con 0x7f1d9c003fa0 2024-04-20T11:34:08.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:08.288+0000 7f1db5400640 1 --2- 172.21.15.152:0/1090282437 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d9c003fa0 0x7f1d9c004800 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f1da4007fd0 tx=0x7f1d9c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e2872b9ea69cc3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:08.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:08.288+0000 7f1db4a00640 1 -- 172.21.15.152:0/1090282437 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da4007d50 con 0x7f1d9c003fa0 2024-04-20T11:34:08.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:08.289+0000 7f1db4a00640 1 -- 172.21.15.152:0/1090282437 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1da4030070 con 0x7f1d9c003fa0 2024-04-20T11:34:08.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:08.289+0000 7f1db4a00640 1 -- 172.21.15.152:0/1090282437 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da402cd40 con 0x7f1d9c003fa0 2024-04-20T11:34:08.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:08.289+0000 7f1dbcc00640 1 -- 172.21.15.152:0/1090282437 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d9c003fa0 msgr2=0x7f1d9c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:08.291 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:08.289+0000 7f1dbcc00640 1 --2- 172.21.15.152:0/1090282437 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d9c003fa0 0x7f1d9c004800 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f1da4007fd0 tx=0x7f1d9c005830 comp rx=0 tx=0).stop 2024-04-20T11:34:08.291 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:08.289+0000 7f1db6800640 1 -- 172.21.15.152:0/1090282437 reap_dead start 2024-04-20T11:34:08.291 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:08.289+0000 7f1dbcc00640 1 -- 172.21.15.152:0/1090282437 shutdown_connections 2024-04-20T11:34:08.291 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:08.289+0000 7f1dbcc00640 1 -- 172.21.15.152:0/1090282437 >> 172.21.15.152:0/1090282437 conn(0x7f1db806b980 msgr2=0x7f1db806bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:08.291 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:08.289+0000 7f1dbcc00640 1 -- 172.21.15.152:0/1090282437 shutdown_connections 2024-04-20T11:34:08.291 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:08.289+0000 7f1dbcc00640 1 -- 172.21.15.152:0/1090282437 wait complete. 2024-04-20T11:34:08.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:08.291+0000 7f1dbcc00640 1 Processor -- start 2024-04-20T11:34:08.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:08.291+0000 7f1dbcc00640 1 -- start start 2024-04-20T11:34:08.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:08.292+0000 7f1dbcc00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d9c003fa0 0x7f1db81f7b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:08.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:08.292+0000 7f1dbcc00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1db813db60 0x7f1db80b0d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:08.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:08.292+0000 7f1dbcc00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1db813bb60 0x7f1db813bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:08.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:08.292+0000 7f1dbcc00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f1db80aeb90 con 0x7f1db813bb60 2024-04-20T11:34:08.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:08.292+0000 7f1dbcc00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1db80b42f0 con 0x7f1d9c003fa0 2024-04-20T11:34:08.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:08.292+0000 7f1dbcc00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1db80b3470 con 0x7f1db813db60 2024-04-20T11:34:08.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:08.293+0000 7f1db6800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1db813bb60 0x7f1db813bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:08.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:08.293+0000 7f1db6800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1db813bb60 0x7f1db813bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:53286/0 (socket says 172.21.15.152:53286) 2024-04-20T11:34:08.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:08.293+0000 7f1db6800640 1 -- 172.21.15.152:0/724487320 learned_addr learned my addr 172.21.15.152:0/724487320 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:08.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:08.293+0000 7f1db5400640 1 --2- 172.21.15.152:0/724487320 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1db813db60 0x7f1db80b0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:08.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:08.293+0000 7f1db5e00640 1 --2- 172.21.15.152:0/724487320 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d9c003fa0 0x7f1db81f7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:08.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:08.293+0000 7f1db6800640 1 -- 172.21.15.152:0/724487320 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1db813db60 msgr2=0x7f1db80b0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:08.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:08.293+0000 7f1db6800640 1 --2- 172.21.15.152:0/724487320 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1db813db60 0x7f1db80b0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:08.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:08.293+0000 7f1db6800640 1 -- 172.21.15.152:0/724487320 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d9c003fa0 msgr2=0x7f1db81f7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:08.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:08.293+0000 7f1db6800640 1 --2- 172.21.15.152:0/724487320 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d9c003fa0 0x7f1db81f7b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:08.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:08.293+0000 7f1db6800640 1 -- 172.21.15.152:0/724487320 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1db81f8a50 con 0x7f1db813bb60 2024-04-20T11:34:08.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:08.294+0000 7f1db6800640 1 --2- 172.21.15.152:0/724487320 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1db813bb60 0x7f1db813bfe0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f1db80b0090 tx=0x7f1db000f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:08.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:08.294+0000 7f1dae000640 1 -- 172.21.15.152:0/724487320 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db001c070 con 0x7f1db813bb60 2024-04-20T11:34:08.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:08.294+0000 7f1dae000640 1 -- 172.21.15.152:0/724487320 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1db0017400 con 0x7f1db813bb60 2024-04-20T11:34:08.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:08.294+0000 7f1dae000640 1 -- 172.21.15.152:0/724487320 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db00177f0 con 0x7f1db813bb60 2024-04-20T11:34:08.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:08.294+0000 7f1dbcc00640 1 -- 172.21.15.152:0/724487320 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1db81f8490 con 0x7f1db813bb60 2024-04-20T11:34:08.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:08.294+0000 7f1dbcc00640 1 -- 172.21.15.152:0/724487320 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1db81601e0 con 0x7f1db813bb60 2024-04-20T11:34:08.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:08.295+0000 7f1dbcc00640 1 -- 172.21.15.152:0/724487320 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1db8068a60 con 0x7f1db813bb60 2024-04-20T11:34:08.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:08.296+0000 7f1dae000640 1 -- 172.21.15.152:0/724487320 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 18) ==== 91295+0+0 (secure 0 0 0) 0x7f1db0005d10 con 0x7f1db813bb60 2024-04-20T11:34:08.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:08.297+0000 7f1dae000640 1 --2- 172.21.15.152:0/724487320 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f1d9806cbc0 0x7f1d9806f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:08.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:08.297+0000 7f1dae000640 1 -- 172.21.15.152:0/724487320 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7f1db008c290 con 0x7f1db813bb60 2024-04-20T11:34:08.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:08.298+0000 7f1db5e00640 1 --2- 172.21.15.152:0/724487320 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f1d9806cbc0 0x7f1d9806f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:08.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:08.299+0000 7f1db5e00640 1 --2- 172.21.15.152:0/724487320 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f1d9806cbc0 0x7f1d9806f080 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1da8009930 tx=0x7f1da8008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:08.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:08.301+0000 7f1dae000640 1 -- 172.21.15.152:0/724487320 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f1db0012070 con 0x7f1db813bb60 2024-04-20T11:34:08.539 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:08.539+0000 7f1dbcc00640 1 -- 172.21.15.152:0/724487320 --> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "vol_name": "cephfs", "sub_name": "subvol48180", "snap_name": "fake", "target": ["mon-mgr", ""]}) -- 0x7f1db80b08b0 con 0x7f1d9806cbc0 2024-04-20T11:34:08.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:08.618+0000 7f1dae000640 1 -- 172.21.15.152:0/724487320 <== mgr.4109 v2:172.21.15.175:6836/4021250441 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f1db80b08b0 con 0x7f1d9806cbc0 2024-04-20T11:34:08.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:08.621+0000 7f1dbcc00640 1 -- 172.21.15.152:0/724487320 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f1d9806cbc0 msgr2=0x7f1d9806f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:08.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:08.621+0000 7f1dbcc00640 1 --2- 172.21.15.152:0/724487320 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f1d9806cbc0 0x7f1d9806f080 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1da8009930 tx=0x7f1da8008040 comp rx=0 tx=0).stop 2024-04-20T11:34:08.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:08.621+0000 7f1dbcc00640 1 -- 172.21.15.152:0/724487320 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1db813bb60 msgr2=0x7f1db813bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:08.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:08.621+0000 7f1dbcc00640 1 --2- 172.21.15.152:0/724487320 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1db813bb60 0x7f1db813bfe0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f1db80b0090 tx=0x7f1db000f4a0 comp rx=0 tx=0).stop 2024-04-20T11:34:08.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:08.622+0000 7f1dbcc00640 1 -- 172.21.15.152:0/724487320 shutdown_connections 2024-04-20T11:34:08.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:08.622+0000 7f1dbcc00640 1 --2- 172.21.15.152:0/724487320 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f1d9806cbc0 0x7f1d9806f080 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:08.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:08.622+0000 7f1dbcc00640 1 --2- 172.21.15.152:0/724487320 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1db813bb60 0x7f1db813bfe0 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:08.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:08.622+0000 7f1dbcc00640 1 --2- 172.21.15.152:0/724487320 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1db813db60 0x7f1db80b0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:08.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:08.622+0000 7f1dbcc00640 1 --2- 172.21.15.152:0/724487320 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d9c003fa0 0x7f1db81f7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:08.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:08.622+0000 7f1dbcc00640 1 -- 172.21.15.152:0/724487320 >> 172.21.15.152:0/724487320 conn(0x7f1db806b980 msgr2=0x7f1db8147c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:08.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:08.622+0000 7f1dbcc00640 1 -- 172.21.15.152:0/724487320 shutdown_connections 2024-04-20T11:34:08.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:08.622+0000 7f1dbcc00640 1 -- 172.21.15.152:0/724487320 wait complete. 2024-04-20T11:34:08.644 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:34:08.644 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_nogroup/subvol48180/.meta) 2024-04-20T11:34:11.343 INFO:teuthology.orchestra.run.smithi152.stdout:[GLOBAL] 2024-04-20T11:34:11.344 INFO:teuthology.orchestra.run.smithi152.stdout:version = 1 2024-04-20T11:34:11.344 INFO:teuthology.orchestra.run.smithi152.stdout:type = subvolume 2024-04-20T11:34:11.344 INFO:teuthology.orchestra.run.smithi152.stdout:path = /volumes/_nogroup/subvol48180/bf9c0317-2e4a-4eaf-b73e-875a6eb33282 2024-04-20T11:34:11.344 INFO:teuthology.orchestra.run.smithi152.stdout:state = complete 2024-04-20T11:34:11.344 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:34:11.350 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol48180 2024-04-20T11:34:11.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:11.561+0000 7f8419600640 1 Processor -- start 2024-04-20T11:34:11.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:11.561+0000 7f8419600640 1 -- start start 2024-04-20T11:34:11.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:11.563+0000 7f8419600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f84140b2880 0x7f841415fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:11.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:11.563+0000 7f8419600640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f841415f1c0 con 0x7f84140cfc40 2024-04-20T11:34:11.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:11.563+0000 7f8419600640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f841406f670 con 0x7f841415b510 2024-04-20T11:34:11.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:11.563+0000 7f8419600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f841406f420 con 0x7f84140b2880 2024-04-20T11:34:11.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:11.563+0000 7f8412000640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f84140cfc40 0x7f84140b4800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:35020/0 (socket says 172.21.15.152:35020) 2024-04-20T11:34:11.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:11.563+0000 7f8412000640 1 -- 172.21.15.152:0/350821779 learned_addr learned my addr 172.21.15.152:0/350821779 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:34:11.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:11.564+0000 7f8413400640 1 --2- 172.21.15.152:0/350821779 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f84140b2880 0x7f841415fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:11.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:11.564+0000 7f8411600640 1 -- 172.21.15.152:0/350821779 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2834666379 0 0) 0x7f841415f1c0 con 0x7f84140cfc40 2024-04-20T11:34:11.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:11.564+0000 7f8411600640 1 -- 172.21.15.152:0/350821779 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f83f8005730 con 0x7f84140cfc40 2024-04-20T11:34:11.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:11.564+0000 7f8411600640 1 -- 172.21.15.152:0/350821779 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2549482657 0 0) 0x7f83f8005730 con 0x7f84140cfc40 2024-04-20T11:34:11.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:11.564+0000 7f8411600640 1 -- 172.21.15.152:0/350821779 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f84140b2880 msgr2=0x7f841415fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:11.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:11.564+0000 7f8411600640 1 --2- 172.21.15.152:0/350821779 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f84140b2880 0x7f841415fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:11.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:11.564+0000 7f8411600640 1 -- 172.21.15.152:0/350821779 >> v1:172.21.15.175:6789/0 conn(0x7f841415b510 legacy=0x7f84140b2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:11.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:11.564+0000 7f8411600640 1 -- 172.21.15.152:0/350821779 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f841415f430 con 0x7f84140cfc40 2024-04-20T11:34:11.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:11.564+0000 7f8411600640 1 -- 172.21.15.152:0/350821779 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f8400001670 con 0x7f84140cfc40 2024-04-20T11:34:11.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:11.564+0000 7f8413400640 1 --2- 172.21.15.152:0/350821779 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f84140b2880 0x7f841415fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:34:11.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:11.564+0000 7f8411600640 1 -- 172.21.15.152:0/350821779 >> v1:172.21.15.152:6789/0 conn(0x7f84140cfc40 legacy=0x7f84140b4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:11.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:11.564+0000 7f8411600640 1 --2- 172.21.15.152:0/350821779 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f83f8005bf0 0x7f83f8006010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:11.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:11.564+0000 7f8411600640 1 --2- 172.21.15.152:0/350821779 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f83f8006a10 0x7f83f8006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:11.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:11.564+0000 7f8411600640 1 --2- 172.21.15.152:0/350821779 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f84140b2880 0x7f83f80073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:11.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:11.564+0000 7f8411600640 1 -- 172.21.15.152:0/350821779 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f841406f420 con 0x7f83f8005bf0 2024-04-20T11:34:11.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:11.564+0000 7f8411600640 1 -- 172.21.15.152:0/350821779 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f83f800dd00 con 0x7f83f8006a10 2024-04-20T11:34:11.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:11.564+0000 7f8411600640 1 -- 172.21.15.152:0/350821779 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f83f800df00 con 0x7f84140b2880 2024-04-20T11:34:11.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:11.565+0000 7f8412a00640 1 --2- 172.21.15.152:0/350821779 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f83f8005bf0 0x7f83f8006010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:11.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:11.565+0000 7f8412000640 1 --2- 172.21.15.152:0/350821779 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f83f8006a10 0x7f83f8006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:11.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:11.565+0000 7f8413400640 1 --2- 172.21.15.152:0/350821779 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f84140b2880 0x7f83f80073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:11.567 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:11.565+0000 7f8412a00640 1 -- 172.21.15.152:0/350821779 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f84140b2880 msgr2=0x7f83f80073d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:11.567 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:11.565+0000 7f8412a00640 1 --2- 172.21.15.152:0/350821779 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f84140b2880 0x7f83f80073d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:11.567 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:11.565+0000 7f8412a00640 1 -- 172.21.15.152:0/350821779 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f83f8006a10 msgr2=0x7f83f8006e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:11.567 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:11.565+0000 7f8412a00640 1 --2- 172.21.15.152:0/350821779 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f83f8006a10 0x7f83f8006e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:11.567 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:11.565+0000 7f8412a00640 1 -- 172.21.15.152:0/350821779 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83f8005730 con 0x7f83f8005bf0 2024-04-20T11:34:11.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:11.566+0000 7f8412a00640 1 --2- 172.21.15.152:0/350821779 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f83f8005bf0 0x7f83f8006010 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f8404001750 tx=0x7f83f8007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6abd33a5ac5190a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:11.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:11.566+0000 7f8411600640 1 -- 172.21.15.152:0/350821779 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8404001370 con 0x7f83f8005bf0 2024-04-20T11:34:11.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:11.566+0000 7f8411600640 1 -- 172.21.15.152:0/350821779 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84040052d0 con 0x7f83f8005bf0 2024-04-20T11:34:11.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:11.566+0000 7f8411600640 1 -- 172.21.15.152:0/350821779 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8404003830 con 0x7f83f8005bf0 2024-04-20T11:34:11.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:11.567+0000 7f8419600640 1 -- 172.21.15.152:0/350821779 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f83f8005bf0 msgr2=0x7f83f8006010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:11.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:11.567+0000 7f8419600640 1 --2- 172.21.15.152:0/350821779 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f83f8005bf0 0x7f83f8006010 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f8404001750 tx=0x7f83f8007a40 comp rx=0 tx=0).stop 2024-04-20T11:34:11.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:11.567+0000 7f8413400640 1 -- 172.21.15.152:0/350821779 reap_dead start 2024-04-20T11:34:11.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:11.567+0000 7f8419600640 1 -- 172.21.15.152:0/350821779 shutdown_connections 2024-04-20T11:34:11.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:11.567+0000 7f8419600640 1 -- 172.21.15.152:0/350821779 >> 172.21.15.152:0/350821779 conn(0x7f841406b980 msgr2=0x7f841406bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:11.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:11.567+0000 7f8419600640 1 -- 172.21.15.152:0/350821779 shutdown_connections 2024-04-20T11:34:11.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:11.567+0000 7f8419600640 1 -- 172.21.15.152:0/350821779 wait complete. 2024-04-20T11:34:11.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:11.569+0000 7f8419600640 1 Processor -- start 2024-04-20T11:34:11.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:11.570+0000 7f8419600640 1 -- start start 2024-04-20T11:34:11.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:11.570+0000 7f8419600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f83f8005bf0 0x7f8414141da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:11.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:11.570+0000 7f8419600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f84140b0f70 0x7f84140b13f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:11.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:11.570+0000 7f8419600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f841415ba70 0x7f841415bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:11.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:11.571+0000 7f8419600640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f841406f950 con 0x7f841415ba70 2024-04-20T11:34:11.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:11.571+0000 7f8419600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f841415b8f0 con 0x7f84140b0f70 2024-04-20T11:34:11.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:11.571+0000 7f8419600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f841415fc30 con 0x7f83f8005bf0 2024-04-20T11:34:11.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:11.571+0000 7f8413400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f841415ba70 0x7f841415bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:11.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:11.571+0000 7f8413400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f841415ba70 0x7f841415bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:53306/0 (socket says 172.21.15.152:53306) 2024-04-20T11:34:11.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:11.571+0000 7f8413400640 1 -- 172.21.15.152:0/3958068133 learned_addr learned my addr 172.21.15.152:0/3958068133 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:11.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:11.571+0000 7f8412000640 1 --2- 172.21.15.152:0/3958068133 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f84140b0f70 0x7f84140b13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:11.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:11.571+0000 7f8413400640 1 -- 172.21.15.152:0/3958068133 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f83f8005bf0 msgr2=0x7f8414141da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:11.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:11.571+0000 7f8412a00640 1 --2- 172.21.15.152:0/3958068133 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f83f8005bf0 0x7f8414141da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:11.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:11.571+0000 7f8413400640 1 --2- 172.21.15.152:0/3958068133 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f83f8005bf0 0x7f8414141da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:11.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:11.571+0000 7f8413400640 1 -- 172.21.15.152:0/3958068133 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f84140b0f70 msgr2=0x7f84140b13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:11.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:11.571+0000 7f8413400640 1 --2- 172.21.15.152:0/3958068133 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f84140b0f70 0x7f84140b13f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:11.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:11.571+0000 7f8413400640 1 -- 172.21.15.152:0/3958068133 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f841415c430 con 0x7f841415ba70 2024-04-20T11:34:11.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:11.572+0000 7f8413400640 1 --2- 172.21.15.152:0/3958068133 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f841415ba70 0x7f841415bef0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f84140d03e0 tx=0x7f840c004760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:11.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:11.572+0000 7f840aa00640 1 -- 172.21.15.152:0/3958068133 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f840c019070 con 0x7f841415ba70 2024-04-20T11:34:11.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:11.572+0000 7f840aa00640 1 -- 172.21.15.152:0/3958068133 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f840c007cf0 con 0x7f841415ba70 2024-04-20T11:34:11.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:11.572+0000 7f8419600640 1 -- 172.21.15.152:0/3958068133 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f841415c6c0 con 0x7f841415ba70 2024-04-20T11:34:11.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:11.572+0000 7f840aa00640 1 -- 172.21.15.152:0/3958068133 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f840c014640 con 0x7f841415ba70 2024-04-20T11:34:11.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:11.572+0000 7f8419600640 1 -- 172.21.15.152:0/3958068133 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f841415cc80 con 0x7f841415ba70 2024-04-20T11:34:11.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:11.573+0000 7f8419600640 1 -- 172.21.15.152:0/3958068133 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84140689f0 con 0x7f841415ba70 2024-04-20T11:34:11.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:11.574+0000 7f840aa00640 1 -- 172.21.15.152:0/3958068133 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 18) ==== 91295+0+0 (secure 0 0 0) 0x7f840c01c070 con 0x7f841415ba70 2024-04-20T11:34:11.574 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:11.575+0000 7f840aa00640 1 --2- 172.21.15.152:0/3958068133 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f83f406cbc0 0x7f83f406f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:11.574 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:11.575+0000 7f840aa00640 1 -- 172.21.15.152:0/3958068133 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7f840c08bbd0 con 0x7f841415ba70 2024-04-20T11:34:11.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:11.576+0000 7f8412a00640 1 --2- 172.21.15.152:0/3958068133 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f83f406cbc0 0x7f83f406f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:11.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:11.577+0000 7f8412a00640 1 --2- 172.21.15.152:0/3958068133 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f83f406cbc0 0x7f83f406f080 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8404033000 tx=0x7f8404009030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:11.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:11.580+0000 7f840aa00640 1 -- 172.21.15.152:0/3958068133 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f840c059b90 con 0x7f841415ba70 2024-04-20T11:34:11.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:11.807+0000 7f8419600640 1 -- 172.21.15.152:0/3958068133 --> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol48180", "target": ["mon-mgr", ""]}) -- 0x7f841406ecf0 con 0x7f83f406cbc0 2024-04-20T11:34:11.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:11.920+0000 7f840aa00640 1 -- 172.21.15.152:0/3958068133 <== mgr.4109 v2:172.21.15.175:6836/4021250441 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f841406ecf0 con 0x7f83f406cbc0 2024-04-20T11:34:11.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:11.923+0000 7f8419600640 1 -- 172.21.15.152:0/3958068133 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f83f406cbc0 msgr2=0x7f83f406f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:11.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:11.923+0000 7f8419600640 1 --2- 172.21.15.152:0/3958068133 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f83f406cbc0 0x7f83f406f080 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8404033000 tx=0x7f8404009030 comp rx=0 tx=0).stop 2024-04-20T11:34:11.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:11.923+0000 7f8419600640 1 -- 172.21.15.152:0/3958068133 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f841415ba70 msgr2=0x7f841415bef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:11.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:11.923+0000 7f8419600640 1 --2- 172.21.15.152:0/3958068133 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f841415ba70 0x7f841415bef0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f84140d03e0 tx=0x7f840c004760 comp rx=0 tx=0).stop 2024-04-20T11:34:11.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:11.923+0000 7f8419600640 1 -- 172.21.15.152:0/3958068133 shutdown_connections 2024-04-20T11:34:11.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:11.923+0000 7f8419600640 1 --2- 172.21.15.152:0/3958068133 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f83f406cbc0 0x7f83f406f080 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:11.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:11.923+0000 7f8419600640 1 --2- 172.21.15.152:0/3958068133 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f841415ba70 0x7f841415bef0 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:11.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:11.923+0000 7f8419600640 1 --2- 172.21.15.152:0/3958068133 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f84140b0f70 0x7f84140b13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:11.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:11.923+0000 7f8419600640 1 --2- 172.21.15.152:0/3958068133 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f83f8005bf0 0x7f8414141da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:11.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:11.923+0000 7f8419600640 1 -- 172.21.15.152:0/3958068133 >> 172.21.15.152:0/3958068133 conn(0x7f841406b980 msgr2=0x7f84140cf3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:11.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:11.923+0000 7f8419600640 1 -- 172.21.15.152:0/3958068133 shutdown_connections 2024-04-20T11:34:11.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:11.923+0000 7f8419600640 1 -- 172.21.15.152:0/3958068133 wait complete. 2024-04-20T11:34:11.930 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:34:11.930+0000 7f8a5f400640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:34:11.930 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:34:11.930+0000 7f8a5f400640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:34:11.930 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:34:11.930+0000 7f8a5f400640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:34:11.930 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:34:11.930+0000 7f8a5f400640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:34:11.930 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:34:11.930+0000 7f8a5f400640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:34:11.948 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol_clone89010 2024-04-20T11:34:11.963 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:34:11.963+0000 7f8a5fe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:34:11.963 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:34:11.963+0000 7f8a5fe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:34:11.963 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:34:11.963+0000 7f8a5fe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:34:11.963 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:34:11.963+0000 7f8a5fe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:34:11.963 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:34:11.963+0000 7f8a5fe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:34:12.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.160+0000 7fc99b000640 1 Processor -- start 2024-04-20T11:34:12.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.160+0000 7fc99b000640 1 -- start start 2024-04-20T11:34:12.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.161+0000 7fc99b000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fc9940b2880 0x7fc9940b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:12.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.161+0000 7fc99b000640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9941463e0 con 0x7fc994146000 2024-04-20T11:34:12.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.161+0000 7fc99b000640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9940b3e20 con 0x7fc9940cd1b0 2024-04-20T11:34:12.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.161+0000 7fc99b000640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fc9940b4750 con 0x7fc9940b2880 2024-04-20T11:34:12.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.162+0000 7fc993400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fc9940b2880 0x7fc9940b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:12.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.162+0000 7fc993e00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fc994146000 0x7fc9940b2170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:35022/0 (socket says 172.21.15.152:35022) 2024-04-20T11:34:12.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.162+0000 7fc993e00640 1 -- 172.21.15.152:0/1042302040 learned_addr learned my addr 172.21.15.152:0/1042302040 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:34:12.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.162+0000 7fc992a00640 1 -- 172.21.15.152:0/1042302040 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1813223099 0 0) 0x7fc9941463e0 con 0x7fc994146000 2024-04-20T11:34:12.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.162+0000 7fc992a00640 1 -- 172.21.15.152:0/1042302040 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc978003520 con 0x7fc994146000 2024-04-20T11:34:12.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.162+0000 7fc993400640 1 -- 172.21.15.152:0/1042302040 >> v1:172.21.15.175:6789/0 conn(0x7fc9940cd1b0 legacy=0x7fc9940cd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:12.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.162+0000 7fc993400640 1 -- 172.21.15.152:0/1042302040 >> v1:172.21.15.152:6789/0 conn(0x7fc994146000 legacy=0x7fc9940b2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:12.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.162+0000 7fc993400640 1 -- 172.21.15.152:0/1042302040 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9940b4100 con 0x7fc9940b2880 2024-04-20T11:34:12.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.163+0000 7fc993400640 1 --2- 172.21.15.152:0/1042302040 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc9940b2880 0x7fc9940b4be0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7fc9800097a0 tx=0x7fc98002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cca5509ac00bbe90 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:12.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.164+0000 7fc992a00640 1 -- 172.21.15.152:0/1042302040 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc980035020 con 0x7fc9940b2880 2024-04-20T11:34:12.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.164+0000 7fc992a00640 1 -- 172.21.15.152:0/1042302040 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc980031830 con 0x7fc9940b2880 2024-04-20T11:34:12.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.164+0000 7fc992a00640 1 -- 172.21.15.152:0/1042302040 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc980031c00 con 0x7fc9940b2880 2024-04-20T11:34:12.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.164+0000 7fc99b000640 1 -- 172.21.15.152:0/1042302040 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc9940b2880 msgr2=0x7fc9940b4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:12.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.164+0000 7fc99b000640 1 --2- 172.21.15.152:0/1042302040 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc9940b2880 0x7fc9940b4be0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7fc9800097a0 tx=0x7fc98002b8d0 comp rx=0 tx=0).stop 2024-04-20T11:34:12.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.164+0000 7fc99b000640 1 -- 172.21.15.152:0/1042302040 shutdown_connections 2024-04-20T11:34:12.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.164+0000 7fc99b000640 1 --2- 172.21.15.152:0/1042302040 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc9940b2880 0x7fc9940b4be0 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:12.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.164+0000 7fc99b000640 1 -- 172.21.15.152:0/1042302040 >> 172.21.15.152:0/1042302040 conn(0x7fc99406b980 msgr2=0x7fc99406bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:12.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.164+0000 7fc99b000640 1 -- 172.21.15.152:0/1042302040 shutdown_connections 2024-04-20T11:34:12.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.164+0000 7fc99b000640 1 -- 172.21.15.152:0/1042302040 wait complete. 2024-04-20T11:34:12.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.166+0000 7fc99b000640 1 Processor -- start 2024-04-20T11:34:12.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.166+0000 7fc99b000640 1 -- start start 2024-04-20T11:34:12.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.167+0000 7fc99b000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc9940b2880 0x7fc99415c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:12.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.167+0000 7fc99b000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc9940cd1b0 0x7fc994143610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:12.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.167+0000 7fc99b000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc994146000 0x7fc994143b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:12.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.167+0000 7fc99b000640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fc99415ce70 con 0x7fc9940b2880 2024-04-20T11:34:12.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.167+0000 7fc99b000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc99415ccf0 con 0x7fc9940cd1b0 2024-04-20T11:34:12.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.167+0000 7fc99b000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc99415cff0 con 0x7fc994146000 2024-04-20T11:34:12.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.167+0000 7fc993e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc9940b2880 0x7fc99415c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:12.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.167+0000 7fc993e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc9940b2880 0x7fc99415c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:53332/0 (socket says 172.21.15.152:53332) 2024-04-20T11:34:12.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.167+0000 7fc993e00640 1 -- 172.21.15.152:0/3059435295 learned_addr learned my addr 172.21.15.152:0/3059435295 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:12.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.168+0000 7fc993400640 1 --2- 172.21.15.152:0/3059435295 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc9940cd1b0 0x7fc994143610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:12.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.168+0000 7fc999000640 1 --2- 172.21.15.152:0/3059435295 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc994146000 0x7fc994143b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:12.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.168+0000 7fc993e00640 1 -- 172.21.15.152:0/3059435295 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc994146000 msgr2=0x7fc994143b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:12.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.168+0000 7fc993e00640 1 --2- 172.21.15.152:0/3059435295 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc994146000 0x7fc994143b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:12.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.168+0000 7fc993e00640 1 -- 172.21.15.152:0/3059435295 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc9940cd1b0 msgr2=0x7fc994143610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:12.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.168+0000 7fc993e00640 1 --2- 172.21.15.152:0/3059435295 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc9940cd1b0 0x7fc994143610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:12.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.168+0000 7fc993e00640 1 -- 172.21.15.152:0/3059435295 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9941f5150 con 0x7fc9940b2880 2024-04-20T11:34:12.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.168+0000 7fc993e00640 1 --2- 172.21.15.152:0/3059435295 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc9940b2880 0x7fc99415c120 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7fc98400c410 tx=0x7fc98400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:12.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.169+0000 7fc98be00640 1 -- 172.21.15.152:0/3059435295 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc984017070 con 0x7fc9940b2880 2024-04-20T11:34:12.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.169+0000 7fc98be00640 1 -- 172.21.15.152:0/3059435295 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc984005ce0 con 0x7fc9940b2880 2024-04-20T11:34:12.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.169+0000 7fc98be00640 1 -- 172.21.15.152:0/3059435295 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9840072f0 con 0x7fc9940b2880 2024-04-20T11:34:12.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.169+0000 7fc99b000640 1 -- 172.21.15.152:0/3059435295 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9941f5340 con 0x7fc9940b2880 2024-04-20T11:34:12.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.169+0000 7fc99b000640 1 -- 172.21.15.152:0/3059435295 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9941f4e70 con 0x7fc9940b2880 2024-04-20T11:34:12.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.170+0000 7fc99b000640 1 -- 172.21.15.152:0/3059435295 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9940689f0 con 0x7fc9940b2880 2024-04-20T11:34:12.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.171+0000 7fc98be00640 1 -- 172.21.15.152:0/3059435295 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 18) ==== 91295+0+0 (secure 0 0 0) 0x7fc98401a070 con 0x7fc9940b2880 2024-04-20T11:34:12.172 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.172+0000 7fc98be00640 1 --2- 172.21.15.152:0/3059435295 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fc97406cbf0 0x7fc97406f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:12.172 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.172+0000 7fc98be00640 1 -- 172.21.15.152:0/3059435295 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5765+0+0 (secure 0 0 0) 0x7fc98408b440 con 0x7fc9940b2880 2024-04-20T11:34:12.172 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.173+0000 7fc993400640 1 --2- 172.21.15.152:0/3059435295 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fc97406cbf0 0x7fc97406f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:12.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.174+0000 7fc993400640 1 --2- 172.21.15.152:0/3059435295 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fc97406cbf0 0x7fc97406f0b0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc980036000 tx=0x7fc980008030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:12.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.177+0000 7fc98be00640 1 -- 172.21.15.152:0/3059435295 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fc9840595a0 con 0x7fc9940b2880 2024-04-20T11:34:12.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.416+0000 7fc99b000640 1 -- 172.21.15.152:0/3059435295 --> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol_clone89010", "target": ["mon-mgr", ""]}) -- 0x7fc99406cd90 con 0x7fc97406cbf0 2024-04-20T11:34:12.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.458+0000 7fc98be00640 1 -- 172.21.15.152:0/3059435295 <== mgr.4109 v2:172.21.15.175:6836/4021250441 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc99406cd90 con 0x7fc97406cbf0 2024-04-20T11:34:12.461 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.461+0000 7fc99b000640 1 -- 172.21.15.152:0/3059435295 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fc97406cbf0 msgr2=0x7fc97406f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:12.461 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.461+0000 7fc99b000640 1 --2- 172.21.15.152:0/3059435295 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fc97406cbf0 0x7fc97406f0b0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc980036000 tx=0x7fc980008030 comp rx=0 tx=0).stop 2024-04-20T11:34:12.461 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.461+0000 7fc99b000640 1 -- 172.21.15.152:0/3059435295 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc9940b2880 msgr2=0x7fc99415c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:12.461 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.461+0000 7fc99b000640 1 --2- 172.21.15.152:0/3059435295 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc9940b2880 0x7fc99415c120 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7fc98400c410 tx=0x7fc98400c8d0 comp rx=0 tx=0).stop 2024-04-20T11:34:12.462 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.461+0000 7fc99b000640 1 -- 172.21.15.152:0/3059435295 shutdown_connections 2024-04-20T11:34:12.462 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.461+0000 7fc99b000640 1 --2- 172.21.15.152:0/3059435295 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fc97406cbf0 0x7fc97406f0b0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:12.462 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.461+0000 7fc99b000640 1 --2- 172.21.15.152:0/3059435295 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc994146000 0x7fc994143b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:12.462 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.461+0000 7fc99b000640 1 --2- 172.21.15.152:0/3059435295 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc9940cd1b0 0x7fc994143610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:12.462 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.461+0000 7fc99b000640 1 --2- 172.21.15.152:0/3059435295 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc9940b2880 0x7fc99415c120 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:12.462 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.461+0000 7fc99b000640 1 -- 172.21.15.152:0/3059435295 >> 172.21.15.152:0/3059435295 conn(0x7fc99406b980 msgr2=0x7fc99415f390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:12.462 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.462+0000 7fc99b000640 1 -- 172.21.15.152:0/3059435295 shutdown_connections 2024-04-20T11:34:12.462 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.462+0000 7fc99b000640 1 -- 172.21.15.152:0/3059435295 wait complete. 2024-04-20T11:34:12.484 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol_clone68251 2024-04-20T11:34:12.692 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.693+0000 7fccf3000640 1 Processor -- start 2024-04-20T11:34:12.692 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.693+0000 7fccf3000640 1 -- start start 2024-04-20T11:34:12.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.694+0000 7fccf3000640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fccec160220 0x7fccec1606b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:12.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.694+0000 7fccf3000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fccec141450 0x7fccec1418e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:12.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.694+0000 7fccf3000640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fccec06fb40 con 0x7fccec071cd0 2024-04-20T11:34:12.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.694+0000 7fccf3000640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fccec13f8e0 con 0x7fccec141450 2024-04-20T11:34:12.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.694+0000 7fccf3000640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fccec160f90 con 0x7fccec160220 2024-04-20T11:34:12.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.695+0000 7fccf1c00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fccec071cd0 0x7fccec0720b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:35026/0 (socket says 172.21.15.152:35026) 2024-04-20T11:34:12.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.695+0000 7fccf2600640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fccec141450 0x7fccec1418e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:12.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.695+0000 7fccf1c00640 1 -- 172.21.15.152:0/3367834894 learned_addr learned my addr 172.21.15.152:0/3367834894 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:34:12.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.695+0000 7fccf1200640 1 --2- 172.21.15.152:0/3367834894 >> v2:172.21.15.175:3300/0 conn(0x7fccec160220 0x7fccec1606b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:12.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.695+0000 7fccf2600640 1 -- 172.21.15.152:0/3367834894 >> v2:172.21.15.175:3300/0 conn(0x7fccec160220 msgr2=0x7fccec1606b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:12.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.695+0000 7fccf2600640 1 --2- 172.21.15.152:0/3367834894 >> v2:172.21.15.175:3300/0 conn(0x7fccec160220 0x7fccec1606b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:12.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.695+0000 7fccf2600640 1 -- 172.21.15.152:0/3367834894 >> v1:172.21.15.152:6789/0 conn(0x7fccec071cd0 legacy=0x7fccec0720b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:12.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.695+0000 7fccf2600640 1 -- 172.21.15.152:0/3367834894 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccec06dd90 con 0x7fccec141450 2024-04-20T11:34:12.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.695+0000 7fccebe00640 1 -- 172.21.15.152:0/3367834894 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3659507233 0 0) 0x7fccec06fb40 con 0x7fccec071cd0 2024-04-20T11:34:12.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.696+0000 7fccf2600640 1 --2- 172.21.15.152:0/3367834894 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fccec141450 0x7fccec1418e0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7fcce4009d90 tx=0x7fcce402be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c09739630e3347b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:12.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.696+0000 7fccebe00640 1 -- 172.21.15.152:0/3367834894 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce4039070 con 0x7fccec141450 2024-04-20T11:34:12.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.696+0000 7fccebe00640 1 -- 172.21.15.152:0/3367834894 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcce402cbd0 con 0x7fccec141450 2024-04-20T11:34:12.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.696+0000 7fccebe00640 1 -- 172.21.15.152:0/3367834894 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce4034440 con 0x7fccec141450 2024-04-20T11:34:12.697 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.697+0000 7fccf3000640 1 -- 172.21.15.152:0/3367834894 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fccec141450 msgr2=0x7fccec1418e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:12.697 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.697+0000 7fccf3000640 1 --2- 172.21.15.152:0/3367834894 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fccec141450 0x7fccec1418e0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7fcce4009d90 tx=0x7fcce402be20 comp rx=0 tx=0).stop 2024-04-20T11:34:12.697 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.697+0000 7fccf3000640 1 -- 172.21.15.152:0/3367834894 shutdown_connections 2024-04-20T11:34:12.697 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.697+0000 7fccf3000640 1 --2- 172.21.15.152:0/3367834894 >> v2:172.21.15.175:3300/0 conn(0x7fccec160220 0x7fccec1606b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:12.697 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.697+0000 7fccf3000640 1 --2- 172.21.15.152:0/3367834894 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fccec141450 0x7fccec1418e0 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:12.697 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.697+0000 7fccf3000640 1 -- 172.21.15.152:0/3367834894 >> 172.21.15.152:0/3367834894 conn(0x7fccec067ae0 msgr2=0x7fccec06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:12.698 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.697+0000 7fccf3000640 1 -- 172.21.15.152:0/3367834894 shutdown_connections 2024-04-20T11:34:12.698 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.697+0000 7fccf3000640 1 -- 172.21.15.152:0/3367834894 wait complete. 2024-04-20T11:34:12.698 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.699+0000 7fccf3000640 1 Processor -- start 2024-04-20T11:34:12.698 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.699+0000 7fccf3000640 1 -- start start 2024-04-20T11:34:12.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.700+0000 7fccf3000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fccec071cd0 0x7fccec13cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:12.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.700+0000 7fccf3000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fccec141450 0x7fccec1e6520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:12.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.700+0000 7fccf3000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fccec160220 0x7fccec1e8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:12.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.700+0000 7fccf3000640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fccec06e630 con 0x7fccec071cd0 2024-04-20T11:34:12.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.700+0000 7fccf3000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fccec06e350 con 0x7fccec160220 2024-04-20T11:34:12.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.700+0000 7fccf3000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fccec160bf0 con 0x7fccec141450 2024-04-20T11:34:12.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.700+0000 7fccf1c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fccec071cd0 0x7fccec13cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:12.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.700+0000 7fccf1c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fccec071cd0 0x7fccec13cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:53356/0 (socket says 172.21.15.152:53356) 2024-04-20T11:34:12.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.700+0000 7fccf1c00640 1 -- 172.21.15.152:0/4061553697 learned_addr learned my addr 172.21.15.152:0/4061553697 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:12.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.701+0000 7fccf1200640 1 --2- 172.21.15.152:0/4061553697 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fccec141450 0x7fccec1e6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:12.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.701+0000 7fccf2600640 1 --2- 172.21.15.152:0/4061553697 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fccec160220 0x7fccec1e8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:12.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.701+0000 7fccf1c00640 1 -- 172.21.15.152:0/4061553697 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fccec141450 msgr2=0x7fccec1e6520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:12.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.701+0000 7fccf1c00640 1 --2- 172.21.15.152:0/4061553697 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fccec141450 0x7fccec1e6520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:12.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.701+0000 7fccf1c00640 1 -- 172.21.15.152:0/4061553697 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fccec160220 msgr2=0x7fccec1e8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:12.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.701+0000 7fccf1c00640 1 --2- 172.21.15.152:0/4061553697 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fccec160220 0x7fccec1e8cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:12.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.701+0000 7fccf1c00640 1 -- 172.21.15.152:0/4061553697 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccec1e6f70 con 0x7fccec071cd0 2024-04-20T11:34:12.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.701+0000 7fccf1c00640 1 --2- 172.21.15.152:0/4061553697 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fccec071cd0 0x7fccec13cb00 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7fcce000c500 tx=0x7fcce000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:12.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.701+0000 7fcce9600640 1 -- 172.21.15.152:0/4061553697 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce0018070 con 0x7fccec071cd0 2024-04-20T11:34:12.703 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.702+0000 7fcce9600640 1 -- 172.21.15.152:0/4061553697 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcce0007580 con 0x7fccec071cd0 2024-04-20T11:34:12.703 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.702+0000 7fcce9600640 1 -- 172.21.15.152:0/4061553697 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce0004a20 con 0x7fccec071cd0 2024-04-20T11:34:12.703 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.702+0000 7fccf3000640 1 -- 172.21.15.152:0/4061553697 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccec1e7250 con 0x7fccec071cd0 2024-04-20T11:34:12.703 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.702+0000 7fccf3000640 1 -- 172.21.15.152:0/4061553697 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccec1e7810 con 0x7fccec071cd0 2024-04-20T11:34:12.703 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.703+0000 7fccf3000640 1 -- 172.21.15.152:0/4061553697 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccec075e60 con 0x7fccec071cd0 2024-04-20T11:34:12.703 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.704+0000 7fcce9600640 1 -- 172.21.15.152:0/4061553697 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 18) ==== 91295+0+0 (secure 0 0 0) 0x7fcce0004bc0 con 0x7fccec071cd0 2024-04-20T11:34:12.705 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.705+0000 7fcce9600640 1 --2- 172.21.15.152:0/4061553697 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fcccc06cbc0 0x7fcccc06f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:12.705 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.705+0000 7fcce9600640 1 -- 172.21.15.152:0/4061553697 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5765+0+0 (secure 0 0 0) 0x7fcce008b310 con 0x7fccec071cd0 2024-04-20T11:34:12.705 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.706+0000 7fccf1200640 1 --2- 172.21.15.152:0/4061553697 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fcccc06cbc0 0x7fcccc06f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:12.707 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.707+0000 7fccf1200640 1 --2- 172.21.15.152:0/4061553697 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fcccc06cbc0 0x7fcccc06f080 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fccd8006fd0 tx=0x7fccd8008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:12.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.710+0000 7fcce9600640 1 -- 172.21.15.152:0/4061553697 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fcce0059400 con 0x7fccec071cd0 2024-04-20T11:34:12.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.943+0000 7fccf3000640 1 -- 172.21.15.152:0/4061553697 --> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol_clone68251", "target": ["mon-mgr", ""]}) -- 0x7fccec06c740 con 0x7fcccc06cbc0 2024-04-20T11:34:12.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.983+0000 7fcce9600640 1 -- 172.21.15.152:0/4061553697 <== mgr.4109 v2:172.21.15.175:6836/4021250441 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fccec06c740 con 0x7fcccc06cbc0 2024-04-20T11:34:12.986 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.986+0000 7fccf3000640 1 -- 172.21.15.152:0/4061553697 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fcccc06cbc0 msgr2=0x7fcccc06f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:12.986 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.986+0000 7fccf3000640 1 --2- 172.21.15.152:0/4061553697 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fcccc06cbc0 0x7fcccc06f080 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fccd8006fd0 tx=0x7fccd8008040 comp rx=0 tx=0).stop 2024-04-20T11:34:12.986 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.986+0000 7fccf3000640 1 -- 172.21.15.152:0/4061553697 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fccec071cd0 msgr2=0x7fccec13cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:12.986 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.986+0000 7fccf3000640 1 --2- 172.21.15.152:0/4061553697 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fccec071cd0 0x7fccec13cb00 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7fcce000c500 tx=0x7fcce000c9c0 comp rx=0 tx=0).stop 2024-04-20T11:34:12.986 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.986+0000 7fccf3000640 1 -- 172.21.15.152:0/4061553697 shutdown_connections 2024-04-20T11:34:12.986 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.986+0000 7fccf3000640 1 --2- 172.21.15.152:0/4061553697 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fcccc06cbc0 0x7fcccc06f080 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:12.986 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.986+0000 7fccf3000640 1 --2- 172.21.15.152:0/4061553697 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fccec160220 0x7fccec1e8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:12.986 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.986+0000 7fccf3000640 1 --2- 172.21.15.152:0/4061553697 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fccec141450 0x7fccec1e6520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:12.986 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.986+0000 7fccf3000640 1 --2- 172.21.15.152:0/4061553697 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fccec071cd0 0x7fccec13cb00 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:12.986 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.986+0000 7fccf3000640 1 -- 172.21.15.152:0/4061553697 >> 172.21.15.152:0/4061553697 conn(0x7fccec067ae0 msgr2=0x7fccec0b0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:12.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.987+0000 7fccf3000640 1 -- 172.21.15.152:0/4061553697 shutdown_connections 2024-04-20T11:34:12.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:12.987+0000 7fccf3000640 1 -- 172.21.15.152:0/4061553697 wait complete. 2024-04-20T11:34:13.008 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-04-20T11:34:13.109 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:13 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-20T11:34:13.119 INFO:teuthology.orchestra.run.smithi152.stdout:2 2024-04-20T11:34:13.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T11:34:13.132 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-20T11:34:13.133 DEBUG:teuthology.orchestra.run.smithi152:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-20T11:34:13.222 DEBUG:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-20T11:34:13.340 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-20T11:34:13.340 INFO:tasks.cephfs.mount:Terminating background process 2024-04-20T11:34:13.340 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-20T11:34:13.341 DEBUG:teuthology.orchestra.run.smithi152:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:34:13.396 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:34:13.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:13.598+0000 7f4ef2400640 1 Processor -- start 2024-04-20T11:34:13.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:13.599+0000 7f4ef2400640 1 -- start start 2024-04-20T11:34:13.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:13.600+0000 7f4ef2400640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f4eec0b0340 0x7f4eec0ae310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:13.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:13.600+0000 7f4ef2400640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4eec0b07f0 con 0x7f4eec06b980 2024-04-20T11:34:13.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:13.600+0000 7f4ef2400640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4eec0b63e0 con 0x7f4eec15ec00 2024-04-20T11:34:13.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:13.600+0000 7f4ef2400640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f4eec0b66c0 con 0x7f4eec0b0340 2024-04-20T11:34:13.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:13.600+0000 7f4eeb400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f4eec06b980 0x7f4eec06bd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:35036/0 (socket says 172.21.15.152:35036) 2024-04-20T11:34:13.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:13.600+0000 7f4eeb400640 1 -- 172.21.15.152:0/889399638 learned_addr learned my addr 172.21.15.152:0/889399638 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:34:13.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:13.601+0000 7f4eeaa00640 1 -- 172.21.15.152:0/889399638 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1909866047 0 0) 0x7f4eec0b07f0 con 0x7f4eec06b980 2024-04-20T11:34:13.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:13.601+0000 7f4eeaa00640 1 -- 172.21.15.152:0/889399638 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4ed0003520 con 0x7f4eec06b980 2024-04-20T11:34:13.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:13.601+0000 7f4eebe00640 1 --2- 172.21.15.152:0/889399638 >> v2:172.21.15.175:3300/0 conn(0x7f4eec0b0340 0x7f4eec0ae310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:13.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:13.601+0000 7f4eeaa00640 1 -- 172.21.15.152:0/889399638 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3867809285 0 0) 0x7f4ed0003520 con 0x7f4eec06b980 2024-04-20T11:34:13.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:13.601+0000 7f4eeaa00640 1 -- 172.21.15.152:0/889399638 >> v2:172.21.15.175:3300/0 conn(0x7f4eec0b0340 msgr2=0x7f4eec0ae310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:13.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:13.601+0000 7f4eeaa00640 1 --2- 172.21.15.152:0/889399638 >> v2:172.21.15.175:3300/0 conn(0x7f4eec0b0340 0x7f4eec0ae310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:13.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:13.601+0000 7f4eeaa00640 1 -- 172.21.15.152:0/889399638 >> v1:172.21.15.175:6789/0 conn(0x7f4eec15ec00 legacy=0x7f4eec06fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:13.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:13.601+0000 7f4eeaa00640 1 -- 172.21.15.152:0/889399638 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4eec0b44d0 con 0x7f4eec06b980 2024-04-20T11:34:13.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:13.601+0000 7f4eeaa00640 1 -- 172.21.15.152:0/889399638 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f4edc001670 con 0x7f4eec06b980 2024-04-20T11:34:13.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:13.601+0000 7f4eeaa00640 1 -- 172.21.15.152:0/889399638 >> v1:172.21.15.152:6789/0 conn(0x7f4eec06b980 legacy=0x7f4eec06bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:13.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:13.601+0000 7f4eeaa00640 1 --2- 172.21.15.152:0/889399638 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ed0003a20 0x7f4ed0003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:13.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:13.602+0000 7f4eeaa00640 1 --2- 172.21.15.152:0/889399638 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4ed0004800 0x7f4ed0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:13.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:13.602+0000 7f4eeaa00640 1 --2- 172.21.15.152:0/889399638 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4ed0005370 0x7f4ed000b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:13.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:13.602+0000 7f4eeaa00640 1 -- 172.21.15.152:0/889399638 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f4eec0b66c0 con 0x7f4ed0003a20 2024-04-20T11:34:13.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:13.602+0000 7f4eeaa00640 1 -- 172.21.15.152:0/889399638 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4ed000c2c0 con 0x7f4ed0004800 2024-04-20T11:34:13.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:13.602+0000 7f4eeaa00640 1 -- 172.21.15.152:0/889399638 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4ed000c4c0 con 0x7f4ed0005370 2024-04-20T11:34:13.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:13.602+0000 7f4eeb400640 1 --2- 172.21.15.152:0/889399638 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ed0003a20 0x7f4ed0003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:13.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:13.602+0000 7f4ee3e00640 1 --2- 172.21.15.152:0/889399638 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4ed0004800 0x7f4ed0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:13.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:13.602+0000 7f4eeb400640 1 -- 172.21.15.152:0/889399638 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4ed0005370 msgr2=0x7f4ed000b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:13.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:13.602+0000 7f4eeb400640 1 --2- 172.21.15.152:0/889399638 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4ed0005370 0x7f4ed000b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:13.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:13.602+0000 7f4eeb400640 1 -- 172.21.15.152:0/889399638 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4ed0004800 msgr2=0x7f4ed0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:13.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:13.602+0000 7f4eeb400640 1 --2- 172.21.15.152:0/889399638 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4ed0004800 0x7f4ed0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:13.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:13.602+0000 7f4eebe00640 1 -- 172.21.15.152:0/889399638 reap_dead start 2024-04-20T11:34:13.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:13.602+0000 7f4eeb400640 1 -- 172.21.15.152:0/889399638 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ed0003520 con 0x7f4ed0003a20 2024-04-20T11:34:13.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:13.602+0000 7f4ee3e00640 1 --2- 172.21.15.152:0/889399638 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4ed0004800 0x7f4ed0004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:34:13.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:13.603+0000 7f4eeb400640 1 --2- 172.21.15.152:0/889399638 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ed0003a20 0x7f4ed0003e40 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f4edc00c210 tx=0x7f4ed0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63a98f54935d5a66 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:13.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:13.604+0000 7f4eeaa00640 1 -- 172.21.15.152:0/889399638 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4edc007d30 con 0x7f4ed0003a20 2024-04-20T11:34:13.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:13.604+0000 7f4eeaa00640 1 -- 172.21.15.152:0/889399638 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4edc030070 con 0x7f4ed0003a20 2024-04-20T11:34:13.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:13.604+0000 7f4eeaa00640 1 -- 172.21.15.152:0/889399638 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4edc02cda0 con 0x7f4ed0003a20 2024-04-20T11:34:13.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:13.604+0000 7f4ef2400640 1 -- 172.21.15.152:0/889399638 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ed0003a20 msgr2=0x7f4ed0003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:13.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:13.604+0000 7f4ef2400640 1 --2- 172.21.15.152:0/889399638 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ed0003a20 0x7f4ed0003e40 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f4edc00c210 tx=0x7f4ed0005830 comp rx=0 tx=0).stop 2024-04-20T11:34:13.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:13.604+0000 7f4ef2400640 1 -- 172.21.15.152:0/889399638 shutdown_connections 2024-04-20T11:34:13.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:13.604+0000 7f4ef2400640 1 --2- 172.21.15.152:0/889399638 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ed0003a20 0x7f4ed0003e40 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:13.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:13.604+0000 7f4ef2400640 1 -- 172.21.15.152:0/889399638 >> 172.21.15.152:0/889399638 conn(0x7f4eec09aa40 msgr2=0x7f4eec0b68c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:13.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:13.604+0000 7f4ef2400640 1 -- 172.21.15.152:0/889399638 shutdown_connections 2024-04-20T11:34:13.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:13.604+0000 7f4ef2400640 1 -- 172.21.15.152:0/889399638 wait complete. 2024-04-20T11:34:13.608 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:13.606+0000 7f4ef2400640 1 Processor -- start 2024-04-20T11:34:13.608 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:13.606+0000 7f4ef2400640 1 -- start start 2024-04-20T11:34:13.608 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:13.607+0000 7f4ef2400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4ed0003a20 0x7f4eec15c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:13.608 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:13.607+0000 7f4ef2400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4eec0ccdb0 0x7f4eec0aee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:13.608 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:13.607+0000 7f4ef2400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4eec0af3a0 0x7f4eec157ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:13.608 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:13.607+0000 7f4ef2400640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f4eec0b0c50 con 0x7f4eec0af3a0 2024-04-20T11:34:13.608 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:13.607+0000 7f4ef2400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4eec0b4a90 con 0x7f4ed0003a20 2024-04-20T11:34:13.608 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:13.607+0000 7f4ef2400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4eec0b0dd0 con 0x7f4eec0ccdb0 2024-04-20T11:34:13.608 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:13.608+0000 7f4eebe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4eec0af3a0 0x7f4eec157ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:13.608 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:13.608+0000 7f4eebe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4eec0af3a0 0x7f4eec157ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:53368/0 (socket says 172.21.15.152:53368) 2024-04-20T11:34:13.608 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:13.608+0000 7f4eebe00640 1 -- 172.21.15.152:0/3264014673 learned_addr learned my addr 172.21.15.152:0/3264014673 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:13.608 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:13.608+0000 7f4eeb400640 1 --2- 172.21.15.152:0/3264014673 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4ed0003a20 0x7f4eec15c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:13.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:13.608+0000 7f4ee3e00640 1 --2- 172.21.15.152:0/3264014673 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4eec0ccdb0 0x7f4eec0aee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:13.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:13.608+0000 7f4eebe00640 1 -- 172.21.15.152:0/3264014673 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4eec0ccdb0 msgr2=0x7f4eec0aee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:13.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:13.608+0000 7f4eebe00640 1 --2- 172.21.15.152:0/3264014673 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4eec0ccdb0 0x7f4eec0aee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:13.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:13.608+0000 7f4eebe00640 1 -- 172.21.15.152:0/3264014673 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4ed0003a20 msgr2=0x7f4eec15c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:13.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:13.608+0000 7f4eebe00640 1 --2- 172.21.15.152:0/3264014673 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4ed0003a20 0x7f4eec15c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:13.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:13.608+0000 7f4eebe00640 1 -- 172.21.15.152:0/3264014673 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4eec0cd270 con 0x7f4eec0af3a0 2024-04-20T11:34:13.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:13.608+0000 7f4eebe00640 1 --2- 172.21.15.152:0/3264014673 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4eec0af3a0 0x7f4eec157ac0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f4eec09ae50 tx=0x7f4ee400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:13.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:13.609+0000 7f4ee3400640 1 -- 172.21.15.152:0/3264014673 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee400ed90 con 0x7f4eec0af3a0 2024-04-20T11:34:13.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:13.609+0000 7f4ee3400640 1 -- 172.21.15.152:0/3264014673 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ee4019070 con 0x7f4eec0af3a0 2024-04-20T11:34:13.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:13.609+0000 7f4ef2400640 1 -- 172.21.15.152:0/3264014673 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4eec0cd550 con 0x7f4eec0af3a0 2024-04-20T11:34:13.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:13.609+0000 7f4ee3400640 1 -- 172.21.15.152:0/3264014673 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee4014660 con 0x7f4eec0af3a0 2024-04-20T11:34:13.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:13.609+0000 7f4ef2400640 1 -- 172.21.15.152:0/3264014673 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4eec1572c0 con 0x7f4eec0af3a0 2024-04-20T11:34:13.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:13.610+0000 7f4ef2400640 1 -- 172.21.15.152:0/3264014673 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4eec069100 con 0x7f4eec0af3a0 2024-04-20T11:34:13.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:13.611+0000 7f4ee3400640 1 -- 172.21.15.152:0/3264014673 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 18) ==== 91295+0+0 (secure 0 0 0) 0x7f4ee4014800 con 0x7f4eec0af3a0 2024-04-20T11:34:13.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:13.612+0000 7f4ee3400640 1 --2- 172.21.15.152:0/3264014673 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4ec806cbc0 0x7f4ec806f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:13.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:13.613+0000 7f4ee3400640 1 -- 172.21.15.152:0/3264014673 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5781+0+0 (secure 0 0 0) 0x7f4ee408c4e0 con 0x7f4eec0af3a0 2024-04-20T11:34:13.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:13.613+0000 7f4eeb400640 1 --2- 172.21.15.152:0/3264014673 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4ec806cbc0 0x7f4ec806f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:13.614 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:13.614+0000 7f4eeb400640 1 --2- 172.21.15.152:0/3264014673 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4ec806cbc0 0x7f4ec806f080 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4edc034000 tx=0x7f4edc009030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:13.616 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:13.617+0000 7f4ee3400640 1 -- 172.21.15.152:0/3264014673 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f4ee405a530 con 0x7f4eec0af3a0 2024-04-20T11:34:13.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:13.816+0000 7f4ef2400640 1 -- 172.21.15.152:0/3264014673 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4eec06c580 con 0x7f4eec0af3a0 2024-04-20T11:34:13.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:13.819+0000 7f4ee3400640 1 -- 172.21.15.152:0/3264014673 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v105) ==== 74+0+16821 (secure 0 0 0) 0x7f4ee4059e30 con 0x7f4eec0af3a0 2024-04-20T11:34:13.819 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:34:13.819 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":105,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:34:13.116969+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:33:25.062365+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-20T11:33:26.260945+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":105,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[{"pool":17,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:34:13.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:13.821+0000 7f4ef2400640 1 -- 172.21.15.152:0/3264014673 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4ec806cbc0 msgr2=0x7f4ec806f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:13.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:13.821+0000 7f4ef2400640 1 --2- 172.21.15.152:0/3264014673 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4ec806cbc0 0x7f4ec806f080 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4edc034000 tx=0x7f4edc009030 comp rx=0 tx=0).stop 2024-04-20T11:34:13.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:13.822+0000 7f4ef2400640 1 -- 172.21.15.152:0/3264014673 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4eec0af3a0 msgr2=0x7f4eec157ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:13.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:13.822+0000 7f4ef2400640 1 --2- 172.21.15.152:0/3264014673 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4eec0af3a0 0x7f4eec157ac0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f4eec09ae50 tx=0x7f4ee400e4b0 comp rx=0 tx=0).stop 2024-04-20T11:34:13.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:13.822+0000 7f4ef2400640 1 -- 172.21.15.152:0/3264014673 shutdown_connections 2024-04-20T11:34:13.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:13.822+0000 7f4ef2400640 1 --2- 172.21.15.152:0/3264014673 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4ec806cbc0 0x7f4ec806f080 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:13.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:13.822+0000 7f4ef2400640 1 --2- 172.21.15.152:0/3264014673 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4eec0af3a0 0x7f4eec157ac0 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:13.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:13.822+0000 7f4ef2400640 1 --2- 172.21.15.152:0/3264014673 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4eec0ccdb0 0x7f4eec0aee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:13.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:13.822+0000 7f4ef2400640 1 --2- 172.21.15.152:0/3264014673 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4ed0003a20 0x7f4eec15c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:13.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:13.822+0000 7f4ef2400640 1 -- 172.21.15.152:0/3264014673 >> 172.21.15.152:0/3264014673 conn(0x7f4eec09aa40 msgr2=0x7f4eec0b2730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:13.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:13.822+0000 7f4ef2400640 1 -- 172.21.15.152:0/3264014673 shutdown_connections 2024-04-20T11:34:13.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:13.822+0000 7f4ef2400640 1 -- 172.21.15.152:0/3264014673 wait complete. 2024-04-20T11:34:13.841 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T11:26:47.246571+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T11:33:25.062365+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '102', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T11:33:26.260945+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '105', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 5, 'snap_epoch': 105, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T11:34:13.842 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T11:34:14.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.047+0000 7f8535c00640 1 Processor -- start 2024-04-20T11:34:14.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.048+0000 7f8535c00640 1 -- start start 2024-04-20T11:34:14.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.049+0000 7f8535c00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f85300b3930 0x7f853009b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:14.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.049+0000 7f8535c00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85300d0dc0 con 0x7f8530162d90 2024-04-20T11:34:14.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.049+0000 7f8535c00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85300d10a0 con 0x7f85300d0800 2024-04-20T11:34:14.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.049+0000 7f8535c00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f85300d19d0 con 0x7f85300b3930 2024-04-20T11:34:14.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.049+0000 7f852f400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f8530162d90 0x7f853015e1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:35042/0 (socket says 172.21.15.152:35042) 2024-04-20T11:34:14.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.049+0000 7f852f400640 1 -- 172.21.15.152:0/1567556078 learned_addr learned my addr 172.21.15.152:0/1567556078 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:34:14.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.049+0000 7f852ea00640 1 --2- 172.21.15.152:0/1567556078 >> v2:172.21.15.152:3300/0 conn(0x7f85300b3930 0x7f853009b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:14.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.050+0000 7f852ea00640 1 -- 172.21.15.152:0/1567556078 >> v1:172.21.15.175:6789/0 conn(0x7f85300d0800 legacy=0x7f8530162680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:14.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.050+0000 7f852ea00640 1 -- 172.21.15.152:0/1567556078 >> v1:172.21.15.152:6789/0 conn(0x7f8530162d90 legacy=0x7f853015e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:14.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.050+0000 7f852d600640 1 -- 172.21.15.152:0/1567556078 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3054633403 0 0) 0x7f85300d0dc0 con 0x7f8530162d90 2024-04-20T11:34:14.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.050+0000 7f852ea00640 1 -- 172.21.15.152:0/1567556078 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85300d1380 con 0x7f85300b3930 2024-04-20T11:34:14.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.051+0000 7f852ea00640 1 --2- 172.21.15.152:0/1567556078 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f85300b3930 0x7f853009b6a0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f8520009d90 tx=0x7f852002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e8c70cb1251d2d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:14.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.051+0000 7f852d600640 1 -- 172.21.15.152:0/1567556078 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8520039070 con 0x7f85300b3930 2024-04-20T11:34:14.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.051+0000 7f852d600640 1 -- 172.21.15.152:0/1567556078 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f852002cc60 con 0x7f85300b3930 2024-04-20T11:34:14.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.051+0000 7f852d600640 1 -- 172.21.15.152:0/1567556078 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8520034660 con 0x7f85300b3930 2024-04-20T11:34:14.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.052+0000 7f8535c00640 1 -- 172.21.15.152:0/1567556078 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f85300b3930 msgr2=0x7f853009b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:14.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.052+0000 7f8535c00640 1 --2- 172.21.15.152:0/1567556078 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f85300b3930 0x7f853009b6a0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f8520009d90 tx=0x7f852002beb0 comp rx=0 tx=0).stop 2024-04-20T11:34:14.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.052+0000 7f8535c00640 1 -- 172.21.15.152:0/1567556078 shutdown_connections 2024-04-20T11:34:14.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.052+0000 7f8535c00640 1 --2- 172.21.15.152:0/1567556078 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f85300b3930 0x7f853009b6a0 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:14.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.052+0000 7f8535c00640 1 -- 172.21.15.152:0/1567556078 >> 172.21.15.152:0/1567556078 conn(0x7f85300b4020 msgr2=0x7f85300b4430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:14.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.052+0000 7f8535c00640 1 -- 172.21.15.152:0/1567556078 shutdown_connections 2024-04-20T11:34:14.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.052+0000 7f8535c00640 1 -- 172.21.15.152:0/1567556078 wait complete. 2024-04-20T11:34:14.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.054+0000 7f8535c00640 1 Processor -- start 2024-04-20T11:34:14.054 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.054+0000 7f8535c00640 1 -- start start 2024-04-20T11:34:14.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.055+0000 7f8535c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f85300b3930 0x7f853015a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:14.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.055+0000 7f8535c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85300d0800 0x7f853015c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:14.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.055+0000 7f8535c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8530162d90 0x7f853014a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:14.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.055+0000 7f8535c00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f85300b3d10 con 0x7f8530162d90 2024-04-20T11:34:14.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.055+0000 7f8535c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f853009bbe0 con 0x7f85300d0800 2024-04-20T11:34:14.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.055+0000 7f8535c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8530159f10 con 0x7f85300b3930 2024-04-20T11:34:14.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.055+0000 7f852f400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8530162d90 0x7f853014a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:14.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.055+0000 7f852f400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8530162d90 0x7f853014a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:53376/0 (socket says 172.21.15.152:53376) 2024-04-20T11:34:14.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.055+0000 7f852f400640 1 -- 172.21.15.152:0/3956396547 learned_addr learned my addr 172.21.15.152:0/3956396547 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:14.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.056+0000 7f852e000640 1 --2- 172.21.15.152:0/3956396547 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85300d0800 0x7f853015c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:14.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.056+0000 7f852ea00640 1 --2- 172.21.15.152:0/3956396547 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f85300b3930 0x7f853015a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:14.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.056+0000 7f852f400640 1 -- 172.21.15.152:0/3956396547 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f85300b3930 msgr2=0x7f853015a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:14.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.056+0000 7f852f400640 1 --2- 172.21.15.152:0/3956396547 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f85300b3930 0x7f853015a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:14.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.056+0000 7f852f400640 1 -- 172.21.15.152:0/3956396547 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85300d0800 msgr2=0x7f853015c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:14.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.056+0000 7f852f400640 1 --2- 172.21.15.152:0/3956396547 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85300d0800 0x7f853015c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:14.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.056+0000 7f852f400640 1 -- 172.21.15.152:0/3956396547 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8530162000 con 0x7f8530162d90 2024-04-20T11:34:14.057 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.056+0000 7f852f400640 1 --2- 172.21.15.152:0/3956396547 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8530162d90 0x7f853014a4b0 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f852800c410 tx=0x7f852800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:14.057 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.056+0000 7f8526a00640 1 -- 172.21.15.152:0/3956396547 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85280074e0 con 0x7f8530162d90 2024-04-20T11:34:14.057 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.056+0000 7f8526a00640 1 -- 172.21.15.152:0/3956396547 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85280047e0 con 0x7f8530162d90 2024-04-20T11:34:14.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.056+0000 7f8535c00640 1 -- 172.21.15.152:0/3956396547 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8530159300 con 0x7f8530162d90 2024-04-20T11:34:14.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.057+0000 7f8526a00640 1 -- 172.21.15.152:0/3956396547 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8528004bd0 con 0x7f8530162d90 2024-04-20T11:34:14.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.057+0000 7f8535c00640 1 -- 172.21.15.152:0/3956396547 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8530159ba0 con 0x7f8530162d90 2024-04-20T11:34:14.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.058+0000 7f8535c00640 1 -- 172.21.15.152:0/3956396547 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85300689f0 con 0x7f8530162d90 2024-04-20T11:34:14.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.059+0000 7f8526a00640 1 -- 172.21.15.152:0/3956396547 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 18) ==== 91295+0+0 (secure 0 0 0) 0x7f8528011070 con 0x7f8530162d90 2024-04-20T11:34:14.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.060+0000 7f8526a00640 1 --2- 172.21.15.152:0/3956396547 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f850c06cbc0 0x7f850c06f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:14.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.060+0000 7f8526a00640 1 -- 172.21.15.152:0/3956396547 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5781+0+0 (secure 0 0 0) 0x7f852808bb80 con 0x7f8530162d90 2024-04-20T11:34:14.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.060+0000 7f852ea00640 1 --2- 172.21.15.152:0/3956396547 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f850c06cbc0 0x7f850c06f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:14.061 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.061+0000 7f852ea00640 1 --2- 172.21.15.152:0/3956396547 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f850c06cbc0 0x7f850c06f080 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8520036000 tx=0x7f8520030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:14.063 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.064+0000 7f8526a00640 1 -- 172.21.15.152:0/3956396547 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f8530162000 con 0x7f8530162d90 2024-04-20T11:34:14.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.168+0000 7f8526a00640 1 -- 172.21.15.152:0/3956396547 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mgrmap(e 19) ==== 91397+0+0 (secure 0 0 0) 0x7f8528055b40 con 0x7f8530162d90 2024-04-20T11:34:14.270 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.270+0000 7f8535c00640 1 -- 172.21.15.152:0/3956396547 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f85300b4c80 con 0x7f8530162d90 2024-04-20T11:34:14.270 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.271+0000 7f8526a00640 1 -- 172.21.15.152:0/3956396547 <== mon.0 v2:172.21.15.152:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v105) ==== 93+0+10 (secure 0 0 0) 0x7f8528056b20 con 0x7f8530162d90 2024-04-20T11:34:14.271 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 1 2024-04-20T11:34:14.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.273+0000 7f8535c00640 1 -- 172.21.15.152:0/3956396547 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f850c06cbc0 msgr2=0x7f850c06f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:14.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.273+0000 7f8535c00640 1 --2- 172.21.15.152:0/3956396547 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f850c06cbc0 0x7f850c06f080 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8520036000 tx=0x7f8520030000 comp rx=0 tx=0).stop 2024-04-20T11:34:14.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.273+0000 7f8535c00640 1 -- 172.21.15.152:0/3956396547 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8530162d90 msgr2=0x7f853014a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:14.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.273+0000 7f8535c00640 1 --2- 172.21.15.152:0/3956396547 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8530162d90 0x7f853014a4b0 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f852800c410 tx=0x7f852800c8d0 comp rx=0 tx=0).stop 2024-04-20T11:34:14.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.274+0000 7f8535c00640 1 -- 172.21.15.152:0/3956396547 shutdown_connections 2024-04-20T11:34:14.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.274+0000 7f8535c00640 1 --2- 172.21.15.152:0/3956396547 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f850c06cbc0 0x7f850c06f080 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:14.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.274+0000 7f8535c00640 1 --2- 172.21.15.152:0/3956396547 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8530162d90 0x7f853014a4b0 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:14.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.274+0000 7f8535c00640 1 --2- 172.21.15.152:0/3956396547 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85300d0800 0x7f853015c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:14.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.274+0000 7f8535c00640 1 --2- 172.21.15.152:0/3956396547 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f85300b3930 0x7f853015a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:14.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.274+0000 7f8535c00640 1 -- 172.21.15.152:0/3956396547 >> 172.21.15.152:0/3956396547 conn(0x7f85300b4020 msgr2=0x7f85300b7b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:14.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.274+0000 7f8535c00640 1 -- 172.21.15.152:0/3956396547 shutdown_connections 2024-04-20T11:34:14.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.274+0000 7f8535c00640 1 -- 172.21.15.152:0/3956396547 wait complete. 2024-04-20T11:34:14.293 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T11:34:14.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.492+0000 7feef0200640 1 Processor -- start 2024-04-20T11:34:14.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.492+0000 7feef0200640 1 -- start start 2024-04-20T11:34:14.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.493+0000 7feef0200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feee80cfee0 0x7feee815ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:14.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.494+0000 7feef0200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7feee80b2170 0x7feee80b25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:14.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.494+0000 7feef0200640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feee80b45e0 con 0x7feee815f0f0 2024-04-20T11:34:14.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.494+0000 7feef0200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7feee8158840 con 0x7feee80b2170 2024-04-20T11:34:14.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.494+0000 7feef0200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7feee80b2b10 con 0x7feee80cfee0 2024-04-20T11:34:14.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.494+0000 7feeee200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7feee80b2170 0x7feee80b25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:14.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.494+0000 7feeece00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7feee815f0f0 0x7feee80b3ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:35048/0 (socket says 172.21.15.152:35048) 2024-04-20T11:34:14.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.494+0000 7feeece00640 1 -- 172.21.15.152:0/2651188549 learned_addr learned my addr 172.21.15.152:0/2651188549 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:34:14.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.494+0000 7feeed800640 1 --2- 172.21.15.152:0/2651188549 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feee80cfee0 0x7feee815ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:14.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.495+0000 7feeee200640 1 -- 172.21.15.152:0/2651188549 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feee80cfee0 msgr2=0x7feee815ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:14.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.495+0000 7feeee200640 1 --2- 172.21.15.152:0/2651188549 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feee80cfee0 0x7feee815ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:14.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.495+0000 7feeee200640 1 -- 172.21.15.152:0/2651188549 >> v1:172.21.15.152:6789/0 conn(0x7feee815f0f0 legacy=0x7feee80b3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:14.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.495+0000 7feeee200640 1 -- 172.21.15.152:0/2651188549 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feee80b4890 con 0x7feee80b2170 2024-04-20T11:34:14.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.496+0000 7feeee200640 1 --2- 172.21.15.152:0/2651188549 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feee80b2170 0x7feee80b25d0 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7feee0009d90 tx=0x7feee002cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9e68344b1d2136b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:14.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.496+0000 7feee7e00640 1 -- 172.21.15.152:0/2651188549 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee003a070 con 0x7feee80b2170 2024-04-20T11:34:14.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.496+0000 7feee7e00640 1 -- 172.21.15.152:0/2651188549 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feee002dce0 con 0x7feee80b2170 2024-04-20T11:34:14.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.496+0000 7feee7e00640 1 -- 172.21.15.152:0/2651188549 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee0035660 con 0x7feee80b2170 2024-04-20T11:34:14.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.497+0000 7feef0200640 1 -- 172.21.15.152:0/2651188549 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feee80b2170 msgr2=0x7feee80b25d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:14.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.497+0000 7feef0200640 1 --2- 172.21.15.152:0/2651188549 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feee80b2170 0x7feee80b25d0 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7feee0009d90 tx=0x7feee002cf30 comp rx=0 tx=0).stop 2024-04-20T11:34:14.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.497+0000 7feef0200640 1 -- 172.21.15.152:0/2651188549 shutdown_connections 2024-04-20T11:34:14.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.497+0000 7feef0200640 1 --2- 172.21.15.152:0/2651188549 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feee80b2170 0x7feee80b25d0 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:14.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.497+0000 7feef0200640 1 --2- 172.21.15.152:0/2651188549 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feee80cfee0 0x7feee815ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:14.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.497+0000 7feef0200640 1 -- 172.21.15.152:0/2651188549 >> 172.21.15.152:0/2651188549 conn(0x7feee806b980 msgr2=0x7feee806bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:14.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.497+0000 7feef0200640 1 -- 172.21.15.152:0/2651188549 shutdown_connections 2024-04-20T11:34:14.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.497+0000 7feef0200640 1 -- 172.21.15.152:0/2651188549 wait complete. 2024-04-20T11:34:14.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.499+0000 7feef0200640 1 Processor -- start 2024-04-20T11:34:14.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.499+0000 7feef0200640 1 -- start start 2024-04-20T11:34:14.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.500+0000 7feef0200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feee80b2170 0x7feee8155e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:14.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.500+0000 7feef0200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feee80cfee0 0x7feee8152f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:14.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.500+0000 7feef0200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feee815f0f0 0x7feee80ce630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:14.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.500+0000 7feef0200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7feee815b5a0 con 0x7feee80b2170 2024-04-20T11:34:14.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.500+0000 7feef0200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7feee80b4b70 con 0x7feee815f0f0 2024-04-20T11:34:14.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.500+0000 7feef0200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7feee815b230 con 0x7feee80cfee0 2024-04-20T11:34:14.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.500+0000 7feeed800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feee80b2170 0x7feee8155e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:14.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.501+0000 7feeed800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feee80b2170 0x7feee8155e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:53394/0 (socket says 172.21.15.152:53394) 2024-04-20T11:34:14.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.501+0000 7feeed800640 1 -- 172.21.15.152:0/3077017229 learned_addr learned my addr 172.21.15.152:0/3077017229 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:14.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.501+0000 7feeee200640 1 --2- 172.21.15.152:0/3077017229 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feee815f0f0 0x7feee80ce630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:14.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.501+0000 7feeed800640 1 -- 172.21.15.152:0/3077017229 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feee80cfee0 msgr2=0x7feee8152f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:14.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.501+0000 7feeece00640 1 --2- 172.21.15.152:0/3077017229 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feee80cfee0 0x7feee8152f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:14.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.501+0000 7feeed800640 1 --2- 172.21.15.152:0/3077017229 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feee80cfee0 0x7feee8152f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:14.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.501+0000 7feeed800640 1 -- 172.21.15.152:0/3077017229 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feee815f0f0 msgr2=0x7feee80ce630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:14.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.501+0000 7feeed800640 1 --2- 172.21.15.152:0/3077017229 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feee815f0f0 0x7feee80ce630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:14.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.501+0000 7feeed800640 1 -- 172.21.15.152:0/3077017229 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feee80cd870 con 0x7feee80b2170 2024-04-20T11:34:14.502 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.501+0000 7feeee200640 1 --2- 172.21.15.152:0/3077017229 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feee815f0f0 0x7feee80ce630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:34:14.503 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.501+0000 7feeed800640 1 --2- 172.21.15.152:0/3077017229 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feee80b2170 0x7feee8155e70 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7feedc00c7f0 tx=0x7feedc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:14.503 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.502+0000 7feee5600640 1 -- 172.21.15.152:0/3077017229 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feedc015070 con 0x7feee80b2170 2024-04-20T11:34:14.503 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.502+0000 7feee5600640 1 -- 172.21.15.152:0/3077017229 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feedc004240 con 0x7feee80b2170 2024-04-20T11:34:14.503 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.502+0000 7feee5600640 1 -- 172.21.15.152:0/3077017229 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feedc007620 con 0x7feee80b2170 2024-04-20T11:34:14.503 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.502+0000 7feef0200640 1 -- 172.21.15.152:0/3077017229 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feee80cdb50 con 0x7feee80b2170 2024-04-20T11:34:14.503 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.502+0000 7feef0200640 1 -- 172.21.15.152:0/3077017229 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feee80ceb70 con 0x7feee80b2170 2024-04-20T11:34:14.503 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.503+0000 7feef0200640 1 -- 172.21.15.152:0/3077017229 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feee80689f0 con 0x7feee80b2170 2024-04-20T11:34:14.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.505+0000 7feee5600640 1 -- 172.21.15.152:0/3077017229 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 19) ==== 91397+0+0 (secure 0 0 0) 0x7feedc0079d0 con 0x7feee80b2170 2024-04-20T11:34:14.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.505+0000 7feee5600640 1 --2- 172.21.15.152:0/3077017229 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7feecc06cce0 0x7feecc06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:14.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.506+0000 7feee5600640 1 -- 172.21.15.152:0/3077017229 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5781+0+0 (secure 0 0 0) 0x7feedc08c860 con 0x7feee80b2170 2024-04-20T11:34:14.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.506+0000 7feeece00640 1 --2- 172.21.15.152:0/3077017229 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7feecc06cce0 0x7feecc06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:14.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.507+0000 7feeece00640 1 --2- 172.21.15.152:0/3077017229 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7feecc06cce0 0x7feecc06f1a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7feee8144860 tx=0x7feed8009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:14.510 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.511+0000 7feee5600640 1 -- 172.21.15.152:0/3077017229 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7feedc05a5a0 con 0x7feee80b2170 2024-04-20T11:34:14.710 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.710+0000 7feef0200640 1 -- 172.21.15.152:0/3077017229 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7feee806fc60 con 0x7feee80b2170 2024-04-20T11:34:14.710 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.711+0000 7feee5600640 1 -- 172.21.15.152:0/3077017229 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v105) ==== 104+0+11 (secure 0 0 0) 0x7feedc059ea0 con 0x7feee80b2170 2024-04-20T11:34:14.711 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 32 2024-04-20T11:34:14.714 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.714+0000 7feef0200640 1 -- 172.21.15.152:0/3077017229 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7feecc06cce0 msgr2=0x7feecc06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:14.714 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.714+0000 7feef0200640 1 --2- 172.21.15.152:0/3077017229 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7feecc06cce0 0x7feecc06f1a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7feee8144860 tx=0x7feed8009040 comp rx=0 tx=0).stop 2024-04-20T11:34:14.714 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.714+0000 7feef0200640 1 -- 172.21.15.152:0/3077017229 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feee80b2170 msgr2=0x7feee8155e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:14.714 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.714+0000 7feef0200640 1 --2- 172.21.15.152:0/3077017229 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feee80b2170 0x7feee8155e70 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7feedc00c7f0 tx=0x7feedc00ccb0 comp rx=0 tx=0).stop 2024-04-20T11:34:14.714 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.714+0000 7feef0200640 1 -- 172.21.15.152:0/3077017229 shutdown_connections 2024-04-20T11:34:14.714 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.714+0000 7feef0200640 1 --2- 172.21.15.152:0/3077017229 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7feecc06cce0 0x7feecc06f1a0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:14.714 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.714+0000 7feef0200640 1 --2- 172.21.15.152:0/3077017229 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feee815f0f0 0x7feee80ce630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:14.714 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.714+0000 7feef0200640 1 --2- 172.21.15.152:0/3077017229 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feee80cfee0 0x7feee8152f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:14.714 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.714+0000 7feef0200640 1 --2- 172.21.15.152:0/3077017229 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feee80b2170 0x7feee8155e70 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:14.714 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.714+0000 7feef0200640 1 -- 172.21.15.152:0/3077017229 >> 172.21.15.152:0/3077017229 conn(0x7feee806b980 msgr2=0x7feee815e230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:14.714 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.714+0000 7feef0200640 1 -- 172.21.15.152:0/3077017229 shutdown_connections 2024-04-20T11:34:14.715 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.714+0000 7feef0200640 1 -- 172.21.15.152:0/3077017229 wait complete. 2024-04-20T11:34:14.734 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T11:34:14.938 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.938+0000 7fb49ac00640 1 Processor -- start 2024-04-20T11:34:14.938 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.938+0000 7fb49ac00640 1 -- start start 2024-04-20T11:34:14.939 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.939+0000 7fb49ac00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fb4940cf5b0 0x7fb4940d4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:14.939 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.939+0000 7fb49ac00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fb4940ceeb0 0x7fb4940cf990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:14.939 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.939+0000 7fb49ac00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb494162430 0x7fb4941628e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:14.939 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.939+0000 7fb49ac00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fb4940d5420 con 0x7fb4940ceeb0 2024-04-20T11:34:14.939 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.939+0000 7fb49ac00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fb4940cfed0 con 0x7fb4940cf5b0 2024-04-20T11:34:14.939 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.939+0000 7fb49ac00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb4940d2b70 con 0x7fb494162430 2024-04-20T11:34:14.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.940+0000 7fb498e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fb4940ceeb0 0x7fb4940cf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:14.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.940+0000 7fb498e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fb4940ceeb0 0x7fb4940cf990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:53396/0 (socket says 172.21.15.152:53396) 2024-04-20T11:34:14.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.940+0000 7fb498e00640 1 -- 172.21.15.152:0/3165787258 learned_addr learned my addr 172.21.15.152:0/3165787258 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:14.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.940+0000 7fb499800640 1 --2- 172.21.15.152:0/3165787258 >> v2:172.21.15.175:3300/0 conn(0x7fb4940cf5b0 0x7fb4940d4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:14.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.940+0000 7fb49a200640 1 --2- 172.21.15.152:0/3165787258 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb494162430 0x7fb4941628e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:14.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.940+0000 7fb498e00640 1 -- 172.21.15.152:0/3165787258 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb494162430 msgr2=0x7fb4941628e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:14.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.940+0000 7fb498e00640 1 --2- 172.21.15.152:0/3165787258 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb494162430 0x7fb4941628e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:14.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.940+0000 7fb498e00640 1 -- 172.21.15.152:0/3165787258 >> v2:172.21.15.175:3300/0 conn(0x7fb4940cf5b0 msgr2=0x7fb4940d4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:14.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.940+0000 7fb498e00640 1 --2- 172.21.15.152:0/3165787258 >> v2:172.21.15.175:3300/0 conn(0x7fb4940cf5b0 0x7fb4940d4ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:14.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.940+0000 7fb498e00640 1 -- 172.21.15.152:0/3165787258 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4940d2e50 con 0x7fb4940ceeb0 2024-04-20T11:34:14.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.940+0000 7fb499800640 1 --2- 172.21.15.152:0/3165787258 >> v2:172.21.15.175:3300/0 conn(0x7fb4940cf5b0 0x7fb4940d4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:34:14.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.941+0000 7fb498e00640 1 --2- 172.21.15.152:0/3165787258 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb4940ceeb0 0x7fb4940cf990 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7fb49408cc10 tx=0x7fb48002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe26e5875eb8107 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:14.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.942+0000 7fb493e00640 1 -- 172.21.15.152:0/3165787258 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb480004230 con 0x7fb4940ceeb0 2024-04-20T11:34:14.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.942+0000 7fb493e00640 1 -- 172.21.15.152:0/3165787258 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4800043d0 con 0x7fb4940ceeb0 2024-04-20T11:34:14.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.942+0000 7fb493e00640 1 -- 172.21.15.152:0/3165787258 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb480030840 con 0x7fb4940ceeb0 2024-04-20T11:34:14.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.942+0000 7fb49ac00640 1 -- 172.21.15.152:0/3165787258 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb4940ceeb0 msgr2=0x7fb4940cf990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:14.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.942+0000 7fb49ac00640 1 --2- 172.21.15.152:0/3165787258 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb4940ceeb0 0x7fb4940cf990 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7fb49408cc10 tx=0x7fb48002b980 comp rx=0 tx=0).stop 2024-04-20T11:34:14.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.942+0000 7fb49ac00640 1 -- 172.21.15.152:0/3165787258 shutdown_connections 2024-04-20T11:34:14.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.942+0000 7fb49ac00640 1 --2- 172.21.15.152:0/3165787258 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb494162430 0x7fb4941628e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:14.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.943+0000 7fb49ac00640 1 --2- 172.21.15.152:0/3165787258 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb4940ceeb0 0x7fb4940cf990 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:14.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.943+0000 7fb49ac00640 1 --2- 172.21.15.152:0/3165787258 >> v2:172.21.15.175:3300/0 conn(0x7fb4940cf5b0 0x7fb4940d4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:14.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.943+0000 7fb49ac00640 1 -- 172.21.15.152:0/3165787258 >> 172.21.15.152:0/3165787258 conn(0x7fb49408bfd0 msgr2=0x7fb4940d4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:14.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.943+0000 7fb49ac00640 1 -- 172.21.15.152:0/3165787258 shutdown_connections 2024-04-20T11:34:14.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.943+0000 7fb49ac00640 1 -- 172.21.15.152:0/3165787258 wait complete. 2024-04-20T11:34:14.944 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.944+0000 7fb49ac00640 1 Processor -- start 2024-04-20T11:34:14.944 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.945+0000 7fb49ac00640 1 -- start start 2024-04-20T11:34:14.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.945+0000 7fb49ac00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4940ceeb0 0x7fb4941611f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:14.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.945+0000 7fb49ac00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb4940cf5b0 0x7fb4941e8230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:14.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.945+0000 7fb49ac00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb494162430 0x7fb4941e8770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:14.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.946+0000 7fb49ac00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fb4940d36f0 con 0x7fb494162430 2024-04-20T11:34:14.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.946+0000 7fb49ac00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb4940d3410 con 0x7fb4940ceeb0 2024-04-20T11:34:14.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.946+0000 7fb49ac00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb4940d3c10 con 0x7fb4940cf5b0 2024-04-20T11:34:14.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.946+0000 7fb49a200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb494162430 0x7fb4941e8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:14.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.946+0000 7fb49a200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb494162430 0x7fb4941e8770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:53398/0 (socket says 172.21.15.152:53398) 2024-04-20T11:34:14.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.946+0000 7fb49a200640 1 -- 172.21.15.152:0/1702489664 learned_addr learned my addr 172.21.15.152:0/1702489664 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:14.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.946+0000 7fb499800640 1 --2- 172.21.15.152:0/1702489664 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4940ceeb0 0x7fb4941611f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:14.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.946+0000 7fb498e00640 1 --2- 172.21.15.152:0/1702489664 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb4940cf5b0 0x7fb4941e8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:14.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.946+0000 7fb49a200640 1 -- 172.21.15.152:0/1702489664 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb4940cf5b0 msgr2=0x7fb4941e8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:14.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.946+0000 7fb49a200640 1 --2- 172.21.15.152:0/1702489664 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb4940cf5b0 0x7fb4941e8230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:14.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.946+0000 7fb49a200640 1 -- 172.21.15.152:0/1702489664 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4940ceeb0 msgr2=0x7fb4941611f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:14.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.946+0000 7fb49a200640 1 --2- 172.21.15.152:0/1702489664 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4940ceeb0 0x7fb4941611f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:14.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.946+0000 7fb49a200640 1 -- 172.21.15.152:0/1702489664 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4941e8cb0 con 0x7fb494162430 2024-04-20T11:34:14.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.946+0000 7fb499800640 1 --2- 172.21.15.152:0/1702489664 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4940ceeb0 0x7fb4941611f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:34:14.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.947+0000 7fb49a200640 1 --2- 172.21.15.152:0/1702489664 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb494162430 0x7fb4941e8770 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7fb48c00c7f0 tx=0x7fb48c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:14.948 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.947+0000 7fb491600640 1 -- 172.21.15.152:0/1702489664 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb48c015070 con 0x7fb494162430 2024-04-20T11:34:14.948 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.947+0000 7fb491600640 1 -- 172.21.15.152:0/1702489664 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb48c004220 con 0x7fb494162430 2024-04-20T11:34:14.948 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.947+0000 7fb491600640 1 -- 172.21.15.152:0/1702489664 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb48c003610 con 0x7fb494162430 2024-04-20T11:34:14.948 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.947+0000 7fb49ac00640 1 -- 172.21.15.152:0/1702489664 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4941e8f60 con 0x7fb494162430 2024-04-20T11:34:14.948 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.947+0000 7fb49ac00640 1 -- 172.21.15.152:0/1702489664 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4941e9800 con 0x7fb494162430 2024-04-20T11:34:14.948 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.948+0000 7fb49ac00640 1 -- 172.21.15.152:0/1702489664 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb494075e60 con 0x7fb494162430 2024-04-20T11:34:14.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.949+0000 7fb491600640 1 -- 172.21.15.152:0/1702489664 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 19) ==== 91397+0+0 (secure 0 0 0) 0x7fb48c0039c0 con 0x7fb494162430 2024-04-20T11:34:14.950 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.951+0000 7fb491600640 1 --2- 172.21.15.152:0/1702489664 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb47406cd10 0x7fb47406f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:14.950 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.951+0000 7fb491600640 1 -- 172.21.15.152:0/1702489664 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5781+0+0 (secure 0 0 0) 0x7fb48c08c460 con 0x7fb494162430 2024-04-20T11:34:14.950 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.951+0000 7fb499800640 1 --2- 172.21.15.152:0/1702489664 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb47406cd10 0x7fb47406f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:14.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.952+0000 7fb499800640 1 --2- 172.21.15.152:0/1702489664 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb47406cd10 0x7fb47406f1d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb484008fc0 tx=0x7fb484008510 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:14.955 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:14.956+0000 7fb491600640 1 -- 172.21.15.152:0/1702489664 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fb48c05a0c0 con 0x7fb494162430 2024-04-20T11:34:15.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:15.156+0000 7fb49ac00640 1 -- 172.21.15.152:0/1702489664 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb4940cf390 con 0x7fb494162430 2024-04-20T11:34:15.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:15.157+0000 7fb491600640 1 -- 172.21.15.152:0/1702489664 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v105) ==== 100+0+11 (secure 0 0 0) 0x7fb48c0599c0 con 0x7fb494162430 2024-04-20T11:34:15.157 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 64 2024-04-20T11:34:15.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:15.159+0000 7fb49ac00640 1 -- 172.21.15.152:0/1702489664 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb47406cd10 msgr2=0x7fb47406f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:15.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:15.159+0000 7fb49ac00640 1 --2- 172.21.15.152:0/1702489664 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb47406cd10 0x7fb47406f1d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb484008fc0 tx=0x7fb484008510 comp rx=0 tx=0).stop 2024-04-20T11:34:15.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:15.159+0000 7fb49ac00640 1 -- 172.21.15.152:0/1702489664 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb494162430 msgr2=0x7fb4941e8770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:15.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:15.159+0000 7fb49ac00640 1 --2- 172.21.15.152:0/1702489664 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb494162430 0x7fb4941e8770 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7fb48c00c7f0 tx=0x7fb48c00ccb0 comp rx=0 tx=0).stop 2024-04-20T11:34:15.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:15.160+0000 7fb49ac00640 1 -- 172.21.15.152:0/1702489664 shutdown_connections 2024-04-20T11:34:15.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:15.160+0000 7fb49ac00640 1 --2- 172.21.15.152:0/1702489664 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb47406cd10 0x7fb47406f1d0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:15.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:15.160+0000 7fb49ac00640 1 --2- 172.21.15.152:0/1702489664 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb494162430 0x7fb4941e8770 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:15.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:15.160+0000 7fb49ac00640 1 --2- 172.21.15.152:0/1702489664 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb4940cf5b0 0x7fb4941e8230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:15.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:15.160+0000 7fb49ac00640 1 --2- 172.21.15.152:0/1702489664 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4940ceeb0 0x7fb4941611f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:15.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:15.160+0000 7fb49ac00640 1 -- 172.21.15.152:0/1702489664 >> 172.21.15.152:0/1702489664 conn(0x7fb49408bfd0 msgr2=0x7fb4940c9690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:15.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:15.160+0000 7fb49ac00640 1 -- 172.21.15.152:0/1702489664 shutdown_connections 2024-04-20T11:34:15.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:15.160+0000 7fb49ac00640 1 -- 172.21.15.152:0/1702489664 wait complete. 2024-04-20T11:34:15.180 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-20T11:34:15.384 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:15.385+0000 7f4bcbe00640 1 Processor -- start 2024-04-20T11:34:15.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:15.385+0000 7f4bcbe00640 1 -- start start 2024-04-20T11:34:15.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:15.387+0000 7f4bcbe00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f4bcc06c090 0x7f4bcc09aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:15.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:15.387+0000 7f4bcbe00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f4bcc0ae250 0x7f4bcc0afc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:15.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:15.387+0000 7f4bcbe00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4bcc160d70 con 0x7f4bcc145fd0 2024-04-20T11:34:15.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:15.387+0000 7f4bcbe00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f4bcc0b3680 con 0x7f4bcc0ae250 2024-04-20T11:34:15.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:15.387+0000 7f4bcbe00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f4bcc161010 con 0x7f4bcc06c090 2024-04-20T11:34:15.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:15.387+0000 7f4bcb400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f4bcc0ae250 0x7f4bcc0afc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:15.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:15.387+0000 7f4bcb400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f4bcc0ae250 0x7f4bcc0afc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:53400/0 (socket says 172.21.15.152:53400) 2024-04-20T11:34:15.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:15.387+0000 7f4bcb400640 1 -- 172.21.15.152:0/3983428891 learned_addr learned my addr 172.21.15.152:0/3983428891 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:15.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:15.387+0000 7f4bca000640 1 --2- 172.21.15.152:0/3983428891 >> v2:172.21.15.175:3300/0 conn(0x7f4bcc06c090 0x7f4bcc09aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:15.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:15.387+0000 7f4bcb400640 1 -- 172.21.15.152:0/3983428891 >> v2:172.21.15.175:3300/0 conn(0x7f4bcc06c090 msgr2=0x7f4bcc09aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:15.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:15.388+0000 7f4bcb400640 1 --2- 172.21.15.152:0/3983428891 >> v2:172.21.15.175:3300/0 conn(0x7f4bcc06c090 0x7f4bcc09aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:15.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:15.388+0000 7f4bcb400640 1 -- 172.21.15.152:0/3983428891 >> v1:172.21.15.175:6789/0 conn(0x7f4bcc145fd0 legacy=0x7f4bcc06b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:15.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:15.388+0000 7f4bcb400640 1 -- 172.21.15.152:0/3983428891 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bcc0b3940 con 0x7f4bcc0ae250 2024-04-20T11:34:15.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:15.388+0000 7f4bca000640 1 --2- 172.21.15.152:0/3983428891 >> v2:172.21.15.175:3300/0 conn(0x7f4bcc06c090 0x7f4bcc09aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:34:15.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:15.389+0000 7f4bcb400640 1 --2- 172.21.15.152:0/3983428891 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4bcc0ae250 0x7f4bcc0afc80 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc072210 tx=0x7f4bc402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f97d9abfe482cd8d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:15.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:15.389+0000 7f4bc9600640 1 -- 172.21.15.152:0/3983428891 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bc4035020 con 0x7f4bcc0ae250 2024-04-20T11:34:15.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:15.389+0000 7f4bc9600640 1 -- 172.21.15.152:0/3983428891 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bc4031640 con 0x7f4bcc0ae250 2024-04-20T11:34:15.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:15.389+0000 7f4bc9600640 1 -- 172.21.15.152:0/3983428891 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bc402d040 con 0x7f4bcc0ae250 2024-04-20T11:34:15.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:15.390+0000 7f4bcbe00640 1 -- 172.21.15.152:0/3983428891 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4bcc0ae250 msgr2=0x7f4bcc0afc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:15.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:15.390+0000 7f4bcbe00640 1 --2- 172.21.15.152:0/3983428891 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4bcc0ae250 0x7f4bcc0afc80 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc072210 tx=0x7f4bc402b8f0 comp rx=0 tx=0).stop 2024-04-20T11:34:15.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:15.390+0000 7f4bcbe00640 1 -- 172.21.15.152:0/3983428891 shutdown_connections 2024-04-20T11:34:15.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:15.390+0000 7f4bcbe00640 1 --2- 172.21.15.152:0/3983428891 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4bcc0ae250 0x7f4bcc0afc80 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:15.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:15.390+0000 7f4bcbe00640 1 --2- 172.21.15.152:0/3983428891 >> v2:172.21.15.175:3300/0 conn(0x7f4bcc06c090 0x7f4bcc09aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:15.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:15.390+0000 7f4bcbe00640 1 -- 172.21.15.152:0/3983428891 >> 172.21.15.152:0/3983428891 conn(0x7f4bcc0731a0 msgr2=0x7f4bcc0b2f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:15.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:15.390+0000 7f4bcbe00640 1 -- 172.21.15.152:0/3983428891 shutdown_connections 2024-04-20T11:34:15.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:15.390+0000 7f4bcbe00640 1 -- 172.21.15.152:0/3983428891 wait complete. 2024-04-20T11:34:15.391 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:15.391+0000 7f4bcbe00640 1 Processor -- start 2024-04-20T11:34:15.391 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:15.392+0000 7f4bcbe00640 1 -- start start 2024-04-20T11:34:15.393 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:15.392+0000 7f4bcbe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4bcc06c090 0x7f4bcc15ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:15.393 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:15.393+0000 7f4bcbe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4bcc0ae250 0x7f4bcc0cb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:15.393 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:15.393+0000 7f4bcbe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4bcc145fd0 0x7f4bcc0cbee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:15.393 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:15.393+0000 7f4bcbe00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f4bcc0b0340 con 0x7f4bcc145fd0 2024-04-20T11:34:15.393 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:15.393+0000 7f4bcbe00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4bcc0b01c0 con 0x7f4bcc0ae250 2024-04-20T11:34:15.393 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:15.393+0000 7f4bcbe00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4bcc0b04c0 con 0x7f4bcc06c090 2024-04-20T11:34:15.394 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:15.393+0000 7f4bcb400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4bcc145fd0 0x7f4bcc0cbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:15.394 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:15.393+0000 7f4bcb400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4bcc145fd0 0x7f4bcc0cbee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:53408/0 (socket says 172.21.15.152:53408) 2024-04-20T11:34:15.394 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:15.393+0000 7f4bcb400640 1 -- 172.21.15.152:0/673022135 learned_addr learned my addr 172.21.15.152:0/673022135 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:15.394 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:15.393+0000 7f4bca000640 1 --2- 172.21.15.152:0/673022135 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4bcc0ae250 0x7f4bcc0cb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:15.394 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:15.393+0000 7f4bcaa00640 1 --2- 172.21.15.152:0/673022135 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4bcc06c090 0x7f4bcc15ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:15.394 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:15.393+0000 7f4bcb400640 1 -- 172.21.15.152:0/673022135 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4bcc06c090 msgr2=0x7f4bcc15ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:15.394 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:15.393+0000 7f4bcb400640 1 --2- 172.21.15.152:0/673022135 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4bcc06c090 0x7f4bcc15ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:15.394 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:15.393+0000 7f4bcb400640 1 -- 172.21.15.152:0/673022135 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4bcc0ae250 msgr2=0x7f4bcc0cb9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:15.394 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:15.393+0000 7f4bcb400640 1 --2- 172.21.15.152:0/673022135 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4bcc0ae250 0x7f4bcc0cb9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:15.394 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:15.393+0000 7f4bcb400640 1 -- 172.21.15.152:0/673022135 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bcc0cc560 con 0x7f4bcc145fd0 2024-04-20T11:34:15.394 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:15.393+0000 7f4bca000640 1 --2- 172.21.15.152:0/673022135 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4bcc0ae250 0x7f4bcc0cb9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:34:15.394 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:15.393+0000 7f4bcaa00640 1 --2- 172.21.15.152:0/673022135 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4bcc06c090 0x7f4bcc15ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:34:15.394 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:15.394+0000 7f4bcb400640 1 --2- 172.21.15.152:0/673022135 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4bcc145fd0 0x7f4bcc0cbee0 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f4bc402b930 tx=0x7f4bc40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:15.395 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:15.394+0000 7f4bc3400640 1 -- 172.21.15.152:0/673022135 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bc4040070 con 0x7f4bcc145fd0 2024-04-20T11:34:15.395 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:15.394+0000 7f4bc3400640 1 -- 172.21.15.152:0/673022135 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bc4039cd0 con 0x7f4bcc145fd0 2024-04-20T11:34:15.395 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:15.394+0000 7f4bcbe00640 1 -- 172.21.15.152:0/673022135 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bcc0cc840 con 0x7f4bcc145fd0 2024-04-20T11:34:15.395 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:15.394+0000 7f4bc3400640 1 -- 172.21.15.152:0/673022135 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bc4035040 con 0x7f4bcc145fd0 2024-04-20T11:34:15.395 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:15.394+0000 7f4bcbe00640 1 -- 172.21.15.152:0/673022135 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bcc0cd9f0 con 0x7f4bcc145fd0 2024-04-20T11:34:15.396 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:15.396+0000 7f4bcbe00640 1 -- 172.21.15.152:0/673022135 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bcc075e60 con 0x7f4bcc145fd0 2024-04-20T11:34:15.396 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:15.396+0000 7f4bc3400640 1 -- 172.21.15.152:0/673022135 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 19) ==== 91397+0+0 (secure 0 0 0) 0x7f4bc4038020 con 0x7f4bcc145fd0 2024-04-20T11:34:15.397 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:15.397+0000 7f4bc3400640 1 --2- 172.21.15.152:0/673022135 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4ba806cce0 0x7f4ba806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:15.397 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:15.397+0000 7f4bc3400640 1 -- 172.21.15.152:0/673022135 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5781+0+0 (secure 0 0 0) 0x7f4bc40adf20 con 0x7f4bcc145fd0 2024-04-20T11:34:15.397 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:15.398+0000 7f4bcaa00640 1 --2- 172.21.15.152:0/673022135 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4ba806cce0 0x7f4ba806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:15.399 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:15.399+0000 7f4bcaa00640 1 --2- 172.21.15.152:0/673022135 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4ba806cce0 0x7f4ba806f1a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc00a830 tx=0x7f4bbc008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:15.401 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:15.402+0000 7f4bc3400640 1 -- 172.21.15.152:0/673022135 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f4bc407bc90 con 0x7f4bcc145fd0 2024-04-20T11:34:15.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:15.620+0000 7f4bcbe00640 1 -- 172.21.15.152:0/673022135 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f4bcc0d1030 con 0x7f4bcc145fd0 2024-04-20T11:34:15.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:15.621+0000 7f4bc3400640 1 -- 172.21.15.152:0/673022135 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v105) ==== 80+0+0 (secure 0 0 0) 0x7f4bc407b590 con 0x7f4bcc145fd0 2024-04-20T11:34:15.620 INFO:teuthology.orchestra.run.smithi152.stderr:listed 0 entries 2024-04-20T11:34:15.623 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:15.623+0000 7f4bcbe00640 1 -- 172.21.15.152:0/673022135 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4ba806cce0 msgr2=0x7f4ba806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:15.623 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:15.623+0000 7f4bcbe00640 1 --2- 172.21.15.152:0/673022135 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4ba806cce0 0x7f4ba806f1a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc00a830 tx=0x7f4bbc008040 comp rx=0 tx=0).stop 2024-04-20T11:34:15.623 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:15.623+0000 7f4bcbe00640 1 -- 172.21.15.152:0/673022135 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4bcc145fd0 msgr2=0x7f4bcc0cbee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:15.623 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:15.623+0000 7f4bcbe00640 1 --2- 172.21.15.152:0/673022135 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4bcc145fd0 0x7f4bcc0cbee0 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f4bc402b930 tx=0x7f4bc40046c0 comp rx=0 tx=0).stop 2024-04-20T11:34:15.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:15.624+0000 7f4bcbe00640 1 -- 172.21.15.152:0/673022135 shutdown_connections 2024-04-20T11:34:15.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:15.624+0000 7f4bcbe00640 1 --2- 172.21.15.152:0/673022135 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4ba806cce0 0x7f4ba806f1a0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:15.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:15.624+0000 7f4bcbe00640 1 --2- 172.21.15.152:0/673022135 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4bcc145fd0 0x7f4bcc0cbee0 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:15.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:15.624+0000 7f4bcbe00640 1 --2- 172.21.15.152:0/673022135 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4bcc0ae250 0x7f4bcc0cb9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:15.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:15.624+0000 7f4bcbe00640 1 --2- 172.21.15.152:0/673022135 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4bcc06c090 0x7f4bcc15ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:15.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:15.624+0000 7f4bcbe00640 1 -- 172.21.15.152:0/673022135 >> 172.21.15.152:0/673022135 conn(0x7f4bcc0731a0 msgr2=0x7f4bcc0b0850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:15.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:15.624+0000 7f4bcbe00640 1 -- 172.21.15.152:0/673022135 shutdown_connections 2024-04-20T11:34:15.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:15.624+0000 7f4bcbe00640 1 -- 172.21.15.152:0/673022135 wait complete. 2024-04-20T11:34:15.641 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi152.front.sepia.ceph.com... 2024-04-20T11:34:15.641 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:34:15.642 DEBUG:teuthology.orchestra.run.smithi152:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-20T11:34:15.715 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi152.stderr:ceph-fuse[54476]: fuse finished with error 0 and tester_r 0 2024-04-20T11:34:15.719 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi152.front.sepia.ceph.com 2024-04-20T11:34:15.719 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:34:15.719 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-20T11:34:15.812 INFO:tasks.cephfs.mount:Terminating background process 2024-04-20T11:34:15.812 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T11:34:15.812 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:34:15.854 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:34:16.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.031+0000 7f6024c00640 1 Processor -- start 2024-04-20T11:34:16.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.031+0000 7f6024c00640 1 -- start start 2024-04-20T11:34:16.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.033+0000 7f6024c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6020145f60 0x7f602006ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:16.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.033+0000 7f6024c00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f60200726f0 0x7f60200cd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:16.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.033+0000 7f6024c00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6020146340 con 0x7f60200cd0d0 2024-04-20T11:34:16.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.033+0000 7f6024c00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f602006e9f0 con 0x7f60200726f0 2024-04-20T11:34:16.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.033+0000 7f6024c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f602006f3c0 con 0x7f6020145f60 2024-04-20T11:34:16.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.033+0000 7f601e800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f60200726f0 0x7f60200cd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:16.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.033+0000 7f601e800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f60200726f0 0x7f60200cd550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52104/0 (socket says 172.21.15.152:52104) 2024-04-20T11:34:16.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.033+0000 7f601e800640 1 -- 172.21.15.152:0/1412798888 learned_addr learned my addr 172.21.15.152:0/1412798888 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:16.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.033+0000 7f6015400640 1 --1- 172.21.15.152:0/1412798888 >> v1:172.21.15.152:6789/0 conn(0x7f60200cd0d0 0x7f6020071fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:40182/0 (socket says 172.21.15.152:40182) 2024-04-20T11:34:16.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.034+0000 7f601de00640 1 --2- 172.21.15.152:0/1412798888 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6020145f60 0x7f602006ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:16.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.034+0000 7f601e800640 1 -- 172.21.15.152:0/1412798888 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6020145f60 msgr2=0x7f602006ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:16.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.034+0000 7f601e800640 1 --2- 172.21.15.152:0/1412798888 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6020145f60 0x7f602006ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:16.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.034+0000 7f601e800640 1 -- 172.21.15.152:0/1412798888 >> v1:172.21.15.152:6789/0 conn(0x7f60200cd0d0 legacy=0x7f6020071fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:16.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.034+0000 7f601e800640 1 -- 172.21.15.152:0/1412798888 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6020072d50 con 0x7f60200726f0 2024-04-20T11:34:16.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.034+0000 7f601d400640 1 -- 172.21.15.152:0/1412798888 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3408298982 0 0) 0x7f6020146340 con 0x7f60200cd0d0 2024-04-20T11:34:16.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.034+0000 7f601de00640 1 --2- 172.21.15.152:0/1412798888 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6020145f60 0x7f602006ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:34:16.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.035+0000 7f601e800640 1 --2- 172.21.15.152:0/1412798888 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f60200726f0 0x7f60200cd550 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f602006eb70 tx=0x7f601802bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebc1734423eae19e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:16.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.035+0000 7f601d400640 1 -- 172.21.15.152:0/1412798888 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f601802cae0 con 0x7f60200726f0 2024-04-20T11:34:16.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.035+0000 7f601d400640 1 -- 172.21.15.152:0/1412798888 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f601802cc80 con 0x7f60200726f0 2024-04-20T11:34:16.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.036+0000 7f601d400640 1 -- 172.21.15.152:0/1412798888 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6018034610 con 0x7f60200726f0 2024-04-20T11:34:16.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.036+0000 7f6024c00640 1 -- 172.21.15.152:0/1412798888 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f60200726f0 msgr2=0x7f60200cd550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:16.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.036+0000 7f6024c00640 1 --2- 172.21.15.152:0/1412798888 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f60200726f0 0x7f60200cd550 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f602006eb70 tx=0x7f601802bf40 comp rx=0 tx=0).stop 2024-04-20T11:34:16.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.036+0000 7f6024c00640 1 -- 172.21.15.152:0/1412798888 shutdown_connections 2024-04-20T11:34:16.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.036+0000 7f6024c00640 1 --2- 172.21.15.152:0/1412798888 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f60200726f0 0x7f60200cd550 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:16.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.036+0000 7f6024c00640 1 --2- 172.21.15.152:0/1412798888 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6020145f60 0x7f602006ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:16.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.036+0000 7f6024c00640 1 -- 172.21.15.152:0/1412798888 >> 172.21.15.152:0/1412798888 conn(0x7f6020067cf0 msgr2=0x7f602006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:16.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.036+0000 7f6024c00640 1 -- 172.21.15.152:0/1412798888 shutdown_connections 2024-04-20T11:34:16.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.036+0000 7f6024c00640 1 -- 172.21.15.152:0/1412798888 wait complete. 2024-04-20T11:34:16.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.038+0000 7f6024c00640 1 Processor -- start 2024-04-20T11:34:16.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.038+0000 7f6024c00640 1 -- start start 2024-04-20T11:34:16.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.039+0000 7f6024c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f60200726f0 0x7f602015e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:16.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.039+0000 7f6024c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f60200cd0d0 0x7f60201440a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:16.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.039+0000 7f6024c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6020145f60 0x7f60200a94a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:16.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.039+0000 7f6024c00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f602006e680 con 0x7f60200cd0d0 2024-04-20T11:34:16.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.039+0000 7f6024c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f602006e3a0 con 0x7f6020145f60 2024-04-20T11:34:16.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.039+0000 7f6024c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f60200cf290 con 0x7f60200726f0 2024-04-20T11:34:16.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.039+0000 7f6015400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f60200cd0d0 0x7f60201440a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:16.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.039+0000 7f6015400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f60200cd0d0 0x7f60201440a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52118/0 (socket says 172.21.15.152:52118) 2024-04-20T11:34:16.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.039+0000 7f6015400640 1 -- 172.21.15.152:0/3596578583 learned_addr learned my addr 172.21.15.152:0/3596578583 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:16.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.040+0000 7f601e800640 1 --2- 172.21.15.152:0/3596578583 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6020145f60 0x7f60200a94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:16.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.040+0000 7f601de00640 1 --2- 172.21.15.152:0/3596578583 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f60200726f0 0x7f602015e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:16.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.040+0000 7f6015400640 1 -- 172.21.15.152:0/3596578583 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f60200726f0 msgr2=0x7f602015e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:16.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.040+0000 7f6015400640 1 --2- 172.21.15.152:0/3596578583 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f60200726f0 0x7f602015e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:16.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.040+0000 7f6015400640 1 -- 172.21.15.152:0/3596578583 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6020145f60 msgr2=0x7f60200a94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:16.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.040+0000 7f6015400640 1 --2- 172.21.15.152:0/3596578583 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6020145f60 0x7f60200a94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:16.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.040+0000 7f6015400640 1 -- 172.21.15.152:0/3596578583 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60200a99e0 con 0x7f60200cd0d0 2024-04-20T11:34:16.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.040+0000 7f6015400640 1 --2- 172.21.15.152:0/3596578583 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f60200cd0d0 0x7f60201440a0 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f600800c4d0 tx=0x7f600800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:16.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.041+0000 7f6016a00640 1 -- 172.21.15.152:0/3596578583 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6008018070 con 0x7f60200cd0d0 2024-04-20T11:34:16.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.041+0000 7f6016a00640 1 -- 172.21.15.152:0/3596578583 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6008007500 con 0x7f60200cd0d0 2024-04-20T11:34:16.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.041+0000 7f6016a00640 1 -- 172.21.15.152:0/3596578583 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6008004a20 con 0x7f60200cd0d0 2024-04-20T11:34:16.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.041+0000 7f6024c00640 1 -- 172.21.15.152:0/3596578583 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60200a9c70 con 0x7f60200cd0d0 2024-04-20T11:34:16.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.041+0000 7f6024c00640 1 -- 172.21.15.152:0/3596578583 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60200aa230 con 0x7f60200cd0d0 2024-04-20T11:34:16.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.042+0000 7f6024c00640 1 -- 172.21.15.152:0/3596578583 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60200d0ef0 con 0x7f60200cd0d0 2024-04-20T11:34:16.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.043+0000 7f6016a00640 1 -- 172.21.15.152:0/3596578583 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 19) ==== 91397+0+0 (secure 0 0 0) 0x7f6008004bc0 con 0x7f60200cd0d0 2024-04-20T11:34:16.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.044+0000 7f6016a00640 1 --2- 172.21.15.152:0/3596578583 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f5ffc06cce0 0x7f5ffc06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:16.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.044+0000 7f6016a00640 1 -- 172.21.15.152:0/3596578583 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5781+0+0 (secure 0 0 0) 0x7f600808b560 con 0x7f60200cd0d0 2024-04-20T11:34:16.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.044+0000 7f601de00640 1 --2- 172.21.15.152:0/3596578583 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f5ffc06cce0 0x7f5ffc06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:16.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.046+0000 7f601de00640 1 --2- 172.21.15.152:0/3596578583 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f5ffc06cce0 0x7f5ffc06f1a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f601000a950 tx=0x7f6010008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:16.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.048+0000 7f6016a00640 1 -- 172.21.15.152:0/3596578583 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f60200d0ef0 con 0x7f60200cd0d0 2024-04-20T11:34:16.255 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.256+0000 7f6024c00640 1 -- 172.21.15.152:0/3596578583 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f60200689f0 con 0x7f60200cd0d0 2024-04-20T11:34:16.257 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.258+0000 7f6016a00640 1 -- 172.21.15.152:0/3596578583 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v105) ==== 74+0+16821 (secure 0 0 0) 0x7f6008089e20 con 0x7f60200cd0d0 2024-04-20T11:34:16.258 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:34:16.258 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":105,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:34:13.116969+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:33:25.062365+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-20T11:33:26.260945+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":105,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[{"pool":17,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:34:16.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.261+0000 7f6024c00640 1 -- 172.21.15.152:0/3596578583 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f5ffc06cce0 msgr2=0x7f5ffc06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:16.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.261+0000 7f6024c00640 1 --2- 172.21.15.152:0/3596578583 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f5ffc06cce0 0x7f5ffc06f1a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f601000a950 tx=0x7f6010008040 comp rx=0 tx=0).stop 2024-04-20T11:34:16.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.261+0000 7f6024c00640 1 -- 172.21.15.152:0/3596578583 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f60200cd0d0 msgr2=0x7f60201440a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:16.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.261+0000 7f6024c00640 1 --2- 172.21.15.152:0/3596578583 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f60200cd0d0 0x7f60201440a0 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f600800c4d0 tx=0x7f600800c990 comp rx=0 tx=0).stop 2024-04-20T11:34:16.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.261+0000 7f6024c00640 1 -- 172.21.15.152:0/3596578583 shutdown_connections 2024-04-20T11:34:16.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.261+0000 7f6024c00640 1 --2- 172.21.15.152:0/3596578583 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f5ffc06cce0 0x7f5ffc06f1a0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:16.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.261+0000 7f6024c00640 1 --2- 172.21.15.152:0/3596578583 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6020145f60 0x7f60200a94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:16.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.261+0000 7f6024c00640 1 --2- 172.21.15.152:0/3596578583 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f60200cd0d0 0x7f60201440a0 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:16.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.261+0000 7f6024c00640 1 --2- 172.21.15.152:0/3596578583 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f60200726f0 0x7f602015e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:16.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.261+0000 7f6024c00640 1 -- 172.21.15.152:0/3596578583 >> 172.21.15.152:0/3596578583 conn(0x7f6020067cf0 msgr2=0x7f60200b3240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:16.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.261+0000 7f6024c00640 1 -- 172.21.15.152:0/3596578583 shutdown_connections 2024-04-20T11:34:16.262 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.261+0000 7f6024c00640 1 -- 172.21.15.152:0/3596578583 wait complete. 2024-04-20T11:34:16.279 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T11:26:47.246571+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T11:33:25.062365+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '102', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T11:33:26.260945+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '105', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 5, 'snap_epoch': 105, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T11:34:16.280 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T11:34:16.485 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.485+0000 7f1fa9800640 1 Processor -- start 2024-04-20T11:34:16.485 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.486+0000 7f1fa9800640 1 -- start start 2024-04-20T11:34:16.487 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.487+0000 7f1fa9800640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f1fa40d46a0 0x7f1fa40d4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:16.487 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.487+0000 7f1fa9800640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1fa40d19a0 con 0x7f1fa40d0d90 2024-04-20T11:34:16.487 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.487+0000 7f1fa9800640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1fa40d1c80 con 0x7f1fa415e9e0 2024-04-20T11:34:16.487 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.487+0000 7f1fa9800640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f1fa406da30 con 0x7f1fa40d46a0 2024-04-20T11:34:16.487 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.488+0000 7f1fa3400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f1fa40d0d90 0x7f1fa40cfda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:40196/0 (socket says 172.21.15.152:40196) 2024-04-20T11:34:16.487 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.488+0000 7f1fa3400640 1 -- 172.21.15.152:0/3877060240 learned_addr learned my addr 172.21.15.152:0/3877060240 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:34:16.489 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.488+0000 7f1fa1600640 1 -- 172.21.15.152:0/3877060240 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 412762784 0 0) 0x7f1fa40d19a0 con 0x7f1fa40d0d90 2024-04-20T11:34:16.489 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.488+0000 7f1fa2000640 1 --2- 172.21.15.152:0/3877060240 >> v2:172.21.15.175:3300/0 conn(0x7f1fa40d46a0 0x7f1fa40d4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:16.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.488+0000 7f1fa1600640 1 -- 172.21.15.152:0/3877060240 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1f84003520 con 0x7f1fa40d0d90 2024-04-20T11:34:16.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.488+0000 7f1fa1600640 1 -- 172.21.15.152:0/3877060240 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2995656625 0 0) 0x7f1f84003520 con 0x7f1fa40d0d90 2024-04-20T11:34:16.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.488+0000 7f1fa1600640 1 -- 172.21.15.152:0/3877060240 >> v2:172.21.15.175:3300/0 conn(0x7f1fa40d46a0 msgr2=0x7f1fa40d4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:16.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.488+0000 7f1fa1600640 1 --2- 172.21.15.152:0/3877060240 >> v2:172.21.15.175:3300/0 conn(0x7f1fa40d46a0 0x7f1fa40d4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:16.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.488+0000 7f1fa1600640 1 -- 172.21.15.152:0/3877060240 >> v1:172.21.15.175:6789/0 conn(0x7f1fa415e9e0 legacy=0x7f1fa406d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:16.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.489+0000 7f1fa1600640 1 -- 172.21.15.152:0/3877060240 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fa40d1f60 con 0x7f1fa40d0d90 2024-04-20T11:34:16.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.489+0000 7f1fa1600640 1 -- 172.21.15.152:0/3877060240 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f1f9c001670 con 0x7f1fa40d0d90 2024-04-20T11:34:16.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.489+0000 7f1fa1600640 1 -- 172.21.15.152:0/3877060240 >> v1:172.21.15.152:6789/0 conn(0x7f1fa40d0d90 legacy=0x7f1fa40cfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:16.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.489+0000 7f1fa1600640 1 --2- 172.21.15.152:0/3877060240 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f840039b0 0x7f1f84003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:16.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.489+0000 7f1fa1600640 1 --2- 172.21.15.152:0/3877060240 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1f84004800 0x7f1f84004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:16.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.489+0000 7f1fa1600640 1 --2- 172.21.15.152:0/3877060240 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1f840051c0 0x7f1f8400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:16.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.489+0000 7f1fa1600640 1 -- 172.21.15.152:0/3877060240 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f1fa406da30 con 0x7f1f84004800 2024-04-20T11:34:16.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.489+0000 7f1fa1600640 1 -- 172.21.15.152:0/3877060240 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1f8400c110 con 0x7f1f840039b0 2024-04-20T11:34:16.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.489+0000 7f1fa1600640 1 -- 172.21.15.152:0/3877060240 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1f8400c310 con 0x7f1f840051c0 2024-04-20T11:34:16.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.489+0000 7f1fa2000640 1 --2- 172.21.15.152:0/3877060240 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1f84004800 0x7f1f84004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:16.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.489+0000 7f1fa2a00640 1 --2- 172.21.15.152:0/3877060240 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f840039b0 0x7f1f84003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:16.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.489+0000 7f1fa2000640 1 -- 172.21.15.152:0/3877060240 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1f840051c0 msgr2=0x7f1f8400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:16.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.489+0000 7f1fa2000640 1 --2- 172.21.15.152:0/3877060240 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1f840051c0 0x7f1f8400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:16.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.489+0000 7f1fa2000640 1 -- 172.21.15.152:0/3877060240 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f840039b0 msgr2=0x7f1f84003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:16.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.489+0000 7f1fa2000640 1 --2- 172.21.15.152:0/3877060240 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f840039b0 0x7f1f84003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:16.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.489+0000 7f1fa2000640 1 -- 172.21.15.152:0/3877060240 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f84003520 con 0x7f1f84004800 2024-04-20T11:34:16.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.489+0000 7f1fa2a00640 1 --2- 172.21.15.152:0/3877060240 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f840039b0 0x7f1f84003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:34:16.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.489+0000 7f1fa3400640 1 -- 172.21.15.152:0/3877060240 reap_dead start 2024-04-20T11:34:16.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.490+0000 7f1fa2000640 1 --2- 172.21.15.152:0/3877060240 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1f84004800 0x7f1f84004c80 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c007360 tx=0x7f1f84005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6679251e1d04c607 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:16.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.491+0000 7f1fa1600640 1 -- 172.21.15.152:0/3877060240 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f8c039070 con 0x7f1f84004800 2024-04-20T11:34:16.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.491+0000 7f1fa1600640 1 -- 172.21.15.152:0/3877060240 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f8c02ccd0 con 0x7f1f84004800 2024-04-20T11:34:16.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.491+0000 7f1fa1600640 1 -- 172.21.15.152:0/3877060240 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f8c009be0 con 0x7f1f84004800 2024-04-20T11:34:16.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.491+0000 7f1fa9800640 1 -- 172.21.15.152:0/3877060240 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1f84004800 msgr2=0x7f1f84004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:16.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.491+0000 7f1fa9800640 1 --2- 172.21.15.152:0/3877060240 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1f84004800 0x7f1f84004c80 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c007360 tx=0x7f1f84005830 comp rx=0 tx=0).stop 2024-04-20T11:34:16.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.491+0000 7f1fa9800640 1 -- 172.21.15.152:0/3877060240 shutdown_connections 2024-04-20T11:34:16.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.491+0000 7f1fa9800640 1 --2- 172.21.15.152:0/3877060240 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1f84004800 0x7f1f84004c80 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:16.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.491+0000 7f1fa9800640 1 -- 172.21.15.152:0/3877060240 >> 172.21.15.152:0/3877060240 conn(0x7f1fa409dab0 msgr2=0x7f1fa408bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:16.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.492+0000 7f1fa9800640 1 -- 172.21.15.152:0/3877060240 shutdown_connections 2024-04-20T11:34:16.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.492+0000 7f1fa9800640 1 -- 172.21.15.152:0/3877060240 wait complete. 2024-04-20T11:34:16.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.493+0000 7f1fa9800640 1 Processor -- start 2024-04-20T11:34:16.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.493+0000 7f1fa9800640 1 -- start start 2024-04-20T11:34:16.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.494+0000 7f1fa9800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f84004800 0x7f1fa415f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:16.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.494+0000 7f1fa9800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1fa41607a0 0x7f1fa4152a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:16.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.494+0000 7f1fa9800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1fa40c8cf0 0x7f1fa40c91a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:16.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.494+0000 7f1fa9800640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f1fa415edc0 con 0x7f1fa41607a0 2024-04-20T11:34:16.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.494+0000 7f1fa9800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1fa40cf880 con 0x7f1f84004800 2024-04-20T11:34:16.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.494+0000 7f1fa9800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1fa415ef40 con 0x7f1fa40c8cf0 2024-04-20T11:34:16.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.495+0000 7f1fa2000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1fa41607a0 0x7f1fa4152a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:16.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.495+0000 7f1fa2000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1fa41607a0 0x7f1fa4152a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52140/0 (socket says 172.21.15.152:52140) 2024-04-20T11:34:16.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.495+0000 7f1fa2000640 1 -- 172.21.15.152:0/3704554677 learned_addr learned my addr 172.21.15.152:0/3704554677 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:16.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.495+0000 7f1fa2a00640 1 --2- 172.21.15.152:0/3704554677 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f84004800 0x7f1fa415f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:16.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.495+0000 7f1fa3400640 1 --2- 172.21.15.152:0/3704554677 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1fa40c8cf0 0x7f1fa40c91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:16.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.495+0000 7f1fa2000640 1 -- 172.21.15.152:0/3704554677 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1fa40c8cf0 msgr2=0x7f1fa40c91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:16.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.495+0000 7f1fa2000640 1 --2- 172.21.15.152:0/3704554677 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1fa40c8cf0 0x7f1fa40c91a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:16.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.495+0000 7f1fa2000640 1 -- 172.21.15.152:0/3704554677 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f84004800 msgr2=0x7f1fa415f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:16.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.495+0000 7f1fa2000640 1 --2- 172.21.15.152:0/3704554677 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f84004800 0x7f1fa415f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:16.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.495+0000 7f1fa2000640 1 -- 172.21.15.152:0/3704554677 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fa415f700 con 0x7f1fa41607a0 2024-04-20T11:34:16.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.495+0000 7f1fa2000640 1 --2- 172.21.15.152:0/3704554677 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1fa41607a0 0x7f1fa4152a20 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c007480 tx=0x7f1f8c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:16.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.496+0000 7f1f9aa00640 1 -- 172.21.15.152:0/3704554677 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f8c044030 con 0x7f1fa41607a0 2024-04-20T11:34:16.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.496+0000 7f1f9aa00640 1 -- 172.21.15.152:0/3704554677 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f8c02cad0 con 0x7f1fa41607a0 2024-04-20T11:34:16.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.496+0000 7f1fa9800640 1 -- 172.21.15.152:0/3704554677 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fa415f9e0 con 0x7f1fa41607a0 2024-04-20T11:34:16.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.496+0000 7f1f9aa00640 1 -- 172.21.15.152:0/3704554677 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f8c039040 con 0x7f1fa41607a0 2024-04-20T11:34:16.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.496+0000 7f1fa9800640 1 -- 172.21.15.152:0/3704554677 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fa415ffa0 con 0x7f1fa41607a0 2024-04-20T11:34:16.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.497+0000 7f1fa9800640 1 -- 172.21.15.152:0/3704554677 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fa40b58d0 con 0x7f1fa41607a0 2024-04-20T11:34:16.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.498+0000 7f1f9aa00640 1 -- 172.21.15.152:0/3704554677 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 19) ==== 91397+0+0 (secure 0 0 0) 0x7f1f8c047070 con 0x7f1fa41607a0 2024-04-20T11:34:16.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.499+0000 7f1f9aa00640 1 --2- 172.21.15.152:0/3704554677 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f1f8006cd10 0x7f1f8006f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:16.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.499+0000 7f1f9aa00640 1 -- 172.21.15.152:0/3704554677 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5781+0+0 (secure 0 0 0) 0x7f1f8c0ae020 con 0x7f1fa41607a0 2024-04-20T11:34:16.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.500+0000 7f1fa2a00640 1 --2- 172.21.15.152:0/3704554677 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f1f8006cd10 0x7f1f8006f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:16.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.501+0000 7f1fa2a00640 1 --2- 172.21.15.152:0/3704554677 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f1f8006cd10 0x7f1f8006f1d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1f94000c00 tx=0x7f1f94009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:16.503 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.503+0000 7f1f9aa00640 1 -- 172.21.15.152:0/3704554677 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f1fa40b58d0 con 0x7f1fa41607a0 2024-04-20T11:34:16.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.702+0000 7f1fa9800640 1 -- 172.21.15.152:0/3704554677 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1fa40689f0 con 0x7f1fa41607a0 2024-04-20T11:34:16.703 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.703+0000 7f1f9aa00640 1 -- 172.21.15.152:0/3704554677 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v105) ==== 93+0+10 (secure 0 0 0) 0x7f1f8c07b940 con 0x7f1fa41607a0 2024-04-20T11:34:16.703 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 1 2024-04-20T11:34:16.705 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.706+0000 7f1fa9800640 1 -- 172.21.15.152:0/3704554677 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f1f8006cd10 msgr2=0x7f1f8006f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:16.706 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.706+0000 7f1fa9800640 1 --2- 172.21.15.152:0/3704554677 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f1f8006cd10 0x7f1f8006f1d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1f94000c00 tx=0x7f1f94009040 comp rx=0 tx=0).stop 2024-04-20T11:34:16.706 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.706+0000 7f1fa9800640 1 -- 172.21.15.152:0/3704554677 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1fa41607a0 msgr2=0x7f1fa4152a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:16.706 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.706+0000 7f1fa9800640 1 --2- 172.21.15.152:0/3704554677 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1fa41607a0 0x7f1fa4152a20 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c007480 tx=0x7f1f8c004ab0 comp rx=0 tx=0).stop 2024-04-20T11:34:16.706 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.706+0000 7f1fa9800640 1 -- 172.21.15.152:0/3704554677 shutdown_connections 2024-04-20T11:34:16.706 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.706+0000 7f1fa9800640 1 --2- 172.21.15.152:0/3704554677 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f1f8006cd10 0x7f1f8006f1d0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:16.706 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.706+0000 7f1fa9800640 1 --2- 172.21.15.152:0/3704554677 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1fa40c8cf0 0x7f1fa40c91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:16.706 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.706+0000 7f1fa9800640 1 --2- 172.21.15.152:0/3704554677 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1fa41607a0 0x7f1fa4152a20 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:16.706 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.706+0000 7f1fa9800640 1 --2- 172.21.15.152:0/3704554677 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f84004800 0x7f1fa415f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:16.706 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.706+0000 7f1fa9800640 1 -- 172.21.15.152:0/3704554677 >> 172.21.15.152:0/3704554677 conn(0x7f1fa409dab0 msgr2=0x7f1fa406dbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:16.706 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.706+0000 7f1fa9800640 1 -- 172.21.15.152:0/3704554677 shutdown_connections 2024-04-20T11:34:16.706 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.706+0000 7f1fa9800640 1 -- 172.21.15.152:0/3704554677 wait complete. 2024-04-20T11:34:16.724 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T11:34:16.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.919+0000 7f00cfc00640 1 Processor -- start 2024-04-20T11:34:16.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.919+0000 7f00cfc00640 1 -- start start 2024-04-20T11:34:16.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.921+0000 7f00cfc00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f00c815e5d0 0x7f00c8162440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:16.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.921+0000 7f00cfc00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f00c80afba0 0x7f00c80ae110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:16.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.921+0000 7f00cfc00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00c80adc30 con 0x7f00c80b3fa0 2024-04-20T11:34:16.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.921+0000 7f00cfc00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f00c8162a70 con 0x7f00c815e5d0 2024-04-20T11:34:16.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.921+0000 7f00cfc00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f00c8162d50 con 0x7f00c80afba0 2024-04-20T11:34:16.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.921+0000 7f00c7e00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f00c80b3fa0 0x7f00c80af490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:40212/0 (socket says 172.21.15.152:40212) 2024-04-20T11:34:16.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.921+0000 7f00c7e00640 1 -- 172.21.15.152:0/1857738860 learned_addr learned my addr 172.21.15.152:0/1857738860 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:34:16.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.921+0000 7f00cdc00640 1 --2- 172.21.15.152:0/1857738860 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f00c80afba0 0x7f00c80ae110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:16.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.921+0000 7f00cd200640 1 --2- 172.21.15.152:0/1857738860 >> v2:172.21.15.175:3300/0 conn(0x7f00c815e5d0 0x7f00c8162440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:16.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.922+0000 7f00c7400640 1 -- 172.21.15.152:0/1857738860 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1082150058 0 0) 0x7f00c80adc30 con 0x7f00c80b3fa0 2024-04-20T11:34:16.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.922+0000 7f00c7400640 1 -- 172.21.15.152:0/1857738860 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f00b0003520 con 0x7f00c80b3fa0 2024-04-20T11:34:16.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.922+0000 7f00c7400640 1 -- 172.21.15.152:0/1857738860 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4227016015 0 0) 0x7f00b0003520 con 0x7f00c80b3fa0 2024-04-20T11:34:16.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.922+0000 7f00c7400640 1 -- 172.21.15.152:0/1857738860 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f00c80afba0 msgr2=0x7f00c80ae110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:16.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.922+0000 7f00c7400640 1 --2- 172.21.15.152:0/1857738860 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f00c80afba0 0x7f00c80ae110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:16.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.922+0000 7f00c7400640 1 -- 172.21.15.152:0/1857738860 >> v2:172.21.15.175:3300/0 conn(0x7f00c815e5d0 msgr2=0x7f00c8162440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:16.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.922+0000 7f00c7400640 1 --2- 172.21.15.152:0/1857738860 >> v2:172.21.15.175:3300/0 conn(0x7f00c815e5d0 0x7f00c8162440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:16.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.922+0000 7f00c7400640 1 -- 172.21.15.152:0/1857738860 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00c8163030 con 0x7f00c80b3fa0 2024-04-20T11:34:16.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.922+0000 7f00c7400640 1 -- 172.21.15.152:0/1857738860 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f00b8001670 con 0x7f00c80b3fa0 2024-04-20T11:34:16.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.922+0000 7f00c7400640 1 -- 172.21.15.152:0/1857738860 >> v1:172.21.15.152:6789/0 conn(0x7f00c80b3fa0 legacy=0x7f00c80af490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:16.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.922+0000 7f00c7400640 1 --2- 172.21.15.152:0/1857738860 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f00b00039b0 0x7f00b0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:16.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.922+0000 7f00c7400640 1 --2- 172.21.15.152:0/1857738860 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00b0004800 0x7f00b0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:16.925 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.922+0000 7f00c7400640 1 --2- 172.21.15.152:0/1857738860 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f00c80afba0 0x7f00b00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:16.925 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.922+0000 7f00c7400640 1 -- 172.21.15.152:0/1857738860 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f00c8162a70 con 0x7f00b00039b0 2024-04-20T11:34:16.925 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.922+0000 7f00c7400640 1 -- 172.21.15.152:0/1857738860 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f00c8162d50 con 0x7f00b0004800 2024-04-20T11:34:16.925 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.922+0000 7f00c7400640 1 -- 172.21.15.152:0/1857738860 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f00b000b9c0 con 0x7f00c80afba0 2024-04-20T11:34:16.925 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.923+0000 7f00cd200640 1 --2- 172.21.15.152:0/1857738860 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f00b00039b0 0x7f00b0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:16.925 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.923+0000 7f00c7e00640 1 --2- 172.21.15.152:0/1857738860 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00b0004800 0x7f00b0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:16.925 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.923+0000 7f00cd200640 1 -- 172.21.15.152:0/1857738860 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f00c80afba0 msgr2=0x7f00b00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:16.925 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.923+0000 7f00cdc00640 1 --2- 172.21.15.152:0/1857738860 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f00c80afba0 0x7f00b00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:16.925 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.923+0000 7f00cd200640 1 --2- 172.21.15.152:0/1857738860 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f00c80afba0 0x7f00b00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:16.925 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.923+0000 7f00cd200640 1 -- 172.21.15.152:0/1857738860 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00b0004800 msgr2=0x7f00b0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:16.925 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.923+0000 7f00cd200640 1 --2- 172.21.15.152:0/1857738860 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00b0004800 0x7f00b0004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:16.925 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.923+0000 7f00cd200640 1 -- 172.21.15.152:0/1857738860 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00b0003520 con 0x7f00b00039b0 2024-04-20T11:34:16.925 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.923+0000 7f00cdc00640 1 --2- 172.21.15.152:0/1857738860 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f00c80afba0 0x7f00b00051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:34:16.925 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.924+0000 7f00cd200640 1 --2- 172.21.15.152:0/1857738860 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f00b00039b0 0x7f00b0003dd0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f00bc009ac0 tx=0x7f00b0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0b168107b8b9ac4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:16.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.924+0000 7f00c7400640 1 -- 172.21.15.152:0/1857738860 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00bc039070 con 0x7f00b00039b0 2024-04-20T11:34:16.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.924+0000 7f00c7400640 1 -- 172.21.15.152:0/1857738860 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00bc02c710 con 0x7f00b00039b0 2024-04-20T11:34:16.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.924+0000 7f00c7400640 1 -- 172.21.15.152:0/1857738860 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00bc009690 con 0x7f00b00039b0 2024-04-20T11:34:16.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.925+0000 7f00cfc00640 1 -- 172.21.15.152:0/1857738860 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f00b00039b0 msgr2=0x7f00b0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:16.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.925+0000 7f00cfc00640 1 --2- 172.21.15.152:0/1857738860 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f00b00039b0 0x7f00b0003dd0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f00bc009ac0 tx=0x7f00b0005830 comp rx=0 tx=0).stop 2024-04-20T11:34:16.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.925+0000 7f00cdc00640 1 -- 172.21.15.152:0/1857738860 reap_dead start 2024-04-20T11:34:16.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.925+0000 7f00cfc00640 1 -- 172.21.15.152:0/1857738860 shutdown_connections 2024-04-20T11:34:16.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.925+0000 7f00cfc00640 1 -- 172.21.15.152:0/1857738860 >> 172.21.15.152:0/1857738860 conn(0x7f00c806b980 msgr2=0x7f00c806bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:16.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.925+0000 7f00cfc00640 1 -- 172.21.15.152:0/1857738860 shutdown_connections 2024-04-20T11:34:16.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.925+0000 7f00cfc00640 1 -- 172.21.15.152:0/1857738860 wait complete. 2024-04-20T11:34:16.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.927+0000 7f00cfc00640 1 Processor -- start 2024-04-20T11:34:16.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.927+0000 7f00cfc00640 1 -- start start 2024-04-20T11:34:16.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.929+0000 7f00cfc00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f00b00039b0 0x7f00c8144ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:16.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.929+0000 7f00cfc00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00c8071db0 0x7f00c8072230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:16.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.929+0000 7f00cfc00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f00c815bdd0 0x7f00c815c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:16.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.929+0000 7f00cfc00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f00c806e0a0 con 0x7f00b00039b0 2024-04-20T11:34:16.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.929+0000 7f00cfc00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f00c8163310 con 0x7f00c8071db0 2024-04-20T11:34:16.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.929+0000 7f00cfc00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f00c806e220 con 0x7f00c815bdd0 2024-04-20T11:34:16.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.929+0000 7f00cd200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f00b00039b0 0x7f00c8144ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:16.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.929+0000 7f00cd200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f00b00039b0 0x7f00c8144ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52162/0 (socket says 172.21.15.152:52162) 2024-04-20T11:34:16.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.929+0000 7f00cd200640 1 -- 172.21.15.152:0/1200719607 learned_addr learned my addr 172.21.15.152:0/1200719607 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:16.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.929+0000 7f00cd200640 1 -- 172.21.15.152:0/1200719607 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f00c815bdd0 msgr2=0x7f00c815c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:16.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.929+0000 7f00cd200640 1 --2- 172.21.15.152:0/1200719607 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f00c815bdd0 0x7f00c815c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:16.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.929+0000 7f00cd200640 1 -- 172.21.15.152:0/1200719607 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00c8071db0 msgr2=0x7f00c8072230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:16.931 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.929+0000 7f00c7e00640 1 --2- 172.21.15.152:0/1200719607 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00c8071db0 0x7f00c8072230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:16.931 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.929+0000 7f00cd200640 1 --2- 172.21.15.152:0/1200719607 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00c8071db0 0x7f00c8072230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:16.931 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.929+0000 7f00cd200640 1 -- 172.21.15.152:0/1200719607 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00c814f3d0 con 0x7f00b00039b0 2024-04-20T11:34:16.931 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.930+0000 7f00cd200640 1 --2- 172.21.15.152:0/1200719607 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f00b00039b0 0x7f00c8144ff0 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f00bc009be0 tx=0x7f00bc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:16.931 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.930+0000 7f00c4c00640 1 -- 172.21.15.152:0/1200719607 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00bc042020 con 0x7f00b00039b0 2024-04-20T11:34:16.931 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.930+0000 7f00c4c00640 1 -- 172.21.15.152:0/1200719607 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00bc03a040 con 0x7f00b00039b0 2024-04-20T11:34:16.931 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.930+0000 7f00c4c00640 1 -- 172.21.15.152:0/1200719607 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00bc039070 con 0x7f00b00039b0 2024-04-20T11:34:16.931 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.930+0000 7f00cfc00640 1 -- 172.21.15.152:0/1200719607 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00c8145530 con 0x7f00b00039b0 2024-04-20T11:34:16.931 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.930+0000 7f00cfc00640 1 -- 172.21.15.152:0/1200719607 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00c80ccff0 con 0x7f00b00039b0 2024-04-20T11:34:16.931 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.932+0000 7f00cfc00640 1 -- 172.21.15.152:0/1200719607 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00c80689f0 con 0x7f00b00039b0 2024-04-20T11:34:16.932 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.932+0000 7f00c4c00640 1 -- 172.21.15.152:0/1200719607 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 19) ==== 91397+0+0 (secure 0 0 0) 0x7f00bc045020 con 0x7f00b00039b0 2024-04-20T11:34:16.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.933+0000 7f00c4c00640 1 --2- 172.21.15.152:0/1200719607 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f00ac06cce0 0x7f00ac06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:16.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.934+0000 7f00c4c00640 1 -- 172.21.15.152:0/1200719607 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5781+0+0 (secure 0 0 0) 0x7f00bc0ad980 con 0x7f00b00039b0 2024-04-20T11:34:16.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.934+0000 7f00c7e00640 1 --2- 172.21.15.152:0/1200719607 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f00ac06cce0 0x7f00ac06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:16.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.935+0000 7f00c7e00640 1 --2- 172.21.15.152:0/1200719607 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f00ac06cce0 0x7f00ac06f1a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f00b8001eb0 tx=0x7f00b8009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:16.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:16.938+0000 7f00c4c00640 1 -- 172.21.15.152:0/1200719607 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f00bc04c030 con 0x7f00b00039b0 2024-04-20T11:34:17.139 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.139+0000 7f00cfc00640 1 -- 172.21.15.152:0/1200719607 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f00c80af1f0 con 0x7f00b00039b0 2024-04-20T11:34:17.140 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.140+0000 7f00c4c00640 1 -- 172.21.15.152:0/1200719607 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v105) ==== 104+0+11 (secure 0 0 0) 0x7f00c8145530 con 0x7f00b00039b0 2024-04-20T11:34:17.140 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 32 2024-04-20T11:34:17.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.143+0000 7f00cfc00640 1 -- 172.21.15.152:0/1200719607 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f00ac06cce0 msgr2=0x7f00ac06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:17.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.143+0000 7f00cfc00640 1 --2- 172.21.15.152:0/1200719607 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f00ac06cce0 0x7f00ac06f1a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f00b8001eb0 tx=0x7f00b8009040 comp rx=0 tx=0).stop 2024-04-20T11:34:17.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.143+0000 7f00cfc00640 1 -- 172.21.15.152:0/1200719607 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f00b00039b0 msgr2=0x7f00c8144ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:17.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.143+0000 7f00cfc00640 1 --2- 172.21.15.152:0/1200719607 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f00b00039b0 0x7f00c8144ff0 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f00bc009be0 tx=0x7f00bc004730 comp rx=0 tx=0).stop 2024-04-20T11:34:17.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.143+0000 7f00cfc00640 1 -- 172.21.15.152:0/1200719607 shutdown_connections 2024-04-20T11:34:17.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.143+0000 7f00cfc00640 1 --2- 172.21.15.152:0/1200719607 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f00ac06cce0 0x7f00ac06f1a0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:17.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.143+0000 7f00cfc00640 1 --2- 172.21.15.152:0/1200719607 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f00c815bdd0 0x7f00c815c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:17.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.143+0000 7f00cfc00640 1 --2- 172.21.15.152:0/1200719607 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00c8071db0 0x7f00c8072230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:17.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.143+0000 7f00cfc00640 1 --2- 172.21.15.152:0/1200719607 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f00b00039b0 0x7f00c8144ff0 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:17.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.143+0000 7f00cfc00640 1 -- 172.21.15.152:0/1200719607 >> 172.21.15.152:0/1200719607 conn(0x7f00c806b980 msgr2=0x7f00c80ae360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:17.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.144+0000 7f00cfc00640 1 -- 172.21.15.152:0/1200719607 shutdown_connections 2024-04-20T11:34:17.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.144+0000 7f00cfc00640 1 -- 172.21.15.152:0/1200719607 wait complete. 2024-04-20T11:34:17.162 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T11:34:17.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.364+0000 7f84ce200640 1 Processor -- start 2024-04-20T11:34:17.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.364+0000 7f84ce200640 1 -- start start 2024-04-20T11:34:17.366 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.366+0000 7f84ce200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f84c815e5d0 0x7f84c8162440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:17.366 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.366+0000 7f84ce200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f84c80b3fa0 0x7f84c80af490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:17.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.366+0000 7f84ce200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f84c815e9b0 0x7f84c80af9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:17.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.366+0000 7f84ce200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f84c80adc30 con 0x7f84c815e9b0 2024-04-20T11:34:17.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.366+0000 7f84ce200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f84c8162a70 con 0x7f84c80b3fa0 2024-04-20T11:34:17.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.366+0000 7f84ce200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f84c8162d50 con 0x7f84c815e5d0 2024-04-20T11:34:17.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.366+0000 7f84c7e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f84c815e9b0 0x7f84c80af9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:17.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.366+0000 7f84c7e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f84c815e9b0 0x7f84c80af9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52176/0 (socket says 172.21.15.152:52176) 2024-04-20T11:34:17.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.366+0000 7f84c7e00640 1 -- 172.21.15.152:0/2561278860 learned_addr learned my addr 172.21.15.152:0/2561278860 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:17.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.366+0000 7f84c7400640 1 --2- 172.21.15.152:0/2561278860 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f84c815e5d0 0x7f84c8162440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:17.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.367+0000 7f84c6a00640 1 --2- 172.21.15.152:0/2561278860 >> v2:172.21.15.175:3300/0 conn(0x7f84c80b3fa0 0x7f84c80af490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:17.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.367+0000 7f84c7e00640 1 -- 172.21.15.152:0/2561278860 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f84c815e5d0 msgr2=0x7f84c8162440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:17.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.367+0000 7f84c7e00640 1 --2- 172.21.15.152:0/2561278860 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f84c815e5d0 0x7f84c8162440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:17.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.367+0000 7f84c7e00640 1 -- 172.21.15.152:0/2561278860 >> v2:172.21.15.175:3300/0 conn(0x7f84c80b3fa0 msgr2=0x7f84c80af490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:17.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.367+0000 7f84c7e00640 1 --2- 172.21.15.152:0/2561278860 >> v2:172.21.15.175:3300/0 conn(0x7f84c80b3fa0 0x7f84c80af490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:17.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.367+0000 7f84c7e00640 1 -- 172.21.15.152:0/2561278860 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84c8163030 con 0x7f84c815e9b0 2024-04-20T11:34:17.369 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.368+0000 7f84c7e00640 1 --2- 172.21.15.152:0/2561278860 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f84c815e9b0 0x7f84c80af9d0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f84c806ffb0 tx=0x7f84c002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fde0414da441a119 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:17.369 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.368+0000 7f84c6000640 1 -- 172.21.15.152:0/2561278860 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84c002ca70 con 0x7f84c815e9b0 2024-04-20T11:34:17.369 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.368+0000 7f84c6000640 1 -- 172.21.15.152:0/2561278860 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84c002cc10 con 0x7f84c815e9b0 2024-04-20T11:34:17.369 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.368+0000 7f84c6000640 1 -- 172.21.15.152:0/2561278860 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84c00343f0 con 0x7f84c815e9b0 2024-04-20T11:34:17.369 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.369+0000 7f84ce200640 1 -- 172.21.15.152:0/2561278860 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f84c815e9b0 msgr2=0x7f84c80af9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:17.369 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.369+0000 7f84ce200640 1 --2- 172.21.15.152:0/2561278860 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f84c815e9b0 0x7f84c80af9d0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f84c806ffb0 tx=0x7f84c002bed0 comp rx=0 tx=0).stop 2024-04-20T11:34:17.369 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.369+0000 7f84ce200640 1 -- 172.21.15.152:0/2561278860 shutdown_connections 2024-04-20T11:34:17.369 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.369+0000 7f84ce200640 1 --2- 172.21.15.152:0/2561278860 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f84c815e9b0 0x7f84c80af9d0 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:17.369 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.369+0000 7f84ce200640 1 --2- 172.21.15.152:0/2561278860 >> v2:172.21.15.175:3300/0 conn(0x7f84c80b3fa0 0x7f84c80af490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:17.369 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.369+0000 7f84ce200640 1 --2- 172.21.15.152:0/2561278860 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f84c815e5d0 0x7f84c8162440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:17.370 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.369+0000 7f84ce200640 1 -- 172.21.15.152:0/2561278860 >> 172.21.15.152:0/2561278860 conn(0x7f84c806b980 msgr2=0x7f84c806bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:17.370 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.369+0000 7f84ce200640 1 -- 172.21.15.152:0/2561278860 shutdown_connections 2024-04-20T11:34:17.370 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.369+0000 7f84ce200640 1 -- 172.21.15.152:0/2561278860 wait complete. 2024-04-20T11:34:17.371 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.371+0000 7f84ce200640 1 Processor -- start 2024-04-20T11:34:17.371 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.372+0000 7f84ce200640 1 -- start start 2024-04-20T11:34:17.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.373+0000 7f84ce200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f84c80b3fa0 0x7f84c814e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:17.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.373+0000 7f84ce200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f84c815e5d0 0x7f84c8152780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:17.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.373+0000 7f84ce200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f84c815e9b0 0x7f84c80cd340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:17.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.373+0000 7f84ce200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f84c806e0a0 con 0x7f84c815e5d0 2024-04-20T11:34:17.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.373+0000 7f84ce200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f84c8163310 con 0x7f84c815e9b0 2024-04-20T11:34:17.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.373+0000 7f84ce200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f84c806e220 con 0x7f84c80b3fa0 2024-04-20T11:34:17.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.373+0000 7f84c6a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f84c815e5d0 0x7f84c8152780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:17.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.373+0000 7f84c6a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f84c815e5d0 0x7f84c8152780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52180/0 (socket says 172.21.15.152:52180) 2024-04-20T11:34:17.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.373+0000 7f84c6a00640 1 -- 172.21.15.152:0/3660834378 learned_addr learned my addr 172.21.15.152:0/3660834378 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:17.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.373+0000 7f84c7400640 1 --2- 172.21.15.152:0/3660834378 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f84c80b3fa0 0x7f84c814e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:17.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.373+0000 7f84c7e00640 1 --2- 172.21.15.152:0/3660834378 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f84c815e9b0 0x7f84c80cd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:17.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.374+0000 7f84c6a00640 1 -- 172.21.15.152:0/3660834378 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f84c80b3fa0 msgr2=0x7f84c814e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:17.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.374+0000 7f84c6a00640 1 --2- 172.21.15.152:0/3660834378 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f84c80b3fa0 0x7f84c814e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:17.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.374+0000 7f84c6a00640 1 -- 172.21.15.152:0/3660834378 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f84c815e9b0 msgr2=0x7f84c80cd340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:17.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.374+0000 7f84c6a00640 1 --2- 172.21.15.152:0/3660834378 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f84c815e9b0 0x7f84c80cd340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:17.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.374+0000 7f84c6a00640 1 -- 172.21.15.152:0/3660834378 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84c81516e0 con 0x7f84c815e5d0 2024-04-20T11:34:17.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.374+0000 7f84c6a00640 1 --2- 172.21.15.152:0/3660834378 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f84c815e5d0 0x7f84c8152780 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f84b400d830 tx=0x7f84b400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:17.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.374+0000 7f84bf400640 1 -- 172.21.15.152:0/3660834378 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b400b6c0 con 0x7f84c815e5d0 2024-04-20T11:34:17.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.374+0000 7f84bf400640 1 -- 172.21.15.152:0/3660834378 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84b4019070 con 0x7f84c815e5d0 2024-04-20T11:34:17.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.374+0000 7f84bf400640 1 -- 172.21.15.152:0/3660834378 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b400ba80 con 0x7f84c815e5d0 2024-04-20T11:34:17.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.374+0000 7f84ce200640 1 -- 172.21.15.152:0/3660834378 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84c81519c0 con 0x7f84c815e5d0 2024-04-20T11:34:17.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.374+0000 7f84ce200640 1 -- 172.21.15.152:0/3660834378 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84c8151f80 con 0x7f84c815e5d0 2024-04-20T11:34:17.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.376+0000 7f84ce200640 1 -- 172.21.15.152:0/3660834378 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84c80afd00 con 0x7f84c815e5d0 2024-04-20T11:34:17.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.377+0000 7f84bf400640 1 -- 172.21.15.152:0/3660834378 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 19) ==== 91397+0+0 (secure 0 0 0) 0x7f84b401e460 con 0x7f84c815e5d0 2024-04-20T11:34:17.377 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.378+0000 7f84bf400640 1 --2- 172.21.15.152:0/3660834378 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f84a806cce0 0x7f84a806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:17.377 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.378+0000 7f84bf400640 1 -- 172.21.15.152:0/3660834378 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5781+0+0 (secure 0 0 0) 0x7f84b408c9d0 con 0x7f84c815e5d0 2024-04-20T11:34:17.378 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.378+0000 7f84c7400640 1 --2- 172.21.15.152:0/3660834378 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f84a806cce0 0x7f84a806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:17.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.380+0000 7f84c7400640 1 --2- 172.21.15.152:0/3660834378 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f84a806cce0 0x7f84a806f1a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f84b8006fd0 tx=0x7f84b8008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:17.383 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.383+0000 7f84bf400640 1 -- 172.21.15.152:0/3660834378 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f84c80afd00 con 0x7f84c815e5d0 2024-04-20T11:34:17.583 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.583+0000 7f84ce200640 1 -- 172.21.15.152:0/3660834378 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f84c806c7e0 con 0x7f84c815e5d0 2024-04-20T11:34:17.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.584+0000 7f84bf400640 1 -- 172.21.15.152:0/3660834378 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v105) ==== 100+0+11 (secure 0 0 0) 0x7f84b405a7d0 con 0x7f84c815e5d0 2024-04-20T11:34:17.584 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 64 2024-04-20T11:34:17.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.587+0000 7f84ce200640 1 -- 172.21.15.152:0/3660834378 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f84a806cce0 msgr2=0x7f84a806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:17.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.587+0000 7f84ce200640 1 --2- 172.21.15.152:0/3660834378 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f84a806cce0 0x7f84a806f1a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f84b8006fd0 tx=0x7f84b8008040 comp rx=0 tx=0).stop 2024-04-20T11:34:17.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.587+0000 7f84ce200640 1 -- 172.21.15.152:0/3660834378 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f84c815e5d0 msgr2=0x7f84c8152780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:17.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.587+0000 7f84ce200640 1 --2- 172.21.15.152:0/3660834378 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f84c815e5d0 0x7f84c8152780 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f84b400d830 tx=0x7f84b400dcf0 comp rx=0 tx=0).stop 2024-04-20T11:34:17.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.587+0000 7f84ce200640 1 -- 172.21.15.152:0/3660834378 shutdown_connections 2024-04-20T11:34:17.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.587+0000 7f84ce200640 1 --2- 172.21.15.152:0/3660834378 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f84a806cce0 0x7f84a806f1a0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:17.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.587+0000 7f84ce200640 1 --2- 172.21.15.152:0/3660834378 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f84c815e9b0 0x7f84c80cd340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:17.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.587+0000 7f84ce200640 1 --2- 172.21.15.152:0/3660834378 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f84c815e5d0 0x7f84c8152780 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:17.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.587+0000 7f84ce200640 1 --2- 172.21.15.152:0/3660834378 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f84c80b3fa0 0x7f84c814e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:17.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.587+0000 7f84ce200640 1 -- 172.21.15.152:0/3660834378 >> 172.21.15.152:0/3660834378 conn(0x7f84c806b980 msgr2=0x7f84c80ae010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:17.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.588+0000 7f84ce200640 1 -- 172.21.15.152:0/3660834378 shutdown_connections 2024-04-20T11:34:17.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.588+0000 7f84ce200640 1 -- 172.21.15.152:0/3660834378 wait complete. 2024-04-20T11:34:17.606 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-20T11:34:17.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.808+0000 7f0e45c00640 1 Processor -- start 2024-04-20T11:34:17.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.809+0000 7f0e45c00640 1 -- start start 2024-04-20T11:34:17.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.810+0000 7f0e45c00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f0e4006fa50 0x7f0e400b3470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:17.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.810+0000 7f0e45c00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e400b4130 con 0x7f0e4015fd60 2024-04-20T11:34:17.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.810+0000 7f0e45c00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e4015f280 con 0x7f0e4015b2b0 2024-04-20T11:34:17.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.810+0000 7f0e45c00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f0e4015f8d0 con 0x7f0e4006fa50 2024-04-20T11:34:17.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.811+0000 7f0e3f400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f0e4015fd60 0x7f0e400b4790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:40222/0 (socket says 172.21.15.152:40222) 2024-04-20T11:34:17.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.811+0000 7f0e3f400640 1 -- 172.21.15.152:0/808623675 learned_addr learned my addr 172.21.15.152:0/808623675 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:34:17.812 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.811+0000 7f0e37e00640 1 --2- 172.21.15.152:0/808623675 >> v2:172.21.15.175:3300/0 conn(0x7f0e4006fa50 0x7f0e400b3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:17.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.811+0000 7f0e3e000640 1 -- 172.21.15.152:0/808623675 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 942799167 0 0) 0x7f0e400b4130 con 0x7f0e4015fd60 2024-04-20T11:34:17.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.811+0000 7f0e3e000640 1 -- 172.21.15.152:0/808623675 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0e24005730 con 0x7f0e4015fd60 2024-04-20T11:34:17.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.812+0000 7f0e3e000640 1 -- 172.21.15.152:0/808623675 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2815371686 0 0) 0x7f0e24005730 con 0x7f0e4015fd60 2024-04-20T11:34:17.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.812+0000 7f0e3e000640 1 -- 172.21.15.152:0/808623675 >> v2:172.21.15.175:3300/0 conn(0x7f0e4006fa50 msgr2=0x7f0e400b3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:17.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.812+0000 7f0e3e000640 1 --2- 172.21.15.152:0/808623675 >> v2:172.21.15.175:3300/0 conn(0x7f0e4006fa50 0x7f0e400b3470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:17.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.812+0000 7f0e3e000640 1 -- 172.21.15.152:0/808623675 >> v1:172.21.15.175:6789/0 conn(0x7f0e4015b2b0 legacy=0x7f0e4006f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:17.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.812+0000 7f0e3e000640 1 -- 172.21.15.152:0/808623675 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e4015f560 con 0x7f0e4015fd60 2024-04-20T11:34:17.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.812+0000 7f0e3e000640 1 -- 172.21.15.152:0/808623675 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f0e380019f0 con 0x7f0e4015fd60 2024-04-20T11:34:17.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.812+0000 7f0e3e000640 1 -- 172.21.15.152:0/808623675 >> v1:172.21.15.152:6789/0 conn(0x7f0e4015fd60 legacy=0x7f0e400b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:17.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.812+0000 7f0e3e000640 1 --2- 172.21.15.152:0/808623675 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0e24005bc0 0x7f0e24005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:17.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.812+0000 7f0e3e000640 1 --2- 172.21.15.152:0/808623675 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0e24006a10 0x7f0e24006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:17.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.812+0000 7f0e3e000640 1 --2- 172.21.15.152:0/808623675 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e240073d0 0x7f0e2400dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:17.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.812+0000 7f0e3e000640 1 -- 172.21.15.152:0/808623675 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f0e4015f8d0 con 0x7f0e24006a10 2024-04-20T11:34:17.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.812+0000 7f0e3e000640 1 -- 172.21.15.152:0/808623675 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0e2400e270 con 0x7f0e240073d0 2024-04-20T11:34:17.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.812+0000 7f0e3e000640 1 -- 172.21.15.152:0/808623675 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0e2400e470 con 0x7f0e24005bc0 2024-04-20T11:34:17.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.812+0000 7f0e37e00640 1 --2- 172.21.15.152:0/808623675 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0e24006a10 0x7f0e24006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:17.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.812+0000 7f0e3f400640 1 --2- 172.21.15.152:0/808623675 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e240073d0 0x7f0e2400dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:17.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.812+0000 7f0e3ea00640 1 --2- 172.21.15.152:0/808623675 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0e24005bc0 0x7f0e24005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:17.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.813+0000 7f0e37e00640 1 -- 172.21.15.152:0/808623675 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0e24005bc0 msgr2=0x7f0e24005fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:17.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.813+0000 7f0e37e00640 1 --2- 172.21.15.152:0/808623675 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0e24005bc0 0x7f0e24005fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:17.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.813+0000 7f0e37e00640 1 -- 172.21.15.152:0/808623675 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e240073d0 msgr2=0x7f0e2400dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:17.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.813+0000 7f0e37e00640 1 --2- 172.21.15.152:0/808623675 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e240073d0 0x7f0e2400dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:17.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.813+0000 7f0e37e00640 1 -- 172.21.15.152:0/808623675 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e24005730 con 0x7f0e24006a10 2024-04-20T11:34:17.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.813+0000 7f0e3ea00640 1 --2- 172.21.15.152:0/808623675 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0e24005bc0 0x7f0e24005fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:34:17.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.813+0000 7f0e3f400640 1 --2- 172.21.15.152:0/808623675 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e240073d0 0x7f0e2400dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:34:17.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.813+0000 7f0e3f400640 1 -- 172.21.15.152:0/808623675 reap_dead start 2024-04-20T11:34:17.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.814+0000 7f0e37e00640 1 --2- 172.21.15.152:0/808623675 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0e24006a10 0x7f0e24006e90 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f0e28009ac0 tx=0x7f0e24007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e07089f5f72439d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:17.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.814+0000 7f0e3e000640 1 -- 172.21.15.152:0/808623675 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e28039070 con 0x7f0e24006a10 2024-04-20T11:34:17.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.814+0000 7f0e3e000640 1 -- 172.21.15.152:0/808623675 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e2802c6b0 con 0x7f0e24006a10 2024-04-20T11:34:17.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.814+0000 7f0e3e000640 1 -- 172.21.15.152:0/808623675 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e28009690 con 0x7f0e24006a10 2024-04-20T11:34:17.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.815+0000 7f0e45c00640 1 -- 172.21.15.152:0/808623675 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0e24006a10 msgr2=0x7f0e24006e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:17.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.815+0000 7f0e45c00640 1 --2- 172.21.15.152:0/808623675 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0e24006a10 0x7f0e24006e90 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f0e28009ac0 tx=0x7f0e24007a40 comp rx=0 tx=0).stop 2024-04-20T11:34:17.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.815+0000 7f0e45c00640 1 -- 172.21.15.152:0/808623675 shutdown_connections 2024-04-20T11:34:17.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.815+0000 7f0e45c00640 1 --2- 172.21.15.152:0/808623675 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0e24006a10 0x7f0e24006e90 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:17.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.815+0000 7f0e45c00640 1 -- 172.21.15.152:0/808623675 >> 172.21.15.152:0/808623675 conn(0x7f0e40067cf0 msgr2=0x7f0e4006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:17.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.815+0000 7f0e45c00640 1 -- 172.21.15.152:0/808623675 shutdown_connections 2024-04-20T11:34:17.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.815+0000 7f0e45c00640 1 -- 172.21.15.152:0/808623675 wait complete. 2024-04-20T11:34:17.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.817+0000 7f0e45c00640 1 Processor -- start 2024-04-20T11:34:17.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.817+0000 7f0e45c00640 1 -- start start 2024-04-20T11:34:17.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.818+0000 7f0e45c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e24006a10 0x7f0e4013c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:17.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.818+0000 7f0e45c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0e4015bfe0 0x7f0e400cf550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:17.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.818+0000 7f0e45c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0e4013ef50 0x7f0e4013f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:17.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.818+0000 7f0e45c00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f0e4006f340 con 0x7f0e4013ef50 2024-04-20T11:34:17.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.818+0000 7f0e45c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0e4015b8a0 con 0x7f0e24006a10 2024-04-20T11:34:17.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.818+0000 7f0e45c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0e4006f4c0 con 0x7f0e4015bfe0 2024-04-20T11:34:17.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.818+0000 7f0e3f400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0e4013ef50 0x7f0e4013f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:17.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.818+0000 7f0e3f400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0e4013ef50 0x7f0e4013f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52194/0 (socket says 172.21.15.152:52194) 2024-04-20T11:34:17.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.818+0000 7f0e3f400640 1 -- 172.21.15.152:0/232257643 learned_addr learned my addr 172.21.15.152:0/232257643 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:17.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.818+0000 7f0e37e00640 1 --2- 172.21.15.152:0/232257643 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0e4015bfe0 0x7f0e400cf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:17.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.818+0000 7f0e3ea00640 1 --2- 172.21.15.152:0/232257643 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e24006a10 0x7f0e4013c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:17.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.818+0000 7f0e3f400640 1 -- 172.21.15.152:0/232257643 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0e4015bfe0 msgr2=0x7f0e400cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:17.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.818+0000 7f0e3f400640 1 --2- 172.21.15.152:0/232257643 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0e4015bfe0 0x7f0e400cf550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:17.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.818+0000 7f0e3f400640 1 -- 172.21.15.152:0/232257643 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e24006a10 msgr2=0x7f0e4013c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:17.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.818+0000 7f0e3f400640 1 --2- 172.21.15.152:0/232257643 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e24006a10 0x7f0e4013c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:17.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.818+0000 7f0e3f400640 1 -- 172.21.15.152:0/232257643 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e400cfa90 con 0x7f0e4013ef50 2024-04-20T11:34:17.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.818+0000 7f0e37e00640 1 --2- 172.21.15.152:0/232257643 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0e4015bfe0 0x7f0e400cf550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:34:17.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.818+0000 7f0e3ea00640 1 --2- 172.21.15.152:0/232257643 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e24006a10 0x7f0e4013c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:34:17.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.819+0000 7f0e3f400640 1 --2- 172.21.15.152:0/232257643 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0e4013ef50 0x7f0e4013f400 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f0e400ccec0 tx=0x7f0e3800b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:17.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.819+0000 7f0e36a00640 1 -- 172.21.15.152:0/232257643 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e38004d60 con 0x7f0e4013ef50 2024-04-20T11:34:17.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.819+0000 7f0e36a00640 1 -- 172.21.15.152:0/232257643 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e3800bb20 con 0x7f0e4013ef50 2024-04-20T11:34:17.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.819+0000 7f0e36a00640 1 -- 172.21.15.152:0/232257643 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e38007110 con 0x7f0e4013ef50 2024-04-20T11:34:17.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.819+0000 7f0e45c00640 1 -- 172.21.15.152:0/232257643 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e401571d0 con 0x7f0e4013ef50 2024-04-20T11:34:17.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.819+0000 7f0e45c00640 1 -- 172.21.15.152:0/232257643 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e400ced50 con 0x7f0e4013ef50 2024-04-20T11:34:17.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.821+0000 7f0e45c00640 1 -- 172.21.15.152:0/232257643 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e401576f0 con 0x7f0e4013ef50 2024-04-20T11:34:17.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.822+0000 7f0e36a00640 1 -- 172.21.15.152:0/232257643 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 19) ==== 91397+0+0 (secure 0 0 0) 0x7f0e3801b020 con 0x7f0e4013ef50 2024-04-20T11:34:17.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.823+0000 7f0e36a00640 1 --2- 172.21.15.152:0/232257643 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f0e1c06cd10 0x7f0e1c06f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:17.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.823+0000 7f0e36a00640 1 -- 172.21.15.152:0/232257643 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5781+0+0 (secure 0 0 0) 0x7f0e3808c5d0 con 0x7f0e4013ef50 2024-04-20T11:34:17.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.823+0000 7f0e3ea00640 1 --2- 172.21.15.152:0/232257643 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f0e1c06cd10 0x7f0e1c06f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:17.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.825+0000 7f0e3ea00640 1 --2- 172.21.15.152:0/232257643 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f0e1c06cd10 0x7f0e1c06f1d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0e40076cd0 tx=0x7f0e30009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:17.826 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:17.827+0000 7f0e36a00640 1 -- 172.21.15.152:0/232257643 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f0e401576f0 con 0x7f0e4013ef50 2024-04-20T11:34:18.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:18.042+0000 7f0e45c00640 1 -- 172.21.15.152:0/232257643 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f0e400ceb40 con 0x7f0e4013ef50 2024-04-20T11:34:18.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:18.043+0000 7f0e36a00640 1 -- 172.21.15.152:0/232257643 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v105) ==== 80+0+0 (secure 0 0 0) 0x7f0e3805a280 con 0x7f0e4013ef50 2024-04-20T11:34:18.043 INFO:teuthology.orchestra.run.smithi152.stderr:listed 0 entries 2024-04-20T11:34:18.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:18.045+0000 7f0e45c00640 1 -- 172.21.15.152:0/232257643 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f0e1c06cd10 msgr2=0x7f0e1c06f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:18.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:18.045+0000 7f0e45c00640 1 --2- 172.21.15.152:0/232257643 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f0e1c06cd10 0x7f0e1c06f1d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0e40076cd0 tx=0x7f0e30009040 comp rx=0 tx=0).stop 2024-04-20T11:34:18.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:18.045+0000 7f0e45c00640 1 -- 172.21.15.152:0/232257643 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0e4013ef50 msgr2=0x7f0e4013f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:18.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:18.045+0000 7f0e45c00640 1 --2- 172.21.15.152:0/232257643 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0e4013ef50 0x7f0e4013f400 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f0e400ccec0 tx=0x7f0e3800b4a0 comp rx=0 tx=0).stop 2024-04-20T11:34:18.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:18.046+0000 7f0e45c00640 1 -- 172.21.15.152:0/232257643 shutdown_connections 2024-04-20T11:34:18.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:18.046+0000 7f0e45c00640 1 --2- 172.21.15.152:0/232257643 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f0e1c06cd10 0x7f0e1c06f1d0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:18.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:18.046+0000 7f0e45c00640 1 --2- 172.21.15.152:0/232257643 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0e4013ef50 0x7f0e4013f400 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:18.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:18.046+0000 7f0e45c00640 1 --2- 172.21.15.152:0/232257643 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0e4015bfe0 0x7f0e400cf550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:18.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:18.046+0000 7f0e45c00640 1 --2- 172.21.15.152:0/232257643 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e24006a10 0x7f0e4013c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:18.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:18.046+0000 7f0e45c00640 1 -- 172.21.15.152:0/232257643 >> 172.21.15.152:0/232257643 conn(0x7f0e40067cf0 msgr2=0x7f0e40072790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:18.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:18.046+0000 7f0e45c00640 1 -- 172.21.15.152:0/232257643 shutdown_connections 2024-04-20T11:34:18.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:18.046+0000 7f0e45c00640 1 -- 172.21.15.152:0/232257643 wait complete. 2024-04-20T11:34:18.065 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi175.front.sepia.ceph.com... 2024-04-20T11:34:18.065 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:34:18.065 DEBUG:teuthology.orchestra.run.smithi175:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-04-20T11:34:18.148 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi175.stderr:ceph-fuse[33724]: fuse finished with error 0 and tester_r 0 2024-04-20T11:34:18.152 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi175.front.sepia.ceph.com 2024-04-20T11:34:18.152 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:34:18.152 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-20T11:34:18.197 INFO:tasks.cephfs.mount:Terminating background process 2024-04-20T11:34:18.197 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-20T11:34:18.197 DEBUG:teuthology.orchestra.run.smithi152:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:34:18.254 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-20T11:34:18.254 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi152.front.sepia.ceph.com 2024-04-20T11:34:18.254 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:34:18.254 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-20T11:34:18.302 INFO:teuthology.orchestra.run.smithi152.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-20T11:34:18.303 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T11:34:18.303 INFO:tasks.cephfs.mount:Terminating background process 2024-04-20T11:34:18.303 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T11:34:18.304 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:34:18.359 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-20T11:34:18.359 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi175.front.sepia.ceph.com 2024-04-20T11:34:18.359 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:34:18.360 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-20T11:34:18.413 INFO:teuthology.orchestra.run.smithi175.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-20T11:34:18.414 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T11:34:18.415 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:34:18.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:18.624+0000 7fe688c00640 1 Processor -- start 2024-04-20T11:34:18.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:18.624+0000 7fe688c00640 1 -- start start 2024-04-20T11:34:18.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:18.626+0000 7fe688c00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fe68415c900 0x7fe6840cfd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:18.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:18.626+0000 7fe688c00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fe6840676c0 0x7fe6840d0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:18.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:18.626+0000 7fe688c00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe684160750 con 0x7fe684070090 2024-04-20T11:34:18.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:18.626+0000 7fe688c00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fe684160500 con 0x7fe68415c900 2024-04-20T11:34:18.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:18.626+0000 7fe688c00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fe684161080 con 0x7fe6840676c0 2024-04-20T11:34:18.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:18.626+0000 7fe681e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fe68415c900 0x7fe6840cfd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:18.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:18.626+0000 7fe681e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fe68415c900 0x7fe6840cfd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52202/0 (socket says 172.21.15.152:52202) 2024-04-20T11:34:18.627 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:18.626+0000 7fe681e00640 1 -- 172.21.15.152:0/3296816223 learned_addr learned my addr 172.21.15.152:0/3296816223 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:18.627 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:18.626+0000 7fe681e00640 1 -- 172.21.15.152:0/3296816223 >> v2:172.21.15.175:3300/0 conn(0x7fe6840676c0 msgr2=0x7fe6840d0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:18.627 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:18.626+0000 7fe681e00640 1 --2- 172.21.15.152:0/3296816223 >> v2:172.21.15.175:3300/0 conn(0x7fe6840676c0 0x7fe6840d0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:18.627 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:18.626+0000 7fe681e00640 1 -- 172.21.15.152:0/3296816223 >> v1:172.21.15.175:6789/0 conn(0x7fe684070090 legacy=0x7fe6840ceb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:18.627 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:18.626+0000 7fe681e00640 1 -- 172.21.15.152:0/3296816223 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe684160a30 con 0x7fe68415c900 2024-04-20T11:34:18.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:18.628+0000 7fe681e00640 1 --2- 172.21.15.152:0/3296816223 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe68415c900 0x7fe6840cfd50 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7fe684050c50 tx=0x7fe67402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9a735589aa576e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:18.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:18.628+0000 7fe680a00640 1 -- 172.21.15.152:0/3296816223 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe674034070 con 0x7fe68415c900 2024-04-20T11:34:18.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:18.628+0000 7fe680a00640 1 -- 172.21.15.152:0/3296816223 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe674030860 con 0x7fe68415c900 2024-04-20T11:34:18.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:18.628+0000 7fe680a00640 1 -- 172.21.15.152:0/3296816223 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe674030c30 con 0x7fe68415c900 2024-04-20T11:34:18.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:18.629+0000 7fe688c00640 1 -- 172.21.15.152:0/3296816223 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe68415c900 msgr2=0x7fe6840cfd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:18.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:18.629+0000 7fe688c00640 1 --2- 172.21.15.152:0/3296816223 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe68415c900 0x7fe6840cfd50 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7fe684050c50 tx=0x7fe67402b9a0 comp rx=0 tx=0).stop 2024-04-20T11:34:18.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:18.629+0000 7fe688c00640 1 -- 172.21.15.152:0/3296816223 shutdown_connections 2024-04-20T11:34:18.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:18.629+0000 7fe688c00640 1 --2- 172.21.15.152:0/3296816223 >> v2:172.21.15.175:3300/0 conn(0x7fe6840676c0 0x7fe6840d0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:18.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:18.629+0000 7fe688c00640 1 --2- 172.21.15.152:0/3296816223 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe68415c900 0x7fe6840cfd50 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:18.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:18.629+0000 7fe688c00640 1 -- 172.21.15.152:0/3296816223 >> 172.21.15.152:0/3296816223 conn(0x7fe6840cd6f0 msgr2=0x7fe6840cdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:18.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:18.629+0000 7fe688c00640 1 -- 172.21.15.152:0/3296816223 shutdown_connections 2024-04-20T11:34:18.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:18.629+0000 7fe688c00640 1 -- 172.21.15.152:0/3296816223 wait complete. 2024-04-20T11:34:18.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:18.630+0000 7fe688c00640 1 Processor -- start 2024-04-20T11:34:18.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:18.631+0000 7fe688c00640 1 -- start start 2024-04-20T11:34:18.632 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:18.631+0000 7fe688c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe6840676c0 0x7fe68415d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:18.632 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:18.632+0000 7fe688c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe684070090 0x7fe6840b3da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:18.632 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:18.632+0000 7fe688c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe68415c900 0x7fe6840af650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:18.632 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:18.632+0000 7fe688c00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fe6840ce3c0 con 0x7fe68415c900 2024-04-20T11:34:18.632 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:18.632+0000 7fe688c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe68406b980 con 0x7fe684070090 2024-04-20T11:34:18.632 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:18.632+0000 7fe688c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe6840ce540 con 0x7fe6840676c0 2024-04-20T11:34:18.632 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:18.632+0000 7fe682800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe68415c900 0x7fe6840af650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:18.632 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:18.632+0000 7fe682800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe68415c900 0x7fe6840af650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52208/0 (socket says 172.21.15.152:52208) 2024-04-20T11:34:18.632 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:18.632+0000 7fe682800640 1 -- 172.21.15.152:0/136682366 learned_addr learned my addr 172.21.15.152:0/136682366 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:18.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:18.632+0000 7fe682800640 1 -- 172.21.15.152:0/136682366 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe6840676c0 msgr2=0x7fe68415d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:18.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:18.632+0000 7fe682800640 1 --2- 172.21.15.152:0/136682366 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe6840676c0 0x7fe68415d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:18.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:18.632+0000 7fe682800640 1 -- 172.21.15.152:0/136682366 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe684070090 msgr2=0x7fe6840b3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:18.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:18.632+0000 7fe682800640 1 --2- 172.21.15.152:0/136682366 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe684070090 0x7fe6840b3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:18.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:18.632+0000 7fe682800640 1 -- 172.21.15.152:0/136682366 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6840b45f0 con 0x7fe68415c900 2024-04-20T11:34:18.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:18.633+0000 7fe682800640 1 --2- 172.21.15.152:0/136682366 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe68415c900 0x7fe6840af650 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7fe67c00c390 tx=0x7fe67c00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:18.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:18.633+0000 7fe67a000640 1 -- 172.21.15.152:0/136682366 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe67c017070 con 0x7fe68415c900 2024-04-20T11:34:18.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:18.633+0000 7fe67a000640 1 -- 172.21.15.152:0/136682366 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe67c0035f0 con 0x7fe68415c900 2024-04-20T11:34:18.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:18.633+0000 7fe67a000640 1 -- 172.21.15.152:0/136682366 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe67c0039e0 con 0x7fe68415c900 2024-04-20T11:34:18.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:18.633+0000 7fe688c00640 1 -- 172.21.15.152:0/136682366 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6840b48d0 con 0x7fe68415c900 2024-04-20T11:34:18.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:18.633+0000 7fe688c00640 1 -- 172.21.15.152:0/136682366 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6840b4e90 con 0x7fe68415c900 2024-04-20T11:34:18.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:18.634+0000 7fe688c00640 1 -- 172.21.15.152:0/136682366 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe684069100 con 0x7fe68415c900 2024-04-20T11:34:18.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:18.635+0000 7fe67a000640 1 -- 172.21.15.152:0/136682366 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 19) ==== 91397+0+0 (secure 0 0 0) 0x7fe67c01a070 con 0x7fe68415c900 2024-04-20T11:34:18.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:18.636+0000 7fe67a000640 1 --2- 172.21.15.152:0/136682366 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe66006cce0 0x7fe66006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:18.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:18.637+0000 7fe67a000640 1 -- 172.21.15.152:0/136682366 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5781+0+0 (secure 0 0 0) 0x7fe67c08b6d0 con 0x7fe68415c900 2024-04-20T11:34:18.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:18.637+0000 7fe681e00640 1 --2- 172.21.15.152:0/136682366 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe66006cce0 0x7fe66006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:18.638 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:18.639+0000 7fe681e00640 1 --2- 172.21.15.152:0/136682366 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe66006cce0 0x7fe66006f1a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe674036000 tx=0x7fe674030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:18.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:18.641+0000 7fe67a000640 1 -- 172.21.15.152:0/136682366 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe67c059550 con 0x7fe68415c900 2024-04-20T11:34:18.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:18.882+0000 7fe688c00640 1 -- 172.21.15.152:0/136682366 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe6840cf340 con 0x7fe68415c900 2024-04-20T11:34:18.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:18.884+0000 7fe67a000640 1 -- 172.21.15.152:0/136682366 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+5210 (secure 0 0 0) 0x7fe67c058e50 con 0x7fe68415c900 2024-04-20T11:34:18.884 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:34:18.884 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":67,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5648,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6834/387198089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":387198089},{"type":"v1","addr":"172.21.15.152:6834","nonce":387198089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":6154,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/1674013715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1674013715},{"type":"v1","addr":"172.21.15.175:6833","nonce":1674013715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":6157,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/3898238490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3898238490},{"type":"v1","addr":"172.21.15.175:6835","nonce":3898238490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T11:33:27.277882+0000","modified":"2024-04-20T11:33:30.342429+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7794},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7794":{"gid":7794,"name":"a","rank":0,"incarnation":62,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6835/3582312712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6833","nonce":3582312712},{"type":"v1","addr":"172.21.15.152:6835","nonce":3582312712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7794,"qdb_cluster":[7794]},"id":8}]} 2024-04-20T11:34:18.884 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 67 2024-04-20T11:34:18.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:18.887+0000 7fe688c00640 1 -- 172.21.15.152:0/136682366 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe66006cce0 msgr2=0x7fe66006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:18.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:18.887+0000 7fe688c00640 1 --2- 172.21.15.152:0/136682366 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe66006cce0 0x7fe66006f1a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe674036000 tx=0x7fe674030000 comp rx=0 tx=0).stop 2024-04-20T11:34:18.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:18.887+0000 7fe688c00640 1 -- 172.21.15.152:0/136682366 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe68415c900 msgr2=0x7fe6840af650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:18.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:18.887+0000 7fe688c00640 1 --2- 172.21.15.152:0/136682366 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe68415c900 0x7fe6840af650 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7fe67c00c390 tx=0x7fe67c00c850 comp rx=0 tx=0).stop 2024-04-20T11:34:18.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:18.887+0000 7fe688c00640 1 -- 172.21.15.152:0/136682366 shutdown_connections 2024-04-20T11:34:18.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:18.887+0000 7fe688c00640 1 --2- 172.21.15.152:0/136682366 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe66006cce0 0x7fe66006f1a0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:18.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:18.887+0000 7fe688c00640 1 --2- 172.21.15.152:0/136682366 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe68415c900 0x7fe6840af650 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:18.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:18.887+0000 7fe688c00640 1 --2- 172.21.15.152:0/136682366 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe684070090 0x7fe6840b3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:18.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:18.887+0000 7fe688c00640 1 --2- 172.21.15.152:0/136682366 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe6840676c0 0x7fe68415d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:18.888 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:18.887+0000 7fe688c00640 1 -- 172.21.15.152:0/136682366 >> 172.21.15.152:0/136682366 conn(0x7fe6840cd6f0 msgr2=0x7fe68406bb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:18.888 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:18.887+0000 7fe688c00640 1 -- 172.21.15.152:0/136682366 shutdown_connections 2024-04-20T11:34:18.888 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:18.887+0000 7fe688c00640 1 -- 172.21.15.152:0/136682366 wait complete. 2024-04-20T11:34:18.906 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:34:19.113 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.114+0000 7f8385a00640 1 Processor -- start 2024-04-20T11:34:19.114 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.114+0000 7f8385a00640 1 -- start start 2024-04-20T11:34:19.116 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.115+0000 7f8385a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f838006eb40 0x7f838006f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:19.116 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.115+0000 7f8385a00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f838006fca0 con 0x7f838015b5e0 2024-04-20T11:34:19.116 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.115+0000 7f8385a00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83800b5770 con 0x7f838006f880 2024-04-20T11:34:19.116 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.115+0000 7f8385a00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f838006de10 con 0x7f838006eb40 2024-04-20T11:34:19.116 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.116+0000 7f837e000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f838006eb40 0x7f838006f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:19.116 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.116+0000 7f837e000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f838006eb40 0x7f838006f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52210/0 (socket says 172.21.15.152:52210) 2024-04-20T11:34:19.116 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.116+0000 7f837ea00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f838015b5e0 0x7f83800b4c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:40234/0 (socket says 172.21.15.152:40234) 2024-04-20T11:34:19.116 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.116+0000 7f837e000640 1 -- 172.21.15.152:0/2532944739 learned_addr learned my addr 172.21.15.152:0/2532944739 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:19.116 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.116+0000 7f837e000640 1 -- 172.21.15.152:0/2532944739 >> v1:172.21.15.175:6789/0 conn(0x7f838006f880 legacy=0x7f838015cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:19.116 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.116+0000 7f837e000640 1 -- 172.21.15.152:0/2532944739 >> v1:172.21.15.152:6789/0 conn(0x7f838015b5e0 legacy=0x7f83800b4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:19.116 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.116+0000 7f837e000640 1 -- 172.21.15.152:0/2532944739 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f838006e060 con 0x7f838006eb40 2024-04-20T11:34:19.116 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.116+0000 7f837d600640 1 -- 172.21.15.152:0/2532944739 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 462767255 0 0) 0x7f838006fca0 con 0x7f838015b5e0 2024-04-20T11:34:19.118 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.117+0000 7f837e000640 1 --2- 172.21.15.152:0/2532944739 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f838006eb40 0x7f838006f340 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f836c009870 tx=0x7f836c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3515ead28f805219 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:19.118 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.118+0000 7f837d600640 1 -- 172.21.15.152:0/2532944739 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f836c034070 con 0x7f838006eb40 2024-04-20T11:34:19.118 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.118+0000 7f837d600640 1 -- 172.21.15.152:0/2532944739 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f836c030860 con 0x7f838006eb40 2024-04-20T11:34:19.118 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.118+0000 7f837d600640 1 -- 172.21.15.152:0/2532944739 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f836c030c30 con 0x7f838006eb40 2024-04-20T11:34:19.118 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.118+0000 7f8385a00640 1 -- 172.21.15.152:0/2532944739 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f838006eb40 msgr2=0x7f838006f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:19.118 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.118+0000 7f8385a00640 1 --2- 172.21.15.152:0/2532944739 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f838006eb40 0x7f838006f340 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f836c009870 tx=0x7f836c02b9a0 comp rx=0 tx=0).stop 2024-04-20T11:34:19.118 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.118+0000 7f8385a00640 1 -- 172.21.15.152:0/2532944739 shutdown_connections 2024-04-20T11:34:19.118 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.118+0000 7f8385a00640 1 --2- 172.21.15.152:0/2532944739 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f838006eb40 0x7f838006f340 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:19.118 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.118+0000 7f8385a00640 1 -- 172.21.15.152:0/2532944739 >> 172.21.15.152:0/2532944739 conn(0x7f8380067cf0 msgr2=0x7f838006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:19.118 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.118+0000 7f8385a00640 1 -- 172.21.15.152:0/2532944739 shutdown_connections 2024-04-20T11:34:19.118 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.118+0000 7f8385a00640 1 -- 172.21.15.152:0/2532944739 wait complete. 2024-04-20T11:34:19.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.120+0000 7f8385a00640 1 Processor -- start 2024-04-20T11:34:19.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.120+0000 7f8385a00640 1 -- start start 2024-04-20T11:34:19.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.121+0000 7f8385a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f838006eb40 0x7f8380151590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:19.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.121+0000 7f8385a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f838006f880 0x7f8380150c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:19.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.121+0000 7f8385a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f838015b5e0 0x7f8380146160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:19.122 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.121+0000 7f8385a00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f838015c9d0 con 0x7f838006eb40 2024-04-20T11:34:19.122 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.121+0000 7f8385a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f838006e620 con 0x7f838006f880 2024-04-20T11:34:19.122 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.121+0000 7f8385a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f83800b42d0 con 0x7f838015b5e0 2024-04-20T11:34:19.122 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.121+0000 7f837ea00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f838006eb40 0x7f8380151590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:19.122 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.121+0000 7f837ea00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f838006eb40 0x7f8380151590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52216/0 (socket says 172.21.15.152:52216) 2024-04-20T11:34:19.122 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.121+0000 7f837ea00640 1 -- 172.21.15.152:0/3986779847 learned_addr learned my addr 172.21.15.152:0/3986779847 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:19.122 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.122+0000 7f837e000640 1 --2- 172.21.15.152:0/3986779847 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f838006f880 0x7f8380150c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:19.122 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.122+0000 7f837ea00640 1 -- 172.21.15.152:0/3986779847 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f838015b5e0 msgr2=0x7f8380146160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:19.122 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.122+0000 7f837ea00640 1 --2- 172.21.15.152:0/3986779847 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f838015b5e0 0x7f8380146160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:19.122 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.122+0000 7f837ea00640 1 -- 172.21.15.152:0/3986779847 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f838006f880 msgr2=0x7f8380150c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:19.122 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.122+0000 7f837ea00640 1 --2- 172.21.15.152:0/3986779847 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f838006f880 0x7f8380150c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:19.122 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.122+0000 7f837ea00640 1 -- 172.21.15.152:0/3986779847 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8380151140 con 0x7f838006eb40 2024-04-20T11:34:19.122 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.122+0000 7f837ea00640 1 --2- 172.21.15.152:0/3986779847 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f838006eb40 0x7f8380151590 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f837000c4d0 tx=0x7f837000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:19.123 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.122+0000 7f8376a00640 1 -- 172.21.15.152:0/3986779847 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8370018070 con 0x7f838006eb40 2024-04-20T11:34:19.123 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.122+0000 7f8376a00640 1 -- 172.21.15.152:0/3986779847 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8370007500 con 0x7f838006eb40 2024-04-20T11:34:19.123 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.122+0000 7f8376a00640 1 -- 172.21.15.152:0/3986779847 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8370004a20 con 0x7f838006eb40 2024-04-20T11:34:19.124 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.122+0000 7f8385a00640 1 -- 172.21.15.152:0/3986779847 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83801541a0 con 0x7f838006eb40 2024-04-20T11:34:19.124 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.122+0000 7f8385a00640 1 -- 172.21.15.152:0/3986779847 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83801537b0 con 0x7f838006eb40 2024-04-20T11:34:19.124 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.123+0000 7f8385a00640 1 -- 172.21.15.152:0/3986779847 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83800689f0 con 0x7f838006eb40 2024-04-20T11:34:19.124 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.124+0000 7f8376a00640 1 -- 172.21.15.152:0/3986779847 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 19) ==== 91397+0+0 (secure 0 0 0) 0x7f837001b070 con 0x7f838006eb40 2024-04-20T11:34:19.125 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.125+0000 7f8376a00640 1 --2- 172.21.15.152:0/3986779847 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f836006cce0 0x7f836006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:19.125 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.125+0000 7f8376a00640 1 -- 172.21.15.152:0/3986779847 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 5737+0+0 (secure 0 0 0) 0x7f837008b5a0 con 0x7f838006eb40 2024-04-20T11:34:19.125 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.126+0000 7f837e000640 1 --2- 172.21.15.152:0/3986779847 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f836006cce0 0x7f836006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:19.127 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.127+0000 7f837e000640 1 --2- 172.21.15.152:0/3986779847 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f836006cce0 0x7f836006f1a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f836c036000 tx=0x7f836c008030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:19.130 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.130+0000 7f8376a00640 1 -- 172.21.15.152:0/3986779847 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f8370059390 con 0x7f838006eb40 2024-04-20T11:34:19.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.346+0000 7f8385a00640 1 -- 172.21.15.152:0/3986779847 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f83800d07d0 con 0x7f838006eb40 2024-04-20T11:34:19.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.348+0000 7f8376a00640 1 -- 172.21.15.152:0/3986779847 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) ==== 74+0+16754 (secure 0 0 0) 0x7f8370058c90 con 0x7f838006eb40 2024-04-20T11:34:19.349 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:34:19.349 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":106,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:34:19.060050+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:33:25.062365+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-20T11:33:26.260945+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":105,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:34:19.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.351+0000 7f8385a00640 1 -- 172.21.15.152:0/3986779847 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f836006cce0 msgr2=0x7f836006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:19.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.351+0000 7f8385a00640 1 --2- 172.21.15.152:0/3986779847 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f836006cce0 0x7f836006f1a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f836c036000 tx=0x7f836c008030 comp rx=0 tx=0).stop 2024-04-20T11:34:19.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.351+0000 7f8385a00640 1 -- 172.21.15.152:0/3986779847 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f838006eb40 msgr2=0x7f8380151590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:19.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.351+0000 7f8385a00640 1 --2- 172.21.15.152:0/3986779847 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f838006eb40 0x7f8380151590 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f837000c4d0 tx=0x7f837000c990 comp rx=0 tx=0).stop 2024-04-20T11:34:19.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.351+0000 7f8385a00640 1 -- 172.21.15.152:0/3986779847 shutdown_connections 2024-04-20T11:34:19.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.351+0000 7f8385a00640 1 --2- 172.21.15.152:0/3986779847 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f836006cce0 0x7f836006f1a0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:19.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.351+0000 7f8385a00640 1 --2- 172.21.15.152:0/3986779847 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f838015b5e0 0x7f8380146160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:19.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.351+0000 7f8385a00640 1 --2- 172.21.15.152:0/3986779847 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f838006f880 0x7f8380150c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:19.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.351+0000 7f8385a00640 1 --2- 172.21.15.152:0/3986779847 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f838006eb40 0x7f8380151590 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:19.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.351+0000 7f8385a00640 1 -- 172.21.15.152:0/3986779847 >> 172.21.15.152:0/3986779847 conn(0x7f8380067cf0 msgr2=0x7f8380159410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:19.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.351+0000 7f8385a00640 1 -- 172.21.15.152:0/3986779847 shutdown_connections 2024-04-20T11:34:19.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.351+0000 7f8385a00640 1 -- 172.21.15.152:0/3986779847 wait complete. 2024-04-20T11:34:19.369 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T11:26:47.246571+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T11:33:25.062365+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '102', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T11:33:26.260945+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '105', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 5, 'snap_epoch': 105, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T11:34:19.369 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T11:34:19.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.566+0000 7fcd93200640 1 Processor -- start 2024-04-20T11:34:19.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.567+0000 7fcd93200640 1 -- start start 2024-04-20T11:34:19.567 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.568+0000 7fcd93200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fcd940cf5b0 0x7fcd940d4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:19.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.568+0000 7fcd93200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fcd940ceeb0 0x7fcd940cf990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:19.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.568+0000 7fcd93200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd940cfed0 con 0x7fcd94162430 2024-04-20T11:34:19.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.568+0000 7fcd93200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fcd940d5420 con 0x7fcd940cf5b0 2024-04-20T11:34:19.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.568+0000 7fcd93200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fcd940d2b70 con 0x7fcd940ceeb0 2024-04-20T11:34:19.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.568+0000 7fcd91e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fcd940cf5b0 0x7fcd940d4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:19.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.569+0000 7fcd91e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fcd940cf5b0 0x7fcd940d4ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52230/0 (socket says 172.21.15.152:52230) 2024-04-20T11:34:19.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.569+0000 7fcd91e00640 1 -- 172.21.15.152:0/1859194737 learned_addr learned my addr 172.21.15.152:0/1859194737 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:19.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.569+0000 7fcd91400640 1 --2- 172.21.15.152:0/1859194737 >> v2:172.21.15.175:3300/0 conn(0x7fcd940ceeb0 0x7fcd940cf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:19.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.569+0000 7fcd91e00640 1 -- 172.21.15.152:0/1859194737 >> v2:172.21.15.175:3300/0 conn(0x7fcd940ceeb0 msgr2=0x7fcd940cf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:19.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.569+0000 7fcd91e00640 1 --2- 172.21.15.152:0/1859194737 >> v2:172.21.15.175:3300/0 conn(0x7fcd940ceeb0 0x7fcd940cf990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:19.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.569+0000 7fcd91e00640 1 -- 172.21.15.152:0/1859194737 >> v1:172.21.15.175:6789/0 conn(0x7fcd94162430 legacy=0x7fcd941628c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:19.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.569+0000 7fcd91e00640 1 -- 172.21.15.152:0/1859194737 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd940d2e50 con 0x7fcd940cf5b0 2024-04-20T11:34:19.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.570+0000 7fcd91e00640 1 --2- 172.21.15.152:0/1859194737 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcd940cf5b0 0x7fcd940d4ee0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7fcd9408cb50 tx=0x7fcd8402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4b4688aeda915e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:19.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.571+0000 7fcd90a00640 1 -- 172.21.15.152:0/1859194737 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd84034070 con 0x7fcd940cf5b0 2024-04-20T11:34:19.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.571+0000 7fcd90a00640 1 -- 172.21.15.152:0/1859194737 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd84030860 con 0x7fcd940cf5b0 2024-04-20T11:34:19.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.571+0000 7fcd90a00640 1 -- 172.21.15.152:0/1859194737 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd84030c30 con 0x7fcd940cf5b0 2024-04-20T11:34:19.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.572+0000 7fcd93200640 1 -- 172.21.15.152:0/1859194737 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcd940cf5b0 msgr2=0x7fcd940d4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:19.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.572+0000 7fcd93200640 1 --2- 172.21.15.152:0/1859194737 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcd940cf5b0 0x7fcd940d4ee0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7fcd9408cb50 tx=0x7fcd8402b9a0 comp rx=0 tx=0).stop 2024-04-20T11:34:19.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.572+0000 7fcd93200640 1 -- 172.21.15.152:0/1859194737 shutdown_connections 2024-04-20T11:34:19.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.572+0000 7fcd93200640 1 --2- 172.21.15.152:0/1859194737 >> v2:172.21.15.175:3300/0 conn(0x7fcd940ceeb0 0x7fcd940cf990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:19.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.572+0000 7fcd93200640 1 --2- 172.21.15.152:0/1859194737 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcd940cf5b0 0x7fcd940d4ee0 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:19.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.572+0000 7fcd93200640 1 -- 172.21.15.152:0/1859194737 >> 172.21.15.152:0/1859194737 conn(0x7fcd9408bfd0 msgr2=0x7fcd940d4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:19.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.572+0000 7fcd93200640 1 -- 172.21.15.152:0/1859194737 shutdown_connections 2024-04-20T11:34:19.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.572+0000 7fcd93200640 1 -- 172.21.15.152:0/1859194737 wait complete. 2024-04-20T11:34:19.574 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.574+0000 7fcd93200640 1 Processor -- start 2024-04-20T11:34:19.574 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.574+0000 7fcd93200640 1 -- start start 2024-04-20T11:34:19.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.576+0000 7fcd93200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcd940ceeb0 0x7fcd94151ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:19.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.576+0000 7fcd93200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcd940cf5b0 0x7fcd941e97a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:19.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.576+0000 7fcd93200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd94162430 0x7fcd941f5b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:19.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.576+0000 7fcd93200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fcd940d36f0 con 0x7fcd940ceeb0 2024-04-20T11:34:19.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.576+0000 7fcd93200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcd940d3410 con 0x7fcd94162430 2024-04-20T11:34:19.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.576+0000 7fcd93200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fcd94161240 con 0x7fcd940cf5b0 2024-04-20T11:34:19.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.576+0000 7fcd91e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcd940ceeb0 0x7fcd94151ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:19.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.576+0000 7fcd91e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcd940ceeb0 0x7fcd94151ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52232/0 (socket says 172.21.15.152:52232) 2024-04-20T11:34:19.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.576+0000 7fcd91e00640 1 -- 172.21.15.152:0/3712320192 learned_addr learned my addr 172.21.15.152:0/3712320192 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:19.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.576+0000 7fcd91e00640 1 -- 172.21.15.152:0/3712320192 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcd940cf5b0 msgr2=0x7fcd941e97a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:19.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.576+0000 7fcd91e00640 1 --2- 172.21.15.152:0/3712320192 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcd940cf5b0 0x7fcd941e97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:19.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.576+0000 7fcd91e00640 1 -- 172.21.15.152:0/3712320192 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd94162430 msgr2=0x7fcd941f5b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:19.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.576+0000 7fcd91e00640 1 --2- 172.21.15.152:0/3712320192 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd94162430 0x7fcd941f5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:19.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.576+0000 7fcd91e00640 1 -- 172.21.15.152:0/3712320192 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd941e8700 con 0x7fcd940ceeb0 2024-04-20T11:34:19.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.577+0000 7fcd91e00640 1 --2- 172.21.15.152:0/3712320192 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcd940ceeb0 0x7fcd94151ac0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7fcd84000c00 tx=0x7fcd84004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:19.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.577+0000 7fcd8a000640 1 -- 172.21.15.152:0/3712320192 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd84048070 con 0x7fcd940ceeb0 2024-04-20T11:34:19.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.577+0000 7fcd8a000640 1 -- 172.21.15.152:0/3712320192 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd84039d90 con 0x7fcd940ceeb0 2024-04-20T11:34:19.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.577+0000 7fcd8a000640 1 -- 172.21.15.152:0/3712320192 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd84034040 con 0x7fcd940ceeb0 2024-04-20T11:34:19.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.577+0000 7fcd93200640 1 -- 172.21.15.152:0/3712320192 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd941e89e0 con 0x7fcd940ceeb0 2024-04-20T11:34:19.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.577+0000 7fcd93200640 1 -- 172.21.15.152:0/3712320192 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd941e8fa0 con 0x7fcd940ceeb0 2024-04-20T11:34:19.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.578+0000 7fcd93200640 1 -- 172.21.15.152:0/3712320192 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd9415fe50 con 0x7fcd940ceeb0 2024-04-20T11:34:19.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.579+0000 7fcd8a000640 1 -- 172.21.15.152:0/3712320192 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 19) ==== 91397+0+0 (secure 0 0 0) 0x7fcd8404b070 con 0x7fcd940ceeb0 2024-04-20T11:34:19.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.581+0000 7fcd8a000640 1 --2- 172.21.15.152:0/3712320192 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fcd7006cce0 0x7fcd7006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:19.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.582+0000 7fcd8a000640 1 -- 172.21.15.152:0/3712320192 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 5737+0+0 (secure 0 0 0) 0x7fcd840ae750 con 0x7fcd940ceeb0 2024-04-20T11:34:19.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.582+0000 7fcd91400640 1 --2- 172.21.15.152:0/3712320192 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fcd7006cce0 0x7fcd7006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:19.583 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.583+0000 7fcd91400640 1 --2- 172.21.15.152:0/3712320192 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fcd7006cce0 0x7fcd7006f1a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fcd941f85c0 tx=0x7fcd7c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:19.586 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.587+0000 7fcd8a000640 1 -- 172.21.15.152:0/3712320192 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fcd9415fe50 con 0x7fcd940ceeb0 2024-04-20T11:34:19.795 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.795+0000 7fcd93200640 1 -- 172.21.15.152:0/3712320192 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fcd94075e60 con 0x7fcd940ceeb0 2024-04-20T11:34:19.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.796+0000 7fcd8a000640 1 -- 172.21.15.152:0/3712320192 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v106) ==== 93+0+10 (secure 0 0 0) 0x7fcd8407c490 con 0x7fcd940ceeb0 2024-04-20T11:34:19.796 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 1 2024-04-20T11:34:19.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.799+0000 7fcd93200640 1 -- 172.21.15.152:0/3712320192 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fcd7006cce0 msgr2=0x7fcd7006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:19.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.799+0000 7fcd93200640 1 --2- 172.21.15.152:0/3712320192 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fcd7006cce0 0x7fcd7006f1a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fcd941f85c0 tx=0x7fcd7c008040 comp rx=0 tx=0).stop 2024-04-20T11:34:19.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.799+0000 7fcd93200640 1 -- 172.21.15.152:0/3712320192 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcd940ceeb0 msgr2=0x7fcd94151ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:19.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.799+0000 7fcd93200640 1 --2- 172.21.15.152:0/3712320192 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcd940ceeb0 0x7fcd94151ac0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7fcd84000c00 tx=0x7fcd84004730 comp rx=0 tx=0).stop 2024-04-20T11:34:19.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.799+0000 7fcd93200640 1 -- 172.21.15.152:0/3712320192 shutdown_connections 2024-04-20T11:34:19.800 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.799+0000 7fcd93200640 1 --2- 172.21.15.152:0/3712320192 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fcd7006cce0 0x7fcd7006f1a0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:19.800 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.799+0000 7fcd93200640 1 --2- 172.21.15.152:0/3712320192 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd94162430 0x7fcd941f5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:19.800 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.799+0000 7fcd93200640 1 --2- 172.21.15.152:0/3712320192 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcd940cf5b0 0x7fcd941e97a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:19.800 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.799+0000 7fcd93200640 1 --2- 172.21.15.152:0/3712320192 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcd940ceeb0 0x7fcd94151ac0 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:19.800 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.799+0000 7fcd93200640 1 -- 172.21.15.152:0/3712320192 >> 172.21.15.152:0/3712320192 conn(0x7fcd9408bfd0 msgr2=0x7fcd940cda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:19.800 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.799+0000 7fcd93200640 1 -- 172.21.15.152:0/3712320192 shutdown_connections 2024-04-20T11:34:19.800 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:19.799+0000 7fcd93200640 1 -- 172.21.15.152:0/3712320192 wait complete. 2024-04-20T11:34:19.818 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T11:34:20.019 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.019+0000 7fb881400640 1 Processor -- start 2024-04-20T11:34:20.019 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.019+0000 7fb881400640 1 -- start start 2024-04-20T11:34:20.021 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.021+0000 7fb881400640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fb87c06ee00 0x7fb87c06f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:20.021 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.021+0000 7fb881400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fb87c15fbf0 0x7fb87c15c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:20.021 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.021+0000 7fb881400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb87c0cf110 0x7fb87c0b3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:20.021 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.021+0000 7fb881400640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fb87c0d00d0 con 0x7fb87c15fbf0 2024-04-20T11:34:20.021 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.021+0000 7fb881400640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fb87c06f720 con 0x7fb87c06ee00 2024-04-20T11:34:20.021 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.021+0000 7fb881400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb87c0cfb10 con 0x7fb87c0cf110 2024-04-20T11:34:20.021 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.021+0000 7fb87b400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fb87c15fbf0 0x7fb87c15c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:20.021 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.021+0000 7fb87b400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fb87c15fbf0 0x7fb87c15c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52234/0 (socket says 172.21.15.152:52234) 2024-04-20T11:34:20.022 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.021+0000 7fb87b400640 1 -- 172.21.15.152:0/2899187133 learned_addr learned my addr 172.21.15.152:0/2899187133 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:20.022 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.021+0000 7fb87b400640 1 -- 172.21.15.152:0/2899187133 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb87c0cf110 msgr2=0x7fb87c0b3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:20.022 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.021+0000 7fb87b400640 1 --2- 172.21.15.152:0/2899187133 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb87c0cf110 0x7fb87c0b3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:20.022 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.021+0000 7fb87b400640 1 -- 172.21.15.152:0/2899187133 >> v2:172.21.15.175:3300/0 conn(0x7fb87c06ee00 msgr2=0x7fb87c06f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:20.022 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.021+0000 7fb87b400640 1 --2- 172.21.15.152:0/2899187133 >> v2:172.21.15.175:3300/0 conn(0x7fb87c06ee00 0x7fb87c06f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:20.022 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.022+0000 7fb87b400640 1 -- 172.21.15.152:0/2899187133 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb87c0cfdf0 con 0x7fb87c15fbf0 2024-04-20T11:34:20.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.023+0000 7fb87b400640 1 --2- 172.21.15.152:0/2899187133 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb87c15fbf0 0x7fb87c15c390 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7fb87c06ffc0 tx=0x7fb86802b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=754831a058453a71 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:20.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.023+0000 7fb87aa00640 1 -- 172.21.15.152:0/2899187133 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb868035020 con 0x7fb87c15fbf0 2024-04-20T11:34:20.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.023+0000 7fb87aa00640 1 -- 172.21.15.152:0/2899187133 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb86802d040 con 0x7fb87c15fbf0 2024-04-20T11:34:20.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.023+0000 7fb87aa00640 1 -- 172.21.15.152:0/2899187133 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb868031a80 con 0x7fb87c15fbf0 2024-04-20T11:34:20.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.024+0000 7fb881400640 1 -- 172.21.15.152:0/2899187133 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb87c15fbf0 msgr2=0x7fb87c15c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:20.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.024+0000 7fb881400640 1 --2- 172.21.15.152:0/2899187133 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb87c15fbf0 0x7fb87c15c390 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7fb87c06ffc0 tx=0x7fb86802b900 comp rx=0 tx=0).stop 2024-04-20T11:34:20.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.024+0000 7fb881400640 1 -- 172.21.15.152:0/2899187133 shutdown_connections 2024-04-20T11:34:20.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.024+0000 7fb881400640 1 --2- 172.21.15.152:0/2899187133 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb87c0cf110 0x7fb87c0b3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:20.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.024+0000 7fb881400640 1 --2- 172.21.15.152:0/2899187133 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb87c15fbf0 0x7fb87c15c390 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:20.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.024+0000 7fb881400640 1 --2- 172.21.15.152:0/2899187133 >> v2:172.21.15.175:3300/0 conn(0x7fb87c06ee00 0x7fb87c06f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:20.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.024+0000 7fb881400640 1 -- 172.21.15.152:0/2899187133 >> 172.21.15.152:0/2899187133 conn(0x7fb87c067ae0 msgr2=0x7fb87c06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:20.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.024+0000 7fb881400640 1 -- 172.21.15.152:0/2899187133 shutdown_connections 2024-04-20T11:34:20.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.024+0000 7fb881400640 1 -- 172.21.15.152:0/2899187133 wait complete. 2024-04-20T11:34:20.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.026+0000 7fb881400640 1 Processor -- start 2024-04-20T11:34:20.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.026+0000 7fb881400640 1 -- start start 2024-04-20T11:34:20.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.027+0000 7fb881400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb87c06ee00 0x7fb87c1f4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:20.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.027+0000 7fb881400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb87c0cf110 0x7fb87c15cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:20.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.027+0000 7fb881400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb87c15fbf0 0x7fb87c13dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:20.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.027+0000 7fb881400640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fb87c15c8d0 con 0x7fb87c06ee00 2024-04-20T11:34:20.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.027+0000 7fb881400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb87c15be70 con 0x7fb87c15fbf0 2024-04-20T11:34:20.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.027+0000 7fb881400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb87c144970 con 0x7fb87c0cf110 2024-04-20T11:34:20.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.028+0000 7fb87be00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb87c06ee00 0x7fb87c1f4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:20.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.028+0000 7fb87be00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb87c06ee00 0x7fb87c1f4fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52248/0 (socket says 172.21.15.152:52248) 2024-04-20T11:34:20.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.028+0000 7fb87be00640 1 -- 172.21.15.152:0/1480297149 learned_addr learned my addr 172.21.15.152:0/1480297149 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:20.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.028+0000 7fb87be00640 1 -- 172.21.15.152:0/1480297149 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb87c0cf110 msgr2=0x7fb87c15cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:20.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.028+0000 7fb87be00640 1 --2- 172.21.15.152:0/1480297149 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb87c0cf110 0x7fb87c15cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:20.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.028+0000 7fb87be00640 1 -- 172.21.15.152:0/1480297149 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb87c15fbf0 msgr2=0x7fb87c13dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:20.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.028+0000 7fb87be00640 1 --2- 172.21.15.152:0/1480297149 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb87c15fbf0 0x7fb87c13dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:20.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.028+0000 7fb87be00640 1 -- 172.21.15.152:0/1480297149 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb87c15ed30 con 0x7fb87c06ee00 2024-04-20T11:34:20.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.028+0000 7fb87be00640 1 --2- 172.21.15.152:0/1480297149 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb87c06ee00 0x7fb87c1f4fc0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7fb86c00b410 tx=0x7fb86c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:20.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.029+0000 7fb873e00640 1 -- 172.21.15.152:0/1480297149 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb86c015020 con 0x7fb87c06ee00 2024-04-20T11:34:20.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.029+0000 7fb873e00640 1 -- 172.21.15.152:0/1480297149 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb86c011850 con 0x7fb87c06ee00 2024-04-20T11:34:20.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.029+0000 7fb873e00640 1 -- 172.21.15.152:0/1480297149 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb86c011c40 con 0x7fb87c06ee00 2024-04-20T11:34:20.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.029+0000 7fb881400640 1 -- 172.21.15.152:0/1480297149 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb87c1f44e0 con 0x7fb87c06ee00 2024-04-20T11:34:20.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.029+0000 7fb881400640 1 -- 172.21.15.152:0/1480297149 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb87c1f4aa0 con 0x7fb87c06ee00 2024-04-20T11:34:20.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.030+0000 7fb881400640 1 -- 172.21.15.152:0/1480297149 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb87c075e60 con 0x7fb87c06ee00 2024-04-20T11:34:20.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.031+0000 7fb873e00640 1 -- 172.21.15.152:0/1480297149 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 19) ==== 91397+0+0 (secure 0 0 0) 0x7fb86c018020 con 0x7fb87c06ee00 2024-04-20T11:34:20.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.032+0000 7fb873e00640 1 --2- 172.21.15.152:0/1480297149 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb85c06cce0 0x7fb85c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:20.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.032+0000 7fb873e00640 1 -- 172.21.15.152:0/1480297149 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 5737+0+0 (secure 0 0 0) 0x7fb86c08c280 con 0x7fb87c06ee00 2024-04-20T11:34:20.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.032+0000 7fb87b400640 1 --2- 172.21.15.152:0/1480297149 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb85c06cce0 0x7fb85c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:20.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.034+0000 7fb87b400640 1 --2- 172.21.15.152:0/1480297149 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb85c06cce0 0x7fb85c06f1a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb868036000 tx=0x7fb868030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:20.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.038+0000 7fb873e00640 1 -- 172.21.15.152:0/1480297149 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fb86c01f020 con 0x7fb87c06ee00 2024-04-20T11:34:20.222 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.222+0000 7fb881400640 1 -- 172.21.15.152:0/1480297149 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb87c0b3490 con 0x7fb87c06ee00 2024-04-20T11:34:20.222 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.223+0000 7fb873e00640 1 -- 172.21.15.152:0/1480297149 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v106) ==== 104+0+11 (secure 0 0 0) 0x7fb86c0172c0 con 0x7fb87c06ee00 2024-04-20T11:34:20.223 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 32 2024-04-20T11:34:20.225 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.225+0000 7fb881400640 1 -- 172.21.15.152:0/1480297149 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb85c06cce0 msgr2=0x7fb85c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:20.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.225+0000 7fb881400640 1 --2- 172.21.15.152:0/1480297149 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb85c06cce0 0x7fb85c06f1a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb868036000 tx=0x7fb868030000 comp rx=0 tx=0).stop 2024-04-20T11:34:20.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.225+0000 7fb881400640 1 -- 172.21.15.152:0/1480297149 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb87c06ee00 msgr2=0x7fb87c1f4fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:20.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.225+0000 7fb881400640 1 --2- 172.21.15.152:0/1480297149 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb87c06ee00 0x7fb87c1f4fc0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7fb86c00b410 tx=0x7fb86c00b8d0 comp rx=0 tx=0).stop 2024-04-20T11:34:20.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.226+0000 7fb881400640 1 -- 172.21.15.152:0/1480297149 shutdown_connections 2024-04-20T11:34:20.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.226+0000 7fb881400640 1 --2- 172.21.15.152:0/1480297149 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb85c06cce0 0x7fb85c06f1a0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:20.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.226+0000 7fb881400640 1 --2- 172.21.15.152:0/1480297149 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb87c15fbf0 0x7fb87c13dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:20.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.226+0000 7fb881400640 1 --2- 172.21.15.152:0/1480297149 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb87c0cf110 0x7fb87c15cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:20.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.226+0000 7fb881400640 1 --2- 172.21.15.152:0/1480297149 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb87c06ee00 0x7fb87c1f4fc0 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:20.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.226+0000 7fb881400640 1 -- 172.21.15.152:0/1480297149 >> 172.21.15.152:0/1480297149 conn(0x7fb87c067ae0 msgr2=0x7fb87c06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:20.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.226+0000 7fb881400640 1 -- 172.21.15.152:0/1480297149 shutdown_connections 2024-04-20T11:34:20.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.226+0000 7fb881400640 1 -- 172.21.15.152:0/1480297149 wait complete. 2024-04-20T11:34:20.244 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T11:34:20.436 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.436+0000 7fb9e2000640 1 Processor -- start 2024-04-20T11:34:20.436 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.436+0000 7fb9e2000640 1 -- start start 2024-04-20T11:34:20.438 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.438+0000 7fb9e2000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fb9dc15b510 0x7fb9dc15fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:20.438 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.438+0000 7fb9e2000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb9dc0b2880 0x7fb9dc0b3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:20.438 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.438+0000 7fb9e2000640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9dc15f1c0 con 0x7fb9dc0cfc40 2024-04-20T11:34:20.438 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.438+0000 7fb9e2000640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fb9dc06f420 con 0x7fb9dc15b510 2024-04-20T11:34:20.438 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.438+0000 7fb9e2000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb9dc0b4430 con 0x7fb9dc0b2880 2024-04-20T11:34:20.438 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.438+0000 7fb9db400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fb9dc15b510 0x7fb9dc15fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:20.438 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.438+0000 7fb9db400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fb9dc15b510 0x7fb9dc15fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52262/0 (socket says 172.21.15.152:52262) 2024-04-20T11:34:20.438 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.438+0000 7fb9db400640 1 -- 172.21.15.152:0/2933367940 learned_addr learned my addr 172.21.15.152:0/2933367940 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:20.439 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.439+0000 7fb9db400640 1 -- 172.21.15.152:0/2933367940 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb9dc0b2880 msgr2=0x7fb9dc0b3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:20.439 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.439+0000 7fb9db400640 1 --2- 172.21.15.152:0/2933367940 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb9dc0b2880 0x7fb9dc0b3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:20.439 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.439+0000 7fb9db400640 1 -- 172.21.15.152:0/2933367940 >> v1:172.21.15.175:6789/0 conn(0x7fb9dc0cfc40 legacy=0x7fb9dc0b2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:20.439 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.439+0000 7fb9db400640 1 -- 172.21.15.152:0/2933367940 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9dc06f670 con 0x7fb9dc15b510 2024-04-20T11:34:20.439 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.440+0000 7fb9db400640 1 --2- 172.21.15.152:0/2933367940 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb9dc15b510 0x7fb9dc15fc30 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7fb9cc009e50 tx=0x7fb9cc02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1f145afa66a5b11 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:20.440 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.440+0000 7fb9daa00640 1 -- 172.21.15.152:0/2933367940 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9cc02ca70 con 0x7fb9dc15b510 2024-04-20T11:34:20.440 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.440+0000 7fb9daa00640 1 -- 172.21.15.152:0/2933367940 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9cc02cc10 con 0x7fb9dc15b510 2024-04-20T11:34:20.440 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.440+0000 7fb9daa00640 1 -- 172.21.15.152:0/2933367940 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9cc0343f0 con 0x7fb9dc15b510 2024-04-20T11:34:20.441 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.441+0000 7fb9e2000640 1 -- 172.21.15.152:0/2933367940 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb9dc15b510 msgr2=0x7fb9dc15fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:20.441 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.441+0000 7fb9e2000640 1 --2- 172.21.15.152:0/2933367940 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb9dc15b510 0x7fb9dc15fc30 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7fb9cc009e50 tx=0x7fb9cc02bed0 comp rx=0 tx=0).stop 2024-04-20T11:34:20.441 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.441+0000 7fb9e2000640 1 -- 172.21.15.152:0/2933367940 shutdown_connections 2024-04-20T11:34:20.441 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.441+0000 7fb9e2000640 1 --2- 172.21.15.152:0/2933367940 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb9dc0b2880 0x7fb9dc0b3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:20.441 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.441+0000 7fb9e2000640 1 --2- 172.21.15.152:0/2933367940 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb9dc15b510 0x7fb9dc15fc30 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:20.441 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.441+0000 7fb9e2000640 1 -- 172.21.15.152:0/2933367940 >> 172.21.15.152:0/2933367940 conn(0x7fb9dc06b980 msgr2=0x7fb9dc06bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:20.441 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.441+0000 7fb9e2000640 1 -- 172.21.15.152:0/2933367940 shutdown_connections 2024-04-20T11:34:20.441 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.441+0000 7fb9e2000640 1 -- 172.21.15.152:0/2933367940 wait complete. 2024-04-20T11:34:20.442 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.443+0000 7fb9e2000640 1 Processor -- start 2024-04-20T11:34:20.442 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.443+0000 7fb9e2000640 1 -- start start 2024-04-20T11:34:20.444 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.444+0000 7fb9e2000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9dc0b2880 0x7fb9dc1440e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:20.445 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.444+0000 7fb9e2000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb9dc0cfc40 0x7fb9dc15ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:20.445 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.444+0000 7fb9e2000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb9dc15b510 0x7fb9dc153550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:20.445 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.444+0000 7fb9e2000640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fb9dc15b8f0 con 0x7fb9dc15b510 2024-04-20T11:34:20.445 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.444+0000 7fb9e2000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb9dc15f710 con 0x7fb9dc0b2880 2024-04-20T11:34:20.445 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.444+0000 7fb9e2000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb9dc06f950 con 0x7fb9dc0cfc40 2024-04-20T11:34:20.445 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.444+0000 7fb9dbe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb9dc15b510 0x7fb9dc153550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:20.445 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.444+0000 7fb9dbe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb9dc15b510 0x7fb9dc153550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52278/0 (socket says 172.21.15.152:52278) 2024-04-20T11:34:20.445 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.444+0000 7fb9dbe00640 1 -- 172.21.15.152:0/2380748373 learned_addr learned my addr 172.21.15.152:0/2380748373 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:20.445 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.444+0000 7fb9db400640 1 --2- 172.21.15.152:0/2380748373 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9dc0b2880 0x7fb9dc1440e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:20.445 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.444+0000 7fb9d2a00640 1 --2- 172.21.15.152:0/2380748373 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb9dc0cfc40 0x7fb9dc15ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:20.445 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.444+0000 7fb9dbe00640 1 -- 172.21.15.152:0/2380748373 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb9dc0cfc40 msgr2=0x7fb9dc15ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:20.445 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.444+0000 7fb9dbe00640 1 --2- 172.21.15.152:0/2380748373 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb9dc0cfc40 0x7fb9dc15ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:20.445 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.444+0000 7fb9dbe00640 1 -- 172.21.15.152:0/2380748373 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9dc0b2880 msgr2=0x7fb9dc1440e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:20.445 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.444+0000 7fb9dbe00640 1 --2- 172.21.15.152:0/2380748373 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9dc0b2880 0x7fb9dc1440e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:20.445 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.445+0000 7fb9dbe00640 1 -- 172.21.15.152:0/2380748373 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9dc15c450 con 0x7fb9dc15b510 2024-04-20T11:34:20.446 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.445+0000 7fb9dbe00640 1 --2- 172.21.15.152:0/2380748373 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb9dc15b510 0x7fb9dc153550 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7fb9d4009fd0 tx=0x7fb9d400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:20.446 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.445+0000 7fb9d3e00640 1 -- 172.21.15.152:0/2380748373 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9d4011830 con 0x7fb9dc15b510 2024-04-20T11:34:20.446 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.445+0000 7fb9d3e00640 1 -- 172.21.15.152:0/2380748373 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9d4016070 con 0x7fb9dc15b510 2024-04-20T11:34:20.446 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.445+0000 7fb9d3e00640 1 -- 172.21.15.152:0/2380748373 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9d4011bf0 con 0x7fb9dc15b510 2024-04-20T11:34:20.446 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.445+0000 7fb9e2000640 1 -- 172.21.15.152:0/2380748373 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9dc15c730 con 0x7fb9dc15b510 2024-04-20T11:34:20.446 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.445+0000 7fb9e2000640 1 -- 172.21.15.152:0/2380748373 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9dc15ccf0 con 0x7fb9dc15b510 2024-04-20T11:34:20.446 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.446+0000 7fb9e2000640 1 -- 172.21.15.152:0/2380748373 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9dc0689f0 con 0x7fb9dc15b510 2024-04-20T11:34:20.447 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.447+0000 7fb9d3e00640 1 -- 172.21.15.152:0/2380748373 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 19) ==== 91397+0+0 (secure 0 0 0) 0x7fb9d401b460 con 0x7fb9dc15b510 2024-04-20T11:34:20.448 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.448+0000 7fb9d3e00640 1 --2- 172.21.15.152:0/2380748373 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb9b806cce0 0x7fb9b806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:20.448 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.448+0000 7fb9d3e00640 1 -- 172.21.15.152:0/2380748373 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 5737+0+0 (secure 0 0 0) 0x7fb9d408c0b0 con 0x7fb9dc15b510 2024-04-20T11:34:20.448 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.449+0000 7fb9db400640 1 --2- 172.21.15.152:0/2380748373 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb9b806cce0 0x7fb9b806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:20.449 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.450+0000 7fb9db400640 1 --2- 172.21.15.152:0/2380748373 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb9b806cce0 0x7fb9b806f1a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb9cc035000 tx=0x7fb9cc030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:20.452 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.453+0000 7fb9d3e00640 1 -- 172.21.15.152:0/2380748373 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fb9d4059ea0 con 0x7fb9dc15b510 2024-04-20T11:34:20.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.654+0000 7fb9e2000640 1 -- 172.21.15.152:0/2380748373 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb9dc06ecf0 con 0x7fb9dc15b510 2024-04-20T11:34:20.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.655+0000 7fb9d3e00640 1 -- 172.21.15.152:0/2380748373 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v106) ==== 100+0+11 (secure 0 0 0) 0x7fb9d40597a0 con 0x7fb9dc15b510 2024-04-20T11:34:20.655 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 64 2024-04-20T11:34:20.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.658+0000 7fb9e2000640 1 -- 172.21.15.152:0/2380748373 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb9b806cce0 msgr2=0x7fb9b806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:20.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.658+0000 7fb9e2000640 1 --2- 172.21.15.152:0/2380748373 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb9b806cce0 0x7fb9b806f1a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb9cc035000 tx=0x7fb9cc030000 comp rx=0 tx=0).stop 2024-04-20T11:34:20.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.658+0000 7fb9e2000640 1 -- 172.21.15.152:0/2380748373 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb9dc15b510 msgr2=0x7fb9dc153550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:20.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.658+0000 7fb9e2000640 1 --2- 172.21.15.152:0/2380748373 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb9dc15b510 0x7fb9dc153550 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7fb9d4009fd0 tx=0x7fb9d400b8e0 comp rx=0 tx=0).stop 2024-04-20T11:34:20.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.658+0000 7fb9e2000640 1 -- 172.21.15.152:0/2380748373 shutdown_connections 2024-04-20T11:34:20.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.658+0000 7fb9e2000640 1 --2- 172.21.15.152:0/2380748373 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb9b806cce0 0x7fb9b806f1a0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:20.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.658+0000 7fb9e2000640 1 --2- 172.21.15.152:0/2380748373 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb9dc15b510 0x7fb9dc153550 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:20.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.658+0000 7fb9e2000640 1 --2- 172.21.15.152:0/2380748373 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb9dc0cfc40 0x7fb9dc15ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:20.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.658+0000 7fb9e2000640 1 --2- 172.21.15.152:0/2380748373 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9dc0b2880 0x7fb9dc1440e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:20.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.658+0000 7fb9e2000640 1 -- 172.21.15.152:0/2380748373 >> 172.21.15.152:0/2380748373 conn(0x7fb9dc06b980 msgr2=0x7fb9dc0cef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:20.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.658+0000 7fb9e2000640 1 -- 172.21.15.152:0/2380748373 shutdown_connections 2024-04-20T11:34:20.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.658+0000 7fb9e2000640 1 -- 172.21.15.152:0/2380748373 wait complete. 2024-04-20T11:34:20.677 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:34:20.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.865+0000 7fd035800640 1 Processor -- start 2024-04-20T11:34:20.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.865+0000 7fd035800640 1 -- start start 2024-04-20T11:34:20.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.867+0000 7fd035800640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fd030145f60 0x7fd03006ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:20.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.867+0000 7fd035800640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd030146340 con 0x7fd0300cd0d0 2024-04-20T11:34:20.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.867+0000 7fd035800640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd03006f3c0 con 0x7fd0300726f0 2024-04-20T11:34:20.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.867+0000 7fd035800640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fd03006e9f0 con 0x7fd030145f60 2024-04-20T11:34:20.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.867+0000 7fd026000640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fd0300cd0d0 0x7fd030071fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:40246/0 (socket says 172.21.15.152:40246) 2024-04-20T11:34:20.867 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.867+0000 7fd026000640 1 -- 172.21.15.152:0/807845458 learned_addr learned my addr 172.21.15.152:0/807845458 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:34:20.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.867+0000 7fd02e000640 1 -- 172.21.15.152:0/807845458 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2595055931 0 0) 0x7fd030146340 con 0x7fd0300cd0d0 2024-04-20T11:34:20.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.868+0000 7fd02ea00640 1 --2- 172.21.15.152:0/807845458 >> v2:172.21.15.175:3300/0 conn(0x7fd030145f60 0x7fd03006ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:20.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.868+0000 7fd02e000640 1 -- 172.21.15.152:0/807845458 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd014003520 con 0x7fd0300cd0d0 2024-04-20T11:34:20.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.868+0000 7fd02e000640 1 -- 172.21.15.152:0/807845458 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4053260238 0 0) 0x7fd014003520 con 0x7fd0300cd0d0 2024-04-20T11:34:20.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.868+0000 7fd02e000640 1 -- 172.21.15.152:0/807845458 >> v2:172.21.15.175:3300/0 conn(0x7fd030145f60 msgr2=0x7fd03006ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:20.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.868+0000 7fd02e000640 1 --2- 172.21.15.152:0/807845458 >> v2:172.21.15.175:3300/0 conn(0x7fd030145f60 0x7fd03006ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:20.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.868+0000 7fd02e000640 1 -- 172.21.15.152:0/807845458 >> v1:172.21.15.175:6789/0 conn(0x7fd0300726f0 legacy=0x7fd0300cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:20.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.868+0000 7fd02e000640 1 -- 172.21.15.152:0/807845458 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd030072d50 con 0x7fd0300cd0d0 2024-04-20T11:34:20.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.868+0000 7fd02e000640 1 -- 172.21.15.152:0/807845458 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7fd0180019f0 con 0x7fd0300cd0d0 2024-04-20T11:34:20.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.868+0000 7fd02e000640 1 -- 172.21.15.152:0/807845458 >> v1:172.21.15.152:6789/0 conn(0x7fd0300cd0d0 legacy=0x7fd030071fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:20.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.868+0000 7fd02e000640 1 --2- 172.21.15.152:0/807845458 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd014003a20 0x7fd014003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:20.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.868+0000 7fd02e000640 1 --2- 172.21.15.152:0/807845458 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd014004800 0x7fd014004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:20.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.868+0000 7fd02e000640 1 --2- 172.21.15.152:0/807845458 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd0140051c0 0x7fd01400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:20.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.868+0000 7fd02e000640 1 -- 172.21.15.152:0/807845458 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fd03006e9f0 con 0x7fd014004800 2024-04-20T11:34:20.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.868+0000 7fd02e000640 1 -- 172.21.15.152:0/807845458 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd01400c0f0 con 0x7fd014003a20 2024-04-20T11:34:20.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.868+0000 7fd02e000640 1 -- 172.21.15.152:0/807845458 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd01400c2f0 con 0x7fd0140051c0 2024-04-20T11:34:20.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.868+0000 7fd026000640 1 --2- 172.21.15.152:0/807845458 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd014004800 0x7fd014004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:20.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.869+0000 7fd02f400640 1 --2- 172.21.15.152:0/807845458 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd0140051c0 0x7fd01400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:20.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.869+0000 7fd026000640 1 -- 172.21.15.152:0/807845458 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd0140051c0 msgr2=0x7fd01400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:20.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.869+0000 7fd026000640 1 --2- 172.21.15.152:0/807845458 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd0140051c0 0x7fd01400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:20.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.869+0000 7fd026000640 1 -- 172.21.15.152:0/807845458 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd014003a20 msgr2=0x7fd014003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:20.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.869+0000 7fd02ea00640 1 --2- 172.21.15.152:0/807845458 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd014003a20 0x7fd014003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:20.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.869+0000 7fd026000640 1 --2- 172.21.15.152:0/807845458 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd014003a20 0x7fd014003e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:20.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.869+0000 7fd026000640 1 -- 172.21.15.152:0/807845458 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd014003520 con 0x7fd014004800 2024-04-20T11:34:20.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.869+0000 7fd02f400640 1 --2- 172.21.15.152:0/807845458 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd0140051c0 0x7fd01400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:34:20.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.869+0000 7fd02f400640 1 -- 172.21.15.152:0/807845458 reap_dead start 2024-04-20T11:34:20.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.870+0000 7fd026000640 1 --2- 172.21.15.152:0/807845458 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd014004800 0x7fd014004c80 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7fd01800a840 tx=0x7fd014005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=715f76a51f50773b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:20.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.870+0000 7fd02e000640 1 -- 172.21.15.152:0/807845458 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd01800a460 con 0x7fd014004800 2024-04-20T11:34:20.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.870+0000 7fd02e000640 1 -- 172.21.15.152:0/807845458 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd018034650 con 0x7fd014004800 2024-04-20T11:34:20.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.870+0000 7fd02e000640 1 -- 172.21.15.152:0/807845458 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd018034a80 con 0x7fd014004800 2024-04-20T11:34:20.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.871+0000 7fd035800640 1 -- 172.21.15.152:0/807845458 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd014004800 msgr2=0x7fd014004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:20.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.871+0000 7fd035800640 1 --2- 172.21.15.152:0/807845458 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd014004800 0x7fd014004c80 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7fd01800a840 tx=0x7fd014005830 comp rx=0 tx=0).stop 2024-04-20T11:34:20.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.871+0000 7fd035800640 1 -- 172.21.15.152:0/807845458 shutdown_connections 2024-04-20T11:34:20.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.871+0000 7fd035800640 1 --2- 172.21.15.152:0/807845458 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd014004800 0x7fd014004c80 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:20.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.871+0000 7fd035800640 1 -- 172.21.15.152:0/807845458 >> 172.21.15.152:0/807845458 conn(0x7fd030067cf0 msgr2=0x7fd03006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:20.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.871+0000 7fd035800640 1 -- 172.21.15.152:0/807845458 shutdown_connections 2024-04-20T11:34:20.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.871+0000 7fd035800640 1 -- 172.21.15.152:0/807845458 wait complete. 2024-04-20T11:34:20.873 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.873+0000 7fd035800640 1 Processor -- start 2024-04-20T11:34:20.873 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.873+0000 7fd035800640 1 -- start start 2024-04-20T11:34:20.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.874+0000 7fd035800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd014004800 0x7fd030159fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:20.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.874+0000 7fd035800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd0300aa5d0 0x7fd0300a94a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:20.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.874+0000 7fd035800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd0300aaa30 0x7fd0300a99e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:20.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.874+0000 7fd035800640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fd03006e680 con 0x7fd0300aaa30 2024-04-20T11:34:20.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.874+0000 7fd035800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd03006e3a0 con 0x7fd0300aa5d0 2024-04-20T11:34:20.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.874+0000 7fd035800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd03006ee80 con 0x7fd014004800 2024-04-20T11:34:20.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.874+0000 7fd02f400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd0300aaa30 0x7fd0300a99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:20.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.874+0000 7fd02f400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd0300aaa30 0x7fd0300a99e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52310/0 (socket says 172.21.15.152:52310) 2024-04-20T11:34:20.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.874+0000 7fd02f400640 1 -- 172.21.15.152:0/1617087392 learned_addr learned my addr 172.21.15.152:0/1617087392 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:20.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.875+0000 7fd02ea00640 1 --2- 172.21.15.152:0/1617087392 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd014004800 0x7fd030159fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:20.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.875+0000 7fd026000640 1 --2- 172.21.15.152:0/1617087392 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd0300aa5d0 0x7fd0300a94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:20.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.875+0000 7fd02f400640 1 -- 172.21.15.152:0/1617087392 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd014004800 msgr2=0x7fd030159fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:20.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.875+0000 7fd02f400640 1 --2- 172.21.15.152:0/1617087392 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd014004800 0x7fd030159fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:20.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.875+0000 7fd02f400640 1 -- 172.21.15.152:0/1617087392 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd0300aa5d0 msgr2=0x7fd0300a94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:20.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.875+0000 7fd02f400640 1 --2- 172.21.15.152:0/1617087392 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd0300aa5d0 0x7fd0300a94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:20.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.875+0000 7fd02f400640 1 -- 172.21.15.152:0/1617087392 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd03015a510 con 0x7fd0300aaa30 2024-04-20T11:34:20.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.875+0000 7fd02f400640 1 --2- 172.21.15.152:0/1617087392 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd0300aaa30 0x7fd0300a99e0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7fd02800d400 tx=0x7fd02800d8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:20.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.875+0000 7fd027400640 1 -- 172.21.15.152:0/1617087392 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd028018070 con 0x7fd0300aaa30 2024-04-20T11:34:20.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.876+0000 7fd027400640 1 -- 172.21.15.152:0/1617087392 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0280035f0 con 0x7fd0300aaa30 2024-04-20T11:34:20.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.876+0000 7fd027400640 1 -- 172.21.15.152:0/1617087392 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0280039e0 con 0x7fd0300aaa30 2024-04-20T11:34:20.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.876+0000 7fd035800640 1 -- 172.21.15.152:0/1617087392 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd03015ce10 con 0x7fd0300aaa30 2024-04-20T11:34:20.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.876+0000 7fd035800640 1 -- 172.21.15.152:0/1617087392 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd03015d3b0 con 0x7fd0300aaa30 2024-04-20T11:34:20.877 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.877+0000 7fd035800640 1 -- 172.21.15.152:0/1617087392 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0300689f0 con 0x7fd0300aaa30 2024-04-20T11:34:20.877 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.878+0000 7fd027400640 1 -- 172.21.15.152:0/1617087392 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 19) ==== 91397+0+0 (secure 0 0 0) 0x7fd02801b070 con 0x7fd0300aaa30 2024-04-20T11:34:20.878 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.879+0000 7fd027400640 1 --2- 172.21.15.152:0/1617087392 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd00c06cce0 0x7fd00c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:20.878 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.879+0000 7fd027400640 1 -- 172.21.15.152:0/1617087392 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 5737+0+0 (secure 0 0 0) 0x7fd02808b850 con 0x7fd0300aaa30 2024-04-20T11:34:20.879 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.880+0000 7fd02ea00640 1 --2- 172.21.15.152:0/1617087392 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd00c06cce0 0x7fd00c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:20.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.881+0000 7fd02ea00640 1 --2- 172.21.15.152:0/1617087392 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd00c06cce0 0x7fd00c06f1a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd020009a90 tx=0x7fd020008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:20.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:20.883+0000 7fd027400640 1 -- 172.21.15.152:0/1617087392 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fd028059570 con 0x7fd0300aaa30 2024-04-20T11:34:21.128 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.128+0000 7fd035800640 1 -- 172.21.15.152:0/1617087392 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd0300b54c0 con 0x7fd0300aaa30 2024-04-20T11:34:21.129 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.129+0000 7fd027400640 1 -- 172.21.15.152:0/1617087392 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+5210 (secure 0 0 0) 0x7fd028058e70 con 0x7fd0300aaa30 2024-04-20T11:34:21.129 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:34:21.129 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":67,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5648,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6834/387198089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":387198089},{"type":"v1","addr":"172.21.15.152:6834","nonce":387198089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":6154,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/1674013715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1674013715},{"type":"v1","addr":"172.21.15.175:6833","nonce":1674013715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":6157,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/3898238490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3898238490},{"type":"v1","addr":"172.21.15.175:6835","nonce":3898238490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T11:33:27.277882+0000","modified":"2024-04-20T11:33:30.342429+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7794},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7794":{"gid":7794,"name":"a","rank":0,"incarnation":62,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6835/3582312712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6833","nonce":3582312712},{"type":"v1","addr":"172.21.15.152:6835","nonce":3582312712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7794,"qdb_cluster":[7794]},"id":8}]} 2024-04-20T11:34:21.130 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 67 2024-04-20T11:34:21.132 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.132+0000 7fd035800640 1 -- 172.21.15.152:0/1617087392 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd00c06cce0 msgr2=0x7fd00c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:21.132 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.132+0000 7fd035800640 1 --2- 172.21.15.152:0/1617087392 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd00c06cce0 0x7fd00c06f1a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd020009a90 tx=0x7fd020008040 comp rx=0 tx=0).stop 2024-04-20T11:34:21.132 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.132+0000 7fd035800640 1 -- 172.21.15.152:0/1617087392 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd0300aaa30 msgr2=0x7fd0300a99e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:21.132 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.132+0000 7fd035800640 1 --2- 172.21.15.152:0/1617087392 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd0300aaa30 0x7fd0300a99e0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7fd02800d400 tx=0x7fd02800d8c0 comp rx=0 tx=0).stop 2024-04-20T11:34:21.132 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.132+0000 7fd035800640 1 -- 172.21.15.152:0/1617087392 shutdown_connections 2024-04-20T11:34:21.132 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.132+0000 7fd035800640 1 --2- 172.21.15.152:0/1617087392 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd00c06cce0 0x7fd00c06f1a0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:21.132 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.132+0000 7fd035800640 1 --2- 172.21.15.152:0/1617087392 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd0300aaa30 0x7fd0300a99e0 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:21.132 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.132+0000 7fd035800640 1 --2- 172.21.15.152:0/1617087392 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd0300aa5d0 0x7fd0300a94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:21.133 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.132+0000 7fd035800640 1 --2- 172.21.15.152:0/1617087392 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd014004800 0x7fd030159fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:21.133 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.132+0000 7fd035800640 1 -- 172.21.15.152:0/1617087392 >> 172.21.15.152:0/1617087392 conn(0x7fd030067cf0 msgr2=0x7fd030160170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:21.133 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.132+0000 7fd035800640 1 -- 172.21.15.152:0/1617087392 shutdown_connections 2024-04-20T11:34:21.133 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.132+0000 7fd035800640 1 -- 172.21.15.152:0/1617087392 wait complete. 2024-04-20T11:34:21.150 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:34:21.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.362+0000 7fa2e2c00640 1 Processor -- start 2024-04-20T11:34:21.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.362+0000 7fa2e2c00640 1 -- start start 2024-04-20T11:34:21.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.363+0000 7fa2e2c00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fa2dc15eb80 0x7fa2dc0b79a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:21.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.363+0000 7fa2e2c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa2dc0ba520 0x7fa2dc0d28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:21.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.363+0000 7fa2e2c00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2dc0b8330 con 0x7fa2dc0d2e30 2024-04-20T11:34:21.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.363+0000 7fa2e2c00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fa2dc0b8610 con 0x7fa2dc15eb80 2024-04-20T11:34:21.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.363+0000 7fa2e2c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa2dc0b88f0 con 0x7fa2dc0ba520 2024-04-20T11:34:21.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.363+0000 7fa2e0c00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fa2dc0d2e30 0x7fa2dc0d1810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:40250/0 (socket says 172.21.15.152:40250) 2024-04-20T11:34:21.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.363+0000 7fa2e0c00640 1 -- 172.21.15.152:0/2413240987 learned_addr learned my addr 172.21.15.152:0/2413240987 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:34:21.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.364+0000 7fa2dbe00640 1 --2- 172.21.15.152:0/2413240987 >> v2:172.21.15.175:3300/0 conn(0x7fa2dc15eb80 0x7fa2dc0b79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:21.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.364+0000 7fa2db400640 1 --2- 172.21.15.152:0/2413240987 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa2dc0ba520 0x7fa2dc0d28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:21.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.364+0000 7fa2daa00640 1 -- 172.21.15.152:0/2413240987 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2534274306 0 0) 0x7fa2dc0b8330 con 0x7fa2dc0d2e30 2024-04-20T11:34:21.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.364+0000 7fa2daa00640 1 -- 172.21.15.152:0/2413240987 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa2c0003520 con 0x7fa2dc0d2e30 2024-04-20T11:34:21.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.364+0000 7fa2daa00640 1 -- 172.21.15.152:0/2413240987 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2061817782 0 0) 0x7fa2c0003520 con 0x7fa2dc0d2e30 2024-04-20T11:34:21.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.364+0000 7fa2daa00640 1 -- 172.21.15.152:0/2413240987 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa2dc0ba520 msgr2=0x7fa2dc0d28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:21.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.364+0000 7fa2daa00640 1 --2- 172.21.15.152:0/2413240987 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa2dc0ba520 0x7fa2dc0d28f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:21.366 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.364+0000 7fa2daa00640 1 -- 172.21.15.152:0/2413240987 >> v2:172.21.15.175:3300/0 conn(0x7fa2dc15eb80 msgr2=0x7fa2dc0b79a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:21.366 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.364+0000 7fa2daa00640 1 --2- 172.21.15.152:0/2413240987 >> v2:172.21.15.175:3300/0 conn(0x7fa2dc15eb80 0x7fa2dc0b79a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:21.366 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.364+0000 7fa2daa00640 1 -- 172.21.15.152:0/2413240987 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2dc0b8bd0 con 0x7fa2dc0d2e30 2024-04-20T11:34:21.366 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.364+0000 7fa2daa00640 1 -- 172.21.15.152:0/2413240987 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7fa2d40019f0 con 0x7fa2dc0d2e30 2024-04-20T11:34:21.366 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.364+0000 7fa2daa00640 1 -- 172.21.15.152:0/2413240987 >> v1:172.21.15.152:6789/0 conn(0x7fa2dc0d2e30 legacy=0x7fa2dc0d1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:21.366 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.364+0000 7fa2daa00640 1 --2- 172.21.15.152:0/2413240987 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa2dc0ba520 0x7fa2c0003a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:21.366 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.365+0000 7fa2daa00640 1 --2- 172.21.15.152:0/2413240987 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa2c0003fa0 0x7fa2c0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:21.366 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.365+0000 7fa2daa00640 1 --2- 172.21.15.152:0/2413240987 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2c0004d40 0x7fa2c00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:21.366 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.365+0000 7fa2daa00640 1 -- 172.21.15.152:0/2413240987 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fa2dc0b8610 con 0x7fa2c0003fa0 2024-04-20T11:34:21.366 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.365+0000 7fa2daa00640 1 -- 172.21.15.152:0/2413240987 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa2dc0b88f0 con 0x7fa2c0004d40 2024-04-20T11:34:21.366 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.365+0000 7fa2daa00640 1 -- 172.21.15.152:0/2413240987 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa2c000ba70 con 0x7fa2dc0ba520 2024-04-20T11:34:21.366 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.365+0000 7fa2db400640 1 --2- 172.21.15.152:0/2413240987 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa2c0003fa0 0x7fa2c0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:21.366 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.365+0000 7fa2dbe00640 1 --2- 172.21.15.152:0/2413240987 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa2dc0ba520 0x7fa2c0003a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:21.366 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.365+0000 7fa2db400640 1 -- 172.21.15.152:0/2413240987 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa2dc0ba520 msgr2=0x7fa2c0003a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:21.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.365+0000 7fa2db400640 1 --2- 172.21.15.152:0/2413240987 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa2dc0ba520 0x7fa2c0003a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:21.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.365+0000 7fa2db400640 1 -- 172.21.15.152:0/2413240987 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2c0004d40 msgr2=0x7fa2c00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:21.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.365+0000 7fa2e0c00640 1 --2- 172.21.15.152:0/2413240987 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2c0004d40 0x7fa2c00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:21.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.365+0000 7fa2db400640 1 --2- 172.21.15.152:0/2413240987 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2c0004d40 0x7fa2c00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:21.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.365+0000 7fa2db400640 1 -- 172.21.15.152:0/2413240987 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2c0003520 con 0x7fa2c0003fa0 2024-04-20T11:34:21.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.365+0000 7fa2e0c00640 1 --2- 172.21.15.152:0/2413240987 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2c0004d40 0x7fa2c00051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:34:21.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.366+0000 7fa2db400640 1 --2- 172.21.15.152:0/2413240987 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa2c0003fa0 0x7fa2c0004800 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7fa2c8009ac0 tx=0x7fa2c0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16e59ab19c87e4f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:21.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.367+0000 7fa2daa00640 1 -- 172.21.15.152:0/2413240987 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c8039070 con 0x7fa2c0003fa0 2024-04-20T11:34:21.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.367+0000 7fa2daa00640 1 -- 172.21.15.152:0/2413240987 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2c802c6b0 con 0x7fa2c0003fa0 2024-04-20T11:34:21.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.367+0000 7fa2daa00640 1 -- 172.21.15.152:0/2413240987 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c8009690 con 0x7fa2c0003fa0 2024-04-20T11:34:21.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.367+0000 7fa2e2c00640 1 -- 172.21.15.152:0/2413240987 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa2c0003fa0 msgr2=0x7fa2c0004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:21.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.367+0000 7fa2e2c00640 1 --2- 172.21.15.152:0/2413240987 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa2c0003fa0 0x7fa2c0004800 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7fa2c8009ac0 tx=0x7fa2c0005830 comp rx=0 tx=0).stop 2024-04-20T11:34:21.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.367+0000 7fa2e0c00640 1 -- 172.21.15.152:0/2413240987 reap_dead start 2024-04-20T11:34:21.369 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.367+0000 7fa2e2c00640 1 -- 172.21.15.152:0/2413240987 shutdown_connections 2024-04-20T11:34:21.369 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.367+0000 7fa2e2c00640 1 -- 172.21.15.152:0/2413240987 >> 172.21.15.152:0/2413240987 conn(0x7fa2dc067ae0 msgr2=0x7fa2dc067ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:21.369 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.367+0000 7fa2e2c00640 1 -- 172.21.15.152:0/2413240987 shutdown_connections 2024-04-20T11:34:21.369 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.368+0000 7fa2e2c00640 1 -- 172.21.15.152:0/2413240987 wait complete. 2024-04-20T11:34:21.369 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.369+0000 7fa2e2c00640 1 Processor -- start 2024-04-20T11:34:21.369 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.369+0000 7fa2e2c00640 1 -- start start 2024-04-20T11:34:21.371 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.370+0000 7fa2e2c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa2c0003fa0 0x7fa2dc1f7c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:21.371 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.370+0000 7fa2e2c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2dc156270 0x7fa2dc1566f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:21.371 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.370+0000 7fa2e2c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa2dc0ccb80 0x7fa2dc0cd000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:21.371 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.370+0000 7fa2e2c00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fa2dc0d1f20 con 0x7fa2c0003fa0 2024-04-20T11:34:21.371 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.370+0000 7fa2e2c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa2dc0b80f0 con 0x7fa2dc156270 2024-04-20T11:34:21.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.370+0000 7fa2e2c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa2dc0d20a0 con 0x7fa2dc0ccb80 2024-04-20T11:34:21.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.371+0000 7fa2dbe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa2c0003fa0 0x7fa2dc1f7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:21.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.371+0000 7fa2dbe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa2c0003fa0 0x7fa2dc1f7c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52338/0 (socket says 172.21.15.152:52338) 2024-04-20T11:34:21.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.371+0000 7fa2dbe00640 1 -- 172.21.15.152:0/2014168902 learned_addr learned my addr 172.21.15.152:0/2014168902 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:21.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.371+0000 7fa2db400640 1 --2- 172.21.15.152:0/2014168902 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2dc156270 0x7fa2dc1566f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:21.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.371+0000 7fa2dbe00640 1 -- 172.21.15.152:0/2014168902 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa2dc0ccb80 msgr2=0x7fa2dc0cd000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:21.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.371+0000 7fa2e0c00640 1 --2- 172.21.15.152:0/2014168902 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa2dc0ccb80 0x7fa2dc0cd000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:21.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.371+0000 7fa2dbe00640 1 --2- 172.21.15.152:0/2014168902 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa2dc0ccb80 0x7fa2dc0cd000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:21.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.371+0000 7fa2dbe00640 1 -- 172.21.15.152:0/2014168902 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2dc156270 msgr2=0x7fa2dc1566f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:21.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.371+0000 7fa2dbe00640 1 --2- 172.21.15.152:0/2014168902 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2dc156270 0x7fa2dc1566f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:21.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.371+0000 7fa2dbe00640 1 -- 172.21.15.152:0/2014168902 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2dc156e70 con 0x7fa2c0003fa0 2024-04-20T11:34:21.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.371+0000 7fa2dbe00640 1 --2- 172.21.15.152:0/2014168902 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa2c0003fa0 0x7fa2dc1f7c90 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc0108e0 tx=0x7fa2cc010da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:21.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.372+0000 7fa2d3e00640 1 -- 172.21.15.152:0/2014168902 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc00f040 con 0x7fa2c0003fa0 2024-04-20T11:34:21.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.372+0000 7fa2d3e00640 1 -- 172.21.15.152:0/2014168902 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2cc00eb60 con 0x7fa2c0003fa0 2024-04-20T11:34:21.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.372+0000 7fa2d3e00640 1 -- 172.21.15.152:0/2014168902 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc01c400 con 0x7fa2c0003fa0 2024-04-20T11:34:21.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.372+0000 7fa2e2c00640 1 -- 172.21.15.152:0/2014168902 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2dc157150 con 0x7fa2c0003fa0 2024-04-20T11:34:21.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.372+0000 7fa2e2c00640 1 -- 172.21.15.152:0/2014168902 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2dc157710 con 0x7fa2c0003fa0 2024-04-20T11:34:21.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.373+0000 7fa2e2c00640 1 -- 172.21.15.152:0/2014168902 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2dc0689f0 con 0x7fa2c0003fa0 2024-04-20T11:34:21.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.374+0000 7fa2d3e00640 1 -- 172.21.15.152:0/2014168902 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 19) ==== 91397+0+0 (secure 0 0 0) 0x7fa2cc01c5a0 con 0x7fa2c0003fa0 2024-04-20T11:34:21.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.375+0000 7fa2d3e00640 1 --2- 172.21.15.152:0/2014168902 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fa2bc06cd10 0x7fa2bc06f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:21.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.375+0000 7fa2d3e00640 1 -- 172.21.15.152:0/2014168902 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 5737+0+0 (secure 0 0 0) 0x7fa2cc08ca60 con 0x7fa2c0003fa0 2024-04-20T11:34:21.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.375+0000 7fa2db400640 1 --2- 172.21.15.152:0/2014168902 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fa2bc06cd10 0x7fa2bc06f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:21.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.377+0000 7fa2db400640 1 --2- 172.21.15.152:0/2014168902 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fa2bc06cd10 0x7fa2bc06f1d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa2c8036000 tx=0x7fa2c8030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:21.378 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.379+0000 7fa2d3e00640 1 -- 172.21.15.152:0/2014168902 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fa2cc018030 con 0x7fa2c0003fa0 2024-04-20T11:34:21.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.581+0000 7fa2e2c00640 1 -- 172.21.15.152:0/2014168902 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa2dc157e40 con 0x7fa2c0003fa0 2024-04-20T11:34:21.583 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.583+0000 7fa2d3e00640 1 -- 172.21.15.152:0/2014168902 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) ==== 74+0+16754 (secure 0 0 0) 0x7fa2cc05a200 con 0x7fa2c0003fa0 2024-04-20T11:34:21.583 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:34:21.584 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":106,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:34:19.060050+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:33:25.062365+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-20T11:33:26.260945+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":105,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:34:21.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.586+0000 7fa2e2c00640 1 -- 172.21.15.152:0/2014168902 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fa2bc06cd10 msgr2=0x7fa2bc06f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:21.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.586+0000 7fa2e2c00640 1 --2- 172.21.15.152:0/2014168902 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fa2bc06cd10 0x7fa2bc06f1d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa2c8036000 tx=0x7fa2c8030000 comp rx=0 tx=0).stop 2024-04-20T11:34:21.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.586+0000 7fa2e2c00640 1 -- 172.21.15.152:0/2014168902 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa2c0003fa0 msgr2=0x7fa2dc1f7c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:21.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.586+0000 7fa2e2c00640 1 --2- 172.21.15.152:0/2014168902 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa2c0003fa0 0x7fa2dc1f7c90 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc0108e0 tx=0x7fa2cc010da0 comp rx=0 tx=0).stop 2024-04-20T11:34:21.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.586+0000 7fa2e2c00640 1 -- 172.21.15.152:0/2014168902 shutdown_connections 2024-04-20T11:34:21.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.586+0000 7fa2e2c00640 1 --2- 172.21.15.152:0/2014168902 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fa2bc06cd10 0x7fa2bc06f1d0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:21.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.586+0000 7fa2e2c00640 1 --2- 172.21.15.152:0/2014168902 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa2dc0ccb80 0x7fa2dc0cd000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:21.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.586+0000 7fa2e2c00640 1 --2- 172.21.15.152:0/2014168902 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2dc156270 0x7fa2dc1566f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:21.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.586+0000 7fa2e2c00640 1 --2- 172.21.15.152:0/2014168902 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa2c0003fa0 0x7fa2dc1f7c90 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:21.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.586+0000 7fa2e2c00640 1 -- 172.21.15.152:0/2014168902 >> 172.21.15.152:0/2014168902 conn(0x7fa2dc067ae0 msgr2=0x7fa2dc15f200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:21.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.586+0000 7fa2e2c00640 1 -- 172.21.15.152:0/2014168902 shutdown_connections 2024-04-20T11:34:21.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.586+0000 7fa2e2c00640 1 -- 172.21.15.152:0/2014168902 wait complete. 2024-04-20T11:34:21.606 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-20T11:34:21.607 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:34:21.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.805+0000 7f2dfbe00640 1 Processor -- start 2024-04-20T11:34:21.805 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.805+0000 7f2dfbe00640 1 -- start start 2024-04-20T11:34:21.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.806+0000 7f2dfbe00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f2dfc15d1e0 0x7f2dfc160b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:21.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.807+0000 7f2dfbe00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f2dfc15d5c0 0x7f2dfc0cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:21.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.807+0000 7f2dfbe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2dfc071cd0 0x7f2dfc072160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:21.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.807+0000 7f2dfbe00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f2dfc0726a0 con 0x7f2dfc15d1e0 2024-04-20T11:34:21.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.807+0000 7f2dfbe00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f2dfc0ce8c0 con 0x7f2dfc15d5c0 2024-04-20T11:34:21.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.807+0000 7f2dfbe00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2dfc1600b0 con 0x7f2dfc071cd0 2024-04-20T11:34:21.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.807+0000 7f2dfaa00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f2dfc15d1e0 0x7f2dfc160b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:21.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.807+0000 7f2dfaa00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f2dfc15d1e0 0x7f2dfc160b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52348/0 (socket says 172.21.15.152:52348) 2024-04-20T11:34:21.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.807+0000 7f2dfaa00640 1 -- 172.21.15.152:0/3129925687 learned_addr learned my addr 172.21.15.152:0/3129925687 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:21.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.807+0000 7f2dfaa00640 1 -- 172.21.15.152:0/3129925687 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2dfc071cd0 msgr2=0x7f2dfc072160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:21.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.807+0000 7f2dfaa00640 1 --2- 172.21.15.152:0/3129925687 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2dfc071cd0 0x7f2dfc072160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:21.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.807+0000 7f2dfaa00640 1 -- 172.21.15.152:0/3129925687 >> v2:172.21.15.175:3300/0 conn(0x7f2dfc15d5c0 msgr2=0x7f2dfc0cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:21.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.807+0000 7f2df3e00640 1 --2- 172.21.15.152:0/3129925687 >> v2:172.21.15.175:3300/0 conn(0x7f2dfc15d5c0 0x7f2dfc0cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:21.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.807+0000 7f2dfaa00640 1 --2- 172.21.15.152:0/3129925687 >> v2:172.21.15.175:3300/0 conn(0x7f2dfc15d5c0 0x7f2dfc0cdfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:21.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.807+0000 7f2dfaa00640 1 -- 172.21.15.152:0/3129925687 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dfc160390 con 0x7f2dfc15d1e0 2024-04-20T11:34:21.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.809+0000 7f2dfaa00640 1 --2- 172.21.15.152:0/3129925687 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2dfc15d1e0 0x7f2dfc160b90 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f2dec009870 tx=0x7f2dec02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75108b83ce2ea07b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:21.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.809+0000 7f2dfa000640 1 -- 172.21.15.152:0/3129925687 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dec035020 con 0x7f2dfc15d1e0 2024-04-20T11:34:21.809 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.809+0000 7f2dfa000640 1 -- 172.21.15.152:0/3129925687 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2dec031640 con 0x7f2dfc15d1e0 2024-04-20T11:34:21.809 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.809+0000 7f2dfa000640 1 -- 172.21.15.152:0/3129925687 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dec02d040 con 0x7f2dfc15d1e0 2024-04-20T11:34:21.809 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.810+0000 7f2dfbe00640 1 -- 172.21.15.152:0/3129925687 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2dfc15d1e0 msgr2=0x7f2dfc160b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:21.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.810+0000 7f2dfbe00640 1 --2- 172.21.15.152:0/3129925687 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2dfc15d1e0 0x7f2dfc160b90 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f2dec009870 tx=0x7f2dec02b8f0 comp rx=0 tx=0).stop 2024-04-20T11:34:21.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.810+0000 7f2dfbe00640 1 -- 172.21.15.152:0/3129925687 shutdown_connections 2024-04-20T11:34:21.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.810+0000 7f2dfbe00640 1 --2- 172.21.15.152:0/3129925687 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2dfc071cd0 0x7f2dfc072160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:21.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.810+0000 7f2dfbe00640 1 --2- 172.21.15.152:0/3129925687 >> v2:172.21.15.175:3300/0 conn(0x7f2dfc15d5c0 0x7f2dfc0cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:21.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.810+0000 7f2dfbe00640 1 --2- 172.21.15.152:0/3129925687 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2dfc15d1e0 0x7f2dfc160b90 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:21.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.810+0000 7f2dfbe00640 1 -- 172.21.15.152:0/3129925687 >> 172.21.15.152:0/3129925687 conn(0x7f2dfc067ae0 msgr2=0x7f2dfc06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:21.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.810+0000 7f2dfbe00640 1 -- 172.21.15.152:0/3129925687 shutdown_connections 2024-04-20T11:34:21.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.810+0000 7f2dfbe00640 1 -- 172.21.15.152:0/3129925687 wait complete. 2024-04-20T11:34:21.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.812+0000 7f2dfbe00640 1 Processor -- start 2024-04-20T11:34:21.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.812+0000 7f2dfbe00640 1 -- start start 2024-04-20T11:34:21.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.813+0000 7f2dfbe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2dfc071cd0 0x7f2dfc152ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:21.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.813+0000 7f2dfbe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2dfc15d1e0 0x7f2dfc156490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:21.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.813+0000 7f2dfbe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2dfc15d5c0 0x7f2dfc1569d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:21.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.813+0000 7f2dfbe00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f2dfc0ce520 con 0x7f2dfc15d1e0 2024-04-20T11:34:21.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.813+0000 7f2dfbe00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2dfc0ceba0 con 0x7f2dfc15d5c0 2024-04-20T11:34:21.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.813+0000 7f2dfbe00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2dfc0ce6a0 con 0x7f2dfc071cd0 2024-04-20T11:34:21.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.813+0000 7f2df3e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2dfc15d1e0 0x7f2dfc156490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:21.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.813+0000 7f2df3e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2dfc15d1e0 0x7f2dfc156490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52362/0 (socket says 172.21.15.152:52362) 2024-04-20T11:34:21.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.813+0000 7f2df3e00640 1 -- 172.21.15.152:0/3227327285 learned_addr learned my addr 172.21.15.152:0/3227327285 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:21.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.813+0000 7f2dfaa00640 1 --2- 172.21.15.152:0/3227327285 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2dfc071cd0 0x7f2dfc152ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:21.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.814+0000 7f2dfb400640 1 --2- 172.21.15.152:0/3227327285 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2dfc15d5c0 0x7f2dfc1569d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:21.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.814+0000 7f2df3e00640 1 -- 172.21.15.152:0/3227327285 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2dfc071cd0 msgr2=0x7f2dfc152ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:21.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.814+0000 7f2df3e00640 1 --2- 172.21.15.152:0/3227327285 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2dfc071cd0 0x7f2dfc152ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:21.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.814+0000 7f2df3e00640 1 -- 172.21.15.152:0/3227327285 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2dfc15d5c0 msgr2=0x7f2dfc1569d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:21.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.814+0000 7f2df3e00640 1 --2- 172.21.15.152:0/3227327285 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2dfc15d5c0 0x7f2dfc1569d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:21.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.814+0000 7f2df3e00640 1 -- 172.21.15.152:0/3227327285 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dfc1598c0 con 0x7f2dfc15d1e0 2024-04-20T11:34:21.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.814+0000 7f2df3e00640 1 --2- 172.21.15.152:0/3227327285 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2dfc15d1e0 0x7f2dfc156490 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f2de400c820 tx=0x7f2de400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:21.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.814+0000 7f2df2a00640 1 -- 172.21.15.152:0/3227327285 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de4015070 con 0x7f2dfc15d1e0 2024-04-20T11:34:21.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.814+0000 7f2df2a00640 1 -- 172.21.15.152:0/3227327285 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2de4004240 con 0x7f2dfc15d1e0 2024-04-20T11:34:21.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.814+0000 7f2df2a00640 1 -- 172.21.15.152:0/3227327285 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de400ce10 con 0x7f2dfc15d1e0 2024-04-20T11:34:21.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.814+0000 7f2dfbe00640 1 -- 172.21.15.152:0/3227327285 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2dfc159ba0 con 0x7f2dfc15d1e0 2024-04-20T11:34:21.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.814+0000 7f2dfbe00640 1 -- 172.21.15.152:0/3227327285 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2dfc1539d0 con 0x7f2dfc15d1e0 2024-04-20T11:34:21.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.816+0000 7f2dfbe00640 1 -- 172.21.15.152:0/3227327285 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2dfc1537f0 con 0x7f2dfc15d1e0 2024-04-20T11:34:21.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.817+0000 7f2df2a00640 1 -- 172.21.15.152:0/3227327285 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 19) ==== 91397+0+0 (secure 0 0 0) 0x7f2de4007830 con 0x7f2dfc15d1e0 2024-04-20T11:34:21.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.818+0000 7f2df2a00640 1 --2- 172.21.15.152:0/3227327285 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f2dd806cd10 0x7f2dd806f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:21.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.818+0000 7f2df2a00640 1 -- 172.21.15.152:0/3227327285 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 5737+0+0 (secure 0 0 0) 0x7f2de408c260 con 0x7f2dfc15d1e0 2024-04-20T11:34:21.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.819+0000 7f2dfaa00640 1 --2- 172.21.15.152:0/3227327285 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f2dd806cd10 0x7f2dd806f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:21.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.820+0000 7f2dfaa00640 1 --2- 172.21.15.152:0/3227327285 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f2dd806cd10 0x7f2dd806f1d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f2dec036000 tx=0x7f2dec03c040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:21.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:21.823+0000 7f2df2a00640 1 -- 172.21.15.152:0/3227327285 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f2dfc1537f0 con 0x7f2dfc15d1e0 2024-04-20T11:34:22.061 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.061+0000 7f2dfbe00640 1 -- 172.21.15.152:0/3227327285 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2dfc075e60 con 0x7f2dfc15d1e0 2024-04-20T11:34:22.061 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.062+0000 7f2df2a00640 1 -- 172.21.15.152:0/3227327285 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+5210 (secure 0 0 0) 0x7f2de4059fd0 con 0x7f2dfc15d1e0 2024-04-20T11:34:22.062 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:34:22.062 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":67,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5648,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6834/387198089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":387198089},{"type":"v1","addr":"172.21.15.152:6834","nonce":387198089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":6154,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/1674013715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1674013715},{"type":"v1","addr":"172.21.15.175:6833","nonce":1674013715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":6157,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/3898238490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3898238490},{"type":"v1","addr":"172.21.15.175:6835","nonce":3898238490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T11:33:27.277882+0000","modified":"2024-04-20T11:33:30.342429+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7794},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7794":{"gid":7794,"name":"a","rank":0,"incarnation":62,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6835/3582312712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6833","nonce":3582312712},{"type":"v1","addr":"172.21.15.152:6835","nonce":3582312712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7794,"qdb_cluster":[7794]},"id":8}]} 2024-04-20T11:34:22.062 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 67 2024-04-20T11:34:22.065 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.065+0000 7f2dfbe00640 1 -- 172.21.15.152:0/3227327285 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f2dd806cd10 msgr2=0x7f2dd806f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:22.065 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.065+0000 7f2dfbe00640 1 --2- 172.21.15.152:0/3227327285 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f2dd806cd10 0x7f2dd806f1d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f2dec036000 tx=0x7f2dec03c040 comp rx=0 tx=0).stop 2024-04-20T11:34:22.065 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.065+0000 7f2dfbe00640 1 -- 172.21.15.152:0/3227327285 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2dfc15d1e0 msgr2=0x7f2dfc156490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:22.065 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.065+0000 7f2dfbe00640 1 --2- 172.21.15.152:0/3227327285 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2dfc15d1e0 0x7f2dfc156490 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f2de400c820 tx=0x7f2de400cce0 comp rx=0 tx=0).stop 2024-04-20T11:34:22.065 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.065+0000 7f2dfbe00640 1 -- 172.21.15.152:0/3227327285 shutdown_connections 2024-04-20T11:34:22.065 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.065+0000 7f2dfbe00640 1 --2- 172.21.15.152:0/3227327285 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f2dd806cd10 0x7f2dd806f1d0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:22.065 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.065+0000 7f2dfbe00640 1 --2- 172.21.15.152:0/3227327285 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2dfc15d5c0 0x7f2dfc1569d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:22.065 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.065+0000 7f2dfbe00640 1 --2- 172.21.15.152:0/3227327285 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2dfc15d1e0 0x7f2dfc156490 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:22.065 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.065+0000 7f2dfbe00640 1 --2- 172.21.15.152:0/3227327285 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2dfc071cd0 0x7f2dfc152ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:22.065 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.065+0000 7f2dfbe00640 1 -- 172.21.15.152:0/3227327285 >> 172.21.15.152:0/3227327285 conn(0x7f2dfc067ae0 msgr2=0x7f2dfc06e5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:22.065 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.065+0000 7f2dfbe00640 1 -- 172.21.15.152:0/3227327285 shutdown_connections 2024-04-20T11:34:22.066 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.065+0000 7f2dfbe00640 1 -- 172.21.15.152:0/3227327285 wait complete. 2024-04-20T11:34:22.085 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:34:22.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.284+0000 7fca7fe00640 1 Processor -- start 2024-04-20T11:34:22.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.284+0000 7fca7fe00640 1 -- start start 2024-04-20T11:34:22.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.286+0000 7fca7fe00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fca800723e0 0x7fca800afa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:22.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.286+0000 7fca7fe00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca800aec90 con 0x7fca80147410 2024-04-20T11:34:22.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.286+0000 7fca7fe00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca800aef70 con 0x7fca8006e010 2024-04-20T11:34:22.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.286+0000 7fca7fe00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fca800af250 con 0x7fca800723e0 2024-04-20T11:34:22.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.286+0000 7fca76000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fca800723e0 0x7fca800afa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:22.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.286+0000 7fca7ea00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fca80147410 0x7fca80071cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:40264/0 (socket says 172.21.15.152:40264) 2024-04-20T11:34:22.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.286+0000 7fca7ea00640 1 -- 172.21.15.152:0/1712816937 learned_addr learned my addr 172.21.15.152:0/1712816937 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:34:22.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.287+0000 7fca76000640 1 -- 172.21.15.152:0/1712816937 >> v1:172.21.15.175:6789/0 conn(0x7fca8006e010 legacy=0x7fca8006e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:22.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.287+0000 7fca76000640 1 -- 172.21.15.152:0/1712816937 >> v1:172.21.15.152:6789/0 conn(0x7fca80147410 legacy=0x7fca80071cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:22.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.287+0000 7fca76000640 1 -- 172.21.15.152:0/1712816937 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca800af530 con 0x7fca800723e0 2024-04-20T11:34:22.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.287+0000 7fca7e000640 1 -- 172.21.15.152:0/1712816937 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2444880693 0 0) 0x7fca800aec90 con 0x7fca80147410 2024-04-20T11:34:22.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.288+0000 7fca76000640 1 --2- 172.21.15.152:0/1712816937 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fca800723e0 0x7fca800afa50 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7fca680097a0 tx=0x7fca6802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d907f5b7cf0d9e4c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:22.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.288+0000 7fca7e000640 1 -- 172.21.15.152:0/1712816937 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca68034070 con 0x7fca800723e0 2024-04-20T11:34:22.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.288+0000 7fca7e000640 1 -- 172.21.15.152:0/1712816937 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca68030610 con 0x7fca800723e0 2024-04-20T11:34:22.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.288+0000 7fca7e000640 1 -- 172.21.15.152:0/1712816937 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca680309e0 con 0x7fca800723e0 2024-04-20T11:34:22.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.289+0000 7fca7fe00640 1 -- 172.21.15.152:0/1712816937 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fca800723e0 msgr2=0x7fca800afa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:22.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.289+0000 7fca7fe00640 1 --2- 172.21.15.152:0/1712816937 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fca800723e0 0x7fca800afa50 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7fca680097a0 tx=0x7fca6802b960 comp rx=0 tx=0).stop 2024-04-20T11:34:22.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.289+0000 7fca7fe00640 1 -- 172.21.15.152:0/1712816937 shutdown_connections 2024-04-20T11:34:22.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.289+0000 7fca7fe00640 1 --2- 172.21.15.152:0/1712816937 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fca800723e0 0x7fca800afa50 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:22.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.289+0000 7fca7fe00640 1 -- 172.21.15.152:0/1712816937 >> 172.21.15.152:0/1712816937 conn(0x7fca80067ae0 msgr2=0x7fca8006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:22.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.289+0000 7fca7fe00640 1 -- 172.21.15.152:0/1712816937 shutdown_connections 2024-04-20T11:34:22.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.289+0000 7fca7fe00640 1 -- 172.21.15.152:0/1712816937 wait complete. 2024-04-20T11:34:22.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.291+0000 7fca7fe00640 1 Processor -- start 2024-04-20T11:34:22.291 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.291+0000 7fca7fe00640 1 -- start start 2024-04-20T11:34:22.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.292+0000 7fca7fe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca8006e010 0x7fca80161520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:22.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.293+0000 7fca7fe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fca800723e0 0x7fca80145410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:22.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.293+0000 7fca7fe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fca80147410 0x7fca80160540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:22.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.293+0000 7fca7fe00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fca800b3510 con 0x7fca80147410 2024-04-20T11:34:22.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.293+0000 7fca7fe00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fca800b3390 con 0x7fca8006e010 2024-04-20T11:34:22.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.293+0000 7fca7fe00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fca800b3690 con 0x7fca800723e0 2024-04-20T11:34:22.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.293+0000 7fca7f400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fca80147410 0x7fca80160540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:22.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.293+0000 7fca7f400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fca80147410 0x7fca80160540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52378/0 (socket says 172.21.15.152:52378) 2024-04-20T11:34:22.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.293+0000 7fca7f400640 1 -- 172.21.15.152:0/1560694341 learned_addr learned my addr 172.21.15.152:0/1560694341 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:22.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.293+0000 7fca7ea00640 1 --2- 172.21.15.152:0/1560694341 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca8006e010 0x7fca80161520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:22.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.293+0000 7fca76000640 1 --2- 172.21.15.152:0/1560694341 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fca800723e0 0x7fca80145410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:22.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.293+0000 7fca7f400640 1 -- 172.21.15.152:0/1560694341 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fca800723e0 msgr2=0x7fca80145410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:22.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.293+0000 7fca7f400640 1 --2- 172.21.15.152:0/1560694341 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fca800723e0 0x7fca80145410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:22.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.293+0000 7fca7f400640 1 -- 172.21.15.152:0/1560694341 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca8006e010 msgr2=0x7fca80161520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:22.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.293+0000 7fca7f400640 1 --2- 172.21.15.152:0/1560694341 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca8006e010 0x7fca80161520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:22.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.293+0000 7fca7f400640 1 -- 172.21.15.152:0/1560694341 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca8013e2f0 con 0x7fca80147410 2024-04-20T11:34:22.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.294+0000 7fca7f400640 1 --2- 172.21.15.152:0/1560694341 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fca80147410 0x7fca80160540 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7fca7800c7d0 tx=0x7fca7800cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:22.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.294+0000 7fca77400640 1 -- 172.21.15.152:0/1560694341 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca78003b70 con 0x7fca80147410 2024-04-20T11:34:22.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.294+0000 7fca77400640 1 -- 172.21.15.152:0/1560694341 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca78003d30 con 0x7fca80147410 2024-04-20T11:34:22.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.294+0000 7fca7fe00640 1 -- 172.21.15.152:0/1560694341 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca8013e5d0 con 0x7fca80147410 2024-04-20T11:34:22.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.294+0000 7fca77400640 1 -- 172.21.15.152:0/1560694341 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca7800aa90 con 0x7fca80147410 2024-04-20T11:34:22.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.294+0000 7fca7fe00640 1 -- 172.21.15.152:0/1560694341 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca8013cd20 con 0x7fca80147410 2024-04-20T11:34:22.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.296+0000 7fca7fe00640 1 -- 172.21.15.152:0/1560694341 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca80075e60 con 0x7fca80147410 2024-04-20T11:34:22.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.297+0000 7fca77400640 1 -- 172.21.15.152:0/1560694341 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 19) ==== 91397+0+0 (secure 0 0 0) 0x7fca7800ac30 con 0x7fca80147410 2024-04-20T11:34:22.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.298+0000 7fca77400640 1 --2- 172.21.15.152:0/1560694341 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fca5c06cce0 0x7fca5c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:22.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.298+0000 7fca77400640 1 -- 172.21.15.152:0/1560694341 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 5737+0+0 (secure 0 0 0) 0x7fca7808c150 con 0x7fca80147410 2024-04-20T11:34:22.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.298+0000 7fca7ea00640 1 --2- 172.21.15.152:0/1560694341 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fca5c06cce0 0x7fca5c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:22.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.300+0000 7fca7ea00640 1 --2- 172.21.15.152:0/1560694341 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fca5c06cce0 0x7fca5c06f1a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fca70000f60 tx=0x7fca70007d00 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:22.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.302+0000 7fca77400640 1 -- 172.21.15.152:0/1560694341 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fca7805a010 con 0x7fca80147410 2024-04-20T11:34:22.540 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.540+0000 7fca7fe00640 1 -- 172.21.15.152:0/1560694341 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fca8006f170 con 0x7fca80147410 2024-04-20T11:34:22.541 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.541+0000 7fca77400640 1 -- 172.21.15.152:0/1560694341 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+5210 (secure 0 0 0) 0x7fca78059910 con 0x7fca80147410 2024-04-20T11:34:22.541 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:34:22.541 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":67,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5648,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6834/387198089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":387198089},{"type":"v1","addr":"172.21.15.152:6834","nonce":387198089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":6154,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/1674013715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1674013715},{"type":"v1","addr":"172.21.15.175:6833","nonce":1674013715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":6157,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/3898238490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3898238490},{"type":"v1","addr":"172.21.15.175:6835","nonce":3898238490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T11:33:27.277882+0000","modified":"2024-04-20T11:33:30.342429+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7794},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7794":{"gid":7794,"name":"a","rank":0,"incarnation":62,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6835/3582312712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6833","nonce":3582312712},{"type":"v1","addr":"172.21.15.152:6835","nonce":3582312712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7794,"qdb_cluster":[7794]},"id":8}]} 2024-04-20T11:34:22.542 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 67 2024-04-20T11:34:22.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.544+0000 7fca7fe00640 1 -- 172.21.15.152:0/1560694341 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fca5c06cce0 msgr2=0x7fca5c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:22.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.544+0000 7fca7fe00640 1 --2- 172.21.15.152:0/1560694341 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fca5c06cce0 0x7fca5c06f1a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fca70000f60 tx=0x7fca70007d00 comp rx=0 tx=0).stop 2024-04-20T11:34:22.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.544+0000 7fca7fe00640 1 -- 172.21.15.152:0/1560694341 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fca80147410 msgr2=0x7fca80160540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:22.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.544+0000 7fca7fe00640 1 --2- 172.21.15.152:0/1560694341 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fca80147410 0x7fca80160540 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7fca7800c7d0 tx=0x7fca7800cc90 comp rx=0 tx=0).stop 2024-04-20T11:34:22.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.544+0000 7fca7fe00640 1 -- 172.21.15.152:0/1560694341 shutdown_connections 2024-04-20T11:34:22.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.545+0000 7fca7fe00640 1 --2- 172.21.15.152:0/1560694341 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fca5c06cce0 0x7fca5c06f1a0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:22.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.545+0000 7fca7fe00640 1 --2- 172.21.15.152:0/1560694341 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fca80147410 0x7fca80160540 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:22.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.545+0000 7fca7fe00640 1 --2- 172.21.15.152:0/1560694341 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fca800723e0 0x7fca80145410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:22.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.545+0000 7fca7fe00640 1 --2- 172.21.15.152:0/1560694341 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca8006e010 0x7fca80161520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:22.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.545+0000 7fca7fe00640 1 -- 172.21.15.152:0/1560694341 >> 172.21.15.152:0/1560694341 conn(0x7fca80067ae0 msgr2=0x7fca8006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:22.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.545+0000 7fca7fe00640 1 -- 172.21.15.152:0/1560694341 shutdown_connections 2024-04-20T11:34:22.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.545+0000 7fca7fe00640 1 -- 172.21.15.152:0/1560694341 wait complete. 2024-04-20T11:34:22.564 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:34:22.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.756+0000 7fba49c00640 1 Processor -- start 2024-04-20T11:34:22.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.757+0000 7fba49c00640 1 -- start start 2024-04-20T11:34:22.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.758+0000 7fba49c00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fba441603d0 0x7fba440b5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:22.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.758+0000 7fba49c00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba440d3350 con 0x7fba4415fcc0 2024-04-20T11:34:22.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.758+0000 7fba49c00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba440d3630 con 0x7fba440b5480 2024-04-20T11:34:22.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.758+0000 7fba49c00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fba440d3910 con 0x7fba441603d0 2024-04-20T11:34:22.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.758+0000 7fba43e00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fba4415fcc0 0x7fba440d4e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:40272/0 (socket says 172.21.15.152:40272) 2024-04-20T11:34:22.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.758+0000 7fba43e00640 1 -- 172.21.15.152:0/1635673508 learned_addr learned my addr 172.21.15.152:0/1635673508 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:34:22.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.759+0000 7fba43400640 1 --2- 172.21.15.152:0/1635673508 >> v2:172.21.15.175:3300/0 conn(0x7fba441603d0 0x7fba440b5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:22.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.759+0000 7fba42a00640 1 -- 172.21.15.152:0/1635673508 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1271604150 0 0) 0x7fba440d3350 con 0x7fba4415fcc0 2024-04-20T11:34:22.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.759+0000 7fba42a00640 1 -- 172.21.15.152:0/1635673508 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fba28003520 con 0x7fba4415fcc0 2024-04-20T11:34:22.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.759+0000 7fba42a00640 1 -- 172.21.15.152:0/1635673508 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 767496063 0 0) 0x7fba28003520 con 0x7fba4415fcc0 2024-04-20T11:34:22.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.759+0000 7fba42a00640 1 -- 172.21.15.152:0/1635673508 >> v2:172.21.15.175:3300/0 conn(0x7fba441603d0 msgr2=0x7fba440b5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:22.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.759+0000 7fba42a00640 1 --2- 172.21.15.152:0/1635673508 >> v2:172.21.15.175:3300/0 conn(0x7fba441603d0 0x7fba440b5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:22.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.759+0000 7fba42a00640 1 -- 172.21.15.152:0/1635673508 >> v1:172.21.15.175:6789/0 conn(0x7fba440b5480 legacy=0x7fba440cf240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:22.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.759+0000 7fba42a00640 1 -- 172.21.15.152:0/1635673508 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba440ced20 con 0x7fba4415fcc0 2024-04-20T11:34:22.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.759+0000 7fba42a00640 1 -- 172.21.15.152:0/1635673508 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7fba34001670 con 0x7fba4415fcc0 2024-04-20T11:34:22.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.759+0000 7fba42a00640 1 -- 172.21.15.152:0/1635673508 >> v1:172.21.15.152:6789/0 conn(0x7fba4415fcc0 legacy=0x7fba440d4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:22.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.759+0000 7fba42a00640 1 --2- 172.21.15.152:0/1635673508 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fba28003a00 0x7fba28003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:22.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.759+0000 7fba42a00640 1 --2- 172.21.15.152:0/1635673508 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba28004800 0x7fba28004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:22.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.760+0000 7fba42a00640 1 --2- 172.21.15.152:0/1635673508 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba280051c0 0x7fba2800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:22.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.760+0000 7fba42a00640 1 -- 172.21.15.152:0/1635673508 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fba440d3910 con 0x7fba28003a00 2024-04-20T11:34:22.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.760+0000 7fba42a00640 1 -- 172.21.15.152:0/1635673508 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fba2800c0c0 con 0x7fba280051c0 2024-04-20T11:34:22.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.760+0000 7fba42a00640 1 -- 172.21.15.152:0/1635673508 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fba2800c2c0 con 0x7fba28004800 2024-04-20T11:34:22.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.760+0000 7fba43e00640 1 --2- 172.21.15.152:0/1635673508 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fba28003a00 0x7fba28003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:22.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.760+0000 7fba43e00640 1 -- 172.21.15.152:0/1635673508 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba28004800 msgr2=0x7fba28004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:22.762 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.760+0000 7fba43e00640 1 --2- 172.21.15.152:0/1635673508 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba28004800 0x7fba28004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:22.762 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.760+0000 7fba49200640 1 --2- 172.21.15.152:0/1635673508 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba280051c0 0x7fba2800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:22.762 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.760+0000 7fba43e00640 1 -- 172.21.15.152:0/1635673508 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba280051c0 msgr2=0x7fba2800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:22.762 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.760+0000 7fba43e00640 1 --2- 172.21.15.152:0/1635673508 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba280051c0 0x7fba2800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:22.762 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.760+0000 7fba43e00640 1 -- 172.21.15.152:0/1635673508 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba28003520 con 0x7fba28003a00 2024-04-20T11:34:22.762 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.760+0000 7fba49200640 1 --2- 172.21.15.152:0/1635673508 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba280051c0 0x7fba2800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:34:22.762 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.760+0000 7fba49200640 1 -- 172.21.15.152:0/1635673508 reap_dead start 2024-04-20T11:34:22.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.761+0000 7fba43e00640 1 --2- 172.21.15.152:0/1635673508 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fba28003a00 0x7fba28003e20 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7fba3400c0e0 tx=0x7fba28005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f110599f0ca3081 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:22.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.762+0000 7fba42a00640 1 -- 172.21.15.152:0/1635673508 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba34007c20 con 0x7fba28003a00 2024-04-20T11:34:22.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.762+0000 7fba42a00640 1 -- 172.21.15.152:0/1635673508 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba34030070 con 0x7fba28003a00 2024-04-20T11:34:22.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.762+0000 7fba42a00640 1 -- 172.21.15.152:0/1635673508 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba34005d70 con 0x7fba28003a00 2024-04-20T11:34:22.764 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.762+0000 7fba49c00640 1 -- 172.21.15.152:0/1635673508 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fba28003a00 msgr2=0x7fba28003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:22.764 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.762+0000 7fba49c00640 1 --2- 172.21.15.152:0/1635673508 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fba28003a00 0x7fba28003e20 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7fba3400c0e0 tx=0x7fba28005830 comp rx=0 tx=0).stop 2024-04-20T11:34:22.764 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.762+0000 7fba49c00640 1 -- 172.21.15.152:0/1635673508 shutdown_connections 2024-04-20T11:34:22.764 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.762+0000 7fba49c00640 1 --2- 172.21.15.152:0/1635673508 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fba28003a00 0x7fba28003e20 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:22.764 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.762+0000 7fba49c00640 1 -- 172.21.15.152:0/1635673508 >> 172.21.15.152:0/1635673508 conn(0x7fba4408bfd0 msgr2=0x7fba440d4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:22.764 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.762+0000 7fba49c00640 1 -- 172.21.15.152:0/1635673508 shutdown_connections 2024-04-20T11:34:22.764 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.762+0000 7fba49c00640 1 -- 172.21.15.152:0/1635673508 wait complete. 2024-04-20T11:34:22.764 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.764+0000 7fba49c00640 1 Processor -- start 2024-04-20T11:34:22.766 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.764+0000 7fba49c00640 1 -- start start 2024-04-20T11:34:22.766 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.765+0000 7fba49c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fba28003a00 0x7fba440ca610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:22.766 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.765+0000 7fba49c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba440c90d0 0x7fba4414a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:22.766 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.765+0000 7fba49c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba4414ab20 0x7fba4414afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:22.767 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.765+0000 7fba49c00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fba440d5200 con 0x7fba28003a00 2024-04-20T11:34:22.767 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.765+0000 7fba49c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fba440d5080 con 0x7fba440c90d0 2024-04-20T11:34:22.767 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.765+0000 7fba49c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fba440d5380 con 0x7fba4414ab20 2024-04-20T11:34:22.767 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.765+0000 7fba43e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fba28003a00 0x7fba440ca610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:22.767 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.765+0000 7fba43e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fba28003a00 0x7fba440ca610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52400/0 (socket says 172.21.15.152:52400) 2024-04-20T11:34:22.767 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.765+0000 7fba43e00640 1 -- 172.21.15.152:0/3139979816 learned_addr learned my addr 172.21.15.152:0/3139979816 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:22.767 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.766+0000 7fba43400640 1 --2- 172.21.15.152:0/3139979816 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba440c90d0 0x7fba4414a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:22.767 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.766+0000 7fba49200640 1 --2- 172.21.15.152:0/3139979816 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba4414ab20 0x7fba4414afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:22.767 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.766+0000 7fba43e00640 1 -- 172.21.15.152:0/3139979816 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba4414ab20 msgr2=0x7fba4414afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:22.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.766+0000 7fba43e00640 1 --2- 172.21.15.152:0/3139979816 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba4414ab20 0x7fba4414afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:22.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.766+0000 7fba43e00640 1 -- 172.21.15.152:0/3139979816 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba440c90d0 msgr2=0x7fba4414a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:22.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.766+0000 7fba43e00640 1 --2- 172.21.15.152:0/3139979816 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba440c90d0 0x7fba4414a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:22.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.766+0000 7fba43e00640 1 -- 172.21.15.152:0/3139979816 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba4414a7f0 con 0x7fba28003a00 2024-04-20T11:34:22.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.766+0000 7fba49200640 1 --2- 172.21.15.152:0/3139979816 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba4414ab20 0x7fba4414afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:34:22.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.766+0000 7fba43400640 1 --2- 172.21.15.152:0/3139979816 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba440c90d0 0x7fba4414a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:34:22.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.766+0000 7fba43e00640 1 --2- 172.21.15.152:0/3139979816 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fba28003a00 0x7fba440ca610 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7fba3400c200 tx=0x7fba440b5d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:22.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.766+0000 7fba3be00640 1 -- 172.21.15.152:0/3139979816 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba34004bf0 con 0x7fba28003a00 2024-04-20T11:34:22.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.766+0000 7fba3be00640 1 -- 172.21.15.152:0/3139979816 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba34000ed0 con 0x7fba28003a00 2024-04-20T11:34:22.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.766+0000 7fba3be00640 1 -- 172.21.15.152:0/3139979816 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba34007070 con 0x7fba28003a00 2024-04-20T11:34:22.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.766+0000 7fba49c00640 1 -- 172.21.15.152:0/3139979816 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba440c9570 con 0x7fba28003a00 2024-04-20T11:34:22.769 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.767+0000 7fba49c00640 1 -- 172.21.15.152:0/3139979816 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba440c9b30 con 0x7fba28003a00 2024-04-20T11:34:22.769 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.768+0000 7fba49c00640 1 -- 172.21.15.152:0/3139979816 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba44075e60 con 0x7fba28003a00 2024-04-20T11:34:22.769 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.769+0000 7fba3be00640 1 -- 172.21.15.152:0/3139979816 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 19) ==== 91397+0+0 (secure 0 0 0) 0x7fba34048020 con 0x7fba28003a00 2024-04-20T11:34:22.770 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.770+0000 7fba3be00640 1 --2- 172.21.15.152:0/3139979816 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fba2006cce0 0x7fba2006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:22.770 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.770+0000 7fba3be00640 1 -- 172.21.15.152:0/3139979816 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 5737+0+0 (secure 0 0 0) 0x7fba340ad600 con 0x7fba28003a00 2024-04-20T11:34:22.770 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.770+0000 7fba43400640 1 --2- 172.21.15.152:0/3139979816 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fba2006cce0 0x7fba2006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:22.771 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.772+0000 7fba43400640 1 --2- 172.21.15.152:0/3139979816 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fba2006cce0 0x7fba2006f1a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fba441607f0 tx=0x7fba2c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:22.773 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.774+0000 7fba3be00640 1 -- 172.21.15.152:0/3139979816 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fba3407b4c0 con 0x7fba28003a00 2024-04-20T11:34:22.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.981+0000 7fba49c00640 1 -- 172.21.15.152:0/3139979816 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fba440ba4d0 con 0x7fba28003a00 2024-04-20T11:34:22.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.984+0000 7fba3be00640 1 -- 172.21.15.152:0/3139979816 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) ==== 74+0+16754 (secure 0 0 0) 0x7fba3407adc0 con 0x7fba28003a00 2024-04-20T11:34:22.984 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:34:22.984 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":106,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:34:19.060050+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:33:25.062365+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-20T11:33:26.260945+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":105,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:34:22.986 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.986+0000 7fba49c00640 1 -- 172.21.15.152:0/3139979816 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fba2006cce0 msgr2=0x7fba2006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:22.986 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.986+0000 7fba49c00640 1 --2- 172.21.15.152:0/3139979816 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fba2006cce0 0x7fba2006f1a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fba441607f0 tx=0x7fba2c008040 comp rx=0 tx=0).stop 2024-04-20T11:34:22.986 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.986+0000 7fba49c00640 1 -- 172.21.15.152:0/3139979816 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fba28003a00 msgr2=0x7fba440ca610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:22.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.986+0000 7fba49c00640 1 --2- 172.21.15.152:0/3139979816 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fba28003a00 0x7fba440ca610 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7fba3400c200 tx=0x7fba440b5d20 comp rx=0 tx=0).stop 2024-04-20T11:34:22.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.987+0000 7fba49c00640 1 -- 172.21.15.152:0/3139979816 shutdown_connections 2024-04-20T11:34:22.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.987+0000 7fba49c00640 1 --2- 172.21.15.152:0/3139979816 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fba2006cce0 0x7fba2006f1a0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:22.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.987+0000 7fba49c00640 1 --2- 172.21.15.152:0/3139979816 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba4414ab20 0x7fba4414afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:22.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.987+0000 7fba49c00640 1 --2- 172.21.15.152:0/3139979816 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba440c90d0 0x7fba4414a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:22.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.987+0000 7fba49c00640 1 --2- 172.21.15.152:0/3139979816 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fba28003a00 0x7fba440ca610 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:22.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.987+0000 7fba49c00640 1 -- 172.21.15.152:0/3139979816 >> 172.21.15.152:0/3139979816 conn(0x7fba4408bfd0 msgr2=0x7fba4406e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:22.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.987+0000 7fba49c00640 1 -- 172.21.15.152:0/3139979816 shutdown_connections 2024-04-20T11:34:22.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:22.987+0000 7fba49c00640 1 -- 172.21.15.152:0/3139979816 wait complete. 2024-04-20T11:34:23.006 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-20T11:34:23.204 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:34:23.209 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:34:23.209 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.206+0000 7f4649800640 1 Processor -- start 2024-04-20T11:34:23.209 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.206+0000 7f4649800640 1 -- start start 2024-04-20T11:34:23.210 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.208+0000 7f4649800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f4644145f00 0x7f46440cd0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:23.210 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.208+0000 7f4649800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f46440cd610 0x7f464406e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:23.210 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.208+0000 7f4649800640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f464406f110 con 0x7f464406e900 2024-04-20T11:34:23.210 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.208+0000 7f4649800640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f46440cf9f0 con 0x7f4644145f00 2024-04-20T11:34:23.210 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.208+0000 7f4649800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f46440cda50 con 0x7f46440cd610 2024-04-20T11:34:23.210 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.208+0000 7f4642a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f4644145f00 0x7f46440cd0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:23.210 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.209+0000 7f4642a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f4644145f00 0x7f46440cd0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52414/0 (socket says 172.21.15.152:52414) 2024-04-20T11:34:23.210 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.209+0000 7f4642a00640 1 -- 172.21.15.152:0/2864229065 learned_addr learned my addr 172.21.15.152:0/2864229065 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:23.210 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.209+0000 7f4642000640 1 --2- 172.21.15.152:0/2864229065 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f46440cd610 0x7f464406e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:23.210 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.209+0000 7f4642a00640 1 -- 172.21.15.152:0/2864229065 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f46440cd610 msgr2=0x7f464406e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:23.210 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.209+0000 7f4642a00640 1 --2- 172.21.15.152:0/2864229065 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f46440cd610 0x7f464406e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:23.210 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.209+0000 7f4642a00640 1 -- 172.21.15.152:0/2864229065 >> v1:172.21.15.175:6789/0 conn(0x7f464406e900 legacy=0x7f46441601c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:23.210 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.209+0000 7f4642a00640 1 -- 172.21.15.152:0/2864229065 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f464406f3f0 con 0x7f4644145f00 2024-04-20T11:34:23.212 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.210+0000 7f4642a00640 1 --2- 172.21.15.152:0/2864229065 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4644145f00 0x7f46440cd0d0 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f463c009e50 tx=0x7f463c02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27846bf72f87463b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:23.212 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.211+0000 7f4641600640 1 -- 172.21.15.152:0/2864229065 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f463c02cb00 con 0x7f4644145f00 2024-04-20T11:34:23.212 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.211+0000 7f4641600640 1 -- 172.21.15.152:0/2864229065 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f463c02cca0 con 0x7f4644145f00 2024-04-20T11:34:23.212 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.211+0000 7f4641600640 1 -- 172.21.15.152:0/2864229065 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f463c034610 con 0x7f4644145f00 2024-04-20T11:34:23.213 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.211+0000 7f4649800640 1 -- 172.21.15.152:0/2864229065 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4644145f00 msgr2=0x7f46440cd0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:23.213 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.211+0000 7f4649800640 1 --2- 172.21.15.152:0/2864229065 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4644145f00 0x7f46440cd0d0 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f463c009e50 tx=0x7f463c02bf60 comp rx=0 tx=0).stop 2024-04-20T11:34:23.213 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.211+0000 7f4649800640 1 -- 172.21.15.152:0/2864229065 shutdown_connections 2024-04-20T11:34:23.213 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.211+0000 7f4649800640 1 --2- 172.21.15.152:0/2864229065 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f46440cd610 0x7f464406e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:23.213 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.211+0000 7f4649800640 1 --2- 172.21.15.152:0/2864229065 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4644145f00 0x7f46440cd0d0 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:23.213 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.211+0000 7f4649800640 1 -- 172.21.15.152:0/2864229065 >> 172.21.15.152:0/2864229065 conn(0x7f4644067cf0 msgr2=0x7f464406b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:23.213 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.212+0000 7f4649800640 1 -- 172.21.15.152:0/2864229065 shutdown_connections 2024-04-20T11:34:23.213 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.212+0000 7f4649800640 1 -- 172.21.15.152:0/2864229065 wait complete. 2024-04-20T11:34:23.214 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.213+0000 7f4649800640 1 Processor -- start 2024-04-20T11:34:23.214 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.214+0000 7f4649800640 1 -- start start 2024-04-20T11:34:23.216 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.215+0000 7f4649800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f464406e900 0x7f4644137ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:23.216 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.215+0000 7f4649800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f46440cd610 0x7f4644145030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:23.216 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.215+0000 7f4649800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4644145f00 0x7f46441f4490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:23.216 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.215+0000 7f4649800640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f46441462e0 con 0x7f4644145f00 2024-04-20T11:34:23.216 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.215+0000 7f4649800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f46440cf680 con 0x7f464406e900 2024-04-20T11:34:23.216 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.215+0000 7f4649800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f464406ed90 con 0x7f46440cd610 2024-04-20T11:34:23.216 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.215+0000 7f4643400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4644145f00 0x7f46441f4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:23.216 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.215+0000 7f4643400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4644145f00 0x7f46441f4490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52422/0 (socket says 172.21.15.152:52422) 2024-04-20T11:34:23.216 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.215+0000 7f4643400640 1 -- 172.21.15.152:0/880508277 learned_addr learned my addr 172.21.15.152:0/880508277 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:23.216 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.215+0000 7f4642000640 1 --2- 172.21.15.152:0/880508277 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f46440cd610 0x7f4644145030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:23.216 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.215+0000 7f4642a00640 1 --2- 172.21.15.152:0/880508277 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f464406e900 0x7f4644137ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:23.217 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.216+0000 7f4643400640 1 -- 172.21.15.152:0/880508277 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f46440cd610 msgr2=0x7f4644145030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:23.217 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.216+0000 7f4643400640 1 --2- 172.21.15.152:0/880508277 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f46440cd610 0x7f4644145030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:23.217 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.216+0000 7f4643400640 1 -- 172.21.15.152:0/880508277 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f464406e900 msgr2=0x7f4644137ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:23.217 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.216+0000 7f4643400640 1 --2- 172.21.15.152:0/880508277 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f464406e900 0x7f4644137ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:23.217 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.216+0000 7f4643400640 1 -- 172.21.15.152:0/880508277 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46441f4e00 con 0x7f4644145f00 2024-04-20T11:34:23.217 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.216+0000 7f4642a00640 1 --2- 172.21.15.152:0/880508277 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f464406e900 0x7f4644137ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:34:23.217 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.216+0000 7f4643400640 1 --2- 172.21.15.152:0/880508277 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4644145f00 0x7f46441f4490 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f4634001eb0 tx=0x7f463400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:23.217 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.216+0000 7f463aa00640 1 -- 172.21.15.152:0/880508277 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4634017070 con 0x7f4644145f00 2024-04-20T11:34:23.217 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.216+0000 7f463aa00640 1 -- 172.21.15.152:0/880508277 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46340052d0 con 0x7f4644145f00 2024-04-20T11:34:23.217 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.216+0000 7f4649800640 1 -- 172.21.15.152:0/880508277 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46441f50e0 con 0x7f4644145f00 2024-04-20T11:34:23.217 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.216+0000 7f463aa00640 1 -- 172.21.15.152:0/880508277 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4634003810 con 0x7f4644145f00 2024-04-20T11:34:23.217 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.216+0000 7f4649800640 1 -- 172.21.15.152:0/880508277 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4644138400 con 0x7f4644145f00 2024-04-20T11:34:23.218 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.218+0000 7f4649800640 1 -- 172.21.15.152:0/880508277 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46440689f0 con 0x7f4644145f00 2024-04-20T11:34:23.218 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.219+0000 7f463aa00640 1 -- 172.21.15.152:0/880508277 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 19) ==== 91397+0+0 (secure 0 0 0) 0x7f463401a070 con 0x7f4644145f00 2024-04-20T11:34:23.219 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.220+0000 7f463aa00640 1 --2- 172.21.15.152:0/880508277 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f462006cce0 0x7f462006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:23.219 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.220+0000 7f463aa00640 1 -- 172.21.15.152:0/880508277 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 5737+0+0 (secure 0 0 0) 0x7f463408b600 con 0x7f4644145f00 2024-04-20T11:34:23.220 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.220+0000 7f4642a00640 1 --2- 172.21.15.152:0/880508277 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f462006cce0 0x7f462006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:23.222 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.223+0000 7f4642a00640 1 --2- 172.21.15.152:0/880508277 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f462006cce0 0x7f462006f1a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f463c035000 tx=0x7f463c030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:23.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.230+0000 7f463aa00640 1 -- 172.21.15.152:0/880508277 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f4634059480 con 0x7f4644145f00 2024-04-20T11:34:23.464 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.464+0000 7f4649800640 1 -- 172.21.15.152:0/880508277 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f464415d1b0 con 0x7f4644145f00 2024-04-20T11:34:23.464 INFO:tasks.ceph.mon.a.smithi152.stderr:2024-04-20T11:34:23.465+0000 7f8229800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-20T11:34:23.468 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.469+0000 7f463aa00640 1 -- 172.21.15.152:0/880508277 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v68) ==== 161+0+0 (secure 0 0 0) 0x7f4634058d80 con 0x7f4644145f00 2024-04-20T11:34:23.468 INFO:teuthology.orchestra.run.smithi152.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-20T11:34:23.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.472+0000 7f4649800640 1 -- 172.21.15.152:0/880508277 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f462006cce0 msgr2=0x7f462006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:23.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.472+0000 7f4649800640 1 --2- 172.21.15.152:0/880508277 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f462006cce0 0x7f462006f1a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f463c035000 tx=0x7f463c030000 comp rx=0 tx=0).stop 2024-04-20T11:34:23.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.472+0000 7f4649800640 1 -- 172.21.15.152:0/880508277 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4644145f00 msgr2=0x7f46441f4490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:23.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.472+0000 7f4649800640 1 --2- 172.21.15.152:0/880508277 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4644145f00 0x7f46441f4490 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f4634001eb0 tx=0x7f463400c890 comp rx=0 tx=0).stop 2024-04-20T11:34:23.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.472+0000 7f4649800640 1 -- 172.21.15.152:0/880508277 shutdown_connections 2024-04-20T11:34:23.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.472+0000 7f4649800640 1 --2- 172.21.15.152:0/880508277 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f462006cce0 0x7f462006f1a0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:23.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.472+0000 7f4649800640 1 --2- 172.21.15.152:0/880508277 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4644145f00 0x7f46441f4490 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:23.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.472+0000 7f4649800640 1 --2- 172.21.15.152:0/880508277 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f46440cd610 0x7f4644145030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:23.473 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.472+0000 7f4649800640 1 --2- 172.21.15.152:0/880508277 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f464406e900 0x7f4644137ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:23.473 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.472+0000 7f4649800640 1 -- 172.21.15.152:0/880508277 >> 172.21.15.152:0/880508277 conn(0x7f4644067cf0 msgr2=0x7f464415f500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:23.473 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.472+0000 7f4649800640 1 -- 172.21.15.152:0/880508277 shutdown_connections 2024-04-20T11:34:23.473 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.472+0000 7f4649800640 1 -- 172.21.15.152:0/880508277 wait complete. 2024-04-20T11:34:23.493 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-20T11:34:23.565 INFO:tasks.ceph.mds.a.smithi152.stdout:starting mds.a at 2024-04-20T11:34:23.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.674+0000 7fd222800640 1 Processor -- start 2024-04-20T11:34:23.674 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.674+0000 7fd222800640 1 -- start start 2024-04-20T11:34:23.675 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.675+0000 7fd222800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd21c071cd0 0x7fd21c0720b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:23.675 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.675+0000 7fd222800640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd21c06fb40 con 0x7fd21c160220 2024-04-20T11:34:23.675 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.675+0000 7fd222800640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd21c0725f0 con 0x7fd21c141450 2024-04-20T11:34:23.675 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.675+0000 7fd222800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd21c13f8e0 con 0x7fd21c071cd0 2024-04-20T11:34:23.676 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.676+0000 7fd220a00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fd21c160220 0x7fd21c160680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:40288/0 (socket says 172.21.15.152:40288) 2024-04-20T11:34:23.676 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.676+0000 7fd220a00640 1 -- 172.21.15.152:0/898030407 learned_addr learned my addr 172.21.15.152:0/898030407 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:34:23.678 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.676+0000 7fd221400640 1 --2- 172.21.15.152:0/898030407 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd21c071cd0 0x7fd21c0720b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:23.678 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.676+0000 7fd21be00640 1 -- 172.21.15.152:0/898030407 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3444339507 0 0) 0x7fd21c06fb40 con 0x7fd21c160220 2024-04-20T11:34:23.678 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.676+0000 7fd21be00640 1 -- 172.21.15.152:0/898030407 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd204003520 con 0x7fd21c160220 2024-04-20T11:34:23.678 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.677+0000 7fd21be00640 1 -- 172.21.15.152:0/898030407 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 497391645 0 0) 0x7fd204003520 con 0x7fd21c160220 2024-04-20T11:34:23.678 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.677+0000 7fd21be00640 1 -- 172.21.15.152:0/898030407 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd21c071cd0 msgr2=0x7fd21c0720b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:23.678 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.677+0000 7fd21be00640 1 --2- 172.21.15.152:0/898030407 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd21c071cd0 0x7fd21c0720b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:23.678 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.677+0000 7fd21be00640 1 -- 172.21.15.152:0/898030407 >> v1:172.21.15.175:6789/0 conn(0x7fd21c141450 legacy=0x7fd21c1418e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:23.678 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.677+0000 7fd21be00640 1 -- 172.21.15.152:0/898030407 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd21c160f90 con 0x7fd21c160220 2024-04-20T11:34:23.678 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.677+0000 7fd21be00640 1 -- 172.21.15.152:0/898030407 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7fd208001670 con 0x7fd21c160220 2024-04-20T11:34:23.678 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.677+0000 7fd21be00640 1 -- 172.21.15.152:0/898030407 >> v1:172.21.15.152:6789/0 conn(0x7fd21c160220 legacy=0x7fd21c160680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:23.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.677+0000 7fd21be00640 1 --2- 172.21.15.152:0/898030407 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd2040039e0 0x7fd204003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:23.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.677+0000 7fd21be00640 1 --2- 172.21.15.152:0/898030407 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd204004800 0x7fd204004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:23.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.677+0000 7fd21be00640 1 --2- 172.21.15.152:0/898030407 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd21c071cd0 0x7fd204005270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:23.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.677+0000 7fd21be00640 1 -- 172.21.15.152:0/898030407 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fd21c13f8e0 con 0x7fd2040039e0 2024-04-20T11:34:23.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.677+0000 7fd21be00640 1 -- 172.21.15.152:0/898030407 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd20400bb50 con 0x7fd204004800 2024-04-20T11:34:23.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.677+0000 7fd21be00640 1 -- 172.21.15.152:0/898030407 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd20400bd50 con 0x7fd21c071cd0 2024-04-20T11:34:23.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.677+0000 7fd221400640 1 --2- 172.21.15.152:0/898030407 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd2040039e0 0x7fd204003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:23.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.678+0000 7fd220a00640 1 --2- 172.21.15.152:0/898030407 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd204004800 0x7fd204004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:23.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.678+0000 7fd221400640 1 -- 172.21.15.152:0/898030407 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd21c071cd0 msgr2=0x7fd204005270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:23.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.678+0000 7fd221400640 1 --2- 172.21.15.152:0/898030407 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd21c071cd0 0x7fd204005270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:23.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.678+0000 7fd221400640 1 -- 172.21.15.152:0/898030407 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd204004800 msgr2=0x7fd204004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:23.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.678+0000 7fd221400640 1 --2- 172.21.15.152:0/898030407 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd204004800 0x7fd204004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:23.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.678+0000 7fd221400640 1 -- 172.21.15.152:0/898030407 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd204003520 con 0x7fd2040039e0 2024-04-20T11:34:23.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.679+0000 7fd221400640 1 --2- 172.21.15.152:0/898030407 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd2040039e0 0x7fd204003e00 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7fd210009ac0 tx=0x7fd204005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d994a5c9f5d1ba10 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:23.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.679+0000 7fd21be00640 1 -- 172.21.15.152:0/898030407 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd210039070 con 0x7fd2040039e0 2024-04-20T11:34:23.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.679+0000 7fd21be00640 1 -- 172.21.15.152:0/898030407 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd21002c710 con 0x7fd2040039e0 2024-04-20T11:34:23.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.679+0000 7fd21be00640 1 -- 172.21.15.152:0/898030407 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd210009690 con 0x7fd2040039e0 2024-04-20T11:34:23.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.680+0000 7fd222800640 1 -- 172.21.15.152:0/898030407 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd2040039e0 msgr2=0x7fd204003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:23.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.680+0000 7fd222800640 1 --2- 172.21.15.152:0/898030407 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd2040039e0 0x7fd204003e00 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7fd210009ac0 tx=0x7fd204005830 comp rx=0 tx=0).stop 2024-04-20T11:34:23.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.680+0000 7fd221e00640 1 -- 172.21.15.152:0/898030407 reap_dead start 2024-04-20T11:34:23.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.680+0000 7fd222800640 1 -- 172.21.15.152:0/898030407 shutdown_connections 2024-04-20T11:34:23.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.680+0000 7fd222800640 1 -- 172.21.15.152:0/898030407 >> 172.21.15.152:0/898030407 conn(0x7fd21c067ae0 msgr2=0x7fd21c06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:23.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.680+0000 7fd222800640 1 -- 172.21.15.152:0/898030407 shutdown_connections 2024-04-20T11:34:23.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.680+0000 7fd222800640 1 -- 172.21.15.152:0/898030407 wait complete. 2024-04-20T11:34:23.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.682+0000 7fd222800640 1 Processor -- start 2024-04-20T11:34:23.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.682+0000 7fd222800640 1 -- start start 2024-04-20T11:34:23.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.683+0000 7fd222800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2040039e0 0x7fd21c156290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:23.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.683+0000 7fd222800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd21c1f2540 0x7fd21c15bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:23.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.683+0000 7fd222800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd21c157bb0 0x7fd21c06ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:23.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.683+0000 7fd222800640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fd21c06e350 con 0x7fd21c1f2540 2024-04-20T11:34:23.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.683+0000 7fd222800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd21c06e070 con 0x7fd2040039e0 2024-04-20T11:34:23.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.683+0000 7fd222800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd21c06e630 con 0x7fd21c157bb0 2024-04-20T11:34:23.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.683+0000 7fd220a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd21c1f2540 0x7fd21c15bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:23.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.683+0000 7fd220a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd21c1f2540 0x7fd21c15bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52468/0 (socket says 172.21.15.152:52468) 2024-04-20T11:34:23.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.683+0000 7fd220a00640 1 -- 172.21.15.152:0/1000982079 learned_addr learned my addr 172.21.15.152:0/1000982079 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:23.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.683+0000 7fd221400640 1 --2- 172.21.15.152:0/1000982079 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2040039e0 0x7fd21c156290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:23.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.684+0000 7fd221e00640 1 --2- 172.21.15.152:0/1000982079 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd21c157bb0 0x7fd21c06ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:23.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.684+0000 7fd220a00640 1 -- 172.21.15.152:0/1000982079 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd21c157bb0 msgr2=0x7fd21c06ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:23.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.684+0000 7fd220a00640 1 --2- 172.21.15.152:0/1000982079 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd21c157bb0 0x7fd21c06ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:23.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.684+0000 7fd220a00640 1 -- 172.21.15.152:0/1000982079 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2040039e0 msgr2=0x7fd21c156290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:23.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.684+0000 7fd220a00640 1 --2- 172.21.15.152:0/1000982079 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2040039e0 0x7fd21c156290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:23.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.684+0000 7fd220a00640 1 -- 172.21.15.152:0/1000982079 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd21c158120 con 0x7fd21c1f2540 2024-04-20T11:34:23.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.684+0000 7fd220a00640 1 --2- 172.21.15.152:0/1000982079 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd21c1f2540 0x7fd21c15bfd0 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7fd20800d810 tx=0x7fd20800dcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:23.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.684+0000 7fd219600640 1 -- 172.21.15.152:0/1000982079 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd208019070 con 0x7fd21c1f2540 2024-04-20T11:34:23.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.684+0000 7fd219600640 1 -- 172.21.15.152:0/1000982079 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd208007560 con 0x7fd21c1f2540 2024-04-20T11:34:23.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.684+0000 7fd219600640 1 -- 172.21.15.152:0/1000982079 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd208004a00 con 0x7fd21c1f2540 2024-04-20T11:34:23.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.684+0000 7fd222800640 1 -- 172.21.15.152:0/1000982079 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd21c158400 con 0x7fd21c1f2540 2024-04-20T11:34:23.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.684+0000 7fd222800640 1 -- 172.21.15.152:0/1000982079 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd21c158f40 con 0x7fd21c1f2540 2024-04-20T11:34:23.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.686+0000 7fd222800640 1 -- 172.21.15.152:0/1000982079 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd21c075e60 con 0x7fd21c1f2540 2024-04-20T11:34:23.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.686+0000 7fd219600640 1 -- 172.21.15.152:0/1000982079 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 19) ==== 91397+0+0 (secure 0 0 0) 0x7fd208004ba0 con 0x7fd21c1f2540 2024-04-20T11:34:23.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.687+0000 7fd219600640 1 --2- 172.21.15.152:0/1000982079 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd1fc06cce0 0x7fd1fc06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:23.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.687+0000 7fd219600640 1 -- 172.21.15.152:0/1000982079 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 5853+0+0 (secure 0 0 0) 0x7fd20808bea0 con 0x7fd21c1f2540 2024-04-20T11:34:23.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.688+0000 7fd221400640 1 --2- 172.21.15.152:0/1000982079 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd1fc06cce0 0x7fd1fc06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:23.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.689+0000 7fd221400640 1 --2- 172.21.15.152:0/1000982079 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd1fc06cce0 0x7fd1fc06f1a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd210009ac0 tx=0x7fd210030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:23.691 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.691+0000 7fd219600640 1 -- 172.21.15.152:0/1000982079 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fd20804fb90 con 0x7fd21c1f2540 2024-04-20T11:34:23.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:23.934+0000 7fd222800640 1 -- 172.21.15.152:0/1000982079 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fd21c159110 con 0x7fd21c1f2540 2024-04-20T11:34:24.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:24.472+0000 7fd219600640 1 -- 172.21.15.152:0/1000982079 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v69) ==== 104+0+0 (secure 0 0 0) 0x7fd208059ba0 con 0x7fd21c1f2540 2024-04-20T11:34:24.476 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:24.476+0000 7fd222800640 1 -- 172.21.15.152:0/1000982079 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd1fc06cce0 msgr2=0x7fd1fc06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:24.476 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:24.476+0000 7fd222800640 1 --2- 172.21.15.152:0/1000982079 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd1fc06cce0 0x7fd1fc06f1a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd210009ac0 tx=0x7fd210030000 comp rx=0 tx=0).stop 2024-04-20T11:34:24.476 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:24.476+0000 7fd222800640 1 -- 172.21.15.152:0/1000982079 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd21c1f2540 msgr2=0x7fd21c15bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:24.476 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:24.477+0000 7fd222800640 1 --2- 172.21.15.152:0/1000982079 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd21c1f2540 0x7fd21c15bfd0 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7fd20800d810 tx=0x7fd20800dcd0 comp rx=0 tx=0).stop 2024-04-20T11:34:24.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:24.477+0000 7fd222800640 1 -- 172.21.15.152:0/1000982079 shutdown_connections 2024-04-20T11:34:24.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:24.477+0000 7fd222800640 1 --2- 172.21.15.152:0/1000982079 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd1fc06cce0 0x7fd1fc06f1a0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:24.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:24.477+0000 7fd222800640 1 --2- 172.21.15.152:0/1000982079 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd21c157bb0 0x7fd21c06ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:24.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:24.477+0000 7fd222800640 1 --2- 172.21.15.152:0/1000982079 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd21c1f2540 0x7fd21c15bfd0 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:24.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:24.477+0000 7fd222800640 1 --2- 172.21.15.152:0/1000982079 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2040039e0 0x7fd21c156290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:24.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:24.477+0000 7fd222800640 1 -- 172.21.15.152:0/1000982079 >> 172.21.15.152:0/1000982079 conn(0x7fd21c067ae0 msgr2=0x7fd21c160680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:24.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:24.477+0000 7fd222800640 1 -- 172.21.15.152:0/1000982079 shutdown_connections 2024-04-20T11:34:24.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:24.477+0000 7fd222800640 1 -- 172.21.15.152:0/1000982079 wait complete. 2024-04-20T11:34:24.500 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-20T11:34:24.704 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:24.704+0000 7f0ced800640 1 Processor -- start 2024-04-20T11:34:24.704 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:24.704+0000 7f0ced800640 1 -- start start 2024-04-20T11:34:24.705 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:24.706+0000 7f0ced800640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f0ce80b4ac0 0x7f0ce80b3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:24.705 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:24.706+0000 7f0ced800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0ce815d1f0 0x7f0ce80d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:24.705 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:24.706+0000 7f0ced800640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ce815da60 con 0x7f0ce80d05c0 2024-04-20T11:34:24.706 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:24.706+0000 7f0ced800640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f0ce80d0f60 con 0x7f0ce80b4ac0 2024-04-20T11:34:24.706 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:24.706+0000 7f0ced800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0ce80b3740 con 0x7f0ce815d1f0 2024-04-20T11:34:24.706 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:24.706+0000 7f0cece00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0ce815d1f0 0x7f0ce80d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:24.706 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:24.707+0000 7f0cece00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0ce815d1f0 0x7f0ce80d0a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.152:33238/0 (socket says 172.21.15.152:33238) 2024-04-20T11:34:24.706 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:24.707+0000 7f0cece00640 1 -- 172.21.15.152:0/2794306705 learned_addr learned my addr 172.21.15.152:0/2794306705 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:24.706 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:24.707+0000 7f0ce7e00640 1 --2- 172.21.15.152:0/2794306705 >> v2:172.21.15.175:3300/0 conn(0x7f0ce80b4ac0 0x7f0ce80b3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:24.707 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:24.707+0000 7f0cece00640 1 -- 172.21.15.152:0/2794306705 >> v2:172.21.15.175:3300/0 conn(0x7f0ce80b4ac0 msgr2=0x7f0ce80b3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:24.707 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:24.707+0000 7f0cece00640 1 --2- 172.21.15.152:0/2794306705 >> v2:172.21.15.175:3300/0 conn(0x7f0ce80b4ac0 0x7f0ce80b3bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:24.707 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:24.707+0000 7f0cece00640 1 -- 172.21.15.152:0/2794306705 >> v1:172.21.15.175:6789/0 conn(0x7f0ce80d05c0 legacy=0x7f0ce815cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:24.707 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:24.707+0000 7f0cece00640 1 -- 172.21.15.152:0/2794306705 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ce815d780 con 0x7f0ce815d1f0 2024-04-20T11:34:24.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:24.709+0000 7f0cece00640 1 --2- 172.21.15.152:0/2794306705 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0ce815d1f0 0x7f0ce80d0a20 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0ce0009870 tx=0x7f0ce002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9eafc929d2c31275 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:24.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:24.709+0000 7f0ce7400640 1 -- 172.21.15.152:0/2794306705 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce0034070 con 0x7f0ce815d1f0 2024-04-20T11:34:24.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:24.709+0000 7f0ce7400640 1 -- 172.21.15.152:0/2794306705 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ce0030840 con 0x7f0ce815d1f0 2024-04-20T11:34:24.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:24.709+0000 7f0ce7400640 1 -- 172.21.15.152:0/2794306705 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce0030c10 con 0x7f0ce815d1f0 2024-04-20T11:34:24.710 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:24.710+0000 7f0ced800640 1 -- 172.21.15.152:0/2794306705 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0ce815d1f0 msgr2=0x7f0ce80d0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:24.710 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:24.710+0000 7f0ced800640 1 --2- 172.21.15.152:0/2794306705 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0ce815d1f0 0x7f0ce80d0a20 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0ce0009870 tx=0x7f0ce002b980 comp rx=0 tx=0).stop 2024-04-20T11:34:24.710 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:24.710+0000 7f0ced800640 1 -- 172.21.15.152:0/2794306705 shutdown_connections 2024-04-20T11:34:24.710 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:24.710+0000 7f0ced800640 1 --2- 172.21.15.152:0/2794306705 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0ce815d1f0 0x7f0ce80d0a20 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:24.710 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:24.710+0000 7f0ced800640 1 --2- 172.21.15.152:0/2794306705 >> v2:172.21.15.175:3300/0 conn(0x7f0ce80b4ac0 0x7f0ce80b3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:24.710 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:24.710+0000 7f0ced800640 1 -- 172.21.15.152:0/2794306705 >> 172.21.15.152:0/2794306705 conn(0x7f0ce806bda0 msgr2=0x7f0ce806c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:24.710 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:24.710+0000 7f0ced800640 1 -- 172.21.15.152:0/2794306705 shutdown_connections 2024-04-20T11:34:24.710 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:24.710+0000 7f0ced800640 1 -- 172.21.15.152:0/2794306705 wait complete. 2024-04-20T11:34:24.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:24.712+0000 7f0ced800640 1 Processor -- start 2024-04-20T11:34:24.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:24.713+0000 7f0ced800640 1 -- start start 2024-04-20T11:34:24.716 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:24.714+0000 7f0ced800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0ce80b4ac0 0x7f0ce814b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:24.717 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:24.714+0000 7f0ced800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0ce80d05c0 0x7f0ce8072c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:24.717 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:24.714+0000 7f0ced800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ce815d1f0 0x7f0ce8073150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:24.717 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:24.714+0000 7f0ced800640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f0ce80b4ea0 con 0x7f0ce80b4ac0 2024-04-20T11:34:24.717 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:24.714+0000 7f0ced800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0ce80b33d0 con 0x7f0ce815d1f0 2024-04-20T11:34:24.717 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:24.714+0000 7f0ced800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0ce80ccb80 con 0x7f0ce80d05c0 2024-04-20T11:34:24.717 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:24.714+0000 7f0ce7e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0ce80b4ac0 0x7f0ce814b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:24.717 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:24.714+0000 7f0ce7e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0ce80b4ac0 0x7f0ce814b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52476/0 (socket says 172.21.15.152:52476) 2024-04-20T11:34:24.717 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:24.714+0000 7f0ce7e00640 1 -- 172.21.15.152:0/3521050338 learned_addr learned my addr 172.21.15.152:0/3521050338 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:24.717 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:24.714+0000 7f0ce7e00640 1 -- 172.21.15.152:0/3521050338 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0ce80d05c0 msgr2=0x7f0ce8072c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:24.718 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:24.714+0000 7f0ce7e00640 1 --2- 172.21.15.152:0/3521050338 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0ce80d05c0 0x7f0ce8072c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:24.718 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:24.714+0000 7f0ce7e00640 1 -- 172.21.15.152:0/3521050338 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ce815d1f0 msgr2=0x7f0ce8073150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:24.718 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:24.714+0000 7f0ce7e00640 1 --2- 172.21.15.152:0/3521050338 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ce815d1f0 0x7f0ce8073150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:24.718 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:24.714+0000 7f0ce7e00640 1 -- 172.21.15.152:0/3521050338 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ce814b990 con 0x7f0ce80b4ac0 2024-04-20T11:34:24.718 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:24.715+0000 7f0ce7e00640 1 --2- 172.21.15.152:0/3521050338 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0ce80b4ac0 0x7f0ce814b450 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f0cd8009db0 tx=0x7f0cd800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:24.718 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:24.715+0000 7f0ce4c00640 1 -- 172.21.15.152:0/3521050338 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd8018020 con 0x7f0ce80b4ac0 2024-04-20T11:34:24.718 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:24.715+0000 7f0ce4c00640 1 -- 172.21.15.152:0/3521050338 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cd8014480 con 0x7f0ce80b4ac0 2024-04-20T11:34:24.718 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:24.715+0000 7f0ced800640 1 -- 172.21.15.152:0/3521050338 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ce809bc10 con 0x7f0ce80b4ac0 2024-04-20T11:34:24.718 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:24.715+0000 7f0ce4c00640 1 -- 172.21.15.152:0/3521050338 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd8014850 con 0x7f0ce80b4ac0 2024-04-20T11:34:24.719 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:24.715+0000 7f0ced800640 1 -- 172.21.15.152:0/3521050338 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ce809c4b0 con 0x7f0ce80b4ac0 2024-04-20T11:34:24.719 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:24.716+0000 7f0ced800640 1 -- 172.21.15.152:0/3521050338 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ce80689f0 con 0x7f0ce80b4ac0 2024-04-20T11:34:24.719 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:24.718+0000 7f0ce4c00640 1 -- 172.21.15.152:0/3521050338 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 19) ==== 91397+0+0 (secure 0 0 0) 0x7f0cd8005d10 con 0x7f0ce80b4ac0 2024-04-20T11:34:24.720 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:24.719+0000 7f0ce4c00640 1 --2- 172.21.15.152:0/3521050338 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f0cc406cce0 0x7f0cc406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:24.720 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:24.719+0000 7f0ce4c00640 1 -- 172.21.15.152:0/3521050338 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 5853+0+0 (secure 0 0 0) 0x7f0cd808cf70 con 0x7f0ce80b4ac0 2024-04-20T11:34:24.720 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:24.719+0000 7f0cdf400640 1 --2- 172.21.15.152:0/3521050338 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f0cc406cce0 0x7f0cc406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:24.720 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:24.721+0000 7f0cdf400640 1 --2- 172.21.15.152:0/3521050338 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f0cc406cce0 0x7f0cc406f1a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f0cd0000f60 tx=0x7f0cd00079d0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:24.723 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:24.724+0000 7f0ce4c00640 1 -- 172.21.15.152:0/3521050338 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f0cd8022070 con 0x7f0ce80b4ac0 2024-04-20T11:34:24.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:24.911+0000 7f0ced800640 1 -- 172.21.15.152:0/3521050338 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f0ce806eef0 con 0x7f0ce80b4ac0 2024-04-20T11:34:25.485 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:25.485+0000 7f0ce4c00640 1 -- 172.21.15.152:0/3521050338 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v109) ==== 181+0+0 (secure 0 0 0) 0x7f0cd8010040 con 0x7f0ce80b4ac0 2024-04-20T11:34:25.485 INFO:teuthology.orchestra.run.smithi152.stderr:pool 'cephfs_metadata' removed 2024-04-20T11:34:25.488 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:25.489+0000 7f0ced800640 1 -- 172.21.15.152:0/3521050338 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f0cc406cce0 msgr2=0x7f0cc406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:25.488 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:25.489+0000 7f0ced800640 1 --2- 172.21.15.152:0/3521050338 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f0cc406cce0 0x7f0cc406f1a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f0cd0000f60 tx=0x7f0cd00079d0 comp rx=0 tx=0).stop 2024-04-20T11:34:25.488 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:25.489+0000 7f0ced800640 1 -- 172.21.15.152:0/3521050338 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0ce80b4ac0 msgr2=0x7f0ce814b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:25.488 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:25.489+0000 7f0ced800640 1 --2- 172.21.15.152:0/3521050338 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0ce80b4ac0 0x7f0ce814b450 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f0cd8009db0 tx=0x7f0cd800c490 comp rx=0 tx=0).stop 2024-04-20T11:34:25.489 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:25.489+0000 7f0ced800640 1 -- 172.21.15.152:0/3521050338 shutdown_connections 2024-04-20T11:34:25.489 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:25.489+0000 7f0ced800640 1 --2- 172.21.15.152:0/3521050338 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f0cc406cce0 0x7f0cc406f1a0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:25.489 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:25.489+0000 7f0ced800640 1 --2- 172.21.15.152:0/3521050338 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ce815d1f0 0x7f0ce8073150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:25.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:25.489+0000 7f0ced800640 1 --2- 172.21.15.152:0/3521050338 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0ce80d05c0 0x7f0ce8072c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:25.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:25.489+0000 7f0ced800640 1 --2- 172.21.15.152:0/3521050338 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0ce80b4ac0 0x7f0ce814b450 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:25.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:25.489+0000 7f0ced800640 1 -- 172.21.15.152:0/3521050338 >> 172.21.15.152:0/3521050338 conn(0x7f0ce806bda0 msgr2=0x7f0ce806f410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:25.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:25.489+0000 7f0ced800640 1 -- 172.21.15.152:0/3521050338 shutdown_connections 2024-04-20T11:34:25.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:25.490+0000 7f0ced800640 1 -- 172.21.15.152:0/3521050338 wait complete. 2024-04-20T11:34:25.517 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-20T11:34:25.698 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:25.699+0000 7fec21200640 1 Processor -- start 2024-04-20T11:34:25.699 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:25.699+0000 7fec21200640 1 -- start start 2024-04-20T11:34:25.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:25.700+0000 7fec21200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fec1c0cd8e0 0x7fec1c06e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:25.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:25.700+0000 7fec21200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fec1c0cf220 0x7fec1c0b4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:25.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:25.700+0000 7fec21200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fec1c144d70 0x7fec1c0b4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:25.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:25.700+0000 7fec21200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fec1c0d0240 con 0x7fec1c144d70 2024-04-20T11:34:25.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:25.700+0000 7fec21200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fec1c06eab0 con 0x7fec1c0cd8e0 2024-04-20T11:34:25.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:25.700+0000 7fec21200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fec1c0cf6c0 con 0x7fec1c0cf220 2024-04-20T11:34:25.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:25.701+0000 7fec1a800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fec1c144d70 0x7fec1c0b4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:25.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:25.701+0000 7fec1a800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fec1c144d70 0x7fec1c0b4bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52490/0 (socket says 172.21.15.152:52490) 2024-04-20T11:34:25.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:25.701+0000 7fec1a800640 1 -- 172.21.15.152:0/381754339 learned_addr learned my addr 172.21.15.152:0/381754339 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:25.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:25.701+0000 7fec11400640 1 --2- 172.21.15.152:0/381754339 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fec1c0cf220 0x7fec1c0b4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:25.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:25.701+0000 7fec19e00640 1 --2- 172.21.15.152:0/381754339 >> v2:172.21.15.175:3300/0 conn(0x7fec1c0cd8e0 0x7fec1c06e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:25.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:25.701+0000 7fec1a800640 1 -- 172.21.15.152:0/381754339 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fec1c0cf220 msgr2=0x7fec1c0b4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:25.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:25.701+0000 7fec1a800640 1 --2- 172.21.15.152:0/381754339 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fec1c0cf220 0x7fec1c0b4680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:25.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:25.701+0000 7fec1a800640 1 -- 172.21.15.152:0/381754339 >> v2:172.21.15.175:3300/0 conn(0x7fec1c0cd8e0 msgr2=0x7fec1c06e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:25.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:25.701+0000 7fec1a800640 1 --2- 172.21.15.152:0/381754339 >> v2:172.21.15.175:3300/0 conn(0x7fec1c0cd8e0 0x7fec1c06e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:25.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:25.701+0000 7fec1a800640 1 -- 172.21.15.152:0/381754339 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec1c0cf9a0 con 0x7fec1c144d70 2024-04-20T11:34:25.703 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:25.701+0000 7fec19e00640 1 --2- 172.21.15.152:0/381754339 >> v2:172.21.15.175:3300/0 conn(0x7fec1c0cd8e0 0x7fec1c06e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:34:25.703 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:25.703+0000 7fec1a800640 1 --2- 172.21.15.152:0/381754339 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fec1c144d70 0x7fec1c0b4bc0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7fec1c073680 tx=0x7fec1402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81ea6aff0285bb84 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:25.704 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:25.703+0000 7fec19400640 1 -- 172.21.15.152:0/381754339 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec1402c9e0 con 0x7fec1c144d70 2024-04-20T11:34:25.704 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:25.703+0000 7fec19400640 1 -- 172.21.15.152:0/381754339 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec1402cb80 con 0x7fec1c144d70 2024-04-20T11:34:25.704 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:25.703+0000 7fec19400640 1 -- 172.21.15.152:0/381754339 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec140343f0 con 0x7fec1c144d70 2024-04-20T11:34:25.704 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:25.704+0000 7fec21200640 1 -- 172.21.15.152:0/381754339 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fec1c144d70 msgr2=0x7fec1c0b4bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:25.704 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:25.704+0000 7fec21200640 1 --2- 172.21.15.152:0/381754339 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fec1c144d70 0x7fec1c0b4bc0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7fec1c073680 tx=0x7fec1402be40 comp rx=0 tx=0).stop 2024-04-20T11:34:25.704 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:25.704+0000 7fec21200640 1 -- 172.21.15.152:0/381754339 shutdown_connections 2024-04-20T11:34:25.704 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:25.704+0000 7fec21200640 1 --2- 172.21.15.152:0/381754339 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fec1c144d70 0x7fec1c0b4bc0 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:25.704 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:25.704+0000 7fec21200640 1 --2- 172.21.15.152:0/381754339 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fec1c0cf220 0x7fec1c0b4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:25.705 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:25.704+0000 7fec21200640 1 --2- 172.21.15.152:0/381754339 >> v2:172.21.15.175:3300/0 conn(0x7fec1c0cd8e0 0x7fec1c06e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:25.705 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:25.704+0000 7fec21200640 1 -- 172.21.15.152:0/381754339 >> 172.21.15.152:0/381754339 conn(0x7fec1c06b980 msgr2=0x7fec1c06bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:25.705 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:25.704+0000 7fec21200640 1 -- 172.21.15.152:0/381754339 shutdown_connections 2024-04-20T11:34:25.705 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:25.704+0000 7fec21200640 1 -- 172.21.15.152:0/381754339 wait complete. 2024-04-20T11:34:25.705 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:25.706+0000 7fec21200640 1 Processor -- start 2024-04-20T11:34:25.706 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:25.706+0000 7fec21200640 1 -- start start 2024-04-20T11:34:25.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:25.707+0000 7fec21200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fec1c0cd8e0 0x7fec1c1e8170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:25.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:25.707+0000 7fec21200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fec1c0cf220 0x7fec1c141170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:25.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:25.707+0000 7fec21200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fec1c144d70 0x7fec1c15cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:25.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:25.707+0000 7fec21200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fec1c0cdcc0 con 0x7fec1c0cd8e0 2024-04-20T11:34:25.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:25.707+0000 7fec21200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fec1c0cff60 con 0x7fec1c144d70 2024-04-20T11:34:25.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:25.707+0000 7fec21200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fec1c1447b0 con 0x7fec1c0cf220 2024-04-20T11:34:25.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:25.707+0000 7fec19e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fec1c0cd8e0 0x7fec1c1e8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:25.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:25.707+0000 7fec19e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fec1c0cd8e0 0x7fec1c1e8170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52502/0 (socket says 172.21.15.152:52502) 2024-04-20T11:34:25.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:25.707+0000 7fec19e00640 1 -- 172.21.15.152:0/3213876251 learned_addr learned my addr 172.21.15.152:0/3213876251 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:25.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:25.708+0000 7fec19e00640 1 -- 172.21.15.152:0/3213876251 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fec1c0cf220 msgr2=0x7fec1c141170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:25.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:25.708+0000 7fec11400640 1 --2- 172.21.15.152:0/3213876251 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fec1c0cf220 0x7fec1c141170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:25.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:25.708+0000 7fec19e00640 1 --2- 172.21.15.152:0/3213876251 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fec1c0cf220 0x7fec1c141170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:25.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:25.708+0000 7fec19e00640 1 -- 172.21.15.152:0/3213876251 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fec1c144d70 msgr2=0x7fec1c15cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:25.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:25.708+0000 7fec19e00640 1 --2- 172.21.15.152:0/3213876251 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fec1c144d70 0x7fec1c15cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:25.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:25.708+0000 7fec19e00640 1 -- 172.21.15.152:0/3213876251 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec1c0aa330 con 0x7fec1c0cd8e0 2024-04-20T11:34:25.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:25.708+0000 7fec19e00640 1 --2- 172.21.15.152:0/3213876251 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fec1c0cd8e0 0x7fec1c1e8170 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7fec0c00c7f0 tx=0x7fec0c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:25.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:25.708+0000 7fec12a00640 1 -- 172.21.15.152:0/3213876251 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec0c015070 con 0x7fec1c0cd8e0 2024-04-20T11:34:25.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:25.708+0000 7fec12a00640 1 -- 172.21.15.152:0/3213876251 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec0c0077c0 con 0x7fec1c0cd8e0 2024-04-20T11:34:25.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:25.708+0000 7fec21200640 1 -- 172.21.15.152:0/3213876251 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec1c1e70d0 con 0x7fec1c0cd8e0 2024-04-20T11:34:25.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:25.708+0000 7fec12a00640 1 -- 172.21.15.152:0/3213876251 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec0c007b70 con 0x7fec1c0cd8e0 2024-04-20T11:34:25.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:25.709+0000 7fec21200640 1 -- 172.21.15.152:0/3213876251 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec1c1e7690 con 0x7fec1c0cd8e0 2024-04-20T11:34:25.710 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:25.709+0000 7fec21200640 1 -- 172.21.15.152:0/3213876251 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec1c0ccea0 con 0x7fec1c0cd8e0 2024-04-20T11:34:25.710 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:25.710+0000 7fec12a00640 1 -- 172.21.15.152:0/3213876251 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 19) ==== 91397+0+0 (secure 0 0 0) 0x7fec0c018070 con 0x7fec1c0cd8e0 2024-04-20T11:34:25.714 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:25.711+0000 7fec12a00640 1 --2- 172.21.15.152:0/3213876251 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7febf806cce0 0x7febf806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:25.714 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:25.711+0000 7fec12a00640 1 -- 172.21.15.152:0/3213876251 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5415+0+0 (secure 0 0 0) 0x7fec0c08c2f0 con 0x7fec1c0cd8e0 2024-04-20T11:34:25.714 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:25.712+0000 7fec11400640 1 --2- 172.21.15.152:0/3213876251 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7febf806cce0 0x7febf806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:25.714 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:25.713+0000 7fec11400640 1 --2- 172.21.15.152:0/3213876251 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7febf806cce0 0x7febf806f1a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fec0400b7e0 tx=0x7fec04008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:25.715 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:25.716+0000 7fec12a00640 1 -- 172.21.15.152:0/3213876251 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fec1c0ccea0 con 0x7fec1c0cd8e0 2024-04-20T11:34:25.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:25.902+0000 7fec21200640 1 -- 172.21.15.152:0/3213876251 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fec1c068a60 con 0x7fec1c0cd8e0 2024-04-20T11:34:26.489 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:26.489+0000 7fec12a00640 1 -- 172.21.15.152:0/3213876251 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v110) ==== 169+0+0 (secure 0 0 0) 0x7fec0c05a1a0 con 0x7fec1c0cd8e0 2024-04-20T11:34:26.489 INFO:teuthology.orchestra.run.smithi152.stderr:pool 'cephfs_data' removed 2024-04-20T11:34:26.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:26.493+0000 7fec21200640 1 -- 172.21.15.152:0/3213876251 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7febf806cce0 msgr2=0x7febf806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:26.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:26.493+0000 7fec21200640 1 --2- 172.21.15.152:0/3213876251 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7febf806cce0 0x7febf806f1a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fec0400b7e0 tx=0x7fec04008040 comp rx=0 tx=0).stop 2024-04-20T11:34:26.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:26.493+0000 7fec21200640 1 -- 172.21.15.152:0/3213876251 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fec1c0cd8e0 msgr2=0x7fec1c1e8170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:26.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:26.493+0000 7fec21200640 1 --2- 172.21.15.152:0/3213876251 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fec1c0cd8e0 0x7fec1c1e8170 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7fec0c00c7f0 tx=0x7fec0c00ccb0 comp rx=0 tx=0).stop 2024-04-20T11:34:26.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:26.493+0000 7fec21200640 1 -- 172.21.15.152:0/3213876251 shutdown_connections 2024-04-20T11:34:26.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:26.493+0000 7fec21200640 1 --2- 172.21.15.152:0/3213876251 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7febf806cce0 0x7febf806f1a0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:26.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:26.493+0000 7fec21200640 1 --2- 172.21.15.152:0/3213876251 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fec1c144d70 0x7fec1c15cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:26.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:26.493+0000 7fec21200640 1 --2- 172.21.15.152:0/3213876251 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fec1c0cf220 0x7fec1c141170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:26.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:26.493+0000 7fec21200640 1 --2- 172.21.15.152:0/3213876251 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fec1c0cd8e0 0x7fec1c1e8170 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:26.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:26.493+0000 7fec21200640 1 -- 172.21.15.152:0/3213876251 >> 172.21.15.152:0/3213876251 conn(0x7fec1c06b980 msgr2=0x7fec1c0b4920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:26.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:26.494+0000 7fec21200640 1 -- 172.21.15.152:0/3213876251 shutdown_connections 2024-04-20T11:34:26.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:26.495+0000 7fec21200640 1 -- 172.21.15.152:0/3213876251 wait complete. 2024-04-20T11:34:26.522 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-20T11:34:26.719 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:26.719+0000 7fa3da600640 1 Processor -- start 2024-04-20T11:34:26.719 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:26.719+0000 7fa3da600640 1 -- start start 2024-04-20T11:34:26.721 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:26.721+0000 7fa3da600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa3d4073210 0x7fa3d406c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:26.721 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:26.721+0000 7fa3da600640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fa3d40af250 0x7fa3d406ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:26.722 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:26.721+0000 7fa3da600640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3d415fdf0 con 0x7fa3d40aee70 2024-04-20T11:34:26.722 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:26.721+0000 7fa3da600640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fa3d415f830 con 0x7fa3d40af250 2024-04-20T11:34:26.722 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:26.721+0000 7fa3da600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa3d415fb10 con 0x7fa3d4073210 2024-04-20T11:34:26.722 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:26.722+0000 7fa3d3e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fa3d40af250 0x7fa3d406ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:26.722 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:26.722+0000 7fa3d3e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fa3d40af250 0x7fa3d406ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:33428/0 (socket says 172.21.15.152:33428) 2024-04-20T11:34:26.722 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:26.722+0000 7fa3d3e00640 1 -- 172.21.15.152:0/1711720812 learned_addr learned my addr 172.21.15.152:0/1711720812 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:26.722 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:26.722+0000 7fa3d3e00640 1 -- 172.21.15.152:0/1711720812 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa3d4073210 msgr2=0x7fa3d406c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:26.722 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:26.722+0000 7fa3d3e00640 1 --2- 172.21.15.152:0/1711720812 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa3d4073210 0x7fa3d406c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:26.722 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:26.722+0000 7fa3d3e00640 1 -- 172.21.15.152:0/1711720812 >> v1:172.21.15.175:6789/0 conn(0x7fa3d40aee70 legacy=0x7fa3d415ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:26.722 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:26.722+0000 7fa3d3e00640 1 -- 172.21.15.152:0/1711720812 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3d41600f0 con 0x7fa3d40af250 2024-04-20T11:34:26.723 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:26.723+0000 7fa3d3e00640 1 --2- 172.21.15.152:0/1711720812 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa3d40af250 0x7fa3d406ced0 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc009e50 tx=0x7fa3cc02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=248c9741820ae639 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:26.723 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:26.723+0000 7fa3d2a00640 1 -- 172.21.15.152:0/1711720812 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc02ca70 con 0x7fa3d40af250 2024-04-20T11:34:26.723 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:26.724+0000 7fa3d2a00640 1 -- 172.21.15.152:0/1711720812 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3cc02cc10 con 0x7fa3d40af250 2024-04-20T11:34:26.723 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:26.724+0000 7fa3d2a00640 1 -- 172.21.15.152:0/1711720812 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc0343f0 con 0x7fa3d40af250 2024-04-20T11:34:26.724 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:26.724+0000 7fa3da600640 1 -- 172.21.15.152:0/1711720812 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa3d40af250 msgr2=0x7fa3d406ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:26.724 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:26.724+0000 7fa3da600640 1 --2- 172.21.15.152:0/1711720812 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa3d40af250 0x7fa3d406ced0 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc009e50 tx=0x7fa3cc02bed0 comp rx=0 tx=0).stop 2024-04-20T11:34:26.724 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:26.724+0000 7fa3da600640 1 -- 172.21.15.152:0/1711720812 shutdown_connections 2024-04-20T11:34:26.724 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:26.724+0000 7fa3da600640 1 --2- 172.21.15.152:0/1711720812 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa3d40af250 0x7fa3d406ced0 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:26.725 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:26.724+0000 7fa3da600640 1 --2- 172.21.15.152:0/1711720812 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa3d4073210 0x7fa3d406c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:26.725 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:26.724+0000 7fa3da600640 1 -- 172.21.15.152:0/1711720812 >> 172.21.15.152:0/1711720812 conn(0x7fa3d4072620 msgr2=0x7fa3d4072a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:26.725 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:26.725+0000 7fa3da600640 1 -- 172.21.15.152:0/1711720812 shutdown_connections 2024-04-20T11:34:26.725 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:26.725+0000 7fa3da600640 1 -- 172.21.15.152:0/1711720812 wait complete. 2024-04-20T11:34:26.727 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:26.727+0000 7fa3da600640 1 Processor -- start 2024-04-20T11:34:26.727 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:26.727+0000 7fa3da600640 1 -- start start 2024-04-20T11:34:26.728 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:26.728+0000 7fa3da600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa3d4073210 0x7fa3d41465e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:26.728 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:26.728+0000 7fa3da600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa3d40aee70 0x7fa3d41f38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:26.728 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:26.728+0000 7fa3da600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3d40af250 0x7fa3d41f3e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:26.728 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:26.728+0000 7fa3da600640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d41608f0 con 0x7fa3d40aee70 2024-04-20T11:34:26.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:26.728+0000 7fa3da600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d415f530 con 0x7fa3d40af250 2024-04-20T11:34:26.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:26.728+0000 7fa3da600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa3d4160a70 con 0x7fa3d4073210 2024-04-20T11:34:26.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:26.729+0000 7fa3cbe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa3d40aee70 0x7fa3d41f38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:26.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:26.729+0000 7fa3cbe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa3d40aee70 0x7fa3d41f38d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:33430/0 (socket says 172.21.15.152:33430) 2024-04-20T11:34:26.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:26.729+0000 7fa3cbe00640 1 -- 172.21.15.152:0/2219535275 learned_addr learned my addr 172.21.15.152:0/2219535275 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:26.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:26.729+0000 7fa3d3e00640 1 --2- 172.21.15.152:0/2219535275 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3d40af250 0x7fa3d41f3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:26.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:26.729+0000 7fa3d3400640 1 --2- 172.21.15.152:0/2219535275 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa3d4073210 0x7fa3d41465e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:26.730 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:26.729+0000 7fa3cbe00640 1 -- 172.21.15.152:0/2219535275 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa3d4073210 msgr2=0x7fa3d41465e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:26.730 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:26.729+0000 7fa3cbe00640 1 --2- 172.21.15.152:0/2219535275 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa3d4073210 0x7fa3d41465e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:26.730 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:26.729+0000 7fa3cbe00640 1 -- 172.21.15.152:0/2219535275 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3d40af250 msgr2=0x7fa3d41f3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:26.731 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:26.729+0000 7fa3cbe00640 1 --2- 172.21.15.152:0/2219535275 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3d40af250 0x7fa3d41f3e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:26.731 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:26.729+0000 7fa3cbe00640 1 -- 172.21.15.152:0/2219535275 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3d4144410 con 0x7fa3d40aee70 2024-04-20T11:34:26.731 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:26.730+0000 7fa3cbe00640 1 --2- 172.21.15.152:0/2219535275 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa3d40aee70 0x7fa3d41f38d0 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc00b410 tx=0x7fa3bc00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:26.731 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:26.730+0000 7fa3cb400640 1 -- 172.21.15.152:0/2219535275 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3bc015020 con 0x7fa3d40aee70 2024-04-20T11:34:26.731 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:26.730+0000 7fa3cb400640 1 -- 172.21.15.152:0/2219535275 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3bc011850 con 0x7fa3d40aee70 2024-04-20T11:34:26.731 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:26.730+0000 7fa3cb400640 1 -- 172.21.15.152:0/2219535275 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3bc011c40 con 0x7fa3d40aee70 2024-04-20T11:34:26.731 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:26.730+0000 7fa3da600640 1 -- 172.21.15.152:0/2219535275 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3d41446f0 con 0x7fa3d40aee70 2024-04-20T11:34:26.731 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:26.730+0000 7fa3da600640 1 -- 172.21.15.152:0/2219535275 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3d4146b20 con 0x7fa3d40aee70 2024-04-20T11:34:26.732 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:26.731+0000 7fa3da600640 1 -- 172.21.15.152:0/2219535275 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3d40689f0 con 0x7fa3d40aee70 2024-04-20T11:34:26.732 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:26.732+0000 7fa3cb400640 1 -- 172.21.15.152:0/2219535275 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 19) ==== 91397+0+0 (secure 0 0 0) 0x7fa3bc018020 con 0x7fa3d40aee70 2024-04-20T11:34:26.733 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:26.734+0000 7fa3cb400640 1 --2- 172.21.15.152:0/2219535275 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fa3b006cce0 0x7fa3b006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:26.733 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:26.734+0000 7fa3cb400640 1 -- 172.21.15.152:0/2219535275 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 4943+0+0 (secure 0 0 0) 0x7fa3bc08bf00 con 0x7fa3d40aee70 2024-04-20T11:34:26.734 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:26.734+0000 7fa3d3400640 1 --2- 172.21.15.152:0/2219535275 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fa3b006cce0 0x7fa3b006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:26.735 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:26.736+0000 7fa3d3400640 1 --2- 172.21.15.152:0/2219535275 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fa3b006cce0 0x7fa3b006f1a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa3c4004ea0 tx=0x7fa3c4009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:26.738 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:26.738+0000 7fa3cb400640 1 -- 172.21.15.152:0/2219535275 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fa3bc01f020 con 0x7fa3d40aee70 2024-04-20T11:34:26.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:26.940+0000 7fa3da600640 1 -- 172.21.15.152:0/2219535275 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fa3d4161000 con 0x7fa3d40aee70 2024-04-20T11:34:26.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:26.947+0000 7fa3cb400640 1 -- 172.21.15.152:0/2219535275 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v21) ==== 104+0+0 (secure 0 0 0) 0x7fa3bc0172c0 con 0x7fa3d40aee70 2024-04-20T11:34:26.950 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:26.950+0000 7fa3da600640 1 -- 172.21.15.152:0/2219535275 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fa3b006cce0 msgr2=0x7fa3b006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:26.950 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:26.950+0000 7fa3da600640 1 --2- 172.21.15.152:0/2219535275 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fa3b006cce0 0x7fa3b006f1a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa3c4004ea0 tx=0x7fa3c4009040 comp rx=0 tx=0).stop 2024-04-20T11:34:26.950 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:26.950+0000 7fa3da600640 1 -- 172.21.15.152:0/2219535275 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa3d40aee70 msgr2=0x7fa3d41f38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:26.950 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:26.950+0000 7fa3da600640 1 --2- 172.21.15.152:0/2219535275 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa3d40aee70 0x7fa3d41f38d0 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc00b410 tx=0x7fa3bc00b8d0 comp rx=0 tx=0).stop 2024-04-20T11:34:26.950 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:26.950+0000 7fa3da600640 1 -- 172.21.15.152:0/2219535275 shutdown_connections 2024-04-20T11:34:26.950 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:26.950+0000 7fa3da600640 1 --2- 172.21.15.152:0/2219535275 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fa3b006cce0 0x7fa3b006f1a0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:26.950 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:26.950+0000 7fa3da600640 1 --2- 172.21.15.152:0/2219535275 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3d40af250 0x7fa3d41f3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:26.951 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:26.950+0000 7fa3da600640 1 --2- 172.21.15.152:0/2219535275 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa3d40aee70 0x7fa3d41f38d0 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:26.951 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:26.950+0000 7fa3da600640 1 --2- 172.21.15.152:0/2219535275 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa3d4073210 0x7fa3d41465e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:26.951 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:26.950+0000 7fa3da600640 1 -- 172.21.15.152:0/2219535275 >> 172.21.15.152:0/2219535275 conn(0x7fa3d4072620 msgr2=0x7fa3d415edb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:26.951 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:26.951+0000 7fa3da600640 1 -- 172.21.15.152:0/2219535275 shutdown_connections 2024-04-20T11:34:26.951 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:26.951+0000 7fa3da600640 1 -- 172.21.15.152:0/2219535275 wait complete. 2024-04-20T11:34:26.969 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_sanity' 2024-04-20T11:34:27.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.166+0000 7f5fb6400640 1 Processor -- start 2024-04-20T11:34:27.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.166+0000 7f5fb6400640 1 -- start start 2024-04-20T11:34:27.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.167+0000 7f5fb6400640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f5fb0160b90 0x7f5fb015d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:27.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.167+0000 7f5fb6400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5fb0071cd0 0x7f5fb0072160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:27.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.167+0000 7f5fb6400640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5fb00ce8c0 con 0x7f5fb015d1e0 2024-04-20T11:34:27.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.167+0000 7f5fb6400640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f5fb00726a0 con 0x7f5fb0160b90 2024-04-20T11:34:27.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.167+0000 7f5fb6400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5fb01600b0 con 0x7f5fb0071cd0 2024-04-20T11:34:27.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.168+0000 7f5fb5000640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f5fb015d1e0 0x7f5fb00cdfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:42690/0 (socket says 172.21.15.152:42690) 2024-04-20T11:34:27.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.168+0000 7f5fb5000640 1 -- 172.21.15.152:0/1677417113 learned_addr learned my addr 172.21.15.152:0/1677417113 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:34:27.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.168+0000 7f5fb5a00640 1 --2- 172.21.15.152:0/1677417113 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5fb0071cd0 0x7f5fb0072160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:27.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.168+0000 7f5faf400640 1 -- 172.21.15.152:0/1677417113 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4108936559 0 0) 0x7f5fb00ce8c0 con 0x7f5fb015d1e0 2024-04-20T11:34:27.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.168+0000 7f5fafe00640 1 --2- 172.21.15.152:0/1677417113 >> v2:172.21.15.175:3300/0 conn(0x7f5fb0160b90 0x7f5fb015d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:27.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.168+0000 7f5faf400640 1 -- 172.21.15.152:0/1677417113 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5f98003520 con 0x7f5fb015d1e0 2024-04-20T11:34:27.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.169+0000 7f5faf400640 1 -- 172.21.15.152:0/1677417113 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 58784846 0 0) 0x7f5f98003520 con 0x7f5fb015d1e0 2024-04-20T11:34:27.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.169+0000 7f5faf400640 1 -- 172.21.15.152:0/1677417113 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5fb0071cd0 msgr2=0x7f5fb0072160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:27.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.169+0000 7f5faf400640 1 --2- 172.21.15.152:0/1677417113 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5fb0071cd0 0x7f5fb0072160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:27.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.169+0000 7f5faf400640 1 -- 172.21.15.152:0/1677417113 >> v2:172.21.15.175:3300/0 conn(0x7f5fb0160b90 msgr2=0x7f5fb015d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:27.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.169+0000 7f5faf400640 1 --2- 172.21.15.152:0/1677417113 >> v2:172.21.15.175:3300/0 conn(0x7f5fb0160b90 0x7f5fb015d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:27.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.169+0000 7f5faf400640 1 -- 172.21.15.152:0/1677417113 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fb0160390 con 0x7f5fb015d1e0 2024-04-20T11:34:27.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.169+0000 7f5faf400640 1 -- 172.21.15.152:0/1677417113 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f5fa4001670 con 0x7f5fb015d1e0 2024-04-20T11:34:27.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.169+0000 7f5faf400640 1 -- 172.21.15.152:0/1677417113 >> v1:172.21.15.152:6789/0 conn(0x7f5fb015d1e0 legacy=0x7f5fb00cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:27.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.169+0000 7f5faf400640 1 --2- 172.21.15.152:0/1677417113 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5f980039b0 0x7f5f98003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:27.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.169+0000 7f5faf400640 1 --2- 172.21.15.152:0/1677417113 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5fb0071cd0 0x7f5f98004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:27.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.169+0000 7f5faf400640 1 --2- 172.21.15.152:0/1677417113 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f98004d40 0x7f5f980051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:27.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.169+0000 7f5faf400640 1 -- 172.21.15.152:0/1677417113 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb00726a0 con 0x7f5f980039b0 2024-04-20T11:34:27.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.169+0000 7f5faf400640 1 -- 172.21.15.152:0/1677417113 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb01600b0 con 0x7f5f98004d40 2024-04-20T11:34:27.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.169+0000 7f5faf400640 1 -- 172.21.15.152:0/1677417113 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5f9800b9c0 con 0x7f5fb0071cd0 2024-04-20T11:34:27.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.169+0000 7f5fb5000640 1 --2- 172.21.15.152:0/1677417113 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5f980039b0 0x7f5f98003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:27.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.170+0000 7f5fafe00640 1 --2- 172.21.15.152:0/1677417113 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5fb0071cd0 0x7f5f98004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:27.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.170+0000 7f5fb5a00640 1 --2- 172.21.15.152:0/1677417113 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f98004d40 0x7f5f980051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:27.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.170+0000 7f5fb5000640 1 -- 172.21.15.152:0/1677417113 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5fb0071cd0 msgr2=0x7f5f98004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:27.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.170+0000 7f5fb5000640 1 --2- 172.21.15.152:0/1677417113 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5fb0071cd0 0x7f5f98004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:27.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.170+0000 7f5fb5000640 1 -- 172.21.15.152:0/1677417113 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f98004d40 msgr2=0x7f5f980051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:27.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.170+0000 7f5fb5000640 1 --2- 172.21.15.152:0/1677417113 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f98004d40 0x7f5f980051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:27.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.170+0000 7f5fb5000640 1 -- 172.21.15.152:0/1677417113 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f98003520 con 0x7f5f980039b0 2024-04-20T11:34:27.172 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.171+0000 7f5fb5000640 1 --2- 172.21.15.152:0/1677417113 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5f980039b0 0x7f5f98003dd0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f5fa400c210 tx=0x7f5f98005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f17c18d44e97b4c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:27.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.171+0000 7f5faf400640 1 -- 172.21.15.152:0/1677417113 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa4007d10 con 0x7f5f980039b0 2024-04-20T11:34:27.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.171+0000 7f5faf400640 1 -- 172.21.15.152:0/1677417113 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fa4030070 con 0x7f5f980039b0 2024-04-20T11:34:27.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.171+0000 7f5faf400640 1 -- 172.21.15.152:0/1677417113 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa4005d30 con 0x7f5f980039b0 2024-04-20T11:34:27.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.172+0000 7f5fb6400640 1 -- 172.21.15.152:0/1677417113 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5f980039b0 msgr2=0x7f5f98003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:27.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.172+0000 7f5fb6400640 1 --2- 172.21.15.152:0/1677417113 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5f980039b0 0x7f5f98003dd0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f5fa400c210 tx=0x7f5f98005830 comp rx=0 tx=0).stop 2024-04-20T11:34:27.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.172+0000 7f5fb5a00640 1 -- 172.21.15.152:0/1677417113 reap_dead start 2024-04-20T11:34:27.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.172+0000 7f5fb6400640 1 -- 172.21.15.152:0/1677417113 shutdown_connections 2024-04-20T11:34:27.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.172+0000 7f5fb6400640 1 -- 172.21.15.152:0/1677417113 >> 172.21.15.152:0/1677417113 conn(0x7f5fb0067ae0 msgr2=0x7f5fb006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:27.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.172+0000 7f5fb6400640 1 -- 172.21.15.152:0/1677417113 shutdown_connections 2024-04-20T11:34:27.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.172+0000 7f5fb6400640 1 -- 172.21.15.152:0/1677417113 wait complete. 2024-04-20T11:34:27.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.174+0000 7f5fb6400640 1 Processor -- start 2024-04-20T11:34:27.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.174+0000 7f5fb6400640 1 -- start start 2024-04-20T11:34:27.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.175+0000 7f5fb6400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5f980039b0 0x7f5fb00af6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:27.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.175+0000 7f5fb6400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5fb00c5300 0x7f5fb014d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:27.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.175+0000 7f5fb6400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5fb015ac50 0x7f5fb015b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:27.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.175+0000 7f5fb6400640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb0072160 con 0x7f5fb015ac50 2024-04-20T11:34:27.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.175+0000 7f5fb6400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb00ceba0 con 0x7f5fb00c5300 2024-04-20T11:34:27.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.175+0000 7f5fb6400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5fb00722e0 con 0x7f5f980039b0 2024-04-20T11:34:27.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.175+0000 7f5fb5a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5fb015ac50 0x7f5fb015b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:27.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.175+0000 7f5fb5a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5fb015ac50 0x7f5fb015b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:33436/0 (socket says 172.21.15.152:33436) 2024-04-20T11:34:27.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.175+0000 7f5fb5a00640 1 -- 172.21.15.152:0/3987901823 learned_addr learned my addr 172.21.15.152:0/3987901823 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:27.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.176+0000 7f5fb5000640 1 --2- 172.21.15.152:0/3987901823 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5f980039b0 0x7f5fb00af6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:27.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.176+0000 7f5fafe00640 1 --2- 172.21.15.152:0/3987901823 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5fb00c5300 0x7f5fb014d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:27.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.176+0000 7f5fb5a00640 1 -- 172.21.15.152:0/3987901823 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5f980039b0 msgr2=0x7f5fb00af6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:27.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.176+0000 7f5fb5a00640 1 --2- 172.21.15.152:0/3987901823 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5f980039b0 0x7f5fb00af6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:27.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.176+0000 7f5fb5a00640 1 -- 172.21.15.152:0/3987901823 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5fb00c5300 msgr2=0x7f5fb014d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:27.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.176+0000 7f5fb5a00640 1 --2- 172.21.15.152:0/3987901823 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5fb00c5300 0x7f5fb014d850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:27.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.176+0000 7f5fb5a00640 1 -- 172.21.15.152:0/3987901823 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fb01f4790 con 0x7f5fb015ac50 2024-04-20T11:34:27.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.176+0000 7f5fb5a00640 1 --2- 172.21.15.152:0/3987901823 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5fb015ac50 0x7f5fb015b0d0 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f5fb006fe80 tx=0x7f5fa800f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:27.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.176+0000 7f5facc00640 1 -- 172.21.15.152:0/3987901823 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa801c070 con 0x7f5fb015ac50 2024-04-20T11:34:27.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.176+0000 7f5facc00640 1 -- 172.21.15.152:0/3987901823 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fa8017400 con 0x7f5fb015ac50 2024-04-20T11:34:27.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.176+0000 7f5fb6400640 1 -- 172.21.15.152:0/3987901823 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fb01f4a70 con 0x7f5fb015ac50 2024-04-20T11:34:27.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.176+0000 7f5facc00640 1 -- 172.21.15.152:0/3987901823 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa80177f0 con 0x7f5fb015ac50 2024-04-20T11:34:27.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.177+0000 7f5fb6400640 1 -- 172.21.15.152:0/3987901823 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fb01f5030 con 0x7f5fb015ac50 2024-04-20T11:34:27.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.178+0000 7f5fb6400640 1 -- 172.21.15.152:0/3987901823 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fb015b610 con 0x7f5fb015ac50 2024-04-20T11:34:27.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.179+0000 7f5facc00640 1 -- 172.21.15.152:0/3987901823 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 19) ==== 91397+0+0 (secure 0 0 0) 0x7f5fa801f070 con 0x7f5fb015ac50 2024-04-20T11:34:27.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.180+0000 7f5facc00640 1 --2- 172.21.15.152:0/3987901823 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f5f9006cce0 0x7f5f9006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:27.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.180+0000 7f5facc00640 1 -- 172.21.15.152:0/3987901823 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 4943+0+0 (secure 0 0 0) 0x7f5fa808c080 con 0x7f5fb015ac50 2024-04-20T11:34:27.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.180+0000 7f5fb5000640 1 --2- 172.21.15.152:0/3987901823 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f5f9006cce0 0x7f5f9006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:27.181 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.182+0000 7f5fb5000640 1 --2- 172.21.15.152:0/3987901823 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f5f9006cce0 0x7f5f9006f1a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5fa4034000 tx=0x7f5fa4007000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:27.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.184+0000 7f5facc00640 1 -- 172.21.15.152:0/3987901823 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f5fa8012070 con 0x7f5fb015ac50 2024-04-20T11:34:27.462 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.462+0000 7f5fb6400640 1 -- 172.21.15.152:0/3987901823 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_sanity"]} v 0) -- 0x7f5fb006ef10 con 0x7f5fb015ac50 2024-04-20T11:34:27.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.500+0000 7f5facc00640 1 -- 172.21.15.152:0/3987901823 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_sanity"]}]=0 v398) ==== 149+0+0 (secure 0 0 0) 0x7f5fb006ef10 con 0x7f5fb015ac50 2024-04-20T11:34:27.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.504+0000 7f5fb6400640 1 -- 172.21.15.152:0/3987901823 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f5f9006cce0 msgr2=0x7f5f9006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:27.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.504+0000 7f5fb6400640 1 --2- 172.21.15.152:0/3987901823 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f5f9006cce0 0x7f5f9006f1a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5fa4034000 tx=0x7f5fa4007000 comp rx=0 tx=0).stop 2024-04-20T11:34:27.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.504+0000 7f5fb6400640 1 -- 172.21.15.152:0/3987901823 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5fb015ac50 msgr2=0x7f5fb015b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:27.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.504+0000 7f5fb6400640 1 --2- 172.21.15.152:0/3987901823 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5fb015ac50 0x7f5fb015b0d0 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f5fb006fe80 tx=0x7f5fa800f4a0 comp rx=0 tx=0).stop 2024-04-20T11:34:27.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.504+0000 7f5fb6400640 1 -- 172.21.15.152:0/3987901823 shutdown_connections 2024-04-20T11:34:27.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.504+0000 7f5fb6400640 1 --2- 172.21.15.152:0/3987901823 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f5f9006cce0 0x7f5f9006f1a0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:27.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.504+0000 7f5fb6400640 1 --2- 172.21.15.152:0/3987901823 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5fb015ac50 0x7f5fb015b0d0 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:27.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.504+0000 7f5fb6400640 1 --2- 172.21.15.152:0/3987901823 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5fb00c5300 0x7f5fb014d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:27.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.504+0000 7f5fb6400640 1 --2- 172.21.15.152:0/3987901823 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5f980039b0 0x7f5fb00af6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:27.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.504+0000 7f5fb6400640 1 -- 172.21.15.152:0/3987901823 >> 172.21.15.152:0/3987901823 conn(0x7f5fb0067ae0 msgr2=0x7f5fb006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:27.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.504+0000 7f5fb6400640 1 -- 172.21.15.152:0/3987901823 shutdown_connections 2024-04-20T11:34:27.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.504+0000 7f5fb6400640 1 -- 172.21.15.152:0/3987901823 wait complete. 2024-04-20T11:34:27.524 INFO:tasks.cephfs_test_runner:poor man's upgrade test -- theme continues... ... ok 2024-04-20T11:34:27.524 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_no_upgrade_v1_to_v2 (tasks.cephfs.test_volumes.TestMisc) poor man's upgrade test -- theme continues... 2024-04-20T11:34:27.524 INFO:tasks.cephfs_test_runner:test_subvolume_no_upgrade_v1_to_v2 (tasks.cephfs.test_volumes.TestMisc) 2024-04-20T11:34:27.525 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:34:27.717 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.717+0000 7fc876c00640 1 Processor -- start 2024-04-20T11:34:27.717 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.718+0000 7fc876c00640 1 -- start start 2024-04-20T11:34:27.719 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.719+0000 7fc876c00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fc87015c680 0x7fc8700d02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:27.719 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.719+0000 7fc876c00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fc8700d1ad0 0x7fc8700cede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:27.719 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.719+0000 7fc876c00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8700b4ec0 con 0x7fc8700cf320 2024-04-20T11:34:27.719 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.719+0000 7fc876c00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fc8700d1f40 con 0x7fc87015c680 2024-04-20T11:34:27.719 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.719+0000 7fc876c00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fc8700cf770 con 0x7fc8700d1ad0 2024-04-20T11:34:27.719 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.719+0000 7fc875800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fc87015c680 0x7fc8700d02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:27.719 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.719+0000 7fc875800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fc87015c680 0x7fc8700d02f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:33440/0 (socket says 172.21.15.152:33440) 2024-04-20T11:34:27.719 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.719+0000 7fc875800640 1 -- 172.21.15.152:0/292931889 learned_addr learned my addr 172.21.15.152:0/292931889 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:27.720 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.720+0000 7fc875800640 1 -- 172.21.15.152:0/292931889 >> v2:172.21.15.175:3300/0 conn(0x7fc8700d1ad0 msgr2=0x7fc8700cede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:27.720 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.720+0000 7fc875800640 1 --2- 172.21.15.152:0/292931889 >> v2:172.21.15.175:3300/0 conn(0x7fc8700d1ad0 0x7fc8700cede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:27.720 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.720+0000 7fc875800640 1 -- 172.21.15.152:0/292931889 >> v1:172.21.15.175:6789/0 conn(0x7fc8700cf320 legacy=0x7fc8701624c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:27.720 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.720+0000 7fc875800640 1 -- 172.21.15.152:0/292931889 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc87015fad0 con 0x7fc87015c680 2024-04-20T11:34:27.721 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.721+0000 7fc875800640 1 --2- 172.21.15.152:0/292931889 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc87015c680 0x7fc8700d02f0 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7fc8600098a0 tx=0x7fc86002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=343c82991052c41b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:27.721 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.722+0000 7fc86fe00640 1 -- 172.21.15.152:0/292931889 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8600317c0 con 0x7fc87015c680 2024-04-20T11:34:27.721 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.722+0000 7fc86fe00640 1 -- 172.21.15.152:0/292931889 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc860037070 con 0x7fc87015c680 2024-04-20T11:34:27.722 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.722+0000 7fc86fe00640 1 -- 172.21.15.152:0/292931889 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc860031b80 con 0x7fc87015c680 2024-04-20T11:34:27.722 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.722+0000 7fc876c00640 1 -- 172.21.15.152:0/292931889 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc87015c680 msgr2=0x7fc8700d02f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:27.723 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.722+0000 7fc876c00640 1 --2- 172.21.15.152:0/292931889 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc87015c680 0x7fc8700d02f0 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7fc8600098a0 tx=0x7fc86002b860 comp rx=0 tx=0).stop 2024-04-20T11:34:27.723 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.722+0000 7fc876c00640 1 -- 172.21.15.152:0/292931889 shutdown_connections 2024-04-20T11:34:27.723 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.722+0000 7fc876c00640 1 --2- 172.21.15.152:0/292931889 >> v2:172.21.15.175:3300/0 conn(0x7fc8700d1ad0 0x7fc8700cede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:27.723 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.723+0000 7fc876c00640 1 --2- 172.21.15.152:0/292931889 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc87015c680 0x7fc8700d02f0 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:27.723 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.723+0000 7fc876c00640 1 -- 172.21.15.152:0/292931889 >> 172.21.15.152:0/292931889 conn(0x7fc8700b3fe0 msgr2=0x7fc8700b43f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:27.723 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.723+0000 7fc876c00640 1 -- 172.21.15.152:0/292931889 shutdown_connections 2024-04-20T11:34:27.723 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.723+0000 7fc876c00640 1 -- 172.21.15.152:0/292931889 wait complete. 2024-04-20T11:34:27.724 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.725+0000 7fc876c00640 1 Processor -- start 2024-04-20T11:34:27.724 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.725+0000 7fc876c00640 1 -- start start 2024-04-20T11:34:27.727 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.726+0000 7fc876c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8700cf320 0x7fc87014f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:27.727 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.726+0000 7fc876c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc8700d1ad0 0x7fc87006d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:27.727 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.726+0000 7fc876c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc87015c680 0x7fc87006dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:27.727 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.726+0000 7fc876c00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fc8700b3c20 con 0x7fc87015c680 2024-04-20T11:34:27.727 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.726+0000 7fc876c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc8700b3940 con 0x7fc8700cf320 2024-04-20T11:34:27.727 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.726+0000 7fc876c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc87015fdb0 con 0x7fc8700d1ad0 2024-04-20T11:34:27.728 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.726+0000 7fc876200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc87015c680 0x7fc87006dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:27.728 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.726+0000 7fc876200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc87015c680 0x7fc87006dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:33442/0 (socket says 172.21.15.152:33442) 2024-04-20T11:34:27.728 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.726+0000 7fc876200640 1 -- 172.21.15.152:0/1545524590 learned_addr learned my addr 172.21.15.152:0/1545524590 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:27.728 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.726+0000 7fc875800640 1 --2- 172.21.15.152:0/1545524590 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8700cf320 0x7fc87014f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:27.728 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.726+0000 7fc874e00640 1 --2- 172.21.15.152:0/1545524590 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc8700d1ad0 0x7fc87006d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:27.728 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.726+0000 7fc876200640 1 -- 172.21.15.152:0/1545524590 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc8700d1ad0 msgr2=0x7fc87006d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:27.728 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.726+0000 7fc876200640 1 --2- 172.21.15.152:0/1545524590 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc8700d1ad0 0x7fc87006d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:27.728 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.726+0000 7fc876200640 1 -- 172.21.15.152:0/1545524590 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8700cf320 msgr2=0x7fc87014f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:27.728 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.726+0000 7fc876200640 1 --2- 172.21.15.152:0/1545524590 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8700cf320 0x7fc87014f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:27.728 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.726+0000 7fc876200640 1 -- 172.21.15.152:0/1545524590 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc870159ca0 con 0x7fc87015c680 2024-04-20T11:34:27.728 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.727+0000 7fc875800640 1 --2- 172.21.15.152:0/1545524590 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8700cf320 0x7fc87014f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:34:27.728 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.727+0000 7fc874e00640 1 --2- 172.21.15.152:0/1545524590 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc8700d1ad0 0x7fc87006d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:34:27.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.727+0000 7fc876200640 1 --2- 172.21.15.152:0/1545524590 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc87015c680 0x7fc87006dae0 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7fc86800b490 tx=0x7fc86800b950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:27.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.727+0000 7fc86d600640 1 -- 172.21.15.152:0/1545524590 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc868015020 con 0x7fc87015c680 2024-04-20T11:34:27.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.727+0000 7fc86d600640 1 -- 172.21.15.152:0/1545524590 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc868011850 con 0x7fc87015c680 2024-04-20T11:34:27.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.727+0000 7fc86d600640 1 -- 172.21.15.152:0/1545524590 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc868011c40 con 0x7fc87015c680 2024-04-20T11:34:27.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.727+0000 7fc876c00640 1 -- 172.21.15.152:0/1545524590 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc870159f40 con 0x7fc87015c680 2024-04-20T11:34:27.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.727+0000 7fc876c00640 1 -- 172.21.15.152:0/1545524590 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc87015a500 con 0x7fc87015c680 2024-04-20T11:34:27.730 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.728+0000 7fc876c00640 1 -- 172.21.15.152:0/1545524590 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc870075e60 con 0x7fc87015c680 2024-04-20T11:34:27.730 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.729+0000 7fc86d600640 1 -- 172.21.15.152:0/1545524590 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 19) ==== 91397+0+0 (secure 0 0 0) 0x7fc868018020 con 0x7fc87015c680 2024-04-20T11:34:27.730 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.730+0000 7fc86d600640 1 --2- 172.21.15.152:0/1545524590 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fc85006cce0 0x7fc85006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:27.730 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.730+0000 7fc86d600640 1 -- 172.21.15.152:0/1545524590 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 4943+0+0 (secure 0 0 0) 0x7fc86808bdc0 con 0x7fc87015c680 2024-04-20T11:34:27.731 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.731+0000 7fc875800640 1 --2- 172.21.15.152:0/1545524590 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fc85006cce0 0x7fc85006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:27.732 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.732+0000 7fc875800640 1 --2- 172.21.15.152:0/1545524590 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fc85006cce0 0x7fc85006f1a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc860036000 tx=0x7fc860008030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:27.734 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.735+0000 7fc86d600640 1 -- 172.21.15.152:0/1545524590 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fc86801f020 con 0x7fc87015c680 2024-04-20T11:34:27.925 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.925+0000 7fc876c00640 1 -- 172.21.15.152:0/1545524590 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc87006cda0 con 0x7fc87015c680 2024-04-20T11:34:27.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.927+0000 7fc86d600640 1 -- 172.21.15.152:0/1545524590 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) ==== 74+0+13092 (secure 0 0 0) 0x7fc8680172c0 con 0x7fc87015c680 2024-04-20T11:34:27.927 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:34:27.927 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":110,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:34:26.485562+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.152:6835/3582312712":"2024-04-21T11:34:23.465831+0000","172.21.15.152:6833/3582312712":"2024-04-21T11:34:23.465831+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:34:27.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.929+0000 7fc876c00640 1 -- 172.21.15.152:0/1545524590 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fc85006cce0 msgr2=0x7fc85006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:27.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.929+0000 7fc876c00640 1 --2- 172.21.15.152:0/1545524590 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fc85006cce0 0x7fc85006f1a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc860036000 tx=0x7fc860008030 comp rx=0 tx=0).stop 2024-04-20T11:34:27.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.929+0000 7fc876c00640 1 -- 172.21.15.152:0/1545524590 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc87015c680 msgr2=0x7fc87006dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:27.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.929+0000 7fc876c00640 1 --2- 172.21.15.152:0/1545524590 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc87015c680 0x7fc87006dae0 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7fc86800b490 tx=0x7fc86800b950 comp rx=0 tx=0).stop 2024-04-20T11:34:27.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.930+0000 7fc876c00640 1 -- 172.21.15.152:0/1545524590 shutdown_connections 2024-04-20T11:34:27.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.930+0000 7fc876c00640 1 --2- 172.21.15.152:0/1545524590 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fc85006cce0 0x7fc85006f1a0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:27.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.930+0000 7fc876c00640 1 --2- 172.21.15.152:0/1545524590 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc87015c680 0x7fc87006dae0 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:27.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.930+0000 7fc876c00640 1 --2- 172.21.15.152:0/1545524590 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc8700d1ad0 0x7fc87006d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:27.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.930+0000 7fc876c00640 1 --2- 172.21.15.152:0/1545524590 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8700cf320 0x7fc87014f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:27.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.930+0000 7fc876c00640 1 -- 172.21.15.152:0/1545524590 >> 172.21.15.152:0/1545524590 conn(0x7fc8700b3fe0 msgr2=0x7fc870071670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:27.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.930+0000 7fc876c00640 1 -- 172.21.15.152:0/1545524590 shutdown_connections 2024-04-20T11:34:27.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:27.930+0000 7fc876c00640 1 -- 172.21.15.152:0/1545524590 wait complete. 2024-04-20T11:34:27.947 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T11:26:47.246571+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T11:34:27.947 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T11:34:28.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.154+0000 7f39a3a00640 1 Processor -- start 2024-04-20T11:34:28.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.154+0000 7f39a3a00640 1 -- start start 2024-04-20T11:34:28.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.156+0000 7f39a3a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f399c0b2880 0x7f399c0b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:28.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.156+0000 7f39a3a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f399c0cd1b0 0x7f399c0cd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:28.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.156+0000 7f39a3a00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f399c1463e0 con 0x7f399c146000 2024-04-20T11:34:28.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.156+0000 7f39a3a00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f399c0b4750 con 0x7f399c0b2880 2024-04-20T11:34:28.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.156+0000 7f39a3a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f399c0b3e20 con 0x7f399c0cd1b0 2024-04-20T11:34:28.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.156+0000 7f399be00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f399c0b2880 0x7f399c0b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:28.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.156+0000 7f39a1000640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f399c146000 0x7f399c0b2170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:42704/0 (socket says 172.21.15.152:42704) 2024-04-20T11:34:28.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.156+0000 7f39a1000640 1 -- 172.21.15.152:0/1028653327 learned_addr learned my addr 172.21.15.152:0/1028653327 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:34:28.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.157+0000 7f39a1a00640 1 --2- 172.21.15.152:0/1028653327 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f399c0cd1b0 0x7f399c0cd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:28.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.157+0000 7f399be00640 1 -- 172.21.15.152:0/1028653327 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f399c0cd1b0 msgr2=0x7f399c0cd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:28.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.157+0000 7f399be00640 1 --2- 172.21.15.152:0/1028653327 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f399c0cd1b0 0x7f399c0cd5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:28.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.157+0000 7f399be00640 1 -- 172.21.15.152:0/1028653327 >> v1:172.21.15.152:6789/0 conn(0x7f399c146000 legacy=0x7f399c0b2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:28.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.157+0000 7f399be00640 1 -- 172.21.15.152:0/1028653327 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f399c0b4100 con 0x7f399c0b2880 2024-04-20T11:34:28.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.158+0000 7f399be00640 1 --2- 172.21.15.152:0/1028653327 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f399c0b2880 0x7f399c0b4be0 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f39880097a0 tx=0x7f398802b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0f1bb0f789c15bc server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:28.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.158+0000 7f399b400640 1 -- 172.21.15.152:0/1028653327 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3988031780 con 0x7f399c0b2880 2024-04-20T11:34:28.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.158+0000 7f399b400640 1 -- 172.21.15.152:0/1028653327 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3988037070 con 0x7f399c0b2880 2024-04-20T11:34:28.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.158+0000 7f399b400640 1 -- 172.21.15.152:0/1028653327 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3988031b40 con 0x7f399c0b2880 2024-04-20T11:34:28.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.159+0000 7f39a3a00640 1 -- 172.21.15.152:0/1028653327 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f399c0b2880 msgr2=0x7f399c0b4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:28.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.159+0000 7f39a3a00640 1 --2- 172.21.15.152:0/1028653327 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f399c0b2880 0x7f399c0b4be0 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f39880097a0 tx=0x7f398802b820 comp rx=0 tx=0).stop 2024-04-20T11:34:28.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.159+0000 7f39a3a00640 1 -- 172.21.15.152:0/1028653327 shutdown_connections 2024-04-20T11:34:28.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.159+0000 7f39a3a00640 1 --2- 172.21.15.152:0/1028653327 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f399c0cd1b0 0x7f399c0cd5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:28.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.159+0000 7f39a3a00640 1 --2- 172.21.15.152:0/1028653327 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f399c0b2880 0x7f399c0b4be0 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:28.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.159+0000 7f39a3a00640 1 -- 172.21.15.152:0/1028653327 >> 172.21.15.152:0/1028653327 conn(0x7f399c06b980 msgr2=0x7f399c06bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:28.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.159+0000 7f39a3a00640 1 -- 172.21.15.152:0/1028653327 shutdown_connections 2024-04-20T11:34:28.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.159+0000 7f39a3a00640 1 -- 172.21.15.152:0/1028653327 wait complete. 2024-04-20T11:34:28.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.161+0000 7f39a3a00640 1 Processor -- start 2024-04-20T11:34:28.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.161+0000 7f39a3a00640 1 -- start start 2024-04-20T11:34:28.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.162+0000 7f39a3a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f399c0b2880 0x7f399c15c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:28.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.162+0000 7f39a3a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f399c0cd1b0 0x7f399c06e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:28.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.162+0000 7f39a3a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f399c146000 0x7f399c15e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:28.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.162+0000 7f39a3a00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f399c0cdb30 con 0x7f399c0cd1b0 2024-04-20T11:34:28.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.162+0000 7f39a3a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f399c15ccf0 con 0x7f399c146000 2024-04-20T11:34:28.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.162+0000 7f39a3a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f399c15c5c0 con 0x7f399c0b2880 2024-04-20T11:34:28.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.163+0000 7f399be00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f399c0cd1b0 0x7f399c06e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:28.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.163+0000 7f399be00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f399c0cd1b0 0x7f399c06e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:33466/0 (socket says 172.21.15.152:33466) 2024-04-20T11:34:28.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.163+0000 7f399be00640 1 -- 172.21.15.152:0/3906750679 learned_addr learned my addr 172.21.15.152:0/3906750679 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:28.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.163+0000 7f39a1000640 1 --2- 172.21.15.152:0/3906750679 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f399c0b2880 0x7f399c15c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:28.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.163+0000 7f399be00640 1 -- 172.21.15.152:0/3906750679 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f399c0b2880 msgr2=0x7f399c15c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:28.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.163+0000 7f39a1a00640 1 --2- 172.21.15.152:0/3906750679 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f399c146000 0x7f399c15e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:28.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.163+0000 7f399be00640 1 --2- 172.21.15.152:0/3906750679 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f399c0b2880 0x7f399c15c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:28.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.163+0000 7f399be00640 1 -- 172.21.15.152:0/3906750679 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f399c146000 msgr2=0x7f399c15e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:28.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.163+0000 7f399be00640 1 --2- 172.21.15.152:0/3906750679 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f399c146000 0x7f399c15e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:28.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.163+0000 7f399be00640 1 -- 172.21.15.152:0/3906750679 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f399c13dbe0 con 0x7f399c0cd1b0 2024-04-20T11:34:28.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.164+0000 7f399be00640 1 --2- 172.21.15.152:0/3906750679 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f399c0cd1b0 0x7f399c06e570 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f3988034040 tx=0x7f39880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:28.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.164+0000 7f3998c00640 1 -- 172.21.15.152:0/3906750679 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3988032820 con 0x7f399c0cd1b0 2024-04-20T11:34:28.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.164+0000 7f3998c00640 1 -- 172.21.15.152:0/3906750679 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39880329e0 con 0x7f399c0cd1b0 2024-04-20T11:34:28.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.164+0000 7f3998c00640 1 -- 172.21.15.152:0/3906750679 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3988032dd0 con 0x7f399c0cd1b0 2024-04-20T11:34:28.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.164+0000 7f39a3a00640 1 -- 172.21.15.152:0/3906750679 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f399c13dec0 con 0x7f399c0cd1b0 2024-04-20T11:34:28.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.164+0000 7f39a3a00640 1 -- 172.21.15.152:0/3906750679 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f399c13e480 con 0x7f399c0cd1b0 2024-04-20T11:34:28.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.165+0000 7f39a3a00640 1 -- 172.21.15.152:0/3906750679 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f399c0689f0 con 0x7f399c0cd1b0 2024-04-20T11:34:28.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.166+0000 7f3998c00640 1 -- 172.21.15.152:0/3906750679 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 19) ==== 91397+0+0 (secure 0 0 0) 0x7f398804e440 con 0x7f399c0cd1b0 2024-04-20T11:34:28.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.167+0000 7f3998c00640 1 --2- 172.21.15.152:0/3906750679 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f397c06cce0 0x7f397c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:28.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.167+0000 7f3998c00640 1 -- 172.21.15.152:0/3906750679 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 4943+0+0 (secure 0 0 0) 0x7f39880addf0 con 0x7f399c0cd1b0 2024-04-20T11:34:28.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.168+0000 7f39a1000640 1 --2- 172.21.15.152:0/3906750679 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f397c06cce0 0x7f397c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:28.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.169+0000 7f39a1000640 1 --2- 172.21.15.152:0/3906750679 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f397c06cce0 0x7f397c06f1a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3990000f60 tx=0x7f3990009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:28.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.172+0000 7f3998c00640 1 -- 172.21.15.152:0/3906750679 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f3988040070 con 0x7f399c0cd1b0 2024-04-20T11:34:28.369 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.369+0000 7f39a3a00640 1 -- 172.21.15.152:0/3906750679 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f399c0b0d00 con 0x7f399c0cd1b0 2024-04-20T11:34:28.370 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.370+0000 7f3998c00640 1 -- 172.21.15.152:0/3906750679 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v110) ==== 93+0+10 (secure 0 0 0) 0x7f3988080e20 con 0x7f399c0cd1b0 2024-04-20T11:34:28.370 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 1 2024-04-20T11:34:28.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.373+0000 7f39a3a00640 1 -- 172.21.15.152:0/3906750679 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f397c06cce0 msgr2=0x7f397c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:28.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.373+0000 7f39a3a00640 1 --2- 172.21.15.152:0/3906750679 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f397c06cce0 0x7f397c06f1a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3990000f60 tx=0x7f3990009040 comp rx=0 tx=0).stop 2024-04-20T11:34:28.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.373+0000 7f39a3a00640 1 -- 172.21.15.152:0/3906750679 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f399c0cd1b0 msgr2=0x7f399c06e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:28.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.373+0000 7f39a3a00640 1 --2- 172.21.15.152:0/3906750679 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f399c0cd1b0 0x7f399c06e570 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f3988034040 tx=0x7f39880046c0 comp rx=0 tx=0).stop 2024-04-20T11:34:28.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.373+0000 7f39a3a00640 1 -- 172.21.15.152:0/3906750679 shutdown_connections 2024-04-20T11:34:28.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.373+0000 7f39a3a00640 1 --2- 172.21.15.152:0/3906750679 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f397c06cce0 0x7f397c06f1a0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:28.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.373+0000 7f39a3a00640 1 --2- 172.21.15.152:0/3906750679 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f399c146000 0x7f399c15e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:28.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.373+0000 7f39a3a00640 1 --2- 172.21.15.152:0/3906750679 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f399c0cd1b0 0x7f399c06e570 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:28.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.373+0000 7f39a3a00640 1 --2- 172.21.15.152:0/3906750679 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f399c0b2880 0x7f399c15c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:28.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.373+0000 7f39a3a00640 1 -- 172.21.15.152:0/3906750679 >> 172.21.15.152:0/3906750679 conn(0x7f399c06b980 msgr2=0x7f399c0b1a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:28.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.373+0000 7f39a3a00640 1 -- 172.21.15.152:0/3906750679 shutdown_connections 2024-04-20T11:34:28.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.373+0000 7f39a3a00640 1 -- 172.21.15.152:0/3906750679 wait complete. 2024-04-20T11:34:28.392 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_to_v2' 2024-04-20T11:34:28.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.592+0000 7fbcfb200640 1 Processor -- start 2024-04-20T11:34:28.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.592+0000 7fbcfb200640 1 -- start start 2024-04-20T11:34:28.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.594+0000 7fbcfb200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fbcfc147410 0x7fbcfc0afa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:28.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.594+0000 7fbcfb200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fbcfc071cd0 0x7fbcfc072130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:28.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.594+0000 7fbcfb200640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbcfc0aec90 con 0x7fbcfc06e010 2024-04-20T11:34:28.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.594+0000 7fbcfb200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fbcfc072670 con 0x7fbcfc147410 2024-04-20T11:34:28.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.594+0000 7fbcfb200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fbcfc0aef70 con 0x7fbcfc071cd0 2024-04-20T11:34:28.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.594+0000 7fbcf9e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fbcfc147410 0x7fbcfc0afa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:28.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.594+0000 7fbcfa800640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fbcfc06e010 0x7fbcfc06e4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:42716/0 (socket says 172.21.15.152:42716) 2024-04-20T11:34:28.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.594+0000 7fbcfa800640 1 -- 172.21.15.152:0/866220195 learned_addr learned my addr 172.21.15.152:0/866220195 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:34:28.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.595+0000 7fbcf1400640 1 --2- 172.21.15.152:0/866220195 >> v2:172.21.15.175:3300/0 conn(0x7fbcfc071cd0 0x7fbcfc072130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:28.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.595+0000 7fbcf9e00640 1 -- 172.21.15.152:0/866220195 >> v2:172.21.15.175:3300/0 conn(0x7fbcfc071cd0 msgr2=0x7fbcfc072130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:28.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.595+0000 7fbcf9e00640 1 --2- 172.21.15.152:0/866220195 >> v2:172.21.15.175:3300/0 conn(0x7fbcfc071cd0 0x7fbcfc072130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:28.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.595+0000 7fbcf9e00640 1 -- 172.21.15.152:0/866220195 >> v1:172.21.15.152:6789/0 conn(0x7fbcfc06e010 legacy=0x7fbcfc06e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:28.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.595+0000 7fbcf9e00640 1 -- 172.21.15.152:0/866220195 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcfc0af250 con 0x7fbcfc147410 2024-04-20T11:34:28.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.595+0000 7fbcf9400640 1 -- 172.21.15.152:0/866220195 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 695119929 0 0) 0x7fbcfc0aec90 con 0x7fbcfc06e010 2024-04-20T11:34:28.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.596+0000 7fbcf9e00640 1 --2- 172.21.15.152:0/866220195 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbcfc147410 0x7fbcfc0afa50 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7fbcfc073170 tx=0x7fbcec02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcf9b35466f4e2f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:28.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.596+0000 7fbcf9400640 1 -- 172.21.15.152:0/866220195 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcec039070 con 0x7fbcfc147410 2024-04-20T11:34:28.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.596+0000 7fbcf9400640 1 -- 172.21.15.152:0/866220195 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcec02cc80 con 0x7fbcfc147410 2024-04-20T11:34:28.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.596+0000 7fbcf9400640 1 -- 172.21.15.152:0/866220195 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcec034660 con 0x7fbcfc147410 2024-04-20T11:34:28.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.597+0000 7fbcfb200640 1 -- 172.21.15.152:0/866220195 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbcfc147410 msgr2=0x7fbcfc0afa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:28.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.597+0000 7fbcfb200640 1 --2- 172.21.15.152:0/866220195 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbcfc147410 0x7fbcfc0afa50 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7fbcfc073170 tx=0x7fbcec02bed0 comp rx=0 tx=0).stop 2024-04-20T11:34:28.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.597+0000 7fbcfb200640 1 -- 172.21.15.152:0/866220195 shutdown_connections 2024-04-20T11:34:28.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.597+0000 7fbcfb200640 1 --2- 172.21.15.152:0/866220195 >> v2:172.21.15.175:3300/0 conn(0x7fbcfc071cd0 0x7fbcfc072130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:28.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.597+0000 7fbcfb200640 1 --2- 172.21.15.152:0/866220195 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbcfc147410 0x7fbcfc0afa50 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:28.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.597+0000 7fbcfb200640 1 -- 172.21.15.152:0/866220195 >> 172.21.15.152:0/866220195 conn(0x7fbcfc067ae0 msgr2=0x7fbcfc06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:28.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.597+0000 7fbcfb200640 1 -- 172.21.15.152:0/866220195 shutdown_connections 2024-04-20T11:34:28.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.597+0000 7fbcfb200640 1 -- 172.21.15.152:0/866220195 wait complete. 2024-04-20T11:34:28.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.599+0000 7fbcfb200640 1 Processor -- start 2024-04-20T11:34:28.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.599+0000 7fbcfb200640 1 -- start start 2024-04-20T11:34:28.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.600+0000 7fbcfb200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbcfc06e010 0x7fbcfc0b5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:28.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.600+0000 7fbcfb200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbcfc071cd0 0x7fbcfc139370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:28.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.601+0000 7fbcfb200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbcfc147410 0x7fbcfc1398b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:28.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.601+0000 7fbcfb200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fbcfc0b3390 con 0x7fbcfc06e010 2024-04-20T11:34:28.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.601+0000 7fbcfb200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbcfc0ae980 con 0x7fbcfc147410 2024-04-20T11:34:28.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.601+0000 7fbcfb200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fbcfc0b3510 con 0x7fbcfc071cd0 2024-04-20T11:34:28.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.601+0000 7fbcf9e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbcfc06e010 0x7fbcfc0b5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:28.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.601+0000 7fbcf9e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbcfc06e010 0x7fbcfc0b5180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:33492/0 (socket says 172.21.15.152:33492) 2024-04-20T11:34:28.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.601+0000 7fbcf9e00640 1 -- 172.21.15.152:0/4201218906 learned_addr learned my addr 172.21.15.152:0/4201218906 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:28.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.601+0000 7fbcf9e00640 1 -- 172.21.15.152:0/4201218906 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbcfc071cd0 msgr2=0x7fbcfc139370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:28.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.601+0000 7fbcf1400640 1 --2- 172.21.15.152:0/4201218906 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbcfc071cd0 0x7fbcfc139370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:28.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.601+0000 7fbcf9e00640 1 --2- 172.21.15.152:0/4201218906 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbcfc071cd0 0x7fbcfc139370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:28.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.601+0000 7fbcf9e00640 1 -- 172.21.15.152:0/4201218906 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbcfc147410 msgr2=0x7fbcfc1398b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:28.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.601+0000 7fbcf9e00640 1 --2- 172.21.15.152:0/4201218906 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbcfc147410 0x7fbcfc1398b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:28.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.601+0000 7fbcf9e00640 1 -- 172.21.15.152:0/4201218906 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcfc13b670 con 0x7fbcfc06e010 2024-04-20T11:34:28.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.602+0000 7fbcf9e00640 1 --2- 172.21.15.152:0/4201218906 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbcfc06e010 0x7fbcfc0b5180 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7fbcec037040 tx=0x7fbcec004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:28.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.602+0000 7fbcf2a00640 1 -- 172.21.15.152:0/4201218906 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcec039050 con 0x7fbcfc06e010 2024-04-20T11:34:28.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.602+0000 7fbcf2a00640 1 -- 172.21.15.152:0/4201218906 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcec045070 con 0x7fbcfc06e010 2024-04-20T11:34:28.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.602+0000 7fbcf2a00640 1 -- 172.21.15.152:0/4201218906 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcec03e850 con 0x7fbcfc06e010 2024-04-20T11:34:28.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.602+0000 7fbcfb200640 1 -- 172.21.15.152:0/4201218906 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcfc13b950 con 0x7fbcfc06e010 2024-04-20T11:34:28.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.602+0000 7fbcfb200640 1 -- 172.21.15.152:0/4201218906 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcfc13bf10 con 0x7fbcfc06e010 2024-04-20T11:34:28.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.603+0000 7fbcfb200640 1 -- 172.21.15.152:0/4201218906 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcfc075e60 con 0x7fbcfc06e010 2024-04-20T11:34:28.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.604+0000 7fbcf2a00640 1 -- 172.21.15.152:0/4201218906 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 19) ==== 91397+0+0 (secure 0 0 0) 0x7fbcec03e9f0 con 0x7fbcfc06e010 2024-04-20T11:34:28.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.605+0000 7fbcf2a00640 1 --2- 172.21.15.152:0/4201218906 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fbcd806cce0 0x7fbcd806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:28.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.605+0000 7fbcf2a00640 1 -- 172.21.15.152:0/4201218906 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 4943+0+0 (secure 0 0 0) 0x7fbcec0aedb0 con 0x7fbcfc06e010 2024-04-20T11:34:28.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.606+0000 7fbcf1400640 1 --2- 172.21.15.152:0/4201218906 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fbcd806cce0 0x7fbcd806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:28.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.607+0000 7fbcf1400640 1 --2- 172.21.15.152:0/4201218906 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fbcd806cce0 0x7fbcd806f1a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fbcfc15de60 tx=0x7fbce4008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:28.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.610+0000 7fbcf2a00640 1 -- 172.21.15.152:0/4201218906 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fbcec0b3070 con 0x7fbcfc06e010 2024-04-20T11:34:28.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:28.871+0000 7fbcfb200640 1 -- 172.21.15.152:0/4201218906 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_to_v2"]} v 0) -- 0x7fbcfc06bc60 con 0x7fbcfc06e010 2024-04-20T11:34:29.513 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:29.513+0000 7fbcf2a00640 1 -- 172.21.15.152:0/4201218906 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_to_v2"]}]=0 v400) ==== 151+0+0 (secure 0 0 0) 0x7fbcec032e20 con 0x7fbcfc06e010 2024-04-20T11:34:29.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:29.517+0000 7fbcfb200640 1 -- 172.21.15.152:0/4201218906 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fbcd806cce0 msgr2=0x7fbcd806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:29.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:29.517+0000 7fbcfb200640 1 --2- 172.21.15.152:0/4201218906 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fbcd806cce0 0x7fbcd806f1a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fbcfc15de60 tx=0x7fbce4008040 comp rx=0 tx=0).stop 2024-04-20T11:34:29.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:29.517+0000 7fbcfb200640 1 -- 172.21.15.152:0/4201218906 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbcfc06e010 msgr2=0x7fbcfc0b5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:29.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:29.517+0000 7fbcfb200640 1 --2- 172.21.15.152:0/4201218906 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbcfc06e010 0x7fbcfc0b5180 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7fbcec037040 tx=0x7fbcec004a40 comp rx=0 tx=0).stop 2024-04-20T11:34:29.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:29.517+0000 7fbcfb200640 1 -- 172.21.15.152:0/4201218906 shutdown_connections 2024-04-20T11:34:29.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:29.517+0000 7fbcfb200640 1 --2- 172.21.15.152:0/4201218906 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fbcd806cce0 0x7fbcd806f1a0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:29.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:29.517+0000 7fbcfb200640 1 --2- 172.21.15.152:0/4201218906 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbcfc147410 0x7fbcfc1398b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:29.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:29.517+0000 7fbcfb200640 1 --2- 172.21.15.152:0/4201218906 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbcfc071cd0 0x7fbcfc139370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:29.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:29.517+0000 7fbcfb200640 1 --2- 172.21.15.152:0/4201218906 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbcfc06e010 0x7fbcfc0b5180 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:29.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:29.517+0000 7fbcfb200640 1 -- 172.21.15.152:0/4201218906 >> 172.21.15.152:0/4201218906 conn(0x7fbcfc067ae0 msgr2=0x7fbcfc0b0620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:29.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:29.517+0000 7fbcfb200640 1 -- 172.21.15.152:0/4201218906 shutdown_connections 2024-04-20T11:34:29.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:29.517+0000 7fbcfb200640 1 -- 172.21.15.152:0/4201218906 wait complete. 2024-04-20T11:34:29.539 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-20T11:34:29.743 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:29.743+0000 7f69cf200640 1 Processor -- start 2024-04-20T11:34:29.743 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:29.743+0000 7f69cf200640 1 -- start start 2024-04-20T11:34:29.744 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:29.745+0000 7f69cf200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f69d0160220 0x7f69d01606b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:29.745 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:29.745+0000 7f69cf200640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69d006fb40 con 0x7f69d0071cd0 2024-04-20T11:34:29.745 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:29.745+0000 7f69cf200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69d0160f90 con 0x7f69d0141450 2024-04-20T11:34:29.745 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:29.745+0000 7f69cf200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f69d013f8e0 con 0x7f69d0160220 2024-04-20T11:34:29.745 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:29.745+0000 7f69cde00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f69d0071cd0 0x7f69d00720b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:42720/0 (socket says 172.21.15.152:42720) 2024-04-20T11:34:29.745 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:29.745+0000 7f69cde00640 1 -- 172.21.15.152:0/3830500111 learned_addr learned my addr 172.21.15.152:0/3830500111 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:34:29.747 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:29.746+0000 7f69cd400640 1 --2- 172.21.15.152:0/3830500111 >> v2:172.21.15.175:3300/0 conn(0x7f69d0160220 0x7f69d01606b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:29.747 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:29.746+0000 7f69cca00640 1 -- 172.21.15.152:0/3830500111 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2905448017 0 0) 0x7f69d006fb40 con 0x7f69d0071cd0 2024-04-20T11:34:29.747 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:29.746+0000 7f69cca00640 1 -- 172.21.15.152:0/3830500111 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f69b4003520 con 0x7f69d0071cd0 2024-04-20T11:34:29.747 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:29.746+0000 7f69cca00640 1 -- 172.21.15.152:0/3830500111 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 736277109 0 0) 0x7f69b4003520 con 0x7f69d0071cd0 2024-04-20T11:34:29.747 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:29.746+0000 7f69cca00640 1 -- 172.21.15.152:0/3830500111 >> v2:172.21.15.175:3300/0 conn(0x7f69d0160220 msgr2=0x7f69d01606b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:29.748 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:29.746+0000 7f69cca00640 1 --2- 172.21.15.152:0/3830500111 >> v2:172.21.15.175:3300/0 conn(0x7f69d0160220 0x7f69d01606b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:29.748 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:29.746+0000 7f69cca00640 1 -- 172.21.15.152:0/3830500111 >> v1:172.21.15.175:6789/0 conn(0x7f69d0141450 legacy=0x7f69d01418e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:29.748 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:29.746+0000 7f69cca00640 1 -- 172.21.15.152:0/3830500111 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69d006dd90 con 0x7f69d0071cd0 2024-04-20T11:34:29.748 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:29.746+0000 7f69cca00640 1 -- 172.21.15.152:0/3830500111 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f69c00019f0 con 0x7f69d0071cd0 2024-04-20T11:34:29.748 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:29.746+0000 7f69cca00640 1 -- 172.21.15.152:0/3830500111 >> v1:172.21.15.152:6789/0 conn(0x7f69d0071cd0 legacy=0x7f69d00720b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:29.748 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:29.746+0000 7f69cca00640 1 --2- 172.21.15.152:0/3830500111 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f69b4003a30 0x7f69b4003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:29.748 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:29.747+0000 7f69cca00640 1 --2- 172.21.15.152:0/3830500111 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f69b4004800 0x7f69b4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:29.748 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:29.747+0000 7f69cca00640 1 --2- 172.21.15.152:0/3830500111 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f69b4005230 0x7f69b400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:29.748 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:29.747+0000 7f69cca00640 1 -- 172.21.15.152:0/3830500111 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f69d013f8e0 con 0x7f69b4003a30 2024-04-20T11:34:29.748 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:29.747+0000 7f69cca00640 1 -- 172.21.15.152:0/3830500111 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f69b400c190 con 0x7f69b4004800 2024-04-20T11:34:29.748 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:29.747+0000 7f69cca00640 1 -- 172.21.15.152:0/3830500111 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f69b400c390 con 0x7f69b4005230 2024-04-20T11:34:29.748 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:29.747+0000 7f69cde00640 1 --2- 172.21.15.152:0/3830500111 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f69b4003a30 0x7f69b4003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:29.748 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:29.747+0000 7f69cde00640 1 -- 172.21.15.152:0/3830500111 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f69b4005230 msgr2=0x7f69b400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:29.748 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:29.747+0000 7f69cde00640 1 --2- 172.21.15.152:0/3830500111 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f69b4005230 0x7f69b400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:29.748 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:29.747+0000 7f69cde00640 1 -- 172.21.15.152:0/3830500111 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f69b4004800 msgr2=0x7f69b4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:29.749 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:29.747+0000 7f69cde00640 1 --2- 172.21.15.152:0/3830500111 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f69b4004800 0x7f69b4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:29.749 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:29.747+0000 7f69cde00640 1 -- 172.21.15.152:0/3830500111 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69b4003520 con 0x7f69b4003a30 2024-04-20T11:34:29.749 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:29.747+0000 7f69ce800640 1 -- 172.21.15.152:0/3830500111 reap_dead start 2024-04-20T11:34:29.750 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:29.748+0000 7f69cde00640 1 --2- 172.21.15.152:0/3830500111 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f69b4003a30 0x7f69b4003e50 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f69c000a770 tx=0x7f69b4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e03a4fb2a7a7193c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:29.750 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:29.748+0000 7f69cca00640 1 -- 172.21.15.152:0/3830500111 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69c000a390 con 0x7f69b4003a30 2024-04-20T11:34:29.750 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:29.748+0000 7f69cca00640 1 -- 172.21.15.152:0/3830500111 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69c0034650 con 0x7f69b4003a30 2024-04-20T11:34:29.751 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:29.748+0000 7f69cca00640 1 -- 172.21.15.152:0/3830500111 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69c0034a80 con 0x7f69b4003a30 2024-04-20T11:34:29.751 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:29.749+0000 7f69cf200640 1 -- 172.21.15.152:0/3830500111 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f69b4003a30 msgr2=0x7f69b4003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:29.751 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:29.749+0000 7f69cf200640 1 --2- 172.21.15.152:0/3830500111 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f69b4003a30 0x7f69b4003e50 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f69c000a770 tx=0x7f69b4005830 comp rx=0 tx=0).stop 2024-04-20T11:34:29.751 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:29.749+0000 7f69cf200640 1 -- 172.21.15.152:0/3830500111 shutdown_connections 2024-04-20T11:34:29.751 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:29.749+0000 7f69cf200640 1 --2- 172.21.15.152:0/3830500111 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f69b4003a30 0x7f69b4003e50 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:29.751 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:29.749+0000 7f69cf200640 1 -- 172.21.15.152:0/3830500111 >> 172.21.15.152:0/3830500111 conn(0x7f69d0067ae0 msgr2=0x7f69d006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:29.751 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:29.749+0000 7f69cf200640 1 -- 172.21.15.152:0/3830500111 shutdown_connections 2024-04-20T11:34:29.751 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:29.749+0000 7f69cf200640 1 -- 172.21.15.152:0/3830500111 wait complete. 2024-04-20T11:34:29.751 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:29.751+0000 7f69cf200640 1 Processor -- start 2024-04-20T11:34:29.751 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:29.751+0000 7f69cf200640 1 -- start start 2024-04-20T11:34:29.753 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:29.752+0000 7f69cf200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f69b4003a30 0x7f69d0155070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:29.753 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:29.752+0000 7f69cf200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f69d0151660 0x7f69d013a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:29.753 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:29.752+0000 7f69cf200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f69d013c050 0x7f69d0157860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:29.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:29.752+0000 7f69cf200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f69d006e630 con 0x7f69d013c050 2024-04-20T11:34:29.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:29.752+0000 7f69cf200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f69d006e350 con 0x7f69b4003a30 2024-04-20T11:34:29.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:29.752+0000 7f69cf200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f69d013fb90 con 0x7f69d0151660 2024-04-20T11:34:29.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:29.752+0000 7f69ce800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f69d013c050 0x7f69d0157860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:29.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:29.752+0000 7f69ce800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f69d013c050 0x7f69d0157860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:33508/0 (socket says 172.21.15.152:33508) 2024-04-20T11:34:29.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:29.752+0000 7f69ce800640 1 -- 172.21.15.152:0/490929801 learned_addr learned my addr 172.21.15.152:0/490929801 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:29.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:29.753+0000 7f69cd400640 1 --2- 172.21.15.152:0/490929801 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f69d0151660 0x7f69d013a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:29.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:29.753+0000 7f69cde00640 1 --2- 172.21.15.152:0/490929801 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f69b4003a30 0x7f69d0155070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:29.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:29.753+0000 7f69ce800640 1 -- 172.21.15.152:0/490929801 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f69d0151660 msgr2=0x7f69d013a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:29.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:29.753+0000 7f69ce800640 1 --2- 172.21.15.152:0/490929801 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f69d0151660 0x7f69d013a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:29.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:29.753+0000 7f69ce800640 1 -- 172.21.15.152:0/490929801 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f69b4003a30 msgr2=0x7f69d0155070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:29.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:29.753+0000 7f69ce800640 1 --2- 172.21.15.152:0/490929801 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f69b4003a30 0x7f69d0155070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:29.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:29.753+0000 7f69ce800640 1 -- 172.21.15.152:0/490929801 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69d013afb0 con 0x7f69d013c050 2024-04-20T11:34:29.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:29.753+0000 7f69ce800640 1 --2- 172.21.15.152:0/490929801 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f69d013c050 0x7f69d0157860 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f69c800c390 tx=0x7f69c800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:29.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:29.753+0000 7f69c6000640 1 -- 172.21.15.152:0/490929801 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69c8007480 con 0x7f69d013c050 2024-04-20T11:34:29.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:29.753+0000 7f69c6000640 1 -- 172.21.15.152:0/490929801 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69c8004020 con 0x7f69d013c050 2024-04-20T11:34:29.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:29.753+0000 7f69c6000640 1 -- 172.21.15.152:0/490929801 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69c8007640 con 0x7f69d013c050 2024-04-20T11:34:29.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:29.753+0000 7f69cf200640 1 -- 172.21.15.152:0/490929801 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69d013b290 con 0x7f69d013c050 2024-04-20T11:34:29.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:29.754+0000 7f69cf200640 1 -- 172.21.15.152:0/490929801 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69d013b850 con 0x7f69d013c050 2024-04-20T11:34:29.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:29.755+0000 7f69cf200640 1 -- 172.21.15.152:0/490929801 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69d0075e60 con 0x7f69d013c050 2024-04-20T11:34:29.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:29.756+0000 7f69c6000640 1 -- 172.21.15.152:0/490929801 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 19) ==== 91397+0+0 (secure 0 0 0) 0x7f69c8011070 con 0x7f69d013c050 2024-04-20T11:34:29.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:29.757+0000 7f69c6000640 1 --2- 172.21.15.152:0/490929801 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f69ac06cd10 0x7f69ac06f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:29.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:29.757+0000 7f69c6000640 1 -- 172.21.15.152:0/490929801 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 4943+0+0 (secure 0 0 0) 0x7f69c808ad00 con 0x7f69d013c050 2024-04-20T11:34:29.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:29.757+0000 7f69cde00640 1 --2- 172.21.15.152:0/490929801 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f69ac06cd10 0x7f69ac06f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:29.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:29.758+0000 7f69cde00640 1 --2- 172.21.15.152:0/490929801 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f69ac06cd10 0x7f69ac06f1d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f69c0034000 tx=0x7f69c0009030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:29.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:29.761+0000 7f69c6000640 1 -- 172.21.15.152:0/490929801 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f69c801a030 con 0x7f69d013c050 2024-04-20T11:34:29.964 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:29.964+0000 7f69cf200640 1 -- 172.21.15.152:0/490929801 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f69d0072d30 con 0x7f69d013c050 2024-04-20T11:34:29.969 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:29.970+0000 7f69c6000640 1 -- 172.21.15.152:0/490929801 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) ==== 122+0+0 (secure 0 0 0) 0x7f69c8058e50 con 0x7f69d013c050 2024-04-20T11:34:29.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:29.973+0000 7f69cf200640 1 -- 172.21.15.152:0/490929801 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f69ac06cd10 msgr2=0x7f69ac06f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:29.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:29.973+0000 7f69cf200640 1 --2- 172.21.15.152:0/490929801 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f69ac06cd10 0x7f69ac06f1d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f69c0034000 tx=0x7f69c0009030 comp rx=0 tx=0).stop 2024-04-20T11:34:29.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:29.973+0000 7f69cf200640 1 -- 172.21.15.152:0/490929801 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f69d013c050 msgr2=0x7f69d0157860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:29.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:29.973+0000 7f69cf200640 1 --2- 172.21.15.152:0/490929801 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f69d013c050 0x7f69d0157860 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f69c800c390 tx=0x7f69c800c850 comp rx=0 tx=0).stop 2024-04-20T11:34:29.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:29.973+0000 7f69cf200640 1 -- 172.21.15.152:0/490929801 shutdown_connections 2024-04-20T11:34:29.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:29.973+0000 7f69cf200640 1 --2- 172.21.15.152:0/490929801 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f69ac06cd10 0x7f69ac06f1d0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:29.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:29.973+0000 7f69cf200640 1 --2- 172.21.15.152:0/490929801 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f69d013c050 0x7f69d0157860 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:29.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:29.973+0000 7f69cf200640 1 --2- 172.21.15.152:0/490929801 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f69d0151660 0x7f69d013a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:29.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:29.973+0000 7f69cf200640 1 --2- 172.21.15.152:0/490929801 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f69b4003a30 0x7f69d0155070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:29.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:29.973+0000 7f69cf200640 1 -- 172.21.15.152:0/490929801 >> 172.21.15.152:0/490929801 conn(0x7f69d0067ae0 msgr2=0x7f69d015c0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:29.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:29.973+0000 7f69cf200640 1 -- 172.21.15.152:0/490929801 shutdown_connections 2024-04-20T11:34:29.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:29.973+0000 7f69cf200640 1 -- 172.21.15.152:0/490929801 wait complete. 2024-04-20T11:34:29.992 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-20T11:34:29.992 DEBUG:teuthology.orchestra.run.smithi152:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-20T11:34:30.091 DEBUG:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-20T11:34:30.209 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-20T11:34:30.209 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-20T11:34:30.209 DEBUG:teuthology.orchestra.run.smithi152:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:34:30.268 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-20T11:34:30.268 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T11:34:30.268 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:34:30.325 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-20T11:34:30.326 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-20T11:34:30.326 DEBUG:teuthology.orchestra.run.smithi152:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:34:30.383 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-20T11:34:30.383 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T11:34:30.383 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:34:30.440 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-20T11:34:30.440 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-20T11:34:30.440 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:34:30.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:30.651+0000 7f33ed600640 1 Processor -- start 2024-04-20T11:34:30.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:30.651+0000 7f33ed600640 1 -- start start 2024-04-20T11:34:30.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:30.652+0000 7f33ed600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33e806f510 0x7f33e8140dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:30.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:30.652+0000 7f33ed600640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f33e8141310 0x7f33e8071cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:30.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:30.652+0000 7f33ed600640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33e80cf0a0 con 0x7f33e813e530 2024-04-20T11:34:30.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:30.652+0000 7f33ed600640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f33e8141b00 con 0x7f33e8141310 2024-04-20T11:34:30.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:30.652+0000 7f33ed600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f33e80cfc50 con 0x7f33e806f510 2024-04-20T11:34:30.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:30.653+0000 7f33ecc00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f33e8141310 0x7f33e8071cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:30.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:30.653+0000 7f33ecc00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f33e8141310 0x7f33e8071cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:33516/0 (socket says 172.21.15.152:33516) 2024-04-20T11:34:30.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:30.653+0000 7f33ecc00640 1 -- 172.21.15.152:0/1946620585 learned_addr learned my addr 172.21.15.152:0/1946620585 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:30.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:30.653+0000 7f33df400640 1 --2- 172.21.15.152:0/1946620585 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33e806f510 0x7f33e8140dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:30.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:30.653+0000 7f33ecc00640 1 -- 172.21.15.152:0/1946620585 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33e806f510 msgr2=0x7f33e8140dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:30.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:30.653+0000 7f33ecc00640 1 --2- 172.21.15.152:0/1946620585 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33e806f510 0x7f33e8140dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:30.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:30.653+0000 7f33ecc00640 1 -- 172.21.15.152:0/1946620585 >> v1:172.21.15.175:6789/0 conn(0x7f33e813e530 legacy=0x7f33e806ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:30.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:30.653+0000 7f33ecc00640 1 -- 172.21.15.152:0/1946620585 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33e80d00e0 con 0x7f33e8141310 2024-04-20T11:34:30.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:30.654+0000 7f33ecc00640 1 --2- 172.21.15.152:0/1946620585 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f33e8141310 0x7f33e8071cd0 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f33e0009e50 tx=0x7f33e002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f928c203e129bf0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:30.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:30.655+0000 7f33e7400640 1 -- 172.21.15.152:0/1946620585 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33e002ca70 con 0x7f33e8141310 2024-04-20T11:34:30.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:30.655+0000 7f33e7400640 1 -- 172.21.15.152:0/1946620585 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33e002cc10 con 0x7f33e8141310 2024-04-20T11:34:30.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:30.655+0000 7f33e7400640 1 -- 172.21.15.152:0/1946620585 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33e00343f0 con 0x7f33e8141310 2024-04-20T11:34:30.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:30.655+0000 7f33ed600640 1 -- 172.21.15.152:0/1946620585 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f33e8141310 msgr2=0x7f33e8071cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:30.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:30.655+0000 7f33ed600640 1 --2- 172.21.15.152:0/1946620585 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f33e8141310 0x7f33e8071cd0 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f33e0009e50 tx=0x7f33e002bed0 comp rx=0 tx=0).stop 2024-04-20T11:34:30.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:30.655+0000 7f33ed600640 1 -- 172.21.15.152:0/1946620585 shutdown_connections 2024-04-20T11:34:30.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:30.655+0000 7f33ed600640 1 --2- 172.21.15.152:0/1946620585 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f33e8141310 0x7f33e8071cd0 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:30.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:30.655+0000 7f33ed600640 1 --2- 172.21.15.152:0/1946620585 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33e806f510 0x7f33e8140dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:30.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:30.655+0000 7f33ed600640 1 -- 172.21.15.152:0/1946620585 >> 172.21.15.152:0/1946620585 conn(0x7f33e8067ae0 msgr2=0x7f33e806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:30.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:30.656+0000 7f33ed600640 1 -- 172.21.15.152:0/1946620585 shutdown_connections 2024-04-20T11:34:30.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:30.656+0000 7f33ed600640 1 -- 172.21.15.152:0/1946620585 wait complete. 2024-04-20T11:34:30.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:30.657+0000 7f33ed600640 1 Processor -- start 2024-04-20T11:34:30.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:30.658+0000 7f33ed600640 1 -- start start 2024-04-20T11:34:30.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:30.658+0000 7f33ed600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33e806f510 0x7f33e80b1130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:30.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:30.658+0000 7f33ed600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f33e813e530 0x7f33e81f46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:30.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:30.659+0000 7f33ed600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33e8141310 0x7f33e813da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:30.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:30.659+0000 7f33ed600640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f33e80cf8e0 con 0x7f33e813e530 2024-04-20T11:34:30.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:30.659+0000 7f33ed600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f33e80cf600 con 0x7f33e8141310 2024-04-20T11:34:30.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:30.659+0000 7f33ed600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f33e81417c0 con 0x7f33e806f510 2024-04-20T11:34:30.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:30.659+0000 7f33df400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f33e813e530 0x7f33e81f46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:30.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:30.659+0000 7f33df400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f33e813e530 0x7f33e81f46f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:33520/0 (socket says 172.21.15.152:33520) 2024-04-20T11:34:30.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:30.659+0000 7f33df400640 1 -- 172.21.15.152:0/2262573951 learned_addr learned my addr 172.21.15.152:0/2262573951 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:30.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:30.659+0000 7f33ecc00640 1 --2- 172.21.15.152:0/2262573951 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33e8141310 0x7f33e813da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:30.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:30.659+0000 7f33e7e00640 1 --2- 172.21.15.152:0/2262573951 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33e806f510 0x7f33e80b1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:30.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:30.659+0000 7f33df400640 1 -- 172.21.15.152:0/2262573951 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33e806f510 msgr2=0x7f33e80b1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:30.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:30.659+0000 7f33df400640 1 --2- 172.21.15.152:0/2262573951 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33e806f510 0x7f33e80b1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:30.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:30.659+0000 7f33df400640 1 -- 172.21.15.152:0/2262573951 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33e8141310 msgr2=0x7f33e813da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:30.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:30.659+0000 7f33df400640 1 --2- 172.21.15.152:0/2262573951 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33e8141310 0x7f33e813da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:30.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:30.659+0000 7f33df400640 1 -- 172.21.15.152:0/2262573951 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33e80b19f0 con 0x7f33e813e530 2024-04-20T11:34:30.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:30.659+0000 7f33ecc00640 1 --2- 172.21.15.152:0/2262573951 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33e8141310 0x7f33e813da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:34:30.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:30.659+0000 7f33e7e00640 1 --2- 172.21.15.152:0/2262573951 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33e806f510 0x7f33e80b1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:34:30.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:30.660+0000 7f33df400640 1 --2- 172.21.15.152:0/2262573951 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f33e813e530 0x7f33e81f46f0 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f33d000d800 tx=0x7f33d000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:30.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:30.660+0000 7f33e4c00640 1 -- 172.21.15.152:0/2262573951 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33d000b6c0 con 0x7f33e813e530 2024-04-20T11:34:30.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:30.660+0000 7f33e4c00640 1 -- 172.21.15.152:0/2262573951 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33d0019070 con 0x7f33e813e530 2024-04-20T11:34:30.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:30.660+0000 7f33ed600640 1 -- 172.21.15.152:0/2262573951 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33e80b1cd0 con 0x7f33e813e530 2024-04-20T11:34:30.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:30.660+0000 7f33e4c00640 1 -- 172.21.15.152:0/2262573951 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33d000ba80 con 0x7f33e813e530 2024-04-20T11:34:30.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:30.660+0000 7f33ed600640 1 -- 172.21.15.152:0/2262573951 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33e80b2290 con 0x7f33e813e530 2024-04-20T11:34:30.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:30.661+0000 7f33ed600640 1 -- 172.21.15.152:0/2262573951 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33e8075e60 con 0x7f33e813e530 2024-04-20T11:34:30.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:30.662+0000 7f33e4c00640 1 -- 172.21.15.152:0/2262573951 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 19) ==== 91397+0+0 (secure 0 0 0) 0x7f33d000bc20 con 0x7f33e813e530 2024-04-20T11:34:30.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:30.663+0000 7f33e4c00640 1 --2- 172.21.15.152:0/2262573951 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f33c406cce0 0x7f33c406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:30.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:30.663+0000 7f33e4c00640 1 -- 172.21.15.152:0/2262573951 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 4943+0+0 (secure 0 0 0) 0x7f33d008c3f0 con 0x7f33e813e530 2024-04-20T11:34:30.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:30.664+0000 7f33e7e00640 1 --2- 172.21.15.152:0/2262573951 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f33c406cce0 0x7f33c406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:30.664 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:30.665+0000 7f33e7e00640 1 --2- 172.21.15.152:0/2262573951 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f33c406cce0 0x7f33c406f1a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f33e8140fb0 tx=0x7f33d8009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:30.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:30.668+0000 7f33e4c00640 1 -- 172.21.15.152:0/2262573951 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f33d005a5f0 con 0x7f33e813e530 2024-04-20T11:34:30.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:30.907+0000 7f33ed600640 1 -- 172.21.15.152:0/2262573951 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f33e80b3490 con 0x7f33e813e530 2024-04-20T11:34:30.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:30.908+0000 7f33e4c00640 1 -- 172.21.15.152:0/2262573951 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) ==== 94+0+3878 (secure 0 0 0) 0x7f33d0059ef0 con 0x7f33e813e530 2024-04-20T11:34:30.908 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:34:30.908 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":69,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5648,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6834/387198089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":387198089},{"type":"v1","addr":"172.21.15.152:6834","nonce":387198089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":6154,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/1674013715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1674013715},{"type":"v1","addr":"172.21.15.175:6833","nonce":1674013715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":6157,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/3898238490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3898238490},{"type":"v1","addr":"172.21.15.175:6835","nonce":3898238490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":8328,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/3838238799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6833","nonce":3838238799},{"type":"v1","addr":"172.21.15.152:6835","nonce":3838238799}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69}],"filesystems":[]} 2024-04-20T11:34:30.908 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 69 2024-04-20T11:34:30.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:30.911+0000 7f33ed600640 1 -- 172.21.15.152:0/2262573951 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f33c406cce0 msgr2=0x7f33c406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:30.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:30.911+0000 7f33ed600640 1 --2- 172.21.15.152:0/2262573951 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f33c406cce0 0x7f33c406f1a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f33e8140fb0 tx=0x7f33d8009040 comp rx=0 tx=0).stop 2024-04-20T11:34:30.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:30.911+0000 7f33ed600640 1 -- 172.21.15.152:0/2262573951 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f33e813e530 msgr2=0x7f33e81f46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:30.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:30.911+0000 7f33ed600640 1 --2- 172.21.15.152:0/2262573951 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f33e813e530 0x7f33e81f46f0 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f33d000d800 tx=0x7f33d000dcc0 comp rx=0 tx=0).stop 2024-04-20T11:34:30.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:30.911+0000 7f33ed600640 1 -- 172.21.15.152:0/2262573951 shutdown_connections 2024-04-20T11:34:30.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:30.911+0000 7f33ed600640 1 --2- 172.21.15.152:0/2262573951 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f33c406cce0 0x7f33c406f1a0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:30.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:30.911+0000 7f33ed600640 1 --2- 172.21.15.152:0/2262573951 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33e8141310 0x7f33e813da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:30.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:30.911+0000 7f33ed600640 1 --2- 172.21.15.152:0/2262573951 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f33e813e530 0x7f33e81f46f0 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:30.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:30.911+0000 7f33ed600640 1 --2- 172.21.15.152:0/2262573951 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33e806f510 0x7f33e80b1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:30.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:30.911+0000 7f33ed600640 1 -- 172.21.15.152:0/2262573951 >> 172.21.15.152:0/2262573951 conn(0x7f33e8067ae0 msgr2=0x7f33e815a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:30.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:30.911+0000 7f33ed600640 1 -- 172.21.15.152:0/2262573951 shutdown_connections 2024-04-20T11:34:30.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:30.911+0000 7f33ed600640 1 -- 172.21.15.152:0/2262573951 wait complete. 2024-04-20T11:34:30.930 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-20T11:34:30.930 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-20T11:34:30.930 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-20T11:34:30.930 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T11:34:30.931 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-20T11:34:30.931 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-20T11:34:30.931 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-20T11:34:30.931 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T11:34:30.931 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-20T11:34:30.931 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-20T11:34:30.932 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-20T11:34:30.932 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T11:34:30.932 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-20T11:34:30.932 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-20T11:34:30.932 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-20T11:34:30.932 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T11:34:30.950 INFO:tasks.ceph.mds.a:Stopped 2024-04-20T11:34:30.950 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-20T11:34:30.954 INFO:tasks.ceph.mds.b:Stopped 2024-04-20T11:34:30.954 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-20T11:34:30.956 INFO:tasks.ceph.mds.c:Stopped 2024-04-20T11:34:30.956 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-20T11:34:30.958 INFO:tasks.ceph.mds.d:Stopped 2024-04-20T11:34:30.958 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-20T11:34:31.003 INFO:tasks.ceph.mds.a:Started 2024-04-20T11:34:31.003 DEBUG:teuthology.parallel:result is None 2024-04-20T11:34:31.005 INFO:tasks.ceph.mds.b:Started 2024-04-20T11:34:31.005 DEBUG:teuthology.parallel:result is None 2024-04-20T11:34:31.009 INFO:tasks.ceph.mds.c:Started 2024-04-20T11:34:31.009 DEBUG:teuthology.parallel:result is None 2024-04-20T11:34:31.011 INFO:tasks.ceph.mds.d:Started 2024-04-20T11:34:31.011 DEBUG:teuthology.parallel:result is None 2024-04-20T11:34:31.011 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-20T11:34:31.154 INFO:tasks.ceph.mds.a.smithi152.stdout:starting mds.a at 2024-04-20T11:34:31.154 INFO:tasks.ceph.mds.c.smithi152.stdout:starting mds.c at 2024-04-20T11:34:31.156 INFO:tasks.ceph.mds.d.smithi175.stdout:starting mds.d at 2024-04-20T11:34:31.157 INFO:tasks.ceph.mds.b.smithi175.stdout:starting mds.b at 2024-04-20T11:34:31.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.189+0000 7f761fa00640 1 Processor -- start 2024-04-20T11:34:31.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.189+0000 7f761fa00640 1 -- start start 2024-04-20T11:34:31.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.190+0000 7f761fa00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f76180cd7e0 0x7f761806e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:31.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.190+0000 7f761fa00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f761806f110 con 0x7f7618145f00 2024-04-20T11:34:31.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.190+0000 7f761fa00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f761806f3f0 con 0x7f761806e900 2024-04-20T11:34:31.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.190+0000 7f761fa00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f76180cf9f0 con 0x7f76180cd7e0 2024-04-20T11:34:31.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.191+0000 7f761d000640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f7618145f00 0x7f76180cd0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:42764/0 (socket says 172.21.15.152:42764) 2024-04-20T11:34:31.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.191+0000 7f761d000640 1 -- 172.21.15.152:0/4050820973 learned_addr learned my addr 172.21.15.152:0/4050820973 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:34:31.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.191+0000 7f760fe00640 1 --2- 172.21.15.152:0/4050820973 >> v2:172.21.15.175:3300/0 conn(0x7f76180cd7e0 0x7f761806e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:31.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.191+0000 7f760fe00640 1 -- 172.21.15.152:0/4050820973 >> v1:172.21.15.175:6789/0 conn(0x7f761806e900 legacy=0x7f76180cfe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:31.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.191+0000 7f760fe00640 1 -- 172.21.15.152:0/4050820973 >> v1:172.21.15.152:6789/0 conn(0x7f7618145f00 legacy=0x7f76180cd0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:31.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.191+0000 7f760fe00640 1 -- 172.21.15.152:0/4050820973 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76180cf3a0 con 0x7f76180cd7e0 2024-04-20T11:34:31.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.192+0000 7f760fe00640 1 --2- 172.21.15.152:0/4050820973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f76180cd7e0 0x7f761806e3c0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f76180cfb70 tx=0x7f760002b940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96c97217ab9fa927 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:31.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.193+0000 7f7617e00640 1 -- 172.21.15.152:0/4050820973 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7600034070 con 0x7f76180cd7e0 2024-04-20T11:34:31.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.193+0000 7f7617e00640 1 -- 172.21.15.152:0/4050820973 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7600030800 con 0x7f76180cd7e0 2024-04-20T11:34:31.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.193+0000 7f7617e00640 1 -- 172.21.15.152:0/4050820973 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7600030bd0 con 0x7f76180cd7e0 2024-04-20T11:34:31.193 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.193+0000 7f761fa00640 1 -- 172.21.15.152:0/4050820973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f76180cd7e0 msgr2=0x7f761806e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:31.193 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.193+0000 7f761fa00640 1 --2- 172.21.15.152:0/4050820973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f76180cd7e0 0x7f761806e3c0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f76180cfb70 tx=0x7f760002b940 comp rx=0 tx=0).stop 2024-04-20T11:34:31.193 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.193+0000 7f761fa00640 1 -- 172.21.15.152:0/4050820973 shutdown_connections 2024-04-20T11:34:31.193 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.193+0000 7f761fa00640 1 --2- 172.21.15.152:0/4050820973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f76180cd7e0 0x7f761806e3c0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:31.193 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.193+0000 7f761fa00640 1 -- 172.21.15.152:0/4050820973 >> 172.21.15.152:0/4050820973 conn(0x7f7618067cf0 msgr2=0x7f761806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:31.193 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.193+0000 7f761fa00640 1 -- 172.21.15.152:0/4050820973 shutdown_connections 2024-04-20T11:34:31.193 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.193+0000 7f761fa00640 1 -- 172.21.15.152:0/4050820973 wait complete. 2024-04-20T11:34:31.195 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.195+0000 7f761fa00640 1 Processor -- start 2024-04-20T11:34:31.195 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.195+0000 7f761fa00640 1 -- start start 2024-04-20T11:34:31.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.196+0000 7f761fa00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f761806e900 0x7f761813f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:31.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.196+0000 7f761fa00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f76180cd7e0 0x7f76180ca520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:31.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.196+0000 7f761fa00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7618145f00 0x7f7618144b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:31.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.196+0000 7f761fa00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f761806ed50 con 0x7f76180cd7e0 2024-04-20T11:34:31.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.196+0000 7f761fa00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f76181462e0 con 0x7f761806e900 2024-04-20T11:34:31.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.196+0000 7f761fa00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f761806eed0 con 0x7f7618145f00 2024-04-20T11:34:31.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.196+0000 7f760fe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f76180cd7e0 0x7f76180ca520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:31.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.197+0000 7f760fe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f76180cd7e0 0x7f76180ca520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:33546/0 (socket says 172.21.15.152:33546) 2024-04-20T11:34:31.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.197+0000 7f761d000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f761806e900 0x7f761813f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:31.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.197+0000 7f760fe00640 1 -- 172.21.15.152:0/314401167 learned_addr learned my addr 172.21.15.152:0/314401167 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:31.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.197+0000 7f761da00640 1 --2- 172.21.15.152:0/314401167 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7618145f00 0x7f7618144b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:31.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.197+0000 7f760fe00640 1 -- 172.21.15.152:0/314401167 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7618145f00 msgr2=0x7f7618144b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:31.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.197+0000 7f760fe00640 1 --2- 172.21.15.152:0/314401167 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7618145f00 0x7f7618144b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:31.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.197+0000 7f760fe00640 1 -- 172.21.15.152:0/314401167 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f761806e900 msgr2=0x7f761813f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:31.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.197+0000 7f760fe00640 1 --2- 172.21.15.152:0/314401167 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f761806e900 0x7f761813f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:31.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.197+0000 7f760fe00640 1 -- 172.21.15.152:0/314401167 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76180cc650 con 0x7f76180cd7e0 2024-04-20T11:34:31.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.197+0000 7f760fe00640 1 --2- 172.21.15.152:0/314401167 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f76180cd7e0 0x7f76180ca520 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f760002bf50 tx=0x7f76000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:31.199 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.198+0000 7f7615600640 1 -- 172.21.15.152:0/314401167 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7600046020 con 0x7f76180cd7e0 2024-04-20T11:34:31.199 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.198+0000 7f7615600640 1 -- 172.21.15.152:0/314401167 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7600039d10 con 0x7f76180cd7e0 2024-04-20T11:34:31.199 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.198+0000 7f7615600640 1 -- 172.21.15.152:0/314401167 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7600034040 con 0x7f76180cd7e0 2024-04-20T11:34:31.199 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.198+0000 7f761fa00640 1 -- 172.21.15.152:0/314401167 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76180cc930 con 0x7f76180cd7e0 2024-04-20T11:34:31.199 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.198+0000 7f761fa00640 1 -- 172.21.15.152:0/314401167 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76180ccef0 con 0x7f76180cd7e0 2024-04-20T11:34:31.199 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.199+0000 7f761fa00640 1 -- 172.21.15.152:0/314401167 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76180689f0 con 0x7f76180cd7e0 2024-04-20T11:34:31.199 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.200+0000 7f7615600640 1 -- 172.21.15.152:0/314401167 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 19) ==== 91397+0+0 (secure 0 0 0) 0x7f7600037070 con 0x7f76180cd7e0 2024-04-20T11:34:31.201 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.202+0000 7f7615600640 1 --2- 172.21.15.152:0/314401167 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f75f406cd10 0x7f75f406f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:31.201 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.202+0000 7f7615600640 1 -- 172.21.15.152:0/314401167 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 4943+0+0 (secure 0 0 0) 0x7f76000adfc0 con 0x7f76180cd7e0 2024-04-20T11:34:31.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.202+0000 7f761d000640 1 --2- 172.21.15.152:0/314401167 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f75f406cd10 0x7f75f406f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:31.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.203+0000 7f761d000640 1 --2- 172.21.15.152:0/314401167 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f75f406cd10 0x7f75f406f1d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7608005880 tx=0x7f7608009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:31.205 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.206+0000 7f7615600640 1 -- 172.21.15.152:0/314401167 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f7600050020 con 0x7f76180cd7e0 2024-04-20T11:34:31.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.418+0000 7f761fa00640 1 -- 172.21.15.152:0/314401167 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f76180b1ec0 con 0x7f76180cd7e0 2024-04-20T11:34:31.541 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.541+0000 7f7615600640 1 -- 172.21.15.152:0/314401167 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v112) ==== 97+0+0 (secure 0 0 0) 0x7f760007ba00 con 0x7f76180cd7e0 2024-04-20T11:34:31.541 INFO:teuthology.orchestra.run.smithi152.stderr: removed all blocklist entries 2024-04-20T11:34:31.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.546+0000 7f761fa00640 1 -- 172.21.15.152:0/314401167 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f75f406cd10 msgr2=0x7f75f406f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:31.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.546+0000 7f761fa00640 1 --2- 172.21.15.152:0/314401167 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f75f406cd10 0x7f75f406f1d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7608005880 tx=0x7f7608009040 comp rx=0 tx=0).stop 2024-04-20T11:34:31.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.546+0000 7f761fa00640 1 -- 172.21.15.152:0/314401167 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f76180cd7e0 msgr2=0x7f76180ca520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:31.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.546+0000 7f761fa00640 1 --2- 172.21.15.152:0/314401167 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f76180cd7e0 0x7f76180ca520 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f760002bf50 tx=0x7f76000046c0 comp rx=0 tx=0).stop 2024-04-20T11:34:31.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.546+0000 7f761fa00640 1 -- 172.21.15.152:0/314401167 shutdown_connections 2024-04-20T11:34:31.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.546+0000 7f761fa00640 1 --2- 172.21.15.152:0/314401167 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f75f406cd10 0x7f75f406f1d0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:31.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.546+0000 7f761fa00640 1 --2- 172.21.15.152:0/314401167 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7618145f00 0x7f7618144b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:31.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.546+0000 7f761fa00640 1 --2- 172.21.15.152:0/314401167 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f76180cd7e0 0x7f76180ca520 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:31.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.546+0000 7f761fa00640 1 --2- 172.21.15.152:0/314401167 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f761806e900 0x7f761813f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:31.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.546+0000 7f761fa00640 1 -- 172.21.15.152:0/314401167 >> 172.21.15.152:0/314401167 conn(0x7f7618067cf0 msgr2=0x7f761815ccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:31.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.546+0000 7f761fa00640 1 -- 172.21.15.152:0/314401167 shutdown_connections 2024-04-20T11:34:31.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.546+0000 7f761fa00640 1 -- 172.21.15.152:0/314401167 wait complete. 2024-04-20T11:34:31.568 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-20T11:34:31.750 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.750+0000 7f1a6de00640 1 Processor -- start 2024-04-20T11:34:31.750 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.751+0000 7f1a6de00640 1 -- start start 2024-04-20T11:34:31.752 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.752+0000 7f1a6de00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f1a680d1790 0x7f1a680d3a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:31.752 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.752+0000 7f1a6de00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a680cf470 con 0x7f1a680d1c20 2024-04-20T11:34:31.752 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.752+0000 7f1a6de00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a680cfa50 con 0x7f1a6815e9b0 2024-04-20T11:34:31.752 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.752+0000 7f1a6de00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f1a680d0010 con 0x7f1a680d1790 2024-04-20T11:34:31.752 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.753+0000 7f1a67400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f1a680d1c20 0x7f1a680d3fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:42768/0 (socket says 172.21.15.152:42768) 2024-04-20T11:34:31.752 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.753+0000 7f1a67400640 1 -- 172.21.15.152:0/232849902 learned_addr learned my addr 172.21.15.152:0/232849902 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:34:31.752 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.753+0000 7f1a66000640 1 --2- 172.21.15.152:0/232849902 >> v2:172.21.15.175:3300/0 conn(0x7f1a680d1790 0x7f1a680d3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:31.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.753+0000 7f1a65600640 1 -- 172.21.15.152:0/232849902 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 816248328 0 0) 0x7f1a680cf470 con 0x7f1a680d1c20 2024-04-20T11:34:31.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.753+0000 7f1a65600640 1 -- 172.21.15.152:0/232849902 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1a48003520 con 0x7f1a680d1c20 2024-04-20T11:34:31.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.754+0000 7f1a65600640 1 -- 172.21.15.152:0/232849902 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3829514223 0 0) 0x7f1a48003520 con 0x7f1a680d1c20 2024-04-20T11:34:31.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.754+0000 7f1a65600640 1 -- 172.21.15.152:0/232849902 >> v2:172.21.15.175:3300/0 conn(0x7f1a680d1790 msgr2=0x7f1a680d3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:31.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.754+0000 7f1a65600640 1 --2- 172.21.15.152:0/232849902 >> v2:172.21.15.175:3300/0 conn(0x7f1a680d1790 0x7f1a680d3a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:31.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.754+0000 7f1a65600640 1 -- 172.21.15.152:0/232849902 >> v1:172.21.15.175:6789/0 conn(0x7f1a6815e9b0 legacy=0x7f1a680d1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:31.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.754+0000 7f1a65600640 1 -- 172.21.15.152:0/232849902 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a680cfd30 con 0x7f1a680d1c20 2024-04-20T11:34:31.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.754+0000 7f1a65600640 1 -- 172.21.15.152:0/232849902 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f1a60001670 con 0x7f1a680d1c20 2024-04-20T11:34:31.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.754+0000 7f1a65600640 1 -- 172.21.15.152:0/232849902 >> v1:172.21.15.152:6789/0 conn(0x7f1a680d1c20 legacy=0x7f1a680d3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:31.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.754+0000 7f1a65600640 1 --2- 172.21.15.152:0/232849902 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1a48003a70 0x7f1a48003e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:31.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.754+0000 7f1a65600640 1 --2- 172.21.15.152:0/232849902 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1a48004800 0x7f1a48004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:31.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.754+0000 7f1a65600640 1 --2- 172.21.15.152:0/232849902 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1a480052e0 0x7f1a4800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:31.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.754+0000 7f1a65600640 1 -- 172.21.15.152:0/232849902 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f1a680d0010 con 0x7f1a48004800 2024-04-20T11:34:31.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.754+0000 7f1a65600640 1 -- 172.21.15.152:0/232849902 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1a4800c1b0 con 0x7f1a48003a70 2024-04-20T11:34:31.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.754+0000 7f1a65600640 1 -- 172.21.15.152:0/232849902 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1a4800c3b0 con 0x7f1a480052e0 2024-04-20T11:34:31.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.754+0000 7f1a66000640 1 --2- 172.21.15.152:0/232849902 >> v2:172.21.15.175:3300/0 conn(0x7f1a680d1790 0x7f1a680d3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:34:31.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.754+0000 7f1a66000640 1 --2- 172.21.15.152:0/232849902 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1a48004800 0x7f1a48004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:31.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.754+0000 7f1a66a00640 1 --2- 172.21.15.152:0/232849902 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1a48003a70 0x7f1a48003e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:31.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.755+0000 7f1a66000640 1 -- 172.21.15.152:0/232849902 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1a480052e0 msgr2=0x7f1a4800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:31.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.755+0000 7f1a66000640 1 --2- 172.21.15.152:0/232849902 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1a480052e0 0x7f1a4800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:31.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.755+0000 7f1a66000640 1 -- 172.21.15.152:0/232849902 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1a48003a70 msgr2=0x7f1a48003e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:31.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.755+0000 7f1a66000640 1 --2- 172.21.15.152:0/232849902 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1a48003a70 0x7f1a48003e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:31.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.755+0000 7f1a66000640 1 -- 172.21.15.152:0/232849902 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a48003520 con 0x7f1a48004800 2024-04-20T11:34:31.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.755+0000 7f1a67400640 1 -- 172.21.15.152:0/232849902 reap_dead start 2024-04-20T11:34:31.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.756+0000 7f1a66000640 1 --2- 172.21.15.152:0/232849902 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1a48004800 0x7f1a48004c80 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f1a5000e350 tx=0x7f1a48005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=311d569835cf1cc server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:31.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.756+0000 7f1a65600640 1 -- 172.21.15.152:0/232849902 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a50009dc0 con 0x7f1a48004800 2024-04-20T11:34:31.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.756+0000 7f1a65600640 1 -- 172.21.15.152:0/232849902 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a5002cd70 con 0x7f1a48004800 2024-04-20T11:34:31.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.756+0000 7f1a65600640 1 -- 172.21.15.152:0/232849902 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a50003b90 con 0x7f1a48004800 2024-04-20T11:34:31.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.757+0000 7f1a6de00640 1 -- 172.21.15.152:0/232849902 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1a48004800 msgr2=0x7f1a48004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:31.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.757+0000 7f1a6de00640 1 --2- 172.21.15.152:0/232849902 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1a48004800 0x7f1a48004c80 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f1a5000e350 tx=0x7f1a48005830 comp rx=0 tx=0).stop 2024-04-20T11:34:31.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.757+0000 7f1a6de00640 1 -- 172.21.15.152:0/232849902 shutdown_connections 2024-04-20T11:34:31.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.757+0000 7f1a6de00640 1 --2- 172.21.15.152:0/232849902 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1a48004800 0x7f1a48004c80 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:31.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.757+0000 7f1a6de00640 1 -- 172.21.15.152:0/232849902 >> 172.21.15.152:0/232849902 conn(0x7f1a6806cc70 msgr2=0x7f1a6806d080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:31.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.757+0000 7f1a6de00640 1 -- 172.21.15.152:0/232849902 shutdown_connections 2024-04-20T11:34:31.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.757+0000 7f1a6de00640 1 -- 172.21.15.152:0/232849902 wait complete. 2024-04-20T11:34:31.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.758+0000 7f1a6de00640 1 Processor -- start 2024-04-20T11:34:31.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.759+0000 7f1a6de00640 1 -- start start 2024-04-20T11:34:31.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.759+0000 7f1a6de00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1a48004800 0x7f1a68157bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:31.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.759+0000 7f1a6de00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1a680cb870 0x7f1a68153570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:31.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.759+0000 7f1a6de00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1a680cc0f0 0x7f1a680cc570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:31.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.759+0000 7f1a6de00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f1a6815ed90 con 0x7f1a680cb870 2024-04-20T11:34:31.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.759+0000 7f1a6de00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1a680cf800 con 0x7f1a48004800 2024-04-20T11:34:31.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.759+0000 7f1a6de00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1a6815ef10 con 0x7f1a680cc0f0 2024-04-20T11:34:31.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.760+0000 7f1a66000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1a680cb870 0x7f1a68153570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:31.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.760+0000 7f1a66000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1a680cb870 0x7f1a68153570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:33556/0 (socket says 172.21.15.152:33556) 2024-04-20T11:34:31.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.760+0000 7f1a66000640 1 -- 172.21.15.152:0/483890774 learned_addr learned my addr 172.21.15.152:0/483890774 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:31.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.760+0000 7f1a66a00640 1 --2- 172.21.15.152:0/483890774 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1a48004800 0x7f1a68157bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:31.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.760+0000 7f1a67400640 1 --2- 172.21.15.152:0/483890774 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1a680cc0f0 0x7f1a680cc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:31.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.760+0000 7f1a66000640 1 -- 172.21.15.152:0/483890774 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1a680cc0f0 msgr2=0x7f1a680cc570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:31.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.760+0000 7f1a66000640 1 --2- 172.21.15.152:0/483890774 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1a680cc0f0 0x7f1a680cc570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:31.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.760+0000 7f1a66000640 1 -- 172.21.15.152:0/483890774 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1a48004800 msgr2=0x7f1a68157bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:31.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.760+0000 7f1a66000640 1 --2- 172.21.15.152:0/483890774 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1a48004800 0x7f1a68157bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:31.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.760+0000 7f1a66000640 1 -- 172.21.15.152:0/483890774 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a680cdf80 con 0x7f1a680cb870 2024-04-20T11:34:31.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.761+0000 7f1a66000640 1 --2- 172.21.15.152:0/483890774 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1a680cb870 0x7f1a68153570 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f1a5000e470 tx=0x7f1a50009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:31.762 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.761+0000 7f1a5f400640 1 -- 172.21.15.152:0/483890774 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a50044020 con 0x7f1a680cb870 2024-04-20T11:34:31.762 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.761+0000 7f1a5f400640 1 -- 172.21.15.152:0/483890774 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a50040680 con 0x7f1a680cb870 2024-04-20T11:34:31.762 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.761+0000 7f1a5f400640 1 -- 172.21.15.152:0/483890774 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a50009dc0 con 0x7f1a680cb870 2024-04-20T11:34:31.762 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.761+0000 7f1a6de00640 1 -- 172.21.15.152:0/483890774 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a680ce220 con 0x7f1a680cb870 2024-04-20T11:34:31.762 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.761+0000 7f1a6de00640 1 -- 172.21.15.152:0/483890774 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a680ce7e0 con 0x7f1a680cb870 2024-04-20T11:34:31.762 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.762+0000 7f1a6de00640 1 -- 172.21.15.152:0/483890774 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a68069100 con 0x7f1a680cb870 2024-04-20T11:34:31.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.764+0000 7f1a5f400640 1 -- 172.21.15.152:0/483890774 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 19) ==== 91397+0+0 (secure 0 0 0) 0x7f1a50047020 con 0x7f1a680cb870 2024-04-20T11:34:31.764 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.764+0000 7f1a5f400640 1 --2- 172.21.15.152:0/483890774 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f1a4406cd10 0x7f1a4406f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:31.764 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.765+0000 7f1a5f400640 1 -- 172.21.15.152:0/483890774 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 4857+0+0 (secure 0 0 0) 0x7f1a500aeff0 con 0x7f1a680cb870 2024-04-20T11:34:31.764 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.765+0000 7f1a66a00640 1 --2- 172.21.15.152:0/483890774 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f1a4406cd10 0x7f1a4406f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:31.766 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.766+0000 7f1a66a00640 1 --2- 172.21.15.152:0/483890774 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f1a4406cd10 0x7f1a4406f1d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1a58007fd0 tx=0x7f1a58009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:31.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:31.769+0000 7f1a5f400640 1 -- 172.21.15.152:0/483890774 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f1a5004e070 con 0x7f1a680cb870 2024-04-20T11:34:32.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.040+0000 7f1a6de00640 1 -- 172.21.15.152:0/483890774 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f1a680d2520 con 0x7f1a680cb870 2024-04-20T11:34:32.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.041+0000 7f1a5f400640 1 -- 172.21.15.152:0/483890774 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) ==== 80+0+5304 (secure 0 0 0) 0x7f1a5007d0c0 con 0x7f1a680cb870 2024-04-20T11:34:32.041 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:34:32.041 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-04-20T11:34:32.041 INFO:teuthology.orchestra.run.smithi152.stdout: "auth_dump": [ 2024-04-20T11:34:32.041 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:34:32.041 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "mds.a", 2024-04-20T11:34:32.041 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBCpiNmyFRJIBAAj2aIMXdb65nqRgKFDBLLzA==", 2024-04-20T11:34:32.041 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:34:32.041 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow", 2024-04-20T11:34:32.041 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow *", 2024-04-20T11:34:32.041 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow *", 2024-04-20T11:34:32.041 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:34:32.041 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:34:32.041 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:34:32.042 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:34:32.042 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "mds.b", 2024-04-20T11:34:32.042 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBCpiNmVPytOhAAzQkvNOJaNy+MeKETWhYLyg==", 2024-04-20T11:34:32.042 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:34:32.042 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow", 2024-04-20T11:34:32.042 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow *", 2024-04-20T11:34:32.042 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow *", 2024-04-20T11:34:32.042 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:34:32.042 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:34:32.042 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:34:32.043 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:34:32.043 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "mds.c", 2024-04-20T11:34:32.043 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBCpiNm5ViKLRAAg7VBQG4hFTLYUpJMpBAaVA==", 2024-04-20T11:34:32.043 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:34:32.043 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow", 2024-04-20T11:34:32.043 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow *", 2024-04-20T11:34:32.044 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow *", 2024-04-20T11:34:32.044 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:34:32.044 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:34:32.044 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:34:32.044 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:34:32.044 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "mds.d", 2024-04-20T11:34:32.044 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBDpiNmLWhaCxAA2e4qd/7+TKenJlWICIsvGg==", 2024-04-20T11:34:32.044 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:34:32.045 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow", 2024-04-20T11:34:32.045 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow *", 2024-04-20T11:34:32.045 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow *", 2024-04-20T11:34:32.045 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:34:32.045 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:34:32.045 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:34:32.045 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:34:32.045 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "osd.0", 2024-04-20T11:34:32.045 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBHpiNmpuhyDBAAsQXX9DNvWlo0UYh53lb+FA==", 2024-04-20T11:34:32.045 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:34:32.045 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow profile osd", 2024-04-20T11:34:32.045 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile osd", 2024-04-20T11:34:32.046 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:34:32.046 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:34:32.046 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:34:32.046 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:34:32.046 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "osd.1", 2024-04-20T11:34:32.046 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBKpiNmxBiLJhAAHon1adC8G2jbYoDcWdE9jA==", 2024-04-20T11:34:32.046 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:34:32.046 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow profile osd", 2024-04-20T11:34:32.046 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile osd", 2024-04-20T11:34:32.046 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:34:32.046 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:34:32.046 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:34:32.047 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:34:32.047 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "osd.2", 2024-04-20T11:34:32.047 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBNpiNmdn0zMhAAqwIO2kKahdSzTVh8/p3FCg==", 2024-04-20T11:34:32.047 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:34:32.047 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow profile osd", 2024-04-20T11:34:32.047 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile osd", 2024-04-20T11:34:32.047 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:34:32.047 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:34:32.047 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:34:32.047 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:34:32.047 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "osd.3", 2024-04-20T11:34:32.047 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBRpiNmTPCYAhAAHFNHX/xKn21jEFHDkHMjEw==", 2024-04-20T11:34:32.048 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:34:32.048 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow profile osd", 2024-04-20T11:34:32.048 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile osd", 2024-04-20T11:34:32.048 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:34:32.048 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:34:32.048 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:34:32.048 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:34:32.048 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "osd.4", 2024-04-20T11:34:32.048 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBXpiNmM8cVFxAAJl9Hu+MvlkEiMglutI89eg==", 2024-04-20T11:34:32.048 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:34:32.048 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow profile osd", 2024-04-20T11:34:32.048 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile osd", 2024-04-20T11:34:32.049 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:34:32.049 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:34:32.049 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:34:32.049 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:34:32.049 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "osd.5", 2024-04-20T11:34:32.049 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBapiNmnGxJMRAA90P0T+VsTA8LCTcexTVr7w==", 2024-04-20T11:34:32.049 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:34:32.049 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow profile osd", 2024-04-20T11:34:32.049 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile osd", 2024-04-20T11:34:32.049 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:34:32.049 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:34:32.049 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:34:32.049 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:34:32.049 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "osd.6", 2024-04-20T11:34:32.050 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBepiNmyLXWDxAAhtHD5h478mLovE5IejAFng==", 2024-04-20T11:34:32.050 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:34:32.050 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow profile osd", 2024-04-20T11:34:32.050 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile osd", 2024-04-20T11:34:32.050 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:34:32.050 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:34:32.050 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:34:32.050 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:34:32.050 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "osd.7", 2024-04-20T11:34:32.050 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBhpiNmB3jwHBAAqco9om6r2N/GqAV6rVKiaA==", 2024-04-20T11:34:32.050 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:34:32.050 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow profile osd", 2024-04-20T11:34:32.050 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile osd", 2024-04-20T11:34:32.051 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:34:32.051 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:34:32.051 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:34:32.051 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:34:32.051 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "client.0", 2024-04-20T11:34:32.051 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBDpiNmtsBWFhAArq9A7r33CcwgMmOU2pAGPw==", 2024-04-20T11:34:32.051 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:34:32.051 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow", 2024-04-20T11:34:32.051 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow r", 2024-04-20T11:34:32.051 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-20T11:34:32.051 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:34:32.051 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:34:32.051 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:34:32.051 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "client.1", 2024-04-20T11:34:32.052 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBDpiNm6aNnJxAAB431GSGytZ0IKWMh4DMalw==", 2024-04-20T11:34:32.052 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:34:32.052 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow", 2024-04-20T11:34:32.052 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow r", 2024-04-20T11:34:32.052 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow rw", 2024-04-20T11:34:32.052 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow rwx" 2024-04-20T11:34:32.052 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:34:32.052 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:34:32.052 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:34:32.052 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "client.2", 2024-04-20T11:34:32.052 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBDpiNm5QLtHRAAa2kHTDVdgSh/B4XGZaKa5Q==", 2024-04-20T11:34:32.052 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:34:32.053 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow", 2024-04-20T11:34:32.053 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow r", 2024-04-20T11:34:32.053 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow rw", 2024-04-20T11:34:32.053 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow rwx" 2024-04-20T11:34:32.053 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:34:32.053 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:34:32.053 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:34:32.053 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "client.3", 2024-04-20T11:34:32.053 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBDpiNmw95SMBAAdodzTuPp4WdKSEf+jI8Uuw==", 2024-04-20T11:34:32.053 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:34:32.053 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow", 2024-04-20T11:34:32.053 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow r", 2024-04-20T11:34:32.053 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow rw", 2024-04-20T11:34:32.054 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow rwx" 2024-04-20T11:34:32.054 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:34:32.054 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:34:32.054 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:34:32.054 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "client.admin", 2024-04-20T11:34:32.054 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBBpiNm3xz6HhAA82+N+trDW/AmBr3q8l4ECw==", 2024-04-20T11:34:32.054 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:34:32.054 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow *", 2024-04-20T11:34:32.054 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow *", 2024-04-20T11:34:32.054 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow *", 2024-04-20T11:34:32.054 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:34:32.054 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:34:32.055 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:34:32.055 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:34:32.055 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "mgr.x", 2024-04-20T11:34:32.055 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBCpiNmuhalFxAA9dTl/zSbUovCCZAanAipww==", 2024-04-20T11:34:32.055 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:34:32.055 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow *", 2024-04-20T11:34:32.055 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile mgr", 2024-04-20T11:34:32.055 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:34:32.055 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:34:32.055 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:34:32.055 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:34:32.055 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "mgr.y", 2024-04-20T11:34:32.056 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBCpiNmP1+5DRAAs6bVAEOr0i6FdPmSEyo/vQ==", 2024-04-20T11:34:32.056 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:34:32.056 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow *", 2024-04-20T11:34:32.056 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile mgr", 2024-04-20T11:34:32.056 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:34:32.056 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:34:32.056 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:34:32.056 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-04-20T11:34:32.056 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-04-20T11:34:32.057 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.044+0000 7f1a6de00640 1 -- 172.21.15.152:0/483890774 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f1a4406cd10 msgr2=0x7f1a4406f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:32.057 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.044+0000 7f1a6de00640 1 --2- 172.21.15.152:0/483890774 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f1a4406cd10 0x7f1a4406f1d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1a58007fd0 tx=0x7f1a58009040 comp rx=0 tx=0).stop 2024-04-20T11:34:32.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.044+0000 7f1a6de00640 1 -- 172.21.15.152:0/483890774 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1a680cb870 msgr2=0x7f1a68153570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:32.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.044+0000 7f1a6de00640 1 --2- 172.21.15.152:0/483890774 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1a680cb870 0x7f1a68153570 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f1a5000e470 tx=0x7f1a50009750 comp rx=0 tx=0).stop 2024-04-20T11:34:32.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.044+0000 7f1a6de00640 1 -- 172.21.15.152:0/483890774 shutdown_connections 2024-04-20T11:34:32.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.044+0000 7f1a6de00640 1 --2- 172.21.15.152:0/483890774 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f1a4406cd10 0x7f1a4406f1d0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:32.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.044+0000 7f1a6de00640 1 --2- 172.21.15.152:0/483890774 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1a680cc0f0 0x7f1a680cc570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:32.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.044+0000 7f1a6de00640 1 --2- 172.21.15.152:0/483890774 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1a680cb870 0x7f1a68153570 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:32.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.044+0000 7f1a6de00640 1 --2- 172.21.15.152:0/483890774 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1a48004800 0x7f1a68157bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:32.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.044+0000 7f1a6de00640 1 -- 172.21.15.152:0/483890774 >> 172.21.15.152:0/483890774 conn(0x7f1a6806cc70 msgr2=0x7f1a680b7bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:32.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.044+0000 7f1a6de00640 1 -- 172.21.15.152:0/483890774 shutdown_connections 2024-04-20T11:34:32.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.044+0000 7f1a6de00640 1 -- 172.21.15.152:0/483890774 wait complete. 2024-04-20T11:34:32.062 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:34:32.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.240+0000 7fb700600640 1 Processor -- start 2024-04-20T11:34:32.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.240+0000 7fb700600640 1 -- start start 2024-04-20T11:34:32.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.242+0000 7fb700600640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fb6f815eb80 0x7fb6f80b79a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:32.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.242+0000 7fb700600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb6f80ba520 0x7fb6f80d28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:32.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.242+0000 7fb700600640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6f80b8330 con 0x7fb6f80d2e30 2024-04-20T11:34:32.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.242+0000 7fb700600640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fb6f80b8610 con 0x7fb6f815eb80 2024-04-20T11:34:32.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.242+0000 7fb700600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb6f80b88f0 con 0x7fb6f80ba520 2024-04-20T11:34:32.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.242+0000 7fb6fdc00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fb6f815eb80 0x7fb6f80b79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:32.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.242+0000 7fb6fe600640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fb6f80d2e30 0x7fb6f80d1810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:42780/0 (socket says 172.21.15.152:42780) 2024-04-20T11:34:32.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.242+0000 7fb6fe600640 1 -- 172.21.15.152:0/342089864 learned_addr learned my addr 172.21.15.152:0/342089864 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:34:32.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.243+0000 7fb6fd200640 1 --2- 172.21.15.152:0/342089864 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb6f80ba520 0x7fb6f80d28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:32.243 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.243+0000 7fb6fdc00640 1 -- 172.21.15.152:0/342089864 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb6f80ba520 msgr2=0x7fb6f80d28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:32.243 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.243+0000 7fb6fdc00640 1 --2- 172.21.15.152:0/342089864 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb6f80ba520 0x7fb6f80d28f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:32.243 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.243+0000 7fb6fdc00640 1 -- 172.21.15.152:0/342089864 >> v1:172.21.15.152:6789/0 conn(0x7fb6f80d2e30 legacy=0x7fb6f80d1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:32.243 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.243+0000 7fb6fdc00640 1 -- 172.21.15.152:0/342089864 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6f80b8bd0 con 0x7fb6f815eb80 2024-04-20T11:34:32.243 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.243+0000 7fb6f7e00640 1 -- 172.21.15.152:0/342089864 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 544346653 0 0) 0x7fb6f80b8330 con 0x7fb6f80d2e30 2024-04-20T11:34:32.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.244+0000 7fb6fdc00640 1 --2- 172.21.15.152:0/342089864 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb6f815eb80 0x7fb6f80b79a0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec0097a0 tx=0x7fb6ec02b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc60e150c75106f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:32.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.244+0000 7fb6f7e00640 1 -- 172.21.15.152:0/342089864 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6ec031780 con 0x7fb6f815eb80 2024-04-20T11:34:32.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.244+0000 7fb6f7e00640 1 -- 172.21.15.152:0/342089864 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6ec037070 con 0x7fb6f815eb80 2024-04-20T11:34:32.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.244+0000 7fb6f7e00640 1 -- 172.21.15.152:0/342089864 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6ec031b40 con 0x7fb6f815eb80 2024-04-20T11:34:32.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.245+0000 7fb700600640 1 -- 172.21.15.152:0/342089864 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb6f815eb80 msgr2=0x7fb6f80b79a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:32.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.245+0000 7fb700600640 1 --2- 172.21.15.152:0/342089864 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb6f815eb80 0x7fb6f80b79a0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec0097a0 tx=0x7fb6ec02b820 comp rx=0 tx=0).stop 2024-04-20T11:34:32.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.245+0000 7fb700600640 1 -- 172.21.15.152:0/342089864 shutdown_connections 2024-04-20T11:34:32.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.245+0000 7fb700600640 1 --2- 172.21.15.152:0/342089864 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb6f80ba520 0x7fb6f80d28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:32.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.245+0000 7fb700600640 1 --2- 172.21.15.152:0/342089864 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb6f815eb80 0x7fb6f80b79a0 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:32.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.245+0000 7fb700600640 1 -- 172.21.15.152:0/342089864 >> 172.21.15.152:0/342089864 conn(0x7fb6f8067ae0 msgr2=0x7fb6f8067ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:32.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.245+0000 7fb700600640 1 -- 172.21.15.152:0/342089864 shutdown_connections 2024-04-20T11:34:32.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.245+0000 7fb700600640 1 -- 172.21.15.152:0/342089864 wait complete. 2024-04-20T11:34:32.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.247+0000 7fb700600640 1 Processor -- start 2024-04-20T11:34:32.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.247+0000 7fb700600640 1 -- start start 2024-04-20T11:34:32.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.248+0000 7fb700600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb6f80ba520 0x7fb6f8160a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:32.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.248+0000 7fb700600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb6f80d2e30 0x7fb6f8154c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:32.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.248+0000 7fb700600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb6f815eb80 0x7fb6f8158f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:32.249 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.248+0000 7fb700600640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f80d1f20 con 0x7fb6f80ba520 2024-04-20T11:34:32.249 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.248+0000 7fb700600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f80b80f0 con 0x7fb6f815eb80 2024-04-20T11:34:32.249 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.248+0000 7fb700600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb6f80d20a0 con 0x7fb6f80d2e30 2024-04-20T11:34:32.249 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.248+0000 7fb6fdc00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb6f80ba520 0x7fb6f8160a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:32.249 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.248+0000 7fb6fdc00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb6f80ba520 0x7fb6f8160a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:33572/0 (socket says 172.21.15.152:33572) 2024-04-20T11:34:32.249 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.248+0000 7fb6fdc00640 1 -- 172.21.15.152:0/229624423 learned_addr learned my addr 172.21.15.152:0/229624423 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:32.249 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.248+0000 7fb6fdc00640 1 -- 172.21.15.152:0/229624423 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb6f80d2e30 msgr2=0x7fb6f8154c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:32.249 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.248+0000 7fb6fdc00640 1 --2- 172.21.15.152:0/229624423 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb6f80d2e30 0x7fb6f8154c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:32.249 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.248+0000 7fb6fdc00640 1 -- 172.21.15.152:0/229624423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb6f815eb80 msgr2=0x7fb6f8158f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:32.249 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.248+0000 7fb6fdc00640 1 --2- 172.21.15.152:0/229624423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb6f815eb80 0x7fb6f8158f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:32.249 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.248+0000 7fb6fdc00640 1 -- 172.21.15.152:0/229624423 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6f8151970 con 0x7fb6f80ba520 2024-04-20T11:34:32.249 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.249+0000 7fb6fdc00640 1 --2- 172.21.15.152:0/229624423 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb6f80ba520 0x7fb6f8160a90 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec034040 tx=0x7fb6ec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:32.249 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.249+0000 7fb6f5600640 1 -- 172.21.15.152:0/229624423 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6ec032820 con 0x7fb6f80ba520 2024-04-20T11:34:32.249 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.249+0000 7fb6f5600640 1 -- 172.21.15.152:0/229624423 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6ec0329e0 con 0x7fb6f80ba520 2024-04-20T11:34:32.250 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.249+0000 7fb6f5600640 1 -- 172.21.15.152:0/229624423 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6ec032dd0 con 0x7fb6f80ba520 2024-04-20T11:34:32.250 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.249+0000 7fb700600640 1 -- 172.21.15.152:0/229624423 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6f8151c50 con 0x7fb6f80ba520 2024-04-20T11:34:32.250 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.249+0000 7fb700600640 1 -- 172.21.15.152:0/229624423 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6f8152210 con 0x7fb6f80ba520 2024-04-20T11:34:32.250 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.250+0000 7fb700600640 1 -- 172.21.15.152:0/229624423 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6f80d1790 con 0x7fb6f80ba520 2024-04-20T11:34:32.250 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.251+0000 7fb6f5600640 1 -- 172.21.15.152:0/229624423 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 19) ==== 91397+0+0 (secure 0 0 0) 0x7fb6ec04e440 con 0x7fb6f80ba520 2024-04-20T11:34:32.252 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.253+0000 7fb6f5600640 1 --2- 172.21.15.152:0/229624423 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb6dc06cce0 0x7fb6dc06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:32.252 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.253+0000 7fb6f5600640 1 -- 172.21.15.152:0/229624423 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 4857+0+0 (secure 0 0 0) 0x7fb6ec0ade10 con 0x7fb6f80ba520 2024-04-20T11:34:32.252 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.253+0000 7fb6fd200640 1 --2- 172.21.15.152:0/229624423 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb6dc06cce0 0x7fb6dc06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:32.254 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.254+0000 7fb6fd200640 1 --2- 172.21.15.152:0/229624423 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb6dc06cce0 0x7fb6dc06f1a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb6e80070b0 tx=0x7fb6e8007040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:32.257 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.257+0000 7fb6f5600640 1 -- 172.21.15.152:0/229624423 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fb6ec040070 con 0x7fb6f80ba520 2024-04-20T11:34:32.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.455+0000 7fb700600640 1 -- 172.21.15.152:0/229624423 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb6f80689f0 con 0x7fb6f80ba520 2024-04-20T11:34:32.457 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.457+0000 7fb6f5600640 1 -- 172.21.15.152:0/229624423 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v114) ==== 74+0+12961 (secure 0 0 0) 0x7fb6ec02f050 con 0x7fb6f80ba520 2024-04-20T11:34:32.457 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:34:32.457 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":114,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:34:31.560252+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:34:32.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.459+0000 7fb700600640 1 -- 172.21.15.152:0/229624423 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb6dc06cce0 msgr2=0x7fb6dc06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:32.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.459+0000 7fb700600640 1 --2- 172.21.15.152:0/229624423 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb6dc06cce0 0x7fb6dc06f1a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb6e80070b0 tx=0x7fb6e8007040 comp rx=0 tx=0).stop 2024-04-20T11:34:32.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.459+0000 7fb700600640 1 -- 172.21.15.152:0/229624423 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb6f80ba520 msgr2=0x7fb6f8160a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:32.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.459+0000 7fb700600640 1 --2- 172.21.15.152:0/229624423 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb6f80ba520 0x7fb6f8160a90 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec034040 tx=0x7fb6ec0046c0 comp rx=0 tx=0).stop 2024-04-20T11:34:32.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.460+0000 7fb700600640 1 -- 172.21.15.152:0/229624423 shutdown_connections 2024-04-20T11:34:32.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.460+0000 7fb700600640 1 --2- 172.21.15.152:0/229624423 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb6dc06cce0 0x7fb6dc06f1a0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:32.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.460+0000 7fb700600640 1 --2- 172.21.15.152:0/229624423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb6f815eb80 0x7fb6f8158f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:32.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.460+0000 7fb700600640 1 --2- 172.21.15.152:0/229624423 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb6f80d2e30 0x7fb6f8154c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:32.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.460+0000 7fb700600640 1 --2- 172.21.15.152:0/229624423 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb6f80ba520 0x7fb6f8160a90 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:32.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.460+0000 7fb700600640 1 -- 172.21.15.152:0/229624423 >> 172.21.15.152:0/229624423 conn(0x7fb6f8067ae0 msgr2=0x7fb6f806d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:32.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.460+0000 7fb700600640 1 -- 172.21.15.152:0/229624423 shutdown_connections 2024-04-20T11:34:32.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.460+0000 7fb700600640 1 -- 172.21.15.152:0/229624423 wait complete. 2024-04-20T11:34:32.478 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T11:26:47.246571+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T11:34:32.479 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T11:34:32.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.681+0000 7f0800e00640 1 Processor -- start 2024-04-20T11:34:32.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.681+0000 7f0800e00640 1 -- start start 2024-04-20T11:34:32.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.683+0000 7f0800e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f07fc15ca40 0x7f07fc15cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:32.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.683+0000 7f0800e00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07fc15d5c0 con 0x7f07fc06bda0 2024-04-20T11:34:32.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.683+0000 7f0800e00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07fc15d8a0 con 0x7f07fc156c00 2024-04-20T11:34:32.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.683+0000 7f0800e00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f07fc160290 con 0x7f07fc15ca40 2024-04-20T11:34:32.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.683+0000 7f07fbe00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f07fc06bda0 0x7f07fc0b30c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:42786/0 (socket says 172.21.15.152:42786) 2024-04-20T11:34:32.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.683+0000 7f07fbe00640 1 -- 172.21.15.152:0/3757398799 learned_addr learned my addr 172.21.15.152:0/3757398799 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:34:32.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.683+0000 7f07faa00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f07fc15ca40 0x7f07fc15cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:32.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.683+0000 7f07fa000640 1 -- 172.21.15.152:0/3757398799 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 890357119 0 0) 0x7f07fc15d5c0 con 0x7f07fc06bda0 2024-04-20T11:34:32.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.683+0000 7f07faa00640 1 -- 172.21.15.152:0/3757398799 >> v1:172.21.15.175:6789/0 conn(0x7f07fc156c00 legacy=0x7f07fc160a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:32.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.683+0000 7f07faa00640 1 -- 172.21.15.152:0/3757398799 >> v1:172.21.15.152:6789/0 conn(0x7f07fc06bda0 legacy=0x7f07fc0b30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:32.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.683+0000 7f07faa00640 1 -- 172.21.15.152:0/3757398799 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07fc0ad2e0 con 0x7f07fc15ca40 2024-04-20T11:34:32.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.685+0000 7f07faa00640 1 --2- 172.21.15.152:0/3757398799 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f07fc15ca40 0x7f07fc15cec0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f07e80097a0 tx=0x7f07e802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62a42e5c07302805 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:32.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.685+0000 7f07fa000640 1 -- 172.21.15.152:0/3757398799 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07e8035020 con 0x7f07fc15ca40 2024-04-20T11:34:32.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.685+0000 7f07fa000640 1 -- 172.21.15.152:0/3757398799 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07e8031830 con 0x7f07fc15ca40 2024-04-20T11:34:32.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.685+0000 7f07fa000640 1 -- 172.21.15.152:0/3757398799 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07e8031c00 con 0x7f07fc15ca40 2024-04-20T11:34:32.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.685+0000 7f0800e00640 1 -- 172.21.15.152:0/3757398799 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f07fc15ca40 msgr2=0x7f07fc15cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:32.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.685+0000 7f0800e00640 1 --2- 172.21.15.152:0/3757398799 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f07fc15ca40 0x7f07fc15cec0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f07e80097a0 tx=0x7f07e802b8d0 comp rx=0 tx=0).stop 2024-04-20T11:34:32.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.685+0000 7f0800e00640 1 -- 172.21.15.152:0/3757398799 shutdown_connections 2024-04-20T11:34:32.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.685+0000 7f0800e00640 1 --2- 172.21.15.152:0/3757398799 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f07fc15ca40 0x7f07fc15cec0 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:32.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.685+0000 7f0800e00640 1 -- 172.21.15.152:0/3757398799 >> 172.21.15.152:0/3757398799 conn(0x7f07fc0b21e0 msgr2=0x7f07fc0b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:32.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.686+0000 7f0800e00640 1 -- 172.21.15.152:0/3757398799 shutdown_connections 2024-04-20T11:34:32.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.686+0000 7f0800e00640 1 -- 172.21.15.152:0/3757398799 wait complete. 2024-04-20T11:34:32.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.687+0000 7f0800e00640 1 Processor -- start 2024-04-20T11:34:32.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.687+0000 7f0800e00640 1 -- start start 2024-04-20T11:34:32.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.688+0000 7f0800e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f07fc06bda0 0x7f07fc15f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:32.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.688+0000 7f0800e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f07fc156c00 0x7f07fc154170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:32.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.688+0000 7f0800e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f07fc15ca40 0x7f07fc14e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:32.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.688+0000 7f0800e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f07fc0b3b60 con 0x7f07fc06bda0 2024-04-20T11:34:32.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.688+0000 7f0800e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f07fc0b39e0 con 0x7f07fc156c00 2024-04-20T11:34:32.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.688+0000 7f0800e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f07fc0b3ce0 con 0x7f07fc15ca40 2024-04-20T11:34:32.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.688+0000 7f07fb400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f07fc06bda0 0x7f07fc15f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:32.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.689+0000 7f07fb400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f07fc06bda0 0x7f07fc15f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:33592/0 (socket says 172.21.15.152:33592) 2024-04-20T11:34:32.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.689+0000 7f07fb400640 1 -- 172.21.15.152:0/1582469106 learned_addr learned my addr 172.21.15.152:0/1582469106 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:32.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.689+0000 7f07fbe00640 1 --2- 172.21.15.152:0/1582469106 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f07fc15ca40 0x7f07fc14e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:32.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.689+0000 7f07faa00640 1 --2- 172.21.15.152:0/1582469106 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f07fc156c00 0x7f07fc154170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:32.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.689+0000 7f07fb400640 1 -- 172.21.15.152:0/1582469106 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f07fc15ca40 msgr2=0x7f07fc14e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:32.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.689+0000 7f07fb400640 1 --2- 172.21.15.152:0/1582469106 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f07fc15ca40 0x7f07fc14e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:32.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.689+0000 7f07fb400640 1 -- 172.21.15.152:0/1582469106 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f07fc156c00 msgr2=0x7f07fc154170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:32.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.689+0000 7f07fb400640 1 --2- 172.21.15.152:0/1582469106 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f07fc156c00 0x7f07fc154170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:32.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.689+0000 7f07fb400640 1 -- 172.21.15.152:0/1582469106 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07fc1f44e0 con 0x7f07fc06bda0 2024-04-20T11:34:32.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.689+0000 7f07fb400640 1 --2- 172.21.15.152:0/1582469106 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f07fc06bda0 0x7f07fc15f260 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f07ec00c460 tx=0x7f07ec00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:32.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.690+0000 7f07f3400640 1 -- 172.21.15.152:0/1582469106 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07ec011790 con 0x7f07fc06bda0 2024-04-20T11:34:32.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.690+0000 7f07f3400640 1 -- 172.21.15.152:0/1582469106 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07ec017070 con 0x7f07fc06bda0 2024-04-20T11:34:32.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.690+0000 7f0800e00640 1 -- 172.21.15.152:0/1582469106 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07fc1f47c0 con 0x7f07fc06bda0 2024-04-20T11:34:32.691 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.690+0000 7f07f3400640 1 -- 172.21.15.152:0/1582469106 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07ec011bb0 con 0x7f07fc06bda0 2024-04-20T11:34:32.691 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.690+0000 7f0800e00640 1 -- 172.21.15.152:0/1582469106 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07fc1f4d80 con 0x7f07fc06bda0 2024-04-20T11:34:32.691 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.691+0000 7f0800e00640 1 -- 172.21.15.152:0/1582469106 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07fc1f5360 con 0x7f07fc06bda0 2024-04-20T11:34:32.691 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.692+0000 7f07f3400640 1 -- 172.21.15.152:0/1582469106 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 19) ==== 91397+0+0 (secure 0 0 0) 0x7f07ec01c460 con 0x7f07fc06bda0 2024-04-20T11:34:32.692 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.693+0000 7f07f3400640 1 --2- 172.21.15.152:0/1582469106 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f07dc06cce0 0x7f07dc06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:32.692 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.693+0000 7f07f3400640 1 -- 172.21.15.152:0/1582469106 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 4857+0+0 (secure 0 0 0) 0x7f07ec08bdc0 con 0x7f07fc06bda0 2024-04-20T11:34:32.693 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.694+0000 7f07faa00640 1 --2- 172.21.15.152:0/1582469106 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f07dc06cce0 0x7f07dc06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:32.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.695+0000 7f07faa00640 1 --2- 172.21.15.152:0/1582469106 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f07dc06cce0 0x7f07dc06f1a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f07e8036000 tx=0x7f07e8008030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:32.697 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.698+0000 7f07f3400640 1 -- 172.21.15.152:0/1582469106 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f07fc1f5360 con 0x7f07fc06bda0 2024-04-20T11:34:32.879 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.880+0000 7f0800e00640 1 -- 172.21.15.152:0/1582469106 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f07fc075e60 con 0x7f07fc06bda0 2024-04-20T11:34:32.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.881+0000 7f07f3400640 1 -- 172.21.15.152:0/1582469106 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v114) ==== 93+0+10 (secure 0 0 0) 0x7f07ec059f20 con 0x7f07fc06bda0 2024-04-20T11:34:32.880 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 1 2024-04-20T11:34:32.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.883+0000 7f0800e00640 1 -- 172.21.15.152:0/1582469106 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f07dc06cce0 msgr2=0x7f07dc06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:32.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.883+0000 7f0800e00640 1 --2- 172.21.15.152:0/1582469106 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f07dc06cce0 0x7f07dc06f1a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f07e8036000 tx=0x7f07e8008030 comp rx=0 tx=0).stop 2024-04-20T11:34:32.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.883+0000 7f0800e00640 1 -- 172.21.15.152:0/1582469106 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f07fc06bda0 msgr2=0x7f07fc15f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:32.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.883+0000 7f0800e00640 1 --2- 172.21.15.152:0/1582469106 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f07fc06bda0 0x7f07fc15f260 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f07ec00c460 tx=0x7f07ec00c920 comp rx=0 tx=0).stop 2024-04-20T11:34:32.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.883+0000 7f0800e00640 1 -- 172.21.15.152:0/1582469106 shutdown_connections 2024-04-20T11:34:32.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.883+0000 7f0800e00640 1 --2- 172.21.15.152:0/1582469106 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f07dc06cce0 0x7f07dc06f1a0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:32.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.883+0000 7f0800e00640 1 --2- 172.21.15.152:0/1582469106 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f07fc15ca40 0x7f07fc14e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:32.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.883+0000 7f0800e00640 1 --2- 172.21.15.152:0/1582469106 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f07fc156c00 0x7f07fc154170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:32.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.883+0000 7f0800e00640 1 --2- 172.21.15.152:0/1582469106 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f07fc06bda0 0x7f07fc15f260 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:32.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.883+0000 7f0800e00640 1 -- 172.21.15.152:0/1582469106 >> 172.21.15.152:0/1582469106 conn(0x7f07fc0b21e0 msgr2=0x7f07fc072340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:32.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.883+0000 7f0800e00640 1 -- 172.21.15.152:0/1582469106 shutdown_connections 2024-04-20T11:34:32.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:32.883+0000 7f0800e00640 1 -- 172.21.15.152:0/1582469106 wait complete. 2024-04-20T11:34:32.901 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-20T11:34:33.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.094+0000 7fcbc9600640 1 Processor -- start 2024-04-20T11:34:33.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.094+0000 7fcbc9600640 1 -- start start 2024-04-20T11:34:33.098 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.096+0000 7fcbc9600640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fcbc4145f60 0x7fcbc406ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:33.098 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.096+0000 7fcbc9600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcbc40cd0d0 0x7fcbc40cd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:33.098 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.096+0000 7fcbc9600640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcbc4146340 con 0x7fcbc4071fe0 2024-04-20T11:34:33.098 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.096+0000 7fcbc9600640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fcbc406e9f0 con 0x7fcbc4145f60 2024-04-20T11:34:33.098 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.096+0000 7fcbc9600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fcbc406f3c0 con 0x7fcbc40cd0d0 2024-04-20T11:34:33.098 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.096+0000 7fcbc3400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fcbc4071fe0 0x7fcbc4072470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:42798/0 (socket says 172.21.15.152:42798) 2024-04-20T11:34:33.098 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.096+0000 7fcbc3400640 1 -- 172.21.15.152:0/2757138425 learned_addr learned my addr 172.21.15.152:0/2757138425 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:34:33.098 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.096+0000 7fcbc2a00640 1 --2- 172.21.15.152:0/2757138425 >> v2:172.21.15.175:3300/0 conn(0x7fcbc4145f60 0x7fcbc406ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:33.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.096+0000 7fcbc2000640 1 --2- 172.21.15.152:0/2757138425 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcbc40cd0d0 0x7fcbc40cd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:33.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.097+0000 7fcbc1600640 1 -- 172.21.15.152:0/2757138425 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1795680722 0 0) 0x7fcbc4146340 con 0x7fcbc4071fe0 2024-04-20T11:34:33.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.097+0000 7fcbc1600640 1 -- 172.21.15.152:0/2757138425 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcba8003520 con 0x7fcbc4071fe0 2024-04-20T11:34:33.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.097+0000 7fcbc1600640 1 -- 172.21.15.152:0/2757138425 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 729506284 0 0) 0x7fcba8003520 con 0x7fcbc4071fe0 2024-04-20T11:34:33.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.097+0000 7fcbc1600640 1 -- 172.21.15.152:0/2757138425 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcbc40cd0d0 msgr2=0x7fcbc40cd550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:33.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.097+0000 7fcbc1600640 1 --2- 172.21.15.152:0/2757138425 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcbc40cd0d0 0x7fcbc40cd550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:33.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.097+0000 7fcbc1600640 1 -- 172.21.15.152:0/2757138425 >> v2:172.21.15.175:3300/0 conn(0x7fcbc4145f60 msgr2=0x7fcbc406ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:33.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.097+0000 7fcbc1600640 1 --2- 172.21.15.152:0/2757138425 >> v2:172.21.15.175:3300/0 conn(0x7fcbc4145f60 0x7fcbc406ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:33.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.097+0000 7fcbc1600640 1 -- 172.21.15.152:0/2757138425 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbc4072d50 con 0x7fcbc4071fe0 2024-04-20T11:34:33.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.097+0000 7fcbc1600640 1 -- 172.21.15.152:0/2757138425 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7fcbbc0019f0 con 0x7fcbc4071fe0 2024-04-20T11:34:33.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.097+0000 7fcbc1600640 1 -- 172.21.15.152:0/2757138425 >> v1:172.21.15.152:6789/0 conn(0x7fcbc4071fe0 legacy=0x7fcbc4072470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:33.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.097+0000 7fcbc1600640 1 --2- 172.21.15.152:0/2757138425 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcba8003a20 0x7fcba8003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:33.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.097+0000 7fcbc1600640 1 --2- 172.21.15.152:0/2757138425 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcbc40cd0d0 0x7fcba8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:33.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.097+0000 7fcbc1600640 1 --2- 172.21.15.152:0/2757138425 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcba8004d40 0x7fcba80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:33.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.097+0000 7fcbc1600640 1 -- 172.21.15.152:0/2757138425 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc406e9f0 con 0x7fcba8004d40 2024-04-20T11:34:33.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.097+0000 7fcbc1600640 1 -- 172.21.15.152:0/2757138425 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc406f3c0 con 0x7fcba8003a20 2024-04-20T11:34:33.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.097+0000 7fcbc1600640 1 -- 172.21.15.152:0/2757138425 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fcba800ba40 con 0x7fcbc40cd0d0 2024-04-20T11:34:33.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.098+0000 7fcbc3400640 1 --2- 172.21.15.152:0/2757138425 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcba8004d40 0x7fcba80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:33.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.098+0000 7fcbc2a00640 1 --2- 172.21.15.152:0/2757138425 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcba8003a20 0x7fcba8003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:33.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.098+0000 7fcbc2000640 1 --2- 172.21.15.152:0/2757138425 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcbc40cd0d0 0x7fcba8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:33.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.098+0000 7fcbc3400640 1 -- 172.21.15.152:0/2757138425 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcbc40cd0d0 msgr2=0x7fcba8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:33.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.098+0000 7fcbc3400640 1 --2- 172.21.15.152:0/2757138425 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcbc40cd0d0 0x7fcba8004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:33.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.098+0000 7fcbc3400640 1 -- 172.21.15.152:0/2757138425 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcba8003a20 msgr2=0x7fcba8003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:33.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.098+0000 7fcbc3400640 1 --2- 172.21.15.152:0/2757138425 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcba8003a20 0x7fcba8003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:33.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.098+0000 7fcbc3400640 1 -- 172.21.15.152:0/2757138425 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcba8003520 con 0x7fcba8004d40 2024-04-20T11:34:33.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.099+0000 7fcbc3400640 1 --2- 172.21.15.152:0/2757138425 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcba8004d40 0x7fcba80051c0 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7fcbbc00a880 tx=0x7fcba8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=884f3de873983612 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:33.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.099+0000 7fcbc1600640 1 -- 172.21.15.152:0/2757138425 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbbc00a4a0 con 0x7fcba8004d40 2024-04-20T11:34:33.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.099+0000 7fcbc1600640 1 -- 172.21.15.152:0/2757138425 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbbc034650 con 0x7fcba8004d40 2024-04-20T11:34:33.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.099+0000 7fcbc1600640 1 -- 172.21.15.152:0/2757138425 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbbc034a40 con 0x7fcba8004d40 2024-04-20T11:34:33.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.100+0000 7fcbc9600640 1 -- 172.21.15.152:0/2757138425 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcba8004d40 msgr2=0x7fcba80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:33.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.100+0000 7fcbc9600640 1 --2- 172.21.15.152:0/2757138425 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcba8004d40 0x7fcba80051c0 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7fcbbc00a880 tx=0x7fcba8005830 comp rx=0 tx=0).stop 2024-04-20T11:34:33.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.100+0000 7fcbc3400640 1 -- 172.21.15.152:0/2757138425 reap_dead start 2024-04-20T11:34:33.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.100+0000 7fcbc9600640 1 -- 172.21.15.152:0/2757138425 shutdown_connections 2024-04-20T11:34:33.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.100+0000 7fcbc9600640 1 -- 172.21.15.152:0/2757138425 >> 172.21.15.152:0/2757138425 conn(0x7fcbc4067cf0 msgr2=0x7fcbc406b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:33.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.100+0000 7fcbc9600640 1 -- 172.21.15.152:0/2757138425 shutdown_connections 2024-04-20T11:34:33.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.100+0000 7fcbc9600640 1 -- 172.21.15.152:0/2757138425 wait complete. 2024-04-20T11:34:33.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.102+0000 7fcbc9600640 1 Processor -- start 2024-04-20T11:34:33.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.102+0000 7fcbc9600640 1 -- start start 2024-04-20T11:34:33.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.103+0000 7fcbc9600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcba8004d40 0x7fcbc413bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:33.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.103+0000 7fcbc9600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcbc4144000 0x7fcbc415f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:33.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.103+0000 7fcbc9600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcbc415ddc0 0x7fcbc40ae900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:33.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.103+0000 7fcbc9600640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc406e680 con 0x7fcba8004d40 2024-04-20T11:34:33.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.103+0000 7fcbc9600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc406e3a0 con 0x7fcbc4144000 2024-04-20T11:34:33.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.103+0000 7fcbc9600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fcbc40cd550 con 0x7fcbc415ddc0 2024-04-20T11:34:33.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.103+0000 7fcbc2a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcba8004d40 0x7fcbc413bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:33.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.103+0000 7fcbc2a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcba8004d40 0x7fcbc413bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:33614/0 (socket says 172.21.15.152:33614) 2024-04-20T11:34:33.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.103+0000 7fcbc2a00640 1 -- 172.21.15.152:0/2963434974 learned_addr learned my addr 172.21.15.152:0/2963434974 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:33.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.103+0000 7fcbc2a00640 1 -- 172.21.15.152:0/2963434974 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcbc415ddc0 msgr2=0x7fcbc40ae900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:33.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.103+0000 7fcbc2a00640 1 --2- 172.21.15.152:0/2963434974 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcbc415ddc0 0x7fcbc40ae900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:33.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.103+0000 7fcbc2a00640 1 -- 172.21.15.152:0/2963434974 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcbc4144000 msgr2=0x7fcbc415f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:33.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.103+0000 7fcbc2a00640 1 --2- 172.21.15.152:0/2963434974 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcbc4144000 0x7fcbc415f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:33.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.103+0000 7fcbc2a00640 1 -- 172.21.15.152:0/2963434974 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbc415e730 con 0x7fcba8004d40 2024-04-20T11:34:33.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.104+0000 7fcbc2a00640 1 --2- 172.21.15.152:0/2963434974 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcba8004d40 0x7fcbc413bda0 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7fcbb40116d0 tx=0x7fcbb4011b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:33.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.104+0000 7fcbbaa00640 1 -- 172.21.15.152:0/2963434974 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb400f960 con 0x7fcba8004d40 2024-04-20T11:34:33.107 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.104+0000 7fcbbaa00640 1 -- 172.21.15.152:0/2963434974 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbb400fb00 con 0x7fcba8004d40 2024-04-20T11:34:33.107 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.104+0000 7fcbc9600640 1 -- 172.21.15.152:0/2963434974 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbc415ea10 con 0x7fcba8004d40 2024-04-20T11:34:33.107 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.104+0000 7fcbbaa00640 1 -- 172.21.15.152:0/2963434974 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb4015460 con 0x7fcba8004d40 2024-04-20T11:34:33.107 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.104+0000 7fcbc9600640 1 -- 172.21.15.152:0/2963434974 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbc415efd0 con 0x7fcba8004d40 2024-04-20T11:34:33.107 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.105+0000 7fcbc9600640 1 -- 172.21.15.152:0/2963434974 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbc40689f0 con 0x7fcba8004d40 2024-04-20T11:34:33.108 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.106+0000 7fcbbaa00640 1 -- 172.21.15.152:0/2963434974 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 19) ==== 91397+0+0 (secure 0 0 0) 0x7fcbb4015600 con 0x7fcba8004d40 2024-04-20T11:34:33.108 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.107+0000 7fcbbaa00640 1 --2- 172.21.15.152:0/2963434974 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fcba406cd10 0x7fcba406f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:33.108 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.107+0000 7fcbbaa00640 1 -- 172.21.15.152:0/2963434974 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 4857+0+0 (secure 0 0 0) 0x7fcbb408c880 con 0x7fcba8004d40 2024-04-20T11:34:33.108 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.108+0000 7fcbc2000640 1 --2- 172.21.15.152:0/2963434974 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fcba406cd10 0x7fcba406f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:33.108 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.109+0000 7fcbc2000640 1 --2- 172.21.15.152:0/2963434974 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fcba406cd10 0x7fcba406f1d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fcbb0006a90 tx=0x7fcbb0006a20 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:33.111 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.111+0000 7fcbbaa00640 1 -- 172.21.15.152:0/2963434974 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fcbb4019030 con 0x7fcba8004d40 2024-04-20T11:34:33.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.312+0000 7fcbc9600640 1 -- 172.21.15.152:0/2963434974 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fcbc40afc70 con 0x7fcba8004d40 2024-04-20T11:34:33.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.313+0000 7fcbbaa00640 1 -- 172.21.15.152:0/2963434974 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v114) ==== 84+0+65 (secure 0 0 0) 0x7fcbb405aa20 con 0x7fcba8004d40 2024-04-20T11:34:33.313 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:34:33.313 INFO:teuthology.orchestra.run.smithi152.stdout:[ 2024-04-20T11:34:33.313 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:34:33.313 INFO:teuthology.orchestra.run.smithi152.stdout: "poolnum": 1, 2024-04-20T11:34:33.313 INFO:teuthology.orchestra.run.smithi152.stdout: "poolname": ".mgr" 2024-04-20T11:34:33.313 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:34:33.313 INFO:teuthology.orchestra.run.smithi152.stdout:] 2024-04-20T11:34:33.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.315+0000 7fcbc9600640 1 -- 172.21.15.152:0/2963434974 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fcba406cd10 msgr2=0x7fcba406f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:33.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.315+0000 7fcbc9600640 1 --2- 172.21.15.152:0/2963434974 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fcba406cd10 0x7fcba406f1d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fcbb0006a90 tx=0x7fcbb0006a20 comp rx=0 tx=0).stop 2024-04-20T11:34:33.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.315+0000 7fcbc9600640 1 -- 172.21.15.152:0/2963434974 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcba8004d40 msgr2=0x7fcbc413bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:33.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.315+0000 7fcbc9600640 1 --2- 172.21.15.152:0/2963434974 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcba8004d40 0x7fcbc413bda0 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7fcbb40116d0 tx=0x7fcbb4011b90 comp rx=0 tx=0).stop 2024-04-20T11:34:33.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.316+0000 7fcbc9600640 1 -- 172.21.15.152:0/2963434974 shutdown_connections 2024-04-20T11:34:33.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.316+0000 7fcbc9600640 1 --2- 172.21.15.152:0/2963434974 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fcba406cd10 0x7fcba406f1d0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:33.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.316+0000 7fcbc9600640 1 --2- 172.21.15.152:0/2963434974 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcbc415ddc0 0x7fcbc40ae900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:33.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.316+0000 7fcbc9600640 1 --2- 172.21.15.152:0/2963434974 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcbc4144000 0x7fcbc415f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:33.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.316+0000 7fcbc9600640 1 --2- 172.21.15.152:0/2963434974 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcba8004d40 0x7fcbc413bda0 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:33.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.316+0000 7fcbc9600640 1 -- 172.21.15.152:0/2963434974 >> 172.21.15.152:0/2963434974 conn(0x7fcbc4067cf0 msgr2=0x7fcbc406b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:33.317 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.316+0000 7fcbc9600640 1 -- 172.21.15.152:0/2963434974 shutdown_connections 2024-04-20T11:34:33.317 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.316+0000 7fcbc9600640 1 -- 172.21.15.152:0/2963434974 wait complete. 2024-04-20T11:34:33.335 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-20T11:34:33.335 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-20T11:34:33.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.546+0000 7f2f8ca00640 1 Processor -- start 2024-04-20T11:34:33.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.546+0000 7f2f8ca00640 1 -- start start 2024-04-20T11:34:33.548 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.547+0000 7f2f8ca00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f2f8815e870 0x7f2f880d3370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:33.548 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.547+0000 7f2f8ca00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f2f880d0fa0 0x7f2f880d1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:33.548 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.547+0000 7f2f8ca00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f8815ec50 con 0x7f2f880cff90 2024-04-20T11:34:33.548 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.547+0000 7f2f8ca00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f2f880d2ee0 con 0x7f2f880d0fa0 2024-04-20T11:34:33.548 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.547+0000 7f2f8ca00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f2f880d38b0 con 0x7f2f8815e870 2024-04-20T11:34:33.548 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.548+0000 7f2f85e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f2f880d0fa0 0x7f2f880d1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:33.548 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.548+0000 7f2f85e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f2f880d0fa0 0x7f2f880d1450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:33628/0 (socket says 172.21.15.152:33628) 2024-04-20T11:34:33.548 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.548+0000 7f2f85e00640 1 -- 172.21.15.152:0/2841345610 learned_addr learned my addr 172.21.15.152:0/2841345610 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:33.548 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.548+0000 7f2f85e00640 1 -- 172.21.15.152:0/2841345610 >> v2:172.21.15.175:3300/0 conn(0x7f2f8815e870 msgr2=0x7f2f880d3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:33.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.548+0000 7f2f85e00640 1 --2- 172.21.15.152:0/2841345610 >> v2:172.21.15.175:3300/0 conn(0x7f2f8815e870 0x7f2f880d3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:33.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.548+0000 7f2f85e00640 1 -- 172.21.15.152:0/2841345610 >> v1:172.21.15.175:6789/0 conn(0x7f2f880cff90 legacy=0x7f2f880d0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:33.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.548+0000 7f2f85e00640 1 -- 172.21.15.152:0/2841345610 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f880d22d0 con 0x7f2f880d0fa0 2024-04-20T11:34:33.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.549+0000 7f2f85e00640 1 --2- 172.21.15.152:0/2841345610 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2f880d0fa0 0x7f2f880d1450 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f2f8808e460 tx=0x7f2f7402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee05bfd1bed56d58 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:33.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.550+0000 7f2f85400640 1 -- 172.21.15.152:0/2841345610 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f74037070 con 0x7f2f880d0fa0 2024-04-20T11:34:33.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.550+0000 7f2f85400640 1 -- 172.21.15.152:0/2841345610 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f74004230 con 0x7f2f880d0fa0 2024-04-20T11:34:33.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.550+0000 7f2f85400640 1 -- 172.21.15.152:0/2841345610 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f74003610 con 0x7f2f880d0fa0 2024-04-20T11:34:33.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.550+0000 7f2f8ca00640 1 -- 172.21.15.152:0/2841345610 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2f880d0fa0 msgr2=0x7f2f880d1450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:33.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.550+0000 7f2f8ca00640 1 --2- 172.21.15.152:0/2841345610 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2f880d0fa0 0x7f2f880d1450 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f2f8808e460 tx=0x7f2f7402b860 comp rx=0 tx=0).stop 2024-04-20T11:34:33.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.551+0000 7f2f8ca00640 1 -- 172.21.15.152:0/2841345610 shutdown_connections 2024-04-20T11:34:33.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.551+0000 7f2f8ca00640 1 --2- 172.21.15.152:0/2841345610 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2f880d0fa0 0x7f2f880d1450 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:33.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.551+0000 7f2f8ca00640 1 --2- 172.21.15.152:0/2841345610 >> v2:172.21.15.175:3300/0 conn(0x7f2f8815e870 0x7f2f880d3370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:33.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.551+0000 7f2f8ca00640 1 -- 172.21.15.152:0/2841345610 >> 172.21.15.152:0/2841345610 conn(0x7f2f8808bfd0 msgr2=0x7f2f880d45d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:33.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.551+0000 7f2f8ca00640 1 -- 172.21.15.152:0/2841345610 shutdown_connections 2024-04-20T11:34:33.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.551+0000 7f2f8ca00640 1 -- 172.21.15.152:0/2841345610 wait complete. 2024-04-20T11:34:33.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.552+0000 7f2f8ca00640 1 Processor -- start 2024-04-20T11:34:33.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.552+0000 7f2f8ca00640 1 -- start start 2024-04-20T11:34:33.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.553+0000 7f2f8ca00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2f880cff90 0x7f2f880ca9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:33.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.553+0000 7f2f8ca00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2f880d0fa0 0x7f2f880c8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:33.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.553+0000 7f2f8ca00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2f8815e870 0x7f2f8815ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:33.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.553+0000 7f2f8ca00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f2f880d2b70 con 0x7f2f8815e870 2024-04-20T11:34:33.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.553+0000 7f2f8ca00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2f880d2890 con 0x7f2f880d0fa0 2024-04-20T11:34:33.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.553+0000 7f2f8ca00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2f8806ca80 con 0x7f2f880cff90 2024-04-20T11:34:33.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.554+0000 7f2f87200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2f8815e870 0x7f2f8815ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:33.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.554+0000 7f2f87200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2f8815e870 0x7f2f8815ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:33640/0 (socket says 172.21.15.152:33640) 2024-04-20T11:34:33.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.554+0000 7f2f87200640 1 -- 172.21.15.152:0/3381412860 learned_addr learned my addr 172.21.15.152:0/3381412860 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:33.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.554+0000 7f2f85e00640 1 --2- 172.21.15.152:0/3381412860 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2f880d0fa0 0x7f2f880c8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:33.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.554+0000 7f2f86800640 1 --2- 172.21.15.152:0/3381412860 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2f880cff90 0x7f2f880ca9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:33.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.554+0000 7f2f87200640 1 -- 172.21.15.152:0/3381412860 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2f880cff90 msgr2=0x7f2f880ca9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:33.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.554+0000 7f2f87200640 1 --2- 172.21.15.152:0/3381412860 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2f880cff90 0x7f2f880ca9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:33.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.554+0000 7f2f87200640 1 -- 172.21.15.152:0/3381412860 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2f880d0fa0 msgr2=0x7f2f880c8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:33.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.554+0000 7f2f87200640 1 --2- 172.21.15.152:0/3381412860 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2f880d0fa0 0x7f2f880c8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:33.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.554+0000 7f2f87200640 1 -- 172.21.15.152:0/3381412860 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f880c8b50 con 0x7f2f8815e870 2024-04-20T11:34:33.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.554+0000 7f2f85e00640 1 --2- 172.21.15.152:0/3381412860 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2f880d0fa0 0x7f2f880c8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:34:33.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.554+0000 7f2f86800640 1 --2- 172.21.15.152:0/3381412860 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2f880cff90 0x7f2f880ca9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:34:33.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.555+0000 7f2f87200640 1 --2- 172.21.15.152:0/3381412860 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2f8815e870 0x7f2f8815ee40 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f2f8000c780 tx=0x7f2f8000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:33.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.555+0000 7f2f7ea00640 1 -- 172.21.15.152:0/3381412860 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f80017070 con 0x7f2f8815e870 2024-04-20T11:34:33.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.555+0000 7f2f7ea00640 1 -- 172.21.15.152:0/3381412860 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f80003da0 con 0x7f2f8815e870 2024-04-20T11:34:33.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.555+0000 7f2f8ca00640 1 -- 172.21.15.152:0/3381412860 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f880c8e30 con 0x7f2f8815e870 2024-04-20T11:34:33.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.555+0000 7f2f7ea00640 1 -- 172.21.15.152:0/3381412860 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f800052b0 con 0x7f2f8815e870 2024-04-20T11:34:33.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.555+0000 7f2f8ca00640 1 -- 172.21.15.152:0/3381412860 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f880c9110 con 0x7f2f8815e870 2024-04-20T11:34:33.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.557+0000 7f2f8ca00640 1 -- 172.21.15.152:0/3381412860 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f88075e60 con 0x7f2f8815e870 2024-04-20T11:34:33.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.558+0000 7f2f7ea00640 1 -- 172.21.15.152:0/3381412860 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 19) ==== 91397+0+0 (secure 0 0 0) 0x7f2f8001a070 con 0x7f2f8815e870 2024-04-20T11:34:33.558 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.559+0000 7f2f7ea00640 1 --2- 172.21.15.152:0/3381412860 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f2f6806cce0 0x7f2f6806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:33.558 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.559+0000 7f2f7ea00640 1 -- 172.21.15.152:0/3381412860 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 4857+0+0 (secure 0 0 0) 0x7f2f8008b5e0 con 0x7f2f8815e870 2024-04-20T11:34:33.558 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.559+0000 7f2f86800640 1 --2- 172.21.15.152:0/3381412860 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f2f6806cce0 0x7f2f6806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:33.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.561+0000 7f2f86800640 1 --2- 172.21.15.152:0/3381412860 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f2f6806cce0 0x7f2f6806f1a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2f880b55b0 tx=0x7f2f78008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:33.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.563+0000 7f2f7ea00640 1 -- 172.21.15.152:0/3381412860 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f2f80059780 con 0x7f2f8815e870 2024-04-20T11:34:33.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:33.763+0000 7f2f8ca00640 1 -- 172.21.15.152:0/3381412860 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f2f880ba550 con 0x7f2f8815e870 2024-04-20T11:34:34.567 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:34.567+0000 7f2f7ea00640 1 -- 172.21.15.152:0/3381412860 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v115) ==== 138+0+0 (secure 0 0 0) 0x7f2f80059080 con 0x7f2f8815e870 2024-04-20T11:34:34.567 INFO:teuthology.orchestra.run.smithi152.stderr:pool 'cephfs_metadata' created 2024-04-20T11:34:34.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:34.571+0000 7f2f8ca00640 1 -- 172.21.15.152:0/3381412860 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f2f6806cce0 msgr2=0x7f2f6806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:34.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:34.571+0000 7f2f8ca00640 1 --2- 172.21.15.152:0/3381412860 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f2f6806cce0 0x7f2f6806f1a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2f880b55b0 tx=0x7f2f78008040 comp rx=0 tx=0).stop 2024-04-20T11:34:34.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:34.571+0000 7f2f8ca00640 1 -- 172.21.15.152:0/3381412860 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2f8815e870 msgr2=0x7f2f8815ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:34.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:34.571+0000 7f2f8ca00640 1 --2- 172.21.15.152:0/3381412860 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2f8815e870 0x7f2f8815ee40 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f2f8000c780 tx=0x7f2f8000cc40 comp rx=0 tx=0).stop 2024-04-20T11:34:34.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:34.572+0000 7f2f8ca00640 1 -- 172.21.15.152:0/3381412860 shutdown_connections 2024-04-20T11:34:34.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:34.572+0000 7f2f8ca00640 1 --2- 172.21.15.152:0/3381412860 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f2f6806cce0 0x7f2f6806f1a0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:34.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:34.572+0000 7f2f8ca00640 1 --2- 172.21.15.152:0/3381412860 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2f8815e870 0x7f2f8815ee40 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:34.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:34.572+0000 7f2f8ca00640 1 --2- 172.21.15.152:0/3381412860 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2f880d0fa0 0x7f2f880c8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:34.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:34.572+0000 7f2f8ca00640 1 --2- 172.21.15.152:0/3381412860 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2f880cff90 0x7f2f880ca9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:34.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:34.572+0000 7f2f8ca00640 1 -- 172.21.15.152:0/3381412860 >> 172.21.15.152:0/3381412860 conn(0x7f2f8808bfd0 msgr2=0x7f2f8815cd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:34.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:34.572+0000 7f2f8ca00640 1 -- 172.21.15.152:0/3381412860 shutdown_connections 2024-04-20T11:34:34.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:34.572+0000 7f2f8ca00640 1 -- 172.21.15.152:0/3381412860 wait complete. 2024-04-20T11:34:34.601 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-20T11:34:34.797 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:34.797+0000 7fc93fe00640 1 Processor -- start 2024-04-20T11:34:34.797 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:34.798+0000 7fc93fe00640 1 -- start start 2024-04-20T11:34:34.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:34.799+0000 7fc93fe00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fc9380cf880 0x7fc9380cfce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:34.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:34.799+0000 7fc93fe00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fc9380b7d50 0x7fc9380b8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:34.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:34.799+0000 7fc93fe00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9380d3000 con 0x7fc93815ed40 2024-04-20T11:34:34.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:34.799+0000 7fc93fe00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fc9380d0220 con 0x7fc9380b7d50 2024-04-20T11:34:34.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:34.799+0000 7fc93fe00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fc9380d2db0 con 0x7fc9380cf880 2024-04-20T11:34:34.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:34.800+0000 7fc93de00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fc9380b7d50 0x7fc9380b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:34.800 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:34.800+0000 7fc93de00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fc9380b7d50 0x7fc9380b8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:33656/0 (socket says 172.21.15.152:33656) 2024-04-20T11:34:34.800 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:34.800+0000 7fc93de00640 1 -- 172.21.15.152:0/2144722517 learned_addr learned my addr 172.21.15.152:0/2144722517 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:34.800 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:34.800+0000 7fc93ca00640 1 --2- 172.21.15.152:0/2144722517 >> v2:172.21.15.175:3300/0 conn(0x7fc9380cf880 0x7fc9380cfce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:34.800 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:34.800+0000 7fc93de00640 1 -- 172.21.15.152:0/2144722517 >> v2:172.21.15.175:3300/0 conn(0x7fc9380cf880 msgr2=0x7fc9380cfce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:34.801 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:34.800+0000 7fc93de00640 1 --2- 172.21.15.152:0/2144722517 >> v2:172.21.15.175:3300/0 conn(0x7fc9380cf880 0x7fc9380cfce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:34.801 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:34.800+0000 7fc93de00640 1 -- 172.21.15.152:0/2144722517 >> v1:172.21.15.175:6789/0 conn(0x7fc93815ed40 legacy=0x7fc9380d3dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:34.801 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:34.800+0000 7fc93de00640 1 -- 172.21.15.152:0/2144722517 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9380d32e0 con 0x7fc9380b7d50 2024-04-20T11:34:34.801 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:34.800+0000 7fc93ca00640 1 --2- 172.21.15.152:0/2144722517 >> v2:172.21.15.175:3300/0 conn(0x7fc9380cf880 0x7fc9380cfce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:34:34.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:34.801+0000 7fc93de00640 1 --2- 172.21.15.152:0/2144722517 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc9380b7d50 0x7fc9380b8190 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7fc9380d2930 tx=0x7fc93002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cfce84e0bfcd8ff server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:34.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:34.802+0000 7fc937e00640 1 -- 172.21.15.152:0/2144722517 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc93002ca80 con 0x7fc9380b7d50 2024-04-20T11:34:34.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:34.802+0000 7fc937e00640 1 -- 172.21.15.152:0/2144722517 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc93002cc20 con 0x7fc9380b7d50 2024-04-20T11:34:34.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:34.802+0000 7fc937e00640 1 -- 172.21.15.152:0/2144722517 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9300343f0 con 0x7fc9380b7d50 2024-04-20T11:34:34.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:34.802+0000 7fc93fe00640 1 -- 172.21.15.152:0/2144722517 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc9380b7d50 msgr2=0x7fc9380b8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:34.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:34.802+0000 7fc93fe00640 1 --2- 172.21.15.152:0/2144722517 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc9380b7d50 0x7fc9380b8190 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7fc9380d2930 tx=0x7fc93002bee0 comp rx=0 tx=0).stop 2024-04-20T11:34:34.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:34.803+0000 7fc93fe00640 1 -- 172.21.15.152:0/2144722517 shutdown_connections 2024-04-20T11:34:34.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:34.803+0000 7fc93fe00640 1 --2- 172.21.15.152:0/2144722517 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc9380b7d50 0x7fc9380b8190 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:34.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:34.803+0000 7fc93fe00640 1 --2- 172.21.15.152:0/2144722517 >> v2:172.21.15.175:3300/0 conn(0x7fc9380cf880 0x7fc9380cfce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:34.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:34.803+0000 7fc93fe00640 1 -- 172.21.15.152:0/2144722517 >> 172.21.15.152:0/2144722517 conn(0x7fc938067d50 msgr2=0x7fc93806cc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:34.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:34.803+0000 7fc93fe00640 1 -- 172.21.15.152:0/2144722517 shutdown_connections 2024-04-20T11:34:34.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:34.803+0000 7fc93fe00640 1 -- 172.21.15.152:0/2144722517 wait complete. 2024-04-20T11:34:34.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:34.804+0000 7fc93fe00640 1 Processor -- start 2024-04-20T11:34:34.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:34.804+0000 7fc93fe00640 1 -- start start 2024-04-20T11:34:34.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:34.805+0000 7fc93fe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc9380b7d50 0x7fc93814cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:34.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:34.805+0000 7fc93fe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc9380cf880 0x7fc938149850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:34.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:34.805+0000 7fc93fe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc93815ed40 0x7fc9381612d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:34.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:34.805+0000 7fc93fe00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fc9380b86d0 con 0x7fc93815ed40 2024-04-20T11:34:34.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:34.805+0000 7fc93fe00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc9380d3930 con 0x7fc9380cf880 2024-04-20T11:34:34.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:34.805+0000 7fc93fe00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc9380d44d0 con 0x7fc9380b7d50 2024-04-20T11:34:34.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:34.806+0000 7fc93de00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc93815ed40 0x7fc9381612d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:34.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:34.806+0000 7fc93de00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc93815ed40 0x7fc9381612d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:33664/0 (socket says 172.21.15.152:33664) 2024-04-20T11:34:34.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:34.806+0000 7fc93de00640 1 -- 172.21.15.152:0/450781870 learned_addr learned my addr 172.21.15.152:0/450781870 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:34.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:34.806+0000 7fc93ca00640 1 --2- 172.21.15.152:0/450781870 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc9380cf880 0x7fc938149850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:34.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:34.806+0000 7fc93d400640 1 --2- 172.21.15.152:0/450781870 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc9380b7d50 0x7fc93814cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:34.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:34.806+0000 7fc93de00640 1 -- 172.21.15.152:0/450781870 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc9380b7d50 msgr2=0x7fc93814cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:34.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:34.806+0000 7fc93de00640 1 --2- 172.21.15.152:0/450781870 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc9380b7d50 0x7fc93814cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:34.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:34.806+0000 7fc93de00640 1 -- 172.21.15.152:0/450781870 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc9380cf880 msgr2=0x7fc938149850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:34.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:34.806+0000 7fc93de00640 1 --2- 172.21.15.152:0/450781870 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc9380cf880 0x7fc938149850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:34.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:34.806+0000 7fc93de00640 1 -- 172.21.15.152:0/450781870 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc93815e060 con 0x7fc93815ed40 2024-04-20T11:34:34.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:34.806+0000 7fc93d400640 1 --2- 172.21.15.152:0/450781870 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc9380b7d50 0x7fc93814cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:34:34.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:34.807+0000 7fc93de00640 1 --2- 172.21.15.152:0/450781870 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc93815ed40 0x7fc9381612d0 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7fc930036040 tx=0x7fc930004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:34.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:34.807+0000 7fc935600640 1 -- 172.21.15.152:0/450781870 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc930043020 con 0x7fc93815ed40 2024-04-20T11:34:34.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:34.807+0000 7fc935600640 1 -- 172.21.15.152:0/450781870 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc93003e450 con 0x7fc93815ed40 2024-04-20T11:34:34.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:34.807+0000 7fc935600640 1 -- 172.21.15.152:0/450781870 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc93003e860 con 0x7fc93815ed40 2024-04-20T11:34:34.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:34.807+0000 7fc93fe00640 1 -- 172.21.15.152:0/450781870 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc93814a2d0 con 0x7fc93815ed40 2024-04-20T11:34:34.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:34.807+0000 7fc93fe00640 1 -- 172.21.15.152:0/450781870 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc93814a850 con 0x7fc93815ed40 2024-04-20T11:34:34.809 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:34.808+0000 7fc93fe00640 1 -- 172.21.15.152:0/450781870 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc938069100 con 0x7fc93815ed40 2024-04-20T11:34:34.809 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:34.810+0000 7fc935600640 1 -- 172.21.15.152:0/450781870 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 19) ==== 91397+0+0 (secure 0 0 0) 0x7fc930046020 con 0x7fc93815ed40 2024-04-20T11:34:34.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:34.810+0000 7fc935600640 1 --2- 172.21.15.152:0/450781870 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fc91806cce0 0x7fc91806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:34.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:34.811+0000 7fc935600640 1 -- 172.21.15.152:0/450781870 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5227+0+0 (secure 0 0 0) 0x7fc9300ae9e0 con 0x7fc93815ed40 2024-04-20T11:34:34.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:34.811+0000 7fc93d400640 1 --2- 172.21.15.152:0/450781870 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fc91806cce0 0x7fc91806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:34.812 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:34.812+0000 7fc93d400640 1 --2- 172.21.15.152:0/450781870 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fc91806cce0 0x7fc91806f1a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc9380b7350 tx=0x7fc928008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:34.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:34.816+0000 7fc935600640 1 -- 172.21.15.152:0/450781870 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fc93004d030 con 0x7fc93815ed40 2024-04-20T11:34:35.018 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:35.018+0000 7fc93fe00640 1 -- 172.21.15.152:0/450781870 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fc9380d2960 con 0x7fc93815ed40 2024-04-20T11:34:35.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:35.579+0000 7fc935600640 1 -- 172.21.15.152:0/450781870 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v116) ==== 170+0+0 (secure 0 0 0) 0x7fc93007ca60 con 0x7fc93815ed40 2024-04-20T11:34:35.579 INFO:teuthology.orchestra.run.smithi152.stderr:pool 'cephfs_data' created 2024-04-20T11:34:35.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:35.583+0000 7fc93fe00640 1 -- 172.21.15.152:0/450781870 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fc91806cce0 msgr2=0x7fc91806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:35.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:35.583+0000 7fc93fe00640 1 --2- 172.21.15.152:0/450781870 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fc91806cce0 0x7fc91806f1a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc9380b7350 tx=0x7fc928008040 comp rx=0 tx=0).stop 2024-04-20T11:34:35.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:35.583+0000 7fc93fe00640 1 -- 172.21.15.152:0/450781870 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc93815ed40 msgr2=0x7fc9381612d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:35.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:35.583+0000 7fc93fe00640 1 --2- 172.21.15.152:0/450781870 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc93815ed40 0x7fc9381612d0 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7fc930036040 tx=0x7fc930004a40 comp rx=0 tx=0).stop 2024-04-20T11:34:35.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:35.584+0000 7fc93fe00640 1 -- 172.21.15.152:0/450781870 shutdown_connections 2024-04-20T11:34:35.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:35.584+0000 7fc93fe00640 1 --2- 172.21.15.152:0/450781870 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fc91806cce0 0x7fc91806f1a0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:35.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:35.584+0000 7fc93fe00640 1 --2- 172.21.15.152:0/450781870 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc93815ed40 0x7fc9381612d0 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:35.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:35.584+0000 7fc93fe00640 1 --2- 172.21.15.152:0/450781870 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc9380cf880 0x7fc938149850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:35.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:35.584+0000 7fc93fe00640 1 --2- 172.21.15.152:0/450781870 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc9380b7d50 0x7fc93814cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:35.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:35.584+0000 7fc93fe00640 1 -- 172.21.15.152:0/450781870 >> 172.21.15.152:0/450781870 conn(0x7fc938067d50 msgr2=0x7fc9380b8850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:35.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:35.584+0000 7fc93fe00640 1 -- 172.21.15.152:0/450781870 shutdown_connections 2024-04-20T11:34:35.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:35.584+0000 7fc93fe00640 1 -- 172.21.15.152:0/450781870 wait complete. 2024-04-20T11:34:35.618 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-20T11:34:35.797 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:35.797+0000 7fc608000640 1 Processor -- start 2024-04-20T11:34:35.797 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:35.797+0000 7fc608000640 1 -- start start 2024-04-20T11:34:35.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:35.799+0000 7fc608000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fc60015ec30 0x7fc60015f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:35.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:35.799+0000 7fc608000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc6000b0660 0x7fc6000ae3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:35.800 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:35.799+0000 7fc608000640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc60015f6a0 con 0x7fc6000b23e0 2024-04-20T11:34:35.800 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:35.799+0000 7fc608000640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fc60015f980 con 0x7fc60015ec30 2024-04-20T11:34:35.800 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:35.799+0000 7fc608000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc60015fc60 con 0x7fc6000b0660 2024-04-20T11:34:35.800 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:35.799+0000 7fc605600640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fc60015ec30 0x7fc60015f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:35.800 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:35.799+0000 7fc605600640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fc60015ec30 0x7fc60015f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:50714/0 (socket says 172.21.15.152:50714) 2024-04-20T11:34:35.800 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:35.799+0000 7fc605600640 1 -- 172.21.15.152:0/404004918 learned_addr learned my addr 172.21.15.152:0/404004918 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:35.800 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:35.800+0000 7fc604c00640 1 --2- 172.21.15.152:0/404004918 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc6000b0660 0x7fc6000ae3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:35.800 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:35.800+0000 7fc605600640 1 -- 172.21.15.152:0/404004918 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc6000b0660 msgr2=0x7fc6000ae3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:35.800 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:35.800+0000 7fc605600640 1 --2- 172.21.15.152:0/404004918 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc6000b0660 0x7fc6000ae3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:35.800 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:35.800+0000 7fc605600640 1 -- 172.21.15.152:0/404004918 >> v1:172.21.15.175:6789/0 conn(0x7fc6000b23e0 legacy=0x7fc6000b2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:35.800 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:35.800+0000 7fc605600640 1 -- 172.21.15.152:0/404004918 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc60015ff40 con 0x7fc60015ec30 2024-04-20T11:34:35.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:35.801+0000 7fc605600640 1 --2- 172.21.15.152:0/404004918 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc60015ec30 0x7fc60015f010 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7fc5f4009870 tx=0x7fc5f402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bca252ab974800f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:35.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:35.801+0000 7fc5ffe00640 1 -- 172.21.15.152:0/404004918 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f4035020 con 0x7fc60015ec30 2024-04-20T11:34:35.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:35.801+0000 7fc5ffe00640 1 -- 172.21.15.152:0/404004918 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5f4031640 con 0x7fc60015ec30 2024-04-20T11:34:35.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:35.801+0000 7fc5ffe00640 1 -- 172.21.15.152:0/404004918 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f402d040 con 0x7fc60015ec30 2024-04-20T11:34:35.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:35.802+0000 7fc608000640 1 -- 172.21.15.152:0/404004918 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc60015ec30 msgr2=0x7fc60015f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:35.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:35.802+0000 7fc608000640 1 --2- 172.21.15.152:0/404004918 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc60015ec30 0x7fc60015f010 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7fc5f4009870 tx=0x7fc5f402b8f0 comp rx=0 tx=0).stop 2024-04-20T11:34:35.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:35.802+0000 7fc608000640 1 -- 172.21.15.152:0/404004918 shutdown_connections 2024-04-20T11:34:35.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:35.802+0000 7fc608000640 1 --2- 172.21.15.152:0/404004918 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc6000b0660 0x7fc6000ae3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:35.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:35.802+0000 7fc608000640 1 --2- 172.21.15.152:0/404004918 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc60015ec30 0x7fc60015f010 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:35.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:35.802+0000 7fc608000640 1 -- 172.21.15.152:0/404004918 >> 172.21.15.152:0/404004918 conn(0x7fc6000b6b00 msgr2=0x7fc60009aa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:35.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:35.802+0000 7fc608000640 1 -- 172.21.15.152:0/404004918 shutdown_connections 2024-04-20T11:34:35.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:35.802+0000 7fc608000640 1 -- 172.21.15.152:0/404004918 wait complete. 2024-04-20T11:34:35.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:35.804+0000 7fc608000640 1 Processor -- start 2024-04-20T11:34:35.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:35.804+0000 7fc608000640 1 -- start start 2024-04-20T11:34:35.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:35.805+0000 7fc608000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc6000b0660 0x7fc6000b4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:35.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:35.805+0000 7fc608000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc6000b23e0 0x7fc6000cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:35.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:35.805+0000 7fc608000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc60015ec30 0x7fc600149400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:35.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:35.805+0000 7fc608000640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fc600073640 con 0x7fc60015ec30 2024-04-20T11:34:35.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:35.805+0000 7fc608000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc6000734c0 con 0x7fc6000b0660 2024-04-20T11:34:35.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:35.805+0000 7fc608000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc60006bdd0 con 0x7fc6000b23e0 2024-04-20T11:34:35.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:35.805+0000 7fc606000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc60015ec30 0x7fc600149400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:35.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:35.805+0000 7fc606000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc60015ec30 0x7fc600149400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:50720/0 (socket says 172.21.15.152:50720) 2024-04-20T11:34:35.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:35.805+0000 7fc606000640 1 -- 172.21.15.152:0/2302706777 learned_addr learned my addr 172.21.15.152:0/2302706777 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:35.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:35.805+0000 7fc604c00640 1 --2- 172.21.15.152:0/2302706777 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc6000b23e0 0x7fc6000cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:35.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:35.806+0000 7fc605600640 1 --2- 172.21.15.152:0/2302706777 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc6000b0660 0x7fc6000b4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:35.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:35.806+0000 7fc606000640 1 -- 172.21.15.152:0/2302706777 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc6000b23e0 msgr2=0x7fc6000cf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:35.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:35.806+0000 7fc606000640 1 --2- 172.21.15.152:0/2302706777 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc6000b23e0 0x7fc6000cf5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:35.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:35.806+0000 7fc606000640 1 -- 172.21.15.152:0/2302706777 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc6000b0660 msgr2=0x7fc6000b4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:35.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:35.806+0000 7fc606000640 1 --2- 172.21.15.152:0/2302706777 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc6000b0660 0x7fc6000b4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:35.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:35.806+0000 7fc606000640 1 -- 172.21.15.152:0/2302706777 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc600148c20 con 0x7fc60015ec30 2024-04-20T11:34:35.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:35.806+0000 7fc604c00640 1 --2- 172.21.15.152:0/2302706777 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc6000b23e0 0x7fc6000cf5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:34:35.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:35.806+0000 7fc605600640 1 --2- 172.21.15.152:0/2302706777 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc6000b0660 0x7fc6000b4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:34:35.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:35.806+0000 7fc606000640 1 --2- 172.21.15.152:0/2302706777 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc60015ec30 0x7fc600149400 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7fc5f800c780 tx=0x7fc5f800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:35.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:35.806+0000 7fc5fd600640 1 -- 172.21.15.152:0/2302706777 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f8004b60 con 0x7fc60015ec30 2024-04-20T11:34:35.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:35.806+0000 7fc5fd600640 1 -- 172.21.15.152:0/2302706777 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5f8004d00 con 0x7fc60015ec30 2024-04-20T11:34:35.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:35.806+0000 7fc5fd600640 1 -- 172.21.15.152:0/2302706777 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f800aa30 con 0x7fc60015ec30 2024-04-20T11:34:35.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:35.806+0000 7fc608000640 1 -- 172.21.15.152:0/2302706777 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6000b5af0 con 0x7fc60015ec30 2024-04-20T11:34:35.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:35.806+0000 7fc608000640 1 -- 172.21.15.152:0/2302706777 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6000b5810 con 0x7fc60015ec30 2024-04-20T11:34:35.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:35.808+0000 7fc608000640 1 -- 172.21.15.152:0/2302706777 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc600069100 con 0x7fc60015ec30 2024-04-20T11:34:35.809 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:35.809+0000 7fc5fd600640 1 -- 172.21.15.152:0/2302706777 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 19) ==== 91397+0+0 (secure 0 0 0) 0x7fc5f800abd0 con 0x7fc60015ec30 2024-04-20T11:34:35.809 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:35.810+0000 7fc5fd600640 1 --2- 172.21.15.152:0/2302706777 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fc5e006cce0 0x7fc5e006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:35.809 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:35.810+0000 7fc5fd600640 1 -- 172.21.15.152:0/2302706777 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5609+0+0 (secure 0 0 0) 0x7fc5f808c440 con 0x7fc60015ec30 2024-04-20T11:34:35.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:35.810+0000 7fc605600640 1 --2- 172.21.15.152:0/2302706777 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fc5e006cce0 0x7fc5e006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:35.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:35.812+0000 7fc605600640 1 --2- 172.21.15.152:0/2302706777 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fc5e006cce0 0x7fc5e006f1a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc5f4036000 tx=0x7fc5f4008030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:35.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:35.815+0000 7fc5fd600640 1 -- 172.21.15.152:0/2302706777 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fc5f8020070 con 0x7fc60015ec30 2024-04-20T11:34:36.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:36.049+0000 7fc608000640 1 -- 172.21.15.152:0/2302706777 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fc60006c3e0 con 0x7fc60015ec30 2024-04-20T11:34:36.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:36.584+0000 7fc5fd600640 1 -- 172.21.15.152:0/2302706777 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 18 and data pool 19 v72) ==== 174+0+0 (secure 0 0 0) 0x7fc5f805a340 con 0x7fc60015ec30 2024-04-20T11:34:36.584 INFO:teuthology.orchestra.run.smithi152.stderr:new fs with metadata pool 18 and data pool 19 2024-04-20T11:34:36.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:36.588+0000 7fc608000640 1 -- 172.21.15.152:0/2302706777 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fc5e006cce0 msgr2=0x7fc5e006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:36.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:36.588+0000 7fc608000640 1 --2- 172.21.15.152:0/2302706777 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fc5e006cce0 0x7fc5e006f1a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc5f4036000 tx=0x7fc5f4008030 comp rx=0 tx=0).stop 2024-04-20T11:34:36.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:36.588+0000 7fc608000640 1 -- 172.21.15.152:0/2302706777 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc60015ec30 msgr2=0x7fc600149400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:36.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:36.588+0000 7fc608000640 1 --2- 172.21.15.152:0/2302706777 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc60015ec30 0x7fc600149400 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7fc5f800c780 tx=0x7fc5f800cc40 comp rx=0 tx=0).stop 2024-04-20T11:34:36.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:36.588+0000 7fc608000640 1 -- 172.21.15.152:0/2302706777 shutdown_connections 2024-04-20T11:34:36.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:36.588+0000 7fc608000640 1 --2- 172.21.15.152:0/2302706777 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fc5e006cce0 0x7fc5e006f1a0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:36.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:36.588+0000 7fc608000640 1 --2- 172.21.15.152:0/2302706777 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc60015ec30 0x7fc600149400 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:36.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:36.588+0000 7fc608000640 1 --2- 172.21.15.152:0/2302706777 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc6000b23e0 0x7fc6000cf5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:36.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:36.588+0000 7fc608000640 1 --2- 172.21.15.152:0/2302706777 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc6000b0660 0x7fc6000b4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:36.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:36.588+0000 7fc608000640 1 -- 172.21.15.152:0/2302706777 >> 172.21.15.152:0/2302706777 conn(0x7fc6000b6b00 msgr2=0x7fc6000af310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:36.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:36.589+0000 7fc608000640 1 -- 172.21.15.152:0/2302706777 shutdown_connections 2024-04-20T11:34:36.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:36.589+0000 7fc608000640 1 -- 172.21.15.152:0/2302706777 wait complete. 2024-04-20T11:34:36.635 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:34:36.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:36.806+0000 7f6927400640 1 Processor -- start 2024-04-20T11:34:36.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:36.806+0000 7f6927400640 1 -- start start 2024-04-20T11:34:36.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:36.808+0000 7f6927400640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f692015c560 0x7f69200b3df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:36.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:36.808+0000 7f6927400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f69200b4330 0x7f69200d0100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:36.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:36.808+0000 7f6927400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f69200d0640 0x7f6920071db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:36.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:36.808+0000 7f6927400640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f692015fd20 con 0x7f69200b4330 2024-04-20T11:34:36.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:36.808+0000 7f6927400640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f692015ef60 con 0x7f692015c560 2024-04-20T11:34:36.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:36.808+0000 7f6927400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f692015f240 con 0x7f69200d0640 2024-04-20T11:34:36.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:36.808+0000 7f691fe00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f69200b4330 0x7f69200d0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:36.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:36.808+0000 7f691fe00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f69200b4330 0x7f69200d0100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:50722/0 (socket says 172.21.15.152:50722) 2024-04-20T11:34:36.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:36.808+0000 7f691fe00640 1 -- 172.21.15.152:0/1218196952 learned_addr learned my addr 172.21.15.152:0/1218196952 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:36.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:36.808+0000 7f6925400640 1 --2- 172.21.15.152:0/1218196952 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f69200d0640 0x7f6920071db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:36.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:36.808+0000 7f6924a00640 1 --2- 172.21.15.152:0/1218196952 >> v2:172.21.15.175:3300/0 conn(0x7f692015c560 0x7f69200b3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:36.809 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:36.809+0000 7f691fe00640 1 -- 172.21.15.152:0/1218196952 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f69200d0640 msgr2=0x7f6920071db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:36.809 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:36.809+0000 7f691fe00640 1 --2- 172.21.15.152:0/1218196952 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f69200d0640 0x7f6920071db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:36.809 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:36.809+0000 7f691fe00640 1 -- 172.21.15.152:0/1218196952 >> v2:172.21.15.175:3300/0 conn(0x7f692015c560 msgr2=0x7f69200b3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:36.809 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:36.809+0000 7f691fe00640 1 --2- 172.21.15.152:0/1218196952 >> v2:172.21.15.175:3300/0 conn(0x7f692015c560 0x7f69200b3df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:36.809 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:36.809+0000 7f691fe00640 1 -- 172.21.15.152:0/1218196952 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f692015f520 con 0x7f69200b4330 2024-04-20T11:34:36.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:36.810+0000 7f691fe00640 1 --2- 172.21.15.152:0/1218196952 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f69200b4330 0x7f69200d0100 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f692006ff80 tx=0x7f690c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe6be4f4415d2349 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:36.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:36.810+0000 7f691f400640 1 -- 172.21.15.152:0/1218196952 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f690c02e070 con 0x7f69200b4330 2024-04-20T11:34:36.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:36.810+0000 7f691f400640 1 -- 172.21.15.152:0/1218196952 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f690c037070 con 0x7f69200b4330 2024-04-20T11:34:36.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:36.810+0000 7f691f400640 1 -- 172.21.15.152:0/1218196952 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f690c031a00 con 0x7f69200b4330 2024-04-20T11:34:36.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:36.811+0000 7f6927400640 1 -- 172.21.15.152:0/1218196952 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f69200b4330 msgr2=0x7f69200d0100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:36.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:36.811+0000 7f6927400640 1 --2- 172.21.15.152:0/1218196952 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f69200b4330 0x7f69200d0100 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f692006ff80 tx=0x7f690c02b880 comp rx=0 tx=0).stop 2024-04-20T11:34:36.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:36.811+0000 7f6927400640 1 -- 172.21.15.152:0/1218196952 shutdown_connections 2024-04-20T11:34:36.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:36.811+0000 7f6927400640 1 --2- 172.21.15.152:0/1218196952 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f69200d0640 0x7f6920071db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:36.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:36.811+0000 7f6927400640 1 --2- 172.21.15.152:0/1218196952 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f69200b4330 0x7f69200d0100 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:36.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:36.811+0000 7f6927400640 1 --2- 172.21.15.152:0/1218196952 >> v2:172.21.15.175:3300/0 conn(0x7f692015c560 0x7f69200b3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:36.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:36.811+0000 7f6927400640 1 -- 172.21.15.152:0/1218196952 >> 172.21.15.152:0/1218196952 conn(0x7f6920067cf0 msgr2=0x7f692006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:36.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:36.811+0000 7f6927400640 1 -- 172.21.15.152:0/1218196952 shutdown_connections 2024-04-20T11:34:36.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:36.811+0000 7f6927400640 1 -- 172.21.15.152:0/1218196952 wait complete. 2024-04-20T11:34:36.812 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:36.813+0000 7f6927400640 1 Processor -- start 2024-04-20T11:34:36.812 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:36.813+0000 7f6927400640 1 -- start start 2024-04-20T11:34:36.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:36.814+0000 7f6927400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f69200b4330 0x7f69201f45c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:36.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:36.814+0000 7f6927400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f69200d0640 0x7f6920155d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:36.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:36.814+0000 7f6927400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f692015c560 0x7f69200cfc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:36.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:36.814+0000 7f6927400640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f69200d0a80 con 0x7f69200d0640 2024-04-20T11:34:36.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:36.814+0000 7f6927400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f69200d0dc0 con 0x7f692015c560 2024-04-20T11:34:36.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:36.814+0000 7f6927400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f69200722f0 con 0x7f69200b4330 2024-04-20T11:34:36.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:36.814+0000 7f691fe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f69200d0640 0x7f6920155d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:36.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:36.814+0000 7f691fe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f69200d0640 0x7f6920155d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:50728/0 (socket says 172.21.15.152:50728) 2024-04-20T11:34:36.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:36.814+0000 7f691fe00640 1 -- 172.21.15.152:0/1580812437 learned_addr learned my addr 172.21.15.152:0/1580812437 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:36.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:36.815+0000 7f6924a00640 1 --2- 172.21.15.152:0/1580812437 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f69200b4330 0x7f69201f45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:36.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:36.815+0000 7f6925400640 1 --2- 172.21.15.152:0/1580812437 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f692015c560 0x7f69200cfc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:36.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:36.815+0000 7f691fe00640 1 -- 172.21.15.152:0/1580812437 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f69200b4330 msgr2=0x7f69201f45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:36.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:36.815+0000 7f691fe00640 1 --2- 172.21.15.152:0/1580812437 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f69200b4330 0x7f69201f45c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:36.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:36.815+0000 7f691fe00640 1 -- 172.21.15.152:0/1580812437 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f692015c560 msgr2=0x7f69200cfc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:36.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:36.815+0000 7f691fe00640 1 --2- 172.21.15.152:0/1580812437 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f692015c560 0x7f69200cfc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:36.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:36.815+0000 7f691fe00640 1 -- 172.21.15.152:0/1580812437 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69200cdf60 con 0x7f69200d0640 2024-04-20T11:34:36.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:36.815+0000 7f691fe00640 1 --2- 172.21.15.152:0/1580812437 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f69200d0640 0x7f6920155d10 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f690c02b8c0 tx=0x7f690c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:36.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:36.815+0000 7f691cc00640 1 -- 172.21.15.152:0/1580812437 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f690c032880 con 0x7f69200d0640 2024-04-20T11:34:36.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:36.815+0000 7f691cc00640 1 -- 172.21.15.152:0/1580812437 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f690c032a40 con 0x7f69200d0640 2024-04-20T11:34:36.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:36.815+0000 7f691cc00640 1 -- 172.21.15.152:0/1580812437 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f690c032e30 con 0x7f69200d0640 2024-04-20T11:34:36.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:36.815+0000 7f6927400640 1 -- 172.21.15.152:0/1580812437 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69200ce240 con 0x7f69200d0640 2024-04-20T11:34:36.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:36.816+0000 7f6927400640 1 -- 172.21.15.152:0/1580812437 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6920157930 con 0x7f69200d0640 2024-04-20T11:34:36.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:36.817+0000 7f6927400640 1 -- 172.21.15.152:0/1580812437 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69200689f0 con 0x7f69200d0640 2024-04-20T11:34:36.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:36.818+0000 7f691cc00640 1 -- 172.21.15.152:0/1580812437 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 19) ==== 91397+0+0 (secure 0 0 0) 0x7f690c0453f0 con 0x7f69200d0640 2024-04-20T11:34:36.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:36.819+0000 7f691cc00640 1 --2- 172.21.15.152:0/1580812437 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f690006cce0 0x7f690006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:36.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:36.819+0000 7f691cc00640 1 -- 172.21.15.152:0/1580812437 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5709+0+0 (secure 0 0 0) 0x7f690c0ae590 con 0x7f69200d0640 2024-04-20T11:34:36.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:36.820+0000 7f6924a00640 1 --2- 172.21.15.152:0/1580812437 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f690006cce0 0x7f690006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:36.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:36.821+0000 7f6924a00640 1 --2- 172.21.15.152:0/1580812437 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f690006cce0 0x7f690006f1a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6914006fd0 tx=0x7f6914008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:36.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:36.825+0000 7f691cc00640 1 -- 172.21.15.152:0/1580812437 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f690c040070 con 0x7f69200d0640 2024-04-20T11:34:37.010 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.010+0000 7f6927400640 1 -- 172.21.15.152:0/1580812437 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6920159e90 con 0x7f69200d0640 2024-04-20T11:34:37.013 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.013+0000 7f691cc00640 1 -- 172.21.15.152:0/1580812437 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v117) ==== 74+0+16752 (secure 0 0 0) 0x7f690c07c240 con 0x7f69200d0640 2024-04-20T11:34:37.013 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:34:37.013 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":117,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:34:36.051011+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:34:33.781054+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-04-20T11:34:35.031728+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:34:37.016 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.016+0000 7f6927400640 1 -- 172.21.15.152:0/1580812437 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f690006cce0 msgr2=0x7f690006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:37.016 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.016+0000 7f6927400640 1 --2- 172.21.15.152:0/1580812437 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f690006cce0 0x7f690006f1a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6914006fd0 tx=0x7f6914008040 comp rx=0 tx=0).stop 2024-04-20T11:34:37.016 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.016+0000 7f6927400640 1 -- 172.21.15.152:0/1580812437 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f69200d0640 msgr2=0x7f6920155d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:37.016 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.016+0000 7f6927400640 1 --2- 172.21.15.152:0/1580812437 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f69200d0640 0x7f6920155d10 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f690c02b8c0 tx=0x7f690c0046c0 comp rx=0 tx=0).stop 2024-04-20T11:34:37.016 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.016+0000 7f6927400640 1 -- 172.21.15.152:0/1580812437 shutdown_connections 2024-04-20T11:34:37.016 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.016+0000 7f6927400640 1 --2- 172.21.15.152:0/1580812437 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f690006cce0 0x7f690006f1a0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:37.016 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.016+0000 7f6927400640 1 --2- 172.21.15.152:0/1580812437 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f692015c560 0x7f69200cfc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:37.016 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.016+0000 7f6927400640 1 --2- 172.21.15.152:0/1580812437 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f69200d0640 0x7f6920155d10 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:37.016 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.016+0000 7f6927400640 1 --2- 172.21.15.152:0/1580812437 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f69200b4330 0x7f69201f45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:37.016 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.016+0000 7f6927400640 1 -- 172.21.15.152:0/1580812437 >> 172.21.15.152:0/1580812437 conn(0x7f6920067cf0 msgr2=0x7f692015a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:37.016 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.016+0000 7f6927400640 1 -- 172.21.15.152:0/1580812437 shutdown_connections 2024-04-20T11:34:37.016 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.016+0000 7f6927400640 1 -- 172.21.15.152:0/1580812437 wait complete. 2024-04-20T11:34:37.034 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:34:37.223 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.224+0000 7f4a7b200640 1 Processor -- start 2024-04-20T11:34:37.223 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.224+0000 7f4a7b200640 1 -- start start 2024-04-20T11:34:37.227 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.225+0000 7f4a7b200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4a7c147410 0x7f4a7c0afa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:37.227 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.225+0000 7f4a7b200640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a7c0aec90 con 0x7f4a7c071cd0 2024-04-20T11:34:37.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.226+0000 7f4a7b200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a7c0aef70 con 0x7f4a7c06e720 2024-04-20T11:34:37.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.226+0000 7f4a7b200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4a7c0af250 con 0x7f4a7c147410 2024-04-20T11:34:37.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.226+0000 7f4a79400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f4a7c071cd0 0x7f4a7c06e010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:41150/0 (socket says 172.21.15.152:41150) 2024-04-20T11:34:37.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.226+0000 7f4a79400640 1 -- 172.21.15.152:0/2999920261 learned_addr learned my addr 172.21.15.152:0/2999920261 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:34:37.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.226+0000 7f4a78a00640 1 -- 172.21.15.152:0/2999920261 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2838081814 0 0) 0x7f4a7c0aec90 con 0x7f4a7c071cd0 2024-04-20T11:34:37.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.226+0000 7f4a78a00640 1 -- 172.21.15.152:0/2999920261 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4a60003520 con 0x7f4a7c071cd0 2024-04-20T11:34:37.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.226+0000 7f4a79e00640 1 --2- 172.21.15.152:0/2999920261 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4a7c147410 0x7f4a7c0afa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:37.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.227+0000 7f4a78a00640 1 -- 172.21.15.152:0/2999920261 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1264657686 0 0) 0x7f4a60003520 con 0x7f4a7c071cd0 2024-04-20T11:34:37.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.227+0000 7f4a78a00640 1 -- 172.21.15.152:0/2999920261 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4a7c147410 msgr2=0x7f4a7c0afa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:37.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.227+0000 7f4a78a00640 1 --2- 172.21.15.152:0/2999920261 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4a7c147410 0x7f4a7c0afa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:37.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.227+0000 7f4a78a00640 1 -- 172.21.15.152:0/2999920261 >> v1:172.21.15.175:6789/0 conn(0x7f4a7c06e720 legacy=0x7f4a7c0b3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:37.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.227+0000 7f4a78a00640 1 -- 172.21.15.152:0/2999920261 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a7c0af530 con 0x7f4a7c071cd0 2024-04-20T11:34:37.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.227+0000 7f4a78a00640 1 -- 172.21.15.152:0/2999920261 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f4a68001670 con 0x7f4a7c071cd0 2024-04-20T11:34:37.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.227+0000 7f4a78a00640 1 -- 172.21.15.152:0/2999920261 >> v1:172.21.15.152:6789/0 conn(0x7f4a7c071cd0 legacy=0x7f4a7c06e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:37.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.227+0000 7f4a78a00640 1 --2- 172.21.15.152:0/2999920261 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4a600039e0 0x7f4a60003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:37.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.227+0000 7f4a78a00640 1 --2- 172.21.15.152:0/2999920261 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4a60004800 0x7f4a60004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:37.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.227+0000 7f4a78a00640 1 --2- 172.21.15.152:0/2999920261 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4a7c147410 0x7f4a60005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:37.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.227+0000 7f4a78a00640 1 -- 172.21.15.152:0/2999920261 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f4a7c0af250 con 0x7f4a60004800 2024-04-20T11:34:37.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.227+0000 7f4a78a00640 1 -- 172.21.15.152:0/2999920261 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4a6000baf0 con 0x7f4a600039e0 2024-04-20T11:34:37.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.227+0000 7f4a78a00640 1 -- 172.21.15.152:0/2999920261 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4a6000bcf0 con 0x7f4a7c147410 2024-04-20T11:34:37.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.227+0000 7f4a79400640 1 --2- 172.21.15.152:0/2999920261 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4a60004800 0x7f4a60004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:37.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.227+0000 7f4a79e00640 1 --2- 172.21.15.152:0/2999920261 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4a600039e0 0x7f4a60003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:37.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.227+0000 7f4a79400640 1 -- 172.21.15.152:0/2999920261 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4a7c147410 msgr2=0x7f4a60005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:37.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.227+0000 7f4a79400640 1 --2- 172.21.15.152:0/2999920261 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4a7c147410 0x7f4a60005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:37.230 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.227+0000 7f4a79400640 1 -- 172.21.15.152:0/2999920261 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4a600039e0 msgr2=0x7f4a60003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:37.230 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.227+0000 7f4a79400640 1 --2- 172.21.15.152:0/2999920261 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4a600039e0 0x7f4a60003e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:37.230 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.227+0000 7f4a79400640 1 -- 172.21.15.152:0/2999920261 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a60003520 con 0x7f4a60004800 2024-04-20T11:34:37.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.228+0000 7f4a79400640 1 --2- 172.21.15.152:0/2999920261 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4a60004800 0x7f4a60004c80 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f4a6800c240 tx=0x7f4a60005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5a24a33ace52899 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:37.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.229+0000 7f4a78a00640 1 -- 172.21.15.152:0/2999920261 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a68007cf0 con 0x7f4a60004800 2024-04-20T11:34:37.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.229+0000 7f4a78a00640 1 -- 172.21.15.152:0/2999920261 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a68030070 con 0x7f4a60004800 2024-04-20T11:34:37.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.229+0000 7f4a78a00640 1 -- 172.21.15.152:0/2999920261 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a68005d30 con 0x7f4a60004800 2024-04-20T11:34:37.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.229+0000 7f4a7b200640 1 -- 172.21.15.152:0/2999920261 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4a60004800 msgr2=0x7f4a60004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:37.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.229+0000 7f4a7b200640 1 --2- 172.21.15.152:0/2999920261 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4a60004800 0x7f4a60004c80 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f4a6800c240 tx=0x7f4a60005830 comp rx=0 tx=0).stop 2024-04-20T11:34:37.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.229+0000 7f4a7a800640 1 -- 172.21.15.152:0/2999920261 reap_dead start 2024-04-20T11:34:37.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.229+0000 7f4a7b200640 1 -- 172.21.15.152:0/2999920261 shutdown_connections 2024-04-20T11:34:37.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.229+0000 7f4a7b200640 1 -- 172.21.15.152:0/2999920261 >> 172.21.15.152:0/2999920261 conn(0x7f4a7c067ae0 msgr2=0x7f4a7c06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:37.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.229+0000 7f4a7b200640 1 -- 172.21.15.152:0/2999920261 shutdown_connections 2024-04-20T11:34:37.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.229+0000 7f4a7b200640 1 -- 172.21.15.152:0/2999920261 wait complete. 2024-04-20T11:34:37.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.231+0000 7f4a7b200640 1 Processor -- start 2024-04-20T11:34:37.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.231+0000 7f4a7b200640 1 -- start start 2024-04-20T11:34:37.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.232+0000 7f4a7b200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4a60004800 0x7f4a7c144c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:37.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.232+0000 7f4a7b200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4a7c13eb70 0x7f4a7c13cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:37.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.232+0000 7f4a7b200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4a7c1400a0 0x7f4a7c0b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:37.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.232+0000 7f4a7b200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f4a7c0afbd0 con 0x7f4a7c1400a0 2024-04-20T11:34:37.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.232+0000 7f4a7b200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4a7c0afa50 con 0x7f4a60004800 2024-04-20T11:34:37.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.232+0000 7f4a7b200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4a7c0afd50 con 0x7f4a7c13eb70 2024-04-20T11:34:37.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.232+0000 7f4a7a800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4a7c1400a0 0x7f4a7c0b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:37.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.233+0000 7f4a7a800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4a7c1400a0 0x7f4a7c0b4be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:50750/0 (socket says 172.21.15.152:50750) 2024-04-20T11:34:37.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.233+0000 7f4a7a800640 1 -- 172.21.15.152:0/3900388557 learned_addr learned my addr 172.21.15.152:0/3900388557 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:37.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.233+0000 7f4a79400640 1 --2- 172.21.15.152:0/3900388557 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4a7c13eb70 0x7f4a7c13cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:37.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.233+0000 7f4a79e00640 1 --2- 172.21.15.152:0/3900388557 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4a60004800 0x7f4a7c144c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:37.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.233+0000 7f4a7a800640 1 -- 172.21.15.152:0/3900388557 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4a7c13eb70 msgr2=0x7f4a7c13cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:37.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.233+0000 7f4a7a800640 1 --2- 172.21.15.152:0/3900388557 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4a7c13eb70 0x7f4a7c13cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:37.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.233+0000 7f4a7a800640 1 -- 172.21.15.152:0/3900388557 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4a60004800 msgr2=0x7f4a7c144c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:37.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.233+0000 7f4a7a800640 1 --2- 172.21.15.152:0/3900388557 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4a60004800 0x7f4a7c144c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:37.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.233+0000 7f4a7a800640 1 -- 172.21.15.152:0/3900388557 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a7c13f000 con 0x7f4a7c1400a0 2024-04-20T11:34:37.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.233+0000 7f4a7a800640 1 --2- 172.21.15.152:0/3900388557 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4a7c1400a0 0x7f4a7c0b4be0 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c0b30e0 tx=0x7f4a7400cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:37.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.234+0000 7f4a72000640 1 -- 172.21.15.152:0/3900388557 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a74017070 con 0x7f4a7c1400a0 2024-04-20T11:34:37.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.234+0000 7f4a72000640 1 -- 172.21.15.152:0/3900388557 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a74005d60 con 0x7f4a7c1400a0 2024-04-20T11:34:37.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.234+0000 7f4a7b200640 1 -- 172.21.15.152:0/3900388557 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a7c13f2e0 con 0x7f4a7c1400a0 2024-04-20T11:34:37.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.234+0000 7f4a72000640 1 -- 172.21.15.152:0/3900388557 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a7400a7f0 con 0x7f4a7c1400a0 2024-04-20T11:34:37.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.234+0000 7f4a7b200640 1 -- 172.21.15.152:0/3900388557 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a7c13f8a0 con 0x7f4a7c1400a0 2024-04-20T11:34:37.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.235+0000 7f4a7b200640 1 -- 172.21.15.152:0/3900388557 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a7c075e60 con 0x7f4a7c1400a0 2024-04-20T11:34:37.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.236+0000 7f4a72000640 1 -- 172.21.15.152:0/3900388557 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 19) ==== 91397+0+0 (secure 0 0 0) 0x7f4a7401a070 con 0x7f4a7c1400a0 2024-04-20T11:34:37.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.237+0000 7f4a72000640 1 --2- 172.21.15.152:0/3900388557 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4a5c06cd10 0x7f4a5c06f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:37.237 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.237+0000 7f4a72000640 1 -- 172.21.15.152:0/3900388557 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5709+0+0 (secure 0 0 0) 0x7f4a7408b5e0 con 0x7f4a7c1400a0 2024-04-20T11:34:37.237 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.237+0000 7f4a79e00640 1 --2- 172.21.15.152:0/3900388557 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4a5c06cd10 0x7f4a5c06f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:37.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.239+0000 7f4a79e00640 1 --2- 172.21.15.152:0/3900388557 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4a5c06cd10 0x7f4a5c06f1d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c13eb00 tx=0x7f4a6c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:37.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.241+0000 7f4a72000640 1 -- 172.21.15.152:0/3900388557 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f4a740593f0 con 0x7f4a7c1400a0 2024-04-20T11:34:37.439 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.440+0000 7f4a7b200640 1 -- 172.21.15.152:0/3900388557 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4a7c0b0f20 con 0x7f4a7c1400a0 2024-04-20T11:34:37.442 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.443+0000 7f4a72000640 1 -- 172.21.15.152:0/3900388557 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v117) ==== 74+0+16752 (secure 0 0 0) 0x7f4a74058cf0 con 0x7f4a7c1400a0 2024-04-20T11:34:37.443 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:34:37.443 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":117,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:34:36.051011+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:34:33.781054+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-04-20T11:34:35.031728+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:34:37.445 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.445+0000 7f4a7b200640 1 -- 172.21.15.152:0/3900388557 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4a5c06cd10 msgr2=0x7f4a5c06f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:37.445 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.445+0000 7f4a7b200640 1 --2- 172.21.15.152:0/3900388557 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4a5c06cd10 0x7f4a5c06f1d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c13eb00 tx=0x7f4a6c008040 comp rx=0 tx=0).stop 2024-04-20T11:34:37.445 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.445+0000 7f4a7b200640 1 -- 172.21.15.152:0/3900388557 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4a7c1400a0 msgr2=0x7f4a7c0b4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:37.446 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.445+0000 7f4a7b200640 1 --2- 172.21.15.152:0/3900388557 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4a7c1400a0 0x7f4a7c0b4be0 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c0b30e0 tx=0x7f4a7400cb90 comp rx=0 tx=0).stop 2024-04-20T11:34:37.446 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.445+0000 7f4a7b200640 1 -- 172.21.15.152:0/3900388557 shutdown_connections 2024-04-20T11:34:37.446 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.445+0000 7f4a7b200640 1 --2- 172.21.15.152:0/3900388557 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4a5c06cd10 0x7f4a5c06f1d0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:37.446 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.445+0000 7f4a7b200640 1 --2- 172.21.15.152:0/3900388557 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4a7c1400a0 0x7f4a7c0b4be0 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:37.446 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.445+0000 7f4a7b200640 1 --2- 172.21.15.152:0/3900388557 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4a7c13eb70 0x7f4a7c13cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:37.446 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.445+0000 7f4a7b200640 1 --2- 172.21.15.152:0/3900388557 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4a60004800 0x7f4a7c144c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:37.446 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.445+0000 7f4a7b200640 1 -- 172.21.15.152:0/3900388557 >> 172.21.15.152:0/3900388557 conn(0x7f4a7c067ae0 msgr2=0x7f4a7c06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:37.446 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.446+0000 7f4a7b200640 1 -- 172.21.15.152:0/3900388557 shutdown_connections 2024-04-20T11:34:37.446 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.446+0000 7f4a7b200640 1 -- 172.21.15.152:0/3900388557 wait complete. 2024-04-20T11:34:37.466 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-20T11:34:37.677 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.678+0000 7ff553a00640 1 Processor -- start 2024-04-20T11:34:37.677 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.678+0000 7ff553a00640 1 -- start start 2024-04-20T11:34:37.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.679+0000 7ff553a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff54c15e9e0 0x7ff54c06d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:37.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.679+0000 7ff553a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7ff54c0d46a0 0x7ff54c0d4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:37.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.679+0000 7ff553a00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff54c0d19a0 con 0x7ff54c0d0d90 2024-04-20T11:34:37.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.679+0000 7ff553a00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7ff54c0d5040 con 0x7ff54c0d46a0 2024-04-20T11:34:37.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.679+0000 7ff553a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff54c0d1c80 con 0x7ff54c15e9e0 2024-04-20T11:34:37.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.679+0000 7ff543e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7ff54c0d46a0 0x7ff54c0d4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:37.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.679+0000 7ff543e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7ff54c0d46a0 0x7ff54c0d4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:50760/0 (socket says 172.21.15.152:50760) 2024-04-20T11:34:37.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.679+0000 7ff543e00640 1 -- 172.21.15.152:0/3752022597 learned_addr learned my addr 172.21.15.152:0/3752022597 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:37.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.680+0000 7ff551000640 1 --2- 172.21.15.152:0/3752022597 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff54c15e9e0 0x7ff54c06d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:37.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.680+0000 7ff543e00640 1 -- 172.21.15.152:0/3752022597 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff54c15e9e0 msgr2=0x7ff54c06d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:37.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.680+0000 7ff543e00640 1 --2- 172.21.15.152:0/3752022597 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff54c15e9e0 0x7ff54c06d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:37.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.680+0000 7ff543e00640 1 -- 172.21.15.152:0/3752022597 >> v1:172.21.15.175:6789/0 conn(0x7ff54c0d0d90 legacy=0x7ff54c0cfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:37.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.680+0000 7ff543e00640 1 -- 172.21.15.152:0/3752022597 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff54c0d1f60 con 0x7ff54c0d46a0 2024-04-20T11:34:37.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.681+0000 7ff543e00640 1 --2- 172.21.15.152:0/3752022597 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff54c0d46a0 0x7ff54c0d4b00 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7ff534009870 tx=0x7ff53402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8158ccc354754996 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:37.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.681+0000 7ff54be00640 1 -- 172.21.15.152:0/3752022597 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff534035020 con 0x7ff54c0d46a0 2024-04-20T11:34:37.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.681+0000 7ff54be00640 1 -- 172.21.15.152:0/3752022597 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff534031640 con 0x7ff54c0d46a0 2024-04-20T11:34:37.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.681+0000 7ff54be00640 1 -- 172.21.15.152:0/3752022597 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff53402d040 con 0x7ff54c0d46a0 2024-04-20T11:34:37.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.682+0000 7ff553a00640 1 -- 172.21.15.152:0/3752022597 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff54c0d46a0 msgr2=0x7ff54c0d4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:37.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.682+0000 7ff553a00640 1 --2- 172.21.15.152:0/3752022597 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff54c0d46a0 0x7ff54c0d4b00 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7ff534009870 tx=0x7ff53402b8f0 comp rx=0 tx=0).stop 2024-04-20T11:34:37.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.682+0000 7ff553a00640 1 -- 172.21.15.152:0/3752022597 shutdown_connections 2024-04-20T11:34:37.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.682+0000 7ff553a00640 1 --2- 172.21.15.152:0/3752022597 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff54c0d46a0 0x7ff54c0d4b00 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:37.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.682+0000 7ff553a00640 1 --2- 172.21.15.152:0/3752022597 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff54c15e9e0 0x7ff54c06d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:37.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.682+0000 7ff553a00640 1 -- 172.21.15.152:0/3752022597 >> 172.21.15.152:0/3752022597 conn(0x7ff54c09dab0 msgr2=0x7ff54c08bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:37.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.682+0000 7ff553a00640 1 -- 172.21.15.152:0/3752022597 shutdown_connections 2024-04-20T11:34:37.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.682+0000 7ff553a00640 1 -- 172.21.15.152:0/3752022597 wait complete. 2024-04-20T11:34:37.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.684+0000 7ff553a00640 1 Processor -- start 2024-04-20T11:34:37.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.684+0000 7ff553a00640 1 -- start start 2024-04-20T11:34:37.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.685+0000 7ff553a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff54c0d0d90 0x7ff54c153f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:37.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.685+0000 7ff553a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff54c0d46a0 0x7ff54c158620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:37.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.685+0000 7ff553a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff54c15e9e0 0x7ff54c0b6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:37.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.685+0000 7ff553a00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7ff54c06d860 con 0x7ff54c15e9e0 2024-04-20T11:34:37.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.685+0000 7ff553a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff54c0cf880 con 0x7ff54c0d0d90 2024-04-20T11:34:37.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.685+0000 7ff553a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff54c06d9e0 con 0x7ff54c0d46a0 2024-04-20T11:34:37.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.685+0000 7ff551a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff54c15e9e0 0x7ff54c0b6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:37.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.685+0000 7ff551a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff54c15e9e0 0x7ff54c0b6210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:50772/0 (socket says 172.21.15.152:50772) 2024-04-20T11:34:37.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.685+0000 7ff551a00640 1 -- 172.21.15.152:0/2831491274 learned_addr learned my addr 172.21.15.152:0/2831491274 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:37.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.686+0000 7ff551000640 1 --2- 172.21.15.152:0/2831491274 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff54c0d0d90 0x7ff54c153f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:37.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.686+0000 7ff551a00640 1 -- 172.21.15.152:0/2831491274 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff54c0d46a0 msgr2=0x7ff54c158620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:37.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.686+0000 7ff551a00640 1 --2- 172.21.15.152:0/2831491274 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff54c0d46a0 0x7ff54c158620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:37.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.686+0000 7ff551a00640 1 -- 172.21.15.152:0/2831491274 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff54c0d0d90 msgr2=0x7ff54c153f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:37.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.686+0000 7ff551a00640 1 --2- 172.21.15.152:0/2831491274 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff54c0d0d90 0x7ff54c153f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:37.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.686+0000 7ff551a00640 1 -- 172.21.15.152:0/2831491274 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff54c0cc470 con 0x7ff54c15e9e0 2024-04-20T11:34:37.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.686+0000 7ff551000640 1 --2- 172.21.15.152:0/2831491274 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff54c0d0d90 0x7ff54c153f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:34:37.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.686+0000 7ff551a00640 1 --2- 172.21.15.152:0/2831491274 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff54c15e9e0 0x7ff54c0b6210 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7ff54400c7b0 tx=0x7ff54400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:37.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.686+0000 7ff549600640 1 -- 172.21.15.152:0/2831491274 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff544004b60 con 0x7ff54c15e9e0 2024-04-20T11:34:37.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.686+0000 7ff549600640 1 -- 172.21.15.152:0/2831491274 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff544004d00 con 0x7ff54c15e9e0 2024-04-20T11:34:37.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.686+0000 7ff549600640 1 -- 172.21.15.152:0/2831491274 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff54400aa90 con 0x7ff54c15e9e0 2024-04-20T11:34:37.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.686+0000 7ff553a00640 1 -- 172.21.15.152:0/2831491274 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff54c0cc750 con 0x7ff54c15e9e0 2024-04-20T11:34:37.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.686+0000 7ff553a00640 1 -- 172.21.15.152:0/2831491274 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff54c158b60 con 0x7ff54c15e9e0 2024-04-20T11:34:37.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.688+0000 7ff553a00640 1 -- 172.21.15.152:0/2831491274 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff54c0689f0 con 0x7ff54c15e9e0 2024-04-20T11:34:37.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.689+0000 7ff549600640 1 -- 172.21.15.152:0/2831491274 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 19) ==== 91397+0+0 (secure 0 0 0) 0x7ff54400ac30 con 0x7ff54c15e9e0 2024-04-20T11:34:37.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.690+0000 7ff549600640 1 --2- 172.21.15.152:0/2831491274 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7ff52806cce0 0x7ff52806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:37.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.690+0000 7ff549600640 1 -- 172.21.15.152:0/2831491274 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5709+0+0 (secure 0 0 0) 0x7ff54408c5d0 con 0x7ff54c15e9e0 2024-04-20T11:34:37.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.690+0000 7ff551000640 1 --2- 172.21.15.152:0/2831491274 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7ff52806cce0 0x7ff52806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:37.691 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.691+0000 7ff551000640 1 --2- 172.21.15.152:0/2831491274 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7ff52806cce0 0x7ff52806f1a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff53c006fd0 tx=0x7ff53c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:37.693 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.694+0000 7ff549600640 1 -- 172.21.15.152:0/2831491274 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ff544020070 con 0x7ff54c15e9e0 2024-04-20T11:34:37.932 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:37.933+0000 7ff553a00640 1 -- 172.21.15.152:0/2831491274 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7ff54c06cdb0 con 0x7ff54c15e9e0 2024-04-20T11:34:38.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:38.631+0000 7ff549600640 1 -- 172.21.15.152:0/2831491274 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v75) ==== 118+0+0 (secure 0 0 0) 0x7ff54405a3e0 con 0x7ff54c15e9e0 2024-04-20T11:34:38.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:38.635+0000 7ff553a00640 1 -- 172.21.15.152:0/2831491274 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7ff52806cce0 msgr2=0x7ff52806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:38.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:38.635+0000 7ff553a00640 1 --2- 172.21.15.152:0/2831491274 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7ff52806cce0 0x7ff52806f1a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff53c006fd0 tx=0x7ff53c008040 comp rx=0 tx=0).stop 2024-04-20T11:34:38.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:38.635+0000 7ff553a00640 1 -- 172.21.15.152:0/2831491274 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff54c15e9e0 msgr2=0x7ff54c0b6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:38.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:38.635+0000 7ff553a00640 1 --2- 172.21.15.152:0/2831491274 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff54c15e9e0 0x7ff54c0b6210 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7ff54400c7b0 tx=0x7ff54400cc70 comp rx=0 tx=0).stop 2024-04-20T11:34:38.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:38.635+0000 7ff553a00640 1 -- 172.21.15.152:0/2831491274 shutdown_connections 2024-04-20T11:34:38.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:38.635+0000 7ff553a00640 1 --2- 172.21.15.152:0/2831491274 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7ff52806cce0 0x7ff52806f1a0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:38.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:38.635+0000 7ff553a00640 1 --2- 172.21.15.152:0/2831491274 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff54c15e9e0 0x7ff54c0b6210 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:38.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:38.635+0000 7ff553a00640 1 --2- 172.21.15.152:0/2831491274 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff54c0d46a0 0x7ff54c158620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:38.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:38.635+0000 7ff553a00640 1 --2- 172.21.15.152:0/2831491274 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff54c0d0d90 0x7ff54c153f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:38.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:38.635+0000 7ff553a00640 1 -- 172.21.15.152:0/2831491274 >> 172.21.15.152:0/2831491274 conn(0x7ff54c09dab0 msgr2=0x7ff54c0d11e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:38.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:38.635+0000 7ff553a00640 1 -- 172.21.15.152:0/2831491274 shutdown_connections 2024-04-20T11:34:38.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:38.635+0000 7ff553a00640 1 -- 172.21.15.152:0/2831491274 wait complete. 2024-04-20T11:34:38.656 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-20T11:34:38.656 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-20T11:34:38.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:38.862+0000 7f4f63000640 1 Processor -- start 2024-04-20T11:34:38.863 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:38.863+0000 7f4f63000640 1 -- start start 2024-04-20T11:34:38.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:38.864+0000 7f4f63000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f4f5c0d35a0 0x7f4f5c1698d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:38.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:38.864+0000 7f4f63000640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f5c0d19c0 con 0x7f4f5c163230 2024-04-20T11:34:38.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:38.864+0000 7f4f63000640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f5c162750 con 0x7f4f5c15eb50 2024-04-20T11:34:38.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:38.864+0000 7f4f63000640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f4f5c162a30 con 0x7f4f5c0d35a0 2024-04-20T11:34:38.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:38.864+0000 7f4f62600640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f4f5c0d35a0 0x7f4f5c1698d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:38.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:38.864+0000 7f4f61200640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f4f5c163230 0x7f4f5c0d2e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:41166/0 (socket says 172.21.15.152:41166) 2024-04-20T11:34:38.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:38.864+0000 7f4f61200640 1 -- 172.21.15.152:0/374088214 learned_addr learned my addr 172.21.15.152:0/374088214 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:34:38.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:38.865+0000 7f4f62600640 1 -- 172.21.15.152:0/374088214 >> v1:172.21.15.175:6789/0 conn(0x7f4f5c15eb50 legacy=0x7f4f5c0d0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:38.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:38.865+0000 7f4f62600640 1 -- 172.21.15.152:0/374088214 >> v1:172.21.15.152:6789/0 conn(0x7f4f5c163230 legacy=0x7f4f5c0d2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:38.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:38.865+0000 7f4f62600640 1 -- 172.21.15.152:0/374088214 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f5c162d10 con 0x7f4f5c0d35a0 2024-04-20T11:34:38.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:38.865+0000 7f4f5be00640 1 -- 172.21.15.152:0/374088214 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 568089899 0 0) 0x7f4f5c0d19c0 con 0x7f4f5c163230 2024-04-20T11:34:38.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:38.866+0000 7f4f62600640 1 --2- 172.21.15.152:0/374088214 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4f5c0d35a0 0x7f4f5c1698d0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f4f54009d90 tx=0x7f4f5402be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0ceb0f8c0520a10 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:38.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:38.867+0000 7f4f5be00640 1 -- 172.21.15.152:0/374088214 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f54039070 con 0x7f4f5c0d35a0 2024-04-20T11:34:38.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:38.867+0000 7f4f5be00640 1 -- 172.21.15.152:0/374088214 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f5402cbd0 con 0x7f4f5c0d35a0 2024-04-20T11:34:38.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:38.867+0000 7f4f5be00640 1 -- 172.21.15.152:0/374088214 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f54034440 con 0x7f4f5c0d35a0 2024-04-20T11:34:38.867 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:38.867+0000 7f4f63000640 1 -- 172.21.15.152:0/374088214 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4f5c0d35a0 msgr2=0x7f4f5c1698d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:38.867 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:38.867+0000 7f4f63000640 1 --2- 172.21.15.152:0/374088214 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4f5c0d35a0 0x7f4f5c1698d0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f4f54009d90 tx=0x7f4f5402be20 comp rx=0 tx=0).stop 2024-04-20T11:34:38.867 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:38.867+0000 7f4f63000640 1 -- 172.21.15.152:0/374088214 shutdown_connections 2024-04-20T11:34:38.867 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:38.867+0000 7f4f63000640 1 --2- 172.21.15.152:0/374088214 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4f5c0d35a0 0x7f4f5c1698d0 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:38.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:38.867+0000 7f4f63000640 1 -- 172.21.15.152:0/374088214 >> 172.21.15.152:0/374088214 conn(0x7f4f5c08bfd0 msgr2=0x7f4f5c0d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:38.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:38.868+0000 7f4f63000640 1 -- 172.21.15.152:0/374088214 shutdown_connections 2024-04-20T11:34:38.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:38.868+0000 7f4f63000640 1 -- 172.21.15.152:0/374088214 wait complete. 2024-04-20T11:34:38.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:38.869+0000 7f4f63000640 1 Processor -- start 2024-04-20T11:34:38.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:38.869+0000 7f4f63000640 1 -- start start 2024-04-20T11:34:38.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:38.870+0000 7f4f63000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4f5c0d35a0 0x7f4f5c0b8180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:38.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:38.870+0000 7f4f63000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4f5c15eb50 0x7f4f5c0b4200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:38.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:38.870+0000 7f4f63000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4f5c163230 0x7f4f5c143120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:38.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:38.870+0000 7f4f63000640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f4f5c0d1610 con 0x7f4f5c0d35a0 2024-04-20T11:34:38.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:38.870+0000 7f4f63000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4f5c169e10 con 0x7f4f5c15eb50 2024-04-20T11:34:38.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:38.870+0000 7f4f63000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4f5c159a40 con 0x7f4f5c163230 2024-04-20T11:34:38.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:38.871+0000 7f4f61c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4f5c0d35a0 0x7f4f5c0b8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:38.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:38.871+0000 7f4f61c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4f5c0d35a0 0x7f4f5c0b8180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:50784/0 (socket says 172.21.15.152:50784) 2024-04-20T11:34:38.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:38.871+0000 7f4f61c00640 1 -- 172.21.15.152:0/1960182960 learned_addr learned my addr 172.21.15.152:0/1960182960 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:38.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:38.871+0000 7f4f61c00640 1 -- 172.21.15.152:0/1960182960 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4f5c163230 msgr2=0x7f4f5c143120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:38.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:38.871+0000 7f4f61c00640 1 --2- 172.21.15.152:0/1960182960 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4f5c163230 0x7f4f5c143120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:38.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:38.871+0000 7f4f61c00640 1 -- 172.21.15.152:0/1960182960 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4f5c15eb50 msgr2=0x7f4f5c0b4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:38.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:38.871+0000 7f4f61c00640 1 --2- 172.21.15.152:0/1960182960 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4f5c15eb50 0x7f4f5c0b4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:38.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:38.871+0000 7f4f61c00640 1 -- 172.21.15.152:0/1960182960 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f5c0b8810 con 0x7f4f5c0d35a0 2024-04-20T11:34:38.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:38.871+0000 7f4f61c00640 1 --2- 172.21.15.152:0/1960182960 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4f5c0d35a0 0x7f4f5c0b8180 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f4f5000c460 tx=0x7f4f5000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:38.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:38.872+0000 7f4f59600640 1 -- 172.21.15.152:0/1960182960 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f50016020 con 0x7f4f5c0d35a0 2024-04-20T11:34:38.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:38.872+0000 7f4f59600640 1 -- 172.21.15.152:0/1960182960 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f50012870 con 0x7f4f5c0d35a0 2024-04-20T11:34:38.873 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:38.872+0000 7f4f59600640 1 -- 172.21.15.152:0/1960182960 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f50012c60 con 0x7f4f5c0d35a0 2024-04-20T11:34:38.873 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:38.872+0000 7f4f63000640 1 -- 172.21.15.152:0/1960182960 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f5c0b8af0 con 0x7f4f5c0d35a0 2024-04-20T11:34:38.873 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:38.872+0000 7f4f63000640 1 -- 172.21.15.152:0/1960182960 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f5c0b90b0 con 0x7f4f5c0d35a0 2024-04-20T11:34:38.873 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:38.873+0000 7f4f63000640 1 -- 172.21.15.152:0/1960182960 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f5c075e60 con 0x7f4f5c0d35a0 2024-04-20T11:34:38.873 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:38.874+0000 7f4f59600640 1 -- 172.21.15.152:0/1960182960 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 19) ==== 91397+0+0 (secure 0 0 0) 0x7f4f50019020 con 0x7f4f5c0d35a0 2024-04-20T11:34:38.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:38.875+0000 7f4f59600640 1 --2- 172.21.15.152:0/1960182960 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4f3c06cd10 0x7f4f3c06f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:38.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:38.875+0000 7f4f59600640 1 -- 172.21.15.152:0/1960182960 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5709+0+0 (secure 0 0 0) 0x7f4f5008c280 con 0x7f4f5c0d35a0 2024-04-20T11:34:38.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:38.875+0000 7f4f61200640 1 --2- 172.21.15.152:0/1960182960 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4f3c06cd10 0x7f4f3c06f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:38.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:38.877+0000 7f4f61200640 1 --2- 172.21.15.152:0/1960182960 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4f3c06cd10 0x7f4f3c06f1d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c0b9de0 tx=0x7f4f48007910 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:38.879 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:38.880+0000 7f4f59600640 1 -- 172.21.15.152:0/1960182960 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f4f50020020 con 0x7f4f5c0d35a0 2024-04-20T11:34:39.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:39.115+0000 7f4f63000640 1 -- 172.21.15.152:0/1960182960 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f4f5c08c680 con 0x7f4f5c0d35a0 2024-04-20T11:34:39.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:39.641+0000 7f4f59600640 1 -- 172.21.15.152:0/1960182960 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v76) ==== 122+0+0 (secure 0 0 0) 0x7f4f500182c0 con 0x7f4f5c0d35a0 2024-04-20T11:34:39.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:39.645+0000 7f4f63000640 1 -- 172.21.15.152:0/1960182960 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4f3c06cd10 msgr2=0x7f4f3c06f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:39.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:39.645+0000 7f4f63000640 1 --2- 172.21.15.152:0/1960182960 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4f3c06cd10 0x7f4f3c06f1d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c0b9de0 tx=0x7f4f48007910 comp rx=0 tx=0).stop 2024-04-20T11:34:39.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:39.645+0000 7f4f63000640 1 -- 172.21.15.152:0/1960182960 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4f5c0d35a0 msgr2=0x7f4f5c0b8180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:39.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:39.645+0000 7f4f63000640 1 --2- 172.21.15.152:0/1960182960 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4f5c0d35a0 0x7f4f5c0b8180 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f4f5000c460 tx=0x7f4f5000c920 comp rx=0 tx=0).stop 2024-04-20T11:34:39.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:39.645+0000 7f4f63000640 1 -- 172.21.15.152:0/1960182960 shutdown_connections 2024-04-20T11:34:39.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:39.645+0000 7f4f63000640 1 --2- 172.21.15.152:0/1960182960 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4f3c06cd10 0x7f4f3c06f1d0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:39.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:39.645+0000 7f4f63000640 1 --2- 172.21.15.152:0/1960182960 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4f5c163230 0x7f4f5c143120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:39.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:39.645+0000 7f4f63000640 1 --2- 172.21.15.152:0/1960182960 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4f5c15eb50 0x7f4f5c0b4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:39.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:39.645+0000 7f4f63000640 1 --2- 172.21.15.152:0/1960182960 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4f5c0d35a0 0x7f4f5c0b8180 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:39.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:39.645+0000 7f4f63000640 1 -- 172.21.15.152:0/1960182960 >> 172.21.15.152:0/1960182960 conn(0x7f4f5c08bfd0 msgr2=0x7f4f5c0d02c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:39.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:39.645+0000 7f4f63000640 1 -- 172.21.15.152:0/1960182960 shutdown_connections 2024-04-20T11:34:39.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:39.646+0000 7f4f63000640 1 -- 172.21.15.152:0/1960182960 wait complete. 2024-04-20T11:34:39.667 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-20T11:34:39.667 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:34:39.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:39.872+0000 7f0486800640 1 Processor -- start 2024-04-20T11:34:39.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:39.872+0000 7f0486800640 1 -- start start 2024-04-20T11:34:39.873 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:39.874+0000 7f0486800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f0480160b90 0x7f048015d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:39.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:39.874+0000 7f0486800640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04800ce8c0 con 0x7f0480071cd0 2024-04-20T11:34:39.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:39.874+0000 7f0486800640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04801600b0 con 0x7f048015d1e0 2024-04-20T11:34:39.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:39.874+0000 7f0486800640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f0480160390 con 0x7f0480160b90 2024-04-20T11:34:39.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:39.874+0000 7f0484a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f0480160b90 0x7f048015d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:39.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:39.874+0000 7f0485e00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f0480071cd0 0x7f0480072120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:41170/0 (socket says 172.21.15.152:41170) 2024-04-20T11:34:39.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:39.874+0000 7f0485e00640 1 -- 172.21.15.152:0/2153129987 learned_addr learned my addr 172.21.15.152:0/2153129987 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:34:39.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:39.875+0000 7f0484a00640 1 -- 172.21.15.152:0/2153129987 >> v1:172.21.15.175:6789/0 conn(0x7f048015d1e0 legacy=0x7f04800cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:39.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:39.875+0000 7f0484a00640 1 -- 172.21.15.152:0/2153129987 >> v1:172.21.15.152:6789/0 conn(0x7f0480071cd0 legacy=0x7f0480072120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:39.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:39.875+0000 7f0484a00640 1 -- 172.21.15.152:0/2153129987 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0480160670 con 0x7f0480160b90 2024-04-20T11:34:39.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:39.876+0000 7f0484a00640 1 --2- 172.21.15.152:0/2153129987 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0480160b90 0x7f048015d5c0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f046c0097d0 tx=0x7f046c02b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=254b2781bcf1597c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:39.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:39.876+0000 7f047fe00640 1 -- 172.21.15.152:0/2153129987 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f046c035020 con 0x7f0480160b90 2024-04-20T11:34:39.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:39.876+0000 7f047fe00640 1 -- 172.21.15.152:0/2153129987 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f046c031810 con 0x7f0480160b90 2024-04-20T11:34:39.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:39.876+0000 7f047fe00640 1 -- 172.21.15.152:0/2153129987 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f046c031be0 con 0x7f0480160b90 2024-04-20T11:34:39.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:39.877+0000 7f0486800640 1 -- 172.21.15.152:0/2153129987 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0480160b90 msgr2=0x7f048015d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:39.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:39.877+0000 7f0486800640 1 --2- 172.21.15.152:0/2153129987 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0480160b90 0x7f048015d5c0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f046c0097d0 tx=0x7f046c02b8b0 comp rx=0 tx=0).stop 2024-04-20T11:34:39.877 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:39.877+0000 7f0486800640 1 -- 172.21.15.152:0/2153129987 shutdown_connections 2024-04-20T11:34:39.877 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:39.877+0000 7f0486800640 1 --2- 172.21.15.152:0/2153129987 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0480160b90 0x7f048015d5c0 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:39.877 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:39.877+0000 7f0486800640 1 -- 172.21.15.152:0/2153129987 >> 172.21.15.152:0/2153129987 conn(0x7f0480067ae0 msgr2=0x7f048006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:39.877 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:39.877+0000 7f0486800640 1 -- 172.21.15.152:0/2153129987 shutdown_connections 2024-04-20T11:34:39.877 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:39.877+0000 7f0486800640 1 -- 172.21.15.152:0/2153129987 wait complete. 2024-04-20T11:34:39.878 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:39.878+0000 7f0486800640 1 Processor -- start 2024-04-20T11:34:39.878 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:39.879+0000 7f0486800640 1 -- start start 2024-04-20T11:34:39.879 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:39.879+0000 7f0486800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0480071cd0 0x7f0480156980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:39.879 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:39.879+0000 7f0486800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f048015d1e0 0x7f0480159860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:39.879 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:39.879+0000 7f0486800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0480160b90 0x7f0480144320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:39.879 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:39.879+0000 7f0486800640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f04800ccef0 con 0x7f0480071cd0 2024-04-20T11:34:39.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:39.879+0000 7f0486800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f04800ccd70 con 0x7f0480160b90 2024-04-20T11:34:39.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:39.879+0000 7f0486800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f04800cd070 con 0x7f048015d1e0 2024-04-20T11:34:39.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:39.880+0000 7f0485400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0480071cd0 0x7f0480156980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:39.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:39.880+0000 7f0485400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0480071cd0 0x7f0480156980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:50802/0 (socket says 172.21.15.152:50802) 2024-04-20T11:34:39.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:39.880+0000 7f0485400640 1 -- 172.21.15.152:0/2766188160 learned_addr learned my addr 172.21.15.152:0/2766188160 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:39.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:39.880+0000 7f0485400640 1 -- 172.21.15.152:0/2766188160 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f048015d1e0 msgr2=0x7f0480159860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:39.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:39.880+0000 7f0485400640 1 --2- 172.21.15.152:0/2766188160 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f048015d1e0 0x7f0480159860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:39.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:39.880+0000 7f0485400640 1 -- 172.21.15.152:0/2766188160 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0480160b90 msgr2=0x7f0480144320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:39.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:39.880+0000 7f0485400640 1 --2- 172.21.15.152:0/2766188160 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0480160b90 0x7f0480144320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:39.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:39.880+0000 7f0485400640 1 -- 172.21.15.152:0/2766188160 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f048015f570 con 0x7f0480071cd0 2024-04-20T11:34:39.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:39.881+0000 7f0485400640 1 --2- 172.21.15.152:0/2766188160 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0480071cd0 0x7f0480156980 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f047400c430 tx=0x7f047400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:39.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:39.881+0000 7f047d600640 1 -- 172.21.15.152:0/2766188160 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0474011790 con 0x7f0480071cd0 2024-04-20T11:34:39.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:39.881+0000 7f047d600640 1 -- 172.21.15.152:0/2766188160 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0474017070 con 0x7f0480071cd0 2024-04-20T11:34:39.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:39.881+0000 7f047d600640 1 -- 172.21.15.152:0/2766188160 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0474011bb0 con 0x7f0480071cd0 2024-04-20T11:34:39.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:39.881+0000 7f0486800640 1 -- 172.21.15.152:0/2766188160 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f048015e7b0 con 0x7f0480071cd0 2024-04-20T11:34:39.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:39.881+0000 7f0486800640 1 -- 172.21.15.152:0/2766188160 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f048015ed70 con 0x7f0480071cd0 2024-04-20T11:34:39.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:39.882+0000 7f0486800640 1 -- 172.21.15.152:0/2766188160 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0480075e60 con 0x7f0480071cd0 2024-04-20T11:34:39.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:39.883+0000 7f047d600640 1 -- 172.21.15.152:0/2766188160 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 19) ==== 91397+0+0 (secure 0 0 0) 0x7f047401c460 con 0x7f0480071cd0 2024-04-20T11:34:39.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:39.884+0000 7f047d600640 1 --2- 172.21.15.152:0/2766188160 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f046006cd10 0x7f046006f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:39.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:39.885+0000 7f047d600640 1 -- 172.21.15.152:0/2766188160 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5709+0+0 (secure 0 0 0) 0x7f047408c150 con 0x7f0480071cd0 2024-04-20T11:34:39.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:39.885+0000 7f0484a00640 1 --2- 172.21.15.152:0/2766188160 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f046006cd10 0x7f046006f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:39.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:39.886+0000 7f0484a00640 1 --2- 172.21.15.152:0/2766188160 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f046006cd10 0x7f046006f1d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f046c036000 tx=0x7f046c030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:39.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:39.890+0000 7f047d600640 1 -- 172.21.15.152:0/2766188160 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f0474059f60 con 0x7f0480071cd0 2024-04-20T11:34:40.128 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.129+0000 7f0486800640 1 -- 172.21.15.152:0/2766188160 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f04800b1e00 con 0x7f0480071cd0 2024-04-20T11:34:40.129 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.130+0000 7f047d600640 1 -- 172.21.15.152:0/2766188160 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) ==== 94+0+5207 (secure 0 0 0) 0x7f0474059860 con 0x7f0480071cd0 2024-04-20T11:34:40.129 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:34:40.130 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":76,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5891,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/508638056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6833","nonce":508638056},{"type":"v1","addr":"172.21.15.175:6835","nonce":508638056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":6448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6834/3447077466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3447077466},{"type":"v1","addr":"172.21.15.175:6834","nonce":3447077466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":8409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6834/751442105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":751442105},{"type":"v1","addr":"172.21.15.152:6834","nonce":751442105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T11:34:36.050950+0000","modified":"2024-04-20T11:34:39.117003+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8412},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8412":{"gid":8412,"name":"c","rank":0,"incarnation":72,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6835/2477488828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6833","nonce":2477488828},{"type":"v1","addr":"172.21.15.152:6835","nonce":2477488828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8412,"qdb_cluster":[8412]},"id":9}]} 2024-04-20T11:34:40.130 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 76 2024-04-20T11:34:40.132 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.132+0000 7f0486800640 1 -- 172.21.15.152:0/2766188160 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f046006cd10 msgr2=0x7f046006f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:40.132 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.132+0000 7f0486800640 1 --2- 172.21.15.152:0/2766188160 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f046006cd10 0x7f046006f1d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f046c036000 tx=0x7f046c030000 comp rx=0 tx=0).stop 2024-04-20T11:34:40.132 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.132+0000 7f0486800640 1 -- 172.21.15.152:0/2766188160 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0480071cd0 msgr2=0x7f0480156980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:40.132 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.132+0000 7f0486800640 1 --2- 172.21.15.152:0/2766188160 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0480071cd0 0x7f0480156980 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f047400c430 tx=0x7f047400c8f0 comp rx=0 tx=0).stop 2024-04-20T11:34:40.132 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.132+0000 7f0486800640 1 -- 172.21.15.152:0/2766188160 shutdown_connections 2024-04-20T11:34:40.132 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.132+0000 7f0486800640 1 --2- 172.21.15.152:0/2766188160 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f046006cd10 0x7f046006f1d0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:40.132 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.132+0000 7f0486800640 1 --2- 172.21.15.152:0/2766188160 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0480160b90 0x7f0480144320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:40.132 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.132+0000 7f0486800640 1 --2- 172.21.15.152:0/2766188160 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f048015d1e0 0x7f0480159860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:40.132 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.132+0000 7f0486800640 1 --2- 172.21.15.152:0/2766188160 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0480071cd0 0x7f0480156980 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:40.133 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.132+0000 7f0486800640 1 -- 172.21.15.152:0/2766188160 >> 172.21.15.152:0/2766188160 conn(0x7f0480067ae0 msgr2=0x7f04800aec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:40.133 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.132+0000 7f0486800640 1 -- 172.21.15.152:0/2766188160 shutdown_connections 2024-04-20T11:34:40.133 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.133+0000 7f0486800640 1 -- 172.21.15.152:0/2766188160 wait complete. 2024-04-20T11:34:40.150 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:34:40.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.352+0000 7f103b200640 1 Processor -- start 2024-04-20T11:34:40.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.352+0000 7f103b200640 1 -- start start 2024-04-20T11:34:40.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.353+0000 7f103b200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f103c13e530 0x7f103c06ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:40.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.353+0000 7f103b200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f103c1414e0 0x7f103c071cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:40.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.353+0000 7f103b200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f103c0cf0a0 con 0x7f103c06f340 2024-04-20T11:34:40.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.353+0000 7f103b200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f103c141b00 con 0x7f103c13e530 2024-04-20T11:34:40.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.353+0000 7f103b200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f103c0cfc50 con 0x7f103c1414e0 2024-04-20T11:34:40.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.354+0000 7f1039e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f103c13e530 0x7f103c06ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:40.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.354+0000 7f1039e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f103c13e530 0x7f103c06ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:50808/0 (socket says 172.21.15.152:50808) 2024-04-20T11:34:40.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.354+0000 7f1039e00640 1 -- 172.21.15.152:0/4063405295 learned_addr learned my addr 172.21.15.152:0/4063405295 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:40.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.354+0000 7f103a800640 1 --2- 172.21.15.152:0/4063405295 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f103c1414e0 0x7f103c071cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:40.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.354+0000 7f1039e00640 1 -- 172.21.15.152:0/4063405295 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f103c1414e0 msgr2=0x7f103c071cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:40.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.354+0000 7f1039e00640 1 --2- 172.21.15.152:0/4063405295 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f103c1414e0 0x7f103c071cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:40.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.354+0000 7f1039e00640 1 -- 172.21.15.152:0/4063405295 >> v1:172.21.15.175:6789/0 conn(0x7f103c06f340 legacy=0x7f103c140dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:40.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.354+0000 7f1039e00640 1 -- 172.21.15.152:0/4063405295 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f103c0d00e0 con 0x7f103c13e530 2024-04-20T11:34:40.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.354+0000 7f103a800640 1 --2- 172.21.15.152:0/4063405295 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f103c1414e0 0x7f103c071cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:34:40.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.355+0000 7f1039e00640 1 --2- 172.21.15.152:0/4063405295 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f103c13e530 0x7f103c06ee00 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f102c00bb10 tx=0x7f102c02e0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38e14309b66c3c89 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:40.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.356+0000 7f1039400640 1 -- 172.21.15.152:0/4063405295 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f102c03b070 con 0x7f103c13e530 2024-04-20T11:34:40.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.356+0000 7f1039400640 1 -- 172.21.15.152:0/4063405295 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f102c036440 con 0x7f103c13e530 2024-04-20T11:34:40.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.356+0000 7f1039400640 1 -- 172.21.15.152:0/4063405295 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f102c036810 con 0x7f103c13e530 2024-04-20T11:34:40.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.356+0000 7f103b200640 1 -- 172.21.15.152:0/4063405295 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f103c13e530 msgr2=0x7f103c06ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:40.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.356+0000 7f103b200640 1 --2- 172.21.15.152:0/4063405295 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f103c13e530 0x7f103c06ee00 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f102c00bb10 tx=0x7f102c02e0c0 comp rx=0 tx=0).stop 2024-04-20T11:34:40.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.357+0000 7f103b200640 1 -- 172.21.15.152:0/4063405295 shutdown_connections 2024-04-20T11:34:40.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.357+0000 7f103b200640 1 --2- 172.21.15.152:0/4063405295 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f103c1414e0 0x7f103c071cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:40.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.357+0000 7f103b200640 1 --2- 172.21.15.152:0/4063405295 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f103c13e530 0x7f103c06ee00 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:40.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.357+0000 7f103b200640 1 -- 172.21.15.152:0/4063405295 >> 172.21.15.152:0/4063405295 conn(0x7f103c067ae0 msgr2=0x7f103c06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:40.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.357+0000 7f103b200640 1 -- 172.21.15.152:0/4063405295 shutdown_connections 2024-04-20T11:34:40.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.357+0000 7f103b200640 1 -- 172.21.15.152:0/4063405295 wait complete. 2024-04-20T11:34:40.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.358+0000 7f103b200640 1 Processor -- start 2024-04-20T11:34:40.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.358+0000 7f103b200640 1 -- start start 2024-04-20T11:34:40.359 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.359+0000 7f103b200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f103c06f340 0x7f103c13db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:40.359 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.359+0000 7f103b200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f103c13e530 0x7f103c1e4460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:40.359 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.359+0000 7f103b200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f103c1414e0 0x7f103c0b1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:40.359 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.359+0000 7f103b200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f103c0cf8e0 con 0x7f103c13e530 2024-04-20T11:34:40.359 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.359+0000 7f103b200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f103c0cf600 con 0x7f103c06f340 2024-04-20T11:34:40.359 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.359+0000 7f103b200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f103c072210 con 0x7f103c1414e0 2024-04-20T11:34:40.359 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.360+0000 7f1033e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f103c13e530 0x7f103c1e4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:40.359 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.360+0000 7f1033e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f103c13e530 0x7f103c1e4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:50820/0 (socket says 172.21.15.152:50820) 2024-04-20T11:34:40.359 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.360+0000 7f1033e00640 1 -- 172.21.15.152:0/1968356825 learned_addr learned my addr 172.21.15.152:0/1968356825 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:40.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.360+0000 7f1039e00640 1 --2- 172.21.15.152:0/1968356825 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f103c06f340 0x7f103c13db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:40.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.360+0000 7f1033e00640 1 -- 172.21.15.152:0/1968356825 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f103c1414e0 msgr2=0x7f103c0b1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:40.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.360+0000 7f1033e00640 1 --2- 172.21.15.152:0/1968356825 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f103c1414e0 0x7f103c0b1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:40.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.360+0000 7f1033e00640 1 -- 172.21.15.152:0/1968356825 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f103c06f340 msgr2=0x7f103c13db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:40.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.360+0000 7f1033e00640 1 --2- 172.21.15.152:0/1968356825 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f103c06f340 0x7f103c13db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:40.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.360+0000 7f1033e00640 1 -- 172.21.15.152:0/1968356825 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f103c14ee90 con 0x7f103c13e530 2024-04-20T11:34:40.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.360+0000 7f1033e00640 1 --2- 172.21.15.152:0/1968356825 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f103c13e530 0x7f103c1e4460 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f102400c430 tx=0x7f102400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:40.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.361+0000 7f1032000640 1 -- 172.21.15.152:0/1968356825 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1024011790 con 0x7f103c13e530 2024-04-20T11:34:40.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.361+0000 7f1032000640 1 -- 172.21.15.152:0/1968356825 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1024017070 con 0x7f103c13e530 2024-04-20T11:34:40.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.361+0000 7f1032000640 1 -- 172.21.15.152:0/1968356825 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1024011bb0 con 0x7f103c13e530 2024-04-20T11:34:40.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.361+0000 7f103b200640 1 -- 172.21.15.152:0/1968356825 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f103c0b28a0 con 0x7f103c13e530 2024-04-20T11:34:40.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.361+0000 7f103b200640 1 -- 172.21.15.152:0/1968356825 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f103c13e070 con 0x7f103c13e530 2024-04-20T11:34:40.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.362+0000 7f103b200640 1 -- 172.21.15.152:0/1968356825 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f103c0b2620 con 0x7f103c13e530 2024-04-20T11:34:40.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.363+0000 7f1032000640 1 -- 172.21.15.152:0/1968356825 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 19) ==== 91397+0+0 (secure 0 0 0) 0x7f102401c460 con 0x7f103c13e530 2024-04-20T11:34:40.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.364+0000 7f1032000640 1 --2- 172.21.15.152:0/1968356825 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f101806cce0 0x7f101806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:40.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.364+0000 7f1032000640 1 -- 172.21.15.152:0/1968356825 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5709+0+0 (secure 0 0 0) 0x7f102408c130 con 0x7f103c13e530 2024-04-20T11:34:40.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.365+0000 7f1039e00640 1 --2- 172.21.15.152:0/1968356825 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f101806cce0 0x7f101806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:40.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.366+0000 7f1039e00640 1 --2- 172.21.15.152:0/1968356825 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f101806cce0 0x7f101806f1a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f102c038000 tx=0x7f102c032000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:40.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.368+0000 7f1032000640 1 -- 172.21.15.152:0/1968356825 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f103c0b2620 con 0x7f103c13e530 2024-04-20T11:34:40.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.570+0000 7f103b200640 1 -- 172.21.15.152:0/1968356825 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f103c075e60 con 0x7f103c13e530 2024-04-20T11:34:40.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.572+0000 7f1032000640 1 -- 172.21.15.152:0/1968356825 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v118) ==== 74+0+16739 (secure 0 0 0) 0x7f102405a010 con 0x7f103c13e530 2024-04-20T11:34:40.572 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:34:40.572 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":118,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:34:37.584133+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:34:33.781054+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-04-20T11:34:35.031728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:34:40.574 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.575+0000 7f103b200640 1 -- 172.21.15.152:0/1968356825 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f101806cce0 msgr2=0x7f101806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:40.574 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.575+0000 7f103b200640 1 --2- 172.21.15.152:0/1968356825 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f101806cce0 0x7f101806f1a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f102c038000 tx=0x7f102c032000 comp rx=0 tx=0).stop 2024-04-20T11:34:40.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.575+0000 7f103b200640 1 -- 172.21.15.152:0/1968356825 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f103c13e530 msgr2=0x7f103c1e4460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:40.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.575+0000 7f103b200640 1 --2- 172.21.15.152:0/1968356825 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f103c13e530 0x7f103c1e4460 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f102400c430 tx=0x7f102400c8f0 comp rx=0 tx=0).stop 2024-04-20T11:34:40.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.575+0000 7f103b200640 1 -- 172.21.15.152:0/1968356825 shutdown_connections 2024-04-20T11:34:40.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.575+0000 7f103b200640 1 --2- 172.21.15.152:0/1968356825 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f101806cce0 0x7f101806f1a0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:40.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.575+0000 7f103b200640 1 --2- 172.21.15.152:0/1968356825 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f103c1414e0 0x7f103c0b1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:40.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.575+0000 7f103b200640 1 --2- 172.21.15.152:0/1968356825 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f103c13e530 0x7f103c1e4460 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:40.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.575+0000 7f103b200640 1 --2- 172.21.15.152:0/1968356825 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f103c06f340 0x7f103c13db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:40.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.575+0000 7f103b200640 1 -- 172.21.15.152:0/1968356825 >> 172.21.15.152:0/1968356825 conn(0x7f103c067ae0 msgr2=0x7f103c0b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:40.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.575+0000 7f103b200640 1 -- 172.21.15.152:0/1968356825 shutdown_connections 2024-04-20T11:34:40.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.575+0000 7f103b200640 1 -- 172.21.15.152:0/1968356825 wait complete. 2024-04-20T11:34:40.593 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-20T11:34:40.593 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T11:34:40.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.788+0000 7f63a5200640 1 Processor -- start 2024-04-20T11:34:40.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.788+0000 7f63a5200640 1 -- start start 2024-04-20T11:34:40.789 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.789+0000 7f63a5200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f63a0071cd0 0x7f63a00720b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:40.789 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.789+0000 7f63a5200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f63a006ea20 0x7f63a00b4a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:40.789 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.790+0000 7f63a5200640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63a00725f0 con 0x7f63a00b4f90 2024-04-20T11:34:40.789 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.790+0000 7f63a5200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f63a006dcf0 con 0x7f63a0071cd0 2024-04-20T11:34:40.789 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.790+0000 7f63a5200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f63a00b5510 con 0x7f63a006ea20 2024-04-20T11:34:40.790 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.790+0000 7f639f400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f63a0071cd0 0x7f63a00720b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:40.790 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.790+0000 7f639fe00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f63a00b4f90 0x7f63a00cdeb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:41180/0 (socket says 172.21.15.152:41180) 2024-04-20T11:34:40.790 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.790+0000 7f639fe00640 1 -- 172.21.15.152:0/168799189 learned_addr learned my addr 172.21.15.152:0/168799189 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:34:40.790 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.790+0000 7f639ea00640 1 --2- 172.21.15.152:0/168799189 >> v2:172.21.15.175:3300/0 conn(0x7f63a006ea20 0x7f63a00b4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:40.790 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.790+0000 7f639e000640 1 -- 172.21.15.152:0/168799189 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3522070188 0 0) 0x7f63a00725f0 con 0x7f63a00b4f90 2024-04-20T11:34:40.791 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.790+0000 7f639f400640 1 -- 172.21.15.152:0/168799189 >> v2:172.21.15.175:3300/0 conn(0x7f63a006ea20 msgr2=0x7f63a00b4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:40.791 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.790+0000 7f639f400640 1 --2- 172.21.15.152:0/168799189 >> v2:172.21.15.175:3300/0 conn(0x7f63a006ea20 0x7f63a00b4a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:40.791 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.790+0000 7f639f400640 1 -- 172.21.15.152:0/168799189 >> v1:172.21.15.152:6789/0 conn(0x7f63a00b4f90 legacy=0x7f63a00cdeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:40.791 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.790+0000 7f639f400640 1 -- 172.21.15.152:0/168799189 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63a006df40 con 0x7f63a0071cd0 2024-04-20T11:34:40.792 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.792+0000 7f639f400640 1 --2- 172.21.15.152:0/168799189 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f63a0071cd0 0x7f63a00720b0 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f63a01641d0 tx=0x7f639002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18df94ea19d524ce server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:40.792 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.792+0000 7f639e000640 1 -- 172.21.15.152:0/168799189 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6390039070 con 0x7f63a0071cd0 2024-04-20T11:34:40.792 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.792+0000 7f639e000640 1 -- 172.21.15.152:0/168799189 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f639002cc80 con 0x7f63a0071cd0 2024-04-20T11:34:40.792 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.792+0000 7f639e000640 1 -- 172.21.15.152:0/168799189 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6390034660 con 0x7f63a0071cd0 2024-04-20T11:34:40.792 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.792+0000 7f63a5200640 1 -- 172.21.15.152:0/168799189 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f63a0071cd0 msgr2=0x7f63a00720b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:40.792 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.792+0000 7f63a5200640 1 --2- 172.21.15.152:0/168799189 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f63a0071cd0 0x7f63a00720b0 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f63a01641d0 tx=0x7f639002bed0 comp rx=0 tx=0).stop 2024-04-20T11:34:40.792 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.792+0000 7f63a5200640 1 -- 172.21.15.152:0/168799189 shutdown_connections 2024-04-20T11:34:40.792 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.792+0000 7f63a5200640 1 --2- 172.21.15.152:0/168799189 >> v2:172.21.15.175:3300/0 conn(0x7f63a006ea20 0x7f63a00b4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:40.793 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.792+0000 7f63a5200640 1 --2- 172.21.15.152:0/168799189 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f63a0071cd0 0x7f63a00720b0 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:40.793 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.792+0000 7f63a5200640 1 -- 172.21.15.152:0/168799189 >> 172.21.15.152:0/168799189 conn(0x7f63a0067ae0 msgr2=0x7f63a006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:40.793 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.793+0000 7f63a5200640 1 -- 172.21.15.152:0/168799189 shutdown_connections 2024-04-20T11:34:40.793 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.793+0000 7f63a5200640 1 -- 172.21.15.152:0/168799189 wait complete. 2024-04-20T11:34:40.793 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.794+0000 7f63a5200640 1 Processor -- start 2024-04-20T11:34:40.794 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.794+0000 7f63a5200640 1 -- start start 2024-04-20T11:34:40.795 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.795+0000 7f63a5200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f63a006ea20 0x7f63a01595c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:40.795 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.795+0000 7f63a5200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f63a0071cd0 0x7f63a0165160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:40.795 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.795+0000 7f63a5200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63a00b4f90 0x7f63a01656a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:40.795 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.795+0000 7f63a5200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f63a0163e60 con 0x7f63a0071cd0 2024-04-20T11:34:40.795 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.795+0000 7f63a5200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f63a006e500 con 0x7f63a00b4f90 2024-04-20T11:34:40.795 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.795+0000 7f63a5200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f63a00b4180 con 0x7f63a006ea20 2024-04-20T11:34:40.795 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.795+0000 7f639ea00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f63a0071cd0 0x7f63a0165160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:40.795 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.795+0000 7f639ea00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f63a0071cd0 0x7f63a0165160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:50846/0 (socket says 172.21.15.152:50846) 2024-04-20T11:34:40.795 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.795+0000 7f639ea00640 1 -- 172.21.15.152:0/3297268195 learned_addr learned my addr 172.21.15.152:0/3297268195 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:40.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.796+0000 7f639ea00640 1 -- 172.21.15.152:0/3297268195 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f63a006ea20 msgr2=0x7f63a01595c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:40.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.796+0000 7f639ea00640 1 --2- 172.21.15.152:0/3297268195 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f63a006ea20 0x7f63a01595c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:40.797 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.796+0000 7f639ea00640 1 -- 172.21.15.152:0/3297268195 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63a00b4f90 msgr2=0x7f63a01656a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:40.797 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.796+0000 7f639ea00640 1 --2- 172.21.15.152:0/3297268195 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63a00b4f90 0x7f63a01656a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:40.797 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.796+0000 7f639ea00640 1 -- 172.21.15.152:0/3297268195 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63a0159b00 con 0x7f63a0071cd0 2024-04-20T11:34:40.797 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.796+0000 7f639ea00640 1 --2- 172.21.15.152:0/3297268195 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f63a0071cd0 0x7f63a0165160 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f638c00d800 tx=0x7f638c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:40.797 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.796+0000 7f6397400640 1 -- 172.21.15.152:0/3297268195 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f638c00b6c0 con 0x7f63a0071cd0 2024-04-20T11:34:40.797 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.796+0000 7f6397400640 1 -- 172.21.15.152:0/3297268195 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f638c019070 con 0x7f63a0071cd0 2024-04-20T11:34:40.797 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.796+0000 7f6397400640 1 -- 172.21.15.152:0/3297268195 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f638c00ba80 con 0x7f63a0071cd0 2024-04-20T11:34:40.797 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.796+0000 7f63a5200640 1 -- 172.21.15.152:0/3297268195 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63a01f8610 con 0x7f63a0071cd0 2024-04-20T11:34:40.797 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.797+0000 7f63a5200640 1 -- 172.21.15.152:0/3297268195 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63a015ae60 con 0x7f63a0071cd0 2024-04-20T11:34:40.797 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.798+0000 7f63a5200640 1 -- 172.21.15.152:0/3297268195 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63a0075e60 con 0x7f63a0071cd0 2024-04-20T11:34:40.798 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.799+0000 7f6397400640 1 -- 172.21.15.152:0/3297268195 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 19) ==== 91397+0+0 (secure 0 0 0) 0x7f638c00bc20 con 0x7f63a0071cd0 2024-04-20T11:34:40.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.800+0000 7f6397400640 1 --2- 172.21.15.152:0/3297268195 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f638006cce0 0x7f638006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:40.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.800+0000 7f6397400640 1 -- 172.21.15.152:0/3297268195 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5709+0+0 (secure 0 0 0) 0x7f638c08c6f0 con 0x7f63a0071cd0 2024-04-20T11:34:40.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.800+0000 7f639f400640 1 --2- 172.21.15.152:0/3297268195 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f638006cce0 0x7f638006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:40.801 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.801+0000 7f639f400640 1 --2- 172.21.15.152:0/3297268195 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f638006cce0 0x7f638006f1a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f6390036000 tx=0x7f639002e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:40.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.804+0000 7f6397400640 1 -- 172.21.15.152:0/3297268195 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f638c05a5f0 con 0x7f63a0071cd0 2024-04-20T11:34:40.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:40.993+0000 7f63a5200640 1 -- 172.21.15.152:0/3297268195 --> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f63a0160000 con 0x7f638006cce0 2024-04-20T11:34:41.011 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.011+0000 7f6397400640 1 -- 172.21.15.152:0/3297268195 <== mgr.4109 v2:172.21.15.175:6836/4021250441 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252466 (secure 0 0 0) 0x7f63a0160000 con 0x7f638006cce0 2024-04-20T11:34:41.014 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:34:41.017 INFO:teuthology.orchestra.run.smithi152.stderr:dumped all 2024-04-20T11:34:41.018 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.017+0000 7f63a5200640 1 -- 172.21.15.152:0/3297268195 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f638006cce0 msgr2=0x7f638006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:41.018 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.017+0000 7f63a5200640 1 --2- 172.21.15.152:0/3297268195 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f638006cce0 0x7f638006f1a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f6390036000 tx=0x7f639002e000 comp rx=0 tx=0).stop 2024-04-20T11:34:41.018 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.018+0000 7f63a5200640 1 -- 172.21.15.152:0/3297268195 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f63a0071cd0 msgr2=0x7f63a0165160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:41.018 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.018+0000 7f63a5200640 1 --2- 172.21.15.152:0/3297268195 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f63a0071cd0 0x7f63a0165160 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f638c00d800 tx=0x7f638c00dcc0 comp rx=0 tx=0).stop 2024-04-20T11:34:41.018 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.018+0000 7f63a5200640 1 -- 172.21.15.152:0/3297268195 shutdown_connections 2024-04-20T11:34:41.018 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.018+0000 7f63a5200640 1 --2- 172.21.15.152:0/3297268195 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f638006cce0 0x7f638006f1a0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:41.019 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.018+0000 7f63a5200640 1 --2- 172.21.15.152:0/3297268195 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63a00b4f90 0x7f63a01656a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:41.019 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.018+0000 7f63a5200640 1 --2- 172.21.15.152:0/3297268195 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f63a0071cd0 0x7f63a0165160 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:41.019 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.018+0000 7f63a5200640 1 --2- 172.21.15.152:0/3297268195 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f63a006ea20 0x7f63a01595c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:41.019 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.018+0000 7f63a5200640 1 -- 172.21.15.152:0/3297268195 >> 172.21.15.152:0/3297268195 conn(0x7f63a0067ae0 msgr2=0x7f63a00b3180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:41.019 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.018+0000 7f63a5200640 1 -- 172.21.15.152:0/3297268195 shutdown_connections 2024-04-20T11:34:41.019 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.018+0000 7f63a5200640 1 -- 172.21.15.152:0/3297268195 wait complete. 2024-04-20T11:34:41.035 INFO:teuthology.orchestra.run.smithi152.stdout:{"pg_ready":true,"pg_map":{"version":119,"stamp":"2024-04-20T11:34:40.646272+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":223,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":166,"ondisk_log_size":166,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":140,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":262168,"kb_used_data":12696,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754712552,"statfs":{"total":773094113280,"available":772825653248,"internally_reserved":0,"allocated":13000704,"data_stored":6368108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12746,"internal_metadata":255315510},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001757"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:37.592773+0000","last_change":"2024-04-20T11:34:36.635642+0000","last_active":"2024-04-20T11:34:37.592773+0000","last_peered":"2024-04-20T11:34:37.592773+0000","last_clean":"2024-04-20T11:34:37.592773+0000","last_became_active":"2024-04-20T11:34:36.635300+0000","last_became_peered":"2024-04-20T11:34:36.635300+0000","last_unstale":"2024-04-20T11:34:37.592773+0000","last_undegraded":"2024-04-20T11:34:37.592773+0000","last_fullsized":"2024-04-20T11:34:37.592773+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:52:31.695928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:38.211244+0000","last_change":"2024-04-20T11:34:36.626518+0000","last_active":"2024-04-20T11:34:38.211244+0000","last_peered":"2024-04-20T11:34:38.211244+0000","last_clean":"2024-04-20T11:34:38.211244+0000","last_became_active":"2024-04-20T11:34:36.626011+0000","last_became_peered":"2024-04-20T11:34:36.626011+0000","last_unstale":"2024-04-20T11:34:38.211244+0000","last_undegraded":"2024-04-20T11:34:38.211244+0000","last_fullsized":"2024-04-20T11:34:38.211244+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:09:01.470522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-20T11:34:36.621117+0000","last_change":"2024-04-20T11:34:36.618158+0000","last_active":"2024-04-20T11:34:36.621117+0000","last_peered":"2024-04-20T11:34:36.621117+0000","last_clean":"2024-04-20T11:34:36.621117+0000","last_became_active":"2024-04-20T11:34:36.617752+0000","last_became_peered":"2024-04-20T11:34:36.617752+0000","last_unstale":"2024-04-20T11:34:36.621117+0000","last_undegraded":"2024-04-20T11:34:36.621117+0000","last_fullsized":"2024-04-20T11:34:36.621117+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:09:01.470522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:37.592633+0000","last_change":"2024-04-20T11:34:36.616117+0000","last_active":"2024-04-20T11:34:37.592633+0000","last_peered":"2024-04-20T11:34:37.592633+0000","last_clean":"2024-04-20T11:34:37.592633+0000","last_became_active":"2024-04-20T11:34:36.615824+0000","last_became_peered":"2024-04-20T11:34:36.615824+0000","last_unstale":"2024-04-20T11:34:37.592633+0000","last_undegraded":"2024-04-20T11:34:37.592633+0000","last_fullsized":"2024-04-20T11:34:37.592633+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:07:11.853057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:37.633031+0000","last_change":"2024-04-20T11:34:36.627663+0000","last_active":"2024-04-20T11:34:37.633031+0000","last_peered":"2024-04-20T11:34:37.633031+0000","last_clean":"2024-04-20T11:34:37.633031+0000","last_became_active":"2024-04-20T11:34:36.627006+0000","last_became_peered":"2024-04-20T11:34:36.627006+0000","last_unstale":"2024-04-20T11:34:37.633031+0000","last_undegraded":"2024-04-20T11:34:37.633031+0000","last_fullsized":"2024-04-20T11:34:37.633031+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:32:52.568340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:38.484235+0000","last_change":"2024-04-20T11:34:36.618445+0000","last_active":"2024-04-20T11:34:38.484235+0000","last_peered":"2024-04-20T11:34:38.484235+0000","last_clean":"2024-04-20T11:34:38.484235+0000","last_became_active":"2024-04-20T11:34:36.618118+0000","last_became_peered":"2024-04-20T11:34:36.618118+0000","last_unstale":"2024-04-20T11:34:38.484235+0000","last_undegraded":"2024-04-20T11:34:38.484235+0000","last_fullsized":"2024-04-20T11:34:38.484235+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:58:16.330485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:38.483876+0000","last_change":"2024-04-20T11:34:36.622278+0000","last_active":"2024-04-20T11:34:38.483876+0000","last_peered":"2024-04-20T11:34:38.483876+0000","last_clean":"2024-04-20T11:34:38.483876+0000","last_became_active":"2024-04-20T11:34:36.621928+0000","last_became_peered":"2024-04-20T11:34:36.621928+0000","last_unstale":"2024-04-20T11:34:38.483876+0000","last_undegraded":"2024-04-20T11:34:38.483876+0000","last_fullsized":"2024-04-20T11:34:38.483876+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:11:38.577132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-20T11:34:36.635428+0000","last_change":"2024-04-20T11:34:36.623959+0000","last_active":"2024-04-20T11:34:36.635428+0000","last_peered":"2024-04-20T11:34:36.635428+0000","last_clean":"2024-04-20T11:34:36.635428+0000","last_became_active":"2024-04-20T11:34:36.623378+0000","last_became_peered":"2024-04-20T11:34:36.623378+0000","last_unstale":"2024-04-20T11:34:36.635428+0000","last_undegraded":"2024-04-20T11:34:36.635428+0000","last_fullsized":"2024-04-20T11:34:36.635428+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:11:14.001030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-20T11:34:36.637430+0000","last_change":"2024-04-20T11:34:36.631864+0000","last_active":"2024-04-20T11:34:36.637430+0000","last_peered":"2024-04-20T11:34:36.637430+0000","last_clean":"2024-04-20T11:34:36.637430+0000","last_became_active":"2024-04-20T11:34:36.631579+0000","last_became_peered":"2024-04-20T11:34:36.631579+0000","last_unstale":"2024-04-20T11:34:36.637430+0000","last_undegraded":"2024-04-20T11:34:36.637430+0000","last_fullsized":"2024-04-20T11:34:36.637430+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:26:57.597691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:38.211671+0000","last_change":"2024-04-20T11:34:36.626289+0000","last_active":"2024-04-20T11:34:38.211671+0000","last_peered":"2024-04-20T11:34:38.211671+0000","last_clean":"2024-04-20T11:34:38.211671+0000","last_became_active":"2024-04-20T11:34:36.625628+0000","last_became_peered":"2024-04-20T11:34:36.625628+0000","last_unstale":"2024-04-20T11:34:38.211671+0000","last_undegraded":"2024-04-20T11:34:38.211671+0000","last_fullsized":"2024-04-20T11:34:38.211671+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:12:30.998570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-20T11:34:36.617104+0000","last_change":"2024-04-20T11:34:36.613031+0000","last_active":"2024-04-20T11:34:36.617104+0000","last_peered":"2024-04-20T11:34:36.617104+0000","last_clean":"2024-04-20T11:34:36.617104+0000","last_became_active":"2024-04-20T11:34:36.612719+0000","last_became_peered":"2024-04-20T11:34:36.612719+0000","last_unstale":"2024-04-20T11:34:36.617104+0000","last_undegraded":"2024-04-20T11:34:36.617104+0000","last_fullsized":"2024-04-20T11:34:36.617104+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:02:52.819367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-20T11:34:36.638413+0000","last_change":"2024-04-20T11:34:36.633778+0000","last_active":"2024-04-20T11:34:36.638413+0000","last_peered":"2024-04-20T11:34:36.638413+0000","last_clean":"2024-04-20T11:34:36.638413+0000","last_became_active":"2024-04-20T11:34:36.633447+0000","last_became_peered":"2024-04-20T11:34:36.633447+0000","last_unstale":"2024-04-20T11:34:36.638413+0000","last_undegraded":"2024-04-20T11:34:36.638413+0000","last_fullsized":"2024-04-20T11:34:36.638413+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:12:30.998570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:37.593962+0000","last_change":"2024-04-20T11:34:36.622688+0000","last_active":"2024-04-20T11:34:37.593962+0000","last_peered":"2024-04-20T11:34:37.593962+0000","last_clean":"2024-04-20T11:34:37.593962+0000","last_became_active":"2024-04-20T11:34:36.622093+0000","last_became_peered":"2024-04-20T11:34:36.622093+0000","last_unstale":"2024-04-20T11:34:37.593962+0000","last_undegraded":"2024-04-20T11:34:37.593962+0000","last_fullsized":"2024-04-20T11:34:37.593962+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:26:00.441310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:38.211615+0000","last_change":"2024-04-20T11:34:36.609394+0000","last_active":"2024-04-20T11:34:38.211615+0000","last_peered":"2024-04-20T11:34:38.211615+0000","last_clean":"2024-04-20T11:34:38.211615+0000","last_became_active":"2024-04-20T11:34:36.609113+0000","last_became_peered":"2024-04-20T11:34:36.609113+0000","last_unstale":"2024-04-20T11:34:38.211615+0000","last_undegraded":"2024-04-20T11:34:38.211615+0000","last_fullsized":"2024-04-20T11:34:38.211615+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:26:57.597691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:37.591825+0000","last_change":"2024-04-20T11:34:36.618030+0000","last_active":"2024-04-20T11:34:37.591825+0000","last_peered":"2024-04-20T11:34:37.591825+0000","last_clean":"2024-04-20T11:34:37.591825+0000","last_became_active":"2024-04-20T11:34:36.617557+0000","last_became_peered":"2024-04-20T11:34:36.617557+0000","last_unstale":"2024-04-20T11:34:37.591825+0000","last_undegraded":"2024-04-20T11:34:37.591825+0000","last_fullsized":"2024-04-20T11:34:37.591825+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:56:31.623540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:37.597187+0000","last_change":"2024-04-20T11:34:36.609197+0000","last_active":"2024-04-20T11:34:37.597187+0000","last_peered":"2024-04-20T11:34:37.597187+0000","last_clean":"2024-04-20T11:34:37.597187+0000","last_became_active":"2024-04-20T11:34:36.608876+0000","last_became_peered":"2024-04-20T11:34:36.608876+0000","last_unstale":"2024-04-20T11:34:37.597187+0000","last_undegraded":"2024-04-20T11:34:37.597187+0000","last_fullsized":"2024-04-20T11:34:37.597187+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:57:35.372036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:38.484512+0000","last_change":"2024-04-20T11:34:36.620346+0000","last_active":"2024-04-20T11:34:38.484512+0000","last_peered":"2024-04-20T11:34:38.484512+0000","last_clean":"2024-04-20T11:34:38.484512+0000","last_became_active":"2024-04-20T11:34:36.620020+0000","last_became_peered":"2024-04-20T11:34:36.620020+0000","last_unstale":"2024-04-20T11:34:38.484512+0000","last_undegraded":"2024-04-20T11:34:38.484512+0000","last_fullsized":"2024-04-20T11:34:38.484512+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:32:52.568340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:37.598062+0000","last_change":"2024-04-20T11:34:36.625093+0000","last_active":"2024-04-20T11:34:37.598062+0000","last_peered":"2024-04-20T11:34:37.598062+0000","last_clean":"2024-04-20T11:34:37.598062+0000","last_became_active":"2024-04-20T11:34:36.624797+0000","last_became_peered":"2024-04-20T11:34:36.624797+0000","last_unstale":"2024-04-20T11:34:37.598062+0000","last_undegraded":"2024-04-20T11:34:37.598062+0000","last_fullsized":"2024-04-20T11:34:37.598062+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:45:55.429475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-20T11:34:36.622175+0000","last_change":"2024-04-20T11:34:36.619604+0000","last_active":"2024-04-20T11:34:36.622175+0000","last_peered":"2024-04-20T11:34:36.622175+0000","last_clean":"2024-04-20T11:34:36.622175+0000","last_became_active":"2024-04-20T11:34:36.619335+0000","last_became_peered":"2024-04-20T11:34:36.619335+0000","last_unstale":"2024-04-20T11:34:36.622175+0000","last_undegraded":"2024-04-20T11:34:36.622175+0000","last_fullsized":"2024-04-20T11:34:36.622175+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:17:02.644047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:37.597946+0000","last_change":"2024-04-20T11:34:36.613325+0000","last_active":"2024-04-20T11:34:37.597946+0000","last_peered":"2024-04-20T11:34:37.597946+0000","last_clean":"2024-04-20T11:34:37.597946+0000","last_became_active":"2024-04-20T11:34:36.613039+0000","last_became_peered":"2024-04-20T11:34:36.613039+0000","last_unstale":"2024-04-20T11:34:37.597946+0000","last_undegraded":"2024-04-20T11:34:37.597946+0000","last_fullsized":"2024-04-20T11:34:37.597946+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:15:12.163320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:37.632489+0000","last_change":"2024-04-20T11:34:36.627409+0000","last_active":"2024-04-20T11:34:37.632489+0000","last_peered":"2024-04-20T11:34:37.632489+0000","last_clean":"2024-04-20T11:34:37.632489+0000","last_became_active":"2024-04-20T11:34:36.626614+0000","last_became_peered":"2024-04-20T11:34:36.626614+0000","last_unstale":"2024-04-20T11:34:37.632489+0000","last_undegraded":"2024-04-20T11:34:37.632489+0000","last_fullsized":"2024-04-20T11:34:37.632489+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:28:42.034218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-20T11:34:36.614609+0000","last_change":"2024-04-20T11:34:36.611348+0000","last_active":"2024-04-20T11:34:36.614609+0000","last_peered":"2024-04-20T11:34:36.614609+0000","last_clean":"2024-04-20T11:34:36.614609+0000","last_became_active":"2024-04-20T11:34:36.611017+0000","last_became_peered":"2024-04-20T11:34:36.611017+0000","last_unstale":"2024-04-20T11:34:36.614609+0000","last_undegraded":"2024-04-20T11:34:36.614609+0000","last_fullsized":"2024-04-20T11:34:36.614609+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:51:17.501994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-20T11:34:36.638539+0000","last_change":"2024-04-20T11:34:36.634391+0000","last_active":"2024-04-20T11:34:36.638539+0000","last_peered":"2024-04-20T11:34:36.638539+0000","last_clean":"2024-04-20T11:34:36.638539+0000","last_became_active":"2024-04-20T11:34:36.634010+0000","last_became_peered":"2024-04-20T11:34:36.634010+0000","last_unstale":"2024-04-20T11:34:36.638539+0000","last_undegraded":"2024-04-20T11:34:36.638539+0000","last_fullsized":"2024-04-20T11:34:36.638539+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:15:10.532889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-20T11:34:36.603001+0000","last_change":"2024-04-20T11:34:36.600729+0000","last_active":"2024-04-20T11:34:36.603001+0000","last_peered":"2024-04-20T11:34:36.603001+0000","last_clean":"2024-04-20T11:34:36.603001+0000","last_became_active":"2024-04-20T11:34:36.600432+0000","last_became_peered":"2024-04-20T11:34:36.600432+0000","last_unstale":"2024-04-20T11:34:36.603001+0000","last_undegraded":"2024-04-20T11:34:36.603001+0000","last_fullsized":"2024-04-20T11:34:36.603001+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:24:26.808424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:38.211448+0000","last_change":"2024-04-20T11:34:36.613944+0000","last_active":"2024-04-20T11:34:38.211448+0000","last_peered":"2024-04-20T11:34:38.211448+0000","last_clean":"2024-04-20T11:34:38.211448+0000","last_became_active":"2024-04-20T11:34:36.613668+0000","last_became_peered":"2024-04-20T11:34:36.613668+0000","last_unstale":"2024-04-20T11:34:38.211448+0000","last_undegraded":"2024-04-20T11:34:38.211448+0000","last_fullsized":"2024-04-20T11:34:38.211448+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:11:14.001030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:37.598182+0000","last_change":"2024-04-20T11:34:36.616892+0000","last_active":"2024-04-20T11:34:37.598182+0000","last_peered":"2024-04-20T11:34:37.598182+0000","last_clean":"2024-04-20T11:34:37.598182+0000","last_became_active":"2024-04-20T11:34:36.616534+0000","last_became_peered":"2024-04-20T11:34:36.616534+0000","last_unstale":"2024-04-20T11:34:37.598182+0000","last_undegraded":"2024-04-20T11:34:37.598182+0000","last_fullsized":"2024-04-20T11:34:37.598182+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:33:13.548986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:37.592770+0000","last_change":"2024-04-20T11:34:36.623551+0000","last_active":"2024-04-20T11:34:37.592770+0000","last_peered":"2024-04-20T11:34:37.592770+0000","last_clean":"2024-04-20T11:34:37.592770+0000","last_became_active":"2024-04-20T11:34:36.622433+0000","last_became_peered":"2024-04-20T11:34:36.622433+0000","last_unstale":"2024-04-20T11:34:37.592770+0000","last_undegraded":"2024-04-20T11:34:37.592770+0000","last_fullsized":"2024-04-20T11:34:37.592770+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:13:36.429019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:37.598052+0000","last_change":"2024-04-20T11:34:36.621406+0000","last_active":"2024-04-20T11:34:37.598052+0000","last_peered":"2024-04-20T11:34:37.598052+0000","last_clean":"2024-04-20T11:34:37.598052+0000","last_became_active":"2024-04-20T11:34:36.619272+0000","last_became_peered":"2024-04-20T11:34:36.619272+0000","last_unstale":"2024-04-20T11:34:37.598052+0000","last_undegraded":"2024-04-20T11:34:37.598052+0000","last_fullsized":"2024-04-20T11:34:37.598052+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:33:59.359931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:37.593901+0000","last_change":"2024-04-20T11:34:36.621651+0000","last_active":"2024-04-20T11:34:37.593901+0000","last_peered":"2024-04-20T11:34:37.593901+0000","last_clean":"2024-04-20T11:34:37.593901+0000","last_became_active":"2024-04-20T11:34:36.619380+0000","last_became_peered":"2024-04-20T11:34:36.619380+0000","last_unstale":"2024-04-20T11:34:37.593901+0000","last_undegraded":"2024-04-20T11:34:37.593901+0000","last_fullsized":"2024-04-20T11:34:37.593901+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:06:56.343485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:37.592502+0000","last_change":"2024-04-20T11:34:36.616643+0000","last_active":"2024-04-20T11:34:37.592502+0000","last_peered":"2024-04-20T11:34:37.592502+0000","last_clean":"2024-04-20T11:34:37.592502+0000","last_became_active":"2024-04-20T11:34:36.616355+0000","last_became_peered":"2024-04-20T11:34:36.616355+0000","last_unstale":"2024-04-20T11:34:37.592502+0000","last_undegraded":"2024-04-20T11:34:37.592502+0000","last_fullsized":"2024-04-20T11:34:37.592502+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:32:56.437813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:37.632467+0000","last_change":"2024-04-20T11:34:36.630349+0000","last_active":"2024-04-20T11:34:37.632467+0000","last_peered":"2024-04-20T11:34:37.632467+0000","last_clean":"2024-04-20T11:34:37.632467+0000","last_became_active":"2024-04-20T11:34:36.630031+0000","last_became_peered":"2024-04-20T11:34:36.630031+0000","last_unstale":"2024-04-20T11:34:37.632467+0000","last_undegraded":"2024-04-20T11:34:37.632467+0000","last_fullsized":"2024-04-20T11:34:37.632467+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:40:30.539389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:38.483777+0000","last_change":"2024-04-20T11:34:36.624413+0000","last_active":"2024-04-20T11:34:38.483777+0000","last_peered":"2024-04-20T11:34:38.483777+0000","last_clean":"2024-04-20T11:34:38.483777+0000","last_became_active":"2024-04-20T11:34:36.624046+0000","last_became_peered":"2024-04-20T11:34:36.624046+0000","last_unstale":"2024-04-20T11:34:38.483777+0000","last_undegraded":"2024-04-20T11:34:38.483777+0000","last_fullsized":"2024-04-20T11:34:38.483777+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:06:04.637607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:37.598381+0000","last_change":"2024-04-20T11:34:36.615431+0000","last_active":"2024-04-20T11:34:37.598381+0000","last_peered":"2024-04-20T11:34:37.598381+0000","last_clean":"2024-04-20T11:34:37.598381+0000","last_became_active":"2024-04-20T11:34:36.615180+0000","last_became_peered":"2024-04-20T11:34:36.615180+0000","last_unstale":"2024-04-20T11:34:37.598381+0000","last_undegraded":"2024-04-20T11:34:37.598381+0000","last_fullsized":"2024-04-20T11:34:37.598381+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:32:08.712923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"117'4","reported_seq":18,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:37.633862+0000","last_change":"2024-04-20T11:34:35.622313+0000","last_active":"2024-04-20T11:34:37.633862+0000","last_peered":"2024-04-20T11:34:37.633862+0000","last_clean":"2024-04-20T11:34:37.633862+0000","last_became_active":"2024-04-20T11:34:35.621973+0000","last_became_peered":"2024-04-20T11:34:35.621973+0000","last_unstale":"2024-04-20T11:34:37.633862+0000","last_undegraded":"2024-04-20T11:34:37.633862+0000","last_fullsized":"2024-04-20T11:34:37.633862+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_clean_scrub_stamp":"2024-04-20T11:34:34.554915+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:57:35.086112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-20T11:34:36.616968+0000","last_change":"2024-04-20T11:34:36.615193+0000","last_active":"2024-04-20T11:34:36.616968+0000","last_peered":"2024-04-20T11:34:36.616968+0000","last_clean":"2024-04-20T11:34:36.616968+0000","last_became_active":"2024-04-20T11:34:36.614726+0000","last_became_peered":"2024-04-20T11:34:36.614726+0000","last_unstale":"2024-04-20T11:34:36.616968+0000","last_undegraded":"2024-04-20T11:34:36.616968+0000","last_fullsized":"2024-04-20T11:34:36.616968+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:32:13.140771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"117'1","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:37.598283+0000","last_change":"2024-04-20T11:34:35.608959+0000","last_active":"2024-04-20T11:34:37.598283+0000","last_peered":"2024-04-20T11:34:37.598283+0000","last_clean":"2024-04-20T11:34:37.598283+0000","last_became_active":"2024-04-20T11:34:35.607680+0000","last_became_peered":"2024-04-20T11:34:35.607680+0000","last_unstale":"2024-04-20T11:34:37.598283+0000","last_undegraded":"2024-04-20T11:34:37.598283+0000","last_fullsized":"2024-04-20T11:34:37.598283+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_clean_scrub_stamp":"2024-04-20T11:34:34.554915+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:53:43.135494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:37.633574+0000","last_change":"2024-04-20T11:34:36.626517+0000","last_active":"2024-04-20T11:34:37.633574+0000","last_peered":"2024-04-20T11:34:37.633574+0000","last_clean":"2024-04-20T11:34:37.633574+0000","last_became_active":"2024-04-20T11:34:36.626172+0000","last_became_peered":"2024-04-20T11:34:36.626172+0000","last_unstale":"2024-04-20T11:34:37.633574+0000","last_undegraded":"2024-04-20T11:34:37.633574+0000","last_fullsized":"2024-04-20T11:34:37.633574+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:12:50.192585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"117'2","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-20T11:34:36.626006+0000","last_change":"2024-04-20T11:34:35.613298+0000","last_active":"2024-04-20T11:34:36.626006+0000","last_peered":"2024-04-20T11:34:36.626006+0000","last_clean":"2024-04-20T11:34:36.626006+0000","last_became_active":"2024-04-20T11:34:35.612904+0000","last_became_peered":"2024-04-20T11:34:35.612904+0000","last_unstale":"2024-04-20T11:34:36.626006+0000","last_undegraded":"2024-04-20T11:34:36.626006+0000","last_fullsized":"2024-04-20T11:34:36.626006+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_clean_scrub_stamp":"2024-04-20T11:34:34.554915+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:09:59.218104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:37.633923+0000","last_change":"2024-04-20T11:34:36.624996+0000","last_active":"2024-04-20T11:34:37.633923+0000","last_peered":"2024-04-20T11:34:37.633923+0000","last_clean":"2024-04-20T11:34:37.633923+0000","last_became_active":"2024-04-20T11:34:36.623069+0000","last_became_peered":"2024-04-20T11:34:36.623069+0000","last_unstale":"2024-04-20T11:34:37.633923+0000","last_undegraded":"2024-04-20T11:34:37.633923+0000","last_fullsized":"2024-04-20T11:34:37.633923+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:58:27.089946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"117'2","reported_seq":16,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:38.211619+0000","last_change":"2024-04-20T11:34:35.607678+0000","last_active":"2024-04-20T11:34:38.211619+0000","last_peered":"2024-04-20T11:34:38.211619+0000","last_clean":"2024-04-20T11:34:38.211619+0000","last_became_active":"2024-04-20T11:34:35.606290+0000","last_became_peered":"2024-04-20T11:34:35.606290+0000","last_unstale":"2024-04-20T11:34:38.211619+0000","last_undegraded":"2024-04-20T11:34:38.211619+0000","last_fullsized":"2024-04-20T11:34:38.211619+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_clean_scrub_stamp":"2024-04-20T11:34:34.554915+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:50:25.165644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:37.597171+0000","last_change":"2024-04-20T11:34:36.625289+0000","last_active":"2024-04-20T11:34:37.597171+0000","last_peered":"2024-04-20T11:34:37.597171+0000","last_clean":"2024-04-20T11:34:37.597171+0000","last_became_active":"2024-04-20T11:34:36.624831+0000","last_became_peered":"2024-04-20T11:34:36.624831+0000","last_unstale":"2024-04-20T11:34:37.597171+0000","last_undegraded":"2024-04-20T11:34:37.597171+0000","last_fullsized":"2024-04-20T11:34:37.597171+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:12:50.192585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"117'1","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:38.210745+0000","last_change":"2024-04-20T11:34:35.604975+0000","last_active":"2024-04-20T11:34:38.210745+0000","last_peered":"2024-04-20T11:34:38.210745+0000","last_clean":"2024-04-20T11:34:38.210745+0000","last_became_active":"2024-04-20T11:34:35.604665+0000","last_became_peered":"2024-04-20T11:34:35.604665+0000","last_unstale":"2024-04-20T11:34:38.210745+0000","last_undegraded":"2024-04-20T11:34:38.210745+0000","last_fullsized":"2024-04-20T11:34:38.210745+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_clean_scrub_stamp":"2024-04-20T11:34:34.554915+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:28:46.049807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:38.484369+0000","last_change":"2024-04-20T11:34:36.620690+0000","last_active":"2024-04-20T11:34:38.484369+0000","last_peered":"2024-04-20T11:34:38.484369+0000","last_clean":"2024-04-20T11:34:38.484369+0000","last_became_active":"2024-04-20T11:34:36.618612+0000","last_became_peered":"2024-04-20T11:34:36.618612+0000","last_unstale":"2024-04-20T11:34:38.484369+0000","last_undegraded":"2024-04-20T11:34:38.484369+0000","last_fullsized":"2024-04-20T11:34:38.484369+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:40:30.539389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"117'1","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:37.597066+0000","last_change":"2024-04-20T11:34:35.607616+0000","last_active":"2024-04-20T11:34:37.597066+0000","last_peered":"2024-04-20T11:34:37.597066+0000","last_clean":"2024-04-20T11:34:37.597066+0000","last_became_active":"2024-04-20T11:34:35.607208+0000","last_became_peered":"2024-04-20T11:34:35.607208+0000","last_unstale":"2024-04-20T11:34:37.597066+0000","last_undegraded":"2024-04-20T11:34:37.597066+0000","last_fullsized":"2024-04-20T11:34:37.597066+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_clean_scrub_stamp":"2024-04-20T11:34:34.554915+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:57:37.447390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-20T11:34:36.635407+0000","last_change":"2024-04-20T11:34:36.623690+0000","last_active":"2024-04-20T11:34:36.635407+0000","last_peered":"2024-04-20T11:34:36.635407+0000","last_clean":"2024-04-20T11:34:36.635407+0000","last_became_active":"2024-04-20T11:34:36.622667+0000","last_became_peered":"2024-04-20T11:34:36.622667+0000","last_unstale":"2024-04-20T11:34:36.635407+0000","last_undegraded":"2024-04-20T11:34:36.635407+0000","last_fullsized":"2024-04-20T11:34:36.635407+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:51:39.025826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:38.483916+0000","last_change":"2024-04-20T11:34:35.622543+0000","last_active":"2024-04-20T11:34:38.483916+0000","last_peered":"2024-04-20T11:34:38.483916+0000","last_clean":"2024-04-20T11:34:38.483916+0000","last_became_active":"2024-04-20T11:34:35.622198+0000","last_became_peered":"2024-04-20T11:34:35.622198+0000","last_unstale":"2024-04-20T11:34:38.483916+0000","last_undegraded":"2024-04-20T11:34:38.483916+0000","last_fullsized":"2024-04-20T11:34:38.483916+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_clean_scrub_stamp":"2024-04-20T11:34:34.554915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:25:10.731059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:38.210906+0000","last_change":"2024-04-20T11:34:36.610956+0000","last_active":"2024-04-20T11:34:38.210906+0000","last_peered":"2024-04-20T11:34:38.210906+0000","last_clean":"2024-04-20T11:34:38.210906+0000","last_became_active":"2024-04-20T11:34:36.610029+0000","last_became_peered":"2024-04-20T11:34:36.610029+0000","last_unstale":"2024-04-20T11:34:38.210906+0000","last_undegraded":"2024-04-20T11:34:38.210906+0000","last_fullsized":"2024-04-20T11:34:38.210906+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:10:40.601449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:38.211227+0000","last_change":"2024-04-20T11:34:35.613214+0000","last_active":"2024-04-20T11:34:38.211227+0000","last_peered":"2024-04-20T11:34:38.211227+0000","last_clean":"2024-04-20T11:34:38.211227+0000","last_became_active":"2024-04-20T11:34:35.612894+0000","last_became_peered":"2024-04-20T11:34:35.612894+0000","last_unstale":"2024-04-20T11:34:38.211227+0000","last_undegraded":"2024-04-20T11:34:38.211227+0000","last_fullsized":"2024-04-20T11:34:38.211227+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_clean_scrub_stamp":"2024-04-20T11:34:34.554915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:32:27.780925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:37.592360+0000","last_change":"2024-04-20T11:34:36.603509+0000","last_active":"2024-04-20T11:34:37.592360+0000","last_peered":"2024-04-20T11:34:37.592360+0000","last_clean":"2024-04-20T11:34:37.592360+0000","last_became_active":"2024-04-20T11:34:36.603209+0000","last_became_peered":"2024-04-20T11:34:36.603209+0000","last_unstale":"2024-04-20T11:34:37.592360+0000","last_undegraded":"2024-04-20T11:34:37.592360+0000","last_fullsized":"2024-04-20T11:34:37.592360+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:47:27.221121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:37.598512+0000","last_change":"2024-04-20T11:34:35.610647+0000","last_active":"2024-04-20T11:34:37.598512+0000","last_peered":"2024-04-20T11:34:37.598512+0000","last_clean":"2024-04-20T11:34:37.598512+0000","last_became_active":"2024-04-20T11:34:35.606382+0000","last_became_peered":"2024-04-20T11:34:35.606382+0000","last_unstale":"2024-04-20T11:34:37.598512+0000","last_undegraded":"2024-04-20T11:34:37.598512+0000","last_fullsized":"2024-04-20T11:34:37.598512+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_clean_scrub_stamp":"2024-04-20T11:34:34.554915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:47:55.225440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:37.598255+0000","last_change":"2024-04-20T11:34:36.614740+0000","last_active":"2024-04-20T11:34:37.598255+0000","last_peered":"2024-04-20T11:34:37.598255+0000","last_clean":"2024-04-20T11:34:37.598255+0000","last_became_active":"2024-04-20T11:34:36.614424+0000","last_became_peered":"2024-04-20T11:34:36.614424+0000","last_unstale":"2024-04-20T11:34:37.598255+0000","last_undegraded":"2024-04-20T11:34:37.598255+0000","last_fullsized":"2024-04-20T11:34:37.598255+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:06:56.712443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:37.598341+0000","last_change":"2024-04-20T11:34:35.608494+0000","last_active":"2024-04-20T11:34:37.598341+0000","last_peered":"2024-04-20T11:34:37.598341+0000","last_clean":"2024-04-20T11:34:37.598341+0000","last_became_active":"2024-04-20T11:34:35.608058+0000","last_became_peered":"2024-04-20T11:34:35.608058+0000","last_unstale":"2024-04-20T11:34:37.598341+0000","last_undegraded":"2024-04-20T11:34:37.598341+0000","last_fullsized":"2024-04-20T11:34:37.598341+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_clean_scrub_stamp":"2024-04-20T11:34:34.554915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:27:10.390559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:37.633327+0000","last_change":"2024-04-20T11:34:36.627090+0000","last_active":"2024-04-20T11:34:37.633327+0000","last_peered":"2024-04-20T11:34:37.633327+0000","last_clean":"2024-04-20T11:34:37.633327+0000","last_became_active":"2024-04-20T11:34:36.626704+0000","last_became_peered":"2024-04-20T11:34:36.626704+0000","last_unstale":"2024-04-20T11:34:37.633327+0000","last_undegraded":"2024-04-20T11:34:37.633327+0000","last_fullsized":"2024-04-20T11:34:37.633327+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:17:52.125796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"117'3","reported_seq":17,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:37.597915+0000","last_change":"2024-04-20T11:34:35.601616+0000","last_active":"2024-04-20T11:34:37.597915+0000","last_peered":"2024-04-20T11:34:37.597915+0000","last_clean":"2024-04-20T11:34:37.597915+0000","last_became_active":"2024-04-20T11:34:35.601094+0000","last_became_peered":"2024-04-20T11:34:35.601094+0000","last_unstale":"2024-04-20T11:34:37.597915+0000","last_undegraded":"2024-04-20T11:34:37.597915+0000","last_fullsized":"2024-04-20T11:34:37.597915+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_clean_scrub_stamp":"2024-04-20T11:34:34.554915+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:50:25.165644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:37.633877+0000","last_change":"2024-04-20T11:34:36.621913+0000","last_active":"2024-04-20T11:34:37.633877+0000","last_peered":"2024-04-20T11:34:37.633877+0000","last_clean":"2024-04-20T11:34:37.633877+0000","last_became_active":"2024-04-20T11:34:36.621372+0000","last_became_peered":"2024-04-20T11:34:36.621372+0000","last_unstale":"2024-04-20T11:34:37.633877+0000","last_undegraded":"2024-04-20T11:34:37.633877+0000","last_fullsized":"2024-04-20T11:34:37.633877+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:57:35.372036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"117'6","reported_seq":18,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-20T11:34:36.641314+0000","last_change":"2024-04-20T11:34:35.619954+0000","last_active":"2024-04-20T11:34:36.641314+0000","last_peered":"2024-04-20T11:34:36.641314+0000","last_clean":"2024-04-20T11:34:36.641314+0000","last_became_active":"2024-04-20T11:34:35.619625+0000","last_became_peered":"2024-04-20T11:34:35.619625+0000","last_unstale":"2024-04-20T11:34:36.641314+0000","last_undegraded":"2024-04-20T11:34:36.641314+0000","last_fullsized":"2024-04-20T11:34:36.641314+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_clean_scrub_stamp":"2024-04-20T11:34:34.554915+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:10:39.587624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:37.594170+0000","last_change":"2024-04-20T11:34:36.603729+0000","last_active":"2024-04-20T11:34:37.594170+0000","last_peered":"2024-04-20T11:34:37.594170+0000","last_clean":"2024-04-20T11:34:37.594170+0000","last_became_active":"2024-04-20T11:34:36.603447+0000","last_became_peered":"2024-04-20T11:34:36.603447+0000","last_unstale":"2024-04-20T11:34:37.594170+0000","last_undegraded":"2024-04-20T11:34:37.594170+0000","last_fullsized":"2024-04-20T11:34:37.594170+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:59:02.109644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:37.593157+0000","last_change":"2024-04-20T11:34:35.611133+0000","last_active":"2024-04-20T11:34:37.593157+0000","last_peered":"2024-04-20T11:34:37.593157+0000","last_clean":"2024-04-20T11:34:37.593157+0000","last_became_active":"2024-04-20T11:34:35.610837+0000","last_became_peered":"2024-04-20T11:34:35.610837+0000","last_unstale":"2024-04-20T11:34:37.593157+0000","last_undegraded":"2024-04-20T11:34:37.593157+0000","last_fullsized":"2024-04-20T11:34:37.593157+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_clean_scrub_stamp":"2024-04-20T11:34:34.554915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:17:10.243879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:38.210906+0000","last_change":"2024-04-20T11:34:36.616647+0000","last_active":"2024-04-20T11:34:38.210906+0000","last_peered":"2024-04-20T11:34:38.210906+0000","last_clean":"2024-04-20T11:34:38.210906+0000","last_became_active":"2024-04-20T11:34:36.616341+0000","last_became_peered":"2024-04-20T11:34:36.616341+0000","last_unstale":"2024-04-20T11:34:38.210906+0000","last_undegraded":"2024-04-20T11:34:38.210906+0000","last_fullsized":"2024-04-20T11:34:38.210906+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:19:21.488409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:38.211479+0000","last_change":"2024-04-20T11:34:35.620071+0000","last_active":"2024-04-20T11:34:38.211479+0000","last_peered":"2024-04-20T11:34:38.211479+0000","last_clean":"2024-04-20T11:34:38.211479+0000","last_became_active":"2024-04-20T11:34:35.619745+0000","last_became_peered":"2024-04-20T11:34:35.619745+0000","last_unstale":"2024-04-20T11:34:38.211479+0000","last_undegraded":"2024-04-20T11:34:38.211479+0000","last_fullsized":"2024-04-20T11:34:38.211479+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_clean_scrub_stamp":"2024-04-20T11:34:34.554915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:53:43.135494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-20T11:34:36.638562+0000","last_change":"2024-04-20T11:34:36.634023+0000","last_active":"2024-04-20T11:34:36.638562+0000","last_peered":"2024-04-20T11:34:36.638562+0000","last_clean":"2024-04-20T11:34:36.638562+0000","last_became_active":"2024-04-20T11:34:36.633702+0000","last_became_peered":"2024-04-20T11:34:36.633702+0000","last_unstale":"2024-04-20T11:34:36.638562+0000","last_undegraded":"2024-04-20T11:34:36.638562+0000","last_fullsized":"2024-04-20T11:34:36.638562+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:59:18.372456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"117'8","reported_seq":20,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-20T11:34:36.639817+0000","last_change":"2024-04-20T11:34:35.615732+0000","last_active":"2024-04-20T11:34:36.639817+0000","last_peered":"2024-04-20T11:34:36.639817+0000","last_clean":"2024-04-20T11:34:36.639817+0000","last_became_active":"2024-04-20T11:34:35.615094+0000","last_became_peered":"2024-04-20T11:34:35.615094+0000","last_unstale":"2024-04-20T11:34:36.639817+0000","last_undegraded":"2024-04-20T11:34:36.639817+0000","last_fullsized":"2024-04-20T11:34:36.639817+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_clean_scrub_stamp":"2024-04-20T11:34:34.554915+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:19:01.727664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-20T11:34:36.637464+0000","last_change":"2024-04-20T11:34:36.632021+0000","last_active":"2024-04-20T11:34:36.637464+0000","last_peered":"2024-04-20T11:34:36.637464+0000","last_clean":"2024-04-20T11:34:36.637464+0000","last_became_active":"2024-04-20T11:34:36.631728+0000","last_became_peered":"2024-04-20T11:34:36.631728+0000","last_unstale":"2024-04-20T11:34:36.637464+0000","last_undegraded":"2024-04-20T11:34:36.637464+0000","last_fullsized":"2024-04-20T11:34:36.637464+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:23:45.660339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"117'1","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:37.598695+0000","last_change":"2024-04-20T11:34:35.619853+0000","last_active":"2024-04-20T11:34:37.598695+0000","last_peered":"2024-04-20T11:34:37.598695+0000","last_clean":"2024-04-20T11:34:37.598695+0000","last_became_active":"2024-04-20T11:34:35.619540+0000","last_became_peered":"2024-04-20T11:34:35.619540+0000","last_unstale":"2024-04-20T11:34:37.598695+0000","last_undegraded":"2024-04-20T11:34:37.598695+0000","last_fullsized":"2024-04-20T11:34:37.598695+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_clean_scrub_stamp":"2024-04-20T11:34:34.554915+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:19:48.867052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:37.598403+0000","last_change":"2024-04-20T11:34:36.602442+0000","last_active":"2024-04-20T11:34:37.598403+0000","last_peered":"2024-04-20T11:34:37.598403+0000","last_clean":"2024-04-20T11:34:37.598403+0000","last_became_active":"2024-04-20T11:34:36.601419+0000","last_became_peered":"2024-04-20T11:34:36.601419+0000","last_unstale":"2024-04-20T11:34:37.598403+0000","last_undegraded":"2024-04-20T11:34:37.598403+0000","last_fullsized":"2024-04-20T11:34:37.598403+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:57:06.686709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"117'2","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-20T11:34:36.634522+0000","last_change":"2024-04-20T11:34:35.621744+0000","last_active":"2024-04-20T11:34:36.634522+0000","last_peered":"2024-04-20T11:34:36.634522+0000","last_clean":"2024-04-20T11:34:36.634522+0000","last_became_active":"2024-04-20T11:34:35.621430+0000","last_became_peered":"2024-04-20T11:34:35.621430+0000","last_unstale":"2024-04-20T11:34:36.634522+0000","last_undegraded":"2024-04-20T11:34:36.634522+0000","last_fullsized":"2024-04-20T11:34:36.634522+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_clean_scrub_stamp":"2024-04-20T11:34:34.554915+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:44:24.876930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:37.598199+0000","last_change":"2024-04-20T11:34:36.609780+0000","last_active":"2024-04-20T11:34:37.598199+0000","last_peered":"2024-04-20T11:34:37.598199+0000","last_clean":"2024-04-20T11:34:37.598199+0000","last_became_active":"2024-04-20T11:34:36.608521+0000","last_became_peered":"2024-04-20T11:34:36.608521+0000","last_unstale":"2024-04-20T11:34:37.598199+0000","last_undegraded":"2024-04-20T11:34:37.598199+0000","last_fullsized":"2024-04-20T11:34:37.598199+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:09:01.413284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.d","version":"117'1","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:38.484629+0000","last_change":"2024-04-20T11:34:35.616950+0000","last_active":"2024-04-20T11:34:38.484629+0000","last_peered":"2024-04-20T11:34:38.484629+0000","last_clean":"2024-04-20T11:34:38.484629+0000","last_became_active":"2024-04-20T11:34:35.616595+0000","last_became_peered":"2024-04-20T11:34:35.616595+0000","last_unstale":"2024-04-20T11:34:38.484629+0000","last_undegraded":"2024-04-20T11:34:38.484629+0000","last_fullsized":"2024-04-20T11:34:38.484629+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_clean_scrub_stamp":"2024-04-20T11:34:34.554915+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:57:35.086112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:38.211229+0000","last_change":"2024-04-20T11:34:36.612368+0000","last_active":"2024-04-20T11:34:38.211229+0000","last_peered":"2024-04-20T11:34:38.211229+0000","last_clean":"2024-04-20T11:34:38.211229+0000","last_became_active":"2024-04-20T11:34:36.611025+0000","last_became_peered":"2024-04-20T11:34:36.611025+0000","last_unstale":"2024-04-20T11:34:38.211229+0000","last_undegraded":"2024-04-20T11:34:38.211229+0000","last_fullsized":"2024-04-20T11:34:38.211229+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:32:13.140771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:37.597881+0000","last_change":"2024-04-20T11:34:35.606708+0000","last_active":"2024-04-20T11:34:37.597881+0000","last_peered":"2024-04-20T11:34:37.597881+0000","last_clean":"2024-04-20T11:34:37.597881+0000","last_became_active":"2024-04-20T11:34:35.606275+0000","last_became_peered":"2024-04-20T11:34:35.606275+0000","last_unstale":"2024-04-20T11:34:37.597881+0000","last_undegraded":"2024-04-20T11:34:37.597881+0000","last_fullsized":"2024-04-20T11:34:37.597881+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_clean_scrub_stamp":"2024-04-20T11:34:34.554915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:22:57.415194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:37.592917+0000","last_change":"2024-04-20T11:34:36.635812+0000","last_active":"2024-04-20T11:34:37.592917+0000","last_peered":"2024-04-20T11:34:37.592917+0000","last_clean":"2024-04-20T11:34:37.592917+0000","last_became_active":"2024-04-20T11:34:36.635082+0000","last_became_peered":"2024-04-20T11:34:36.635082+0000","last_unstale":"2024-04-20T11:34:37.592917+0000","last_undegraded":"2024-04-20T11:34:37.592917+0000","last_fullsized":"2024-04-20T11:34:37.592917+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:06:42.203253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"117'2","reported_seq":16,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:37.597518+0000","last_change":"2024-04-20T11:34:35.609152+0000","last_active":"2024-04-20T11:34:37.597518+0000","last_peered":"2024-04-20T11:34:37.597518+0000","last_clean":"2024-04-20T11:34:37.597518+0000","last_became_active":"2024-04-20T11:34:35.607949+0000","last_became_peered":"2024-04-20T11:34:35.607949+0000","last_unstale":"2024-04-20T11:34:37.597518+0000","last_undegraded":"2024-04-20T11:34:37.597518+0000","last_fullsized":"2024-04-20T11:34:37.597518+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_clean_scrub_stamp":"2024-04-20T11:34:34.554915+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:40:21.726230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:38.484492+0000","last_change":"2024-04-20T11:34:36.622992+0000","last_active":"2024-04-20T11:34:38.484492+0000","last_peered":"2024-04-20T11:34:38.484492+0000","last_clean":"2024-04-20T11:34:38.484492+0000","last_became_active":"2024-04-20T11:34:36.622448+0000","last_became_peered":"2024-04-20T11:34:36.622448+0000","last_unstale":"2024-04-20T11:34:38.484492+0000","last_undegraded":"2024-04-20T11:34:38.484492+0000","last_fullsized":"2024-04-20T11:34:38.484492+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:28:42.034218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:37.597371+0000","last_change":"2024-04-20T11:34:35.610851+0000","last_active":"2024-04-20T11:34:37.597371+0000","last_peered":"2024-04-20T11:34:37.597371+0000","last_clean":"2024-04-20T11:34:37.597371+0000","last_became_active":"2024-04-20T11:34:35.610583+0000","last_became_peered":"2024-04-20T11:34:35.610583+0000","last_unstale":"2024-04-20T11:34:37.597371+0000","last_undegraded":"2024-04-20T11:34:37.597371+0000","last_fullsized":"2024-04-20T11:34:37.597371+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_clean_scrub_stamp":"2024-04-20T11:34:34.554915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:04:39.272610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:38.211000+0000","last_change":"2024-04-20T11:34:36.606406+0000","last_active":"2024-04-20T11:34:38.211000+0000","last_peered":"2024-04-20T11:34:38.211000+0000","last_clean":"2024-04-20T11:34:38.211000+0000","last_became_active":"2024-04-20T11:34:36.606111+0000","last_became_peered":"2024-04-20T11:34:36.606111+0000","last_unstale":"2024-04-20T11:34:38.211000+0000","last_undegraded":"2024-04-20T11:34:38.211000+0000","last_fullsized":"2024-04-20T11:34:38.211000+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:19:02.741488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:37.596838+0000","last_change":"2024-04-20T11:34:35.606580+0000","last_active":"2024-04-20T11:34:37.596838+0000","last_peered":"2024-04-20T11:34:37.596838+0000","last_clean":"2024-04-20T11:34:37.596838+0000","last_became_active":"2024-04-20T11:34:35.606321+0000","last_became_peered":"2024-04-20T11:34:35.606321+0000","last_unstale":"2024-04-20T11:34:37.596838+0000","last_undegraded":"2024-04-20T11:34:37.596838+0000","last_fullsized":"2024-04-20T11:34:37.596838+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_clean_scrub_stamp":"2024-04-20T11:34:34.554915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:15:51.824961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:37.632210+0000","last_change":"2024-04-20T11:34:36.629614+0000","last_active":"2024-04-20T11:34:37.632210+0000","last_peered":"2024-04-20T11:34:37.632210+0000","last_clean":"2024-04-20T11:34:37.632210+0000","last_became_active":"2024-04-20T11:34:36.628425+0000","last_became_peered":"2024-04-20T11:34:36.628425+0000","last_unstale":"2024-04-20T11:34:37.632210+0000","last_undegraded":"2024-04-20T11:34:37.632210+0000","last_fullsized":"2024-04-20T11:34:37.632210+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:11:38.577132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-20T11:34:36.590101+0000","last_change":"2024-04-20T11:34:35.618857+0000","last_active":"2024-04-20T11:34:36.590101+0000","last_peered":"2024-04-20T11:34:36.590101+0000","last_clean":"2024-04-20T11:34:36.590101+0000","last_became_active":"2024-04-20T11:34:35.618473+0000","last_became_peered":"2024-04-20T11:34:35.618473+0000","last_unstale":"2024-04-20T11:34:36.590101+0000","last_undegraded":"2024-04-20T11:34:36.590101+0000","last_fullsized":"2024-04-20T11:34:36.590101+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_clean_scrub_stamp":"2024-04-20T11:34:34.554915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:28:05.739229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:38.484719+0000","last_change":"2024-04-20T11:34:36.636756+0000","last_active":"2024-04-20T11:34:38.484719+0000","last_peered":"2024-04-20T11:34:38.484719+0000","last_clean":"2024-04-20T11:34:38.484719+0000","last_became_active":"2024-04-20T11:34:36.636404+0000","last_became_peered":"2024-04-20T11:34:36.636404+0000","last_unstale":"2024-04-20T11:34:38.484719+0000","last_undegraded":"2024-04-20T11:34:38.484719+0000","last_fullsized":"2024-04-20T11:34:38.484719+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:17:52.125796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.7","version":"117'2","reported_seq":16,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:37.593155+0000","last_change":"2024-04-20T11:34:35.608692+0000","last_active":"2024-04-20T11:34:37.593155+0000","last_peered":"2024-04-20T11:34:37.593155+0000","last_clean":"2024-04-20T11:34:37.593155+0000","last_became_active":"2024-04-20T11:34:35.608400+0000","last_became_peered":"2024-04-20T11:34:35.608400+0000","last_unstale":"2024-04-20T11:34:37.593155+0000","last_undegraded":"2024-04-20T11:34:37.593155+0000","last_fullsized":"2024-04-20T11:34:37.593155+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_clean_scrub_stamp":"2024-04-20T11:34:34.554915+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:32:04.614712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:37.593041+0000","last_change":"2024-04-20T11:34:36.615472+0000","last_active":"2024-04-20T11:34:37.593041+0000","last_peered":"2024-04-20T11:34:37.593041+0000","last_clean":"2024-04-20T11:34:37.593041+0000","last_became_active":"2024-04-20T11:34:36.615142+0000","last_became_peered":"2024-04-20T11:34:36.615142+0000","last_unstale":"2024-04-20T11:34:37.593041+0000","last_undegraded":"2024-04-20T11:34:37.593041+0000","last_fullsized":"2024-04-20T11:34:37.593041+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:35:52.566260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:37.633192+0000","last_change":"2024-04-20T11:34:35.624251+0000","last_active":"2024-04-20T11:34:37.633192+0000","last_peered":"2024-04-20T11:34:37.633192+0000","last_clean":"2024-04-20T11:34:37.633192+0000","last_became_active":"2024-04-20T11:34:35.623867+0000","last_became_peered":"2024-04-20T11:34:35.623867+0000","last_unstale":"2024-04-20T11:34:37.633192+0000","last_undegraded":"2024-04-20T11:34:37.633192+0000","last_fullsized":"2024-04-20T11:34:37.633192+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_clean_scrub_stamp":"2024-04-20T11:34:34.554915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:03:31.045549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:38.211667+0000","last_change":"2024-04-20T11:34:36.620993+0000","last_active":"2024-04-20T11:34:38.211667+0000","last_peered":"2024-04-20T11:34:38.211667+0000","last_clean":"2024-04-20T11:34:38.211667+0000","last_became_active":"2024-04-20T11:34:36.618315+0000","last_became_peered":"2024-04-20T11:34:36.618315+0000","last_unstale":"2024-04-20T11:34:38.211667+0000","last_undegraded":"2024-04-20T11:34:38.211667+0000","last_fullsized":"2024-04-20T11:34:38.211667+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:23:45.660339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:38.211247+0000","last_change":"2024-04-20T11:34:35.617625+0000","last_active":"2024-04-20T11:34:38.211247+0000","last_peered":"2024-04-20T11:34:38.211247+0000","last_clean":"2024-04-20T11:34:38.211247+0000","last_became_active":"2024-04-20T11:34:35.617320+0000","last_became_peered":"2024-04-20T11:34:35.617320+0000","last_unstale":"2024-04-20T11:34:38.211247+0000","last_undegraded":"2024-04-20T11:34:38.211247+0000","last_fullsized":"2024-04-20T11:34:38.211247+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_clean_scrub_stamp":"2024-04-20T11:34:34.554915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:40:21.726230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:38.484503+0000","last_change":"2024-04-20T11:34:36.635337+0000","last_active":"2024-04-20T11:34:38.484503+0000","last_peered":"2024-04-20T11:34:38.484503+0000","last_clean":"2024-04-20T11:34:38.484503+0000","last_became_active":"2024-04-20T11:34:36.634947+0000","last_became_peered":"2024-04-20T11:34:36.634947+0000","last_unstale":"2024-04-20T11:34:38.484503+0000","last_undegraded":"2024-04-20T11:34:38.484503+0000","last_fullsized":"2024-04-20T11:34:38.484503+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:34:54.762775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-20T11:34:36.591565+0000","last_change":"2024-04-20T11:34:35.606436+0000","last_active":"2024-04-20T11:34:36.591565+0000","last_peered":"2024-04-20T11:34:36.591565+0000","last_clean":"2024-04-20T11:34:36.591565+0000","last_became_active":"2024-04-20T11:34:35.606114+0000","last_became_peered":"2024-04-20T11:34:35.606114+0000","last_unstale":"2024-04-20T11:34:36.591565+0000","last_undegraded":"2024-04-20T11:34:36.591565+0000","last_fullsized":"2024-04-20T11:34:36.591565+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_clean_scrub_stamp":"2024-04-20T11:34:34.554915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:07:27.769385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:37.632798+0000","last_change":"2024-04-20T11:34:36.628297+0000","last_active":"2024-04-20T11:34:37.632798+0000","last_peered":"2024-04-20T11:34:37.632798+0000","last_clean":"2024-04-20T11:34:37.632798+0000","last_became_active":"2024-04-20T11:34:36.627978+0000","last_became_peered":"2024-04-20T11:34:36.627978+0000","last_unstale":"2024-04-20T11:34:37.632798+0000","last_undegraded":"2024-04-20T11:34:37.632798+0000","last_fullsized":"2024-04-20T11:34:37.632798+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:34:54.762775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:38.211214+0000","last_change":"2024-04-20T11:34:35.622211+0000","last_active":"2024-04-20T11:34:38.211214+0000","last_peered":"2024-04-20T11:34:38.211214+0000","last_clean":"2024-04-20T11:34:38.211214+0000","last_became_active":"2024-04-20T11:34:35.621869+0000","last_became_peered":"2024-04-20T11:34:35.621869+0000","last_unstale":"2024-04-20T11:34:38.211214+0000","last_undegraded":"2024-04-20T11:34:38.211214+0000","last_fullsized":"2024-04-20T11:34:38.211214+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_clean_scrub_stamp":"2024-04-20T11:34:34.554915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:26:01.834165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:37.597600+0000","last_change":"2024-04-20T11:34:36.610415+0000","last_active":"2024-04-20T11:34:37.597600+0000","last_peered":"2024-04-20T11:34:37.597600+0000","last_clean":"2024-04-20T11:34:37.597600+0000","last_became_active":"2024-04-20T11:34:36.610052+0000","last_became_peered":"2024-04-20T11:34:36.610052+0000","last_unstale":"2024-04-20T11:34:37.597600+0000","last_undegraded":"2024-04-20T11:34:37.597600+0000","last_fullsized":"2024-04-20T11:34:37.597600+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:52:00.000858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:37.633636+0000","last_change":"2024-04-20T11:34:35.622833+0000","last_active":"2024-04-20T11:34:37.633636+0000","last_peered":"2024-04-20T11:34:37.633636+0000","last_clean":"2024-04-20T11:34:37.633636+0000","last_became_active":"2024-04-20T11:34:35.621851+0000","last_became_peered":"2024-04-20T11:34:35.621851+0000","last_unstale":"2024-04-20T11:34:37.633636+0000","last_undegraded":"2024-04-20T11:34:37.633636+0000","last_fullsized":"2024-04-20T11:34:37.633636+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_clean_scrub_stamp":"2024-04-20T11:34:34.554915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:39:56.418943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:37.597482+0000","last_change":"2024-04-20T11:34:36.621243+0000","last_active":"2024-04-20T11:34:37.597482+0000","last_peered":"2024-04-20T11:34:37.597482+0000","last_clean":"2024-04-20T11:34:37.597482+0000","last_became_active":"2024-04-20T11:34:36.620968+0000","last_became_peered":"2024-04-20T11:34:36.620968+0000","last_unstale":"2024-04-20T11:34:37.597482+0000","last_undegraded":"2024-04-20T11:34:37.597482+0000","last_fullsized":"2024-04-20T11:34:37.597482+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:58:27.089946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:37.597053+0000","last_change":"2024-04-20T11:34:35.611518+0000","last_active":"2024-04-20T11:34:37.597053+0000","last_peered":"2024-04-20T11:34:37.597053+0000","last_clean":"2024-04-20T11:34:37.597053+0000","last_became_active":"2024-04-20T11:34:35.609737+0000","last_became_peered":"2024-04-20T11:34:35.609737+0000","last_unstale":"2024-04-20T11:34:37.597053+0000","last_undegraded":"2024-04-20T11:34:37.597053+0000","last_fullsized":"2024-04-20T11:34:37.597053+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_clean_scrub_stamp":"2024-04-20T11:34:34.554915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:29:58.591093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:38.211261+0000","last_change":"2024-04-20T11:34:36.614617+0000","last_active":"2024-04-20T11:34:38.211261+0000","last_peered":"2024-04-20T11:34:38.211261+0000","last_clean":"2024-04-20T11:34:38.211261+0000","last_became_active":"2024-04-20T11:34:36.614320+0000","last_became_peered":"2024-04-20T11:34:36.614320+0000","last_unstale":"2024-04-20T11:34:38.211261+0000","last_undegraded":"2024-04-20T11:34:38.211261+0000","last_fullsized":"2024-04-20T11:34:38.211261+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:51:39.025826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-20T11:34:36.590317+0000","last_change":"2024-04-20T11:34:35.613736+0000","last_active":"2024-04-20T11:34:36.590317+0000","last_peered":"2024-04-20T11:34:36.590317+0000","last_clean":"2024-04-20T11:34:36.590317+0000","last_became_active":"2024-04-20T11:34:35.612303+0000","last_became_peered":"2024-04-20T11:34:35.612303+0000","last_unstale":"2024-04-20T11:34:36.590317+0000","last_undegraded":"2024-04-20T11:34:36.590317+0000","last_fullsized":"2024-04-20T11:34:36.590317+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_clean_scrub_stamp":"2024-04-20T11:34:34.554915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:54:21.076905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:37.632380+0000","last_change":"2024-04-20T11:34:36.630553+0000","last_active":"2024-04-20T11:34:37.632380+0000","last_peered":"2024-04-20T11:34:37.632380+0000","last_clean":"2024-04-20T11:34:37.632380+0000","last_became_active":"2024-04-20T11:34:36.630239+0000","last_became_peered":"2024-04-20T11:34:36.630239+0000","last_unstale":"2024-04-20T11:34:37.632380+0000","last_undegraded":"2024-04-20T11:34:37.632380+0000","last_fullsized":"2024-04-20T11:34:37.632380+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:58:16.330485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"117'2","reported_seq":16,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:37.633357+0000","last_change":"2024-04-20T11:34:35.623699+0000","last_active":"2024-04-20T11:34:37.633357+0000","last_peered":"2024-04-20T11:34:37.633357+0000","last_clean":"2024-04-20T11:34:37.633357+0000","last_became_active":"2024-04-20T11:34:35.623391+0000","last_became_peered":"2024-04-20T11:34:35.623391+0000","last_unstale":"2024-04-20T11:34:37.633357+0000","last_undegraded":"2024-04-20T11:34:37.633357+0000","last_fullsized":"2024-04-20T11:34:37.633357+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_clean_scrub_stamp":"2024-04-20T11:34:34.554915+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:44:14.415607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"72'128","reported_seq":384,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:37.631852+0000","last_change":"2024-04-20T11:26:48.661897+0000","last_active":"2024-04-20T11:34:37.631852+0000","last_peered":"2024-04-20T11:34:37.631852+0000","last_clean":"2024-04-20T11:34:37.631852+0000","last_became_active":"2024-04-20T11:26:48.656535+0000","last_became_peered":"2024-04-20T11:26:48.656535+0000","last_unstale":"2024-04-20T11:34:37.631852+0000","last_undegraded":"2024-04-20T11:34:37.631852+0000","last_fullsized":"2024-04-20T11:34:37.631852+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:47.543004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:47.543004+0000","last_clean_scrub_stamp":"2024-04-20T11:26:47.543004+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:58:35.527105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574948,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33224,"kb_used_data":2044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338616,"statfs":{"total":96636764160,"available":96602742784,"internally_reserved":0,"allocated":2093056,"data_stored":1243587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574945,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32620,"kb_used_data":1440,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339220,"statfs":{"total":96636764160,"available":96603361280,"internally_reserved":0,"allocated":1474560,"data_stored":646281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574947,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32628,"kb_used_data":1448,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339212,"statfs":{"total":96636764160,"available":96603353088,"internally_reserved":0,"allocated":1482752,"data_stored":651081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574945,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32620,"kb_used_data":1432,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339220,"statfs":{"total":96636764160,"available":96603361280,"internally_reserved":0,"allocated":1466368,"data_stored":644739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574946,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32632,"kb_used_data":1448,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339208,"statfs":{"total":96636764160,"available":96603348992,"internally_reserved":0,"allocated":1482752,"data_stored":651129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574947,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32632,"kb_used_data":1448,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339208,"statfs":{"total":96636764160,"available":96603348992,"internally_reserved":0,"allocated":1482752,"data_stored":651585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574945,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32616,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339224,"statfs":{"total":96636764160,"available":96603365376,"internally_reserved":0,"allocated":1462272,"data_stored":644669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574946,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33196,"kb_used_data":2008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338644,"statfs":{"total":96636764160,"available":96602771456,"internally_reserved":0,"allocated":2056192,"data_stored":1235037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T11:34:41.040 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T11:34:41.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.273+0000 7f85fec00640 1 Processor -- start 2024-04-20T11:34:41.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.273+0000 7f85fec00640 1 -- start start 2024-04-20T11:34:41.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.274+0000 7f85fec00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f85f813e530 0x7f85f806ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:41.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.274+0000 7f85fec00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f85f806f340 0x7f85f8140dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:41.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.274+0000 7f85fec00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f85f8141310 0x7f85f815efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:41.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.274+0000 7f85fec00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f85f8141b00 con 0x7f85f813e530 2024-04-20T11:34:41.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.274+0000 7f85fec00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f85f80cfc50 con 0x7f85f806f340 2024-04-20T11:34:41.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.274+0000 7f85fec00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f85f80cf0a0 con 0x7f85f8141310 2024-04-20T11:34:41.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.274+0000 7f85fd800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f85f813e530 0x7f85f806ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:41.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.274+0000 7f85fd800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f85f813e530 0x7f85f806ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:50852/0 (socket says 172.21.15.152:50852) 2024-04-20T11:34:41.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.274+0000 7f85fd800640 1 -- 172.21.15.152:0/1488250086 learned_addr learned my addr 172.21.15.152:0/1488250086 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:41.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.275+0000 7f85fce00640 1 --2- 172.21.15.152:0/1488250086 >> v2:172.21.15.175:3300/0 conn(0x7f85f806f340 0x7f85f8140dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:41.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.275+0000 7f85fe200640 1 --2- 172.21.15.152:0/1488250086 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f85f8141310 0x7f85f815efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:41.277 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.275+0000 7f85fd800640 1 -- 172.21.15.152:0/1488250086 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f85f8141310 msgr2=0x7f85f815efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:41.277 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.275+0000 7f85fd800640 1 --2- 172.21.15.152:0/1488250086 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f85f8141310 0x7f85f815efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:41.277 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.275+0000 7f85fd800640 1 -- 172.21.15.152:0/1488250086 >> v2:172.21.15.175:3300/0 conn(0x7f85f806f340 msgr2=0x7f85f8140dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:41.277 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.275+0000 7f85fd800640 1 --2- 172.21.15.152:0/1488250086 >> v2:172.21.15.175:3300/0 conn(0x7f85f806f340 0x7f85f8140dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:41.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.275+0000 7f85fd800640 1 -- 172.21.15.152:0/1488250086 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85f80d00e0 con 0x7f85f813e530 2024-04-20T11:34:41.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.275+0000 7f85fce00640 1 --2- 172.21.15.152:0/1488250086 >> v2:172.21.15.175:3300/0 conn(0x7f85f806f340 0x7f85f8140dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:34:41.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.275+0000 7f85fe200640 1 --2- 172.21.15.152:0/1488250086 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f85f8141310 0x7f85f815efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:34:41.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.276+0000 7f85fd800640 1 --2- 172.21.15.152:0/1488250086 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f85f813e530 0x7f85f806ee00 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f85f806feb0 tx=0x7f85ec02e0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=667257101f9c0bf4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:41.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.276+0000 7f85f7e00640 1 -- 172.21.15.152:0/1488250086 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85ec03b070 con 0x7f85f813e530 2024-04-20T11:34:41.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.276+0000 7f85f7e00640 1 -- 172.21.15.152:0/1488250086 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85ec036440 con 0x7f85f813e530 2024-04-20T11:34:41.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.276+0000 7f85f7e00640 1 -- 172.21.15.152:0/1488250086 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85ec036810 con 0x7f85f813e530 2024-04-20T11:34:41.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.277+0000 7f85fec00640 1 -- 172.21.15.152:0/1488250086 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f85f813e530 msgr2=0x7f85f806ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:41.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.277+0000 7f85fec00640 1 --2- 172.21.15.152:0/1488250086 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f85f813e530 0x7f85f806ee00 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f85f806feb0 tx=0x7f85ec02e0d0 comp rx=0 tx=0).stop 2024-04-20T11:34:41.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.277+0000 7f85fec00640 1 -- 172.21.15.152:0/1488250086 shutdown_connections 2024-04-20T11:34:41.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.277+0000 7f85fec00640 1 --2- 172.21.15.152:0/1488250086 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f85f8141310 0x7f85f815efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:41.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.277+0000 7f85fec00640 1 --2- 172.21.15.152:0/1488250086 >> v2:172.21.15.175:3300/0 conn(0x7f85f806f340 0x7f85f8140dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:41.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.277+0000 7f85fec00640 1 --2- 172.21.15.152:0/1488250086 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f85f813e530 0x7f85f806ee00 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:41.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.277+0000 7f85fec00640 1 -- 172.21.15.152:0/1488250086 >> 172.21.15.152:0/1488250086 conn(0x7f85f8067ae0 msgr2=0x7f85f806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:41.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.277+0000 7f85fec00640 1 -- 172.21.15.152:0/1488250086 shutdown_connections 2024-04-20T11:34:41.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.277+0000 7f85fec00640 1 -- 172.21.15.152:0/1488250086 wait complete. 2024-04-20T11:34:41.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.279+0000 7f85fec00640 1 Processor -- start 2024-04-20T11:34:41.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.279+0000 7f85fec00640 1 -- start start 2024-04-20T11:34:41.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.280+0000 7f85fec00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f85f806f340 0x7f85f813da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:41.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.280+0000 7f85fec00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85f813e530 0x7f85f8153180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:41.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.280+0000 7f85fec00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f85f8141310 0x7f85f813fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:41.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.280+0000 7f85fec00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f85f80cf8e0 con 0x7f85f8141310 2024-04-20T11:34:41.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.280+0000 7f85fec00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f85f80cf600 con 0x7f85f813e530 2024-04-20T11:34:41.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.280+0000 7f85fec00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f85f81417a0 con 0x7f85f806f340 2024-04-20T11:34:41.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.280+0000 7f85fe200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f85f8141310 0x7f85f813fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:41.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.280+0000 7f85fe200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f85f8141310 0x7f85f813fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:50856/0 (socket says 172.21.15.152:50856) 2024-04-20T11:34:41.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.280+0000 7f85fe200640 1 -- 172.21.15.152:0/3161095808 learned_addr learned my addr 172.21.15.152:0/3161095808 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:41.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.281+0000 7f85fce00640 1 --2- 172.21.15.152:0/3161095808 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85f813e530 0x7f85f8153180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:41.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.281+0000 7f85fd800640 1 --2- 172.21.15.152:0/3161095808 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f85f806f340 0x7f85f813da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:41.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.281+0000 7f85fe200640 1 -- 172.21.15.152:0/3161095808 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f85f806f340 msgr2=0x7f85f813da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:41.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.281+0000 7f85fe200640 1 --2- 172.21.15.152:0/3161095808 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f85f806f340 0x7f85f813da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:41.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.281+0000 7f85fe200640 1 -- 172.21.15.152:0/3161095808 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85f813e530 msgr2=0x7f85f8153180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:41.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.281+0000 7f85fe200640 1 --2- 172.21.15.152:0/3161095808 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85f813e530 0x7f85f8153180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:41.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.281+0000 7f85fe200640 1 -- 172.21.15.152:0/3161095808 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85f81546a0 con 0x7f85f8141310 2024-04-20T11:34:41.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.281+0000 7f85fce00640 1 --2- 172.21.15.152:0/3161095808 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85f813e530 0x7f85f8153180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:34:41.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.281+0000 7f85fe200640 1 --2- 172.21.15.152:0/3161095808 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f85f8141310 0x7f85f813fb70 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f85f000cd50 tx=0x7f85f000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:41.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.281+0000 7f85f5600640 1 -- 172.21.15.152:0/3161095808 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f000f680 con 0x7f85f8141310 2024-04-20T11:34:41.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.281+0000 7f85f5600640 1 -- 172.21.15.152:0/3161095808 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85f000f840 con 0x7f85f8141310 2024-04-20T11:34:41.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.281+0000 7f85f5600640 1 -- 172.21.15.152:0/3161095808 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f000fc30 con 0x7f85f8141310 2024-04-20T11:34:41.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.281+0000 7f85fec00640 1 -- 172.21.15.152:0/3161095808 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85f8153b20 con 0x7f85f8141310 2024-04-20T11:34:41.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.282+0000 7f85fec00640 1 -- 172.21.15.152:0/3161095808 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85f81540e0 con 0x7f85f8141310 2024-04-20T11:34:41.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.283+0000 7f85fec00640 1 -- 172.21.15.152:0/3161095808 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85f8072cb0 con 0x7f85f8141310 2024-04-20T11:34:41.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.284+0000 7f85f5600640 1 -- 172.21.15.152:0/3161095808 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 19) ==== 91397+0+0 (secure 0 0 0) 0x7f85f00213f0 con 0x7f85f8141310 2024-04-20T11:34:41.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.284+0000 7f85f5600640 1 --2- 172.21.15.152:0/3161095808 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f85d806cce0 0x7f85d806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:41.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.285+0000 7f85f5600640 1 -- 172.21.15.152:0/3161095808 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5709+0+0 (secure 0 0 0) 0x7f85f008d100 con 0x7f85f8141310 2024-04-20T11:34:41.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.285+0000 7f85fd800640 1 --2- 172.21.15.152:0/3161095808 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f85d806cce0 0x7f85d806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:41.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.286+0000 7f85fd800640 1 --2- 172.21.15.152:0/3161095808 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f85d806cce0 0x7f85d806f1a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f85ec038000 tx=0x7f85ec008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:41.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.289+0000 7f85f5600640 1 -- 172.21.15.152:0/3161095808 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f85f8072cb0 con 0x7f85f8141310 2024-04-20T11:34:41.471 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.471+0000 7f85fec00640 1 -- 172.21.15.152:0/3161095808 --> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f85f8075e60 con 0x7f85d806cce0 2024-04-20T11:34:41.548 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.490+0000 7f85f5600640 1 -- 172.21.15.152:0/3161095808 <== mgr.4109 v2:172.21.15.175:6836/4021250441 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252466 (secure 0 0 0) 0x7f85ec082a50 con 0x7f85d806cce0 2024-04-20T11:34:41.551 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:34:41.552 INFO:teuthology.orchestra.run.smithi152.stderr:dumped all 2024-04-20T11:34:41.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.495+0000 7f85fec00640 1 -- 172.21.15.152:0/3161095808 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f85d806cce0 msgr2=0x7f85d806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:41.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.495+0000 7f85fec00640 1 --2- 172.21.15.152:0/3161095808 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f85d806cce0 0x7f85d806f1a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f85ec038000 tx=0x7f85ec008040 comp rx=0 tx=0).stop 2024-04-20T11:34:41.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.495+0000 7f85fec00640 1 -- 172.21.15.152:0/3161095808 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f85f8141310 msgr2=0x7f85f813fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:41.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.495+0000 7f85fec00640 1 --2- 172.21.15.152:0/3161095808 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f85f8141310 0x7f85f813fb70 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f85f000cd50 tx=0x7f85f000a650 comp rx=0 tx=0).stop 2024-04-20T11:34:41.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.495+0000 7f85fec00640 1 -- 172.21.15.152:0/3161095808 shutdown_connections 2024-04-20T11:34:41.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.495+0000 7f85fec00640 1 --2- 172.21.15.152:0/3161095808 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f85d806cce0 0x7f85d806f1a0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:41.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.495+0000 7f85fec00640 1 --2- 172.21.15.152:0/3161095808 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f85f8141310 0x7f85f813fb70 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:41.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.495+0000 7f85fec00640 1 --2- 172.21.15.152:0/3161095808 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85f813e530 0x7f85f8153180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:41.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.495+0000 7f85fec00640 1 --2- 172.21.15.152:0/3161095808 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f85f806f340 0x7f85f813da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:41.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.495+0000 7f85fec00640 1 -- 172.21.15.152:0/3161095808 >> 172.21.15.152:0/3161095808 conn(0x7f85f8067ae0 msgr2=0x7f85f8159570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:41.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.495+0000 7f85fec00640 1 -- 172.21.15.152:0/3161095808 shutdown_connections 2024-04-20T11:34:41.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.495+0000 7f85fec00640 1 -- 172.21.15.152:0/3161095808 wait complete. 2024-04-20T11:34:41.555 INFO:teuthology.orchestra.run.smithi152.stdout:{"pg_ready":true,"pg_map":{"version":119,"stamp":"2024-04-20T11:34:40.646272+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":223,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":166,"ondisk_log_size":166,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":140,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":262168,"kb_used_data":12696,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754712552,"statfs":{"total":773094113280,"available":772825653248,"internally_reserved":0,"allocated":13000704,"data_stored":6368108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12746,"internal_metadata":255315510},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001757"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:37.592773+0000","last_change":"2024-04-20T11:34:36.635642+0000","last_active":"2024-04-20T11:34:37.592773+0000","last_peered":"2024-04-20T11:34:37.592773+0000","last_clean":"2024-04-20T11:34:37.592773+0000","last_became_active":"2024-04-20T11:34:36.635300+0000","last_became_peered":"2024-04-20T11:34:36.635300+0000","last_unstale":"2024-04-20T11:34:37.592773+0000","last_undegraded":"2024-04-20T11:34:37.592773+0000","last_fullsized":"2024-04-20T11:34:37.592773+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:52:31.695928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:38.211244+0000","last_change":"2024-04-20T11:34:36.626518+0000","last_active":"2024-04-20T11:34:38.211244+0000","last_peered":"2024-04-20T11:34:38.211244+0000","last_clean":"2024-04-20T11:34:38.211244+0000","last_became_active":"2024-04-20T11:34:36.626011+0000","last_became_peered":"2024-04-20T11:34:36.626011+0000","last_unstale":"2024-04-20T11:34:38.211244+0000","last_undegraded":"2024-04-20T11:34:38.211244+0000","last_fullsized":"2024-04-20T11:34:38.211244+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:09:01.470522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-20T11:34:36.621117+0000","last_change":"2024-04-20T11:34:36.618158+0000","last_active":"2024-04-20T11:34:36.621117+0000","last_peered":"2024-04-20T11:34:36.621117+0000","last_clean":"2024-04-20T11:34:36.621117+0000","last_became_active":"2024-04-20T11:34:36.617752+0000","last_became_peered":"2024-04-20T11:34:36.617752+0000","last_unstale":"2024-04-20T11:34:36.621117+0000","last_undegraded":"2024-04-20T11:34:36.621117+0000","last_fullsized":"2024-04-20T11:34:36.621117+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:09:01.470522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:37.592633+0000","last_change":"2024-04-20T11:34:36.616117+0000","last_active":"2024-04-20T11:34:37.592633+0000","last_peered":"2024-04-20T11:34:37.592633+0000","last_clean":"2024-04-20T11:34:37.592633+0000","last_became_active":"2024-04-20T11:34:36.615824+0000","last_became_peered":"2024-04-20T11:34:36.615824+0000","last_unstale":"2024-04-20T11:34:37.592633+0000","last_undegraded":"2024-04-20T11:34:37.592633+0000","last_fullsized":"2024-04-20T11:34:37.592633+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:07:11.853057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:37.633031+0000","last_change":"2024-04-20T11:34:36.627663+0000","last_active":"2024-04-20T11:34:37.633031+0000","last_peered":"2024-04-20T11:34:37.633031+0000","last_clean":"2024-04-20T11:34:37.633031+0000","last_became_active":"2024-04-20T11:34:36.627006+0000","last_became_peered":"2024-04-20T11:34:36.627006+0000","last_unstale":"2024-04-20T11:34:37.633031+0000","last_undegraded":"2024-04-20T11:34:37.633031+0000","last_fullsized":"2024-04-20T11:34:37.633031+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:32:52.568340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:38.484235+0000","last_change":"2024-04-20T11:34:36.618445+0000","last_active":"2024-04-20T11:34:38.484235+0000","last_peered":"2024-04-20T11:34:38.484235+0000","last_clean":"2024-04-20T11:34:38.484235+0000","last_became_active":"2024-04-20T11:34:36.618118+0000","last_became_peered":"2024-04-20T11:34:36.618118+0000","last_unstale":"2024-04-20T11:34:38.484235+0000","last_undegraded":"2024-04-20T11:34:38.484235+0000","last_fullsized":"2024-04-20T11:34:38.484235+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:58:16.330485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:38.483876+0000","last_change":"2024-04-20T11:34:36.622278+0000","last_active":"2024-04-20T11:34:38.483876+0000","last_peered":"2024-04-20T11:34:38.483876+0000","last_clean":"2024-04-20T11:34:38.483876+0000","last_became_active":"2024-04-20T11:34:36.621928+0000","last_became_peered":"2024-04-20T11:34:36.621928+0000","last_unstale":"2024-04-20T11:34:38.483876+0000","last_undegraded":"2024-04-20T11:34:38.483876+0000","last_fullsized":"2024-04-20T11:34:38.483876+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:11:38.577132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-20T11:34:36.635428+0000","last_change":"2024-04-20T11:34:36.623959+0000","last_active":"2024-04-20T11:34:36.635428+0000","last_peered":"2024-04-20T11:34:36.635428+0000","last_clean":"2024-04-20T11:34:36.635428+0000","last_became_active":"2024-04-20T11:34:36.623378+0000","last_became_peered":"2024-04-20T11:34:36.623378+0000","last_unstale":"2024-04-20T11:34:36.635428+0000","last_undegraded":"2024-04-20T11:34:36.635428+0000","last_fullsized":"2024-04-20T11:34:36.635428+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:11:14.001030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-20T11:34:36.637430+0000","last_change":"2024-04-20T11:34:36.631864+0000","last_active":"2024-04-20T11:34:36.637430+0000","last_peered":"2024-04-20T11:34:36.637430+0000","last_clean":"2024-04-20T11:34:36.637430+0000","last_became_active":"2024-04-20T11:34:36.631579+0000","last_became_peered":"2024-04-20T11:34:36.631579+0000","last_unstale":"2024-04-20T11:34:36.637430+0000","last_undegraded":"2024-04-20T11:34:36.637430+0000","last_fullsized":"2024-04-20T11:34:36.637430+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:26:57.597691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:38.211671+0000","last_change":"2024-04-20T11:34:36.626289+0000","last_active":"2024-04-20T11:34:38.211671+0000","last_peered":"2024-04-20T11:34:38.211671+0000","last_clean":"2024-04-20T11:34:38.211671+0000","last_became_active":"2024-04-20T11:34:36.625628+0000","last_became_peered":"2024-04-20T11:34:36.625628+0000","last_unstale":"2024-04-20T11:34:38.211671+0000","last_undegraded":"2024-04-20T11:34:38.211671+0000","last_fullsized":"2024-04-20T11:34:38.211671+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:12:30.998570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-20T11:34:36.617104+0000","last_change":"2024-04-20T11:34:36.613031+0000","last_active":"2024-04-20T11:34:36.617104+0000","last_peered":"2024-04-20T11:34:36.617104+0000","last_clean":"2024-04-20T11:34:36.617104+0000","last_became_active":"2024-04-20T11:34:36.612719+0000","last_became_peered":"2024-04-20T11:34:36.612719+0000","last_unstale":"2024-04-20T11:34:36.617104+0000","last_undegraded":"2024-04-20T11:34:36.617104+0000","last_fullsized":"2024-04-20T11:34:36.617104+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:02:52.819367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-20T11:34:36.638413+0000","last_change":"2024-04-20T11:34:36.633778+0000","last_active":"2024-04-20T11:34:36.638413+0000","last_peered":"2024-04-20T11:34:36.638413+0000","last_clean":"2024-04-20T11:34:36.638413+0000","last_became_active":"2024-04-20T11:34:36.633447+0000","last_became_peered":"2024-04-20T11:34:36.633447+0000","last_unstale":"2024-04-20T11:34:36.638413+0000","last_undegraded":"2024-04-20T11:34:36.638413+0000","last_fullsized":"2024-04-20T11:34:36.638413+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:12:30.998570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:37.593962+0000","last_change":"2024-04-20T11:34:36.622688+0000","last_active":"2024-04-20T11:34:37.593962+0000","last_peered":"2024-04-20T11:34:37.593962+0000","last_clean":"2024-04-20T11:34:37.593962+0000","last_became_active":"2024-04-20T11:34:36.622093+0000","last_became_peered":"2024-04-20T11:34:36.622093+0000","last_unstale":"2024-04-20T11:34:37.593962+0000","last_undegraded":"2024-04-20T11:34:37.593962+0000","last_fullsized":"2024-04-20T11:34:37.593962+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:26:00.441310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:38.211615+0000","last_change":"2024-04-20T11:34:36.609394+0000","last_active":"2024-04-20T11:34:38.211615+0000","last_peered":"2024-04-20T11:34:38.211615+0000","last_clean":"2024-04-20T11:34:38.211615+0000","last_became_active":"2024-04-20T11:34:36.609113+0000","last_became_peered":"2024-04-20T11:34:36.609113+0000","last_unstale":"2024-04-20T11:34:38.211615+0000","last_undegraded":"2024-04-20T11:34:38.211615+0000","last_fullsized":"2024-04-20T11:34:38.211615+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:26:57.597691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:37.591825+0000","last_change":"2024-04-20T11:34:36.618030+0000","last_active":"2024-04-20T11:34:37.591825+0000","last_peered":"2024-04-20T11:34:37.591825+0000","last_clean":"2024-04-20T11:34:37.591825+0000","last_became_active":"2024-04-20T11:34:36.617557+0000","last_became_peered":"2024-04-20T11:34:36.617557+0000","last_unstale":"2024-04-20T11:34:37.591825+0000","last_undegraded":"2024-04-20T11:34:37.591825+0000","last_fullsized":"2024-04-20T11:34:37.591825+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:56:31.623540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:37.597187+0000","last_change":"2024-04-20T11:34:36.609197+0000","last_active":"2024-04-20T11:34:37.597187+0000","last_peered":"2024-04-20T11:34:37.597187+0000","last_clean":"2024-04-20T11:34:37.597187+0000","last_became_active":"2024-04-20T11:34:36.608876+0000","last_became_peered":"2024-04-20T11:34:36.608876+0000","last_unstale":"2024-04-20T11:34:37.597187+0000","last_undegraded":"2024-04-20T11:34:37.597187+0000","last_fullsized":"2024-04-20T11:34:37.597187+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:57:35.372036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:38.484512+0000","last_change":"2024-04-20T11:34:36.620346+0000","last_active":"2024-04-20T11:34:38.484512+0000","last_peered":"2024-04-20T11:34:38.484512+0000","last_clean":"2024-04-20T11:34:38.484512+0000","last_became_active":"2024-04-20T11:34:36.620020+0000","last_became_peered":"2024-04-20T11:34:36.620020+0000","last_unstale":"2024-04-20T11:34:38.484512+0000","last_undegraded":"2024-04-20T11:34:38.484512+0000","last_fullsized":"2024-04-20T11:34:38.484512+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:32:52.568340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:37.598062+0000","last_change":"2024-04-20T11:34:36.625093+0000","last_active":"2024-04-20T11:34:37.598062+0000","last_peered":"2024-04-20T11:34:37.598062+0000","last_clean":"2024-04-20T11:34:37.598062+0000","last_became_active":"2024-04-20T11:34:36.624797+0000","last_became_peered":"2024-04-20T11:34:36.624797+0000","last_unstale":"2024-04-20T11:34:37.598062+0000","last_undegraded":"2024-04-20T11:34:37.598062+0000","last_fullsized":"2024-04-20T11:34:37.598062+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:45:55.429475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-20T11:34:36.622175+0000","last_change":"2024-04-20T11:34:36.619604+0000","last_active":"2024-04-20T11:34:36.622175+0000","last_peered":"2024-04-20T11:34:36.622175+0000","last_clean":"2024-04-20T11:34:36.622175+0000","last_became_active":"2024-04-20T11:34:36.619335+0000","last_became_peered":"2024-04-20T11:34:36.619335+0000","last_unstale":"2024-04-20T11:34:36.622175+0000","last_undegraded":"2024-04-20T11:34:36.622175+0000","last_fullsized":"2024-04-20T11:34:36.622175+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:17:02.644047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:37.597946+0000","last_change":"2024-04-20T11:34:36.613325+0000","last_active":"2024-04-20T11:34:37.597946+0000","last_peered":"2024-04-20T11:34:37.597946+0000","last_clean":"2024-04-20T11:34:37.597946+0000","last_became_active":"2024-04-20T11:34:36.613039+0000","last_became_peered":"2024-04-20T11:34:36.613039+0000","last_unstale":"2024-04-20T11:34:37.597946+0000","last_undegraded":"2024-04-20T11:34:37.597946+0000","last_fullsized":"2024-04-20T11:34:37.597946+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:15:12.163320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:37.632489+0000","last_change":"2024-04-20T11:34:36.627409+0000","last_active":"2024-04-20T11:34:37.632489+0000","last_peered":"2024-04-20T11:34:37.632489+0000","last_clean":"2024-04-20T11:34:37.632489+0000","last_became_active":"2024-04-20T11:34:36.626614+0000","last_became_peered":"2024-04-20T11:34:36.626614+0000","last_unstale":"2024-04-20T11:34:37.632489+0000","last_undegraded":"2024-04-20T11:34:37.632489+0000","last_fullsized":"2024-04-20T11:34:37.632489+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:28:42.034218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-20T11:34:36.614609+0000","last_change":"2024-04-20T11:34:36.611348+0000","last_active":"2024-04-20T11:34:36.614609+0000","last_peered":"2024-04-20T11:34:36.614609+0000","last_clean":"2024-04-20T11:34:36.614609+0000","last_became_active":"2024-04-20T11:34:36.611017+0000","last_became_peered":"2024-04-20T11:34:36.611017+0000","last_unstale":"2024-04-20T11:34:36.614609+0000","last_undegraded":"2024-04-20T11:34:36.614609+0000","last_fullsized":"2024-04-20T11:34:36.614609+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:51:17.501994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-20T11:34:36.638539+0000","last_change":"2024-04-20T11:34:36.634391+0000","last_active":"2024-04-20T11:34:36.638539+0000","last_peered":"2024-04-20T11:34:36.638539+0000","last_clean":"2024-04-20T11:34:36.638539+0000","last_became_active":"2024-04-20T11:34:36.634010+0000","last_became_peered":"2024-04-20T11:34:36.634010+0000","last_unstale":"2024-04-20T11:34:36.638539+0000","last_undegraded":"2024-04-20T11:34:36.638539+0000","last_fullsized":"2024-04-20T11:34:36.638539+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:15:10.532889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-20T11:34:36.603001+0000","last_change":"2024-04-20T11:34:36.600729+0000","last_active":"2024-04-20T11:34:36.603001+0000","last_peered":"2024-04-20T11:34:36.603001+0000","last_clean":"2024-04-20T11:34:36.603001+0000","last_became_active":"2024-04-20T11:34:36.600432+0000","last_became_peered":"2024-04-20T11:34:36.600432+0000","last_unstale":"2024-04-20T11:34:36.603001+0000","last_undegraded":"2024-04-20T11:34:36.603001+0000","last_fullsized":"2024-04-20T11:34:36.603001+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:24:26.808424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:38.211448+0000","last_change":"2024-04-20T11:34:36.613944+0000","last_active":"2024-04-20T11:34:38.211448+0000","last_peered":"2024-04-20T11:34:38.211448+0000","last_clean":"2024-04-20T11:34:38.211448+0000","last_became_active":"2024-04-20T11:34:36.613668+0000","last_became_peered":"2024-04-20T11:34:36.613668+0000","last_unstale":"2024-04-20T11:34:38.211448+0000","last_undegraded":"2024-04-20T11:34:38.211448+0000","last_fullsized":"2024-04-20T11:34:38.211448+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:11:14.001030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:37.598182+0000","last_change":"2024-04-20T11:34:36.616892+0000","last_active":"2024-04-20T11:34:37.598182+0000","last_peered":"2024-04-20T11:34:37.598182+0000","last_clean":"2024-04-20T11:34:37.598182+0000","last_became_active":"2024-04-20T11:34:36.616534+0000","last_became_peered":"2024-04-20T11:34:36.616534+0000","last_unstale":"2024-04-20T11:34:37.598182+0000","last_undegraded":"2024-04-20T11:34:37.598182+0000","last_fullsized":"2024-04-20T11:34:37.598182+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:33:13.548986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:37.592770+0000","last_change":"2024-04-20T11:34:36.623551+0000","last_active":"2024-04-20T11:34:37.592770+0000","last_peered":"2024-04-20T11:34:37.592770+0000","last_clean":"2024-04-20T11:34:37.592770+0000","last_became_active":"2024-04-20T11:34:36.622433+0000","last_became_peered":"2024-04-20T11:34:36.622433+0000","last_unstale":"2024-04-20T11:34:37.592770+0000","last_undegraded":"2024-04-20T11:34:37.592770+0000","last_fullsized":"2024-04-20T11:34:37.592770+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:13:36.429019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:37.598052+0000","last_change":"2024-04-20T11:34:36.621406+0000","last_active":"2024-04-20T11:34:37.598052+0000","last_peered":"2024-04-20T11:34:37.598052+0000","last_clean":"2024-04-20T11:34:37.598052+0000","last_became_active":"2024-04-20T11:34:36.619272+0000","last_became_peered":"2024-04-20T11:34:36.619272+0000","last_unstale":"2024-04-20T11:34:37.598052+0000","last_undegraded":"2024-04-20T11:34:37.598052+0000","last_fullsized":"2024-04-20T11:34:37.598052+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:33:59.359931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:37.593901+0000","last_change":"2024-04-20T11:34:36.621651+0000","last_active":"2024-04-20T11:34:37.593901+0000","last_peered":"2024-04-20T11:34:37.593901+0000","last_clean":"2024-04-20T11:34:37.593901+0000","last_became_active":"2024-04-20T11:34:36.619380+0000","last_became_peered":"2024-04-20T11:34:36.619380+0000","last_unstale":"2024-04-20T11:34:37.593901+0000","last_undegraded":"2024-04-20T11:34:37.593901+0000","last_fullsized":"2024-04-20T11:34:37.593901+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:06:56.343485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:37.592502+0000","last_change":"2024-04-20T11:34:36.616643+0000","last_active":"2024-04-20T11:34:37.592502+0000","last_peered":"2024-04-20T11:34:37.592502+0000","last_clean":"2024-04-20T11:34:37.592502+0000","last_became_active":"2024-04-20T11:34:36.616355+0000","last_became_peered":"2024-04-20T11:34:36.616355+0000","last_unstale":"2024-04-20T11:34:37.592502+0000","last_undegraded":"2024-04-20T11:34:37.592502+0000","last_fullsized":"2024-04-20T11:34:37.592502+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:32:56.437813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:37.632467+0000","last_change":"2024-04-20T11:34:36.630349+0000","last_active":"2024-04-20T11:34:37.632467+0000","last_peered":"2024-04-20T11:34:37.632467+0000","last_clean":"2024-04-20T11:34:37.632467+0000","last_became_active":"2024-04-20T11:34:36.630031+0000","last_became_peered":"2024-04-20T11:34:36.630031+0000","last_unstale":"2024-04-20T11:34:37.632467+0000","last_undegraded":"2024-04-20T11:34:37.632467+0000","last_fullsized":"2024-04-20T11:34:37.632467+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:40:30.539389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:38.483777+0000","last_change":"2024-04-20T11:34:36.624413+0000","last_active":"2024-04-20T11:34:38.483777+0000","last_peered":"2024-04-20T11:34:38.483777+0000","last_clean":"2024-04-20T11:34:38.483777+0000","last_became_active":"2024-04-20T11:34:36.624046+0000","last_became_peered":"2024-04-20T11:34:36.624046+0000","last_unstale":"2024-04-20T11:34:38.483777+0000","last_undegraded":"2024-04-20T11:34:38.483777+0000","last_fullsized":"2024-04-20T11:34:38.483777+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:06:04.637607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:37.598381+0000","last_change":"2024-04-20T11:34:36.615431+0000","last_active":"2024-04-20T11:34:37.598381+0000","last_peered":"2024-04-20T11:34:37.598381+0000","last_clean":"2024-04-20T11:34:37.598381+0000","last_became_active":"2024-04-20T11:34:36.615180+0000","last_became_peered":"2024-04-20T11:34:36.615180+0000","last_unstale":"2024-04-20T11:34:37.598381+0000","last_undegraded":"2024-04-20T11:34:37.598381+0000","last_fullsized":"2024-04-20T11:34:37.598381+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:32:08.712923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"117'4","reported_seq":18,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:37.633862+0000","last_change":"2024-04-20T11:34:35.622313+0000","last_active":"2024-04-20T11:34:37.633862+0000","last_peered":"2024-04-20T11:34:37.633862+0000","last_clean":"2024-04-20T11:34:37.633862+0000","last_became_active":"2024-04-20T11:34:35.621973+0000","last_became_peered":"2024-04-20T11:34:35.621973+0000","last_unstale":"2024-04-20T11:34:37.633862+0000","last_undegraded":"2024-04-20T11:34:37.633862+0000","last_fullsized":"2024-04-20T11:34:37.633862+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_clean_scrub_stamp":"2024-04-20T11:34:34.554915+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:57:35.086112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-20T11:34:36.616968+0000","last_change":"2024-04-20T11:34:36.615193+0000","last_active":"2024-04-20T11:34:36.616968+0000","last_peered":"2024-04-20T11:34:36.616968+0000","last_clean":"2024-04-20T11:34:36.616968+0000","last_became_active":"2024-04-20T11:34:36.614726+0000","last_became_peered":"2024-04-20T11:34:36.614726+0000","last_unstale":"2024-04-20T11:34:36.616968+0000","last_undegraded":"2024-04-20T11:34:36.616968+0000","last_fullsized":"2024-04-20T11:34:36.616968+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:32:13.140771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"117'1","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:37.598283+0000","last_change":"2024-04-20T11:34:35.608959+0000","last_active":"2024-04-20T11:34:37.598283+0000","last_peered":"2024-04-20T11:34:37.598283+0000","last_clean":"2024-04-20T11:34:37.598283+0000","last_became_active":"2024-04-20T11:34:35.607680+0000","last_became_peered":"2024-04-20T11:34:35.607680+0000","last_unstale":"2024-04-20T11:34:37.598283+0000","last_undegraded":"2024-04-20T11:34:37.598283+0000","last_fullsized":"2024-04-20T11:34:37.598283+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_clean_scrub_stamp":"2024-04-20T11:34:34.554915+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:53:43.135494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:37.633574+0000","last_change":"2024-04-20T11:34:36.626517+0000","last_active":"2024-04-20T11:34:37.633574+0000","last_peered":"2024-04-20T11:34:37.633574+0000","last_clean":"2024-04-20T11:34:37.633574+0000","last_became_active":"2024-04-20T11:34:36.626172+0000","last_became_peered":"2024-04-20T11:34:36.626172+0000","last_unstale":"2024-04-20T11:34:37.633574+0000","last_undegraded":"2024-04-20T11:34:37.633574+0000","last_fullsized":"2024-04-20T11:34:37.633574+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:12:50.192585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"117'2","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-20T11:34:36.626006+0000","last_change":"2024-04-20T11:34:35.613298+0000","last_active":"2024-04-20T11:34:36.626006+0000","last_peered":"2024-04-20T11:34:36.626006+0000","last_clean":"2024-04-20T11:34:36.626006+0000","last_became_active":"2024-04-20T11:34:35.612904+0000","last_became_peered":"2024-04-20T11:34:35.612904+0000","last_unstale":"2024-04-20T11:34:36.626006+0000","last_undegraded":"2024-04-20T11:34:36.626006+0000","last_fullsized":"2024-04-20T11:34:36.626006+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_clean_scrub_stamp":"2024-04-20T11:34:34.554915+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:09:59.218104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:37.633923+0000","last_change":"2024-04-20T11:34:36.624996+0000","last_active":"2024-04-20T11:34:37.633923+0000","last_peered":"2024-04-20T11:34:37.633923+0000","last_clean":"2024-04-20T11:34:37.633923+0000","last_became_active":"2024-04-20T11:34:36.623069+0000","last_became_peered":"2024-04-20T11:34:36.623069+0000","last_unstale":"2024-04-20T11:34:37.633923+0000","last_undegraded":"2024-04-20T11:34:37.633923+0000","last_fullsized":"2024-04-20T11:34:37.633923+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:58:27.089946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"117'2","reported_seq":16,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:38.211619+0000","last_change":"2024-04-20T11:34:35.607678+0000","last_active":"2024-04-20T11:34:38.211619+0000","last_peered":"2024-04-20T11:34:38.211619+0000","last_clean":"2024-04-20T11:34:38.211619+0000","last_became_active":"2024-04-20T11:34:35.606290+0000","last_became_peered":"2024-04-20T11:34:35.606290+0000","last_unstale":"2024-04-20T11:34:38.211619+0000","last_undegraded":"2024-04-20T11:34:38.211619+0000","last_fullsized":"2024-04-20T11:34:38.211619+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_clean_scrub_stamp":"2024-04-20T11:34:34.554915+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:50:25.165644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:37.597171+0000","last_change":"2024-04-20T11:34:36.625289+0000","last_active":"2024-04-20T11:34:37.597171+0000","last_peered":"2024-04-20T11:34:37.597171+0000","last_clean":"2024-04-20T11:34:37.597171+0000","last_became_active":"2024-04-20T11:34:36.624831+0000","last_became_peered":"2024-04-20T11:34:36.624831+0000","last_unstale":"2024-04-20T11:34:37.597171+0000","last_undegraded":"2024-04-20T11:34:37.597171+0000","last_fullsized":"2024-04-20T11:34:37.597171+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:12:50.192585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"117'1","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:38.210745+0000","last_change":"2024-04-20T11:34:35.604975+0000","last_active":"2024-04-20T11:34:38.210745+0000","last_peered":"2024-04-20T11:34:38.210745+0000","last_clean":"2024-04-20T11:34:38.210745+0000","last_became_active":"2024-04-20T11:34:35.604665+0000","last_became_peered":"2024-04-20T11:34:35.604665+0000","last_unstale":"2024-04-20T11:34:38.210745+0000","last_undegraded":"2024-04-20T11:34:38.210745+0000","last_fullsized":"2024-04-20T11:34:38.210745+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_clean_scrub_stamp":"2024-04-20T11:34:34.554915+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:28:46.049807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:38.484369+0000","last_change":"2024-04-20T11:34:36.620690+0000","last_active":"2024-04-20T11:34:38.484369+0000","last_peered":"2024-04-20T11:34:38.484369+0000","last_clean":"2024-04-20T11:34:38.484369+0000","last_became_active":"2024-04-20T11:34:36.618612+0000","last_became_peered":"2024-04-20T11:34:36.618612+0000","last_unstale":"2024-04-20T11:34:38.484369+0000","last_undegraded":"2024-04-20T11:34:38.484369+0000","last_fullsized":"2024-04-20T11:34:38.484369+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:40:30.539389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"117'1","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:37.597066+0000","last_change":"2024-04-20T11:34:35.607616+0000","last_active":"2024-04-20T11:34:37.597066+0000","last_peered":"2024-04-20T11:34:37.597066+0000","last_clean":"2024-04-20T11:34:37.597066+0000","last_became_active":"2024-04-20T11:34:35.607208+0000","last_became_peered":"2024-04-20T11:34:35.607208+0000","last_unstale":"2024-04-20T11:34:37.597066+0000","last_undegraded":"2024-04-20T11:34:37.597066+0000","last_fullsized":"2024-04-20T11:34:37.597066+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_clean_scrub_stamp":"2024-04-20T11:34:34.554915+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:57:37.447390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-20T11:34:36.635407+0000","last_change":"2024-04-20T11:34:36.623690+0000","last_active":"2024-04-20T11:34:36.635407+0000","last_peered":"2024-04-20T11:34:36.635407+0000","last_clean":"2024-04-20T11:34:36.635407+0000","last_became_active":"2024-04-20T11:34:36.622667+0000","last_became_peered":"2024-04-20T11:34:36.622667+0000","last_unstale":"2024-04-20T11:34:36.635407+0000","last_undegraded":"2024-04-20T11:34:36.635407+0000","last_fullsized":"2024-04-20T11:34:36.635407+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:51:39.025826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:38.483916+0000","last_change":"2024-04-20T11:34:35.622543+0000","last_active":"2024-04-20T11:34:38.483916+0000","last_peered":"2024-04-20T11:34:38.483916+0000","last_clean":"2024-04-20T11:34:38.483916+0000","last_became_active":"2024-04-20T11:34:35.622198+0000","last_became_peered":"2024-04-20T11:34:35.622198+0000","last_unstale":"2024-04-20T11:34:38.483916+0000","last_undegraded":"2024-04-20T11:34:38.483916+0000","last_fullsized":"2024-04-20T11:34:38.483916+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_clean_scrub_stamp":"2024-04-20T11:34:34.554915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:25:10.731059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:38.210906+0000","last_change":"2024-04-20T11:34:36.610956+0000","last_active":"2024-04-20T11:34:38.210906+0000","last_peered":"2024-04-20T11:34:38.210906+0000","last_clean":"2024-04-20T11:34:38.210906+0000","last_became_active":"2024-04-20T11:34:36.610029+0000","last_became_peered":"2024-04-20T11:34:36.610029+0000","last_unstale":"2024-04-20T11:34:38.210906+0000","last_undegraded":"2024-04-20T11:34:38.210906+0000","last_fullsized":"2024-04-20T11:34:38.210906+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:10:40.601449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:38.211227+0000","last_change":"2024-04-20T11:34:35.613214+0000","last_active":"2024-04-20T11:34:38.211227+0000","last_peered":"2024-04-20T11:34:38.211227+0000","last_clean":"2024-04-20T11:34:38.211227+0000","last_became_active":"2024-04-20T11:34:35.612894+0000","last_became_peered":"2024-04-20T11:34:35.612894+0000","last_unstale":"2024-04-20T11:34:38.211227+0000","last_undegraded":"2024-04-20T11:34:38.211227+0000","last_fullsized":"2024-04-20T11:34:38.211227+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_clean_scrub_stamp":"2024-04-20T11:34:34.554915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:32:27.780925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:37.592360+0000","last_change":"2024-04-20T11:34:36.603509+0000","last_active":"2024-04-20T11:34:37.592360+0000","last_peered":"2024-04-20T11:34:37.592360+0000","last_clean":"2024-04-20T11:34:37.592360+0000","last_became_active":"2024-04-20T11:34:36.603209+0000","last_became_peered":"2024-04-20T11:34:36.603209+0000","last_unstale":"2024-04-20T11:34:37.592360+0000","last_undegraded":"2024-04-20T11:34:37.592360+0000","last_fullsized":"2024-04-20T11:34:37.592360+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:47:27.221121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:37.598512+0000","last_change":"2024-04-20T11:34:35.610647+0000","last_active":"2024-04-20T11:34:37.598512+0000","last_peered":"2024-04-20T11:34:37.598512+0000","last_clean":"2024-04-20T11:34:37.598512+0000","last_became_active":"2024-04-20T11:34:35.606382+0000","last_became_peered":"2024-04-20T11:34:35.606382+0000","last_unstale":"2024-04-20T11:34:37.598512+0000","last_undegraded":"2024-04-20T11:34:37.598512+0000","last_fullsized":"2024-04-20T11:34:37.598512+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_clean_scrub_stamp":"2024-04-20T11:34:34.554915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:47:55.225440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:37.598255+0000","last_change":"2024-04-20T11:34:36.614740+0000","last_active":"2024-04-20T11:34:37.598255+0000","last_peered":"2024-04-20T11:34:37.598255+0000","last_clean":"2024-04-20T11:34:37.598255+0000","last_became_active":"2024-04-20T11:34:36.614424+0000","last_became_peered":"2024-04-20T11:34:36.614424+0000","last_unstale":"2024-04-20T11:34:37.598255+0000","last_undegraded":"2024-04-20T11:34:37.598255+0000","last_fullsized":"2024-04-20T11:34:37.598255+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:06:56.712443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:37.598341+0000","last_change":"2024-04-20T11:34:35.608494+0000","last_active":"2024-04-20T11:34:37.598341+0000","last_peered":"2024-04-20T11:34:37.598341+0000","last_clean":"2024-04-20T11:34:37.598341+0000","last_became_active":"2024-04-20T11:34:35.608058+0000","last_became_peered":"2024-04-20T11:34:35.608058+0000","last_unstale":"2024-04-20T11:34:37.598341+0000","last_undegraded":"2024-04-20T11:34:37.598341+0000","last_fullsized":"2024-04-20T11:34:37.598341+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_clean_scrub_stamp":"2024-04-20T11:34:34.554915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:27:10.390559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:37.633327+0000","last_change":"2024-04-20T11:34:36.627090+0000","last_active":"2024-04-20T11:34:37.633327+0000","last_peered":"2024-04-20T11:34:37.633327+0000","last_clean":"2024-04-20T11:34:37.633327+0000","last_became_active":"2024-04-20T11:34:36.626704+0000","last_became_peered":"2024-04-20T11:34:36.626704+0000","last_unstale":"2024-04-20T11:34:37.633327+0000","last_undegraded":"2024-04-20T11:34:37.633327+0000","last_fullsized":"2024-04-20T11:34:37.633327+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:17:52.125796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"117'3","reported_seq":17,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:37.597915+0000","last_change":"2024-04-20T11:34:35.601616+0000","last_active":"2024-04-20T11:34:37.597915+0000","last_peered":"2024-04-20T11:34:37.597915+0000","last_clean":"2024-04-20T11:34:37.597915+0000","last_became_active":"2024-04-20T11:34:35.601094+0000","last_became_peered":"2024-04-20T11:34:35.601094+0000","last_unstale":"2024-04-20T11:34:37.597915+0000","last_undegraded":"2024-04-20T11:34:37.597915+0000","last_fullsized":"2024-04-20T11:34:37.597915+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_clean_scrub_stamp":"2024-04-20T11:34:34.554915+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:50:25.165644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:37.633877+0000","last_change":"2024-04-20T11:34:36.621913+0000","last_active":"2024-04-20T11:34:37.633877+0000","last_peered":"2024-04-20T11:34:37.633877+0000","last_clean":"2024-04-20T11:34:37.633877+0000","last_became_active":"2024-04-20T11:34:36.621372+0000","last_became_peered":"2024-04-20T11:34:36.621372+0000","last_unstale":"2024-04-20T11:34:37.633877+0000","last_undegraded":"2024-04-20T11:34:37.633877+0000","last_fullsized":"2024-04-20T11:34:37.633877+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:57:35.372036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"117'6","reported_seq":18,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-20T11:34:36.641314+0000","last_change":"2024-04-20T11:34:35.619954+0000","last_active":"2024-04-20T11:34:36.641314+0000","last_peered":"2024-04-20T11:34:36.641314+0000","last_clean":"2024-04-20T11:34:36.641314+0000","last_became_active":"2024-04-20T11:34:35.619625+0000","last_became_peered":"2024-04-20T11:34:35.619625+0000","last_unstale":"2024-04-20T11:34:36.641314+0000","last_undegraded":"2024-04-20T11:34:36.641314+0000","last_fullsized":"2024-04-20T11:34:36.641314+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_clean_scrub_stamp":"2024-04-20T11:34:34.554915+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:10:39.587624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:37.594170+0000","last_change":"2024-04-20T11:34:36.603729+0000","last_active":"2024-04-20T11:34:37.594170+0000","last_peered":"2024-04-20T11:34:37.594170+0000","last_clean":"2024-04-20T11:34:37.594170+0000","last_became_active":"2024-04-20T11:34:36.603447+0000","last_became_peered":"2024-04-20T11:34:36.603447+0000","last_unstale":"2024-04-20T11:34:37.594170+0000","last_undegraded":"2024-04-20T11:34:37.594170+0000","last_fullsized":"2024-04-20T11:34:37.594170+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:59:02.109644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:37.593157+0000","last_change":"2024-04-20T11:34:35.611133+0000","last_active":"2024-04-20T11:34:37.593157+0000","last_peered":"2024-04-20T11:34:37.593157+0000","last_clean":"2024-04-20T11:34:37.593157+0000","last_became_active":"2024-04-20T11:34:35.610837+0000","last_became_peered":"2024-04-20T11:34:35.610837+0000","last_unstale":"2024-04-20T11:34:37.593157+0000","last_undegraded":"2024-04-20T11:34:37.593157+0000","last_fullsized":"2024-04-20T11:34:37.593157+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_clean_scrub_stamp":"2024-04-20T11:34:34.554915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:17:10.243879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:38.210906+0000","last_change":"2024-04-20T11:34:36.616647+0000","last_active":"2024-04-20T11:34:38.210906+0000","last_peered":"2024-04-20T11:34:38.210906+0000","last_clean":"2024-04-20T11:34:38.210906+0000","last_became_active":"2024-04-20T11:34:36.616341+0000","last_became_peered":"2024-04-20T11:34:36.616341+0000","last_unstale":"2024-04-20T11:34:38.210906+0000","last_undegraded":"2024-04-20T11:34:38.210906+0000","last_fullsized":"2024-04-20T11:34:38.210906+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:19:21.488409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:38.211479+0000","last_change":"2024-04-20T11:34:35.620071+0000","last_active":"2024-04-20T11:34:38.211479+0000","last_peered":"2024-04-20T11:34:38.211479+0000","last_clean":"2024-04-20T11:34:38.211479+0000","last_became_active":"2024-04-20T11:34:35.619745+0000","last_became_peered":"2024-04-20T11:34:35.619745+0000","last_unstale":"2024-04-20T11:34:38.211479+0000","last_undegraded":"2024-04-20T11:34:38.211479+0000","last_fullsized":"2024-04-20T11:34:38.211479+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_clean_scrub_stamp":"2024-04-20T11:34:34.554915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:53:43.135494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-20T11:34:36.638562+0000","last_change":"2024-04-20T11:34:36.634023+0000","last_active":"2024-04-20T11:34:36.638562+0000","last_peered":"2024-04-20T11:34:36.638562+0000","last_clean":"2024-04-20T11:34:36.638562+0000","last_became_active":"2024-04-20T11:34:36.633702+0000","last_became_peered":"2024-04-20T11:34:36.633702+0000","last_unstale":"2024-04-20T11:34:36.638562+0000","last_undegraded":"2024-04-20T11:34:36.638562+0000","last_fullsized":"2024-04-20T11:34:36.638562+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:59:18.372456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"117'8","reported_seq":20,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-20T11:34:36.639817+0000","last_change":"2024-04-20T11:34:35.615732+0000","last_active":"2024-04-20T11:34:36.639817+0000","last_peered":"2024-04-20T11:34:36.639817+0000","last_clean":"2024-04-20T11:34:36.639817+0000","last_became_active":"2024-04-20T11:34:35.615094+0000","last_became_peered":"2024-04-20T11:34:35.615094+0000","last_unstale":"2024-04-20T11:34:36.639817+0000","last_undegraded":"2024-04-20T11:34:36.639817+0000","last_fullsized":"2024-04-20T11:34:36.639817+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_clean_scrub_stamp":"2024-04-20T11:34:34.554915+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:19:01.727664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-20T11:34:36.637464+0000","last_change":"2024-04-20T11:34:36.632021+0000","last_active":"2024-04-20T11:34:36.637464+0000","last_peered":"2024-04-20T11:34:36.637464+0000","last_clean":"2024-04-20T11:34:36.637464+0000","last_became_active":"2024-04-20T11:34:36.631728+0000","last_became_peered":"2024-04-20T11:34:36.631728+0000","last_unstale":"2024-04-20T11:34:36.637464+0000","last_undegraded":"2024-04-20T11:34:36.637464+0000","last_fullsized":"2024-04-20T11:34:36.637464+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:23:45.660339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"117'1","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:37.598695+0000","last_change":"2024-04-20T11:34:35.619853+0000","last_active":"2024-04-20T11:34:37.598695+0000","last_peered":"2024-04-20T11:34:37.598695+0000","last_clean":"2024-04-20T11:34:37.598695+0000","last_became_active":"2024-04-20T11:34:35.619540+0000","last_became_peered":"2024-04-20T11:34:35.619540+0000","last_unstale":"2024-04-20T11:34:37.598695+0000","last_undegraded":"2024-04-20T11:34:37.598695+0000","last_fullsized":"2024-04-20T11:34:37.598695+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_clean_scrub_stamp":"2024-04-20T11:34:34.554915+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:19:48.867052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:37.598403+0000","last_change":"2024-04-20T11:34:36.602442+0000","last_active":"2024-04-20T11:34:37.598403+0000","last_peered":"2024-04-20T11:34:37.598403+0000","last_clean":"2024-04-20T11:34:37.598403+0000","last_became_active":"2024-04-20T11:34:36.601419+0000","last_became_peered":"2024-04-20T11:34:36.601419+0000","last_unstale":"2024-04-20T11:34:37.598403+0000","last_undegraded":"2024-04-20T11:34:37.598403+0000","last_fullsized":"2024-04-20T11:34:37.598403+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:57:06.686709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"117'2","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-20T11:34:36.634522+0000","last_change":"2024-04-20T11:34:35.621744+0000","last_active":"2024-04-20T11:34:36.634522+0000","last_peered":"2024-04-20T11:34:36.634522+0000","last_clean":"2024-04-20T11:34:36.634522+0000","last_became_active":"2024-04-20T11:34:35.621430+0000","last_became_peered":"2024-04-20T11:34:35.621430+0000","last_unstale":"2024-04-20T11:34:36.634522+0000","last_undegraded":"2024-04-20T11:34:36.634522+0000","last_fullsized":"2024-04-20T11:34:36.634522+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_clean_scrub_stamp":"2024-04-20T11:34:34.554915+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:44:24.876930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:37.598199+0000","last_change":"2024-04-20T11:34:36.609780+0000","last_active":"2024-04-20T11:34:37.598199+0000","last_peered":"2024-04-20T11:34:37.598199+0000","last_clean":"2024-04-20T11:34:37.598199+0000","last_became_active":"2024-04-20T11:34:36.608521+0000","last_became_peered":"2024-04-20T11:34:36.608521+0000","last_unstale":"2024-04-20T11:34:37.598199+0000","last_undegraded":"2024-04-20T11:34:37.598199+0000","last_fullsized":"2024-04-20T11:34:37.598199+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:09:01.413284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.d","version":"117'1","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:38.484629+0000","last_change":"2024-04-20T11:34:35.616950+0000","last_active":"2024-04-20T11:34:38.484629+0000","last_peered":"2024-04-20T11:34:38.484629+0000","last_clean":"2024-04-20T11:34:38.484629+0000","last_became_active":"2024-04-20T11:34:35.616595+0000","last_became_peered":"2024-04-20T11:34:35.616595+0000","last_unstale":"2024-04-20T11:34:38.484629+0000","last_undegraded":"2024-04-20T11:34:38.484629+0000","last_fullsized":"2024-04-20T11:34:38.484629+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_clean_scrub_stamp":"2024-04-20T11:34:34.554915+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:57:35.086112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:38.211229+0000","last_change":"2024-04-20T11:34:36.612368+0000","last_active":"2024-04-20T11:34:38.211229+0000","last_peered":"2024-04-20T11:34:38.211229+0000","last_clean":"2024-04-20T11:34:38.211229+0000","last_became_active":"2024-04-20T11:34:36.611025+0000","last_became_peered":"2024-04-20T11:34:36.611025+0000","last_unstale":"2024-04-20T11:34:38.211229+0000","last_undegraded":"2024-04-20T11:34:38.211229+0000","last_fullsized":"2024-04-20T11:34:38.211229+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:32:13.140771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:37.597881+0000","last_change":"2024-04-20T11:34:35.606708+0000","last_active":"2024-04-20T11:34:37.597881+0000","last_peered":"2024-04-20T11:34:37.597881+0000","last_clean":"2024-04-20T11:34:37.597881+0000","last_became_active":"2024-04-20T11:34:35.606275+0000","last_became_peered":"2024-04-20T11:34:35.606275+0000","last_unstale":"2024-04-20T11:34:37.597881+0000","last_undegraded":"2024-04-20T11:34:37.597881+0000","last_fullsized":"2024-04-20T11:34:37.597881+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_clean_scrub_stamp":"2024-04-20T11:34:34.554915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:22:57.415194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:37.592917+0000","last_change":"2024-04-20T11:34:36.635812+0000","last_active":"2024-04-20T11:34:37.592917+0000","last_peered":"2024-04-20T11:34:37.592917+0000","last_clean":"2024-04-20T11:34:37.592917+0000","last_became_active":"2024-04-20T11:34:36.635082+0000","last_became_peered":"2024-04-20T11:34:36.635082+0000","last_unstale":"2024-04-20T11:34:37.592917+0000","last_undegraded":"2024-04-20T11:34:37.592917+0000","last_fullsized":"2024-04-20T11:34:37.592917+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:06:42.203253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"117'2","reported_seq":16,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:37.597518+0000","last_change":"2024-04-20T11:34:35.609152+0000","last_active":"2024-04-20T11:34:37.597518+0000","last_peered":"2024-04-20T11:34:37.597518+0000","last_clean":"2024-04-20T11:34:37.597518+0000","last_became_active":"2024-04-20T11:34:35.607949+0000","last_became_peered":"2024-04-20T11:34:35.607949+0000","last_unstale":"2024-04-20T11:34:37.597518+0000","last_undegraded":"2024-04-20T11:34:37.597518+0000","last_fullsized":"2024-04-20T11:34:37.597518+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_clean_scrub_stamp":"2024-04-20T11:34:34.554915+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:40:21.726230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:38.484492+0000","last_change":"2024-04-20T11:34:36.622992+0000","last_active":"2024-04-20T11:34:38.484492+0000","last_peered":"2024-04-20T11:34:38.484492+0000","last_clean":"2024-04-20T11:34:38.484492+0000","last_became_active":"2024-04-20T11:34:36.622448+0000","last_became_peered":"2024-04-20T11:34:36.622448+0000","last_unstale":"2024-04-20T11:34:38.484492+0000","last_undegraded":"2024-04-20T11:34:38.484492+0000","last_fullsized":"2024-04-20T11:34:38.484492+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:28:42.034218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:37.597371+0000","last_change":"2024-04-20T11:34:35.610851+0000","last_active":"2024-04-20T11:34:37.597371+0000","last_peered":"2024-04-20T11:34:37.597371+0000","last_clean":"2024-04-20T11:34:37.597371+0000","last_became_active":"2024-04-20T11:34:35.610583+0000","last_became_peered":"2024-04-20T11:34:35.610583+0000","last_unstale":"2024-04-20T11:34:37.597371+0000","last_undegraded":"2024-04-20T11:34:37.597371+0000","last_fullsized":"2024-04-20T11:34:37.597371+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_clean_scrub_stamp":"2024-04-20T11:34:34.554915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:04:39.272610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:38.211000+0000","last_change":"2024-04-20T11:34:36.606406+0000","last_active":"2024-04-20T11:34:38.211000+0000","last_peered":"2024-04-20T11:34:38.211000+0000","last_clean":"2024-04-20T11:34:38.211000+0000","last_became_active":"2024-04-20T11:34:36.606111+0000","last_became_peered":"2024-04-20T11:34:36.606111+0000","last_unstale":"2024-04-20T11:34:38.211000+0000","last_undegraded":"2024-04-20T11:34:38.211000+0000","last_fullsized":"2024-04-20T11:34:38.211000+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:19:02.741488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:37.596838+0000","last_change":"2024-04-20T11:34:35.606580+0000","last_active":"2024-04-20T11:34:37.596838+0000","last_peered":"2024-04-20T11:34:37.596838+0000","last_clean":"2024-04-20T11:34:37.596838+0000","last_became_active":"2024-04-20T11:34:35.606321+0000","last_became_peered":"2024-04-20T11:34:35.606321+0000","last_unstale":"2024-04-20T11:34:37.596838+0000","last_undegraded":"2024-04-20T11:34:37.596838+0000","last_fullsized":"2024-04-20T11:34:37.596838+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_clean_scrub_stamp":"2024-04-20T11:34:34.554915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:15:51.824961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:37.632210+0000","last_change":"2024-04-20T11:34:36.629614+0000","last_active":"2024-04-20T11:34:37.632210+0000","last_peered":"2024-04-20T11:34:37.632210+0000","last_clean":"2024-04-20T11:34:37.632210+0000","last_became_active":"2024-04-20T11:34:36.628425+0000","last_became_peered":"2024-04-20T11:34:36.628425+0000","last_unstale":"2024-04-20T11:34:37.632210+0000","last_undegraded":"2024-04-20T11:34:37.632210+0000","last_fullsized":"2024-04-20T11:34:37.632210+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:11:38.577132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-20T11:34:36.590101+0000","last_change":"2024-04-20T11:34:35.618857+0000","last_active":"2024-04-20T11:34:36.590101+0000","last_peered":"2024-04-20T11:34:36.590101+0000","last_clean":"2024-04-20T11:34:36.590101+0000","last_became_active":"2024-04-20T11:34:35.618473+0000","last_became_peered":"2024-04-20T11:34:35.618473+0000","last_unstale":"2024-04-20T11:34:36.590101+0000","last_undegraded":"2024-04-20T11:34:36.590101+0000","last_fullsized":"2024-04-20T11:34:36.590101+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_clean_scrub_stamp":"2024-04-20T11:34:34.554915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:28:05.739229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:38.484719+0000","last_change":"2024-04-20T11:34:36.636756+0000","last_active":"2024-04-20T11:34:38.484719+0000","last_peered":"2024-04-20T11:34:38.484719+0000","last_clean":"2024-04-20T11:34:38.484719+0000","last_became_active":"2024-04-20T11:34:36.636404+0000","last_became_peered":"2024-04-20T11:34:36.636404+0000","last_unstale":"2024-04-20T11:34:38.484719+0000","last_undegraded":"2024-04-20T11:34:38.484719+0000","last_fullsized":"2024-04-20T11:34:38.484719+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:17:52.125796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.7","version":"117'2","reported_seq":16,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:37.593155+0000","last_change":"2024-04-20T11:34:35.608692+0000","last_active":"2024-04-20T11:34:37.593155+0000","last_peered":"2024-04-20T11:34:37.593155+0000","last_clean":"2024-04-20T11:34:37.593155+0000","last_became_active":"2024-04-20T11:34:35.608400+0000","last_became_peered":"2024-04-20T11:34:35.608400+0000","last_unstale":"2024-04-20T11:34:37.593155+0000","last_undegraded":"2024-04-20T11:34:37.593155+0000","last_fullsized":"2024-04-20T11:34:37.593155+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_clean_scrub_stamp":"2024-04-20T11:34:34.554915+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:32:04.614712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:37.593041+0000","last_change":"2024-04-20T11:34:36.615472+0000","last_active":"2024-04-20T11:34:37.593041+0000","last_peered":"2024-04-20T11:34:37.593041+0000","last_clean":"2024-04-20T11:34:37.593041+0000","last_became_active":"2024-04-20T11:34:36.615142+0000","last_became_peered":"2024-04-20T11:34:36.615142+0000","last_unstale":"2024-04-20T11:34:37.593041+0000","last_undegraded":"2024-04-20T11:34:37.593041+0000","last_fullsized":"2024-04-20T11:34:37.593041+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:35:52.566260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:37.633192+0000","last_change":"2024-04-20T11:34:35.624251+0000","last_active":"2024-04-20T11:34:37.633192+0000","last_peered":"2024-04-20T11:34:37.633192+0000","last_clean":"2024-04-20T11:34:37.633192+0000","last_became_active":"2024-04-20T11:34:35.623867+0000","last_became_peered":"2024-04-20T11:34:35.623867+0000","last_unstale":"2024-04-20T11:34:37.633192+0000","last_undegraded":"2024-04-20T11:34:37.633192+0000","last_fullsized":"2024-04-20T11:34:37.633192+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_clean_scrub_stamp":"2024-04-20T11:34:34.554915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:03:31.045549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:38.211667+0000","last_change":"2024-04-20T11:34:36.620993+0000","last_active":"2024-04-20T11:34:38.211667+0000","last_peered":"2024-04-20T11:34:38.211667+0000","last_clean":"2024-04-20T11:34:38.211667+0000","last_became_active":"2024-04-20T11:34:36.618315+0000","last_became_peered":"2024-04-20T11:34:36.618315+0000","last_unstale":"2024-04-20T11:34:38.211667+0000","last_undegraded":"2024-04-20T11:34:38.211667+0000","last_fullsized":"2024-04-20T11:34:38.211667+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:23:45.660339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:38.211247+0000","last_change":"2024-04-20T11:34:35.617625+0000","last_active":"2024-04-20T11:34:38.211247+0000","last_peered":"2024-04-20T11:34:38.211247+0000","last_clean":"2024-04-20T11:34:38.211247+0000","last_became_active":"2024-04-20T11:34:35.617320+0000","last_became_peered":"2024-04-20T11:34:35.617320+0000","last_unstale":"2024-04-20T11:34:38.211247+0000","last_undegraded":"2024-04-20T11:34:38.211247+0000","last_fullsized":"2024-04-20T11:34:38.211247+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_clean_scrub_stamp":"2024-04-20T11:34:34.554915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:40:21.726230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:38.484503+0000","last_change":"2024-04-20T11:34:36.635337+0000","last_active":"2024-04-20T11:34:38.484503+0000","last_peered":"2024-04-20T11:34:38.484503+0000","last_clean":"2024-04-20T11:34:38.484503+0000","last_became_active":"2024-04-20T11:34:36.634947+0000","last_became_peered":"2024-04-20T11:34:36.634947+0000","last_unstale":"2024-04-20T11:34:38.484503+0000","last_undegraded":"2024-04-20T11:34:38.484503+0000","last_fullsized":"2024-04-20T11:34:38.484503+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:34:54.762775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-20T11:34:36.591565+0000","last_change":"2024-04-20T11:34:35.606436+0000","last_active":"2024-04-20T11:34:36.591565+0000","last_peered":"2024-04-20T11:34:36.591565+0000","last_clean":"2024-04-20T11:34:36.591565+0000","last_became_active":"2024-04-20T11:34:35.606114+0000","last_became_peered":"2024-04-20T11:34:35.606114+0000","last_unstale":"2024-04-20T11:34:36.591565+0000","last_undegraded":"2024-04-20T11:34:36.591565+0000","last_fullsized":"2024-04-20T11:34:36.591565+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_clean_scrub_stamp":"2024-04-20T11:34:34.554915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:07:27.769385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:37.632798+0000","last_change":"2024-04-20T11:34:36.628297+0000","last_active":"2024-04-20T11:34:37.632798+0000","last_peered":"2024-04-20T11:34:37.632798+0000","last_clean":"2024-04-20T11:34:37.632798+0000","last_became_active":"2024-04-20T11:34:36.627978+0000","last_became_peered":"2024-04-20T11:34:36.627978+0000","last_unstale":"2024-04-20T11:34:37.632798+0000","last_undegraded":"2024-04-20T11:34:37.632798+0000","last_fullsized":"2024-04-20T11:34:37.632798+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:34:54.762775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:38.211214+0000","last_change":"2024-04-20T11:34:35.622211+0000","last_active":"2024-04-20T11:34:38.211214+0000","last_peered":"2024-04-20T11:34:38.211214+0000","last_clean":"2024-04-20T11:34:38.211214+0000","last_became_active":"2024-04-20T11:34:35.621869+0000","last_became_peered":"2024-04-20T11:34:35.621869+0000","last_unstale":"2024-04-20T11:34:38.211214+0000","last_undegraded":"2024-04-20T11:34:38.211214+0000","last_fullsized":"2024-04-20T11:34:38.211214+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_clean_scrub_stamp":"2024-04-20T11:34:34.554915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:26:01.834165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:37.597600+0000","last_change":"2024-04-20T11:34:36.610415+0000","last_active":"2024-04-20T11:34:37.597600+0000","last_peered":"2024-04-20T11:34:37.597600+0000","last_clean":"2024-04-20T11:34:37.597600+0000","last_became_active":"2024-04-20T11:34:36.610052+0000","last_became_peered":"2024-04-20T11:34:36.610052+0000","last_unstale":"2024-04-20T11:34:37.597600+0000","last_undegraded":"2024-04-20T11:34:37.597600+0000","last_fullsized":"2024-04-20T11:34:37.597600+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:52:00.000858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:37.633636+0000","last_change":"2024-04-20T11:34:35.622833+0000","last_active":"2024-04-20T11:34:37.633636+0000","last_peered":"2024-04-20T11:34:37.633636+0000","last_clean":"2024-04-20T11:34:37.633636+0000","last_became_active":"2024-04-20T11:34:35.621851+0000","last_became_peered":"2024-04-20T11:34:35.621851+0000","last_unstale":"2024-04-20T11:34:37.633636+0000","last_undegraded":"2024-04-20T11:34:37.633636+0000","last_fullsized":"2024-04-20T11:34:37.633636+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_clean_scrub_stamp":"2024-04-20T11:34:34.554915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:39:56.418943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:37.597482+0000","last_change":"2024-04-20T11:34:36.621243+0000","last_active":"2024-04-20T11:34:37.597482+0000","last_peered":"2024-04-20T11:34:37.597482+0000","last_clean":"2024-04-20T11:34:37.597482+0000","last_became_active":"2024-04-20T11:34:36.620968+0000","last_became_peered":"2024-04-20T11:34:36.620968+0000","last_unstale":"2024-04-20T11:34:37.597482+0000","last_undegraded":"2024-04-20T11:34:37.597482+0000","last_fullsized":"2024-04-20T11:34:37.597482+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:58:27.089946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:37.597053+0000","last_change":"2024-04-20T11:34:35.611518+0000","last_active":"2024-04-20T11:34:37.597053+0000","last_peered":"2024-04-20T11:34:37.597053+0000","last_clean":"2024-04-20T11:34:37.597053+0000","last_became_active":"2024-04-20T11:34:35.609737+0000","last_became_peered":"2024-04-20T11:34:35.609737+0000","last_unstale":"2024-04-20T11:34:37.597053+0000","last_undegraded":"2024-04-20T11:34:37.597053+0000","last_fullsized":"2024-04-20T11:34:37.597053+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_clean_scrub_stamp":"2024-04-20T11:34:34.554915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:29:58.591093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:38.211261+0000","last_change":"2024-04-20T11:34:36.614617+0000","last_active":"2024-04-20T11:34:38.211261+0000","last_peered":"2024-04-20T11:34:38.211261+0000","last_clean":"2024-04-20T11:34:38.211261+0000","last_became_active":"2024-04-20T11:34:36.614320+0000","last_became_peered":"2024-04-20T11:34:36.614320+0000","last_unstale":"2024-04-20T11:34:38.211261+0000","last_undegraded":"2024-04-20T11:34:38.211261+0000","last_fullsized":"2024-04-20T11:34:38.211261+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:51:39.025826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-20T11:34:36.590317+0000","last_change":"2024-04-20T11:34:35.613736+0000","last_active":"2024-04-20T11:34:36.590317+0000","last_peered":"2024-04-20T11:34:36.590317+0000","last_clean":"2024-04-20T11:34:36.590317+0000","last_became_active":"2024-04-20T11:34:35.612303+0000","last_became_peered":"2024-04-20T11:34:35.612303+0000","last_unstale":"2024-04-20T11:34:36.590317+0000","last_undegraded":"2024-04-20T11:34:36.590317+0000","last_fullsized":"2024-04-20T11:34:36.590317+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_clean_scrub_stamp":"2024-04-20T11:34:34.554915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:54:21.076905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:37.632380+0000","last_change":"2024-04-20T11:34:36.630553+0000","last_active":"2024-04-20T11:34:37.632380+0000","last_peered":"2024-04-20T11:34:37.632380+0000","last_clean":"2024-04-20T11:34:37.632380+0000","last_became_active":"2024-04-20T11:34:36.630239+0000","last_became_peered":"2024-04-20T11:34:36.630239+0000","last_unstale":"2024-04-20T11:34:37.632380+0000","last_undegraded":"2024-04-20T11:34:37.632380+0000","last_fullsized":"2024-04-20T11:34:37.632380+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:35.568740+0000","last_clean_scrub_stamp":"2024-04-20T11:34:35.568740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:58:16.330485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"117'2","reported_seq":16,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:37.633357+0000","last_change":"2024-04-20T11:34:35.623699+0000","last_active":"2024-04-20T11:34:37.633357+0000","last_peered":"2024-04-20T11:34:37.633357+0000","last_clean":"2024-04-20T11:34:37.633357+0000","last_became_active":"2024-04-20T11:34:35.623391+0000","last_became_peered":"2024-04-20T11:34:35.623391+0000","last_unstale":"2024-04-20T11:34:37.633357+0000","last_undegraded":"2024-04-20T11:34:37.633357+0000","last_fullsized":"2024-04-20T11:34:37.633357+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:34:34.554915+0000","last_clean_scrub_stamp":"2024-04-20T11:34:34.554915+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:44:14.415607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"72'128","reported_seq":384,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-20T11:34:37.631852+0000","last_change":"2024-04-20T11:26:48.661897+0000","last_active":"2024-04-20T11:34:37.631852+0000","last_peered":"2024-04-20T11:34:37.631852+0000","last_clean":"2024-04-20T11:34:37.631852+0000","last_became_active":"2024-04-20T11:26:48.656535+0000","last_became_peered":"2024-04-20T11:26:48.656535+0000","last_unstale":"2024-04-20T11:34:37.631852+0000","last_undegraded":"2024-04-20T11:34:37.631852+0000","last_fullsized":"2024-04-20T11:34:37.631852+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:47.543004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:47.543004+0000","last_clean_scrub_stamp":"2024-04-20T11:26:47.543004+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:58:35.527105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574948,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33224,"kb_used_data":2044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338616,"statfs":{"total":96636764160,"available":96602742784,"internally_reserved":0,"allocated":2093056,"data_stored":1243587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574945,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32620,"kb_used_data":1440,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339220,"statfs":{"total":96636764160,"available":96603361280,"internally_reserved":0,"allocated":1474560,"data_stored":646281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574947,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32628,"kb_used_data":1448,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339212,"statfs":{"total":96636764160,"available":96603353088,"internally_reserved":0,"allocated":1482752,"data_stored":651081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574945,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32620,"kb_used_data":1432,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339220,"statfs":{"total":96636764160,"available":96603361280,"internally_reserved":0,"allocated":1466368,"data_stored":644739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574946,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32632,"kb_used_data":1448,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339208,"statfs":{"total":96636764160,"available":96603348992,"internally_reserved":0,"allocated":1482752,"data_stored":651129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574947,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32632,"kb_used_data":1448,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339208,"statfs":{"total":96636764160,"available":96603348992,"internally_reserved":0,"allocated":1482752,"data_stored":651585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574945,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32616,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339224,"statfs":{"total":96636764160,"available":96603365376,"internally_reserved":0,"allocated":1462272,"data_stored":644669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574946,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33196,"kb_used_data":2008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338644,"statfs":{"total":96636764160,"available":96602771456,"internally_reserved":0,"allocated":2056192,"data_stored":1235037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T11:34:41.584 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-20T11:34:41.584 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-20T11:34:41.805 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.805+0000 7f02d6600640 1 Processor -- start 2024-04-20T11:34:41.805 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.806+0000 7f02d6600640 1 -- start start 2024-04-20T11:34:41.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.807+0000 7f02d6600640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f02d0161c20 0x7f02d00d5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:41.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.807+0000 7f02d6600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f02d00d0f80 0x7f02d00d13f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:41.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.807+0000 7f02d6600640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02d00d0400 con 0x7f02d00cfcb0 2024-04-20T11:34:41.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.807+0000 7f02d6600640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f02d00d1930 con 0x7f02d0161c20 2024-04-20T11:34:41.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.807+0000 7f02d6600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f02d00d2160 con 0x7f02d00d0f80 2024-04-20T11:34:41.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.807+0000 7f02d5200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f02d0161c20 0x7f02d00d5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:41.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.807+0000 7f02d5200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f02d0161c20 0x7f02d00d5060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:50864/0 (socket says 172.21.15.152:50864) 2024-04-20T11:34:41.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.807+0000 7f02d5200640 1 -- 172.21.15.152:0/3104504826 learned_addr learned my addr 172.21.15.152:0/3104504826 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:41.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.808+0000 7f02d5c00640 1 --2- 172.21.15.152:0/3104504826 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f02d00d0f80 0x7f02d00d13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:41.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.808+0000 7f02d5200640 1 -- 172.21.15.152:0/3104504826 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f02d00d0f80 msgr2=0x7f02d00d13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:41.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.808+0000 7f02d5200640 1 --2- 172.21.15.152:0/3104504826 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f02d00d0f80 0x7f02d00d13f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:41.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.808+0000 7f02d5200640 1 -- 172.21.15.152:0/3104504826 >> v1:172.21.15.175:6789/0 conn(0x7f02d00cfcb0 legacy=0x7f02d00d2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:41.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.808+0000 7f02d5200640 1 -- 172.21.15.152:0/3104504826 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02d00d23b0 con 0x7f02d0161c20 2024-04-20T11:34:41.809 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.808+0000 7f02d5c00640 1 --2- 172.21.15.152:0/3104504826 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f02d00d0f80 0x7f02d00d13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:34:41.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.809+0000 7f02d5200640 1 --2- 172.21.15.152:0/3104504826 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f02d0161c20 0x7f02d00d5060 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f02d008d0f0 tx=0x7f02b802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1919a0f3dec5bdee server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:41.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.809+0000 7f02cf400640 1 -- 172.21.15.152:0/3104504826 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02b80315a0 con 0x7f02d0161c20 2024-04-20T11:34:41.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.809+0000 7f02cf400640 1 -- 172.21.15.152:0/3104504826 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02b8037070 con 0x7f02d0161c20 2024-04-20T11:34:41.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.809+0000 7f02cf400640 1 -- 172.21.15.152:0/3104504826 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02b8031960 con 0x7f02d0161c20 2024-04-20T11:34:41.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.810+0000 7f02d6600640 1 -- 172.21.15.152:0/3104504826 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f02d0161c20 msgr2=0x7f02d00d5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:41.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.810+0000 7f02d6600640 1 --2- 172.21.15.152:0/3104504826 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f02d0161c20 0x7f02d00d5060 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f02d008d0f0 tx=0x7f02b802b840 comp rx=0 tx=0).stop 2024-04-20T11:34:41.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.810+0000 7f02d6600640 1 -- 172.21.15.152:0/3104504826 shutdown_connections 2024-04-20T11:34:41.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.810+0000 7f02d6600640 1 --2- 172.21.15.152:0/3104504826 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f02d00d0f80 0x7f02d00d13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:41.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.810+0000 7f02d6600640 1 --2- 172.21.15.152:0/3104504826 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f02d0161c20 0x7f02d00d5060 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:41.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.810+0000 7f02d6600640 1 -- 172.21.15.152:0/3104504826 >> 172.21.15.152:0/3104504826 conn(0x7f02d008bfd0 msgr2=0x7f02d00d4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:41.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.810+0000 7f02d6600640 1 -- 172.21.15.152:0/3104504826 shutdown_connections 2024-04-20T11:34:41.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.810+0000 7f02d6600640 1 -- 172.21.15.152:0/3104504826 wait complete. 2024-04-20T11:34:41.812 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.812+0000 7f02d6600640 1 Processor -- start 2024-04-20T11:34:41.812 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.812+0000 7f02d6600640 1 -- start start 2024-04-20T11:34:41.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.813+0000 7f02d6600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02d00cfcb0 0x7f02d015beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:41.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.813+0000 7f02d6600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f02d00d0f80 0x7f02d00c9b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:41.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.813+0000 7f02d6600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f02d0161c20 0x7f02d00ca060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:41.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.813+0000 7f02d6600640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f02d00d0120 con 0x7f02d0161c20 2024-04-20T11:34:41.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.813+0000 7f02d6600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f02d00d2a00 con 0x7f02d00cfcb0 2024-04-20T11:34:41.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.813+0000 7f02d6600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f02d00d35a0 con 0x7f02d00d0f80 2024-04-20T11:34:41.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.814+0000 7f02d5c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f02d0161c20 0x7f02d00ca060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:41.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.814+0000 7f02d5c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f02d0161c20 0x7f02d00ca060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:50868/0 (socket says 172.21.15.152:50868) 2024-04-20T11:34:41.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.814+0000 7f02d5c00640 1 -- 172.21.15.152:0/1658144362 learned_addr learned my addr 172.21.15.152:0/1658144362 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:41.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.814+0000 7f02d5200640 1 --2- 172.21.15.152:0/1658144362 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02d00cfcb0 0x7f02d015beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:41.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.814+0000 7f02cfe00640 1 --2- 172.21.15.152:0/1658144362 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f02d00d0f80 0x7f02d00c9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:41.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.814+0000 7f02d5c00640 1 -- 172.21.15.152:0/1658144362 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f02d00d0f80 msgr2=0x7f02d00c9b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:41.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.814+0000 7f02d5c00640 1 --2- 172.21.15.152:0/1658144362 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f02d00d0f80 0x7f02d00c9b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:41.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.814+0000 7f02d5c00640 1 -- 172.21.15.152:0/1658144362 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02d00cfcb0 msgr2=0x7f02d015beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:41.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.814+0000 7f02d5c00640 1 --2- 172.21.15.152:0/1658144362 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02d00cfcb0 0x7f02d015beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:41.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.814+0000 7f02d5c00640 1 -- 172.21.15.152:0/1658144362 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02d015d260 con 0x7f02d0161c20 2024-04-20T11:34:41.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.814+0000 7f02d5200640 1 --2- 172.21.15.152:0/1658144362 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02d00cfcb0 0x7f02d015beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:34:41.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.814+0000 7f02d5c00640 1 --2- 172.21.15.152:0/1658144362 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f02d0161c20 0x7f02d00ca060 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f02c800c7f0 tx=0x7f02c800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:41.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.815+0000 7f02ccc00640 1 -- 172.21.15.152:0/1658144362 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02c8015070 con 0x7f02d0161c20 2024-04-20T11:34:41.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.815+0000 7f02ccc00640 1 -- 172.21.15.152:0/1658144362 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02c8007820 con 0x7f02d0161c20 2024-04-20T11:34:41.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.815+0000 7f02d6600640 1 -- 172.21.15.152:0/1658144362 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02d015d540 con 0x7f02d0161c20 2024-04-20T11:34:41.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.815+0000 7f02ccc00640 1 -- 172.21.15.152:0/1658144362 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02c8007c10 con 0x7f02d0161c20 2024-04-20T11:34:41.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.815+0000 7f02d6600640 1 -- 172.21.15.152:0/1658144362 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02d015db00 con 0x7f02d0161c20 2024-04-20T11:34:41.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.816+0000 7f02d6600640 1 -- 172.21.15.152:0/1658144362 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02d0075e60 con 0x7f02d0161c20 2024-04-20T11:34:41.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.817+0000 7f02ccc00640 1 -- 172.21.15.152:0/1658144362 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 19) ==== 91397+0+0 (secure 0 0 0) 0x7f02c8018070 con 0x7f02d0161c20 2024-04-20T11:34:41.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.818+0000 7f02ccc00640 1 --2- 172.21.15.152:0/1658144362 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f02ac06cce0 0x7f02ac06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:41.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.818+0000 7f02ccc00640 1 -- 172.21.15.152:0/1658144362 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5709+0+0 (secure 0 0 0) 0x7f02c808c4b0 con 0x7f02d0161c20 2024-04-20T11:34:41.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.818+0000 7f02d5200640 1 --2- 172.21.15.152:0/1658144362 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f02ac06cce0 0x7f02ac06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:41.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.820+0000 7f02d5200640 1 --2- 172.21.15.152:0/1658144362 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f02ac06cce0 0x7f02ac06f1a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f02b8036000 tx=0x7f02b8008030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:41.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:41.823+0000 7f02ccc00640 1 -- 172.21.15.152:0/1658144362 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f02c805a280 con 0x7f02d0161c20 2024-04-20T11:34:42.091 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:42.091+0000 7f02d6600640 1 -- 172.21.15.152:0/1658144362 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f02d00d1c80 con 0x7f02d0161c20 2024-04-20T11:34:42.096 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:42.096+0000 7f02ccc00640 1 -- 172.21.15.152:0/1658144362 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v18) ==== 190+0+147 (secure 0 0 0) 0x7f02c8059b80 con 0x7f02d0161c20 2024-04-20T11:34:42.096 INFO:teuthology.orchestra.run.smithi152.stdout:[client.0] 2024-04-20T11:34:42.096 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQBDpiNmtsBWFhAArq9A7r33CcwgMmOU2pAGPw== 2024-04-20T11:34:42.097 INFO:teuthology.orchestra.run.smithi152.stdout: caps mds = "allow" 2024-04-20T11:34:42.097 INFO:teuthology.orchestra.run.smithi152.stdout: caps mon = "allow r" 2024-04-20T11:34:42.097 INFO:teuthology.orchestra.run.smithi152.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-20T11:34:42.097 INFO:teuthology.orchestra.run.smithi152.stderr:updated caps for client.0 2024-04-20T11:34:42.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:42.099+0000 7f02d6600640 1 -- 172.21.15.152:0/1658144362 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f02ac06cce0 msgr2=0x7f02ac06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:42.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:42.099+0000 7f02d6600640 1 --2- 172.21.15.152:0/1658144362 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f02ac06cce0 0x7f02ac06f1a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f02b8036000 tx=0x7f02b8008030 comp rx=0 tx=0).stop 2024-04-20T11:34:42.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:42.099+0000 7f02d6600640 1 -- 172.21.15.152:0/1658144362 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f02d0161c20 msgr2=0x7f02d00ca060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:42.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:42.099+0000 7f02d6600640 1 --2- 172.21.15.152:0/1658144362 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f02d0161c20 0x7f02d00ca060 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f02c800c7f0 tx=0x7f02c800ccb0 comp rx=0 tx=0).stop 2024-04-20T11:34:42.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:42.099+0000 7f02d6600640 1 -- 172.21.15.152:0/1658144362 shutdown_connections 2024-04-20T11:34:42.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:42.099+0000 7f02d6600640 1 --2- 172.21.15.152:0/1658144362 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f02ac06cce0 0x7f02ac06f1a0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:42.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:42.099+0000 7f02d6600640 1 --2- 172.21.15.152:0/1658144362 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f02d0161c20 0x7f02d00ca060 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:42.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:42.099+0000 7f02d6600640 1 --2- 172.21.15.152:0/1658144362 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f02d00d0f80 0x7f02d00c9b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:42.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:42.099+0000 7f02d6600640 1 --2- 172.21.15.152:0/1658144362 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02d00cfcb0 0x7f02d015beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:42.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:42.099+0000 7f02d6600640 1 -- 172.21.15.152:0/1658144362 >> 172.21.15.152:0/1658144362 conn(0x7f02d008bfd0 msgr2=0x7f02d00d4f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:42.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:42.099+0000 7f02d6600640 1 -- 172.21.15.152:0/1658144362 shutdown_connections 2024-04-20T11:34:42.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:42.099+0000 7f02d6600640 1 -- 172.21.15.152:0/1658144362 wait complete. 2024-04-20T11:34:42.119 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:34:42.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:42.300+0000 7f6a21e00640 1 Processor -- start 2024-04-20T11:34:42.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:42.301+0000 7f6a21e00640 1 -- start start 2024-04-20T11:34:42.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:42.302+0000 7f6a21e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6a1c0b4500 0x7f6a1c0d0100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:42.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:42.302+0000 7f6a21e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f6a1c0d0640 0x7f6a1c0ada90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:42.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:42.302+0000 7f6a21e00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a1c15fd20 con 0x7f6a1c15c560 2024-04-20T11:34:42.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:42.302+0000 7f6a21e00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f6a1c15ef60 con 0x7f6a1c0d0640 2024-04-20T11:34:42.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:42.302+0000 7f6a21e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6a1c15f240 con 0x7f6a1c0b4500 2024-04-20T11:34:42.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:42.302+0000 7f6a1b400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f6a1c0d0640 0x7f6a1c0ada90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:42.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:42.303+0000 7f6a1aa00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f6a1c15c560 0x7f6a1c0b3df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:41196/0 (socket says 172.21.15.152:41196) 2024-04-20T11:34:42.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:42.303+0000 7f6a1aa00640 1 -- 172.21.15.152:0/3294536343 learned_addr learned my addr 172.21.15.152:0/3294536343 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:34:42.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:42.303+0000 7f6a13e00640 1 --2- 172.21.15.152:0/3294536343 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6a1c0b4500 0x7f6a1c0d0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:42.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:42.303+0000 7f6a1b400640 1 -- 172.21.15.152:0/3294536343 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6a1c0b4500 msgr2=0x7f6a1c0d0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:42.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:42.303+0000 7f6a1b400640 1 --2- 172.21.15.152:0/3294536343 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6a1c0b4500 0x7f6a1c0d0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:42.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:42.303+0000 7f6a1b400640 1 -- 172.21.15.152:0/3294536343 >> v1:172.21.15.152:6789/0 conn(0x7f6a1c15c560 legacy=0x7f6a1c0b3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:42.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:42.303+0000 7f6a1b400640 1 -- 172.21.15.152:0/3294536343 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a1c15f520 con 0x7f6a1c0d0640 2024-04-20T11:34:42.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:42.303+0000 7f6a1a000640 1 -- 172.21.15.152:0/3294536343 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1710789107 0 0) 0x7f6a1c15fd20 con 0x7f6a1c15c560 2024-04-20T11:34:42.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:42.304+0000 7f6a1b400640 1 --2- 172.21.15.152:0/3294536343 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6a1c0d0640 0x7f6a1c0ada90 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f6a14009d90 tx=0x7f6a1402be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5093275e17afd46b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:42.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:42.304+0000 7f6a1a000640 1 -- 172.21.15.152:0/3294536343 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a14039070 con 0x7f6a1c0d0640 2024-04-20T11:34:42.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:42.304+0000 7f6a1a000640 1 -- 172.21.15.152:0/3294536343 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a1402cbc0 con 0x7f6a1c0d0640 2024-04-20T11:34:42.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:42.304+0000 7f6a1a000640 1 -- 172.21.15.152:0/3294536343 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a14034440 con 0x7f6a1c0d0640 2024-04-20T11:34:42.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:42.305+0000 7f6a21e00640 1 -- 172.21.15.152:0/3294536343 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6a1c0d0640 msgr2=0x7f6a1c0ada90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:42.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:42.305+0000 7f6a21e00640 1 --2- 172.21.15.152:0/3294536343 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6a1c0d0640 0x7f6a1c0ada90 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f6a14009d90 tx=0x7f6a1402be10 comp rx=0 tx=0).stop 2024-04-20T11:34:42.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:42.305+0000 7f6a21e00640 1 -- 172.21.15.152:0/3294536343 shutdown_connections 2024-04-20T11:34:42.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:42.305+0000 7f6a21e00640 1 --2- 172.21.15.152:0/3294536343 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6a1c0b4500 0x7f6a1c0d0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:42.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:42.305+0000 7f6a21e00640 1 --2- 172.21.15.152:0/3294536343 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6a1c0d0640 0x7f6a1c0ada90 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:42.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:42.305+0000 7f6a21e00640 1 -- 172.21.15.152:0/3294536343 >> 172.21.15.152:0/3294536343 conn(0x7f6a1c067cf0 msgr2=0x7f6a1c06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:42.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:42.305+0000 7f6a21e00640 1 -- 172.21.15.152:0/3294536343 shutdown_connections 2024-04-20T11:34:42.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:42.305+0000 7f6a21e00640 1 -- 172.21.15.152:0/3294536343 wait complete. 2024-04-20T11:34:42.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:42.307+0000 7f6a21e00640 1 Processor -- start 2024-04-20T11:34:42.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:42.307+0000 7f6a21e00640 1 -- start start 2024-04-20T11:34:42.308 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:42.308+0000 7f6a21e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6a1c0b4500 0x7f6a1c145d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:42.308 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:42.308+0000 7f6a21e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6a1c0d0640 0x7f6a1c150770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:42.308 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:42.308+0000 7f6a21e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a1c15c560 0x7f6a1c150cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:42.308 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:42.308+0000 7f6a21e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f6a1c0d0aa0 con 0x7f6a1c0d0640 2024-04-20T11:34:42.308 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:42.308+0000 7f6a21e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6a1c0d0dc0 con 0x7f6a1c15c560 2024-04-20T11:34:42.308 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:42.308+0000 7f6a21e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6a1c0adfd0 con 0x7f6a1c0b4500 2024-04-20T11:34:42.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:42.308+0000 7f6a13e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6a1c0d0640 0x7f6a1c150770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:42.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:42.308+0000 7f6a13e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6a1c0d0640 0x7f6a1c150770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:50892/0 (socket says 172.21.15.152:50892) 2024-04-20T11:34:42.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:42.308+0000 7f6a13e00640 1 -- 172.21.15.152:0/1436549603 learned_addr learned my addr 172.21.15.152:0/1436549603 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:42.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:42.309+0000 7f6a13e00640 1 -- 172.21.15.152:0/1436549603 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6a1c0b4500 msgr2=0x7f6a1c145d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:42.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:42.309+0000 7f6a13e00640 1 --2- 172.21.15.152:0/1436549603 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6a1c0b4500 0x7f6a1c145d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:42.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:42.309+0000 7f6a13e00640 1 -- 172.21.15.152:0/1436549603 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a1c15c560 msgr2=0x7f6a1c150cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:42.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:42.309+0000 7f6a13e00640 1 --2- 172.21.15.152:0/1436549603 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a1c15c560 0x7f6a1c150cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:42.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:42.309+0000 7f6a13e00640 1 -- 172.21.15.152:0/1436549603 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a1c1531c0 con 0x7f6a1c0d0640 2024-04-20T11:34:42.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:42.309+0000 7f6a13e00640 1 --2- 172.21.15.152:0/1436549603 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6a1c0d0640 0x7f6a1c150770 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f6a0400d800 tx=0x7f6a0400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:42.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:42.309+0000 7f6a13400640 1 -- 172.21.15.152:0/1436549603 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a0400b6c0 con 0x7f6a1c0d0640 2024-04-20T11:34:42.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:42.309+0000 7f6a13400640 1 -- 172.21.15.152:0/1436549603 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a04019070 con 0x7f6a1c0d0640 2024-04-20T11:34:42.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:42.309+0000 7f6a13400640 1 -- 172.21.15.152:0/1436549603 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a0400ba80 con 0x7f6a1c0d0640 2024-04-20T11:34:42.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:42.309+0000 7f6a21e00640 1 -- 172.21.15.152:0/1436549603 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a1c152ee0 con 0x7f6a1c0d0640 2024-04-20T11:34:42.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:42.309+0000 7f6a21e00640 1 -- 172.21.15.152:0/1436549603 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a1c15d760 con 0x7f6a1c0d0640 2024-04-20T11:34:42.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:42.310+0000 7f6a21e00640 1 -- 172.21.15.152:0/1436549603 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a1c0689f0 con 0x7f6a1c0d0640 2024-04-20T11:34:42.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:42.311+0000 7f6a13400640 1 -- 172.21.15.152:0/1436549603 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 19) ==== 91397+0+0 (secure 0 0 0) 0x7f6a0400bc20 con 0x7f6a1c0d0640 2024-04-20T11:34:42.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:42.312+0000 7f6a13400640 1 --2- 172.21.15.152:0/1436549603 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f69f806cce0 0x7f69f806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:42.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:42.312+0000 7f6a13400640 1 -- 172.21.15.152:0/1436549603 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5709+0+0 (secure 0 0 0) 0x7f6a0408c6f0 con 0x7f6a1c0d0640 2024-04-20T11:34:42.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:42.313+0000 7f6a1aa00640 1 --2- 172.21.15.152:0/1436549603 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f69f806cce0 0x7f69f806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:42.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:42.314+0000 7f6a1aa00640 1 --2- 172.21.15.152:0/1436549603 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f69f806cce0 0x7f69f806f1a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c007fd0 tx=0x7f6a0c007960 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:42.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:42.317+0000 7f6a13400640 1 -- 172.21.15.152:0/1436549603 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f6a1c1531c0 con 0x7f6a1c0d0640 2024-04-20T11:34:42.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:42.550+0000 7f6a21e00640 1 -- 172.21.15.152:0/1436549603 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6a1c0b1ee0 con 0x7f6a1c0d0640 2024-04-20T11:34:42.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:42.551+0000 7f6a13400640 1 -- 172.21.15.152:0/1436549603 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) ==== 94+0+5207 (secure 0 0 0) 0x7f6a0405a5f0 con 0x7f6a1c0d0640 2024-04-20T11:34:42.551 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:34:42.551 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":76,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5891,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/508638056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6833","nonce":508638056},{"type":"v1","addr":"172.21.15.175:6835","nonce":508638056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":6448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6834/3447077466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3447077466},{"type":"v1","addr":"172.21.15.175:6834","nonce":3447077466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":8409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6834/751442105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":751442105},{"type":"v1","addr":"172.21.15.152:6834","nonce":751442105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T11:34:36.050950+0000","modified":"2024-04-20T11:34:39.117003+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8412},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8412":{"gid":8412,"name":"c","rank":0,"incarnation":72,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6835/2477488828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6833","nonce":2477488828},{"type":"v1","addr":"172.21.15.152:6835","nonce":2477488828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8412,"qdb_cluster":[8412]},"id":9}]} 2024-04-20T11:34:42.551 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 76 2024-04-20T11:34:42.551 INFO:tasks.ceph.mds.c.smithi152.stderr:2024-04-20T11:34:42.551+0000 7fb6e7000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-20T11:34:42.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:42.553+0000 7f6a21e00640 1 -- 172.21.15.152:0/1436549603 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f69f806cce0 msgr2=0x7f69f806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:42.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:42.553+0000 7f6a21e00640 1 --2- 172.21.15.152:0/1436549603 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f69f806cce0 0x7f69f806f1a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c007fd0 tx=0x7f6a0c007960 comp rx=0 tx=0).stop 2024-04-20T11:34:42.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:42.553+0000 7f6a21e00640 1 -- 172.21.15.152:0/1436549603 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6a1c0d0640 msgr2=0x7f6a1c150770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:42.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:42.553+0000 7f6a21e00640 1 --2- 172.21.15.152:0/1436549603 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6a1c0d0640 0x7f6a1c150770 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f6a0400d800 tx=0x7f6a0400dcc0 comp rx=0 tx=0).stop 2024-04-20T11:34:42.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:42.554+0000 7f6a21e00640 1 -- 172.21.15.152:0/1436549603 shutdown_connections 2024-04-20T11:34:42.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:42.554+0000 7f6a21e00640 1 --2- 172.21.15.152:0/1436549603 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f69f806cce0 0x7f69f806f1a0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:42.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:42.554+0000 7f6a21e00640 1 --2- 172.21.15.152:0/1436549603 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a1c15c560 0x7f6a1c150cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:42.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:42.554+0000 7f6a21e00640 1 --2- 172.21.15.152:0/1436549603 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6a1c0d0640 0x7f6a1c150770 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:42.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:42.554+0000 7f6a21e00640 1 --2- 172.21.15.152:0/1436549603 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6a1c0b4500 0x7f6a1c145d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:42.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:42.554+0000 7f6a21e00640 1 -- 172.21.15.152:0/1436549603 >> 172.21.15.152:0/1436549603 conn(0x7f6a1c067cf0 msgr2=0x7f6a1c0ae3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:42.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:42.554+0000 7f6a21e00640 1 -- 172.21.15.152:0/1436549603 shutdown_connections 2024-04-20T11:34:42.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:42.554+0000 7f6a21e00640 1 -- 172.21.15.152:0/1436549603 wait complete. 2024-04-20T11:34:42.973 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 76, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-20T11:34:36.050950+0000', 'modified': '2024-04-20T11:34:39.117003+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'snap_rstat': True, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8412}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8412': {'gid': 8412, 'name': 'c', 'rank': 0, 'incarnation': 72, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.152:6835/2477488828', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6833', 'nonce': 2477488828}, {'type': 'v1', 'addr': '172.21.15.152:6835', 'nonce': 2477488828}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [19], 'metadata_pool': 18, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8412, 'qdb_cluster': [8412]} 2024-04-20T11:34:42.974 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-20T11:34:42.974 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-20T11:34:43.182 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.182+0000 7f7eec000640 1 Processor -- start 2024-04-20T11:34:43.182 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.182+0000 7f7eec000640 1 -- start start 2024-04-20T11:34:43.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.184+0000 7f7eec000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7ee415ff30 0x7f7ee409aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:43.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.184+0000 7f7eec000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f7ee409af80 0x7f7ee40d06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:43.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.184+0000 7f7eec000640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ee40d1010 con 0x7f7ee40d0be0 2024-04-20T11:34:43.184 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.184+0000 7f7eec000640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f7ee409b3e0 con 0x7f7ee409af80 2024-04-20T11:34:43.184 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.184+0000 7f7eec000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7ee40cd970 con 0x7f7ee415ff30 2024-04-20T11:34:43.184 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.184+0000 7f7ee8c00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f7ee409af80 0x7f7ee40d06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:43.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.184+0000 7f7ee8c00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f7ee409af80 0x7f7ee40d06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:50906/0 (socket says 172.21.15.152:50906) 2024-04-20T11:34:43.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.184+0000 7f7ee8c00640 1 -- 172.21.15.152:0/1597615330 learned_addr learned my addr 172.21.15.152:0/1597615330 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:43.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.185+0000 7f7ee9600640 1 --2- 172.21.15.152:0/1597615330 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7ee415ff30 0x7f7ee409aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:43.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.185+0000 7f7ee8c00640 1 -- 172.21.15.152:0/1597615330 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7ee415ff30 msgr2=0x7f7ee409aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:43.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.185+0000 7f7ee8c00640 1 --2- 172.21.15.152:0/1597615330 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7ee415ff30 0x7f7ee409aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:43.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.185+0000 7f7ee8c00640 1 -- 172.21.15.152:0/1597615330 >> v1:172.21.15.175:6789/0 conn(0x7f7ee40d0be0 legacy=0x7f7ee4070010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:43.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.185+0000 7f7ee8c00640 1 -- 172.21.15.152:0/1597615330 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ee40cee10 con 0x7f7ee409af80 2024-04-20T11:34:43.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.185+0000 7f7ee9600640 1 --2- 172.21.15.152:0/1597615330 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7ee415ff30 0x7f7ee409aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:34:43.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.186+0000 7f7ee8c00640 1 --2- 172.21.15.152:0/1597615330 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7ee409af80 0x7f7ee40d06a0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f7ee40cdaf0 tx=0x7f7ed002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d677dc52546753d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:43.187 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.186+0000 7f7ee3e00640 1 -- 172.21.15.152:0/1597615330 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed00317c0 con 0x7f7ee409af80 2024-04-20T11:34:43.187 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.186+0000 7f7ee3e00640 1 -- 172.21.15.152:0/1597615330 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ed0037070 con 0x7f7ee409af80 2024-04-20T11:34:43.187 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.186+0000 7f7ee3e00640 1 -- 172.21.15.152:0/1597615330 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed0031b80 con 0x7f7ee409af80 2024-04-20T11:34:43.187 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.187+0000 7f7eec000640 1 -- 172.21.15.152:0/1597615330 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7ee409af80 msgr2=0x7f7ee40d06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:43.187 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.187+0000 7f7eec000640 1 --2- 172.21.15.152:0/1597615330 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7ee409af80 0x7f7ee40d06a0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f7ee40cdaf0 tx=0x7f7ed002b860 comp rx=0 tx=0).stop 2024-04-20T11:34:43.187 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.187+0000 7f7eec000640 1 -- 172.21.15.152:0/1597615330 shutdown_connections 2024-04-20T11:34:43.187 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.187+0000 7f7eec000640 1 --2- 172.21.15.152:0/1597615330 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7ee409af80 0x7f7ee40d06a0 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:43.187 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.187+0000 7f7eec000640 1 --2- 172.21.15.152:0/1597615330 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7ee415ff30 0x7f7ee409aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:43.187 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.187+0000 7f7eec000640 1 -- 172.21.15.152:0/1597615330 >> 172.21.15.152:0/1597615330 conn(0x7f7ee40b4090 msgr2=0x7f7ee40b44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:43.187 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.187+0000 7f7eec000640 1 -- 172.21.15.152:0/1597615330 shutdown_connections 2024-04-20T11:34:43.187 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.187+0000 7f7eec000640 1 -- 172.21.15.152:0/1597615330 wait complete. 2024-04-20T11:34:43.188 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.189+0000 7f7eec000640 1 Processor -- start 2024-04-20T11:34:43.188 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.189+0000 7f7eec000640 1 -- start start 2024-04-20T11:34:43.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.190+0000 7f7eec000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ee409af80 0x7f7ee40ca9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:43.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.190+0000 7f7eec000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7ee40d0be0 0x7f7ee415c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:43.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.190+0000 7f7eec000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7ee415ff30 0x7f7ee4159bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:43.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.190+0000 7f7eec000640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee40cd320 con 0x7f7ee415ff30 2024-04-20T11:34:43.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.190+0000 7f7eec000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee40b4ed0 con 0x7f7ee409af80 2024-04-20T11:34:43.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.190+0000 7f7eec000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7ee40cd600 con 0x7f7ee40d0be0 2024-04-20T11:34:43.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.190+0000 7f7eea000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7ee415ff30 0x7f7ee4159bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:43.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.190+0000 7f7eea000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7ee415ff30 0x7f7ee4159bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:50916/0 (socket says 172.21.15.152:50916) 2024-04-20T11:34:43.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.190+0000 7f7eea000640 1 -- 172.21.15.152:0/2937599651 learned_addr learned my addr 172.21.15.152:0/2937599651 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:43.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.190+0000 7f7ee8c00640 1 --2- 172.21.15.152:0/2937599651 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7ee40d0be0 0x7f7ee415c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:43.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.190+0000 7f7ee9600640 1 --2- 172.21.15.152:0/2937599651 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ee409af80 0x7f7ee40ca9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:43.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.190+0000 7f7eea000640 1 -- 172.21.15.152:0/2937599651 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7ee40d0be0 msgr2=0x7f7ee415c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:43.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.191+0000 7f7eea000640 1 --2- 172.21.15.152:0/2937599651 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7ee40d0be0 0x7f7ee415c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:43.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.191+0000 7f7eea000640 1 -- 172.21.15.152:0/2937599651 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ee409af80 msgr2=0x7f7ee40ca9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:43.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.191+0000 7f7eea000640 1 --2- 172.21.15.152:0/2937599651 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ee409af80 0x7f7ee40ca9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:43.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.191+0000 7f7eea000640 1 -- 172.21.15.152:0/2937599651 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ee40caef0 con 0x7f7ee415ff30 2024-04-20T11:34:43.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.191+0000 7f7eea000640 1 --2- 172.21.15.152:0/2937599651 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7ee415ff30 0x7f7ee4159bd0 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f7edc00c780 tx=0x7f7edc00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:43.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.191+0000 7f7ee1600640 1 -- 172.21.15.152:0/2937599651 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7edc004b60 con 0x7f7ee415ff30 2024-04-20T11:34:43.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.191+0000 7f7ee1600640 1 -- 172.21.15.152:0/2937599651 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7edc004d00 con 0x7f7ee415ff30 2024-04-20T11:34:43.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.191+0000 7f7ee1600640 1 -- 172.21.15.152:0/2937599651 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7edc00aa90 con 0x7f7ee415ff30 2024-04-20T11:34:43.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.191+0000 7f7eec000640 1 -- 172.21.15.152:0/2937599651 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ee415b0c0 con 0x7f7ee415ff30 2024-04-20T11:34:43.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.191+0000 7f7eec000640 1 -- 172.21.15.152:0/2937599651 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ee415b680 con 0x7f7ee415ff30 2024-04-20T11:34:43.193 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.193+0000 7f7ee1600640 1 -- 172.21.15.152:0/2937599651 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 19) ==== 91397+0+0 (secure 0 0 0) 0x7f7edc00ac30 con 0x7f7ee415ff30 2024-04-20T11:34:43.194 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.194+0000 7f7ee1600640 1 --2- 172.21.15.152:0/2937599651 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f7ec406cce0 0x7f7ec406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:43.194 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.194+0000 7f7ee1600640 1 -- 172.21.15.152:0/2937599651 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5709+0+0 (secure 0 0 0) 0x7f7edc08c530 con 0x7f7ee415ff30 2024-04-20T11:34:43.194 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.195+0000 7f7ee9600640 1 --2- 172.21.15.152:0/2937599651 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f7ec406cce0 0x7f7ec406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:43.194 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.195+0000 7f7eeed82740 1 Processor -- start 2024-04-20T11:34:43.195 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.195+0000 7f7eeed82740 1 -- start start 2024-04-20T11:34:43.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.196+0000 7f7eeed82740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa21e09460 0x55aa21dc0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:43.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.196+0000 7f7eeed82740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aa21dbf4e0 0x55aa21dc0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:43.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.196+0000 7f7eeed82740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55aa21dbb930 0x55aa21deb770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:43.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.196+0000 7f7eeed82740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55aa21d279d0 con 0x55aa21e09460 2024-04-20T11:34:43.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.196+0000 7f7eeed82740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55aa21cce270 con 0x55aa21dbf4e0 2024-04-20T11:34:43.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.196+0000 7f7eeed82740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x55aa21d23990 con 0x55aa21dbb930 2024-04-20T11:34:43.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.196+0000 7f7ee9600640 1 --2- 172.21.15.152:0/2937599651 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f7ec406cce0 0x7f7ec406f1a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7ed400a950 tx=0x7f7ed4008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:43.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.196+0000 7f7ee9600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa21e09460 0x55aa21dc0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:43.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.196+0000 7f7ee9600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa21e09460 0x55aa21dc0a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:50920/0 (socket says 172.21.15.152:50920) 2024-04-20T11:34:43.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.196+0000 7f7ee9600640 1 -- 172.21.15.152:0/978026503 learned_addr learned my addr 172.21.15.152:0/978026503 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:43.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.197+0000 7f7eea000640 1 --2- 172.21.15.152:0/978026503 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55aa21dbb930 0x55aa21deb770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:43.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.197+0000 7f7ee8c00640 1 --2- 172.21.15.152:0/978026503 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aa21dbf4e0 0x55aa21dc0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:43.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.197+0000 7f7ee9600640 1 -- 172.21.15.152:0/978026503 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55aa21dbb930 msgr2=0x55aa21deb770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:43.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.197+0000 7f7ee9600640 1 --2- 172.21.15.152:0/978026503 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55aa21dbb930 0x55aa21deb770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:43.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.197+0000 7f7ee9600640 1 -- 172.21.15.152:0/978026503 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aa21dbf4e0 msgr2=0x55aa21dc0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:43.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.197+0000 7f7ee9600640 1 --2- 172.21.15.152:0/978026503 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aa21dbf4e0 0x55aa21dc0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:43.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.197+0000 7f7ee9600640 1 -- 172.21.15.152:0/978026503 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa21a3a2d0 con 0x55aa21e09460 2024-04-20T11:34:43.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.197+0000 7f7ee8c00640 1 --2- 172.21.15.152:0/978026503 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aa21dbf4e0 0x55aa21dc0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:34:43.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.197+0000 7f7ee9600640 1 --2- 172.21.15.152:0/978026503 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa21e09460 0x55aa21dc0a90 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f7ed400ebd0 tx=0x7f7ed4011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fc9af8b45237fdc server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:43.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.197+0000 7f7ed9c00640 1 -- 172.21.15.152:0/978026503 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed401a050 con 0x55aa21e09460 2024-04-20T11:34:43.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.197+0000 7f7ed9c00640 1 -- 172.21.15.152:0/978026503 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ed4024070 con 0x55aa21e09460 2024-04-20T11:34:43.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.197+0000 7f7ed9c00640 1 -- 172.21.15.152:0/978026503 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed401f440 con 0x55aa21e09460 2024-04-20T11:34:43.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.198+0000 7f7eeed82740 1 -- 172.21.15.152:0/978026503 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa21e09460 msgr2=0x55aa21dc0a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:43.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.198+0000 7f7eeed82740 1 --2- 172.21.15.152:0/978026503 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa21e09460 0x55aa21dc0a90 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f7ed400ebd0 tx=0x7f7ed4011000 comp rx=0 tx=0).stop 2024-04-20T11:34:43.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.198+0000 7f7eeed82740 1 -- 172.21.15.152:0/978026503 shutdown_connections 2024-04-20T11:34:43.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.198+0000 7f7eeed82740 1 --2- 172.21.15.152:0/978026503 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55aa21dbb930 0x55aa21deb770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:43.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.198+0000 7f7eeed82740 1 --2- 172.21.15.152:0/978026503 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aa21dbf4e0 0x55aa21dc0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:43.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.198+0000 7f7eeed82740 1 --2- 172.21.15.152:0/978026503 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa21e09460 0x55aa21dc0a90 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:43.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.198+0000 7f7eeed82740 1 -- 172.21.15.152:0/978026503 >> 172.21.15.152:0/978026503 conn(0x55aa21dbdbf0 msgr2=0x55aa21e06770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:43.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.198+0000 7f7eeed82740 1 -- 172.21.15.152:0/978026503 shutdown_connections 2024-04-20T11:34:43.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.198+0000 7f7eeed82740 1 -- 172.21.15.152:0/978026503 wait complete. 2024-04-20T11:34:43.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.199+0000 7f7eeed82740 1 -- start start 2024-04-20T11:34:43.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.199+0000 7f7eeed82740 1 Processor -- start 2024-04-20T11:34:43.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.200+0000 7f7eeed82740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa21dbf4e0 0x55aa21de2d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:43.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.200+0000 7f7eeed82740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aa21e09460 0x55aa21e07e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:43.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.200+0000 7f7eeed82740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55aa21de3ed0 0x55aa21e08360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:43.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.200+0000 7f7eeed82740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55aa21dbf9f0 con 0x55aa21dbf4e0 2024-04-20T11:34:43.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.200+0000 7f7eeed82740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55aa21ce15e0 con 0x55aa21e09460 2024-04-20T11:34:43.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.200+0000 7f7eeed82740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x55aa21dbfcd0 con 0x55aa21de3ed0 2024-04-20T11:34:43.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.200+0000 7f7eeed82740 10 client.0 _ll_register_callbacks cb 0x55aa21a39ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:34:43.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.200+0000 7f7ee9600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa21dbf4e0 0x55aa21de2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:43.204 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.200+0000 7f7ee9600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa21dbf4e0 0x55aa21de2d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:50932/0 (socket says 172.21.15.152:50932) 2024-04-20T11:34:43.204 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.200+0000 7f7ee9600640 1 -- 172.21.15.152:0/2917449759 learned_addr learned my addr 172.21.15.152:0/2917449759 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:43.204 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.201+0000 7f7ee8c00640 1 --2- 172.21.15.152:0/2917449759 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aa21e09460 0x55aa21e07e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:43.204 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.201+0000 7f7eea000640 1 --2- 172.21.15.152:0/2917449759 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55aa21de3ed0 0x55aa21e08360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:43.204 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.201+0000 7f7ee9600640 1 -- 172.21.15.152:0/2917449759 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55aa21de3ed0 msgr2=0x55aa21e08360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:43.204 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.201+0000 7f7ee9600640 1 --2- 172.21.15.152:0/2917449759 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55aa21de3ed0 0x55aa21e08360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:43.204 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.201+0000 7f7ee9600640 1 -- 172.21.15.152:0/2917449759 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aa21e09460 msgr2=0x55aa21e07e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:43.204 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.201+0000 7f7ee9600640 1 --2- 172.21.15.152:0/2917449759 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aa21e09460 0x55aa21e07e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:43.204 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.201+0000 7f7ee9600640 1 -- 172.21.15.152:0/2917449759 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa21b046b0 con 0x55aa21dbf4e0 2024-04-20T11:34:43.204 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.201+0000 7f7eea000640 1 --2- 172.21.15.152:0/2917449759 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55aa21de3ed0 0x55aa21e08360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:34:43.204 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.201+0000 7f7ee8c00640 1 --2- 172.21.15.152:0/2917449759 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aa21e09460 0x55aa21e07e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:34:43.204 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.201+0000 7f7ee9600640 1 --2- 172.21.15.152:0/2917449759 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa21dbf4e0 0x55aa21de2d00 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f7ed400efd0 tx=0x7f7ed4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75e6bf54289d4d1c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:43.204 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.201+0000 7f7ece000640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-04-20T11:34:43.204 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.201+0000 7f7ece000640 1 -- 172.21.15.152:0/2917449759 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed4005990 con 0x55aa21dbf4e0 2024-04-20T11:34:43.205 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.202+0000 7f7ece000640 1 -- 172.21.15.152:0/2917449759 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 5709+0+0 (secure 0 0 0) 0x7f7ed4005b30 con 0x55aa21dbf4e0 2024-04-20T11:34:43.205 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.202+0000 7f7eeed82740 1 -- 172.21.15.152:0/2917449759 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ed4024070 con 0x55aa21dbf4e0 2024-04-20T11:34:43.205 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.202+0000 7f7ece000640 1 -- 172.21.15.152:0/2917449759 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f7ed401a020 con 0x55aa21dbf4e0 2024-04-20T11:34:43.205 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.202+0000 7f7eeed82740 10 client.8556 fetch_fsmap learned FSMap version 76 2024-04-20T11:34:43.205 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.202+0000 7f7eeed82740 1 -- 172.21.15.152:0/2917449759 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=2+}) -- 0x55aa21de3240 con 0x55aa21dbf4e0 2024-04-20T11:34:43.205 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.202+0000 7f7ece000640 1 -- 172.21.15.152:0/2917449759 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ed4024070 con 0x55aa21dbf4e0 2024-04-20T11:34:43.205 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.202+0000 7f7ece000640 1 -- 172.21.15.152:0/2917449759 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 76) ==== 3495+0+0 (secure 0 0 0) 0x7f7ed4006030 con 0x55aa21dbf4e0 2024-04-20T11:34:43.205 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.202+0000 7f7eeed82740 10 client.8556 fetch_fsmap finished waiting for FSMap version 76 2024-04-20T11:34:43.205 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.202+0000 7f7eeed82740 10 client.8556 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-20T11:34:43.205 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.202+0000 7f7eeed82740 20 client.8556 populate_metadata read hostname 'smithi152' 2024-04-20T11:34:43.205 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.202+0000 7f7eeed82740 1 --2- 172.21.15.152:0/2917449759 >> [v2:172.21.15.152:6833/2477488828,v1:172.21.15.152:6835/2477488828] conn(0x55aa21e0d270 0x55aa21e0d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:43.205 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.202+0000 7f7eeed82740 4 client.8556 mds_command: new command op to 8412 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:34:43.205 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.202+0000 7f7eeed82740 1 -- 172.21.15.152:0/2917449759 --> [v2:172.21.15.152:6833/2477488828,v1:172.21.15.152:6835/2477488828] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aa21d353f0 con 0x55aa21e0d270 2024-04-20T11:34:43.205 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.203+0000 7f7ee8c00640 1 --2- 172.21.15.152:0/2917449759 >> [v2:172.21.15.152:6833/2477488828,v1:172.21.15.152:6835/2477488828] conn(0x55aa21e0d270 0x55aa21e0d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:43.207 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.203+0000 7f7ee8c00640 1 --2- 172.21.15.152:0/2917449759 >> [v2:172.21.15.152:6833/2477488828,v1:172.21.15.152:6835/2477488828] conn(0x55aa21e0d270 0x55aa21e0d650 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=483eb6657977fa8b server_cookie=8c81e2fd94b5b172 in_seq=0 out_seq=0 2024-04-20T11:34:43.207 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.203+0000 7f7ece000640 10 client.8556 ms_handle_connect on v2:172.21.15.152:6833/2477488828 2024-04-20T11:34:43.207 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.204+0000 7f7ece000640 1 -- 172.21.15.152:0/2917449759 <== mds.0 v2:172.21.15.152:6833/2477488828 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55aa21d353f0 con 0x55aa21e0d270 2024-04-20T11:34:43.207 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.204+0000 7f7ece000640 10 client.8556 handle_command_reply: tid=0 2024-04-20T11:34:43.207 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.204+0000 7f7eeed82740 1 client.8556 shutdown 2024-04-20T11:34:43.207 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.204+0000 7f7eeed82740 1 -- 172.21.15.152:0/2917449759 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa21dbf4e0 msgr2=0x55aa21de2d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:43.207 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.204+0000 7f7eeed82740 1 --2- 172.21.15.152:0/2917449759 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa21dbf4e0 0x55aa21de2d00 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f7ed400efd0 tx=0x7f7ed4004730 comp rx=0 tx=0).stop 2024-04-20T11:34:43.207 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.204+0000 7f7eeed82740 1 -- 172.21.15.152:0/2917449759 shutdown_connections 2024-04-20T11:34:43.207 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.204+0000 7f7eeed82740 1 --2- 172.21.15.152:0/2917449759 >> [v2:172.21.15.152:6833/2477488828,v1:172.21.15.152:6835/2477488828] conn(0x55aa21e0d270 0x55aa21e0d650 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:43.207 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.204+0000 7f7eeed82740 1 --2- 172.21.15.152:0/2917449759 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55aa21de3ed0 0x55aa21e08360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:43.208 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.204+0000 7f7eeed82740 1 --2- 172.21.15.152:0/2917449759 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aa21e09460 0x55aa21e07e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:43.208 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.204+0000 7f7ece000640 0 client.8556 ms_handle_reset on v2:172.21.15.152:6833/2477488828 2024-04-20T11:34:43.208 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.204+0000 7f7eeed82740 1 --2- 172.21.15.152:0/2917449759 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa21dbf4e0 0x55aa21de2d00 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:43.208 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.204+0000 7f7eeed82740 1 -- 172.21.15.152:0/2917449759 >> 172.21.15.152:0/2917449759 conn(0x55aa21dbb930 msgr2=0x55aa21e06570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:43.208 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.205+0000 7f7eeed82740 1 -- 172.21.15.152:0/2917449759 shutdown_connections 2024-04-20T11:34:43.208 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.205+0000 7f7eeed82740 1 -- 172.21.15.152:0/2917449759 wait complete. 2024-04-20T11:34:43.208 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.205+0000 7f7eeed82740 20 client.8556 trim_cache size 0 max 16384 2024-04-20T11:34:43.224 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.224+0000 7f7eeed82740 1 Processor -- start 2024-04-20T11:34:43.224 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.224+0000 7f7eeed82740 1 -- start start 2024-04-20T11:34:43.224 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.225+0000 7f7eeed82740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aa21dbf4e0 0x55aa21de9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:43.224 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.225+0000 7f7eeed82740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55aa21de3ed0 0x55aa21e05370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:43.225 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.225+0000 7f7eeed82740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa21e09460 0x55aa21e058b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:43.225 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.225+0000 7f7eeed82740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee41fff40 con 0x55aa21e09460 2024-04-20T11:34:43.225 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.225+0000 7f7eeed82740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee406e8a0 con 0x55aa21dbf4e0 2024-04-20T11:34:43.225 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.225+0000 7f7eeed82740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7ee40c6860 con 0x55aa21de3ed0 2024-04-20T11:34:43.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.226+0000 7f7eea000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa21e09460 0x55aa21e058b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:43.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.226+0000 7f7eea000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa21e09460 0x55aa21e058b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:50946/0 (socket says 172.21.15.152:50946) 2024-04-20T11:34:43.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.226+0000 7f7eea000640 1 -- 172.21.15.152:0/246092506 learned_addr learned my addr 172.21.15.152:0/246092506 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:43.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.226+0000 7f7ee8c00640 1 --2- 172.21.15.152:0/246092506 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55aa21de3ed0 0x55aa21e05370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:43.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.226+0000 7f7ee9600640 1 --2- 172.21.15.152:0/246092506 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aa21dbf4e0 0x55aa21de9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:43.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.226+0000 7f7eea000640 1 -- 172.21.15.152:0/246092506 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55aa21de3ed0 msgr2=0x55aa21e05370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:43.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.226+0000 7f7eea000640 1 --2- 172.21.15.152:0/246092506 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55aa21de3ed0 0x55aa21e05370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:43.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.226+0000 7f7eea000640 1 -- 172.21.15.152:0/246092506 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aa21dbf4e0 msgr2=0x55aa21de9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:43.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.226+0000 7f7eea000640 1 --2- 172.21.15.152:0/246092506 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aa21dbf4e0 0x55aa21de9d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:43.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.226+0000 7f7eea000640 1 -- 172.21.15.152:0/246092506 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa21dbd300 con 0x55aa21e09460 2024-04-20T11:34:43.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.226+0000 7f7ee9600640 1 --2- 172.21.15.152:0/246092506 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aa21dbf4e0 0x55aa21de9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:34:43.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.226+0000 7f7eea000640 1 --2- 172.21.15.152:0/246092506 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa21e09460 0x55aa21e058b0 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f7edc059c60 tx=0x7f7edc056020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab5863e442ad48da server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:43.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.226+0000 7f7ed9200640 1 -- 172.21.15.152:0/246092506 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7edc05e080 con 0x55aa21e09460 2024-04-20T11:34:43.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.227+0000 7f7ed9200640 1 -- 172.21.15.152:0/246092506 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7edc0628b0 con 0x55aa21e09460 2024-04-20T11:34:43.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.227+0000 7f7ed9200640 1 -- 172.21.15.152:0/246092506 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7edc057fd0 con 0x55aa21e09460 2024-04-20T11:34:43.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.227+0000 7f7eeed82740 1 -- 172.21.15.152:0/246092506 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa21e09460 msgr2=0x55aa21e058b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:43.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.227+0000 7f7eeed82740 1 --2- 172.21.15.152:0/246092506 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa21e09460 0x55aa21e058b0 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f7edc059c60 tx=0x7f7edc056020 comp rx=0 tx=0).stop 2024-04-20T11:34:43.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.227+0000 7f7eeed82740 1 -- 172.21.15.152:0/246092506 shutdown_connections 2024-04-20T11:34:43.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.227+0000 7f7eeed82740 1 --2- 172.21.15.152:0/246092506 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa21e09460 0x55aa21e058b0 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:43.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.227+0000 7f7eeed82740 1 --2- 172.21.15.152:0/246092506 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55aa21de3ed0 0x55aa21e05370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:43.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.227+0000 7f7eeed82740 1 --2- 172.21.15.152:0/246092506 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aa21dbf4e0 0x55aa21de9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:43.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.227+0000 7f7eeed82740 1 -- 172.21.15.152:0/246092506 >> 172.21.15.152:0/246092506 conn(0x55aa21dbb930 msgr2=0x55aa21db6ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:43.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.227+0000 7f7eeed82740 1 -- 172.21.15.152:0/246092506 shutdown_connections 2024-04-20T11:34:43.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.227+0000 7f7eeed82740 1 -- 172.21.15.152:0/246092506 wait complete. 2024-04-20T11:34:43.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.228+0000 7f7eeed82740 1 -- start start 2024-04-20T11:34:43.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.228+0000 7f7eeed82740 1 Processor -- start 2024-04-20T11:34:43.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.229+0000 7f7eeed82740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55aa21dbf4e0 0x55aa21e079e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:43.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.229+0000 7f7eeed82740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa21de3ed0 0x55aa21e669d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:43.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.229+0000 7f7eeed82740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aa21e09460 0x55aa21dc1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:43.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.229+0000 7f7eeed82740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55aa21e0de40 con 0x55aa21de3ed0 2024-04-20T11:34:43.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.229+0000 7f7eeed82740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55aa21defb00 con 0x55aa21e09460 2024-04-20T11:34:43.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.229+0000 7f7eeed82740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x55aa21e0b630 con 0x55aa21dbf4e0 2024-04-20T11:34:43.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.229+0000 7f7eeed82740 10 client.0 _ll_register_callbacks cb 0x7f7ed400ead0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:34:43.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.229+0000 7f7ee8c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa21de3ed0 0x55aa21e669d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:43.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.229+0000 7f7ee8c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa21de3ed0 0x55aa21e669d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:50956/0 (socket says 172.21.15.152:50956) 2024-04-20T11:34:43.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.229+0000 7f7ee8c00640 1 -- 172.21.15.152:0/3076577188 learned_addr learned my addr 172.21.15.152:0/3076577188 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:43.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.230+0000 7f7ee8c00640 1 -- 172.21.15.152:0/3076577188 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55aa21dbf4e0 msgr2=0x55aa21e079e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:43.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.230+0000 7f7ee8c00640 1 --2- 172.21.15.152:0/3076577188 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55aa21dbf4e0 0x55aa21e079e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:43.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.230+0000 7f7ee8c00640 1 -- 172.21.15.152:0/3076577188 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aa21e09460 msgr2=0x55aa21dc1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:43.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.230+0000 7f7eea000640 1 --2- 172.21.15.152:0/3076577188 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aa21e09460 0x55aa21dc1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:43.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.230+0000 7f7ee8c00640 1 --2- 172.21.15.152:0/3076577188 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aa21e09460 0x55aa21dc1680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:43.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.230+0000 7f7ee8c00640 1 -- 172.21.15.152:0/3076577188 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa21e08210 con 0x55aa21de3ed0 2024-04-20T11:34:43.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.230+0000 7f7ee8c00640 1 --2- 172.21.15.152:0/3076577188 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa21de3ed0 0x55aa21e669d0 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f7ed00090b0 tx=0x7f7ed0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=300b5b7c33199a6c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:43.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.230+0000 7f7ece000640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-04-20T11:34:43.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.230+0000 7f7ece000640 1 -- 172.21.15.152:0/3076577188 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed0031e00 con 0x55aa21de3ed0 2024-04-20T11:34:43.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.230+0000 7f7ece000640 1 -- 172.21.15.152:0/3076577188 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 5709+0+0 (secure 0 0 0) 0x7f7ed00346e0 con 0x55aa21de3ed0 2024-04-20T11:34:43.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.230+0000 7f7eeed82740 1 -- 172.21.15.152:0/3076577188 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7edc0628b0 con 0x55aa21de3ed0 2024-04-20T11:34:43.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.230+0000 7f7ece000640 1 -- 172.21.15.152:0/3076577188 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f7ed00348e0 con 0x55aa21de3ed0 2024-04-20T11:34:43.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.230+0000 7f7eeed82740 10 client.8562 fetch_fsmap learned FSMap version 76 2024-04-20T11:34:43.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.230+0000 7f7eeed82740 1 -- 172.21.15.152:0/3076577188 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=2+}) -- 0x55aa21e84d40 con 0x55aa21de3ed0 2024-04-20T11:34:43.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.231+0000 7f7ece000640 1 -- 172.21.15.152:0/3076577188 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7edc0628b0 con 0x55aa21de3ed0 2024-04-20T11:34:43.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.231+0000 7f7ece000640 1 -- 172.21.15.152:0/3076577188 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 76) ==== 3495+0+0 (secure 0 0 0) 0x7f7ed0045040 con 0x55aa21de3ed0 2024-04-20T11:34:43.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.231+0000 7f7eeed82740 10 client.8562 fetch_fsmap finished waiting for FSMap version 76 2024-04-20T11:34:43.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.231+0000 7f7eeed82740 10 client.8562 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-20T11:34:43.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.231+0000 7f7eeed82740 20 client.8562 populate_metadata read hostname 'smithi152' 2024-04-20T11:34:43.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.231+0000 7f7eeed82740 1 --2- 172.21.15.152:0/3076577188 >> [v2:172.21.15.152:6833/2477488828,v1:172.21.15.152:6835/2477488828] conn(0x55aa21e84820 0x55aa21de6a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:43.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.231+0000 7f7eeed82740 4 client.8562 mds_command: new command op to 8412 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-20T11:34:43.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.231+0000 7f7eeed82740 1 -- 172.21.15.152:0/3076577188 --> [v2:172.21.15.152:6833/2477488828,v1:172.21.15.152:6835/2477488828] -- command(tid 0: {"prefix": "status"}) -- 0x55aa21d2f260 con 0x55aa21e84820 2024-04-20T11:34:43.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.231+0000 7f7ee9600640 1 --2- 172.21.15.152:0/3076577188 >> [v2:172.21.15.152:6833/2477488828,v1:172.21.15.152:6835/2477488828] conn(0x55aa21e84820 0x55aa21de6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:43.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.232+0000 7f7ece000640 10 client.8562 ms_handle_connect on v2:172.21.15.152:6833/2477488828 2024-04-20T11:34:43.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.232+0000 7f7ee9600640 1 --2- 172.21.15.152:0/3076577188 >> [v2:172.21.15.152:6833/2477488828,v1:172.21.15.152:6835/2477488828] conn(0x55aa21e84820 0x55aa21de6a70 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c6bbc2d3871c796 server_cookie=6e80218682333df8 in_seq=0 out_seq=0 2024-04-20T11:34:43.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.232+0000 7f7ece000640 1 -- 172.21.15.152:0/3076577188 <== mds.0 v2:172.21.15.152:6833/2477488828 1 ==== command_reply(tid 0: 0 ) ==== 8+0+320 (crc 0 0 0) 0x55aa21d2f260 con 0x55aa21e84820 2024-04-20T11:34:43.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.232+0000 7f7ece000640 10 client.8562 handle_command_reply: tid=0 2024-04-20T11:34:43.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.232+0000 7f7eeed82740 1 client.8562 shutdown 2024-04-20T11:34:43.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.232+0000 7f7eeed82740 1 -- 172.21.15.152:0/3076577188 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa21de3ed0 msgr2=0x55aa21e669d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:43.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.232+0000 7f7eeed82740 1 --2- 172.21.15.152:0/3076577188 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa21de3ed0 0x55aa21e669d0 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f7ed00090b0 tx=0x7f7ed0004730 comp rx=0 tx=0).stop 2024-04-20T11:34:43.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.232+0000 7f7eeed82740 1 -- 172.21.15.152:0/3076577188 shutdown_connections 2024-04-20T11:34:43.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.232+0000 7f7eeed82740 1 --2- 172.21.15.152:0/3076577188 >> [v2:172.21.15.152:6833/2477488828,v1:172.21.15.152:6835/2477488828] conn(0x55aa21e84820 0x55aa21de6a70 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:43.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.232+0000 7f7eeed82740 1 --2- 172.21.15.152:0/3076577188 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aa21e09460 0x55aa21dc1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:43.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.232+0000 7f7eeed82740 1 --2- 172.21.15.152:0/3076577188 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa21de3ed0 0x55aa21e669d0 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:43.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.232+0000 7f7ece000640 0 client.8562 ms_handle_reset on v2:172.21.15.152:6833/2477488828 2024-04-20T11:34:43.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.232+0000 7f7eeed82740 1 --2- 172.21.15.152:0/3076577188 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55aa21dbf4e0 0x55aa21e079e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:43.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.232+0000 7f7eeed82740 1 -- 172.21.15.152:0/3076577188 >> 172.21.15.152:0/3076577188 conn(0x55aa21dbb930 msgr2=0x55aa21de9d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:43.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.232+0000 7f7eeed82740 1 -- 172.21.15.152:0/3076577188 shutdown_connections 2024-04-20T11:34:43.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.232+0000 7f7eeed82740 1 -- 172.21.15.152:0/3076577188 wait complete. 2024-04-20T11:34:43.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.233+0000 7f7eeed82740 20 client.8562 trim_cache size 0 max 16384 2024-04-20T11:34:43.236 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-04-20T11:34:43.236 INFO:teuthology.orchestra.run.smithi152.stdout: "cluster_fsid": "c8a72131-0d4f-42c7-892f-3f3af4d2bd3d", 2024-04-20T11:34:43.236 INFO:teuthology.orchestra.run.smithi152.stdout: "whoami": 0, 2024-04-20T11:34:43.236 INFO:teuthology.orchestra.run.smithi152.stdout: "id": 8412, 2024-04-20T11:34:43.237 INFO:teuthology.orchestra.run.smithi152.stdout: "want_state": "up:active", 2024-04-20T11:34:43.237 INFO:teuthology.orchestra.run.smithi152.stdout: "state": "up:active", 2024-04-20T11:34:43.237 INFO:teuthology.orchestra.run.smithi152.stdout: "fs_name": "cephfs", 2024-04-20T11:34:43.237 INFO:teuthology.orchestra.run.smithi152.stdout: "rank_uptime": 6.646544982, 2024-04-20T11:34:43.237 INFO:teuthology.orchestra.run.smithi152.stdout: "mdsmap_epoch": 76, 2024-04-20T11:34:43.237 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_epoch": 118, 2024-04-20T11:34:43.237 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_epoch_barrier": 118, 2024-04-20T11:34:43.237 INFO:teuthology.orchestra.run.smithi152.stdout: "uptime": 12.072702387 2024-04-20T11:34:43.237 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-04-20T11:34:43.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.233+0000 7f7eec000640 1 -- 172.21.15.152:0/2937599651 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f7ec406cce0 msgr2=0x7f7ec406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:43.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.233+0000 7f7eec000640 1 --2- 172.21.15.152:0/2937599651 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f7ec406cce0 0x7f7ec406f1a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7ed400a950 tx=0x7f7ed4008040 comp rx=0 tx=0).stop 2024-04-20T11:34:43.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.233+0000 7f7eec000640 1 -- 172.21.15.152:0/2937599651 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7ee415ff30 msgr2=0x7f7ee4159bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:43.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.233+0000 7f7eec000640 1 --2- 172.21.15.152:0/2937599651 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7ee415ff30 0x7f7ee4159bd0 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f7edc00c780 tx=0x7f7edc00cc40 comp rx=0 tx=0).stop 2024-04-20T11:34:43.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.233+0000 7f7eec000640 1 -- 172.21.15.152:0/2937599651 shutdown_connections 2024-04-20T11:34:43.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.233+0000 7f7eec000640 1 --2- 172.21.15.152:0/2937599651 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f7ec406cce0 0x7f7ec406f1a0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:43.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.233+0000 7f7eec000640 1 --2- 172.21.15.152:0/2937599651 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7ee415ff30 0x7f7ee4159bd0 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:43.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.233+0000 7f7eec000640 1 --2- 172.21.15.152:0/2937599651 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7ee40d0be0 0x7f7ee415c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:43.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.233+0000 7f7eec000640 1 --2- 172.21.15.152:0/2937599651 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ee409af80 0x7f7ee40ca9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:43.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.233+0000 7f7eec000640 1 -- 172.21.15.152:0/2937599651 >> 172.21.15.152:0/2937599651 conn(0x7f7ee40b4090 msgr2=0x7f7ee415eaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:43.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.233+0000 7f7eec000640 1 -- 172.21.15.152:0/2937599651 shutdown_connections 2024-04-20T11:34:43.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.233+0000 7f7eec000640 1 -- 172.21.15.152:0/2937599651 wait complete. 2024-04-20T11:34:43.249 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-20T11:34:43.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.430+0000 7f28c7200640 1 Processor -- start 2024-04-20T11:34:43.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.430+0000 7f28c7200640 1 -- start start 2024-04-20T11:34:43.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.431+0000 7f28c7200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f28c815d1e0 0x7f28c8160b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:43.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.431+0000 7f28c7200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f28c815d5c0 0x7f28c80cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:43.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.431+0000 7f28c7200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f28c8071cd0 0x7f28c8072160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:43.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.431+0000 7f28c7200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f28c80726a0 con 0x7f28c815d1e0 2024-04-20T11:34:43.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.431+0000 7f28c7200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f28c80ce8c0 con 0x7f28c815d5c0 2024-04-20T11:34:43.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.431+0000 7f28c7200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f28c81600b0 con 0x7f28c8071cd0 2024-04-20T11:34:43.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.431+0000 7f28c5e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f28c815d1e0 0x7f28c8160b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:43.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.432+0000 7f28c5e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f28c815d1e0 0x7f28c8160b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:50962/0 (socket says 172.21.15.152:50962) 2024-04-20T11:34:43.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.432+0000 7f28c5e00640 1 -- 172.21.15.152:0/3519949006 learned_addr learned my addr 172.21.15.152:0/3519949006 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:43.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.432+0000 7f28c5400640 1 --2- 172.21.15.152:0/3519949006 >> v2:172.21.15.175:3300/0 conn(0x7f28c815d5c0 0x7f28c80cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:43.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.432+0000 7f28c6800640 1 --2- 172.21.15.152:0/3519949006 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f28c8071cd0 0x7f28c8072160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:43.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.432+0000 7f28c5e00640 1 -- 172.21.15.152:0/3519949006 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f28c8071cd0 msgr2=0x7f28c8072160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:43.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.432+0000 7f28c5e00640 1 --2- 172.21.15.152:0/3519949006 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f28c8071cd0 0x7f28c8072160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:43.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.432+0000 7f28c5e00640 1 -- 172.21.15.152:0/3519949006 >> v2:172.21.15.175:3300/0 conn(0x7f28c815d5c0 msgr2=0x7f28c80cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:43.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.432+0000 7f28c5e00640 1 --2- 172.21.15.152:0/3519949006 >> v2:172.21.15.175:3300/0 conn(0x7f28c815d5c0 0x7f28c80cdfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:43.433 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.432+0000 7f28c5e00640 1 -- 172.21.15.152:0/3519949006 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28c8160390 con 0x7f28c815d1e0 2024-04-20T11:34:43.433 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.432+0000 7f28c5400640 1 --2- 172.21.15.152:0/3519949006 >> v2:172.21.15.175:3300/0 conn(0x7f28c815d5c0 0x7f28c80cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:34:43.433 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.432+0000 7f28c6800640 1 --2- 172.21.15.152:0/3519949006 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f28c8071cd0 0x7f28c8072160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:34:43.433 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.433+0000 7f28c5e00640 1 --2- 172.21.15.152:0/3519949006 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f28c815d1e0 0x7f28c8160b90 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f28b8009870 tx=0x7f28b802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=987cd271f3c792e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:43.433 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.434+0000 7f28c4a00640 1 -- 172.21.15.152:0/3519949006 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b8035020 con 0x7f28c815d1e0 2024-04-20T11:34:43.433 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.434+0000 7f28c4a00640 1 -- 172.21.15.152:0/3519949006 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28b8031640 con 0x7f28c815d1e0 2024-04-20T11:34:43.433 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.434+0000 7f28c4a00640 1 -- 172.21.15.152:0/3519949006 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b802d040 con 0x7f28c815d1e0 2024-04-20T11:34:43.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.434+0000 7f28c7200640 1 -- 172.21.15.152:0/3519949006 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f28c815d1e0 msgr2=0x7f28c8160b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:43.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.434+0000 7f28c7200640 1 --2- 172.21.15.152:0/3519949006 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f28c815d1e0 0x7f28c8160b90 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f28b8009870 tx=0x7f28b802b8f0 comp rx=0 tx=0).stop 2024-04-20T11:34:43.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.435+0000 7f28c7200640 1 -- 172.21.15.152:0/3519949006 shutdown_connections 2024-04-20T11:34:43.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.435+0000 7f28c7200640 1 --2- 172.21.15.152:0/3519949006 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f28c8071cd0 0x7f28c8072160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:43.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.435+0000 7f28c7200640 1 --2- 172.21.15.152:0/3519949006 >> v2:172.21.15.175:3300/0 conn(0x7f28c815d5c0 0x7f28c80cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:43.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.435+0000 7f28c7200640 1 --2- 172.21.15.152:0/3519949006 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f28c815d1e0 0x7f28c8160b90 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:43.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.435+0000 7f28c7200640 1 -- 172.21.15.152:0/3519949006 >> 172.21.15.152:0/3519949006 conn(0x7f28c8067ae0 msgr2=0x7f28c806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:43.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.435+0000 7f28c7200640 1 -- 172.21.15.152:0/3519949006 shutdown_connections 2024-04-20T11:34:43.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.435+0000 7f28c7200640 1 -- 172.21.15.152:0/3519949006 wait complete. 2024-04-20T11:34:43.436 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.436+0000 7f28c7200640 1 Processor -- start 2024-04-20T11:34:43.436 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.436+0000 7f28c7200640 1 -- start start 2024-04-20T11:34:43.437 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.437+0000 7f28c7200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f28c8071cd0 0x7f28c81f4220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:43.437 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.437+0000 7f28c7200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f28c815d1e0 0x7f28c814fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:43.437 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.437+0000 7f28c7200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28c815d5c0 0x7f28c8150410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:43.438 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.437+0000 7f28c7200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f28c80ce520 con 0x7f28c8071cd0 2024-04-20T11:34:43.438 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.437+0000 7f28c7200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f28c80ceba0 con 0x7f28c815d5c0 2024-04-20T11:34:43.438 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.437+0000 7f28c7200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f28c80ce6a0 con 0x7f28c815d1e0 2024-04-20T11:34:43.438 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.438+0000 7f28c5e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f28c8071cd0 0x7f28c81f4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:43.438 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.438+0000 7f28c5e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f28c8071cd0 0x7f28c81f4220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:50968/0 (socket says 172.21.15.152:50968) 2024-04-20T11:34:43.438 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.438+0000 7f28c5e00640 1 -- 172.21.15.152:0/976325769 learned_addr learned my addr 172.21.15.152:0/976325769 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:43.438 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.438+0000 7f28c5e00640 1 -- 172.21.15.152:0/976325769 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f28c815d1e0 msgr2=0x7f28c814fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:43.438 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.438+0000 7f28c5e00640 1 --2- 172.21.15.152:0/976325769 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f28c815d1e0 0x7f28c814fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:43.438 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.438+0000 7f28c5e00640 1 -- 172.21.15.152:0/976325769 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28c815d5c0 msgr2=0x7f28c8150410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:43.438 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.438+0000 7f28c5e00640 1 --2- 172.21.15.152:0/976325769 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28c815d5c0 0x7f28c8150410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:43.438 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.438+0000 7f28c5e00640 1 -- 172.21.15.152:0/976325769 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28c8150950 con 0x7f28c8071cd0 2024-04-20T11:34:43.439 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.438+0000 7f28c5e00640 1 --2- 172.21.15.152:0/976325769 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f28c8071cd0 0x7f28c81f4220 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f28b8000c00 tx=0x7f28b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:43.439 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.439+0000 7f28be000640 1 -- 172.21.15.152:0/976325769 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b8040070 con 0x7f28c8071cd0 2024-04-20T11:34:43.439 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.439+0000 7f28be000640 1 -- 172.21.15.152:0/976325769 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28b8039bf0 con 0x7f28c8071cd0 2024-04-20T11:34:43.439 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.439+0000 7f28be000640 1 -- 172.21.15.152:0/976325769 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b8035040 con 0x7f28c8071cd0 2024-04-20T11:34:43.439 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.439+0000 7f28c7200640 1 -- 172.21.15.152:0/976325769 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28c8150c00 con 0x7f28c8071cd0 2024-04-20T11:34:43.439 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.439+0000 7f28c7200640 1 -- 172.21.15.152:0/976325769 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28c81514a0 con 0x7f28c8071cd0 2024-04-20T11:34:43.440 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.440+0000 7f28c7200640 1 -- 172.21.15.152:0/976325769 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28c8150ff0 con 0x7f28c8071cd0 2024-04-20T11:34:43.440 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.441+0000 7f28be000640 1 -- 172.21.15.152:0/976325769 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 19) ==== 91397+0+0 (secure 0 0 0) 0x7f28b804b060 con 0x7f28c8071cd0 2024-04-20T11:34:43.441 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.442+0000 7f28be000640 1 --2- 172.21.15.152:0/976325769 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f28a806cce0 0x7f28a806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:43.441 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.442+0000 7f28be000640 1 -- 172.21.15.152:0/976325769 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5709+0+0 (secure 0 0 0) 0x7f28b80add40 con 0x7f28c8071cd0 2024-04-20T11:34:43.442 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.442+0000 7f28c5400640 1 --2- 172.21.15.152:0/976325769 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f28a806cce0 0x7f28a806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:43.443 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.444+0000 7f28c5400640 1 --2- 172.21.15.152:0/976325769 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f28a806cce0 0x7f28a806f1a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f28b4006fd0 tx=0x7f28b4008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:43.445 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.446+0000 7f28be000640 1 -- 172.21.15.152:0/976325769 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f28b8031610 con 0x7f28c8071cd0 2024-04-20T11:34:43.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.686+0000 7f28c7200640 1 -- 172.21.15.152:0/976325769 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f28c8160b90 con 0x7f28c8071cd0 2024-04-20T11:34:43.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.686+0000 7f28be000640 1 -- 172.21.15.152:0/976325769 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v76) ==== 53+0+73 (secure 0 0 0) 0x7f28b807bc20 con 0x7f28c8071cd0 2024-04-20T11:34:43.686 INFO:teuthology.orchestra.run.smithi152.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-20T11:34:43.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.689+0000 7f28c7200640 1 -- 172.21.15.152:0/976325769 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f28a806cce0 msgr2=0x7f28a806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:43.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.689+0000 7f28c7200640 1 --2- 172.21.15.152:0/976325769 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f28a806cce0 0x7f28a806f1a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f28b4006fd0 tx=0x7f28b4008040 comp rx=0 tx=0).stop 2024-04-20T11:34:43.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.689+0000 7f28c7200640 1 -- 172.21.15.152:0/976325769 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f28c8071cd0 msgr2=0x7f28c81f4220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:43.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.689+0000 7f28c7200640 1 --2- 172.21.15.152:0/976325769 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f28c8071cd0 0x7f28c81f4220 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f28b8000c00 tx=0x7f28b80046c0 comp rx=0 tx=0).stop 2024-04-20T11:34:43.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.689+0000 7f28c7200640 1 -- 172.21.15.152:0/976325769 shutdown_connections 2024-04-20T11:34:43.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.689+0000 7f28c7200640 1 --2- 172.21.15.152:0/976325769 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f28a806cce0 0x7f28a806f1a0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:43.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.689+0000 7f28c7200640 1 --2- 172.21.15.152:0/976325769 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28c815d5c0 0x7f28c8150410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:43.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.689+0000 7f28c7200640 1 --2- 172.21.15.152:0/976325769 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f28c815d1e0 0x7f28c814fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:43.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.689+0000 7f28c7200640 1 --2- 172.21.15.152:0/976325769 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f28c8071cd0 0x7f28c81f4220 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:43.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.689+0000 7f28c7200640 1 -- 172.21.15.152:0/976325769 >> 172.21.15.152:0/976325769 conn(0x7f28c8067ae0 msgr2=0x7f28c806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:43.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.689+0000 7f28c7200640 1 -- 172.21.15.152:0/976325769 shutdown_connections 2024-04-20T11:34:43.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:43.689+0000 7f28c7200640 1 -- 172.21.15.152:0/976325769 wait complete. 2024-04-20T11:34:43.707 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-20T11:34:43.707 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-20T11:34:43.707 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi152.front.sepia.ceph.com 2024-04-20T11:34:43.707 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-20T11:34:43.707 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-20T11:34:43.707 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-20T11:34:43.707 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-20T11:34:43.707 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-20T11:34:43.707 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-20T11:34:43.707 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:34:43.708 DEBUG:teuthology.orchestra.run.smithi152:> ip addr 2024-04-20T11:34:43.755 INFO:teuthology.orchestra.run.smithi152.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-20T11:34:43.755 INFO:teuthology.orchestra.run.smithi152.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-20T11:34:43.755 INFO:teuthology.orchestra.run.smithi152.stdout: inet 127.0.0.1/8 scope host lo 2024-04-20T11:34:43.755 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:34:43.755 INFO:teuthology.orchestra.run.smithi152.stdout: inet6 ::1/128 scope host 2024-04-20T11:34:43.755 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:34:43.755 INFO:teuthology.orchestra.run.smithi152.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T11:34:43.756 INFO:teuthology.orchestra.run.smithi152.stdout: link/ether 0c:c4:7a:da:6c:44 brd ff:ff:ff:ff:ff:ff 2024-04-20T11:34:43.756 INFO:teuthology.orchestra.run.smithi152.stdout: altname enp7s0f0 2024-04-20T11:34:43.756 INFO:teuthology.orchestra.run.smithi152.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T11:34:43.756 INFO:teuthology.orchestra.run.smithi152.stdout: link/ether 0c:c4:7a:da:6c:45 brd ff:ff:ff:ff:ff:ff 2024-04-20T11:34:43.756 INFO:teuthology.orchestra.run.smithi152.stdout: altname enp7s0f1 2024-04-20T11:34:43.756 INFO:teuthology.orchestra.run.smithi152.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T11:34:43.756 INFO:teuthology.orchestra.run.smithi152.stdout: link/ether 0c:c4:7a:88:6d:e2 brd ff:ff:ff:ff:ff:ff 2024-04-20T11:34:43.756 INFO:teuthology.orchestra.run.smithi152.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-20T11:34:43.756 INFO:teuthology.orchestra.run.smithi152.stdout: link/ether 0c:c4:7a:88:6d:e3 brd ff:ff:ff:ff:ff:ff 2024-04-20T11:34:43.756 INFO:teuthology.orchestra.run.smithi152.stdout: inet 172.21.15.152/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-20T11:34:43.756 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft 42322sec preferred_lft 42322sec 2024-04-20T11:34:43.756 INFO:teuthology.orchestra.run.smithi152.stdout: inet6 fe80::ec4:7aff:fe88:6de3/64 scope link 2024-04-20T11:34:43.756 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:34:43.756 INFO:teuthology.orchestra.run.smithi152.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-20T11:34:43.756 INFO:teuthology.orchestra.run.smithi152.stdout: link/ether 06:3e:9a:d3:49:1e brd ff:ff:ff:ff:ff:ff 2024-04-20T11:34:43.756 INFO:teuthology.orchestra.run.smithi152.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-20T11:34:43.757 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:34:43.757 INFO:teuthology.orchestra.run.smithi152.stdout: inet6 fe80::a019:d8ff:fe70:1185/64 scope link 2024-04-20T11:34:43.757 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:34:43.757 INFO:teuthology.orchestra.run.smithi152.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-20T11:34:43.757 INFO:teuthology.orchestra.run.smithi152.stdout: link/ether 16:e5:08:78:20:47 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T11:34:43.757 INFO:teuthology.orchestra.run.smithi152.stdout: inet6 fe80::14e5:8ff:fe78:2047/64 scope link 2024-04-20T11:34:43.757 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:34:43.757 INFO:teuthology.orchestra.run.smithi152.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-20T11:34:43.757 INFO:teuthology.orchestra.run.smithi152.stdout: link/ether 06:3e:9a:d3:49:1e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T11:34:43.757 INFO:teuthology.orchestra.run.smithi152.stdout: inet6 fe80::43e:9aff:fed3:491e/64 scope link 2024-04-20T11:34:43.757 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:34:43.758 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:34:43.758 DEBUG:teuthology.orchestra.run.smithi152:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-20T11:34:43.879 INFO:teuthology.orchestra.run.smithi152.stdout:1 2024-04-20T11:34:43.883 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:34:43.883 DEBUG:teuthology.orchestra.run.smithi152:> route 2024-04-20T11:34:43.931 INFO:teuthology.orchestra.run.smithi152.stdout:Kernel IP routing table 2024-04-20T11:34:43.931 INFO:teuthology.orchestra.run.smithi152.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-20T11:34:43.931 INFO:teuthology.orchestra.run.smithi152.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-20T11:34:43.932 INFO:teuthology.orchestra.run.smithi152.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-20T11:34:43.932 INFO:teuthology.orchestra.run.smithi152.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-20T11:34:43.932 DEBUG:teuthology.orchestra.run.smithi152:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T11:34:43.932 DEBUG:teuthology.orchestra.run.smithi152:> set -e 2024-04-20T11:34:43.932 DEBUG:teuthology.orchestra.run.smithi152:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-20T11:34:43.932 DEBUG:teuthology.orchestra.run.smithi152:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-20T11:34:43.932 DEBUG:teuthology.orchestra.run.smithi152:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-20T11:34:43.932 DEBUG:teuthology.orchestra.run.smithi152:> ') 2024-04-20T11:34:44.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T11:34:44.135 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T11:34:44.145 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:34:44.145 DEBUG:teuthology.orchestra.run.smithi152:> ip netns list 2024-04-20T11:34:44.193 INFO:teuthology.orchestra.run.smithi152.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-20T11:34:44.193 INFO:teuthology.orchestra.run.smithi152.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-20T11:34:44.194 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-20T11:34:44.194 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:34:44.194 DEBUG:teuthology.orchestra.run.smithi152:> sudo ip link set brx.0 up 2024-04-20T11:34:44.273 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-20T11:34:44.273 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T11:34:44.273 DEBUG:teuthology.orchestra.run.smithi152:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-20T11:34:44.327 INFO:teuthology.orchestra.run.smithi152.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-20T11:34:44.327 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T11:34:44.327 DEBUG:teuthology.orchestra.run.smithi152:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-20T11:34:44.376 DEBUG:teuthology.orchestra.run.smithi152:> sudo modprobe fuse 2024-04-20T11:34:44.454 DEBUG:teuthology.orchestra.run.smithi152:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T11:34:44.507 INFO:teuthology.orchestra.run.smithi152.stdout:/proc 2024-04-20T11:34:44.507 INFO:teuthology.orchestra.run.smithi152.stdout:/sys 2024-04-20T11:34:44.507 INFO:teuthology.orchestra.run.smithi152.stdout:/dev 2024-04-20T11:34:44.507 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/security 2024-04-20T11:34:44.507 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/shm 2024-04-20T11:34:44.507 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/pts 2024-04-20T11:34:44.507 INFO:teuthology.orchestra.run.smithi152.stdout:/run 2024-04-20T11:34:44.507 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/cgroup 2024-04-20T11:34:44.507 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/pstore 2024-04-20T11:34:44.508 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/bpf 2024-04-20T11:34:44.508 INFO:teuthology.orchestra.run.smithi152.stdout:/ 2024-04-20T11:34:44.508 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/selinux 2024-04-20T11:34:44.508 INFO:teuthology.orchestra.run.smithi152.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T11:34:44.508 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/mqueue 2024-04-20T11:34:44.508 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/hugepages 2024-04-20T11:34:44.508 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/debug 2024-04-20T11:34:44.508 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/tracing 2024-04-20T11:34:44.508 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/fuse/connections 2024-04-20T11:34:44.508 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/config 2024-04-20T11:34:44.508 INFO:teuthology.orchestra.run.smithi152.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T11:34:44.508 INFO:teuthology.orchestra.run.smithi152.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T11:34:44.508 INFO:teuthology.orchestra.run.smithi152.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T11:34:44.508 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T11:34:44.508 INFO:teuthology.orchestra.run.smithi152.stdout:/run/user/1000 2024-04-20T11:34:44.509 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph 2024-04-20T11:34:44.509 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph/osd/ceph-0 2024-04-20T11:34:44.509 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph/osd/ceph-1 2024-04-20T11:34:44.509 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph/osd/ceph-2 2024-04-20T11:34:44.509 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph/osd/ceph-3 2024-04-20T11:34:44.509 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns 2024-04-20T11:34:44.509 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T11:34:44.509 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T11:34:44.509 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T11:34:44.509 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T11:34:44.510 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:34:44.510 DEBUG:teuthology.orchestra.run.smithi152:> ls /sys/fs/fuse/connections 2024-04-20T11:34:44.559 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-20T11:34:44.560 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-20T11:34:44.564 DEBUG:teuthology.orchestra.run.smithi152:> sudo modprobe fuse 2024-04-20T11:34:44.640 DEBUG:teuthology.orchestra.run.smithi152:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T11:34:44.681 INFO:teuthology.orchestra.run.smithi152.stdout:/proc 2024-04-20T11:34:44.681 INFO:teuthology.orchestra.run.smithi152.stdout:/sys 2024-04-20T11:34:44.681 INFO:teuthology.orchestra.run.smithi152.stdout:/dev 2024-04-20T11:34:44.681 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/security 2024-04-20T11:34:44.681 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/shm 2024-04-20T11:34:44.681 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/pts 2024-04-20T11:34:44.681 INFO:teuthology.orchestra.run.smithi152.stdout:/run 2024-04-20T11:34:44.681 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/cgroup 2024-04-20T11:34:44.681 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/pstore 2024-04-20T11:34:44.682 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/bpf 2024-04-20T11:34:44.682 INFO:teuthology.orchestra.run.smithi152.stdout:/ 2024-04-20T11:34:44.682 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/selinux 2024-04-20T11:34:44.682 INFO:teuthology.orchestra.run.smithi152.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T11:34:44.682 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/mqueue 2024-04-20T11:34:44.682 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/hugepages 2024-04-20T11:34:44.682 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/debug 2024-04-20T11:34:44.682 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/tracing 2024-04-20T11:34:44.682 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/fuse/connections 2024-04-20T11:34:44.682 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/config 2024-04-20T11:34:44.682 INFO:teuthology.orchestra.run.smithi152.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T11:34:44.682 INFO:teuthology.orchestra.run.smithi152.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T11:34:44.682 INFO:teuthology.orchestra.run.smithi152.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T11:34:44.683 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T11:34:44.683 INFO:teuthology.orchestra.run.smithi152.stdout:/run/user/1000 2024-04-20T11:34:44.683 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph 2024-04-20T11:34:44.683 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph/osd/ceph-0 2024-04-20T11:34:44.683 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph/osd/ceph-1 2024-04-20T11:34:44.683 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph/osd/ceph-2 2024-04-20T11:34:44.683 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph/osd/ceph-3 2024-04-20T11:34:44.683 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns 2024-04-20T11:34:44.683 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T11:34:44.683 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T11:34:44.683 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T11:34:44.683 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T11:34:44.684 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:34:44.684 DEBUG:teuthology.orchestra.run.smithi152:> ls /sys/fs/fuse/connections 2024-04-20T11:34:44.741 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi152.stderr:2024-04-20T11:34:44.742+0000 7ff93943c480 -1 init, newargv = 0x559a202094d0 newargc=15 2024-04-20T11:34:44.742 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi152.stderr:ceph-fuse[60265]: starting ceph client 2024-04-20T11:34:44.752 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi152.stderr:ceph-fuse[60265]: starting fuse 2024-04-20T11:34:44.768 INFO:teuthology.orchestra.run.smithi152.stdout:42 2024-04-20T11:34:44.768 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-04-20T11:34:44.768 DEBUG:teuthology.orchestra.run.smithi152:> sudo stdin-killer -- python3 -c ' 2024-04-20T11:34:44.768 DEBUG:teuthology.orchestra.run.smithi152:> import glob 2024-04-20T11:34:44.768 DEBUG:teuthology.orchestra.run.smithi152:> import re 2024-04-20T11:34:44.768 DEBUG:teuthology.orchestra.run.smithi152:> import os 2024-04-20T11:34:44.768 DEBUG:teuthology.orchestra.run.smithi152:> import subprocess 2024-04-20T11:34:44.768 DEBUG:teuthology.orchestra.run.smithi152:> 2024-04-20T11:34:44.768 DEBUG:teuthology.orchestra.run.smithi152:> def _find_admin_socket(client_name): 2024-04-20T11:34:44.769 DEBUG:teuthology.orchestra.run.smithi152:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-20T11:34:44.769 DEBUG:teuthology.orchestra.run.smithi152:> files = glob.glob(asok_path) 2024-04-20T11:34:44.769 DEBUG:teuthology.orchestra.run.smithi152:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-20T11:34:44.769 DEBUG:teuthology.orchestra.run.smithi152:> 2024-04-20T11:34:44.769 DEBUG:teuthology.orchestra.run.smithi152:> # Given a non-glob path, it better be there 2024-04-20T11:34:44.769 DEBUG:teuthology.orchestra.run.smithi152:> if "*" not in asok_path: 2024-04-20T11:34:44.769 DEBUG:teuthology.orchestra.run.smithi152:> assert(len(files) == 1) 2024-04-20T11:34:44.769 DEBUG:teuthology.orchestra.run.smithi152:> return files[0] 2024-04-20T11:34:44.769 DEBUG:teuthology.orchestra.run.smithi152:> 2024-04-20T11:34:44.769 DEBUG:teuthology.orchestra.run.smithi152:> for f in files: 2024-04-20T11:34:44.769 DEBUG:teuthology.orchestra.run.smithi152:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-20T11:34:44.769 DEBUG:teuthology.orchestra.run.smithi152:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-20T11:34:44.769 DEBUG:teuthology.orchestra.run.smithi152:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-20T11:34:44.769 DEBUG:teuthology.orchestra.run.smithi152:> contents = proc_f.read() 2024-04-20T11:34:44.769 DEBUG:teuthology.orchestra.run.smithi152:> if mountpoint in contents: 2024-04-20T11:34:44.769 DEBUG:teuthology.orchestra.run.smithi152:> return f 2024-04-20T11:34:44.770 DEBUG:teuthology.orchestra.run.smithi152:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-20T11:34:44.770 DEBUG:teuthology.orchestra.run.smithi152:> 2024-04-20T11:34:44.770 DEBUG:teuthology.orchestra.run.smithi152:> print(_find_admin_socket("client.0")) 2024-04-20T11:34:44.770 DEBUG:teuthology.orchestra.run.smithi152:> ' 2024-04-20T11:34:44.956 INFO:teuthology.orchestra.run.smithi152.stdout:/var/run/ceph/ceph-client.0.60265.asok 2024-04-20T11:34:44.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T11:34:44.972 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.60265.asok 2024-04-20T11:34:44.972 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:34:44.972 DEBUG:teuthology.orchestra.run.smithi152:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.60265.asok status 2024-04-20T11:34:45.199 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-04-20T11:34:45.199 INFO:teuthology.orchestra.run.smithi152.stdout: "metadata": { 2024-04-20T11:34:45.199 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_sha1": "a8c84f5997950fd7d008204839253291d52e6eda", 2024-04-20T11:34:45.200 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", 2024-04-20T11:34:45.200 INFO:teuthology.orchestra.run.smithi152.stdout: "entity_id": "0", 2024-04-20T11:34:45.200 INFO:teuthology.orchestra.run.smithi152.stdout: "hostname": "smithi152", 2024-04-20T11:34:45.200 INFO:teuthology.orchestra.run.smithi152.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-20T11:34:45.200 INFO:teuthology.orchestra.run.smithi152.stdout: "pid": "60265", 2024-04-20T11:34:45.200 INFO:teuthology.orchestra.run.smithi152.stdout: "root": "/" 2024-04-20T11:34:45.200 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:34:45.200 INFO:teuthology.orchestra.run.smithi152.stdout: "dentry_count": 0, 2024-04-20T11:34:45.200 INFO:teuthology.orchestra.run.smithi152.stdout: "dentry_pinned_count": 0, 2024-04-20T11:34:45.200 INFO:teuthology.orchestra.run.smithi152.stdout: "id": 8574, 2024-04-20T11:34:45.200 INFO:teuthology.orchestra.run.smithi152.stdout: "inst": { 2024-04-20T11:34:45.200 INFO:teuthology.orchestra.run.smithi152.stdout: "name": { 2024-04-20T11:34:45.200 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "client", 2024-04-20T11:34:45.200 INFO:teuthology.orchestra.run.smithi152.stdout: "num": 8574 2024-04-20T11:34:45.201 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:34:45.201 INFO:teuthology.orchestra.run.smithi152.stdout: "addr": { 2024-04-20T11:34:45.201 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "v1", 2024-04-20T11:34:45.201 INFO:teuthology.orchestra.run.smithi152.stdout: "addr": "192.168.0.1:0", 2024-04-20T11:34:45.201 INFO:teuthology.orchestra.run.smithi152.stdout: "nonce": 3557015060 2024-04-20T11:34:45.201 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:34:45.201 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:34:45.201 INFO:teuthology.orchestra.run.smithi152.stdout: "addr": { 2024-04-20T11:34:45.201 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "v1", 2024-04-20T11:34:45.201 INFO:teuthology.orchestra.run.smithi152.stdout: "addr": "192.168.0.1:0", 2024-04-20T11:34:45.201 INFO:teuthology.orchestra.run.smithi152.stdout: "nonce": 3557015060 2024-04-20T11:34:45.201 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:34:45.201 INFO:teuthology.orchestra.run.smithi152.stdout: "inst_str": "client.8574 192.168.0.1:0/3557015060", 2024-04-20T11:34:45.201 INFO:teuthology.orchestra.run.smithi152.stdout: "addr_str": "192.168.0.1:0/3557015060", 2024-04-20T11:34:45.201 INFO:teuthology.orchestra.run.smithi152.stdout: "inode_count": 1, 2024-04-20T11:34:45.201 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_epoch": 76, 2024-04-20T11:34:45.202 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_epoch": 118, 2024-04-20T11:34:45.202 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_epoch_barrier": 0, 2024-04-20T11:34:45.202 INFO:teuthology.orchestra.run.smithi152.stdout: "blocklisted": false, 2024-04-20T11:34:45.202 INFO:teuthology.orchestra.run.smithi152.stdout: "fs_name": "cephfs" 2024-04-20T11:34:45.202 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-04-20T11:34:45.214 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:34:45.214 DEBUG:teuthology.orchestra.run.smithi152:> stat --file-system '--printf=%T 2024-04-20T11:34:45.214 DEBUG:teuthology.orchestra.run.smithi152:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-20T11:34:45.265 INFO:teuthology.orchestra.run.smithi152.stdout:fuseblk 2024-04-20T11:34:45.265 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-20T11:34:45.265 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:34:45.266 DEBUG:teuthology.orchestra.run.smithi152:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-20T11:34:45.342 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-20T11:34:45.543 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:45.544+0000 7fc985800640 1 Processor -- start 2024-04-20T11:34:45.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:45.544+0000 7fc985800640 1 -- start start 2024-04-20T11:34:45.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:45.545+0000 7fc985800640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fc98015c560 0x7fc9800b3df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:45.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:45.546+0000 7fc985800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc9800d0810 0x7fc9800ada90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:45.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:45.546+0000 7fc985800640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc98015fd20 con 0x7fc9800b4330 2024-04-20T11:34:45.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:45.546+0000 7fc985800640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fc98015ef60 con 0x7fc98015c560 2024-04-20T11:34:45.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:45.546+0000 7fc985800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc98015f240 con 0x7fc9800d0810 2024-04-20T11:34:45.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:45.546+0000 7fc97e000640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fc9800b4330 0x7fc9800d0100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:41200/0 (socket says 172.21.15.152:41200) 2024-04-20T11:34:45.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:45.546+0000 7fc97e000640 1 -- 172.21.15.152:0/1840428330 learned_addr learned my addr 172.21.15.152:0/1840428330 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:34:45.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:45.546+0000 7fc97f400640 1 --2- 172.21.15.152:0/1840428330 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc9800d0810 0x7fc9800ada90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:45.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:45.546+0000 7fc97ea00640 1 --2- 172.21.15.152:0/1840428330 >> v2:172.21.15.175:3300/0 conn(0x7fc98015c560 0x7fc9800b3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:45.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:45.547+0000 7fc97d600640 1 -- 172.21.15.152:0/1840428330 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 383413340 0 0) 0x7fc98015fd20 con 0x7fc9800b4330 2024-04-20T11:34:45.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:45.547+0000 7fc97d600640 1 -- 172.21.15.152:0/1840428330 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc964003520 con 0x7fc9800b4330 2024-04-20T11:34:45.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:45.547+0000 7fc97d600640 1 -- 172.21.15.152:0/1840428330 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1012733450 0 0) 0x7fc964003520 con 0x7fc9800b4330 2024-04-20T11:34:45.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:45.547+0000 7fc97f400640 1 -- 172.21.15.152:0/1840428330 >> v2:172.21.15.175:3300/0 conn(0x7fc98015c560 msgr2=0x7fc9800b3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:45.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:45.547+0000 7fc97f400640 1 --2- 172.21.15.152:0/1840428330 >> v2:172.21.15.175:3300/0 conn(0x7fc98015c560 0x7fc9800b3df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:45.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:45.547+0000 7fc97f400640 1 -- 172.21.15.152:0/1840428330 >> v1:172.21.15.152:6789/0 conn(0x7fc9800b4330 legacy=0x7fc9800d0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:45.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:45.547+0000 7fc97f400640 1 -- 172.21.15.152:0/1840428330 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc98015f520 con 0x7fc9800d0810 2024-04-20T11:34:45.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:45.547+0000 7fc97ea00640 1 --2- 172.21.15.152:0/1840428330 >> v2:172.21.15.175:3300/0 conn(0x7fc98015c560 0x7fc9800b3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:34:45.548 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:45.548+0000 7fc97f400640 1 --2- 172.21.15.152:0/1840428330 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc9800d0810 0x7fc9800ada90 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc978009d90 tx=0x7fc97802bce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4be550b3c89682d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:45.548 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:45.549+0000 7fc97d600640 1 -- 172.21.15.152:0/1840428330 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc978039070 con 0x7fc9800d0810 2024-04-20T11:34:45.548 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:45.549+0000 7fc97d600640 1 -- 172.21.15.152:0/1840428330 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc97802c810 con 0x7fc9800d0810 2024-04-20T11:34:45.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:45.549+0000 7fc97d600640 1 -- 172.21.15.152:0/1840428330 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc96c001670 con 0x7fc9800d0810 2024-04-20T11:34:45.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:45.550+0000 7fc985800640 1 -- 172.21.15.152:0/1840428330 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc9800d0810 msgr2=0x7fc9800ada90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:45.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:45.550+0000 7fc985800640 1 --2- 172.21.15.152:0/1840428330 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc9800d0810 0x7fc9800ada90 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc978009d90 tx=0x7fc97802bce0 comp rx=0 tx=0).stop 2024-04-20T11:34:45.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:45.550+0000 7fc985800640 1 -- 172.21.15.152:0/1840428330 shutdown_connections 2024-04-20T11:34:45.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:45.550+0000 7fc985800640 1 --2- 172.21.15.152:0/1840428330 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc9800d0810 0x7fc9800ada90 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:45.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:45.550+0000 7fc985800640 1 --2- 172.21.15.152:0/1840428330 >> v2:172.21.15.175:3300/0 conn(0x7fc98015c560 0x7fc9800b3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:45.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:45.550+0000 7fc985800640 1 -- 172.21.15.152:0/1840428330 >> 172.21.15.152:0/1840428330 conn(0x7fc980067cf0 msgr2=0x7fc98006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:45.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:45.550+0000 7fc985800640 1 -- 172.21.15.152:0/1840428330 shutdown_connections 2024-04-20T11:34:45.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:45.550+0000 7fc985800640 1 -- 172.21.15.152:0/1840428330 wait complete. 2024-04-20T11:34:45.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:45.552+0000 7fc985800640 1 Processor -- start 2024-04-20T11:34:45.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:45.552+0000 7fc985800640 1 -- start start 2024-04-20T11:34:45.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:45.553+0000 7fc985800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc9800b4330 0x7fc98015ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:45.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:45.553+0000 7fc985800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc9800d0810 0x7fc98014ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:45.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:45.553+0000 7fc985800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc98015c560 0x7fc980161790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:45.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:45.553+0000 7fc985800640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fc9800adfd0 con 0x7fc9800b4330 2024-04-20T11:34:45.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:45.553+0000 7fc985800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc9800d0dc0 con 0x7fc98015c560 2024-04-20T11:34:45.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:45.553+0000 7fc985800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc9800ae150 con 0x7fc9800d0810 2024-04-20T11:34:45.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:45.553+0000 7fc97ea00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc9800b4330 0x7fc98015ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:45.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:45.553+0000 7fc97ea00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc9800b4330 0x7fc98015ff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:50976/0 (socket says 172.21.15.152:50976) 2024-04-20T11:34:45.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:45.553+0000 7fc97ea00640 1 -- 172.21.15.152:0/3721342214 learned_addr learned my addr 172.21.15.152:0/3721342214 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:45.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:45.553+0000 7fc97e000640 1 --2- 172.21.15.152:0/3721342214 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc9800d0810 0x7fc98014ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:45.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:45.554+0000 7fc97f400640 1 --2- 172.21.15.152:0/3721342214 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc98015c560 0x7fc980161790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:45.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:45.554+0000 7fc97ea00640 1 -- 172.21.15.152:0/3721342214 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc9800d0810 msgr2=0x7fc98014ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:45.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:45.554+0000 7fc97ea00640 1 --2- 172.21.15.152:0/3721342214 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc9800d0810 0x7fc98014ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:45.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:45.554+0000 7fc97ea00640 1 -- 172.21.15.152:0/3721342214 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc98015c560 msgr2=0x7fc980161790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:45.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:45.554+0000 7fc97ea00640 1 --2- 172.21.15.152:0/3721342214 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc98015c560 0x7fc980161790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:45.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:45.554+0000 7fc97ea00640 1 -- 172.21.15.152:0/3721342214 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc980160450 con 0x7fc9800b4330 2024-04-20T11:34:45.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:45.554+0000 7fc97f400640 1 --2- 172.21.15.152:0/3721342214 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc98015c560 0x7fc980161790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:34:45.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:45.554+0000 7fc97ea00640 1 --2- 172.21.15.152:0/3721342214 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc9800b4330 0x7fc98015ff10 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7fc97000e960 tx=0x7fc97000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:45.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:45.554+0000 7fc976a00640 1 -- 172.21.15.152:0/3721342214 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc970017070 con 0x7fc9800b4330 2024-04-20T11:34:45.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:45.554+0000 7fc976a00640 1 -- 172.21.15.152:0/3721342214 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc97000ca00 con 0x7fc9800b4330 2024-04-20T11:34:45.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:45.554+0000 7fc976a00640 1 -- 172.21.15.152:0/3721342214 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc97000cdd0 con 0x7fc9800b4330 2024-04-20T11:34:45.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:45.554+0000 7fc985800640 1 -- 172.21.15.152:0/3721342214 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9801606f0 con 0x7fc9800b4330 2024-04-20T11:34:45.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:45.554+0000 7fc985800640 1 -- 172.21.15.152:0/3721342214 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc980160cb0 con 0x7fc9800b4330 2024-04-20T11:34:45.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:45.556+0000 7fc985800640 1 -- 172.21.15.152:0/3721342214 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9800689f0 con 0x7fc9800b4330 2024-04-20T11:34:45.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:45.557+0000 7fc976a00640 1 -- 172.21.15.152:0/3721342214 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 19) ==== 91397+0+0 (secure 0 0 0) 0x7fc97001a070 con 0x7fc9800b4330 2024-04-20T11:34:45.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:45.557+0000 7fc976a00640 1 --2- 172.21.15.152:0/3721342214 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fc96006cd10 0x7fc96006f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:45.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:45.558+0000 7fc976a00640 1 -- 172.21.15.152:0/3721342214 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5709+0+0 (secure 0 0 0) 0x7fc97008c260 con 0x7fc9800b4330 2024-04-20T11:34:45.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:45.558+0000 7fc97e000640 1 --2- 172.21.15.152:0/3721342214 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fc96006cd10 0x7fc96006f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:45.559 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:45.559+0000 7fc97e000640 1 --2- 172.21.15.152:0/3721342214 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fc96006cd10 0x7fc96006f1d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fc9800ae7b0 tx=0x7fc96c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:45.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:45.562+0000 7fc976a00640 1 -- 172.21.15.152:0/3721342214 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fc970059fe0 con 0x7fc9800b4330 2024-04-20T11:34:45.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:45.805+0000 7fc985800640 1 -- 172.21.15.152:0/3721342214 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fc9800d0060 con 0x7fc9800b4330 2024-04-20T11:34:45.805 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:45.806+0000 7fc976a00640 1 -- 172.21.15.152:0/3721342214 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v76) ==== 53+0+73 (secure 0 0 0) 0x7fc9700598e0 con 0x7fc9800b4330 2024-04-20T11:34:45.805 INFO:teuthology.orchestra.run.smithi152.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-20T11:34:45.809 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:45.809+0000 7fc985800640 1 -- 172.21.15.152:0/3721342214 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fc96006cd10 msgr2=0x7fc96006f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:45.809 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:45.809+0000 7fc985800640 1 --2- 172.21.15.152:0/3721342214 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fc96006cd10 0x7fc96006f1d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fc9800ae7b0 tx=0x7fc96c009040 comp rx=0 tx=0).stop 2024-04-20T11:34:45.809 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:45.809+0000 7fc985800640 1 -- 172.21.15.152:0/3721342214 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc9800b4330 msgr2=0x7fc98015ff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:45.809 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:45.809+0000 7fc985800640 1 --2- 172.21.15.152:0/3721342214 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc9800b4330 0x7fc98015ff10 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7fc97000e960 tx=0x7fc97000ee20 comp rx=0 tx=0).stop 2024-04-20T11:34:45.809 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:45.809+0000 7fc985800640 1 -- 172.21.15.152:0/3721342214 shutdown_connections 2024-04-20T11:34:45.809 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:45.809+0000 7fc985800640 1 --2- 172.21.15.152:0/3721342214 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fc96006cd10 0x7fc96006f1d0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:45.809 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:45.809+0000 7fc985800640 1 --2- 172.21.15.152:0/3721342214 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc98015c560 0x7fc980161790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:45.809 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:45.809+0000 7fc985800640 1 --2- 172.21.15.152:0/3721342214 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc9800d0810 0x7fc98014ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:45.809 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:45.809+0000 7fc985800640 1 --2- 172.21.15.152:0/3721342214 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc9800b4330 0x7fc98015ff10 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:45.809 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:45.809+0000 7fc985800640 1 -- 172.21.15.152:0/3721342214 >> 172.21.15.152:0/3721342214 conn(0x7fc980067cf0 msgr2=0x7fc98006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:45.809 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:45.809+0000 7fc985800640 1 -- 172.21.15.152:0/3721342214 shutdown_connections 2024-04-20T11:34:45.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:45.809+0000 7fc985800640 1 -- 172.21.15.152:0/3721342214 wait complete. 2024-04-20T11:34:45.827 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-20T11:34:45.827 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-20T11:34:45.827 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi175.front.sepia.ceph.com 2024-04-20T11:34:45.827 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-20T11:34:45.827 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-20T11:34:45.827 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-20T11:34:45.827 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-20T11:34:45.827 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-20T11:34:45.827 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-20T11:34:45.827 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:34:45.828 DEBUG:teuthology.orchestra.run.smithi175:> ip addr 2024-04-20T11:34:45.883 INFO:teuthology.orchestra.run.smithi175.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-20T11:34:45.884 INFO:teuthology.orchestra.run.smithi175.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-20T11:34:45.884 INFO:teuthology.orchestra.run.smithi175.stdout: inet 127.0.0.1/8 scope host lo 2024-04-20T11:34:45.884 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:34:45.884 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 ::1/128 scope host 2024-04-20T11:34:45.884 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:34:45.884 INFO:teuthology.orchestra.run.smithi175.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T11:34:45.884 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:d9:39:ee brd ff:ff:ff:ff:ff:ff 2024-04-20T11:34:45.884 INFO:teuthology.orchestra.run.smithi175.stdout: altname enp7s0f0 2024-04-20T11:34:45.884 INFO:teuthology.orchestra.run.smithi175.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T11:34:45.884 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:d9:39:ef brd ff:ff:ff:ff:ff:ff 2024-04-20T11:34:45.884 INFO:teuthology.orchestra.run.smithi175.stdout: altname enp7s0f1 2024-04-20T11:34:45.884 INFO:teuthology.orchestra.run.smithi175.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T11:34:45.884 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:88:71:f0 brd ff:ff:ff:ff:ff:ff 2024-04-20T11:34:45.884 INFO:teuthology.orchestra.run.smithi175.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-20T11:34:45.885 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:88:71:f1 brd ff:ff:ff:ff:ff:ff 2024-04-20T11:34:45.885 INFO:teuthology.orchestra.run.smithi175.stdout: inet 172.21.15.175/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-20T11:34:45.885 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft 42295sec preferred_lft 42295sec 2024-04-20T11:34:45.885 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::ec4:7aff:fe88:71f1/64 scope link 2024-04-20T11:34:45.885 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:34:45.885 INFO:teuthology.orchestra.run.smithi175.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-20T11:34:45.885 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 02:d6:c6:31:26:70 brd ff:ff:ff:ff:ff:ff 2024-04-20T11:34:45.885 INFO:teuthology.orchestra.run.smithi175.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-20T11:34:45.885 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:34:45.885 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::5091:d5ff:fee7:54b3/64 scope link 2024-04-20T11:34:45.885 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:34:45.885 INFO:teuthology.orchestra.run.smithi175.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-20T11:34:45.885 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 02:d6:c6:31:26:70 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-20T11:34:45.885 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::d6:c6ff:fe31:2670/64 scope link 2024-04-20T11:34:45.886 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:34:45.886 INFO:teuthology.orchestra.run.smithi175.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-20T11:34:45.886 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 9e:14:f3:e2:18:1e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-20T11:34:45.886 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::9c14:f3ff:fee2:181e/64 scope link 2024-04-20T11:34:45.886 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:34:45.886 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:34:45.886 DEBUG:teuthology.orchestra.run.smithi175:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-20T11:34:45.960 INFO:teuthology.orchestra.run.smithi175.stdout:1 2024-04-20T11:34:45.964 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:34:45.964 DEBUG:teuthology.orchestra.run.smithi175:> route 2024-04-20T11:34:46.014 INFO:teuthology.orchestra.run.smithi175.stdout:Kernel IP routing table 2024-04-20T11:34:46.014 INFO:teuthology.orchestra.run.smithi175.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-20T11:34:46.014 INFO:teuthology.orchestra.run.smithi175.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-20T11:34:46.014 INFO:teuthology.orchestra.run.smithi175.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-20T11:34:46.014 INFO:teuthology.orchestra.run.smithi175.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-20T11:34:46.015 DEBUG:teuthology.orchestra.run.smithi175:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T11:34:46.015 DEBUG:teuthology.orchestra.run.smithi175:> set -e 2024-04-20T11:34:46.015 DEBUG:teuthology.orchestra.run.smithi175:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-20T11:34:46.015 DEBUG:teuthology.orchestra.run.smithi175:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-20T11:34:46.015 DEBUG:teuthology.orchestra.run.smithi175:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-20T11:34:46.015 DEBUG:teuthology.orchestra.run.smithi175:> ') 2024-04-20T11:34:46.111 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:34:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T11:34:46.204 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:34:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T11:34:46.213 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:34:46.213 DEBUG:teuthology.orchestra.run.smithi175:> ip netns list 2024-04-20T11:34:46.261 INFO:teuthology.orchestra.run.smithi175.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-20T11:34:46.261 INFO:teuthology.orchestra.run.smithi175.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-20T11:34:46.262 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-20T11:34:46.262 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:34:46.263 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip link set brx.0 up 2024-04-20T11:34:46.341 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-04-20T11:34:46.341 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T11:34:46.341 DEBUG:teuthology.orchestra.run.smithi175:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-20T11:34:46.392 INFO:teuthology.orchestra.run.smithi175.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-20T11:34:46.392 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T11:34:46.393 DEBUG:teuthology.orchestra.run.smithi175:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-20T11:34:46.441 DEBUG:teuthology.orchestra.run.smithi175:> sudo modprobe fuse 2024-04-20T11:34:46.522 DEBUG:teuthology.orchestra.run.smithi175:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T11:34:46.575 INFO:teuthology.orchestra.run.smithi175.stdout:/proc 2024-04-20T11:34:46.575 INFO:teuthology.orchestra.run.smithi175.stdout:/sys 2024-04-20T11:34:46.575 INFO:teuthology.orchestra.run.smithi175.stdout:/dev 2024-04-20T11:34:46.575 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/security 2024-04-20T11:34:46.575 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/shm 2024-04-20T11:34:46.575 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/pts 2024-04-20T11:34:46.575 INFO:teuthology.orchestra.run.smithi175.stdout:/run 2024-04-20T11:34:46.575 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/cgroup 2024-04-20T11:34:46.576 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/pstore 2024-04-20T11:34:46.576 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/bpf 2024-04-20T11:34:46.576 INFO:teuthology.orchestra.run.smithi175.stdout:/ 2024-04-20T11:34:46.576 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/selinux 2024-04-20T11:34:46.576 INFO:teuthology.orchestra.run.smithi175.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T11:34:46.576 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/mqueue 2024-04-20T11:34:46.576 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/debug 2024-04-20T11:34:46.576 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/hugepages 2024-04-20T11:34:46.576 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/tracing 2024-04-20T11:34:46.576 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/fuse/connections 2024-04-20T11:34:46.576 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/config 2024-04-20T11:34:46.576 INFO:teuthology.orchestra.run.smithi175.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T11:34:46.576 INFO:teuthology.orchestra.run.smithi175.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T11:34:46.576 INFO:teuthology.orchestra.run.smithi175.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T11:34:46.577 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T11:34:46.577 INFO:teuthology.orchestra.run.smithi175.stdout:/run/user/1000 2024-04-20T11:34:46.577 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph 2024-04-20T11:34:46.577 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-4 2024-04-20T11:34:46.577 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-5 2024-04-20T11:34:46.577 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-6 2024-04-20T11:34:46.577 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-7 2024-04-20T11:34:46.577 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns 2024-04-20T11:34:46.577 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-20T11:34:46.577 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-20T11:34:46.577 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-20T11:34:46.577 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-20T11:34:46.578 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:34:46.578 DEBUG:teuthology.orchestra.run.smithi175:> ls /sys/fs/fuse/connections 2024-04-20T11:34:46.629 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-20T11:34:46.629 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-04-20T11:34:46.634 DEBUG:teuthology.orchestra.run.smithi175:> sudo modprobe fuse 2024-04-20T11:34:46.699 DEBUG:teuthology.orchestra.run.smithi175:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T11:34:46.739 INFO:teuthology.orchestra.run.smithi175.stdout:/proc 2024-04-20T11:34:46.739 INFO:teuthology.orchestra.run.smithi175.stdout:/sys 2024-04-20T11:34:46.739 INFO:teuthology.orchestra.run.smithi175.stdout:/dev 2024-04-20T11:34:46.740 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/security 2024-04-20T11:34:46.740 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/shm 2024-04-20T11:34:46.740 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/pts 2024-04-20T11:34:46.740 INFO:teuthology.orchestra.run.smithi175.stdout:/run 2024-04-20T11:34:46.740 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/cgroup 2024-04-20T11:34:46.740 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/pstore 2024-04-20T11:34:46.740 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/bpf 2024-04-20T11:34:46.740 INFO:teuthology.orchestra.run.smithi175.stdout:/ 2024-04-20T11:34:46.740 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/selinux 2024-04-20T11:34:46.740 INFO:teuthology.orchestra.run.smithi175.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T11:34:46.740 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/mqueue 2024-04-20T11:34:46.741 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/debug 2024-04-20T11:34:46.741 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/hugepages 2024-04-20T11:34:46.741 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/tracing 2024-04-20T11:34:46.741 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/fuse/connections 2024-04-20T11:34:46.741 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/config 2024-04-20T11:34:46.741 INFO:teuthology.orchestra.run.smithi175.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T11:34:46.741 INFO:teuthology.orchestra.run.smithi175.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T11:34:46.741 INFO:teuthology.orchestra.run.smithi175.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T11:34:46.741 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T11:34:46.741 INFO:teuthology.orchestra.run.smithi175.stdout:/run/user/1000 2024-04-20T11:34:46.741 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph 2024-04-20T11:34:46.741 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-4 2024-04-20T11:34:46.742 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-5 2024-04-20T11:34:46.742 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-6 2024-04-20T11:34:46.742 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-7 2024-04-20T11:34:46.742 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns 2024-04-20T11:34:46.742 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-20T11:34:46.742 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-20T11:34:46.742 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-20T11:34:46.742 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-20T11:34:46.743 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:34:46.744 DEBUG:teuthology.orchestra.run.smithi175:> ls /sys/fs/fuse/connections 2024-04-20T11:34:46.802 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi175.stderr:2024-04-20T11:34:46.802+0000 7f90e3e40480 -1 init, newargv = 0x55950f773310 newargc=15 2024-04-20T11:34:46.802 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi175.stderr:ceph-fuse[34437]: starting ceph client 2024-04-20T11:34:46.811 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi175.stderr:ceph-fuse[34437]: starting fuse 2024-04-20T11:34:46.828 INFO:teuthology.orchestra.run.smithi175.stdout:42 2024-04-20T11:34:46.829 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-04-20T11:34:46.829 DEBUG:teuthology.orchestra.run.smithi175:> sudo stdin-killer -- python3 -c ' 2024-04-20T11:34:46.829 DEBUG:teuthology.orchestra.run.smithi175:> import glob 2024-04-20T11:34:46.829 DEBUG:teuthology.orchestra.run.smithi175:> import re 2024-04-20T11:34:46.829 DEBUG:teuthology.orchestra.run.smithi175:> import os 2024-04-20T11:34:46.829 DEBUG:teuthology.orchestra.run.smithi175:> import subprocess 2024-04-20T11:34:46.829 DEBUG:teuthology.orchestra.run.smithi175:> 2024-04-20T11:34:46.829 DEBUG:teuthology.orchestra.run.smithi175:> def _find_admin_socket(client_name): 2024-04-20T11:34:46.829 DEBUG:teuthology.orchestra.run.smithi175:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-04-20T11:34:46.829 DEBUG:teuthology.orchestra.run.smithi175:> files = glob.glob(asok_path) 2024-04-20T11:34:46.830 DEBUG:teuthology.orchestra.run.smithi175:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-04-20T11:34:46.830 DEBUG:teuthology.orchestra.run.smithi175:> 2024-04-20T11:34:46.830 DEBUG:teuthology.orchestra.run.smithi175:> # Given a non-glob path, it better be there 2024-04-20T11:34:46.830 DEBUG:teuthology.orchestra.run.smithi175:> if "*" not in asok_path: 2024-04-20T11:34:46.830 DEBUG:teuthology.orchestra.run.smithi175:> assert(len(files) == 1) 2024-04-20T11:34:46.830 DEBUG:teuthology.orchestra.run.smithi175:> return files[0] 2024-04-20T11:34:46.830 DEBUG:teuthology.orchestra.run.smithi175:> 2024-04-20T11:34:46.830 DEBUG:teuthology.orchestra.run.smithi175:> for f in files: 2024-04-20T11:34:46.830 DEBUG:teuthology.orchestra.run.smithi175:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-20T11:34:46.830 DEBUG:teuthology.orchestra.run.smithi175:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-20T11:34:46.830 DEBUG:teuthology.orchestra.run.smithi175:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-20T11:34:46.830 DEBUG:teuthology.orchestra.run.smithi175:> contents = proc_f.read() 2024-04-20T11:34:46.830 DEBUG:teuthology.orchestra.run.smithi175:> if mountpoint in contents: 2024-04-20T11:34:46.830 DEBUG:teuthology.orchestra.run.smithi175:> return f 2024-04-20T11:34:46.830 DEBUG:teuthology.orchestra.run.smithi175:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-20T11:34:46.830 DEBUG:teuthology.orchestra.run.smithi175:> 2024-04-20T11:34:46.831 DEBUG:teuthology.orchestra.run.smithi175:> print(_find_admin_socket("client.1")) 2024-04-20T11:34:46.831 DEBUG:teuthology.orchestra.run.smithi175:> ' 2024-04-20T11:34:47.018 INFO:teuthology.orchestra.run.smithi175.stdout:/var/run/ceph/ceph-client.1.34437.asok 2024-04-20T11:34:47.023 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:34:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T11:34:47.035 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.34437.asok 2024-04-20T11:34:47.035 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:34:47.036 DEBUG:teuthology.orchestra.run.smithi175:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.34437.asok status 2024-04-20T11:34:47.258 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-04-20T11:34:47.259 INFO:teuthology.orchestra.run.smithi175.stdout: "metadata": { 2024-04-20T11:34:47.259 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_sha1": "a8c84f5997950fd7d008204839253291d52e6eda", 2024-04-20T11:34:47.259 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", 2024-04-20T11:34:47.259 INFO:teuthology.orchestra.run.smithi175.stdout: "entity_id": "1", 2024-04-20T11:34:47.259 INFO:teuthology.orchestra.run.smithi175.stdout: "hostname": "smithi175", 2024-04-20T11:34:47.259 INFO:teuthology.orchestra.run.smithi175.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-04-20T11:34:47.259 INFO:teuthology.orchestra.run.smithi175.stdout: "pid": "34437", 2024-04-20T11:34:47.259 INFO:teuthology.orchestra.run.smithi175.stdout: "root": "/" 2024-04-20T11:34:47.259 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-20T11:34:47.259 INFO:teuthology.orchestra.run.smithi175.stdout: "dentry_count": 0, 2024-04-20T11:34:47.259 INFO:teuthology.orchestra.run.smithi175.stdout: "dentry_pinned_count": 0, 2024-04-20T11:34:47.259 INFO:teuthology.orchestra.run.smithi175.stdout: "id": 6496, 2024-04-20T11:34:47.260 INFO:teuthology.orchestra.run.smithi175.stdout: "inst": { 2024-04-20T11:34:47.260 INFO:teuthology.orchestra.run.smithi175.stdout: "name": { 2024-04-20T11:34:47.260 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "client", 2024-04-20T11:34:47.260 INFO:teuthology.orchestra.run.smithi175.stdout: "num": 6496 2024-04-20T11:34:47.260 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-20T11:34:47.260 INFO:teuthology.orchestra.run.smithi175.stdout: "addr": { 2024-04-20T11:34:47.260 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "v1", 2024-04-20T11:34:47.260 INFO:teuthology.orchestra.run.smithi175.stdout: "addr": "192.168.0.1:0", 2024-04-20T11:34:47.260 INFO:teuthology.orchestra.run.smithi175.stdout: "nonce": 2898024093 2024-04-20T11:34:47.260 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-04-20T11:34:47.260 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-20T11:34:47.260 INFO:teuthology.orchestra.run.smithi175.stdout: "addr": { 2024-04-20T11:34:47.260 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "v1", 2024-04-20T11:34:47.260 INFO:teuthology.orchestra.run.smithi175.stdout: "addr": "192.168.0.1:0", 2024-04-20T11:34:47.261 INFO:teuthology.orchestra.run.smithi175.stdout: "nonce": 2898024093 2024-04-20T11:34:47.261 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-20T11:34:47.261 INFO:teuthology.orchestra.run.smithi175.stdout: "inst_str": "client.6496 192.168.0.1:0/2898024093", 2024-04-20T11:34:47.261 INFO:teuthology.orchestra.run.smithi175.stdout: "addr_str": "192.168.0.1:0/2898024093", 2024-04-20T11:34:47.261 INFO:teuthology.orchestra.run.smithi175.stdout: "inode_count": 1, 2024-04-20T11:34:47.261 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_epoch": 76, 2024-04-20T11:34:47.261 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_epoch": 118, 2024-04-20T11:34:47.261 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_epoch_barrier": 0, 2024-04-20T11:34:47.261 INFO:teuthology.orchestra.run.smithi175.stdout: "blocklisted": false, 2024-04-20T11:34:47.261 INFO:teuthology.orchestra.run.smithi175.stdout: "fs_name": "cephfs" 2024-04-20T11:34:47.261 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-04-20T11:34:47.273 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:34:47.273 DEBUG:teuthology.orchestra.run.smithi175:> stat --file-system '--printf=%T 2024-04-20T11:34:47.273 DEBUG:teuthology.orchestra.run.smithi175:> ' -- /home/ubuntu/cephtest/mnt.1 2024-04-20T11:34:47.325 INFO:teuthology.orchestra.run.smithi175.stdout:fuseblk 2024-04-20T11:34:47.326 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-04-20T11:34:47.326 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:34:47.326 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-20T11:34:47.408 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-04-20T11:34:47.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:47.626+0000 7f13f2400640 1 Processor -- start 2024-04-20T11:34:47.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:47.626+0000 7f13f2400640 1 -- start start 2024-04-20T11:34:47.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:47.628+0000 7f13f2400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f13ec15a450 0x7f13ec0cd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:47.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:47.628+0000 7f13f2400640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13ec0cdcf0 con 0x7f13ec0ce1e0 2024-04-20T11:34:47.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:47.628+0000 7f13f2400640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13ec15d710 con 0x7f13ec071db0 2024-04-20T11:34:47.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:47.628+0000 7f13f2400640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f13ec0cd320 con 0x7f13ec15a450 2024-04-20T11:34:47.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:47.628+0000 7f13eb400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f13ec15a450 0x7f13ec0cd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:47.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:47.628+0000 7f13e2a00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f13ec0ce1e0 0x7f13ec15cc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:50942/0 (socket says 172.21.15.152:50942) 2024-04-20T11:34:47.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:47.628+0000 7f13e2a00640 1 -- 172.21.15.152:0/1186787450 learned_addr learned my addr 172.21.15.152:0/1186787450 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:34:47.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:47.628+0000 7f13eb400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f13ec15a450 0x7f13ec0cd7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49264/0 (socket says 172.21.15.152:49264) 2024-04-20T11:34:47.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:47.629+0000 7f13eb400640 1 -- 172.21.15.152:0/1186787450 >> v1:172.21.15.175:6789/0 conn(0x7f13ec071db0 legacy=0x7f13ec06ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:47.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:47.629+0000 7f13eb400640 1 -- 172.21.15.152:0/1186787450 >> v1:172.21.15.152:6789/0 conn(0x7f13ec0ce1e0 legacy=0x7f13ec15cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:47.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:47.629+0000 7f13eb400640 1 -- 172.21.15.152:0/1186787450 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13ec15d9f0 con 0x7f13ec15a450 2024-04-20T11:34:47.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:47.629+0000 7f13eaa00640 1 -- 172.21.15.152:0/1186787450 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2017624196 0 0) 0x7f13d4001110 con 0x7f13ec0ce1e0 2024-04-20T11:34:47.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:47.630+0000 7f13eb400640 1 --2- 172.21.15.152:0/1186787450 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f13ec15a450 0x7f13ec0cd7b0 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f13dc0097a0 tx=0x7f13dc02b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c908cc579d3899b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:47.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:47.631+0000 7f13eaa00640 1 -- 172.21.15.152:0/1186787450 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13dc02f810 con 0x7f13ec15a450 2024-04-20T11:34:47.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:47.631+0000 7f13eaa00640 1 -- 172.21.15.152:0/1186787450 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13dc02f9b0 con 0x7f13ec15a450 2024-04-20T11:34:47.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:47.631+0000 7f13eaa00640 1 -- 172.21.15.152:0/1186787450 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13dc02fd80 con 0x7f13ec15a450 2024-04-20T11:34:47.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:47.631+0000 7f13f2400640 1 -- 172.21.15.152:0/1186787450 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f13ec15a450 msgr2=0x7f13ec0cd7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:47.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:47.631+0000 7f13f2400640 1 --2- 172.21.15.152:0/1186787450 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f13ec15a450 0x7f13ec0cd7b0 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f13dc0097a0 tx=0x7f13dc02b9f0 comp rx=0 tx=0).stop 2024-04-20T11:34:47.632 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:47.632+0000 7f13f2400640 1 -- 172.21.15.152:0/1186787450 shutdown_connections 2024-04-20T11:34:47.632 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:47.632+0000 7f13f2400640 1 --2- 172.21.15.152:0/1186787450 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f13ec15a450 0x7f13ec0cd7b0 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:47.632 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:47.632+0000 7f13f2400640 1 -- 172.21.15.152:0/1186787450 >> 172.21.15.152:0/1186787450 conn(0x7f13ec067cf0 msgr2=0x7f13ec06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:47.632 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:47.632+0000 7f13f2400640 1 -- 172.21.15.152:0/1186787450 shutdown_connections 2024-04-20T11:34:47.632 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:47.632+0000 7f13f2400640 1 -- 172.21.15.152:0/1186787450 wait complete. 2024-04-20T11:34:47.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:47.633+0000 7f13f2400640 1 Processor -- start 2024-04-20T11:34:47.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:47.634+0000 7f13f2400640 1 -- start start 2024-04-20T11:34:47.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:47.634+0000 7f13f2400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f13ec071db0 0x7f13ec14cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:47.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:47.635+0000 7f13f2400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f13ec0ce1e0 0x7f13ec14e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:47.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:47.635+0000 7f13f2400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f13ec15a450 0x7f13ec0c97e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:47.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:47.635+0000 7f13f2400640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f13ec06e4e0 con 0x7f13ec0ce1e0 2024-04-20T11:34:47.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:47.635+0000 7f13f2400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f13ec15d380 con 0x7f13ec15a450 2024-04-20T11:34:47.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:47.635+0000 7f13f2400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f13ec06e660 con 0x7f13ec071db0 2024-04-20T11:34:47.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:47.635+0000 7f13e2a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f13ec0ce1e0 0x7f13ec14e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:47.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:47.635+0000 7f13e2a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f13ec0ce1e0 0x7f13ec14e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49274/0 (socket says 172.21.15.152:49274) 2024-04-20T11:34:47.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:47.635+0000 7f13e2a00640 1 -- 172.21.15.152:0/1374303 learned_addr learned my addr 172.21.15.152:0/1374303 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:47.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:47.635+0000 7f13ebe00640 1 --2- 172.21.15.152:0/1374303 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f13ec15a450 0x7f13ec0c97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:47.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:47.635+0000 7f13eb400640 1 --2- 172.21.15.152:0/1374303 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f13ec071db0 0x7f13ec14cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:47.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:47.635+0000 7f13e2a00640 1 -- 172.21.15.152:0/1374303 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f13ec071db0 msgr2=0x7f13ec14cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:47.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:47.635+0000 7f13e2a00640 1 --2- 172.21.15.152:0/1374303 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f13ec071db0 0x7f13ec14cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:47.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:47.635+0000 7f13e2a00640 1 -- 172.21.15.152:0/1374303 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f13ec15a450 msgr2=0x7f13ec0c97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:47.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:47.635+0000 7f13e2a00640 1 --2- 172.21.15.152:0/1374303 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f13ec15a450 0x7f13ec0c97e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:47.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:47.635+0000 7f13e2a00640 1 -- 172.21.15.152:0/1374303 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13ec14d920 con 0x7f13ec0ce1e0 2024-04-20T11:34:47.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:47.635+0000 7f13ebe00640 1 --2- 172.21.15.152:0/1374303 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f13ec15a450 0x7f13ec0c97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:34:47.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:47.636+0000 7f13e2a00640 1 --2- 172.21.15.152:0/1374303 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f13ec0ce1e0 0x7f13ec14e910 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f13d400c780 tx=0x7f13d400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:47.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:47.636+0000 7f13e3e00640 1 -- 172.21.15.152:0/1374303 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d4018070 con 0x7f13ec0ce1e0 2024-04-20T11:34:47.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:47.636+0000 7f13e3e00640 1 -- 172.21.15.152:0/1374303 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13d40072d0 con 0x7f13ec0ce1e0 2024-04-20T11:34:47.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:47.636+0000 7f13e3e00640 1 -- 172.21.15.152:0/1374303 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d40047e0 con 0x7f13ec0ce1e0 2024-04-20T11:34:47.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:47.636+0000 7f13f2400640 1 -- 172.21.15.152:0/1374303 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13ec14dc00 con 0x7f13ec0ce1e0 2024-04-20T11:34:47.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:47.636+0000 7f13f2400640 1 -- 172.21.15.152:0/1374303 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13ec14e1c0 con 0x7f13ec0ce1e0 2024-04-20T11:34:47.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:47.637+0000 7f13f2400640 1 -- 172.21.15.152:0/1374303 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13ec14e390 con 0x7f13ec0ce1e0 2024-04-20T11:34:47.638 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:47.638+0000 7f13e3e00640 1 -- 172.21.15.152:0/1374303 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 19) ==== 91397+0+0 (secure 0 0 0) 0x7f13d4004980 con 0x7f13ec0ce1e0 2024-04-20T11:34:47.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:47.639+0000 7f13e3e00640 1 --2- 172.21.15.152:0/1374303 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f13c806cce0 0x7f13c806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:47.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:47.639+0000 7f13e3e00640 1 -- 172.21.15.152:0/1374303 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5709+0+0 (secure 0 0 0) 0x7f13d408b610 con 0x7f13ec0ce1e0 2024-04-20T11:34:47.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:47.640+0000 7f13eb400640 1 --2- 172.21.15.152:0/1374303 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f13c806cce0 0x7f13c806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:47.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:47.641+0000 7f13eb400640 1 --2- 172.21.15.152:0/1374303 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f13c806cce0 0x7f13c806f1a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f13dc036000 tx=0x7f13dc039040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:47.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:47.644+0000 7f13e3e00640 1 -- 172.21.15.152:0/1374303 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f13ec0cdcf0 con 0x7f13ec0ce1e0 2024-04-20T11:34:47.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:47.881+0000 7f13f2400640 1 -- 172.21.15.152:0/1374303 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f13ec068a60 con 0x7f13ec0ce1e0 2024-04-20T11:34:48.114 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.114+0000 7f13e3e00640 1 -- 172.21.15.152:0/1374303 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v77) ==== 136+0+0 (secure 0 0 0) 0x7f13d4004c70 con 0x7f13ec0ce1e0 2024-04-20T11:34:48.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.117+0000 7f13f2400640 1 -- 172.21.15.152:0/1374303 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f13c806cce0 msgr2=0x7f13c806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:48.118 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.118+0000 7f13f2400640 1 --2- 172.21.15.152:0/1374303 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f13c806cce0 0x7f13c806f1a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f13dc036000 tx=0x7f13dc039040 comp rx=0 tx=0).stop 2024-04-20T11:34:48.118 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.118+0000 7f13f2400640 1 -- 172.21.15.152:0/1374303 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f13ec0ce1e0 msgr2=0x7f13ec14e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:48.118 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.118+0000 7f13f2400640 1 --2- 172.21.15.152:0/1374303 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f13ec0ce1e0 0x7f13ec14e910 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f13d400c780 tx=0x7f13d400cc40 comp rx=0 tx=0).stop 2024-04-20T11:34:48.118 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.118+0000 7f13f2400640 1 -- 172.21.15.152:0/1374303 shutdown_connections 2024-04-20T11:34:48.118 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.118+0000 7f13f2400640 1 --2- 172.21.15.152:0/1374303 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f13c806cce0 0x7f13c806f1a0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:48.118 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.118+0000 7f13f2400640 1 --2- 172.21.15.152:0/1374303 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f13ec15a450 0x7f13ec0c97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:48.118 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.118+0000 7f13f2400640 1 --2- 172.21.15.152:0/1374303 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f13ec0ce1e0 0x7f13ec14e910 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:48.118 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.118+0000 7f13f2400640 1 --2- 172.21.15.152:0/1374303 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f13ec071db0 0x7f13ec14cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:48.118 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.118+0000 7f13f2400640 1 -- 172.21.15.152:0/1374303 >> 172.21.15.152:0/1374303 conn(0x7f13ec067cf0 msgr2=0x7f13ec15eb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:48.118 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.118+0000 7f13f2400640 1 -- 172.21.15.152:0/1374303 shutdown_connections 2024-04-20T11:34:48.118 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.118+0000 7f13f2400640 1 -- 172.21.15.152:0/1374303 wait complete. 2024-04-20T11:34:48.140 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-04-20T11:34:48.330 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.330+0000 7f15a4000640 1 Processor -- start 2024-04-20T11:34:48.330 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.331+0000 7f15a4000640 1 -- start start 2024-04-20T11:34:48.331 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.332+0000 7f15a4000640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f159c06ded0 0x7f159c0d18e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:48.332 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.332+0000 7f15a4000640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f159c06ce30 con 0x7f159c09b150 2024-04-20T11:34:48.332 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.332+0000 7f15a4000640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f159c06d110 con 0x7f159c15e900 2024-04-20T11:34:48.332 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.332+0000 7f15a4000640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f159c06da40 con 0x7f159c06ded0 2024-04-20T11:34:48.332 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.332+0000 7f15a0c00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f159c09b150 0x7f159c166880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:50944/0 (socket says 172.21.15.152:50944) 2024-04-20T11:34:48.332 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.332+0000 7f15a0c00640 1 -- 172.21.15.152:0/2593020488 learned_addr learned my addr 172.21.15.152:0/2593020488 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:34:48.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.333+0000 7f15a2000640 1 --2- 172.21.15.152:0/2593020488 >> v2:172.21.15.175:3300/0 conn(0x7f159c06ded0 0x7f159c0d18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:48.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.333+0000 7f159be00640 1 -- 172.21.15.152:0/2593020488 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1494268925 0 0) 0x7f159c06ce30 con 0x7f159c09b150 2024-04-20T11:34:48.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.333+0000 7f159be00640 1 -- 172.21.15.152:0/2593020488 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1580003520 con 0x7f159c09b150 2024-04-20T11:34:48.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.333+0000 7f159be00640 1 -- 172.21.15.152:0/2593020488 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3693199518 0 0) 0x7f1580003520 con 0x7f159c09b150 2024-04-20T11:34:48.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.333+0000 7f159be00640 1 -- 172.21.15.152:0/2593020488 >> v2:172.21.15.175:3300/0 conn(0x7f159c06ded0 msgr2=0x7f159c0d18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:48.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.333+0000 7f159be00640 1 --2- 172.21.15.152:0/2593020488 >> v2:172.21.15.175:3300/0 conn(0x7f159c06ded0 0x7f159c0d18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:48.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.333+0000 7f159be00640 1 -- 172.21.15.152:0/2593020488 >> v1:172.21.15.175:6789/0 conn(0x7f159c15e900 legacy=0x7f159c09aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:48.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.333+0000 7f159be00640 1 -- 172.21.15.152:0/2593020488 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f159c06d3f0 con 0x7f159c09b150 2024-04-20T11:34:48.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.333+0000 7f159be00640 1 -- 172.21.15.152:0/2593020488 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f1588001670 con 0x7f159c09b150 2024-04-20T11:34:48.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.333+0000 7f159be00640 1 -- 172.21.15.152:0/2593020488 >> v1:172.21.15.152:6789/0 conn(0x7f159c09b150 legacy=0x7f159c166880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:48.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.333+0000 7f159be00640 1 --2- 172.21.15.152:0/2593020488 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f15800039b0 0x7f1580003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:48.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.333+0000 7f159be00640 1 --2- 172.21.15.152:0/2593020488 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1580004800 0x7f1580004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:48.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.333+0000 7f159be00640 1 --2- 172.21.15.152:0/2593020488 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15800051c0 0x7f158000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:48.335 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.333+0000 7f159be00640 1 -- 172.21.15.152:0/2593020488 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f159c06da40 con 0x7f1580004800 2024-04-20T11:34:48.335 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.333+0000 7f159be00640 1 -- 172.21.15.152:0/2593020488 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f158000c060 con 0x7f15800051c0 2024-04-20T11:34:48.335 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.333+0000 7f159be00640 1 -- 172.21.15.152:0/2593020488 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f158000c260 con 0x7f15800039b0 2024-04-20T11:34:48.335 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.334+0000 7f15a0c00640 1 --2- 172.21.15.152:0/2593020488 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1580004800 0x7f1580004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:48.335 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.334+0000 7f15a0c00640 1 -- 172.21.15.152:0/2593020488 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f15800039b0 msgr2=0x7f1580003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:48.335 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.334+0000 7f15a0c00640 1 --2- 172.21.15.152:0/2593020488 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f15800039b0 0x7f1580003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:48.335 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.334+0000 7f15a0c00640 1 -- 172.21.15.152:0/2593020488 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15800051c0 msgr2=0x7f158000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:48.335 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.334+0000 7f15a0c00640 1 --2- 172.21.15.152:0/2593020488 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15800051c0 0x7f158000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:48.335 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.334+0000 7f15a0c00640 1 -- 172.21.15.152:0/2593020488 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1580003520 con 0x7f1580004800 2024-04-20T11:34:48.335 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.334+0000 7f15a2000640 1 -- 172.21.15.152:0/2593020488 reap_dead start 2024-04-20T11:34:48.336 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.335+0000 7f15a0c00640 1 --2- 172.21.15.152:0/2593020488 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1580004800 0x7f1580004c80 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f158800c210 tx=0x7f1580005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1e7697797e00ad7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:48.336 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.335+0000 7f159be00640 1 -- 172.21.15.152:0/2593020488 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1588007d30 con 0x7f1580004800 2024-04-20T11:34:48.336 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.335+0000 7f159be00640 1 -- 172.21.15.152:0/2593020488 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1588030070 con 0x7f1580004800 2024-04-20T11:34:48.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.335+0000 7f159be00640 1 -- 172.21.15.152:0/2593020488 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f158802cda0 con 0x7f1580004800 2024-04-20T11:34:48.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.336+0000 7f15a4000640 1 -- 172.21.15.152:0/2593020488 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1580004800 msgr2=0x7f1580004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:48.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.336+0000 7f15a4000640 1 --2- 172.21.15.152:0/2593020488 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1580004800 0x7f1580004c80 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f158800c210 tx=0x7f1580005830 comp rx=0 tx=0).stop 2024-04-20T11:34:48.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.336+0000 7f15a4000640 1 -- 172.21.15.152:0/2593020488 shutdown_connections 2024-04-20T11:34:48.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.336+0000 7f15a4000640 1 --2- 172.21.15.152:0/2593020488 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1580004800 0x7f1580004c80 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:48.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.336+0000 7f15a4000640 1 -- 172.21.15.152:0/2593020488 >> 172.21.15.152:0/2593020488 conn(0x7f159c0b40c0 msgr2=0x7f159c0b44d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:48.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.336+0000 7f15a4000640 1 -- 172.21.15.152:0/2593020488 shutdown_connections 2024-04-20T11:34:48.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.336+0000 7f15a4000640 1 -- 172.21.15.152:0/2593020488 wait complete. 2024-04-20T11:34:48.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.338+0000 7f15a4000640 1 Processor -- start 2024-04-20T11:34:48.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.338+0000 7f15a4000640 1 -- start start 2024-04-20T11:34:48.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.339+0000 7f15a4000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1580004800 0x7f159c15f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:48.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.339+0000 7f15a4000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f159c1f5100 0x7f159c161020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:48.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.339+0000 7f15a4000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f159c161560 0x7f159c1f14b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:48.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.339+0000 7f15a4000640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f159c06cc10 con 0x7f159c161560 2024-04-20T11:34:48.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.339+0000 7f15a4000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f159c06ca90 con 0x7f1580004800 2024-04-20T11:34:48.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.339+0000 7f15a4000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f159c0d0bc0 con 0x7f159c1f5100 2024-04-20T11:34:48.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.339+0000 7f15a2000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f159c161560 0x7f159c1f14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:48.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.339+0000 7f15a2000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f159c161560 0x7f159c1f14b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49300/0 (socket says 172.21.15.152:49300) 2024-04-20T11:34:48.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.339+0000 7f15a2000640 1 -- 172.21.15.152:0/2060472897 learned_addr learned my addr 172.21.15.152:0/2060472897 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:48.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.339+0000 7f15a1600640 1 --2- 172.21.15.152:0/2060472897 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1580004800 0x7f159c15f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:48.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.339+0000 7f15a0c00640 1 --2- 172.21.15.152:0/2060472897 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f159c1f5100 0x7f159c161020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:48.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.340+0000 7f15a2000640 1 -- 172.21.15.152:0/2060472897 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f159c1f5100 msgr2=0x7f159c161020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:48.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.340+0000 7f15a2000640 1 --2- 172.21.15.152:0/2060472897 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f159c1f5100 0x7f159c161020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:48.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.340+0000 7f15a2000640 1 -- 172.21.15.152:0/2060472897 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1580004800 msgr2=0x7f159c15f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:48.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.340+0000 7f15a2000640 1 --2- 172.21.15.152:0/2060472897 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1580004800 0x7f159c15f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:48.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.340+0000 7f15a2000640 1 -- 172.21.15.152:0/2060472897 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f159c161a00 con 0x7f159c161560 2024-04-20T11:34:48.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.340+0000 7f15a0c00640 1 --2- 172.21.15.152:0/2060472897 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f159c1f5100 0x7f159c161020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:34:48.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.340+0000 7f15a2000640 1 --2- 172.21.15.152:0/2060472897 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f159c161560 0x7f159c1f14b0 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f159c06fa80 tx=0x7f159400eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:48.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.340+0000 7f1599600640 1 -- 172.21.15.152:0/2060472897 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1594018020 con 0x7f159c161560 2024-04-20T11:34:48.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.340+0000 7f1599600640 1 -- 172.21.15.152:0/2060472897 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f159400bab0 con 0x7f159c161560 2024-04-20T11:34:48.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.340+0000 7f1599600640 1 -- 172.21.15.152:0/2060472897 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1594014400 con 0x7f159c161560 2024-04-20T11:34:48.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.340+0000 7f15a4000640 1 -- 172.21.15.152:0/2060472897 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f159c161ce0 con 0x7f159c161560 2024-04-20T11:34:48.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.340+0000 7f15a4000640 1 -- 172.21.15.152:0/2060472897 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f159c0b6610 con 0x7f159c161560 2024-04-20T11:34:48.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.342+0000 7f15a4000640 1 -- 172.21.15.152:0/2060472897 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f159c069100 con 0x7f159c161560 2024-04-20T11:34:48.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.342+0000 7f1599600640 1 -- 172.21.15.152:0/2060472897 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 19) ==== 91397+0+0 (secure 0 0 0) 0x7f1594005d10 con 0x7f159c161560 2024-04-20T11:34:48.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.343+0000 7f1599600640 1 --2- 172.21.15.152:0/2060472897 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f157c06cd10 0x7f157c06f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:48.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.343+0000 7f1599600640 1 -- 172.21.15.152:0/2060472897 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5709+0+0 (secure 0 0 0) 0x7f159408ca70 con 0x7f159c161560 2024-04-20T11:34:48.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.344+0000 7f15a1600640 1 --2- 172.21.15.152:0/2060472897 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f157c06cd10 0x7f157c06f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:48.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.345+0000 7f15a1600640 1 --2- 172.21.15.152:0/2060472897 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f157c06cd10 0x7f157c06f1d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f158c007fd0 tx=0x7f158c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:48.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.347+0000 7f1599600640 1 -- 172.21.15.152:0/2060472897 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f1594022030 con 0x7f159c161560 2024-04-20T11:34:48.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.576+0000 7f15a4000640 1 -- 172.21.15.152:0/2060472897 --> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f159c0b4760 con 0x7f157c06cd10 2024-04-20T11:34:48.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.580+0000 7f1599600640 1 -- 172.21.15.152:0/2060472897 <== mgr.4109 v2:172.21.15.175:6836/4021250441 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f159c0b4760 con 0x7f157c06cd10 2024-04-20T11:34:48.580 INFO:teuthology.orchestra.run.smithi152.stdout:[ 2024-04-20T11:34:48.580 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:34:48.580 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "cephfs" 2024-04-20T11:34:48.580 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:34:48.580 INFO:teuthology.orchestra.run.smithi152.stdout:] 2024-04-20T11:34:48.583 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.583+0000 7f15a4000640 1 -- 172.21.15.152:0/2060472897 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f157c06cd10 msgr2=0x7f157c06f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:48.583 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.583+0000 7f15a4000640 1 --2- 172.21.15.152:0/2060472897 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f157c06cd10 0x7f157c06f1d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f158c007fd0 tx=0x7f158c009040 comp rx=0 tx=0).stop 2024-04-20T11:34:48.583 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.583+0000 7f15a4000640 1 -- 172.21.15.152:0/2060472897 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f159c161560 msgr2=0x7f159c1f14b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:48.583 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.583+0000 7f15a4000640 1 --2- 172.21.15.152:0/2060472897 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f159c161560 0x7f159c1f14b0 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f159c06fa80 tx=0x7f159400eea0 comp rx=0 tx=0).stop 2024-04-20T11:34:48.583 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.583+0000 7f15a4000640 1 -- 172.21.15.152:0/2060472897 shutdown_connections 2024-04-20T11:34:48.583 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.583+0000 7f15a4000640 1 --2- 172.21.15.152:0/2060472897 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f157c06cd10 0x7f157c06f1d0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:48.583 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.583+0000 7f15a4000640 1 --2- 172.21.15.152:0/2060472897 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f159c161560 0x7f159c1f14b0 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:48.583 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.583+0000 7f15a4000640 1 --2- 172.21.15.152:0/2060472897 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f159c1f5100 0x7f159c161020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:48.583 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.583+0000 7f15a4000640 1 --2- 172.21.15.152:0/2060472897 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1580004800 0x7f159c15f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:48.583 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.583+0000 7f15a4000640 1 -- 172.21.15.152:0/2060472897 >> 172.21.15.152:0/2060472897 conn(0x7f159c0b40c0 msgr2=0x7f159c09aa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:48.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.583+0000 7f15a4000640 1 -- 172.21.15.152:0/2060472897 shutdown_connections 2024-04-20T11:34:48.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.583+0000 7f15a4000640 1 -- 172.21.15.152:0/2060472897 wait complete. 2024-04-20T11:34:48.602 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-20T11:34:48.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.807+0000 7fcecfe00640 1 Processor -- start 2024-04-20T11:34:48.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.807+0000 7fcecfe00640 1 -- start start 2024-04-20T11:34:48.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.808+0000 7fcecfe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fced006ee00 0x7fced006f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:48.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.808+0000 7fcecfe00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fced015fbf0 0x7fced015c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:48.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.808+0000 7fcecfe00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fced00cf110 0x7fced00b3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:48.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.808+0000 7fcecfe00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fced00d00d0 con 0x7fced00cf110 2024-04-20T11:34:48.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.808+0000 7fcecfe00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fced006f720 con 0x7fced015fbf0 2024-04-20T11:34:48.809 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.808+0000 7fcecfe00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fced00cfb10 con 0x7fced006ee00 2024-04-20T11:34:48.809 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.809+0000 7fcecf400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fced00cf110 0x7fced00b3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:48.809 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.809+0000 7fcecf400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fced00cf110 0x7fced00b3ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49314/0 (socket says 172.21.15.152:49314) 2024-04-20T11:34:48.809 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.809+0000 7fcecf400640 1 -- 172.21.15.152:0/2302602179 learned_addr learned my addr 172.21.15.152:0/2302602179 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:48.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.809+0000 7fcece000640 1 --2- 172.21.15.152:0/2302602179 >> v2:172.21.15.175:3300/0 conn(0x7fced015fbf0 0x7fced015c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:48.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.809+0000 7fcecf400640 1 -- 172.21.15.152:0/2302602179 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fced006ee00 msgr2=0x7fced006f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:48.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.809+0000 7fcecea00640 1 --2- 172.21.15.152:0/2302602179 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fced006ee00 0x7fced006f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:48.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.809+0000 7fcecf400640 1 --2- 172.21.15.152:0/2302602179 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fced006ee00 0x7fced006f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:48.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.809+0000 7fcecf400640 1 -- 172.21.15.152:0/2302602179 >> v2:172.21.15.175:3300/0 conn(0x7fced015fbf0 msgr2=0x7fced015c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:48.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.809+0000 7fcecf400640 1 --2- 172.21.15.152:0/2302602179 >> v2:172.21.15.175:3300/0 conn(0x7fced015fbf0 0x7fced015c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:48.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.809+0000 7fcecf400640 1 -- 172.21.15.152:0/2302602179 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fced00cfdf0 con 0x7fced00cf110 2024-04-20T11:34:48.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.809+0000 7fcece000640 1 --2- 172.21.15.152:0/2302602179 >> v2:172.21.15.175:3300/0 conn(0x7fced015fbf0 0x7fced015c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:34:48.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.809+0000 7fcecea00640 1 --2- 172.21.15.152:0/2302602179 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fced006ee00 0x7fced006f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:34:48.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.810+0000 7fcecf400640 1 --2- 172.21.15.152:0/2302602179 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fced00cf110 0x7fced00b3ba0 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7fced0073170 tx=0x7fcec802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79a9b32681bc41ab server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:48.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.811+0000 7fcecd600640 1 -- 172.21.15.152:0/2302602179 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec802ca80 con 0x7fced00cf110 2024-04-20T11:34:48.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.811+0000 7fcecd600640 1 -- 172.21.15.152:0/2302602179 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcec802cc20 con 0x7fced00cf110 2024-04-20T11:34:48.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.811+0000 7fcecd600640 1 -- 172.21.15.152:0/2302602179 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec80343f0 con 0x7fced00cf110 2024-04-20T11:34:48.812 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.811+0000 7fcecfe00640 1 -- 172.21.15.152:0/2302602179 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fced00cf110 msgr2=0x7fced00b3ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:48.812 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.811+0000 7fcecfe00640 1 --2- 172.21.15.152:0/2302602179 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fced00cf110 0x7fced00b3ba0 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7fced0073170 tx=0x7fcec802bee0 comp rx=0 tx=0).stop 2024-04-20T11:34:48.812 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.811+0000 7fcecfe00640 1 -- 172.21.15.152:0/2302602179 shutdown_connections 2024-04-20T11:34:48.812 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.811+0000 7fcecfe00640 1 --2- 172.21.15.152:0/2302602179 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fced00cf110 0x7fced00b3ba0 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:48.812 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.811+0000 7fcecfe00640 1 --2- 172.21.15.152:0/2302602179 >> v2:172.21.15.175:3300/0 conn(0x7fced015fbf0 0x7fced015c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:48.812 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.811+0000 7fcecfe00640 1 --2- 172.21.15.152:0/2302602179 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fced006ee00 0x7fced006f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:48.812 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.811+0000 7fcecfe00640 1 -- 172.21.15.152:0/2302602179 >> 172.21.15.152:0/2302602179 conn(0x7fced0067ae0 msgr2=0x7fced006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:48.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.812+0000 7fcecfe00640 1 -- 172.21.15.152:0/2302602179 shutdown_connections 2024-04-20T11:34:48.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.812+0000 7fcecfe00640 1 -- 172.21.15.152:0/2302602179 wait complete. 2024-04-20T11:34:48.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.813+0000 7fcecfe00640 1 Processor -- start 2024-04-20T11:34:48.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.813+0000 7fcecfe00640 1 -- start start 2024-04-20T11:34:48.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.814+0000 7fcecfe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fced006ee00 0x7fced013a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:48.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.814+0000 7fcecfe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fced00cf110 0x7fced013bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:48.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.814+0000 7fcecfe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fced015fbf0 0x7fced0141010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:48.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.814+0000 7fcecfe00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fced015c8d0 con 0x7fced015fbf0 2024-04-20T11:34:48.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.814+0000 7fcecfe00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fced015be70 con 0x7fced00cf110 2024-04-20T11:34:48.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.814+0000 7fcecfe00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fced0144970 con 0x7fced006ee00 2024-04-20T11:34:48.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.815+0000 7fcecf400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fced015fbf0 0x7fced0141010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:48.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.815+0000 7fcecf400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fced015fbf0 0x7fced0141010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49330/0 (socket says 172.21.15.152:49330) 2024-04-20T11:34:48.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.815+0000 7fcecf400640 1 -- 172.21.15.152:0/3853073909 learned_addr learned my addr 172.21.15.152:0/3853073909 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:48.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.815+0000 7fcecea00640 1 --2- 172.21.15.152:0/3853073909 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fced006ee00 0x7fced013a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:48.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.815+0000 7fcece000640 1 --2- 172.21.15.152:0/3853073909 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fced00cf110 0x7fced013bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:48.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.815+0000 7fcecf400640 1 -- 172.21.15.152:0/3853073909 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fced006ee00 msgr2=0x7fced013a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:48.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.815+0000 7fcecf400640 1 --2- 172.21.15.152:0/3853073909 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fced006ee00 0x7fced013a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:48.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.815+0000 7fcecf400640 1 -- 172.21.15.152:0/3853073909 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fced00cf110 msgr2=0x7fced013bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:48.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.815+0000 7fcecf400640 1 --2- 172.21.15.152:0/3853073909 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fced00cf110 0x7fced013bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:48.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.815+0000 7fcecf400640 1 -- 172.21.15.152:0/3853073909 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fced013ad50 con 0x7fced015fbf0 2024-04-20T11:34:48.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.815+0000 7fcecf400640 1 --2- 172.21.15.152:0/3853073909 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fced015fbf0 0x7fced0141010 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7fcec8009f40 tx=0x7fcec8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:48.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.816+0000 7fcec6a00640 1 -- 172.21.15.152:0/3853073909 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec8043020 con 0x7fced015fbf0 2024-04-20T11:34:48.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.816+0000 7fcec6a00640 1 -- 172.21.15.152:0/3853073909 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcec803e450 con 0x7fced015fbf0 2024-04-20T11:34:48.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.816+0000 7fcecfe00640 1 -- 172.21.15.152:0/3853073909 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fced013b030 con 0x7fced015fbf0 2024-04-20T11:34:48.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.816+0000 7fcec6a00640 1 -- 172.21.15.152:0/3853073909 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec803e820 con 0x7fced015fbf0 2024-04-20T11:34:48.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.816+0000 7fcecfe00640 1 -- 172.21.15.152:0/3853073909 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fced013b5f0 con 0x7fced015fbf0 2024-04-20T11:34:48.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.817+0000 7fcecfe00640 1 -- 172.21.15.152:0/3853073909 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fced0075e60 con 0x7fced015fbf0 2024-04-20T11:34:48.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.818+0000 7fcec6a00640 1 -- 172.21.15.152:0/3853073909 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 19) ==== 91397+0+0 (secure 0 0 0) 0x7fcec8046020 con 0x7fced015fbf0 2024-04-20T11:34:48.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.819+0000 7fcec6a00640 1 --2- 172.21.15.152:0/3853073909 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fceb006cce0 0x7fceb006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:48.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.819+0000 7fcec6a00640 1 -- 172.21.15.152:0/3853073909 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5709+0+0 (secure 0 0 0) 0x7fcec80aebe0 con 0x7fced015fbf0 2024-04-20T11:34:48.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.820+0000 7fcecea00640 1 --2- 172.21.15.152:0/3853073909 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fceb006cce0 0x7fceb006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:48.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.821+0000 7fcecea00640 1 --2- 172.21.15.152:0/3853073909 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fceb006cce0 0x7fceb006f1a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fcec000a980 tx=0x7fcec0008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:48.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:48.823+0000 7fcec6a00640 1 -- 172.21.15.152:0/3853073909 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fcec804d030 con 0x7fced015fbf0 2024-04-20T11:34:49.021 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:49.021+0000 7fcecfe00640 1 -- 172.21.15.152:0/3853073909 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fced00b3980 con 0x7fced015fbf0 2024-04-20T11:34:49.022 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:49.022+0000 7fcec6a00640 1 -- 172.21.15.152:0/3853073909 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) ==== 122+0+0 (secure 0 0 0) 0x7fcec807ca80 con 0x7fced015fbf0 2024-04-20T11:34:49.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:49.024+0000 7fcecfe00640 1 -- 172.21.15.152:0/3853073909 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fceb006cce0 msgr2=0x7fceb006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:49.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:49.024+0000 7fcecfe00640 1 --2- 172.21.15.152:0/3853073909 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fceb006cce0 0x7fceb006f1a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fcec000a980 tx=0x7fcec0008040 comp rx=0 tx=0).stop 2024-04-20T11:34:49.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:49.024+0000 7fcecfe00640 1 -- 172.21.15.152:0/3853073909 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fced015fbf0 msgr2=0x7fced0141010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:49.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:49.024+0000 7fcecfe00640 1 --2- 172.21.15.152:0/3853073909 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fced015fbf0 0x7fced0141010 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7fcec8009f40 tx=0x7fcec8004a40 comp rx=0 tx=0).stop 2024-04-20T11:34:49.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:49.025+0000 7fcecfe00640 1 -- 172.21.15.152:0/3853073909 shutdown_connections 2024-04-20T11:34:49.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:49.025+0000 7fcecfe00640 1 --2- 172.21.15.152:0/3853073909 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fceb006cce0 0x7fceb006f1a0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:49.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:49.025+0000 7fcecfe00640 1 --2- 172.21.15.152:0/3853073909 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fced015fbf0 0x7fced0141010 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:49.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:49.025+0000 7fcecfe00640 1 --2- 172.21.15.152:0/3853073909 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fced00cf110 0x7fced013bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:49.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:49.025+0000 7fcecfe00640 1 --2- 172.21.15.152:0/3853073909 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fced006ee00 0x7fced013a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:49.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:49.025+0000 7fcecfe00640 1 -- 172.21.15.152:0/3853073909 >> 172.21.15.152:0/3853073909 conn(0x7fced0067ae0 msgr2=0x7fced006e670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:49.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:49.025+0000 7fcecfe00640 1 -- 172.21.15.152:0/3853073909 shutdown_connections 2024-04-20T11:34:49.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:49.025+0000 7fcecfe00640 1 -- 172.21.15.152:0/3853073909 wait complete. 2024-04-20T11:34:49.043 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:34:49.044 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/_nogroup/subvol36640/c31a948e-91ce-43b1-bdd3-e504a43c9761) 2024-04-20T11:34:49.150 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:34:49.150 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/_nogroup/subvol36640/c31a948e-91ce-43b1-bdd3-e504a43c9761/.snap/fake) 2024-04-20T11:34:49.259 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:34:49.259 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2024-04-20T11:34:49.313 INFO:teuthology.orchestra.run.smithi152.stdout:cephfs_data 2024-04-20T11:34:49.314 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:34:49.314 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data volumes/_nogroup/subvol36640/c31a948e-91ce-43b1-bdd3-e504a43c9761) 2024-04-20T11:34:49.404 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-20T11:34:49.404 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/home/ubuntu/cephtest/mnt.0/volumes/_nogroup/subvol36640/.meta 2024-04-20T11:34:49.494 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:34:49.494 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/subvol_grp47320/subvol43081/076b3aa2-4a8c-45ef-84df-615379a10ac6) 2024-04-20T11:34:49.642 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:34:49.642 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/subvol_grp47320/subvol43081/076b3aa2-4a8c-45ef-84df-615379a10ac6/.snap/fake) 2024-04-20T11:34:49.749 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:34:49.749 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2024-04-20T11:34:49.806 INFO:teuthology.orchestra.run.smithi152.stdout:cephfs_data 2024-04-20T11:34:49.806 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:34:49.806 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data volumes/subvol_grp47320/subvol43081/076b3aa2-4a8c-45ef-84df-615379a10ac6) 2024-04-20T11:34:49.898 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-20T11:34:49.898 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/home/ubuntu/cephtest/mnt.0/volumes/subvol_grp47320/subvol43081/.meta 2024-04-20T11:34:49.976 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:34:49.977 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/_nogroup/subvol90142/e2f77d43-a542-4f2a-beaa-47d82b3daced) 2024-04-20T11:34:50.114 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:34:50.114 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2024-04-20T11:34:50.171 INFO:teuthology.orchestra.run.smithi152.stdout:cephfs_data 2024-04-20T11:34:50.172 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:34:50.172 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data volumes/_nogroup/subvol90142/e2f77d43-a542-4f2a-beaa-47d82b3daced) 2024-04-20T11:34:50.266 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-20T11:34:50.266 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/home/ubuntu/cephtest/mnt.0/volumes/_nogroup/subvol90142/.meta 2024-04-20T11:34:50.353 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol36640 2024-04-20T11:34:50.608 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:50.608+0000 7f8625000640 1 Processor -- start 2024-04-20T11:34:50.608 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:50.609+0000 7f8625000640 1 -- start start 2024-04-20T11:34:50.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:50.610+0000 7f8625000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8620160220 0x7f86201606b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:50.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:50.610+0000 7f8625000640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f862006fb40 con 0x7f8620071cd0 2024-04-20T11:34:50.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:50.610+0000 7f8625000640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8620160f90 con 0x7f8620141450 2024-04-20T11:34:50.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:50.610+0000 7f8625000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f862013f8e0 con 0x7f8620160220 2024-04-20T11:34:50.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:50.611+0000 7f861f400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f8620071cd0 0x7f86200720b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:50960/0 (socket says 172.21.15.152:50960) 2024-04-20T11:34:50.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:50.611+0000 7f861f400640 1 -- 172.21.15.152:0/1732332452 learned_addr learned my addr 172.21.15.152:0/1732332452 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:34:50.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:50.611+0000 7f861e000640 1 -- 172.21.15.152:0/1732332452 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1048863259 0 0) 0x7f862006fb40 con 0x7f8620071cd0 2024-04-20T11:34:50.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:50.611+0000 7f861ea00640 1 --2- 172.21.15.152:0/1732332452 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8620160220 0x7f86201606b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:50.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:50.611+0000 7f861e000640 1 -- 172.21.15.152:0/1732332452 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8600003520 con 0x7f8620071cd0 2024-04-20T11:34:50.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:50.612+0000 7f861e000640 1 -- 172.21.15.152:0/1732332452 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4043657695 0 0) 0x7f8600003520 con 0x7f8620071cd0 2024-04-20T11:34:50.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:50.612+0000 7f861e000640 1 -- 172.21.15.152:0/1732332452 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8620160220 msgr2=0x7f86201606b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:50.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:50.612+0000 7f861e000640 1 --2- 172.21.15.152:0/1732332452 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8620160220 0x7f86201606b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:50.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:50.612+0000 7f861e000640 1 -- 172.21.15.152:0/1732332452 >> v1:172.21.15.175:6789/0 conn(0x7f8620141450 legacy=0x7f86201418e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:50.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:50.612+0000 7f861e000640 1 -- 172.21.15.152:0/1732332452 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f862006dd90 con 0x7f8620071cd0 2024-04-20T11:34:50.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:50.612+0000 7f861e000640 1 -- 172.21.15.152:0/1732332452 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f8610001670 con 0x7f8620071cd0 2024-04-20T11:34:50.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:50.612+0000 7f861e000640 1 -- 172.21.15.152:0/1732332452 >> v1:172.21.15.152:6789/0 conn(0x7f8620071cd0 legacy=0x7f86200720b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:50.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:50.612+0000 7f861e000640 1 --2- 172.21.15.152:0/1732332452 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f86000039e0 0x7f8600003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:50.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:50.612+0000 7f861e000640 1 --2- 172.21.15.152:0/1732332452 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8620160220 0x7f8600004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:50.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:50.612+0000 7f861e000640 1 --2- 172.21.15.152:0/1732332452 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8600004d40 0x7f86000051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:50.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:50.612+0000 7f861e000640 1 -- 172.21.15.152:0/1732332452 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f862013f8e0 con 0x7f86000039e0 2024-04-20T11:34:50.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:50.612+0000 7f861e000640 1 -- 172.21.15.152:0/1732332452 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f860000bbb0 con 0x7f8600004d40 2024-04-20T11:34:50.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:50.612+0000 7f861e000640 1 -- 172.21.15.152:0/1732332452 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f860000bdb0 con 0x7f8620160220 2024-04-20T11:34:50.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:50.612+0000 7f861f400640 1 --2- 172.21.15.152:0/1732332452 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f86000039e0 0x7f8600003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:50.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:50.612+0000 7f861f400640 1 -- 172.21.15.152:0/1732332452 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8620160220 msgr2=0x7f8600004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:50.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:50.612+0000 7f861f400640 1 --2- 172.21.15.152:0/1732332452 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8620160220 0x7f8600004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:50.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:50.612+0000 7f861f400640 1 -- 172.21.15.152:0/1732332452 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8600004d40 msgr2=0x7f86000051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:50.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:50.612+0000 7f861f400640 1 --2- 172.21.15.152:0/1732332452 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8600004d40 0x7f86000051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:50.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:50.612+0000 7f861f400640 1 -- 172.21.15.152:0/1732332452 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8600003520 con 0x7f86000039e0 2024-04-20T11:34:50.614 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:50.614+0000 7f861f400640 1 --2- 172.21.15.152:0/1732332452 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f86000039e0 0x7f8600003e00 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f861000c110 tx=0x7f8600005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3a9483c6ee95100 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:50.614 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:50.614+0000 7f861e000640 1 -- 172.21.15.152:0/1732332452 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8610007c40 con 0x7f86000039e0 2024-04-20T11:34:50.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:50.614+0000 7f861e000640 1 -- 172.21.15.152:0/1732332452 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8610030070 con 0x7f86000039e0 2024-04-20T11:34:50.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:50.614+0000 7f861e000640 1 -- 172.21.15.152:0/1732332452 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8610005d30 con 0x7f86000039e0 2024-04-20T11:34:50.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:50.614+0000 7f8625000640 1 -- 172.21.15.152:0/1732332452 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f86000039e0 msgr2=0x7f8600003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:50.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:50.614+0000 7f8625000640 1 --2- 172.21.15.152:0/1732332452 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f86000039e0 0x7f8600003e00 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f861000c110 tx=0x7f8600005830 comp rx=0 tx=0).stop 2024-04-20T11:34:50.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:50.614+0000 7f861fe00640 1 -- 172.21.15.152:0/1732332452 reap_dead start 2024-04-20T11:34:50.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:50.615+0000 7f8625000640 1 -- 172.21.15.152:0/1732332452 shutdown_connections 2024-04-20T11:34:50.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:50.615+0000 7f8625000640 1 -- 172.21.15.152:0/1732332452 >> 172.21.15.152:0/1732332452 conn(0x7f8620067ae0 msgr2=0x7f862006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:50.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:50.615+0000 7f8625000640 1 -- 172.21.15.152:0/1732332452 shutdown_connections 2024-04-20T11:34:50.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:50.615+0000 7f8625000640 1 -- 172.21.15.152:0/1732332452 wait complete. 2024-04-20T11:34:50.616 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:50.616+0000 7f8625000640 1 Processor -- start 2024-04-20T11:34:50.616 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:50.617+0000 7f8625000640 1 -- start start 2024-04-20T11:34:50.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:50.617+0000 7f8625000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f86000039e0 0x7f8620154400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:50.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:50.617+0000 7f8625000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8620155550 0x7f86201559d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:50.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:50.617+0000 7f8625000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f862015e400 0x7f862015e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:50.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:50.618+0000 7f8625000640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f862006e630 con 0x7f86000039e0 2024-04-20T11:34:50.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:50.618+0000 7f8625000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f862006e350 con 0x7f8620155550 2024-04-20T11:34:50.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:50.618+0000 7f8625000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f862013fb90 con 0x7f862015e400 2024-04-20T11:34:50.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:50.618+0000 7f861f400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f86000039e0 0x7f8620154400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:50.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:50.618+0000 7f861f400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f86000039e0 0x7f8620154400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49352/0 (socket says 172.21.15.152:49352) 2024-04-20T11:34:50.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:50.618+0000 7f861f400640 1 -- 172.21.15.152:0/4549618 learned_addr learned my addr 172.21.15.152:0/4549618 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:50.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:50.618+0000 7f861f400640 1 -- 172.21.15.152:0/4549618 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f862015e400 msgr2=0x7f862015e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:50.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:50.618+0000 7f861f400640 1 --2- 172.21.15.152:0/4549618 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f862015e400 0x7f862015e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:50.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:50.618+0000 7f861f400640 1 -- 172.21.15.152:0/4549618 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8620155550 msgr2=0x7f86201559d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:50.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:50.618+0000 7f861f400640 1 --2- 172.21.15.152:0/4549618 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8620155550 0x7f86201559d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:50.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:50.618+0000 7f861f400640 1 -- 172.21.15.152:0/4549618 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86201597d0 con 0x7f86000039e0 2024-04-20T11:34:50.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:50.619+0000 7f861f400640 1 --2- 172.21.15.152:0/4549618 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f86000039e0 0x7f8620154400 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f861002c4a0 tx=0x7f862006df60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:50.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:50.619+0000 7f8617400640 1 -- 172.21.15.152:0/4549618 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8610045070 con 0x7f86000039e0 2024-04-20T11:34:50.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:50.619+0000 7f8617400640 1 -- 172.21.15.152:0/4549618 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8610000ed0 con 0x7f86000039e0 2024-04-20T11:34:50.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:50.619+0000 7f8617400640 1 -- 172.21.15.152:0/4549618 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8610004df0 con 0x7f86000039e0 2024-04-20T11:34:50.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:50.619+0000 7f8625000640 1 -- 172.21.15.152:0/4549618 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8620159ab0 con 0x7f86000039e0 2024-04-20T11:34:50.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:50.619+0000 7f8625000640 1 -- 172.21.15.152:0/4549618 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8620136380 con 0x7f86000039e0 2024-04-20T11:34:50.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:50.620+0000 7f8625000640 1 -- 172.21.15.152:0/4549618 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8620075e60 con 0x7f86000039e0 2024-04-20T11:34:50.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:50.621+0000 7f8617400640 1 -- 172.21.15.152:0/4549618 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 19) ==== 91397+0+0 (secure 0 0 0) 0x7f8610007070 con 0x7f86000039e0 2024-04-20T11:34:50.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:50.622+0000 7f8617400640 1 --2- 172.21.15.152:0/4549618 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f85fc06cce0 0x7f85fc06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:50.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:50.622+0000 7f8617400640 1 -- 172.21.15.152:0/4549618 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5709+0+0 (secure 0 0 0) 0x7f86100ad280 con 0x7f86000039e0 2024-04-20T11:34:50.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:50.623+0000 7f861ea00640 1 --2- 172.21.15.152:0/4549618 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f85fc06cce0 0x7f85fc06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:50.623 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:50.624+0000 7f861ea00640 1 --2- 172.21.15.152:0/4549618 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f85fc06cce0 0x7f85fc06f1a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f86080070b0 tx=0x7f860800d040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:50.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:50.627+0000 7f8617400640 1 -- 172.21.15.152:0/4549618 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f861007ab50 con 0x7f86000039e0 2024-04-20T11:34:50.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:50.855+0000 7f8625000640 1 -- 172.21.15.152:0/4549618 --> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol36640", "target": ["mon-mgr", ""]}) -- 0x7f8620160d30 con 0x7f85fc06cce0 2024-04-20T11:34:50.870 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:34:50.871+0000 7f8a5cc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:34:50.871 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:34:50.871+0000 7f8a5cc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:34:50.871 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:34:50.871+0000 7f8a5cc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:34:50.871 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:34:50.871+0000 7f8a5cc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:34:50.871 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:34:50.871+0000 7f8a5cc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:34:50.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:50.930+0000 7f8617400640 1 -- 172.21.15.152:0/4549618 <== mgr.4109 v2:172.21.15.175:6836/4021250441 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+67 (secure 0 0 0) 0x7f860800ebc0 con 0x7f85fc06cce0 2024-04-20T11:34:50.930 INFO:teuthology.orchestra.run.smithi152.stdout:/volumes/_nogroup/subvol36640/c31a948e-91ce-43b1-bdd3-e504a43c9761 2024-04-20T11:34:50.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:50.933+0000 7f8625000640 1 -- 172.21.15.152:0/4549618 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f85fc06cce0 msgr2=0x7f85fc06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:50.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:50.933+0000 7f8625000640 1 --2- 172.21.15.152:0/4549618 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f85fc06cce0 0x7f85fc06f1a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f86080070b0 tx=0x7f860800d040 comp rx=0 tx=0).stop 2024-04-20T11:34:50.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:50.934+0000 7f8625000640 1 -- 172.21.15.152:0/4549618 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f86000039e0 msgr2=0x7f8620154400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:50.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:50.934+0000 7f8625000640 1 --2- 172.21.15.152:0/4549618 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f86000039e0 0x7f8620154400 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f861002c4a0 tx=0x7f862006df60 comp rx=0 tx=0).stop 2024-04-20T11:34:50.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:50.934+0000 7f8625000640 1 -- 172.21.15.152:0/4549618 shutdown_connections 2024-04-20T11:34:50.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:50.934+0000 7f8625000640 1 --2- 172.21.15.152:0/4549618 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f85fc06cce0 0x7f85fc06f1a0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:50.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:50.934+0000 7f8625000640 1 --2- 172.21.15.152:0/4549618 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f862015e400 0x7f862015e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:50.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:50.934+0000 7f8625000640 1 --2- 172.21.15.152:0/4549618 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8620155550 0x7f86201559d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:50.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:50.934+0000 7f8625000640 1 --2- 172.21.15.152:0/4549618 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f86000039e0 0x7f8620154400 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:50.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:50.934+0000 7f8625000640 1 -- 172.21.15.152:0/4549618 >> 172.21.15.152:0/4549618 conn(0x7f8620067ae0 msgr2=0x7f86200aeb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:50.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:50.934+0000 7f8625000640 1 -- 172.21.15.152:0/4549618 shutdown_connections 2024-04-20T11:34:50.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:50.934+0000 7f8625000640 1 -- 172.21.15.152:0/4549618 wait complete. 2024-04-20T11:34:50.955 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol43081 subvol_grp47320 2024-04-20T11:34:51.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.163+0000 7fa1dac00640 1 Processor -- start 2024-04-20T11:34:51.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.164+0000 7fa1dac00640 1 -- start start 2024-04-20T11:34:51.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.165+0000 7fa1dac00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fa1d40d05f0 0x7fa1d40d09d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:51.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.165+0000 7fa1dac00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fa1d40ae180 0x7fa1d40ae600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:51.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.165+0000 7fa1dac00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1d40d0f10 con 0x7fa1d415eee0 2024-04-20T11:34:51.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.165+0000 7fa1dac00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fa1d40aeef0 con 0x7fa1d40d05f0 2024-04-20T11:34:51.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.165+0000 7fa1dac00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fa1d40af1d0 con 0x7fa1d40ae180 2024-04-20T11:34:51.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.165+0000 7fa1d9800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fa1d40d05f0 0x7fa1d40d09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:51.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.165+0000 7fa1da200640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fa1d415eee0 0x7fa1d415f330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:50974/0 (socket says 172.21.15.152:50974) 2024-04-20T11:34:51.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.165+0000 7fa1da200640 1 -- 172.21.15.152:0/189040325 learned_addr learned my addr 172.21.15.152:0/189040325 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:34:51.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.166+0000 7fa1d8e00640 1 --2- 172.21.15.152:0/189040325 >> v2:172.21.15.175:3300/0 conn(0x7fa1d40ae180 0x7fa1d40ae600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:51.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.166+0000 7fa1d9800640 1 -- 172.21.15.152:0/189040325 >> v2:172.21.15.175:3300/0 conn(0x7fa1d40ae180 msgr2=0x7fa1d40ae600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:51.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.166+0000 7fa1d9800640 1 --2- 172.21.15.152:0/189040325 >> v2:172.21.15.175:3300/0 conn(0x7fa1d40ae180 0x7fa1d40ae600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:51.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.166+0000 7fa1d9800640 1 -- 172.21.15.152:0/189040325 >> v1:172.21.15.152:6789/0 conn(0x7fa1d415eee0 legacy=0x7fa1d415f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:51.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.166+0000 7fa1d9800640 1 -- 172.21.15.152:0/189040325 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1d40af4b0 con 0x7fa1d40d05f0 2024-04-20T11:34:51.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.167+0000 7fa1d9800640 1 --2- 172.21.15.152:0/189040325 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa1d40d05f0 0x7fa1d40d09d0 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7fa1d40b6360 tx=0x7fa1c802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c23a138c5ccda22f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:51.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.167+0000 7fa1d3e00640 1 -- 172.21.15.152:0/189040325 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c8035020 con 0x7fa1d40d05f0 2024-04-20T11:34:51.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.167+0000 7fa1d3e00640 1 -- 172.21.15.152:0/189040325 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1c8031830 con 0x7fa1d40d05f0 2024-04-20T11:34:51.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.167+0000 7fa1d3e00640 1 -- 172.21.15.152:0/189040325 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c8031c00 con 0x7fa1d40d05f0 2024-04-20T11:34:51.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.168+0000 7fa1dac00640 1 -- 172.21.15.152:0/189040325 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa1d40d05f0 msgr2=0x7fa1d40d09d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:51.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.168+0000 7fa1dac00640 1 --2- 172.21.15.152:0/189040325 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa1d40d05f0 0x7fa1d40d09d0 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7fa1d40b6360 tx=0x7fa1c802b8d0 comp rx=0 tx=0).stop 2024-04-20T11:34:51.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.168+0000 7fa1dac00640 1 -- 172.21.15.152:0/189040325 shutdown_connections 2024-04-20T11:34:51.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.168+0000 7fa1dac00640 1 --2- 172.21.15.152:0/189040325 >> v2:172.21.15.175:3300/0 conn(0x7fa1d40ae180 0x7fa1d40ae600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:51.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.168+0000 7fa1dac00640 1 --2- 172.21.15.152:0/189040325 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa1d40d05f0 0x7fa1d40d09d0 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:51.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.168+0000 7fa1dac00640 1 -- 172.21.15.152:0/189040325 >> 172.21.15.152:0/189040325 conn(0x7fa1d40b21e0 msgr2=0x7fa1d40b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:51.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.168+0000 7fa1dac00640 1 -- 172.21.15.152:0/189040325 shutdown_connections 2024-04-20T11:34:51.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.168+0000 7fa1dac00640 1 -- 172.21.15.152:0/189040325 wait complete. 2024-04-20T11:34:51.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.170+0000 7fa1dac00640 1 Processor -- start 2024-04-20T11:34:51.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.170+0000 7fa1dac00640 1 -- start start 2024-04-20T11:34:51.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.171+0000 7fa1dac00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa1d40ae180 0x7fa1d40ce0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:51.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.171+0000 7fa1dac00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa1d40d05f0 0x7fa1d41e5f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:51.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.171+0000 7fa1dac00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa1d415eee0 0x7fa1d40917f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:51.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.171+0000 7fa1dac00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d4155ca0 con 0x7fa1d40d05f0 2024-04-20T11:34:51.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.171+0000 7fa1dac00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d40afa70 con 0x7fa1d415eee0 2024-04-20T11:34:51.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.171+0000 7fa1dac00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa1d40cf2d0 con 0x7fa1d40ae180 2024-04-20T11:34:51.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.172+0000 7fa1d8e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa1d40d05f0 0x7fa1d41e5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:51.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.172+0000 7fa1d8e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa1d40d05f0 0x7fa1d41e5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49378/0 (socket says 172.21.15.152:49378) 2024-04-20T11:34:51.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.172+0000 7fa1d8e00640 1 -- 172.21.15.152:0/3987038772 learned_addr learned my addr 172.21.15.152:0/3987038772 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:51.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.172+0000 7fa1da200640 1 --2- 172.21.15.152:0/3987038772 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa1d415eee0 0x7fa1d40917f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:51.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.172+0000 7fa1d9800640 1 --2- 172.21.15.152:0/3987038772 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa1d40ae180 0x7fa1d40ce0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:51.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.172+0000 7fa1d8e00640 1 -- 172.21.15.152:0/3987038772 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa1d40ae180 msgr2=0x7fa1d40ce0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:51.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.172+0000 7fa1d8e00640 1 --2- 172.21.15.152:0/3987038772 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa1d40ae180 0x7fa1d40ce0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:51.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.172+0000 7fa1d8e00640 1 -- 172.21.15.152:0/3987038772 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa1d415eee0 msgr2=0x7fa1d40917f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:51.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.172+0000 7fa1d8e00640 1 --2- 172.21.15.152:0/3987038772 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa1d415eee0 0x7fa1d40917f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:51.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.172+0000 7fa1d8e00640 1 -- 172.21.15.152:0/3987038772 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1d40ce6c0 con 0x7fa1d40d05f0 2024-04-20T11:34:51.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.172+0000 7fa1d9800640 1 --2- 172.21.15.152:0/3987038772 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa1d40ae180 0x7fa1d40ce0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:34:51.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.173+0000 7fa1d8e00640 1 --2- 172.21.15.152:0/3987038772 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa1d40d05f0 0x7fa1d41e5f40 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7fa1c000d800 tx=0x7fa1c000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:51.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.173+0000 7fa1d1600640 1 -- 172.21.15.152:0/3987038772 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c000b6c0 con 0x7fa1d40d05f0 2024-04-20T11:34:51.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.173+0000 7fa1d1600640 1 -- 172.21.15.152:0/3987038772 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1c0019070 con 0x7fa1d40d05f0 2024-04-20T11:34:51.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.173+0000 7fa1d1600640 1 -- 172.21.15.152:0/3987038772 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c000ba80 con 0x7fa1d40d05f0 2024-04-20T11:34:51.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.173+0000 7fa1dac00640 1 -- 172.21.15.152:0/3987038772 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1d40ce9a0 con 0x7fa1d40d05f0 2024-04-20T11:34:51.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.173+0000 7fa1dac00640 1 -- 172.21.15.152:0/3987038772 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1d40cef60 con 0x7fa1d40d05f0 2024-04-20T11:34:51.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.175+0000 7fa1dac00640 1 -- 172.21.15.152:0/3987038772 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1d4075e60 con 0x7fa1d40d05f0 2024-04-20T11:34:51.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.176+0000 7fa1d1600640 1 -- 172.21.15.152:0/3987038772 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 19) ==== 91397+0+0 (secure 0 0 0) 0x7fa1c000bc20 con 0x7fa1d40d05f0 2024-04-20T11:34:51.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.176+0000 7fa1d1600640 1 --2- 172.21.15.152:0/3987038772 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fa1b406cce0 0x7fa1b406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:51.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.177+0000 7fa1d1600640 1 -- 172.21.15.152:0/3987038772 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5709+0+0 (secure 0 0 0) 0x7fa1c008c780 con 0x7fa1d40d05f0 2024-04-20T11:34:51.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.177+0000 7fa1d9800640 1 --2- 172.21.15.152:0/3987038772 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fa1b406cce0 0x7fa1b406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:51.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.178+0000 7fa1d9800640 1 --2- 172.21.15.152:0/3987038772 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fa1b406cce0 0x7fa1b406f1a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa1c8036000 tx=0x7fa1c8008030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:51.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.181+0000 7fa1d1600640 1 -- 172.21.15.152:0/3987038772 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fa1c005a5f0 con 0x7fa1d40d05f0 2024-04-20T11:34:51.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.413+0000 7fa1dac00640 1 -- 172.21.15.152:0/3987038772 --> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol43081", "group_name": "subvol_grp47320", "target": ["mon-mgr", ""]}) -- 0x7fa1d4091d30 con 0x7fa1b406cce0 2024-04-20T11:34:51.476 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.476+0000 7fa1d1600640 1 -- 172.21.15.152:0/3987038772 <== mgr.4109 v2:172.21.15.175:6836/4021250441 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+74 (secure 0 0 0) 0x7fa1d4091d30 con 0x7fa1b406cce0 2024-04-20T11:34:51.476 INFO:teuthology.orchestra.run.smithi152.stdout:/volumes/subvol_grp47320/subvol43081/076b3aa2-4a8c-45ef-84df-615379a10ac6 2024-04-20T11:34:51.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.479+0000 7fa1dac00640 1 -- 172.21.15.152:0/3987038772 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fa1b406cce0 msgr2=0x7fa1b406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:51.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.479+0000 7fa1dac00640 1 --2- 172.21.15.152:0/3987038772 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fa1b406cce0 0x7fa1b406f1a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa1c8036000 tx=0x7fa1c8008030 comp rx=0 tx=0).stop 2024-04-20T11:34:51.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.479+0000 7fa1dac00640 1 -- 172.21.15.152:0/3987038772 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa1d40d05f0 msgr2=0x7fa1d41e5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:51.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.479+0000 7fa1dac00640 1 --2- 172.21.15.152:0/3987038772 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa1d40d05f0 0x7fa1d41e5f40 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7fa1c000d800 tx=0x7fa1c000dcc0 comp rx=0 tx=0).stop 2024-04-20T11:34:51.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.479+0000 7fa1dac00640 1 -- 172.21.15.152:0/3987038772 shutdown_connections 2024-04-20T11:34:51.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.479+0000 7fa1dac00640 1 --2- 172.21.15.152:0/3987038772 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fa1b406cce0 0x7fa1b406f1a0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:51.480 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.479+0000 7fa1dac00640 1 --2- 172.21.15.152:0/3987038772 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa1d415eee0 0x7fa1d40917f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:51.480 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.479+0000 7fa1dac00640 1 --2- 172.21.15.152:0/3987038772 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa1d40d05f0 0x7fa1d41e5f40 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:51.480 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.479+0000 7fa1dac00640 1 --2- 172.21.15.152:0/3987038772 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa1d40ae180 0x7fa1d40ce0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:51.480 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.479+0000 7fa1dac00640 1 -- 172.21.15.152:0/3987038772 >> 172.21.15.152:0/3987038772 conn(0x7fa1d40b21e0 msgr2=0x7fa1d415f330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:51.480 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.480+0000 7fa1dac00640 1 -- 172.21.15.152:0/3987038772 shutdown_connections 2024-04-20T11:34:51.480 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.480+0000 7fa1dac00640 1 -- 172.21.15.152:0/3987038772 wait complete. 2024-04-20T11:34:51.500 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol90142 2024-04-20T11:34:51.705 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.705+0000 7f7901000640 1 Processor -- start 2024-04-20T11:34:51.705 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.705+0000 7f7901000640 1 -- start start 2024-04-20T11:34:51.706 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.707+0000 7f7901000640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f78fc0b2880 0x7f78fc0b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:51.706 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.707+0000 7f7901000640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78fc1463e0 con 0x7f78fc0cd1b0 2024-04-20T11:34:51.707 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.707+0000 7f7901000640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78fc0b3e20 con 0x7f78fc146000 2024-04-20T11:34:51.707 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.707+0000 7f7901000640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f78fc0b4750 con 0x7f78fc0b2880 2024-04-20T11:34:51.707 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.707+0000 7f78fa800640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f78fc0cd1b0 0x7f78fc0cd5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:50984/0 (socket says 172.21.15.152:50984) 2024-04-20T11:34:51.707 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.707+0000 7f78fa800640 1 -- 172.21.15.152:0/1304584303 learned_addr learned my addr 172.21.15.152:0/1304584303 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:34:51.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.708+0000 7f78f9400640 1 -- 172.21.15.152:0/1304584303 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3453908772 0 0) 0x7f78fc1463e0 con 0x7f78fc0cd1b0 2024-04-20T11:34:51.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.708+0000 7f78f9400640 1 -- 172.21.15.152:0/1304584303 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f78e0003520 con 0x7f78fc0cd1b0 2024-04-20T11:34:51.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.708+0000 7f78f3e00640 1 --2- 172.21.15.152:0/1304584303 >> v2:172.21.15.175:3300/0 conn(0x7f78fc0b2880 0x7f78fc0b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:51.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.708+0000 7f78f9400640 1 -- 172.21.15.152:0/1304584303 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1494839447 0 0) 0x7f78e0003520 con 0x7f78fc0cd1b0 2024-04-20T11:34:51.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.708+0000 7f78f9400640 1 -- 172.21.15.152:0/1304584303 >> v2:172.21.15.175:3300/0 conn(0x7f78fc0b2880 msgr2=0x7f78fc0b4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:51.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.708+0000 7f78f9400640 1 --2- 172.21.15.152:0/1304584303 >> v2:172.21.15.175:3300/0 conn(0x7f78fc0b2880 0x7f78fc0b4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:51.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.708+0000 7f78f9400640 1 -- 172.21.15.152:0/1304584303 >> v1:172.21.15.175:6789/0 conn(0x7f78fc146000 legacy=0x7f78fc0b2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:51.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.708+0000 7f78f9400640 1 -- 172.21.15.152:0/1304584303 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78fc0b4100 con 0x7f78fc0cd1b0 2024-04-20T11:34:51.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.708+0000 7f78f9400640 1 -- 172.21.15.152:0/1304584303 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f78f40019f0 con 0x7f78fc0cd1b0 2024-04-20T11:34:51.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.708+0000 7f78f9400640 1 -- 172.21.15.152:0/1304584303 >> v1:172.21.15.152:6789/0 conn(0x7f78fc0cd1b0 legacy=0x7f78fc0cd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:51.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.708+0000 7f78f9400640 1 --2- 172.21.15.152:0/1304584303 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78e00039b0 0x7f78e0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:51.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.708+0000 7f78f9400640 1 --2- 172.21.15.152:0/1304584303 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f78e0004800 0x7f78e0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:51.710 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.708+0000 7f78f9400640 1 --2- 172.21.15.152:0/1304584303 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f78e00052e0 0x7f78e000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:51.710 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.708+0000 7f78f9400640 1 -- 172.21.15.152:0/1304584303 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f78fc0b4750 con 0x7f78e00052e0 2024-04-20T11:34:51.710 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.708+0000 7f78f9400640 1 -- 172.21.15.152:0/1304584303 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f78e000c1b0 con 0x7f78e00039b0 2024-04-20T11:34:51.710 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.708+0000 7f78f9400640 1 -- 172.21.15.152:0/1304584303 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f78e000c3b0 con 0x7f78e0004800 2024-04-20T11:34:51.710 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.709+0000 7f78fa800640 1 --2- 172.21.15.152:0/1304584303 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f78e00052e0 0x7f78e000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:51.710 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.709+0000 7f78f9e00640 1 --2- 172.21.15.152:0/1304584303 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78e00039b0 0x7f78e0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:51.710 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.709+0000 7f78f3e00640 1 --2- 172.21.15.152:0/1304584303 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f78e0004800 0x7f78e0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:51.710 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.709+0000 7f78fa800640 1 -- 172.21.15.152:0/1304584303 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f78e0004800 msgr2=0x7f78e0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:51.710 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.709+0000 7f78fa800640 1 --2- 172.21.15.152:0/1304584303 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f78e0004800 0x7f78e0004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:51.710 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.709+0000 7f78fa800640 1 -- 172.21.15.152:0/1304584303 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78e00039b0 msgr2=0x7f78e0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:51.710 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.709+0000 7f78fa800640 1 --2- 172.21.15.152:0/1304584303 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78e00039b0 0x7f78e0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:51.710 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.709+0000 7f78fa800640 1 -- 172.21.15.152:0/1304584303 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78e0003520 con 0x7f78e00052e0 2024-04-20T11:34:51.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.710+0000 7f78fa800640 1 -- 172.21.15.152:0/1304584303 reap_dead start 2024-04-20T11:34:51.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.710+0000 7f78fa800640 1 --2- 172.21.15.152:0/1304584303 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f78e00052e0 0x7f78e000b9c0 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f78f400a830 tx=0x7f78e0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eab3cd1dd9f5b67 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:51.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.710+0000 7f78f9400640 1 -- 172.21.15.152:0/1304584303 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78f400a480 con 0x7f78e00052e0 2024-04-20T11:34:51.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.710+0000 7f78f9400640 1 -- 172.21.15.152:0/1304584303 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78f4034630 con 0x7f78e00052e0 2024-04-20T11:34:51.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.710+0000 7f78f9400640 1 -- 172.21.15.152:0/1304584303 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78f4034a00 con 0x7f78e00052e0 2024-04-20T11:34:51.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.711+0000 7f7901000640 1 -- 172.21.15.152:0/1304584303 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f78e00052e0 msgr2=0x7f78e000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:51.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.711+0000 7f7901000640 1 --2- 172.21.15.152:0/1304584303 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f78e00052e0 0x7f78e000b9c0 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f78f400a830 tx=0x7f78e0005830 comp rx=0 tx=0).stop 2024-04-20T11:34:51.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.711+0000 7f7901000640 1 -- 172.21.15.152:0/1304584303 shutdown_connections 2024-04-20T11:34:51.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.711+0000 7f7901000640 1 --2- 172.21.15.152:0/1304584303 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f78e00052e0 0x7f78e000b9c0 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:51.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.711+0000 7f7901000640 1 -- 172.21.15.152:0/1304584303 >> 172.21.15.152:0/1304584303 conn(0x7f78fc06b980 msgr2=0x7f78fc06bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:51.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.711+0000 7f7901000640 1 -- 172.21.15.152:0/1304584303 shutdown_connections 2024-04-20T11:34:51.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.711+0000 7f7901000640 1 -- 172.21.15.152:0/1304584303 wait complete. 2024-04-20T11:34:51.713 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.713+0000 7f7901000640 1 Processor -- start 2024-04-20T11:34:51.713 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.713+0000 7f7901000640 1 -- start start 2024-04-20T11:34:51.714 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.714+0000 7f7901000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78e00052e0 0x7f78fc133e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:51.714 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.714+0000 7f7901000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f78fc1e8f00 0x7f78fc1416d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:51.714 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.714+0000 7f7901000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f78fc1e9e20 0x7f78fc1ea2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:51.714 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.714+0000 7f7901000640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f78fc0b4d60 con 0x7f78fc1e8f00 2024-04-20T11:34:51.714 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.714+0000 7f7901000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f78fc0b4be0 con 0x7f78e00052e0 2024-04-20T11:34:51.714 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.714+0000 7f7901000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f78fc0b4ee0 con 0x7f78fc1e9e20 2024-04-20T11:34:51.714 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.714+0000 7f78f3e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f78fc1e8f00 0x7f78fc1416d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:51.714 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.714+0000 7f78f3e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f78fc1e8f00 0x7f78fc1416d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49394/0 (socket says 172.21.15.152:49394) 2024-04-20T11:34:51.714 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.714+0000 7f78f3e00640 1 -- 172.21.15.152:0/188306855 learned_addr learned my addr 172.21.15.152:0/188306855 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:51.715 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.715+0000 7f78f3e00640 1 -- 172.21.15.152:0/188306855 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f78fc1e9e20 msgr2=0x7f78fc1ea2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:51.715 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.715+0000 7f78f3e00640 1 --2- 172.21.15.152:0/188306855 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f78fc1e9e20 0x7f78fc1ea2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:51.716 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.715+0000 7f78f3e00640 1 -- 172.21.15.152:0/188306855 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78e00052e0 msgr2=0x7f78fc133e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:51.716 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.715+0000 7f78f3e00640 1 --2- 172.21.15.152:0/188306855 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78e00052e0 0x7f78fc133e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:51.716 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.715+0000 7f78f3e00640 1 -- 172.21.15.152:0/188306855 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78fc141f20 con 0x7f78fc1e8f00 2024-04-20T11:34:51.716 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.715+0000 7f78f3e00640 1 --2- 172.21.15.152:0/188306855 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f78fc1e8f00 0x7f78fc1416d0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f78e400e5c0 tx=0x7f78e400ea80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:51.716 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.715+0000 7f78f2000640 1 -- 172.21.15.152:0/188306855 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78e4017070 con 0x7f78fc1e8f00 2024-04-20T11:34:51.716 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.715+0000 7f78f2000640 1 -- 172.21.15.152:0/188306855 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78e4013610 con 0x7f78fc1e8f00 2024-04-20T11:34:51.716 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.715+0000 7f78f2000640 1 -- 172.21.15.152:0/188306855 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78e400f040 con 0x7f78fc1e8f00 2024-04-20T11:34:51.716 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.715+0000 7f7901000640 1 -- 172.21.15.152:0/188306855 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78fc142200 con 0x7f78fc1e8f00 2024-04-20T11:34:51.716 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.716+0000 7f7901000640 1 -- 172.21.15.152:0/188306855 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78fc1427c0 con 0x7f78fc1e8f00 2024-04-20T11:34:51.716 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.717+0000 7f7901000640 1 -- 172.21.15.152:0/188306855 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78fc0689f0 con 0x7f78fc1e8f00 2024-04-20T11:34:51.717 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.718+0000 7f78f2000640 1 -- 172.21.15.152:0/188306855 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 19) ==== 91397+0+0 (secure 0 0 0) 0x7f78e401a070 con 0x7f78fc1e8f00 2024-04-20T11:34:51.718 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.719+0000 7f78f2000640 1 --2- 172.21.15.152:0/188306855 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f78d806cd10 0x7f78d806f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:51.718 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.719+0000 7f78f2000640 1 -- 172.21.15.152:0/188306855 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5709+0+0 (secure 0 0 0) 0x7f78e408be00 con 0x7f78fc1e8f00 2024-04-20T11:34:51.718 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.719+0000 7f78f9e00640 1 --2- 172.21.15.152:0/188306855 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f78d806cd10 0x7f78d806f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:51.720 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.721+0000 7f78f9e00640 1 --2- 172.21.15.152:0/188306855 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f78d806cd10 0x7f78d806f1d0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f78fc144860 tx=0x7f78ec009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:51.722 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.723+0000 7f78f2000640 1 -- 172.21.15.152:0/188306855 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f78e4059bd0 con 0x7f78fc1e8f00 2024-04-20T11:34:51.945 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.945+0000 7f7901000640 1 -- 172.21.15.152:0/188306855 --> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol90142", "target": ["mon-mgr", ""]}) -- 0x7f78fc15c9d0 con 0x7f78d806cd10 2024-04-20T11:34:51.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.993+0000 7f78f2000640 1 -- 172.21.15.152:0/188306855 <== mgr.4109 v2:172.21.15.175:6836/4021250441 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+144 (secure 0 0 0) 0x7f78fc15c9d0 con 0x7f78d806cd10 2024-04-20T11:34:51.993 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-04-20T11:34:51.993 INFO:teuthology.orchestra.run.smithi152.stdout: "status": { 2024-04-20T11:34:51.993 INFO:teuthology.orchestra.run.smithi152.stdout: "state": "pending", 2024-04-20T11:34:51.993 INFO:teuthology.orchestra.run.smithi152.stdout: "source": { 2024-04-20T11:34:51.993 INFO:teuthology.orchestra.run.smithi152.stdout: "volume": "fake", 2024-04-20T11:34:51.993 INFO:teuthology.orchestra.run.smithi152.stdout: "subvolume": "fake", 2024-04-20T11:34:51.993 INFO:teuthology.orchestra.run.smithi152.stdout: "snapshot": "fake" 2024-04-20T11:34:51.993 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:34:51.994 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:34:51.994 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-04-20T11:34:51.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.995+0000 7f7901000640 1 -- 172.21.15.152:0/188306855 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f78d806cd10 msgr2=0x7f78d806f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:51.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.996+0000 7f7901000640 1 --2- 172.21.15.152:0/188306855 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f78d806cd10 0x7f78d806f1d0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f78fc144860 tx=0x7f78ec009040 comp rx=0 tx=0).stop 2024-04-20T11:34:51.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.996+0000 7f7901000640 1 -- 172.21.15.152:0/188306855 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f78fc1e8f00 msgr2=0x7f78fc1416d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:51.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.996+0000 7f7901000640 1 --2- 172.21.15.152:0/188306855 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f78fc1e8f00 0x7f78fc1416d0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f78e400e5c0 tx=0x7f78e400ea80 comp rx=0 tx=0).stop 2024-04-20T11:34:51.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.996+0000 7f7901000640 1 -- 172.21.15.152:0/188306855 shutdown_connections 2024-04-20T11:34:51.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.996+0000 7f7901000640 1 --2- 172.21.15.152:0/188306855 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f78d806cd10 0x7f78d806f1d0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:51.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.996+0000 7f7901000640 1 --2- 172.21.15.152:0/188306855 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f78fc1e9e20 0x7f78fc1ea2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:51.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.996+0000 7f7901000640 1 --2- 172.21.15.152:0/188306855 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f78fc1e8f00 0x7f78fc1416d0 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:51.997 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.996+0000 7f7901000640 1 --2- 172.21.15.152:0/188306855 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78e00052e0 0x7f78fc133e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:51.997 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.996+0000 7f7901000640 1 -- 172.21.15.152:0/188306855 >> 172.21.15.152:0/188306855 conn(0x7f78fc06b980 msgr2=0x7f78fc06dfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:51.997 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.996+0000 7f7901000640 1 -- 172.21.15.152:0/188306855 shutdown_connections 2024-04-20T11:34:51.997 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:51.996+0000 7f7901000640 1 -- 172.21.15.152:0/188306855 wait complete. 2024-04-20T11:34:52.014 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm cephfs subvol36640 fake 2024-04-20T11:34:52.220 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.220+0000 7f11de600640 1 Processor -- start 2024-04-20T11:34:52.220 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.220+0000 7f11de600640 1 -- start start 2024-04-20T11:34:52.222 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.222+0000 7f11de600640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f11d80d1b60 0x7f11d8162aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:52.222 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.222+0000 7f11de600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f11d8162fe0 0x7f11d80b3030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:52.222 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.222+0000 7f11de600640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11d80c79b0 con 0x7f11d80c73f0 2024-04-20T11:34:52.222 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.222+0000 7f11de600640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f11d8162610 con 0x7f11d80d1b60 2024-04-20T11:34:52.222 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.222+0000 7f11de600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f11d80c7c70 con 0x7f11d8162fe0 2024-04-20T11:34:52.222 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.222+0000 7f11cea00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f11d80d1b60 0x7f11d8162aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:52.222 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.222+0000 7f11d7400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f11d80c73f0 0x7f11d80c6310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:50992/0 (socket says 172.21.15.152:50992) 2024-04-20T11:34:52.222 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.222+0000 7f11d7400640 1 -- 172.21.15.152:0/1198299704 learned_addr learned my addr 172.21.15.152:0/1198299704 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:34:52.223 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.223+0000 7f11cea00640 1 -- 172.21.15.152:0/1198299704 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f11d8162fe0 msgr2=0x7f11d80b3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:52.223 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.223+0000 7f11d7e00640 1 --2- 172.21.15.152:0/1198299704 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f11d8162fe0 0x7f11d80b3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:52.224 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.223+0000 7f11cea00640 1 --2- 172.21.15.152:0/1198299704 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f11d8162fe0 0x7f11d80b3030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:52.224 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.223+0000 7f11cea00640 1 -- 172.21.15.152:0/1198299704 >> v1:172.21.15.152:6789/0 conn(0x7f11d80c73f0 legacy=0x7f11d80c6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:52.224 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.223+0000 7f11d6a00640 1 -- 172.21.15.152:0/1198299704 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3716997928 0 0) 0x7f11d80c79b0 con 0x7f11d80c73f0 2024-04-20T11:34:52.224 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.223+0000 7f11cea00640 1 -- 172.21.15.152:0/1198299704 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11d80c7f50 con 0x7f11d80d1b60 2024-04-20T11:34:52.224 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.224+0000 7f11cea00640 1 --2- 172.21.15.152:0/1198299704 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f11d80d1b60 0x7f11d8162aa0 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f11c00097a0 tx=0x7f11c002b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c27906662569cd9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:52.224 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.224+0000 7f11d6a00640 1 -- 172.21.15.152:0/1198299704 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c0035020 con 0x7f11d80d1b60 2024-04-20T11:34:52.224 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.224+0000 7f11d6a00640 1 -- 172.21.15.152:0/1198299704 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11c0031810 con 0x7f11d80d1b60 2024-04-20T11:34:52.224 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.224+0000 7f11d6a00640 1 -- 172.21.15.152:0/1198299704 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c0031be0 con 0x7f11d80d1b60 2024-04-20T11:34:52.225 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.224+0000 7f11de600640 1 -- 172.21.15.152:0/1198299704 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f11d80d1b60 msgr2=0x7f11d8162aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:52.225 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.224+0000 7f11de600640 1 --2- 172.21.15.152:0/1198299704 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f11d80d1b60 0x7f11d8162aa0 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f11c00097a0 tx=0x7f11c002b8b0 comp rx=0 tx=0).stop 2024-04-20T11:34:52.225 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.225+0000 7f11de600640 1 -- 172.21.15.152:0/1198299704 shutdown_connections 2024-04-20T11:34:52.225 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.225+0000 7f11de600640 1 --2- 172.21.15.152:0/1198299704 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f11d8162fe0 0x7f11d80b3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:52.225 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.225+0000 7f11de600640 1 --2- 172.21.15.152:0/1198299704 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f11d80d1b60 0x7f11d8162aa0 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:52.225 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.225+0000 7f11de600640 1 -- 172.21.15.152:0/1198299704 >> 172.21.15.152:0/1198299704 conn(0x7f11d809aa40 msgr2=0x7f11d80d1450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:52.225 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.225+0000 7f11de600640 1 -- 172.21.15.152:0/1198299704 shutdown_connections 2024-04-20T11:34:52.225 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.225+0000 7f11de600640 1 -- 172.21.15.152:0/1198299704 wait complete. 2024-04-20T11:34:52.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.226+0000 7f11de600640 1 Processor -- start 2024-04-20T11:34:52.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.226+0000 7f11de600640 1 -- start start 2024-04-20T11:34:52.227 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.227+0000 7f11de600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f11d80c73f0 0x7f11d8157620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:52.227 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.227+0000 7f11de600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f11d80d1b60 0x7f11d815ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:52.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.227+0000 7f11de600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f11d8162fe0 0x7f11d80ca700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:52.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.227+0000 7f11de600640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f11d80c6a20 con 0x7f11d80d1b60 2024-04-20T11:34:52.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.227+0000 7f11de600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f11d80d1090 con 0x7f11d8162fe0 2024-04-20T11:34:52.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.227+0000 7f11de600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f11d80c8490 con 0x7f11d80c73f0 2024-04-20T11:34:52.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.227+0000 7f11cea00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f11d80d1b60 0x7f11d815ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:52.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.227+0000 7f11cea00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f11d80d1b60 0x7f11d815ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49408/0 (socket says 172.21.15.152:49408) 2024-04-20T11:34:52.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.227+0000 7f11cea00640 1 -- 172.21.15.152:0/1774606684 learned_addr learned my addr 172.21.15.152:0/1774606684 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:52.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.228+0000 7f11d7400640 1 --2- 172.21.15.152:0/1774606684 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f11d80c73f0 0x7f11d8157620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:52.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.228+0000 7f11cea00640 1 -- 172.21.15.152:0/1774606684 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f11d80c73f0 msgr2=0x7f11d8157620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:52.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.228+0000 7f11cea00640 1 --2- 172.21.15.152:0/1774606684 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f11d80c73f0 0x7f11d8157620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:52.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.228+0000 7f11cea00640 1 -- 172.21.15.152:0/1774606684 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f11d8162fe0 msgr2=0x7f11d80ca700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:52.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.228+0000 7f11cea00640 1 --2- 172.21.15.152:0/1774606684 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f11d8162fe0 0x7f11d80ca700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:52.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.228+0000 7f11cea00640 1 -- 172.21.15.152:0/1774606684 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11d80b1a70 con 0x7f11d80d1b60 2024-04-20T11:34:52.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.228+0000 7f11cea00640 1 --2- 172.21.15.152:0/1774606684 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f11d80d1b60 0x7f11d815ecb0 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f11c0000c00 tx=0x7f11c00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:52.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.228+0000 7f11cfe00640 1 -- 172.21.15.152:0/1774606684 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c0035050 con 0x7f11d80d1b60 2024-04-20T11:34:52.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.228+0000 7f11cfe00640 1 -- 172.21.15.152:0/1774606684 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11c003bd70 con 0x7f11d80d1b60 2024-04-20T11:34:52.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.228+0000 7f11cfe00640 1 -- 172.21.15.152:0/1774606684 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c0004270 con 0x7f11d80d1b60 2024-04-20T11:34:52.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.228+0000 7f11de600640 1 -- 172.21.15.152:0/1774606684 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11d815f600 con 0x7f11d80d1b60 2024-04-20T11:34:52.230 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.229+0000 7f11de600640 1 -- 172.21.15.152:0/1774606684 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11d815fbc0 con 0x7f11d80d1b60 2024-04-20T11:34:52.230 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.230+0000 7f11de600640 1 -- 172.21.15.152:0/1774606684 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11d8069100 con 0x7f11d80d1b60 2024-04-20T11:34:52.230 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.231+0000 7f11cfe00640 1 -- 172.21.15.152:0/1774606684 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 20) ==== 91448+0+0 (secure 0 0 0) 0x7f11c0038020 con 0x7f11d80d1b60 2024-04-20T11:34:52.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.231+0000 7f11cfe00640 1 --2- 172.21.15.152:0/1774606684 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f11b406cd70 0x7f11b406f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:52.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.232+0000 7f11cfe00640 1 -- 172.21.15.152:0/1774606684 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5709+0+0 (secure 0 0 0) 0x7f11c00ae650 con 0x7f11d80d1b60 2024-04-20T11:34:52.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.232+0000 7f11d7400640 1 --2- 172.21.15.152:0/1774606684 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f11b406cd70 0x7f11b406f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:52.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.233+0000 7f11d7400640 1 --2- 172.21.15.152:0/1774606684 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f11b406cd70 0x7f11b406f230 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f11c8007fd0 tx=0x7f11c8009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:52.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.236+0000 7f11cfe00640 1 -- 172.21.15.152:0/1774606684 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f11d80b1a70 con 0x7f11d80d1b60 2024-04-20T11:34:52.457 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.457+0000 7f11de600640 1 -- 172.21.15.152:0/1774606684 --> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "vol_name": "cephfs", "sub_name": "subvol36640", "snap_name": "fake", "target": ["mon-mgr", ""]}) -- 0x7f11d80cb2a0 con 0x7f11b406cd70 2024-04-20T11:34:52.531 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.532+0000 7f11cfe00640 1 -- 172.21.15.152:0/1774606684 <== mgr.4109 v2:172.21.15.175:6836/4021250441 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f11d80cb2a0 con 0x7f11b406cd70 2024-04-20T11:34:52.535 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.535+0000 7f11de600640 1 -- 172.21.15.152:0/1774606684 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f11b406cd70 msgr2=0x7f11b406f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:52.535 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.535+0000 7f11de600640 1 --2- 172.21.15.152:0/1774606684 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f11b406cd70 0x7f11b406f230 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f11c8007fd0 tx=0x7f11c8009040 comp rx=0 tx=0).stop 2024-04-20T11:34:52.535 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.535+0000 7f11de600640 1 -- 172.21.15.152:0/1774606684 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f11d80d1b60 msgr2=0x7f11d815ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:52.535 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.535+0000 7f11de600640 1 --2- 172.21.15.152:0/1774606684 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f11d80d1b60 0x7f11d815ecb0 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f11c0000c00 tx=0x7f11c00046c0 comp rx=0 tx=0).stop 2024-04-20T11:34:52.535 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.535+0000 7f11de600640 1 -- 172.21.15.152:0/1774606684 shutdown_connections 2024-04-20T11:34:52.536 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.535+0000 7f11de600640 1 --2- 172.21.15.152:0/1774606684 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f11b406cd70 0x7f11b406f230 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:52.536 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.535+0000 7f11de600640 1 --2- 172.21.15.152:0/1774606684 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f11d8162fe0 0x7f11d80ca700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:52.536 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.535+0000 7f11de600640 1 --2- 172.21.15.152:0/1774606684 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f11d80d1b60 0x7f11d815ecb0 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:52.536 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.535+0000 7f11de600640 1 --2- 172.21.15.152:0/1774606684 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f11d80c73f0 0x7f11d8157620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:52.536 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.535+0000 7f11de600640 1 -- 172.21.15.152:0/1774606684 >> 172.21.15.152:0/1774606684 conn(0x7f11d809aa40 msgr2=0x7f11d80d1420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:52.536 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.535+0000 7f11de600640 1 -- 172.21.15.152:0/1774606684 shutdown_connections 2024-04-20T11:34:52.536 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.536+0000 7f11de600640 1 -- 172.21.15.152:0/1774606684 wait complete. 2024-04-20T11:34:52.553 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm cephfs subvol43081 fake subvol_grp47320 2024-04-20T11:34:52.750 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.750+0000 7f2aea200640 1 Processor -- start 2024-04-20T11:34:52.750 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.751+0000 7f2aea200640 1 -- start start 2024-04-20T11:34:52.752 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.752+0000 7f2aea200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2ae40b2880 0x7f2ae40b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:52.752 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.752+0000 7f2aea200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f2ae40cd1b0 0x7f2ae40cd600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:52.752 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.752+0000 7f2aea200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ae41463e0 con 0x7f2ae4146000 2024-04-20T11:34:52.752 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.752+0000 7f2aea200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f2ae40b4750 con 0x7f2ae40cd1b0 2024-04-20T11:34:52.752 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.752+0000 7f2aea200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2ae40b3e20 con 0x7f2ae40b2880 2024-04-20T11:34:52.753 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.753+0000 7f2ae2a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2ae40b2880 0x7f2ae40b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:52.753 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.753+0000 7f2ae3400640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7f2ae4146000 0x7f2ae40b2170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.152:56036/0 (socket says 172.21.15.152:56036) 2024-04-20T11:34:52.753 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.753+0000 7f2ae2a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2ae40b2880 0x7f2ae40b4be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.152:33326/0 (socket says 172.21.15.152:33326) 2024-04-20T11:34:52.753 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.753+0000 7f2ae3400640 1 -- 172.21.15.152:0/3440522360 learned_addr learned my addr 172.21.15.152:0/3440522360 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:34:52.753 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.753+0000 7f2ae3e00640 1 --2- 172.21.15.152:0/3440522360 >> v2:172.21.15.175:3300/0 conn(0x7f2ae40cd1b0 0x7f2ae40cd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:52.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.754+0000 7f2ae2a00640 1 -- 172.21.15.152:0/3440522360 >> v2:172.21.15.175:3300/0 conn(0x7f2ae40cd1b0 msgr2=0x7f2ae40cd600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:52.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.754+0000 7f2ae2a00640 1 --2- 172.21.15.152:0/3440522360 >> v2:172.21.15.175:3300/0 conn(0x7f2ae40cd1b0 0x7f2ae40cd600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:52.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.754+0000 7f2ae2a00640 1 -- 172.21.15.152:0/3440522360 >> v1:172.21.15.175:6789/0 conn(0x7f2ae4146000 legacy=0x7f2ae40b2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:52.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.754+0000 7f2ae2a00640 1 -- 172.21.15.152:0/3440522360 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ae40b4100 con 0x7f2ae40b2880 2024-04-20T11:34:52.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.755+0000 7f2ae2a00640 1 --2- 172.21.15.152:0/3440522360 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2ae40b2880 0x7f2ae40b4be0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2ad00097a0 tx=0x7f2ad002b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45713432006ffa48 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:52.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.756+0000 7f2ae2000640 1 -- 172.21.15.152:0/3440522360 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad00315a0 con 0x7f2ae40b2880 2024-04-20T11:34:52.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.756+0000 7f2ae2000640 1 -- 172.21.15.152:0/3440522360 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ad0037070 con 0x7f2ae40b2880 2024-04-20T11:34:52.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.756+0000 7f2ae2000640 1 -- 172.21.15.152:0/3440522360 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad0031960 con 0x7f2ae40b2880 2024-04-20T11:34:52.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.757+0000 7f2aea200640 1 -- 172.21.15.152:0/3440522360 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2ae40b2880 msgr2=0x7f2ae40b4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:52.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.757+0000 7f2aea200640 1 --2- 172.21.15.152:0/3440522360 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2ae40b2880 0x7f2ae40b4be0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2ad00097a0 tx=0x7f2ad002b840 comp rx=0 tx=0).stop 2024-04-20T11:34:52.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.757+0000 7f2aea200640 1 -- 172.21.15.152:0/3440522360 shutdown_connections 2024-04-20T11:34:52.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.757+0000 7f2aea200640 1 --2- 172.21.15.152:0/3440522360 >> v2:172.21.15.175:3300/0 conn(0x7f2ae40cd1b0 0x7f2ae40cd600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:52.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.757+0000 7f2aea200640 1 --2- 172.21.15.152:0/3440522360 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2ae40b2880 0x7f2ae40b4be0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:52.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.757+0000 7f2aea200640 1 -- 172.21.15.152:0/3440522360 >> 172.21.15.152:0/3440522360 conn(0x7f2ae406b980 msgr2=0x7f2ae406bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:52.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.757+0000 7f2aea200640 1 -- 172.21.15.152:0/3440522360 shutdown_connections 2024-04-20T11:34:52.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.757+0000 7f2aea200640 1 -- 172.21.15.152:0/3440522360 wait complete. 2024-04-20T11:34:52.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.759+0000 7f2aea200640 1 Processor -- start 2024-04-20T11:34:52.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.759+0000 7f2aea200640 1 -- start start 2024-04-20T11:34:52.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.761+0000 7f2aea200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ae40b2880 0x7f2ae41e6940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:52.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.761+0000 7f2aea200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ae40cd1b0 0x7f2ae41e4f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:52.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.761+0000 7f2aea200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2ae4146000 0x7f2ae40cb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:52.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.761+0000 7f2aea200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f2ae40cdcc0 con 0x7f2ae40b2880 2024-04-20T11:34:52.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.761+0000 7f2aea200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2ae40cdb40 con 0x7f2ae40cd1b0 2024-04-20T11:34:52.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.761+0000 7f2aea200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2ae415ccf0 con 0x7f2ae4146000 2024-04-20T11:34:52.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.761+0000 7f2ae3400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ae40b2880 0x7f2ae41e6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:52.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.761+0000 7f2ae3400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ae40b2880 0x7f2ae41e6940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49412/0 (socket says 172.21.15.152:49412) 2024-04-20T11:34:52.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.761+0000 7f2ae3400640 1 -- 172.21.15.152:0/697062052 learned_addr learned my addr 172.21.15.152:0/697062052 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:52.762 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.761+0000 7f2ae2a00640 1 --2- 172.21.15.152:0/697062052 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ae40cd1b0 0x7f2ae41e4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:52.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.761+0000 7f2ae3e00640 1 --2- 172.21.15.152:0/697062052 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2ae4146000 0x7f2ae40cb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:52.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.761+0000 7f2ae3400640 1 -- 172.21.15.152:0/697062052 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2ae4146000 msgr2=0x7f2ae40cb6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:52.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.761+0000 7f2ae3400640 1 --2- 172.21.15.152:0/697062052 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2ae4146000 0x7f2ae40cb6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:52.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.761+0000 7f2ae3400640 1 -- 172.21.15.152:0/697062052 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ae40cd1b0 msgr2=0x7f2ae41e4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:52.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.761+0000 7f2ae3400640 1 --2- 172.21.15.152:0/697062052 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ae40cd1b0 0x7f2ae41e4f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:52.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.761+0000 7f2ae3400640 1 -- 172.21.15.152:0/697062052 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ae41e59e0 con 0x7f2ae40b2880 2024-04-20T11:34:52.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.761+0000 7f2ae2a00640 1 --2- 172.21.15.152:0/697062052 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ae40cd1b0 0x7f2ae41e4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:34:52.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.762+0000 7f2ae3400640 1 --2- 172.21.15.152:0/697062052 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ae40b2880 0x7f2ae41e6940 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f2ad400c3c0 tx=0x7f2ad400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:52.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.762+0000 7f2adb400640 1 -- 172.21.15.152:0/697062052 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad4017070 con 0x7f2ae40b2880 2024-04-20T11:34:52.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.762+0000 7f2adb400640 1 -- 172.21.15.152:0/697062052 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ad4005ce0 con 0x7f2ae40b2880 2024-04-20T11:34:52.764 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.762+0000 7f2adb400640 1 -- 172.21.15.152:0/697062052 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad40072f0 con 0x7f2ae40b2880 2024-04-20T11:34:52.764 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.762+0000 7f2aea200640 1 -- 172.21.15.152:0/697062052 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ae41e5cc0 con 0x7f2ae40b2880 2024-04-20T11:34:52.764 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.762+0000 7f2aea200640 1 -- 172.21.15.152:0/697062052 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ae41e6280 con 0x7f2ae40b2880 2024-04-20T11:34:52.764 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.763+0000 7f2aea200640 1 -- 172.21.15.152:0/697062052 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ae40689f0 con 0x7f2ae40b2880 2024-04-20T11:34:52.764 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.764+0000 7f2adb400640 1 -- 172.21.15.152:0/697062052 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 20) ==== 91448+0+0 (secure 0 0 0) 0x7f2ad401a070 con 0x7f2ae40b2880 2024-04-20T11:34:52.765 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.765+0000 7f2adb400640 1 --2- 172.21.15.152:0/697062052 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f2ac406cd70 0x7f2ac406f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:52.765 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.765+0000 7f2adb400640 1 -- 172.21.15.152:0/697062052 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5709+0+0 (secure 0 0 0) 0x7f2ad408b720 con 0x7f2ae40b2880 2024-04-20T11:34:52.765 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.766+0000 7f2ae2a00640 1 --2- 172.21.15.152:0/697062052 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f2ac406cd70 0x7f2ac406f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:52.766 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.767+0000 7f2ae2a00640 1 --2- 172.21.15.152:0/697062052 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f2ac406cd70 0x7f2ac406f230 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2ad0036000 tx=0x7f2ad0008030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:52.770 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:52.771+0000 7f2adb400640 1 -- 172.21.15.152:0/697062052 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f2ad40594b0 con 0x7f2ae40b2880 2024-04-20T11:34:53.011 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:53.012+0000 7f2aea200640 1 -- 172.21.15.152:0/697062052 --> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "vol_name": "cephfs", "sub_name": "subvol43081", "snap_name": "fake", "group_name": "subvol_grp47320", "target": ["mon-mgr", ""]}) -- 0x7f2ae40ad8b0 con 0x7f2ac406cd70 2024-04-20T11:34:53.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:53.094+0000 7f2adb400640 1 -- 172.21.15.152:0/697062052 <== mgr.4109 v2:172.21.15.175:6836/4021250441 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f2ae40ad8b0 con 0x7f2ac406cd70 2024-04-20T11:34:53.098 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:53.098+0000 7f2aea200640 1 -- 172.21.15.152:0/697062052 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f2ac406cd70 msgr2=0x7f2ac406f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:53.098 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:53.098+0000 7f2aea200640 1 --2- 172.21.15.152:0/697062052 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f2ac406cd70 0x7f2ac406f230 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2ad0036000 tx=0x7f2ad0008030 comp rx=0 tx=0).stop 2024-04-20T11:34:53.098 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:53.098+0000 7f2aea200640 1 -- 172.21.15.152:0/697062052 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ae40b2880 msgr2=0x7f2ae41e6940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:53.098 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:53.098+0000 7f2aea200640 1 --2- 172.21.15.152:0/697062052 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ae40b2880 0x7f2ae41e6940 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f2ad400c3c0 tx=0x7f2ad400c880 comp rx=0 tx=0).stop 2024-04-20T11:34:53.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:53.098+0000 7f2aea200640 1 -- 172.21.15.152:0/697062052 shutdown_connections 2024-04-20T11:34:53.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:53.098+0000 7f2aea200640 1 --2- 172.21.15.152:0/697062052 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f2ac406cd70 0x7f2ac406f230 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:53.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:53.098+0000 7f2aea200640 1 --2- 172.21.15.152:0/697062052 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2ae4146000 0x7f2ae40cb6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:53.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:53.098+0000 7f2aea200640 1 --2- 172.21.15.152:0/697062052 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ae40cd1b0 0x7f2ae41e4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:53.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:53.098+0000 7f2aea200640 1 --2- 172.21.15.152:0/697062052 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ae40b2880 0x7f2ae41e6940 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:53.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:53.098+0000 7f2aea200640 1 -- 172.21.15.152:0/697062052 >> 172.21.15.152:0/697062052 conn(0x7f2ae406b980 msgr2=0x7f2ae40afef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:53.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:53.099+0000 7f2aea200640 1 -- 172.21.15.152:0/697062052 shutdown_connections 2024-04-20T11:34:53.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:53.099+0000 7f2aea200640 1 -- 172.21.15.152:0/697062052 wait complete. 2024-04-20T11:34:53.118 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:34:53.118 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_nogroup/subvol36640/.meta) 2024-04-20T11:34:53.202 INFO:teuthology.orchestra.run.smithi152.stdout:[GLOBAL] 2024-04-20T11:34:53.203 INFO:teuthology.orchestra.run.smithi152.stdout:version = 1 2024-04-20T11:34:53.203 INFO:teuthology.orchestra.run.smithi152.stdout:type = subvolume 2024-04-20T11:34:53.203 INFO:teuthology.orchestra.run.smithi152.stdout:path = /volumes/_nogroup/subvol36640/c31a948e-91ce-43b1-bdd3-e504a43c9761 2024-04-20T11:34:53.203 INFO:teuthology.orchestra.run.smithi152.stdout:state = complete 2024-04-20T11:34:53.203 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:34:53.207 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:34:53.208 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/subvol_grp47320/subvol43081/.meta) 2024-04-20T11:34:53.296 INFO:teuthology.orchestra.run.smithi152.stdout:[GLOBAL] 2024-04-20T11:34:53.296 INFO:teuthology.orchestra.run.smithi152.stdout:version = 1 2024-04-20T11:34:53.297 INFO:teuthology.orchestra.run.smithi152.stdout:type = subvolume 2024-04-20T11:34:53.297 INFO:teuthology.orchestra.run.smithi152.stdout:path = /volumes/subvol_grp47320/subvol43081/076b3aa2-4a8c-45ef-84df-615379a10ac6 2024-04-20T11:34:53.297 INFO:teuthology.orchestra.run.smithi152.stdout:state = complete 2024-04-20T11:34:53.297 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:34:53.302 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol36640 2024-04-20T11:34:53.325 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:34:53.355 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:34:53.503 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:53.503+0000 7f4d71e00640 1 Processor -- start 2024-04-20T11:34:53.503 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:53.504+0000 7f4d71e00640 1 -- start start 2024-04-20T11:34:53.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:53.505+0000 7f4d71e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f4d6c0b7890 0x7f4d6c0d2860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:53.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:53.505+0000 7f4d71e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f4d6c0cf4f0 0x7f4d6c0cf920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:53.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:53.505+0000 7f4d71e00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d6c0b7d00 con 0x7f4d6c161e50 2024-04-20T11:34:53.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:53.505+0000 7f4d71e00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f4d6c162230 con 0x7f4d6c0b7890 2024-04-20T11:34:53.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:53.505+0000 7f4d71e00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f4d6c0b7f70 con 0x7f4d6c0cf4f0 2024-04-20T11:34:53.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:53.505+0000 7f4d6a000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f4d6c0b7890 0x7f4d6c0d2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:53.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:53.506+0000 7f4d6a000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f4d6c0b7890 0x7f4d6c0d2860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49420/0 (socket says 172.21.15.152:49420) 2024-04-20T11:34:53.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:53.506+0000 7f4d6a000640 1 -- 172.21.15.152:0/1412185452 learned_addr learned my addr 172.21.15.152:0/1412185452 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:53.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:53.506+0000 7f4d6b400640 1 --2- 172.21.15.152:0/1412185452 >> v2:172.21.15.175:3300/0 conn(0x7f4d6c0cf4f0 0x7f4d6c0cf920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:53.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:53.506+0000 7f4d6a000640 1 -- 172.21.15.152:0/1412185452 >> v2:172.21.15.175:3300/0 conn(0x7f4d6c0cf4f0 msgr2=0x7f4d6c0cf920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:53.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:53.506+0000 7f4d6a000640 1 --2- 172.21.15.152:0/1412185452 >> v2:172.21.15.175:3300/0 conn(0x7f4d6c0cf4f0 0x7f4d6c0cf920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:53.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:53.506+0000 7f4d6a000640 1 -- 172.21.15.152:0/1412185452 >> v1:172.21.15.152:6789/0 conn(0x7f4d6c161e50 legacy=0x7f4d6c0ba180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:53.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:53.506+0000 7f4d6a000640 1 -- 172.21.15.152:0/1412185452 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d6c0b8250 con 0x7f4d6c0b7890 2024-04-20T11:34:53.508 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:53.507+0000 7f4d6a000640 1 --2- 172.21.15.152:0/1412185452 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d6c0b7890 0x7f4d6c0d2860 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c0b70c0 tx=0x7f4d5402ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50fe19047041e27a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:53.508 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:53.508+0000 7f4d69600640 1 -- 172.21.15.152:0/1412185452 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d5402f850 con 0x7f4d6c0b7890 2024-04-20T11:34:53.508 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:53.508+0000 7f4d69600640 1 -- 172.21.15.152:0/1412185452 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d5402f9f0 con 0x7f4d6c0b7890 2024-04-20T11:34:53.509 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:53.508+0000 7f4d69600640 1 -- 172.21.15.152:0/1412185452 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d5402fdc0 con 0x7f4d6c0b7890 2024-04-20T11:34:53.509 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:53.508+0000 7f4d71e00640 1 -- 172.21.15.152:0/1412185452 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d6c0b7890 msgr2=0x7f4d6c0d2860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:53.509 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:53.508+0000 7f4d71e00640 1 --2- 172.21.15.152:0/1412185452 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d6c0b7890 0x7f4d6c0d2860 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c0b70c0 tx=0x7f4d5402ba30 comp rx=0 tx=0).stop 2024-04-20T11:34:53.509 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:53.508+0000 7f4d71e00640 1 -- 172.21.15.152:0/1412185452 shutdown_connections 2024-04-20T11:34:53.509 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:53.508+0000 7f4d71e00640 1 --2- 172.21.15.152:0/1412185452 >> v2:172.21.15.175:3300/0 conn(0x7f4d6c0cf4f0 0x7f4d6c0cf920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:53.509 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:53.508+0000 7f4d71e00640 1 --2- 172.21.15.152:0/1412185452 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d6c0b7890 0x7f4d6c0d2860 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:53.509 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:53.508+0000 7f4d71e00640 1 -- 172.21.15.152:0/1412185452 >> 172.21.15.152:0/1412185452 conn(0x7f4d6c067ae0 msgr2=0x7f4d6c067ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:53.509 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:53.508+0000 7f4d71e00640 1 -- 172.21.15.152:0/1412185452 shutdown_connections 2024-04-20T11:34:53.509 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:53.508+0000 7f4d71e00640 1 -- 172.21.15.152:0/1412185452 wait complete. 2024-04-20T11:34:53.510 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:53.510+0000 7f4d71e00640 1 Processor -- start 2024-04-20T11:34:53.510 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:53.510+0000 7f4d71e00640 1 -- start start 2024-04-20T11:34:53.512 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:53.511+0000 7f4d71e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d6c0b7890 0x7f4d6c0c6990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:53.512 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:53.511+0000 7f4d71e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d6c0cf4f0 0x7f4d6c1608f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:53.513 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:53.511+0000 7f4d71e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4d6c161e50 0x7f4d6c160e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:53.513 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:53.511+0000 7f4d71e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f4d6c0b8af0 con 0x7f4d6c0b7890 2024-04-20T11:34:53.513 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:53.511+0000 7f4d71e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4d6c0b8810 con 0x7f4d6c0cf4f0 2024-04-20T11:34:53.513 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:53.511+0000 7f4d71e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4d6c0cfe60 con 0x7f4d6c161e50 2024-04-20T11:34:53.513 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:53.511+0000 7f4d6aa00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d6c0b7890 0x7f4d6c0c6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:53.513 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:53.511+0000 7f4d6aa00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d6c0b7890 0x7f4d6c0c6990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49426/0 (socket says 172.21.15.152:49426) 2024-04-20T11:34:53.513 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:53.511+0000 7f4d6aa00640 1 -- 172.21.15.152:0/1411220452 learned_addr learned my addr 172.21.15.152:0/1411220452 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:53.513 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:53.512+0000 7f4d6a000640 1 --2- 172.21.15.152:0/1411220452 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d6c0cf4f0 0x7f4d6c1608f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:53.513 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:53.512+0000 7f4d6b400640 1 --2- 172.21.15.152:0/1411220452 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4d6c161e50 0x7f4d6c160e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:53.513 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:53.512+0000 7f4d6aa00640 1 -- 172.21.15.152:0/1411220452 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4d6c161e50 msgr2=0x7f4d6c160e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:53.513 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:53.512+0000 7f4d6aa00640 1 --2- 172.21.15.152:0/1411220452 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4d6c161e50 0x7f4d6c160e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:53.514 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:53.512+0000 7f4d6aa00640 1 -- 172.21.15.152:0/1411220452 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d6c0cf4f0 msgr2=0x7f4d6c1608f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:53.514 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:53.512+0000 7f4d6aa00640 1 --2- 172.21.15.152:0/1411220452 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d6c0cf4f0 0x7f4d6c1608f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:53.514 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:53.512+0000 7f4d6aa00640 1 -- 172.21.15.152:0/1411220452 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d6c0c9e00 con 0x7f4d6c0b7890 2024-04-20T11:34:53.514 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:53.512+0000 7f4d6b400640 1 --2- 172.21.15.152:0/1411220452 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4d6c161e50 0x7f4d6c160e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:34:53.514 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:53.512+0000 7f4d6aa00640 1 --2- 172.21.15.152:0/1411220452 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d6c0b7890 0x7f4d6c0c6990 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c00c760 tx=0x7f4d5c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:53.514 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:53.512+0000 7f4d63400640 1 -- 172.21.15.152:0/1411220452 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d5c016020 con 0x7f4d6c0b7890 2024-04-20T11:34:53.514 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:53.512+0000 7f4d63400640 1 -- 172.21.15.152:0/1411220452 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d5c006d50 con 0x7f4d6c0b7890 2024-04-20T11:34:53.514 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:53.513+0000 7f4d71e00640 1 -- 172.21.15.152:0/1411220452 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d6c0ca0e0 con 0x7f4d6c0b7890 2024-04-20T11:34:53.514 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:53.513+0000 7f4d63400640 1 -- 172.21.15.152:0/1411220452 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d5c003bb0 con 0x7f4d6c0b7890 2024-04-20T11:34:53.514 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:53.513+0000 7f4d71e00640 1 -- 172.21.15.152:0/1411220452 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d6c0ca6a0 con 0x7f4d6c0b7890 2024-04-20T11:34:53.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:53.514+0000 7f4d71e00640 1 -- 172.21.15.152:0/1411220452 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d6c0689f0 con 0x7f4d6c0b7890 2024-04-20T11:34:53.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:53.515+0000 7f4d63400640 1 -- 172.21.15.152:0/1411220452 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 20) ==== 91448+0+0 (secure 0 0 0) 0x7f4d5c00a850 con 0x7f4d6c0b7890 2024-04-20T11:34:53.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:53.516+0000 7f4d63400640 1 --2- 172.21.15.152:0/1411220452 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4d4806cd70 0x7f4d4806f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:53.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:53.516+0000 7f4d63400640 1 -- 172.21.15.152:0/1411220452 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5709+0+0 (secure 0 0 0) 0x7f4d5c08bfe0 con 0x7f4d6c0b7890 2024-04-20T11:34:53.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:53.517+0000 7f4d6a000640 1 --2- 172.21.15.152:0/1411220452 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4d4806cd70 0x7f4d4806f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:53.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:53.518+0000 7f4d6a000640 1 --2- 172.21.15.152:0/1411220452 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4d4806cd70 0x7f4d4806f230 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f4d54036000 tx=0x7f4d54008030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:53.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:53.520+0000 7f4d63400640 1 -- 172.21.15.152:0/1411220452 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f4d5c020020 con 0x7f4d6c0b7890 2024-04-20T11:34:53.751 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:53.751+0000 7f4d71e00640 1 -- 172.21.15.152:0/1411220452 --> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol36640", "target": ["mon-mgr", ""]}) -- 0x7f4d6c0d0650 con 0x7f4d4806cd70 2024-04-20T11:34:53.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:53.855+0000 7f4d63400640 1 -- 172.21.15.152:0/1411220452 <== mgr.4109 v2:172.21.15.175:6836/4021250441 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4d6c0d0650 con 0x7f4d4806cd70 2024-04-20T11:34:53.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:53.859+0000 7f4d71e00640 1 -- 172.21.15.152:0/1411220452 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4d4806cd70 msgr2=0x7f4d4806f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:53.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:53.859+0000 7f4d71e00640 1 --2- 172.21.15.152:0/1411220452 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4d4806cd70 0x7f4d4806f230 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f4d54036000 tx=0x7f4d54008030 comp rx=0 tx=0).stop 2024-04-20T11:34:53.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:53.859+0000 7f4d71e00640 1 -- 172.21.15.152:0/1411220452 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d6c0b7890 msgr2=0x7f4d6c0c6990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:53.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:53.859+0000 7f4d71e00640 1 --2- 172.21.15.152:0/1411220452 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d6c0b7890 0x7f4d6c0c6990 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c00c760 tx=0x7f4d5c00cc20 comp rx=0 tx=0).stop 2024-04-20T11:34:53.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:53.859+0000 7f4d71e00640 1 -- 172.21.15.152:0/1411220452 shutdown_connections 2024-04-20T11:34:53.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:53.859+0000 7f4d71e00640 1 --2- 172.21.15.152:0/1411220452 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4d4806cd70 0x7f4d4806f230 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:53.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:53.859+0000 7f4d71e00640 1 --2- 172.21.15.152:0/1411220452 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4d6c161e50 0x7f4d6c160e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:53.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:53.859+0000 7f4d71e00640 1 --2- 172.21.15.152:0/1411220452 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d6c0cf4f0 0x7f4d6c1608f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:53.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:53.859+0000 7f4d71e00640 1 --2- 172.21.15.152:0/1411220452 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d6c0b7890 0x7f4d6c0c6990 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:53.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:53.859+0000 7f4d71e00640 1 -- 172.21.15.152:0/1411220452 >> 172.21.15.152:0/1411220452 conn(0x7f4d6c067ae0 msgr2=0x7f4d6c15c630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:53.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:53.859+0000 7f4d71e00640 1 -- 172.21.15.152:0/1411220452 shutdown_connections 2024-04-20T11:34:53.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:53.859+0000 7f4d71e00640 1 -- 172.21.15.152:0/1411220452 wait complete. 2024-04-20T11:34:53.872 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:34:53.873+0000 7f8a5fe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:34:53.872 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:34:53.873+0000 7f8a5fe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:34:53.872 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:34:53.873+0000 7f8a5fe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:34:53.872 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:34:53.873+0000 7f8a5fe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:34:53.872 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:34:53.873+0000 7f8a5fe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:34:53.880 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol43081 subvol_grp47320 2024-04-20T11:34:53.902 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:34:53.903+0000 7f8a5f400640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:34:53.902 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:34:53.903+0000 7f8a5f400640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:34:53.902 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:34:53.903+0000 7f8a5f400640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:34:53.902 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:34:53.903+0000 7f8a5f400640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:34:53.902 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:34:53.903+0000 7f8a5f400640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:34:54.077 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.077+0000 7f2341800640 1 Processor -- start 2024-04-20T11:34:54.077 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.077+0000 7f2341800640 1 -- start start 2024-04-20T11:34:54.078 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.078+0000 7f2341800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f233c0d22d0 0x7f233c0cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:54.079 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.078+0000 7f2341800640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f233c0d40e0 con 0x7f233c0cfa10 2024-04-20T11:34:54.079 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.078+0000 7f2341800640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f233c0d43c0 con 0x7f233c15e910 2024-04-20T11:34:54.079 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.078+0000 7f2341800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f233c0d46a0 con 0x7f233c0d22d0 2024-04-20T11:34:54.079 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.079+0000 7f233b400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f233c0cfa10 0x7f233c0b78c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:51008/0 (socket says 172.21.15.152:51008) 2024-04-20T11:34:54.079 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.079+0000 7f233b400640 1 -- 172.21.15.152:0/970589751 learned_addr learned my addr 172.21.15.152:0/970589751 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:34:54.079 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.079+0000 7f233a000640 1 --2- 172.21.15.152:0/970589751 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f233c0d22d0 0x7f233c0cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:54.079 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.079+0000 7f2339600640 1 -- 172.21.15.152:0/970589751 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1164507482 0 0) 0x7f233c0d40e0 con 0x7f233c0cfa10 2024-04-20T11:34:54.079 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.079+0000 7f2339600640 1 -- 172.21.15.152:0/970589751 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2320003520 con 0x7f233c0cfa10 2024-04-20T11:34:54.081 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.080+0000 7f2339600640 1 -- 172.21.15.152:0/970589751 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3119990486 0 0) 0x7f2320003520 con 0x7f233c0cfa10 2024-04-20T11:34:54.081 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.080+0000 7f2339600640 1 -- 172.21.15.152:0/970589751 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f233c0d22d0 msgr2=0x7f233c0cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:54.081 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.080+0000 7f2339600640 1 --2- 172.21.15.152:0/970589751 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f233c0d22d0 0x7f233c0cf4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:54.081 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.080+0000 7f2339600640 1 -- 172.21.15.152:0/970589751 >> v1:172.21.15.175:6789/0 conn(0x7f233c15e910 legacy=0x7f233c0d3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:54.081 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.080+0000 7f2339600640 1 -- 172.21.15.152:0/970589751 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f233c0d4980 con 0x7f233c0cfa10 2024-04-20T11:34:54.081 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.080+0000 7f2339600640 1 -- 172.21.15.152:0/970589751 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f23340019f0 con 0x7f233c0cfa10 2024-04-20T11:34:54.081 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.080+0000 7f2339600640 1 -- 172.21.15.152:0/970589751 >> v1:172.21.15.152:6789/0 conn(0x7f233c0cfa10 legacy=0x7f233c0b78c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:54.081 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.080+0000 7f2339600640 1 --2- 172.21.15.152:0/970589751 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f233c0d22d0 0x7f2320003a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:54.081 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.080+0000 7f2339600640 1 --2- 172.21.15.152:0/970589751 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f23200040d0 0x7f2320004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:54.081 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.080+0000 7f2339600640 1 --2- 172.21.15.152:0/970589751 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2320004db0 0x7f2320005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:54.081 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.080+0000 7f2339600640 1 -- 172.21.15.152:0/970589751 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f233c0d46a0 con 0x7f23200040d0 2024-04-20T11:34:54.081 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.080+0000 7f2339600640 1 -- 172.21.15.152:0/970589751 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f232000bc20 con 0x7f2320004db0 2024-04-20T11:34:54.082 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.080+0000 7f2339600640 1 -- 172.21.15.152:0/970589751 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f232000be20 con 0x7f233c0d22d0 2024-04-20T11:34:54.082 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.080+0000 7f233a000640 1 --2- 172.21.15.152:0/970589751 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f23200040d0 0x7f2320004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:54.082 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.080+0000 7f233a000640 1 -- 172.21.15.152:0/970589751 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f233c0d22d0 msgr2=0x7f2320003a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:54.082 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.080+0000 7f233a000640 1 --2- 172.21.15.152:0/970589751 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f233c0d22d0 0x7f2320003a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:54.082 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.080+0000 7f233a000640 1 -- 172.21.15.152:0/970589751 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2320004db0 msgr2=0x7f2320005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:54.082 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.080+0000 7f233a000640 1 --2- 172.21.15.152:0/970589751 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2320004db0 0x7f2320005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:54.082 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.080+0000 7f233a000640 1 -- 172.21.15.152:0/970589751 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2320003520 con 0x7f23200040d0 2024-04-20T11:34:54.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.082+0000 7f233a000640 1 --2- 172.21.15.152:0/970589751 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f23200040d0 0x7f2320004800 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f232400ba30 tx=0x7f2320005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3ac31ef8d5953c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:54.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.082+0000 7f2339600640 1 -- 172.21.15.152:0/970589751 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2324036020 con 0x7f23200040d0 2024-04-20T11:34:54.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.082+0000 7f2339600640 1 -- 172.21.15.152:0/970589751 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2324032680 con 0x7f23200040d0 2024-04-20T11:34:54.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.082+0000 7f2339600640 1 -- 172.21.15.152:0/970589751 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f232400b600 con 0x7f23200040d0 2024-04-20T11:34:54.084 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.083+0000 7f2341800640 1 -- 172.21.15.152:0/970589751 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f23200040d0 msgr2=0x7f2320004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:54.084 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.083+0000 7f2341800640 1 --2- 172.21.15.152:0/970589751 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f23200040d0 0x7f2320004800 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f232400ba30 tx=0x7f2320005830 comp rx=0 tx=0).stop 2024-04-20T11:34:54.084 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.083+0000 7f233b400640 1 -- 172.21.15.152:0/970589751 reap_dead start 2024-04-20T11:34:54.084 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.083+0000 7f2341800640 1 -- 172.21.15.152:0/970589751 shutdown_connections 2024-04-20T11:34:54.084 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.083+0000 7f2341800640 1 -- 172.21.15.152:0/970589751 >> 172.21.15.152:0/970589751 conn(0x7f233c067ae0 msgr2=0x7f233c067ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:54.084 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.083+0000 7f2341800640 1 -- 172.21.15.152:0/970589751 shutdown_connections 2024-04-20T11:34:54.084 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.083+0000 7f2341800640 1 -- 172.21.15.152:0/970589751 wait complete. 2024-04-20T11:34:54.086 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.084+0000 7f2341800640 1 Processor -- start 2024-04-20T11:34:54.086 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.085+0000 7f2341800640 1 -- start start 2024-04-20T11:34:54.086 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.085+0000 7f2341800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f23200040d0 0x7f233c1f51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:54.086 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.086+0000 7f2341800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f233c0ca8b0 0x7f233c161750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:54.086 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.086+0000 7f2341800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f233c15f7e0 0x7f233c15fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:54.086 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.086+0000 7f2341800640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f233c15ecf0 con 0x7f233c0ca8b0 2024-04-20T11:34:54.086 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.086+0000 7f2341800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f233c0cfe60 con 0x7f23200040d0 2024-04-20T11:34:54.086 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.086+0000 7f2341800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f233c0cf4d0 con 0x7f233c15f7e0 2024-04-20T11:34:54.086 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.086+0000 7f233a000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f233c0ca8b0 0x7f233c161750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:54.087 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.086+0000 7f233a000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f233c0ca8b0 0x7f233c161750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49432/0 (socket says 172.21.15.152:49432) 2024-04-20T11:34:54.087 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.086+0000 7f233a000640 1 -- 172.21.15.152:0/4225171616 learned_addr learned my addr 172.21.15.152:0/4225171616 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:54.087 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.086+0000 7f233a000640 1 -- 172.21.15.152:0/4225171616 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f233c15f7e0 msgr2=0x7f233c15fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:54.087 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.086+0000 7f233a000640 1 --2- 172.21.15.152:0/4225171616 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f233c15f7e0 0x7f233c15fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:54.087 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.086+0000 7f233a000640 1 -- 172.21.15.152:0/4225171616 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f23200040d0 msgr2=0x7f233c1f51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:54.087 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.086+0000 7f233a000640 1 --2- 172.21.15.152:0/4225171616 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f23200040d0 0x7f233c1f51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:54.087 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.086+0000 7f233a000640 1 -- 172.21.15.152:0/4225171616 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f233c1f56e0 con 0x7f233c0ca8b0 2024-04-20T11:34:54.087 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.087+0000 7f233a000640 1 --2- 172.21.15.152:0/4225171616 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f233c0ca8b0 0x7f233c161750 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f232400bb50 tx=0x7f2324004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:54.088 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.087+0000 7f2332a00640 1 -- 172.21.15.152:0/4225171616 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2324041070 con 0x7f233c0ca8b0 2024-04-20T11:34:54.088 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.087+0000 7f2332a00640 1 -- 172.21.15.152:0/4225171616 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2324030a90 con 0x7f233c0ca8b0 2024-04-20T11:34:54.088 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.087+0000 7f2341800640 1 -- 172.21.15.152:0/4225171616 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f233c15ee70 con 0x7f233c0ca8b0 2024-04-20T11:34:54.088 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.087+0000 7f2332a00640 1 -- 172.21.15.152:0/4225171616 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2324036040 con 0x7f233c0ca8b0 2024-04-20T11:34:54.088 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.087+0000 7f2341800640 1 -- 172.21.15.152:0/4225171616 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f233c160c70 con 0x7f233c0ca8b0 2024-04-20T11:34:54.088 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.088+0000 7f2341800640 1 -- 172.21.15.152:0/4225171616 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f233c068a60 con 0x7f233c0ca8b0 2024-04-20T11:34:54.089 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.089+0000 7f2332a00640 1 -- 172.21.15.152:0/4225171616 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 20) ==== 91448+0+0 (secure 0 0 0) 0x7f2324039020 con 0x7f233c0ca8b0 2024-04-20T11:34:54.089 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.090+0000 7f2332a00640 1 --2- 172.21.15.152:0/4225171616 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f231806cd70 0x7f231806f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:54.089 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.090+0000 7f2332a00640 1 -- 172.21.15.152:0/4225171616 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5709+0+0 (secure 0 0 0) 0x7f23240ae900 con 0x7f233c0ca8b0 2024-04-20T11:34:54.090 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.091+0000 7f233aa00640 1 --2- 172.21.15.152:0/4225171616 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f231806cd70 0x7f231806f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:54.091 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.092+0000 7f233aa00640 1 --2- 172.21.15.152:0/4225171616 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f231806cd70 0x7f231806f230 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f232c001600 tx=0x7f232c007990 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:54.096 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.097+0000 7f2332a00640 1 -- 172.21.15.152:0/4225171616 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f233c1f56e0 con 0x7f233c0ca8b0 2024-04-20T11:34:54.327 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.328+0000 7f2341800640 1 -- 172.21.15.152:0/4225171616 --> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol43081", "group_name": "subvol_grp47320", "target": ["mon-mgr", ""]}) -- 0x7f233c068090 con 0x7f231806cd70 2024-04-20T11:34:54.422 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.422+0000 7f2332a00640 1 -- 172.21.15.152:0/4225171616 <== mgr.4109 v2:172.21.15.175:6836/4021250441 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f233c068090 con 0x7f231806cd70 2024-04-20T11:34:54.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.426+0000 7f2341800640 1 -- 172.21.15.152:0/4225171616 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f231806cd70 msgr2=0x7f231806f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:54.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.426+0000 7f2341800640 1 --2- 172.21.15.152:0/4225171616 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f231806cd70 0x7f231806f230 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f232c001600 tx=0x7f232c007990 comp rx=0 tx=0).stop 2024-04-20T11:34:54.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.426+0000 7f2341800640 1 -- 172.21.15.152:0/4225171616 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f233c0ca8b0 msgr2=0x7f233c161750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:54.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.426+0000 7f2341800640 1 --2- 172.21.15.152:0/4225171616 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f233c0ca8b0 0x7f233c161750 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f232400bb50 tx=0x7f2324004730 comp rx=0 tx=0).stop 2024-04-20T11:34:54.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.426+0000 7f2341800640 1 -- 172.21.15.152:0/4225171616 shutdown_connections 2024-04-20T11:34:54.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.426+0000 7f2341800640 1 --2- 172.21.15.152:0/4225171616 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f231806cd70 0x7f231806f230 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:54.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.426+0000 7f2341800640 1 --2- 172.21.15.152:0/4225171616 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f233c15f7e0 0x7f233c15fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:54.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.426+0000 7f2341800640 1 --2- 172.21.15.152:0/4225171616 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f233c0ca8b0 0x7f233c161750 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:54.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.426+0000 7f2341800640 1 --2- 172.21.15.152:0/4225171616 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f23200040d0 0x7f233c1f51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:54.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.426+0000 7f2341800640 1 -- 172.21.15.152:0/4225171616 >> 172.21.15.152:0/4225171616 conn(0x7f233c067ae0 msgr2=0x7f233c0cc6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:54.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.426+0000 7f2341800640 1 -- 172.21.15.152:0/4225171616 shutdown_connections 2024-04-20T11:34:54.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.426+0000 7f2341800640 1 -- 172.21.15.152:0/4225171616 wait complete. 2024-04-20T11:34:54.446 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol90142 2024-04-20T11:34:54.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.641+0000 7fbd2b200640 1 Processor -- start 2024-04-20T11:34:54.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.642+0000 7fbd2b200640 1 -- start start 2024-04-20T11:34:54.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.643+0000 7fbd2b200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fbd2c15eb50 0x7fbd2c0d0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:54.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.643+0000 7fbd2b200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd2c163230 0x7fbd2c0d2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:54.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.643+0000 7fbd2b200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fbd2c0d1440 0x7fbd2c0d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:54.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.643+0000 7fbd2b200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fbd2c0d19c0 con 0x7fbd2c15eb50 2024-04-20T11:34:54.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.643+0000 7fbd2b200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fbd2c162750 con 0x7fbd2c0d1440 2024-04-20T11:34:54.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.643+0000 7fbd2b200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fbd2c162a30 con 0x7fbd2c163230 2024-04-20T11:34:54.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.643+0000 7fbd29e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fbd2c15eb50 0x7fbd2c0d0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:54.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.643+0000 7fbd29e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fbd2c15eb50 0x7fbd2c0d0f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49434/0 (socket says 172.21.15.152:49434) 2024-04-20T11:34:54.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.643+0000 7fbd29e00640 1 -- 172.21.15.152:0/3279418150 learned_addr learned my addr 172.21.15.152:0/3279418150 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:54.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.644+0000 7fbd29e00640 1 -- 172.21.15.152:0/3279418150 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd2c163230 msgr2=0x7fbd2c0d2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:54.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.644+0000 7fbd29e00640 1 --2- 172.21.15.152:0/3279418150 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd2c163230 0x7fbd2c0d2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:54.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.644+0000 7fbd29e00640 1 -- 172.21.15.152:0/3279418150 >> v2:172.21.15.175:3300/0 conn(0x7fbd2c0d1440 msgr2=0x7fbd2c0d33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:54.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.644+0000 7fbd29e00640 1 --2- 172.21.15.152:0/3279418150 >> v2:172.21.15.175:3300/0 conn(0x7fbd2c0d1440 0x7fbd2c0d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:54.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.644+0000 7fbd29e00640 1 -- 172.21.15.152:0/3279418150 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd2c162d10 con 0x7fbd2c15eb50 2024-04-20T11:34:54.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.645+0000 7fbd29e00640 1 --2- 172.21.15.152:0/3279418150 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd2c15eb50 0x7fbd2c0d0f00 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c0d1ff0 tx=0x7fbd1c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9724779359ef319 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:54.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.645+0000 7fbd28a00640 1 -- 172.21.15.152:0/3279418150 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd1c02e070 con 0x7fbd2c15eb50 2024-04-20T11:34:54.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.645+0000 7fbd28a00640 1 -- 172.21.15.152:0/3279418150 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd1c037070 con 0x7fbd2c15eb50 2024-04-20T11:34:54.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.645+0000 7fbd28a00640 1 -- 172.21.15.152:0/3279418150 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd1c031a00 con 0x7fbd2c15eb50 2024-04-20T11:34:54.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.646+0000 7fbd2b200640 1 -- 172.21.15.152:0/3279418150 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd2c15eb50 msgr2=0x7fbd2c0d0f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:54.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.646+0000 7fbd2b200640 1 --2- 172.21.15.152:0/3279418150 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd2c15eb50 0x7fbd2c0d0f00 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c0d1ff0 tx=0x7fbd1c02b880 comp rx=0 tx=0).stop 2024-04-20T11:34:54.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.646+0000 7fbd2b200640 1 -- 172.21.15.152:0/3279418150 shutdown_connections 2024-04-20T11:34:54.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.646+0000 7fbd2b200640 1 --2- 172.21.15.152:0/3279418150 >> v2:172.21.15.175:3300/0 conn(0x7fbd2c0d1440 0x7fbd2c0d33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:54.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.646+0000 7fbd2b200640 1 --2- 172.21.15.152:0/3279418150 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd2c163230 0x7fbd2c0d2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:54.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.646+0000 7fbd2b200640 1 --2- 172.21.15.152:0/3279418150 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd2c15eb50 0x7fbd2c0d0f00 unknown :-1 s=CLOSED pgs=1636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:54.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.646+0000 7fbd2b200640 1 -- 172.21.15.152:0/3279418150 >> 172.21.15.152:0/3279418150 conn(0x7fbd2c08bfd0 msgr2=0x7fbd2c0d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:54.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.646+0000 7fbd2b200640 1 -- 172.21.15.152:0/3279418150 shutdown_connections 2024-04-20T11:34:54.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.646+0000 7fbd2b200640 1 -- 172.21.15.152:0/3279418150 wait complete. 2024-04-20T11:34:54.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.648+0000 7fbd2b200640 1 Processor -- start 2024-04-20T11:34:54.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.648+0000 7fbd2b200640 1 -- start start 2024-04-20T11:34:54.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.649+0000 7fbd2b200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd2c0d1440 0x7fbd2c1563e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:54.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.649+0000 7fbd2b200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd2c15eb50 0x7fbd2c152380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:54.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.649+0000 7fbd2b200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd2c163230 0x7fbd2c0cc030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:54.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.649+0000 7fbd2b200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fbd2c15f0b0 con 0x7fbd2c0d1440 2024-04-20T11:34:54.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.649+0000 7fbd2b200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbd2c15ef30 con 0x7fbd2c15eb50 2024-04-20T11:34:54.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.649+0000 7fbd2b200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fbd2c15f230 con 0x7fbd2c163230 2024-04-20T11:34:54.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.649+0000 7fbd29e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd2c0d1440 0x7fbd2c1563e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:54.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.649+0000 7fbd29e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd2c0d1440 0x7fbd2c1563e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49436/0 (socket says 172.21.15.152:49436) 2024-04-20T11:34:54.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.649+0000 7fbd29e00640 1 -- 172.21.15.152:0/2859973975 learned_addr learned my addr 172.21.15.152:0/2859973975 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:54.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.650+0000 7fbd29e00640 1 -- 172.21.15.152:0/2859973975 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd2c163230 msgr2=0x7fbd2c0cc030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:54.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.650+0000 7fbd29e00640 1 --2- 172.21.15.152:0/2859973975 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd2c163230 0x7fbd2c0cc030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:54.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.650+0000 7fbd29e00640 1 -- 172.21.15.152:0/2859973975 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd2c15eb50 msgr2=0x7fbd2c152380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:54.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.650+0000 7fbd29e00640 1 --2- 172.21.15.152:0/2859973975 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd2c15eb50 0x7fbd2c152380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:54.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.650+0000 7fbd29e00640 1 -- 172.21.15.152:0/2859973975 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd2c0cb8a0 con 0x7fbd2c0d1440 2024-04-20T11:34:54.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.650+0000 7fbd29e00640 1 --2- 172.21.15.152:0/2859973975 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd2c0d1440 0x7fbd2c1563e0 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c000c00 tx=0x7fbd1c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:54.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.650+0000 7fbd22000640 1 -- 172.21.15.152:0/2859973975 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd1c032880 con 0x7fbd2c0d1440 2024-04-20T11:34:54.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.650+0000 7fbd22000640 1 -- 172.21.15.152:0/2859973975 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd1c032a20 con 0x7fbd2c0d1440 2024-04-20T11:34:54.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.650+0000 7fbd2b200640 1 -- 172.21.15.152:0/2859973975 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd2c0cbb80 con 0x7fbd2c0d1440 2024-04-20T11:34:54.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.650+0000 7fbd22000640 1 -- 172.21.15.152:0/2859973975 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd1c032e10 con 0x7fbd2c0d1440 2024-04-20T11:34:54.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.650+0000 7fbd2b200640 1 -- 172.21.15.152:0/2859973975 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd2c159750 con 0x7fbd2c0d1440 2024-04-20T11:34:54.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.651+0000 7fbd2b200640 1 -- 172.21.15.152:0/2859973975 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd2c0cb630 con 0x7fbd2c0d1440 2024-04-20T11:34:54.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.653+0000 7fbd22000640 1 -- 172.21.15.152:0/2859973975 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 20) ==== 91448+0+0 (secure 0 0 0) 0x7fbd1c04c060 con 0x7fbd2c0d1440 2024-04-20T11:34:54.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.654+0000 7fbd22000640 1 --2- 172.21.15.152:0/2859973975 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fbd0806cd70 0x7fbd0806f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:54.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.654+0000 7fbd22000640 1 -- 172.21.15.152:0/2859973975 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5709+0+0 (secure 0 0 0) 0x7fbd1c0ae1b0 con 0x7fbd2c0d1440 2024-04-20T11:34:54.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.654+0000 7fbd29400640 1 --2- 172.21.15.152:0/2859973975 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fbd0806cd70 0x7fbd0806f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:54.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.655+0000 7fbd29400640 1 --2- 172.21.15.152:0/2859973975 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fbd0806cd70 0x7fbd0806f230 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c0ce660 tx=0x7fbd14009250 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:54.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.658+0000 7fbd22000640 1 -- 172.21.15.152:0/2859973975 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fbd1c040070 con 0x7fbd2c0d1440 2024-04-20T11:34:54.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.892+0000 7fbd2b200640 1 -- 172.21.15.152:0/2859973975 --> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol90142", "target": ["mon-mgr", ""]}) -- 0x7fbd2c075e60 con 0x7fbd0806cd70 2024-04-20T11:34:54.913 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:34:54.914+0000 7f8a5cc00640 -1 mgr.server reply reply (11) Resource temporarily unavailable subvolume 'subvol90142' is not ready for operation rm (use --force to override) 2024-04-20T11:34:54.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:54.914+0000 7fbd22000640 1 -- 172.21.15.152:0/2859973975 <== mgr.4109 v2:172.21.15.175:6836/4021250441 1 ==== mgr_command_reply(tid 0: -11 subvolume 'subvol90142' is not ready for operation rm (use --force to override)) ==== 87+0+0 (secure 0 0 0) 0x7fbd2c075e60 con 0x7fbd0806cd70 2024-04-20T11:34:54.914 INFO:teuthology.orchestra.run.smithi152.stderr:Error EAGAIN: subvolume 'subvol90142' is not ready for operation rm (use --force to override) 2024-04-20T11:34:54.928 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-04-20T11:34:54.929 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:34:54.929 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_nogroup/subvol90142/.meta) 2024-04-20T11:34:55.015 INFO:teuthology.orchestra.run.smithi152.stdout:[GLOBAL] 2024-04-20T11:34:55.015 INFO:teuthology.orchestra.run.smithi152.stdout:version = 1 2024-04-20T11:34:55.015 INFO:teuthology.orchestra.run.smithi152.stdout:type = clone 2024-04-20T11:34:55.015 INFO:teuthology.orchestra.run.smithi152.stdout:path = /volumes/_nogroup/subvol90142/e2f77d43-a542-4f2a-beaa-47d82b3daced 2024-04-20T11:34:55.015 INFO:teuthology.orchestra.run.smithi152.stdout:state = pending 2024-04-20T11:34:55.015 INFO:teuthology.orchestra.run.smithi152.stdout:[source] 2024-04-20T11:34:55.015 INFO:teuthology.orchestra.run.smithi152.stdout:volume = fake 2024-04-20T11:34:55.015 INFO:teuthology.orchestra.run.smithi152.stdout:subvolume = fake 2024-04-20T11:34:55.015 INFO:teuthology.orchestra.run.smithi152.stdout:snapshot = fake 2024-04-20T11:34:55.019 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol90142 --force 2024-04-20T11:34:55.227 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.227+0000 7f08ee200640 1 Processor -- start 2024-04-20T11:34:55.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.227+0000 7f08ee200640 1 -- start start 2024-04-20T11:34:55.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.228+0000 7f08ee200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f08e8072530 0x7f08e8160db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:55.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.228+0000 7f08ee200640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08e80b4f90 con 0x7f08e80b47a0 2024-04-20T11:34:55.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.229+0000 7f08ee200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08e80b4cb0 con 0x7f08e8145340 2024-04-20T11:34:55.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.229+0000 7f08ee200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f08e8145720 con 0x7f08e8072530 2024-04-20T11:34:55.230 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.229+0000 7f08e7e00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f08e80b47a0 0x7f08e815cbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:51022/0 (socket says 172.21.15.152:51022) 2024-04-20T11:34:55.230 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.229+0000 7f08e7e00640 1 -- 172.21.15.152:0/2808908827 learned_addr learned my addr 172.21.15.152:0/2808908827 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:34:55.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.229+0000 7f08e6000640 1 -- 172.21.15.152:0/2808908827 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2321614144 0 0) 0x7f08e80b4f90 con 0x7f08e80b47a0 2024-04-20T11:34:55.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.229+0000 7f08e6000640 1 -- 172.21.15.152:0/2808908827 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f08cc003520 con 0x7f08e80b47a0 2024-04-20T11:34:55.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.229+0000 7f08e6a00640 1 --2- 172.21.15.152:0/2808908827 >> v2:172.21.15.175:3300/0 conn(0x7f08e8072530 0x7f08e8160db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:55.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.230+0000 7f08e6000640 1 -- 172.21.15.152:0/2808908827 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4228826474 0 0) 0x7f08cc003520 con 0x7f08e80b47a0 2024-04-20T11:34:55.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.230+0000 7f08e6000640 1 -- 172.21.15.152:0/2808908827 >> v2:172.21.15.175:3300/0 conn(0x7f08e8072530 msgr2=0x7f08e8160db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:55.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.230+0000 7f08e6000640 1 --2- 172.21.15.152:0/2808908827 >> v2:172.21.15.175:3300/0 conn(0x7f08e8072530 0x7f08e8160db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:55.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.230+0000 7f08e6000640 1 -- 172.21.15.152:0/2808908827 >> v1:172.21.15.175:6789/0 conn(0x7f08e8145340 legacy=0x7f08e8071e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:55.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.230+0000 7f08e6000640 1 -- 172.21.15.152:0/2808908827 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08e81602d0 con 0x7f08e80b47a0 2024-04-20T11:34:55.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.230+0000 7f08e6000640 1 -- 172.21.15.152:0/2808908827 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f08e00019f0 con 0x7f08e80b47a0 2024-04-20T11:34:55.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.230+0000 7f08e6000640 1 -- 172.21.15.152:0/2808908827 >> v1:172.21.15.152:6789/0 conn(0x7f08e80b47a0 legacy=0x7f08e815cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:55.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.230+0000 7f08e6000640 1 --2- 172.21.15.152:0/2808908827 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f08cc003a20 0x7f08cc003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:55.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.230+0000 7f08e6000640 1 --2- 172.21.15.152:0/2808908827 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f08cc004800 0x7f08cc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:55.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.230+0000 7f08e6000640 1 --2- 172.21.15.152:0/2808908827 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f08cc005370 0x7f08cc00b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:55.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.230+0000 7f08e6000640 1 -- 172.21.15.152:0/2808908827 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f08e8145720 con 0x7f08cc003a20 2024-04-20T11:34:55.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.230+0000 7f08e6000640 1 -- 172.21.15.152:0/2808908827 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f08cc00c2c0 con 0x7f08cc004800 2024-04-20T11:34:55.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.230+0000 7f08e6000640 1 -- 172.21.15.152:0/2808908827 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f08cc00c4c0 con 0x7f08cc005370 2024-04-20T11:34:55.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.230+0000 7f08e7400640 1 --2- 172.21.15.152:0/2808908827 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f08cc003a20 0x7f08cc003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:55.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.231+0000 7f08e6a00640 1 --2- 172.21.15.152:0/2808908827 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f08cc004800 0x7f08cc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:55.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.231+0000 7f08e7400640 1 -- 172.21.15.152:0/2808908827 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f08cc005370 msgr2=0x7f08cc00b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:55.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.231+0000 7f08e7e00640 1 --2- 172.21.15.152:0/2808908827 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f08cc005370 0x7f08cc00b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:55.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.231+0000 7f08e7400640 1 --2- 172.21.15.152:0/2808908827 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f08cc005370 0x7f08cc00b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:55.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.231+0000 7f08e7400640 1 -- 172.21.15.152:0/2808908827 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f08cc004800 msgr2=0x7f08cc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:55.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.231+0000 7f08e7400640 1 --2- 172.21.15.152:0/2808908827 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f08cc004800 0x7f08cc004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:55.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.231+0000 7f08e7400640 1 -- 172.21.15.152:0/2808908827 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08cc003520 con 0x7f08cc003a20 2024-04-20T11:34:55.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.231+0000 7f08e7e00640 1 --2- 172.21.15.152:0/2808908827 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f08cc005370 0x7f08cc00b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:34:55.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.231+0000 7f08e7e00640 1 -- 172.21.15.152:0/2808908827 reap_dead start 2024-04-20T11:34:55.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.232+0000 7f08e7400640 1 --2- 172.21.15.152:0/2808908827 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f08cc003a20 0x7f08cc003e40 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f08d8001750 tx=0x7f08cc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abed3baa981e3c54 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:55.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.232+0000 7f08e6000640 1 -- 172.21.15.152:0/2808908827 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d8001370 con 0x7f08cc003a20 2024-04-20T11:34:55.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.232+0000 7f08e6000640 1 -- 172.21.15.152:0/2808908827 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08d80052d0 con 0x7f08cc003a20 2024-04-20T11:34:55.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.232+0000 7f08e6000640 1 -- 172.21.15.152:0/2808908827 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d8003830 con 0x7f08cc003a20 2024-04-20T11:34:55.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.233+0000 7f08ee200640 1 -- 172.21.15.152:0/2808908827 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f08cc003a20 msgr2=0x7f08cc003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:55.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.233+0000 7f08ee200640 1 --2- 172.21.15.152:0/2808908827 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f08cc003a20 0x7f08cc003e40 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f08d8001750 tx=0x7f08cc005830 comp rx=0 tx=0).stop 2024-04-20T11:34:55.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.233+0000 7f08ee200640 1 -- 172.21.15.152:0/2808908827 shutdown_connections 2024-04-20T11:34:55.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.233+0000 7f08ee200640 1 --2- 172.21.15.152:0/2808908827 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f08cc003a20 0x7f08cc003e40 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:55.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.233+0000 7f08ee200640 1 -- 172.21.15.152:0/2808908827 >> 172.21.15.152:0/2808908827 conn(0x7f08e806b980 msgr2=0x7f08e806bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:55.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.233+0000 7f08ee200640 1 -- 172.21.15.152:0/2808908827 shutdown_connections 2024-04-20T11:34:55.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.233+0000 7f08ee200640 1 -- 172.21.15.152:0/2808908827 wait complete. 2024-04-20T11:34:55.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.235+0000 7f08ee200640 1 Processor -- start 2024-04-20T11:34:55.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.235+0000 7f08ee200640 1 -- start start 2024-04-20T11:34:55.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.236+0000 7f08ee200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f08cc003a20 0x7f08e815bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:55.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.236+0000 7f08ee200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f08e81e8c50 0x7f08e81e90d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:55.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.236+0000 7f08ee200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f08e81e6dd0 0x7f08e81e7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:55.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.236+0000 7f08ee200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f08e8160db0 con 0x7f08e81e6dd0 2024-04-20T11:34:55.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.236+0000 7f08ee200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f08e8146010 con 0x7f08e81e8c50 2024-04-20T11:34:55.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.236+0000 7f08ee200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f08e8160f30 con 0x7f08cc003a20 2024-04-20T11:34:55.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.236+0000 7f08e7e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f08e81e6dd0 0x7f08e81e7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:55.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.236+0000 7f08e7e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f08e81e6dd0 0x7f08e81e7250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49448/0 (socket says 172.21.15.152:49448) 2024-04-20T11:34:55.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.236+0000 7f08e7e00640 1 -- 172.21.15.152:0/814263456 learned_addr learned my addr 172.21.15.152:0/814263456 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:55.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.236+0000 7f08e7400640 1 --2- 172.21.15.152:0/814263456 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f08cc003a20 0x7f08e815bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:55.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.237+0000 7f08e6a00640 1 --2- 172.21.15.152:0/814263456 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f08e81e8c50 0x7f08e81e90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:55.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.237+0000 7f08e7e00640 1 -- 172.21.15.152:0/814263456 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f08cc003a20 msgr2=0x7f08e815bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:55.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.237+0000 7f08e7e00640 1 --2- 172.21.15.152:0/814263456 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f08cc003a20 0x7f08e815bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:55.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.237+0000 7f08e7e00640 1 -- 172.21.15.152:0/814263456 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f08e81e8c50 msgr2=0x7f08e81e90d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:55.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.237+0000 7f08e7e00640 1 --2- 172.21.15.152:0/814263456 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f08e81e8c50 0x7f08e81e90d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:55.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.237+0000 7f08e7e00640 1 -- 172.21.15.152:0/814263456 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08e81e7b50 con 0x7f08e81e6dd0 2024-04-20T11:34:55.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.237+0000 7f08e7e00640 1 --2- 172.21.15.152:0/814263456 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f08e81e6dd0 0x7f08e81e7250 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f08e80cfdf0 tx=0x7f08e000b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:55.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.237+0000 7f08df400640 1 -- 172.21.15.152:0/814263456 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08e0004d60 con 0x7f08e81e6dd0 2024-04-20T11:34:55.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.237+0000 7f08df400640 1 -- 172.21.15.152:0/814263456 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08e000bb00 con 0x7f08e81e6dd0 2024-04-20T11:34:55.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.237+0000 7f08ee200640 1 -- 172.21.15.152:0/814263456 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08e81e7e30 con 0x7f08e81e6dd0 2024-04-20T11:34:55.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.237+0000 7f08df400640 1 -- 172.21.15.152:0/814263456 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08e0007070 con 0x7f08e81e6dd0 2024-04-20T11:34:55.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.237+0000 7f08ee200640 1 -- 172.21.15.152:0/814263456 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08e81e83f0 con 0x7f08e81e6dd0 2024-04-20T11:34:55.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.239+0000 7f08ee200640 1 -- 172.21.15.152:0/814263456 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08e8068a60 con 0x7f08e81e6dd0 2024-04-20T11:34:55.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.240+0000 7f08df400640 1 -- 172.21.15.152:0/814263456 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 20) ==== 91448+0+0 (secure 0 0 0) 0x7f08e001b020 con 0x7f08e81e6dd0 2024-04-20T11:34:55.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.240+0000 7f08df400640 1 --2- 172.21.15.152:0/814263456 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f08c806cd70 0x7f08c806f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:55.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.241+0000 7f08df400640 1 -- 172.21.15.152:0/814263456 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5709+0+0 (secure 0 0 0) 0x7f08e008c540 con 0x7f08e81e6dd0 2024-04-20T11:34:55.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.241+0000 7f08e7400640 1 --2- 172.21.15.152:0/814263456 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f08c806cd70 0x7f08c806f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:55.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.242+0000 7f08e7400640 1 --2- 172.21.15.152:0/814263456 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f08c806cd70 0x7f08c806f230 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f08d8033000 tx=0x7f08d8009030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:55.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.245+0000 7f08df400640 1 -- 172.21.15.152:0/814263456 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f08e005a160 con 0x7f08e81e6dd0 2024-04-20T11:34:55.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.478+0000 7f08ee200640 1 -- 172.21.15.152:0/814263456 --> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol90142", "force": true, "target": ["mon-mgr", ""]}) -- 0x7f08e8071ea0 con 0x7f08c806cd70 2024-04-20T11:34:55.510 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.509+0000 7f08df400640 1 -- 172.21.15.152:0/814263456 <== mgr.4109 v2:172.21.15.175:6836/4021250441 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f08e8071ea0 con 0x7f08c806cd70 2024-04-20T11:34:55.514 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.513+0000 7f08ee200640 1 -- 172.21.15.152:0/814263456 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f08c806cd70 msgr2=0x7f08c806f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:55.514 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.513+0000 7f08ee200640 1 --2- 172.21.15.152:0/814263456 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f08c806cd70 0x7f08c806f230 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f08d8033000 tx=0x7f08d8009030 comp rx=0 tx=0).stop 2024-04-20T11:34:55.514 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.513+0000 7f08ee200640 1 -- 172.21.15.152:0/814263456 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f08e81e6dd0 msgr2=0x7f08e81e7250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:55.514 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.513+0000 7f08ee200640 1 --2- 172.21.15.152:0/814263456 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f08e81e6dd0 0x7f08e81e7250 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f08e80cfdf0 tx=0x7f08e000b4a0 comp rx=0 tx=0).stop 2024-04-20T11:34:55.514 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.513+0000 7f08ee200640 1 -- 172.21.15.152:0/814263456 shutdown_connections 2024-04-20T11:34:55.514 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.513+0000 7f08ee200640 1 --2- 172.21.15.152:0/814263456 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f08c806cd70 0x7f08c806f230 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:55.514 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.513+0000 7f08ee200640 1 --2- 172.21.15.152:0/814263456 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f08e81e6dd0 0x7f08e81e7250 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:55.514 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.513+0000 7f08ee200640 1 --2- 172.21.15.152:0/814263456 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f08e81e8c50 0x7f08e81e90d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:55.514 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.513+0000 7f08ee200640 1 --2- 172.21.15.152:0/814263456 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f08cc003a20 0x7f08e815bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:55.514 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.513+0000 7f08ee200640 1 -- 172.21.15.152:0/814263456 >> 172.21.15.152:0/814263456 conn(0x7f08e806b980 msgr2=0x7f08e815e750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:55.514 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.513+0000 7f08ee200640 1 -- 172.21.15.152:0/814263456 shutdown_connections 2024-04-20T11:34:55.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.513+0000 7f08ee200640 1 -- 172.21.15.152:0/814263456 wait complete. 2024-04-20T11:34:55.536 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs 2024-04-20T11:34:55.727 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.726+0000 7f8a97400640 1 Processor -- start 2024-04-20T11:34:55.727 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.726+0000 7f8a97400640 1 -- start start 2024-04-20T11:34:55.728 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.728+0000 7f8a97400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f8a900b7890 0x7f8a900d2860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:55.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.728+0000 7f8a97400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8a900cf4f0 0x7f8a900cf960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:55.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.728+0000 7f8a97400640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a900b7d00 con 0x7f8a90161e50 2024-04-20T11:34:55.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.728+0000 7f8a97400640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f8a90162230 con 0x7f8a900b7890 2024-04-20T11:34:55.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.728+0000 7f8a97400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8a900cfea0 con 0x7f8a900cf4f0 2024-04-20T11:34:55.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.728+0000 7f8a8fe00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f8a900b7890 0x7f8a900d2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:55.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.728+0000 7f8a8fe00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f8a900b7890 0x7f8a900d2860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49458/0 (socket says 172.21.15.152:49458) 2024-04-20T11:34:55.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.728+0000 7f8a8fe00640 1 -- 172.21.15.152:0/4159317799 learned_addr learned my addr 172.21.15.152:0/4159317799 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:55.730 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.729+0000 7f8a95400640 1 --2- 172.21.15.152:0/4159317799 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8a900cf4f0 0x7f8a900cf960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:55.730 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.729+0000 7f8a8fe00640 1 -- 172.21.15.152:0/4159317799 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8a900cf4f0 msgr2=0x7f8a900cf960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:55.730 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.729+0000 7f8a8fe00640 1 --2- 172.21.15.152:0/4159317799 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8a900cf4f0 0x7f8a900cf960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:55.730 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.729+0000 7f8a8fe00640 1 -- 172.21.15.152:0/4159317799 >> v1:172.21.15.152:6789/0 conn(0x7f8a90161e50 legacy=0x7f8a900ba180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:55.730 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.729+0000 7f8a8fe00640 1 -- 172.21.15.152:0/4159317799 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a900b7f70 con 0x7f8a900b7890 2024-04-20T11:34:55.730 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.729+0000 7f8a95400640 1 --2- 172.21.15.152:0/4159317799 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8a900cf4f0 0x7f8a900cf960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:34:55.731 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.730+0000 7f8a8fe00640 1 --2- 172.21.15.152:0/4159317799 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8a900b7890 0x7f8a900d2860 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f8a7c009870 tx=0x7f8a7c02ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f901d2236912ba51 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:55.731 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.730+0000 7f8a8f400640 1 -- 172.21.15.152:0/4159317799 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a7c02f830 con 0x7f8a900b7890 2024-04-20T11:34:55.731 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.730+0000 7f8a8f400640 1 -- 172.21.15.152:0/4159317799 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a7c02f9d0 con 0x7f8a900b7890 2024-04-20T11:34:55.731 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.730+0000 7f8a8f400640 1 -- 172.21.15.152:0/4159317799 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a7c02fda0 con 0x7f8a900b7890 2024-04-20T11:34:55.731 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.731+0000 7f8a97400640 1 -- 172.21.15.152:0/4159317799 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8a900b7890 msgr2=0x7f8a900d2860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:55.731 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.731+0000 7f8a97400640 1 --2- 172.21.15.152:0/4159317799 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8a900b7890 0x7f8a900d2860 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f8a7c009870 tx=0x7f8a7c02ba10 comp rx=0 tx=0).stop 2024-04-20T11:34:55.732 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.731+0000 7f8a97400640 1 -- 172.21.15.152:0/4159317799 shutdown_connections 2024-04-20T11:34:55.732 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.731+0000 7f8a97400640 1 --2- 172.21.15.152:0/4159317799 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8a900cf4f0 0x7f8a900cf960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:55.732 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.731+0000 7f8a97400640 1 --2- 172.21.15.152:0/4159317799 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8a900b7890 0x7f8a900d2860 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:55.732 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.731+0000 7f8a97400640 1 -- 172.21.15.152:0/4159317799 >> 172.21.15.152:0/4159317799 conn(0x7f8a90067ae0 msgr2=0x7f8a90067ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:55.732 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.731+0000 7f8a97400640 1 -- 172.21.15.152:0/4159317799 shutdown_connections 2024-04-20T11:34:55.733 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.731+0000 7f8a97400640 1 -- 172.21.15.152:0/4159317799 wait complete. 2024-04-20T11:34:55.733 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.733+0000 7f8a97400640 1 Processor -- start 2024-04-20T11:34:55.733 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.733+0000 7f8a97400640 1 -- start start 2024-04-20T11:34:55.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.734+0000 7f8a97400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8a900b7890 0x7f8a900c9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:55.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.734+0000 7f8a97400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8a900cf4f0 0x7f8a900cd350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:55.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.734+0000 7f8a97400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8a90161e50 0x7f8a900b53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:55.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.734+0000 7f8a97400640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f8a900b8810 con 0x7f8a90161e50 2024-04-20T11:34:55.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.734+0000 7f8a97400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8a900b8530 con 0x7f8a900b7890 2024-04-20T11:34:55.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.734+0000 7f8a97400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8a900b8af0 con 0x7f8a900cf4f0 2024-04-20T11:34:55.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.734+0000 7f8a95400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8a90161e50 0x7f8a900b53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:55.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.734+0000 7f8a95400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8a90161e50 0x7f8a900b53c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49470/0 (socket says 172.21.15.152:49470) 2024-04-20T11:34:55.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.734+0000 7f8a95400640 1 -- 172.21.15.152:0/3441364383 learned_addr learned my addr 172.21.15.152:0/3441364383 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:55.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.734+0000 7f8a8fe00640 1 --2- 172.21.15.152:0/3441364383 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8a900cf4f0 0x7f8a900cd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:55.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.734+0000 7f8a94a00640 1 --2- 172.21.15.152:0/3441364383 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8a900b7890 0x7f8a900c9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:55.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.734+0000 7f8a95400640 1 -- 172.21.15.152:0/3441364383 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8a900cf4f0 msgr2=0x7f8a900cd350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:55.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.734+0000 7f8a95400640 1 --2- 172.21.15.152:0/3441364383 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8a900cf4f0 0x7f8a900cd350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:55.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.735+0000 7f8a95400640 1 -- 172.21.15.152:0/3441364383 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8a900b7890 msgr2=0x7f8a900c9ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:55.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.735+0000 7f8a95400640 1 --2- 172.21.15.152:0/3441364383 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8a900b7890 0x7f8a900c9ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:55.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.735+0000 7f8a95400640 1 -- 172.21.15.152:0/3441364383 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a900ca180 con 0x7f8a90161e50 2024-04-20T11:34:55.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.735+0000 7f8a95400640 1 --2- 172.21.15.152:0/3441364383 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8a90161e50 0x7f8a900b53c0 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f8a8800cf20 tx=0x7f8a8800a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:55.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.735+0000 7f8a8cc00640 1 -- 172.21.15.152:0/3441364383 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a8801a070 con 0x7f8a90161e50 2024-04-20T11:34:55.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.735+0000 7f8a8cc00640 1 -- 172.21.15.152:0/3441364383 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a8800d840 con 0x7f8a90161e50 2024-04-20T11:34:55.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.735+0000 7f8a97400640 1 -- 172.21.15.152:0/3441364383 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a900ca460 con 0x7f8a90161e50 2024-04-20T11:34:55.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.735+0000 7f8a8cc00640 1 -- 172.21.15.152:0/3441364383 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a8800dc30 con 0x7f8a90161e50 2024-04-20T11:34:55.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.735+0000 7f8a97400640 1 -- 172.21.15.152:0/3441364383 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a900caa20 con 0x7f8a90161e50 2024-04-20T11:34:55.738 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.736+0000 7f8a97400640 1 -- 172.21.15.152:0/3441364383 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a900689f0 con 0x7f8a90161e50 2024-04-20T11:34:55.738 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.737+0000 7f8a8cc00640 1 -- 172.21.15.152:0/3441364383 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 20) ==== 91448+0+0 (secure 0 0 0) 0x7f8a8801d070 con 0x7f8a90161e50 2024-04-20T11:34:55.739 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.738+0000 7f8a8cc00640 1 --2- 172.21.15.152:0/3441364383 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f8a7006cda0 0x7f8a7006f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:55.739 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.739+0000 7f8a8cc00640 1 -- 172.21.15.152:0/3441364383 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5709+0+0 (secure 0 0 0) 0x7f8a8808cbd0 con 0x7f8a90161e50 2024-04-20T11:34:55.739 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.739+0000 7f8a94a00640 1 --2- 172.21.15.152:0/3441364383 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f8a7006cda0 0x7f8a7006f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:55.741 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.740+0000 7f8a94a00640 1 --2- 172.21.15.152:0/3441364383 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f8a7006cda0 0x7f8a7006f260 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8a84007fd0 tx=0x7f8a84009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:55.743 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.743+0000 7f8a8cc00640 1 -- 172.21.15.152:0/3441364383 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f8a8805a990 con 0x7f8a90161e50 2024-04-20T11:34:55.977 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.976+0000 7f8a97400640 1 -- 172.21.15.152:0/3441364383 --> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""]}) -- 0x7f8a90157b80 con 0x7f8a7006cda0 2024-04-20T11:34:55.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.988+0000 7f8a8cc00640 1 -- 172.21.15.152:0/3441364383 <== mgr.4109 v2:172.21.15.175:6836/4021250441 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7f8a90157b80 con 0x7f8a7006cda0 2024-04-20T11:34:55.988 INFO:teuthology.orchestra.run.smithi152.stdout:[] 2024-04-20T11:34:55.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.990+0000 7f8a97400640 1 -- 172.21.15.152:0/3441364383 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f8a7006cda0 msgr2=0x7f8a7006f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:55.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.990+0000 7f8a97400640 1 --2- 172.21.15.152:0/3441364383 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f8a7006cda0 0x7f8a7006f260 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8a84007fd0 tx=0x7f8a84009040 comp rx=0 tx=0).stop 2024-04-20T11:34:55.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.990+0000 7f8a97400640 1 -- 172.21.15.152:0/3441364383 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8a90161e50 msgr2=0x7f8a900b53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:55.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.990+0000 7f8a97400640 1 --2- 172.21.15.152:0/3441364383 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8a90161e50 0x7f8a900b53c0 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f8a8800cf20 tx=0x7f8a8800a7d0 comp rx=0 tx=0).stop 2024-04-20T11:34:55.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.991+0000 7f8a97400640 1 -- 172.21.15.152:0/3441364383 shutdown_connections 2024-04-20T11:34:55.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.991+0000 7f8a97400640 1 --2- 172.21.15.152:0/3441364383 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f8a7006cda0 0x7f8a7006f260 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:55.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.991+0000 7f8a97400640 1 --2- 172.21.15.152:0/3441364383 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8a90161e50 0x7f8a900b53c0 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:55.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.991+0000 7f8a97400640 1 --2- 172.21.15.152:0/3441364383 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8a900cf4f0 0x7f8a900cd350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:55.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.991+0000 7f8a97400640 1 --2- 172.21.15.152:0/3441364383 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8a900b7890 0x7f8a900c9ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:55.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.991+0000 7f8a97400640 1 -- 172.21.15.152:0/3441364383 >> 172.21.15.152:0/3441364383 conn(0x7f8a90067ae0 msgr2=0x7f8a900b49c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:55.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.991+0000 7f8a97400640 1 -- 172.21.15.152:0/3441364383 shutdown_connections 2024-04-20T11:34:55.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:55.991+0000 7f8a97400640 1 -- 172.21.15.152:0/3441364383 wait complete. 2024-04-20T11:34:56.011 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-04-20T11:34:56.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:56 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-20T11:34:56.131 INFO:teuthology.orchestra.run.smithi152.stdout:2 2024-04-20T11:34:56.134 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T11:34:56.141 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-20T11:34:56.141 DEBUG:teuthology.orchestra.run.smithi152:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-20T11:34:56.250 DEBUG:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-20T11:34:56.345 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-20T11:34:56.345 INFO:tasks.cephfs.mount:Terminating background process 2024-04-20T11:34:56.345 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-20T11:34:56.345 DEBUG:teuthology.orchestra.run.smithi152:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:34:56.399 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:34:56.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:56.599+0000 7f2ae6a00640 1 Processor -- start 2024-04-20T11:34:56.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:56.600+0000 7f2ae6a00640 1 -- start start 2024-04-20T11:34:56.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:56.601+0000 7f2ae6a00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f2ae0160b90 0x7f2ae015d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:56.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:56.601+0000 7f2ae6a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f2ae0071cd0 0x7f2ae0072120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:56.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:56.601+0000 7f2ae6a00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ae0072660 con 0x7f2ae015d1e0 2024-04-20T11:34:56.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:56.601+0000 7f2ae6a00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f2ae00ce8c0 con 0x7f2ae0071cd0 2024-04-20T11:34:56.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:56.601+0000 7f2ae6a00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f2ae01600b0 con 0x7f2ae0160b90 2024-04-20T11:34:56.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:56.602+0000 7f2ae6000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f2ae0071cd0 0x7f2ae0072120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:56.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:56.602+0000 7f2ae6000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f2ae0071cd0 0x7f2ae0072120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:42568/0 (socket says 172.21.15.152:42568) 2024-04-20T11:34:56.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:56.602+0000 7f2ae6000640 1 -- 172.21.15.152:0/4035137749 learned_addr learned my addr 172.21.15.152:0/4035137749 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:56.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:56.602+0000 7f2ae4c00640 1 --2- 172.21.15.152:0/4035137749 >> v2:172.21.15.175:3300/0 conn(0x7f2ae0160b90 0x7f2ae015d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:56.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:56.602+0000 7f2ae6000640 1 -- 172.21.15.152:0/4035137749 >> v2:172.21.15.175:3300/0 conn(0x7f2ae0160b90 msgr2=0x7f2ae015d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:56.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:56.602+0000 7f2ae6000640 1 --2- 172.21.15.152:0/4035137749 >> v2:172.21.15.175:3300/0 conn(0x7f2ae0160b90 0x7f2ae015d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:56.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:56.602+0000 7f2ae6000640 1 -- 172.21.15.152:0/4035137749 >> v1:172.21.15.175:6789/0 conn(0x7f2ae015d1e0 legacy=0x7f2ae00cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:56.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:56.602+0000 7f2ae6000640 1 -- 172.21.15.152:0/4035137749 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ae0160390 con 0x7f2ae0071cd0 2024-04-20T11:34:56.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:56.602+0000 7f2ae4c00640 1 --2- 172.21.15.152:0/4035137749 >> v2:172.21.15.175:3300/0 conn(0x7f2ae0160b90 0x7f2ae015d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:34:56.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:56.604+0000 7f2ae6000640 1 --2- 172.21.15.152:0/4035137749 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ae0071cd0 0x7f2ae0072120 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f2ad8009e50 tx=0x7f2ad802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfaf0d0e7c4b0380 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:56.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:56.604+0000 7f2adfe00640 1 -- 172.21.15.152:0/4035137749 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad802ca80 con 0x7f2ae0071cd0 2024-04-20T11:34:56.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:56.604+0000 7f2adfe00640 1 -- 172.21.15.152:0/4035137749 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ad802cc20 con 0x7f2ae0071cd0 2024-04-20T11:34:56.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:56.604+0000 7f2adfe00640 1 -- 172.21.15.152:0/4035137749 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad80343f0 con 0x7f2ae0071cd0 2024-04-20T11:34:56.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:56.605+0000 7f2ae6a00640 1 -- 172.21.15.152:0/4035137749 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ae0071cd0 msgr2=0x7f2ae0072120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:56.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:56.605+0000 7f2ae6a00640 1 --2- 172.21.15.152:0/4035137749 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ae0071cd0 0x7f2ae0072120 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f2ad8009e50 tx=0x7f2ad802bee0 comp rx=0 tx=0).stop 2024-04-20T11:34:56.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:56.605+0000 7f2ae6a00640 1 -- 172.21.15.152:0/4035137749 shutdown_connections 2024-04-20T11:34:56.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:56.605+0000 7f2ae6a00640 1 --2- 172.21.15.152:0/4035137749 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ae0071cd0 0x7f2ae0072120 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:56.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:56.605+0000 7f2ae6a00640 1 --2- 172.21.15.152:0/4035137749 >> v2:172.21.15.175:3300/0 conn(0x7f2ae0160b90 0x7f2ae015d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:56.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:56.605+0000 7f2ae6a00640 1 -- 172.21.15.152:0/4035137749 >> 172.21.15.152:0/4035137749 conn(0x7f2ae0067ae0 msgr2=0x7f2ae006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:56.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:56.605+0000 7f2ae6a00640 1 -- 172.21.15.152:0/4035137749 shutdown_connections 2024-04-20T11:34:56.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:56.605+0000 7f2ae6a00640 1 -- 172.21.15.152:0/4035137749 wait complete. 2024-04-20T11:34:56.607 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:56.607+0000 7f2ae6a00640 1 Processor -- start 2024-04-20T11:34:56.607 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:56.607+0000 7f2ae6a00640 1 -- start start 2024-04-20T11:34:56.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:56.608+0000 7f2ae6a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ae0071cd0 0x7f2ae00af240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:56.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:56.608+0000 7f2ae6a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2ae015d1e0 0x7f2ae0157310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:56.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:56.608+0000 7f2ae6a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ae0160b90 0x7f2ae00b4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:56.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:56.608+0000 7f2ae6a00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f2ae00ccd70 con 0x7f2ae0160b90 2024-04-20T11:34:56.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:56.608+0000 7f2ae6a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2ae00ceba0 con 0x7f2ae0071cd0 2024-04-20T11:34:56.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:56.608+0000 7f2ae6a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2ae00ccef0 con 0x7f2ae015d1e0 2024-04-20T11:34:56.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:56.608+0000 7f2ae6000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ae0160b90 0x7f2ae00b4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:56.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:56.609+0000 7f2ae6000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ae0160b90 0x7f2ae00b4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:42578/0 (socket says 172.21.15.152:42578) 2024-04-20T11:34:56.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:56.609+0000 7f2ae6000640 1 -- 172.21.15.152:0/1385661382 learned_addr learned my addr 172.21.15.152:0/1385661382 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:56.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:56.609+0000 7f2ae6000640 1 -- 172.21.15.152:0/1385661382 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2ae015d1e0 msgr2=0x7f2ae0157310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:56.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:56.609+0000 7f2ae6000640 1 --2- 172.21.15.152:0/1385661382 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2ae015d1e0 0x7f2ae0157310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:56.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:56.609+0000 7f2ae6000640 1 -- 172.21.15.152:0/1385661382 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ae0071cd0 msgr2=0x7f2ae00af240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:56.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:56.609+0000 7f2ae6000640 1 --2- 172.21.15.152:0/1385661382 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ae0071cd0 0x7f2ae00af240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:56.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:56.609+0000 7f2ae6000640 1 -- 172.21.15.152:0/1385661382 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ae00b54b0 con 0x7f2ae0160b90 2024-04-20T11:34:56.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:56.609+0000 7f2ae6000640 1 --2- 172.21.15.152:0/1385661382 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ae0160b90 0x7f2ae00b4370 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f2ad8036040 tx=0x7f2ad8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:56.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:56.610+0000 7f2add600640 1 -- 172.21.15.152:0/1385661382 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad8043020 con 0x7f2ae0160b90 2024-04-20T11:34:56.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:56.610+0000 7f2add600640 1 -- 172.21.15.152:0/1385661382 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ad803e450 con 0x7f2ae0160b90 2024-04-20T11:34:56.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:56.610+0000 7f2add600640 1 -- 172.21.15.152:0/1385661382 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad803e860 con 0x7f2ae0160b90 2024-04-20T11:34:56.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:56.610+0000 7f2ae6a00640 1 -- 172.21.15.152:0/1385661382 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ae01579a0 con 0x7f2ae0160b90 2024-04-20T11:34:56.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:56.610+0000 7f2ae6a00640 1 -- 172.21.15.152:0/1385661382 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ae00b3b70 con 0x7f2ae0160b90 2024-04-20T11:34:56.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:56.611+0000 7f2ae6a00640 1 -- 172.21.15.152:0/1385661382 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ae0157ec0 con 0x7f2ae0160b90 2024-04-20T11:34:56.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:56.613+0000 7f2add600640 1 -- 172.21.15.152:0/1385661382 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 21) ==== 91550+0+0 (secure 0 0 0) 0x7f2ad8046020 con 0x7f2ae0160b90 2024-04-20T11:34:56.614 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:56.614+0000 7f2add600640 1 --2- 172.21.15.152:0/1385661382 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f2ac006ce90 0x7f2ac006f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:56.614 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:56.614+0000 7f2add600640 1 -- 172.21.15.152:0/1385661382 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5765+0+0 (secure 0 0 0) 0x7f2ad80aef70 con 0x7f2ae0160b90 2024-04-20T11:34:56.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:56.615+0000 7f2ae5600640 1 --2- 172.21.15.152:0/1385661382 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f2ac006ce90 0x7f2ac006f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:56.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:56.617+0000 7f2ae5600640 1 --2- 172.21.15.152:0/1385661382 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f2ac006ce90 0x7f2ac006f350 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f2ad4003cc0 tx=0x7f2ad40079b0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:56.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:56.619+0000 7f2add600640 1 -- 172.21.15.152:0/1385661382 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f2ad804d030 con 0x7f2ae0160b90 2024-04-20T11:34:56.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:56.832+0000 7f2ae6a00640 1 -- 172.21.15.152:0/1385661382 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2ae0075e90 con 0x7f2ae0160b90 2024-04-20T11:34:56.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:56.835+0000 7f2add600640 1 -- 172.21.15.152:0/1385661382 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v119) ==== 74+0+16850 (secure 0 0 0) 0x7f2ae0075e90 con 0x7f2ae0160b90 2024-04-20T11:34:56.836 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:34:56.836 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":119,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:34:56.181185+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:34:33.781054+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-04-20T11:34:35.031728+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":119,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:34:56.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:56.838+0000 7f2ae6a00640 1 -- 172.21.15.152:0/1385661382 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f2ac006ce90 msgr2=0x7f2ac006f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:56.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:56.838+0000 7f2ae6a00640 1 --2- 172.21.15.152:0/1385661382 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f2ac006ce90 0x7f2ac006f350 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f2ad4003cc0 tx=0x7f2ad40079b0 comp rx=0 tx=0).stop 2024-04-20T11:34:56.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:56.838+0000 7f2ae6a00640 1 -- 172.21.15.152:0/1385661382 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ae0160b90 msgr2=0x7f2ae00b4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:56.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:56.838+0000 7f2ae6a00640 1 --2- 172.21.15.152:0/1385661382 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ae0160b90 0x7f2ae00b4370 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f2ad8036040 tx=0x7f2ad8004a40 comp rx=0 tx=0).stop 2024-04-20T11:34:56.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:56.838+0000 7f2ae6a00640 1 -- 172.21.15.152:0/1385661382 shutdown_connections 2024-04-20T11:34:56.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:56.838+0000 7f2ae6a00640 1 --2- 172.21.15.152:0/1385661382 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f2ac006ce90 0x7f2ac006f350 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:56.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:56.838+0000 7f2ae6a00640 1 --2- 172.21.15.152:0/1385661382 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ae0160b90 0x7f2ae00b4370 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:56.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:56.838+0000 7f2ae6a00640 1 --2- 172.21.15.152:0/1385661382 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2ae015d1e0 0x7f2ae0157310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:56.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:56.838+0000 7f2ae6a00640 1 --2- 172.21.15.152:0/1385661382 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ae0071cd0 0x7f2ae00af240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:56.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:56.838+0000 7f2ae6a00640 1 -- 172.21.15.152:0/1385661382 >> 172.21.15.152:0/1385661382 conn(0x7f2ae0067ae0 msgr2=0x7f2ae006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:56.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:56.838+0000 7f2ae6a00640 1 -- 172.21.15.152:0/1385661382 shutdown_connections 2024-04-20T11:34:56.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:56.839+0000 7f2ae6a00640 1 -- 172.21.15.152:0/1385661382 wait complete. 2024-04-20T11:34:56.858 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T11:26:47.246571+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T11:34:33.781054+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '117', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T11:34:35.031728+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '119', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 5, 'snap_epoch': 119, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T11:34:56.859 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T11:34:57.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.050+0000 7f8e9c200640 1 Processor -- start 2024-04-20T11:34:57.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.050+0000 7f8e9c200640 1 -- start start 2024-04-20T11:34:57.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.052+0000 7f8e9c200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8e940b2880 0x7f8e940b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:57.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.052+0000 7f8e9c200640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e941463e0 con 0x7f8e94146000 2024-04-20T11:34:57.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.052+0000 7f8e9c200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e940b3e20 con 0x7f8e940cd1b0 2024-04-20T11:34:57.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.052+0000 7f8e9c200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8e940b4750 con 0x7f8e940b2880 2024-04-20T11:34:57.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.052+0000 7f8e99800640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f8e94146000 0x7f8e940b2170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:58688/0 (socket says 172.21.15.152:58688) 2024-04-20T11:34:57.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.052+0000 7f8e99800640 1 -- 172.21.15.152:0/966438430 learned_addr learned my addr 172.21.15.152:0/966438430 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:34:57.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.052+0000 7f8e98e00640 1 --2- 172.21.15.152:0/966438430 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8e940b2880 0x7f8e940b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:57.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.052+0000 7f8e93e00640 1 -- 172.21.15.152:0/966438430 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3993402349 0 0) 0x7f8e941463e0 con 0x7f8e94146000 2024-04-20T11:34:57.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.053+0000 7f8e93e00640 1 -- 172.21.15.152:0/966438430 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8e7c003520 con 0x7f8e94146000 2024-04-20T11:34:57.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.053+0000 7f8e93e00640 1 -- 172.21.15.152:0/966438430 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 817366064 0 0) 0x7f8e7c003520 con 0x7f8e94146000 2024-04-20T11:34:57.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.053+0000 7f8e93e00640 1 -- 172.21.15.152:0/966438430 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8e940b2880 msgr2=0x7f8e940b4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:57.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.053+0000 7f8e93e00640 1 --2- 172.21.15.152:0/966438430 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8e940b2880 0x7f8e940b4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:57.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.053+0000 7f8e93e00640 1 -- 172.21.15.152:0/966438430 >> v1:172.21.15.175:6789/0 conn(0x7f8e940cd1b0 legacy=0x7f8e940cd600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:57.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.053+0000 7f8e93e00640 1 -- 172.21.15.152:0/966438430 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e940b4100 con 0x7f8e94146000 2024-04-20T11:34:57.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.053+0000 7f8e93e00640 1 -- 172.21.15.152:0/966438430 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f8e88001670 con 0x7f8e94146000 2024-04-20T11:34:57.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.053+0000 7f8e93e00640 1 -- 172.21.15.152:0/966438430 >> v1:172.21.15.152:6789/0 conn(0x7f8e94146000 legacy=0x7f8e940b2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:57.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.053+0000 7f8e93e00640 1 --2- 172.21.15.152:0/966438430 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8e7c0039e0 0x7f8e7c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:57.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.053+0000 7f8e93e00640 1 --2- 172.21.15.152:0/966438430 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e7c004800 0x7f8e7c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:57.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.053+0000 7f8e93e00640 1 --2- 172.21.15.152:0/966438430 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8e940b2880 0x7f8e7c005270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:57.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.053+0000 7f8e93e00640 1 -- 172.21.15.152:0/966438430 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f8e940b4750 con 0x7f8e7c0039e0 2024-04-20T11:34:57.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.053+0000 7f8e93e00640 1 -- 172.21.15.152:0/966438430 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8e7c00bb50 con 0x7f8e7c004800 2024-04-20T11:34:57.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.053+0000 7f8e93e00640 1 -- 172.21.15.152:0/966438430 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8e7c00bd50 con 0x7f8e940b2880 2024-04-20T11:34:57.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.053+0000 7f8e99800640 1 --2- 172.21.15.152:0/966438430 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8e7c0039e0 0x7f8e7c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:57.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.054+0000 7f8e98e00640 1 --2- 172.21.15.152:0/966438430 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e7c004800 0x7f8e7c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:57.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.054+0000 7f8e9a200640 1 --2- 172.21.15.152:0/966438430 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8e940b2880 0x7f8e7c005270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:57.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.054+0000 7f8e99800640 1 -- 172.21.15.152:0/966438430 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8e940b2880 msgr2=0x7f8e7c005270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:57.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.054+0000 7f8e99800640 1 --2- 172.21.15.152:0/966438430 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8e940b2880 0x7f8e7c005270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:57.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.054+0000 7f8e99800640 1 -- 172.21.15.152:0/966438430 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e7c004800 msgr2=0x7f8e7c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:57.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.054+0000 7f8e99800640 1 --2- 172.21.15.152:0/966438430 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e7c004800 0x7f8e7c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:57.057 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.054+0000 7f8e99800640 1 -- 172.21.15.152:0/966438430 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e7c003520 con 0x7f8e7c0039e0 2024-04-20T11:34:57.057 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.054+0000 7f8e98e00640 1 --2- 172.21.15.152:0/966438430 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e7c004800 0x7f8e7c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:34:57.057 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.054+0000 7f8e9a200640 1 --2- 172.21.15.152:0/966438430 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8e940b2880 0x7f8e7c005270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:34:57.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.055+0000 7f8e99800640 1 --2- 172.21.15.152:0/966438430 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8e7c0039e0 0x7f8e7c003e00 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f8e8800c110 tx=0x7f8e7c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e6f5492a476c0e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:57.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.055+0000 7f8e93e00640 1 -- 172.21.15.152:0/966438430 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e88007c40 con 0x7f8e7c0039e0 2024-04-20T11:34:57.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.055+0000 7f8e93e00640 1 -- 172.21.15.152:0/966438430 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e88030070 con 0x7f8e7c0039e0 2024-04-20T11:34:57.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.055+0000 7f8e93e00640 1 -- 172.21.15.152:0/966438430 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e88005d30 con 0x7f8e7c0039e0 2024-04-20T11:34:57.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.056+0000 7f8e9c200640 1 -- 172.21.15.152:0/966438430 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8e7c0039e0 msgr2=0x7f8e7c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:57.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.056+0000 7f8e9c200640 1 --2- 172.21.15.152:0/966438430 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8e7c0039e0 0x7f8e7c003e00 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f8e8800c110 tx=0x7f8e7c005830 comp rx=0 tx=0).stop 2024-04-20T11:34:57.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.056+0000 7f8e9a200640 1 -- 172.21.15.152:0/966438430 reap_dead start 2024-04-20T11:34:57.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.056+0000 7f8e9c200640 1 -- 172.21.15.152:0/966438430 shutdown_connections 2024-04-20T11:34:57.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.056+0000 7f8e9c200640 1 -- 172.21.15.152:0/966438430 >> 172.21.15.152:0/966438430 conn(0x7f8e9406b980 msgr2=0x7f8e9406bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:57.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.056+0000 7f8e9c200640 1 -- 172.21.15.152:0/966438430 shutdown_connections 2024-04-20T11:34:57.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.056+0000 7f8e9c200640 1 -- 172.21.15.152:0/966438430 wait complete. 2024-04-20T11:34:57.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.058+0000 7f8e9c200640 1 Processor -- start 2024-04-20T11:34:57.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.058+0000 7f8e9c200640 1 -- start start 2024-04-20T11:34:57.061 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.059+0000 7f8e9c200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8e7c0039e0 0x7f8e9415e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:57.061 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.059+0000 7f8e9c200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e94071db0 0x7f8e94072230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:57.061 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.059+0000 7f8e9c200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8e94143370 0x7f8e941437f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:57.061 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.059+0000 7f8e9c200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f8e940b4d60 con 0x7f8e94143370 2024-04-20T11:34:57.061 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.059+0000 7f8e9c200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8e940b4be0 con 0x7f8e94071db0 2024-04-20T11:34:57.061 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.059+0000 7f8e9c200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8e940b4ee0 con 0x7f8e7c0039e0 2024-04-20T11:34:57.061 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.060+0000 7f8e9a200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8e94143370 0x7f8e941437f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:57.061 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.060+0000 7f8e9a200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8e94143370 0x7f8e941437f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:42600/0 (socket says 172.21.15.152:42600) 2024-04-20T11:34:57.061 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.060+0000 7f8e9a200640 1 -- 172.21.15.152:0/985572339 learned_addr learned my addr 172.21.15.152:0/985572339 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:57.061 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.060+0000 7f8e99800640 1 --2- 172.21.15.152:0/985572339 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8e7c0039e0 0x7f8e9415e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:57.061 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.060+0000 7f8e98e00640 1 --2- 172.21.15.152:0/985572339 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e94071db0 0x7f8e94072230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:57.061 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.060+0000 7f8e9a200640 1 -- 172.21.15.152:0/985572339 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8e7c0039e0 msgr2=0x7f8e9415e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:57.061 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.060+0000 7f8e9a200640 1 --2- 172.21.15.152:0/985572339 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8e7c0039e0 0x7f8e9415e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:57.061 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.060+0000 7f8e9a200640 1 -- 172.21.15.152:0/985572339 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e94071db0 msgr2=0x7f8e94072230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:57.062 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.060+0000 7f8e9a200640 1 --2- 172.21.15.152:0/985572339 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e94071db0 0x7f8e94072230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:57.062 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.060+0000 7f8e9a200640 1 -- 172.21.15.152:0/985572339 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e941f7db0 con 0x7f8e94143370 2024-04-20T11:34:57.062 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.060+0000 7f8e99800640 1 --2- 172.21.15.152:0/985572339 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8e7c0039e0 0x7f8e9415e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:34:57.063 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.061+0000 7f8e9a200640 1 --2- 172.21.15.152:0/985572339 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8e94143370 0x7f8e941437f0 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f8e940700a0 tx=0x7f8e8c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:57.063 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.061+0000 7f8e91600640 1 -- 172.21.15.152:0/985572339 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e8c006b50 con 0x7f8e94143370 2024-04-20T11:34:57.063 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.061+0000 7f8e91600640 1 -- 172.21.15.152:0/985572339 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e8c006cf0 con 0x7f8e94143370 2024-04-20T11:34:57.063 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.061+0000 7f8e91600640 1 -- 172.21.15.152:0/985572339 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e8c003bb0 con 0x7f8e94143370 2024-04-20T11:34:57.063 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.061+0000 7f8e9c200640 1 -- 172.21.15.152:0/985572339 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e941f8090 con 0x7f8e94143370 2024-04-20T11:34:57.063 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.061+0000 7f8e9c200640 1 -- 172.21.15.152:0/985572339 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e941f8650 con 0x7f8e94143370 2024-04-20T11:34:57.063 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.062+0000 7f8e9c200640 1 -- 172.21.15.152:0/985572339 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e940689f0 con 0x7f8e94143370 2024-04-20T11:34:57.063 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.063+0000 7f8e91600640 1 -- 172.21.15.152:0/985572339 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 21) ==== 91550+0+0 (secure 0 0 0) 0x7f8e8c01a030 con 0x7f8e94143370 2024-04-20T11:34:57.065 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.065+0000 7f8e91600640 1 --2- 172.21.15.152:0/985572339 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f8e7406cec0 0x7f8e7406f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:57.065 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.065+0000 7f8e91600640 1 -- 172.21.15.152:0/985572339 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5765+0+0 (secure 0 0 0) 0x7f8e8c08bb70 con 0x7f8e94143370 2024-04-20T11:34:57.065 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.065+0000 7f8e99800640 1 --2- 172.21.15.152:0/985572339 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f8e7406cec0 0x7f8e7406f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:57.067 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.066+0000 7f8e99800640 1 --2- 172.21.15.152:0/985572339 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f8e7406cec0 0x7f8e7406f380 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f8e8800c110 tx=0x7f8e88009030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:57.069 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.069+0000 7f8e91600640 1 -- 172.21.15.152:0/985572339 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f8e8c021020 con 0x7f8e94143370 2024-04-20T11:34:57.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.275+0000 7f8e9c200640 1 -- 172.21.15.152:0/985572339 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8e940b2220 con 0x7f8e94143370 2024-04-20T11:34:57.276 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.276+0000 7f8e91600640 1 -- 172.21.15.152:0/985572339 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v119) ==== 93+0+10 (secure 0 0 0) 0x7f8e8c059790 con 0x7f8e94143370 2024-04-20T11:34:57.276 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 1 2024-04-20T11:34:57.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.278+0000 7f8e9c200640 1 -- 172.21.15.152:0/985572339 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f8e7406cec0 msgr2=0x7f8e7406f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:57.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.278+0000 7f8e9c200640 1 --2- 172.21.15.152:0/985572339 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f8e7406cec0 0x7f8e7406f380 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f8e8800c110 tx=0x7f8e88009030 comp rx=0 tx=0).stop 2024-04-20T11:34:57.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.278+0000 7f8e9c200640 1 -- 172.21.15.152:0/985572339 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8e94143370 msgr2=0x7f8e941437f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:57.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.278+0000 7f8e9c200640 1 --2- 172.21.15.152:0/985572339 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8e94143370 0x7f8e941437f0 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f8e940700a0 tx=0x7f8e8c00dc00 comp rx=0 tx=0).stop 2024-04-20T11:34:57.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.278+0000 7f8e9c200640 1 -- 172.21.15.152:0/985572339 shutdown_connections 2024-04-20T11:34:57.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.278+0000 7f8e9c200640 1 --2- 172.21.15.152:0/985572339 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f8e7406cec0 0x7f8e7406f380 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:57.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.278+0000 7f8e9c200640 1 --2- 172.21.15.152:0/985572339 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8e94143370 0x7f8e941437f0 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:57.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.278+0000 7f8e9c200640 1 --2- 172.21.15.152:0/985572339 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e94071db0 0x7f8e94072230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:57.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.278+0000 7f8e9c200640 1 --2- 172.21.15.152:0/985572339 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8e7c0039e0 0x7f8e9415e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:57.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.278+0000 7f8e9c200640 1 -- 172.21.15.152:0/985572339 >> 172.21.15.152:0/985572339 conn(0x7f8e9406b980 msgr2=0x7f8e9406dc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:57.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.279+0000 7f8e9c200640 1 -- 172.21.15.152:0/985572339 shutdown_connections 2024-04-20T11:34:57.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.279+0000 7f8e9c200640 1 -- 172.21.15.152:0/985572339 wait complete. 2024-04-20T11:34:57.299 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T11:34:57.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.499+0000 7fe7d2400640 1 Processor -- start 2024-04-20T11:34:57.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.500+0000 7fe7d2400640 1 -- start start 2024-04-20T11:34:57.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.501+0000 7fe7d2400640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fe7cc15eb50 0x7fe7cc0d0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:57.502 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.501+0000 7fe7d2400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7cc0d35a0 0x7fe7cc0cf470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:57.502 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.501+0000 7fe7d2400640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7cc0d19c0 con 0x7fe7cc163230 2024-04-20T11:34:57.502 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.501+0000 7fe7d2400640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fe7cc162750 con 0x7fe7cc15eb50 2024-04-20T11:34:57.502 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.501+0000 7fe7d2400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe7cc162a30 con 0x7fe7cc0d35a0 2024-04-20T11:34:57.502 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.502+0000 7fe7d1a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7cc0d35a0 0x7fe7cc0cf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:57.502 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.502+0000 7fe7d1a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7cc0d35a0 0x7fe7cc0cf470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.152:59764/0 (socket says 172.21.15.152:59764) 2024-04-20T11:34:57.503 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.502+0000 7fe7d1a00640 1 -- 172.21.15.152:0/529625104 learned_addr learned my addr 172.21.15.152:0/529625104 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:57.503 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.502+0000 7fe7d1000640 1 --2- 172.21.15.152:0/529625104 >> v2:172.21.15.175:3300/0 conn(0x7fe7cc15eb50 0x7fe7cc0d0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:57.503 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.503+0000 7fe7cb400640 1 -- 172.21.15.152:0/529625104 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3422392118 0 0) 0x7fe7cc0d19c0 con 0x7fe7cc163230 2024-04-20T11:34:57.503 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.503+0000 7fe7cb400640 1 -- 172.21.15.152:0/529625104 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe7b0003520 con 0x7fe7cc163230 2024-04-20T11:34:57.503 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.503+0000 7fe7d1a00640 1 -- 172.21.15.152:0/529625104 >> v2:172.21.15.175:3300/0 conn(0x7fe7cc15eb50 msgr2=0x7fe7cc0d0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:57.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.503+0000 7fe7d1a00640 1 --2- 172.21.15.152:0/529625104 >> v2:172.21.15.175:3300/0 conn(0x7fe7cc15eb50 0x7fe7cc0d0f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:57.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.503+0000 7fe7d1a00640 1 -- 172.21.15.152:0/529625104 >> v1:172.21.15.175:6789/0 conn(0x7fe7cc163230 legacy=0x7fe7cc0d2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:57.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.503+0000 7fe7d1a00640 1 -- 172.21.15.152:0/529625104 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7cc162d10 con 0x7fe7cc0d35a0 2024-04-20T11:34:57.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.504+0000 7fe7d1a00640 1 --2- 172.21.15.152:0/529625104 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7cc0d35a0 0x7fe7cc0cf470 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe7c4009870 tx=0x7fe7c402b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5985a95cc64eee4f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:57.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.505+0000 7fe7cb400640 1 -- 172.21.15.152:0/529625104 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c4035020 con 0x7fe7cc0d35a0 2024-04-20T11:34:57.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.505+0000 7fe7cb400640 1 -- 172.21.15.152:0/529625104 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7c4031640 con 0x7fe7cc0d35a0 2024-04-20T11:34:57.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.505+0000 7fe7cb400640 1 -- 172.21.15.152:0/529625104 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c402d040 con 0x7fe7cc0d35a0 2024-04-20T11:34:57.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.505+0000 7fe7d2400640 1 -- 172.21.15.152:0/529625104 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7cc0d35a0 msgr2=0x7fe7cc0cf470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:57.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.505+0000 7fe7d2400640 1 --2- 172.21.15.152:0/529625104 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7cc0d35a0 0x7fe7cc0cf470 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe7c4009870 tx=0x7fe7c402b8f0 comp rx=0 tx=0).stop 2024-04-20T11:34:57.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.506+0000 7fe7d2400640 1 -- 172.21.15.152:0/529625104 shutdown_connections 2024-04-20T11:34:57.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.506+0000 7fe7d2400640 1 --2- 172.21.15.152:0/529625104 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7cc0d35a0 0x7fe7cc0cf470 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:57.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.506+0000 7fe7d2400640 1 --2- 172.21.15.152:0/529625104 >> v2:172.21.15.175:3300/0 conn(0x7fe7cc15eb50 0x7fe7cc0d0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:57.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.506+0000 7fe7d2400640 1 -- 172.21.15.152:0/529625104 >> 172.21.15.152:0/529625104 conn(0x7fe7cc08bfd0 msgr2=0x7fe7cc0d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:57.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.506+0000 7fe7d2400640 1 -- 172.21.15.152:0/529625104 shutdown_connections 2024-04-20T11:34:57.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.506+0000 7fe7d2400640 1 -- 172.21.15.152:0/529625104 wait complete. 2024-04-20T11:34:57.508 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.507+0000 7fe7d2400640 1 Processor -- start 2024-04-20T11:34:57.508 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.508+0000 7fe7d2400640 1 -- start start 2024-04-20T11:34:57.510 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.509+0000 7fe7d2400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7cc0d35a0 0x7fe7cc1f5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:57.510 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.509+0000 7fe7d2400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7cc15eb50 0x7fe7cc1f4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:57.510 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.509+0000 7fe7d2400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe7cc163230 0x7fe7cc1f7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:57.510 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.509+0000 7fe7d2400640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fe7cc0d15c0 con 0x7fe7cc163230 2024-04-20T11:34:57.510 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.509+0000 7fe7d2400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe7cc0d1440 con 0x7fe7cc0d35a0 2024-04-20T11:34:57.510 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.509+0000 7fe7d2400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe7cc0d1740 con 0x7fe7cc15eb50 2024-04-20T11:34:57.510 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.509+0000 7fe7d1a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe7cc163230 0x7fe7cc1f7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:57.510 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.509+0000 7fe7d1a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe7cc163230 0x7fe7cc1f7bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:42614/0 (socket says 172.21.15.152:42614) 2024-04-20T11:34:57.511 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.509+0000 7fe7d1a00640 1 -- 172.21.15.152:0/2846724494 learned_addr learned my addr 172.21.15.152:0/2846724494 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:57.511 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.509+0000 7fe7d1000640 1 --2- 172.21.15.152:0/2846724494 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7cc0d35a0 0x7fe7cc1f5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:57.511 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.509+0000 7fe7cbe00640 1 --2- 172.21.15.152:0/2846724494 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7cc15eb50 0x7fe7cc1f4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:57.511 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.509+0000 7fe7d1a00640 1 -- 172.21.15.152:0/2846724494 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7cc15eb50 msgr2=0x7fe7cc1f4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:57.511 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.509+0000 7fe7d1a00640 1 --2- 172.21.15.152:0/2846724494 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7cc15eb50 0x7fe7cc1f4cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:57.511 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.509+0000 7fe7d1a00640 1 -- 172.21.15.152:0/2846724494 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7cc0d35a0 msgr2=0x7fe7cc1f5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:57.511 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.509+0000 7fe7d1a00640 1 --2- 172.21.15.152:0/2846724494 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7cc0d35a0 0x7fe7cc1f5e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:57.511 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.509+0000 7fe7d1a00640 1 -- 172.21.15.152:0/2846724494 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7cc0b4060 con 0x7fe7cc163230 2024-04-20T11:34:57.511 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.509+0000 7fe7d1000640 1 --2- 172.21.15.152:0/2846724494 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7cc0d35a0 0x7fe7cc1f5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:34:57.511 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.510+0000 7fe7d1a00640 1 --2- 172.21.15.152:0/2846724494 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe7cc163230 0x7fe7cc1f7bc0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7fe7c4009870 tx=0x7fe7c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:57.512 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.510+0000 7fe7c8c00640 1 -- 172.21.15.152:0/2846724494 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c4040070 con 0x7fe7cc163230 2024-04-20T11:34:57.512 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.510+0000 7fe7c8c00640 1 -- 172.21.15.152:0/2846724494 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7c40399f0 con 0x7fe7cc163230 2024-04-20T11:34:57.512 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.510+0000 7fe7d2400640 1 -- 172.21.15.152:0/2846724494 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7cc1f5300 con 0x7fe7cc163230 2024-04-20T11:34:57.512 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.510+0000 7fe7c8c00640 1 -- 172.21.15.152:0/2846724494 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c4035040 con 0x7fe7cc163230 2024-04-20T11:34:57.512 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.510+0000 7fe7d2400640 1 -- 172.21.15.152:0/2846724494 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7cc1f58c0 con 0x7fe7cc163230 2024-04-20T11:34:57.512 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.512+0000 7fe7d2400640 1 -- 172.21.15.152:0/2846724494 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7cc075e60 con 0x7fe7cc163230 2024-04-20T11:34:57.513 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.512+0000 7fe7c8c00640 1 -- 172.21.15.152:0/2846724494 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 21) ==== 91550+0+0 (secure 0 0 0) 0x7fe7c404b060 con 0x7fe7cc163230 2024-04-20T11:34:57.513 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.513+0000 7fe7c8c00640 1 --2- 172.21.15.152:0/2846724494 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe7a806ce90 0x7fe7a806f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:57.514 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.513+0000 7fe7c8c00640 1 -- 172.21.15.152:0/2846724494 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5765+0+0 (secure 0 0 0) 0x7fe7c40add50 con 0x7fe7cc163230 2024-04-20T11:34:57.514 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.514+0000 7fe7d1000640 1 --2- 172.21.15.152:0/2846724494 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe7a806ce90 0x7fe7a806f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:57.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.515+0000 7fe7d1000640 1 --2- 172.21.15.152:0/2846724494 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe7a806ce90 0x7fe7a806f350 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc0b3640 tx=0x7fe7b4008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:57.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.517+0000 7fe7c8c00640 1 -- 172.21.15.152:0/2846724494 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe7c407b890 con 0x7fe7cc163230 2024-04-20T11:34:57.721 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.721+0000 7fe7d2400640 1 -- 172.21.15.152:0/2846724494 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe7cc0ba500 con 0x7fe7cc163230 2024-04-20T11:34:57.722 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.722+0000 7fe7c8c00640 1 -- 172.21.15.152:0/2846724494 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v119) ==== 104+0+11 (secure 0 0 0) 0x7fe7c407b190 con 0x7fe7cc163230 2024-04-20T11:34:57.722 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 32 2024-04-20T11:34:57.725 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.725+0000 7fe7d2400640 1 -- 172.21.15.152:0/2846724494 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe7a806ce90 msgr2=0x7fe7a806f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:57.726 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.725+0000 7fe7d2400640 1 --2- 172.21.15.152:0/2846724494 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe7a806ce90 0x7fe7a806f350 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc0b3640 tx=0x7fe7b4008040 comp rx=0 tx=0).stop 2024-04-20T11:34:57.726 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.725+0000 7fe7d2400640 1 -- 172.21.15.152:0/2846724494 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe7cc163230 msgr2=0x7fe7cc1f7bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:57.726 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.725+0000 7fe7d2400640 1 --2- 172.21.15.152:0/2846724494 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe7cc163230 0x7fe7cc1f7bc0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7fe7c4009870 tx=0x7fe7c40046c0 comp rx=0 tx=0).stop 2024-04-20T11:34:57.726 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.725+0000 7fe7d2400640 1 -- 172.21.15.152:0/2846724494 shutdown_connections 2024-04-20T11:34:57.726 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.725+0000 7fe7d2400640 1 --2- 172.21.15.152:0/2846724494 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe7a806ce90 0x7fe7a806f350 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:57.726 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.725+0000 7fe7d2400640 1 --2- 172.21.15.152:0/2846724494 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe7cc163230 0x7fe7cc1f7bc0 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:57.726 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.725+0000 7fe7d2400640 1 --2- 172.21.15.152:0/2846724494 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7cc15eb50 0x7fe7cc1f4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:57.726 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.725+0000 7fe7d2400640 1 --2- 172.21.15.152:0/2846724494 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7cc0d35a0 0x7fe7cc1f5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:57.726 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.725+0000 7fe7d2400640 1 -- 172.21.15.152:0/2846724494 >> 172.21.15.152:0/2846724494 conn(0x7fe7cc08bfd0 msgr2=0x7fe7cc0d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:57.726 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.725+0000 7fe7d2400640 1 -- 172.21.15.152:0/2846724494 shutdown_connections 2024-04-20T11:34:57.726 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.725+0000 7fe7d2400640 1 -- 172.21.15.152:0/2846724494 wait complete. 2024-04-20T11:34:57.743 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T11:34:57.951 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.950+0000 7fc61a000640 1 Processor -- start 2024-04-20T11:34:57.951 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.950+0000 7fc61a000640 1 -- start start 2024-04-20T11:34:57.953 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.952+0000 7fc61a000640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fc6140d4ee0 0x7fc614160940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:57.953 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.952+0000 7fc61a000640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6140d5390 con 0x7fc6140cf5b0 2024-04-20T11:34:57.953 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.952+0000 7fc61a000640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6140d2b70 con 0x7fc6140ceeb0 2024-04-20T11:34:57.953 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.952+0000 7fc61a000640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fc6140d2e50 con 0x7fc6140d4ee0 2024-04-20T11:34:57.953 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.952+0000 7fc618c00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fc6140cf5b0 0x7fc6140cf990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:58696/0 (socket says 172.21.15.152:58696) 2024-04-20T11:34:57.953 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.952+0000 7fc618c00640 1 -- 172.21.15.152:0/1073976453 learned_addr learned my addr 172.21.15.152:0/1073976453 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:34:57.955 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.953+0000 7fc619600640 1 --2- 172.21.15.152:0/1073976453 >> v2:172.21.15.175:3300/0 conn(0x7fc6140d4ee0 0x7fc614160940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:57.955 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.953+0000 7fc613400640 1 -- 172.21.15.152:0/1073976453 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1841874573 0 0) 0x7fc6140d5390 con 0x7fc6140cf5b0 2024-04-20T11:34:57.955 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.953+0000 7fc613400640 1 -- 172.21.15.152:0/1073976453 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc5f8003520 con 0x7fc6140cf5b0 2024-04-20T11:34:57.955 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.953+0000 7fc613400640 1 -- 172.21.15.152:0/1073976453 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1588328732 0 0) 0x7fc5f8003520 con 0x7fc6140cf5b0 2024-04-20T11:34:57.955 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.953+0000 7fc613400640 1 -- 172.21.15.152:0/1073976453 >> v2:172.21.15.175:3300/0 conn(0x7fc6140d4ee0 msgr2=0x7fc614160940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:57.955 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.953+0000 7fc613400640 1 --2- 172.21.15.152:0/1073976453 >> v2:172.21.15.175:3300/0 conn(0x7fc6140d4ee0 0x7fc614160940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:57.955 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.953+0000 7fc613400640 1 -- 172.21.15.152:0/1073976453 >> v1:172.21.15.175:6789/0 conn(0x7fc6140ceeb0 legacy=0x7fc614160230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:57.955 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.953+0000 7fc613400640 1 -- 172.21.15.152:0/1073976453 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6140d3130 con 0x7fc6140cf5b0 2024-04-20T11:34:57.955 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.953+0000 7fc613400640 1 -- 172.21.15.152:0/1073976453 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7fc5fc001670 con 0x7fc6140cf5b0 2024-04-20T11:34:57.955 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.953+0000 7fc613400640 1 -- 172.21.15.152:0/1073976453 >> v1:172.21.15.152:6789/0 conn(0x7fc6140cf5b0 legacy=0x7fc6140cf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:57.955 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.953+0000 7fc613400640 1 --2- 172.21.15.152:0/1073976453 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5f8003a00 0x7fc5f8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:57.956 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.953+0000 7fc613400640 1 --2- 172.21.15.152:0/1073976453 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc5f8004800 0x7fc5f8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:57.956 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.953+0000 7fc613400640 1 --2- 172.21.15.152:0/1073976453 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc5f80051c0 0x7fc5f800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:57.956 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.953+0000 7fc613400640 1 -- 172.21.15.152:0/1073976453 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fc6140d2e50 con 0x7fc5f80051c0 2024-04-20T11:34:57.956 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.953+0000 7fc613400640 1 -- 172.21.15.152:0/1073976453 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc5f800c0a0 con 0x7fc5f8003a00 2024-04-20T11:34:57.956 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.953+0000 7fc613400640 1 -- 172.21.15.152:0/1073976453 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc5f800c2a0 con 0x7fc5f8004800 2024-04-20T11:34:57.956 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.954+0000 7fc619600640 1 --2- 172.21.15.152:0/1073976453 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc5f80051c0 0x7fc5f800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:57.956 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.954+0000 7fc618c00640 1 --2- 172.21.15.152:0/1073976453 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5f8003a00 0x7fc5f8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:57.956 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.954+0000 7fc619600640 1 -- 172.21.15.152:0/1073976453 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc5f8004800 msgr2=0x7fc5f8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:57.956 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.954+0000 7fc619600640 1 --2- 172.21.15.152:0/1073976453 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc5f8004800 0x7fc5f8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:57.956 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.954+0000 7fc619600640 1 -- 172.21.15.152:0/1073976453 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5f8003a00 msgr2=0x7fc5f8003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:57.956 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.954+0000 7fc619600640 1 --2- 172.21.15.152:0/1073976453 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5f8003a00 0x7fc5f8003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:57.956 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.954+0000 7fc619600640 1 -- 172.21.15.152:0/1073976453 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5f8003520 con 0x7fc5f80051c0 2024-04-20T11:34:57.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.955+0000 7fc619600640 1 -- 172.21.15.152:0/1073976453 reap_dead start 2024-04-20T11:34:57.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.955+0000 7fc619600640 1 --2- 172.21.15.152:0/1073976453 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc5f80051c0 0x7fc5f800b9c0 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7fc60c009ac0 tx=0x7fc5f8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fd9a7461584d6cc server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:57.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.955+0000 7fc613400640 1 -- 172.21.15.152:0/1073976453 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc60c009690 con 0x7fc5f80051c0 2024-04-20T11:34:57.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.956+0000 7fc613400640 1 -- 172.21.15.152:0/1073976453 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc60c004020 con 0x7fc5f80051c0 2024-04-20T11:34:57.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.956+0000 7fc613400640 1 -- 172.21.15.152:0/1073976453 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc60c0043d0 con 0x7fc5f80051c0 2024-04-20T11:34:57.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.956+0000 7fc61a000640 1 -- 172.21.15.152:0/1073976453 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc5f80051c0 msgr2=0x7fc5f800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:57.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.956+0000 7fc61a000640 1 --2- 172.21.15.152:0/1073976453 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc5f80051c0 0x7fc5f800b9c0 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7fc60c009ac0 tx=0x7fc5f8005830 comp rx=0 tx=0).stop 2024-04-20T11:34:57.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.956+0000 7fc61a000640 1 -- 172.21.15.152:0/1073976453 shutdown_connections 2024-04-20T11:34:57.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.956+0000 7fc61a000640 1 --2- 172.21.15.152:0/1073976453 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc5f80051c0 0x7fc5f800b9c0 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:57.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.956+0000 7fc61a000640 1 -- 172.21.15.152:0/1073976453 >> 172.21.15.152:0/1073976453 conn(0x7fc61408bfd0 msgr2=0x7fc6140d4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:57.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.956+0000 7fc61a000640 1 -- 172.21.15.152:0/1073976453 shutdown_connections 2024-04-20T11:34:57.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.956+0000 7fc61a000640 1 -- 172.21.15.152:0/1073976453 wait complete. 2024-04-20T11:34:57.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.958+0000 7fc61a000640 1 Processor -- start 2024-04-20T11:34:57.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.958+0000 7fc61a000640 1 -- start start 2024-04-20T11:34:57.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.959+0000 7fc61a000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5f80051c0 0x7fc6140cdff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:57.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.959+0000 7fc61a000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc6140b7c10 0x7fc614156390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:57.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.959+0000 7fc61a000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc6140b9440 0x7fc6140b98c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:57.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.959+0000 7fc61a000640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fc6140d3c10 con 0x7fc6140b7c10 2024-04-20T11:34:57.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.959+0000 7fc61a000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc6140d36f0 con 0x7fc5f80051c0 2024-04-20T11:34:57.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.959+0000 7fc61a000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc6140d2790 con 0x7fc6140b9440 2024-04-20T11:34:57.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.959+0000 7fc613e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc6140b7c10 0x7fc614156390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:57.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.959+0000 7fc613e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc6140b7c10 0x7fc614156390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:42642/0 (socket says 172.21.15.152:42642) 2024-04-20T11:34:57.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.959+0000 7fc613e00640 1 -- 172.21.15.152:0/617902602 learned_addr learned my addr 172.21.15.152:0/617902602 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:57.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.960+0000 7fc618c00640 1 --2- 172.21.15.152:0/617902602 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5f80051c0 0x7fc6140cdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:57.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.960+0000 7fc613e00640 1 -- 172.21.15.152:0/617902602 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc6140b9440 msgr2=0x7fc6140b98c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:57.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.960+0000 7fc613e00640 1 --2- 172.21.15.152:0/617902602 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc6140b9440 0x7fc6140b98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:57.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.960+0000 7fc613e00640 1 -- 172.21.15.152:0/617902602 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5f80051c0 msgr2=0x7fc6140cdff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:57.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.960+0000 7fc613e00640 1 --2- 172.21.15.152:0/617902602 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5f80051c0 0x7fc6140cdff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:57.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.960+0000 7fc613e00640 1 -- 172.21.15.152:0/617902602 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6140cd1d0 con 0x7fc6140b7c10 2024-04-20T11:34:57.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.960+0000 7fc613e00640 1 --2- 172.21.15.152:0/617902602 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc6140b7c10 0x7fc614156390 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7fc60400c8d0 tx=0x7fc60400cd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:57.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.960+0000 7fc610c00640 1 -- 172.21.15.152:0/617902602 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc604003bc0 con 0x7fc6140b7c10 2024-04-20T11:34:57.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.961+0000 7fc610c00640 1 -- 172.21.15.152:0/617902602 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc604003d60 con 0x7fc6140b7c10 2024-04-20T11:34:57.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.961+0000 7fc61a000640 1 -- 172.21.15.152:0/617902602 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6140cd4b0 con 0x7fc6140b7c10 2024-04-20T11:34:57.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.961+0000 7fc610c00640 1 -- 172.21.15.152:0/617902602 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc60400aa90 con 0x7fc6140b7c10 2024-04-20T11:34:57.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.961+0000 7fc61a000640 1 -- 172.21.15.152:0/617902602 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6140b9e00 con 0x7fc6140b7c10 2024-04-20T11:34:57.963 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.962+0000 7fc61a000640 1 -- 172.21.15.152:0/617902602 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc614075e60 con 0x7fc6140b7c10 2024-04-20T11:34:57.963 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.963+0000 7fc610c00640 1 -- 172.21.15.152:0/617902602 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 21) ==== 91550+0+0 (secure 0 0 0) 0x7fc60400ac30 con 0x7fc6140b7c10 2024-04-20T11:34:57.964 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.964+0000 7fc610c00640 1 --2- 172.21.15.152:0/617902602 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fc5f006ce90 0x7fc5f006f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:57.964 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.964+0000 7fc610c00640 1 -- 172.21.15.152:0/617902602 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5765+0+0 (secure 0 0 0) 0x7fc60408c620 con 0x7fc6140b7c10 2024-04-20T11:34:57.965 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.965+0000 7fc618c00640 1 --2- 172.21.15.152:0/617902602 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fc5f006ce90 0x7fc5f006f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:57.966 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.966+0000 7fc618c00640 1 --2- 172.21.15.152:0/617902602 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fc5f006ce90 0x7fc5f006f350 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc001eb0 tx=0x7fc5fc000f30 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:57.969 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:57.969+0000 7fc610c00640 1 -- 172.21.15.152:0/617902602 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fc60405a1a0 con 0x7fc6140b7c10 2024-04-20T11:34:58.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:58.167+0000 7fc61a000640 1 -- 172.21.15.152:0/617902602 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc6140cf390 con 0x7fc6140b7c10 2024-04-20T11:34:58.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:58.168+0000 7fc610c00640 1 -- 172.21.15.152:0/617902602 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v119) ==== 100+0+11 (secure 0 0 0) 0x7fc604059aa0 con 0x7fc6140b7c10 2024-04-20T11:34:58.168 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 64 2024-04-20T11:34:58.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:58.170+0000 7fc61a000640 1 -- 172.21.15.152:0/617902602 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fc5f006ce90 msgr2=0x7fc5f006f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:58.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:58.170+0000 7fc61a000640 1 --2- 172.21.15.152:0/617902602 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fc5f006ce90 0x7fc5f006f350 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc001eb0 tx=0x7fc5fc000f30 comp rx=0 tx=0).stop 2024-04-20T11:34:58.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:58.170+0000 7fc61a000640 1 -- 172.21.15.152:0/617902602 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc6140b7c10 msgr2=0x7fc614156390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:58.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:58.170+0000 7fc61a000640 1 --2- 172.21.15.152:0/617902602 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc6140b7c10 0x7fc614156390 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7fc60400c8d0 tx=0x7fc60400cd90 comp rx=0 tx=0).stop 2024-04-20T11:34:58.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:58.170+0000 7fc61a000640 1 -- 172.21.15.152:0/617902602 shutdown_connections 2024-04-20T11:34:58.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:58.170+0000 7fc61a000640 1 --2- 172.21.15.152:0/617902602 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fc5f006ce90 0x7fc5f006f350 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:58.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:58.170+0000 7fc61a000640 1 --2- 172.21.15.152:0/617902602 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc6140b9440 0x7fc6140b98c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:58.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:58.170+0000 7fc61a000640 1 --2- 172.21.15.152:0/617902602 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc6140b7c10 0x7fc614156390 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:58.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:58.170+0000 7fc61a000640 1 --2- 172.21.15.152:0/617902602 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5f80051c0 0x7fc6140cdff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:58.172 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:58.170+0000 7fc61a000640 1 -- 172.21.15.152:0/617902602 >> 172.21.15.152:0/617902602 conn(0x7fc61408bfd0 msgr2=0x7fc61415d0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:58.172 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:58.170+0000 7fc61a000640 1 -- 172.21.15.152:0/617902602 shutdown_connections 2024-04-20T11:34:58.172 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:58.170+0000 7fc61a000640 1 -- 172.21.15.152:0/617902602 wait complete. 2024-04-20T11:34:58.190 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-20T11:34:58.394 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:58.393+0000 7f3455600640 1 Processor -- start 2024-04-20T11:34:58.394 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:58.394+0000 7f3455600640 1 -- start start 2024-04-20T11:34:58.396 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:58.395+0000 7f3455600640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f34500d0930 0x7f345006de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:58.396 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:58.395+0000 7f3455600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f34500afcc0 0x7f345015ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:58.396 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:58.395+0000 7f3455600640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34500af7e0 con 0x7f345015c460 2024-04-20T11:34:58.396 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:58.395+0000 7f3455600640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f34501624c0 con 0x7f34500d0930 2024-04-20T11:34:58.396 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:58.395+0000 7f3455600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f3450162710 con 0x7f34500afcc0 2024-04-20T11:34:58.396 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:58.396+0000 7f3447e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f34500d0930 0x7f345006de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:58.396 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:58.396+0000 7f3447e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f34500d0930 0x7f345006de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:42656/0 (socket says 172.21.15.152:42656) 2024-04-20T11:34:58.397 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:58.396+0000 7f3447e00640 1 -- 172.21.15.152:0/3592924804 learned_addr learned my addr 172.21.15.152:0/3592924804 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:58.397 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:58.396+0000 7f344ea00640 1 --1- 172.21.15.152:0/3592924804 >> v1:172.21.15.152:6789/0 conn(0x7f345015c460 0x7f34500d0220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:58700/0 (socket says 172.21.15.152:58700) 2024-04-20T11:34:58.397 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:58.396+0000 7f3447e00640 1 -- 172.21.15.152:0/3592924804 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f34500afcc0 msgr2=0x7f345015ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:58.397 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:58.396+0000 7f344f400640 1 --2- 172.21.15.152:0/3592924804 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f34500afcc0 0x7f345015ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:58.397 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:58.396+0000 7f3447e00640 1 --2- 172.21.15.152:0/3592924804 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f34500afcc0 0x7f345015ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:58.397 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:58.396+0000 7f3447e00640 1 -- 172.21.15.152:0/3592924804 >> v1:172.21.15.152:6789/0 conn(0x7f345015c460 legacy=0x7f34500d0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:58.397 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:58.396+0000 7f344e000640 1 -- 172.21.15.152:0/3592924804 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1434125381 0 0) 0x7f34500af7e0 con 0x7f345015c460 2024-04-20T11:34:58.398 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:58.396+0000 7f3447e00640 1 -- 172.21.15.152:0/3592924804 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34501629f0 con 0x7f34500d0930 2024-04-20T11:34:58.398 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:58.396+0000 7f344f400640 1 --2- 172.21.15.152:0/3592924804 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f34500afcc0 0x7f345015ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:34:58.398 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:58.397+0000 7f3447e00640 1 --2- 172.21.15.152:0/3592924804 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f34500d0930 0x7f345006de10 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f3438009870 tx=0x7f343802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1bb1d7aa02ef868 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:58.398 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:58.398+0000 7f344e000640 1 -- 172.21.15.152:0/3592924804 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34380317c0 con 0x7f34500d0930 2024-04-20T11:34:58.398 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:58.398+0000 7f344e000640 1 -- 172.21.15.152:0/3592924804 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3438037070 con 0x7f34500d0930 2024-04-20T11:34:58.399 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:58.398+0000 7f344e000640 1 -- 172.21.15.152:0/3592924804 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3438031b80 con 0x7f34500d0930 2024-04-20T11:34:58.399 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:58.398+0000 7f3455600640 1 -- 172.21.15.152:0/3592924804 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f34500d0930 msgr2=0x7f345006de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:58.399 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:58.398+0000 7f3455600640 1 --2- 172.21.15.152:0/3592924804 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f34500d0930 0x7f345006de10 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f3438009870 tx=0x7f343802b860 comp rx=0 tx=0).stop 2024-04-20T11:34:58.399 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:58.398+0000 7f3455600640 1 -- 172.21.15.152:0/3592924804 shutdown_connections 2024-04-20T11:34:58.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:58.398+0000 7f3455600640 1 --2- 172.21.15.152:0/3592924804 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f34500afcc0 0x7f345015ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:58.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:58.398+0000 7f3455600640 1 --2- 172.21.15.152:0/3592924804 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f34500d0930 0x7f345006de10 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:58.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:58.398+0000 7f3455600640 1 -- 172.21.15.152:0/3592924804 >> 172.21.15.152:0/3592924804 conn(0x7f3450067cf0 msgr2=0x7f345006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:58.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:58.399+0000 7f3455600640 1 -- 172.21.15.152:0/3592924804 shutdown_connections 2024-04-20T11:34:58.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:58.399+0000 7f3455600640 1 -- 172.21.15.152:0/3592924804 wait complete. 2024-04-20T11:34:58.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:58.400+0000 7f3455600640 1 Processor -- start 2024-04-20T11:34:58.401 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:58.400+0000 7f3455600640 1 -- start start 2024-04-20T11:34:58.403 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:58.401+0000 7f3455600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f34500afcc0 0x7f34501f81b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:58.403 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:58.401+0000 7f3455600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f34500d0930 0x7f34501f4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:58.403 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:58.401+0000 7f3455600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f345015c460 0x7f345006e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:58.403 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:58.401+0000 7f3455600640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f34500af490 con 0x7f34500d0930 2024-04-20T11:34:58.403 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:58.401+0000 7f3455600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f345015c840 con 0x7f34500afcc0 2024-04-20T11:34:58.403 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:58.401+0000 7f3455600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f34501631f0 con 0x7f345015c460 2024-04-20T11:34:58.403 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:58.402+0000 7f3447e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f34500d0930 0x7f34501f4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:58.403 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:58.402+0000 7f3447e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f34500d0930 0x7f34501f4180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:42658/0 (socket says 172.21.15.152:42658) 2024-04-20T11:34:58.404 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:58.402+0000 7f3447e00640 1 -- 172.21.15.152:0/1846211778 learned_addr learned my addr 172.21.15.152:0/1846211778 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:58.404 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:58.402+0000 7f344ea00640 1 --2- 172.21.15.152:0/1846211778 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f34500afcc0 0x7f34501f81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:58.404 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:58.402+0000 7f3447e00640 1 -- 172.21.15.152:0/1846211778 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f345015c460 msgr2=0x7f345006e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:58.404 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:58.402+0000 7f344f400640 1 --2- 172.21.15.152:0/1846211778 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f345015c460 0x7f345006e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:58.404 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:58.402+0000 7f3447e00640 1 --2- 172.21.15.152:0/1846211778 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f345015c460 0x7f345006e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:58.404 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:58.402+0000 7f3447e00640 1 -- 172.21.15.152:0/1846211778 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f34500afcc0 msgr2=0x7f34501f81b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:58.404 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:58.402+0000 7f3447e00640 1 --2- 172.21.15.152:0/1846211778 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f34500afcc0 0x7f34501f81b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:58.404 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:58.402+0000 7f3447e00640 1 -- 172.21.15.152:0/1846211778 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34501f4920 con 0x7f34500d0930 2024-04-20T11:34:58.404 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:58.402+0000 7f344ea00640 1 --2- 172.21.15.152:0/1846211778 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f34500afcc0 0x7f34501f81b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:34:58.404 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:58.403+0000 7f3447e00640 1 --2- 172.21.15.152:0/1846211778 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f34500d0930 0x7f34501f4180 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f3438034040 tx=0x7f34380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:58.405 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:58.403+0000 7f3446a00640 1 -- 172.21.15.152:0/1846211778 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3438004230 con 0x7f34500d0930 2024-04-20T11:34:58.405 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:58.403+0000 7f3446a00640 1 -- 172.21.15.152:0/1846211778 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34380043d0 con 0x7f34500d0930 2024-04-20T11:34:58.405 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:58.403+0000 7f3446a00640 1 -- 172.21.15.152:0/1846211778 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3438003850 con 0x7f34500d0930 2024-04-20T11:34:58.405 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:58.403+0000 7f3455600640 1 -- 172.21.15.152:0/1846211778 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34501f4c00 con 0x7f34500d0930 2024-04-20T11:34:58.405 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:58.403+0000 7f3455600640 1 -- 172.21.15.152:0/1846211778 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f345006ec40 con 0x7f34500d0930 2024-04-20T11:34:58.405 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:58.404+0000 7f3455600640 1 -- 172.21.15.152:0/1846211778 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34500689f0 con 0x7f34500d0930 2024-04-20T11:34:58.406 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:58.405+0000 7f3446a00640 1 -- 172.21.15.152:0/1846211778 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 21) ==== 91550+0+0 (secure 0 0 0) 0x7f343804c060 con 0x7f34500d0930 2024-04-20T11:34:58.406 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:58.406+0000 7f3446a00640 1 --2- 172.21.15.152:0/1846211778 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f342c06cec0 0x7f342c06f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:58.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:58.406+0000 7f3446a00640 1 -- 172.21.15.152:0/1846211778 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5765+0+0 (secure 0 0 0) 0x7f34380ae1e0 con 0x7f34500d0930 2024-04-20T11:34:58.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:58.407+0000 7f344ea00640 1 --2- 172.21.15.152:0/1846211778 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f342c06cec0 0x7f342c06f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:58.408 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:58.408+0000 7f344ea00640 1 --2- 172.21.15.152:0/1846211778 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f342c06cec0 0x7f342c06f380 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f3440000f60 tx=0x7f3440009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:58.411 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:58.411+0000 7f3446a00640 1 -- 172.21.15.152:0/1846211778 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f3438040070 con 0x7f34500d0930 2024-04-20T11:34:58.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:58.629+0000 7f3455600640 1 -- 172.21.15.152:0/1846211778 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f345006f160 con 0x7f34500d0930 2024-04-20T11:34:58.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:58.630+0000 7f3446a00640 1 -- 172.21.15.152:0/1846211778 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v119) ==== 80+0+0 (secure 0 0 0) 0x7f343807bd60 con 0x7f34500d0930 2024-04-20T11:34:58.630 INFO:teuthology.orchestra.run.smithi152.stderr:listed 0 entries 2024-04-20T11:34:58.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:58.632+0000 7f3455600640 1 -- 172.21.15.152:0/1846211778 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f342c06cec0 msgr2=0x7f342c06f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:58.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:58.633+0000 7f3455600640 1 --2- 172.21.15.152:0/1846211778 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f342c06cec0 0x7f342c06f380 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f3440000f60 tx=0x7f3440009040 comp rx=0 tx=0).stop 2024-04-20T11:34:58.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:58.633+0000 7f3455600640 1 -- 172.21.15.152:0/1846211778 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f34500d0930 msgr2=0x7f34501f4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:58.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:58.633+0000 7f3455600640 1 --2- 172.21.15.152:0/1846211778 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f34500d0930 0x7f34501f4180 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f3438034040 tx=0x7f34380046c0 comp rx=0 tx=0).stop 2024-04-20T11:34:58.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:58.633+0000 7f3455600640 1 -- 172.21.15.152:0/1846211778 shutdown_connections 2024-04-20T11:34:58.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:58.633+0000 7f3455600640 1 --2- 172.21.15.152:0/1846211778 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f342c06cec0 0x7f342c06f380 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:58.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:58.633+0000 7f3455600640 1 --2- 172.21.15.152:0/1846211778 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f345015c460 0x7f345006e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:58.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:58.633+0000 7f3455600640 1 --2- 172.21.15.152:0/1846211778 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f34500d0930 0x7f34501f4180 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:58.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:58.633+0000 7f3455600640 1 --2- 172.21.15.152:0/1846211778 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f34500afcc0 0x7f34501f81b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:58.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:58.633+0000 7f3455600640 1 -- 172.21.15.152:0/1846211778 >> 172.21.15.152:0/1846211778 conn(0x7f3450067cf0 msgr2=0x7f34501577a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:58.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:58.633+0000 7f3455600640 1 -- 172.21.15.152:0/1846211778 shutdown_connections 2024-04-20T11:34:58.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:58.633+0000 7f3455600640 1 -- 172.21.15.152:0/1846211778 wait complete. 2024-04-20T11:34:58.652 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi152.front.sepia.ceph.com... 2024-04-20T11:34:58.652 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:34:58.652 DEBUG:teuthology.orchestra.run.smithi152:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-20T11:34:58.729 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi152.stderr:ceph-fuse[60265]: fuse finished with error 0 and tester_r 0 2024-04-20T11:34:58.733 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi152.front.sepia.ceph.com 2024-04-20T11:34:58.733 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:34:58.734 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-20T11:34:58.936 INFO:tasks.cephfs.mount:Terminating background process 2024-04-20T11:34:58.936 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T11:34:58.936 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:34:58.988 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:34:59.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.190+0000 7fdd6e600640 1 Processor -- start 2024-04-20T11:34:59.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.190+0000 7fdd6e600640 1 -- start start 2024-04-20T11:34:59.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.191+0000 7fdd6e600640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fdd681414e0 0x7fdd68166360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:59.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.191+0000 7fdd6e600640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd680cf0a0 con 0x7fdd6813e530 2024-04-20T11:34:59.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.191+0000 7fdd6e600640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd680d00e0 con 0x7fdd6806f510 2024-04-20T11:34:59.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.191+0000 7fdd6e600640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fdd68141b00 con 0x7fdd681414e0 2024-04-20T11:34:59.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.192+0000 7fdd6d200640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fdd6813e530 0x7fdd6806ee00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:58706/0 (socket says 172.21.15.152:58706) 2024-04-20T11:34:59.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.192+0000 7fdd6d200640 1 -- 172.21.15.152:0/415461382 learned_addr learned my addr 172.21.15.152:0/415461382 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:34:59.195 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.192+0000 7fdd67e00640 1 -- 172.21.15.152:0/415461382 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 824749746 0 0) 0x7fdd58001110 con 0x7fdd6813e530 2024-04-20T11:34:59.195 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.192+0000 7fdd67e00640 1 -- 172.21.15.152:0/415461382 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdd4c003520 con 0x7fdd6813e530 2024-04-20T11:34:59.195 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.192+0000 7fdd6dc00640 1 --2- 172.21.15.152:0/415461382 >> v2:172.21.15.175:3300/0 conn(0x7fdd681414e0 0x7fdd68166360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:59.195 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.192+0000 7fdd67e00640 1 -- 172.21.15.152:0/415461382 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1349291542 0 0) 0x7fdd4c003520 con 0x7fdd6813e530 2024-04-20T11:34:59.195 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.192+0000 7fdd67e00640 1 -- 172.21.15.152:0/415461382 >> v2:172.21.15.175:3300/0 conn(0x7fdd681414e0 msgr2=0x7fdd68166360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:59.195 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.192+0000 7fdd67e00640 1 --2- 172.21.15.152:0/415461382 >> v2:172.21.15.175:3300/0 conn(0x7fdd681414e0 0x7fdd68166360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:59.195 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.193+0000 7fdd67e00640 1 -- 172.21.15.152:0/415461382 >> v1:172.21.15.175:6789/0 conn(0x7fdd6806f510 legacy=0x7fdd68140dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:59.195 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.193+0000 7fdd67e00640 1 -- 172.21.15.152:0/415461382 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd680cf320 con 0x7fdd6813e530 2024-04-20T11:34:59.195 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.193+0000 7fdd67e00640 1 -- 172.21.15.152:0/415461382 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7fdd58001840 con 0x7fdd6813e530 2024-04-20T11:34:59.195 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.193+0000 7fdd67e00640 1 -- 172.21.15.152:0/415461382 >> v1:172.21.15.152:6789/0 conn(0x7fdd6813e530 legacy=0x7fdd6806ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:59.195 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.193+0000 7fdd67e00640 1 --2- 172.21.15.152:0/415461382 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdd4c003a00 0x7fdd4c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:59.195 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.193+0000 7fdd67e00640 1 --2- 172.21.15.152:0/415461382 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd4c004800 0x7fdd4c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:59.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.193+0000 7fdd67e00640 1 --2- 172.21.15.152:0/415461382 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd4c005210 0x7fdd4c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:59.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.193+0000 7fdd67e00640 1 -- 172.21.15.152:0/415461382 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fdd4c00c130 con 0x7fdd4c003a00 2024-04-20T11:34:59.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.193+0000 7fdd67e00640 1 -- 172.21.15.152:0/415461382 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdd4c00c2e0 con 0x7fdd4c005210 2024-04-20T11:34:59.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.193+0000 7fdd67e00640 1 -- 172.21.15.152:0/415461382 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fdd4c00c4e0 con 0x7fdd4c004800 2024-04-20T11:34:59.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.193+0000 7fdd6d200640 1 --2- 172.21.15.152:0/415461382 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdd4c003a00 0x7fdd4c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:59.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.193+0000 7fdd5fe00640 1 --2- 172.21.15.152:0/415461382 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd4c004800 0x7fdd4c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:59.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.193+0000 7fdd6d200640 1 -- 172.21.15.152:0/415461382 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd4c004800 msgr2=0x7fdd4c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:59.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.193+0000 7fdd6dc00640 1 --2- 172.21.15.152:0/415461382 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd4c005210 0x7fdd4c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:59.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.193+0000 7fdd6d200640 1 --2- 172.21.15.152:0/415461382 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd4c004800 0x7fdd4c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:59.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.193+0000 7fdd6d200640 1 -- 172.21.15.152:0/415461382 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd4c005210 msgr2=0x7fdd4c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:59.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.193+0000 7fdd6d200640 1 --2- 172.21.15.152:0/415461382 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd4c005210 0x7fdd4c00b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:59.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.193+0000 7fdd6d200640 1 -- 172.21.15.152:0/415461382 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd4c003520 con 0x7fdd4c003a00 2024-04-20T11:34:59.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.193+0000 7fdd6dc00640 1 -- 172.21.15.152:0/415461382 reap_dead start 2024-04-20T11:34:59.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.193+0000 7fdd5fe00640 1 --2- 172.21.15.152:0/415461382 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd4c004800 0x7fdd4c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:34:59.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.195+0000 7fdd6d200640 1 --2- 172.21.15.152:0/415461382 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdd4c003a00 0x7fdd4c003e20 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7fdd5800c320 tx=0x7fdd4c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5136e5eb118710d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:59.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.195+0000 7fdd67e00640 1 -- 172.21.15.152:0/415461382 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd58007e00 con 0x7fdd4c003a00 2024-04-20T11:34:59.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.195+0000 7fdd67e00640 1 -- 172.21.15.152:0/415461382 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd58030070 con 0x7fdd4c003a00 2024-04-20T11:34:59.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.195+0000 7fdd67e00640 1 -- 172.21.15.152:0/415461382 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd580058d0 con 0x7fdd4c003a00 2024-04-20T11:34:59.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.195+0000 7fdd6e600640 1 -- 172.21.15.152:0/415461382 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdd4c003a00 msgr2=0x7fdd4c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:59.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.195+0000 7fdd6e600640 1 --2- 172.21.15.152:0/415461382 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdd4c003a00 0x7fdd4c003e20 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7fdd5800c320 tx=0x7fdd4c005830 comp rx=0 tx=0).stop 2024-04-20T11:34:59.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.195+0000 7fdd6e600640 1 -- 172.21.15.152:0/415461382 shutdown_connections 2024-04-20T11:34:59.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.195+0000 7fdd6e600640 1 --2- 172.21.15.152:0/415461382 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdd4c003a00 0x7fdd4c003e20 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:59.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.195+0000 7fdd6e600640 1 -- 172.21.15.152:0/415461382 >> 172.21.15.152:0/415461382 conn(0x7fdd68067ae0 msgr2=0x7fdd6806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:59.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.196+0000 7fdd6e600640 1 -- 172.21.15.152:0/415461382 shutdown_connections 2024-04-20T11:34:59.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.196+0000 7fdd6e600640 1 -- 172.21.15.152:0/415461382 wait complete. 2024-04-20T11:34:59.199 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.197+0000 7fdd6e600640 1 Processor -- start 2024-04-20T11:34:59.199 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.198+0000 7fdd6e600640 1 -- start start 2024-04-20T11:34:59.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.198+0000 7fdd6e600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdd4c003a00 0x7fdd6813cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:59.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.198+0000 7fdd6e600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd6813db40 0x7fdd68155030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:59.201 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.198+0000 7fdd6e600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd6813d4d0 0x7fdd68141b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:59.201 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.198+0000 7fdd6e600640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fdd680cf8e0 con 0x7fdd4c003a00 2024-04-20T11:34:59.201 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.198+0000 7fdd6e600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdd680cfc50 con 0x7fdd6813d4d0 2024-04-20T11:34:59.201 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.198+0000 7fdd6e600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fdd68140dd0 con 0x7fdd6813db40 2024-04-20T11:34:59.201 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.199+0000 7fdd6d200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdd4c003a00 0x7fdd6813cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:59.201 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.199+0000 7fdd6d200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdd4c003a00 0x7fdd6813cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:42664/0 (socket says 172.21.15.152:42664) 2024-04-20T11:34:59.201 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.199+0000 7fdd6d200640 1 -- 172.21.15.152:0/1347926462 learned_addr learned my addr 172.21.15.152:0/1347926462 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:59.201 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.199+0000 7fdd6dc00640 1 --2- 172.21.15.152:0/1347926462 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd6813d4d0 0x7fdd68141b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:59.201 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.199+0000 7fdd5fe00640 1 --2- 172.21.15.152:0/1347926462 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd6813db40 0x7fdd68155030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:59.201 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.199+0000 7fdd6d200640 1 -- 172.21.15.152:0/1347926462 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd6813db40 msgr2=0x7fdd68155030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:59.201 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.199+0000 7fdd6d200640 1 --2- 172.21.15.152:0/1347926462 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd6813db40 0x7fdd68155030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:59.201 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.199+0000 7fdd6d200640 1 -- 172.21.15.152:0/1347926462 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd6813d4d0 msgr2=0x7fdd68141b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:59.201 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.199+0000 7fdd6d200640 1 --2- 172.21.15.152:0/1347926462 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd6813d4d0 0x7fdd68141b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:59.201 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.199+0000 7fdd6d200640 1 -- 172.21.15.152:0/1347926462 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd68154270 con 0x7fdd4c003a00 2024-04-20T11:34:59.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.199+0000 7fdd5fe00640 1 --2- 172.21.15.152:0/1347926462 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd6813db40 0x7fdd68155030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:34:59.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.199+0000 7fdd6dc00640 1 --2- 172.21.15.152:0/1347926462 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd6813d4d0 0x7fdd68141b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:34:59.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.200+0000 7fdd6d200640 1 --2- 172.21.15.152:0/1347926462 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdd4c003a00 0x7fdd6813cf20 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7fdd5800c410 tx=0x7fdd6815b9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:59.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.200+0000 7fdd65600640 1 -- 172.21.15.152:0/1347926462 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd58007e00 con 0x7fdd4c003a00 2024-04-20T11:34:59.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.200+0000 7fdd65600640 1 -- 172.21.15.152:0/1347926462 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd58000ed0 con 0x7fdd4c003a00 2024-04-20T11:34:59.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.200+0000 7fdd65600640 1 -- 172.21.15.152:0/1347926462 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd58007470 con 0x7fdd4c003a00 2024-04-20T11:34:59.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.200+0000 7fdd6e600640 1 -- 172.21.15.152:0/1347926462 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd68154550 con 0x7fdd4c003a00 2024-04-20T11:34:59.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.200+0000 7fdd6e600640 1 -- 172.21.15.152:0/1347926462 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd68154b10 con 0x7fdd4c003a00 2024-04-20T11:34:59.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.201+0000 7fdd6e600640 1 -- 172.21.15.152:0/1347926462 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd68075e60 con 0x7fdd4c003a00 2024-04-20T11:34:59.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.202+0000 7fdd65600640 1 -- 172.21.15.152:0/1347926462 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 21) ==== 91550+0+0 (secure 0 0 0) 0x7fdd58048020 con 0x7fdd4c003a00 2024-04-20T11:34:59.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.203+0000 7fdd65600640 1 --2- 172.21.15.152:0/1347926462 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fdd4406cec0 0x7fdd4406f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:59.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.203+0000 7fdd65600640 1 -- 172.21.15.152:0/1347926462 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5765+0+0 (secure 0 0 0) 0x7fdd580ad6e0 con 0x7fdd4c003a00 2024-04-20T11:34:59.204 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.204+0000 7fdd5fe00640 1 --2- 172.21.15.152:0/1347926462 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fdd4406cec0 0x7fdd4406f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:59.205 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.205+0000 7fdd5fe00640 1 --2- 172.21.15.152:0/1347926462 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fdd4406cec0 0x7fdd4406f380 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fdd68144730 tx=0x7fdd50009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:59.208 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.208+0000 7fdd65600640 1 -- 172.21.15.152:0/1347926462 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fdd680cf0a0 con 0x7fdd4c003a00 2024-04-20T11:34:59.409 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.409+0000 7fdd6e600640 1 -- 172.21.15.152:0/1347926462 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdd680b3490 con 0x7fdd4c003a00 2024-04-20T11:34:59.411 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.411+0000 7fdd65600640 1 -- 172.21.15.152:0/1347926462 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v119) ==== 74+0+16850 (secure 0 0 0) 0x7fdd5807b550 con 0x7fdd4c003a00 2024-04-20T11:34:59.412 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:34:59.412 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":119,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:34:56.181185+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:34:33.781054+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-04-20T11:34:35.031728+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":119,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:34:59.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.413+0000 7fdd6e600640 1 -- 172.21.15.152:0/1347926462 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fdd4406cec0 msgr2=0x7fdd4406f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:59.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.413+0000 7fdd6e600640 1 --2- 172.21.15.152:0/1347926462 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fdd4406cec0 0x7fdd4406f380 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fdd68144730 tx=0x7fdd50009040 comp rx=0 tx=0).stop 2024-04-20T11:34:59.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.413+0000 7fdd6e600640 1 -- 172.21.15.152:0/1347926462 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdd4c003a00 msgr2=0x7fdd6813cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:59.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.413+0000 7fdd6e600640 1 --2- 172.21.15.152:0/1347926462 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdd4c003a00 0x7fdd6813cf20 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7fdd5800c410 tx=0x7fdd6815b9e0 comp rx=0 tx=0).stop 2024-04-20T11:34:59.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.413+0000 7fdd6e600640 1 -- 172.21.15.152:0/1347926462 shutdown_connections 2024-04-20T11:34:59.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.413+0000 7fdd6e600640 1 --2- 172.21.15.152:0/1347926462 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fdd4406cec0 0x7fdd4406f380 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:59.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.414+0000 7fdd6e600640 1 --2- 172.21.15.152:0/1347926462 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd6813d4d0 0x7fdd68141b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:59.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.414+0000 7fdd6e600640 1 --2- 172.21.15.152:0/1347926462 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd6813db40 0x7fdd68155030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:59.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.414+0000 7fdd6e600640 1 --2- 172.21.15.152:0/1347926462 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdd4c003a00 0x7fdd6813cf20 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:59.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.414+0000 7fdd6e600640 1 -- 172.21.15.152:0/1347926462 >> 172.21.15.152:0/1347926462 conn(0x7fdd68067ae0 msgr2=0x7fdd6815ce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:59.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.414+0000 7fdd6e600640 1 -- 172.21.15.152:0/1347926462 shutdown_connections 2024-04-20T11:34:59.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.414+0000 7fdd6e600640 1 -- 172.21.15.152:0/1347926462 wait complete. 2024-04-20T11:34:59.433 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T11:26:47.246571+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T11:34:33.781054+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '117', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T11:34:35.031728+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '119', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 5, 'snap_epoch': 119, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T11:34:59.434 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T11:34:59.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.644+0000 7f0e49a00640 1 Processor -- start 2024-04-20T11:34:59.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.644+0000 7f0e49a00640 1 -- start start 2024-04-20T11:34:59.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.645+0000 7f0e49a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0e44152450 0x7f0e4415c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:59.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.645+0000 7f0e49a00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f0e440b3520 0x7f0e440ae470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:59.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.645+0000 7f0e49a00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e44152830 con 0x7f0e440b30c0 2024-04-20T11:34:59.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.646+0000 7f0e49a00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f0e440b39c0 con 0x7f0e440b3520 2024-04-20T11:34:59.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.646+0000 7f0e49a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0e440b3ca0 con 0x7f0e44152450 2024-04-20T11:34:59.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.646+0000 7f0e43e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0e44152450 0x7f0e4415c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:59.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.646+0000 7f0e49000640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f0e440b3520 0x7f0e440ae470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:59.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.646+0000 7f0e43e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0e44152450 0x7f0e4415c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.152:59834/0 (socket says 172.21.15.152:59834) 2024-04-20T11:34:59.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.646+0000 7f0e43e00640 1 -- 172.21.15.152:0/745736406 learned_addr learned my addr 172.21.15.152:0/745736406 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:59.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.646+0000 7f0e3b400640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7f0e440b30c0 0x7f0e4406bda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.152:43506/0 (socket says 172.21.15.152:43506) 2024-04-20T11:34:59.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.647+0000 7f0e49000640 1 -- 172.21.15.152:0/745736406 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0e44152450 msgr2=0x7f0e4415c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:34:59.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.647+0000 7f0e49000640 1 --2- 172.21.15.152:0/745736406 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0e44152450 0x7f0e4415c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:59.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.647+0000 7f0e49000640 1 -- 172.21.15.152:0/745736406 >> v1:172.21.15.175:6789/0 conn(0x7f0e440b30c0 legacy=0x7f0e4406bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:59.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.647+0000 7f0e49000640 1 -- 172.21.15.152:0/745736406 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e440b3f80 con 0x7f0e440b3520 2024-04-20T11:34:59.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.648+0000 7f0e49000640 1 --2- 172.21.15.152:0/745736406 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e440b3520 0x7f0e440ae470 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c0097a0 tx=0x7f0e3c02b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b885e9a4dd47c329 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:59.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.649+0000 7f0e43400640 1 -- 172.21.15.152:0/745736406 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e3c035020 con 0x7f0e440b3520 2024-04-20T11:34:59.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.649+0000 7f0e43400640 1 -- 172.21.15.152:0/745736406 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e3c031830 con 0x7f0e440b3520 2024-04-20T11:34:59.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.649+0000 7f0e43400640 1 -- 172.21.15.152:0/745736406 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e3c031c00 con 0x7f0e440b3520 2024-04-20T11:34:59.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.650+0000 7f0e49a00640 1 -- 172.21.15.152:0/745736406 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e440b3520 msgr2=0x7f0e440ae470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:59.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.650+0000 7f0e49a00640 1 --2- 172.21.15.152:0/745736406 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e440b3520 0x7f0e440ae470 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c0097a0 tx=0x7f0e3c02b8d0 comp rx=0 tx=0).stop 2024-04-20T11:34:59.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.650+0000 7f0e49a00640 1 -- 172.21.15.152:0/745736406 shutdown_connections 2024-04-20T11:34:59.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.650+0000 7f0e49a00640 1 --2- 172.21.15.152:0/745736406 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e440b3520 0x7f0e440ae470 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:59.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.650+0000 7f0e49a00640 1 --2- 172.21.15.152:0/745736406 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0e44152450 0x7f0e4415c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:59.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.650+0000 7f0e49a00640 1 -- 172.21.15.152:0/745736406 >> 172.21.15.152:0/745736406 conn(0x7f0e440b21e0 msgr2=0x7f0e440b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:59.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.650+0000 7f0e49a00640 1 -- 172.21.15.152:0/745736406 shutdown_connections 2024-04-20T11:34:59.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.650+0000 7f0e49a00640 1 -- 172.21.15.152:0/745736406 wait complete. 2024-04-20T11:34:59.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.652+0000 7f0e49a00640 1 Processor -- start 2024-04-20T11:34:59.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.652+0000 7f0e49a00640 1 -- start start 2024-04-20T11:34:59.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.653+0000 7f0e49a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0e440b30c0 0x7f0e4414d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:59.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.653+0000 7f0e49a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0e440b3520 0x7f0e44153930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:59.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.653+0000 7f0e49a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e44152450 0x7f0e441f4910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:59.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.653+0000 7f0e49a00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f0e4406c4b0 con 0x7f0e440b30c0 2024-04-20T11:34:59.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.653+0000 7f0e49a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0e440adf00 con 0x7f0e44152450 2024-04-20T11:34:59.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.653+0000 7f0e49a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0e4406c630 con 0x7f0e440b3520 2024-04-20T11:34:59.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.653+0000 7f0e43e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0e440b30c0 0x7f0e4414d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:59.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.653+0000 7f0e43e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0e440b30c0 0x7f0e4414d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:42678/0 (socket says 172.21.15.152:42678) 2024-04-20T11:34:59.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.653+0000 7f0e43e00640 1 -- 172.21.15.152:0/1610014877 learned_addr learned my addr 172.21.15.152:0/1610014877 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:34:59.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.653+0000 7f0e49000640 1 --2- 172.21.15.152:0/1610014877 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e44152450 0x7f0e441f4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:59.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.654+0000 7f0e3b400640 1 --2- 172.21.15.152:0/1610014877 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0e440b3520 0x7f0e44153930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:59.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.654+0000 7f0e43e00640 1 -- 172.21.15.152:0/1610014877 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0e440b3520 msgr2=0x7f0e44153930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:59.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.654+0000 7f0e43e00640 1 --2- 172.21.15.152:0/1610014877 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0e440b3520 0x7f0e44153930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:59.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.654+0000 7f0e43e00640 1 -- 172.21.15.152:0/1610014877 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e44152450 msgr2=0x7f0e441f4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:59.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.654+0000 7f0e43e00640 1 --2- 172.21.15.152:0/1610014877 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e44152450 0x7f0e441f4910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:59.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.654+0000 7f0e43e00640 1 -- 172.21.15.152:0/1610014877 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e4415c590 con 0x7f0e440b30c0 2024-04-20T11:34:59.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.654+0000 7f0e43e00640 1 --2- 172.21.15.152:0/1610014877 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0e440b30c0 0x7f0e4414d730 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f0e3400ea90 tx=0x7f0e3400ef50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:59.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.654+0000 7f0e40c00640 1 -- 172.21.15.152:0/1610014877 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e34019480 con 0x7f0e440b30c0 2024-04-20T11:34:59.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.654+0000 7f0e40c00640 1 -- 172.21.15.152:0/1610014877 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e34019620 con 0x7f0e440b30c0 2024-04-20T11:34:59.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.654+0000 7f0e40c00640 1 -- 172.21.15.152:0/1610014877 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e34019a10 con 0x7f0e440b30c0 2024-04-20T11:34:59.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.654+0000 7f0e49a00640 1 -- 172.21.15.152:0/1610014877 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e4415c870 con 0x7f0e440b30c0 2024-04-20T11:34:59.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.655+0000 7f0e49a00640 1 -- 172.21.15.152:0/1610014877 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e4415ce30 con 0x7f0e440b30c0 2024-04-20T11:34:59.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.656+0000 7f0e49a00640 1 -- 172.21.15.152:0/1610014877 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e44075e60 con 0x7f0e440b30c0 2024-04-20T11:34:59.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.657+0000 7f0e40c00640 1 -- 172.21.15.152:0/1610014877 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 21) ==== 91550+0+0 (secure 0 0 0) 0x7f0e34019bb0 con 0x7f0e440b30c0 2024-04-20T11:34:59.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.658+0000 7f0e40c00640 1 --2- 172.21.15.152:0/1610014877 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f0e2006cec0 0x7f0e2006f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:59.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.659+0000 7f0e40c00640 1 -- 172.21.15.152:0/1610014877 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5765+0+0 (secure 0 0 0) 0x7f0e3408d240 con 0x7f0e440b30c0 2024-04-20T11:34:59.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.659+0000 7f0e3b400640 1 --2- 172.21.15.152:0/1610014877 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f0e2006cec0 0x7f0e2006f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:59.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.660+0000 7f0e3b400640 1 --2- 172.21.15.152:0/1610014877 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f0e2006cec0 0x7f0e2006f380 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0e4414f7f0 tx=0x7f0e2c005b70 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:59.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.663+0000 7f0e40c00640 1 -- 172.21.15.152:0/1610014877 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f0e34016330 con 0x7f0e440b30c0 2024-04-20T11:34:59.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.863+0000 7f0e49a00640 1 -- 172.21.15.152:0/1610014877 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0e440b1ba0 con 0x7f0e440b30c0 2024-04-20T11:34:59.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.864+0000 7f0e40c00640 1 -- 172.21.15.152:0/1610014877 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v119) ==== 93+0+10 (secure 0 0 0) 0x7f0e3405ad00 con 0x7f0e440b30c0 2024-04-20T11:34:59.865 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 1 2024-04-20T11:34:59.867 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.867+0000 7f0e49a00640 1 -- 172.21.15.152:0/1610014877 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f0e2006cec0 msgr2=0x7f0e2006f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:59.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.867+0000 7f0e49a00640 1 --2- 172.21.15.152:0/1610014877 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f0e2006cec0 0x7f0e2006f380 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0e4414f7f0 tx=0x7f0e2c005b70 comp rx=0 tx=0).stop 2024-04-20T11:34:59.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.867+0000 7f0e49a00640 1 -- 172.21.15.152:0/1610014877 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0e440b30c0 msgr2=0x7f0e4414d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:59.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.867+0000 7f0e49a00640 1 --2- 172.21.15.152:0/1610014877 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0e440b30c0 0x7f0e4414d730 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f0e3400ea90 tx=0x7f0e3400ef50 comp rx=0 tx=0).stop 2024-04-20T11:34:59.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.867+0000 7f0e49a00640 1 -- 172.21.15.152:0/1610014877 shutdown_connections 2024-04-20T11:34:59.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.867+0000 7f0e49a00640 1 --2- 172.21.15.152:0/1610014877 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f0e2006cec0 0x7f0e2006f380 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:59.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.867+0000 7f0e49a00640 1 --2- 172.21.15.152:0/1610014877 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e44152450 0x7f0e441f4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:59.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.867+0000 7f0e49a00640 1 --2- 172.21.15.152:0/1610014877 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0e440b3520 0x7f0e44153930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:59.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.867+0000 7f0e49a00640 1 --2- 172.21.15.152:0/1610014877 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0e440b30c0 0x7f0e4414d730 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:59.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.867+0000 7f0e49a00640 1 -- 172.21.15.152:0/1610014877 >> 172.21.15.152:0/1610014877 conn(0x7f0e440b21e0 msgr2=0x7f0e4415ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:59.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.867+0000 7f0e49a00640 1 -- 172.21.15.152:0/1610014877 shutdown_connections 2024-04-20T11:34:59.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:34:59.868+0000 7f0e49a00640 1 -- 172.21.15.152:0/1610014877 wait complete. 2024-04-20T11:34:59.888 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T11:35:00.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.093+0000 7f29c7200640 1 Processor -- start 2024-04-20T11:35:00.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.093+0000 7f29c7200640 1 -- start start 2024-04-20T11:35:00.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.094+0000 7f29c7200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f29c806ee00 0x7f29c806f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:00.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.094+0000 7f29c7200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f29c815fbf0 0x7f29c815c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:00.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.094+0000 7f29c7200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29c80d00d0 con 0x7f29c80cf110 2024-04-20T11:35:00.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.094+0000 7f29c7200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f29c806f720 con 0x7f29c815fbf0 2024-04-20T11:35:00.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.094+0000 7f29c7200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f29c80cfb10 con 0x7f29c806ee00 2024-04-20T11:35:00.096 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.095+0000 7f29c5400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f29c815fbf0 0x7f29c815c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:00.096 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.095+0000 7f29c5400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f29c815fbf0 0x7f29c815c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:42680/0 (socket says 172.21.15.152:42680) 2024-04-20T11:35:00.096 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.095+0000 7f29c5400640 1 -- 172.21.15.152:0/1469671982 learned_addr learned my addr 172.21.15.152:0/1469671982 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:00.096 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.095+0000 7f29c5400640 1 -- 172.21.15.152:0/1469671982 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f29c806ee00 msgr2=0x7f29c806f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:00.096 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.095+0000 7f29c5400640 1 --2- 172.21.15.152:0/1469671982 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f29c806ee00 0x7f29c806f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:00.096 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.095+0000 7f29c5400640 1 -- 172.21.15.152:0/1469671982 >> v1:172.21.15.175:6789/0 conn(0x7f29c80cf110 legacy=0x7f29c80b3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:00.096 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.095+0000 7f29c5400640 1 -- 172.21.15.152:0/1469671982 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29c80cfdf0 con 0x7f29c815fbf0 2024-04-20T11:35:00.097 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.096+0000 7f29c5400640 1 --2- 172.21.15.152:0/1469671982 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f29c815fbf0 0x7f29c815c390 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f29b4009870 tx=0x7f29b402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e766991464017cec server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:00.097 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.097+0000 7f29c4a00640 1 -- 172.21.15.152:0/1469671982 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b4035020 con 0x7f29c815fbf0 2024-04-20T11:35:00.097 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.097+0000 7f29c4a00640 1 -- 172.21.15.152:0/1469671982 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29b4031640 con 0x7f29c815fbf0 2024-04-20T11:35:00.097 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.097+0000 7f29c4a00640 1 -- 172.21.15.152:0/1469671982 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b402d040 con 0x7f29c815fbf0 2024-04-20T11:35:00.098 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.097+0000 7f29c7200640 1 -- 172.21.15.152:0/1469671982 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f29c815fbf0 msgr2=0x7f29c815c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:00.098 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.097+0000 7f29c7200640 1 --2- 172.21.15.152:0/1469671982 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f29c815fbf0 0x7f29c815c390 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f29b4009870 tx=0x7f29b402b8f0 comp rx=0 tx=0).stop 2024-04-20T11:35:00.098 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.097+0000 7f29c7200640 1 -- 172.21.15.152:0/1469671982 shutdown_connections 2024-04-20T11:35:00.098 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.097+0000 7f29c7200640 1 --2- 172.21.15.152:0/1469671982 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f29c815fbf0 0x7f29c815c390 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:00.098 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.097+0000 7f29c7200640 1 --2- 172.21.15.152:0/1469671982 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f29c806ee00 0x7f29c806f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:00.098 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.097+0000 7f29c7200640 1 -- 172.21.15.152:0/1469671982 >> 172.21.15.152:0/1469671982 conn(0x7f29c8067ae0 msgr2=0x7f29c806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:00.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.098+0000 7f29c7200640 1 -- 172.21.15.152:0/1469671982 shutdown_connections 2024-04-20T11:35:00.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.098+0000 7f29c7200640 1 -- 172.21.15.152:0/1469671982 wait complete. 2024-04-20T11:35:00.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.099+0000 7f29c7200640 1 Processor -- start 2024-04-20T11:35:00.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.100+0000 7f29c7200640 1 -- start start 2024-04-20T11:35:00.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.100+0000 7f29c7200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f29c806ee00 0x7f29c81e9f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:00.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.100+0000 7f29c7200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f29c80cf110 0x7f29c81e9100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:00.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.100+0000 7f29c7200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f29c815fbf0 0x7f29c81335d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:00.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.101+0000 7f29c7200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f29c815c8d0 con 0x7f29c815fbf0 2024-04-20T11:35:00.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.101+0000 7f29c7200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f29c815be70 con 0x7f29c80cf110 2024-04-20T11:35:00.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.101+0000 7f29c7200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f29c8144970 con 0x7f29c806ee00 2024-04-20T11:35:00.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.101+0000 7f29c6800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f29c815fbf0 0x7f29c81335d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:00.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.101+0000 7f29c6800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f29c815fbf0 0x7f29c81335d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:42696/0 (socket says 172.21.15.152:42696) 2024-04-20T11:35:00.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.101+0000 7f29c6800640 1 -- 172.21.15.152:0/361774836 learned_addr learned my addr 172.21.15.152:0/361774836 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:00.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.101+0000 7f29c5400640 1 --2- 172.21.15.152:0/361774836 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f29c80cf110 0x7f29c81e9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:00.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.101+0000 7f29c5e00640 1 --2- 172.21.15.152:0/361774836 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f29c806ee00 0x7f29c81e9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:00.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.101+0000 7f29c6800640 1 -- 172.21.15.152:0/361774836 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f29c806ee00 msgr2=0x7f29c81e9f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:00.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.101+0000 7f29c6800640 1 --2- 172.21.15.152:0/361774836 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f29c806ee00 0x7f29c81e9f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:00.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.101+0000 7f29c6800640 1 -- 172.21.15.152:0/361774836 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f29c80cf110 msgr2=0x7f29c81e9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:00.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.101+0000 7f29c6800640 1 --2- 172.21.15.152:0/361774836 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f29c80cf110 0x7f29c81e9100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:00.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.101+0000 7f29c6800640 1 -- 172.21.15.152:0/361774836 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29c81e8060 con 0x7f29c815fbf0 2024-04-20T11:35:00.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.102+0000 7f29c6800640 1 --2- 172.21.15.152:0/361774836 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f29c815fbf0 0x7f29c81335d0 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f29c000c780 tx=0x7f29c000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:00.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.102+0000 7f29be000640 1 -- 172.21.15.152:0/361774836 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29c0016030 con 0x7f29c815fbf0 2024-04-20T11:35:00.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.102+0000 7f29be000640 1 -- 172.21.15.152:0/361774836 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29c0006b50 con 0x7f29c815fbf0 2024-04-20T11:35:00.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.102+0000 7f29be000640 1 -- 172.21.15.152:0/361774836 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29c000a850 con 0x7f29c815fbf0 2024-04-20T11:35:00.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.102+0000 7f29c7200640 1 -- 172.21.15.152:0/361774836 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29c81e8340 con 0x7f29c815fbf0 2024-04-20T11:35:00.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.102+0000 7f29c7200640 1 -- 172.21.15.152:0/361774836 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29c81e8900 con 0x7f29c815fbf0 2024-04-20T11:35:00.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.103+0000 7f29c7200640 1 -- 172.21.15.152:0/361774836 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29c8075e60 con 0x7f29c815fbf0 2024-04-20T11:35:00.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.104+0000 7f29be000640 1 -- 172.21.15.152:0/361774836 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 21) ==== 91550+0+0 (secure 0 0 0) 0x7f29c0019070 con 0x7f29c815fbf0 2024-04-20T11:35:00.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.105+0000 7f29be000640 1 --2- 172.21.15.152:0/361774836 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f29a806ce90 0x7f29a806f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:00.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.105+0000 7f29be000640 1 -- 172.21.15.152:0/361774836 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5765+0+0 (secure 0 0 0) 0x7f29c008c5e0 con 0x7f29c815fbf0 2024-04-20T11:35:00.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.106+0000 7f29c5e00640 1 --2- 172.21.15.152:0/361774836 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f29a806ce90 0x7f29a806f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:00.107 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.107+0000 7f29c5e00640 1 --2- 172.21.15.152:0/361774836 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f29a806ce90 0x7f29a806f350 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f29b8009890 tx=0x7f29b8008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:00.110 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.110+0000 7f29be000640 1 -- 172.21.15.152:0/361774836 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f29c005a120 con 0x7f29c815fbf0 2024-04-20T11:35:00.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.309+0000 7f29c7200640 1 -- 172.21.15.152:0/361774836 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f29c80b3980 con 0x7f29c815fbf0 2024-04-20T11:35:00.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.310+0000 7f29be000640 1 -- 172.21.15.152:0/361774836 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v119) ==== 104+0+11 (secure 0 0 0) 0x7f29c0059a20 con 0x7f29c815fbf0 2024-04-20T11:35:00.311 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 32 2024-04-20T11:35:00.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.312+0000 7f29c7200640 1 -- 172.21.15.152:0/361774836 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f29a806ce90 msgr2=0x7f29a806f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:00.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.312+0000 7f29c7200640 1 --2- 172.21.15.152:0/361774836 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f29a806ce90 0x7f29a806f350 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f29b8009890 tx=0x7f29b8008040 comp rx=0 tx=0).stop 2024-04-20T11:35:00.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.312+0000 7f29c7200640 1 -- 172.21.15.152:0/361774836 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f29c815fbf0 msgr2=0x7f29c81335d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:00.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.312+0000 7f29c7200640 1 --2- 172.21.15.152:0/361774836 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f29c815fbf0 0x7f29c81335d0 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f29c000c780 tx=0x7f29c000cc40 comp rx=0 tx=0).stop 2024-04-20T11:35:00.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.312+0000 7f29c7200640 1 -- 172.21.15.152:0/361774836 shutdown_connections 2024-04-20T11:35:00.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.313+0000 7f29c7200640 1 --2- 172.21.15.152:0/361774836 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f29a806ce90 0x7f29a806f350 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:00.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.313+0000 7f29c7200640 1 --2- 172.21.15.152:0/361774836 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f29c815fbf0 0x7f29c81335d0 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:00.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.313+0000 7f29c7200640 1 --2- 172.21.15.152:0/361774836 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f29c80cf110 0x7f29c81e9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:00.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.313+0000 7f29c7200640 1 --2- 172.21.15.152:0/361774836 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f29c806ee00 0x7f29c81e9f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:00.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.313+0000 7f29c7200640 1 -- 172.21.15.152:0/361774836 >> 172.21.15.152:0/361774836 conn(0x7f29c8067ae0 msgr2=0x7f29c80b0d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:00.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.313+0000 7f29c7200640 1 -- 172.21.15.152:0/361774836 shutdown_connections 2024-04-20T11:35:00.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.313+0000 7f29c7200640 1 -- 172.21.15.152:0/361774836 wait complete. 2024-04-20T11:35:00.332 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T11:35:00.537 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.536+0000 7fe8bf200640 1 Processor -- start 2024-04-20T11:35:00.537 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.537+0000 7fe8bf200640 1 -- start start 2024-04-20T11:35:00.539 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.538+0000 7fe8bf200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8c0156c00 0x7fe8c0160a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:00.539 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.538+0000 7fe8bf200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fe8c006c4b0 0x7fe8c015cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:00.539 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.538+0000 7fe8bf200640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8c0160fd0 con 0x7fe8c015ca40 2024-04-20T11:35:00.539 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.538+0000 7fe8bf200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fe8c015d5c0 con 0x7fe8c006c4b0 2024-04-20T11:35:00.539 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.538+0000 7fe8bf200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe8c015d8a0 con 0x7fe8c0156c00 2024-04-20T11:35:00.539 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.538+0000 7fe8bd400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fe8c015ca40 0x7fe8c006bda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:58722/0 (socket says 172.21.15.152:58722) 2024-04-20T11:35:00.539 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.538+0000 7fe8bd400640 1 -- 172.21.15.152:0/481036816 learned_addr learned my addr 172.21.15.152:0/481036816 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:35:00.539 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.539+0000 7fe8be800640 1 --2- 172.21.15.152:0/481036816 >> v2:172.21.15.152:3300/0 conn(0x7fe8c006c4b0 0x7fe8c015cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:00.540 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.539+0000 7fe8bde00640 1 --2- 172.21.15.152:0/481036816 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8c0156c00 0x7fe8c0160a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:00.540 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.539+0000 7fe8bca00640 1 -- 172.21.15.152:0/481036816 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 420544946 0 0) 0x7fe8c0160fd0 con 0x7fe8c015ca40 2024-04-20T11:35:00.540 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.539+0000 7fe8bca00640 1 -- 172.21.15.152:0/481036816 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe8a4003520 con 0x7fe8c015ca40 2024-04-20T11:35:00.540 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.539+0000 7fe8be800640 1 -- 172.21.15.152:0/481036816 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8c0156c00 msgr2=0x7fe8c0160a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:00.540 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.539+0000 7fe8be800640 1 --2- 172.21.15.152:0/481036816 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8c0156c00 0x7fe8c0160a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:00.541 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.539+0000 7fe8be800640 1 -- 172.21.15.152:0/481036816 >> v1:172.21.15.152:6789/0 conn(0x7fe8c015ca40 legacy=0x7fe8c006bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:00.541 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.539+0000 7fe8be800640 1 -- 172.21.15.152:0/481036816 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8c0160290 con 0x7fe8c006c4b0 2024-04-20T11:35:00.541 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.540+0000 7fe8be800640 1 --2- 172.21.15.152:0/481036816 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe8c006c4b0 0x7fe8c015cec0 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7fe8b8009d90 tx=0x7fe8b802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=277e33d0feff9939 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:00.541 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.541+0000 7fe8bca00640 1 -- 172.21.15.152:0/481036816 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b8039070 con 0x7fe8c006c4b0 2024-04-20T11:35:00.541 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.541+0000 7fe8bca00640 1 -- 172.21.15.152:0/481036816 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8b802cc60 con 0x7fe8c006c4b0 2024-04-20T11:35:00.542 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.541+0000 7fe8bca00640 1 -- 172.21.15.152:0/481036816 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b8034660 con 0x7fe8c006c4b0 2024-04-20T11:35:00.542 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.541+0000 7fe8bf200640 1 -- 172.21.15.152:0/481036816 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe8c006c4b0 msgr2=0x7fe8c015cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:00.542 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.541+0000 7fe8bf200640 1 --2- 172.21.15.152:0/481036816 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe8c006c4b0 0x7fe8c015cec0 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7fe8b8009d90 tx=0x7fe8b802beb0 comp rx=0 tx=0).stop 2024-04-20T11:35:00.543 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.542+0000 7fe8bf200640 1 -- 172.21.15.152:0/481036816 shutdown_connections 2024-04-20T11:35:00.543 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.542+0000 7fe8bf200640 1 --2- 172.21.15.152:0/481036816 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe8c006c4b0 0x7fe8c015cec0 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:00.543 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.542+0000 7fe8bf200640 1 --2- 172.21.15.152:0/481036816 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8c0156c00 0x7fe8c0160a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:00.543 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.542+0000 7fe8bf200640 1 -- 172.21.15.152:0/481036816 >> 172.21.15.152:0/481036816 conn(0x7fe8c00b21e0 msgr2=0x7fe8c00b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:00.543 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.542+0000 7fe8bf200640 1 -- 172.21.15.152:0/481036816 shutdown_connections 2024-04-20T11:35:00.543 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.542+0000 7fe8bf200640 1 -- 172.21.15.152:0/481036816 wait complete. 2024-04-20T11:35:00.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.543+0000 7fe8bf200640 1 Processor -- start 2024-04-20T11:35:00.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.544+0000 7fe8bf200640 1 -- start start 2024-04-20T11:35:00.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.545+0000 7fe8bf200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8c006c4b0 0x7fe8c00b0360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:00.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.545+0000 7fe8bf200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8c0156c00 0x7fe8c0099310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:00.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.545+0000 7fe8bf200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe8c015ca40 0x7fe8c014ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:00.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.545+0000 7fe8bf200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c00b3240 con 0x7fe8c015ca40 2024-04-20T11:35:00.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.545+0000 7fe8bf200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c00b30c0 con 0x7fe8c006c4b0 2024-04-20T11:35:00.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.545+0000 7fe8bf200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe8c00b33c0 con 0x7fe8c0156c00 2024-04-20T11:35:00.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.545+0000 7fe8be800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe8c015ca40 0x7fe8c014ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:00.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.545+0000 7fe8be800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe8c015ca40 0x7fe8c014ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:42702/0 (socket says 172.21.15.152:42702) 2024-04-20T11:35:00.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.545+0000 7fe8be800640 1 -- 172.21.15.152:0/1782716955 learned_addr learned my addr 172.21.15.152:0/1782716955 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:00.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.545+0000 7fe8bd400640 1 --2- 172.21.15.152:0/1782716955 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8c0156c00 0x7fe8c0099310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:00.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.545+0000 7fe8bde00640 1 --2- 172.21.15.152:0/1782716955 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8c006c4b0 0x7fe8c00b0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:00.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.546+0000 7fe8be800640 1 -- 172.21.15.152:0/1782716955 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8c0156c00 msgr2=0x7fe8c0099310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:00.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.546+0000 7fe8be800640 1 --2- 172.21.15.152:0/1782716955 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8c0156c00 0x7fe8c0099310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:00.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.546+0000 7fe8be800640 1 -- 172.21.15.152:0/1782716955 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8c006c4b0 msgr2=0x7fe8c00b0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:00.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.546+0000 7fe8be800640 1 --2- 172.21.15.152:0/1782716955 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8c006c4b0 0x7fe8c00b0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:00.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.546+0000 7fe8be800640 1 -- 172.21.15.152:0/1782716955 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8c00b1370 con 0x7fe8c015ca40 2024-04-20T11:35:00.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.546+0000 7fe8bde00640 1 --2- 172.21.15.152:0/1782716955 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8c006c4b0 0x7fe8c00b0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:35:00.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.546+0000 7fe8be800640 1 --2- 172.21.15.152:0/1782716955 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe8c015ca40 0x7fe8c014ee80 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7fe8b8009d90 tx=0x7fe8b8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:00.548 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.546+0000 7fe8b6000640 1 -- 172.21.15.152:0/1782716955 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b8039050 con 0x7fe8c015ca40 2024-04-20T11:35:00.548 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.546+0000 7fe8b6000640 1 -- 172.21.15.152:0/1782716955 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8b8045070 con 0x7fe8c015ca40 2024-04-20T11:35:00.548 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.546+0000 7fe8bf200640 1 -- 172.21.15.152:0/1782716955 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8c00b0ad0 con 0x7fe8c015ca40 2024-04-20T11:35:00.548 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.546+0000 7fe8b6000640 1 -- 172.21.15.152:0/1782716955 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b8009450 con 0x7fe8c015ca40 2024-04-20T11:35:00.548 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.546+0000 7fe8bf200640 1 -- 172.21.15.152:0/1782716955 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8c00b1090 con 0x7fe8c015ca40 2024-04-20T11:35:00.548 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.547+0000 7fe8bf200640 1 -- 172.21.15.152:0/1782716955 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8c0075e60 con 0x7fe8c015ca40 2024-04-20T11:35:00.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.549+0000 7fe8b6000640 1 -- 172.21.15.152:0/1782716955 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 21) ==== 91550+0+0 (secure 0 0 0) 0x7fe8b803e850 con 0x7fe8c015ca40 2024-04-20T11:35:00.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.549+0000 7fe8b6000640 1 --2- 172.21.15.152:0/1782716955 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe89c06ce90 0x7fe89c06f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:00.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.550+0000 7fe8b6000640 1 -- 172.21.15.152:0/1782716955 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5765+0+0 (secure 0 0 0) 0x7fe8b80af470 con 0x7fe8c015ca40 2024-04-20T11:35:00.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.550+0000 7fe8bde00640 1 --2- 172.21.15.152:0/1782716955 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe89c06ce90 0x7fe89c06f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:00.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.551+0000 7fe8bde00640 1 --2- 172.21.15.152:0/1782716955 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe89c06ce90 0x7fe89c06f350 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe8b0009ac0 tx=0x7fe8b0008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:00.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.554+0000 7fe8b6000640 1 -- 172.21.15.152:0/1782716955 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe8b807d160 con 0x7fe8c015ca40 2024-04-20T11:35:00.762 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.761+0000 7fe8bf200640 1 -- 172.21.15.152:0/1782716955 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fe8c006f390 con 0x7fe8c015ca40 2024-04-20T11:35:00.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.762+0000 7fe8b6000640 1 -- 172.21.15.152:0/1782716955 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v119) ==== 100+0+11 (secure 0 0 0) 0x7fe8b807ca60 con 0x7fe8c015ca40 2024-04-20T11:35:00.763 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 64 2024-04-20T11:35:00.766 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.765+0000 7fe8bf200640 1 -- 172.21.15.152:0/1782716955 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe89c06ce90 msgr2=0x7fe89c06f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:00.766 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.765+0000 7fe8bf200640 1 --2- 172.21.15.152:0/1782716955 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe89c06ce90 0x7fe89c06f350 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe8b0009ac0 tx=0x7fe8b0008040 comp rx=0 tx=0).stop 2024-04-20T11:35:00.766 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.765+0000 7fe8bf200640 1 -- 172.21.15.152:0/1782716955 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe8c015ca40 msgr2=0x7fe8c014ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:00.766 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.765+0000 7fe8bf200640 1 --2- 172.21.15.152:0/1782716955 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe8c015ca40 0x7fe8c014ee80 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7fe8b8009d90 tx=0x7fe8b8004a40 comp rx=0 tx=0).stop 2024-04-20T11:35:00.766 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.765+0000 7fe8bf200640 1 -- 172.21.15.152:0/1782716955 shutdown_connections 2024-04-20T11:35:00.766 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.765+0000 7fe8bf200640 1 --2- 172.21.15.152:0/1782716955 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe89c06ce90 0x7fe89c06f350 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:00.767 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.765+0000 7fe8bf200640 1 --2- 172.21.15.152:0/1782716955 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe8c015ca40 0x7fe8c014ee80 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:00.767 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.765+0000 7fe8bf200640 1 --2- 172.21.15.152:0/1782716955 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8c0156c00 0x7fe8c0099310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:00.767 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.765+0000 7fe8bf200640 1 --2- 172.21.15.152:0/1782716955 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8c006c4b0 0x7fe8c00b0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:00.767 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.765+0000 7fe8bf200640 1 -- 172.21.15.152:0/1782716955 >> 172.21.15.152:0/1782716955 conn(0x7fe8c00b21e0 msgr2=0x7fe8c0160a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:00.767 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.766+0000 7fe8bf200640 1 -- 172.21.15.152:0/1782716955 shutdown_connections 2024-04-20T11:35:00.767 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.766+0000 7fe8bf200640 1 -- 172.21.15.152:0/1782716955 wait complete. 2024-04-20T11:35:00.785 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-20T11:35:00.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.982+0000 7f945be00640 1 Processor -- start 2024-04-20T11:35:00.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.983+0000 7f945be00640 1 -- start start 2024-04-20T11:35:00.985 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.984+0000 7f945be00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f945c15ec10 0x7f945c0cee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:00.985 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.984+0000 7f945be00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f945c0b29f0 0x7f945c0cc370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:00.985 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.984+0000 7f945be00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f945c15eff0 con 0x7f945c0cdcb0 2024-04-20T11:35:00.985 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.984+0000 7f945be00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f945c0d0060 con 0x7f945c0b29f0 2024-04-20T11:35:00.985 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.984+0000 7f945be00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f945c0cf7c0 con 0x7f945c15ec10 2024-04-20T11:35:00.985 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.985+0000 7f945b400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f945c0b29f0 0x7f945c0cc370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:00.986 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.985+0000 7f945b400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f945c0b29f0 0x7f945c0cc370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:42714/0 (socket says 172.21.15.152:42714) 2024-04-20T11:35:00.986 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.985+0000 7f945b400640 1 -- 172.21.15.152:0/541449973 learned_addr learned my addr 172.21.15.152:0/541449973 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:00.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.985+0000 7f945b400640 1 -- 172.21.15.152:0/541449973 >> v2:172.21.15.175:3300/0 conn(0x7f945c15ec10 msgr2=0x7f945c0cee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:00.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.985+0000 7f945b400640 1 --2- 172.21.15.152:0/541449973 >> v2:172.21.15.175:3300/0 conn(0x7f945c15ec10 0x7f945c0cee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:00.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.985+0000 7f945b400640 1 -- 172.21.15.152:0/541449973 >> v1:172.21.15.175:6789/0 conn(0x7f945c0cdcb0 legacy=0x7f945c0b22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:00.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.985+0000 7f945b400640 1 -- 172.21.15.152:0/541449973 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f945c0cfaa0 con 0x7f945c0b29f0 2024-04-20T11:35:00.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.986+0000 7f945b400640 1 --2- 172.21.15.152:0/541449973 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f945c0b29f0 0x7f945c0cc370 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f945c075890 tx=0x7f945402bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d7419a26cadf056 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:00.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.986+0000 7f945a000640 1 -- 172.21.15.152:0/541449973 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f945402cb20 con 0x7f945c0b29f0 2024-04-20T11:35:00.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.986+0000 7f945a000640 1 -- 172.21.15.152:0/541449973 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f945402ccc0 con 0x7f945c0b29f0 2024-04-20T11:35:00.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.986+0000 7f945a000640 1 -- 172.21.15.152:0/541449973 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9454034610 con 0x7f945c0b29f0 2024-04-20T11:35:00.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.987+0000 7f945be00640 1 -- 172.21.15.152:0/541449973 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f945c0b29f0 msgr2=0x7f945c0cc370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:00.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.987+0000 7f945be00640 1 --2- 172.21.15.152:0/541449973 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f945c0b29f0 0x7f945c0cc370 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f945c075890 tx=0x7f945402bf80 comp rx=0 tx=0).stop 2024-04-20T11:35:00.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.987+0000 7f945be00640 1 -- 172.21.15.152:0/541449973 shutdown_connections 2024-04-20T11:35:00.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.987+0000 7f945be00640 1 --2- 172.21.15.152:0/541449973 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f945c0b29f0 0x7f945c0cc370 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:00.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.987+0000 7f945be00640 1 --2- 172.21.15.152:0/541449973 >> v2:172.21.15.175:3300/0 conn(0x7f945c15ec10 0x7f945c0cee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:00.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.987+0000 7f945be00640 1 -- 172.21.15.152:0/541449973 >> 172.21.15.152:0/541449973 conn(0x7f945c0b2e70 msgr2=0x7f945c0b3280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:00.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.987+0000 7f945be00640 1 -- 172.21.15.152:0/541449973 shutdown_connections 2024-04-20T11:35:00.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.987+0000 7f945be00640 1 -- 172.21.15.152:0/541449973 wait complete. 2024-04-20T11:35:00.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.989+0000 7f945be00640 1 Processor -- start 2024-04-20T11:35:00.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.989+0000 7f945be00640 1 -- start start 2024-04-20T11:35:00.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.990+0000 7f945be00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f945c0b29f0 0x7f945c0b6990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:00.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.990+0000 7f945be00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f945c0cdcb0 0x7f945c0b3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:00.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.990+0000 7f945be00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f945c15ec10 0x7f945c14b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:00.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.990+0000 7f945be00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f945c0cf390 con 0x7f945c0cdcb0 2024-04-20T11:35:00.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.990+0000 7f945be00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f945c0cbe50 con 0x7f945c0b29f0 2024-04-20T11:35:00.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.990+0000 7f945be00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f945c0cf510 con 0x7f945c15ec10 2024-04-20T11:35:00.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.990+0000 7f9453e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f945c0cdcb0 0x7f945c0b3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:00.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.990+0000 7f9453e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f945c0cdcb0 0x7f945c0b3ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:42720/0 (socket says 172.21.15.152:42720) 2024-04-20T11:35:00.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.990+0000 7f9453e00640 1 -- 172.21.15.152:0/3747581931 learned_addr learned my addr 172.21.15.152:0/3747581931 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:00.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.991+0000 7f9453e00640 1 -- 172.21.15.152:0/3747581931 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f945c15ec10 msgr2=0x7f945c14b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:00.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.991+0000 7f9453e00640 1 --2- 172.21.15.152:0/3747581931 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f945c15ec10 0x7f945c14b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:00.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.991+0000 7f9453e00640 1 -- 172.21.15.152:0/3747581931 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f945c0b29f0 msgr2=0x7f945c0b6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:00.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.991+0000 7f9453e00640 1 --2- 172.21.15.152:0/3747581931 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f945c0b29f0 0x7f945c0b6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:00.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.991+0000 7f9453e00640 1 -- 172.21.15.152:0/3747581931 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f945c14bc70 con 0x7f945c0cdcb0 2024-04-20T11:35:00.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.991+0000 7f9453e00640 1 --2- 172.21.15.152:0/3747581931 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f945c0cdcb0 0x7f945c0b3ff0 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f944400c430 tx=0x7f944400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:00.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.991+0000 7f9452a00640 1 -- 172.21.15.152:0/3747581931 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9444017070 con 0x7f945c0cdcb0 2024-04-20T11:35:00.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.991+0000 7f9452a00640 1 -- 172.21.15.152:0/3747581931 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94440035f0 con 0x7f945c0cdcb0 2024-04-20T11:35:00.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.991+0000 7f945be00640 1 -- 172.21.15.152:0/3747581931 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f945c0b5930 con 0x7f945c0cdcb0 2024-04-20T11:35:00.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.991+0000 7f9452a00640 1 -- 172.21.15.152:0/3747581931 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94440039e0 con 0x7f945c0cdcb0 2024-04-20T11:35:00.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.991+0000 7f945be00640 1 -- 172.21.15.152:0/3747581931 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f945c0b5eb0 con 0x7f945c0cdcb0 2024-04-20T11:35:00.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.992+0000 7f945be00640 1 -- 172.21.15.152:0/3747581931 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f945c075e60 con 0x7f945c0cdcb0 2024-04-20T11:35:00.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.993+0000 7f9452a00640 1 -- 172.21.15.152:0/3747581931 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 21) ==== 91550+0+0 (secure 0 0 0) 0x7f944401a070 con 0x7f945c0cdcb0 2024-04-20T11:35:00.995 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.994+0000 7f9452a00640 1 --2- 172.21.15.152:0/3747581931 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f943806ce90 0x7f943806f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:00.995 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.994+0000 7f9452a00640 1 -- 172.21.15.152:0/3747581931 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5765+0+0 (secure 0 0 0) 0x7f944408b930 con 0x7f945c0cdcb0 2024-04-20T11:35:00.995 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.995+0000 7f945aa00640 1 --2- 172.21.15.152:0/3747581931 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f943806ce90 0x7f943806f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:00.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.996+0000 7f945aa00640 1 --2- 172.21.15.152:0/3747581931 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f943806ce90 0x7f943806f350 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f945c15cbe0 tx=0x7f944c006bf0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:00.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:00.999+0000 7f9452a00640 1 -- 172.21.15.152:0/3747581931 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f9444058e60 con 0x7f945c0cdcb0 2024-04-20T11:35:01.212 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:01.211+0000 7f945be00640 1 -- 172.21.15.152:0/3747581931 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f945c06dc30 con 0x7f945c0cdcb0 2024-04-20T11:35:01.213 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:01.212+0000 7f9452a00640 1 -- 172.21.15.152:0/3747581931 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v119) ==== 80+0+0 (secure 0 0 0) 0x7f944405d160 con 0x7f945c0cdcb0 2024-04-20T11:35:01.213 INFO:teuthology.orchestra.run.smithi152.stderr:listed 0 entries 2024-04-20T11:35:01.215 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:01.215+0000 7f945be00640 1 -- 172.21.15.152:0/3747581931 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f943806ce90 msgr2=0x7f943806f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:01.216 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:01.215+0000 7f945be00640 1 --2- 172.21.15.152:0/3747581931 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f943806ce90 0x7f943806f350 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f945c15cbe0 tx=0x7f944c006bf0 comp rx=0 tx=0).stop 2024-04-20T11:35:01.216 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:01.215+0000 7f945be00640 1 -- 172.21.15.152:0/3747581931 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f945c0cdcb0 msgr2=0x7f945c0b3ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:01.216 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:01.215+0000 7f945be00640 1 --2- 172.21.15.152:0/3747581931 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f945c0cdcb0 0x7f945c0b3ff0 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f944400c430 tx=0x7f944400c8f0 comp rx=0 tx=0).stop 2024-04-20T11:35:01.216 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:01.215+0000 7f945be00640 1 -- 172.21.15.152:0/3747581931 shutdown_connections 2024-04-20T11:35:01.216 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:01.215+0000 7f945be00640 1 --2- 172.21.15.152:0/3747581931 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f943806ce90 0x7f943806f350 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:01.216 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:01.215+0000 7f945be00640 1 --2- 172.21.15.152:0/3747581931 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f945c15ec10 0x7f945c14b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:01.216 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:01.215+0000 7f945be00640 1 --2- 172.21.15.152:0/3747581931 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f945c0cdcb0 0x7f945c0b3ff0 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:01.216 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:01.215+0000 7f945be00640 1 --2- 172.21.15.152:0/3747581931 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f945c0b29f0 0x7f945c0b6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:01.216 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:01.215+0000 7f945be00640 1 -- 172.21.15.152:0/3747581931 >> 172.21.15.152:0/3747581931 conn(0x7f945c0b2e70 msgr2=0x7f945c06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:01.217 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:01.215+0000 7f945be00640 1 -- 172.21.15.152:0/3747581931 shutdown_connections 2024-04-20T11:35:01.217 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:01.216+0000 7f945be00640 1 -- 172.21.15.152:0/3747581931 wait complete. 2024-04-20T11:35:01.234 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi175.front.sepia.ceph.com... 2024-04-20T11:35:01.235 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:35:01.235 DEBUG:teuthology.orchestra.run.smithi175:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-04-20T11:35:01.309 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi175.stderr:ceph-fuse[34437]: fuse finished with error 0 and tester_r 0 2024-04-20T11:35:01.312 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi175.front.sepia.ceph.com 2024-04-20T11:35:01.313 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:35:01.313 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-20T11:35:01.413 INFO:tasks.cephfs.mount:Terminating background process 2024-04-20T11:35:01.414 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-20T11:35:01.414 DEBUG:teuthology.orchestra.run.smithi152:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:35:01.473 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-20T11:35:01.473 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi152.front.sepia.ceph.com 2024-04-20T11:35:01.473 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:35:01.473 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-20T11:35:01.527 INFO:teuthology.orchestra.run.smithi152.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-20T11:35:01.528 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T11:35:01.529 INFO:tasks.cephfs.mount:Terminating background process 2024-04-20T11:35:01.529 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T11:35:01.529 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:35:01.585 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-20T11:35:01.585 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi175.front.sepia.ceph.com 2024-04-20T11:35:01.586 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:35:01.586 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-20T11:35:01.639 INFO:teuthology.orchestra.run.smithi175.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-20T11:35:01.640 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T11:35:01.641 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:35:01.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:01.849+0000 7fbfbdc00640 1 Processor -- start 2024-04-20T11:35:01.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:01.849+0000 7fbfbdc00640 1 -- start start 2024-04-20T11:35:01.852 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:01.851+0000 7fbfbdc00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fbfb80cd0d0 0x7fbfb80cd580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:01.852 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:01.851+0000 7fbfbdc00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbfb8071fe0 0x7fbfb8072470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:01.852 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:01.851+0000 7fbfbdc00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbfb8146340 con 0x7fbfb8145f60 2024-04-20T11:35:01.852 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:01.851+0000 7fbfbdc00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fbfb806e9f0 con 0x7fbfb80cd0d0 2024-04-20T11:35:01.852 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:01.851+0000 7fbfbdc00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fbfb8072d50 con 0x7fbfb8071fe0 2024-04-20T11:35:01.852 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:01.851+0000 7fbfb6000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fbfb80cd0d0 0x7fbfb80cd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:01.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:01.851+0000 7fbfb6a00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fbfb8145f60 0x7fbfb806ee80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:58730/0 (socket says 172.21.15.152:58730) 2024-04-20T11:35:01.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:01.851+0000 7fbfb6a00640 1 -- 172.21.15.152:0/3148377870 learned_addr learned my addr 172.21.15.152:0/3148377870 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:35:01.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:01.852+0000 7fbfb7400640 1 --2- 172.21.15.152:0/3148377870 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbfb8071fe0 0x7fbfb8072470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:01.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:01.852+0000 7fbfb6000640 1 -- 172.21.15.152:0/3148377870 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbfb8071fe0 msgr2=0x7fbfb8072470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:01.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:01.852+0000 7fbfb6000640 1 --2- 172.21.15.152:0/3148377870 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbfb8071fe0 0x7fbfb8072470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:01.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:01.852+0000 7fbfb6000640 1 -- 172.21.15.152:0/3148377870 >> v1:172.21.15.152:6789/0 conn(0x7fbfb8145f60 legacy=0x7fbfb806ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:01.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:01.852+0000 7fbfb6000640 1 -- 172.21.15.152:0/3148377870 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfb806e0c0 con 0x7fbfb80cd0d0 2024-04-20T11:35:01.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:01.852+0000 7fbfb5600640 1 -- 172.21.15.152:0/3148377870 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1695910453 0 0) 0x7fbfb8146340 con 0x7fbfb8145f60 2024-04-20T11:35:01.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:01.853+0000 7fbfb6000640 1 --2- 172.21.15.152:0/3148377870 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbfb80cd0d0 0x7fbfb80cd580 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7fbfa40097a0 tx=0x7fbfa402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b016d30a94b1786 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:01.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:01.853+0000 7fbfb5600640 1 -- 172.21.15.152:0/3148377870 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa4035020 con 0x7fbfb80cd0d0 2024-04-20T11:35:01.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:01.853+0000 7fbfb5600640 1 -- 172.21.15.152:0/3148377870 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfa4031810 con 0x7fbfb80cd0d0 2024-04-20T11:35:01.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:01.853+0000 7fbfb5600640 1 -- 172.21.15.152:0/3148377870 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa4031be0 con 0x7fbfb80cd0d0 2024-04-20T11:35:01.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:01.854+0000 7fbfbdc00640 1 -- 172.21.15.152:0/3148377870 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbfb80cd0d0 msgr2=0x7fbfb80cd580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:01.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:01.854+0000 7fbfbdc00640 1 --2- 172.21.15.152:0/3148377870 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbfb80cd0d0 0x7fbfb80cd580 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7fbfa40097a0 tx=0x7fbfa402b8b0 comp rx=0 tx=0).stop 2024-04-20T11:35:01.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:01.854+0000 7fbfbdc00640 1 -- 172.21.15.152:0/3148377870 shutdown_connections 2024-04-20T11:35:01.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:01.854+0000 7fbfbdc00640 1 --2- 172.21.15.152:0/3148377870 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbfb8071fe0 0x7fbfb8072470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:01.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:01.854+0000 7fbfbdc00640 1 --2- 172.21.15.152:0/3148377870 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbfb80cd0d0 0x7fbfb80cd580 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:01.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:01.854+0000 7fbfbdc00640 1 -- 172.21.15.152:0/3148377870 >> 172.21.15.152:0/3148377870 conn(0x7fbfb8067cf0 msgr2=0x7fbfb806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:01.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:01.854+0000 7fbfbdc00640 1 -- 172.21.15.152:0/3148377870 shutdown_connections 2024-04-20T11:35:01.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:01.854+0000 7fbfbdc00640 1 -- 172.21.15.152:0/3148377870 wait complete. 2024-04-20T11:35:01.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:01.856+0000 7fbfbdc00640 1 Processor -- start 2024-04-20T11:35:01.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:01.856+0000 7fbfbdc00640 1 -- start start 2024-04-20T11:35:01.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:01.857+0000 7fbfbdc00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbfb8071fe0 0x7fbfb8136e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:01.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:01.857+0000 7fbfbdc00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbfb80cd0d0 0x7fbfb81f4500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:01.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:01.857+0000 7fbfbdc00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbfb8145f60 0x7fbfb80b1270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:01.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:01.857+0000 7fbfbdc00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb80729b0 con 0x7fbfb8145f60 2024-04-20T11:35:01.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:01.857+0000 7fbfbdc00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb806e680 con 0x7fbfb80cd0d0 2024-04-20T11:35:01.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:01.857+0000 7fbfbdc00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fbfb8072b30 con 0x7fbfb8071fe0 2024-04-20T11:35:01.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:01.857+0000 7fbfb7400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbfb8145f60 0x7fbfb80b1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:01.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:01.857+0000 7fbfb7400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbfb8145f60 0x7fbfb80b1270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:42738/0 (socket says 172.21.15.152:42738) 2024-04-20T11:35:01.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:01.857+0000 7fbfb7400640 1 -- 172.21.15.152:0/1687489585 learned_addr learned my addr 172.21.15.152:0/1687489585 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:01.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:01.857+0000 7fbfb7400640 1 -- 172.21.15.152:0/1687489585 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbfb8071fe0 msgr2=0x7fbfb8136e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:01.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:01.857+0000 7fbfb7400640 1 --2- 172.21.15.152:0/1687489585 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbfb8071fe0 0x7fbfb8136e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:01.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:01.857+0000 7fbfb7400640 1 -- 172.21.15.152:0/1687489585 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbfb80cd0d0 msgr2=0x7fbfb81f4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:01.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:01.857+0000 7fbfb7400640 1 --2- 172.21.15.152:0/1687489585 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbfb80cd0d0 0x7fbfb81f4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:01.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:01.857+0000 7fbfb7400640 1 -- 172.21.15.152:0/1687489585 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfb80ae6c0 con 0x7fbfb8145f60 2024-04-20T11:35:01.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:01.858+0000 7fbfb7400640 1 --2- 172.21.15.152:0/1687489585 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbfb8145f60 0x7fbfb80b1270 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7fbfb0007e40 tx=0x7fbfb000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:01.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:01.858+0000 7fbfaea00640 1 -- 172.21.15.152:0/1687489585 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb0014620 con 0x7fbfb8145f60 2024-04-20T11:35:01.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:01.858+0000 7fbfaea00640 1 -- 172.21.15.152:0/1687489585 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfb0019070 con 0x7fbfb8145f60 2024-04-20T11:35:01.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:01.858+0000 7fbfaea00640 1 -- 172.21.15.152:0/1687489585 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb0014a20 con 0x7fbfb8145f60 2024-04-20T11:35:01.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:01.858+0000 7fbfbdc00640 1 -- 172.21.15.152:0/1687489585 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfb80b36c0 con 0x7fbfb8145f60 2024-04-20T11:35:01.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:01.858+0000 7fbfbdc00640 1 -- 172.21.15.152:0/1687489585 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfb80b3c80 con 0x7fbfb8145f60 2024-04-20T11:35:01.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:01.859+0000 7fbfbdc00640 1 -- 172.21.15.152:0/1687489585 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfb80689f0 con 0x7fbfb8145f60 2024-04-20T11:35:01.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:01.860+0000 7fbfaea00640 1 -- 172.21.15.152:0/1687489585 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 21) ==== 91550+0+0 (secure 0 0 0) 0x7fbfb0014bc0 con 0x7fbfb8145f60 2024-04-20T11:35:01.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:01.861+0000 7fbfaea00640 1 --2- 172.21.15.152:0/1687489585 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fbf9806ce90 0x7fbf9806f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:01.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:01.861+0000 7fbfaea00640 1 -- 172.21.15.152:0/1687489585 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5765+0+0 (secure 0 0 0) 0x7fbfb008cf80 con 0x7fbfb8145f60 2024-04-20T11:35:01.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:01.862+0000 7fbfb6a00640 1 --2- 172.21.15.152:0/1687489585 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fbf9806ce90 0x7fbf9806f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:01.863 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:01.863+0000 7fbfb6a00640 1 --2- 172.21.15.152:0/1687489585 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fbf9806ce90 0x7fbf9806f350 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fbfa8005880 tx=0x7fbfa8007a00 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:01.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:01.865+0000 7fbfaea00640 1 -- 172.21.15.152:0/1687489585 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fbfb005aac0 con 0x7fbfb8145f60 2024-04-20T11:35:02.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.104+0000 7fbfbdc00640 1 -- 172.21.15.152:0/1687489585 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbfb80b0180 con 0x7fbfb8145f60 2024-04-20T11:35:02.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.105+0000 7fbfaea00640 1 -- 172.21.15.152:0/1687489585 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) ==== 94+0+5207 (secure 0 0 0) 0x7fbfb005a3c0 con 0x7fbfb8145f60 2024-04-20T11:35:02.106 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:35:02.106 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":77,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5891,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/508638056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6833","nonce":508638056},{"type":"v1","addr":"172.21.15.175:6835","nonce":508638056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":6448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6834/3447077466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3447077466},{"type":"v1","addr":"172.21.15.175:6834","nonce":3447077466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":8409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6834/751442105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":751442105},{"type":"v1","addr":"172.21.15.152:6834","nonce":751442105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T11:34:36.050950+0000","modified":"2024-04-20T11:34:39.117003+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8412},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8412":{"gid":8412,"name":"c","rank":0,"incarnation":72,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6835/2477488828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6833","nonce":2477488828},{"type":"v1","addr":"172.21.15.152:6835","nonce":2477488828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8412,"qdb_cluster":[8412]},"id":9}]} 2024-04-20T11:35:02.106 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 77 2024-04-20T11:35:02.110 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.107+0000 7fbfbdc00640 1 -- 172.21.15.152:0/1687489585 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fbf9806ce90 msgr2=0x7fbf9806f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:02.110 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.107+0000 7fbfbdc00640 1 --2- 172.21.15.152:0/1687489585 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fbf9806ce90 0x7fbf9806f350 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fbfa8005880 tx=0x7fbfa8007a00 comp rx=0 tx=0).stop 2024-04-20T11:35:02.111 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.107+0000 7fbfbdc00640 1 -- 172.21.15.152:0/1687489585 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbfb8145f60 msgr2=0x7fbfb80b1270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:02.111 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.107+0000 7fbfbdc00640 1 --2- 172.21.15.152:0/1687489585 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbfb8145f60 0x7fbfb80b1270 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7fbfb0007e40 tx=0x7fbfb000e4b0 comp rx=0 tx=0).stop 2024-04-20T11:35:02.111 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.107+0000 7fbfbdc00640 1 -- 172.21.15.152:0/1687489585 shutdown_connections 2024-04-20T11:35:02.111 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.107+0000 7fbfbdc00640 1 --2- 172.21.15.152:0/1687489585 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fbf9806ce90 0x7fbf9806f350 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:02.111 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.107+0000 7fbfbdc00640 1 --2- 172.21.15.152:0/1687489585 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbfb8145f60 0x7fbfb80b1270 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:02.111 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.107+0000 7fbfbdc00640 1 --2- 172.21.15.152:0/1687489585 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbfb80cd0d0 0x7fbfb81f4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:02.111 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.107+0000 7fbfbdc00640 1 --2- 172.21.15.152:0/1687489585 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbfb8071fe0 0x7fbfb8136e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:02.111 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.107+0000 7fbfbdc00640 1 -- 172.21.15.152:0/1687489585 >> 172.21.15.152:0/1687489585 conn(0x7fbfb8067cf0 msgr2=0x7fbfb8144490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:02.111 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.108+0000 7fbfbdc00640 1 -- 172.21.15.152:0/1687489585 shutdown_connections 2024-04-20T11:35:02.111 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.108+0000 7fbfbdc00640 1 -- 172.21.15.152:0/1687489585 wait complete. 2024-04-20T11:35:02.127 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:35:02.319 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.318+0000 7f3415e00640 1 Processor -- start 2024-04-20T11:35:02.319 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.318+0000 7f3415e00640 1 -- start start 2024-04-20T11:35:02.320 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.319+0000 7f3415e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f34100723e0 0x7f341015d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:02.320 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.319+0000 7f3415e00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34100ce8c0 con 0x7f3410160b90 2024-04-20T11:35:02.320 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.319+0000 7f3415e00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34101600b0 con 0x7f341015d1e0 2024-04-20T11:35:02.320 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.319+0000 7f3415e00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f3410160390 con 0x7f34100723e0 2024-04-20T11:35:02.320 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.320+0000 7f340fe00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f3410160b90 0x7f3410071cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:58736/0 (socket says 172.21.15.152:58736) 2024-04-20T11:35:02.320 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.320+0000 7f340fe00640 1 -- 172.21.15.152:0/2846738264 learned_addr learned my addr 172.21.15.152:0/2846738264 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:35:02.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.320+0000 7f340f400640 1 -- 172.21.15.152:0/2846738264 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 671330518 0 0) 0x7f34100ce8c0 con 0x7f3410160b90 2024-04-20T11:35:02.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.320+0000 7f3415400640 1 --2- 172.21.15.152:0/2846738264 >> v2:172.21.15.175:3300/0 conn(0x7f34100723e0 0x7f341015d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:02.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.320+0000 7f340f400640 1 -- 172.21.15.152:0/2846738264 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f33f4003520 con 0x7f3410160b90 2024-04-20T11:35:02.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.320+0000 7f340f400640 1 -- 172.21.15.152:0/2846738264 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3880440911 0 0) 0x7f33f4003520 con 0x7f3410160b90 2024-04-20T11:35:02.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.320+0000 7f340f400640 1 -- 172.21.15.152:0/2846738264 >> v2:172.21.15.175:3300/0 conn(0x7f34100723e0 msgr2=0x7f341015d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:02.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.320+0000 7f340f400640 1 --2- 172.21.15.152:0/2846738264 >> v2:172.21.15.175:3300/0 conn(0x7f34100723e0 0x7f341015d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:02.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.320+0000 7f340f400640 1 -- 172.21.15.152:0/2846738264 >> v1:172.21.15.175:6789/0 conn(0x7f341015d1e0 legacy=0x7f34100cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:02.323 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.320+0000 7f340f400640 1 -- 172.21.15.152:0/2846738264 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3410160670 con 0x7f3410160b90 2024-04-20T11:35:02.323 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.321+0000 7f340f400640 1 -- 172.21.15.152:0/2846738264 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f33f8001670 con 0x7f3410160b90 2024-04-20T11:35:02.323 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.321+0000 7f340f400640 1 -- 172.21.15.152:0/2846738264 >> v1:172.21.15.152:6789/0 conn(0x7f3410160b90 legacy=0x7f3410071cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:02.323 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.321+0000 7f340f400640 1 --2- 172.21.15.152:0/2846738264 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f33f40039b0 0x7f33f4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:02.323 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.321+0000 7f340f400640 1 --2- 172.21.15.152:0/2846738264 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33f4004800 0x7f33f4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:02.323 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.321+0000 7f340f400640 1 --2- 172.21.15.152:0/2846738264 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33f40051c0 0x7f33f400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:02.323 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.321+0000 7f340f400640 1 -- 172.21.15.152:0/2846738264 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f3410160390 con 0x7f33f40039b0 2024-04-20T11:35:02.323 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.321+0000 7f340f400640 1 -- 172.21.15.152:0/2846738264 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f33f400c110 con 0x7f33f40051c0 2024-04-20T11:35:02.323 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.321+0000 7f340f400640 1 -- 172.21.15.152:0/2846738264 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f33f400c310 con 0x7f33f4004800 2024-04-20T11:35:02.323 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.321+0000 7f3414a00640 1 --2- 172.21.15.152:0/2846738264 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f33f40039b0 0x7f33f4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:02.323 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.321+0000 7f3414a00640 1 -- 172.21.15.152:0/2846738264 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33f4004800 msgr2=0x7f33f4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:02.323 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.321+0000 7f3414a00640 1 --2- 172.21.15.152:0/2846738264 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33f4004800 0x7f33f4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:02.323 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.321+0000 7f3414a00640 1 -- 172.21.15.152:0/2846738264 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33f40051c0 msgr2=0x7f33f400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:02.324 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.321+0000 7f3414a00640 1 --2- 172.21.15.152:0/2846738264 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33f40051c0 0x7f33f400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:02.324 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.321+0000 7f3414a00640 1 -- 172.21.15.152:0/2846738264 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33f4003520 con 0x7f33f40039b0 2024-04-20T11:35:02.324 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.321+0000 7f3415400640 1 -- 172.21.15.152:0/2846738264 reap_dead start 2024-04-20T11:35:02.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.322+0000 7f3414a00640 1 --2- 172.21.15.152:0/2846738264 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f33f40039b0 0x7f33f4003dd0 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f3400001e50 tx=0x7f33f4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a369dba1e7ffd5fe server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:02.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.322+0000 7f340f400640 1 -- 172.21.15.152:0/2846738264 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3400001a70 con 0x7f33f40039b0 2024-04-20T11:35:02.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.323+0000 7f340f400640 1 -- 172.21.15.152:0/2846738264 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3400003b70 con 0x7f33f40039b0 2024-04-20T11:35:02.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.323+0000 7f340f400640 1 -- 172.21.15.152:0/2846738264 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34000050d0 con 0x7f33f40039b0 2024-04-20T11:35:02.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.323+0000 7f3415e00640 1 -- 172.21.15.152:0/2846738264 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f33f40039b0 msgr2=0x7f33f4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:02.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.323+0000 7f3415e00640 1 --2- 172.21.15.152:0/2846738264 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f33f40039b0 0x7f33f4003dd0 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f3400001e50 tx=0x7f33f4005830 comp rx=0 tx=0).stop 2024-04-20T11:35:02.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.323+0000 7f3415e00640 1 -- 172.21.15.152:0/2846738264 shutdown_connections 2024-04-20T11:35:02.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.323+0000 7f3415e00640 1 --2- 172.21.15.152:0/2846738264 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f33f40039b0 0x7f33f4003dd0 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:02.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.323+0000 7f3415e00640 1 -- 172.21.15.152:0/2846738264 >> 172.21.15.152:0/2846738264 conn(0x7f3410067ae0 msgr2=0x7f341006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:02.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.323+0000 7f3415e00640 1 -- 172.21.15.152:0/2846738264 shutdown_connections 2024-04-20T11:35:02.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.323+0000 7f3415e00640 1 -- 172.21.15.152:0/2846738264 wait complete. 2024-04-20T11:35:02.326 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.325+0000 7f3415e00640 1 Processor -- start 2024-04-20T11:35:02.326 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.325+0000 7f3415e00640 1 -- start start 2024-04-20T11:35:02.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.326+0000 7f3415e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33f40039b0 0x7f3410143a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:02.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.326+0000 7f3415e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3410159700 0x7f3410146cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:02.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.326+0000 7f3415e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f34101580a0 0x7f34100a94a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:02.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.326+0000 7f3415e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f341015d740 con 0x7f34101580a0 2024-04-20T11:35:02.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.326+0000 7f3415e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f341015d5c0 con 0x7f3410159700 2024-04-20T11:35:02.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.326+0000 7f3415e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f341015d8c0 con 0x7f33f40039b0 2024-04-20T11:35:02.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.326+0000 7f3415400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f34101580a0 0x7f34100a94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:02.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.326+0000 7f3415400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f34101580a0 0x7f34100a94a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:42766/0 (socket says 172.21.15.152:42766) 2024-04-20T11:35:02.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.326+0000 7f3415400640 1 -- 172.21.15.152:0/3427759824 learned_addr learned my addr 172.21.15.152:0/3427759824 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:02.329 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.327+0000 7f3415400640 1 -- 172.21.15.152:0/3427759824 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33f40039b0 msgr2=0x7f3410143a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:02.329 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.327+0000 7f3414a00640 1 --2- 172.21.15.152:0/3427759824 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33f40039b0 0x7f3410143a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:02.329 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.327+0000 7f340fe00640 1 --2- 172.21.15.152:0/3427759824 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3410159700 0x7f3410146cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:02.329 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.327+0000 7f3415400640 1 --2- 172.21.15.152:0/3427759824 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33f40039b0 0x7f3410143a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:02.329 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.327+0000 7f3415400640 1 -- 172.21.15.152:0/3427759824 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3410159700 msgr2=0x7f3410146cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:02.329 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.327+0000 7f3415400640 1 --2- 172.21.15.152:0/3427759824 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3410159700 0x7f3410146cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:02.329 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.327+0000 7f3415400640 1 -- 172.21.15.152:0/3427759824 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3410143fa0 con 0x7f34101580a0 2024-04-20T11:35:02.329 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.327+0000 7f3415400640 1 --2- 172.21.15.152:0/3427759824 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f34101580a0 0x7f34100a94a0 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f3410072810 tx=0x7f34080078a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:02.329 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.327+0000 7f340cc00640 1 -- 172.21.15.152:0/3427759824 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3408018070 con 0x7f34101580a0 2024-04-20T11:35:02.329 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.327+0000 7f340cc00640 1 -- 172.21.15.152:0/3427759824 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3408004300 con 0x7f34101580a0 2024-04-20T11:35:02.329 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.327+0000 7f3415e00640 1 -- 172.21.15.152:0/3427759824 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3410147230 con 0x7f34101580a0 2024-04-20T11:35:02.329 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.327+0000 7f340cc00640 1 -- 172.21.15.152:0/3427759824 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3408003850 con 0x7f34101580a0 2024-04-20T11:35:02.329 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.328+0000 7f3415e00640 1 -- 172.21.15.152:0/3427759824 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3410158940 con 0x7f34101580a0 2024-04-20T11:35:02.330 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.328+0000 7f3415e00640 1 -- 172.21.15.152:0/3427759824 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34100721a0 con 0x7f34101580a0 2024-04-20T11:35:02.330 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.330+0000 7f340cc00640 1 -- 172.21.15.152:0/3427759824 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 21) ==== 91550+0+0 (secure 0 0 0) 0x7f340801b070 con 0x7f34101580a0 2024-04-20T11:35:02.331 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.330+0000 7f340cc00640 1 --2- 172.21.15.152:0/3427759824 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f33ec06ce90 0x7f33ec06f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:02.331 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.331+0000 7f340cc00640 1 -- 172.21.15.152:0/3427759824 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5765+0+0 (secure 0 0 0) 0x7f340808b920 con 0x7f34101580a0 2024-04-20T11:35:02.331 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.331+0000 7f3414a00640 1 --2- 172.21.15.152:0/3427759824 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f33ec06ce90 0x7f33ec06f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:02.333 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.332+0000 7f3414a00640 1 --2- 172.21.15.152:0/3427759824 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f33ec06ce90 0x7f33ec06f350 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3400034000 tx=0x7f3400009030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:02.335 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.335+0000 7f340cc00640 1 -- 172.21.15.152:0/3427759824 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f34100721a0 con 0x7f34101580a0 2024-04-20T11:35:02.534 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.533+0000 7f3415e00640 1 -- 172.21.15.152:0/3427759824 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f341006ef10 con 0x7f34101580a0 2024-04-20T11:35:02.537 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.536+0000 7f340cc00640 1 -- 172.21.15.152:0/3427759824 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v119) ==== 74+0+16850 (secure 0 0 0) 0x7f341006ef10 con 0x7f34101580a0 2024-04-20T11:35:02.537 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:35:02.538 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":119,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:34:56.181185+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:34:33.781054+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-04-20T11:34:35.031728+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":119,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:35:02.540 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.539+0000 7f3415e00640 1 -- 172.21.15.152:0/3427759824 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f33ec06ce90 msgr2=0x7f33ec06f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:02.540 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.539+0000 7f3415e00640 1 --2- 172.21.15.152:0/3427759824 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f33ec06ce90 0x7f33ec06f350 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3400034000 tx=0x7f3400009030 comp rx=0 tx=0).stop 2024-04-20T11:35:02.540 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.539+0000 7f3415e00640 1 -- 172.21.15.152:0/3427759824 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f34101580a0 msgr2=0x7f34100a94a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:02.540 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.539+0000 7f3415e00640 1 --2- 172.21.15.152:0/3427759824 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f34101580a0 0x7f34100a94a0 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f3410072810 tx=0x7f34080078a0 comp rx=0 tx=0).stop 2024-04-20T11:35:02.540 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.539+0000 7f3415e00640 1 -- 172.21.15.152:0/3427759824 shutdown_connections 2024-04-20T11:35:02.540 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.539+0000 7f3415e00640 1 --2- 172.21.15.152:0/3427759824 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f33ec06ce90 0x7f33ec06f350 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:02.540 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.539+0000 7f3415e00640 1 --2- 172.21.15.152:0/3427759824 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f34101580a0 0x7f34100a94a0 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:02.540 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.539+0000 7f3415e00640 1 --2- 172.21.15.152:0/3427759824 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3410159700 0x7f3410146cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:02.541 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.539+0000 7f3415e00640 1 --2- 172.21.15.152:0/3427759824 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33f40039b0 0x7f3410143a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:02.541 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.539+0000 7f3415e00640 1 -- 172.21.15.152:0/3427759824 >> 172.21.15.152:0/3427759824 conn(0x7f3410067ae0 msgr2=0x7f34100b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:02.541 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.540+0000 7f3415e00640 1 -- 172.21.15.152:0/3427759824 shutdown_connections 2024-04-20T11:35:02.541 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.540+0000 7f3415e00640 1 -- 172.21.15.152:0/3427759824 wait complete. 2024-04-20T11:35:02.560 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T11:26:47.246571+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T11:34:33.781054+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '117', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T11:34:35.031728+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '119', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 5, 'snap_epoch': 119, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T11:35:02.560 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T11:35:02.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.755+0000 7fe530600640 1 Processor -- start 2024-04-20T11:35:02.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.755+0000 7fe530600640 1 -- start start 2024-04-20T11:35:02.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.757+0000 7fe530600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe5280cfc40 0x7fe52815fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:02.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.757+0000 7fe530600640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fe5280b2880 0x7fe5280b3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:02.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.757+0000 7fe530600640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe52815f1c0 con 0x7fe52815b510 2024-04-20T11:35:02.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.757+0000 7fe530600640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fe52806f420 con 0x7fe5280b2880 2024-04-20T11:35:02.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.757+0000 7fe530600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe5280b4430 con 0x7fe5280cfc40 2024-04-20T11:35:02.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.757+0000 7fe52dc00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fe52815b510 0x7fe5280b2170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:58738/0 (socket says 172.21.15.152:58738) 2024-04-20T11:35:02.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.757+0000 7fe52dc00640 1 -- 172.21.15.152:0/1719074190 learned_addr learned my addr 172.21.15.152:0/1719074190 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:35:02.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.758+0000 7fe52d200640 1 --2- 172.21.15.152:0/1719074190 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe5280cfc40 0x7fe52815fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:02.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.758+0000 7fe52e600640 1 --2- 172.21.15.152:0/1719074190 >> v2:172.21.15.175:3300/0 conn(0x7fe5280b2880 0x7fe5280b3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:02.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.758+0000 7fe527e00640 1 -- 172.21.15.152:0/1719074190 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1038537743 0 0) 0x7fe52815f1c0 con 0x7fe52815b510 2024-04-20T11:35:02.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.758+0000 7fe527e00640 1 -- 172.21.15.152:0/1719074190 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe510003520 con 0x7fe52815b510 2024-04-20T11:35:02.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.758+0000 7fe527e00640 1 -- 172.21.15.152:0/1719074190 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3199079648 0 0) 0x7fe510003520 con 0x7fe52815b510 2024-04-20T11:35:02.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.758+0000 7fe527e00640 1 -- 172.21.15.152:0/1719074190 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe5280cfc40 msgr2=0x7fe52815fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:02.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.758+0000 7fe527e00640 1 --2- 172.21.15.152:0/1719074190 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe5280cfc40 0x7fe52815fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:02.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.758+0000 7fe527e00640 1 -- 172.21.15.152:0/1719074190 >> v2:172.21.15.175:3300/0 conn(0x7fe5280b2880 msgr2=0x7fe5280b3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:02.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.758+0000 7fe527e00640 1 --2- 172.21.15.152:0/1719074190 >> v2:172.21.15.175:3300/0 conn(0x7fe5280b2880 0x7fe5280b3ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:02.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.759+0000 7fe527e00640 1 -- 172.21.15.152:0/1719074190 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe52806f670 con 0x7fe52815b510 2024-04-20T11:35:02.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.759+0000 7fe527e00640 1 -- 172.21.15.152:0/1719074190 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7fe51c001670 con 0x7fe52815b510 2024-04-20T11:35:02.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.759+0000 7fe527e00640 1 -- 172.21.15.152:0/1719074190 >> v1:172.21.15.152:6789/0 conn(0x7fe52815b510 legacy=0x7fe5280b2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:02.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.759+0000 7fe527e00640 1 --2- 172.21.15.152:0/1719074190 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe5280cfc40 0x7fe5100039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:02.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.759+0000 7fe527e00640 1 --2- 172.21.15.152:0/1719074190 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe510003f80 0x7fe510004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:02.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.759+0000 7fe527e00640 1 --2- 172.21.15.152:0/1719074190 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe510004d40 0x7fe5100051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:02.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.759+0000 7fe527e00640 1 -- 172.21.15.152:0/1719074190 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fe52806f420 con 0x7fe510003f80 2024-04-20T11:35:02.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.759+0000 7fe527e00640 1 -- 172.21.15.152:0/1719074190 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe5280b4430 con 0x7fe510004d40 2024-04-20T11:35:02.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.759+0000 7fe527e00640 1 -- 172.21.15.152:0/1719074190 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe51000dc80 con 0x7fe5280cfc40 2024-04-20T11:35:02.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.759+0000 7fe52d200640 1 --2- 172.21.15.152:0/1719074190 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe510003f80 0x7fe510004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:02.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.759+0000 7fe52d200640 1 -- 172.21.15.152:0/1719074190 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe5280cfc40 msgr2=0x7fe5100039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:02.762 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.759+0000 7fe52d200640 1 --2- 172.21.15.152:0/1719074190 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe5280cfc40 0x7fe5100039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:02.762 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.759+0000 7fe52d200640 1 -- 172.21.15.152:0/1719074190 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe510004d40 msgr2=0x7fe5100051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:02.762 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.759+0000 7fe52d200640 1 --2- 172.21.15.152:0/1719074190 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe510004d40 0x7fe5100051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:02.762 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.759+0000 7fe52d200640 1 -- 172.21.15.152:0/1719074190 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe510003520 con 0x7fe510003f80 2024-04-20T11:35:02.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.760+0000 7fe52d200640 1 --2- 172.21.15.152:0/1719074190 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe510003f80 0x7fe510004800 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7fe514009ac0 tx=0x7fe510005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd04d1f7bad9bad4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:02.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.761+0000 7fe527e00640 1 -- 172.21.15.152:0/1719074190 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe514039070 con 0x7fe510003f80 2024-04-20T11:35:02.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.761+0000 7fe527e00640 1 -- 172.21.15.152:0/1719074190 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe51402c710 con 0x7fe510003f80 2024-04-20T11:35:02.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.761+0000 7fe527e00640 1 -- 172.21.15.152:0/1719074190 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe514009690 con 0x7fe510003f80 2024-04-20T11:35:02.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.761+0000 7fe530600640 1 -- 172.21.15.152:0/1719074190 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe510003f80 msgr2=0x7fe510004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:02.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.761+0000 7fe530600640 1 --2- 172.21.15.152:0/1719074190 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe510003f80 0x7fe510004800 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7fe514009ac0 tx=0x7fe510005830 comp rx=0 tx=0).stop 2024-04-20T11:35:02.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.761+0000 7fe52e600640 1 -- 172.21.15.152:0/1719074190 reap_dead start 2024-04-20T11:35:02.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.761+0000 7fe530600640 1 -- 172.21.15.152:0/1719074190 shutdown_connections 2024-04-20T11:35:02.764 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.761+0000 7fe530600640 1 -- 172.21.15.152:0/1719074190 >> 172.21.15.152:0/1719074190 conn(0x7fe52806b980 msgr2=0x7fe52806bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:02.764 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.761+0000 7fe530600640 1 -- 172.21.15.152:0/1719074190 shutdown_connections 2024-04-20T11:35:02.764 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.761+0000 7fe530600640 1 -- 172.21.15.152:0/1719074190 wait complete. 2024-04-20T11:35:02.764 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.763+0000 7fe530600640 1 Processor -- start 2024-04-20T11:35:02.764 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.763+0000 7fe530600640 1 -- start start 2024-04-20T11:35:02.765 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.764+0000 7fe530600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe510003f80 0x7fe5281f4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:02.765 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.764+0000 7fe530600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe5281f4d80 0x7fe52814c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:02.765 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.764+0000 7fe530600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe52814c740 0x7fe5280b2f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:02.765 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.764+0000 7fe530600640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fe52815b8f0 con 0x7fe510003f80 2024-04-20T11:35:02.765 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.764+0000 7fe530600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe52815f710 con 0x7fe52814c740 2024-04-20T11:35:02.765 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.764+0000 7fe530600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe52806f950 con 0x7fe5281f4d80 2024-04-20T11:35:02.765 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.765+0000 7fe52dc00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe510003f80 0x7fe5281f4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:02.766 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.765+0000 7fe52dc00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe510003f80 0x7fe5281f4840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:42792/0 (socket says 172.21.15.152:42792) 2024-04-20T11:35:02.766 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.765+0000 7fe52dc00640 1 -- 172.21.15.152:0/3556367250 learned_addr learned my addr 172.21.15.152:0/3556367250 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:02.767 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.765+0000 7fe52dc00640 1 -- 172.21.15.152:0/3556367250 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe5281f4d80 msgr2=0x7fe52814c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:02.767 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.765+0000 7fe52dc00640 1 --2- 172.21.15.152:0/3556367250 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe5281f4d80 0x7fe52814c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:02.767 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.765+0000 7fe52e600640 1 --2- 172.21.15.152:0/3556367250 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe52814c740 0x7fe5280b2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:02.767 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.765+0000 7fe52dc00640 1 -- 172.21.15.152:0/3556367250 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe52814c740 msgr2=0x7fe5280b2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:02.767 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.765+0000 7fe52dc00640 1 --2- 172.21.15.152:0/3556367250 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe52814c740 0x7fe5280b2f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:02.767 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.765+0000 7fe52dc00640 1 -- 172.21.15.152:0/3556367250 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5281f5220 con 0x7fe510003f80 2024-04-20T11:35:02.767 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.765+0000 7fe52dc00640 1 --2- 172.21.15.152:0/3556367250 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe510003f80 0x7fe5281f4840 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7fe51c00c760 tx=0x7fe51c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:02.767 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.766+0000 7fe525600640 1 -- 172.21.15.152:0/3556367250 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe51c018070 con 0x7fe510003f80 2024-04-20T11:35:02.767 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.766+0000 7fe525600640 1 -- 172.21.15.152:0/3556367250 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe51c0072d0 con 0x7fe510003f80 2024-04-20T11:35:02.767 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.766+0000 7fe525600640 1 -- 172.21.15.152:0/3556367250 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe51c0047e0 con 0x7fe510003f80 2024-04-20T11:35:02.767 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.766+0000 7fe530600640 1 -- 172.21.15.152:0/3556367250 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5281f5500 con 0x7fe510003f80 2024-04-20T11:35:02.767 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.766+0000 7fe530600640 1 -- 172.21.15.152:0/3556367250 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe528149eb0 con 0x7fe510003f80 2024-04-20T11:35:02.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.767+0000 7fe530600640 1 -- 172.21.15.152:0/3556367250 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5280689f0 con 0x7fe510003f80 2024-04-20T11:35:02.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.768+0000 7fe525600640 1 -- 172.21.15.152:0/3556367250 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 21) ==== 91550+0+0 (secure 0 0 0) 0x7fe51c004980 con 0x7fe510003f80 2024-04-20T11:35:02.769 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.769+0000 7fe525600640 1 --2- 172.21.15.152:0/3556367250 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe50806ce90 0x7fe50806f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:02.769 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.769+0000 7fe525600640 1 -- 172.21.15.152:0/3556367250 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5765+0+0 (secure 0 0 0) 0x7fe51c08bb20 con 0x7fe510003f80 2024-04-20T11:35:02.770 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.770+0000 7fe52d200640 1 --2- 172.21.15.152:0/3556367250 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe50806ce90 0x7fe50806f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:02.771 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.771+0000 7fe52d200640 1 --2- 172.21.15.152:0/3556367250 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe50806ce90 0x7fe50806f350 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe514036000 tx=0x7fe51402e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:02.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.774+0000 7fe525600640 1 -- 172.21.15.152:0/3556367250 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe51c004c70 con 0x7fe510003f80 2024-04-20T11:35:02.978 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.977+0000 7fe530600640 1 -- 172.21.15.152:0/3556367250 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe52806ecf0 con 0x7fe510003f80 2024-04-20T11:35:02.978 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.978+0000 7fe525600640 1 -- 172.21.15.152:0/3556367250 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v119) ==== 93+0+10 (secure 0 0 0) 0x7fe51c059730 con 0x7fe510003f80 2024-04-20T11:35:02.979 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 1 2024-04-20T11:35:02.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.982+0000 7fe530600640 1 -- 172.21.15.152:0/3556367250 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe50806ce90 msgr2=0x7fe50806f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:02.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.982+0000 7fe530600640 1 --2- 172.21.15.152:0/3556367250 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe50806ce90 0x7fe50806f350 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe514036000 tx=0x7fe51402e000 comp rx=0 tx=0).stop 2024-04-20T11:35:02.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.982+0000 7fe530600640 1 -- 172.21.15.152:0/3556367250 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe510003f80 msgr2=0x7fe5281f4840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:02.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.982+0000 7fe530600640 1 --2- 172.21.15.152:0/3556367250 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe510003f80 0x7fe5281f4840 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7fe51c00c760 tx=0x7fe51c00cc20 comp rx=0 tx=0).stop 2024-04-20T11:35:02.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.982+0000 7fe530600640 1 -- 172.21.15.152:0/3556367250 shutdown_connections 2024-04-20T11:35:02.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.982+0000 7fe530600640 1 --2- 172.21.15.152:0/3556367250 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe50806ce90 0x7fe50806f350 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:02.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.982+0000 7fe530600640 1 --2- 172.21.15.152:0/3556367250 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe52814c740 0x7fe5280b2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:02.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.982+0000 7fe530600640 1 --2- 172.21.15.152:0/3556367250 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe5281f4d80 0x7fe52814c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:02.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.982+0000 7fe530600640 1 --2- 172.21.15.152:0/3556367250 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe510003f80 0x7fe5281f4840 unknown :-1 s=CLOSED pgs=1675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:02.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.982+0000 7fe530600640 1 -- 172.21.15.152:0/3556367250 >> 172.21.15.152:0/3556367250 conn(0x7fe52806b980 msgr2=0x7fe5280ce2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:02.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.982+0000 7fe530600640 1 -- 172.21.15.152:0/3556367250 shutdown_connections 2024-04-20T11:35:02.984 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:02.983+0000 7fe530600640 1 -- 172.21.15.152:0/3556367250 wait complete. 2024-04-20T11:35:03.002 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T11:35:03.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.202+0000 7f320b200640 1 Processor -- start 2024-04-20T11:35:03.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.202+0000 7f320b200640 1 -- start start 2024-04-20T11:35:03.204 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.203+0000 7f320b200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f320c071cd0 0x7f320c0720b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:03.204 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.204+0000 7f320b200640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f320c06fb40 con 0x7f320c141450 2024-04-20T11:35:03.204 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.204+0000 7f320b200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f320c0725f0 con 0x7f320c160220 2024-04-20T11:35:03.205 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.204+0000 7f320b200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f320c13f8e0 con 0x7f320c071cd0 2024-04-20T11:35:03.205 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.204+0000 7f320a800640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f320c141450 0x7f320c1418e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:58742/0 (socket says 172.21.15.152:58742) 2024-04-20T11:35:03.205 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.204+0000 7f3209e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f320c071cd0 0x7f320c0720b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:03.205 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.204+0000 7f320a800640 1 -- 172.21.15.152:0/2908938818 learned_addr learned my addr 172.21.15.152:0/2908938818 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:35:03.205 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.204+0000 7f3209400640 1 -- 172.21.15.152:0/2908938818 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2098913462 0 0) 0x7f320c06fb40 con 0x7f320c141450 2024-04-20T11:35:03.205 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.205+0000 7f3209400640 1 -- 172.21.15.152:0/2908938818 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f31f0003520 con 0x7f320c141450 2024-04-20T11:35:03.205 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.205+0000 7f3209e00640 1 -- 172.21.15.152:0/2908938818 >> v1:172.21.15.175:6789/0 conn(0x7f320c160220 legacy=0x7f320c1606b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:03.206 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.205+0000 7f3209e00640 1 -- 172.21.15.152:0/2908938818 >> v1:172.21.15.152:6789/0 conn(0x7f320c141450 legacy=0x7f320c1418e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:03.206 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.205+0000 7f3209e00640 1 -- 172.21.15.152:0/2908938818 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f320c160f90 con 0x7f320c071cd0 2024-04-20T11:35:03.206 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.206+0000 7f3209e00640 1 --2- 172.21.15.152:0/2908938818 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f320c071cd0 0x7f320c0720b0 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f31fc0097a0 tx=0x7f31fc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7594c1061bc79e96 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:03.208 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.206+0000 7f3209400640 1 -- 172.21.15.152:0/2908938818 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31fc035020 con 0x7f320c071cd0 2024-04-20T11:35:03.208 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.207+0000 7f3209400640 1 -- 172.21.15.152:0/2908938818 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31fc031830 con 0x7f320c071cd0 2024-04-20T11:35:03.208 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.207+0000 7f3209400640 1 -- 172.21.15.152:0/2908938818 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31fc031be0 con 0x7f320c071cd0 2024-04-20T11:35:03.208 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.207+0000 7f320b200640 1 -- 172.21.15.152:0/2908938818 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f320c071cd0 msgr2=0x7f320c0720b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:03.208 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.207+0000 7f320b200640 1 --2- 172.21.15.152:0/2908938818 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f320c071cd0 0x7f320c0720b0 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f31fc0097a0 tx=0x7f31fc02b8d0 comp rx=0 tx=0).stop 2024-04-20T11:35:03.208 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.207+0000 7f320b200640 1 -- 172.21.15.152:0/2908938818 shutdown_connections 2024-04-20T11:35:03.208 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.207+0000 7f320b200640 1 --2- 172.21.15.152:0/2908938818 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f320c071cd0 0x7f320c0720b0 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:03.208 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.207+0000 7f320b200640 1 -- 172.21.15.152:0/2908938818 >> 172.21.15.152:0/2908938818 conn(0x7f320c067ae0 msgr2=0x7f320c06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:03.209 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.207+0000 7f320b200640 1 -- 172.21.15.152:0/2908938818 shutdown_connections 2024-04-20T11:35:03.209 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.207+0000 7f320b200640 1 -- 172.21.15.152:0/2908938818 wait complete. 2024-04-20T11:35:03.209 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.209+0000 7f320b200640 1 Processor -- start 2024-04-20T11:35:03.209 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.209+0000 7f320b200640 1 -- start start 2024-04-20T11:35:03.211 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.210+0000 7f320b200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f320c071cd0 0x7f320c0ac8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:03.212 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.210+0000 7f320b200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f320c141450 0x7f320c15dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:03.212 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.210+0000 7f320b200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f320c160220 0x7f320c13c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:03.212 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.210+0000 7f320b200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f320c06e350 con 0x7f320c141450 2024-04-20T11:35:03.212 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.210+0000 7f320b200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f320c06e070 con 0x7f320c071cd0 2024-04-20T11:35:03.212 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.210+0000 7f320b200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f320c06e630 con 0x7f320c160220 2024-04-20T11:35:03.212 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.210+0000 7f3201400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f320c141450 0x7f320c15dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:03.212 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.210+0000 7f3201400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f320c141450 0x7f320c15dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:42820/0 (socket says 172.21.15.152:42820) 2024-04-20T11:35:03.212 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.210+0000 7f3201400640 1 -- 172.21.15.152:0/3240274893 learned_addr learned my addr 172.21.15.152:0/3240274893 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:03.212 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.211+0000 7f3209e00640 1 --2- 172.21.15.152:0/3240274893 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f320c071cd0 0x7f320c0ac8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:03.212 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.211+0000 7f320a800640 1 --2- 172.21.15.152:0/3240274893 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f320c160220 0x7f320c13c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:03.213 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.211+0000 7f3201400640 1 -- 172.21.15.152:0/3240274893 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f320c160220 msgr2=0x7f320c13c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:03.213 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.211+0000 7f3201400640 1 --2- 172.21.15.152:0/3240274893 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f320c160220 0x7f320c13c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:03.213 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.211+0000 7f3201400640 1 -- 172.21.15.152:0/3240274893 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f320c071cd0 msgr2=0x7f320c0ac8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:03.213 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.211+0000 7f3201400640 1 --2- 172.21.15.152:0/3240274893 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f320c071cd0 0x7f320c0ac8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:03.213 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.211+0000 7f3201400640 1 -- 172.21.15.152:0/3240274893 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f320c0ab850 con 0x7f320c141450 2024-04-20T11:35:03.213 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.211+0000 7f3201400640 1 --2- 172.21.15.152:0/3240274893 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f320c141450 0x7f320c15dd80 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f31f400c3c0 tx=0x7f31f400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:03.214 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.212+0000 7f3202a00640 1 -- 172.21.15.152:0/3240274893 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31f40052d0 con 0x7f320c141450 2024-04-20T11:35:03.214 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.212+0000 7f3202a00640 1 -- 172.21.15.152:0/3240274893 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31f4005490 con 0x7f320c141450 2024-04-20T11:35:03.214 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.212+0000 7f3202a00640 1 -- 172.21.15.152:0/3240274893 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31f4007260 con 0x7f320c141450 2024-04-20T11:35:03.214 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.212+0000 7f320b200640 1 -- 172.21.15.152:0/3240274893 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f320c0abb30 con 0x7f320c141450 2024-04-20T11:35:03.214 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.212+0000 7f320b200640 1 -- 172.21.15.152:0/3240274893 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f320c0ac3d0 con 0x7f320c141450 2024-04-20T11:35:03.214 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.213+0000 7f320b200640 1 -- 172.21.15.152:0/3240274893 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f320c0abf20 con 0x7f320c141450 2024-04-20T11:35:03.215 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.214+0000 7f3202a00640 1 -- 172.21.15.152:0/3240274893 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 21) ==== 91550+0+0 (secure 0 0 0) 0x7f31f4007400 con 0x7f320c141450 2024-04-20T11:35:03.216 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.215+0000 7f3202a00640 1 --2- 172.21.15.152:0/3240274893 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f31e806ce90 0x7f31e806f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:03.216 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.216+0000 7f3202a00640 1 -- 172.21.15.152:0/3240274893 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5765+0+0 (secure 0 0 0) 0x7f31f408ba60 con 0x7f320c141450 2024-04-20T11:35:03.216 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.216+0000 7f3209e00640 1 --2- 172.21.15.152:0/3240274893 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f31e806ce90 0x7f31e806f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:03.218 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.217+0000 7f3209e00640 1 --2- 172.21.15.152:0/3240274893 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f31e806ce90 0x7f31e806f350 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f31fc036000 tx=0x7f31fc008030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:03.220 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.220+0000 7f3202a00640 1 -- 172.21.15.152:0/3240274893 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f31f4059670 con 0x7f320c141450 2024-04-20T11:35:03.418 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.417+0000 7f320b200640 1 -- 172.21.15.152:0/3240274893 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f320c075e60 con 0x7f320c141450 2024-04-20T11:35:03.418 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.418+0000 7f3202a00640 1 -- 172.21.15.152:0/3240274893 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v119) ==== 104+0+11 (secure 0 0 0) 0x7f31f4058f70 con 0x7f320c141450 2024-04-20T11:35:03.419 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 32 2024-04-20T11:35:03.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.420+0000 7f320b200640 1 -- 172.21.15.152:0/3240274893 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f31e806ce90 msgr2=0x7f31e806f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:03.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.420+0000 7f320b200640 1 --2- 172.21.15.152:0/3240274893 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f31e806ce90 0x7f31e806f350 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f31fc036000 tx=0x7f31fc008030 comp rx=0 tx=0).stop 2024-04-20T11:35:03.422 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.420+0000 7f320b200640 1 -- 172.21.15.152:0/3240274893 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f320c141450 msgr2=0x7f320c15dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:03.422 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.420+0000 7f320b200640 1 --2- 172.21.15.152:0/3240274893 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f320c141450 0x7f320c15dd80 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f31f400c3c0 tx=0x7f31f400c880 comp rx=0 tx=0).stop 2024-04-20T11:35:03.422 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.421+0000 7f320b200640 1 -- 172.21.15.152:0/3240274893 shutdown_connections 2024-04-20T11:35:03.422 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.421+0000 7f320b200640 1 --2- 172.21.15.152:0/3240274893 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f31e806ce90 0x7f31e806f350 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:03.422 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.421+0000 7f320b200640 1 --2- 172.21.15.152:0/3240274893 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f320c160220 0x7f320c13c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:03.422 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.421+0000 7f320b200640 1 --2- 172.21.15.152:0/3240274893 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f320c141450 0x7f320c15dd80 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:03.422 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.421+0000 7f320b200640 1 --2- 172.21.15.152:0/3240274893 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f320c071cd0 0x7f320c0ac8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:03.422 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.421+0000 7f320b200640 1 -- 172.21.15.152:0/3240274893 >> 172.21.15.152:0/3240274893 conn(0x7f320c067ae0 msgr2=0x7f320c13fb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:03.422 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.421+0000 7f320b200640 1 -- 172.21.15.152:0/3240274893 shutdown_connections 2024-04-20T11:35:03.422 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.421+0000 7f320b200640 1 -- 172.21.15.152:0/3240274893 wait complete. 2024-04-20T11:35:03.439 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T11:35:03.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.635+0000 7f0317e00640 1 Processor -- start 2024-04-20T11:35:03.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.635+0000 7f0317e00640 1 -- start start 2024-04-20T11:35:03.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.637+0000 7f0317e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f03180cfe00 0x7f03180d31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:03.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.637+0000 7f0317e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f03180d3720 0x7f031815e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:03.638 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.637+0000 7f0317e00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03180ced60 con 0x7f03181622c0 2024-04-20T11:35:03.638 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.637+0000 7f0317e00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f0318163170 con 0x7f03180d3720 2024-04-20T11:35:03.638 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.637+0000 7f0317e00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f03180cf970 con 0x7f03180cfe00 2024-04-20T11:35:03.638 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.637+0000 7f0316a00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f03181622c0 0x7f03181626a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:58746/0 (socket says 172.21.15.152:58746) 2024-04-20T11:35:03.638 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.637+0000 7f0316a00640 1 -- 172.21.15.152:0/2754712807 learned_addr learned my addr 172.21.15.152:0/2754712807 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:35:03.638 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.637+0000 7f0317400640 1 --2- 172.21.15.152:0/2754712807 >> v2:172.21.15.152:3300/0 conn(0x7f03180d3720 0x7f031815e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:03.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.637+0000 7f0316000640 1 --2- 172.21.15.152:0/2754712807 >> v2:172.21.15.175:3300/0 conn(0x7f03180cfe00 0x7f03180d31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:03.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.637+0000 7f0315600640 1 -- 172.21.15.152:0/2754712807 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1118800365 0 0) 0x7f03180ced60 con 0x7f03181622c0 2024-04-20T11:35:03.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.637+0000 7f0315600640 1 -- 172.21.15.152:0/2754712807 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f02fc003520 con 0x7f03181622c0 2024-04-20T11:35:03.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.638+0000 7f0317400640 1 -- 172.21.15.152:0/2754712807 >> v2:172.21.15.175:3300/0 conn(0x7f03180cfe00 msgr2=0x7f03180d31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:03.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.638+0000 7f0317400640 1 --2- 172.21.15.152:0/2754712807 >> v2:172.21.15.175:3300/0 conn(0x7f03180cfe00 0x7f03180d31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:03.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.638+0000 7f0317400640 1 -- 172.21.15.152:0/2754712807 >> v1:172.21.15.152:6789/0 conn(0x7f03181622c0 legacy=0x7f03181626a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:03.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.638+0000 7f0317400640 1 -- 172.21.15.152:0/2754712807 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03180cf040 con 0x7f03180d3720 2024-04-20T11:35:03.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.639+0000 7f0317400640 1 --2- 172.21.15.152:0/2754712807 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f03180d3720 0x7f031815e9b0 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f03180d55e0 tx=0x7f031002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a994a644960fd870 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:03.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.639+0000 7f0315600640 1 -- 172.21.15.152:0/2754712807 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0310039070 con 0x7f03180d3720 2024-04-20T11:35:03.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.639+0000 7f0315600640 1 -- 172.21.15.152:0/2754712807 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f031002cc80 con 0x7f03180d3720 2024-04-20T11:35:03.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.639+0000 7f0315600640 1 -- 172.21.15.152:0/2754712807 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0310034660 con 0x7f03180d3720 2024-04-20T11:35:03.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.639+0000 7f0317e00640 1 -- 172.21.15.152:0/2754712807 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f03180d3720 msgr2=0x7f031815e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:03.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.640+0000 7f0317e00640 1 --2- 172.21.15.152:0/2754712807 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f03180d3720 0x7f031815e9b0 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f03180d55e0 tx=0x7f031002bed0 comp rx=0 tx=0).stop 2024-04-20T11:35:03.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.640+0000 7f0317e00640 1 -- 172.21.15.152:0/2754712807 shutdown_connections 2024-04-20T11:35:03.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.640+0000 7f0317e00640 1 --2- 172.21.15.152:0/2754712807 >> v2:172.21.15.175:3300/0 conn(0x7f03180cfe00 0x7f03180d31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:03.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.640+0000 7f0317e00640 1 --2- 172.21.15.152:0/2754712807 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f03180d3720 0x7f031815e9b0 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:03.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.640+0000 7f0317e00640 1 -- 172.21.15.152:0/2754712807 >> 172.21.15.152:0/2754712807 conn(0x7f031808bfd0 msgr2=0x7f03180d4570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:03.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.640+0000 7f0317e00640 1 -- 172.21.15.152:0/2754712807 shutdown_connections 2024-04-20T11:35:03.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.640+0000 7f0317e00640 1 -- 172.21.15.152:0/2754712807 wait complete. 2024-04-20T11:35:03.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.641+0000 7f0317e00640 1 Processor -- start 2024-04-20T11:35:03.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.641+0000 7f0317e00640 1 -- start start 2024-04-20T11:35:03.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.642+0000 7f0317e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f03180cfe00 0x7f03181f4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:03.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.642+0000 7f0317e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f03180d3720 0x7f031814cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:03.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.642+0000 7f0317e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03181622c0 0x7f031814fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:03.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.642+0000 7f0317e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f031815eef0 con 0x7f03180d3720 2024-04-20T11:35:03.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.642+0000 7f0317e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f03180cf600 con 0x7f03180cfe00 2024-04-20T11:35:03.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.642+0000 7f0317e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f031815f070 con 0x7f03181622c0 2024-04-20T11:35:03.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.643+0000 7f0316000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f03180d3720 0x7f031814cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:03.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.643+0000 7f0316000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f03180d3720 0x7f031814cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:42838/0 (socket says 172.21.15.152:42838) 2024-04-20T11:35:03.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.643+0000 7f0316000640 1 -- 172.21.15.152:0/1619072864 learned_addr learned my addr 172.21.15.152:0/1619072864 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:03.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.643+0000 7f0316000640 1 -- 172.21.15.152:0/1619072864 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03181622c0 msgr2=0x7f031814fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:03.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.643+0000 7f0316000640 1 --2- 172.21.15.152:0/1619072864 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03181622c0 0x7f031814fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:03.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.643+0000 7f0316000640 1 -- 172.21.15.152:0/1619072864 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f03180cfe00 msgr2=0x7f03181f4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:03.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.643+0000 7f0316000640 1 --2- 172.21.15.152:0/1619072864 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f03180cfe00 0x7f03181f4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:03.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.643+0000 7f0316000640 1 -- 172.21.15.152:0/1619072864 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f031814d4f0 con 0x7f03180d3720 2024-04-20T11:35:03.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.643+0000 7f0316000640 1 --2- 172.21.15.152:0/1619072864 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f03180d3720 0x7f031814cd20 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f030400d7d0 tx=0x7f030400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:03.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.644+0000 7f030ea00640 1 -- 172.21.15.152:0/1619072864 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0304004020 con 0x7f03180d3720 2024-04-20T11:35:03.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.644+0000 7f030ea00640 1 -- 172.21.15.152:0/1619072864 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03040041c0 con 0x7f03180d3720 2024-04-20T11:35:03.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.644+0000 7f030ea00640 1 -- 172.21.15.152:0/1619072864 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03040043b0 con 0x7f03180d3720 2024-04-20T11:35:03.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.644+0000 7f0317e00640 1 -- 172.21.15.152:0/1619072864 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03181f4f50 con 0x7f03180d3720 2024-04-20T11:35:03.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.644+0000 7f0317e00640 1 -- 172.21.15.152:0/1619072864 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03181f5510 con 0x7f03180d3720 2024-04-20T11:35:03.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.645+0000 7f0317e00640 1 -- 172.21.15.152:0/1619072864 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03180d0cc0 con 0x7f03180d3720 2024-04-20T11:35:03.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.646+0000 7f030ea00640 1 -- 172.21.15.152:0/1619072864 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 21) ==== 91550+0+0 (secure 0 0 0) 0x7f030401b020 con 0x7f03180d3720 2024-04-20T11:35:03.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.647+0000 7f030ea00640 1 --2- 172.21.15.152:0/1619072864 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f02f806ce90 0x7f02f806f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:03.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.647+0000 7f030ea00640 1 -- 172.21.15.152:0/1619072864 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5765+0+0 (secure 0 0 0) 0x7f030408c0c0 con 0x7f03180d3720 2024-04-20T11:35:03.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.648+0000 7f0316a00640 1 --2- 172.21.15.152:0/1619072864 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f02f806ce90 0x7f02f806f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:03.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.649+0000 7f0316a00640 1 --2- 172.21.15.152:0/1619072864 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f02f806ce90 0x7f02f806f350 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f0308000f60 tx=0x7f03080078f0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:03.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.653+0000 7f030ea00640 1 -- 172.21.15.152:0/1619072864 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f03180d0cc0 con 0x7f03180d3720 2024-04-20T11:35:03.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.855+0000 7f0317e00640 1 -- 172.21.15.152:0/1619072864 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f0318075e60 con 0x7f03180d3720 2024-04-20T11:35:03.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.856+0000 7f030ea00640 1 -- 172.21.15.152:0/1619072864 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v119) ==== 100+0+11 (secure 0 0 0) 0x7f0304022020 con 0x7f03180d3720 2024-04-20T11:35:03.856 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 64 2024-04-20T11:35:03.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.858+0000 7f0317e00640 1 -- 172.21.15.152:0/1619072864 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f02f806ce90 msgr2=0x7f02f806f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:03.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.858+0000 7f0317e00640 1 --2- 172.21.15.152:0/1619072864 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f02f806ce90 0x7f02f806f350 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f0308000f60 tx=0x7f03080078f0 comp rx=0 tx=0).stop 2024-04-20T11:35:03.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.858+0000 7f0317e00640 1 -- 172.21.15.152:0/1619072864 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f03180d3720 msgr2=0x7f031814cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:03.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.858+0000 7f0317e00640 1 --2- 172.21.15.152:0/1619072864 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f03180d3720 0x7f031814cd20 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f030400d7d0 tx=0x7f030400dc90 comp rx=0 tx=0).stop 2024-04-20T11:35:03.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.858+0000 7f0317e00640 1 -- 172.21.15.152:0/1619072864 shutdown_connections 2024-04-20T11:35:03.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.858+0000 7f0317e00640 1 --2- 172.21.15.152:0/1619072864 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f02f806ce90 0x7f02f806f350 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:03.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.858+0000 7f0317e00640 1 --2- 172.21.15.152:0/1619072864 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03181622c0 0x7f031814fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:03.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.858+0000 7f0317e00640 1 --2- 172.21.15.152:0/1619072864 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f03180d3720 0x7f031814cd20 unknown :-1 s=CLOSED pgs=1681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:03.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.858+0000 7f0317e00640 1 --2- 172.21.15.152:0/1619072864 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f03180cfe00 0x7f03181f4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:03.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.858+0000 7f0317e00640 1 -- 172.21.15.152:0/1619072864 >> 172.21.15.152:0/1619072864 conn(0x7f031808bfd0 msgr2=0x7f03180b6340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:03.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.859+0000 7f0317e00640 1 -- 172.21.15.152:0/1619072864 shutdown_connections 2024-04-20T11:35:03.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:03.859+0000 7f0317e00640 1 -- 172.21.15.152:0/1619072864 wait complete. 2024-04-20T11:35:03.878 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:35:04.076 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.075+0000 7ff1a6800640 1 Processor -- start 2024-04-20T11:35:04.076 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.075+0000 7ff1a6800640 1 -- start start 2024-04-20T11:35:04.078 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.077+0000 7ff1a6800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7ff1a015d1e0 0x7ff1a0160b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:04.078 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.077+0000 7ff1a6800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff1a0071cd0 0x7ff1a0072160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:04.078 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.077+0000 7ff1a6800640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1a00ce8c0 con 0x7ff1a015d5c0 2024-04-20T11:35:04.078 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.077+0000 7ff1a6800640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7ff1a00726a0 con 0x7ff1a015d1e0 2024-04-20T11:35:04.078 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.077+0000 7ff1a6800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff1a01600b0 con 0x7ff1a0071cd0 2024-04-20T11:35:04.078 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.077+0000 7ff1a4a00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7ff1a015d5c0 0x7ff1a00cdfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:58754/0 (socket says 172.21.15.152:58754) 2024-04-20T11:35:04.078 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.077+0000 7ff1a4a00640 1 -- 172.21.15.152:0/3615163475 learned_addr learned my addr 172.21.15.152:0/3615163475 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:35:04.079 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.078+0000 7ff1a5400640 1 --2- 172.21.15.152:0/3615163475 >> v2:172.21.15.152:3300/0 conn(0x7ff1a015d1e0 0x7ff1a0160b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:04.079 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.078+0000 7ff1a5e00640 1 --2- 172.21.15.152:0/3615163475 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff1a0071cd0 0x7ff1a0072160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:04.079 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.078+0000 7ff19fe00640 1 -- 172.21.15.152:0/3615163475 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 704591049 0 0) 0x7ff1a00ce8c0 con 0x7ff1a015d5c0 2024-04-20T11:35:04.079 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.078+0000 7ff19fe00640 1 -- 172.21.15.152:0/3615163475 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff188007930 con 0x7ff1a015d5c0 2024-04-20T11:35:04.080 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.078+0000 7ff1a5400640 1 -- 172.21.15.152:0/3615163475 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff1a0071cd0 msgr2=0x7ff1a0072160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:04.080 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.078+0000 7ff1a5400640 1 --2- 172.21.15.152:0/3615163475 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff1a0071cd0 0x7ff1a0072160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:04.080 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.078+0000 7ff1a5400640 1 -- 172.21.15.152:0/3615163475 >> v1:172.21.15.152:6789/0 conn(0x7ff1a015d5c0 legacy=0x7ff1a00cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:04.080 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.078+0000 7ff1a5400640 1 -- 172.21.15.152:0/3615163475 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1a0160390 con 0x7ff1a015d1e0 2024-04-20T11:35:04.080 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.080+0000 7ff1a5400640 1 --2- 172.21.15.152:0/3615163475 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff1a015d1e0 0x7ff1a0160b90 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7ff1940097d0 tx=0x7ff19402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dab1135e541c65a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:04.084 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.084+0000 7ff19fe00640 1 -- 172.21.15.152:0/3615163475 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff194035020 con 0x7ff1a015d1e0 2024-04-20T11:35:04.085 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.084+0000 7ff19fe00640 1 -- 172.21.15.152:0/3615163475 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff194031810 con 0x7ff1a015d1e0 2024-04-20T11:35:04.085 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.084+0000 7ff19fe00640 1 -- 172.21.15.152:0/3615163475 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff194031be0 con 0x7ff1a015d1e0 2024-04-20T11:35:04.085 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.085+0000 7ff1a6800640 1 -- 172.21.15.152:0/3615163475 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff1a015d1e0 msgr2=0x7ff1a0160b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:04.086 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.085+0000 7ff1a6800640 1 --2- 172.21.15.152:0/3615163475 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff1a015d1e0 0x7ff1a0160b90 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7ff1940097d0 tx=0x7ff19402b8b0 comp rx=0 tx=0).stop 2024-04-20T11:35:04.086 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.085+0000 7ff1a6800640 1 -- 172.21.15.152:0/3615163475 shutdown_connections 2024-04-20T11:35:04.086 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.085+0000 7ff1a6800640 1 --2- 172.21.15.152:0/3615163475 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff1a0071cd0 0x7ff1a0072160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:04.086 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.085+0000 7ff1a6800640 1 --2- 172.21.15.152:0/3615163475 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff1a015d1e0 0x7ff1a0160b90 secure :-1 s=CLOSED pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7ff1940097d0 tx=0x7ff19402b8b0 comp rx=0 tx=0).stop 2024-04-20T11:35:04.087 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.086+0000 7ff1a6800640 1 -- 172.21.15.152:0/3615163475 >> 172.21.15.152:0/3615163475 conn(0x7ff1a0067ae0 msgr2=0x7ff1a006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:04.087 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.086+0000 7ff1a6800640 1 -- 172.21.15.152:0/3615163475 shutdown_connections 2024-04-20T11:35:04.087 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.086+0000 7ff1a6800640 1 -- 172.21.15.152:0/3615163475 wait complete. 2024-04-20T11:35:04.088 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.088+0000 7ff1a6800640 1 Processor -- start 2024-04-20T11:35:04.088 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.088+0000 7ff1a6800640 1 -- start start 2024-04-20T11:35:04.091 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.089+0000 7ff1a6800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff1a0071cd0 0x7ff1a0152520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:04.091 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.089+0000 7ff1a6800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff1a015d5c0 0x7ff1a014f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:04.091 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.089+0000 7ff1a6800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1a00b3110 0x7ff1a00b3f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:04.091 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.089+0000 7ff1a6800640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7ff1a00ccd70 con 0x7ff1a015d5c0 2024-04-20T11:35:04.091 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.089+0000 7ff1a6800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff1a00ceba0 con 0x7ff1a00b3110 2024-04-20T11:35:04.091 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.089+0000 7ff1a6800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff1a00ccef0 con 0x7ff1a0071cd0 2024-04-20T11:35:04.091 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.089+0000 7ff1a4a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff1a015d5c0 0x7ff1a014f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:04.091 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.089+0000 7ff1a4a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff1a015d5c0 0x7ff1a014f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:42852/0 (socket says 172.21.15.152:42852) 2024-04-20T11:35:04.091 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.089+0000 7ff1a4a00640 1 -- 172.21.15.152:0/2251153390 learned_addr learned my addr 172.21.15.152:0/2251153390 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:04.091 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.089+0000 7ff1a4a00640 1 -- 172.21.15.152:0/2251153390 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff1a0071cd0 msgr2=0x7ff1a0152520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:04.091 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.089+0000 7ff1a4a00640 1 --2- 172.21.15.152:0/2251153390 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff1a0071cd0 0x7ff1a0152520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:04.091 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.089+0000 7ff1a4a00640 1 -- 172.21.15.152:0/2251153390 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1a00b3110 msgr2=0x7ff1a00b3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:04.091 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.089+0000 7ff1a4a00640 1 --2- 172.21.15.152:0/2251153390 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1a00b3110 0x7ff1a00b3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:04.092 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.089+0000 7ff1a4a00640 1 -- 172.21.15.152:0/2251153390 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1a0152a60 con 0x7ff1a015d5c0 2024-04-20T11:35:04.092 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.090+0000 7ff1a4a00640 1 --2- 172.21.15.152:0/2251153390 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff1a015d5c0 0x7ff1a014f4b0 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7ff18c00c390 tx=0x7ff18c00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:04.092 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.090+0000 7ff19d600640 1 -- 172.21.15.152:0/2251153390 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff18c017070 con 0x7ff1a015d5c0 2024-04-20T11:35:04.092 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.090+0000 7ff19d600640 1 -- 172.21.15.152:0/2251153390 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff18c005ce0 con 0x7ff1a015d5c0 2024-04-20T11:35:04.092 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.090+0000 7ff1a6800640 1 -- 172.21.15.152:0/2251153390 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1a0150950 con 0x7ff1a015d5c0 2024-04-20T11:35:04.092 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.090+0000 7ff19d600640 1 -- 172.21.15.152:0/2251153390 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff18c0072f0 con 0x7ff1a015d5c0 2024-04-20T11:35:04.092 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.090+0000 7ff1a6800640 1 -- 172.21.15.152:0/2251153390 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1a0150f10 con 0x7ff1a015d5c0 2024-04-20T11:35:04.092 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.092+0000 7ff1a6800640 1 -- 172.21.15.152:0/2251153390 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1a00cfcf0 con 0x7ff1a015d5c0 2024-04-20T11:35:04.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.093+0000 7ff19d600640 1 -- 172.21.15.152:0/2251153390 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 21) ==== 91550+0+0 (secure 0 0 0) 0x7ff18c01a070 con 0x7ff1a015d5c0 2024-04-20T11:35:04.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.094+0000 7ff19d600640 1 --2- 172.21.15.152:0/2251153390 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7ff18006ce90 0x7ff18006f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:04.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.094+0000 7ff19d600640 1 -- 172.21.15.152:0/2251153390 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5737+0+0 (secure 0 0 0) 0x7ff18c08ba60 con 0x7ff1a015d5c0 2024-04-20T11:35:04.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.094+0000 7ff1a5400640 1 --2- 172.21.15.152:0/2251153390 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7ff18006ce90 0x7ff18006f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:04.096 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.095+0000 7ff1a5400640 1 --2- 172.21.15.152:0/2251153390 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7ff18006ce90 0x7ff18006f350 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff194036000 tx=0x7ff194030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:04.098 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.098+0000 7ff19d600640 1 -- 172.21.15.152:0/2251153390 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ff1a00cfcf0 con 0x7ff1a015d5c0 2024-04-20T11:35:04.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.346+0000 7ff1a6800640 1 -- 172.21.15.152:0/2251153390 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff1a0075e60 con 0x7ff1a015d5c0 2024-04-20T11:35:04.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.347+0000 7ff19d600640 1 -- 172.21.15.152:0/2251153390 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) ==== 94+0+5207 (secure 0 0 0) 0x7ff18c0595c0 con 0x7ff1a015d5c0 2024-04-20T11:35:04.348 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:35:04.348 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":77,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5891,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/508638056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6833","nonce":508638056},{"type":"v1","addr":"172.21.15.175:6835","nonce":508638056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":6448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6834/3447077466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3447077466},{"type":"v1","addr":"172.21.15.175:6834","nonce":3447077466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":8409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6834/751442105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":751442105},{"type":"v1","addr":"172.21.15.152:6834","nonce":751442105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T11:34:36.050950+0000","modified":"2024-04-20T11:34:39.117003+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8412},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8412":{"gid":8412,"name":"c","rank":0,"incarnation":72,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6835/2477488828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6833","nonce":2477488828},{"type":"v1","addr":"172.21.15.152:6835","nonce":2477488828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8412,"qdb_cluster":[8412]},"id":9}]} 2024-04-20T11:35:04.348 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 77 2024-04-20T11:35:04.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.350+0000 7ff1a6800640 1 -- 172.21.15.152:0/2251153390 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7ff18006ce90 msgr2=0x7ff18006f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:04.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.350+0000 7ff1a6800640 1 --2- 172.21.15.152:0/2251153390 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7ff18006ce90 0x7ff18006f350 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff194036000 tx=0x7ff194030000 comp rx=0 tx=0).stop 2024-04-20T11:35:04.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.350+0000 7ff1a6800640 1 -- 172.21.15.152:0/2251153390 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff1a015d5c0 msgr2=0x7ff1a014f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:04.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.350+0000 7ff1a6800640 1 --2- 172.21.15.152:0/2251153390 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff1a015d5c0 0x7ff1a014f4b0 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7ff18c00c390 tx=0x7ff18c00c850 comp rx=0 tx=0).stop 2024-04-20T11:35:04.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.350+0000 7ff1a6800640 1 -- 172.21.15.152:0/2251153390 shutdown_connections 2024-04-20T11:35:04.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.350+0000 7ff1a6800640 1 --2- 172.21.15.152:0/2251153390 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7ff18006ce90 0x7ff18006f350 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:04.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.350+0000 7ff1a6800640 1 --2- 172.21.15.152:0/2251153390 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1a00b3110 0x7ff1a00b3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:04.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.350+0000 7ff1a6800640 1 --2- 172.21.15.152:0/2251153390 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff1a015d5c0 0x7ff1a014f4b0 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:04.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.350+0000 7ff1a6800640 1 --2- 172.21.15.152:0/2251153390 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff1a0071cd0 0x7ff1a0152520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:04.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.350+0000 7ff1a6800640 1 -- 172.21.15.152:0/2251153390 >> 172.21.15.152:0/2251153390 conn(0x7ff1a0067ae0 msgr2=0x7ff1a015ab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:04.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.351+0000 7ff1a6800640 1 -- 172.21.15.152:0/2251153390 shutdown_connections 2024-04-20T11:35:04.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.351+0000 7ff1a6800640 1 -- 172.21.15.152:0/2251153390 wait complete. 2024-04-20T11:35:04.371 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:35:04.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.559+0000 7ff493c00640 1 Processor -- start 2024-04-20T11:35:04.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.560+0000 7ff493c00640 1 -- start start 2024-04-20T11:35:04.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.561+0000 7ff493c00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ff48c145f60 0x7ff48c06ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:04.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.561+0000 7ff493c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff48c0cd0d0 0x7ff48c0cd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:04.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.561+0000 7ff493c00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff48c146340 con 0x7ff48c071fe0 2024-04-20T11:35:04.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.561+0000 7ff493c00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7ff48c06e9f0 con 0x7ff48c145f60 2024-04-20T11:35:04.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.561+0000 7ff493c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff48c06f3c0 con 0x7ff48c0cd0d0 2024-04-20T11:35:04.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.562+0000 7ff491200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ff48c145f60 0x7ff48c06ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:04.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.562+0000 7ff491200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ff48c145f60 0x7ff48c06ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.152:57304/0 (socket says 172.21.15.152:57304) 2024-04-20T11:35:04.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.562+0000 7ff491200640 1 -- 172.21.15.152:0/2111741903 learned_addr learned my addr 172.21.15.152:0/2111741903 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:04.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.562+0000 7ff483e00640 1 --2- 172.21.15.152:0/2111741903 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff48c0cd0d0 0x7ff48c0cd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:04.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.563+0000 7ff491200640 1 -- 172.21.15.152:0/2111741903 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff48c0cd0d0 msgr2=0x7ff48c0cd550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:04.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.563+0000 7ff491200640 1 --2- 172.21.15.152:0/2111741903 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff48c0cd0d0 0x7ff48c0cd550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:04.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.563+0000 7ff491200640 1 -- 172.21.15.152:0/2111741903 >> v1:172.21.15.175:6789/0 conn(0x7ff48c071fe0 legacy=0x7ff48c072470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:04.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.563+0000 7ff491200640 1 -- 172.21.15.152:0/2111741903 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff48c072d50 con 0x7ff48c145f60 2024-04-20T11:35:04.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.564+0000 7ff491200640 1 --2- 172.21.15.152:0/2111741903 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff48c145f60 0x7ff48c06ee80 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7ff48c06eb70 tx=0x7ff47c02b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a760c80ee87f9dc server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:04.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.565+0000 7ff48be00640 1 -- 172.21.15.152:0/2111741903 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff47c034070 con 0x7ff48c145f60 2024-04-20T11:35:04.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.565+0000 7ff48be00640 1 -- 172.21.15.152:0/2111741903 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff47c030840 con 0x7ff48c145f60 2024-04-20T11:35:04.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.565+0000 7ff48be00640 1 -- 172.21.15.152:0/2111741903 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff47c030c10 con 0x7ff48c145f60 2024-04-20T11:35:04.567 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.566+0000 7ff493c00640 1 -- 172.21.15.152:0/2111741903 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff48c145f60 msgr2=0x7ff48c06ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:04.567 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.566+0000 7ff493c00640 1 --2- 172.21.15.152:0/2111741903 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff48c145f60 0x7ff48c06ee80 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7ff48c06eb70 tx=0x7ff47c02b980 comp rx=0 tx=0).stop 2024-04-20T11:35:04.567 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.566+0000 7ff493c00640 1 -- 172.21.15.152:0/2111741903 shutdown_connections 2024-04-20T11:35:04.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.566+0000 7ff493c00640 1 --2- 172.21.15.152:0/2111741903 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff48c0cd0d0 0x7ff48c0cd550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:04.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.566+0000 7ff493c00640 1 --2- 172.21.15.152:0/2111741903 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff48c145f60 0x7ff48c06ee80 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:04.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.566+0000 7ff493c00640 1 -- 172.21.15.152:0/2111741903 >> 172.21.15.152:0/2111741903 conn(0x7ff48c067cf0 msgr2=0x7ff48c06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:04.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.566+0000 7ff493c00640 1 -- 172.21.15.152:0/2111741903 shutdown_connections 2024-04-20T11:35:04.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.566+0000 7ff493c00640 1 -- 172.21.15.152:0/2111741903 wait complete. 2024-04-20T11:35:04.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.569+0000 7ff493c00640 1 Processor -- start 2024-04-20T11:35:04.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.569+0000 7ff493c00640 1 -- start start 2024-04-20T11:35:04.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.570+0000 7ff493c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff48c071fe0 0x7ff48c142e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:04.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.570+0000 7ff493c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff48c0cd0d0 0x7ff48c0b3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:04.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.570+0000 7ff493c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff48c145f60 0x7ff48c0b3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:04.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.570+0000 7ff493c00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7ff48c06e680 con 0x7ff48c145f60 2024-04-20T11:35:04.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.570+0000 7ff493c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff48c06e3a0 con 0x7ff48c0cd0d0 2024-04-20T11:35:04.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.570+0000 7ff493c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff48c0cf290 con 0x7ff48c071fe0 2024-04-20T11:35:04.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.570+0000 7ff491c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff48c145f60 0x7ff48c0b3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:04.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.571+0000 7ff491c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff48c145f60 0x7ff48c0b3ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:42862/0 (socket says 172.21.15.152:42862) 2024-04-20T11:35:04.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.571+0000 7ff491c00640 1 -- 172.21.15.152:0/924691633 learned_addr learned my addr 172.21.15.152:0/924691633 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:04.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.571+0000 7ff491200640 1 --2- 172.21.15.152:0/924691633 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff48c071fe0 0x7ff48c142e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:04.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.571+0000 7ff483e00640 1 --2- 172.21.15.152:0/924691633 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff48c0cd0d0 0x7ff48c0b3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:04.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.571+0000 7ff491c00640 1 -- 172.21.15.152:0/924691633 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff48c071fe0 msgr2=0x7ff48c142e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:04.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.571+0000 7ff491c00640 1 --2- 172.21.15.152:0/924691633 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff48c071fe0 0x7ff48c142e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:04.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.571+0000 7ff491c00640 1 -- 172.21.15.152:0/924691633 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff48c0cd0d0 msgr2=0x7ff48c0b3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:04.574 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.571+0000 7ff491c00640 1 --2- 172.21.15.152:0/924691633 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff48c0cd0d0 0x7ff48c0b3570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:04.574 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.571+0000 7ff491c00640 1 -- 172.21.15.152:0/924691633 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff48c0c72d0 con 0x7ff48c145f60 2024-04-20T11:35:04.574 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.571+0000 7ff483e00640 1 --2- 172.21.15.152:0/924691633 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff48c0cd0d0 0x7ff48c0b3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:35:04.574 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.571+0000 7ff491c00640 1 --2- 172.21.15.152:0/924691633 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff48c145f60 0x7ff48c0b3ab0 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7ff48400c7a0 tx=0x7ff48400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:04.574 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.572+0000 7ff489600640 1 -- 172.21.15.152:0/924691633 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff484003b70 con 0x7ff48c145f60 2024-04-20T11:35:04.574 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.572+0000 7ff489600640 1 -- 172.21.15.152:0/924691633 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff484003d30 con 0x7ff48c145f60 2024-04-20T11:35:04.574 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.572+0000 7ff489600640 1 -- 172.21.15.152:0/924691633 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff48400aa90 con 0x7ff48c145f60 2024-04-20T11:35:04.574 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.572+0000 7ff493c00640 1 -- 172.21.15.152:0/924691633 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff48c0c75b0 con 0x7ff48c145f60 2024-04-20T11:35:04.574 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.572+0000 7ff493c00640 1 -- 172.21.15.152:0/924691633 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff48c0cad40 con 0x7ff48c145f60 2024-04-20T11:35:04.574 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.573+0000 7ff493c00640 1 -- 172.21.15.152:0/924691633 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff48c0c7ad0 con 0x7ff48c145f60 2024-04-20T11:35:04.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.574+0000 7ff489600640 1 -- 172.21.15.152:0/924691633 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 21) ==== 91550+0+0 (secure 0 0 0) 0x7ff48400ac30 con 0x7ff48c145f60 2024-04-20T11:35:04.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.575+0000 7ff489600640 1 --2- 172.21.15.152:0/924691633 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7ff46806ce90 0x7ff46806f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:04.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.575+0000 7ff489600640 1 -- 172.21.15.152:0/924691633 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5737+0+0 (secure 0 0 0) 0x7ff48408c540 con 0x7ff48c145f60 2024-04-20T11:35:04.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.576+0000 7ff491200640 1 --2- 172.21.15.152:0/924691633 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7ff46806ce90 0x7ff46806f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:04.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.577+0000 7ff491200640 1 --2- 172.21.15.152:0/924691633 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7ff46806ce90 0x7ff46806f350 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7ff47c036000 tx=0x7ff47c03b040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:04.580 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.580+0000 7ff489600640 1 -- 172.21.15.152:0/924691633 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ff48c0c7ad0 con 0x7ff48c145f60 2024-04-20T11:35:04.770 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.770+0000 7ff493c00640 1 -- 172.21.15.152:0/924691633 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff48c0689f0 con 0x7ff48c145f60 2024-04-20T11:35:04.772 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.772+0000 7ff489600640 1 -- 172.21.15.152:0/924691633 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v120) ==== 74+0+16806 (secure 0 0 0) 0x7ff48405a130 con 0x7ff48c145f60 2024-04-20T11:35:04.773 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:35:04.773 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":120,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:35:04.078143+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:34:33.781054+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-04-20T11:34:35.031728+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":119,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":19,"snaps":[{"begin":2,"length":4}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:35:04.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.774+0000 7ff493c00640 1 -- 172.21.15.152:0/924691633 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7ff46806ce90 msgr2=0x7ff46806f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:04.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.774+0000 7ff493c00640 1 --2- 172.21.15.152:0/924691633 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7ff46806ce90 0x7ff46806f350 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7ff47c036000 tx=0x7ff47c03b040 comp rx=0 tx=0).stop 2024-04-20T11:35:04.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.774+0000 7ff493c00640 1 -- 172.21.15.152:0/924691633 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff48c145f60 msgr2=0x7ff48c0b3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:04.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.774+0000 7ff493c00640 1 --2- 172.21.15.152:0/924691633 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff48c145f60 0x7ff48c0b3ab0 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7ff48400c7a0 tx=0x7ff48400cc60 comp rx=0 tx=0).stop 2024-04-20T11:35:04.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.774+0000 7ff493c00640 1 -- 172.21.15.152:0/924691633 shutdown_connections 2024-04-20T11:35:04.776 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.774+0000 7ff493c00640 1 --2- 172.21.15.152:0/924691633 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7ff46806ce90 0x7ff46806f350 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:04.776 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.774+0000 7ff493c00640 1 --2- 172.21.15.152:0/924691633 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff48c145f60 0x7ff48c0b3ab0 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:04.776 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.774+0000 7ff493c00640 1 --2- 172.21.15.152:0/924691633 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff48c0cd0d0 0x7ff48c0b3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:04.776 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.774+0000 7ff493c00640 1 --2- 172.21.15.152:0/924691633 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff48c071fe0 0x7ff48c142e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:04.776 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.774+0000 7ff493c00640 1 -- 172.21.15.152:0/924691633 >> 172.21.15.152:0/924691633 conn(0x7ff48c067cf0 msgr2=0x7ff48c0d08d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:04.776 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.775+0000 7ff493c00640 1 -- 172.21.15.152:0/924691633 shutdown_connections 2024-04-20T11:35:04.776 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.775+0000 7ff493c00640 1 -- 172.21.15.152:0/924691633 wait complete. 2024-04-20T11:35:04.794 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-20T11:35:04.794 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:35:04.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.988+0000 7fb632400640 1 Processor -- start 2024-04-20T11:35:04.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.988+0000 7fb632400640 1 -- start start 2024-04-20T11:35:04.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.989+0000 7fb632400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fb62c0cd0d0 0x7fb62c0cd580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:04.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.990+0000 7fb632400640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fb62c071fe0 0x7fb62c072470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:04.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.990+0000 7fb632400640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb62c146340 con 0x7fb62c145f60 2024-04-20T11:35:04.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.990+0000 7fb632400640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fb62c06e9f0 con 0x7fb62c0cd0d0 2024-04-20T11:35:04.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.990+0000 7fb632400640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fb62c072d50 con 0x7fb62c071fe0 2024-04-20T11:35:04.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.990+0000 7fb62aa00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fb62c0cd0d0 0x7fb62c0cd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:04.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.990+0000 7fb62b400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fb62c145f60 0x7fb62c06ee80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:58760/0 (socket says 172.21.15.152:58760) 2024-04-20T11:35:04.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.990+0000 7fb62b400640 1 -- 172.21.15.152:0/1320540967 learned_addr learned my addr 172.21.15.152:0/1320540967 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:35:04.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.990+0000 7fb62aa00640 1 -- 172.21.15.152:0/1320540967 >> v2:172.21.15.175:3300/0 conn(0x7fb62c071fe0 msgr2=0x7fb62c072470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:04.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.990+0000 7fb62aa00640 1 --2- 172.21.15.152:0/1320540967 >> v2:172.21.15.175:3300/0 conn(0x7fb62c071fe0 0x7fb62c072470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:04.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.990+0000 7fb62aa00640 1 -- 172.21.15.152:0/1320540967 >> v1:172.21.15.152:6789/0 conn(0x7fb62c145f60 legacy=0x7fb62c06ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:04.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.990+0000 7fb62aa00640 1 -- 172.21.15.152:0/1320540967 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb62c06e0c0 con 0x7fb62c0cd0d0 2024-04-20T11:35:04.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.990+0000 7fb62a000640 1 -- 172.21.15.152:0/1320540967 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3252878253 0 0) 0x7fb62c146340 con 0x7fb62c145f60 2024-04-20T11:35:04.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.992+0000 7fb62aa00640 1 --2- 172.21.15.152:0/1320540967 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb62c0cd0d0 0x7fb62c0cd580 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7fb62c06eb70 tx=0x7fb61802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27748ff50d5ca797 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:04.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.992+0000 7fb62a000640 1 -- 172.21.15.152:0/1320540967 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb618035020 con 0x7fb62c0cd0d0 2024-04-20T11:35:04.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.992+0000 7fb62a000640 1 -- 172.21.15.152:0/1320540967 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb618031830 con 0x7fb62c0cd0d0 2024-04-20T11:35:04.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.992+0000 7fb62a000640 1 -- 172.21.15.152:0/1320540967 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb618031c00 con 0x7fb62c0cd0d0 2024-04-20T11:35:04.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.992+0000 7fb632400640 1 -- 172.21.15.152:0/1320540967 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb62c0cd0d0 msgr2=0x7fb62c0cd580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:04.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.992+0000 7fb632400640 1 --2- 172.21.15.152:0/1320540967 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb62c0cd0d0 0x7fb62c0cd580 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7fb62c06eb70 tx=0x7fb61802b8d0 comp rx=0 tx=0).stop 2024-04-20T11:35:04.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.992+0000 7fb632400640 1 -- 172.21.15.152:0/1320540967 shutdown_connections 2024-04-20T11:35:04.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.992+0000 7fb632400640 1 --2- 172.21.15.152:0/1320540967 >> v2:172.21.15.175:3300/0 conn(0x7fb62c071fe0 0x7fb62c072470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:04.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.993+0000 7fb632400640 1 --2- 172.21.15.152:0/1320540967 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb62c0cd0d0 0x7fb62c0cd580 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:04.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.993+0000 7fb632400640 1 -- 172.21.15.152:0/1320540967 >> 172.21.15.152:0/1320540967 conn(0x7fb62c067cf0 msgr2=0x7fb62c06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:04.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.993+0000 7fb632400640 1 -- 172.21.15.152:0/1320540967 shutdown_connections 2024-04-20T11:35:04.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.993+0000 7fb632400640 1 -- 172.21.15.152:0/1320540967 wait complete. 2024-04-20T11:35:04.995 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.994+0000 7fb632400640 1 Processor -- start 2024-04-20T11:35:04.995 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.994+0000 7fb632400640 1 -- start start 2024-04-20T11:35:04.997 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.995+0000 7fb632400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb62c071fe0 0x7fb62c0d0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:04.997 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.995+0000 7fb632400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb62c0cd0d0 0x7fb62c131c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:04.997 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.995+0000 7fb632400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb62c145f60 0x7fb62c0a9760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:04.997 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.995+0000 7fb632400640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fb62c0729b0 con 0x7fb62c0cd0d0 2024-04-20T11:35:04.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.995+0000 7fb632400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb62c06e680 con 0x7fb62c145f60 2024-04-20T11:35:04.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.995+0000 7fb632400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb62c072b30 con 0x7fb62c071fe0 2024-04-20T11:35:04.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.996+0000 7fb62aa00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb62c0cd0d0 0x7fb62c131c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:04.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.996+0000 7fb62aa00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb62c0cd0d0 0x7fb62c131c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:42884/0 (socket says 172.21.15.152:42884) 2024-04-20T11:35:04.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.996+0000 7fb62aa00640 1 -- 172.21.15.152:0/3382200365 learned_addr learned my addr 172.21.15.152:0/3382200365 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:04.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.996+0000 7fb62b400640 1 --2- 172.21.15.152:0/3382200365 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb62c071fe0 0x7fb62c0d0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:04.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.996+0000 7fb62aa00640 1 -- 172.21.15.152:0/3382200365 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb62c071fe0 msgr2=0x7fb62c0d0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:04.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.996+0000 7fb62aa00640 1 --2- 172.21.15.152:0/3382200365 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb62c071fe0 0x7fb62c0d0400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:04.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.996+0000 7fb62aa00640 1 -- 172.21.15.152:0/3382200365 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb62c145f60 msgr2=0x7fb62c0a9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:04.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.996+0000 7fb62be00640 1 --2- 172.21.15.152:0/3382200365 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb62c145f60 0x7fb62c0a9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:04.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.996+0000 7fb62aa00640 1 --2- 172.21.15.152:0/3382200365 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb62c145f60 0x7fb62c0a9760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:04.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.996+0000 7fb62aa00640 1 -- 172.21.15.152:0/3382200365 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb62c1326e0 con 0x7fb62c0cd0d0 2024-04-20T11:35:04.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.997+0000 7fb62aa00640 1 --2- 172.21.15.152:0/3382200365 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb62c0cd0d0 0x7fb62c131c20 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7fb618009860 tx=0x7fb6180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:04.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.997+0000 7fb623400640 1 -- 172.21.15.152:0/3382200365 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb618004230 con 0x7fb62c0cd0d0 2024-04-20T11:35:04.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.997+0000 7fb623400640 1 -- 172.21.15.152:0/3382200365 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6180043d0 con 0x7fb62c0cd0d0 2024-04-20T11:35:04.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.997+0000 7fb623400640 1 -- 172.21.15.152:0/3382200365 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb61803cad0 con 0x7fb62c0cd0d0 2024-04-20T11:35:04.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.997+0000 7fb632400640 1 -- 172.21.15.152:0/3382200365 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb62c1329c0 con 0x7fb62c0cd0d0 2024-04-20T11:35:04.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.997+0000 7fb632400640 1 -- 172.21.15.152:0/3382200365 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb62c130080 con 0x7fb62c0cd0d0 2024-04-20T11:35:04.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.998+0000 7fb632400640 1 -- 172.21.15.152:0/3382200365 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb62c0689f0 con 0x7fb62c0cd0d0 2024-04-20T11:35:04.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:04.999+0000 7fb623400640 1 -- 172.21.15.152:0/3382200365 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 21) ==== 91550+0+0 (secure 0 0 0) 0x7fb618038020 con 0x7fb62c0cd0d0 2024-04-20T11:35:05.000 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.000+0000 7fb623400640 1 --2- 172.21.15.152:0/3382200365 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb60c06ce90 0x7fb60c06f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:05.000 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.000+0000 7fb623400640 1 -- 172.21.15.152:0/3382200365 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5737+0+0 (secure 0 0 0) 0x7fb6180aeaa0 con 0x7fb62c0cd0d0 2024-04-20T11:35:05.001 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.001+0000 7fb62b400640 1 --2- 172.21.15.152:0/3382200365 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb60c06ce90 0x7fb60c06f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:05.002 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.002+0000 7fb62b400640 1 --2- 172.21.15.152:0/3382200365 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb60c06ce90 0x7fb60c06f350 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb61c000f60 tx=0x7fb61c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:05.004 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.004+0000 7fb623400640 1 -- 172.21.15.152:0/3382200365 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fb618041070 con 0x7fb62c0cd0d0 2024-04-20T11:35:05.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.238+0000 7fb632400640 1 -- 172.21.15.152:0/3382200365 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb62c0cf610 con 0x7fb62c0cd0d0 2024-04-20T11:35:05.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.240+0000 7fb623400640 1 -- 172.21.15.152:0/3382200365 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) ==== 94+0+5207 (secure 0 0 0) 0x7fb61807c460 con 0x7fb62c0cd0d0 2024-04-20T11:35:05.241 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:35:05.241 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":77,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5891,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/508638056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6833","nonce":508638056},{"type":"v1","addr":"172.21.15.175:6835","nonce":508638056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":6448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6834/3447077466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3447077466},{"type":"v1","addr":"172.21.15.175:6834","nonce":3447077466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":8409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6834/751442105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":751442105},{"type":"v1","addr":"172.21.15.152:6834","nonce":751442105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T11:34:36.050950+0000","modified":"2024-04-20T11:34:39.117003+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8412},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8412":{"gid":8412,"name":"c","rank":0,"incarnation":72,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6835/2477488828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6833","nonce":2477488828},{"type":"v1","addr":"172.21.15.152:6835","nonce":2477488828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8412,"qdb_cluster":[8412]},"id":9}]} 2024-04-20T11:35:05.241 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 77 2024-04-20T11:35:05.243 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.242+0000 7fb632400640 1 -- 172.21.15.152:0/3382200365 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb60c06ce90 msgr2=0x7fb60c06f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:05.243 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.242+0000 7fb632400640 1 --2- 172.21.15.152:0/3382200365 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb60c06ce90 0x7fb60c06f350 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb61c000f60 tx=0x7fb61c009040 comp rx=0 tx=0).stop 2024-04-20T11:35:05.243 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.242+0000 7fb632400640 1 -- 172.21.15.152:0/3382200365 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb62c0cd0d0 msgr2=0x7fb62c131c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:05.243 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.242+0000 7fb632400640 1 --2- 172.21.15.152:0/3382200365 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb62c0cd0d0 0x7fb62c131c20 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7fb618009860 tx=0x7fb6180046c0 comp rx=0 tx=0).stop 2024-04-20T11:35:05.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.243+0000 7fb632400640 1 -- 172.21.15.152:0/3382200365 shutdown_connections 2024-04-20T11:35:05.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.243+0000 7fb632400640 1 --2- 172.21.15.152:0/3382200365 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb60c06ce90 0x7fb60c06f350 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:05.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.243+0000 7fb632400640 1 --2- 172.21.15.152:0/3382200365 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb62c145f60 0x7fb62c0a9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:05.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.243+0000 7fb632400640 1 --2- 172.21.15.152:0/3382200365 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb62c0cd0d0 0x7fb62c131c20 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:05.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.243+0000 7fb632400640 1 --2- 172.21.15.152:0/3382200365 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb62c071fe0 0x7fb62c0d0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:05.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.243+0000 7fb632400640 1 -- 172.21.15.152:0/3382200365 >> 172.21.15.152:0/3382200365 conn(0x7fb62c067cf0 msgr2=0x7fb62c159ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:05.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.243+0000 7fb632400640 1 -- 172.21.15.152:0/3382200365 shutdown_connections 2024-04-20T11:35:05.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.243+0000 7fb632400640 1 -- 172.21.15.152:0/3382200365 wait complete. 2024-04-20T11:35:05.264 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:35:05.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.451+0000 7f56bec00640 1 Processor -- start 2024-04-20T11:35:05.452 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.451+0000 7f56bec00640 1 -- start start 2024-04-20T11:35:05.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.452+0000 7f56bec00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f56b806f510 0x7f56b8140dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:05.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.452+0000 7f56bec00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56b80cf0a0 con 0x7f56b813e530 2024-04-20T11:35:05.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.453+0000 7f56bec00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56b80d00e0 con 0x7f56b8141310 2024-04-20T11:35:05.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.453+0000 7f56bec00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f56b8141b00 con 0x7f56b806f510 2024-04-20T11:35:05.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.453+0000 7f56bd800640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f56b813e530 0x7f56b806ee00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:58772/0 (socket says 172.21.15.152:58772) 2024-04-20T11:35:05.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.453+0000 7f56bd800640 1 -- 172.21.15.152:0/2492434646 learned_addr learned my addr 172.21.15.152:0/2492434646 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:35:05.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.453+0000 7f56bce00640 1 --2- 172.21.15.152:0/2492434646 >> v2:172.21.15.175:3300/0 conn(0x7f56b806f510 0x7f56b8140dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:05.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.453+0000 7f56b7e00640 1 -- 172.21.15.152:0/2492434646 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2258692278 0 0) 0x7f56a8001110 con 0x7f56b813e530 2024-04-20T11:35:05.456 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.453+0000 7f56b7e00640 1 -- 172.21.15.152:0/2492434646 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f56a0003520 con 0x7f56b813e530 2024-04-20T11:35:05.456 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.454+0000 7f56b7e00640 1 -- 172.21.15.152:0/2492434646 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3560795563 0 0) 0x7f56a0003520 con 0x7f56b813e530 2024-04-20T11:35:05.456 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.454+0000 7f56b7e00640 1 -- 172.21.15.152:0/2492434646 >> v2:172.21.15.175:3300/0 conn(0x7f56b806f510 msgr2=0x7f56b8140dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:05.456 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.454+0000 7f56b7e00640 1 --2- 172.21.15.152:0/2492434646 >> v2:172.21.15.175:3300/0 conn(0x7f56b806f510 0x7f56b8140dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:05.456 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.454+0000 7f56b7e00640 1 -- 172.21.15.152:0/2492434646 >> v1:172.21.15.175:6789/0 conn(0x7f56b8141310 legacy=0x7f56b8071cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:05.456 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.454+0000 7f56b7e00640 1 -- 172.21.15.152:0/2492434646 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56b80cf320 con 0x7f56b813e530 2024-04-20T11:35:05.456 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.454+0000 7f56b7e00640 1 -- 172.21.15.152:0/2492434646 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f56a8001840 con 0x7f56b813e530 2024-04-20T11:35:05.456 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.454+0000 7f56b7e00640 1 -- 172.21.15.152:0/2492434646 >> v1:172.21.15.152:6789/0 conn(0x7f56b813e530 legacy=0x7f56b806ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:05.456 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.454+0000 7f56b7e00640 1 --2- 172.21.15.152:0/2492434646 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f56a0003a00 0x7f56a0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:05.456 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.454+0000 7f56b7e00640 1 --2- 172.21.15.152:0/2492434646 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f56a0004800 0x7f56a0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:05.456 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.454+0000 7f56b7e00640 1 --2- 172.21.15.152:0/2492434646 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f56a00051c0 0x7f56a000dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:05.456 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.454+0000 7f56b7e00640 1 -- 172.21.15.152:0/2492434646 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f56a000e2a0 con 0x7f56a00051c0 2024-04-20T11:35:05.456 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.454+0000 7f56b7e00640 1 -- 172.21.15.152:0/2492434646 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f56a000e450 con 0x7f56a0003a00 2024-04-20T11:35:05.456 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.454+0000 7f56b7e00640 1 -- 172.21.15.152:0/2492434646 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f56a000e650 con 0x7f56a0004800 2024-04-20T11:35:05.457 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.454+0000 7f56be200640 1 --2- 172.21.15.152:0/2492434646 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f56a00051c0 0x7f56a000dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:05.457 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.454+0000 7f56bd800640 1 --2- 172.21.15.152:0/2492434646 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f56a0003a00 0x7f56a0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:05.457 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.454+0000 7f56bce00640 1 --2- 172.21.15.152:0/2492434646 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f56a0004800 0x7f56a0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:05.457 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.455+0000 7f56be200640 1 -- 172.21.15.152:0/2492434646 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f56a0004800 msgr2=0x7f56a0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:05.457 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.455+0000 7f56be200640 1 --2- 172.21.15.152:0/2492434646 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f56a0004800 0x7f56a0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:05.457 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.455+0000 7f56be200640 1 -- 172.21.15.152:0/2492434646 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f56a0003a00 msgr2=0x7f56a0003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:05.457 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.455+0000 7f56be200640 1 --2- 172.21.15.152:0/2492434646 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f56a0003a00 0x7f56a0003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:05.457 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.455+0000 7f56be200640 1 -- 172.21.15.152:0/2492434646 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56a0003520 con 0x7f56a00051c0 2024-04-20T11:35:05.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.456+0000 7f56be200640 1 -- 172.21.15.152:0/2492434646 reap_dead start 2024-04-20T11:35:05.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.456+0000 7f56be200640 1 --2- 172.21.15.152:0/2492434646 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f56a00051c0 0x7f56a000dbd0 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f56b0001e50 tx=0x7f56a0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2541e3dd7cde028 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:05.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.456+0000 7f56b7e00640 1 -- 172.21.15.152:0/2492434646 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b0001a70 con 0x7f56a00051c0 2024-04-20T11:35:05.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.456+0000 7f56b7e00640 1 -- 172.21.15.152:0/2492434646 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56b0003b90 con 0x7f56a00051c0 2024-04-20T11:35:05.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.456+0000 7f56b7e00640 1 -- 172.21.15.152:0/2492434646 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b00050d0 con 0x7f56a00051c0 2024-04-20T11:35:05.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.457+0000 7f56bec00640 1 -- 172.21.15.152:0/2492434646 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f56a00051c0 msgr2=0x7f56a000dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:05.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.457+0000 7f56bec00640 1 --2- 172.21.15.152:0/2492434646 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f56a00051c0 0x7f56a000dbd0 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f56b0001e50 tx=0x7f56a0005830 comp rx=0 tx=0).stop 2024-04-20T11:35:05.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.457+0000 7f56bec00640 1 -- 172.21.15.152:0/2492434646 shutdown_connections 2024-04-20T11:35:05.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.457+0000 7f56bec00640 1 --2- 172.21.15.152:0/2492434646 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f56a00051c0 0x7f56a000dbd0 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:05.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.457+0000 7f56bec00640 1 -- 172.21.15.152:0/2492434646 >> 172.21.15.152:0/2492434646 conn(0x7f56b8067ae0 msgr2=0x7f56b806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:05.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.457+0000 7f56bec00640 1 -- 172.21.15.152:0/2492434646 shutdown_connections 2024-04-20T11:35:05.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.457+0000 7f56bec00640 1 -- 172.21.15.152:0/2492434646 wait complete. 2024-04-20T11:35:05.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.459+0000 7f56bec00640 1 Processor -- start 2024-04-20T11:35:05.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.459+0000 7f56bec00640 1 -- start start 2024-04-20T11:35:05.462 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.460+0000 7f56bec00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f56a00051c0 0x7f56b81f2f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:05.462 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.460+0000 7f56bec00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f56b8155ff0 0x7f56b8156470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:05.462 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.460+0000 7f56bec00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f56b8158170 0x7f56b81585f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:05.462 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.460+0000 7f56bec00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f56b80cf8e0 con 0x7f56a00051c0 2024-04-20T11:35:05.462 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.460+0000 7f56bec00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f56b80cfc50 con 0x7f56b8155ff0 2024-04-20T11:35:05.462 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.460+0000 7f56bec00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f56b81417a0 con 0x7f56b8158170 2024-04-20T11:35:05.462 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.460+0000 7f56bd800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f56a00051c0 0x7f56b81f2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:05.462 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.460+0000 7f56bd800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f56a00051c0 0x7f56b81f2f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:42904/0 (socket says 172.21.15.152:42904) 2024-04-20T11:35:05.462 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.460+0000 7f56bd800640 1 -- 172.21.15.152:0/3117408239 learned_addr learned my addr 172.21.15.152:0/3117408239 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:05.462 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.461+0000 7f56bd800640 1 -- 172.21.15.152:0/3117408239 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f56b8158170 msgr2=0x7f56b81585f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:05.462 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.461+0000 7f56be200640 1 --2- 172.21.15.152:0/3117408239 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f56b8158170 0x7f56b81585f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:05.462 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.461+0000 7f56bce00640 1 --2- 172.21.15.152:0/3117408239 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f56b8155ff0 0x7f56b8156470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:05.463 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.461+0000 7f56bd800640 1 --2- 172.21.15.152:0/3117408239 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f56b8158170 0x7f56b81585f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:05.463 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.461+0000 7f56bd800640 1 -- 172.21.15.152:0/3117408239 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f56b8155ff0 msgr2=0x7f56b8156470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:05.463 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.461+0000 7f56bd800640 1 --2- 172.21.15.152:0/3117408239 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f56b8155ff0 0x7f56b8156470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:05.463 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.461+0000 7f56bd800640 1 -- 172.21.15.152:0/3117408239 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56b813dda0 con 0x7f56a00051c0 2024-04-20T11:35:05.463 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.461+0000 7f56bd800640 1 --2- 172.21.15.152:0/3117408239 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f56a00051c0 0x7f56b81f2f30 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f56a8001050 tx=0x7f56a800eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:05.463 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.461+0000 7f56b5600640 1 -- 172.21.15.152:0/3117408239 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56a800c6c0 con 0x7f56a00051c0 2024-04-20T11:35:05.463 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.461+0000 7f56b5600640 1 -- 172.21.15.152:0/3117408239 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56a8018070 con 0x7f56a00051c0 2024-04-20T11:35:05.463 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.461+0000 7f56bec00640 1 -- 172.21.15.152:0/3117408239 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56b81f3470 con 0x7f56a00051c0 2024-04-20T11:35:05.463 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.461+0000 7f56b5600640 1 -- 172.21.15.152:0/3117408239 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56a800caa0 con 0x7f56a00051c0 2024-04-20T11:35:05.463 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.461+0000 7f56bec00640 1 -- 172.21.15.152:0/3117408239 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56b80b08b0 con 0x7f56a00051c0 2024-04-20T11:35:05.464 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.463+0000 7f56bec00640 1 -- 172.21.15.152:0/3117408239 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56b80b57e0 con 0x7f56a00051c0 2024-04-20T11:35:05.464 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.463+0000 7f56b5600640 1 -- 172.21.15.152:0/3117408239 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 21) ==== 91550+0+0 (secure 0 0 0) 0x7f56a800cc40 con 0x7f56a00051c0 2024-04-20T11:35:05.466 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.465+0000 7f56b5600640 1 --2- 172.21.15.152:0/3117408239 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f569c06ce90 0x7f569c06f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:05.466 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.466+0000 7f56b5600640 1 -- 172.21.15.152:0/3117408239 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5737+0+0 (secure 0 0 0) 0x7f56a808c440 con 0x7f56a00051c0 2024-04-20T11:35:05.466 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.466+0000 7f56bce00640 1 --2- 172.21.15.152:0/3117408239 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f569c06ce90 0x7f569c06f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:05.468 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.467+0000 7f56bce00640 1 --2- 172.21.15.152:0/3117408239 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f569c06ce90 0x7f569c06f350 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f56b80cd980 tx=0x7f56ac008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:05.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.470+0000 7f56b5600640 1 -- 172.21.15.152:0/3117408239 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f56b80b57e0 con 0x7f56a00051c0 2024-04-20T11:35:05.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.707+0000 7f56bec00640 1 -- 172.21.15.152:0/3117408239 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f56b8075e60 con 0x7f56a00051c0 2024-04-20T11:35:05.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.708+0000 7f56b5600640 1 -- 172.21.15.152:0/3117408239 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) ==== 94+0+5207 (secure 0 0 0) 0x7f56a805ee20 con 0x7f56a00051c0 2024-04-20T11:35:05.709 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:35:05.709 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":77,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5891,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/508638056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6833","nonce":508638056},{"type":"v1","addr":"172.21.15.175:6835","nonce":508638056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":6448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6834/3447077466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3447077466},{"type":"v1","addr":"172.21.15.175:6834","nonce":3447077466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":8409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6834/751442105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":751442105},{"type":"v1","addr":"172.21.15.152:6834","nonce":751442105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T11:34:36.050950+0000","modified":"2024-04-20T11:34:39.117003+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8412},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8412":{"gid":8412,"name":"c","rank":0,"incarnation":72,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6835/2477488828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6833","nonce":2477488828},{"type":"v1","addr":"172.21.15.152:6835","nonce":2477488828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8412,"qdb_cluster":[8412]},"id":9}]} 2024-04-20T11:35:05.710 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 77 2024-04-20T11:35:05.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.711+0000 7f56bec00640 1 -- 172.21.15.152:0/3117408239 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f569c06ce90 msgr2=0x7f569c06f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:05.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.711+0000 7f56bec00640 1 --2- 172.21.15.152:0/3117408239 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f569c06ce90 0x7f569c06f350 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f56b80cd980 tx=0x7f56ac008040 comp rx=0 tx=0).stop 2024-04-20T11:35:05.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.711+0000 7f56bec00640 1 -- 172.21.15.152:0/3117408239 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f56a00051c0 msgr2=0x7f56b81f2f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:05.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.711+0000 7f56bec00640 1 --2- 172.21.15.152:0/3117408239 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f56a00051c0 0x7f56b81f2f30 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f56a8001050 tx=0x7f56a800eb90 comp rx=0 tx=0).stop 2024-04-20T11:35:05.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.711+0000 7f56bec00640 1 -- 172.21.15.152:0/3117408239 shutdown_connections 2024-04-20T11:35:05.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.711+0000 7f56bec00640 1 --2- 172.21.15.152:0/3117408239 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f569c06ce90 0x7f569c06f350 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:05.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.711+0000 7f56bec00640 1 --2- 172.21.15.152:0/3117408239 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f56b8158170 0x7f56b81585f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:05.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.711+0000 7f56bec00640 1 --2- 172.21.15.152:0/3117408239 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f56b8155ff0 0x7f56b8156470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:05.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.711+0000 7f56bec00640 1 --2- 172.21.15.152:0/3117408239 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f56a00051c0 0x7f56b81f2f30 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:05.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.711+0000 7f56bec00640 1 -- 172.21.15.152:0/3117408239 >> 172.21.15.152:0/3117408239 conn(0x7f56b8067ae0 msgr2=0x7f56b813fec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:05.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.711+0000 7f56bec00640 1 -- 172.21.15.152:0/3117408239 shutdown_connections 2024-04-20T11:35:05.713 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.711+0000 7f56bec00640 1 -- 172.21.15.152:0/3117408239 wait complete. 2024-04-20T11:35:05.730 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:35:05.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.927+0000 7f3731800640 1 Processor -- start 2024-04-20T11:35:05.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.928+0000 7f3731800640 1 -- start start 2024-04-20T11:35:05.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.929+0000 7f3731800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f372c1603d0 0x7f372c0b5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:05.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.929+0000 7f3731800640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f372c0b5480 0x7f372c0b63f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:05.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.929+0000 7f3731800640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f372c0d3350 con 0x7f372c15fcc0 2024-04-20T11:35:05.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.929+0000 7f3731800640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f372c0d3630 con 0x7f372c0b5480 2024-04-20T11:35:05.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.929+0000 7f3731800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f372c0d3910 con 0x7f372c1603d0 2024-04-20T11:35:05.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.929+0000 7f372be00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f372c15fcc0 0x7f372c0d4e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:54330/0 (socket says 172.21.15.152:54330) 2024-04-20T11:35:05.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.929+0000 7f372be00640 1 -- 172.21.15.152:0/4292100196 learned_addr learned my addr 172.21.15.152:0/4292100196 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:35:05.932 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.930+0000 7f3730e00640 1 --2- 172.21.15.152:0/4292100196 >> v2:172.21.15.175:3300/0 conn(0x7f372c0b5480 0x7f372c0b63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:05.932 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.930+0000 7f372b400640 1 --2- 172.21.15.152:0/4292100196 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f372c1603d0 0x7f372c0b5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:05.932 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.930+0000 7f372aa00640 1 -- 172.21.15.152:0/4292100196 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2423053800 0 0) 0x7f372c0d3350 con 0x7f372c15fcc0 2024-04-20T11:35:05.932 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.930+0000 7f372aa00640 1 -- 172.21.15.152:0/4292100196 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3710003520 con 0x7f372c15fcc0 2024-04-20T11:35:05.932 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.930+0000 7f372aa00640 1 -- 172.21.15.152:0/4292100196 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1030201820 0 0) 0x7f3710003520 con 0x7f372c15fcc0 2024-04-20T11:35:05.932 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.930+0000 7f372aa00640 1 -- 172.21.15.152:0/4292100196 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f372c1603d0 msgr2=0x7f372c0b5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:05.932 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.930+0000 7f372aa00640 1 --2- 172.21.15.152:0/4292100196 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f372c1603d0 0x7f372c0b5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:05.932 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.930+0000 7f372aa00640 1 -- 172.21.15.152:0/4292100196 >> v2:172.21.15.175:3300/0 conn(0x7f372c0b5480 msgr2=0x7f372c0b63f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:05.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.930+0000 7f372aa00640 1 --2- 172.21.15.152:0/4292100196 >> v2:172.21.15.175:3300/0 conn(0x7f372c0b5480 0x7f372c0b63f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:05.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.930+0000 7f372aa00640 1 -- 172.21.15.152:0/4292100196 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f372c0ced20 con 0x7f372c15fcc0 2024-04-20T11:35:05.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.930+0000 7f372aa00640 1 -- 172.21.15.152:0/4292100196 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f371c0019f0 con 0x7f372c15fcc0 2024-04-20T11:35:05.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.930+0000 7f372aa00640 1 -- 172.21.15.152:0/4292100196 >> v1:172.21.15.152:6789/0 conn(0x7f372c15fcc0 legacy=0x7f372c0d4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:05.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.931+0000 7f372aa00640 1 --2- 172.21.15.152:0/4292100196 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f37100039b0 0x7f3710003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:05.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.931+0000 7f372aa00640 1 --2- 172.21.15.152:0/4292100196 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f372c1603d0 0x7f3710004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:05.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.931+0000 7f372aa00640 1 --2- 172.21.15.152:0/4292100196 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3710004d40 0x7f37100051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:05.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.931+0000 7f372aa00640 1 -- 172.21.15.152:0/4292100196 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f372c0d3630 con 0x7f3710004d40 2024-04-20T11:35:05.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.931+0000 7f372aa00640 1 -- 172.21.15.152:0/4292100196 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f372c0d3910 con 0x7f37100039b0 2024-04-20T11:35:05.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.931+0000 7f372aa00640 1 -- 172.21.15.152:0/4292100196 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f371000b9c0 con 0x7f372c1603d0 2024-04-20T11:35:05.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.931+0000 7f3730e00640 1 --2- 172.21.15.152:0/4292100196 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3710004d40 0x7f37100051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:05.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.931+0000 7f3730e00640 1 -- 172.21.15.152:0/4292100196 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f372c1603d0 msgr2=0x7f3710004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:05.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.931+0000 7f372be00640 1 --2- 172.21.15.152:0/4292100196 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f37100039b0 0x7f3710003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:05.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.931+0000 7f3730e00640 1 --2- 172.21.15.152:0/4292100196 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f372c1603d0 0x7f3710004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:05.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.931+0000 7f3730e00640 1 -- 172.21.15.152:0/4292100196 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f37100039b0 msgr2=0x7f3710003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:05.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.931+0000 7f3730e00640 1 --2- 172.21.15.152:0/4292100196 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f37100039b0 0x7f3710003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:05.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.931+0000 7f3730e00640 1 -- 172.21.15.152:0/4292100196 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3710003520 con 0x7f3710004d40 2024-04-20T11:35:05.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.932+0000 7f3730e00640 1 --2- 172.21.15.152:0/4292100196 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3710004d40 0x7f37100051c0 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f3724009ac0 tx=0x7f3710005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56e53703e96a52e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:05.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.933+0000 7f372aa00640 1 -- 172.21.15.152:0/4292100196 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3724039070 con 0x7f3710004d40 2024-04-20T11:35:05.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.933+0000 7f372aa00640 1 -- 172.21.15.152:0/4292100196 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f372402c710 con 0x7f3710004d40 2024-04-20T11:35:05.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.933+0000 7f372aa00640 1 -- 172.21.15.152:0/4292100196 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3724009690 con 0x7f3710004d40 2024-04-20T11:35:05.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.933+0000 7f3731800640 1 -- 172.21.15.152:0/4292100196 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3710004d40 msgr2=0x7f37100051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:05.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.933+0000 7f3731800640 1 --2- 172.21.15.152:0/4292100196 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3710004d40 0x7f37100051c0 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f3724009ac0 tx=0x7f3710005830 comp rx=0 tx=0).stop 2024-04-20T11:35:05.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.933+0000 7f3730e00640 1 -- 172.21.15.152:0/4292100196 reap_dead start 2024-04-20T11:35:05.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.933+0000 7f3731800640 1 -- 172.21.15.152:0/4292100196 shutdown_connections 2024-04-20T11:35:05.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.933+0000 7f3731800640 1 -- 172.21.15.152:0/4292100196 >> 172.21.15.152:0/4292100196 conn(0x7f372c08bfd0 msgr2=0x7f372c0d4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:05.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.934+0000 7f3731800640 1 -- 172.21.15.152:0/4292100196 shutdown_connections 2024-04-20T11:35:05.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.934+0000 7f3731800640 1 -- 172.21.15.152:0/4292100196 wait complete. 2024-04-20T11:35:05.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.935+0000 7f3731800640 1 Processor -- start 2024-04-20T11:35:05.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.935+0000 7f3731800640 1 -- start start 2024-04-20T11:35:05.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.936+0000 7f3731800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3710004d40 0x7f372c14ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:05.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.936+0000 7f3731800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f372c15e750 0x7f372c0a1b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:05.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.936+0000 7f3731800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f372c14e580 0x7f372c06dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:05.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.936+0000 7f3731800640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f372c0b6240 con 0x7f372c14e580 2024-04-20T11:35:05.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.936+0000 7f3731800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f372c0b60c0 con 0x7f372c15e750 2024-04-20T11:35:05.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.936+0000 7f3731800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f372c0cf240 con 0x7f3710004d40 2024-04-20T11:35:05.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.937+0000 7f3730e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f372c14e580 0x7f372c06dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:05.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.937+0000 7f3730e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f372c14e580 0x7f372c06dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:56708/0 (socket says 172.21.15.152:56708) 2024-04-20T11:35:05.938 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.937+0000 7f3730e00640 1 -- 172.21.15.152:0/555576426 learned_addr learned my addr 172.21.15.152:0/555576426 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:05.939 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.937+0000 7f372b400640 1 --2- 172.21.15.152:0/555576426 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f372c15e750 0x7f372c0a1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:05.939 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.937+0000 7f3730e00640 1 -- 172.21.15.152:0/555576426 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3710004d40 msgr2=0x7f372c14ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:05.939 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.937+0000 7f3730e00640 1 --2- 172.21.15.152:0/555576426 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3710004d40 0x7f372c14ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:05.939 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.937+0000 7f3730e00640 1 -- 172.21.15.152:0/555576426 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f372c15e750 msgr2=0x7f372c0a1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:05.939 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.937+0000 7f3730e00640 1 --2- 172.21.15.152:0/555576426 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f372c15e750 0x7f372c0a1b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:05.939 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.937+0000 7f3730e00640 1 -- 172.21.15.152:0/555576426 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f372c14ef20 con 0x7f372c14e580 2024-04-20T11:35:05.939 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.937+0000 7f3730e00640 1 --2- 172.21.15.152:0/555576426 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f372c14e580 0x7f372c06dcc0 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f3724009be0 tx=0x7f3724004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:05.939 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.938+0000 7f3723e00640 1 -- 172.21.15.152:0/555576426 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3724039070 con 0x7f372c14e580 2024-04-20T11:35:05.939 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.938+0000 7f3723e00640 1 -- 172.21.15.152:0/555576426 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3724043070 con 0x7f372c14e580 2024-04-20T11:35:05.939 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.938+0000 7f3723e00640 1 -- 172.21.15.152:0/555576426 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3724003a50 con 0x7f372c14e580 2024-04-20T11:35:05.939 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.938+0000 7f3731800640 1 -- 172.21.15.152:0/555576426 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f372c14f200 con 0x7f372c14e580 2024-04-20T11:35:05.939 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.938+0000 7f3731800640 1 -- 172.21.15.152:0/555576426 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f372c14f7c0 con 0x7f372c14e580 2024-04-20T11:35:05.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.939+0000 7f3731800640 1 -- 172.21.15.152:0/555576426 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f372c075e60 con 0x7f372c14e580 2024-04-20T11:35:05.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.940+0000 7f3723e00640 1 -- 172.21.15.152:0/555576426 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 21) ==== 91550+0+0 (secure 0 0 0) 0x7f3724003bf0 con 0x7f372c14e580 2024-04-20T11:35:05.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.941+0000 7f3723e00640 1 --2- 172.21.15.152:0/555576426 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f370806cec0 0x7f370806f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:05.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.941+0000 7f3723e00640 1 -- 172.21.15.152:0/555576426 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5737+0+0 (secure 0 0 0) 0x7f37240adc40 con 0x7f372c14e580 2024-04-20T11:35:05.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.942+0000 7f372be00640 1 --2- 172.21.15.152:0/555576426 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f370806cec0 0x7f370806f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:05.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.943+0000 7f372be00640 1 --2- 172.21.15.152:0/555576426 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f370806cec0 0x7f370806f380 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f371c007c40 tx=0x7f371c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:05.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:05.945+0000 7f3723e00640 1 -- 172.21.15.152:0/555576426 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f372404d020 con 0x7f372c14e580 2024-04-20T11:35:06.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.147+0000 7f3731800640 1 -- 172.21.15.152:0/555576426 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f372c0b7e60 con 0x7f372c14e580 2024-04-20T11:35:06.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.149+0000 7f3723e00640 1 -- 172.21.15.152:0/555576426 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v120) ==== 74+0+16806 (secure 0 0 0) 0x7f37240044a0 con 0x7f372c14e580 2024-04-20T11:35:06.150 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:35:06.150 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":120,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:35:04.078143+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:34:33.781054+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-04-20T11:34:35.031728+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":119,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":19,"snaps":[{"begin":2,"length":4}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:35:06.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.152+0000 7f3731800640 1 -- 172.21.15.152:0/555576426 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f370806cec0 msgr2=0x7f370806f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:06.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.152+0000 7f3731800640 1 --2- 172.21.15.152:0/555576426 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f370806cec0 0x7f370806f380 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f371c007c40 tx=0x7f371c009040 comp rx=0 tx=0).stop 2024-04-20T11:35:06.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.152+0000 7f3731800640 1 -- 172.21.15.152:0/555576426 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f372c14e580 msgr2=0x7f372c06dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:06.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.152+0000 7f3731800640 1 --2- 172.21.15.152:0/555576426 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f372c14e580 0x7f372c06dcc0 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f3724009be0 tx=0x7f3724004730 comp rx=0 tx=0).stop 2024-04-20T11:35:06.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.152+0000 7f3731800640 1 -- 172.21.15.152:0/555576426 shutdown_connections 2024-04-20T11:35:06.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.152+0000 7f3731800640 1 --2- 172.21.15.152:0/555576426 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f370806cec0 0x7f370806f380 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:06.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.152+0000 7f3731800640 1 --2- 172.21.15.152:0/555576426 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f372c14e580 0x7f372c06dcc0 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:06.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.152+0000 7f3731800640 1 --2- 172.21.15.152:0/555576426 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f372c15e750 0x7f372c0a1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:06.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.152+0000 7f3731800640 1 --2- 172.21.15.152:0/555576426 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3710004d40 0x7f372c14ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:06.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.152+0000 7f3731800640 1 -- 172.21.15.152:0/555576426 >> 172.21.15.152:0/555576426 conn(0x7f372c08bfd0 msgr2=0x7f372c06cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:06.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.152+0000 7f3731800640 1 -- 172.21.15.152:0/555576426 shutdown_connections 2024-04-20T11:35:06.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.152+0000 7f3731800640 1 -- 172.21.15.152:0/555576426 wait complete. 2024-04-20T11:35:06.171 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-20T11:35:06.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.372+0000 7f8558e00640 1 Processor -- start 2024-04-20T11:35:06.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.373+0000 7f8558e00640 1 -- start start 2024-04-20T11:35:06.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.374+0000 7f8558e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f8554161d30 0x7f85540d0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:06.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.374+0000 7f8558e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f85540d4ec0 0x7f85540d15e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:06.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.374+0000 7f8558e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f85540d2b80 0x7f85540d3010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:06.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.374+0000 7f8558e00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f85540d0540 con 0x7f85540d2b80 2024-04-20T11:35:06.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.374+0000 7f8558e00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f85540cfb70 con 0x7f8554161d30 2024-04-20T11:35:06.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.374+0000 7f8558e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f85540d1b20 con 0x7f85540d4ec0 2024-04-20T11:35:06.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.374+0000 7f8553e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f85540d2b80 0x7f85540d3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:06.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.374+0000 7f8553e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f85540d2b80 0x7f85540d3010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:56710/0 (socket says 172.21.15.152:56710) 2024-04-20T11:35:06.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.374+0000 7f8553e00640 1 -- 172.21.15.152:0/3111973059 learned_addr learned my addr 172.21.15.152:0/3111973059 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:06.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.375+0000 7f8553400640 1 --2- 172.21.15.152:0/3111973059 >> v2:172.21.15.175:3300/0 conn(0x7f8554161d30 0x7f85540d0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:06.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.375+0000 7f8552a00640 1 --2- 172.21.15.152:0/3111973059 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f85540d4ec0 0x7f85540d15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:06.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.375+0000 7f8553e00640 1 -- 172.21.15.152:0/3111973059 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f85540d4ec0 msgr2=0x7f85540d15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:06.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.375+0000 7f8553e00640 1 --2- 172.21.15.152:0/3111973059 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f85540d4ec0 0x7f85540d15e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:06.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.375+0000 7f8553e00640 1 -- 172.21.15.152:0/3111973059 >> v2:172.21.15.175:3300/0 conn(0x7f8554161d30 msgr2=0x7f85540d0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:06.377 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.375+0000 7f8553e00640 1 --2- 172.21.15.152:0/3111973059 >> v2:172.21.15.175:3300/0 conn(0x7f8554161d30 0x7f85540d0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:06.377 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.375+0000 7f8553e00640 1 -- 172.21.15.152:0/3111973059 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85540d26a0 con 0x7f85540d2b80 2024-04-20T11:35:06.377 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.375+0000 7f8553400640 1 --2- 172.21.15.152:0/3111973059 >> v2:172.21.15.175:3300/0 conn(0x7f8554161d30 0x7f85540d0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:35:06.377 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.375+0000 7f8552a00640 1 --2- 172.21.15.152:0/3111973059 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f85540d4ec0 0x7f85540d15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:35:06.378 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.376+0000 7f8553e00640 1 --2- 172.21.15.152:0/3111973059 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f85540d2b80 0x7f85540d3010 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f855408ced0 tx=0x7f854c02b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1326d7a06991cb31 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:06.378 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.376+0000 7f8552000640 1 -- 172.21.15.152:0/3111973059 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f854c035020 con 0x7f85540d2b80 2024-04-20T11:35:06.378 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.376+0000 7f8552000640 1 -- 172.21.15.152:0/3111973059 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f854c031880 con 0x7f85540d2b80 2024-04-20T11:35:06.378 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.376+0000 7f8552000640 1 -- 172.21.15.152:0/3111973059 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f854c031c50 con 0x7f85540d2b80 2024-04-20T11:35:06.378 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.377+0000 7f8558e00640 1 -- 172.21.15.152:0/3111973059 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f85540d2b80 msgr2=0x7f85540d3010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:06.378 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.377+0000 7f8558e00640 1 --2- 172.21.15.152:0/3111973059 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f85540d2b80 0x7f85540d3010 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f855408ced0 tx=0x7f854c02b920 comp rx=0 tx=0).stop 2024-04-20T11:35:06.378 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.377+0000 7f8558e00640 1 -- 172.21.15.152:0/3111973059 shutdown_connections 2024-04-20T11:35:06.378 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.377+0000 7f8558e00640 1 --2- 172.21.15.152:0/3111973059 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f85540d2b80 0x7f85540d3010 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:06.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.377+0000 7f8558e00640 1 --2- 172.21.15.152:0/3111973059 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f85540d4ec0 0x7f85540d15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:06.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.377+0000 7f8558e00640 1 --2- 172.21.15.152:0/3111973059 >> v2:172.21.15.175:3300/0 conn(0x7f8554161d30 0x7f85540d0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:06.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.377+0000 7f8558e00640 1 -- 172.21.15.152:0/3111973059 >> 172.21.15.152:0/3111973059 conn(0x7f855408bfd0 msgr2=0x7f85540d45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:06.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.377+0000 7f8558e00640 1 -- 172.21.15.152:0/3111973059 shutdown_connections 2024-04-20T11:35:06.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.377+0000 7f8558e00640 1 -- 172.21.15.152:0/3111973059 wait complete. 2024-04-20T11:35:06.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.379+0000 7f8558e00640 1 Processor -- start 2024-04-20T11:35:06.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.379+0000 7f8558e00640 1 -- start start 2024-04-20T11:35:06.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.380+0000 7f8558e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f85540d2b80 0x7f855415a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:06.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.380+0000 7f8558e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f85540d4ec0 0x7f85541f5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:06.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.380+0000 7f8558e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8554161d30 0x7f855415cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:06.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.380+0000 7f8558e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f85540cf800 con 0x7f85540d4ec0 2024-04-20T11:35:06.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.380+0000 7f8558e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f85540cf520 con 0x7f8554161d30 2024-04-20T11:35:06.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.380+0000 7f8558e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f85540cf1b0 con 0x7f85540d2b80 2024-04-20T11:35:06.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.380+0000 7f8552a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f85540d4ec0 0x7f85541f5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:06.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.380+0000 7f8552a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f85540d4ec0 0x7f85541f5330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:56726/0 (socket says 172.21.15.152:56726) 2024-04-20T11:35:06.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.380+0000 7f8552a00640 1 -- 172.21.15.152:0/1328692998 learned_addr learned my addr 172.21.15.152:0/1328692998 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:06.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.381+0000 7f8553e00640 1 --2- 172.21.15.152:0/1328692998 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8554161d30 0x7f855415cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:06.383 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.381+0000 7f8552a00640 1 -- 172.21.15.152:0/1328692998 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f85540d2b80 msgr2=0x7f855415a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:06.383 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.381+0000 7f8553400640 1 --2- 172.21.15.152:0/1328692998 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f85540d2b80 0x7f855415a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:06.383 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.381+0000 7f8552a00640 1 --2- 172.21.15.152:0/1328692998 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f85540d2b80 0x7f855415a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:06.383 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.381+0000 7f8552a00640 1 -- 172.21.15.152:0/1328692998 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8554161d30 msgr2=0x7f855415cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:06.383 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.381+0000 7f8552a00640 1 --2- 172.21.15.152:0/1328692998 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8554161d30 0x7f855415cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:06.383 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.381+0000 7f8552a00640 1 -- 172.21.15.152:0/1328692998 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8554157b90 con 0x7f85540d4ec0 2024-04-20T11:35:06.383 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.381+0000 7f8553e00640 1 --2- 172.21.15.152:0/1328692998 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8554161d30 0x7f855415cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:35:06.383 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.381+0000 7f8552a00640 1 --2- 172.21.15.152:0/1328692998 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f85540d4ec0 0x7f85541f5330 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f854400a390 tx=0x7f854400a850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:06.383 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.381+0000 7f854b400640 1 -- 172.21.15.152:0/1328692998 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8544019070 con 0x7f85540d4ec0 2024-04-20T11:35:06.383 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.381+0000 7f854b400640 1 -- 172.21.15.152:0/1328692998 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8544008580 con 0x7f85540d4ec0 2024-04-20T11:35:06.383 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.381+0000 7f854b400640 1 -- 172.21.15.152:0/1328692998 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8544008950 con 0x7f85540d4ec0 2024-04-20T11:35:06.383 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.381+0000 7f8558e00640 1 -- 172.21.15.152:0/1328692998 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8554157e70 con 0x7f85540d4ec0 2024-04-20T11:35:06.383 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.382+0000 7f8558e00640 1 -- 172.21.15.152:0/1328692998 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8554158430 con 0x7f85540d4ec0 2024-04-20T11:35:06.384 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.383+0000 7f8558e00640 1 -- 172.21.15.152:0/1328692998 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8554075e60 con 0x7f85540d4ec0 2024-04-20T11:35:06.384 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.384+0000 7f854b400640 1 -- 172.21.15.152:0/1328692998 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 21) ==== 91550+0+0 (secure 0 0 0) 0x7f854401c070 con 0x7f85540d4ec0 2024-04-20T11:35:06.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.384+0000 7f854b400640 1 --2- 172.21.15.152:0/1328692998 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f853006ce90 0x7f853006f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:06.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.385+0000 7f854b400640 1 -- 172.21.15.152:0/1328692998 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5737+0+0 (secure 0 0 0) 0x7f854408c9f0 con 0x7f85540d4ec0 2024-04-20T11:35:06.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.385+0000 7f8553400640 1 --2- 172.21.15.152:0/1328692998 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f853006ce90 0x7f853006f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:06.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.386+0000 7f8553400640 1 --2- 172.21.15.152:0/1328692998 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f853006ce90 0x7f853006f350 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f85541f76c0 tx=0x7f853c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:06.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.389+0000 7f854b400640 1 -- 172.21.15.152:0/1328692998 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f8544059df0 con 0x7f85540d4ec0 2024-04-20T11:35:06.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.629+0000 7f8558e00640 1 -- 172.21.15.152:0/1328692998 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f85540d1d70 con 0x7f85540d4ec0 2024-04-20T11:35:06.631 INFO:tasks.ceph.mon.a.smithi152.stderr:2024-04-20T11:35:06.631+0000 7f8229800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-20T11:35:06.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.634+0000 7f854b400640 1 -- 172.21.15.152:0/1328692998 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v78) ==== 161+0+0 (secure 0 0 0) 0x7f854405e0f0 con 0x7f85540d4ec0 2024-04-20T11:35:06.635 INFO:teuthology.orchestra.run.smithi152.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-20T11:35:06.638 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.637+0000 7f8558e00640 1 -- 172.21.15.152:0/1328692998 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f853006ce90 msgr2=0x7f853006f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:06.638 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.637+0000 7f8558e00640 1 --2- 172.21.15.152:0/1328692998 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f853006ce90 0x7f853006f350 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f85541f76c0 tx=0x7f853c008040 comp rx=0 tx=0).stop 2024-04-20T11:35:06.638 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.637+0000 7f8558e00640 1 -- 172.21.15.152:0/1328692998 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f85540d4ec0 msgr2=0x7f85541f5330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:06.638 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.637+0000 7f8558e00640 1 --2- 172.21.15.152:0/1328692998 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f85540d4ec0 0x7f85541f5330 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f854400a390 tx=0x7f854400a850 comp rx=0 tx=0).stop 2024-04-20T11:35:06.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.638+0000 7f8558e00640 1 -- 172.21.15.152:0/1328692998 shutdown_connections 2024-04-20T11:35:06.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.638+0000 7f8558e00640 1 --2- 172.21.15.152:0/1328692998 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f853006ce90 0x7f853006f350 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:06.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.638+0000 7f8558e00640 1 --2- 172.21.15.152:0/1328692998 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8554161d30 0x7f855415cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:06.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.638+0000 7f8558e00640 1 --2- 172.21.15.152:0/1328692998 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f85540d4ec0 0x7f85541f5330 unknown :-1 s=CLOSED pgs=1696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:06.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.638+0000 7f8558e00640 1 --2- 172.21.15.152:0/1328692998 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f85540d2b80 0x7f855415a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:06.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.638+0000 7f8558e00640 1 -- 172.21.15.152:0/1328692998 >> 172.21.15.152:0/1328692998 conn(0x7f855408bfd0 msgr2=0x7f85540cb7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:06.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.638+0000 7f8558e00640 1 -- 172.21.15.152:0/1328692998 shutdown_connections 2024-04-20T11:35:06.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.638+0000 7f8558e00640 1 -- 172.21.15.152:0/1328692998 wait complete. 2024-04-20T11:35:06.660 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-20T11:35:06.725 INFO:tasks.ceph.mds.c.smithi152.stdout:starting mds.c at 2024-04-20T11:35:06.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.836+0000 7fca21600640 1 Processor -- start 2024-04-20T11:35:06.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.836+0000 7fca21600640 1 -- start start 2024-04-20T11:35:06.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.837+0000 7fca21600640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fca1c15eb50 0x7fca1c0d0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:06.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.837+0000 7fca21600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fca1c163230 0x7fca1c0d2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:06.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.837+0000 7fca21600640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca1c0d19c0 con 0x7fca1c0d1440 2024-04-20T11:35:06.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.837+0000 7fca21600640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fca1c162750 con 0x7fca1c15eb50 2024-04-20T11:35:06.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.837+0000 7fca21600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fca1c162a30 con 0x7fca1c163230 2024-04-20T11:35:06.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.838+0000 7fca1be00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fca1c15eb50 0x7fca1c0d0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:06.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.838+0000 7fca1be00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fca1c15eb50 0x7fca1c0d0f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:56756/0 (socket says 172.21.15.152:56756) 2024-04-20T11:35:06.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.838+0000 7fca1be00640 1 -- 172.21.15.152:0/3370007176 learned_addr learned my addr 172.21.15.152:0/3370007176 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:06.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.838+0000 7fca1be00640 1 -- 172.21.15.152:0/3370007176 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fca1c163230 msgr2=0x7fca1c0d2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:06.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.838+0000 7fca1b400640 1 --2- 172.21.15.152:0/3370007176 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fca1c163230 0x7fca1c0d2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:06.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.838+0000 7fca1be00640 1 --2- 172.21.15.152:0/3370007176 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fca1c163230 0x7fca1c0d2e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:06.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.838+0000 7fca1be00640 1 -- 172.21.15.152:0/3370007176 >> v1:172.21.15.175:6789/0 conn(0x7fca1c0d1440 legacy=0x7fca1c15ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:06.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.838+0000 7fca1be00640 1 -- 172.21.15.152:0/3370007176 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca1c162d10 con 0x7fca1c15eb50 2024-04-20T11:35:06.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.839+0000 7fca1be00640 1 --2- 172.21.15.152:0/3370007176 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fca1c15eb50 0x7fca1c0d0f00 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7fca14009e50 tx=0x7fca1402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45f35683bd9d9adc server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:06.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.840+0000 7fca1aa00640 1 -- 172.21.15.152:0/3370007176 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca1402c9e0 con 0x7fca1c15eb50 2024-04-20T11:35:06.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.840+0000 7fca1aa00640 1 -- 172.21.15.152:0/3370007176 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca1402cb80 con 0x7fca1c15eb50 2024-04-20T11:35:06.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.840+0000 7fca1aa00640 1 -- 172.21.15.152:0/3370007176 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca140343f0 con 0x7fca1c15eb50 2024-04-20T11:35:06.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.840+0000 7fca21600640 1 -- 172.21.15.152:0/3370007176 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fca1c15eb50 msgr2=0x7fca1c0d0f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:06.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.840+0000 7fca21600640 1 --2- 172.21.15.152:0/3370007176 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fca1c15eb50 0x7fca1c0d0f00 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7fca14009e50 tx=0x7fca1402be40 comp rx=0 tx=0).stop 2024-04-20T11:35:06.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.840+0000 7fca21600640 1 -- 172.21.15.152:0/3370007176 shutdown_connections 2024-04-20T11:35:06.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.840+0000 7fca21600640 1 --2- 172.21.15.152:0/3370007176 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fca1c163230 0x7fca1c0d2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:06.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.840+0000 7fca21600640 1 --2- 172.21.15.152:0/3370007176 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fca1c15eb50 0x7fca1c0d0f00 unknown :-1 s=CLOSED pgs=1700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:06.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.840+0000 7fca21600640 1 -- 172.21.15.152:0/3370007176 >> 172.21.15.152:0/3370007176 conn(0x7fca1c08bfd0 msgr2=0x7fca1c0d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:06.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.840+0000 7fca21600640 1 -- 172.21.15.152:0/3370007176 shutdown_connections 2024-04-20T11:35:06.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.840+0000 7fca21600640 1 -- 172.21.15.152:0/3370007176 wait complete. 2024-04-20T11:35:06.843 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.842+0000 7fca21600640 1 Processor -- start 2024-04-20T11:35:06.843 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.842+0000 7fca21600640 1 -- start start 2024-04-20T11:35:06.844 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.843+0000 7fca21600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca1c0d1440 0x7fca1c0b5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:06.844 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.843+0000 7fca21600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fca1c15eb50 0x7fca1c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:06.845 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.843+0000 7fca21600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fca1c163230 0x7fca1c0b6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:06.845 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.843+0000 7fca21600640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fca1c0d3550 con 0x7fca1c163230 2024-04-20T11:35:06.845 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.843+0000 7fca21600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fca1c0d33d0 con 0x7fca1c0d1440 2024-04-20T11:35:06.845 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.843+0000 7fca21600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fca1c0d36d0 con 0x7fca1c15eb50 2024-04-20T11:35:06.845 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.843+0000 7fca20c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fca1c163230 0x7fca1c0b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:06.845 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.843+0000 7fca20c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fca1c163230 0x7fca1c0b6390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:56762/0 (socket says 172.21.15.152:56762) 2024-04-20T11:35:06.845 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.843+0000 7fca20c00640 1 -- 172.21.15.152:0/2049919279 learned_addr learned my addr 172.21.15.152:0/2049919279 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:06.845 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.844+0000 7fca1b400640 1 --2- 172.21.15.152:0/2049919279 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fca1c15eb50 0x7fca1c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:06.845 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.844+0000 7fca1be00640 1 --2- 172.21.15.152:0/2049919279 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca1c0d1440 0x7fca1c0b5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:06.845 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.844+0000 7fca20c00640 1 -- 172.21.15.152:0/2049919279 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fca1c15eb50 msgr2=0x7fca1c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:06.846 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.844+0000 7fca20c00640 1 --2- 172.21.15.152:0/2049919279 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fca1c15eb50 0x7fca1c15d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:06.846 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.844+0000 7fca20c00640 1 -- 172.21.15.152:0/2049919279 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca1c0d1440 msgr2=0x7fca1c0b5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:06.846 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.844+0000 7fca20c00640 1 --2- 172.21.15.152:0/2049919279 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca1c0d1440 0x7fca1c0b5a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:06.846 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.844+0000 7fca20c00640 1 -- 172.21.15.152:0/2049919279 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca1c160830 con 0x7fca1c163230 2024-04-20T11:35:06.846 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.844+0000 7fca20c00640 1 --2- 172.21.15.152:0/2049919279 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fca1c163230 0x7fca1c0b6390 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7fca0c00b490 tx=0x7fca0c00b950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:06.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.844+0000 7fca13e00640 1 -- 172.21.15.152:0/2049919279 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca0c015020 con 0x7fca1c163230 2024-04-20T11:35:06.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.844+0000 7fca13e00640 1 -- 172.21.15.152:0/2049919279 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca0c011850 con 0x7fca1c163230 2024-04-20T11:35:06.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.844+0000 7fca13e00640 1 -- 172.21.15.152:0/2049919279 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca0c011c40 con 0x7fca1c163230 2024-04-20T11:35:06.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.844+0000 7fca21600640 1 -- 172.21.15.152:0/2049919279 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca1c15fa70 con 0x7fca1c163230 2024-04-20T11:35:06.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.845+0000 7fca21600640 1 -- 172.21.15.152:0/2049919279 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca1c160030 con 0x7fca1c163230 2024-04-20T11:35:06.848 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.846+0000 7fca21600640 1 -- 172.21.15.152:0/2049919279 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca1c075e60 con 0x7fca1c163230 2024-04-20T11:35:06.848 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.847+0000 7fca13e00640 1 -- 172.21.15.152:0/2049919279 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 21) ==== 91550+0+0 (secure 0 0 0) 0x7fca0c018020 con 0x7fca1c163230 2024-04-20T11:35:06.848 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.848+0000 7fca13e00640 1 --2- 172.21.15.152:0/2049919279 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fc9f806ce90 0x7fc9f806f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:06.848 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.848+0000 7fca13e00640 1 -- 172.21.15.152:0/2049919279 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5795+0+0 (secure 0 0 0) 0x7fca0c08c4c0 con 0x7fca1c163230 2024-04-20T11:35:06.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.848+0000 7fca1be00640 1 --2- 172.21.15.152:0/2049919279 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fc9f806ce90 0x7fc9f806f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:06.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.849+0000 7fca1be00640 1 --2- 172.21.15.152:0/2049919279 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fc9f806ce90 0x7fc9f806f350 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fca14035000 tx=0x7fca14030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:06.852 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:06.852+0000 7fca13e00640 1 -- 172.21.15.152:0/2049919279 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fca0c01f020 con 0x7fca1c163230 2024-04-20T11:35:07.080 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:07.079+0000 7fca21600640 1 -- 172.21.15.152:0/2049919279 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fca1c162190 con 0x7fca1c163230 2024-04-20T11:35:07.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:07.645+0000 7fca13e00640 1 -- 172.21.15.152:0/2049919279 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v79) ==== 104+0+0 (secure 0 0 0) 0x7fca0c0172c0 con 0x7fca1c163230 2024-04-20T11:35:07.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:07.649+0000 7fca21600640 1 -- 172.21.15.152:0/2049919279 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fc9f806ce90 msgr2=0x7fc9f806f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:07.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:07.649+0000 7fca21600640 1 --2- 172.21.15.152:0/2049919279 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fc9f806ce90 0x7fc9f806f350 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fca14035000 tx=0x7fca14030000 comp rx=0 tx=0).stop 2024-04-20T11:35:07.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:07.649+0000 7fca21600640 1 -- 172.21.15.152:0/2049919279 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fca1c163230 msgr2=0x7fca1c0b6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:07.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:07.649+0000 7fca21600640 1 --2- 172.21.15.152:0/2049919279 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fca1c163230 0x7fca1c0b6390 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7fca0c00b490 tx=0x7fca0c00b950 comp rx=0 tx=0).stop 2024-04-20T11:35:07.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:07.649+0000 7fca21600640 1 -- 172.21.15.152:0/2049919279 shutdown_connections 2024-04-20T11:35:07.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:07.649+0000 7fca21600640 1 --2- 172.21.15.152:0/2049919279 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fc9f806ce90 0x7fc9f806f350 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:07.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:07.649+0000 7fca21600640 1 --2- 172.21.15.152:0/2049919279 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fca1c163230 0x7fca1c0b6390 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:07.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:07.649+0000 7fca21600640 1 --2- 172.21.15.152:0/2049919279 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fca1c15eb50 0x7fca1c15d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:07.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:07.649+0000 7fca21600640 1 --2- 172.21.15.152:0/2049919279 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca1c0d1440 0x7fca1c0b5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:07.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:07.649+0000 7fca21600640 1 -- 172.21.15.152:0/2049919279 >> 172.21.15.152:0/2049919279 conn(0x7fca1c08bfd0 msgr2=0x7fca1c0b9140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:07.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:07.649+0000 7fca21600640 1 -- 172.21.15.152:0/2049919279 shutdown_connections 2024-04-20T11:35:07.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:07.650+0000 7fca21600640 1 -- 172.21.15.152:0/2049919279 wait complete. 2024-04-20T11:35:07.673 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-20T11:35:07.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:07.876+0000 7f98e0a00640 1 Processor -- start 2024-04-20T11:35:07.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:07.876+0000 7f98e0a00640 1 -- start start 2024-04-20T11:35:07.878 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:07.877+0000 7f98e0a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f98dc15b5c0 0x7f98dc071f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:07.878 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:07.877+0000 7f98e0a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f98dc0cebd0 0x7f98dc0b3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:07.878 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:07.877+0000 7f98e0a00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f98dc0b4240 0x7f98dc0d05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:07.878 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:07.877+0000 7f98e0a00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f98dc15b9a0 con 0x7f98dc15b5c0 2024-04-20T11:35:07.878 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:07.877+0000 7f98e0a00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f98dc072610 con 0x7f98dc0b4240 2024-04-20T11:35:07.878 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:07.877+0000 7f98e0a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f98dc0728f0 con 0x7f98dc0cebd0 2024-04-20T11:35:07.878 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:07.878+0000 7f98db400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f98dc15b5c0 0x7f98dc071f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:07.878 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:07.878+0000 7f98db400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f98dc15b5c0 0x7f98dc071f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:56768/0 (socket says 172.21.15.152:56768) 2024-04-20T11:35:07.879 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:07.878+0000 7f98db400640 1 -- 172.21.15.152:0/1257769864 learned_addr learned my addr 172.21.15.152:0/1257769864 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:07.879 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:07.878+0000 7f98db400640 1 -- 172.21.15.152:0/1257769864 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f98dc0cebd0 msgr2=0x7f98dc0b3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:07.879 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:07.878+0000 7f98db400640 1 --2- 172.21.15.152:0/1257769864 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f98dc0cebd0 0x7f98dc0b3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:07.879 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:07.878+0000 7f98db400640 1 -- 172.21.15.152:0/1257769864 >> v2:172.21.15.175:3300/0 conn(0x7f98dc0b4240 msgr2=0x7f98dc0d05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:07.879 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:07.878+0000 7f98db400640 1 --2- 172.21.15.152:0/1257769864 >> v2:172.21.15.175:3300/0 conn(0x7f98dc0b4240 0x7f98dc0d05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:07.879 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:07.878+0000 7f98db400640 1 -- 172.21.15.152:0/1257769864 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98dc0b3500 con 0x7f98dc15b5c0 2024-04-20T11:35:07.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:07.879+0000 7f98db400640 1 --2- 172.21.15.152:0/1257769864 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f98dc15b5c0 0x7f98dc071f10 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f98dc073b80 tx=0x7f98cc02b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fa77249f5e39b92 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:07.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:07.879+0000 7f98daa00640 1 -- 172.21.15.152:0/1257769864 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98cc035020 con 0x7f98dc15b5c0 2024-04-20T11:35:07.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:07.879+0000 7f98daa00640 1 -- 172.21.15.152:0/1257769864 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98cc031800 con 0x7f98dc15b5c0 2024-04-20T11:35:07.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:07.880+0000 7f98daa00640 1 -- 172.21.15.152:0/1257769864 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98cc031bd0 con 0x7f98dc15b5c0 2024-04-20T11:35:07.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:07.880+0000 7f98e0a00640 1 -- 172.21.15.152:0/1257769864 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f98dc15b5c0 msgr2=0x7f98dc071f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:07.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:07.880+0000 7f98e0a00640 1 --2- 172.21.15.152:0/1257769864 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f98dc15b5c0 0x7f98dc071f10 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f98dc073b80 tx=0x7f98cc02b8a0 comp rx=0 tx=0).stop 2024-04-20T11:35:07.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:07.880+0000 7f98e0a00640 1 -- 172.21.15.152:0/1257769864 shutdown_connections 2024-04-20T11:35:07.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:07.880+0000 7f98e0a00640 1 --2- 172.21.15.152:0/1257769864 >> v2:172.21.15.175:3300/0 conn(0x7f98dc0b4240 0x7f98dc0d05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:07.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:07.880+0000 7f98e0a00640 1 --2- 172.21.15.152:0/1257769864 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f98dc0cebd0 0x7f98dc0b3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:07.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:07.880+0000 7f98e0a00640 1 --2- 172.21.15.152:0/1257769864 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f98dc15b5c0 0x7f98dc071f10 unknown :-1 s=CLOSED pgs=1702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:07.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:07.880+0000 7f98e0a00640 1 -- 172.21.15.152:0/1257769864 >> 172.21.15.152:0/1257769864 conn(0x7f98dc06bda0 msgr2=0x7f98dc06c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:07.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:07.880+0000 7f98e0a00640 1 -- 172.21.15.152:0/1257769864 shutdown_connections 2024-04-20T11:35:07.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:07.880+0000 7f98e0a00640 1 -- 172.21.15.152:0/1257769864 wait complete. 2024-04-20T11:35:07.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:07.882+0000 7f98e0a00640 1 Processor -- start 2024-04-20T11:35:07.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:07.882+0000 7f98e0a00640 1 -- start start 2024-04-20T11:35:07.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:07.883+0000 7f98e0a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f98dc0b4240 0x7f98dc157f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:07.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:07.883+0000 7f98e0a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98dc0cebd0 0x7f98dc1f4520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:07.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:07.883+0000 7f98e0a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f98dc15b5c0 0x7f98dc1f4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:07.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:07.883+0000 7f98e0a00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f98dc0d0b00 con 0x7f98dc0b4240 2024-04-20T11:35:07.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:07.883+0000 7f98e0a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f98dc0cf030 con 0x7f98dc0cebd0 2024-04-20T11:35:07.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:07.883+0000 7f98e0a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f98dc0d0c80 con 0x7f98dc15b5c0 2024-04-20T11:35:07.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:07.883+0000 7f98db400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f98dc0b4240 0x7f98dc157f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:07.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:07.883+0000 7f98db400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f98dc0b4240 0x7f98dc157f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:56772/0 (socket says 172.21.15.152:56772) 2024-04-20T11:35:07.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:07.883+0000 7f98db400640 1 -- 172.21.15.152:0/1099627466 learned_addr learned my addr 172.21.15.152:0/1099627466 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:07.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:07.884+0000 7f98db400640 1 -- 172.21.15.152:0/1099627466 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f98dc15b5c0 msgr2=0x7f98dc1f4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:07.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:07.884+0000 7f98dbe00640 1 --2- 172.21.15.152:0/1099627466 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f98dc15b5c0 0x7f98dc1f4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:07.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:07.884+0000 7f98d2a00640 1 --2- 172.21.15.152:0/1099627466 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98dc0cebd0 0x7f98dc1f4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:07.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:07.884+0000 7f98db400640 1 --2- 172.21.15.152:0/1099627466 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f98dc15b5c0 0x7f98dc1f4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:07.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:07.884+0000 7f98db400640 1 -- 172.21.15.152:0/1099627466 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98dc0cebd0 msgr2=0x7f98dc1f4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:07.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:07.884+0000 7f98db400640 1 --2- 172.21.15.152:0/1099627466 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98dc0cebd0 0x7f98dc1f4520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:07.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:07.884+0000 7f98db400640 1 -- 172.21.15.152:0/1099627466 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98dc0aefa0 con 0x7f98dc0b4240 2024-04-20T11:35:07.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:07.884+0000 7f98dbe00640 1 --2- 172.21.15.152:0/1099627466 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f98dc15b5c0 0x7f98dc1f4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:35:07.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:07.884+0000 7f98db400640 1 --2- 172.21.15.152:0/1099627466 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f98dc0b4240 0x7f98dc157f30 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f98cc000c00 tx=0x7f98cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:07.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:07.884+0000 7f98d3e00640 1 -- 172.21.15.152:0/1099627466 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98cc03ad60 con 0x7f98dc0b4240 2024-04-20T11:35:07.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:07.884+0000 7f98d3e00640 1 -- 172.21.15.152:0/1099627466 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98cc0443f0 con 0x7f98dc0b4240 2024-04-20T11:35:07.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:07.884+0000 7f98d3e00640 1 -- 172.21.15.152:0/1099627466 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98cc0447e0 con 0x7f98dc0b4240 2024-04-20T11:35:07.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:07.885+0000 7f98e0a00640 1 -- 172.21.15.152:0/1099627466 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98dc0ae2d0 con 0x7f98dc0b4240 2024-04-20T11:35:07.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:07.885+0000 7f98e0a00640 1 -- 172.21.15.152:0/1099627466 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98dc0aebc0 con 0x7f98dc0b4240 2024-04-20T11:35:07.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:07.886+0000 7f98e0a00640 1 -- 172.21.15.152:0/1099627466 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98dc0689f0 con 0x7f98dc0b4240 2024-04-20T11:35:07.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:07.887+0000 7f98d3e00640 1 -- 172.21.15.152:0/1099627466 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 21) ==== 91550+0+0 (secure 0 0 0) 0x7f98cc04b060 con 0x7f98dc0b4240 2024-04-20T11:35:07.888 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:07.887+0000 7f98d3e00640 1 --2- 172.21.15.152:0/1099627466 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f98b806cec0 0x7f98b806f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:07.888 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:07.888+0000 7f98d3e00640 1 -- 172.21.15.152:0/1099627466 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5795+0+0 (secure 0 0 0) 0x7f98cc0ae640 con 0x7f98dc0b4240 2024-04-20T11:35:07.888 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:07.888+0000 7f98d2a00640 1 --2- 172.21.15.152:0/1099627466 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f98b806cec0 0x7f98b806f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:07.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:07.889+0000 7f98d2a00640 1 --2- 172.21.15.152:0/1099627466 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f98b806cec0 0x7f98b806f380 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f98dc155bb0 tx=0x7f98c4008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:07.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:07.892+0000 7f98d3e00640 1 -- 172.21.15.152:0/1099627466 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f98cc03f070 con 0x7f98dc0b4240 2024-04-20T11:35:08.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:08.094+0000 7f98e0a00640 1 -- 172.21.15.152:0/1099627466 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f98dc0cfe30 con 0x7f98dc0b4240 2024-04-20T11:35:08.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:08.649+0000 7f98d3e00640 1 -- 172.21.15.152:0/1099627466 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v122) ==== 181+0+0 (secure 0 0 0) 0x7f98cc07c050 con 0x7f98dc0b4240 2024-04-20T11:35:08.651 INFO:teuthology.orchestra.run.smithi152.stderr:pool 'cephfs_metadata' removed 2024-04-20T11:35:08.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:08.653+0000 7f98e0a00640 1 -- 172.21.15.152:0/1099627466 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f98b806cec0 msgr2=0x7f98b806f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:08.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:08.653+0000 7f98e0a00640 1 --2- 172.21.15.152:0/1099627466 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f98b806cec0 0x7f98b806f380 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f98dc155bb0 tx=0x7f98c4008040 comp rx=0 tx=0).stop 2024-04-20T11:35:08.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:08.653+0000 7f98e0a00640 1 -- 172.21.15.152:0/1099627466 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f98dc0b4240 msgr2=0x7f98dc157f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:08.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:08.653+0000 7f98e0a00640 1 --2- 172.21.15.152:0/1099627466 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f98dc0b4240 0x7f98dc157f30 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f98cc000c00 tx=0x7f98cc0046c0 comp rx=0 tx=0).stop 2024-04-20T11:35:08.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:08.653+0000 7f98e0a00640 1 -- 172.21.15.152:0/1099627466 shutdown_connections 2024-04-20T11:35:08.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:08.653+0000 7f98e0a00640 1 --2- 172.21.15.152:0/1099627466 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f98b806cec0 0x7f98b806f380 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:08.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:08.653+0000 7f98e0a00640 1 --2- 172.21.15.152:0/1099627466 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f98dc15b5c0 0x7f98dc1f4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:08.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:08.653+0000 7f98e0a00640 1 --2- 172.21.15.152:0/1099627466 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98dc0cebd0 0x7f98dc1f4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:08.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:08.653+0000 7f98e0a00640 1 --2- 172.21.15.152:0/1099627466 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f98dc0b4240 0x7f98dc157f30 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:08.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:08.653+0000 7f98e0a00640 1 -- 172.21.15.152:0/1099627466 >> 172.21.15.152:0/1099627466 conn(0x7f98dc06bda0 msgr2=0x7f98dc15bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:08.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:08.653+0000 7f98e0a00640 1 -- 172.21.15.152:0/1099627466 shutdown_connections 2024-04-20T11:35:08.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:08.654+0000 7f98e0a00640 1 -- 172.21.15.152:0/1099627466 wait complete. 2024-04-20T11:35:08.683 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-20T11:35:08.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:08.880+0000 7fe369a00640 1 Processor -- start 2024-04-20T11:35:08.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:08.880+0000 7fe369a00640 1 -- start start 2024-04-20T11:35:08.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:08.881+0000 7fe369a00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fe3641569d0 0x7fe3640ad760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:08.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:08.882+0000 7fe369a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe36406e970 0x7fe3640d05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:08.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:08.882+0000 7fe369a00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe36415d8d0 con 0x7fe36406e510 2024-04-20T11:35:08.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:08.882+0000 7fe369a00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fe36415d310 con 0x7fe3641569d0 2024-04-20T11:35:08.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:08.882+0000 7fe369a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe36415d5f0 con 0x7fe36406e970 2024-04-20T11:35:08.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:08.882+0000 7fe35b400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fe36406e510 0x7fe36415ca10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:54348/0 (socket says 172.21.15.152:54348) 2024-04-20T11:35:08.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:08.882+0000 7fe35b400640 1 -- 172.21.15.152:0/1975694801 learned_addr learned my addr 172.21.15.152:0/1975694801 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:35:08.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:08.882+0000 7fe363e00640 1 --2- 172.21.15.152:0/1975694801 >> v2:172.21.15.175:3300/0 conn(0x7fe3641569d0 0x7fe3640ad760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:08.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:08.882+0000 7fe369000640 1 --2- 172.21.15.152:0/1975694801 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe36406e970 0x7fe3640d05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:08.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:08.882+0000 7fe363400640 1 -- 172.21.15.152:0/1975694801 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1394748509 0 0) 0x7fe36415d8d0 con 0x7fe36406e510 2024-04-20T11:35:08.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:08.882+0000 7fe363400640 1 -- 172.21.15.152:0/1975694801 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe348003520 con 0x7fe36406e510 2024-04-20T11:35:08.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:08.883+0000 7fe363400640 1 -- 172.21.15.152:0/1975694801 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1084228983 0 0) 0x7fe348003520 con 0x7fe36406e510 2024-04-20T11:35:08.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:08.883+0000 7fe363400640 1 -- 172.21.15.152:0/1975694801 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe36406e970 msgr2=0x7fe3640d05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:08.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:08.883+0000 7fe363400640 1 --2- 172.21.15.152:0/1975694801 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe36406e970 0x7fe3640d05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:08.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:08.883+0000 7fe363400640 1 -- 172.21.15.152:0/1975694801 >> v2:172.21.15.175:3300/0 conn(0x7fe3641569d0 msgr2=0x7fe3640ad760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:08.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:08.883+0000 7fe363400640 1 --2- 172.21.15.152:0/1975694801 >> v2:172.21.15.175:3300/0 conn(0x7fe3641569d0 0x7fe3640ad760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:08.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:08.883+0000 7fe363400640 1 -- 172.21.15.152:0/1975694801 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe36406dd10 con 0x7fe36406e510 2024-04-20T11:35:08.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:08.883+0000 7fe363400640 1 -- 172.21.15.152:0/1975694801 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7fe354001670 con 0x7fe36406e510 2024-04-20T11:35:08.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:08.883+0000 7fe363400640 1 -- 172.21.15.152:0/1975694801 >> v1:172.21.15.152:6789/0 conn(0x7fe36406e510 legacy=0x7fe36415ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:08.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:08.883+0000 7fe363400640 1 --2- 172.21.15.152:0/1975694801 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe3480039b0 0x7fe348003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:08.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:08.883+0000 7fe363400640 1 --2- 172.21.15.152:0/1975694801 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe348004360 0x7fe348004840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:08.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:08.883+0000 7fe363400640 1 --2- 172.21.15.152:0/1975694801 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe348004f60 0x7fe34800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:08.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:08.883+0000 7fe363400640 1 -- 172.21.15.152:0/1975694801 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fe36415d310 con 0x7fe348004f60 2024-04-20T11:35:08.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:08.883+0000 7fe363400640 1 -- 172.21.15.152:0/1975694801 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe36415d5f0 con 0x7fe348004360 2024-04-20T11:35:08.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:08.883+0000 7fe363400640 1 -- 172.21.15.152:0/1975694801 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe34800bf00 con 0x7fe3480039b0 2024-04-20T11:35:08.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:08.883+0000 7fe369000640 1 --2- 172.21.15.152:0/1975694801 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe348004f60 0x7fe34800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:08.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:08.884+0000 7fe35b400640 1 --2- 172.21.15.152:0/1975694801 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe348004360 0x7fe348004840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:08.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:08.884+0000 7fe363e00640 1 --2- 172.21.15.152:0/1975694801 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe3480039b0 0x7fe348003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:08.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:08.884+0000 7fe369000640 1 -- 172.21.15.152:0/1975694801 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe3480039b0 msgr2=0x7fe348003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:08.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:08.884+0000 7fe369000640 1 --2- 172.21.15.152:0/1975694801 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe3480039b0 0x7fe348003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:08.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:08.884+0000 7fe369000640 1 -- 172.21.15.152:0/1975694801 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe348004360 msgr2=0x7fe348004840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:08.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:08.884+0000 7fe369000640 1 --2- 172.21.15.152:0/1975694801 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe348004360 0x7fe348004840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:08.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:08.884+0000 7fe369000640 1 -- 172.21.15.152:0/1975694801 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe348003520 con 0x7fe348004f60 2024-04-20T11:35:08.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:08.885+0000 7fe369000640 1 --2- 172.21.15.152:0/1975694801 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe348004f60 0x7fe34800b9c0 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7fe35c009fc0 tx=0x7fe348005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8844a4bc845cc8d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:08.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:08.886+0000 7fe363400640 1 -- 172.21.15.152:0/1975694801 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe35c039070 con 0x7fe348004f60 2024-04-20T11:35:08.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:08.886+0000 7fe363400640 1 -- 172.21.15.152:0/1975694801 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe35c02cbd0 con 0x7fe348004f60 2024-04-20T11:35:08.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:08.886+0000 7fe363400640 1 -- 172.21.15.152:0/1975694801 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe35c009c10 con 0x7fe348004f60 2024-04-20T11:35:08.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:08.886+0000 7fe369a00640 1 -- 172.21.15.152:0/1975694801 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe348004f60 msgr2=0x7fe34800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:08.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:08.886+0000 7fe369a00640 1 --2- 172.21.15.152:0/1975694801 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe348004f60 0x7fe34800b9c0 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7fe35c009fc0 tx=0x7fe348005830 comp rx=0 tx=0).stop 2024-04-20T11:35:08.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:08.886+0000 7fe369000640 1 -- 172.21.15.152:0/1975694801 reap_dead start 2024-04-20T11:35:08.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:08.886+0000 7fe369a00640 1 -- 172.21.15.152:0/1975694801 shutdown_connections 2024-04-20T11:35:08.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:08.886+0000 7fe369a00640 1 -- 172.21.15.152:0/1975694801 >> 172.21.15.152:0/1975694801 conn(0x7fe3640b21e0 msgr2=0x7fe3640b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:08.890 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:08.886+0000 7fe369a00640 1 -- 172.21.15.152:0/1975694801 shutdown_connections 2024-04-20T11:35:08.890 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:08.886+0000 7fe369a00640 1 -- 172.21.15.152:0/1975694801 wait complete. 2024-04-20T11:35:08.890 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:08.888+0000 7fe369a00640 1 Processor -- start 2024-04-20T11:35:08.890 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:08.888+0000 7fe369a00640 1 -- start start 2024-04-20T11:35:08.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:08.889+0000 7fe369a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe3641f19c0 0x7fe364072b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:08.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:08.889+0000 7fe369a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe36415ee80 0x7fe3640b3e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:08.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:08.889+0000 7fe369a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe3640b4360 0x7fe3641f5db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:08.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:08.889+0000 7fe369a00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fe3640d05f0 con 0x7fe3640b4360 2024-04-20T11:35:08.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:08.889+0000 7fe369a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe364156db0 con 0x7fe3641f19c0 2024-04-20T11:35:08.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:08.889+0000 7fe369a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe3640d0770 con 0x7fe36415ee80 2024-04-20T11:35:08.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:08.890+0000 7fe369000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe3640b4360 0x7fe3641f5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:08.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:08.890+0000 7fe369000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe3640b4360 0x7fe3641f5db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:56794/0 (socket says 172.21.15.152:56794) 2024-04-20T11:35:08.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:08.890+0000 7fe369000640 1 -- 172.21.15.152:0/2914132125 learned_addr learned my addr 172.21.15.152:0/2914132125 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:08.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:08.890+0000 7fe369000640 1 -- 172.21.15.152:0/2914132125 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe36415ee80 msgr2=0x7fe3640b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:08.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:08.890+0000 7fe369000640 1 --2- 172.21.15.152:0/2914132125 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe36415ee80 0x7fe3640b3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:08.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:08.890+0000 7fe369000640 1 -- 172.21.15.152:0/2914132125 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe3641f19c0 msgr2=0x7fe364072b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:08.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:08.890+0000 7fe369000640 1 --2- 172.21.15.152:0/2914132125 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe3641f19c0 0x7fe364072b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:08.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:08.890+0000 7fe369000640 1 -- 172.21.15.152:0/2914132125 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3641f4d90 con 0x7fe3640b4360 2024-04-20T11:35:08.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:08.890+0000 7fe369000640 1 --2- 172.21.15.152:0/2914132125 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe3640b4360 0x7fe3641f5db0 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7fe35c00e200 tx=0x7fe35c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:08.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:08.891+0000 7fe360c00640 1 -- 172.21.15.152:0/2914132125 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe35c039070 con 0x7fe3640b4360 2024-04-20T11:35:08.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:08.891+0000 7fe360c00640 1 -- 172.21.15.152:0/2914132125 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe35c03a040 con 0x7fe3640b4360 2024-04-20T11:35:08.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:08.891+0000 7fe360c00640 1 -- 172.21.15.152:0/2914132125 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe35c009c10 con 0x7fe3640b4360 2024-04-20T11:35:08.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:08.891+0000 7fe369a00640 1 -- 172.21.15.152:0/2914132125 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3641f5070 con 0x7fe3640b4360 2024-04-20T11:35:08.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:08.891+0000 7fe369a00640 1 -- 172.21.15.152:0/2914132125 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3640b4ef0 con 0x7fe3640b4360 2024-04-20T11:35:08.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:08.892+0000 7fe369a00640 1 -- 172.21.15.152:0/2914132125 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3640b25f0 con 0x7fe3640b4360 2024-04-20T11:35:08.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:08.893+0000 7fe360c00640 1 -- 172.21.15.152:0/2914132125 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 21) ==== 91550+0+0 (secure 0 0 0) 0x7fe35c046070 con 0x7fe3640b4360 2024-04-20T11:35:08.894 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:08.894+0000 7fe360c00640 1 --2- 172.21.15.152:0/2914132125 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe34006cec0 0x7fe34006f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:08.894 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:08.894+0000 7fe360c00640 1 -- 172.21.15.152:0/2914132125 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5357+0+0 (secure 0 0 0) 0x7fe35c0adda0 con 0x7fe3640b4360 2024-04-20T11:35:08.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:08.895+0000 7fe363e00640 1 --2- 172.21.15.152:0/2914132125 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe34006cec0 0x7fe34006f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:08.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:08.896+0000 7fe363e00640 1 --2- 172.21.15.152:0/2914132125 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe34006cec0 0x7fe34006f380 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe34c006a90 tx=0x7fe34c006a20 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:08.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:08.899+0000 7fe360c00640 1 -- 172.21.15.152:0/2914132125 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe3640b25f0 con 0x7fe3640b4360 2024-04-20T11:35:09.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:09.099+0000 7fe369a00640 1 -- 172.21.15.152:0/2914132125 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fe364075e60 con 0x7fe3640b4360 2024-04-20T11:35:09.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:09.653+0000 7fe360c00640 1 -- 172.21.15.152:0/2914132125 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v123) ==== 169+0+0 (secure 0 0 0) 0x7fe35c07bc70 con 0x7fe3640b4360 2024-04-20T11:35:09.654 INFO:teuthology.orchestra.run.smithi152.stderr:pool 'cephfs_data' removed 2024-04-20T11:35:09.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:09.657+0000 7fe369a00640 1 -- 172.21.15.152:0/2914132125 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe34006cec0 msgr2=0x7fe34006f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:09.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:09.657+0000 7fe369a00640 1 --2- 172.21.15.152:0/2914132125 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe34006cec0 0x7fe34006f380 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe34c006a90 tx=0x7fe34c006a20 comp rx=0 tx=0).stop 2024-04-20T11:35:09.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:09.657+0000 7fe369a00640 1 -- 172.21.15.152:0/2914132125 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe3640b4360 msgr2=0x7fe3641f5db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:09.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:09.657+0000 7fe369a00640 1 --2- 172.21.15.152:0/2914132125 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe3640b4360 0x7fe3641f5db0 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7fe35c00e200 tx=0x7fe35c009750 comp rx=0 tx=0).stop 2024-04-20T11:35:09.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:09.657+0000 7fe369a00640 1 -- 172.21.15.152:0/2914132125 shutdown_connections 2024-04-20T11:35:09.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:09.657+0000 7fe369a00640 1 --2- 172.21.15.152:0/2914132125 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe34006cec0 0x7fe34006f380 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:09.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:09.657+0000 7fe369a00640 1 --2- 172.21.15.152:0/2914132125 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe3640b4360 0x7fe3641f5db0 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:09.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:09.657+0000 7fe369a00640 1 --2- 172.21.15.152:0/2914132125 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe36415ee80 0x7fe3640b3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:09.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:09.657+0000 7fe369a00640 1 --2- 172.21.15.152:0/2914132125 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe3641f19c0 0x7fe364072b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:09.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:09.657+0000 7fe369a00640 1 -- 172.21.15.152:0/2914132125 >> 172.21.15.152:0/2914132125 conn(0x7fe3640b21e0 msgr2=0x7fe3640ad0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:09.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:09.658+0000 7fe369a00640 1 -- 172.21.15.152:0/2914132125 shutdown_connections 2024-04-20T11:35:09.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:09.658+0000 7fe369a00640 1 -- 172.21.15.152:0/2914132125 wait complete. 2024-04-20T11:35:09.695 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-20T11:35:09.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:09.871+0000 7fc39d600640 1 Processor -- start 2024-04-20T11:35:09.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:09.871+0000 7fc39d600640 1 -- start start 2024-04-20T11:35:09.873 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:09.873+0000 7fc39d600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc39809b890 0x7fc39815ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:09.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:09.873+0000 7fc39d600640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fc39809b090 0x7fc3980d30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:09.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:09.873+0000 7fc39d600640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3980d0ff0 con 0x7fc39809acb0 2024-04-20T11:35:09.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:09.873+0000 7fc39d600640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fc39806dc00 con 0x7fc39809b090 2024-04-20T11:35:09.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:09.873+0000 7fc39d600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc3980d1290 con 0x7fc39809b890 2024-04-20T11:35:09.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:09.874+0000 7fc397400640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fc39809b090 0x7fc3980d30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:09.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:09.874+0000 7fc38e000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc39809b890 0x7fc39815ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:09.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:09.874+0000 7fc397400640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fc39809b090 0x7fc3980d30d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.152:42130/0 (socket says 172.21.15.152:42130) 2024-04-20T11:35:09.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:09.874+0000 7fc38e000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc39809b890 0x7fc39815ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.152:56480/0 (socket says 172.21.15.152:56480) 2024-04-20T11:35:09.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:09.874+0000 7fc397400640 1 -- 172.21.15.152:0/1725795371 learned_addr learned my addr 172.21.15.152:0/1725795371 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:09.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:09.875+0000 7fc397400640 1 -- 172.21.15.152:0/1725795371 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc39809b890 msgr2=0x7fc39815ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:09.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:09.875+0000 7fc397400640 1 --2- 172.21.15.152:0/1725795371 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc39809b890 0x7fc39815ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:09.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:09.875+0000 7fc397400640 1 -- 172.21.15.152:0/1725795371 >> v1:172.21.15.175:6789/0 conn(0x7fc39809acb0 legacy=0x7fc39815e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:09.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:09.875+0000 7fc397400640 1 -- 172.21.15.152:0/1725795371 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc39806dea0 con 0x7fc39809b090 2024-04-20T11:35:09.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:09.875+0000 7fc38e000640 1 --2- 172.21.15.152:0/1725795371 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc39809b890 0x7fc39815ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:35:09.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:09.876+0000 7fc397400640 1 --2- 172.21.15.152:0/1725795371 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc39809b090 0x7fc3980d30d0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fc390009870 tx=0x7fc39002b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7a115d722c0b268 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:09.877 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:09.876+0000 7fc396000640 1 -- 172.21.15.152:0/1725795371 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc390035020 con 0x7fc39809b090 2024-04-20T11:35:09.877 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:09.876+0000 7fc396000640 1 -- 172.21.15.152:0/1725795371 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc390031870 con 0x7fc39809b090 2024-04-20T11:35:09.877 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:09.877+0000 7fc396000640 1 -- 172.21.15.152:0/1725795371 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc390031c40 con 0x7fc39809b090 2024-04-20T11:35:09.878 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:09.877+0000 7fc39d600640 1 -- 172.21.15.152:0/1725795371 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc39809b090 msgr2=0x7fc3980d30d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:09.878 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:09.877+0000 7fc39d600640 1 --2- 172.21.15.152:0/1725795371 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc39809b090 0x7fc3980d30d0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fc390009870 tx=0x7fc39002b910 comp rx=0 tx=0).stop 2024-04-20T11:35:09.878 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:09.877+0000 7fc39d600640 1 -- 172.21.15.152:0/1725795371 shutdown_connections 2024-04-20T11:35:09.878 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:09.877+0000 7fc39d600640 1 --2- 172.21.15.152:0/1725795371 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc39809b090 0x7fc3980d30d0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:09.878 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:09.877+0000 7fc39d600640 1 --2- 172.21.15.152:0/1725795371 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc39809b890 0x7fc39815ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:09.878 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:09.877+0000 7fc39d600640 1 -- 172.21.15.152:0/1725795371 >> 172.21.15.152:0/1725795371 conn(0x7fc3980d25b0 msgr2=0x7fc3980d29c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:09.878 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:09.877+0000 7fc39d600640 1 -- 172.21.15.152:0/1725795371 shutdown_connections 2024-04-20T11:35:09.878 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:09.877+0000 7fc39d600640 1 -- 172.21.15.152:0/1725795371 wait complete. 2024-04-20T11:35:09.879 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:09.879+0000 7fc39d600640 1 Processor -- start 2024-04-20T11:35:09.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:09.879+0000 7fc39d600640 1 -- start start 2024-04-20T11:35:09.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:09.880+0000 7fc39d600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc39809acb0 0x7fc3981f8d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:09.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:09.880+0000 7fc39d600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc39809b090 0x7fc3980713e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:09.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:09.880+0000 7fc39d600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc39809b890 0x7fc398071920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:09.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:09.880+0000 7fc39d600640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fc39806e540 con 0x7fc39809b090 2024-04-20T11:35:09.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:09.880+0000 7fc39d600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc39806e3c0 con 0x7fc39809acb0 2024-04-20T11:35:09.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:09.880+0000 7fc39d600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc39806e6c0 con 0x7fc39809b890 2024-04-20T11:35:09.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:09.880+0000 7fc38e000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc39809b090 0x7fc3980713e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:09.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:09.881+0000 7fc38e000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc39809b090 0x7fc3980713e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:56808/0 (socket says 172.21.15.152:56808) 2024-04-20T11:35:09.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:09.881+0000 7fc38e000640 1 -- 172.21.15.152:0/1650722143 learned_addr learned my addr 172.21.15.152:0/1650722143 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:09.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:09.881+0000 7fc396a00640 1 --2- 172.21.15.152:0/1650722143 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc39809acb0 0x7fc3981f8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:09.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:09.881+0000 7fc397400640 1 --2- 172.21.15.152:0/1650722143 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc39809b890 0x7fc398071920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:09.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:09.881+0000 7fc38e000640 1 -- 172.21.15.152:0/1650722143 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc39809b890 msgr2=0x7fc398071920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:09.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:09.881+0000 7fc38e000640 1 --2- 172.21.15.152:0/1650722143 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc39809b890 0x7fc398071920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:09.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:09.881+0000 7fc38e000640 1 -- 172.21.15.152:0/1650722143 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc39809acb0 msgr2=0x7fc3981f8d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:09.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:09.881+0000 7fc38e000640 1 --2- 172.21.15.152:0/1650722143 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc39809acb0 0x7fc3981f8d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:09.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:09.881+0000 7fc38e000640 1 -- 172.21.15.152:0/1650722143 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3981f4d30 con 0x7fc39809b090 2024-04-20T11:35:09.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:09.881+0000 7fc396a00640 1 --2- 172.21.15.152:0/1650722143 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc39809acb0 0x7fc3981f8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:35:09.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:09.881+0000 7fc38e000640 1 --2- 172.21.15.152:0/1650722143 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc39809b090 0x7fc3980713e0 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7fc38000ebe0 tx=0x7fc38000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:09.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:09.882+0000 7fc38f400640 1 -- 172.21.15.152:0/1650722143 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc380017070 con 0x7fc39809b090 2024-04-20T11:35:09.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:09.882+0000 7fc38f400640 1 -- 172.21.15.152:0/1650722143 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc38000cc30 con 0x7fc39809b090 2024-04-20T11:35:09.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:09.882+0000 7fc38f400640 1 -- 172.21.15.152:0/1650722143 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc380013470 con 0x7fc39809b090 2024-04-20T11:35:09.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:09.882+0000 7fc39d600640 1 -- 172.21.15.152:0/1650722143 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3981f4ff0 con 0x7fc39809b090 2024-04-20T11:35:09.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:09.882+0000 7fc39d600640 1 -- 172.21.15.152:0/1650722143 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3981f55b0 con 0x7fc39809b090 2024-04-20T11:35:09.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:09.883+0000 7fc39d600640 1 -- 172.21.15.152:0/1650722143 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3980689f0 con 0x7fc39809b090 2024-04-20T11:35:09.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:09.884+0000 7fc38f400640 1 -- 172.21.15.152:0/1650722143 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 21) ==== 91550+0+0 (secure 0 0 0) 0x7fc38001a070 con 0x7fc39809b090 2024-04-20T11:35:09.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:09.885+0000 7fc38f400640 1 --2- 172.21.15.152:0/1650722143 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fc37406ce90 0x7fc37406f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:09.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:09.885+0000 7fc38f400640 1 -- 172.21.15.152:0/1650722143 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 4943+0+0 (secure 0 0 0) 0x7fc38008c240 con 0x7fc39809b090 2024-04-20T11:35:09.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:09.886+0000 7fc396a00640 1 --2- 172.21.15.152:0/1650722143 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fc37406ce90 0x7fc37406f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:09.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:09.887+0000 7fc396a00640 1 --2- 172.21.15.152:0/1650722143 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fc37406ce90 0x7fc37406f350 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc388001cd0 tx=0x7fc388009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:09.891 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:09.890+0000 7fc38f400640 1 -- 172.21.15.152:0/1650722143 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fc38005a110 con 0x7fc39809b090 2024-04-20T11:35:10.085 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.084+0000 7fc39d600640 1 -- 172.21.15.152:0/1650722143 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fc39809bde0 con 0x7fc39809b090 2024-04-20T11:35:10.090 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.090+0000 7fc38f400640 1 -- 172.21.15.152:0/1650722143 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v23) ==== 104+0+0 (secure 0 0 0) 0x7fc380059a10 con 0x7fc39809b090 2024-04-20T11:35:10.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.092+0000 7fc39d600640 1 -- 172.21.15.152:0/1650722143 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fc37406ce90 msgr2=0x7fc37406f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:10.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.092+0000 7fc39d600640 1 --2- 172.21.15.152:0/1650722143 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fc37406ce90 0x7fc37406f350 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc388001cd0 tx=0x7fc388009040 comp rx=0 tx=0).stop 2024-04-20T11:35:10.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.093+0000 7fc39d600640 1 -- 172.21.15.152:0/1650722143 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc39809b090 msgr2=0x7fc3980713e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:10.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.093+0000 7fc39d600640 1 --2- 172.21.15.152:0/1650722143 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc39809b090 0x7fc3980713e0 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7fc38000ebe0 tx=0x7fc38000c520 comp rx=0 tx=0).stop 2024-04-20T11:35:10.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.093+0000 7fc39d600640 1 -- 172.21.15.152:0/1650722143 shutdown_connections 2024-04-20T11:35:10.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.093+0000 7fc39d600640 1 --2- 172.21.15.152:0/1650722143 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fc37406ce90 0x7fc37406f350 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:10.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.093+0000 7fc39d600640 1 --2- 172.21.15.152:0/1650722143 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc39809b890 0x7fc398071920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:10.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.093+0000 7fc39d600640 1 --2- 172.21.15.152:0/1650722143 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc39809b090 0x7fc3980713e0 unknown :-1 s=CLOSED pgs=1707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:10.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.093+0000 7fc39d600640 1 --2- 172.21.15.152:0/1650722143 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc39809acb0 0x7fc3981f8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:10.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.093+0000 7fc39d600640 1 -- 172.21.15.152:0/1650722143 >> 172.21.15.152:0/1650722143 conn(0x7fc3980d25b0 msgr2=0x7fc3980b7660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:10.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.093+0000 7fc39d600640 1 -- 172.21.15.152:0/1650722143 shutdown_connections 2024-04-20T11:35:10.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.093+0000 7fc39d600640 1 -- 172.21.15.152:0/1650722143 wait complete. 2024-04-20T11:35:10.111 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_to_v2' 2024-04-20T11:35:10.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.296+0000 7fa026800640 1 Processor -- start 2024-04-20T11:35:10.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.297+0000 7fa026800640 1 -- start start 2024-04-20T11:35:10.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.298+0000 7fa026800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa02015fcc0 0x7fa0200d4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:10.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.298+0000 7fa026800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fa0200b65c0 0x7fa0200cf240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:10.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.298+0000 7fa026800640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0200d3350 con 0x7fa0201603d0 2024-04-20T11:35:10.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.298+0000 7fa026800640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fa0200d3630 con 0x7fa0200b65c0 2024-04-20T11:35:10.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.298+0000 7fa026800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa0200d3910 con 0x7fa02015fcc0 2024-04-20T11:35:10.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.299+0000 7fa024a00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fa0201603d0 0x7fa0200b5eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:54354/0 (socket says 172.21.15.152:54354) 2024-04-20T11:35:10.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.299+0000 7fa024a00640 1 -- 172.21.15.152:0/3841845804 learned_addr learned my addr 172.21.15.152:0/3841845804 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:35:10.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.299+0000 7fa025e00640 1 --2- 172.21.15.152:0/3841845804 >> v2:172.21.15.152:3300/0 conn(0x7fa0200b65c0 0x7fa0200cf240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:10.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.299+0000 7fa025400640 1 --2- 172.21.15.152:0/3841845804 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa02015fcc0 0x7fa0200d4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:10.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.299+0000 7fa025e00640 1 -- 172.21.15.152:0/3841845804 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa02015fcc0 msgr2=0x7fa0200d4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:10.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.299+0000 7fa025e00640 1 --2- 172.21.15.152:0/3841845804 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa02015fcc0 0x7fa0200d4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:10.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.299+0000 7fa025e00640 1 -- 172.21.15.152:0/3841845804 >> v1:172.21.15.152:6789/0 conn(0x7fa0201603d0 legacy=0x7fa0200b5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:10.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.299+0000 7fa025e00640 1 -- 172.21.15.152:0/3841845804 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0200ced20 con 0x7fa0200b65c0 2024-04-20T11:35:10.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.300+0000 7fa025e00640 1 --2- 172.21.15.152:0/3841845804 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa0200b65c0 0x7fa0200cf240 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7fa0180097a0 tx=0x7fa01802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b836eeed5c30e05 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:10.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.300+0000 7fa01fe00640 1 -- 172.21.15.152:0/3841845804 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa018035020 con 0x7fa0200b65c0 2024-04-20T11:35:10.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.301+0000 7fa01fe00640 1 -- 172.21.15.152:0/3841845804 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa018031810 con 0x7fa0200b65c0 2024-04-20T11:35:10.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.301+0000 7fa01fe00640 1 -- 172.21.15.152:0/3841845804 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa018031be0 con 0x7fa0200b65c0 2024-04-20T11:35:10.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.301+0000 7fa026800640 1 -- 172.21.15.152:0/3841845804 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa0200b65c0 msgr2=0x7fa0200cf240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:10.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.301+0000 7fa026800640 1 --2- 172.21.15.152:0/3841845804 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa0200b65c0 0x7fa0200cf240 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7fa0180097a0 tx=0x7fa01802b8b0 comp rx=0 tx=0).stop 2024-04-20T11:35:10.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.301+0000 7fa026800640 1 -- 172.21.15.152:0/3841845804 shutdown_connections 2024-04-20T11:35:10.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.301+0000 7fa026800640 1 --2- 172.21.15.152:0/3841845804 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa0200b65c0 0x7fa0200cf240 unknown :-1 s=CLOSED pgs=1709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:10.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.301+0000 7fa026800640 1 --2- 172.21.15.152:0/3841845804 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa02015fcc0 0x7fa0200d4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:10.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.301+0000 7fa026800640 1 -- 172.21.15.152:0/3841845804 >> 172.21.15.152:0/3841845804 conn(0x7fa02008bfd0 msgr2=0x7fa0200d4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:10.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.301+0000 7fa026800640 1 -- 172.21.15.152:0/3841845804 shutdown_connections 2024-04-20T11:35:10.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.301+0000 7fa026800640 1 -- 172.21.15.152:0/3841845804 wait complete. 2024-04-20T11:35:10.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.303+0000 7fa026800640 1 Processor -- start 2024-04-20T11:35:10.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.303+0000 7fa026800640 1 -- start start 2024-04-20T11:35:10.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.304+0000 7fa026800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa0200b65c0 0x7fa02014b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:10.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.304+0000 7fa026800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa02015fcc0 0x7fa020158520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:10.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.304+0000 7fa026800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa0201603d0 0x7fa0200cb620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:10.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.304+0000 7fa026800640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fa0200b5600 con 0x7fa02015fcc0 2024-04-20T11:35:10.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.304+0000 7fa026800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa0200b5480 con 0x7fa0201603d0 2024-04-20T11:35:10.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.304+0000 7fa026800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa0200cf780 con 0x7fa0200b65c0 2024-04-20T11:35:10.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.304+0000 7fa024a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa02015fcc0 0x7fa020158520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:10.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.304+0000 7fa024a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa02015fcc0 0x7fa020158520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:56820/0 (socket says 172.21.15.152:56820) 2024-04-20T11:35:10.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.304+0000 7fa024a00640 1 -- 172.21.15.152:0/618991788 learned_addr learned my addr 172.21.15.152:0/618991788 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:10.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.305+0000 7fa025e00640 1 --2- 172.21.15.152:0/618991788 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa0201603d0 0x7fa0200cb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:10.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.305+0000 7fa024a00640 1 -- 172.21.15.152:0/618991788 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa0200b65c0 msgr2=0x7fa02014b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:10.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.305+0000 7fa024a00640 1 --2- 172.21.15.152:0/618991788 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa0200b65c0 0x7fa02014b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:10.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.305+0000 7fa024a00640 1 -- 172.21.15.152:0/618991788 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa0201603d0 msgr2=0x7fa0200cb620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:10.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.305+0000 7fa024a00640 1 --2- 172.21.15.152:0/618991788 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa0201603d0 0x7fa0200cb620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:10.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.305+0000 7fa024a00640 1 -- 172.21.15.152:0/618991788 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0200cc0c0 con 0x7fa02015fcc0 2024-04-20T11:35:10.308 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.305+0000 7fa024a00640 1 --2- 172.21.15.152:0/618991788 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa02015fcc0 0x7fa020158520 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7fa01400c730 tx=0x7fa01400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:10.308 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.305+0000 7fa01d600640 1 -- 172.21.15.152:0/618991788 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa014017070 con 0x7fa02015fcc0 2024-04-20T11:35:10.308 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.305+0000 7fa01d600640 1 -- 172.21.15.152:0/618991788 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa01400ce40 con 0x7fa02015fcc0 2024-04-20T11:35:10.308 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.305+0000 7fa01d600640 1 -- 172.21.15.152:0/618991788 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa01400a840 con 0x7fa02015fcc0 2024-04-20T11:35:10.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.305+0000 7fa026800640 1 -- 172.21.15.152:0/618991788 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0200cc3a0 con 0x7fa02015fcc0 2024-04-20T11:35:10.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.305+0000 7fa026800640 1 -- 172.21.15.152:0/618991788 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0200cd340 con 0x7fa02015fcc0 2024-04-20T11:35:10.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.306+0000 7fa026800640 1 -- 172.21.15.152:0/618991788 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0200cd540 con 0x7fa02015fcc0 2024-04-20T11:35:10.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.309+0000 7fa01d600640 1 -- 172.21.15.152:0/618991788 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 21) ==== 91550+0+0 (secure 0 0 0) 0x7fa01401a070 con 0x7fa02015fcc0 2024-04-20T11:35:10.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.310+0000 7fa01d600640 1 --2- 172.21.15.152:0/618991788 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fa00406ce90 0x7fa00406f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:10.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.310+0000 7fa01d600640 1 -- 172.21.15.152:0/618991788 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 4943+0+0 (secure 0 0 0) 0x7fa01408b6f0 con 0x7fa02015fcc0 2024-04-20T11:35:10.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.311+0000 7fa025400640 1 --2- 172.21.15.152:0/618991788 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fa00406ce90 0x7fa00406f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:10.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.312+0000 7fa025400640 1 --2- 172.21.15.152:0/618991788 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fa00406ce90 0x7fa00406f350 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa0100098f0 tx=0x7fa010008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:10.315 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.315+0000 7fa01d600640 1 -- 172.21.15.152:0/618991788 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fa0200cd540 con 0x7fa02015fcc0 2024-04-20T11:35:10.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.568+0000 7fa026800640 1 -- 172.21.15.152:0/618991788 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_to_v2"]} v 0) -- 0x7fa0200d0400 con 0x7fa02015fcc0 2024-04-20T11:35:10.675 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.675+0000 7fa01d600640 1 -- 172.21.15.152:0/618991788 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_to_v2"]}]=0 v437) ==== 148+0+0 (secure 0 0 0) 0x7fa014058ff0 con 0x7fa02015fcc0 2024-04-20T11:35:10.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.678+0000 7fa026800640 1 -- 172.21.15.152:0/618991788 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fa00406ce90 msgr2=0x7fa00406f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:10.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.678+0000 7fa026800640 1 --2- 172.21.15.152:0/618991788 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fa00406ce90 0x7fa00406f350 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa0100098f0 tx=0x7fa010008040 comp rx=0 tx=0).stop 2024-04-20T11:35:10.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.678+0000 7fa026800640 1 -- 172.21.15.152:0/618991788 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa02015fcc0 msgr2=0x7fa020158520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:10.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.678+0000 7fa026800640 1 --2- 172.21.15.152:0/618991788 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa02015fcc0 0x7fa020158520 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7fa01400c730 tx=0x7fa01400cbf0 comp rx=0 tx=0).stop 2024-04-20T11:35:10.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.678+0000 7fa026800640 1 -- 172.21.15.152:0/618991788 shutdown_connections 2024-04-20T11:35:10.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.678+0000 7fa026800640 1 --2- 172.21.15.152:0/618991788 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fa00406ce90 0x7fa00406f350 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:10.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.679+0000 7fa026800640 1 --2- 172.21.15.152:0/618991788 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa0201603d0 0x7fa0200cb620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:10.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.679+0000 7fa026800640 1 --2- 172.21.15.152:0/618991788 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa02015fcc0 0x7fa020158520 unknown :-1 s=CLOSED pgs=1710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:10.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.679+0000 7fa026800640 1 --2- 172.21.15.152:0/618991788 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa0200b65c0 0x7fa02014b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:10.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.679+0000 7fa026800640 1 -- 172.21.15.152:0/618991788 >> 172.21.15.152:0/618991788 conn(0x7fa02008bfd0 msgr2=0x7fa0201618c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:10.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.679+0000 7fa026800640 1 -- 172.21.15.152:0/618991788 shutdown_connections 2024-04-20T11:35:10.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.679+0000 7fa026800640 1 -- 172.21.15.152:0/618991788 wait complete. 2024-04-20T11:35:10.698 INFO:tasks.cephfs_test_runner:poor man's upgrade test -- theme continues... ... ok 2024-04-20T11:35:10.699 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_ops_on_nonexistent_vol (tasks.cephfs.test_volumes.TestMisc) 2024-04-20T11:35:10.699 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:35:10.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.894+0000 7f1803e00640 1 Processor -- start 2024-04-20T11:35:10.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.894+0000 7f1803e00640 1 -- start start 2024-04-20T11:35:10.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.896+0000 7f1803e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f1804147410 0x7f18040afa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:10.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.896+0000 7f1803e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f180406e720 0x7f1804072130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:10.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.896+0000 7f1803e00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18040aec90 con 0x7f1804071cd0 2024-04-20T11:35:10.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.896+0000 7f1803e00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f1804072670 con 0x7f1804147410 2024-04-20T11:35:10.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.896+0000 7f1803e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f18040aef70 con 0x7f180406e720 2024-04-20T11:35:10.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.897+0000 7f1803400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f180406e720 0x7f1804072130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:10.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.897+0000 7f1802a00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f1804147410 0x7f18040afa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:10.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.897+0000 7f1802000640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7f1804071cd0 0x7f180406e010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.152:42390/0 (socket says 172.21.15.152:42390) 2024-04-20T11:35:10.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.897+0000 7f1802000640 1 -- 172.21.15.152:0/2221671164 learned_addr learned my addr 172.21.15.152:0/2221671164 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:35:10.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.897+0000 7f1802a00640 1 -- 172.21.15.152:0/2221671164 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f180406e720 msgr2=0x7f1804072130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:10.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.897+0000 7f1802a00640 1 --2- 172.21.15.152:0/2221671164 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f180406e720 0x7f1804072130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:10.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.898+0000 7f1802a00640 1 -- 172.21.15.152:0/2221671164 >> v1:172.21.15.175:6789/0 conn(0x7f1804071cd0 legacy=0x7f180406e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:10.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.898+0000 7f1802a00640 1 -- 172.21.15.152:0/2221671164 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18040af250 con 0x7f1804147410 2024-04-20T11:35:10.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.899+0000 7f1802a00640 1 --2- 172.21.15.152:0/2221671164 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1804147410 0x7f18040afa50 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f17fc009d90 tx=0x7f17fc02bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6fd904ba6266975 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:10.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.900+0000 7f1801600640 1 -- 172.21.15.152:0/2221671164 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17fc039070 con 0x7f1804147410 2024-04-20T11:35:10.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.900+0000 7f1801600640 1 -- 172.21.15.152:0/2221671164 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17fc02cc50 con 0x7f1804147410 2024-04-20T11:35:10.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.900+0000 7f1801600640 1 -- 172.21.15.152:0/2221671164 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17fc034440 con 0x7f1804147410 2024-04-20T11:35:10.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.901+0000 7f1803e00640 1 -- 172.21.15.152:0/2221671164 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1804147410 msgr2=0x7f18040afa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:10.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.901+0000 7f1803e00640 1 --2- 172.21.15.152:0/2221671164 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1804147410 0x7f18040afa50 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f17fc009d90 tx=0x7f17fc02bea0 comp rx=0 tx=0).stop 2024-04-20T11:35:10.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.901+0000 7f1803e00640 1 -- 172.21.15.152:0/2221671164 shutdown_connections 2024-04-20T11:35:10.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.901+0000 7f1803e00640 1 --2- 172.21.15.152:0/2221671164 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f180406e720 0x7f1804072130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:10.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.901+0000 7f1803e00640 1 --2- 172.21.15.152:0/2221671164 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1804147410 0x7f18040afa50 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:10.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.901+0000 7f1803e00640 1 -- 172.21.15.152:0/2221671164 >> 172.21.15.152:0/2221671164 conn(0x7f1804067ae0 msgr2=0x7f180406b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:10.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.901+0000 7f1803e00640 1 -- 172.21.15.152:0/2221671164 shutdown_connections 2024-04-20T11:35:10.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.901+0000 7f1803e00640 1 -- 172.21.15.152:0/2221671164 wait complete. 2024-04-20T11:35:10.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.903+0000 7f1803e00640 1 Processor -- start 2024-04-20T11:35:10.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.903+0000 7f1803e00640 1 -- start start 2024-04-20T11:35:10.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.904+0000 7f1803e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f180406e720 0x7f180415d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:10.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.904+0000 7f1803e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1804071cd0 0x7f180413ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:10.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.904+0000 7f1803e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1804147410 0x7f1804160040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:10.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.904+0000 7f1803e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f18040b3390 con 0x7f1804147410 2024-04-20T11:35:10.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.904+0000 7f1803e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f18040ae980 con 0x7f1804071cd0 2024-04-20T11:35:10.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.904+0000 7f1803e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f18040b3510 con 0x7f180406e720 2024-04-20T11:35:10.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.905+0000 7f1803400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1804147410 0x7f1804160040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:10.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.905+0000 7f1803400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1804147410 0x7f1804160040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:56824/0 (socket says 172.21.15.152:56824) 2024-04-20T11:35:10.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.905+0000 7f1803400640 1 -- 172.21.15.152:0/230124336 learned_addr learned my addr 172.21.15.152:0/230124336 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:10.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.905+0000 7f1802000640 1 --2- 172.21.15.152:0/230124336 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1804071cd0 0x7f180413ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:10.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.905+0000 7f1802a00640 1 --2- 172.21.15.152:0/230124336 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f180406e720 0x7f180415d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:10.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.905+0000 7f1803400640 1 -- 172.21.15.152:0/230124336 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f180406e720 msgr2=0x7f180415d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:10.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.905+0000 7f1803400640 1 --2- 172.21.15.152:0/230124336 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f180406e720 0x7f180415d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:10.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.905+0000 7f1803400640 1 -- 172.21.15.152:0/230124336 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1804071cd0 msgr2=0x7f180413ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:10.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.905+0000 7f1803400640 1 --2- 172.21.15.152:0/230124336 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1804071cd0 0x7f180413ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:10.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.905+0000 7f1803400640 1 -- 172.21.15.152:0/230124336 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f180413a3a0 con 0x7f1804147410 2024-04-20T11:35:10.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.905+0000 7f1802000640 1 --2- 172.21.15.152:0/230124336 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1804071cd0 0x7f180413ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:35:10.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.905+0000 7f1803400640 1 --2- 172.21.15.152:0/230124336 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1804147410 0x7f1804160040 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f17f400e930 tx=0x7f17f400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:10.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.906+0000 7f17faa00640 1 -- 172.21.15.152:0/230124336 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f4017070 con 0x7f1804147410 2024-04-20T11:35:10.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.906+0000 7f17faa00640 1 -- 172.21.15.152:0/230124336 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17f400c980 con 0x7f1804147410 2024-04-20T11:35:10.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.906+0000 7f1803e00640 1 -- 172.21.15.152:0/230124336 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f180413a680 con 0x7f1804147410 2024-04-20T11:35:10.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.906+0000 7f17faa00640 1 -- 172.21.15.152:0/230124336 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f400cd70 con 0x7f1804147410 2024-04-20T11:35:10.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.906+0000 7f1803e00640 1 -- 172.21.15.152:0/230124336 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f180413ea70 con 0x7f1804147410 2024-04-20T11:35:10.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.907+0000 7f1803e00640 1 -- 172.21.15.152:0/230124336 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1804072f90 con 0x7f1804147410 2024-04-20T11:35:10.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.908+0000 7f17faa00640 1 -- 172.21.15.152:0/230124336 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 21) ==== 91550+0+0 (secure 0 0 0) 0x7f17f401a070 con 0x7f1804147410 2024-04-20T11:35:10.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.909+0000 7f17faa00640 1 --2- 172.21.15.152:0/230124336 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f17e006cec0 0x7f17e006f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:10.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.909+0000 7f17faa00640 1 -- 172.21.15.152:0/230124336 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 4943+0+0 (secure 0 0 0) 0x7f17f408c650 con 0x7f1804147410 2024-04-20T11:35:10.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.910+0000 7f1802a00640 1 --2- 172.21.15.152:0/230124336 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f17e006cec0 0x7f17e006f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:10.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.911+0000 7f1802a00640 1 --2- 172.21.15.152:0/230124336 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f17e006cec0 0x7f17e006f380 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f17fc036000 tx=0x7f17fc030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:10.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:10.914+0000 7f17faa00640 1 -- 172.21.15.152:0/230124336 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f1804072f90 con 0x7f1804147410 2024-04-20T11:35:11.118 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.118+0000 7f1803e00640 1 -- 172.21.15.152:0/230124336 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f180413ec70 con 0x7f1804147410 2024-04-20T11:35:11.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.120+0000 7f17faa00640 1 -- 172.21.15.152:0/230124336 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v123) ==== 74+0+13092 (secure 0 0 0) 0x7f180413ec70 con 0x7f1804147410 2024-04-20T11:35:11.121 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:35:11.121 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":123,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:35:09.650961+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.152:6835/2477488828":"2024-04-21T11:35:06.631957+0000","172.21.15.152:6833/2477488828":"2024-04-21T11:35:06.631957+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:35:11.123 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.122+0000 7f1803e00640 1 -- 172.21.15.152:0/230124336 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f17e006cec0 msgr2=0x7f17e006f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:11.123 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.122+0000 7f1803e00640 1 --2- 172.21.15.152:0/230124336 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f17e006cec0 0x7f17e006f380 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f17fc036000 tx=0x7f17fc030000 comp rx=0 tx=0).stop 2024-04-20T11:35:11.123 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.122+0000 7f1803e00640 1 -- 172.21.15.152:0/230124336 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1804147410 msgr2=0x7f1804160040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:11.123 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.122+0000 7f1803e00640 1 --2- 172.21.15.152:0/230124336 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1804147410 0x7f1804160040 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f17f400e930 tx=0x7f17f400edf0 comp rx=0 tx=0).stop 2024-04-20T11:35:11.124 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.123+0000 7f1803e00640 1 -- 172.21.15.152:0/230124336 shutdown_connections 2024-04-20T11:35:11.124 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.123+0000 7f1803e00640 1 --2- 172.21.15.152:0/230124336 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f17e006cec0 0x7f17e006f380 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:11.124 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.123+0000 7f1803e00640 1 --2- 172.21.15.152:0/230124336 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1804147410 0x7f1804160040 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:11.124 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.123+0000 7f1803e00640 1 --2- 172.21.15.152:0/230124336 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1804071cd0 0x7f180413ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:11.124 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.123+0000 7f1803e00640 1 --2- 172.21.15.152:0/230124336 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f180406e720 0x7f180415d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:11.124 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.123+0000 7f1803e00640 1 -- 172.21.15.152:0/230124336 >> 172.21.15.152:0/230124336 conn(0x7f1804067ae0 msgr2=0x7f180406f050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:11.124 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.123+0000 7f1803e00640 1 -- 172.21.15.152:0/230124336 shutdown_connections 2024-04-20T11:35:11.124 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.123+0000 7f1803e00640 1 -- 172.21.15.152:0/230124336 wait complete. 2024-04-20T11:35:11.142 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T11:26:47.246571+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T11:35:11.142 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T11:35:11.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.345+0000 7fc738a00640 1 Processor -- start 2024-04-20T11:35:11.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.346+0000 7fc738a00640 1 -- start start 2024-04-20T11:35:11.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.347+0000 7fc738a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc734156c00 0x7fc734160a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:11.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.347+0000 7fc738a00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fc73406c4b0 0x7fc73415cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:11.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.347+0000 7fc738a00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc734160fd0 con 0x7fc73415ca40 2024-04-20T11:35:11.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.347+0000 7fc738a00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fc73415d5c0 con 0x7fc73406c4b0 2024-04-20T11:35:11.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.347+0000 7fc738a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc73415d8a0 con 0x7fc734156c00 2024-04-20T11:35:11.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.347+0000 7fc729e00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fc73415ca40 0x7fc73406bda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:54362/0 (socket says 172.21.15.152:54362) 2024-04-20T11:35:11.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.348+0000 7fc729e00640 1 -- 172.21.15.152:0/529882065 learned_addr learned my addr 172.21.15.152:0/529882065 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:35:11.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.348+0000 7fc733200640 1 --2- 172.21.15.152:0/529882065 >> v2:172.21.15.175:3300/0 conn(0x7fc73406c4b0 0x7fc73415cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:11.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.348+0000 7fc732800640 1 --2- 172.21.15.152:0/529882065 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc734156c00 0x7fc734160a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:11.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.348+0000 7fc731e00640 1 -- 172.21.15.152:0/529882065 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4004222664 0 0) 0x7fc734160fd0 con 0x7fc73415ca40 2024-04-20T11:35:11.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.348+0000 7fc731e00640 1 -- 172.21.15.152:0/529882065 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc718003520 con 0x7fc73415ca40 2024-04-20T11:35:11.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.348+0000 7fc731e00640 1 -- 172.21.15.152:0/529882065 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4248062658 0 0) 0x7fc718003520 con 0x7fc73415ca40 2024-04-20T11:35:11.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.348+0000 7fc731e00640 1 -- 172.21.15.152:0/529882065 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc734156c00 msgr2=0x7fc734160a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:11.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.348+0000 7fc731e00640 1 --2- 172.21.15.152:0/529882065 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc734156c00 0x7fc734160a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:11.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.348+0000 7fc731e00640 1 -- 172.21.15.152:0/529882065 >> v2:172.21.15.175:3300/0 conn(0x7fc73406c4b0 msgr2=0x7fc73415cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:11.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.348+0000 7fc731e00640 1 --2- 172.21.15.152:0/529882065 >> v2:172.21.15.175:3300/0 conn(0x7fc73406c4b0 0x7fc73415cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:11.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.348+0000 7fc731e00640 1 -- 172.21.15.152:0/529882065 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc734160290 con 0x7fc73415ca40 2024-04-20T11:35:11.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.348+0000 7fc731e00640 1 -- 172.21.15.152:0/529882065 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7fc71c001690 con 0x7fc73415ca40 2024-04-20T11:35:11.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.348+0000 7fc731e00640 1 -- 172.21.15.152:0/529882065 >> v1:172.21.15.152:6789/0 conn(0x7fc73415ca40 legacy=0x7fc73406bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:11.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.349+0000 7fc731e00640 1 --2- 172.21.15.152:0/529882065 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc7180039b0 0x7fc718003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:11.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.349+0000 7fc731e00640 1 --2- 172.21.15.152:0/529882065 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc718004800 0x7fc718004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:11.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.349+0000 7fc731e00640 1 --2- 172.21.15.152:0/529882065 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc734156c00 0x7fc7180051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:11.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.349+0000 7fc731e00640 1 -- 172.21.15.152:0/529882065 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fc73415d5c0 con 0x7fc718004800 2024-04-20T11:35:11.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.349+0000 7fc731e00640 1 -- 172.21.15.152:0/529882065 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc73415d8a0 con 0x7fc7180039b0 2024-04-20T11:35:11.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.349+0000 7fc731e00640 1 -- 172.21.15.152:0/529882065 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc71800b9c0 con 0x7fc734156c00 2024-04-20T11:35:11.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.349+0000 7fc729e00640 1 --2- 172.21.15.152:0/529882065 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc718004800 0x7fc718004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:11.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.349+0000 7fc729e00640 1 -- 172.21.15.152:0/529882065 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc734156c00 msgr2=0x7fc7180051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:11.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.349+0000 7fc729e00640 1 --2- 172.21.15.152:0/529882065 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc734156c00 0x7fc7180051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:11.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.349+0000 7fc729e00640 1 -- 172.21.15.152:0/529882065 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc7180039b0 msgr2=0x7fc718003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:11.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.349+0000 7fc729e00640 1 --2- 172.21.15.152:0/529882065 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc7180039b0 0x7fc718003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:11.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.349+0000 7fc729e00640 1 -- 172.21.15.152:0/529882065 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc718003520 con 0x7fc718004800 2024-04-20T11:35:11.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.350+0000 7fc729e00640 1 --2- 172.21.15.152:0/529882065 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc718004800 0x7fc718004c80 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7fc7341611a0 tx=0x7fc718005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1f8b92918a9be2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:11.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.350+0000 7fc731e00640 1 -- 172.21.15.152:0/529882065 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc71c007ca0 con 0x7fc718004800 2024-04-20T11:35:11.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.351+0000 7fc731e00640 1 -- 172.21.15.152:0/529882065 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc71c030070 con 0x7fc718004800 2024-04-20T11:35:11.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.351+0000 7fc731e00640 1 -- 172.21.15.152:0/529882065 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc71c005d10 con 0x7fc718004800 2024-04-20T11:35:11.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.351+0000 7fc738a00640 1 -- 172.21.15.152:0/529882065 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc718004800 msgr2=0x7fc718004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:11.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.351+0000 7fc738a00640 1 --2- 172.21.15.152:0/529882065 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc718004800 0x7fc718004c80 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7fc7341611a0 tx=0x7fc718005830 comp rx=0 tx=0).stop 2024-04-20T11:35:11.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.351+0000 7fc733200640 1 -- 172.21.15.152:0/529882065 reap_dead start 2024-04-20T11:35:11.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.351+0000 7fc738a00640 1 -- 172.21.15.152:0/529882065 shutdown_connections 2024-04-20T11:35:11.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.351+0000 7fc738a00640 1 -- 172.21.15.152:0/529882065 >> 172.21.15.152:0/529882065 conn(0x7fc7340b21e0 msgr2=0x7fc7340b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:11.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.351+0000 7fc738a00640 1 -- 172.21.15.152:0/529882065 shutdown_connections 2024-04-20T11:35:11.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.351+0000 7fc738a00640 1 -- 172.21.15.152:0/529882065 wait complete. 2024-04-20T11:35:11.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.353+0000 7fc738a00640 1 Processor -- start 2024-04-20T11:35:11.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.353+0000 7fc738a00640 1 -- start start 2024-04-20T11:35:11.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.354+0000 7fc738a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc718004800 0x7fc7340992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:11.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.354+0000 7fc738a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc73414e6c0 0x7fc7340b0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:11.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.354+0000 7fc738a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc73414eb40 0x7fc7340b0970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:11.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.354+0000 7fc738a00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fc734160c10 con 0x7fc73414e6c0 2024-04-20T11:35:11.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.354+0000 7fc738a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc734160a90 con 0x7fc73414eb40 2024-04-20T11:35:11.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.354+0000 7fc738a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc734160d90 con 0x7fc718004800 2024-04-20T11:35:11.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.354+0000 7fc729e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc73414e6c0 0x7fc7340b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:11.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.354+0000 7fc729e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc73414e6c0 0x7fc7340b0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:56848/0 (socket says 172.21.15.152:56848) 2024-04-20T11:35:11.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.354+0000 7fc729e00640 1 -- 172.21.15.152:0/1967229477 learned_addr learned my addr 172.21.15.152:0/1967229477 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:11.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.355+0000 7fc732800640 1 --2- 172.21.15.152:0/1967229477 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc718004800 0x7fc7340992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:11.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.355+0000 7fc729e00640 1 -- 172.21.15.152:0/1967229477 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc718004800 msgr2=0x7fc7340992d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:11.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.355+0000 7fc729e00640 1 --2- 172.21.15.152:0/1967229477 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc718004800 0x7fc7340992d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:11.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.355+0000 7fc729e00640 1 -- 172.21.15.152:0/1967229477 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc73414eb40 msgr2=0x7fc7340b0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:11.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.355+0000 7fc729e00640 1 --2- 172.21.15.152:0/1967229477 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc73414eb40 0x7fc7340b0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:11.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.355+0000 7fc729e00640 1 -- 172.21.15.152:0/1967229477 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc73414d900 con 0x7fc73414e6c0 2024-04-20T11:35:11.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.355+0000 7fc729e00640 1 --2- 172.21.15.152:0/1967229477 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc73414e6c0 0x7fc7340b0430 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7fc71c02c4e0 tx=0x7fc734160460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:11.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.355+0000 7fc72b400640 1 -- 172.21.15.152:0/1967229477 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc71c004bf0 con 0x7fc73414e6c0 2024-04-20T11:35:11.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.355+0000 7fc72b400640 1 -- 172.21.15.152:0/1967229477 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc71c004d90 con 0x7fc73414e6c0 2024-04-20T11:35:11.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.355+0000 7fc72b400640 1 -- 172.21.15.152:0/1967229477 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc71c007290 con 0x7fc73414e6c0 2024-04-20T11:35:11.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.355+0000 7fc738a00640 1 -- 172.21.15.152:0/1967229477 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc73414dbe0 con 0x7fc73414e6c0 2024-04-20T11:35:11.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.355+0000 7fc738a00640 1 -- 172.21.15.152:0/1967229477 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc73414e1a0 con 0x7fc73414e6c0 2024-04-20T11:35:11.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.356+0000 7fc738a00640 1 -- 172.21.15.152:0/1967229477 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc734075e60 con 0x7fc73414e6c0 2024-04-20T11:35:11.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.357+0000 7fc72b400640 1 -- 172.21.15.152:0/1967229477 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 21) ==== 91550+0+0 (secure 0 0 0) 0x7fc71c03f070 con 0x7fc73414e6c0 2024-04-20T11:35:11.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.358+0000 7fc72b400640 1 --2- 172.21.15.152:0/1967229477 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fc71006ce90 0x7fc71006f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:11.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.358+0000 7fc72b400640 1 -- 172.21.15.152:0/1967229477 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 4943+0+0 (secure 0 0 0) 0x7fc71c0ad6d0 con 0x7fc73414e6c0 2024-04-20T11:35:11.359 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.359+0000 7fc732800640 1 --2- 172.21.15.152:0/1967229477 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fc71006ce90 0x7fc71006f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:11.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.360+0000 7fc732800640 1 --2- 172.21.15.152:0/1967229477 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fc71006ce90 0x7fc71006f350 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fc724007230 tx=0x7fc72400b040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:11.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.363+0000 7fc72b400640 1 -- 172.21.15.152:0/1967229477 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fc71c048e20 con 0x7fc73414e6c0 2024-04-20T11:35:11.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.560+0000 7fc738a00640 1 -- 172.21.15.152:0/1967229477 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc73406f390 con 0x7fc73414e6c0 2024-04-20T11:35:11.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.561+0000 7fc72b400640 1 -- 172.21.15.152:0/1967229477 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v123) ==== 93+0+10 (secure 0 0 0) 0x7fc71c07b720 con 0x7fc73414e6c0 2024-04-20T11:35:11.562 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 1 2024-04-20T11:35:11.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.564+0000 7fc738a00640 1 -- 172.21.15.152:0/1967229477 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fc71006ce90 msgr2=0x7fc71006f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:11.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.564+0000 7fc738a00640 1 --2- 172.21.15.152:0/1967229477 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fc71006ce90 0x7fc71006f350 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fc724007230 tx=0x7fc72400b040 comp rx=0 tx=0).stop 2024-04-20T11:35:11.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.564+0000 7fc738a00640 1 -- 172.21.15.152:0/1967229477 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc73414e6c0 msgr2=0x7fc7340b0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:11.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.564+0000 7fc738a00640 1 --2- 172.21.15.152:0/1967229477 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc73414e6c0 0x7fc7340b0430 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7fc71c02c4e0 tx=0x7fc734160460 comp rx=0 tx=0).stop 2024-04-20T11:35:11.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.564+0000 7fc738a00640 1 -- 172.21.15.152:0/1967229477 shutdown_connections 2024-04-20T11:35:11.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.564+0000 7fc738a00640 1 --2- 172.21.15.152:0/1967229477 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fc71006ce90 0x7fc71006f350 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:11.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.564+0000 7fc738a00640 1 --2- 172.21.15.152:0/1967229477 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc73414eb40 0x7fc7340b0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:11.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.564+0000 7fc738a00640 1 --2- 172.21.15.152:0/1967229477 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc73414e6c0 0x7fc7340b0430 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:11.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.564+0000 7fc738a00640 1 --2- 172.21.15.152:0/1967229477 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc718004800 0x7fc7340992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:11.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.564+0000 7fc738a00640 1 -- 172.21.15.152:0/1967229477 >> 172.21.15.152:0/1967229477 conn(0x7fc7340b21e0 msgr2=0x7fc73415df10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:11.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.565+0000 7fc738a00640 1 -- 172.21.15.152:0/1967229477 shutdown_connections 2024-04-20T11:35:11.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.565+0000 7fc738a00640 1 -- 172.21.15.152:0/1967229477 wait complete. 2024-04-20T11:35:11.584 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_ops_on_nonexistent_vol' 2024-04-20T11:35:11.772 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.772+0000 7f7cde800640 1 Processor -- start 2024-04-20T11:35:11.773 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.772+0000 7f7cde800640 1 -- start start 2024-04-20T11:35:11.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.774+0000 7f7cde800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f7cd80726f0 0x7f7cd80cd580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:11.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.774+0000 7f7cde800640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7cd8146340 con 0x7f7cd80cd0d0 2024-04-20T11:35:11.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.774+0000 7f7cde800640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7cd8072d50 con 0x7f7cd8145f60 2024-04-20T11:35:11.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.774+0000 7f7cde800640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f7cd806e9f0 con 0x7f7cd80726f0 2024-04-20T11:35:11.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.774+0000 7f7cd7e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f7cd80726f0 0x7f7cd80cd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:11.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.774+0000 7f7cd6a00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f7cd80cd0d0 0x7f7cd8071fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:54368/0 (socket says 172.21.15.152:54368) 2024-04-20T11:35:11.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.774+0000 7f7cd6a00640 1 -- 172.21.15.152:0/212788821 learned_addr learned my addr 172.21.15.152:0/212788821 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:35:11.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.775+0000 7f7cd7e00640 1 -- 172.21.15.152:0/212788821 >> v1:172.21.15.175:6789/0 conn(0x7f7cd8145f60 legacy=0x7f7cd806ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:11.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.775+0000 7f7cd7e00640 1 -- 172.21.15.152:0/212788821 >> v1:172.21.15.152:6789/0 conn(0x7f7cd80cd0d0 legacy=0x7f7cd8071fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:11.776 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.775+0000 7f7cd7e00640 1 -- 172.21.15.152:0/212788821 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cd806e0c0 con 0x7f7cd80726f0 2024-04-20T11:35:11.776 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.776+0000 7f7cd7e00640 1 --2- 172.21.15.152:0/212788821 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7cd80726f0 0x7f7cd80cd580 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f7cd00097a0 tx=0x7f7cd002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf3086b8b8fabfd8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:11.776 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.776+0000 7f7cd6000640 1 -- 172.21.15.152:0/212788821 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cd0035020 con 0x7f7cd80726f0 2024-04-20T11:35:11.777 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.776+0000 7f7cd6000640 1 -- 172.21.15.152:0/212788821 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cd0031830 con 0x7f7cd80726f0 2024-04-20T11:35:11.777 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.776+0000 7f7cd6000640 1 -- 172.21.15.152:0/212788821 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cd0031c00 con 0x7f7cd80726f0 2024-04-20T11:35:11.777 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.777+0000 7f7cde800640 1 -- 172.21.15.152:0/212788821 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7cd80726f0 msgr2=0x7f7cd80cd580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:11.777 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.777+0000 7f7cde800640 1 --2- 172.21.15.152:0/212788821 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7cd80726f0 0x7f7cd80cd580 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f7cd00097a0 tx=0x7f7cd002b8d0 comp rx=0 tx=0).stop 2024-04-20T11:35:11.778 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.777+0000 7f7cde800640 1 -- 172.21.15.152:0/212788821 shutdown_connections 2024-04-20T11:35:11.778 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.777+0000 7f7cde800640 1 --2- 172.21.15.152:0/212788821 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7cd80726f0 0x7f7cd80cd580 unknown :-1 s=CLOSED pgs=1716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:11.778 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.777+0000 7f7cde800640 1 -- 172.21.15.152:0/212788821 >> 172.21.15.152:0/212788821 conn(0x7f7cd8067cf0 msgr2=0x7f7cd806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:11.778 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.777+0000 7f7cde800640 1 -- 172.21.15.152:0/212788821 shutdown_connections 2024-04-20T11:35:11.778 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.777+0000 7f7cde800640 1 -- 172.21.15.152:0/212788821 wait complete. 2024-04-20T11:35:11.779 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.779+0000 7f7cde800640 1 Processor -- start 2024-04-20T11:35:11.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.779+0000 7f7cde800640 1 -- start start 2024-04-20T11:35:11.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.780+0000 7f7cde800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7cd80726f0 0x7f7cd8159000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:11.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.780+0000 7f7cde800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7cd80cd0d0 0x7f7cd80b4820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:11.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.780+0000 7f7cde800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7cd8145f60 0x7f7cd80c7c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:11.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.781+0000 7f7cde800640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f7cd80cf290 con 0x7f7cd8145f60 2024-04-20T11:35:11.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.781+0000 7f7cde800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7cd806e680 con 0x7f7cd80cd0d0 2024-04-20T11:35:11.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.781+0000 7f7cde800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7cd80cf410 con 0x7f7cd80726f0 2024-04-20T11:35:11.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.781+0000 7f7cd7e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7cd8145f60 0x7f7cd80c7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:11.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.781+0000 7f7cd7e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7cd8145f60 0x7f7cd80c7c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:56880/0 (socket says 172.21.15.152:56880) 2024-04-20T11:35:11.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.781+0000 7f7cd7e00640 1 -- 172.21.15.152:0/300638292 learned_addr learned my addr 172.21.15.152:0/300638292 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:11.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.781+0000 7f7cd7400640 1 --2- 172.21.15.152:0/300638292 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7cd80726f0 0x7f7cd8159000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:11.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.781+0000 7f7cd6a00640 1 --2- 172.21.15.152:0/300638292 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7cd80cd0d0 0x7f7cd80b4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:11.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.781+0000 7f7cd7e00640 1 -- 172.21.15.152:0/300638292 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7cd80726f0 msgr2=0x7f7cd8159000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:11.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.781+0000 7f7cd7e00640 1 --2- 172.21.15.152:0/300638292 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7cd80726f0 0x7f7cd8159000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:11.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.781+0000 7f7cd7e00640 1 -- 172.21.15.152:0/300638292 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7cd80cd0d0 msgr2=0x7f7cd80b4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:11.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.781+0000 7f7cd7e00640 1 --2- 172.21.15.152:0/300638292 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7cd80cd0d0 0x7f7cd80b4820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:11.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.781+0000 7f7cd7e00640 1 -- 172.21.15.152:0/300638292 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cd80c59e0 con 0x7f7cd8145f60 2024-04-20T11:35:11.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.781+0000 7f7cd7400640 1 --2- 172.21.15.152:0/300638292 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7cd80726f0 0x7f7cd8159000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:35:11.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.782+0000 7f7cd7e00640 1 --2- 172.21.15.152:0/300638292 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7cd8145f60 0x7f7cd80c7c40 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f7cd00098c0 tx=0x7f7cd00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:11.784 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.782+0000 7f7ccf400640 1 -- 172.21.15.152:0/300638292 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cd0041070 con 0x7f7cd8145f60 2024-04-20T11:35:11.784 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.782+0000 7f7ccf400640 1 -- 172.21.15.152:0/300638292 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cd003a8d0 con 0x7f7cd8145f60 2024-04-20T11:35:11.784 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.782+0000 7f7ccf400640 1 -- 172.21.15.152:0/300638292 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cd0035040 con 0x7f7cd8145f60 2024-04-20T11:35:11.784 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.782+0000 7f7cde800640 1 -- 172.21.15.152:0/300638292 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cd80c5cc0 con 0x7f7cd8145f60 2024-04-20T11:35:11.784 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.782+0000 7f7cde800640 1 -- 172.21.15.152:0/300638292 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cd80c6280 con 0x7f7cd8145f60 2024-04-20T11:35:11.784 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.783+0000 7f7cde800640 1 -- 172.21.15.152:0/300638292 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cd80689f0 con 0x7f7cd8145f60 2024-04-20T11:35:11.785 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.784+0000 7f7ccf400640 1 -- 172.21.15.152:0/300638292 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 21) ==== 91550+0+0 (secure 0 0 0) 0x7f7cd0038020 con 0x7f7cd8145f60 2024-04-20T11:35:11.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.785+0000 7f7ccf400640 1 --2- 172.21.15.152:0/300638292 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f7cb806ce90 0x7f7cb806f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:11.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.786+0000 7f7ccf400640 1 -- 172.21.15.152:0/300638292 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 4943+0+0 (secure 0 0 0) 0x7f7cd00ade10 con 0x7f7cd8145f60 2024-04-20T11:35:11.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.786+0000 7f7cd7400640 1 --2- 172.21.15.152:0/300638292 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f7cb806ce90 0x7f7cb806f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:11.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.788+0000 7f7cd7400640 1 --2- 172.21.15.152:0/300638292 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f7cb806ce90 0x7f7cb806f350 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f7cd815c660 tx=0x7f7cc80073c0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:11.790 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:11.790+0000 7f7ccf400640 1 -- 172.21.15.152:0/300638292 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f7cd007bc90 con 0x7f7cd8145f60 2024-04-20T11:35:12.063 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:12.062+0000 7f7cde800640 1 -- 172.21.15.152:0/300638292 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_ops_on_nonexistent_vol"]} v 0) -- 0x7f7cd80d0a50 con 0x7f7cd8145f60 2024-04-20T11:35:12.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:12.690+0000 7f7ccf400640 1 -- 172.21.15.152:0/300638292 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_ops_on_nonexistent_vol"]}]=0 v439) ==== 154+0+0 (secure 0 0 0) 0x7f7cd007b590 con 0x7f7cd8145f60 2024-04-20T11:35:12.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:12.693+0000 7f7cde800640 1 -- 172.21.15.152:0/300638292 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f7cb806ce90 msgr2=0x7f7cb806f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:12.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:12.693+0000 7f7cde800640 1 --2- 172.21.15.152:0/300638292 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f7cb806ce90 0x7f7cb806f350 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f7cd815c660 tx=0x7f7cc80073c0 comp rx=0 tx=0).stop 2024-04-20T11:35:12.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:12.693+0000 7f7cde800640 1 -- 172.21.15.152:0/300638292 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7cd8145f60 msgr2=0x7f7cd80c7c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:12.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:12.693+0000 7f7cde800640 1 --2- 172.21.15.152:0/300638292 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7cd8145f60 0x7f7cd80c7c40 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f7cd00098c0 tx=0x7f7cd00046c0 comp rx=0 tx=0).stop 2024-04-20T11:35:12.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:12.694+0000 7f7cde800640 1 -- 172.21.15.152:0/300638292 shutdown_connections 2024-04-20T11:35:12.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:12.694+0000 7f7cde800640 1 --2- 172.21.15.152:0/300638292 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f7cb806ce90 0x7f7cb806f350 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:12.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:12.694+0000 7f7cde800640 1 --2- 172.21.15.152:0/300638292 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7cd8145f60 0x7f7cd80c7c40 unknown :-1 s=CLOSED pgs=1717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:12.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:12.694+0000 7f7cde800640 1 --2- 172.21.15.152:0/300638292 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7cd80cd0d0 0x7f7cd80b4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:12.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:12.694+0000 7f7cde800640 1 --2- 172.21.15.152:0/300638292 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7cd80726f0 0x7f7cd8159000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:12.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:12.694+0000 7f7cde800640 1 -- 172.21.15.152:0/300638292 >> 172.21.15.152:0/300638292 conn(0x7f7cd8067cf0 msgr2=0x7f7cd8144c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:12.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:12.694+0000 7f7cde800640 1 -- 172.21.15.152:0/300638292 shutdown_connections 2024-04-20T11:35:12.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:12.694+0000 7f7cde800640 1 -- 172.21.15.152:0/300638292 wait complete. 2024-04-20T11:35:12.718 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-20T11:35:12.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:12.926+0000 7fc449e00640 1 Processor -- start 2024-04-20T11:35:12.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:12.926+0000 7fc449e00640 1 -- start start 2024-04-20T11:35:12.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:12.927+0000 7fc449e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc4440afba0 0x7fc444162440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:12.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:12.928+0000 7fc449e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fc4440b3fa0 0x7fc4440ae110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:12.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:12.928+0000 7fc449e00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4440adc30 con 0x7fc44415e5d0 2024-04-20T11:35:12.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:12.928+0000 7fc449e00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fc444162a70 con 0x7fc4440b3fa0 2024-04-20T11:35:12.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:12.928+0000 7fc449e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc444162d50 con 0x7fc4440afba0 2024-04-20T11:35:12.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:12.928+0000 7fc442a00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fc44415e5d0 0x7fc4440af490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:54384/0 (socket says 172.21.15.152:54384) 2024-04-20T11:35:12.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:12.928+0000 7fc442a00640 1 -- 172.21.15.152:0/2250592744 learned_addr learned my addr 172.21.15.152:0/2250592744 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:35:12.931 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:12.928+0000 7fc442000640 1 -- 172.21.15.152:0/2250592744 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2985449735 0 0) 0x7fc4440adc30 con 0x7fc44415e5d0 2024-04-20T11:35:12.931 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:12.928+0000 7fc442000640 1 -- 172.21.15.152:0/2250592744 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc428003520 con 0x7fc44415e5d0 2024-04-20T11:35:12.931 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:12.928+0000 7fc443400640 1 --2- 172.21.15.152:0/2250592744 >> v2:172.21.15.175:3300/0 conn(0x7fc4440b3fa0 0x7fc4440ae110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:12.931 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:12.928+0000 7fc43a000640 1 --2- 172.21.15.152:0/2250592744 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc4440afba0 0x7fc444162440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:12.931 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:12.929+0000 7fc442000640 1 -- 172.21.15.152:0/2250592744 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2422324003 0 0) 0x7fc428003520 con 0x7fc44415e5d0 2024-04-20T11:35:12.931 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:12.929+0000 7fc442000640 1 -- 172.21.15.152:0/2250592744 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc4440afba0 msgr2=0x7fc444162440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:12.931 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:12.929+0000 7fc442000640 1 --2- 172.21.15.152:0/2250592744 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc4440afba0 0x7fc444162440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:12.931 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:12.929+0000 7fc442000640 1 -- 172.21.15.152:0/2250592744 >> v2:172.21.15.175:3300/0 conn(0x7fc4440b3fa0 msgr2=0x7fc4440ae110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:12.931 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:12.929+0000 7fc442000640 1 --2- 172.21.15.152:0/2250592744 >> v2:172.21.15.175:3300/0 conn(0x7fc4440b3fa0 0x7fc4440ae110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:12.931 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:12.929+0000 7fc442000640 1 -- 172.21.15.152:0/2250592744 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc444163030 con 0x7fc44415e5d0 2024-04-20T11:35:12.931 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:12.929+0000 7fc442000640 1 -- 172.21.15.152:0/2250592744 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7fc434001670 con 0x7fc44415e5d0 2024-04-20T11:35:12.931 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:12.929+0000 7fc442000640 1 -- 172.21.15.152:0/2250592744 >> v1:172.21.15.152:6789/0 conn(0x7fc44415e5d0 legacy=0x7fc4440af490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:12.931 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:12.929+0000 7fc442000640 1 --2- 172.21.15.152:0/2250592744 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc4440afba0 0x7fc4280039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:12.932 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:12.929+0000 7fc442000640 1 --2- 172.21.15.152:0/2250592744 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc428004030 0x7fc428004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:12.932 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:12.929+0000 7fc442000640 1 --2- 172.21.15.152:0/2250592744 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc428004d40 0x7fc4280051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:12.932 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:12.929+0000 7fc442000640 1 -- 172.21.15.152:0/2250592744 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fc444162a70 con 0x7fc428004d40 2024-04-20T11:35:12.932 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:12.929+0000 7fc442000640 1 -- 172.21.15.152:0/2250592744 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc444162d50 con 0x7fc428004030 2024-04-20T11:35:12.932 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:12.929+0000 7fc442000640 1 -- 172.21.15.152:0/2250592744 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc42800bb20 con 0x7fc4440afba0 2024-04-20T11:35:12.932 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:12.929+0000 7fc443400640 1 --2- 172.21.15.152:0/2250592744 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc428004d40 0x7fc4280051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:12.932 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:12.930+0000 7fc443400640 1 -- 172.21.15.152:0/2250592744 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc4440afba0 msgr2=0x7fc4280039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:12.932 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:12.930+0000 7fc442a00640 1 --2- 172.21.15.152:0/2250592744 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc4440afba0 0x7fc4280039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:12.932 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:12.930+0000 7fc443400640 1 --2- 172.21.15.152:0/2250592744 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc4440afba0 0x7fc4280039b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:12.932 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:12.930+0000 7fc443400640 1 -- 172.21.15.152:0/2250592744 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc428004030 msgr2=0x7fc428004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:12.932 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:12.930+0000 7fc43a000640 1 --2- 172.21.15.152:0/2250592744 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc428004030 0x7fc428004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:12.932 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:12.930+0000 7fc443400640 1 --2- 172.21.15.152:0/2250592744 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc428004030 0x7fc428004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:12.932 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:12.930+0000 7fc443400640 1 -- 172.21.15.152:0/2250592744 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc428003520 con 0x7fc428004d40 2024-04-20T11:35:12.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:12.931+0000 7fc443400640 1 --2- 172.21.15.152:0/2250592744 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc428004d40 0x7fc4280051c0 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7fc43c007360 tx=0x7fc428005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9fe334037b852a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:12.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:12.931+0000 7fc442000640 1 -- 172.21.15.152:0/2250592744 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc43c039070 con 0x7fc428004d40 2024-04-20T11:35:12.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:12.931+0000 7fc442000640 1 -- 172.21.15.152:0/2250592744 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc43c02ccf0 con 0x7fc428004d40 2024-04-20T11:35:12.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:12.931+0000 7fc442000640 1 -- 172.21.15.152:0/2250592744 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc43c009be0 con 0x7fc428004d40 2024-04-20T11:35:12.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:12.932+0000 7fc449e00640 1 -- 172.21.15.152:0/2250592744 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc428004d40 msgr2=0x7fc4280051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:12.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:12.932+0000 7fc449e00640 1 --2- 172.21.15.152:0/2250592744 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc428004d40 0x7fc4280051c0 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7fc43c007360 tx=0x7fc428005830 comp rx=0 tx=0).stop 2024-04-20T11:35:12.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:12.932+0000 7fc443400640 1 -- 172.21.15.152:0/2250592744 reap_dead start 2024-04-20T11:35:12.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:12.932+0000 7fc449e00640 1 -- 172.21.15.152:0/2250592744 shutdown_connections 2024-04-20T11:35:12.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:12.932+0000 7fc449e00640 1 -- 172.21.15.152:0/2250592744 >> 172.21.15.152:0/2250592744 conn(0x7fc44406b980 msgr2=0x7fc44406bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:12.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:12.932+0000 7fc449e00640 1 -- 172.21.15.152:0/2250592744 shutdown_connections 2024-04-20T11:35:12.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:12.932+0000 7fc449e00640 1 -- 172.21.15.152:0/2250592744 wait complete. 2024-04-20T11:35:12.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:12.933+0000 7fc449e00640 1 Processor -- start 2024-04-20T11:35:12.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:12.934+0000 7fc449e00640 1 -- start start 2024-04-20T11:35:12.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:12.935+0000 7fc449e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc428004d40 0x7fc444155960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:12.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:12.935+0000 7fc449e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc44415e9b0 0x7fc444147d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:12.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:12.935+0000 7fc449e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc444160510 0x7fc444160990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:12.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:12.935+0000 7fc449e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fc44406e0a0 con 0x7fc44415e9b0 2024-04-20T11:35:12.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:12.935+0000 7fc449e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc444163310 con 0x7fc428004d40 2024-04-20T11:35:12.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:12.935+0000 7fc449e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc44406e220 con 0x7fc444160510 2024-04-20T11:35:12.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:12.935+0000 7fc43a000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc44415e9b0 0x7fc444147d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:12.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:12.935+0000 7fc43a000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc44415e9b0 0x7fc444147d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:56912/0 (socket says 172.21.15.152:56912) 2024-04-20T11:35:12.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:12.935+0000 7fc43a000640 1 -- 172.21.15.152:0/348862324 learned_addr learned my addr 172.21.15.152:0/348862324 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:12.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:12.935+0000 7fc43a000640 1 -- 172.21.15.152:0/348862324 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc444160510 msgr2=0x7fc444160990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:12.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:12.935+0000 7fc43a000640 1 --2- 172.21.15.152:0/348862324 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc444160510 0x7fc444160990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:12.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:12.935+0000 7fc43a000640 1 -- 172.21.15.152:0/348862324 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc428004d40 msgr2=0x7fc444155960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:12.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:12.935+0000 7fc43a000640 1 --2- 172.21.15.152:0/348862324 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc428004d40 0x7fc444155960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:12.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:12.935+0000 7fc43a000640 1 -- 172.21.15.152:0/348862324 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4441482a0 con 0x7fc44415e9b0 2024-04-20T11:35:12.938 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:12.936+0000 7fc43a000640 1 --2- 172.21.15.152:0/348862324 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc44415e9b0 0x7fc444147d60 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7fc42c00e840 tx=0x7fc42c00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:12.938 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:12.936+0000 7fc43be00640 1 -- 172.21.15.152:0/348862324 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc42c017070 con 0x7fc44415e9b0 2024-04-20T11:35:12.938 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:12.936+0000 7fc43be00640 1 -- 172.21.15.152:0/348862324 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc42c00c850 con 0x7fc44415e9b0 2024-04-20T11:35:12.938 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:12.936+0000 7fc43be00640 1 -- 172.21.15.152:0/348862324 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc42c00f040 con 0x7fc44415e9b0 2024-04-20T11:35:12.938 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:12.936+0000 7fc449e00640 1 -- 172.21.15.152:0/348862324 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4441611e0 con 0x7fc44415e9b0 2024-04-20T11:35:12.938 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:12.936+0000 7fc449e00640 1 -- 172.21.15.152:0/348862324 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4441617a0 con 0x7fc44415e9b0 2024-04-20T11:35:12.938 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:12.937+0000 7fc449e00640 1 -- 172.21.15.152:0/348862324 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4440689f0 con 0x7fc44415e9b0 2024-04-20T11:35:12.939 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:12.939+0000 7fc43be00640 1 -- 172.21.15.152:0/348862324 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 21) ==== 91550+0+0 (secure 0 0 0) 0x7fc42c01a070 con 0x7fc44415e9b0 2024-04-20T11:35:12.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:12.940+0000 7fc43be00640 1 --2- 172.21.15.152:0/348862324 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fc42006ce90 0x7fc42006f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:12.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:12.940+0000 7fc43be00640 1 -- 172.21.15.152:0/348862324 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 4943+0+0 (secure 0 0 0) 0x7fc42c08be60 con 0x7fc44415e9b0 2024-04-20T11:35:12.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:12.941+0000 7fc442a00640 1 --2- 172.21.15.152:0/348862324 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fc42006ce90 0x7fc42006f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:12.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:12.942+0000 7fc442a00640 1 --2- 172.21.15.152:0/348862324 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fc42006ce90 0x7fc42006f350 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc434001eb0 tx=0x7fc434009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:12.945 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:12.945+0000 7fc43be00640 1 -- 172.21.15.152:0/348862324 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fc42c059c10 con 0x7fc44415e9b0 2024-04-20T11:35:13.146 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:13.146+0000 7fc449e00640 1 -- 172.21.15.152:0/348862324 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fc44415dbd0 con 0x7fc44415e9b0 2024-04-20T11:35:13.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:13.151+0000 7fc43be00640 1 -- 172.21.15.152:0/348862324 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v24)=0 v24) ==== 122+0+0 (secure 0 0 0) 0x7fc42c059510 con 0x7fc44415e9b0 2024-04-20T11:35:13.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:13.153+0000 7fc449e00640 1 -- 172.21.15.152:0/348862324 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fc42006ce90 msgr2=0x7fc42006f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:13.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:13.153+0000 7fc449e00640 1 --2- 172.21.15.152:0/348862324 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fc42006ce90 0x7fc42006f350 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc434001eb0 tx=0x7fc434009040 comp rx=0 tx=0).stop 2024-04-20T11:35:13.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:13.153+0000 7fc449e00640 1 -- 172.21.15.152:0/348862324 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc44415e9b0 msgr2=0x7fc444147d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:13.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:13.153+0000 7fc449e00640 1 --2- 172.21.15.152:0/348862324 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc44415e9b0 0x7fc444147d60 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7fc42c00e840 tx=0x7fc42c00ed00 comp rx=0 tx=0).stop 2024-04-20T11:35:13.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:13.153+0000 7fc449e00640 1 -- 172.21.15.152:0/348862324 shutdown_connections 2024-04-20T11:35:13.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:13.153+0000 7fc449e00640 1 --2- 172.21.15.152:0/348862324 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fc42006ce90 0x7fc42006f350 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:13.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:13.153+0000 7fc449e00640 1 --2- 172.21.15.152:0/348862324 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc444160510 0x7fc444160990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:13.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:13.153+0000 7fc449e00640 1 --2- 172.21.15.152:0/348862324 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc44415e9b0 0x7fc444147d60 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:13.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:13.153+0000 7fc449e00640 1 --2- 172.21.15.152:0/348862324 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc428004d40 0x7fc444155960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:13.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:13.153+0000 7fc449e00640 1 -- 172.21.15.152:0/348862324 >> 172.21.15.152:0/348862324 conn(0x7fc44406b980 msgr2=0x7fc44406f690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:13.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:13.154+0000 7fc449e00640 1 -- 172.21.15.152:0/348862324 shutdown_connections 2024-04-20T11:35:13.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:13.154+0000 7fc449e00640 1 -- 172.21.15.152:0/348862324 wait complete. 2024-04-20T11:35:13.173 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-20T11:35:13.173 DEBUG:teuthology.orchestra.run.smithi152:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-20T11:35:13.275 DEBUG:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-20T11:35:13.388 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-20T11:35:13.388 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-20T11:35:13.389 DEBUG:teuthology.orchestra.run.smithi152:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:35:13.448 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-20T11:35:13.448 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T11:35:13.448 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:35:13.505 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-20T11:35:13.505 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-20T11:35:13.505 DEBUG:teuthology.orchestra.run.smithi152:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:35:13.566 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-20T11:35:13.566 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T11:35:13.566 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:35:13.626 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-20T11:35:13.627 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-20T11:35:13.627 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:35:13.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:13.834+0000 7fdc52e00640 1 Processor -- start 2024-04-20T11:35:13.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:13.835+0000 7fdc52e00640 1 -- start start 2024-04-20T11:35:13.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:13.836+0000 7fdc52e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdc4c0726f0 0x7fdc4c0cd580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:13.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:13.836+0000 7fdc52e00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc4c146340 con 0x7fdc4c0cd0d0 2024-04-20T11:35:13.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:13.836+0000 7fdc52e00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc4c072d50 con 0x7fdc4c145f60 2024-04-20T11:35:13.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:13.836+0000 7fdc52e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fdc4c06e9f0 con 0x7fdc4c0726f0 2024-04-20T11:35:13.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:13.836+0000 7fdc43400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fdc4c0cd0d0 0x7fdc4c071fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:54396/0 (socket says 172.21.15.152:54396) 2024-04-20T11:35:13.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:13.837+0000 7fdc43400640 1 -- 172.21.15.152:0/1859194083 learned_addr learned my addr 172.21.15.152:0/1859194083 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:35:13.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:13.837+0000 7fdc50e00640 1 --2- 172.21.15.152:0/1859194083 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdc4c0726f0 0x7fdc4c0cd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:13.843 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:13.837+0000 7fdc4b400640 1 -- 172.21.15.152:0/1859194083 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 990146787 0 0) 0x7fdc4c146340 con 0x7fdc4c0cd0d0 2024-04-20T11:35:13.843 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:13.837+0000 7fdc4b400640 1 -- 172.21.15.152:0/1859194083 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdc30003520 con 0x7fdc4c0cd0d0 2024-04-20T11:35:13.843 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:13.837+0000 7fdc4b400640 1 -- 172.21.15.152:0/1859194083 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1514119601 0 0) 0x7fdc30003520 con 0x7fdc4c0cd0d0 2024-04-20T11:35:13.843 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:13.837+0000 7fdc4b400640 1 -- 172.21.15.152:0/1859194083 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdc4c0726f0 msgr2=0x7fdc4c0cd580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:13.843 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:13.837+0000 7fdc4b400640 1 --2- 172.21.15.152:0/1859194083 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdc4c0726f0 0x7fdc4c0cd580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:13.843 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:13.837+0000 7fdc4b400640 1 -- 172.21.15.152:0/1859194083 >> v1:172.21.15.175:6789/0 conn(0x7fdc4c145f60 legacy=0x7fdc4c06ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:13.843 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:13.837+0000 7fdc4b400640 1 -- 172.21.15.152:0/1859194083 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc4c06e0c0 con 0x7fdc4c0cd0d0 2024-04-20T11:35:13.844 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:13.837+0000 7fdc4b400640 1 -- 172.21.15.152:0/1859194083 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7fdc34001690 con 0x7fdc4c0cd0d0 2024-04-20T11:35:13.844 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:13.837+0000 7fdc4b400640 1 -- 172.21.15.152:0/1859194083 >> v1:172.21.15.152:6789/0 conn(0x7fdc4c0cd0d0 legacy=0x7fdc4c071fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:13.844 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:13.838+0000 7fdc4b400640 1 --2- 172.21.15.152:0/1859194083 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdc4c0726f0 0x7fdc30003a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:13.844 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:13.838+0000 7fdc4b400640 1 --2- 172.21.15.152:0/1859194083 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc30003fd0 0x7fdc30004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:13.844 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:13.838+0000 7fdc4b400640 1 --2- 172.21.15.152:0/1859194083 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdc30004d40 0x7fdc300051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:13.844 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:13.838+0000 7fdc4b400640 1 -- 172.21.15.152:0/1859194083 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fdc4c06e9f0 con 0x7fdc30004d40 2024-04-20T11:35:13.844 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:13.838+0000 7fdc4b400640 1 -- 172.21.15.152:0/1859194083 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdc3000bb20 con 0x7fdc30003fd0 2024-04-20T11:35:13.845 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:13.838+0000 7fdc4b400640 1 -- 172.21.15.152:0/1859194083 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fdc3000bd20 con 0x7fdc4c0726f0 2024-04-20T11:35:13.845 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:13.838+0000 7fdc50e00640 1 --2- 172.21.15.152:0/1859194083 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdc30004d40 0x7fdc300051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:13.845 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:13.838+0000 7fdc43400640 1 --2- 172.21.15.152:0/1859194083 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc30003fd0 0x7fdc30004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:13.845 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:13.838+0000 7fdc4be00640 1 --2- 172.21.15.152:0/1859194083 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdc4c0726f0 0x7fdc30003a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:13.845 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:13.838+0000 7fdc50e00640 1 -- 172.21.15.152:0/1859194083 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdc4c0726f0 msgr2=0x7fdc30003a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:13.845 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:13.838+0000 7fdc50e00640 1 --2- 172.21.15.152:0/1859194083 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdc4c0726f0 0x7fdc30003a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:13.845 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:13.838+0000 7fdc50e00640 1 -- 172.21.15.152:0/1859194083 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc30003fd0 msgr2=0x7fdc30004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:13.846 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:13.838+0000 7fdc50e00640 1 --2- 172.21.15.152:0/1859194083 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc30003fd0 0x7fdc30004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:13.846 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:13.838+0000 7fdc50e00640 1 -- 172.21.15.152:0/1859194083 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc30003520 con 0x7fdc30004d40 2024-04-20T11:35:13.846 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:13.838+0000 7fdc43400640 1 --2- 172.21.15.152:0/1859194083 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc30003fd0 0x7fdc30004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:35:13.846 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:13.838+0000 7fdc4be00640 1 --2- 172.21.15.152:0/1859194083 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdc4c0726f0 0x7fdc30003a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:35:13.846 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:13.840+0000 7fdc50e00640 1 --2- 172.21.15.152:0/1859194083 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdc30004d40 0x7fdc300051c0 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7fdc44009fc0 tx=0x7fdc30005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9e4ea58c019c28a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:13.846 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:13.840+0000 7fdc4b400640 1 -- 172.21.15.152:0/1859194083 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc44039070 con 0x7fdc30004d40 2024-04-20T11:35:13.846 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:13.840+0000 7fdc4b400640 1 -- 172.21.15.152:0/1859194083 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc4402cc20 con 0x7fdc30004d40 2024-04-20T11:35:13.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:13.840+0000 7fdc4b400640 1 -- 172.21.15.152:0/1859194083 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc44009c10 con 0x7fdc30004d40 2024-04-20T11:35:13.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:13.840+0000 7fdc52e00640 1 -- 172.21.15.152:0/1859194083 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdc30004d40 msgr2=0x7fdc300051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:13.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:13.840+0000 7fdc52e00640 1 --2- 172.21.15.152:0/1859194083 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdc30004d40 0x7fdc300051c0 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7fdc44009fc0 tx=0x7fdc30005830 comp rx=0 tx=0).stop 2024-04-20T11:35:13.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:13.840+0000 7fdc50e00640 1 -- 172.21.15.152:0/1859194083 reap_dead start 2024-04-20T11:35:13.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:13.840+0000 7fdc52e00640 1 -- 172.21.15.152:0/1859194083 shutdown_connections 2024-04-20T11:35:13.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:13.840+0000 7fdc52e00640 1 -- 172.21.15.152:0/1859194083 >> 172.21.15.152:0/1859194083 conn(0x7fdc4c067cf0 msgr2=0x7fdc4c06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:13.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:13.841+0000 7fdc52e00640 1 -- 172.21.15.152:0/1859194083 shutdown_connections 2024-04-20T11:35:13.848 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:13.841+0000 7fdc52e00640 1 -- 172.21.15.152:0/1859194083 wait complete. 2024-04-20T11:35:13.852 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:13.842+0000 7fdc52e00640 1 Processor -- start 2024-04-20T11:35:13.852 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:13.842+0000 7fdc52e00640 1 -- start start 2024-04-20T11:35:13.852 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:13.843+0000 7fdc52e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc4c159910 0x7fdc4c1e81e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:13.852 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:13.843+0000 7fdc52e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdc4c1e8720 0x7fdc4c15f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:13.852 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:13.843+0000 7fdc52e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdc4c159cf0 0x7fdc4c15fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:13.852 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:13.843+0000 7fdc52e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fdc4c0cd580 con 0x7fdc4c159cf0 2024-04-20T11:35:13.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:13.843+0000 7fdc52e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdc4c06e680 con 0x7fdc4c159910 2024-04-20T11:35:13.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:13.843+0000 7fdc52e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fdc4c0cd700 con 0x7fdc4c1e8720 2024-04-20T11:35:13.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:13.844+0000 7fdc50e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdc4c159cf0 0x7fdc4c15fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:13.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:13.844+0000 7fdc50e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdc4c159cf0 0x7fdc4c15fab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:56930/0 (socket says 172.21.15.152:56930) 2024-04-20T11:35:13.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:13.844+0000 7fdc50e00640 1 -- 172.21.15.152:0/1282270448 learned_addr learned my addr 172.21.15.152:0/1282270448 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:13.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:13.844+0000 7fdc4be00640 1 --2- 172.21.15.152:0/1282270448 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc4c159910 0x7fdc4c1e81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:13.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:13.844+0000 7fdc50e00640 1 -- 172.21.15.152:0/1282270448 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdc4c1e8720 msgr2=0x7fdc4c15f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:13.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:13.844+0000 7fdc50e00640 1 --2- 172.21.15.152:0/1282270448 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdc4c1e8720 0x7fdc4c15f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:13.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:13.844+0000 7fdc50e00640 1 -- 172.21.15.152:0/1282270448 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc4c159910 msgr2=0x7fdc4c1e81e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:13.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:13.844+0000 7fdc50e00640 1 --2- 172.21.15.152:0/1282270448 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc4c159910 0x7fdc4c1e81e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:13.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:13.844+0000 7fdc50e00640 1 -- 172.21.15.152:0/1282270448 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc4c13f5c0 con 0x7fdc4c159cf0 2024-04-20T11:35:13.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:13.844+0000 7fdc50e00640 1 --2- 172.21.15.152:0/1282270448 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdc4c159cf0 0x7fdc4c15fab0 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7fdc44002230 tx=0x7fdc44009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:13.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:13.845+0000 7fdc48c00640 1 -- 172.21.15.152:0/1282270448 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc44039030 con 0x7fdc4c159cf0 2024-04-20T11:35:13.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:13.845+0000 7fdc48c00640 1 -- 172.21.15.152:0/1282270448 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc44043070 con 0x7fdc4c159cf0 2024-04-20T11:35:13.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:13.845+0000 7fdc48c00640 1 -- 172.21.15.152:0/1282270448 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc44009c10 con 0x7fdc4c159cf0 2024-04-20T11:35:13.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:13.845+0000 7fdc52e00640 1 -- 172.21.15.152:0/1282270448 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc4c1e7140 con 0x7fdc4c159cf0 2024-04-20T11:35:13.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:13.845+0000 7fdc52e00640 1 -- 172.21.15.152:0/1282270448 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc4c1e7700 con 0x7fdc4c159cf0 2024-04-20T11:35:13.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:13.846+0000 7fdc52e00640 1 -- 172.21.15.152:0/1282270448 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc4c06bdd0 con 0x7fdc4c159cf0 2024-04-20T11:35:13.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:13.847+0000 7fdc48c00640 1 -- 172.21.15.152:0/1282270448 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 21) ==== 91550+0+0 (secure 0 0 0) 0x7fdc44005d10 con 0x7fdc4c159cf0 2024-04-20T11:35:13.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:13.848+0000 7fdc48c00640 1 --2- 172.21.15.152:0/1282270448 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fdc2806cec0 0x7fdc2806f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:13.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:13.848+0000 7fdc48c00640 1 -- 172.21.15.152:0/1282270448 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 4943+0+0 (secure 0 0 0) 0x7fdc440ae360 con 0x7fdc4c159cf0 2024-04-20T11:35:13.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:13.849+0000 7fdc4be00640 1 --2- 172.21.15.152:0/1282270448 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fdc2806cec0 0x7fdc2806f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:13.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:13.850+0000 7fdc4be00640 1 --2- 172.21.15.152:0/1282270448 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fdc2806cec0 0x7fdc2806f380 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c0015b0 tx=0x7fdc3c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:13.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:13.853+0000 7fdc48c00640 1 -- 172.21.15.152:0/1282270448 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fdc4c06bdd0 con 0x7fdc4c159cf0 2024-04-20T11:35:14.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.093+0000 7fdc52e00640 1 -- 172.21.15.152:0/1282270448 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdc4c0689f0 con 0x7fdc4c159cf0 2024-04-20T11:35:14.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.094+0000 7fdc48c00640 1 -- 172.21.15.152:0/1282270448 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) ==== 94+0+3875 (secure 0 0 0) 0x7fdc4407c270 con 0x7fdc4c159cf0 2024-04-20T11:35:14.095 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:35:14.095 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":79,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5891,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/508638056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6833","nonce":508638056},{"type":"v1","addr":"172.21.15.175:6835","nonce":508638056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":6448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6834/3447077466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3447077466},{"type":"v1","addr":"172.21.15.175:6834","nonce":3447077466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":8409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6834/751442105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":751442105},{"type":"v1","addr":"172.21.15.152:6834","nonce":751442105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":8832,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/3352993292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6833","nonce":3352993292},{"type":"v1","addr":"172.21.15.152:6835","nonce":3352993292}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[]} 2024-04-20T11:35:14.095 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 79 2024-04-20T11:35:14.098 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.097+0000 7fdc52e00640 1 -- 172.21.15.152:0/1282270448 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fdc2806cec0 msgr2=0x7fdc2806f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:14.098 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.097+0000 7fdc52e00640 1 --2- 172.21.15.152:0/1282270448 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fdc2806cec0 0x7fdc2806f380 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c0015b0 tx=0x7fdc3c009040 comp rx=0 tx=0).stop 2024-04-20T11:35:14.098 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.097+0000 7fdc52e00640 1 -- 172.21.15.152:0/1282270448 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdc4c159cf0 msgr2=0x7fdc4c15fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:14.098 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.097+0000 7fdc52e00640 1 --2- 172.21.15.152:0/1282270448 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdc4c159cf0 0x7fdc4c15fab0 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7fdc44002230 tx=0x7fdc44009750 comp rx=0 tx=0).stop 2024-04-20T11:35:14.098 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.097+0000 7fdc52e00640 1 -- 172.21.15.152:0/1282270448 shutdown_connections 2024-04-20T11:35:14.098 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.097+0000 7fdc52e00640 1 --2- 172.21.15.152:0/1282270448 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fdc2806cec0 0x7fdc2806f380 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:14.098 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.097+0000 7fdc52e00640 1 --2- 172.21.15.152:0/1282270448 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdc4c159cf0 0x7fdc4c15fab0 unknown :-1 s=CLOSED pgs=1723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:14.098 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.097+0000 7fdc52e00640 1 --2- 172.21.15.152:0/1282270448 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdc4c1e8720 0x7fdc4c15f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:14.098 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.097+0000 7fdc52e00640 1 --2- 172.21.15.152:0/1282270448 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc4c159910 0x7fdc4c1e81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:14.098 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.097+0000 7fdc52e00640 1 -- 172.21.15.152:0/1282270448 >> 172.21.15.152:0/1282270448 conn(0x7fdc4c067cf0 msgr2=0x7fdc4c0cf250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:14.098 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.097+0000 7fdc52e00640 1 -- 172.21.15.152:0/1282270448 shutdown_connections 2024-04-20T11:35:14.098 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.097+0000 7fdc52e00640 1 -- 172.21.15.152:0/1282270448 wait complete. 2024-04-20T11:35:14.118 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-20T11:35:14.118 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-20T11:35:14.118 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-20T11:35:14.118 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T11:35:14.119 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-20T11:35:14.119 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-20T11:35:14.119 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-20T11:35:14.119 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T11:35:14.119 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-20T11:35:14.119 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-20T11:35:14.120 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-20T11:35:14.120 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T11:35:14.120 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-20T11:35:14.120 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-20T11:35:14.120 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-20T11:35:14.120 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T11:35:14.136 INFO:tasks.ceph.mds.a:Stopped 2024-04-20T11:35:14.136 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-20T11:35:14.139 INFO:tasks.ceph.mds.c:Stopped 2024-04-20T11:35:14.139 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-20T11:35:14.140 INFO:tasks.ceph.mds.b:Stopped 2024-04-20T11:35:14.141 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-20T11:35:14.142 INFO:tasks.ceph.mds.d:Stopped 2024-04-20T11:35:14.142 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-20T11:35:14.186 INFO:tasks.ceph.mds.b:Started 2024-04-20T11:35:14.187 DEBUG:teuthology.parallel:result is None 2024-04-20T11:35:14.187 INFO:tasks.ceph.mds.a:Started 2024-04-20T11:35:14.188 DEBUG:teuthology.parallel:result is None 2024-04-20T11:35:14.193 INFO:tasks.ceph.mds.d:Started 2024-04-20T11:35:14.194 DEBUG:teuthology.parallel:result is None 2024-04-20T11:35:14.194 INFO:tasks.ceph.mds.c:Started 2024-04-20T11:35:14.195 DEBUG:teuthology.parallel:result is None 2024-04-20T11:35:14.195 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-20T11:35:14.341 INFO:tasks.ceph.mds.c.smithi152.stdout:starting mds.c at 2024-04-20T11:35:14.342 INFO:tasks.ceph.mds.a.smithi152.stdout:starting mds.a at 2024-04-20T11:35:14.342 INFO:tasks.ceph.mds.b.smithi175.stdout:starting mds.b at 2024-04-20T11:35:14.343 INFO:tasks.ceph.mds.d.smithi175.stdout:starting mds.d at 2024-04-20T11:35:14.377 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.377+0000 7fd61d400640 1 Processor -- start 2024-04-20T11:35:14.378 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.377+0000 7fd61d400640 1 -- start start 2024-04-20T11:35:14.378 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.378+0000 7fd61d400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fd618161d30 0x7fd6180d0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:14.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.378+0000 7fd61d400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd6180d4ec0 0x7fd6180d15e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:14.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.378+0000 7fd61d400640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6180d1b20 con 0x7fd6180d2b80 2024-04-20T11:35:14.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.378+0000 7fd61d400640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fd6180d0540 con 0x7fd618161d30 2024-04-20T11:35:14.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.378+0000 7fd61d400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd6180cfb70 con 0x7fd6180d4ec0 2024-04-20T11:35:14.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.378+0000 7fd617e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fd618161d30 0x7fd6180d0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:14.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.379+0000 7fd617e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fd618161d30 0x7fd6180d0000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:56970/0 (socket says 172.21.15.152:56970) 2024-04-20T11:35:14.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.379+0000 7fd617e00640 1 -- 172.21.15.152:0/2508555328 learned_addr learned my addr 172.21.15.152:0/2508555328 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:14.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.379+0000 7fd617400640 1 --2- 172.21.15.152:0/2508555328 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd6180d4ec0 0x7fd6180d15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:14.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.379+0000 7fd617e00640 1 -- 172.21.15.152:0/2508555328 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd6180d4ec0 msgr2=0x7fd6180d15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:14.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.379+0000 7fd617e00640 1 --2- 172.21.15.152:0/2508555328 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd6180d4ec0 0x7fd6180d15e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:14.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.379+0000 7fd617e00640 1 -- 172.21.15.152:0/2508555328 >> v1:172.21.15.175:6789/0 conn(0x7fd6180d2b80 legacy=0x7fd61806c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:14.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.379+0000 7fd617e00640 1 -- 172.21.15.152:0/2508555328 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6180d26a0 con 0x7fd618161d30 2024-04-20T11:35:14.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.380+0000 7fd617e00640 1 --2- 172.21.15.152:0/2508555328 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd618161d30 0x7fd6180d0000 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7fd608009870 tx=0x7fd60802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1eb62092935104a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:14.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.380+0000 7fd616a00640 1 -- 172.21.15.152:0/2508555328 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd608034070 con 0x7fd618161d30 2024-04-20T11:35:14.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.380+0000 7fd616a00640 1 -- 172.21.15.152:0/2508555328 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd608030840 con 0x7fd618161d30 2024-04-20T11:35:14.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.380+0000 7fd616a00640 1 -- 172.21.15.152:0/2508555328 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd608030c10 con 0x7fd618161d30 2024-04-20T11:35:14.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.381+0000 7fd61d400640 1 -- 172.21.15.152:0/2508555328 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd618161d30 msgr2=0x7fd6180d0000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:14.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.381+0000 7fd61d400640 1 --2- 172.21.15.152:0/2508555328 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd618161d30 0x7fd6180d0000 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7fd608009870 tx=0x7fd60802b980 comp rx=0 tx=0).stop 2024-04-20T11:35:14.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.381+0000 7fd61d400640 1 -- 172.21.15.152:0/2508555328 shutdown_connections 2024-04-20T11:35:14.383 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.381+0000 7fd61d400640 1 --2- 172.21.15.152:0/2508555328 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd6180d4ec0 0x7fd6180d15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:14.383 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.381+0000 7fd61d400640 1 --2- 172.21.15.152:0/2508555328 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd618161d30 0x7fd6180d0000 unknown :-1 s=CLOSED pgs=1731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:14.383 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.381+0000 7fd61d400640 1 -- 172.21.15.152:0/2508555328 >> 172.21.15.152:0/2508555328 conn(0x7fd61808bfd0 msgr2=0x7fd6180d45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:14.383 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.381+0000 7fd61d400640 1 -- 172.21.15.152:0/2508555328 shutdown_connections 2024-04-20T11:35:14.383 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.381+0000 7fd61d400640 1 -- 172.21.15.152:0/2508555328 wait complete. 2024-04-20T11:35:14.383 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.383+0000 7fd61d400640 1 Processor -- start 2024-04-20T11:35:14.384 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.383+0000 7fd61d400640 1 -- start start 2024-04-20T11:35:14.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.384+0000 7fd61d400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd6180d2b80 0x7fd6180b6810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:14.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.384+0000 7fd61d400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd6180d4ec0 0x7fd6180cb670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:14.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.384+0000 7fd61d400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd618161d30 0x7fd6180b1440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:14.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.384+0000 7fd61d400640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fd6180cf800 con 0x7fd618161d30 2024-04-20T11:35:14.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.384+0000 7fd61d400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd6180cf520 con 0x7fd6180d2b80 2024-04-20T11:35:14.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.384+0000 7fd61d400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd6180cf1b0 con 0x7fd6180d4ec0 2024-04-20T11:35:14.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.384+0000 7fd61ca00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd618161d30 0x7fd6180b1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:14.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.384+0000 7fd617e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd6180d2b80 0x7fd6180b6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:14.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.384+0000 7fd61ca00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd618161d30 0x7fd6180b1440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:56980/0 (socket says 172.21.15.152:56980) 2024-04-20T11:35:14.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.384+0000 7fd61ca00640 1 -- 172.21.15.152:0/3319684846 learned_addr learned my addr 172.21.15.152:0/3319684846 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:14.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.384+0000 7fd617400640 1 --2- 172.21.15.152:0/3319684846 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd6180d4ec0 0x7fd6180cb670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:14.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.385+0000 7fd617e00640 1 -- 172.21.15.152:0/3319684846 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd6180d4ec0 msgr2=0x7fd6180cb670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:14.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.385+0000 7fd617e00640 1 --2- 172.21.15.152:0/3319684846 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd6180d4ec0 0x7fd6180cb670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:14.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.385+0000 7fd617e00640 1 -- 172.21.15.152:0/3319684846 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd618161d30 msgr2=0x7fd6180b1440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:14.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.385+0000 7fd617e00640 1 --2- 172.21.15.152:0/3319684846 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd618161d30 0x7fd6180b1440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:14.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.385+0000 7fd617e00640 1 -- 172.21.15.152:0/3319684846 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6180cbbb0 con 0x7fd6180d2b80 2024-04-20T11:35:14.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.385+0000 7fd61ca00640 1 --2- 172.21.15.152:0/3319684846 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd618161d30 0x7fd6180b1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:35:14.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.385+0000 7fd617e00640 1 --2- 172.21.15.152:0/3319684846 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd6180d2b80 0x7fd6180b6810 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fd608033000 tx=0x7fd608004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:14.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.386+0000 7fd60fe00640 1 -- 172.21.15.152:0/3319684846 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd608048070 con 0x7fd6180d2b80 2024-04-20T11:35:14.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.386+0000 7fd60fe00640 1 -- 172.21.15.152:0/3319684846 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd608030640 con 0x7fd6180d2b80 2024-04-20T11:35:14.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.386+0000 7fd60fe00640 1 -- 172.21.15.152:0/3319684846 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd608034040 con 0x7fd6180d2b80 2024-04-20T11:35:14.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.386+0000 7fd61d400640 1 -- 172.21.15.152:0/3319684846 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6180cbe30 con 0x7fd6180d2b80 2024-04-20T11:35:14.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.386+0000 7fd61d400640 1 -- 172.21.15.152:0/3319684846 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6180cc3f0 con 0x7fd6180d2b80 2024-04-20T11:35:14.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.387+0000 7fd61d400640 1 -- 172.21.15.152:0/3319684846 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd618075e60 con 0x7fd6180d2b80 2024-04-20T11:35:14.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.389+0000 7fd60fe00640 1 -- 172.21.15.152:0/3319684846 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 21) ==== 91550+0+0 (secure 0 0 0) 0x7fd608037070 con 0x7fd6180d2b80 2024-04-20T11:35:14.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.390+0000 7fd60fe00640 1 --2- 172.21.15.152:0/3319684846 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd5f806ce90 0x7fd5f806f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:14.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.390+0000 7fd60fe00640 1 -- 172.21.15.152:0/3319684846 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 4943+0+0 (secure 0 0 0) 0x7fd6080ae0b0 con 0x7fd6180d2b80 2024-04-20T11:35:14.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.390+0000 7fd617400640 1 --2- 172.21.15.152:0/3319684846 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd5f806ce90 0x7fd5f806f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:14.391 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.391+0000 7fd617400640 1 --2- 172.21.15.152:0/3319684846 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd5f806ce90 0x7fd5f806f350 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd6180ba280 tx=0x7fd604008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:14.395 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.395+0000 7fd60fe00640 1 -- 172.21.15.152:0/3319684846 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fd6180cbbb0 con 0x7fd6180d2b80 2024-04-20T11:35:14.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.609+0000 7fd61d400640 1 -- 172.21.15.152:0/3319684846 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fd6180c7a20 con 0x7fd6180d2b80 2024-04-20T11:35:14.727 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.726+0000 7fd60fe00640 1 -- 172.21.15.152:0/3319684846 <== mon.1 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v125) ==== 97+0+0 (secure 0 0 0) 0x7fd60807b9c0 con 0x7fd6180d2b80 2024-04-20T11:35:14.727 INFO:teuthology.orchestra.run.smithi152.stderr: removed all blocklist entries 2024-04-20T11:35:14.731 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.730+0000 7fd61d400640 1 -- 172.21.15.152:0/3319684846 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd5f806ce90 msgr2=0x7fd5f806f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:14.731 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.730+0000 7fd61d400640 1 --2- 172.21.15.152:0/3319684846 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd5f806ce90 0x7fd5f806f350 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd6180ba280 tx=0x7fd604008040 comp rx=0 tx=0).stop 2024-04-20T11:35:14.731 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.730+0000 7fd61d400640 1 -- 172.21.15.152:0/3319684846 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd6180d2b80 msgr2=0x7fd6180b6810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:14.731 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.730+0000 7fd61d400640 1 --2- 172.21.15.152:0/3319684846 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd6180d2b80 0x7fd6180b6810 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fd608033000 tx=0x7fd608004730 comp rx=0 tx=0).stop 2024-04-20T11:35:14.731 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.730+0000 7fd61d400640 1 -- 172.21.15.152:0/3319684846 shutdown_connections 2024-04-20T11:35:14.731 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.730+0000 7fd61d400640 1 --2- 172.21.15.152:0/3319684846 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd5f806ce90 0x7fd5f806f350 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:14.731 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.730+0000 7fd61d400640 1 --2- 172.21.15.152:0/3319684846 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd618161d30 0x7fd6180b1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:14.731 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.730+0000 7fd61d400640 1 --2- 172.21.15.152:0/3319684846 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd6180d4ec0 0x7fd6180cb670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:14.731 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.730+0000 7fd61d400640 1 --2- 172.21.15.152:0/3319684846 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd6180d2b80 0x7fd6180b6810 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:14.731 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.730+0000 7fd61d400640 1 -- 172.21.15.152:0/3319684846 >> 172.21.15.152:0/3319684846 conn(0x7fd61808bfd0 msgr2=0x7fd6180c8250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:14.731 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.730+0000 7fd61d400640 1 -- 172.21.15.152:0/3319684846 shutdown_connections 2024-04-20T11:35:14.731 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.730+0000 7fd61d400640 1 -- 172.21.15.152:0/3319684846 wait complete. 2024-04-20T11:35:14.754 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-20T11:35:14.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.946+0000 7fc84aa00640 1 Processor -- start 2024-04-20T11:35:14.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.946+0000 7fc84aa00640 1 -- start start 2024-04-20T11:35:14.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.948+0000 7fc84aa00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fc84415e800 0x7fc8440d4f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:14.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.948+0000 7fc84aa00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc84415ebe0 0x7fc8440b2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:14.950 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.948+0000 7fc84aa00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fc8440b39f0 0x7fc8440b3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:14.950 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.948+0000 7fc84aa00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fc8440b2920 con 0x7fc84415e800 2024-04-20T11:35:14.950 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.948+0000 7fc84aa00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fc8440b1d10 con 0x7fc8440b39f0 2024-04-20T11:35:14.950 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.948+0000 7fc84aa00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc8440b1ff0 con 0x7fc84415ebe0 2024-04-20T11:35:14.950 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.948+0000 7fc849600640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fc84415e800 0x7fc8440d4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:14.950 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.948+0000 7fc849600640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fc84415e800 0x7fc8440d4f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:56990/0 (socket says 172.21.15.152:56990) 2024-04-20T11:35:14.950 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.948+0000 7fc849600640 1 -- 172.21.15.152:0/3832015545 learned_addr learned my addr 172.21.15.152:0/3832015545 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:14.950 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.949+0000 7fc849600640 1 -- 172.21.15.152:0/3832015545 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc84415ebe0 msgr2=0x7fc8440b2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:14.950 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.949+0000 7fc849600640 1 --2- 172.21.15.152:0/3832015545 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc84415ebe0 0x7fc8440b2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:14.951 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.949+0000 7fc849600640 1 -- 172.21.15.152:0/3832015545 >> v2:172.21.15.175:3300/0 conn(0x7fc8440b39f0 msgr2=0x7fc8440b3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:14.951 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.949+0000 7fc849600640 1 --2- 172.21.15.152:0/3832015545 >> v2:172.21.15.175:3300/0 conn(0x7fc8440b39f0 0x7fc8440b3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:14.951 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.949+0000 7fc849600640 1 -- 172.21.15.152:0/3832015545 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8440b22d0 con 0x7fc84415e800 2024-04-20T11:35:14.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.950+0000 7fc849600640 1 --2- 172.21.15.152:0/3832015545 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc84415e800 0x7fc8440d4f10 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7fc84408e260 tx=0x7fc83802b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32200538dfee6431 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:14.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.950+0000 7fc843e00640 1 -- 172.21.15.152:0/3832015545 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc838035020 con 0x7fc84415e800 2024-04-20T11:35:14.953 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.950+0000 7fc843e00640 1 -- 172.21.15.152:0/3832015545 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc83802d040 con 0x7fc84415e800 2024-04-20T11:35:14.953 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.950+0000 7fc843e00640 1 -- 172.21.15.152:0/3832015545 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc838031a80 con 0x7fc84415e800 2024-04-20T11:35:14.953 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.951+0000 7fc84aa00640 1 -- 172.21.15.152:0/3832015545 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc84415e800 msgr2=0x7fc8440d4f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:14.953 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.951+0000 7fc84aa00640 1 --2- 172.21.15.152:0/3832015545 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc84415e800 0x7fc8440d4f10 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7fc84408e260 tx=0x7fc83802b900 comp rx=0 tx=0).stop 2024-04-20T11:35:14.953 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.951+0000 7fc84aa00640 1 -- 172.21.15.152:0/3832015545 shutdown_connections 2024-04-20T11:35:14.953 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.951+0000 7fc84aa00640 1 --2- 172.21.15.152:0/3832015545 >> v2:172.21.15.175:3300/0 conn(0x7fc8440b39f0 0x7fc8440b3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:14.953 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.951+0000 7fc84aa00640 1 --2- 172.21.15.152:0/3832015545 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc84415ebe0 0x7fc8440b2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:14.953 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.951+0000 7fc84aa00640 1 --2- 172.21.15.152:0/3832015545 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc84415e800 0x7fc8440d4f10 unknown :-1 s=CLOSED pgs=1732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:14.953 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.951+0000 7fc84aa00640 1 -- 172.21.15.152:0/3832015545 >> 172.21.15.152:0/3832015545 conn(0x7fc84408bfd0 msgr2=0x7fc8440d45f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:14.954 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.951+0000 7fc84aa00640 1 -- 172.21.15.152:0/3832015545 shutdown_connections 2024-04-20T11:35:14.954 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.951+0000 7fc84aa00640 1 -- 172.21.15.152:0/3832015545 wait complete. 2024-04-20T11:35:14.956 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.953+0000 7fc84aa00640 1 Processor -- start 2024-04-20T11:35:14.956 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.953+0000 7fc84aa00640 1 -- start start 2024-04-20T11:35:14.956 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.953+0000 7fc84aa00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc8440b39f0 0x7fc8440ce130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:14.957 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.954+0000 7fc84aa00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc84415e800 0x7fc8440d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:14.957 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.954+0000 7fc84aa00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8440cc0a0 0x7fc8440d14d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:14.957 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.954+0000 7fc84aa00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fc8440b4520 con 0x7fc84415e800 2024-04-20T11:35:14.957 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.954+0000 7fc84aa00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc8440b43a0 con 0x7fc8440cc0a0 2024-04-20T11:35:14.957 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.954+0000 7fc84aa00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc8440d28b0 con 0x7fc8440b39f0 2024-04-20T11:35:14.957 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.954+0000 7fc848c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc84415e800 0x7fc8440d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:14.957 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.954+0000 7fc848c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc84415e800 0x7fc8440d0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:57004/0 (socket says 172.21.15.152:57004) 2024-04-20T11:35:14.957 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.954+0000 7fc848c00640 1 -- 172.21.15.152:0/508805840 learned_addr learned my addr 172.21.15.152:0/508805840 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:14.957 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.954+0000 7fc848c00640 1 -- 172.21.15.152:0/508805840 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc8440b39f0 msgr2=0x7fc8440ce130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:14.957 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.954+0000 7fc848c00640 1 --2- 172.21.15.152:0/508805840 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc8440b39f0 0x7fc8440ce130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:14.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.954+0000 7fc848c00640 1 -- 172.21.15.152:0/508805840 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8440cc0a0 msgr2=0x7fc8440d14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:14.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.954+0000 7fc848c00640 1 --2- 172.21.15.152:0/508805840 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8440cc0a0 0x7fc8440d14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:14.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.954+0000 7fc848c00640 1 -- 172.21.15.152:0/508805840 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8440cc650 con 0x7fc84415e800 2024-04-20T11:35:14.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.955+0000 7fc848c00640 1 --2- 172.21.15.152:0/508805840 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc84415e800 0x7fc8440d0f90 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7fc83000b410 tx=0x7fc83000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:14.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.955+0000 7fc841600640 1 -- 172.21.15.152:0/508805840 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc830015020 con 0x7fc84415e800 2024-04-20T11:35:14.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.955+0000 7fc841600640 1 -- 172.21.15.152:0/508805840 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc830011850 con 0x7fc84415e800 2024-04-20T11:35:14.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.955+0000 7fc841600640 1 -- 172.21.15.152:0/508805840 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc830011c40 con 0x7fc84415e800 2024-04-20T11:35:14.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.955+0000 7fc84aa00640 1 -- 172.21.15.152:0/508805840 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8440cc930 con 0x7fc84415e800 2024-04-20T11:35:14.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.955+0000 7fc84aa00640 1 -- 172.21.15.152:0/508805840 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc84415f3e0 con 0x7fc84415e800 2024-04-20T11:35:14.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.956+0000 7fc84aa00640 1 -- 172.21.15.152:0/508805840 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc84415f5e0 con 0x7fc84415e800 2024-04-20T11:35:14.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.957+0000 7fc841600640 1 -- 172.21.15.152:0/508805840 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 21) ==== 91550+0+0 (secure 0 0 0) 0x7fc830018020 con 0x7fc84415e800 2024-04-20T11:35:14.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.958+0000 7fc841600640 1 --2- 172.21.15.152:0/508805840 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fc82406ce90 0x7fc82406f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:14.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.958+0000 7fc841600640 1 -- 172.21.15.152:0/508805840 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 4857+0+0 (secure 0 0 0) 0x7fc83008c220 con 0x7fc84415e800 2024-04-20T11:35:14.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.959+0000 7fc849600640 1 --2- 172.21.15.152:0/508805840 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fc82406ce90 0x7fc82406f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:14.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.960+0000 7fc849600640 1 --2- 172.21.15.152:0/508805840 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fc82406ce90 0x7fc82406f350 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc838036000 tx=0x7fc838030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:14.963 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:14.963+0000 7fc841600640 1 -- 172.21.15.152:0/508805840 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fc84415f5e0 con 0x7fc84415e800 2024-04-20T11:35:15.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.231+0000 7fc84aa00640 1 -- 172.21.15.152:0/508805840 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fc844075e60 con 0x7fc84415e800 2024-04-20T11:35:15.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.232+0000 7fc841600640 1 -- 172.21.15.152:0/508805840 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v18) ==== 80+0+5304 (secure 0 0 0) 0x7fc83001f020 con 0x7fc84415e800 2024-04-20T11:35:15.233 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:35:15.233 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-04-20T11:35:15.233 INFO:teuthology.orchestra.run.smithi152.stdout: "auth_dump": [ 2024-04-20T11:35:15.234 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:35:15.234 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "mds.a", 2024-04-20T11:35:15.234 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBCpiNmyFRJIBAAj2aIMXdb65nqRgKFDBLLzA==", 2024-04-20T11:35:15.234 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:35:15.234 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow", 2024-04-20T11:35:15.234 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow *", 2024-04-20T11:35:15.234 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow *", 2024-04-20T11:35:15.234 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:35:15.234 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:35:15.234 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:35:15.235 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:35:15.235 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "mds.b", 2024-04-20T11:35:15.235 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBCpiNmVPytOhAAzQkvNOJaNy+MeKETWhYLyg==", 2024-04-20T11:35:15.235 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:35:15.235 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow", 2024-04-20T11:35:15.235 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow *", 2024-04-20T11:35:15.235 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow *", 2024-04-20T11:35:15.235 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:35:15.235 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:35:15.235 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:35:15.235 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:35:15.236 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "mds.c", 2024-04-20T11:35:15.236 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBCpiNm5ViKLRAAg7VBQG4hFTLYUpJMpBAaVA==", 2024-04-20T11:35:15.236 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:35:15.236 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow", 2024-04-20T11:35:15.236 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow *", 2024-04-20T11:35:15.236 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow *", 2024-04-20T11:35:15.236 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:35:15.236 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:35:15.237 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:35:15.237 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:35:15.237 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "mds.d", 2024-04-20T11:35:15.237 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBDpiNmLWhaCxAA2e4qd/7+TKenJlWICIsvGg==", 2024-04-20T11:35:15.237 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:35:15.237 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow", 2024-04-20T11:35:15.237 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow *", 2024-04-20T11:35:15.237 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow *", 2024-04-20T11:35:15.237 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:35:15.237 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:35:15.237 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:35:15.238 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:35:15.238 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "osd.0", 2024-04-20T11:35:15.238 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBHpiNmpuhyDBAAsQXX9DNvWlo0UYh53lb+FA==", 2024-04-20T11:35:15.238 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:35:15.238 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow profile osd", 2024-04-20T11:35:15.238 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile osd", 2024-04-20T11:35:15.238 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:35:15.238 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:35:15.238 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:35:15.238 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:35:15.238 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "osd.1", 2024-04-20T11:35:15.239 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBKpiNmxBiLJhAAHon1adC8G2jbYoDcWdE9jA==", 2024-04-20T11:35:15.239 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:35:15.239 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow profile osd", 2024-04-20T11:35:15.239 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile osd", 2024-04-20T11:35:15.239 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:35:15.239 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:35:15.239 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:35:15.239 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:35:15.239 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "osd.2", 2024-04-20T11:35:15.239 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBNpiNmdn0zMhAAqwIO2kKahdSzTVh8/p3FCg==", 2024-04-20T11:35:15.239 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:35:15.240 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow profile osd", 2024-04-20T11:35:15.240 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile osd", 2024-04-20T11:35:15.240 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:35:15.240 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:35:15.240 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:35:15.240 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:35:15.240 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "osd.3", 2024-04-20T11:35:15.240 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBRpiNmTPCYAhAAHFNHX/xKn21jEFHDkHMjEw==", 2024-04-20T11:35:15.240 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:35:15.240 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow profile osd", 2024-04-20T11:35:15.240 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile osd", 2024-04-20T11:35:15.241 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:35:15.241 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:35:15.241 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:35:15.241 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:35:15.241 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "osd.4", 2024-04-20T11:35:15.241 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBXpiNmM8cVFxAAJl9Hu+MvlkEiMglutI89eg==", 2024-04-20T11:35:15.241 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:35:15.241 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow profile osd", 2024-04-20T11:35:15.241 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile osd", 2024-04-20T11:35:15.241 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:35:15.241 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:35:15.242 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:35:15.242 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:35:15.242 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "osd.5", 2024-04-20T11:35:15.242 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBapiNmnGxJMRAA90P0T+VsTA8LCTcexTVr7w==", 2024-04-20T11:35:15.242 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:35:15.242 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow profile osd", 2024-04-20T11:35:15.242 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile osd", 2024-04-20T11:35:15.243 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:35:15.243 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:35:15.243 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:35:15.243 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:35:15.243 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "osd.6", 2024-04-20T11:35:15.243 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBepiNmyLXWDxAAhtHD5h478mLovE5IejAFng==", 2024-04-20T11:35:15.243 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:35:15.243 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow profile osd", 2024-04-20T11:35:15.244 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile osd", 2024-04-20T11:35:15.244 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:35:15.244 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:35:15.244 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:35:15.244 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:35:15.244 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "osd.7", 2024-04-20T11:35:15.244 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBhpiNmB3jwHBAAqco9om6r2N/GqAV6rVKiaA==", 2024-04-20T11:35:15.244 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:35:15.244 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow profile osd", 2024-04-20T11:35:15.244 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile osd", 2024-04-20T11:35:15.244 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:35:15.245 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:35:15.245 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:35:15.245 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:35:15.245 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "client.0", 2024-04-20T11:35:15.245 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBDpiNmtsBWFhAArq9A7r33CcwgMmOU2pAGPw==", 2024-04-20T11:35:15.245 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:35:15.245 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow", 2024-04-20T11:35:15.245 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow r", 2024-04-20T11:35:15.245 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-20T11:35:15.245 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:35:15.245 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:35:15.246 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:35:15.246 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "client.1", 2024-04-20T11:35:15.246 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBDpiNm6aNnJxAAB431GSGytZ0IKWMh4DMalw==", 2024-04-20T11:35:15.246 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:35:15.246 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow", 2024-04-20T11:35:15.246 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow r", 2024-04-20T11:35:15.246 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow rw", 2024-04-20T11:35:15.246 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow rwx" 2024-04-20T11:35:15.246 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:35:15.246 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:35:15.247 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:35:15.247 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "client.2", 2024-04-20T11:35:15.247 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBDpiNm5QLtHRAAa2kHTDVdgSh/B4XGZaKa5Q==", 2024-04-20T11:35:15.247 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:35:15.247 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow", 2024-04-20T11:35:15.247 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow r", 2024-04-20T11:35:15.247 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow rw", 2024-04-20T11:35:15.247 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow rwx" 2024-04-20T11:35:15.247 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:35:15.247 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:35:15.247 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:35:15.247 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "client.3", 2024-04-20T11:35:15.248 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBDpiNmw95SMBAAdodzTuPp4WdKSEf+jI8Uuw==", 2024-04-20T11:35:15.248 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:35:15.248 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow", 2024-04-20T11:35:15.248 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow r", 2024-04-20T11:35:15.248 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow rw", 2024-04-20T11:35:15.248 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow rwx" 2024-04-20T11:35:15.248 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:35:15.248 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:35:15.248 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:35:15.248 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "client.admin", 2024-04-20T11:35:15.248 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBBpiNm3xz6HhAA82+N+trDW/AmBr3q8l4ECw==", 2024-04-20T11:35:15.249 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:35:15.249 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow *", 2024-04-20T11:35:15.249 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow *", 2024-04-20T11:35:15.249 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow *", 2024-04-20T11:35:15.249 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:35:15.249 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:35:15.249 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:35:15.249 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:35:15.249 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "mgr.x", 2024-04-20T11:35:15.249 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBCpiNmuhalFxAA9dTl/zSbUovCCZAanAipww==", 2024-04-20T11:35:15.249 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:35:15.249 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow *", 2024-04-20T11:35:15.250 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile mgr", 2024-04-20T11:35:15.250 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:35:15.250 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:35:15.250 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:35:15.250 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:35:15.250 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "mgr.y", 2024-04-20T11:35:15.250 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBCpiNmP1+5DRAAs6bVAEOr0i6FdPmSEyo/vQ==", 2024-04-20T11:35:15.250 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:35:15.250 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow *", 2024-04-20T11:35:15.250 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile mgr", 2024-04-20T11:35:15.250 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:35:15.251 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:35:15.251 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:35:15.251 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-04-20T11:35:15.251 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-04-20T11:35:15.252 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.234+0000 7fc84aa00640 1 -- 172.21.15.152:0/508805840 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fc82406ce90 msgr2=0x7fc82406f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:15.252 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.234+0000 7fc84aa00640 1 --2- 172.21.15.152:0/508805840 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fc82406ce90 0x7fc82406f350 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc838036000 tx=0x7fc838030000 comp rx=0 tx=0).stop 2024-04-20T11:35:15.252 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.234+0000 7fc84aa00640 1 -- 172.21.15.152:0/508805840 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc84415e800 msgr2=0x7fc8440d0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:15.252 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.234+0000 7fc84aa00640 1 --2- 172.21.15.152:0/508805840 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc84415e800 0x7fc8440d0f90 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7fc83000b410 tx=0x7fc83000b8d0 comp rx=0 tx=0).stop 2024-04-20T11:35:15.253 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.234+0000 7fc84aa00640 1 -- 172.21.15.152:0/508805840 shutdown_connections 2024-04-20T11:35:15.253 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.234+0000 7fc84aa00640 1 --2- 172.21.15.152:0/508805840 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fc82406ce90 0x7fc82406f350 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:15.253 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.234+0000 7fc84aa00640 1 --2- 172.21.15.152:0/508805840 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8440cc0a0 0x7fc8440d14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:15.253 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.234+0000 7fc84aa00640 1 --2- 172.21.15.152:0/508805840 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc84415e800 0x7fc8440d0f90 unknown :-1 s=CLOSED pgs=1733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:15.253 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.234+0000 7fc84aa00640 1 --2- 172.21.15.152:0/508805840 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc8440b39f0 0x7fc8440ce130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:15.253 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.234+0000 7fc84aa00640 1 -- 172.21.15.152:0/508805840 >> 172.21.15.152:0/508805840 conn(0x7fc84408bfd0 msgr2=0x7fc84415cbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:15.253 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.234+0000 7fc84aa00640 1 -- 172.21.15.152:0/508805840 shutdown_connections 2024-04-20T11:35:15.253 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.235+0000 7fc84aa00640 1 -- 172.21.15.152:0/508805840 wait complete. 2024-04-20T11:35:15.254 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:35:15.444 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.443+0000 7ff08fe00640 1 Processor -- start 2024-04-20T11:35:15.444 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.443+0000 7ff08fe00640 1 -- start start 2024-04-20T11:35:15.446 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.445+0000 7ff08fe00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ff0900723e0 0x7ff0900afa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:15.446 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.445+0000 7ff08fe00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7ff09006e010 0x7ff09006e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:15.446 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.445+0000 7ff08fe00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0900aec90 con 0x7ff090147410 2024-04-20T11:35:15.446 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.445+0000 7ff08fe00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7ff0900aef70 con 0x7ff09006e010 2024-04-20T11:35:15.446 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.445+0000 7ff08fe00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7ff0900af250 con 0x7ff0900723e0 2024-04-20T11:35:15.446 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.445+0000 7ff08f400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7ff09006e010 0x7ff09006e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:15.446 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.445+0000 7ff08f400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7ff09006e010 0x7ff09006e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:57012/0 (socket says 172.21.15.152:57012) 2024-04-20T11:35:15.446 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.445+0000 7ff08f400640 1 -- 172.21.15.152:0/260629233 learned_addr learned my addr 172.21.15.152:0/260629233 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:15.447 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.446+0000 7ff08f400640 1 -- 172.21.15.152:0/260629233 >> v2:172.21.15.175:3300/0 conn(0x7ff0900723e0 msgr2=0x7ff0900afa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:15.447 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.446+0000 7ff08f400640 1 --2- 172.21.15.152:0/260629233 >> v2:172.21.15.175:3300/0 conn(0x7ff0900723e0 0x7ff0900afa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:15.447 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.446+0000 7ff08f400640 1 -- 172.21.15.152:0/260629233 >> v1:172.21.15.175:6789/0 conn(0x7ff090147410 legacy=0x7ff090071cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:15.447 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.446+0000 7ff08f400640 1 -- 172.21.15.152:0/260629233 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0900af530 con 0x7ff09006e010 2024-04-20T11:35:15.447 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.447+0000 7ff08f400640 1 --2- 172.21.15.152:0/260629233 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff09006e010 0x7ff09006e480 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7ff090073170 tx=0x7ff08802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e70f1fef07ba473b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:15.448 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.447+0000 7ff08d600640 1 -- 172.21.15.152:0/260629233 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08802ca80 con 0x7ff09006e010 2024-04-20T11:35:15.448 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.447+0000 7ff08d600640 1 -- 172.21.15.152:0/260629233 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff08802cc20 con 0x7ff09006e010 2024-04-20T11:35:15.449 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.447+0000 7ff08d600640 1 -- 172.21.15.152:0/260629233 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0880343f0 con 0x7ff09006e010 2024-04-20T11:35:15.449 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.448+0000 7ff08fe00640 1 -- 172.21.15.152:0/260629233 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff09006e010 msgr2=0x7ff09006e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:15.449 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.448+0000 7ff08fe00640 1 --2- 172.21.15.152:0/260629233 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff09006e010 0x7ff09006e480 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7ff090073170 tx=0x7ff08802bee0 comp rx=0 tx=0).stop 2024-04-20T11:35:15.449 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.448+0000 7ff08fe00640 1 -- 172.21.15.152:0/260629233 shutdown_connections 2024-04-20T11:35:15.449 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.448+0000 7ff08fe00640 1 --2- 172.21.15.152:0/260629233 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff09006e010 0x7ff09006e480 unknown :-1 s=CLOSED pgs=1734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:15.449 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.448+0000 7ff08fe00640 1 --2- 172.21.15.152:0/260629233 >> v2:172.21.15.175:3300/0 conn(0x7ff0900723e0 0x7ff0900afa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:15.449 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.448+0000 7ff08fe00640 1 -- 172.21.15.152:0/260629233 >> 172.21.15.152:0/260629233 conn(0x7ff090067ae0 msgr2=0x7ff09006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:15.449 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.448+0000 7ff08fe00640 1 -- 172.21.15.152:0/260629233 shutdown_connections 2024-04-20T11:35:15.449 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.448+0000 7ff08fe00640 1 -- 172.21.15.152:0/260629233 wait complete. 2024-04-20T11:35:15.450 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.449+0000 7ff08fe00640 1 Processor -- start 2024-04-20T11:35:15.450 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.449+0000 7ff08fe00640 1 -- start start 2024-04-20T11:35:15.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.450+0000 7ff08fe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff09006e010 0x7ff0901f3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:15.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.450+0000 7ff08fe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff0900723e0 0x7ff09015faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:15.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.450+0000 7ff08fe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff090147410 0x7ff09015ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:15.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.450+0000 7ff08fe00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7ff09006eb40 con 0x7ff09006e010 2024-04-20T11:35:15.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.450+0000 7ff08fe00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff09006e9c0 con 0x7ff090147410 2024-04-20T11:35:15.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.450+0000 7ff08fe00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff0900b3390 con 0x7ff0900723e0 2024-04-20T11:35:15.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.450+0000 7ff08ea00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff09006e010 0x7ff0901f3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:15.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.451+0000 7ff08ea00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff09006e010 0x7ff0901f3830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:57022/0 (socket says 172.21.15.152:57022) 2024-04-20T11:35:15.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.451+0000 7ff08ea00640 1 -- 172.21.15.152:0/4286237521 learned_addr learned my addr 172.21.15.152:0/4286237521 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:15.452 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.451+0000 7ff08ea00640 1 -- 172.21.15.152:0/4286237521 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff0900723e0 msgr2=0x7ff09015faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:15.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.451+0000 7ff08ea00640 1 --2- 172.21.15.152:0/4286237521 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff0900723e0 0x7ff09015faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:15.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.451+0000 7ff08ea00640 1 -- 172.21.15.152:0/4286237521 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff090147410 msgr2=0x7ff09015ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:15.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.451+0000 7ff08ea00640 1 --2- 172.21.15.152:0/4286237521 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff090147410 0x7ff09015ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:15.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.451+0000 7ff08ea00640 1 -- 172.21.15.152:0/4286237521 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0901611d0 con 0x7ff09006e010 2024-04-20T11:35:15.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.451+0000 7ff08ea00640 1 --2- 172.21.15.152:0/4286237521 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff09006e010 0x7ff0901f3830 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7ff08000b410 tx=0x7ff08000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:15.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.451+0000 7ff086a00640 1 -- 172.21.15.152:0/4286237521 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff080015020 con 0x7ff09006e010 2024-04-20T11:35:15.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.451+0000 7ff086a00640 1 -- 172.21.15.152:0/4286237521 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff080011830 con 0x7ff09006e010 2024-04-20T11:35:15.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.451+0000 7ff086a00640 1 -- 172.21.15.152:0/4286237521 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff080011c20 con 0x7ff09006e010 2024-04-20T11:35:15.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.452+0000 7ff08fe00640 1 -- 172.21.15.152:0/4286237521 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff090160610 con 0x7ff09006e010 2024-04-20T11:35:15.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.452+0000 7ff08fe00640 1 -- 172.21.15.152:0/4286237521 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff090141620 con 0x7ff09006e010 2024-04-20T11:35:15.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.453+0000 7ff08fe00640 1 -- 172.21.15.152:0/4286237521 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff090075e60 con 0x7ff09006e010 2024-04-20T11:35:15.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.454+0000 7ff086a00640 1 -- 172.21.15.152:0/4286237521 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 21) ==== 91550+0+0 (secure 0 0 0) 0x7ff080018020 con 0x7ff09006e010 2024-04-20T11:35:15.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.455+0000 7ff086a00640 1 --2- 172.21.15.152:0/4286237521 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7ff07006ce90 0x7ff07006f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:15.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.455+0000 7ff086a00640 1 -- 172.21.15.152:0/4286237521 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 4857+0+0 (secure 0 0 0) 0x7ff08008c200 con 0x7ff09006e010 2024-04-20T11:35:15.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.455+0000 7ff08e000640 1 --2- 172.21.15.152:0/4286237521 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7ff07006ce90 0x7ff07006f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:15.456 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.456+0000 7ff08e000640 1 --2- 172.21.15.152:0/4286237521 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7ff07006ce90 0x7ff07006f350 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff09006bfa0 tx=0x7ff07c006c90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:15.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.459+0000 7ff086a00640 1 -- 172.21.15.152:0/4286237521 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ff08001f020 con 0x7ff09006e010 2024-04-20T11:35:15.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.648+0000 7ff08fe00640 1 -- 172.21.15.152:0/4286237521 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff09006f920 con 0x7ff09006e010 2024-04-20T11:35:15.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.650+0000 7ff086a00640 1 -- 172.21.15.152:0/4286237521 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v127) ==== 74+0+12961 (secure 0 0 0) 0x7ff0800172c0 con 0x7ff09006e010 2024-04-20T11:35:15.651 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:35:15.651 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":127,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:35:14.735643+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:35:15.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.653+0000 7ff08fe00640 1 -- 172.21.15.152:0/4286237521 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7ff07006ce90 msgr2=0x7ff07006f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:15.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.653+0000 7ff08fe00640 1 --2- 172.21.15.152:0/4286237521 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7ff07006ce90 0x7ff07006f350 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff09006bfa0 tx=0x7ff07c006c90 comp rx=0 tx=0).stop 2024-04-20T11:35:15.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.653+0000 7ff08fe00640 1 -- 172.21.15.152:0/4286237521 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff09006e010 msgr2=0x7ff0901f3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:15.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.653+0000 7ff08fe00640 1 --2- 172.21.15.152:0/4286237521 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff09006e010 0x7ff0901f3830 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7ff08000b410 tx=0x7ff08000b8d0 comp rx=0 tx=0).stop 2024-04-20T11:35:15.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.653+0000 7ff08fe00640 1 -- 172.21.15.152:0/4286237521 shutdown_connections 2024-04-20T11:35:15.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.653+0000 7ff08fe00640 1 --2- 172.21.15.152:0/4286237521 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7ff07006ce90 0x7ff07006f350 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:15.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.653+0000 7ff08fe00640 1 --2- 172.21.15.152:0/4286237521 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff090147410 0x7ff09015ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:15.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.653+0000 7ff08fe00640 1 --2- 172.21.15.152:0/4286237521 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff0900723e0 0x7ff09015faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:15.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.653+0000 7ff08fe00640 1 --2- 172.21.15.152:0/4286237521 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff09006e010 0x7ff0901f3830 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:15.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.653+0000 7ff08fe00640 1 -- 172.21.15.152:0/4286237521 >> 172.21.15.152:0/4286237521 conn(0x7ff090067ae0 msgr2=0x7ff0900b3730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:15.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.653+0000 7ff08fe00640 1 -- 172.21.15.152:0/4286237521 shutdown_connections 2024-04-20T11:35:15.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.653+0000 7ff08fe00640 1 -- 172.21.15.152:0/4286237521 wait complete. 2024-04-20T11:35:15.671 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T11:26:47.246571+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T11:35:15.672 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T11:35:15.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.860+0000 7fae04a00640 1 Processor -- start 2024-04-20T11:35:15.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.860+0000 7fae04a00640 1 -- start start 2024-04-20T11:35:15.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.861+0000 7fae04a00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fae00146000 0x7fae000b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:15.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.861+0000 7fae04a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fae000cd8c0 0x7fae000b2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:15.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.861+0000 7fae04a00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae001463e0 con 0x7fae000b2170 2024-04-20T11:35:15.863 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.862+0000 7fae04a00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fae000cdca0 con 0x7fae000cd8c0 2024-04-20T11:35:15.863 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.862+0000 7fae04a00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fae000b4750 con 0x7fae00146000 2024-04-20T11:35:15.863 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.862+0000 7fadfe800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fae000cd8c0 0x7fae000b2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:15.863 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.862+0000 7fadfe800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fae000cd8c0 0x7fae000b2620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52206/0 (socket says 172.21.15.152:52206) 2024-04-20T11:35:15.863 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.862+0000 7fadfe800640 1 -- 172.21.15.152:0/4232694181 learned_addr learned my addr 172.21.15.152:0/4232694181 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:15.863 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.862+0000 7fadfe800640 1 -- 172.21.15.152:0/4232694181 >> v2:172.21.15.175:3300/0 conn(0x7fae00146000 msgr2=0x7fae000b4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:15.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.862+0000 7fadfde00640 1 --2- 172.21.15.152:0/4232694181 >> v2:172.21.15.175:3300/0 conn(0x7fae00146000 0x7fae000b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:15.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.862+0000 7fadfe800640 1 --2- 172.21.15.152:0/4232694181 >> v2:172.21.15.175:3300/0 conn(0x7fae00146000 0x7fae000b4be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:15.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.862+0000 7fadfe800640 1 -- 172.21.15.152:0/4232694181 >> v1:172.21.15.175:6789/0 conn(0x7fae000b2170 legacy=0x7fae000cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:15.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.862+0000 7fadfe800640 1 -- 172.21.15.152:0/4232694181 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae000b3e20 con 0x7fae000cd8c0 2024-04-20T11:35:15.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.864+0000 7fadfe800640 1 --2- 172.21.15.152:0/4232694181 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fae000cd8c0 0x7fae000b2620 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7fae00070190 tx=0x7fadf802be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74f6fadf31c6a354 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:15.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.864+0000 7fadfca00640 1 -- 172.21.15.152:0/4232694181 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf802c9f0 con 0x7fae000cd8c0 2024-04-20T11:35:15.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.864+0000 7fadfca00640 1 -- 172.21.15.152:0/4232694181 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadf802cb90 con 0x7fae000cd8c0 2024-04-20T11:35:15.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.864+0000 7fadfca00640 1 -- 172.21.15.152:0/4232694181 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf80343f0 con 0x7fae000cd8c0 2024-04-20T11:35:15.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.864+0000 7fae04a00640 1 -- 172.21.15.152:0/4232694181 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fae000cd8c0 msgr2=0x7fae000b2620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:15.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.864+0000 7fae04a00640 1 --2- 172.21.15.152:0/4232694181 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fae000cd8c0 0x7fae000b2620 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7fae00070190 tx=0x7fadf802be50 comp rx=0 tx=0).stop 2024-04-20T11:35:15.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.864+0000 7fae04a00640 1 -- 172.21.15.152:0/4232694181 shutdown_connections 2024-04-20T11:35:15.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.864+0000 7fae04a00640 1 --2- 172.21.15.152:0/4232694181 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fae000cd8c0 0x7fae000b2620 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:15.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.865+0000 7fae04a00640 1 --2- 172.21.15.152:0/4232694181 >> v2:172.21.15.175:3300/0 conn(0x7fae00146000 0x7fae000b4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:15.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.865+0000 7fae04a00640 1 -- 172.21.15.152:0/4232694181 >> 172.21.15.152:0/4232694181 conn(0x7fae0006b980 msgr2=0x7fae0006bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:15.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.865+0000 7fae04a00640 1 -- 172.21.15.152:0/4232694181 shutdown_connections 2024-04-20T11:35:15.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.865+0000 7fae04a00640 1 -- 172.21.15.152:0/4232694181 wait complete. 2024-04-20T11:35:15.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.866+0000 7fae04a00640 1 Processor -- start 2024-04-20T11:35:15.867 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.866+0000 7fae04a00640 1 -- start start 2024-04-20T11:35:15.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.867+0000 7fae04a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae000b2170 0x7fae0013bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:15.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.867+0000 7fae04a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fae000cd8c0 0x7fae0013aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:15.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.867+0000 7fae04a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fae00146000 0x7fae0006f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:15.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.867+0000 7fae04a00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fae0015ccf0 con 0x7fae00146000 2024-04-20T11:35:15.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.867+0000 7fae04a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fae000b43e0 con 0x7fae000b2170 2024-04-20T11:35:15.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.867+0000 7fae04a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fae0015ce70 con 0x7fae000cd8c0 2024-04-20T11:35:15.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.868+0000 7fadfe800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fae00146000 0x7fae0006f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:15.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.868+0000 7fadfe800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fae00146000 0x7fae0006f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52212/0 (socket says 172.21.15.152:52212) 2024-04-20T11:35:15.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.868+0000 7fadfe800640 1 -- 172.21.15.152:0/687803423 learned_addr learned my addr 172.21.15.152:0/687803423 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:15.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.868+0000 7fadfde00640 1 --2- 172.21.15.152:0/687803423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae000b2170 0x7fae0013bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:15.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.868+0000 7fadfe800640 1 -- 172.21.15.152:0/687803423 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fae000cd8c0 msgr2=0x7fae0013aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:15.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.868+0000 7fadfe800640 1 --2- 172.21.15.152:0/687803423 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fae000cd8c0 0x7fae0013aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:15.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.868+0000 7fadfe800640 1 -- 172.21.15.152:0/687803423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae000b2170 msgr2=0x7fae0013bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:15.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.868+0000 7fadfe800640 1 --2- 172.21.15.152:0/687803423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae000b2170 0x7fae0013bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:15.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.868+0000 7fadfe800640 1 -- 172.21.15.152:0/687803423 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae0013c430 con 0x7fae00146000 2024-04-20T11:35:15.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.868+0000 7fadfde00640 1 --2- 172.21.15.152:0/687803423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae000b2170 0x7fae0013bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:35:15.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.868+0000 7fadfe800640 1 --2- 172.21.15.152:0/687803423 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fae00146000 0x7fae0006f640 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7fadf802c460 tx=0x7fadf8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:15.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.869+0000 7fadf6000640 1 -- 172.21.15.152:0/687803423 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf8043020 con 0x7fae00146000 2024-04-20T11:35:15.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.869+0000 7fadf6000640 1 -- 172.21.15.152:0/687803423 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadf803e450 con 0x7fae00146000 2024-04-20T11:35:15.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.869+0000 7fadf6000640 1 -- 172.21.15.152:0/687803423 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf8009510 con 0x7fae00146000 2024-04-20T11:35:15.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.869+0000 7fae04a00640 1 -- 172.21.15.152:0/687803423 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae0006e2c0 con 0x7fae00146000 2024-04-20T11:35:15.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.869+0000 7fae04a00640 1 -- 172.21.15.152:0/687803423 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae0006e880 con 0x7fae00146000 2024-04-20T11:35:15.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.870+0000 7fae04a00640 1 -- 172.21.15.152:0/687803423 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae000689f0 con 0x7fae00146000 2024-04-20T11:35:15.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.871+0000 7fadf6000640 1 -- 172.21.15.152:0/687803423 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 21) ==== 91550+0+0 (secure 0 0 0) 0x7fadf8046020 con 0x7fae00146000 2024-04-20T11:35:15.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.872+0000 7fadf6000640 1 --2- 172.21.15.152:0/687803423 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fade006ce90 0x7fade006f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:15.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.872+0000 7fadf6000640 1 -- 172.21.15.152:0/687803423 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 4857+0+0 (secure 0 0 0) 0x7fadf80aefa0 con 0x7fae00146000 2024-04-20T11:35:15.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.872+0000 7fadfde00640 1 --2- 172.21.15.152:0/687803423 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fade006ce90 0x7fade006f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:15.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.874+0000 7fadfde00640 1 --2- 172.21.15.152:0/687803423 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fade006ce90 0x7fade006f350 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fadf000b820 tx=0x7fadf0008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:15.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:15.876+0000 7fadf6000640 1 -- 172.21.15.152:0/687803423 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fadf804d020 con 0x7fae00146000 2024-04-20T11:35:16.057 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.057+0000 7fae04a00640 1 -- 172.21.15.152:0/687803423 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fae000b2f80 con 0x7fae00146000 2024-04-20T11:35:16.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.057+0000 7fadf6000640 1 -- 172.21.15.152:0/687803423 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v127) ==== 93+0+10 (secure 0 0 0) 0x7fadf807cde0 con 0x7fae00146000 2024-04-20T11:35:16.058 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 1 2024-04-20T11:35:16.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.060+0000 7fae04a00640 1 -- 172.21.15.152:0/687803423 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fade006ce90 msgr2=0x7fade006f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:16.061 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.060+0000 7fae04a00640 1 --2- 172.21.15.152:0/687803423 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fade006ce90 0x7fade006f350 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fadf000b820 tx=0x7fadf0008040 comp rx=0 tx=0).stop 2024-04-20T11:35:16.061 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.060+0000 7fae04a00640 1 -- 172.21.15.152:0/687803423 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fae00146000 msgr2=0x7fae0006f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:16.061 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.060+0000 7fae04a00640 1 --2- 172.21.15.152:0/687803423 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fae00146000 0x7fae0006f640 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7fadf802c460 tx=0x7fadf8004a40 comp rx=0 tx=0).stop 2024-04-20T11:35:16.061 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.060+0000 7fae04a00640 1 -- 172.21.15.152:0/687803423 shutdown_connections 2024-04-20T11:35:16.061 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.060+0000 7fae04a00640 1 --2- 172.21.15.152:0/687803423 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fade006ce90 0x7fade006f350 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:16.061 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.060+0000 7fae04a00640 1 --2- 172.21.15.152:0/687803423 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fae00146000 0x7fae0006f640 unknown :-1 s=CLOSED pgs=1737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:16.061 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.060+0000 7fae04a00640 1 --2- 172.21.15.152:0/687803423 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fae000cd8c0 0x7fae0013aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:16.061 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.060+0000 7fae04a00640 1 --2- 172.21.15.152:0/687803423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae000b2170 0x7fae0013bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:16.061 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.060+0000 7fae04a00640 1 -- 172.21.15.152:0/687803423 >> 172.21.15.152:0/687803423 conn(0x7fae0006b980 msgr2=0x7fae000cd180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:16.061 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.060+0000 7fae04a00640 1 -- 172.21.15.152:0/687803423 shutdown_connections 2024-04-20T11:35:16.061 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.060+0000 7fae04a00640 1 -- 172.21.15.152:0/687803423 wait complete. 2024-04-20T11:35:16.080 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-20T11:35:16.276 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.275+0000 7fb6fd800640 1 Processor -- start 2024-04-20T11:35:16.276 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.275+0000 7fb6fd800640 1 -- start start 2024-04-20T11:35:16.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.277+0000 7fb6fd800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fb6f81603d0 0x7fb6f80b5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:16.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.277+0000 7fb6fd800640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fb6f80b5480 0x7fb6f80b63f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:16.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.277+0000 7fb6fd800640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6f80d3350 con 0x7fb6f815fcc0 2024-04-20T11:35:16.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.277+0000 7fb6fd800640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fb6f80d3630 con 0x7fb6f81603d0 2024-04-20T11:35:16.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.277+0000 7fb6fd800640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fb6f80d3910 con 0x7fb6f80b5480 2024-04-20T11:35:16.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.277+0000 7fb6f7e00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fb6f815fcc0 0x7fb6f80d4e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:39712/0 (socket says 172.21.15.152:39712) 2024-04-20T11:35:16.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.277+0000 7fb6f7e00640 1 -- 172.21.15.152:0/535602382 learned_addr learned my addr 172.21.15.152:0/535602382 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:35:16.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.277+0000 7fb6ef400640 1 --2- 172.21.15.152:0/535602382 >> v2:172.21.15.152:3300/0 conn(0x7fb6f81603d0 0x7fb6f80b5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:16.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.278+0000 7fb6fce00640 1 --2- 172.21.15.152:0/535602382 >> v2:172.21.15.175:3300/0 conn(0x7fb6f80b5480 0x7fb6f80b63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:16.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.278+0000 7fb6f7400640 1 -- 172.21.15.152:0/535602382 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1353783210 0 0) 0x7fb6f80d3350 con 0x7fb6f815fcc0 2024-04-20T11:35:16.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.278+0000 7fb6f7400640 1 -- 172.21.15.152:0/535602382 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb6dc003520 con 0x7fb6f815fcc0 2024-04-20T11:35:16.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.278+0000 7fb6ef400640 1 -- 172.21.15.152:0/535602382 >> v2:172.21.15.175:3300/0 conn(0x7fb6f80b5480 msgr2=0x7fb6f80b63f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:16.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.278+0000 7fb6ef400640 1 --2- 172.21.15.152:0/535602382 >> v2:172.21.15.175:3300/0 conn(0x7fb6f80b5480 0x7fb6f80b63f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:16.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.278+0000 7fb6ef400640 1 -- 172.21.15.152:0/535602382 >> v1:172.21.15.152:6789/0 conn(0x7fb6f815fcc0 legacy=0x7fb6f80d4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:16.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.278+0000 7fb6ef400640 1 -- 172.21.15.152:0/535602382 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6f80ced20 con 0x7fb6f81603d0 2024-04-20T11:35:16.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.279+0000 7fb6ef400640 1 --2- 172.21.15.152:0/535602382 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb6f81603d0 0x7fb6f80b5eb0 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7fb6f808e7f0 tx=0x7fb6e002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae91b70e86ce15f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:16.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.279+0000 7fb6f7400640 1 -- 172.21.15.152:0/535602382 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e0035020 con 0x7fb6f81603d0 2024-04-20T11:35:16.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.279+0000 7fb6f7400640 1 -- 172.21.15.152:0/535602382 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6e0031830 con 0x7fb6f81603d0 2024-04-20T11:35:16.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.279+0000 7fb6f7400640 1 -- 172.21.15.152:0/535602382 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e0031c00 con 0x7fb6f81603d0 2024-04-20T11:35:16.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.280+0000 7fb6fd800640 1 -- 172.21.15.152:0/535602382 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb6f81603d0 msgr2=0x7fb6f80b5eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:16.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.280+0000 7fb6fd800640 1 --2- 172.21.15.152:0/535602382 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb6f81603d0 0x7fb6f80b5eb0 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7fb6f808e7f0 tx=0x7fb6e002b8d0 comp rx=0 tx=0).stop 2024-04-20T11:35:16.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.280+0000 7fb6fd800640 1 -- 172.21.15.152:0/535602382 shutdown_connections 2024-04-20T11:35:16.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.280+0000 7fb6fd800640 1 --2- 172.21.15.152:0/535602382 >> v2:172.21.15.175:3300/0 conn(0x7fb6f80b5480 0x7fb6f80b63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:16.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.280+0000 7fb6fd800640 1 --2- 172.21.15.152:0/535602382 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb6f81603d0 0x7fb6f80b5eb0 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:16.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.280+0000 7fb6fd800640 1 -- 172.21.15.152:0/535602382 >> 172.21.15.152:0/535602382 conn(0x7fb6f808bfd0 msgr2=0x7fb6f80d4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:16.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.280+0000 7fb6fd800640 1 -- 172.21.15.152:0/535602382 shutdown_connections 2024-04-20T11:35:16.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.280+0000 7fb6fd800640 1 -- 172.21.15.152:0/535602382 wait complete. 2024-04-20T11:35:16.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.282+0000 7fb6fd800640 1 Processor -- start 2024-04-20T11:35:16.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.282+0000 7fb6fd800640 1 -- start start 2024-04-20T11:35:16.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.283+0000 7fb6fd800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb6f80b5480 0x7fb6f8157360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:16.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.283+0000 7fb6fd800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb6f815fcc0 0x7fb6f80c9f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:16.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.283+0000 7fb6fd800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb6f81603d0 0x7fb6f80ca4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:16.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.283+0000 7fb6fd800640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f80cf610 con 0x7fb6f80b5480 2024-04-20T11:35:16.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.283+0000 7fb6fd800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f80cf490 con 0x7fb6f815fcc0 2024-04-20T11:35:16.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.283+0000 7fb6fd800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb6f80cf790 con 0x7fb6f81603d0 2024-04-20T11:35:16.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.283+0000 7fb6f7e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb6f80b5480 0x7fb6f8157360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:16.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.283+0000 7fb6f7e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb6f80b5480 0x7fb6f8157360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52236/0 (socket says 172.21.15.152:52236) 2024-04-20T11:35:16.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.283+0000 7fb6f7e00640 1 -- 172.21.15.152:0/1377633640 learned_addr learned my addr 172.21.15.152:0/1377633640 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:16.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.283+0000 7fb6f7e00640 1 -- 172.21.15.152:0/1377633640 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb6f81603d0 msgr2=0x7fb6f80ca4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:16.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.284+0000 7fb6f7e00640 1 --2- 172.21.15.152:0/1377633640 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb6f81603d0 0x7fb6f80ca4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:16.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.284+0000 7fb6f7e00640 1 -- 172.21.15.152:0/1377633640 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb6f815fcc0 msgr2=0x7fb6f80c9f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:16.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.284+0000 7fb6ef400640 1 --2- 172.21.15.152:0/1377633640 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb6f815fcc0 0x7fb6f80c9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:16.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.284+0000 7fb6f7e00640 1 --2- 172.21.15.152:0/1377633640 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb6f815fcc0 0x7fb6f80c9f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:16.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.284+0000 7fb6f7e00640 1 -- 172.21.15.152:0/1377633640 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6f80c8f90 con 0x7fb6f80b5480 2024-04-20T11:35:16.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.284+0000 7fb6f7e00640 1 --2- 172.21.15.152:0/1377633640 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb6f80b5480 0x7fb6f8157360 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7fb6e800c430 tx=0x7fb6e800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:16.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.284+0000 7fb6f4c00640 1 -- 172.21.15.152:0/1377633640 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e8017070 con 0x7fb6f80b5480 2024-04-20T11:35:16.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.284+0000 7fb6f4c00640 1 -- 172.21.15.152:0/1377633640 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6e8005ce0 con 0x7fb6f80b5480 2024-04-20T11:35:16.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.284+0000 7fb6f4c00640 1 -- 172.21.15.152:0/1377633640 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e80072f0 con 0x7fb6f80b5480 2024-04-20T11:35:16.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.284+0000 7fb6fd800640 1 -- 172.21.15.152:0/1377633640 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6f80c9270 con 0x7fb6f80b5480 2024-04-20T11:35:16.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.284+0000 7fb6fd800640 1 -- 172.21.15.152:0/1377633640 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6f80c9830 con 0x7fb6f80b5480 2024-04-20T11:35:16.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.285+0000 7fb6fd800640 1 -- 172.21.15.152:0/1377633640 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6f80d5410 con 0x7fb6f80b5480 2024-04-20T11:35:16.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.286+0000 7fb6f4c00640 1 -- 172.21.15.152:0/1377633640 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 21) ==== 91550+0+0 (secure 0 0 0) 0x7fb6e801a070 con 0x7fb6f80b5480 2024-04-20T11:35:16.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.287+0000 7fb6f4c00640 1 --2- 172.21.15.152:0/1377633640 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb6d406ce90 0x7fb6d406f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:16.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.287+0000 7fb6f4c00640 1 -- 172.21.15.152:0/1377633640 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 4857+0+0 (secure 0 0 0) 0x7fb6e808b790 con 0x7fb6f80b5480 2024-04-20T11:35:16.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.288+0000 7fb6ef400640 1 --2- 172.21.15.152:0/1377633640 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb6d406ce90 0x7fb6d406f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:16.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.289+0000 7fb6ef400640 1 --2- 172.21.15.152:0/1377633640 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb6d406ce90 0x7fb6d406f350 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fb6e0036000 tx=0x7fb6e0008030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:16.292 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.292+0000 7fb6f4c00640 1 -- 172.21.15.152:0/1377633640 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fb6f80d5410 con 0x7fb6f80b5480 2024-04-20T11:35:16.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.490+0000 7fb6fd800640 1 -- 172.21.15.152:0/1377633640 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fb6f80d2f30 con 0x7fb6f80b5480 2024-04-20T11:35:16.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.491+0000 7fb6f4c00640 1 -- 172.21.15.152:0/1377633640 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v127) ==== 84+0+65 (secure 0 0 0) 0x7fb6f80d2f30 con 0x7fb6f80b5480 2024-04-20T11:35:16.492 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:35:16.492 INFO:teuthology.orchestra.run.smithi152.stdout:[ 2024-04-20T11:35:16.492 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:35:16.493 INFO:teuthology.orchestra.run.smithi152.stdout: "poolnum": 1, 2024-04-20T11:35:16.493 INFO:teuthology.orchestra.run.smithi152.stdout: "poolname": ".mgr" 2024-04-20T11:35:16.493 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:35:16.493 INFO:teuthology.orchestra.run.smithi152.stdout:] 2024-04-20T11:35:16.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.494+0000 7fb6fd800640 1 -- 172.21.15.152:0/1377633640 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb6d406ce90 msgr2=0x7fb6d406f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:16.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.494+0000 7fb6fd800640 1 --2- 172.21.15.152:0/1377633640 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb6d406ce90 0x7fb6d406f350 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fb6e0036000 tx=0x7fb6e0008030 comp rx=0 tx=0).stop 2024-04-20T11:35:16.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.494+0000 7fb6fd800640 1 -- 172.21.15.152:0/1377633640 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb6f80b5480 msgr2=0x7fb6f8157360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:16.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.494+0000 7fb6fd800640 1 --2- 172.21.15.152:0/1377633640 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb6f80b5480 0x7fb6f8157360 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7fb6e800c430 tx=0x7fb6e800c8f0 comp rx=0 tx=0).stop 2024-04-20T11:35:16.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.494+0000 7fb6fd800640 1 -- 172.21.15.152:0/1377633640 shutdown_connections 2024-04-20T11:35:16.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.494+0000 7fb6fd800640 1 --2- 172.21.15.152:0/1377633640 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb6d406ce90 0x7fb6d406f350 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:16.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.494+0000 7fb6fd800640 1 --2- 172.21.15.152:0/1377633640 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb6f81603d0 0x7fb6f80ca4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:16.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.494+0000 7fb6fd800640 1 --2- 172.21.15.152:0/1377633640 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb6f815fcc0 0x7fb6f80c9f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:16.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.494+0000 7fb6fd800640 1 --2- 172.21.15.152:0/1377633640 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb6f80b5480 0x7fb6f8157360 unknown :-1 s=CLOSED pgs=1740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:16.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.494+0000 7fb6fd800640 1 -- 172.21.15.152:0/1377633640 >> 172.21.15.152:0/1377633640 conn(0x7fb6f808bfd0 msgr2=0x7fb6f80d3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:16.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.494+0000 7fb6fd800640 1 -- 172.21.15.152:0/1377633640 shutdown_connections 2024-04-20T11:35:16.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.494+0000 7fb6fd800640 1 -- 172.21.15.152:0/1377633640 wait complete. 2024-04-20T11:35:16.514 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-20T11:35:16.514 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-20T11:35:16.725 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.725+0000 7fb3d3200640 1 Processor -- start 2024-04-20T11:35:16.725 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.725+0000 7fb3d3200640 1 -- start start 2024-04-20T11:35:16.726 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.726+0000 7fb3d3200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fb3d415e900 0x7fb3d40b39f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:16.727 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.726+0000 7fb3d3200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fb3d40d32e0 0x7fb3d40d01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:16.727 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.726+0000 7fb3d3200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3d40d5310 con 0x7fb3d406ca80 2024-04-20T11:35:16.727 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.726+0000 7fb3d3200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fb3d40cf3e0 con 0x7fb3d40d32e0 2024-04-20T11:35:16.727 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.726+0000 7fb3d3200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fb3d40cf6c0 con 0x7fb3d415e900 2024-04-20T11:35:16.727 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.727+0000 7fb3d2800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fb3d40d32e0 0x7fb3d40d01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:16.727 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.727+0000 7fb3d2800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fb3d40d32e0 0x7fb3d40d01a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52238/0 (socket says 172.21.15.152:52238) 2024-04-20T11:35:16.728 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.727+0000 7fb3d2800640 1 -- 172.21.15.152:0/3301688236 learned_addr learned my addr 172.21.15.152:0/3301688236 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:16.728 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.727+0000 7fb3d1e00640 1 --2- 172.21.15.152:0/3301688236 >> v2:172.21.15.175:3300/0 conn(0x7fb3d415e900 0x7fb3d40b39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:16.728 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.728+0000 7fb3d2800640 1 -- 172.21.15.152:0/3301688236 >> v2:172.21.15.175:3300/0 conn(0x7fb3d415e900 msgr2=0x7fb3d40b39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:16.728 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.728+0000 7fb3d2800640 1 --2- 172.21.15.152:0/3301688236 >> v2:172.21.15.175:3300/0 conn(0x7fb3d415e900 0x7fb3d40b39f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:16.728 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.728+0000 7fb3d2800640 1 -- 172.21.15.152:0/3301688236 >> v1:172.21.15.175:6789/0 conn(0x7fb3d406ca80 legacy=0x7fb3d40d2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:16.728 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.728+0000 7fb3d2800640 1 -- 172.21.15.152:0/3301688236 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3d40cf9a0 con 0x7fb3d40d32e0 2024-04-20T11:35:16.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.729+0000 7fb3d2800640 1 --2- 172.21.15.152:0/3301688236 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb3d40d32e0 0x7fb3d40d01a0 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7fb3d408cb00 tx=0x7fb3cc02bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=922c2e325c6b9394 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:16.731 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.730+0000 7fb3d0a00640 1 -- 172.21.15.152:0/3301688236 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3cc02cb20 con 0x7fb3d40d32e0 2024-04-20T11:35:16.731 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.730+0000 7fb3d0a00640 1 -- 172.21.15.152:0/3301688236 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3cc02ccc0 con 0x7fb3d40d32e0 2024-04-20T11:35:16.731 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.730+0000 7fb3d0a00640 1 -- 172.21.15.152:0/3301688236 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3cc034610 con 0x7fb3d40d32e0 2024-04-20T11:35:16.732 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.731+0000 7fb3d3200640 1 -- 172.21.15.152:0/3301688236 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb3d40d32e0 msgr2=0x7fb3d40d01a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:16.732 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.731+0000 7fb3d3200640 1 --2- 172.21.15.152:0/3301688236 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb3d40d32e0 0x7fb3d40d01a0 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7fb3d408cb00 tx=0x7fb3cc02bf80 comp rx=0 tx=0).stop 2024-04-20T11:35:16.732 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.731+0000 7fb3d3200640 1 -- 172.21.15.152:0/3301688236 shutdown_connections 2024-04-20T11:35:16.732 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.731+0000 7fb3d3200640 1 --2- 172.21.15.152:0/3301688236 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb3d40d32e0 0x7fb3d40d01a0 unknown :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:16.732 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.731+0000 7fb3d3200640 1 --2- 172.21.15.152:0/3301688236 >> v2:172.21.15.175:3300/0 conn(0x7fb3d415e900 0x7fb3d40b39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:16.732 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.731+0000 7fb3d3200640 1 -- 172.21.15.152:0/3301688236 >> 172.21.15.152:0/3301688236 conn(0x7fb3d408bfd0 msgr2=0x7fb3d40d45f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:16.732 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.731+0000 7fb3d3200640 1 -- 172.21.15.152:0/3301688236 shutdown_connections 2024-04-20T11:35:16.732 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.731+0000 7fb3d3200640 1 -- 172.21.15.152:0/3301688236 wait complete. 2024-04-20T11:35:16.734 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.734+0000 7fb3d3200640 1 Processor -- start 2024-04-20T11:35:16.734 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.734+0000 7fb3d3200640 1 -- start start 2024-04-20T11:35:16.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.735+0000 7fb3d3200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb3d406ca80 0x7fb3d4153a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:16.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.735+0000 7fb3d3200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb3d40d32e0 0x7fb3d40cd890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:16.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.735+0000 7fb3d3200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb3d415e900 0x7fb3d40cddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:16.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.735+0000 7fb3d3200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d40d2710 con 0x7fb3d40d32e0 2024-04-20T11:35:16.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.735+0000 7fb3d3200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d40d2590 con 0x7fb3d415e900 2024-04-20T11:35:16.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.735+0000 7fb3d3200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb3d40d4f10 con 0x7fb3d406ca80 2024-04-20T11:35:16.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.735+0000 7fb3d1400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb3d40d32e0 0x7fb3d40cd890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:16.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.735+0000 7fb3d1400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb3d40d32e0 0x7fb3d40cd890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52244/0 (socket says 172.21.15.152:52244) 2024-04-20T11:35:16.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.735+0000 7fb3d1400640 1 -- 172.21.15.152:0/2182382567 learned_addr learned my addr 172.21.15.152:0/2182382567 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:16.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.736+0000 7fb3d1e00640 1 --2- 172.21.15.152:0/2182382567 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb3d406ca80 0x7fb3d4153a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:16.738 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.736+0000 7fb3d2800640 1 --2- 172.21.15.152:0/2182382567 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb3d415e900 0x7fb3d40cddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:16.738 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.736+0000 7fb3d1400640 1 -- 172.21.15.152:0/2182382567 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb3d406ca80 msgr2=0x7fb3d4153a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:16.738 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.736+0000 7fb3d1400640 1 --2- 172.21.15.152:0/2182382567 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb3d406ca80 0x7fb3d4153a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:16.738 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.736+0000 7fb3d1400640 1 -- 172.21.15.152:0/2182382567 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb3d415e900 msgr2=0x7fb3d40cddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:16.738 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.736+0000 7fb3d1400640 1 --2- 172.21.15.152:0/2182382567 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb3d415e900 0x7fb3d40cddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:16.738 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.736+0000 7fb3d1400640 1 -- 172.21.15.152:0/2182382567 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3d40a1720 con 0x7fb3d40d32e0 2024-04-20T11:35:16.738 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.736+0000 7fb3d1400640 1 --2- 172.21.15.152:0/2182382567 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb3d40d32e0 0x7fb3d40cd890 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7fb3c000c430 tx=0x7fb3c000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:16.738 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.736+0000 7fb3ca000640 1 -- 172.21.15.152:0/2182382567 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3c00052d0 con 0x7fb3d40d32e0 2024-04-20T11:35:16.738 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.736+0000 7fb3ca000640 1 -- 172.21.15.152:0/2182382567 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3c0005490 con 0x7fb3d40d32e0 2024-04-20T11:35:16.738 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.736+0000 7fb3d3200640 1 -- 172.21.15.152:0/2182382567 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3d40a1a00 con 0x7fb3d40d32e0 2024-04-20T11:35:16.738 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.736+0000 7fb3ca000640 1 -- 172.21.15.152:0/2182382567 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3c0007260 con 0x7fb3d40d32e0 2024-04-20T11:35:16.738 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.737+0000 7fb3d3200640 1 -- 172.21.15.152:0/2182382567 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3d40a1fc0 con 0x7fb3d40d32e0 2024-04-20T11:35:16.739 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.738+0000 7fb3d3200640 1 -- 172.21.15.152:0/2182382567 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3d4075e60 con 0x7fb3d40d32e0 2024-04-20T11:35:16.739 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.739+0000 7fb3ca000640 1 -- 172.21.15.152:0/2182382567 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 21) ==== 91550+0+0 (secure 0 0 0) 0x7fb3c0007400 con 0x7fb3d40d32e0 2024-04-20T11:35:16.741 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.740+0000 7fb3ca000640 1 --2- 172.21.15.152:0/2182382567 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb3b406ce90 0x7fb3b406f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:16.741 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.740+0000 7fb3ca000640 1 -- 172.21.15.152:0/2182382567 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 4857+0+0 (secure 0 0 0) 0x7fb3c008b750 con 0x7fb3d40d32e0 2024-04-20T11:35:16.741 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.741+0000 7fb3d1e00640 1 --2- 172.21.15.152:0/2182382567 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb3b406ce90 0x7fb3b406f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:16.742 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.742+0000 7fb3d1e00640 1 --2- 172.21.15.152:0/2182382567 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb3b406ce90 0x7fb3b406f350 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb3c4009a30 tx=0x7fb3c4008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:16.745 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.745+0000 7fb3ca000640 1 -- 172.21.15.152:0/2182382567 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fb3c00596f0 con 0x7fb3d40d32e0 2024-04-20T11:35:16.945 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:16.945+0000 7fb3d3200640 1 -- 172.21.15.152:0/2182382567 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fb3d40d29b0 con 0x7fb3d40d32e0 2024-04-20T11:35:17.750 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:17.749+0000 7fb3ca000640 1 -- 172.21.15.152:0/2182382567 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v128) ==== 138+0+0 (secure 0 0 0) 0x7fb3c0058ff0 con 0x7fb3d40d32e0 2024-04-20T11:35:17.751 INFO:teuthology.orchestra.run.smithi152.stderr:pool 'cephfs_metadata' created 2024-04-20T11:35:17.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:17.753+0000 7fb3d3200640 1 -- 172.21.15.152:0/2182382567 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb3b406ce90 msgr2=0x7fb3b406f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:17.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:17.753+0000 7fb3d3200640 1 --2- 172.21.15.152:0/2182382567 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb3b406ce90 0x7fb3b406f350 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb3c4009a30 tx=0x7fb3c4008040 comp rx=0 tx=0).stop 2024-04-20T11:35:17.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:17.753+0000 7fb3d3200640 1 -- 172.21.15.152:0/2182382567 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb3d40d32e0 msgr2=0x7fb3d40cd890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:17.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:17.753+0000 7fb3d3200640 1 --2- 172.21.15.152:0/2182382567 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb3d40d32e0 0x7fb3d40cd890 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7fb3c000c430 tx=0x7fb3c000c8f0 comp rx=0 tx=0).stop 2024-04-20T11:35:17.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:17.753+0000 7fb3d3200640 1 -- 172.21.15.152:0/2182382567 shutdown_connections 2024-04-20T11:35:17.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:17.753+0000 7fb3d3200640 1 --2- 172.21.15.152:0/2182382567 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb3b406ce90 0x7fb3b406f350 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:17.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:17.754+0000 7fb3d3200640 1 --2- 172.21.15.152:0/2182382567 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb3d415e900 0x7fb3d40cddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:17.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:17.754+0000 7fb3d3200640 1 --2- 172.21.15.152:0/2182382567 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb3d40d32e0 0x7fb3d40cd890 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:17.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:17.754+0000 7fb3d3200640 1 --2- 172.21.15.152:0/2182382567 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb3d406ca80 0x7fb3d4153a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:17.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:17.754+0000 7fb3d3200640 1 -- 172.21.15.152:0/2182382567 >> 172.21.15.152:0/2182382567 conn(0x7fb3d408bfd0 msgr2=0x7fb3d40b9d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:17.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:17.754+0000 7fb3d3200640 1 -- 172.21.15.152:0/2182382567 shutdown_connections 2024-04-20T11:35:17.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:17.754+0000 7fb3d3200640 1 -- 172.21.15.152:0/2182382567 wait complete. 2024-04-20T11:35:17.783 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-20T11:35:17.977 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:17.975+0000 7fa75be00640 1 Processor -- start 2024-04-20T11:35:17.977 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:17.976+0000 7fa75be00640 1 -- start start 2024-04-20T11:35:17.979 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:17.977+0000 7fa75be00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa75c159040 0x7fa75c0cf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:17.979 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:17.977+0000 7fa75be00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fa75c06f6a0 0x7fa75c0ce140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:17.979 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:17.977+0000 7fa75be00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa75c159420 con 0x7fa75c162530 2024-04-20T11:35:17.980 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:17.977+0000 7fa75be00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fa75c0ce680 con 0x7fa75c06f6a0 2024-04-20T11:35:17.980 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:17.977+0000 7fa75be00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa75c06fb00 con 0x7fa75c159040 2024-04-20T11:35:17.980 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:17.978+0000 7fa753e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fa75c06f6a0 0x7fa75c0ce140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:17.980 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:17.978+0000 7fa75b400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fa75c162530 0x7fa75c0b32d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:39728/0 (socket says 172.21.15.152:39728) 2024-04-20T11:35:17.980 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:17.978+0000 7fa75b400640 1 -- 172.21.15.152:0/2292903873 learned_addr learned my addr 172.21.15.152:0/2292903873 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:35:17.980 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:17.978+0000 7fa75aa00640 1 --2- 172.21.15.152:0/2292903873 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa75c159040 0x7fa75c0cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:17.980 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:17.978+0000 7fa753e00640 1 -- 172.21.15.152:0/2292903873 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa75c159040 msgr2=0x7fa75c0cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:17.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:17.978+0000 7fa753e00640 1 --2- 172.21.15.152:0/2292903873 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa75c159040 0x7fa75c0cf950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:17.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:17.978+0000 7fa753e00640 1 -- 172.21.15.152:0/2292903873 >> v1:172.21.15.152:6789/0 conn(0x7fa75c162530 legacy=0x7fa75c0b32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:17.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:17.978+0000 7fa753e00640 1 -- 172.21.15.152:0/2292903873 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa75c0ce8b0 con 0x7fa75c06f6a0 2024-04-20T11:35:17.984 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:17.980+0000 7fa753e00640 1 --2- 172.21.15.152:0/2292903873 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa75c06f6a0 0x7fa75c0ce140 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7fa7440097a0 tx=0x7fa74402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74185d3f86c4322f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:17.984 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:17.980+0000 7fa75a000640 1 -- 172.21.15.152:0/2292903873 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa744035020 con 0x7fa75c06f6a0 2024-04-20T11:35:17.984 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:17.980+0000 7fa75a000640 1 -- 172.21.15.152:0/2292903873 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa744031810 con 0x7fa75c06f6a0 2024-04-20T11:35:17.984 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:17.980+0000 7fa75a000640 1 -- 172.21.15.152:0/2292903873 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa744031be0 con 0x7fa75c06f6a0 2024-04-20T11:35:17.984 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:17.981+0000 7fa75be00640 1 -- 172.21.15.152:0/2292903873 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa75c06f6a0 msgr2=0x7fa75c0ce140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:17.984 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:17.981+0000 7fa75be00640 1 --2- 172.21.15.152:0/2292903873 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa75c06f6a0 0x7fa75c0ce140 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7fa7440097a0 tx=0x7fa74402b8b0 comp rx=0 tx=0).stop 2024-04-20T11:35:17.985 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:17.981+0000 7fa75be00640 1 -- 172.21.15.152:0/2292903873 shutdown_connections 2024-04-20T11:35:17.985 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:17.981+0000 7fa75be00640 1 --2- 172.21.15.152:0/2292903873 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa75c06f6a0 0x7fa75c0ce140 unknown :-1 s=CLOSED pgs=1744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:17.985 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:17.981+0000 7fa75be00640 1 --2- 172.21.15.152:0/2292903873 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa75c159040 0x7fa75c0cf950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:17.985 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:17.981+0000 7fa75be00640 1 -- 172.21.15.152:0/2292903873 >> 172.21.15.152:0/2292903873 conn(0x7fa75c0d0260 msgr2=0x7fa75c0d0670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:17.985 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:17.981+0000 7fa75be00640 1 -- 172.21.15.152:0/2292903873 shutdown_connections 2024-04-20T11:35:17.985 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:17.981+0000 7fa75be00640 1 -- 172.21.15.152:0/2292903873 wait complete. 2024-04-20T11:35:17.986 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:17.983+0000 7fa75be00640 1 Processor -- start 2024-04-20T11:35:17.986 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:17.983+0000 7fa75be00640 1 -- start start 2024-04-20T11:35:17.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:17.984+0000 7fa75be00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa75c06f6a0 0x7fa75c0b1200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:17.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:17.984+0000 7fa75be00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa75c159040 0x7fa75c0c9650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:17.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:17.984+0000 7fa75be00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa75c162530 0x7fa75c071c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:17.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:17.984+0000 7fa75be00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fa75c0cf150 con 0x7fa75c06f6a0 2024-04-20T11:35:17.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:17.984+0000 7fa75be00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa75c0cee70 con 0x7fa75c159040 2024-04-20T11:35:17.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:17.984+0000 7fa75be00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa75c0cf430 con 0x7fa75c162530 2024-04-20T11:35:17.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:17.984+0000 7fa75aa00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa75c06f6a0 0x7fa75c0b1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:17.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:17.984+0000 7fa75aa00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa75c06f6a0 0x7fa75c0b1200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52266/0 (socket says 172.21.15.152:52266) 2024-04-20T11:35:17.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:17.984+0000 7fa75aa00640 1 -- 172.21.15.152:0/974074071 learned_addr learned my addr 172.21.15.152:0/974074071 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:17.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:17.984+0000 7fa753e00640 1 --2- 172.21.15.152:0/974074071 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa75c159040 0x7fa75c0c9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:17.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:17.984+0000 7fa75b400640 1 --2- 172.21.15.152:0/974074071 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa75c162530 0x7fa75c071c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:17.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:17.984+0000 7fa75aa00640 1 -- 172.21.15.152:0/974074071 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa75c162530 msgr2=0x7fa75c071c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:17.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:17.984+0000 7fa75aa00640 1 --2- 172.21.15.152:0/974074071 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa75c162530 0x7fa75c071c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:17.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:17.984+0000 7fa75aa00640 1 -- 172.21.15.152:0/974074071 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa75c159040 msgr2=0x7fa75c0c9650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:17.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:17.984+0000 7fa75aa00640 1 --2- 172.21.15.152:0/974074071 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa75c159040 0x7fa75c0c9650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:17.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:17.984+0000 7fa75aa00640 1 -- 172.21.15.152:0/974074071 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa75c0726e0 con 0x7fa75c06f6a0 2024-04-20T11:35:17.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:17.984+0000 7fa753e00640 1 --2- 172.21.15.152:0/974074071 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa75c159040 0x7fa75c0c9650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:35:17.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:17.985+0000 7fa75aa00640 1 --2- 172.21.15.152:0/974074071 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa75c06f6a0 0x7fa75c0b1200 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7fa74c00d5f0 tx=0x7fa74c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:17.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:17.985+0000 7fa752a00640 1 -- 172.21.15.152:0/974074071 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa74c0125f0 con 0x7fa75c06f6a0 2024-04-20T11:35:17.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:17.985+0000 7fa752a00640 1 -- 172.21.15.152:0/974074071 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa74c019070 con 0x7fa75c06f6a0 2024-04-20T11:35:17.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:17.985+0000 7fa752a00640 1 -- 172.21.15.152:0/974074071 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa74c0129d0 con 0x7fa75c06f6a0 2024-04-20T11:35:17.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:17.985+0000 7fa75be00640 1 -- 172.21.15.152:0/974074071 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa75c072c70 con 0x7fa75c06f6a0 2024-04-20T11:35:17.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:17.985+0000 7fa75be00640 1 -- 172.21.15.152:0/974074071 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa75c09a6a0 con 0x7fa75c06f6a0 2024-04-20T11:35:17.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:17.986+0000 7fa75be00640 1 -- 172.21.15.152:0/974074071 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa75c075e60 con 0x7fa75c06f6a0 2024-04-20T11:35:17.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:17.988+0000 7fa752a00640 1 -- 172.21.15.152:0/974074071 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 21) ==== 91550+0+0 (secure 0 0 0) 0x7fa74c012b70 con 0x7fa75c06f6a0 2024-04-20T11:35:17.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:17.989+0000 7fa752a00640 1 --2- 172.21.15.152:0/974074071 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fa73806ce90 0x7fa73806f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:17.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:17.989+0000 7fa752a00640 1 -- 172.21.15.152:0/974074071 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 5227+0+0 (secure 0 0 0) 0x7fa74c08c7a0 con 0x7fa75c06f6a0 2024-04-20T11:35:17.995 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:17.989+0000 7fa753e00640 1 --2- 172.21.15.152:0/974074071 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fa73806ce90 0x7fa73806f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:17.995 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:17.991+0000 7fa753e00640 1 --2- 172.21.15.152:0/974074071 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fa73806ce90 0x7fa73806f350 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa744036000 tx=0x7fa744008030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:17.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:17.994+0000 7fa752a00640 1 -- 172.21.15.152:0/974074071 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fa74c05a5d0 con 0x7fa75c06f6a0 2024-04-20T11:35:18.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:18.199+0000 7fa75be00640 1 -- 172.21.15.152:0/974074071 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fa75c0cd990 con 0x7fa75c06f6a0 2024-04-20T11:35:18.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:18.757+0000 7fa752a00640 1 -- 172.21.15.152:0/974074071 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v129) ==== 170+0+0 (secure 0 0 0) 0x7fa74c059ed0 con 0x7fa75c06f6a0 2024-04-20T11:35:18.758 INFO:teuthology.orchestra.run.smithi152.stderr:pool 'cephfs_data' created 2024-04-20T11:35:18.762 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:18.761+0000 7fa75be00640 1 -- 172.21.15.152:0/974074071 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fa73806ce90 msgr2=0x7fa73806f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:18.762 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:18.761+0000 7fa75be00640 1 --2- 172.21.15.152:0/974074071 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fa73806ce90 0x7fa73806f350 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa744036000 tx=0x7fa744008030 comp rx=0 tx=0).stop 2024-04-20T11:35:18.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:18.761+0000 7fa75be00640 1 -- 172.21.15.152:0/974074071 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa75c06f6a0 msgr2=0x7fa75c0b1200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:18.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:18.761+0000 7fa75be00640 1 --2- 172.21.15.152:0/974074071 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa75c06f6a0 0x7fa75c0b1200 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7fa74c00d5f0 tx=0x7fa74c00dab0 comp rx=0 tx=0).stop 2024-04-20T11:35:18.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:18.762+0000 7fa75be00640 1 -- 172.21.15.152:0/974074071 shutdown_connections 2024-04-20T11:35:18.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:18.762+0000 7fa75be00640 1 --2- 172.21.15.152:0/974074071 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fa73806ce90 0x7fa73806f350 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:18.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:18.762+0000 7fa75be00640 1 --2- 172.21.15.152:0/974074071 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa75c162530 0x7fa75c071c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:18.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:18.762+0000 7fa75be00640 1 --2- 172.21.15.152:0/974074071 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa75c159040 0x7fa75c0c9650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:18.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:18.762+0000 7fa75be00640 1 --2- 172.21.15.152:0/974074071 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa75c06f6a0 0x7fa75c0b1200 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:18.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:18.762+0000 7fa75be00640 1 -- 172.21.15.152:0/974074071 >> 172.21.15.152:0/974074071 conn(0x7fa75c0d0260 msgr2=0x7fa75c1528b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:18.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:18.762+0000 7fa75be00640 1 -- 172.21.15.152:0/974074071 shutdown_connections 2024-04-20T11:35:18.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:18.762+0000 7fa75be00640 1 -- 172.21.15.152:0/974074071 wait complete. 2024-04-20T11:35:18.797 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-20T11:35:18.974 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:18.974+0000 7f9feaa00640 1 Processor -- start 2024-04-20T11:35:18.975 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:18.974+0000 7f9feaa00640 1 -- start start 2024-04-20T11:35:18.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:18.975+0000 7f9feaa00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f9fe415eb50 0x7f9fe40d0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:18.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:18.975+0000 7f9feaa00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f9fe4163230 0x7f9fe40d2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:18.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:18.975+0000 7f9feaa00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9fe40d1440 0x7f9fe40d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:18.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:18.975+0000 7f9feaa00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f9fe40d19c0 con 0x7f9fe4163230 2024-04-20T11:35:18.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:18.975+0000 7f9feaa00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f9fe4162750 con 0x7f9fe415eb50 2024-04-20T11:35:18.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:18.975+0000 7f9feaa00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f9fe4162a30 con 0x7f9fe40d1440 2024-04-20T11:35:18.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:18.975+0000 7f9fe8c00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f9fe4163230 0x7f9fe40d2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:18.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:18.976+0000 7f9fe8c00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f9fe4163230 0x7f9fe40d2e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52272/0 (socket says 172.21.15.152:52272) 2024-04-20T11:35:18.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:18.976+0000 7f9fe8c00640 1 -- 172.21.15.152:0/3861450803 learned_addr learned my addr 172.21.15.152:0/3861450803 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:18.977 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:18.976+0000 7f9fea000640 1 --2- 172.21.15.152:0/3861450803 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9fe40d1440 0x7f9fe40d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:18.977 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:18.976+0000 7f9fe9600640 1 --2- 172.21.15.152:0/3861450803 >> v2:172.21.15.175:3300/0 conn(0x7f9fe415eb50 0x7f9fe40d0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:18.977 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:18.976+0000 7f9fe8c00640 1 -- 172.21.15.152:0/3861450803 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9fe40d1440 msgr2=0x7f9fe40d33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:18.978 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:18.976+0000 7f9fe8c00640 1 --2- 172.21.15.152:0/3861450803 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9fe40d1440 0x7f9fe40d33d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:18.978 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:18.976+0000 7f9fe8c00640 1 -- 172.21.15.152:0/3861450803 >> v2:172.21.15.175:3300/0 conn(0x7f9fe415eb50 msgr2=0x7f9fe40d0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:18.978 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:18.976+0000 7f9fe8c00640 1 --2- 172.21.15.152:0/3861450803 >> v2:172.21.15.175:3300/0 conn(0x7f9fe415eb50 0x7f9fe40d0f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:18.978 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:18.976+0000 7f9fe8c00640 1 -- 172.21.15.152:0/3861450803 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fe4162d10 con 0x7f9fe4163230 2024-04-20T11:35:18.978 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:18.976+0000 7f9fe9600640 1 --2- 172.21.15.152:0/3861450803 >> v2:172.21.15.175:3300/0 conn(0x7f9fe415eb50 0x7f9fe40d0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:35:18.978 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:18.977+0000 7f9fe8c00640 1 --2- 172.21.15.152:0/3861450803 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9fe4163230 0x7f9fe40d2e90 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f9fe40b7080 tx=0x7f9fd402be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80f78f4b75e51c82 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:18.979 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:18.977+0000 7f9fe3e00640 1 -- 172.21.15.152:0/3861450803 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fd402ca30 con 0x7f9fe4163230 2024-04-20T11:35:18.979 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:18.977+0000 7f9fe3e00640 1 -- 172.21.15.152:0/3861450803 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fd402cbd0 con 0x7f9fe4163230 2024-04-20T11:35:18.979 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:18.977+0000 7f9fe3e00640 1 -- 172.21.15.152:0/3861450803 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fd40343f0 con 0x7f9fe4163230 2024-04-20T11:35:18.979 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:18.978+0000 7f9feaa00640 1 -- 172.21.15.152:0/3861450803 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9fe4163230 msgr2=0x7f9fe40d2e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:18.979 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:18.978+0000 7f9feaa00640 1 --2- 172.21.15.152:0/3861450803 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9fe4163230 0x7f9fe40d2e90 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f9fe40b7080 tx=0x7f9fd402be90 comp rx=0 tx=0).stop 2024-04-20T11:35:18.979 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:18.978+0000 7f9feaa00640 1 -- 172.21.15.152:0/3861450803 shutdown_connections 2024-04-20T11:35:18.979 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:18.978+0000 7f9feaa00640 1 --2- 172.21.15.152:0/3861450803 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9fe40d1440 0x7f9fe40d33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:18.979 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:18.978+0000 7f9feaa00640 1 --2- 172.21.15.152:0/3861450803 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9fe4163230 0x7f9fe40d2e90 unknown :-1 s=CLOSED pgs=1746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:18.979 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:18.978+0000 7f9feaa00640 1 --2- 172.21.15.152:0/3861450803 >> v2:172.21.15.175:3300/0 conn(0x7f9fe415eb50 0x7f9fe40d0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:18.980 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:18.978+0000 7f9feaa00640 1 -- 172.21.15.152:0/3861450803 >> 172.21.15.152:0/3861450803 conn(0x7f9fe408bfd0 msgr2=0x7f9fe40d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:18.980 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:18.978+0000 7f9feaa00640 1 -- 172.21.15.152:0/3861450803 shutdown_connections 2024-04-20T11:35:18.980 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:18.978+0000 7f9feaa00640 1 -- 172.21.15.152:0/3861450803 wait complete. 2024-04-20T11:35:18.980 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:18.980+0000 7f9feaa00640 1 Processor -- start 2024-04-20T11:35:18.980 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:18.980+0000 7f9feaa00640 1 -- start start 2024-04-20T11:35:18.982 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:18.981+0000 7f9feaa00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9fe40d1440 0x7f9fe4152320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:18.982 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:18.981+0000 7f9feaa00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9fe415eb50 0x7f9fe4161380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:18.982 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:18.981+0000 7f9feaa00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9fe4163230 0x7f9fe41618c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:18.982 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:18.981+0000 7f9feaa00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe415f0b0 con 0x7f9fe4163230 2024-04-20T11:35:18.982 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:18.981+0000 7f9feaa00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe415ef30 con 0x7f9fe415eb50 2024-04-20T11:35:18.982 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:18.981+0000 7f9feaa00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f9fe415f230 con 0x7f9fe40d1440 2024-04-20T11:35:18.982 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:18.981+0000 7f9fea000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9fe4163230 0x7f9fe41618c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:18.982 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:18.981+0000 7f9fea000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9fe4163230 0x7f9fe41618c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52284/0 (socket says 172.21.15.152:52284) 2024-04-20T11:35:18.982 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:18.981+0000 7f9fea000640 1 -- 172.21.15.152:0/3928662935 learned_addr learned my addr 172.21.15.152:0/3928662935 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:18.982 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:18.982+0000 7f9fe9600640 1 --2- 172.21.15.152:0/3928662935 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9fe40d1440 0x7f9fe4152320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:18.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:18.982+0000 7f9fe8c00640 1 --2- 172.21.15.152:0/3928662935 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9fe415eb50 0x7f9fe4161380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:18.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:18.982+0000 7f9fea000640 1 -- 172.21.15.152:0/3928662935 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9fe40d1440 msgr2=0x7f9fe4152320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:18.984 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:18.982+0000 7f9fea000640 1 --2- 172.21.15.152:0/3928662935 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9fe40d1440 0x7f9fe4152320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:18.984 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:18.982+0000 7f9fea000640 1 -- 172.21.15.152:0/3928662935 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9fe415eb50 msgr2=0x7f9fe4161380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:18.984 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:18.982+0000 7f9fea000640 1 --2- 172.21.15.152:0/3928662935 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9fe415eb50 0x7f9fe4161380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:18.984 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:18.982+0000 7f9fea000640 1 -- 172.21.15.152:0/3928662935 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fe40b44f0 con 0x7f9fe4163230 2024-04-20T11:35:18.984 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:18.982+0000 7f9fea000640 1 --2- 172.21.15.152:0/3928662935 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9fe4163230 0x7f9fe41618c0 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc00c5a0 tx=0x7f9fdc00ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:18.984 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:18.982+0000 7f9fe1600640 1 -- 172.21.15.152:0/3928662935 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fdc015070 con 0x7f9fe4163230 2024-04-20T11:35:18.984 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:18.982+0000 7f9fe1600640 1 -- 172.21.15.152:0/3928662935 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fdc004220 con 0x7f9fe4163230 2024-04-20T11:35:18.984 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:18.982+0000 7f9fe1600640 1 -- 172.21.15.152:0/3928662935 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fdc003610 con 0x7f9fe4163230 2024-04-20T11:35:18.984 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:18.983+0000 7f9feaa00640 1 -- 172.21.15.152:0/3928662935 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fe40b4790 con 0x7f9fe4163230 2024-04-20T11:35:18.984 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:18.983+0000 7f9feaa00640 1 -- 172.21.15.152:0/3928662935 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fe40b4d50 con 0x7f9fe4163230 2024-04-20T11:35:18.985 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:18.984+0000 7f9feaa00640 1 -- 172.21.15.152:0/3928662935 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fe4075e60 con 0x7f9fe4163230 2024-04-20T11:35:18.986 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:18.986+0000 7f9fe1600640 1 -- 172.21.15.152:0/3928662935 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 21) ==== 91550+0+0 (secure 0 0 0) 0x7f9fdc0037b0 con 0x7f9fe4163230 2024-04-20T11:35:18.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:18.986+0000 7f9fe1600640 1 --2- 172.21.15.152:0/3928662935 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f9fc806cec0 0x7f9fc806f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:18.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:18.987+0000 7f9fe1600640 1 -- 172.21.15.152:0/3928662935 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5609+0+0 (secure 0 0 0) 0x7f9fdc08c7b0 con 0x7f9fe4163230 2024-04-20T11:35:18.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:18.987+0000 7f9fe9600640 1 --2- 172.21.15.152:0/3928662935 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f9fc806cec0 0x7f9fc806f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:18.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:18.988+0000 7f9fe9600640 1 --2- 172.21.15.152:0/3928662935 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f9fc806cec0 0x7f9fc806f380 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f9fd800a990 tx=0x7f9fd8008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:18.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:18.990+0000 7f9fe1600640 1 -- 172.21.15.152:0/3928662935 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f9fdc05a3c0 con 0x7f9fe4163230 2024-04-20T11:35:19.230 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:19.229+0000 7f9feaa00640 1 -- 172.21.15.152:0/3928662935 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f9fe408c680 con 0x7f9fe4163230 2024-04-20T11:35:20.092 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.091+0000 7f9fe1600640 1 -- 172.21.15.152:0/3928662935 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 20 and data pool 21 v82) ==== 174+0+0 (secure 0 0 0) 0x7f9fdc059cc0 con 0x7f9fe4163230 2024-04-20T11:35:20.092 INFO:teuthology.orchestra.run.smithi152.stderr:new fs with metadata pool 20 and data pool 21 2024-04-20T11:35:20.096 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.095+0000 7f9feaa00640 1 -- 172.21.15.152:0/3928662935 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f9fc806cec0 msgr2=0x7f9fc806f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:20.096 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.095+0000 7f9feaa00640 1 --2- 172.21.15.152:0/3928662935 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f9fc806cec0 0x7f9fc806f380 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f9fd800a990 tx=0x7f9fd8008040 comp rx=0 tx=0).stop 2024-04-20T11:35:20.096 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.095+0000 7f9feaa00640 1 -- 172.21.15.152:0/3928662935 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9fe4163230 msgr2=0x7f9fe41618c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:20.096 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.095+0000 7f9feaa00640 1 --2- 172.21.15.152:0/3928662935 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9fe4163230 0x7f9fe41618c0 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc00c5a0 tx=0x7f9fdc00ca60 comp rx=0 tx=0).stop 2024-04-20T11:35:20.096 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.095+0000 7f9feaa00640 1 -- 172.21.15.152:0/3928662935 shutdown_connections 2024-04-20T11:35:20.096 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.095+0000 7f9feaa00640 1 --2- 172.21.15.152:0/3928662935 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f9fc806cec0 0x7f9fc806f380 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:20.096 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.095+0000 7f9feaa00640 1 --2- 172.21.15.152:0/3928662935 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9fe4163230 0x7f9fe41618c0 unknown :-1 s=CLOSED pgs=1747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:20.096 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.095+0000 7f9feaa00640 1 --2- 172.21.15.152:0/3928662935 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9fe415eb50 0x7f9fe4161380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:20.096 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.095+0000 7f9feaa00640 1 --2- 172.21.15.152:0/3928662935 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9fe40d1440 0x7f9fe4152320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:20.096 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.095+0000 7f9feaa00640 1 -- 172.21.15.152:0/3928662935 >> 172.21.15.152:0/3928662935 conn(0x7f9fe408bfd0 msgr2=0x7f9fe40cf980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:20.097 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.096+0000 7f9feaa00640 1 -- 172.21.15.152:0/3928662935 shutdown_connections 2024-04-20T11:35:20.097 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.096+0000 7f9feaa00640 1 -- 172.21.15.152:0/3928662935 wait complete. 2024-04-20T11:35:20.124 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:35:20.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.299+0000 7ff1ebe00640 1 Processor -- start 2024-04-20T11:35:20.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.299+0000 7ff1ebe00640 1 -- start start 2024-04-20T11:35:20.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.300+0000 7ff1ebe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff1ec15d1e0 0x7ff1ec160b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:20.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.300+0000 7ff1ebe00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7ff1ec15d5c0 0x7ff1ec0cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:20.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.300+0000 7ff1ebe00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1ec0ce8c0 con 0x7ff1ec071cd0 2024-04-20T11:35:20.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.301+0000 7ff1ebe00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7ff1ec1600b0 con 0x7ff1ec15d5c0 2024-04-20T11:35:20.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.301+0000 7ff1ebe00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff1ec160390 con 0x7ff1ec15d1e0 2024-04-20T11:35:20.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.301+0000 7ff1ea000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7ff1ec15d5c0 0x7ff1ec0cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:20.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.301+0000 7ff1eb400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7ff1ec071cd0 0x7ff1ec0ccd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:39734/0 (socket says 172.21.15.152:39734) 2024-04-20T11:35:20.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.301+0000 7ff1eb400640 1 -- 172.21.15.152:0/944656347 learned_addr learned my addr 172.21.15.152:0/944656347 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:35:20.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.301+0000 7ff1eaa00640 1 --2- 172.21.15.152:0/944656347 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff1ec15d1e0 0x7ff1ec160b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:20.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.301+0000 7ff1ea000640 1 -- 172.21.15.152:0/944656347 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff1ec15d1e0 msgr2=0x7ff1ec160b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:20.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.301+0000 7ff1ea000640 1 --2- 172.21.15.152:0/944656347 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff1ec15d1e0 0x7ff1ec160b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:20.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.301+0000 7ff1ea000640 1 -- 172.21.15.152:0/944656347 >> v1:172.21.15.152:6789/0 conn(0x7ff1ec071cd0 legacy=0x7ff1ec0ccd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:20.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.301+0000 7ff1ea000640 1 -- 172.21.15.152:0/944656347 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1ec160670 con 0x7ff1ec15d5c0 2024-04-20T11:35:20.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.302+0000 7ff1ea000640 1 --2- 172.21.15.152:0/944656347 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff1ec15d5c0 0x7ff1ec0cdfe0 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7ff1d40097a0 tx=0x7ff1d402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f20e65db564c0b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:20.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.303+0000 7ff1e9600640 1 -- 172.21.15.152:0/944656347 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1d4035020 con 0x7ff1ec15d5c0 2024-04-20T11:35:20.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.303+0000 7ff1e9600640 1 -- 172.21.15.152:0/944656347 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1d4031830 con 0x7ff1ec15d5c0 2024-04-20T11:35:20.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.303+0000 7ff1e9600640 1 -- 172.21.15.152:0/944656347 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1d4031c00 con 0x7ff1ec15d5c0 2024-04-20T11:35:20.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.303+0000 7ff1ebe00640 1 -- 172.21.15.152:0/944656347 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff1ec15d5c0 msgr2=0x7ff1ec0cdfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:20.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.303+0000 7ff1ebe00640 1 --2- 172.21.15.152:0/944656347 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff1ec15d5c0 0x7ff1ec0cdfe0 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7ff1d40097a0 tx=0x7ff1d402b8d0 comp rx=0 tx=0).stop 2024-04-20T11:35:20.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.303+0000 7ff1ebe00640 1 -- 172.21.15.152:0/944656347 shutdown_connections 2024-04-20T11:35:20.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.303+0000 7ff1ebe00640 1 --2- 172.21.15.152:0/944656347 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff1ec15d5c0 0x7ff1ec0cdfe0 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:20.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.303+0000 7ff1ebe00640 1 --2- 172.21.15.152:0/944656347 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff1ec15d1e0 0x7ff1ec160b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:20.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.303+0000 7ff1ebe00640 1 -- 172.21.15.152:0/944656347 >> 172.21.15.152:0/944656347 conn(0x7ff1ec067ae0 msgr2=0x7ff1ec06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:20.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.303+0000 7ff1ebe00640 1 -- 172.21.15.152:0/944656347 shutdown_connections 2024-04-20T11:35:20.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.304+0000 7ff1ebe00640 1 -- 172.21.15.152:0/944656347 wait complete. 2024-04-20T11:35:20.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.305+0000 7ff1ebe00640 1 Processor -- start 2024-04-20T11:35:20.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.305+0000 7ff1ebe00640 1 -- start start 2024-04-20T11:35:20.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.306+0000 7ff1ebe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1ec071cd0 0x7ff1ec1599f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:20.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.306+0000 7ff1ebe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff1ec15d1e0 0x7ff1ec157a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:20.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.306+0000 7ff1ebe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff1ec15d5c0 0x7ff1ec1f6f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:20.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.306+0000 7ff1ebe00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7ff1ec0ce6a0 con 0x7ff1ec15d1e0 2024-04-20T11:35:20.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.306+0000 7ff1ebe00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff1ec0ce520 con 0x7ff1ec071cd0 2024-04-20T11:35:20.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.306+0000 7ff1ebe00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff1ec072160 con 0x7ff1ec15d5c0 2024-04-20T11:35:20.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.306+0000 7ff1ea000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff1ec15d1e0 0x7ff1ec157a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:20.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.306+0000 7ff1ea000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff1ec15d1e0 0x7ff1ec157a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52304/0 (socket says 172.21.15.152:52304) 2024-04-20T11:35:20.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.306+0000 7ff1ea000640 1 -- 172.21.15.152:0/4090711921 learned_addr learned my addr 172.21.15.152:0/4090711921 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:20.308 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.307+0000 7ff1ea000640 1 -- 172.21.15.152:0/4090711921 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff1ec15d5c0 msgr2=0x7ff1ec1f6f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:20.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.307+0000 7ff1eb400640 1 --2- 172.21.15.152:0/4090711921 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff1ec15d5c0 0x7ff1ec1f6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:20.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.307+0000 7ff1eaa00640 1 --2- 172.21.15.152:0/4090711921 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1ec071cd0 0x7ff1ec1599f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:20.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.307+0000 7ff1ea000640 1 --2- 172.21.15.152:0/4090711921 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff1ec15d5c0 0x7ff1ec1f6f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:20.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.307+0000 7ff1ea000640 1 -- 172.21.15.152:0/4090711921 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1ec071cd0 msgr2=0x7ff1ec1599f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:20.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.307+0000 7ff1ea000640 1 --2- 172.21.15.152:0/4090711921 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1ec071cd0 0x7ff1ec1599f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:20.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.307+0000 7ff1ea000640 1 -- 172.21.15.152:0/4090711921 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1ec1f74b0 con 0x7ff1ec15d1e0 2024-04-20T11:35:20.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.307+0000 7ff1ea000640 1 --2- 172.21.15.152:0/4090711921 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff1ec15d1e0 0x7ff1ec157a60 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7ff1d4034040 tx=0x7ff1d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:20.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.307+0000 7ff1e2a00640 1 -- 172.21.15.152:0/4090711921 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1d403b9c0 con 0x7ff1ec15d1e0 2024-04-20T11:35:20.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.307+0000 7ff1e2a00640 1 -- 172.21.15.152:0/4090711921 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1d403bba0 con 0x7ff1ec15d1e0 2024-04-20T11:35:20.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.308+0000 7ff1e2a00640 1 -- 172.21.15.152:0/4090711921 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1d40453f0 con 0x7ff1ec15d1e0 2024-04-20T11:35:20.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.308+0000 7ff1ebe00640 1 -- 172.21.15.152:0/4090711921 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1ec1581d0 con 0x7ff1ec15d1e0 2024-04-20T11:35:20.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.308+0000 7ff1ebe00640 1 -- 172.21.15.152:0/4090711921 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1ec158790 con 0x7ff1ec15d1e0 2024-04-20T11:35:20.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.309+0000 7ff1ebe00640 1 -- 172.21.15.152:0/4090711921 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1ec075e60 con 0x7ff1ec15d1e0 2024-04-20T11:35:20.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.310+0000 7ff1e2a00640 1 -- 172.21.15.152:0/4090711921 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 21) ==== 91550+0+0 (secure 0 0 0) 0x7ff1d404c060 con 0x7ff1ec15d1e0 2024-04-20T11:35:20.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.310+0000 7ff1e2a00640 1 --2- 172.21.15.152:0/4090711921 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7ff1c806ce90 0x7ff1c806f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:20.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.311+0000 7ff1e2a00640 1 -- 172.21.15.152:0/4090711921 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5709+0+0 (secure 0 0 0) 0x7ff1d40ae3d0 con 0x7ff1ec15d1e0 2024-04-20T11:35:20.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.311+0000 7ff1eaa00640 1 --2- 172.21.15.152:0/4090711921 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7ff1c806ce90 0x7ff1c806f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:20.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.312+0000 7ff1eaa00640 1 --2- 172.21.15.152:0/4090711921 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7ff1c806ce90 0x7ff1c806f350 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc006fd0 tx=0x7ff1dc008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:20.318 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.315+0000 7ff1e2a00640 1 -- 172.21.15.152:0/4090711921 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ff1d4040070 con 0x7ff1ec15d1e0 2024-04-20T11:35:20.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.516+0000 7ff1ebe00640 1 -- 172.21.15.152:0/4090711921 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff1ec072c30 con 0x7ff1ec15d1e0 2024-04-20T11:35:20.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.519+0000 7ff1e2a00640 1 -- 172.21.15.152:0/4090711921 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v130) ==== 74+0+16752 (secure 0 0 0) 0x7ff1d407be40 con 0x7ff1ec15d1e0 2024-04-20T11:35:20.520 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:35:20.520 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":130,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:35:19.231721+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:35:16.963520+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"130","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-04-20T11:35:18.215890+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"130","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:35:20.522 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.521+0000 7ff1ebe00640 1 -- 172.21.15.152:0/4090711921 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7ff1c806ce90 msgr2=0x7ff1c806f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:20.522 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.521+0000 7ff1ebe00640 1 --2- 172.21.15.152:0/4090711921 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7ff1c806ce90 0x7ff1c806f350 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc006fd0 tx=0x7ff1dc008040 comp rx=0 tx=0).stop 2024-04-20T11:35:20.522 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.521+0000 7ff1ebe00640 1 -- 172.21.15.152:0/4090711921 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff1ec15d1e0 msgr2=0x7ff1ec157a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:20.522 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.521+0000 7ff1ebe00640 1 --2- 172.21.15.152:0/4090711921 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff1ec15d1e0 0x7ff1ec157a60 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7ff1d4034040 tx=0x7ff1d40046c0 comp rx=0 tx=0).stop 2024-04-20T11:35:20.522 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.521+0000 7ff1ebe00640 1 -- 172.21.15.152:0/4090711921 shutdown_connections 2024-04-20T11:35:20.522 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.521+0000 7ff1ebe00640 1 --2- 172.21.15.152:0/4090711921 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7ff1c806ce90 0x7ff1c806f350 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:20.523 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.521+0000 7ff1ebe00640 1 --2- 172.21.15.152:0/4090711921 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff1ec15d5c0 0x7ff1ec1f6f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:20.523 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.521+0000 7ff1ebe00640 1 --2- 172.21.15.152:0/4090711921 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff1ec15d1e0 0x7ff1ec157a60 unknown :-1 s=CLOSED pgs=1750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:20.523 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.521+0000 7ff1ebe00640 1 --2- 172.21.15.152:0/4090711921 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1ec071cd0 0x7ff1ec1599f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:20.523 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.521+0000 7ff1ebe00640 1 -- 172.21.15.152:0/4090711921 >> 172.21.15.152:0/4090711921 conn(0x7ff1ec067ae0 msgr2=0x7ff1ec0b1880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:20.523 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.522+0000 7ff1ebe00640 1 -- 172.21.15.152:0/4090711921 shutdown_connections 2024-04-20T11:35:20.523 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.522+0000 7ff1ebe00640 1 -- 172.21.15.152:0/4090711921 wait complete. 2024-04-20T11:35:20.542 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:35:20.742 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.741+0000 7fcf52c00640 1 Processor -- start 2024-04-20T11:35:20.742 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.741+0000 7fcf52c00640 1 -- start start 2024-04-20T11:35:20.743 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.743+0000 7fcf52c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcf4c15d1e0 0x7fcf4c160b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:20.743 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.743+0000 7fcf52c00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fcf4c15d5c0 0x7fcf4c0cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:20.743 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.743+0000 7fcf52c00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf4c0ce8c0 con 0x7fcf4c071cd0 2024-04-20T11:35:20.743 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.743+0000 7fcf52c00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fcf4c1600b0 con 0x7fcf4c15d5c0 2024-04-20T11:35:20.743 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.743+0000 7fcf52c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fcf4c160390 con 0x7fcf4c15d1e0 2024-04-20T11:35:20.744 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.743+0000 7fcf50e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fcf4c15d5c0 0x7fcf4c0cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:20.744 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.743+0000 7fcf50e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fcf4c15d5c0 0x7fcf4c0cdfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52306/0 (socket says 172.21.15.152:52306) 2024-04-20T11:35:20.744 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.743+0000 7fcf50e00640 1 -- 172.21.15.152:0/2221251728 learned_addr learned my addr 172.21.15.152:0/2221251728 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:20.744 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.743+0000 7fcf51800640 1 --2- 172.21.15.152:0/2221251728 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcf4c15d1e0 0x7fcf4c160b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:20.745 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.744+0000 7fcf50e00640 1 -- 172.21.15.152:0/2221251728 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcf4c15d1e0 msgr2=0x7fcf4c160b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:20.745 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.744+0000 7fcf50e00640 1 --2- 172.21.15.152:0/2221251728 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcf4c15d1e0 0x7fcf4c160b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:20.745 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.744+0000 7fcf50e00640 1 -- 172.21.15.152:0/2221251728 >> v1:172.21.15.175:6789/0 conn(0x7fcf4c071cd0 legacy=0x7fcf4c0ccd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:20.745 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.744+0000 7fcf50e00640 1 -- 172.21.15.152:0/2221251728 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf4c160670 con 0x7fcf4c15d5c0 2024-04-20T11:35:20.745 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.744+0000 7fcf51800640 1 --2- 172.21.15.152:0/2221251728 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcf4c15d1e0 0x7fcf4c160b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:35:20.745 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.745+0000 7fcf50e00640 1 --2- 172.21.15.152:0/2221251728 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcf4c15d5c0 0x7fcf4c0cdfe0 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7fcf38009870 tx=0x7fcf3802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df8538ec48a6efa1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:20.746 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.745+0000 7fcf4be00640 1 -- 172.21.15.152:0/2221251728 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf38035020 con 0x7fcf4c15d5c0 2024-04-20T11:35:20.746 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.745+0000 7fcf4be00640 1 -- 172.21.15.152:0/2221251728 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf38031640 con 0x7fcf4c15d5c0 2024-04-20T11:35:20.746 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.745+0000 7fcf4be00640 1 -- 172.21.15.152:0/2221251728 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf3802d040 con 0x7fcf4c15d5c0 2024-04-20T11:35:20.747 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.746+0000 7fcf52c00640 1 -- 172.21.15.152:0/2221251728 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcf4c15d5c0 msgr2=0x7fcf4c0cdfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:20.747 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.746+0000 7fcf52c00640 1 --2- 172.21.15.152:0/2221251728 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcf4c15d5c0 0x7fcf4c0cdfe0 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7fcf38009870 tx=0x7fcf3802b8f0 comp rx=0 tx=0).stop 2024-04-20T11:35:20.747 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.746+0000 7fcf52c00640 1 -- 172.21.15.152:0/2221251728 shutdown_connections 2024-04-20T11:35:20.747 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.746+0000 7fcf52c00640 1 --2- 172.21.15.152:0/2221251728 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcf4c15d5c0 0x7fcf4c0cdfe0 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:20.747 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.746+0000 7fcf52c00640 1 --2- 172.21.15.152:0/2221251728 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcf4c15d1e0 0x7fcf4c160b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:20.747 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.746+0000 7fcf52c00640 1 -- 172.21.15.152:0/2221251728 >> 172.21.15.152:0/2221251728 conn(0x7fcf4c067ae0 msgr2=0x7fcf4c06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:20.747 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.746+0000 7fcf52c00640 1 -- 172.21.15.152:0/2221251728 shutdown_connections 2024-04-20T11:35:20.747 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.746+0000 7fcf52c00640 1 -- 172.21.15.152:0/2221251728 wait complete. 2024-04-20T11:35:20.748 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.748+0000 7fcf52c00640 1 Processor -- start 2024-04-20T11:35:20.748 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.748+0000 7fcf52c00640 1 -- start start 2024-04-20T11:35:20.750 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.749+0000 7fcf52c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcf4c071cd0 0x7fcf4c15f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:20.750 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.749+0000 7fcf52c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcf4c15d1e0 0x7fcf4c13b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:20.750 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.749+0000 7fcf52c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcf4c15d5c0 0x7fcf4c14c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:20.750 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.749+0000 7fcf52c00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fcf4c0ce6a0 con 0x7fcf4c071cd0 2024-04-20T11:35:20.750 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.749+0000 7fcf52c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcf4c0ce520 con 0x7fcf4c15d5c0 2024-04-20T11:35:20.750 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.749+0000 7fcf52c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fcf4c072160 con 0x7fcf4c15d1e0 2024-04-20T11:35:20.750 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.749+0000 7fcf51800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcf4c071cd0 0x7fcf4c15f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:20.750 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.749+0000 7fcf51800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcf4c071cd0 0x7fcf4c15f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52312/0 (socket says 172.21.15.152:52312) 2024-04-20T11:35:20.750 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.749+0000 7fcf51800640 1 -- 172.21.15.152:0/2839997856 learned_addr learned my addr 172.21.15.152:0/2839997856 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:20.751 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.749+0000 7fcf51800640 1 -- 172.21.15.152:0/2839997856 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcf4c15d1e0 msgr2=0x7fcf4c13b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:20.751 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.749+0000 7fcf51800640 1 --2- 172.21.15.152:0/2839997856 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcf4c15d1e0 0x7fcf4c13b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:20.751 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.749+0000 7fcf52200640 1 --2- 172.21.15.152:0/2839997856 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcf4c15d5c0 0x7fcf4c14c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:20.751 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.749+0000 7fcf51800640 1 -- 172.21.15.152:0/2839997856 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcf4c15d5c0 msgr2=0x7fcf4c14c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:20.751 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.749+0000 7fcf51800640 1 --2- 172.21.15.152:0/2839997856 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcf4c15d5c0 0x7fcf4c14c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:20.751 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.750+0000 7fcf51800640 1 -- 172.21.15.152:0/2839997856 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf4c14e360 con 0x7fcf4c071cd0 2024-04-20T11:35:20.752 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.750+0000 7fcf51800640 1 --2- 172.21.15.152:0/2839997856 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcf4c071cd0 0x7fcf4c15f7c0 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7fcf4000c820 tx=0x7fcf4000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:20.752 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.750+0000 7fcf49600640 1 -- 172.21.15.152:0/2839997856 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf40015070 con 0x7fcf4c071cd0 2024-04-20T11:35:20.752 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.750+0000 7fcf49600640 1 -- 172.21.15.152:0/2839997856 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf40007a30 con 0x7fcf4c071cd0 2024-04-20T11:35:20.752 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.750+0000 7fcf49600640 1 -- 172.21.15.152:0/2839997856 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf40007e00 con 0x7fcf4c071cd0 2024-04-20T11:35:20.752 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.750+0000 7fcf52c00640 1 -- 172.21.15.152:0/2839997856 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf4c14e600 con 0x7fcf4c071cd0 2024-04-20T11:35:20.752 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.750+0000 7fcf52c00640 1 -- 172.21.15.152:0/2839997856 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf4c14ebc0 con 0x7fcf4c071cd0 2024-04-20T11:35:20.752 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.752+0000 7fcf52c00640 1 -- 172.21.15.152:0/2839997856 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf4c075e60 con 0x7fcf4c071cd0 2024-04-20T11:35:20.753 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.753+0000 7fcf49600640 1 -- 172.21.15.152:0/2839997856 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 21) ==== 91550+0+0 (secure 0 0 0) 0x7fcf40018070 con 0x7fcf4c071cd0 2024-04-20T11:35:20.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.753+0000 7fcf49600640 1 --2- 172.21.15.152:0/2839997856 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fcf2c06cec0 0x7fcf2c06f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:20.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.754+0000 7fcf49600640 1 -- 172.21.15.152:0/2839997856 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5709+0+0 (secure 0 0 0) 0x7fcf4008c6a0 con 0x7fcf4c071cd0 2024-04-20T11:35:20.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.754+0000 7fcf50e00640 1 --2- 172.21.15.152:0/2839997856 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fcf2c06cec0 0x7fcf2c06f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:20.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.755+0000 7fcf50e00640 1 --2- 172.21.15.152:0/2839997856 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fcf2c06cec0 0x7fcf2c06f380 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fcf38036000 tx=0x7fcf38030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:20.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.758+0000 7fcf49600640 1 -- 172.21.15.152:0/2839997856 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fcf4005a0c0 con 0x7fcf4c071cd0 2024-04-20T11:35:20.957 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.956+0000 7fcf52c00640 1 -- 172.21.15.152:0/2839997856 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcf4c072c90 con 0x7fcf4c071cd0 2024-04-20T11:35:20.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.958+0000 7fcf49600640 1 -- 172.21.15.152:0/2839997856 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v131) ==== 74+0+16739 (secure 0 0 0) 0x7fcf400599c0 con 0x7fcf4c071cd0 2024-04-20T11:35:20.959 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:35:20.959 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":131,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:35:20.781260+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:35:16.963520+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"130","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-04-20T11:35:18.215890+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:35:20.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.960+0000 7fcf52c00640 1 -- 172.21.15.152:0/2839997856 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fcf2c06cec0 msgr2=0x7fcf2c06f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:20.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.960+0000 7fcf52c00640 1 --2- 172.21.15.152:0/2839997856 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fcf2c06cec0 0x7fcf2c06f380 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fcf38036000 tx=0x7fcf38030000 comp rx=0 tx=0).stop 2024-04-20T11:35:20.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.960+0000 7fcf52c00640 1 -- 172.21.15.152:0/2839997856 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcf4c071cd0 msgr2=0x7fcf4c15f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:20.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.960+0000 7fcf52c00640 1 --2- 172.21.15.152:0/2839997856 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcf4c071cd0 0x7fcf4c15f7c0 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7fcf4000c820 tx=0x7fcf4000cc20 comp rx=0 tx=0).stop 2024-04-20T11:35:20.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.960+0000 7fcf52c00640 1 -- 172.21.15.152:0/2839997856 shutdown_connections 2024-04-20T11:35:20.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.960+0000 7fcf52c00640 1 --2- 172.21.15.152:0/2839997856 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fcf2c06cec0 0x7fcf2c06f380 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:20.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.960+0000 7fcf52c00640 1 --2- 172.21.15.152:0/2839997856 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcf4c15d5c0 0x7fcf4c14c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:20.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.960+0000 7fcf52c00640 1 --2- 172.21.15.152:0/2839997856 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcf4c15d1e0 0x7fcf4c13b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:20.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.960+0000 7fcf52c00640 1 --2- 172.21.15.152:0/2839997856 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcf4c071cd0 0x7fcf4c15f7c0 unknown :-1 s=CLOSED pgs=1752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:20.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.960+0000 7fcf52c00640 1 -- 172.21.15.152:0/2839997856 >> 172.21.15.152:0/2839997856 conn(0x7fcf4c067ae0 msgr2=0x7fcf4c06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:20.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.961+0000 7fcf52c00640 1 -- 172.21.15.152:0/2839997856 shutdown_connections 2024-04-20T11:35:20.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:20.961+0000 7fcf52c00640 1 -- 172.21.15.152:0/2839997856 wait complete. 2024-04-20T11:35:20.980 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-20T11:35:21.184 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:21.184+0000 7fefabe00640 1 Processor -- start 2024-04-20T11:35:21.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:21.184+0000 7fefabe00640 1 -- start start 2024-04-20T11:35:21.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:21.186+0000 7fefabe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fefa415b510 0x7fefa415fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:21.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:21.186+0000 7fefabe00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fefa40b2880 0x7fefa40b3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:21.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:21.186+0000 7fefabe00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fefa415f1c0 con 0x7fefa40cfc40 2024-04-20T11:35:21.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:21.186+0000 7fefabe00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fefa406f420 con 0x7fefa40b2880 2024-04-20T11:35:21.187 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:21.186+0000 7fefabe00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fefa40b4430 con 0x7fefa415b510 2024-04-20T11:35:21.187 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:21.186+0000 7fefa8a00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fefa40cfc40 0x7fefa40b2170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:39736/0 (socket says 172.21.15.152:39736) 2024-04-20T11:35:21.187 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:21.186+0000 7fefa8a00640 1 -- 172.21.15.152:0/1610444784 learned_addr learned my addr 172.21.15.152:0/1610444784 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:35:21.188 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:21.186+0000 7fefa9e00640 1 --2- 172.21.15.152:0/1610444784 >> v2:172.21.15.175:3300/0 conn(0x7fefa40b2880 0x7fefa40b3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:21.188 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:21.186+0000 7fefa9400640 1 --2- 172.21.15.152:0/1610444784 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fefa415b510 0x7fefa415fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:21.188 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:21.187+0000 7fefa3e00640 1 -- 172.21.15.152:0/1610444784 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2661763158 0 0) 0x7fefa415f1c0 con 0x7fefa40cfc40 2024-04-20T11:35:21.188 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:21.187+0000 7fefa3e00640 1 -- 172.21.15.152:0/1610444784 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fef8c003520 con 0x7fefa40cfc40 2024-04-20T11:35:21.188 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:21.187+0000 7fefa3e00640 1 -- 172.21.15.152:0/1610444784 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2048704083 0 0) 0x7fef8c003520 con 0x7fefa40cfc40 2024-04-20T11:35:21.188 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:21.187+0000 7fefa3e00640 1 -- 172.21.15.152:0/1610444784 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fefa415b510 msgr2=0x7fefa415fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:21.188 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:21.187+0000 7fefa3e00640 1 --2- 172.21.15.152:0/1610444784 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fefa415b510 0x7fefa415fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:21.188 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:21.187+0000 7fefa3e00640 1 -- 172.21.15.152:0/1610444784 >> v2:172.21.15.175:3300/0 conn(0x7fefa40b2880 msgr2=0x7fefa40b3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:21.188 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:21.187+0000 7fefa3e00640 1 --2- 172.21.15.152:0/1610444784 >> v2:172.21.15.175:3300/0 conn(0x7fefa40b2880 0x7fefa40b3ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:21.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:21.187+0000 7fefa3e00640 1 -- 172.21.15.152:0/1610444784 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefa406f670 con 0x7fefa40cfc40 2024-04-20T11:35:21.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:21.187+0000 7fefa3e00640 1 -- 172.21.15.152:0/1610444784 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7fef90001670 con 0x7fefa40cfc40 2024-04-20T11:35:21.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:21.187+0000 7fefa3e00640 1 -- 172.21.15.152:0/1610444784 >> v1:172.21.15.152:6789/0 conn(0x7fefa40cfc40 legacy=0x7fefa40b2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:21.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:21.187+0000 7fefa3e00640 1 --2- 172.21.15.152:0/1610444784 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef8c0039b0 0x7fef8c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:21.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:21.187+0000 7fefa3e00640 1 --2- 172.21.15.152:0/1610444784 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fefa415b510 0x7fef8c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:21.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:21.187+0000 7fefa3e00640 1 --2- 172.21.15.152:0/1610444784 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef8c004d40 0x7fef8c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:21.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:21.187+0000 7fefa3e00640 1 -- 172.21.15.152:0/1610444784 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fefa406f420 con 0x7fef8c0039b0 2024-04-20T11:35:21.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:21.187+0000 7fefa3e00640 1 -- 172.21.15.152:0/1610444784 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fefa40b4430 con 0x7fef8c004d40 2024-04-20T11:35:21.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:21.187+0000 7fefa3e00640 1 -- 172.21.15.152:0/1610444784 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fef8c00dc00 con 0x7fefa415b510 2024-04-20T11:35:21.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:21.188+0000 7fefa9400640 1 --2- 172.21.15.152:0/1610444784 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef8c0039b0 0x7fef8c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:21.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:21.188+0000 7fefa8a00640 1 --2- 172.21.15.152:0/1610444784 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fefa415b510 0x7fef8c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:21.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:21.188+0000 7fefa9400640 1 -- 172.21.15.152:0/1610444784 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fefa415b510 msgr2=0x7fef8c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:21.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:21.188+0000 7fefa9400640 1 --2- 172.21.15.152:0/1610444784 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fefa415b510 0x7fef8c004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:21.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:21.188+0000 7fefa9400640 1 -- 172.21.15.152:0/1610444784 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef8c004d40 msgr2=0x7fef8c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:21.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:21.188+0000 7fefa9400640 1 --2- 172.21.15.152:0/1610444784 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef8c004d40 0x7fef8c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:21.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:21.188+0000 7fefa9400640 1 -- 172.21.15.152:0/1610444784 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef8c003520 con 0x7fef8c0039b0 2024-04-20T11:35:21.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:21.189+0000 7fefa9400640 1 --2- 172.21.15.152:0/1610444784 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef8c0039b0 0x7fef8c003dd0 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7fef98009ac0 tx=0x7fef8c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c95e68bd84f9da0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:21.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:21.189+0000 7fefa3e00640 1 -- 172.21.15.152:0/1610444784 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef98039070 con 0x7fef8c0039b0 2024-04-20T11:35:21.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:21.189+0000 7fefa3e00640 1 -- 172.21.15.152:0/1610444784 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef9802c6b0 con 0x7fef8c0039b0 2024-04-20T11:35:21.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:21.189+0000 7fefa3e00640 1 -- 172.21.15.152:0/1610444784 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef98009690 con 0x7fef8c0039b0 2024-04-20T11:35:21.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:21.190+0000 7fefabe00640 1 -- 172.21.15.152:0/1610444784 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef8c0039b0 msgr2=0x7fef8c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:21.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:21.190+0000 7fefabe00640 1 --2- 172.21.15.152:0/1610444784 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef8c0039b0 0x7fef8c003dd0 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7fef98009ac0 tx=0x7fef8c005830 comp rx=0 tx=0).stop 2024-04-20T11:35:21.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:21.190+0000 7fefa9e00640 1 -- 172.21.15.152:0/1610444784 reap_dead start 2024-04-20T11:35:21.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:21.190+0000 7fefabe00640 1 -- 172.21.15.152:0/1610444784 shutdown_connections 2024-04-20T11:35:21.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:21.190+0000 7fefabe00640 1 -- 172.21.15.152:0/1610444784 >> 172.21.15.152:0/1610444784 conn(0x7fefa406b980 msgr2=0x7fefa406bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:21.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:21.190+0000 7fefabe00640 1 -- 172.21.15.152:0/1610444784 shutdown_connections 2024-04-20T11:35:21.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:21.190+0000 7fefabe00640 1 -- 172.21.15.152:0/1610444784 wait complete. 2024-04-20T11:35:21.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:21.192+0000 7fefabe00640 1 Processor -- start 2024-04-20T11:35:21.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:21.192+0000 7fefabe00640 1 -- start start 2024-04-20T11:35:21.194 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:21.193+0000 7fefabe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef8c0039b0 0x7fefa415c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:21.194 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:21.193+0000 7fefabe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fefa414d300 0x7fefa415d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:21.194 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:21.193+0000 7fefabe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fefa4152860 0x7fefa4152ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:21.194 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:21.193+0000 7fefabe00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fefa415b8f0 con 0x7fef8c0039b0 2024-04-20T11:35:21.194 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:21.193+0000 7fefabe00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fefa415f710 con 0x7fefa414d300 2024-04-20T11:35:21.194 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:21.193+0000 7fefabe00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fefa406f950 con 0x7fefa4152860 2024-04-20T11:35:21.194 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:21.193+0000 7fefa9400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef8c0039b0 0x7fefa415c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:21.194 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:21.193+0000 7fefa9400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef8c0039b0 0x7fefa415c5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52332/0 (socket says 172.21.15.152:52332) 2024-04-20T11:35:21.194 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:21.193+0000 7fefa9400640 1 -- 172.21.15.152:0/720647138 learned_addr learned my addr 172.21.15.152:0/720647138 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:21.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:21.194+0000 7fefa8a00640 1 --2- 172.21.15.152:0/720647138 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fefa414d300 0x7fefa415d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:21.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:21.194+0000 7fefa9e00640 1 --2- 172.21.15.152:0/720647138 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fefa4152860 0x7fefa4152ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:21.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:21.194+0000 7fefa9400640 1 -- 172.21.15.152:0/720647138 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fefa4152860 msgr2=0x7fefa4152ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:21.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:21.194+0000 7fefa9400640 1 --2- 172.21.15.152:0/720647138 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fefa4152860 0x7fefa4152ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:21.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:21.194+0000 7fefa9400640 1 -- 172.21.15.152:0/720647138 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fefa414d300 msgr2=0x7fefa415d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:21.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:21.194+0000 7fefa9400640 1 --2- 172.21.15.152:0/720647138 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fefa414d300 0x7fefa415d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:21.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:21.194+0000 7fefa9400640 1 -- 172.21.15.152:0/720647138 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefa415cb20 con 0x7fef8c0039b0 2024-04-20T11:35:21.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:21.194+0000 7fefa8a00640 1 --2- 172.21.15.152:0/720647138 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fefa414d300 0x7fefa415d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:35:21.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:21.194+0000 7fefa9400640 1 --2- 172.21.15.152:0/720647138 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef8c0039b0 0x7fefa415c5e0 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7fef98009be0 tx=0x7fef98004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:21.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:21.194+0000 7fefa1600640 1 -- 172.21.15.152:0/720647138 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef98042020 con 0x7fef8c0039b0 2024-04-20T11:35:21.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:21.194+0000 7fefa1600640 1 -- 172.21.15.152:0/720647138 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef9803a040 con 0x7fef8c0039b0 2024-04-20T11:35:21.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:21.194+0000 7fefabe00640 1 -- 172.21.15.152:0/720647138 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefa415e0d0 con 0x7fef8c0039b0 2024-04-20T11:35:21.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:21.195+0000 7fefa1600640 1 -- 172.21.15.152:0/720647138 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef98039070 con 0x7fef8c0039b0 2024-04-20T11:35:21.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:21.195+0000 7fefabe00640 1 -- 172.21.15.152:0/720647138 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefa415bb00 con 0x7fef8c0039b0 2024-04-20T11:35:21.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:21.196+0000 7fefabe00640 1 -- 172.21.15.152:0/720647138 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefa40689f0 con 0x7fef8c0039b0 2024-04-20T11:35:21.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:21.197+0000 7fefa1600640 1 -- 172.21.15.152:0/720647138 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 21) ==== 91550+0+0 (secure 0 0 0) 0x7fef98045020 con 0x7fef8c0039b0 2024-04-20T11:35:21.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:21.198+0000 7fefa1600640 1 --2- 172.21.15.152:0/720647138 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fef8406ce90 0x7fef8406f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:21.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:21.198+0000 7fefa1600640 1 -- 172.21.15.152:0/720647138 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5709+0+0 (secure 0 0 0) 0x7fef980ada70 con 0x7fef8c0039b0 2024-04-20T11:35:21.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:21.198+0000 7fefa8a00640 1 --2- 172.21.15.152:0/720647138 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fef8406ce90 0x7fef8406f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:21.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:21.199+0000 7fefa8a00640 1 --2- 172.21.15.152:0/720647138 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fef8406ce90 0x7fef8406f350 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fef900017e0 tx=0x7fef90009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:21.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:21.202+0000 7fefa1600640 1 -- 172.21.15.152:0/720647138 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fef9804c070 con 0x7fef8c0039b0 2024-04-20T11:35:21.445 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:21.444+0000 7fefabe00640 1 -- 172.21.15.152:0/720647138 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fefa406ecf0 con 0x7fef8c0039b0 2024-04-20T11:35:22.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:22.105+0000 7fefa1600640 1 -- 172.21.15.152:0/720647138 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v85) ==== 118+0+0 (secure 0 0 0) 0x7fef9807b5f0 con 0x7fef8c0039b0 2024-04-20T11:35:22.110 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:22.109+0000 7fefabe00640 1 -- 172.21.15.152:0/720647138 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fef8406ce90 msgr2=0x7fef8406f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:22.110 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:22.109+0000 7fefabe00640 1 --2- 172.21.15.152:0/720647138 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fef8406ce90 0x7fef8406f350 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fef900017e0 tx=0x7fef90009040 comp rx=0 tx=0).stop 2024-04-20T11:35:22.110 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:22.109+0000 7fefabe00640 1 -- 172.21.15.152:0/720647138 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef8c0039b0 msgr2=0x7fefa415c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:22.110 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:22.109+0000 7fefabe00640 1 --2- 172.21.15.152:0/720647138 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef8c0039b0 0x7fefa415c5e0 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7fef98009be0 tx=0x7fef98004730 comp rx=0 tx=0).stop 2024-04-20T11:35:22.110 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:22.109+0000 7fefabe00640 1 -- 172.21.15.152:0/720647138 shutdown_connections 2024-04-20T11:35:22.110 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:22.109+0000 7fefabe00640 1 --2- 172.21.15.152:0/720647138 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fef8406ce90 0x7fef8406f350 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:22.110 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:22.109+0000 7fefabe00640 1 --2- 172.21.15.152:0/720647138 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fefa4152860 0x7fefa4152ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:22.111 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:22.109+0000 7fefabe00640 1 --2- 172.21.15.152:0/720647138 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fefa414d300 0x7fefa415d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:22.111 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:22.109+0000 7fefabe00640 1 --2- 172.21.15.152:0/720647138 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef8c0039b0 0x7fefa415c5e0 unknown :-1 s=CLOSED pgs=1755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:22.111 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:22.109+0000 7fefabe00640 1 -- 172.21.15.152:0/720647138 >> 172.21.15.152:0/720647138 conn(0x7fefa406b980 msgr2=0x7fefa40cc6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:22.111 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:22.109+0000 7fefabe00640 1 -- 172.21.15.152:0/720647138 shutdown_connections 2024-04-20T11:35:22.111 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:22.109+0000 7fefabe00640 1 -- 172.21.15.152:0/720647138 wait complete. 2024-04-20T11:35:22.133 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-20T11:35:22.133 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-20T11:35:22.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:22.340+0000 7f952ca00640 1 Processor -- start 2024-04-20T11:35:22.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:22.341+0000 7f952ca00640 1 -- start start 2024-04-20T11:35:22.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:22.342+0000 7f952ca00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f9528161e50 0x7f95280b7890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:22.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:22.342+0000 7f952ca00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f95280d2860 0x7f95280ba180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:22.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:22.342+0000 7f952ca00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f95280cf4f0 0x7f95280cf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:22.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:22.342+0000 7f952ca00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f9528162230 con 0x7f9528161e50 2024-04-20T11:35:22.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:22.342+0000 7f952ca00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f95280d2ca0 con 0x7f95280cf4f0 2024-04-20T11:35:22.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:22.342+0000 7f952ca00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f95280cfec0 con 0x7f95280d2860 2024-04-20T11:35:22.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:22.342+0000 7f9525e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f9528161e50 0x7f95280b7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:22.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:22.342+0000 7f9525e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f9528161e50 0x7f95280b7890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52336/0 (socket says 172.21.15.152:52336) 2024-04-20T11:35:22.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:22.342+0000 7f9525e00640 1 -- 172.21.15.152:0/339754181 learned_addr learned my addr 172.21.15.152:0/339754181 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:22.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:22.343+0000 7f9526800640 1 --2- 172.21.15.152:0/339754181 >> v2:172.21.15.175:3300/0 conn(0x7f95280cf4f0 0x7f95280cf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:22.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:22.343+0000 7f9525e00640 1 -- 172.21.15.152:0/339754181 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f95280d2860 msgr2=0x7f95280ba180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:22.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:22.343+0000 7f9525400640 1 --2- 172.21.15.152:0/339754181 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f95280d2860 0x7f95280ba180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:22.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:22.343+0000 7f9525e00640 1 --2- 172.21.15.152:0/339754181 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f95280d2860 0x7f95280ba180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:22.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:22.343+0000 7f9525e00640 1 -- 172.21.15.152:0/339754181 >> v2:172.21.15.175:3300/0 conn(0x7f95280cf4f0 msgr2=0x7f95280cf980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:22.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:22.343+0000 7f9525e00640 1 --2- 172.21.15.152:0/339754181 >> v2:172.21.15.175:3300/0 conn(0x7f95280cf4f0 0x7f95280cf980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:22.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:22.343+0000 7f9525e00640 1 -- 172.21.15.152:0/339754181 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95280b7f70 con 0x7f9528161e50 2024-04-20T11:35:22.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:22.343+0000 7f9526800640 1 --2- 172.21.15.152:0/339754181 >> v2:172.21.15.175:3300/0 conn(0x7f95280cf4f0 0x7f95280cf980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:35:22.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:22.343+0000 7f9525400640 1 --2- 172.21.15.152:0/339754181 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f95280d2860 0x7f95280ba180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:35:22.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:22.344+0000 7f9525e00640 1 --2- 172.21.15.152:0/339754181 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9528161e50 0x7f95280b7890 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f95280b4230 tx=0x7f951802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9611e8680b1ccb8a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:22.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:22.344+0000 7f9524a00640 1 -- 172.21.15.152:0/339754181 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9518034070 con 0x7f9528161e50 2024-04-20T11:35:22.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:22.344+0000 7f9524a00640 1 -- 172.21.15.152:0/339754181 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9518030860 con 0x7f9528161e50 2024-04-20T11:35:22.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:22.344+0000 7f9524a00640 1 -- 172.21.15.152:0/339754181 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9518030c30 con 0x7f9528161e50 2024-04-20T11:35:22.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:22.345+0000 7f952ca00640 1 -- 172.21.15.152:0/339754181 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9528161e50 msgr2=0x7f95280b7890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:22.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:22.345+0000 7f952ca00640 1 --2- 172.21.15.152:0/339754181 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9528161e50 0x7f95280b7890 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f95280b4230 tx=0x7f951802b9a0 comp rx=0 tx=0).stop 2024-04-20T11:35:22.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:22.345+0000 7f952ca00640 1 -- 172.21.15.152:0/339754181 shutdown_connections 2024-04-20T11:35:22.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:22.345+0000 7f952ca00640 1 --2- 172.21.15.152:0/339754181 >> v2:172.21.15.175:3300/0 conn(0x7f95280cf4f0 0x7f95280cf980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:22.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:22.345+0000 7f952ca00640 1 --2- 172.21.15.152:0/339754181 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f95280d2860 0x7f95280ba180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:22.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:22.345+0000 7f952ca00640 1 --2- 172.21.15.152:0/339754181 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9528161e50 0x7f95280b7890 unknown :-1 s=CLOSED pgs=1756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:22.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:22.345+0000 7f952ca00640 1 -- 172.21.15.152:0/339754181 >> 172.21.15.152:0/339754181 conn(0x7f9528067ae0 msgr2=0x7f9528067ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:22.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:22.345+0000 7f952ca00640 1 -- 172.21.15.152:0/339754181 shutdown_connections 2024-04-20T11:35:22.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:22.345+0000 7f952ca00640 1 -- 172.21.15.152:0/339754181 wait complete. 2024-04-20T11:35:22.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:22.347+0000 7f952ca00640 1 Processor -- start 2024-04-20T11:35:22.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:22.347+0000 7f952ca00640 1 -- start start 2024-04-20T11:35:22.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:22.348+0000 7f952ca00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f95280cf4f0 0x7f95280c80d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:22.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:22.348+0000 7f952ca00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f95280d2860 0x7f95281509f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:22.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:22.348+0000 7f952ca00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9528161e50 0x7f95280d1f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:22.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:22.348+0000 7f952ca00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f95280b8810 con 0x7f95280d2860 2024-04-20T11:35:22.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:22.348+0000 7f952ca00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f95280b8530 con 0x7f95280cf4f0 2024-04-20T11:35:22.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:22.348+0000 7f952ca00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f95280b8af0 con 0x7f9528161e50 2024-04-20T11:35:22.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:22.348+0000 7f9525400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f95280d2860 0x7f95281509f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:22.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:22.348+0000 7f9525400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f95280d2860 0x7f95281509f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52346/0 (socket says 172.21.15.152:52346) 2024-04-20T11:35:22.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:22.348+0000 7f9525400640 1 -- 172.21.15.152:0/2888515062 learned_addr learned my addr 172.21.15.152:0/2888515062 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:22.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:22.349+0000 7f9525e00640 1 --2- 172.21.15.152:0/2888515062 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f95280cf4f0 0x7f95280c80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:22.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:22.349+0000 7f9525400640 1 -- 172.21.15.152:0/2888515062 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9528161e50 msgr2=0x7f95280d1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:22.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:22.349+0000 7f9526800640 1 --2- 172.21.15.152:0/2888515062 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9528161e50 0x7f95280d1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:22.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:22.349+0000 7f9525400640 1 --2- 172.21.15.152:0/2888515062 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9528161e50 0x7f95280d1f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:22.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:22.349+0000 7f9525400640 1 -- 172.21.15.152:0/2888515062 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f95280cf4f0 msgr2=0x7f95280c80d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:22.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:22.349+0000 7f9525400640 1 --2- 172.21.15.152:0/2888515062 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f95280cf4f0 0x7f95280c80d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:22.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:22.349+0000 7f9525400640 1 -- 172.21.15.152:0/2888515062 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95280c87b0 con 0x7f95280d2860 2024-04-20T11:35:22.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:22.349+0000 7f9525e00640 1 --2- 172.21.15.152:0/2888515062 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f95280cf4f0 0x7f95280c80d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:35:22.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:22.349+0000 7f9525400640 1 --2- 172.21.15.152:0/2888515062 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f95280d2860 0x7f95281509f0 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f951000cf60 tx=0x7f951000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:22.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:22.349+0000 7f951e000640 1 -- 172.21.15.152:0/2888515062 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f951001a070 con 0x7f95280d2860 2024-04-20T11:35:22.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:22.350+0000 7f951e000640 1 -- 172.21.15.152:0/2888515062 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f951000d840 con 0x7f95280d2860 2024-04-20T11:35:22.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:22.350+0000 7f952ca00640 1 -- 172.21.15.152:0/2888515062 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95280c8a90 con 0x7f95280d2860 2024-04-20T11:35:22.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:22.350+0000 7f951e000640 1 -- 172.21.15.152:0/2888515062 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f951000dc10 con 0x7f95280d2860 2024-04-20T11:35:22.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:22.350+0000 7f952ca00640 1 -- 172.21.15.152:0/2888515062 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95280c9050 con 0x7f95280d2860 2024-04-20T11:35:22.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:22.351+0000 7f952ca00640 1 -- 172.21.15.152:0/2888515062 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9528157e30 con 0x7f95280d2860 2024-04-20T11:35:22.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:22.352+0000 7f951e000640 1 -- 172.21.15.152:0/2888515062 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 21) ==== 91550+0+0 (secure 0 0 0) 0x7f951001d070 con 0x7f95280d2860 2024-04-20T11:35:22.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:22.353+0000 7f951e000640 1 --2- 172.21.15.152:0/2888515062 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f950406ce90 0x7f950406f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:22.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:22.353+0000 7f951e000640 1 -- 172.21.15.152:0/2888515062 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5709+0+0 (secure 0 0 0) 0x7f951008cde0 con 0x7f95280d2860 2024-04-20T11:35:22.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:22.354+0000 7f9525e00640 1 --2- 172.21.15.152:0/2888515062 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f950406ce90 0x7f950406f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:22.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:22.355+0000 7f9525e00640 1 --2- 172.21.15.152:0/2888515062 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f950406ce90 0x7f950406f350 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f9518036000 tx=0x7f951803b040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:22.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:22.358+0000 7f951e000640 1 -- 172.21.15.152:0/2888515062 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f9528157e30 con 0x7f95280d2860 2024-04-20T11:35:22.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:22.595+0000 7f952ca00640 1 -- 172.21.15.152:0/2888515062 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f95280689f0 con 0x7f95280d2860 2024-04-20T11:35:23.116 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.115+0000 7f951e000640 1 -- 172.21.15.152:0/2888515062 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v86) ==== 122+0+0 (secure 0 0 0) 0x7f951005a350 con 0x7f95280d2860 2024-04-20T11:35:23.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.119+0000 7f952ca00640 1 -- 172.21.15.152:0/2888515062 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f950406ce90 msgr2=0x7f950406f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:23.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.119+0000 7f952ca00640 1 --2- 172.21.15.152:0/2888515062 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f950406ce90 0x7f950406f350 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f9518036000 tx=0x7f951803b040 comp rx=0 tx=0).stop 2024-04-20T11:35:23.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.119+0000 7f952ca00640 1 -- 172.21.15.152:0/2888515062 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f95280d2860 msgr2=0x7f95281509f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:23.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.119+0000 7f952ca00640 1 --2- 172.21.15.152:0/2888515062 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f95280d2860 0x7f95281509f0 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f951000cf60 tx=0x7f951000cf90 comp rx=0 tx=0).stop 2024-04-20T11:35:23.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.119+0000 7f952ca00640 1 -- 172.21.15.152:0/2888515062 shutdown_connections 2024-04-20T11:35:23.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.119+0000 7f952ca00640 1 --2- 172.21.15.152:0/2888515062 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f950406ce90 0x7f950406f350 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:23.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.119+0000 7f952ca00640 1 --2- 172.21.15.152:0/2888515062 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9528161e50 0x7f95280d1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:23.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.119+0000 7f952ca00640 1 --2- 172.21.15.152:0/2888515062 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f95280d2860 0x7f95281509f0 unknown :-1 s=CLOSED pgs=1757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:23.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.119+0000 7f952ca00640 1 --2- 172.21.15.152:0/2888515062 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f95280cf4f0 0x7f95280c80d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:23.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.119+0000 7f952ca00640 1 -- 172.21.15.152:0/2888515062 >> 172.21.15.152:0/2888515062 conn(0x7f9528067ae0 msgr2=0x7f95280d30a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:23.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.119+0000 7f952ca00640 1 -- 172.21.15.152:0/2888515062 shutdown_connections 2024-04-20T11:35:23.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.119+0000 7f952ca00640 1 -- 172.21.15.152:0/2888515062 wait complete. 2024-04-20T11:35:23.142 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-20T11:35:23.142 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:35:23.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.342+0000 7fb101800640 1 Processor -- start 2024-04-20T11:35:23.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.342+0000 7fb101800640 1 -- start start 2024-04-20T11:35:23.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.344+0000 7fb101800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb0fc15d1e0 0x7fb0fc160b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:23.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.344+0000 7fb101800640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fb0fc15d5c0 0x7fb0fc0cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:23.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.344+0000 7fb101800640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0fc0ce8c0 con 0x7fb0fc071cd0 2024-04-20T11:35:23.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.344+0000 7fb101800640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fb0fc1600b0 con 0x7fb0fc15d5c0 2024-04-20T11:35:23.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.344+0000 7fb101800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb0fc160390 con 0x7fb0fc15d1e0 2024-04-20T11:35:23.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.344+0000 7fb0fbe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb0fc15d1e0 0x7fb0fc160b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:23.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.345+0000 7fb100e00640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7fb0fc071cd0 0x7fb0fc0ccd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.152:48650/0 (socket says 172.21.15.152:48650) 2024-04-20T11:35:23.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.345+0000 7fb100e00640 1 -- 172.21.15.152:0/3649968122 learned_addr learned my addr 172.21.15.152:0/3649968122 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:35:23.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.345+0000 7fb0fbe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb0fc15d1e0 0x7fb0fc160b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.152:53124/0 (socket says 172.21.15.152:53124) 2024-04-20T11:35:23.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.345+0000 7fb0fb400640 1 --2- 172.21.15.152:0/3649968122 >> v2:172.21.15.175:3300/0 conn(0x7fb0fc15d5c0 0x7fb0fc0cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:23.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.345+0000 7fb0faa00640 1 -- 172.21.15.152:0/3649968122 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1909451998 0 0) 0x7fb0fc0ce8c0 con 0x7fb0fc071cd0 2024-04-20T11:35:23.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.345+0000 7fb0fbe00640 1 -- 172.21.15.152:0/3649968122 >> v2:172.21.15.175:3300/0 conn(0x7fb0fc15d5c0 msgr2=0x7fb0fc0cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:23.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.345+0000 7fb0fbe00640 1 --2- 172.21.15.152:0/3649968122 >> v2:172.21.15.175:3300/0 conn(0x7fb0fc15d5c0 0x7fb0fc0cdfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:23.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.345+0000 7fb0fbe00640 1 -- 172.21.15.152:0/3649968122 >> v1:172.21.15.175:6789/0 conn(0x7fb0fc071cd0 legacy=0x7fb0fc0ccd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:23.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.345+0000 7fb0fbe00640 1 -- 172.21.15.152:0/3649968122 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0fc160670 con 0x7fb0fc15d1e0 2024-04-20T11:35:23.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.347+0000 7fb0fbe00640 1 --2- 172.21.15.152:0/3649968122 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb0fc15d1e0 0x7fb0fc160b90 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec0097a0 tx=0x7fb0ec02b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3237e9ab20f1139 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:23.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.347+0000 7fb0faa00640 1 -- 172.21.15.152:0/3649968122 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0ec035020 con 0x7fb0fc15d1e0 2024-04-20T11:35:23.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.347+0000 7fb0faa00640 1 -- 172.21.15.152:0/3649968122 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0ec031830 con 0x7fb0fc15d1e0 2024-04-20T11:35:23.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.348+0000 7fb0faa00640 1 -- 172.21.15.152:0/3649968122 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0ec031c00 con 0x7fb0fc15d1e0 2024-04-20T11:35:23.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.348+0000 7fb101800640 1 -- 172.21.15.152:0/3649968122 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb0fc15d1e0 msgr2=0x7fb0fc160b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:23.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.348+0000 7fb101800640 1 --2- 172.21.15.152:0/3649968122 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb0fc15d1e0 0x7fb0fc160b90 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec0097a0 tx=0x7fb0ec02b8d0 comp rx=0 tx=0).stop 2024-04-20T11:35:23.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.348+0000 7fb101800640 1 -- 172.21.15.152:0/3649968122 shutdown_connections 2024-04-20T11:35:23.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.348+0000 7fb101800640 1 --2- 172.21.15.152:0/3649968122 >> v2:172.21.15.175:3300/0 conn(0x7fb0fc15d5c0 0x7fb0fc0cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:23.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.348+0000 7fb101800640 1 --2- 172.21.15.152:0/3649968122 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb0fc15d1e0 0x7fb0fc160b90 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:23.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.348+0000 7fb101800640 1 -- 172.21.15.152:0/3649968122 >> 172.21.15.152:0/3649968122 conn(0x7fb0fc067ae0 msgr2=0x7fb0fc06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:23.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.348+0000 7fb101800640 1 -- 172.21.15.152:0/3649968122 shutdown_connections 2024-04-20T11:35:23.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.348+0000 7fb101800640 1 -- 172.21.15.152:0/3649968122 wait complete. 2024-04-20T11:35:23.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.350+0000 7fb101800640 1 Processor -- start 2024-04-20T11:35:23.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.350+0000 7fb101800640 1 -- start start 2024-04-20T11:35:23.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.351+0000 7fb101800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb0fc071cd0 0x7fb0fc14cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:23.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.351+0000 7fb101800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb0fc15d1e0 0x7fb0fc0b3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:23.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.351+0000 7fb101800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb0fc15d5c0 0x7fb0fc0ae900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:23.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.351+0000 7fb101800640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fb0fc0ce6a0 con 0x7fb0fc15d5c0 2024-04-20T11:35:23.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.351+0000 7fb101800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb0fc0ce520 con 0x7fb0fc15d1e0 2024-04-20T11:35:23.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.351+0000 7fb101800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb0fc072160 con 0x7fb0fc071cd0 2024-04-20T11:35:23.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.351+0000 7fb100e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb0fc15d5c0 0x7fb0fc0ae900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:23.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.351+0000 7fb100e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb0fc15d5c0 0x7fb0fc0ae900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52354/0 (socket says 172.21.15.152:52354) 2024-04-20T11:35:23.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.351+0000 7fb100e00640 1 -- 172.21.15.152:0/3823740176 learned_addr learned my addr 172.21.15.152:0/3823740176 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:23.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.352+0000 7fb0fbe00640 1 --2- 172.21.15.152:0/3823740176 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb0fc071cd0 0x7fb0fc14cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:23.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.352+0000 7fb100e00640 1 -- 172.21.15.152:0/3823740176 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb0fc071cd0 msgr2=0x7fb0fc14cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:23.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.352+0000 7fb100e00640 1 --2- 172.21.15.152:0/3823740176 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb0fc071cd0 0x7fb0fc14cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:23.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.352+0000 7fb100e00640 1 -- 172.21.15.152:0/3823740176 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb0fc15d1e0 msgr2=0x7fb0fc0b3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:23.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.352+0000 7fb100e00640 1 --2- 172.21.15.152:0/3823740176 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb0fc15d1e0 0x7fb0fc0b3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:23.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.352+0000 7fb100e00640 1 -- 172.21.15.152:0/3823740176 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0fc0b48f0 con 0x7fb0fc15d5c0 2024-04-20T11:35:23.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.352+0000 7fb100e00640 1 --2- 172.21.15.152:0/3823740176 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb0fc15d5c0 0x7fb0fc0ae900 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7fb0f400c7a0 tx=0x7fb0f400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:23.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.352+0000 7fb0f3e00640 1 -- 172.21.15.152:0/3823740176 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f4018070 con 0x7fb0fc15d5c0 2024-04-20T11:35:23.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.352+0000 7fb0f3e00640 1 -- 172.21.15.152:0/3823740176 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0f400a640 con 0x7fb0fc15d5c0 2024-04-20T11:35:23.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.352+0000 7fb0f3e00640 1 -- 172.21.15.152:0/3823740176 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f400aa70 con 0x7fb0fc15d5c0 2024-04-20T11:35:23.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.352+0000 7fb101800640 1 -- 172.21.15.152:0/3823740176 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0fc0c5300 con 0x7fb0fc15d5c0 2024-04-20T11:35:23.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.353+0000 7fb101800640 1 -- 172.21.15.152:0/3823740176 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0fc0c58c0 con 0x7fb0fc15d5c0 2024-04-20T11:35:23.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.354+0000 7fb101800640 1 -- 172.21.15.152:0/3823740176 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0fc075e60 con 0x7fb0fc15d5c0 2024-04-20T11:35:23.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.355+0000 7fb0f3e00640 1 -- 172.21.15.152:0/3823740176 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 21) ==== 91550+0+0 (secure 0 0 0) 0x7fb0f401b070 con 0x7fb0fc15d5c0 2024-04-20T11:35:23.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.356+0000 7fb0f3e00640 1 --2- 172.21.15.152:0/3823740176 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb0dc06ce90 0x7fb0dc06f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:23.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.356+0000 7fb0f3e00640 1 -- 172.21.15.152:0/3823740176 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5709+0+0 (secure 0 0 0) 0x7fb0f408bc10 con 0x7fb0fc15d5c0 2024-04-20T11:35:23.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.356+0000 7fb0fbe00640 1 --2- 172.21.15.152:0/3823740176 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb0dc06ce90 0x7fb0dc06f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:23.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.358+0000 7fb0fbe00640 1 --2- 172.21.15.152:0/3823740176 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb0dc06ce90 0x7fb0dc06f350 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec036000 tx=0x7fb0ec03c040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:23.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.360+0000 7fb0f3e00640 1 -- 172.21.15.152:0/3823740176 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fb0f40597d0 con 0x7fb0fc15d5c0 2024-04-20T11:35:23.490 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:35:23.498 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:35:23.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.600+0000 7fb101800640 1 -- 172.21.15.152:0/3823740176 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb0fc072c90 con 0x7fb0fc15d5c0 2024-04-20T11:35:23.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.600+0000 7fb0f3e00640 1 -- 172.21.15.152:0/3823740176 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) ==== 94+0+5212 (secure 0 0 0) 0x7fb0f40590d0 con 0x7fb0fc15d5c0 2024-04-20T11:35:23.601 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:35:23.601 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":86,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6062,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1303172876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6833","nonce":1303172876},{"type":"v1","addr":"172.21.15.175:6835","nonce":1303172876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6667,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6834/1975208598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1975208598},{"type":"v1","addr":"172.21.15.175:6834","nonce":1975208598}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6670,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6834/287411071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":287411071},{"type":"v1","addr":"172.21.15.152:6834","nonce":287411071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T11:35:19.231687+0000","modified":"2024-04-20T11:35:22.597626+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8919},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8919":{"gid":8919,"name":"a","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6835/1884827433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6833","nonce":1884827433},{"type":"v1","addr":"172.21.15.152:6835","nonce":1884827433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8919,"qdb_cluster":[8919]},"id":10}]} 2024-04-20T11:35:23.602 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 86 2024-04-20T11:35:23.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.603+0000 7fb101800640 1 -- 172.21.15.152:0/3823740176 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb0dc06ce90 msgr2=0x7fb0dc06f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:23.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.603+0000 7fb101800640 1 --2- 172.21.15.152:0/3823740176 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb0dc06ce90 0x7fb0dc06f350 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec036000 tx=0x7fb0ec03c040 comp rx=0 tx=0).stop 2024-04-20T11:35:23.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.603+0000 7fb101800640 1 -- 172.21.15.152:0/3823740176 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb0fc15d5c0 msgr2=0x7fb0fc0ae900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:23.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.603+0000 7fb101800640 1 --2- 172.21.15.152:0/3823740176 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb0fc15d5c0 0x7fb0fc0ae900 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7fb0f400c7a0 tx=0x7fb0f400cc60 comp rx=0 tx=0).stop 2024-04-20T11:35:23.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.603+0000 7fb101800640 1 -- 172.21.15.152:0/3823740176 shutdown_connections 2024-04-20T11:35:23.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.603+0000 7fb101800640 1 --2- 172.21.15.152:0/3823740176 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb0dc06ce90 0x7fb0dc06f350 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:23.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.603+0000 7fb101800640 1 --2- 172.21.15.152:0/3823740176 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb0fc15d5c0 0x7fb0fc0ae900 unknown :-1 s=CLOSED pgs=1758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:23.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.603+0000 7fb101800640 1 --2- 172.21.15.152:0/3823740176 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb0fc15d1e0 0x7fb0fc0b3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:23.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.604+0000 7fb101800640 1 --2- 172.21.15.152:0/3823740176 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb0fc071cd0 0x7fb0fc14cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:23.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.604+0000 7fb101800640 1 -- 172.21.15.152:0/3823740176 >> 172.21.15.152:0/3823740176 conn(0x7fb0fc067ae0 msgr2=0x7fb0fc06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:23.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.604+0000 7fb101800640 1 -- 172.21.15.152:0/3823740176 shutdown_connections 2024-04-20T11:35:23.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.604+0000 7fb101800640 1 -- 172.21.15.152:0/3823740176 wait complete. 2024-04-20T11:35:23.624 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:35:23.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.870+0000 7f476dc00640 1 Processor -- start 2024-04-20T11:35:23.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.870+0000 7f476dc00640 1 -- start start 2024-04-20T11:35:23.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.871+0000 7f476dc00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f476815b5e0 0x7f476806eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:23.873 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.871+0000 7f476dc00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f47680b4c50 0x7f47680b50d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:23.873 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.871+0000 7f476dc00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f476806f340 0x7f476806f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:23.873 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.871+0000 7f476dc00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f476806fcd0 con 0x7f476806f340 2024-04-20T11:35:23.873 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.871+0000 7f476dc00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f476806de10 con 0x7f476815b5e0 2024-04-20T11:35:23.873 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.871+0000 7f476dc00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f47680b5770 con 0x7f47680b4c50 2024-04-20T11:35:23.873 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.871+0000 7f4767400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f476806f340 0x7f476806f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:23.873 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.872+0000 7f4767400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f476806f340 0x7f476806f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52364/0 (socket says 172.21.15.152:52364) 2024-04-20T11:35:23.873 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.872+0000 7f4767400640 1 -- 172.21.15.152:0/409206792 learned_addr learned my addr 172.21.15.152:0/409206792 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:23.873 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.872+0000 7f4766000640 1 --2- 172.21.15.152:0/409206792 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f47680b4c50 0x7f47680b50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:23.873 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.872+0000 7f4766a00640 1 --2- 172.21.15.152:0/409206792 >> v2:172.21.15.175:3300/0 conn(0x7f476815b5e0 0x7f476806eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:23.873 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.872+0000 7f4767400640 1 -- 172.21.15.152:0/409206792 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f47680b4c50 msgr2=0x7f47680b50d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:23.873 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.872+0000 7f4767400640 1 --2- 172.21.15.152:0/409206792 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f47680b4c50 0x7f47680b50d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:23.873 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.872+0000 7f4767400640 1 -- 172.21.15.152:0/409206792 >> v2:172.21.15.175:3300/0 conn(0x7f476815b5e0 msgr2=0x7f476806eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:23.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.872+0000 7f4767400640 1 --2- 172.21.15.152:0/409206792 >> v2:172.21.15.175:3300/0 conn(0x7f476815b5e0 0x7f476806eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:23.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.872+0000 7f4767400640 1 -- 172.21.15.152:0/409206792 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f476806e060 con 0x7f476806f340 2024-04-20T11:35:23.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.873+0000 7f4767400640 1 --2- 172.21.15.152:0/409206792 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f476806f340 0x7f476806f790 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f4768072fa0 tx=0x7f476002e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e751a0e422f0ed51 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:23.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.874+0000 7f4765600640 1 -- 172.21.15.152:0/409206792 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f476002ea80 con 0x7f476806f340 2024-04-20T11:35:23.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.874+0000 7f4765600640 1 -- 172.21.15.152:0/409206792 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f476002ec20 con 0x7f476806f340 2024-04-20T11:35:23.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.874+0000 7f4765600640 1 -- 172.21.15.152:0/409206792 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4760009600 con 0x7f476806f340 2024-04-20T11:35:23.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.874+0000 7f476dc00640 1 -- 172.21.15.152:0/409206792 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f476806f340 msgr2=0x7f476806f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:23.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.874+0000 7f476dc00640 1 --2- 172.21.15.152:0/409206792 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f476806f340 0x7f476806f790 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f4768072fa0 tx=0x7f476002e0e0 comp rx=0 tx=0).stop 2024-04-20T11:35:23.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.874+0000 7f476dc00640 1 -- 172.21.15.152:0/409206792 shutdown_connections 2024-04-20T11:35:23.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.874+0000 7f476dc00640 1 --2- 172.21.15.152:0/409206792 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f476806f340 0x7f476806f790 unknown :-1 s=CLOSED pgs=1759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:23.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.874+0000 7f476dc00640 1 --2- 172.21.15.152:0/409206792 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f47680b4c50 0x7f47680b50d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:23.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.874+0000 7f476dc00640 1 --2- 172.21.15.152:0/409206792 >> v2:172.21.15.175:3300/0 conn(0x7f476815b5e0 0x7f476806eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:23.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.874+0000 7f476dc00640 1 -- 172.21.15.152:0/409206792 >> 172.21.15.152:0/409206792 conn(0x7f4768067cf0 msgr2=0x7f476806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:23.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.874+0000 7f476dc00640 1 -- 172.21.15.152:0/409206792 shutdown_connections 2024-04-20T11:35:23.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.874+0000 7f476dc00640 1 -- 172.21.15.152:0/409206792 wait complete. 2024-04-20T11:35:23.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.876+0000 7f476dc00640 1 Processor -- start 2024-04-20T11:35:23.877 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.876+0000 7f476dc00640 1 -- start start 2024-04-20T11:35:23.878 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.877+0000 7f476dc00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f476806f340 0x7f476814ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:23.878 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.877+0000 7f476dc00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f47680b4c50 0x7f4768152ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:23.878 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.877+0000 7f476dc00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f476815b5e0 0x7f4768158230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:23.878 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.877+0000 7f476dc00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f47680b42d0 con 0x7f47680b4c50 2024-04-20T11:35:23.878 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.877+0000 7f476dc00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f476806e620 con 0x7f476806f340 2024-04-20T11:35:23.878 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.877+0000 7f476dc00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f476815cd70 con 0x7f476815b5e0 2024-04-20T11:35:23.878 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.877+0000 7f4766000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f47680b4c50 0x7f4768152ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:23.878 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.878+0000 7f4766000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f47680b4c50 0x7f4768152ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52376/0 (socket says 172.21.15.152:52376) 2024-04-20T11:35:23.879 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.878+0000 7f4766000640 1 -- 172.21.15.152:0/2342231980 learned_addr learned my addr 172.21.15.152:0/2342231980 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:23.879 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.878+0000 7f4766000640 1 -- 172.21.15.152:0/2342231980 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f476815b5e0 msgr2=0x7f4768158230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:23.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.878+0000 7f4766000640 1 --2- 172.21.15.152:0/2342231980 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f476815b5e0 0x7f4768158230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:23.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.878+0000 7f4766000640 1 -- 172.21.15.152:0/2342231980 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f476806f340 msgr2=0x7f476814ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:23.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.878+0000 7f4766000640 1 --2- 172.21.15.152:0/2342231980 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f476806f340 0x7f476814ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:23.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.878+0000 7f4766000640 1 -- 172.21.15.152:0/2342231980 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f476814b3b0 con 0x7f47680b4c50 2024-04-20T11:35:23.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.878+0000 7f4766000640 1 --2- 172.21.15.152:0/2342231980 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f47680b4c50 0x7f4768152ca0 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f475400d830 tx=0x7f475400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:23.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.878+0000 7f475ea00640 1 -- 172.21.15.152:0/2342231980 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4754018020 con 0x7f47680b4c50 2024-04-20T11:35:23.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.879+0000 7f475ea00640 1 -- 172.21.15.152:0/2342231980 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47540096a0 con 0x7f47680b4c50 2024-04-20T11:35:23.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.879+0000 7f475ea00640 1 -- 172.21.15.152:0/2342231980 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4754009a70 con 0x7f47680b4c50 2024-04-20T11:35:23.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.879+0000 7f476dc00640 1 -- 172.21.15.152:0/2342231980 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4768151c00 con 0x7f47680b4c50 2024-04-20T11:35:23.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.879+0000 7f476dc00640 1 -- 172.21.15.152:0/2342231980 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47681521c0 con 0x7f47680b4c50 2024-04-20T11:35:23.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.880+0000 7f476dc00640 1 -- 172.21.15.152:0/2342231980 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47680689f0 con 0x7f47680b4c50 2024-04-20T11:35:23.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.881+0000 7f475ea00640 1 -- 172.21.15.152:0/2342231980 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 21) ==== 91550+0+0 (secure 0 0 0) 0x7f475401b020 con 0x7f47680b4c50 2024-04-20T11:35:23.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.882+0000 7f475ea00640 1 --2- 172.21.15.152:0/2342231980 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f474806ce90 0x7f474806f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:23.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.882+0000 7f475ea00640 1 -- 172.21.15.152:0/2342231980 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5709+0+0 (secure 0 0 0) 0x7f475408ca50 con 0x7f47680b4c50 2024-04-20T11:35:23.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.883+0000 7f4766a00640 1 --2- 172.21.15.152:0/2342231980 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f474806ce90 0x7f474806f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:23.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.884+0000 7f4766a00640 1 --2- 172.21.15.152:0/2342231980 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f474806ce90 0x7f474806f350 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f4758006fd0 tx=0x7f4758008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:23.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:23.886+0000 7f475ea00640 1 -- 172.21.15.152:0/2342231980 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f4754022030 con 0x7f47680b4c50 2024-04-20T11:35:24.088 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.087+0000 7f476dc00640 1 -- 172.21.15.152:0/2342231980 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f476806cd90 con 0x7f47680b4c50 2024-04-20T11:35:24.091 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.090+0000 7f475ea00640 1 -- 172.21.15.152:0/2342231980 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v131) ==== 74+0+16739 (secure 0 0 0) 0x7f47540102e0 con 0x7f47680b4c50 2024-04-20T11:35:24.091 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:35:24.092 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":131,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:35:20.781260+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:35:16.963520+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"130","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-04-20T11:35:18.215890+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:35:24.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.093+0000 7f476dc00640 1 -- 172.21.15.152:0/2342231980 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f474806ce90 msgr2=0x7f474806f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:24.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.093+0000 7f476dc00640 1 --2- 172.21.15.152:0/2342231980 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f474806ce90 0x7f474806f350 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f4758006fd0 tx=0x7f4758008040 comp rx=0 tx=0).stop 2024-04-20T11:35:24.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.093+0000 7f476dc00640 1 -- 172.21.15.152:0/2342231980 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f47680b4c50 msgr2=0x7f4768152ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:24.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.093+0000 7f476dc00640 1 --2- 172.21.15.152:0/2342231980 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f47680b4c50 0x7f4768152ca0 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f475400d830 tx=0x7f475400dcf0 comp rx=0 tx=0).stop 2024-04-20T11:35:24.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.093+0000 7f476dc00640 1 -- 172.21.15.152:0/2342231980 shutdown_connections 2024-04-20T11:35:24.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.093+0000 7f476dc00640 1 --2- 172.21.15.152:0/2342231980 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f474806ce90 0x7f474806f350 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:24.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.093+0000 7f476dc00640 1 --2- 172.21.15.152:0/2342231980 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f476815b5e0 0x7f4768158230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:24.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.093+0000 7f476dc00640 1 --2- 172.21.15.152:0/2342231980 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f47680b4c50 0x7f4768152ca0 unknown :-1 s=CLOSED pgs=1760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:24.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.093+0000 7f476dc00640 1 --2- 172.21.15.152:0/2342231980 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f476806f340 0x7f476814ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:24.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.093+0000 7f476dc00640 1 -- 172.21.15.152:0/2342231980 >> 172.21.15.152:0/2342231980 conn(0x7f4768067cf0 msgr2=0x7f476806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:24.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.094+0000 7f476dc00640 1 -- 172.21.15.152:0/2342231980 shutdown_connections 2024-04-20T11:35:24.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.094+0000 7f476dc00640 1 -- 172.21.15.152:0/2342231980 wait complete. 2024-04-20T11:35:24.114 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-20T11:35:24.114 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T11:35:24.315 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.314+0000 7f31f3400640 1 Processor -- start 2024-04-20T11:35:24.315 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.314+0000 7f31f3400640 1 -- start start 2024-04-20T11:35:24.317 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.316+0000 7f31f3400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f31ec0b17d0 0x7f31ec0ccbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:24.317 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.316+0000 7f31f3400640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f31ec06da70 0x7f31ec06cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:24.317 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.316+0000 7f31f3400640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31ec0d1af0 con 0x7f31ec15eb80 2024-04-20T11:35:24.317 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.316+0000 7f31f3400640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f31ec09ae50 con 0x7f31ec06da70 2024-04-20T11:35:24.317 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.316+0000 7f31f3400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f31ec0d1250 con 0x7f31ec0b17d0 2024-04-20T11:35:24.317 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.317+0000 7f31f0a00640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7f31ec15eb80 0x7f31ec06d360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.152:48658/0 (socket says 172.21.15.152:48658) 2024-04-20T11:35:24.317 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.317+0000 7f31f0a00640 1 -- 172.21.15.152:0/3449630144 learned_addr learned my addr 172.21.15.152:0/3449630144 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:35:24.318 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.317+0000 7f31f1400640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f31ec06da70 0x7f31ec06cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:24.318 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.317+0000 7f31ebe00640 1 --2- 172.21.15.152:0/3449630144 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f31ec0b17d0 0x7f31ec0ccbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:24.318 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.317+0000 7f31eb400640 1 -- 172.21.15.152:0/3449630144 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3204870726 0 0) 0x7f31ec0d1af0 con 0x7f31ec15eb80 2024-04-20T11:35:24.318 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.318+0000 7f31eb400640 1 -- 172.21.15.152:0/3449630144 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f31d4003520 con 0x7f31ec15eb80 2024-04-20T11:35:24.319 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.318+0000 7f31f1400640 1 -- 172.21.15.152:0/3449630144 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f31ec0b17d0 msgr2=0x7f31ec0ccbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:24.319 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.318+0000 7f31f1400640 1 --2- 172.21.15.152:0/3449630144 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f31ec0b17d0 0x7f31ec0ccbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:24.319 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.318+0000 7f31f1400640 1 -- 172.21.15.152:0/3449630144 >> v1:172.21.15.175:6789/0 conn(0x7f31ec15eb80 legacy=0x7f31ec06d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:24.319 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.318+0000 7f31f1400640 1 -- 172.21.15.152:0/3449630144 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31ec0d1530 con 0x7f31ec06da70 2024-04-20T11:35:24.320 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.319+0000 7f31f1400640 1 --2- 172.21.15.152:0/3449630144 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f31ec06da70 0x7f31ec06cca0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f31e4009d90 tx=0x7f31e402beb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3cf160f92266766f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:24.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.320+0000 7f31eb400640 1 -- 172.21.15.152:0/3449630144 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31e4039070 con 0x7f31ec06da70 2024-04-20T11:35:24.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.320+0000 7f31eb400640 1 -- 172.21.15.152:0/3449630144 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31e402cc60 con 0x7f31ec06da70 2024-04-20T11:35:24.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.320+0000 7f31eb400640 1 -- 172.21.15.152:0/3449630144 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31e4034640 con 0x7f31ec06da70 2024-04-20T11:35:24.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.321+0000 7f31f3400640 1 -- 172.21.15.152:0/3449630144 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f31ec06da70 msgr2=0x7f31ec06cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:24.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.321+0000 7f31f3400640 1 --2- 172.21.15.152:0/3449630144 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f31ec06da70 0x7f31ec06cca0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f31e4009d90 tx=0x7f31e402beb0 comp rx=0 tx=0).stop 2024-04-20T11:35:24.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.321+0000 7f31f3400640 1 -- 172.21.15.152:0/3449630144 shutdown_connections 2024-04-20T11:35:24.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.321+0000 7f31f3400640 1 --2- 172.21.15.152:0/3449630144 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f31ec06da70 0x7f31ec06cca0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:24.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.321+0000 7f31f3400640 1 --2- 172.21.15.152:0/3449630144 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f31ec0b17d0 0x7f31ec0ccbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:24.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.321+0000 7f31f3400640 1 -- 172.21.15.152:0/3449630144 >> 172.21.15.152:0/3449630144 conn(0x7f31ec0b4080 msgr2=0x7f31ec0b4490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:24.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.321+0000 7f31f3400640 1 -- 172.21.15.152:0/3449630144 shutdown_connections 2024-04-20T11:35:24.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.321+0000 7f31f3400640 1 -- 172.21.15.152:0/3449630144 wait complete. 2024-04-20T11:35:24.323 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.323+0000 7f31f3400640 1 Processor -- start 2024-04-20T11:35:24.323 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.323+0000 7f31f3400640 1 -- start start 2024-04-20T11:35:24.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.324+0000 7f31f3400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f31ec06da70 0x7f31ec158410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:24.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.324+0000 7f31f3400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f31ec0b17d0 0x7f31ec0c7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:24.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.324+0000 7f31f3400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f31ec15eb80 0x7f31ec0c8310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:24.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.324+0000 7f31f3400640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f31ec15ef60 con 0x7f31ec15eb80 2024-04-20T11:35:24.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.324+0000 7f31f3400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f31ec09aae0 con 0x7f31ec0b17d0 2024-04-20T11:35:24.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.324+0000 7f31f3400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f31ec09b2e0 con 0x7f31ec06da70 2024-04-20T11:35:24.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.324+0000 7f31f1400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f31ec15eb80 0x7f31ec0c8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:24.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.324+0000 7f31f1400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f31ec15eb80 0x7f31ec0c8310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52382/0 (socket says 172.21.15.152:52382) 2024-04-20T11:35:24.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.324+0000 7f31f1400640 1 -- 172.21.15.152:0/3122188204 learned_addr learned my addr 172.21.15.152:0/3122188204 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:24.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.325+0000 7f31f0a00640 1 --2- 172.21.15.152:0/3122188204 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f31ec06da70 0x7f31ec158410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:24.326 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.325+0000 7f31ebe00640 1 --2- 172.21.15.152:0/3122188204 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f31ec0b17d0 0x7f31ec0c7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:24.327 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.325+0000 7f31f1400640 1 -- 172.21.15.152:0/3122188204 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f31ec06da70 msgr2=0x7f31ec158410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:24.327 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.325+0000 7f31f1400640 1 --2- 172.21.15.152:0/3122188204 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f31ec06da70 0x7f31ec158410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:24.327 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.325+0000 7f31f1400640 1 -- 172.21.15.152:0/3122188204 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f31ec0b17d0 msgr2=0x7f31ec0c7a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:24.327 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.325+0000 7f31f1400640 1 --2- 172.21.15.152:0/3122188204 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f31ec0b17d0 0x7f31ec0c7a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:24.327 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.325+0000 7f31f1400640 1 -- 172.21.15.152:0/3122188204 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31ec0b7ea0 con 0x7f31ec15eb80 2024-04-20T11:35:24.327 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.325+0000 7f31f0a00640 1 --2- 172.21.15.152:0/3122188204 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f31ec06da70 0x7f31ec158410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:35:24.327 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.325+0000 7f31ebe00640 1 --2- 172.21.15.152:0/3122188204 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f31ec0b17d0 0x7f31ec0c7a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:35:24.327 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.325+0000 7f31f1400640 1 --2- 172.21.15.152:0/3122188204 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f31ec15eb80 0x7f31ec0c8310 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f31e4009eb0 tx=0x7f31e4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:24.327 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.326+0000 7f31e8c00640 1 -- 172.21.15.152:0/3122188204 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31e4039050 con 0x7f31ec15eb80 2024-04-20T11:35:24.327 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.326+0000 7f31e8c00640 1 -- 172.21.15.152:0/3122188204 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31e4045070 con 0x7f31ec15eb80 2024-04-20T11:35:24.327 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.326+0000 7f31e8c00640 1 -- 172.21.15.152:0/3122188204 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31e4009450 con 0x7f31ec15eb80 2024-04-20T11:35:24.327 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.326+0000 7f31f3400640 1 -- 172.21.15.152:0/3122188204 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31ec0c6360 con 0x7f31ec15eb80 2024-04-20T11:35:24.327 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.326+0000 7f31f3400640 1 -- 172.21.15.152:0/3122188204 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31ec0c6920 con 0x7f31ec15eb80 2024-04-20T11:35:24.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.327+0000 7f31f3400640 1 -- 172.21.15.152:0/3122188204 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31ec0689f0 con 0x7f31ec15eb80 2024-04-20T11:35:24.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.328+0000 7f31e8c00640 1 -- 172.21.15.152:0/3122188204 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 21) ==== 91550+0+0 (secure 0 0 0) 0x7f31e403e830 con 0x7f31ec15eb80 2024-04-20T11:35:24.330 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.329+0000 7f31e8c00640 1 --2- 172.21.15.152:0/3122188204 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f31cc06cec0 0x7f31cc06f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:24.330 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.329+0000 7f31e8c00640 1 -- 172.21.15.152:0/3122188204 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5709+0+0 (secure 0 0 0) 0x7f31e40af110 con 0x7f31ec15eb80 2024-04-20T11:35:24.330 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.330+0000 7f31f0a00640 1 --2- 172.21.15.152:0/3122188204 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f31cc06cec0 0x7f31cc06f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:24.331 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.331+0000 7f31f0a00640 1 --2- 172.21.15.152:0/3122188204 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f31cc06cec0 0x7f31cc06f380 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f31e0007fd0 tx=0x7f31e0009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:24.335 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.334+0000 7f31e8c00640 1 -- 172.21.15.152:0/3122188204 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f31ec0b7ea0 con 0x7f31ec15eb80 2024-04-20T11:35:24.531 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.531+0000 7f31f3400640 1 -- 172.21.15.152:0/3122188204 --> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f31ec162110 con 0x7f31cc06cec0 2024-04-20T11:35:24.548 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.548+0000 7f31e8c00640 1 -- 172.21.15.152:0/3122188204 <== mgr.4109 v2:172.21.15.175:6836/4021250441 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+240297 (secure 0 0 0) 0x7f31ec162110 con 0x7f31cc06cec0 2024-04-20T11:35:24.551 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:35:24.553 INFO:teuthology.orchestra.run.smithi152.stderr:dumped all 2024-04-20T11:35:24.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.553+0000 7f31f3400640 1 -- 172.21.15.152:0/3122188204 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f31cc06cec0 msgr2=0x7f31cc06f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:24.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.553+0000 7f31f3400640 1 --2- 172.21.15.152:0/3122188204 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f31cc06cec0 0x7f31cc06f380 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f31e0007fd0 tx=0x7f31e0009040 comp rx=0 tx=0).stop 2024-04-20T11:35:24.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.553+0000 7f31f3400640 1 -- 172.21.15.152:0/3122188204 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f31ec15eb80 msgr2=0x7f31ec0c8310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:24.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.553+0000 7f31f3400640 1 --2- 172.21.15.152:0/3122188204 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f31ec15eb80 0x7f31ec0c8310 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f31e4009eb0 tx=0x7f31e4004a40 comp rx=0 tx=0).stop 2024-04-20T11:35:24.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.554+0000 7f31f3400640 1 -- 172.21.15.152:0/3122188204 shutdown_connections 2024-04-20T11:35:24.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.554+0000 7f31f3400640 1 --2- 172.21.15.152:0/3122188204 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f31cc06cec0 0x7f31cc06f380 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:24.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.554+0000 7f31f3400640 1 --2- 172.21.15.152:0/3122188204 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f31ec15eb80 0x7f31ec0c8310 unknown :-1 s=CLOSED pgs=1761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:24.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.554+0000 7f31f3400640 1 --2- 172.21.15.152:0/3122188204 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f31ec0b17d0 0x7f31ec0c7a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:24.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.554+0000 7f31f3400640 1 --2- 172.21.15.152:0/3122188204 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f31ec06da70 0x7f31ec158410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:24.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.554+0000 7f31f3400640 1 -- 172.21.15.152:0/3122188204 >> 172.21.15.152:0/3122188204 conn(0x7f31ec0b4080 msgr2=0x7f31ec0cec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:24.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.554+0000 7f31f3400640 1 -- 172.21.15.152:0/3122188204 shutdown_connections 2024-04-20T11:35:24.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.554+0000 7f31f3400640 1 -- 172.21.15.152:0/3122188204 wait complete. 2024-04-20T11:35:24.574 INFO:teuthology.orchestra.run.smithi152.stdout:{"pg_ready":true,"pg_map":{"version":153,"stamp":"2024-04-20T11:35:22.661646+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":263132,"kb_used_data":13692,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754711588,"statfs":{"total":773094113280,"available":772824666112,"internally_reserved":0,"allocated":14020608,"data_stored":6832263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12746,"internal_metadata":255315510},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.908287"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:35:18.750729+0000","last_change":"2024-04-20T11:35:18.750729+0000","last_active":"2024-04-20T11:35:18.750729+0000","last_peered":"2024-04-20T11:35:18.750729+0000","last_clean":"2024-04-20T11:35:18.750729+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:35:18.750729+0000","last_undegraded":"2024-04-20T11:35:18.750729+0000","last_fullsized":"2024-04-20T11:35:18.750729+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:35:18.750729+0000","last_change":"2024-04-20T11:35:18.750729+0000","last_active":"2024-04-20T11:35:18.750729+0000","last_peered":"2024-04-20T11:35:18.750729+0000","last_clean":"2024-04-20T11:35:18.750729+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:35:18.750729+0000","last_undegraded":"2024-04-20T11:35:18.750729+0000","last_fullsized":"2024-04-20T11:35:18.750729+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:35:18.750729+0000","last_change":"2024-04-20T11:35:18.750729+0000","last_active":"2024-04-20T11:35:18.750729+0000","last_peered":"2024-04-20T11:35:18.750729+0000","last_clean":"2024-04-20T11:35:18.750729+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:35:18.750729+0000","last_undegraded":"2024-04-20T11:35:18.750729+0000","last_fullsized":"2024-04-20T11:35:18.750729+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:35:18.750729+0000","last_change":"2024-04-20T11:35:18.750729+0000","last_active":"2024-04-20T11:35:18.750729+0000","last_peered":"2024-04-20T11:35:18.750729+0000","last_clean":"2024-04-20T11:35:18.750729+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:35:18.750729+0000","last_undegraded":"2024-04-20T11:35:18.750729+0000","last_fullsized":"2024-04-20T11:35:18.750729+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:35:18.750729+0000","last_change":"2024-04-20T11:35:18.750729+0000","last_active":"2024-04-20T11:35:18.750729+0000","last_peered":"2024-04-20T11:35:18.750729+0000","last_clean":"2024-04-20T11:35:18.750729+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:35:18.750729+0000","last_undegraded":"2024-04-20T11:35:18.750729+0000","last_fullsized":"2024-04-20T11:35:18.750729+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:35:18.750729+0000","last_change":"2024-04-20T11:35:18.750729+0000","last_active":"2024-04-20T11:35:18.750729+0000","last_peered":"2024-04-20T11:35:18.750729+0000","last_clean":"2024-04-20T11:35:18.750729+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:35:18.750729+0000","last_undegraded":"2024-04-20T11:35:18.750729+0000","last_fullsized":"2024-04-20T11:35:18.750729+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:35:18.750729+0000","last_change":"2024-04-20T11:35:18.750729+0000","last_active":"2024-04-20T11:35:18.750729+0000","last_peered":"2024-04-20T11:35:18.750729+0000","last_clean":"2024-04-20T11:35:18.750729+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:35:18.750729+0000","last_undegraded":"2024-04-20T11:35:18.750729+0000","last_fullsized":"2024-04-20T11:35:18.750729+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:35:18.750729+0000","last_change":"2024-04-20T11:35:18.750729+0000","last_active":"2024-04-20T11:35:18.750729+0000","last_peered":"2024-04-20T11:35:18.750729+0000","last_clean":"2024-04-20T11:35:18.750729+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:35:18.750729+0000","last_undegraded":"2024-04-20T11:35:18.750729+0000","last_fullsized":"2024-04-20T11:35:18.750729+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:35:18.750729+0000","last_change":"2024-04-20T11:35:18.750729+0000","last_active":"2024-04-20T11:35:18.750729+0000","last_peered":"2024-04-20T11:35:18.750729+0000","last_clean":"2024-04-20T11:35:18.750729+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:35:18.750729+0000","last_undegraded":"2024-04-20T11:35:18.750729+0000","last_fullsized":"2024-04-20T11:35:18.750729+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:35:18.750729+0000","last_change":"2024-04-20T11:35:18.750729+0000","last_active":"2024-04-20T11:35:18.750729+0000","last_peered":"2024-04-20T11:35:18.750729+0000","last_clean":"2024-04-20T11:35:18.750729+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:35:18.750729+0000","last_undegraded":"2024-04-20T11:35:18.750729+0000","last_fullsized":"2024-04-20T11:35:18.750729+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:35:18.750729+0000","last_change":"2024-04-20T11:35:18.750729+0000","last_active":"2024-04-20T11:35:18.750729+0000","last_peered":"2024-04-20T11:35:18.750729+0000","last_clean":"2024-04-20T11:35:18.750729+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:35:18.750729+0000","last_undegraded":"2024-04-20T11:35:18.750729+0000","last_fullsized":"2024-04-20T11:35:18.750729+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:35:18.750729+0000","last_change":"2024-04-20T11:35:18.750729+0000","last_active":"2024-04-20T11:35:18.750729+0000","last_peered":"2024-04-20T11:35:18.750729+0000","last_clean":"2024-04-20T11:35:18.750729+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:35:18.750729+0000","last_undegraded":"2024-04-20T11:35:18.750729+0000","last_fullsized":"2024-04-20T11:35:18.750729+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:35:18.750729+0000","last_change":"2024-04-20T11:35:18.750729+0000","last_active":"2024-04-20T11:35:18.750729+0000","last_peered":"2024-04-20T11:35:18.750729+0000","last_clean":"2024-04-20T11:35:18.750729+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:35:18.750729+0000","last_undegraded":"2024-04-20T11:35:18.750729+0000","last_fullsized":"2024-04-20T11:35:18.750729+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:35:18.750729+0000","last_change":"2024-04-20T11:35:18.750729+0000","last_active":"2024-04-20T11:35:18.750729+0000","last_peered":"2024-04-20T11:35:18.750729+0000","last_clean":"2024-04-20T11:35:18.750729+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:35:18.750729+0000","last_undegraded":"2024-04-20T11:35:18.750729+0000","last_fullsized":"2024-04-20T11:35:18.750729+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:35:18.750729+0000","last_change":"2024-04-20T11:35:18.750729+0000","last_active":"2024-04-20T11:35:18.750729+0000","last_peered":"2024-04-20T11:35:18.750729+0000","last_clean":"2024-04-20T11:35:18.750729+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:35:18.750729+0000","last_undegraded":"2024-04-20T11:35:18.750729+0000","last_fullsized":"2024-04-20T11:35:18.750729+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:35:18.750729+0000","last_change":"2024-04-20T11:35:18.750729+0000","last_active":"2024-04-20T11:35:18.750729+0000","last_peered":"2024-04-20T11:35:18.750729+0000","last_clean":"2024-04-20T11:35:18.750729+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:35:18.750729+0000","last_undegraded":"2024-04-20T11:35:18.750729+0000","last_fullsized":"2024-04-20T11:35:18.750729+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:35:18.750729+0000","last_change":"2024-04-20T11:35:18.750729+0000","last_active":"2024-04-20T11:35:18.750729+0000","last_peered":"2024-04-20T11:35:18.750729+0000","last_clean":"2024-04-20T11:35:18.750729+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:35:18.750729+0000","last_undegraded":"2024-04-20T11:35:18.750729+0000","last_fullsized":"2024-04-20T11:35:18.750729+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:35:18.750729+0000","last_change":"2024-04-20T11:35:18.750729+0000","last_active":"2024-04-20T11:35:18.750729+0000","last_peered":"2024-04-20T11:35:18.750729+0000","last_clean":"2024-04-20T11:35:18.750729+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:35:18.750729+0000","last_undegraded":"2024-04-20T11:35:18.750729+0000","last_fullsized":"2024-04-20T11:35:18.750729+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:35:18.750729+0000","last_change":"2024-04-20T11:35:18.750729+0000","last_active":"2024-04-20T11:35:18.750729+0000","last_peered":"2024-04-20T11:35:18.750729+0000","last_clean":"2024-04-20T11:35:18.750729+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:35:18.750729+0000","last_undegraded":"2024-04-20T11:35:18.750729+0000","last_fullsized":"2024-04-20T11:35:18.750729+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:35:18.750729+0000","last_change":"2024-04-20T11:35:18.750729+0000","last_active":"2024-04-20T11:35:18.750729+0000","last_peered":"2024-04-20T11:35:18.750729+0000","last_clean":"2024-04-20T11:35:18.750729+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:35:18.750729+0000","last_undegraded":"2024-04-20T11:35:18.750729+0000","last_fullsized":"2024-04-20T11:35:18.750729+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:35:18.750729+0000","last_change":"2024-04-20T11:35:18.750729+0000","last_active":"2024-04-20T11:35:18.750729+0000","last_peered":"2024-04-20T11:35:18.750729+0000","last_clean":"2024-04-20T11:35:18.750729+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:35:18.750729+0000","last_undegraded":"2024-04-20T11:35:18.750729+0000","last_fullsized":"2024-04-20T11:35:18.750729+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:35:18.750729+0000","last_change":"2024-04-20T11:35:18.750729+0000","last_active":"2024-04-20T11:35:18.750729+0000","last_peered":"2024-04-20T11:35:18.750729+0000","last_clean":"2024-04-20T11:35:18.750729+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:35:18.750729+0000","last_undegraded":"2024-04-20T11:35:18.750729+0000","last_fullsized":"2024-04-20T11:35:18.750729+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:35:18.750729+0000","last_change":"2024-04-20T11:35:18.750729+0000","last_active":"2024-04-20T11:35:18.750729+0000","last_peered":"2024-04-20T11:35:18.750729+0000","last_clean":"2024-04-20T11:35:18.750729+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:35:18.750729+0000","last_undegraded":"2024-04-20T11:35:18.750729+0000","last_fullsized":"2024-04-20T11:35:18.750729+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:35:18.750729+0000","last_change":"2024-04-20T11:35:18.750729+0000","last_active":"2024-04-20T11:35:18.750729+0000","last_peered":"2024-04-20T11:35:18.750729+0000","last_clean":"2024-04-20T11:35:18.750729+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:35:18.750729+0000","last_undegraded":"2024-04-20T11:35:18.750729+0000","last_fullsized":"2024-04-20T11:35:18.750729+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:35:18.750729+0000","last_change":"2024-04-20T11:35:18.750729+0000","last_active":"2024-04-20T11:35:18.750729+0000","last_peered":"2024-04-20T11:35:18.750729+0000","last_clean":"2024-04-20T11:35:18.750729+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:35:18.750729+0000","last_undegraded":"2024-04-20T11:35:18.750729+0000","last_fullsized":"2024-04-20T11:35:18.750729+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:35:18.750729+0000","last_change":"2024-04-20T11:35:18.750729+0000","last_active":"2024-04-20T11:35:18.750729+0000","last_peered":"2024-04-20T11:35:18.750729+0000","last_clean":"2024-04-20T11:35:18.750729+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:35:18.750729+0000","last_undegraded":"2024-04-20T11:35:18.750729+0000","last_fullsized":"2024-04-20T11:35:18.750729+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:35:18.750729+0000","last_change":"2024-04-20T11:35:18.750729+0000","last_active":"2024-04-20T11:35:18.750729+0000","last_peered":"2024-04-20T11:35:18.750729+0000","last_clean":"2024-04-20T11:35:18.750729+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:35:18.750729+0000","last_undegraded":"2024-04-20T11:35:18.750729+0000","last_fullsized":"2024-04-20T11:35:18.750729+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:35:18.750729+0000","last_change":"2024-04-20T11:35:18.750729+0000","last_active":"2024-04-20T11:35:18.750729+0000","last_peered":"2024-04-20T11:35:18.750729+0000","last_clean":"2024-04-20T11:35:18.750729+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:35:18.750729+0000","last_undegraded":"2024-04-20T11:35:18.750729+0000","last_fullsized":"2024-04-20T11:35:18.750729+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:35:18.750729+0000","last_change":"2024-04-20T11:35:18.750729+0000","last_active":"2024-04-20T11:35:18.750729+0000","last_peered":"2024-04-20T11:35:18.750729+0000","last_clean":"2024-04-20T11:35:18.750729+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:35:18.750729+0000","last_undegraded":"2024-04-20T11:35:18.750729+0000","last_fullsized":"2024-04-20T11:35:18.750729+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:35:18.750729+0000","last_change":"2024-04-20T11:35:18.750729+0000","last_active":"2024-04-20T11:35:18.750729+0000","last_peered":"2024-04-20T11:35:18.750729+0000","last_clean":"2024-04-20T11:35:18.750729+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:35:18.750729+0000","last_undegraded":"2024-04-20T11:35:18.750729+0000","last_fullsized":"2024-04-20T11:35:18.750729+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:35:18.750729+0000","last_change":"2024-04-20T11:35:18.750729+0000","last_active":"2024-04-20T11:35:18.750729+0000","last_peered":"2024-04-20T11:35:18.750729+0000","last_clean":"2024-04-20T11:35:18.750729+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:35:18.750729+0000","last_undegraded":"2024-04-20T11:35:18.750729+0000","last_fullsized":"2024-04-20T11:35:18.750729+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:35:18.750729+0000","last_change":"2024-04-20T11:35:18.750729+0000","last_active":"2024-04-20T11:35:18.750729+0000","last_peered":"2024-04-20T11:35:18.750729+0000","last_clean":"2024-04-20T11:35:18.750729+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:35:18.750729+0000","last_undegraded":"2024-04-20T11:35:18.750729+0000","last_fullsized":"2024-04-20T11:35:18.750729+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:35:18.750729+0000","last_change":"2024-04-20T11:35:18.750729+0000","last_active":"2024-04-20T11:35:18.750729+0000","last_peered":"2024-04-20T11:35:18.750729+0000","last_clean":"2024-04-20T11:35:18.750729+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:35:18.750729+0000","last_undegraded":"2024-04-20T11:35:18.750729+0000","last_fullsized":"2024-04-20T11:35:18.750729+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":1,"reported_epoch":128,"state":"creating+peering","last_fresh":"2024-04-20T11:35:17.765538+0000","last_change":"2024-04-20T11:35:17.765538+0000","last_active":"2024-04-20T11:35:17.736274+0000","last_peered":"2024-04-20T11:35:17.736274+0000","last_clean":"2024-04-20T11:35:17.736274+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:35:17.765538+0000","last_undegraded":"2024-04-20T11:35:17.765538+0000","last_fullsized":"2024-04-20T11:35:17.765538+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_clean_scrub_stamp":"2024-04-20T11:35:17.736274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:35:18.750729+0000","last_change":"2024-04-20T11:35:18.750729+0000","last_active":"2024-04-20T11:35:18.750729+0000","last_peered":"2024-04-20T11:35:18.750729+0000","last_clean":"2024-04-20T11:35:18.750729+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:35:18.750729+0000","last_undegraded":"2024-04-20T11:35:18.750729+0000","last_fullsized":"2024-04-20T11:35:18.750729+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":1,"reported_epoch":128,"state":"creating+peering","last_fresh":"2024-04-20T11:35:17.757884+0000","last_change":"2024-04-20T11:35:17.757884+0000","last_active":"2024-04-20T11:35:17.736274+0000","last_peered":"2024-04-20T11:35:17.736274+0000","last_clean":"2024-04-20T11:35:17.736274+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:35:17.757884+0000","last_undegraded":"2024-04-20T11:35:17.757884+0000","last_fullsized":"2024-04-20T11:35:17.757884+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_clean_scrub_stamp":"2024-04-20T11:35:17.736274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:35:18.750729+0000","last_change":"2024-04-20T11:35:18.750729+0000","last_active":"2024-04-20T11:35:18.750729+0000","last_peered":"2024-04-20T11:35:18.750729+0000","last_clean":"2024-04-20T11:35:18.750729+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:35:18.750729+0000","last_undegraded":"2024-04-20T11:35:18.750729+0000","last_fullsized":"2024-04-20T11:35:18.750729+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":1,"reported_epoch":128,"state":"creating+peering","last_fresh":"2024-04-20T11:35:17.770361+0000","last_change":"2024-04-20T11:35:17.770361+0000","last_active":"2024-04-20T11:35:17.736274+0000","last_peered":"2024-04-20T11:35:17.736274+0000","last_clean":"2024-04-20T11:35:17.736274+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:35:17.770361+0000","last_undegraded":"2024-04-20T11:35:17.770361+0000","last_fullsized":"2024-04-20T11:35:17.770361+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_clean_scrub_stamp":"2024-04-20T11:35:17.736274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:35:18.750729+0000","last_change":"2024-04-20T11:35:18.750729+0000","last_active":"2024-04-20T11:35:18.750729+0000","last_peered":"2024-04-20T11:35:18.750729+0000","last_clean":"2024-04-20T11:35:18.750729+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:35:18.750729+0000","last_undegraded":"2024-04-20T11:35:18.750729+0000","last_fullsized":"2024-04-20T11:35:18.750729+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":1,"reported_epoch":128,"state":"creating+peering","last_fresh":"2024-04-20T11:35:17.770224+0000","last_change":"2024-04-20T11:35:17.770224+0000","last_active":"2024-04-20T11:35:17.736274+0000","last_peered":"2024-04-20T11:35:17.736274+0000","last_clean":"2024-04-20T11:35:17.736274+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:35:17.770224+0000","last_undegraded":"2024-04-20T11:35:17.770224+0000","last_fullsized":"2024-04-20T11:35:17.770224+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_clean_scrub_stamp":"2024-04-20T11:35:17.736274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:35:18.750729+0000","last_change":"2024-04-20T11:35:18.750729+0000","last_active":"2024-04-20T11:35:18.750729+0000","last_peered":"2024-04-20T11:35:18.750729+0000","last_clean":"2024-04-20T11:35:18.750729+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:35:18.750729+0000","last_undegraded":"2024-04-20T11:35:18.750729+0000","last_fullsized":"2024-04-20T11:35:18.750729+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":1,"reported_epoch":128,"state":"creating+peering","last_fresh":"2024-04-20T11:35:17.763029+0000","last_change":"2024-04-20T11:35:17.763029+0000","last_active":"2024-04-20T11:35:17.736274+0000","last_peered":"2024-04-20T11:35:17.736274+0000","last_clean":"2024-04-20T11:35:17.736274+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:35:17.763029+0000","last_undegraded":"2024-04-20T11:35:17.763029+0000","last_fullsized":"2024-04-20T11:35:17.763029+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_clean_scrub_stamp":"2024-04-20T11:35:17.736274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:35:18.750729+0000","last_change":"2024-04-20T11:35:18.750729+0000","last_active":"2024-04-20T11:35:18.750729+0000","last_peered":"2024-04-20T11:35:18.750729+0000","last_clean":"2024-04-20T11:35:18.750729+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:35:18.750729+0000","last_undegraded":"2024-04-20T11:35:18.750729+0000","last_fullsized":"2024-04-20T11:35:18.750729+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":1,"reported_epoch":128,"state":"creating+peering","last_fresh":"2024-04-20T11:35:17.758623+0000","last_change":"2024-04-20T11:35:17.758623+0000","last_active":"2024-04-20T11:35:17.736274+0000","last_peered":"2024-04-20T11:35:17.736274+0000","last_clean":"2024-04-20T11:35:17.736274+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:35:17.758623+0000","last_undegraded":"2024-04-20T11:35:17.758623+0000","last_fullsized":"2024-04-20T11:35:17.758623+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_clean_scrub_stamp":"2024-04-20T11:35:17.736274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:35:18.750729+0000","last_change":"2024-04-20T11:35:18.750729+0000","last_active":"2024-04-20T11:35:18.750729+0000","last_peered":"2024-04-20T11:35:18.750729+0000","last_clean":"2024-04-20T11:35:18.750729+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:35:18.750729+0000","last_undegraded":"2024-04-20T11:35:18.750729+0000","last_fullsized":"2024-04-20T11:35:18.750729+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":1,"reported_epoch":128,"state":"creating+peering","last_fresh":"2024-04-20T11:35:17.763500+0000","last_change":"2024-04-20T11:35:17.763500+0000","last_active":"2024-04-20T11:35:17.736274+0000","last_peered":"2024-04-20T11:35:17.736274+0000","last_clean":"2024-04-20T11:35:17.736274+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:35:17.763500+0000","last_undegraded":"2024-04-20T11:35:17.763500+0000","last_fullsized":"2024-04-20T11:35:17.763500+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_clean_scrub_stamp":"2024-04-20T11:35:17.736274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:35:18.750729+0000","last_change":"2024-04-20T11:35:18.750729+0000","last_active":"2024-04-20T11:35:18.750729+0000","last_peered":"2024-04-20T11:35:18.750729+0000","last_clean":"2024-04-20T11:35:18.750729+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:35:18.750729+0000","last_undegraded":"2024-04-20T11:35:18.750729+0000","last_fullsized":"2024-04-20T11:35:18.750729+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":1,"reported_epoch":128,"state":"creating+peering","last_fresh":"2024-04-20T11:35:17.768926+0000","last_change":"2024-04-20T11:35:17.768926+0000","last_active":"2024-04-20T11:35:17.736274+0000","last_peered":"2024-04-20T11:35:17.736274+0000","last_clean":"2024-04-20T11:35:17.736274+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:35:17.768926+0000","last_undegraded":"2024-04-20T11:35:17.768926+0000","last_fullsized":"2024-04-20T11:35:17.768926+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_clean_scrub_stamp":"2024-04-20T11:35:17.736274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:35:18.750729+0000","last_change":"2024-04-20T11:35:18.750729+0000","last_active":"2024-04-20T11:35:18.750729+0000","last_peered":"2024-04-20T11:35:18.750729+0000","last_clean":"2024-04-20T11:35:18.750729+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:35:18.750729+0000","last_undegraded":"2024-04-20T11:35:18.750729+0000","last_fullsized":"2024-04-20T11:35:18.750729+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":1,"reported_epoch":128,"state":"creating+peering","last_fresh":"2024-04-20T11:35:17.757390+0000","last_change":"2024-04-20T11:35:17.757390+0000","last_active":"2024-04-20T11:35:17.736274+0000","last_peered":"2024-04-20T11:35:17.736274+0000","last_clean":"2024-04-20T11:35:17.736274+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:35:17.757390+0000","last_undegraded":"2024-04-20T11:35:17.757390+0000","last_fullsized":"2024-04-20T11:35:17.757390+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_clean_scrub_stamp":"2024-04-20T11:35:17.736274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:35:18.750729+0000","last_change":"2024-04-20T11:35:18.750729+0000","last_active":"2024-04-20T11:35:18.750729+0000","last_peered":"2024-04-20T11:35:18.750729+0000","last_clean":"2024-04-20T11:35:18.750729+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:35:18.750729+0000","last_undegraded":"2024-04-20T11:35:18.750729+0000","last_fullsized":"2024-04-20T11:35:18.750729+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":1,"reported_epoch":128,"state":"creating+peering","last_fresh":"2024-04-20T11:35:17.764810+0000","last_change":"2024-04-20T11:35:17.764810+0000","last_active":"2024-04-20T11:35:17.736274+0000","last_peered":"2024-04-20T11:35:17.736274+0000","last_clean":"2024-04-20T11:35:17.736274+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:35:17.764810+0000","last_undegraded":"2024-04-20T11:35:17.764810+0000","last_fullsized":"2024-04-20T11:35:17.764810+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_clean_scrub_stamp":"2024-04-20T11:35:17.736274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:35:18.750729+0000","last_change":"2024-04-20T11:35:18.750729+0000","last_active":"2024-04-20T11:35:18.750729+0000","last_peered":"2024-04-20T11:35:18.750729+0000","last_clean":"2024-04-20T11:35:18.750729+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:35:18.750729+0000","last_undegraded":"2024-04-20T11:35:18.750729+0000","last_fullsized":"2024-04-20T11:35:18.750729+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":1,"reported_epoch":128,"state":"creating+peering","last_fresh":"2024-04-20T11:35:17.759505+0000","last_change":"2024-04-20T11:35:17.759505+0000","last_active":"2024-04-20T11:35:17.736274+0000","last_peered":"2024-04-20T11:35:17.736274+0000","last_clean":"2024-04-20T11:35:17.736274+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:35:17.759505+0000","last_undegraded":"2024-04-20T11:35:17.759505+0000","last_fullsized":"2024-04-20T11:35:17.759505+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_clean_scrub_stamp":"2024-04-20T11:35:17.736274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:35:18.750729+0000","last_change":"2024-04-20T11:35:18.750729+0000","last_active":"2024-04-20T11:35:18.750729+0000","last_peered":"2024-04-20T11:35:18.750729+0000","last_clean":"2024-04-20T11:35:18.750729+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:35:18.750729+0000","last_undegraded":"2024-04-20T11:35:18.750729+0000","last_fullsized":"2024-04-20T11:35:18.750729+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":1,"reported_epoch":128,"state":"creating+peering","last_fresh":"2024-04-20T11:35:17.763365+0000","last_change":"2024-04-20T11:35:17.763365+0000","last_active":"2024-04-20T11:35:17.736274+0000","last_peered":"2024-04-20T11:35:17.736274+0000","last_clean":"2024-04-20T11:35:17.736274+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:35:17.763365+0000","last_undegraded":"2024-04-20T11:35:17.763365+0000","last_fullsized":"2024-04-20T11:35:17.763365+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_clean_scrub_stamp":"2024-04-20T11:35:17.736274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:35:18.750729+0000","last_change":"2024-04-20T11:35:18.750729+0000","last_active":"2024-04-20T11:35:18.750729+0000","last_peered":"2024-04-20T11:35:18.750729+0000","last_clean":"2024-04-20T11:35:18.750729+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:35:18.750729+0000","last_undegraded":"2024-04-20T11:35:18.750729+0000","last_fullsized":"2024-04-20T11:35:18.750729+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":1,"reported_epoch":128,"state":"creating+peering","last_fresh":"2024-04-20T11:35:17.763639+0000","last_change":"2024-04-20T11:35:17.763639+0000","last_active":"2024-04-20T11:35:17.736274+0000","last_peered":"2024-04-20T11:35:17.736274+0000","last_clean":"2024-04-20T11:35:17.736274+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:35:17.763639+0000","last_undegraded":"2024-04-20T11:35:17.763639+0000","last_fullsized":"2024-04-20T11:35:17.763639+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_clean_scrub_stamp":"2024-04-20T11:35:17.736274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:35:18.750729+0000","last_change":"2024-04-20T11:35:18.750729+0000","last_active":"2024-04-20T11:35:18.750729+0000","last_peered":"2024-04-20T11:35:18.750729+0000","last_clean":"2024-04-20T11:35:18.750729+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:35:18.750729+0000","last_undegraded":"2024-04-20T11:35:18.750729+0000","last_fullsized":"2024-04-20T11:35:18.750729+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":1,"reported_epoch":128,"state":"creating+peering","last_fresh":"2024-04-20T11:35:17.762179+0000","last_change":"2024-04-20T11:35:17.762179+0000","last_active":"2024-04-20T11:35:17.736274+0000","last_peered":"2024-04-20T11:35:17.736274+0000","last_clean":"2024-04-20T11:35:17.736274+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:35:17.762179+0000","last_undegraded":"2024-04-20T11:35:17.762179+0000","last_fullsized":"2024-04-20T11:35:17.762179+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_clean_scrub_stamp":"2024-04-20T11:35:17.736274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:35:18.750729+0000","last_change":"2024-04-20T11:35:18.750729+0000","last_active":"2024-04-20T11:35:18.750729+0000","last_peered":"2024-04-20T11:35:18.750729+0000","last_clean":"2024-04-20T11:35:18.750729+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:35:18.750729+0000","last_undegraded":"2024-04-20T11:35:18.750729+0000","last_fullsized":"2024-04-20T11:35:18.750729+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":1,"reported_epoch":128,"state":"creating+peering","last_fresh":"2024-04-20T11:35:17.761634+0000","last_change":"2024-04-20T11:35:17.761634+0000","last_active":"2024-04-20T11:35:17.736274+0000","last_peered":"2024-04-20T11:35:17.736274+0000","last_clean":"2024-04-20T11:35:17.736274+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:35:17.761634+0000","last_undegraded":"2024-04-20T11:35:17.761634+0000","last_fullsized":"2024-04-20T11:35:17.761634+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_clean_scrub_stamp":"2024-04-20T11:35:17.736274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:35:18.750729+0000","last_change":"2024-04-20T11:35:18.750729+0000","last_active":"2024-04-20T11:35:18.750729+0000","last_peered":"2024-04-20T11:35:18.750729+0000","last_clean":"2024-04-20T11:35:18.750729+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:35:18.750729+0000","last_undegraded":"2024-04-20T11:35:18.750729+0000","last_fullsized":"2024-04-20T11:35:18.750729+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":1,"reported_epoch":128,"state":"creating+peering","last_fresh":"2024-04-20T11:35:17.758670+0000","last_change":"2024-04-20T11:35:17.758670+0000","last_active":"2024-04-20T11:35:17.736274+0000","last_peered":"2024-04-20T11:35:17.736274+0000","last_clean":"2024-04-20T11:35:17.736274+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:35:17.758670+0000","last_undegraded":"2024-04-20T11:35:17.758670+0000","last_fullsized":"2024-04-20T11:35:17.758670+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_clean_scrub_stamp":"2024-04-20T11:35:17.736274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:35:18.750729+0000","last_change":"2024-04-20T11:35:18.750729+0000","last_active":"2024-04-20T11:35:18.750729+0000","last_peered":"2024-04-20T11:35:18.750729+0000","last_clean":"2024-04-20T11:35:18.750729+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:35:18.750729+0000","last_undegraded":"2024-04-20T11:35:18.750729+0000","last_fullsized":"2024-04-20T11:35:18.750729+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":1,"reported_epoch":128,"state":"creating+peering","last_fresh":"2024-04-20T11:35:17.757256+0000","last_change":"2024-04-20T11:35:17.757256+0000","last_active":"2024-04-20T11:35:17.736274+0000","last_peered":"2024-04-20T11:35:17.736274+0000","last_clean":"2024-04-20T11:35:17.736274+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:35:17.757256+0000","last_undegraded":"2024-04-20T11:35:17.757256+0000","last_fullsized":"2024-04-20T11:35:17.757256+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_clean_scrub_stamp":"2024-04-20T11:35:17.736274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:35:18.750729+0000","last_change":"2024-04-20T11:35:18.750729+0000","last_active":"2024-04-20T11:35:18.750729+0000","last_peered":"2024-04-20T11:35:18.750729+0000","last_clean":"2024-04-20T11:35:18.750729+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:35:18.750729+0000","last_undegraded":"2024-04-20T11:35:18.750729+0000","last_fullsized":"2024-04-20T11:35:18.750729+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":1,"reported_epoch":128,"state":"creating+peering","last_fresh":"2024-04-20T11:35:17.765519+0000","last_change":"2024-04-20T11:35:17.765519+0000","last_active":"2024-04-20T11:35:17.736274+0000","last_peered":"2024-04-20T11:35:17.736274+0000","last_clean":"2024-04-20T11:35:17.736274+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:35:17.765519+0000","last_undegraded":"2024-04-20T11:35:17.765519+0000","last_fullsized":"2024-04-20T11:35:17.765519+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_clean_scrub_stamp":"2024-04-20T11:35:17.736274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:35:18.750729+0000","last_change":"2024-04-20T11:35:18.750729+0000","last_active":"2024-04-20T11:35:18.750729+0000","last_peered":"2024-04-20T11:35:18.750729+0000","last_clean":"2024-04-20T11:35:18.750729+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:35:18.750729+0000","last_undegraded":"2024-04-20T11:35:18.750729+0000","last_fullsized":"2024-04-20T11:35:18.750729+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":1,"reported_epoch":128,"state":"creating+peering","last_fresh":"2024-04-20T11:35:17.756742+0000","last_change":"2024-04-20T11:35:17.756742+0000","last_active":"2024-04-20T11:35:17.736274+0000","last_peered":"2024-04-20T11:35:17.736274+0000","last_clean":"2024-04-20T11:35:17.736274+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:35:17.756742+0000","last_undegraded":"2024-04-20T11:35:17.756742+0000","last_fullsized":"2024-04-20T11:35:17.756742+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_clean_scrub_stamp":"2024-04-20T11:35:17.736274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:35:18.750729+0000","last_change":"2024-04-20T11:35:18.750729+0000","last_active":"2024-04-20T11:35:18.750729+0000","last_peered":"2024-04-20T11:35:18.750729+0000","last_clean":"2024-04-20T11:35:18.750729+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:35:18.750729+0000","last_undegraded":"2024-04-20T11:35:18.750729+0000","last_fullsized":"2024-04-20T11:35:18.750729+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":1,"reported_epoch":128,"state":"creating+peering","last_fresh":"2024-04-20T11:35:17.760230+0000","last_change":"2024-04-20T11:35:17.760230+0000","last_active":"2024-04-20T11:35:17.736274+0000","last_peered":"2024-04-20T11:35:17.736274+0000","last_clean":"2024-04-20T11:35:17.736274+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:35:17.760230+0000","last_undegraded":"2024-04-20T11:35:17.760230+0000","last_fullsized":"2024-04-20T11:35:17.760230+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_clean_scrub_stamp":"2024-04-20T11:35:17.736274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:35:18.750729+0000","last_change":"2024-04-20T11:35:18.750729+0000","last_active":"2024-04-20T11:35:18.750729+0000","last_peered":"2024-04-20T11:35:18.750729+0000","last_clean":"2024-04-20T11:35:18.750729+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:35:18.750729+0000","last_undegraded":"2024-04-20T11:35:18.750729+0000","last_fullsized":"2024-04-20T11:35:18.750729+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":1,"reported_epoch":128,"state":"creating+peering","last_fresh":"2024-04-20T11:35:17.765300+0000","last_change":"2024-04-20T11:35:17.765300+0000","last_active":"2024-04-20T11:35:17.736274+0000","last_peered":"2024-04-20T11:35:17.736274+0000","last_clean":"2024-04-20T11:35:17.736274+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:35:17.765300+0000","last_undegraded":"2024-04-20T11:35:17.765300+0000","last_fullsized":"2024-04-20T11:35:17.765300+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_clean_scrub_stamp":"2024-04-20T11:35:17.736274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:35:18.750729+0000","last_change":"2024-04-20T11:35:18.750729+0000","last_active":"2024-04-20T11:35:18.750729+0000","last_peered":"2024-04-20T11:35:18.750729+0000","last_clean":"2024-04-20T11:35:18.750729+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:35:18.750729+0000","last_undegraded":"2024-04-20T11:35:18.750729+0000","last_fullsized":"2024-04-20T11:35:18.750729+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":1,"reported_epoch":128,"state":"creating+peering","last_fresh":"2024-04-20T11:35:17.763179+0000","last_change":"2024-04-20T11:35:17.763179+0000","last_active":"2024-04-20T11:35:17.736274+0000","last_peered":"2024-04-20T11:35:17.736274+0000","last_clean":"2024-04-20T11:35:17.736274+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:35:17.763179+0000","last_undegraded":"2024-04-20T11:35:17.763179+0000","last_fullsized":"2024-04-20T11:35:17.763179+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_clean_scrub_stamp":"2024-04-20T11:35:17.736274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:35:18.750729+0000","last_change":"2024-04-20T11:35:18.750729+0000","last_active":"2024-04-20T11:35:18.750729+0000","last_peered":"2024-04-20T11:35:18.750729+0000","last_clean":"2024-04-20T11:35:18.750729+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:35:18.750729+0000","last_undegraded":"2024-04-20T11:35:18.750729+0000","last_fullsized":"2024-04-20T11:35:18.750729+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":1,"reported_epoch":128,"state":"creating+peering","last_fresh":"2024-04-20T11:35:17.763700+0000","last_change":"2024-04-20T11:35:17.763700+0000","last_active":"2024-04-20T11:35:17.736274+0000","last_peered":"2024-04-20T11:35:17.736274+0000","last_clean":"2024-04-20T11:35:17.736274+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:35:17.763700+0000","last_undegraded":"2024-04-20T11:35:17.763700+0000","last_fullsized":"2024-04-20T11:35:17.763700+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_clean_scrub_stamp":"2024-04-20T11:35:17.736274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:35:18.750729+0000","last_change":"2024-04-20T11:35:18.750729+0000","last_active":"2024-04-20T11:35:18.750729+0000","last_peered":"2024-04-20T11:35:18.750729+0000","last_clean":"2024-04-20T11:35:18.750729+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:35:18.750729+0000","last_undegraded":"2024-04-20T11:35:18.750729+0000","last_fullsized":"2024-04-20T11:35:18.750729+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":1,"reported_epoch":128,"state":"creating+peering","last_fresh":"2024-04-20T11:35:17.764314+0000","last_change":"2024-04-20T11:35:17.764314+0000","last_active":"2024-04-20T11:35:17.736274+0000","last_peered":"2024-04-20T11:35:17.736274+0000","last_clean":"2024-04-20T11:35:17.736274+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:35:17.764314+0000","last_undegraded":"2024-04-20T11:35:17.764314+0000","last_fullsized":"2024-04-20T11:35:17.764314+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_clean_scrub_stamp":"2024-04-20T11:35:17.736274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:35:18.750729+0000","last_change":"2024-04-20T11:35:18.750729+0000","last_active":"2024-04-20T11:35:18.750729+0000","last_peered":"2024-04-20T11:35:18.750729+0000","last_clean":"2024-04-20T11:35:18.750729+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:35:18.750729+0000","last_undegraded":"2024-04-20T11:35:18.750729+0000","last_fullsized":"2024-04-20T11:35:18.750729+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":1,"reported_epoch":128,"state":"creating+peering","last_fresh":"2024-04-20T11:35:17.762056+0000","last_change":"2024-04-20T11:35:17.762056+0000","last_active":"2024-04-20T11:35:17.736274+0000","last_peered":"2024-04-20T11:35:17.736274+0000","last_clean":"2024-04-20T11:35:17.736274+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:35:17.762056+0000","last_undegraded":"2024-04-20T11:35:17.762056+0000","last_fullsized":"2024-04-20T11:35:17.762056+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_clean_scrub_stamp":"2024-04-20T11:35:17.736274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:35:18.750729+0000","last_change":"2024-04-20T11:35:18.750729+0000","last_active":"2024-04-20T11:35:18.750729+0000","last_peered":"2024-04-20T11:35:18.750729+0000","last_clean":"2024-04-20T11:35:18.750729+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:35:18.750729+0000","last_undegraded":"2024-04-20T11:35:18.750729+0000","last_fullsized":"2024-04-20T11:35:18.750729+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":1,"reported_epoch":128,"state":"creating+peering","last_fresh":"2024-04-20T11:35:17.764505+0000","last_change":"2024-04-20T11:35:17.764505+0000","last_active":"2024-04-20T11:35:17.736274+0000","last_peered":"2024-04-20T11:35:17.736274+0000","last_clean":"2024-04-20T11:35:17.736274+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:35:17.764505+0000","last_undegraded":"2024-04-20T11:35:17.764505+0000","last_fullsized":"2024-04-20T11:35:17.764505+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_clean_scrub_stamp":"2024-04-20T11:35:17.736274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:35:18.750729+0000","last_change":"2024-04-20T11:35:18.750729+0000","last_active":"2024-04-20T11:35:18.750729+0000","last_peered":"2024-04-20T11:35:18.750729+0000","last_clean":"2024-04-20T11:35:18.750729+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:35:18.750729+0000","last_undegraded":"2024-04-20T11:35:18.750729+0000","last_fullsized":"2024-04-20T11:35:18.750729+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":1,"reported_epoch":128,"state":"creating+peering","last_fresh":"2024-04-20T11:35:17.762071+0000","last_change":"2024-04-20T11:35:17.762071+0000","last_active":"2024-04-20T11:35:17.736274+0000","last_peered":"2024-04-20T11:35:17.736274+0000","last_clean":"2024-04-20T11:35:17.736274+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:35:17.762071+0000","last_undegraded":"2024-04-20T11:35:17.762071+0000","last_fullsized":"2024-04-20T11:35:17.762071+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_clean_scrub_stamp":"2024-04-20T11:35:17.736274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:35:18.750729+0000","last_change":"2024-04-20T11:35:18.750729+0000","last_active":"2024-04-20T11:35:18.750729+0000","last_peered":"2024-04-20T11:35:18.750729+0000","last_clean":"2024-04-20T11:35:18.750729+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:35:18.750729+0000","last_undegraded":"2024-04-20T11:35:18.750729+0000","last_fullsized":"2024-04-20T11:35:18.750729+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":1,"reported_epoch":128,"state":"creating+peering","last_fresh":"2024-04-20T11:35:17.759489+0000","last_change":"2024-04-20T11:35:17.759489+0000","last_active":"2024-04-20T11:35:17.736274+0000","last_peered":"2024-04-20T11:35:17.736274+0000","last_clean":"2024-04-20T11:35:17.736274+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:35:17.759489+0000","last_undegraded":"2024-04-20T11:35:17.759489+0000","last_fullsized":"2024-04-20T11:35:17.759489+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_clean_scrub_stamp":"2024-04-20T11:35:17.736274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:35:18.750729+0000","last_change":"2024-04-20T11:35:18.750729+0000","last_active":"2024-04-20T11:35:18.750729+0000","last_peered":"2024-04-20T11:35:18.750729+0000","last_clean":"2024-04-20T11:35:18.750729+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:35:18.750729+0000","last_undegraded":"2024-04-20T11:35:18.750729+0000","last_fullsized":"2024-04-20T11:35:18.750729+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":1,"reported_epoch":128,"state":"creating+peering","last_fresh":"2024-04-20T11:35:17.768872+0000","last_change":"2024-04-20T11:35:17.768872+0000","last_active":"2024-04-20T11:35:17.736274+0000","last_peered":"2024-04-20T11:35:17.736274+0000","last_clean":"2024-04-20T11:35:17.736274+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:35:17.768872+0000","last_undegraded":"2024-04-20T11:35:17.768872+0000","last_fullsized":"2024-04-20T11:35:17.768872+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_clean_scrub_stamp":"2024-04-20T11:35:17.736274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:35:18.750729+0000","last_change":"2024-04-20T11:35:18.750729+0000","last_active":"2024-04-20T11:35:18.750729+0000","last_peered":"2024-04-20T11:35:18.750729+0000","last_clean":"2024-04-20T11:35:18.750729+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:35:18.750729+0000","last_undegraded":"2024-04-20T11:35:18.750729+0000","last_fullsized":"2024-04-20T11:35:18.750729+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":1,"reported_epoch":128,"state":"creating+peering","last_fresh":"2024-04-20T11:35:17.763199+0000","last_change":"2024-04-20T11:35:17.763199+0000","last_active":"2024-04-20T11:35:17.736274+0000","last_peered":"2024-04-20T11:35:17.736274+0000","last_clean":"2024-04-20T11:35:17.736274+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:35:17.763199+0000","last_undegraded":"2024-04-20T11:35:17.763199+0000","last_fullsized":"2024-04-20T11:35:17.763199+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_clean_scrub_stamp":"2024-04-20T11:35:17.736274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:35:18.750729+0000","last_change":"2024-04-20T11:35:18.750729+0000","last_active":"2024-04-20T11:35:18.750729+0000","last_peered":"2024-04-20T11:35:18.750729+0000","last_clean":"2024-04-20T11:35:18.750729+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:35:18.750729+0000","last_undegraded":"2024-04-20T11:35:18.750729+0000","last_fullsized":"2024-04-20T11:35:18.750729+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":1,"reported_epoch":128,"state":"creating+peering","last_fresh":"2024-04-20T11:35:17.757334+0000","last_change":"2024-04-20T11:35:17.757334+0000","last_active":"2024-04-20T11:35:17.736274+0000","last_peered":"2024-04-20T11:35:17.736274+0000","last_clean":"2024-04-20T11:35:17.736274+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:35:17.757334+0000","last_undegraded":"2024-04-20T11:35:17.757334+0000","last_fullsized":"2024-04-20T11:35:17.757334+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_clean_scrub_stamp":"2024-04-20T11:35:17.736274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:35:18.750729+0000","last_change":"2024-04-20T11:35:18.750729+0000","last_active":"2024-04-20T11:35:18.750729+0000","last_peered":"2024-04-20T11:35:18.750729+0000","last_clean":"2024-04-20T11:35:18.750729+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:35:18.750729+0000","last_undegraded":"2024-04-20T11:35:18.750729+0000","last_fullsized":"2024-04-20T11:35:18.750729+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":1,"reported_epoch":128,"state":"creating+peering","last_fresh":"2024-04-20T11:35:17.763780+0000","last_change":"2024-04-20T11:35:17.763780+0000","last_active":"2024-04-20T11:35:17.736274+0000","last_peered":"2024-04-20T11:35:17.736274+0000","last_clean":"2024-04-20T11:35:17.736274+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:35:17.763780+0000","last_undegraded":"2024-04-20T11:35:17.763780+0000","last_fullsized":"2024-04-20T11:35:17.763780+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_clean_scrub_stamp":"2024-04-20T11:35:17.736274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"72'128","reported_seq":401,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-04-20T11:35:17.773789+0000","last_change":"2024-04-20T11:26:48.661897+0000","last_active":"2024-04-20T11:35:17.773789+0000","last_peered":"2024-04-20T11:35:17.773789+0000","last_clean":"2024-04-20T11:35:17.773789+0000","last_became_active":"2024-04-20T11:26:48.656535+0000","last_became_peered":"2024-04-20T11:26:48.656535+0000","last_unstale":"2024-04-20T11:35:17.773789+0000","last_undegraded":"2024-04-20T11:35:17.773789+0000","last_fullsized":"2024-04-20T11:35:17.773789+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:47.543004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:47.543004+0000","last_clean_scrub_stamp":"2024-04-20T11:26:47.543004+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:58:35.527105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574956,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33328,"kb_used_data":2148,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338512,"statfs":{"total":96636764160,"available":96602636288,"internally_reserved":0,"allocated":2199552,"data_stored":1297536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574953,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32748,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339092,"statfs":{"total":96636764160,"available":96603230208,"internally_reserved":0,"allocated":1605632,"data_stored":707168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574955,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32748,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339092,"statfs":{"total":96636764160,"available":96603230208,"internally_reserved":0,"allocated":1605632,"data_stored":707168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574953,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32736,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339104,"statfs":{"total":96636764160,"available":96603242496,"internally_reserved":0,"allocated":1593344,"data_stored":701351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574954,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32748,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339092,"statfs":{"total":96636764160,"available":96603230208,"internally_reserved":0,"allocated":1605632,"data_stored":707168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574955,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32748,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339092,"statfs":{"total":96636764160,"available":96603230208,"internally_reserved":0,"allocated":1605632,"data_stored":707168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574953,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32748,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339092,"statfs":{"total":96636764160,"available":96603230208,"internally_reserved":0,"allocated":1605632,"data_stored":707168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574954,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33328,"kb_used_data":2148,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338512,"statfs":{"total":96636764160,"available":96602636288,"internally_reserved":0,"allocated":2199552,"data_stored":1297536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T11:35:24.578 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T11:35:24.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.821+0000 7f882e600640 1 Processor -- start 2024-04-20T11:35:24.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.821+0000 7f882e600640 1 -- start start 2024-04-20T11:35:24.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.823+0000 7f882e600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f882815fcc0 0x7f88280d4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:24.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.823+0000 7f882e600640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f88281603d0 0x7f88280b5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:24.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.823+0000 7f882e600640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f88280b5480 0x7f88280b63f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:24.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.823+0000 7f882e600640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f88280d3350 con 0x7f88280b5480 2024-04-20T11:35:24.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.823+0000 7f882e600640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f88280d3630 con 0x7f88281603d0 2024-04-20T11:35:24.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.823+0000 7f882e600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f88280d3910 con 0x7f882815fcc0 2024-04-20T11:35:24.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.823+0000 7f882dc00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f88280b5480 0x7f88280b63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:24.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.823+0000 7f882dc00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f88280b5480 0x7f88280b63f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52388/0 (socket says 172.21.15.152:52388) 2024-04-20T11:35:24.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.823+0000 7f882dc00640 1 -- 172.21.15.152:0/3848391497 learned_addr learned my addr 172.21.15.152:0/3848391497 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:24.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.823+0000 7f882d200640 1 --2- 172.21.15.152:0/3848391497 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f882815fcc0 0x7f88280d4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:24.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.823+0000 7f8827e00640 1 --2- 172.21.15.152:0/3848391497 >> v2:172.21.15.175:3300/0 conn(0x7f88281603d0 0x7f88280b5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:24.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.823+0000 7f882dc00640 1 -- 172.21.15.152:0/3848391497 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f882815fcc0 msgr2=0x7f88280d4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:24.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.823+0000 7f882dc00640 1 --2- 172.21.15.152:0/3848391497 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f882815fcc0 0x7f88280d4e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:24.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.823+0000 7f882dc00640 1 -- 172.21.15.152:0/3848391497 >> v2:172.21.15.175:3300/0 conn(0x7f88281603d0 msgr2=0x7f88280b5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:24.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.823+0000 7f882dc00640 1 --2- 172.21.15.152:0/3848391497 >> v2:172.21.15.175:3300/0 conn(0x7f88281603d0 0x7f88280b5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:24.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.824+0000 7f882dc00640 1 -- 172.21.15.152:0/3848391497 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88280ced20 con 0x7f88280b5480 2024-04-20T11:35:24.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.824+0000 7f8827e00640 1 --2- 172.21.15.152:0/3848391497 >> v2:172.21.15.175:3300/0 conn(0x7f88281603d0 0x7f88280b5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:35:24.826 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.825+0000 7f882dc00640 1 --2- 172.21.15.152:0/3848391497 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f88280b5480 0x7f88280b63f0 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f882808e560 tx=0x7f882002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b504181f6f932 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:24.826 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.825+0000 7f8827400640 1 -- 172.21.15.152:0/3848391497 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f882002ca70 con 0x7f88280b5480 2024-04-20T11:35:24.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.825+0000 7f8827400640 1 -- 172.21.15.152:0/3848391497 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f882002cc10 con 0x7f88280b5480 2024-04-20T11:35:24.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.825+0000 7f8827400640 1 -- 172.21.15.152:0/3848391497 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88200343f0 con 0x7f88280b5480 2024-04-20T11:35:24.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.825+0000 7f882e600640 1 -- 172.21.15.152:0/3848391497 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f88280b5480 msgr2=0x7f88280b63f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:24.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.825+0000 7f882e600640 1 --2- 172.21.15.152:0/3848391497 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f88280b5480 0x7f88280b63f0 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f882808e560 tx=0x7f882002bed0 comp rx=0 tx=0).stop 2024-04-20T11:35:24.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.826+0000 7f882e600640 1 -- 172.21.15.152:0/3848391497 shutdown_connections 2024-04-20T11:35:24.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.826+0000 7f882e600640 1 --2- 172.21.15.152:0/3848391497 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f88280b5480 0x7f88280b63f0 unknown :-1 s=CLOSED pgs=1762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:24.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.826+0000 7f882e600640 1 --2- 172.21.15.152:0/3848391497 >> v2:172.21.15.175:3300/0 conn(0x7f88281603d0 0x7f88280b5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:24.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.826+0000 7f882e600640 1 --2- 172.21.15.152:0/3848391497 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f882815fcc0 0x7f88280d4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:24.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.826+0000 7f882e600640 1 -- 172.21.15.152:0/3848391497 >> 172.21.15.152:0/3848391497 conn(0x7f882808bfd0 msgr2=0x7f88280d4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:24.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.826+0000 7f882e600640 1 -- 172.21.15.152:0/3848391497 shutdown_connections 2024-04-20T11:35:24.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.826+0000 7f882e600640 1 -- 172.21.15.152:0/3848391497 wait complete. 2024-04-20T11:35:24.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.827+0000 7f882e600640 1 Processor -- start 2024-04-20T11:35:24.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.828+0000 7f882e600640 1 -- start start 2024-04-20T11:35:24.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.828+0000 7f882e600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f88280b5480 0x7f882815a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:24.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.828+0000 7f882e600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f882815fcc0 0x7f882814a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:24.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.829+0000 7f882e600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f88281603d0 0x7f8828159ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:24.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.829+0000 7f882e600640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f8828162650 con 0x7f88281603d0 2024-04-20T11:35:24.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.829+0000 7f882e600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f88281624d0 con 0x7f882815fcc0 2024-04-20T11:35:24.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.829+0000 7f882e600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f88281627d0 con 0x7f88280b5480 2024-04-20T11:35:24.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.829+0000 7f882dc00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f88281603d0 0x7f8828159ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:24.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.829+0000 7f882dc00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f88281603d0 0x7f8828159ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52392/0 (socket says 172.21.15.152:52392) 2024-04-20T11:35:24.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.829+0000 7f882dc00640 1 -- 172.21.15.152:0/2149394820 learned_addr learned my addr 172.21.15.152:0/2149394820 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:24.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.829+0000 7f882d200640 1 --2- 172.21.15.152:0/2149394820 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f88280b5480 0x7f882815a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:24.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.829+0000 7f882dc00640 1 -- 172.21.15.152:0/2149394820 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f88280b5480 msgr2=0x7f882815a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:24.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.829+0000 7f882dc00640 1 --2- 172.21.15.152:0/2149394820 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f88280b5480 0x7f882815a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:24.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.829+0000 7f882dc00640 1 -- 172.21.15.152:0/2149394820 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f882815fcc0 msgr2=0x7f882814a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:24.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.829+0000 7f882dc00640 1 --2- 172.21.15.152:0/2149394820 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f882815fcc0 0x7f882814a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:24.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.829+0000 7f882dc00640 1 -- 172.21.15.152:0/2149394820 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88280c95e0 con 0x7f88281603d0 2024-04-20T11:35:24.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.830+0000 7f882dc00640 1 --2- 172.21.15.152:0/2149394820 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f88281603d0 0x7f8828159ba0 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f8820036040 tx=0x7f8820004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:24.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.830+0000 7f8824c00640 1 -- 172.21.15.152:0/2149394820 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8820043020 con 0x7f88281603d0 2024-04-20T11:35:24.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.830+0000 7f8824c00640 1 -- 172.21.15.152:0/2149394820 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f882003e450 con 0x7f88281603d0 2024-04-20T11:35:24.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.830+0000 7f8824c00640 1 -- 172.21.15.152:0/2149394820 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8820009510 con 0x7f88281603d0 2024-04-20T11:35:24.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.830+0000 7f882e600640 1 -- 172.21.15.152:0/2149394820 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f882815ada0 con 0x7f88281603d0 2024-04-20T11:35:24.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.830+0000 7f882e600640 1 -- 172.21.15.152:0/2149394820 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88280c9b30 con 0x7f88281603d0 2024-04-20T11:35:24.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.831+0000 7f882e600640 1 -- 172.21.15.152:0/2149394820 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8828075e60 con 0x7f88281603d0 2024-04-20T11:35:24.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.832+0000 7f8824c00640 1 -- 172.21.15.152:0/2149394820 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 21) ==== 91550+0+0 (secure 0 0 0) 0x7f8820046020 con 0x7f88281603d0 2024-04-20T11:35:24.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.833+0000 7f8824c00640 1 --2- 172.21.15.152:0/2149394820 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f880806ce90 0x7f880806f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:24.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.833+0000 7f8824c00640 1 -- 172.21.15.152:0/2149394820 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5709+0+0 (secure 0 0 0) 0x7f88200aeec0 con 0x7f88281603d0 2024-04-20T11:35:24.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.834+0000 7f882d200640 1 --2- 172.21.15.152:0/2149394820 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f880806ce90 0x7f880806f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:24.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.835+0000 7f882d200640 1 --2- 172.21.15.152:0/2149394820 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f880806ce90 0x7f880806f350 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f8818006fd0 tx=0x7f8818008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:24.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:24.838+0000 7f8824c00640 1 -- 172.21.15.152:0/2149394820 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f882004d030 con 0x7f88281603d0 2024-04-20T11:35:25.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.037+0000 7f882e600640 1 -- 172.21.15.152:0/2149394820 --> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f88280ba4d0 con 0x7f880806ce90 2024-04-20T11:35:25.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.055+0000 7f8824c00640 1 -- 172.21.15.152:0/2149394820 <== mgr.4109 v2:172.21.15.175:6836/4021250441 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252488 (secure 0 0 0) 0x7f88280ba4d0 con 0x7f880806ce90 2024-04-20T11:35:25.060 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:35:25.061 INFO:teuthology.orchestra.run.smithi152.stderr:dumped all 2024-04-20T11:35:25.062 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.061+0000 7f882e600640 1 -- 172.21.15.152:0/2149394820 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f880806ce90 msgr2=0x7f880806f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:25.062 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.061+0000 7f882e600640 1 --2- 172.21.15.152:0/2149394820 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f880806ce90 0x7f880806f350 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f8818006fd0 tx=0x7f8818008040 comp rx=0 tx=0).stop 2024-04-20T11:35:25.062 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.061+0000 7f882e600640 1 -- 172.21.15.152:0/2149394820 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f88281603d0 msgr2=0x7f8828159ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:25.063 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.061+0000 7f882e600640 1 --2- 172.21.15.152:0/2149394820 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f88281603d0 0x7f8828159ba0 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f8820036040 tx=0x7f8820004a40 comp rx=0 tx=0).stop 2024-04-20T11:35:25.063 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.061+0000 7f882e600640 1 -- 172.21.15.152:0/2149394820 shutdown_connections 2024-04-20T11:35:25.063 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.061+0000 7f882e600640 1 --2- 172.21.15.152:0/2149394820 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f880806ce90 0x7f880806f350 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:25.063 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.061+0000 7f882e600640 1 --2- 172.21.15.152:0/2149394820 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f88281603d0 0x7f8828159ba0 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:25.063 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.061+0000 7f882e600640 1 --2- 172.21.15.152:0/2149394820 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f882815fcc0 0x7f882814a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:25.063 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.061+0000 7f882e600640 1 --2- 172.21.15.152:0/2149394820 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f88280b5480 0x7f882815a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:25.063 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.061+0000 7f882e600640 1 -- 172.21.15.152:0/2149394820 >> 172.21.15.152:0/2149394820 conn(0x7f882808bfd0 msgr2=0x7f882806e390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:25.063 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.062+0000 7f882e600640 1 -- 172.21.15.152:0/2149394820 shutdown_connections 2024-04-20T11:35:25.063 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.062+0000 7f882e600640 1 -- 172.21.15.152:0/2149394820 wait complete. 2024-04-20T11:35:25.082 INFO:teuthology.orchestra.run.smithi152.stdout:{"pg_ready":true,"pg_map":{"version":154,"stamp":"2024-04-20T11:35:24.662395+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":224,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":167,"ondisk_log_size":167,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":263528,"kb_used_data":14056,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754711192,"statfs":{"total":773094113280,"available":772824260608,"internally_reserved":0,"allocated":14393344,"data_stored":6997700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12746,"internal_metadata":255315510},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002281"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-20T11:35:21.108424+0000","last_change":"2024-04-20T11:35:19.818389+0000","last_active":"2024-04-20T11:35:21.108424+0000","last_peered":"2024-04-20T11:35:21.108424+0000","last_clean":"2024-04-20T11:35:21.108424+0000","last_became_active":"2024-04-20T11:35:19.818115+0000","last_became_peered":"2024-04-20T11:35:19.818115+0000","last_unstale":"2024-04-20T11:35:21.108424+0000","last_undegraded":"2024-04-20T11:35:21.108424+0000","last_fullsized":"2024-04-20T11:35:21.108424+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:03:36.001355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-20T11:35:21.846286+0000","last_change":"2024-04-20T11:35:19.823213+0000","last_active":"2024-04-20T11:35:21.846286+0000","last_peered":"2024-04-20T11:35:21.846286+0000","last_clean":"2024-04-20T11:35:21.846286+0000","last_became_active":"2024-04-20T11:35:19.821569+0000","last_became_peered":"2024-04-20T11:35:19.821569+0000","last_unstale":"2024-04-20T11:35:21.846286+0000","last_undegraded":"2024-04-20T11:35:21.846286+0000","last_fullsized":"2024-04-20T11:35:21.846286+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:24:44.606634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-20T11:35:21.847586+0000","last_change":"2024-04-20T11:35:19.822183+0000","last_active":"2024-04-20T11:35:21.847586+0000","last_peered":"2024-04-20T11:35:21.847586+0000","last_clean":"2024-04-20T11:35:21.847586+0000","last_became_active":"2024-04-20T11:35:19.821389+0000","last_became_peered":"2024-04-20T11:35:19.821389+0000","last_unstale":"2024-04-20T11:35:21.847586+0000","last_undegraded":"2024-04-20T11:35:21.847586+0000","last_fullsized":"2024-04-20T11:35:21.847586+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:54:48.834448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-20T11:35:20.794922+0000","last_change":"2024-04-20T11:35:19.820616+0000","last_active":"2024-04-20T11:35:20.794922+0000","last_peered":"2024-04-20T11:35:20.794922+0000","last_clean":"2024-04-20T11:35:20.794922+0000","last_became_active":"2024-04-20T11:35:19.820344+0000","last_became_peered":"2024-04-20T11:35:19.820344+0000","last_unstale":"2024-04-20T11:35:20.794922+0000","last_undegraded":"2024-04-20T11:35:20.794922+0000","last_fullsized":"2024-04-20T11:35:20.794922+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:19:26.597643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-20T11:35:20.792837+0000","last_change":"2024-04-20T11:35:19.813999+0000","last_active":"2024-04-20T11:35:20.792837+0000","last_peered":"2024-04-20T11:35:20.792837+0000","last_clean":"2024-04-20T11:35:20.792837+0000","last_became_active":"2024-04-20T11:35:19.813356+0000","last_became_peered":"2024-04-20T11:35:19.813356+0000","last_unstale":"2024-04-20T11:35:20.792837+0000","last_undegraded":"2024-04-20T11:35:20.792837+0000","last_fullsized":"2024-04-20T11:35:20.792837+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:06:47.819595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-20T11:35:21.107659+0000","last_change":"2024-04-20T11:35:19.820708+0000","last_active":"2024-04-20T11:35:21.107659+0000","last_peered":"2024-04-20T11:35:21.107659+0000","last_clean":"2024-04-20T11:35:21.107659+0000","last_became_active":"2024-04-20T11:35:19.820405+0000","last_became_peered":"2024-04-20T11:35:19.820405+0000","last_unstale":"2024-04-20T11:35:21.107659+0000","last_undegraded":"2024-04-20T11:35:21.107659+0000","last_fullsized":"2024-04-20T11:35:21.107659+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:45:09.072743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-20T11:35:21.847571+0000","last_change":"2024-04-20T11:35:19.811846+0000","last_active":"2024-04-20T11:35:21.847571+0000","last_peered":"2024-04-20T11:35:21.847571+0000","last_clean":"2024-04-20T11:35:21.847571+0000","last_became_active":"2024-04-20T11:35:19.811373+0000","last_became_peered":"2024-04-20T11:35:19.811373+0000","last_unstale":"2024-04-20T11:35:21.847571+0000","last_undegraded":"2024-04-20T11:35:21.847571+0000","last_fullsized":"2024-04-20T11:35:21.847571+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:02:27.756599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-20T11:35:21.107316+0000","last_change":"2024-04-20T11:35:19.803570+0000","last_active":"2024-04-20T11:35:21.107316+0000","last_peered":"2024-04-20T11:35:21.107316+0000","last_clean":"2024-04-20T11:35:21.107316+0000","last_became_active":"2024-04-20T11:35:19.803048+0000","last_became_peered":"2024-04-20T11:35:19.803048+0000","last_unstale":"2024-04-20T11:35:21.107316+0000","last_undegraded":"2024-04-20T11:35:21.107316+0000","last_fullsized":"2024-04-20T11:35:21.107316+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:43:26.851924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-20T11:35:21.846631+0000","last_change":"2024-04-20T11:35:19.824950+0000","last_active":"2024-04-20T11:35:21.846631+0000","last_peered":"2024-04-20T11:35:21.846631+0000","last_clean":"2024-04-20T11:35:21.846631+0000","last_became_active":"2024-04-20T11:35:19.824652+0000","last_became_peered":"2024-04-20T11:35:19.824652+0000","last_unstale":"2024-04-20T11:35:21.846631+0000","last_undegraded":"2024-04-20T11:35:21.846631+0000","last_fullsized":"2024-04-20T11:35:21.846631+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:59:42.637631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-20T11:35:21.114268+0000","last_change":"2024-04-20T11:35:19.826228+0000","last_active":"2024-04-20T11:35:21.114268+0000","last_peered":"2024-04-20T11:35:21.114268+0000","last_clean":"2024-04-20T11:35:21.114268+0000","last_became_active":"2024-04-20T11:35:19.825771+0000","last_became_peered":"2024-04-20T11:35:19.825771+0000","last_unstale":"2024-04-20T11:35:21.114268+0000","last_undegraded":"2024-04-20T11:35:21.114268+0000","last_fullsized":"2024-04-20T11:35:21.114268+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:52:48.176754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-20T11:35:21.107597+0000","last_change":"2024-04-20T11:35:19.808167+0000","last_active":"2024-04-20T11:35:21.107597+0000","last_peered":"2024-04-20T11:35:21.107597+0000","last_clean":"2024-04-20T11:35:21.107597+0000","last_became_active":"2024-04-20T11:35:19.806792+0000","last_became_peered":"2024-04-20T11:35:19.806792+0000","last_unstale":"2024-04-20T11:35:21.107597+0000","last_undegraded":"2024-04-20T11:35:21.107597+0000","last_fullsized":"2024-04-20T11:35:21.107597+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:28:49.935042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-20T11:35:21.114090+0000","last_change":"2024-04-20T11:35:19.820240+0000","last_active":"2024-04-20T11:35:21.114090+0000","last_peered":"2024-04-20T11:35:21.114090+0000","last_clean":"2024-04-20T11:35:21.114090+0000","last_became_active":"2024-04-20T11:35:19.819917+0000","last_became_peered":"2024-04-20T11:35:19.819917+0000","last_unstale":"2024-04-20T11:35:21.114090+0000","last_undegraded":"2024-04-20T11:35:21.114090+0000","last_fullsized":"2024-04-20T11:35:21.114090+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:42:27.760559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-20T11:35:20.793273+0000","last_change":"2024-04-20T11:35:19.812741+0000","last_active":"2024-04-20T11:35:20.793273+0000","last_peered":"2024-04-20T11:35:20.793273+0000","last_clean":"2024-04-20T11:35:20.793273+0000","last_became_active":"2024-04-20T11:35:19.812462+0000","last_became_peered":"2024-04-20T11:35:19.812462+0000","last_unstale":"2024-04-20T11:35:20.793273+0000","last_undegraded":"2024-04-20T11:35:20.793273+0000","last_fullsized":"2024-04-20T11:35:20.793273+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:58:59.512474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-20T11:35:20.793278+0000","last_change":"2024-04-20T11:35:19.813662+0000","last_active":"2024-04-20T11:35:20.793278+0000","last_peered":"2024-04-20T11:35:20.793278+0000","last_clean":"2024-04-20T11:35:20.793278+0000","last_became_active":"2024-04-20T11:35:19.813387+0000","last_became_peered":"2024-04-20T11:35:19.813387+0000","last_unstale":"2024-04-20T11:35:20.793278+0000","last_undegraded":"2024-04-20T11:35:20.793278+0000","last_fullsized":"2024-04-20T11:35:20.793278+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:13:36.704521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-20T11:35:21.113405+0000","last_change":"2024-04-20T11:35:19.821315+0000","last_active":"2024-04-20T11:35:21.113405+0000","last_peered":"2024-04-20T11:35:21.113405+0000","last_clean":"2024-04-20T11:35:21.113405+0000","last_became_active":"2024-04-20T11:35:19.819593+0000","last_became_peered":"2024-04-20T11:35:19.819593+0000","last_unstale":"2024-04-20T11:35:21.113405+0000","last_undegraded":"2024-04-20T11:35:21.113405+0000","last_fullsized":"2024-04-20T11:35:21.113405+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:53:01.857261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-20T11:35:20.787722+0000","last_change":"2024-04-20T11:35:19.809286+0000","last_active":"2024-04-20T11:35:20.787722+0000","last_peered":"2024-04-20T11:35:20.787722+0000","last_clean":"2024-04-20T11:35:20.787722+0000","last_became_active":"2024-04-20T11:35:19.808926+0000","last_became_peered":"2024-04-20T11:35:19.808926+0000","last_unstale":"2024-04-20T11:35:20.787722+0000","last_undegraded":"2024-04-20T11:35:20.787722+0000","last_fullsized":"2024-04-20T11:35:20.787722+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:47:59.036285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-20T11:35:21.114699+0000","last_change":"2024-04-20T11:35:19.822085+0000","last_active":"2024-04-20T11:35:21.114699+0000","last_peered":"2024-04-20T11:35:21.114699+0000","last_clean":"2024-04-20T11:35:21.114699+0000","last_became_active":"2024-04-20T11:35:19.821815+0000","last_became_peered":"2024-04-20T11:35:19.821815+0000","last_unstale":"2024-04-20T11:35:21.114699+0000","last_undegraded":"2024-04-20T11:35:21.114699+0000","last_fullsized":"2024-04-20T11:35:21.114699+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:51:44.303955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-20T11:35:21.847421+0000","last_change":"2024-04-20T11:35:19.827852+0000","last_active":"2024-04-20T11:35:21.847421+0000","last_peered":"2024-04-20T11:35:21.847421+0000","last_clean":"2024-04-20T11:35:21.847421+0000","last_became_active":"2024-04-20T11:35:19.827238+0000","last_became_peered":"2024-04-20T11:35:19.827238+0000","last_unstale":"2024-04-20T11:35:21.847421+0000","last_undegraded":"2024-04-20T11:35:21.847421+0000","last_fullsized":"2024-04-20T11:35:21.847421+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:35:05.070222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-20T11:35:21.107813+0000","last_change":"2024-04-20T11:35:19.814108+0000","last_active":"2024-04-20T11:35:21.107813+0000","last_peered":"2024-04-20T11:35:21.107813+0000","last_clean":"2024-04-20T11:35:21.107813+0000","last_became_active":"2024-04-20T11:35:19.810747+0000","last_became_peered":"2024-04-20T11:35:19.810747+0000","last_unstale":"2024-04-20T11:35:21.107813+0000","last_undegraded":"2024-04-20T11:35:21.107813+0000","last_fullsized":"2024-04-20T11:35:21.107813+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:10:43.413917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-20T11:35:20.789048+0000","last_change":"2024-04-20T11:35:19.820892+0000","last_active":"2024-04-20T11:35:20.789048+0000","last_peered":"2024-04-20T11:35:20.789048+0000","last_clean":"2024-04-20T11:35:20.789048+0000","last_became_active":"2024-04-20T11:35:19.820528+0000","last_became_peered":"2024-04-20T11:35:19.820528+0000","last_unstale":"2024-04-20T11:35:20.789048+0000","last_undegraded":"2024-04-20T11:35:20.789048+0000","last_fullsized":"2024-04-20T11:35:20.789048+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:01:21.940443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-20T11:35:21.113994+0000","last_change":"2024-04-20T11:35:19.826278+0000","last_active":"2024-04-20T11:35:21.113994+0000","last_peered":"2024-04-20T11:35:21.113994+0000","last_clean":"2024-04-20T11:35:21.113994+0000","last_became_active":"2024-04-20T11:35:19.825977+0000","last_became_peered":"2024-04-20T11:35:19.825977+0000","last_unstale":"2024-04-20T11:35:21.113994+0000","last_undegraded":"2024-04-20T11:35:21.113994+0000","last_fullsized":"2024-04-20T11:35:21.113994+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:18:37.671058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-20T11:35:20.793696+0000","last_change":"2024-04-20T11:35:19.823585+0000","last_active":"2024-04-20T11:35:20.793696+0000","last_peered":"2024-04-20T11:35:20.793696+0000","last_clean":"2024-04-20T11:35:20.793696+0000","last_became_active":"2024-04-20T11:35:19.823022+0000","last_became_peered":"2024-04-20T11:35:19.823022+0000","last_unstale":"2024-04-20T11:35:20.793696+0000","last_undegraded":"2024-04-20T11:35:20.793696+0000","last_fullsized":"2024-04-20T11:35:20.793696+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:54:41.358394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-20T11:35:20.792534+0000","last_change":"2024-04-20T11:35:19.806908+0000","last_active":"2024-04-20T11:35:20.792534+0000","last_peered":"2024-04-20T11:35:20.792534+0000","last_clean":"2024-04-20T11:35:20.792534+0000","last_became_active":"2024-04-20T11:35:19.806535+0000","last_became_peered":"2024-04-20T11:35:19.806535+0000","last_unstale":"2024-04-20T11:35:20.792534+0000","last_undegraded":"2024-04-20T11:35:20.792534+0000","last_fullsized":"2024-04-20T11:35:20.792534+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:00:55.047883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-20T11:35:21.113241+0000","last_change":"2024-04-20T11:35:19.825891+0000","last_active":"2024-04-20T11:35:21.113241+0000","last_peered":"2024-04-20T11:35:21.113241+0000","last_clean":"2024-04-20T11:35:21.113241+0000","last_became_active":"2024-04-20T11:35:19.825585+0000","last_became_peered":"2024-04-20T11:35:19.825585+0000","last_unstale":"2024-04-20T11:35:21.113241+0000","last_undegraded":"2024-04-20T11:35:21.113241+0000","last_fullsized":"2024-04-20T11:35:21.113241+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:36:10.595750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-20T11:35:20.788191+0000","last_change":"2024-04-20T11:35:19.813130+0000","last_active":"2024-04-20T11:35:20.788191+0000","last_peered":"2024-04-20T11:35:20.788191+0000","last_clean":"2024-04-20T11:35:20.788191+0000","last_became_active":"2024-04-20T11:35:19.812870+0000","last_became_peered":"2024-04-20T11:35:19.812870+0000","last_unstale":"2024-04-20T11:35:20.788191+0000","last_undegraded":"2024-04-20T11:35:20.788191+0000","last_fullsized":"2024-04-20T11:35:20.788191+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:58:21.643203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-20T11:35:21.108055+0000","last_change":"2024-04-20T11:35:19.812381+0000","last_active":"2024-04-20T11:35:21.108055+0000","last_peered":"2024-04-20T11:35:21.108055+0000","last_clean":"2024-04-20T11:35:21.108055+0000","last_became_active":"2024-04-20T11:35:19.812033+0000","last_became_peered":"2024-04-20T11:35:19.812033+0000","last_unstale":"2024-04-20T11:35:21.108055+0000","last_undegraded":"2024-04-20T11:35:21.108055+0000","last_fullsized":"2024-04-20T11:35:21.108055+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:25:09.990413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-20T11:35:20.788428+0000","last_change":"2024-04-20T11:35:19.810432+0000","last_active":"2024-04-20T11:35:20.788428+0000","last_peered":"2024-04-20T11:35:20.788428+0000","last_clean":"2024-04-20T11:35:20.788428+0000","last_became_active":"2024-04-20T11:35:19.810117+0000","last_became_peered":"2024-04-20T11:35:19.810117+0000","last_unstale":"2024-04-20T11:35:20.788428+0000","last_undegraded":"2024-04-20T11:35:20.788428+0000","last_fullsized":"2024-04-20T11:35:20.788428+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:05:23.468423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-20T11:35:21.847446+0000","last_change":"2024-04-20T11:35:19.826822+0000","last_active":"2024-04-20T11:35:21.847446+0000","last_peered":"2024-04-20T11:35:21.847446+0000","last_clean":"2024-04-20T11:35:21.847446+0000","last_became_active":"2024-04-20T11:35:19.826468+0000","last_became_peered":"2024-04-20T11:35:19.826468+0000","last_unstale":"2024-04-20T11:35:21.847446+0000","last_undegraded":"2024-04-20T11:35:21.847446+0000","last_fullsized":"2024-04-20T11:35:21.847446+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:51:44.303955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-20T11:35:21.107495+0000","last_change":"2024-04-20T11:35:19.816866+0000","last_active":"2024-04-20T11:35:21.107495+0000","last_peered":"2024-04-20T11:35:21.107495+0000","last_clean":"2024-04-20T11:35:21.107495+0000","last_became_active":"2024-04-20T11:35:19.816544+0000","last_became_peered":"2024-04-20T11:35:19.816544+0000","last_unstale":"2024-04-20T11:35:21.107495+0000","last_undegraded":"2024-04-20T11:35:21.107495+0000","last_fullsized":"2024-04-20T11:35:21.107495+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:42:24.987162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-20T11:35:20.794068+0000","last_change":"2024-04-20T11:35:19.818099+0000","last_active":"2024-04-20T11:35:20.794068+0000","last_peered":"2024-04-20T11:35:20.794068+0000","last_clean":"2024-04-20T11:35:20.794068+0000","last_became_active":"2024-04-20T11:35:19.817686+0000","last_became_peered":"2024-04-20T11:35:19.817686+0000","last_unstale":"2024-04-20T11:35:20.794068+0000","last_undegraded":"2024-04-20T11:35:20.794068+0000","last_fullsized":"2024-04-20T11:35:20.794068+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:11:09.848684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-20T11:35:21.845962+0000","last_change":"2024-04-20T11:35:19.829599+0000","last_active":"2024-04-20T11:35:21.845962+0000","last_peered":"2024-04-20T11:35:21.845962+0000","last_clean":"2024-04-20T11:35:21.845962+0000","last_became_active":"2024-04-20T11:35:19.829293+0000","last_became_peered":"2024-04-20T11:35:19.829293+0000","last_unstale":"2024-04-20T11:35:21.845962+0000","last_undegraded":"2024-04-20T11:35:21.845962+0000","last_fullsized":"2024-04-20T11:35:21.845962+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:51:51.392165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-20T11:35:20.788945+0000","last_change":"2024-04-20T11:35:19.811818+0000","last_active":"2024-04-20T11:35:20.788945+0000","last_peered":"2024-04-20T11:35:20.788945+0000","last_clean":"2024-04-20T11:35:20.788945+0000","last_became_active":"2024-04-20T11:35:19.811508+0000","last_became_peered":"2024-04-20T11:35:19.811508+0000","last_unstale":"2024-04-20T11:35:20.788945+0000","last_undegraded":"2024-04-20T11:35:20.788945+0000","last_fullsized":"2024-04-20T11:35:20.788945+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:27:54.586372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-20T11:35:21.114416+0000","last_change":"2024-04-20T11:35:19.821315+0000","last_active":"2024-04-20T11:35:21.114416+0000","last_peered":"2024-04-20T11:35:21.114416+0000","last_clean":"2024-04-20T11:35:21.114416+0000","last_became_active":"2024-04-20T11:35:19.820947+0000","last_became_peered":"2024-04-20T11:35:19.820947+0000","last_unstale":"2024-04-20T11:35:21.114416+0000","last_undegraded":"2024-04-20T11:35:21.114416+0000","last_fullsized":"2024-04-20T11:35:21.114416+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:38:07.754757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1f","version":"130'4","reported_seq":18,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-20T11:35:21.114284+0000","last_change":"2024-04-20T11:35:18.802413+0000","last_active":"2024-04-20T11:35:21.114284+0000","last_peered":"2024-04-20T11:35:21.114284+0000","last_clean":"2024-04-20T11:35:21.114284+0000","last_became_active":"2024-04-20T11:35:18.801686+0000","last_became_peered":"2024-04-20T11:35:18.801686+0000","last_unstale":"2024-04-20T11:35:21.114284+0000","last_undegraded":"2024-04-20T11:35:21.114284+0000","last_fullsized":"2024-04-20T11:35:21.114284+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_clean_scrub_stamp":"2024-04-20T11:35:17.736274+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:13:35.690067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-20T11:35:21.847019+0000","last_change":"2024-04-20T11:35:19.822905+0000","last_active":"2024-04-20T11:35:21.847019+0000","last_peered":"2024-04-20T11:35:21.847019+0000","last_clean":"2024-04-20T11:35:21.847019+0000","last_became_active":"2024-04-20T11:35:19.821861+0000","last_became_peered":"2024-04-20T11:35:19.821861+0000","last_unstale":"2024-04-20T11:35:21.847019+0000","last_undegraded":"2024-04-20T11:35:21.847019+0000","last_fullsized":"2024-04-20T11:35:21.847019+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:18:37.671058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"130'1","reported_seq":15,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-20T11:35:21.846100+0000","last_change":"2024-04-20T11:35:18.798504+0000","last_active":"2024-04-20T11:35:21.846100+0000","last_peered":"2024-04-20T11:35:21.846100+0000","last_clean":"2024-04-20T11:35:21.846100+0000","last_became_active":"2024-04-20T11:35:18.797829+0000","last_became_peered":"2024-04-20T11:35:18.797829+0000","last_unstale":"2024-04-20T11:35:21.846100+0000","last_undegraded":"2024-04-20T11:35:21.846100+0000","last_fullsized":"2024-04-20T11:35:21.846100+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_clean_scrub_stamp":"2024-04-20T11:35:17.736274+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:56:02.363714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-20T11:35:21.108221+0000","last_change":"2024-04-20T11:35:19.804439+0000","last_active":"2024-04-20T11:35:21.108221+0000","last_peered":"2024-04-20T11:35:21.108221+0000","last_clean":"2024-04-20T11:35:21.108221+0000","last_became_active":"2024-04-20T11:35:19.804106+0000","last_became_peered":"2024-04-20T11:35:19.804106+0000","last_unstale":"2024-04-20T11:35:21.108221+0000","last_undegraded":"2024-04-20T11:35:21.108221+0000","last_fullsized":"2024-04-20T11:35:21.108221+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:52:00.683982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1d","version":"130'2","reported_seq":16,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-20T11:35:20.794515+0000","last_change":"2024-04-20T11:35:18.790200+0000","last_active":"2024-04-20T11:35:20.794515+0000","last_peered":"2024-04-20T11:35:20.794515+0000","last_clean":"2024-04-20T11:35:20.794515+0000","last_became_active":"2024-04-20T11:35:18.789904+0000","last_became_peered":"2024-04-20T11:35:18.789904+0000","last_unstale":"2024-04-20T11:35:20.794515+0000","last_undegraded":"2024-04-20T11:35:20.794515+0000","last_fullsized":"2024-04-20T11:35:20.794515+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_clean_scrub_stamp":"2024-04-20T11:35:17.736274+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:35:04.055768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-20T11:35:21.114767+0000","last_change":"2024-04-20T11:35:19.821001+0000","last_active":"2024-04-20T11:35:21.114767+0000","last_peered":"2024-04-20T11:35:21.114767+0000","last_clean":"2024-04-20T11:35:21.114767+0000","last_became_active":"2024-04-20T11:35:19.820639+0000","last_became_peered":"2024-04-20T11:35:19.820639+0000","last_unstale":"2024-04-20T11:35:21.114767+0000","last_undegraded":"2024-04-20T11:35:21.114767+0000","last_fullsized":"2024-04-20T11:35:21.114767+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:55:38.723982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"130'2","reported_seq":16,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-20T11:35:20.794794+0000","last_change":"2024-04-20T11:35:18.788583+0000","last_active":"2024-04-20T11:35:20.794794+0000","last_peered":"2024-04-20T11:35:20.794794+0000","last_clean":"2024-04-20T11:35:20.794794+0000","last_became_active":"2024-04-20T11:35:18.787817+0000","last_became_peered":"2024-04-20T11:35:18.787817+0000","last_unstale":"2024-04-20T11:35:20.794794+0000","last_undegraded":"2024-04-20T11:35:20.794794+0000","last_fullsized":"2024-04-20T11:35:20.794794+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_clean_scrub_stamp":"2024-04-20T11:35:17.736274+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:18:02.859222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-20T11:35:21.846070+0000","last_change":"2024-04-20T11:35:19.829431+0000","last_active":"2024-04-20T11:35:21.846070+0000","last_peered":"2024-04-20T11:35:21.846070+0000","last_clean":"2024-04-20T11:35:21.846070+0000","last_became_active":"2024-04-20T11:35:19.829176+0000","last_became_peered":"2024-04-20T11:35:19.829176+0000","last_unstale":"2024-04-20T11:35:21.846070+0000","last_undegraded":"2024-04-20T11:35:21.846070+0000","last_fullsized":"2024-04-20T11:35:21.846070+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:46:19.356066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1b","version":"130'1","reported_seq":15,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-20T11:35:20.788242+0000","last_change":"2024-04-20T11:35:18.800817+0000","last_active":"2024-04-20T11:35:20.788242+0000","last_peered":"2024-04-20T11:35:20.788242+0000","last_clean":"2024-04-20T11:35:20.788242+0000","last_became_active":"2024-04-20T11:35:18.800393+0000","last_became_peered":"2024-04-20T11:35:18.800393+0000","last_unstale":"2024-04-20T11:35:20.788242+0000","last_undegraded":"2024-04-20T11:35:20.788242+0000","last_fullsized":"2024-04-20T11:35:20.788242+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_clean_scrub_stamp":"2024-04-20T11:35:17.736274+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:10:04.195314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-20T11:35:20.793738+0000","last_change":"2024-04-20T11:35:19.822314+0000","last_active":"2024-04-20T11:35:20.793738+0000","last_peered":"2024-04-20T11:35:20.793738+0000","last_clean":"2024-04-20T11:35:20.793738+0000","last_became_active":"2024-04-20T11:35:19.822035+0000","last_became_peered":"2024-04-20T11:35:19.822035+0000","last_unstale":"2024-04-20T11:35:20.793738+0000","last_undegraded":"2024-04-20T11:35:20.793738+0000","last_fullsized":"2024-04-20T11:35:20.793738+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:35:20.892407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1a","version":"130'1","reported_seq":15,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-20T11:35:20.792119+0000","last_change":"2024-04-20T11:35:18.798437+0000","last_active":"2024-04-20T11:35:20.792119+0000","last_peered":"2024-04-20T11:35:20.792119+0000","last_clean":"2024-04-20T11:35:20.792119+0000","last_became_active":"2024-04-20T11:35:18.798150+0000","last_became_peered":"2024-04-20T11:35:18.798150+0000","last_unstale":"2024-04-20T11:35:20.792119+0000","last_undegraded":"2024-04-20T11:35:20.792119+0000","last_fullsized":"2024-04-20T11:35:20.792119+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_clean_scrub_stamp":"2024-04-20T11:35:17.736274+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:44:57.596966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-20T11:35:21.846108+0000","last_change":"2024-04-20T11:35:19.823614+0000","last_active":"2024-04-20T11:35:21.846108+0000","last_peered":"2024-04-20T11:35:21.846108+0000","last_clean":"2024-04-20T11:35:21.846108+0000","last_became_active":"2024-04-20T11:35:19.823295+0000","last_became_peered":"2024-04-20T11:35:19.823295+0000","last_unstale":"2024-04-20T11:35:21.846108+0000","last_undegraded":"2024-04-20T11:35:21.846108+0000","last_fullsized":"2024-04-20T11:35:21.846108+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:43:00.704377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":14,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-20T11:35:20.787963+0000","last_change":"2024-04-20T11:35:18.791013+0000","last_active":"2024-04-20T11:35:20.787963+0000","last_peered":"2024-04-20T11:35:20.787963+0000","last_clean":"2024-04-20T11:35:20.787963+0000","last_became_active":"2024-04-20T11:35:18.790498+0000","last_became_peered":"2024-04-20T11:35:18.790498+0000","last_unstale":"2024-04-20T11:35:20.787963+0000","last_undegraded":"2024-04-20T11:35:20.787963+0000","last_fullsized":"2024-04-20T11:35:20.787963+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_clean_scrub_stamp":"2024-04-20T11:35:17.736274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:23:37.793733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-20T11:35:20.790949+0000","last_change":"2024-04-20T11:35:19.802347+0000","last_active":"2024-04-20T11:35:20.790949+0000","last_peered":"2024-04-20T11:35:20.790949+0000","last_clean":"2024-04-20T11:35:20.790949+0000","last_became_active":"2024-04-20T11:35:19.801498+0000","last_became_peered":"2024-04-20T11:35:19.801498+0000","last_unstale":"2024-04-20T11:35:20.790949+0000","last_undegraded":"2024-04-20T11:35:20.790949+0000","last_fullsized":"2024-04-20T11:35:20.790949+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:16:36.020774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":14,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-20T11:35:20.791444+0000","last_change":"2024-04-20T11:35:18.797817+0000","last_active":"2024-04-20T11:35:20.791444+0000","last_peered":"2024-04-20T11:35:20.791444+0000","last_clean":"2024-04-20T11:35:20.791444+0000","last_became_active":"2024-04-20T11:35:18.797489+0000","last_became_peered":"2024-04-20T11:35:18.797489+0000","last_unstale":"2024-04-20T11:35:20.791444+0000","last_undegraded":"2024-04-20T11:35:20.791444+0000","last_fullsized":"2024-04-20T11:35:20.791444+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_clean_scrub_stamp":"2024-04-20T11:35:17.736274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:17:44.811582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-20T11:35:20.792239+0000","last_change":"2024-04-20T11:35:19.816022+0000","last_active":"2024-04-20T11:35:20.792239+0000","last_peered":"2024-04-20T11:35:20.792239+0000","last_clean":"2024-04-20T11:35:20.792239+0000","last_became_active":"2024-04-20T11:35:19.815663+0000","last_became_peered":"2024-04-20T11:35:19.815663+0000","last_unstale":"2024-04-20T11:35:20.792239+0000","last_undegraded":"2024-04-20T11:35:20.792239+0000","last_fullsized":"2024-04-20T11:35:20.792239+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:27:54.586372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":14,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-20T11:35:21.847147+0000","last_change":"2024-04-20T11:35:18.787522+0000","last_active":"2024-04-20T11:35:21.847147+0000","last_peered":"2024-04-20T11:35:21.847147+0000","last_clean":"2024-04-20T11:35:21.847147+0000","last_became_active":"2024-04-20T11:35:18.787197+0000","last_became_peered":"2024-04-20T11:35:18.787197+0000","last_unstale":"2024-04-20T11:35:21.847147+0000","last_undegraded":"2024-04-20T11:35:21.847147+0000","last_fullsized":"2024-04-20T11:35:21.847147+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_clean_scrub_stamp":"2024-04-20T11:35:17.736274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:13:35.690067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-20T11:35:21.114576+0000","last_change":"2024-04-20T11:35:19.822388+0000","last_active":"2024-04-20T11:35:21.114576+0000","last_peered":"2024-04-20T11:35:21.114576+0000","last_clean":"2024-04-20T11:35:21.114576+0000","last_became_active":"2024-04-20T11:35:19.822084+0000","last_became_peered":"2024-04-20T11:35:19.822084+0000","last_unstale":"2024-04-20T11:35:21.114576+0000","last_undegraded":"2024-04-20T11:35:21.114576+0000","last_fullsized":"2024-04-20T11:35:21.114576+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:58:29.466877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":14,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-20T11:35:21.114616+0000","last_change":"2024-04-20T11:35:18.803847+0000","last_active":"2024-04-20T11:35:21.114616+0000","last_peered":"2024-04-20T11:35:21.114616+0000","last_clean":"2024-04-20T11:35:21.114616+0000","last_became_active":"2024-04-20T11:35:18.803563+0000","last_became_peered":"2024-04-20T11:35:18.803563+0000","last_unstale":"2024-04-20T11:35:21.114616+0000","last_undegraded":"2024-04-20T11:35:21.114616+0000","last_fullsized":"2024-04-20T11:35:21.114616+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_clean_scrub_stamp":"2024-04-20T11:35:17.736274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:35:19.877953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-20T11:35:20.788631+0000","last_change":"2024-04-20T11:35:19.808079+0000","last_active":"2024-04-20T11:35:20.788631+0000","last_peered":"2024-04-20T11:35:20.788631+0000","last_clean":"2024-04-20T11:35:20.788631+0000","last_became_active":"2024-04-20T11:35:19.807763+0000","last_became_peered":"2024-04-20T11:35:19.807763+0000","last_unstale":"2024-04-20T11:35:20.788631+0000","last_undegraded":"2024-04-20T11:35:20.788631+0000","last_fullsized":"2024-04-20T11:35:20.788631+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:30:42.786906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.14","version":"130'6","reported_seq":20,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-20T11:35:21.847632+0000","last_change":"2024-04-20T11:35:18.787417+0000","last_active":"2024-04-20T11:35:21.847632+0000","last_peered":"2024-04-20T11:35:21.847632+0000","last_clean":"2024-04-20T11:35:21.847632+0000","last_became_active":"2024-04-20T11:35:18.786463+0000","last_became_peered":"2024-04-20T11:35:18.786463+0000","last_unstale":"2024-04-20T11:35:21.847632+0000","last_undegraded":"2024-04-20T11:35:21.847632+0000","last_fullsized":"2024-04-20T11:35:21.847632+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_clean_scrub_stamp":"2024-04-20T11:35:17.736274+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:36:09.581296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-20T11:35:21.846780+0000","last_change":"2024-04-20T11:35:19.824595+0000","last_active":"2024-04-20T11:35:21.846780+0000","last_peered":"2024-04-20T11:35:21.846780+0000","last_clean":"2024-04-20T11:35:21.846780+0000","last_became_active":"2024-04-20T11:35:19.824123+0000","last_became_peered":"2024-04-20T11:35:19.824123+0000","last_unstale":"2024-04-20T11:35:21.846780+0000","last_undegraded":"2024-04-20T11:35:21.846780+0000","last_fullsized":"2024-04-20T11:35:21.846780+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:03:36.001355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.13","version":"130'2","reported_seq":16,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-20T11:35:21.847008+0000","last_change":"2024-04-20T11:35:18.791174+0000","last_active":"2024-04-20T11:35:21.847008+0000","last_peered":"2024-04-20T11:35:21.847008+0000","last_clean":"2024-04-20T11:35:21.847008+0000","last_became_active":"2024-04-20T11:35:18.790927+0000","last_became_peered":"2024-04-20T11:35:18.790927+0000","last_unstale":"2024-04-20T11:35:21.847008+0000","last_undegraded":"2024-04-20T11:35:21.847008+0000","last_fullsized":"2024-04-20T11:35:21.847008+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_clean_scrub_stamp":"2024-04-20T11:35:17.736274+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:38:12.744465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-20T11:35:21.846662+0000","last_change":"2024-04-20T11:35:19.823377+0000","last_active":"2024-04-20T11:35:21.846662+0000","last_peered":"2024-04-20T11:35:21.846662+0000","last_clean":"2024-04-20T11:35:21.846662+0000","last_became_active":"2024-04-20T11:35:19.822161+0000","last_became_peered":"2024-04-20T11:35:19.822161+0000","last_unstale":"2024-04-20T11:35:21.846662+0000","last_undegraded":"2024-04-20T11:35:21.846662+0000","last_fullsized":"2024-04-20T11:35:21.846662+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:53:01.857261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":14,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-20T11:35:21.846716+0000","last_change":"2024-04-20T11:35:18.795236+0000","last_active":"2024-04-20T11:35:21.846716+0000","last_peered":"2024-04-20T11:35:21.846716+0000","last_clean":"2024-04-20T11:35:21.846716+0000","last_became_active":"2024-04-20T11:35:18.794911+0000","last_became_peered":"2024-04-20T11:35:18.794911+0000","last_unstale":"2024-04-20T11:35:21.846716+0000","last_undegraded":"2024-04-20T11:35:21.846716+0000","last_fullsized":"2024-04-20T11:35:21.846716+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_clean_scrub_stamp":"2024-04-20T11:35:17.736274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:19:23.441510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-20T11:35:20.791022+0000","last_change":"2024-04-20T11:35:19.812130+0000","last_active":"2024-04-20T11:35:20.791022+0000","last_peered":"2024-04-20T11:35:20.791022+0000","last_clean":"2024-04-20T11:35:20.791022+0000","last_became_active":"2024-04-20T11:35:19.811815+0000","last_became_peered":"2024-04-20T11:35:19.811815+0000","last_unstale":"2024-04-20T11:35:20.791022+0000","last_undegraded":"2024-04-20T11:35:20.791022+0000","last_fullsized":"2024-04-20T11:35:20.791022+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:30:42.786906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":14,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-20T11:35:21.107356+0000","last_change":"2024-04-20T11:35:18.790431+0000","last_active":"2024-04-20T11:35:21.107356+0000","last_peered":"2024-04-20T11:35:21.107356+0000","last_clean":"2024-04-20T11:35:21.107356+0000","last_became_active":"2024-04-20T11:35:18.790143+0000","last_became_peered":"2024-04-20T11:35:18.790143+0000","last_unstale":"2024-04-20T11:35:21.107356+0000","last_undegraded":"2024-04-20T11:35:21.107356+0000","last_fullsized":"2024-04-20T11:35:21.107356+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_clean_scrub_stamp":"2024-04-20T11:35:17.736274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:35:36.930309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-20T11:35:21.846569+0000","last_change":"2024-04-20T11:35:19.819754+0000","last_active":"2024-04-20T11:35:21.846569+0000","last_peered":"2024-04-20T11:35:21.846569+0000","last_clean":"2024-04-20T11:35:21.846569+0000","last_became_active":"2024-04-20T11:35:19.819454+0000","last_became_peered":"2024-04-20T11:35:19.819454+0000","last_unstale":"2024-04-20T11:35:21.846569+0000","last_undegraded":"2024-04-20T11:35:21.846569+0000","last_fullsized":"2024-04-20T11:35:21.846569+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:52:00.683982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"130'8","reported_seq":22,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-20T11:35:20.787894+0000","last_change":"2024-04-20T11:35:18.795278+0000","last_active":"2024-04-20T11:35:20.787894+0000","last_peered":"2024-04-20T11:35:20.787894+0000","last_clean":"2024-04-20T11:35:20.787894+0000","last_became_active":"2024-04-20T11:35:18.794711+0000","last_became_peered":"2024-04-20T11:35:18.794711+0000","last_unstale":"2024-04-20T11:35:20.787894+0000","last_undegraded":"2024-04-20T11:35:20.787894+0000","last_fullsized":"2024-04-20T11:35:20.787894+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_clean_scrub_stamp":"2024-04-20T11:35:17.736274+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:37:39.706989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-20T11:35:21.114501+0000","last_change":"2024-04-20T11:35:19.821526+0000","last_active":"2024-04-20T11:35:21.114501+0000","last_peered":"2024-04-20T11:35:21.114501+0000","last_clean":"2024-04-20T11:35:21.114501+0000","last_became_active":"2024-04-20T11:35:19.821187+0000","last_became_peered":"2024-04-20T11:35:19.821187+0000","last_unstale":"2024-04-20T11:35:21.114501+0000","last_undegraded":"2024-04-20T11:35:21.114501+0000","last_fullsized":"2024-04-20T11:35:21.114501+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:48:53.645253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"130'1","reported_seq":15,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-20T11:35:20.791616+0000","last_change":"2024-04-20T11:35:18.800211+0000","last_active":"2024-04-20T11:35:20.791616+0000","last_peered":"2024-04-20T11:35:20.791616+0000","last_clean":"2024-04-20T11:35:20.791616+0000","last_became_active":"2024-04-20T11:35:18.798350+0000","last_became_peered":"2024-04-20T11:35:18.798350+0000","last_unstale":"2024-04-20T11:35:20.791616+0000","last_undegraded":"2024-04-20T11:35:20.791616+0000","last_fullsized":"2024-04-20T11:35:20.791616+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_clean_scrub_stamp":"2024-04-20T11:35:17.736274+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:40:39.600302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-20T11:35:20.788559+0000","last_change":"2024-04-20T11:35:19.816006+0000","last_active":"2024-04-20T11:35:20.788559+0000","last_peered":"2024-04-20T11:35:20.788559+0000","last_clean":"2024-04-20T11:35:20.788559+0000","last_became_active":"2024-04-20T11:35:19.814355+0000","last_became_peered":"2024-04-20T11:35:19.814355+0000","last_unstale":"2024-04-20T11:35:20.788559+0000","last_undegraded":"2024-04-20T11:35:20.788559+0000","last_fullsized":"2024-04-20T11:35:20.788559+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:16:36.020774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"130'2","reported_seq":16,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-20T11:35:21.846398+0000","last_change":"2024-04-20T11:35:18.789467+0000","last_active":"2024-04-20T11:35:21.846398+0000","last_peered":"2024-04-20T11:35:21.846398+0000","last_clean":"2024-04-20T11:35:21.846398+0000","last_became_active":"2024-04-20T11:35:18.789121+0000","last_became_peered":"2024-04-20T11:35:18.789121+0000","last_unstale":"2024-04-20T11:35:21.846398+0000","last_undegraded":"2024-04-20T11:35:21.846398+0000","last_fullsized":"2024-04-20T11:35:21.846398+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_clean_scrub_stamp":"2024-04-20T11:35:17.736274+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:00:54.033429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-20T11:35:21.847300+0000","last_change":"2024-04-20T11:35:19.819838+0000","last_active":"2024-04-20T11:35:21.847300+0000","last_peered":"2024-04-20T11:35:21.847300+0000","last_clean":"2024-04-20T11:35:21.847300+0000","last_became_active":"2024-04-20T11:35:19.819554+0000","last_became_peered":"2024-04-20T11:35:19.819554+0000","last_unstale":"2024-04-20T11:35:21.847300+0000","last_undegraded":"2024-04-20T11:35:21.847300+0000","last_fullsized":"2024-04-20T11:35:21.847300+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:17:45.826036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.d","version":"130'1","reported_seq":15,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-20T11:35:20.788491+0000","last_change":"2024-04-20T11:35:18.797019+0000","last_active":"2024-04-20T11:35:20.788491+0000","last_peered":"2024-04-20T11:35:20.788491+0000","last_clean":"2024-04-20T11:35:20.788491+0000","last_became_active":"2024-04-20T11:35:18.796745+0000","last_became_peered":"2024-04-20T11:35:18.796745+0000","last_unstale":"2024-04-20T11:35:20.788491+0000","last_undegraded":"2024-04-20T11:35:20.788491+0000","last_fullsized":"2024-04-20T11:35:20.788491+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_clean_scrub_stamp":"2024-04-20T11:35:17.736274+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:29:06.986258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-20T11:35:21.107709+0000","last_change":"2024-04-20T11:35:19.811672+0000","last_active":"2024-04-20T11:35:21.107709+0000","last_peered":"2024-04-20T11:35:21.107709+0000","last_clean":"2024-04-20T11:35:21.107709+0000","last_became_active":"2024-04-20T11:35:19.810903+0000","last_became_peered":"2024-04-20T11:35:19.810903+0000","last_unstale":"2024-04-20T11:35:21.107709+0000","last_undegraded":"2024-04-20T11:35:21.107709+0000","last_fullsized":"2024-04-20T11:35:21.107709+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:55:05.272718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":14,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-20T11:35:21.846402+0000","last_change":"2024-04-20T11:35:18.787420+0000","last_active":"2024-04-20T11:35:21.846402+0000","last_peered":"2024-04-20T11:35:21.846402+0000","last_clean":"2024-04-20T11:35:21.846402+0000","last_became_active":"2024-04-20T11:35:18.787098+0000","last_became_peered":"2024-04-20T11:35:18.787098+0000","last_unstale":"2024-04-20T11:35:21.846402+0000","last_undegraded":"2024-04-20T11:35:21.846402+0000","last_fullsized":"2024-04-20T11:35:21.846402+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_clean_scrub_stamp":"2024-04-20T11:35:17.736274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:54:40.343940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-20T11:35:20.794479+0000","last_change":"2024-04-20T11:35:19.818725+0000","last_active":"2024-04-20T11:35:20.794479+0000","last_peered":"2024-04-20T11:35:20.794479+0000","last_clean":"2024-04-20T11:35:20.794479+0000","last_became_active":"2024-04-20T11:35:19.818146+0000","last_became_peered":"2024-04-20T11:35:19.818146+0000","last_unstale":"2024-04-20T11:35:20.794479+0000","last_undegraded":"2024-04-20T11:35:20.794479+0000","last_fullsized":"2024-04-20T11:35:20.794479+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:33:52.968975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.b","version":"130'2","reported_seq":16,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-20T11:35:21.847548+0000","last_change":"2024-04-20T11:35:18.789669+0000","last_active":"2024-04-20T11:35:21.847548+0000","last_peered":"2024-04-20T11:35:21.847548+0000","last_clean":"2024-04-20T11:35:21.847548+0000","last_became_active":"2024-04-20T11:35:18.787987+0000","last_became_peered":"2024-04-20T11:35:18.787987+0000","last_unstale":"2024-04-20T11:35:21.847548+0000","last_undegraded":"2024-04-20T11:35:21.847548+0000","last_fullsized":"2024-04-20T11:35:21.847548+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_clean_scrub_stamp":"2024-04-20T11:35:17.736274+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:24:43.592179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-20T11:35:21.847472+0000","last_change":"2024-04-20T11:35:19.809385+0000","last_active":"2024-04-20T11:35:21.847472+0000","last_peered":"2024-04-20T11:35:21.847472+0000","last_clean":"2024-04-20T11:35:21.847472+0000","last_became_active":"2024-04-20T11:35:19.808905+0000","last_became_peered":"2024-04-20T11:35:19.808905+0000","last_unstale":"2024-04-20T11:35:21.847472+0000","last_undegraded":"2024-04-20T11:35:21.847472+0000","last_fullsized":"2024-04-20T11:35:21.847472+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:55:38.723982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":14,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-20T11:35:21.113331+0000","last_change":"2024-04-20T11:35:18.803232+0000","last_active":"2024-04-20T11:35:21.113331+0000","last_peered":"2024-04-20T11:35:21.113331+0000","last_clean":"2024-04-20T11:35:21.113331+0000","last_became_active":"2024-04-20T11:35:18.802923+0000","last_became_peered":"2024-04-20T11:35:18.802923+0000","last_unstale":"2024-04-20T11:35:21.113331+0000","last_undegraded":"2024-04-20T11:35:21.113331+0000","last_fullsized":"2024-04-20T11:35:21.113331+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_clean_scrub_stamp":"2024-04-20T11:35:17.736274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:00:54.033429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-20T11:35:20.791301+0000","last_change":"2024-04-20T11:35:19.805864+0000","last_active":"2024-04-20T11:35:20.791301+0000","last_peered":"2024-04-20T11:35:20.791301+0000","last_clean":"2024-04-20T11:35:20.791301+0000","last_became_active":"2024-04-20T11:35:19.803852+0000","last_became_peered":"2024-04-20T11:35:19.803852+0000","last_unstale":"2024-04-20T11:35:20.791301+0000","last_undegraded":"2024-04-20T11:35:20.791301+0000","last_fullsized":"2024-04-20T11:35:20.791301+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:56:03.378168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":14,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-20T11:35:21.113470+0000","last_change":"2024-04-20T11:35:18.802174+0000","last_active":"2024-04-20T11:35:21.113470+0000","last_peered":"2024-04-20T11:35:21.113470+0000","last_clean":"2024-04-20T11:35:21.113470+0000","last_became_active":"2024-04-20T11:35:18.801743+0000","last_became_peered":"2024-04-20T11:35:18.801743+0000","last_unstale":"2024-04-20T11:35:21.113470+0000","last_undegraded":"2024-04-20T11:35:21.113470+0000","last_fullsized":"2024-04-20T11:35:21.113470+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_clean_scrub_stamp":"2024-04-20T11:35:17.736274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:56:02.363714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-20T11:35:21.847687+0000","last_change":"2024-04-20T11:35:19.812070+0000","last_active":"2024-04-20T11:35:21.847687+0000","last_peered":"2024-04-20T11:35:21.847687+0000","last_clean":"2024-04-20T11:35:21.847687+0000","last_became_active":"2024-04-20T11:35:19.811709+0000","last_became_peered":"2024-04-20T11:35:19.811709+0000","last_unstale":"2024-04-20T11:35:21.847687+0000","last_undegraded":"2024-04-20T11:35:21.847687+0000","last_fullsized":"2024-04-20T11:35:21.847687+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:48:37.410168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":14,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-20T11:35:21.846161+0000","last_change":"2024-04-20T11:35:18.795018+0000","last_active":"2024-04-20T11:35:21.846161+0000","last_peered":"2024-04-20T11:35:21.846161+0000","last_clean":"2024-04-20T11:35:21.846161+0000","last_became_active":"2024-04-20T11:35:18.794690+0000","last_became_peered":"2024-04-20T11:35:18.794690+0000","last_unstale":"2024-04-20T11:35:21.846161+0000","last_undegraded":"2024-04-20T11:35:21.846161+0000","last_fullsized":"2024-04-20T11:35:21.846161+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_clean_scrub_stamp":"2024-04-20T11:35:17.736274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:32:50.880457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-20T11:35:20.792648+0000","last_change":"2024-04-20T11:35:19.815399+0000","last_active":"2024-04-20T11:35:20.792648+0000","last_peered":"2024-04-20T11:35:20.792648+0000","last_clean":"2024-04-20T11:35:20.792648+0000","last_became_active":"2024-04-20T11:35:19.814580+0000","last_became_peered":"2024-04-20T11:35:19.814580+0000","last_unstale":"2024-04-20T11:35:20.792648+0000","last_undegraded":"2024-04-20T11:35:20.792648+0000","last_fullsized":"2024-04-20T11:35:20.792648+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:01:21.940443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.7","version":"130'2","reported_seq":16,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-20T11:35:21.114715+0000","last_change":"2024-04-20T11:35:18.802536+0000","last_active":"2024-04-20T11:35:21.114715+0000","last_peered":"2024-04-20T11:35:21.114715+0000","last_clean":"2024-04-20T11:35:21.114715+0000","last_became_active":"2024-04-20T11:35:18.801972+0000","last_became_peered":"2024-04-20T11:35:18.801972+0000","last_unstale":"2024-04-20T11:35:21.114715+0000","last_undegraded":"2024-04-20T11:35:21.114715+0000","last_fullsized":"2024-04-20T11:35:21.114715+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_clean_scrub_stamp":"2024-04-20T11:35:17.736274+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:54:40.343940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-20T11:35:21.847386+0000","last_change":"2024-04-20T11:35:19.820725+0000","last_active":"2024-04-20T11:35:21.847386+0000","last_peered":"2024-04-20T11:35:21.847386+0000","last_clean":"2024-04-20T11:35:21.847386+0000","last_became_active":"2024-04-20T11:35:19.818958+0000","last_became_peered":"2024-04-20T11:35:19.818958+0000","last_unstale":"2024-04-20T11:35:21.847386+0000","last_undegraded":"2024-04-20T11:35:21.847386+0000","last_fullsized":"2024-04-20T11:35:21.847386+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:18:03.873677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.6","version":"131'1","reported_seq":15,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-20T11:35:21.108006+0000","last_change":"2024-04-20T11:35:18.798170+0000","last_active":"2024-04-20T11:35:21.108006+0000","last_peered":"2024-04-20T11:35:21.108006+0000","last_clean":"2024-04-20T11:35:21.108006+0000","last_became_active":"2024-04-20T11:35:18.797823+0000","last_became_peered":"2024-04-20T11:35:18.797823+0000","last_unstale":"2024-04-20T11:35:21.108006+0000","last_undegraded":"2024-04-20T11:35:21.108006+0000","last_fullsized":"2024-04-20T11:35:21.108006+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_clean_scrub_stamp":"2024-04-20T11:35:17.736274+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:18:34.293331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-20T11:35:21.846344+0000","last_change":"2024-04-20T11:35:19.826131+0000","last_active":"2024-04-20T11:35:21.846344+0000","last_peered":"2024-04-20T11:35:21.846344+0000","last_clean":"2024-04-20T11:35:21.846344+0000","last_became_active":"2024-04-20T11:35:19.825836+0000","last_became_peered":"2024-04-20T11:35:19.825836+0000","last_unstale":"2024-04-20T11:35:21.846344+0000","last_undegraded":"2024-04-20T11:35:21.846344+0000","last_fullsized":"2024-04-20T11:35:21.846344+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:36:10.595750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":14,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-20T11:35:21.107847+0000","last_change":"2024-04-20T11:35:18.790532+0000","last_active":"2024-04-20T11:35:21.107847+0000","last_peered":"2024-04-20T11:35:21.107847+0000","last_clean":"2024-04-20T11:35:21.107847+0000","last_became_active":"2024-04-20T11:35:18.790269+0000","last_became_peered":"2024-04-20T11:35:18.790269+0000","last_unstale":"2024-04-20T11:35:21.107847+0000","last_undegraded":"2024-04-20T11:35:21.107847+0000","last_fullsized":"2024-04-20T11:35:21.107847+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_clean_scrub_stamp":"2024-04-20T11:35:17.736274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:33:34.735874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-20T11:35:21.107856+0000","last_change":"2024-04-20T11:35:19.803963+0000","last_active":"2024-04-20T11:35:21.107856+0000","last_peered":"2024-04-20T11:35:21.107856+0000","last_clean":"2024-04-20T11:35:21.107856+0000","last_became_active":"2024-04-20T11:35:19.803447+0000","last_became_peered":"2024-04-20T11:35:19.803447+0000","last_unstale":"2024-04-20T11:35:21.107856+0000","last_undegraded":"2024-04-20T11:35:21.107856+0000","last_fullsized":"2024-04-20T11:35:21.107856+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:08:11.965198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":14,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-20T11:35:21.107912+0000","last_change":"2024-04-20T11:35:18.790203+0000","last_active":"2024-04-20T11:35:21.107912+0000","last_peered":"2024-04-20T11:35:21.107912+0000","last_clean":"2024-04-20T11:35:21.107912+0000","last_became_active":"2024-04-20T11:35:18.789887+0000","last_became_peered":"2024-04-20T11:35:18.789887+0000","last_unstale":"2024-04-20T11:35:21.107912+0000","last_undegraded":"2024-04-20T11:35:21.107912+0000","last_fullsized":"2024-04-20T11:35:21.107912+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_clean_scrub_stamp":"2024-04-20T11:35:17.736274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:29:24.201752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-20T11:35:20.788062+0000","last_change":"2024-04-20T11:35:19.809276+0000","last_active":"2024-04-20T11:35:20.788062+0000","last_peered":"2024-04-20T11:35:20.788062+0000","last_clean":"2024-04-20T11:35:20.788062+0000","last_became_active":"2024-04-20T11:35:19.808569+0000","last_became_peered":"2024-04-20T11:35:19.808569+0000","last_unstale":"2024-04-20T11:35:20.788062+0000","last_undegraded":"2024-04-20T11:35:20.788062+0000","last_fullsized":"2024-04-20T11:35:20.788062+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:17:12.229603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":14,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-20T11:35:21.847857+0000","last_change":"2024-04-20T11:35:18.786130+0000","last_active":"2024-04-20T11:35:21.847857+0000","last_peered":"2024-04-20T11:35:21.847857+0000","last_clean":"2024-04-20T11:35:21.847857+0000","last_became_active":"2024-04-20T11:35:18.785850+0000","last_became_peered":"2024-04-20T11:35:18.785850+0000","last_unstale":"2024-04-20T11:35:21.847857+0000","last_undegraded":"2024-04-20T11:35:21.847857+0000","last_fullsized":"2024-04-20T11:35:21.847857+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_clean_scrub_stamp":"2024-04-20T11:35:17.736274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:59:41.623176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-20T11:35:20.793274+0000","last_change":"2024-04-20T11:35:19.821133+0000","last_active":"2024-04-20T11:35:20.793274+0000","last_peered":"2024-04-20T11:35:20.793274+0000","last_clean":"2024-04-20T11:35:20.793274+0000","last_became_active":"2024-04-20T11:35:19.820843+0000","last_became_peered":"2024-04-20T11:35:19.820843+0000","last_unstale":"2024-04-20T11:35:20.793274+0000","last_undegraded":"2024-04-20T11:35:20.793274+0000","last_fullsized":"2024-04-20T11:35:20.793274+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:12:21.759120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":14,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-20T11:35:21.113068+0000","last_change":"2024-04-20T11:35:18.802564+0000","last_active":"2024-04-20T11:35:21.113068+0000","last_peered":"2024-04-20T11:35:21.113068+0000","last_clean":"2024-04-20T11:35:21.113068+0000","last_became_active":"2024-04-20T11:35:18.801697+0000","last_became_peered":"2024-04-20T11:35:18.801697+0000","last_unstale":"2024-04-20T11:35:21.113068+0000","last_undegraded":"2024-04-20T11:35:21.113068+0000","last_fullsized":"2024-04-20T11:35:21.113068+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_clean_scrub_stamp":"2024-04-20T11:35:17.736274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:02:15.829312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-20T11:35:21.113335+0000","last_change":"2024-04-20T11:35:19.820151+0000","last_active":"2024-04-20T11:35:21.113335+0000","last_peered":"2024-04-20T11:35:21.113335+0000","last_clean":"2024-04-20T11:35:21.113335+0000","last_became_active":"2024-04-20T11:35:19.818848+0000","last_became_peered":"2024-04-20T11:35:19.818848+0000","last_unstale":"2024-04-20T11:35:21.113335+0000","last_undegraded":"2024-04-20T11:35:21.113335+0000","last_fullsized":"2024-04-20T11:35:21.113335+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:59:42.637631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":14,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-20T11:35:21.846476+0000","last_change":"2024-04-20T11:35:18.793935+0000","last_active":"2024-04-20T11:35:21.846476+0000","last_peered":"2024-04-20T11:35:21.846476+0000","last_clean":"2024-04-20T11:35:21.846476+0000","last_became_active":"2024-04-20T11:35:18.793641+0000","last_became_peered":"2024-04-20T11:35:18.793641+0000","last_unstale":"2024-04-20T11:35:21.846476+0000","last_undegraded":"2024-04-20T11:35:21.846476+0000","last_fullsized":"2024-04-20T11:35:21.846476+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_clean_scrub_stamp":"2024-04-20T11:35:17.736274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:14:29.346855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-20T11:35:20.793636+0000","last_change":"2024-04-20T11:35:19.818467+0000","last_active":"2024-04-20T11:35:20.793636+0000","last_peered":"2024-04-20T11:35:20.793636+0000","last_clean":"2024-04-20T11:35:20.793636+0000","last_became_active":"2024-04-20T11:35:19.818183+0000","last_became_peered":"2024-04-20T11:35:19.818183+0000","last_unstale":"2024-04-20T11:35:20.793636+0000","last_undegraded":"2024-04-20T11:35:20.793636+0000","last_fullsized":"2024-04-20T11:35:20.793636+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:41:13.721378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":14,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-20T11:35:21.845585+0000","last_change":"2024-04-20T11:35:18.798773+0000","last_active":"2024-04-20T11:35:21.845585+0000","last_peered":"2024-04-20T11:35:21.845585+0000","last_clean":"2024-04-20T11:35:21.845585+0000","last_became_active":"2024-04-20T11:35:18.797908+0000","last_became_peered":"2024-04-20T11:35:18.797908+0000","last_unstale":"2024-04-20T11:35:21.845585+0000","last_undegraded":"2024-04-20T11:35:21.845585+0000","last_fullsized":"2024-04-20T11:35:21.845585+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_clean_scrub_stamp":"2024-04-20T11:35:17.736274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:02:15.829312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-20T11:35:21.847827+0000","last_change":"2024-04-20T11:35:19.805122+0000","last_active":"2024-04-20T11:35:21.847827+0000","last_peered":"2024-04-20T11:35:21.847827+0000","last_clean":"2024-04-20T11:35:21.847827+0000","last_became_active":"2024-04-20T11:35:19.804372+0000","last_became_peered":"2024-04-20T11:35:19.804372+0000","last_unstale":"2024-04-20T11:35:21.847827+0000","last_undegraded":"2024-04-20T11:35:21.847827+0000","last_fullsized":"2024-04-20T11:35:21.847827+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:18.750729+0000","last_clean_scrub_stamp":"2024-04-20T11:35:18.750729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:40:38.426556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.15","version":"130'3","reported_seq":17,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-20T11:35:21.847569+0000","last_change":"2024-04-20T11:35:18.790212+0000","last_active":"2024-04-20T11:35:21.847569+0000","last_peered":"2024-04-20T11:35:21.847569+0000","last_clean":"2024-04-20T11:35:21.847569+0000","last_became_active":"2024-04-20T11:35:18.789914+0000","last_became_peered":"2024-04-20T11:35:18.789914+0000","last_unstale":"2024-04-20T11:35:21.847569+0000","last_undegraded":"2024-04-20T11:35:21.847569+0000","last_fullsized":"2024-04-20T11:35:21.847569+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:35:17.736274+0000","last_clean_scrub_stamp":"2024-04-20T11:35:17.736274+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:43:25.837469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"72'128","reported_seq":407,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-20T11:35:21.113576+0000","last_change":"2024-04-20T11:26:48.661897+0000","last_active":"2024-04-20T11:35:21.113576+0000","last_peered":"2024-04-20T11:35:21.113576+0000","last_clean":"2024-04-20T11:35:21.113576+0000","last_became_active":"2024-04-20T11:26:48.656535+0000","last_became_peered":"2024-04-20T11:26:48.656535+0000","last_unstale":"2024-04-20T11:35:21.113576+0000","last_undegraded":"2024-04-20T11:35:21.113576+0000","last_fullsized":"2024-04-20T11:35:21.113576+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:47.543004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:47.543004+0000","last_clean_scrub_stamp":"2024-04-20T11:26:47.543004+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:58:35.527105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574957,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33380,"kb_used_data":2192,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338460,"statfs":{"total":96636764160,"available":96602583040,"internally_reserved":0,"allocated":2244608,"data_stored":1317538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574954,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32796,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339044,"statfs":{"total":96636764160,"available":96603181056,"internally_reserved":0,"allocated":1654784,"data_stored":726664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574956,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32792,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339048,"statfs":{"total":96636764160,"available":96603185152,"internally_reserved":0,"allocated":1646592,"data_stored":726540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574954,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32788,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339052,"statfs":{"total":96636764160,"available":96603189248,"internally_reserved":0,"allocated":1642496,"data_stored":726518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574955,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32788,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339052,"statfs":{"total":96636764160,"available":96603189248,"internally_reserved":0,"allocated":1646592,"data_stored":726608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574956,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32808,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339032,"statfs":{"total":96636764160,"available":96603168768,"internally_reserved":0,"allocated":1658880,"data_stored":728200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574954,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32792,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339048,"statfs":{"total":96636764160,"available":96603185152,"internally_reserved":0,"allocated":1646592,"data_stored":727100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574955,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33384,"kb_used_data":2200,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338456,"statfs":{"total":96636764160,"available":96602578944,"internally_reserved":0,"allocated":2252800,"data_stored":1318532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T11:35:25.086 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-20T11:35:25.086 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-20T11:35:25.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.338+0000 7f4be3200640 1 Processor -- start 2024-04-20T11:35:25.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.338+0000 7f4be3200640 1 -- start start 2024-04-20T11:35:25.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.340+0000 7f4be3200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4bdc159320 0x7f4bdc15bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:25.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.340+0000 7f4be3200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f4bdc0d0e60 0x7f4bdc06d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:25.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.340+0000 7f4be3200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4bdc0cfef0 con 0x7f4bdc06df20 2024-04-20T11:35:25.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.340+0000 7f4be3200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f4bdc06e370 con 0x7f4bdc0d0e60 2024-04-20T11:35:25.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.340+0000 7f4be3200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4bdc15c130 con 0x7f4bdc159320 2024-04-20T11:35:25.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.340+0000 7f4be1400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f4bdc0d0e60 0x7f4bdc06d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:25.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.340+0000 7f4be1400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f4bdc0d0e60 0x7f4bdc06d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52404/0 (socket says 172.21.15.152:52404) 2024-04-20T11:35:25.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.340+0000 7f4be1400640 1 -- 172.21.15.152:0/2689584863 learned_addr learned my addr 172.21.15.152:0/2689584863 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:25.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.341+0000 7f4be1e00640 1 --2- 172.21.15.152:0/2689584863 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4bdc159320 0x7f4bdc15bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:25.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.341+0000 7f4be1400640 1 -- 172.21.15.152:0/2689584863 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4bdc159320 msgr2=0x7f4bdc15bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:25.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.341+0000 7f4be1400640 1 --2- 172.21.15.152:0/2689584863 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4bdc159320 0x7f4bdc15bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:25.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.341+0000 7f4be1400640 1 -- 172.21.15.152:0/2689584863 >> v1:172.21.15.175:6789/0 conn(0x7f4bdc06df20 legacy=0x7f4bdc0b38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:25.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.341+0000 7f4be1400640 1 -- 172.21.15.152:0/2689584863 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bdc15c3d0 con 0x7f4bdc0d0e60 2024-04-20T11:35:25.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.341+0000 7f4be1e00640 1 --2- 172.21.15.152:0/2689584863 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4bdc159320 0x7f4bdc15bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:35:25.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.342+0000 7f4be1400640 1 --2- 172.21.15.152:0/2689584863 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4bdc0d0e60 0x7f4bdc06d9e0 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc009870 tx=0x7f4bcc02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3999b2cdccedd7ee server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:25.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.342+0000 7f4be0a00640 1 -- 172.21.15.152:0/2689584863 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bcc0317c0 con 0x7f4bdc0d0e60 2024-04-20T11:35:25.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.342+0000 7f4be0a00640 1 -- 172.21.15.152:0/2689584863 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bcc037070 con 0x7f4bdc0d0e60 2024-04-20T11:35:25.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.342+0000 7f4be0a00640 1 -- 172.21.15.152:0/2689584863 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bcc031b80 con 0x7f4bdc0d0e60 2024-04-20T11:35:25.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.343+0000 7f4be3200640 1 -- 172.21.15.152:0/2689584863 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4bdc0d0e60 msgr2=0x7f4bdc06d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:25.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.343+0000 7f4be3200640 1 --2- 172.21.15.152:0/2689584863 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4bdc0d0e60 0x7f4bdc06d9e0 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc009870 tx=0x7f4bcc02b860 comp rx=0 tx=0).stop 2024-04-20T11:35:25.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.343+0000 7f4be3200640 1 -- 172.21.15.152:0/2689584863 shutdown_connections 2024-04-20T11:35:25.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.343+0000 7f4be3200640 1 --2- 172.21.15.152:0/2689584863 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4bdc0d0e60 0x7f4bdc06d9e0 unknown :-1 s=CLOSED pgs=1764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:25.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.343+0000 7f4be3200640 1 --2- 172.21.15.152:0/2689584863 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4bdc159320 0x7f4bdc15bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:25.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.343+0000 7f4be3200640 1 -- 172.21.15.152:0/2689584863 >> 172.21.15.152:0/2689584863 conn(0x7f4bdc0d0260 msgr2=0x7f4bdc0d0670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:25.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.343+0000 7f4be3200640 1 -- 172.21.15.152:0/2689584863 shutdown_connections 2024-04-20T11:35:25.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.343+0000 7f4be3200640 1 -- 172.21.15.152:0/2689584863 wait complete. 2024-04-20T11:35:25.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.346+0000 7f4be3200640 1 Processor -- start 2024-04-20T11:35:25.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.346+0000 7f4be3200640 1 -- start start 2024-04-20T11:35:25.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.347+0000 7f4be3200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4bdc06df20 0x7f4bdc151470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:25.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.347+0000 7f4be3200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4bdc0d0e60 0x7f4bdc1522c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:25.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.347+0000 7f4be3200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4bdc159320 0x7f4bdc0ae300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:25.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.347+0000 7f4be3200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f4bdc15cf30 con 0x7f4bdc06df20 2024-04-20T11:35:25.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.347+0000 7f4be3200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4bdc15c990 con 0x7f4bdc0d0e60 2024-04-20T11:35:25.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.347+0000 7f4be3200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4bdc15e460 con 0x7f4bdc159320 2024-04-20T11:35:25.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.347+0000 7f4be1e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4bdc06df20 0x7f4bdc151470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:25.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.347+0000 7f4be1e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4bdc06df20 0x7f4bdc151470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52406/0 (socket says 172.21.15.152:52406) 2024-04-20T11:35:25.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.347+0000 7f4be1e00640 1 -- 172.21.15.152:0/1430955882 learned_addr learned my addr 172.21.15.152:0/1430955882 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:25.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.348+0000 7f4be1e00640 1 -- 172.21.15.152:0/1430955882 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4bdc159320 msgr2=0x7f4bdc0ae300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:25.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.348+0000 7f4be2800640 1 --2- 172.21.15.152:0/1430955882 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4bdc159320 0x7f4bdc0ae300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:25.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.348+0000 7f4be1e00640 1 --2- 172.21.15.152:0/1430955882 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4bdc159320 0x7f4bdc0ae300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:25.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.348+0000 7f4be1e00640 1 -- 172.21.15.152:0/1430955882 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4bdc0d0e60 msgr2=0x7f4bdc1522c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:25.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.348+0000 7f4be1e00640 1 --2- 172.21.15.152:0/1430955882 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4bdc0d0e60 0x7f4bdc1522c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:25.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.348+0000 7f4be1e00640 1 -- 172.21.15.152:0/1430955882 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bdc1e35a0 con 0x7f4bdc06df20 2024-04-20T11:35:25.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.348+0000 7f4be1e00640 1 --2- 172.21.15.152:0/1430955882 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4bdc06df20 0x7f4bdc151470 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f4bd000c7f0 tx=0x7f4bd000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:25.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.348+0000 7f4bd6000640 1 -- 172.21.15.152:0/1430955882 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd0015070 con 0x7f4bdc06df20 2024-04-20T11:35:25.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.348+0000 7f4bd6000640 1 -- 172.21.15.152:0/1430955882 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bd00077c0 con 0x7f4bdc06df20 2024-04-20T11:35:25.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.348+0000 7f4be3200640 1 -- 172.21.15.152:0/1430955882 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bdc1f8170 con 0x7f4bdc06df20 2024-04-20T11:35:25.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.348+0000 7f4bd6000640 1 -- 172.21.15.152:0/1430955882 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd0007b70 con 0x7f4bdc06df20 2024-04-20T11:35:25.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.349+0000 7f4be3200640 1 -- 172.21.15.152:0/1430955882 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bdc1f8730 con 0x7f4bdc06df20 2024-04-20T11:35:25.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.350+0000 7f4be3200640 1 -- 172.21.15.152:0/1430955882 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bdc075e60 con 0x7f4bdc06df20 2024-04-20T11:35:25.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.351+0000 7f4bd6000640 1 -- 172.21.15.152:0/1430955882 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 21) ==== 91550+0+0 (secure 0 0 0) 0x7f4bd0018070 con 0x7f4bdc06df20 2024-04-20T11:35:25.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.352+0000 7f4bd6000640 1 --2- 172.21.15.152:0/1430955882 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4bc006ce90 0x7f4bc006f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:25.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.352+0000 7f4bd6000640 1 -- 172.21.15.152:0/1430955882 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5709+0+0 (secure 0 0 0) 0x7f4bd008c690 con 0x7f4bdc06df20 2024-04-20T11:35:25.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.352+0000 7f4be1400640 1 --2- 172.21.15.152:0/1430955882 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4bc006ce90 0x7f4bc006f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:25.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.354+0000 7f4be1400640 1 --2- 172.21.15.152:0/1430955882 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4bc006ce90 0x7f4bc006f350 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc036000 tx=0x7f4bcc030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:25.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.356+0000 7f4bd6000640 1 -- 172.21.15.152:0/1430955882 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f4bd005a250 con 0x7f4bdc06df20 2024-04-20T11:35:25.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.628+0000 7f4be3200640 1 -- 172.21.15.152:0/1430955882 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f4bdc071e00 con 0x7f4bdc06df20 2024-04-20T11:35:25.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.633+0000 7f4bd6000640 1 -- 172.21.15.152:0/1430955882 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v19) ==== 190+0+147 (secure 0 0 0) 0x7f4bd0059b50 con 0x7f4bdc06df20 2024-04-20T11:35:25.634 INFO:teuthology.orchestra.run.smithi152.stdout:[client.0] 2024-04-20T11:35:25.634 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQBDpiNmtsBWFhAArq9A7r33CcwgMmOU2pAGPw== 2024-04-20T11:35:25.634 INFO:teuthology.orchestra.run.smithi152.stdout: caps mds = "allow" 2024-04-20T11:35:25.634 INFO:teuthology.orchestra.run.smithi152.stdout: caps mon = "allow r" 2024-04-20T11:35:25.634 INFO:teuthology.orchestra.run.smithi152.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-20T11:35:25.635 INFO:teuthology.orchestra.run.smithi152.stderr:updated caps for client.0 2024-04-20T11:35:25.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.636+0000 7f4be3200640 1 -- 172.21.15.152:0/1430955882 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4bc006ce90 msgr2=0x7f4bc006f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:25.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.636+0000 7f4be3200640 1 --2- 172.21.15.152:0/1430955882 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4bc006ce90 0x7f4bc006f350 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc036000 tx=0x7f4bcc030000 comp rx=0 tx=0).stop 2024-04-20T11:35:25.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.636+0000 7f4be3200640 1 -- 172.21.15.152:0/1430955882 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4bdc06df20 msgr2=0x7f4bdc151470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:25.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.636+0000 7f4be3200640 1 --2- 172.21.15.152:0/1430955882 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4bdc06df20 0x7f4bdc151470 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f4bd000c7f0 tx=0x7f4bd000ccb0 comp rx=0 tx=0).stop 2024-04-20T11:35:25.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.636+0000 7f4be3200640 1 -- 172.21.15.152:0/1430955882 shutdown_connections 2024-04-20T11:35:25.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.636+0000 7f4be3200640 1 --2- 172.21.15.152:0/1430955882 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4bc006ce90 0x7f4bc006f350 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:25.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.636+0000 7f4be3200640 1 --2- 172.21.15.152:0/1430955882 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4bdc159320 0x7f4bdc0ae300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:25.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.636+0000 7f4be3200640 1 --2- 172.21.15.152:0/1430955882 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4bdc0d0e60 0x7f4bdc1522c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:25.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.636+0000 7f4be3200640 1 --2- 172.21.15.152:0/1430955882 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4bdc06df20 0x7f4bdc151470 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:25.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.636+0000 7f4be3200640 1 -- 172.21.15.152:0/1430955882 >> 172.21.15.152:0/1430955882 conn(0x7f4bdc0d0260 msgr2=0x7f4bdc0afce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:25.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.636+0000 7f4be3200640 1 -- 172.21.15.152:0/1430955882 shutdown_connections 2024-04-20T11:35:25.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.636+0000 7f4be3200640 1 -- 172.21.15.152:0/1430955882 wait complete. 2024-04-20T11:35:25.656 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:35:25.730 INFO:tasks.ceph.mds.a.smithi152.stderr:2024-04-20T11:35:25.729+0000 7f8784c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-20T11:35:25.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.856+0000 7fbd3b600640 1 Processor -- start 2024-04-20T11:35:25.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.856+0000 7fbd3b600640 1 -- start start 2024-04-20T11:35:25.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.858+0000 7fbd3b600640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fbd340b3d80 0x7fbd340b4210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:25.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.858+0000 7fbd3b600640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fbd341643b0 0x7fbd34164800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:25.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.858+0000 7fbd3b600640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd3406e140 con 0x7fbd34160870 2024-04-20T11:35:25.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.858+0000 7fbd3b600640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fbd34160c50 con 0x7fbd340b3d80 2024-04-20T11:35:25.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.858+0000 7fbd3b600640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fbd3406e420 con 0x7fbd341643b0 2024-04-20T11:35:25.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.858+0000 7fbd2be00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fbd340b3d80 0x7fbd340b4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:25.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.858+0000 7fbd2be00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fbd340b3d80 0x7fbd340b4210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54434/0 (socket says 172.21.15.152:54434) 2024-04-20T11:35:25.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.858+0000 7fbd2be00640 1 -- 172.21.15.152:0/3807949459 learned_addr learned my addr 172.21.15.152:0/3807949459 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:25.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.858+0000 7fbd2be00640 1 -- 172.21.15.152:0/3807949459 >> v2:172.21.15.175:3300/0 conn(0x7fbd341643b0 msgr2=0x7fbd34164800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:25.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.858+0000 7fbd2be00640 1 --2- 172.21.15.152:0/3807949459 >> v2:172.21.15.175:3300/0 conn(0x7fbd341643b0 0x7fbd34164800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:25.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.858+0000 7fbd2be00640 1 -- 172.21.15.152:0/3807949459 >> v1:172.21.15.175:6789/0 conn(0x7fbd34160870 legacy=0x7fbd34163c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:25.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.858+0000 7fbd2be00640 1 -- 172.21.15.152:0/3807949459 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd3406e700 con 0x7fbd340b3d80 2024-04-20T11:35:25.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.859+0000 7fbd2be00640 1 --2- 172.21.15.152:0/3807949459 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd340b3d80 0x7fbd340b4210 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7fbd341631f0 tx=0x7fbd1c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd606acf1489dded server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:25.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.860+0000 7fbd33e00640 1 -- 172.21.15.152:0/3807949459 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd1c035020 con 0x7fbd340b3d80 2024-04-20T11:35:25.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.860+0000 7fbd33e00640 1 -- 172.21.15.152:0/3807949459 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd1c031870 con 0x7fbd340b3d80 2024-04-20T11:35:25.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.860+0000 7fbd33e00640 1 -- 172.21.15.152:0/3807949459 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd1c031c40 con 0x7fbd340b3d80 2024-04-20T11:35:25.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.860+0000 7fbd3b600640 1 -- 172.21.15.152:0/3807949459 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd340b3d80 msgr2=0x7fbd340b4210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:25.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.860+0000 7fbd3b600640 1 --2- 172.21.15.152:0/3807949459 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd340b3d80 0x7fbd340b4210 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7fbd341631f0 tx=0x7fbd1c02b910 comp rx=0 tx=0).stop 2024-04-20T11:35:25.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.861+0000 7fbd3b600640 1 -- 172.21.15.152:0/3807949459 shutdown_connections 2024-04-20T11:35:25.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.861+0000 7fbd3b600640 1 --2- 172.21.15.152:0/3807949459 >> v2:172.21.15.175:3300/0 conn(0x7fbd341643b0 0x7fbd34164800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:25.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.861+0000 7fbd3b600640 1 --2- 172.21.15.152:0/3807949459 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd340b3d80 0x7fbd340b4210 unknown :-1 s=CLOSED pgs=1766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:25.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.861+0000 7fbd3b600640 1 -- 172.21.15.152:0/3807949459 >> 172.21.15.152:0/3807949459 conn(0x7fbd34067cf0 msgr2=0x7fbd3406b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:25.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.861+0000 7fbd3b600640 1 -- 172.21.15.152:0/3807949459 shutdown_connections 2024-04-20T11:35:25.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.861+0000 7fbd3b600640 1 -- 172.21.15.152:0/3807949459 wait complete. 2024-04-20T11:35:25.863 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.862+0000 7fbd3b600640 1 Processor -- start 2024-04-20T11:35:25.863 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.863+0000 7fbd3b600640 1 -- start start 2024-04-20T11:35:25.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.863+0000 7fbd3b600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd340b3d80 0x7fbd340cd060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:25.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.863+0000 7fbd3b600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd34160870 0x7fbd3414e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:25.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.864+0000 7fbd3b600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd341643b0 0x7fbd340ceb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:25.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.864+0000 7fbd3b600640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fbd3406de10 con 0x7fbd34160870 2024-04-20T11:35:25.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.864+0000 7fbd3b600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbd3406ecc0 con 0x7fbd340b3d80 2024-04-20T11:35:25.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.864+0000 7fbd3b600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fbd3406f1e0 con 0x7fbd341643b0 2024-04-20T11:35:25.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.864+0000 7fbd2be00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd34160870 0x7fbd3414e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:25.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.864+0000 7fbd2be00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd34160870 0x7fbd3414e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54442/0 (socket says 172.21.15.152:54442) 2024-04-20T11:35:25.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.864+0000 7fbd2be00640 1 -- 172.21.15.152:0/1675922690 learned_addr learned my addr 172.21.15.152:0/1675922690 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:25.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.864+0000 7fbd2be00640 1 -- 172.21.15.152:0/1675922690 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd341643b0 msgr2=0x7fbd340ceb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:25.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.864+0000 7fbd2be00640 1 --2- 172.21.15.152:0/1675922690 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd341643b0 0x7fbd340ceb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:25.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.864+0000 7fbd2be00640 1 -- 172.21.15.152:0/1675922690 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd340b3d80 msgr2=0x7fbd340cd060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:25.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.864+0000 7fbd2be00640 1 --2- 172.21.15.152:0/1675922690 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd340b3d80 0x7fbd340cd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:25.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.864+0000 7fbd2be00640 1 -- 172.21.15.152:0/1675922690 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd3414eb90 con 0x7fbd34160870 2024-04-20T11:35:25.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.865+0000 7fbd2be00640 1 --2- 172.21.15.152:0/1675922690 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd34160870 0x7fbd3414e650 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c000c00 tx=0x7fbd1c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:25.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.865+0000 7fbd31600640 1 -- 172.21.15.152:0/1675922690 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd1c03ad60 con 0x7fbd34160870 2024-04-20T11:35:25.867 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.865+0000 7fbd31600640 1 -- 172.21.15.152:0/1675922690 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd1c0443f0 con 0x7fbd34160870 2024-04-20T11:35:25.867 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.865+0000 7fbd31600640 1 -- 172.21.15.152:0/1675922690 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd1c0447e0 con 0x7fbd34160870 2024-04-20T11:35:25.867 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.865+0000 7fbd3b600640 1 -- 172.21.15.152:0/1675922690 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd340cf0b0 con 0x7fbd34160870 2024-04-20T11:35:25.867 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.865+0000 7fbd3b600640 1 -- 172.21.15.152:0/1675922690 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd340cdad0 con 0x7fbd34160870 2024-04-20T11:35:25.867 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.866+0000 7fbd3b600640 1 -- 172.21.15.152:0/1675922690 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd340689f0 con 0x7fbd34160870 2024-04-20T11:35:25.867 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.867+0000 7fbd31600640 1 -- 172.21.15.152:0/1675922690 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 21) ==== 91550+0+0 (secure 0 0 0) 0x7fbd1c04b060 con 0x7fbd34160870 2024-04-20T11:35:25.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.868+0000 7fbd31600640 1 --2- 172.21.15.152:0/1675922690 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fbd1006ce90 0x7fbd1006f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:25.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.868+0000 7fbd31600640 1 -- 172.21.15.152:0/1675922690 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5709+0+0 (secure 0 0 0) 0x7fbd1c0aea00 con 0x7fbd34160870 2024-04-20T11:35:25.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.869+0000 7fbd38c00640 1 --2- 172.21.15.152:0/1675922690 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fbd1006ce90 0x7fbd1006f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:25.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.870+0000 7fbd38c00640 1 --2- 172.21.15.152:0/1675922690 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fbd1006ce90 0x7fbd1006f350 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fbd24001560 tx=0x7fbd24007990 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:25.873 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:25.872+0000 7fbd31600640 1 -- 172.21.15.152:0/1675922690 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fbd3414eb90 con 0x7fbd34160870 2024-04-20T11:35:26.112 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.111+0000 7fbd3b600640 1 -- 172.21.15.152:0/1675922690 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbd3406bbd0 con 0x7fbd34160870 2024-04-20T11:35:26.113 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.112+0000 7fbd31600640 1 -- 172.21.15.152:0/1675922690 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) ==== 94+0+5212 (secure 0 0 0) 0x7fbd1c03f070 con 0x7fbd34160870 2024-04-20T11:35:26.113 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:35:26.113 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":86,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6062,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1303172876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6833","nonce":1303172876},{"type":"v1","addr":"172.21.15.175:6835","nonce":1303172876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6667,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6834/1975208598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1975208598},{"type":"v1","addr":"172.21.15.175:6834","nonce":1975208598}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6670,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6834/287411071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":287411071},{"type":"v1","addr":"172.21.15.152:6834","nonce":287411071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T11:35:19.231687+0000","modified":"2024-04-20T11:35:22.597626+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8919},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8919":{"gid":8919,"name":"a","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6835/1884827433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6833","nonce":1884827433},{"type":"v1","addr":"172.21.15.152:6835","nonce":1884827433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8919,"qdb_cluster":[8919]},"id":10}]} 2024-04-20T11:35:26.114 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 86 2024-04-20T11:35:26.116 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.115+0000 7fbd3b600640 1 -- 172.21.15.152:0/1675922690 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fbd1006ce90 msgr2=0x7fbd1006f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:26.116 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.115+0000 7fbd3b600640 1 --2- 172.21.15.152:0/1675922690 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fbd1006ce90 0x7fbd1006f350 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fbd24001560 tx=0x7fbd24007990 comp rx=0 tx=0).stop 2024-04-20T11:35:26.116 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.115+0000 7fbd3b600640 1 -- 172.21.15.152:0/1675922690 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd34160870 msgr2=0x7fbd3414e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:26.116 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.115+0000 7fbd3b600640 1 --2- 172.21.15.152:0/1675922690 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd34160870 0x7fbd3414e650 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c000c00 tx=0x7fbd1c0046c0 comp rx=0 tx=0).stop 2024-04-20T11:35:26.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.115+0000 7fbd3b600640 1 -- 172.21.15.152:0/1675922690 shutdown_connections 2024-04-20T11:35:26.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.115+0000 7fbd3b600640 1 --2- 172.21.15.152:0/1675922690 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fbd1006ce90 0x7fbd1006f350 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:26.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.115+0000 7fbd3b600640 1 --2- 172.21.15.152:0/1675922690 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd341643b0 0x7fbd340ceb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:26.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.115+0000 7fbd3b600640 1 --2- 172.21.15.152:0/1675922690 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd34160870 0x7fbd3414e650 unknown :-1 s=CLOSED pgs=1767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:26.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.116+0000 7fbd3b600640 1 --2- 172.21.15.152:0/1675922690 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd340b3d80 0x7fbd340cd060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:26.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.116+0000 7fbd3b600640 1 -- 172.21.15.152:0/1675922690 >> 172.21.15.152:0/1675922690 conn(0x7fbd34067cf0 msgr2=0x7fbd34165cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:26.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.116+0000 7fbd3b600640 1 -- 172.21.15.152:0/1675922690 shutdown_connections 2024-04-20T11:35:26.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.116+0000 7fbd3b600640 1 -- 172.21.15.152:0/1675922690 wait complete. 2024-04-20T11:35:26.539 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 86, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-20T11:35:19.231687+0000', 'modified': '2024-04-20T11:35:22.597626+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'snap_rstat': True, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8919}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8919': {'gid': 8919, 'name': 'a', 'rank': 0, 'incarnation': 82, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.152:6835/1884827433', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6833', 'nonce': 1884827433}, {'type': 'v1', 'addr': '172.21.15.152:6835', 'nonce': 1884827433}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8919, 'qdb_cluster': [8919]} 2024-04-20T11:35:26.539 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-20T11:35:26.539 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-20T11:35:26.751 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.750+0000 7f26fe400640 1 Processor -- start 2024-04-20T11:35:26.751 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.750+0000 7f26fe400640 1 -- start start 2024-04-20T11:35:26.752 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.752+0000 7f26fe400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f26f8161d30 0x7f26f80d0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:26.753 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.752+0000 7f26fe400640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f26f80d2b80 0x7f26f80d3030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:26.753 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.752+0000 7f26fe400640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26f80d26a0 con 0x7f26f80d4ec0 2024-04-20T11:35:26.753 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.752+0000 7f26fe400640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f26f80d0540 con 0x7f26f8161d30 2024-04-20T11:35:26.753 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.752+0000 7f26fe400640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f26f80cfb70 con 0x7f26f80d2b80 2024-04-20T11:35:26.753 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.752+0000 7f26fd000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f26f8161d30 0x7f26f80d0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:26.753 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.752+0000 7f26fd000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f26f8161d30 0x7f26f80d0000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54444/0 (socket says 172.21.15.152:54444) 2024-04-20T11:35:26.753 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.752+0000 7f26fd000640 1 -- 172.21.15.152:0/1399513148 learned_addr learned my addr 172.21.15.152:0/1399513148 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:26.753 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.753+0000 7f26fd000640 1 -- 172.21.15.152:0/1399513148 >> v2:172.21.15.175:3300/0 conn(0x7f26f80d2b80 msgr2=0x7f26f80d3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:26.753 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.753+0000 7f26fd000640 1 --2- 172.21.15.152:0/1399513148 >> v2:172.21.15.175:3300/0 conn(0x7f26f80d2b80 0x7f26f80d3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:26.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.753+0000 7f26fd000640 1 -- 172.21.15.152:0/1399513148 >> v1:172.21.15.152:6789/0 conn(0x7f26f80d4ec0 legacy=0x7f26f80d15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:26.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.753+0000 7f26fd000640 1 -- 172.21.15.152:0/1399513148 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26f80d53c0 con 0x7f26f8161d30 2024-04-20T11:35:26.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.753+0000 7f26f7400640 1 -- 172.21.15.152:0/1399513148 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3003661077 0 0) 0x7f26f80d26a0 con 0x7f26f80d4ec0 2024-04-20T11:35:26.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.754+0000 7f26fd000640 1 --2- 172.21.15.152:0/1399513148 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f26f8161d30 0x7f26f80d0000 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f26f808e550 tx=0x7f26e002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50a404873dbbfefc server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:26.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.754+0000 7f26f7400640 1 -- 172.21.15.152:0/1399513148 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e0034070 con 0x7f26f8161d30 2024-04-20T11:35:26.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.754+0000 7f26f7400640 1 -- 172.21.15.152:0/1399513148 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26e0030860 con 0x7f26f8161d30 2024-04-20T11:35:26.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.754+0000 7f26f7400640 1 -- 172.21.15.152:0/1399513148 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e0030c30 con 0x7f26f8161d30 2024-04-20T11:35:26.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.755+0000 7f26fe400640 1 -- 172.21.15.152:0/1399513148 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f26f8161d30 msgr2=0x7f26f80d0000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:26.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.755+0000 7f26fe400640 1 --2- 172.21.15.152:0/1399513148 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f26f8161d30 0x7f26f80d0000 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f26f808e550 tx=0x7f26e002b9a0 comp rx=0 tx=0).stop 2024-04-20T11:35:26.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.755+0000 7f26fe400640 1 -- 172.21.15.152:0/1399513148 shutdown_connections 2024-04-20T11:35:26.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.755+0000 7f26fe400640 1 --2- 172.21.15.152:0/1399513148 >> v2:172.21.15.175:3300/0 conn(0x7f26f80d2b80 0x7f26f80d3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:26.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.755+0000 7f26fe400640 1 --2- 172.21.15.152:0/1399513148 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f26f8161d30 0x7f26f80d0000 unknown :-1 s=CLOSED pgs=1769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:26.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.755+0000 7f26fe400640 1 -- 172.21.15.152:0/1399513148 >> 172.21.15.152:0/1399513148 conn(0x7f26f808bfd0 msgr2=0x7f26f80d45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:26.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.755+0000 7f26fe400640 1 -- 172.21.15.152:0/1399513148 shutdown_connections 2024-04-20T11:35:26.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.755+0000 7f26fe400640 1 -- 172.21.15.152:0/1399513148 wait complete. 2024-04-20T11:35:26.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.757+0000 7f26fe400640 1 Processor -- start 2024-04-20T11:35:26.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.757+0000 7f26fe400640 1 -- start start 2024-04-20T11:35:26.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.758+0000 7f26fe400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f26f80d2b80 0x7f26f81f46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:26.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.758+0000 7f26fe400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26f80d4ec0 0x7f26f81532c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:26.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.758+0000 7f26fe400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f26f8161d30 0x7f26f8153800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:26.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.758+0000 7f26fe400640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f26f80cf1b0 con 0x7f26f80d2b80 2024-04-20T11:35:26.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.758+0000 7f26fe400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f26f80cf800 con 0x7f26f80d4ec0 2024-04-20T11:35:26.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.758+0000 7f26fe400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f26f8162110 con 0x7f26f8161d30 2024-04-20T11:35:26.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.758+0000 7f26fd000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f26f80d2b80 0x7f26f81f46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:26.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.758+0000 7f26fd000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f26f80d2b80 0x7f26f81f46e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54446/0 (socket says 172.21.15.152:54446) 2024-04-20T11:35:26.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.758+0000 7f26fd000640 1 -- 172.21.15.152:0/3863466276 learned_addr learned my addr 172.21.15.152:0/3863466276 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:26.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.758+0000 7f26f7e00640 1 --2- 172.21.15.152:0/3863466276 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26f80d4ec0 0x7f26f81532c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:26.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.758+0000 7f26fd000640 1 -- 172.21.15.152:0/3863466276 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f26f8161d30 msgr2=0x7f26f8153800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:26.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.758+0000 7f26fd000640 1 --2- 172.21.15.152:0/3863466276 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f26f8161d30 0x7f26f8153800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:26.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.758+0000 7f26fd000640 1 -- 172.21.15.152:0/3863466276 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26f80d4ec0 msgr2=0x7f26f81532c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:26.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.758+0000 7f26fd000640 1 --2- 172.21.15.152:0/3863466276 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26f80d4ec0 0x7f26f81532c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:26.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.758+0000 7f26fd000640 1 -- 172.21.15.152:0/3863466276 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26f8153f10 con 0x7f26f80d2b80 2024-04-20T11:35:26.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.758+0000 7f26f7e00640 1 --2- 172.21.15.152:0/3863466276 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26f80d4ec0 0x7f26f81532c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:35:26.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.759+0000 7f26fd000640 1 --2- 172.21.15.152:0/3863466276 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f26f80d2b80 0x7f26f81f46e0 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f26e0000c00 tx=0x7f26e0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:26.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.759+0000 7f26f4c00640 1 -- 172.21.15.152:0/3863466276 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e0048070 con 0x7f26f80d2b80 2024-04-20T11:35:26.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.759+0000 7f26f4c00640 1 -- 172.21.15.152:0/3863466276 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26e0030660 con 0x7f26f80d2b80 2024-04-20T11:35:26.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.759+0000 7f26fe400640 1 -- 172.21.15.152:0/3863466276 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26f81541f0 con 0x7f26f80d2b80 2024-04-20T11:35:26.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.759+0000 7f26f4c00640 1 -- 172.21.15.152:0/3863466276 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e0034040 con 0x7f26f80d2b80 2024-04-20T11:35:26.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.759+0000 7f26fe400640 1 -- 172.21.15.152:0/3863466276 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26f81547b0 con 0x7f26f80d2b80 2024-04-20T11:35:26.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.763+0000 7f26f4c00640 1 -- 172.21.15.152:0/3863466276 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 21) ==== 91550+0+0 (secure 0 0 0) 0x7f26e004b070 con 0x7f26f80d2b80 2024-04-20T11:35:26.764 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.764+0000 7f26f4c00640 1 --2- 172.21.15.152:0/3863466276 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f26d406ce90 0x7f26d406f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:26.764 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.764+0000 7f26f4c00640 1 -- 172.21.15.152:0/3863466276 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5709+0+0 (secure 0 0 0) 0x7f26e00ae880 con 0x7f26f80d2b80 2024-04-20T11:35:26.765 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.764+0000 7f2702671740 1 Processor -- start 2024-04-20T11:35:26.765 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.764+0000 7f2702671740 1 -- start start 2024-04-20T11:35:26.765 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.764+0000 7f26f7e00640 1 --2- 172.21.15.152:0/3863466276 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f26d406ce90 0x7f26d406f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:26.766 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.765+0000 7f2702671740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56494864e670 0x564948605ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:26.766 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.765+0000 7f2702671740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5649486046f0 0x5649486061e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:26.766 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.765+0000 7f2702671740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x564948600b90 0x564948630980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:26.766 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.765+0000 7f2702671740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x56494856c9d0 con 0x56494864e670 2024-04-20T11:35:26.766 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.765+0000 7f2702671740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x564948513270 con 0x5649486046f0 2024-04-20T11:35:26.766 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.765+0000 7f2702671740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x564948568990 con 0x564948600b90 2024-04-20T11:35:26.766 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.765+0000 7f26fd000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56494864e670 0x564948605ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:26.766 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.765+0000 7f26fd000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56494864e670 0x564948605ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54454/0 (socket says 172.21.15.152:54454) 2024-04-20T11:35:26.766 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.765+0000 7f26fd000640 1 -- 172.21.15.152:0/1261058872 learned_addr learned my addr 172.21.15.152:0/1261058872 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:26.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.766+0000 7f26f7e00640 1 --2- 172.21.15.152:0/3863466276 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f26d406ce90 0x7f26d406f350 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f26f80ce1f0 tx=0x7f26e8009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:26.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.766+0000 7f26fda00640 1 --2- 172.21.15.152:0/1261058872 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x564948600b90 0x564948630980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:26.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.766+0000 7f26fd000640 1 -- 172.21.15.152:0/1261058872 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x564948600b90 msgr2=0x564948630980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:26.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.766+0000 7f26fd000640 1 --2- 172.21.15.152:0/1261058872 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x564948600b90 0x564948630980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:26.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.766+0000 7f26fd000640 1 -- 172.21.15.152:0/1261058872 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5649486046f0 msgr2=0x5649486061e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:26.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.766+0000 7f26f7e00640 1 --2- 172.21.15.152:0/1261058872 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5649486046f0 0x5649486061e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:26.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.766+0000 7f26fd000640 1 --2- 172.21.15.152:0/1261058872 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5649486046f0 0x5649486061e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:26.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.766+0000 7f26fd000640 1 -- 172.21.15.152:0/1261058872 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56494827f2d0 con 0x56494864e670 2024-04-20T11:35:26.769 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.766+0000 7f26fd000640 1 --2- 172.21.15.152:0/1261058872 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56494864e670 0x564948605ca0 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f26e00842e0 tx=0x7f26e00308f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8525bad2cbbb709 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:26.769 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.766+0000 7f26ed600640 1 -- 172.21.15.152:0/1261058872 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e0082040 con 0x56494864e670 2024-04-20T11:35:26.769 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.766+0000 7f26ed600640 1 -- 172.21.15.152:0/1261058872 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26e007ffe0 con 0x56494864e670 2024-04-20T11:35:26.769 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.767+0000 7f26ed600640 1 -- 172.21.15.152:0/1261058872 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e0084810 con 0x56494864e670 2024-04-20T11:35:26.769 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.767+0000 7f2702671740 1 -- 172.21.15.152:0/1261058872 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56494864e670 msgr2=0x564948605ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:26.769 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.767+0000 7f2702671740 1 --2- 172.21.15.152:0/1261058872 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56494864e670 0x564948605ca0 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f26e00842e0 tx=0x7f26e00308f0 comp rx=0 tx=0).stop 2024-04-20T11:35:26.769 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.767+0000 7f2702671740 1 -- 172.21.15.152:0/1261058872 shutdown_connections 2024-04-20T11:35:26.769 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.767+0000 7f2702671740 1 --2- 172.21.15.152:0/1261058872 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x564948600b90 0x564948630980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:26.769 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.767+0000 7f2702671740 1 --2- 172.21.15.152:0/1261058872 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5649486046f0 0x5649486061e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:26.769 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.767+0000 7f2702671740 1 --2- 172.21.15.152:0/1261058872 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56494864e670 0x564948605ca0 unknown :-1 s=CLOSED pgs=1771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:26.769 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.767+0000 7f2702671740 1 -- 172.21.15.152:0/1261058872 >> 172.21.15.152:0/1261058872 conn(0x564948602e00 msgr2=0x56494864b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:26.769 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.767+0000 7f2702671740 1 -- 172.21.15.152:0/1261058872 shutdown_connections 2024-04-20T11:35:26.769 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.767+0000 7f2702671740 1 -- 172.21.15.152:0/1261058872 wait complete. 2024-04-20T11:35:26.772 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.768+0000 7f2702671740 1 -- start start 2024-04-20T11:35:26.773 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.768+0000 7f2702671740 1 Processor -- start 2024-04-20T11:35:26.773 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.769+0000 7f2702671740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5649486046f0 0x564948628570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:26.773 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.769+0000 7f2702671740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56494864e670 0x564948653570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:26.773 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.769+0000 7f2702671740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564948628df0 0x564948653ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:26.773 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.769+0000 7f2702671740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x56494856c9d0 con 0x5649486046f0 2024-04-20T11:35:26.773 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.769+0000 7f2702671740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5649485265e0 con 0x564948628df0 2024-04-20T11:35:26.773 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.769+0000 7f2702671740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x564948604c00 con 0x56494864e670 2024-04-20T11:35:26.773 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.769+0000 7f2702671740 10 client.0 _ll_register_callbacks cb 0x56494827eac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:35:26.773 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.769+0000 7f26fd000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5649486046f0 0x564948628570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:26.773 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.769+0000 7f26fd000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5649486046f0 0x564948628570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54462/0 (socket says 172.21.15.152:54462) 2024-04-20T11:35:26.773 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.770+0000 7f26fd000640 1 -- 172.21.15.152:0/1135717878 learned_addr learned my addr 172.21.15.152:0/1135717878 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:26.773 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.770+0000 7f26f7e00640 1 --2- 172.21.15.152:0/1135717878 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56494864e670 0x564948653570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:26.773 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.770+0000 7f26fd000640 1 -- 172.21.15.152:0/1135717878 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56494864e670 msgr2=0x564948653570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:26.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.770+0000 7f26fda00640 1 --2- 172.21.15.152:0/1135717878 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564948628df0 0x564948653ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:26.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.770+0000 7f26fd000640 1 --2- 172.21.15.152:0/1135717878 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56494864e670 0x564948653570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:26.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.770+0000 7f26fd000640 1 -- 172.21.15.152:0/1135717878 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564948628df0 msgr2=0x564948653ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:26.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.770+0000 7f26fd000640 1 --2- 172.21.15.152:0/1135717878 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564948628df0 0x564948653ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:26.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.770+0000 7f26fd000640 1 -- 172.21.15.152:0/1135717878 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649483496b0 con 0x5649486046f0 2024-04-20T11:35:26.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.770+0000 7f26f7e00640 1 --2- 172.21.15.152:0/1135717878 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56494864e670 0x564948653570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:35:26.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.770+0000 7f26fda00640 1 --2- 172.21.15.152:0/1135717878 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564948628df0 0x564948653ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:35:26.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.770+0000 7f26fd000640 1 --2- 172.21.15.152:0/1135717878 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5649486046f0 0x564948628570 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f26e007b930 tx=0x7f26e004e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c4020b97b49c758 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:26.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.770+0000 7f26e6000640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-04-20T11:35:26.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.770+0000 7f26e6000640 1 -- 172.21.15.152:0/1135717878 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e0082040 con 0x5649486046f0 2024-04-20T11:35:26.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.770+0000 7f26e6000640 1 -- 172.21.15.152:0/1135717878 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(131..131 src has 1..131) ==== 5709+0+0 (secure 0 0 0) 0x7f26e007b080 con 0x5649486046f0 2024-04-20T11:35:26.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.770+0000 7f2702671740 1 -- 172.21.15.152:0/1135717878 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26e007ffe0 con 0x5649486046f0 2024-04-20T11:35:26.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.771+0000 7f26e6000640 1 -- 172.21.15.152:0/1135717878 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f26e00ac120 con 0x5649486046f0 2024-04-20T11:35:26.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.771+0000 7f2702671740 10 client.9048 fetch_fsmap learned FSMap version 86 2024-04-20T11:35:26.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.771+0000 7f2702671740 1 -- 172.21.15.152:0/1135717878 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=2+}) -- 0x5649486527b0 con 0x5649486046f0 2024-04-20T11:35:26.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.771+0000 7f26e6000640 1 -- 172.21.15.152:0/1135717878 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26e007ffe0 con 0x5649486046f0 2024-04-20T11:35:26.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.771+0000 7f26e6000640 1 -- 172.21.15.152:0/1135717878 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 86) ==== 3495+0+0 (secure 0 0 0) 0x7f26e0042070 con 0x5649486046f0 2024-04-20T11:35:26.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.771+0000 7f2702671740 10 client.9048 fetch_fsmap finished waiting for FSMap version 86 2024-04-20T11:35:26.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.771+0000 7f2702671740 10 client.9048 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-20T11:35:26.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.771+0000 7f2702671740 20 client.9048 populate_metadata read hostname 'smithi152' 2024-04-20T11:35:26.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.771+0000 7f2702671740 1 --2- 172.21.15.152:0/1135717878 >> [v2:172.21.15.152:6833/1884827433,v1:172.21.15.152:6835/1884827433] conn(0x5649486501a0 0x564948650580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:26.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.771+0000 7f2702671740 4 client.9048 mds_command: new command op to 8919 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:35:26.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.771+0000 7f2702671740 1 -- 172.21.15.152:0/1135717878 --> [v2:172.21.15.152:6833/1884827433,v1:172.21.15.152:6835/1884827433] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56494857a3f0 con 0x5649486501a0 2024-04-20T11:35:26.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.772+0000 7f26f7e00640 1 --2- 172.21.15.152:0/1135717878 >> [v2:172.21.15.152:6833/1884827433,v1:172.21.15.152:6835/1884827433] conn(0x5649486501a0 0x564948650580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:26.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.772+0000 7f26f7e00640 1 --2- 172.21.15.152:0/1135717878 >> [v2:172.21.15.152:6833/1884827433,v1:172.21.15.152:6835/1884827433] conn(0x5649486501a0 0x564948650580 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e52ec794afaa6ba5 server_cookie=607578bffec3a4e in_seq=0 out_seq=0 2024-04-20T11:35:26.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.772+0000 7f26e6000640 10 client.9048 ms_handle_connect on v2:172.21.15.152:6833/1884827433 2024-04-20T11:35:26.777 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.773+0000 7f26e6000640 1 -- 172.21.15.152:0/1135717878 <== mds.0 v2:172.21.15.152:6833/1884827433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56494857a3f0 con 0x5649486501a0 2024-04-20T11:35:26.777 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.773+0000 7f26e6000640 10 client.9048 handle_command_reply: tid=0 2024-04-20T11:35:26.777 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.773+0000 7f2702671740 1 client.9048 shutdown 2024-04-20T11:35:26.777 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.773+0000 7f2702671740 1 -- 172.21.15.152:0/1135717878 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5649486046f0 msgr2=0x564948628570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:26.777 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.773+0000 7f2702671740 1 --2- 172.21.15.152:0/1135717878 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5649486046f0 0x564948628570 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f26e007b930 tx=0x7f26e004e000 comp rx=0 tx=0).stop 2024-04-20T11:35:26.777 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.773+0000 7f2702671740 1 -- 172.21.15.152:0/1135717878 shutdown_connections 2024-04-20T11:35:26.777 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.773+0000 7f2702671740 1 --2- 172.21.15.152:0/1135717878 >> [v2:172.21.15.152:6833/1884827433,v1:172.21.15.152:6835/1884827433] conn(0x5649486501a0 0x564948650580 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:26.777 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.773+0000 7f2702671740 1 --2- 172.21.15.152:0/1135717878 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564948628df0 0x564948653ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:26.777 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.773+0000 7f2702671740 1 --2- 172.21.15.152:0/1135717878 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56494864e670 0x564948653570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:26.777 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.773+0000 7f2702671740 1 --2- 172.21.15.152:0/1135717878 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5649486046f0 0x564948628570 unknown :-1 s=CLOSED pgs=1772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:26.777 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.773+0000 7f2702671740 1 -- 172.21.15.152:0/1135717878 >> 172.21.15.152:0/1135717878 conn(0x564948600b90 msgr2=0x5649486067b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:26.778 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.773+0000 7f26e6000640 0 client.9048 ms_handle_reset on v2:172.21.15.152:6833/1884827433 2024-04-20T11:35:26.778 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.774+0000 7f2702671740 1 -- 172.21.15.152:0/1135717878 shutdown_connections 2024-04-20T11:35:26.778 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.774+0000 7f2702671740 1 -- 172.21.15.152:0/1135717878 wait complete. 2024-04-20T11:35:26.778 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.774+0000 7f2702671740 20 client.9048 trim_cache size 0 max 16384 2024-04-20T11:35:26.795 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.795+0000 7f2702671740 1 Processor -- start 2024-04-20T11:35:26.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.795+0000 7f2702671740 1 -- start start 2024-04-20T11:35:26.797 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.796+0000 7f2702671740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5649486046f0 0x564948605810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:26.797 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.796+0000 7f2702671740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564948628df0 0x564948601160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:26.797 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.796+0000 7f2702671740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56494864e670 0x5649486067b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:26.797 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.796+0000 7f2702671740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f26f81ffdb0 con 0x564948628df0 2024-04-20T11:35:26.797 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.796+0000 7f2702671740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f26f8154c60 con 0x5649486046f0 2024-04-20T11:35:26.797 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.796+0000 7f2702671740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f26f81ff760 con 0x56494864e670 2024-04-20T11:35:26.797 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.796+0000 7f26f7e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564948628df0 0x564948601160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:26.797 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.796+0000 7f26f7e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564948628df0 0x564948601160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54464/0 (socket says 172.21.15.152:54464) 2024-04-20T11:35:26.797 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.796+0000 7f26f7e00640 1 -- 172.21.15.152:0/892702422 learned_addr learned my addr 172.21.15.152:0/892702422 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:26.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.797+0000 7f26fd000640 1 --2- 172.21.15.152:0/892702422 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5649486046f0 0x564948605810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:26.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.797+0000 7f26f7e00640 1 -- 172.21.15.152:0/892702422 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56494864e670 msgr2=0x5649486067b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:26.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.797+0000 7f26f7e00640 1 --2- 172.21.15.152:0/892702422 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56494864e670 0x5649486067b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:26.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.797+0000 7f26f7e00640 1 -- 172.21.15.152:0/892702422 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5649486046f0 msgr2=0x564948605810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:26.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.797+0000 7f26f7e00640 1 --2- 172.21.15.152:0/892702422 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5649486046f0 0x564948605810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:26.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.797+0000 7f26f7e00640 1 -- 172.21.15.152:0/892702422 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564948602560 con 0x564948628df0 2024-04-20T11:35:26.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.797+0000 7f26fd000640 1 --2- 172.21.15.152:0/892702422 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5649486046f0 0x564948605810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:35:26.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.797+0000 7f26f7e00640 1 --2- 172.21.15.152:0/892702422 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564948628df0 0x564948601160 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f26e8001620 tx=0x7f26e8011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da4f95de54b59e3d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:26.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.798+0000 7f26ecc00640 1 -- 172.21.15.152:0/892702422 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e80057d0 con 0x564948628df0 2024-04-20T11:35:26.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.798+0000 7f26ecc00640 1 -- 172.21.15.152:0/892702422 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26e8028070 con 0x564948628df0 2024-04-20T11:35:26.800 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.798+0000 7f26ecc00640 1 -- 172.21.15.152:0/892702422 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e8023290 con 0x564948628df0 2024-04-20T11:35:26.800 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.798+0000 7f2702671740 1 -- 172.21.15.152:0/892702422 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564948628df0 msgr2=0x564948601160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:26.800 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.798+0000 7f2702671740 1 --2- 172.21.15.152:0/892702422 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564948628df0 0x564948601160 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f26e8001620 tx=0x7f26e8011000 comp rx=0 tx=0).stop 2024-04-20T11:35:26.800 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.798+0000 7f2702671740 1 -- 172.21.15.152:0/892702422 shutdown_connections 2024-04-20T11:35:26.800 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.798+0000 7f2702671740 1 --2- 172.21.15.152:0/892702422 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56494864e670 0x5649486067b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:26.800 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.798+0000 7f2702671740 1 --2- 172.21.15.152:0/892702422 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564948628df0 0x564948601160 unknown :-1 s=CLOSED pgs=1773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:26.800 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.798+0000 7f2702671740 1 --2- 172.21.15.152:0/892702422 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5649486046f0 0x564948605810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:26.800 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.798+0000 7f2702671740 1 -- 172.21.15.152:0/892702422 >> 172.21.15.152:0/892702422 conn(0x564948600b90 msgr2=0x56494864ea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:26.800 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.798+0000 7f2702671740 1 -- 172.21.15.152:0/892702422 shutdown_connections 2024-04-20T11:35:26.800 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.799+0000 7f2702671740 1 -- 172.21.15.152:0/892702422 wait complete. 2024-04-20T11:35:26.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.799+0000 7f2702671740 1 -- start start 2024-04-20T11:35:26.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.799+0000 7f2702671740 1 Processor -- start 2024-04-20T11:35:26.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.800+0000 7f2702671740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5649486046f0 0x56494865e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:26.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.800+0000 7f2702671740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564948628df0 0x564948650580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:26.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.800+0000 7f2702671740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56494864e670 0x564948650ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:26.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.800+0000 7f2702671740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x56494856c9d0 con 0x564948628df0 2024-04-20T11:35:26.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.800+0000 7f2702671740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5649485265e0 con 0x5649486046f0 2024-04-20T11:35:26.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.800+0000 7f2702671740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x56494864f260 con 0x56494864e670 2024-04-20T11:35:26.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.800+0000 7f2702671740 10 client.0 _ll_register_callbacks cb 0x7f26e007b590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:35:26.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.801+0000 7f26f7e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564948628df0 0x564948650580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:26.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.801+0000 7f26f7e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564948628df0 0x564948650580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54474/0 (socket says 172.21.15.152:54474) 2024-04-20T11:35:26.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.801+0000 7f26f7e00640 1 -- 172.21.15.152:0/1858682935 learned_addr learned my addr 172.21.15.152:0/1858682935 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:26.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.801+0000 7f26f7e00640 1 -- 172.21.15.152:0/1858682935 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56494864e670 msgr2=0x564948650ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:26.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.801+0000 7f26f7e00640 1 --2- 172.21.15.152:0/1858682935 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56494864e670 0x564948650ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:26.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.801+0000 7f26f7e00640 1 -- 172.21.15.152:0/1858682935 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5649486046f0 msgr2=0x56494865e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:26.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.801+0000 7f26fd000640 1 --2- 172.21.15.152:0/1858682935 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5649486046f0 0x56494865e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:26.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.801+0000 7f26f7e00640 1 --2- 172.21.15.152:0/1858682935 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5649486046f0 0x56494865e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:26.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.801+0000 7f26f7e00640 1 -- 172.21.15.152:0/1858682935 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56494865eed0 con 0x564948628df0 2024-04-20T11:35:26.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.801+0000 7f26fd000640 1 --2- 172.21.15.152:0/1858682935 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5649486046f0 0x56494865e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:35:26.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.802+0000 7f26f7e00640 1 --2- 172.21.15.152:0/1858682935 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564948628df0 0x564948650580 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f26e8003d50 tx=0x7f26e8001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68f2ee62a63703de server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:26.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.802+0000 7f26e6000640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-04-20T11:35:26.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.802+0000 7f26e6000640 1 -- 172.21.15.152:0/1858682935 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e8023480 con 0x564948628df0 2024-04-20T11:35:26.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.802+0000 7f26e6000640 1 -- 172.21.15.152:0/1858682935 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(131..131 src has 1..131) ==== 5709+0+0 (secure 0 0 0) 0x7f26e8023620 con 0x564948628df0 2024-04-20T11:35:26.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.802+0000 7f2702671740 1 -- 172.21.15.152:0/1858682935 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26e8028070 con 0x564948628df0 2024-04-20T11:35:26.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.802+0000 7f26e6000640 1 -- 172.21.15.152:0/1858682935 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f26e8023820 con 0x564948628df0 2024-04-20T11:35:26.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.802+0000 7f2702671740 10 client.9054 fetch_fsmap learned FSMap version 86 2024-04-20T11:35:26.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.802+0000 7f2702671740 1 -- 172.21.15.152:0/1858682935 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=2+}) -- 0x564948642290 con 0x564948628df0 2024-04-20T11:35:26.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.802+0000 7f26e6000640 1 -- 172.21.15.152:0/1858682935 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26e8028070 con 0x564948628df0 2024-04-20T11:35:26.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.802+0000 7f26e6000640 1 -- 172.21.15.152:0/1858682935 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 86) ==== 3495+0+0 (secure 0 0 0) 0x7f26e8038470 con 0x564948628df0 2024-04-20T11:35:26.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.802+0000 7f2702671740 10 client.9054 fetch_fsmap finished waiting for FSMap version 86 2024-04-20T11:35:26.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.802+0000 7f2702671740 10 client.9054 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-20T11:35:26.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.802+0000 7f2702671740 20 client.9054 populate_metadata read hostname 'smithi152' 2024-04-20T11:35:26.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.802+0000 7f2702671740 1 --2- 172.21.15.152:0/1858682935 >> [v2:172.21.15.152:6833/1884827433,v1:172.21.15.152:6835/1884827433] conn(0x564948634430 0x5649486d29e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:26.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.803+0000 7f2702671740 4 client.9054 mds_command: new command op to 8919 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-20T11:35:26.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.803+0000 7f2702671740 1 -- 172.21.15.152:0/1858682935 --> [v2:172.21.15.152:6833/1884827433,v1:172.21.15.152:6835/1884827433] -- command(tid 0: {"prefix": "status"}) -- 0x564948574260 con 0x564948634430 2024-04-20T11:35:26.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.803+0000 7f26fd000640 1 --2- 172.21.15.152:0/1858682935 >> [v2:172.21.15.152:6833/1884827433,v1:172.21.15.152:6835/1884827433] conn(0x564948634430 0x5649486d29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:26.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.803+0000 7f26fd000640 1 --2- 172.21.15.152:0/1858682935 >> [v2:172.21.15.152:6833/1884827433,v1:172.21.15.152:6835/1884827433] conn(0x564948634430 0x5649486d29e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=416b1bdcb839f1f server_cookie=5509446252ad89cc in_seq=0 out_seq=0 2024-04-20T11:35:26.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.803+0000 7f26e6000640 10 client.9054 ms_handle_connect on v2:172.21.15.152:6833/1884827433 2024-04-20T11:35:26.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.804+0000 7f26e6000640 1 -- 172.21.15.152:0/1858682935 <== mds.0 v2:172.21.15.152:6833/1884827433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x564948574260 con 0x564948634430 2024-04-20T11:35:26.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.804+0000 7f26e6000640 10 client.9054 handle_command_reply: tid=0 2024-04-20T11:35:26.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.804+0000 7f2702671740 1 client.9054 shutdown 2024-04-20T11:35:26.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.804+0000 7f2702671740 1 -- 172.21.15.152:0/1858682935 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564948628df0 msgr2=0x564948650580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:26.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.804+0000 7f2702671740 1 --2- 172.21.15.152:0/1858682935 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564948628df0 0x564948650580 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f26e8003d50 tx=0x7f26e8001250 comp rx=0 tx=0).stop 2024-04-20T11:35:26.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.804+0000 7f2702671740 1 -- 172.21.15.152:0/1858682935 shutdown_connections 2024-04-20T11:35:26.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.804+0000 7f2702671740 1 --2- 172.21.15.152:0/1858682935 >> [v2:172.21.15.152:6833/1884827433,v1:172.21.15.152:6835/1884827433] conn(0x564948634430 0x5649486d29e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:26.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.804+0000 7f2702671740 1 --2- 172.21.15.152:0/1858682935 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56494864e670 0x564948650ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:26.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.804+0000 7f2702671740 1 --2- 172.21.15.152:0/1858682935 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564948628df0 0x564948650580 unknown :-1 s=CLOSED pgs=1774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:26.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.804+0000 7f2702671740 1 --2- 172.21.15.152:0/1858682935 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5649486046f0 0x56494865e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:26.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.804+0000 7f2702671740 1 -- 172.21.15.152:0/1858682935 >> 172.21.15.152:0/1858682935 conn(0x564948600b90 msgr2=0x564948601160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:26.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.804+0000 7f26e6000640 0 client.9054 ms_handle_reset on v2:172.21.15.152:6833/1884827433 2024-04-20T11:35:26.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.804+0000 7f2702671740 1 -- 172.21.15.152:0/1858682935 shutdown_connections 2024-04-20T11:35:26.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.804+0000 7f2702671740 1 -- 172.21.15.152:0/1858682935 wait complete. 2024-04-20T11:35:26.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.804+0000 7f2702671740 20 client.9054 trim_cache size 0 max 16384 2024-04-20T11:35:26.808 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-04-20T11:35:26.808 INFO:teuthology.orchestra.run.smithi152.stdout: "cluster_fsid": "c8a72131-0d4f-42c7-892f-3f3af4d2bd3d", 2024-04-20T11:35:26.808 INFO:teuthology.orchestra.run.smithi152.stdout: "whoami": 0, 2024-04-20T11:35:26.808 INFO:teuthology.orchestra.run.smithi152.stdout: "id": 8919, 2024-04-20T11:35:26.808 INFO:teuthology.orchestra.run.smithi152.stdout: "want_state": "up:active", 2024-04-20T11:35:26.808 INFO:teuthology.orchestra.run.smithi152.stdout: "state": "up:active", 2024-04-20T11:35:26.808 INFO:teuthology.orchestra.run.smithi152.stdout: "fs_name": "cephfs", 2024-04-20T11:35:26.808 INFO:teuthology.orchestra.run.smithi152.stdout: "rank_uptime": 6.7111119739999996, 2024-04-20T11:35:26.809 INFO:teuthology.orchestra.run.smithi152.stdout: "mdsmap_epoch": 86, 2024-04-20T11:35:26.809 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_epoch": 131, 2024-04-20T11:35:26.809 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_epoch_barrier": 131, 2024-04-20T11:35:26.809 INFO:teuthology.orchestra.run.smithi152.stdout: "uptime": 12.457623598 2024-04-20T11:35:26.809 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-04-20T11:35:26.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.805+0000 7f26fe400640 1 -- 172.21.15.152:0/3863466276 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f26d406ce90 msgr2=0x7f26d406f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:26.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.805+0000 7f26fe400640 1 --2- 172.21.15.152:0/3863466276 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f26d406ce90 0x7f26d406f350 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f26f80ce1f0 tx=0x7f26e8009040 comp rx=0 tx=0).stop 2024-04-20T11:35:26.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.805+0000 7f26fe400640 1 -- 172.21.15.152:0/3863466276 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f26f80d2b80 msgr2=0x7f26f81f46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:26.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.805+0000 7f26fe400640 1 --2- 172.21.15.152:0/3863466276 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f26f80d2b80 0x7f26f81f46e0 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f26e0000c00 tx=0x7f26e0004730 comp rx=0 tx=0).stop 2024-04-20T11:35:26.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.805+0000 7f26fe400640 1 -- 172.21.15.152:0/3863466276 shutdown_connections 2024-04-20T11:35:26.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.805+0000 7f26fe400640 1 --2- 172.21.15.152:0/3863466276 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f26d406ce90 0x7f26d406f350 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:26.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.805+0000 7f26fe400640 1 --2- 172.21.15.152:0/3863466276 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f26f8161d30 0x7f26f8153800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:26.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.805+0000 7f26fe400640 1 --2- 172.21.15.152:0/3863466276 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26f80d4ec0 0x7f26f81532c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:26.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.805+0000 7f26fe400640 1 --2- 172.21.15.152:0/3863466276 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f26f80d2b80 0x7f26f81f46e0 unknown :-1 s=CLOSED pgs=1770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:26.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.805+0000 7f26fe400640 1 -- 172.21.15.152:0/3863466276 >> 172.21.15.152:0/3863466276 conn(0x7f26f808bfd0 msgr2=0x7f26f806d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:26.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.805+0000 7f26fe400640 1 -- 172.21.15.152:0/3863466276 shutdown_connections 2024-04-20T11:35:26.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:26.805+0000 7f26fe400640 1 -- 172.21.15.152:0/3863466276 wait complete. 2024-04-20T11:35:26.822 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-20T11:35:27.004 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:27.003+0000 7fd64ee00640 1 Processor -- start 2024-04-20T11:35:27.004 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:27.003+0000 7fd64ee00640 1 -- start start 2024-04-20T11:35:27.005 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:27.005+0000 7fd64ee00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fd6480726f0 0x7fd6480cd580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:27.006 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:27.005+0000 7fd64ee00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd648146340 con 0x7fd648145f60 2024-04-20T11:35:27.006 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:27.005+0000 7fd64ee00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd648072d50 con 0x7fd6480cd0d0 2024-04-20T11:35:27.006 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:27.005+0000 7fd64ee00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fd64806e9f0 con 0x7fd6480726f0 2024-04-20T11:35:27.006 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:27.005+0000 7fd647e00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fd648145f60 0x7fd64806ee80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:56132/0 (socket says 172.21.15.152:56132) 2024-04-20T11:35:27.006 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:27.005+0000 7fd647e00640 1 -- 172.21.15.152:0/1404421314 learned_addr learned my addr 172.21.15.152:0/1404421314 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:35:27.006 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:27.005+0000 7fd64ce00640 1 --2- 172.21.15.152:0/1404421314 >> v2:172.21.15.152:3300/0 conn(0x7fd6480726f0 0x7fd6480cd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:27.007 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:27.006+0000 7fd64ce00640 1 -- 172.21.15.152:0/1404421314 >> v1:172.21.15.175:6789/0 conn(0x7fd6480cd0d0 legacy=0x7fd648071fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:27.007 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:27.006+0000 7fd64ce00640 1 -- 172.21.15.152:0/1404421314 >> v1:172.21.15.152:6789/0 conn(0x7fd648145f60 legacy=0x7fd64806ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:27.007 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:27.006+0000 7fd64ce00640 1 -- 172.21.15.152:0/1404421314 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd64806e0c0 con 0x7fd6480726f0 2024-04-20T11:35:27.007 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:27.007+0000 7fd64ce00640 1 --2- 172.21.15.152:0/1404421314 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd6480726f0 0x7fd6480cd580 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7fd6400097a0 tx=0x7fd64002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=281200174b40780c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:27.008 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:27.007+0000 7fd647400640 1 -- 172.21.15.152:0/1404421314 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd640035020 con 0x7fd6480726f0 2024-04-20T11:35:27.008 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:27.007+0000 7fd647400640 1 -- 172.21.15.152:0/1404421314 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd640031830 con 0x7fd6480726f0 2024-04-20T11:35:27.008 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:27.007+0000 7fd647400640 1 -- 172.21.15.152:0/1404421314 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd640031c00 con 0x7fd6480726f0 2024-04-20T11:35:27.009 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:27.007+0000 7fd64ee00640 1 -- 172.21.15.152:0/1404421314 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd6480726f0 msgr2=0x7fd6480cd580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:27.009 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:27.007+0000 7fd64ee00640 1 --2- 172.21.15.152:0/1404421314 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd6480726f0 0x7fd6480cd580 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7fd6400097a0 tx=0x7fd64002b8d0 comp rx=0 tx=0).stop 2024-04-20T11:35:27.009 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:27.008+0000 7fd64ee00640 1 -- 172.21.15.152:0/1404421314 shutdown_connections 2024-04-20T11:35:27.009 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:27.008+0000 7fd64ee00640 1 --2- 172.21.15.152:0/1404421314 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd6480726f0 0x7fd6480cd580 unknown :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:27.009 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:27.008+0000 7fd64ee00640 1 -- 172.21.15.152:0/1404421314 >> 172.21.15.152:0/1404421314 conn(0x7fd648067cf0 msgr2=0x7fd64806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:27.009 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:27.008+0000 7fd64ee00640 1 -- 172.21.15.152:0/1404421314 shutdown_connections 2024-04-20T11:35:27.009 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:27.008+0000 7fd64ee00640 1 -- 172.21.15.152:0/1404421314 wait complete. 2024-04-20T11:35:27.010 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:27.009+0000 7fd64ee00640 1 Processor -- start 2024-04-20T11:35:27.010 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:27.010+0000 7fd64ee00640 1 -- start start 2024-04-20T11:35:27.012 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:27.010+0000 7fd64ee00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd6480726f0 0x7fd648140830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:27.012 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:27.010+0000 7fd64ee00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd6480cd0d0 0x7fd6481f7070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:27.012 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:27.010+0000 7fd64ee00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd648145f60 0x7fd6481f75b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:27.012 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:27.010+0000 7fd64ee00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fd6480cf290 con 0x7fd6480726f0 2024-04-20T11:35:27.012 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:27.010+0000 7fd64ee00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd64806e680 con 0x7fd6480cd0d0 2024-04-20T11:35:27.012 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:27.010+0000 7fd64ee00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd6480cf410 con 0x7fd648145f60 2024-04-20T11:35:27.012 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:27.011+0000 7fd647e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd6480726f0 0x7fd648140830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:27.012 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:27.011+0000 7fd647e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd6480726f0 0x7fd648140830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54494/0 (socket says 172.21.15.152:54494) 2024-04-20T11:35:27.013 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:27.011+0000 7fd647e00640 1 -- 172.21.15.152:0/4046003987 learned_addr learned my addr 172.21.15.152:0/4046003987 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:27.013 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:27.011+0000 7fd63f400640 1 --2- 172.21.15.152:0/4046003987 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd6480cd0d0 0x7fd6481f7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:27.013 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:27.011+0000 7fd64ce00640 1 --2- 172.21.15.152:0/4046003987 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd648145f60 0x7fd6481f75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:27.013 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:27.011+0000 7fd647e00640 1 -- 172.21.15.152:0/4046003987 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd648145f60 msgr2=0x7fd6481f75b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:27.013 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:27.011+0000 7fd647e00640 1 --2- 172.21.15.152:0/4046003987 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd648145f60 0x7fd6481f75b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:27.013 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:27.011+0000 7fd647e00640 1 -- 172.21.15.152:0/4046003987 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd6480cd0d0 msgr2=0x7fd6481f7070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:27.013 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:27.011+0000 7fd647e00640 1 --2- 172.21.15.152:0/4046003987 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd6480cd0d0 0x7fd6481f7070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:27.013 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:27.011+0000 7fd647e00640 1 -- 172.21.15.152:0/4046003987 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6481f7af0 con 0x7fd6480726f0 2024-04-20T11:35:27.013 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:27.012+0000 7fd647e00640 1 --2- 172.21.15.152:0/4046003987 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd6480726f0 0x7fd648140830 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7fd63800ca70 tx=0x7fd63800cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:27.014 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:27.012+0000 7fd644c00640 1 -- 172.21.15.152:0/4046003987 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd638017070 con 0x7fd6480726f0 2024-04-20T11:35:27.014 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:27.012+0000 7fd644c00640 1 -- 172.21.15.152:0/4046003987 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd63800abd0 con 0x7fd6480726f0 2024-04-20T11:35:27.014 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:27.012+0000 7fd644c00640 1 -- 172.21.15.152:0/4046003987 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd638012400 con 0x7fd6480726f0 2024-04-20T11:35:27.014 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:27.012+0000 7fd64ee00640 1 -- 172.21.15.152:0/4046003987 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd64813df90 con 0x7fd6480726f0 2024-04-20T11:35:27.014 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:27.012+0000 7fd64ee00640 1 -- 172.21.15.152:0/4046003987 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6481412c0 con 0x7fd6480726f0 2024-04-20T11:35:27.014 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:27.013+0000 7fd64ee00640 1 -- 172.21.15.152:0/4046003987 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6480689f0 con 0x7fd6480726f0 2024-04-20T11:35:27.015 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:27.014+0000 7fd644c00640 1 -- 172.21.15.152:0/4046003987 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 21) ==== 91550+0+0 (secure 0 0 0) 0x7fd63801a070 con 0x7fd6480726f0 2024-04-20T11:35:27.015 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:27.015+0000 7fd644c00640 1 --2- 172.21.15.152:0/4046003987 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd62406ce90 0x7fd62406f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:27.016 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:27.015+0000 7fd644c00640 1 -- 172.21.15.152:0/4046003987 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5709+0+0 (secure 0 0 0) 0x7fd63808c2e0 con 0x7fd6480726f0 2024-04-20T11:35:27.016 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:27.016+0000 7fd63f400640 1 --2- 172.21.15.152:0/4046003987 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd62406ce90 0x7fd62406f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:27.017 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:27.017+0000 7fd63f400640 1 --2- 172.21.15.152:0/4046003987 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd62406ce90 0x7fd62406f350 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fd64813d030 tx=0x7fd630009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:27.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:27.020+0000 7fd644c00640 1 -- 172.21.15.152:0/4046003987 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fd638059ea0 con 0x7fd6480726f0 2024-04-20T11:35:27.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:27.260+0000 7fd64ee00640 1 -- 172.21.15.152:0/4046003987 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fd648159d30 con 0x7fd6480726f0 2024-04-20T11:35:27.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:27.261+0000 7fd644c00640 1 -- 172.21.15.152:0/4046003987 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v86) ==== 53+0+73 (secure 0 0 0) 0x7fd6380597a0 con 0x7fd6480726f0 2024-04-20T11:35:27.261 INFO:teuthology.orchestra.run.smithi152.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-20T11:35:27.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:27.263+0000 7fd64ee00640 1 -- 172.21.15.152:0/4046003987 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd62406ce90 msgr2=0x7fd62406f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:27.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:27.263+0000 7fd64ee00640 1 --2- 172.21.15.152:0/4046003987 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd62406ce90 0x7fd62406f350 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fd64813d030 tx=0x7fd630009040 comp rx=0 tx=0).stop 2024-04-20T11:35:27.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:27.263+0000 7fd64ee00640 1 -- 172.21.15.152:0/4046003987 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd6480726f0 msgr2=0x7fd648140830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:27.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:27.263+0000 7fd64ee00640 1 --2- 172.21.15.152:0/4046003987 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd6480726f0 0x7fd648140830 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7fd63800ca70 tx=0x7fd63800cf30 comp rx=0 tx=0).stop 2024-04-20T11:35:27.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:27.263+0000 7fd64ee00640 1 -- 172.21.15.152:0/4046003987 shutdown_connections 2024-04-20T11:35:27.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:27.263+0000 7fd64ee00640 1 --2- 172.21.15.152:0/4046003987 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd62406ce90 0x7fd62406f350 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:27.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:27.263+0000 7fd64ee00640 1 --2- 172.21.15.152:0/4046003987 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd648145f60 0x7fd6481f75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:27.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:27.263+0000 7fd64ee00640 1 --2- 172.21.15.152:0/4046003987 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd6480cd0d0 0x7fd6481f7070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:27.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:27.263+0000 7fd64ee00640 1 --2- 172.21.15.152:0/4046003987 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd6480726f0 0x7fd648140830 unknown :-1 s=CLOSED pgs=1777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:27.265 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:27.263+0000 7fd64ee00640 1 -- 172.21.15.152:0/4046003987 >> 172.21.15.152:0/4046003987 conn(0x7fd648067cf0 msgr2=0x7fd648144d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:27.265 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:27.263+0000 7fd64ee00640 1 -- 172.21.15.152:0/4046003987 shutdown_connections 2024-04-20T11:35:27.265 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:27.263+0000 7fd64ee00640 1 -- 172.21.15.152:0/4046003987 wait complete. 2024-04-20T11:35:27.283 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-20T11:35:27.283 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-20T11:35:27.283 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi152.front.sepia.ceph.com 2024-04-20T11:35:27.283 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-20T11:35:27.283 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-20T11:35:27.283 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-20T11:35:27.283 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-20T11:35:27.283 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-20T11:35:27.284 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-20T11:35:27.284 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:35:27.284 DEBUG:teuthology.orchestra.run.smithi152:> ip addr 2024-04-20T11:35:27.334 INFO:teuthology.orchestra.run.smithi152.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-20T11:35:27.334 INFO:teuthology.orchestra.run.smithi152.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-20T11:35:27.334 INFO:teuthology.orchestra.run.smithi152.stdout: inet 127.0.0.1/8 scope host lo 2024-04-20T11:35:27.334 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:35:27.334 INFO:teuthology.orchestra.run.smithi152.stdout: inet6 ::1/128 scope host 2024-04-20T11:35:27.334 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:35:27.335 INFO:teuthology.orchestra.run.smithi152.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T11:35:27.335 INFO:teuthology.orchestra.run.smithi152.stdout: link/ether 0c:c4:7a:da:6c:44 brd ff:ff:ff:ff:ff:ff 2024-04-20T11:35:27.335 INFO:teuthology.orchestra.run.smithi152.stdout: altname enp7s0f0 2024-04-20T11:35:27.335 INFO:teuthology.orchestra.run.smithi152.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T11:35:27.335 INFO:teuthology.orchestra.run.smithi152.stdout: link/ether 0c:c4:7a:da:6c:45 brd ff:ff:ff:ff:ff:ff 2024-04-20T11:35:27.335 INFO:teuthology.orchestra.run.smithi152.stdout: altname enp7s0f1 2024-04-20T11:35:27.335 INFO:teuthology.orchestra.run.smithi152.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T11:35:27.335 INFO:teuthology.orchestra.run.smithi152.stdout: link/ether 0c:c4:7a:88:6d:e2 brd ff:ff:ff:ff:ff:ff 2024-04-20T11:35:27.335 INFO:teuthology.orchestra.run.smithi152.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-20T11:35:27.335 INFO:teuthology.orchestra.run.smithi152.stdout: link/ether 0c:c4:7a:88:6d:e3 brd ff:ff:ff:ff:ff:ff 2024-04-20T11:35:27.335 INFO:teuthology.orchestra.run.smithi152.stdout: inet 172.21.15.152/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-20T11:35:27.336 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft 42279sec preferred_lft 42279sec 2024-04-20T11:35:27.336 INFO:teuthology.orchestra.run.smithi152.stdout: inet6 fe80::ec4:7aff:fe88:6de3/64 scope link 2024-04-20T11:35:27.336 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:35:27.336 INFO:teuthology.orchestra.run.smithi152.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-20T11:35:27.336 INFO:teuthology.orchestra.run.smithi152.stdout: link/ether 06:3e:9a:d3:49:1e brd ff:ff:ff:ff:ff:ff 2024-04-20T11:35:27.336 INFO:teuthology.orchestra.run.smithi152.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-20T11:35:27.336 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:35:27.336 INFO:teuthology.orchestra.run.smithi152.stdout: inet6 fe80::a019:d8ff:fe70:1185/64 scope link 2024-04-20T11:35:27.336 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:35:27.336 INFO:teuthology.orchestra.run.smithi152.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-20T11:35:27.336 INFO:teuthology.orchestra.run.smithi152.stdout: link/ether 16:e5:08:78:20:47 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T11:35:27.336 INFO:teuthology.orchestra.run.smithi152.stdout: inet6 fe80::14e5:8ff:fe78:2047/64 scope link 2024-04-20T11:35:27.337 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:35:27.337 INFO:teuthology.orchestra.run.smithi152.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-20T11:35:27.337 INFO:teuthology.orchestra.run.smithi152.stdout: link/ether 06:3e:9a:d3:49:1e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T11:35:27.337 INFO:teuthology.orchestra.run.smithi152.stdout: inet6 fe80::43e:9aff:fed3:491e/64 scope link 2024-04-20T11:35:27.337 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:35:27.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:35:27.337 DEBUG:teuthology.orchestra.run.smithi152:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-20T11:35:27.406 INFO:teuthology.orchestra.run.smithi152.stdout:1 2024-04-20T11:35:27.410 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:35:27.410 DEBUG:teuthology.orchestra.run.smithi152:> route 2024-04-20T11:35:27.458 INFO:teuthology.orchestra.run.smithi152.stdout:Kernel IP routing table 2024-04-20T11:35:27.458 INFO:teuthology.orchestra.run.smithi152.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-20T11:35:27.458 INFO:teuthology.orchestra.run.smithi152.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-20T11:35:27.458 INFO:teuthology.orchestra.run.smithi152.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-20T11:35:27.459 INFO:teuthology.orchestra.run.smithi152.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-20T11:35:27.459 DEBUG:teuthology.orchestra.run.smithi152:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T11:35:27.459 DEBUG:teuthology.orchestra.run.smithi152:> set -e 2024-04-20T11:35:27.459 DEBUG:teuthology.orchestra.run.smithi152:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-20T11:35:27.459 DEBUG:teuthology.orchestra.run.smithi152:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-20T11:35:27.459 DEBUG:teuthology.orchestra.run.smithi152:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-20T11:35:27.459 DEBUG:teuthology.orchestra.run.smithi152:> ') 2024-04-20T11:35:27.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:27 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T11:35:27.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T11:35:27.676 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:35:27.676 DEBUG:teuthology.orchestra.run.smithi152:> ip netns list 2024-04-20T11:35:27.730 INFO:teuthology.orchestra.run.smithi152.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-20T11:35:27.730 INFO:teuthology.orchestra.run.smithi152.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-20T11:35:27.731 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-20T11:35:27.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:35:27.731 DEBUG:teuthology.orchestra.run.smithi152:> sudo ip link set brx.0 up 2024-04-20T11:35:27.859 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-20T11:35:27.859 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T11:35:27.859 DEBUG:teuthology.orchestra.run.smithi152:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-20T11:35:27.907 INFO:teuthology.orchestra.run.smithi152.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-20T11:35:27.907 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T11:35:27.907 DEBUG:teuthology.orchestra.run.smithi152:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-20T11:35:28.007 DEBUG:teuthology.orchestra.run.smithi152:> sudo modprobe fuse 2024-04-20T11:35:28.096 DEBUG:teuthology.orchestra.run.smithi152:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T11:35:28.153 INFO:teuthology.orchestra.run.smithi152.stdout:/proc 2024-04-20T11:35:28.153 INFO:teuthology.orchestra.run.smithi152.stdout:/sys 2024-04-20T11:35:28.154 INFO:teuthology.orchestra.run.smithi152.stdout:/dev 2024-04-20T11:35:28.154 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/security 2024-04-20T11:35:28.154 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/shm 2024-04-20T11:35:28.154 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/pts 2024-04-20T11:35:28.154 INFO:teuthology.orchestra.run.smithi152.stdout:/run 2024-04-20T11:35:28.154 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/cgroup 2024-04-20T11:35:28.154 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/pstore 2024-04-20T11:35:28.154 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/bpf 2024-04-20T11:35:28.154 INFO:teuthology.orchestra.run.smithi152.stdout:/ 2024-04-20T11:35:28.154 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/selinux 2024-04-20T11:35:28.154 INFO:teuthology.orchestra.run.smithi152.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T11:35:28.154 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/mqueue 2024-04-20T11:35:28.154 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/hugepages 2024-04-20T11:35:28.155 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/debug 2024-04-20T11:35:28.155 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/tracing 2024-04-20T11:35:28.155 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/fuse/connections 2024-04-20T11:35:28.155 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/config 2024-04-20T11:35:28.155 INFO:teuthology.orchestra.run.smithi152.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T11:35:28.155 INFO:teuthology.orchestra.run.smithi152.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T11:35:28.155 INFO:teuthology.orchestra.run.smithi152.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T11:35:28.155 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T11:35:28.155 INFO:teuthology.orchestra.run.smithi152.stdout:/run/user/1000 2024-04-20T11:35:28.155 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph 2024-04-20T11:35:28.155 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph/osd/ceph-0 2024-04-20T11:35:28.155 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph/osd/ceph-1 2024-04-20T11:35:28.155 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph/osd/ceph-2 2024-04-20T11:35:28.155 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph/osd/ceph-3 2024-04-20T11:35:28.156 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns 2024-04-20T11:35:28.156 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T11:35:28.156 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T11:35:28.156 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T11:35:28.156 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T11:35:28.157 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:35:28.157 DEBUG:teuthology.orchestra.run.smithi152:> ls /sys/fs/fuse/connections 2024-04-20T11:35:28.207 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-20T11:35:28.207 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-20T11:35:28.213 DEBUG:teuthology.orchestra.run.smithi152:> sudo modprobe fuse 2024-04-20T11:35:28.280 DEBUG:teuthology.orchestra.run.smithi152:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T11:35:28.321 INFO:teuthology.orchestra.run.smithi152.stdout:/proc 2024-04-20T11:35:28.321 INFO:teuthology.orchestra.run.smithi152.stdout:/sys 2024-04-20T11:35:28.321 INFO:teuthology.orchestra.run.smithi152.stdout:/dev 2024-04-20T11:35:28.321 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/security 2024-04-20T11:35:28.321 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/shm 2024-04-20T11:35:28.321 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/pts 2024-04-20T11:35:28.321 INFO:teuthology.orchestra.run.smithi152.stdout:/run 2024-04-20T11:35:28.321 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/cgroup 2024-04-20T11:35:28.321 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/pstore 2024-04-20T11:35:28.321 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/bpf 2024-04-20T11:35:28.321 INFO:teuthology.orchestra.run.smithi152.stdout:/ 2024-04-20T11:35:28.321 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/selinux 2024-04-20T11:35:28.322 INFO:teuthology.orchestra.run.smithi152.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T11:35:28.322 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/mqueue 2024-04-20T11:35:28.322 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/hugepages 2024-04-20T11:35:28.322 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/debug 2024-04-20T11:35:28.322 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/tracing 2024-04-20T11:35:28.322 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/fuse/connections 2024-04-20T11:35:28.322 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/config 2024-04-20T11:35:28.322 INFO:teuthology.orchestra.run.smithi152.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T11:35:28.322 INFO:teuthology.orchestra.run.smithi152.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T11:35:28.322 INFO:teuthology.orchestra.run.smithi152.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T11:35:28.322 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T11:35:28.322 INFO:teuthology.orchestra.run.smithi152.stdout:/run/user/1000 2024-04-20T11:35:28.322 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph 2024-04-20T11:35:28.322 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph/osd/ceph-0 2024-04-20T11:35:28.323 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph/osd/ceph-1 2024-04-20T11:35:28.323 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph/osd/ceph-2 2024-04-20T11:35:28.323 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph/osd/ceph-3 2024-04-20T11:35:28.323 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns 2024-04-20T11:35:28.323 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T11:35:28.323 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T11:35:28.323 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T11:35:28.323 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T11:35:28.324 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:35:28.324 DEBUG:teuthology.orchestra.run.smithi152:> ls /sys/fs/fuse/connections 2024-04-20T11:35:28.381 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi152.stderr:2024-04-20T11:35:28.381+0000 7ff7d6837480 -1 init, newargv = 0x5595d6987be0 newargc=15 2024-04-20T11:35:28.381 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi152.stderr:ceph-fuse[64036]: starting ceph client 2024-04-20T11:35:28.389 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi152.stderr:ceph-fuse[64036]: starting fuse 2024-04-20T11:35:28.408 INFO:teuthology.orchestra.run.smithi152.stdout:42 2024-04-20T11:35:28.409 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-04-20T11:35:28.409 DEBUG:teuthology.orchestra.run.smithi152:> sudo stdin-killer -- python3 -c ' 2024-04-20T11:35:28.409 DEBUG:teuthology.orchestra.run.smithi152:> import glob 2024-04-20T11:35:28.409 DEBUG:teuthology.orchestra.run.smithi152:> import re 2024-04-20T11:35:28.409 DEBUG:teuthology.orchestra.run.smithi152:> import os 2024-04-20T11:35:28.409 DEBUG:teuthology.orchestra.run.smithi152:> import subprocess 2024-04-20T11:35:28.409 DEBUG:teuthology.orchestra.run.smithi152:> 2024-04-20T11:35:28.409 DEBUG:teuthology.orchestra.run.smithi152:> def _find_admin_socket(client_name): 2024-04-20T11:35:28.409 DEBUG:teuthology.orchestra.run.smithi152:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-20T11:35:28.409 DEBUG:teuthology.orchestra.run.smithi152:> files = glob.glob(asok_path) 2024-04-20T11:35:28.410 DEBUG:teuthology.orchestra.run.smithi152:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-20T11:35:28.410 DEBUG:teuthology.orchestra.run.smithi152:> 2024-04-20T11:35:28.410 DEBUG:teuthology.orchestra.run.smithi152:> # Given a non-glob path, it better be there 2024-04-20T11:35:28.410 DEBUG:teuthology.orchestra.run.smithi152:> if "*" not in asok_path: 2024-04-20T11:35:28.410 DEBUG:teuthology.orchestra.run.smithi152:> assert(len(files) == 1) 2024-04-20T11:35:28.410 DEBUG:teuthology.orchestra.run.smithi152:> return files[0] 2024-04-20T11:35:28.410 DEBUG:teuthology.orchestra.run.smithi152:> 2024-04-20T11:35:28.410 DEBUG:teuthology.orchestra.run.smithi152:> for f in files: 2024-04-20T11:35:28.410 DEBUG:teuthology.orchestra.run.smithi152:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-20T11:35:28.410 DEBUG:teuthology.orchestra.run.smithi152:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-20T11:35:28.411 DEBUG:teuthology.orchestra.run.smithi152:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-20T11:35:28.411 DEBUG:teuthology.orchestra.run.smithi152:> contents = proc_f.read() 2024-04-20T11:35:28.411 DEBUG:teuthology.orchestra.run.smithi152:> if mountpoint in contents: 2024-04-20T11:35:28.411 DEBUG:teuthology.orchestra.run.smithi152:> return f 2024-04-20T11:35:28.411 DEBUG:teuthology.orchestra.run.smithi152:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-20T11:35:28.411 DEBUG:teuthology.orchestra.run.smithi152:> 2024-04-20T11:35:28.411 DEBUG:teuthology.orchestra.run.smithi152:> print(_find_admin_socket("client.0")) 2024-04-20T11:35:28.411 DEBUG:teuthology.orchestra.run.smithi152:> ' 2024-04-20T11:35:28.597 INFO:teuthology.orchestra.run.smithi152.stdout:/var/run/ceph/ceph-client.0.64036.asok 2024-04-20T11:35:28.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T11:35:28.615 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.64036.asok 2024-04-20T11:35:28.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:35:28.615 DEBUG:teuthology.orchestra.run.smithi152:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.64036.asok status 2024-04-20T11:35:28.831 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-04-20T11:35:28.831 INFO:teuthology.orchestra.run.smithi152.stdout: "metadata": { 2024-04-20T11:35:28.831 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_sha1": "a8c84f5997950fd7d008204839253291d52e6eda", 2024-04-20T11:35:28.831 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", 2024-04-20T11:35:28.831 INFO:teuthology.orchestra.run.smithi152.stdout: "entity_id": "0", 2024-04-20T11:35:28.832 INFO:teuthology.orchestra.run.smithi152.stdout: "hostname": "smithi152", 2024-04-20T11:35:28.832 INFO:teuthology.orchestra.run.smithi152.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-20T11:35:28.832 INFO:teuthology.orchestra.run.smithi152.stdout: "pid": "64036", 2024-04-20T11:35:28.832 INFO:teuthology.orchestra.run.smithi152.stdout: "root": "/" 2024-04-20T11:35:28.832 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:35:28.832 INFO:teuthology.orchestra.run.smithi152.stdout: "dentry_count": 0, 2024-04-20T11:35:28.832 INFO:teuthology.orchestra.run.smithi152.stdout: "dentry_pinned_count": 0, 2024-04-20T11:35:28.832 INFO:teuthology.orchestra.run.smithi152.stdout: "id": 9069, 2024-04-20T11:35:28.832 INFO:teuthology.orchestra.run.smithi152.stdout: "inst": { 2024-04-20T11:35:28.832 INFO:teuthology.orchestra.run.smithi152.stdout: "name": { 2024-04-20T11:35:28.832 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "client", 2024-04-20T11:35:28.832 INFO:teuthology.orchestra.run.smithi152.stdout: "num": 9069 2024-04-20T11:35:28.832 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:35:28.833 INFO:teuthology.orchestra.run.smithi152.stdout: "addr": { 2024-04-20T11:35:28.833 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "v1", 2024-04-20T11:35:28.833 INFO:teuthology.orchestra.run.smithi152.stdout: "addr": "172.21.15.152:0", 2024-04-20T11:35:28.833 INFO:teuthology.orchestra.run.smithi152.stdout: "nonce": 3716053811 2024-04-20T11:35:28.833 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:35:28.833 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:35:28.833 INFO:teuthology.orchestra.run.smithi152.stdout: "addr": { 2024-04-20T11:35:28.833 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "v1", 2024-04-20T11:35:28.833 INFO:teuthology.orchestra.run.smithi152.stdout: "addr": "172.21.15.152:0", 2024-04-20T11:35:28.833 INFO:teuthology.orchestra.run.smithi152.stdout: "nonce": 3716053811 2024-04-20T11:35:28.833 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:35:28.833 INFO:teuthology.orchestra.run.smithi152.stdout: "inst_str": "client.9069 172.21.15.152:0/3716053811", 2024-04-20T11:35:28.833 INFO:teuthology.orchestra.run.smithi152.stdout: "addr_str": "172.21.15.152:0/3716053811", 2024-04-20T11:35:28.833 INFO:teuthology.orchestra.run.smithi152.stdout: "inode_count": 1, 2024-04-20T11:35:28.834 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_epoch": 86, 2024-04-20T11:35:28.834 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_epoch": 131, 2024-04-20T11:35:28.834 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_epoch_barrier": 0, 2024-04-20T11:35:28.834 INFO:teuthology.orchestra.run.smithi152.stdout: "blocklisted": false, 2024-04-20T11:35:28.834 INFO:teuthology.orchestra.run.smithi152.stdout: "fs_name": "cephfs" 2024-04-20T11:35:28.834 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-04-20T11:35:28.846 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:35:28.846 DEBUG:teuthology.orchestra.run.smithi152:> stat --file-system '--printf=%T 2024-04-20T11:35:28.846 DEBUG:teuthology.orchestra.run.smithi152:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-20T11:35:28.901 INFO:teuthology.orchestra.run.smithi152.stdout:fuseblk 2024-04-20T11:35:28.901 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-20T11:35:28.901 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:35:28.902 DEBUG:teuthology.orchestra.run.smithi152:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-20T11:35:28.989 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-20T11:35:29.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:29.196+0000 7fa55b200640 1 Processor -- start 2024-04-20T11:35:29.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:29.196+0000 7fa55b200640 1 -- start start 2024-04-20T11:35:29.199 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:29.198+0000 7fa55b200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fa55c15d1e0 0x7fa55c160b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:29.199 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:29.198+0000 7fa55b200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa55c15d5c0 0x7fa55c0cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:29.199 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:29.198+0000 7fa55b200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fa55c071cd0 0x7fa55c072160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:29.199 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:29.198+0000 7fa55b200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fa55c0726a0 con 0x7fa55c15d1e0 2024-04-20T11:35:29.199 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:29.198+0000 7fa55b200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fa55c0ce8c0 con 0x7fa55c071cd0 2024-04-20T11:35:29.199 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:29.198+0000 7fa55b200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa55c1600b0 con 0x7fa55c15d5c0 2024-04-20T11:35:29.199 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:29.198+0000 7fa559e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fa55c15d1e0 0x7fa55c160b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:29.199 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:29.198+0000 7fa559e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fa55c15d1e0 0x7fa55c160b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54498/0 (socket says 172.21.15.152:54498) 2024-04-20T11:35:29.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:29.198+0000 7fa559e00640 1 -- 172.21.15.152:0/2933678694 learned_addr learned my addr 172.21.15.152:0/2933678694 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:29.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:29.198+0000 7fa55a800640 1 --2- 172.21.15.152:0/2933678694 >> v2:172.21.15.175:3300/0 conn(0x7fa55c071cd0 0x7fa55c072160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:29.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:29.198+0000 7fa559e00640 1 -- 172.21.15.152:0/2933678694 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa55c15d5c0 msgr2=0x7fa55c0cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:29.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:29.198+0000 7fa559e00640 1 --2- 172.21.15.152:0/2933678694 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa55c15d5c0 0x7fa55c0cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:29.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:29.198+0000 7fa559e00640 1 -- 172.21.15.152:0/2933678694 >> v2:172.21.15.175:3300/0 conn(0x7fa55c071cd0 msgr2=0x7fa55c072160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:29.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:29.198+0000 7fa559e00640 1 --2- 172.21.15.152:0/2933678694 >> v2:172.21.15.175:3300/0 conn(0x7fa55c071cd0 0x7fa55c072160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:29.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:29.198+0000 7fa559e00640 1 -- 172.21.15.152:0/2933678694 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa55c160390 con 0x7fa55c15d1e0 2024-04-20T11:35:29.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:29.198+0000 7fa55a800640 1 --2- 172.21.15.152:0/2933678694 >> v2:172.21.15.175:3300/0 conn(0x7fa55c071cd0 0x7fa55c072160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:35:29.201 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:29.199+0000 7fa559e00640 1 --2- 172.21.15.152:0/2933678694 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa55c15d1e0 0x7fa55c160b90 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7fa55c06c4f0 tx=0x7fa54c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2faed732c0a9c04e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:29.201 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:29.200+0000 7fa558a00640 1 -- 172.21.15.152:0/2933678694 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa54c02ca70 con 0x7fa55c15d1e0 2024-04-20T11:35:29.201 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:29.200+0000 7fa558a00640 1 -- 172.21.15.152:0/2933678694 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa54c02cc10 con 0x7fa55c15d1e0 2024-04-20T11:35:29.201 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:29.200+0000 7fa558a00640 1 -- 172.21.15.152:0/2933678694 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa54c0343f0 con 0x7fa55c15d1e0 2024-04-20T11:35:29.201 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:29.202+0000 7fa55b200640 1 -- 172.21.15.152:0/2933678694 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa55c15d1e0 msgr2=0x7fa55c160b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:29.201 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:29.202+0000 7fa55b200640 1 --2- 172.21.15.152:0/2933678694 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa55c15d1e0 0x7fa55c160b90 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7fa55c06c4f0 tx=0x7fa54c02bed0 comp rx=0 tx=0).stop 2024-04-20T11:35:29.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:29.202+0000 7fa55b200640 1 -- 172.21.15.152:0/2933678694 shutdown_connections 2024-04-20T11:35:29.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:29.202+0000 7fa55b200640 1 --2- 172.21.15.152:0/2933678694 >> v2:172.21.15.175:3300/0 conn(0x7fa55c071cd0 0x7fa55c072160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:29.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:29.202+0000 7fa55b200640 1 --2- 172.21.15.152:0/2933678694 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa55c15d5c0 0x7fa55c0cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:29.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:29.202+0000 7fa55b200640 1 --2- 172.21.15.152:0/2933678694 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa55c15d1e0 0x7fa55c160b90 unknown :-1 s=CLOSED pgs=1781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:29.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:29.202+0000 7fa55b200640 1 -- 172.21.15.152:0/2933678694 >> 172.21.15.152:0/2933678694 conn(0x7fa55c067ae0 msgr2=0x7fa55c06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:29.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:29.202+0000 7fa55b200640 1 -- 172.21.15.152:0/2933678694 shutdown_connections 2024-04-20T11:35:29.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:29.202+0000 7fa55b200640 1 -- 172.21.15.152:0/2933678694 wait complete. 2024-04-20T11:35:29.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:29.203+0000 7fa55b200640 1 Processor -- start 2024-04-20T11:35:29.204 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:29.203+0000 7fa55b200640 1 -- start start 2024-04-20T11:35:29.205 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:29.204+0000 7fa55b200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa55c071cd0 0x7fa55c157230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:29.205 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:29.205+0000 7fa55b200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa55c15d1e0 0x7fa55c144c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:29.205 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:29.205+0000 7fa55b200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa55c15d5c0 0x7fa55c1474c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:29.205 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:29.205+0000 7fa55b200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fa55c0ce520 con 0x7fa55c15d5c0 2024-04-20T11:35:29.205 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:29.205+0000 7fa55b200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa55c0ceba0 con 0x7fa55c071cd0 2024-04-20T11:35:29.206 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:29.205+0000 7fa55b200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa55c0ce6a0 con 0x7fa55c15d1e0 2024-04-20T11:35:29.206 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:29.205+0000 7fa55a800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa55c15d5c0 0x7fa55c1474c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:29.206 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:29.205+0000 7fa55a800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa55c15d5c0 0x7fa55c1474c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54506/0 (socket says 172.21.15.152:54506) 2024-04-20T11:35:29.206 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:29.205+0000 7fa55a800640 1 -- 172.21.15.152:0/2277872667 learned_addr learned my addr 172.21.15.152:0/2277872667 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:29.207 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:29.205+0000 7fa559e00640 1 --2- 172.21.15.152:0/2277872667 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa55c071cd0 0x7fa55c157230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:29.207 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:29.205+0000 7fa55a800640 1 -- 172.21.15.152:0/2277872667 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa55c15d1e0 msgr2=0x7fa55c144c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:29.207 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:29.205+0000 7fa55a800640 1 --2- 172.21.15.152:0/2277872667 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa55c15d1e0 0x7fa55c144c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:29.207 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:29.205+0000 7fa55a800640 1 -- 172.21.15.152:0/2277872667 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa55c071cd0 msgr2=0x7fa55c157230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:29.207 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:29.205+0000 7fa55a800640 1 --2- 172.21.15.152:0/2277872667 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa55c071cd0 0x7fa55c157230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:29.207 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:29.205+0000 7fa55a800640 1 -- 172.21.15.152:0/2277872667 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa55c145150 con 0x7fa55c15d5c0 2024-04-20T11:35:29.207 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:29.205+0000 7fa559e00640 1 --2- 172.21.15.152:0/2277872667 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa55c071cd0 0x7fa55c157230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:35:29.207 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:29.206+0000 7fa55a800640 1 --2- 172.21.15.152:0/2277872667 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa55c15d5c0 0x7fa55c1474c0 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7fa55400c7f0 tx=0x7fa55400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:29.207 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:29.206+0000 7fa552000640 1 -- 172.21.15.152:0/2277872667 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa554015070 con 0x7fa55c15d5c0 2024-04-20T11:35:29.208 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:29.206+0000 7fa552000640 1 -- 172.21.15.152:0/2277872667 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa554007820 con 0x7fa55c15d5c0 2024-04-20T11:35:29.208 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:29.206+0000 7fa552000640 1 -- 172.21.15.152:0/2277872667 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa554007c50 con 0x7fa55c15d5c0 2024-04-20T11:35:29.208 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:29.206+0000 7fa55b200640 1 -- 172.21.15.152:0/2277872667 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa55c0af9c0 con 0x7fa55c15d5c0 2024-04-20T11:35:29.208 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:29.206+0000 7fa55b200640 1 -- 172.21.15.152:0/2277872667 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa55c0b38a0 con 0x7fa55c15d5c0 2024-04-20T11:35:29.208 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:29.207+0000 7fa55b200640 1 -- 172.21.15.152:0/2277872667 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa55c0cfa10 con 0x7fa55c15d5c0 2024-04-20T11:35:29.209 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:29.208+0000 7fa552000640 1 -- 172.21.15.152:0/2277872667 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 21) ==== 91550+0+0 (secure 0 0 0) 0x7fa554018070 con 0x7fa55c15d5c0 2024-04-20T11:35:29.210 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:29.209+0000 7fa552000640 1 --2- 172.21.15.152:0/2277872667 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fa53c06cec0 0x7fa53c06f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:29.210 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:29.209+0000 7fa552000640 1 -- 172.21.15.152:0/2277872667 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5709+0+0 (secure 0 0 0) 0x7fa55408c700 con 0x7fa55c15d5c0 2024-04-20T11:35:29.210 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:29.210+0000 7fa559e00640 1 --2- 172.21.15.152:0/2277872667 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fa53c06cec0 0x7fa53c06f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:29.212 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:29.211+0000 7fa559e00640 1 --2- 172.21.15.152:0/2277872667 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fa53c06cec0 0x7fa53c06f380 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fa54c035000 tx=0x7fa54c030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:29.215 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:29.214+0000 7fa552000640 1 -- 172.21.15.152:0/2277872667 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fa55c0cfa10 con 0x7fa55c15d5c0 2024-04-20T11:35:29.463 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:29.462+0000 7fa55b200640 1 -- 172.21.15.152:0/2277872667 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fa55c075e60 con 0x7fa55c15d5c0 2024-04-20T11:35:29.463 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:29.463+0000 7fa552000640 1 -- 172.21.15.152:0/2277872667 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v86) ==== 53+0+73 (secure 0 0 0) 0x7fa55c145150 con 0x7fa55c15d5c0 2024-04-20T11:35:29.464 INFO:teuthology.orchestra.run.smithi152.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-20T11:35:29.466 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:29.466+0000 7fa55b200640 1 -- 172.21.15.152:0/2277872667 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fa53c06cec0 msgr2=0x7fa53c06f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:29.467 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:29.466+0000 7fa55b200640 1 --2- 172.21.15.152:0/2277872667 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fa53c06cec0 0x7fa53c06f380 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fa54c035000 tx=0x7fa54c030000 comp rx=0 tx=0).stop 2024-04-20T11:35:29.467 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:29.466+0000 7fa55b200640 1 -- 172.21.15.152:0/2277872667 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa55c15d5c0 msgr2=0x7fa55c1474c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:29.467 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:29.466+0000 7fa55b200640 1 --2- 172.21.15.152:0/2277872667 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa55c15d5c0 0x7fa55c1474c0 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7fa55400c7f0 tx=0x7fa55400ccb0 comp rx=0 tx=0).stop 2024-04-20T11:35:29.467 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:29.466+0000 7fa55b200640 1 -- 172.21.15.152:0/2277872667 shutdown_connections 2024-04-20T11:35:29.467 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:29.466+0000 7fa55b200640 1 --2- 172.21.15.152:0/2277872667 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fa53c06cec0 0x7fa53c06f380 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:29.467 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:29.466+0000 7fa55b200640 1 --2- 172.21.15.152:0/2277872667 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa55c15d5c0 0x7fa55c1474c0 unknown :-1 s=CLOSED pgs=1782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:29.467 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:29.466+0000 7fa55b200640 1 --2- 172.21.15.152:0/2277872667 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa55c15d1e0 0x7fa55c144c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:29.467 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:29.466+0000 7fa55b200640 1 --2- 172.21.15.152:0/2277872667 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa55c071cd0 0x7fa55c157230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:29.467 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:29.466+0000 7fa55b200640 1 -- 172.21.15.152:0/2277872667 >> 172.21.15.152:0/2277872667 conn(0x7fa55c067ae0 msgr2=0x7fa55c06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:29.467 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:29.466+0000 7fa55b200640 1 -- 172.21.15.152:0/2277872667 shutdown_connections 2024-04-20T11:35:29.467 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:29.466+0000 7fa55b200640 1 -- 172.21.15.152:0/2277872667 wait complete. 2024-04-20T11:35:29.487 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-20T11:35:29.487 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-20T11:35:29.487 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi175.front.sepia.ceph.com 2024-04-20T11:35:29.487 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-20T11:35:29.487 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-20T11:35:29.487 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-20T11:35:29.488 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-20T11:35:29.488 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-20T11:35:29.488 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-20T11:35:29.488 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:35:29.488 DEBUG:teuthology.orchestra.run.smithi175:> ip addr 2024-04-20T11:35:29.545 INFO:teuthology.orchestra.run.smithi175.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-20T11:35:29.546 INFO:teuthology.orchestra.run.smithi175.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-20T11:35:29.546 INFO:teuthology.orchestra.run.smithi175.stdout: inet 127.0.0.1/8 scope host lo 2024-04-20T11:35:29.546 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:35:29.546 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 ::1/128 scope host 2024-04-20T11:35:29.546 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:35:29.546 INFO:teuthology.orchestra.run.smithi175.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T11:35:29.546 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:d9:39:ee brd ff:ff:ff:ff:ff:ff 2024-04-20T11:35:29.546 INFO:teuthology.orchestra.run.smithi175.stdout: altname enp7s0f0 2024-04-20T11:35:29.546 INFO:teuthology.orchestra.run.smithi175.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T11:35:29.546 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:d9:39:ef brd ff:ff:ff:ff:ff:ff 2024-04-20T11:35:29.546 INFO:teuthology.orchestra.run.smithi175.stdout: altname enp7s0f1 2024-04-20T11:35:29.546 INFO:teuthology.orchestra.run.smithi175.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T11:35:29.547 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:88:71:f0 brd ff:ff:ff:ff:ff:ff 2024-04-20T11:35:29.547 INFO:teuthology.orchestra.run.smithi175.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-20T11:35:29.547 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:88:71:f1 brd ff:ff:ff:ff:ff:ff 2024-04-20T11:35:29.547 INFO:teuthology.orchestra.run.smithi175.stdout: inet 172.21.15.175/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-20T11:35:29.547 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft 42252sec preferred_lft 42252sec 2024-04-20T11:35:29.547 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::ec4:7aff:fe88:71f1/64 scope link 2024-04-20T11:35:29.547 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:35:29.547 INFO:teuthology.orchestra.run.smithi175.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-20T11:35:29.547 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 02:d6:c6:31:26:70 brd ff:ff:ff:ff:ff:ff 2024-04-20T11:35:29.547 INFO:teuthology.orchestra.run.smithi175.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-20T11:35:29.547 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:35:29.547 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::5091:d5ff:fee7:54b3/64 scope link 2024-04-20T11:35:29.547 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:35:29.547 INFO:teuthology.orchestra.run.smithi175.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-20T11:35:29.548 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 02:d6:c6:31:26:70 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-20T11:35:29.548 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::d6:c6ff:fe31:2670/64 scope link 2024-04-20T11:35:29.548 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:35:29.548 INFO:teuthology.orchestra.run.smithi175.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-20T11:35:29.548 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 9e:14:f3:e2:18:1e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-20T11:35:29.548 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::9c14:f3ff:fee2:181e/64 scope link 2024-04-20T11:35:29.548 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:35:29.549 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:35:29.549 DEBUG:teuthology.orchestra.run.smithi175:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-20T11:35:29.629 INFO:teuthology.orchestra.run.smithi175.stdout:1 2024-04-20T11:35:29.633 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:35:29.633 DEBUG:teuthology.orchestra.run.smithi175:> route 2024-04-20T11:35:29.685 INFO:teuthology.orchestra.run.smithi175.stdout:Kernel IP routing table 2024-04-20T11:35:29.685 INFO:teuthology.orchestra.run.smithi175.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-20T11:35:29.685 INFO:teuthology.orchestra.run.smithi175.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-20T11:35:29.685 INFO:teuthology.orchestra.run.smithi175.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-20T11:35:29.685 INFO:teuthology.orchestra.run.smithi175.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-20T11:35:29.686 DEBUG:teuthology.orchestra.run.smithi175:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T11:35:29.686 DEBUG:teuthology.orchestra.run.smithi175:> set -e 2024-04-20T11:35:29.686 DEBUG:teuthology.orchestra.run.smithi175:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-20T11:35:29.686 DEBUG:teuthology.orchestra.run.smithi175:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-20T11:35:29.686 DEBUG:teuthology.orchestra.run.smithi175:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-20T11:35:29.686 DEBUG:teuthology.orchestra.run.smithi175:> ') 2024-04-20T11:35:29.789 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:35:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T11:35:29.885 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:35:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T11:35:29.895 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:35:29.895 DEBUG:teuthology.orchestra.run.smithi175:> ip netns list 2024-04-20T11:35:29.942 INFO:teuthology.orchestra.run.smithi175.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-20T11:35:29.942 INFO:teuthology.orchestra.run.smithi175.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-20T11:35:29.942 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-20T11:35:29.943 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:35:29.943 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip link set brx.0 up 2024-04-20T11:35:30.072 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-04-20T11:35:30.072 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T11:35:30.072 DEBUG:teuthology.orchestra.run.smithi175:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-20T11:35:30.127 INFO:teuthology.orchestra.run.smithi175.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-20T11:35:30.127 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T11:35:30.128 DEBUG:teuthology.orchestra.run.smithi175:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-20T11:35:30.179 DEBUG:teuthology.orchestra.run.smithi175:> sudo modprobe fuse 2024-04-20T11:35:30.253 DEBUG:teuthology.orchestra.run.smithi175:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T11:35:30.305 INFO:teuthology.orchestra.run.smithi175.stdout:/proc 2024-04-20T11:35:30.305 INFO:teuthology.orchestra.run.smithi175.stdout:/sys 2024-04-20T11:35:30.305 INFO:teuthology.orchestra.run.smithi175.stdout:/dev 2024-04-20T11:35:30.305 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/security 2024-04-20T11:35:30.305 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/shm 2024-04-20T11:35:30.305 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/pts 2024-04-20T11:35:30.306 INFO:teuthology.orchestra.run.smithi175.stdout:/run 2024-04-20T11:35:30.306 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/cgroup 2024-04-20T11:35:30.306 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/pstore 2024-04-20T11:35:30.306 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/bpf 2024-04-20T11:35:30.306 INFO:teuthology.orchestra.run.smithi175.stdout:/ 2024-04-20T11:35:30.306 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/selinux 2024-04-20T11:35:30.306 INFO:teuthology.orchestra.run.smithi175.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T11:35:30.306 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/mqueue 2024-04-20T11:35:30.306 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/debug 2024-04-20T11:35:30.306 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/hugepages 2024-04-20T11:35:30.306 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/tracing 2024-04-20T11:35:30.306 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/fuse/connections 2024-04-20T11:35:30.306 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/config 2024-04-20T11:35:30.307 INFO:teuthology.orchestra.run.smithi175.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T11:35:30.307 INFO:teuthology.orchestra.run.smithi175.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T11:35:30.307 INFO:teuthology.orchestra.run.smithi175.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T11:35:30.307 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T11:35:30.307 INFO:teuthology.orchestra.run.smithi175.stdout:/run/user/1000 2024-04-20T11:35:30.307 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph 2024-04-20T11:35:30.307 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-4 2024-04-20T11:35:30.307 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-5 2024-04-20T11:35:30.307 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-6 2024-04-20T11:35:30.307 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-7 2024-04-20T11:35:30.307 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns 2024-04-20T11:35:30.307 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-20T11:35:30.307 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-20T11:35:30.308 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-20T11:35:30.308 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-20T11:35:30.308 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:35:30.309 DEBUG:teuthology.orchestra.run.smithi175:> ls /sys/fs/fuse/connections 2024-04-20T11:35:30.362 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-20T11:35:30.363 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-04-20T11:35:30.367 DEBUG:teuthology.orchestra.run.smithi175:> sudo modprobe fuse 2024-04-20T11:35:30.435 DEBUG:teuthology.orchestra.run.smithi175:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T11:35:30.482 INFO:teuthology.orchestra.run.smithi175.stdout:/proc 2024-04-20T11:35:30.482 INFO:teuthology.orchestra.run.smithi175.stdout:/sys 2024-04-20T11:35:30.482 INFO:teuthology.orchestra.run.smithi175.stdout:/dev 2024-04-20T11:35:30.483 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/security 2024-04-20T11:35:30.483 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/shm 2024-04-20T11:35:30.483 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/pts 2024-04-20T11:35:30.483 INFO:teuthology.orchestra.run.smithi175.stdout:/run 2024-04-20T11:35:30.483 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/cgroup 2024-04-20T11:35:30.483 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/pstore 2024-04-20T11:35:30.483 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/bpf 2024-04-20T11:35:30.483 INFO:teuthology.orchestra.run.smithi175.stdout:/ 2024-04-20T11:35:30.483 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/selinux 2024-04-20T11:35:30.483 INFO:teuthology.orchestra.run.smithi175.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T11:35:30.483 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/mqueue 2024-04-20T11:35:30.483 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/debug 2024-04-20T11:35:30.483 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/hugepages 2024-04-20T11:35:30.484 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/tracing 2024-04-20T11:35:30.484 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/fuse/connections 2024-04-20T11:35:30.484 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/config 2024-04-20T11:35:30.484 INFO:teuthology.orchestra.run.smithi175.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T11:35:30.484 INFO:teuthology.orchestra.run.smithi175.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T11:35:30.484 INFO:teuthology.orchestra.run.smithi175.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T11:35:30.484 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T11:35:30.484 INFO:teuthology.orchestra.run.smithi175.stdout:/run/user/1000 2024-04-20T11:35:30.484 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph 2024-04-20T11:35:30.484 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-4 2024-04-20T11:35:30.484 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-5 2024-04-20T11:35:30.485 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-6 2024-04-20T11:35:30.485 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-7 2024-04-20T11:35:30.485 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns 2024-04-20T11:35:30.485 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-20T11:35:30.485 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-20T11:35:30.485 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-20T11:35:30.485 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-20T11:35:30.486 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:35:30.486 DEBUG:teuthology.orchestra.run.smithi175:> ls /sys/fs/fuse/connections 2024-04-20T11:35:30.532 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi175.stderr:2024-04-20T11:35:30.532+0000 7fd0931db480 -1 init, newargv = 0x558f107076b0 newargc=15 2024-04-20T11:35:30.532 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi175.stderr:ceph-fuse[35123]: starting ceph client 2024-04-20T11:35:30.545 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi175.stderr:ceph-fuse[35123]: starting fuse 2024-04-20T11:35:31.630 DEBUG:teuthology.orchestra.run.smithi175:> sudo modprobe fuse 2024-04-20T11:35:31.722 DEBUG:teuthology.orchestra.run.smithi175:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T11:35:31.775 INFO:teuthology.orchestra.run.smithi175.stdout:/proc 2024-04-20T11:35:31.775 INFO:teuthology.orchestra.run.smithi175.stdout:/sys 2024-04-20T11:35:31.775 INFO:teuthology.orchestra.run.smithi175.stdout:/dev 2024-04-20T11:35:31.775 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/security 2024-04-20T11:35:31.776 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/shm 2024-04-20T11:35:31.776 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/pts 2024-04-20T11:35:31.776 INFO:teuthology.orchestra.run.smithi175.stdout:/run 2024-04-20T11:35:31.776 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/cgroup 2024-04-20T11:35:31.776 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/pstore 2024-04-20T11:35:31.776 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/bpf 2024-04-20T11:35:31.776 INFO:teuthology.orchestra.run.smithi175.stdout:/ 2024-04-20T11:35:31.776 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/selinux 2024-04-20T11:35:31.776 INFO:teuthology.orchestra.run.smithi175.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T11:35:31.776 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/mqueue 2024-04-20T11:35:31.776 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/debug 2024-04-20T11:35:31.776 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/hugepages 2024-04-20T11:35:31.776 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/tracing 2024-04-20T11:35:31.776 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/fuse/connections 2024-04-20T11:35:31.776 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/config 2024-04-20T11:35:31.777 INFO:teuthology.orchestra.run.smithi175.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T11:35:31.777 INFO:teuthology.orchestra.run.smithi175.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T11:35:31.777 INFO:teuthology.orchestra.run.smithi175.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T11:35:31.777 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T11:35:31.777 INFO:teuthology.orchestra.run.smithi175.stdout:/run/user/1000 2024-04-20T11:35:31.777 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph 2024-04-20T11:35:31.777 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-4 2024-04-20T11:35:31.777 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-5 2024-04-20T11:35:31.777 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-6 2024-04-20T11:35:31.777 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-7 2024-04-20T11:35:31.777 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns 2024-04-20T11:35:31.777 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-20T11:35:31.777 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-20T11:35:31.778 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-20T11:35:31.778 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-20T11:35:31.778 INFO:teuthology.orchestra.run.smithi175.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-20T11:35:31.779 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:35:31.779 DEBUG:teuthology.orchestra.run.smithi175:> ls /sys/fs/fuse/connections 2024-04-20T11:35:31.829 INFO:teuthology.orchestra.run.smithi175.stdout:42 2024-04-20T11:35:31.829 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-04-20T11:35:31.830 DEBUG:teuthology.orchestra.run.smithi175:> sudo stdin-killer -- python3 -c ' 2024-04-20T11:35:31.830 DEBUG:teuthology.orchestra.run.smithi175:> import glob 2024-04-20T11:35:31.830 DEBUG:teuthology.orchestra.run.smithi175:> import re 2024-04-20T11:35:31.830 DEBUG:teuthology.orchestra.run.smithi175:> import os 2024-04-20T11:35:31.830 DEBUG:teuthology.orchestra.run.smithi175:> import subprocess 2024-04-20T11:35:31.830 DEBUG:teuthology.orchestra.run.smithi175:> 2024-04-20T11:35:31.830 DEBUG:teuthology.orchestra.run.smithi175:> def _find_admin_socket(client_name): 2024-04-20T11:35:31.830 DEBUG:teuthology.orchestra.run.smithi175:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-04-20T11:35:31.830 DEBUG:teuthology.orchestra.run.smithi175:> files = glob.glob(asok_path) 2024-04-20T11:35:31.830 DEBUG:teuthology.orchestra.run.smithi175:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-04-20T11:35:31.830 DEBUG:teuthology.orchestra.run.smithi175:> 2024-04-20T11:35:31.830 DEBUG:teuthology.orchestra.run.smithi175:> # Given a non-glob path, it better be there 2024-04-20T11:35:31.830 DEBUG:teuthology.orchestra.run.smithi175:> if "*" not in asok_path: 2024-04-20T11:35:31.830 DEBUG:teuthology.orchestra.run.smithi175:> assert(len(files) == 1) 2024-04-20T11:35:31.831 DEBUG:teuthology.orchestra.run.smithi175:> return files[0] 2024-04-20T11:35:31.831 DEBUG:teuthology.orchestra.run.smithi175:> 2024-04-20T11:35:31.831 DEBUG:teuthology.orchestra.run.smithi175:> for f in files: 2024-04-20T11:35:31.831 DEBUG:teuthology.orchestra.run.smithi175:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-20T11:35:31.831 DEBUG:teuthology.orchestra.run.smithi175:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-20T11:35:31.831 DEBUG:teuthology.orchestra.run.smithi175:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-20T11:35:31.831 DEBUG:teuthology.orchestra.run.smithi175:> contents = proc_f.read() 2024-04-20T11:35:31.831 DEBUG:teuthology.orchestra.run.smithi175:> if mountpoint in contents: 2024-04-20T11:35:31.831 DEBUG:teuthology.orchestra.run.smithi175:> return f 2024-04-20T11:35:31.831 DEBUG:teuthology.orchestra.run.smithi175:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-20T11:35:31.831 DEBUG:teuthology.orchestra.run.smithi175:> 2024-04-20T11:35:31.831 DEBUG:teuthology.orchestra.run.smithi175:> print(_find_admin_socket("client.1")) 2024-04-20T11:35:31.831 DEBUG:teuthology.orchestra.run.smithi175:> ' 2024-04-20T11:35:31.993 INFO:teuthology.orchestra.run.smithi175.stdout:/var/run/ceph/ceph-client.1.35123.asok 2024-04-20T11:35:31.997 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:35:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T11:35:32.010 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.35123.asok 2024-04-20T11:35:32.010 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:35:32.010 DEBUG:teuthology.orchestra.run.smithi175:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.35123.asok status 2024-04-20T11:35:32.227 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-04-20T11:35:32.228 INFO:teuthology.orchestra.run.smithi175.stdout: "metadata": { 2024-04-20T11:35:32.228 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_sha1": "a8c84f5997950fd7d008204839253291d52e6eda", 2024-04-20T11:35:32.228 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", 2024-04-20T11:35:32.228 INFO:teuthology.orchestra.run.smithi175.stdout: "entity_id": "1", 2024-04-20T11:35:32.228 INFO:teuthology.orchestra.run.smithi175.stdout: "hostname": "smithi175", 2024-04-20T11:35:32.228 INFO:teuthology.orchestra.run.smithi175.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-04-20T11:35:32.228 INFO:teuthology.orchestra.run.smithi175.stdout: "pid": "35123", 2024-04-20T11:35:32.228 INFO:teuthology.orchestra.run.smithi175.stdout: "root": "/" 2024-04-20T11:35:32.228 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-20T11:35:32.228 INFO:teuthology.orchestra.run.smithi175.stdout: "dentry_count": 0, 2024-04-20T11:35:32.228 INFO:teuthology.orchestra.run.smithi175.stdout: "dentry_pinned_count": 0, 2024-04-20T11:35:32.228 INFO:teuthology.orchestra.run.smithi175.stdout: "id": 6727, 2024-04-20T11:35:32.229 INFO:teuthology.orchestra.run.smithi175.stdout: "inst": { 2024-04-20T11:35:32.229 INFO:teuthology.orchestra.run.smithi175.stdout: "name": { 2024-04-20T11:35:32.229 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "client", 2024-04-20T11:35:32.229 INFO:teuthology.orchestra.run.smithi175.stdout: "num": 6727 2024-04-20T11:35:32.229 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-20T11:35:32.229 INFO:teuthology.orchestra.run.smithi175.stdout: "addr": { 2024-04-20T11:35:32.229 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "v1", 2024-04-20T11:35:32.229 INFO:teuthology.orchestra.run.smithi175.stdout: "addr": "192.168.0.1:0", 2024-04-20T11:35:32.229 INFO:teuthology.orchestra.run.smithi175.stdout: "nonce": 3572222535 2024-04-20T11:35:32.229 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-04-20T11:35:32.229 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-20T11:35:32.229 INFO:teuthology.orchestra.run.smithi175.stdout: "addr": { 2024-04-20T11:35:32.229 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "v1", 2024-04-20T11:35:32.229 INFO:teuthology.orchestra.run.smithi175.stdout: "addr": "192.168.0.1:0", 2024-04-20T11:35:32.229 INFO:teuthology.orchestra.run.smithi175.stdout: "nonce": 3572222535 2024-04-20T11:35:32.230 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-20T11:35:32.230 INFO:teuthology.orchestra.run.smithi175.stdout: "inst_str": "client.6727 192.168.0.1:0/3572222535", 2024-04-20T11:35:32.230 INFO:teuthology.orchestra.run.smithi175.stdout: "addr_str": "192.168.0.1:0/3572222535", 2024-04-20T11:35:32.230 INFO:teuthology.orchestra.run.smithi175.stdout: "inode_count": 1, 2024-04-20T11:35:32.230 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_epoch": 86, 2024-04-20T11:35:32.230 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_epoch": 131, 2024-04-20T11:35:32.230 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_epoch_barrier": 0, 2024-04-20T11:35:32.230 INFO:teuthology.orchestra.run.smithi175.stdout: "blocklisted": false, 2024-04-20T11:35:32.230 INFO:teuthology.orchestra.run.smithi175.stdout: "fs_name": "cephfs" 2024-04-20T11:35:32.230 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-04-20T11:35:32.242 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:35:32.242 DEBUG:teuthology.orchestra.run.smithi175:> stat --file-system '--printf=%T 2024-04-20T11:35:32.242 DEBUG:teuthology.orchestra.run.smithi175:> ' -- /home/ubuntu/cephtest/mnt.1 2024-04-20T11:35:32.291 INFO:teuthology.orchestra.run.smithi175.stdout:fuseblk 2024-04-20T11:35:32.291 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-04-20T11:35:32.291 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:35:32.291 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-20T11:35:32.372 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-04-20T11:35:32.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:32.585+0000 7f5066e00640 1 Processor -- start 2024-04-20T11:35:32.585 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:32.585+0000 7f5066e00640 1 -- start start 2024-04-20T11:35:32.586 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:32.586+0000 7f5066e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5060158fc0 0x7f50600d05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:32.586 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:32.586+0000 7f5066e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f50600d0b00 0x7f506015efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:32.586 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:32.586+0000 7f5066e00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f506015fcf0 con 0x7f506015f520 2024-04-20T11:35:32.586 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:32.587+0000 7f5066e00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f50601593a0 con 0x7f50600d0b00 2024-04-20T11:35:32.586 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:32.587+0000 7f5066e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5060162db0 con 0x7f5060158fc0 2024-04-20T11:35:32.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:32.587+0000 7f5066400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f506015f520 0x7f5060072f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:56146/0 (socket says 172.21.15.152:56146) 2024-04-20T11:35:32.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:32.587+0000 7f5066400640 1 -- 172.21.15.152:0/1663912515 learned_addr learned my addr 172.21.15.152:0/1663912515 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:35:32.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:32.587+0000 7f5065a00640 1 --2- 172.21.15.152:0/1663912515 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5060158fc0 0x7f50600d05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:32.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:32.587+0000 7f5065000640 1 --2- 172.21.15.152:0/1663912515 >> v2:172.21.15.175:3300/0 conn(0x7f50600d0b00 0x7f506015efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:32.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:32.588+0000 7f505fe00640 1 -- 172.21.15.152:0/1663912515 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 212902629 0 0) 0x7f506015fcf0 con 0x7f506015f520 2024-04-20T11:35:32.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:32.588+0000 7f505fe00640 1 -- 172.21.15.152:0/1663912515 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5044003520 con 0x7f506015f520 2024-04-20T11:35:32.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:32.588+0000 7f505fe00640 1 -- 172.21.15.152:0/1663912515 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2345259515 0 0) 0x7f5044003520 con 0x7f506015f520 2024-04-20T11:35:32.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:32.588+0000 7f5065a00640 1 -- 172.21.15.152:0/1663912515 >> v2:172.21.15.175:3300/0 conn(0x7f50600d0b00 msgr2=0x7f506015efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:32.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:32.588+0000 7f5065a00640 1 --2- 172.21.15.152:0/1663912515 >> v2:172.21.15.175:3300/0 conn(0x7f50600d0b00 0x7f506015efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:32.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:32.588+0000 7f5065a00640 1 -- 172.21.15.152:0/1663912515 >> v1:172.21.15.152:6789/0 conn(0x7f506015f520 legacy=0x7f5060072f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:32.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:32.588+0000 7f5065a00640 1 -- 172.21.15.152:0/1663912515 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5060162480 con 0x7f5060158fc0 2024-04-20T11:35:32.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:32.589+0000 7f5065a00640 1 --2- 172.21.15.152:0/1663912515 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5060158fc0 0x7f50600d05c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f50500097a0 tx=0x7f505002b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2adbef876f2642ee server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:32.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:32.590+0000 7f505fe00640 1 -- 172.21.15.152:0/1663912515 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5050037070 con 0x7f5060158fc0 2024-04-20T11:35:32.590 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:32.590+0000 7f505fe00640 1 -- 172.21.15.152:0/1663912515 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5050030510 con 0x7f5060158fc0 2024-04-20T11:35:32.590 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:32.590+0000 7f505fe00640 1 -- 172.21.15.152:0/1663912515 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50580019f0 con 0x7f5060158fc0 2024-04-20T11:35:32.590 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:32.590+0000 7f5066e00640 1 -- 172.21.15.152:0/1663912515 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5060158fc0 msgr2=0x7f50600d05c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:32.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:32.590+0000 7f5066e00640 1 --2- 172.21.15.152:0/1663912515 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5060158fc0 0x7f50600d05c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f50500097a0 tx=0x7f505002b790 comp rx=0 tx=0).stop 2024-04-20T11:35:32.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:32.591+0000 7f5066e00640 1 -- 172.21.15.152:0/1663912515 shutdown_connections 2024-04-20T11:35:32.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:32.591+0000 7f5066e00640 1 --2- 172.21.15.152:0/1663912515 >> v2:172.21.15.175:3300/0 conn(0x7f50600d0b00 0x7f506015efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:32.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:32.591+0000 7f5066e00640 1 --2- 172.21.15.152:0/1663912515 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5060158fc0 0x7f50600d05c0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:32.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:32.591+0000 7f5066e00640 1 -- 172.21.15.152:0/1663912515 >> 172.21.15.152:0/1663912515 conn(0x7f506006bda0 msgr2=0x7f506006c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:32.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:32.591+0000 7f5066e00640 1 -- 172.21.15.152:0/1663912515 shutdown_connections 2024-04-20T11:35:32.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:32.591+0000 7f5066e00640 1 -- 172.21.15.152:0/1663912515 wait complete. 2024-04-20T11:35:32.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:32.592+0000 7f5066e00640 1 Processor -- start 2024-04-20T11:35:32.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:32.593+0000 7f5066e00640 1 -- start start 2024-04-20T11:35:32.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:32.593+0000 7f5066e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f50600d0b00 0x7f5060157560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:32.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:32.593+0000 7f5066e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5060158fc0 0x7f5060150b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:32.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:32.593+0000 7f5066e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f506015f520 0x7f506006eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:32.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:32.593+0000 7f5066e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f5060162a40 con 0x7f5060158fc0 2024-04-20T11:35:32.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:32.593+0000 7f5066e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f50600b4060 con 0x7f50600d0b00 2024-04-20T11:35:32.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:32.593+0000 7f5066e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f506015f980 con 0x7f506015f520 2024-04-20T11:35:32.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:32.594+0000 7f5065000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5060158fc0 0x7f5060150b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:32.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:32.594+0000 7f5065000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5060158fc0 0x7f5060150b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54522/0 (socket says 172.21.15.152:54522) 2024-04-20T11:35:32.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:32.594+0000 7f5065000640 1 -- 172.21.15.152:0/3627667929 learned_addr learned my addr 172.21.15.152:0/3627667929 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:32.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:32.594+0000 7f5065a00640 1 --2- 172.21.15.152:0/3627667929 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f50600d0b00 0x7f5060157560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:32.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:32.594+0000 7f5066400640 1 --2- 172.21.15.152:0/3627667929 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f506015f520 0x7f506006eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:32.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:32.594+0000 7f5065000640 1 -- 172.21.15.152:0/3627667929 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f506015f520 msgr2=0x7f506006eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:32.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:32.594+0000 7f5065000640 1 --2- 172.21.15.152:0/3627667929 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f506015f520 0x7f506006eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:32.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:32.594+0000 7f5065000640 1 -- 172.21.15.152:0/3627667929 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f50600d0b00 msgr2=0x7f5060157560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:32.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:32.594+0000 7f5065000640 1 --2- 172.21.15.152:0/3627667929 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f50600d0b00 0x7f5060157560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:32.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:32.594+0000 7f5065000640 1 -- 172.21.15.152:0/3627667929 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f506014e8f0 con 0x7f5060158fc0 2024-04-20T11:35:32.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:32.594+0000 7f5065a00640 1 --2- 172.21.15.152:0/3627667929 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f50600d0b00 0x7f5060157560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:35:32.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:32.594+0000 7f5066400640 1 --2- 172.21.15.152:0/3627667929 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f506015f520 0x7f506006eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:35:32.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:32.595+0000 7f5065000640 1 --2- 172.21.15.152:0/3627667929 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5060158fc0 0x7f5060150b30 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f504c00e870 tx=0x7f504c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:32.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:32.595+0000 7f505d600640 1 -- 172.21.15.152:0/3627667929 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f504c017070 con 0x7f5060158fc0 2024-04-20T11:35:32.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:32.595+0000 7f505d600640 1 -- 172.21.15.152:0/3627667929 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f504c00c940 con 0x7f5060158fc0 2024-04-20T11:35:32.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:32.595+0000 7f505d600640 1 -- 172.21.15.152:0/3627667929 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f504c00cd30 con 0x7f5060158fc0 2024-04-20T11:35:32.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:32.595+0000 7f5066e00640 1 -- 172.21.15.152:0/3627667929 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f506014ebd0 con 0x7f5060158fc0 2024-04-20T11:35:32.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:32.595+0000 7f5066e00640 1 -- 172.21.15.152:0/3627667929 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f506014f190 con 0x7f5060158fc0 2024-04-20T11:35:32.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:32.596+0000 7f5066e00640 1 -- 172.21.15.152:0/3627667929 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50600689f0 con 0x7f5060158fc0 2024-04-20T11:35:32.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:32.597+0000 7f505d600640 1 -- 172.21.15.152:0/3627667929 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 21) ==== 91550+0+0 (secure 0 0 0) 0x7f504c01a070 con 0x7f5060158fc0 2024-04-20T11:35:32.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:32.598+0000 7f505d600640 1 --2- 172.21.15.152:0/3627667929 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f504006cec0 0x7f504006f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:32.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:32.598+0000 7f505d600640 1 -- 172.21.15.152:0/3627667929 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5709+0+0 (secure 0 0 0) 0x7f504c08c4e0 con 0x7f5060158fc0 2024-04-20T11:35:32.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:32.599+0000 7f5065a00640 1 --2- 172.21.15.152:0/3627667929 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f504006cec0 0x7f504006f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:32.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:32.600+0000 7f5065a00640 1 --2- 172.21.15.152:0/3627667929 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f504006cec0 0x7f504006f380 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f5050034000 tx=0x7f505002f000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:32.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:32.603+0000 7f505d600640 1 -- 172.21.15.152:0/3627667929 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f504c05a060 con 0x7f5060158fc0 2024-04-20T11:35:32.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:32.841+0000 7f5066e00640 1 -- 172.21.15.152:0/3627667929 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f50600cecd0 con 0x7f5060158fc0 2024-04-20T11:35:33.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.154+0000 7f505d600640 1 -- 172.21.15.152:0/3627667929 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v87) ==== 136+0+0 (secure 0 0 0) 0x7f504c059960 con 0x7f5060158fc0 2024-04-20T11:35:33.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.157+0000 7f5066e00640 1 -- 172.21.15.152:0/3627667929 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f504006cec0 msgr2=0x7f504006f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:33.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.157+0000 7f5066e00640 1 --2- 172.21.15.152:0/3627667929 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f504006cec0 0x7f504006f380 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f5050034000 tx=0x7f505002f000 comp rx=0 tx=0).stop 2024-04-20T11:35:33.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.157+0000 7f5066e00640 1 -- 172.21.15.152:0/3627667929 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5060158fc0 msgr2=0x7f5060150b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:33.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.157+0000 7f5066e00640 1 --2- 172.21.15.152:0/3627667929 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5060158fc0 0x7f5060150b30 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f504c00e870 tx=0x7f504c00ed30 comp rx=0 tx=0).stop 2024-04-20T11:35:33.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.157+0000 7f5066e00640 1 -- 172.21.15.152:0/3627667929 shutdown_connections 2024-04-20T11:35:33.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.157+0000 7f5066e00640 1 --2- 172.21.15.152:0/3627667929 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f504006cec0 0x7f504006f380 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:33.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.157+0000 7f5066e00640 1 --2- 172.21.15.152:0/3627667929 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f506015f520 0x7f506006eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:33.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.157+0000 7f5066e00640 1 --2- 172.21.15.152:0/3627667929 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5060158fc0 0x7f5060150b30 unknown :-1 s=CLOSED pgs=1784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:33.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.157+0000 7f5066e00640 1 --2- 172.21.15.152:0/3627667929 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f50600d0b00 0x7f5060157560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:33.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.157+0000 7f5066e00640 1 -- 172.21.15.152:0/3627667929 >> 172.21.15.152:0/3627667929 conn(0x7f506006bda0 msgr2=0x7f50600b3860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:33.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.158+0000 7f5066e00640 1 -- 172.21.15.152:0/3627667929 shutdown_connections 2024-04-20T11:35:33.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.158+0000 7f5066e00640 1 -- 172.21.15.152:0/3627667929 wait complete. 2024-04-20T11:35:33.178 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-04-20T11:35:33.384 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.384+0000 7faf35200640 1 Processor -- start 2024-04-20T11:35:33.384 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.385+0000 7faf35200640 1 -- start start 2024-04-20T11:35:33.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.386+0000 7faf35200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7faf3006e0f0 0x7faf30166220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:33.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.386+0000 7faf35200640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf300cfef0 con 0x7faf30159320 2024-04-20T11:35:33.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.386+0000 7faf35200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf3015c3d0 con 0x7faf300d0e60 2024-04-20T11:35:33.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.386+0000 7faf35200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7faf3015c6b0 con 0x7faf3006e0f0 2024-04-20T11:35:33.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.387+0000 7faf2f400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7faf30159320 0x7faf3015bbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:56148/0 (socket says 172.21.15.152:56148) 2024-04-20T11:35:33.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.387+0000 7faf2f400640 1 -- 172.21.15.152:0/3434433653 learned_addr learned my addr 172.21.15.152:0/3434433653 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:35:33.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.387+0000 7faf2fe00640 1 --2- 172.21.15.152:0/3434433653 >> v2:172.21.15.152:3300/0 conn(0x7faf3006e0f0 0x7faf30166220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:33.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.387+0000 7faf2ea00640 1 -- 172.21.15.152:0/3434433653 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4051575324 0 0) 0x7faf300cfef0 con 0x7faf30159320 2024-04-20T11:35:33.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.387+0000 7faf2fe00640 1 -- 172.21.15.152:0/3434433653 >> v1:172.21.15.175:6789/0 conn(0x7faf300d0e60 legacy=0x7faf3006d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:33.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.387+0000 7faf2fe00640 1 -- 172.21.15.152:0/3434433653 >> v1:172.21.15.152:6789/0 conn(0x7faf30159320 legacy=0x7faf3015bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:33.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.387+0000 7faf2fe00640 1 -- 172.21.15.152:0/3434433653 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf3015c990 con 0x7faf3006e0f0 2024-04-20T11:35:33.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.388+0000 7faf2fe00640 1 --2- 172.21.15.152:0/3434433653 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faf3006e0f0 0x7faf30166220 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7faf28009d90 tx=0x7faf2802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f13e6900abca6be server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:33.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.389+0000 7faf2ea00640 1 -- 172.21.15.152:0/3434433653 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf28039070 con 0x7faf3006e0f0 2024-04-20T11:35:33.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.389+0000 7faf2ea00640 1 -- 172.21.15.152:0/3434433653 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf2802cc60 con 0x7faf3006e0f0 2024-04-20T11:35:33.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.389+0000 7faf2ea00640 1 -- 172.21.15.152:0/3434433653 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf28034660 con 0x7faf3006e0f0 2024-04-20T11:35:33.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.389+0000 7faf35200640 1 -- 172.21.15.152:0/3434433653 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faf3006e0f0 msgr2=0x7faf30166220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:33.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.389+0000 7faf35200640 1 --2- 172.21.15.152:0/3434433653 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faf3006e0f0 0x7faf30166220 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7faf28009d90 tx=0x7faf2802beb0 comp rx=0 tx=0).stop 2024-04-20T11:35:33.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.389+0000 7faf35200640 1 -- 172.21.15.152:0/3434433653 shutdown_connections 2024-04-20T11:35:33.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.389+0000 7faf35200640 1 --2- 172.21.15.152:0/3434433653 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faf3006e0f0 0x7faf30166220 unknown :-1 s=CLOSED pgs=1786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:33.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.389+0000 7faf35200640 1 -- 172.21.15.152:0/3434433653 >> 172.21.15.152:0/3434433653 conn(0x7faf300d0260 msgr2=0x7faf300d0670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:33.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.389+0000 7faf35200640 1 -- 172.21.15.152:0/3434433653 shutdown_connections 2024-04-20T11:35:33.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.389+0000 7faf35200640 1 -- 172.21.15.152:0/3434433653 wait complete. 2024-04-20T11:35:33.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.391+0000 7faf35200640 1 Processor -- start 2024-04-20T11:35:33.391 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.391+0000 7faf35200640 1 -- start start 2024-04-20T11:35:33.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.392+0000 7faf35200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faf3006e0f0 0x7faf3015a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:33.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.392+0000 7faf35200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faf300d0e60 0x7faf300ad2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:33.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.392+0000 7faf35200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faf30159320 0x7faf301f4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:33.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.392+0000 7faf35200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7faf3015e5e0 con 0x7faf300d0e60 2024-04-20T11:35:33.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.392+0000 7faf35200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7faf3015e460 con 0x7faf30159320 2024-04-20T11:35:33.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.392+0000 7faf35200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7faf3015e760 con 0x7faf3006e0f0 2024-04-20T11:35:33.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.393+0000 7faf26a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faf300d0e60 0x7faf300ad2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:33.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.393+0000 7faf26a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faf300d0e60 0x7faf300ad2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54540/0 (socket says 172.21.15.152:54540) 2024-04-20T11:35:33.393 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.393+0000 7faf26a00640 1 -- 172.21.15.152:0/3432645484 learned_addr learned my addr 172.21.15.152:0/3432645484 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:33.394 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.393+0000 7faf2fe00640 1 --2- 172.21.15.152:0/3432645484 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faf30159320 0x7faf301f4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:33.394 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.393+0000 7faf26a00640 1 -- 172.21.15.152:0/3432645484 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faf3006e0f0 msgr2=0x7faf3015a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:33.394 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.393+0000 7faf2f400640 1 --2- 172.21.15.152:0/3432645484 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faf3006e0f0 0x7faf3015a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:33.394 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.393+0000 7faf26a00640 1 --2- 172.21.15.152:0/3432645484 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faf3006e0f0 0x7faf3015a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:33.394 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.393+0000 7faf26a00640 1 -- 172.21.15.152:0/3432645484 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faf30159320 msgr2=0x7faf301f4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:33.394 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.393+0000 7faf26a00640 1 --2- 172.21.15.152:0/3432645484 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faf30159320 0x7faf301f4180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:33.394 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.393+0000 7faf26a00640 1 -- 172.21.15.152:0/3432645484 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf3015a7a0 con 0x7faf300d0e60 2024-04-20T11:35:33.394 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.393+0000 7faf2fe00640 1 --2- 172.21.15.152:0/3432645484 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faf30159320 0x7faf301f4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:35:33.394 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.393+0000 7faf2f400640 1 --2- 172.21.15.152:0/3432645484 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faf3006e0f0 0x7faf3015a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:35:33.394 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.394+0000 7faf26a00640 1 --2- 172.21.15.152:0/3432645484 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faf300d0e60 0x7faf300ad2b0 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7faf1800c430 tx=0x7faf1800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:33.395 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.394+0000 7faf27e00640 1 -- 172.21.15.152:0/3432645484 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf18011790 con 0x7faf300d0e60 2024-04-20T11:35:33.395 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.394+0000 7faf27e00640 1 -- 172.21.15.152:0/3432645484 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf18017070 con 0x7faf300d0e60 2024-04-20T11:35:33.395 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.394+0000 7faf35200640 1 -- 172.21.15.152:0/3432645484 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf3015aa80 con 0x7faf300d0e60 2024-04-20T11:35:33.395 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.394+0000 7faf27e00640 1 -- 172.21.15.152:0/3432645484 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf18011bb0 con 0x7faf300d0e60 2024-04-20T11:35:33.395 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.394+0000 7faf35200640 1 -- 172.21.15.152:0/3432645484 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf30145e80 con 0x7faf300d0e60 2024-04-20T11:35:33.395 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.395+0000 7faf35200640 1 -- 172.21.15.152:0/3432645484 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf30075e60 con 0x7faf300d0e60 2024-04-20T11:35:33.395 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.396+0000 7faf27e00640 1 -- 172.21.15.152:0/3432645484 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 21) ==== 91550+0+0 (secure 0 0 0) 0x7faf1801c460 con 0x7faf300d0e60 2024-04-20T11:35:33.396 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.397+0000 7faf27e00640 1 --2- 172.21.15.152:0/3432645484 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7faf0c06ce90 0x7faf0c06f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:33.396 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.397+0000 7faf27e00640 1 -- 172.21.15.152:0/3432645484 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5709+0+0 (secure 0 0 0) 0x7faf1808c420 con 0x7faf300d0e60 2024-04-20T11:35:33.397 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.398+0000 7faf2f400640 1 --2- 172.21.15.152:0/3432645484 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7faf0c06ce90 0x7faf0c06f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:33.398 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.399+0000 7faf2f400640 1 --2- 172.21.15.152:0/3432645484 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7faf0c06ce90 0x7faf0c06f350 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7faf20000f60 tx=0x7faf20009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:33.401 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.401+0000 7faf27e00640 1 -- 172.21.15.152:0/3432645484 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7faf1805a070 con 0x7faf300d0e60 2024-04-20T11:35:33.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.624+0000 7faf35200640 1 -- 172.21.15.152:0/3432645484 --> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7faf3006dbc0 con 0x7faf0c06ce90 2024-04-20T11:35:33.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.628+0000 7faf27e00640 1 -- 172.21.15.152:0/3432645484 <== mgr.4109 v2:172.21.15.175:6836/4021250441 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7faf3006dbc0 con 0x7faf0c06ce90 2024-04-20T11:35:33.628 INFO:teuthology.orchestra.run.smithi152.stdout:[ 2024-04-20T11:35:33.628 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:35:33.628 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "cephfs" 2024-04-20T11:35:33.628 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:35:33.628 INFO:teuthology.orchestra.run.smithi152.stdout:] 2024-04-20T11:35:33.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.630+0000 7faf35200640 1 -- 172.21.15.152:0/3432645484 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7faf0c06ce90 msgr2=0x7faf0c06f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:33.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.630+0000 7faf35200640 1 --2- 172.21.15.152:0/3432645484 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7faf0c06ce90 0x7faf0c06f350 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7faf20000f60 tx=0x7faf20009040 comp rx=0 tx=0).stop 2024-04-20T11:35:33.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.630+0000 7faf35200640 1 -- 172.21.15.152:0/3432645484 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faf300d0e60 msgr2=0x7faf300ad2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:33.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.630+0000 7faf35200640 1 --2- 172.21.15.152:0/3432645484 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faf300d0e60 0x7faf300ad2b0 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7faf1800c430 tx=0x7faf1800c8f0 comp rx=0 tx=0).stop 2024-04-20T11:35:33.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.631+0000 7faf35200640 1 -- 172.21.15.152:0/3432645484 shutdown_connections 2024-04-20T11:35:33.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.631+0000 7faf35200640 1 --2- 172.21.15.152:0/3432645484 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7faf0c06ce90 0x7faf0c06f350 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:33.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.631+0000 7faf35200640 1 --2- 172.21.15.152:0/3432645484 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faf30159320 0x7faf301f4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:33.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.631+0000 7faf35200640 1 --2- 172.21.15.152:0/3432645484 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faf300d0e60 0x7faf300ad2b0 unknown :-1 s=CLOSED pgs=1787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:33.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.631+0000 7faf35200640 1 --2- 172.21.15.152:0/3432645484 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faf3006e0f0 0x7faf3015a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:33.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.631+0000 7faf35200640 1 -- 172.21.15.152:0/3432645484 >> 172.21.15.152:0/3432645484 conn(0x7faf300d0260 msgr2=0x7faf300b44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:33.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.631+0000 7faf35200640 1 -- 172.21.15.152:0/3432645484 shutdown_connections 2024-04-20T11:35:33.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.631+0000 7faf35200640 1 -- 172.21.15.152:0/3432645484 wait complete. 2024-04-20T11:35:33.649 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-20T11:35:33.848 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.849+0000 7f8c87e00640 1 Processor -- start 2024-04-20T11:35:33.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.849+0000 7f8c87e00640 1 -- start start 2024-04-20T11:35:33.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.850+0000 7f8c87e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8c88160220 0x7f8c881606b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:33.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.850+0000 7f8c87e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f8c88141450 0x7f8c881418e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:33.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.850+0000 7f8c87e00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c8806fb40 con 0x7f8c88071cd0 2024-04-20T11:35:33.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.850+0000 7f8c87e00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f8c8813f8e0 con 0x7f8c88141450 2024-04-20T11:35:33.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.850+0000 7f8c87e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8c88160f90 con 0x7f8c88160220 2024-04-20T11:35:33.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.851+0000 7f8c87400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f8c88141450 0x7f8c881418e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:33.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.851+0000 7f8c87400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f8c88141450 0x7f8c881418e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54556/0 (socket says 172.21.15.152:54556) 2024-04-20T11:35:33.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.851+0000 7f8c87400640 1 -- 172.21.15.152:0/4019021403 learned_addr learned my addr 172.21.15.152:0/4019021403 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:33.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.851+0000 7f8c86000640 1 --2- 172.21.15.152:0/4019021403 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8c88160220 0x7f8c881606b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:33.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.851+0000 7f8c87400640 1 -- 172.21.15.152:0/4019021403 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8c88160220 msgr2=0x7f8c881606b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:33.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.851+0000 7f8c87400640 1 --2- 172.21.15.152:0/4019021403 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8c88160220 0x7f8c881606b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:33.852 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.851+0000 7f8c87400640 1 -- 172.21.15.152:0/4019021403 >> v1:172.21.15.175:6789/0 conn(0x7f8c88071cd0 legacy=0x7f8c880720b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:33.852 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.851+0000 7f8c87400640 1 -- 172.21.15.152:0/4019021403 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c8806dd90 con 0x7f8c88141450 2024-04-20T11:35:33.852 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.851+0000 7f8c86000640 1 --2- 172.21.15.152:0/4019021403 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8c88160220 0x7f8c881606b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:35:33.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.852+0000 7f8c87400640 1 --2- 172.21.15.152:0/4019021403 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c88141450 0x7f8c881418e0 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f8c80009e80 tx=0x7f8c8002be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf6d2f51d4ca08f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:33.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.853+0000 7f8c85600640 1 -- 172.21.15.152:0/4019021403 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c8002c9c0 con 0x7f8c88141450 2024-04-20T11:35:33.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.853+0000 7f8c85600640 1 -- 172.21.15.152:0/4019021403 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c8002cb60 con 0x7f8c88141450 2024-04-20T11:35:33.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.853+0000 7f8c85600640 1 -- 172.21.15.152:0/4019021403 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c800343f0 con 0x7f8c88141450 2024-04-20T11:35:33.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.853+0000 7f8c87e00640 1 -- 172.21.15.152:0/4019021403 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c88141450 msgr2=0x7f8c881418e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:33.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.853+0000 7f8c87e00640 1 --2- 172.21.15.152:0/4019021403 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c88141450 0x7f8c881418e0 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f8c80009e80 tx=0x7f8c8002be20 comp rx=0 tx=0).stop 2024-04-20T11:35:33.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.853+0000 7f8c87e00640 1 -- 172.21.15.152:0/4019021403 shutdown_connections 2024-04-20T11:35:33.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.853+0000 7f8c87e00640 1 --2- 172.21.15.152:0/4019021403 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c88141450 0x7f8c881418e0 unknown :-1 s=CLOSED pgs=1788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:33.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.853+0000 7f8c87e00640 1 --2- 172.21.15.152:0/4019021403 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8c88160220 0x7f8c881606b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:33.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.853+0000 7f8c87e00640 1 -- 172.21.15.152:0/4019021403 >> 172.21.15.152:0/4019021403 conn(0x7f8c88067ae0 msgr2=0x7f8c8806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:33.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.853+0000 7f8c87e00640 1 -- 172.21.15.152:0/4019021403 shutdown_connections 2024-04-20T11:35:33.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.853+0000 7f8c87e00640 1 -- 172.21.15.152:0/4019021403 wait complete. 2024-04-20T11:35:33.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.855+0000 7f8c87e00640 1 Processor -- start 2024-04-20T11:35:33.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.855+0000 7f8c87e00640 1 -- start start 2024-04-20T11:35:33.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.856+0000 7f8c87e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8c88071cd0 0x7f8c880af030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:33.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.856+0000 7f8c87e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8c88141450 0x7f8c881f67e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:33.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.856+0000 7f8c87e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c88160220 0x7f8c881e7b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:33.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.856+0000 7f8c87e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f8c8806e630 con 0x7f8c88160220 2024-04-20T11:35:33.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.856+0000 7f8c87e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8c8806e350 con 0x7f8c88071cd0 2024-04-20T11:35:33.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.856+0000 7f8c87e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8c88160bf0 con 0x7f8c88141450 2024-04-20T11:35:33.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.856+0000 7f8c87400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c88160220 0x7f8c881e7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:33.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.856+0000 7f8c87400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c88160220 0x7f8c881e7b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54566/0 (socket says 172.21.15.152:54566) 2024-04-20T11:35:33.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.856+0000 7f8c87400640 1 -- 172.21.15.152:0/2266208606 learned_addr learned my addr 172.21.15.152:0/2266208606 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:33.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.857+0000 7f8c86a00640 1 --2- 172.21.15.152:0/2266208606 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8c88071cd0 0x7f8c880af030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:33.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.857+0000 7f8c86000640 1 --2- 172.21.15.152:0/2266208606 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8c88141450 0x7f8c881f67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:33.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.857+0000 7f8c87400640 1 -- 172.21.15.152:0/2266208606 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8c88141450 msgr2=0x7f8c881f67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:33.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.857+0000 7f8c87400640 1 --2- 172.21.15.152:0/2266208606 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8c88141450 0x7f8c881f67e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:33.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.857+0000 7f8c87400640 1 -- 172.21.15.152:0/2266208606 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8c88071cd0 msgr2=0x7f8c880af030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:33.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.857+0000 7f8c87400640 1 --2- 172.21.15.152:0/2266208606 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8c88071cd0 0x7f8c880af030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:33.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.857+0000 7f8c87400640 1 -- 172.21.15.152:0/2266208606 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c88156a80 con 0x7f8c88160220 2024-04-20T11:35:33.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.857+0000 7f8c87400640 1 --2- 172.21.15.152:0/2266208606 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c88160220 0x7f8c881e7b20 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f8c80009fa0 tx=0x7f8c80004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:33.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.857+0000 7f8c7ea00640 1 -- 172.21.15.152:0/2266208606 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c80043020 con 0x7f8c88160220 2024-04-20T11:35:33.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.857+0000 7f8c7ea00640 1 -- 172.21.15.152:0/2266208606 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c8003e450 con 0x7f8c88160220 2024-04-20T11:35:33.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.857+0000 7f8c7ea00640 1 -- 172.21.15.152:0/2266208606 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c80009510 con 0x7f8c88160220 2024-04-20T11:35:33.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.857+0000 7f8c87e00640 1 -- 172.21.15.152:0/2266208606 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c881576a0 con 0x7f8c88160220 2024-04-20T11:35:33.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.858+0000 7f8c87e00640 1 -- 172.21.15.152:0/2266208606 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c88157c60 con 0x7f8c88160220 2024-04-20T11:35:33.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.859+0000 7f8c87e00640 1 -- 172.21.15.152:0/2266208606 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c8815d380 con 0x7f8c88160220 2024-04-20T11:35:33.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.860+0000 7f8c7ea00640 1 -- 172.21.15.152:0/2266208606 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 21) ==== 91550+0+0 (secure 0 0 0) 0x7f8c80046020 con 0x7f8c88160220 2024-04-20T11:35:33.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.860+0000 7f8c7ea00640 1 --2- 172.21.15.152:0/2266208606 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f8c6406ce90 0x7f8c6406f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:33.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.861+0000 7f8c7ea00640 1 -- 172.21.15.152:0/2266208606 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5709+0+0 (secure 0 0 0) 0x7f8c800aee40 con 0x7f8c88160220 2024-04-20T11:35:33.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.861+0000 7f8c86a00640 1 --2- 172.21.15.152:0/2266208606 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f8c6406ce90 0x7f8c6406f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:33.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.862+0000 7f8c86a00640 1 --2- 172.21.15.152:0/2266208606 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f8c6406ce90 0x7f8c6406f350 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f8c78004ea0 tx=0x7f8c78009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:33.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:33.865+0000 7f8c7ea00640 1 -- 172.21.15.152:0/2266208606 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f8c8815d380 con 0x7f8c88160220 2024-04-20T11:35:34.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.048+0000 7f8c87e00640 1 -- 172.21.15.152:0/2266208606 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f8c880cfda0 con 0x7f8c88160220 2024-04-20T11:35:34.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.049+0000 7f8c7ea00640 1 -- 172.21.15.152:0/2266208606 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v24)=0 v24) ==== 122+0+0 (secure 0 0 0) 0x7f8c880cfda0 con 0x7f8c88160220 2024-04-20T11:35:34.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.051+0000 7f8c87e00640 1 -- 172.21.15.152:0/2266208606 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f8c6406ce90 msgr2=0x7f8c6406f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:34.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.051+0000 7f8c87e00640 1 --2- 172.21.15.152:0/2266208606 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f8c6406ce90 0x7f8c6406f350 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f8c78004ea0 tx=0x7f8c78009040 comp rx=0 tx=0).stop 2024-04-20T11:35:34.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.051+0000 7f8c87e00640 1 -- 172.21.15.152:0/2266208606 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c88160220 msgr2=0x7f8c881e7b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:34.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.051+0000 7f8c87e00640 1 --2- 172.21.15.152:0/2266208606 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c88160220 0x7f8c881e7b20 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f8c80009fa0 tx=0x7f8c80004a40 comp rx=0 tx=0).stop 2024-04-20T11:35:34.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.051+0000 7f8c87e00640 1 -- 172.21.15.152:0/2266208606 shutdown_connections 2024-04-20T11:35:34.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.051+0000 7f8c87e00640 1 --2- 172.21.15.152:0/2266208606 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f8c6406ce90 0x7f8c6406f350 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:34.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.051+0000 7f8c87e00640 1 --2- 172.21.15.152:0/2266208606 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c88160220 0x7f8c881e7b20 unknown :-1 s=CLOSED pgs=1789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:34.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.051+0000 7f8c87e00640 1 --2- 172.21.15.152:0/2266208606 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8c88141450 0x7f8c881f67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:34.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.051+0000 7f8c87e00640 1 --2- 172.21.15.152:0/2266208606 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8c88071cd0 0x7f8c880af030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:34.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.052+0000 7f8c87e00640 1 -- 172.21.15.152:0/2266208606 >> 172.21.15.152:0/2266208606 conn(0x7f8c88067ae0 msgr2=0x7f8c8815cc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:34.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.052+0000 7f8c87e00640 1 -- 172.21.15.152:0/2266208606 shutdown_connections 2024-04-20T11:35:34.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.052+0000 7f8c87e00640 1 -- 172.21.15.152:0/2266208606 wait complete. 2024-04-20T11:35:34.070 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create non_existent_subvolume subvolume_1 2024-04-20T11:35:34.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.278+0000 7f0f73200640 1 Processor -- start 2024-04-20T11:35:34.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.278+0000 7f0f73200640 1 -- start start 2024-04-20T11:35:34.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.280+0000 7f0f73200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f0f74161c70 0x7f0f740d4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:34.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.280+0000 7f0f73200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0f740cf5d0 0x7f0f741623e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:34.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.280+0000 7f0f73200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f740d53d0 con 0x7f0f74162920 2024-04-20T11:35:34.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.280+0000 7f0f73200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f0f74163260 con 0x7f0f74161c70 2024-04-20T11:35:34.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.280+0000 7f0f73200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0f740b77d0 con 0x7f0f740cf5d0 2024-04-20T11:35:34.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.280+0000 7f0f71e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f0f74161c70 0x7f0f740d4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:34.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.280+0000 7f0f71e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f0f74161c70 0x7f0f740d4e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54574/0 (socket says 172.21.15.152:54574) 2024-04-20T11:35:34.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.280+0000 7f0f71e00640 1 -- 172.21.15.152:0/2334804537 learned_addr learned my addr 172.21.15.152:0/2334804537 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:34.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.280+0000 7f0f71400640 1 --2- 172.21.15.152:0/2334804537 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0f740cf5d0 0x7f0f741623e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:34.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.281+0000 7f0f71e00640 1 -- 172.21.15.152:0/2334804537 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0f740cf5d0 msgr2=0x7f0f741623e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:34.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.281+0000 7f0f71e00640 1 --2- 172.21.15.152:0/2334804537 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0f740cf5d0 0x7f0f741623e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:34.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.281+0000 7f0f71e00640 1 -- 172.21.15.152:0/2334804537 >> v1:172.21.15.175:6789/0 conn(0x7f0f74162920 legacy=0x7f0f740d2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:34.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.281+0000 7f0f71e00640 1 -- 172.21.15.152:0/2334804537 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f740cfa60 con 0x7f0f74161c70 2024-04-20T11:35:34.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.282+0000 7f0f71e00640 1 --2- 172.21.15.152:0/2334804537 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0f74161c70 0x7f0f740d4e90 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f0f64009870 tx=0x7f0f6402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dc76a5d47130823 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:34.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.283+0000 7f0f70a00640 1 -- 172.21.15.152:0/2334804537 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f64034070 con 0x7f0f74161c70 2024-04-20T11:35:34.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.283+0000 7f0f70a00640 1 -- 172.21.15.152:0/2334804537 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f64030840 con 0x7f0f74161c70 2024-04-20T11:35:34.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.283+0000 7f0f70a00640 1 -- 172.21.15.152:0/2334804537 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f64030c10 con 0x7f0f74161c70 2024-04-20T11:35:34.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.284+0000 7f0f73200640 1 -- 172.21.15.152:0/2334804537 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0f74161c70 msgr2=0x7f0f740d4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:34.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.284+0000 7f0f73200640 1 --2- 172.21.15.152:0/2334804537 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0f74161c70 0x7f0f740d4e90 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f0f64009870 tx=0x7f0f6402b980 comp rx=0 tx=0).stop 2024-04-20T11:35:34.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.284+0000 7f0f73200640 1 -- 172.21.15.152:0/2334804537 shutdown_connections 2024-04-20T11:35:34.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.284+0000 7f0f73200640 1 --2- 172.21.15.152:0/2334804537 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0f740cf5d0 0x7f0f741623e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:34.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.284+0000 7f0f73200640 1 --2- 172.21.15.152:0/2334804537 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0f74161c70 0x7f0f740d4e90 unknown :-1 s=CLOSED pgs=1790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:34.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.284+0000 7f0f73200640 1 -- 172.21.15.152:0/2334804537 >> 172.21.15.152:0/2334804537 conn(0x7f0f7408bfd0 msgr2=0x7f0f740d4570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:34.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.284+0000 7f0f73200640 1 -- 172.21.15.152:0/2334804537 shutdown_connections 2024-04-20T11:35:34.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.284+0000 7f0f73200640 1 -- 172.21.15.152:0/2334804537 wait complete. 2024-04-20T11:35:34.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.286+0000 7f0f73200640 1 Processor -- start 2024-04-20T11:35:34.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.286+0000 7f0f73200640 1 -- start start 2024-04-20T11:35:34.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.287+0000 7f0f73200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0f74161c70 0x7f0f740ca840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:34.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.287+0000 7f0f73200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0f74162920 0x7f0f741f5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:34.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.287+0000 7f0f73200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0f740a0420 0x7f0f740a1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:34.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.287+0000 7f0f73200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f0f740d0020 con 0x7f0f74162920 2024-04-20T11:35:34.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.288+0000 7f0f73200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0f740b7460 con 0x7f0f74161c70 2024-04-20T11:35:34.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.288+0000 7f0f73200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0f74162050 con 0x7f0f740a0420 2024-04-20T11:35:34.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.288+0000 7f0f71400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0f74162920 0x7f0f741f5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:34.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.288+0000 7f0f71400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0f74162920 0x7f0f741f5790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54584/0 (socket says 172.21.15.152:54584) 2024-04-20T11:35:34.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.288+0000 7f0f71400640 1 -- 172.21.15.152:0/328722037 learned_addr learned my addr 172.21.15.152:0/328722037 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:34.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.288+0000 7f0f72800640 1 --2- 172.21.15.152:0/328722037 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0f740a0420 0x7f0f740a1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:34.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.288+0000 7f0f71e00640 1 --2- 172.21.15.152:0/328722037 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0f74161c70 0x7f0f740ca840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:34.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.288+0000 7f0f71400640 1 -- 172.21.15.152:0/328722037 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0f740a0420 msgr2=0x7f0f740a1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:34.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.288+0000 7f0f71400640 1 --2- 172.21.15.152:0/328722037 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0f740a0420 0x7f0f740a1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:34.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.288+0000 7f0f71400640 1 -- 172.21.15.152:0/328722037 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0f74161c70 msgr2=0x7f0f740ca840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:34.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.288+0000 7f0f71400640 1 --2- 172.21.15.152:0/328722037 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0f74161c70 0x7f0f740ca840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:34.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.288+0000 7f0f71400640 1 -- 172.21.15.152:0/328722037 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f741f5cd0 con 0x7f0f74162920 2024-04-20T11:35:34.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.288+0000 7f0f72800640 1 --2- 172.21.15.152:0/328722037 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0f740a0420 0x7f0f740a1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:35:34.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.289+0000 7f0f71400640 1 --2- 172.21.15.152:0/328722037 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0f74162920 0x7f0f741f5790 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f0f6000e9b0 tx=0x7f0f6000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:34.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.289+0000 7f0f6a000640 1 -- 172.21.15.152:0/328722037 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f60017070 con 0x7f0f74162920 2024-04-20T11:35:34.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.289+0000 7f0f6a000640 1 -- 172.21.15.152:0/328722037 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f6000c9a0 con 0x7f0f74162920 2024-04-20T11:35:34.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.289+0000 7f0f6a000640 1 -- 172.21.15.152:0/328722037 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f6000cd90 con 0x7f0f74162920 2024-04-20T11:35:34.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.289+0000 7f0f73200640 1 -- 172.21.15.152:0/328722037 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f740a09c0 con 0x7f0f74162920 2024-04-20T11:35:34.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.289+0000 7f0f73200640 1 -- 172.21.15.152:0/328722037 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f740a0f80 con 0x7f0f74162920 2024-04-20T11:35:34.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.290+0000 7f0f73200640 1 -- 172.21.15.152:0/328722037 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f74075e60 con 0x7f0f74162920 2024-04-20T11:35:34.291 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.292+0000 7f0f6a000640 1 -- 172.21.15.152:0/328722037 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 21) ==== 91550+0+0 (secure 0 0 0) 0x7f0f6001a070 con 0x7f0f74162920 2024-04-20T11:35:34.292 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.293+0000 7f0f6a000640 1 --2- 172.21.15.152:0/328722037 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f0f5406ce90 0x7f0f5406f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:34.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.293+0000 7f0f6a000640 1 -- 172.21.15.152:0/328722037 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5709+0+0 (secure 0 0 0) 0x7f0f6008c5c0 con 0x7f0f74162920 2024-04-20T11:35:34.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.294+0000 7f0f71e00640 1 --2- 172.21.15.152:0/328722037 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f0f5406ce90 0x7f0f5406f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:34.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.295+0000 7f0f71e00640 1 --2- 172.21.15.152:0/328722037 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f0f5406ce90 0x7f0f5406f350 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f0f64036000 tx=0x7f0f6403b040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:34.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.298+0000 7f0f6a000640 1 -- 172.21.15.152:0/328722037 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f0f741f5cd0 con 0x7f0f74162920 2024-04-20T11:35:34.525 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.526+0000 7f0f73200640 1 -- 172.21.15.152:0/328722037 --> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "target": ["mon-mgr", ""]}) -- 0x7f0f740d5040 con 0x7f0f5406ce90 2024-04-20T11:35:34.529 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:35:34.529+0000 7f8a5cc00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-04-20T11:35:34.530 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.530+0000 7f0f6a000640 1 -- 172.21.15.152:0/328722037 <== mgr.4109 v2:172.21.15.175:6836/4021250441 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f0f740d5040 con 0x7f0f5406ce90 2024-04-20T11:35:34.530 INFO:teuthology.orchestra.run.smithi152.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-04-20T11:35:34.545 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-20T11:35:34.546 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm non_existent_subvolume subvolume_1 2024-04-20T11:35:34.750 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.750+0000 7effe6e00640 1 Processor -- start 2024-04-20T11:35:34.750 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.750+0000 7effe6e00640 1 -- start start 2024-04-20T11:35:34.751 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.752+0000 7effe6e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7effe00b7890 0x7effe00d2860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:34.751 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.752+0000 7effe6e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7effe00cf4f0 0x7effe00cf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:34.752 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.752+0000 7effe6e00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7effe00b7cd0 con 0x7effe0161e50 2024-04-20T11:35:34.752 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.752+0000 7effe6e00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7effe0162230 con 0x7effe00cf4f0 2024-04-20T11:35:34.752 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.752+0000 7effe6e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7effe00cfec0 con 0x7effe00b7890 2024-04-20T11:35:34.752 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.752+0000 7effdfe00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7effe0161e50 0x7effe00ba180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:56154/0 (socket says 172.21.15.152:56154) 2024-04-20T11:35:34.752 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.752+0000 7effdfe00640 1 -- 172.21.15.152:0/1975279366 learned_addr learned my addr 172.21.15.152:0/1975279366 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:35:34.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.752+0000 7effdf400640 1 --2- 172.21.15.152:0/1975279366 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7effe00b7890 0x7effe00d2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:34.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.753+0000 7effe4e00640 1 --2- 172.21.15.152:0/1975279366 >> v2:172.21.15.175:3300/0 conn(0x7effe00cf4f0 0x7effe00cf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:34.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.753+0000 7effdea00640 1 -- 172.21.15.152:0/1975279366 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3270765859 0 0) 0x7effe00b7cd0 con 0x7effe0161e50 2024-04-20T11:35:34.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.753+0000 7effdea00640 1 -- 172.21.15.152:0/1975279366 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7effc4003520 con 0x7effe0161e50 2024-04-20T11:35:34.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.753+0000 7effdea00640 1 -- 172.21.15.152:0/1975279366 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1290357795 0 0) 0x7effc4003520 con 0x7effe0161e50 2024-04-20T11:35:34.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.753+0000 7effdea00640 1 -- 172.21.15.152:0/1975279366 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7effe00b7890 msgr2=0x7effe00d2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:34.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.753+0000 7effdea00640 1 --2- 172.21.15.152:0/1975279366 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7effe00b7890 0x7effe00d2860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:34.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.753+0000 7effdea00640 1 -- 172.21.15.152:0/1975279366 >> v2:172.21.15.175:3300/0 conn(0x7effe00cf4f0 msgr2=0x7effe00cf980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:34.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.753+0000 7effdea00640 1 --2- 172.21.15.152:0/1975279366 >> v2:172.21.15.175:3300/0 conn(0x7effe00cf4f0 0x7effe00cf980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:34.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.753+0000 7effdea00640 1 -- 172.21.15.152:0/1975279366 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effe00b7f70 con 0x7effe0161e50 2024-04-20T11:35:34.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.753+0000 7effdea00640 1 -- 172.21.15.152:0/1975279366 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7effd0001670 con 0x7effe0161e50 2024-04-20T11:35:34.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.753+0000 7effdea00640 1 -- 172.21.15.152:0/1975279366 >> v1:172.21.15.152:6789/0 conn(0x7effe0161e50 legacy=0x7effe00ba180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:34.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.753+0000 7effdea00640 1 --2- 172.21.15.152:0/1975279366 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7effc40039b0 0x7effc4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:34.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.753+0000 7effdea00640 1 --2- 172.21.15.152:0/1975279366 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7effc4004800 0x7effc4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:34.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.753+0000 7effdea00640 1 --2- 172.21.15.152:0/1975279366 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7effe00b7890 0x7effc40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:34.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.753+0000 7effdea00640 1 -- 172.21.15.152:0/1975279366 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7effe00cfec0 con 0x7effc40039b0 2024-04-20T11:35:34.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.753+0000 7effdea00640 1 -- 172.21.15.152:0/1975279366 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7effc400b9c0 con 0x7effc4004800 2024-04-20T11:35:34.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.753+0000 7effdea00640 1 -- 172.21.15.152:0/1975279366 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7effc400bbc0 con 0x7effe00b7890 2024-04-20T11:35:34.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.753+0000 7effdfe00640 1 --2- 172.21.15.152:0/1975279366 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7effc40039b0 0x7effc4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:34.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.754+0000 7effe4e00640 1 --2- 172.21.15.152:0/1975279366 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7effe00b7890 0x7effc40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:34.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.754+0000 7effdf400640 1 --2- 172.21.15.152:0/1975279366 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7effc4004800 0x7effc4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:34.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.754+0000 7effdfe00640 1 -- 172.21.15.152:0/1975279366 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7effe00b7890 msgr2=0x7effc40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:34.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.754+0000 7effdfe00640 1 --2- 172.21.15.152:0/1975279366 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7effe00b7890 0x7effc40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:34.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.754+0000 7effdfe00640 1 -- 172.21.15.152:0/1975279366 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7effc4004800 msgr2=0x7effc4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:34.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.754+0000 7effdfe00640 1 --2- 172.21.15.152:0/1975279366 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7effc4004800 0x7effc4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:34.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.754+0000 7effdfe00640 1 -- 172.21.15.152:0/1975279366 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effc4003520 con 0x7effc40039b0 2024-04-20T11:35:34.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.754+0000 7effdf400640 1 --2- 172.21.15.152:0/1975279366 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7effc4004800 0x7effc4004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:35:34.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.755+0000 7effdfe00640 1 --2- 172.21.15.152:0/1975279366 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7effc40039b0 0x7effc4003dd0 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7effd000c190 tx=0x7effc4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae979b6f0c5f7b5d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:34.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.756+0000 7effdea00640 1 -- 172.21.15.152:0/1975279366 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effd0007c60 con 0x7effc40039b0 2024-04-20T11:35:34.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.756+0000 7effdea00640 1 -- 172.21.15.152:0/1975279366 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effd0030070 con 0x7effc40039b0 2024-04-20T11:35:34.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.756+0000 7effdea00640 1 -- 172.21.15.152:0/1975279366 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effd0005d30 con 0x7effc40039b0 2024-04-20T11:35:34.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.756+0000 7effe6e00640 1 -- 172.21.15.152:0/1975279366 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7effc40039b0 msgr2=0x7effc4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:34.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.756+0000 7effe6e00640 1 --2- 172.21.15.152:0/1975279366 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7effc40039b0 0x7effc4003dd0 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7effd000c190 tx=0x7effc4005830 comp rx=0 tx=0).stop 2024-04-20T11:35:34.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.756+0000 7effe4e00640 1 -- 172.21.15.152:0/1975279366 reap_dead start 2024-04-20T11:35:34.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.756+0000 7effe6e00640 1 -- 172.21.15.152:0/1975279366 shutdown_connections 2024-04-20T11:35:34.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.756+0000 7effe6e00640 1 -- 172.21.15.152:0/1975279366 >> 172.21.15.152:0/1975279366 conn(0x7effe0067ae0 msgr2=0x7effe0067ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:34.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.756+0000 7effe6e00640 1 -- 172.21.15.152:0/1975279366 shutdown_connections 2024-04-20T11:35:34.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.756+0000 7effe6e00640 1 -- 172.21.15.152:0/1975279366 wait complete. 2024-04-20T11:35:34.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.758+0000 7effe6e00640 1 Processor -- start 2024-04-20T11:35:34.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.758+0000 7effe6e00640 1 -- start start 2024-04-20T11:35:34.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.759+0000 7effe6e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7effc40039b0 0x7effe01f6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:34.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.759+0000 7effe6e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7effe015e270 0x7effe01f7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:34.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.759+0000 7effe6e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7effe015b020 0x7effe01475e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:34.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.759+0000 7effe6e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7effe00b8810 con 0x7effe015b020 2024-04-20T11:35:34.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.759+0000 7effe6e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7effe00b8530 con 0x7effc40039b0 2024-04-20T11:35:34.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.759+0000 7effe6e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7effe00b8af0 con 0x7effe015e270 2024-04-20T11:35:34.762 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.760+0000 7effe4e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7effe015b020 0x7effe01475e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:34.762 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.760+0000 7effe4e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7effe015b020 0x7effe01475e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54600/0 (socket says 172.21.15.152:54600) 2024-04-20T11:35:34.762 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.760+0000 7effe4e00640 1 -- 172.21.15.152:0/2562283973 learned_addr learned my addr 172.21.15.152:0/2562283973 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:34.762 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.760+0000 7effdfe00640 1 --2- 172.21.15.152:0/2562283973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7effc40039b0 0x7effe01f6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:34.762 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.760+0000 7effe4e00640 1 -- 172.21.15.152:0/2562283973 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7effe015e270 msgr2=0x7effe01f7230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:34.762 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.760+0000 7effdf400640 1 --2- 172.21.15.152:0/2562283973 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7effe015e270 0x7effe01f7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:34.762 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.760+0000 7effe4e00640 1 --2- 172.21.15.152:0/2562283973 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7effe015e270 0x7effe01f7230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:34.762 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.760+0000 7effe4e00640 1 -- 172.21.15.152:0/2562283973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7effc40039b0 msgr2=0x7effe01f6cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:34.762 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.760+0000 7effe4e00640 1 --2- 172.21.15.152:0/2562283973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7effc40039b0 0x7effe01f6cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:34.762 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.760+0000 7effe4e00640 1 -- 172.21.15.152:0/2562283973 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effe015ea10 con 0x7effe015b020 2024-04-20T11:35:34.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.760+0000 7effdfe00640 1 --2- 172.21.15.152:0/2562283973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7effc40039b0 0x7effe01f6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:35:34.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.760+0000 7effe4e00640 1 --2- 172.21.15.152:0/2562283973 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7effe015b020 0x7effe01475e0 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7effe00b70c0 tx=0x7effd80115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:34.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.761+0000 7effd7e00640 1 -- 172.21.15.152:0/2562283973 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effd8011e40 con 0x7effe015b020 2024-04-20T11:35:34.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.761+0000 7effd7e00640 1 -- 172.21.15.152:0/2562283973 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effd801b070 con 0x7effe015b020 2024-04-20T11:35:34.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.761+0000 7effd7e00640 1 -- 172.21.15.152:0/2562283973 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effd8003990 con 0x7effe015b020 2024-04-20T11:35:34.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.761+0000 7effe6e00640 1 -- 172.21.15.152:0/2562283973 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effe015e730 con 0x7effe015b020 2024-04-20T11:35:34.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.761+0000 7effe6e00640 1 -- 172.21.15.152:0/2562283973 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effe015a820 con 0x7effe015b020 2024-04-20T11:35:34.764 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.762+0000 7effe6e00640 1 -- 172.21.15.152:0/2562283973 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effe00689f0 con 0x7effe015b020 2024-04-20T11:35:34.764 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.763+0000 7effd7e00640 1 -- 172.21.15.152:0/2562283973 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 21) ==== 91550+0+0 (secure 0 0 0) 0x7effd8005d10 con 0x7effe015b020 2024-04-20T11:35:34.765 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.764+0000 7effd7e00640 1 --2- 172.21.15.152:0/2562283973 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7effc006cec0 0x7effc006f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:34.765 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.764+0000 7effd7e00640 1 -- 172.21.15.152:0/2562283973 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5709+0+0 (secure 0 0 0) 0x7effd808c990 con 0x7effe015b020 2024-04-20T11:35:34.765 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.765+0000 7effdfe00640 1 --2- 172.21.15.152:0/2562283973 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7effc006cec0 0x7effc006f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:34.765 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.766+0000 7effdfe00640 1 --2- 172.21.15.152:0/2562283973 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7effc006cec0 0x7effc006f380 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7effd0034000 tx=0x7effd0007000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:34.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:34.768+0000 7effd7e00640 1 -- 172.21.15.152:0/2562283973 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7effd8025020 con 0x7effe015b020 2024-04-20T11:35:35.000 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.000+0000 7effe6e00640 1 -- 172.21.15.152:0/2562283973 --> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "target": ["mon-mgr", ""]}) -- 0x7effe015cd50 con 0x7effc006cec0 2024-04-20T11:35:35.003 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:35:35.003+0000 7f8a5cc00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-04-20T11:35:35.003 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.004+0000 7effd7e00640 1 -- 172.21.15.152:0/2562283973 <== mgr.4109 v2:172.21.15.175:6836/4021250441 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7effe015cd50 con 0x7effc006cec0 2024-04-20T11:35:35.003 INFO:teuthology.orchestra.run.smithi152.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-04-20T11:35:35.025 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-20T11:35:35.026 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath non_existent_subvolume subvolume_1 2024-04-20T11:35:35.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.235+0000 7feff8800640 1 Processor -- start 2024-04-20T11:35:35.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.235+0000 7feff8800640 1 -- start start 2024-04-20T11:35:35.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.237+0000 7feff8800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7feff00b2880 0x7feff015fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:35.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.237+0000 7feff8800640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feff015f1c0 con 0x7feff00cfc40 2024-04-20T11:35:35.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.237+0000 7feff8800640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feff006f670 con 0x7feff015b510 2024-04-20T11:35:35.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.237+0000 7feff8800640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7feff006f420 con 0x7feff00b2880 2024-04-20T11:35:35.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.237+0000 7feff6800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7feff00b2880 0x7feff015fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:35.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.237+0000 7feff5400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7feff00cfc40 0x7feff00b4800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:56170/0 (socket says 172.21.15.152:56170) 2024-04-20T11:35:35.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.237+0000 7feff5400640 1 -- 172.21.15.152:0/4174078944 learned_addr learned my addr 172.21.15.152:0/4174078944 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:35:35.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.238+0000 7feff6800640 1 -- 172.21.15.152:0/4174078944 >> v1:172.21.15.175:6789/0 conn(0x7feff015b510 legacy=0x7feff00b2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:35.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.238+0000 7feff6800640 1 -- 172.21.15.152:0/4174078944 >> v1:172.21.15.152:6789/0 conn(0x7feff00cfc40 legacy=0x7feff00b4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:35.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.238+0000 7feff4a00640 1 -- 172.21.15.152:0/4174078944 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 369802746 0 0) 0x7feff015f1c0 con 0x7feff00cfc40 2024-04-20T11:35:35.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.238+0000 7feff6800640 1 -- 172.21.15.152:0/4174078944 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feff015f430 con 0x7feff00b2880 2024-04-20T11:35:35.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.239+0000 7feff6800640 1 --2- 172.21.15.152:0/4174078944 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feff00b2880 0x7feff015fc30 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7fefec009d90 tx=0x7fefec02be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a524b9ed87ce7ac server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:35.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.239+0000 7feff4a00640 1 -- 172.21.15.152:0/4174078944 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefec039070 con 0x7feff00b2880 2024-04-20T11:35:35.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.239+0000 7feff4a00640 1 -- 172.21.15.152:0/4174078944 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefec02cbd0 con 0x7feff00b2880 2024-04-20T11:35:35.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.239+0000 7feff4a00640 1 -- 172.21.15.152:0/4174078944 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefec034440 con 0x7feff00b2880 2024-04-20T11:35:35.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.240+0000 7feff8800640 1 -- 172.21.15.152:0/4174078944 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feff00b2880 msgr2=0x7feff015fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:35.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.240+0000 7feff8800640 1 --2- 172.21.15.152:0/4174078944 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feff00b2880 0x7feff015fc30 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7fefec009d90 tx=0x7fefec02be20 comp rx=0 tx=0).stop 2024-04-20T11:35:35.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.240+0000 7feff8800640 1 -- 172.21.15.152:0/4174078944 shutdown_connections 2024-04-20T11:35:35.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.240+0000 7feff8800640 1 --2- 172.21.15.152:0/4174078944 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feff00b2880 0x7feff015fc30 unknown :-1 s=CLOSED pgs=1796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:35.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.240+0000 7feff8800640 1 -- 172.21.15.152:0/4174078944 >> 172.21.15.152:0/4174078944 conn(0x7feff006b980 msgr2=0x7feff006bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:35.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.240+0000 7feff8800640 1 -- 172.21.15.152:0/4174078944 shutdown_connections 2024-04-20T11:35:35.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.240+0000 7feff8800640 1 -- 172.21.15.152:0/4174078944 wait complete. 2024-04-20T11:35:35.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.241+0000 7feff8800640 1 Processor -- start 2024-04-20T11:35:35.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.242+0000 7feff8800640 1 -- start start 2024-04-20T11:35:35.243 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.242+0000 7feff8800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feff00b2880 0x7feff0159b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:35.243 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.242+0000 7feff8800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feff00cfc40 0x7feff0144270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:35.243 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.243+0000 7feff8800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feff015b510 0x7feff00ce320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:35.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.243+0000 7feff8800640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7feff006f950 con 0x7feff015b510 2024-04-20T11:35:35.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.243+0000 7feff8800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7feff015b8f0 con 0x7feff00b2880 2024-04-20T11:35:35.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.243+0000 7feff8800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7feff00b3ef0 con 0x7feff00cfc40 2024-04-20T11:35:35.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.243+0000 7feff6800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feff015b510 0x7feff00ce320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:35.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.243+0000 7feff6800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feff015b510 0x7feff00ce320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54624/0 (socket says 172.21.15.152:54624) 2024-04-20T11:35:35.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.243+0000 7feff6800640 1 -- 172.21.15.152:0/2600406844 learned_addr learned my addr 172.21.15.152:0/2600406844 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:35.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.243+0000 7feff6800640 1 -- 172.21.15.152:0/2600406844 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feff00cfc40 msgr2=0x7feff0144270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:35.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.243+0000 7feff6800640 1 --2- 172.21.15.152:0/2600406844 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feff00cfc40 0x7feff0144270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:35.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.243+0000 7feff6800640 1 -- 172.21.15.152:0/2600406844 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feff00b2880 msgr2=0x7feff0159b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:35.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.243+0000 7feff6800640 1 --2- 172.21.15.152:0/2600406844 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feff00b2880 0x7feff0159b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:35.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.243+0000 7feff6800640 1 -- 172.21.15.152:0/2600406844 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feff015e5f0 con 0x7feff015b510 2024-04-20T11:35:35.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.244+0000 7feff6800640 1 --2- 172.21.15.152:0/2600406844 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feff015b510 0x7feff00ce320 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7fefec009d90 tx=0x7fefec004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:35.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.244+0000 7fefea000640 1 -- 172.21.15.152:0/2600406844 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefec044020 con 0x7feff015b510 2024-04-20T11:35:35.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.244+0000 7fefea000640 1 -- 172.21.15.152:0/2600406844 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefec03f470 con 0x7feff015b510 2024-04-20T11:35:35.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.244+0000 7fefea000640 1 -- 172.21.15.152:0/2600406844 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefec039040 con 0x7feff015b510 2024-04-20T11:35:35.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.244+0000 7feff8800640 1 -- 172.21.15.152:0/2600406844 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feff015e8d0 con 0x7feff015b510 2024-04-20T11:35:35.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.244+0000 7feff8800640 1 -- 172.21.15.152:0/2600406844 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feff014feb0 con 0x7feff015b510 2024-04-20T11:35:35.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.245+0000 7feff8800640 1 -- 172.21.15.152:0/2600406844 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feff00689f0 con 0x7feff015b510 2024-04-20T11:35:35.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.246+0000 7fefea000640 1 -- 172.21.15.152:0/2600406844 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 21) ==== 91550+0+0 (secure 0 0 0) 0x7fefec047020 con 0x7feff015b510 2024-04-20T11:35:35.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.247+0000 7fefea000640 1 --2- 172.21.15.152:0/2600406844 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fefd406cec0 0x7fefd406f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:35.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.248+0000 7fefea000640 1 -- 172.21.15.152:0/2600406844 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5709+0+0 (secure 0 0 0) 0x7fefec0aefe0 con 0x7feff015b510 2024-04-20T11:35:35.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.248+0000 7feff5e00640 1 --2- 172.21.15.152:0/2600406844 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fefd406cec0 0x7fefd406f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:35.249 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.249+0000 7feff5e00640 1 --2- 172.21.15.152:0/2600406844 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fefd406cec0 0x7fefd406f380 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fefe4006fd0 tx=0x7fefe4006bc0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:35.251 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.252+0000 7fefea000640 1 -- 172.21.15.152:0/2600406844 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fefec04e070 con 0x7feff015b510 2024-04-20T11:35:35.480 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.481+0000 7feff8800640 1 -- 172.21.15.152:0/2600406844 --> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "target": ["mon-mgr", ""]}) -- 0x7feff015a090 con 0x7fefd406cec0 2024-04-20T11:35:35.484 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:35:35.484+0000 7f8a5cc00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-04-20T11:35:35.484 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.485+0000 7fefea000640 1 -- 172.21.15.152:0/2600406844 <== mgr.4109 v2:172.21.15.175:6836/4021250441 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7fefe4004020 con 0x7fefd406cec0 2024-04-20T11:35:35.484 INFO:teuthology.orchestra.run.smithi152.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-04-20T11:35:35.500 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-20T11:35:35.501 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume info non_existent_subvolume subvolume_1 2024-04-20T11:35:35.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.708+0000 7f8c2fc00640 1 Processor -- start 2024-04-20T11:35:35.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.708+0000 7f8c2fc00640 1 -- start start 2024-04-20T11:35:35.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.710+0000 7f8c2fc00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f8c280b2880 0x7f8c280b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:35.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.710+0000 7f8c2fc00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8c280cd1b0 0x7f8c280cd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:35.710 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.710+0000 7f8c2fc00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c281463e0 con 0x7f8c28146000 2024-04-20T11:35:35.710 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.710+0000 7f8c2fc00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f8c280b4750 con 0x7f8c280b2880 2024-04-20T11:35:35.710 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.710+0000 7f8c2fc00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8c280b3e20 con 0x7f8c280cd1b0 2024-04-20T11:35:35.711 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.711+0000 7f8c2d200640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7f8c28146000 0x7f8c280b2170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.152:54158/0 (socket says 172.21.15.152:54158) 2024-04-20T11:35:35.711 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.711+0000 7f8c2d200640 1 -- 172.21.15.152:0/3877415784 learned_addr learned my addr 172.21.15.152:0/3877415784 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:35:35.711 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.711+0000 7f8c2dc00640 1 --2- 172.21.15.152:0/3877415784 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8c280cd1b0 0x7f8c280cd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:35.711 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.711+0000 7f8c27e00640 1 --2- 172.21.15.152:0/3877415784 >> v2:172.21.15.175:3300/0 conn(0x7f8c280b2880 0x7f8c280b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:35.711 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.712+0000 7f8c27400640 1 -- 172.21.15.152:0/3877415784 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1263053073 0 0) 0x7f8c281463e0 con 0x7f8c28146000 2024-04-20T11:35:35.711 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.712+0000 7f8c2dc00640 1 -- 172.21.15.152:0/3877415784 >> v2:172.21.15.175:3300/0 conn(0x7f8c280b2880 msgr2=0x7f8c280b4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:35.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.712+0000 7f8c2dc00640 1 --2- 172.21.15.152:0/3877415784 >> v2:172.21.15.175:3300/0 conn(0x7f8c280b2880 0x7f8c280b4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:35.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.712+0000 7f8c2dc00640 1 -- 172.21.15.152:0/3877415784 >> v1:172.21.15.175:6789/0 conn(0x7f8c28146000 legacy=0x7f8c280b2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:35.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.712+0000 7f8c2dc00640 1 -- 172.21.15.152:0/3877415784 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c280b4100 con 0x7f8c280cd1b0 2024-04-20T11:35:35.713 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.713+0000 7f8c2dc00640 1 --2- 172.21.15.152:0/3877415784 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8c280cd1b0 0x7f8c280cd5f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8c20009d90 tx=0x7f8c2002be20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8958fffc3ef1f10 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:35.713 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.713+0000 7f8c27400640 1 -- 172.21.15.152:0/3877415784 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c20039070 con 0x7f8c280cd1b0 2024-04-20T11:35:35.713 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.713+0000 7f8c27400640 1 -- 172.21.15.152:0/3877415784 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c2002cbd0 con 0x7f8c280cd1b0 2024-04-20T11:35:35.713 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.713+0000 7f8c27400640 1 -- 172.21.15.152:0/3877415784 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c20034440 con 0x7f8c280cd1b0 2024-04-20T11:35:35.714 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.714+0000 7f8c2fc00640 1 -- 172.21.15.152:0/3877415784 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8c280cd1b0 msgr2=0x7f8c280cd5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:35.714 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.714+0000 7f8c2fc00640 1 --2- 172.21.15.152:0/3877415784 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8c280cd1b0 0x7f8c280cd5f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8c20009d90 tx=0x7f8c2002be20 comp rx=0 tx=0).stop 2024-04-20T11:35:35.714 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.714+0000 7f8c2fc00640 1 -- 172.21.15.152:0/3877415784 shutdown_connections 2024-04-20T11:35:35.714 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.714+0000 7f8c2fc00640 1 --2- 172.21.15.152:0/3877415784 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8c280cd1b0 0x7f8c280cd5f0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:35.715 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.714+0000 7f8c2fc00640 1 --2- 172.21.15.152:0/3877415784 >> v2:172.21.15.175:3300/0 conn(0x7f8c280b2880 0x7f8c280b4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:35.715 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.714+0000 7f8c2fc00640 1 -- 172.21.15.152:0/3877415784 >> 172.21.15.152:0/3877415784 conn(0x7f8c2806b980 msgr2=0x7f8c2806bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:35.715 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.714+0000 7f8c2fc00640 1 -- 172.21.15.152:0/3877415784 shutdown_connections 2024-04-20T11:35:35.715 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.714+0000 7f8c2fc00640 1 -- 172.21.15.152:0/3877415784 wait complete. 2024-04-20T11:35:35.715 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.716+0000 7f8c2fc00640 1 Processor -- start 2024-04-20T11:35:35.716 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.716+0000 7f8c2fc00640 1 -- start start 2024-04-20T11:35:35.719 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.717+0000 7f8c2fc00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8c280b2880 0x7f8c28144410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:35.719 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.717+0000 7f8c2fc00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c280cd1b0 0x7f8c2815e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:35.719 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.717+0000 7f8c2fc00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8c28146000 0x7f8c2815e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:35.719 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.717+0000 7f8c2fc00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f8c280cdb30 con 0x7f8c280cd1b0 2024-04-20T11:35:35.719 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.717+0000 7f8c2fc00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8c2815ccf0 con 0x7f8c28146000 2024-04-20T11:35:35.719 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.717+0000 7f8c2fc00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8c28144950 con 0x7f8c280b2880 2024-04-20T11:35:35.719 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.718+0000 7f8c27e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c280cd1b0 0x7f8c2815e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:35.719 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.718+0000 7f8c27e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c280cd1b0 0x7f8c2815e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54628/0 (socket says 172.21.15.152:54628) 2024-04-20T11:35:35.720 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.718+0000 7f8c27e00640 1 -- 172.21.15.152:0/1521674250 learned_addr learned my addr 172.21.15.152:0/1521674250 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:35.720 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.718+0000 7f8c2d200640 1 --2- 172.21.15.152:0/1521674250 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8c280b2880 0x7f8c28144410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:35.720 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.718+0000 7f8c2dc00640 1 --2- 172.21.15.152:0/1521674250 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8c28146000 0x7f8c2815e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:35.720 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.718+0000 7f8c27e00640 1 -- 172.21.15.152:0/1521674250 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8c280b2880 msgr2=0x7f8c28144410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:35.720 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.718+0000 7f8c27e00640 1 --2- 172.21.15.152:0/1521674250 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8c280b2880 0x7f8c28144410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:35.720 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.718+0000 7f8c27e00640 1 -- 172.21.15.152:0/1521674250 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8c28146000 msgr2=0x7f8c2815e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:35.720 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.718+0000 7f8c27e00640 1 --2- 172.21.15.152:0/1521674250 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8c28146000 0x7f8c2815e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:35.720 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.718+0000 7f8c27e00640 1 -- 172.21.15.152:0/1521674250 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c281e7500 con 0x7f8c280cd1b0 2024-04-20T11:35:35.720 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.718+0000 7f8c27e00640 1 --2- 172.21.15.152:0/1521674250 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c280cd1b0 0x7f8c2815e100 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7f8c1400e8e0 tx=0x7f8c1400eda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:35.720 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.719+0000 7f8c24c00640 1 -- 172.21.15.152:0/1521674250 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c14017070 con 0x7f8c280cd1b0 2024-04-20T11:35:35.720 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.719+0000 7f8c24c00640 1 -- 172.21.15.152:0/1521674250 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c1400c940 con 0x7f8c280cd1b0 2024-04-20T11:35:35.720 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.719+0000 7f8c2fc00640 1 -- 172.21.15.152:0/1521674250 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c281e77e0 con 0x7f8c280cd1b0 2024-04-20T11:35:35.720 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.719+0000 7f8c24c00640 1 -- 172.21.15.152:0/1521674250 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c1400cd30 con 0x7f8c280cd1b0 2024-04-20T11:35:35.720 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.719+0000 7f8c2fc00640 1 -- 172.21.15.152:0/1521674250 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c281e7da0 con 0x7f8c280cd1b0 2024-04-20T11:35:35.721 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.720+0000 7f8c2fc00640 1 -- 172.21.15.152:0/1521674250 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c280689f0 con 0x7f8c280cd1b0 2024-04-20T11:35:35.721 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.721+0000 7f8c24c00640 1 -- 172.21.15.152:0/1521674250 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 21) ==== 91550+0+0 (secure 0 0 0) 0x7f8c1401a070 con 0x7f8c280cd1b0 2024-04-20T11:35:35.722 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.722+0000 7f8c24c00640 1 --2- 172.21.15.152:0/1521674250 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f8c0806ce90 0x7f8c0806f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:35.722 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.723+0000 7f8c24c00640 1 -- 172.21.15.152:0/1521674250 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5709+0+0 (secure 0 0 0) 0x7f8c1408ca80 con 0x7f8c280cd1b0 2024-04-20T11:35:35.722 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.723+0000 7f8c2d200640 1 --2- 172.21.15.152:0/1521674250 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f8c0806ce90 0x7f8c0806f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:35.723 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.724+0000 7f8c2d200640 1 --2- 172.21.15.152:0/1521674250 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f8c0806ce90 0x7f8c0806f350 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c000f60 tx=0x7f8c1c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:35.727 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.728+0000 7f8c24c00640 1 -- 172.21.15.152:0/1521674250 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f8c1405a600 con 0x7f8c280cd1b0 2024-04-20T11:35:35.966 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.966+0000 7f8c2fc00640 1 -- 172.21.15.152:0/1521674250 --> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] -- mgr_command(tid 0: {"prefix": "fs subvolume info", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "target": ["mon-mgr", ""]}) -- 0x7f8c2815c7f0 con 0x7f8c0806ce90 2024-04-20T11:35:35.969 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:35:35.969+0000 7f8a5cc00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-04-20T11:35:35.969 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:35.970+0000 7f8c24c00640 1 -- 172.21.15.152:0/1521674250 <== mgr.4109 v2:172.21.15.175:6836/4021250441 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f8c2815c7f0 con 0x7f8c0806ce90 2024-04-20T11:35:35.970 INFO:teuthology.orchestra.run.smithi152.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-04-20T11:35:35.984 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-20T11:35:35.985 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume resize non_existent_subvolume subvolname_1 inf 2024-04-20T11:35:36.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.189+0000 7f8325400640 1 Processor -- start 2024-04-20T11:35:36.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.189+0000 7f8325400640 1 -- start start 2024-04-20T11:35:36.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.191+0000 7f8325400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f83200d05f0 0x7f83200d09d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:36.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.191+0000 7f8325400640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83200d0f10 con 0x7f8320071f20 2024-04-20T11:35:36.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.191+0000 7f8325400640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83200aeef0 con 0x7f83200ae180 2024-04-20T11:35:36.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.191+0000 7f8325400640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f83200af1d0 con 0x7f83200d05f0 2024-04-20T11:35:36.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.191+0000 7f831fe00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f83200d05f0 0x7f83200d09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:36.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.191+0000 7f831fe00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f83200d05f0 0x7f83200d09d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:37834/0 (socket says 172.21.15.152:37834) 2024-04-20T11:35:36.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.191+0000 7f831fe00640 1 -- 172.21.15.152:0/1010042500 learned_addr learned my addr 172.21.15.152:0/1010042500 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:36.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.192+0000 7f831fe00640 1 -- 172.21.15.152:0/1010042500 >> v1:172.21.15.175:6789/0 conn(0x7f83200ae180 legacy=0x7f83200ae600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:36.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.192+0000 7f831fe00640 1 -- 172.21.15.152:0/1010042500 >> v1:172.21.15.152:6789/0 conn(0x7f8320071f20 legacy=0x7f8320072370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:36.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.192+0000 7f831fe00640 1 -- 172.21.15.152:0/1010042500 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83200af4b0 con 0x7f83200d05f0 2024-04-20T11:35:36.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.193+0000 7f831fe00640 1 --2- 172.21.15.152:0/1010042500 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f83200d05f0 0x7f83200d09d0 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7f8310009870 tx=0x7f831002b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=333594a0b88c115e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:36.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.193+0000 7f831f400640 1 -- 172.21.15.152:0/1010042500 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8310034070 con 0x7f83200d05f0 2024-04-20T11:35:36.193 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.193+0000 7f831f400640 1 -- 172.21.15.152:0/1010042500 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8310030850 con 0x7f83200d05f0 2024-04-20T11:35:36.193 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.193+0000 7f831f400640 1 -- 172.21.15.152:0/1010042500 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8310030c20 con 0x7f83200d05f0 2024-04-20T11:35:36.193 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.194+0000 7f8325400640 1 -- 172.21.15.152:0/1010042500 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f83200d05f0 msgr2=0x7f83200d09d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:36.193 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.194+0000 7f8325400640 1 --2- 172.21.15.152:0/1010042500 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f83200d05f0 0x7f83200d09d0 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7f8310009870 tx=0x7f831002b990 comp rx=0 tx=0).stop 2024-04-20T11:35:36.194 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.194+0000 7f8325400640 1 -- 172.21.15.152:0/1010042500 shutdown_connections 2024-04-20T11:35:36.194 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.194+0000 7f8325400640 1 --2- 172.21.15.152:0/1010042500 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f83200d05f0 0x7f83200d09d0 unknown :-1 s=CLOSED pgs=1800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:36.194 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.194+0000 7f8325400640 1 -- 172.21.15.152:0/1010042500 >> 172.21.15.152:0/1010042500 conn(0x7f83200b21e0 msgr2=0x7f83200b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:36.194 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.194+0000 7f8325400640 1 -- 172.21.15.152:0/1010042500 shutdown_connections 2024-04-20T11:35:36.194 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.194+0000 7f8325400640 1 -- 172.21.15.152:0/1010042500 wait complete. 2024-04-20T11:35:36.195 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.196+0000 7f8325400640 1 Processor -- start 2024-04-20T11:35:36.195 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.196+0000 7f8325400640 1 -- start start 2024-04-20T11:35:36.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.197+0000 7f8325400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8320071f20 0x7f8320155fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:36.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.197+0000 7f8325400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f83200ae180 0x7f832009d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:36.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.197+0000 7f8325400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f83200d05f0 0x7f83200ce030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:36.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.197+0000 7f8325400640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f8320155ca0 con 0x7f8320071f20 2024-04-20T11:35:36.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.197+0000 7f8325400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f83200afa70 con 0x7f83200ae180 2024-04-20T11:35:36.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.197+0000 7f8325400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f832009da30 con 0x7f83200d05f0 2024-04-20T11:35:36.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.197+0000 7f831fe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8320071f20 0x7f8320155fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:36.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.197+0000 7f831fe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8320071f20 0x7f8320155fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:37838/0 (socket says 172.21.15.152:37838) 2024-04-20T11:35:36.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.197+0000 7f831fe00640 1 -- 172.21.15.152:0/235245924 learned_addr learned my addr 172.21.15.152:0/235245924 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:36.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.198+0000 7f831fe00640 1 -- 172.21.15.152:0/235245924 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f83200d05f0 msgr2=0x7f83200ce030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:36.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.198+0000 7f831fe00640 1 --2- 172.21.15.152:0/235245924 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f83200d05f0 0x7f83200ce030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:36.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.198+0000 7f831fe00640 1 -- 172.21.15.152:0/235245924 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f83200ae180 msgr2=0x7f832009d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:36.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.198+0000 7f831fe00640 1 --2- 172.21.15.152:0/235245924 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f83200ae180 0x7f832009d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:36.199 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.198+0000 7f831fe00640 1 -- 172.21.15.152:0/235245924 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83200ce7d0 con 0x7f8320071f20 2024-04-20T11:35:36.199 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.198+0000 7f831fe00640 1 --2- 172.21.15.152:0/235245924 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8320071f20 0x7f8320155fb0 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f8310000c00 tx=0x7f8310004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:36.199 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.198+0000 7f831cc00640 1 -- 172.21.15.152:0/235245924 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8310048070 con 0x7f8320071f20 2024-04-20T11:35:36.199 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.198+0000 7f831cc00640 1 -- 172.21.15.152:0/235245924 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8310039ce0 con 0x7f8320071f20 2024-04-20T11:35:36.199 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.198+0000 7f831cc00640 1 -- 172.21.15.152:0/235245924 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8310034040 con 0x7f8320071f20 2024-04-20T11:35:36.199 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.198+0000 7f8325400640 1 -- 172.21.15.152:0/235245924 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83200ceab0 con 0x7f8320071f20 2024-04-20T11:35:36.199 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.198+0000 7f8325400640 1 -- 172.21.15.152:0/235245924 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83200cf070 con 0x7f8320071f20 2024-04-20T11:35:36.199 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.200+0000 7f8325400640 1 -- 172.21.15.152:0/235245924 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8320075e60 con 0x7f8320071f20 2024-04-20T11:35:36.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.200+0000 7f831cc00640 1 -- 172.21.15.152:0/235245924 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 21) ==== 91550+0+0 (secure 0 0 0) 0x7f831004b070 con 0x7f8320071f20 2024-04-20T11:35:36.201 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.201+0000 7f831cc00640 1 --2- 172.21.15.152:0/235245924 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f82fc06ce90 0x7f82fc06f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:36.201 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.202+0000 7f831cc00640 1 -- 172.21.15.152:0/235245924 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5709+0+0 (secure 0 0 0) 0x7f83100ae7d0 con 0x7f8320071f20 2024-04-20T11:35:36.201 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.202+0000 7f8317e00640 1 --2- 172.21.15.152:0/235245924 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f82fc06ce90 0x7f82fc06f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:36.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.203+0000 7f8317e00640 1 --2- 172.21.15.152:0/235245924 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f82fc06ce90 0x7f82fc06f350 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f8320144ee0 tx=0x7f8308007990 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:36.205 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.206+0000 7f831cc00640 1 -- 172.21.15.152:0/235245924 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f831007c310 con 0x7f8320071f20 2024-04-20T11:35:36.438 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.439+0000 7f8325400640 1 -- 172.21.15.152:0/235245924 --> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] -- mgr_command(tid 0: {"prefix": "fs subvolume resize", "vol_name": "non_existent_subvolume", "sub_name": "subvolname_1", "new_size": "inf", "target": ["mon-mgr", ""]}) -- 0x7f83200ce570 con 0x7f82fc06ce90 2024-04-20T11:35:36.442 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:35:36.442+0000 7f8a5cc00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-04-20T11:35:36.443 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.443+0000 7f831cc00640 1 -- 172.21.15.152:0/235245924 <== mgr.4109 v2:172.21.15.175:6836/4021250441 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f83080050d0 con 0x7f82fc06ce90 2024-04-20T11:35:36.443 INFO:teuthology.orchestra.run.smithi152.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-04-20T11:35:36.457 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-20T11:35:36.458 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume pin non_existent_subvolume subvolname_1 export 1 2024-04-20T11:35:36.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.665+0000 7fb176a00640 1 Processor -- start 2024-04-20T11:35:36.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.666+0000 7fb176a00640 1 -- start start 2024-04-20T11:35:36.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.667+0000 7fb176a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fb170145340 0x7fb170160db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:36.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.668+0000 7fb176a00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1700b4f90 con 0x7fb1700b47a0 2024-04-20T11:35:36.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.668+0000 7fb176a00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1700b4cb0 con 0x7fb170072530 2024-04-20T11:35:36.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.668+0000 7fb176a00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fb170145720 con 0x7fb170145340 2024-04-20T11:35:36.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.668+0000 7fb16fe00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fb170145340 0x7fb170160db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:36.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.668+0000 7fb167400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fb1700b47a0 0x7fb170071e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:40496/0 (socket says 172.21.15.152:40496) 2024-04-20T11:35:36.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.668+0000 7fb167400640 1 -- 172.21.15.152:0/3110198953 learned_addr learned my addr 172.21.15.152:0/3110198953 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:35:36.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.668+0000 7fb16f400640 1 -- 172.21.15.152:0/3110198953 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1100156524 0 0) 0x7fb1700b4f90 con 0x7fb1700b47a0 2024-04-20T11:35:36.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.669+0000 7fb16f400640 1 -- 172.21.15.152:0/3110198953 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb154003520 con 0x7fb1700b47a0 2024-04-20T11:35:36.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.669+0000 7fb16fe00640 1 -- 172.21.15.152:0/3110198953 >> v1:172.21.15.175:6789/0 conn(0x7fb170072530 legacy=0x7fb17015cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:36.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.669+0000 7fb16fe00640 1 -- 172.21.15.152:0/3110198953 >> v1:172.21.15.152:6789/0 conn(0x7fb1700b47a0 legacy=0x7fb170071e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:36.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.669+0000 7fb16fe00640 1 -- 172.21.15.152:0/3110198953 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1701602d0 con 0x7fb170145340 2024-04-20T11:35:36.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.670+0000 7fb16fe00640 1 --2- 172.21.15.152:0/3110198953 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb170145340 0x7fb170160db0 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7fb1600097a0 tx=0x7fb16002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81fa3cef9f867c99 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:36.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.670+0000 7fb16f400640 1 -- 172.21.15.152:0/3110198953 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1600315a0 con 0x7fb170145340 2024-04-20T11:35:36.670 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.670+0000 7fb16f400640 1 -- 172.21.15.152:0/3110198953 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb160037070 con 0x7fb170145340 2024-04-20T11:35:36.670 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.670+0000 7fb16f400640 1 -- 172.21.15.152:0/3110198953 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb160031960 con 0x7fb170145340 2024-04-20T11:35:36.670 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.671+0000 7fb176a00640 1 -- 172.21.15.152:0/3110198953 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb170145340 msgr2=0x7fb170160db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:36.671 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.671+0000 7fb176a00640 1 --2- 172.21.15.152:0/3110198953 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb170145340 0x7fb170160db0 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7fb1600097a0 tx=0x7fb16002b840 comp rx=0 tx=0).stop 2024-04-20T11:35:36.671 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.671+0000 7fb176a00640 1 -- 172.21.15.152:0/3110198953 shutdown_connections 2024-04-20T11:35:36.671 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.671+0000 7fb176a00640 1 --2- 172.21.15.152:0/3110198953 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb170145340 0x7fb170160db0 unknown :-1 s=CLOSED pgs=1803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:36.671 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.671+0000 7fb176a00640 1 -- 172.21.15.152:0/3110198953 >> 172.21.15.152:0/3110198953 conn(0x7fb17006b980 msgr2=0x7fb17006bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:36.671 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.671+0000 7fb176a00640 1 -- 172.21.15.152:0/3110198953 shutdown_connections 2024-04-20T11:35:36.671 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.671+0000 7fb176a00640 1 -- 172.21.15.152:0/3110198953 wait complete. 2024-04-20T11:35:36.672 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.673+0000 7fb176a00640 1 Processor -- start 2024-04-20T11:35:36.672 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.673+0000 7fb176a00640 1 -- start start 2024-04-20T11:35:36.675 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.674+0000 7fb176a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb170072530 0x7fb1701f8520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:36.675 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.674+0000 7fb176a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1700b47a0 0x7fb17006eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:36.675 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.674+0000 7fb176a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb170145340 0x7fb17006f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:36.675 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.674+0000 7fb176a00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fb1700ceaa0 con 0x7fb170145340 2024-04-20T11:35:36.675 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.674+0000 7fb176a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb170146010 con 0x7fb1700b47a0 2024-04-20T11:35:36.675 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.674+0000 7fb176a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb1700cec20 con 0x7fb170072530 2024-04-20T11:35:36.675 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.674+0000 7fb174a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb170145340 0x7fb17006f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:36.675 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.674+0000 7fb174a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb170145340 0x7fb17006f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:37858/0 (socket says 172.21.15.152:37858) 2024-04-20T11:35:36.675 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.674+0000 7fb174a00640 1 -- 172.21.15.152:0/282590655 learned_addr learned my addr 172.21.15.152:0/282590655 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:36.675 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.674+0000 7fb167400640 1 --2- 172.21.15.152:0/282590655 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1700b47a0 0x7fb17006eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:36.675 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.674+0000 7fb16fe00640 1 --2- 172.21.15.152:0/282590655 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb170072530 0x7fb1701f8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:36.676 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.675+0000 7fb174a00640 1 -- 172.21.15.152:0/282590655 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb170072530 msgr2=0x7fb1701f8520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:36.676 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.675+0000 7fb174a00640 1 --2- 172.21.15.152:0/282590655 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb170072530 0x7fb1701f8520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:36.676 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.675+0000 7fb174a00640 1 -- 172.21.15.152:0/282590655 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1700b47a0 msgr2=0x7fb17006eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:36.676 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.675+0000 7fb174a00640 1 --2- 172.21.15.152:0/282590655 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1700b47a0 0x7fb17006eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:36.676 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.675+0000 7fb174a00640 1 -- 172.21.15.152:0/282590655 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1701f90c0 con 0x7fb170145340 2024-04-20T11:35:36.676 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.675+0000 7fb174a00640 1 --2- 172.21.15.152:0/282590655 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb170145340 0x7fb17006f050 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7fb16800c7d0 tx=0x7fb16800cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:36.676 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.675+0000 7fb16cc00640 1 -- 172.21.15.152:0/282590655 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb168003b70 con 0x7fb170145340 2024-04-20T11:35:36.676 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.675+0000 7fb16cc00640 1 -- 172.21.15.152:0/282590655 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb168003d30 con 0x7fb170145340 2024-04-20T11:35:36.676 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.675+0000 7fb176a00640 1 -- 172.21.15.152:0/282590655 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb17006f6f0 con 0x7fb170145340 2024-04-20T11:35:36.676 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.675+0000 7fb16cc00640 1 -- 172.21.15.152:0/282590655 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb16800aa90 con 0x7fb170145340 2024-04-20T11:35:36.676 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.675+0000 7fb176a00640 1 -- 172.21.15.152:0/282590655 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb17006fcb0 con 0x7fb170145340 2024-04-20T11:35:36.677 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.677+0000 7fb176a00640 1 -- 172.21.15.152:0/282590655 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb170068a60 con 0x7fb170145340 2024-04-20T11:35:36.677 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.678+0000 7fb16cc00640 1 -- 172.21.15.152:0/282590655 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 21) ==== 91550+0+0 (secure 0 0 0) 0x7fb16800ac30 con 0x7fb170145340 2024-04-20T11:35:36.678 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.678+0000 7fb16cc00640 1 --2- 172.21.15.152:0/282590655 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb14c06cec0 0x7fb14c06f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:36.678 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.679+0000 7fb16cc00640 1 -- 172.21.15.152:0/282590655 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5709+0+0 (secure 0 0 0) 0x7fb16808c4c0 con 0x7fb170145340 2024-04-20T11:35:36.678 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.679+0000 7fb16fe00640 1 --2- 172.21.15.152:0/282590655 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb14c06cec0 0x7fb14c06f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:36.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.680+0000 7fb16fe00640 1 --2- 172.21.15.152:0/282590655 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb14c06cec0 0x7fb14c06f380 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb160036000 tx=0x7fb16003d040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:36.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.683+0000 7fb16cc00640 1 -- 172.21.15.152:0/282590655 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fb1701f90c0 con 0x7fb170145340 2024-04-20T11:35:36.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.920+0000 7fb176a00640 1 -- 172.21.15.152:0/282590655 --> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] -- mgr_command(tid 0: {"prefix": "fs subvolume pin", "vol_name": "non_existent_subvolume", "sub_name": "subvolname_1", "pin_type": "export", "pin_setting": "1", "target": ["mon-mgr", ""]}) -- 0x7fb1700b4580 con 0x7fb14c06cec0 2024-04-20T11:35:36.923 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:35:36.923+0000 7f8a5cc00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-04-20T11:35:36.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:36.924+0000 7fb16cc00640 1 -- 172.21.15.152:0/282590655 <== mgr.4109 v2:172.21.15.175:6836/4021250441 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7fb1700b4580 con 0x7fb14c06cec0 2024-04-20T11:35:36.924 INFO:teuthology.orchestra.run.smithi152.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-04-20T11:35:36.945 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-20T11:35:36.946 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls non_existent_subvolume 2024-04-20T11:35:37.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.149+0000 7f1878600640 1 Processor -- start 2024-04-20T11:35:37.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.149+0000 7f1878600640 1 -- start start 2024-04-20T11:35:37.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.150+0000 7f1878600640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f1870146000 0x7f18700b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:37.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.150+0000 7f1878600640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f18700b2170 0x7f18700b2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:37.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.150+0000 7f1878600640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18701463e0 con 0x7f18700cd1b0 2024-04-20T11:35:37.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.150+0000 7f1878600640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f18700cdcc0 con 0x7f1870146000 2024-04-20T11:35:37.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.150+0000 7f1878600640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f18700b4750 con 0x7f18700b2170 2024-04-20T11:35:37.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.151+0000 7f1876600640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f18700cd1b0 0x7f18700cd5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:40512/0 (socket says 172.21.15.152:40512) 2024-04-20T11:35:37.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.151+0000 7f1876600640 1 -- 172.21.15.152:0/3802403209 learned_addr learned my addr 172.21.15.152:0/3802403209 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:35:37.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.151+0000 7f1875c00640 1 --2- 172.21.15.152:0/3802403209 >> v2:172.21.15.152:3300/0 conn(0x7f1870146000 0x7f18700b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:37.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.151+0000 7f1875200640 1 --2- 172.21.15.152:0/3802403209 >> v2:172.21.15.175:3300/0 conn(0x7f18700b2170 0x7f18700b2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:37.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.151+0000 7f186fe00640 1 -- 172.21.15.152:0/3802403209 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 734132918 0 0) 0x7f18701463e0 con 0x7f18700cd1b0 2024-04-20T11:35:37.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.151+0000 7f186fe00640 1 -- 172.21.15.152:0/3802403209 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1858003520 con 0x7f18700cd1b0 2024-04-20T11:35:37.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.152+0000 7f1875c00640 1 -- 172.21.15.152:0/3802403209 >> v2:172.21.15.175:3300/0 conn(0x7f18700b2170 msgr2=0x7f18700b2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:37.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.152+0000 7f1875c00640 1 --2- 172.21.15.152:0/3802403209 >> v2:172.21.15.175:3300/0 conn(0x7f18700b2170 0x7f18700b2620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:37.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.152+0000 7f1875c00640 1 -- 172.21.15.152:0/3802403209 >> v1:172.21.15.152:6789/0 conn(0x7f18700cd1b0 legacy=0x7f18700cd5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:37.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.152+0000 7f1875c00640 1 -- 172.21.15.152:0/3802403209 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18700b3e20 con 0x7f1870146000 2024-04-20T11:35:37.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.153+0000 7f1875c00640 1 --2- 172.21.15.152:0/3802403209 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1870146000 0x7f18700b4be0 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f18700b3280 tx=0x7f186402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=890ac2b41ec0bc25 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:37.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.153+0000 7f186fe00640 1 -- 172.21.15.152:0/3802403209 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1864039070 con 0x7f1870146000 2024-04-20T11:35:37.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.153+0000 7f186fe00640 1 -- 172.21.15.152:0/3802403209 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f186402cbf0 con 0x7f1870146000 2024-04-20T11:35:37.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.153+0000 7f186fe00640 1 -- 172.21.15.152:0/3802403209 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1864034440 con 0x7f1870146000 2024-04-20T11:35:37.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.153+0000 7f1878600640 1 -- 172.21.15.152:0/3802403209 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1870146000 msgr2=0x7f18700b4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:37.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.153+0000 7f1878600640 1 --2- 172.21.15.152:0/3802403209 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1870146000 0x7f18700b4be0 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f18700b3280 tx=0x7f186402be40 comp rx=0 tx=0).stop 2024-04-20T11:35:37.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.154+0000 7f1878600640 1 -- 172.21.15.152:0/3802403209 shutdown_connections 2024-04-20T11:35:37.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.154+0000 7f1878600640 1 --2- 172.21.15.152:0/3802403209 >> v2:172.21.15.175:3300/0 conn(0x7f18700b2170 0x7f18700b2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:37.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.154+0000 7f1878600640 1 --2- 172.21.15.152:0/3802403209 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1870146000 0x7f18700b4be0 unknown :-1 s=CLOSED pgs=1806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:37.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.154+0000 7f1878600640 1 -- 172.21.15.152:0/3802403209 >> 172.21.15.152:0/3802403209 conn(0x7f187006b980 msgr2=0x7f187006bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:37.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.154+0000 7f1878600640 1 -- 172.21.15.152:0/3802403209 shutdown_connections 2024-04-20T11:35:37.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.154+0000 7f1878600640 1 -- 172.21.15.152:0/3802403209 wait complete. 2024-04-20T11:35:37.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.155+0000 7f1878600640 1 Processor -- start 2024-04-20T11:35:37.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.156+0000 7f1878600640 1 -- start start 2024-04-20T11:35:37.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.156+0000 7f1878600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f18700b2170 0x7f187013de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:37.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.156+0000 7f1878600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f18700cd1b0 0x7f1870141720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:37.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.156+0000 7f1878600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1870146000 0x7f1870141c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:37.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.157+0000 7f1878600640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f187015ccf0 con 0x7f18700cd1b0 2024-04-20T11:35:37.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.157+0000 7f1878600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f18700b43e0 con 0x7f1870146000 2024-04-20T11:35:37.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.157+0000 7f1878600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f187015ce70 con 0x7f18700b2170 2024-04-20T11:35:37.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.157+0000 7f1875200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f18700cd1b0 0x7f1870141720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:37.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.157+0000 7f1875200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f18700cd1b0 0x7f1870141720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:37878/0 (socket says 172.21.15.152:37878) 2024-04-20T11:35:37.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.157+0000 7f1875200640 1 -- 172.21.15.152:0/3355735295 learned_addr learned my addr 172.21.15.152:0/3355735295 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:37.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.157+0000 7f1875200640 1 -- 172.21.15.152:0/3355735295 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f18700b2170 msgr2=0x7f187013de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:37.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.157+0000 7f1875200640 1 --2- 172.21.15.152:0/3355735295 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f18700b2170 0x7f187013de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:37.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.157+0000 7f1875200640 1 -- 172.21.15.152:0/3355735295 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1870146000 msgr2=0x7f1870141c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:37.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.157+0000 7f1875200640 1 --2- 172.21.15.152:0/3355735295 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1870146000 0x7f1870141c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:37.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.157+0000 7f1875200640 1 -- 172.21.15.152:0/3355735295 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1870140680 con 0x7f18700cd1b0 2024-04-20T11:35:37.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.158+0000 7f1875200640 1 --2- 172.21.15.152:0/3355735295 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f18700cd1b0 0x7f1870141720 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f185c00d800 tx=0x7f185c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:37.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.158+0000 7f186d600640 1 -- 172.21.15.152:0/3355735295 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f185c00b6c0 con 0x7f18700cd1b0 2024-04-20T11:35:37.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.158+0000 7f186d600640 1 -- 172.21.15.152:0/3355735295 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f185c019070 con 0x7f18700cd1b0 2024-04-20T11:35:37.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.158+0000 7f186d600640 1 -- 172.21.15.152:0/3355735295 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f185c00ba80 con 0x7f18700cd1b0 2024-04-20T11:35:37.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.158+0000 7f1878600640 1 -- 172.21.15.152:0/3355735295 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1870140960 con 0x7f18700cd1b0 2024-04-20T11:35:37.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.158+0000 7f1878600640 1 -- 172.21.15.152:0/3355735295 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1870140f20 con 0x7f18700cd1b0 2024-04-20T11:35:37.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.159+0000 7f1878600640 1 -- 172.21.15.152:0/3355735295 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1870070750 con 0x7f18700cd1b0 2024-04-20T11:35:37.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.160+0000 7f186d600640 1 -- 172.21.15.152:0/3355735295 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 21) ==== 91550+0+0 (secure 0 0 0) 0x7f185c00bc20 con 0x7f18700cd1b0 2024-04-20T11:35:37.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.162+0000 7f186d600640 1 --2- 172.21.15.152:0/3355735295 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f185006ce90 0x7f185006f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:37.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.162+0000 7f186d600640 1 -- 172.21.15.152:0/3355735295 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5709+0+0 (secure 0 0 0) 0x7f185c08ca70 con 0x7f18700cd1b0 2024-04-20T11:35:37.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.162+0000 7f1875c00640 1 --2- 172.21.15.152:0/3355735295 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f185006ce90 0x7f185006f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:37.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.163+0000 7f1875c00640 1 --2- 172.21.15.152:0/3355735295 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f185006ce90 0x7f185006f350 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f1864036000 tx=0x7f186402e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:37.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.166+0000 7f186d600640 1 -- 172.21.15.152:0/3355735295 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f1870070750 con 0x7f18700cd1b0 2024-04-20T11:35:37.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.393+0000 7f1878600640 1 -- 172.21.15.152:0/3355735295 --> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "non_existent_subvolume", "target": ["mon-mgr", ""]}) -- 0x7f18700689f0 con 0x7f185006ce90 2024-04-20T11:35:37.396 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:35:37.396+0000 7f8a5cc00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-04-20T11:35:37.396 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.397+0000 7f186d600640 1 -- 172.21.15.152:0/3355735295 <== mgr.4109 v2:172.21.15.175:6836/4021250441 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f18700689f0 con 0x7f185006ce90 2024-04-20T11:35:37.396 INFO:teuthology.orchestra.run.smithi152.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-04-20T11:35:37.408 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-20T11:35:37.408 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create non_existent_subvolume subvolume_1 snapshot_1 2024-04-20T11:35:37.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.619+0000 7f2bdca00640 1 Processor -- start 2024-04-20T11:35:37.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.620+0000 7f2bdca00640 1 -- start start 2024-04-20T11:35:37.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.621+0000 7f2bdca00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2bd815e870 0x7f2bd80d3370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:37.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.621+0000 7f2bdca00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f2bd80d0fa0 0x7f2bd80d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:37.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.621+0000 7f2bdca00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f2bd80cff90 0x7f2bd80d0420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:37.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.621+0000 7f2bdca00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f2bd80d1960 con 0x7f2bd80d0fa0 2024-04-20T11:35:37.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.621+0000 7f2bdca00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f2bd80d2ee0 con 0x7f2bd80cff90 2024-04-20T11:35:37.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.621+0000 7f2bdca00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2bd815ec50 con 0x7f2bd815e870 2024-04-20T11:35:37.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.622+0000 7f2bcea00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f2bd80d0fa0 0x7f2bd80d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:37.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.622+0000 7f2bcea00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f2bd80d0fa0 0x7f2bd80d1420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:37890/0 (socket says 172.21.15.152:37890) 2024-04-20T11:35:37.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.622+0000 7f2bcea00640 1 -- 172.21.15.152:0/1855316305 learned_addr learned my addr 172.21.15.152:0/1855316305 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:37.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.622+0000 7f2bd7400640 1 --2- 172.21.15.152:0/1855316305 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2bd815e870 0x7f2bd80d3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:37.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.622+0000 7f2bd7e00640 1 --2- 172.21.15.152:0/1855316305 >> v2:172.21.15.175:3300/0 conn(0x7f2bd80cff90 0x7f2bd80d0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:37.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.622+0000 7f2bcea00640 1 -- 172.21.15.152:0/1855316305 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2bd815e870 msgr2=0x7f2bd80d3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:37.623 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.622+0000 7f2bcea00640 1 --2- 172.21.15.152:0/1855316305 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2bd815e870 0x7f2bd80d3370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:37.623 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.622+0000 7f2bcea00640 1 -- 172.21.15.152:0/1855316305 >> v2:172.21.15.175:3300/0 conn(0x7f2bd80cff90 msgr2=0x7f2bd80d0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:37.623 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.622+0000 7f2bcea00640 1 --2- 172.21.15.152:0/1855316305 >> v2:172.21.15.175:3300/0 conn(0x7f2bd80cff90 0x7f2bd80d0420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:37.623 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.622+0000 7f2bcea00640 1 -- 172.21.15.152:0/1855316305 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bd80d38b0 con 0x7f2bd80d0fa0 2024-04-20T11:35:37.623 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.622+0000 7f2bd7400640 1 --2- 172.21.15.152:0/1855316305 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2bd815e870 0x7f2bd80d3370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:35:37.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.623+0000 7f2bcea00640 1 --2- 172.21.15.152:0/1855316305 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2bd80d0fa0 0x7f2bd80d1420 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f2bd808ceb0 tx=0x7f2bc002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d32818768e1f1ed6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:37.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.623+0000 7f2bd6a00640 1 -- 172.21.15.152:0/1855316305 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc0037070 con 0x7f2bd80d0fa0 2024-04-20T11:35:37.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.624+0000 7f2bd6a00640 1 -- 172.21.15.152:0/1855316305 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bc0004230 con 0x7f2bd80d0fa0 2024-04-20T11:35:37.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.624+0000 7f2bd6a00640 1 -- 172.21.15.152:0/1855316305 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc0003610 con 0x7f2bd80d0fa0 2024-04-20T11:35:37.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.624+0000 7f2bdca00640 1 -- 172.21.15.152:0/1855316305 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2bd80d0fa0 msgr2=0x7f2bd80d1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:37.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.624+0000 7f2bdca00640 1 --2- 172.21.15.152:0/1855316305 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2bd80d0fa0 0x7f2bd80d1420 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f2bd808ceb0 tx=0x7f2bc002b860 comp rx=0 tx=0).stop 2024-04-20T11:35:37.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.624+0000 7f2bdca00640 1 -- 172.21.15.152:0/1855316305 shutdown_connections 2024-04-20T11:35:37.625 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.624+0000 7f2bdca00640 1 --2- 172.21.15.152:0/1855316305 >> v2:172.21.15.175:3300/0 conn(0x7f2bd80cff90 0x7f2bd80d0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:37.625 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.624+0000 7f2bdca00640 1 --2- 172.21.15.152:0/1855316305 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2bd80d0fa0 0x7f2bd80d1420 unknown :-1 s=CLOSED pgs=1808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:37.625 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.624+0000 7f2bdca00640 1 --2- 172.21.15.152:0/1855316305 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2bd815e870 0x7f2bd80d3370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:37.625 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.624+0000 7f2bdca00640 1 -- 172.21.15.152:0/1855316305 >> 172.21.15.152:0/1855316305 conn(0x7f2bd808bfd0 msgr2=0x7f2bd80d45d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:37.625 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.624+0000 7f2bdca00640 1 -- 172.21.15.152:0/1855316305 shutdown_connections 2024-04-20T11:35:37.625 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.625+0000 7f2bdca00640 1 -- 172.21.15.152:0/1855316305 wait complete. 2024-04-20T11:35:37.625 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.626+0000 7f2bdca00640 1 Processor -- start 2024-04-20T11:35:37.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.626+0000 7f2bdca00640 1 -- start start 2024-04-20T11:35:37.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.627+0000 7f2bdca00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2bd80cff90 0x7f2bd80cd750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:37.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.627+0000 7f2bdca00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2bd80d0fa0 0x7f2bd80cdc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:37.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.627+0000 7f2bdca00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2bd815e870 0x7f2bd81f4650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:37.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.627+0000 7f2bdca00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd80d2890 con 0x7f2bd80cff90 2024-04-20T11:35:37.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.627+0000 7f2bdca00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd80d25b0 con 0x7f2bd815e870 2024-04-20T11:35:37.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.627+0000 7f2bdca00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2bd80d2b70 con 0x7f2bd80d0fa0 2024-04-20T11:35:37.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.628+0000 7f2bd7400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2bd80cff90 0x7f2bd80cd750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:37.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.628+0000 7f2bd7400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2bd80cff90 0x7f2bd80cd750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:37902/0 (socket says 172.21.15.152:37902) 2024-04-20T11:35:37.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.628+0000 7f2bd7400640 1 -- 172.21.15.152:0/266258520 learned_addr learned my addr 172.21.15.152:0/266258520 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:37.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.628+0000 7f2bd7400640 1 -- 172.21.15.152:0/266258520 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2bd80d0fa0 msgr2=0x7f2bd80cdc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:37.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.628+0000 7f2bd7400640 1 --2- 172.21.15.152:0/266258520 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2bd80d0fa0 0x7f2bd80cdc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:37.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.628+0000 7f2bd7400640 1 -- 172.21.15.152:0/266258520 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2bd815e870 msgr2=0x7f2bd81f4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:37.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.628+0000 7f2bd7e00640 1 --2- 172.21.15.152:0/266258520 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2bd815e870 0x7f2bd81f4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:37.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.628+0000 7f2bd7400640 1 --2- 172.21.15.152:0/266258520 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2bd815e870 0x7f2bd81f4650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:37.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.628+0000 7f2bd7400640 1 -- 172.21.15.152:0/266258520 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bd80cbc10 con 0x7f2bd80cff90 2024-04-20T11:35:37.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.629+0000 7f2bd7400640 1 --2- 172.21.15.152:0/266258520 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2bd80cff90 0x7f2bd80cd750 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f2bc800c7f0 tx=0x7f2bc800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:37.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.629+0000 7f2bcfe00640 1 -- 172.21.15.152:0/266258520 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc8015070 con 0x7f2bd80cff90 2024-04-20T11:35:37.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.629+0000 7f2bcfe00640 1 -- 172.21.15.152:0/266258520 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bc8007a30 con 0x7f2bd80cff90 2024-04-20T11:35:37.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.629+0000 7f2bcfe00640 1 -- 172.21.15.152:0/266258520 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc8007e00 con 0x7f2bd80cff90 2024-04-20T11:35:37.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.629+0000 7f2bdca00640 1 -- 172.21.15.152:0/266258520 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bd80cbef0 con 0x7f2bd80cff90 2024-04-20T11:35:37.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.629+0000 7f2bdca00640 1 -- 172.21.15.152:0/266258520 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bd80cc4b0 con 0x7f2bd80cff90 2024-04-20T11:35:37.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.630+0000 7f2bdca00640 1 -- 172.21.15.152:0/266258520 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bd8075e60 con 0x7f2bd80cff90 2024-04-20T11:35:37.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.631+0000 7f2bcfe00640 1 -- 172.21.15.152:0/266258520 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 21) ==== 91550+0+0 (secure 0 0 0) 0x7f2bc8018070 con 0x7f2bd80cff90 2024-04-20T11:35:37.632 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.632+0000 7f2bcfe00640 1 --2- 172.21.15.152:0/266258520 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f2bb406ce90 0x7f2bb406f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:37.632 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.633+0000 7f2bcfe00640 1 -- 172.21.15.152:0/266258520 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5709+0+0 (secure 0 0 0) 0x7f2bc808c590 con 0x7f2bd80cff90 2024-04-20T11:35:37.632 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.633+0000 7f2bcea00640 1 --2- 172.21.15.152:0/266258520 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f2bb406ce90 0x7f2bb406f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:37.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.634+0000 7f2bcea00640 1 --2- 172.21.15.152:0/266258520 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f2bb406ce90 0x7f2bb406f350 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f2bc0034000 tx=0x7f2bc0030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:37.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.637+0000 7f2bcfe00640 1 -- 172.21.15.152:0/266258520 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f2bc805a0d0 con 0x7f2bd80cff90 2024-04-20T11:35:37.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.869+0000 7f2bdca00640 1 -- 172.21.15.152:0/266258520 --> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "snap_name": "snapshot_1", "target": ["mon-mgr", ""]}) -- 0x7f2bd80ba550 con 0x7f2bb406ce90 2024-04-20T11:35:38.205 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:35:37.872+0000 7f8a5cc00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-04-20T11:35:38.205 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-20T11:35:38.206 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:37.873+0000 7f2bcfe00640 1 -- 172.21.15.152:0/266258520 <== mgr.4109 v2:172.21.15.175:6836/4021250441 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f2bd80ba550 con 0x7f2bb406ce90 2024-04-20T11:35:38.206 INFO:teuthology.orchestra.run.smithi152.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-04-20T11:35:38.206 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm non_existent_subvolume subvolume_1 snapshot_1 2024-04-20T11:35:38.424 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.424+0000 7fef13400640 1 Processor -- start 2024-04-20T11:35:38.424 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.424+0000 7fef13400640 1 -- start start 2024-04-20T11:35:38.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.426+0000 7fef13400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef0c0cd8e0 0x7fef0c06e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:38.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.426+0000 7fef13400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fef0c0b4d90 0x7fef0c15f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:38.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.426+0000 7fef13400640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef0c06eab0 con 0x7fef0c0cf220 2024-04-20T11:35:38.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.426+0000 7fef13400640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fef0c0d0240 con 0x7fef0c0b4d90 2024-04-20T11:35:38.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.426+0000 7fef13400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fef0c0cf6c0 con 0x7fef0c0cd8e0 2024-04-20T11:35:38.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.426+0000 7fef11400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fef0c0b4d90 0x7fef0c15f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:38.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.426+0000 7fef11400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fef0c0b4d90 0x7fef0c15f410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:37912/0 (socket says 172.21.15.152:37912) 2024-04-20T11:35:38.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.426+0000 7fef11400640 1 -- 172.21.15.152:0/3372134325 learned_addr learned my addr 172.21.15.152:0/3372134325 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:38.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.426+0000 7fef03e00640 1 --1- 172.21.15.152:0/3372134325 >> v1:172.21.15.152:6789/0 conn(0x7fef0c0cf220 0x7fef0c0b4680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:40514/0 (socket says 172.21.15.152:40514) 2024-04-20T11:35:38.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.427+0000 7fef10a00640 1 --2- 172.21.15.152:0/3372134325 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef0c0cd8e0 0x7fef0c06e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:38.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.427+0000 7fef11400640 1 -- 172.21.15.152:0/3372134325 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef0c0cd8e0 msgr2=0x7fef0c06e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:38.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.427+0000 7fef11400640 1 --2- 172.21.15.152:0/3372134325 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef0c0cd8e0 0x7fef0c06e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:38.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.427+0000 7fef11400640 1 -- 172.21.15.152:0/3372134325 >> v1:172.21.15.152:6789/0 conn(0x7fef0c0cf220 legacy=0x7fef0c0b4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:38.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.427+0000 7fef11400640 1 -- 172.21.15.152:0/3372134325 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef0c0cf9a0 con 0x7fef0c0b4d90 2024-04-20T11:35:38.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.428+0000 7fef11400640 1 --2- 172.21.15.152:0/3372134325 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef0c0b4d90 0x7fef0c15f410 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7fef04009e50 tx=0x7fef0402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ac0fcb03d1922ae server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:38.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.428+0000 7fef0be00640 1 -- 172.21.15.152:0/3372134325 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef0402ca70 con 0x7fef0c0b4d90 2024-04-20T11:35:38.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.428+0000 7fef0be00640 1 -- 172.21.15.152:0/3372134325 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef0402cc10 con 0x7fef0c0b4d90 2024-04-20T11:35:38.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.428+0000 7fef0be00640 1 -- 172.21.15.152:0/3372134325 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef040343f0 con 0x7fef0c0b4d90 2024-04-20T11:35:38.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.429+0000 7fef13400640 1 -- 172.21.15.152:0/3372134325 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef0c0b4d90 msgr2=0x7fef0c15f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:38.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.429+0000 7fef13400640 1 --2- 172.21.15.152:0/3372134325 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef0c0b4d90 0x7fef0c15f410 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7fef04009e50 tx=0x7fef0402bed0 comp rx=0 tx=0).stop 2024-04-20T11:35:38.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.429+0000 7fef13400640 1 -- 172.21.15.152:0/3372134325 shutdown_connections 2024-04-20T11:35:38.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.429+0000 7fef13400640 1 --2- 172.21.15.152:0/3372134325 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef0c0b4d90 0x7fef0c15f410 unknown :-1 s=CLOSED pgs=1811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:38.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.429+0000 7fef13400640 1 --2- 172.21.15.152:0/3372134325 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef0c0cd8e0 0x7fef0c06e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:38.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.429+0000 7fef13400640 1 -- 172.21.15.152:0/3372134325 >> 172.21.15.152:0/3372134325 conn(0x7fef0c06b980 msgr2=0x7fef0c06bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:38.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.429+0000 7fef13400640 1 -- 172.21.15.152:0/3372134325 shutdown_connections 2024-04-20T11:35:38.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.429+0000 7fef13400640 1 -- 172.21.15.152:0/3372134325 wait complete. 2024-04-20T11:35:38.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.431+0000 7fef13400640 1 Processor -- start 2024-04-20T11:35:38.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.431+0000 7fef13400640 1 -- start start 2024-04-20T11:35:38.433 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.432+0000 7fef13400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef0c0b4d90 0x7fef0c15a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:38.433 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.432+0000 7fef13400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef0c0cd8e0 0x7fef0c168d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:38.433 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.432+0000 7fef13400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef0c0cf220 0x7fef0c142f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:38.433 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.432+0000 7fef13400640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fef0c0cdcc0 con 0x7fef0c0cd8e0 2024-04-20T11:35:38.433 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.432+0000 7fef13400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fef0c0cff60 con 0x7fef0c0b4d90 2024-04-20T11:35:38.433 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.432+0000 7fef13400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fef0c144d70 con 0x7fef0c0cf220 2024-04-20T11:35:38.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.432+0000 7fef03e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef0c0cd8e0 0x7fef0c168d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:38.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.432+0000 7fef03e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef0c0cd8e0 0x7fef0c168d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:37914/0 (socket says 172.21.15.152:37914) 2024-04-20T11:35:38.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.432+0000 7fef03e00640 1 -- 172.21.15.152:0/3420778152 learned_addr learned my addr 172.21.15.152:0/3420778152 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:38.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.433+0000 7fef11400640 1 --2- 172.21.15.152:0/3420778152 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef0c0cf220 0x7fef0c142f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:38.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.433+0000 7fef10a00640 1 --2- 172.21.15.152:0/3420778152 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef0c0b4d90 0x7fef0c15a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:38.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.433+0000 7fef03e00640 1 -- 172.21.15.152:0/3420778152 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef0c0cf220 msgr2=0x7fef0c142f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:38.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.433+0000 7fef03e00640 1 --2- 172.21.15.152:0/3420778152 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef0c0cf220 0x7fef0c142f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:38.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.433+0000 7fef03e00640 1 -- 172.21.15.152:0/3420778152 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef0c0b4d90 msgr2=0x7fef0c15a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:38.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.433+0000 7fef03e00640 1 --2- 172.21.15.152:0/3420778152 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef0c0b4d90 0x7fef0c15a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:38.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.433+0000 7fef03e00640 1 -- 172.21.15.152:0/3420778152 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef0c0a9640 con 0x7fef0c0cd8e0 2024-04-20T11:35:38.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.433+0000 7fef03e00640 1 --2- 172.21.15.152:0/3420778152 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef0c0cd8e0 0x7fef0c168d00 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7feef400c430 tx=0x7feef400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:38.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.433+0000 7fef09600640 1 -- 172.21.15.152:0/3420778152 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef40074e0 con 0x7fef0c0cd8e0 2024-04-20T11:35:38.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.433+0000 7fef09600640 1 -- 172.21.15.152:0/3420778152 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feef4007680 con 0x7fef0c0cd8e0 2024-04-20T11:35:38.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.433+0000 7fef09600640 1 -- 172.21.15.152:0/3420778152 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef4004a00 con 0x7fef0c0cd8e0 2024-04-20T11:35:38.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.433+0000 7fef13400640 1 -- 172.21.15.152:0/3420778152 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef0c0a9920 con 0x7fef0c0cd8e0 2024-04-20T11:35:38.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.434+0000 7fef13400640 1 -- 172.21.15.152:0/3420778152 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef0c13c5e0 con 0x7fef0c0cd8e0 2024-04-20T11:35:38.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.435+0000 7fef13400640 1 -- 172.21.15.152:0/3420778152 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef0c068a60 con 0x7fef0c0cd8e0 2024-04-20T11:35:38.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.436+0000 7fef09600640 1 -- 172.21.15.152:0/3420778152 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 21) ==== 91550+0+0 (secure 0 0 0) 0x7feef401a020 con 0x7fef0c0cd8e0 2024-04-20T11:35:38.436 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.437+0000 7fef09600640 1 --2- 172.21.15.152:0/3420778152 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7feee806ce90 0x7feee806f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:38.436 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.437+0000 7fef09600640 1 -- 172.21.15.152:0/3420778152 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5709+0+0 (secure 0 0 0) 0x7feef408bdb0 con 0x7fef0c0cd8e0 2024-04-20T11:35:38.436 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.437+0000 7fef10a00640 1 --2- 172.21.15.152:0/3420778152 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7feee806ce90 0x7feee806f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:38.438 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.438+0000 7fef10a00640 1 --2- 172.21.15.152:0/3420778152 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7feee806ce90 0x7feee806f350 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7feefc006fd0 tx=0x7feefc008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:38.440 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.441+0000 7fef09600640 1 -- 172.21.15.152:0/3420778152 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7feef4059970 con 0x7fef0c0cd8e0 2024-04-20T11:35:38.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.682+0000 7fef13400640 1 -- 172.21.15.152:0/3420778152 --> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "snap_name": "snapshot_1", "target": ["mon-mgr", ""]}) -- 0x7fef0c06ff40 con 0x7feee806ce90 2024-04-20T11:35:38.685 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:35:38.685+0000 7f8a5cc00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-04-20T11:35:38.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.686+0000 7fef09600640 1 -- 172.21.15.152:0/3420778152 <== mgr.4109 v2:172.21.15.175:6836/4021250441 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7fef0c06ff40 con 0x7feee806ce90 2024-04-20T11:35:38.685 INFO:teuthology.orchestra.run.smithi152.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-04-20T11:35:38.697 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-20T11:35:38.698 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot info non_existent_subvolume subvolume_1 snapshot_1 2024-04-20T11:35:38.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.908+0000 7fd9f7e00640 1 Processor -- start 2024-04-20T11:35:38.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.908+0000 7fd9f7e00640 1 -- start start 2024-04-20T11:35:38.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.909+0000 7fd9f7e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd9f80d05f0 0x7fd9f80d09d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:38.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.909+0000 7fd9f7e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fd9f80ae180 0x7fd9f80ae600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:38.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.909+0000 7fd9f7e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fd9f815eee0 0x7fd9f815f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:38.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.909+0000 7fd9f7e00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fd9f80d0f10 con 0x7fd9f815eee0 2024-04-20T11:35:38.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.909+0000 7fd9f7e00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fd9f80aeef0 con 0x7fd9f80ae180 2024-04-20T11:35:38.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.909+0000 7fd9f7e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd9f80af1d0 con 0x7fd9f80d05f0 2024-04-20T11:35:38.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.910+0000 7fd9f7400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fd9f815eee0 0x7fd9f815f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:38.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.910+0000 7fd9f7400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fd9f815eee0 0x7fd9f815f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:37920/0 (socket says 172.21.15.152:37920) 2024-04-20T11:35:38.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.910+0000 7fd9f7400640 1 -- 172.21.15.152:0/1545565989 learned_addr learned my addr 172.21.15.152:0/1545565989 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:38.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.910+0000 7fd9f6a00640 1 --2- 172.21.15.152:0/1545565989 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd9f80d05f0 0x7fd9f80d09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:38.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.910+0000 7fd9f7400640 1 -- 172.21.15.152:0/1545565989 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd9f80d05f0 msgr2=0x7fd9f80d09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:38.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.910+0000 7fd9f7400640 1 --2- 172.21.15.152:0/1545565989 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd9f80d05f0 0x7fd9f80d09d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:38.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.910+0000 7fd9f7400640 1 -- 172.21.15.152:0/1545565989 >> v2:172.21.15.175:3300/0 conn(0x7fd9f80ae180 msgr2=0x7fd9f80ae600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:38.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.910+0000 7fd9f7400640 1 --2- 172.21.15.152:0/1545565989 >> v2:172.21.15.175:3300/0 conn(0x7fd9f80ae180 0x7fd9f80ae600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:38.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.910+0000 7fd9f7400640 1 -- 172.21.15.152:0/1545565989 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9f80af4b0 con 0x7fd9f815eee0 2024-04-20T11:35:38.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.911+0000 7fd9f7400640 1 --2- 172.21.15.152:0/1545565989 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd9f815eee0 0x7fd9f815f370 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7fd9f8070150 tx=0x7fd9f002e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b524493eedfb3035 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:38.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.912+0000 7fd9f6000640 1 -- 172.21.15.152:0/1545565989 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f002ea80 con 0x7fd9f815eee0 2024-04-20T11:35:38.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.912+0000 7fd9f6000640 1 -- 172.21.15.152:0/1545565989 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9f002ec20 con 0x7fd9f815eee0 2024-04-20T11:35:38.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.912+0000 7fd9f6000640 1 -- 172.21.15.152:0/1545565989 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f0009600 con 0x7fd9f815eee0 2024-04-20T11:35:38.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.912+0000 7fd9f7e00640 1 -- 172.21.15.152:0/1545565989 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd9f815eee0 msgr2=0x7fd9f815f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:38.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.912+0000 7fd9f7e00640 1 --2- 172.21.15.152:0/1545565989 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd9f815eee0 0x7fd9f815f370 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7fd9f8070150 tx=0x7fd9f002e0e0 comp rx=0 tx=0).stop 2024-04-20T11:35:38.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.913+0000 7fd9f7e00640 1 -- 172.21.15.152:0/1545565989 shutdown_connections 2024-04-20T11:35:38.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.913+0000 7fd9f7e00640 1 --2- 172.21.15.152:0/1545565989 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd9f815eee0 0x7fd9f815f370 unknown :-1 s=CLOSED pgs=1813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:38.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.913+0000 7fd9f7e00640 1 --2- 172.21.15.152:0/1545565989 >> v2:172.21.15.175:3300/0 conn(0x7fd9f80ae180 0x7fd9f80ae600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:38.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.913+0000 7fd9f7e00640 1 --2- 172.21.15.152:0/1545565989 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd9f80d05f0 0x7fd9f80d09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:38.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.913+0000 7fd9f7e00640 1 -- 172.21.15.152:0/1545565989 >> 172.21.15.152:0/1545565989 conn(0x7fd9f80b21e0 msgr2=0x7fd9f80b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:38.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.913+0000 7fd9f7e00640 1 -- 172.21.15.152:0/1545565989 shutdown_connections 2024-04-20T11:35:38.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.913+0000 7fd9f7e00640 1 -- 172.21.15.152:0/1545565989 wait complete. 2024-04-20T11:35:38.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.914+0000 7fd9f7e00640 1 Processor -- start 2024-04-20T11:35:38.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.914+0000 7fd9f7e00640 1 -- start start 2024-04-20T11:35:38.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.915+0000 7fd9f7e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd9f80ae180 0x7fd9f8154e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:38.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.915+0000 7fd9f7e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd9f80d05f0 0x7fd9f80ccf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:38.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.915+0000 7fd9f7e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd9f815eee0 0x7fd9f809b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:38.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.915+0000 7fd9f7e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fd9f8155ca0 con 0x7fd9f80ae180 2024-04-20T11:35:38.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.915+0000 7fd9f7e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd9f80afa70 con 0x7fd9f815eee0 2024-04-20T11:35:38.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.915+0000 7fd9f7e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd9f8155350 con 0x7fd9f80d05f0 2024-04-20T11:35:38.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.916+0000 7fd9f6a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd9f80ae180 0x7fd9f8154e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:38.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.916+0000 7fd9f6a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd9f80ae180 0x7fd9f8154e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:37924/0 (socket says 172.21.15.152:37924) 2024-04-20T11:35:38.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.916+0000 7fd9f6a00640 1 -- 172.21.15.152:0/3575495615 learned_addr learned my addr 172.21.15.152:0/3575495615 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:38.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.916+0000 7fd9f6a00640 1 -- 172.21.15.152:0/3575495615 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd9f80d05f0 msgr2=0x7fd9f80ccf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:38.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.916+0000 7fd9f6a00640 1 --2- 172.21.15.152:0/3575495615 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd9f80d05f0 0x7fd9f80ccf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:38.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.916+0000 7fd9f6a00640 1 -- 172.21.15.152:0/3575495615 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd9f815eee0 msgr2=0x7fd9f809b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:38.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.916+0000 7fd9f7400640 1 --2- 172.21.15.152:0/3575495615 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd9f815eee0 0x7fd9f809b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:38.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.916+0000 7fd9f6a00640 1 --2- 172.21.15.152:0/3575495615 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd9f815eee0 0x7fd9f809b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:38.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.916+0000 7fd9f6a00640 1 -- 172.21.15.152:0/3575495615 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9f80cd8c0 con 0x7fd9f80ae180 2024-04-20T11:35:38.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.917+0000 7fd9f6a00640 1 --2- 172.21.15.152:0/3575495615 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd9f80ae180 0x7fd9f8154e10 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7fd9e800c7f0 tx=0x7fd9e800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:38.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.917+0000 7fd9eea00640 1 -- 172.21.15.152:0/3575495615 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9e8015070 con 0x7fd9f80ae180 2024-04-20T11:35:38.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.917+0000 7fd9eea00640 1 -- 172.21.15.152:0/3575495615 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9e8004240 con 0x7fd9f80ae180 2024-04-20T11:35:38.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.917+0000 7fd9eea00640 1 -- 172.21.15.152:0/3575495615 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9e8007620 con 0x7fd9f80ae180 2024-04-20T11:35:38.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.917+0000 7fd9f7e00640 1 -- 172.21.15.152:0/3575495615 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9f80cdba0 con 0x7fd9f80ae180 2024-04-20T11:35:38.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.917+0000 7fd9f7e00640 1 -- 172.21.15.152:0/3575495615 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9f80ce160 con 0x7fd9f80ae180 2024-04-20T11:35:38.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.918+0000 7fd9f7e00640 1 -- 172.21.15.152:0/3575495615 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9f8075e60 con 0x7fd9f80ae180 2024-04-20T11:35:38.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.919+0000 7fd9eea00640 1 -- 172.21.15.152:0/3575495615 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 21) ==== 91550+0+0 (secure 0 0 0) 0x7fd9e80079d0 con 0x7fd9f80ae180 2024-04-20T11:35:38.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.920+0000 7fd9eea00640 1 --2- 172.21.15.152:0/3575495615 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd9d406cec0 0x7fd9d406f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:38.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.920+0000 7fd9eea00640 1 -- 172.21.15.152:0/3575495615 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5709+0+0 (secure 0 0 0) 0x7fd9e808c640 con 0x7fd9f80ae180 2024-04-20T11:35:38.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.921+0000 7fd9efe00640 1 --2- 172.21.15.152:0/3575495615 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd9d406cec0 0x7fd9d406f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:38.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.922+0000 7fd9efe00640 1 --2- 172.21.15.152:0/3575495615 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd9d406cec0 0x7fd9d406f380 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fd9f809c240 tx=0x7fd9e0006c90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:38.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:38.925+0000 7fd9eea00640 1 -- 172.21.15.152:0/3575495615 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fd9e805a280 con 0x7fd9f80ae180 2024-04-20T11:35:39.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.160+0000 7fd9f7e00640 1 -- 172.21.15.152:0/3575495615 --> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot info", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "snap_name": "snapshot_1", "target": ["mon-mgr", ""]}) -- 0x7fd9f815bf80 con 0x7fd9d406cec0 2024-04-20T11:35:39.163 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:35:39.163+0000 7f8a5cc00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-04-20T11:35:39.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.164+0000 7fd9eea00640 1 -- 172.21.15.152:0/3575495615 <== mgr.4109 v2:172.21.15.175:6836/4021250441 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7fd9f815bf80 con 0x7fd9d406cec0 2024-04-20T11:35:39.163 INFO:teuthology.orchestra.run.smithi152.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-04-20T11:35:39.178 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-20T11:35:39.179 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot protect non_existent_subvolume subvolume_1 snapshot_1 2024-04-20T11:35:39.384 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.385+0000 7ff92d600640 1 Processor -- start 2024-04-20T11:35:39.384 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.385+0000 7ff92d600640 1 -- start start 2024-04-20T11:35:39.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.386+0000 7ff92d600640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ff9280cd8e0 0x7ff92806e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:39.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.386+0000 7ff92d600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff9280cf220 0x7ff9280b4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:39.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.386+0000 7ff92d600640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff92806eab0 con 0x7ff928144d70 2024-04-20T11:35:39.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.386+0000 7ff92d600640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7ff9280d0240 con 0x7ff9280cd8e0 2024-04-20T11:35:39.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.387+0000 7ff92d600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff9280cf6c0 con 0x7ff9280cf220 2024-04-20T11:35:39.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.387+0000 7ff927400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7ff928144d70 0x7ff92815f410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:40518/0 (socket says 172.21.15.152:40518) 2024-04-20T11:35:39.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.387+0000 7ff927400640 1 -- 172.21.15.152:0/2770914917 learned_addr learned my addr 172.21.15.152:0/2770914917 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:35:39.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.387+0000 7ff926000640 1 --2- 172.21.15.152:0/2770914917 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff9280cf220 0x7ff9280b4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:39.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.387+0000 7ff926a00640 1 --2- 172.21.15.152:0/2770914917 >> v2:172.21.15.175:3300/0 conn(0x7ff9280cd8e0 0x7ff92806e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:39.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.387+0000 7ff925600640 1 -- 172.21.15.152:0/2770914917 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3911300213 0 0) 0x7ff92806eab0 con 0x7ff928144d70 2024-04-20T11:35:39.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.387+0000 7ff925600640 1 -- 172.21.15.152:0/2770914917 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff90c003520 con 0x7ff928144d70 2024-04-20T11:35:39.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.388+0000 7ff925600640 1 -- 172.21.15.152:0/2770914917 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 664512587 0 0) 0x7ff90c003520 con 0x7ff928144d70 2024-04-20T11:35:39.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.388+0000 7ff925600640 1 -- 172.21.15.152:0/2770914917 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff9280cf220 msgr2=0x7ff9280b4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:39.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.388+0000 7ff925600640 1 --2- 172.21.15.152:0/2770914917 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff9280cf220 0x7ff9280b4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:39.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.388+0000 7ff925600640 1 -- 172.21.15.152:0/2770914917 >> v2:172.21.15.175:3300/0 conn(0x7ff9280cd8e0 msgr2=0x7ff92806e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:39.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.388+0000 7ff925600640 1 --2- 172.21.15.152:0/2770914917 >> v2:172.21.15.175:3300/0 conn(0x7ff9280cd8e0 0x7ff92806e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:39.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.388+0000 7ff925600640 1 -- 172.21.15.152:0/2770914917 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9280cf9a0 con 0x7ff928144d70 2024-04-20T11:35:39.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.388+0000 7ff925600640 1 -- 172.21.15.152:0/2770914917 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7ff920001a10 con 0x7ff928144d70 2024-04-20T11:35:39.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.388+0000 7ff925600640 1 -- 172.21.15.152:0/2770914917 >> v1:172.21.15.152:6789/0 conn(0x7ff928144d70 legacy=0x7ff92815f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:39.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.388+0000 7ff925600640 1 --2- 172.21.15.152:0/2770914917 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff90c003a20 0x7ff90c003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:39.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.388+0000 7ff925600640 1 --2- 172.21.15.152:0/2770914917 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff90c004800 0x7ff90c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:39.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.388+0000 7ff925600640 1 --2- 172.21.15.152:0/2770914917 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff9280cf220 0x7ff90c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:39.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.388+0000 7ff925600640 1 -- 172.21.15.152:0/2770914917 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7ff9280d0240 con 0x7ff90c003a20 2024-04-20T11:35:39.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.388+0000 7ff925600640 1 -- 172.21.15.152:0/2770914917 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff9280cf6c0 con 0x7ff90c004800 2024-04-20T11:35:39.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.388+0000 7ff925600640 1 -- 172.21.15.152:0/2770914917 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff90c00dc50 con 0x7ff9280cf220 2024-04-20T11:35:39.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.388+0000 7ff926a00640 1 --2- 172.21.15.152:0/2770914917 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff90c003a20 0x7ff90c003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:39.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.389+0000 7ff926000640 1 --2- 172.21.15.152:0/2770914917 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff90c004800 0x7ff90c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:39.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.389+0000 7ff927400640 1 --2- 172.21.15.152:0/2770914917 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff9280cf220 0x7ff90c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:39.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.389+0000 7ff926a00640 1 -- 172.21.15.152:0/2770914917 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff9280cf220 msgr2=0x7ff90c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:39.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.389+0000 7ff926a00640 1 --2- 172.21.15.152:0/2770914917 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff9280cf220 0x7ff90c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:39.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.389+0000 7ff926a00640 1 -- 172.21.15.152:0/2770914917 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff90c004800 msgr2=0x7ff90c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:39.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.389+0000 7ff926a00640 1 --2- 172.21.15.152:0/2770914917 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff90c004800 0x7ff90c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:39.391 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.389+0000 7ff926a00640 1 -- 172.21.15.152:0/2770914917 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff90c003520 con 0x7ff90c003a20 2024-04-20T11:35:39.391 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.389+0000 7ff927400640 1 --2- 172.21.15.152:0/2770914917 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff9280cf220 0x7ff90c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:35:39.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.390+0000 7ff926a00640 1 --2- 172.21.15.152:0/2770914917 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff90c003a20 0x7ff90c003e40 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7ff918009ac0 tx=0x7ff90c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5936154276848ed4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:39.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.390+0000 7ff925600640 1 -- 172.21.15.152:0/2770914917 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff918039070 con 0x7ff90c003a20 2024-04-20T11:35:39.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.390+0000 7ff925600640 1 -- 172.21.15.152:0/2770914917 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff91802c710 con 0x7ff90c003a20 2024-04-20T11:35:39.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.390+0000 7ff925600640 1 -- 172.21.15.152:0/2770914917 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff918009690 con 0x7ff90c003a20 2024-04-20T11:35:39.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.391+0000 7ff92d600640 1 -- 172.21.15.152:0/2770914917 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff90c003a20 msgr2=0x7ff90c003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:39.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.391+0000 7ff92d600640 1 --2- 172.21.15.152:0/2770914917 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff90c003a20 0x7ff90c003e40 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7ff918009ac0 tx=0x7ff90c005830 comp rx=0 tx=0).stop 2024-04-20T11:35:39.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.391+0000 7ff927400640 1 -- 172.21.15.152:0/2770914917 reap_dead start 2024-04-20T11:35:39.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.391+0000 7ff92d600640 1 -- 172.21.15.152:0/2770914917 shutdown_connections 2024-04-20T11:35:39.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.391+0000 7ff92d600640 1 -- 172.21.15.152:0/2770914917 >> 172.21.15.152:0/2770914917 conn(0x7ff92806b980 msgr2=0x7ff92806bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:39.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.391+0000 7ff92d600640 1 -- 172.21.15.152:0/2770914917 shutdown_connections 2024-04-20T11:35:39.393 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.391+0000 7ff92d600640 1 -- 172.21.15.152:0/2770914917 wait complete. 2024-04-20T11:35:39.393 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.393+0000 7ff92d600640 1 Processor -- start 2024-04-20T11:35:39.394 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.393+0000 7ff92d600640 1 -- start start 2024-04-20T11:35:39.394 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.394+0000 7ff92d600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff90c003a20 0x7ff92813cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:39.394 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.394+0000 7ff92d600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff9281e56a0 0x7ff9281e5b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:39.395 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.394+0000 7ff92d600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff9281f81c0 0x7ff9281f8640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:39.395 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.394+0000 7ff92d600640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7ff9280cdcc0 con 0x7ff9281e56a0 2024-04-20T11:35:39.395 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.394+0000 7ff92d600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff9280cff60 con 0x7ff9281f81c0 2024-04-20T11:35:39.395 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.394+0000 7ff92d600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff9281447b0 con 0x7ff90c003a20 2024-04-20T11:35:39.395 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.394+0000 7ff926000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff9281e56a0 0x7ff9281e5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:39.395 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.394+0000 7ff926000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff9281e56a0 0x7ff9281e5b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:37938/0 (socket says 172.21.15.152:37938) 2024-04-20T11:35:39.395 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.394+0000 7ff926000640 1 -- 172.21.15.152:0/3098285504 learned_addr learned my addr 172.21.15.152:0/3098285504 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:39.395 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.394+0000 7ff926a00640 1 --2- 172.21.15.152:0/3098285504 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff90c003a20 0x7ff92813cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:39.395 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.395+0000 7ff927400640 1 --2- 172.21.15.152:0/3098285504 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff9281f81c0 0x7ff9281f8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:39.395 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.395+0000 7ff926000640 1 -- 172.21.15.152:0/3098285504 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff90c003a20 msgr2=0x7ff92813cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:39.395 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.395+0000 7ff926000640 1 --2- 172.21.15.152:0/3098285504 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff90c003a20 0x7ff92813cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:39.395 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.395+0000 7ff926000640 1 -- 172.21.15.152:0/3098285504 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff9281f81c0 msgr2=0x7ff9281f8640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:39.395 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.395+0000 7ff926000640 1 --2- 172.21.15.152:0/3098285504 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff9281f81c0 0x7ff9281f8640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:39.395 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.395+0000 7ff926000640 1 -- 172.21.15.152:0/3098285504 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9281f8b80 con 0x7ff9281e56a0 2024-04-20T11:35:39.396 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.395+0000 7ff926000640 1 --2- 172.21.15.152:0/3098285504 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff9281e56a0 0x7ff9281e5b20 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7ff914011600 tx=0x7ff914011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:39.396 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.395+0000 7ff91ea00640 1 -- 172.21.15.152:0/3098285504 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff91400f920 con 0x7ff9281e56a0 2024-04-20T11:35:39.396 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.395+0000 7ff91ea00640 1 -- 172.21.15.152:0/3098285504 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff91400fac0 con 0x7ff9281e56a0 2024-04-20T11:35:39.396 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.395+0000 7ff92d600640 1 -- 172.21.15.152:0/3098285504 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9281f8e10 con 0x7ff9281e56a0 2024-04-20T11:35:39.397 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.395+0000 7ff91ea00640 1 -- 172.21.15.152:0/3098285504 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff91400fe70 con 0x7ff9281e56a0 2024-04-20T11:35:39.397 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.396+0000 7ff92d600640 1 -- 172.21.15.152:0/3098285504 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9281411e0 con 0x7ff9281e56a0 2024-04-20T11:35:39.397 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.397+0000 7ff92d600640 1 -- 172.21.15.152:0/3098285504 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff92815b560 con 0x7ff9281e56a0 2024-04-20T11:35:39.397 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.398+0000 7ff91ea00640 1 -- 172.21.15.152:0/3098285504 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 21) ==== 91550+0+0 (secure 0 0 0) 0x7ff914015460 con 0x7ff9281e56a0 2024-04-20T11:35:39.398 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.399+0000 7ff91ea00640 1 --2- 172.21.15.152:0/3098285504 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7ff90806ce90 0x7ff90806f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:39.398 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.399+0000 7ff91ea00640 1 -- 172.21.15.152:0/3098285504 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5709+0+0 (secure 0 0 0) 0x7ff91408cfa0 con 0x7ff9281e56a0 2024-04-20T11:35:39.399 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.399+0000 7ff926a00640 1 --2- 172.21.15.152:0/3098285504 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7ff90806ce90 0x7ff90806f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:39.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.401+0000 7ff926a00640 1 --2- 172.21.15.152:0/3098285504 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7ff90806ce90 0x7ff90806f350 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ff918036000 tx=0x7ff918030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:39.403 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.404+0000 7ff91ea00640 1 -- 172.21.15.152:0/3098285504 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ff914019030 con 0x7ff9281e56a0 2024-04-20T11:35:39.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.646+0000 7ff92d600640 1 -- 172.21.15.152:0/3098285504 --> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot protect", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "snap_name": "snapshot_1", "target": ["mon-mgr", ""]}) -- 0x7ff928068a40 con 0x7ff90806ce90 2024-04-20T11:35:39.650 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:35:39.650+0000 7f8a5cc00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-04-20T11:35:39.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.651+0000 7ff91ea00640 1 -- 172.21.15.152:0/3098285504 <== mgr.4109 v2:172.21.15.175:6836/4021250441 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7ff928068a40 con 0x7ff90806ce90 2024-04-20T11:35:39.650 INFO:teuthology.orchestra.run.smithi152.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-04-20T11:35:39.661 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-20T11:35:39.662 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot unprotect non_existent_subvolume subvolume_1 snapshot_1 2024-04-20T11:35:39.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.865+0000 7f3f95e00640 1 Processor -- start 2024-04-20T11:35:39.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.866+0000 7f3f95e00640 1 -- start start 2024-04-20T11:35:39.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.867+0000 7f3f95e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f3f9015e870 0x7f3f900d3370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:39.867 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.867+0000 7f3f95e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f3f900d0fa0 0x7f3f900d1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:39.867 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.867+0000 7f3f95e00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f9015ec50 con 0x7f3f900cff90 2024-04-20T11:35:39.867 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.867+0000 7f3f95e00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f3f900d2ee0 con 0x7f3f900d0fa0 2024-04-20T11:35:39.867 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.867+0000 7f3f95e00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f3f900d38b0 con 0x7f3f9015e870 2024-04-20T11:35:39.867 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.867+0000 7f3f8fe00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f3f900d0fa0 0x7f3f900d1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:39.867 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.867+0000 7f3f8fe00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f3f900d0fa0 0x7f3f900d1450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:37954/0 (socket says 172.21.15.152:37954) 2024-04-20T11:35:39.867 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.867+0000 7f3f8fe00640 1 -- 172.21.15.152:0/3672981196 learned_addr learned my addr 172.21.15.152:0/3672981196 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:39.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.868+0000 7f3f94a00640 1 --2- 172.21.15.152:0/3672981196 >> v2:172.21.15.175:3300/0 conn(0x7f3f9015e870 0x7f3f900d3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:39.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.868+0000 7f3f8fe00640 1 -- 172.21.15.152:0/3672981196 >> v2:172.21.15.175:3300/0 conn(0x7f3f9015e870 msgr2=0x7f3f900d3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:39.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.868+0000 7f3f8fe00640 1 --2- 172.21.15.152:0/3672981196 >> v2:172.21.15.175:3300/0 conn(0x7f3f9015e870 0x7f3f900d3370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:39.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.868+0000 7f3f8fe00640 1 -- 172.21.15.152:0/3672981196 >> v1:172.21.15.175:6789/0 conn(0x7f3f900cff90 legacy=0x7f3f900d0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:39.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.868+0000 7f3f8fe00640 1 -- 172.21.15.152:0/3672981196 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f900d22d0 con 0x7f3f900d0fa0 2024-04-20T11:35:39.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.869+0000 7f3f8fe00640 1 --2- 172.21.15.152:0/3672981196 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3f900d0fa0 0x7f3f900d1450 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7f3f9008e460 tx=0x7f3f7802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=829614628bd29d77 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:39.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.869+0000 7f3f8f400640 1 -- 172.21.15.152:0/3672981196 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f78037070 con 0x7f3f900d0fa0 2024-04-20T11:35:39.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.869+0000 7f3f8f400640 1 -- 172.21.15.152:0/3672981196 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f78004230 con 0x7f3f900d0fa0 2024-04-20T11:35:39.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.869+0000 7f3f8f400640 1 -- 172.21.15.152:0/3672981196 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f78003610 con 0x7f3f900d0fa0 2024-04-20T11:35:39.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.870+0000 7f3f95e00640 1 -- 172.21.15.152:0/3672981196 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3f900d0fa0 msgr2=0x7f3f900d1450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:39.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.870+0000 7f3f95e00640 1 --2- 172.21.15.152:0/3672981196 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3f900d0fa0 0x7f3f900d1450 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7f3f9008e460 tx=0x7f3f7802b860 comp rx=0 tx=0).stop 2024-04-20T11:35:39.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.870+0000 7f3f95e00640 1 -- 172.21.15.152:0/3672981196 shutdown_connections 2024-04-20T11:35:39.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.870+0000 7f3f95e00640 1 --2- 172.21.15.152:0/3672981196 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3f900d0fa0 0x7f3f900d1450 unknown :-1 s=CLOSED pgs=1818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:39.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.870+0000 7f3f95e00640 1 --2- 172.21.15.152:0/3672981196 >> v2:172.21.15.175:3300/0 conn(0x7f3f9015e870 0x7f3f900d3370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:39.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.870+0000 7f3f95e00640 1 -- 172.21.15.152:0/3672981196 >> 172.21.15.152:0/3672981196 conn(0x7f3f9008bfd0 msgr2=0x7f3f900d45d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:39.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.870+0000 7f3f95e00640 1 -- 172.21.15.152:0/3672981196 shutdown_connections 2024-04-20T11:35:39.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.870+0000 7f3f95e00640 1 -- 172.21.15.152:0/3672981196 wait complete. 2024-04-20T11:35:39.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.872+0000 7f3f95e00640 1 Processor -- start 2024-04-20T11:35:39.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.872+0000 7f3f95e00640 1 -- start start 2024-04-20T11:35:39.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.873+0000 7f3f95e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3f900cff90 0x7f3f90152de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:39.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.873+0000 7f3f95e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3f900d0fa0 0x7f3f90160b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:39.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.873+0000 7f3f95e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3f9015e870 0x7f3f90161050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:39.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.873+0000 7f3f95e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f3f900d2b70 con 0x7f3f900cff90 2024-04-20T11:35:39.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.873+0000 7f3f95e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3f900d2890 con 0x7f3f9015e870 2024-04-20T11:35:39.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.873+0000 7f3f95e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f3f9006ca80 con 0x7f3f900d0fa0 2024-04-20T11:35:39.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.873+0000 7f3f94a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3f900cff90 0x7f3f90152de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:39.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.873+0000 7f3f94a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3f900cff90 0x7f3f90152de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:37964/0 (socket says 172.21.15.152:37964) 2024-04-20T11:35:39.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.873+0000 7f3f94a00640 1 -- 172.21.15.152:0/280020244 learned_addr learned my addr 172.21.15.152:0/280020244 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:39.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.874+0000 7f3f94a00640 1 -- 172.21.15.152:0/280020244 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3f900d0fa0 msgr2=0x7f3f90160b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:39.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.874+0000 7f3f94a00640 1 --2- 172.21.15.152:0/280020244 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3f900d0fa0 0x7f3f90160b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:39.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.874+0000 7f3f94a00640 1 -- 172.21.15.152:0/280020244 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3f9015e870 msgr2=0x7f3f90161050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:39.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.874+0000 7f3f95400640 1 --2- 172.21.15.152:0/280020244 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3f9015e870 0x7f3f90161050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:39.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.874+0000 7f3f94a00640 1 --2- 172.21.15.152:0/280020244 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3f9015e870 0x7f3f90161050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:39.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.874+0000 7f3f94a00640 1 -- 172.21.15.152:0/280020244 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f900ca9b0 con 0x7f3f900cff90 2024-04-20T11:35:39.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.874+0000 7f3f95400640 1 --2- 172.21.15.152:0/280020244 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3f9015e870 0x7f3f90161050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:35:39.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.874+0000 7f3f94a00640 1 --2- 172.21.15.152:0/280020244 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3f900cff90 0x7f3f90152de0 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f3f8000d5f0 tx=0x7f3f8000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:39.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.874+0000 7f3f8cc00640 1 -- 172.21.15.152:0/280020244 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f800125f0 con 0x7f3f900cff90 2024-04-20T11:35:39.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.874+0000 7f3f8cc00640 1 -- 172.21.15.152:0/280020244 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f80019070 con 0x7f3f900cff90 2024-04-20T11:35:39.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.874+0000 7f3f8cc00640 1 -- 172.21.15.152:0/280020244 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f800129d0 con 0x7f3f900cff90 2024-04-20T11:35:39.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.874+0000 7f3f95e00640 1 -- 172.21.15.152:0/280020244 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f900cac90 con 0x7f3f900cff90 2024-04-20T11:35:39.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.874+0000 7f3f95e00640 1 -- 172.21.15.152:0/280020244 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f900b7160 con 0x7f3f900cff90 2024-04-20T11:35:39.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.876+0000 7f3f95e00640 1 -- 172.21.15.152:0/280020244 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f90075e60 con 0x7f3f900cff90 2024-04-20T11:35:39.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.877+0000 7f3f8cc00640 1 -- 172.21.15.152:0/280020244 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 21) ==== 91550+0+0 (secure 0 0 0) 0x7f3f80012b70 con 0x7f3f900cff90 2024-04-20T11:35:39.877 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.877+0000 7f3f8cc00640 1 --2- 172.21.15.152:0/280020244 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f3f6c06ce90 0x7f3f6c06f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:39.877 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.878+0000 7f3f8cc00640 1 -- 172.21.15.152:0/280020244 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5709+0+0 (secure 0 0 0) 0x7f3f8008c9a0 con 0x7f3f900cff90 2024-04-20T11:35:39.877 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.878+0000 7f3f8fe00640 1 --2- 172.21.15.152:0/280020244 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f3f6c06ce90 0x7f3f6c06f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:39.879 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.879+0000 7f3f8fe00640 1 --2- 172.21.15.152:0/280020244 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f3f6c06ce90 0x7f3f6c06f350 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f3f78034000 tx=0x7f3f78030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:39.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:39.882+0000 7f3f8cc00640 1 -- 172.21.15.152:0/280020244 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f3f8005a5b0 con 0x7f3f900cff90 2024-04-20T11:35:40.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.116+0000 7f3f95e00640 1 -- 172.21.15.152:0/280020244 --> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot unprotect", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "snap_name": "snapshot_1", "target": ["mon-mgr", ""]}) -- 0x7f3f9015cab0 con 0x7f3f6c06ce90 2024-04-20T11:35:40.119 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:35:40.119+0000 7f8a5cc00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-04-20T11:35:40.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.120+0000 7f3f8cc00640 1 -- 172.21.15.152:0/280020244 <== mgr.4109 v2:172.21.15.175:6836/4021250441 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f3f9015cab0 con 0x7f3f6c06ce90 2024-04-20T11:35:40.119 INFO:teuthology.orchestra.run.smithi152.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-04-20T11:35:40.140 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-20T11:35:40.141 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls non_existent_subvolume subvolume_1 2024-04-20T11:35:40.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.349+0000 7fb056800640 1 Processor -- start 2024-04-20T11:35:40.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.349+0000 7fb056800640 1 -- start start 2024-04-20T11:35:40.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.351+0000 7fb056800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fb0500d05f0 0x7fb0500d09d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:40.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.351+0000 7fb056800640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fb0500ae180 0x7fb0500ae600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:40.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.351+0000 7fb056800640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0500d0f10 con 0x7fb05015eee0 2024-04-20T11:35:40.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.351+0000 7fb056800640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fb0500aeef0 con 0x7fb0500d05f0 2024-04-20T11:35:40.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.351+0000 7fb056800640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fb0500af1d0 con 0x7fb0500ae180 2024-04-20T11:35:40.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.351+0000 7fb055400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fb0500d05f0 0x7fb0500d09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:40.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.351+0000 7fb055400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fb0500d05f0 0x7fb0500d09d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:37978/0 (socket says 172.21.15.152:37978) 2024-04-20T11:35:40.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.351+0000 7fb055400640 1 -- 172.21.15.152:0/1959345634 learned_addr learned my addr 172.21.15.152:0/1959345634 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:40.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.351+0000 7fb054a00640 1 --2- 172.21.15.152:0/1959345634 >> v2:172.21.15.175:3300/0 conn(0x7fb0500ae180 0x7fb0500ae600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:40.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.352+0000 7fb055400640 1 -- 172.21.15.152:0/1959345634 >> v2:172.21.15.175:3300/0 conn(0x7fb0500ae180 msgr2=0x7fb0500ae600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:40.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.352+0000 7fb055400640 1 --2- 172.21.15.152:0/1959345634 >> v2:172.21.15.175:3300/0 conn(0x7fb0500ae180 0x7fb0500ae600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:40.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.352+0000 7fb055400640 1 -- 172.21.15.152:0/1959345634 >> v1:172.21.15.175:6789/0 conn(0x7fb05015eee0 legacy=0x7fb05015f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:40.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.352+0000 7fb055400640 1 -- 172.21.15.152:0/1959345634 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0500af4b0 con 0x7fb0500d05f0 2024-04-20T11:35:40.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.353+0000 7fb055400640 1 --2- 172.21.15.152:0/1959345634 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb0500d05f0 0x7fb0500d09d0 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7fb0500b6360 tx=0x7fb04402db20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f225e605e3ad5a41 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:40.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.353+0000 7fb04fe00640 1 -- 172.21.15.152:0/1959345634 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb04402f8e0 con 0x7fb0500d05f0 2024-04-20T11:35:40.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.353+0000 7fb04fe00640 1 -- 172.21.15.152:0/1959345634 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb04402fa80 con 0x7fb0500d05f0 2024-04-20T11:35:40.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.353+0000 7fb04fe00640 1 -- 172.21.15.152:0/1959345634 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb04402fe50 con 0x7fb0500d05f0 2024-04-20T11:35:40.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.354+0000 7fb056800640 1 -- 172.21.15.152:0/1959345634 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb0500d05f0 msgr2=0x7fb0500d09d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:40.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.354+0000 7fb056800640 1 --2- 172.21.15.152:0/1959345634 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb0500d05f0 0x7fb0500d09d0 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7fb0500b6360 tx=0x7fb04402db20 comp rx=0 tx=0).stop 2024-04-20T11:35:40.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.354+0000 7fb056800640 1 -- 172.21.15.152:0/1959345634 shutdown_connections 2024-04-20T11:35:40.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.354+0000 7fb056800640 1 --2- 172.21.15.152:0/1959345634 >> v2:172.21.15.175:3300/0 conn(0x7fb0500ae180 0x7fb0500ae600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:40.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.354+0000 7fb056800640 1 --2- 172.21.15.152:0/1959345634 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb0500d05f0 0x7fb0500d09d0 unknown :-1 s=CLOSED pgs=1820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:40.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.354+0000 7fb056800640 1 -- 172.21.15.152:0/1959345634 >> 172.21.15.152:0/1959345634 conn(0x7fb0500b21e0 msgr2=0x7fb0500b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:40.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.354+0000 7fb056800640 1 -- 172.21.15.152:0/1959345634 shutdown_connections 2024-04-20T11:35:40.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.354+0000 7fb056800640 1 -- 172.21.15.152:0/1959345634 wait complete. 2024-04-20T11:35:40.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.356+0000 7fb056800640 1 Processor -- start 2024-04-20T11:35:40.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.356+0000 7fb056800640 1 -- start start 2024-04-20T11:35:40.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.357+0000 7fb056800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb0500ae180 0x7fb05015e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:40.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.357+0000 7fb056800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb0500d05f0 0x7fb05015cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:40.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.357+0000 7fb056800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb05015eee0 0x7fb0501f3e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:40.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.357+0000 7fb056800640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fb050155ca0 con 0x7fb0500d05f0 2024-04-20T11:35:40.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.357+0000 7fb056800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb0500afa70 con 0x7fb05015eee0 2024-04-20T11:35:40.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.357+0000 7fb056800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb05015e2d0 con 0x7fb0500ae180 2024-04-20T11:35:40.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.357+0000 7fb054a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb0500d05f0 0x7fb05015cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:40.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.357+0000 7fb054a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb0500d05f0 0x7fb05015cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:37984/0 (socket says 172.21.15.152:37984) 2024-04-20T11:35:40.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.357+0000 7fb054a00640 1 -- 172.21.15.152:0/1103695855 learned_addr learned my addr 172.21.15.152:0/1103695855 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:40.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.357+0000 7fb054a00640 1 -- 172.21.15.152:0/1103695855 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb0500ae180 msgr2=0x7fb05015e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:40.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.357+0000 7fb054a00640 1 --2- 172.21.15.152:0/1103695855 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb0500ae180 0x7fb05015e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:40.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.357+0000 7fb054a00640 1 -- 172.21.15.152:0/1103695855 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb05015eee0 msgr2=0x7fb0501f3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:40.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.357+0000 7fb055e00640 1 --2- 172.21.15.152:0/1103695855 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb05015eee0 0x7fb0501f3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:40.359 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.357+0000 7fb054a00640 1 --2- 172.21.15.152:0/1103695855 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb05015eee0 0x7fb0501f3e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:40.359 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.357+0000 7fb054a00640 1 -- 172.21.15.152:0/1103695855 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb05015d6c0 con 0x7fb0500d05f0 2024-04-20T11:35:40.359 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.358+0000 7fb054a00640 1 --2- 172.21.15.152:0/1103695855 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb0500d05f0 0x7fb05015cde0 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7fb03c00d800 tx=0x7fb03c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:40.359 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.358+0000 7fb04d600640 1 -- 172.21.15.152:0/1103695855 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb03c00b6c0 con 0x7fb0500d05f0 2024-04-20T11:35:40.359 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.358+0000 7fb04d600640 1 -- 172.21.15.152:0/1103695855 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb03c019070 con 0x7fb0500d05f0 2024-04-20T11:35:40.359 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.358+0000 7fb04d600640 1 -- 172.21.15.152:0/1103695855 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb03c00ba80 con 0x7fb0500d05f0 2024-04-20T11:35:40.359 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.358+0000 7fb056800640 1 -- 172.21.15.152:0/1103695855 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb05015d9a0 con 0x7fb0500d05f0 2024-04-20T11:35:40.359 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.358+0000 7fb056800640 1 -- 172.21.15.152:0/1103695855 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb05015df60 con 0x7fb0500d05f0 2024-04-20T11:35:40.359 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.359+0000 7fb056800640 1 -- 172.21.15.152:0/1103695855 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb050075e60 con 0x7fb0500d05f0 2024-04-20T11:35:40.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.360+0000 7fb04d600640 1 -- 172.21.15.152:0/1103695855 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 21) ==== 91550+0+0 (secure 0 0 0) 0x7fb03c00bc20 con 0x7fb0500d05f0 2024-04-20T11:35:40.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.362+0000 7fb04d600640 1 --2- 172.21.15.152:0/1103695855 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb03006ce90 0x7fb03006f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:40.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.362+0000 7fb04d600640 1 -- 172.21.15.152:0/1103695855 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5709+0+0 (secure 0 0 0) 0x7fb03c08ca00 con 0x7fb0500d05f0 2024-04-20T11:35:40.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.362+0000 7fb055400640 1 --2- 172.21.15.152:0/1103695855 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb03006ce90 0x7fb03006f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:40.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.364+0000 7fb055400640 1 --2- 172.21.15.152:0/1103695855 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb03006ce90 0x7fb03006f350 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fb044038000 tx=0x7fb044032000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:40.366 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.366+0000 7fb04d600640 1 -- 172.21.15.152:0/1103695855 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fb03c05a650 con 0x7fb0500d05f0 2024-04-20T11:35:40.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.604+0000 7fb056800640 1 -- 172.21.15.152:0/1103695855 --> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "target": ["mon-mgr", ""]}) -- 0x7fb0500b1950 con 0x7fb03006ce90 2024-04-20T11:35:40.608 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:35:40.608+0000 7f8a5cc00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-04-20T11:35:40.608 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.609+0000 7fb04d600640 1 -- 172.21.15.152:0/1103695855 <== mgr.4109 v2:172.21.15.175:6836/4021250441 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7fb0500b1950 con 0x7fb03006ce90 2024-04-20T11:35:40.608 INFO:teuthology.orchestra.run.smithi152.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-04-20T11:35:40.625 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-20T11:35:40.625 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone non_existent_subvolume subvolume_1 snapshot_1 clone_1 2024-04-20T11:35:40.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.837+0000 7fcc7f200640 1 Processor -- start 2024-04-20T11:35:40.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.837+0000 7fcc7f200640 1 -- start start 2024-04-20T11:35:40.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.839+0000 7fcc7f200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fcc80167110 0x7fcc80071cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:40.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.839+0000 7fcc7f200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcc80072210 0x7fcc8006e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:40.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.839+0000 7fcc7f200640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc80167570 con 0x7fcc80162ac0 2024-04-20T11:35:40.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.839+0000 7fcc7f200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fcc80072660 con 0x7fcc80167110 2024-04-20T11:35:40.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.839+0000 7fcc7f200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fcc800b42f0 con 0x7fcc80072210 2024-04-20T11:35:40.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.839+0000 7fcc7de00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fcc80162ac0 0x7fcc800b46b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:40528/0 (socket says 172.21.15.152:40528) 2024-04-20T11:35:40.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.839+0000 7fcc7de00640 1 -- 172.21.15.152:0/2814104419 learned_addr learned my addr 172.21.15.152:0/2814104419 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:35:40.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.839+0000 7fcc7ca00640 1 -- 172.21.15.152:0/2814104419 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3618574513 0 0) 0x7fcc80167570 con 0x7fcc80162ac0 2024-04-20T11:35:40.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.839+0000 7fcc7d400640 1 --2- 172.21.15.152:0/2814104419 >> v2:172.21.15.175:3300/0 conn(0x7fcc80167110 0x7fcc80071cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:40.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.839+0000 7fcc7ca00640 1 -- 172.21.15.152:0/2814104419 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcc64003520 con 0x7fcc80162ac0 2024-04-20T11:35:40.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.840+0000 7fcc7e800640 1 --2- 172.21.15.152:0/2814104419 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcc80072210 0x7fcc8006e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:40.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.840+0000 7fcc7ca00640 1 -- 172.21.15.152:0/2814104419 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 129221165 0 0) 0x7fcc64003520 con 0x7fcc80162ac0 2024-04-20T11:35:40.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.840+0000 7fcc7ca00640 1 -- 172.21.15.152:0/2814104419 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcc80072210 msgr2=0x7fcc8006e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:40.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.840+0000 7fcc7ca00640 1 --2- 172.21.15.152:0/2814104419 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcc80072210 0x7fcc8006e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:40.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.840+0000 7fcc7ca00640 1 -- 172.21.15.152:0/2814104419 >> v2:172.21.15.175:3300/0 conn(0x7fcc80167110 msgr2=0x7fcc80071cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:40.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.840+0000 7fcc7ca00640 1 --2- 172.21.15.152:0/2814104419 >> v2:172.21.15.175:3300/0 conn(0x7fcc80167110 0x7fcc80071cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:40.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.840+0000 7fcc7ca00640 1 -- 172.21.15.152:0/2814104419 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc800b3770 con 0x7fcc80162ac0 2024-04-20T11:35:40.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.840+0000 7fcc7ca00640 1 -- 172.21.15.152:0/2814104419 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7fcc70001690 con 0x7fcc80162ac0 2024-04-20T11:35:40.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.840+0000 7fcc7d400640 1 --2- 172.21.15.152:0/2814104419 >> v2:172.21.15.175:3300/0 conn(0x7fcc80167110 0x7fcc80071cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:35:40.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.840+0000 7fcc7ca00640 1 -- 172.21.15.152:0/2814104419 >> v1:172.21.15.152:6789/0 conn(0x7fcc80162ac0 legacy=0x7fcc800b46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:40.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.840+0000 7fcc7ca00640 1 --2- 172.21.15.152:0/2814104419 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcc640039b0 0x7fcc64003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:40.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.840+0000 7fcc7ca00640 1 --2- 172.21.15.152:0/2814104419 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcc80072210 0x7fcc64004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:40.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.840+0000 7fcc7ca00640 1 --2- 172.21.15.152:0/2814104419 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcc64004d40 0x7fcc640051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:40.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.840+0000 7fcc7ca00640 1 -- 172.21.15.152:0/2814104419 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fcc80072660 con 0x7fcc640039b0 2024-04-20T11:35:40.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.840+0000 7fcc7ca00640 1 -- 172.21.15.152:0/2814104419 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcc800b42f0 con 0x7fcc64004d40 2024-04-20T11:35:40.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.840+0000 7fcc7ca00640 1 -- 172.21.15.152:0/2814104419 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fcc6400bb20 con 0x7fcc80072210 2024-04-20T11:35:40.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.840+0000 7fcc7de00640 1 --2- 172.21.15.152:0/2814104419 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcc640039b0 0x7fcc64003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:40.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.841+0000 7fcc7d400640 1 --2- 172.21.15.152:0/2814104419 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcc80072210 0x7fcc64004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:40.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.841+0000 7fcc7de00640 1 -- 172.21.15.152:0/2814104419 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcc80072210 msgr2=0x7fcc64004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:40.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.841+0000 7fcc7de00640 1 --2- 172.21.15.152:0/2814104419 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcc80072210 0x7fcc64004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:40.843 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.841+0000 7fcc7e800640 1 --2- 172.21.15.152:0/2814104419 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcc64004d40 0x7fcc640051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:40.843 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.841+0000 7fcc7de00640 1 -- 172.21.15.152:0/2814104419 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcc64004d40 msgr2=0x7fcc640051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:40.843 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.841+0000 7fcc7de00640 1 --2- 172.21.15.152:0/2814104419 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcc64004d40 0x7fcc640051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:40.843 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.841+0000 7fcc7de00640 1 -- 172.21.15.152:0/2814104419 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc64003520 con 0x7fcc640039b0 2024-04-20T11:35:40.844 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.842+0000 7fcc7de00640 1 --2- 172.21.15.152:0/2814104419 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcc640039b0 0x7fcc64003dd0 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7fcc7000c210 tx=0x7fcc64005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d08a080d6a08a2cd server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:40.844 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.842+0000 7fcc7ca00640 1 -- 172.21.15.152:0/2814104419 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc70007d60 con 0x7fcc640039b0 2024-04-20T11:35:40.844 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.842+0000 7fcc7ca00640 1 -- 172.21.15.152:0/2814104419 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc70030070 con 0x7fcc640039b0 2024-04-20T11:35:40.844 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.842+0000 7fcc7ca00640 1 -- 172.21.15.152:0/2814104419 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc7002cd20 con 0x7fcc640039b0 2024-04-20T11:35:40.844 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.842+0000 7fcc7f200640 1 -- 172.21.15.152:0/2814104419 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcc640039b0 msgr2=0x7fcc64003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:40.844 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.842+0000 7fcc7f200640 1 --2- 172.21.15.152:0/2814104419 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcc640039b0 0x7fcc64003dd0 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7fcc7000c210 tx=0x7fcc64005830 comp rx=0 tx=0).stop 2024-04-20T11:35:40.845 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.842+0000 7fcc7e800640 1 -- 172.21.15.152:0/2814104419 reap_dead start 2024-04-20T11:35:40.845 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.843+0000 7fcc7f200640 1 -- 172.21.15.152:0/2814104419 shutdown_connections 2024-04-20T11:35:40.845 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.843+0000 7fcc7f200640 1 -- 172.21.15.152:0/2814104419 >> 172.21.15.152:0/2814104419 conn(0x7fcc80067ae0 msgr2=0x7fcc8006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:40.845 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.843+0000 7fcc7f200640 1 -- 172.21.15.152:0/2814104419 shutdown_connections 2024-04-20T11:35:40.845 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.843+0000 7fcc7f200640 1 -- 172.21.15.152:0/2814104419 wait complete. 2024-04-20T11:35:40.845 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.844+0000 7fcc7f200640 1 Processor -- start 2024-04-20T11:35:40.845 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.844+0000 7fcc7f200640 1 -- start start 2024-04-20T11:35:40.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.845+0000 7fcc7f200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcc640039b0 0x7fcc80165500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:40.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.845+0000 7fcc7f200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcc800b0050 0x7fcc80165a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:40.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.845+0000 7fcc7f200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcc800b09d0 0x7fcc800b0e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:40.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.845+0000 7fcc7f200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fcc800b4010 con 0x7fcc800b0050 2024-04-20T11:35:40.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.845+0000 7fcc7f200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcc800b3d30 con 0x7fcc800b09d0 2024-04-20T11:35:40.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.845+0000 7fcc7f200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fcc8006f530 con 0x7fcc640039b0 2024-04-20T11:35:40.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.846+0000 7fcc7d400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcc800b0050 0x7fcc80165a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:40.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.846+0000 7fcc7d400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcc800b0050 0x7fcc80165a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:38004/0 (socket says 172.21.15.152:38004) 2024-04-20T11:35:40.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.846+0000 7fcc7d400640 1 -- 172.21.15.152:0/514468196 learned_addr learned my addr 172.21.15.152:0/514468196 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:40.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.846+0000 7fcc7de00640 1 --2- 172.21.15.152:0/514468196 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcc640039b0 0x7fcc80165500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:40.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.846+0000 7fcc7d400640 1 -- 172.21.15.152:0/514468196 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcc640039b0 msgr2=0x7fcc80165500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:40.848 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.846+0000 7fcc7d400640 1 --2- 172.21.15.152:0/514468196 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcc640039b0 0x7fcc80165500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:40.848 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.846+0000 7fcc7e800640 1 --2- 172.21.15.152:0/514468196 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcc800b09d0 0x7fcc800b0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:40.848 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.846+0000 7fcc7d400640 1 -- 172.21.15.152:0/514468196 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcc800b09d0 msgr2=0x7fcc800b0e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:40.848 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.846+0000 7fcc7d400640 1 --2- 172.21.15.152:0/514468196 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcc800b09d0 0x7fcc800b0e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:40.848 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.846+0000 7fcc7d400640 1 -- 172.21.15.152:0/514468196 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc800b17c0 con 0x7fcc800b0050 2024-04-20T11:35:40.848 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.846+0000 7fcc7d400640 1 --2- 172.21.15.152:0/514468196 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcc800b0050 0x7fcc80165a40 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c00f570 tx=0x7fcc6c00fa30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:40.848 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.847+0000 7fcc76000640 1 -- 172.21.15.152:0/514468196 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc6c018070 con 0x7fcc800b0050 2024-04-20T11:35:40.848 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.847+0000 7fcc76000640 1 -- 172.21.15.152:0/514468196 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc6c014890 con 0x7fcc800b0050 2024-04-20T11:35:40.848 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.847+0000 7fcc7f200640 1 -- 172.21.15.152:0/514468196 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc8014afe0 con 0x7fcc800b0050 2024-04-20T11:35:40.848 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.847+0000 7fcc76000640 1 -- 172.21.15.152:0/514468196 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc6c014c80 con 0x7fcc800b0050 2024-04-20T11:35:40.848 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.847+0000 7fcc7f200640 1 -- 172.21.15.152:0/514468196 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc8014b5a0 con 0x7fcc800b0050 2024-04-20T11:35:40.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.848+0000 7fcc7f200640 1 -- 172.21.15.152:0/514468196 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc80075e60 con 0x7fcc800b0050 2024-04-20T11:35:40.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.849+0000 7fcc76000640 1 -- 172.21.15.152:0/514468196 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 21) ==== 91550+0+0 (secure 0 0 0) 0x7fcc6c01b070 con 0x7fcc800b0050 2024-04-20T11:35:40.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.850+0000 7fcc76000640 1 --2- 172.21.15.152:0/514468196 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fcc6006cec0 0x7fcc6006f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:40.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.850+0000 7fcc76000640 1 -- 172.21.15.152:0/514468196 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5709+0+0 (secure 0 0 0) 0x7fcc6c08c8d0 con 0x7fcc800b0050 2024-04-20T11:35:40.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.851+0000 7fcc7de00640 1 --2- 172.21.15.152:0/514468196 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fcc6006cec0 0x7fcc6006f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:40.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.852+0000 7fcc7de00640 1 --2- 172.21.15.152:0/514468196 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fcc6006cec0 0x7fcc6006f380 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fcc70034000 tx=0x7fcc70007000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:40.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:40.854+0000 7fcc76000640 1 -- 172.21.15.152:0/514468196 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fcc6c05a3d0 con 0x7fcc800b0050 2024-04-20T11:35:41.088 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.089+0000 7fcc7f200640 1 -- 172.21.15.152:0/514468196 --> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "snap_name": "snapshot_1", "target_sub_name": "clone_1", "target": ["mon-mgr", ""]}) -- 0x7fcc800b1aa0 con 0x7fcc6006cec0 2024-04-20T11:35:41.106 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:35:41.104+0000 7f8a5cc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:35:41.106 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:35:41.104+0000 7f8a5cc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:35:41.106 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:35:41.104+0000 7f8a5cc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:35:41.107 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:35:41.104+0000 7f8a5cc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:35:41.107 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:35:41.104+0000 7f8a5cc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:35:41.116 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:35:41.115+0000 7f8a5cc00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-04-20T11:35:41.116 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.116+0000 7fcc76000640 1 -- 172.21.15.152:0/514468196 <== mgr.4109 v2:172.21.15.175:6836/4021250441 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7fcc800b1aa0 con 0x7fcc6006cec0 2024-04-20T11:35:41.117 INFO:teuthology.orchestra.run.smithi152.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-04-20T11:35:41.134 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-20T11:35:41.134 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status non_existent_subvolume clone_1 2024-04-20T11:35:41.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.341+0000 7f26d9800640 1 Processor -- start 2024-04-20T11:35:41.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.342+0000 7f26d9800640 1 -- start start 2024-04-20T11:35:41.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.343+0000 7f26d9800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f26d4071cd0 0x7f26d40720b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:41.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.343+0000 7f26d9800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f26d4141450 0x7f26d41418e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:41.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.343+0000 7f26d9800640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26d406fb40 con 0x7f26d4160220 2024-04-20T11:35:41.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.343+0000 7f26d9800640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f26d413f8e0 con 0x7f26d4071cd0 2024-04-20T11:35:41.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.343+0000 7f26d9800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f26d40725f0 con 0x7f26d4141450 2024-04-20T11:35:41.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.343+0000 7f26d3e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f26d4071cd0 0x7f26d40720b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:41.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.344+0000 7f26d3e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f26d4071cd0 0x7f26d40720b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:38012/0 (socket says 172.21.15.152:38012) 2024-04-20T11:35:41.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.344+0000 7f26d3e00640 1 -- 172.21.15.152:0/2632993562 learned_addr learned my addr 172.21.15.152:0/2632993562 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:41.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.344+0000 7f26d8e00640 1 --2- 172.21.15.152:0/2632993562 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f26d4141450 0x7f26d41418e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:41.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.344+0000 7f26d3e00640 1 -- 172.21.15.152:0/2632993562 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f26d4141450 msgr2=0x7f26d41418e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:41.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.344+0000 7f26d3e00640 1 --2- 172.21.15.152:0/2632993562 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f26d4141450 0x7f26d41418e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:41.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.344+0000 7f26d3e00640 1 -- 172.21.15.152:0/2632993562 >> v1:172.21.15.152:6789/0 conn(0x7f26d4160220 legacy=0x7f26d41606b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:41.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.344+0000 7f26d3e00640 1 -- 172.21.15.152:0/2632993562 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26d4160f90 con 0x7f26d4071cd0 2024-04-20T11:35:41.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.345+0000 7f26d3e00640 1 --2- 172.21.15.152:0/2632993562 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f26d4071cd0 0x7f26d40720b0 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f26c4009870 tx=0x7f26c402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e269c6219300cf7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:41.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.345+0000 7f26d2a00640 1 -- 172.21.15.152:0/2632993562 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c4035020 con 0x7f26d4071cd0 2024-04-20T11:35:41.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.345+0000 7f26d2a00640 1 -- 172.21.15.152:0/2632993562 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26c4031640 con 0x7f26d4071cd0 2024-04-20T11:35:41.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.345+0000 7f26d2a00640 1 -- 172.21.15.152:0/2632993562 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c402d040 con 0x7f26d4071cd0 2024-04-20T11:35:41.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.346+0000 7f26d9800640 1 -- 172.21.15.152:0/2632993562 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f26d4071cd0 msgr2=0x7f26d40720b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:41.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.346+0000 7f26d9800640 1 --2- 172.21.15.152:0/2632993562 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f26d4071cd0 0x7f26d40720b0 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f26c4009870 tx=0x7f26c402b8f0 comp rx=0 tx=0).stop 2024-04-20T11:35:41.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.346+0000 7f26d9800640 1 -- 172.21.15.152:0/2632993562 shutdown_connections 2024-04-20T11:35:41.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.346+0000 7f26d9800640 1 --2- 172.21.15.152:0/2632993562 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f26d4141450 0x7f26d41418e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:41.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.346+0000 7f26d9800640 1 --2- 172.21.15.152:0/2632993562 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f26d4071cd0 0x7f26d40720b0 unknown :-1 s=CLOSED pgs=1826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:41.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.346+0000 7f26d9800640 1 -- 172.21.15.152:0/2632993562 >> 172.21.15.152:0/2632993562 conn(0x7f26d4067ae0 msgr2=0x7f26d406b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:41.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.346+0000 7f26d9800640 1 -- 172.21.15.152:0/2632993562 shutdown_connections 2024-04-20T11:35:41.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.346+0000 7f26d9800640 1 -- 172.21.15.152:0/2632993562 wait complete. 2024-04-20T11:35:41.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.348+0000 7f26d9800640 1 Processor -- start 2024-04-20T11:35:41.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.348+0000 7f26d9800640 1 -- start start 2024-04-20T11:35:41.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.349+0000 7f26d9800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f26d4071cd0 0x7f26d41491b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:41.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.349+0000 7f26d9800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f26d4141450 0x7f26d4157b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:41.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.349+0000 7f26d9800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26d4160220 0x7f26d4159050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:41.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.349+0000 7f26d9800640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f26d406e350 con 0x7f26d4141450 2024-04-20T11:35:41.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.349+0000 7f26d9800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f26d406e070 con 0x7f26d4160220 2024-04-20T11:35:41.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.349+0000 7f26d9800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f26d406e630 con 0x7f26d4071cd0 2024-04-20T11:35:41.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.349+0000 7f26d3400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f26d4141450 0x7f26d4157b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:41.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.349+0000 7f26d3400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f26d4141450 0x7f26d4157b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:38024/0 (socket says 172.21.15.152:38024) 2024-04-20T11:35:41.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.349+0000 7f26d3400640 1 -- 172.21.15.152:0/1908316684 learned_addr learned my addr 172.21.15.152:0/1908316684 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:41.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.350+0000 7f26d3e00640 1 --2- 172.21.15.152:0/1908316684 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f26d4071cd0 0x7f26d41491b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:41.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.350+0000 7f26d8e00640 1 --2- 172.21.15.152:0/1908316684 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26d4160220 0x7f26d4159050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:41.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.350+0000 7f26d3400640 1 -- 172.21.15.152:0/1908316684 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f26d4071cd0 msgr2=0x7f26d41491b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:41.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.350+0000 7f26d3400640 1 --2- 172.21.15.152:0/1908316684 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f26d4071cd0 0x7f26d41491b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:41.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.350+0000 7f26d3400640 1 -- 172.21.15.152:0/1908316684 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26d4160220 msgr2=0x7f26d4159050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:41.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.350+0000 7f26d3400640 1 --2- 172.21.15.152:0/1908316684 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26d4160220 0x7f26d4159050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:41.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.350+0000 7f26d3400640 1 -- 172.21.15.152:0/1908316684 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26d41496f0 con 0x7f26d4141450 2024-04-20T11:35:41.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.350+0000 7f26d3e00640 1 --2- 172.21.15.152:0/1908316684 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f26d4071cd0 0x7f26d41491b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:35:41.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.350+0000 7f26d3400640 1 --2- 172.21.15.152:0/1908316684 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f26d4141450 0x7f26d4157b60 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f26bc00c3c0 tx=0x7f26bc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:41.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.350+0000 7f26cbe00640 1 -- 172.21.15.152:0/1908316684 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26bc0052d0 con 0x7f26d4141450 2024-04-20T11:35:41.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.350+0000 7f26cbe00640 1 -- 172.21.15.152:0/1908316684 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26bc005490 con 0x7f26d4141450 2024-04-20T11:35:41.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.350+0000 7f26cbe00640 1 -- 172.21.15.152:0/1908316684 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26bc007260 con 0x7f26d4141450 2024-04-20T11:35:41.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.350+0000 7f26d9800640 1 -- 172.21.15.152:0/1908316684 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26d41585e0 con 0x7f26d4141450 2024-04-20T11:35:41.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.351+0000 7f26d9800640 1 -- 172.21.15.152:0/1908316684 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26d41e6280 con 0x7f26d4141450 2024-04-20T11:35:41.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.352+0000 7f26d9800640 1 -- 172.21.15.152:0/1908316684 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26d4075e60 con 0x7f26d4141450 2024-04-20T11:35:41.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.353+0000 7f26cbe00640 1 -- 172.21.15.152:0/1908316684 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 21) ==== 91550+0+0 (secure 0 0 0) 0x7f26bc007400 con 0x7f26d4141450 2024-04-20T11:35:41.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.354+0000 7f26cbe00640 1 --2- 172.21.15.152:0/1908316684 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f26b006ce90 0x7f26b006f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:41.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.354+0000 7f26cbe00640 1 -- 172.21.15.152:0/1908316684 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5709+0+0 (secure 0 0 0) 0x7f26bc08ba60 con 0x7f26d4141450 2024-04-20T11:35:41.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.354+0000 7f26d3e00640 1 --2- 172.21.15.152:0/1908316684 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f26b006ce90 0x7f26b006f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:41.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.356+0000 7f26d3e00640 1 --2- 172.21.15.152:0/1908316684 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f26b006ce90 0x7f26b006f350 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f26c4036000 tx=0x7f26c4008030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:41.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.358+0000 7f26cbe00640 1 -- 172.21.15.152:0/1908316684 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f26bc059670 con 0x7f26d4141450 2024-04-20T11:35:41.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.592+0000 7f26d9800640 1 -- 172.21.15.152:0/1908316684 --> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "non_existent_subvolume", "clone_name": "clone_1", "target": ["mon-mgr", ""]}) -- 0x7f26d41588c0 con 0x7f26b006ce90 2024-04-20T11:35:41.595 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:35:41.595+0000 7f8a5cc00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-04-20T11:35:41.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.596+0000 7f26cbe00640 1 -- 172.21.15.152:0/1908316684 <== mgr.4109 v2:172.21.15.175:6836/4021250441 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f26d41588c0 con 0x7f26b006ce90 2024-04-20T11:35:41.595 INFO:teuthology.orchestra.run.smithi152.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-04-20T11:35:41.611 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-20T11:35:41.611 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create non_existent_subvolume group_1 2024-04-20T11:35:41.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.815+0000 7f47a7e00640 1 Processor -- start 2024-04-20T11:35:41.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.816+0000 7f47a7e00640 1 -- start start 2024-04-20T11:35:41.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.817+0000 7f47a7e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f47a815e880 0x7f47a80b5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:41.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.817+0000 7f47a7e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f47a80d5190 0x7f47a80b6530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:41.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.817+0000 7f47a7e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f47a80d4c80 0x7f47a8162290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:41.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.817+0000 7f47a7e00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f47a80d32d0 con 0x7f47a815e880 2024-04-20T11:35:41.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.817+0000 7f47a7e00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f47a80d29a0 con 0x7f47a80d4c80 2024-04-20T11:35:41.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.817+0000 7f47a7e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f47a80d2c80 con 0x7f47a80d5190 2024-04-20T11:35:41.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.817+0000 7f47a6a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f47a815e880 0x7f47a80b5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:41.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.818+0000 7f47a6a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f47a815e880 0x7f47a80b5ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:38040/0 (socket says 172.21.15.152:38040) 2024-04-20T11:35:41.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.818+0000 7f47a6a00640 1 -- 172.21.15.152:0/1094893265 learned_addr learned my addr 172.21.15.152:0/1094893265 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:41.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.818+0000 7f47a6000640 1 --2- 172.21.15.152:0/1094893265 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f47a80d5190 0x7f47a80b6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:41.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.818+0000 7f47a7400640 1 --2- 172.21.15.152:0/1094893265 >> v2:172.21.15.175:3300/0 conn(0x7f47a80d4c80 0x7f47a8162290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:41.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.818+0000 7f47a6a00640 1 -- 172.21.15.152:0/1094893265 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f47a80d5190 msgr2=0x7f47a80b6530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:41.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.818+0000 7f47a6a00640 1 --2- 172.21.15.152:0/1094893265 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f47a80d5190 0x7f47a80b6530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:41.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.818+0000 7f47a6a00640 1 -- 172.21.15.152:0/1094893265 >> v2:172.21.15.175:3300/0 conn(0x7f47a80d4c80 msgr2=0x7f47a8162290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:41.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.818+0000 7f47a6a00640 1 --2- 172.21.15.152:0/1094893265 >> v2:172.21.15.175:3300/0 conn(0x7f47a80d4c80 0x7f47a8162290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:41.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.818+0000 7f47a6a00640 1 -- 172.21.15.152:0/1094893265 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47a80d2f60 con 0x7f47a815e880 2024-04-20T11:35:41.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.818+0000 7f47a7400640 1 --2- 172.21.15.152:0/1094893265 >> v2:172.21.15.175:3300/0 conn(0x7f47a80d4c80 0x7f47a8162290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:35:41.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.819+0000 7f47a6a00640 1 --2- 172.21.15.152:0/1094893265 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f47a815e880 0x7f47a80b5ff0 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f47a81621f0 tx=0x7f479802b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2bbc535003488f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:41.826 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.820+0000 7f47a5600640 1 -- 172.21.15.152:0/1094893265 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4798038070 con 0x7f47a815e880 2024-04-20T11:35:41.826 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.820+0000 7f47a5600640 1 -- 172.21.15.152:0/1094893265 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4798031740 con 0x7f47a815e880 2024-04-20T11:35:41.826 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.820+0000 7f47a5600640 1 -- 172.21.15.152:0/1094893265 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f479802f050 con 0x7f47a815e880 2024-04-20T11:35:41.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.820+0000 7f47a7e00640 1 -- 172.21.15.152:0/1094893265 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f47a815e880 msgr2=0x7f47a80b5ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:41.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.820+0000 7f47a7e00640 1 --2- 172.21.15.152:0/1094893265 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f47a815e880 0x7f47a80b5ff0 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f47a81621f0 tx=0x7f479802b7e0 comp rx=0 tx=0).stop 2024-04-20T11:35:41.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.821+0000 7f47a7e00640 1 -- 172.21.15.152:0/1094893265 shutdown_connections 2024-04-20T11:35:41.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.821+0000 7f47a7e00640 1 --2- 172.21.15.152:0/1094893265 >> v2:172.21.15.175:3300/0 conn(0x7f47a80d4c80 0x7f47a8162290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:41.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.821+0000 7f47a7e00640 1 --2- 172.21.15.152:0/1094893265 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f47a80d5190 0x7f47a80b6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:41.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.821+0000 7f47a7e00640 1 --2- 172.21.15.152:0/1094893265 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f47a815e880 0x7f47a80b5ff0 unknown :-1 s=CLOSED pgs=1828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:41.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.821+0000 7f47a7e00640 1 -- 172.21.15.152:0/1094893265 >> 172.21.15.152:0/1094893265 conn(0x7f47a808bfd0 msgr2=0x7f47a80d4570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:41.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.821+0000 7f47a7e00640 1 -- 172.21.15.152:0/1094893265 shutdown_connections 2024-04-20T11:35:41.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.821+0000 7f47a7e00640 1 -- 172.21.15.152:0/1094893265 wait complete. 2024-04-20T11:35:41.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.822+0000 7f47a7e00640 1 Processor -- start 2024-04-20T11:35:41.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.822+0000 7f47a7e00640 1 -- start start 2024-04-20T11:35:41.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.823+0000 7f47a7e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f47a80d4c80 0x7f47a815fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:41.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.823+0000 7f47a7e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f47a80d5190 0x7f47a80c7f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:41.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.823+0000 7f47a7e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f47a815e880 0x7f47a81f7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:41.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.823+0000 7f47a7e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f47a80cf1f0 con 0x7f47a80d5190 2024-04-20T11:35:41.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.823+0000 7f47a7e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f47a80cf070 con 0x7f47a80d4c80 2024-04-20T11:35:41.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.823+0000 7f47a7e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f47a80d3760 con 0x7f47a815e880 2024-04-20T11:35:41.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.824+0000 7f47a6000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f47a80d5190 0x7f47a80c7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:41.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.824+0000 7f47a6000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f47a80d5190 0x7f47a80c7f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:38054/0 (socket says 172.21.15.152:38054) 2024-04-20T11:35:41.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.824+0000 7f47a6000640 1 -- 172.21.15.152:0/4156276503 learned_addr learned my addr 172.21.15.152:0/4156276503 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:41.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.824+0000 7f47a6a00640 1 --2- 172.21.15.152:0/4156276503 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f47a80d4c80 0x7f47a815fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:41.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.824+0000 7f47a6000640 1 -- 172.21.15.152:0/4156276503 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f47a815e880 msgr2=0x7f47a81f7910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:41.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.824+0000 7f47a7400640 1 --2- 172.21.15.152:0/4156276503 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f47a815e880 0x7f47a81f7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:41.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.824+0000 7f47a6000640 1 --2- 172.21.15.152:0/4156276503 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f47a815e880 0x7f47a81f7910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:41.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.824+0000 7f47a6000640 1 -- 172.21.15.152:0/4156276503 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f47a80d4c80 msgr2=0x7f47a815fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:41.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.824+0000 7f47a6000640 1 --2- 172.21.15.152:0/4156276503 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f47a80d4c80 0x7f47a815fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:41.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.824+0000 7f47a6000640 1 -- 172.21.15.152:0/4156276503 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47a80c8460 con 0x7f47a80d5190 2024-04-20T11:35:41.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.824+0000 7f47a6a00640 1 --2- 172.21.15.152:0/4156276503 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f47a80d4c80 0x7f47a815fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:35:41.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.824+0000 7f47a7400640 1 --2- 172.21.15.152:0/4156276503 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f47a815e880 0x7f47a81f7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:35:41.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.825+0000 7f47a6000640 1 --2- 172.21.15.152:0/4156276503 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f47a80d5190 0x7f47a80c7f20 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f4790009fc0 tx=0x7f479000e570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:41.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.825+0000 7f479ea00640 1 -- 172.21.15.152:0/4156276503 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4790014620 con 0x7f47a80d5190 2024-04-20T11:35:41.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.825+0000 7f479ea00640 1 -- 172.21.15.152:0/4156276503 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4790019070 con 0x7f47a80d5190 2024-04-20T11:35:41.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.825+0000 7f47a7e00640 1 -- 172.21.15.152:0/4156276503 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47a80c8700 con 0x7f47a80d5190 2024-04-20T11:35:41.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.825+0000 7f479ea00640 1 -- 172.21.15.152:0/4156276503 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4790014a00 con 0x7f47a80d5190 2024-04-20T11:35:41.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.825+0000 7f47a7e00640 1 -- 172.21.15.152:0/4156276503 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47a80c8cc0 con 0x7f47a80d5190 2024-04-20T11:35:41.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.826+0000 7f47a7e00640 1 -- 172.21.15.152:0/4156276503 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47a8075e60 con 0x7f47a80d5190 2024-04-20T11:35:41.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.827+0000 7f479ea00640 1 -- 172.21.15.152:0/4156276503 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 21) ==== 91550+0+0 (secure 0 0 0) 0x7f4790014ba0 con 0x7f47a80d5190 2024-04-20T11:35:41.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.828+0000 7f479ea00640 1 --2- 172.21.15.152:0/4156276503 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f478406ce90 0x7f478406f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:41.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.828+0000 7f479ea00640 1 -- 172.21.15.152:0/4156276503 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5709+0+0 (secure 0 0 0) 0x7f479008cec0 con 0x7f47a80d5190 2024-04-20T11:35:41.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.829+0000 7f47a6a00640 1 --2- 172.21.15.152:0/4156276503 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f478406ce90 0x7f478406f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:41.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.830+0000 7f47a6a00640 1 --2- 172.21.15.152:0/4156276503 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f478406ce90 0x7f478406f350 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4798035000 tx=0x7f4798008030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:41.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:41.832+0000 7f479ea00640 1 -- 172.21.15.152:0/4156276503 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f479005ab10 con 0x7f47a80d5190 2024-04-20T11:35:42.063 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.062+0000 7f47a7e00640 1 -- 172.21.15.152:0/4156276503 --> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "non_existent_subvolume", "group_name": "group_1", "target": ["mon-mgr", ""]}) -- 0x7f47a80c9440 con 0x7f478406ce90 2024-04-20T11:35:42.065 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:35:42.064+0000 7f8a5cc00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-04-20T11:35:42.066 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.065+0000 7f479ea00640 1 -- 172.21.15.152:0/4156276503 <== mgr.4109 v2:172.21.15.175:6836/4021250441 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f47a80c9440 con 0x7f478406ce90 2024-04-20T11:35:42.066 INFO:teuthology.orchestra.run.smithi152.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-04-20T11:35:42.080 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-20T11:35:42.081 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup rm non_existent_subvolume group_1 2024-04-20T11:35:42.277 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.277+0000 7fee69a00640 1 Processor -- start 2024-04-20T11:35:42.277 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.277+0000 7fee69a00640 1 -- start start 2024-04-20T11:35:42.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.279+0000 7fee69a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee640cd8c0 0x7fee640b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:42.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.279+0000 7fee69a00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee641463e0 con 0x7fee640b2880 2024-04-20T11:35:42.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.279+0000 7fee69a00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee640b3e20 con 0x7fee64146000 2024-04-20T11:35:42.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.279+0000 7fee69a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fee640b4750 con 0x7fee640cd8c0 2024-04-20T11:35:42.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.279+0000 7fee5a000640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fee640b2880 0x7fee640cd1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:40554/0 (socket says 172.21.15.152:40554) 2024-04-20T11:35:42.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.279+0000 7fee5a000640 1 -- 172.21.15.152:0/324876802 learned_addr learned my addr 172.21.15.152:0/324876802 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:35:42.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.280+0000 7fee63400640 1 --2- 172.21.15.152:0/324876802 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee640cd8c0 0x7fee640b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:42.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.280+0000 7fee62000640 1 -- 172.21.15.152:0/324876802 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3187679554 0 0) 0x7fee641463e0 con 0x7fee640b2880 2024-04-20T11:35:42.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.280+0000 7fee62000640 1 -- 172.21.15.152:0/324876802 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fee48003520 con 0x7fee640b2880 2024-04-20T11:35:42.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.280+0000 7fee62000640 1 -- 172.21.15.152:0/324876802 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 424203755 0 0) 0x7fee48003520 con 0x7fee640b2880 2024-04-20T11:35:42.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.280+0000 7fee62000640 1 -- 172.21.15.152:0/324876802 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee640cd8c0 msgr2=0x7fee640b4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:42.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.280+0000 7fee62000640 1 --2- 172.21.15.152:0/324876802 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee640cd8c0 0x7fee640b4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:42.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.280+0000 7fee62000640 1 -- 172.21.15.152:0/324876802 >> v1:172.21.15.175:6789/0 conn(0x7fee64146000 legacy=0x7fee640b2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:42.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.280+0000 7fee62000640 1 -- 172.21.15.152:0/324876802 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee640b4100 con 0x7fee640b2880 2024-04-20T11:35:42.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.280+0000 7fee62000640 1 -- 172.21.15.152:0/324876802 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7fee4c001690 con 0x7fee640b2880 2024-04-20T11:35:42.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.280+0000 7fee62000640 1 -- 172.21.15.152:0/324876802 >> v1:172.21.15.152:6789/0 conn(0x7fee640b2880 legacy=0x7fee640cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:42.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.280+0000 7fee62000640 1 --2- 172.21.15.152:0/324876802 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee640cd8c0 0x7fee48003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:42.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.280+0000 7fee62000640 1 --2- 172.21.15.152:0/324876802 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fee48003fd0 0x7fee48004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:42.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.280+0000 7fee62000640 1 --2- 172.21.15.152:0/324876802 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee48004d40 0x7fee480051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:42.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.280+0000 7fee62000640 1 -- 172.21.15.152:0/324876802 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fee640b4750 con 0x7fee48003fd0 2024-04-20T11:35:42.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.280+0000 7fee62000640 1 -- 172.21.15.152:0/324876802 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fee4800bbb0 con 0x7fee48004d40 2024-04-20T11:35:42.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.280+0000 7fee62000640 1 -- 172.21.15.152:0/324876802 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fee4800bdb0 con 0x7fee640cd8c0 2024-04-20T11:35:42.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.281+0000 7fee5a000640 1 --2- 172.21.15.152:0/324876802 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fee48003fd0 0x7fee48004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:42.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.281+0000 7fee62a00640 1 --2- 172.21.15.152:0/324876802 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee640cd8c0 0x7fee48003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:42.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.281+0000 7fee63400640 1 --2- 172.21.15.152:0/324876802 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee48004d40 0x7fee480051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:42.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.281+0000 7fee5a000640 1 -- 172.21.15.152:0/324876802 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee640cd8c0 msgr2=0x7fee48003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:42.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.281+0000 7fee5a000640 1 --2- 172.21.15.152:0/324876802 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee640cd8c0 0x7fee48003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:42.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.281+0000 7fee5a000640 1 -- 172.21.15.152:0/324876802 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee48004d40 msgr2=0x7fee480051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:42.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.281+0000 7fee5a000640 1 --2- 172.21.15.152:0/324876802 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee48004d40 0x7fee480051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:42.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.281+0000 7fee5a000640 1 -- 172.21.15.152:0/324876802 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee48003520 con 0x7fee48003fd0 2024-04-20T11:35:42.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.282+0000 7fee5a000640 1 --2- 172.21.15.152:0/324876802 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fee48003fd0 0x7fee48004800 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7fee64070160 tx=0x7fee48005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58cb7148c5c87bc8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:42.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.282+0000 7fee62000640 1 -- 172.21.15.152:0/324876802 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee4c007c40 con 0x7fee48003fd0 2024-04-20T11:35:42.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.283+0000 7fee62000640 1 -- 172.21.15.152:0/324876802 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee4c030070 con 0x7fee48003fd0 2024-04-20T11:35:42.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.283+0000 7fee62000640 1 -- 172.21.15.152:0/324876802 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee4c005d50 con 0x7fee48003fd0 2024-04-20T11:35:42.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.283+0000 7fee69a00640 1 -- 172.21.15.152:0/324876802 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fee48003fd0 msgr2=0x7fee48004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:42.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.283+0000 7fee69a00640 1 --2- 172.21.15.152:0/324876802 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fee48003fd0 0x7fee48004800 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7fee64070160 tx=0x7fee48005830 comp rx=0 tx=0).stop 2024-04-20T11:35:42.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.283+0000 7fee63400640 1 -- 172.21.15.152:0/324876802 reap_dead start 2024-04-20T11:35:42.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.283+0000 7fee69a00640 1 -- 172.21.15.152:0/324876802 shutdown_connections 2024-04-20T11:35:42.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.283+0000 7fee69a00640 1 -- 172.21.15.152:0/324876802 >> 172.21.15.152:0/324876802 conn(0x7fee6406b980 msgr2=0x7fee6406bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:42.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.283+0000 7fee69a00640 1 -- 172.21.15.152:0/324876802 shutdown_connections 2024-04-20T11:35:42.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.283+0000 7fee69a00640 1 -- 172.21.15.152:0/324876802 wait complete. 2024-04-20T11:35:42.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.285+0000 7fee69a00640 1 Processor -- start 2024-04-20T11:35:42.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.285+0000 7fee69a00640 1 -- start start 2024-04-20T11:35:42.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.286+0000 7fee69a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee48003fd0 0x7fee6415bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:42.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.286+0000 7fee69a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fee6413fce0 0x7fee64140160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:42.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.286+0000 7fee69a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee640ad920 0x7fee640adda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:42.291 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.286+0000 7fee69a00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fee640b4d60 con 0x7fee6413fce0 2024-04-20T11:35:42.291 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.286+0000 7fee69a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fee640b4be0 con 0x7fee48003fd0 2024-04-20T11:35:42.291 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.286+0000 7fee69a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fee640b4ee0 con 0x7fee640ad920 2024-04-20T11:35:42.291 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.286+0000 7fee5a000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fee6413fce0 0x7fee64140160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:42.291 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.286+0000 7fee5a000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fee6413fce0 0x7fee64140160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:38066/0 (socket says 172.21.15.152:38066) 2024-04-20T11:35:42.291 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.286+0000 7fee5a000640 1 -- 172.21.15.152:0/2395024270 learned_addr learned my addr 172.21.15.152:0/2395024270 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:42.291 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.287+0000 7fee63400640 1 --2- 172.21.15.152:0/2395024270 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee640ad920 0x7fee640adda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:42.291 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.287+0000 7fee5a000640 1 -- 172.21.15.152:0/2395024270 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee640ad920 msgr2=0x7fee640adda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:42.291 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.287+0000 7fee5a000640 1 --2- 172.21.15.152:0/2395024270 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee640ad920 0x7fee640adda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:42.291 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.287+0000 7fee5a000640 1 -- 172.21.15.152:0/2395024270 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee48003fd0 msgr2=0x7fee6415bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:42.291 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.287+0000 7fee62a00640 1 --2- 172.21.15.152:0/2395024270 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee48003fd0 0x7fee6415bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:42.291 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.287+0000 7fee5a000640 1 --2- 172.21.15.152:0/2395024270 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee48003fd0 0x7fee6415bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:42.292 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.287+0000 7fee5a000640 1 -- 172.21.15.152:0/2395024270 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee640ae690 con 0x7fee6413fce0 2024-04-20T11:35:42.292 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.287+0000 7fee5a000640 1 --2- 172.21.15.152:0/2395024270 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fee6413fce0 0x7fee64140160 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7fee4c02c3e0 tx=0x7fee640b3ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:42.292 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.288+0000 7fee5b400640 1 -- 172.21.15.152:0/2395024270 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee4c001b80 con 0x7fee6413fce0 2024-04-20T11:35:42.292 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.288+0000 7fee5b400640 1 -- 172.21.15.152:0/2395024270 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee4c007270 con 0x7fee6413fce0 2024-04-20T11:35:42.292 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.288+0000 7fee5b400640 1 -- 172.21.15.152:0/2395024270 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee4c007640 con 0x7fee6413fce0 2024-04-20T11:35:42.292 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.288+0000 7fee69a00640 1 -- 172.21.15.152:0/2395024270 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee640ae970 con 0x7fee6413fce0 2024-04-20T11:35:42.292 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.288+0000 7fee69a00640 1 -- 172.21.15.152:0/2395024270 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee640aef30 con 0x7fee6413fce0 2024-04-20T11:35:42.292 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.289+0000 7fee69a00640 1 -- 172.21.15.152:0/2395024270 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee640689f0 con 0x7fee6413fce0 2024-04-20T11:35:42.292 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.290+0000 7fee5b400640 1 -- 172.21.15.152:0/2395024270 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 22) ==== 91601+0+0 (secure 0 0 0) 0x7fee4c02c770 con 0x7fee6413fce0 2024-04-20T11:35:42.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.291+0000 7fee5b400640 1 --2- 172.21.15.152:0/2395024270 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fee4006cf50 0x7fee4006f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:42.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.291+0000 7fee5b400640 1 -- 172.21.15.152:0/2395024270 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5709+0+0 (secure 0 0 0) 0x7fee4c0ad8f0 con 0x7fee6413fce0 2024-04-20T11:35:42.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.291+0000 7fee62a00640 1 --2- 172.21.15.152:0/2395024270 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fee4006cf50 0x7fee4006f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:42.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.293+0000 7fee62a00640 1 --2- 172.21.15.152:0/2395024270 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fee4006cf50 0x7fee4006f410 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fee54001550 tx=0x7fee54009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:42.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.295+0000 7fee5b400640 1 -- 172.21.15.152:0/2395024270 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fee4c07b430 con 0x7fee6413fce0 2024-04-20T11:35:42.503 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.503+0000 7fee69a00640 1 -- 172.21.15.152:0/2395024270 --> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup rm", "vol_name": "non_existent_subvolume", "group_name": "group_1", "target": ["mon-mgr", ""]}) -- 0x7fee640afe10 con 0x7fee4006cf50 2024-04-20T11:35:42.506 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:35:42.506+0000 7f8a5cc00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-04-20T11:35:42.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.507+0000 7fee5b400640 1 -- 172.21.15.152:0/2395024270 <== mgr.4109 v2:172.21.15.175:6836/4021250441 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7fee640afe10 con 0x7fee4006cf50 2024-04-20T11:35:42.507 INFO:teuthology.orchestra.run.smithi152.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-04-20T11:35:42.519 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-20T11:35:42.519 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup getpath non_existent_subvolume group_1 2024-04-20T11:35:42.726 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.725+0000 7fd88e000640 1 Processor -- start 2024-04-20T11:35:42.726 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.726+0000 7fd88e000640 1 -- start start 2024-04-20T11:35:42.727 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.727+0000 7fd88e000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fd88815eb50 0x7fd8880d0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:42.727 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.727+0000 7fd88e000640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fd8880d35a0 0x7fd8880cf470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:42.727 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.727+0000 7fd88e000640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8880d19c0 con 0x7fd888163230 2024-04-20T11:35:42.727 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.727+0000 7fd88e000640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fd888162750 con 0x7fd88815eb50 2024-04-20T11:35:42.728 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.727+0000 7fd88e000640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fd888162a30 con 0x7fd8880d35a0 2024-04-20T11:35:42.728 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.727+0000 7fd88cc00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fd88815eb50 0x7fd8880d0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:42.728 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.727+0000 7fd88cc00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fd88815eb50 0x7fd8880d0f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:38072/0 (socket says 172.21.15.152:38072) 2024-04-20T11:35:42.728 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.727+0000 7fd88cc00640 1 -- 172.21.15.152:0/392804926 learned_addr learned my addr 172.21.15.152:0/392804926 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:42.728 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.728+0000 7fd88d600640 1 --2- 172.21.15.152:0/392804926 >> v2:172.21.15.175:3300/0 conn(0x7fd8880d35a0 0x7fd8880cf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:42.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.728+0000 7fd88cc00640 1 -- 172.21.15.152:0/392804926 >> v2:172.21.15.175:3300/0 conn(0x7fd8880d35a0 msgr2=0x7fd8880cf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:42.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.728+0000 7fd88cc00640 1 --2- 172.21.15.152:0/392804926 >> v2:172.21.15.175:3300/0 conn(0x7fd8880d35a0 0x7fd8880cf470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:42.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.728+0000 7fd88cc00640 1 -- 172.21.15.152:0/392804926 >> v1:172.21.15.152:6789/0 conn(0x7fd888163230 legacy=0x7fd8880d2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:42.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.728+0000 7fd88cc00640 1 -- 172.21.15.152:0/392804926 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd888162d10 con 0x7fd88815eb50 2024-04-20T11:35:42.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.728+0000 7fd88d600640 1 --2- 172.21.15.152:0/392804926 >> v2:172.21.15.175:3300/0 conn(0x7fd8880d35a0 0x7fd8880cf470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:35:42.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.729+0000 7fd88cc00640 1 --2- 172.21.15.152:0/392804926 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd88815eb50 0x7fd8880d0f00 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7fd8880cefb0 tx=0x7fd87802b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98f88b9b674b55e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:42.731 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.729+0000 7fd887400640 1 -- 172.21.15.152:0/392804926 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd878035020 con 0x7fd88815eb50 2024-04-20T11:35:42.731 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.729+0000 7fd887400640 1 -- 172.21.15.152:0/392804926 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd878031890 con 0x7fd88815eb50 2024-04-20T11:35:42.731 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.729+0000 7fd887400640 1 -- 172.21.15.152:0/392804926 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd878031c60 con 0x7fd88815eb50 2024-04-20T11:35:42.731 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.730+0000 7fd88e000640 1 -- 172.21.15.152:0/392804926 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd88815eb50 msgr2=0x7fd8880d0f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:42.731 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.730+0000 7fd88e000640 1 --2- 172.21.15.152:0/392804926 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd88815eb50 0x7fd8880d0f00 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7fd8880cefb0 tx=0x7fd87802b930 comp rx=0 tx=0).stop 2024-04-20T11:35:42.731 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.730+0000 7fd88e000640 1 -- 172.21.15.152:0/392804926 shutdown_connections 2024-04-20T11:35:42.731 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.730+0000 7fd88e000640 1 --2- 172.21.15.152:0/392804926 >> v2:172.21.15.175:3300/0 conn(0x7fd8880d35a0 0x7fd8880cf470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:42.731 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.730+0000 7fd88e000640 1 --2- 172.21.15.152:0/392804926 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd88815eb50 0x7fd8880d0f00 unknown :-1 s=CLOSED pgs=1834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:42.732 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.730+0000 7fd88e000640 1 -- 172.21.15.152:0/392804926 >> 172.21.15.152:0/392804926 conn(0x7fd88808bfd0 msgr2=0x7fd8880d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:42.732 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.730+0000 7fd88e000640 1 -- 172.21.15.152:0/392804926 shutdown_connections 2024-04-20T11:35:42.732 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.730+0000 7fd88e000640 1 -- 172.21.15.152:0/392804926 wait complete. 2024-04-20T11:35:42.732 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.732+0000 7fd88e000640 1 Processor -- start 2024-04-20T11:35:42.732 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.732+0000 7fd88e000640 1 -- start start 2024-04-20T11:35:42.733 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.733+0000 7fd88e000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd8880d35a0 0x7fd88814da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:42.733 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.733+0000 7fd88e000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd88815eb50 0x7fd8880ccc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:42.735 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.733+0000 7fd88e000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd888163230 0x7fd8880cd180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:42.735 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.733+0000 7fd88e000640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fd8880d15c0 con 0x7fd888163230 2024-04-20T11:35:42.735 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.733+0000 7fd88e000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd8880d1440 con 0x7fd88815eb50 2024-04-20T11:35:42.735 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.733+0000 7fd88e000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd8880d1740 con 0x7fd8880d35a0 2024-04-20T11:35:42.735 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.733+0000 7fd88d600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd888163230 0x7fd8880cd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:42.735 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.733+0000 7fd88d600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd888163230 0x7fd8880cd180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:38086/0 (socket says 172.21.15.152:38086) 2024-04-20T11:35:42.735 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.733+0000 7fd88d600640 1 -- 172.21.15.152:0/4063038156 learned_addr learned my addr 172.21.15.152:0/4063038156 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:42.735 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.734+0000 7fd88d600640 1 -- 172.21.15.152:0/4063038156 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd8880d35a0 msgr2=0x7fd88814da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:42.735 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.734+0000 7fd88cc00640 1 --2- 172.21.15.152:0/4063038156 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd8880d35a0 0x7fd88814da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:42.735 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.734+0000 7fd887e00640 1 --2- 172.21.15.152:0/4063038156 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd88815eb50 0x7fd8880ccc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:42.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.734+0000 7fd88d600640 1 --2- 172.21.15.152:0/4063038156 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd8880d35a0 0x7fd88814da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:42.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.734+0000 7fd88d600640 1 -- 172.21.15.152:0/4063038156 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd88815eb50 msgr2=0x7fd8880ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:42.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.734+0000 7fd88d600640 1 --2- 172.21.15.152:0/4063038156 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd88815eb50 0x7fd8880ccc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:42.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.734+0000 7fd88d600640 1 -- 172.21.15.152:0/4063038156 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8880b59e0 con 0x7fd888163230 2024-04-20T11:35:42.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.734+0000 7fd88cc00640 1 --2- 172.21.15.152:0/4063038156 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd8880d35a0 0x7fd88814da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:35:42.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.734+0000 7fd887e00640 1 --2- 172.21.15.152:0/4063038156 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd88815eb50 0x7fd8880ccc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:35:42.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.734+0000 7fd88d600640 1 --2- 172.21.15.152:0/4063038156 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd888163230 0x7fd8880cd180 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7fd88000ceb0 tx=0x7fd88000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:42.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.734+0000 7fd884c00640 1 -- 172.21.15.152:0/4063038156 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd88001a070 con 0x7fd888163230 2024-04-20T11:35:42.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.734+0000 7fd884c00640 1 -- 172.21.15.152:0/4063038156 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd88000d840 con 0x7fd888163230 2024-04-20T11:35:42.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.734+0000 7fd88e000640 1 -- 172.21.15.152:0/4063038156 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8880b5cc0 con 0x7fd888163230 2024-04-20T11:35:42.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.734+0000 7fd884c00640 1 -- 172.21.15.152:0/4063038156 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd88000dc30 con 0x7fd888163230 2024-04-20T11:35:42.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.735+0000 7fd88e000640 1 -- 172.21.15.152:0/4063038156 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd88815f820 con 0x7fd888163230 2024-04-20T11:35:42.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.736+0000 7fd88e000640 1 -- 172.21.15.152:0/4063038156 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd88815fa20 con 0x7fd888163230 2024-04-20T11:35:42.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.737+0000 7fd884c00640 1 -- 172.21.15.152:0/4063038156 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 22) ==== 91601+0+0 (secure 0 0 0) 0x7fd88001d070 con 0x7fd888163230 2024-04-20T11:35:42.738 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.738+0000 7fd884c00640 1 --2- 172.21.15.152:0/4063038156 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd86806cf20 0x7fd86806f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:42.738 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.738+0000 7fd884c00640 1 -- 172.21.15.152:0/4063038156 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5709+0+0 (secure 0 0 0) 0x7fd88008cfb0 con 0x7fd888163230 2024-04-20T11:35:42.738 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.739+0000 7fd88cc00640 1 --2- 172.21.15.152:0/4063038156 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd86806cf20 0x7fd86806f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:42.739 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.740+0000 7fd88cc00640 1 --2- 172.21.15.152:0/4063038156 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd86806cf20 0x7fd86806f3e0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fd878036000 tx=0x7fd878008030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:42.742 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.743+0000 7fd884c00640 1 -- 172.21.15.152:0/4063038156 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fd88005aa10 con 0x7fd888163230 2024-04-20T11:35:42.975 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.974+0000 7fd88e000640 1 -- 172.21.15.152:0/4063038156 --> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup getpath", "vol_name": "non_existent_subvolume", "group_name": "group_1", "target": ["mon-mgr", ""]}) -- 0x7fd888075e60 con 0x7fd86806cf20 2024-04-20T11:35:42.978 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:35:42.978+0000 7f8a5cc00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-04-20T11:35:42.979 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:42.979+0000 7fd884c00640 1 -- 172.21.15.152:0/4063038156 <== mgr.4109 v2:172.21.15.175:6836/4021250441 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7fd888075e60 con 0x7fd86806cf20 2024-04-20T11:35:42.979 INFO:teuthology.orchestra.run.smithi152.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-04-20T11:35:42.995 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-20T11:35:42.996 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup pin non_existent_subvolume group_1 export 0 2024-04-20T11:35:43.208 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.208+0000 7faceb200640 1 Processor -- start 2024-04-20T11:35:43.208 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.209+0000 7faceb200640 1 -- start start 2024-04-20T11:35:43.209 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.210+0000 7faceb200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7facec0d05f0 0x7facec0d09d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:43.210 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.210+0000 7faceb200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7facec071f20 0x7facec0723b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:43.210 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.210+0000 7faceb200640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7facec0d0f10 con 0x7facec0ae180 2024-04-20T11:35:43.210 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.210+0000 7faceb200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7facec0aeef0 con 0x7facec071f20 2024-04-20T11:35:43.210 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.210+0000 7faceb200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7facec0af1d0 con 0x7facec0d05f0 2024-04-20T11:35:43.210 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.210+0000 7face9400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7facec0ae180 0x7facec0ae600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:40572/0 (socket says 172.21.15.152:40572) 2024-04-20T11:35:43.210 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.210+0000 7face9400640 1 -- 172.21.15.152:0/2759067203 learned_addr learned my addr 172.21.15.152:0/2759067203 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:35:43.214 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.211+0000 7face9e00640 1 --2- 172.21.15.152:0/2759067203 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7facec0d05f0 0x7facec0d09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:43.215 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.211+0000 7facea800640 1 --2- 172.21.15.152:0/2759067203 >> v2:172.21.15.175:3300/0 conn(0x7facec071f20 0x7facec0723b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:43.215 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.211+0000 7face8a00640 1 -- 172.21.15.152:0/2759067203 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1579476563 0 0) 0x7facec0d0f10 con 0x7facec0ae180 2024-04-20T11:35:43.215 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.211+0000 7face8a00640 1 -- 172.21.15.152:0/2759067203 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7facd0003520 con 0x7facec0ae180 2024-04-20T11:35:43.215 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.211+0000 7face8a00640 1 -- 172.21.15.152:0/2759067203 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3940480691 0 0) 0x7facd0003520 con 0x7facec0ae180 2024-04-20T11:35:43.215 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.211+0000 7face8a00640 1 -- 172.21.15.152:0/2759067203 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7facec0d05f0 msgr2=0x7facec0d09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:43.215 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.211+0000 7face8a00640 1 --2- 172.21.15.152:0/2759067203 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7facec0d05f0 0x7facec0d09d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:43.215 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.211+0000 7face8a00640 1 -- 172.21.15.152:0/2759067203 >> v2:172.21.15.175:3300/0 conn(0x7facec071f20 msgr2=0x7facec0723b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:43.215 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.211+0000 7face8a00640 1 --2- 172.21.15.152:0/2759067203 >> v2:172.21.15.175:3300/0 conn(0x7facec071f20 0x7facec0723b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:43.215 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.211+0000 7face8a00640 1 -- 172.21.15.152:0/2759067203 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facec0af4b0 con 0x7facec0ae180 2024-04-20T11:35:43.215 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.211+0000 7face8a00640 1 -- 172.21.15.152:0/2759067203 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7facdc0019f0 con 0x7facec0ae180 2024-04-20T11:35:43.215 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.211+0000 7face8a00640 1 -- 172.21.15.152:0/2759067203 >> v1:172.21.15.152:6789/0 conn(0x7facec0ae180 legacy=0x7facec0ae600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:43.215 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.211+0000 7face8a00640 1 --2- 172.21.15.152:0/2759067203 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7facd00039b0 0x7facd0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:43.215 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.211+0000 7face8a00640 1 --2- 172.21.15.152:0/2759067203 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7facd0004800 0x7facd0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:43.216 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.212+0000 7face8a00640 1 --2- 172.21.15.152:0/2759067203 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7facec0d05f0 0x7facd00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:43.216 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.212+0000 7face8a00640 1 -- 172.21.15.152:0/2759067203 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7facec0aeef0 con 0x7facd00039b0 2024-04-20T11:35:43.216 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.212+0000 7face8a00640 1 -- 172.21.15.152:0/2759067203 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7facec0af1d0 con 0x7facd0004800 2024-04-20T11:35:43.216 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.212+0000 7face8a00640 1 -- 172.21.15.152:0/2759067203 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7facd000b9c0 con 0x7facec0d05f0 2024-04-20T11:35:43.216 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.212+0000 7face9e00640 1 --2- 172.21.15.152:0/2759067203 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7facd00039b0 0x7facd0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:43.216 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.212+0000 7face9400640 1 --2- 172.21.15.152:0/2759067203 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7facd0004800 0x7facd0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:43.216 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.212+0000 7face9e00640 1 -- 172.21.15.152:0/2759067203 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7facec0d05f0 msgr2=0x7facd00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:43.216 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.212+0000 7face9e00640 1 --2- 172.21.15.152:0/2759067203 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7facec0d05f0 0x7facd00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:43.216 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.212+0000 7face9e00640 1 -- 172.21.15.152:0/2759067203 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7facd0004800 msgr2=0x7facd0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:43.216 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.212+0000 7face9e00640 1 --2- 172.21.15.152:0/2759067203 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7facd0004800 0x7facd0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:43.217 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.212+0000 7face9e00640 1 -- 172.21.15.152:0/2759067203 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facd0003520 con 0x7facd00039b0 2024-04-20T11:35:43.217 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.212+0000 7face9400640 1 --2- 172.21.15.152:0/2759067203 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7facd0004800 0x7facd0004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:35:43.217 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.213+0000 7face9e00640 1 --2- 172.21.15.152:0/2759067203 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7facd00039b0 0x7facd0003dd0 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7facd4009ac0 tx=0x7facd0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c7c222c415f053c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:43.217 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.214+0000 7face8a00640 1 -- 172.21.15.152:0/2759067203 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd4039070 con 0x7facd00039b0 2024-04-20T11:35:43.217 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.214+0000 7face8a00640 1 -- 172.21.15.152:0/2759067203 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facd402c710 con 0x7facd00039b0 2024-04-20T11:35:43.217 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.214+0000 7face8a00640 1 -- 172.21.15.152:0/2759067203 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd4009690 con 0x7facd00039b0 2024-04-20T11:35:43.217 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.214+0000 7faceb200640 1 -- 172.21.15.152:0/2759067203 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7facd00039b0 msgr2=0x7facd0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:43.217 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.214+0000 7faceb200640 1 --2- 172.21.15.152:0/2759067203 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7facd00039b0 0x7facd0003dd0 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7facd4009ac0 tx=0x7facd0005830 comp rx=0 tx=0).stop 2024-04-20T11:35:43.217 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.214+0000 7facea800640 1 -- 172.21.15.152:0/2759067203 reap_dead start 2024-04-20T11:35:43.217 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.214+0000 7faceb200640 1 -- 172.21.15.152:0/2759067203 shutdown_connections 2024-04-20T11:35:43.217 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.214+0000 7faceb200640 1 -- 172.21.15.152:0/2759067203 >> 172.21.15.152:0/2759067203 conn(0x7facec0b21e0 msgr2=0x7facec0b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:43.217 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.215+0000 7faceb200640 1 -- 172.21.15.152:0/2759067203 shutdown_connections 2024-04-20T11:35:43.217 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.215+0000 7faceb200640 1 -- 172.21.15.152:0/2759067203 wait complete. 2024-04-20T11:35:43.220 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.216+0000 7faceb200640 1 Processor -- start 2024-04-20T11:35:43.220 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.216+0000 7faceb200640 1 -- start start 2024-04-20T11:35:43.220 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.217+0000 7faceb200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7facd00039b0 0x7facec1f1e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:43.220 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.217+0000 7faceb200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7facec091230 0x7facec1569c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:43.220 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.217+0000 7faceb200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7facec154ca0 0x7facec0cedb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:43.220 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.217+0000 7faceb200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7facec155ca0 con 0x7facd00039b0 2024-04-20T11:35:43.220 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.217+0000 7faceb200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7facec0afa70 con 0x7facec154ca0 2024-04-20T11:35:43.220 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.217+0000 7faceb200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7facec0917b0 con 0x7facec091230 2024-04-20T11:35:43.220 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.218+0000 7face9e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7facd00039b0 0x7facec1f1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:43.220 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.218+0000 7face9e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7facd00039b0 0x7facec1f1e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:38106/0 (socket says 172.21.15.152:38106) 2024-04-20T11:35:43.220 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.218+0000 7face9e00640 1 -- 172.21.15.152:0/3990331416 learned_addr learned my addr 172.21.15.152:0/3990331416 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:43.220 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.218+0000 7face9e00640 1 -- 172.21.15.152:0/3990331416 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7facec091230 msgr2=0x7facec1569c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:43.221 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.218+0000 7face9e00640 1 --2- 172.21.15.152:0/3990331416 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7facec091230 0x7facec1569c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:43.221 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.218+0000 7face9e00640 1 -- 172.21.15.152:0/3990331416 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7facec154ca0 msgr2=0x7facec0cedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:43.221 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.218+0000 7face9e00640 1 --2- 172.21.15.152:0/3990331416 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7facec154ca0 0x7facec0cedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:43.221 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.218+0000 7face9e00640 1 -- 172.21.15.152:0/3990331416 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facec091a90 con 0x7facd00039b0 2024-04-20T11:35:43.221 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.218+0000 7face9e00640 1 --2- 172.21.15.152:0/3990331416 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7facd00039b0 0x7facec1f1e60 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7facd4037040 tx=0x7facd4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:43.221 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.219+0000 7face2000640 1 -- 172.21.15.152:0/3990331416 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd4039050 con 0x7facd00039b0 2024-04-20T11:35:43.221 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.219+0000 7face2000640 1 -- 172.21.15.152:0/3990331416 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facd4045070 con 0x7facd00039b0 2024-04-20T11:35:43.221 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.219+0000 7face2000640 1 -- 172.21.15.152:0/3990331416 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd4003a30 con 0x7facd00039b0 2024-04-20T11:35:43.221 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.219+0000 7faceb200640 1 -- 172.21.15.152:0/3990331416 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facec091d70 con 0x7facd00039b0 2024-04-20T11:35:43.221 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.219+0000 7faceb200640 1 -- 172.21.15.152:0/3990331416 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facec092330 con 0x7facd00039b0 2024-04-20T11:35:43.227 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.220+0000 7faceb200640 1 -- 172.21.15.152:0/3990331416 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facec075e60 con 0x7facd00039b0 2024-04-20T11:35:43.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.221+0000 7face2000640 1 -- 172.21.15.152:0/3990331416 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 22) ==== 91601+0+0 (secure 0 0 0) 0x7facd4003bd0 con 0x7facd00039b0 2024-04-20T11:35:43.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.222+0000 7face2000640 1 --2- 172.21.15.152:0/3990331416 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7facc806cf50 0x7facc806f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:43.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.222+0000 7face2000640 1 -- 172.21.15.152:0/3990331416 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5709+0+0 (secure 0 0 0) 0x7facd40ae230 con 0x7facd00039b0 2024-04-20T11:35:43.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.223+0000 7face9400640 1 --2- 172.21.15.152:0/3990331416 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7facc806cf50 0x7facc806f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:43.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.224+0000 7face9400640 1 --2- 172.21.15.152:0/3990331416 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7facc806cf50 0x7facc806f410 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7facdc001ca0 tx=0x7facdc009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:43.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.227+0000 7face2000640 1 -- 172.21.15.152:0/3990331416 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7facd404e020 con 0x7facd00039b0 2024-04-20T11:35:43.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.459+0000 7faceb200640 1 -- 172.21.15.152:0/3990331416 --> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "non_existent_subvolume", "group_name": "group_1", "pin_type": "export", "pin_setting": "0", "target": ["mon-mgr", ""]}) -- 0x7facec0d0d00 con 0x7facc806cf50 2024-04-20T11:35:43.462 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:35:43.462+0000 7f8a5cc00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-04-20T11:35:43.463 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.463+0000 7face2000640 1 -- 172.21.15.152:0/3990331416 <== mgr.4109 v2:172.21.15.175:6836/4021250441 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7facec0d0d00 con 0x7facc806cf50 2024-04-20T11:35:43.463 INFO:teuthology.orchestra.run.smithi152.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-04-20T11:35:43.473 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-20T11:35:43.474 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup ls non_existent_subvolume 2024-04-20T11:35:43.677 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.678+0000 7f8fbdc00640 1 Processor -- start 2024-04-20T11:35:43.678 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.678+0000 7f8fbdc00640 1 -- start start 2024-04-20T11:35:43.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.679+0000 7f8fbdc00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f8fb8146000 0x7f8fb80b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:43.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.679+0000 7f8fbdc00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8fb80b2170 0x7f8fb80b25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:43.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.679+0000 7f8fbdc00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8fb81463e0 con 0x7f8fb80cd1b0 2024-04-20T11:35:43.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.679+0000 7f8fbdc00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f8fb80b4750 con 0x7f8fb8146000 2024-04-20T11:35:43.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.679+0000 7f8fbdc00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8fb80b3e20 con 0x7f8fb80b2170 2024-04-20T11:35:43.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.680+0000 7f8fb6a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f8fb8146000 0x7f8fb80b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:43.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.680+0000 7f8fb6a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f8fb8146000 0x7f8fb80b4be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:38114/0 (socket says 172.21.15.152:38114) 2024-04-20T11:35:43.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.680+0000 7f8fb6a00640 1 -- 172.21.15.152:0/1690219318 learned_addr learned my addr 172.21.15.152:0/1690219318 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:43.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.680+0000 7f8fb7400640 1 --1- 172.21.15.152:0/1690219318 >> v1:172.21.15.152:6789/0 conn(0x7f8fb80cd1b0 0x7f8fb80cd5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:40574/0 (socket says 172.21.15.152:40574) 2024-04-20T11:35:43.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.680+0000 7f8fb6a00640 1 -- 172.21.15.152:0/1690219318 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8fb80b2170 msgr2=0x7f8fb80b25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:43.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.680+0000 7f8fae000640 1 --2- 172.21.15.152:0/1690219318 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8fb80b2170 0x7f8fb80b25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:43.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.680+0000 7f8fb6a00640 1 --2- 172.21.15.152:0/1690219318 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8fb80b2170 0x7f8fb80b25f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:43.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.680+0000 7f8fb6a00640 1 -- 172.21.15.152:0/1690219318 >> v1:172.21.15.152:6789/0 conn(0x7f8fb80cd1b0 legacy=0x7f8fb80cd5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:43.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.680+0000 7f8fb6000640 1 -- 172.21.15.152:0/1690219318 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1442736737 0 0) 0x7f8fb81463e0 con 0x7f8fb80cd1b0 2024-04-20T11:35:43.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.680+0000 7f8fb6a00640 1 -- 172.21.15.152:0/1690219318 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fb80b4100 con 0x7f8fb8146000 2024-04-20T11:35:43.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.680+0000 7f8fae000640 1 --2- 172.21.15.152:0/1690219318 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8fb80b2170 0x7f8fb80b25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:35:43.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.682+0000 7f8fb6a00640 1 --2- 172.21.15.152:0/1690219318 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8fb8146000 0x7f8fb80b4be0 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f8fa8009870 tx=0x7f8fa802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a33222c2d44a484f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:43.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.682+0000 7f8fb6000640 1 -- 172.21.15.152:0/1690219318 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa8035020 con 0x7f8fb8146000 2024-04-20T11:35:43.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.682+0000 7f8fb6000640 1 -- 172.21.15.152:0/1690219318 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fa8031640 con 0x7f8fb8146000 2024-04-20T11:35:43.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.682+0000 7f8fb6000640 1 -- 172.21.15.152:0/1690219318 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa802d040 con 0x7f8fb8146000 2024-04-20T11:35:43.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.682+0000 7f8fbdc00640 1 -- 172.21.15.152:0/1690219318 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8fb8146000 msgr2=0x7f8fb80b4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:43.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.682+0000 7f8fbdc00640 1 --2- 172.21.15.152:0/1690219318 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8fb8146000 0x7f8fb80b4be0 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f8fa8009870 tx=0x7f8fa802b8f0 comp rx=0 tx=0).stop 2024-04-20T11:35:43.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.683+0000 7f8fbdc00640 1 -- 172.21.15.152:0/1690219318 shutdown_connections 2024-04-20T11:35:43.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.683+0000 7f8fbdc00640 1 --2- 172.21.15.152:0/1690219318 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8fb80b2170 0x7f8fb80b25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:43.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.683+0000 7f8fbdc00640 1 --2- 172.21.15.152:0/1690219318 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8fb8146000 0x7f8fb80b4be0 unknown :-1 s=CLOSED pgs=1840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:43.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.683+0000 7f8fbdc00640 1 -- 172.21.15.152:0/1690219318 >> 172.21.15.152:0/1690219318 conn(0x7f8fb806b980 msgr2=0x7f8fb806bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:43.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.683+0000 7f8fbdc00640 1 -- 172.21.15.152:0/1690219318 shutdown_connections 2024-04-20T11:35:43.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.683+0000 7f8fbdc00640 1 -- 172.21.15.152:0/1690219318 wait complete. 2024-04-20T11:35:43.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.684+0000 7f8fbdc00640 1 Processor -- start 2024-04-20T11:35:43.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.684+0000 7f8fbdc00640 1 -- start start 2024-04-20T11:35:43.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.685+0000 7f8fbdc00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8fb80b2170 0x7f8fb80c9b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:43.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.685+0000 7f8fbdc00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8fb80cd1b0 0x7f8fb815c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:43.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.685+0000 7f8fbdc00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8fb8146000 0x7f8fb815ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:43.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.685+0000 7f8fbdc00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb815ce70 con 0x7f8fb80cd1b0 2024-04-20T11:35:43.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.685+0000 7f8fbdc00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb815ccf0 con 0x7f8fb8146000 2024-04-20T11:35:43.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.685+0000 7f8fbdc00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8fb815cff0 con 0x7f8fb80b2170 2024-04-20T11:35:43.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.686+0000 7f8fae000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8fb80cd1b0 0x7f8fb815c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:43.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.686+0000 7f8fae000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8fb80cd1b0 0x7f8fb815c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:38128/0 (socket says 172.21.15.152:38128) 2024-04-20T11:35:43.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.686+0000 7f8fae000640 1 -- 172.21.15.152:0/1769585353 learned_addr learned my addr 172.21.15.152:0/1769585353 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:43.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.686+0000 7f8fb6a00640 1 --2- 172.21.15.152:0/1769585353 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8fb80b2170 0x7f8fb80c9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:43.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.686+0000 7f8fae000640 1 -- 172.21.15.152:0/1769585353 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8fb80b2170 msgr2=0x7f8fb80c9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:43.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.686+0000 7f8fae000640 1 --2- 172.21.15.152:0/1769585353 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8fb80b2170 0x7f8fb80c9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:43.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.686+0000 7f8fae000640 1 -- 172.21.15.152:0/1769585353 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8fb8146000 msgr2=0x7f8fb815ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:43.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.686+0000 7f8fae000640 1 --2- 172.21.15.152:0/1769585353 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8fb8146000 0x7f8fb815ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:43.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.686+0000 7f8fae000640 1 -- 172.21.15.152:0/1769585353 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fb80c8ae0 con 0x7f8fb80cd1b0 2024-04-20T11:35:43.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.686+0000 7f8fb6a00640 1 --2- 172.21.15.152:0/1769585353 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8fb80b2170 0x7f8fb80c9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:35:43.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.686+0000 7f8fae000640 1 --2- 172.21.15.152:0/1769585353 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8fb80cd1b0 0x7f8fb815c1a0 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f8fa000c7f0 tx=0x7f8fa000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:43.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.687+0000 7f8faf400640 1 -- 172.21.15.152:0/1769585353 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa0015070 con 0x7f8fb80cd1b0 2024-04-20T11:35:43.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.687+0000 7f8faf400640 1 -- 172.21.15.152:0/1769585353 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fa0007a30 con 0x7f8fb80cd1b0 2024-04-20T11:35:43.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.687+0000 7f8faf400640 1 -- 172.21.15.152:0/1769585353 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa0007e00 con 0x7f8fb80cd1b0 2024-04-20T11:35:43.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.687+0000 7f8fbdc00640 1 -- 172.21.15.152:0/1769585353 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fb80c8dc0 con 0x7f8fb80cd1b0 2024-04-20T11:35:43.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.687+0000 7f8fbdc00640 1 -- 172.21.15.152:0/1769585353 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fb80c9380 con 0x7f8fb80cd1b0 2024-04-20T11:35:43.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.688+0000 7f8fbdc00640 1 -- 172.21.15.152:0/1769585353 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fb80689f0 con 0x7f8fb80cd1b0 2024-04-20T11:35:43.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.689+0000 7f8faf400640 1 -- 172.21.15.152:0/1769585353 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 22) ==== 91601+0+0 (secure 0 0 0) 0x7f8fa0018070 con 0x7f8fb80cd1b0 2024-04-20T11:35:43.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.690+0000 7f8faf400640 1 --2- 172.21.15.152:0/1769585353 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f8f9406cf50 0x7f8f9406f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:43.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.690+0000 7f8faf400640 1 -- 172.21.15.152:0/1769585353 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5709+0+0 (secure 0 0 0) 0x7f8fa008c6a0 con 0x7f8fb80cd1b0 2024-04-20T11:35:43.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.691+0000 7f8fb6a00640 1 --2- 172.21.15.152:0/1769585353 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f8f9406cf50 0x7f8f9406f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:43.691 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.692+0000 7f8fb6a00640 1 --2- 172.21.15.152:0/1769585353 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f8f9406cf50 0x7f8f9406f410 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f8fa8036000 tx=0x7f8fa8008030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:43.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.694+0000 7f8faf400640 1 -- 172.21.15.152:0/1769585353 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f8fa005a1d0 con 0x7f8fb80cd1b0 2024-04-20T11:35:43.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.925+0000 7f8fbdc00640 1 -- 172.21.15.152:0/1769585353 --> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup ls", "vol_name": "non_existent_subvolume", "target": ["mon-mgr", ""]}) -- 0x7f8fb8142530 con 0x7f8f9406cf50 2024-04-20T11:35:43.928 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:35:43.928+0000 7f8a5cc00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-04-20T11:35:43.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:43.929+0000 7f8faf400640 1 -- 172.21.15.152:0/1769585353 <== mgr.4109 v2:172.21.15.175:6836/4021250441 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f8fb8142530 con 0x7f8f9406cf50 2024-04-20T11:35:43.929 INFO:teuthology.orchestra.run.smithi152.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-04-20T11:35:43.944 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-20T11:35:43.945 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup snapshot create non_existent_subvolume group_1 snapshot_1 2024-04-20T11:35:44.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.157+0000 7fd0e8a00640 1 Processor -- start 2024-04-20T11:35:44.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.158+0000 7fd0e8a00640 1 -- start start 2024-04-20T11:35:44.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.159+0000 7fd0e8a00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fd0e40b88d0 0x7fd0e4169520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:44.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.159+0000 7fd0e8a00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0e40d40e0 con 0x7fd0e40d22d0 2024-04-20T11:35:44.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.159+0000 7fd0e8a00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0e40d43c0 con 0x7fd0e415e910 2024-04-20T11:35:44.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.159+0000 7fd0e8a00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fd0e40d46a0 con 0x7fd0e40b88d0 2024-04-20T11:35:44.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.159+0000 7fd0e1400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fd0e40d22d0 0x7fd0e40cf4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:40578/0 (socket says 172.21.15.152:40578) 2024-04-20T11:35:44.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.159+0000 7fd0e1400640 1 -- 172.21.15.152:0/2612898505 learned_addr learned my addr 172.21.15.152:0/2612898505 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:35:44.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.160+0000 7fd0e2800640 1 --2- 172.21.15.152:0/2612898505 >> v2:172.21.15.175:3300/0 conn(0x7fd0e40b88d0 0x7fd0e4169520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:44.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.160+0000 7fd0e0a00640 1 -- 172.21.15.152:0/2612898505 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 90881284 0 0) 0x7fd0e40d40e0 con 0x7fd0e40d22d0 2024-04-20T11:35:44.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.160+0000 7fd0e0a00640 1 -- 172.21.15.152:0/2612898505 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd0c8003520 con 0x7fd0e40d22d0 2024-04-20T11:35:44.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.160+0000 7fd0e0a00640 1 -- 172.21.15.152:0/2612898505 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 419637595 0 0) 0x7fd0c8003520 con 0x7fd0e40d22d0 2024-04-20T11:35:44.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.160+0000 7fd0e0a00640 1 -- 172.21.15.152:0/2612898505 >> v2:172.21.15.175:3300/0 conn(0x7fd0e40b88d0 msgr2=0x7fd0e4169520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:44.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.160+0000 7fd0e0a00640 1 --2- 172.21.15.152:0/2612898505 >> v2:172.21.15.175:3300/0 conn(0x7fd0e40b88d0 0x7fd0e4169520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:44.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.160+0000 7fd0e0a00640 1 -- 172.21.15.152:0/2612898505 >> v1:172.21.15.175:6789/0 conn(0x7fd0e415e910 legacy=0x7fd0e40d3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:44.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.160+0000 7fd0e0a00640 1 -- 172.21.15.152:0/2612898505 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0e40d4980 con 0x7fd0e40d22d0 2024-04-20T11:35:44.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.160+0000 7fd0e0a00640 1 -- 172.21.15.152:0/2612898505 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7fd0d0001670 con 0x7fd0e40d22d0 2024-04-20T11:35:44.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.160+0000 7fd0e0a00640 1 -- 172.21.15.152:0/2612898505 >> v1:172.21.15.152:6789/0 conn(0x7fd0e40d22d0 legacy=0x7fd0e40cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:44.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.161+0000 7fd0e0a00640 1 --2- 172.21.15.152:0/2612898505 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd0c8003a70 0x7fd0c8003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:44.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.161+0000 7fd0e0a00640 1 --2- 172.21.15.152:0/2612898505 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd0c8004800 0x7fd0c8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:44.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.161+0000 7fd0e0a00640 1 --2- 172.21.15.152:0/2612898505 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd0c80051c0 0x7fd0c800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:44.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.161+0000 7fd0e0a00640 1 -- 172.21.15.152:0/2612898505 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fd0e40d46a0 con 0x7fd0c80051c0 2024-04-20T11:35:44.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.161+0000 7fd0e0a00640 1 -- 172.21.15.152:0/2612898505 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd0c800c160 con 0x7fd0c8003a70 2024-04-20T11:35:44.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.161+0000 7fd0e0a00640 1 -- 172.21.15.152:0/2612898505 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd0c800c360 con 0x7fd0c8004800 2024-04-20T11:35:44.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.161+0000 7fd0e2800640 1 --2- 172.21.15.152:0/2612898505 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd0c80051c0 0x7fd0c800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:44.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.161+0000 7fd0e1400640 1 --2- 172.21.15.152:0/2612898505 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd0c8004800 0x7fd0c8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:44.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.161+0000 7fd0e1e00640 1 --2- 172.21.15.152:0/2612898505 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd0c8003a70 0x7fd0c8003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:44.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.162+0000 7fd0e2800640 1 -- 172.21.15.152:0/2612898505 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd0c8004800 msgr2=0x7fd0c8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:44.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.162+0000 7fd0e2800640 1 --2- 172.21.15.152:0/2612898505 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd0c8004800 0x7fd0c8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:44.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.162+0000 7fd0e2800640 1 -- 172.21.15.152:0/2612898505 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd0c8003a70 msgr2=0x7fd0c8003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:44.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.162+0000 7fd0e2800640 1 --2- 172.21.15.152:0/2612898505 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd0c8003a70 0x7fd0c8003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:44.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.162+0000 7fd0e2800640 1 -- 172.21.15.152:0/2612898505 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0c8003520 con 0x7fd0c80051c0 2024-04-20T11:35:44.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.162+0000 7fd0e1400640 1 --2- 172.21.15.152:0/2612898505 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd0c8004800 0x7fd0c8004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:35:44.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.163+0000 7fd0e2800640 1 -- 172.21.15.152:0/2612898505 reap_dead start 2024-04-20T11:35:44.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.163+0000 7fd0e2800640 1 --2- 172.21.15.152:0/2612898505 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd0c80051c0 0x7fd0c800b9c0 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc009ac0 tx=0x7fd0c8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87ffa0961f138467 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:44.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.163+0000 7fd0e0a00640 1 -- 172.21.15.152:0/2612898505 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0dc009690 con 0x7fd0c80051c0 2024-04-20T11:35:44.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.163+0000 7fd0e0a00640 1 -- 172.21.15.152:0/2612898505 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0dc004020 con 0x7fd0c80051c0 2024-04-20T11:35:44.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.163+0000 7fd0e0a00640 1 -- 172.21.15.152:0/2612898505 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0dc004410 con 0x7fd0c80051c0 2024-04-20T11:35:44.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.164+0000 7fd0e8a00640 1 -- 172.21.15.152:0/2612898505 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd0c80051c0 msgr2=0x7fd0c800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:44.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.164+0000 7fd0e8a00640 1 --2- 172.21.15.152:0/2612898505 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd0c80051c0 0x7fd0c800b9c0 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc009ac0 tx=0x7fd0c8005830 comp rx=0 tx=0).stop 2024-04-20T11:35:44.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.164+0000 7fd0e8a00640 1 -- 172.21.15.152:0/2612898505 shutdown_connections 2024-04-20T11:35:44.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.164+0000 7fd0e8a00640 1 --2- 172.21.15.152:0/2612898505 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd0c80051c0 0x7fd0c800b9c0 unknown :-1 s=CLOSED pgs=1843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:44.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.164+0000 7fd0e8a00640 1 -- 172.21.15.152:0/2612898505 >> 172.21.15.152:0/2612898505 conn(0x7fd0e4067ae0 msgr2=0x7fd0e4067ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:44.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.164+0000 7fd0e8a00640 1 -- 172.21.15.152:0/2612898505 shutdown_connections 2024-04-20T11:35:44.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.164+0000 7fd0e8a00640 1 -- 172.21.15.152:0/2612898505 wait complete. 2024-04-20T11:35:44.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.166+0000 7fd0e8a00640 1 Processor -- start 2024-04-20T11:35:44.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.166+0000 7fd0e8a00640 1 -- start start 2024-04-20T11:35:44.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.167+0000 7fd0e8a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd0c80051c0 0x7fd0e40ca370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:44.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.167+0000 7fd0e8a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd0e41f86c0 0x7fd0e4161790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:44.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.167+0000 7fd0e8a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd0e41e7180 0x7fd0e41e7610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:44.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.167+0000 7fd0e8a00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fd0e40d3870 con 0x7fd0e41f86c0 2024-04-20T11:35:44.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.167+0000 7fd0e8a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd0e415ecf0 con 0x7fd0e41e7180 2024-04-20T11:35:44.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.167+0000 7fd0e8a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd0e40d39f0 con 0x7fd0c80051c0 2024-04-20T11:35:44.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.167+0000 7fd0e1400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd0e41f86c0 0x7fd0e4161790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:44.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.167+0000 7fd0e1400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd0e41f86c0 0x7fd0e4161790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:38142/0 (socket says 172.21.15.152:38142) 2024-04-20T11:35:44.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.167+0000 7fd0e1400640 1 -- 172.21.15.152:0/1760001689 learned_addr learned my addr 172.21.15.152:0/1760001689 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:44.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.167+0000 7fd0e1e00640 1 --2- 172.21.15.152:0/1760001689 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd0c80051c0 0x7fd0e40ca370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:44.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.167+0000 7fd0e2800640 1 --2- 172.21.15.152:0/1760001689 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd0e41e7180 0x7fd0e41e7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:44.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.168+0000 7fd0e1400640 1 -- 172.21.15.152:0/1760001689 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd0c80051c0 msgr2=0x7fd0e40ca370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:44.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.168+0000 7fd0e1400640 1 --2- 172.21.15.152:0/1760001689 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd0c80051c0 0x7fd0e40ca370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:44.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.168+0000 7fd0e1400640 1 -- 172.21.15.152:0/1760001689 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd0e41e7180 msgr2=0x7fd0e41e7610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:44.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.168+0000 7fd0e1400640 1 --2- 172.21.15.152:0/1760001689 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd0e41e7180 0x7fd0e41e7610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:44.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.168+0000 7fd0e1400640 1 -- 172.21.15.152:0/1760001689 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0e41f8bc0 con 0x7fd0e41f86c0 2024-04-20T11:35:44.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.168+0000 7fd0e2800640 1 --2- 172.21.15.152:0/1760001689 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd0e41e7180 0x7fd0e41e7610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:35:44.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.168+0000 7fd0e1400640 1 --2- 172.21.15.152:0/1760001689 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd0e41f86c0 0x7fd0e4161790 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7fd0d000e780 tx=0x7fd0d000ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:44.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.168+0000 7fd0da000640 1 -- 172.21.15.152:0/1760001689 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0d000c660 con 0x7fd0e41f86c0 2024-04-20T11:35:44.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.168+0000 7fd0da000640 1 -- 172.21.15.152:0/1760001689 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0d0018070 con 0x7fd0e41f86c0 2024-04-20T11:35:44.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.168+0000 7fd0da000640 1 -- 172.21.15.152:0/1760001689 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0d000ca20 con 0x7fd0e41f86c0 2024-04-20T11:35:44.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.168+0000 7fd0e8a00640 1 -- 172.21.15.152:0/1760001689 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0e41f8ea0 con 0x7fd0e41f86c0 2024-04-20T11:35:44.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.168+0000 7fd0e8a00640 1 -- 172.21.15.152:0/1760001689 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0e41f9460 con 0x7fd0e41f86c0 2024-04-20T11:35:44.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.169+0000 7fd0e8a00640 1 -- 172.21.15.152:0/1760001689 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0e4068a60 con 0x7fd0e41f86c0 2024-04-20T11:35:44.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.171+0000 7fd0da000640 1 -- 172.21.15.152:0/1760001689 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 22) ==== 91601+0+0 (secure 0 0 0) 0x7fd0d000cbc0 con 0x7fd0e41f86c0 2024-04-20T11:35:44.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.171+0000 7fd0da000640 1 --2- 172.21.15.152:0/1760001689 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd0c406cf50 0x7fd0c406f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:44.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.172+0000 7fd0da000640 1 -- 172.21.15.152:0/1760001689 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5709+0+0 (secure 0 0 0) 0x7fd0d008c360 con 0x7fd0e41f86c0 2024-04-20T11:35:44.172 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.172+0000 7fd0e1e00640 1 --2- 172.21.15.152:0/1760001689 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd0c406cf50 0x7fd0c406f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:44.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.173+0000 7fd0e1e00640 1 --2- 172.21.15.152:0/1760001689 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd0c406cf50 0x7fd0c406f410 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fd0d4005600 tx=0x7fd0d4009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:44.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.176+0000 7fd0da000640 1 -- 172.21.15.152:0/1760001689 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fd0d0059ee0 con 0x7fd0e41f86c0 2024-04-20T11:35:44.406 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.407+0000 7fd0e8a00640 1 -- 172.21.15.152:0/1760001689 --> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup snapshot create", "vol_name": "non_existent_subvolume", "group_name": "group_1", "snap_name": "snapshot_1", "target": ["mon-mgr", ""]}) -- 0x7fd0e40b5210 con 0x7fd0c406cf50 2024-04-20T11:35:44.410 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:35:44.410+0000 7f8a5cc00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-04-20T11:35:44.411 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.411+0000 7fd0da000640 1 -- 172.21.15.152:0/1760001689 <== mgr.4109 v2:172.21.15.175:6836/4021250441 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7fd0e40b5210 con 0x7fd0c406cf50 2024-04-20T11:35:44.411 INFO:teuthology.orchestra.run.smithi152.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-04-20T11:35:44.425 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-20T11:35:44.426 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup snapshot rm non_existent_subvolume group_1 snapshot_1 2024-04-20T11:35:44.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.623+0000 7f411bc00640 1 Processor -- start 2024-04-20T11:35:44.623 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.623+0000 7f411bc00640 1 -- start start 2024-04-20T11:35:44.625 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.625+0000 7f411bc00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f41140c73f0 0x7f41140c6310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:44.625 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.625+0000 7f411bc00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f41140d1b60 0x7f4114162aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:44.625 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.625+0000 7f411bc00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41140c79b0 con 0x7f41140c6850 2024-04-20T11:35:44.625 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.625+0000 7f411bc00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f4114162610 con 0x7f41140c73f0 2024-04-20T11:35:44.625 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.625+0000 7f411bc00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f41140c7c70 con 0x7f41140d1b60 2024-04-20T11:35:44.625 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.625+0000 7f4119200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f41140c73f0 0x7f41140c6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:44.625 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.625+0000 7f4119200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f41140c73f0 0x7f41140c6310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:38158/0 (socket says 172.21.15.152:38158) 2024-04-20T11:35:44.625 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.625+0000 7f4119200640 1 -- 172.21.15.152:0/3531308366 learned_addr learned my addr 172.21.15.152:0/3531308366 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:44.625 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.625+0000 7f4119c00640 1 --1- 172.21.15.152:0/3531308366 >> v1:172.21.15.152:6789/0 conn(0x7f41140c6850 0x7f411406b980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:40586/0 (socket says 172.21.15.152:40586) 2024-04-20T11:35:44.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.626+0000 7f4119200640 1 -- 172.21.15.152:0/3531308366 >> v2:172.21.15.175:3300/0 conn(0x7f41140d1b60 msgr2=0x7f4114162aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:44.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.626+0000 7f4119200640 1 --2- 172.21.15.152:0/3531308366 >> v2:172.21.15.175:3300/0 conn(0x7f41140d1b60 0x7f4114162aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:44.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.626+0000 7f4119200640 1 -- 172.21.15.152:0/3531308366 >> v1:172.21.15.152:6789/0 conn(0x7f41140c6850 legacy=0x7f411406b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:44.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.626+0000 7f4119200640 1 -- 172.21.15.152:0/3531308366 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41140c7f50 con 0x7f41140c73f0 2024-04-20T11:35:44.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.627+0000 7f4119200640 1 --2- 172.21.15.152:0/3531308366 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f41140c73f0 0x7f41140c6310 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f41140703b0 tx=0x7f410802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6562e032e7505656 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:44.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.627+0000 7f4113400640 1 -- 172.21.15.152:0/3531308366 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4108034070 con 0x7f41140c73f0 2024-04-20T11:35:44.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.627+0000 7f4113400640 1 -- 172.21.15.152:0/3531308366 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4108030860 con 0x7f41140c73f0 2024-04-20T11:35:44.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.627+0000 7f4113400640 1 -- 172.21.15.152:0/3531308366 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4108030c30 con 0x7f41140c73f0 2024-04-20T11:35:44.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.628+0000 7f411bc00640 1 -- 172.21.15.152:0/3531308366 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f41140c73f0 msgr2=0x7f41140c6310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:44.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.628+0000 7f411bc00640 1 --2- 172.21.15.152:0/3531308366 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f41140c73f0 0x7f41140c6310 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f41140703b0 tx=0x7f410802b9a0 comp rx=0 tx=0).stop 2024-04-20T11:35:44.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.628+0000 7f411bc00640 1 -- 172.21.15.152:0/3531308366 shutdown_connections 2024-04-20T11:35:44.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.628+0000 7f411bc00640 1 --2- 172.21.15.152:0/3531308366 >> v2:172.21.15.175:3300/0 conn(0x7f41140d1b60 0x7f4114162aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:44.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.628+0000 7f411bc00640 1 --2- 172.21.15.152:0/3531308366 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f41140c73f0 0x7f41140c6310 unknown :-1 s=CLOSED pgs=1846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:44.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.628+0000 7f411bc00640 1 -- 172.21.15.152:0/3531308366 >> 172.21.15.152:0/3531308366 conn(0x7f411409aa40 msgr2=0x7f41140d1450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:44.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.628+0000 7f411bc00640 1 -- 172.21.15.152:0/3531308366 shutdown_connections 2024-04-20T11:35:44.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.628+0000 7f411bc00640 1 -- 172.21.15.152:0/3531308366 wait complete. 2024-04-20T11:35:44.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.630+0000 7f411bc00640 1 Processor -- start 2024-04-20T11:35:44.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.630+0000 7f411bc00640 1 -- start start 2024-04-20T11:35:44.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.631+0000 7f411bc00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41140c6850 0x7f41141536e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:44.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.631+0000 7f411bc00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f41140c73f0 0x7f41140c9d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:44.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.631+0000 7f411bc00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f41140d1b60 0x7f41140ca2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:44.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.631+0000 7f411bc00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f41140d1210 con 0x7f41140c73f0 2024-04-20T11:35:44.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.631+0000 7f411bc00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f41140d1090 con 0x7f41140c6850 2024-04-20T11:35:44.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.631+0000 7f411bc00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4114162fe0 con 0x7f41140d1b60 2024-04-20T11:35:44.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.631+0000 7f4113e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f41140c73f0 0x7f41140c9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:44.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.631+0000 7f4113e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f41140c73f0 0x7f41140c9d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:38168/0 (socket says 172.21.15.152:38168) 2024-04-20T11:35:44.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.631+0000 7f4113e00640 1 -- 172.21.15.152:0/703014177 learned_addr learned my addr 172.21.15.152:0/703014177 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:44.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.631+0000 7f4119200640 1 --2- 172.21.15.152:0/703014177 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41140c6850 0x7f41141536e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:44.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.631+0000 7f4119c00640 1 --2- 172.21.15.152:0/703014177 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f41140d1b60 0x7f41140ca2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:44.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.631+0000 7f4113e00640 1 -- 172.21.15.152:0/703014177 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f41140d1b60 msgr2=0x7f41140ca2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:44.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.631+0000 7f4113e00640 1 --2- 172.21.15.152:0/703014177 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f41140d1b60 0x7f41140ca2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:44.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.631+0000 7f4113e00640 1 -- 172.21.15.152:0/703014177 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41140c6850 msgr2=0x7f41141536e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:44.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.631+0000 7f4113e00640 1 --2- 172.21.15.152:0/703014177 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41140c6850 0x7f41141536e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:44.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.631+0000 7f4113e00640 1 -- 172.21.15.152:0/703014177 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41140ca8e0 con 0x7f41140c73f0 2024-04-20T11:35:44.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.631+0000 7f4119200640 1 --2- 172.21.15.152:0/703014177 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41140c6850 0x7f41141536e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:35:44.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.632+0000 7f4113e00640 1 --2- 172.21.15.152:0/703014177 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f41140c73f0 0x7f41140c9d70 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7f410000b440 tx=0x7f410000b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:44.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.632+0000 7f4110c00640 1 -- 172.21.15.152:0/703014177 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4100015020 con 0x7f41140c73f0 2024-04-20T11:35:44.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.632+0000 7f4110c00640 1 -- 172.21.15.152:0/703014177 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4100011890 con 0x7f41140c73f0 2024-04-20T11:35:44.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.632+0000 7f4110c00640 1 -- 172.21.15.152:0/703014177 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4100011c80 con 0x7f41140c73f0 2024-04-20T11:35:44.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.632+0000 7f411bc00640 1 -- 172.21.15.152:0/703014177 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41140cabc0 con 0x7f41140c73f0 2024-04-20T11:35:44.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.632+0000 7f411bc00640 1 -- 172.21.15.152:0/703014177 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41140cb180 con 0x7f41140c73f0 2024-04-20T11:35:44.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.633+0000 7f411bc00640 1 -- 172.21.15.152:0/703014177 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4114153c20 con 0x7f41140c73f0 2024-04-20T11:35:44.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.634+0000 7f4110c00640 1 -- 172.21.15.152:0/703014177 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 22) ==== 91601+0+0 (secure 0 0 0) 0x7f4100018020 con 0x7f41140c73f0 2024-04-20T11:35:44.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.635+0000 7f4110c00640 1 --2- 172.21.15.152:0/703014177 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f40f406cf20 0x7f40f406f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:44.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.635+0000 7f4110c00640 1 -- 172.21.15.152:0/703014177 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5709+0+0 (secure 0 0 0) 0x7f410008c830 con 0x7f41140c73f0 2024-04-20T11:35:44.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.636+0000 7f4119200640 1 --2- 172.21.15.152:0/703014177 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f40f406cf20 0x7f40f406f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:44.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.637+0000 7f4119200640 1 --2- 172.21.15.152:0/703014177 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f40f406cf20 0x7f40f406f3e0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f4108036000 tx=0x7f410803b040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:44.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.640+0000 7f4110c00640 1 -- 172.21.15.152:0/703014177 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f410001f020 con 0x7f41140c73f0 2024-04-20T11:35:44.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.871+0000 7f411bc00640 1 -- 172.21.15.152:0/703014177 --> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup snapshot rm", "vol_name": "non_existent_subvolume", "group_name": "group_1", "snap_name": "snapshot_1", "target": ["mon-mgr", ""]}) -- 0x7f41141622c0 con 0x7f40f406cf20 2024-04-20T11:35:44.874 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:35:44.874+0000 7f8a5cc00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-04-20T11:35:44.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:44.875+0000 7f4110c00640 1 -- 172.21.15.152:0/703014177 <== mgr.4109 v2:172.21.15.175:6836/4021250441 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f41141622c0 con 0x7f40f406cf20 2024-04-20T11:35:44.874 INFO:teuthology.orchestra.run.smithi152.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-04-20T11:35:44.887 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-20T11:35:44.888 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup snapshot ls non_existent_subvolume group_1 2024-04-20T11:35:45.084 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.084+0000 7fcd11200640 1 Processor -- start 2024-04-20T11:35:45.084 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.085+0000 7fcd11200640 1 -- start start 2024-04-20T11:35:45.085 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.086+0000 7fcd11200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcd0c0cf220 0x7fcd0c0b4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:45.086 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.086+0000 7fcd11200640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd0c0cf6c0 con 0x7fcd0c0cd8e0 2024-04-20T11:35:45.086 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.086+0000 7fcd11200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd0c0cf9a0 con 0x7fcd0c144d70 2024-04-20T11:35:45.086 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.086+0000 7fcd11200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fcd0c0d0240 con 0x7fcd0c0cf220 2024-04-20T11:35:45.088 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.086+0000 7fcd09e00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fcd0c0cd8e0 0x7fcd0c06e570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:40592/0 (socket says 172.21.15.152:40592) 2024-04-20T11:35:45.088 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.086+0000 7fcd09e00640 1 -- 172.21.15.152:0/1652494385 learned_addr learned my addr 172.21.15.152:0/1652494385 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:35:45.088 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.087+0000 7fcd03e00640 1 --2- 172.21.15.152:0/1652494385 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcd0c0cf220 0x7fcd0c0b4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:45.088 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.087+0000 7fcd09400640 1 -- 172.21.15.152:0/1652494385 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2015629327 0 0) 0x7fcd0c0cf6c0 con 0x7fcd0c0cd8e0 2024-04-20T11:35:45.088 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.087+0000 7fcd09400640 1 -- 172.21.15.152:0/1652494385 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fccf0003520 con 0x7fcd0c0cd8e0 2024-04-20T11:35:45.088 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.087+0000 7fcd09400640 1 -- 172.21.15.152:0/1652494385 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3298253307 0 0) 0x7fccf0003520 con 0x7fcd0c0cd8e0 2024-04-20T11:35:45.088 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.087+0000 7fcd09400640 1 -- 172.21.15.152:0/1652494385 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcd0c0cf220 msgr2=0x7fcd0c0b4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:45.089 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.087+0000 7fcd09400640 1 --2- 172.21.15.152:0/1652494385 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcd0c0cf220 0x7fcd0c0b4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:45.089 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.087+0000 7fcd09400640 1 -- 172.21.15.152:0/1652494385 >> v1:172.21.15.175:6789/0 conn(0x7fcd0c144d70 legacy=0x7fcd0c15f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:45.089 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.087+0000 7fcd09400640 1 -- 172.21.15.152:0/1652494385 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd0c0cfc80 con 0x7fcd0c0cd8e0 2024-04-20T11:35:45.089 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.088+0000 7fcd09400640 1 -- 172.21.15.152:0/1652494385 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7fccfc001670 con 0x7fcd0c0cd8e0 2024-04-20T11:35:45.089 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.088+0000 7fcd09400640 1 -- 172.21.15.152:0/1652494385 >> v1:172.21.15.152:6789/0 conn(0x7fcd0c0cd8e0 legacy=0x7fcd0c06e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:45.089 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.088+0000 7fcd09400640 1 --2- 172.21.15.152:0/1652494385 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcd0c0cf220 0x7fccf0003a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:45.089 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.088+0000 7fcd09400640 1 --2- 172.21.15.152:0/1652494385 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fccf0003fd0 0x7fccf0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:45.089 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.088+0000 7fcd09400640 1 --2- 172.21.15.152:0/1652494385 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fccf0004d40 0x7fccf00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:45.089 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.088+0000 7fcd09400640 1 -- 172.21.15.152:0/1652494385 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fcd0c0d0240 con 0x7fccf0004d40 2024-04-20T11:35:45.089 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.088+0000 7fcd09400640 1 -- 172.21.15.152:0/1652494385 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fccf000ddc0 con 0x7fccf0003fd0 2024-04-20T11:35:45.089 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.088+0000 7fcd09400640 1 -- 172.21.15.152:0/1652494385 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fccf000dfc0 con 0x7fcd0c0cf220 2024-04-20T11:35:45.089 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.088+0000 7fcd0a800640 1 --2- 172.21.15.152:0/1652494385 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fccf0004d40 0x7fccf00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:45.089 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.088+0000 7fcd09e00640 1 --2- 172.21.15.152:0/1652494385 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcd0c0cf220 0x7fccf0003a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:45.089 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.088+0000 7fcd03e00640 1 --2- 172.21.15.152:0/1652494385 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fccf0003fd0 0x7fccf0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:45.090 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.089+0000 7fcd0a800640 1 -- 172.21.15.152:0/1652494385 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcd0c0cf220 msgr2=0x7fccf0003a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:45.090 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.089+0000 7fcd0a800640 1 --2- 172.21.15.152:0/1652494385 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcd0c0cf220 0x7fccf0003a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:45.090 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.089+0000 7fcd0a800640 1 -- 172.21.15.152:0/1652494385 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fccf0003fd0 msgr2=0x7fccf0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:45.090 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.089+0000 7fcd0a800640 1 --2- 172.21.15.152:0/1652494385 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fccf0003fd0 0x7fccf0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:45.090 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.089+0000 7fcd0a800640 1 -- 172.21.15.152:0/1652494385 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccf0003520 con 0x7fccf0004d40 2024-04-20T11:35:45.090 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.089+0000 7fcd09e00640 1 --2- 172.21.15.152:0/1652494385 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcd0c0cf220 0x7fccf0003a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:35:45.091 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.090+0000 7fcd0a800640 1 --2- 172.21.15.152:0/1652494385 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fccf0004d40 0x7fccf00051c0 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7fcd04001e50 tx=0x7fccf0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae9558c1e399ae59 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:45.091 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.090+0000 7fcd09400640 1 -- 172.21.15.152:0/1652494385 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd04001a70 con 0x7fccf0004d40 2024-04-20T11:35:45.091 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.090+0000 7fcd09400640 1 -- 172.21.15.152:0/1652494385 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd04003b70 con 0x7fccf0004d40 2024-04-20T11:35:45.091 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.090+0000 7fcd09400640 1 -- 172.21.15.152:0/1652494385 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd040050d0 con 0x7fccf0004d40 2024-04-20T11:35:45.091 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.090+0000 7fcd11200640 1 -- 172.21.15.152:0/1652494385 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fccf0004d40 msgr2=0x7fccf00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:45.091 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.090+0000 7fcd11200640 1 --2- 172.21.15.152:0/1652494385 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fccf0004d40 0x7fccf00051c0 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7fcd04001e50 tx=0x7fccf0005830 comp rx=0 tx=0).stop 2024-04-20T11:35:45.091 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.090+0000 7fcd0a800640 1 -- 172.21.15.152:0/1652494385 reap_dead start 2024-04-20T11:35:45.092 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.091+0000 7fcd11200640 1 -- 172.21.15.152:0/1652494385 shutdown_connections 2024-04-20T11:35:45.092 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.091+0000 7fcd11200640 1 -- 172.21.15.152:0/1652494385 >> 172.21.15.152:0/1652494385 conn(0x7fcd0c06b980 msgr2=0x7fcd0c06bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:45.092 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.091+0000 7fcd11200640 1 -- 172.21.15.152:0/1652494385 shutdown_connections 2024-04-20T11:35:45.092 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.091+0000 7fcd11200640 1 -- 172.21.15.152:0/1652494385 wait complete. 2024-04-20T11:35:45.092 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.092+0000 7fcd11200640 1 Processor -- start 2024-04-20T11:35:45.092 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.093+0000 7fcd11200640 1 -- start start 2024-04-20T11:35:45.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.093+0000 7fcd11200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fccf0004d40 0x7fcd0c139b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:45.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.094+0000 7fcd11200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd0c0cb430 0x7fcd0c15af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:45.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.094+0000 7fcd11200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcd0c0cbaf0 0x7fcd0c0ae900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:45.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.094+0000 7fcd11200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fcd0c1447b0 con 0x7fccf0004d40 2024-04-20T11:35:45.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.094+0000 7fcd11200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcd0c0cdcc0 con 0x7fcd0c0cb430 2024-04-20T11:35:45.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.094+0000 7fcd11200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fcd0c144930 con 0x7fcd0c0cbaf0 2024-04-20T11:35:45.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.094+0000 7fcd09e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fccf0004d40 0x7fcd0c139b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:45.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.094+0000 7fcd09e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fccf0004d40 0x7fcd0c139b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:38178/0 (socket says 172.21.15.152:38178) 2024-04-20T11:35:45.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.094+0000 7fcd09e00640 1 -- 172.21.15.152:0/1866868645 learned_addr learned my addr 172.21.15.152:0/1866868645 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:45.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.094+0000 7fcd09e00640 1 -- 172.21.15.152:0/1866868645 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcd0c0cbaf0 msgr2=0x7fcd0c0ae900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:45.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.094+0000 7fcd09e00640 1 --2- 172.21.15.152:0/1866868645 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcd0c0cbaf0 0x7fcd0c0ae900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:45.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.094+0000 7fcd09e00640 1 -- 172.21.15.152:0/1866868645 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd0c0cb430 msgr2=0x7fcd0c15af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:45.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.094+0000 7fcd03e00640 1 --2- 172.21.15.152:0/1866868645 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd0c0cb430 0x7fcd0c15af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:45.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.094+0000 7fcd09e00640 1 --2- 172.21.15.152:0/1866868645 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd0c0cb430 0x7fcd0c15af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:45.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.094+0000 7fcd09e00640 1 -- 172.21.15.152:0/1866868645 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd0c1366d0 con 0x7fccf0004d40 2024-04-20T11:35:45.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.095+0000 7fcd09e00640 1 --2- 172.21.15.152:0/1866868645 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fccf0004d40 0x7fcd0c139b20 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7fccfc00e7a0 tx=0x7fccfc00ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:45.096 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.095+0000 7fcd02000640 1 -- 172.21.15.152:0/1866868645 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccfc00c8d0 con 0x7fccf0004d40 2024-04-20T11:35:45.096 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.095+0000 7fcd02000640 1 -- 172.21.15.152:0/1866868645 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fccfc018070 con 0x7fccf0004d40 2024-04-20T11:35:45.096 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.095+0000 7fcd02000640 1 -- 172.21.15.152:0/1866868645 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccfc00cc70 con 0x7fccf0004d40 2024-04-20T11:35:45.096 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.095+0000 7fcd11200640 1 -- 172.21.15.152:0/1866868645 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd0c1369b0 con 0x7fccf0004d40 2024-04-20T11:35:45.096 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.095+0000 7fcd11200640 1 -- 172.21.15.152:0/1866868645 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd0c136f70 con 0x7fccf0004d40 2024-04-20T11:35:45.096 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.096+0000 7fcd11200640 1 -- 172.21.15.152:0/1866868645 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd0c068a60 con 0x7fccf0004d40 2024-04-20T11:35:45.097 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.097+0000 7fcd02000640 1 -- 172.21.15.152:0/1866868645 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 22) ==== 91601+0+0 (secure 0 0 0) 0x7fccfc01d460 con 0x7fccf0004d40 2024-04-20T11:35:45.098 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.099+0000 7fcd02000640 1 --2- 172.21.15.152:0/1866868645 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fcce806cf20 0x7fcce806f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:45.098 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.099+0000 7fcd02000640 1 -- 172.21.15.152:0/1866868645 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5709+0+0 (secure 0 0 0) 0x7fccfc08c6a0 con 0x7fccf0004d40 2024-04-20T11:35:45.098 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.099+0000 7fcd03e00640 1 --2- 172.21.15.152:0/1866868645 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fcce806cf20 0x7fcce806f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:45.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.101+0000 7fcd03e00640 1 --2- 172.21.15.152:0/1866868645 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fcce806cf20 0x7fcce806f3e0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c0afde0 tx=0x7fccf4008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:45.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.103+0000 7fcd02000640 1 -- 172.21.15.152:0/1866868645 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fccfc05a0c0 con 0x7fccf0004d40 2024-04-20T11:35:45.333 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.333+0000 7fcd11200640 1 -- 172.21.15.152:0/1866868645 --> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup snapshot ls", "vol_name": "non_existent_subvolume", "group_name": "group_1", "target": ["mon-mgr", ""]}) -- 0x7fcd0c0b32e0 con 0x7fcce806cf20 2024-04-20T11:35:45.336 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:35:45.336+0000 7f8a5cc00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-04-20T11:35:45.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.337+0000 7fcd02000640 1 -- 172.21.15.152:0/1866868645 <== mgr.4109 v2:172.21.15.175:6836/4021250441 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7fcd0c0b32e0 con 0x7fcce806cf20 2024-04-20T11:35:45.337 INFO:teuthology.orchestra.run.smithi152.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-04-20T11:35:45.350 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-20T11:35:45.351 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-20T11:35:45.351 DEBUG:teuthology.orchestra.run.smithi152:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-20T11:35:45.458 DEBUG:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-20T11:35:45.567 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-20T11:35:45.567 INFO:tasks.cephfs.mount:Terminating background process 2024-04-20T11:35:45.567 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-20T11:35:45.567 DEBUG:teuthology.orchestra.run.smithi152:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:35:45.627 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:35:45.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.833+0000 7fc61fe00640 1 Processor -- start 2024-04-20T11:35:45.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.833+0000 7fc61fe00640 1 -- start start 2024-04-20T11:35:45.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.834+0000 7fc61fe00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fc618145f60 0x7fc61806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:45.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.834+0000 7fc61fe00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc618146340 con 0x7fc6180cd0d0 2024-04-20T11:35:45.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.835+0000 7fc61fe00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc61806f3c0 con 0x7fc6180726f0 2024-04-20T11:35:45.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.835+0000 7fc61fe00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fc61806e9f0 con 0x7fc618145f60 2024-04-20T11:35:45.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.835+0000 7fc61d400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fc618145f60 0x7fc61806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:45.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.835+0000 7fc61d400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fc618145f60 0x7fc61806ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:60390/0 (socket says 172.21.15.152:60390) 2024-04-20T11:35:45.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.835+0000 7fc61d400640 1 -- 172.21.15.152:0/2350596438 learned_addr learned my addr 172.21.15.152:0/2350596438 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:45.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.835+0000 7fc61ca00640 1 --1- 172.21.15.152:0/2350596438 >> v1:172.21.15.152:6789/0 conn(0x7fc6180cd0d0 0x7fc618071fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:60534/0 (socket says 172.21.15.152:60534) 2024-04-20T11:35:45.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.835+0000 7fc61d400640 1 -- 172.21.15.152:0/2350596438 >> v1:172.21.15.175:6789/0 conn(0x7fc6180726f0 legacy=0x7fc6180cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:45.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.835+0000 7fc61d400640 1 -- 172.21.15.152:0/2350596438 >> v1:172.21.15.152:6789/0 conn(0x7fc6180cd0d0 legacy=0x7fc618071fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:45.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.835+0000 7fc61d400640 1 -- 172.21.15.152:0/2350596438 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc618072d50 con 0x7fc618145f60 2024-04-20T11:35:45.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.836+0000 7fc61d400640 1 --2- 172.21.15.152:0/2350596438 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc618145f60 0x7fc61806ee80 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7fc60c009870 tx=0x7fc60c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=613877f17fb4ec47 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:45.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.837+0000 7fc617e00640 1 -- 172.21.15.152:0/2350596438 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc60c034070 con 0x7fc618145f60 2024-04-20T11:35:45.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.837+0000 7fc617e00640 1 -- 172.21.15.152:0/2350596438 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc60c030860 con 0x7fc618145f60 2024-04-20T11:35:45.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.837+0000 7fc617e00640 1 -- 172.21.15.152:0/2350596438 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc60c030c30 con 0x7fc618145f60 2024-04-20T11:35:45.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.837+0000 7fc61fe00640 1 -- 172.21.15.152:0/2350596438 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc618145f60 msgr2=0x7fc61806ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:45.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.837+0000 7fc61fe00640 1 --2- 172.21.15.152:0/2350596438 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc618145f60 0x7fc61806ee80 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7fc60c009870 tx=0x7fc60c02b9a0 comp rx=0 tx=0).stop 2024-04-20T11:35:45.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.837+0000 7fc61fe00640 1 -- 172.21.15.152:0/2350596438 shutdown_connections 2024-04-20T11:35:45.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.837+0000 7fc61fe00640 1 --2- 172.21.15.152:0/2350596438 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc618145f60 0x7fc61806ee80 unknown :-1 s=CLOSED pgs=1852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:45.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.837+0000 7fc61fe00640 1 -- 172.21.15.152:0/2350596438 >> 172.21.15.152:0/2350596438 conn(0x7fc618067cf0 msgr2=0x7fc61806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:45.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.838+0000 7fc61fe00640 1 -- 172.21.15.152:0/2350596438 shutdown_connections 2024-04-20T11:35:45.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.838+0000 7fc61fe00640 1 -- 172.21.15.152:0/2350596438 wait complete. 2024-04-20T11:35:45.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.839+0000 7fc61fe00640 1 Processor -- start 2024-04-20T11:35:45.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.839+0000 7fc61fe00640 1 -- start start 2024-04-20T11:35:45.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.840+0000 7fc61fe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc6180726f0 0x7fc6181e5e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:45.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.840+0000 7fc61fe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc6180cd0d0 0x7fc6181f3e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:45.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.840+0000 7fc61fe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc618145f60 0x7fc6181f59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:45.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.840+0000 7fc61fe00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fc61806e680 con 0x7fc6180cd0d0 2024-04-20T11:35:45.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.840+0000 7fc61fe00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc61806e3a0 con 0x7fc6180726f0 2024-04-20T11:35:45.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.840+0000 7fc61fe00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc6180cd580 con 0x7fc618145f60 2024-04-20T11:35:45.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.840+0000 7fc61ca00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc6180cd0d0 0x7fc6181f3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:45.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.841+0000 7fc61ca00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc6180cd0d0 0x7fc6181f3e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:60398/0 (socket says 172.21.15.152:60398) 2024-04-20T11:35:45.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.841+0000 7fc61ca00640 1 -- 172.21.15.152:0/323557399 learned_addr learned my addr 172.21.15.152:0/323557399 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:45.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.841+0000 7fc61de00640 1 --2- 172.21.15.152:0/323557399 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc618145f60 0x7fc6181f59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:45.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.841+0000 7fc61d400640 1 --2- 172.21.15.152:0/323557399 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc6180726f0 0x7fc6181e5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:45.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.841+0000 7fc61ca00640 1 -- 172.21.15.152:0/323557399 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc618145f60 msgr2=0x7fc6181f59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:45.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.841+0000 7fc61ca00640 1 --2- 172.21.15.152:0/323557399 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc618145f60 0x7fc6181f59a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:45.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.841+0000 7fc61ca00640 1 -- 172.21.15.152:0/323557399 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc6180726f0 msgr2=0x7fc6181e5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:45.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.841+0000 7fc61ca00640 1 --2- 172.21.15.152:0/323557399 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc6180726f0 0x7fc6181e5e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:45.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.841+0000 7fc61ca00640 1 -- 172.21.15.152:0/323557399 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6181f4910 con 0x7fc6180cd0d0 2024-04-20T11:35:45.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.841+0000 7fc61ca00640 1 --2- 172.21.15.152:0/323557399 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc6180cd0d0 0x7fc6181f3e50 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7fc60400c400 tx=0x7fc60400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:45.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.842+0000 7fc615600640 1 -- 172.21.15.152:0/323557399 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6040074e0 con 0x7fc6180cd0d0 2024-04-20T11:35:45.843 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.842+0000 7fc615600640 1 -- 172.21.15.152:0/323557399 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc604007680 con 0x7fc6180cd0d0 2024-04-20T11:35:45.843 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.842+0000 7fc615600640 1 -- 172.21.15.152:0/323557399 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc604004a00 con 0x7fc6180cd0d0 2024-04-20T11:35:45.843 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.842+0000 7fc61fe00640 1 -- 172.21.15.152:0/323557399 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6181f4bf0 con 0x7fc6180cd0d0 2024-04-20T11:35:45.843 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.842+0000 7fc61fe00640 1 -- 172.21.15.152:0/323557399 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6181f51b0 con 0x7fc6180cd0d0 2024-04-20T11:35:45.843 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.843+0000 7fc61fe00640 1 -- 172.21.15.152:0/323557399 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6180689f0 con 0x7fc6180cd0d0 2024-04-20T11:35:45.843 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.844+0000 7fc615600640 1 -- 172.21.15.152:0/323557399 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 22) ==== 91601+0+0 (secure 0 0 0) 0x7fc60401a020 con 0x7fc6180cd0d0 2024-04-20T11:35:45.844 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.845+0000 7fc615600640 1 --2- 172.21.15.152:0/323557399 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fc5f806cf20 0x7fc5f806f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:45.844 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.845+0000 7fc615600640 1 -- 172.21.15.152:0/323557399 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5709+0+0 (secure 0 0 0) 0x7fc60408bea0 con 0x7fc6180cd0d0 2024-04-20T11:35:45.845 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.845+0000 7fc61d400640 1 --2- 172.21.15.152:0/323557399 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fc5f806cf20 0x7fc5f806f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:45.846 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.847+0000 7fc61d400640 1 --2- 172.21.15.152:0/323557399 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fc5f806cf20 0x7fc5f806f3e0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fc60c036000 tx=0x7fc60c03b040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:45.848 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:45.849+0000 7fc615600640 1 -- 172.21.15.152:0/323557399 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fc60408f030 con 0x7fc6180cd0d0 2024-04-20T11:35:46.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.054+0000 7fc61fe00640 1 -- 172.21.15.152:0/323557399 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc61815a340 con 0x7fc6180cd0d0 2024-04-20T11:35:46.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.055+0000 7fc615600640 1 -- 172.21.15.152:0/323557399 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v131) ==== 74+0+16739 (secure 0 0 0) 0x7fc604059a20 con 0x7fc6180cd0d0 2024-04-20T11:35:46.059 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:35:46.059 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":131,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:35:20.781260+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:35:16.963520+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"130","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-04-20T11:35:18.215890+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:35:46.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.058+0000 7fc61fe00640 1 -- 172.21.15.152:0/323557399 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fc5f806cf20 msgr2=0x7fc5f806f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:46.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.058+0000 7fc61fe00640 1 --2- 172.21.15.152:0/323557399 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fc5f806cf20 0x7fc5f806f3e0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fc60c036000 tx=0x7fc60c03b040 comp rx=0 tx=0).stop 2024-04-20T11:35:46.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.058+0000 7fc61fe00640 1 -- 172.21.15.152:0/323557399 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc6180cd0d0 msgr2=0x7fc6181f3e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:46.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.058+0000 7fc61fe00640 1 --2- 172.21.15.152:0/323557399 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc6180cd0d0 0x7fc6181f3e50 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7fc60400c400 tx=0x7fc60400c8c0 comp rx=0 tx=0).stop 2024-04-20T11:35:46.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.058+0000 7fc61fe00640 1 -- 172.21.15.152:0/323557399 shutdown_connections 2024-04-20T11:35:46.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.058+0000 7fc61fe00640 1 --2- 172.21.15.152:0/323557399 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fc5f806cf20 0x7fc5f806f3e0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:46.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.058+0000 7fc61fe00640 1 --2- 172.21.15.152:0/323557399 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc618145f60 0x7fc6181f59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:46.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.058+0000 7fc61fe00640 1 --2- 172.21.15.152:0/323557399 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc6180cd0d0 0x7fc6181f3e50 unknown :-1 s=CLOSED pgs=1853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:46.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.058+0000 7fc61fe00640 1 --2- 172.21.15.152:0/323557399 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc6180726f0 0x7fc6181e5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:46.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.058+0000 7fc61fe00640 1 -- 172.21.15.152:0/323557399 >> 172.21.15.152:0/323557399 conn(0x7fc618067cf0 msgr2=0x7fc6180affe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:46.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.058+0000 7fc61fe00640 1 -- 172.21.15.152:0/323557399 shutdown_connections 2024-04-20T11:35:46.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.058+0000 7fc61fe00640 1 -- 172.21.15.152:0/323557399 wait complete. 2024-04-20T11:35:46.078 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T11:26:47.246571+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 20, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T11:35:16.963520+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '130', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 21, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T11:35:18.215890+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '131', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T11:35:46.078 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T11:35:46.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.283+0000 7f4d9c600640 1 Processor -- start 2024-04-20T11:35:46.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.283+0000 7f4d9c600640 1 -- start start 2024-04-20T11:35:46.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.285+0000 7f4d9c600640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f4d9415e9e0 0x7f4d9406d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:46.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.285+0000 7f4d9c600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4d940d14a0 0x7f4d940d4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:46.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.285+0000 7f4d9c600640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d940d19a0 con 0x7f4d940d46a0 2024-04-20T11:35:46.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.285+0000 7f4d9c600640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f4d940d5040 con 0x7f4d9415e9e0 2024-04-20T11:35:46.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.285+0000 7f4d9c600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4d940d1c80 con 0x7f4d940d14a0 2024-04-20T11:35:46.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.285+0000 7f4d99c00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f4d9415e9e0 0x7f4d9406d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:46.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.285+0000 7f4d99200640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f4d940d46a0 0x7f4d940d0d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:60540/0 (socket says 172.21.15.152:60540) 2024-04-20T11:35:46.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.285+0000 7f4d99200640 1 -- 172.21.15.152:0/3124053385 learned_addr learned my addr 172.21.15.152:0/3124053385 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:35:46.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.286+0000 7f4d99c00640 1 -- 172.21.15.152:0/3124053385 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4d940d14a0 msgr2=0x7f4d940d4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:46.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.286+0000 7f4d99c00640 1 --2- 172.21.15.152:0/3124053385 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4d940d14a0 0x7f4d940d4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:46.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.286+0000 7f4d99c00640 1 -- 172.21.15.152:0/3124053385 >> v1:172.21.15.152:6789/0 conn(0x7f4d940d46a0 legacy=0x7f4d940d0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:46.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.286+0000 7f4d99c00640 1 -- 172.21.15.152:0/3124053385 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d940d1f60 con 0x7f4d9415e9e0 2024-04-20T11:35:46.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.287+0000 7f4d99c00640 1 --2- 172.21.15.152:0/3124053385 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d9415e9e0 0x7f4d9406d320 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f4d880097a0 tx=0x7f4d8802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d3458723a6718c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:46.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.287+0000 7f4d93e00640 1 -- 172.21.15.152:0/3124053385 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d88035020 con 0x7f4d9415e9e0 2024-04-20T11:35:46.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.287+0000 7f4d93e00640 1 -- 172.21.15.152:0/3124053385 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d88031810 con 0x7f4d9415e9e0 2024-04-20T11:35:46.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.287+0000 7f4d93e00640 1 -- 172.21.15.152:0/3124053385 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d88031be0 con 0x7f4d9415e9e0 2024-04-20T11:35:46.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.288+0000 7f4d9c600640 1 -- 172.21.15.152:0/3124053385 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d9415e9e0 msgr2=0x7f4d9406d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:46.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.288+0000 7f4d9c600640 1 --2- 172.21.15.152:0/3124053385 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d9415e9e0 0x7f4d9406d320 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f4d880097a0 tx=0x7f4d8802b8b0 comp rx=0 tx=0).stop 2024-04-20T11:35:46.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.288+0000 7f4d9c600640 1 -- 172.21.15.152:0/3124053385 shutdown_connections 2024-04-20T11:35:46.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.288+0000 7f4d9c600640 1 --2- 172.21.15.152:0/3124053385 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4d940d14a0 0x7f4d940d4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:46.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.288+0000 7f4d9c600640 1 --2- 172.21.15.152:0/3124053385 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d9415e9e0 0x7f4d9406d320 unknown :-1 s=CLOSED pgs=1855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:46.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.288+0000 7f4d9c600640 1 -- 172.21.15.152:0/3124053385 >> 172.21.15.152:0/3124053385 conn(0x7f4d9409dab0 msgr2=0x7f4d9408bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:46.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.288+0000 7f4d9c600640 1 -- 172.21.15.152:0/3124053385 shutdown_connections 2024-04-20T11:35:46.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.288+0000 7f4d9c600640 1 -- 172.21.15.152:0/3124053385 wait complete. 2024-04-20T11:35:46.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.290+0000 7f4d9c600640 1 Processor -- start 2024-04-20T11:35:46.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.290+0000 7f4d9c600640 1 -- start start 2024-04-20T11:35:46.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.291+0000 7f4d9c600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d940d14a0 0x7f4d94161900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:46.291 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.291+0000 7f4d9c600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d940d46a0 0x7f4d94152de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:46.291 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.291+0000 7f4d9c600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4d9415e9e0 0x7f4d9415ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:46.291 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.291+0000 7f4d9c600640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f4d9406d860 con 0x7f4d940d14a0 2024-04-20T11:35:46.291 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.291+0000 7f4d9c600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4d940cf880 con 0x7f4d940d46a0 2024-04-20T11:35:46.291 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.291+0000 7f4d9c600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4d9406d9e0 con 0x7f4d9415e9e0 2024-04-20T11:35:46.291 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.291+0000 7f4d99c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d940d14a0 0x7f4d94161900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:46.291 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.291+0000 7f4d99c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d940d14a0 0x7f4d94161900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:60418/0 (socket says 172.21.15.152:60418) 2024-04-20T11:35:46.291 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.291+0000 7f4d99c00640 1 -- 172.21.15.152:0/511470331 learned_addr learned my addr 172.21.15.152:0/511470331 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:46.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.292+0000 7f4d99200640 1 --2- 172.21.15.152:0/511470331 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d940d46a0 0x7f4d94152de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:46.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.292+0000 7f4d9a600640 1 --2- 172.21.15.152:0/511470331 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4d9415e9e0 0x7f4d9415ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:46.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.292+0000 7f4d99c00640 1 -- 172.21.15.152:0/511470331 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4d9415e9e0 msgr2=0x7f4d9415ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:46.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.292+0000 7f4d99c00640 1 --2- 172.21.15.152:0/511470331 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4d9415e9e0 0x7f4d9415ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:46.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.292+0000 7f4d99c00640 1 -- 172.21.15.152:0/511470331 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d940d46a0 msgr2=0x7f4d94152de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:46.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.292+0000 7f4d99c00640 1 --2- 172.21.15.152:0/511470331 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d940d46a0 0x7f4d94152de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:46.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.292+0000 7f4d99c00640 1 -- 172.21.15.152:0/511470331 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d94160860 con 0x7f4d940d14a0 2024-04-20T11:35:46.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.292+0000 7f4d99200640 1 --2- 172.21.15.152:0/511470331 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d940d46a0 0x7f4d94152de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:35:46.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.292+0000 7f4d9a600640 1 --2- 172.21.15.152:0/511470331 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4d9415e9e0 0x7f4d9415ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:35:46.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.292+0000 7f4d99c00640 1 --2- 172.21.15.152:0/511470331 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d940d14a0 0x7f4d94161900 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f4d880098c0 tx=0x7f4d880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:46.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.292+0000 7f4d91600640 1 -- 172.21.15.152:0/511470331 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d8803ad60 con 0x7f4d940d14a0 2024-04-20T11:35:46.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.292+0000 7f4d91600640 1 -- 172.21.15.152:0/511470331 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d88045450 con 0x7f4d940d14a0 2024-04-20T11:35:46.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.292+0000 7f4d9c600640 1 -- 172.21.15.152:0/511470331 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d94160b40 con 0x7f4d940d14a0 2024-04-20T11:35:46.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.292+0000 7f4d91600640 1 -- 172.21.15.152:0/511470331 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d88045840 con 0x7f4d940d14a0 2024-04-20T11:35:46.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.293+0000 7f4d9c600640 1 -- 172.21.15.152:0/511470331 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d94161100 con 0x7f4d940d14a0 2024-04-20T11:35:46.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.294+0000 7f4d9c600640 1 -- 172.21.15.152:0/511470331 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d940689f0 con 0x7f4d940d14a0 2024-04-20T11:35:46.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.295+0000 7f4d91600640 1 -- 172.21.15.152:0/511470331 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 22) ==== 91601+0+0 (secure 0 0 0) 0x7f4d88038020 con 0x7f4d940d14a0 2024-04-20T11:35:46.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.296+0000 7f4d91600640 1 --2- 172.21.15.152:0/511470331 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4d7806cf20 0x7f4d7806f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:46.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.296+0000 7f4d91600640 1 -- 172.21.15.152:0/511470331 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5709+0+0 (secure 0 0 0) 0x7f4d880ae9a0 con 0x7f4d940d14a0 2024-04-20T11:35:46.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.297+0000 7f4d99200640 1 --2- 172.21.15.152:0/511470331 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4d7806cf20 0x7f4d7806f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:46.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.298+0000 7f4d99200640 1 --2- 172.21.15.152:0/511470331 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4d7806cf20 0x7f4d7806f3e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f4d941f7e10 tx=0x7f4d84009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:46.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.300+0000 7f4d91600640 1 -- 172.21.15.152:0/511470331 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f4d8807c3d0 con 0x7f4d940d14a0 2024-04-20T11:35:46.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.500+0000 7f4d9c600640 1 -- 172.21.15.152:0/511470331 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4d940cb2f0 con 0x7f4d940d14a0 2024-04-20T11:35:46.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.501+0000 7f4d91600640 1 -- 172.21.15.152:0/511470331 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v131) ==== 93+0+10 (secure 0 0 0) 0x7f4d8807bcd0 con 0x7f4d940d14a0 2024-04-20T11:35:46.501 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 1 2024-04-20T11:35:46.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.504+0000 7f4d9c600640 1 -- 172.21.15.152:0/511470331 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4d7806cf20 msgr2=0x7f4d7806f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:46.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.504+0000 7f4d9c600640 1 --2- 172.21.15.152:0/511470331 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4d7806cf20 0x7f4d7806f3e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f4d941f7e10 tx=0x7f4d84009040 comp rx=0 tx=0).stop 2024-04-20T11:35:46.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.504+0000 7f4d9c600640 1 -- 172.21.15.152:0/511470331 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d940d14a0 msgr2=0x7f4d94161900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:46.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.504+0000 7f4d9c600640 1 --2- 172.21.15.152:0/511470331 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d940d14a0 0x7f4d94161900 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f4d880098c0 tx=0x7f4d880046c0 comp rx=0 tx=0).stop 2024-04-20T11:35:46.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.504+0000 7f4d9c600640 1 -- 172.21.15.152:0/511470331 shutdown_connections 2024-04-20T11:35:46.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.504+0000 7f4d9c600640 1 --2- 172.21.15.152:0/511470331 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4d7806cf20 0x7f4d7806f3e0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:46.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.504+0000 7f4d9c600640 1 --2- 172.21.15.152:0/511470331 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4d9415e9e0 0x7f4d9415ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:46.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.504+0000 7f4d9c600640 1 --2- 172.21.15.152:0/511470331 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d940d46a0 0x7f4d94152de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:46.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.504+0000 7f4d9c600640 1 --2- 172.21.15.152:0/511470331 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d940d14a0 0x7f4d94161900 unknown :-1 s=CLOSED pgs=1856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:46.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.504+0000 7f4d9c600640 1 -- 172.21.15.152:0/511470331 >> 172.21.15.152:0/511470331 conn(0x7f4d9409dab0 msgr2=0x7f4d940b15f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:46.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.504+0000 7f4d9c600640 1 -- 172.21.15.152:0/511470331 shutdown_connections 2024-04-20T11:35:46.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.504+0000 7f4d9c600640 1 -- 172.21.15.152:0/511470331 wait complete. 2024-04-20T11:35:46.522 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T11:35:46.726 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.726+0000 7fb733e00640 1 Processor -- start 2024-04-20T11:35:46.726 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.727+0000 7fb733e00640 1 -- start start 2024-04-20T11:35:46.727 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.728+0000 7fb733e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fb7340d35a0 0x7fb7341698d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:46.728 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.728+0000 7fb733e00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7340d19c0 con 0x7fb734163230 2024-04-20T11:35:46.728 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.728+0000 7fb733e00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb734162750 con 0x7fb73415eb50 2024-04-20T11:35:46.728 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.728+0000 7fb733e00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fb734162a30 con 0x7fb7340d35a0 2024-04-20T11:35:46.728 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.728+0000 7fb732000640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fb734163230 0x7fb7340d2e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:60554/0 (socket says 172.21.15.152:60554) 2024-04-20T11:35:46.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.728+0000 7fb732000640 1 -- 172.21.15.152:0/1323132788 learned_addr learned my addr 172.21.15.152:0/1323132788 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:35:46.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.728+0000 7fb733400640 1 --2- 172.21.15.152:0/1323132788 >> v2:172.21.15.152:3300/0 conn(0x7fb7340d35a0 0x7fb7341698d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:46.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.729+0000 7fb731600640 1 -- 172.21.15.152:0/1323132788 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3684465621 0 0) 0x7fb7340d19c0 con 0x7fb734163230 2024-04-20T11:35:46.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.729+0000 7fb731600640 1 -- 172.21.15.152:0/1323132788 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb714003520 con 0x7fb734163230 2024-04-20T11:35:46.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.729+0000 7fb733400640 1 -- 172.21.15.152:0/1323132788 >> v1:172.21.15.175:6789/0 conn(0x7fb73415eb50 legacy=0x7fb7340d0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:46.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.729+0000 7fb733400640 1 -- 172.21.15.152:0/1323132788 >> v1:172.21.15.152:6789/0 conn(0x7fb734163230 legacy=0x7fb7340d2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:46.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.729+0000 7fb733400640 1 -- 172.21.15.152:0/1323132788 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb734162d10 con 0x7fb7340d35a0 2024-04-20T11:35:46.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.730+0000 7fb733400640 1 --2- 172.21.15.152:0/1323132788 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb7340d35a0 0x7fb7341698d0 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7fb72c0097a0 tx=0x7fb72c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9531b2a90ecc846e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:46.730 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.730+0000 7fb731600640 1 -- 172.21.15.152:0/1323132788 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb72c0315a0 con 0x7fb7340d35a0 2024-04-20T11:35:46.730 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.730+0000 7fb731600640 1 -- 172.21.15.152:0/1323132788 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb72c037070 con 0x7fb7340d35a0 2024-04-20T11:35:46.730 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.730+0000 7fb731600640 1 -- 172.21.15.152:0/1323132788 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb72c031940 con 0x7fb7340d35a0 2024-04-20T11:35:46.731 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.731+0000 7fb733e00640 1 -- 172.21.15.152:0/1323132788 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb7340d35a0 msgr2=0x7fb7341698d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:46.731 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.731+0000 7fb733e00640 1 --2- 172.21.15.152:0/1323132788 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb7340d35a0 0x7fb7341698d0 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7fb72c0097a0 tx=0x7fb72c02b840 comp rx=0 tx=0).stop 2024-04-20T11:35:46.731 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.731+0000 7fb733e00640 1 -- 172.21.15.152:0/1323132788 shutdown_connections 2024-04-20T11:35:46.731 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.731+0000 7fb733e00640 1 --2- 172.21.15.152:0/1323132788 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb7340d35a0 0x7fb7341698d0 unknown :-1 s=CLOSED pgs=1858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:46.731 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.731+0000 7fb733e00640 1 -- 172.21.15.152:0/1323132788 >> 172.21.15.152:0/1323132788 conn(0x7fb73408bfd0 msgr2=0x7fb7340d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:46.732 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.731+0000 7fb733e00640 1 -- 172.21.15.152:0/1323132788 shutdown_connections 2024-04-20T11:35:46.732 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.731+0000 7fb733e00640 1 -- 172.21.15.152:0/1323132788 wait complete. 2024-04-20T11:35:46.732 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.733+0000 7fb733e00640 1 Processor -- start 2024-04-20T11:35:46.732 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.733+0000 7fb733e00640 1 -- start start 2024-04-20T11:35:46.735 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.734+0000 7fb733e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7340d35a0 0x7fb734159950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:46.735 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.734+0000 7fb733e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb73415eb50 0x7fb7340b8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:46.735 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.734+0000 7fb733e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb734163230 0x7fb7341568d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:46.735 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.734+0000 7fb733e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fb7340d1610 con 0x7fb73415eb50 2024-04-20T11:35:46.735 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.734+0000 7fb733e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb734169e10 con 0x7fb7340d35a0 2024-04-20T11:35:46.735 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.734+0000 7fb733e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb7340b6950 con 0x7fb734163230 2024-04-20T11:35:46.735 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.734+0000 7fb732000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb73415eb50 0x7fb7340b8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:46.735 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.734+0000 7fb732000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb73415eb50 0x7fb7340b8990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:60430/0 (socket says 172.21.15.152:60430) 2024-04-20T11:35:46.735 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.734+0000 7fb732000640 1 -- 172.21.15.152:0/2699811515 learned_addr learned my addr 172.21.15.152:0/2699811515 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:46.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.734+0000 7fb732a00640 1 --2- 172.21.15.152:0/2699811515 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7340d35a0 0x7fb734159950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:46.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.734+0000 7fb733400640 1 --2- 172.21.15.152:0/2699811515 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb734163230 0x7fb7341568d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:46.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.735+0000 7fb732000640 1 -- 172.21.15.152:0/2699811515 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb734163230 msgr2=0x7fb7341568d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:46.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.735+0000 7fb732000640 1 --2- 172.21.15.152:0/2699811515 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb734163230 0x7fb7341568d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:46.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.735+0000 7fb732000640 1 -- 172.21.15.152:0/2699811515 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7340d35a0 msgr2=0x7fb734159950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:46.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.735+0000 7fb732000640 1 --2- 172.21.15.152:0/2699811515 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7340d35a0 0x7fb734159950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:46.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.735+0000 7fb732000640 1 -- 172.21.15.152:0/2699811515 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7340b6060 con 0x7fb73415eb50 2024-04-20T11:35:46.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.735+0000 7fb732000640 1 --2- 172.21.15.152:0/2699811515 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb73415eb50 0x7fb7340b8990 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7fb71c00c770 tx=0x7fb71c00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:46.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.735+0000 7fb72aa00640 1 -- 172.21.15.152:0/2699811515 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb71c017070 con 0x7fb73415eb50 2024-04-20T11:35:46.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.735+0000 7fb72aa00640 1 -- 172.21.15.152:0/2699811515 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb71c00a5e0 con 0x7fb73415eb50 2024-04-20T11:35:46.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.735+0000 7fb72aa00640 1 -- 172.21.15.152:0/2699811515 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb71c00aa10 con 0x7fb73415eb50 2024-04-20T11:35:46.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.735+0000 7fb733e00640 1 -- 172.21.15.152:0/2699811515 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb734159470 con 0x7fb73415eb50 2024-04-20T11:35:46.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.735+0000 7fb733e00640 1 -- 172.21.15.152:0/2699811515 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7340b6300 con 0x7fb73415eb50 2024-04-20T11:35:46.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.736+0000 7fb733e00640 1 -- 172.21.15.152:0/2699811515 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb734075e60 con 0x7fb73415eb50 2024-04-20T11:35:46.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.737+0000 7fb72aa00640 1 -- 172.21.15.152:0/2699811515 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 22) ==== 91601+0+0 (secure 0 0 0) 0x7fb71c01a070 con 0x7fb73415eb50 2024-04-20T11:35:46.738 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.739+0000 7fb72aa00640 1 --2- 172.21.15.152:0/2699811515 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb71006cf50 0x7fb71006f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:46.738 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.739+0000 7fb72aa00640 1 -- 172.21.15.152:0/2699811515 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5709+0+0 (secure 0 0 0) 0x7fb71c08bd70 con 0x7fb73415eb50 2024-04-20T11:35:46.739 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.739+0000 7fb732a00640 1 --2- 172.21.15.152:0/2699811515 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb71006cf50 0x7fb71006f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:46.740 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.741+0000 7fb732a00640 1 --2- 172.21.15.152:0/2699811515 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb71006cf50 0x7fb71006f410 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb724003d90 tx=0x7fb724009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:46.743 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.743+0000 7fb72aa00640 1 -- 172.21.15.152:0/2699811515 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fb71c059820 con 0x7fb73415eb50 2024-04-20T11:35:46.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.941+0000 7fb733e00640 1 -- 172.21.15.152:0/2699811515 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb73408c680 con 0x7fb73415eb50 2024-04-20T11:35:46.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.942+0000 7fb72aa00640 1 -- 172.21.15.152:0/2699811515 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v131) ==== 104+0+11 (secure 0 0 0) 0x7fb71c059120 con 0x7fb73415eb50 2024-04-20T11:35:46.942 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 32 2024-04-20T11:35:46.945 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.945+0000 7fb733e00640 1 -- 172.21.15.152:0/2699811515 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb71006cf50 msgr2=0x7fb71006f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:46.945 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.945+0000 7fb733e00640 1 --2- 172.21.15.152:0/2699811515 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb71006cf50 0x7fb71006f410 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb724003d90 tx=0x7fb724009040 comp rx=0 tx=0).stop 2024-04-20T11:35:46.945 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.945+0000 7fb733e00640 1 -- 172.21.15.152:0/2699811515 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb73415eb50 msgr2=0x7fb7340b8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:46.945 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.945+0000 7fb733e00640 1 --2- 172.21.15.152:0/2699811515 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb73415eb50 0x7fb7340b8990 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7fb71c00c770 tx=0x7fb71c00cc30 comp rx=0 tx=0).stop 2024-04-20T11:35:46.945 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.945+0000 7fb733e00640 1 -- 172.21.15.152:0/2699811515 shutdown_connections 2024-04-20T11:35:46.945 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.945+0000 7fb733e00640 1 --2- 172.21.15.152:0/2699811515 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb71006cf50 0x7fb71006f410 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:46.945 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.945+0000 7fb733e00640 1 --2- 172.21.15.152:0/2699811515 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb734163230 0x7fb7341568d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:46.945 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.945+0000 7fb733e00640 1 --2- 172.21.15.152:0/2699811515 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb73415eb50 0x7fb7340b8990 unknown :-1 s=CLOSED pgs=1859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:46.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.945+0000 7fb733e00640 1 --2- 172.21.15.152:0/2699811515 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7340d35a0 0x7fb734159950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:46.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.945+0000 7fb733e00640 1 -- 172.21.15.152:0/2699811515 >> 172.21.15.152:0/2699811515 conn(0x7fb73408bfd0 msgr2=0x7fb7340d02c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:46.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.945+0000 7fb733e00640 1 -- 172.21.15.152:0/2699811515 shutdown_connections 2024-04-20T11:35:46.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:46.945+0000 7fb733e00640 1 -- 172.21.15.152:0/2699811515 wait complete. 2024-04-20T11:35:46.963 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T11:35:47.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.166+0000 7fa167200640 1 Processor -- start 2024-04-20T11:35:47.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.166+0000 7fa167200640 1 -- start start 2024-04-20T11:35:47.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.167+0000 7fa167200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa16806c4b0 0x7fa16815cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:47.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.167+0000 7fa167200640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa16815d5c0 con 0x7fa16815ca40 2024-04-20T11:35:47.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.167+0000 7fa167200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa16815d8a0 con 0x7fa168156c00 2024-04-20T11:35:47.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.167+0000 7fa167200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa168160290 con 0x7fa16806c4b0 2024-04-20T11:35:47.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.168+0000 7fa165400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fa16815ca40 0x7fa16806bda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:60566/0 (socket says 172.21.15.152:60566) 2024-04-20T11:35:47.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.168+0000 7fa165400640 1 -- 172.21.15.152:0/3744643538 learned_addr learned my addr 172.21.15.152:0/3744643538 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:35:47.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.168+0000 7fa166800640 1 --2- 172.21.15.152:0/3744643538 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa16806c4b0 0x7fa16815cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:47.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.168+0000 7fa164a00640 1 -- 172.21.15.152:0/3744643538 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 990335472 0 0) 0x7fa16815d5c0 con 0x7fa16815ca40 2024-04-20T11:35:47.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.168+0000 7fa164a00640 1 -- 172.21.15.152:0/3744643538 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa14c003520 con 0x7fa16815ca40 2024-04-20T11:35:47.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.169+0000 7fa164a00640 1 -- 172.21.15.152:0/3744643538 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 573241196 0 0) 0x7fa14c003520 con 0x7fa16815ca40 2024-04-20T11:35:47.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.169+0000 7fa164a00640 1 -- 172.21.15.152:0/3744643538 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa16806c4b0 msgr2=0x7fa16815cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:47.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.169+0000 7fa164a00640 1 --2- 172.21.15.152:0/3744643538 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa16806c4b0 0x7fa16815cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:47.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.169+0000 7fa164a00640 1 -- 172.21.15.152:0/3744643538 >> v1:172.21.15.175:6789/0 conn(0x7fa168156c00 legacy=0x7fa168160a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:47.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.169+0000 7fa164a00640 1 -- 172.21.15.152:0/3744643538 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1680ad2e0 con 0x7fa16815ca40 2024-04-20T11:35:47.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.169+0000 7fa164a00640 1 -- 172.21.15.152:0/3744643538 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7fa154001690 con 0x7fa16815ca40 2024-04-20T11:35:47.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.169+0000 7fa164a00640 1 -- 172.21.15.152:0/3744643538 >> v1:172.21.15.152:6789/0 conn(0x7fa16815ca40 legacy=0x7fa16806bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:47.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.169+0000 7fa164a00640 1 --2- 172.21.15.152:0/3744643538 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa14c0039e0 0x7fa14c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:47.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.169+0000 7fa164a00640 1 --2- 172.21.15.152:0/3744643538 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa16806c4b0 0x7fa14c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:47.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.169+0000 7fa164a00640 1 --2- 172.21.15.152:0/3744643538 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa14c004d40 0x7fa14c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:47.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.169+0000 7fa164a00640 1 -- 172.21.15.152:0/3744643538 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fa168160290 con 0x7fa14c0039e0 2024-04-20T11:35:47.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.169+0000 7fa164a00640 1 -- 172.21.15.152:0/3744643538 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa14c00baa0 con 0x7fa14c004d40 2024-04-20T11:35:47.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.169+0000 7fa164a00640 1 -- 172.21.15.152:0/3744643538 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa14c00bca0 con 0x7fa16806c4b0 2024-04-20T11:35:47.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.169+0000 7fa165e00640 1 --2- 172.21.15.152:0/3744643538 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa14c0039e0 0x7fa14c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:47.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.169+0000 7fa165e00640 1 -- 172.21.15.152:0/3744643538 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa16806c4b0 msgr2=0x7fa14c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:47.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.169+0000 7fa165e00640 1 --2- 172.21.15.152:0/3744643538 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa16806c4b0 0x7fa14c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:47.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.169+0000 7fa165e00640 1 -- 172.21.15.152:0/3744643538 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa14c004d40 msgr2=0x7fa14c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:47.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.169+0000 7fa165e00640 1 --2- 172.21.15.152:0/3744643538 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa14c004d40 0x7fa14c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:47.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.169+0000 7fa165e00640 1 -- 172.21.15.152:0/3744643538 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa14c003520 con 0x7fa14c0039e0 2024-04-20T11:35:47.172 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.170+0000 7fa165e00640 1 --2- 172.21.15.152:0/3744643538 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa14c0039e0 0x7fa14c003e00 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7fa158001720 tx=0x7fa14c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=199095e960822ddf server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:47.172 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.171+0000 7fa164a00640 1 -- 172.21.15.152:0/3744643538 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa158001370 con 0x7fa14c0039e0 2024-04-20T11:35:47.172 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.171+0000 7fa164a00640 1 -- 172.21.15.152:0/3744643538 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1580052d0 con 0x7fa14c0039e0 2024-04-20T11:35:47.172 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.171+0000 7fa164a00640 1 -- 172.21.15.152:0/3744643538 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa158003830 con 0x7fa14c0039e0 2024-04-20T11:35:47.172 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.171+0000 7fa167200640 1 -- 172.21.15.152:0/3744643538 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa14c0039e0 msgr2=0x7fa14c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:47.172 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.171+0000 7fa167200640 1 --2- 172.21.15.152:0/3744643538 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa14c0039e0 0x7fa14c003e00 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7fa158001720 tx=0x7fa14c005830 comp rx=0 tx=0).stop 2024-04-20T11:35:47.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.171+0000 7fa166800640 1 -- 172.21.15.152:0/3744643538 reap_dead start 2024-04-20T11:35:47.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.171+0000 7fa167200640 1 -- 172.21.15.152:0/3744643538 shutdown_connections 2024-04-20T11:35:47.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.171+0000 7fa167200640 1 -- 172.21.15.152:0/3744643538 >> 172.21.15.152:0/3744643538 conn(0x7fa1680b21e0 msgr2=0x7fa1680b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:47.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.171+0000 7fa167200640 1 -- 172.21.15.152:0/3744643538 shutdown_connections 2024-04-20T11:35:47.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.171+0000 7fa167200640 1 -- 172.21.15.152:0/3744643538 wait complete. 2024-04-20T11:35:47.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.173+0000 7fa167200640 1 Processor -- start 2024-04-20T11:35:47.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.173+0000 7fa167200640 1 -- start start 2024-04-20T11:35:47.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.174+0000 7fa167200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa14c0039e0 0x7fa16809a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:47.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.174+0000 7fa167200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa1681f50f0 0x7fa16809c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:47.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.174+0000 7fa167200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa1680d04b0 0x7fa1680d0910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:47.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.174+0000 7fa167200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fa16815d2d0 con 0x7fa14c0039e0 2024-04-20T11:35:47.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.174+0000 7fa167200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa16815d150 con 0x7fa1681f50f0 2024-04-20T11:35:47.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.174+0000 7fa167200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa1680b30c0 con 0x7fa1680d04b0 2024-04-20T11:35:47.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.174+0000 7fa165e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa14c0039e0 0x7fa16809a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:47.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.174+0000 7fa165e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa14c0039e0 0x7fa16809a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:60448/0 (socket says 172.21.15.152:60448) 2024-04-20T11:35:47.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.175+0000 7fa165e00640 1 -- 172.21.15.152:0/1651894218 learned_addr learned my addr 172.21.15.152:0/1651894218 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:47.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.175+0000 7fa165400640 1 --2- 172.21.15.152:0/1651894218 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa1681f50f0 0x7fa16809c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:47.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.175+0000 7fa165e00640 1 -- 172.21.15.152:0/1651894218 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa1680d04b0 msgr2=0x7fa1680d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:47.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.175+0000 7fa166800640 1 --2- 172.21.15.152:0/1651894218 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa1680d04b0 0x7fa1680d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:47.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.175+0000 7fa165e00640 1 --2- 172.21.15.152:0/1651894218 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa1680d04b0 0x7fa1680d0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:47.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.175+0000 7fa165e00640 1 -- 172.21.15.152:0/1651894218 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa1681f50f0 msgr2=0x7fa16809c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:47.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.175+0000 7fa165e00640 1 --2- 172.21.15.152:0/1651894218 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa1681f50f0 0x7fa16809c5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:47.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.175+0000 7fa165e00640 1 -- 172.21.15.152:0/1651894218 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa16809a790 con 0x7fa14c0039e0 2024-04-20T11:35:47.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.175+0000 7fa165e00640 1 --2- 172.21.15.152:0/1651894218 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa14c0039e0 0x7fa16809a1b0 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7fa158008fd0 tx=0x7fa158001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:47.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.175+0000 7fa15e000640 1 -- 172.21.15.152:0/1651894218 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa158041070 con 0x7fa14c0039e0 2024-04-20T11:35:47.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.176+0000 7fa15e000640 1 -- 172.21.15.152:0/1651894218 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa158005480 con 0x7fa14c0039e0 2024-04-20T11:35:47.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.176+0000 7fa15e000640 1 -- 172.21.15.152:0/1651894218 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa158007a30 con 0x7fa14c0039e0 2024-04-20T11:35:47.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.176+0000 7fa167200640 1 -- 172.21.15.152:0/1651894218 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa16809aa70 con 0x7fa14c0039e0 2024-04-20T11:35:47.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.176+0000 7fa167200640 1 -- 172.21.15.152:0/1651894218 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa16809b030 con 0x7fa14c0039e0 2024-04-20T11:35:47.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.177+0000 7fa167200640 1 -- 172.21.15.152:0/1651894218 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa168075e60 con 0x7fa14c0039e0 2024-04-20T11:35:47.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.178+0000 7fa15e000640 1 -- 172.21.15.152:0/1651894218 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 22) ==== 91601+0+0 (secure 0 0 0) 0x7fa158044070 con 0x7fa14c0039e0 2024-04-20T11:35:47.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.179+0000 7fa15e000640 1 --2- 172.21.15.152:0/1651894218 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fa14806cf50 0x7fa14806f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:47.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.179+0000 7fa15e000640 1 -- 172.21.15.152:0/1651894218 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5709+0+0 (secure 0 0 0) 0x7fa1580ae190 con 0x7fa14c0039e0 2024-04-20T11:35:47.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.179+0000 7fa165400640 1 --2- 172.21.15.152:0/1651894218 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fa14806cf50 0x7fa14806f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:47.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.181+0000 7fa165400640 1 --2- 172.21.15.152:0/1651894218 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fa14806cf50 0x7fa14806f410 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fa154001880 tx=0x7fa154009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:47.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.183+0000 7fa15e000640 1 -- 172.21.15.152:0/1651894218 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fa15807bd30 con 0x7fa14c0039e0 2024-04-20T11:35:47.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.381+0000 7fa167200640 1 -- 172.21.15.152:0/1651894218 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa16806f390 con 0x7fa14c0039e0 2024-04-20T11:35:47.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.382+0000 7fa15e000640 1 -- 172.21.15.152:0/1651894218 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v131) ==== 100+0+11 (secure 0 0 0) 0x7fa15807b630 con 0x7fa14c0039e0 2024-04-20T11:35:47.382 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 64 2024-04-20T11:35:47.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.385+0000 7fa167200640 1 -- 172.21.15.152:0/1651894218 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fa14806cf50 msgr2=0x7fa14806f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:47.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.385+0000 7fa167200640 1 --2- 172.21.15.152:0/1651894218 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fa14806cf50 0x7fa14806f410 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fa154001880 tx=0x7fa154009040 comp rx=0 tx=0).stop 2024-04-20T11:35:47.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.385+0000 7fa167200640 1 -- 172.21.15.152:0/1651894218 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa14c0039e0 msgr2=0x7fa16809a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:47.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.385+0000 7fa167200640 1 --2- 172.21.15.152:0/1651894218 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa14c0039e0 0x7fa16809a1b0 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7fa158008fd0 tx=0x7fa158001120 comp rx=0 tx=0).stop 2024-04-20T11:35:47.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.385+0000 7fa167200640 1 -- 172.21.15.152:0/1651894218 shutdown_connections 2024-04-20T11:35:47.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.385+0000 7fa167200640 1 --2- 172.21.15.152:0/1651894218 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fa14806cf50 0x7fa14806f410 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:47.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.385+0000 7fa167200640 1 --2- 172.21.15.152:0/1651894218 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa1680d04b0 0x7fa1680d0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:47.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.385+0000 7fa167200640 1 --2- 172.21.15.152:0/1651894218 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa1681f50f0 0x7fa16809c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:47.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.385+0000 7fa167200640 1 --2- 172.21.15.152:0/1651894218 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa14c0039e0 0x7fa16809a1b0 unknown :-1 s=CLOSED pgs=1862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:47.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.385+0000 7fa167200640 1 -- 172.21.15.152:0/1651894218 >> 172.21.15.152:0/1651894218 conn(0x7fa1680b21e0 msgr2=0x7fa168072380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:47.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.385+0000 7fa167200640 1 -- 172.21.15.152:0/1651894218 shutdown_connections 2024-04-20T11:35:47.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.385+0000 7fa167200640 1 -- 172.21.15.152:0/1651894218 wait complete. 2024-04-20T11:35:47.403 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-20T11:35:47.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.600+0000 7f0317e00640 1 Processor -- start 2024-04-20T11:35:47.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.601+0000 7f0317e00640 1 -- start start 2024-04-20T11:35:47.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.602+0000 7f0317e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0318161d30 0x7f03180d0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:47.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.602+0000 7f0317e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f03180d2b80 0x7f03180d3030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:47.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.602+0000 7f0317e00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03180d26a0 con 0x7f03180d4ec0 2024-04-20T11:35:47.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.602+0000 7f0317e00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f03180d0540 con 0x7f03180d2b80 2024-04-20T11:35:47.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.602+0000 7f0317e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f03180cfb70 con 0x7f0318161d30 2024-04-20T11:35:47.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.602+0000 7f0316000640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f03180d4ec0 0x7f03180d15e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:60580/0 (socket says 172.21.15.152:60580) 2024-04-20T11:35:47.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.602+0000 7f0316000640 1 -- 172.21.15.152:0/4233416926 learned_addr learned my addr 172.21.15.152:0/4233416926 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:35:47.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.603+0000 7f0317400640 1 --2- 172.21.15.152:0/4233416926 >> v2:172.21.15.175:3300/0 conn(0x7f03180d2b80 0x7f03180d3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:47.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.603+0000 7f0315600640 1 -- 172.21.15.152:0/4233416926 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3469974791 0 0) 0x7f03180d26a0 con 0x7f03180d4ec0 2024-04-20T11:35:47.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.603+0000 7f0316a00640 1 --2- 172.21.15.152:0/4233416926 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0318161d30 0x7f03180d0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:47.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.603+0000 7f0315600640 1 -- 172.21.15.152:0/4233416926 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f02f8003520 con 0x7f03180d4ec0 2024-04-20T11:35:47.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.603+0000 7f0315600640 1 -- 172.21.15.152:0/4233416926 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 177207785 0 0) 0x7f02f8003520 con 0x7f03180d4ec0 2024-04-20T11:35:47.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.603+0000 7f0315600640 1 -- 172.21.15.152:0/4233416926 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0318161d30 msgr2=0x7f03180d0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:47.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.603+0000 7f0315600640 1 --2- 172.21.15.152:0/4233416926 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0318161d30 0x7f03180d0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:47.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.603+0000 7f0315600640 1 -- 172.21.15.152:0/4233416926 >> v2:172.21.15.175:3300/0 conn(0x7f03180d2b80 msgr2=0x7f03180d3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:47.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.603+0000 7f0315600640 1 --2- 172.21.15.152:0/4233416926 >> v2:172.21.15.175:3300/0 conn(0x7f03180d2b80 0x7f03180d3030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:47.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.603+0000 7f0315600640 1 -- 172.21.15.152:0/4233416926 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03180d53c0 con 0x7f03180d4ec0 2024-04-20T11:35:47.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.603+0000 7f0315600640 1 -- 172.21.15.152:0/4233416926 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f03000017f0 con 0x7f03180d4ec0 2024-04-20T11:35:47.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.603+0000 7f0317400640 1 --2- 172.21.15.152:0/4233416926 >> v2:172.21.15.175:3300/0 conn(0x7f03180d2b80 0x7f03180d3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:35:47.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.603+0000 7f0315600640 1 -- 172.21.15.152:0/4233416926 >> v1:172.21.15.152:6789/0 conn(0x7f03180d4ec0 legacy=0x7f03180d15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:47.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.603+0000 7f0315600640 1 --2- 172.21.15.152:0/4233416926 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02f80039b0 0x7f02f8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:47.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.603+0000 7f0315600640 1 --2- 172.21.15.152:0/4233416926 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f02f8004800 0x7f02f8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:47.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.603+0000 7f0315600640 1 --2- 172.21.15.152:0/4233416926 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0318161d30 0x7f02f80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:47.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.604+0000 7f0315600640 1 -- 172.21.15.152:0/4233416926 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f03180d0540 con 0x7f02f8004800 2024-04-20T11:35:47.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.604+0000 7f0315600640 1 -- 172.21.15.152:0/4233416926 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f03180cfb70 con 0x7f02f80039b0 2024-04-20T11:35:47.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.604+0000 7f0315600640 1 -- 172.21.15.152:0/4233416926 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f02f800ba70 con 0x7f0318161d30 2024-04-20T11:35:47.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.604+0000 7f0316000640 1 --2- 172.21.15.152:0/4233416926 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f02f8004800 0x7f02f8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:47.607 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.604+0000 7f0316a00640 1 --2- 172.21.15.152:0/4233416926 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02f80039b0 0x7f02f8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:47.607 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.604+0000 7f0317400640 1 --2- 172.21.15.152:0/4233416926 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0318161d30 0x7f02f80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:47.607 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.604+0000 7f0316000640 1 -- 172.21.15.152:0/4233416926 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0318161d30 msgr2=0x7f02f80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:47.607 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.604+0000 7f0316000640 1 --2- 172.21.15.152:0/4233416926 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0318161d30 0x7f02f80051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:47.607 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.604+0000 7f0316000640 1 -- 172.21.15.152:0/4233416926 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02f80039b0 msgr2=0x7f02f8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:47.607 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.604+0000 7f0316000640 1 --2- 172.21.15.152:0/4233416926 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02f80039b0 0x7f02f8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:47.607 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.604+0000 7f0316000640 1 -- 172.21.15.152:0/4233416926 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02f8003520 con 0x7f02f8004800 2024-04-20T11:35:47.607 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.604+0000 7f0316a00640 1 --2- 172.21.15.152:0/4233416926 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02f80039b0 0x7f02f8003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:35:47.607 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.605+0000 7f0316000640 1 --2- 172.21.15.152:0/4233416926 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f02f8004800 0x7f02f8004c80 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f03180d1cf0 tx=0x7f02f8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2708ee36dab209c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:47.607 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.606+0000 7f0315600640 1 -- 172.21.15.152:0/4233416926 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0300007d30 con 0x7f02f8004800 2024-04-20T11:35:47.607 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.606+0000 7f0315600640 1 -- 172.21.15.152:0/4233416926 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0300030070 con 0x7f02f8004800 2024-04-20T11:35:47.607 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.606+0000 7f0315600640 1 -- 172.21.15.152:0/4233416926 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03000074f0 con 0x7f02f8004800 2024-04-20T11:35:47.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.606+0000 7f0317e00640 1 -- 172.21.15.152:0/4233416926 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f02f8004800 msgr2=0x7f02f8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:47.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.606+0000 7f0317e00640 1 --2- 172.21.15.152:0/4233416926 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f02f8004800 0x7f02f8004c80 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f03180d1cf0 tx=0x7f02f8005830 comp rx=0 tx=0).stop 2024-04-20T11:35:47.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.606+0000 7f0317400640 1 -- 172.21.15.152:0/4233416926 reap_dead start 2024-04-20T11:35:47.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.606+0000 7f0317e00640 1 -- 172.21.15.152:0/4233416926 shutdown_connections 2024-04-20T11:35:47.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.606+0000 7f0317e00640 1 -- 172.21.15.152:0/4233416926 >> 172.21.15.152:0/4233416926 conn(0x7f031808bfd0 msgr2=0x7f03180d45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:47.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.606+0000 7f0317e00640 1 -- 172.21.15.152:0/4233416926 shutdown_connections 2024-04-20T11:35:47.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.606+0000 7f0317e00640 1 -- 172.21.15.152:0/4233416926 wait complete. 2024-04-20T11:35:47.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.608+0000 7f0317e00640 1 Processor -- start 2024-04-20T11:35:47.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.608+0000 7f0317e00640 1 -- start start 2024-04-20T11:35:47.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.609+0000 7f0317e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02f8004800 0x7f031815bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:47.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.609+0000 7f0317e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f031815e420 0x7f03181545f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:47.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.609+0000 7f0317e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f031815c500 0x7f031815c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:47.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.609+0000 7f0317e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f03180cf1b0 con 0x7f031815e420 2024-04-20T11:35:47.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.609+0000 7f0317e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f03180cf800 con 0x7f02f8004800 2024-04-20T11:35:47.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.609+0000 7f0317e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0318162110 con 0x7f031815c500 2024-04-20T11:35:47.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.609+0000 7f0316000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f031815e420 0x7f03181545f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:47.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.609+0000 7f0316000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f031815e420 0x7f03181545f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:60472/0 (socket says 172.21.15.152:60472) 2024-04-20T11:35:47.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.609+0000 7f0316000640 1 -- 172.21.15.152:0/620350626 learned_addr learned my addr 172.21.15.152:0/620350626 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:47.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.610+0000 7f0316000640 1 -- 172.21.15.152:0/620350626 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f031815c500 msgr2=0x7f031815c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:47.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.610+0000 7f0316a00640 1 --2- 172.21.15.152:0/620350626 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02f8004800 0x7f031815bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:47.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.610+0000 7f0316000640 1 --2- 172.21.15.152:0/620350626 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f031815c500 0x7f031815c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:47.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.610+0000 7f0316000640 1 -- 172.21.15.152:0/620350626 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02f8004800 msgr2=0x7f031815bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:47.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.610+0000 7f0316000640 1 --2- 172.21.15.152:0/620350626 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02f8004800 0x7f031815bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:47.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.610+0000 7f0316000640 1 -- 172.21.15.152:0/620350626 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03180b8090 con 0x7f031815e420 2024-04-20T11:35:47.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.610+0000 7f0316000640 1 --2- 172.21.15.152:0/620350626 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f031815e420 0x7f03181545f0 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f030000c300 tx=0x7f031808d840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:47.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.611+0000 7f030f400640 1 -- 172.21.15.152:0/620350626 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0300000f70 con 0x7f031815e420 2024-04-20T11:35:47.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.611+0000 7f030f400640 1 -- 172.21.15.152:0/620350626 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03000074a0 con 0x7f031815e420 2024-04-20T11:35:47.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.611+0000 7f030f400640 1 -- 172.21.15.152:0/620350626 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03000059e0 con 0x7f031815e420 2024-04-20T11:35:47.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.611+0000 7f0317e00640 1 -- 172.21.15.152:0/620350626 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f031815e9b0 con 0x7f031815e420 2024-04-20T11:35:47.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.611+0000 7f0317e00640 1 -- 172.21.15.152:0/620350626 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03181584a0 con 0x7f031815e420 2024-04-20T11:35:47.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.612+0000 7f0317e00640 1 -- 172.21.15.152:0/620350626 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0318075e60 con 0x7f031815e420 2024-04-20T11:35:47.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.613+0000 7f030f400640 1 -- 172.21.15.152:0/620350626 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 22) ==== 91601+0+0 (secure 0 0 0) 0x7f0300005b80 con 0x7f031815e420 2024-04-20T11:35:47.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.614+0000 7f030f400640 1 --2- 172.21.15.152:0/620350626 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f02f406cf20 0x7f02f406f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:47.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.614+0000 7f030f400640 1 -- 172.21.15.152:0/620350626 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5709+0+0 (secure 0 0 0) 0x7f03000ae1e0 con 0x7f031815e420 2024-04-20T11:35:47.614 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.615+0000 7f0316a00640 1 --2- 172.21.15.152:0/620350626 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f02f406cf20 0x7f02f406f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:47.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.616+0000 7f0316a00640 1 --2- 172.21.15.152:0/620350626 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f02f406cf20 0x7f02f406f3e0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f0308006fd0 tx=0x7f0308008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:47.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.618+0000 7f030f400640 1 -- 172.21.15.152:0/620350626 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f03000343d0 con 0x7f031815e420 2024-04-20T11:35:47.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.829+0000 7f0317e00640 1 -- 172.21.15.152:0/620350626 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f03180caf00 con 0x7f031815e420 2024-04-20T11:35:47.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.829+0000 7f030f400640 1 -- 172.21.15.152:0/620350626 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v131) ==== 80+0+0 (secure 0 0 0) 0x7f030007bc90 con 0x7f031815e420 2024-04-20T11:35:47.829 INFO:teuthology.orchestra.run.smithi152.stderr:listed 0 entries 2024-04-20T11:35:47.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.832+0000 7f0317e00640 1 -- 172.21.15.152:0/620350626 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f02f406cf20 msgr2=0x7f02f406f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:47.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.832+0000 7f0317e00640 1 --2- 172.21.15.152:0/620350626 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f02f406cf20 0x7f02f406f3e0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f0308006fd0 tx=0x7f0308008040 comp rx=0 tx=0).stop 2024-04-20T11:35:47.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.832+0000 7f0317e00640 1 -- 172.21.15.152:0/620350626 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f031815e420 msgr2=0x7f03181545f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:47.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.832+0000 7f0317e00640 1 --2- 172.21.15.152:0/620350626 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f031815e420 0x7f03181545f0 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f030000c300 tx=0x7f031808d840 comp rx=0 tx=0).stop 2024-04-20T11:35:47.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.832+0000 7f0317e00640 1 -- 172.21.15.152:0/620350626 shutdown_connections 2024-04-20T11:35:47.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.832+0000 7f0317e00640 1 --2- 172.21.15.152:0/620350626 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f02f406cf20 0x7f02f406f3e0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:47.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.832+0000 7f0317e00640 1 --2- 172.21.15.152:0/620350626 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f031815c500 0x7f031815c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:47.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.832+0000 7f0317e00640 1 --2- 172.21.15.152:0/620350626 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f031815e420 0x7f03181545f0 unknown :-1 s=CLOSED pgs=1865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:47.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.832+0000 7f0317e00640 1 --2- 172.21.15.152:0/620350626 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02f8004800 0x7f031815bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:47.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.832+0000 7f0317e00640 1 -- 172.21.15.152:0/620350626 >> 172.21.15.152:0/620350626 conn(0x7f031808bfd0 msgr2=0x7f031806d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:47.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.832+0000 7f0317e00640 1 -- 172.21.15.152:0/620350626 shutdown_connections 2024-04-20T11:35:47.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:47.832+0000 7f0317e00640 1 -- 172.21.15.152:0/620350626 wait complete. 2024-04-20T11:35:47.850 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi152.front.sepia.ceph.com... 2024-04-20T11:35:47.850 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:35:47.850 DEBUG:teuthology.orchestra.run.smithi152:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-20T11:35:47.925 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi152.stderr:ceph-fuse[64036]: fuse finished with error 0 and tester_r 0 2024-04-20T11:35:47.931 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi152.front.sepia.ceph.com 2024-04-20T11:35:47.931 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:35:47.931 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-20T11:35:47.987 INFO:tasks.cephfs.mount:Terminating background process 2024-04-20T11:35:47.987 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T11:35:47.987 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:35:48.046 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:35:48.257 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.257+0000 7fea75e00640 1 Processor -- start 2024-04-20T11:35:48.258 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.258+0000 7fea75e00640 1 -- start start 2024-04-20T11:35:48.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.259+0000 7fea75e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fea7015d1e0 0x7fea70160b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:48.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.259+0000 7fea75e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fea7015d5c0 0x7fea700cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:48.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.259+0000 7fea75e00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea700ce8c0 con 0x7fea70071cd0 2024-04-20T11:35:48.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.259+0000 7fea75e00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fea701600b0 con 0x7fea7015d1e0 2024-04-20T11:35:48.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.259+0000 7fea75e00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fea70160390 con 0x7fea7015d5c0 2024-04-20T11:35:48.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.260+0000 7fea74a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fea7015d1e0 0x7fea70160b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:48.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.260+0000 7fea74a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fea7015d1e0 0x7fea70160b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:60482/0 (socket says 172.21.15.152:60482) 2024-04-20T11:35:48.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.260+0000 7fea74a00640 1 -- 172.21.15.152:0/2089080077 learned_addr learned my addr 172.21.15.152:0/2089080077 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:48.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.260+0000 7fea67e00640 1 --2- 172.21.15.152:0/2089080077 >> v2:172.21.15.175:3300/0 conn(0x7fea7015d5c0 0x7fea700cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:48.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.260+0000 7fea74a00640 1 -- 172.21.15.152:0/2089080077 >> v2:172.21.15.175:3300/0 conn(0x7fea7015d5c0 msgr2=0x7fea700cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:48.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.260+0000 7fea74a00640 1 --2- 172.21.15.152:0/2089080077 >> v2:172.21.15.175:3300/0 conn(0x7fea7015d5c0 0x7fea700cdfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:48.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.260+0000 7fea74a00640 1 -- 172.21.15.152:0/2089080077 >> v1:172.21.15.175:6789/0 conn(0x7fea70071cd0 legacy=0x7fea70072120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:48.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.260+0000 7fea74a00640 1 -- 172.21.15.152:0/2089080077 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea70160670 con 0x7fea7015d1e0 2024-04-20T11:35:48.262 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.261+0000 7fea74a00640 1 --2- 172.21.15.152:0/2089080077 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fea7015d1e0 0x7fea70160b90 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7fea60009870 tx=0x7fea6002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecc96cfc85027b82 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:48.262 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.262+0000 7fea6fe00640 1 -- 172.21.15.152:0/2089080077 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea60035020 con 0x7fea7015d1e0 2024-04-20T11:35:48.262 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.262+0000 7fea6fe00640 1 -- 172.21.15.152:0/2089080077 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea60031870 con 0x7fea7015d1e0 2024-04-20T11:35:48.262 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.262+0000 7fea6fe00640 1 -- 172.21.15.152:0/2089080077 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea60031c40 con 0x7fea7015d1e0 2024-04-20T11:35:48.263 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.262+0000 7fea75e00640 1 -- 172.21.15.152:0/2089080077 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fea7015d1e0 msgr2=0x7fea70160b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:48.263 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.262+0000 7fea75e00640 1 --2- 172.21.15.152:0/2089080077 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fea7015d1e0 0x7fea70160b90 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7fea60009870 tx=0x7fea6002b910 comp rx=0 tx=0).stop 2024-04-20T11:35:48.263 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.262+0000 7fea75e00640 1 -- 172.21.15.152:0/2089080077 shutdown_connections 2024-04-20T11:35:48.263 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.262+0000 7fea75e00640 1 --2- 172.21.15.152:0/2089080077 >> v2:172.21.15.175:3300/0 conn(0x7fea7015d5c0 0x7fea700cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:48.263 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.262+0000 7fea75e00640 1 --2- 172.21.15.152:0/2089080077 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fea7015d1e0 0x7fea70160b90 unknown :-1 s=CLOSED pgs=1866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:48.263 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.262+0000 7fea75e00640 1 -- 172.21.15.152:0/2089080077 >> 172.21.15.152:0/2089080077 conn(0x7fea70067ae0 msgr2=0x7fea7006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:48.263 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.262+0000 7fea75e00640 1 -- 172.21.15.152:0/2089080077 shutdown_connections 2024-04-20T11:35:48.263 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.263+0000 7fea75e00640 1 -- 172.21.15.152:0/2089080077 wait complete. 2024-04-20T11:35:48.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.265+0000 7fea75e00640 1 Processor -- start 2024-04-20T11:35:48.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.265+0000 7fea75e00640 1 -- start start 2024-04-20T11:35:48.266 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.266+0000 7fea75e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fea70071cd0 0x7fea701f79d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:48.266 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.266+0000 7fea75e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fea7015d1e0 0x7fea7015e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:48.266 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.266+0000 7fea75e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fea7015d5c0 0x7fea700b4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:48.266 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.266+0000 7fea75e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fea700ce6a0 con 0x7fea70071cd0 2024-04-20T11:35:48.266 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.266+0000 7fea75e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fea700ce520 con 0x7fea7015d5c0 2024-04-20T11:35:48.266 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.266+0000 7fea75e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fea700ccd70 con 0x7fea7015d1e0 2024-04-20T11:35:48.266 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.266+0000 7fea74a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fea70071cd0 0x7fea701f79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:48.266 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.266+0000 7fea74a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fea70071cd0 0x7fea701f79d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:60492/0 (socket says 172.21.15.152:60492) 2024-04-20T11:35:48.266 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.266+0000 7fea74a00640 1 -- 172.21.15.152:0/823715827 learned_addr learned my addr 172.21.15.152:0/823715827 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:48.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.267+0000 7fea67e00640 1 --2- 172.21.15.152:0/823715827 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fea7015d1e0 0x7fea7015e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:48.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.267+0000 7fea75400640 1 --2- 172.21.15.152:0/823715827 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fea7015d5c0 0x7fea700b4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:48.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.267+0000 7fea74a00640 1 -- 172.21.15.152:0/823715827 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fea7015d1e0 msgr2=0x7fea7015e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:48.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.267+0000 7fea74a00640 1 --2- 172.21.15.152:0/823715827 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fea7015d1e0 0x7fea7015e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:48.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.267+0000 7fea74a00640 1 -- 172.21.15.152:0/823715827 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fea7015d5c0 msgr2=0x7fea700b4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:48.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.267+0000 7fea74a00640 1 --2- 172.21.15.152:0/823715827 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fea7015d5c0 0x7fea700b4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:48.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.267+0000 7fea74a00640 1 -- 172.21.15.152:0/823715827 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea7015ed90 con 0x7fea70071cd0 2024-04-20T11:35:48.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.267+0000 7fea67e00640 1 --2- 172.21.15.152:0/823715827 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fea7015d1e0 0x7fea7015e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:35:48.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.267+0000 7fea75400640 1 --2- 172.21.15.152:0/823715827 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fea7015d5c0 0x7fea700b4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:35:48.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.267+0000 7fea74a00640 1 --2- 172.21.15.152:0/823715827 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fea70071cd0 0x7fea701f79d0 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7fea60000c00 tx=0x7fea600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:48.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.268+0000 7fea6d600640 1 -- 172.21.15.152:0/823715827 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea60040070 con 0x7fea70071cd0 2024-04-20T11:35:48.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.268+0000 7fea6d600640 1 -- 172.21.15.152:0/823715827 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea6003a420 con 0x7fea70071cd0 2024-04-20T11:35:48.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.268+0000 7fea75e00640 1 -- 172.21.15.152:0/823715827 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea7015f070 con 0x7fea70071cd0 2024-04-20T11:35:48.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.268+0000 7fea6d600640 1 -- 172.21.15.152:0/823715827 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea60035040 con 0x7fea70071cd0 2024-04-20T11:35:48.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.268+0000 7fea75e00640 1 -- 172.21.15.152:0/823715827 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea7015f910 con 0x7fea70071cd0 2024-04-20T11:35:48.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.269+0000 7fea75e00640 1 -- 172.21.15.152:0/823715827 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea70075e60 con 0x7fea70071cd0 2024-04-20T11:35:48.270 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.271+0000 7fea6d600640 1 -- 172.21.15.152:0/823715827 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 22) ==== 91601+0+0 (secure 0 0 0) 0x7fea6004b060 con 0x7fea70071cd0 2024-04-20T11:35:48.271 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.272+0000 7fea6d600640 1 --2- 172.21.15.152:0/823715827 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fea4c06cf50 0x7fea4c06f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:48.271 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.272+0000 7fea6d600640 1 -- 172.21.15.152:0/823715827 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5709+0+0 (secure 0 0 0) 0x7fea600ae590 con 0x7fea70071cd0 2024-04-20T11:35:48.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.272+0000 7fea67e00640 1 --2- 172.21.15.152:0/823715827 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fea4c06cf50 0x7fea4c06f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:48.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.274+0000 7fea67e00640 1 --2- 172.21.15.152:0/823715827 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fea4c06cf50 0x7fea4c06f410 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fea700af610 tx=0x7fea58008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:48.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.276+0000 7fea6d600640 1 -- 172.21.15.152:0/823715827 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fea6007c120 con 0x7fea70071cd0 2024-04-20T11:35:48.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.481+0000 7fea75e00640 1 -- 172.21.15.152:0/823715827 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fea70160c20 con 0x7fea70071cd0 2024-04-20T11:35:48.484 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.484+0000 7fea6d600640 1 -- 172.21.15.152:0/823715827 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v131) ==== 74+0+16739 (secure 0 0 0) 0x7fea6007ba20 con 0x7fea70071cd0 2024-04-20T11:35:48.484 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:35:48.484 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":131,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:35:20.781260+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:35:16.963520+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"130","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-04-20T11:35:18.215890+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:35:48.487 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.487+0000 7fea75e00640 1 -- 172.21.15.152:0/823715827 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fea4c06cf50 msgr2=0x7fea4c06f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:48.487 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.487+0000 7fea75e00640 1 --2- 172.21.15.152:0/823715827 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fea4c06cf50 0x7fea4c06f410 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fea700af610 tx=0x7fea58008040 comp rx=0 tx=0).stop 2024-04-20T11:35:48.487 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.487+0000 7fea75e00640 1 -- 172.21.15.152:0/823715827 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fea70071cd0 msgr2=0x7fea701f79d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:48.487 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.487+0000 7fea75e00640 1 --2- 172.21.15.152:0/823715827 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fea70071cd0 0x7fea701f79d0 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7fea60000c00 tx=0x7fea600046c0 comp rx=0 tx=0).stop 2024-04-20T11:35:48.487 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.487+0000 7fea75e00640 1 -- 172.21.15.152:0/823715827 shutdown_connections 2024-04-20T11:35:48.487 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.487+0000 7fea75e00640 1 --2- 172.21.15.152:0/823715827 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fea4c06cf50 0x7fea4c06f410 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:48.487 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.487+0000 7fea75e00640 1 --2- 172.21.15.152:0/823715827 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fea7015d5c0 0x7fea700b4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:48.487 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.487+0000 7fea75e00640 1 --2- 172.21.15.152:0/823715827 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fea7015d1e0 0x7fea7015e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:48.488 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.487+0000 7fea75e00640 1 --2- 172.21.15.152:0/823715827 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fea70071cd0 0x7fea701f79d0 unknown :-1 s=CLOSED pgs=1867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:48.488 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.487+0000 7fea75e00640 1 -- 172.21.15.152:0/823715827 >> 172.21.15.152:0/823715827 conn(0x7fea70067ae0 msgr2=0x7fea700b07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:48.488 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.487+0000 7fea75e00640 1 -- 172.21.15.152:0/823715827 shutdown_connections 2024-04-20T11:35:48.488 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.487+0000 7fea75e00640 1 -- 172.21.15.152:0/823715827 wait complete. 2024-04-20T11:35:48.507 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T11:26:47.246571+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 20, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T11:35:16.963520+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '130', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 21, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T11:35:18.215890+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '131', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T11:35:48.509 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T11:35:48.720 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.720+0000 7f78b2e00640 1 Processor -- start 2024-04-20T11:35:48.720 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.720+0000 7f78b2e00640 1 -- start start 2024-04-20T11:35:48.722 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.722+0000 7f78b2e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f78ac06e630 0x7f78ac162540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:48.722 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.722+0000 7f78b2e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f78ac0d2330 0x7f78ac0d2790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:48.722 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.722+0000 7f78b2e00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78ac0cf510 con 0x7f78ac0d0570 2024-04-20T11:35:48.722 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.722+0000 7f78b2e00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f78ac0cf7b0 con 0x7f78ac06e630 2024-04-20T11:35:48.722 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.722+0000 7f78b2e00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f78ac0cfa90 con 0x7f78ac0d2330 2024-04-20T11:35:48.722 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.722+0000 7f78abe00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f78ac06e630 0x7f78ac162540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:48.722 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.722+0000 7f78abe00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f78ac06e630 0x7f78ac162540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:60506/0 (socket says 172.21.15.152:60506) 2024-04-20T11:35:48.722 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.722+0000 7f78a3400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f78ac0d0570 0x7f78ac162a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:60586/0 (socket says 172.21.15.152:60586) 2024-04-20T11:35:48.722 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.722+0000 7f78abe00640 1 -- 172.21.15.152:0/1560071705 learned_addr learned my addr 172.21.15.152:0/1560071705 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:48.723 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.723+0000 7f78b0e00640 1 --2- 172.21.15.152:0/1560071705 >> v2:172.21.15.175:3300/0 conn(0x7f78ac0d2330 0x7f78ac0d2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:48.723 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.723+0000 7f78abe00640 1 -- 172.21.15.152:0/1560071705 >> v2:172.21.15.175:3300/0 conn(0x7f78ac0d2330 msgr2=0x7f78ac0d2790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:48.723 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.723+0000 7f78abe00640 1 --2- 172.21.15.152:0/1560071705 >> v2:172.21.15.175:3300/0 conn(0x7f78ac0d2330 0x7f78ac0d2790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:48.723 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.723+0000 7f78abe00640 1 -- 172.21.15.152:0/1560071705 >> v1:172.21.15.152:6789/0 conn(0x7f78ac0d0570 legacy=0x7f78ac162a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:48.724 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.723+0000 7f78abe00640 1 -- 172.21.15.152:0/1560071705 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78ac0cfd70 con 0x7f78ac06e630 2024-04-20T11:35:48.724 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.723+0000 7f78ab400640 1 -- 172.21.15.152:0/1560071705 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 789724819 0 0) 0x7f78ac0cf510 con 0x7f78ac0d0570 2024-04-20T11:35:48.724 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.724+0000 7f78abe00640 1 --2- 172.21.15.152:0/1560071705 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f78ac06e630 0x7f78ac162540 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7f78ac0cb7b0 tx=0x7f789c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8873028a4b8db165 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:48.725 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.724+0000 7f78ab400640 1 -- 172.21.15.152:0/1560071705 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f789c034070 con 0x7f78ac06e630 2024-04-20T11:35:48.725 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.724+0000 7f78ab400640 1 -- 172.21.15.152:0/1560071705 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f789c030860 con 0x7f78ac06e630 2024-04-20T11:35:48.725 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.724+0000 7f78ab400640 1 -- 172.21.15.152:0/1560071705 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f789c030c30 con 0x7f78ac06e630 2024-04-20T11:35:48.725 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.725+0000 7f78b2e00640 1 -- 172.21.15.152:0/1560071705 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f78ac06e630 msgr2=0x7f78ac162540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:48.725 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.725+0000 7f78b2e00640 1 --2- 172.21.15.152:0/1560071705 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f78ac06e630 0x7f78ac162540 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7f78ac0cb7b0 tx=0x7f789c02b9a0 comp rx=0 tx=0).stop 2024-04-20T11:35:48.725 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.725+0000 7f78b2e00640 1 -- 172.21.15.152:0/1560071705 shutdown_connections 2024-04-20T11:35:48.725 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.725+0000 7f78b2e00640 1 --2- 172.21.15.152:0/1560071705 >> v2:172.21.15.175:3300/0 conn(0x7f78ac0d2330 0x7f78ac0d2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:48.725 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.725+0000 7f78b2e00640 1 --2- 172.21.15.152:0/1560071705 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f78ac06e630 0x7f78ac162540 unknown :-1 s=CLOSED pgs=1869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:48.726 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.725+0000 7f78b2e00640 1 -- 172.21.15.152:0/1560071705 >> 172.21.15.152:0/1560071705 conn(0x7f78ac08bfd0 msgr2=0x7f78ac0d4760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:48.726 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.725+0000 7f78b2e00640 1 -- 172.21.15.152:0/1560071705 shutdown_connections 2024-04-20T11:35:48.726 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.725+0000 7f78b2e00640 1 -- 172.21.15.152:0/1560071705 wait complete. 2024-04-20T11:35:48.726 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.727+0000 7f78b2e00640 1 Processor -- start 2024-04-20T11:35:48.726 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.727+0000 7f78b2e00640 1 -- start start 2024-04-20T11:35:48.728 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.728+0000 7f78b2e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f78ac06e630 0x7f78ac152ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:48.728 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.728+0000 7f78b2e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f78ac0d2330 0x7f78ac1e9810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:48.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.728+0000 7f78b2e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78ac1e9d50 0x7f78ac0ccac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:48.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.728+0000 7f78b2e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f78ac06ea10 con 0x7f78ac0d2330 2024-04-20T11:35:48.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.728+0000 7f78b2e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f78ac0d5370 con 0x7f78ac1e9d50 2024-04-20T11:35:48.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.728+0000 7f78b2e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f78ac163190 con 0x7f78ac06e630 2024-04-20T11:35:48.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.728+0000 7f78a3400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f78ac0d2330 0x7f78ac1e9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:48.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.728+0000 7f78a3400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f78ac0d2330 0x7f78ac1e9810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:60518/0 (socket says 172.21.15.152:60518) 2024-04-20T11:35:48.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.728+0000 7f78a3400640 1 -- 172.21.15.152:0/1081111842 learned_addr learned my addr 172.21.15.152:0/1081111842 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:48.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.729+0000 7f78abe00640 1 --2- 172.21.15.152:0/1081111842 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f78ac06e630 0x7f78ac152ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:48.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.729+0000 7f78b0e00640 1 --2- 172.21.15.152:0/1081111842 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78ac1e9d50 0x7f78ac0ccac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:48.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.729+0000 7f78a3400640 1 -- 172.21.15.152:0/1081111842 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f78ac06e630 msgr2=0x7f78ac152ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:48.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.729+0000 7f78a3400640 1 --2- 172.21.15.152:0/1081111842 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f78ac06e630 0x7f78ac152ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:48.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.729+0000 7f78a3400640 1 -- 172.21.15.152:0/1081111842 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78ac1e9d50 msgr2=0x7f78ac0ccac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:48.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.729+0000 7f78a3400640 1 --2- 172.21.15.152:0/1081111842 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78ac1e9d50 0x7f78ac0ccac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:48.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.729+0000 7f78a3400640 1 -- 172.21.15.152:0/1081111842 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78ac0ba3b0 con 0x7f78ac0d2330 2024-04-20T11:35:48.730 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.729+0000 7f78a3400640 1 --2- 172.21.15.152:0/1081111842 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f78ac0d2330 0x7f78ac1e9810 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f789400c470 tx=0x7f789400c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:48.730 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.729+0000 7f78a8c00640 1 -- 172.21.15.152:0/1081111842 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7894018070 con 0x7f78ac0d2330 2024-04-20T11:35:48.730 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.730+0000 7f78a8c00640 1 -- 172.21.15.152:0/1081111842 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78940074a0 con 0x7f78ac0d2330 2024-04-20T11:35:48.731 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.730+0000 7f78a8c00640 1 -- 172.21.15.152:0/1081111842 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7894004a20 con 0x7f78ac0d2330 2024-04-20T11:35:48.731 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.730+0000 7f78b2e00640 1 -- 172.21.15.152:0/1081111842 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78ac0cd6d0 con 0x7f78ac0d2330 2024-04-20T11:35:48.731 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.730+0000 7f78b2e00640 1 -- 172.21.15.152:0/1081111842 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78ac0cdc90 con 0x7f78ac0d2330 2024-04-20T11:35:48.731 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.731+0000 7f78b2e00640 1 -- 172.21.15.152:0/1081111842 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78ac069100 con 0x7f78ac0d2330 2024-04-20T11:35:48.731 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.732+0000 7f78a8c00640 1 -- 172.21.15.152:0/1081111842 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 22) ==== 91601+0+0 (secure 0 0 0) 0x7f789401b070 con 0x7f78ac0d2330 2024-04-20T11:35:48.732 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.733+0000 7f78a8c00640 1 --2- 172.21.15.152:0/1081111842 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f788806cf20 0x7f788806f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:48.732 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.733+0000 7f78a8c00640 1 -- 172.21.15.152:0/1081111842 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5709+0+0 (secure 0 0 0) 0x7f789408b9d0 con 0x7f78ac0d2330 2024-04-20T11:35:48.733 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.733+0000 7f78abe00640 1 --2- 172.21.15.152:0/1081111842 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f788806cf20 0x7f788806f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:48.734 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.735+0000 7f78abe00640 1 --2- 172.21.15.152:0/1081111842 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f788806cf20 0x7f788806f3e0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f789c036000 tx=0x7f789c03b040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:48.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.737+0000 7f78a8c00640 1 -- 172.21.15.152:0/1081111842 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f78ac0ba3b0 con 0x7f78ac0d2330 2024-04-20T11:35:48.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.937+0000 7f78b2e00640 1 -- 172.21.15.152:0/1081111842 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f78ac0b70d0 con 0x7f78ac0d2330 2024-04-20T11:35:48.938 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.938+0000 7f78a8c00640 1 -- 172.21.15.152:0/1081111842 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v131) ==== 93+0+10 (secure 0 0 0) 0x7f7894007650 con 0x7f78ac0d2330 2024-04-20T11:35:48.938 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 1 2024-04-20T11:35:48.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.940+0000 7f78b2e00640 1 -- 172.21.15.152:0/1081111842 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f788806cf20 msgr2=0x7f788806f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:48.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.940+0000 7f78b2e00640 1 --2- 172.21.15.152:0/1081111842 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f788806cf20 0x7f788806f3e0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f789c036000 tx=0x7f789c03b040 comp rx=0 tx=0).stop 2024-04-20T11:35:48.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.940+0000 7f78b2e00640 1 -- 172.21.15.152:0/1081111842 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f78ac0d2330 msgr2=0x7f78ac1e9810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:48.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.940+0000 7f78b2e00640 1 --2- 172.21.15.152:0/1081111842 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f78ac0d2330 0x7f78ac1e9810 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f789400c470 tx=0x7f789400c930 comp rx=0 tx=0).stop 2024-04-20T11:35:48.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.940+0000 7f78b2e00640 1 -- 172.21.15.152:0/1081111842 shutdown_connections 2024-04-20T11:35:48.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.940+0000 7f78b2e00640 1 --2- 172.21.15.152:0/1081111842 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f788806cf20 0x7f788806f3e0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:48.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.941+0000 7f78b2e00640 1 --2- 172.21.15.152:0/1081111842 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78ac1e9d50 0x7f78ac0ccac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:48.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.941+0000 7f78b2e00640 1 --2- 172.21.15.152:0/1081111842 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f78ac0d2330 0x7f78ac1e9810 unknown :-1 s=CLOSED pgs=1870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:48.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.941+0000 7f78b2e00640 1 --2- 172.21.15.152:0/1081111842 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f78ac06e630 0x7f78ac152ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:48.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.941+0000 7f78b2e00640 1 -- 172.21.15.152:0/1081111842 >> 172.21.15.152:0/1081111842 conn(0x7f78ac08bfd0 msgr2=0x7f78ac0b3550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:48.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.941+0000 7f78b2e00640 1 -- 172.21.15.152:0/1081111842 shutdown_connections 2024-04-20T11:35:48.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:48.941+0000 7f78b2e00640 1 -- 172.21.15.152:0/1081111842 wait complete. 2024-04-20T11:35:48.960 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T11:35:49.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.162+0000 7fbd29c00640 1 Processor -- start 2024-04-20T11:35:49.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.162+0000 7fbd29c00640 1 -- start start 2024-04-20T11:35:49.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.163+0000 7fbd29c00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fbd24146000 0x7fbd240b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:49.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.164+0000 7fbd29c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd240b2170 0x7fbd240b25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:49.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.164+0000 7fbd29c00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd241463e0 con 0x7fbd240cd1b0 2024-04-20T11:35:49.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.164+0000 7fbd29c00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fbd240b4750 con 0x7fbd24146000 2024-04-20T11:35:49.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.164+0000 7fbd29c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fbd240b3e20 con 0x7fbd240b2170 2024-04-20T11:35:49.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.164+0000 7fbd22a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fbd24146000 0x7fbd240b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:49.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.164+0000 7fbd23400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fbd240cd1b0 0x7fbd240cd5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:60588/0 (socket says 172.21.15.152:60588) 2024-04-20T11:35:49.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.164+0000 7fbd23400640 1 -- 172.21.15.152:0/3959016633 learned_addr learned my addr 172.21.15.152:0/3959016633 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:35:49.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.164+0000 7fbd22000640 1 --2- 172.21.15.152:0/3959016633 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd240b2170 0x7fbd240b25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:49.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.164+0000 7fbd22a00640 1 -- 172.21.15.152:0/3959016633 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd240b2170 msgr2=0x7fbd240b25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:49.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.164+0000 7fbd22a00640 1 --2- 172.21.15.152:0/3959016633 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd240b2170 0x7fbd240b25f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:49.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.164+0000 7fbd22a00640 1 -- 172.21.15.152:0/3959016633 >> v1:172.21.15.152:6789/0 conn(0x7fbd240cd1b0 legacy=0x7fbd240cd5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:49.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.165+0000 7fbd22a00640 1 -- 172.21.15.152:0/3959016633 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd240b4100 con 0x7fbd24146000 2024-04-20T11:35:49.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.164+0000 7fbd21600640 1 -- 172.21.15.152:0/3959016633 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2701428911 0 0) 0x7fbd241463e0 con 0x7fbd240cd1b0 2024-04-20T11:35:49.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.165+0000 7fbd22000640 1 --2- 172.21.15.152:0/3959016633 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd240b2170 0x7fbd240b25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:35:49.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.166+0000 7fbd22a00640 1 --2- 172.21.15.152:0/3959016633 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd24146000 0x7fbd240b4be0 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7fbd140097a0 tx=0x7fbd1402b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e72d5eb58a98003b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:49.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.166+0000 7fbd21600640 1 -- 172.21.15.152:0/3959016633 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd14031780 con 0x7fbd24146000 2024-04-20T11:35:49.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.166+0000 7fbd21600640 1 -- 172.21.15.152:0/3959016633 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd14037070 con 0x7fbd24146000 2024-04-20T11:35:49.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.166+0000 7fbd21600640 1 -- 172.21.15.152:0/3959016633 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd14031b40 con 0x7fbd24146000 2024-04-20T11:35:49.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.167+0000 7fbd29c00640 1 -- 172.21.15.152:0/3959016633 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd24146000 msgr2=0x7fbd240b4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:49.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.167+0000 7fbd29c00640 1 --2- 172.21.15.152:0/3959016633 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd24146000 0x7fbd240b4be0 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7fbd140097a0 tx=0x7fbd1402b820 comp rx=0 tx=0).stop 2024-04-20T11:35:49.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.167+0000 7fbd29c00640 1 -- 172.21.15.152:0/3959016633 shutdown_connections 2024-04-20T11:35:49.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.167+0000 7fbd29c00640 1 --2- 172.21.15.152:0/3959016633 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd240b2170 0x7fbd240b25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:49.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.167+0000 7fbd29c00640 1 --2- 172.21.15.152:0/3959016633 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd24146000 0x7fbd240b4be0 unknown :-1 s=CLOSED pgs=1872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:49.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.167+0000 7fbd29c00640 1 -- 172.21.15.152:0/3959016633 >> 172.21.15.152:0/3959016633 conn(0x7fbd2406b980 msgr2=0x7fbd2406bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:49.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.167+0000 7fbd29c00640 1 -- 172.21.15.152:0/3959016633 shutdown_connections 2024-04-20T11:35:49.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.167+0000 7fbd29c00640 1 -- 172.21.15.152:0/3959016633 wait complete. 2024-04-20T11:35:49.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.169+0000 7fbd29c00640 1 Processor -- start 2024-04-20T11:35:49.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.169+0000 7fbd29c00640 1 -- start start 2024-04-20T11:35:49.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.170+0000 7fbd29c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd240b2170 0x7fbd240c88a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:49.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.170+0000 7fbd29c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd240cd1b0 0x7fbd241f75a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:49.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.170+0000 7fbd29c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd24146000 0x7fbd240c91d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:49.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.170+0000 7fbd29c00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fbd2415ce70 con 0x7fbd240b2170 2024-04-20T11:35:49.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.170+0000 7fbd29c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbd2415ccf0 con 0x7fbd24146000 2024-04-20T11:35:49.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.170+0000 7fbd29c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fbd2415cff0 con 0x7fbd240cd1b0 2024-04-20T11:35:49.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.170+0000 7fbd22a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd240b2170 0x7fbd240c88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:49.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.170+0000 7fbd22a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd240b2170 0x7fbd240c88a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:60542/0 (socket says 172.21.15.152:60542) 2024-04-20T11:35:49.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.170+0000 7fbd22a00640 1 -- 172.21.15.152:0/3143433881 learned_addr learned my addr 172.21.15.152:0/3143433881 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:49.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.170+0000 7fbd22a00640 1 -- 172.21.15.152:0/3143433881 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd240cd1b0 msgr2=0x7fbd241f75a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:49.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.170+0000 7fbd22000640 1 --2- 172.21.15.152:0/3143433881 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd240cd1b0 0x7fbd241f75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:49.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.170+0000 7fbd22a00640 1 --2- 172.21.15.152:0/3143433881 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd240cd1b0 0x7fbd241f75a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:49.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.170+0000 7fbd22a00640 1 -- 172.21.15.152:0/3143433881 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd24146000 msgr2=0x7fbd240c91d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:49.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.170+0000 7fbd22a00640 1 --2- 172.21.15.152:0/3143433881 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd24146000 0x7fbd240c91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:49.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.170+0000 7fbd22a00640 1 -- 172.21.15.152:0/3143433881 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd2413d660 con 0x7fbd240b2170 2024-04-20T11:35:49.172 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.171+0000 7fbd22a00640 1 --2- 172.21.15.152:0/3143433881 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd240b2170 0x7fbd240c88a0 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7fbd14034040 tx=0x7fbd140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:49.172 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.171+0000 7fbd1aa00640 1 -- 172.21.15.152:0/3143433881 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd14032820 con 0x7fbd240b2170 2024-04-20T11:35:49.172 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.171+0000 7fbd1aa00640 1 -- 172.21.15.152:0/3143433881 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd140329e0 con 0x7fbd240b2170 2024-04-20T11:35:49.172 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.171+0000 7fbd1aa00640 1 -- 172.21.15.152:0/3143433881 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd14032dd0 con 0x7fbd240b2170 2024-04-20T11:35:49.172 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.171+0000 7fbd29c00640 1 -- 172.21.15.152:0/3143433881 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd240cbf50 con 0x7fbd240b2170 2024-04-20T11:35:49.172 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.171+0000 7fbd29c00640 1 -- 172.21.15.152:0/3143433881 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd2406dfd0 con 0x7fbd240b2170 2024-04-20T11:35:49.172 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.172+0000 7fbd29c00640 1 -- 172.21.15.152:0/3143433881 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd240689f0 con 0x7fbd240b2170 2024-04-20T11:35:49.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.173+0000 7fbd1aa00640 1 -- 172.21.15.152:0/3143433881 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 22) ==== 91601+0+0 (secure 0 0 0) 0x7fbd1404e440 con 0x7fbd240b2170 2024-04-20T11:35:49.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.174+0000 7fbd1aa00640 1 --2- 172.21.15.152:0/3143433881 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fbd0406cf20 0x7fbd0406f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:49.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.174+0000 7fbd1aa00640 1 -- 172.21.15.152:0/3143433881 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5709+0+0 (secure 0 0 0) 0x7fbd140ae8f0 con 0x7fbd240b2170 2024-04-20T11:35:49.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.175+0000 7fbd22000640 1 --2- 172.21.15.152:0/3143433881 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fbd0406cf20 0x7fbd0406f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:49.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.176+0000 7fbd22000640 1 --2- 172.21.15.152:0/3143433881 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fbd0406cf20 0x7fbd0406f3e0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fbd240cab70 tx=0x7fbd10008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:49.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.179+0000 7fbd1aa00640 1 -- 172.21.15.152:0/3143433881 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fbd14040070 con 0x7fbd240b2170 2024-04-20T11:35:49.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.381+0000 7fbd29c00640 1 -- 172.21.15.152:0/3143433881 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fbd240b1e70 con 0x7fbd240b2170 2024-04-20T11:35:49.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.382+0000 7fbd1aa00640 1 -- 172.21.15.152:0/3143433881 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v131) ==== 104+0+11 (secure 0 0 0) 0x7fbd1402f050 con 0x7fbd240b2170 2024-04-20T11:35:49.382 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 32 2024-04-20T11:35:49.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.385+0000 7fbd29c00640 1 -- 172.21.15.152:0/3143433881 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fbd0406cf20 msgr2=0x7fbd0406f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:49.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.385+0000 7fbd29c00640 1 --2- 172.21.15.152:0/3143433881 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fbd0406cf20 0x7fbd0406f3e0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fbd240cab70 tx=0x7fbd10008040 comp rx=0 tx=0).stop 2024-04-20T11:35:49.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.385+0000 7fbd29c00640 1 -- 172.21.15.152:0/3143433881 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd240b2170 msgr2=0x7fbd240c88a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:49.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.385+0000 7fbd29c00640 1 --2- 172.21.15.152:0/3143433881 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd240b2170 0x7fbd240c88a0 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7fbd14034040 tx=0x7fbd140046c0 comp rx=0 tx=0).stop 2024-04-20T11:35:49.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.385+0000 7fbd29c00640 1 -- 172.21.15.152:0/3143433881 shutdown_connections 2024-04-20T11:35:49.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.385+0000 7fbd29c00640 1 --2- 172.21.15.152:0/3143433881 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fbd0406cf20 0x7fbd0406f3e0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:49.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.385+0000 7fbd29c00640 1 --2- 172.21.15.152:0/3143433881 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd24146000 0x7fbd240c91d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:49.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.385+0000 7fbd29c00640 1 --2- 172.21.15.152:0/3143433881 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd240cd1b0 0x7fbd241f75a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:49.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.385+0000 7fbd29c00640 1 --2- 172.21.15.152:0/3143433881 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd240b2170 0x7fbd240c88a0 unknown :-1 s=CLOSED pgs=1873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:49.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.385+0000 7fbd29c00640 1 -- 172.21.15.152:0/3143433881 >> 172.21.15.152:0/3143433881 conn(0x7fbd2406b980 msgr2=0x7fbd2406bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:49.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.385+0000 7fbd29c00640 1 -- 172.21.15.152:0/3143433881 shutdown_connections 2024-04-20T11:35:49.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.385+0000 7fbd29c00640 1 -- 172.21.15.152:0/3143433881 wait complete. 2024-04-20T11:35:49.405 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T11:35:49.608 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.608+0000 7fbc1b200640 1 Processor -- start 2024-04-20T11:35:49.608 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.609+0000 7fbc1b200640 1 -- start start 2024-04-20T11:35:49.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.610+0000 7fbc1b200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fbc1c0cf5b0 0x7fbc1c0d4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:49.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.610+0000 7fbc1b200640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc1c0d2b70 con 0x7fbc1c160230 2024-04-20T11:35:49.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.610+0000 7fbc1b200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc1c0d2e50 con 0x7fbc1c0ceeb0 2024-04-20T11:35:49.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.610+0000 7fbc1b200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fbc1c0d5420 con 0x7fbc1c0cf5b0 2024-04-20T11:35:49.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.610+0000 7fbc1a800640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fbc1c160230 0x7fbc1c1606e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:60604/0 (socket says 172.21.15.152:60604) 2024-04-20T11:35:49.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.610+0000 7fbc1a800640 1 -- 172.21.15.152:0/516144920 learned_addr learned my addr 172.21.15.152:0/516144920 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:35:49.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.611+0000 7fbc19e00640 1 --2- 172.21.15.152:0/516144920 >> v2:172.21.15.175:3300/0 conn(0x7fbc1c0cf5b0 0x7fbc1c0d4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:49.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.611+0000 7fbc19400640 1 -- 172.21.15.152:0/516144920 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4209159234 0 0) 0x7fbc1c0d2b70 con 0x7fbc1c160230 2024-04-20T11:35:49.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.611+0000 7fbc19400640 1 -- 172.21.15.152:0/516144920 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbc00003520 con 0x7fbc1c160230 2024-04-20T11:35:49.616 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.611+0000 7fbc19400640 1 -- 172.21.15.152:0/516144920 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4271959201 0 0) 0x7fbc00003520 con 0x7fbc1c160230 2024-04-20T11:35:49.616 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.611+0000 7fbc19400640 1 -- 172.21.15.152:0/516144920 >> v2:172.21.15.175:3300/0 conn(0x7fbc1c0cf5b0 msgr2=0x7fbc1c0d4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:49.616 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.611+0000 7fbc19400640 1 --2- 172.21.15.152:0/516144920 >> v2:172.21.15.175:3300/0 conn(0x7fbc1c0cf5b0 0x7fbc1c0d4ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:49.616 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.611+0000 7fbc19400640 1 -- 172.21.15.152:0/516144920 >> v1:172.21.15.175:6789/0 conn(0x7fbc1c0ceeb0 legacy=0x7fbc1c0cf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:49.616 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.611+0000 7fbc19400640 1 -- 172.21.15.152:0/516144920 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc1c0d3130 con 0x7fbc1c160230 2024-04-20T11:35:49.616 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.611+0000 7fbc19400640 1 -- 172.21.15.152:0/516144920 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7fbc140019f0 con 0x7fbc1c160230 2024-04-20T11:35:49.616 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.611+0000 7fbc19400640 1 -- 172.21.15.152:0/516144920 >> v1:172.21.15.152:6789/0 conn(0x7fbc1c160230 legacy=0x7fbc1c1606e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:49.616 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.611+0000 7fbc19400640 1 --2- 172.21.15.152:0/516144920 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbc000039b0 0x7fbc00003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:49.616 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.611+0000 7fbc19400640 1 --2- 172.21.15.152:0/516144920 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc00004800 0x7fbc00004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:49.616 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.611+0000 7fbc19400640 1 --2- 172.21.15.152:0/516144920 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbc000051c0 0x7fbc0000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:49.616 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.611+0000 7fbc19400640 1 -- 172.21.15.152:0/516144920 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fbc1c0d5420 con 0x7fbc00004800 2024-04-20T11:35:49.616 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.611+0000 7fbc19400640 1 -- 172.21.15.152:0/516144920 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbc0000c070 con 0x7fbc000051c0 2024-04-20T11:35:49.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.611+0000 7fbc19400640 1 -- 172.21.15.152:0/516144920 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fbc0000c270 con 0x7fbc000039b0 2024-04-20T11:35:49.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.612+0000 7fbc13e00640 1 --2- 172.21.15.152:0/516144920 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc00004800 0x7fbc00004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:49.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.612+0000 7fbc19e00640 1 --2- 172.21.15.152:0/516144920 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbc000039b0 0x7fbc00003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:49.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.612+0000 7fbc13e00640 1 -- 172.21.15.152:0/516144920 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbc000039b0 msgr2=0x7fbc00003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:49.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.612+0000 7fbc13e00640 1 --2- 172.21.15.152:0/516144920 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbc000039b0 0x7fbc00003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:49.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.612+0000 7fbc13e00640 1 -- 172.21.15.152:0/516144920 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbc000051c0 msgr2=0x7fbc0000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:49.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.612+0000 7fbc13e00640 1 --2- 172.21.15.152:0/516144920 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbc000051c0 0x7fbc0000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:49.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.612+0000 7fbc1a800640 1 -- 172.21.15.152:0/516144920 reap_dead start 2024-04-20T11:35:49.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.612+0000 7fbc13e00640 1 -- 172.21.15.152:0/516144920 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc00003520 con 0x7fbc00004800 2024-04-20T11:35:49.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.613+0000 7fbc13e00640 1 --2- 172.21.15.152:0/516144920 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc00004800 0x7fbc00004c80 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7fbc04001770 tx=0x7fbc00005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d241b884687e309 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:49.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.613+0000 7fbc19400640 1 -- 172.21.15.152:0/516144920 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc040013c0 con 0x7fbc00004800 2024-04-20T11:35:49.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.613+0000 7fbc19400640 1 -- 172.21.15.152:0/516144920 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc040052d0 con 0x7fbc00004800 2024-04-20T11:35:49.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.613+0000 7fbc19400640 1 -- 172.21.15.152:0/516144920 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc04003640 con 0x7fbc00004800 2024-04-20T11:35:49.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.614+0000 7fbc1b200640 1 -- 172.21.15.152:0/516144920 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc00004800 msgr2=0x7fbc00004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:49.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.614+0000 7fbc1b200640 1 --2- 172.21.15.152:0/516144920 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc00004800 0x7fbc00004c80 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7fbc04001770 tx=0x7fbc00005830 comp rx=0 tx=0).stop 2024-04-20T11:35:49.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.614+0000 7fbc1b200640 1 -- 172.21.15.152:0/516144920 shutdown_connections 2024-04-20T11:35:49.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.614+0000 7fbc1b200640 1 --2- 172.21.15.152:0/516144920 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc00004800 0x7fbc00004c80 unknown :-1 s=CLOSED pgs=1875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:49.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.614+0000 7fbc1b200640 1 -- 172.21.15.152:0/516144920 >> 172.21.15.152:0/516144920 conn(0x7fbc1c08bfd0 msgr2=0x7fbc1c0d4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:49.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.614+0000 7fbc1b200640 1 -- 172.21.15.152:0/516144920 shutdown_connections 2024-04-20T11:35:49.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.614+0000 7fbc1b200640 1 -- 172.21.15.152:0/516144920 wait complete. 2024-04-20T11:35:49.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.616+0000 7fbc1b200640 1 Processor -- start 2024-04-20T11:35:49.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.616+0000 7fbc1b200640 1 -- start start 2024-04-20T11:35:49.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.617+0000 7fbc1b200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc00004800 0x7fbc1c159c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:49.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.617+0000 7fbc1b200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbc1c0b9180 0x7fbc1c0b9600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:49.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.617+0000 7fbc1b200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbc1c15dd40 0x7fbc1c15e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:49.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.617+0000 7fbc1b200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fbc1c0d3c10 con 0x7fbc00004800 2024-04-20T11:35:49.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.617+0000 7fbc1b200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbc1c0d36f0 con 0x7fbc1c0b9180 2024-04-20T11:35:49.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.617+0000 7fbc1b200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fbc1c0d2790 con 0x7fbc1c15dd40 2024-04-20T11:35:49.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.617+0000 7fbc19e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc00004800 0x7fbc1c159c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:49.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.617+0000 7fbc19e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc00004800 0x7fbc1c159c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:60556/0 (socket says 172.21.15.152:60556) 2024-04-20T11:35:49.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.617+0000 7fbc19e00640 1 -- 172.21.15.152:0/4289502262 learned_addr learned my addr 172.21.15.152:0/4289502262 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:49.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.618+0000 7fbc1a800640 1 --2- 172.21.15.152:0/4289502262 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbc1c15dd40 0x7fbc1c15e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:49.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.618+0000 7fbc13e00640 1 --2- 172.21.15.152:0/4289502262 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbc1c0b9180 0x7fbc1c0b9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:49.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.618+0000 7fbc19e00640 1 -- 172.21.15.152:0/4289502262 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbc1c15dd40 msgr2=0x7fbc1c15e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:49.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.618+0000 7fbc19e00640 1 --2- 172.21.15.152:0/4289502262 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbc1c15dd40 0x7fbc1c15e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:49.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.618+0000 7fbc19e00640 1 -- 172.21.15.152:0/4289502262 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbc1c0b9180 msgr2=0x7fbc1c0b9600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:49.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.618+0000 7fbc19e00640 1 --2- 172.21.15.152:0/4289502262 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbc1c0b9180 0x7fbc1c0b9600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:49.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.618+0000 7fbc19e00640 1 -- 172.21.15.152:0/4289502262 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc1c15e700 con 0x7fbc00004800 2024-04-20T11:35:49.623 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.618+0000 7fbc1a800640 1 --2- 172.21.15.152:0/4289502262 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbc1c15dd40 0x7fbc1c15e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:35:49.623 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.618+0000 7fbc19e00640 1 --2- 172.21.15.152:0/4289502262 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc00004800 0x7fbc1c159c60 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c0116d0 tx=0x7fbc0c011b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:49.623 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.618+0000 7fbc12000640 1 -- 172.21.15.152:0/4289502262 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc0c00f9c0 con 0x7fbc00004800 2024-04-20T11:35:49.623 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.619+0000 7fbc12000640 1 -- 172.21.15.152:0/4289502262 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc0c00fb60 con 0x7fbc00004800 2024-04-20T11:35:49.623 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.619+0000 7fbc12000640 1 -- 172.21.15.152:0/4289502262 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc0c015460 con 0x7fbc00004800 2024-04-20T11:35:49.623 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.619+0000 7fbc1b200640 1 -- 172.21.15.152:0/4289502262 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc1c0c7d80 con 0x7fbc00004800 2024-04-20T11:35:49.623 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.619+0000 7fbc1b200640 1 -- 172.21.15.152:0/4289502262 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc1c0c8300 con 0x7fbc00004800 2024-04-20T11:35:49.623 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.620+0000 7fbc1b200640 1 -- 172.21.15.152:0/4289502262 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc1c075e60 con 0x7fbc00004800 2024-04-20T11:35:49.623 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.621+0000 7fbc12000640 1 -- 172.21.15.152:0/4289502262 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 22) ==== 91601+0+0 (secure 0 0 0) 0x7fbc0c015600 con 0x7fbc00004800 2024-04-20T11:35:49.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.622+0000 7fbc12000640 1 --2- 172.21.15.152:0/4289502262 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fbbf806cf50 0x7fbbf806f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:49.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.622+0000 7fbc12000640 1 -- 172.21.15.152:0/4289502262 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5709+0+0 (secure 0 0 0) 0x7fbc0c08d140 con 0x7fbc00004800 2024-04-20T11:35:49.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.623+0000 7fbc13e00640 1 --2- 172.21.15.152:0/4289502262 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fbbf806cf50 0x7fbbf806f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:49.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.624+0000 7fbc13e00640 1 --2- 172.21.15.152:0/4289502262 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fbbf806cf50 0x7fbbf806f410 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fbc04033000 tx=0x7fbc04009030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:49.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.626+0000 7fbc12000640 1 -- 172.21.15.152:0/4289502262 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fbc0c019030 con 0x7fbc00004800 2024-04-20T11:35:49.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.827+0000 7fbc1b200640 1 -- 172.21.15.152:0/4289502262 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fbc1c0cf390 con 0x7fbc00004800 2024-04-20T11:35:49.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.828+0000 7fbc12000640 1 -- 172.21.15.152:0/4289502262 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v131) ==== 100+0+11 (secure 0 0 0) 0x7fbc0c05abf0 con 0x7fbc00004800 2024-04-20T11:35:49.828 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 64 2024-04-20T11:35:49.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.831+0000 7fbc1b200640 1 -- 172.21.15.152:0/4289502262 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fbbf806cf50 msgr2=0x7fbbf806f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:49.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.831+0000 7fbc1b200640 1 --2- 172.21.15.152:0/4289502262 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fbbf806cf50 0x7fbbf806f410 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fbc04033000 tx=0x7fbc04009030 comp rx=0 tx=0).stop 2024-04-20T11:35:49.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.831+0000 7fbc1b200640 1 -- 172.21.15.152:0/4289502262 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc00004800 msgr2=0x7fbc1c159c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:49.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.831+0000 7fbc1b200640 1 --2- 172.21.15.152:0/4289502262 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc00004800 0x7fbc1c159c60 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c0116d0 tx=0x7fbc0c011b90 comp rx=0 tx=0).stop 2024-04-20T11:35:49.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.831+0000 7fbc1b200640 1 -- 172.21.15.152:0/4289502262 shutdown_connections 2024-04-20T11:35:49.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.831+0000 7fbc1b200640 1 --2- 172.21.15.152:0/4289502262 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fbbf806cf50 0x7fbbf806f410 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:49.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.831+0000 7fbc1b200640 1 --2- 172.21.15.152:0/4289502262 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbc1c15dd40 0x7fbc1c15e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:49.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.831+0000 7fbc1b200640 1 --2- 172.21.15.152:0/4289502262 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbc1c0b9180 0x7fbc1c0b9600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:49.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.831+0000 7fbc1b200640 1 --2- 172.21.15.152:0/4289502262 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc00004800 0x7fbc1c159c60 unknown :-1 s=CLOSED pgs=1876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:49.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.831+0000 7fbc1b200640 1 -- 172.21.15.152:0/4289502262 >> 172.21.15.152:0/4289502262 conn(0x7fbc1c08bfd0 msgr2=0x7fbc1c06d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:49.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.831+0000 7fbc1b200640 1 -- 172.21.15.152:0/4289502262 shutdown_connections 2024-04-20T11:35:49.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:49.831+0000 7fbc1b200640 1 -- 172.21.15.152:0/4289502262 wait complete. 2024-04-20T11:35:49.851 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-20T11:35:50.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.053+0000 7efea2800640 1 Processor -- start 2024-04-20T11:35:50.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.053+0000 7efea2800640 1 -- start start 2024-04-20T11:35:50.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.054+0000 7efea2800640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7efe9c072b70 0x7efe9c073b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:50.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.054+0000 7efea2800640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe9c072f70 con 0x7efe9c0b4870 2024-04-20T11:35:50.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.054+0000 7efea2800640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe9c0ce390 con 0x7efe9c15c7f0 2024-04-20T11:35:50.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.054+0000 7efea2800640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7efe9c0ce140 con 0x7efe9c072b70 2024-04-20T11:35:50.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.055+0000 7efea0a00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7efe9c0b4870 0x7efe9c0cf430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:60620/0 (socket says 172.21.15.152:60620) 2024-04-20T11:35:50.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.055+0000 7efea0a00640 1 -- 172.21.15.152:0/2891128103 learned_addr learned my addr 172.21.15.152:0/2891128103 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:35:50.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.055+0000 7efea1e00640 1 --2- 172.21.15.152:0/2891128103 >> v2:172.21.15.175:3300/0 conn(0x7efe9c072b70 0x7efe9c073b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:50.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.055+0000 7efe9be00640 1 -- 172.21.15.152:0/2891128103 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2858729150 0 0) 0x7efe8c001110 con 0x7efe9c0b4870 2024-04-20T11:35:50.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.055+0000 7efe9be00640 1 -- 172.21.15.152:0/2891128103 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efe84003520 con 0x7efe9c0b4870 2024-04-20T11:35:50.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.056+0000 7efe9be00640 1 -- 172.21.15.152:0/2891128103 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2555029725 0 0) 0x7efe84003520 con 0x7efe9c0b4870 2024-04-20T11:35:50.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.056+0000 7efe9be00640 1 -- 172.21.15.152:0/2891128103 >> v2:172.21.15.175:3300/0 conn(0x7efe9c072b70 msgr2=0x7efe9c073b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:50.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.056+0000 7efe9be00640 1 --2- 172.21.15.152:0/2891128103 >> v2:172.21.15.175:3300/0 conn(0x7efe9c072b70 0x7efe9c073b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:50.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.056+0000 7efe9be00640 1 -- 172.21.15.152:0/2891128103 >> v1:172.21.15.175:6789/0 conn(0x7efe9c15c7f0 legacy=0x7efe9c0b4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:50.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.056+0000 7efe9be00640 1 -- 172.21.15.152:0/2891128103 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe9c0ce670 con 0x7efe9c0b4870 2024-04-20T11:35:50.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.056+0000 7efe9be00640 1 -- 172.21.15.152:0/2891128103 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7efe8c001840 con 0x7efe9c0b4870 2024-04-20T11:35:50.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.056+0000 7efe9be00640 1 -- 172.21.15.152:0/2891128103 >> v1:172.21.15.152:6789/0 conn(0x7efe9c0b4870 legacy=0x7efe9c0cf430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:50.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.056+0000 7efe9be00640 1 --2- 172.21.15.152:0/2891128103 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efe840039b0 0x7efe84003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:50.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.056+0000 7efe9be00640 1 --2- 172.21.15.152:0/2891128103 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efe84004800 0x7efe84004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:50.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.056+0000 7efe9be00640 1 --2- 172.21.15.152:0/2891128103 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efe84005210 0x7efe8400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:50.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.056+0000 7efe9be00640 1 -- 172.21.15.152:0/2891128103 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7efe9c0ce140 con 0x7efe840039b0 2024-04-20T11:35:50.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.056+0000 7efe9be00640 1 -- 172.21.15.152:0/2891128103 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7efe8400c140 con 0x7efe84005210 2024-04-20T11:35:50.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.056+0000 7efe9be00640 1 -- 172.21.15.152:0/2891128103 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7efe8400c340 con 0x7efe84004800 2024-04-20T11:35:50.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.056+0000 7efea1400640 1 --2- 172.21.15.152:0/2891128103 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efe840039b0 0x7efe84003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:50.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.056+0000 7efea0a00640 1 --2- 172.21.15.152:0/2891128103 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efe84004800 0x7efe84004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:50.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.056+0000 7efea1e00640 1 --2- 172.21.15.152:0/2891128103 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efe84005210 0x7efe8400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:50.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.057+0000 7efea1400640 1 -- 172.21.15.152:0/2891128103 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efe84004800 msgr2=0x7efe84004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:50.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.057+0000 7efea1400640 1 --2- 172.21.15.152:0/2891128103 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efe84004800 0x7efe84004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:50.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.057+0000 7efea1400640 1 -- 172.21.15.152:0/2891128103 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efe84005210 msgr2=0x7efe8400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:50.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.057+0000 7efea1400640 1 --2- 172.21.15.152:0/2891128103 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efe84005210 0x7efe8400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:50.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.057+0000 7efea1400640 1 -- 172.21.15.152:0/2891128103 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe84003520 con 0x7efe840039b0 2024-04-20T11:35:50.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.057+0000 7efea1e00640 1 --2- 172.21.15.152:0/2891128103 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efe84005210 0x7efe8400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:35:50.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.057+0000 7efea1e00640 1 -- 172.21.15.152:0/2891128103 reap_dead start 2024-04-20T11:35:50.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.058+0000 7efea1400640 1 --2- 172.21.15.152:0/2891128103 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efe840039b0 0x7efe84003df0 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7efe90001750 tx=0x7efe84005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c258789bf34dc749 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:50.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.058+0000 7efe9be00640 1 -- 172.21.15.152:0/2891128103 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe90001370 con 0x7efe840039b0 2024-04-20T11:35:50.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.058+0000 7efe9be00640 1 -- 172.21.15.152:0/2891128103 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe900052d0 con 0x7efe840039b0 2024-04-20T11:35:50.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.058+0000 7efe9be00640 1 -- 172.21.15.152:0/2891128103 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe90003830 con 0x7efe840039b0 2024-04-20T11:35:50.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.059+0000 7efea2800640 1 -- 172.21.15.152:0/2891128103 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efe840039b0 msgr2=0x7efe84003df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:50.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.059+0000 7efea2800640 1 --2- 172.21.15.152:0/2891128103 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efe840039b0 0x7efe84003df0 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7efe90001750 tx=0x7efe84005830 comp rx=0 tx=0).stop 2024-04-20T11:35:50.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.059+0000 7efea2800640 1 -- 172.21.15.152:0/2891128103 shutdown_connections 2024-04-20T11:35:50.061 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.059+0000 7efea2800640 1 --2- 172.21.15.152:0/2891128103 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efe840039b0 0x7efe84003df0 unknown :-1 s=CLOSED pgs=1878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:50.061 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.059+0000 7efea2800640 1 -- 172.21.15.152:0/2891128103 >> 172.21.15.152:0/2891128103 conn(0x7efe9c0d0260 msgr2=0x7efe9c0d0670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:50.061 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.059+0000 7efea2800640 1 -- 172.21.15.152:0/2891128103 shutdown_connections 2024-04-20T11:35:50.061 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.059+0000 7efea2800640 1 -- 172.21.15.152:0/2891128103 wait complete. 2024-04-20T11:35:50.063 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.061+0000 7efea2800640 1 Processor -- start 2024-04-20T11:35:50.063 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.061+0000 7efea2800640 1 -- start start 2024-04-20T11:35:50.063 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.062+0000 7efea2800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efe840039b0 0x7efe9c09b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:50.063 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.062+0000 7efea2800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efe9c14b690 0x7efe9c09d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:50.063 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.062+0000 7efea2800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efe9c158700 0x7efe9c158b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:50.063 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.062+0000 7efea2800640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7efe9c0cef10 con 0x7efe840039b0 2024-04-20T11:35:50.064 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.062+0000 7efea2800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7efe9c0cec30 con 0x7efe9c158700 2024-04-20T11:35:50.064 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.062+0000 7efea2800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7efe9c073b60 con 0x7efe9c14b690 2024-04-20T11:35:50.064 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.062+0000 7efea1400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efe840039b0 0x7efe9c09b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:50.064 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.062+0000 7efea1400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efe840039b0 0x7efe9c09b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:60576/0 (socket says 172.21.15.152:60576) 2024-04-20T11:35:50.064 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.062+0000 7efea1400640 1 -- 172.21.15.152:0/1526055069 learned_addr learned my addr 172.21.15.152:0/1526055069 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:50.064 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.063+0000 7efea0a00640 1 --2- 172.21.15.152:0/1526055069 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efe9c14b690 0x7efe9c09d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:50.064 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.063+0000 7efea1e00640 1 --2- 172.21.15.152:0/1526055069 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efe9c158700 0x7efe9c158b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:50.064 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.063+0000 7efea1400640 1 -- 172.21.15.152:0/1526055069 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efe9c14b690 msgr2=0x7efe9c09d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:50.064 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.063+0000 7efea1400640 1 --2- 172.21.15.152:0/1526055069 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efe9c14b690 0x7efe9c09d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:50.064 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.063+0000 7efea1400640 1 -- 172.21.15.152:0/1526055069 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efe9c158700 msgr2=0x7efe9c158b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:50.064 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.063+0000 7efea1400640 1 --2- 172.21.15.152:0/1526055069 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efe9c158700 0x7efe9c158b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:50.064 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.063+0000 7efea1400640 1 -- 172.21.15.152:0/1526055069 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe9c1f8360 con 0x7efe840039b0 2024-04-20T11:35:50.065 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.063+0000 7efea0a00640 1 --2- 172.21.15.152:0/1526055069 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efe9c14b690 0x7efe9c09d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:35:50.065 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.063+0000 7efea1400640 1 --2- 172.21.15.152:0/1526055069 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efe840039b0 0x7efe9c09b410 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7efe90005040 tx=0x7efe90001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:50.065 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.063+0000 7efe99600640 1 -- 172.21.15.152:0/1526055069 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe90041020 con 0x7efe840039b0 2024-04-20T11:35:50.065 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.063+0000 7efe99600640 1 -- 172.21.15.152:0/1526055069 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe90005480 con 0x7efe840039b0 2024-04-20T11:35:50.065 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.063+0000 7efe99600640 1 -- 172.21.15.152:0/1526055069 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe90007a30 con 0x7efe840039b0 2024-04-20T11:35:50.065 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.063+0000 7efea2800640 1 -- 172.21.15.152:0/1526055069 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe9c1f8640 con 0x7efe840039b0 2024-04-20T11:35:50.065 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.064+0000 7efea2800640 1 -- 172.21.15.152:0/1526055069 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe9c1f8c00 con 0x7efe840039b0 2024-04-20T11:35:50.065 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.065+0000 7efea2800640 1 -- 172.21.15.152:0/1526055069 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe9c06ea30 con 0x7efe840039b0 2024-04-20T11:35:50.066 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.066+0000 7efe99600640 1 -- 172.21.15.152:0/1526055069 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 22) ==== 91601+0+0 (secure 0 0 0) 0x7efe90044020 con 0x7efe840039b0 2024-04-20T11:35:50.066 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.067+0000 7efe99600640 1 --2- 172.21.15.152:0/1526055069 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7efe8006cf20 0x7efe8006f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:50.066 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.067+0000 7efe99600640 1 -- 172.21.15.152:0/1526055069 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5709+0+0 (secure 0 0 0) 0x7efe900ae100 con 0x7efe840039b0 2024-04-20T11:35:50.067 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.068+0000 7efea0a00640 1 --2- 172.21.15.152:0/1526055069 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7efe8006cf20 0x7efe8006f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:50.068 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.069+0000 7efea0a00640 1 --2- 172.21.15.152:0/1526055069 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7efe8006cf20 0x7efe8006f3e0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7efe9c0ae850 tx=0x7efe8c000f90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:50.070 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.071+0000 7efe99600640 1 -- 172.21.15.152:0/1526055069 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7efe9004b030 con 0x7efe840039b0 2024-04-20T11:35:50.277 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.278+0000 7efea2800640 1 -- 172.21.15.152:0/1526055069 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7efe9c1f8e40 con 0x7efe840039b0 2024-04-20T11:35:50.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.278+0000 7efe99600640 1 -- 172.21.15.152:0/1526055069 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v131) ==== 80+0+0 (secure 0 0 0) 0x7efe900303c0 con 0x7efe840039b0 2024-04-20T11:35:50.278 INFO:teuthology.orchestra.run.smithi152.stderr:listed 0 entries 2024-04-20T11:35:50.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.281+0000 7efea2800640 1 -- 172.21.15.152:0/1526055069 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7efe8006cf20 msgr2=0x7efe8006f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:50.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.281+0000 7efea2800640 1 --2- 172.21.15.152:0/1526055069 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7efe8006cf20 0x7efe8006f3e0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7efe9c0ae850 tx=0x7efe8c000f90 comp rx=0 tx=0).stop 2024-04-20T11:35:50.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.281+0000 7efea2800640 1 -- 172.21.15.152:0/1526055069 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efe840039b0 msgr2=0x7efe9c09b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:50.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.281+0000 7efea2800640 1 --2- 172.21.15.152:0/1526055069 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efe840039b0 0x7efe9c09b410 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7efe90005040 tx=0x7efe90001120 comp rx=0 tx=0).stop 2024-04-20T11:35:50.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.281+0000 7efea2800640 1 -- 172.21.15.152:0/1526055069 shutdown_connections 2024-04-20T11:35:50.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.281+0000 7efea2800640 1 --2- 172.21.15.152:0/1526055069 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7efe8006cf20 0x7efe8006f3e0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:50.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.281+0000 7efea2800640 1 --2- 172.21.15.152:0/1526055069 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efe9c158700 0x7efe9c158b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:50.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.281+0000 7efea2800640 1 --2- 172.21.15.152:0/1526055069 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efe9c14b690 0x7efe9c09d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:50.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.281+0000 7efea2800640 1 --2- 172.21.15.152:0/1526055069 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efe840039b0 0x7efe9c09b410 unknown :-1 s=CLOSED pgs=1879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:50.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.281+0000 7efea2800640 1 -- 172.21.15.152:0/1526055069 >> 172.21.15.152:0/1526055069 conn(0x7efe9c0d0260 msgr2=0x7efe9c15a350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:50.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.281+0000 7efea2800640 1 -- 172.21.15.152:0/1526055069 shutdown_connections 2024-04-20T11:35:50.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.281+0000 7efea2800640 1 -- 172.21.15.152:0/1526055069 wait complete. 2024-04-20T11:35:50.299 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi175.front.sepia.ceph.com... 2024-04-20T11:35:50.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:35:50.300 DEBUG:teuthology.orchestra.run.smithi175:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-04-20T11:35:50.385 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi175.stderr:ceph-fuse[35123]: fuse finished with error 0 and tester_r 0 2024-04-20T11:35:50.391 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi175.front.sepia.ceph.com 2024-04-20T11:35:50.391 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:35:50.391 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-20T11:35:50.559 INFO:tasks.cephfs.mount:Terminating background process 2024-04-20T11:35:50.559 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-20T11:35:50.559 DEBUG:teuthology.orchestra.run.smithi152:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:35:50.618 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-20T11:35:50.618 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi152.front.sepia.ceph.com 2024-04-20T11:35:50.618 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:35:50.618 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-20T11:35:50.666 INFO:teuthology.orchestra.run.smithi152.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-20T11:35:50.667 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T11:35:50.667 INFO:tasks.cephfs.mount:Terminating background process 2024-04-20T11:35:50.667 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T11:35:50.668 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:35:50.725 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-20T11:35:50.725 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi175.front.sepia.ceph.com 2024-04-20T11:35:50.726 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:35:50.726 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-20T11:35:50.778 INFO:teuthology.orchestra.run.smithi175.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-20T11:35:50.779 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T11:35:50.780 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:35:50.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.989+0000 7f85fb600640 1 Processor -- start 2024-04-20T11:35:50.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.990+0000 7f85fb600640 1 -- start start 2024-04-20T11:35:50.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.991+0000 7f85fb600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f85f40b8070 0x7f85f409aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:50.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.991+0000 7f85fb600640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f85f40d13a0 0x7f85f409af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:50.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.991+0000 7f85fb600640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85f40cd630 con 0x7f85f409b670 2024-04-20T11:35:50.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.991+0000 7f85fb600640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f85f41603f0 con 0x7f85f40d13a0 2024-04-20T11:35:50.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.991+0000 7f85fb600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f85f40b6430 con 0x7f85f40b8070 2024-04-20T11:35:50.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.991+0000 7f85f3e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f85f40d13a0 0x7f85f409af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:50.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.992+0000 7f85f3e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f85f40d13a0 0x7f85f409af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:60580/0 (socket says 172.21.15.152:60580) 2024-04-20T11:35:50.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.992+0000 7f85f3e00640 1 -- 172.21.15.152:0/2210603440 learned_addr learned my addr 172.21.15.152:0/2210603440 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:50.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.992+0000 7f85f8c00640 1 --2- 172.21.15.152:0/2210603440 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f85f40b8070 0x7f85f409aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:50.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.992+0000 7f85f3e00640 1 -- 172.21.15.152:0/2210603440 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f85f40b8070 msgr2=0x7f85f409aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:50.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.992+0000 7f85f3e00640 1 --2- 172.21.15.152:0/2210603440 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f85f40b8070 0x7f85f409aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:50.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.992+0000 7f85f3e00640 1 -- 172.21.15.152:0/2210603440 >> v1:172.21.15.152:6789/0 conn(0x7f85f409b670 legacy=0x7f85f41605e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:50.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.992+0000 7f85f3e00640 1 -- 172.21.15.152:0/2210603440 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85f40cd910 con 0x7f85f40d13a0 2024-04-20T11:35:50.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.993+0000 7f85f3e00640 1 --2- 172.21.15.152:0/2210603440 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f85f40d13a0 0x7f85f409af80 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f85e4009e50 tx=0x7f85e402bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2adc2300898c0ee8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:50.995 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.993+0000 7f85f3400640 1 -- 172.21.15.152:0/2210603440 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e402cb00 con 0x7f85f40d13a0 2024-04-20T11:35:50.995 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.993+0000 7f85f3400640 1 -- 172.21.15.152:0/2210603440 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85e402cca0 con 0x7f85f40d13a0 2024-04-20T11:35:50.995 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.994+0000 7f85f3400640 1 -- 172.21.15.152:0/2210603440 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e4034610 con 0x7f85f40d13a0 2024-04-20T11:35:50.995 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.994+0000 7f85fb600640 1 -- 172.21.15.152:0/2210603440 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f85f40d13a0 msgr2=0x7f85f409af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:50.995 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.994+0000 7f85fb600640 1 --2- 172.21.15.152:0/2210603440 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f85f40d13a0 0x7f85f409af80 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f85e4009e50 tx=0x7f85e402bf60 comp rx=0 tx=0).stop 2024-04-20T11:35:50.995 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.994+0000 7f85fb600640 1 -- 172.21.15.152:0/2210603440 shutdown_connections 2024-04-20T11:35:50.995 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.994+0000 7f85fb600640 1 --2- 172.21.15.152:0/2210603440 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f85f40d13a0 0x7f85f409af80 unknown :-1 s=CLOSED pgs=1881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:50.995 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.994+0000 7f85fb600640 1 --2- 172.21.15.152:0/2210603440 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f85f40b8070 0x7f85f409aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:50.995 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.994+0000 7f85fb600640 1 -- 172.21.15.152:0/2210603440 >> 172.21.15.152:0/2210603440 conn(0x7f85f40b40c0 msgr2=0x7f85f40b44d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:50.995 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.994+0000 7f85fb600640 1 -- 172.21.15.152:0/2210603440 shutdown_connections 2024-04-20T11:35:50.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.994+0000 7f85fb600640 1 -- 172.21.15.152:0/2210603440 wait complete. 2024-04-20T11:35:50.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.996+0000 7f85fb600640 1 Processor -- start 2024-04-20T11:35:50.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.996+0000 7f85fb600640 1 -- start start 2024-04-20T11:35:50.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.997+0000 7f85fb600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85f409b670 0x7f85f41f5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:50.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.997+0000 7f85fb600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f85f40b8070 0x7f85f40cbc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:50.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.997+0000 7f85fb600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f85f40d13a0 0x7f85f41516a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:50.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.997+0000 7f85fb600640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f85f40ce1b0 con 0x7f85f40b8070 2024-04-20T11:35:50.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.997+0000 7f85fb600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f85f40cded0 con 0x7f85f409b670 2024-04-20T11:35:50.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.997+0000 7f85fb600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f85f40ce6d0 con 0x7f85f40d13a0 2024-04-20T11:35:50.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.998+0000 7f85f3e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f85f40b8070 0x7f85f40cbc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:50.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.998+0000 7f85f3e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f85f40b8070 0x7f85f40cbc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:60592/0 (socket says 172.21.15.152:60592) 2024-04-20T11:35:50.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.998+0000 7f85f3e00640 1 -- 172.21.15.152:0/4279971147 learned_addr learned my addr 172.21.15.152:0/4279971147 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:50.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.998+0000 7f85f8c00640 1 --2- 172.21.15.152:0/4279971147 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85f409b670 0x7f85f41f5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:50.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.998+0000 7f85f3e00640 1 -- 172.21.15.152:0/4279971147 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f85f40d13a0 msgr2=0x7f85f41516a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:50.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.998+0000 7f85f3e00640 1 --2- 172.21.15.152:0/4279971147 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f85f40d13a0 0x7f85f41516a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:51.000 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.998+0000 7f85f3e00640 1 -- 172.21.15.152:0/4279971147 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85f409b670 msgr2=0x7f85f41f5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:51.000 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.998+0000 7f85f3e00640 1 --2- 172.21.15.152:0/4279971147 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85f409b670 0x7f85f41f5a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:51.000 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.998+0000 7f85f3e00640 1 -- 172.21.15.152:0/4279971147 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85f41f4c90 con 0x7f85f40b8070 2024-04-20T11:35:51.000 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.998+0000 7f85f3e00640 1 --2- 172.21.15.152:0/4279971147 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f85f40b8070 0x7f85f40cbc90 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f85e4036040 tx=0x7f85e4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:51.000 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.999+0000 7f85f0c00640 1 -- 172.21.15.152:0/4279971147 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e4043020 con 0x7f85f40b8070 2024-04-20T11:35:51.000 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.999+0000 7f85f0c00640 1 -- 172.21.15.152:0/4279971147 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85e402ce50 con 0x7f85f40b8070 2024-04-20T11:35:51.000 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.999+0000 7f85f0c00640 1 -- 172.21.15.152:0/4279971147 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e4009510 con 0x7f85f40b8070 2024-04-20T11:35:51.000 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.999+0000 7f85fb600640 1 -- 172.21.15.152:0/4279971147 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85f40ca5b0 con 0x7f85f40b8070 2024-04-20T11:35:51.000 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:50.999+0000 7f85fb600640 1 -- 172.21.15.152:0/4279971147 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85f41f5220 con 0x7f85f40b8070 2024-04-20T11:35:51.001 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.000+0000 7f85fb600640 1 -- 172.21.15.152:0/4279971147 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85f40689f0 con 0x7f85f40b8070 2024-04-20T11:35:51.001 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.001+0000 7f85f0c00640 1 -- 172.21.15.152:0/4279971147 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 22) ==== 91601+0+0 (secure 0 0 0) 0x7f85e4046020 con 0x7f85f40b8070 2024-04-20T11:35:51.002 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.002+0000 7f85f0c00640 1 --2- 172.21.15.152:0/4279971147 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f85d006cf20 0x7f85d006f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:51.002 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.002+0000 7f85f0c00640 1 -- 172.21.15.152:0/4279971147 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5709+0+0 (secure 0 0 0) 0x7f85e40aed80 con 0x7f85f40b8070 2024-04-20T11:35:51.002 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.003+0000 7f85f8c00640 1 --2- 172.21.15.152:0/4279971147 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f85d006cf20 0x7f85d006f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:51.003 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.004+0000 7f85f8c00640 1 --2- 172.21.15.152:0/4279971147 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f85d006cf20 0x7f85d006f3e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f85dc006fd0 tx=0x7f85dc008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:51.006 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.007+0000 7f85f0c00640 1 -- 172.21.15.152:0/4279971147 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f85e404d030 con 0x7f85f40b8070 2024-04-20T11:35:51.243 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.243+0000 7f85fb600640 1 -- 172.21.15.152:0/4279971147 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f85f406fda0 con 0x7f85f40b8070 2024-04-20T11:35:51.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.244+0000 7f85f0c00640 1 -- 172.21.15.152:0/4279971147 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) ==== 94+0+5212 (secure 0 0 0) 0x7f85e407c900 con 0x7f85f40b8070 2024-04-20T11:35:51.244 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:35:51.244 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":87,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6062,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1303172876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6833","nonce":1303172876},{"type":"v1","addr":"172.21.15.175:6835","nonce":1303172876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6667,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6834/1975208598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1975208598},{"type":"v1","addr":"172.21.15.175:6834","nonce":1975208598}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6670,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6834/287411071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":287411071},{"type":"v1","addr":"172.21.15.152:6834","nonce":287411071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T11:35:19.231687+0000","modified":"2024-04-20T11:35:22.597626+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8919},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8919":{"gid":8919,"name":"a","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6835/1884827433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6833","nonce":1884827433},{"type":"v1","addr":"172.21.15.152:6835","nonce":1884827433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8919,"qdb_cluster":[8919]},"id":10}]} 2024-04-20T11:35:51.244 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 87 2024-04-20T11:35:51.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.247+0000 7f85fb600640 1 -- 172.21.15.152:0/4279971147 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f85d006cf20 msgr2=0x7f85d006f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:51.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.247+0000 7f85fb600640 1 --2- 172.21.15.152:0/4279971147 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f85d006cf20 0x7f85d006f3e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f85dc006fd0 tx=0x7f85dc008040 comp rx=0 tx=0).stop 2024-04-20T11:35:51.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.247+0000 7f85fb600640 1 -- 172.21.15.152:0/4279971147 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f85f40b8070 msgr2=0x7f85f40cbc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:51.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.247+0000 7f85fb600640 1 --2- 172.21.15.152:0/4279971147 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f85f40b8070 0x7f85f40cbc90 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f85e4036040 tx=0x7f85e4004a40 comp rx=0 tx=0).stop 2024-04-20T11:35:51.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.247+0000 7f85fb600640 1 -- 172.21.15.152:0/4279971147 shutdown_connections 2024-04-20T11:35:51.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.247+0000 7f85fb600640 1 --2- 172.21.15.152:0/4279971147 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f85d006cf20 0x7f85d006f3e0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:51.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.247+0000 7f85fb600640 1 --2- 172.21.15.152:0/4279971147 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f85f40d13a0 0x7f85f41516a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:51.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.247+0000 7f85fb600640 1 --2- 172.21.15.152:0/4279971147 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f85f40b8070 0x7f85f40cbc90 unknown :-1 s=CLOSED pgs=1882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:51.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.247+0000 7f85fb600640 1 --2- 172.21.15.152:0/4279971147 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85f409b670 0x7f85f41f5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:51.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.247+0000 7f85fb600640 1 -- 172.21.15.152:0/4279971147 >> 172.21.15.152:0/4279971147 conn(0x7f85f40b40c0 msgr2=0x7f85f40b5720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:51.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.247+0000 7f85fb600640 1 -- 172.21.15.152:0/4279971147 shutdown_connections 2024-04-20T11:35:51.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.247+0000 7f85fb600640 1 -- 172.21.15.152:0/4279971147 wait complete. 2024-04-20T11:35:51.266 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:35:51.466 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.466+0000 7f7277e00640 1 Processor -- start 2024-04-20T11:35:51.466 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.466+0000 7f7277e00640 1 -- start start 2024-04-20T11:35:51.471 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.468+0000 7f7277e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f72780723e0 0x7f727815d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:51.471 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.468+0000 7f7277e00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72780ce8c0 con 0x7f7278160b90 2024-04-20T11:35:51.471 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.468+0000 7f7277e00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72781600b0 con 0x7f727815d1e0 2024-04-20T11:35:51.471 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.468+0000 7f7277e00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f7278160390 con 0x7f72780723e0 2024-04-20T11:35:51.471 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.468+0000 7f726e000640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f7278160b90 0x7f7278071cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:60640/0 (socket says 172.21.15.152:60640) 2024-04-20T11:35:51.471 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.468+0000 7f726e000640 1 -- 172.21.15.152:0/781327285 learned_addr learned my addr 172.21.15.152:0/781327285 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:35:51.471 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.469+0000 7f7277400640 1 --2- 172.21.15.152:0/781327285 >> v2:172.21.15.175:3300/0 conn(0x7f72780723e0 0x7f727815d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:51.471 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.469+0000 7f7276000640 1 -- 172.21.15.152:0/781327285 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3153671262 0 0) 0x7f72780ce8c0 con 0x7f7278160b90 2024-04-20T11:35:51.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.469+0000 7f7276000640 1 -- 172.21.15.152:0/781327285 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f725c003520 con 0x7f7278160b90 2024-04-20T11:35:51.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.469+0000 7f7276000640 1 -- 172.21.15.152:0/781327285 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1556570540 0 0) 0x7f725c003520 con 0x7f7278160b90 2024-04-20T11:35:51.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.469+0000 7f7276000640 1 -- 172.21.15.152:0/781327285 >> v2:172.21.15.175:3300/0 conn(0x7f72780723e0 msgr2=0x7f727815d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:51.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.469+0000 7f7276000640 1 --2- 172.21.15.152:0/781327285 >> v2:172.21.15.175:3300/0 conn(0x7f72780723e0 0x7f727815d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:51.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.469+0000 7f7276000640 1 -- 172.21.15.152:0/781327285 >> v1:172.21.15.175:6789/0 conn(0x7f727815d1e0 legacy=0x7f72780cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:51.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.469+0000 7f7276000640 1 -- 172.21.15.152:0/781327285 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7278160670 con 0x7f7278160b90 2024-04-20T11:35:51.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.469+0000 7f7276000640 1 -- 172.21.15.152:0/781327285 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f7260001670 con 0x7f7278160b90 2024-04-20T11:35:51.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.469+0000 7f7276000640 1 -- 172.21.15.152:0/781327285 >> v1:172.21.15.152:6789/0 conn(0x7f7278160b90 legacy=0x7f7278071cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:51.473 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.469+0000 7f7276000640 1 --2- 172.21.15.152:0/781327285 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f725c0039b0 0x7f725c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:51.473 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.469+0000 7f7276000640 1 --2- 172.21.15.152:0/781327285 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f725c004800 0x7f725c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:51.473 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.469+0000 7f7276000640 1 --2- 172.21.15.152:0/781327285 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f725c0051c0 0x7f725c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:51.473 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.469+0000 7f7276000640 1 -- 172.21.15.152:0/781327285 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f7278160390 con 0x7f725c004800 2024-04-20T11:35:51.473 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.469+0000 7f7276000640 1 -- 172.21.15.152:0/781327285 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f725c00c060 con 0x7f725c0039b0 2024-04-20T11:35:51.473 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.469+0000 7f7276000640 1 -- 172.21.15.152:0/781327285 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f725c00c260 con 0x7f725c0051c0 2024-04-20T11:35:51.473 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.469+0000 7f726e000640 1 --2- 172.21.15.152:0/781327285 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f725c004800 0x7f725c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:51.473 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.470+0000 7f7276a00640 1 --2- 172.21.15.152:0/781327285 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f725c0039b0 0x7f725c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:51.473 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.470+0000 7f7277400640 1 --2- 172.21.15.152:0/781327285 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f725c0051c0 0x7f725c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:51.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.470+0000 7f726e000640 1 -- 172.21.15.152:0/781327285 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f725c0051c0 msgr2=0x7f725c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:51.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.470+0000 7f726e000640 1 --2- 172.21.15.152:0/781327285 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f725c0051c0 0x7f725c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:51.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.470+0000 7f726e000640 1 -- 172.21.15.152:0/781327285 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f725c0039b0 msgr2=0x7f725c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:51.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.470+0000 7f726e000640 1 --2- 172.21.15.152:0/781327285 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f725c0039b0 0x7f725c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:51.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.470+0000 7f726e000640 1 -- 172.21.15.152:0/781327285 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f725c003520 con 0x7f725c004800 2024-04-20T11:35:51.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.470+0000 7f7276a00640 1 --2- 172.21.15.152:0/781327285 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f725c0039b0 0x7f725c003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:35:51.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.470+0000 7f7277400640 1 -- 172.21.15.152:0/781327285 reap_dead start 2024-04-20T11:35:51.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.471+0000 7f726e000640 1 --2- 172.21.15.152:0/781327285 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f725c004800 0x7f725c004c80 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f726000c240 tx=0x7f725c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa6a90476aa61e2b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:51.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.471+0000 7f7276000640 1 -- 172.21.15.152:0/781327285 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7260007d10 con 0x7f725c004800 2024-04-20T11:35:51.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.471+0000 7f7276000640 1 -- 172.21.15.152:0/781327285 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7260030070 con 0x7f725c004800 2024-04-20T11:35:51.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.471+0000 7f7276000640 1 -- 172.21.15.152:0/781327285 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7260005d90 con 0x7f725c004800 2024-04-20T11:35:51.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.472+0000 7f7277e00640 1 -- 172.21.15.152:0/781327285 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f725c004800 msgr2=0x7f725c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:51.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.472+0000 7f7277e00640 1 --2- 172.21.15.152:0/781327285 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f725c004800 0x7f725c004c80 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f726000c240 tx=0x7f725c005830 comp rx=0 tx=0).stop 2024-04-20T11:35:51.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.472+0000 7f7277e00640 1 -- 172.21.15.152:0/781327285 shutdown_connections 2024-04-20T11:35:51.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.472+0000 7f7277e00640 1 --2- 172.21.15.152:0/781327285 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f725c004800 0x7f725c004c80 unknown :-1 s=CLOSED pgs=1884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:51.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.472+0000 7f7277e00640 1 -- 172.21.15.152:0/781327285 >> 172.21.15.152:0/781327285 conn(0x7f7278067ae0 msgr2=0x7f727806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:51.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.472+0000 7f7277e00640 1 -- 172.21.15.152:0/781327285 shutdown_connections 2024-04-20T11:35:51.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.472+0000 7f7277e00640 1 -- 172.21.15.152:0/781327285 wait complete. 2024-04-20T11:35:51.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.474+0000 7f7277e00640 1 Processor -- start 2024-04-20T11:35:51.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.474+0000 7f7277e00640 1 -- start start 2024-04-20T11:35:51.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.475+0000 7f7277e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f725c004800 0x7f727814ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:51.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.475+0000 7f7277e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72780af340 0x7f72780af7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:51.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.475+0000 7f7277e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f72780b0300 0x7f72780b0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:51.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.475+0000 7f7277e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f727815d740 con 0x7f725c004800 2024-04-20T11:35:51.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.475+0000 7f7277e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f727815d5c0 con 0x7f72780af340 2024-04-20T11:35:51.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.475+0000 7f7277e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f727815d8c0 con 0x7f72780b0300 2024-04-20T11:35:51.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.476+0000 7f7276a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f725c004800 0x7f727814ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:51.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.476+0000 7f7276a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f725c004800 0x7f727814ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:60620/0 (socket says 172.21.15.152:60620) 2024-04-20T11:35:51.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.476+0000 7f7276a00640 1 -- 172.21.15.152:0/1472012599 learned_addr learned my addr 172.21.15.152:0/1472012599 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:51.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.476+0000 7f7277400640 1 --2- 172.21.15.152:0/1472012599 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f72780b0300 0x7f72780b0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:51.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.476+0000 7f7276a00640 1 -- 172.21.15.152:0/1472012599 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f72780b0300 msgr2=0x7f72780b0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:51.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.476+0000 7f7276a00640 1 --2- 172.21.15.152:0/1472012599 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f72780b0300 0x7f72780b0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:51.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.476+0000 7f7276a00640 1 -- 172.21.15.152:0/1472012599 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72780af340 msgr2=0x7f72780af7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:51.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.476+0000 7f7276a00640 1 --2- 172.21.15.152:0/1472012599 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72780af340 0x7f72780af7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:51.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.476+0000 7f7276a00640 1 -- 172.21.15.152:0/1472012599 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f727814d3f0 con 0x7f725c004800 2024-04-20T11:35:51.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.476+0000 7f7277400640 1 --2- 172.21.15.152:0/1472012599 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f72780b0300 0x7f72780b0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:35:51.480 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.476+0000 7f7276a00640 1 --2- 172.21.15.152:0/1472012599 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f725c004800 0x7f727814ceb0 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f726800e4b0 tx=0x7f726800e970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:51.480 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.477+0000 7f726fe00640 1 -- 172.21.15.152:0/1472012599 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72680052e0 con 0x7f725c004800 2024-04-20T11:35:51.480 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.477+0000 7f726fe00640 1 -- 172.21.15.152:0/1472012599 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7268007020 con 0x7f725c004800 2024-04-20T11:35:51.480 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.477+0000 7f7277e00640 1 -- 172.21.15.152:0/1472012599 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7278157050 con 0x7f725c004800 2024-04-20T11:35:51.480 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.477+0000 7f726fe00640 1 -- 172.21.15.152:0/1472012599 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7268007210 con 0x7f725c004800 2024-04-20T11:35:51.480 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.477+0000 7f7277e00640 1 -- 172.21.15.152:0/1472012599 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7278157610 con 0x7f725c004800 2024-04-20T11:35:51.480 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.478+0000 7f7277e00640 1 -- 172.21.15.152:0/1472012599 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7278075e60 con 0x7f725c004800 2024-04-20T11:35:51.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.479+0000 7f726fe00640 1 -- 172.21.15.152:0/1472012599 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 22) ==== 91601+0+0 (secure 0 0 0) 0x7f726801a020 con 0x7f725c004800 2024-04-20T11:35:51.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.480+0000 7f726fe00640 1 --2- 172.21.15.152:0/1472012599 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f725406cf20 0x7f725406f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:51.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.480+0000 7f726fe00640 1 -- 172.21.15.152:0/1472012599 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5709+0+0 (secure 0 0 0) 0x7f726808bbe0 con 0x7f725c004800 2024-04-20T11:35:51.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.480+0000 7f726e000640 1 --2- 172.21.15.152:0/1472012599 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f725406cf20 0x7f725406f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:51.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.482+0000 7f726e000640 1 --2- 172.21.15.152:0/1472012599 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f725406cf20 0x7f725406f3e0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f7260034000 tx=0x7f7260007000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:51.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.484+0000 7f726fe00640 1 -- 172.21.15.152:0/1472012599 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f727814d3f0 con 0x7f725c004800 2024-04-20T11:35:51.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.686+0000 7f7277e00640 1 -- 172.21.15.152:0/1472012599 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f72780afd30 con 0x7f725c004800 2024-04-20T11:35:51.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.689+0000 7f726fe00640 1 -- 172.21.15.152:0/1472012599 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v131) ==== 74+0+16739 (secure 0 0 0) 0x7f726801a310 con 0x7f725c004800 2024-04-20T11:35:51.689 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:35:51.689 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":131,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:35:20.781260+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:35:16.963520+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"130","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-04-20T11:35:18.215890+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:35:51.692 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.692+0000 7f7277e00640 1 -- 172.21.15.152:0/1472012599 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f725406cf20 msgr2=0x7f725406f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:51.692 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.692+0000 7f7277e00640 1 --2- 172.21.15.152:0/1472012599 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f725406cf20 0x7f725406f3e0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f7260034000 tx=0x7f7260007000 comp rx=0 tx=0).stop 2024-04-20T11:35:51.692 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.692+0000 7f7277e00640 1 -- 172.21.15.152:0/1472012599 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f725c004800 msgr2=0x7f727814ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:51.692 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.692+0000 7f7277e00640 1 --2- 172.21.15.152:0/1472012599 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f725c004800 0x7f727814ceb0 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f726800e4b0 tx=0x7f726800e970 comp rx=0 tx=0).stop 2024-04-20T11:35:51.692 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.692+0000 7f7277e00640 1 -- 172.21.15.152:0/1472012599 shutdown_connections 2024-04-20T11:35:51.693 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.692+0000 7f7277e00640 1 --2- 172.21.15.152:0/1472012599 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f725406cf20 0x7f725406f3e0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:51.693 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.692+0000 7f7277e00640 1 --2- 172.21.15.152:0/1472012599 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f72780b0300 0x7f72780b0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:51.693 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.692+0000 7f7277e00640 1 --2- 172.21.15.152:0/1472012599 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72780af340 0x7f72780af7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:51.693 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.692+0000 7f7277e00640 1 --2- 172.21.15.152:0/1472012599 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f725c004800 0x7f727814ceb0 unknown :-1 s=CLOSED pgs=1885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:51.693 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.692+0000 7f7277e00640 1 -- 172.21.15.152:0/1472012599 >> 172.21.15.152:0/1472012599 conn(0x7f7278067ae0 msgr2=0x7f72780b2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:51.693 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.693+0000 7f7277e00640 1 -- 172.21.15.152:0/1472012599 shutdown_connections 2024-04-20T11:35:51.693 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.693+0000 7f7277e00640 1 -- 172.21.15.152:0/1472012599 wait complete. 2024-04-20T11:35:51.712 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T11:26:47.246571+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 20, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T11:35:16.963520+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '130', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 21, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T11:35:18.215890+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '131', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T11:35:51.712 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T11:35:51.922 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.922+0000 7f7281200640 1 Processor -- start 2024-04-20T11:35:51.922 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.922+0000 7f7281200640 1 -- start start 2024-04-20T11:35:51.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.924+0000 7f7281200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f727c15abe0 0x7f727c06c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:51.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.924+0000 7f7281200640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f727c15f6d0 con 0x7f727c0b4f40 2024-04-20T11:35:51.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.924+0000 7f7281200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f727c0b3ff0 con 0x7f727c070930 2024-04-20T11:35:51.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.924+0000 7f7281200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f727c0b42d0 con 0x7f727c15abe0 2024-04-20T11:35:51.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.924+0000 7f7279e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f727c15abe0 0x7f727c06c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:51.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.924+0000 7f7271400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f727c0b4f40 0x7f727c15e8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:60648/0 (socket says 172.21.15.152:60648) 2024-04-20T11:35:51.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.924+0000 7f7271400640 1 -- 172.21.15.152:0/267925366 learned_addr learned my addr 172.21.15.152:0/267925366 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:35:51.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.925+0000 7f7279e00640 1 -- 172.21.15.152:0/267925366 >> v1:172.21.15.175:6789/0 conn(0x7f727c070930 legacy=0x7f727c15efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:51.925 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.925+0000 7f7279e00640 1 -- 172.21.15.152:0/267925366 >> v1:172.21.15.152:6789/0 conn(0x7f727c0b4f40 legacy=0x7f727c15e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:51.925 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.925+0000 7f7279e00640 1 -- 172.21.15.152:0/267925366 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f727c0b45b0 con 0x7f727c15abe0 2024-04-20T11:35:51.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.926+0000 7f7279e00640 1 --2- 172.21.15.152:0/267925366 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f727c15abe0 0x7f727c06c990 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f726c00a940 tx=0x7f726c02cd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=341c96f33182f5a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:51.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.927+0000 7f7279400640 1 -- 172.21.15.152:0/267925366 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f726c03a070 con 0x7f727c15abe0 2024-04-20T11:35:51.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.927+0000 7f7279400640 1 -- 172.21.15.152:0/267925366 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f726c02db40 con 0x7f727c15abe0 2024-04-20T11:35:51.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.927+0000 7f7279400640 1 -- 172.21.15.152:0/267925366 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f726c00b040 con 0x7f727c15abe0 2024-04-20T11:35:51.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.927+0000 7f7281200640 1 -- 172.21.15.152:0/267925366 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f727c15abe0 msgr2=0x7f727c06c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:51.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.927+0000 7f7281200640 1 --2- 172.21.15.152:0/267925366 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f727c15abe0 0x7f727c06c990 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f726c00a940 tx=0x7f726c02cd90 comp rx=0 tx=0).stop 2024-04-20T11:35:51.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.928+0000 7f7281200640 1 -- 172.21.15.152:0/267925366 shutdown_connections 2024-04-20T11:35:51.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.928+0000 7f7281200640 1 --2- 172.21.15.152:0/267925366 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f727c15abe0 0x7f727c06c990 unknown :-1 s=CLOSED pgs=1887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:51.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.928+0000 7f7281200640 1 -- 172.21.15.152:0/267925366 >> 172.21.15.152:0/267925366 conn(0x7f727c072620 msgr2=0x7f727c072a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:51.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.928+0000 7f7281200640 1 -- 172.21.15.152:0/267925366 shutdown_connections 2024-04-20T11:35:51.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.928+0000 7f7281200640 1 -- 172.21.15.152:0/267925366 wait complete. 2024-04-20T11:35:51.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.929+0000 7f7281200640 1 Processor -- start 2024-04-20T11:35:51.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.930+0000 7f7281200640 1 -- start start 2024-04-20T11:35:51.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.930+0000 7f7281200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f727c070930 0x7f727c145f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:51.931 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.930+0000 7f7281200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f727c0b4f40 0x7f727c1e9220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:51.931 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.931+0000 7f7281200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f727c15abe0 0x7f727c1e6860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:51.931 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.931+0000 7f7281200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f727c0aef40 con 0x7f727c070930 2024-04-20T11:35:51.931 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.931+0000 7f7281200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f727c0b3ca0 con 0x7f727c15abe0 2024-04-20T11:35:51.931 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.931+0000 7f7281200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f727c06efd0 con 0x7f727c0b4f40 2024-04-20T11:35:51.931 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.931+0000 7f7279e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f727c070930 0x7f727c145f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:51.931 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.931+0000 7f7279e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f727c070930 0x7f727c145f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:60634/0 (socket says 172.21.15.152:60634) 2024-04-20T11:35:51.931 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.931+0000 7f7279e00640 1 -- 172.21.15.152:0/4127845167 learned_addr learned my addr 172.21.15.152:0/4127845167 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:51.931 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.931+0000 7f7279e00640 1 -- 172.21.15.152:0/4127845167 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f727c0b4f40 msgr2=0x7f727c1e9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:51.931 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.931+0000 7f7279e00640 1 --2- 172.21.15.152:0/4127845167 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f727c0b4f40 0x7f727c1e9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:51.931 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.931+0000 7f7279e00640 1 -- 172.21.15.152:0/4127845167 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f727c15abe0 msgr2=0x7f727c1e6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:51.931 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.931+0000 7f7279e00640 1 --2- 172.21.15.152:0/4127845167 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f727c15abe0 0x7f727c1e6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:51.931 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.931+0000 7f7279e00640 1 -- 172.21.15.152:0/4127845167 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f727c1e9760 con 0x7f727c070930 2024-04-20T11:35:51.932 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.931+0000 7f7279e00640 1 --2- 172.21.15.152:0/4127845167 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f727c070930 0x7f727c145f40 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f726c00a940 tx=0x7f726c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:51.932 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.932+0000 7f7272a00640 1 -- 172.21.15.152:0/4127845167 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f726c042020 con 0x7f727c070930 2024-04-20T11:35:51.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.932+0000 7f7272a00640 1 -- 172.21.15.152:0/4127845167 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f726c03e410 con 0x7f727c070930 2024-04-20T11:35:51.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.932+0000 7f7272a00640 1 -- 172.21.15.152:0/4127845167 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f726c03a040 con 0x7f727c070930 2024-04-20T11:35:51.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.932+0000 7f7281200640 1 -- 172.21.15.152:0/4127845167 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f727c1465a0 con 0x7f727c070930 2024-04-20T11:35:51.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.932+0000 7f7281200640 1 -- 172.21.15.152:0/4127845167 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f727c146b60 con 0x7f727c070930 2024-04-20T11:35:51.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.933+0000 7f7281200640 1 -- 172.21.15.152:0/4127845167 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f727c0689f0 con 0x7f727c070930 2024-04-20T11:35:51.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.934+0000 7f7272a00640 1 -- 172.21.15.152:0/4127845167 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 22) ==== 91601+0+0 (secure 0 0 0) 0x7f726c045020 con 0x7f727c070930 2024-04-20T11:35:51.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.935+0000 7f7272a00640 1 --2- 172.21.15.152:0/4127845167 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f725806cf20 0x7f725806f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:51.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.935+0000 7f7272a00640 1 -- 172.21.15.152:0/4127845167 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5709+0+0 (secure 0 0 0) 0x7f726c0ae540 con 0x7f727c070930 2024-04-20T11:35:51.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.936+0000 7f7271400640 1 --2- 172.21.15.152:0/4127845167 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f725806cf20 0x7f725806f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:51.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.937+0000 7f7271400640 1 --2- 172.21.15.152:0/4127845167 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f725806cf20 0x7f725806f3e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f7264000f60 tx=0x7f7264007a80 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:51.939 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:51.939+0000 7f7272a00640 1 -- 172.21.15.152:0/4127845167 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f726c04c030 con 0x7f727c070930 2024-04-20T11:35:52.139 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.139+0000 7f7281200640 1 -- 172.21.15.152:0/4127845167 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f727c15a230 con 0x7f727c070930 2024-04-20T11:35:52.140 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.140+0000 7f7272a00640 1 -- 172.21.15.152:0/4127845167 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v131) ==== 93+0+10 (secure 0 0 0) 0x7f727c1e9760 con 0x7f727c070930 2024-04-20T11:35:52.140 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 1 2024-04-20T11:35:52.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.143+0000 7f7281200640 1 -- 172.21.15.152:0/4127845167 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f725806cf20 msgr2=0x7f725806f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:52.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.143+0000 7f7281200640 1 --2- 172.21.15.152:0/4127845167 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f725806cf20 0x7f725806f3e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f7264000f60 tx=0x7f7264007a80 comp rx=0 tx=0).stop 2024-04-20T11:35:52.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.143+0000 7f7281200640 1 -- 172.21.15.152:0/4127845167 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f727c070930 msgr2=0x7f727c145f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:52.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.143+0000 7f7281200640 1 --2- 172.21.15.152:0/4127845167 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f727c070930 0x7f727c145f40 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f726c00a940 tx=0x7f726c0046c0 comp rx=0 tx=0).stop 2024-04-20T11:35:52.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.143+0000 7f7281200640 1 -- 172.21.15.152:0/4127845167 shutdown_connections 2024-04-20T11:35:52.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.143+0000 7f7281200640 1 --2- 172.21.15.152:0/4127845167 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f725806cf20 0x7f725806f3e0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:52.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.143+0000 7f7281200640 1 --2- 172.21.15.152:0/4127845167 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f727c15abe0 0x7f727c1e6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:52.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.143+0000 7f7281200640 1 --2- 172.21.15.152:0/4127845167 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f727c0b4f40 0x7f727c1e9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:52.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.143+0000 7f7281200640 1 --2- 172.21.15.152:0/4127845167 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f727c070930 0x7f727c145f40 unknown :-1 s=CLOSED pgs=1888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:52.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.143+0000 7f7281200640 1 -- 172.21.15.152:0/4127845167 >> 172.21.15.152:0/4127845167 conn(0x7f727c072620 msgr2=0x7f727c158500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:52.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.143+0000 7f7281200640 1 -- 172.21.15.152:0/4127845167 shutdown_connections 2024-04-20T11:35:52.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.144+0000 7f7281200640 1 -- 172.21.15.152:0/4127845167 wait complete. 2024-04-20T11:35:52.163 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T11:35:52.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.367+0000 7f2f04a00640 1 Processor -- start 2024-04-20T11:35:52.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.367+0000 7f2f04a00640 1 -- start start 2024-04-20T11:35:52.369 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.369+0000 7f2f04a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2f000cd8c0 0x7f2f000b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:52.369 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.369+0000 7f2f04a00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f001463e0 con 0x7f2f00146000 2024-04-20T11:35:52.369 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.369+0000 7f2f04a00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f000b3e20 con 0x7f2f000b2880 2024-04-20T11:35:52.369 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.369+0000 7f2f04a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2f000b4750 con 0x7f2f000cd8c0 2024-04-20T11:35:52.369 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.369+0000 7f2efde00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f2f00146000 0x7f2f000b2170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:60658/0 (socket says 172.21.15.152:60658) 2024-04-20T11:35:52.369 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.369+0000 7f2efde00640 1 -- 172.21.15.152:0/2493152757 learned_addr learned my addr 172.21.15.152:0/2493152757 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:35:52.371 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.370+0000 7f2efe800640 1 --2- 172.21.15.152:0/2493152757 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2f000cd8c0 0x7f2f000b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:52.371 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.370+0000 7f2efd400640 1 -- 172.21.15.152:0/2493152757 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1527135757 0 0) 0x7f2f001463e0 con 0x7f2f00146000 2024-04-20T11:35:52.371 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.370+0000 7f2efd400640 1 -- 172.21.15.152:0/2493152757 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2ee4003520 con 0x7f2f00146000 2024-04-20T11:35:52.371 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.370+0000 7f2efd400640 1 -- 172.21.15.152:0/2493152757 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 382069518 0 0) 0x7f2ee4003520 con 0x7f2f00146000 2024-04-20T11:35:52.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.370+0000 7f2efd400640 1 -- 172.21.15.152:0/2493152757 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2f000cd8c0 msgr2=0x7f2f000b4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:52.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.370+0000 7f2efd400640 1 --2- 172.21.15.152:0/2493152757 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2f000cd8c0 0x7f2f000b4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:52.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.370+0000 7f2efd400640 1 -- 172.21.15.152:0/2493152757 >> v1:172.21.15.175:6789/0 conn(0x7f2f000b2880 legacy=0x7f2f000cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:52.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.370+0000 7f2efd400640 1 -- 172.21.15.152:0/2493152757 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f000b4100 con 0x7f2f00146000 2024-04-20T11:35:52.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.370+0000 7f2efd400640 1 -- 172.21.15.152:0/2493152757 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f2ef0001690 con 0x7f2f00146000 2024-04-20T11:35:52.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.371+0000 7f2efd400640 1 -- 172.21.15.152:0/2493152757 >> v1:172.21.15.152:6789/0 conn(0x7f2f00146000 legacy=0x7f2f000b2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:52.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.371+0000 7f2efd400640 1 --2- 172.21.15.152:0/2493152757 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2f000cd8c0 0x7f2ee4003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:52.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.371+0000 7f2efd400640 1 --2- 172.21.15.152:0/2493152757 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ee4003f80 0x7f2ee4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:52.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.371+0000 7f2efd400640 1 --2- 172.21.15.152:0/2493152757 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ee4004d40 0x7f2ee40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:52.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.371+0000 7f2efd400640 1 -- 172.21.15.152:0/2493152757 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f2f000b4750 con 0x7f2ee4004d40 2024-04-20T11:35:52.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.371+0000 7f2efd400640 1 -- 172.21.15.152:0/2493152757 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee400bb80 con 0x7f2ee4003f80 2024-04-20T11:35:52.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.371+0000 7f2efd400640 1 -- 172.21.15.152:0/2493152757 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2ee400bd80 con 0x7f2f000cd8c0 2024-04-20T11:35:52.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.371+0000 7f2efe800640 1 --2- 172.21.15.152:0/2493152757 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ee4004d40 0x7f2ee40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:52.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.371+0000 7f2efde00640 1 --2- 172.21.15.152:0/2493152757 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2f000cd8c0 0x7f2ee4003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:52.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.371+0000 7f2efe800640 1 -- 172.21.15.152:0/2493152757 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2f000cd8c0 msgr2=0x7f2ee4003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:52.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.371+0000 7f2efe800640 1 --2- 172.21.15.152:0/2493152757 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2f000cd8c0 0x7f2ee4003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:52.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.371+0000 7f2efe800640 1 -- 172.21.15.152:0/2493152757 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ee4003f80 msgr2=0x7f2ee4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:52.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.371+0000 7f2ef5400640 1 --2- 172.21.15.152:0/2493152757 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ee4003f80 0x7f2ee4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:52.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.372+0000 7f2efe800640 1 --2- 172.21.15.152:0/2493152757 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ee4003f80 0x7f2ee4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:52.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.372+0000 7f2efe800640 1 -- 172.21.15.152:0/2493152757 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ee4003520 con 0x7f2ee4004d40 2024-04-20T11:35:52.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.372+0000 7f2ef5400640 1 --2- 172.21.15.152:0/2493152757 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ee4003f80 0x7f2ee4004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:35:52.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.373+0000 7f2efe800640 1 --2- 172.21.15.152:0/2493152757 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ee4004d40 0x7f2ee40051c0 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7f2ef8009fc0 tx=0x7f2ee4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=334f13454bb8da85 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:52.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.373+0000 7f2efd400640 1 -- 172.21.15.152:0/2493152757 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef8039070 con 0x7f2ee4004d40 2024-04-20T11:35:52.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.373+0000 7f2efd400640 1 -- 172.21.15.152:0/2493152757 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ef802cc20 con 0x7f2ee4004d40 2024-04-20T11:35:52.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.373+0000 7f2efd400640 1 -- 172.21.15.152:0/2493152757 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef8009c10 con 0x7f2ee4004d40 2024-04-20T11:35:52.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.374+0000 7f2f04a00640 1 -- 172.21.15.152:0/2493152757 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ee4004d40 msgr2=0x7f2ee40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:52.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.374+0000 7f2f04a00640 1 --2- 172.21.15.152:0/2493152757 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ee4004d40 0x7f2ee40051c0 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7f2ef8009fc0 tx=0x7f2ee4005830 comp rx=0 tx=0).stop 2024-04-20T11:35:52.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.374+0000 7f2efe800640 1 -- 172.21.15.152:0/2493152757 reap_dead start 2024-04-20T11:35:52.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.374+0000 7f2f04a00640 1 -- 172.21.15.152:0/2493152757 shutdown_connections 2024-04-20T11:35:52.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.374+0000 7f2f04a00640 1 -- 172.21.15.152:0/2493152757 >> 172.21.15.152:0/2493152757 conn(0x7f2f0006b980 msgr2=0x7f2f0006bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:52.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.374+0000 7f2f04a00640 1 -- 172.21.15.152:0/2493152757 shutdown_connections 2024-04-20T11:35:52.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.374+0000 7f2f04a00640 1 -- 172.21.15.152:0/2493152757 wait complete. 2024-04-20T11:35:52.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.376+0000 7f2f04a00640 1 Processor -- start 2024-04-20T11:35:52.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.376+0000 7f2f04a00640 1 -- start start 2024-04-20T11:35:52.378 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.377+0000 7f2f04a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2ee4004d40 0x7f2f0013e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:52.378 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.377+0000 7f2f04a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2f001f2d20 0x7f2f0013d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:52.378 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.377+0000 7f2f04a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2f0013f210 0x7f2f0013f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:52.378 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.377+0000 7f2f04a00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f2f000b4d60 con 0x7f2f0013f210 2024-04-20T11:35:52.378 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.377+0000 7f2f04a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2f000b4be0 con 0x7f2f001f2d20 2024-04-20T11:35:52.378 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.377+0000 7f2f04a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2f000b4ee0 con 0x7f2ee4004d40 2024-04-20T11:35:52.378 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.377+0000 7f2efe800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2f0013f210 0x7f2f0013f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:52.378 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.377+0000 7f2efe800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2f0013f210 0x7f2f0013f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:60652/0 (socket says 172.21.15.152:60652) 2024-04-20T11:35:52.378 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.377+0000 7f2efe800640 1 -- 172.21.15.152:0/1408633034 learned_addr learned my addr 172.21.15.152:0/1408633034 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:52.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.377+0000 7f2efde00640 1 --2- 172.21.15.152:0/1408633034 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2ee4004d40 0x7f2f0013e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:52.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.377+0000 7f2ef5400640 1 --2- 172.21.15.152:0/1408633034 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2f001f2d20 0x7f2f0013d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:52.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.377+0000 7f2efe800640 1 -- 172.21.15.152:0/1408633034 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2ee4004d40 msgr2=0x7f2f0013e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:52.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.377+0000 7f2efe800640 1 --2- 172.21.15.152:0/1408633034 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2ee4004d40 0x7f2f0013e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:52.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.377+0000 7f2efe800640 1 -- 172.21.15.152:0/1408633034 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2f001f2d20 msgr2=0x7f2f0013d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:52.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.378+0000 7f2efe800640 1 --2- 172.21.15.152:0/1408633034 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2f001f2d20 0x7f2f0013d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:52.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.378+0000 7f2efe800640 1 -- 172.21.15.152:0/1408633034 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f0013e980 con 0x7f2f0013f210 2024-04-20T11:35:52.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.378+0000 7f2ef5400640 1 --2- 172.21.15.152:0/1408633034 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2f001f2d20 0x7f2f0013d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:35:52.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.378+0000 7f2efe800640 1 --2- 172.21.15.152:0/1408633034 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2f0013f210 0x7f2f0013f690 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7f2ef8002230 tx=0x7f2ef8009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:52.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.378+0000 7f2ef6a00640 1 -- 172.21.15.152:0/1408633034 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef8039030 con 0x7f2f0013f210 2024-04-20T11:35:52.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.378+0000 7f2ef6a00640 1 -- 172.21.15.152:0/1408633034 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ef8043070 con 0x7f2f0013f210 2024-04-20T11:35:52.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.378+0000 7f2ef6a00640 1 -- 172.21.15.152:0/1408633034 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef8009c10 con 0x7f2f0013f210 2024-04-20T11:35:52.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.378+0000 7f2f04a00640 1 -- 172.21.15.152:0/1408633034 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f00140bb0 con 0x7f2f0013f210 2024-04-20T11:35:52.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.378+0000 7f2f04a00640 1 -- 172.21.15.152:0/1408633034 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f00141170 con 0x7f2f0013f210 2024-04-20T11:35:52.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.380+0000 7f2f04a00640 1 -- 172.21.15.152:0/1408633034 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f000689f0 con 0x7f2f0013f210 2024-04-20T11:35:52.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.381+0000 7f2ef6a00640 1 -- 172.21.15.152:0/1408633034 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 22) ==== 91601+0+0 (secure 0 0 0) 0x7f2ef803d7f0 con 0x7f2f0013f210 2024-04-20T11:35:52.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.381+0000 7f2ef6a00640 1 --2- 172.21.15.152:0/1408633034 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f2edc06cf50 0x7f2edc06f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:52.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.382+0000 7f2ef6a00640 1 -- 172.21.15.152:0/1408633034 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5709+0+0 (secure 0 0 0) 0x7f2ef80aeaf0 con 0x7f2f0013f210 2024-04-20T11:35:52.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.382+0000 7f2efde00640 1 --2- 172.21.15.152:0/1408633034 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f2edc06cf50 0x7f2edc06f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:52.383 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.384+0000 7f2efde00640 1 --2- 172.21.15.152:0/1408633034 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f2edc06cf50 0x7f2edc06f410 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f2f0013bc40 tx=0x7f2ef0009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:52.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.386+0000 7f2ef6a00640 1 -- 172.21.15.152:0/1408633034 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f2ef807c7a0 con 0x7f2f0013f210 2024-04-20T11:35:52.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.589+0000 7f2f04a00640 1 -- 172.21.15.152:0/1408633034 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f2f000b0620 con 0x7f2f0013f210 2024-04-20T11:35:52.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.590+0000 7f2ef6a00640 1 -- 172.21.15.152:0/1408633034 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v131) ==== 104+0+11 (secure 0 0 0) 0x7f2ef807c0a0 con 0x7f2f0013f210 2024-04-20T11:35:52.590 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 32 2024-04-20T11:35:52.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.593+0000 7f2f04a00640 1 -- 172.21.15.152:0/1408633034 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f2edc06cf50 msgr2=0x7f2edc06f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:52.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.593+0000 7f2f04a00640 1 --2- 172.21.15.152:0/1408633034 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f2edc06cf50 0x7f2edc06f410 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f2f0013bc40 tx=0x7f2ef0009040 comp rx=0 tx=0).stop 2024-04-20T11:35:52.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.593+0000 7f2f04a00640 1 -- 172.21.15.152:0/1408633034 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2f0013f210 msgr2=0x7f2f0013f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:52.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.593+0000 7f2f04a00640 1 --2- 172.21.15.152:0/1408633034 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2f0013f210 0x7f2f0013f690 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7f2ef8002230 tx=0x7f2ef8009750 comp rx=0 tx=0).stop 2024-04-20T11:35:52.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.593+0000 7f2f04a00640 1 -- 172.21.15.152:0/1408633034 shutdown_connections 2024-04-20T11:35:52.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.593+0000 7f2f04a00640 1 --2- 172.21.15.152:0/1408633034 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f2edc06cf50 0x7f2edc06f410 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:52.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.593+0000 7f2f04a00640 1 --2- 172.21.15.152:0/1408633034 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2f0013f210 0x7f2f0013f690 unknown :-1 s=CLOSED pgs=1891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:52.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.593+0000 7f2f04a00640 1 --2- 172.21.15.152:0/1408633034 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2f001f2d20 0x7f2f0013d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:52.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.593+0000 7f2f04a00640 1 --2- 172.21.15.152:0/1408633034 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2ee4004d40 0x7f2f0013e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:52.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.593+0000 7f2f04a00640 1 -- 172.21.15.152:0/1408633034 >> 172.21.15.152:0/1408633034 conn(0x7f2f0006b980 msgr2=0x7f2f000b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:52.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.593+0000 7f2f04a00640 1 -- 172.21.15.152:0/1408633034 shutdown_connections 2024-04-20T11:35:52.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.593+0000 7f2f04a00640 1 -- 172.21.15.152:0/1408633034 wait complete. 2024-04-20T11:35:52.612 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T11:35:52.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.818+0000 7f82f7e00640 1 Processor -- start 2024-04-20T11:35:52.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.819+0000 7f82f7e00640 1 -- start start 2024-04-20T11:35:52.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.820+0000 7f82f7e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f82f80ceeb0 0x7f82f80d4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:52.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.820+0000 7f82f7e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f82f8160230 0x7f82f81606e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:52.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.820+0000 7f82f7e00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82f80d2b70 con 0x7f82f80cf5b0 2024-04-20T11:35:52.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.820+0000 7f82f7e00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f82f80d5420 con 0x7f82f8160230 2024-04-20T11:35:52.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.820+0000 7f82f7e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f82f80d2e50 con 0x7f82f80ceeb0 2024-04-20T11:35:52.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.821+0000 7f82f7400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f82f8160230 0x7f82f81606e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:52.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.821+0000 7f82f6a00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f82f80cf5b0 0x7f82f80cf990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:60666/0 (socket says 172.21.15.152:60666) 2024-04-20T11:35:52.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.821+0000 7f82f6a00640 1 -- 172.21.15.152:0/1353310764 learned_addr learned my addr 172.21.15.152:0/1353310764 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:35:52.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.821+0000 7f82f7400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f82f8160230 0x7f82f81606e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:60658/0 (socket says 172.21.15.152:60658) 2024-04-20T11:35:52.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.821+0000 7f82f6000640 1 --2- 172.21.15.152:0/1353310764 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f82f80ceeb0 0x7f82f80d4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:52.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.821+0000 7f82f7400640 1 -- 172.21.15.152:0/1353310764 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f82f80ceeb0 msgr2=0x7f82f80d4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:52.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.821+0000 7f82f7400640 1 --2- 172.21.15.152:0/1353310764 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f82f80ceeb0 0x7f82f80d4ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:52.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.821+0000 7f82f7400640 1 -- 172.21.15.152:0/1353310764 >> v1:172.21.15.152:6789/0 conn(0x7f82f80cf5b0 legacy=0x7f82f80cf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:52.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.821+0000 7f82f7400640 1 -- 172.21.15.152:0/1353310764 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82f80d3130 con 0x7f82f8160230 2024-04-20T11:35:52.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.821+0000 7f82f5600640 1 -- 172.21.15.152:0/1353310764 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2728608422 0 0) 0x7f82f80d2b70 con 0x7f82f80cf5b0 2024-04-20T11:35:52.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.823+0000 7f82f7400640 1 --2- 172.21.15.152:0/1353310764 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f82f8160230 0x7f82f81606e0 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7f82f00097a0 tx=0x7f82f002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48866d19c0b0e6dc server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:52.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.823+0000 7f82f5600640 1 -- 172.21.15.152:0/1353310764 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82f0034070 con 0x7f82f8160230 2024-04-20T11:35:52.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.823+0000 7f82f5600640 1 -- 172.21.15.152:0/1353310764 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82f0030610 con 0x7f82f8160230 2024-04-20T11:35:52.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.823+0000 7f82f5600640 1 -- 172.21.15.152:0/1353310764 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82f00309e0 con 0x7f82f8160230 2024-04-20T11:35:52.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.824+0000 7f82f7e00640 1 -- 172.21.15.152:0/1353310764 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f82f8160230 msgr2=0x7f82f81606e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:52.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.824+0000 7f82f7e00640 1 --2- 172.21.15.152:0/1353310764 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f82f8160230 0x7f82f81606e0 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7f82f00097a0 tx=0x7f82f002b960 comp rx=0 tx=0).stop 2024-04-20T11:35:52.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.824+0000 7f82f7e00640 1 -- 172.21.15.152:0/1353310764 shutdown_connections 2024-04-20T11:35:52.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.824+0000 7f82f7e00640 1 --2- 172.21.15.152:0/1353310764 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f82f80ceeb0 0x7f82f80d4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:52.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.824+0000 7f82f7e00640 1 --2- 172.21.15.152:0/1353310764 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f82f8160230 0x7f82f81606e0 unknown :-1 s=CLOSED pgs=1893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:52.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.824+0000 7f82f7e00640 1 -- 172.21.15.152:0/1353310764 >> 172.21.15.152:0/1353310764 conn(0x7f82f808bfd0 msgr2=0x7f82f80d4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:52.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.824+0000 7f82f7e00640 1 -- 172.21.15.152:0/1353310764 shutdown_connections 2024-04-20T11:35:52.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.824+0000 7f82f7e00640 1 -- 172.21.15.152:0/1353310764 wait complete. 2024-04-20T11:35:52.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.826+0000 7f82f7e00640 1 Processor -- start 2024-04-20T11:35:52.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.826+0000 7f82f7e00640 1 -- start start 2024-04-20T11:35:52.826 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.827+0000 7f82f7e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f82f80ceeb0 0x7f82f81f4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:52.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.827+0000 7f82f7e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f82f80cf5b0 0x7f82f81593c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:52.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.827+0000 7f82f7e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f82f8160230 0x7f82f80cd680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:52.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.827+0000 7f82f7e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f82f80d3c10 con 0x7f82f8160230 2024-04-20T11:35:52.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.827+0000 7f82f7e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f82f80d36f0 con 0x7f82f80cf5b0 2024-04-20T11:35:52.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.827+0000 7f82f7e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f82f80d2790 con 0x7f82f80ceeb0 2024-04-20T11:35:52.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.827+0000 7f82f7400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f82f8160230 0x7f82f80cd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:52.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.827+0000 7f82f7400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f82f8160230 0x7f82f80cd680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:60668/0 (socket says 172.21.15.152:60668) 2024-04-20T11:35:52.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.827+0000 7f82f7400640 1 -- 172.21.15.152:0/180818057 learned_addr learned my addr 172.21.15.152:0/180818057 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:52.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.827+0000 7f82f6a00640 1 --2- 172.21.15.152:0/180818057 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f82f80ceeb0 0x7f82f81f4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:52.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.827+0000 7f82f6000640 1 --2- 172.21.15.152:0/180818057 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f82f80cf5b0 0x7f82f81593c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:52.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.827+0000 7f82f7400640 1 -- 172.21.15.152:0/180818057 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f82f80ceeb0 msgr2=0x7f82f81f4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:52.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.827+0000 7f82f7400640 1 --2- 172.21.15.152:0/180818057 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f82f80ceeb0 0x7f82f81f4ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:52.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.827+0000 7f82f7400640 1 -- 172.21.15.152:0/180818057 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f82f80cf5b0 msgr2=0x7f82f81593c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:52.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.827+0000 7f82f7400640 1 --2- 172.21.15.152:0/180818057 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f82f80cf5b0 0x7f82f81593c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:52.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.827+0000 7f82f7400640 1 -- 172.21.15.152:0/180818057 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82f80cdf60 con 0x7f82f8160230 2024-04-20T11:35:52.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.828+0000 7f82f7400640 1 --2- 172.21.15.152:0/180818057 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f82f8160230 0x7f82f80cd680 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7f82f002b9a0 tx=0x7f82f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:52.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.828+0000 7f82ef400640 1 -- 172.21.15.152:0/180818057 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82f0034040 con 0x7f82f8160230 2024-04-20T11:35:52.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.828+0000 7f82ef400640 1 -- 172.21.15.152:0/180818057 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82f0039910 con 0x7f82f8160230 2024-04-20T11:35:52.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.828+0000 7f82ef400640 1 -- 172.21.15.152:0/180818057 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82f0039d20 con 0x7f82f8160230 2024-04-20T11:35:52.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.828+0000 7f82f7e00640 1 -- 172.21.15.152:0/180818057 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82f80ce240 con 0x7f82f8160230 2024-04-20T11:35:52.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.828+0000 7f82f7e00640 1 -- 172.21.15.152:0/180818057 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82f80ce800 con 0x7f82f8160230 2024-04-20T11:35:52.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.830+0000 7f82f7e00640 1 -- 172.21.15.152:0/180818057 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82f8075e60 con 0x7f82f8160230 2024-04-20T11:35:52.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.830+0000 7f82ef400640 1 -- 172.21.15.152:0/180818057 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 22) ==== 91601+0+0 (secure 0 0 0) 0x7f82f0037070 con 0x7f82f8160230 2024-04-20T11:35:52.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.831+0000 7f82ef400640 1 --2- 172.21.15.152:0/180818057 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f82d406cf20 0x7f82d406f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:52.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.831+0000 7f82ef400640 1 -- 172.21.15.152:0/180818057 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5709+0+0 (secure 0 0 0) 0x7f82f00ae810 con 0x7f82f8160230 2024-04-20T11:35:52.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.832+0000 7f82f6a00640 1 --2- 172.21.15.152:0/180818057 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f82d406cf20 0x7f82d406f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:52.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.833+0000 7f82f6a00640 1 --2- 172.21.15.152:0/180818057 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f82d406cf20 0x7f82d406f3e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f82e8000f60 tx=0x7f82e8007b70 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:52.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:52.836+0000 7f82ef400640 1 -- 172.21.15.152:0/180818057 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f82f007c240 con 0x7f82f8160230 2024-04-20T11:35:53.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.038+0000 7f82f7e00640 1 -- 172.21.15.152:0/180818057 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f82f80cf390 con 0x7f82f8160230 2024-04-20T11:35:53.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.039+0000 7f82ef400640 1 -- 172.21.15.152:0/180818057 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v131) ==== 100+0+11 (secure 0 0 0) 0x7f82f007bb40 con 0x7f82f8160230 2024-04-20T11:35:53.039 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 64 2024-04-20T11:35:53.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.042+0000 7f82f7e00640 1 -- 172.21.15.152:0/180818057 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f82d406cf20 msgr2=0x7f82d406f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:53.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.042+0000 7f82f7e00640 1 --2- 172.21.15.152:0/180818057 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f82d406cf20 0x7f82d406f3e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f82e8000f60 tx=0x7f82e8007b70 comp rx=0 tx=0).stop 2024-04-20T11:35:53.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.042+0000 7f82f7e00640 1 -- 172.21.15.152:0/180818057 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f82f8160230 msgr2=0x7f82f80cd680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:53.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.042+0000 7f82f7e00640 1 --2- 172.21.15.152:0/180818057 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f82f8160230 0x7f82f80cd680 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7f82f002b9a0 tx=0x7f82f00046c0 comp rx=0 tx=0).stop 2024-04-20T11:35:53.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.042+0000 7f82f7e00640 1 -- 172.21.15.152:0/180818057 shutdown_connections 2024-04-20T11:35:53.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.042+0000 7f82f7e00640 1 --2- 172.21.15.152:0/180818057 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f82d406cf20 0x7f82d406f3e0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:53.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.042+0000 7f82f7e00640 1 --2- 172.21.15.152:0/180818057 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f82f8160230 0x7f82f80cd680 unknown :-1 s=CLOSED pgs=1894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:53.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.042+0000 7f82f7e00640 1 --2- 172.21.15.152:0/180818057 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f82f80cf5b0 0x7f82f81593c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:53.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.042+0000 7f82f7e00640 1 --2- 172.21.15.152:0/180818057 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f82f80ceeb0 0x7f82f81f4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:53.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.042+0000 7f82f7e00640 1 -- 172.21.15.152:0/180818057 >> 172.21.15.152:0/180818057 conn(0x7f82f808bfd0 msgr2=0x7f82f80d4530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:53.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.042+0000 7f82f7e00640 1 -- 172.21.15.152:0/180818057 shutdown_connections 2024-04-20T11:35:53.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.042+0000 7f82f7e00640 1 -- 172.21.15.152:0/180818057 wait complete. 2024-04-20T11:35:53.061 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:35:53.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.268+0000 7f84c7e00640 1 Processor -- start 2024-04-20T11:35:53.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.268+0000 7f84c7e00640 1 -- start start 2024-04-20T11:35:53.270 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.270+0000 7f84c7e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f84c806ea20 0x7f84c80b4a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:53.270 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.270+0000 7f84c7e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f84c80b4f90 0x7f84c8162e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:53.270 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.270+0000 7f84c7e00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84c80b5510 con 0x7f84c8071cd0 2024-04-20T11:35:53.270 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.270+0000 7f84c7e00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f84c80ceec0 con 0x7f84c80b4f90 2024-04-20T11:35:53.270 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.270+0000 7f84c7e00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f84c806dcf0 con 0x7f84c806ea20 2024-04-20T11:35:53.270 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.270+0000 7f84c7400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f84c80b4f90 0x7f84c8162e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:53.270 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.271+0000 7f84c7400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f84c80b4f90 0x7f84c8162e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:60670/0 (socket says 172.21.15.152:60670) 2024-04-20T11:35:53.270 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.271+0000 7f84c7400640 1 -- 172.21.15.152:0/651087964 learned_addr learned my addr 172.21.15.152:0/651087964 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:53.271 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.271+0000 7f84c7400640 1 -- 172.21.15.152:0/651087964 >> v2:172.21.15.175:3300/0 conn(0x7f84c806ea20 msgr2=0x7f84c80b4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:53.271 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.271+0000 7f84c7400640 1 --2- 172.21.15.152:0/651087964 >> v2:172.21.15.175:3300/0 conn(0x7f84c806ea20 0x7f84c80b4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:53.271 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.271+0000 7f84c7400640 1 -- 172.21.15.152:0/651087964 >> v1:172.21.15.175:6789/0 conn(0x7f84c8071cd0 legacy=0x7f84c80720b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:53.271 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.271+0000 7f84c7400640 1 -- 172.21.15.152:0/651087964 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84c806df40 con 0x7f84c80b4f90 2024-04-20T11:35:53.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.272+0000 7f84c7400640 1 --2- 172.21.15.152:0/651087964 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f84c80b4f90 0x7f84c8162e50 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7f84c8071a90 tx=0x7f84c002be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5d9c14fbee13cf9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:53.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.272+0000 7f84c6000640 1 -- 172.21.15.152:0/651087964 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84c002c9f0 con 0x7f84c80b4f90 2024-04-20T11:35:53.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.272+0000 7f84c6000640 1 -- 172.21.15.152:0/651087964 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84c002cb90 con 0x7f84c80b4f90 2024-04-20T11:35:53.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.272+0000 7f84c6000640 1 -- 172.21.15.152:0/651087964 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84c00343f0 con 0x7f84c80b4f90 2024-04-20T11:35:53.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.273+0000 7f84c7e00640 1 -- 172.21.15.152:0/651087964 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f84c80b4f90 msgr2=0x7f84c8162e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:53.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.273+0000 7f84c7e00640 1 --2- 172.21.15.152:0/651087964 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f84c80b4f90 0x7f84c8162e50 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7f84c8071a90 tx=0x7f84c002be50 comp rx=0 tx=0).stop 2024-04-20T11:35:53.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.273+0000 7f84c7e00640 1 -- 172.21.15.152:0/651087964 shutdown_connections 2024-04-20T11:35:53.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.273+0000 7f84c7e00640 1 --2- 172.21.15.152:0/651087964 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f84c80b4f90 0x7f84c8162e50 unknown :-1 s=CLOSED pgs=1895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:53.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.273+0000 7f84c7e00640 1 --2- 172.21.15.152:0/651087964 >> v2:172.21.15.175:3300/0 conn(0x7f84c806ea20 0x7f84c80b4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:53.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.273+0000 7f84c7e00640 1 -- 172.21.15.152:0/651087964 >> 172.21.15.152:0/651087964 conn(0x7f84c8067ae0 msgr2=0x7f84c806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:53.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.273+0000 7f84c7e00640 1 -- 172.21.15.152:0/651087964 shutdown_connections 2024-04-20T11:35:53.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.273+0000 7f84c7e00640 1 -- 172.21.15.152:0/651087964 wait complete. 2024-04-20T11:35:53.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.275+0000 7f84c7e00640 1 Processor -- start 2024-04-20T11:35:53.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.275+0000 7f84c7e00640 1 -- start start 2024-04-20T11:35:53.276 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.276+0000 7f84c7e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f84c806ea20 0x7f84c81ebf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:53.276 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.276+0000 7f84c7e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f84c8071cd0 0x7f84c814d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:53.276 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.276+0000 7f84c7e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f84c80b4f90 0x7f84c814d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:53.277 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.276+0000 7f84c7e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f84c80b4180 con 0x7f84c80b4f90 2024-04-20T11:35:53.277 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.276+0000 7f84c7e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f84c806e500 con 0x7f84c806ea20 2024-04-20T11:35:53.277 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.276+0000 7f84c7e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f84c8163390 con 0x7f84c8071cd0 2024-04-20T11:35:53.277 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.276+0000 7f84c7400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f84c80b4f90 0x7f84c814d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:53.277 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.276+0000 7f84c7400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f84c80b4f90 0x7f84c814d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:60684/0 (socket says 172.21.15.152:60684) 2024-04-20T11:35:53.277 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.276+0000 7f84c7400640 1 -- 172.21.15.152:0/605367618 learned_addr learned my addr 172.21.15.152:0/605367618 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:53.277 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.277+0000 7f84c6a00640 1 --2- 172.21.15.152:0/605367618 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f84c806ea20 0x7f84c81ebf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:53.277 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.277+0000 7f84be000640 1 --2- 172.21.15.152:0/605367618 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f84c8071cd0 0x7f84c814d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:53.277 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.277+0000 7f84c7400640 1 -- 172.21.15.152:0/605367618 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f84c8071cd0 msgr2=0x7f84c814d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:53.277 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.277+0000 7f84c7400640 1 --2- 172.21.15.152:0/605367618 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f84c8071cd0 0x7f84c814d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:53.277 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.277+0000 7f84c7400640 1 -- 172.21.15.152:0/605367618 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f84c806ea20 msgr2=0x7f84c81ebf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:53.277 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.277+0000 7f84c7400640 1 --2- 172.21.15.152:0/605367618 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f84c806ea20 0x7f84c81ebf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:53.277 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.277+0000 7f84c7400640 1 -- 172.21.15.152:0/605367618 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84c815ed90 con 0x7f84c80b4f90 2024-04-20T11:35:53.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.277+0000 7f84be000640 1 --2- 172.21.15.152:0/605367618 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f84c8071cd0 0x7f84c814d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:35:53.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.277+0000 7f84c6a00640 1 --2- 172.21.15.152:0/605367618 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f84c806ea20 0x7f84c81ebf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:35:53.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.277+0000 7f84c7400640 1 --2- 172.21.15.152:0/605367618 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f84c80b4f90 0x7f84c814d660 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7f84c0036040 tx=0x7f84c0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:53.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.277+0000 7f84bf400640 1 -- 172.21.15.152:0/605367618 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84c0043020 con 0x7f84c80b4f90 2024-04-20T11:35:53.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.277+0000 7f84bf400640 1 -- 172.21.15.152:0/605367618 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84c003e450 con 0x7f84c80b4f90 2024-04-20T11:35:53.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.277+0000 7f84bf400640 1 -- 172.21.15.152:0/605367618 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84c003e820 con 0x7f84c80b4f90 2024-04-20T11:35:53.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.277+0000 7f84c7e00640 1 -- 172.21.15.152:0/605367618 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84c815f070 con 0x7f84c80b4f90 2024-04-20T11:35:53.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.278+0000 7f84c7e00640 1 -- 172.21.15.152:0/605367618 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84c815f630 con 0x7f84c80b4f90 2024-04-20T11:35:53.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.279+0000 7f84c7e00640 1 -- 172.21.15.152:0/605367618 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84c8075e60 con 0x7f84c80b4f90 2024-04-20T11:35:53.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.280+0000 7f84bf400640 1 -- 172.21.15.152:0/605367618 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 22) ==== 91601+0+0 (secure 0 0 0) 0x7f84c0046020 con 0x7f84c80b4f90 2024-04-20T11:35:53.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.281+0000 7f84bf400640 1 --2- 172.21.15.152:0/605367618 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f84a406cf20 0x7f84a406f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:53.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.281+0000 7f84bf400640 1 -- 172.21.15.152:0/605367618 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5709+0+0 (secure 0 0 0) 0x7f84c00af070 con 0x7f84c80b4f90 2024-04-20T11:35:53.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.281+0000 7f84c6a00640 1 --2- 172.21.15.152:0/605367618 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f84a406cf20 0x7f84a406f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:53.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.283+0000 7f84c6a00640 1 --2- 172.21.15.152:0/605367618 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f84a406cf20 0x7f84a406f3e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f84b8007fd0 tx=0x7f84b8009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:53.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.285+0000 7f84bf400640 1 -- 172.21.15.152:0/605367618 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f84c004d030 con 0x7f84c80b4f90 2024-04-20T11:35:53.526 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.527+0000 7f84c7e00640 1 -- 172.21.15.152:0/605367618 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f84c80ccbf0 con 0x7f84c80b4f90 2024-04-20T11:35:53.527 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.528+0000 7f84bf400640 1 -- 172.21.15.152:0/605367618 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) ==== 94+0+5212 (secure 0 0 0) 0x7f84c007cbb0 con 0x7f84c80b4f90 2024-04-20T11:35:53.528 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:35:53.528 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":87,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6062,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1303172876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6833","nonce":1303172876},{"type":"v1","addr":"172.21.15.175:6835","nonce":1303172876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6667,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6834/1975208598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1975208598},{"type":"v1","addr":"172.21.15.175:6834","nonce":1975208598}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6670,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6834/287411071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":287411071},{"type":"v1","addr":"172.21.15.152:6834","nonce":287411071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T11:35:19.231687+0000","modified":"2024-04-20T11:35:22.597626+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8919},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8919":{"gid":8919,"name":"a","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6835/1884827433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6833","nonce":1884827433},{"type":"v1","addr":"172.21.15.152:6835","nonce":1884827433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8919,"qdb_cluster":[8919]},"id":10}]} 2024-04-20T11:35:53.528 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 87 2024-04-20T11:35:53.530 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.530+0000 7f84c7e00640 1 -- 172.21.15.152:0/605367618 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f84a406cf20 msgr2=0x7f84a406f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:53.531 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.530+0000 7f84c7e00640 1 --2- 172.21.15.152:0/605367618 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f84a406cf20 0x7f84a406f3e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f84b8007fd0 tx=0x7f84b8009040 comp rx=0 tx=0).stop 2024-04-20T11:35:53.531 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.530+0000 7f84c7e00640 1 -- 172.21.15.152:0/605367618 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f84c80b4f90 msgr2=0x7f84c814d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:53.531 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.530+0000 7f84c7e00640 1 --2- 172.21.15.152:0/605367618 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f84c80b4f90 0x7f84c814d660 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7f84c0036040 tx=0x7f84c0004a40 comp rx=0 tx=0).stop 2024-04-20T11:35:53.531 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.531+0000 7f84c7e00640 1 -- 172.21.15.152:0/605367618 shutdown_connections 2024-04-20T11:35:53.531 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.531+0000 7f84c7e00640 1 --2- 172.21.15.152:0/605367618 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f84a406cf20 0x7f84a406f3e0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:53.531 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.531+0000 7f84c7e00640 1 --2- 172.21.15.152:0/605367618 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f84c80b4f90 0x7f84c814d660 unknown :-1 s=CLOSED pgs=1896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:53.531 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.531+0000 7f84c7e00640 1 --2- 172.21.15.152:0/605367618 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f84c8071cd0 0x7f84c814d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:53.531 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.531+0000 7f84c7e00640 1 --2- 172.21.15.152:0/605367618 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f84c806ea20 0x7f84c81ebf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:53.531 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.531+0000 7f84c7e00640 1 -- 172.21.15.152:0/605367618 >> 172.21.15.152:0/605367618 conn(0x7f84c8067ae0 msgr2=0x7f84c8163770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:53.531 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.531+0000 7f84c7e00640 1 -- 172.21.15.152:0/605367618 shutdown_connections 2024-04-20T11:35:53.531 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.531+0000 7f84c7e00640 1 -- 172.21.15.152:0/605367618 wait complete. 2024-04-20T11:35:53.549 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:35:53.654 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:35:53.662 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:35:53.747 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.747+0000 7f0410c00640 1 Processor -- start 2024-04-20T11:35:53.747 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.747+0000 7f0410c00640 1 -- start start 2024-04-20T11:35:53.748 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.749+0000 7f0410c00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f040c145f60 0x7f040c06ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:53.749 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.749+0000 7f0410c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f040c0726f0 0x7f040c0cd580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:53.749 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.749+0000 7f0410c00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f040c146340 con 0x7f040c0cd0d0 2024-04-20T11:35:53.749 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.749+0000 7f0410c00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f040c06e9f0 con 0x7f040c145f60 2024-04-20T11:35:53.749 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.749+0000 7f0410c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f040c06f3c0 con 0x7f040c0726f0 2024-04-20T11:35:53.750 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.749+0000 7f040a800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f040c0726f0 0x7f040c0cd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:53.750 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.749+0000 7f0409e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f040c145f60 0x7f040c06ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:53.750 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.749+0000 7f0403e00640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7f040c0cd0d0 0x7f040c071fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.152:60066/0 (socket says 172.21.15.152:60066) 2024-04-20T11:35:53.750 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.749+0000 7f040a800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f040c0726f0 0x7f040c0cd580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.152:33072/0 (socket says 172.21.15.152:33072) 2024-04-20T11:35:53.750 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.749+0000 7f0403e00640 1 -- 172.21.15.152:0/3380930233 learned_addr learned my addr 172.21.15.152:0/3380930233 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:35:53.750 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.750+0000 7f040a800640 1 -- 172.21.15.152:0/3380930233 >> v2:172.21.15.175:3300/0 conn(0x7f040c145f60 msgr2=0x7f040c06ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:53.750 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.750+0000 7f040a800640 1 --2- 172.21.15.152:0/3380930233 >> v2:172.21.15.175:3300/0 conn(0x7f040c145f60 0x7f040c06ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:53.750 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.750+0000 7f040a800640 1 -- 172.21.15.152:0/3380930233 >> v1:172.21.15.175:6789/0 conn(0x7f040c0cd0d0 legacy=0x7f040c071fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:53.750 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.750+0000 7f040a800640 1 -- 172.21.15.152:0/3380930233 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f040c072d50 con 0x7f040c0726f0 2024-04-20T11:35:53.750 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.750+0000 7f0409e00640 1 --2- 172.21.15.152:0/3380930233 >> v2:172.21.15.175:3300/0 conn(0x7f040c145f60 0x7f040c06ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:35:53.752 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.751+0000 7f040a800640 1 --2- 172.21.15.152:0/3380930233 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f040c0726f0 0x7f040c0cd580 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0404009d90 tx=0x7f040402bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf48f8bda263f253 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:53.752 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.752+0000 7f0409400640 1 -- 172.21.15.152:0/3380930233 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0404039070 con 0x7f040c0726f0 2024-04-20T11:35:53.752 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.752+0000 7f0409400640 1 -- 172.21.15.152:0/3380930233 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f040402cc50 con 0x7f040c0726f0 2024-04-20T11:35:53.752 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.752+0000 7f0409400640 1 -- 172.21.15.152:0/3380930233 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0404034440 con 0x7f040c0726f0 2024-04-20T11:35:53.752 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.752+0000 7f0410c00640 1 -- 172.21.15.152:0/3380930233 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f040c0726f0 msgr2=0x7f040c0cd580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:53.753 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.752+0000 7f0410c00640 1 --2- 172.21.15.152:0/3380930233 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f040c0726f0 0x7f040c0cd580 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0404009d90 tx=0x7f040402bea0 comp rx=0 tx=0).stop 2024-04-20T11:35:53.753 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.752+0000 7f0410c00640 1 -- 172.21.15.152:0/3380930233 shutdown_connections 2024-04-20T11:35:53.753 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.752+0000 7f0410c00640 1 --2- 172.21.15.152:0/3380930233 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f040c0726f0 0x7f040c0cd580 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:53.753 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.752+0000 7f0410c00640 1 --2- 172.21.15.152:0/3380930233 >> v2:172.21.15.175:3300/0 conn(0x7f040c145f60 0x7f040c06ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:53.753 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.752+0000 7f0410c00640 1 -- 172.21.15.152:0/3380930233 >> 172.21.15.152:0/3380930233 conn(0x7f040c067cf0 msgr2=0x7f040c06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:53.753 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.752+0000 7f0410c00640 1 -- 172.21.15.152:0/3380930233 shutdown_connections 2024-04-20T11:35:53.753 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.752+0000 7f0410c00640 1 -- 172.21.15.152:0/3380930233 wait complete. 2024-04-20T11:35:53.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.754+0000 7f0410c00640 1 Processor -- start 2024-04-20T11:35:53.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.754+0000 7f0410c00640 1 -- start start 2024-04-20T11:35:53.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.755+0000 7f0410c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f040c0726f0 0x7f040c1f5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:53.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.755+0000 7f0410c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f040c0cd0d0 0x7f040c0b4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:53.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.755+0000 7f0410c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f040c145f60 0x7f040c13fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:53.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.755+0000 7f0410c00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f040c06e680 con 0x7f040c0cd0d0 2024-04-20T11:35:53.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.755+0000 7f0410c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f040c06e3a0 con 0x7f040c0726f0 2024-04-20T11:35:53.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.755+0000 7f0410c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f040c0cf290 con 0x7f040c145f60 2024-04-20T11:35:53.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.755+0000 7f0403e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f040c0cd0d0 0x7f040c0b4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:53.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.756+0000 7f0409e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f040c0726f0 0x7f040c1f5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:53.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.756+0000 7f0403e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f040c0cd0d0 0x7f040c0b4600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:60694/0 (socket says 172.21.15.152:60694) 2024-04-20T11:35:53.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.756+0000 7f0403e00640 1 -- 172.21.15.152:0/3775443327 learned_addr learned my addr 172.21.15.152:0/3775443327 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:53.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.756+0000 7f040a800640 1 --2- 172.21.15.152:0/3775443327 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f040c145f60 0x7f040c13fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:53.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.756+0000 7f0403e00640 1 -- 172.21.15.152:0/3775443327 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f040c145f60 msgr2=0x7f040c13fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:53.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.756+0000 7f0403e00640 1 --2- 172.21.15.152:0/3775443327 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f040c145f60 0x7f040c13fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:53.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.756+0000 7f0403e00640 1 -- 172.21.15.152:0/3775443327 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f040c0726f0 msgr2=0x7f040c1f5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:53.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.756+0000 7f0403e00640 1 --2- 172.21.15.152:0/3775443327 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f040c0726f0 0x7f040c1f5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:53.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.756+0000 7f0403e00640 1 -- 172.21.15.152:0/3775443327 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f040c0b4d30 con 0x7f040c0cd0d0 2024-04-20T11:35:53.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.756+0000 7f0403e00640 1 --2- 172.21.15.152:0/3775443327 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f040c0cd0d0 0x7f040c0b4600 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f03f400c480 tx=0x7f03f400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:53.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.757+0000 7f0402000640 1 -- 172.21.15.152:0/3775443327 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f4017070 con 0x7f040c0cd0d0 2024-04-20T11:35:53.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.757+0000 7f0402000640 1 -- 172.21.15.152:0/3775443327 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03f4005ce0 con 0x7f040c0cd0d0 2024-04-20T11:35:53.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.757+0000 7f0402000640 1 -- 172.21.15.152:0/3775443327 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f40072b0 con 0x7f040c0cd0d0 2024-04-20T11:35:53.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.757+0000 7f0410c00640 1 -- 172.21.15.152:0/3775443327 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f040c0b5010 con 0x7f040c0cd0d0 2024-04-20T11:35:53.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.757+0000 7f0410c00640 1 -- 172.21.15.152:0/3775443327 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f040c140390 con 0x7f040c0cd0d0 2024-04-20T11:35:53.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.758+0000 7f0410c00640 1 -- 172.21.15.152:0/3775443327 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f040c0689f0 con 0x7f040c0cd0d0 2024-04-20T11:35:53.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.759+0000 7f0402000640 1 -- 172.21.15.152:0/3775443327 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 22) ==== 91601+0+0 (secure 0 0 0) 0x7f03f401a070 con 0x7f040c0cd0d0 2024-04-20T11:35:53.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.760+0000 7f0402000640 1 --2- 172.21.15.152:0/3775443327 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f03e806cf20 0x7f03e806f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:53.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.760+0000 7f0402000640 1 -- 172.21.15.152:0/3775443327 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5709+0+0 (secure 0 0 0) 0x7f03f408b9a0 con 0x7f040c0cd0d0 2024-04-20T11:35:53.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.760+0000 7f0409e00640 1 --2- 172.21.15.152:0/3775443327 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f03e806cf20 0x7f03e806f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:53.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.761+0000 7f0409e00640 1 --2- 172.21.15.152:0/3775443327 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f03e806cf20 0x7f03e806f3e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f040c0b3d70 tx=0x7f03fc009190 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:53.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.764+0000 7f0402000640 1 -- 172.21.15.152:0/3775443327 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f03f4058e90 con 0x7f040c0cd0d0 2024-04-20T11:35:53.965 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.965+0000 7f0410c00640 1 -- 172.21.15.152:0/3775443327 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f040c0cdfb0 con 0x7f040c0cd0d0 2024-04-20T11:35:53.967 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.967+0000 7f0402000640 1 -- 172.21.15.152:0/3775443327 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v131) ==== 74+0+16739 (secure 0 0 0) 0x7f03f405d190 con 0x7f040c0cd0d0 2024-04-20T11:35:53.967 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:35:53.967 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":131,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:35:20.781260+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:35:16.963520+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"130","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-04-20T11:35:18.215890+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:35:53.969 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.970+0000 7f0410c00640 1 -- 172.21.15.152:0/3775443327 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f03e806cf20 msgr2=0x7f03e806f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:53.970 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.970+0000 7f0410c00640 1 --2- 172.21.15.152:0/3775443327 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f03e806cf20 0x7f03e806f3e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f040c0b3d70 tx=0x7f03fc009190 comp rx=0 tx=0).stop 2024-04-20T11:35:53.970 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.970+0000 7f0410c00640 1 -- 172.21.15.152:0/3775443327 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f040c0cd0d0 msgr2=0x7f040c0b4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:53.970 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.970+0000 7f0410c00640 1 --2- 172.21.15.152:0/3775443327 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f040c0cd0d0 0x7f040c0b4600 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f03f400c480 tx=0x7f03f400c940 comp rx=0 tx=0).stop 2024-04-20T11:35:53.970 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.970+0000 7f0410c00640 1 -- 172.21.15.152:0/3775443327 shutdown_connections 2024-04-20T11:35:53.970 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.970+0000 7f0410c00640 1 --2- 172.21.15.152:0/3775443327 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f03e806cf20 0x7f03e806f3e0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:53.970 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.970+0000 7f0410c00640 1 --2- 172.21.15.152:0/3775443327 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f040c145f60 0x7f040c13fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:53.970 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.970+0000 7f0410c00640 1 --2- 172.21.15.152:0/3775443327 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f040c0cd0d0 0x7f040c0b4600 unknown :-1 s=CLOSED pgs=1897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:53.970 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.970+0000 7f0410c00640 1 --2- 172.21.15.152:0/3775443327 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f040c0726f0 0x7f040c1f5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:53.970 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.970+0000 7f0410c00640 1 -- 172.21.15.152:0/3775443327 >> 172.21.15.152:0/3775443327 conn(0x7f040c067cf0 msgr2=0x7f040c143d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:53.970 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.970+0000 7f0410c00640 1 -- 172.21.15.152:0/3775443327 shutdown_connections 2024-04-20T11:35:53.970 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:53.970+0000 7f0410c00640 1 -- 172.21.15.152:0/3775443327 wait complete. 2024-04-20T11:35:53.989 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-20T11:35:53.990 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:35:54.227 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.227+0000 7fc1bb200640 1 Processor -- start 2024-04-20T11:35:54.227 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.227+0000 7fc1bb200640 1 -- start start 2024-04-20T11:35:54.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.228+0000 7fc1bb200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc1bc0723e0 0x7fc1bc15d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:54.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.228+0000 7fc1bb200640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1bc0ce8c0 con 0x7fc1bc15d1e0 2024-04-20T11:35:54.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.228+0000 7fc1bb200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1bc1600b0 con 0x7fc1bc160b90 2024-04-20T11:35:54.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.228+0000 7fc1bb200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc1bc160390 con 0x7fc1bc0723e0 2024-04-20T11:35:54.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.229+0000 7fc1b9e00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fc1bc15d1e0 0x7fc1bc0cdfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:60670/0 (socket says 172.21.15.152:60670) 2024-04-20T11:35:54.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.229+0000 7fc1b9e00640 1 -- 172.21.15.152:0/4263156468 learned_addr learned my addr 172.21.15.152:0/4263156468 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:35:54.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.229+0000 7fc1b9400640 1 -- 172.21.15.152:0/4263156468 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2514877391 0 0) 0x7fc1bc0ce8c0 con 0x7fc1bc15d1e0 2024-04-20T11:35:54.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.229+0000 7fc1b9400640 1 -- 172.21.15.152:0/4263156468 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc1a0003520 con 0x7fc1bc15d1e0 2024-04-20T11:35:54.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.229+0000 7fc1ba800640 1 --2- 172.21.15.152:0/4263156468 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc1bc0723e0 0x7fc1bc15d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:54.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.230+0000 7fc1b9400640 1 -- 172.21.15.152:0/4263156468 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1823750328 0 0) 0x7fc1a0003520 con 0x7fc1bc15d1e0 2024-04-20T11:35:54.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.230+0000 7fc1b9400640 1 -- 172.21.15.152:0/4263156468 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc1bc0723e0 msgr2=0x7fc1bc15d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:54.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.230+0000 7fc1b9400640 1 --2- 172.21.15.152:0/4263156468 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc1bc0723e0 0x7fc1bc15d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:54.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.230+0000 7fc1b9400640 1 -- 172.21.15.152:0/4263156468 >> v1:172.21.15.175:6789/0 conn(0x7fc1bc160b90 legacy=0x7fc1bc071cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:54.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.230+0000 7fc1b9400640 1 -- 172.21.15.152:0/4263156468 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1bc160670 con 0x7fc1bc15d1e0 2024-04-20T11:35:54.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.230+0000 7fc1b9400640 1 -- 172.21.15.152:0/4263156468 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7fc1ac001670 con 0x7fc1bc15d1e0 2024-04-20T11:35:54.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.230+0000 7fc1b9400640 1 -- 172.21.15.152:0/4263156468 >> v1:172.21.15.152:6789/0 conn(0x7fc1bc15d1e0 legacy=0x7fc1bc0cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:54.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.230+0000 7fc1b9400640 1 --2- 172.21.15.152:0/4263156468 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc1a00039e0 0x7fc1a0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:54.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.230+0000 7fc1b9400640 1 --2- 172.21.15.152:0/4263156468 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc1bc0723e0 0x7fc1a0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:54.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.230+0000 7fc1b9400640 1 --2- 172.21.15.152:0/4263156468 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc1a0004d40 0x7fc1a00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:54.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.230+0000 7fc1b9400640 1 -- 172.21.15.152:0/4263156468 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fc1bc160390 con 0x7fc1a00039e0 2024-04-20T11:35:54.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.230+0000 7fc1b9400640 1 -- 172.21.15.152:0/4263156468 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a000bb50 con 0x7fc1a0004d40 2024-04-20T11:35:54.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.230+0000 7fc1b9400640 1 -- 172.21.15.152:0/4263156468 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc1a000bd20 con 0x7fc1bc0723e0 2024-04-20T11:35:54.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.230+0000 7fc1b9e00640 1 --2- 172.21.15.152:0/4263156468 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc1a00039e0 0x7fc1a0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:54.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.230+0000 7fc1ba800640 1 --2- 172.21.15.152:0/4263156468 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc1a0004d40 0x7fc1a00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:54.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.230+0000 7fc1b9e00640 1 -- 172.21.15.152:0/4263156468 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc1bc0723e0 msgr2=0x7fc1a0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:54.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.230+0000 7fc1b3e00640 1 --2- 172.21.15.152:0/4263156468 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc1bc0723e0 0x7fc1a0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:54.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.230+0000 7fc1b9e00640 1 --2- 172.21.15.152:0/4263156468 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc1bc0723e0 0x7fc1a0004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:54.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.230+0000 7fc1b9e00640 1 -- 172.21.15.152:0/4263156468 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc1a0004d40 msgr2=0x7fc1a00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:54.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.230+0000 7fc1b9e00640 1 --2- 172.21.15.152:0/4263156468 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc1a0004d40 0x7fc1a00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:54.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.230+0000 7fc1b9e00640 1 -- 172.21.15.152:0/4263156468 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1a0003520 con 0x7fc1a00039e0 2024-04-20T11:35:54.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.230+0000 7fc1ba800640 1 --2- 172.21.15.152:0/4263156468 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc1a0004d40 0x7fc1a00051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:35:54.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.232+0000 7fc1b9e00640 1 --2- 172.21.15.152:0/4263156468 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc1a00039e0 0x7fc1a0003e00 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac00c270 tx=0x7fc1a0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b498981bc9f51288 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:54.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.232+0000 7fc1b9400640 1 -- 172.21.15.152:0/4263156468 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1ac007d10 con 0x7fc1a00039e0 2024-04-20T11:35:54.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.232+0000 7fc1b9400640 1 -- 172.21.15.152:0/4263156468 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1ac030070 con 0x7fc1a00039e0 2024-04-20T11:35:54.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.232+0000 7fc1b9400640 1 -- 172.21.15.152:0/4263156468 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1ac02cd40 con 0x7fc1a00039e0 2024-04-20T11:35:54.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.232+0000 7fc1bb200640 1 -- 172.21.15.152:0/4263156468 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc1a00039e0 msgr2=0x7fc1a0003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:54.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.232+0000 7fc1bb200640 1 --2- 172.21.15.152:0/4263156468 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc1a00039e0 0x7fc1a0003e00 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac00c270 tx=0x7fc1a0005830 comp rx=0 tx=0).stop 2024-04-20T11:35:54.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.232+0000 7fc1ba800640 1 -- 172.21.15.152:0/4263156468 reap_dead start 2024-04-20T11:35:54.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.232+0000 7fc1bb200640 1 -- 172.21.15.152:0/4263156468 shutdown_connections 2024-04-20T11:35:54.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.232+0000 7fc1bb200640 1 -- 172.21.15.152:0/4263156468 >> 172.21.15.152:0/4263156468 conn(0x7fc1bc067ae0 msgr2=0x7fc1bc06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:54.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.233+0000 7fc1bb200640 1 -- 172.21.15.152:0/4263156468 shutdown_connections 2024-04-20T11:35:54.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.233+0000 7fc1bb200640 1 -- 172.21.15.152:0/4263156468 wait complete. 2024-04-20T11:35:54.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.234+0000 7fc1bb200640 1 Processor -- start 2024-04-20T11:35:54.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.234+0000 7fc1bb200640 1 -- start start 2024-04-20T11:35:54.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.235+0000 7fc1bb200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc1a00039e0 0x7fc1bc0b40a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:54.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.235+0000 7fc1bb200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc1bc0b2140 0x7fc1bc1f5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:54.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.235+0000 7fc1bb200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc1bc0b1560 0x7fc1bc0b1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:54.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.235+0000 7fc1bb200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fc1bc15d740 con 0x7fc1a00039e0 2024-04-20T11:35:54.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.235+0000 7fc1bb200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc1bc15d5c0 con 0x7fc1bc0b1560 2024-04-20T11:35:54.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.235+0000 7fc1bb200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc1bc15d8c0 con 0x7fc1bc0b2140 2024-04-20T11:35:54.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.236+0000 7fc1b9e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc1a00039e0 0x7fc1bc0b40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:54.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.236+0000 7fc1b9e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc1a00039e0 0x7fc1bc0b40a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:60724/0 (socket says 172.21.15.152:60724) 2024-04-20T11:35:54.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.236+0000 7fc1b9e00640 1 -- 172.21.15.152:0/1033572517 learned_addr learned my addr 172.21.15.152:0/1033572517 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:54.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.236+0000 7fc1b9e00640 1 -- 172.21.15.152:0/1033572517 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc1bc0b2140 msgr2=0x7fc1bc1f5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:54.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.236+0000 7fc1b9e00640 1 --2- 172.21.15.152:0/1033572517 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc1bc0b2140 0x7fc1bc1f5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:54.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.236+0000 7fc1b9e00640 1 -- 172.21.15.152:0/1033572517 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc1bc0b1560 msgr2=0x7fc1bc0b1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:54.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.236+0000 7fc1ba800640 1 --2- 172.21.15.152:0/1033572517 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc1bc0b1560 0x7fc1bc0b1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:54.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.236+0000 7fc1b9e00640 1 --2- 172.21.15.152:0/1033572517 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc1bc0b1560 0x7fc1bc0b1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:54.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.236+0000 7fc1b9e00640 1 -- 172.21.15.152:0/1033572517 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1bc0b4ce0 con 0x7fc1a00039e0 2024-04-20T11:35:54.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.236+0000 7fc1b9e00640 1 --2- 172.21.15.152:0/1033572517 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc1a00039e0 0x7fc1bc0b40a0 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac00c210 tx=0x7fc1bc15ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:54.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.237+0000 7fc1b2000640 1 -- 172.21.15.152:0/1033572517 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1ac007d10 con 0x7fc1a00039e0 2024-04-20T11:35:54.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.237+0000 7fc1b2000640 1 -- 172.21.15.152:0/1033572517 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1ac004bf0 con 0x7fc1a00039e0 2024-04-20T11:35:54.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.237+0000 7fc1b2000640 1 -- 172.21.15.152:0/1033572517 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1ac007070 con 0x7fc1a00039e0 2024-04-20T11:35:54.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.237+0000 7fc1bb200640 1 -- 172.21.15.152:0/1033572517 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1bc0b4f50 con 0x7fc1a00039e0 2024-04-20T11:35:54.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.237+0000 7fc1bb200640 1 -- 172.21.15.152:0/1033572517 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1bc0ada30 con 0x7fc1a00039e0 2024-04-20T11:35:54.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.238+0000 7fc1bb200640 1 -- 172.21.15.152:0/1033572517 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1bc06ef10 con 0x7fc1a00039e0 2024-04-20T11:35:54.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.239+0000 7fc1b2000640 1 -- 172.21.15.152:0/1033572517 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 22) ==== 91601+0+0 (secure 0 0 0) 0x7fc1ac007210 con 0x7fc1a00039e0 2024-04-20T11:35:54.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.240+0000 7fc1b2000640 1 --2- 172.21.15.152:0/1033572517 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fc19806cf50 0x7fc19806f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:54.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.240+0000 7fc1b2000640 1 -- 172.21.15.152:0/1033572517 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5709+0+0 (secure 0 0 0) 0x7fc1ac0add20 con 0x7fc1a00039e0 2024-04-20T11:35:54.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.240+0000 7fc1b3e00640 1 --2- 172.21.15.152:0/1033572517 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fc19806cf50 0x7fc19806f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:54.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.242+0000 7fc1b3e00640 1 --2- 172.21.15.152:0/1033572517 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fc19806cf50 0x7fc19806f410 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc06bc00 tx=0x7fc1a4009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:54.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.244+0000 7fc1b2000640 1 -- 172.21.15.152:0/1033572517 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fc1bc06ef10 con 0x7fc1a00039e0 2024-04-20T11:35:54.487 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.487+0000 7fc1bb200640 1 -- 172.21.15.152:0/1033572517 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc1bc0b5470 con 0x7fc1a00039e0 2024-04-20T11:35:54.488 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.489+0000 7fc1b2000640 1 -- 172.21.15.152:0/1033572517 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) ==== 94+0+5212 (secure 0 0 0) 0x7fc1ac07b860 con 0x7fc1a00039e0 2024-04-20T11:35:54.488 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:35:54.489 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":87,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6062,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1303172876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6833","nonce":1303172876},{"type":"v1","addr":"172.21.15.175:6835","nonce":1303172876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6667,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6834/1975208598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1975208598},{"type":"v1","addr":"172.21.15.175:6834","nonce":1975208598}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6670,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6834/287411071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":287411071},{"type":"v1","addr":"172.21.15.152:6834","nonce":287411071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T11:35:19.231687+0000","modified":"2024-04-20T11:35:22.597626+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8919},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8919":{"gid":8919,"name":"a","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6835/1884827433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6833","nonce":1884827433},{"type":"v1","addr":"172.21.15.152:6835","nonce":1884827433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8919,"qdb_cluster":[8919]},"id":10}]} 2024-04-20T11:35:54.489 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 87 2024-04-20T11:35:54.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.491+0000 7fc1bb200640 1 -- 172.21.15.152:0/1033572517 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fc19806cf50 msgr2=0x7fc19806f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:54.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.491+0000 7fc1bb200640 1 --2- 172.21.15.152:0/1033572517 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fc19806cf50 0x7fc19806f410 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc06bc00 tx=0x7fc1a4009040 comp rx=0 tx=0).stop 2024-04-20T11:35:54.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.491+0000 7fc1bb200640 1 -- 172.21.15.152:0/1033572517 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc1a00039e0 msgr2=0x7fc1bc0b40a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:54.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.491+0000 7fc1bb200640 1 --2- 172.21.15.152:0/1033572517 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc1a00039e0 0x7fc1bc0b40a0 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac00c210 tx=0x7fc1bc15ca20 comp rx=0 tx=0).stop 2024-04-20T11:35:54.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.491+0000 7fc1bb200640 1 -- 172.21.15.152:0/1033572517 shutdown_connections 2024-04-20T11:35:54.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.491+0000 7fc1bb200640 1 --2- 172.21.15.152:0/1033572517 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fc19806cf50 0x7fc19806f410 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:54.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.491+0000 7fc1bb200640 1 --2- 172.21.15.152:0/1033572517 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc1bc0b1560 0x7fc1bc0b1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:54.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.491+0000 7fc1bb200640 1 --2- 172.21.15.152:0/1033572517 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc1bc0b2140 0x7fc1bc1f5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:54.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.491+0000 7fc1bb200640 1 --2- 172.21.15.152:0/1033572517 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc1a00039e0 0x7fc1bc0b40a0 unknown :-1 s=CLOSED pgs=1900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:54.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.491+0000 7fc1bb200640 1 -- 172.21.15.152:0/1033572517 >> 172.21.15.152:0/1033572517 conn(0x7fc1bc067ae0 msgr2=0x7fc1bc15a5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:54.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.491+0000 7fc1bb200640 1 -- 172.21.15.152:0/1033572517 shutdown_connections 2024-04-20T11:35:54.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.492+0000 7fc1bb200640 1 -- 172.21.15.152:0/1033572517 wait complete. 2024-04-20T11:35:54.510 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:35:54.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.700+0000 7f0b16000640 1 Processor -- start 2024-04-20T11:35:54.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.701+0000 7f0b16000640 1 -- start start 2024-04-20T11:35:54.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.702+0000 7f0b16000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b1015e890 0x7f0b10162cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:54.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.702+0000 7f0b16000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f0b100add80 0x7f0b100ae1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:54.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.702+0000 7f0b16000640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b100b55a0 con 0x7f0b100af750 2024-04-20T11:35:54.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.702+0000 7f0b16000640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f0b100ae720 con 0x7f0b100add80 2024-04-20T11:35:54.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.702+0000 7f0b16000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0b100b5880 con 0x7f0b1015e890 2024-04-20T11:35:54.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.702+0000 7f0b0fe00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f0b100add80 0x7f0b100ae1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:54.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.702+0000 7f0b0fe00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f0b100add80 0x7f0b100ae1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:60740/0 (socket says 172.21.15.152:60740) 2024-04-20T11:35:54.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.702+0000 7f0b0fe00640 1 -- 172.21.15.152:0/3367143334 learned_addr learned my addr 172.21.15.152:0/3367143334 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:54.703 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.703+0000 7f0b14c00640 1 --2- 172.21.15.152:0/3367143334 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b1015e890 0x7f0b10162cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:54.703 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.703+0000 7f0b0fe00640 1 -- 172.21.15.152:0/3367143334 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b1015e890 msgr2=0x7f0b10162cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:54.703 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.703+0000 7f0b0fe00640 1 --2- 172.21.15.152:0/3367143334 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b1015e890 0x7f0b10162cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:54.703 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.703+0000 7f0b0fe00640 1 -- 172.21.15.152:0/3367143334 >> v1:172.21.15.152:6789/0 conn(0x7f0b100af750 legacy=0x7f0b100afb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:54.703 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.703+0000 7f0b0fe00640 1 -- 172.21.15.152:0/3367143334 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b100b5b60 con 0x7f0b100add80 2024-04-20T11:35:54.704 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.704+0000 7f0b0fe00640 1 --2- 172.21.15.152:0/3367143334 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0b100add80 0x7f0b100ae1e0 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7f0afc009870 tx=0x7f0afc02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f82fa9c421b3ac6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:54.704 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.704+0000 7f0b0f400640 1 -- 172.21.15.152:0/3367143334 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0afc035020 con 0x7f0b100add80 2024-04-20T11:35:54.704 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.704+0000 7f0b0f400640 1 -- 172.21.15.152:0/3367143334 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0afc031640 con 0x7f0b100add80 2024-04-20T11:35:54.704 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.704+0000 7f0b0f400640 1 -- 172.21.15.152:0/3367143334 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0afc02d040 con 0x7f0b100add80 2024-04-20T11:35:54.705 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.705+0000 7f0b16000640 1 -- 172.21.15.152:0/3367143334 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0b100add80 msgr2=0x7f0b100ae1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:54.705 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.705+0000 7f0b16000640 1 --2- 172.21.15.152:0/3367143334 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0b100add80 0x7f0b100ae1e0 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7f0afc009870 tx=0x7f0afc02b8f0 comp rx=0 tx=0).stop 2024-04-20T11:35:54.705 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.705+0000 7f0b16000640 1 -- 172.21.15.152:0/3367143334 shutdown_connections 2024-04-20T11:35:54.705 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.705+0000 7f0b16000640 1 --2- 172.21.15.152:0/3367143334 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0b100add80 0x7f0b100ae1e0 unknown :-1 s=CLOSED pgs=1902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:54.706 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.705+0000 7f0b16000640 1 --2- 172.21.15.152:0/3367143334 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b1015e890 0x7f0b10162cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:54.706 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.705+0000 7f0b16000640 1 -- 172.21.15.152:0/3367143334 >> 172.21.15.152:0/3367143334 conn(0x7f0b10073340 msgr2=0x7f0b1006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:54.706 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.705+0000 7f0b16000640 1 -- 172.21.15.152:0/3367143334 shutdown_connections 2024-04-20T11:35:54.706 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.705+0000 7f0b16000640 1 -- 172.21.15.152:0/3367143334 wait complete. 2024-04-20T11:35:54.706 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.707+0000 7f0b16000640 1 Processor -- start 2024-04-20T11:35:54.706 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.707+0000 7f0b16000640 1 -- start start 2024-04-20T11:35:54.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.708+0000 7f0b16000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b100add80 0x7f0b101f5860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:54.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.708+0000 7f0b16000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b100af750 0x7f0b101523a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:54.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.708+0000 7f0b16000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0b1015e890 0x7f0b10152f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:54.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.708+0000 7f0b16000640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f0b100b5250 con 0x7f0b1015e890 2024-04-20T11:35:54.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.708+0000 7f0b16000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0b101631f0 con 0x7f0b100add80 2024-04-20T11:35:54.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.708+0000 7f0b16000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0b100b6640 con 0x7f0b100af750 2024-04-20T11:35:54.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.708+0000 7f0b15600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0b1015e890 0x7f0b10152f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:54.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.708+0000 7f0b15600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0b1015e890 0x7f0b10152f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:60744/0 (socket says 172.21.15.152:60744) 2024-04-20T11:35:54.710 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.708+0000 7f0b15600640 1 -- 172.21.15.152:0/3739858009 learned_addr learned my addr 172.21.15.152:0/3739858009 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:54.710 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.709+0000 7f0b0fe00640 1 --2- 172.21.15.152:0/3739858009 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b100af750 0x7f0b101523a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:54.710 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.709+0000 7f0b14c00640 1 --2- 172.21.15.152:0/3739858009 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b100add80 0x7f0b101f5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:54.710 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.709+0000 7f0b15600640 1 -- 172.21.15.152:0/3739858009 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b100af750 msgr2=0x7f0b101523a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:54.710 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.709+0000 7f0b15600640 1 --2- 172.21.15.152:0/3739858009 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b100af750 0x7f0b101523a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:54.710 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.709+0000 7f0b15600640 1 -- 172.21.15.152:0/3739858009 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b100add80 msgr2=0x7f0b101f5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:54.710 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.709+0000 7f0b15600640 1 --2- 172.21.15.152:0/3739858009 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b100add80 0x7f0b101f5860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:54.710 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.709+0000 7f0b15600640 1 -- 172.21.15.152:0/3739858009 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b10151650 con 0x7f0b1015e890 2024-04-20T11:35:54.710 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.709+0000 7f0b15600640 1 --2- 172.21.15.152:0/3739858009 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0b1015e890 0x7f0b10152f30 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7f0b0800c7d0 tx=0x7f0b0800cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:54.710 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.709+0000 7f0b0cc00640 1 -- 172.21.15.152:0/3739858009 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b08003b70 con 0x7f0b1015e890 2024-04-20T11:35:54.710 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.709+0000 7f0b0cc00640 1 -- 172.21.15.152:0/3739858009 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b08003d30 con 0x7f0b1015e890 2024-04-20T11:35:54.710 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.709+0000 7f0b0cc00640 1 -- 172.21.15.152:0/3739858009 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b0800aa90 con 0x7f0b1015e890 2024-04-20T11:35:54.710 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.709+0000 7f0b16000640 1 -- 172.21.15.152:0/3739858009 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b101f5da0 con 0x7f0b1015e890 2024-04-20T11:35:54.710 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.710+0000 7f0b16000640 1 -- 172.21.15.152:0/3739858009 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b10150c70 con 0x7f0b1015e890 2024-04-20T11:35:54.711 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.711+0000 7f0b16000640 1 -- 172.21.15.152:0/3739858009 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b10075e60 con 0x7f0b1015e890 2024-04-20T11:35:54.711 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.712+0000 7f0b0cc00640 1 -- 172.21.15.152:0/3739858009 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 22) ==== 91601+0+0 (secure 0 0 0) 0x7f0b0800ac30 con 0x7f0b1015e890 2024-04-20T11:35:54.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.712+0000 7f0b0cc00640 1 --2- 172.21.15.152:0/3739858009 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f0af006cf50 0x7f0af006f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:54.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.713+0000 7f0b0cc00640 1 -- 172.21.15.152:0/3739858009 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5709+0+0 (secure 0 0 0) 0x7f0b0808c6a0 con 0x7f0b1015e890 2024-04-20T11:35:54.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.713+0000 7f0b14c00640 1 --2- 172.21.15.152:0/3739858009 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f0af006cf50 0x7f0af006f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:54.714 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.715+0000 7f0b14c00640 1 --2- 172.21.15.152:0/3739858009 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f0af006cf50 0x7f0af006f410 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f0b04006fd0 tx=0x7f0b04008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:54.716 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.717+0000 7f0b0cc00640 1 -- 172.21.15.152:0/3739858009 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f0b101f5da0 con 0x7f0b1015e890 2024-04-20T11:35:54.954 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.954+0000 7f0b16000640 1 -- 172.21.15.152:0/3739858009 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0b100adb60 con 0x7f0b1015e890 2024-04-20T11:35:54.955 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.955+0000 7f0b0cc00640 1 -- 172.21.15.152:0/3739858009 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) ==== 94+0+5212 (secure 0 0 0) 0x7f0b0805a150 con 0x7f0b1015e890 2024-04-20T11:35:54.955 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:35:54.955 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":87,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6062,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1303172876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6833","nonce":1303172876},{"type":"v1","addr":"172.21.15.175:6835","nonce":1303172876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6667,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6834/1975208598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1975208598},{"type":"v1","addr":"172.21.15.175:6834","nonce":1975208598}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6670,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6834/287411071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":287411071},{"type":"v1","addr":"172.21.15.152:6834","nonce":287411071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T11:35:19.231687+0000","modified":"2024-04-20T11:35:22.597626+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8919},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8919":{"gid":8919,"name":"a","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6835/1884827433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6833","nonce":1884827433},{"type":"v1","addr":"172.21.15.152:6835","nonce":1884827433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8919,"qdb_cluster":[8919]},"id":10}]} 2024-04-20T11:35:54.956 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 87 2024-04-20T11:35:54.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.958+0000 7f0b16000640 1 -- 172.21.15.152:0/3739858009 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f0af006cf50 msgr2=0x7f0af006f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:54.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.958+0000 7f0b16000640 1 --2- 172.21.15.152:0/3739858009 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f0af006cf50 0x7f0af006f410 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f0b04006fd0 tx=0x7f0b04008040 comp rx=0 tx=0).stop 2024-04-20T11:35:54.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.958+0000 7f0b16000640 1 -- 172.21.15.152:0/3739858009 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0b1015e890 msgr2=0x7f0b10152f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:54.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.958+0000 7f0b16000640 1 --2- 172.21.15.152:0/3739858009 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0b1015e890 0x7f0b10152f30 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7f0b0800c7d0 tx=0x7f0b0800cc90 comp rx=0 tx=0).stop 2024-04-20T11:35:54.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.958+0000 7f0b16000640 1 -- 172.21.15.152:0/3739858009 shutdown_connections 2024-04-20T11:35:54.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.958+0000 7f0b16000640 1 --2- 172.21.15.152:0/3739858009 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f0af006cf50 0x7f0af006f410 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:54.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.958+0000 7f0b16000640 1 --2- 172.21.15.152:0/3739858009 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0b1015e890 0x7f0b10152f30 unknown :-1 s=CLOSED pgs=1903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:54.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.958+0000 7f0b16000640 1 --2- 172.21.15.152:0/3739858009 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b100af750 0x7f0b101523a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:54.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.958+0000 7f0b16000640 1 --2- 172.21.15.152:0/3739858009 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b100add80 0x7f0b101f5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:54.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.958+0000 7f0b16000640 1 -- 172.21.15.152:0/3739858009 >> 172.21.15.152:0/3739858009 conn(0x7f0b10073340 msgr2=0x7f0b100afb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:54.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.958+0000 7f0b16000640 1 -- 172.21.15.152:0/3739858009 shutdown_connections 2024-04-20T11:35:54.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:54.958+0000 7f0b16000640 1 -- 172.21.15.152:0/3739858009 wait complete. 2024-04-20T11:35:54.977 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:35:55.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.171+0000 7ff982c00640 1 Processor -- start 2024-04-20T11:35:55.172 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.172+0000 7ff982c00640 1 -- start start 2024-04-20T11:35:55.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.173+0000 7ff982c00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ff97c145f60 0x7ff97c06ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:55.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.173+0000 7ff982c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff97c0726f0 0x7ff97c0cd580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:55.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.173+0000 7ff982c00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff97c146340 con 0x7ff97c0cd0d0 2024-04-20T11:35:55.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.173+0000 7ff982c00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7ff97c06e9f0 con 0x7ff97c145f60 2024-04-20T11:35:55.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.173+0000 7ff982c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff97c06f3c0 con 0x7ff97c0726f0 2024-04-20T11:35:55.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.174+0000 7ff980c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff97c0726f0 0x7ff97c0cd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:55.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.174+0000 7ff980c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff97c0726f0 0x7ff97c0cd580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.152:33134/0 (socket says 172.21.15.152:33134) 2024-04-20T11:35:55.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.174+0000 7ff980c00640 1 -- 172.21.15.152:0/2386100791 learned_addr learned my addr 172.21.15.152:0/2386100791 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:55.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.174+0000 7ff97be00640 1 --2- 172.21.15.152:0/2386100791 >> v2:172.21.15.175:3300/0 conn(0x7ff97c145f60 0x7ff97c06ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:55.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.175+0000 7ff980c00640 1 -- 172.21.15.152:0/2386100791 >> v2:172.21.15.175:3300/0 conn(0x7ff97c145f60 msgr2=0x7ff97c06ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:55.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.175+0000 7ff980c00640 1 --2- 172.21.15.152:0/2386100791 >> v2:172.21.15.175:3300/0 conn(0x7ff97c145f60 0x7ff97c06ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:55.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.175+0000 7ff980c00640 1 -- 172.21.15.152:0/2386100791 >> v1:172.21.15.175:6789/0 conn(0x7ff97c0cd0d0 legacy=0x7ff97c071fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:55.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.175+0000 7ff980c00640 1 -- 172.21.15.152:0/2386100791 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff97c072d50 con 0x7ff97c0726f0 2024-04-20T11:35:55.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.176+0000 7ff980c00640 1 --2- 172.21.15.152:0/2386100791 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff97c0726f0 0x7ff97c0cd580 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff974009e50 tx=0x7ff97402bf50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=302ef546f516f03f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:55.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.177+0000 7ff97b400640 1 -- 172.21.15.152:0/2386100791 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff97402caf0 con 0x7ff97c0726f0 2024-04-20T11:35:55.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.177+0000 7ff97b400640 1 -- 172.21.15.152:0/2386100791 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff97402cc90 con 0x7ff97c0726f0 2024-04-20T11:35:55.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.177+0000 7ff97b400640 1 -- 172.21.15.152:0/2386100791 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff974034610 con 0x7ff97c0726f0 2024-04-20T11:35:55.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.177+0000 7ff982c00640 1 -- 172.21.15.152:0/2386100791 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff97c0726f0 msgr2=0x7ff97c0cd580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:55.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.177+0000 7ff982c00640 1 --2- 172.21.15.152:0/2386100791 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff97c0726f0 0x7ff97c0cd580 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff974009e50 tx=0x7ff97402bf50 comp rx=0 tx=0).stop 2024-04-20T11:35:55.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.177+0000 7ff982c00640 1 -- 172.21.15.152:0/2386100791 shutdown_connections 2024-04-20T11:35:55.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.177+0000 7ff982c00640 1 --2- 172.21.15.152:0/2386100791 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff97c0726f0 0x7ff97c0cd580 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:55.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.177+0000 7ff982c00640 1 --2- 172.21.15.152:0/2386100791 >> v2:172.21.15.175:3300/0 conn(0x7ff97c145f60 0x7ff97c06ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:55.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.177+0000 7ff982c00640 1 -- 172.21.15.152:0/2386100791 >> 172.21.15.152:0/2386100791 conn(0x7ff97c067cf0 msgr2=0x7ff97c06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:55.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.178+0000 7ff982c00640 1 -- 172.21.15.152:0/2386100791 shutdown_connections 2024-04-20T11:35:55.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.178+0000 7ff982c00640 1 -- 172.21.15.152:0/2386100791 wait complete. 2024-04-20T11:35:55.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.179+0000 7ff982c00640 1 Processor -- start 2024-04-20T11:35:55.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.179+0000 7ff982c00640 1 -- start start 2024-04-20T11:35:55.182 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.180+0000 7ff982c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff97c0726f0 0x7ff97c0b4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:55.182 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.180+0000 7ff982c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff97c0cd0d0 0x7ff97c0c5df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:55.182 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.180+0000 7ff982c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff97c145f60 0x7ff97c13de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:55.182 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.180+0000 7ff982c00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7ff97c06e680 con 0x7ff97c145f60 2024-04-20T11:35:55.182 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.180+0000 7ff982c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff97c06e3a0 con 0x7ff97c0cd0d0 2024-04-20T11:35:55.182 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.180+0000 7ff982c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff97c0cf290 con 0x7ff97c0726f0 2024-04-20T11:35:55.182 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.181+0000 7ff980c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff97c145f60 0x7ff97c13de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:55.182 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.181+0000 7ff980c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff97c145f60 0x7ff97c13de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:60752/0 (socket says 172.21.15.152:60752) 2024-04-20T11:35:55.182 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.181+0000 7ff980c00640 1 -- 172.21.15.152:0/992498219 learned_addr learned my addr 172.21.15.152:0/992498219 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:55.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.181+0000 7ff97be00640 1 --2- 172.21.15.152:0/992498219 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff97c0726f0 0x7ff97c0b4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:55.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.181+0000 7ff973400640 1 --2- 172.21.15.152:0/992498219 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff97c0cd0d0 0x7ff97c0c5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:55.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.181+0000 7ff980c00640 1 -- 172.21.15.152:0/992498219 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff97c0726f0 msgr2=0x7ff97c0b4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:55.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.181+0000 7ff980c00640 1 --2- 172.21.15.152:0/992498219 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff97c0726f0 0x7ff97c0b4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:55.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.181+0000 7ff980c00640 1 -- 172.21.15.152:0/992498219 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff97c0cd0d0 msgr2=0x7ff97c0c5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:55.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.181+0000 7ff980c00640 1 --2- 172.21.15.152:0/992498219 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff97c0cd0d0 0x7ff97c0c5df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:55.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.181+0000 7ff980c00640 1 -- 172.21.15.152:0/992498219 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff97c0c7600 con 0x7ff97c145f60 2024-04-20T11:35:55.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.181+0000 7ff980c00640 1 --2- 172.21.15.152:0/992498219 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff97c145f60 0x7ff97c13de50 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7ff974009e50 tx=0x7ff974004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:55.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.182+0000 7ff978c00640 1 -- 172.21.15.152:0/992498219 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff974043020 con 0x7ff97c145f60 2024-04-20T11:35:55.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.182+0000 7ff978c00640 1 -- 172.21.15.152:0/992498219 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff97403e450 con 0x7ff97c145f60 2024-04-20T11:35:55.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.182+0000 7ff978c00640 1 -- 172.21.15.152:0/992498219 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff974009510 con 0x7ff97c145f60 2024-04-20T11:35:55.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.182+0000 7ff982c00640 1 -- 172.21.15.152:0/992498219 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff97c0c6560 con 0x7ff97c145f60 2024-04-20T11:35:55.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.182+0000 7ff982c00640 1 -- 172.21.15.152:0/992498219 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff97c0c6b20 con 0x7ff97c145f60 2024-04-20T11:35:55.184 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.183+0000 7ff982c00640 1 -- 172.21.15.152:0/992498219 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff97c0689f0 con 0x7ff97c145f60 2024-04-20T11:35:55.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.184+0000 7ff978c00640 1 -- 172.21.15.152:0/992498219 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 22) ==== 91601+0+0 (secure 0 0 0) 0x7ff974046020 con 0x7ff97c145f60 2024-04-20T11:35:55.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.185+0000 7ff978c00640 1 --2- 172.21.15.152:0/992498219 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7ff95806cf20 0x7ff95806f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:55.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.185+0000 7ff978c00640 1 -- 172.21.15.152:0/992498219 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5709+0+0 (secure 0 0 0) 0x7ff9740aedd0 con 0x7ff97c145f60 2024-04-20T11:35:55.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.186+0000 7ff97be00640 1 --2- 172.21.15.152:0/992498219 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7ff95806cf20 0x7ff95806f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:55.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.187+0000 7ff97be00640 1 --2- 172.21.15.152:0/992498219 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7ff95806cf20 0x7ff95806f3e0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7ff96c009a30 tx=0x7ff96c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:55.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.189+0000 7ff978c00640 1 -- 172.21.15.152:0/992498219 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ff97404d030 con 0x7ff97c145f60 2024-04-20T11:35:55.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.388+0000 7ff982c00640 1 -- 172.21.15.152:0/992498219 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff97c15b560 con 0x7ff97c145f60 2024-04-20T11:35:55.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.390+0000 7ff978c00640 1 -- 172.21.15.152:0/992498219 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v131) ==== 74+0+16739 (secure 0 0 0) 0x7ff97407c950 con 0x7ff97c145f60 2024-04-20T11:35:55.390 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:35:55.390 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":131,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:35:20.781260+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:35:16.963520+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"130","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-04-20T11:35:18.215890+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:35:55.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.392+0000 7ff982c00640 1 -- 172.21.15.152:0/992498219 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7ff95806cf20 msgr2=0x7ff95806f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:55.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.392+0000 7ff982c00640 1 --2- 172.21.15.152:0/992498219 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7ff95806cf20 0x7ff95806f3e0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7ff96c009a30 tx=0x7ff96c008040 comp rx=0 tx=0).stop 2024-04-20T11:35:55.393 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.392+0000 7ff982c00640 1 -- 172.21.15.152:0/992498219 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff97c145f60 msgr2=0x7ff97c13de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:55.393 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.392+0000 7ff982c00640 1 --2- 172.21.15.152:0/992498219 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff97c145f60 0x7ff97c13de50 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7ff974009e50 tx=0x7ff974004a40 comp rx=0 tx=0).stop 2024-04-20T11:35:55.393 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.392+0000 7ff982c00640 1 -- 172.21.15.152:0/992498219 shutdown_connections 2024-04-20T11:35:55.393 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.392+0000 7ff982c00640 1 --2- 172.21.15.152:0/992498219 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7ff95806cf20 0x7ff95806f3e0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:55.393 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.392+0000 7ff982c00640 1 --2- 172.21.15.152:0/992498219 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff97c145f60 0x7ff97c13de50 unknown :-1 s=CLOSED pgs=1904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:55.393 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.392+0000 7ff982c00640 1 --2- 172.21.15.152:0/992498219 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff97c0cd0d0 0x7ff97c0c5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:55.393 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.392+0000 7ff982c00640 1 --2- 172.21.15.152:0/992498219 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff97c0726f0 0x7ff97c0b4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:55.393 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.392+0000 7ff982c00640 1 -- 172.21.15.152:0/992498219 >> 172.21.15.152:0/992498219 conn(0x7ff97c067cf0 msgr2=0x7ff97c15a1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:55.393 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.393+0000 7ff982c00640 1 -- 172.21.15.152:0/992498219 shutdown_connections 2024-04-20T11:35:55.393 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.393+0000 7ff982c00640 1 -- 172.21.15.152:0/992498219 wait complete. 2024-04-20T11:35:55.411 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-20T11:35:55.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.613+0000 7f9150a00640 1 Processor -- start 2024-04-20T11:35:55.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.613+0000 7f9150a00640 1 -- start start 2024-04-20T11:35:55.614 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.614+0000 7f9150a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f914c0b4870 0x7f914c072b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:55.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.614+0000 7f9150a00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f914c073f60 con 0x7f914c073b60 2024-04-20T11:35:55.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.614+0000 7f9150a00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f914c0ce390 con 0x7f914c15c7f0 2024-04-20T11:35:55.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.614+0000 7f9150a00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f914c0ce140 con 0x7f914c0b4870 2024-04-20T11:35:55.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.614+0000 7f9141e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f914c0b4870 0x7f914c072b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:55.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.614+0000 7f9141e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f914c0b4870 0x7f914c072b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:60756/0 (socket says 172.21.15.152:60756) 2024-04-20T11:35:55.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.614+0000 7f9141e00640 1 -- 172.21.15.152:0/928792602 learned_addr learned my addr 172.21.15.152:0/928792602 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:55.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.614+0000 7f914b200640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f914c073b60 0x7f914c0cf430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:60678/0 (socket says 172.21.15.152:60678) 2024-04-20T11:35:55.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.615+0000 7f9141e00640 1 -- 172.21.15.152:0/928792602 >> v1:172.21.15.175:6789/0 conn(0x7f914c15c7f0 legacy=0x7f914c0b4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:55.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.615+0000 7f9141e00640 1 -- 172.21.15.152:0/928792602 >> v1:172.21.15.152:6789/0 conn(0x7f914c073b60 legacy=0x7f914c0cf430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:55.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.615+0000 7f9141e00640 1 -- 172.21.15.152:0/928792602 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f914c0ce670 con 0x7f914c0b4870 2024-04-20T11:35:55.616 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.616+0000 7f9141e00640 1 --2- 172.21.15.152:0/928792602 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f914c0b4870 0x7f914c072b70 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7f9134009870 tx=0x7f913402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65af2cfa08a89ed0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:55.616 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.616+0000 7f9149e00640 1 -- 172.21.15.152:0/928792602 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9134035020 con 0x7f914c0b4870 2024-04-20T11:35:55.616 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.616+0000 7f9149e00640 1 -- 172.21.15.152:0/928792602 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9134031870 con 0x7f914c0b4870 2024-04-20T11:35:55.616 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.616+0000 7f9149e00640 1 -- 172.21.15.152:0/928792602 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9134031c40 con 0x7f914c0b4870 2024-04-20T11:35:55.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.617+0000 7f9150a00640 1 -- 172.21.15.152:0/928792602 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f914c0b4870 msgr2=0x7f914c072b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:55.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.617+0000 7f9150a00640 1 --2- 172.21.15.152:0/928792602 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f914c0b4870 0x7f914c072b70 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7f9134009870 tx=0x7f913402b910 comp rx=0 tx=0).stop 2024-04-20T11:35:55.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.617+0000 7f9150a00640 1 -- 172.21.15.152:0/928792602 shutdown_connections 2024-04-20T11:35:55.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.617+0000 7f9150a00640 1 --2- 172.21.15.152:0/928792602 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f914c0b4870 0x7f914c072b70 unknown :-1 s=CLOSED pgs=1906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:55.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.617+0000 7f9150a00640 1 -- 172.21.15.152:0/928792602 >> 172.21.15.152:0/928792602 conn(0x7f914c0d0260 msgr2=0x7f914c0d0670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:55.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.617+0000 7f9150a00640 1 -- 172.21.15.152:0/928792602 shutdown_connections 2024-04-20T11:35:55.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.617+0000 7f9150a00640 1 -- 172.21.15.152:0/928792602 wait complete. 2024-04-20T11:35:55.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.619+0000 7f9150a00640 1 Processor -- start 2024-04-20T11:35:55.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.619+0000 7f9150a00640 1 -- start start 2024-04-20T11:35:55.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.620+0000 7f9150a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f914c073b60 0x7f914c1e6b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:55.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.620+0000 7f9150a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f914c0b4870 0x7f914c1e70a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:55.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.620+0000 7f9150a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f914c155f20 0x7f914c1e75e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:55.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.620+0000 7f9150a00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f914c0cef10 con 0x7f914c073b60 2024-04-20T11:35:55.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.620+0000 7f9150a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f914c0cec30 con 0x7f914c0b4870 2024-04-20T11:35:55.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.620+0000 7f9150a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f914c0ad2b0 con 0x7f914c155f20 2024-04-20T11:35:55.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.620+0000 7f914a800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f914c073b60 0x7f914c1e6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:55.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.620+0000 7f914a800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f914c073b60 0x7f914c1e6b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:60760/0 (socket says 172.21.15.152:60760) 2024-04-20T11:35:55.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.620+0000 7f914a800640 1 -- 172.21.15.152:0/3220789483 learned_addr learned my addr 172.21.15.152:0/3220789483 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:55.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.621+0000 7f914a800640 1 -- 172.21.15.152:0/3220789483 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f914c155f20 msgr2=0x7f914c1e75e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:55.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.621+0000 7f914a800640 1 --2- 172.21.15.152:0/3220789483 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f914c155f20 0x7f914c1e75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:55.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.621+0000 7f9141e00640 1 --2- 172.21.15.152:0/3220789483 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f914c0b4870 0x7f914c1e70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:55.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.621+0000 7f914a800640 1 -- 172.21.15.152:0/3220789483 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f914c0b4870 msgr2=0x7f914c1e70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:55.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.621+0000 7f914a800640 1 --2- 172.21.15.152:0/3220789483 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f914c0b4870 0x7f914c1e70a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:55.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.621+0000 7f914a800640 1 -- 172.21.15.152:0/3220789483 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f914c06f320 con 0x7f914c073b60 2024-04-20T11:35:55.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.621+0000 7f9141e00640 1 --2- 172.21.15.152:0/3220789483 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f914c0b4870 0x7f914c1e70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:35:55.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.621+0000 7f914a800640 1 --2- 172.21.15.152:0/3220789483 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f914c073b60 0x7f914c1e6b60 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7f913c00b420 tx=0x7f913c00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:55.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.621+0000 7f9143400640 1 -- 172.21.15.152:0/3220789483 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f913c011830 con 0x7f914c073b60 2024-04-20T11:35:55.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.621+0000 7f9143400640 1 -- 172.21.15.152:0/3220789483 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f913c016070 con 0x7f914c073b60 2024-04-20T11:35:55.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.621+0000 7f9143400640 1 -- 172.21.15.152:0/3220789483 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f913c011c10 con 0x7f914c073b60 2024-04-20T11:35:55.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.621+0000 7f9150a00640 1 -- 172.21.15.152:0/3220789483 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f914c1e5da0 con 0x7f914c073b60 2024-04-20T11:35:55.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.621+0000 7f9150a00640 1 -- 172.21.15.152:0/3220789483 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f914c1e6360 con 0x7f914c073b60 2024-04-20T11:35:55.623 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.623+0000 7f9150a00640 1 -- 172.21.15.152:0/3220789483 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f914c075e60 con 0x7f914c073b60 2024-04-20T11:35:55.623 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.623+0000 7f9143400640 1 -- 172.21.15.152:0/3220789483 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 22) ==== 91601+0+0 (secure 0 0 0) 0x7f913c01b460 con 0x7f914c073b60 2024-04-20T11:35:55.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.624+0000 7f9143400640 1 --2- 172.21.15.152:0/3220789483 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f912806cf20 0x7f912806f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:55.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.624+0000 7f9143400640 1 -- 172.21.15.152:0/3220789483 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5709+0+0 (secure 0 0 0) 0x7f913c08c1c0 con 0x7f914c073b60 2024-04-20T11:35:55.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.625+0000 7f9141e00640 1 --2- 172.21.15.152:0/3220789483 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f912806cf20 0x7f912806f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:55.625 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.626+0000 7f9141e00640 1 --2- 172.21.15.152:0/3220789483 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f912806cf20 0x7f912806f3e0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f9134036000 tx=0x7f9134008030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:55.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.629+0000 7f9143400640 1 -- 172.21.15.152:0/3220789483 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f913c020020 con 0x7f914c073b60 2024-04-20T11:35:55.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.871+0000 7f9150a00640 1 -- 172.21.15.152:0/3220789483 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f914c0b15b0 con 0x7f914c073b60 2024-04-20T11:35:55.872 INFO:tasks.ceph.mon.a.smithi152.stderr:2024-04-20T11:35:55.873+0000 7f8229800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-20T11:35:55.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.876+0000 7f9143400640 1 -- 172.21.15.152:0/3220789483 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v88) ==== 161+0+0 (secure 0 0 0) 0x7f913c059c70 con 0x7f914c073b60 2024-04-20T11:35:55.876 INFO:teuthology.orchestra.run.smithi152.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-20T11:35:55.879 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.880+0000 7f9150a00640 1 -- 172.21.15.152:0/3220789483 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f912806cf20 msgr2=0x7f912806f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:55.879 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.880+0000 7f9150a00640 1 --2- 172.21.15.152:0/3220789483 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f912806cf20 0x7f912806f3e0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f9134036000 tx=0x7f9134008030 comp rx=0 tx=0).stop 2024-04-20T11:35:55.879 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.880+0000 7f9150a00640 1 -- 172.21.15.152:0/3220789483 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f914c073b60 msgr2=0x7f914c1e6b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:55.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.880+0000 7f9150a00640 1 --2- 172.21.15.152:0/3220789483 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f914c073b60 0x7f914c1e6b60 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7f913c00b420 tx=0x7f913c00b8e0 comp rx=0 tx=0).stop 2024-04-20T11:35:55.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.880+0000 7f9150a00640 1 -- 172.21.15.152:0/3220789483 shutdown_connections 2024-04-20T11:35:55.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.880+0000 7f9150a00640 1 --2- 172.21.15.152:0/3220789483 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f912806cf20 0x7f912806f3e0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:55.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.880+0000 7f9150a00640 1 --2- 172.21.15.152:0/3220789483 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f914c155f20 0x7f914c1e75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:55.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.880+0000 7f9150a00640 1 --2- 172.21.15.152:0/3220789483 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f914c0b4870 0x7f914c1e70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:55.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.880+0000 7f9150a00640 1 --2- 172.21.15.152:0/3220789483 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f914c073b60 0x7f914c1e6b60 unknown :-1 s=CLOSED pgs=1907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:55.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.880+0000 7f9150a00640 1 -- 172.21.15.152:0/3220789483 >> 172.21.15.152:0/3220789483 conn(0x7f914c0d0260 msgr2=0x7f914c15b990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:55.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.881+0000 7f9150a00640 1 -- 172.21.15.152:0/3220789483 shutdown_connections 2024-04-20T11:35:55.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:55.881+0000 7f9150a00640 1 -- 172.21.15.152:0/3220789483 wait complete. 2024-04-20T11:35:55.886 INFO:tasks.ceph.mds.a.smithi152.stderr: -2149> 2024-04-20T11:35:25.729+0000 7f8784c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-20T11:35:55.900 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-20T11:35:55.935 INFO:tasks.ceph.mds.a.smithi152.stdout:starting mds.a at 2024-04-20T11:35:56.071 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:56.071+0000 7f0122e00640 1 Processor -- start 2024-04-20T11:35:56.071 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:56.071+0000 7f0122e00640 1 -- start start 2024-04-20T11:35:56.073 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:56.073+0000 7f0122e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f011c15e5d0 0x7f011c162440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:56.073 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:56.073+0000 7f0122e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f011c0b3fa0 0x7f011c0af490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:56.073 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:56.073+0000 7f0122e00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f011c0adc30 con 0x7f011c15e9b0 2024-04-20T11:35:56.073 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:56.073+0000 7f0122e00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f011c162a70 con 0x7f011c0b3fa0 2024-04-20T11:35:56.073 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:56.073+0000 7f0122e00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f011c162d50 con 0x7f011c15e5d0 2024-04-20T11:35:56.073 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:56.073+0000 7f0113400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f011c0b3fa0 0x7f011c0af490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:56.073 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:56.073+0000 7f0113400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f011c0b3fa0 0x7f011c0af490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36260/0 (socket says 172.21.15.152:36260) 2024-04-20T11:35:56.073 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:56.073+0000 7f0113400640 1 -- 172.21.15.152:0/221836105 learned_addr learned my addr 172.21.15.152:0/221836105 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:56.074 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:56.074+0000 7f0113400640 1 -- 172.21.15.152:0/221836105 >> v2:172.21.15.175:3300/0 conn(0x7f011c15e5d0 msgr2=0x7f011c162440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:56.074 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:56.074+0000 7f0113400640 1 --2- 172.21.15.152:0/221836105 >> v2:172.21.15.175:3300/0 conn(0x7f011c15e5d0 0x7f011c162440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:56.074 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:56.074+0000 7f0113400640 1 -- 172.21.15.152:0/221836105 >> v1:172.21.15.175:6789/0 conn(0x7f011c15e9b0 legacy=0x7f011c071db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:56.074 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:56.074+0000 7f0113400640 1 -- 172.21.15.152:0/221836105 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f011c163030 con 0x7f011c0b3fa0 2024-04-20T11:35:56.075 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:56.075+0000 7f0113400640 1 --2- 172.21.15.152:0/221836105 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f011c0b3fa0 0x7f011c0af490 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7f011c06ff00 tx=0x7f010402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2080364300d6aa4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:56.075 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:56.075+0000 7f011b400640 1 -- 172.21.15.152:0/221836105 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0104035020 con 0x7f011c0b3fa0 2024-04-20T11:35:56.075 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:56.075+0000 7f011b400640 1 -- 172.21.15.152:0/221836105 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0104031870 con 0x7f011c0b3fa0 2024-04-20T11:35:56.075 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:56.075+0000 7f011b400640 1 -- 172.21.15.152:0/221836105 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0104031c40 con 0x7f011c0b3fa0 2024-04-20T11:35:56.076 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:56.076+0000 7f0122e00640 1 -- 172.21.15.152:0/221836105 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f011c0b3fa0 msgr2=0x7f011c0af490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:56.076 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:56.076+0000 7f0122e00640 1 --2- 172.21.15.152:0/221836105 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f011c0b3fa0 0x7f011c0af490 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7f011c06ff00 tx=0x7f010402b910 comp rx=0 tx=0).stop 2024-04-20T11:35:56.076 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:56.076+0000 7f0122e00640 1 -- 172.21.15.152:0/221836105 shutdown_connections 2024-04-20T11:35:56.076 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:56.076+0000 7f0122e00640 1 --2- 172.21.15.152:0/221836105 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f011c0b3fa0 0x7f011c0af490 unknown :-1 s=CLOSED pgs=1909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:56.076 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:56.076+0000 7f0122e00640 1 --2- 172.21.15.152:0/221836105 >> v2:172.21.15.175:3300/0 conn(0x7f011c15e5d0 0x7f011c162440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:56.076 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:56.076+0000 7f0122e00640 1 -- 172.21.15.152:0/221836105 >> 172.21.15.152:0/221836105 conn(0x7f011c06b980 msgr2=0x7f011c06bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:56.076 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:56.076+0000 7f0122e00640 1 -- 172.21.15.152:0/221836105 shutdown_connections 2024-04-20T11:35:56.076 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:56.076+0000 7f0122e00640 1 -- 172.21.15.152:0/221836105 wait complete. 2024-04-20T11:35:56.077 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:56.077+0000 7f0122e00640 1 Processor -- start 2024-04-20T11:35:56.077 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:56.078+0000 7f0122e00640 1 -- start start 2024-04-20T11:35:56.079 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:56.078+0000 7f0122e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f011c0b3fa0 0x7f011c0ccec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:56.079 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:56.078+0000 7f0122e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f011c15e5d0 0x7f011c155a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:56.079 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:56.078+0000 7f0122e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f011c15e9b0 0x7f011c0aed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:56.079 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:56.079+0000 7f0122e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f011c06e0a0 con 0x7f011c15e5d0 2024-04-20T11:35:56.079 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:56.079+0000 7f0122e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f011c163310 con 0x7f011c15e9b0 2024-04-20T11:35:56.079 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:56.079+0000 7f0122e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f011c06e220 con 0x7f011c0b3fa0 2024-04-20T11:35:56.079 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:56.079+0000 7f0113400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f011c15e5d0 0x7f011c155a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:56.079 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:56.079+0000 7f0113400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f011c15e5d0 0x7f011c155a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36274/0 (socket says 172.21.15.152:36274) 2024-04-20T11:35:56.080 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:56.079+0000 7f0113400640 1 -- 172.21.15.152:0/2226861086 learned_addr learned my addr 172.21.15.152:0/2226861086 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:56.080 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:56.079+0000 7f0113400640 1 -- 172.21.15.152:0/2226861086 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f011c0b3fa0 msgr2=0x7f011c0ccec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:56.080 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:56.079+0000 7f0113400640 1 --2- 172.21.15.152:0/2226861086 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f011c0b3fa0 0x7f011c0ccec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:56.080 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:56.079+0000 7f0113400640 1 -- 172.21.15.152:0/2226861086 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f011c15e9b0 msgr2=0x7f011c0aed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:56.080 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:56.079+0000 7f0113400640 1 --2- 172.21.15.152:0/2226861086 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f011c15e9b0 0x7f011c0aed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:56.080 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:56.079+0000 7f0113400640 1 -- 172.21.15.152:0/2226861086 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f011c0cd400 con 0x7f011c15e5d0 2024-04-20T11:35:56.080 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:56.079+0000 7f0113400640 1 --2- 172.21.15.152:0/2226861086 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f011c15e5d0 0x7f011c155a90 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7f0104000c00 tx=0x7f01040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:56.081 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:56.080+0000 7f0118c00640 1 -- 172.21.15.152:0/2226861086 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f010403ad60 con 0x7f011c15e5d0 2024-04-20T11:35:56.081 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:56.080+0000 7f0118c00640 1 -- 172.21.15.152:0/2226861086 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01040443f0 con 0x7f011c15e5d0 2024-04-20T11:35:56.081 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:56.080+0000 7f0118c00640 1 -- 172.21.15.152:0/2226861086 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01040447e0 con 0x7f011c15e5d0 2024-04-20T11:35:56.081 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:56.080+0000 7f0122e00640 1 -- 172.21.15.152:0/2226861086 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f011c1604d0 con 0x7f011c15e5d0 2024-04-20T11:35:56.081 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:56.080+0000 7f0122e00640 1 -- 172.21.15.152:0/2226861086 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f011c160a90 con 0x7f011c15e5d0 2024-04-20T11:35:56.081 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:56.081+0000 7f0122e00640 1 -- 172.21.15.152:0/2226861086 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f011c0689f0 con 0x7f011c15e5d0 2024-04-20T11:35:56.082 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:56.082+0000 7f0118c00640 1 -- 172.21.15.152:0/2226861086 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 22) ==== 91601+0+0 (secure 0 0 0) 0x7f0104038020 con 0x7f011c15e5d0 2024-04-20T11:35:56.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:56.083+0000 7f0118c00640 1 --2- 172.21.15.152:0/2226861086 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f00f806cf50 0x7f00f806f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:56.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:56.083+0000 7f0118c00640 1 -- 172.21.15.152:0/2226861086 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5795+0+0 (secure 0 0 0) 0x7f01040aeb50 con 0x7f011c15e5d0 2024-04-20T11:35:56.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:56.084+0000 7f011be00640 1 --2- 172.21.15.152:0/2226861086 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f00f806cf50 0x7f00f806f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:56.085 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:56.085+0000 7f011be00640 1 --2- 172.21.15.152:0/2226861086 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f00f806cf50 0x7f00f806f410 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f010c009710 tx=0x7f010c006c90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:56.087 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:56.088+0000 7f0118c00640 1 -- 172.21.15.152:0/2226861086 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f010403f070 con 0x7f011c15e5d0 2024-04-20T11:35:56.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:56.310+0000 7f0122e00640 1 -- 172.21.15.152:0/2226861086 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f011c0b3530 con 0x7f011c15e5d0 2024-04-20T11:35:56.879 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:56.879+0000 7f0118c00640 1 -- 172.21.15.152:0/2226861086 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v89) ==== 104+0+0 (secure 0 0 0) 0x7f010407c400 con 0x7f011c15e5d0 2024-04-20T11:35:56.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:56.884+0000 7f0122e00640 1 -- 172.21.15.152:0/2226861086 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f00f806cf50 msgr2=0x7f00f806f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:56.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:56.884+0000 7f0122e00640 1 --2- 172.21.15.152:0/2226861086 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f00f806cf50 0x7f00f806f410 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f010c009710 tx=0x7f010c006c90 comp rx=0 tx=0).stop 2024-04-20T11:35:56.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:56.884+0000 7f0122e00640 1 -- 172.21.15.152:0/2226861086 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f011c15e5d0 msgr2=0x7f011c155a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:56.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:56.884+0000 7f0122e00640 1 --2- 172.21.15.152:0/2226861086 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f011c15e5d0 0x7f011c155a90 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7f0104000c00 tx=0x7f01040046c0 comp rx=0 tx=0).stop 2024-04-20T11:35:56.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:56.885+0000 7f0122e00640 1 -- 172.21.15.152:0/2226861086 shutdown_connections 2024-04-20T11:35:56.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:56.885+0000 7f0122e00640 1 --2- 172.21.15.152:0/2226861086 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f00f806cf50 0x7f00f806f410 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:56.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:56.885+0000 7f0122e00640 1 --2- 172.21.15.152:0/2226861086 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f011c15e9b0 0x7f011c0aed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:56.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:56.885+0000 7f0122e00640 1 --2- 172.21.15.152:0/2226861086 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f011c15e5d0 0x7f011c155a90 unknown :-1 s=CLOSED pgs=1910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:56.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:56.885+0000 7f0122e00640 1 --2- 172.21.15.152:0/2226861086 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f011c0b3fa0 0x7f011c0ccec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:56.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:56.885+0000 7f0122e00640 1 -- 172.21.15.152:0/2226861086 >> 172.21.15.152:0/2226861086 conn(0x7f011c06b980 msgr2=0x7f011c15b170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:56.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:56.885+0000 7f0122e00640 1 -- 172.21.15.152:0/2226861086 shutdown_connections 2024-04-20T11:35:56.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:56.885+0000 7f0122e00640 1 -- 172.21.15.152:0/2226861086 wait complete. 2024-04-20T11:35:56.911 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-20T11:35:57.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:57.156+0000 7f5c68400640 1 Processor -- start 2024-04-20T11:35:57.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:57.156+0000 7f5c68400640 1 -- start start 2024-04-20T11:35:57.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:57.157+0000 7f5c68400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5c60144f80 0x7f5c6006eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:57.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:57.157+0000 7f5c68400640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f5c6006f420 0x7f5c600cd220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:57.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:57.158+0000 7f5c68400640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c60072dc0 con 0x7f5c600cd760 2024-04-20T11:35:57.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:57.158+0000 7f5c68400640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f5c6015f1a0 con 0x7f5c6006f420 2024-04-20T11:35:57.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:57.158+0000 7f5c68400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5c6015f480 con 0x7f5c60144f80 2024-04-20T11:35:57.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:57.158+0000 7f5c65a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5c60144f80 0x7f5c6006eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:57.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:57.158+0000 7f5c65a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5c60144f80 0x7f5c6006eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.152:44232/0 (socket says 172.21.15.152:44232) 2024-04-20T11:35:57.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:57.158+0000 7f5c65a00640 1 -- 172.21.15.152:0/1614677851 learned_addr learned my addr 172.21.15.152:0/1614677851 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:57.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:57.159+0000 7f5c65000640 1 --2- 172.21.15.152:0/1614677851 >> v2:172.21.15.175:3300/0 conn(0x7f5c6006f420 0x7f5c600cd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:57.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:57.159+0000 7f5c65a00640 1 -- 172.21.15.152:0/1614677851 >> v2:172.21.15.175:3300/0 conn(0x7f5c6006f420 msgr2=0x7f5c600cd220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:57.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:57.159+0000 7f5c65a00640 1 --2- 172.21.15.152:0/1614677851 >> v2:172.21.15.175:3300/0 conn(0x7f5c6006f420 0x7f5c600cd220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:57.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:57.159+0000 7f5c65a00640 1 -- 172.21.15.152:0/1614677851 >> v1:172.21.15.175:6789/0 conn(0x7f5c600cd760 legacy=0x7f5c600cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:57.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:57.159+0000 7f5c65a00640 1 -- 172.21.15.152:0/1614677851 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c6015f760 con 0x7f5c60144f80 2024-04-20T11:35:57.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:57.160+0000 7f5c65a00640 1 --2- 172.21.15.152:0/1614677851 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5c60144f80 0x7f5c6006eee0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5c54009870 tx=0x7f5c5402b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83560960fdadab4d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:57.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:57.161+0000 7f5c5fe00640 1 -- 172.21.15.152:0/1614677851 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c54034070 con 0x7f5c60144f80 2024-04-20T11:35:57.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:57.161+0000 7f5c5fe00640 1 -- 172.21.15.152:0/1614677851 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c54030840 con 0x7f5c60144f80 2024-04-20T11:35:57.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:57.161+0000 7f5c5fe00640 1 -- 172.21.15.152:0/1614677851 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c54030c10 con 0x7f5c60144f80 2024-04-20T11:35:57.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:57.162+0000 7f5c68400640 1 -- 172.21.15.152:0/1614677851 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5c60144f80 msgr2=0x7f5c6006eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:57.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:57.162+0000 7f5c68400640 1 --2- 172.21.15.152:0/1614677851 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5c60144f80 0x7f5c6006eee0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5c54009870 tx=0x7f5c5402b980 comp rx=0 tx=0).stop 2024-04-20T11:35:57.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:57.162+0000 7f5c68400640 1 -- 172.21.15.152:0/1614677851 shutdown_connections 2024-04-20T11:35:57.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:57.162+0000 7f5c68400640 1 --2- 172.21.15.152:0/1614677851 >> v2:172.21.15.175:3300/0 conn(0x7f5c6006f420 0x7f5c600cd220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:57.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:57.162+0000 7f5c68400640 1 --2- 172.21.15.152:0/1614677851 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5c60144f80 0x7f5c6006eee0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:57.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:57.162+0000 7f5c68400640 1 -- 172.21.15.152:0/1614677851 >> 172.21.15.152:0/1614677851 conn(0x7f5c60067cf0 msgr2=0x7f5c6006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:57.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:57.162+0000 7f5c68400640 1 -- 172.21.15.152:0/1614677851 shutdown_connections 2024-04-20T11:35:57.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:57.162+0000 7f5c68400640 1 -- 172.21.15.152:0/1614677851 wait complete. 2024-04-20T11:35:57.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:57.163+0000 7f5c68400640 1 Processor -- start 2024-04-20T11:35:57.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:57.164+0000 7f5c68400640 1 -- start start 2024-04-20T11:35:57.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:57.164+0000 7f5c68400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5c6006f420 0x7f5c60143200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:57.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:57.165+0000 7f5c68400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5c600cd760 0x7f5c601561b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:57.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:57.165+0000 7f5c68400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5c60144f80 0x7f5c601f4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:57.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:57.165+0000 7f5c68400640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f5c600cdbf0 con 0x7f5c60144f80 2024-04-20T11:35:57.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:57.165+0000 7f5c68400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5c600b3e10 con 0x7f5c6006f420 2024-04-20T11:35:57.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:57.165+0000 7f5c68400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5c600cdd70 con 0x7f5c600cd760 2024-04-20T11:35:57.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:57.165+0000 7f5c66400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5c60144f80 0x7f5c601f4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:57.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:57.165+0000 7f5c66400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5c60144f80 0x7f5c601f4f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36278/0 (socket says 172.21.15.152:36278) 2024-04-20T11:35:57.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:57.165+0000 7f5c66400640 1 -- 172.21.15.152:0/2770223061 learned_addr learned my addr 172.21.15.152:0/2770223061 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:57.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:57.165+0000 7f5c65000640 1 --2- 172.21.15.152:0/2770223061 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5c600cd760 0x7f5c601561b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:57.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:57.165+0000 7f5c65a00640 1 --2- 172.21.15.152:0/2770223061 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5c6006f420 0x7f5c60143200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:57.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:57.165+0000 7f5c66400640 1 -- 172.21.15.152:0/2770223061 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5c600cd760 msgr2=0x7f5c601561b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:57.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:57.165+0000 7f5c66400640 1 --2- 172.21.15.152:0/2770223061 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5c600cd760 0x7f5c601561b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:57.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:57.165+0000 7f5c66400640 1 -- 172.21.15.152:0/2770223061 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5c6006f420 msgr2=0x7f5c60143200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:57.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:57.165+0000 7f5c66400640 1 --2- 172.21.15.152:0/2770223061 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5c6006f420 0x7f5c60143200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:57.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:57.165+0000 7f5c66400640 1 -- 172.21.15.152:0/2770223061 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c60156a90 con 0x7f5c60144f80 2024-04-20T11:35:57.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:57.166+0000 7f5c66400640 1 --2- 172.21.15.152:0/2770223061 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5c60144f80 0x7f5c601f4f50 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f5c5800c7a0 tx=0x7f5c5800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:57.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:57.166+0000 7f5c5d600640 1 -- 172.21.15.152:0/2770223061 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c58003b70 con 0x7f5c60144f80 2024-04-20T11:35:57.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:57.166+0000 7f5c5d600640 1 -- 172.21.15.152:0/2770223061 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c58003d30 con 0x7f5c60144f80 2024-04-20T11:35:57.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:57.166+0000 7f5c5d600640 1 -- 172.21.15.152:0/2770223061 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c5800aa30 con 0x7f5c60144f80 2024-04-20T11:35:57.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:57.166+0000 7f5c68400640 1 -- 172.21.15.152:0/2770223061 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c60156d70 con 0x7f5c60144f80 2024-04-20T11:35:57.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:57.166+0000 7f5c68400640 1 -- 172.21.15.152:0/2770223061 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c60157330 con 0x7f5c60144f80 2024-04-20T11:35:57.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:57.167+0000 7f5c68400640 1 -- 172.21.15.152:0/2770223061 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c6006fc50 con 0x7f5c60144f80 2024-04-20T11:35:57.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:57.168+0000 7f5c5d600640 1 -- 172.21.15.152:0/2770223061 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 22) ==== 91601+0+0 (secure 0 0 0) 0x7f5c5800abd0 con 0x7f5c60144f80 2024-04-20T11:35:57.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:57.169+0000 7f5c5d600640 1 --2- 172.21.15.152:0/2770223061 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f5c4006cf20 0x7f5c4006f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:57.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:57.170+0000 7f5c5d600640 1 -- 172.21.15.152:0/2770223061 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5795+0+0 (secure 0 0 0) 0x7f5c5808c650 con 0x7f5c60144f80 2024-04-20T11:35:57.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:57.170+0000 7f5c65a00640 1 --2- 172.21.15.152:0/2770223061 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f5c4006cf20 0x7f5c4006f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:57.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:57.171+0000 7f5c65a00640 1 --2- 172.21.15.152:0/2770223061 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f5c4006cf20 0x7f5c4006f3e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f5c54036000 tx=0x7f5c5403b040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:57.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:57.174+0000 7f5c5d600640 1 -- 172.21.15.152:0/2770223061 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f5c6006fc50 con 0x7f5c60144f80 2024-04-20T11:35:57.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:57.373+0000 7f5c68400640 1 -- 172.21.15.152:0/2770223061 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f5c600b36e0 con 0x7f5c60144f80 2024-04-20T11:35:58.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:58.345+0000 7f5c5d600640 1 -- 172.21.15.152:0/2770223061 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v133) ==== 181+0+0 (secure 0 0 0) 0x7f5c5805a0a0 con 0x7f5c60144f80 2024-04-20T11:35:58.345 INFO:teuthology.orchestra.run.smithi152.stderr:pool 'cephfs_metadata' removed 2024-04-20T11:35:58.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:58.348+0000 7f5c68400640 1 -- 172.21.15.152:0/2770223061 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f5c4006cf20 msgr2=0x7f5c4006f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:58.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:58.348+0000 7f5c68400640 1 --2- 172.21.15.152:0/2770223061 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f5c4006cf20 0x7f5c4006f3e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f5c54036000 tx=0x7f5c5403b040 comp rx=0 tx=0).stop 2024-04-20T11:35:58.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:58.348+0000 7f5c68400640 1 -- 172.21.15.152:0/2770223061 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5c60144f80 msgr2=0x7f5c601f4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:58.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:58.348+0000 7f5c68400640 1 --2- 172.21.15.152:0/2770223061 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5c60144f80 0x7f5c601f4f50 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f5c5800c7a0 tx=0x7f5c5800cc60 comp rx=0 tx=0).stop 2024-04-20T11:35:58.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:58.348+0000 7f5c68400640 1 -- 172.21.15.152:0/2770223061 shutdown_connections 2024-04-20T11:35:58.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:58.348+0000 7f5c68400640 1 --2- 172.21.15.152:0/2770223061 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f5c4006cf20 0x7f5c4006f3e0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:58.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:58.348+0000 7f5c68400640 1 --2- 172.21.15.152:0/2770223061 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5c60144f80 0x7f5c601f4f50 unknown :-1 s=CLOSED pgs=1911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:58.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:58.348+0000 7f5c68400640 1 --2- 172.21.15.152:0/2770223061 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5c600cd760 0x7f5c601561b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:58.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:58.348+0000 7f5c68400640 1 --2- 172.21.15.152:0/2770223061 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5c6006f420 0x7f5c60143200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:58.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:58.348+0000 7f5c68400640 1 -- 172.21.15.152:0/2770223061 >> 172.21.15.152:0/2770223061 conn(0x7f5c60067cf0 msgr2=0x7f5c600b3f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:58.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:58.349+0000 7f5c68400640 1 -- 172.21.15.152:0/2770223061 shutdown_connections 2024-04-20T11:35:58.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:58.349+0000 7f5c68400640 1 -- 172.21.15.152:0/2770223061 wait complete. 2024-04-20T11:35:58.373 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-20T11:35:58.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:58.571+0000 7f826a600640 1 Processor -- start 2024-04-20T11:35:58.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:58.571+0000 7f826a600640 1 -- start start 2024-04-20T11:35:58.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:58.573+0000 7f826a600640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f82640b3470 0x7f8264145620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:58.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:58.573+0000 7f826a600640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82640b3a50 con 0x7f8264144fa0 2024-04-20T11:35:58.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:58.573+0000 7f826a600640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82640b3d30 con 0x7f8264072530 2024-04-20T11:35:58.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:58.573+0000 7f826a600640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f82640b4010 con 0x7f82640b3470 2024-04-20T11:35:58.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:58.573+0000 7f8263e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f82640b3470 0x7f8264145620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:58.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:58.573+0000 7f8263e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f82640b3470 0x7f8264145620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36284/0 (socket says 172.21.15.152:36284) 2024-04-20T11:35:58.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:58.573+0000 7f8263e00640 1 -- 172.21.15.152:0/2228808437 learned_addr learned my addr 172.21.15.152:0/2228808437 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:58.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:58.573+0000 7f8263400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f8264144fa0 0x7f8264071e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:50016/0 (socket says 172.21.15.152:50016) 2024-04-20T11:35:58.574 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:58.574+0000 7f8263e00640 1 -- 172.21.15.152:0/2228808437 >> v1:172.21.15.175:6789/0 conn(0x7f8264072530 legacy=0x7f82640af100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:58.574 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:58.574+0000 7f8263e00640 1 -- 172.21.15.152:0/2228808437 >> v1:172.21.15.152:6789/0 conn(0x7f8264144fa0 legacy=0x7f8264071e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:58.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:58.574+0000 7f8263e00640 1 -- 172.21.15.152:0/2228808437 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82640b42f0 con 0x7f82640b3470 2024-04-20T11:35:58.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:58.574+0000 7f8262000640 1 -- 172.21.15.152:0/2228808437 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3386942903 0 0) 0x7f82640b3a50 con 0x7f8264144fa0 2024-04-20T11:35:58.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:58.575+0000 7f8263e00640 1 --2- 172.21.15.152:0/2228808437 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f82640b3470 0x7f8264145620 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7f825c009e50 tx=0x7f825c02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4405a50a8d1760e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:58.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:58.575+0000 7f8262000640 1 -- 172.21.15.152:0/2228808437 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f825c02c9e0 con 0x7f82640b3470 2024-04-20T11:35:58.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:58.575+0000 7f8262000640 1 -- 172.21.15.152:0/2228808437 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f825c02cb80 con 0x7f82640b3470 2024-04-20T11:35:58.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:58.575+0000 7f8262000640 1 -- 172.21.15.152:0/2228808437 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f825c0343f0 con 0x7f82640b3470 2024-04-20T11:35:58.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:58.576+0000 7f826a600640 1 -- 172.21.15.152:0/2228808437 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f82640b3470 msgr2=0x7f8264145620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:58.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:58.576+0000 7f826a600640 1 --2- 172.21.15.152:0/2228808437 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f82640b3470 0x7f8264145620 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7f825c009e50 tx=0x7f825c02be40 comp rx=0 tx=0).stop 2024-04-20T11:35:58.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:58.576+0000 7f826a600640 1 -- 172.21.15.152:0/2228808437 shutdown_connections 2024-04-20T11:35:58.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:58.576+0000 7f826a600640 1 --2- 172.21.15.152:0/2228808437 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f82640b3470 0x7f8264145620 unknown :-1 s=CLOSED pgs=1913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:58.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:58.576+0000 7f826a600640 1 -- 172.21.15.152:0/2228808437 >> 172.21.15.152:0/2228808437 conn(0x7f826406b980 msgr2=0x7f826406bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:58.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:58.576+0000 7f826a600640 1 -- 172.21.15.152:0/2228808437 shutdown_connections 2024-04-20T11:35:58.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:58.576+0000 7f826a600640 1 -- 172.21.15.152:0/2228808437 wait complete. 2024-04-20T11:35:58.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:58.578+0000 7f826a600640 1 Processor -- start 2024-04-20T11:35:58.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:58.578+0000 7f826a600640 1 -- start start 2024-04-20T11:35:58.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:58.579+0000 7f826a600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8264072530 0x7f82641f3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:58.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:58.579+0000 7f826a600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f82640b3470 0x7f826415fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:58.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:58.579+0000 7f826a600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8264144fa0 0x7f8264141070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:58.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:58.579+0000 7f826a600640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f826406ea60 con 0x7f8264144fa0 2024-04-20T11:35:58.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:58.579+0000 7f826a600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f826406e8e0 con 0x7f82640b3470 2024-04-20T11:35:58.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:58.579+0000 7f826a600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f826406ebe0 con 0x7f8264072530 2024-04-20T11:35:58.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:58.579+0000 7f8263e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8264144fa0 0x7f8264141070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:58.580 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:58.579+0000 7f8263e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8264144fa0 0x7f8264141070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36296/0 (socket says 172.21.15.152:36296) 2024-04-20T11:35:58.580 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:58.579+0000 7f8263e00640 1 -- 172.21.15.152:0/3520118211 learned_addr learned my addr 172.21.15.152:0/3520118211 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:58.580 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:58.579+0000 7f8263e00640 1 -- 172.21.15.152:0/3520118211 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8264072530 msgr2=0x7f82641f3980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:58.580 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:58.579+0000 7f8263e00640 1 --2- 172.21.15.152:0/3520118211 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8264072530 0x7f82641f3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:58.580 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:58.579+0000 7f8263e00640 1 -- 172.21.15.152:0/3520118211 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f82640b3470 msgr2=0x7f826415fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:58.580 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:58.579+0000 7f8263e00640 1 --2- 172.21.15.152:0/3520118211 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f82640b3470 0x7f826415fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:58.580 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:58.579+0000 7f8263e00640 1 -- 172.21.15.152:0/3520118211 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f826413ea20 con 0x7f8264144fa0 2024-04-20T11:35:58.580 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:58.580+0000 7f8263e00640 1 --2- 172.21.15.152:0/3520118211 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8264144fa0 0x7f8264141070 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7f825c009e50 tx=0x7f825c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:58.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:58.580+0000 7f825b400640 1 -- 172.21.15.152:0/3520118211 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f825c043020 con 0x7f8264144fa0 2024-04-20T11:35:58.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:58.580+0000 7f825b400640 1 -- 172.21.15.152:0/3520118211 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f825c03e450 con 0x7f8264144fa0 2024-04-20T11:35:58.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:58.580+0000 7f825b400640 1 -- 172.21.15.152:0/3520118211 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f825c03e820 con 0x7f8264144fa0 2024-04-20T11:35:58.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:58.580+0000 7f826a600640 1 -- 172.21.15.152:0/3520118211 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8264160870 con 0x7f8264144fa0 2024-04-20T11:35:58.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:58.580+0000 7f826a600640 1 -- 172.21.15.152:0/3520118211 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f826413ecc0 con 0x7f8264144fa0 2024-04-20T11:35:58.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:58.581+0000 7f826a600640 1 -- 172.21.15.152:0/3520118211 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8264068a60 con 0x7f8264144fa0 2024-04-20T11:35:58.582 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:58.582+0000 7f825b400640 1 -- 172.21.15.152:0/3520118211 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 22) ==== 91601+0+0 (secure 0 0 0) 0x7f825c046020 con 0x7f8264144fa0 2024-04-20T11:35:58.583 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:58.583+0000 7f825b400640 1 --2- 172.21.15.152:0/3520118211 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f824006cf20 0x7f824006f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:58.583 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:58.583+0000 7f825b400640 1 -- 172.21.15.152:0/3520118211 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5357+0+0 (secure 0 0 0) 0x7f825c0aef20 con 0x7f8264144fa0 2024-04-20T11:35:58.583 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:58.584+0000 7f8263400640 1 --2- 172.21.15.152:0/3520118211 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f824006cf20 0x7f824006f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:58.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:58.585+0000 7f8263400640 1 --2- 172.21.15.152:0/3520118211 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f824006cf20 0x7f824006f3e0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f8254007fd0 tx=0x7f8254007920 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:58.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:58.588+0000 7f825b400640 1 -- 172.21.15.152:0/3520118211 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f825c04d030 con 0x7f8264144fa0 2024-04-20T11:35:58.793 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:58.793+0000 7f826a600640 1 -- 172.21.15.152:0/3520118211 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f826415eb20 con 0x7f8264144fa0 2024-04-20T11:35:59.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:59.348+0000 7f825b400640 1 -- 172.21.15.152:0/3520118211 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v134) ==== 169+0+0 (secure 0 0 0) 0x7f825c07cb30 con 0x7f8264144fa0 2024-04-20T11:35:59.348 INFO:teuthology.orchestra.run.smithi152.stderr:pool 'cephfs_data' removed 2024-04-20T11:35:59.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:59.353+0000 7f826a600640 1 -- 172.21.15.152:0/3520118211 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f824006cf20 msgr2=0x7f824006f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:59.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:59.353+0000 7f826a600640 1 --2- 172.21.15.152:0/3520118211 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f824006cf20 0x7f824006f3e0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f8254007fd0 tx=0x7f8254007920 comp rx=0 tx=0).stop 2024-04-20T11:35:59.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:59.353+0000 7f826a600640 1 -- 172.21.15.152:0/3520118211 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8264144fa0 msgr2=0x7f8264141070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:59.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:59.353+0000 7f826a600640 1 --2- 172.21.15.152:0/3520118211 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8264144fa0 0x7f8264141070 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7f825c009e50 tx=0x7f825c004a40 comp rx=0 tx=0).stop 2024-04-20T11:35:59.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:59.354+0000 7f826a600640 1 -- 172.21.15.152:0/3520118211 shutdown_connections 2024-04-20T11:35:59.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:59.354+0000 7f826a600640 1 --2- 172.21.15.152:0/3520118211 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f824006cf20 0x7f824006f3e0 secure :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f8254007fd0 tx=0x7f8254007920 comp rx=0 tx=0).stop 2024-04-20T11:35:59.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:59.354+0000 7f826a600640 1 --2- 172.21.15.152:0/3520118211 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8264144fa0 0x7f8264141070 unknown :-1 s=CLOSED pgs=1914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:59.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:59.354+0000 7f826a600640 1 --2- 172.21.15.152:0/3520118211 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f82640b3470 0x7f826415fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:59.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:59.354+0000 7f826a600640 1 --2- 172.21.15.152:0/3520118211 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8264072530 0x7f82641f3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:59.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:59.354+0000 7f826a600640 1 -- 172.21.15.152:0/3520118211 >> 172.21.15.152:0/3520118211 conn(0x7f826406b980 msgr2=0x7f826406e050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:59.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:59.356+0000 7f826a600640 1 -- 172.21.15.152:0/3520118211 shutdown_connections 2024-04-20T11:35:59.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:59.357+0000 7f826a600640 1 -- 172.21.15.152:0/3520118211 wait complete. 2024-04-20T11:35:59.382 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-20T11:35:59.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:59.573+0000 7f4a72e00640 1 Processor -- start 2024-04-20T11:35:59.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:59.573+0000 7f4a72e00640 1 -- start start 2024-04-20T11:35:59.574 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:59.575+0000 7f4a72e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4a6c071cd0 0x7f4a6c0720b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:59.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:59.575+0000 7f4a72e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f4a6c160220 0x7f4a6c160680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:59.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:59.575+0000 7f4a72e00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a6c0725f0 con 0x7f4a6c06eb30 2024-04-20T11:35:59.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:59.575+0000 7f4a72e00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f4a6c13f8e0 con 0x7f4a6c160220 2024-04-20T11:35:59.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:59.575+0000 7f4a72e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4a6c160f90 con 0x7f4a6c071cd0 2024-04-20T11:35:59.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:59.576+0000 7f4a71a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4a6c071cd0 0x7f4a6c0720b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:59.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:59.576+0000 7f4a71a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4a6c071cd0 0x7f4a6c0720b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.152:44268/0 (socket says 172.21.15.152:44268) 2024-04-20T11:35:59.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:59.576+0000 7f4a71a00640 1 -- 172.21.15.152:0/594243025 learned_addr learned my addr 172.21.15.152:0/594243025 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:59.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:59.576+0000 7f4a71000640 1 --2- 172.21.15.152:0/594243025 >> v2:172.21.15.175:3300/0 conn(0x7f4a6c160220 0x7f4a6c160680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:59.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:59.577+0000 7f4a71a00640 1 -- 172.21.15.152:0/594243025 >> v2:172.21.15.175:3300/0 conn(0x7f4a6c160220 msgr2=0x7f4a6c160680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:35:59.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:59.577+0000 7f4a71a00640 1 --2- 172.21.15.152:0/594243025 >> v2:172.21.15.175:3300/0 conn(0x7f4a6c160220 0x7f4a6c160680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:59.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:59.577+0000 7f4a71a00640 1 -- 172.21.15.152:0/594243025 >> v1:172.21.15.175:6789/0 conn(0x7f4a6c06eb30 legacy=0x7f4a6c06efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:59.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:59.577+0000 7f4a71a00640 1 -- 172.21.15.152:0/594243025 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a6c06dd90 con 0x7f4a6c071cd0 2024-04-20T11:35:59.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:59.578+0000 7f4a71a00640 1 --2- 172.21.15.152:0/594243025 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4a6c071cd0 0x7f4a6c0720b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4a60009870 tx=0x7f4a6002b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b91b163e5e5cd9d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:59.580 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:59.579+0000 7f4a6be00640 1 -- 172.21.15.152:0/594243025 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a60035020 con 0x7f4a6c071cd0 2024-04-20T11:35:59.580 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:59.579+0000 7f4a6be00640 1 -- 172.21.15.152:0/594243025 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a60031640 con 0x7f4a6c071cd0 2024-04-20T11:35:59.580 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:59.579+0000 7f4a6be00640 1 -- 172.21.15.152:0/594243025 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a6002d040 con 0x7f4a6c071cd0 2024-04-20T11:35:59.580 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:59.579+0000 7f4a72e00640 1 -- 172.21.15.152:0/594243025 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4a6c071cd0 msgr2=0x7f4a6c0720b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:59.580 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:59.579+0000 7f4a72e00640 1 --2- 172.21.15.152:0/594243025 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4a6c071cd0 0x7f4a6c0720b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4a60009870 tx=0x7f4a6002b8f0 comp rx=0 tx=0).stop 2024-04-20T11:35:59.580 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:59.579+0000 7f4a72e00640 1 -- 172.21.15.152:0/594243025 shutdown_connections 2024-04-20T11:35:59.580 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:59.579+0000 7f4a72e00640 1 --2- 172.21.15.152:0/594243025 >> v2:172.21.15.175:3300/0 conn(0x7f4a6c160220 0x7f4a6c160680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:59.580 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:59.579+0000 7f4a72e00640 1 --2- 172.21.15.152:0/594243025 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4a6c071cd0 0x7f4a6c0720b0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:59.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:59.579+0000 7f4a72e00640 1 -- 172.21.15.152:0/594243025 >> 172.21.15.152:0/594243025 conn(0x7f4a6c067ae0 msgr2=0x7f4a6c06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:59.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:59.579+0000 7f4a72e00640 1 -- 172.21.15.152:0/594243025 shutdown_connections 2024-04-20T11:35:59.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:59.580+0000 7f4a72e00640 1 -- 172.21.15.152:0/594243025 wait complete. 2024-04-20T11:35:59.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:59.581+0000 7f4a72e00640 1 Processor -- start 2024-04-20T11:35:59.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:59.581+0000 7f4a72e00640 1 -- start start 2024-04-20T11:35:59.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:59.582+0000 7f4a72e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4a6c06eb30 0x7f4a6c15b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:59.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:59.582+0000 7f4a72e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4a6c071cd0 0x7f4a6c132410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:59.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:59.582+0000 7f4a72e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4a6c160220 0x7f4a6c152250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:59.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:59.582+0000 7f4a72e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f4a6c06e630 con 0x7f4a6c160220 2024-04-20T11:35:59.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:59.582+0000 7f4a72e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4a6c06e350 con 0x7f4a6c071cd0 2024-04-20T11:35:59.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:59.582+0000 7f4a72e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4a6c13fb90 con 0x7f4a6c06eb30 2024-04-20T11:35:59.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:59.583+0000 7f4a72400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4a6c160220 0x7f4a6c152250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:59.585 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:59.583+0000 7f4a72400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4a6c160220 0x7f4a6c152250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36308/0 (socket says 172.21.15.152:36308) 2024-04-20T11:35:59.585 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:59.583+0000 7f4a72400640 1 -- 172.21.15.152:0/1313553182 learned_addr learned my addr 172.21.15.152:0/1313553182 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:35:59.585 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:59.583+0000 7f4a71a00640 1 --2- 172.21.15.152:0/1313553182 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4a6c06eb30 0x7f4a6c15b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:59.585 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:59.583+0000 7f4a71000640 1 --2- 172.21.15.152:0/1313553182 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4a6c071cd0 0x7f4a6c132410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:59.585 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:59.583+0000 7f4a72400640 1 -- 172.21.15.152:0/1313553182 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4a6c06eb30 msgr2=0x7f4a6c15b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:59.585 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:59.583+0000 7f4a72400640 1 --2- 172.21.15.152:0/1313553182 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4a6c06eb30 0x7f4a6c15b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:59.585 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:59.583+0000 7f4a72400640 1 -- 172.21.15.152:0/1313553182 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4a6c071cd0 msgr2=0x7f4a6c132410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:59.585 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:59.583+0000 7f4a72400640 1 --2- 172.21.15.152:0/1313553182 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4a6c071cd0 0x7f4a6c132410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:59.585 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:59.583+0000 7f4a72400640 1 -- 172.21.15.152:0/1313553182 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a6c151490 con 0x7f4a6c160220 2024-04-20T11:35:59.585 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:59.584+0000 7f4a72400640 1 --2- 172.21.15.152:0/1313553182 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4a6c160220 0x7f4a6c152250 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7f4a6400c710 tx=0x7f4a6400cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:59.585 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:59.584+0000 7f4a69600640 1 -- 172.21.15.152:0/1313553182 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a64016020 con 0x7f4a6c160220 2024-04-20T11:35:59.585 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:59.584+0000 7f4a69600640 1 -- 172.21.15.152:0/1313553182 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a64006d50 con 0x7f4a6c160220 2024-04-20T11:35:59.586 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:59.584+0000 7f4a69600640 1 -- 172.21.15.152:0/1313553182 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a64003bb0 con 0x7f4a6c160220 2024-04-20T11:35:59.586 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:59.584+0000 7f4a72e00640 1 -- 172.21.15.152:0/1313553182 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a6c151770 con 0x7f4a6c160220 2024-04-20T11:35:59.586 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:59.584+0000 7f4a72e00640 1 -- 172.21.15.152:0/1313553182 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a6c151d30 con 0x7f4a6c160220 2024-04-20T11:35:59.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:59.585+0000 7f4a72e00640 1 -- 172.21.15.152:0/1313553182 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a6c075e60 con 0x7f4a6c160220 2024-04-20T11:35:59.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:59.586+0000 7f4a69600640 1 -- 172.21.15.152:0/1313553182 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 22) ==== 91601+0+0 (secure 0 0 0) 0x7f4a6400a850 con 0x7f4a6c160220 2024-04-20T11:35:59.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:59.587+0000 7f4a69600640 1 --2- 172.21.15.152:0/1313553182 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4a4c06cf20 0x7f4a4c06f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:59.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:59.587+0000 7f4a69600640 1 -- 172.21.15.152:0/1313553182 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 4943+0+0 (secure 0 0 0) 0x7f4a6408c090 con 0x7f4a6c160220 2024-04-20T11:35:59.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:59.588+0000 7f4a71a00640 1 --2- 172.21.15.152:0/1313553182 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4a4c06cf20 0x7f4a4c06f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:59.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:59.589+0000 7f4a71a00640 1 --2- 172.21.15.152:0/1313553182 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4a4c06cf20 0x7f4a4c06f3e0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f4a60036000 tx=0x7f4a60008030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:59.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:59.592+0000 7f4a69600640 1 -- 172.21.15.152:0/1313553182 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f4a64020020 con 0x7f4a6c160220 2024-04-20T11:35:59.789 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:59.790+0000 7f4a72e00640 1 -- 172.21.15.152:0/1313553182 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f4a6c151f00 con 0x7f4a6c160220 2024-04-20T11:35:59.795 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:59.796+0000 7f4a69600640 1 -- 172.21.15.152:0/1313553182 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v25) ==== 104+0+0 (secure 0 0 0) 0x7f4a64019310 con 0x7f4a6c160220 2024-04-20T11:35:59.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:59.799+0000 7f4a72e00640 1 -- 172.21.15.152:0/1313553182 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4a4c06cf20 msgr2=0x7f4a4c06f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:59.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:59.799+0000 7f4a72e00640 1 --2- 172.21.15.152:0/1313553182 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4a4c06cf20 0x7f4a4c06f3e0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f4a60036000 tx=0x7f4a60008030 comp rx=0 tx=0).stop 2024-04-20T11:35:59.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:59.799+0000 7f4a72e00640 1 -- 172.21.15.152:0/1313553182 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4a6c160220 msgr2=0x7f4a6c152250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:59.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:59.799+0000 7f4a72e00640 1 --2- 172.21.15.152:0/1313553182 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4a6c160220 0x7f4a6c152250 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7f4a6400c710 tx=0x7f4a6400cbd0 comp rx=0 tx=0).stop 2024-04-20T11:35:59.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:59.799+0000 7f4a72e00640 1 -- 172.21.15.152:0/1313553182 shutdown_connections 2024-04-20T11:35:59.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:59.799+0000 7f4a72e00640 1 --2- 172.21.15.152:0/1313553182 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4a4c06cf20 0x7f4a4c06f3e0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:59.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:59.799+0000 7f4a72e00640 1 --2- 172.21.15.152:0/1313553182 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4a6c160220 0x7f4a6c152250 unknown :-1 s=CLOSED pgs=1915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:59.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:59.799+0000 7f4a72e00640 1 --2- 172.21.15.152:0/1313553182 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4a6c071cd0 0x7f4a6c132410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:59.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:59.799+0000 7f4a72e00640 1 --2- 172.21.15.152:0/1313553182 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4a6c06eb30 0x7f4a6c15b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:59.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:59.799+0000 7f4a72e00640 1 -- 172.21.15.152:0/1313553182 >> 172.21.15.152:0/1313553182 conn(0x7f4a6c067ae0 msgr2=0x7f4a6c06efc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:59.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:59.799+0000 7f4a72e00640 1 -- 172.21.15.152:0/1313553182 shutdown_connections 2024-04-20T11:35:59.800 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:35:59.799+0000 7f4a72e00640 1 -- 172.21.15.152:0/1313553182 wait complete. 2024-04-20T11:35:59.820 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_ops_on_nonexistent_vol' 2024-04-20T11:36:00.018 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.018+0000 7f7acf200640 1 Processor -- start 2024-04-20T11:36:00.018 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.019+0000 7f7acf200640 1 -- start start 2024-04-20T11:36:00.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.020+0000 7f7acf200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f7ad0071cd0 0x7f7ad00720b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:00.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.020+0000 7f7acf200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f7ad0162e50 0x7f7ad0163270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:00.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.020+0000 7f7acf200640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ad00725f0 con 0x7f7ad006ea20 2024-04-20T11:36:00.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.020+0000 7f7acf200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f7ad00ceec0 con 0x7f7ad0071cd0 2024-04-20T11:36:00.021 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.020+0000 7f7acf200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f7ad006dcf0 con 0x7f7ad0162e50 2024-04-20T11:36:00.021 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.020+0000 7f7acde00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f7ad0071cd0 0x7f7ad00720b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:00.021 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.021+0000 7f7ac5400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f7ad006ea20 0x7f7ad00b4a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:50024/0 (socket says 172.21.15.152:50024) 2024-04-20T11:36:00.021 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.021+0000 7f7ac5400640 1 -- 172.21.15.152:0/1207828238 learned_addr learned my addr 172.21.15.152:0/1207828238 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:36:00.021 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.021+0000 7f7acde00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f7ad0071cd0 0x7f7ad00720b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36314/0 (socket says 172.21.15.152:36314) 2024-04-20T11:36:00.021 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.021+0000 7f7ace800640 1 --2- 172.21.15.152:0/1207828238 >> v2:172.21.15.175:3300/0 conn(0x7f7ad0162e50 0x7f7ad0163270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:00.022 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.021+0000 7f7acde00640 1 -- 172.21.15.152:0/1207828238 >> v2:172.21.15.175:3300/0 conn(0x7f7ad0162e50 msgr2=0x7f7ad0163270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:00.022 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.021+0000 7f7acde00640 1 --2- 172.21.15.152:0/1207828238 >> v2:172.21.15.175:3300/0 conn(0x7f7ad0162e50 0x7f7ad0163270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:00.022 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.021+0000 7f7acde00640 1 -- 172.21.15.152:0/1207828238 >> v1:172.21.15.152:6789/0 conn(0x7f7ad006ea20 legacy=0x7f7ad00b4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:00.022 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.021+0000 7f7acde00640 1 -- 172.21.15.152:0/1207828238 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ad00b5510 con 0x7f7ad0071cd0 2024-04-20T11:36:00.022 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.021+0000 7f7acd400640 1 -- 172.21.15.152:0/1207828238 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 261461201 0 0) 0x7f7ad00725f0 con 0x7f7ad006ea20 2024-04-20T11:36:00.022 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.023+0000 7f7acde00640 1 --2- 172.21.15.152:0/1207828238 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7ad0071cd0 0x7f7ad00720b0 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7f7ad006c9a0 tx=0x7f7ac002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fef2cdbe717bcc37 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:00.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.023+0000 7f7acd400640 1 -- 172.21.15.152:0/1207828238 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ac0035020 con 0x7f7ad0071cd0 2024-04-20T11:36:00.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.023+0000 7f7acd400640 1 -- 172.21.15.152:0/1207828238 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ac0031830 con 0x7f7ad0071cd0 2024-04-20T11:36:00.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.023+0000 7f7acd400640 1 -- 172.21.15.152:0/1207828238 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ac0031c00 con 0x7f7ad0071cd0 2024-04-20T11:36:00.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.023+0000 7f7acf200640 1 -- 172.21.15.152:0/1207828238 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7ad0071cd0 msgr2=0x7f7ad00720b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:00.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.023+0000 7f7acf200640 1 --2- 172.21.15.152:0/1207828238 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7ad0071cd0 0x7f7ad00720b0 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7f7ad006c9a0 tx=0x7f7ac002b8d0 comp rx=0 tx=0).stop 2024-04-20T11:36:00.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.024+0000 7f7acf200640 1 -- 172.21.15.152:0/1207828238 shutdown_connections 2024-04-20T11:36:00.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.024+0000 7f7acf200640 1 --2- 172.21.15.152:0/1207828238 >> v2:172.21.15.175:3300/0 conn(0x7f7ad0162e50 0x7f7ad0163270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:00.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.024+0000 7f7acf200640 1 --2- 172.21.15.152:0/1207828238 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7ad0071cd0 0x7f7ad00720b0 unknown :-1 s=CLOSED pgs=1917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:00.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.024+0000 7f7acf200640 1 -- 172.21.15.152:0/1207828238 >> 172.21.15.152:0/1207828238 conn(0x7f7ad0067ae0 msgr2=0x7f7ad006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:00.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.024+0000 7f7acf200640 1 -- 172.21.15.152:0/1207828238 shutdown_connections 2024-04-20T11:36:00.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.024+0000 7f7acf200640 1 -- 172.21.15.152:0/1207828238 wait complete. 2024-04-20T11:36:00.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.025+0000 7f7acf200640 1 Processor -- start 2024-04-20T11:36:00.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.026+0000 7f7acf200640 1 -- start start 2024-04-20T11:36:00.026 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.026+0000 7f7acf200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7ad006ea20 0x7f7ad0146f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:00.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.026+0000 7f7acf200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7ad0071cd0 0x7f7ad0165630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:00.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.026+0000 7f7acf200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ad0162e50 0x7f7ad015d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:00.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.026+0000 7f7acf200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f7ad006e500 con 0x7f7ad0071cd0 2024-04-20T11:36:00.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.026+0000 7f7acf200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7ad006e220 con 0x7f7ad0162e50 2024-04-20T11:36:00.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.026+0000 7f7acf200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7ad00b4180 con 0x7f7ad006ea20 2024-04-20T11:36:00.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.027+0000 7f7ac5400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7ad0071cd0 0x7f7ad0165630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:00.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.027+0000 7f7ac5400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7ad0071cd0 0x7f7ad0165630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36326/0 (socket says 172.21.15.152:36326) 2024-04-20T11:36:00.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.027+0000 7f7ac5400640 1 -- 172.21.15.152:0/3427747787 learned_addr learned my addr 172.21.15.152:0/3427747787 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:00.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.027+0000 7f7ac5400640 1 -- 172.21.15.152:0/3427747787 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7ad006ea20 msgr2=0x7f7ad0146f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:00.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.027+0000 7f7ac5400640 1 --2- 172.21.15.152:0/3427747787 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7ad006ea20 0x7f7ad0146f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:00.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.027+0000 7f7ac5400640 1 -- 172.21.15.152:0/3427747787 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ad0162e50 msgr2=0x7f7ad015d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:00.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.027+0000 7f7ac5400640 1 --2- 172.21.15.152:0/3427747787 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ad0162e50 0x7f7ad015d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:00.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.027+0000 7f7ac5400640 1 -- 172.21.15.152:0/3427747787 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ad0165b70 con 0x7f7ad0071cd0 2024-04-20T11:36:00.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.027+0000 7f7ac5400640 1 --2- 172.21.15.152:0/3427747787 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7ad0071cd0 0x7f7ad0165630 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7f7ab800c4d0 tx=0x7f7ab800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:00.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.028+0000 7f7ac6a00640 1 -- 172.21.15.152:0/3427747787 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab8018070 con 0x7f7ad0071cd0 2024-04-20T11:36:00.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.028+0000 7f7ac6a00640 1 -- 172.21.15.152:0/3427747787 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ab8007500 con 0x7f7ad0071cd0 2024-04-20T11:36:00.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.028+0000 7f7acf200640 1 -- 172.21.15.152:0/3427747787 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ad015d9e0 con 0x7f7ad0071cd0 2024-04-20T11:36:00.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.028+0000 7f7ac6a00640 1 -- 172.21.15.152:0/3427747787 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab8004a20 con 0x7f7ad0071cd0 2024-04-20T11:36:00.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.028+0000 7f7acf200640 1 -- 172.21.15.152:0/3427747787 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ad0164870 con 0x7f7ad0071cd0 2024-04-20T11:36:00.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.029+0000 7f7acf200640 1 -- 172.21.15.152:0/3427747787 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ad0075e60 con 0x7f7ad0071cd0 2024-04-20T11:36:00.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.030+0000 7f7ac6a00640 1 -- 172.21.15.152:0/3427747787 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 22) ==== 91601+0+0 (secure 0 0 0) 0x7f7ab801b070 con 0x7f7ad0071cd0 2024-04-20T11:36:00.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.031+0000 7f7ac6a00640 1 --2- 172.21.15.152:0/3427747787 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f7aac06cf20 0x7f7aac06f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:00.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.031+0000 7f7ac6a00640 1 -- 172.21.15.152:0/3427747787 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 4943+0+0 (secure 0 0 0) 0x7f7ab808b6f0 con 0x7f7ad0071cd0 2024-04-20T11:36:00.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.032+0000 7f7acde00640 1 --2- 172.21.15.152:0/3427747787 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f7aac06cf20 0x7f7aac06f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:00.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.033+0000 7f7acde00640 1 --2- 172.21.15.152:0/3427747787 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f7aac06cf20 0x7f7aac06f3e0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f7ac0036000 tx=0x7f7ac0030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:00.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.035+0000 7f7ac6a00640 1 -- 172.21.15.152:0/3427747787 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f7ab80594e0 con 0x7f7ad0071cd0 2024-04-20T11:36:00.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.311+0000 7f7acf200640 1 -- 172.21.15.152:0/3427747787 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_ops_on_nonexistent_vol"]} v 0) -- 0x7f7ad00b5170 con 0x7f7ad0071cd0 2024-04-20T11:36:00.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.363+0000 7f7ac6a00640 1 -- 172.21.15.152:0/3427747787 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_ops_on_nonexistent_vol"]}]=0 v482) ==== 151+0+0 (secure 0 0 0) 0x7f7ab8058de0 con 0x7f7ad0071cd0 2024-04-20T11:36:00.366 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.366+0000 7f7acf200640 1 -- 172.21.15.152:0/3427747787 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f7aac06cf20 msgr2=0x7f7aac06f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:00.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.366+0000 7f7acf200640 1 --2- 172.21.15.152:0/3427747787 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f7aac06cf20 0x7f7aac06f3e0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f7ac0036000 tx=0x7f7ac0030000 comp rx=0 tx=0).stop 2024-04-20T11:36:00.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.366+0000 7f7acf200640 1 -- 172.21.15.152:0/3427747787 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7ad0071cd0 msgr2=0x7f7ad0165630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:00.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.366+0000 7f7acf200640 1 --2- 172.21.15.152:0/3427747787 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7ad0071cd0 0x7f7ad0165630 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7f7ab800c4d0 tx=0x7f7ab800c990 comp rx=0 tx=0).stop 2024-04-20T11:36:00.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.366+0000 7f7acf200640 1 -- 172.21.15.152:0/3427747787 shutdown_connections 2024-04-20T11:36:00.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.367+0000 7f7acf200640 1 --2- 172.21.15.152:0/3427747787 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f7aac06cf20 0x7f7aac06f3e0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:00.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.367+0000 7f7acf200640 1 --2- 172.21.15.152:0/3427747787 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ad0162e50 0x7f7ad015d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:00.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.367+0000 7f7acf200640 1 --2- 172.21.15.152:0/3427747787 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7ad0071cd0 0x7f7ad0165630 unknown :-1 s=CLOSED pgs=1918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:00.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.367+0000 7f7acf200640 1 --2- 172.21.15.152:0/3427747787 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7ad006ea20 0x7f7ad0146f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:00.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.367+0000 7f7acf200640 1 -- 172.21.15.152:0/3427747787 >> 172.21.15.152:0/3427747787 conn(0x7f7ad0067ae0 msgr2=0x7f7ad006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:00.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.367+0000 7f7acf200640 1 -- 172.21.15.152:0/3427747787 shutdown_connections 2024-04-20T11:36:00.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.367+0000 7f7acf200640 1 -- 172.21.15.152:0/3427747787 wait complete. 2024-04-20T11:36:00.387 INFO:tasks.cephfs_test_runner:test_subvolume_ops_on_nonexistent_vol (tasks.cephfs.test_volumes.TestMisc) ... ok 2024-04-20T11:36:00.387 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_upgrade_legacy_to_v1 (tasks.cephfs.test_volumes.TestMisc) poor man's upgrade test -- rather than going through a full upgrade cycle, 2024-04-20T11:36:00.387 INFO:tasks.cephfs_test_runner:test_subvolume_upgrade_legacy_to_v1 (tasks.cephfs.test_volumes.TestMisc) 2024-04-20T11:36:00.388 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:36:00.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.589+0000 7f9d17000640 1 Processor -- start 2024-04-20T11:36:00.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.590+0000 7f9d17000640 1 -- start start 2024-04-20T11:36:00.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.591+0000 7f9d17000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d10160870 0x7f9d10163c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:00.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.591+0000 7f9d17000640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f9d100b3d80 0x7f9d100b41e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:00.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.591+0000 7f9d17000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f9d10071db0 0x7f9d100721d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:00.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.591+0000 7f9d17000640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f9d100b4720 con 0x7f9d10071db0 2024-04-20T11:36:00.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.591+0000 7f9d17000640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f9d10160c50 con 0x7f9d100b3d80 2024-04-20T11:36:00.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.591+0000 7f9d17000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f9d101641b0 con 0x7f9d10160870 2024-04-20T11:36:00.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.591+0000 7f9d15000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f9d10071db0 0x7f9d100721d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:00.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.591+0000 7f9d15000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f9d10071db0 0x7f9d100721d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36342/0 (socket says 172.21.15.152:36342) 2024-04-20T11:36:00.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.591+0000 7f9d15000640 1 -- 172.21.15.152:0/816834878 learned_addr learned my addr 172.21.15.152:0/816834878 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:00.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.592+0000 7f9d0fe00640 1 --2- 172.21.15.152:0/816834878 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d10160870 0x7f9d10163c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:00.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.592+0000 7f9d07400640 1 --2- 172.21.15.152:0/816834878 >> v2:172.21.15.175:3300/0 conn(0x7f9d100b3d80 0x7f9d100b41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:00.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.592+0000 7f9d15000640 1 -- 172.21.15.152:0/816834878 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d10160870 msgr2=0x7f9d10163c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:00.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.592+0000 7f9d15000640 1 --2- 172.21.15.152:0/816834878 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d10160870 0x7f9d10163c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:00.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.592+0000 7f9d15000640 1 -- 172.21.15.152:0/816834878 >> v2:172.21.15.175:3300/0 conn(0x7f9d100b3d80 msgr2=0x7f9d100b41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:00.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.592+0000 7f9d15000640 1 --2- 172.21.15.152:0/816834878 >> v2:172.21.15.175:3300/0 conn(0x7f9d100b3d80 0x7f9d100b41e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:00.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.592+0000 7f9d15000640 1 -- 172.21.15.152:0/816834878 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d1006e140 con 0x7f9d10071db0 2024-04-20T11:36:00.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.593+0000 7f9d15000640 1 --2- 172.21.15.152:0/816834878 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d10071db0 0x7f9d100721d0 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7f9d1006ffb0 tx=0x7f9d0802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f989a39190917b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:00.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.593+0000 7f9d0f400640 1 -- 172.21.15.152:0/816834878 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d0802ca80 con 0x7f9d10071db0 2024-04-20T11:36:00.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.593+0000 7f9d0f400640 1 -- 172.21.15.152:0/816834878 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d0802cc20 con 0x7f9d10071db0 2024-04-20T11:36:00.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.593+0000 7f9d0f400640 1 -- 172.21.15.152:0/816834878 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d080343f0 con 0x7f9d10071db0 2024-04-20T11:36:00.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.594+0000 7f9d17000640 1 -- 172.21.15.152:0/816834878 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d10071db0 msgr2=0x7f9d100721d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:00.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.594+0000 7f9d17000640 1 --2- 172.21.15.152:0/816834878 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d10071db0 0x7f9d100721d0 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7f9d1006ffb0 tx=0x7f9d0802bee0 comp rx=0 tx=0).stop 2024-04-20T11:36:00.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.594+0000 7f9d17000640 1 -- 172.21.15.152:0/816834878 shutdown_connections 2024-04-20T11:36:00.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.594+0000 7f9d17000640 1 --2- 172.21.15.152:0/816834878 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d10071db0 0x7f9d100721d0 unknown :-1 s=CLOSED pgs=1919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:00.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.594+0000 7f9d17000640 1 --2- 172.21.15.152:0/816834878 >> v2:172.21.15.175:3300/0 conn(0x7f9d100b3d80 0x7f9d100b41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:00.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.594+0000 7f9d17000640 1 --2- 172.21.15.152:0/816834878 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d10160870 0x7f9d10163c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:00.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.594+0000 7f9d17000640 1 -- 172.21.15.152:0/816834878 >> 172.21.15.152:0/816834878 conn(0x7f9d10067cf0 msgr2=0x7f9d1006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:00.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.594+0000 7f9d17000640 1 -- 172.21.15.152:0/816834878 shutdown_connections 2024-04-20T11:36:00.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.594+0000 7f9d17000640 1 -- 172.21.15.152:0/816834878 wait complete. 2024-04-20T11:36:00.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.596+0000 7f9d17000640 1 Processor -- start 2024-04-20T11:36:00.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.596+0000 7f9d17000640 1 -- start start 2024-04-20T11:36:00.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.597+0000 7f9d17000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d10071db0 0x7f9d1014efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:00.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.597+0000 7f9d17000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d100b3d80 0x7f9d101623c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:00.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.597+0000 7f9d17000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d10160870 0x7f9d100ac5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:00.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.597+0000 7f9d17000640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f9d1006e9e0 con 0x7f9d100b3d80 2024-04-20T11:36:00.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.597+0000 7f9d17000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9d1006e700 con 0x7f9d10071db0 2024-04-20T11:36:00.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.597+0000 7f9d17000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f9d1006ecc0 con 0x7f9d10160870 2024-04-20T11:36:00.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.597+0000 7f9d07400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d100b3d80 0x7f9d101623c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:00.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.597+0000 7f9d07400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d100b3d80 0x7f9d101623c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36348/0 (socket says 172.21.15.152:36348) 2024-04-20T11:36:00.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.597+0000 7f9d07400640 1 -- 172.21.15.152:0/1942430512 learned_addr learned my addr 172.21.15.152:0/1942430512 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:00.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.597+0000 7f9d15000640 1 --2- 172.21.15.152:0/1942430512 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d10160870 0x7f9d100ac5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:00.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.597+0000 7f9d0fe00640 1 --2- 172.21.15.152:0/1942430512 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d10071db0 0x7f9d1014efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:00.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.597+0000 7f9d07400640 1 -- 172.21.15.152:0/1942430512 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d10160870 msgr2=0x7f9d100ac5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:00.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.597+0000 7f9d07400640 1 --2- 172.21.15.152:0/1942430512 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d10160870 0x7f9d100ac5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:00.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.597+0000 7f9d07400640 1 -- 172.21.15.152:0/1942430512 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d10071db0 msgr2=0x7f9d1014efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:00.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.597+0000 7f9d07400640 1 --2- 172.21.15.152:0/1942430512 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d10071db0 0x7f9d1014efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:00.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.597+0000 7f9d07400640 1 -- 172.21.15.152:0/1942430512 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d10151b60 con 0x7f9d100b3d80 2024-04-20T11:36:00.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.598+0000 7f9d15000640 1 --2- 172.21.15.152:0/1942430512 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d10160870 0x7f9d100ac5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:36:00.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.598+0000 7f9d07400640 1 --2- 172.21.15.152:0/1942430512 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d100b3d80 0x7f9d101623c0 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7f9cf800d800 tx=0x7f9cf800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:00.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.598+0000 7f9d0cc00640 1 -- 172.21.15.152:0/1942430512 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cf800b6c0 con 0x7f9d100b3d80 2024-04-20T11:36:00.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.598+0000 7f9d0cc00640 1 -- 172.21.15.152:0/1942430512 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cf8019070 con 0x7f9d100b3d80 2024-04-20T11:36:00.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.598+0000 7f9d0cc00640 1 -- 172.21.15.152:0/1942430512 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cf800ba80 con 0x7f9d100b3d80 2024-04-20T11:36:00.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.598+0000 7f9d17000640 1 -- 172.21.15.152:0/1942430512 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d10151e40 con 0x7f9d100b3d80 2024-04-20T11:36:00.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.598+0000 7f9d17000640 1 -- 172.21.15.152:0/1942430512 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d10152400 con 0x7f9d100b3d80 2024-04-20T11:36:00.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.599+0000 7f9d17000640 1 -- 172.21.15.152:0/1942430512 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d100689f0 con 0x7f9d100b3d80 2024-04-20T11:36:00.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.601+0000 7f9d0cc00640 1 -- 172.21.15.152:0/1942430512 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 22) ==== 91601+0+0 (secure 0 0 0) 0x7f9cf800bc20 con 0x7f9d100b3d80 2024-04-20T11:36:00.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.602+0000 7f9d0cc00640 1 --2- 172.21.15.152:0/1942430512 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f9cec06cf20 0x7f9cec06f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:00.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.602+0000 7f9d0cc00640 1 -- 172.21.15.152:0/1942430512 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 4943+0+0 (secure 0 0 0) 0x7f9cf808c880 con 0x7f9d100b3d80 2024-04-20T11:36:00.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.602+0000 7f9d0fe00640 1 --2- 172.21.15.152:0/1942430512 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f9cec06cf20 0x7f9cec06f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:00.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.604+0000 7f9d0fe00640 1 --2- 172.21.15.152:0/1942430512 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f9cec06cf20 0x7f9cec06f3e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f9d00006fd0 tx=0x7f9d00008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:00.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.606+0000 7f9d0cc00640 1 -- 172.21.15.152:0/1942430512 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f9cf805a6c0 con 0x7f9d100b3d80 2024-04-20T11:36:00.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.813+0000 7f9d17000640 1 -- 172.21.15.152:0/1942430512 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9d1014f7a0 con 0x7f9d100b3d80 2024-04-20T11:36:00.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.815+0000 7f9d0cc00640 1 -- 172.21.15.152:0/1942430512 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v134) ==== 74+0+13092 (secure 0 0 0) 0x7f9cf8059fc0 con 0x7f9d100b3d80 2024-04-20T11:36:00.814 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:36:00.815 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":134,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:35:59.345053+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.152:6835/1884827433":"2024-04-21T11:35:55.873081+0000","172.21.15.152:6833/1884827433":"2024-04-21T11:35:55.873081+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:36:00.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.817+0000 7f9d17000640 1 -- 172.21.15.152:0/1942430512 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f9cec06cf20 msgr2=0x7f9cec06f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:00.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.817+0000 7f9d17000640 1 --2- 172.21.15.152:0/1942430512 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f9cec06cf20 0x7f9cec06f3e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f9d00006fd0 tx=0x7f9d00008040 comp rx=0 tx=0).stop 2024-04-20T11:36:00.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.817+0000 7f9d17000640 1 -- 172.21.15.152:0/1942430512 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d100b3d80 msgr2=0x7f9d101623c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:00.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.817+0000 7f9d17000640 1 --2- 172.21.15.152:0/1942430512 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d100b3d80 0x7f9d101623c0 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7f9cf800d800 tx=0x7f9cf800dcc0 comp rx=0 tx=0).stop 2024-04-20T11:36:00.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.817+0000 7f9d17000640 1 -- 172.21.15.152:0/1942430512 shutdown_connections 2024-04-20T11:36:00.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.817+0000 7f9d17000640 1 --2- 172.21.15.152:0/1942430512 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f9cec06cf20 0x7f9cec06f3e0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:00.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.817+0000 7f9d17000640 1 --2- 172.21.15.152:0/1942430512 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d10160870 0x7f9d100ac5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:00.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.817+0000 7f9d17000640 1 --2- 172.21.15.152:0/1942430512 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d100b3d80 0x7f9d101623c0 unknown :-1 s=CLOSED pgs=1920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:00.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.817+0000 7f9d17000640 1 --2- 172.21.15.152:0/1942430512 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d10071db0 0x7f9d1014efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:00.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.817+0000 7f9d17000640 1 -- 172.21.15.152:0/1942430512 >> 172.21.15.152:0/1942430512 conn(0x7f9d10067cf0 msgr2=0x7f9d100affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:00.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.818+0000 7f9d17000640 1 -- 172.21.15.152:0/1942430512 shutdown_connections 2024-04-20T11:36:00.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:00.818+0000 7f9d17000640 1 -- 172.21.15.152:0/1942430512 wait complete. 2024-04-20T11:36:00.836 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T11:26:47.246571+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T11:36:00.837 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T11:36:01.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.046+0000 7f4dc4400640 1 Processor -- start 2024-04-20T11:36:01.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.046+0000 7f4dc4400640 1 -- start start 2024-04-20T11:36:01.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.048+0000 7f4dc4400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4dbc15b510 0x7f4dbc15fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:01.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.048+0000 7f4dc4400640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f4dbc0b2880 0x7f4dbc0b3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:01.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.048+0000 7f4dc4400640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4dbc15f1c0 con 0x7f4dbc0cfc40 2024-04-20T11:36:01.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.048+0000 7f4dc4400640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f4dbc06f420 con 0x7f4dbc0b2880 2024-04-20T11:36:01.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.048+0000 7f4dc4400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4dbc0b4430 con 0x7f4dbc15b510 2024-04-20T11:36:01.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.048+0000 7f4dc1000640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7f4dbc0cfc40 0x7f4dbc0b2170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.152:36008/0 (socket says 172.21.15.152:36008) 2024-04-20T11:36:01.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.048+0000 7f4dc1000640 1 -- 172.21.15.152:0/643861942 learned_addr learned my addr 172.21.15.152:0/643861942 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:36:01.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.048+0000 7f4dc1a00640 1 --2- 172.21.15.152:0/643861942 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4dbc15b510 0x7f4dbc15fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:01.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.049+0000 7f4dc2400640 1 --2- 172.21.15.152:0/643861942 >> v2:172.21.15.175:3300/0 conn(0x7f4dbc0b2880 0x7f4dbc0b3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:01.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.049+0000 7f4dc1a00640 1 -- 172.21.15.152:0/643861942 >> v2:172.21.15.175:3300/0 conn(0x7f4dbc0b2880 msgr2=0x7f4dbc0b3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:01.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.049+0000 7f4dc1a00640 1 --2- 172.21.15.152:0/643861942 >> v2:172.21.15.175:3300/0 conn(0x7f4dbc0b2880 0x7f4dbc0b3ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:01.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.049+0000 7f4dc1a00640 1 -- 172.21.15.152:0/643861942 >> v1:172.21.15.175:6789/0 conn(0x7f4dbc0cfc40 legacy=0x7f4dbc0b2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:01.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.049+0000 7f4dc1a00640 1 -- 172.21.15.152:0/643861942 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dbc06f670 con 0x7f4dbc15b510 2024-04-20T11:36:01.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.049+0000 7f4dbbe00640 1 -- 172.21.15.152:0/643861942 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2798933862 0 0) 0x7f4dbc15f1c0 con 0x7f4dbc0cfc40 2024-04-20T11:36:01.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.050+0000 7f4dc1a00640 1 --2- 172.21.15.152:0/643861942 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4dbc15b510 0x7f4dbc15fc30 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4dac0097a0 tx=0x7f4dac02b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=969aa134a9331e59 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:01.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.051+0000 7f4dbbe00640 1 -- 172.21.15.152:0/643861942 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dac0315a0 con 0x7f4dbc15b510 2024-04-20T11:36:01.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.051+0000 7f4dbbe00640 1 -- 172.21.15.152:0/643861942 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4dac037070 con 0x7f4dbc15b510 2024-04-20T11:36:01.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.051+0000 7f4dbbe00640 1 -- 172.21.15.152:0/643861942 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dac031960 con 0x7f4dbc15b510 2024-04-20T11:36:01.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.051+0000 7f4dc4400640 1 -- 172.21.15.152:0/643861942 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4dbc15b510 msgr2=0x7f4dbc15fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:01.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.051+0000 7f4dc4400640 1 --2- 172.21.15.152:0/643861942 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4dbc15b510 0x7f4dbc15fc30 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4dac0097a0 tx=0x7f4dac02b840 comp rx=0 tx=0).stop 2024-04-20T11:36:01.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.052+0000 7f4dc4400640 1 -- 172.21.15.152:0/643861942 shutdown_connections 2024-04-20T11:36:01.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.052+0000 7f4dc4400640 1 --2- 172.21.15.152:0/643861942 >> v2:172.21.15.175:3300/0 conn(0x7f4dbc0b2880 0x7f4dbc0b3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:01.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.052+0000 7f4dc4400640 1 --2- 172.21.15.152:0/643861942 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4dbc15b510 0x7f4dbc15fc30 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:01.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.052+0000 7f4dc4400640 1 -- 172.21.15.152:0/643861942 >> 172.21.15.152:0/643861942 conn(0x7f4dbc06b980 msgr2=0x7f4dbc06bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:01.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.052+0000 7f4dc4400640 1 -- 172.21.15.152:0/643861942 shutdown_connections 2024-04-20T11:36:01.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.052+0000 7f4dc4400640 1 -- 172.21.15.152:0/643861942 wait complete. 2024-04-20T11:36:01.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.053+0000 7f4dc4400640 1 Processor -- start 2024-04-20T11:36:01.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.054+0000 7f4dc4400640 1 -- start start 2024-04-20T11:36:01.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.054+0000 7f4dc4400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4dbc0b2880 0x7f4dbc15be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:01.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.054+0000 7f4dc4400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4dbc0cfc40 0x7f4dbc1f4950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:01.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.054+0000 7f4dc4400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4dbc15b510 0x7f4dbc1493c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:01.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.054+0000 7f4dc4400640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f4dbc15b8f0 con 0x7f4dbc0cfc40 2024-04-20T11:36:01.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.054+0000 7f4dc4400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4dbc15f710 con 0x7f4dbc15b510 2024-04-20T11:36:01.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.054+0000 7f4dc4400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4dbc06f950 con 0x7f4dbc0b2880 2024-04-20T11:36:01.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.055+0000 7f4dc1000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4dbc0cfc40 0x7f4dbc1f4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:01.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.055+0000 7f4dc1000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4dbc0cfc40 0x7f4dbc1f4950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36358/0 (socket says 172.21.15.152:36358) 2024-04-20T11:36:01.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.055+0000 7f4dc1000640 1 -- 172.21.15.152:0/2251936941 learned_addr learned my addr 172.21.15.152:0/2251936941 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:01.057 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.055+0000 7f4dc1a00640 1 --2- 172.21.15.152:0/2251936941 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4dbc0b2880 0x7f4dbc15be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:01.057 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.055+0000 7f4dc2400640 1 --2- 172.21.15.152:0/2251936941 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4dbc15b510 0x7f4dbc1493c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:01.057 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.055+0000 7f4dc1000640 1 -- 172.21.15.152:0/2251936941 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4dbc0b2880 msgr2=0x7f4dbc15be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:01.057 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.055+0000 7f4dc1000640 1 --2- 172.21.15.152:0/2251936941 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4dbc0b2880 0x7f4dbc15be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:01.057 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.055+0000 7f4dc1000640 1 -- 172.21.15.152:0/2251936941 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4dbc15b510 msgr2=0x7f4dbc1493c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:01.057 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.055+0000 7f4dc1000640 1 --2- 172.21.15.152:0/2251936941 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4dbc15b510 0x7f4dbc1493c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:01.057 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.055+0000 7f4dc1000640 1 -- 172.21.15.152:0/2251936941 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dbc1f4e90 con 0x7f4dbc0cfc40 2024-04-20T11:36:01.057 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.055+0000 7f4dc2400640 1 --2- 172.21.15.152:0/2251936941 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4dbc15b510 0x7f4dbc1493c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:36:01.057 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.056+0000 7f4dc1000640 1 --2- 172.21.15.152:0/2251936941 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4dbc0cfc40 0x7f4dbc1f4950 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7f4da800c4d0 tx=0x7f4da800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:01.057 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.056+0000 7f4db9600640 1 -- 172.21.15.152:0/2251936941 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da8018070 con 0x7f4dbc0cfc40 2024-04-20T11:36:01.057 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.056+0000 7f4db9600640 1 -- 172.21.15.152:0/2251936941 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4da8007500 con 0x7f4dbc0cfc40 2024-04-20T11:36:01.057 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.056+0000 7f4db9600640 1 -- 172.21.15.152:0/2251936941 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da8004a20 con 0x7f4dbc0cfc40 2024-04-20T11:36:01.057 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.056+0000 7f4dc4400640 1 -- 172.21.15.152:0/2251936941 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4dbc1f5130 con 0x7f4dbc0cfc40 2024-04-20T11:36:01.057 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.056+0000 7f4dc4400640 1 -- 172.21.15.152:0/2251936941 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4dbc1f56f0 con 0x7f4dbc0cfc40 2024-04-20T11:36:01.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.057+0000 7f4dc4400640 1 -- 172.21.15.152:0/2251936941 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4dbc0689f0 con 0x7f4dbc0cfc40 2024-04-20T11:36:01.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.058+0000 7f4db9600640 1 -- 172.21.15.152:0/2251936941 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 22) ==== 91601+0+0 (secure 0 0 0) 0x7f4da8004bc0 con 0x7f4dbc0cfc40 2024-04-20T11:36:01.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.059+0000 7f4db9600640 1 --2- 172.21.15.152:0/2251936941 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4d9c06cf20 0x7f4d9c06f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:01.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.059+0000 7f4db9600640 1 -- 172.21.15.152:0/2251936941 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 4943+0+0 (secure 0 0 0) 0x7f4da808b6d0 con 0x7f4dbc0cfc40 2024-04-20T11:36:01.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.060+0000 7f4dc1a00640 1 --2- 172.21.15.152:0/2251936941 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4d9c06cf20 0x7f4d9c06f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:01.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.061+0000 7f4dc1a00640 1 --2- 172.21.15.152:0/2251936941 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4d9c06cf20 0x7f4d9c06f3e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f4dac036000 tx=0x7f4dac008030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:01.063 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.064+0000 7f4db9600640 1 -- 172.21.15.152:0/2251936941 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f4da8059480 con 0x7f4dbc0cfc40 2024-04-20T11:36:01.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.261+0000 7f4dc4400640 1 -- 172.21.15.152:0/2251936941 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4dbc06ecf0 con 0x7f4dbc0cfc40 2024-04-20T11:36:01.262 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.262+0000 7f4db9600640 1 -- 172.21.15.152:0/2251936941 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v134) ==== 93+0+10 (secure 0 0 0) 0x7f4da8058d80 con 0x7f4dbc0cfc40 2024-04-20T11:36:01.262 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 1 2024-04-20T11:36:01.265 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.265+0000 7f4dc4400640 1 -- 172.21.15.152:0/2251936941 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4d9c06cf20 msgr2=0x7f4d9c06f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:01.265 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.265+0000 7f4dc4400640 1 --2- 172.21.15.152:0/2251936941 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4d9c06cf20 0x7f4d9c06f3e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f4dac036000 tx=0x7f4dac008030 comp rx=0 tx=0).stop 2024-04-20T11:36:01.265 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.265+0000 7f4dc4400640 1 -- 172.21.15.152:0/2251936941 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4dbc0cfc40 msgr2=0x7f4dbc1f4950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:01.265 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.265+0000 7f4dc4400640 1 --2- 172.21.15.152:0/2251936941 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4dbc0cfc40 0x7f4dbc1f4950 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7f4da800c4d0 tx=0x7f4da800c990 comp rx=0 tx=0).stop 2024-04-20T11:36:01.265 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.265+0000 7f4dc4400640 1 -- 172.21.15.152:0/2251936941 shutdown_connections 2024-04-20T11:36:01.265 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.265+0000 7f4dc4400640 1 --2- 172.21.15.152:0/2251936941 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4d9c06cf20 0x7f4d9c06f3e0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:01.265 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.265+0000 7f4dc4400640 1 --2- 172.21.15.152:0/2251936941 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4dbc15b510 0x7f4dbc1493c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:01.265 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.265+0000 7f4dc4400640 1 --2- 172.21.15.152:0/2251936941 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4dbc0cfc40 0x7f4dbc1f4950 unknown :-1 s=CLOSED pgs=1921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:01.265 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.265+0000 7f4dc4400640 1 --2- 172.21.15.152:0/2251936941 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4dbc0b2880 0x7f4dbc15be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:01.265 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.265+0000 7f4dc4400640 1 -- 172.21.15.152:0/2251936941 >> 172.21.15.152:0/2251936941 conn(0x7f4dbc06b980 msgr2=0x7f4dbc0cf0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:01.265 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.265+0000 7f4dc4400640 1 -- 172.21.15.152:0/2251936941 shutdown_connections 2024-04-20T11:36:01.266 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.265+0000 7f4dc4400640 1 -- 172.21.15.152:0/2251936941 wait complete. 2024-04-20T11:36:01.284 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_legacy_to_v1' 2024-04-20T11:36:01.484 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.485+0000 7f537aa00640 1 Processor -- start 2024-04-20T11:36:01.485 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.485+0000 7f537aa00640 1 -- start start 2024-04-20T11:36:01.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.486+0000 7f537aa00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f537415ba10 0x7f53740b4a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:01.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.486+0000 7f537aa00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f53740676c0 0x7f53740b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:01.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.487+0000 7f537aa00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53740b5ad0 con 0x7f53740b5510 2024-04-20T11:36:01.487 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.487+0000 7f537aa00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f53740ce1e0 con 0x7f537415ba10 2024-04-20T11:36:01.487 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.487+0000 7f537aa00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f537406ced0 con 0x7f53740676c0 2024-04-20T11:36:01.487 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.487+0000 7f5373400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f53740676c0 0x7f53740b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:01.488 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.488+0000 7f5378a00640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7f53740b5510 0x7f537415f580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.152:36012/0 (socket says 172.21.15.152:36012) 2024-04-20T11:36:01.488 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.488+0000 7f5378a00640 1 -- 172.21.15.152:0/2037589208 learned_addr learned my addr 172.21.15.152:0/2037589208 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:36:01.488 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.488+0000 7f5373e00640 1 --2- 172.21.15.152:0/2037589208 >> v2:172.21.15.175:3300/0 conn(0x7f537415ba10 0x7f53740b4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:01.489 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.488+0000 7f5372a00640 1 -- 172.21.15.152:0/2037589208 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1784637733 0 0) 0x7f53740b5ad0 con 0x7f53740b5510 2024-04-20T11:36:01.489 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.488+0000 7f5372a00640 1 -- 172.21.15.152:0/2037589208 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5358003520 con 0x7f53740b5510 2024-04-20T11:36:01.489 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.489+0000 7f5373e00640 1 -- 172.21.15.152:0/2037589208 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f53740676c0 msgr2=0x7f53740b4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:01.489 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.489+0000 7f5373e00640 1 --2- 172.21.15.152:0/2037589208 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f53740676c0 0x7f53740b4fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:01.489 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.489+0000 7f5373e00640 1 -- 172.21.15.152:0/2037589208 >> v1:172.21.15.175:6789/0 conn(0x7f53740b5510 legacy=0x7f537415f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:01.489 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.489+0000 7f5373e00640 1 -- 172.21.15.152:0/2037589208 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f537406d1b0 con 0x7f537415ba10 2024-04-20T11:36:01.489 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.489+0000 7f5373400640 1 --2- 172.21.15.152:0/2037589208 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f53740676c0 0x7f53740b4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:36:01.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.490+0000 7f5373e00640 1 --2- 172.21.15.152:0/2037589208 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f537415ba10 0x7f53740b4a90 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f535c0097a0 tx=0x7f535c02b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40ad3e2658e2b1ec server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:01.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.491+0000 7f5372a00640 1 -- 172.21.15.152:0/2037589208 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f535c039070 con 0x7f537415ba10 2024-04-20T11:36:01.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.491+0000 7f5372a00640 1 -- 172.21.15.152:0/2037589208 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f535c0316f0 con 0x7f537415ba10 2024-04-20T11:36:01.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.491+0000 7f5372a00640 1 -- 172.21.15.152:0/2037589208 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f535c031aa0 con 0x7f537415ba10 2024-04-20T11:36:01.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.492+0000 7f537aa00640 1 -- 172.21.15.152:0/2037589208 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f537415ba10 msgr2=0x7f53740b4a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:01.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.492+0000 7f537aa00640 1 --2- 172.21.15.152:0/2037589208 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f537415ba10 0x7f53740b4a90 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f535c0097a0 tx=0x7f535c02b790 comp rx=0 tx=0).stop 2024-04-20T11:36:01.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.492+0000 7f537aa00640 1 -- 172.21.15.152:0/2037589208 shutdown_connections 2024-04-20T11:36:01.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.492+0000 7f537aa00640 1 --2- 172.21.15.152:0/2037589208 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f53740676c0 0x7f53740b4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:01.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.492+0000 7f537aa00640 1 --2- 172.21.15.152:0/2037589208 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f537415ba10 0x7f53740b4a90 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:01.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.492+0000 7f537aa00640 1 -- 172.21.15.152:0/2037589208 >> 172.21.15.152:0/2037589208 conn(0x7f53740cd6f0 msgr2=0x7f53740cdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:01.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.492+0000 7f537aa00640 1 -- 172.21.15.152:0/2037589208 shutdown_connections 2024-04-20T11:36:01.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.492+0000 7f537aa00640 1 -- 172.21.15.152:0/2037589208 wait complete. 2024-04-20T11:36:01.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.494+0000 7f537aa00640 1 Processor -- start 2024-04-20T11:36:01.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.494+0000 7f537aa00640 1 -- start start 2024-04-20T11:36:01.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.495+0000 7f537aa00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f53740676c0 0x7f5374157680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:01.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.495+0000 7f537aa00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f53740b5510 0x7f5374159f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:01.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.495+0000 7f537aa00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f537415ba10 0x7f537415a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:01.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.495+0000 7f537aa00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f53740ce670 con 0x7f53740676c0 2024-04-20T11:36:01.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.495+0000 7f537aa00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f53740ceb30 con 0x7f53740b5510 2024-04-20T11:36:01.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.495+0000 7f537aa00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f53741593d0 con 0x7f537415ba10 2024-04-20T11:36:01.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.495+0000 7f5373e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f53740676c0 0x7f5374157680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:01.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.495+0000 7f5373e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f53740676c0 0x7f5374157680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36374/0 (socket says 172.21.15.152:36374) 2024-04-20T11:36:01.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.495+0000 7f5373e00640 1 -- 172.21.15.152:0/3078392192 learned_addr learned my addr 172.21.15.152:0/3078392192 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:01.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.495+0000 7f5373400640 1 --2- 172.21.15.152:0/3078392192 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f53740b5510 0x7f5374159f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:01.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.495+0000 7f5378a00640 1 --2- 172.21.15.152:0/3078392192 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f537415ba10 0x7f537415a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:01.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.496+0000 7f5373e00640 1 -- 172.21.15.152:0/3078392192 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f537415ba10 msgr2=0x7f537415a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:01.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.496+0000 7f5373e00640 1 --2- 172.21.15.152:0/3078392192 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f537415ba10 0x7f537415a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:01.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.496+0000 7f5373e00640 1 -- 172.21.15.152:0/3078392192 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f53740b5510 msgr2=0x7f5374159f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:01.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.496+0000 7f5373e00640 1 --2- 172.21.15.152:0/3078392192 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f53740b5510 0x7f5374159f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:01.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.496+0000 7f5373e00640 1 -- 172.21.15.152:0/3078392192 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5374151170 con 0x7f53740676c0 2024-04-20T11:36:01.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.496+0000 7f5373e00640 1 --2- 172.21.15.152:0/3078392192 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f53740676c0 0x7f5374157680 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f535c000c00 tx=0x7f535c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:01.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.496+0000 7f536be00640 1 -- 172.21.15.152:0/3078392192 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f535c039050 con 0x7f53740676c0 2024-04-20T11:36:01.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.496+0000 7f536be00640 1 -- 172.21.15.152:0/3078392192 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f535c03a040 con 0x7f53740676c0 2024-04-20T11:36:01.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.496+0000 7f536be00640 1 -- 172.21.15.152:0/3078392192 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f535c031ab0 con 0x7f53740676c0 2024-04-20T11:36:01.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.497+0000 7f537aa00640 1 -- 172.21.15.152:0/3078392192 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f537415ab80 con 0x7f53740676c0 2024-04-20T11:36:01.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.497+0000 7f537aa00640 1 -- 172.21.15.152:0/3078392192 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f537415b140 con 0x7f53740676c0 2024-04-20T11:36:01.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.498+0000 7f537aa00640 1 -- 172.21.15.152:0/3078392192 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5374069100 con 0x7f53740676c0 2024-04-20T11:36:01.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.499+0000 7f536be00640 1 -- 172.21.15.152:0/3078392192 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 22) ==== 91601+0+0 (secure 0 0 0) 0x7f535c047020 con 0x7f53740676c0 2024-04-20T11:36:01.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.500+0000 7f536be00640 1 --2- 172.21.15.152:0/3078392192 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f535006cf20 0x7f535006f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:01.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.500+0000 7f536be00640 1 -- 172.21.15.152:0/3078392192 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 4943+0+0 (secure 0 0 0) 0x7f535c0adac0 con 0x7f53740676c0 2024-04-20T11:36:01.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.501+0000 7f5373400640 1 --2- 172.21.15.152:0/3078392192 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f535006cf20 0x7f535006f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:01.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.502+0000 7f5373400640 1 --2- 172.21.15.152:0/3078392192 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f535006cf20 0x7f535006f3e0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f5364009bd0 tx=0x7f536400b040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:01.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.505+0000 7f536be00640 1 -- 172.21.15.152:0/3078392192 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f535c07b910 con 0x7f53740676c0 2024-04-20T11:36:01.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:01.782+0000 7f537aa00640 1 -- 172.21.15.152:0/3078392192 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_legacy_to_v1"]} v 0) -- 0x7f53740cce90 con 0x7f53740676c0 2024-04-20T11:36:02.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:02.376+0000 7f536be00640 1 -- 172.21.15.152:0/3078392192 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_legacy_to_v1"]}]=0 v484) ==== 152+0+0 (secure 0 0 0) 0x7f535c07b210 con 0x7f53740676c0 2024-04-20T11:36:02.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:02.380+0000 7f537aa00640 1 -- 172.21.15.152:0/3078392192 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f535006cf20 msgr2=0x7f535006f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:02.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:02.380+0000 7f537aa00640 1 --2- 172.21.15.152:0/3078392192 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f535006cf20 0x7f535006f3e0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f5364009bd0 tx=0x7f536400b040 comp rx=0 tx=0).stop 2024-04-20T11:36:02.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:02.380+0000 7f537aa00640 1 -- 172.21.15.152:0/3078392192 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f53740676c0 msgr2=0x7f5374157680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:02.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:02.380+0000 7f537aa00640 1 --2- 172.21.15.152:0/3078392192 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f53740676c0 0x7f5374157680 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f535c000c00 tx=0x7f535c0046c0 comp rx=0 tx=0).stop 2024-04-20T11:36:02.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:02.380+0000 7f537aa00640 1 -- 172.21.15.152:0/3078392192 shutdown_connections 2024-04-20T11:36:02.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:02.380+0000 7f537aa00640 1 --2- 172.21.15.152:0/3078392192 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f535006cf20 0x7f535006f3e0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:02.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:02.380+0000 7f537aa00640 1 --2- 172.21.15.152:0/3078392192 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f537415ba10 0x7f537415a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:02.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:02.380+0000 7f537aa00640 1 --2- 172.21.15.152:0/3078392192 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f53740b5510 0x7f5374159f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:02.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:02.380+0000 7f537aa00640 1 --2- 172.21.15.152:0/3078392192 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f53740676c0 0x7f5374157680 unknown :-1 s=CLOSED pgs=1922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:02.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:02.380+0000 7f537aa00640 1 -- 172.21.15.152:0/3078392192 >> 172.21.15.152:0/3078392192 conn(0x7f53740cd6f0 msgr2=0x7f53740b49e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:02.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:02.380+0000 7f537aa00640 1 -- 172.21.15.152:0/3078392192 shutdown_connections 2024-04-20T11:36:02.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:02.381+0000 7f537aa00640 1 -- 172.21.15.152:0/3078392192 wait complete. 2024-04-20T11:36:02.404 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-20T11:36:02.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:02.607+0000 7ff85a400640 1 Processor -- start 2024-04-20T11:36:02.623 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:02.608+0000 7ff85a400640 1 -- start start 2024-04-20T11:36:02.623 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:02.609+0000 7ff85a400640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ff85415e890 0x7ff8540d27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:02.623 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:02.609+0000 7ff85a400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7ff8540d2d20 0x7ff8540b7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:02.623 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:02.609+0000 7ff85a400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff8540b7cc0 0x7ff8540d1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:02.623 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:02.609+0000 7ff85a400640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7ff8540b8e20 con 0x7ff8540d2d20 2024-04-20T11:36:02.623 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:02.609+0000 7ff85a400640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7ff85415ec70 con 0x7ff85415e890 2024-04-20T11:36:02.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:02.609+0000 7ff85a400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff8540b8210 con 0x7ff8540b7cc0 2024-04-20T11:36:02.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:02.609+0000 7ff852a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7ff8540d2d20 0x7ff8540b7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:02.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:02.610+0000 7ff852a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7ff8540d2d20 0x7ff8540b7780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36386/0 (socket says 172.21.15.152:36386) 2024-04-20T11:36:02.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:02.610+0000 7ff852a00640 1 -- 172.21.15.152:0/2416635711 learned_addr learned my addr 172.21.15.152:0/2416635711 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:02.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:02.610+0000 7ff853e00640 1 --2- 172.21.15.152:0/2416635711 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff8540b7cc0 0x7ff8540d1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:02.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:02.610+0000 7ff853400640 1 --2- 172.21.15.152:0/2416635711 >> v2:172.21.15.175:3300/0 conn(0x7ff85415e890 0x7ff8540d27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:02.627 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:02.610+0000 7ff852a00640 1 -- 172.21.15.152:0/2416635711 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff8540b7cc0 msgr2=0x7ff8540d1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:02.627 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:02.610+0000 7ff852a00640 1 --2- 172.21.15.152:0/2416635711 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff8540b7cc0 0x7ff8540d1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:02.627 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:02.610+0000 7ff852a00640 1 -- 172.21.15.152:0/2416635711 >> v2:172.21.15.175:3300/0 conn(0x7ff85415e890 msgr2=0x7ff8540d27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:02.627 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:02.610+0000 7ff852a00640 1 --2- 172.21.15.152:0/2416635711 >> v2:172.21.15.175:3300/0 conn(0x7ff85415e890 0x7ff8540d27e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:02.627 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:02.610+0000 7ff852a00640 1 -- 172.21.15.152:0/2416635711 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8540b84f0 con 0x7ff8540d2d20 2024-04-20T11:36:02.627 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:02.610+0000 7ff853400640 1 --2- 172.21.15.152:0/2416635711 >> v2:172.21.15.175:3300/0 conn(0x7ff85415e890 0x7ff8540d27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:36:02.627 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:02.611+0000 7ff852a00640 1 --2- 172.21.15.152:0/2416635711 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff8540d2d20 0x7ff8540b7780 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7ff8540b64b0 tx=0x7ff84002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2990aa650b039e21 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:02.627 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:02.612+0000 7ff852000640 1 -- 172.21.15.152:0/2416635711 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff840034070 con 0x7ff8540d2d20 2024-04-20T11:36:02.627 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:02.612+0000 7ff852000640 1 -- 172.21.15.152:0/2416635711 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff840030860 con 0x7ff8540d2d20 2024-04-20T11:36:02.627 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:02.612+0000 7ff852000640 1 -- 172.21.15.152:0/2416635711 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff840030c30 con 0x7ff8540d2d20 2024-04-20T11:36:02.627 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:02.613+0000 7ff85a400640 1 -- 172.21.15.152:0/2416635711 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff8540d2d20 msgr2=0x7ff8540b7780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:02.627 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:02.613+0000 7ff85a400640 1 --2- 172.21.15.152:0/2416635711 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff8540d2d20 0x7ff8540b7780 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7ff8540b64b0 tx=0x7ff84002b9a0 comp rx=0 tx=0).stop 2024-04-20T11:36:02.627 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:02.613+0000 7ff85a400640 1 -- 172.21.15.152:0/2416635711 shutdown_connections 2024-04-20T11:36:02.627 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:02.613+0000 7ff85a400640 1 --2- 172.21.15.152:0/2416635711 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff8540b7cc0 0x7ff8540d1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:02.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:02.613+0000 7ff85a400640 1 --2- 172.21.15.152:0/2416635711 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff8540d2d20 0x7ff8540b7780 unknown :-1 s=CLOSED pgs=1923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:02.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:02.613+0000 7ff85a400640 1 --2- 172.21.15.152:0/2416635711 >> v2:172.21.15.175:3300/0 conn(0x7ff85415e890 0x7ff8540d27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:02.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:02.613+0000 7ff85a400640 1 -- 172.21.15.152:0/2416635711 >> 172.21.15.152:0/2416635711 conn(0x7ff854067ae0 msgr2=0x7ff854067ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:02.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:02.613+0000 7ff85a400640 1 -- 172.21.15.152:0/2416635711 shutdown_connections 2024-04-20T11:36:02.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:02.613+0000 7ff85a400640 1 -- 172.21.15.152:0/2416635711 wait complete. 2024-04-20T11:36:02.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:02.615+0000 7ff85a400640 1 Processor -- start 2024-04-20T11:36:02.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:02.615+0000 7ff85a400640 1 -- start start 2024-04-20T11:36:02.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:02.616+0000 7ff85a400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff8540b7cc0 0x7ff85414ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:02.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:02.616+0000 7ff85a400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff85415e890 0x7ff854160310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:02.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:02.616+0000 7ff85a400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff8540cc820 0x7ff8540cd200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:02.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:02.616+0000 7ff85a400640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7ff8540ba430 con 0x7ff8540cc820 2024-04-20T11:36:02.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:02.616+0000 7ff85a400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff8540b8ab0 con 0x7ff8540b7cc0 2024-04-20T11:36:02.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:02.616+0000 7ff85a400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff8540ba5b0 con 0x7ff85415e890 2024-04-20T11:36:02.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:02.616+0000 7ff853e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff8540cc820 0x7ff8540cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:02.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:02.616+0000 7ff853e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff8540cc820 0x7ff8540cd200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36394/0 (socket says 172.21.15.152:36394) 2024-04-20T11:36:02.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:02.616+0000 7ff853e00640 1 -- 172.21.15.152:0/3134689826 learned_addr learned my addr 172.21.15.152:0/3134689826 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:02.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:02.616+0000 7ff852a00640 1 --2- 172.21.15.152:0/3134689826 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff85415e890 0x7ff854160310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:02.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:02.616+0000 7ff853400640 1 --2- 172.21.15.152:0/3134689826 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff8540b7cc0 0x7ff85414ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:02.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:02.616+0000 7ff853e00640 1 -- 172.21.15.152:0/3134689826 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff85415e890 msgr2=0x7ff854160310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:02.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:02.616+0000 7ff853e00640 1 --2- 172.21.15.152:0/3134689826 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff85415e890 0x7ff854160310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:02.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:02.616+0000 7ff853e00640 1 -- 172.21.15.152:0/3134689826 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff8540b7cc0 msgr2=0x7ff85414ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:02.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:02.616+0000 7ff853e00640 1 --2- 172.21.15.152:0/3134689826 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff8540b7cc0 0x7ff85414ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:02.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:02.616+0000 7ff853e00640 1 -- 172.21.15.152:0/3134689826 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff85414cf80 con 0x7ff8540cc820 2024-04-20T11:36:02.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:02.617+0000 7ff853e00640 1 --2- 172.21.15.152:0/3134689826 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff8540cc820 0x7ff8540cd200 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7ff84c00d600 tx=0x7ff84c00dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:02.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:02.617+0000 7ff84b400640 1 -- 172.21.15.152:0/3134689826 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff84c018020 con 0x7ff8540cc820 2024-04-20T11:36:02.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:02.617+0000 7ff84b400640 1 -- 172.21.15.152:0/3134689826 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff84c014620 con 0x7ff8540cc820 2024-04-20T11:36:02.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:02.617+0000 7ff85a400640 1 -- 172.21.15.152:0/3134689826 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff85415f270 con 0x7ff8540cc820 2024-04-20T11:36:02.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:02.617+0000 7ff84b400640 1 -- 172.21.15.152:0/3134689826 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff84c0149f0 con 0x7ff8540cc820 2024-04-20T11:36:02.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:02.617+0000 7ff85a400640 1 -- 172.21.15.152:0/3134689826 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff85415f830 con 0x7ff8540cc820 2024-04-20T11:36:02.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:02.618+0000 7ff85a400640 1 -- 172.21.15.152:0/3134689826 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8540d0bd0 con 0x7ff8540cc820 2024-04-20T11:36:02.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:02.619+0000 7ff84b400640 1 -- 172.21.15.152:0/3134689826 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 22) ==== 91601+0+0 (secure 0 0 0) 0x7ff84c01b020 con 0x7ff8540cc820 2024-04-20T11:36:02.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:02.620+0000 7ff84b400640 1 --2- 172.21.15.152:0/3134689826 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7ff83406cf20 0x7ff83406f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:02.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:02.620+0000 7ff84b400640 1 -- 172.21.15.152:0/3134689826 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 4943+0+0 (secure 0 0 0) 0x7ff84c08c520 con 0x7ff8540cc820 2024-04-20T11:36:02.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:02.621+0000 7ff853400640 1 --2- 172.21.15.152:0/3134689826 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7ff83406cf20 0x7ff83406f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:02.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:02.622+0000 7ff853400640 1 --2- 172.21.15.152:0/3134689826 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7ff83406cf20 0x7ff83406f3e0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7ff844008f80 tx=0x7ff8440084d0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:02.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:02.625+0000 7ff84b400640 1 -- 172.21.15.152:0/3134689826 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ff8540d0bd0 con 0x7ff8540cc820 2024-04-20T11:36:02.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:02.825+0000 7ff85a400640 1 -- 172.21.15.152:0/3134689826 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7ff8540d3c40 con 0x7ff8540cc820 2024-04-20T11:36:02.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:02.832+0000 7ff84b400640 1 -- 172.21.15.152:0/3134689826 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v26)=0 v26) ==== 122+0+0 (secure 0 0 0) 0x7ff84c022030 con 0x7ff8540cc820 2024-04-20T11:36:02.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:02.836+0000 7ff85a400640 1 -- 172.21.15.152:0/3134689826 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7ff83406cf20 msgr2=0x7ff83406f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:02.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:02.836+0000 7ff85a400640 1 --2- 172.21.15.152:0/3134689826 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7ff83406cf20 0x7ff83406f3e0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7ff844008f80 tx=0x7ff8440084d0 comp rx=0 tx=0).stop 2024-04-20T11:36:02.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:02.836+0000 7ff85a400640 1 -- 172.21.15.152:0/3134689826 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff8540cc820 msgr2=0x7ff8540cd200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:02.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:02.836+0000 7ff85a400640 1 --2- 172.21.15.152:0/3134689826 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff8540cc820 0x7ff8540cd200 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7ff84c00d600 tx=0x7ff84c00dac0 comp rx=0 tx=0).stop 2024-04-20T11:36:02.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:02.836+0000 7ff85a400640 1 -- 172.21.15.152:0/3134689826 shutdown_connections 2024-04-20T11:36:02.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:02.836+0000 7ff85a400640 1 --2- 172.21.15.152:0/3134689826 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7ff83406cf20 0x7ff83406f3e0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:02.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:02.836+0000 7ff85a400640 1 --2- 172.21.15.152:0/3134689826 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff8540cc820 0x7ff8540cd200 unknown :-1 s=CLOSED pgs=1924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:02.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:02.836+0000 7ff85a400640 1 --2- 172.21.15.152:0/3134689826 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff85415e890 0x7ff854160310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:02.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:02.836+0000 7ff85a400640 1 --2- 172.21.15.152:0/3134689826 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff8540b7cc0 0x7ff85414ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:02.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:02.836+0000 7ff85a400640 1 -- 172.21.15.152:0/3134689826 >> 172.21.15.152:0/3134689826 conn(0x7ff854067ae0 msgr2=0x7ff8540d3110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:02.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:02.836+0000 7ff85a400640 1 -- 172.21.15.152:0/3134689826 shutdown_connections 2024-04-20T11:36:02.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:02.836+0000 7ff85a400640 1 -- 172.21.15.152:0/3134689826 wait complete. 2024-04-20T11:36:02.855 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-20T11:36:02.855 DEBUG:teuthology.orchestra.run.smithi152:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-20T11:36:02.958 DEBUG:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-20T11:36:03.071 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-20T11:36:03.071 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-20T11:36:03.072 DEBUG:teuthology.orchestra.run.smithi152:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:36:03.129 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-20T11:36:03.129 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T11:36:03.129 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:36:03.188 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-20T11:36:03.188 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-20T11:36:03.188 DEBUG:teuthology.orchestra.run.smithi152:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:36:03.246 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-20T11:36:03.246 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T11:36:03.246 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:36:03.303 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-20T11:36:03.303 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-20T11:36:03.303 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:36:03.513 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:03.513+0000 7fb7f1c00640 1 Processor -- start 2024-04-20T11:36:03.513 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:03.513+0000 7fb7f1c00640 1 -- start start 2024-04-20T11:36:03.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:03.515+0000 7fb7f1c00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fb7ec15d1e0 0x7fb7ec160b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:03.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:03.515+0000 7fb7f1c00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fb7ec15d5c0 0x7fb7ec0cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:03.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:03.515+0000 7fb7f1c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb7ec071cd0 0x7fb7ec072160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:03.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:03.515+0000 7fb7f1c00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fb7ec0726a0 con 0x7fb7ec15d1e0 2024-04-20T11:36:03.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:03.515+0000 7fb7f1c00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fb7ec0ce8c0 con 0x7fb7ec15d5c0 2024-04-20T11:36:03.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:03.515+0000 7fb7f1c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb7ec1600b0 con 0x7fb7ec071cd0 2024-04-20T11:36:03.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:03.515+0000 7fb7ebe00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fb7ec15d1e0 0x7fb7ec160b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:03.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:03.515+0000 7fb7ebe00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fb7ec15d1e0 0x7fb7ec160b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36400/0 (socket says 172.21.15.152:36400) 2024-04-20T11:36:03.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:03.515+0000 7fb7ebe00640 1 -- 172.21.15.152:0/3454622556 learned_addr learned my addr 172.21.15.152:0/3454622556 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:03.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:03.516+0000 7fb7eb400640 1 --2- 172.21.15.152:0/3454622556 >> v2:172.21.15.175:3300/0 conn(0x7fb7ec15d5c0 0x7fb7ec0cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:03.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:03.516+0000 7fb7f1200640 1 --2- 172.21.15.152:0/3454622556 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb7ec071cd0 0x7fb7ec072160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:03.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:03.516+0000 7fb7ebe00640 1 -- 172.21.15.152:0/3454622556 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb7ec071cd0 msgr2=0x7fb7ec072160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:03.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:03.516+0000 7fb7ebe00640 1 --2- 172.21.15.152:0/3454622556 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb7ec071cd0 0x7fb7ec072160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:03.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:03.516+0000 7fb7ebe00640 1 -- 172.21.15.152:0/3454622556 >> v2:172.21.15.175:3300/0 conn(0x7fb7ec15d5c0 msgr2=0x7fb7ec0cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:03.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:03.516+0000 7fb7ebe00640 1 --2- 172.21.15.152:0/3454622556 >> v2:172.21.15.175:3300/0 conn(0x7fb7ec15d5c0 0x7fb7ec0cdfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:03.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:03.516+0000 7fb7ebe00640 1 -- 172.21.15.152:0/3454622556 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7ec160390 con 0x7fb7ec15d1e0 2024-04-20T11:36:03.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:03.518+0000 7fb7ebe00640 1 --2- 172.21.15.152:0/3454622556 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb7ec15d1e0 0x7fb7ec160b90 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc009870 tx=0x7fb7dc02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=624e90901eb126b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:03.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:03.518+0000 7fb7eaa00640 1 -- 172.21.15.152:0/3454622556 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7dc035020 con 0x7fb7ec15d1e0 2024-04-20T11:36:03.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:03.519+0000 7fb7eaa00640 1 -- 172.21.15.152:0/3454622556 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7dc031640 con 0x7fb7ec15d1e0 2024-04-20T11:36:03.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:03.519+0000 7fb7eaa00640 1 -- 172.21.15.152:0/3454622556 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7dc02d040 con 0x7fb7ec15d1e0 2024-04-20T11:36:03.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:03.519+0000 7fb7f1c00640 1 -- 172.21.15.152:0/3454622556 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb7ec15d1e0 msgr2=0x7fb7ec160b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:03.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:03.519+0000 7fb7f1c00640 1 --2- 172.21.15.152:0/3454622556 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb7ec15d1e0 0x7fb7ec160b90 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc009870 tx=0x7fb7dc02b8f0 comp rx=0 tx=0).stop 2024-04-20T11:36:03.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:03.519+0000 7fb7f1c00640 1 -- 172.21.15.152:0/3454622556 shutdown_connections 2024-04-20T11:36:03.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:03.519+0000 7fb7f1c00640 1 --2- 172.21.15.152:0/3454622556 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb7ec071cd0 0x7fb7ec072160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:03.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:03.519+0000 7fb7f1c00640 1 --2- 172.21.15.152:0/3454622556 >> v2:172.21.15.175:3300/0 conn(0x7fb7ec15d5c0 0x7fb7ec0cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:03.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:03.519+0000 7fb7f1c00640 1 --2- 172.21.15.152:0/3454622556 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb7ec15d1e0 0x7fb7ec160b90 unknown :-1 s=CLOSED pgs=1925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:03.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:03.519+0000 7fb7f1c00640 1 -- 172.21.15.152:0/3454622556 >> 172.21.15.152:0/3454622556 conn(0x7fb7ec067ae0 msgr2=0x7fb7ec06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:03.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:03.519+0000 7fb7f1c00640 1 -- 172.21.15.152:0/3454622556 shutdown_connections 2024-04-20T11:36:03.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:03.519+0000 7fb7f1c00640 1 -- 172.21.15.152:0/3454622556 wait complete. 2024-04-20T11:36:03.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:03.521+0000 7fb7f1c00640 1 Processor -- start 2024-04-20T11:36:03.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:03.521+0000 7fb7f1c00640 1 -- start start 2024-04-20T11:36:03.524 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:03.522+0000 7fb7f1c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb7ec071cd0 0x7fb7ec14a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:03.524 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:03.522+0000 7fb7f1c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7ec15d1e0 0x7fb7ec0b41d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:03.524 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:03.522+0000 7fb7f1c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb7ec15d5c0 0x7fb7ec0b4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:03.524 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:03.522+0000 7fb7f1c00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fb7ec0ce520 con 0x7fb7ec15d5c0 2024-04-20T11:36:03.524 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:03.522+0000 7fb7f1c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb7ec0ceba0 con 0x7fb7ec15d1e0 2024-04-20T11:36:03.524 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:03.522+0000 7fb7f1c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb7ec0ce6a0 con 0x7fb7ec071cd0 2024-04-20T11:36:03.524 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:03.523+0000 7fb7f1200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb7ec15d5c0 0x7fb7ec0b4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:03.525 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:03.523+0000 7fb7f1200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb7ec15d5c0 0x7fb7ec0b4710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36416/0 (socket says 172.21.15.152:36416) 2024-04-20T11:36:03.525 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:03.523+0000 7fb7f1200640 1 -- 172.21.15.152:0/2050798112 learned_addr learned my addr 172.21.15.152:0/2050798112 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:03.525 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:03.523+0000 7fb7ebe00640 1 --2- 172.21.15.152:0/2050798112 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb7ec071cd0 0x7fb7ec14a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:03.525 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:03.523+0000 7fb7f1200640 1 -- 172.21.15.152:0/2050798112 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb7ec071cd0 msgr2=0x7fb7ec14a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:03.525 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:03.523+0000 7fb7f1200640 1 --2- 172.21.15.152:0/2050798112 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb7ec071cd0 0x7fb7ec14a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:03.525 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:03.523+0000 7fb7f1200640 1 -- 172.21.15.152:0/2050798112 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7ec15d1e0 msgr2=0x7fb7ec0b41d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:03.525 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:03.523+0000 7fb7f1200640 1 --2- 172.21.15.152:0/2050798112 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7ec15d1e0 0x7fb7ec0b41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:03.525 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:03.523+0000 7fb7f1200640 1 -- 172.21.15.152:0/2050798112 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7ec1575c0 con 0x7fb7ec15d5c0 2024-04-20T11:36:03.525 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:03.523+0000 7fb7f1200640 1 --2- 172.21.15.152:0/2050798112 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb7ec15d5c0 0x7fb7ec0b4710 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7fb7e400eed0 tx=0x7fb7e400b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:03.525 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:03.524+0000 7fb7e3e00640 1 -- 172.21.15.152:0/2050798112 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e4018020 con 0x7fb7ec15d5c0 2024-04-20T11:36:03.526 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:03.524+0000 7fb7e3e00640 1 -- 172.21.15.152:0/2050798112 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7e400bd20 con 0x7fb7ec15d5c0 2024-04-20T11:36:03.526 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:03.524+0000 7fb7e3e00640 1 -- 172.21.15.152:0/2050798112 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e4014640 con 0x7fb7ec15d5c0 2024-04-20T11:36:03.526 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:03.524+0000 7fb7f1c00640 1 -- 172.21.15.152:0/2050798112 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7ec1578a0 con 0x7fb7ec15d5c0 2024-04-20T11:36:03.526 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:03.524+0000 7fb7f1c00640 1 -- 172.21.15.152:0/2050798112 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7ec157e60 con 0x7fb7ec15d5c0 2024-04-20T11:36:03.527 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:03.525+0000 7fb7f1c00640 1 -- 172.21.15.152:0/2050798112 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7ec0b1e00 con 0x7fb7ec15d5c0 2024-04-20T11:36:03.527 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:03.526+0000 7fb7e3e00640 1 -- 172.21.15.152:0/2050798112 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 22) ==== 91601+0+0 (secure 0 0 0) 0x7fb7e4005d10 con 0x7fb7ec15d5c0 2024-04-20T11:36:03.527 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:03.527+0000 7fb7e3e00640 1 --2- 172.21.15.152:0/2050798112 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb7cc06cf20 0x7fb7cc06f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:03.527 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:03.527+0000 7fb7e3e00640 1 -- 172.21.15.152:0/2050798112 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 4943+0+0 (secure 0 0 0) 0x7fb7e408cde0 con 0x7fb7ec15d5c0 2024-04-20T11:36:03.527 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:03.528+0000 7fb7ebe00640 1 --2- 172.21.15.152:0/2050798112 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb7cc06cf20 0x7fb7cc06f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:03.528 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:03.529+0000 7fb7ebe00640 1 --2- 172.21.15.152:0/2050798112 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb7cc06cf20 0x7fb7cc06f3e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc036000 tx=0x7fb7dc03c040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:03.531 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:03.532+0000 7fb7e3e00640 1 -- 172.21.15.152:0/2050798112 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fb7e4093080 con 0x7fb7ec15d5c0 2024-04-20T11:36:03.770 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:03.771+0000 7fb7f1c00640 1 -- 172.21.15.152:0/2050798112 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb7ec075e40 con 0x7fb7ec15d5c0 2024-04-20T11:36:03.771 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:03.772+0000 7fb7e3e00640 1 -- 172.21.15.152:0/2050798112 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) ==== 94+0+3878 (secure 0 0 0) 0x7fb7e4022030 con 0x7fb7ec15d5c0 2024-04-20T11:36:03.771 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:36:03.772 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":89,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6062,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1303172876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6833","nonce":1303172876},{"type":"v1","addr":"172.21.15.175:6835","nonce":1303172876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6667,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6834/1975208598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1975208598},{"type":"v1","addr":"172.21.15.175:6834","nonce":1975208598}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6670,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6834/287411071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":287411071},{"type":"v1","addr":"172.21.15.152:6834","nonce":287411071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":9417,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/3718750921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6833","nonce":3718750921},{"type":"v1","addr":"172.21.15.152:6835","nonce":3718750921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":89}],"filesystems":[]} 2024-04-20T11:36:03.772 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 89 2024-04-20T11:36:03.773 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:03.774+0000 7fb7f1c00640 1 -- 172.21.15.152:0/2050798112 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb7cc06cf20 msgr2=0x7fb7cc06f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:03.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:03.774+0000 7fb7f1c00640 1 --2- 172.21.15.152:0/2050798112 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb7cc06cf20 0x7fb7cc06f3e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc036000 tx=0x7fb7dc03c040 comp rx=0 tx=0).stop 2024-04-20T11:36:03.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:03.774+0000 7fb7f1c00640 1 -- 172.21.15.152:0/2050798112 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb7ec15d5c0 msgr2=0x7fb7ec0b4710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:03.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:03.774+0000 7fb7f1c00640 1 --2- 172.21.15.152:0/2050798112 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb7ec15d5c0 0x7fb7ec0b4710 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7fb7e400eed0 tx=0x7fb7e400b480 comp rx=0 tx=0).stop 2024-04-20T11:36:03.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:03.774+0000 7fb7f1c00640 1 -- 172.21.15.152:0/2050798112 shutdown_connections 2024-04-20T11:36:03.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:03.774+0000 7fb7f1c00640 1 --2- 172.21.15.152:0/2050798112 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb7cc06cf20 0x7fb7cc06f3e0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:03.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:03.774+0000 7fb7f1c00640 1 --2- 172.21.15.152:0/2050798112 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb7ec15d5c0 0x7fb7ec0b4710 unknown :-1 s=CLOSED pgs=1926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:03.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:03.774+0000 7fb7f1c00640 1 --2- 172.21.15.152:0/2050798112 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7ec15d1e0 0x7fb7ec0b41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:03.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:03.774+0000 7fb7f1c00640 1 --2- 172.21.15.152:0/2050798112 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb7ec071cd0 0x7fb7ec14a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:03.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:03.774+0000 7fb7f1c00640 1 -- 172.21.15.152:0/2050798112 >> 172.21.15.152:0/2050798112 conn(0x7fb7ec067ae0 msgr2=0x7fb7ec0cd600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:03.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:03.775+0000 7fb7f1c00640 1 -- 172.21.15.152:0/2050798112 shutdown_connections 2024-04-20T11:36:03.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:03.775+0000 7fb7f1c00640 1 -- 172.21.15.152:0/2050798112 wait complete. 2024-04-20T11:36:03.794 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-20T11:36:03.794 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-20T11:36:03.794 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-20T11:36:03.794 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T11:36:03.794 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-20T11:36:03.794 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-20T11:36:03.794 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-20T11:36:03.795 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T11:36:03.795 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-20T11:36:03.795 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-20T11:36:03.795 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-20T11:36:03.795 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T11:36:03.795 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-20T11:36:03.795 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-20T11:36:03.795 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-20T11:36:03.796 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T11:36:03.814 INFO:tasks.ceph.mds.a:Stopped 2024-04-20T11:36:03.814 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-20T11:36:03.816 INFO:tasks.ceph.mds.b:Stopped 2024-04-20T11:36:03.816 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-20T11:36:03.819 INFO:tasks.ceph.mds.c:Stopped 2024-04-20T11:36:03.819 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-20T11:36:03.825 INFO:tasks.ceph.mds.d:Stopped 2024-04-20T11:36:03.826 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-20T11:36:03.866 INFO:tasks.ceph.mds.a:Started 2024-04-20T11:36:03.867 DEBUG:teuthology.parallel:result is None 2024-04-20T11:36:03.872 INFO:tasks.ceph.mds.b:Started 2024-04-20T11:36:03.873 DEBUG:teuthology.parallel:result is None 2024-04-20T11:36:03.873 INFO:tasks.ceph.mds.c:Started 2024-04-20T11:36:03.873 DEBUG:teuthology.parallel:result is None 2024-04-20T11:36:03.878 INFO:tasks.ceph.mds.d:Started 2024-04-20T11:36:03.879 DEBUG:teuthology.parallel:result is None 2024-04-20T11:36:03.879 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-20T11:36:04.017 INFO:tasks.ceph.mds.a.smithi152.stdout:starting mds.a at 2024-04-20T11:36:04.017 INFO:tasks.ceph.mds.c.smithi152.stdout:starting mds.c at 2024-04-20T11:36:04.029 INFO:tasks.ceph.mds.b.smithi175.stdout:starting mds.b at 2024-04-20T11:36:04.030 INFO:tasks.ceph.mds.d.smithi175.stdout:starting mds.d at 2024-04-20T11:36:04.031 INFO:tasks.ceph.mds.b.smithi175.stderr:2024-04-20T11:36:04.032+0000 7fb138400640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2024-04-20T11:36:04.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.059+0000 7f7d82800640 1 Processor -- start 2024-04-20T11:36:04.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.059+0000 7f7d82800640 1 -- start start 2024-04-20T11:36:04.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.061+0000 7f7d82800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f7d7c15b330 0x7f7d7c06e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:04.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.061+0000 7f7d82800640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d7c158cd0 con 0x7f7d7c0b4b70 2024-04-20T11:36:04.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.061+0000 7f7d82800640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d7c15b7c0 con 0x7f7d7c1588f0 2024-04-20T11:36:04.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.061+0000 7f7d82800640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f7d7c15baa0 con 0x7f7d7c15b330 2024-04-20T11:36:04.061 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.061+0000 7f7d72a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f7d7c15b330 0x7f7d7c06e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:04.061 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.061+0000 7f7d7be00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f7d7c0b4b70 0x7f7d7c06e710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:50046/0 (socket says 172.21.15.152:50046) 2024-04-20T11:36:04.061 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.061+0000 7f7d7be00640 1 -- 172.21.15.152:0/1990594752 learned_addr learned my addr 172.21.15.152:0/1990594752 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:36:04.061 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.062+0000 7f7d72a00640 1 -- 172.21.15.152:0/1990594752 >> v1:172.21.15.175:6789/0 conn(0x7f7d7c1588f0 legacy=0x7f7d7c0b4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:04.061 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.062+0000 7f7d72a00640 1 -- 172.21.15.152:0/1990594752 >> v1:172.21.15.152:6789/0 conn(0x7f7d7c0b4b70 legacy=0x7f7d7c06e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:04.061 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.062+0000 7f7d72a00640 1 -- 172.21.15.152:0/1990594752 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d7c15bd80 con 0x7f7d7c15b330 2024-04-20T11:36:04.061 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.062+0000 7f7d7aa00640 1 -- 172.21.15.152:0/1990594752 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1731449699 0 0) 0x7f7d7c158cd0 con 0x7f7d7c0b4b70 2024-04-20T11:36:04.062 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.063+0000 7f7d72a00640 1 --2- 172.21.15.152:0/1990594752 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7d7c15b330 0x7f7d7c06e1d0 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7f7d6400a940 tx=0x7f7d6402df40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3971bb56cdbe6a5f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:04.062 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.063+0000 7f7d7aa00640 1 -- 172.21.15.152:0/1990594752 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d6403b070 con 0x7f7d7c15b330 2024-04-20T11:36:04.062 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.063+0000 7f7d7aa00640 1 -- 172.21.15.152:0/1990594752 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d6402ecf0 con 0x7f7d7c15b330 2024-04-20T11:36:04.063 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.063+0000 7f7d7aa00640 1 -- 172.21.15.152:0/1990594752 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d6400b040 con 0x7f7d7c15b330 2024-04-20T11:36:04.063 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.064+0000 7f7d82800640 1 -- 172.21.15.152:0/1990594752 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7d7c15b330 msgr2=0x7f7d7c06e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:04.063 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.064+0000 7f7d82800640 1 --2- 172.21.15.152:0/1990594752 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7d7c15b330 0x7f7d7c06e1d0 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7f7d6400a940 tx=0x7f7d6402df40 comp rx=0 tx=0).stop 2024-04-20T11:36:04.063 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.064+0000 7f7d82800640 1 -- 172.21.15.152:0/1990594752 shutdown_connections 2024-04-20T11:36:04.063 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.064+0000 7f7d82800640 1 --2- 172.21.15.152:0/1990594752 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7d7c15b330 0x7f7d7c06e1d0 unknown :-1 s=CLOSED pgs=1937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:04.064 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.064+0000 7f7d82800640 1 -- 172.21.15.152:0/1990594752 >> 172.21.15.152:0/1990594752 conn(0x7f7d7c067cf0 msgr2=0x7f7d7c06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:04.064 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.064+0000 7f7d82800640 1 -- 172.21.15.152:0/1990594752 shutdown_connections 2024-04-20T11:36:04.064 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.064+0000 7f7d82800640 1 -- 172.21.15.152:0/1990594752 wait complete. 2024-04-20T11:36:04.065 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.065+0000 7f7d82800640 1 Processor -- start 2024-04-20T11:36:04.065 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.066+0000 7f7d82800640 1 -- start start 2024-04-20T11:36:04.067 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.066+0000 7f7d82800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7d7c0b4b70 0x7f7d7c1e4310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:04.067 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.066+0000 7f7d82800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7d7c1588f0 0x7f7d7c1e5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:04.068 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.067+0000 7f7d82800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d7c156d30 0x7f7d7c1e5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:04.068 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.067+0000 7f7d82800640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f7d7c0b3ee0 con 0x7f7d7c1588f0 2024-04-20T11:36:04.068 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.067+0000 7f7d82800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7d7c0b3d60 con 0x7f7d7c156d30 2024-04-20T11:36:04.068 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.067+0000 7f7d82800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7d7c0b4060 con 0x7f7d7c0b4b70 2024-04-20T11:36:04.068 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.067+0000 7f7d72a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7d7c1588f0 0x7f7d7c1e5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:04.068 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.067+0000 7f7d72a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7d7c1588f0 0x7f7d7c1e5400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36478/0 (socket says 172.21.15.152:36478) 2024-04-20T11:36:04.068 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.067+0000 7f7d72a00640 1 -- 172.21.15.152:0/3506934695 learned_addr learned my addr 172.21.15.152:0/3506934695 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:04.068 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.067+0000 7f7d7b400640 1 --2- 172.21.15.152:0/3506934695 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7d7c0b4b70 0x7f7d7c1e4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:04.068 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.067+0000 7f7d7be00640 1 --2- 172.21.15.152:0/3506934695 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d7c156d30 0x7f7d7c1e5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:04.068 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.067+0000 7f7d72a00640 1 -- 172.21.15.152:0/3506934695 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7d7c0b4b70 msgr2=0x7f7d7c1e4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:04.068 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.067+0000 7f7d72a00640 1 --2- 172.21.15.152:0/3506934695 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7d7c0b4b70 0x7f7d7c1e4310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:04.068 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.067+0000 7f7d72a00640 1 -- 172.21.15.152:0/3506934695 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d7c156d30 msgr2=0x7f7d7c1e5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:04.068 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.067+0000 7f7d72a00640 1 --2- 172.21.15.152:0/3506934695 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d7c156d30 0x7f7d7c1e5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:04.068 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.067+0000 7f7d72a00640 1 -- 172.21.15.152:0/3506934695 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d7c0cfb10 con 0x7f7d7c1588f0 2024-04-20T11:36:04.069 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.068+0000 7f7d72a00640 1 --2- 172.21.15.152:0/3506934695 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7d7c1588f0 0x7f7d7c1e5400 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7f7d64000c00 tx=0x7f7d640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:04.069 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.068+0000 7f7d73e00640 1 -- 172.21.15.152:0/3506934695 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d6403b040 con 0x7f7d7c1588f0 2024-04-20T11:36:04.069 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.068+0000 7f7d73e00640 1 -- 172.21.15.152:0/3506934695 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d6403f420 con 0x7f7d7c1588f0 2024-04-20T11:36:04.069 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.068+0000 7f7d73e00640 1 -- 172.21.15.152:0/3506934695 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d6403f810 con 0x7f7d7c1588f0 2024-04-20T11:36:04.069 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.068+0000 7f7d82800640 1 -- 172.21.15.152:0/3506934695 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d7c0cfdf0 con 0x7f7d7c1588f0 2024-04-20T11:36:04.069 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.068+0000 7f7d82800640 1 -- 172.21.15.152:0/3506934695 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d7c1e4850 con 0x7f7d7c1588f0 2024-04-20T11:36:04.069 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.069+0000 7f7d82800640 1 -- 172.21.15.152:0/3506934695 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d7c15d180 con 0x7f7d7c1588f0 2024-04-20T11:36:04.070 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.070+0000 7f7d73e00640 1 -- 172.21.15.152:0/3506934695 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 22) ==== 91601+0+0 (secure 0 0 0) 0x7f7d6403f9b0 con 0x7f7d7c1588f0 2024-04-20T11:36:04.071 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.071+0000 7f7d73e00640 1 --2- 172.21.15.152:0/3506934695 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f7d5806cf20 0x7f7d5806f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:04.071 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.071+0000 7f7d73e00640 1 -- 172.21.15.152:0/3506934695 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 4943+0+0 (secure 0 0 0) 0x7f7d640ae630 con 0x7f7d7c1588f0 2024-04-20T11:36:04.071 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.072+0000 7f7d7b400640 1 --2- 172.21.15.152:0/3506934695 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f7d5806cf20 0x7f7d5806f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:04.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.072+0000 7f7d7b400640 1 --2- 172.21.15.152:0/3506934695 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f7d5806cf20 0x7f7d5806f3e0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c0012e0 tx=0x7f7d6c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:04.075 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.076+0000 7f7d73e00640 1 -- 172.21.15.152:0/3506934695 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f7d7c15d180 con 0x7f7d7c1588f0 2024-04-20T11:36:04.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.286+0000 7f7d82800640 1 -- 172.21.15.152:0/3506934695 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f7d7c0689f0 con 0x7f7d7c1588f0 2024-04-20T11:36:04.410 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.410+0000 7f7d73e00640 1 -- 172.21.15.152:0/3506934695 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v136) ==== 97+0+0 (secure 0 0 0) 0x7f7d6407c470 con 0x7f7d7c1588f0 2024-04-20T11:36:04.410 INFO:teuthology.orchestra.run.smithi152.stderr: removed all blocklist entries 2024-04-20T11:36:04.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.413+0000 7f7d82800640 1 -- 172.21.15.152:0/3506934695 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f7d5806cf20 msgr2=0x7f7d5806f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:04.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.413+0000 7f7d82800640 1 --2- 172.21.15.152:0/3506934695 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f7d5806cf20 0x7f7d5806f3e0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c0012e0 tx=0x7f7d6c009040 comp rx=0 tx=0).stop 2024-04-20T11:36:04.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.413+0000 7f7d82800640 1 -- 172.21.15.152:0/3506934695 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7d7c1588f0 msgr2=0x7f7d7c1e5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:04.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.413+0000 7f7d82800640 1 --2- 172.21.15.152:0/3506934695 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7d7c1588f0 0x7f7d7c1e5400 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7f7d64000c00 tx=0x7f7d640046c0 comp rx=0 tx=0).stop 2024-04-20T11:36:04.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.414+0000 7f7d82800640 1 -- 172.21.15.152:0/3506934695 shutdown_connections 2024-04-20T11:36:04.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.414+0000 7f7d82800640 1 --2- 172.21.15.152:0/3506934695 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f7d5806cf20 0x7f7d5806f3e0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:04.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.414+0000 7f7d82800640 1 --2- 172.21.15.152:0/3506934695 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d7c156d30 0x7f7d7c1e5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:04.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.414+0000 7f7d82800640 1 --2- 172.21.15.152:0/3506934695 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7d7c1588f0 0x7f7d7c1e5400 unknown :-1 s=CLOSED pgs=1938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:04.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.414+0000 7f7d82800640 1 --2- 172.21.15.152:0/3506934695 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7d7c0b4b70 0x7f7d7c1e4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:04.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.414+0000 7f7d82800640 1 -- 172.21.15.152:0/3506934695 >> 172.21.15.152:0/3506934695 conn(0x7f7d7c067cf0 msgr2=0x7f7d7c15ca70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:04.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.414+0000 7f7d82800640 1 -- 172.21.15.152:0/3506934695 shutdown_connections 2024-04-20T11:36:04.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.414+0000 7f7d82800640 1 -- 172.21.15.152:0/3506934695 wait complete. 2024-04-20T11:36:04.435 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-20T11:36:04.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.624+0000 7f5f5b200640 1 Processor -- start 2024-04-20T11:36:04.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.624+0000 7f5f5b200640 1 -- start start 2024-04-20T11:36:04.625 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.626+0000 7f5f5b200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f5f54145f60 0x7f5f5406ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:04.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.626+0000 7f5f5b200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5f540726f0 0x7f5f540cd580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:04.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.626+0000 7f5f5b200640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f54146340 con 0x7f5f540cd0d0 2024-04-20T11:36:04.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.626+0000 7f5f5b200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f5f5406e9f0 con 0x7f5f54145f60 2024-04-20T11:36:04.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.626+0000 7f5f5b200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5f5406f3c0 con 0x7f5f540726f0 2024-04-20T11:36:04.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.626+0000 7f5f53400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f5f540cd0d0 0x7f5f54071fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:50056/0 (socket says 172.21.15.152:50056) 2024-04-20T11:36:04.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.626+0000 7f5f53400640 1 -- 172.21.15.152:0/808060843 learned_addr learned my addr 172.21.15.152:0/808060843 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:36:04.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.627+0000 7f5f59200640 1 --2- 172.21.15.152:0/808060843 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5f540726f0 0x7f5f540cd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:04.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.627+0000 7f5f53e00640 1 --2- 172.21.15.152:0/808060843 >> v2:172.21.15.175:3300/0 conn(0x7f5f54145f60 0x7f5f5406ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:04.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.627+0000 7f5f52a00640 1 -- 172.21.15.152:0/808060843 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2604814317 0 0) 0x7f5f54146340 con 0x7f5f540cd0d0 2024-04-20T11:36:04.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.627+0000 7f5f52a00640 1 -- 172.21.15.152:0/808060843 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5f38003520 con 0x7f5f540cd0d0 2024-04-20T11:36:04.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.627+0000 7f5f52a00640 1 -- 172.21.15.152:0/808060843 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 783492822 0 0) 0x7f5f38003520 con 0x7f5f540cd0d0 2024-04-20T11:36:04.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.627+0000 7f5f52a00640 1 -- 172.21.15.152:0/808060843 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5f540726f0 msgr2=0x7f5f540cd580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:04.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.627+0000 7f5f52a00640 1 --2- 172.21.15.152:0/808060843 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5f540726f0 0x7f5f540cd580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:04.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.627+0000 7f5f52a00640 1 -- 172.21.15.152:0/808060843 >> v2:172.21.15.175:3300/0 conn(0x7f5f54145f60 msgr2=0x7f5f5406ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:04.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.627+0000 7f5f52a00640 1 --2- 172.21.15.152:0/808060843 >> v2:172.21.15.175:3300/0 conn(0x7f5f54145f60 0x7f5f5406ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:04.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.627+0000 7f5f52a00640 1 -- 172.21.15.152:0/808060843 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f54072d50 con 0x7f5f540cd0d0 2024-04-20T11:36:04.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.627+0000 7f5f52a00640 1 -- 172.21.15.152:0/808060843 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f5f40001690 con 0x7f5f540cd0d0 2024-04-20T11:36:04.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.627+0000 7f5f52a00640 1 -- 172.21.15.152:0/808060843 >> v1:172.21.15.152:6789/0 conn(0x7f5f540cd0d0 legacy=0x7f5f54071fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:04.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.627+0000 7f5f52a00640 1 --2- 172.21.15.152:0/808060843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f38003a20 0x7f5f38003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:04.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.627+0000 7f5f52a00640 1 --2- 172.21.15.152:0/808060843 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5f540726f0 0x7f5f38004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:04.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.627+0000 7f5f52a00640 1 --2- 172.21.15.152:0/808060843 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5f38004d40 0x7f5f380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:04.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.627+0000 7f5f52a00640 1 -- 172.21.15.152:0/808060843 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f5f5406e9f0 con 0x7f5f38004d40 2024-04-20T11:36:04.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.627+0000 7f5f52a00640 1 -- 172.21.15.152:0/808060843 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5f5406f3c0 con 0x7f5f38003a20 2024-04-20T11:36:04.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.627+0000 7f5f52a00640 1 -- 172.21.15.152:0/808060843 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5f3800ba70 con 0x7f5f540726f0 2024-04-20T11:36:04.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.628+0000 7f5f59200640 1 --2- 172.21.15.152:0/808060843 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5f38004d40 0x7f5f380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:04.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.628+0000 7f5f53400640 1 --2- 172.21.15.152:0/808060843 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5f540726f0 0x7f5f38004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:04.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.628+0000 7f5f59200640 1 -- 172.21.15.152:0/808060843 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5f540726f0 msgr2=0x7f5f38004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:04.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.628+0000 7f5f59200640 1 --2- 172.21.15.152:0/808060843 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5f540726f0 0x7f5f38004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:04.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.628+0000 7f5f59200640 1 -- 172.21.15.152:0/808060843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f38003a20 msgr2=0x7f5f38003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:04.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.628+0000 7f5f59200640 1 --2- 172.21.15.152:0/808060843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f38003a20 0x7f5f38003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:04.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.628+0000 7f5f59200640 1 -- 172.21.15.152:0/808060843 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f38003520 con 0x7f5f38004d40 2024-04-20T11:36:04.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.629+0000 7f5f59200640 1 --2- 172.21.15.152:0/808060843 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5f38004d40 0x7f5f380051c0 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7f5f4c009fc0 tx=0x7f5f38005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5e3ae8b16245cf1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:04.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.629+0000 7f5f52a00640 1 -- 172.21.15.152:0/808060843 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f4c039070 con 0x7f5f38004d40 2024-04-20T11:36:04.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.629+0000 7f5f52a00640 1 -- 172.21.15.152:0/808060843 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f4c02cc20 con 0x7f5f38004d40 2024-04-20T11:36:04.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.629+0000 7f5f52a00640 1 -- 172.21.15.152:0/808060843 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f4c009c10 con 0x7f5f38004d40 2024-04-20T11:36:04.632 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.630+0000 7f5f5b200640 1 -- 172.21.15.152:0/808060843 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5f38004d40 msgr2=0x7f5f380051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:04.632 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.630+0000 7f5f5b200640 1 --2- 172.21.15.152:0/808060843 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5f38004d40 0x7f5f380051c0 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7f5f4c009fc0 tx=0x7f5f38005830 comp rx=0 tx=0).stop 2024-04-20T11:36:04.632 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.630+0000 7f5f59200640 1 -- 172.21.15.152:0/808060843 reap_dead start 2024-04-20T11:36:04.632 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.630+0000 7f5f5b200640 1 -- 172.21.15.152:0/808060843 shutdown_connections 2024-04-20T11:36:04.632 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.630+0000 7f5f5b200640 1 -- 172.21.15.152:0/808060843 >> 172.21.15.152:0/808060843 conn(0x7f5f54067cf0 msgr2=0x7f5f5406b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:04.632 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.630+0000 7f5f5b200640 1 -- 172.21.15.152:0/808060843 shutdown_connections 2024-04-20T11:36:04.632 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.630+0000 7f5f5b200640 1 -- 172.21.15.152:0/808060843 wait complete. 2024-04-20T11:36:04.632 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.632+0000 7f5f5b200640 1 Processor -- start 2024-04-20T11:36:04.632 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.632+0000 7f5f5b200640 1 -- start start 2024-04-20T11:36:04.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.633+0000 7f5f5b200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f38004d40 0x7f5f54140800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:04.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.633+0000 7f5f5b200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5f54144880 0x7f5f5415cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:04.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.633+0000 7f5f5b200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5f54144d00 0x7f5f5415d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:04.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.633+0000 7f5f5b200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f5f5406e3a0 con 0x7f5f54144d00 2024-04-20T11:36:04.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.633+0000 7f5f5b200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5f54072b00 con 0x7f5f38004d40 2024-04-20T11:36:04.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.633+0000 7f5f5b200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5f5406e680 con 0x7f5f54144880 2024-04-20T11:36:04.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.633+0000 7f5f59200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5f54144d00 0x7f5f5415d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:04.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.633+0000 7f5f59200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5f54144d00 0x7f5f5415d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36506/0 (socket says 172.21.15.152:36506) 2024-04-20T11:36:04.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.633+0000 7f5f59200640 1 -- 172.21.15.152:0/4021377544 learned_addr learned my addr 172.21.15.152:0/4021377544 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:04.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.634+0000 7f5f53400640 1 --2- 172.21.15.152:0/4021377544 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5f54144880 0x7f5f5415cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:04.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.634+0000 7f5f59200640 1 -- 172.21.15.152:0/4021377544 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5f54144880 msgr2=0x7f5f5415cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:04.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.634+0000 7f5f59200640 1 --2- 172.21.15.152:0/4021377544 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5f54144880 0x7f5f5415cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:04.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.634+0000 7f5f59200640 1 -- 172.21.15.152:0/4021377544 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f38004d40 msgr2=0x7f5f54140800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:04.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.634+0000 7f5f53e00640 1 --2- 172.21.15.152:0/4021377544 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f38004d40 0x7f5f54140800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:04.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.634+0000 7f5f59200640 1 --2- 172.21.15.152:0/4021377544 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f38004d40 0x7f5f54140800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:04.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.634+0000 7f5f59200640 1 -- 172.21.15.152:0/4021377544 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f5415dd10 con 0x7f5f54144d00 2024-04-20T11:36:04.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.634+0000 7f5f53e00640 1 --2- 172.21.15.152:0/4021377544 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f38004d40 0x7f5f54140800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:36:04.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.634+0000 7f5f59200640 1 --2- 172.21.15.152:0/4021377544 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5f54144d00 0x7f5f5415d490 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7f5f4c002230 tx=0x7f5f4c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:04.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.634+0000 7f5f4be00640 1 -- 172.21.15.152:0/4021377544 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f4c039030 con 0x7f5f54144d00 2024-04-20T11:36:04.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.634+0000 7f5f4be00640 1 -- 172.21.15.152:0/4021377544 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f4c043070 con 0x7f5f54144d00 2024-04-20T11:36:04.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.634+0000 7f5f5b200640 1 -- 172.21.15.152:0/4021377544 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f5415e760 con 0x7f5f54144d00 2024-04-20T11:36:04.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.634+0000 7f5f4be00640 1 -- 172.21.15.152:0/4021377544 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f4c03d7f0 con 0x7f5f54144d00 2024-04-20T11:36:04.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.635+0000 7f5f5b200640 1 -- 172.21.15.152:0/4021377544 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f5415ed20 con 0x7f5f54144d00 2024-04-20T11:36:04.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.636+0000 7f5f5b200640 1 -- 172.21.15.152:0/4021377544 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f540689f0 con 0x7f5f54144d00 2024-04-20T11:36:04.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.637+0000 7f5f4be00640 1 -- 172.21.15.152:0/4021377544 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 22) ==== 91601+0+0 (secure 0 0 0) 0x7f5f4c005d10 con 0x7f5f54144d00 2024-04-20T11:36:04.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.638+0000 7f5f4be00640 1 --2- 172.21.15.152:0/4021377544 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f5f3406cf50 0x7f5f3406f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:04.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.638+0000 7f5f4be00640 1 -- 172.21.15.152:0/4021377544 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 4857+0+0 (secure 0 0 0) 0x7f5f4c0ae4c0 con 0x7f5f54144d00 2024-04-20T11:36:04.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.638+0000 7f5f53e00640 1 --2- 172.21.15.152:0/4021377544 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f5f3406cf50 0x7f5f3406f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:04.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.639+0000 7f5f53e00640 1 --2- 172.21.15.152:0/4021377544 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f5f3406cf50 0x7f5f3406f410 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f5f44006fd0 tx=0x7f5f44008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:04.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.642+0000 7f5f4be00640 1 -- 172.21.15.152:0/4021377544 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f5f4c07c4c0 con 0x7f5f54144d00 2024-04-20T11:36:04.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.915+0000 7f5f5b200640 1 -- 172.21.15.152:0/4021377544 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f5f540afbf0 con 0x7f5f54144d00 2024-04-20T11:36:04.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.916+0000 7f5f4be00640 1 -- 172.21.15.152:0/4021377544 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v20) ==== 80+0+5304 (secure 0 0 0) 0x7f5f4c07bdc0 con 0x7f5f54144d00 2024-04-20T11:36:04.916 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:36:04.916 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-04-20T11:36:04.916 INFO:teuthology.orchestra.run.smithi152.stdout: "auth_dump": [ 2024-04-20T11:36:04.916 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:36:04.916 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "mds.a", 2024-04-20T11:36:04.917 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBCpiNmyFRJIBAAj2aIMXdb65nqRgKFDBLLzA==", 2024-04-20T11:36:04.917 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:36:04.917 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow", 2024-04-20T11:36:04.917 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow *", 2024-04-20T11:36:04.917 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow *", 2024-04-20T11:36:04.917 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:36:04.917 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:36:04.917 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:36:04.917 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:36:04.917 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "mds.b", 2024-04-20T11:36:04.917 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBCpiNmVPytOhAAzQkvNOJaNy+MeKETWhYLyg==", 2024-04-20T11:36:04.917 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:36:04.918 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow", 2024-04-20T11:36:04.918 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow *", 2024-04-20T11:36:04.918 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow *", 2024-04-20T11:36:04.918 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:36:04.918 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:36:04.918 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:36:04.918 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:36:04.918 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "mds.c", 2024-04-20T11:36:04.918 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBCpiNm5ViKLRAAg7VBQG4hFTLYUpJMpBAaVA==", 2024-04-20T11:36:04.918 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:36:04.918 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow", 2024-04-20T11:36:04.918 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow *", 2024-04-20T11:36:04.919 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow *", 2024-04-20T11:36:04.919 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:36:04.919 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:36:04.919 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:36:04.919 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:36:04.919 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "mds.d", 2024-04-20T11:36:04.919 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBDpiNmLWhaCxAA2e4qd/7+TKenJlWICIsvGg==", 2024-04-20T11:36:04.919 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:36:04.919 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow", 2024-04-20T11:36:04.919 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow *", 2024-04-20T11:36:04.919 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow *", 2024-04-20T11:36:04.919 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:36:04.919 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:36:04.920 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:36:04.920 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:36:04.920 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "osd.0", 2024-04-20T11:36:04.920 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBHpiNmpuhyDBAAsQXX9DNvWlo0UYh53lb+FA==", 2024-04-20T11:36:04.920 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:36:04.920 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow profile osd", 2024-04-20T11:36:04.920 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile osd", 2024-04-20T11:36:04.920 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:36:04.920 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:36:04.920 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:36:04.920 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:36:04.920 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "osd.1", 2024-04-20T11:36:04.921 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBKpiNmxBiLJhAAHon1adC8G2jbYoDcWdE9jA==", 2024-04-20T11:36:04.921 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:36:04.921 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow profile osd", 2024-04-20T11:36:04.921 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile osd", 2024-04-20T11:36:04.921 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:36:04.921 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:36:04.921 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:36:04.921 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:36:04.921 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "osd.2", 2024-04-20T11:36:04.921 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBNpiNmdn0zMhAAqwIO2kKahdSzTVh8/p3FCg==", 2024-04-20T11:36:04.921 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:36:04.921 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow profile osd", 2024-04-20T11:36:04.921 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile osd", 2024-04-20T11:36:04.922 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:36:04.922 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:36:04.922 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:36:04.922 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:36:04.922 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "osd.3", 2024-04-20T11:36:04.922 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBRpiNmTPCYAhAAHFNHX/xKn21jEFHDkHMjEw==", 2024-04-20T11:36:04.922 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:36:04.922 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow profile osd", 2024-04-20T11:36:04.922 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile osd", 2024-04-20T11:36:04.922 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:36:04.922 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:36:04.922 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:36:04.923 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:36:04.923 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "osd.4", 2024-04-20T11:36:04.923 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBXpiNmM8cVFxAAJl9Hu+MvlkEiMglutI89eg==", 2024-04-20T11:36:04.923 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:36:04.923 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow profile osd", 2024-04-20T11:36:04.923 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile osd", 2024-04-20T11:36:04.923 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:36:04.923 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:36:04.923 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:36:04.923 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:36:04.923 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "osd.5", 2024-04-20T11:36:04.923 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBapiNmnGxJMRAA90P0T+VsTA8LCTcexTVr7w==", 2024-04-20T11:36:04.924 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:36:04.924 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow profile osd", 2024-04-20T11:36:04.924 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile osd", 2024-04-20T11:36:04.924 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:36:04.924 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:36:04.924 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:36:04.924 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:36:04.924 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "osd.6", 2024-04-20T11:36:04.924 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBepiNmyLXWDxAAhtHD5h478mLovE5IejAFng==", 2024-04-20T11:36:04.924 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:36:04.924 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow profile osd", 2024-04-20T11:36:04.924 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile osd", 2024-04-20T11:36:04.925 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:36:04.925 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:36:04.925 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:36:04.925 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:36:04.925 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "osd.7", 2024-04-20T11:36:04.925 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBhpiNmB3jwHBAAqco9om6r2N/GqAV6rVKiaA==", 2024-04-20T11:36:04.925 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:36:04.925 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow profile osd", 2024-04-20T11:36:04.925 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile osd", 2024-04-20T11:36:04.925 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:36:04.925 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:36:04.925 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:36:04.926 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:36:04.926 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "client.0", 2024-04-20T11:36:04.926 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBDpiNmtsBWFhAArq9A7r33CcwgMmOU2pAGPw==", 2024-04-20T11:36:04.926 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:36:04.926 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow", 2024-04-20T11:36:04.926 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow r", 2024-04-20T11:36:04.926 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-20T11:36:04.926 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:36:04.926 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:36:04.926 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:36:04.926 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "client.1", 2024-04-20T11:36:04.926 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBDpiNm6aNnJxAAB431GSGytZ0IKWMh4DMalw==", 2024-04-20T11:36:04.927 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:36:04.927 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow", 2024-04-20T11:36:04.927 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow r", 2024-04-20T11:36:04.927 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow rw", 2024-04-20T11:36:04.927 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow rwx" 2024-04-20T11:36:04.927 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:36:04.927 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:36:04.927 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:36:04.927 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "client.2", 2024-04-20T11:36:04.927 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBDpiNm5QLtHRAAa2kHTDVdgSh/B4XGZaKa5Q==", 2024-04-20T11:36:04.927 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:36:04.927 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow", 2024-04-20T11:36:04.928 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow r", 2024-04-20T11:36:04.928 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow rw", 2024-04-20T11:36:04.928 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow rwx" 2024-04-20T11:36:04.928 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:36:04.928 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:36:04.928 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:36:04.928 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "client.3", 2024-04-20T11:36:04.928 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBDpiNmw95SMBAAdodzTuPp4WdKSEf+jI8Uuw==", 2024-04-20T11:36:04.928 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:36:04.928 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow", 2024-04-20T11:36:04.928 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow r", 2024-04-20T11:36:04.929 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow rw", 2024-04-20T11:36:04.929 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow rwx" 2024-04-20T11:36:04.929 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:36:04.929 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:36:04.929 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:36:04.929 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "client.admin", 2024-04-20T11:36:04.929 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBBpiNm3xz6HhAA82+N+trDW/AmBr3q8l4ECw==", 2024-04-20T11:36:04.929 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:36:04.929 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow *", 2024-04-20T11:36:04.929 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow *", 2024-04-20T11:36:04.930 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow *", 2024-04-20T11:36:04.930 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:36:04.930 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:36:04.930 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:36:04.930 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:36:04.930 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "mgr.x", 2024-04-20T11:36:04.930 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBCpiNmuhalFxAA9dTl/zSbUovCCZAanAipww==", 2024-04-20T11:36:04.930 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:36:04.930 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow *", 2024-04-20T11:36:04.930 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile mgr", 2024-04-20T11:36:04.930 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:36:04.931 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:36:04.931 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:36:04.931 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:36:04.931 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "mgr.y", 2024-04-20T11:36:04.931 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBCpiNmP1+5DRAAs6bVAEOr0i6FdPmSEyo/vQ==", 2024-04-20T11:36:04.931 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:36:04.931 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow *", 2024-04-20T11:36:04.931 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile mgr", 2024-04-20T11:36:04.931 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:36:04.931 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:36:04.931 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:36:04.931 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-04-20T11:36:04.931 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-04-20T11:36:04.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.919+0000 7f5f5b200640 1 -- 172.21.15.152:0/4021377544 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f5f3406cf50 msgr2=0x7f5f3406f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:04.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.919+0000 7f5f5b200640 1 --2- 172.21.15.152:0/4021377544 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f5f3406cf50 0x7f5f3406f410 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f5f44006fd0 tx=0x7f5f44008040 comp rx=0 tx=0).stop 2024-04-20T11:36:04.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.919+0000 7f5f5b200640 1 -- 172.21.15.152:0/4021377544 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5f54144d00 msgr2=0x7f5f5415d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:04.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.919+0000 7f5f5b200640 1 --2- 172.21.15.152:0/4021377544 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5f54144d00 0x7f5f5415d490 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7f5f4c002230 tx=0x7f5f4c009750 comp rx=0 tx=0).stop 2024-04-20T11:36:04.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.919+0000 7f5f5b200640 1 -- 172.21.15.152:0/4021377544 shutdown_connections 2024-04-20T11:36:04.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.919+0000 7f5f5b200640 1 --2- 172.21.15.152:0/4021377544 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f5f3406cf50 0x7f5f3406f410 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:04.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.919+0000 7f5f5b200640 1 --2- 172.21.15.152:0/4021377544 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5f54144d00 0x7f5f5415d490 unknown :-1 s=CLOSED pgs=1941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:04.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.919+0000 7f5f5b200640 1 --2- 172.21.15.152:0/4021377544 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5f54144880 0x7f5f5415cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:04.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.919+0000 7f5f5b200640 1 --2- 172.21.15.152:0/4021377544 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f38004d40 0x7f5f54140800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:04.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.919+0000 7f5f5b200640 1 -- 172.21.15.152:0/4021377544 >> 172.21.15.152:0/4021377544 conn(0x7f5f54067cf0 msgr2=0x7f5f540cf250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:04.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.919+0000 7f5f5b200640 1 -- 172.21.15.152:0/4021377544 shutdown_connections 2024-04-20T11:36:04.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:04.919+0000 7f5f5b200640 1 -- 172.21.15.152:0/4021377544 wait complete. 2024-04-20T11:36:04.940 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:36:05.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.142+0000 7fdd06400640 1 Processor -- start 2024-04-20T11:36:05.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.142+0000 7fdd06400640 1 -- start start 2024-04-20T11:36:05.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.143+0000 7fdd06400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd00145f60 0x7fdd0006ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:05.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.144+0000 7fdd06400640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fdd000726f0 0x7fdd000cd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:05.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.144+0000 7fdd06400640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd00146340 con 0x7fdd000cd0d0 2024-04-20T11:36:05.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.144+0000 7fdd06400640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fdd0006e9f0 con 0x7fdd000726f0 2024-04-20T11:36:05.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.144+0000 7fdd06400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fdd0006f3c0 con 0x7fdd00145f60 2024-04-20T11:36:05.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.144+0000 7fdcf7e00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fdd000cd0d0 0x7fdd00071fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:50068/0 (socket says 172.21.15.152:50068) 2024-04-20T11:36:05.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.144+0000 7fdcf7e00640 1 -- 172.21.15.152:0/3251675311 learned_addr learned my addr 172.21.15.152:0/3251675311 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:36:05.146 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.144+0000 7fdcff400640 1 --2- 172.21.15.152:0/3251675311 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd00145f60 0x7fdd0006ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:05.146 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.144+0000 7fdcfea00640 1 -- 172.21.15.152:0/3251675311 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1464620977 0 0) 0x7fdd00146340 con 0x7fdd000cd0d0 2024-04-20T11:36:05.146 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.144+0000 7fdcfea00640 1 -- 172.21.15.152:0/3251675311 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdce4003520 con 0x7fdd000cd0d0 2024-04-20T11:36:05.146 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.144+0000 7fdcffe00640 1 --2- 172.21.15.152:0/3251675311 >> v2:172.21.15.175:3300/0 conn(0x7fdd000726f0 0x7fdd000cd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:05.146 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.145+0000 7fdcfea00640 1 -- 172.21.15.152:0/3251675311 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1203980910 0 0) 0x7fdce4003520 con 0x7fdd000cd0d0 2024-04-20T11:36:05.146 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.145+0000 7fdcfea00640 1 -- 172.21.15.152:0/3251675311 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd00145f60 msgr2=0x7fdd0006ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:05.146 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.145+0000 7fdcfea00640 1 --2- 172.21.15.152:0/3251675311 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd00145f60 0x7fdd0006ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:05.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.145+0000 7fdcfea00640 1 -- 172.21.15.152:0/3251675311 >> v2:172.21.15.175:3300/0 conn(0x7fdd000726f0 msgr2=0x7fdd000cd550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:05.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.145+0000 7fdcfea00640 1 --2- 172.21.15.152:0/3251675311 >> v2:172.21.15.175:3300/0 conn(0x7fdd000726f0 0x7fdd000cd550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:05.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.145+0000 7fdcfea00640 1 -- 172.21.15.152:0/3251675311 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd00072d50 con 0x7fdd000cd0d0 2024-04-20T11:36:05.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.145+0000 7fdcfea00640 1 -- 172.21.15.152:0/3251675311 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7fdce8001670 con 0x7fdd000cd0d0 2024-04-20T11:36:05.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.145+0000 7fdcfea00640 1 -- 172.21.15.152:0/3251675311 >> v1:172.21.15.152:6789/0 conn(0x7fdd000cd0d0 legacy=0x7fdd00071fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:05.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.145+0000 7fdcfea00640 1 --2- 172.21.15.152:0/3251675311 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdce4003a20 0x7fdce4003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:05.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.145+0000 7fdcfea00640 1 --2- 172.21.15.152:0/3251675311 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd00145f60 0x7fdce4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:05.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.145+0000 7fdcfea00640 1 --2- 172.21.15.152:0/3251675311 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdce4004d40 0x7fdce40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:05.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.145+0000 7fdcfea00640 1 -- 172.21.15.152:0/3251675311 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fdd0006e9f0 con 0x7fdce4003a20 2024-04-20T11:36:05.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.145+0000 7fdcfea00640 1 -- 172.21.15.152:0/3251675311 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdd0006f3c0 con 0x7fdce4004d40 2024-04-20T11:36:05.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.145+0000 7fdcfea00640 1 -- 172.21.15.152:0/3251675311 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fdce400baa0 con 0x7fdd00145f60 2024-04-20T11:36:05.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.145+0000 7fdcff400640 1 --2- 172.21.15.152:0/3251675311 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdce4003a20 0x7fdce4003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:05.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.146+0000 7fdcf7e00640 1 --2- 172.21.15.152:0/3251675311 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd00145f60 0x7fdce4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:05.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.146+0000 7fdcff400640 1 -- 172.21.15.152:0/3251675311 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd00145f60 msgr2=0x7fdce4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:05.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.146+0000 7fdcff400640 1 --2- 172.21.15.152:0/3251675311 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd00145f60 0x7fdce4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:05.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.146+0000 7fdcff400640 1 -- 172.21.15.152:0/3251675311 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdce4004d40 msgr2=0x7fdce40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:05.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.146+0000 7fdcff400640 1 --2- 172.21.15.152:0/3251675311 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdce4004d40 0x7fdce40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:05.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.146+0000 7fdcff400640 1 -- 172.21.15.152:0/3251675311 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdce4003520 con 0x7fdce4003a20 2024-04-20T11:36:05.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.146+0000 7fdcf7e00640 1 --2- 172.21.15.152:0/3251675311 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd00145f60 0x7fdce4004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:36:05.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.147+0000 7fdcff400640 1 --2- 172.21.15.152:0/3251675311 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdce4003a20 0x7fdce4003e60 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7fdcf0009ac0 tx=0x7fdce4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc185dc3770986da server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:05.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.147+0000 7fdcfea00640 1 -- 172.21.15.152:0/3251675311 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcf0039070 con 0x7fdce4003a20 2024-04-20T11:36:05.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.147+0000 7fdcfea00640 1 -- 172.21.15.152:0/3251675311 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcf002c6b0 con 0x7fdce4003a20 2024-04-20T11:36:05.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.147+0000 7fdcfea00640 1 -- 172.21.15.152:0/3251675311 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcf0009690 con 0x7fdce4003a20 2024-04-20T11:36:05.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.148+0000 7fdd06400640 1 -- 172.21.15.152:0/3251675311 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdce4003a20 msgr2=0x7fdce4003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:05.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.148+0000 7fdd06400640 1 --2- 172.21.15.152:0/3251675311 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdce4003a20 0x7fdce4003e60 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7fdcf0009ac0 tx=0x7fdce4005830 comp rx=0 tx=0).stop 2024-04-20T11:36:05.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.148+0000 7fdcffe00640 1 -- 172.21.15.152:0/3251675311 reap_dead start 2024-04-20T11:36:05.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.148+0000 7fdd06400640 1 -- 172.21.15.152:0/3251675311 shutdown_connections 2024-04-20T11:36:05.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.148+0000 7fdd06400640 1 -- 172.21.15.152:0/3251675311 >> 172.21.15.152:0/3251675311 conn(0x7fdd00067cf0 msgr2=0x7fdd0006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:05.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.148+0000 7fdd06400640 1 -- 172.21.15.152:0/3251675311 shutdown_connections 2024-04-20T11:36:05.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.148+0000 7fdd06400640 1 -- 172.21.15.152:0/3251675311 wait complete. 2024-04-20T11:36:05.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.150+0000 7fdd06400640 1 Processor -- start 2024-04-20T11:36:05.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.150+0000 7fdd06400640 1 -- start start 2024-04-20T11:36:05.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.151+0000 7fdd06400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdce4003a20 0x7fdd000c6b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:05.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.151+0000 7fdd06400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdd001424f0 0x7fdd0013b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:05.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.151+0000 7fdd06400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd0013b5e0 0x7fdd0013fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:05.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.151+0000 7fdd06400640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fdd0006e680 con 0x7fdd001424f0 2024-04-20T11:36:05.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.151+0000 7fdd06400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdd0006e3a0 con 0x7fdd0013b5e0 2024-04-20T11:36:05.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.151+0000 7fdd06400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fdd0006ee80 con 0x7fdce4003a20 2024-04-20T11:36:05.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.151+0000 7fdcf7e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdd001424f0 0x7fdd0013b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:05.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.151+0000 7fdcf7e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdd001424f0 0x7fdd0013b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36520/0 (socket says 172.21.15.152:36520) 2024-04-20T11:36:05.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.151+0000 7fdcf7e00640 1 -- 172.21.15.152:0/1616094444 learned_addr learned my addr 172.21.15.152:0/1616094444 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:05.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.151+0000 7fdcff400640 1 --2- 172.21.15.152:0/1616094444 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdce4003a20 0x7fdd000c6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:05.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.151+0000 7fdcf7e00640 1 -- 172.21.15.152:0/1616094444 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdce4003a20 msgr2=0x7fdd000c6b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:05.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.151+0000 7fdcf7e00640 1 --2- 172.21.15.152:0/1616094444 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdce4003a20 0x7fdd000c6b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:05.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.151+0000 7fdcf7e00640 1 -- 172.21.15.152:0/1616094444 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd0013b5e0 msgr2=0x7fdd0013fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:05.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.151+0000 7fdcf7e00640 1 --2- 172.21.15.152:0/1616094444 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd0013b5e0 0x7fdd0013fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:05.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.151+0000 7fdcf7e00640 1 -- 172.21.15.152:0/1616094444 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd00143010 con 0x7fdd001424f0 2024-04-20T11:36:05.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.152+0000 7fdcf7e00640 1 --2- 172.21.15.152:0/1616094444 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdd001424f0 0x7fdd0013b0a0 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7fdce800d860 tx=0x7fdce800dd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:05.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.152+0000 7fdcf7400640 1 -- 172.21.15.152:0/1616094444 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce8019070 con 0x7fdd001424f0 2024-04-20T11:36:05.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.152+0000 7fdcf7400640 1 -- 172.21.15.152:0/1616094444 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdce80074e0 con 0x7fdd001424f0 2024-04-20T11:36:05.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.152+0000 7fdcf7400640 1 -- 172.21.15.152:0/1616094444 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce8004a00 con 0x7fdd001424f0 2024-04-20T11:36:05.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.152+0000 7fdd06400640 1 -- 172.21.15.152:0/1616094444 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd001432f0 con 0x7fdd001424f0 2024-04-20T11:36:05.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.152+0000 7fdd06400640 1 -- 172.21.15.152:0/1616094444 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd001438b0 con 0x7fdd001424f0 2024-04-20T11:36:05.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.153+0000 7fdd06400640 1 -- 172.21.15.152:0/1616094444 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd000689f0 con 0x7fdd001424f0 2024-04-20T11:36:05.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.154+0000 7fdcf7400640 1 -- 172.21.15.152:0/1616094444 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 22) ==== 91601+0+0 (secure 0 0 0) 0x7fdce801c070 con 0x7fdd001424f0 2024-04-20T11:36:05.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.155+0000 7fdcf7400640 1 --2- 172.21.15.152:0/1616094444 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fdcdc06cf20 0x7fdcdc06f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:05.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.155+0000 7fdcf7400640 1 -- 172.21.15.152:0/1616094444 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 4857+0+0 (secure 0 0 0) 0x7fdce808bf80 con 0x7fdd001424f0 2024-04-20T11:36:05.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.156+0000 7fdcff400640 1 --2- 172.21.15.152:0/1616094444 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fdcdc06cf20 0x7fdcdc06f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:05.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.157+0000 7fdcff400640 1 --2- 172.21.15.152:0/1616094444 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fdcdc06cf20 0x7fdcdc06f3e0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fdcf0036000 tx=0x7fdcf0030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:05.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.159+0000 7fdcf7400640 1 -- 172.21.15.152:0/1616094444 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fdce8059d00 con 0x7fdd001424f0 2024-04-20T11:36:05.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.364+0000 7fdd06400640 1 -- 172.21.15.152:0/1616094444 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdd0006f970 con 0x7fdd001424f0 2024-04-20T11:36:05.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.366+0000 7fdcf7400640 1 -- 172.21.15.152:0/1616094444 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v138) ==== 74+0+12961 (secure 0 0 0) 0x7fdce8059600 con 0x7fdd001424f0 2024-04-20T11:36:05.366 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:36:05.366 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":138,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:36:04.421909+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:36:05.369 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.369+0000 7fdd06400640 1 -- 172.21.15.152:0/1616094444 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fdcdc06cf20 msgr2=0x7fdcdc06f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:05.369 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.369+0000 7fdd06400640 1 --2- 172.21.15.152:0/1616094444 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fdcdc06cf20 0x7fdcdc06f3e0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fdcf0036000 tx=0x7fdcf0030000 comp rx=0 tx=0).stop 2024-04-20T11:36:05.369 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.369+0000 7fdd06400640 1 -- 172.21.15.152:0/1616094444 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdd001424f0 msgr2=0x7fdd0013b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:05.369 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.369+0000 7fdd06400640 1 --2- 172.21.15.152:0/1616094444 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdd001424f0 0x7fdd0013b0a0 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7fdce800d860 tx=0x7fdce800dd20 comp rx=0 tx=0).stop 2024-04-20T11:36:05.369 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.369+0000 7fdd06400640 1 -- 172.21.15.152:0/1616094444 shutdown_connections 2024-04-20T11:36:05.369 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.369+0000 7fdd06400640 1 --2- 172.21.15.152:0/1616094444 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fdcdc06cf20 0x7fdcdc06f3e0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:05.369 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.369+0000 7fdd06400640 1 --2- 172.21.15.152:0/1616094444 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd0013b5e0 0x7fdd0013fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:05.369 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.369+0000 7fdd06400640 1 --2- 172.21.15.152:0/1616094444 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdd001424f0 0x7fdd0013b0a0 unknown :-1 s=CLOSED pgs=1944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:05.370 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.369+0000 7fdd06400640 1 --2- 172.21.15.152:0/1616094444 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdce4003a20 0x7fdd000c6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:05.370 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.369+0000 7fdd06400640 1 -- 172.21.15.152:0/1616094444 >> 172.21.15.152:0/1616094444 conn(0x7fdd00067cf0 msgr2=0x7fdd00140d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:05.371 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.369+0000 7fdd06400640 1 -- 172.21.15.152:0/1616094444 shutdown_connections 2024-04-20T11:36:05.371 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.369+0000 7fdd06400640 1 -- 172.21.15.152:0/1616094444 wait complete. 2024-04-20T11:36:05.388 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T11:26:47.246571+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T11:36:05.388 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T11:36:05.580 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.578+0000 7f6e10400640 1 Processor -- start 2024-04-20T11:36:05.580 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.578+0000 7f6e10400640 1 -- start start 2024-04-20T11:36:05.585 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.579+0000 7f6e10400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f6e08146000 0x7f6e080b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:05.585 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.579+0000 7f6e10400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e080b2170 0x7f6e080b25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:05.585 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.580+0000 7f6e10400640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e081463e0 con 0x7f6e080cd1b0 2024-04-20T11:36:05.585 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.580+0000 7f6e10400640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f6e080b4750 con 0x7f6e08146000 2024-04-20T11:36:05.585 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.580+0000 7f6e10400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6e080b3e20 con 0x7f6e080b2170 2024-04-20T11:36:05.585 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.580+0000 7f6e0da00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f6e08146000 0x7f6e080b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:05.586 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.580+0000 7f6e0da00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f6e08146000 0x7f6e080b4be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36532/0 (socket says 172.21.15.152:36532) 2024-04-20T11:36:05.586 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.580+0000 7f6e0da00640 1 -- 172.21.15.152:0/1805365349 learned_addr learned my addr 172.21.15.152:0/1805365349 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:05.586 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.580+0000 7f6e0d000640 1 --2- 172.21.15.152:0/1805365349 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e080b2170 0x7f6e080b25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:05.586 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.580+0000 7f6e0da00640 1 -- 172.21.15.152:0/1805365349 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e080b2170 msgr2=0x7f6e080b25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:05.586 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.580+0000 7f6e0da00640 1 --2- 172.21.15.152:0/1805365349 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e080b2170 0x7f6e080b25f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:05.586 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.580+0000 7f6e0da00640 1 -- 172.21.15.152:0/1805365349 >> v1:172.21.15.175:6789/0 conn(0x7f6e080cd1b0 legacy=0x7f6e080cd5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:05.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.580+0000 7f6e0da00640 1 -- 172.21.15.152:0/1805365349 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e080b4100 con 0x7f6e08146000 2024-04-20T11:36:05.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.582+0000 7f6e0da00640 1 --2- 172.21.15.152:0/1805365349 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6e08146000 0x7f6e080b4be0 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc009870 tx=0x7f6dfc02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef67ce06d272eabb server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:05.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.582+0000 7f6e07e00640 1 -- 172.21.15.152:0/1805365349 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dfc0317c0 con 0x7f6e08146000 2024-04-20T11:36:05.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.582+0000 7f6e07e00640 1 -- 172.21.15.152:0/1805365349 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6dfc037070 con 0x7f6e08146000 2024-04-20T11:36:05.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.582+0000 7f6e07e00640 1 -- 172.21.15.152:0/1805365349 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dfc031b80 con 0x7f6e08146000 2024-04-20T11:36:05.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.583+0000 7f6e10400640 1 -- 172.21.15.152:0/1805365349 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6e08146000 msgr2=0x7f6e080b4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:05.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.583+0000 7f6e10400640 1 --2- 172.21.15.152:0/1805365349 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6e08146000 0x7f6e080b4be0 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc009870 tx=0x7f6dfc02b860 comp rx=0 tx=0).stop 2024-04-20T11:36:05.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.583+0000 7f6e10400640 1 -- 172.21.15.152:0/1805365349 shutdown_connections 2024-04-20T11:36:05.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.583+0000 7f6e10400640 1 --2- 172.21.15.152:0/1805365349 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e080b2170 0x7f6e080b25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:05.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.583+0000 7f6e10400640 1 --2- 172.21.15.152:0/1805365349 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6e08146000 0x7f6e080b4be0 unknown :-1 s=CLOSED pgs=1945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:05.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.583+0000 7f6e10400640 1 -- 172.21.15.152:0/1805365349 >> 172.21.15.152:0/1805365349 conn(0x7f6e0806b980 msgr2=0x7f6e0806bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:05.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.583+0000 7f6e10400640 1 -- 172.21.15.152:0/1805365349 shutdown_connections 2024-04-20T11:36:05.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.583+0000 7f6e10400640 1 -- 172.21.15.152:0/1805365349 wait complete. 2024-04-20T11:36:05.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.585+0000 7f6e10400640 1 Processor -- start 2024-04-20T11:36:05.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.585+0000 7f6e10400640 1 -- start start 2024-04-20T11:36:05.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.586+0000 7f6e10400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e080b2170 0x7f6e080c5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:05.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.586+0000 7f6e10400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6e080cd1b0 0x7f6e08140e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:05.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.586+0000 7f6e10400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e08146000 0x7f6e0815c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:05.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.586+0000 7f6e10400640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f6e0815ce70 con 0x7f6e080cd1b0 2024-04-20T11:36:05.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.586+0000 7f6e10400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6e0815ccf0 con 0x7f6e08146000 2024-04-20T11:36:05.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.586+0000 7f6e10400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6e0815cff0 con 0x7f6e080b2170 2024-04-20T11:36:05.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.586+0000 7f6e0d000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6e080cd1b0 0x7f6e08140e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:05.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.586+0000 7f6e0d000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6e080cd1b0 0x7f6e08140e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36540/0 (socket says 172.21.15.152:36540) 2024-04-20T11:36:05.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.586+0000 7f6e0d000640 1 -- 172.21.15.152:0/4174874951 learned_addr learned my addr 172.21.15.152:0/4174874951 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:05.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.586+0000 7f6e0da00640 1 --2- 172.21.15.152:0/4174874951 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e080b2170 0x7f6e080c5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:05.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.587+0000 7f6e0d000640 1 -- 172.21.15.152:0/4174874951 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e080b2170 msgr2=0x7f6e080c5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:05.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.587+0000 7f6e0d000640 1 --2- 172.21.15.152:0/4174874951 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e080b2170 0x7f6e080c5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:05.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.587+0000 7f6e0e400640 1 --2- 172.21.15.152:0/4174874951 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e08146000 0x7f6e0815c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:05.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.587+0000 7f6e0d000640 1 -- 172.21.15.152:0/4174874951 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e08146000 msgr2=0x7f6e0815c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:05.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.587+0000 7f6e0d000640 1 --2- 172.21.15.152:0/4174874951 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e08146000 0x7f6e0815c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:05.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.587+0000 7f6e0d000640 1 -- 172.21.15.152:0/4174874951 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e081413a0 con 0x7f6e080cd1b0 2024-04-20T11:36:05.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.587+0000 7f6e0da00640 1 --2- 172.21.15.152:0/4174874951 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e080b2170 0x7f6e080c5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:36:05.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.587+0000 7f6e0e400640 1 --2- 172.21.15.152:0/4174874951 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e08146000 0x7f6e0815c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:36:05.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.587+0000 7f6e0d000640 1 --2- 172.21.15.152:0/4174874951 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6e080cd1b0 0x7f6e08140e60 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7f6df400c7f0 tx=0x7f6df400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:05.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.587+0000 7f6e05600640 1 -- 172.21.15.152:0/4174874951 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df4015070 con 0x7f6e080cd1b0 2024-04-20T11:36:05.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.587+0000 7f6e05600640 1 -- 172.21.15.152:0/4174874951 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6df4004240 con 0x7f6e080cd1b0 2024-04-20T11:36:05.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.587+0000 7f6e05600640 1 -- 172.21.15.152:0/4174874951 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df4007620 con 0x7f6e080cd1b0 2024-04-20T11:36:05.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.587+0000 7f6e10400640 1 -- 172.21.15.152:0/4174874951 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e081e5270 con 0x7f6e080cd1b0 2024-04-20T11:36:05.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.587+0000 7f6e10400640 1 -- 172.21.15.152:0/4174874951 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e081e5830 con 0x7f6e080cd1b0 2024-04-20T11:36:05.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.588+0000 7f6e10400640 1 -- 172.21.15.152:0/4174874951 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e080689f0 con 0x7f6e080cd1b0 2024-04-20T11:36:05.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.589+0000 7f6e05600640 1 -- 172.21.15.152:0/4174874951 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 22) ==== 91601+0+0 (secure 0 0 0) 0x7f6df40079d0 con 0x7f6e080cd1b0 2024-04-20T11:36:05.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.590+0000 7f6e05600640 1 --2- 172.21.15.152:0/4174874951 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f6de806cf50 0x7f6de806f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:05.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.590+0000 7f6e05600640 1 -- 172.21.15.152:0/4174874951 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 4857+0+0 (secure 0 0 0) 0x7f6df408c4c0 con 0x7f6e080cd1b0 2024-04-20T11:36:05.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.591+0000 7f6e0da00640 1 --2- 172.21.15.152:0/4174874951 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f6de806cf50 0x7f6de806f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:05.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.592+0000 7f6e0da00640 1 --2- 172.21.15.152:0/4174874951 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f6de806cf50 0x7f6de806f410 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc036000 tx=0x7f6dfc008030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:05.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.595+0000 7f6e05600640 1 -- 172.21.15.152:0/4174874951 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f6df405a240 con 0x7f6e080cd1b0 2024-04-20T11:36:05.792 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.791+0000 7f6e10400640 1 -- 172.21.15.152:0/4174874951 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6e0815e530 con 0x7f6e080cd1b0 2024-04-20T11:36:05.792 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.792+0000 7f6e05600640 1 -- 172.21.15.152:0/4174874951 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v138) ==== 93+0+10 (secure 0 0 0) 0x7f6df4059b40 con 0x7f6e080cd1b0 2024-04-20T11:36:05.792 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 1 2024-04-20T11:36:05.795 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.795+0000 7f6e10400640 1 -- 172.21.15.152:0/4174874951 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f6de806cf50 msgr2=0x7f6de806f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:05.795 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.795+0000 7f6e10400640 1 --2- 172.21.15.152:0/4174874951 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f6de806cf50 0x7f6de806f410 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc036000 tx=0x7f6dfc008030 comp rx=0 tx=0).stop 2024-04-20T11:36:05.795 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.795+0000 7f6e10400640 1 -- 172.21.15.152:0/4174874951 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6e080cd1b0 msgr2=0x7f6e08140e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:05.795 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.795+0000 7f6e10400640 1 --2- 172.21.15.152:0/4174874951 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6e080cd1b0 0x7f6e08140e60 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7f6df400c7f0 tx=0x7f6df400ccb0 comp rx=0 tx=0).stop 2024-04-20T11:36:05.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.795+0000 7f6e10400640 1 -- 172.21.15.152:0/4174874951 shutdown_connections 2024-04-20T11:36:05.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.795+0000 7f6e10400640 1 --2- 172.21.15.152:0/4174874951 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f6de806cf50 0x7f6de806f410 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:05.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.795+0000 7f6e10400640 1 --2- 172.21.15.152:0/4174874951 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e08146000 0x7f6e0815c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:05.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.795+0000 7f6e10400640 1 --2- 172.21.15.152:0/4174874951 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6e080cd1b0 0x7f6e08140e60 unknown :-1 s=CLOSED pgs=1946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:05.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.795+0000 7f6e10400640 1 --2- 172.21.15.152:0/4174874951 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e080b2170 0x7f6e080c5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:05.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.795+0000 7f6e10400640 1 -- 172.21.15.152:0/4174874951 >> 172.21.15.152:0/4174874951 conn(0x7f6e0806b980 msgr2=0x7f6e08071db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:05.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.795+0000 7f6e10400640 1 -- 172.21.15.152:0/4174874951 shutdown_connections 2024-04-20T11:36:05.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:05.795+0000 7f6e10400640 1 -- 172.21.15.152:0/4174874951 wait complete. 2024-04-20T11:36:05.814 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-20T11:36:06.013 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.013+0000 7f46f6200640 1 Processor -- start 2024-04-20T11:36:06.013 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.013+0000 7f46f6200640 1 -- start start 2024-04-20T11:36:06.014 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.015+0000 7f46f6200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f46f015d1e0 0x7f46f0160b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:06.014 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.015+0000 7f46f6200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f46f0071cd0 0x7f46f0072160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:06.015 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.015+0000 7f46f6200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46f00ce8c0 con 0x7f46f015d5c0 2024-04-20T11:36:06.015 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.015+0000 7f46f6200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f46f00726a0 con 0x7f46f0071cd0 2024-04-20T11:36:06.015 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.015+0000 7f46f6200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f46f01600b0 con 0x7f46f015d1e0 2024-04-20T11:36:06.015 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.015+0000 7f46f5800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f46f0071cd0 0x7f46f0072160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:06.015 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.015+0000 7f46f5800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f46f0071cd0 0x7f46f0072160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35668/0 (socket says 172.21.15.152:35668) 2024-04-20T11:36:06.015 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.015+0000 7f46f5800640 1 -- 172.21.15.152:0/1771461706 learned_addr learned my addr 172.21.15.152:0/1771461706 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:06.016 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.015+0000 7f46f4e00640 1 --2- 172.21.15.152:0/1771461706 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f46f015d1e0 0x7f46f0160b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:06.016 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.016+0000 7f46f5800640 1 -- 172.21.15.152:0/1771461706 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f46f015d1e0 msgr2=0x7f46f0160b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:06.016 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.016+0000 7f46f5800640 1 --2- 172.21.15.152:0/1771461706 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f46f015d1e0 0x7f46f0160b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:06.016 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.016+0000 7f46f5800640 1 -- 172.21.15.152:0/1771461706 >> v1:172.21.15.175:6789/0 conn(0x7f46f015d5c0 legacy=0x7f46f00cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:06.016 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.016+0000 7f46f5800640 1 -- 172.21.15.152:0/1771461706 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46f0160390 con 0x7f46f0071cd0 2024-04-20T11:36:06.018 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.017+0000 7f46f5800640 1 --2- 172.21.15.152:0/1771461706 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f46f0071cd0 0x7f46f0072160 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7f46e8009e50 tx=0x7f46e802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=284a0a6708ba6366 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:06.018 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.018+0000 7f46ef400640 1 -- 172.21.15.152:0/1771461706 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e802ca70 con 0x7f46f0071cd0 2024-04-20T11:36:06.018 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.018+0000 7f46ef400640 1 -- 172.21.15.152:0/1771461706 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46e802cc10 con 0x7f46f0071cd0 2024-04-20T11:36:06.018 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.018+0000 7f46ef400640 1 -- 172.21.15.152:0/1771461706 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e80343f0 con 0x7f46f0071cd0 2024-04-20T11:36:06.018 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.018+0000 7f46f6200640 1 -- 172.21.15.152:0/1771461706 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f46f0071cd0 msgr2=0x7f46f0072160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:06.018 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.018+0000 7f46f6200640 1 --2- 172.21.15.152:0/1771461706 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f46f0071cd0 0x7f46f0072160 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7f46e8009e50 tx=0x7f46e802bed0 comp rx=0 tx=0).stop 2024-04-20T11:36:06.018 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.018+0000 7f46f6200640 1 -- 172.21.15.152:0/1771461706 shutdown_connections 2024-04-20T11:36:06.019 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.018+0000 7f46f6200640 1 --2- 172.21.15.152:0/1771461706 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f46f0071cd0 0x7f46f0072160 unknown :-1 s=CLOSED pgs=1947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:06.019 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.018+0000 7f46f6200640 1 --2- 172.21.15.152:0/1771461706 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f46f015d1e0 0x7f46f0160b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:06.019 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.018+0000 7f46f6200640 1 -- 172.21.15.152:0/1771461706 >> 172.21.15.152:0/1771461706 conn(0x7f46f0067ae0 msgr2=0x7f46f006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:06.019 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.018+0000 7f46f6200640 1 -- 172.21.15.152:0/1771461706 shutdown_connections 2024-04-20T11:36:06.019 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.018+0000 7f46f6200640 1 -- 172.21.15.152:0/1771461706 wait complete. 2024-04-20T11:36:06.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.020+0000 7f46f6200640 1 Processor -- start 2024-04-20T11:36:06.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.021+0000 7f46f6200640 1 -- start start 2024-04-20T11:36:06.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.022+0000 7f46f6200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f46f0071cd0 0x7f46f00ae2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:06.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.022+0000 7f46f6200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f46f015d1e0 0x7f46f01f4300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:06.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.022+0000 7f46f6200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f46f015d5c0 0x7f46f01f4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:06.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.022+0000 7f46f6200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f46f00ccd70 con 0x7f46f0071cd0 2024-04-20T11:36:06.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.022+0000 7f46f6200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f46f00ceba0 con 0x7f46f015d5c0 2024-04-20T11:36:06.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.022+0000 7f46f6200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f46f00ccef0 con 0x7f46f015d1e0 2024-04-20T11:36:06.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.022+0000 7f46f4e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f46f0071cd0 0x7f46f00ae2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:06.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.022+0000 7f46f4e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f46f0071cd0 0x7f46f00ae2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35672/0 (socket says 172.21.15.152:35672) 2024-04-20T11:36:06.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.022+0000 7f46f4e00640 1 -- 172.21.15.152:0/3306735523 learned_addr learned my addr 172.21.15.152:0/3306735523 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:06.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.023+0000 7f46efe00640 1 --2- 172.21.15.152:0/3306735523 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f46f015d1e0 0x7f46f01f4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:06.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.023+0000 7f46f4e00640 1 -- 172.21.15.152:0/3306735523 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f46f015d1e0 msgr2=0x7f46f01f4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:06.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.023+0000 7f46f4e00640 1 --2- 172.21.15.152:0/3306735523 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f46f015d1e0 0x7f46f01f4300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:06.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.023+0000 7f46f4e00640 1 -- 172.21.15.152:0/3306735523 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f46f015d5c0 msgr2=0x7f46f01f4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:06.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.023+0000 7f46f5800640 1 --2- 172.21.15.152:0/3306735523 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f46f015d5c0 0x7f46f01f4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:06.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.023+0000 7f46f4e00640 1 --2- 172.21.15.152:0/3306735523 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f46f015d5c0 0x7f46f01f4840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:06.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.023+0000 7f46f4e00640 1 -- 172.21.15.152:0/3306735523 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46f00aeb10 con 0x7f46f0071cd0 2024-04-20T11:36:06.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.023+0000 7f46f4e00640 1 --2- 172.21.15.152:0/3306735523 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f46f0071cd0 0x7f46f00ae2c0 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7f46e400d830 tx=0x7f46e400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:06.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.023+0000 7f46ecc00640 1 -- 172.21.15.152:0/3306735523 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e400b6c0 con 0x7f46f0071cd0 2024-04-20T11:36:06.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.023+0000 7f46ecc00640 1 -- 172.21.15.152:0/3306735523 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46e4019070 con 0x7f46f0071cd0 2024-04-20T11:36:06.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.023+0000 7f46f6200640 1 -- 172.21.15.152:0/3306735523 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46f00aedf0 con 0x7f46f0071cd0 2024-04-20T11:36:06.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.023+0000 7f46ecc00640 1 -- 172.21.15.152:0/3306735523 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e400ba80 con 0x7f46f0071cd0 2024-04-20T11:36:06.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.023+0000 7f46f6200640 1 -- 172.21.15.152:0/3306735523 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46f00ae800 con 0x7f46f0071cd0 2024-04-20T11:36:06.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.024+0000 7f46f6200640 1 -- 172.21.15.152:0/3306735523 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46f0075e60 con 0x7f46f0071cd0 2024-04-20T11:36:06.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.026+0000 7f46ecc00640 1 -- 172.21.15.152:0/3306735523 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 22) ==== 91601+0+0 (secure 0 0 0) 0x7f46e401e460 con 0x7f46f0071cd0 2024-04-20T11:36:06.026 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.027+0000 7f46ecc00640 1 --2- 172.21.15.152:0/3306735523 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f46d006cf20 0x7f46d006f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:06.026 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.027+0000 7f46ecc00640 1 -- 172.21.15.152:0/3306735523 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 4857+0+0 (secure 0 0 0) 0x7f46e408cad0 con 0x7f46f0071cd0 2024-04-20T11:36:06.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.027+0000 7f46efe00640 1 --2- 172.21.15.152:0/3306735523 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f46d006cf20 0x7f46d006f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:06.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.029+0000 7f46efe00640 1 --2- 172.21.15.152:0/3306735523 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f46d006cf20 0x7f46d006f3e0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f46f015fba0 tx=0x7f46dc008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:06.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.031+0000 7f46ecc00640 1 -- 172.21.15.152:0/3306735523 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f46e405a8d0 con 0x7f46f0071cd0 2024-04-20T11:36:06.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.234+0000 7f46f6200640 1 -- 172.21.15.152:0/3306735523 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f46f0072c90 con 0x7f46f0071cd0 2024-04-20T11:36:06.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.235+0000 7f46ecc00640 1 -- 172.21.15.152:0/3306735523 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v138) ==== 84+0+65 (secure 0 0 0) 0x7f46e405a1d0 con 0x7f46f0071cd0 2024-04-20T11:36:06.236 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:36:06.236 INFO:teuthology.orchestra.run.smithi152.stdout:[ 2024-04-20T11:36:06.236 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:36:06.237 INFO:teuthology.orchestra.run.smithi152.stdout: "poolnum": 1, 2024-04-20T11:36:06.237 INFO:teuthology.orchestra.run.smithi152.stdout: "poolname": ".mgr" 2024-04-20T11:36:06.237 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:36:06.237 INFO:teuthology.orchestra.run.smithi152.stdout:] 2024-04-20T11:36:06.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.238+0000 7f46f6200640 1 -- 172.21.15.152:0/3306735523 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f46d006cf20 msgr2=0x7f46d006f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:06.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.238+0000 7f46f6200640 1 --2- 172.21.15.152:0/3306735523 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f46d006cf20 0x7f46d006f3e0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f46f015fba0 tx=0x7f46dc008040 comp rx=0 tx=0).stop 2024-04-20T11:36:06.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.238+0000 7f46f6200640 1 -- 172.21.15.152:0/3306735523 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f46f0071cd0 msgr2=0x7f46f00ae2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:06.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.238+0000 7f46f6200640 1 --2- 172.21.15.152:0/3306735523 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f46f0071cd0 0x7f46f00ae2c0 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7f46e400d830 tx=0x7f46e400dcf0 comp rx=0 tx=0).stop 2024-04-20T11:36:06.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.239+0000 7f46f6200640 1 -- 172.21.15.152:0/3306735523 shutdown_connections 2024-04-20T11:36:06.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.239+0000 7f46f6200640 1 --2- 172.21.15.152:0/3306735523 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f46d006cf20 0x7f46d006f3e0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:06.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.239+0000 7f46f6200640 1 --2- 172.21.15.152:0/3306735523 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f46f015d5c0 0x7f46f01f4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:06.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.239+0000 7f46f6200640 1 --2- 172.21.15.152:0/3306735523 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f46f015d1e0 0x7f46f01f4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:06.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.239+0000 7f46f6200640 1 --2- 172.21.15.152:0/3306735523 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f46f0071cd0 0x7f46f00ae2c0 unknown :-1 s=CLOSED pgs=1948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:06.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.239+0000 7f46f6200640 1 -- 172.21.15.152:0/3306735523 >> 172.21.15.152:0/3306735523 conn(0x7f46f0067ae0 msgr2=0x7f46f00b2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:06.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.239+0000 7f46f6200640 1 -- 172.21.15.152:0/3306735523 shutdown_connections 2024-04-20T11:36:06.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.239+0000 7f46f6200640 1 -- 172.21.15.152:0/3306735523 wait complete. 2024-04-20T11:36:06.258 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-20T11:36:06.259 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-20T11:36:06.469 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.469+0000 7fe9f2200640 1 Processor -- start 2024-04-20T11:36:06.469 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.469+0000 7fe9f2200640 1 -- start start 2024-04-20T11:36:06.484 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.471+0000 7fe9f2200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe9ec1569d0 0x7fe9ec0ad760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:06.484 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.471+0000 7fe9f2200640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9ec15d8d0 con 0x7fe9ec06e970 2024-04-20T11:36:06.485 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.471+0000 7fe9f2200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9ec15d310 con 0x7fe9ec06e510 2024-04-20T11:36:06.485 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.471+0000 7fe9f2200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe9ec15d5f0 con 0x7fe9ec1569d0 2024-04-20T11:36:06.485 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.471+0000 7fe9f1800640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fe9ec06e970 0x7fe9ec0d05f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:42386/0 (socket says 172.21.15.152:42386) 2024-04-20T11:36:06.485 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.471+0000 7fe9f1800640 1 -- 172.21.15.152:0/1994116139 learned_addr learned my addr 172.21.15.152:0/1994116139 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:36:06.485 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.472+0000 7fe9f0e00640 1 --2- 172.21.15.152:0/1994116139 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe9ec1569d0 0x7fe9ec0ad760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:06.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.472+0000 7fe9eb400640 1 -- 172.21.15.152:0/1994116139 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3768468174 0 0) 0x7fe9ec15d8d0 con 0x7fe9ec06e970 2024-04-20T11:36:06.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.472+0000 7fe9eb400640 1 -- 172.21.15.152:0/1994116139 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe9d4003520 con 0x7fe9ec06e970 2024-04-20T11:36:06.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.472+0000 7fe9eb400640 1 -- 172.21.15.152:0/1994116139 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4107125634 0 0) 0x7fe9d4003520 con 0x7fe9ec06e970 2024-04-20T11:36:06.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.473+0000 7fe9eb400640 1 -- 172.21.15.152:0/1994116139 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe9ec1569d0 msgr2=0x7fe9ec0ad760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:06.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.473+0000 7fe9eb400640 1 --2- 172.21.15.152:0/1994116139 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe9ec1569d0 0x7fe9ec0ad760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:06.487 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.473+0000 7fe9eb400640 1 -- 172.21.15.152:0/1994116139 >> v1:172.21.15.175:6789/0 conn(0x7fe9ec06e510 legacy=0x7fe9ec15ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:06.487 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.473+0000 7fe9eb400640 1 -- 172.21.15.152:0/1994116139 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9ec06dd10 con 0x7fe9ec06e970 2024-04-20T11:36:06.487 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.473+0000 7fe9eb400640 1 -- 172.21.15.152:0/1994116139 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7fe9e4001a10 con 0x7fe9ec06e970 2024-04-20T11:36:06.487 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.473+0000 7fe9eb400640 1 -- 172.21.15.152:0/1994116139 >> v1:172.21.15.152:6789/0 conn(0x7fe9ec06e970 legacy=0x7fe9ec0d05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:06.488 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.473+0000 7fe9eb400640 1 --2- 172.21.15.152:0/1994116139 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe9d40039e0 0x7fe9d4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:06.488 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.473+0000 7fe9eb400640 1 --2- 172.21.15.152:0/1994116139 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe9d4004390 0x7fe9d4004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:06.488 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.473+0000 7fe9eb400640 1 --2- 172.21.15.152:0/1994116139 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe9ec1569d0 0x7fe9d4005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:06.488 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.473+0000 7fe9eb400640 1 -- 172.21.15.152:0/1994116139 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fe9ec15d5f0 con 0x7fe9d4004390 2024-04-20T11:36:06.488 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.473+0000 7fe9eb400640 1 -- 172.21.15.152:0/1994116139 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe9d400ba70 con 0x7fe9d40039e0 2024-04-20T11:36:06.489 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.473+0000 7fe9eb400640 1 -- 172.21.15.152:0/1994116139 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe9d400bc70 con 0x7fe9ec1569d0 2024-04-20T11:36:06.489 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.473+0000 7fe9ebe00640 1 --2- 172.21.15.152:0/1994116139 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe9d4004390 0x7fe9d4004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:06.489 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.473+0000 7fe9f0e00640 1 --2- 172.21.15.152:0/1994116139 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe9d40039e0 0x7fe9d4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:06.489 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.473+0000 7fe9ebe00640 1 -- 172.21.15.152:0/1994116139 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe9ec1569d0 msgr2=0x7fe9d4005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:06.489 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.473+0000 7fe9f1800640 1 --2- 172.21.15.152:0/1994116139 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe9ec1569d0 0x7fe9d4005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:06.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.474+0000 7fe9ebe00640 1 --2- 172.21.15.152:0/1994116139 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe9ec1569d0 0x7fe9d4005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:06.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.474+0000 7fe9ebe00640 1 -- 172.21.15.152:0/1994116139 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe9d40039e0 msgr2=0x7fe9d4003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:06.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.474+0000 7fe9ebe00640 1 --2- 172.21.15.152:0/1994116139 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe9d40039e0 0x7fe9d4003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:06.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.474+0000 7fe9ebe00640 1 -- 172.21.15.152:0/1994116139 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9d4003520 con 0x7fe9d4004390 2024-04-20T11:36:06.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.474+0000 7fe9f1800640 1 --2- 172.21.15.152:0/1994116139 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe9ec1569d0 0x7fe9d4005160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:36:06.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.474+0000 7fe9f0e00640 1 --2- 172.21.15.152:0/1994116139 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe9d40039e0 0x7fe9d4003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:36:06.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.475+0000 7fe9ebe00640 1 --2- 172.21.15.152:0/1994116139 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe9d4004390 0x7fe9d4004860 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7fe9d8001910 tx=0x7fe9d4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b49fc84779af41a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:06.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.475+0000 7fe9eb400640 1 -- 172.21.15.152:0/1994116139 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9d80014e0 con 0x7fe9d4004390 2024-04-20T11:36:06.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.475+0000 7fe9eb400640 1 -- 172.21.15.152:0/1994116139 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9d8037070 con 0x7fe9d4004390 2024-04-20T11:36:06.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.475+0000 7fe9eb400640 1 -- 172.21.15.152:0/1994116139 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9d8032840 con 0x7fe9d4004390 2024-04-20T11:36:06.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.475+0000 7fe9f2200640 1 -- 172.21.15.152:0/1994116139 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe9d4004390 msgr2=0x7fe9d4004860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:06.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.475+0000 7fe9f2200640 1 --2- 172.21.15.152:0/1994116139 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe9d4004390 0x7fe9d4004860 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7fe9d8001910 tx=0x7fe9d4005830 comp rx=0 tx=0).stop 2024-04-20T11:36:06.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.476+0000 7fe9f1800640 1 -- 172.21.15.152:0/1994116139 reap_dead start 2024-04-20T11:36:06.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.476+0000 7fe9f2200640 1 -- 172.21.15.152:0/1994116139 shutdown_connections 2024-04-20T11:36:06.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.476+0000 7fe9f2200640 1 -- 172.21.15.152:0/1994116139 >> 172.21.15.152:0/1994116139 conn(0x7fe9ec0b21e0 msgr2=0x7fe9ec0b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:06.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.476+0000 7fe9f2200640 1 -- 172.21.15.152:0/1994116139 shutdown_connections 2024-04-20T11:36:06.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.476+0000 7fe9f2200640 1 -- 172.21.15.152:0/1994116139 wait complete. 2024-04-20T11:36:06.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.477+0000 7fe9f2200640 1 Processor -- start 2024-04-20T11:36:06.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.478+0000 7fe9f2200640 1 -- start start 2024-04-20T11:36:06.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.478+0000 7fe9f2200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe9d4004390 0x7fe9ec09d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:06.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.479+0000 7fe9f2200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe9ec09a1b0 0x7fe9ec09a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:06.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.479+0000 7fe9f2200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe9ec0b01c0 0x7fe9ec0b0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:06.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.479+0000 7fe9f2200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fe9ec0d0d00 con 0x7fe9ec09a1b0 2024-04-20T11:36:06.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.479+0000 7fe9f2200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe9ec156db0 con 0x7fe9d4004390 2024-04-20T11:36:06.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.479+0000 7fe9f2200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe9ec0d0e80 con 0x7fe9ec0b01c0 2024-04-20T11:36:06.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.479+0000 7fe9ebe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe9ec09a1b0 0x7fe9ec09a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:06.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.479+0000 7fe9ebe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe9ec09a1b0 0x7fe9ec09a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35682/0 (socket says 172.21.15.152:35682) 2024-04-20T11:36:06.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.479+0000 7fe9ebe00640 1 -- 172.21.15.152:0/2512080709 learned_addr learned my addr 172.21.15.152:0/2512080709 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:06.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.479+0000 7fe9ebe00640 1 -- 172.21.15.152:0/2512080709 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe9ec0b01c0 msgr2=0x7fe9ec0b0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:06.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.479+0000 7fe9ebe00640 1 --2- 172.21.15.152:0/2512080709 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe9ec0b01c0 0x7fe9ec0b0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:06.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.479+0000 7fe9ebe00640 1 -- 172.21.15.152:0/2512080709 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe9d4004390 msgr2=0x7fe9ec09d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:06.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.479+0000 7fe9f0e00640 1 --2- 172.21.15.152:0/2512080709 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe9d4004390 0x7fe9ec09d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:06.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.479+0000 7fe9ebe00640 1 --2- 172.21.15.152:0/2512080709 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe9d4004390 0x7fe9ec09d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:06.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.479+0000 7fe9ebe00640 1 -- 172.21.15.152:0/2512080709 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9ec09bfc0 con 0x7fe9ec09a1b0 2024-04-20T11:36:06.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.480+0000 7fe9ebe00640 1 --2- 172.21.15.152:0/2512080709 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe9ec09a1b0 0x7fe9ec09a630 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7fe9d8001a30 tx=0x7fe9d80012b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:06.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.480+0000 7fe9e8c00640 1 -- 172.21.15.152:0/2512080709 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9d8006e50 con 0x7fe9ec09a1b0 2024-04-20T11:36:06.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.480+0000 7fe9e8c00640 1 -- 172.21.15.152:0/2512080709 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9d8040070 con 0x7fe9ec09a1b0 2024-04-20T11:36:06.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.480+0000 7fe9e8c00640 1 -- 172.21.15.152:0/2512080709 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9d8007260 con 0x7fe9ec09a1b0 2024-04-20T11:36:06.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.480+0000 7fe9f2200640 1 -- 172.21.15.152:0/2512080709 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9ec09c2a0 con 0x7fe9ec09a1b0 2024-04-20T11:36:06.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.480+0000 7fe9f2200640 1 -- 172.21.15.152:0/2512080709 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9ec09c860 con 0x7fe9ec09a1b0 2024-04-20T11:36:06.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.481+0000 7fe9f2200640 1 -- 172.21.15.152:0/2512080709 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9ec075e60 con 0x7fe9ec09a1b0 2024-04-20T11:36:06.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.483+0000 7fe9e8c00640 1 -- 172.21.15.152:0/2512080709 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 22) ==== 91601+0+0 (secure 0 0 0) 0x7fe9d8007400 con 0x7fe9ec09a1b0 2024-04-20T11:36:06.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.483+0000 7fe9e8c00640 1 --2- 172.21.15.152:0/2512080709 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe9cc06d0f0 0x7fe9cc06f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:06.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.484+0000 7fe9e8c00640 1 -- 172.21.15.152:0/2512080709 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 4857+0+0 (secure 0 0 0) 0x7fe9d80ae090 con 0x7fe9ec09a1b0 2024-04-20T11:36:06.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.484+0000 7fe9f0e00640 1 --2- 172.21.15.152:0/2512080709 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe9cc06d0f0 0x7fe9cc06f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:06.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.485+0000 7fe9f0e00640 1 --2- 172.21.15.152:0/2512080709 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe9cc06d0f0 0x7fe9cc06f5b0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec0b13f0 tx=0x7fe9e0008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:06.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.487+0000 7fe9e8c00640 1 -- 172.21.15.152:0/2512080709 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe9d803a380 con 0x7fe9ec09a1b0 2024-04-20T11:36:06.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:06.684+0000 7fe9f2200640 1 -- 172.21.15.152:0/2512080709 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fe9ec06f440 con 0x7fe9ec09a1b0 2024-04-20T11:36:07.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:07.433+0000 7fe9e8c00640 1 -- 172.21.15.152:0/2512080709 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v139) ==== 138+0+0 (secure 0 0 0) 0x7fe9d807c030 con 0x7fe9ec09a1b0 2024-04-20T11:36:07.434 INFO:teuthology.orchestra.run.smithi152.stderr:pool 'cephfs_metadata' created 2024-04-20T11:36:07.438 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:07.437+0000 7fe9f2200640 1 -- 172.21.15.152:0/2512080709 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe9cc06d0f0 msgr2=0x7fe9cc06f5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:07.438 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:07.437+0000 7fe9f2200640 1 --2- 172.21.15.152:0/2512080709 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe9cc06d0f0 0x7fe9cc06f5b0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec0b13f0 tx=0x7fe9e0008040 comp rx=0 tx=0).stop 2024-04-20T11:36:07.438 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:07.437+0000 7fe9f2200640 1 -- 172.21.15.152:0/2512080709 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe9ec09a1b0 msgr2=0x7fe9ec09a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:07.438 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:07.437+0000 7fe9f2200640 1 --2- 172.21.15.152:0/2512080709 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe9ec09a1b0 0x7fe9ec09a630 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7fe9d8001a30 tx=0x7fe9d80012b0 comp rx=0 tx=0).stop 2024-04-20T11:36:07.438 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:07.437+0000 7fe9f2200640 1 -- 172.21.15.152:0/2512080709 shutdown_connections 2024-04-20T11:36:07.439 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:07.437+0000 7fe9f2200640 1 --2- 172.21.15.152:0/2512080709 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe9cc06d0f0 0x7fe9cc06f5b0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:07.439 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:07.437+0000 7fe9f2200640 1 --2- 172.21.15.152:0/2512080709 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe9ec0b01c0 0x7fe9ec0b0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:07.439 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:07.437+0000 7fe9f2200640 1 --2- 172.21.15.152:0/2512080709 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe9ec09a1b0 0x7fe9ec09a630 unknown :-1 s=CLOSED pgs=1951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:07.440 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:07.437+0000 7fe9f2200640 1 --2- 172.21.15.152:0/2512080709 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe9d4004390 0x7fe9ec09d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:07.440 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:07.437+0000 7fe9f2200640 1 -- 172.21.15.152:0/2512080709 >> 172.21.15.152:0/2512080709 conn(0x7fe9ec0b21e0 msgr2=0x7fe9ec15b130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:07.440 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:07.437+0000 7fe9f2200640 1 -- 172.21.15.152:0/2512080709 shutdown_connections 2024-04-20T11:36:07.440 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:07.438+0000 7fe9f2200640 1 -- 172.21.15.152:0/2512080709 wait complete. 2024-04-20T11:36:07.469 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-20T11:36:07.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:07.674+0000 7f4ec4a00640 1 Processor -- start 2024-04-20T11:36:07.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:07.674+0000 7f4ec4a00640 1 -- start start 2024-04-20T11:36:07.675 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:07.675+0000 7f4ec4a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f4ec0161c20 0x7f4ec00d5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:07.675 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:07.675+0000 7f4ec4a00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f4ec00cfcb0 0x7f4ec00d2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:07.675 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:07.675+0000 7f4ec4a00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ec00d0400 con 0x7f4ec00d33d0 2024-04-20T11:36:07.675 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:07.675+0000 7f4ec4a00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f4ec00d2160 con 0x7f4ec0161c20 2024-04-20T11:36:07.675 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:07.675+0000 7f4ec4a00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f4ec00d2a00 con 0x7f4ec00cfcb0 2024-04-20T11:36:07.675 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:07.676+0000 7f4ebe800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f4ec0161c20 0x7f4ec00d5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:07.676 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:07.676+0000 7f4ebe800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f4ec0161c20 0x7f4ec00d5060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35684/0 (socket says 172.21.15.152:35684) 2024-04-20T11:36:07.676 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:07.676+0000 7f4ebe800640 1 -- 172.21.15.152:0/2963429284 learned_addr learned my addr 172.21.15.152:0/2963429284 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:07.676 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:07.676+0000 7f4ebe800640 1 -- 172.21.15.152:0/2963429284 >> v2:172.21.15.175:3300/0 conn(0x7f4ec00cfcb0 msgr2=0x7f4ec00d2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:07.676 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:07.676+0000 7f4ebe800640 1 --2- 172.21.15.152:0/2963429284 >> v2:172.21.15.175:3300/0 conn(0x7f4ec00cfcb0 0x7f4ec00d2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:07.676 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:07.676+0000 7f4ebe800640 1 -- 172.21.15.152:0/2963429284 >> v1:172.21.15.175:6789/0 conn(0x7f4ec00d33d0 legacy=0x7f4ec00d0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:07.676 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:07.676+0000 7f4ebe800640 1 -- 172.21.15.152:0/2963429284 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ec00d23b0 con 0x7f4ec0161c20 2024-04-20T11:36:07.677 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:07.677+0000 7f4ebe800640 1 --2- 172.21.15.152:0/2963429284 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ec0161c20 0x7f4ec00d5060 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7f4ec008dde0 tx=0x7f4eb002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df2ce5f07ca7fe6e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:07.677 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:07.678+0000 7f4ebde00640 1 -- 172.21.15.152:0/2963429284 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb002e070 con 0x7f4ec0161c20 2024-04-20T11:36:07.677 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:07.678+0000 7f4ebde00640 1 -- 172.21.15.152:0/2963429284 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4eb0037070 con 0x7f4ec0161c20 2024-04-20T11:36:07.677 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:07.678+0000 7f4ebde00640 1 -- 172.21.15.152:0/2963429284 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb0031a00 con 0x7f4ec0161c20 2024-04-20T11:36:07.678 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:07.678+0000 7f4ec4a00640 1 -- 172.21.15.152:0/2963429284 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ec0161c20 msgr2=0x7f4ec00d5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:07.678 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:07.678+0000 7f4ec4a00640 1 --2- 172.21.15.152:0/2963429284 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ec0161c20 0x7f4ec00d5060 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7f4ec008dde0 tx=0x7f4eb002b880 comp rx=0 tx=0).stop 2024-04-20T11:36:07.678 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:07.678+0000 7f4ec4a00640 1 -- 172.21.15.152:0/2963429284 shutdown_connections 2024-04-20T11:36:07.678 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:07.678+0000 7f4ec4a00640 1 --2- 172.21.15.152:0/2963429284 >> v2:172.21.15.175:3300/0 conn(0x7f4ec00cfcb0 0x7f4ec00d2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:07.678 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:07.678+0000 7f4ec4a00640 1 --2- 172.21.15.152:0/2963429284 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ec0161c20 0x7f4ec00d5060 unknown :-1 s=CLOSED pgs=1952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:07.678 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:07.678+0000 7f4ec4a00640 1 -- 172.21.15.152:0/2963429284 >> 172.21.15.152:0/2963429284 conn(0x7f4ec008bfd0 msgr2=0x7f4ec00d4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:07.678 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:07.678+0000 7f4ec4a00640 1 -- 172.21.15.152:0/2963429284 shutdown_connections 2024-04-20T11:36:07.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:07.679+0000 7f4ec4a00640 1 -- 172.21.15.152:0/2963429284 wait complete. 2024-04-20T11:36:07.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:07.680+0000 7f4ec4a00640 1 Processor -- start 2024-04-20T11:36:07.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:07.680+0000 7f4ec4a00640 1 -- start start 2024-04-20T11:36:07.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:07.681+0000 7f4ec4a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4ec00cfcb0 0x7f4ec014ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:07.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:07.681+0000 7f4ec4a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4ec00d33d0 0x7f4ec015bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:07.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:07.681+0000 7f4ec4a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ec0161c20 0x7f4ec015c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:07.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:07.681+0000 7f4ec4a00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec0162000 con 0x7f4ec0161c20 2024-04-20T11:36:07.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:07.681+0000 7f4ec4a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec00d0120 con 0x7f4ec00cfcb0 2024-04-20T11:36:07.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:07.681+0000 7f4ec4a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4ec0162180 con 0x7f4ec00d33d0 2024-04-20T11:36:07.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:07.682+0000 7f4ebf200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ec0161c20 0x7f4ec015c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:07.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:07.682+0000 7f4ebf200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ec0161c20 0x7f4ec015c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35696/0 (socket says 172.21.15.152:35696) 2024-04-20T11:36:07.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:07.682+0000 7f4ebf200640 1 -- 172.21.15.152:0/3542874223 learned_addr learned my addr 172.21.15.152:0/3542874223 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:07.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:07.682+0000 7f4ebe800640 1 --2- 172.21.15.152:0/3542874223 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4ec00cfcb0 0x7f4ec014ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:07.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:07.682+0000 7f4eb5e00640 1 --2- 172.21.15.152:0/3542874223 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4ec00d33d0 0x7f4ec015bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:07.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:07.682+0000 7f4ebf200640 1 -- 172.21.15.152:0/3542874223 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4ec00d33d0 msgr2=0x7f4ec015bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:07.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:07.682+0000 7f4ebf200640 1 --2- 172.21.15.152:0/3542874223 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4ec00d33d0 0x7f4ec015bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:07.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:07.682+0000 7f4ebf200640 1 -- 172.21.15.152:0/3542874223 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4ec00cfcb0 msgr2=0x7f4ec014ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:07.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:07.682+0000 7f4ebf200640 1 --2- 172.21.15.152:0/3542874223 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4ec00cfcb0 0x7f4ec014ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:07.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:07.682+0000 7f4ebf200640 1 -- 172.21.15.152:0/3542874223 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ec015c860 con 0x7f4ec0161c20 2024-04-20T11:36:07.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:07.682+0000 7f4ebf200640 1 --2- 172.21.15.152:0/3542874223 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ec0161c20 0x7f4ec015c120 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7f4eb800c780 tx=0x7f4eb800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:07.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:07.683+0000 7f4eb7400640 1 -- 172.21.15.152:0/3542874223 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb8017070 con 0x7f4ec0161c20 2024-04-20T11:36:07.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:07.683+0000 7f4eb7400640 1 -- 172.21.15.152:0/3542874223 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4eb8003da0 con 0x7f4ec0161c20 2024-04-20T11:36:07.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:07.683+0000 7f4eb7400640 1 -- 172.21.15.152:0/3542874223 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb80052b0 con 0x7f4ec0161c20 2024-04-20T11:36:07.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:07.683+0000 7f4ec4a00640 1 -- 172.21.15.152:0/3542874223 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ec015cb40 con 0x7f4ec0161c20 2024-04-20T11:36:07.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:07.683+0000 7f4ec4a00640 1 -- 172.21.15.152:0/3542874223 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ec015d100 con 0x7f4ec0161c20 2024-04-20T11:36:07.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:07.684+0000 7f4ec4a00640 1 -- 172.21.15.152:0/3542874223 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ec0075e60 con 0x7f4ec0161c20 2024-04-20T11:36:07.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:07.686+0000 7f4eb7400640 1 -- 172.21.15.152:0/3542874223 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 22) ==== 91601+0+0 (secure 0 0 0) 0x7f4eb8006b10 con 0x7f4ec0161c20 2024-04-20T11:36:07.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:07.686+0000 7f4eb7400640 1 --2- 172.21.15.152:0/3542874223 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4e9c06cf20 0x7f4e9c06f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:07.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:07.687+0000 7f4eb7400640 1 -- 172.21.15.152:0/3542874223 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(139..139 src has 1..139) ==== 5227+0+0 (secure 0 0 0) 0x7f4eb808b980 con 0x7f4ec0161c20 2024-04-20T11:36:07.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:07.687+0000 7f4ebe800640 1 --2- 172.21.15.152:0/3542874223 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4e9c06cf20 0x7f4e9c06f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:07.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:07.688+0000 7f4ebe800640 1 --2- 172.21.15.152:0/3542874223 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4e9c06cf20 0x7f4e9c06f3e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f4eb0036000 tx=0x7f4eb003d040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:07.691 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:07.692+0000 7f4eb7400640 1 -- 172.21.15.152:0/3542874223 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f4eb8058fc0 con 0x7f4ec0161c20 2024-04-20T11:36:07.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:07.894+0000 7f4ec4a00640 1 -- 172.21.15.152:0/3542874223 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f4ec00d1c80 con 0x7f4ec0161c20 2024-04-20T11:36:08.444 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:08.440+0000 7f4eb7400640 1 -- 172.21.15.152:0/3542874223 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v140) ==== 170+0+0 (secure 0 0 0) 0x7f4eb805d2c0 con 0x7f4ec0161c20 2024-04-20T11:36:08.444 INFO:teuthology.orchestra.run.smithi152.stderr:pool 'cephfs_data' created 2024-04-20T11:36:08.452 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:08.444+0000 7f4ec4a00640 1 -- 172.21.15.152:0/3542874223 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4e9c06cf20 msgr2=0x7f4e9c06f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:08.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:08.444+0000 7f4ec4a00640 1 --2- 172.21.15.152:0/3542874223 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4e9c06cf20 0x7f4e9c06f3e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f4eb0036000 tx=0x7f4eb003d040 comp rx=0 tx=0).stop 2024-04-20T11:36:08.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:08.444+0000 7f4ec4a00640 1 -- 172.21.15.152:0/3542874223 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ec0161c20 msgr2=0x7f4ec015c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:08.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:08.444+0000 7f4ec4a00640 1 --2- 172.21.15.152:0/3542874223 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ec0161c20 0x7f4ec015c120 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7f4eb800c780 tx=0x7f4eb800cc40 comp rx=0 tx=0).stop 2024-04-20T11:36:08.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:08.444+0000 7f4ec4a00640 1 -- 172.21.15.152:0/3542874223 shutdown_connections 2024-04-20T11:36:08.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:08.444+0000 7f4ec4a00640 1 --2- 172.21.15.152:0/3542874223 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4e9c06cf20 0x7f4e9c06f3e0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:08.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:08.444+0000 7f4ec4a00640 1 --2- 172.21.15.152:0/3542874223 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ec0161c20 0x7f4ec015c120 unknown :-1 s=CLOSED pgs=1953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:08.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:08.444+0000 7f4ec4a00640 1 --2- 172.21.15.152:0/3542874223 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4ec00d33d0 0x7f4ec015bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:08.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:08.444+0000 7f4ec4a00640 1 --2- 172.21.15.152:0/3542874223 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4ec00cfcb0 0x7f4ec014ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:08.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:08.444+0000 7f4ec4a00640 1 -- 172.21.15.152:0/3542874223 >> 172.21.15.152:0/3542874223 conn(0x7f4ec008bfd0 msgr2=0x7f4ec006cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:08.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:08.444+0000 7f4ec4a00640 1 -- 172.21.15.152:0/3542874223 shutdown_connections 2024-04-20T11:36:08.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:08.444+0000 7f4ec4a00640 1 -- 172.21.15.152:0/3542874223 wait complete. 2024-04-20T11:36:08.479 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-20T11:36:08.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:08.667+0000 7f5214000640 1 Processor -- start 2024-04-20T11:36:08.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:08.667+0000 7f5214000640 1 -- start start 2024-04-20T11:36:08.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:08.669+0000 7f5214000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f520c0b2880 0x7f520c0b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:08.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:08.669+0000 7f5214000640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f520c1463e0 con 0x7f520c0cd1b0 2024-04-20T11:36:08.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:08.669+0000 7f5214000640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f520c0b3e20 con 0x7f520c146000 2024-04-20T11:36:08.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:08.669+0000 7f5214000640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f520c0b4750 con 0x7f520c0b2880 2024-04-20T11:36:08.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:08.669+0000 7f5210c00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f520c0b2880 0x7f520c0b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:08.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:08.669+0000 7f5212000640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f520c0cd1b0 0x7f520c0cd5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:42392/0 (socket says 172.21.15.152:42392) 2024-04-20T11:36:08.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:08.669+0000 7f5212000640 1 -- 172.21.15.152:0/1162057371 learned_addr learned my addr 172.21.15.152:0/1162057371 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:36:08.670 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:08.670+0000 7f5210c00640 1 -- 172.21.15.152:0/1162057371 >> v1:172.21.15.175:6789/0 conn(0x7f520c146000 legacy=0x7f520c0b2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:08.670 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:08.670+0000 7f5210c00640 1 -- 172.21.15.152:0/1162057371 >> v1:172.21.15.152:6789/0 conn(0x7f520c0cd1b0 legacy=0x7f520c0cd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:08.670 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:08.670+0000 7f520be00640 1 -- 172.21.15.152:0/1162057371 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 412285945 0 0) 0x7f520c1463e0 con 0x7f520c0cd1b0 2024-04-20T11:36:08.671 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:08.670+0000 7f5210c00640 1 -- 172.21.15.152:0/1162057371 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f520c0b4100 con 0x7f520c0b2880 2024-04-20T11:36:08.671 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:08.671+0000 7f5210c00640 1 --2- 172.21.15.152:0/1162057371 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f520c0b2880 0x7f520c0b4be0 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7f51f80097a0 tx=0x7f51f802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5e8cdf296b25d90 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:08.671 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:08.671+0000 7f520be00640 1 -- 172.21.15.152:0/1162057371 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f8035020 con 0x7f520c0b2880 2024-04-20T11:36:08.671 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:08.671+0000 7f520be00640 1 -- 172.21.15.152:0/1162057371 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51f8031830 con 0x7f520c0b2880 2024-04-20T11:36:08.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:08.671+0000 7f520be00640 1 -- 172.21.15.152:0/1162057371 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f8031c00 con 0x7f520c0b2880 2024-04-20T11:36:08.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:08.672+0000 7f5214000640 1 -- 172.21.15.152:0/1162057371 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f520c0b2880 msgr2=0x7f520c0b4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:08.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:08.672+0000 7f5214000640 1 --2- 172.21.15.152:0/1162057371 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f520c0b2880 0x7f520c0b4be0 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7f51f80097a0 tx=0x7f51f802b8d0 comp rx=0 tx=0).stop 2024-04-20T11:36:08.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:08.672+0000 7f5214000640 1 -- 172.21.15.152:0/1162057371 shutdown_connections 2024-04-20T11:36:08.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:08.672+0000 7f5214000640 1 --2- 172.21.15.152:0/1162057371 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f520c0b2880 0x7f520c0b4be0 unknown :-1 s=CLOSED pgs=1955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:08.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:08.672+0000 7f5214000640 1 -- 172.21.15.152:0/1162057371 >> 172.21.15.152:0/1162057371 conn(0x7f520c06b980 msgr2=0x7f520c06bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:08.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:08.672+0000 7f5214000640 1 -- 172.21.15.152:0/1162057371 shutdown_connections 2024-04-20T11:36:08.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:08.672+0000 7f5214000640 1 -- 172.21.15.152:0/1162057371 wait complete. 2024-04-20T11:36:08.674 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:08.674+0000 7f5214000640 1 Processor -- start 2024-04-20T11:36:08.674 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:08.674+0000 7f5214000640 1 -- start start 2024-04-20T11:36:08.676 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:08.675+0000 7f5214000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f520c0b2880 0x7f520c13e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:08.676 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:08.675+0000 7f5214000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f520c0cd1b0 0x7f520c06bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:08.676 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:08.675+0000 7f5214000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f520c146000 0x7f520c06c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:08.676 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:08.675+0000 7f5214000640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f520c15ce70 con 0x7f520c0cd1b0 2024-04-20T11:36:08.676 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:08.675+0000 7f5214000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f520c15ccf0 con 0x7f520c146000 2024-04-20T11:36:08.676 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:08.675+0000 7f5214000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f520c15cff0 con 0x7f520c0b2880 2024-04-20T11:36:08.677 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:08.675+0000 7f5210c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f520c0cd1b0 0x7f520c06bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:08.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:08.675+0000 7f5210c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f520c0cd1b0 0x7f520c06bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35726/0 (socket says 172.21.15.152:35726) 2024-04-20T11:36:08.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:08.675+0000 7f5210c00640 1 -- 172.21.15.152:0/4202093569 learned_addr learned my addr 172.21.15.152:0/4202093569 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:08.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:08.676+0000 7f5211600640 1 --2- 172.21.15.152:0/4202093569 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f520c0b2880 0x7f520c13e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:08.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:08.676+0000 7f5212000640 1 --2- 172.21.15.152:0/4202093569 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f520c146000 0x7f520c06c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:08.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:08.676+0000 7f5210c00640 1 -- 172.21.15.152:0/4202093569 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f520c0b2880 msgr2=0x7f520c13e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:08.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:08.676+0000 7f5210c00640 1 --2- 172.21.15.152:0/4202093569 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f520c0b2880 0x7f520c13e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:08.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:08.676+0000 7f5210c00640 1 -- 172.21.15.152:0/4202093569 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f520c146000 msgr2=0x7f520c06c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:08.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:08.676+0000 7f5210c00640 1 --2- 172.21.15.152:0/4202093569 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f520c146000 0x7f520c06c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:08.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:08.676+0000 7f5210c00640 1 -- 172.21.15.152:0/4202093569 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f520c13e5a0 con 0x7f520c0cd1b0 2024-04-20T11:36:08.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:08.676+0000 7f5212000640 1 --2- 172.21.15.152:0/4202093569 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f520c146000 0x7f520c06c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:36:08.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:08.676+0000 7f5210c00640 1 --2- 172.21.15.152:0/4202093569 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f520c0cd1b0 0x7f520c06bd60 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7f51f80097a0 tx=0x7f51f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:08.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:08.676+0000 7f5209600640 1 -- 172.21.15.152:0/4202093569 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f8035050 con 0x7f520c0cd1b0 2024-04-20T11:36:08.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:08.677+0000 7f5209600640 1 -- 172.21.15.152:0/4202093569 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51f8004230 con 0x7f520c0cd1b0 2024-04-20T11:36:08.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:08.677+0000 7f5209600640 1 -- 172.21.15.152:0/4202093569 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f803b740 con 0x7f520c0cd1b0 2024-04-20T11:36:08.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:08.677+0000 7f5214000640 1 -- 172.21.15.152:0/4202093569 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f520c1e4da0 con 0x7f520c0cd1b0 2024-04-20T11:36:08.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:08.677+0000 7f5214000640 1 -- 172.21.15.152:0/4202093569 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f520c1e5360 con 0x7f520c0cd1b0 2024-04-20T11:36:08.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:08.678+0000 7f5214000640 1 -- 172.21.15.152:0/4202093569 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f520c0689f0 con 0x7f520c0cd1b0 2024-04-20T11:36:08.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:08.679+0000 7f5209600640 1 -- 172.21.15.152:0/4202093569 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 22) ==== 91601+0+0 (secure 0 0 0) 0x7f51f8038020 con 0x7f520c0cd1b0 2024-04-20T11:36:08.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:08.680+0000 7f5209600640 1 --2- 172.21.15.152:0/4202093569 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f51ec06cf20 0x7f51ec06f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:08.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:08.680+0000 7f5209600640 1 -- 172.21.15.152:0/4202093569 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 5609+0+0 (secure 0 0 0) 0x7f51f80ae600 con 0x7f520c0cd1b0 2024-04-20T11:36:08.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:08.681+0000 7f5211600640 1 --2- 172.21.15.152:0/4202093569 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f51ec06cf20 0x7f51ec06f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:08.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:08.682+0000 7f5211600640 1 --2- 172.21.15.152:0/4202093569 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f51ec06cf20 0x7f51ec06f3e0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f5200000f60 tx=0x7f5200009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:08.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:08.685+0000 7f5209600640 1 -- 172.21.15.152:0/4202093569 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f51f8040070 con 0x7f520c0cd1b0 2024-04-20T11:36:08.922 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:08.918+0000 7f5214000640 1 -- 172.21.15.152:0/4202093569 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f520c0b0180 con 0x7f520c0cd1b0 2024-04-20T11:36:09.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:09.485+0000 7f5209600640 1 -- 172.21.15.152:0/4202093569 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 22 and data pool 23 v92) ==== 174+0+0 (secure 0 0 0) 0x7f51f807c000 con 0x7f520c0cd1b0 2024-04-20T11:36:09.486 INFO:teuthology.orchestra.run.smithi152.stderr:new fs with metadata pool 22 and data pool 23 2024-04-20T11:36:09.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:09.493+0000 7f5214000640 1 -- 172.21.15.152:0/4202093569 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f51ec06cf20 msgr2=0x7f51ec06f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:09.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:09.493+0000 7f5214000640 1 --2- 172.21.15.152:0/4202093569 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f51ec06cf20 0x7f51ec06f3e0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f5200000f60 tx=0x7f5200009040 comp rx=0 tx=0).stop 2024-04-20T11:36:09.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:09.493+0000 7f5214000640 1 -- 172.21.15.152:0/4202093569 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f520c0cd1b0 msgr2=0x7f520c06bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:09.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:09.493+0000 7f5214000640 1 --2- 172.21.15.152:0/4202093569 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f520c0cd1b0 0x7f520c06bd60 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7f51f80097a0 tx=0x7f51f80046c0 comp rx=0 tx=0).stop 2024-04-20T11:36:09.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:09.494+0000 7f5214000640 1 -- 172.21.15.152:0/4202093569 shutdown_connections 2024-04-20T11:36:09.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:09.494+0000 7f5214000640 1 --2- 172.21.15.152:0/4202093569 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f51ec06cf20 0x7f51ec06f3e0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:09.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:09.495+0000 7f5214000640 1 --2- 172.21.15.152:0/4202093569 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f520c146000 0x7f520c06c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:09.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:09.495+0000 7f5214000640 1 --2- 172.21.15.152:0/4202093569 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f520c0cd1b0 0x7f520c06bd60 unknown :-1 s=CLOSED pgs=1956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:09.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:09.495+0000 7f5214000640 1 --2- 172.21.15.152:0/4202093569 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f520c0b2880 0x7f520c13e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:09.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:09.495+0000 7f5214000640 1 -- 172.21.15.152:0/4202093569 >> 172.21.15.152:0/4202093569 conn(0x7f520c06b980 msgr2=0x7f520c0b06a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:09.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:09.496+0000 7f5214000640 1 -- 172.21.15.152:0/4202093569 shutdown_connections 2024-04-20T11:36:09.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:09.496+0000 7f5214000640 1 -- 172.21.15.152:0/4202093569 wait complete. 2024-04-20T11:36:09.527 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:36:09.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:09.700+0000 7ffa64c00640 1 Processor -- start 2024-04-20T11:36:09.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:09.700+0000 7ffa64c00640 1 -- start start 2024-04-20T11:36:09.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:09.702+0000 7ffa64c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa600cce30 0x7ffa600b1800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:09.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:09.702+0000 7ffa64c00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7ffa600cdda0 0x7ffa600ce200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:09.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:09.702+0000 7ffa64c00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ffa600b2410 0x7ffa600b28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:09.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:09.702+0000 7ffa64c00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7ffa600b1370 con 0x7ffa600cdda0 2024-04-20T11:36:09.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:09.702+0000 7ffa64c00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7ffa600b0d90 con 0x7ffa600b2410 2024-04-20T11:36:09.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:09.702+0000 7ffa64c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ffa600b1d40 con 0x7ffa600cce30 2024-04-20T11:36:09.703 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:09.702+0000 7ffa5d400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7ffa600cdda0 0x7ffa600ce200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:09.703 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:09.702+0000 7ffa5d400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7ffa600cdda0 0x7ffa600ce200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35736/0 (socket says 172.21.15.152:35736) 2024-04-20T11:36:09.703 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:09.702+0000 7ffa5d400640 1 -- 172.21.15.152:0/2566732341 learned_addr learned my addr 172.21.15.152:0/2566732341 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:09.703 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:09.702+0000 7ffa5d400640 1 -- 172.21.15.152:0/2566732341 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa600cce30 msgr2=0x7ffa600b1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:09.703 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:09.702+0000 7ffa5e800640 1 --2- 172.21.15.152:0/2566732341 >> v2:172.21.15.175:3300/0 conn(0x7ffa600b2410 0x7ffa600b28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:09.703 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:09.702+0000 7ffa5de00640 1 --2- 172.21.15.152:0/2566732341 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa600cce30 0x7ffa600b1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:09.703 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:09.702+0000 7ffa5d400640 1 --2- 172.21.15.152:0/2566732341 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa600cce30 0x7ffa600b1800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:09.704 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:09.702+0000 7ffa5d400640 1 -- 172.21.15.152:0/2566732341 >> v2:172.21.15.175:3300/0 conn(0x7ffa600b2410 msgr2=0x7ffa600b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:09.704 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:09.702+0000 7ffa5d400640 1 --2- 172.21.15.152:0/2566732341 >> v2:172.21.15.175:3300/0 conn(0x7ffa600b2410 0x7ffa600b28a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:09.704 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:09.702+0000 7ffa5d400640 1 -- 172.21.15.152:0/2566732341 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa600cd970 con 0x7ffa600cdda0 2024-04-20T11:36:09.704 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:09.702+0000 7ffa5e800640 1 --2- 172.21.15.152:0/2566732341 >> v2:172.21.15.175:3300/0 conn(0x7ffa600b2410 0x7ffa600b28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:36:09.705 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:09.704+0000 7ffa5d400640 1 --2- 172.21.15.152:0/2566732341 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffa600cdda0 0x7ffa600ce200 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7ffa600cb9f0 tx=0x7ffa4c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db90041ca9bf7917 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:09.705 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:09.704+0000 7ffa5ca00640 1 -- 172.21.15.152:0/2566732341 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa4c035020 con 0x7ffa600cdda0 2024-04-20T11:36:09.705 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:09.704+0000 7ffa5ca00640 1 -- 172.21.15.152:0/2566732341 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa4c031870 con 0x7ffa600cdda0 2024-04-20T11:36:09.705 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:09.704+0000 7ffa5ca00640 1 -- 172.21.15.152:0/2566732341 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa4c031c40 con 0x7ffa600cdda0 2024-04-20T11:36:09.705 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:09.705+0000 7ffa64c00640 1 -- 172.21.15.152:0/2566732341 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffa600cdda0 msgr2=0x7ffa600ce200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:09.705 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:09.705+0000 7ffa64c00640 1 --2- 172.21.15.152:0/2566732341 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffa600cdda0 0x7ffa600ce200 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7ffa600cb9f0 tx=0x7ffa4c02b910 comp rx=0 tx=0).stop 2024-04-20T11:36:09.706 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:09.705+0000 7ffa64c00640 1 -- 172.21.15.152:0/2566732341 shutdown_connections 2024-04-20T11:36:09.706 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:09.705+0000 7ffa64c00640 1 --2- 172.21.15.152:0/2566732341 >> v2:172.21.15.175:3300/0 conn(0x7ffa600b2410 0x7ffa600b28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:09.706 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:09.705+0000 7ffa64c00640 1 --2- 172.21.15.152:0/2566732341 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffa600cdda0 0x7ffa600ce200 unknown :-1 s=CLOSED pgs=1957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:09.706 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:09.705+0000 7ffa64c00640 1 --2- 172.21.15.152:0/2566732341 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa600cce30 0x7ffa600b1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:09.706 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:09.705+0000 7ffa64c00640 1 -- 172.21.15.152:0/2566732341 >> 172.21.15.152:0/2566732341 conn(0x7ffa600b2fc0 msgr2=0x7ffa600b33a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:09.706 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:09.705+0000 7ffa64c00640 1 -- 172.21.15.152:0/2566732341 shutdown_connections 2024-04-20T11:36:09.706 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:09.705+0000 7ffa64c00640 1 -- 172.21.15.152:0/2566732341 wait complete. 2024-04-20T11:36:09.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:09.706+0000 7ffa64c00640 1 Processor -- start 2024-04-20T11:36:09.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:09.707+0000 7ffa64c00640 1 -- start start 2024-04-20T11:36:09.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:09.707+0000 7ffa64c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa600b2410 0x7ffa60155680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:09.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:09.707+0000 7ffa64c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffa600cce30 0x7ffa600b65a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:09.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:09.708+0000 7ffa64c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa600cdda0 0x7ffa60154b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:09.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:09.708+0000 7ffa64c00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7ffa600cd210 con 0x7ffa600cce30 2024-04-20T11:36:09.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:09.708+0000 7ffa64c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ffa600b1000 con 0x7ffa600cdda0 2024-04-20T11:36:09.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:09.708+0000 7ffa64c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ffa600cebf0 con 0x7ffa600b2410 2024-04-20T11:36:09.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:09.708+0000 7ffa5d400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffa600cce30 0x7ffa600b65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:09.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:09.708+0000 7ffa5d400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffa600cce30 0x7ffa600b65a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35738/0 (socket says 172.21.15.152:35738) 2024-04-20T11:36:09.710 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:09.708+0000 7ffa5d400640 1 -- 172.21.15.152:0/4147311451 learned_addr learned my addr 172.21.15.152:0/4147311451 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:09.710 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:09.708+0000 7ffa5d400640 1 -- 172.21.15.152:0/4147311451 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa600b2410 msgr2=0x7ffa60155680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:09.710 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:09.708+0000 7ffa5d400640 1 --2- 172.21.15.152:0/4147311451 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa600b2410 0x7ffa60155680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:09.710 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:09.708+0000 7ffa5d400640 1 -- 172.21.15.152:0/4147311451 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa600cdda0 msgr2=0x7ffa60154b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:09.710 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:09.708+0000 7ffa5d400640 1 --2- 172.21.15.152:0/4147311451 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa600cdda0 0x7ffa60154b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:09.710 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:09.708+0000 7ffa5d400640 1 -- 172.21.15.152:0/4147311451 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa60155d60 con 0x7ffa600cce30 2024-04-20T11:36:09.710 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:09.709+0000 7ffa5d400640 1 --2- 172.21.15.152:0/4147311451 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffa600cce30 0x7ffa600b65a0 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c02b950 tx=0x7ffa4c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:09.710 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:09.709+0000 7ffa56000640 1 -- 172.21.15.152:0/4147311451 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa4c040070 con 0x7ffa600cce30 2024-04-20T11:36:09.710 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:09.709+0000 7ffa56000640 1 -- 172.21.15.152:0/4147311451 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa4c03a420 con 0x7ffa600cce30 2024-04-20T11:36:09.710 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:09.709+0000 7ffa56000640 1 -- 172.21.15.152:0/4147311451 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa4c035040 con 0x7ffa600cce30 2024-04-20T11:36:09.710 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:09.709+0000 7ffa64c00640 1 -- 172.21.15.152:0/4147311451 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa60156040 con 0x7ffa600cce30 2024-04-20T11:36:09.711 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:09.709+0000 7ffa64c00640 1 -- 172.21.15.152:0/4147311451 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa60156600 con 0x7ffa600cce30 2024-04-20T11:36:09.711 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:09.710+0000 7ffa64c00640 1 -- 172.21.15.152:0/4147311451 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa600689f0 con 0x7ffa600cce30 2024-04-20T11:36:09.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:09.711+0000 7ffa56000640 1 -- 172.21.15.152:0/4147311451 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 22) ==== 91601+0+0 (secure 0 0 0) 0x7ffa4c04b060 con 0x7ffa600cce30 2024-04-20T11:36:09.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:09.712+0000 7ffa56000640 1 --2- 172.21.15.152:0/4147311451 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7ffa4006cf50 0x7ffa4006f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:09.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:09.712+0000 7ffa56000640 1 -- 172.21.15.152:0/4147311451 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 5709+0+0 (secure 0 0 0) 0x7ffa4c0ae4b0 con 0x7ffa600cce30 2024-04-20T11:36:09.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:09.712+0000 7ffa5de00640 1 --2- 172.21.15.152:0/4147311451 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7ffa4006cf50 0x7ffa4006f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:09.713 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:09.714+0000 7ffa5de00640 1 --2- 172.21.15.152:0/4147311451 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7ffa4006cf50 0x7ffa4006f410 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7ffa50006fd0 tx=0x7ffa50008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:09.716 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:09.717+0000 7ffa56000640 1 -- 172.21.15.152:0/4147311451 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ffa4c07c0d0 con 0x7ffa600cce30 2024-04-20T11:36:09.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:09.917+0000 7ffa64c00640 1 -- 172.21.15.152:0/4147311451 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ffa600b0a80 con 0x7ffa600cce30 2024-04-20T11:36:09.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:09.920+0000 7ffa56000640 1 -- 172.21.15.152:0/4147311451 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v141) ==== 74+0+16744 (secure 0 0 0) 0x7ffa4c07b9d0 con 0x7ffa600cce30 2024-04-20T11:36:09.920 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:36:09.921 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":141,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:36:08.919648+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:36:06.702770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"141","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-04-20T11:36:07.911633+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"141","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:36:09.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:09.923+0000 7ffa64c00640 1 -- 172.21.15.152:0/4147311451 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7ffa4006cf50 msgr2=0x7ffa4006f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:09.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:09.923+0000 7ffa64c00640 1 --2- 172.21.15.152:0/4147311451 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7ffa4006cf50 0x7ffa4006f410 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7ffa50006fd0 tx=0x7ffa50008040 comp rx=0 tx=0).stop 2024-04-20T11:36:09.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:09.923+0000 7ffa64c00640 1 -- 172.21.15.152:0/4147311451 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffa600cce30 msgr2=0x7ffa600b65a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:09.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:09.923+0000 7ffa64c00640 1 --2- 172.21.15.152:0/4147311451 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffa600cce30 0x7ffa600b65a0 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c02b950 tx=0x7ffa4c0046c0 comp rx=0 tx=0).stop 2024-04-20T11:36:09.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:09.923+0000 7ffa64c00640 1 -- 172.21.15.152:0/4147311451 shutdown_connections 2024-04-20T11:36:09.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:09.923+0000 7ffa64c00640 1 --2- 172.21.15.152:0/4147311451 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7ffa4006cf50 0x7ffa4006f410 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:09.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:09.923+0000 7ffa64c00640 1 --2- 172.21.15.152:0/4147311451 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa600cdda0 0x7ffa60154b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:09.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:09.923+0000 7ffa64c00640 1 --2- 172.21.15.152:0/4147311451 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffa600cce30 0x7ffa600b65a0 unknown :-1 s=CLOSED pgs=1958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:09.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:09.923+0000 7ffa64c00640 1 --2- 172.21.15.152:0/4147311451 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa600b2410 0x7ffa60155680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:09.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:09.923+0000 7ffa64c00640 1 -- 172.21.15.152:0/4147311451 >> 172.21.15.152:0/4147311451 conn(0x7ffa600b2fc0 msgr2=0x7ffa600d0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:09.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:09.923+0000 7ffa64c00640 1 -- 172.21.15.152:0/4147311451 shutdown_connections 2024-04-20T11:36:09.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:09.924+0000 7ffa64c00640 1 -- 172.21.15.152:0/4147311451 wait complete. 2024-04-20T11:36:09.942 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:36:10.133 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.133+0000 7f305aa00640 1 Processor -- start 2024-04-20T11:36:10.133 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.133+0000 7f305aa00640 1 -- start start 2024-04-20T11:36:10.137 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.135+0000 7f305aa00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f305406f510 0x7f3054140dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:10.137 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.135+0000 7f305aa00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30540cf0a0 con 0x7f305413e530 2024-04-20T11:36:10.137 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.135+0000 7f305aa00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30540d00e0 con 0x7f3054141310 2024-04-20T11:36:10.137 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.135+0000 7f305aa00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f3054141b00 con 0x7f305406f510 2024-04-20T11:36:10.137 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.135+0000 7f3059600640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f305413e530 0x7f305406ee00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:42404/0 (socket says 172.21.15.152:42404) 2024-04-20T11:36:10.137 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.135+0000 7f3059600640 1 -- 172.21.15.152:0/2527322436 learned_addr learned my addr 172.21.15.152:0/2527322436 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:36:10.137 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.136+0000 7f3058c00640 1 --2- 172.21.15.152:0/2527322436 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f305406f510 0x7f3054140dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:10.137 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.136+0000 7f3053e00640 1 -- 172.21.15.152:0/2527322436 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1385215230 0 0) 0x7f3048001110 con 0x7f305413e530 2024-04-20T11:36:10.137 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.136+0000 7f3053e00640 1 -- 172.21.15.152:0/2527322436 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f303c003520 con 0x7f305413e530 2024-04-20T11:36:10.138 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.136+0000 7f3053e00640 1 -- 172.21.15.152:0/2527322436 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3374458556 0 0) 0x7f303c003520 con 0x7f305413e530 2024-04-20T11:36:10.138 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.136+0000 7f3053e00640 1 -- 172.21.15.152:0/2527322436 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f305406f510 msgr2=0x7f3054140dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:10.138 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.136+0000 7f3053e00640 1 --2- 172.21.15.152:0/2527322436 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f305406f510 0x7f3054140dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:10.138 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.136+0000 7f3053e00640 1 -- 172.21.15.152:0/2527322436 >> v1:172.21.15.175:6789/0 conn(0x7f3054141310 legacy=0x7f3054071cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:10.138 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.136+0000 7f3053e00640 1 -- 172.21.15.152:0/2527322436 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30540cf320 con 0x7f305413e530 2024-04-20T11:36:10.138 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.136+0000 7f3053e00640 1 -- 172.21.15.152:0/2527322436 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f3048001840 con 0x7f305413e530 2024-04-20T11:36:10.138 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.136+0000 7f3053e00640 1 -- 172.21.15.152:0/2527322436 >> v1:172.21.15.152:6789/0 conn(0x7f305413e530 legacy=0x7f305406ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:10.138 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.136+0000 7f3053e00640 1 --2- 172.21.15.152:0/2527322436 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f303c0039e0 0x7f303c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:10.138 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.136+0000 7f3053e00640 1 --2- 172.21.15.152:0/2527322436 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f305406f510 0x7f303c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:10.138 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.136+0000 7f3053e00640 1 --2- 172.21.15.152:0/2527322436 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f303c004d40 0x7f303c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:10.138 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.136+0000 7f3053e00640 1 -- 172.21.15.152:0/2527322436 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f303c00dc80 con 0x7f303c004d40 2024-04-20T11:36:10.139 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.136+0000 7f3053e00640 1 -- 172.21.15.152:0/2527322436 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f303c00de00 con 0x7f303c0039e0 2024-04-20T11:36:10.139 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.136+0000 7f3053e00640 1 -- 172.21.15.152:0/2527322436 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f303c00e000 con 0x7f305406f510 2024-04-20T11:36:10.139 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.137+0000 7f305a000640 1 --2- 172.21.15.152:0/2527322436 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f303c004d40 0x7f303c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:10.139 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.137+0000 7f3059600640 1 --2- 172.21.15.152:0/2527322436 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f303c0039e0 0x7f303c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:10.139 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.137+0000 7f3058c00640 1 --2- 172.21.15.152:0/2527322436 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f305406f510 0x7f303c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:10.139 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.137+0000 7f305a000640 1 -- 172.21.15.152:0/2527322436 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f305406f510 msgr2=0x7f303c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:10.139 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.137+0000 7f305a000640 1 --2- 172.21.15.152:0/2527322436 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f305406f510 0x7f303c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:10.139 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.137+0000 7f305a000640 1 -- 172.21.15.152:0/2527322436 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f303c0039e0 msgr2=0x7f303c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:10.139 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.137+0000 7f305a000640 1 --2- 172.21.15.152:0/2527322436 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f303c0039e0 0x7f303c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:10.139 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.137+0000 7f305a000640 1 -- 172.21.15.152:0/2527322436 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f303c003520 con 0x7f303c004d40 2024-04-20T11:36:10.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.138+0000 7f305a000640 1 --2- 172.21.15.152:0/2527322436 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f303c004d40 0x7f303c0051c0 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7f304c001e20 tx=0x7f303c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=103cb26340d4cd92 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:10.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.139+0000 7f3053e00640 1 -- 172.21.15.152:0/2527322436 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f304c001a70 con 0x7f303c004d40 2024-04-20T11:36:10.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.139+0000 7f3053e00640 1 -- 172.21.15.152:0/2527322436 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f304c003b70 con 0x7f303c004d40 2024-04-20T11:36:10.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.139+0000 7f3053e00640 1 -- 172.21.15.152:0/2527322436 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f304c0050d0 con 0x7f303c004d40 2024-04-20T11:36:10.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.139+0000 7f305aa00640 1 -- 172.21.15.152:0/2527322436 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f303c004d40 msgr2=0x7f303c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:10.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.139+0000 7f305aa00640 1 --2- 172.21.15.152:0/2527322436 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f303c004d40 0x7f303c0051c0 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7f304c001e20 tx=0x7f303c005830 comp rx=0 tx=0).stop 2024-04-20T11:36:10.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.139+0000 7f305a000640 1 -- 172.21.15.152:0/2527322436 reap_dead start 2024-04-20T11:36:10.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.139+0000 7f305aa00640 1 -- 172.21.15.152:0/2527322436 shutdown_connections 2024-04-20T11:36:10.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.139+0000 7f305aa00640 1 -- 172.21.15.152:0/2527322436 >> 172.21.15.152:0/2527322436 conn(0x7f3054067ae0 msgr2=0x7f305406b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:10.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.139+0000 7f305aa00640 1 -- 172.21.15.152:0/2527322436 shutdown_connections 2024-04-20T11:36:10.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.140+0000 7f305aa00640 1 -- 172.21.15.152:0/2527322436 wait complete. 2024-04-20T11:36:10.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.141+0000 7f305aa00640 1 Processor -- start 2024-04-20T11:36:10.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.142+0000 7f305aa00640 1 -- start start 2024-04-20T11:36:10.145 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.142+0000 7f305aa00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f303c004d40 0x7f30540b1130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:10.145 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.142+0000 7f305aa00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f30541f5c70 0x7f30541f5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:10.145 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.142+0000 7f305aa00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f30541f77f0 0x7f30540b3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:10.145 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.142+0000 7f305aa00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f30540cf8e0 con 0x7f30541f77f0 2024-04-20T11:36:10.145 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.143+0000 7f305aa00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f30540cfc50 con 0x7f303c004d40 2024-04-20T11:36:10.145 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.143+0000 7f305aa00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f30541417c0 con 0x7f30541f5c70 2024-04-20T11:36:10.145 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.143+0000 7f305a000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f30541f77f0 0x7f30540b3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:10.145 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.143+0000 7f305a000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f30541f77f0 0x7f30540b3860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35754/0 (socket says 172.21.15.152:35754) 2024-04-20T11:36:10.146 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.143+0000 7f305a000640 1 -- 172.21.15.152:0/4270191831 learned_addr learned my addr 172.21.15.152:0/4270191831 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:10.146 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.143+0000 7f305a000640 1 -- 172.21.15.152:0/4270191831 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f30541f5c70 msgr2=0x7f30541f5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:10.146 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.143+0000 7f3059600640 1 --2- 172.21.15.152:0/4270191831 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f303c004d40 0x7f30540b1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:10.146 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.143+0000 7f305a000640 1 --2- 172.21.15.152:0/4270191831 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f30541f5c70 0x7f30541f5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:10.146 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.143+0000 7f305a000640 1 -- 172.21.15.152:0/4270191831 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f303c004d40 msgr2=0x7f30540b1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:10.146 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.143+0000 7f305a000640 1 --2- 172.21.15.152:0/4270191831 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f303c004d40 0x7f30540b1130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:10.146 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.143+0000 7f305a000640 1 -- 172.21.15.152:0/4270191831 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30541f5a70 con 0x7f30541f77f0 2024-04-20T11:36:10.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.144+0000 7f305a000640 1 --2- 172.21.15.152:0/4270191831 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f30541f77f0 0x7f30540b3860 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7f304c005040 tx=0x7f304c0014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:10.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.144+0000 7f3051600640 1 -- 172.21.15.152:0/4270191831 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f304c007db0 con 0x7f30541f77f0 2024-04-20T11:36:10.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.144+0000 7f3051600640 1 -- 172.21.15.152:0/4270191831 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f304c003970 con 0x7f30541f77f0 2024-04-20T11:36:10.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.144+0000 7f305aa00640 1 -- 172.21.15.152:0/4270191831 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30540b1710 con 0x7f30541f77f0 2024-04-20T11:36:10.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.144+0000 7f3051600640 1 -- 172.21.15.152:0/4270191831 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f304c006d50 con 0x7f30541f77f0 2024-04-20T11:36:10.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.144+0000 7f305aa00640 1 -- 172.21.15.152:0/4270191831 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30540b1cd0 con 0x7f30541f77f0 2024-04-20T11:36:10.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.145+0000 7f305aa00640 1 -- 172.21.15.152:0/4270191831 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3054075e60 con 0x7f30541f77f0 2024-04-20T11:36:10.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.146+0000 7f3051600640 1 -- 172.21.15.152:0/4270191831 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 22) ==== 91601+0+0 (secure 0 0 0) 0x7f304c041030 con 0x7f30541f77f0 2024-04-20T11:36:10.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.147+0000 7f3051600640 1 --2- 172.21.15.152:0/4270191831 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f303406cf20 0x7f303406f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:10.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.147+0000 7f3051600640 1 -- 172.21.15.152:0/4270191831 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 5709+0+0 (secure 0 0 0) 0x7f304c0adc80 con 0x7f30541f77f0 2024-04-20T11:36:10.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.148+0000 7f3059600640 1 --2- 172.21.15.152:0/4270191831 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f303406cf20 0x7f303406f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:10.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.149+0000 7f3059600640 1 --2- 172.21.15.152:0/4270191831 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f303406cf20 0x7f303406f3e0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f30480013d0 tx=0x7f3048001320 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:10.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.151+0000 7f3051600640 1 -- 172.21.15.152:0/4270191831 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f30541f5a70 con 0x7f30541f77f0 2024-04-20T11:36:10.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.358+0000 7f305aa00640 1 -- 172.21.15.152:0/4270191831 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f30540b3490 con 0x7f30541f77f0 2024-04-20T11:36:10.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.360+0000 7f3051600640 1 -- 172.21.15.152:0/4270191831 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v141) ==== 74+0+16744 (secure 0 0 0) 0x7f304c033020 con 0x7f30541f77f0 2024-04-20T11:36:10.361 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:36:10.361 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":141,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:36:08.919648+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:36:06.702770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"141","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-04-20T11:36:07.911633+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"141","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:36:10.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.363+0000 7f305aa00640 1 -- 172.21.15.152:0/4270191831 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f303406cf20 msgr2=0x7f303406f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:10.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.363+0000 7f305aa00640 1 --2- 172.21.15.152:0/4270191831 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f303406cf20 0x7f303406f3e0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f30480013d0 tx=0x7f3048001320 comp rx=0 tx=0).stop 2024-04-20T11:36:10.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.363+0000 7f305aa00640 1 -- 172.21.15.152:0/4270191831 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f30541f77f0 msgr2=0x7f30540b3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:10.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.363+0000 7f305aa00640 1 --2- 172.21.15.152:0/4270191831 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f30541f77f0 0x7f30540b3860 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7f304c005040 tx=0x7f304c0014a0 comp rx=0 tx=0).stop 2024-04-20T11:36:10.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.363+0000 7f305aa00640 1 -- 172.21.15.152:0/4270191831 shutdown_connections 2024-04-20T11:36:10.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.363+0000 7f305aa00640 1 --2- 172.21.15.152:0/4270191831 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f303406cf20 0x7f303406f3e0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:10.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.363+0000 7f305aa00640 1 --2- 172.21.15.152:0/4270191831 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f30541f77f0 0x7f30540b3860 unknown :-1 s=CLOSED pgs=1961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:10.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.363+0000 7f305aa00640 1 --2- 172.21.15.152:0/4270191831 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f30541f5c70 0x7f30541f5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:10.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.363+0000 7f305aa00640 1 --2- 172.21.15.152:0/4270191831 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f303c004d40 0x7f30540b1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:10.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.363+0000 7f305aa00640 1 -- 172.21.15.152:0/4270191831 >> 172.21.15.152:0/4270191831 conn(0x7f3054067ae0 msgr2=0x7f305413e910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:10.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.364+0000 7f305aa00640 1 -- 172.21.15.152:0/4270191831 shutdown_connections 2024-04-20T11:36:10.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.364+0000 7f305aa00640 1 -- 172.21.15.152:0/4270191831 wait complete. 2024-04-20T11:36:10.383 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-20T11:36:10.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.588+0000 7f7818e00640 1 Processor -- start 2024-04-20T11:36:10.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.588+0000 7f7818e00640 1 -- start start 2024-04-20T11:36:10.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.590+0000 7f7818e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f7814146000 0x7f78140b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:10.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.590+0000 7f7818e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f78140cd8c0 0x7f78140b2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:10.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.590+0000 7f7818e00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78141463e0 con 0x7f78140b2170 2024-04-20T11:36:10.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.590+0000 7f7818e00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f78140cdca0 con 0x7f7814146000 2024-04-20T11:36:10.590 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.590+0000 7f7818e00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f78140b4750 con 0x7f78140cd8c0 2024-04-20T11:36:10.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.590+0000 7f7811e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f7814146000 0x7f78140b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:10.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.590+0000 7f7811400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f78140b2170 0x7f78140cd1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:42412/0 (socket says 172.21.15.152:42412) 2024-04-20T11:36:10.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.590+0000 7f7811400640 1 -- 172.21.15.152:0/3259383895 learned_addr learned my addr 172.21.15.152:0/3259383895 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:36:10.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.590+0000 7f7812800640 1 --2- 172.21.15.152:0/3259383895 >> v2:172.21.15.175:3300/0 conn(0x7f78140cd8c0 0x7f78140b2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:10.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.591+0000 7f7810a00640 1 -- 172.21.15.152:0/3259383895 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3331282613 0 0) 0x7f78141463e0 con 0x7f78140b2170 2024-04-20T11:36:10.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.591+0000 7f7810a00640 1 -- 172.21.15.152:0/3259383895 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f77f8003520 con 0x7f78140b2170 2024-04-20T11:36:10.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.591+0000 7f7811e00640 1 -- 172.21.15.152:0/3259383895 >> v2:172.21.15.175:3300/0 conn(0x7f78140cd8c0 msgr2=0x7f78140b2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:10.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.591+0000 7f7811e00640 1 --2- 172.21.15.152:0/3259383895 >> v2:172.21.15.175:3300/0 conn(0x7f78140cd8c0 0x7f78140b2620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:10.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.591+0000 7f7811e00640 1 -- 172.21.15.152:0/3259383895 >> v1:172.21.15.152:6789/0 conn(0x7f78140b2170 legacy=0x7f78140cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:10.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.591+0000 7f7811e00640 1 -- 172.21.15.152:0/3259383895 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78140b3e20 con 0x7f7814146000 2024-04-20T11:36:10.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.592+0000 7f7811e00640 1 --2- 172.21.15.152:0/3259383895 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7814146000 0x7f78140b4be0 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7f78141601b0 tx=0x7f77fc02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=399ba950cea422ce server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:10.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.593+0000 7f7810a00640 1 -- 172.21.15.152:0/3259383895 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77fc0315a0 con 0x7f7814146000 2024-04-20T11:36:10.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.593+0000 7f7810a00640 1 -- 172.21.15.152:0/3259383895 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77fc037070 con 0x7f7814146000 2024-04-20T11:36:10.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.593+0000 7f7810a00640 1 -- 172.21.15.152:0/3259383895 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77fc031960 con 0x7f7814146000 2024-04-20T11:36:10.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.593+0000 7f7818e00640 1 -- 172.21.15.152:0/3259383895 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7814146000 msgr2=0x7f78140b4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:10.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.593+0000 7f7818e00640 1 --2- 172.21.15.152:0/3259383895 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7814146000 0x7f78140b4be0 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7f78141601b0 tx=0x7f77fc02b840 comp rx=0 tx=0).stop 2024-04-20T11:36:10.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.594+0000 7f7818e00640 1 -- 172.21.15.152:0/3259383895 shutdown_connections 2024-04-20T11:36:10.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.594+0000 7f7818e00640 1 --2- 172.21.15.152:0/3259383895 >> v2:172.21.15.175:3300/0 conn(0x7f78140cd8c0 0x7f78140b2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:10.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.594+0000 7f7818e00640 1 --2- 172.21.15.152:0/3259383895 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7814146000 0x7f78140b4be0 unknown :-1 s=CLOSED pgs=1963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:10.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.594+0000 7f7818e00640 1 -- 172.21.15.152:0/3259383895 >> 172.21.15.152:0/3259383895 conn(0x7f781406b980 msgr2=0x7f781406bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:10.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.594+0000 7f7818e00640 1 -- 172.21.15.152:0/3259383895 shutdown_connections 2024-04-20T11:36:10.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.594+0000 7f7818e00640 1 -- 172.21.15.152:0/3259383895 wait complete. 2024-04-20T11:36:10.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.595+0000 7f7818e00640 1 Processor -- start 2024-04-20T11:36:10.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.596+0000 7f7818e00640 1 -- start start 2024-04-20T11:36:10.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.596+0000 7f7818e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f78140b2170 0x7f78140b1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:10.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.596+0000 7f7818e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78140cd8c0 0x7f78141f6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:10.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.596+0000 7f7818e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7814146000 0x7f78140af890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:10.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.597+0000 7f7818e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f781415ccf0 con 0x7f7814146000 2024-04-20T11:36:10.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.597+0000 7f7818e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f78140b43e0 con 0x7f78140cd8c0 2024-04-20T11:36:10.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.597+0000 7f7818e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f781415ce70 con 0x7f78140b2170 2024-04-20T11:36:10.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.597+0000 7f7812800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7814146000 0x7f78140af890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:10.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.597+0000 7f7812800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7814146000 0x7f78140af890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35760/0 (socket says 172.21.15.152:35760) 2024-04-20T11:36:10.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.597+0000 7f7812800640 1 -- 172.21.15.152:0/306505920 learned_addr learned my addr 172.21.15.152:0/306505920 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:10.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.597+0000 7f7811400640 1 --2- 172.21.15.152:0/306505920 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78140cd8c0 0x7f78141f6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:10.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.597+0000 7f7812800640 1 -- 172.21.15.152:0/306505920 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f78140b2170 msgr2=0x7f78140b1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:10.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.597+0000 7f7812800640 1 --2- 172.21.15.152:0/306505920 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f78140b2170 0x7f78140b1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:10.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.597+0000 7f7812800640 1 -- 172.21.15.152:0/306505920 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78140cd8c0 msgr2=0x7f78141f6ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:10.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.597+0000 7f7812800640 1 --2- 172.21.15.152:0/306505920 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78140cd8c0 0x7f78141f6ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:10.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.597+0000 7f7812800640 1 -- 172.21.15.152:0/306505920 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78140ae510 con 0x7f7814146000 2024-04-20T11:36:10.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.598+0000 7f7812800640 1 --2- 172.21.15.152:0/306505920 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7814146000 0x7f78140af890 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7f780c00efc0 tx=0x7f780c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:10.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.598+0000 7f780a000640 1 -- 172.21.15.152:0/306505920 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f780c018020 con 0x7f7814146000 2024-04-20T11:36:10.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.598+0000 7f780a000640 1 -- 172.21.15.152:0/306505920 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f780c014480 con 0x7f7814146000 2024-04-20T11:36:10.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.598+0000 7f780a000640 1 -- 172.21.15.152:0/306505920 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f780c014870 con 0x7f7814146000 2024-04-20T11:36:10.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.598+0000 7f7818e00640 1 -- 172.21.15.152:0/306505920 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78140ae7f0 con 0x7f7814146000 2024-04-20T11:36:10.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.598+0000 7f7818e00640 1 -- 172.21.15.152:0/306505920 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78141426b0 con 0x7f7814146000 2024-04-20T11:36:10.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.599+0000 7f7818e00640 1 -- 172.21.15.152:0/306505920 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78140ae2a0 con 0x7f7814146000 2024-04-20T11:36:10.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.600+0000 7f780a000640 1 -- 172.21.15.152:0/306505920 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 22) ==== 91601+0+0 (secure 0 0 0) 0x7f780c005d10 con 0x7f7814146000 2024-04-20T11:36:10.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.602+0000 7f780a000640 1 --2- 172.21.15.152:0/306505920 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f77f006cf20 0x7f77f006f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:10.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.602+0000 7f780a000640 1 -- 172.21.15.152:0/306505920 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5709+0+0 (secure 0 0 0) 0x7f780c08d1a0 con 0x7f7814146000 2024-04-20T11:36:10.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.603+0000 7f7811e00640 1 --2- 172.21.15.152:0/306505920 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f77f006cf20 0x7f77f006f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:10.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.604+0000 7f7811e00640 1 --2- 172.21.15.152:0/306505920 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f77f006cf20 0x7f77f006f3e0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f77fc036000 tx=0x7f77fc030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:10.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.607+0000 7f780a000640 1 -- 172.21.15.152:0/306505920 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f780c022070 con 0x7f7814146000 2024-04-20T11:36:10.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:10.850+0000 7f7818e00640 1 -- 172.21.15.152:0/306505920 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f78140689f0 con 0x7f7814146000 2024-04-20T11:36:11.523 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:11.523+0000 7f780a000640 1 -- 172.21.15.152:0/306505920 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v95) ==== 118+0+0 (secure 0 0 0) 0x7f780c010040 con 0x7f7814146000 2024-04-20T11:36:11.526 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:11.527+0000 7f7818e00640 1 -- 172.21.15.152:0/306505920 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f77f006cf20 msgr2=0x7f77f006f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:11.526 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:11.527+0000 7f7818e00640 1 --2- 172.21.15.152:0/306505920 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f77f006cf20 0x7f77f006f3e0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f77fc036000 tx=0x7f77fc030000 comp rx=0 tx=0).stop 2024-04-20T11:36:11.527 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:11.527+0000 7f7818e00640 1 -- 172.21.15.152:0/306505920 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7814146000 msgr2=0x7f78140af890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:11.527 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:11.527+0000 7f7818e00640 1 --2- 172.21.15.152:0/306505920 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7814146000 0x7f78140af890 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7f780c00efc0 tx=0x7f780c00c490 comp rx=0 tx=0).stop 2024-04-20T11:36:11.527 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:11.527+0000 7f7818e00640 1 -- 172.21.15.152:0/306505920 shutdown_connections 2024-04-20T11:36:11.527 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:11.527+0000 7f7818e00640 1 --2- 172.21.15.152:0/306505920 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f77f006cf20 0x7f77f006f3e0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:11.527 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:11.527+0000 7f7818e00640 1 --2- 172.21.15.152:0/306505920 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7814146000 0x7f78140af890 unknown :-1 s=CLOSED pgs=1964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:11.527 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:11.527+0000 7f7818e00640 1 --2- 172.21.15.152:0/306505920 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78140cd8c0 0x7f78141f6ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:11.528 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:11.527+0000 7f7818e00640 1 --2- 172.21.15.152:0/306505920 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f78140b2170 0x7f78140b1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:11.528 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:11.527+0000 7f7818e00640 1 -- 172.21.15.152:0/306505920 >> 172.21.15.152:0/306505920 conn(0x7f781406b980 msgr2=0x7f781406efd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:11.528 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:11.527+0000 7f7818e00640 1 -- 172.21.15.152:0/306505920 shutdown_connections 2024-04-20T11:36:11.528 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:11.527+0000 7f7818e00640 1 -- 172.21.15.152:0/306505920 wait complete. 2024-04-20T11:36:11.551 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-20T11:36:11.552 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-20T11:36:11.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:11.756+0000 7fb0bb200640 1 Processor -- start 2024-04-20T11:36:11.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:11.756+0000 7fb0bb200640 1 -- start start 2024-04-20T11:36:11.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:11.757+0000 7fb0bb200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb0bc06ee00 0x7fb0bc06f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:11.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:11.757+0000 7fb0bb200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fb0bc15fbf0 0x7fb0bc15c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:11.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:11.758+0000 7fb0bb200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fb0bc0cf110 0x7fb0bc0b3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:11.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:11.758+0000 7fb0bb200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fb0bc0d00d0 con 0x7fb0bc0cf110 2024-04-20T11:36:11.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:11.758+0000 7fb0bb200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fb0bc06f720 con 0x7fb0bc15fbf0 2024-04-20T11:36:11.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:11.758+0000 7fb0bb200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb0bc0cfb10 con 0x7fb0bc06ee00 2024-04-20T11:36:11.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:11.758+0000 7fb0ba800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fb0bc0cf110 0x7fb0bc0b3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:11.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:11.758+0000 7fb0ba800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fb0bc0cf110 0x7fb0bc0b3ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35762/0 (socket says 172.21.15.152:35762) 2024-04-20T11:36:11.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:11.758+0000 7fb0ba800640 1 -- 172.21.15.152:0/2299304391 learned_addr learned my addr 172.21.15.152:0/2299304391 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:11.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:11.758+0000 7fb0b9e00640 1 --2- 172.21.15.152:0/2299304391 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb0bc06ee00 0x7fb0bc06f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:11.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:11.758+0000 7fb0b3e00640 1 --2- 172.21.15.152:0/2299304391 >> v2:172.21.15.175:3300/0 conn(0x7fb0bc15fbf0 0x7fb0bc15c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:11.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:11.758+0000 7fb0ba800640 1 -- 172.21.15.152:0/2299304391 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb0bc06ee00 msgr2=0x7fb0bc06f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:11.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:11.758+0000 7fb0ba800640 1 --2- 172.21.15.152:0/2299304391 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb0bc06ee00 0x7fb0bc06f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:11.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:11.758+0000 7fb0ba800640 1 -- 172.21.15.152:0/2299304391 >> v2:172.21.15.175:3300/0 conn(0x7fb0bc15fbf0 msgr2=0x7fb0bc15c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:11.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:11.758+0000 7fb0ba800640 1 --2- 172.21.15.152:0/2299304391 >> v2:172.21.15.175:3300/0 conn(0x7fb0bc15fbf0 0x7fb0bc15c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:11.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:11.758+0000 7fb0ba800640 1 -- 172.21.15.152:0/2299304391 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0bc0cfdf0 con 0x7fb0bc0cf110 2024-04-20T11:36:11.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:11.759+0000 7fb0b3e00640 1 --2- 172.21.15.152:0/2299304391 >> v2:172.21.15.175:3300/0 conn(0x7fb0bc15fbf0 0x7fb0bc15c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:36:11.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:11.760+0000 7fb0ba800640 1 --2- 172.21.15.152:0/2299304391 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb0bc0cf110 0x7fb0bc0b3ba0 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc073170 tx=0x7fb0b402e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1ae0bf64c7210bc server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:11.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:11.760+0000 7fb0b9400640 1 -- 172.21.15.152:0/2299304391 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b402ea80 con 0x7fb0bc0cf110 2024-04-20T11:36:11.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:11.760+0000 7fb0b9400640 1 -- 172.21.15.152:0/2299304391 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0b402ec20 con 0x7fb0bc0cf110 2024-04-20T11:36:11.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:11.760+0000 7fb0b9400640 1 -- 172.21.15.152:0/2299304391 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b4009600 con 0x7fb0bc0cf110 2024-04-20T11:36:11.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:11.761+0000 7fb0bb200640 1 -- 172.21.15.152:0/2299304391 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb0bc0cf110 msgr2=0x7fb0bc0b3ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:11.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:11.761+0000 7fb0bb200640 1 --2- 172.21.15.152:0/2299304391 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb0bc0cf110 0x7fb0bc0b3ba0 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc073170 tx=0x7fb0b402e0e0 comp rx=0 tx=0).stop 2024-04-20T11:36:11.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:11.761+0000 7fb0bb200640 1 -- 172.21.15.152:0/2299304391 shutdown_connections 2024-04-20T11:36:11.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:11.761+0000 7fb0bb200640 1 --2- 172.21.15.152:0/2299304391 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb0bc0cf110 0x7fb0bc0b3ba0 unknown :-1 s=CLOSED pgs=1965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:11.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:11.761+0000 7fb0bb200640 1 --2- 172.21.15.152:0/2299304391 >> v2:172.21.15.175:3300/0 conn(0x7fb0bc15fbf0 0x7fb0bc15c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:11.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:11.761+0000 7fb0bb200640 1 --2- 172.21.15.152:0/2299304391 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb0bc06ee00 0x7fb0bc06f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:11.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:11.761+0000 7fb0bb200640 1 -- 172.21.15.152:0/2299304391 >> 172.21.15.152:0/2299304391 conn(0x7fb0bc067ae0 msgr2=0x7fb0bc06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:11.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:11.761+0000 7fb0bb200640 1 -- 172.21.15.152:0/2299304391 shutdown_connections 2024-04-20T11:36:11.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:11.761+0000 7fb0bb200640 1 -- 172.21.15.152:0/2299304391 wait complete. 2024-04-20T11:36:11.762 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:11.762+0000 7fb0bb200640 1 Processor -- start 2024-04-20T11:36:11.762 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:11.763+0000 7fb0bb200640 1 -- start start 2024-04-20T11:36:11.765 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:11.764+0000 7fb0bb200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb0bc06ee00 0x7fb0bc13f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:11.765 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:11.764+0000 7fb0bb200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb0bc0cf110 0x7fb0bc142900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:11.765 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:11.764+0000 7fb0bb200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb0bc15fbf0 0x7fb0bc0af240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:11.765 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:11.764+0000 7fb0bb200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fb0bc15c8d0 con 0x7fb0bc15fbf0 2024-04-20T11:36:11.765 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:11.764+0000 7fb0bb200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb0bc15be70 con 0x7fb0bc06ee00 2024-04-20T11:36:11.765 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:11.764+0000 7fb0bb200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb0bc144970 con 0x7fb0bc0cf110 2024-04-20T11:36:11.765 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:11.764+0000 7fb0ba800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb0bc15fbf0 0x7fb0bc0af240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:11.765 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:11.764+0000 7fb0ba800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb0bc15fbf0 0x7fb0bc0af240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35778/0 (socket says 172.21.15.152:35778) 2024-04-20T11:36:11.765 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:11.764+0000 7fb0ba800640 1 -- 172.21.15.152:0/1400970504 learned_addr learned my addr 172.21.15.152:0/1400970504 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:11.765 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:11.764+0000 7fb0b3e00640 1 --2- 172.21.15.152:0/1400970504 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb0bc0cf110 0x7fb0bc142900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:11.765 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:11.764+0000 7fb0b9e00640 1 --2- 172.21.15.152:0/1400970504 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb0bc06ee00 0x7fb0bc13f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:11.765 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:11.764+0000 7fb0ba800640 1 -- 172.21.15.152:0/1400970504 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb0bc0cf110 msgr2=0x7fb0bc142900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:11.765 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:11.764+0000 7fb0ba800640 1 --2- 172.21.15.152:0/1400970504 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb0bc0cf110 0x7fb0bc142900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:11.766 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:11.764+0000 7fb0ba800640 1 -- 172.21.15.152:0/1400970504 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb0bc06ee00 msgr2=0x7fb0bc13f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:11.766 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:11.764+0000 7fb0ba800640 1 --2- 172.21.15.152:0/1400970504 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb0bc06ee00 0x7fb0bc13f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:11.766 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:11.764+0000 7fb0ba800640 1 -- 172.21.15.152:0/1400970504 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0bc1effb0 con 0x7fb0bc15fbf0 2024-04-20T11:36:11.766 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:11.764+0000 7fb0b3e00640 1 --2- 172.21.15.152:0/1400970504 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb0bc0cf110 0x7fb0bc142900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:36:11.766 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:11.765+0000 7fb0ba800640 1 --2- 172.21.15.152:0/1400970504 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb0bc15fbf0 0x7fb0bc0af240 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7fb0b4038040 tx=0x7fb0b4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:11.766 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:11.765+0000 7fb0b2000640 1 -- 172.21.15.152:0/1400970504 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b4045020 con 0x7fb0bc15fbf0 2024-04-20T11:36:11.766 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:11.765+0000 7fb0b2000640 1 -- 172.21.15.152:0/1400970504 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0b4040660 con 0x7fb0bc15fbf0 2024-04-20T11:36:11.766 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:11.765+0000 7fb0b2000640 1 -- 172.21.15.152:0/1400970504 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b402ea80 con 0x7fb0bc15fbf0 2024-04-20T11:36:11.766 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:11.765+0000 7fb0bb200640 1 -- 172.21.15.152:0/1400970504 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0bc1432c0 con 0x7fb0bc15fbf0 2024-04-20T11:36:11.766 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:11.765+0000 7fb0bb200640 1 -- 172.21.15.152:0/1400970504 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0bc0aad20 con 0x7fb0bc15fbf0 2024-04-20T11:36:11.767 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:11.766+0000 7fb0bb200640 1 -- 172.21.15.152:0/1400970504 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0bc075e60 con 0x7fb0bc15fbf0 2024-04-20T11:36:11.767 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:11.767+0000 7fb0b2000640 1 -- 172.21.15.152:0/1400970504 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 22) ==== 91601+0+0 (secure 0 0 0) 0x7fb0b4048020 con 0x7fb0bc15fbf0 2024-04-20T11:36:11.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:11.768+0000 7fb0b2000640 1 --2- 172.21.15.152:0/1400970504 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb09806cf20 0x7fb09806f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:11.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:11.768+0000 7fb0b2000640 1 -- 172.21.15.152:0/1400970504 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5709+0+0 (secure 0 0 0) 0x7fb0b40b11e0 con 0x7fb0bc15fbf0 2024-04-20T11:36:11.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:11.769+0000 7fb0b9e00640 1 --2- 172.21.15.152:0/1400970504 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb09806cf20 0x7fb09806f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:11.769 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:11.770+0000 7fb0b9e00640 1 --2- 172.21.15.152:0/1400970504 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb09806cf20 0x7fb09806f3e0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac006fd0 tx=0x7fb0ac008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:11.772 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:11.773+0000 7fb0b2000640 1 -- 172.21.15.152:0/1400970504 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fb0b404f030 con 0x7fb0bc15fbf0 2024-04-20T11:36:12.018 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:12.019+0000 7fb0bb200640 1 -- 172.21.15.152:0/1400970504 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fb0bc0b3490 con 0x7fb0bc15fbf0 2024-04-20T11:36:12.527 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:12.527+0000 7fb0b2000640 1 -- 172.21.15.152:0/1400970504 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v96) ==== 122+0+0 (secure 0 0 0) 0x7fb0b407eca0 con 0x7fb0bc15fbf0 2024-04-20T11:36:12.532 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:12.531+0000 7fb0bb200640 1 -- 172.21.15.152:0/1400970504 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb09806cf20 msgr2=0x7fb09806f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:12.532 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:12.531+0000 7fb0bb200640 1 --2- 172.21.15.152:0/1400970504 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb09806cf20 0x7fb09806f3e0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac006fd0 tx=0x7fb0ac008040 comp rx=0 tx=0).stop 2024-04-20T11:36:12.533 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:12.531+0000 7fb0bb200640 1 -- 172.21.15.152:0/1400970504 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb0bc15fbf0 msgr2=0x7fb0bc0af240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:12.533 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:12.531+0000 7fb0bb200640 1 --2- 172.21.15.152:0/1400970504 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb0bc15fbf0 0x7fb0bc0af240 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7fb0b4038040 tx=0x7fb0b4004a40 comp rx=0 tx=0).stop 2024-04-20T11:36:12.533 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:12.532+0000 7fb0bb200640 1 -- 172.21.15.152:0/1400970504 shutdown_connections 2024-04-20T11:36:12.533 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:12.532+0000 7fb0bb200640 1 --2- 172.21.15.152:0/1400970504 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb09806cf20 0x7fb09806f3e0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:12.533 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:12.532+0000 7fb0bb200640 1 --2- 172.21.15.152:0/1400970504 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb0bc15fbf0 0x7fb0bc0af240 unknown :-1 s=CLOSED pgs=1966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:12.533 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:12.532+0000 7fb0bb200640 1 --2- 172.21.15.152:0/1400970504 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb0bc0cf110 0x7fb0bc142900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:12.534 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:12.532+0000 7fb0bb200640 1 --2- 172.21.15.152:0/1400970504 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb0bc06ee00 0x7fb0bc13f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:12.534 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:12.532+0000 7fb0bb200640 1 -- 172.21.15.152:0/1400970504 >> 172.21.15.152:0/1400970504 conn(0x7fb0bc067ae0 msgr2=0x7fb0bc06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:12.534 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:12.532+0000 7fb0bb200640 1 -- 172.21.15.152:0/1400970504 shutdown_connections 2024-04-20T11:36:12.534 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:12.532+0000 7fb0bb200640 1 -- 172.21.15.152:0/1400970504 wait complete. 2024-04-20T11:36:12.553 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-20T11:36:12.553 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:36:12.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:12.759+0000 7f3504800640 1 Processor -- start 2024-04-20T11:36:12.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:12.759+0000 7f3504800640 1 -- start start 2024-04-20T11:36:12.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:12.761+0000 7f3504800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f34fc06fa50 0x7f34fc15cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:12.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:12.761+0000 7f3504800640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34fc0b5770 con 0x7f34fc15b5e0 2024-04-20T11:36:12.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:12.761+0000 7f3504800640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34fc06e060 con 0x7f34fc06eb40 2024-04-20T11:36:12.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:12.761+0000 7f3504800640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f34fc06de10 con 0x7f34fc06fa50 2024-04-20T11:36:12.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:12.761+0000 7f3501e00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f34fc15b5e0 0x7f34fc0b4c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:42426/0 (socket says 172.21.15.152:42426) 2024-04-20T11:36:12.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:12.761+0000 7f3501e00640 1 -- 172.21.15.152:0/656843356 learned_addr learned my addr 172.21.15.152:0/656843356 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:36:12.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:12.761+0000 7f3502800640 1 --2- 172.21.15.152:0/656843356 >> v2:172.21.15.152:3300/0 conn(0x7f34fc06fa50 0x7f34fc15cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:12.762 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:12.762+0000 7f3500a00640 1 -- 172.21.15.152:0/656843356 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 198912750 0 0) 0x7f34fc0b5770 con 0x7f34fc15b5e0 2024-04-20T11:36:12.762 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:12.762+0000 7f3500a00640 1 -- 172.21.15.152:0/656843356 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f34e4003520 con 0x7f34fc15b5e0 2024-04-20T11:36:12.762 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:12.762+0000 7f3502800640 1 -- 172.21.15.152:0/656843356 >> v1:172.21.15.175:6789/0 conn(0x7f34fc06eb40 legacy=0x7f34fc06f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:12.762 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:12.762+0000 7f3502800640 1 -- 172.21.15.152:0/656843356 >> v1:172.21.15.152:6789/0 conn(0x7f34fc15b5e0 legacy=0x7f34fc0b4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:12.762 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:12.762+0000 7f3502800640 1 -- 172.21.15.152:0/656843356 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34fc06e340 con 0x7f34fc06fa50 2024-04-20T11:36:12.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:12.763+0000 7f3502800640 1 --2- 172.21.15.152:0/656843356 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f34fc06fa50 0x7f34fc15cd70 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7f34fc06df90 tx=0x7f34f802bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40462fecfed12863 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:12.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:12.763+0000 7f3500a00640 1 -- 172.21.15.152:0/656843356 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f8039070 con 0x7f34fc06fa50 2024-04-20T11:36:12.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:12.763+0000 7f3500a00640 1 -- 172.21.15.152:0/656843356 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34f802cc50 con 0x7f34fc06fa50 2024-04-20T11:36:12.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:12.763+0000 7f3500a00640 1 -- 172.21.15.152:0/656843356 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f8034440 con 0x7f34fc06fa50 2024-04-20T11:36:12.764 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:12.764+0000 7f3504800640 1 -- 172.21.15.152:0/656843356 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f34fc06fa50 msgr2=0x7f34fc15cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:12.764 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:12.764+0000 7f3504800640 1 --2- 172.21.15.152:0/656843356 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f34fc06fa50 0x7f34fc15cd70 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7f34fc06df90 tx=0x7f34f802bea0 comp rx=0 tx=0).stop 2024-04-20T11:36:12.764 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:12.764+0000 7f3504800640 1 -- 172.21.15.152:0/656843356 shutdown_connections 2024-04-20T11:36:12.764 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:12.764+0000 7f3504800640 1 --2- 172.21.15.152:0/656843356 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f34fc06fa50 0x7f34fc15cd70 unknown :-1 s=CLOSED pgs=1968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:12.764 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:12.764+0000 7f3504800640 1 -- 172.21.15.152:0/656843356 >> 172.21.15.152:0/656843356 conn(0x7f34fc067cf0 msgr2=0x7f34fc06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:12.764 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:12.764+0000 7f3504800640 1 -- 172.21.15.152:0/656843356 shutdown_connections 2024-04-20T11:36:12.764 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:12.764+0000 7f3504800640 1 -- 172.21.15.152:0/656843356 wait complete. 2024-04-20T11:36:12.765 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:12.766+0000 7f3504800640 1 Processor -- start 2024-04-20T11:36:12.765 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:12.766+0000 7f3504800640 1 -- start start 2024-04-20T11:36:12.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:12.767+0000 7f3504800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f34fc06eb40 0x7f34fc1e57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:12.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:12.767+0000 7f3504800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f34fc06fa50 0x7f34fc0aea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:12.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:12.767+0000 7f3504800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f34fc15b5e0 0x7f34fc0aef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:12.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:12.767+0000 7f3504800640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f34fc0b42d0 con 0x7f34fc06fa50 2024-04-20T11:36:12.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:12.767+0000 7f3504800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f34fc15c9d0 con 0x7f34fc06eb40 2024-04-20T11:36:12.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:12.767+0000 7f3504800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f34fc0b5360 con 0x7f34fc15b5e0 2024-04-20T11:36:12.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:12.767+0000 7f3501400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f34fc06fa50 0x7f34fc0aea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:12.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:12.767+0000 7f3501400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f34fc06fa50 0x7f34fc0aea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35792/0 (socket says 172.21.15.152:35792) 2024-04-20T11:36:12.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:12.767+0000 7f3501400640 1 -- 172.21.15.152:0/1903618824 learned_addr learned my addr 172.21.15.152:0/1903618824 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:12.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:12.768+0000 7f3501e00640 1 --2- 172.21.15.152:0/1903618824 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f34fc06eb40 0x7f34fc1e57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:12.769 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:12.768+0000 7f3501400640 1 -- 172.21.15.152:0/1903618824 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f34fc15b5e0 msgr2=0x7f34fc0aef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:12.769 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:12.768+0000 7f3502800640 1 --2- 172.21.15.152:0/1903618824 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f34fc15b5e0 0x7f34fc0aef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:12.769 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:12.768+0000 7f3501400640 1 --2- 172.21.15.152:0/1903618824 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f34fc15b5e0 0x7f34fc0aef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:12.769 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:12.768+0000 7f3501400640 1 -- 172.21.15.152:0/1903618824 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f34fc06eb40 msgr2=0x7f34fc1e57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:12.769 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:12.768+0000 7f3501400640 1 --2- 172.21.15.152:0/1903618824 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f34fc06eb40 0x7f34fc1e57c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:12.769 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:12.768+0000 7f3501400640 1 -- 172.21.15.152:0/1903618824 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34fc149b30 con 0x7f34fc06fa50 2024-04-20T11:36:12.769 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:12.768+0000 7f3501e00640 1 --2- 172.21.15.152:0/1903618824 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f34fc06eb40 0x7f34fc1e57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:36:12.769 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:12.768+0000 7f3501400640 1 --2- 172.21.15.152:0/1903618824 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f34fc06fa50 0x7f34fc0aea30 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7f34ec00b420 tx=0x7f34ec00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:12.769 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:12.768+0000 7f34f6000640 1 -- 172.21.15.152:0/1903618824 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34ec011830 con 0x7f34fc06fa50 2024-04-20T11:36:12.769 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:12.768+0000 7f34f6000640 1 -- 172.21.15.152:0/1903618824 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34ec016070 con 0x7f34fc06fa50 2024-04-20T11:36:12.769 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:12.768+0000 7f34f6000640 1 -- 172.21.15.152:0/1903618824 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34ec011c10 con 0x7f34fc06fa50 2024-04-20T11:36:12.769 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:12.768+0000 7f3504800640 1 -- 172.21.15.152:0/1903618824 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34fc149e10 con 0x7f34fc06fa50 2024-04-20T11:36:12.769 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:12.768+0000 7f3504800640 1 -- 172.21.15.152:0/1903618824 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34fc14a3d0 con 0x7f34fc06fa50 2024-04-20T11:36:12.770 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:12.770+0000 7f3504800640 1 -- 172.21.15.152:0/1903618824 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34fc0689f0 con 0x7f34fc06fa50 2024-04-20T11:36:12.770 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:12.771+0000 7f34f6000640 1 -- 172.21.15.152:0/1903618824 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 22) ==== 91601+0+0 (secure 0 0 0) 0x7f34ec01b460 con 0x7f34fc06fa50 2024-04-20T11:36:12.771 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:12.771+0000 7f34f6000640 1 --2- 172.21.15.152:0/1903618824 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f34e006cf20 0x7f34e006f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:12.771 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:12.772+0000 7f34f6000640 1 -- 172.21.15.152:0/1903618824 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5709+0+0 (secure 0 0 0) 0x7f34ec08c1e0 con 0x7f34fc06fa50 2024-04-20T11:36:12.771 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:12.772+0000 7f3501e00640 1 --2- 172.21.15.152:0/1903618824 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f34e006cf20 0x7f34e006f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:12.773 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:12.773+0000 7f3501e00640 1 --2- 172.21.15.152:0/1903618824 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f34e006cf20 0x7f34e006f3e0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f34f0005880 tx=0x7f34f0009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:12.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:12.776+0000 7f34f6000640 1 -- 172.21.15.152:0/1903618824 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f34ec020020 con 0x7f34fc06fa50 2024-04-20T11:36:13.011 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.011+0000 7f3504800640 1 -- 172.21.15.152:0/1903618824 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f34fc06cd90 con 0x7f34fc06fa50 2024-04-20T11:36:13.012 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.013+0000 7f34f6000640 1 -- 172.21.15.152:0/1903618824 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) ==== 94+0+5209 (secure 0 0 0) 0x7f34ec059c90 con 0x7f34fc06fa50 2024-04-20T11:36:13.013 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:36:13.013 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":96,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6275,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6834/721066448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":721066448},{"type":"v1","addr":"172.21.15.175:6834","nonce":721066448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":91},{"gid":9483,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6834/3026417634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":3026417634},{"type":"v1","addr":"172.21.15.152:6834","nonce":3026417634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":90},{"gid":9486,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/3399678659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6833","nonce":3399678659},{"type":"v1","addr":"172.21.15.152:6835","nonce":3399678659}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":91}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T11:36:08.919586+0000","modified":"2024-04-20T11:36:12.020458+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9501},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9501":{"gid":9501,"name":"b","rank":0,"incarnation":92,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6835/189289100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6833","nonce":189289100},{"type":"v1","addr":"172.21.15.175:6835","nonce":189289100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9501,"qdb_cluster":[9501]},"id":11}]} 2024-04-20T11:36:13.013 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 96 2024-04-20T11:36:13.015 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.015+0000 7f3504800640 1 -- 172.21.15.152:0/1903618824 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f34e006cf20 msgr2=0x7f34e006f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:13.015 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.015+0000 7f3504800640 1 --2- 172.21.15.152:0/1903618824 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f34e006cf20 0x7f34e006f3e0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f34f0005880 tx=0x7f34f0009040 comp rx=0 tx=0).stop 2024-04-20T11:36:13.015 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.015+0000 7f3504800640 1 -- 172.21.15.152:0/1903618824 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f34fc06fa50 msgr2=0x7f34fc0aea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:13.015 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.015+0000 7f3504800640 1 --2- 172.21.15.152:0/1903618824 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f34fc06fa50 0x7f34fc0aea30 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7f34ec00b420 tx=0x7f34ec00b8e0 comp rx=0 tx=0).stop 2024-04-20T11:36:13.015 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.016+0000 7f3504800640 1 -- 172.21.15.152:0/1903618824 shutdown_connections 2024-04-20T11:36:13.016 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.016+0000 7f3504800640 1 --2- 172.21.15.152:0/1903618824 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f34e006cf20 0x7f34e006f3e0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:13.016 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.016+0000 7f3504800640 1 --2- 172.21.15.152:0/1903618824 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f34fc15b5e0 0x7f34fc0aef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:13.016 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.016+0000 7f3504800640 1 --2- 172.21.15.152:0/1903618824 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f34fc06fa50 0x7f34fc0aea30 unknown :-1 s=CLOSED pgs=1969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:13.016 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.016+0000 7f3504800640 1 --2- 172.21.15.152:0/1903618824 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f34fc06eb40 0x7f34fc1e57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:13.016 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.016+0000 7f3504800640 1 -- 172.21.15.152:0/1903618824 >> 172.21.15.152:0/1903618824 conn(0x7f34fc067cf0 msgr2=0x7f34fc0b29e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:13.016 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.016+0000 7f3504800640 1 -- 172.21.15.152:0/1903618824 shutdown_connections 2024-04-20T11:36:13.016 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.016+0000 7f3504800640 1 -- 172.21.15.152:0/1903618824 wait complete. 2024-04-20T11:36:13.033 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:36:13.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.226+0000 7f6847a00640 1 Processor -- start 2024-04-20T11:36:13.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.227+0000 7f6847a00640 1 -- start start 2024-04-20T11:36:13.227 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.228+0000 7f6847a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f684015e9b0 0x7f68400d1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:13.227 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.228+0000 7f6847a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f68400d4a90 0x7f68400d1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:13.227 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.228+0000 7f6847a00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68400cf470 con 0x7f68400d15c0 2024-04-20T11:36:13.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.228+0000 7f6847a00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f68400d0010 con 0x7f68400d4a90 2024-04-20T11:36:13.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.228+0000 7f6847a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f68400cf800 con 0x7f684015e9b0 2024-04-20T11:36:13.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.228+0000 7f6845a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f68400d4a90 0x7f68400d1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:13.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.228+0000 7f6845a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f68400d4a90 0x7f68400d1a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35800/0 (socket says 172.21.15.152:35800) 2024-04-20T11:36:13.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.228+0000 7f6845a00640 1 -- 172.21.15.152:0/3518318469 learned_addr learned my addr 172.21.15.152:0/3518318469 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:13.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.228+0000 7f6845000640 1 --2- 172.21.15.152:0/3518318469 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f684015e9b0 0x7f68400d1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:13.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.228+0000 7f6845a00640 1 -- 172.21.15.152:0/3518318469 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f684015e9b0 msgr2=0x7f68400d1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:13.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.228+0000 7f6845a00640 1 --2- 172.21.15.152:0/3518318469 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f684015e9b0 0x7f68400d1080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:13.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.228+0000 7f6845a00640 1 -- 172.21.15.152:0/3518318469 >> v1:172.21.15.175:6789/0 conn(0x7f68400d15c0 legacy=0x7f6840162610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:13.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.228+0000 7f6845a00640 1 -- 172.21.15.152:0/3518318469 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68400cfa50 con 0x7f68400d4a90 2024-04-20T11:36:13.230 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.230+0000 7f6845a00640 1 --2- 172.21.15.152:0/3518318469 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f68400d4a90 0x7f68400d1a20 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7f6830009870 tx=0x7f683002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7482e315a80b6478 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:13.230 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.230+0000 7f683f400640 1 -- 172.21.15.152:0/3518318469 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68300317c0 con 0x7f68400d4a90 2024-04-20T11:36:13.230 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.230+0000 7f683f400640 1 -- 172.21.15.152:0/3518318469 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6830037070 con 0x7f68400d4a90 2024-04-20T11:36:13.230 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.230+0000 7f683f400640 1 -- 172.21.15.152:0/3518318469 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6830031b80 con 0x7f68400d4a90 2024-04-20T11:36:13.230 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.230+0000 7f6847a00640 1 -- 172.21.15.152:0/3518318469 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f68400d4a90 msgr2=0x7f68400d1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:13.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.230+0000 7f6847a00640 1 --2- 172.21.15.152:0/3518318469 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f68400d4a90 0x7f68400d1a20 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7f6830009870 tx=0x7f683002b860 comp rx=0 tx=0).stop 2024-04-20T11:36:13.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.230+0000 7f6847a00640 1 -- 172.21.15.152:0/3518318469 shutdown_connections 2024-04-20T11:36:13.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.230+0000 7f6847a00640 1 --2- 172.21.15.152:0/3518318469 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f68400d4a90 0x7f68400d1a20 unknown :-1 s=CLOSED pgs=1970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:13.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.230+0000 7f6847a00640 1 --2- 172.21.15.152:0/3518318469 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f684015e9b0 0x7f68400d1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:13.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.230+0000 7f6847a00640 1 -- 172.21.15.152:0/3518318469 >> 172.21.15.152:0/3518318469 conn(0x7f684006cc70 msgr2=0x7f684006d080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:13.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.231+0000 7f6847a00640 1 -- 172.21.15.152:0/3518318469 shutdown_connections 2024-04-20T11:36:13.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.231+0000 7f6847a00640 1 -- 172.21.15.152:0/3518318469 wait complete. 2024-04-20T11:36:13.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.232+0000 7f6847a00640 1 Processor -- start 2024-04-20T11:36:13.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.232+0000 7f6847a00640 1 -- start start 2024-04-20T11:36:13.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.233+0000 7f6847a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68400d15c0 0x7f68400c8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:13.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.233+0000 7f6847a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f68400d4a90 0x7f6840152c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:13.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.233+0000 7f6847a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f684015e9b0 0x7f68401f7b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:13.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.233+0000 7f6847a00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f68400b3e20 con 0x7f684015e9b0 2024-04-20T11:36:13.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.233+0000 7f6847a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f68400b3ca0 con 0x7f68400d15c0 2024-04-20T11:36:13.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.233+0000 7f6847a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f684015ed90 con 0x7f68400d4a90 2024-04-20T11:36:13.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.234+0000 7f6845a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f684015e9b0 0x7f68401f7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:13.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.234+0000 7f6845a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f684015e9b0 0x7f68401f7b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35802/0 (socket says 172.21.15.152:35802) 2024-04-20T11:36:13.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.234+0000 7f6845a00640 1 -- 172.21.15.152:0/3872352903 learned_addr learned my addr 172.21.15.152:0/3872352903 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:13.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.234+0000 7f683fe00640 1 --2- 172.21.15.152:0/3872352903 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f68400d4a90 0x7f6840152c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:13.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.234+0000 7f6845000640 1 --2- 172.21.15.152:0/3872352903 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68400d15c0 0x7f68400c8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:13.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.234+0000 7f6845a00640 1 -- 172.21.15.152:0/3872352903 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f68400d4a90 msgr2=0x7f6840152c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:13.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.234+0000 7f6845a00640 1 --2- 172.21.15.152:0/3872352903 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f68400d4a90 0x7f6840152c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:13.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.234+0000 7f6845a00640 1 -- 172.21.15.152:0/3872352903 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68400d15c0 msgr2=0x7f68400c8ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:13.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.234+0000 7f6845a00640 1 --2- 172.21.15.152:0/3872352903 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68400d15c0 0x7f68400c8ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:13.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.234+0000 7f6845a00640 1 -- 172.21.15.152:0/3872352903 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f684015f2c0 con 0x7f684015e9b0 2024-04-20T11:36:13.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.234+0000 7f683fe00640 1 --2- 172.21.15.152:0/3872352903 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f68400d4a90 0x7f6840152c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:36:13.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.235+0000 7f6845a00640 1 --2- 172.21.15.152:0/3872352903 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f684015e9b0 0x7f68401f7b50 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7f683002b8a0 tx=0x7f68300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:13.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.235+0000 7f683cc00640 1 -- 172.21.15.152:0/3872352903 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6830032860 con 0x7f684015e9b0 2024-04-20T11:36:13.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.235+0000 7f683cc00640 1 -- 172.21.15.152:0/3872352903 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6830032a20 con 0x7f684015e9b0 2024-04-20T11:36:13.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.235+0000 7f683cc00640 1 -- 172.21.15.152:0/3872352903 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6830032e10 con 0x7f684015e9b0 2024-04-20T11:36:13.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.235+0000 7f6847a00640 1 -- 172.21.15.152:0/3872352903 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68400c92c0 con 0x7f684015e9b0 2024-04-20T11:36:13.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.235+0000 7f6847a00640 1 -- 172.21.15.152:0/3872352903 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68400c9880 con 0x7f684015e9b0 2024-04-20T11:36:13.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.236+0000 7f6847a00640 1 -- 172.21.15.152:0/3872352903 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6840069100 con 0x7f684015e9b0 2024-04-20T11:36:13.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.237+0000 7f683cc00640 1 -- 172.21.15.152:0/3872352903 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 22) ==== 91601+0+0 (secure 0 0 0) 0x7f683004c060 con 0x7f684015e9b0 2024-04-20T11:36:13.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.238+0000 7f683cc00640 1 --2- 172.21.15.152:0/3872352903 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f681c06cf20 0x7f681c06f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:13.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.238+0000 7f683cc00640 1 -- 172.21.15.152:0/3872352903 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5709+0+0 (secure 0 0 0) 0x7f68300ae160 con 0x7f684015e9b0 2024-04-20T11:36:13.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.239+0000 7f6845000640 1 --2- 172.21.15.152:0/3872352903 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f681c06cf20 0x7f681c06f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:13.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.240+0000 7f6845000640 1 --2- 172.21.15.152:0/3872352903 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f681c06cf20 0x7f681c06f3e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f683800f510 tx=0x7f68380077b0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:13.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.242+0000 7f683cc00640 1 -- 172.21.15.152:0/3872352903 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f6830040070 con 0x7f684015e9b0 2024-04-20T11:36:13.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.427+0000 7f6847a00640 1 -- 172.21.15.152:0/3872352903 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f68400d2520 con 0x7f684015e9b0 2024-04-20T11:36:13.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.429+0000 7f683cc00640 1 -- 172.21.15.152:0/3872352903 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v142) ==== 74+0+16731 (secure 0 0 0) 0x7f683007bc10 con 0x7f684015e9b0 2024-04-20T11:36:13.429 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:36:13.430 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":142,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:36:10.484514+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:36:06.702770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"141","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-04-20T11:36:07.911633+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:36:13.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.432+0000 7f6847a00640 1 -- 172.21.15.152:0/3872352903 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f681c06cf20 msgr2=0x7f681c06f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:13.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.432+0000 7f6847a00640 1 --2- 172.21.15.152:0/3872352903 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f681c06cf20 0x7f681c06f3e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f683800f510 tx=0x7f68380077b0 comp rx=0 tx=0).stop 2024-04-20T11:36:13.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.432+0000 7f6847a00640 1 -- 172.21.15.152:0/3872352903 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f684015e9b0 msgr2=0x7f68401f7b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:13.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.432+0000 7f6847a00640 1 --2- 172.21.15.152:0/3872352903 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f684015e9b0 0x7f68401f7b50 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7f683002b8a0 tx=0x7f68300046c0 comp rx=0 tx=0).stop 2024-04-20T11:36:13.433 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.433+0000 7f6847a00640 1 -- 172.21.15.152:0/3872352903 shutdown_connections 2024-04-20T11:36:13.433 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.433+0000 7f6847a00640 1 --2- 172.21.15.152:0/3872352903 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f681c06cf20 0x7f681c06f3e0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:13.433 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.433+0000 7f6847a00640 1 --2- 172.21.15.152:0/3872352903 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f684015e9b0 0x7f68401f7b50 unknown :-1 s=CLOSED pgs=1971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:13.433 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.433+0000 7f6847a00640 1 --2- 172.21.15.152:0/3872352903 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f68400d4a90 0x7f6840152c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:13.433 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.433+0000 7f6847a00640 1 --2- 172.21.15.152:0/3872352903 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68400d15c0 0x7f68400c8ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:13.433 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.433+0000 7f6847a00640 1 -- 172.21.15.152:0/3872352903 >> 172.21.15.152:0/3872352903 conn(0x7f684006cc70 msgr2=0x7f68400d4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:13.433 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.433+0000 7f6847a00640 1 -- 172.21.15.152:0/3872352903 shutdown_connections 2024-04-20T11:36:13.433 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.433+0000 7f6847a00640 1 -- 172.21.15.152:0/3872352903 wait complete. 2024-04-20T11:36:13.451 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-20T11:36:13.452 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T11:36:13.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.655+0000 7f5d51800640 1 Processor -- start 2024-04-20T11:36:13.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.655+0000 7f5d51800640 1 -- start start 2024-04-20T11:36:13.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.657+0000 7f5d51800640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f5d4c145f60 0x7f5d4c06ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:13.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.657+0000 7f5d51800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5d4c0cd0d0 0x7f5d4c0cd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:13.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.657+0000 7f5d51800640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d4c146340 con 0x7f5d4c071fe0 2024-04-20T11:36:13.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.657+0000 7f5d51800640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f5d4c06e9f0 con 0x7f5d4c145f60 2024-04-20T11:36:13.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.657+0000 7f5d51800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5d4c06f3c0 con 0x7f5d4c0cd0d0 2024-04-20T11:36:13.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.658+0000 7f5d4a000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5d4c0cd0d0 0x7f5d4c0cd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:13.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.658+0000 7f5d4aa00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f5d4c145f60 0x7f5d4c06ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:13.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.658+0000 7f5d4b400640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7f5d4c071fe0 0x7f5d4c072470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.152:52524/0 (socket says 172.21.15.152:52524) 2024-04-20T11:36:13.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.658+0000 7f5d4a000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5d4c0cd0d0 0x7f5d4c0cd550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.152:46724/0 (socket says 172.21.15.152:46724) 2024-04-20T11:36:13.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.658+0000 7f5d4b400640 1 -- 172.21.15.152:0/3629319881 learned_addr learned my addr 172.21.15.152:0/3629319881 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:36:13.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.659+0000 7f5d4aa00640 1 -- 172.21.15.152:0/3629319881 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5d4c0cd0d0 msgr2=0x7f5d4c0cd550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:13.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.659+0000 7f5d4aa00640 1 --2- 172.21.15.152:0/3629319881 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5d4c0cd0d0 0x7f5d4c0cd550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:13.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.659+0000 7f5d4aa00640 1 -- 172.21.15.152:0/3629319881 >> v1:172.21.15.175:6789/0 conn(0x7f5d4c071fe0 legacy=0x7f5d4c072470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:13.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.659+0000 7f5d4aa00640 1 -- 172.21.15.152:0/3629319881 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d4c072d50 con 0x7f5d4c145f60 2024-04-20T11:36:13.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.659+0000 7f5d4a000640 1 --2- 172.21.15.152:0/3629319881 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5d4c0cd0d0 0x7f5d4c0cd550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:36:13.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.660+0000 7f5d4aa00640 1 --2- 172.21.15.152:0/3629319881 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d4c145f60 0x7f5d4c06ee80 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c06eb70 tx=0x7f5d3c02b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85b00e21bc218edb server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:13.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.660+0000 7f5d49600640 1 -- 172.21.15.152:0/3629319881 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d3c035020 con 0x7f5d4c145f60 2024-04-20T11:36:13.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.661+0000 7f5d49600640 1 -- 172.21.15.152:0/3629319881 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d3c031810 con 0x7f5d4c145f60 2024-04-20T11:36:13.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.661+0000 7f5d49600640 1 -- 172.21.15.152:0/3629319881 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d3c031be0 con 0x7f5d4c145f60 2024-04-20T11:36:13.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.661+0000 7f5d51800640 1 -- 172.21.15.152:0/3629319881 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d4c145f60 msgr2=0x7f5d4c06ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:13.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.661+0000 7f5d51800640 1 --2- 172.21.15.152:0/3629319881 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d4c145f60 0x7f5d4c06ee80 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c06eb70 tx=0x7f5d3c02b8b0 comp rx=0 tx=0).stop 2024-04-20T11:36:13.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.661+0000 7f5d51800640 1 -- 172.21.15.152:0/3629319881 shutdown_connections 2024-04-20T11:36:13.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.661+0000 7f5d51800640 1 --2- 172.21.15.152:0/3629319881 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5d4c0cd0d0 0x7f5d4c0cd550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:13.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.661+0000 7f5d51800640 1 --2- 172.21.15.152:0/3629319881 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d4c145f60 0x7f5d4c06ee80 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:13.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.661+0000 7f5d51800640 1 -- 172.21.15.152:0/3629319881 >> 172.21.15.152:0/3629319881 conn(0x7f5d4c067cf0 msgr2=0x7f5d4c06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:13.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.661+0000 7f5d51800640 1 -- 172.21.15.152:0/3629319881 shutdown_connections 2024-04-20T11:36:13.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.661+0000 7f5d51800640 1 -- 172.21.15.152:0/3629319881 wait complete. 2024-04-20T11:36:13.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.663+0000 7f5d51800640 1 Processor -- start 2024-04-20T11:36:13.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.663+0000 7f5d51800640 1 -- start start 2024-04-20T11:36:13.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.664+0000 7f5d51800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5d4c071fe0 0x7f5d4c1f2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:13.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.664+0000 7f5d51800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d4c0cd0d0 0x7f5d4c13cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:13.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.664+0000 7f5d51800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5d4c145f60 0x7f5d4c0c5d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:13.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.664+0000 7f5d51800640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f5d4c06e680 con 0x7f5d4c145f60 2024-04-20T11:36:13.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.664+0000 7f5d51800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5d4c06e3a0 con 0x7f5d4c0cd0d0 2024-04-20T11:36:13.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.664+0000 7f5d51800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5d4c0cf290 con 0x7f5d4c071fe0 2024-04-20T11:36:13.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.664+0000 7f5d4b400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5d4c145f60 0x7f5d4c0c5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:13.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.664+0000 7f5d4b400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5d4c145f60 0x7f5d4c0c5d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35812/0 (socket says 172.21.15.152:35812) 2024-04-20T11:36:13.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.664+0000 7f5d4b400640 1 -- 172.21.15.152:0/3691272213 learned_addr learned my addr 172.21.15.152:0/3691272213 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:13.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.665+0000 7f5d4aa00640 1 --2- 172.21.15.152:0/3691272213 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5d4c071fe0 0x7f5d4c1f2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:13.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.665+0000 7f5d4a000640 1 --2- 172.21.15.152:0/3691272213 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d4c0cd0d0 0x7f5d4c13cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:13.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.665+0000 7f5d4b400640 1 -- 172.21.15.152:0/3691272213 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5d4c071fe0 msgr2=0x7f5d4c1f2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:13.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.665+0000 7f5d4b400640 1 --2- 172.21.15.152:0/3691272213 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5d4c071fe0 0x7f5d4c1f2bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:13.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.665+0000 7f5d4b400640 1 -- 172.21.15.152:0/3691272213 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d4c0cd0d0 msgr2=0x7f5d4c13cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:13.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.665+0000 7f5d4b400640 1 --2- 172.21.15.152:0/3691272213 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d4c0cd0d0 0x7f5d4c13cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:13.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.665+0000 7f5d4b400640 1 -- 172.21.15.152:0/3691272213 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d4c1f3210 con 0x7f5d4c145f60 2024-04-20T11:36:13.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.665+0000 7f5d4a000640 1 --2- 172.21.15.152:0/3691272213 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d4c0cd0d0 0x7f5d4c13cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:36:13.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.665+0000 7f5d4b400640 1 --2- 172.21.15.152:0/3691272213 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5d4c145f60 0x7f5d4c0c5d80 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7f5d4400cbc0 tx=0x7f5d4400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:13.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.665+0000 7f5d42a00640 1 -- 172.21.15.152:0/3691272213 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d44017070 con 0x7f5d4c145f60 2024-04-20T11:36:13.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.665+0000 7f5d42a00640 1 -- 172.21.15.152:0/3691272213 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d4400ad40 con 0x7f5d4c145f60 2024-04-20T11:36:13.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.665+0000 7f5d51800640 1 -- 172.21.15.152:0/3691272213 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d4c1f34f0 con 0x7f5d4c145f60 2024-04-20T11:36:13.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.665+0000 7f5d42a00640 1 -- 172.21.15.152:0/3691272213 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d44012600 con 0x7f5d4c145f60 2024-04-20T11:36:13.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.665+0000 7f5d51800640 1 -- 172.21.15.152:0/3691272213 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d4c1f3ab0 con 0x7f5d4c145f60 2024-04-20T11:36:13.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.667+0000 7f5d51800640 1 -- 172.21.15.152:0/3691272213 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d4c0689f0 con 0x7f5d4c145f60 2024-04-20T11:36:13.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.668+0000 7f5d42a00640 1 -- 172.21.15.152:0/3691272213 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 22) ==== 91601+0+0 (secure 0 0 0) 0x7f5d4401a070 con 0x7f5d4c145f60 2024-04-20T11:36:13.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.668+0000 7f5d42a00640 1 --2- 172.21.15.152:0/3691272213 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f5d2c06cf20 0x7f5d2c06f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:13.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.668+0000 7f5d42a00640 1 -- 172.21.15.152:0/3691272213 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5709+0+0 (secure 0 0 0) 0x7f5d4408c390 con 0x7f5d4c145f60 2024-04-20T11:36:13.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.669+0000 7f5d4aa00640 1 --2- 172.21.15.152:0/3691272213 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f5d2c06cf20 0x7f5d2c06f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:13.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.670+0000 7f5d4aa00640 1 --2- 172.21.15.152:0/3691272213 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f5d2c06cf20 0x7f5d2c06f3e0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c036000 tx=0x7f5d3c03c040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:13.672 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.673+0000 7f5d42a00640 1 -- 172.21.15.152:0/3691272213 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f5d44059880 con 0x7f5d4c145f60 2024-04-20T11:36:13.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.869+0000 7f5d51800640 1 -- 172.21.15.152:0/3691272213 --> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5d4c06bd90 con 0x7f5d2c06cf20 2024-04-20T11:36:13.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.887+0000 7f5d42a00640 1 -- 172.21.15.152:0/3691272213 <== mgr.4109 v2:172.21.15.175:6836/4021250441 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247089 (secure 0 0 0) 0x7f5d4c06bd90 con 0x7f5d2c06cf20 2024-04-20T11:36:13.890 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:36:13.892 INFO:teuthology.orchestra.run.smithi152.stderr:dumped all 2024-04-20T11:36:13.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.893+0000 7f5d51800640 1 -- 172.21.15.152:0/3691272213 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f5d2c06cf20 msgr2=0x7f5d2c06f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:13.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.893+0000 7f5d51800640 1 --2- 172.21.15.152:0/3691272213 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f5d2c06cf20 0x7f5d2c06f3e0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c036000 tx=0x7f5d3c03c040 comp rx=0 tx=0).stop 2024-04-20T11:36:13.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.893+0000 7f5d51800640 1 -- 172.21.15.152:0/3691272213 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5d4c145f60 msgr2=0x7f5d4c0c5d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:13.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.893+0000 7f5d51800640 1 --2- 172.21.15.152:0/3691272213 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5d4c145f60 0x7f5d4c0c5d80 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7f5d4400cbc0 tx=0x7f5d4400a430 comp rx=0 tx=0).stop 2024-04-20T11:36:13.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.893+0000 7f5d51800640 1 -- 172.21.15.152:0/3691272213 shutdown_connections 2024-04-20T11:36:13.894 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.893+0000 7f5d51800640 1 --2- 172.21.15.152:0/3691272213 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f5d2c06cf20 0x7f5d2c06f3e0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:13.894 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.893+0000 7f5d51800640 1 --2- 172.21.15.152:0/3691272213 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5d4c145f60 0x7f5d4c0c5d80 unknown :-1 s=CLOSED pgs=1972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:13.894 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.893+0000 7f5d51800640 1 --2- 172.21.15.152:0/3691272213 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d4c0cd0d0 0x7f5d4c13cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:13.894 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.893+0000 7f5d51800640 1 --2- 172.21.15.152:0/3691272213 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5d4c071fe0 0x7f5d4c1f2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:13.894 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.893+0000 7f5d51800640 1 -- 172.21.15.152:0/3691272213 >> 172.21.15.152:0/3691272213 conn(0x7f5d4c067cf0 msgr2=0x7f5d4c0cc760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:13.894 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.893+0000 7f5d51800640 1 -- 172.21.15.152:0/3691272213 shutdown_connections 2024-04-20T11:36:13.894 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:13.893+0000 7f5d51800640 1 -- 172.21.15.152:0/3691272213 wait complete. 2024-04-20T11:36:13.913 INFO:teuthology.orchestra.run.smithi152.stdout:{"pg_ready":true,"pg_map":{"version":189,"stamp":"2024-04-20T11:36:12.681287+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":40,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":264284,"kb_used_data":14836,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754710436,"statfs":{"total":773094113280,"available":772823486464,"internally_reserved":0,"allocated":15192064,"data_stored":7349955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12746,"internal_metadata":255315510},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.244562"},"pg_stats":[{"pgid":"23.3f","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.469159+0000","last_change":"2024-04-20T11:36:08.469159+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.469159+0000","last_undegraded":"2024-04-20T11:36:08.469159+0000","last_fullsized":"2024-04-20T11:36:08.469159+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.3e","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.469300+0000","last_change":"2024-04-20T11:36:08.469300+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.469300+0000","last_undegraded":"2024-04-20T11:36:08.469300+0000","last_fullsized":"2024-04-20T11:36:08.469300+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3d","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.452592+0000","last_change":"2024-04-20T11:36:08.452592+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.452592+0000","last_undegraded":"2024-04-20T11:36:08.452592+0000","last_fullsized":"2024-04-20T11:36:08.452592+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.3c","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.473704+0000","last_change":"2024-04-20T11:36:08.473704+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.473704+0000","last_undegraded":"2024-04-20T11:36:08.473704+0000","last_fullsized":"2024-04-20T11:36:08.473704+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3b","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.464837+0000","last_change":"2024-04-20T11:36:08.464837+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.464837+0000","last_undegraded":"2024-04-20T11:36:08.464837+0000","last_fullsized":"2024-04-20T11:36:08.464837+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3a","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.465384+0000","last_change":"2024-04-20T11:36:08.465384+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.465384+0000","last_undegraded":"2024-04-20T11:36:08.465384+0000","last_fullsized":"2024-04-20T11:36:08.465384+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.39","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.463782+0000","last_change":"2024-04-20T11:36:08.463782+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.463782+0000","last_undegraded":"2024-04-20T11:36:08.463782+0000","last_fullsized":"2024-04-20T11:36:08.463782+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.38","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.465904+0000","last_change":"2024-04-20T11:36:08.465904+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.465904+0000","last_undegraded":"2024-04-20T11:36:08.465904+0000","last_fullsized":"2024-04-20T11:36:08.465904+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.37","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.472065+0000","last_change":"2024-04-20T11:36:08.472065+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.472065+0000","last_undegraded":"2024-04-20T11:36:08.472065+0000","last_fullsized":"2024-04-20T11:36:08.472065+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.36","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.468695+0000","last_change":"2024-04-20T11:36:08.468695+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.468695+0000","last_undegraded":"2024-04-20T11:36:08.468695+0000","last_fullsized":"2024-04-20T11:36:08.468695+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.35","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.471960+0000","last_change":"2024-04-20T11:36:08.471960+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.471960+0000","last_undegraded":"2024-04-20T11:36:08.471960+0000","last_fullsized":"2024-04-20T11:36:08.471960+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.34","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.472788+0000","last_change":"2024-04-20T11:36:08.472788+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.472788+0000","last_undegraded":"2024-04-20T11:36:08.472788+0000","last_fullsized":"2024-04-20T11:36:08.472788+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.33","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.451393+0000","last_change":"2024-04-20T11:36:08.451393+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.451393+0000","last_undegraded":"2024-04-20T11:36:08.451393+0000","last_fullsized":"2024-04-20T11:36:08.451393+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.32","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.451863+0000","last_change":"2024-04-20T11:36:08.451863+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.451863+0000","last_undegraded":"2024-04-20T11:36:08.451863+0000","last_fullsized":"2024-04-20T11:36:08.451863+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.31","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.469280+0000","last_change":"2024-04-20T11:36:08.469280+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.469280+0000","last_undegraded":"2024-04-20T11:36:08.469280+0000","last_fullsized":"2024-04-20T11:36:08.469280+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.30","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.464992+0000","last_change":"2024-04-20T11:36:08.464992+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.464992+0000","last_undegraded":"2024-04-20T11:36:08.464992+0000","last_fullsized":"2024-04-20T11:36:08.464992+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.2f","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.463046+0000","last_change":"2024-04-20T11:36:08.463046+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.463046+0000","last_undegraded":"2024-04-20T11:36:08.463046+0000","last_fullsized":"2024-04-20T11:36:08.463046+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2e","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.470932+0000","last_change":"2024-04-20T11:36:08.470932+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.470932+0000","last_undegraded":"2024-04-20T11:36:08.470932+0000","last_fullsized":"2024-04-20T11:36:08.470932+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2d","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.464842+0000","last_change":"2024-04-20T11:36:08.464842+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.464842+0000","last_undegraded":"2024-04-20T11:36:08.464842+0000","last_fullsized":"2024-04-20T11:36:08.464842+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2c","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.450798+0000","last_change":"2024-04-20T11:36:08.450798+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.450798+0000","last_undegraded":"2024-04-20T11:36:08.450798+0000","last_fullsized":"2024-04-20T11:36:08.450798+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.2b","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.462875+0000","last_change":"2024-04-20T11:36:08.462875+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.462875+0000","last_undegraded":"2024-04-20T11:36:08.462875+0000","last_fullsized":"2024-04-20T11:36:08.462875+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2a","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.469036+0000","last_change":"2024-04-20T11:36:08.469036+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.469036+0000","last_undegraded":"2024-04-20T11:36:08.469036+0000","last_fullsized":"2024-04-20T11:36:08.469036+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.29","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.462179+0000","last_change":"2024-04-20T11:36:08.462179+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.462179+0000","last_undegraded":"2024-04-20T11:36:08.462179+0000","last_fullsized":"2024-04-20T11:36:08.462179+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.28","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.465454+0000","last_change":"2024-04-20T11:36:08.465454+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.465454+0000","last_undegraded":"2024-04-20T11:36:08.465454+0000","last_fullsized":"2024-04-20T11:36:08.465454+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.27","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.449266+0000","last_change":"2024-04-20T11:36:08.449266+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.449266+0000","last_undegraded":"2024-04-20T11:36:08.449266+0000","last_fullsized":"2024-04-20T11:36:08.449266+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.26","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.463771+0000","last_change":"2024-04-20T11:36:08.463771+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.463771+0000","last_undegraded":"2024-04-20T11:36:08.463771+0000","last_fullsized":"2024-04-20T11:36:08.463771+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.25","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.451044+0000","last_change":"2024-04-20T11:36:08.451044+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.451044+0000","last_undegraded":"2024-04-20T11:36:08.451044+0000","last_fullsized":"2024-04-20T11:36:08.451044+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.24","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.473278+0000","last_change":"2024-04-20T11:36:08.473278+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.473278+0000","last_undegraded":"2024-04-20T11:36:08.473278+0000","last_fullsized":"2024-04-20T11:36:08.473278+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.23","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.450005+0000","last_change":"2024-04-20T11:36:08.450005+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.450005+0000","last_undegraded":"2024-04-20T11:36:08.450005+0000","last_fullsized":"2024-04-20T11:36:08.450005+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.22","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.452543+0000","last_change":"2024-04-20T11:36:08.452543+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.452543+0000","last_undegraded":"2024-04-20T11:36:08.452543+0000","last_fullsized":"2024-04-20T11:36:08.452543+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.21","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.456445+0000","last_change":"2024-04-20T11:36:08.456445+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.456445+0000","last_undegraded":"2024-04-20T11:36:08.456445+0000","last_fullsized":"2024-04-20T11:36:08.456445+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.20","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.457564+0000","last_change":"2024-04-20T11:36:08.457564+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.457564+0000","last_undegraded":"2024-04-20T11:36:08.457564+0000","last_fullsized":"2024-04-20T11:36:08.457564+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.463194+0000","last_change":"2024-04-20T11:36:08.463194+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.463194+0000","last_undegraded":"2024-04-20T11:36:08.463194+0000","last_fullsized":"2024-04-20T11:36:08.463194+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-20T11:36:08.485343+0000","last_change":"2024-04-20T11:36:08.483872+0000","last_active":"2024-04-20T11:36:08.485343+0000","last_peered":"2024-04-20T11:36:08.485343+0000","last_clean":"2024-04-20T11:36:08.485343+0000","last_became_active":"2024-04-20T11:36:08.483196+0000","last_became_peered":"2024-04-20T11:36:08.483196+0000","last_unstale":"2024-04-20T11:36:08.485343+0000","last_undegraded":"2024-04-20T11:36:08.485343+0000","last_fullsized":"2024-04-20T11:36:08.485343+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:03:08.918074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.457823+0000","last_change":"2024-04-20T11:36:08.457823+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.457823+0000","last_undegraded":"2024-04-20T11:36:08.457823+0000","last_fullsized":"2024-04-20T11:36:08.457823+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-20T11:36:08.479720+0000","last_change":"2024-04-20T11:36:08.478064+0000","last_active":"2024-04-20T11:36:08.479720+0000","last_peered":"2024-04-20T11:36:08.479720+0000","last_clean":"2024-04-20T11:36:08.479720+0000","last_became_active":"2024-04-20T11:36:08.477788+0000","last_became_peered":"2024-04-20T11:36:08.477788+0000","last_unstale":"2024-04-20T11:36:08.479720+0000","last_undegraded":"2024-04-20T11:36:08.479720+0000","last_fullsized":"2024-04-20T11:36:08.479720+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:03:16.423053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.461517+0000","last_change":"2024-04-20T11:36:08.461517+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.461517+0000","last_undegraded":"2024-04-20T11:36:08.461517+0000","last_fullsized":"2024-04-20T11:36:08.461517+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-20T11:36:08.481347+0000","last_change":"2024-04-20T11:36:08.479424+0000","last_active":"2024-04-20T11:36:08.481347+0000","last_peered":"2024-04-20T11:36:08.481347+0000","last_clean":"2024-04-20T11:36:08.481347+0000","last_became_active":"2024-04-20T11:36:08.479137+0000","last_became_peered":"2024-04-20T11:36:08.479137+0000","last_unstale":"2024-04-20T11:36:08.481347+0000","last_undegraded":"2024-04-20T11:36:08.481347+0000","last_fullsized":"2024-04-20T11:36:08.481347+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:20:15.264097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.449526+0000","last_change":"2024-04-20T11:36:08.449526+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.449526+0000","last_undegraded":"2024-04-20T11:36:08.449526+0000","last_fullsized":"2024-04-20T11:36:08.449526+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-20T11:36:08.482454+0000","last_change":"2024-04-20T11:36:08.481345+0000","last_active":"2024-04-20T11:36:08.482454+0000","last_peered":"2024-04-20T11:36:08.482454+0000","last_clean":"2024-04-20T11:36:08.482454+0000","last_became_active":"2024-04-20T11:36:08.480807+0000","last_became_peered":"2024-04-20T11:36:08.480807+0000","last_unstale":"2024-04-20T11:36:08.482454+0000","last_undegraded":"2024-04-20T11:36:08.482454+0000","last_fullsized":"2024-04-20T11:36:08.482454+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:27:21.500530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.464353+0000","last_change":"2024-04-20T11:36:08.464353+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.464353+0000","last_undegraded":"2024-04-20T11:36:08.464353+0000","last_fullsized":"2024-04-20T11:36:08.464353+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-20T11:36:08.477073+0000","last_change":"2024-04-20T11:36:08.473692+0000","last_active":"2024-04-20T11:36:08.477073+0000","last_peered":"2024-04-20T11:36:08.477073+0000","last_clean":"2024-04-20T11:36:08.477073+0000","last_became_active":"2024-04-20T11:36:08.473388+0000","last_became_peered":"2024-04-20T11:36:08.473388+0000","last_unstale":"2024-04-20T11:36:08.477073+0000","last_undegraded":"2024-04-20T11:36:08.477073+0000","last_fullsized":"2024-04-20T11:36:08.477073+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:29:38.601496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.466526+0000","last_change":"2024-04-20T11:36:08.466526+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.466526+0000","last_undegraded":"2024-04-20T11:36:08.466526+0000","last_fullsized":"2024-04-20T11:36:08.466526+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-20T11:36:08.473814+0000","last_change":"2024-04-20T11:36:08.472809+0000","last_active":"2024-04-20T11:36:08.473814+0000","last_peered":"2024-04-20T11:36:08.473814+0000","last_clean":"2024-04-20T11:36:08.473814+0000","last_became_active":"2024-04-20T11:36:08.472197+0000","last_became_peered":"2024-04-20T11:36:08.472197+0000","last_unstale":"2024-04-20T11:36:08.473814+0000","last_undegraded":"2024-04-20T11:36:08.473814+0000","last_fullsized":"2024-04-20T11:36:08.473814+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:36:09.558861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.465753+0000","last_change":"2024-04-20T11:36:08.465753+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.465753+0000","last_undegraded":"2024-04-20T11:36:08.465753+0000","last_fullsized":"2024-04-20T11:36:08.465753+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-20T11:36:08.483751+0000","last_change":"2024-04-20T11:36:08.481523+0000","last_active":"2024-04-20T11:36:08.483751+0000","last_peered":"2024-04-20T11:36:08.483751+0000","last_clean":"2024-04-20T11:36:08.483751+0000","last_became_active":"2024-04-20T11:36:08.481220+0000","last_became_peered":"2024-04-20T11:36:08.481220+0000","last_unstale":"2024-04-20T11:36:08.483751+0000","last_undegraded":"2024-04-20T11:36:08.483751+0000","last_fullsized":"2024-04-20T11:36:08.483751+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:20:15.264097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.461553+0000","last_change":"2024-04-20T11:36:08.461553+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.461553+0000","last_undegraded":"2024-04-20T11:36:08.461553+0000","last_fullsized":"2024-04-20T11:36:08.461553+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-20T11:36:08.465848+0000","last_change":"2024-04-20T11:36:08.459181+0000","last_active":"2024-04-20T11:36:08.465848+0000","last_peered":"2024-04-20T11:36:08.465848+0000","last_clean":"2024-04-20T11:36:08.465848+0000","last_became_active":"2024-04-20T11:36:08.455877+0000","last_became_peered":"2024-04-20T11:36:08.455877+0000","last_unstale":"2024-04-20T11:36:08.465848+0000","last_undegraded":"2024-04-20T11:36:08.465848+0000","last_fullsized":"2024-04-20T11:36:08.465848+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:53:28.067443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.469199+0000","last_change":"2024-04-20T11:36:08.469199+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.469199+0000","last_undegraded":"2024-04-20T11:36:08.469199+0000","last_fullsized":"2024-04-20T11:36:08.469199+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-20T11:36:08.473900+0000","last_change":"2024-04-20T11:36:08.472135+0000","last_active":"2024-04-20T11:36:08.473900+0000","last_peered":"2024-04-20T11:36:08.473900+0000","last_clean":"2024-04-20T11:36:08.473900+0000","last_became_active":"2024-04-20T11:36:08.471529+0000","last_became_peered":"2024-04-20T11:36:08.471529+0000","last_unstale":"2024-04-20T11:36:08.473900+0000","last_undegraded":"2024-04-20T11:36:08.473900+0000","last_fullsized":"2024-04-20T11:36:08.473900+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:39:02.425373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.457492+0000","last_change":"2024-04-20T11:36:08.457492+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.457492+0000","last_undegraded":"2024-04-20T11:36:08.457492+0000","last_fullsized":"2024-04-20T11:36:08.457492+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-20T11:36:08.478390+0000","last_change":"2024-04-20T11:36:08.474633+0000","last_active":"2024-04-20T11:36:08.478390+0000","last_peered":"2024-04-20T11:36:08.478390+0000","last_clean":"2024-04-20T11:36:08.478390+0000","last_became_active":"2024-04-20T11:36:08.474310+0000","last_became_peered":"2024-04-20T11:36:08.474310+0000","last_unstale":"2024-04-20T11:36:08.478390+0000","last_undegraded":"2024-04-20T11:36:08.478390+0000","last_fullsized":"2024-04-20T11:36:08.478390+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:44:15.518378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.15","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.455081+0000","last_change":"2024-04-20T11:36:08.455081+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.455081+0000","last_undegraded":"2024-04-20T11:36:08.455081+0000","last_fullsized":"2024-04-20T11:36:08.455081+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-20T11:36:08.482877+0000","last_change":"2024-04-20T11:36:08.481226+0000","last_active":"2024-04-20T11:36:08.482877+0000","last_peered":"2024-04-20T11:36:08.482877+0000","last_clean":"2024-04-20T11:36:08.482877+0000","last_became_active":"2024-04-20T11:36:08.480392+0000","last_became_peered":"2024-04-20T11:36:08.480392+0000","last_unstale":"2024-04-20T11:36:08.482877+0000","last_undegraded":"2024-04-20T11:36:08.482877+0000","last_fullsized":"2024-04-20T11:36:08.482877+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:49:26.076623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.458679+0000","last_change":"2024-04-20T11:36:08.458679+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.458679+0000","last_undegraded":"2024-04-20T11:36:08.458679+0000","last_fullsized":"2024-04-20T11:36:08.458679+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-20T11:36:08.482352+0000","last_change":"2024-04-20T11:36:08.481075+0000","last_active":"2024-04-20T11:36:08.482352+0000","last_peered":"2024-04-20T11:36:08.482352+0000","last_clean":"2024-04-20T11:36:08.482352+0000","last_became_active":"2024-04-20T11:36:08.480423+0000","last_became_peered":"2024-04-20T11:36:08.480423+0000","last_unstale":"2024-04-20T11:36:08.482352+0000","last_undegraded":"2024-04-20T11:36:08.482352+0000","last_fullsized":"2024-04-20T11:36:08.482352+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:42:49.736624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.458961+0000","last_change":"2024-04-20T11:36:08.458961+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.458961+0000","last_undegraded":"2024-04-20T11:36:08.458961+0000","last_fullsized":"2024-04-20T11:36:08.458961+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-20T11:36:08.472605+0000","last_change":"2024-04-20T11:36:08.469986+0000","last_active":"2024-04-20T11:36:08.472605+0000","last_peered":"2024-04-20T11:36:08.472605+0000","last_clean":"2024-04-20T11:36:08.472605+0000","last_became_active":"2024-04-20T11:36:08.469638+0000","last_became_peered":"2024-04-20T11:36:08.469638+0000","last_unstale":"2024-04-20T11:36:08.472605+0000","last_undegraded":"2024-04-20T11:36:08.472605+0000","last_fullsized":"2024-04-20T11:36:08.472605+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:29:33.311708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.456411+0000","last_change":"2024-04-20T11:36:08.456411+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.456411+0000","last_undegraded":"2024-04-20T11:36:08.456411+0000","last_fullsized":"2024-04-20T11:36:08.456411+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-20T11:36:08.478403+0000","last_change":"2024-04-20T11:36:08.477055+0000","last_active":"2024-04-20T11:36:08.478403+0000","last_peered":"2024-04-20T11:36:08.478403+0000","last_clean":"2024-04-20T11:36:08.478403+0000","last_became_active":"2024-04-20T11:36:08.476730+0000","last_became_peered":"2024-04-20T11:36:08.476730+0000","last_unstale":"2024-04-20T11:36:08.478403+0000","last_undegraded":"2024-04-20T11:36:08.478403+0000","last_fullsized":"2024-04-20T11:36:08.478403+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:11:58.515138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.456036+0000","last_change":"2024-04-20T11:36:08.456036+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.456036+0000","last_undegraded":"2024-04-20T11:36:08.456036+0000","last_fullsized":"2024-04-20T11:36:08.456036+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-20T11:36:08.479428+0000","last_change":"2024-04-20T11:36:08.476771+0000","last_active":"2024-04-20T11:36:08.479428+0000","last_peered":"2024-04-20T11:36:08.479428+0000","last_clean":"2024-04-20T11:36:08.479428+0000","last_became_active":"2024-04-20T11:36:08.476124+0000","last_became_peered":"2024-04-20T11:36:08.476124+0000","last_unstale":"2024-04-20T11:36:08.479428+0000","last_undegraded":"2024-04-20T11:36:08.479428+0000","last_fullsized":"2024-04-20T11:36:08.479428+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:56:27.390436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.470400+0000","last_change":"2024-04-20T11:36:08.470400+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.470400+0000","last_undegraded":"2024-04-20T11:36:08.470400+0000","last_fullsized":"2024-04-20T11:36:08.470400+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-20T11:36:08.482369+0000","last_change":"2024-04-20T11:36:08.479096+0000","last_active":"2024-04-20T11:36:08.482369+0000","last_peered":"2024-04-20T11:36:08.482369+0000","last_clean":"2024-04-20T11:36:08.482369+0000","last_became_active":"2024-04-20T11:36:08.478807+0000","last_became_peered":"2024-04-20T11:36:08.478807+0000","last_unstale":"2024-04-20T11:36:08.482369+0000","last_undegraded":"2024-04-20T11:36:08.482369+0000","last_fullsized":"2024-04-20T11:36:08.482369+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:55:53.939172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.468258+0000","last_change":"2024-04-20T11:36:08.468258+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.468258+0000","last_undegraded":"2024-04-20T11:36:08.468258+0000","last_fullsized":"2024-04-20T11:36:08.468258+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-20T11:36:08.480764+0000","last_change":"2024-04-20T11:36:08.475925+0000","last_active":"2024-04-20T11:36:08.480764+0000","last_peered":"2024-04-20T11:36:08.480764+0000","last_clean":"2024-04-20T11:36:08.480764+0000","last_became_active":"2024-04-20T11:36:08.474739+0000","last_became_peered":"2024-04-20T11:36:08.474739+0000","last_unstale":"2024-04-20T11:36:08.480764+0000","last_undegraded":"2024-04-20T11:36:08.480764+0000","last_fullsized":"2024-04-20T11:36:08.480764+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:45:57.739197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.472822+0000","last_change":"2024-04-20T11:36:08.472822+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.472822+0000","last_undegraded":"2024-04-20T11:36:08.472822+0000","last_fullsized":"2024-04-20T11:36:08.472822+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-20T11:36:08.485361+0000","last_change":"2024-04-20T11:36:08.484073+0000","last_active":"2024-04-20T11:36:08.485361+0000","last_peered":"2024-04-20T11:36:08.485361+0000","last_clean":"2024-04-20T11:36:08.485361+0000","last_became_active":"2024-04-20T11:36:08.483348+0000","last_became_peered":"2024-04-20T11:36:08.483348+0000","last_unstale":"2024-04-20T11:36:08.485361+0000","last_undegraded":"2024-04-20T11:36:08.485361+0000","last_fullsized":"2024-04-20T11:36:08.485361+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:40:48.346717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.456912+0000","last_change":"2024-04-20T11:36:08.456912+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.456912+0000","last_undegraded":"2024-04-20T11:36:08.456912+0000","last_fullsized":"2024-04-20T11:36:08.456912+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-20T11:36:08.484928+0000","last_change":"2024-04-20T11:36:08.481420+0000","last_active":"2024-04-20T11:36:08.484928+0000","last_peered":"2024-04-20T11:36:08.484928+0000","last_clean":"2024-04-20T11:36:08.484928+0000","last_became_active":"2024-04-20T11:36:08.481118+0000","last_became_peered":"2024-04-20T11:36:08.481118+0000","last_unstale":"2024-04-20T11:36:08.484928+0000","last_undegraded":"2024-04-20T11:36:08.484928+0000","last_fullsized":"2024-04-20T11:36:08.484928+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:53:28.067443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.450480+0000","last_change":"2024-04-20T11:36:08.450480+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.450480+0000","last_undegraded":"2024-04-20T11:36:08.450480+0000","last_fullsized":"2024-04-20T11:36:08.450480+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-20T11:36:08.481759+0000","last_change":"2024-04-20T11:36:08.479935+0000","last_active":"2024-04-20T11:36:08.481759+0000","last_peered":"2024-04-20T11:36:08.481759+0000","last_clean":"2024-04-20T11:36:08.481759+0000","last_became_active":"2024-04-20T11:36:08.479361+0000","last_became_peered":"2024-04-20T11:36:08.479361+0000","last_unstale":"2024-04-20T11:36:08.481759+0000","last_undegraded":"2024-04-20T11:36:08.481759+0000","last_fullsized":"2024-04-20T11:36:08.481759+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:14:53.523329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.455740+0000","last_change":"2024-04-20T11:36:08.455740+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.455740+0000","last_undegraded":"2024-04-20T11:36:08.455740+0000","last_fullsized":"2024-04-20T11:36:08.455740+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-20T11:36:08.481737+0000","last_change":"2024-04-20T11:36:08.480015+0000","last_active":"2024-04-20T11:36:08.481737+0000","last_peered":"2024-04-20T11:36:08.481737+0000","last_clean":"2024-04-20T11:36:08.481737+0000","last_became_active":"2024-04-20T11:36:08.479699+0000","last_became_peered":"2024-04-20T11:36:08.479699+0000","last_unstale":"2024-04-20T11:36:08.481737+0000","last_undegraded":"2024-04-20T11:36:08.481737+0000","last_fullsized":"2024-04-20T11:36:08.481737+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:55:26.964426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.465378+0000","last_change":"2024-04-20T11:36:08.465378+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.465378+0000","last_undegraded":"2024-04-20T11:36:08.465378+0000","last_fullsized":"2024-04-20T11:36:08.465378+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-20T11:36:08.479541+0000","last_change":"2024-04-20T11:36:08.477340+0000","last_active":"2024-04-20T11:36:08.479541+0000","last_peered":"2024-04-20T11:36:08.479541+0000","last_clean":"2024-04-20T11:36:08.479541+0000","last_became_active":"2024-04-20T11:36:08.476853+0000","last_became_peered":"2024-04-20T11:36:08.476853+0000","last_unstale":"2024-04-20T11:36:08.479541+0000","last_undegraded":"2024-04-20T11:36:08.479541+0000","last_fullsized":"2024-04-20T11:36:08.479541+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:34:41.635429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.458969+0000","last_change":"2024-04-20T11:36:08.458969+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.458969+0000","last_undegraded":"2024-04-20T11:36:08.458969+0000","last_fullsized":"2024-04-20T11:36:08.458969+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-20T11:36:08.484801+0000","last_change":"2024-04-20T11:36:08.481304+0000","last_active":"2024-04-20T11:36:08.484801+0000","last_peered":"2024-04-20T11:36:08.484801+0000","last_clean":"2024-04-20T11:36:08.484801+0000","last_became_active":"2024-04-20T11:36:08.480957+0000","last_became_peered":"2024-04-20T11:36:08.480957+0000","last_unstale":"2024-04-20T11:36:08.484801+0000","last_undegraded":"2024-04-20T11:36:08.484801+0000","last_fullsized":"2024-04-20T11:36:08.484801+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:51:22.490812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.464091+0000","last_change":"2024-04-20T11:36:08.464091+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.464091+0000","last_undegraded":"2024-04-20T11:36:08.464091+0000","last_fullsized":"2024-04-20T11:36:08.464091+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-20T11:36:08.482995+0000","last_change":"2024-04-20T11:36:08.482021+0000","last_active":"2024-04-20T11:36:08.482995+0000","last_peered":"2024-04-20T11:36:08.482995+0000","last_clean":"2024-04-20T11:36:08.482995+0000","last_became_active":"2024-04-20T11:36:08.481134+0000","last_became_peered":"2024-04-20T11:36:08.481134+0000","last_unstale":"2024-04-20T11:36:08.482995+0000","last_undegraded":"2024-04-20T11:36:08.482995+0000","last_fullsized":"2024-04-20T11:36:08.482995+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:44:23.567819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.455304+0000","last_change":"2024-04-20T11:36:08.455304+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.455304+0000","last_undegraded":"2024-04-20T11:36:08.455304+0000","last_fullsized":"2024-04-20T11:36:08.455304+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-20T11:36:08.475795+0000","last_change":"2024-04-20T11:36:08.471585+0000","last_active":"2024-04-20T11:36:08.475795+0000","last_peered":"2024-04-20T11:36:08.475795+0000","last_clean":"2024-04-20T11:36:08.475795+0000","last_became_active":"2024-04-20T11:36:08.471286+0000","last_became_peered":"2024-04-20T11:36:08.471286+0000","last_unstale":"2024-04-20T11:36:08.475795+0000","last_undegraded":"2024-04-20T11:36:08.475795+0000","last_fullsized":"2024-04-20T11:36:08.475795+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:43:13.653616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.455911+0000","last_change":"2024-04-20T11:36:08.455911+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.455911+0000","last_undegraded":"2024-04-20T11:36:08.455911+0000","last_fullsized":"2024-04-20T11:36:08.455911+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-20T11:36:08.485164+0000","last_change":"2024-04-20T11:36:08.483684+0000","last_active":"2024-04-20T11:36:08.485164+0000","last_peered":"2024-04-20T11:36:08.485164+0000","last_clean":"2024-04-20T11:36:08.485164+0000","last_became_active":"2024-04-20T11:36:08.483405+0000","last_became_peered":"2024-04-20T11:36:08.483405+0000","last_unstale":"2024-04-20T11:36:08.485164+0000","last_undegraded":"2024-04-20T11:36:08.485164+0000","last_fullsized":"2024-04-20T11:36:08.485164+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:14:53.523329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.462439+0000","last_change":"2024-04-20T11:36:08.462439+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.462439+0000","last_undegraded":"2024-04-20T11:36:08.462439+0000","last_fullsized":"2024-04-20T11:36:08.462439+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-20T11:36:08.475521+0000","last_change":"2024-04-20T11:36:08.472571+0000","last_active":"2024-04-20T11:36:08.475521+0000","last_peered":"2024-04-20T11:36:08.475521+0000","last_clean":"2024-04-20T11:36:08.475521+0000","last_became_active":"2024-04-20T11:36:08.470365+0000","last_became_peered":"2024-04-20T11:36:08.470365+0000","last_unstale":"2024-04-20T11:36:08.475521+0000","last_undegraded":"2024-04-20T11:36:08.475521+0000","last_fullsized":"2024-04-20T11:36:08.475521+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:07:05.531843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.458037+0000","last_change":"2024-04-20T11:36:08.458037+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.458037+0000","last_undegraded":"2024-04-20T11:36:08.458037+0000","last_fullsized":"2024-04-20T11:36:08.458037+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-20T11:36:08.476839+0000","last_change":"2024-04-20T11:36:08.471003+0000","last_active":"2024-04-20T11:36:08.476839+0000","last_peered":"2024-04-20T11:36:08.476839+0000","last_clean":"2024-04-20T11:36:08.476839+0000","last_became_active":"2024-04-20T11:36:08.470689+0000","last_became_peered":"2024-04-20T11:36:08.470689+0000","last_unstale":"2024-04-20T11:36:08.476839+0000","last_undegraded":"2024-04-20T11:36:08.476839+0000","last_fullsized":"2024-04-20T11:36:08.476839+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:59:18.133331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.457122+0000","last_change":"2024-04-20T11:36:08.457122+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.457122+0000","last_undegraded":"2024-04-20T11:36:08.457122+0000","last_fullsized":"2024-04-20T11:36:08.457122+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-20T11:36:08.473321+0000","last_change":"2024-04-20T11:36:08.472251+0000","last_active":"2024-04-20T11:36:08.473321+0000","last_peered":"2024-04-20T11:36:08.473321+0000","last_clean":"2024-04-20T11:36:08.473321+0000","last_became_active":"2024-04-20T11:36:08.469968+0000","last_became_peered":"2024-04-20T11:36:08.469968+0000","last_unstale":"2024-04-20T11:36:08.473321+0000","last_undegraded":"2024-04-20T11:36:08.473321+0000","last_fullsized":"2024-04-20T11:36:08.473321+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:55:30.024848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.457273+0000","last_change":"2024-04-20T11:36:08.457273+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.457273+0000","last_undegraded":"2024-04-20T11:36:08.457273+0000","last_fullsized":"2024-04-20T11:36:08.457273+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-20T11:36:08.481658+0000","last_change":"2024-04-20T11:36:08.475717+0000","last_active":"2024-04-20T11:36:08.481658+0000","last_peered":"2024-04-20T11:36:08.481658+0000","last_clean":"2024-04-20T11:36:08.481658+0000","last_became_active":"2024-04-20T11:36:08.475397+0000","last_became_peered":"2024-04-20T11:36:08.475397+0000","last_unstale":"2024-04-20T11:36:08.481658+0000","last_undegraded":"2024-04-20T11:36:08.481658+0000","last_fullsized":"2024-04-20T11:36:08.481658+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:34:41.635429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.448436+0000","last_change":"2024-04-20T11:36:08.448436+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.448436+0000","last_undegraded":"2024-04-20T11:36:08.448436+0000","last_fullsized":"2024-04-20T11:36:08.448436+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-20T11:36:08.471280+0000","last_change":"2024-04-20T11:36:08.467047+0000","last_active":"2024-04-20T11:36:08.471280+0000","last_peered":"2024-04-20T11:36:08.471280+0000","last_clean":"2024-04-20T11:36:08.471280+0000","last_became_active":"2024-04-20T11:36:08.466681+0000","last_became_peered":"2024-04-20T11:36:08.466681+0000","last_unstale":"2024-04-20T11:36:08.471280+0000","last_undegraded":"2024-04-20T11:36:08.471280+0000","last_fullsized":"2024-04-20T11:36:08.471280+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:49:42.311707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.464467+0000","last_change":"2024-04-20T11:36:08.464467+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.464467+0000","last_undegraded":"2024-04-20T11:36:08.464467+0000","last_fullsized":"2024-04-20T11:36:08.464467+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-20T11:36:08.477773+0000","last_change":"2024-04-20T11:36:08.473393+0000","last_active":"2024-04-20T11:36:08.477773+0000","last_peered":"2024-04-20T11:36:08.477773+0000","last_clean":"2024-04-20T11:36:08.477773+0000","last_became_active":"2024-04-20T11:36:08.473104+0000","last_became_peered":"2024-04-20T11:36:08.473104+0000","last_unstale":"2024-04-20T11:36:08.477773+0000","last_undegraded":"2024-04-20T11:36:08.477773+0000","last_fullsized":"2024-04-20T11:36:08.477773+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:52:32.970409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"72'128","reported_seq":423,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-20T11:36:08.472747+0000","last_change":"2024-04-20T11:26:48.661897+0000","last_active":"2024-04-20T11:36:08.472747+0000","last_peered":"2024-04-20T11:36:08.472747+0000","last_clean":"2024-04-20T11:36:08.472747+0000","last_became_active":"2024-04-20T11:26:48.656535+0000","last_became_peered":"2024-04-20T11:26:48.656535+0000","last_unstale":"2024-04-20T11:36:08.472747+0000","last_undegraded":"2024-04-20T11:36:08.472747+0000","last_fullsized":"2024-04-20T11:36:08.472747+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:47.543004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:47.543004+0000","last_clean_scrub_stamp":"2024-04-20T11:26:47.543004+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:58:35.527105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":23,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":22,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574966,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33468,"kb_used_data":2288,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338372,"statfs":{"total":96636764160,"available":96602492928,"internally_reserved":0,"allocated":2342912,"data_stored":1360736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574963,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32896,"kb_used_data":1708,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338944,"statfs":{"total":96636764160,"available":96603078656,"internally_reserved":0,"allocated":1748992,"data_stored":770368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574965,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32888,"kb_used_data":1708,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338952,"statfs":{"total":96636764160,"available":96603086848,"internally_reserved":0,"allocated":1748992,"data_stored":770368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574963,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32888,"kb_used_data":1708,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338952,"statfs":{"total":96636764160,"available":96603086848,"internally_reserved":0,"allocated":1748992,"data_stored":770368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574964,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32888,"kb_used_data":1708,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338952,"statfs":{"total":96636764160,"available":96603086848,"internally_reserved":0,"allocated":1748992,"data_stored":770368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574965,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32888,"kb_used_data":1708,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338952,"statfs":{"total":96636764160,"available":96603086848,"internally_reserved":0,"allocated":1748992,"data_stored":770368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574963,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32888,"kb_used_data":1708,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338952,"statfs":{"total":96636764160,"available":96603086848,"internally_reserved":0,"allocated":1748992,"data_stored":770368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574964,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33480,"kb_used_data":2300,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338360,"statfs":{"total":96636764160,"available":96602480640,"internally_reserved":0,"allocated":2355200,"data_stored":1367011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T11:36:13.918 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T11:36:14.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.159+0000 7facae800640 1 Processor -- start 2024-04-20T11:36:14.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.159+0000 7facae800640 1 -- start start 2024-04-20T11:36:14.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.161+0000 7facae800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faca813e530 0x7faca806ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:14.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.161+0000 7facae800640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7faca81414e0 0x7faca8071cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:14.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.161+0000 7facae800640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faca80cf0a0 con 0x7faca806f340 2024-04-20T11:36:14.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.161+0000 7facae800640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7faca8141b00 con 0x7faca81414e0 2024-04-20T11:36:14.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.161+0000 7facae800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7faca80cfc50 con 0x7faca813e530 2024-04-20T11:36:14.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.161+0000 7facad400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faca813e530 0x7faca806ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:14.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.162+0000 7facaca00640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7faca806f340 0x7faca8140dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.152:52530/0 (socket says 172.21.15.152:52530) 2024-04-20T11:36:14.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.162+0000 7facaca00640 1 -- 172.21.15.152:0/872404841 learned_addr learned my addr 172.21.15.152:0/872404841 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:36:14.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.162+0000 7facad400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faca813e530 0x7faca806ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.152:46740/0 (socket says 172.21.15.152:46740) 2024-04-20T11:36:14.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.162+0000 7facade00640 1 --2- 172.21.15.152:0/872404841 >> v2:172.21.15.175:3300/0 conn(0x7faca81414e0 0x7faca8071cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:14.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.162+0000 7facad400640 1 -- 172.21.15.152:0/872404841 >> v2:172.21.15.175:3300/0 conn(0x7faca81414e0 msgr2=0x7faca8071cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:14.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.162+0000 7facad400640 1 --2- 172.21.15.152:0/872404841 >> v2:172.21.15.175:3300/0 conn(0x7faca81414e0 0x7faca8071cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:14.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.162+0000 7facad400640 1 -- 172.21.15.152:0/872404841 >> v1:172.21.15.175:6789/0 conn(0x7faca806f340 legacy=0x7faca8140dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:14.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.162+0000 7facad400640 1 -- 172.21.15.152:0/872404841 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faca80d00e0 con 0x7faca813e530 2024-04-20T11:36:14.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.162+0000 7faca7e00640 1 -- 172.21.15.152:0/872404841 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3565544583 0 0) 0x7fac94001490 con 0x7faca806f340 2024-04-20T11:36:14.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.164+0000 7facad400640 1 --2- 172.21.15.152:0/872404841 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faca813e530 0x7faca806ee00 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fac980097a0 tx=0x7fac9802b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c0bc17f1f6c98f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:14.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.164+0000 7faca7e00640 1 -- 172.21.15.152:0/872404841 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac98035020 con 0x7faca813e530 2024-04-20T11:36:14.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.164+0000 7faca7e00640 1 -- 172.21.15.152:0/872404841 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac98031830 con 0x7faca813e530 2024-04-20T11:36:14.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.165+0000 7faca7e00640 1 -- 172.21.15.152:0/872404841 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac98031c00 con 0x7faca813e530 2024-04-20T11:36:14.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.165+0000 7facae800640 1 -- 172.21.15.152:0/872404841 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faca813e530 msgr2=0x7faca806ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:14.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.165+0000 7facae800640 1 --2- 172.21.15.152:0/872404841 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faca813e530 0x7faca806ee00 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fac980097a0 tx=0x7fac9802b8d0 comp rx=0 tx=0).stop 2024-04-20T11:36:14.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.165+0000 7facae800640 1 -- 172.21.15.152:0/872404841 shutdown_connections 2024-04-20T11:36:14.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.165+0000 7facae800640 1 --2- 172.21.15.152:0/872404841 >> v2:172.21.15.175:3300/0 conn(0x7faca81414e0 0x7faca8071cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:14.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.165+0000 7facae800640 1 --2- 172.21.15.152:0/872404841 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faca813e530 0x7faca806ee00 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:14.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.165+0000 7facae800640 1 -- 172.21.15.152:0/872404841 >> 172.21.15.152:0/872404841 conn(0x7faca8067ae0 msgr2=0x7faca806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:14.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.165+0000 7facae800640 1 -- 172.21.15.152:0/872404841 shutdown_connections 2024-04-20T11:36:14.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.165+0000 7facae800640 1 -- 172.21.15.152:0/872404841 wait complete. 2024-04-20T11:36:14.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.167+0000 7facae800640 1 Processor -- start 2024-04-20T11:36:14.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.167+0000 7facae800640 1 -- start start 2024-04-20T11:36:14.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.168+0000 7facae800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faca806f340 0x7faca813d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:14.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.168+0000 7facae800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faca813e530 0x7faca814de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:14.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.168+0000 7facae800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faca81414e0 0x7faca815a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:14.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.168+0000 7facae800640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7faca80cf8e0 con 0x7faca806f340 2024-04-20T11:36:14.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.168+0000 7facae800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7faca80cf600 con 0x7faca813e530 2024-04-20T11:36:14.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.168+0000 7facae800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7faca8072210 con 0x7faca81414e0 2024-04-20T11:36:14.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.169+0000 7facad400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faca806f340 0x7faca813d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:14.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.169+0000 7facad400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faca806f340 0x7faca813d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35822/0 (socket says 172.21.15.152:35822) 2024-04-20T11:36:14.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.169+0000 7facad400640 1 -- 172.21.15.152:0/3687015474 learned_addr learned my addr 172.21.15.152:0/3687015474 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:14.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.169+0000 7facade00640 1 --2- 172.21.15.152:0/3687015474 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faca81414e0 0x7faca815a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:14.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.169+0000 7facad400640 1 -- 172.21.15.152:0/3687015474 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faca81414e0 msgr2=0x7faca815a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:14.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.169+0000 7facad400640 1 --2- 172.21.15.152:0/3687015474 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faca81414e0 0x7faca815a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:14.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.169+0000 7facad400640 1 -- 172.21.15.152:0/3687015474 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faca813e530 msgr2=0x7faca814de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:14.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.169+0000 7facaca00640 1 --2- 172.21.15.152:0/3687015474 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faca813e530 0x7faca814de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:14.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.169+0000 7facad400640 1 --2- 172.21.15.152:0/3687015474 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faca813e530 0x7faca814de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:14.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.169+0000 7facad400640 1 -- 172.21.15.152:0/3687015474 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faca814e850 con 0x7faca806f340 2024-04-20T11:36:14.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.169+0000 7facaca00640 1 --2- 172.21.15.152:0/3687015474 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faca813e530 0x7faca814de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:36:14.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.170+0000 7facad400640 1 --2- 172.21.15.152:0/3687015474 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faca806f340 0x7faca813d870 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7fac980097a0 tx=0x7fac980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:14.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.170+0000 7faca5600640 1 -- 172.21.15.152:0/3687015474 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac98035070 con 0x7faca806f340 2024-04-20T11:36:14.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.170+0000 7faca5600640 1 -- 172.21.15.152:0/3687015474 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac98004230 con 0x7faca806f340 2024-04-20T11:36:14.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.170+0000 7faca5600640 1 -- 172.21.15.152:0/3687015474 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac9803a630 con 0x7faca806f340 2024-04-20T11:36:14.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.170+0000 7facae800640 1 -- 172.21.15.152:0/3687015474 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faca814eb30 con 0x7faca806f340 2024-04-20T11:36:14.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.170+0000 7facae800640 1 -- 172.21.15.152:0/3687015474 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faca80b2c90 con 0x7faca806f340 2024-04-20T11:36:14.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.171+0000 7facae800640 1 -- 172.21.15.152:0/3687015474 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faca8075e60 con 0x7faca806f340 2024-04-20T11:36:14.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.172+0000 7faca5600640 1 -- 172.21.15.152:0/3687015474 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 22) ==== 91601+0+0 (secure 0 0 0) 0x7fac98038020 con 0x7faca806f340 2024-04-20T11:36:14.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.174+0000 7faca5600640 1 --2- 172.21.15.152:0/3687015474 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fac8806cf20 0x7fac8806f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:14.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.174+0000 7faca5600640 1 -- 172.21.15.152:0/3687015474 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5709+0+0 (secure 0 0 0) 0x7fac980ae430 con 0x7faca806f340 2024-04-20T11:36:14.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.174+0000 7facaca00640 1 --2- 172.21.15.152:0/3687015474 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fac8806cf20 0x7fac8806f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:14.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.176+0000 7facaca00640 1 --2- 172.21.15.152:0/3687015474 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fac8806cf20 0x7fac8806f3e0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7faca814ddc0 tx=0x7fac94009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:14.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.179+0000 7faca5600640 1 -- 172.21.15.152:0/3687015474 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fac98040070 con 0x7faca806f340 2024-04-20T11:36:14.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.374+0000 7facae800640 1 -- 172.21.15.152:0/3687015474 --> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7faca814f050 con 0x7fac8806cf20 2024-04-20T11:36:14.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.393+0000 7faca5600640 1 -- 172.21.15.152:0/3687015474 <== mgr.4109 v2:172.21.15.175:6836/4021250441 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247089 (secure 0 0 0) 0x7faca814f050 con 0x7fac8806cf20 2024-04-20T11:36:14.396 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:36:14.398 INFO:teuthology.orchestra.run.smithi152.stderr:dumped all 2024-04-20T11:36:14.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.398+0000 7facae800640 1 -- 172.21.15.152:0/3687015474 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fac8806cf20 msgr2=0x7fac8806f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:14.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.398+0000 7facae800640 1 --2- 172.21.15.152:0/3687015474 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fac8806cf20 0x7fac8806f3e0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7faca814ddc0 tx=0x7fac94009040 comp rx=0 tx=0).stop 2024-04-20T11:36:14.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.398+0000 7facae800640 1 -- 172.21.15.152:0/3687015474 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faca806f340 msgr2=0x7faca813d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:14.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.398+0000 7facae800640 1 --2- 172.21.15.152:0/3687015474 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faca806f340 0x7faca813d870 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7fac980097a0 tx=0x7fac980046c0 comp rx=0 tx=0).stop 2024-04-20T11:36:14.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.398+0000 7facae800640 1 -- 172.21.15.152:0/3687015474 shutdown_connections 2024-04-20T11:36:14.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.398+0000 7facae800640 1 --2- 172.21.15.152:0/3687015474 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fac8806cf20 0x7fac8806f3e0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:14.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.398+0000 7facae800640 1 --2- 172.21.15.152:0/3687015474 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faca81414e0 0x7faca815a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:14.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.398+0000 7facae800640 1 --2- 172.21.15.152:0/3687015474 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faca813e530 0x7faca814de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:14.401 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.398+0000 7facae800640 1 --2- 172.21.15.152:0/3687015474 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faca806f340 0x7faca813d870 unknown :-1 s=CLOSED pgs=1973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:14.401 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.398+0000 7facae800640 1 -- 172.21.15.152:0/3687015474 >> 172.21.15.152:0/3687015474 conn(0x7faca8067ae0 msgr2=0x7faca815c7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:14.401 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.398+0000 7facae800640 1 -- 172.21.15.152:0/3687015474 shutdown_connections 2024-04-20T11:36:14.401 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.398+0000 7facae800640 1 -- 172.21.15.152:0/3687015474 wait complete. 2024-04-20T11:36:14.417 INFO:teuthology.orchestra.run.smithi152.stdout:{"pg_ready":true,"pg_map":{"version":189,"stamp":"2024-04-20T11:36:12.681287+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":40,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":264284,"kb_used_data":14836,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754710436,"statfs":{"total":773094113280,"available":772823486464,"internally_reserved":0,"allocated":15192064,"data_stored":7349955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12746,"internal_metadata":255315510},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.244562"},"pg_stats":[{"pgid":"23.3f","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.469159+0000","last_change":"2024-04-20T11:36:08.469159+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.469159+0000","last_undegraded":"2024-04-20T11:36:08.469159+0000","last_fullsized":"2024-04-20T11:36:08.469159+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.3e","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.469300+0000","last_change":"2024-04-20T11:36:08.469300+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.469300+0000","last_undegraded":"2024-04-20T11:36:08.469300+0000","last_fullsized":"2024-04-20T11:36:08.469300+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3d","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.452592+0000","last_change":"2024-04-20T11:36:08.452592+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.452592+0000","last_undegraded":"2024-04-20T11:36:08.452592+0000","last_fullsized":"2024-04-20T11:36:08.452592+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.3c","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.473704+0000","last_change":"2024-04-20T11:36:08.473704+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.473704+0000","last_undegraded":"2024-04-20T11:36:08.473704+0000","last_fullsized":"2024-04-20T11:36:08.473704+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3b","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.464837+0000","last_change":"2024-04-20T11:36:08.464837+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.464837+0000","last_undegraded":"2024-04-20T11:36:08.464837+0000","last_fullsized":"2024-04-20T11:36:08.464837+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3a","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.465384+0000","last_change":"2024-04-20T11:36:08.465384+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.465384+0000","last_undegraded":"2024-04-20T11:36:08.465384+0000","last_fullsized":"2024-04-20T11:36:08.465384+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.39","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.463782+0000","last_change":"2024-04-20T11:36:08.463782+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.463782+0000","last_undegraded":"2024-04-20T11:36:08.463782+0000","last_fullsized":"2024-04-20T11:36:08.463782+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.38","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.465904+0000","last_change":"2024-04-20T11:36:08.465904+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.465904+0000","last_undegraded":"2024-04-20T11:36:08.465904+0000","last_fullsized":"2024-04-20T11:36:08.465904+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.37","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.472065+0000","last_change":"2024-04-20T11:36:08.472065+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.472065+0000","last_undegraded":"2024-04-20T11:36:08.472065+0000","last_fullsized":"2024-04-20T11:36:08.472065+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.36","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.468695+0000","last_change":"2024-04-20T11:36:08.468695+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.468695+0000","last_undegraded":"2024-04-20T11:36:08.468695+0000","last_fullsized":"2024-04-20T11:36:08.468695+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.35","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.471960+0000","last_change":"2024-04-20T11:36:08.471960+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.471960+0000","last_undegraded":"2024-04-20T11:36:08.471960+0000","last_fullsized":"2024-04-20T11:36:08.471960+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.34","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.472788+0000","last_change":"2024-04-20T11:36:08.472788+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.472788+0000","last_undegraded":"2024-04-20T11:36:08.472788+0000","last_fullsized":"2024-04-20T11:36:08.472788+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.33","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.451393+0000","last_change":"2024-04-20T11:36:08.451393+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.451393+0000","last_undegraded":"2024-04-20T11:36:08.451393+0000","last_fullsized":"2024-04-20T11:36:08.451393+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.32","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.451863+0000","last_change":"2024-04-20T11:36:08.451863+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.451863+0000","last_undegraded":"2024-04-20T11:36:08.451863+0000","last_fullsized":"2024-04-20T11:36:08.451863+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.31","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.469280+0000","last_change":"2024-04-20T11:36:08.469280+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.469280+0000","last_undegraded":"2024-04-20T11:36:08.469280+0000","last_fullsized":"2024-04-20T11:36:08.469280+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.30","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.464992+0000","last_change":"2024-04-20T11:36:08.464992+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.464992+0000","last_undegraded":"2024-04-20T11:36:08.464992+0000","last_fullsized":"2024-04-20T11:36:08.464992+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.2f","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.463046+0000","last_change":"2024-04-20T11:36:08.463046+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.463046+0000","last_undegraded":"2024-04-20T11:36:08.463046+0000","last_fullsized":"2024-04-20T11:36:08.463046+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2e","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.470932+0000","last_change":"2024-04-20T11:36:08.470932+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.470932+0000","last_undegraded":"2024-04-20T11:36:08.470932+0000","last_fullsized":"2024-04-20T11:36:08.470932+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2d","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.464842+0000","last_change":"2024-04-20T11:36:08.464842+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.464842+0000","last_undegraded":"2024-04-20T11:36:08.464842+0000","last_fullsized":"2024-04-20T11:36:08.464842+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2c","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.450798+0000","last_change":"2024-04-20T11:36:08.450798+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.450798+0000","last_undegraded":"2024-04-20T11:36:08.450798+0000","last_fullsized":"2024-04-20T11:36:08.450798+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.2b","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.462875+0000","last_change":"2024-04-20T11:36:08.462875+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.462875+0000","last_undegraded":"2024-04-20T11:36:08.462875+0000","last_fullsized":"2024-04-20T11:36:08.462875+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2a","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.469036+0000","last_change":"2024-04-20T11:36:08.469036+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.469036+0000","last_undegraded":"2024-04-20T11:36:08.469036+0000","last_fullsized":"2024-04-20T11:36:08.469036+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.29","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.462179+0000","last_change":"2024-04-20T11:36:08.462179+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.462179+0000","last_undegraded":"2024-04-20T11:36:08.462179+0000","last_fullsized":"2024-04-20T11:36:08.462179+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.28","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.465454+0000","last_change":"2024-04-20T11:36:08.465454+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.465454+0000","last_undegraded":"2024-04-20T11:36:08.465454+0000","last_fullsized":"2024-04-20T11:36:08.465454+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.27","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.449266+0000","last_change":"2024-04-20T11:36:08.449266+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.449266+0000","last_undegraded":"2024-04-20T11:36:08.449266+0000","last_fullsized":"2024-04-20T11:36:08.449266+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.26","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.463771+0000","last_change":"2024-04-20T11:36:08.463771+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.463771+0000","last_undegraded":"2024-04-20T11:36:08.463771+0000","last_fullsized":"2024-04-20T11:36:08.463771+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.25","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.451044+0000","last_change":"2024-04-20T11:36:08.451044+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.451044+0000","last_undegraded":"2024-04-20T11:36:08.451044+0000","last_fullsized":"2024-04-20T11:36:08.451044+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.24","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.473278+0000","last_change":"2024-04-20T11:36:08.473278+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.473278+0000","last_undegraded":"2024-04-20T11:36:08.473278+0000","last_fullsized":"2024-04-20T11:36:08.473278+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.23","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.450005+0000","last_change":"2024-04-20T11:36:08.450005+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.450005+0000","last_undegraded":"2024-04-20T11:36:08.450005+0000","last_fullsized":"2024-04-20T11:36:08.450005+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.22","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.452543+0000","last_change":"2024-04-20T11:36:08.452543+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.452543+0000","last_undegraded":"2024-04-20T11:36:08.452543+0000","last_fullsized":"2024-04-20T11:36:08.452543+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.21","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.456445+0000","last_change":"2024-04-20T11:36:08.456445+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.456445+0000","last_undegraded":"2024-04-20T11:36:08.456445+0000","last_fullsized":"2024-04-20T11:36:08.456445+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.20","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.457564+0000","last_change":"2024-04-20T11:36:08.457564+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.457564+0000","last_undegraded":"2024-04-20T11:36:08.457564+0000","last_fullsized":"2024-04-20T11:36:08.457564+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.463194+0000","last_change":"2024-04-20T11:36:08.463194+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.463194+0000","last_undegraded":"2024-04-20T11:36:08.463194+0000","last_fullsized":"2024-04-20T11:36:08.463194+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-20T11:36:08.485343+0000","last_change":"2024-04-20T11:36:08.483872+0000","last_active":"2024-04-20T11:36:08.485343+0000","last_peered":"2024-04-20T11:36:08.485343+0000","last_clean":"2024-04-20T11:36:08.485343+0000","last_became_active":"2024-04-20T11:36:08.483196+0000","last_became_peered":"2024-04-20T11:36:08.483196+0000","last_unstale":"2024-04-20T11:36:08.485343+0000","last_undegraded":"2024-04-20T11:36:08.485343+0000","last_fullsized":"2024-04-20T11:36:08.485343+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:03:08.918074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.457823+0000","last_change":"2024-04-20T11:36:08.457823+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.457823+0000","last_undegraded":"2024-04-20T11:36:08.457823+0000","last_fullsized":"2024-04-20T11:36:08.457823+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-20T11:36:08.479720+0000","last_change":"2024-04-20T11:36:08.478064+0000","last_active":"2024-04-20T11:36:08.479720+0000","last_peered":"2024-04-20T11:36:08.479720+0000","last_clean":"2024-04-20T11:36:08.479720+0000","last_became_active":"2024-04-20T11:36:08.477788+0000","last_became_peered":"2024-04-20T11:36:08.477788+0000","last_unstale":"2024-04-20T11:36:08.479720+0000","last_undegraded":"2024-04-20T11:36:08.479720+0000","last_fullsized":"2024-04-20T11:36:08.479720+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:03:16.423053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.461517+0000","last_change":"2024-04-20T11:36:08.461517+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.461517+0000","last_undegraded":"2024-04-20T11:36:08.461517+0000","last_fullsized":"2024-04-20T11:36:08.461517+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-20T11:36:08.481347+0000","last_change":"2024-04-20T11:36:08.479424+0000","last_active":"2024-04-20T11:36:08.481347+0000","last_peered":"2024-04-20T11:36:08.481347+0000","last_clean":"2024-04-20T11:36:08.481347+0000","last_became_active":"2024-04-20T11:36:08.479137+0000","last_became_peered":"2024-04-20T11:36:08.479137+0000","last_unstale":"2024-04-20T11:36:08.481347+0000","last_undegraded":"2024-04-20T11:36:08.481347+0000","last_fullsized":"2024-04-20T11:36:08.481347+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:20:15.264097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.449526+0000","last_change":"2024-04-20T11:36:08.449526+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.449526+0000","last_undegraded":"2024-04-20T11:36:08.449526+0000","last_fullsized":"2024-04-20T11:36:08.449526+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-20T11:36:08.482454+0000","last_change":"2024-04-20T11:36:08.481345+0000","last_active":"2024-04-20T11:36:08.482454+0000","last_peered":"2024-04-20T11:36:08.482454+0000","last_clean":"2024-04-20T11:36:08.482454+0000","last_became_active":"2024-04-20T11:36:08.480807+0000","last_became_peered":"2024-04-20T11:36:08.480807+0000","last_unstale":"2024-04-20T11:36:08.482454+0000","last_undegraded":"2024-04-20T11:36:08.482454+0000","last_fullsized":"2024-04-20T11:36:08.482454+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:27:21.500530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.464353+0000","last_change":"2024-04-20T11:36:08.464353+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.464353+0000","last_undegraded":"2024-04-20T11:36:08.464353+0000","last_fullsized":"2024-04-20T11:36:08.464353+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-20T11:36:08.477073+0000","last_change":"2024-04-20T11:36:08.473692+0000","last_active":"2024-04-20T11:36:08.477073+0000","last_peered":"2024-04-20T11:36:08.477073+0000","last_clean":"2024-04-20T11:36:08.477073+0000","last_became_active":"2024-04-20T11:36:08.473388+0000","last_became_peered":"2024-04-20T11:36:08.473388+0000","last_unstale":"2024-04-20T11:36:08.477073+0000","last_undegraded":"2024-04-20T11:36:08.477073+0000","last_fullsized":"2024-04-20T11:36:08.477073+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:29:38.601496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.466526+0000","last_change":"2024-04-20T11:36:08.466526+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.466526+0000","last_undegraded":"2024-04-20T11:36:08.466526+0000","last_fullsized":"2024-04-20T11:36:08.466526+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-20T11:36:08.473814+0000","last_change":"2024-04-20T11:36:08.472809+0000","last_active":"2024-04-20T11:36:08.473814+0000","last_peered":"2024-04-20T11:36:08.473814+0000","last_clean":"2024-04-20T11:36:08.473814+0000","last_became_active":"2024-04-20T11:36:08.472197+0000","last_became_peered":"2024-04-20T11:36:08.472197+0000","last_unstale":"2024-04-20T11:36:08.473814+0000","last_undegraded":"2024-04-20T11:36:08.473814+0000","last_fullsized":"2024-04-20T11:36:08.473814+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:36:09.558861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.465753+0000","last_change":"2024-04-20T11:36:08.465753+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.465753+0000","last_undegraded":"2024-04-20T11:36:08.465753+0000","last_fullsized":"2024-04-20T11:36:08.465753+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-20T11:36:08.483751+0000","last_change":"2024-04-20T11:36:08.481523+0000","last_active":"2024-04-20T11:36:08.483751+0000","last_peered":"2024-04-20T11:36:08.483751+0000","last_clean":"2024-04-20T11:36:08.483751+0000","last_became_active":"2024-04-20T11:36:08.481220+0000","last_became_peered":"2024-04-20T11:36:08.481220+0000","last_unstale":"2024-04-20T11:36:08.483751+0000","last_undegraded":"2024-04-20T11:36:08.483751+0000","last_fullsized":"2024-04-20T11:36:08.483751+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:20:15.264097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.461553+0000","last_change":"2024-04-20T11:36:08.461553+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.461553+0000","last_undegraded":"2024-04-20T11:36:08.461553+0000","last_fullsized":"2024-04-20T11:36:08.461553+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-20T11:36:08.465848+0000","last_change":"2024-04-20T11:36:08.459181+0000","last_active":"2024-04-20T11:36:08.465848+0000","last_peered":"2024-04-20T11:36:08.465848+0000","last_clean":"2024-04-20T11:36:08.465848+0000","last_became_active":"2024-04-20T11:36:08.455877+0000","last_became_peered":"2024-04-20T11:36:08.455877+0000","last_unstale":"2024-04-20T11:36:08.465848+0000","last_undegraded":"2024-04-20T11:36:08.465848+0000","last_fullsized":"2024-04-20T11:36:08.465848+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:53:28.067443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.469199+0000","last_change":"2024-04-20T11:36:08.469199+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.469199+0000","last_undegraded":"2024-04-20T11:36:08.469199+0000","last_fullsized":"2024-04-20T11:36:08.469199+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-20T11:36:08.473900+0000","last_change":"2024-04-20T11:36:08.472135+0000","last_active":"2024-04-20T11:36:08.473900+0000","last_peered":"2024-04-20T11:36:08.473900+0000","last_clean":"2024-04-20T11:36:08.473900+0000","last_became_active":"2024-04-20T11:36:08.471529+0000","last_became_peered":"2024-04-20T11:36:08.471529+0000","last_unstale":"2024-04-20T11:36:08.473900+0000","last_undegraded":"2024-04-20T11:36:08.473900+0000","last_fullsized":"2024-04-20T11:36:08.473900+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:39:02.425373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.457492+0000","last_change":"2024-04-20T11:36:08.457492+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.457492+0000","last_undegraded":"2024-04-20T11:36:08.457492+0000","last_fullsized":"2024-04-20T11:36:08.457492+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-20T11:36:08.478390+0000","last_change":"2024-04-20T11:36:08.474633+0000","last_active":"2024-04-20T11:36:08.478390+0000","last_peered":"2024-04-20T11:36:08.478390+0000","last_clean":"2024-04-20T11:36:08.478390+0000","last_became_active":"2024-04-20T11:36:08.474310+0000","last_became_peered":"2024-04-20T11:36:08.474310+0000","last_unstale":"2024-04-20T11:36:08.478390+0000","last_undegraded":"2024-04-20T11:36:08.478390+0000","last_fullsized":"2024-04-20T11:36:08.478390+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:44:15.518378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.15","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.455081+0000","last_change":"2024-04-20T11:36:08.455081+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.455081+0000","last_undegraded":"2024-04-20T11:36:08.455081+0000","last_fullsized":"2024-04-20T11:36:08.455081+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-20T11:36:08.482877+0000","last_change":"2024-04-20T11:36:08.481226+0000","last_active":"2024-04-20T11:36:08.482877+0000","last_peered":"2024-04-20T11:36:08.482877+0000","last_clean":"2024-04-20T11:36:08.482877+0000","last_became_active":"2024-04-20T11:36:08.480392+0000","last_became_peered":"2024-04-20T11:36:08.480392+0000","last_unstale":"2024-04-20T11:36:08.482877+0000","last_undegraded":"2024-04-20T11:36:08.482877+0000","last_fullsized":"2024-04-20T11:36:08.482877+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:49:26.076623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.458679+0000","last_change":"2024-04-20T11:36:08.458679+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.458679+0000","last_undegraded":"2024-04-20T11:36:08.458679+0000","last_fullsized":"2024-04-20T11:36:08.458679+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-20T11:36:08.482352+0000","last_change":"2024-04-20T11:36:08.481075+0000","last_active":"2024-04-20T11:36:08.482352+0000","last_peered":"2024-04-20T11:36:08.482352+0000","last_clean":"2024-04-20T11:36:08.482352+0000","last_became_active":"2024-04-20T11:36:08.480423+0000","last_became_peered":"2024-04-20T11:36:08.480423+0000","last_unstale":"2024-04-20T11:36:08.482352+0000","last_undegraded":"2024-04-20T11:36:08.482352+0000","last_fullsized":"2024-04-20T11:36:08.482352+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:42:49.736624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.458961+0000","last_change":"2024-04-20T11:36:08.458961+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.458961+0000","last_undegraded":"2024-04-20T11:36:08.458961+0000","last_fullsized":"2024-04-20T11:36:08.458961+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-20T11:36:08.472605+0000","last_change":"2024-04-20T11:36:08.469986+0000","last_active":"2024-04-20T11:36:08.472605+0000","last_peered":"2024-04-20T11:36:08.472605+0000","last_clean":"2024-04-20T11:36:08.472605+0000","last_became_active":"2024-04-20T11:36:08.469638+0000","last_became_peered":"2024-04-20T11:36:08.469638+0000","last_unstale":"2024-04-20T11:36:08.472605+0000","last_undegraded":"2024-04-20T11:36:08.472605+0000","last_fullsized":"2024-04-20T11:36:08.472605+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:29:33.311708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.456411+0000","last_change":"2024-04-20T11:36:08.456411+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.456411+0000","last_undegraded":"2024-04-20T11:36:08.456411+0000","last_fullsized":"2024-04-20T11:36:08.456411+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-20T11:36:08.478403+0000","last_change":"2024-04-20T11:36:08.477055+0000","last_active":"2024-04-20T11:36:08.478403+0000","last_peered":"2024-04-20T11:36:08.478403+0000","last_clean":"2024-04-20T11:36:08.478403+0000","last_became_active":"2024-04-20T11:36:08.476730+0000","last_became_peered":"2024-04-20T11:36:08.476730+0000","last_unstale":"2024-04-20T11:36:08.478403+0000","last_undegraded":"2024-04-20T11:36:08.478403+0000","last_fullsized":"2024-04-20T11:36:08.478403+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:11:58.515138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.456036+0000","last_change":"2024-04-20T11:36:08.456036+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.456036+0000","last_undegraded":"2024-04-20T11:36:08.456036+0000","last_fullsized":"2024-04-20T11:36:08.456036+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-20T11:36:08.479428+0000","last_change":"2024-04-20T11:36:08.476771+0000","last_active":"2024-04-20T11:36:08.479428+0000","last_peered":"2024-04-20T11:36:08.479428+0000","last_clean":"2024-04-20T11:36:08.479428+0000","last_became_active":"2024-04-20T11:36:08.476124+0000","last_became_peered":"2024-04-20T11:36:08.476124+0000","last_unstale":"2024-04-20T11:36:08.479428+0000","last_undegraded":"2024-04-20T11:36:08.479428+0000","last_fullsized":"2024-04-20T11:36:08.479428+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:56:27.390436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.470400+0000","last_change":"2024-04-20T11:36:08.470400+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.470400+0000","last_undegraded":"2024-04-20T11:36:08.470400+0000","last_fullsized":"2024-04-20T11:36:08.470400+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-20T11:36:08.482369+0000","last_change":"2024-04-20T11:36:08.479096+0000","last_active":"2024-04-20T11:36:08.482369+0000","last_peered":"2024-04-20T11:36:08.482369+0000","last_clean":"2024-04-20T11:36:08.482369+0000","last_became_active":"2024-04-20T11:36:08.478807+0000","last_became_peered":"2024-04-20T11:36:08.478807+0000","last_unstale":"2024-04-20T11:36:08.482369+0000","last_undegraded":"2024-04-20T11:36:08.482369+0000","last_fullsized":"2024-04-20T11:36:08.482369+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:55:53.939172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.468258+0000","last_change":"2024-04-20T11:36:08.468258+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.468258+0000","last_undegraded":"2024-04-20T11:36:08.468258+0000","last_fullsized":"2024-04-20T11:36:08.468258+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-20T11:36:08.480764+0000","last_change":"2024-04-20T11:36:08.475925+0000","last_active":"2024-04-20T11:36:08.480764+0000","last_peered":"2024-04-20T11:36:08.480764+0000","last_clean":"2024-04-20T11:36:08.480764+0000","last_became_active":"2024-04-20T11:36:08.474739+0000","last_became_peered":"2024-04-20T11:36:08.474739+0000","last_unstale":"2024-04-20T11:36:08.480764+0000","last_undegraded":"2024-04-20T11:36:08.480764+0000","last_fullsized":"2024-04-20T11:36:08.480764+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:45:57.739197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.472822+0000","last_change":"2024-04-20T11:36:08.472822+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.472822+0000","last_undegraded":"2024-04-20T11:36:08.472822+0000","last_fullsized":"2024-04-20T11:36:08.472822+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-20T11:36:08.485361+0000","last_change":"2024-04-20T11:36:08.484073+0000","last_active":"2024-04-20T11:36:08.485361+0000","last_peered":"2024-04-20T11:36:08.485361+0000","last_clean":"2024-04-20T11:36:08.485361+0000","last_became_active":"2024-04-20T11:36:08.483348+0000","last_became_peered":"2024-04-20T11:36:08.483348+0000","last_unstale":"2024-04-20T11:36:08.485361+0000","last_undegraded":"2024-04-20T11:36:08.485361+0000","last_fullsized":"2024-04-20T11:36:08.485361+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:40:48.346717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.456912+0000","last_change":"2024-04-20T11:36:08.456912+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.456912+0000","last_undegraded":"2024-04-20T11:36:08.456912+0000","last_fullsized":"2024-04-20T11:36:08.456912+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-20T11:36:08.484928+0000","last_change":"2024-04-20T11:36:08.481420+0000","last_active":"2024-04-20T11:36:08.484928+0000","last_peered":"2024-04-20T11:36:08.484928+0000","last_clean":"2024-04-20T11:36:08.484928+0000","last_became_active":"2024-04-20T11:36:08.481118+0000","last_became_peered":"2024-04-20T11:36:08.481118+0000","last_unstale":"2024-04-20T11:36:08.484928+0000","last_undegraded":"2024-04-20T11:36:08.484928+0000","last_fullsized":"2024-04-20T11:36:08.484928+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:53:28.067443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.450480+0000","last_change":"2024-04-20T11:36:08.450480+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.450480+0000","last_undegraded":"2024-04-20T11:36:08.450480+0000","last_fullsized":"2024-04-20T11:36:08.450480+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-20T11:36:08.481759+0000","last_change":"2024-04-20T11:36:08.479935+0000","last_active":"2024-04-20T11:36:08.481759+0000","last_peered":"2024-04-20T11:36:08.481759+0000","last_clean":"2024-04-20T11:36:08.481759+0000","last_became_active":"2024-04-20T11:36:08.479361+0000","last_became_peered":"2024-04-20T11:36:08.479361+0000","last_unstale":"2024-04-20T11:36:08.481759+0000","last_undegraded":"2024-04-20T11:36:08.481759+0000","last_fullsized":"2024-04-20T11:36:08.481759+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:14:53.523329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.455740+0000","last_change":"2024-04-20T11:36:08.455740+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.455740+0000","last_undegraded":"2024-04-20T11:36:08.455740+0000","last_fullsized":"2024-04-20T11:36:08.455740+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-20T11:36:08.481737+0000","last_change":"2024-04-20T11:36:08.480015+0000","last_active":"2024-04-20T11:36:08.481737+0000","last_peered":"2024-04-20T11:36:08.481737+0000","last_clean":"2024-04-20T11:36:08.481737+0000","last_became_active":"2024-04-20T11:36:08.479699+0000","last_became_peered":"2024-04-20T11:36:08.479699+0000","last_unstale":"2024-04-20T11:36:08.481737+0000","last_undegraded":"2024-04-20T11:36:08.481737+0000","last_fullsized":"2024-04-20T11:36:08.481737+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:55:26.964426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.465378+0000","last_change":"2024-04-20T11:36:08.465378+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.465378+0000","last_undegraded":"2024-04-20T11:36:08.465378+0000","last_fullsized":"2024-04-20T11:36:08.465378+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-20T11:36:08.479541+0000","last_change":"2024-04-20T11:36:08.477340+0000","last_active":"2024-04-20T11:36:08.479541+0000","last_peered":"2024-04-20T11:36:08.479541+0000","last_clean":"2024-04-20T11:36:08.479541+0000","last_became_active":"2024-04-20T11:36:08.476853+0000","last_became_peered":"2024-04-20T11:36:08.476853+0000","last_unstale":"2024-04-20T11:36:08.479541+0000","last_undegraded":"2024-04-20T11:36:08.479541+0000","last_fullsized":"2024-04-20T11:36:08.479541+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:34:41.635429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.458969+0000","last_change":"2024-04-20T11:36:08.458969+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.458969+0000","last_undegraded":"2024-04-20T11:36:08.458969+0000","last_fullsized":"2024-04-20T11:36:08.458969+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-20T11:36:08.484801+0000","last_change":"2024-04-20T11:36:08.481304+0000","last_active":"2024-04-20T11:36:08.484801+0000","last_peered":"2024-04-20T11:36:08.484801+0000","last_clean":"2024-04-20T11:36:08.484801+0000","last_became_active":"2024-04-20T11:36:08.480957+0000","last_became_peered":"2024-04-20T11:36:08.480957+0000","last_unstale":"2024-04-20T11:36:08.484801+0000","last_undegraded":"2024-04-20T11:36:08.484801+0000","last_fullsized":"2024-04-20T11:36:08.484801+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:51:22.490812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.464091+0000","last_change":"2024-04-20T11:36:08.464091+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.464091+0000","last_undegraded":"2024-04-20T11:36:08.464091+0000","last_fullsized":"2024-04-20T11:36:08.464091+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-20T11:36:08.482995+0000","last_change":"2024-04-20T11:36:08.482021+0000","last_active":"2024-04-20T11:36:08.482995+0000","last_peered":"2024-04-20T11:36:08.482995+0000","last_clean":"2024-04-20T11:36:08.482995+0000","last_became_active":"2024-04-20T11:36:08.481134+0000","last_became_peered":"2024-04-20T11:36:08.481134+0000","last_unstale":"2024-04-20T11:36:08.482995+0000","last_undegraded":"2024-04-20T11:36:08.482995+0000","last_fullsized":"2024-04-20T11:36:08.482995+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:44:23.567819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.455304+0000","last_change":"2024-04-20T11:36:08.455304+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.455304+0000","last_undegraded":"2024-04-20T11:36:08.455304+0000","last_fullsized":"2024-04-20T11:36:08.455304+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-20T11:36:08.475795+0000","last_change":"2024-04-20T11:36:08.471585+0000","last_active":"2024-04-20T11:36:08.475795+0000","last_peered":"2024-04-20T11:36:08.475795+0000","last_clean":"2024-04-20T11:36:08.475795+0000","last_became_active":"2024-04-20T11:36:08.471286+0000","last_became_peered":"2024-04-20T11:36:08.471286+0000","last_unstale":"2024-04-20T11:36:08.475795+0000","last_undegraded":"2024-04-20T11:36:08.475795+0000","last_fullsized":"2024-04-20T11:36:08.475795+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:43:13.653616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.455911+0000","last_change":"2024-04-20T11:36:08.455911+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.455911+0000","last_undegraded":"2024-04-20T11:36:08.455911+0000","last_fullsized":"2024-04-20T11:36:08.455911+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-20T11:36:08.485164+0000","last_change":"2024-04-20T11:36:08.483684+0000","last_active":"2024-04-20T11:36:08.485164+0000","last_peered":"2024-04-20T11:36:08.485164+0000","last_clean":"2024-04-20T11:36:08.485164+0000","last_became_active":"2024-04-20T11:36:08.483405+0000","last_became_peered":"2024-04-20T11:36:08.483405+0000","last_unstale":"2024-04-20T11:36:08.485164+0000","last_undegraded":"2024-04-20T11:36:08.485164+0000","last_fullsized":"2024-04-20T11:36:08.485164+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:14:53.523329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.462439+0000","last_change":"2024-04-20T11:36:08.462439+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.462439+0000","last_undegraded":"2024-04-20T11:36:08.462439+0000","last_fullsized":"2024-04-20T11:36:08.462439+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-20T11:36:08.475521+0000","last_change":"2024-04-20T11:36:08.472571+0000","last_active":"2024-04-20T11:36:08.475521+0000","last_peered":"2024-04-20T11:36:08.475521+0000","last_clean":"2024-04-20T11:36:08.475521+0000","last_became_active":"2024-04-20T11:36:08.470365+0000","last_became_peered":"2024-04-20T11:36:08.470365+0000","last_unstale":"2024-04-20T11:36:08.475521+0000","last_undegraded":"2024-04-20T11:36:08.475521+0000","last_fullsized":"2024-04-20T11:36:08.475521+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:07:05.531843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.458037+0000","last_change":"2024-04-20T11:36:08.458037+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.458037+0000","last_undegraded":"2024-04-20T11:36:08.458037+0000","last_fullsized":"2024-04-20T11:36:08.458037+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-20T11:36:08.476839+0000","last_change":"2024-04-20T11:36:08.471003+0000","last_active":"2024-04-20T11:36:08.476839+0000","last_peered":"2024-04-20T11:36:08.476839+0000","last_clean":"2024-04-20T11:36:08.476839+0000","last_became_active":"2024-04-20T11:36:08.470689+0000","last_became_peered":"2024-04-20T11:36:08.470689+0000","last_unstale":"2024-04-20T11:36:08.476839+0000","last_undegraded":"2024-04-20T11:36:08.476839+0000","last_fullsized":"2024-04-20T11:36:08.476839+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:59:18.133331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.457122+0000","last_change":"2024-04-20T11:36:08.457122+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.457122+0000","last_undegraded":"2024-04-20T11:36:08.457122+0000","last_fullsized":"2024-04-20T11:36:08.457122+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-20T11:36:08.473321+0000","last_change":"2024-04-20T11:36:08.472251+0000","last_active":"2024-04-20T11:36:08.473321+0000","last_peered":"2024-04-20T11:36:08.473321+0000","last_clean":"2024-04-20T11:36:08.473321+0000","last_became_active":"2024-04-20T11:36:08.469968+0000","last_became_peered":"2024-04-20T11:36:08.469968+0000","last_unstale":"2024-04-20T11:36:08.473321+0000","last_undegraded":"2024-04-20T11:36:08.473321+0000","last_fullsized":"2024-04-20T11:36:08.473321+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:55:30.024848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.457273+0000","last_change":"2024-04-20T11:36:08.457273+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.457273+0000","last_undegraded":"2024-04-20T11:36:08.457273+0000","last_fullsized":"2024-04-20T11:36:08.457273+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-20T11:36:08.481658+0000","last_change":"2024-04-20T11:36:08.475717+0000","last_active":"2024-04-20T11:36:08.481658+0000","last_peered":"2024-04-20T11:36:08.481658+0000","last_clean":"2024-04-20T11:36:08.481658+0000","last_became_active":"2024-04-20T11:36:08.475397+0000","last_became_peered":"2024-04-20T11:36:08.475397+0000","last_unstale":"2024-04-20T11:36:08.481658+0000","last_undegraded":"2024-04-20T11:36:08.481658+0000","last_fullsized":"2024-04-20T11:36:08.481658+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:34:41.635429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.448436+0000","last_change":"2024-04-20T11:36:08.448436+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.448436+0000","last_undegraded":"2024-04-20T11:36:08.448436+0000","last_fullsized":"2024-04-20T11:36:08.448436+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-20T11:36:08.471280+0000","last_change":"2024-04-20T11:36:08.467047+0000","last_active":"2024-04-20T11:36:08.471280+0000","last_peered":"2024-04-20T11:36:08.471280+0000","last_clean":"2024-04-20T11:36:08.471280+0000","last_became_active":"2024-04-20T11:36:08.466681+0000","last_became_peered":"2024-04-20T11:36:08.466681+0000","last_unstale":"2024-04-20T11:36:08.471280+0000","last_undegraded":"2024-04-20T11:36:08.471280+0000","last_fullsized":"2024-04-20T11:36:08.471280+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:49:42.311707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":1,"reported_epoch":140,"state":"creating+peering","last_fresh":"2024-04-20T11:36:08.464467+0000","last_change":"2024-04-20T11:36:08.464467+0000","last_active":"2024-04-20T11:36:08.433765+0000","last_peered":"2024-04-20T11:36:08.433765+0000","last_clean":"2024-04-20T11:36:08.433765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:08.464467+0000","last_undegraded":"2024-04-20T11:36:08.464467+0000","last_fullsized":"2024-04-20T11:36:08.464467+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-20T11:36:08.477773+0000","last_change":"2024-04-20T11:36:08.473393+0000","last_active":"2024-04-20T11:36:08.477773+0000","last_peered":"2024-04-20T11:36:08.477773+0000","last_clean":"2024-04-20T11:36:08.477773+0000","last_became_active":"2024-04-20T11:36:08.473104+0000","last_became_peered":"2024-04-20T11:36:08.473104+0000","last_unstale":"2024-04-20T11:36:08.477773+0000","last_undegraded":"2024-04-20T11:36:08.477773+0000","last_fullsized":"2024-04-20T11:36:08.477773+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:52:32.970409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"72'128","reported_seq":423,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-20T11:36:08.472747+0000","last_change":"2024-04-20T11:26:48.661897+0000","last_active":"2024-04-20T11:36:08.472747+0000","last_peered":"2024-04-20T11:36:08.472747+0000","last_clean":"2024-04-20T11:36:08.472747+0000","last_became_active":"2024-04-20T11:26:48.656535+0000","last_became_peered":"2024-04-20T11:26:48.656535+0000","last_unstale":"2024-04-20T11:36:08.472747+0000","last_undegraded":"2024-04-20T11:36:08.472747+0000","last_fullsized":"2024-04-20T11:36:08.472747+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:47.543004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:47.543004+0000","last_clean_scrub_stamp":"2024-04-20T11:26:47.543004+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:58:35.527105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":23,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":22,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574966,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33468,"kb_used_data":2288,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338372,"statfs":{"total":96636764160,"available":96602492928,"internally_reserved":0,"allocated":2342912,"data_stored":1360736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574963,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32896,"kb_used_data":1708,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338944,"statfs":{"total":96636764160,"available":96603078656,"internally_reserved":0,"allocated":1748992,"data_stored":770368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574965,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32888,"kb_used_data":1708,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338952,"statfs":{"total":96636764160,"available":96603086848,"internally_reserved":0,"allocated":1748992,"data_stored":770368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574963,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32888,"kb_used_data":1708,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338952,"statfs":{"total":96636764160,"available":96603086848,"internally_reserved":0,"allocated":1748992,"data_stored":770368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574964,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32888,"kb_used_data":1708,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338952,"statfs":{"total":96636764160,"available":96603086848,"internally_reserved":0,"allocated":1748992,"data_stored":770368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574965,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32888,"kb_used_data":1708,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338952,"statfs":{"total":96636764160,"available":96603086848,"internally_reserved":0,"allocated":1748992,"data_stored":770368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574963,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32888,"kb_used_data":1708,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338952,"statfs":{"total":96636764160,"available":96603086848,"internally_reserved":0,"allocated":1748992,"data_stored":770368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574964,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33480,"kb_used_data":2300,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338360,"statfs":{"total":96636764160,"available":96602480640,"internally_reserved":0,"allocated":2355200,"data_stored":1367011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T11:36:14.422 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T11:36:14.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.662+0000 7f2a83200640 1 Processor -- start 2024-04-20T11:36:14.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.662+0000 7f2a83200640 1 -- start start 2024-04-20T11:36:14.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.664+0000 7f2a83200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f2a8406ea20 0x7f2a840b4a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:14.664 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.664+0000 7f2a83200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a840b4f90 0x7f2a84162e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:14.664 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.664+0000 7f2a83200640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a840b5510 con 0x7f2a84071cd0 2024-04-20T11:36:14.664 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.664+0000 7f2a83200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f2a840ceec0 con 0x7f2a8406ea20 2024-04-20T11:36:14.664 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.664+0000 7f2a83200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2a8406dcf0 con 0x7f2a840b4f90 2024-04-20T11:36:14.664 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.664+0000 7f2a81e00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f2a84071cd0 0x7f2a840720b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:42428/0 (socket says 172.21.15.152:42428) 2024-04-20T11:36:14.664 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.664+0000 7f2a81e00640 1 -- 172.21.15.152:0/1341834160 learned_addr learned my addr 172.21.15.152:0/1341834160 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:36:14.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.665+0000 7f2a82800640 1 --2- 172.21.15.152:0/1341834160 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a840b4f90 0x7f2a84162e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:14.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.665+0000 7f2a81400640 1 --2- 172.21.15.152:0/1341834160 >> v2:172.21.15.175:3300/0 conn(0x7f2a8406ea20 0x7f2a840b4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:14.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.665+0000 7f2a80a00640 1 -- 172.21.15.152:0/1341834160 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4034543430 0 0) 0x7f2a840b5510 con 0x7f2a84071cd0 2024-04-20T11:36:14.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.665+0000 7f2a80a00640 1 -- 172.21.15.152:0/1341834160 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2a68003520 con 0x7f2a84071cd0 2024-04-20T11:36:14.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.665+0000 7f2a80a00640 1 -- 172.21.15.152:0/1341834160 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 830409370 0 0) 0x7f2a68003520 con 0x7f2a84071cd0 2024-04-20T11:36:14.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.665+0000 7f2a80a00640 1 -- 172.21.15.152:0/1341834160 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a840b4f90 msgr2=0x7f2a84162e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:14.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.665+0000 7f2a80a00640 1 --2- 172.21.15.152:0/1341834160 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a840b4f90 0x7f2a84162e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:14.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.665+0000 7f2a80a00640 1 -- 172.21.15.152:0/1341834160 >> v2:172.21.15.175:3300/0 conn(0x7f2a8406ea20 msgr2=0x7f2a840b4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:14.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.665+0000 7f2a80a00640 1 --2- 172.21.15.152:0/1341834160 >> v2:172.21.15.175:3300/0 conn(0x7f2a8406ea20 0x7f2a840b4a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:14.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.665+0000 7f2a80a00640 1 -- 172.21.15.152:0/1341834160 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a8406df40 con 0x7f2a84071cd0 2024-04-20T11:36:14.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.665+0000 7f2a80a00640 1 -- 172.21.15.152:0/1341834160 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f2a74001670 con 0x7f2a84071cd0 2024-04-20T11:36:14.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.665+0000 7f2a80a00640 1 -- 172.21.15.152:0/1341834160 >> v1:172.21.15.152:6789/0 conn(0x7f2a84071cd0 legacy=0x7f2a840720b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:14.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.665+0000 7f2a80a00640 1 --2- 172.21.15.152:0/1341834160 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2a680039b0 0x7f2a68003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:14.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.665+0000 7f2a80a00640 1 --2- 172.21.15.152:0/1341834160 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a68004800 0x7f2a68004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:14.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.665+0000 7f2a80a00640 1 --2- 172.21.15.152:0/1341834160 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a840b4f90 0x7f2a680051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:14.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.665+0000 7f2a80a00640 1 -- 172.21.15.152:0/1341834160 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f2a840ceec0 con 0x7f2a680039b0 2024-04-20T11:36:14.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.665+0000 7f2a80a00640 1 -- 172.21.15.152:0/1341834160 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2a8406dcf0 con 0x7f2a68004800 2024-04-20T11:36:14.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.665+0000 7f2a80a00640 1 -- 172.21.15.152:0/1341834160 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2a6800b9c0 con 0x7f2a840b4f90 2024-04-20T11:36:14.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.666+0000 7f2a81e00640 1 --2- 172.21.15.152:0/1341834160 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2a680039b0 0x7f2a68003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:14.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.666+0000 7f2a81400640 1 --2- 172.21.15.152:0/1341834160 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a68004800 0x7f2a68004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:14.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.666+0000 7f2a82800640 1 --2- 172.21.15.152:0/1341834160 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a840b4f90 0x7f2a680051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:14.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.666+0000 7f2a81e00640 1 -- 172.21.15.152:0/1341834160 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a840b4f90 msgr2=0x7f2a680051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:14.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.666+0000 7f2a81e00640 1 --2- 172.21.15.152:0/1341834160 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a840b4f90 0x7f2a680051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:14.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.666+0000 7f2a81e00640 1 -- 172.21.15.152:0/1341834160 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a68004800 msgr2=0x7f2a68004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:14.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.666+0000 7f2a81e00640 1 --2- 172.21.15.152:0/1341834160 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a68004800 0x7f2a68004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:14.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.666+0000 7f2a81e00640 1 -- 172.21.15.152:0/1341834160 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a68003520 con 0x7f2a680039b0 2024-04-20T11:36:14.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.667+0000 7f2a81e00640 1 --2- 172.21.15.152:0/1341834160 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2a680039b0 0x7f2a68003dd0 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7f2a7400c210 tx=0x7f2a68005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bea463513a72142 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:14.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.668+0000 7f2a80a00640 1 -- 172.21.15.152:0/1341834160 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a74007d50 con 0x7f2a680039b0 2024-04-20T11:36:14.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.668+0000 7f2a80a00640 1 -- 172.21.15.152:0/1341834160 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a74030070 con 0x7f2a680039b0 2024-04-20T11:36:14.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.668+0000 7f2a80a00640 1 -- 172.21.15.152:0/1341834160 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a7402cd80 con 0x7f2a680039b0 2024-04-20T11:36:14.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.668+0000 7f2a83200640 1 -- 172.21.15.152:0/1341834160 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2a680039b0 msgr2=0x7f2a68003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:14.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.668+0000 7f2a83200640 1 --2- 172.21.15.152:0/1341834160 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2a680039b0 0x7f2a68003dd0 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7f2a7400c210 tx=0x7f2a68005830 comp rx=0 tx=0).stop 2024-04-20T11:36:14.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.668+0000 7f2a82800640 1 -- 172.21.15.152:0/1341834160 reap_dead start 2024-04-20T11:36:14.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.668+0000 7f2a83200640 1 -- 172.21.15.152:0/1341834160 shutdown_connections 2024-04-20T11:36:14.670 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.668+0000 7f2a83200640 1 -- 172.21.15.152:0/1341834160 >> 172.21.15.152:0/1341834160 conn(0x7f2a84067ae0 msgr2=0x7f2a8406b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:14.670 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.668+0000 7f2a83200640 1 -- 172.21.15.152:0/1341834160 shutdown_connections 2024-04-20T11:36:14.670 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.668+0000 7f2a83200640 1 -- 172.21.15.152:0/1341834160 wait complete. 2024-04-20T11:36:14.670 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.670+0000 7f2a83200640 1 Processor -- start 2024-04-20T11:36:14.670 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.670+0000 7f2a83200640 1 -- start start 2024-04-20T11:36:14.672 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.671+0000 7f2a83200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a680039b0 0x7f2a84148360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:14.672 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.671+0000 7f2a83200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a8415ebc0 0x7f2a841654d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:14.672 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.671+0000 7f2a83200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2a8415f040 0x7f2a84155370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:14.672 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.671+0000 7f2a83200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f2a840b4180 con 0x7f2a8415f040 2024-04-20T11:36:14.672 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.671+0000 7f2a83200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2a8406e500 con 0x7f2a8415ebc0 2024-04-20T11:36:14.672 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.671+0000 7f2a83200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2a84162e50 con 0x7f2a680039b0 2024-04-20T11:36:14.672 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.672+0000 7f2a82800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2a8415f040 0x7f2a84155370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:14.672 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.672+0000 7f2a82800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2a8415f040 0x7f2a84155370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35842/0 (socket says 172.21.15.152:35842) 2024-04-20T11:36:14.672 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.672+0000 7f2a82800640 1 -- 172.21.15.152:0/2144592855 learned_addr learned my addr 172.21.15.152:0/2144592855 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:14.672 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.672+0000 7f2a81e00640 1 --2- 172.21.15.152:0/2144592855 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a680039b0 0x7f2a84148360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:14.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.672+0000 7f2a82800640 1 -- 172.21.15.152:0/2144592855 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a680039b0 msgr2=0x7f2a84148360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:14.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.672+0000 7f2a81400640 1 --2- 172.21.15.152:0/2144592855 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a8415ebc0 0x7f2a841654d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:14.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.672+0000 7f2a82800640 1 --2- 172.21.15.152:0/2144592855 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a680039b0 0x7f2a84148360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:14.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.672+0000 7f2a82800640 1 -- 172.21.15.152:0/2144592855 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a8415ebc0 msgr2=0x7f2a841654d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:14.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.672+0000 7f2a82800640 1 --2- 172.21.15.152:0/2144592855 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a8415ebc0 0x7f2a841654d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:14.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.672+0000 7f2a82800640 1 -- 172.21.15.152:0/2144592855 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a84164430 con 0x7f2a8415f040 2024-04-20T11:36:14.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.672+0000 7f2a82800640 1 --2- 172.21.15.152:0/2144592855 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2a8415f040 0x7f2a84155370 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7f2a841641f0 tx=0x7f2a7c0115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:14.674 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.673+0000 7f2a7a000640 1 -- 172.21.15.152:0/2144592855 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a7c003990 con 0x7f2a8415f040 2024-04-20T11:36:14.674 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.673+0000 7f2a7a000640 1 -- 172.21.15.152:0/2144592855 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a7c01b070 con 0x7f2a8415f040 2024-04-20T11:36:14.674 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.673+0000 7f2a7a000640 1 -- 172.21.15.152:0/2144592855 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a7c003b70 con 0x7f2a8415f040 2024-04-20T11:36:14.674 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.673+0000 7f2a83200640 1 -- 172.21.15.152:0/2144592855 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a84164710 con 0x7f2a8415f040 2024-04-20T11:36:14.674 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.673+0000 7f2a83200640 1 -- 172.21.15.152:0/2144592855 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a84164cd0 con 0x7f2a8415f040 2024-04-20T11:36:14.674 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.674+0000 7f2a83200640 1 -- 172.21.15.152:0/2144592855 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a840b37d0 con 0x7f2a8415f040 2024-04-20T11:36:14.675 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.675+0000 7f2a7a000640 1 -- 172.21.15.152:0/2144592855 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 22) ==== 91601+0+0 (secure 0 0 0) 0x7f2a7c003d10 con 0x7f2a8415f040 2024-04-20T11:36:14.676 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.676+0000 7f2a7a000640 1 --2- 172.21.15.152:0/2144592855 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f2a6406cf20 0x7f2a6406f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:14.676 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.676+0000 7f2a7a000640 1 -- 172.21.15.152:0/2144592855 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5709+0+0 (secure 0 0 0) 0x7f2a7c08d1c0 con 0x7f2a8415f040 2024-04-20T11:36:14.676 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.677+0000 7f2a81e00640 1 --2- 172.21.15.152:0/2144592855 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f2a6406cf20 0x7f2a6406f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:14.677 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.678+0000 7f2a81e00640 1 --2- 172.21.15.152:0/2144592855 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f2a6406cf20 0x7f2a6406f3e0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f2a74034000 tx=0x7f2a74007000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:14.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.681+0000 7f2a7a000640 1 -- 172.21.15.152:0/2144592855 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f2a840b37d0 con 0x7f2a8415f040 2024-04-20T11:36:14.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.866+0000 7f2a83200640 1 -- 172.21.15.152:0/2144592855 --> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2a84075e60 con 0x7f2a6406cf20 2024-04-20T11:36:14.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.885+0000 7f2a7a000640 1 -- 172.21.15.152:0/2144592855 <== mgr.4109 v2:172.21.15.175:6836/4021250441 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252484 (secure 0 0 0) 0x7f2a84075e60 con 0x7f2a6406cf20 2024-04-20T11:36:14.888 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:36:14.889 INFO:teuthology.orchestra.run.smithi152.stderr:dumped all 2024-04-20T11:36:14.891 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.890+0000 7f2a83200640 1 -- 172.21.15.152:0/2144592855 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f2a6406cf20 msgr2=0x7f2a6406f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:14.891 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.890+0000 7f2a83200640 1 --2- 172.21.15.152:0/2144592855 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f2a6406cf20 0x7f2a6406f3e0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f2a74034000 tx=0x7f2a74007000 comp rx=0 tx=0).stop 2024-04-20T11:36:14.891 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.890+0000 7f2a83200640 1 -- 172.21.15.152:0/2144592855 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2a8415f040 msgr2=0x7f2a84155370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:14.891 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.890+0000 7f2a83200640 1 --2- 172.21.15.152:0/2144592855 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2a8415f040 0x7f2a84155370 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7f2a841641f0 tx=0x7f2a7c0115a0 comp rx=0 tx=0).stop 2024-04-20T11:36:14.891 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.891+0000 7f2a83200640 1 -- 172.21.15.152:0/2144592855 shutdown_connections 2024-04-20T11:36:14.891 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.891+0000 7f2a83200640 1 --2- 172.21.15.152:0/2144592855 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f2a6406cf20 0x7f2a6406f3e0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:14.891 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.891+0000 7f2a83200640 1 --2- 172.21.15.152:0/2144592855 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2a8415f040 0x7f2a84155370 unknown :-1 s=CLOSED pgs=1976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:14.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.891+0000 7f2a83200640 1 --2- 172.21.15.152:0/2144592855 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a8415ebc0 0x7f2a841654d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:14.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.891+0000 7f2a83200640 1 --2- 172.21.15.152:0/2144592855 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a680039b0 0x7f2a84148360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:14.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.891+0000 7f2a83200640 1 -- 172.21.15.152:0/2144592855 >> 172.21.15.152:0/2144592855 conn(0x7f2a84067ae0 msgr2=0x7f2a84162000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:14.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.891+0000 7f2a83200640 1 -- 172.21.15.152:0/2144592855 shutdown_connections 2024-04-20T11:36:14.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:14.891+0000 7f2a83200640 1 -- 172.21.15.152:0/2144592855 wait complete. 2024-04-20T11:36:14.909 INFO:teuthology.orchestra.run.smithi152.stdout:{"pg_ready":true,"pg_map":{"version":190,"stamp":"2024-04-20T11:36:14.681879+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":224,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":167,"ondisk_log_size":167,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":169,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":264628,"kb_used_data":15176,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754710092,"statfs":{"total":773094113280,"available":772823134208,"internally_reserved":0,"allocated":15540224,"data_stored":7503300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12746,"internal_metadata":255315510},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.237427"},"pg_stats":[{"pgid":"23.3f","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.498594+0000","last_change":"2024-04-20T11:36:09.498245+0000","last_active":"2024-04-20T11:36:10.498594+0000","last_peered":"2024-04-20T11:36:10.498594+0000","last_clean":"2024-04-20T11:36:10.498594+0000","last_became_active":"2024-04-20T11:36:09.497956+0000","last_became_peered":"2024-04-20T11:36:09.497956+0000","last_unstale":"2024-04-20T11:36:10.498594+0000","last_undegraded":"2024-04-20T11:36:10.498594+0000","last_fullsized":"2024-04-20T11:36:10.498594+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:06:21.617876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.3e","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.674949+0000","last_change":"2024-04-20T11:36:09.511171+0000","last_active":"2024-04-20T11:36:10.674949+0000","last_peered":"2024-04-20T11:36:10.674949+0000","last_clean":"2024-04-20T11:36:10.674949+0000","last_became_active":"2024-04-20T11:36:09.510715+0000","last_became_peered":"2024-04-20T11:36:09.510715+0000","last_unstale":"2024-04-20T11:36:10.674949+0000","last_undegraded":"2024-04-20T11:36:10.674949+0000","last_fullsized":"2024-04-20T11:36:10.674949+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:56:28.407018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3d","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.505591+0000","last_change":"2024-04-20T11:36:09.517211+0000","last_active":"2024-04-20T11:36:11.505591+0000","last_peered":"2024-04-20T11:36:11.505591+0000","last_clean":"2024-04-20T11:36:11.505591+0000","last_became_active":"2024-04-20T11:36:09.516886+0000","last_became_peered":"2024-04-20T11:36:09.516886+0000","last_unstale":"2024-04-20T11:36:11.505591+0000","last_undegraded":"2024-04-20T11:36:11.505591+0000","last_fullsized":"2024-04-20T11:36:11.505591+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:11:59.531721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.3c","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.808485+0000","last_change":"2024-04-20T11:36:09.491911+0000","last_active":"2024-04-20T11:36:11.808485+0000","last_peered":"2024-04-20T11:36:11.808485+0000","last_clean":"2024-04-20T11:36:11.808485+0000","last_became_active":"2024-04-20T11:36:09.491136+0000","last_became_peered":"2024-04-20T11:36:09.491136+0000","last_unstale":"2024-04-20T11:36:11.808485+0000","last_undegraded":"2024-04-20T11:36:11.808485+0000","last_fullsized":"2024-04-20T11:36:11.808485+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:41:28.109593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3b","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.674463+0000","last_change":"2024-04-20T11:36:09.515606+0000","last_active":"2024-04-20T11:36:10.674463+0000","last_peered":"2024-04-20T11:36:10.674463+0000","last_clean":"2024-04-20T11:36:10.674463+0000","last_became_active":"2024-04-20T11:36:09.515282+0000","last_became_peered":"2024-04-20T11:36:09.515282+0000","last_unstale":"2024-04-20T11:36:10.674463+0000","last_undegraded":"2024-04-20T11:36:10.674463+0000","last_fullsized":"2024-04-20T11:36:10.674463+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:49:43.328289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3a","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.807103+0000","last_change":"2024-04-20T11:36:09.505767+0000","last_active":"2024-04-20T11:36:11.807103+0000","last_peered":"2024-04-20T11:36:11.807103+0000","last_clean":"2024-04-20T11:36:11.807103+0000","last_became_active":"2024-04-20T11:36:09.502268+0000","last_became_peered":"2024-04-20T11:36:09.502268+0000","last_unstale":"2024-04-20T11:36:11.807103+0000","last_undegraded":"2024-04-20T11:36:11.807103+0000","last_fullsized":"2024-04-20T11:36:11.807103+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:40:06.723507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.39","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.492148+0000","last_change":"2024-04-20T11:36:09.510744+0000","last_active":"2024-04-20T11:36:10.492148+0000","last_peered":"2024-04-20T11:36:10.492148+0000","last_clean":"2024-04-20T11:36:10.492148+0000","last_became_active":"2024-04-20T11:36:09.510385+0000","last_became_peered":"2024-04-20T11:36:09.510385+0000","last_unstale":"2024-04-20T11:36:10.492148+0000","last_undegraded":"2024-04-20T11:36:10.492148+0000","last_fullsized":"2024-04-20T11:36:10.492148+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:44:53.347896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.38","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.675349+0000","last_change":"2024-04-20T11:36:09.509705+0000","last_active":"2024-04-20T11:36:10.675349+0000","last_peered":"2024-04-20T11:36:10.675349+0000","last_clean":"2024-04-20T11:36:10.675349+0000","last_became_active":"2024-04-20T11:36:09.509111+0000","last_became_peered":"2024-04-20T11:36:09.509111+0000","last_unstale":"2024-04-20T11:36:10.675349+0000","last_undegraded":"2024-04-20T11:36:10.675349+0000","last_fullsized":"2024-04-20T11:36:10.675349+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:30:15.532693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.37","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.675032+0000","last_change":"2024-04-20T11:36:09.507670+0000","last_active":"2024-04-20T11:36:10.675032+0000","last_peered":"2024-04-20T11:36:10.675032+0000","last_clean":"2024-04-20T11:36:10.675032+0000","last_became_active":"2024-04-20T11:36:09.507308+0000","last_became_peered":"2024-04-20T11:36:09.507308+0000","last_unstale":"2024-04-20T11:36:10.675032+0000","last_undegraded":"2024-04-20T11:36:10.675032+0000","last_fullsized":"2024-04-20T11:36:10.675032+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:49:27.093205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.36","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.675225+0000","last_change":"2024-04-20T11:36:09.510613+0000","last_active":"2024-04-20T11:36:10.675225+0000","last_peered":"2024-04-20T11:36:10.675225+0000","last_clean":"2024-04-20T11:36:10.675225+0000","last_became_active":"2024-04-20T11:36:09.510280+0000","last_became_peered":"2024-04-20T11:36:09.510280+0000","last_unstale":"2024-04-20T11:36:10.675225+0000","last_undegraded":"2024-04-20T11:36:10.675225+0000","last_fullsized":"2024-04-20T11:36:10.675225+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:41:28.109593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.35","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.674568+0000","last_change":"2024-04-20T11:36:09.516333+0000","last_active":"2024-04-20T11:36:10.674568+0000","last_peered":"2024-04-20T11:36:10.674568+0000","last_clean":"2024-04-20T11:36:10.674568+0000","last_became_active":"2024-04-20T11:36:09.516038+0000","last_became_peered":"2024-04-20T11:36:09.516038+0000","last_unstale":"2024-04-20T11:36:10.674568+0000","last_undegraded":"2024-04-20T11:36:10.674568+0000","last_fullsized":"2024-04-20T11:36:10.674568+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:59:19.149913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.34","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.675001+0000","last_change":"2024-04-20T11:36:09.503669+0000","last_active":"2024-04-20T11:36:10.675001+0000","last_peered":"2024-04-20T11:36:10.675001+0000","last_clean":"2024-04-20T11:36:10.675001+0000","last_became_active":"2024-04-20T11:36:09.503351+0000","last_became_peered":"2024-04-20T11:36:09.503351+0000","last_unstale":"2024-04-20T11:36:10.675001+0000","last_undegraded":"2024-04-20T11:36:10.675001+0000","last_fullsized":"2024-04-20T11:36:10.675001+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:03:17.439635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.33","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.505133+0000","last_change":"2024-04-20T11:36:09.500004+0000","last_active":"2024-04-20T11:36:11.505133+0000","last_peered":"2024-04-20T11:36:11.505133+0000","last_clean":"2024-04-20T11:36:11.505133+0000","last_became_active":"2024-04-20T11:36:09.497333+0000","last_became_peered":"2024-04-20T11:36:09.497333+0000","last_unstale":"2024-04-20T11:36:11.505133+0000","last_undegraded":"2024-04-20T11:36:11.505133+0000","last_fullsized":"2024-04-20T11:36:11.505133+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:55:31.041430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.32","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.506115+0000","last_change":"2024-04-20T11:36:09.513751+0000","last_active":"2024-04-20T11:36:11.506115+0000","last_peered":"2024-04-20T11:36:11.506115+0000","last_clean":"2024-04-20T11:36:11.506115+0000","last_became_active":"2024-04-20T11:36:09.512773+0000","last_became_peered":"2024-04-20T11:36:09.512773+0000","last_unstale":"2024-04-20T11:36:11.506115+0000","last_undegraded":"2024-04-20T11:36:11.506115+0000","last_fullsized":"2024-04-20T11:36:11.506115+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:42:50.753207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.31","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.497845+0000","last_change":"2024-04-20T11:36:09.494578+0000","last_active":"2024-04-20T11:36:10.497845+0000","last_peered":"2024-04-20T11:36:10.497845+0000","last_clean":"2024-04-20T11:36:10.497845+0000","last_became_active":"2024-04-20T11:36:09.494216+0000","last_became_peered":"2024-04-20T11:36:09.494216+0000","last_unstale":"2024-04-20T11:36:10.497845+0000","last_undegraded":"2024-04-20T11:36:10.497845+0000","last_fullsized":"2024-04-20T11:36:10.497845+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:08:37.699106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.30","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.491449+0000","last_change":"2024-04-20T11:36:09.506793+0000","last_active":"2024-04-20T11:36:10.491449+0000","last_peered":"2024-04-20T11:36:10.491449+0000","last_clean":"2024-04-20T11:36:10.491449+0000","last_became_active":"2024-04-20T11:36:09.506217+0000","last_became_peered":"2024-04-20T11:36:09.506217+0000","last_unstale":"2024-04-20T11:36:10.491449+0000","last_undegraded":"2024-04-20T11:36:10.491449+0000","last_fullsized":"2024-04-20T11:36:10.491449+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:08:37.699106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.2f","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.807341+0000","last_change":"2024-04-20T11:36:09.502941+0000","last_active":"2024-04-20T11:36:11.807341+0000","last_peered":"2024-04-20T11:36:11.807341+0000","last_clean":"2024-04-20T11:36:11.807341+0000","last_became_active":"2024-04-20T11:36:09.501204+0000","last_became_peered":"2024-04-20T11:36:09.501204+0000","last_unstale":"2024-04-20T11:36:11.807341+0000","last_undegraded":"2024-04-20T11:36:11.807341+0000","last_fullsized":"2024-04-20T11:36:11.807341+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:41:49.723887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2e","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.496340+0000","last_change":"2024-04-20T11:36:09.497994+0000","last_active":"2024-04-20T11:36:10.496340+0000","last_peered":"2024-04-20T11:36:10.496340+0000","last_clean":"2024-04-20T11:36:10.496340+0000","last_became_active":"2024-04-20T11:36:09.497691+0000","last_became_peered":"2024-04-20T11:36:09.497691+0000","last_unstale":"2024-04-20T11:36:10.496340+0000","last_undegraded":"2024-04-20T11:36:10.496340+0000","last_fullsized":"2024-04-20T11:36:10.496340+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:40:28.853425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2d","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.807963+0000","last_change":"2024-04-20T11:36:09.517462+0000","last_active":"2024-04-20T11:36:11.807963+0000","last_peered":"2024-04-20T11:36:11.807963+0000","last_clean":"2024-04-20T11:36:11.807963+0000","last_became_active":"2024-04-20T11:36:09.517192+0000","last_became_peered":"2024-04-20T11:36:09.517192+0000","last_unstale":"2024-04-20T11:36:11.807963+0000","last_undegraded":"2024-04-20T11:36:11.807963+0000","last_fullsized":"2024-04-20T11:36:11.807963+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:33:01.539999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2c","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.505035+0000","last_change":"2024-04-20T11:36:09.513981+0000","last_active":"2024-04-20T11:36:11.505035+0000","last_peered":"2024-04-20T11:36:11.505035+0000","last_clean":"2024-04-20T11:36:11.505035+0000","last_became_active":"2024-04-20T11:36:09.513653+0000","last_became_peered":"2024-04-20T11:36:09.513653+0000","last_unstale":"2024-04-20T11:36:11.505035+0000","last_undegraded":"2024-04-20T11:36:11.505035+0000","last_fullsized":"2024-04-20T11:36:11.505035+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:14:26.387558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.2b","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.807325+0000","last_change":"2024-04-20T11:36:09.513386+0000","last_active":"2024-04-20T11:36:11.807325+0000","last_peered":"2024-04-20T11:36:11.807325+0000","last_clean":"2024-04-20T11:36:11.807325+0000","last_became_active":"2024-04-20T11:36:09.512484+0000","last_became_peered":"2024-04-20T11:36:09.512484+0000","last_unstale":"2024-04-20T11:36:11.807325+0000","last_undegraded":"2024-04-20T11:36:11.807325+0000","last_fullsized":"2024-04-20T11:36:11.807325+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:49:44.982942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2a","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.675443+0000","last_change":"2024-04-20T11:36:09.501933+0000","last_active":"2024-04-20T11:36:10.675443+0000","last_peered":"2024-04-20T11:36:10.675443+0000","last_clean":"2024-04-20T11:36:10.675443+0000","last_became_active":"2024-04-20T11:36:09.501168+0000","last_became_peered":"2024-04-20T11:36:09.501168+0000","last_unstale":"2024-04-20T11:36:10.675443+0000","last_undegraded":"2024-04-20T11:36:10.675443+0000","last_fullsized":"2024-04-20T11:36:10.675443+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:10:11.720624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.29","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.806672+0000","last_change":"2024-04-20T11:36:09.501305+0000","last_active":"2024-04-20T11:36:11.806672+0000","last_peered":"2024-04-20T11:36:11.806672+0000","last_clean":"2024-04-20T11:36:11.806672+0000","last_became_active":"2024-04-20T11:36:09.500714+0000","last_became_peered":"2024-04-20T11:36:09.500714+0000","last_unstale":"2024-04-20T11:36:11.806672+0000","last_undegraded":"2024-04-20T11:36:11.806672+0000","last_fullsized":"2024-04-20T11:36:11.806672+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:55:38.517484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.28","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.674679+0000","last_change":"2024-04-20T11:36:09.503752+0000","last_active":"2024-04-20T11:36:10.674679+0000","last_peered":"2024-04-20T11:36:10.674679+0000","last_clean":"2024-04-20T11:36:10.674679+0000","last_became_active":"2024-04-20T11:36:09.501356+0000","last_became_peered":"2024-04-20T11:36:09.501356+0000","last_unstale":"2024-04-20T11:36:10.674679+0000","last_undegraded":"2024-04-20T11:36:10.674679+0000","last_fullsized":"2024-04-20T11:36:10.674679+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:52:33.986991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.27","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.505645+0000","last_change":"2024-04-20T11:36:09.508345+0000","last_active":"2024-04-20T11:36:11.505645+0000","last_peered":"2024-04-20T11:36:11.505645+0000","last_clean":"2024-04-20T11:36:11.505645+0000","last_became_active":"2024-04-20T11:36:09.508006+0000","last_became_peered":"2024-04-20T11:36:09.508006+0000","last_unstale":"2024-04-20T11:36:11.505645+0000","last_undegraded":"2024-04-20T11:36:11.505645+0000","last_fullsized":"2024-04-20T11:36:11.505645+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:34:42.652012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.26","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.491806+0000","last_change":"2024-04-20T11:36:09.502314+0000","last_active":"2024-04-20T11:36:10.491806+0000","last_peered":"2024-04-20T11:36:10.491806+0000","last_clean":"2024-04-20T11:36:10.491806+0000","last_became_active":"2024-04-20T11:36:09.502005+0000","last_became_peered":"2024-04-20T11:36:09.502005+0000","last_unstale":"2024-04-20T11:36:10.491806+0000","last_undegraded":"2024-04-20T11:36:10.491806+0000","last_fullsized":"2024-04-20T11:36:10.491806+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:10:13.066071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.25","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.506008+0000","last_change":"2024-04-20T11:36:09.498214+0000","last_active":"2024-04-20T11:36:11.506008+0000","last_peered":"2024-04-20T11:36:11.506008+0000","last_clean":"2024-04-20T11:36:11.506008+0000","last_became_active":"2024-04-20T11:36:09.497877+0000","last_became_peered":"2024-04-20T11:36:09.497877+0000","last_unstale":"2024-04-20T11:36:11.506008+0000","last_undegraded":"2024-04-20T11:36:11.506008+0000","last_fullsized":"2024-04-20T11:36:11.506008+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:20:16.280679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.24","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.808688+0000","last_change":"2024-04-20T11:36:09.502782+0000","last_active":"2024-04-20T11:36:11.808688+0000","last_peered":"2024-04-20T11:36:11.808688+0000","last_clean":"2024-04-20T11:36:11.808688+0000","last_became_active":"2024-04-20T11:36:09.502509+0000","last_became_peered":"2024-04-20T11:36:09.502509+0000","last_unstale":"2024-04-20T11:36:11.808688+0000","last_undegraded":"2024-04-20T11:36:11.808688+0000","last_fullsized":"2024-04-20T11:36:11.808688+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:30:15.532693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.23","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.505461+0000","last_change":"2024-04-20T11:36:09.500622+0000","last_active":"2024-04-20T11:36:11.505461+0000","last_peered":"2024-04-20T11:36:11.505461+0000","last_clean":"2024-04-20T11:36:11.505461+0000","last_became_active":"2024-04-20T11:36:09.500341+0000","last_became_peered":"2024-04-20T11:36:09.500341+0000","last_unstale":"2024-04-20T11:36:11.505461+0000","last_undegraded":"2024-04-20T11:36:11.505461+0000","last_fullsized":"2024-04-20T11:36:11.505461+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:36:10.575443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.22","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.512516+0000","last_change":"2024-04-20T11:36:09.502504+0000","last_active":"2024-04-20T11:36:11.512516+0000","last_peered":"2024-04-20T11:36:11.512516+0000","last_clean":"2024-04-20T11:36:11.512516+0000","last_became_active":"2024-04-20T11:36:09.502201+0000","last_became_peered":"2024-04-20T11:36:09.502201+0000","last_unstale":"2024-04-20T11:36:11.512516+0000","last_undegraded":"2024-04-20T11:36:11.512516+0000","last_fullsized":"2024-04-20T11:36:11.512516+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:42:28.478688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.21","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.512231+0000","last_change":"2024-04-20T11:36:09.519295+0000","last_active":"2024-04-20T11:36:11.512231+0000","last_peered":"2024-04-20T11:36:11.512231+0000","last_clean":"2024-04-20T11:36:11.512231+0000","last_became_active":"2024-04-20T11:36:09.518233+0000","last_became_peered":"2024-04-20T11:36:09.518233+0000","last_unstale":"2024-04-20T11:36:11.512231+0000","last_undegraded":"2024-04-20T11:36:11.512231+0000","last_fullsized":"2024-04-20T11:36:11.512231+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:34:50.865938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.20","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.495715+0000","last_change":"2024-04-20T11:36:09.495789+0000","last_active":"2024-04-20T11:36:10.495715+0000","last_peered":"2024-04-20T11:36:10.495715+0000","last_clean":"2024-04-20T11:36:10.495715+0000","last_became_active":"2024-04-20T11:36:09.495472+0000","last_became_peered":"2024-04-20T11:36:09.495472+0000","last_unstale":"2024-04-20T11:36:10.495715+0000","last_undegraded":"2024-04-20T11:36:10.495715+0000","last_fullsized":"2024-04-20T11:36:10.495715+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:34:42.831349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.492044+0000","last_change":"2024-04-20T11:36:09.509822+0000","last_active":"2024-04-20T11:36:10.492044+0000","last_peered":"2024-04-20T11:36:10.492044+0000","last_clean":"2024-04-20T11:36:10.492044+0000","last_became_active":"2024-04-20T11:36:09.509521+0000","last_became_peered":"2024-04-20T11:36:09.509521+0000","last_unstale":"2024-04-20T11:36:10.492044+0000","last_undegraded":"2024-04-20T11:36:10.492044+0000","last_fullsized":"2024-04-20T11:36:10.492044+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:44:16.352910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1f","version":"141'4","reported_seq":18,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.498511+0000","last_change":"2024-04-20T11:36:08.483872+0000","last_active":"2024-04-20T11:36:10.498511+0000","last_peered":"2024-04-20T11:36:10.498511+0000","last_clean":"2024-04-20T11:36:10.498511+0000","last_became_active":"2024-04-20T11:36:08.483196+0000","last_became_peered":"2024-04-20T11:36:08.483196+0000","last_unstale":"2024-04-20T11:36:10.498511+0000","last_undegraded":"2024-04-20T11:36:10.498511+0000","last_fullsized":"2024-04-20T11:36:10.498511+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:37:38.832794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.513632+0000","last_change":"2024-04-20T11:36:09.505474+0000","last_active":"2024-04-20T11:36:11.513632+0000","last_peered":"2024-04-20T11:36:11.513632+0000","last_clean":"2024-04-20T11:36:11.513632+0000","last_became_active":"2024-04-20T11:36:09.502465+0000","last_became_peered":"2024-04-20T11:36:09.502465+0000","last_unstale":"2024-04-20T11:36:11.513632+0000","last_undegraded":"2024-04-20T11:36:11.513632+0000","last_fullsized":"2024-04-20T11:36:11.513632+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:51:23.507394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1e","version":"141'1","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.512968+0000","last_change":"2024-04-20T11:36:08.478064+0000","last_active":"2024-04-20T11:36:11.512968+0000","last_peered":"2024-04-20T11:36:11.512968+0000","last_clean":"2024-04-20T11:36:11.512968+0000","last_became_active":"2024-04-20T11:36:08.477788+0000","last_became_peered":"2024-04-20T11:36:08.477788+0000","last_unstale":"2024-04-20T11:36:11.512968+0000","last_undegraded":"2024-04-20T11:36:11.512968+0000","last_fullsized":"2024-04-20T11:36:11.512968+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:10:10.704042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.808302+0000","last_change":"2024-04-20T11:36:09.506372+0000","last_active":"2024-04-20T11:36:11.808302+0000","last_peered":"2024-04-20T11:36:11.808302+0000","last_clean":"2024-04-20T11:36:11.808302+0000","last_became_active":"2024-04-20T11:36:09.506107+0000","last_became_peered":"2024-04-20T11:36:09.506107+0000","last_unstale":"2024-04-20T11:36:11.808302+0000","last_undegraded":"2024-04-20T11:36:11.808302+0000","last_fullsized":"2024-04-20T11:36:11.808302+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:21:08.136339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1d","version":"141'2","reported_seq":16,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.808049+0000","last_change":"2024-04-20T11:36:08.479424+0000","last_active":"2024-04-20T11:36:11.808049+0000","last_peered":"2024-04-20T11:36:11.808049+0000","last_clean":"2024-04-20T11:36:11.808049+0000","last_became_active":"2024-04-20T11:36:08.479137+0000","last_became_peered":"2024-04-20T11:36:08.479137+0000","last_unstale":"2024-04-20T11:36:11.808049+0000","last_undegraded":"2024-04-20T11:36:11.808049+0000","last_fullsized":"2024-04-20T11:36:11.808049+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:25:33.273088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.506044+0000","last_change":"2024-04-20T11:36:09.498658+0000","last_active":"2024-04-20T11:36:11.506044+0000","last_peered":"2024-04-20T11:36:11.506044+0000","last_clean":"2024-04-20T11:36:11.506044+0000","last_became_active":"2024-04-20T11:36:09.498372+0000","last_became_peered":"2024-04-20T11:36:09.498372+0000","last_unstale":"2024-04-20T11:36:11.506044+0000","last_undegraded":"2024-04-20T11:36:11.506044+0000","last_fullsized":"2024-04-20T11:36:11.506044+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:14:54.539911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1c","version":"141'2","reported_seq":16,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.808844+0000","last_change":"2024-04-20T11:36:08.481345+0000","last_active":"2024-04-20T11:36:11.808844+0000","last_peered":"2024-04-20T11:36:11.808844+0000","last_clean":"2024-04-20T11:36:11.808844+0000","last_became_active":"2024-04-20T11:36:08.480807+0000","last_became_peered":"2024-04-20T11:36:08.480807+0000","last_unstale":"2024-04-20T11:36:11.808844+0000","last_undegraded":"2024-04-20T11:36:11.808844+0000","last_fullsized":"2024-04-20T11:36:11.808844+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:53:50.523715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.491980+0000","last_change":"2024-04-20T11:36:09.498794+0000","last_active":"2024-04-20T11:36:10.491980+0000","last_peered":"2024-04-20T11:36:10.491980+0000","last_clean":"2024-04-20T11:36:10.491980+0000","last_became_active":"2024-04-20T11:36:09.498063+0000","last_became_peered":"2024-04-20T11:36:09.498063+0000","last_unstale":"2024-04-20T11:36:10.491980+0000","last_undegraded":"2024-04-20T11:36:10.491980+0000","last_fullsized":"2024-04-20T11:36:10.491980+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:00:28.716804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1b","version":"141'1","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.505504+0000","last_change":"2024-04-20T11:36:08.473692+0000","last_active":"2024-04-20T11:36:11.505504+0000","last_peered":"2024-04-20T11:36:11.505504+0000","last_clean":"2024-04-20T11:36:11.505504+0000","last_became_active":"2024-04-20T11:36:08.473388+0000","last_became_peered":"2024-04-20T11:36:08.473388+0000","last_unstale":"2024-04-20T11:36:11.505504+0000","last_undegraded":"2024-04-20T11:36:11.505504+0000","last_fullsized":"2024-04-20T11:36:11.505504+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:09:00.631652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.807706+0000","last_change":"2024-04-20T11:36:09.496254+0000","last_active":"2024-04-20T11:36:11.807706+0000","last_peered":"2024-04-20T11:36:11.807706+0000","last_clean":"2024-04-20T11:36:11.807706+0000","last_became_active":"2024-04-20T11:36:09.492502+0000","last_became_peered":"2024-04-20T11:36:09.492502+0000","last_unstale":"2024-04-20T11:36:11.807706+0000","last_undegraded":"2024-04-20T11:36:11.807706+0000","last_fullsized":"2024-04-20T11:36:11.807706+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:49:27.093205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.1a","version":"141'1","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.808305+0000","last_change":"2024-04-20T11:36:08.472809+0000","last_active":"2024-04-20T11:36:11.808305+0000","last_peered":"2024-04-20T11:36:11.808305+0000","last_clean":"2024-04-20T11:36:11.808305+0000","last_became_active":"2024-04-20T11:36:08.472197+0000","last_became_peered":"2024-04-20T11:36:08.472197+0000","last_unstale":"2024-04-20T11:36:11.808305+0000","last_undegraded":"2024-04-20T11:36:11.808305+0000","last_fullsized":"2024-04-20T11:36:11.808305+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:08:24.707643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.807054+0000","last_change":"2024-04-20T11:36:09.483939+0000","last_active":"2024-04-20T11:36:11.807054+0000","last_peered":"2024-04-20T11:36:11.807054+0000","last_clean":"2024-04-20T11:36:11.807054+0000","last_became_active":"2024-04-20T11:36:09.483166+0000","last_became_peered":"2024-04-20T11:36:09.483166+0000","last_unstale":"2024-04-20T11:36:11.807054+0000","last_undegraded":"2024-04-20T11:36:11.807054+0000","last_fullsized":"2024-04-20T11:36:11.807054+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:03:17.439635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.674387+0000","last_change":"2024-04-20T11:36:08.481523+0000","last_active":"2024-04-20T11:36:10.674387+0000","last_peered":"2024-04-20T11:36:10.674387+0000","last_clean":"2024-04-20T11:36:10.674387+0000","last_became_active":"2024-04-20T11:36:08.481220+0000","last_became_peered":"2024-04-20T11:36:08.481220+0000","last_unstale":"2024-04-20T11:36:10.674387+0000","last_undegraded":"2024-04-20T11:36:10.674387+0000","last_fullsized":"2024-04-20T11:36:10.674387+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:27:21.500530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.806610+0000","last_change":"2024-04-20T11:36:09.499280+0000","last_active":"2024-04-20T11:36:11.806610+0000","last_peered":"2024-04-20T11:36:11.806610+0000","last_clean":"2024-04-20T11:36:11.806610+0000","last_became_active":"2024-04-20T11:36:09.499008+0000","last_became_peered":"2024-04-20T11:36:09.499008+0000","last_unstale":"2024-04-20T11:36:11.806610+0000","last_undegraded":"2024-04-20T11:36:11.806610+0000","last_fullsized":"2024-04-20T11:36:11.806610+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:35:54.753258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.491574+0000","last_change":"2024-04-20T11:36:08.459181+0000","last_active":"2024-04-20T11:36:10.491574+0000","last_peered":"2024-04-20T11:36:10.491574+0000","last_clean":"2024-04-20T11:36:10.491574+0000","last_became_active":"2024-04-20T11:36:08.455877+0000","last_became_peered":"2024-04-20T11:36:08.455877+0000","last_unstale":"2024-04-20T11:36:10.491574+0000","last_undegraded":"2024-04-20T11:36:10.491574+0000","last_fullsized":"2024-04-20T11:36:10.491574+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:03:08.918074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.496622+0000","last_change":"2024-04-20T11:36:09.498684+0000","last_active":"2024-04-20T11:36:10.496622+0000","last_peered":"2024-04-20T11:36:10.496622+0000","last_clean":"2024-04-20T11:36:10.496622+0000","last_became_active":"2024-04-20T11:36:09.498116+0000","last_became_peered":"2024-04-20T11:36:09.498116+0000","last_unstale":"2024-04-20T11:36:10.496622+0000","last_undegraded":"2024-04-20T11:36:10.496622+0000","last_fullsized":"2024-04-20T11:36:10.496622+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:06:21.617876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.807834+0000","last_change":"2024-04-20T11:36:08.472135+0000","last_active":"2024-04-20T11:36:11.807834+0000","last_peered":"2024-04-20T11:36:11.807834+0000","last_clean":"2024-04-20T11:36:11.807834+0000","last_became_active":"2024-04-20T11:36:08.471529+0000","last_became_peered":"2024-04-20T11:36:08.471529+0000","last_unstale":"2024-04-20T11:36:11.807834+0000","last_undegraded":"2024-04-20T11:36:11.807834+0000","last_fullsized":"2024-04-20T11:36:11.807834+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:29:38.601496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.496388+0000","last_change":"2024-04-20T11:36:09.497525+0000","last_active":"2024-04-20T11:36:10.496388+0000","last_peered":"2024-04-20T11:36:10.496388+0000","last_clean":"2024-04-20T11:36:10.496388+0000","last_became_active":"2024-04-20T11:36:09.497185+0000","last_became_peered":"2024-04-20T11:36:09.497185+0000","last_unstale":"2024-04-20T11:36:10.496388+0000","last_undegraded":"2024-04-20T11:36:10.496388+0000","last_fullsized":"2024-04-20T11:36:10.496388+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:39:12.611985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.15","version":"141'3","reported_seq":17,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.807992+0000","last_change":"2024-04-20T11:36:08.474633+0000","last_active":"2024-04-20T11:36:11.807992+0000","last_peered":"2024-04-20T11:36:11.807992+0000","last_clean":"2024-04-20T11:36:11.807992+0000","last_became_active":"2024-04-20T11:36:08.474310+0000","last_became_peered":"2024-04-20T11:36:08.474310+0000","last_unstale":"2024-04-20T11:36:11.807992+0000","last_undegraded":"2024-04-20T11:36:11.807992+0000","last_fullsized":"2024-04-20T11:36:11.807992+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:43:13.653616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.15","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.513740+0000","last_change":"2024-04-20T11:36:09.519775+0000","last_active":"2024-04-20T11:36:11.513740+0000","last_peered":"2024-04-20T11:36:11.513740+0000","last_clean":"2024-04-20T11:36:11.513740+0000","last_became_active":"2024-04-20T11:36:09.516787+0000","last_became_peered":"2024-04-20T11:36:09.516787+0000","last_unstale":"2024-04-20T11:36:11.513740+0000","last_undegraded":"2024-04-20T11:36:11.513740+0000","last_fullsized":"2024-04-20T11:36:11.513740+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:53:29.084025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.14","version":"141'6","reported_seq":20,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.512759+0000","last_change":"2024-04-20T11:36:08.481226+0000","last_active":"2024-04-20T11:36:11.512759+0000","last_peered":"2024-04-20T11:36:11.512759+0000","last_clean":"2024-04-20T11:36:11.512759+0000","last_became_active":"2024-04-20T11:36:08.480392+0000","last_became_peered":"2024-04-20T11:36:08.480392+0000","last_unstale":"2024-04-20T11:36:11.512759+0000","last_undegraded":"2024-04-20T11:36:11.512759+0000","last_fullsized":"2024-04-20T11:36:11.512759+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:07:03.390165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.495849+0000","last_change":"2024-04-20T11:36:09.505061+0000","last_active":"2024-04-20T11:36:10.495849+0000","last_peered":"2024-04-20T11:36:10.495849+0000","last_clean":"2024-04-20T11:36:10.495849+0000","last_became_active":"2024-04-20T11:36:09.504620+0000","last_became_peered":"2024-04-20T11:36:09.504620+0000","last_unstale":"2024-04-20T11:36:10.495849+0000","last_undegraded":"2024-04-20T11:36:10.495849+0000","last_fullsized":"2024-04-20T11:36:10.495849+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:16:01.895857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.13","version":"141'2","reported_seq":16,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.807629+0000","last_change":"2024-04-20T11:36:08.481075+0000","last_active":"2024-04-20T11:36:11.807629+0000","last_peered":"2024-04-20T11:36:11.807629+0000","last_clean":"2024-04-20T11:36:11.807629+0000","last_became_active":"2024-04-20T11:36:08.480423+0000","last_became_peered":"2024-04-20T11:36:08.480423+0000","last_unstale":"2024-04-20T11:36:11.807629+0000","last_undegraded":"2024-04-20T11:36:11.807629+0000","last_fullsized":"2024-04-20T11:36:11.807629+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:43:49.370831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.492241+0000","last_change":"2024-04-20T11:36:09.503953+0000","last_active":"2024-04-20T11:36:10.492241+0000","last_peered":"2024-04-20T11:36:10.492241+0000","last_clean":"2024-04-20T11:36:10.492241+0000","last_became_active":"2024-04-20T11:36:09.503682+0000","last_became_peered":"2024-04-20T11:36:09.503682+0000","last_unstale":"2024-04-20T11:36:10.492241+0000","last_undegraded":"2024-04-20T11:36:10.492241+0000","last_fullsized":"2024-04-20T11:36:10.492241+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:32:53.769485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.496025+0000","last_change":"2024-04-20T11:36:08.469986+0000","last_active":"2024-04-20T11:36:10.496025+0000","last_peered":"2024-04-20T11:36:10.496025+0000","last_clean":"2024-04-20T11:36:10.496025+0000","last_became_active":"2024-04-20T11:36:08.469638+0000","last_became_peered":"2024-04-20T11:36:08.469638+0000","last_unstale":"2024-04-20T11:36:10.496025+0000","last_undegraded":"2024-04-20T11:36:10.496025+0000","last_fullsized":"2024-04-20T11:36:10.496025+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:51:13.378758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.807021+0000","last_change":"2024-04-20T11:36:09.503606+0000","last_active":"2024-04-20T11:36:11.807021+0000","last_peered":"2024-04-20T11:36:11.807021+0000","last_clean":"2024-04-20T11:36:11.807021+0000","last_became_active":"2024-04-20T11:36:09.503261+0000","last_became_peered":"2024-04-20T11:36:09.503261+0000","last_unstale":"2024-04-20T11:36:11.807021+0000","last_undegraded":"2024-04-20T11:36:11.807021+0000","last_fullsized":"2024-04-20T11:36:11.807021+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:56:28.407018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.807199+0000","last_change":"2024-04-20T11:36:08.477055+0000","last_active":"2024-04-20T11:36:11.807199+0000","last_peered":"2024-04-20T11:36:11.807199+0000","last_clean":"2024-04-20T11:36:11.807199+0000","last_became_active":"2024-04-20T11:36:08.476730+0000","last_became_peered":"2024-04-20T11:36:08.476730+0000","last_unstale":"2024-04-20T11:36:11.807199+0000","last_undegraded":"2024-04-20T11:36:11.807199+0000","last_fullsized":"2024-04-20T11:36:11.807199+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:00:31.304085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.504738+0000","last_change":"2024-04-20T11:36:09.504805+0000","last_active":"2024-04-20T11:36:11.504738+0000","last_peered":"2024-04-20T11:36:11.504738+0000","last_clean":"2024-04-20T11:36:11.504738+0000","last_became_active":"2024-04-20T11:36:09.504541+0000","last_became_peered":"2024-04-20T11:36:09.504541+0000","last_unstale":"2024-04-20T11:36:11.504738+0000","last_undegraded":"2024-04-20T11:36:11.504738+0000","last_fullsized":"2024-04-20T11:36:11.504738+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:03:06.526803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.10","version":"141'8","reported_seq":22,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.511978+0000","last_change":"2024-04-20T11:36:08.476771+0000","last_active":"2024-04-20T11:36:11.511978+0000","last_peered":"2024-04-20T11:36:11.511978+0000","last_clean":"2024-04-20T11:36:11.511978+0000","last_became_active":"2024-04-20T11:36:08.476124+0000","last_became_peered":"2024-04-20T11:36:08.476124+0000","last_unstale":"2024-04-20T11:36:11.511978+0000","last_undegraded":"2024-04-20T11:36:11.511978+0000","last_fullsized":"2024-04-20T11:36:11.511978+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:41:27.093010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.809290+0000","last_change":"2024-04-20T11:36:09.500469+0000","last_active":"2024-04-20T11:36:11.809290+0000","last_peered":"2024-04-20T11:36:11.809290+0000","last_clean":"2024-04-20T11:36:11.809290+0000","last_became_active":"2024-04-20T11:36:09.495917+0000","last_became_peered":"2024-04-20T11:36:09.495917+0000","last_unstale":"2024-04-20T11:36:11.809290+0000","last_undegraded":"2024-04-20T11:36:11.809290+0000","last_fullsized":"2024-04-20T11:36:11.809290+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:07:04.406747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"141'1","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.505732+0000","last_change":"2024-04-20T11:36:08.479096+0000","last_active":"2024-04-20T11:36:11.505732+0000","last_peered":"2024-04-20T11:36:11.505732+0000","last_clean":"2024-04-20T11:36:11.505732+0000","last_became_active":"2024-04-20T11:36:08.478807+0000","last_became_peered":"2024-04-20T11:36:08.478807+0000","last_unstale":"2024-04-20T11:36:11.505732+0000","last_undegraded":"2024-04-20T11:36:11.505732+0000","last_fullsized":"2024-04-20T11:36:11.505732+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:25:58.656867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.496824+0000","last_change":"2024-04-20T11:36:09.510407+0000","last_active":"2024-04-20T11:36:10.496824+0000","last_peered":"2024-04-20T11:36:10.496824+0000","last_clean":"2024-04-20T11:36:10.496824+0000","last_became_active":"2024-04-20T11:36:09.510132+0000","last_became_peered":"2024-04-20T11:36:09.510132+0000","last_unstale":"2024-04-20T11:36:10.496824+0000","last_undegraded":"2024-04-20T11:36:10.496824+0000","last_fullsized":"2024-04-20T11:36:10.496824+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:13:57.374934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.e","version":"141'2","reported_seq":16,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.505227+0000","last_change":"2024-04-20T11:36:08.475925+0000","last_active":"2024-04-20T11:36:11.505227+0000","last_peered":"2024-04-20T11:36:11.505227+0000","last_clean":"2024-04-20T11:36:11.505227+0000","last_became_active":"2024-04-20T11:36:08.474739+0000","last_became_peered":"2024-04-20T11:36:08.474739+0000","last_unstale":"2024-04-20T11:36:11.505227+0000","last_undegraded":"2024-04-20T11:36:11.505227+0000","last_fullsized":"2024-04-20T11:36:11.505227+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:52:49.350436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.808772+0000","last_change":"2024-04-20T11:36:09.497659+0000","last_active":"2024-04-20T11:36:11.808772+0000","last_peered":"2024-04-20T11:36:11.808772+0000","last_clean":"2024-04-20T11:36:11.808772+0000","last_became_active":"2024-04-20T11:36:09.495859+0000","last_became_peered":"2024-04-20T11:36:09.495859+0000","last_unstale":"2024-04-20T11:36:11.808772+0000","last_undegraded":"2024-04-20T11:36:11.808772+0000","last_fullsized":"2024-04-20T11:36:11.808772+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:10:11.720624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.d","version":"141'1","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.498977+0000","last_change":"2024-04-20T11:36:08.484073+0000","last_active":"2024-04-20T11:36:10.498977+0000","last_peered":"2024-04-20T11:36:10.498977+0000","last_clean":"2024-04-20T11:36:10.498977+0000","last_became_active":"2024-04-20T11:36:08.483348+0000","last_became_peered":"2024-04-20T11:36:08.483348+0000","last_unstale":"2024-04-20T11:36:10.498977+0000","last_undegraded":"2024-04-20T11:36:10.498977+0000","last_fullsized":"2024-04-20T11:36:10.498977+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:58:31.453250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.491566+0000","last_change":"2024-04-20T11:36:09.515509+0000","last_active":"2024-04-20T11:36:10.491566+0000","last_peered":"2024-04-20T11:36:10.491566+0000","last_clean":"2024-04-20T11:36:10.491566+0000","last_became_active":"2024-04-20T11:36:09.515199+0000","last_became_peered":"2024-04-20T11:36:09.515199+0000","last_unstale":"2024-04-20T11:36:10.491566+0000","last_undegraded":"2024-04-20T11:36:10.491566+0000","last_fullsized":"2024-04-20T11:36:10.491566+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:06:21.617876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.498695+0000","last_change":"2024-04-20T11:36:08.481420+0000","last_active":"2024-04-20T11:36:10.498695+0000","last_peered":"2024-04-20T11:36:10.498695+0000","last_clean":"2024-04-20T11:36:10.498695+0000","last_became_active":"2024-04-20T11:36:08.481118+0000","last_became_peered":"2024-04-20T11:36:08.481118+0000","last_unstale":"2024-04-20T11:36:10.498695+0000","last_undegraded":"2024-04-20T11:36:10.498695+0000","last_fullsized":"2024-04-20T11:36:10.498695+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:27:32.804957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.504806+0000","last_change":"2024-04-20T11:36:09.504033+0000","last_active":"2024-04-20T11:36:11.504806+0000","last_peered":"2024-04-20T11:36:11.504806+0000","last_clean":"2024-04-20T11:36:11.504806+0000","last_became_active":"2024-04-20T11:36:09.503676+0000","last_became_peered":"2024-04-20T11:36:09.503676+0000","last_unstale":"2024-04-20T11:36:11.504806+0000","last_undegraded":"2024-04-20T11:36:11.504806+0000","last_fullsized":"2024-04-20T11:36:11.504806+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:56:53.061204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.b","version":"141'2","reported_seq":16,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.807844+0000","last_change":"2024-04-20T11:36:08.479935+0000","last_active":"2024-04-20T11:36:11.807844+0000","last_peered":"2024-04-20T11:36:11.807844+0000","last_clean":"2024-04-20T11:36:11.807844+0000","last_became_active":"2024-04-20T11:36:08.479361+0000","last_became_peered":"2024-04-20T11:36:08.479361+0000","last_unstale":"2024-04-20T11:36:11.807844+0000","last_undegraded":"2024-04-20T11:36:11.807844+0000","last_fullsized":"2024-04-20T11:36:11.807844+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:36:59.262204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.492424+0000","last_change":"2024-04-20T11:36:09.508916+0000","last_active":"2024-04-20T11:36:10.492424+0000","last_peered":"2024-04-20T11:36:10.492424+0000","last_clean":"2024-04-20T11:36:10.492424+0000","last_became_active":"2024-04-20T11:36:09.506508+0000","last_became_peered":"2024-04-20T11:36:09.506508+0000","last_unstale":"2024-04-20T11:36:10.492424+0000","last_undegraded":"2024-04-20T11:36:10.492424+0000","last_fullsized":"2024-04-20T11:36:10.492424+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:37:42.247415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.498687+0000","last_change":"2024-04-20T11:36:08.480015+0000","last_active":"2024-04-20T11:36:10.498687+0000","last_peered":"2024-04-20T11:36:10.498687+0000","last_clean":"2024-04-20T11:36:10.498687+0000","last_became_active":"2024-04-20T11:36:08.479699+0000","last_became_peered":"2024-04-20T11:36:08.479699+0000","last_unstale":"2024-04-20T11:36:10.498687+0000","last_undegraded":"2024-04-20T11:36:10.498687+0000","last_fullsized":"2024-04-20T11:36:10.498687+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:54:19.912464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.497824+0000","last_change":"2024-04-20T11:36:09.503816+0000","last_active":"2024-04-20T11:36:10.497824+0000","last_peered":"2024-04-20T11:36:10.497824+0000","last_clean":"2024-04-20T11:36:10.497824+0000","last_became_active":"2024-04-20T11:36:09.503166+0000","last_became_peered":"2024-04-20T11:36:09.503166+0000","last_unstale":"2024-04-20T11:36:10.497824+0000","last_undegraded":"2024-04-20T11:36:10.497824+0000","last_fullsized":"2024-04-20T11:36:10.497824+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:56:54.406631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.807386+0000","last_change":"2024-04-20T11:36:08.477340+0000","last_active":"2024-04-20T11:36:11.807386+0000","last_peered":"2024-04-20T11:36:11.807386+0000","last_clean":"2024-04-20T11:36:11.807386+0000","last_became_active":"2024-04-20T11:36:08.476853+0000","last_became_peered":"2024-04-20T11:36:08.476853+0000","last_unstale":"2024-04-20T11:36:11.807386+0000","last_undegraded":"2024-04-20T11:36:11.807386+0000","last_fullsized":"2024-04-20T11:36:11.807386+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:52:40.058619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.806778+0000","last_change":"2024-04-20T11:36:09.504017+0000","last_active":"2024-04-20T11:36:11.806778+0000","last_peered":"2024-04-20T11:36:11.806778+0000","last_clean":"2024-04-20T11:36:11.806778+0000","last_became_active":"2024-04-20T11:36:09.503623+0000","last_became_peered":"2024-04-20T11:36:09.503623+0000","last_unstale":"2024-04-20T11:36:11.806778+0000","last_undegraded":"2024-04-20T11:36:11.806778+0000","last_fullsized":"2024-04-20T11:36:11.806778+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:42:34.318459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.497950+0000","last_change":"2024-04-20T11:36:08.481304+0000","last_active":"2024-04-20T11:36:10.497950+0000","last_peered":"2024-04-20T11:36:10.497950+0000","last_clean":"2024-04-20T11:36:10.497950+0000","last_became_active":"2024-04-20T11:36:08.480957+0000","last_became_peered":"2024-04-20T11:36:08.480957+0000","last_unstale":"2024-04-20T11:36:10.497950+0000","last_undegraded":"2024-04-20T11:36:10.497950+0000","last_fullsized":"2024-04-20T11:36:10.497950+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:34:41.814767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.496549+0000","last_change":"2024-04-20T11:36:09.507310+0000","last_active":"2024-04-20T11:36:10.496549+0000","last_peered":"2024-04-20T11:36:10.496549+0000","last_clean":"2024-04-20T11:36:10.496549+0000","last_became_active":"2024-04-20T11:36:09.501972+0000","last_became_peered":"2024-04-20T11:36:09.501972+0000","last_unstale":"2024-04-20T11:36:10.496549+0000","last_undegraded":"2024-04-20T11:36:10.496549+0000","last_fullsized":"2024-04-20T11:36:10.496549+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:08:37.699106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.7","version":"141'2","reported_seq":16,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.496818+0000","last_change":"2024-04-20T11:36:08.482021+0000","last_active":"2024-04-20T11:36:10.496818+0000","last_peered":"2024-04-20T11:36:10.496818+0000","last_clean":"2024-04-20T11:36:10.496818+0000","last_became_active":"2024-04-20T11:36:08.481134+0000","last_became_peered":"2024-04-20T11:36:08.481134+0000","last_unstale":"2024-04-20T11:36:10.496818+0000","last_undegraded":"2024-04-20T11:36:10.496818+0000","last_fullsized":"2024-04-20T11:36:10.496818+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:34:49.849356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.491796+0000","last_change":"2024-04-20T11:36:09.503319+0000","last_active":"2024-04-20T11:36:10.491796+0000","last_peered":"2024-04-20T11:36:10.491796+0000","last_clean":"2024-04-20T11:36:10.491796+0000","last_became_active":"2024-04-20T11:36:09.503019+0000","last_became_peered":"2024-04-20T11:36:09.503019+0000","last_unstale":"2024-04-20T11:36:10.491796+0000","last_undegraded":"2024-04-20T11:36:10.491796+0000","last_fullsized":"2024-04-20T11:36:10.491796+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:13:57.374934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.6","version":"142'1","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.505736+0000","last_change":"2024-04-20T11:36:08.471585+0000","last_active":"2024-04-20T11:36:11.505736+0000","last_peered":"2024-04-20T11:36:11.505736+0000","last_clean":"2024-04-20T11:36:11.505736+0000","last_became_active":"2024-04-20T11:36:08.471286+0000","last_became_peered":"2024-04-20T11:36:08.471286+0000","last_unstale":"2024-04-20T11:36:11.505736+0000","last_undegraded":"2024-04-20T11:36:11.505736+0000","last_fullsized":"2024-04-20T11:36:11.505736+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:04:24.667809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.807695+0000","last_change":"2024-04-20T11:36:09.504584+0000","last_active":"2024-04-20T11:36:11.807695+0000","last_peered":"2024-04-20T11:36:11.807695+0000","last_clean":"2024-04-20T11:36:11.807695+0000","last_became_active":"2024-04-20T11:36:09.502500+0000","last_became_peered":"2024-04-20T11:36:09.502500+0000","last_unstale":"2024-04-20T11:36:11.807695+0000","last_undegraded":"2024-04-20T11:36:11.807695+0000","last_fullsized":"2024-04-20T11:36:11.807695+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:42:53.512494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.674832+0000","last_change":"2024-04-20T11:36:08.483684+0000","last_active":"2024-04-20T11:36:10.674832+0000","last_peered":"2024-04-20T11:36:10.674832+0000","last_clean":"2024-04-20T11:36:10.674832+0000","last_became_active":"2024-04-20T11:36:08.483405+0000","last_became_peered":"2024-04-20T11:36:08.483405+0000","last_unstale":"2024-04-20T11:36:10.674832+0000","last_undegraded":"2024-04-20T11:36:10.674832+0000","last_fullsized":"2024-04-20T11:36:10.674832+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:08:24.707643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.496468+0000","last_change":"2024-04-20T11:36:09.503075+0000","last_active":"2024-04-20T11:36:10.496468+0000","last_peered":"2024-04-20T11:36:10.496468+0000","last_clean":"2024-04-20T11:36:10.496468+0000","last_became_active":"2024-04-20T11:36:09.502712+0000","last_became_peered":"2024-04-20T11:36:09.502712+0000","last_unstale":"2024-04-20T11:36:10.496468+0000","last_undegraded":"2024-04-20T11:36:10.496468+0000","last_fullsized":"2024-04-20T11:36:10.496468+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:56:54.406631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.496193+0000","last_change":"2024-04-20T11:36:08.472571+0000","last_active":"2024-04-20T11:36:10.496193+0000","last_peered":"2024-04-20T11:36:10.496193+0000","last_clean":"2024-04-20T11:36:10.496193+0000","last_became_active":"2024-04-20T11:36:08.470365+0000","last_became_peered":"2024-04-20T11:36:08.470365+0000","last_unstale":"2024-04-20T11:36:10.496193+0000","last_undegraded":"2024-04-20T11:36:10.496193+0000","last_fullsized":"2024-04-20T11:36:10.496193+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:23:09.673966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.496372+0000","last_change":"2024-04-20T11:36:09.496179+0000","last_active":"2024-04-20T11:36:10.496372+0000","last_peered":"2024-04-20T11:36:10.496372+0000","last_clean":"2024-04-20T11:36:10.496372+0000","last_became_active":"2024-04-20T11:36:09.495921+0000","last_became_peered":"2024-04-20T11:36:09.495921+0000","last_unstale":"2024-04-20T11:36:10.496372+0000","last_undegraded":"2024-04-20T11:36:10.496372+0000","last_fullsized":"2024-04-20T11:36:10.496372+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:48:17.725873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.513030+0000","last_change":"2024-04-20T11:36:08.471003+0000","last_active":"2024-04-20T11:36:11.513030+0000","last_peered":"2024-04-20T11:36:11.513030+0000","last_clean":"2024-04-20T11:36:11.513030+0000","last_became_active":"2024-04-20T11:36:08.470689+0000","last_became_peered":"2024-04-20T11:36:08.470689+0000","last_unstale":"2024-04-20T11:36:11.513030+0000","last_undegraded":"2024-04-20T11:36:11.513030+0000","last_fullsized":"2024-04-20T11:36:11.513030+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:29:33.311708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.512894+0000","last_change":"2024-04-20T11:36:09.510186+0000","last_active":"2024-04-20T11:36:11.512894+0000","last_peered":"2024-04-20T11:36:11.512894+0000","last_clean":"2024-04-20T11:36:11.512894+0000","last_became_active":"2024-04-20T11:36:09.509860+0000","last_became_peered":"2024-04-20T11:36:09.509860+0000","last_unstale":"2024-04-20T11:36:11.512894+0000","last_undegraded":"2024-04-20T11:36:11.512894+0000","last_fullsized":"2024-04-20T11:36:11.512894+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:55:27.981008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.808267+0000","last_change":"2024-04-20T11:36:08.472251+0000","last_active":"2024-04-20T11:36:11.808267+0000","last_peered":"2024-04-20T11:36:11.808267+0000","last_clean":"2024-04-20T11:36:11.808267+0000","last_became_active":"2024-04-20T11:36:08.469968+0000","last_became_peered":"2024-04-20T11:36:08.469968+0000","last_unstale":"2024-04-20T11:36:11.808267+0000","last_undegraded":"2024-04-20T11:36:11.808267+0000","last_fullsized":"2024-04-20T11:36:11.808267+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:47:08.022520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.491347+0000","last_change":"2024-04-20T11:36:09.506539+0000","last_active":"2024-04-20T11:36:10.491347+0000","last_peered":"2024-04-20T11:36:10.491347+0000","last_clean":"2024-04-20T11:36:10.491347+0000","last_became_active":"2024-04-20T11:36:09.505746+0000","last_became_peered":"2024-04-20T11:36:09.505746+0000","last_unstale":"2024-04-20T11:36:10.491347+0000","last_undegraded":"2024-04-20T11:36:10.491347+0000","last_fullsized":"2024-04-20T11:36:10.491347+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:56:54.406631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.674650+0000","last_change":"2024-04-20T11:36:08.475717+0000","last_active":"2024-04-20T11:36:10.674650+0000","last_peered":"2024-04-20T11:36:10.674650+0000","last_clean":"2024-04-20T11:36:10.674650+0000","last_became_active":"2024-04-20T11:36:08.475397+0000","last_became_peered":"2024-04-20T11:36:08.475397+0000","last_unstale":"2024-04-20T11:36:10.674650+0000","last_undegraded":"2024-04-20T11:36:10.674650+0000","last_fullsized":"2024-04-20T11:36:10.674650+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:52:40.058619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.504842+0000","last_change":"2024-04-20T11:36:09.504191+0000","last_active":"2024-04-20T11:36:11.504842+0000","last_peered":"2024-04-20T11:36:11.504842+0000","last_clean":"2024-04-20T11:36:11.504842+0000","last_became_active":"2024-04-20T11:36:09.500083+0000","last_became_peered":"2024-04-20T11:36:09.500083+0000","last_unstale":"2024-04-20T11:36:11.504842+0000","last_undegraded":"2024-04-20T11:36:11.504842+0000","last_fullsized":"2024-04-20T11:36:11.504842+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:01:44.730919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.512190+0000","last_change":"2024-04-20T11:36:08.467047+0000","last_active":"2024-04-20T11:36:11.512190+0000","last_peered":"2024-04-20T11:36:11.512190+0000","last_clean":"2024-04-20T11:36:11.512190+0000","last_became_active":"2024-04-20T11:36:08.466681+0000","last_became_peered":"2024-04-20T11:36:08.466681+0000","last_unstale":"2024-04-20T11:36:11.512190+0000","last_undegraded":"2024-04-20T11:36:11.512190+0000","last_fullsized":"2024-04-20T11:36:11.512190+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:30:14.516111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.496672+0000","last_change":"2024-04-20T11:36:09.504230+0000","last_active":"2024-04-20T11:36:10.496672+0000","last_peered":"2024-04-20T11:36:10.496672+0000","last_clean":"2024-04-20T11:36:10.496672+0000","last_became_active":"2024-04-20T11:36:09.503909+0000","last_became_peered":"2024-04-20T11:36:09.503909+0000","last_unstale":"2024-04-20T11:36:10.496672+0000","last_undegraded":"2024-04-20T11:36:10.496672+0000","last_fullsized":"2024-04-20T11:36:10.496672+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:10:13.066071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.513532+0000","last_change":"2024-04-20T11:36:08.473393+0000","last_active":"2024-04-20T11:36:11.513532+0000","last_peered":"2024-04-20T11:36:11.513532+0000","last_clean":"2024-04-20T11:36:11.513532+0000","last_became_active":"2024-04-20T11:36:08.473104+0000","last_became_peered":"2024-04-20T11:36:08.473104+0000","last_unstale":"2024-04-20T11:36:11.513532+0000","last_undegraded":"2024-04-20T11:36:11.513532+0000","last_fullsized":"2024-04-20T11:36:11.513532+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:08:32.469884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"72'128","reported_seq":427,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.498104+0000","last_change":"2024-04-20T11:26:48.661897+0000","last_active":"2024-04-20T11:36:10.498104+0000","last_peered":"2024-04-20T11:36:10.498104+0000","last_clean":"2024-04-20T11:36:10.498104+0000","last_became_active":"2024-04-20T11:26:48.656535+0000","last_became_peered":"2024-04-20T11:26:48.656535+0000","last_unstale":"2024-04-20T11:36:10.498104+0000","last_undegraded":"2024-04-20T11:36:10.498104+0000","last_fullsized":"2024-04-20T11:36:10.498104+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:47.543004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:47.543004+0000","last_clean_scrub_stamp":"2024-04-20T11:26:47.543004+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:58:35.527105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":23,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":22,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574967,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33528,"kb_used_data":2348,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338312,"statfs":{"total":96636764160,"available":96602431488,"internally_reserved":0,"allocated":2404352,"data_stored":1382440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574964,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32932,"kb_used_data":1748,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338908,"statfs":{"total":96636764160,"available":96603041792,"internally_reserved":0,"allocated":1789952,"data_stored":789740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574966,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32932,"kb_used_data":1752,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338908,"statfs":{"total":96636764160,"available":96603041792,"internally_reserved":0,"allocated":1794048,"data_stored":790322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574964,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32940,"kb_used_data":1760,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338900,"statfs":{"total":96636764160,"available":96603033600,"internally_reserved":0,"allocated":1802240,"data_stored":791400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574965,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32928,"kb_used_data":1748,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338912,"statfs":{"total":96636764160,"available":96603045888,"internally_reserved":0,"allocated":1789952,"data_stored":789752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574966,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32928,"kb_used_data":1748,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338912,"statfs":{"total":96636764160,"available":96603045888,"internally_reserved":0,"allocated":1789952,"data_stored":789808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574964,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32928,"kb_used_data":1748,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338912,"statfs":{"total":96636764160,"available":96603045888,"internally_reserved":0,"allocated":1789952,"data_stored":789752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574965,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33512,"kb_used_data":2324,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338328,"statfs":{"total":96636764160,"available":96602447872,"internally_reserved":0,"allocated":2379776,"data_stored":1380086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T11:36:14.913 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-20T11:36:15.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.153+0000 7f317b200640 1 Processor -- start 2024-04-20T11:36:15.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.154+0000 7f317b200640 1 -- start start 2024-04-20T11:36:15.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.155+0000 7f317b200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f317c0723e0 0x7f317c15d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:15.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.155+0000 7f317b200640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f317c0ce8c0 con 0x7f317c160b90 2024-04-20T11:36:15.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.155+0000 7f317b200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f317c1600b0 con 0x7f317c15d1e0 2024-04-20T11:36:15.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.155+0000 7f317b200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f317c160390 con 0x7f317c0723e0 2024-04-20T11:36:15.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.156+0000 7f3173e00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f317c160b90 0x7f317c071cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:42442/0 (socket says 172.21.15.152:42442) 2024-04-20T11:36:15.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.156+0000 7f3173e00640 1 -- 172.21.15.152:0/3308618138 learned_addr learned my addr 172.21.15.152:0/3308618138 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:36:15.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.156+0000 7f317a800640 1 --2- 172.21.15.152:0/3308618138 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f317c0723e0 0x7f317c15d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:15.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.156+0000 7f3179400640 1 -- 172.21.15.152:0/3308618138 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 666437176 0 0) 0x7f317c0ce8c0 con 0x7f317c160b90 2024-04-20T11:36:15.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.156+0000 7f3179400640 1 -- 172.21.15.152:0/3308618138 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3160003520 con 0x7f317c160b90 2024-04-20T11:36:15.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.157+0000 7f3179400640 1 -- 172.21.15.152:0/3308618138 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1667860072 0 0) 0x7f3160003520 con 0x7f317c160b90 2024-04-20T11:36:15.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.157+0000 7f3179400640 1 -- 172.21.15.152:0/3308618138 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f317c0723e0 msgr2=0x7f317c15d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:15.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.157+0000 7f3179400640 1 --2- 172.21.15.152:0/3308618138 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f317c0723e0 0x7f317c15d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:15.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.157+0000 7f3179400640 1 -- 172.21.15.152:0/3308618138 >> v1:172.21.15.175:6789/0 conn(0x7f317c15d1e0 legacy=0x7f317c0cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:15.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.157+0000 7f3179400640 1 -- 172.21.15.152:0/3308618138 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f317c160670 con 0x7f317c160b90 2024-04-20T11:36:15.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.157+0000 7f3179400640 1 -- 172.21.15.152:0/3308618138 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f3164001670 con 0x7f317c160b90 2024-04-20T11:36:15.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.157+0000 7f3179400640 1 -- 172.21.15.152:0/3308618138 >> v1:172.21.15.152:6789/0 conn(0x7f317c160b90 legacy=0x7f317c071cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:15.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.157+0000 7f3179400640 1 --2- 172.21.15.152:0/3308618138 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f31600039e0 0x7f3160003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:15.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.157+0000 7f3179400640 1 --2- 172.21.15.152:0/3308618138 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3160004800 0x7f3160004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:15.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.157+0000 7f3179400640 1 --2- 172.21.15.152:0/3308618138 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f317c0723e0 0x7f31600051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:15.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.157+0000 7f3179400640 1 -- 172.21.15.152:0/3308618138 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f317c160390 con 0x7f3160004800 2024-04-20T11:36:15.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.157+0000 7f3179400640 1 -- 172.21.15.152:0/3308618138 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f316000baa0 con 0x7f31600039e0 2024-04-20T11:36:15.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.157+0000 7f3179400640 1 -- 172.21.15.152:0/3308618138 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f316000bca0 con 0x7f317c0723e0 2024-04-20T11:36:15.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.157+0000 7f3173e00640 1 --2- 172.21.15.152:0/3308618138 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3160004800 0x7f3160004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:15.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.158+0000 7f3173e00640 1 -- 172.21.15.152:0/3308618138 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f317c0723e0 msgr2=0x7f31600051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:15.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.158+0000 7f3173e00640 1 --2- 172.21.15.152:0/3308618138 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f317c0723e0 0x7f31600051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:15.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.158+0000 7f3173e00640 1 -- 172.21.15.152:0/3308618138 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f31600039e0 msgr2=0x7f3160003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:15.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.158+0000 7f3179e00640 1 --2- 172.21.15.152:0/3308618138 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f31600039e0 0x7f3160003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:15.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.158+0000 7f3173e00640 1 --2- 172.21.15.152:0/3308618138 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f31600039e0 0x7f3160003e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:15.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.158+0000 7f3173e00640 1 -- 172.21.15.152:0/3308618138 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3160003520 con 0x7f3160004800 2024-04-20T11:36:15.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.159+0000 7f3173e00640 1 --2- 172.21.15.152:0/3308618138 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3160004800 0x7f3160004c80 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7f316400c0e0 tx=0x7f3160005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7fe0a020e91ec6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:15.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.159+0000 7f3179400640 1 -- 172.21.15.152:0/3308618138 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3164007c40 con 0x7f3160004800 2024-04-20T11:36:15.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.159+0000 7f3179400640 1 -- 172.21.15.152:0/3308618138 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3164030070 con 0x7f3160004800 2024-04-20T11:36:15.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.159+0000 7f3179400640 1 -- 172.21.15.152:0/3308618138 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3164005d30 con 0x7f3160004800 2024-04-20T11:36:15.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.159+0000 7f317b200640 1 -- 172.21.15.152:0/3308618138 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3160004800 msgr2=0x7f3160004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:15.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.159+0000 7f317b200640 1 --2- 172.21.15.152:0/3308618138 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3160004800 0x7f3160004c80 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7f316400c0e0 tx=0x7f3160005830 comp rx=0 tx=0).stop 2024-04-20T11:36:15.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.160+0000 7f317a800640 1 -- 172.21.15.152:0/3308618138 reap_dead start 2024-04-20T11:36:15.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.160+0000 7f317b200640 1 -- 172.21.15.152:0/3308618138 shutdown_connections 2024-04-20T11:36:15.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.160+0000 7f317b200640 1 -- 172.21.15.152:0/3308618138 >> 172.21.15.152:0/3308618138 conn(0x7f317c067ae0 msgr2=0x7f317c06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:15.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.160+0000 7f317b200640 1 -- 172.21.15.152:0/3308618138 shutdown_connections 2024-04-20T11:36:15.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.160+0000 7f317b200640 1 -- 172.21.15.152:0/3308618138 wait complete. 2024-04-20T11:36:15.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.161+0000 7f317b200640 1 Processor -- start 2024-04-20T11:36:15.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.162+0000 7f317b200640 1 -- start start 2024-04-20T11:36:15.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.162+0000 7f317b200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3160004800 0x7f317c1f6900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:15.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.163+0000 7f317b200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f317c1f2990 0x7f317c141910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:15.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.163+0000 7f317b200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f317c14c5c0 0x7f317c14ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:15.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.163+0000 7f317b200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f317c15d740 con 0x7f317c1f2990 2024-04-20T11:36:15.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.163+0000 7f317b200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f317c15d5c0 con 0x7f3160004800 2024-04-20T11:36:15.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.163+0000 7f317b200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f317c15d8c0 con 0x7f317c14c5c0 2024-04-20T11:36:15.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.163+0000 7f3173e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f317c1f2990 0x7f317c141910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:15.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.163+0000 7f3173e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f317c1f2990 0x7f317c141910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35856/0 (socket says 172.21.15.152:35856) 2024-04-20T11:36:15.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.163+0000 7f3173e00640 1 -- 172.21.15.152:0/4227366213 learned_addr learned my addr 172.21.15.152:0/4227366213 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:15.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.163+0000 7f3173e00640 1 -- 172.21.15.152:0/4227366213 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f317c14c5c0 msgr2=0x7f317c14ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:15.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.163+0000 7f317a800640 1 --2- 172.21.15.152:0/4227366213 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f317c14c5c0 0x7f317c14ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:15.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.163+0000 7f3179e00640 1 --2- 172.21.15.152:0/4227366213 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3160004800 0x7f317c1f6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:15.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.163+0000 7f3173e00640 1 --2- 172.21.15.152:0/4227366213 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f317c14c5c0 0x7f317c14ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:15.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.163+0000 7f3173e00640 1 -- 172.21.15.152:0/4227366213 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3160004800 msgr2=0x7f317c1f6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:15.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.163+0000 7f3173e00640 1 --2- 172.21.15.152:0/4227366213 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3160004800 0x7f317c1f6900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:15.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.163+0000 7f3173e00640 1 -- 172.21.15.152:0/4227366213 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f317c142370 con 0x7f317c1f2990 2024-04-20T11:36:15.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.164+0000 7f3173e00640 1 --2- 172.21.15.152:0/4227366213 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f317c1f2990 0x7f317c141910 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7f316402c440 tx=0x7f317c15ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:15.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.164+0000 7f3172000640 1 -- 172.21.15.152:0/4227366213 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3164045070 con 0x7f317c1f2990 2024-04-20T11:36:15.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.164+0000 7f3172000640 1 -- 172.21.15.152:0/4227366213 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3164000ed0 con 0x7f317c1f2990 2024-04-20T11:36:15.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.164+0000 7f3172000640 1 -- 172.21.15.152:0/4227366213 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3164007c40 con 0x7f317c1f2990 2024-04-20T11:36:15.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.164+0000 7f317b200640 1 -- 172.21.15.152:0/4227366213 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f317c1f6e40 con 0x7f317c1f2990 2024-04-20T11:36:15.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.164+0000 7f317b200640 1 -- 172.21.15.152:0/4227366213 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f317c1f3130 con 0x7f317c1f2990 2024-04-20T11:36:15.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.165+0000 7f317b200640 1 -- 172.21.15.152:0/4227366213 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f317c0cfcf0 con 0x7f317c1f2990 2024-04-20T11:36:15.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.166+0000 7f3172000640 1 -- 172.21.15.152:0/4227366213 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 22) ==== 91601+0+0 (secure 0 0 0) 0x7f3164007070 con 0x7f317c1f2990 2024-04-20T11:36:15.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.167+0000 7f3172000640 1 --2- 172.21.15.152:0/4227366213 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f315806cf20 0x7f315806f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:15.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.167+0000 7f3172000640 1 -- 172.21.15.152:0/4227366213 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5709+0+0 (secure 0 0 0) 0x7f31640ad800 con 0x7f317c1f2990 2024-04-20T11:36:15.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.168+0000 7f3179e00640 1 --2- 172.21.15.152:0/4227366213 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f315806cf20 0x7f315806f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:15.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.169+0000 7f3179e00640 1 --2- 172.21.15.152:0/4227366213 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f315806cf20 0x7f315806f3e0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f316c001590 tx=0x7f316c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:15.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.171+0000 7f3172000640 1 -- 172.21.15.152:0/4227366213 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f317c0cfcf0 con 0x7f317c1f2990 2024-04-20T11:36:15.426 INFO:tasks.ceph.mds.b.smithi175.stderr:2024-04-20T11:36:15.426+0000 7fb130c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-20T11:36:15.447 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.447+0000 7f317b200640 1 -- 172.21.15.152:0/4227366213 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f317c141e50 con 0x7f317c1f2990 2024-04-20T11:36:15.448 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.448+0000 7f3172000640 1 -- 172.21.15.152:0/4227366213 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1457 (secure 0 0 0) 0x7f316407ac60 con 0x7f317c1f2990 2024-04-20T11:36:15.448 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:36:15.448 INFO:teuthology.orchestra.run.smithi152.stdout:{"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":581,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":142,"num_osds":8,"num_up_osds":8,"osd_up_since":1713612406,"num_in_osds":8,"osd_in_since":1713612397,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":270626816,"bytes_avail":772823486464,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":96,"id":11,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":11,"rank":0,"name":"b","status":"up:active","gid":9501}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":14,"modified":"2024-04-20T11:35:56.675694+0000","services":{"mds":{"daemons":{"summary":"","b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-20T11:36:15.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.451+0000 7f317b200640 1 -- 172.21.15.152:0/4227366213 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f315806cf20 msgr2=0x7f315806f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:15.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.451+0000 7f317b200640 1 --2- 172.21.15.152:0/4227366213 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f315806cf20 0x7f315806f3e0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f316c001590 tx=0x7f316c009040 comp rx=0 tx=0).stop 2024-04-20T11:36:15.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.451+0000 7f317b200640 1 -- 172.21.15.152:0/4227366213 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f317c1f2990 msgr2=0x7f317c141910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:15.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.451+0000 7f317b200640 1 --2- 172.21.15.152:0/4227366213 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f317c1f2990 0x7f317c141910 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7f316402c440 tx=0x7f317c15ca20 comp rx=0 tx=0).stop 2024-04-20T11:36:15.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.451+0000 7f317b200640 1 -- 172.21.15.152:0/4227366213 shutdown_connections 2024-04-20T11:36:15.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.451+0000 7f317b200640 1 --2- 172.21.15.152:0/4227366213 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f315806cf20 0x7f315806f3e0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:15.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.451+0000 7f317b200640 1 --2- 172.21.15.152:0/4227366213 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f317c14c5c0 0x7f317c14ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:15.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.451+0000 7f317b200640 1 --2- 172.21.15.152:0/4227366213 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f317c1f2990 0x7f317c141910 unknown :-1 s=CLOSED pgs=1979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:15.452 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.451+0000 7f317b200640 1 --2- 172.21.15.152:0/4227366213 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3160004800 0x7f317c1f6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:15.452 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.451+0000 7f317b200640 1 -- 172.21.15.152:0/4227366213 >> 172.21.15.152:0/4227366213 conn(0x7f317c067ae0 msgr2=0x7f317c15ea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:15.452 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.451+0000 7f317b200640 1 -- 172.21.15.152:0/4227366213 shutdown_connections 2024-04-20T11:36:15.452 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.452+0000 7f317b200640 1 -- 172.21.15.152:0/4227366213 wait complete. 2024-04-20T11:36:15.470 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-20T11:36:15.470 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T11:36:15.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.665+0000 7f9d5d000640 1 Processor -- start 2024-04-20T11:36:15.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.665+0000 7f9d5d000640 1 -- start start 2024-04-20T11:36:15.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.666+0000 7f9d5d000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f9d58160b90 0x7f9d5815d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:15.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.666+0000 7f9d5d000640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d580ce8c0 con 0x7f9d58071cd0 2024-04-20T11:36:15.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.667+0000 7f9d5d000640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d581600b0 con 0x7f9d5815d1e0 2024-04-20T11:36:15.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.667+0000 7f9d5d000640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f9d58160390 con 0x7f9d58160b90 2024-04-20T11:36:15.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.667+0000 7f9d56a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f9d58160b90 0x7f9d5815d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:15.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.667+0000 7f9d57e00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f9d58071cd0 0x7f9d58072120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:42446/0 (socket says 172.21.15.152:42446) 2024-04-20T11:36:15.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.667+0000 7f9d57e00640 1 -- 172.21.15.152:0/3117234149 learned_addr learned my addr 172.21.15.152:0/3117234149 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:36:15.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.667+0000 7f9d56a00640 1 -- 172.21.15.152:0/3117234149 >> v1:172.21.15.175:6789/0 conn(0x7f9d5815d1e0 legacy=0x7f9d580cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:15.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.667+0000 7f9d56a00640 1 -- 172.21.15.152:0/3117234149 >> v1:172.21.15.152:6789/0 conn(0x7f9d58071cd0 legacy=0x7f9d58072120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:15.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.668+0000 7f9d56a00640 1 -- 172.21.15.152:0/3117234149 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d58160670 con 0x7f9d58160b90 2024-04-20T11:36:15.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.667+0000 7f9d56000640 1 -- 172.21.15.152:0/3117234149 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4054835358 0 0) 0x7f9d580ce8c0 con 0x7f9d58071cd0 2024-04-20T11:36:15.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.669+0000 7f9d56a00640 1 --2- 172.21.15.152:0/3117234149 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d58160b90 0x7f9d5815d5c0 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7f9d440097d0 tx=0x7f9d4402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82086f97e78d1937 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:15.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.669+0000 7f9d56000640 1 -- 172.21.15.152:0/3117234149 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d44035020 con 0x7f9d58160b90 2024-04-20T11:36:15.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.669+0000 7f9d56000640 1 -- 172.21.15.152:0/3117234149 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d44031830 con 0x7f9d58160b90 2024-04-20T11:36:15.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.669+0000 7f9d56000640 1 -- 172.21.15.152:0/3117234149 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d44031c00 con 0x7f9d58160b90 2024-04-20T11:36:15.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.669+0000 7f9d5d000640 1 -- 172.21.15.152:0/3117234149 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d58160b90 msgr2=0x7f9d5815d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:15.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.669+0000 7f9d5d000640 1 --2- 172.21.15.152:0/3117234149 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d58160b90 0x7f9d5815d5c0 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7f9d440097d0 tx=0x7f9d4402b8d0 comp rx=0 tx=0).stop 2024-04-20T11:36:15.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.670+0000 7f9d5d000640 1 -- 172.21.15.152:0/3117234149 shutdown_connections 2024-04-20T11:36:15.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.670+0000 7f9d5d000640 1 --2- 172.21.15.152:0/3117234149 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d58160b90 0x7f9d5815d5c0 unknown :-1 s=CLOSED pgs=1981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:15.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.670+0000 7f9d5d000640 1 -- 172.21.15.152:0/3117234149 >> 172.21.15.152:0/3117234149 conn(0x7f9d58067ae0 msgr2=0x7f9d5806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:15.670 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.670+0000 7f9d5d000640 1 -- 172.21.15.152:0/3117234149 shutdown_connections 2024-04-20T11:36:15.670 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.670+0000 7f9d5d000640 1 -- 172.21.15.152:0/3117234149 wait complete. 2024-04-20T11:36:15.671 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.671+0000 7f9d5d000640 1 Processor -- start 2024-04-20T11:36:15.671 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.671+0000 7f9d5d000640 1 -- start start 2024-04-20T11:36:15.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.672+0000 7f9d5d000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d58071cd0 0x7f9d58144530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:15.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.672+0000 7f9d5d000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d5815d1e0 0x7f9d581463d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:15.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.672+0000 7f9d5d000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d58160b90 0x7f9d581e5e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:15.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.672+0000 7f9d5d000640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f9d580ccef0 con 0x7f9d58071cd0 2024-04-20T11:36:15.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.672+0000 7f9d5d000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9d580ccd70 con 0x7f9d58160b90 2024-04-20T11:36:15.674 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.672+0000 7f9d5d000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f9d580cd070 con 0x7f9d5815d1e0 2024-04-20T11:36:15.674 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.673+0000 7f9d57400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d58071cd0 0x7f9d58144530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:15.674 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.673+0000 7f9d57400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d58071cd0 0x7f9d58144530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35868/0 (socket says 172.21.15.152:35868) 2024-04-20T11:36:15.674 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.673+0000 7f9d57400640 1 -- 172.21.15.152:0/1004297871 learned_addr learned my addr 172.21.15.152:0/1004297871 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:15.674 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.673+0000 7f9d57400640 1 -- 172.21.15.152:0/1004297871 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d5815d1e0 msgr2=0x7f9d581463d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:15.674 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.673+0000 7f9d57400640 1 --2- 172.21.15.152:0/1004297871 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d5815d1e0 0x7f9d581463d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:15.674 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.673+0000 7f9d57400640 1 -- 172.21.15.152:0/1004297871 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d58160b90 msgr2=0x7f9d581e5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:15.674 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.673+0000 7f9d57400640 1 --2- 172.21.15.152:0/1004297871 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d58160b90 0x7f9d581e5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:15.674 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.673+0000 7f9d57400640 1 -- 172.21.15.152:0/1004297871 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d58146910 con 0x7f9d58071cd0 2024-04-20T11:36:15.674 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.673+0000 7f9d57400640 1 --2- 172.21.15.152:0/1004297871 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d58071cd0 0x7f9d58144530 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7f9d4800c430 tx=0x7f9d4800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:15.675 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.674+0000 7f9d4f400640 1 -- 172.21.15.152:0/1004297871 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d48011790 con 0x7f9d58071cd0 2024-04-20T11:36:15.675 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.674+0000 7f9d4f400640 1 -- 172.21.15.152:0/1004297871 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d48017070 con 0x7f9d58071cd0 2024-04-20T11:36:15.675 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.674+0000 7f9d4f400640 1 -- 172.21.15.152:0/1004297871 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d48011bb0 con 0x7f9d58071cd0 2024-04-20T11:36:15.675 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.674+0000 7f9d5d000640 1 -- 172.21.15.152:0/1004297871 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d581e6d40 con 0x7f9d58071cd0 2024-04-20T11:36:15.675 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.674+0000 7f9d5d000640 1 -- 172.21.15.152:0/1004297871 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d581e6a60 con 0x7f9d58071cd0 2024-04-20T11:36:15.675 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.675+0000 7f9d5d000640 1 -- 172.21.15.152:0/1004297871 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d580cfa10 con 0x7f9d58071cd0 2024-04-20T11:36:15.676 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.676+0000 7f9d4f400640 1 -- 172.21.15.152:0/1004297871 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 22) ==== 91601+0+0 (secure 0 0 0) 0x7f9d4801c460 con 0x7f9d58071cd0 2024-04-20T11:36:15.676 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.677+0000 7f9d4f400640 1 --2- 172.21.15.152:0/1004297871 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f9d3806cf20 0x7f9d3806f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:15.676 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.677+0000 7f9d4f400640 1 -- 172.21.15.152:0/1004297871 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5709+0+0 (secure 0 0 0) 0x7f9d4808c630 con 0x7f9d58071cd0 2024-04-20T11:36:15.677 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.677+0000 7f9d56a00640 1 --2- 172.21.15.152:0/1004297871 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f9d3806cf20 0x7f9d3806f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:15.678 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.679+0000 7f9d56a00640 1 --2- 172.21.15.152:0/1004297871 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f9d3806cf20 0x7f9d3806f3e0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f9d44036000 tx=0x7f9d44030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:15.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.681+0000 7f9d4f400640 1 -- 172.21.15.152:0/1004297871 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f9d580cfa10 con 0x7f9d58071cd0 2024-04-20T11:36:15.877 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.877+0000 7f9d5d000640 1 -- 172.21.15.152:0/1004297871 --> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9d58075e60 con 0x7f9d3806cf20 2024-04-20T11:36:15.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.896+0000 7f9d4f400640 1 -- 172.21.15.152:0/1004297871 <== mgr.4109 v2:172.21.15.175:6836/4021250441 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252484 (secure 0 0 0) 0x7f9d58075e60 con 0x7f9d3806cf20 2024-04-20T11:36:15.897 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:36:15.900 INFO:teuthology.orchestra.run.smithi152.stderr:dumped all 2024-04-20T11:36:15.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.902+0000 7f9d5d000640 1 -- 172.21.15.152:0/1004297871 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f9d3806cf20 msgr2=0x7f9d3806f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:15.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.902+0000 7f9d5d000640 1 --2- 172.21.15.152:0/1004297871 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f9d3806cf20 0x7f9d3806f3e0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f9d44036000 tx=0x7f9d44030000 comp rx=0 tx=0).stop 2024-04-20T11:36:15.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.902+0000 7f9d5d000640 1 -- 172.21.15.152:0/1004297871 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d58071cd0 msgr2=0x7f9d58144530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:15.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.902+0000 7f9d5d000640 1 --2- 172.21.15.152:0/1004297871 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d58071cd0 0x7f9d58144530 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7f9d4800c430 tx=0x7f9d4800c8f0 comp rx=0 tx=0).stop 2024-04-20T11:36:15.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.902+0000 7f9d5d000640 1 -- 172.21.15.152:0/1004297871 shutdown_connections 2024-04-20T11:36:15.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.902+0000 7f9d5d000640 1 --2- 172.21.15.152:0/1004297871 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f9d3806cf20 0x7f9d3806f3e0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:15.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.902+0000 7f9d5d000640 1 --2- 172.21.15.152:0/1004297871 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d58160b90 0x7f9d581e5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:15.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.902+0000 7f9d5d000640 1 --2- 172.21.15.152:0/1004297871 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d5815d1e0 0x7f9d581463d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:15.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.902+0000 7f9d5d000640 1 --2- 172.21.15.152:0/1004297871 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d58071cd0 0x7f9d58144530 unknown :-1 s=CLOSED pgs=1982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:15.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.902+0000 7f9d5d000640 1 -- 172.21.15.152:0/1004297871 >> 172.21.15.152:0/1004297871 conn(0x7f9d58067ae0 msgr2=0x7f9d5806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:15.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.903+0000 7f9d5d000640 1 -- 172.21.15.152:0/1004297871 shutdown_connections 2024-04-20T11:36:15.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:15.903+0000 7f9d5d000640 1 -- 172.21.15.152:0/1004297871 wait complete. 2024-04-20T11:36:15.921 INFO:teuthology.orchestra.run.smithi152.stdout:{"pg_ready":true,"pg_map":{"version":190,"stamp":"2024-04-20T11:36:14.681879+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":224,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":167,"ondisk_log_size":167,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":169,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":264628,"kb_used_data":15176,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754710092,"statfs":{"total":773094113280,"available":772823134208,"internally_reserved":0,"allocated":15540224,"data_stored":7503300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12746,"internal_metadata":255315510},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.237427"},"pg_stats":[{"pgid":"23.3f","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.498594+0000","last_change":"2024-04-20T11:36:09.498245+0000","last_active":"2024-04-20T11:36:10.498594+0000","last_peered":"2024-04-20T11:36:10.498594+0000","last_clean":"2024-04-20T11:36:10.498594+0000","last_became_active":"2024-04-20T11:36:09.497956+0000","last_became_peered":"2024-04-20T11:36:09.497956+0000","last_unstale":"2024-04-20T11:36:10.498594+0000","last_undegraded":"2024-04-20T11:36:10.498594+0000","last_fullsized":"2024-04-20T11:36:10.498594+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:06:21.617876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.3e","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.674949+0000","last_change":"2024-04-20T11:36:09.511171+0000","last_active":"2024-04-20T11:36:10.674949+0000","last_peered":"2024-04-20T11:36:10.674949+0000","last_clean":"2024-04-20T11:36:10.674949+0000","last_became_active":"2024-04-20T11:36:09.510715+0000","last_became_peered":"2024-04-20T11:36:09.510715+0000","last_unstale":"2024-04-20T11:36:10.674949+0000","last_undegraded":"2024-04-20T11:36:10.674949+0000","last_fullsized":"2024-04-20T11:36:10.674949+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:56:28.407018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3d","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.505591+0000","last_change":"2024-04-20T11:36:09.517211+0000","last_active":"2024-04-20T11:36:11.505591+0000","last_peered":"2024-04-20T11:36:11.505591+0000","last_clean":"2024-04-20T11:36:11.505591+0000","last_became_active":"2024-04-20T11:36:09.516886+0000","last_became_peered":"2024-04-20T11:36:09.516886+0000","last_unstale":"2024-04-20T11:36:11.505591+0000","last_undegraded":"2024-04-20T11:36:11.505591+0000","last_fullsized":"2024-04-20T11:36:11.505591+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:11:59.531721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.3c","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.808485+0000","last_change":"2024-04-20T11:36:09.491911+0000","last_active":"2024-04-20T11:36:11.808485+0000","last_peered":"2024-04-20T11:36:11.808485+0000","last_clean":"2024-04-20T11:36:11.808485+0000","last_became_active":"2024-04-20T11:36:09.491136+0000","last_became_peered":"2024-04-20T11:36:09.491136+0000","last_unstale":"2024-04-20T11:36:11.808485+0000","last_undegraded":"2024-04-20T11:36:11.808485+0000","last_fullsized":"2024-04-20T11:36:11.808485+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:41:28.109593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3b","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.674463+0000","last_change":"2024-04-20T11:36:09.515606+0000","last_active":"2024-04-20T11:36:10.674463+0000","last_peered":"2024-04-20T11:36:10.674463+0000","last_clean":"2024-04-20T11:36:10.674463+0000","last_became_active":"2024-04-20T11:36:09.515282+0000","last_became_peered":"2024-04-20T11:36:09.515282+0000","last_unstale":"2024-04-20T11:36:10.674463+0000","last_undegraded":"2024-04-20T11:36:10.674463+0000","last_fullsized":"2024-04-20T11:36:10.674463+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:49:43.328289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3a","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.807103+0000","last_change":"2024-04-20T11:36:09.505767+0000","last_active":"2024-04-20T11:36:11.807103+0000","last_peered":"2024-04-20T11:36:11.807103+0000","last_clean":"2024-04-20T11:36:11.807103+0000","last_became_active":"2024-04-20T11:36:09.502268+0000","last_became_peered":"2024-04-20T11:36:09.502268+0000","last_unstale":"2024-04-20T11:36:11.807103+0000","last_undegraded":"2024-04-20T11:36:11.807103+0000","last_fullsized":"2024-04-20T11:36:11.807103+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:40:06.723507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.39","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.492148+0000","last_change":"2024-04-20T11:36:09.510744+0000","last_active":"2024-04-20T11:36:10.492148+0000","last_peered":"2024-04-20T11:36:10.492148+0000","last_clean":"2024-04-20T11:36:10.492148+0000","last_became_active":"2024-04-20T11:36:09.510385+0000","last_became_peered":"2024-04-20T11:36:09.510385+0000","last_unstale":"2024-04-20T11:36:10.492148+0000","last_undegraded":"2024-04-20T11:36:10.492148+0000","last_fullsized":"2024-04-20T11:36:10.492148+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:44:53.347896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.38","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.675349+0000","last_change":"2024-04-20T11:36:09.509705+0000","last_active":"2024-04-20T11:36:10.675349+0000","last_peered":"2024-04-20T11:36:10.675349+0000","last_clean":"2024-04-20T11:36:10.675349+0000","last_became_active":"2024-04-20T11:36:09.509111+0000","last_became_peered":"2024-04-20T11:36:09.509111+0000","last_unstale":"2024-04-20T11:36:10.675349+0000","last_undegraded":"2024-04-20T11:36:10.675349+0000","last_fullsized":"2024-04-20T11:36:10.675349+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:30:15.532693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.37","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.675032+0000","last_change":"2024-04-20T11:36:09.507670+0000","last_active":"2024-04-20T11:36:10.675032+0000","last_peered":"2024-04-20T11:36:10.675032+0000","last_clean":"2024-04-20T11:36:10.675032+0000","last_became_active":"2024-04-20T11:36:09.507308+0000","last_became_peered":"2024-04-20T11:36:09.507308+0000","last_unstale":"2024-04-20T11:36:10.675032+0000","last_undegraded":"2024-04-20T11:36:10.675032+0000","last_fullsized":"2024-04-20T11:36:10.675032+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:49:27.093205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.36","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.675225+0000","last_change":"2024-04-20T11:36:09.510613+0000","last_active":"2024-04-20T11:36:10.675225+0000","last_peered":"2024-04-20T11:36:10.675225+0000","last_clean":"2024-04-20T11:36:10.675225+0000","last_became_active":"2024-04-20T11:36:09.510280+0000","last_became_peered":"2024-04-20T11:36:09.510280+0000","last_unstale":"2024-04-20T11:36:10.675225+0000","last_undegraded":"2024-04-20T11:36:10.675225+0000","last_fullsized":"2024-04-20T11:36:10.675225+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:41:28.109593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.35","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.674568+0000","last_change":"2024-04-20T11:36:09.516333+0000","last_active":"2024-04-20T11:36:10.674568+0000","last_peered":"2024-04-20T11:36:10.674568+0000","last_clean":"2024-04-20T11:36:10.674568+0000","last_became_active":"2024-04-20T11:36:09.516038+0000","last_became_peered":"2024-04-20T11:36:09.516038+0000","last_unstale":"2024-04-20T11:36:10.674568+0000","last_undegraded":"2024-04-20T11:36:10.674568+0000","last_fullsized":"2024-04-20T11:36:10.674568+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:59:19.149913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.34","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.675001+0000","last_change":"2024-04-20T11:36:09.503669+0000","last_active":"2024-04-20T11:36:10.675001+0000","last_peered":"2024-04-20T11:36:10.675001+0000","last_clean":"2024-04-20T11:36:10.675001+0000","last_became_active":"2024-04-20T11:36:09.503351+0000","last_became_peered":"2024-04-20T11:36:09.503351+0000","last_unstale":"2024-04-20T11:36:10.675001+0000","last_undegraded":"2024-04-20T11:36:10.675001+0000","last_fullsized":"2024-04-20T11:36:10.675001+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:03:17.439635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.33","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.505133+0000","last_change":"2024-04-20T11:36:09.500004+0000","last_active":"2024-04-20T11:36:11.505133+0000","last_peered":"2024-04-20T11:36:11.505133+0000","last_clean":"2024-04-20T11:36:11.505133+0000","last_became_active":"2024-04-20T11:36:09.497333+0000","last_became_peered":"2024-04-20T11:36:09.497333+0000","last_unstale":"2024-04-20T11:36:11.505133+0000","last_undegraded":"2024-04-20T11:36:11.505133+0000","last_fullsized":"2024-04-20T11:36:11.505133+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:55:31.041430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.32","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.506115+0000","last_change":"2024-04-20T11:36:09.513751+0000","last_active":"2024-04-20T11:36:11.506115+0000","last_peered":"2024-04-20T11:36:11.506115+0000","last_clean":"2024-04-20T11:36:11.506115+0000","last_became_active":"2024-04-20T11:36:09.512773+0000","last_became_peered":"2024-04-20T11:36:09.512773+0000","last_unstale":"2024-04-20T11:36:11.506115+0000","last_undegraded":"2024-04-20T11:36:11.506115+0000","last_fullsized":"2024-04-20T11:36:11.506115+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:42:50.753207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.31","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.497845+0000","last_change":"2024-04-20T11:36:09.494578+0000","last_active":"2024-04-20T11:36:10.497845+0000","last_peered":"2024-04-20T11:36:10.497845+0000","last_clean":"2024-04-20T11:36:10.497845+0000","last_became_active":"2024-04-20T11:36:09.494216+0000","last_became_peered":"2024-04-20T11:36:09.494216+0000","last_unstale":"2024-04-20T11:36:10.497845+0000","last_undegraded":"2024-04-20T11:36:10.497845+0000","last_fullsized":"2024-04-20T11:36:10.497845+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:08:37.699106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.30","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.491449+0000","last_change":"2024-04-20T11:36:09.506793+0000","last_active":"2024-04-20T11:36:10.491449+0000","last_peered":"2024-04-20T11:36:10.491449+0000","last_clean":"2024-04-20T11:36:10.491449+0000","last_became_active":"2024-04-20T11:36:09.506217+0000","last_became_peered":"2024-04-20T11:36:09.506217+0000","last_unstale":"2024-04-20T11:36:10.491449+0000","last_undegraded":"2024-04-20T11:36:10.491449+0000","last_fullsized":"2024-04-20T11:36:10.491449+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:08:37.699106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.2f","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.807341+0000","last_change":"2024-04-20T11:36:09.502941+0000","last_active":"2024-04-20T11:36:11.807341+0000","last_peered":"2024-04-20T11:36:11.807341+0000","last_clean":"2024-04-20T11:36:11.807341+0000","last_became_active":"2024-04-20T11:36:09.501204+0000","last_became_peered":"2024-04-20T11:36:09.501204+0000","last_unstale":"2024-04-20T11:36:11.807341+0000","last_undegraded":"2024-04-20T11:36:11.807341+0000","last_fullsized":"2024-04-20T11:36:11.807341+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:41:49.723887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2e","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.496340+0000","last_change":"2024-04-20T11:36:09.497994+0000","last_active":"2024-04-20T11:36:10.496340+0000","last_peered":"2024-04-20T11:36:10.496340+0000","last_clean":"2024-04-20T11:36:10.496340+0000","last_became_active":"2024-04-20T11:36:09.497691+0000","last_became_peered":"2024-04-20T11:36:09.497691+0000","last_unstale":"2024-04-20T11:36:10.496340+0000","last_undegraded":"2024-04-20T11:36:10.496340+0000","last_fullsized":"2024-04-20T11:36:10.496340+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:40:28.853425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2d","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.807963+0000","last_change":"2024-04-20T11:36:09.517462+0000","last_active":"2024-04-20T11:36:11.807963+0000","last_peered":"2024-04-20T11:36:11.807963+0000","last_clean":"2024-04-20T11:36:11.807963+0000","last_became_active":"2024-04-20T11:36:09.517192+0000","last_became_peered":"2024-04-20T11:36:09.517192+0000","last_unstale":"2024-04-20T11:36:11.807963+0000","last_undegraded":"2024-04-20T11:36:11.807963+0000","last_fullsized":"2024-04-20T11:36:11.807963+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:33:01.539999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2c","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.505035+0000","last_change":"2024-04-20T11:36:09.513981+0000","last_active":"2024-04-20T11:36:11.505035+0000","last_peered":"2024-04-20T11:36:11.505035+0000","last_clean":"2024-04-20T11:36:11.505035+0000","last_became_active":"2024-04-20T11:36:09.513653+0000","last_became_peered":"2024-04-20T11:36:09.513653+0000","last_unstale":"2024-04-20T11:36:11.505035+0000","last_undegraded":"2024-04-20T11:36:11.505035+0000","last_fullsized":"2024-04-20T11:36:11.505035+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:14:26.387558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.2b","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.807325+0000","last_change":"2024-04-20T11:36:09.513386+0000","last_active":"2024-04-20T11:36:11.807325+0000","last_peered":"2024-04-20T11:36:11.807325+0000","last_clean":"2024-04-20T11:36:11.807325+0000","last_became_active":"2024-04-20T11:36:09.512484+0000","last_became_peered":"2024-04-20T11:36:09.512484+0000","last_unstale":"2024-04-20T11:36:11.807325+0000","last_undegraded":"2024-04-20T11:36:11.807325+0000","last_fullsized":"2024-04-20T11:36:11.807325+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:49:44.982942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2a","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.675443+0000","last_change":"2024-04-20T11:36:09.501933+0000","last_active":"2024-04-20T11:36:10.675443+0000","last_peered":"2024-04-20T11:36:10.675443+0000","last_clean":"2024-04-20T11:36:10.675443+0000","last_became_active":"2024-04-20T11:36:09.501168+0000","last_became_peered":"2024-04-20T11:36:09.501168+0000","last_unstale":"2024-04-20T11:36:10.675443+0000","last_undegraded":"2024-04-20T11:36:10.675443+0000","last_fullsized":"2024-04-20T11:36:10.675443+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:10:11.720624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.29","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.806672+0000","last_change":"2024-04-20T11:36:09.501305+0000","last_active":"2024-04-20T11:36:11.806672+0000","last_peered":"2024-04-20T11:36:11.806672+0000","last_clean":"2024-04-20T11:36:11.806672+0000","last_became_active":"2024-04-20T11:36:09.500714+0000","last_became_peered":"2024-04-20T11:36:09.500714+0000","last_unstale":"2024-04-20T11:36:11.806672+0000","last_undegraded":"2024-04-20T11:36:11.806672+0000","last_fullsized":"2024-04-20T11:36:11.806672+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:55:38.517484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.28","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.674679+0000","last_change":"2024-04-20T11:36:09.503752+0000","last_active":"2024-04-20T11:36:10.674679+0000","last_peered":"2024-04-20T11:36:10.674679+0000","last_clean":"2024-04-20T11:36:10.674679+0000","last_became_active":"2024-04-20T11:36:09.501356+0000","last_became_peered":"2024-04-20T11:36:09.501356+0000","last_unstale":"2024-04-20T11:36:10.674679+0000","last_undegraded":"2024-04-20T11:36:10.674679+0000","last_fullsized":"2024-04-20T11:36:10.674679+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:52:33.986991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.27","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.505645+0000","last_change":"2024-04-20T11:36:09.508345+0000","last_active":"2024-04-20T11:36:11.505645+0000","last_peered":"2024-04-20T11:36:11.505645+0000","last_clean":"2024-04-20T11:36:11.505645+0000","last_became_active":"2024-04-20T11:36:09.508006+0000","last_became_peered":"2024-04-20T11:36:09.508006+0000","last_unstale":"2024-04-20T11:36:11.505645+0000","last_undegraded":"2024-04-20T11:36:11.505645+0000","last_fullsized":"2024-04-20T11:36:11.505645+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:34:42.652012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.26","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.491806+0000","last_change":"2024-04-20T11:36:09.502314+0000","last_active":"2024-04-20T11:36:10.491806+0000","last_peered":"2024-04-20T11:36:10.491806+0000","last_clean":"2024-04-20T11:36:10.491806+0000","last_became_active":"2024-04-20T11:36:09.502005+0000","last_became_peered":"2024-04-20T11:36:09.502005+0000","last_unstale":"2024-04-20T11:36:10.491806+0000","last_undegraded":"2024-04-20T11:36:10.491806+0000","last_fullsized":"2024-04-20T11:36:10.491806+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:10:13.066071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.25","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.506008+0000","last_change":"2024-04-20T11:36:09.498214+0000","last_active":"2024-04-20T11:36:11.506008+0000","last_peered":"2024-04-20T11:36:11.506008+0000","last_clean":"2024-04-20T11:36:11.506008+0000","last_became_active":"2024-04-20T11:36:09.497877+0000","last_became_peered":"2024-04-20T11:36:09.497877+0000","last_unstale":"2024-04-20T11:36:11.506008+0000","last_undegraded":"2024-04-20T11:36:11.506008+0000","last_fullsized":"2024-04-20T11:36:11.506008+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:20:16.280679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.24","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.808688+0000","last_change":"2024-04-20T11:36:09.502782+0000","last_active":"2024-04-20T11:36:11.808688+0000","last_peered":"2024-04-20T11:36:11.808688+0000","last_clean":"2024-04-20T11:36:11.808688+0000","last_became_active":"2024-04-20T11:36:09.502509+0000","last_became_peered":"2024-04-20T11:36:09.502509+0000","last_unstale":"2024-04-20T11:36:11.808688+0000","last_undegraded":"2024-04-20T11:36:11.808688+0000","last_fullsized":"2024-04-20T11:36:11.808688+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:30:15.532693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.23","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.505461+0000","last_change":"2024-04-20T11:36:09.500622+0000","last_active":"2024-04-20T11:36:11.505461+0000","last_peered":"2024-04-20T11:36:11.505461+0000","last_clean":"2024-04-20T11:36:11.505461+0000","last_became_active":"2024-04-20T11:36:09.500341+0000","last_became_peered":"2024-04-20T11:36:09.500341+0000","last_unstale":"2024-04-20T11:36:11.505461+0000","last_undegraded":"2024-04-20T11:36:11.505461+0000","last_fullsized":"2024-04-20T11:36:11.505461+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:36:10.575443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.22","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.512516+0000","last_change":"2024-04-20T11:36:09.502504+0000","last_active":"2024-04-20T11:36:11.512516+0000","last_peered":"2024-04-20T11:36:11.512516+0000","last_clean":"2024-04-20T11:36:11.512516+0000","last_became_active":"2024-04-20T11:36:09.502201+0000","last_became_peered":"2024-04-20T11:36:09.502201+0000","last_unstale":"2024-04-20T11:36:11.512516+0000","last_undegraded":"2024-04-20T11:36:11.512516+0000","last_fullsized":"2024-04-20T11:36:11.512516+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:42:28.478688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.21","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.512231+0000","last_change":"2024-04-20T11:36:09.519295+0000","last_active":"2024-04-20T11:36:11.512231+0000","last_peered":"2024-04-20T11:36:11.512231+0000","last_clean":"2024-04-20T11:36:11.512231+0000","last_became_active":"2024-04-20T11:36:09.518233+0000","last_became_peered":"2024-04-20T11:36:09.518233+0000","last_unstale":"2024-04-20T11:36:11.512231+0000","last_undegraded":"2024-04-20T11:36:11.512231+0000","last_fullsized":"2024-04-20T11:36:11.512231+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:34:50.865938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.20","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.495715+0000","last_change":"2024-04-20T11:36:09.495789+0000","last_active":"2024-04-20T11:36:10.495715+0000","last_peered":"2024-04-20T11:36:10.495715+0000","last_clean":"2024-04-20T11:36:10.495715+0000","last_became_active":"2024-04-20T11:36:09.495472+0000","last_became_peered":"2024-04-20T11:36:09.495472+0000","last_unstale":"2024-04-20T11:36:10.495715+0000","last_undegraded":"2024-04-20T11:36:10.495715+0000","last_fullsized":"2024-04-20T11:36:10.495715+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:34:42.831349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.492044+0000","last_change":"2024-04-20T11:36:09.509822+0000","last_active":"2024-04-20T11:36:10.492044+0000","last_peered":"2024-04-20T11:36:10.492044+0000","last_clean":"2024-04-20T11:36:10.492044+0000","last_became_active":"2024-04-20T11:36:09.509521+0000","last_became_peered":"2024-04-20T11:36:09.509521+0000","last_unstale":"2024-04-20T11:36:10.492044+0000","last_undegraded":"2024-04-20T11:36:10.492044+0000","last_fullsized":"2024-04-20T11:36:10.492044+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:44:16.352910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1f","version":"141'4","reported_seq":18,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.498511+0000","last_change":"2024-04-20T11:36:08.483872+0000","last_active":"2024-04-20T11:36:10.498511+0000","last_peered":"2024-04-20T11:36:10.498511+0000","last_clean":"2024-04-20T11:36:10.498511+0000","last_became_active":"2024-04-20T11:36:08.483196+0000","last_became_peered":"2024-04-20T11:36:08.483196+0000","last_unstale":"2024-04-20T11:36:10.498511+0000","last_undegraded":"2024-04-20T11:36:10.498511+0000","last_fullsized":"2024-04-20T11:36:10.498511+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:37:38.832794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.513632+0000","last_change":"2024-04-20T11:36:09.505474+0000","last_active":"2024-04-20T11:36:11.513632+0000","last_peered":"2024-04-20T11:36:11.513632+0000","last_clean":"2024-04-20T11:36:11.513632+0000","last_became_active":"2024-04-20T11:36:09.502465+0000","last_became_peered":"2024-04-20T11:36:09.502465+0000","last_unstale":"2024-04-20T11:36:11.513632+0000","last_undegraded":"2024-04-20T11:36:11.513632+0000","last_fullsized":"2024-04-20T11:36:11.513632+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:51:23.507394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1e","version":"141'1","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.512968+0000","last_change":"2024-04-20T11:36:08.478064+0000","last_active":"2024-04-20T11:36:11.512968+0000","last_peered":"2024-04-20T11:36:11.512968+0000","last_clean":"2024-04-20T11:36:11.512968+0000","last_became_active":"2024-04-20T11:36:08.477788+0000","last_became_peered":"2024-04-20T11:36:08.477788+0000","last_unstale":"2024-04-20T11:36:11.512968+0000","last_undegraded":"2024-04-20T11:36:11.512968+0000","last_fullsized":"2024-04-20T11:36:11.512968+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:10:10.704042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.808302+0000","last_change":"2024-04-20T11:36:09.506372+0000","last_active":"2024-04-20T11:36:11.808302+0000","last_peered":"2024-04-20T11:36:11.808302+0000","last_clean":"2024-04-20T11:36:11.808302+0000","last_became_active":"2024-04-20T11:36:09.506107+0000","last_became_peered":"2024-04-20T11:36:09.506107+0000","last_unstale":"2024-04-20T11:36:11.808302+0000","last_undegraded":"2024-04-20T11:36:11.808302+0000","last_fullsized":"2024-04-20T11:36:11.808302+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:21:08.136339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1d","version":"141'2","reported_seq":16,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.808049+0000","last_change":"2024-04-20T11:36:08.479424+0000","last_active":"2024-04-20T11:36:11.808049+0000","last_peered":"2024-04-20T11:36:11.808049+0000","last_clean":"2024-04-20T11:36:11.808049+0000","last_became_active":"2024-04-20T11:36:08.479137+0000","last_became_peered":"2024-04-20T11:36:08.479137+0000","last_unstale":"2024-04-20T11:36:11.808049+0000","last_undegraded":"2024-04-20T11:36:11.808049+0000","last_fullsized":"2024-04-20T11:36:11.808049+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:25:33.273088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.506044+0000","last_change":"2024-04-20T11:36:09.498658+0000","last_active":"2024-04-20T11:36:11.506044+0000","last_peered":"2024-04-20T11:36:11.506044+0000","last_clean":"2024-04-20T11:36:11.506044+0000","last_became_active":"2024-04-20T11:36:09.498372+0000","last_became_peered":"2024-04-20T11:36:09.498372+0000","last_unstale":"2024-04-20T11:36:11.506044+0000","last_undegraded":"2024-04-20T11:36:11.506044+0000","last_fullsized":"2024-04-20T11:36:11.506044+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:14:54.539911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1c","version":"141'2","reported_seq":16,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.808844+0000","last_change":"2024-04-20T11:36:08.481345+0000","last_active":"2024-04-20T11:36:11.808844+0000","last_peered":"2024-04-20T11:36:11.808844+0000","last_clean":"2024-04-20T11:36:11.808844+0000","last_became_active":"2024-04-20T11:36:08.480807+0000","last_became_peered":"2024-04-20T11:36:08.480807+0000","last_unstale":"2024-04-20T11:36:11.808844+0000","last_undegraded":"2024-04-20T11:36:11.808844+0000","last_fullsized":"2024-04-20T11:36:11.808844+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:53:50.523715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.491980+0000","last_change":"2024-04-20T11:36:09.498794+0000","last_active":"2024-04-20T11:36:10.491980+0000","last_peered":"2024-04-20T11:36:10.491980+0000","last_clean":"2024-04-20T11:36:10.491980+0000","last_became_active":"2024-04-20T11:36:09.498063+0000","last_became_peered":"2024-04-20T11:36:09.498063+0000","last_unstale":"2024-04-20T11:36:10.491980+0000","last_undegraded":"2024-04-20T11:36:10.491980+0000","last_fullsized":"2024-04-20T11:36:10.491980+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:00:28.716804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1b","version":"141'1","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.505504+0000","last_change":"2024-04-20T11:36:08.473692+0000","last_active":"2024-04-20T11:36:11.505504+0000","last_peered":"2024-04-20T11:36:11.505504+0000","last_clean":"2024-04-20T11:36:11.505504+0000","last_became_active":"2024-04-20T11:36:08.473388+0000","last_became_peered":"2024-04-20T11:36:08.473388+0000","last_unstale":"2024-04-20T11:36:11.505504+0000","last_undegraded":"2024-04-20T11:36:11.505504+0000","last_fullsized":"2024-04-20T11:36:11.505504+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:09:00.631652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.807706+0000","last_change":"2024-04-20T11:36:09.496254+0000","last_active":"2024-04-20T11:36:11.807706+0000","last_peered":"2024-04-20T11:36:11.807706+0000","last_clean":"2024-04-20T11:36:11.807706+0000","last_became_active":"2024-04-20T11:36:09.492502+0000","last_became_peered":"2024-04-20T11:36:09.492502+0000","last_unstale":"2024-04-20T11:36:11.807706+0000","last_undegraded":"2024-04-20T11:36:11.807706+0000","last_fullsized":"2024-04-20T11:36:11.807706+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:49:27.093205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.1a","version":"141'1","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.808305+0000","last_change":"2024-04-20T11:36:08.472809+0000","last_active":"2024-04-20T11:36:11.808305+0000","last_peered":"2024-04-20T11:36:11.808305+0000","last_clean":"2024-04-20T11:36:11.808305+0000","last_became_active":"2024-04-20T11:36:08.472197+0000","last_became_peered":"2024-04-20T11:36:08.472197+0000","last_unstale":"2024-04-20T11:36:11.808305+0000","last_undegraded":"2024-04-20T11:36:11.808305+0000","last_fullsized":"2024-04-20T11:36:11.808305+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:08:24.707643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.807054+0000","last_change":"2024-04-20T11:36:09.483939+0000","last_active":"2024-04-20T11:36:11.807054+0000","last_peered":"2024-04-20T11:36:11.807054+0000","last_clean":"2024-04-20T11:36:11.807054+0000","last_became_active":"2024-04-20T11:36:09.483166+0000","last_became_peered":"2024-04-20T11:36:09.483166+0000","last_unstale":"2024-04-20T11:36:11.807054+0000","last_undegraded":"2024-04-20T11:36:11.807054+0000","last_fullsized":"2024-04-20T11:36:11.807054+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:03:17.439635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.674387+0000","last_change":"2024-04-20T11:36:08.481523+0000","last_active":"2024-04-20T11:36:10.674387+0000","last_peered":"2024-04-20T11:36:10.674387+0000","last_clean":"2024-04-20T11:36:10.674387+0000","last_became_active":"2024-04-20T11:36:08.481220+0000","last_became_peered":"2024-04-20T11:36:08.481220+0000","last_unstale":"2024-04-20T11:36:10.674387+0000","last_undegraded":"2024-04-20T11:36:10.674387+0000","last_fullsized":"2024-04-20T11:36:10.674387+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:27:21.500530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.806610+0000","last_change":"2024-04-20T11:36:09.499280+0000","last_active":"2024-04-20T11:36:11.806610+0000","last_peered":"2024-04-20T11:36:11.806610+0000","last_clean":"2024-04-20T11:36:11.806610+0000","last_became_active":"2024-04-20T11:36:09.499008+0000","last_became_peered":"2024-04-20T11:36:09.499008+0000","last_unstale":"2024-04-20T11:36:11.806610+0000","last_undegraded":"2024-04-20T11:36:11.806610+0000","last_fullsized":"2024-04-20T11:36:11.806610+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:35:54.753258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.491574+0000","last_change":"2024-04-20T11:36:08.459181+0000","last_active":"2024-04-20T11:36:10.491574+0000","last_peered":"2024-04-20T11:36:10.491574+0000","last_clean":"2024-04-20T11:36:10.491574+0000","last_became_active":"2024-04-20T11:36:08.455877+0000","last_became_peered":"2024-04-20T11:36:08.455877+0000","last_unstale":"2024-04-20T11:36:10.491574+0000","last_undegraded":"2024-04-20T11:36:10.491574+0000","last_fullsized":"2024-04-20T11:36:10.491574+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:03:08.918074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.496622+0000","last_change":"2024-04-20T11:36:09.498684+0000","last_active":"2024-04-20T11:36:10.496622+0000","last_peered":"2024-04-20T11:36:10.496622+0000","last_clean":"2024-04-20T11:36:10.496622+0000","last_became_active":"2024-04-20T11:36:09.498116+0000","last_became_peered":"2024-04-20T11:36:09.498116+0000","last_unstale":"2024-04-20T11:36:10.496622+0000","last_undegraded":"2024-04-20T11:36:10.496622+0000","last_fullsized":"2024-04-20T11:36:10.496622+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:06:21.617876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.807834+0000","last_change":"2024-04-20T11:36:08.472135+0000","last_active":"2024-04-20T11:36:11.807834+0000","last_peered":"2024-04-20T11:36:11.807834+0000","last_clean":"2024-04-20T11:36:11.807834+0000","last_became_active":"2024-04-20T11:36:08.471529+0000","last_became_peered":"2024-04-20T11:36:08.471529+0000","last_unstale":"2024-04-20T11:36:11.807834+0000","last_undegraded":"2024-04-20T11:36:11.807834+0000","last_fullsized":"2024-04-20T11:36:11.807834+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:29:38.601496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.496388+0000","last_change":"2024-04-20T11:36:09.497525+0000","last_active":"2024-04-20T11:36:10.496388+0000","last_peered":"2024-04-20T11:36:10.496388+0000","last_clean":"2024-04-20T11:36:10.496388+0000","last_became_active":"2024-04-20T11:36:09.497185+0000","last_became_peered":"2024-04-20T11:36:09.497185+0000","last_unstale":"2024-04-20T11:36:10.496388+0000","last_undegraded":"2024-04-20T11:36:10.496388+0000","last_fullsized":"2024-04-20T11:36:10.496388+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:39:12.611985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.15","version":"141'3","reported_seq":17,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.807992+0000","last_change":"2024-04-20T11:36:08.474633+0000","last_active":"2024-04-20T11:36:11.807992+0000","last_peered":"2024-04-20T11:36:11.807992+0000","last_clean":"2024-04-20T11:36:11.807992+0000","last_became_active":"2024-04-20T11:36:08.474310+0000","last_became_peered":"2024-04-20T11:36:08.474310+0000","last_unstale":"2024-04-20T11:36:11.807992+0000","last_undegraded":"2024-04-20T11:36:11.807992+0000","last_fullsized":"2024-04-20T11:36:11.807992+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:43:13.653616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.15","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.513740+0000","last_change":"2024-04-20T11:36:09.519775+0000","last_active":"2024-04-20T11:36:11.513740+0000","last_peered":"2024-04-20T11:36:11.513740+0000","last_clean":"2024-04-20T11:36:11.513740+0000","last_became_active":"2024-04-20T11:36:09.516787+0000","last_became_peered":"2024-04-20T11:36:09.516787+0000","last_unstale":"2024-04-20T11:36:11.513740+0000","last_undegraded":"2024-04-20T11:36:11.513740+0000","last_fullsized":"2024-04-20T11:36:11.513740+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:53:29.084025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.14","version":"141'6","reported_seq":20,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.512759+0000","last_change":"2024-04-20T11:36:08.481226+0000","last_active":"2024-04-20T11:36:11.512759+0000","last_peered":"2024-04-20T11:36:11.512759+0000","last_clean":"2024-04-20T11:36:11.512759+0000","last_became_active":"2024-04-20T11:36:08.480392+0000","last_became_peered":"2024-04-20T11:36:08.480392+0000","last_unstale":"2024-04-20T11:36:11.512759+0000","last_undegraded":"2024-04-20T11:36:11.512759+0000","last_fullsized":"2024-04-20T11:36:11.512759+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:07:03.390165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.495849+0000","last_change":"2024-04-20T11:36:09.505061+0000","last_active":"2024-04-20T11:36:10.495849+0000","last_peered":"2024-04-20T11:36:10.495849+0000","last_clean":"2024-04-20T11:36:10.495849+0000","last_became_active":"2024-04-20T11:36:09.504620+0000","last_became_peered":"2024-04-20T11:36:09.504620+0000","last_unstale":"2024-04-20T11:36:10.495849+0000","last_undegraded":"2024-04-20T11:36:10.495849+0000","last_fullsized":"2024-04-20T11:36:10.495849+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:16:01.895857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.13","version":"141'2","reported_seq":16,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.807629+0000","last_change":"2024-04-20T11:36:08.481075+0000","last_active":"2024-04-20T11:36:11.807629+0000","last_peered":"2024-04-20T11:36:11.807629+0000","last_clean":"2024-04-20T11:36:11.807629+0000","last_became_active":"2024-04-20T11:36:08.480423+0000","last_became_peered":"2024-04-20T11:36:08.480423+0000","last_unstale":"2024-04-20T11:36:11.807629+0000","last_undegraded":"2024-04-20T11:36:11.807629+0000","last_fullsized":"2024-04-20T11:36:11.807629+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:43:49.370831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.492241+0000","last_change":"2024-04-20T11:36:09.503953+0000","last_active":"2024-04-20T11:36:10.492241+0000","last_peered":"2024-04-20T11:36:10.492241+0000","last_clean":"2024-04-20T11:36:10.492241+0000","last_became_active":"2024-04-20T11:36:09.503682+0000","last_became_peered":"2024-04-20T11:36:09.503682+0000","last_unstale":"2024-04-20T11:36:10.492241+0000","last_undegraded":"2024-04-20T11:36:10.492241+0000","last_fullsized":"2024-04-20T11:36:10.492241+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:32:53.769485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.496025+0000","last_change":"2024-04-20T11:36:08.469986+0000","last_active":"2024-04-20T11:36:10.496025+0000","last_peered":"2024-04-20T11:36:10.496025+0000","last_clean":"2024-04-20T11:36:10.496025+0000","last_became_active":"2024-04-20T11:36:08.469638+0000","last_became_peered":"2024-04-20T11:36:08.469638+0000","last_unstale":"2024-04-20T11:36:10.496025+0000","last_undegraded":"2024-04-20T11:36:10.496025+0000","last_fullsized":"2024-04-20T11:36:10.496025+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:51:13.378758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.807021+0000","last_change":"2024-04-20T11:36:09.503606+0000","last_active":"2024-04-20T11:36:11.807021+0000","last_peered":"2024-04-20T11:36:11.807021+0000","last_clean":"2024-04-20T11:36:11.807021+0000","last_became_active":"2024-04-20T11:36:09.503261+0000","last_became_peered":"2024-04-20T11:36:09.503261+0000","last_unstale":"2024-04-20T11:36:11.807021+0000","last_undegraded":"2024-04-20T11:36:11.807021+0000","last_fullsized":"2024-04-20T11:36:11.807021+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:56:28.407018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.807199+0000","last_change":"2024-04-20T11:36:08.477055+0000","last_active":"2024-04-20T11:36:11.807199+0000","last_peered":"2024-04-20T11:36:11.807199+0000","last_clean":"2024-04-20T11:36:11.807199+0000","last_became_active":"2024-04-20T11:36:08.476730+0000","last_became_peered":"2024-04-20T11:36:08.476730+0000","last_unstale":"2024-04-20T11:36:11.807199+0000","last_undegraded":"2024-04-20T11:36:11.807199+0000","last_fullsized":"2024-04-20T11:36:11.807199+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:00:31.304085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.504738+0000","last_change":"2024-04-20T11:36:09.504805+0000","last_active":"2024-04-20T11:36:11.504738+0000","last_peered":"2024-04-20T11:36:11.504738+0000","last_clean":"2024-04-20T11:36:11.504738+0000","last_became_active":"2024-04-20T11:36:09.504541+0000","last_became_peered":"2024-04-20T11:36:09.504541+0000","last_unstale":"2024-04-20T11:36:11.504738+0000","last_undegraded":"2024-04-20T11:36:11.504738+0000","last_fullsized":"2024-04-20T11:36:11.504738+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:03:06.526803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.10","version":"141'8","reported_seq":22,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.511978+0000","last_change":"2024-04-20T11:36:08.476771+0000","last_active":"2024-04-20T11:36:11.511978+0000","last_peered":"2024-04-20T11:36:11.511978+0000","last_clean":"2024-04-20T11:36:11.511978+0000","last_became_active":"2024-04-20T11:36:08.476124+0000","last_became_peered":"2024-04-20T11:36:08.476124+0000","last_unstale":"2024-04-20T11:36:11.511978+0000","last_undegraded":"2024-04-20T11:36:11.511978+0000","last_fullsized":"2024-04-20T11:36:11.511978+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:41:27.093010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.809290+0000","last_change":"2024-04-20T11:36:09.500469+0000","last_active":"2024-04-20T11:36:11.809290+0000","last_peered":"2024-04-20T11:36:11.809290+0000","last_clean":"2024-04-20T11:36:11.809290+0000","last_became_active":"2024-04-20T11:36:09.495917+0000","last_became_peered":"2024-04-20T11:36:09.495917+0000","last_unstale":"2024-04-20T11:36:11.809290+0000","last_undegraded":"2024-04-20T11:36:11.809290+0000","last_fullsized":"2024-04-20T11:36:11.809290+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:07:04.406747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"141'1","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.505732+0000","last_change":"2024-04-20T11:36:08.479096+0000","last_active":"2024-04-20T11:36:11.505732+0000","last_peered":"2024-04-20T11:36:11.505732+0000","last_clean":"2024-04-20T11:36:11.505732+0000","last_became_active":"2024-04-20T11:36:08.478807+0000","last_became_peered":"2024-04-20T11:36:08.478807+0000","last_unstale":"2024-04-20T11:36:11.505732+0000","last_undegraded":"2024-04-20T11:36:11.505732+0000","last_fullsized":"2024-04-20T11:36:11.505732+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:25:58.656867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.496824+0000","last_change":"2024-04-20T11:36:09.510407+0000","last_active":"2024-04-20T11:36:10.496824+0000","last_peered":"2024-04-20T11:36:10.496824+0000","last_clean":"2024-04-20T11:36:10.496824+0000","last_became_active":"2024-04-20T11:36:09.510132+0000","last_became_peered":"2024-04-20T11:36:09.510132+0000","last_unstale":"2024-04-20T11:36:10.496824+0000","last_undegraded":"2024-04-20T11:36:10.496824+0000","last_fullsized":"2024-04-20T11:36:10.496824+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:13:57.374934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.e","version":"141'2","reported_seq":16,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.505227+0000","last_change":"2024-04-20T11:36:08.475925+0000","last_active":"2024-04-20T11:36:11.505227+0000","last_peered":"2024-04-20T11:36:11.505227+0000","last_clean":"2024-04-20T11:36:11.505227+0000","last_became_active":"2024-04-20T11:36:08.474739+0000","last_became_peered":"2024-04-20T11:36:08.474739+0000","last_unstale":"2024-04-20T11:36:11.505227+0000","last_undegraded":"2024-04-20T11:36:11.505227+0000","last_fullsized":"2024-04-20T11:36:11.505227+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:52:49.350436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.808772+0000","last_change":"2024-04-20T11:36:09.497659+0000","last_active":"2024-04-20T11:36:11.808772+0000","last_peered":"2024-04-20T11:36:11.808772+0000","last_clean":"2024-04-20T11:36:11.808772+0000","last_became_active":"2024-04-20T11:36:09.495859+0000","last_became_peered":"2024-04-20T11:36:09.495859+0000","last_unstale":"2024-04-20T11:36:11.808772+0000","last_undegraded":"2024-04-20T11:36:11.808772+0000","last_fullsized":"2024-04-20T11:36:11.808772+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:10:11.720624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.d","version":"141'1","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.498977+0000","last_change":"2024-04-20T11:36:08.484073+0000","last_active":"2024-04-20T11:36:10.498977+0000","last_peered":"2024-04-20T11:36:10.498977+0000","last_clean":"2024-04-20T11:36:10.498977+0000","last_became_active":"2024-04-20T11:36:08.483348+0000","last_became_peered":"2024-04-20T11:36:08.483348+0000","last_unstale":"2024-04-20T11:36:10.498977+0000","last_undegraded":"2024-04-20T11:36:10.498977+0000","last_fullsized":"2024-04-20T11:36:10.498977+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:58:31.453250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.491566+0000","last_change":"2024-04-20T11:36:09.515509+0000","last_active":"2024-04-20T11:36:10.491566+0000","last_peered":"2024-04-20T11:36:10.491566+0000","last_clean":"2024-04-20T11:36:10.491566+0000","last_became_active":"2024-04-20T11:36:09.515199+0000","last_became_peered":"2024-04-20T11:36:09.515199+0000","last_unstale":"2024-04-20T11:36:10.491566+0000","last_undegraded":"2024-04-20T11:36:10.491566+0000","last_fullsized":"2024-04-20T11:36:10.491566+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:06:21.617876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.498695+0000","last_change":"2024-04-20T11:36:08.481420+0000","last_active":"2024-04-20T11:36:10.498695+0000","last_peered":"2024-04-20T11:36:10.498695+0000","last_clean":"2024-04-20T11:36:10.498695+0000","last_became_active":"2024-04-20T11:36:08.481118+0000","last_became_peered":"2024-04-20T11:36:08.481118+0000","last_unstale":"2024-04-20T11:36:10.498695+0000","last_undegraded":"2024-04-20T11:36:10.498695+0000","last_fullsized":"2024-04-20T11:36:10.498695+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:27:32.804957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.504806+0000","last_change":"2024-04-20T11:36:09.504033+0000","last_active":"2024-04-20T11:36:11.504806+0000","last_peered":"2024-04-20T11:36:11.504806+0000","last_clean":"2024-04-20T11:36:11.504806+0000","last_became_active":"2024-04-20T11:36:09.503676+0000","last_became_peered":"2024-04-20T11:36:09.503676+0000","last_unstale":"2024-04-20T11:36:11.504806+0000","last_undegraded":"2024-04-20T11:36:11.504806+0000","last_fullsized":"2024-04-20T11:36:11.504806+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:56:53.061204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.b","version":"141'2","reported_seq":16,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.807844+0000","last_change":"2024-04-20T11:36:08.479935+0000","last_active":"2024-04-20T11:36:11.807844+0000","last_peered":"2024-04-20T11:36:11.807844+0000","last_clean":"2024-04-20T11:36:11.807844+0000","last_became_active":"2024-04-20T11:36:08.479361+0000","last_became_peered":"2024-04-20T11:36:08.479361+0000","last_unstale":"2024-04-20T11:36:11.807844+0000","last_undegraded":"2024-04-20T11:36:11.807844+0000","last_fullsized":"2024-04-20T11:36:11.807844+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:36:59.262204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.492424+0000","last_change":"2024-04-20T11:36:09.508916+0000","last_active":"2024-04-20T11:36:10.492424+0000","last_peered":"2024-04-20T11:36:10.492424+0000","last_clean":"2024-04-20T11:36:10.492424+0000","last_became_active":"2024-04-20T11:36:09.506508+0000","last_became_peered":"2024-04-20T11:36:09.506508+0000","last_unstale":"2024-04-20T11:36:10.492424+0000","last_undegraded":"2024-04-20T11:36:10.492424+0000","last_fullsized":"2024-04-20T11:36:10.492424+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:37:42.247415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.498687+0000","last_change":"2024-04-20T11:36:08.480015+0000","last_active":"2024-04-20T11:36:10.498687+0000","last_peered":"2024-04-20T11:36:10.498687+0000","last_clean":"2024-04-20T11:36:10.498687+0000","last_became_active":"2024-04-20T11:36:08.479699+0000","last_became_peered":"2024-04-20T11:36:08.479699+0000","last_unstale":"2024-04-20T11:36:10.498687+0000","last_undegraded":"2024-04-20T11:36:10.498687+0000","last_fullsized":"2024-04-20T11:36:10.498687+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:54:19.912464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.497824+0000","last_change":"2024-04-20T11:36:09.503816+0000","last_active":"2024-04-20T11:36:10.497824+0000","last_peered":"2024-04-20T11:36:10.497824+0000","last_clean":"2024-04-20T11:36:10.497824+0000","last_became_active":"2024-04-20T11:36:09.503166+0000","last_became_peered":"2024-04-20T11:36:09.503166+0000","last_unstale":"2024-04-20T11:36:10.497824+0000","last_undegraded":"2024-04-20T11:36:10.497824+0000","last_fullsized":"2024-04-20T11:36:10.497824+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:56:54.406631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.807386+0000","last_change":"2024-04-20T11:36:08.477340+0000","last_active":"2024-04-20T11:36:11.807386+0000","last_peered":"2024-04-20T11:36:11.807386+0000","last_clean":"2024-04-20T11:36:11.807386+0000","last_became_active":"2024-04-20T11:36:08.476853+0000","last_became_peered":"2024-04-20T11:36:08.476853+0000","last_unstale":"2024-04-20T11:36:11.807386+0000","last_undegraded":"2024-04-20T11:36:11.807386+0000","last_fullsized":"2024-04-20T11:36:11.807386+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:52:40.058619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.806778+0000","last_change":"2024-04-20T11:36:09.504017+0000","last_active":"2024-04-20T11:36:11.806778+0000","last_peered":"2024-04-20T11:36:11.806778+0000","last_clean":"2024-04-20T11:36:11.806778+0000","last_became_active":"2024-04-20T11:36:09.503623+0000","last_became_peered":"2024-04-20T11:36:09.503623+0000","last_unstale":"2024-04-20T11:36:11.806778+0000","last_undegraded":"2024-04-20T11:36:11.806778+0000","last_fullsized":"2024-04-20T11:36:11.806778+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:42:34.318459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.497950+0000","last_change":"2024-04-20T11:36:08.481304+0000","last_active":"2024-04-20T11:36:10.497950+0000","last_peered":"2024-04-20T11:36:10.497950+0000","last_clean":"2024-04-20T11:36:10.497950+0000","last_became_active":"2024-04-20T11:36:08.480957+0000","last_became_peered":"2024-04-20T11:36:08.480957+0000","last_unstale":"2024-04-20T11:36:10.497950+0000","last_undegraded":"2024-04-20T11:36:10.497950+0000","last_fullsized":"2024-04-20T11:36:10.497950+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:34:41.814767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.496549+0000","last_change":"2024-04-20T11:36:09.507310+0000","last_active":"2024-04-20T11:36:10.496549+0000","last_peered":"2024-04-20T11:36:10.496549+0000","last_clean":"2024-04-20T11:36:10.496549+0000","last_became_active":"2024-04-20T11:36:09.501972+0000","last_became_peered":"2024-04-20T11:36:09.501972+0000","last_unstale":"2024-04-20T11:36:10.496549+0000","last_undegraded":"2024-04-20T11:36:10.496549+0000","last_fullsized":"2024-04-20T11:36:10.496549+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:08:37.699106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.7","version":"141'2","reported_seq":16,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.496818+0000","last_change":"2024-04-20T11:36:08.482021+0000","last_active":"2024-04-20T11:36:10.496818+0000","last_peered":"2024-04-20T11:36:10.496818+0000","last_clean":"2024-04-20T11:36:10.496818+0000","last_became_active":"2024-04-20T11:36:08.481134+0000","last_became_peered":"2024-04-20T11:36:08.481134+0000","last_unstale":"2024-04-20T11:36:10.496818+0000","last_undegraded":"2024-04-20T11:36:10.496818+0000","last_fullsized":"2024-04-20T11:36:10.496818+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:34:49.849356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.491796+0000","last_change":"2024-04-20T11:36:09.503319+0000","last_active":"2024-04-20T11:36:10.491796+0000","last_peered":"2024-04-20T11:36:10.491796+0000","last_clean":"2024-04-20T11:36:10.491796+0000","last_became_active":"2024-04-20T11:36:09.503019+0000","last_became_peered":"2024-04-20T11:36:09.503019+0000","last_unstale":"2024-04-20T11:36:10.491796+0000","last_undegraded":"2024-04-20T11:36:10.491796+0000","last_fullsized":"2024-04-20T11:36:10.491796+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:13:57.374934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.6","version":"142'1","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.505736+0000","last_change":"2024-04-20T11:36:08.471585+0000","last_active":"2024-04-20T11:36:11.505736+0000","last_peered":"2024-04-20T11:36:11.505736+0000","last_clean":"2024-04-20T11:36:11.505736+0000","last_became_active":"2024-04-20T11:36:08.471286+0000","last_became_peered":"2024-04-20T11:36:08.471286+0000","last_unstale":"2024-04-20T11:36:11.505736+0000","last_undegraded":"2024-04-20T11:36:11.505736+0000","last_fullsized":"2024-04-20T11:36:11.505736+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:04:24.667809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.807695+0000","last_change":"2024-04-20T11:36:09.504584+0000","last_active":"2024-04-20T11:36:11.807695+0000","last_peered":"2024-04-20T11:36:11.807695+0000","last_clean":"2024-04-20T11:36:11.807695+0000","last_became_active":"2024-04-20T11:36:09.502500+0000","last_became_peered":"2024-04-20T11:36:09.502500+0000","last_unstale":"2024-04-20T11:36:11.807695+0000","last_undegraded":"2024-04-20T11:36:11.807695+0000","last_fullsized":"2024-04-20T11:36:11.807695+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:42:53.512494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.674832+0000","last_change":"2024-04-20T11:36:08.483684+0000","last_active":"2024-04-20T11:36:10.674832+0000","last_peered":"2024-04-20T11:36:10.674832+0000","last_clean":"2024-04-20T11:36:10.674832+0000","last_became_active":"2024-04-20T11:36:08.483405+0000","last_became_peered":"2024-04-20T11:36:08.483405+0000","last_unstale":"2024-04-20T11:36:10.674832+0000","last_undegraded":"2024-04-20T11:36:10.674832+0000","last_fullsized":"2024-04-20T11:36:10.674832+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:08:24.707643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.496468+0000","last_change":"2024-04-20T11:36:09.503075+0000","last_active":"2024-04-20T11:36:10.496468+0000","last_peered":"2024-04-20T11:36:10.496468+0000","last_clean":"2024-04-20T11:36:10.496468+0000","last_became_active":"2024-04-20T11:36:09.502712+0000","last_became_peered":"2024-04-20T11:36:09.502712+0000","last_unstale":"2024-04-20T11:36:10.496468+0000","last_undegraded":"2024-04-20T11:36:10.496468+0000","last_fullsized":"2024-04-20T11:36:10.496468+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:56:54.406631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.496193+0000","last_change":"2024-04-20T11:36:08.472571+0000","last_active":"2024-04-20T11:36:10.496193+0000","last_peered":"2024-04-20T11:36:10.496193+0000","last_clean":"2024-04-20T11:36:10.496193+0000","last_became_active":"2024-04-20T11:36:08.470365+0000","last_became_peered":"2024-04-20T11:36:08.470365+0000","last_unstale":"2024-04-20T11:36:10.496193+0000","last_undegraded":"2024-04-20T11:36:10.496193+0000","last_fullsized":"2024-04-20T11:36:10.496193+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:23:09.673966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.496372+0000","last_change":"2024-04-20T11:36:09.496179+0000","last_active":"2024-04-20T11:36:10.496372+0000","last_peered":"2024-04-20T11:36:10.496372+0000","last_clean":"2024-04-20T11:36:10.496372+0000","last_became_active":"2024-04-20T11:36:09.495921+0000","last_became_peered":"2024-04-20T11:36:09.495921+0000","last_unstale":"2024-04-20T11:36:10.496372+0000","last_undegraded":"2024-04-20T11:36:10.496372+0000","last_fullsized":"2024-04-20T11:36:10.496372+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:48:17.725873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.513030+0000","last_change":"2024-04-20T11:36:08.471003+0000","last_active":"2024-04-20T11:36:11.513030+0000","last_peered":"2024-04-20T11:36:11.513030+0000","last_clean":"2024-04-20T11:36:11.513030+0000","last_became_active":"2024-04-20T11:36:08.470689+0000","last_became_peered":"2024-04-20T11:36:08.470689+0000","last_unstale":"2024-04-20T11:36:11.513030+0000","last_undegraded":"2024-04-20T11:36:11.513030+0000","last_fullsized":"2024-04-20T11:36:11.513030+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:29:33.311708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.512894+0000","last_change":"2024-04-20T11:36:09.510186+0000","last_active":"2024-04-20T11:36:11.512894+0000","last_peered":"2024-04-20T11:36:11.512894+0000","last_clean":"2024-04-20T11:36:11.512894+0000","last_became_active":"2024-04-20T11:36:09.509860+0000","last_became_peered":"2024-04-20T11:36:09.509860+0000","last_unstale":"2024-04-20T11:36:11.512894+0000","last_undegraded":"2024-04-20T11:36:11.512894+0000","last_fullsized":"2024-04-20T11:36:11.512894+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:55:27.981008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.808267+0000","last_change":"2024-04-20T11:36:08.472251+0000","last_active":"2024-04-20T11:36:11.808267+0000","last_peered":"2024-04-20T11:36:11.808267+0000","last_clean":"2024-04-20T11:36:11.808267+0000","last_became_active":"2024-04-20T11:36:08.469968+0000","last_became_peered":"2024-04-20T11:36:08.469968+0000","last_unstale":"2024-04-20T11:36:11.808267+0000","last_undegraded":"2024-04-20T11:36:11.808267+0000","last_fullsized":"2024-04-20T11:36:11.808267+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:47:08.022520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.491347+0000","last_change":"2024-04-20T11:36:09.506539+0000","last_active":"2024-04-20T11:36:10.491347+0000","last_peered":"2024-04-20T11:36:10.491347+0000","last_clean":"2024-04-20T11:36:10.491347+0000","last_became_active":"2024-04-20T11:36:09.505746+0000","last_became_peered":"2024-04-20T11:36:09.505746+0000","last_unstale":"2024-04-20T11:36:10.491347+0000","last_undegraded":"2024-04-20T11:36:10.491347+0000","last_fullsized":"2024-04-20T11:36:10.491347+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:56:54.406631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.674650+0000","last_change":"2024-04-20T11:36:08.475717+0000","last_active":"2024-04-20T11:36:10.674650+0000","last_peered":"2024-04-20T11:36:10.674650+0000","last_clean":"2024-04-20T11:36:10.674650+0000","last_became_active":"2024-04-20T11:36:08.475397+0000","last_became_peered":"2024-04-20T11:36:08.475397+0000","last_unstale":"2024-04-20T11:36:10.674650+0000","last_undegraded":"2024-04-20T11:36:10.674650+0000","last_fullsized":"2024-04-20T11:36:10.674650+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:52:40.058619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.504842+0000","last_change":"2024-04-20T11:36:09.504191+0000","last_active":"2024-04-20T11:36:11.504842+0000","last_peered":"2024-04-20T11:36:11.504842+0000","last_clean":"2024-04-20T11:36:11.504842+0000","last_became_active":"2024-04-20T11:36:09.500083+0000","last_became_peered":"2024-04-20T11:36:09.500083+0000","last_unstale":"2024-04-20T11:36:11.504842+0000","last_undegraded":"2024-04-20T11:36:11.504842+0000","last_fullsized":"2024-04-20T11:36:11.504842+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:01:44.730919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.512190+0000","last_change":"2024-04-20T11:36:08.467047+0000","last_active":"2024-04-20T11:36:11.512190+0000","last_peered":"2024-04-20T11:36:11.512190+0000","last_clean":"2024-04-20T11:36:11.512190+0000","last_became_active":"2024-04-20T11:36:08.466681+0000","last_became_peered":"2024-04-20T11:36:08.466681+0000","last_unstale":"2024-04-20T11:36:11.512190+0000","last_undegraded":"2024-04-20T11:36:11.512190+0000","last_fullsized":"2024-04-20T11:36:11.512190+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:30:14.516111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.496672+0000","last_change":"2024-04-20T11:36:09.504230+0000","last_active":"2024-04-20T11:36:10.496672+0000","last_peered":"2024-04-20T11:36:10.496672+0000","last_clean":"2024-04-20T11:36:10.496672+0000","last_became_active":"2024-04-20T11:36:09.503909+0000","last_became_peered":"2024-04-20T11:36:09.503909+0000","last_unstale":"2024-04-20T11:36:10.496672+0000","last_undegraded":"2024-04-20T11:36:10.496672+0000","last_fullsized":"2024-04-20T11:36:10.496672+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:10:13.066071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.513532+0000","last_change":"2024-04-20T11:36:08.473393+0000","last_active":"2024-04-20T11:36:11.513532+0000","last_peered":"2024-04-20T11:36:11.513532+0000","last_clean":"2024-04-20T11:36:11.513532+0000","last_became_active":"2024-04-20T11:36:08.473104+0000","last_became_peered":"2024-04-20T11:36:08.473104+0000","last_unstale":"2024-04-20T11:36:11.513532+0000","last_undegraded":"2024-04-20T11:36:11.513532+0000","last_fullsized":"2024-04-20T11:36:11.513532+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:08:32.469884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"72'128","reported_seq":427,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.498104+0000","last_change":"2024-04-20T11:26:48.661897+0000","last_active":"2024-04-20T11:36:10.498104+0000","last_peered":"2024-04-20T11:36:10.498104+0000","last_clean":"2024-04-20T11:36:10.498104+0000","last_became_active":"2024-04-20T11:26:48.656535+0000","last_became_peered":"2024-04-20T11:26:48.656535+0000","last_unstale":"2024-04-20T11:36:10.498104+0000","last_undegraded":"2024-04-20T11:36:10.498104+0000","last_fullsized":"2024-04-20T11:36:10.498104+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:47.543004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:47.543004+0000","last_clean_scrub_stamp":"2024-04-20T11:26:47.543004+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:58:35.527105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":23,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":22,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574967,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33528,"kb_used_data":2348,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338312,"statfs":{"total":96636764160,"available":96602431488,"internally_reserved":0,"allocated":2404352,"data_stored":1382440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574964,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32932,"kb_used_data":1748,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338908,"statfs":{"total":96636764160,"available":96603041792,"internally_reserved":0,"allocated":1789952,"data_stored":789740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574966,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32932,"kb_used_data":1752,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338908,"statfs":{"total":96636764160,"available":96603041792,"internally_reserved":0,"allocated":1794048,"data_stored":790322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574964,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32940,"kb_used_data":1760,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338900,"statfs":{"total":96636764160,"available":96603033600,"internally_reserved":0,"allocated":1802240,"data_stored":791400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574965,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32928,"kb_used_data":1748,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338912,"statfs":{"total":96636764160,"available":96603045888,"internally_reserved":0,"allocated":1789952,"data_stored":789752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574966,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32928,"kb_used_data":1748,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338912,"statfs":{"total":96636764160,"available":96603045888,"internally_reserved":0,"allocated":1789952,"data_stored":789808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574964,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32928,"kb_used_data":1748,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338912,"statfs":{"total":96636764160,"available":96603045888,"internally_reserved":0,"allocated":1789952,"data_stored":789752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574965,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33512,"kb_used_data":2324,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338328,"statfs":{"total":96636764160,"available":96602447872,"internally_reserved":0,"allocated":2379776,"data_stored":1380086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T11:36:18.926 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T11:36:19.136 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.137+0000 7f345d600640 1 Processor -- start 2024-04-20T11:36:19.136 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.137+0000 7f345d600640 1 -- start start 2024-04-20T11:36:19.138 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.138+0000 7f345d600640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f3458160b90 0x7f345815d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:19.138 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.138+0000 7f345d600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3458071cd0 0x7f3458072160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:19.138 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.139+0000 7f345d600640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34580ce8c0 con 0x7f345815d1e0 2024-04-20T11:36:19.138 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.139+0000 7f345d600640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f34580726a0 con 0x7f3458160b90 2024-04-20T11:36:19.138 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.139+0000 7f345d600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f34581600b0 con 0x7f3458071cd0 2024-04-20T11:36:19.139 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.139+0000 7f3457e00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f345815d1e0 0x7f34580cdfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:44304/0 (socket says 172.21.15.152:44304) 2024-04-20T11:36:19.139 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.139+0000 7f3457e00640 1 -- 172.21.15.152:0/1692967121 learned_addr learned my addr 172.21.15.152:0/1692967121 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:36:19.140 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.139+0000 7f345cc00640 1 --2- 172.21.15.152:0/1692967121 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3458071cd0 0x7f3458072160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:19.140 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.139+0000 7f344f400640 1 --2- 172.21.15.152:0/1692967121 >> v2:172.21.15.175:3300/0 conn(0x7f3458160b90 0x7f345815d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:19.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.139+0000 7f3457400640 1 -- 172.21.15.152:0/1692967121 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1782139627 0 0) 0x7f34580ce8c0 con 0x7f345815d1e0 2024-04-20T11:36:19.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.139+0000 7f3457400640 1 -- 172.21.15.152:0/1692967121 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f343c003520 con 0x7f345815d1e0 2024-04-20T11:36:19.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.140+0000 7f3457400640 1 -- 172.21.15.152:0/1692967121 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1253320649 0 0) 0x7f343c003520 con 0x7f345815d1e0 2024-04-20T11:36:19.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.140+0000 7f3457400640 1 -- 172.21.15.152:0/1692967121 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3458071cd0 msgr2=0x7f3458072160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:19.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.140+0000 7f3457400640 1 --2- 172.21.15.152:0/1692967121 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3458071cd0 0x7f3458072160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:19.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.140+0000 7f3457400640 1 -- 172.21.15.152:0/1692967121 >> v2:172.21.15.175:3300/0 conn(0x7f3458160b90 msgr2=0x7f345815d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:19.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.140+0000 7f3457400640 1 --2- 172.21.15.152:0/1692967121 >> v2:172.21.15.175:3300/0 conn(0x7f3458160b90 0x7f345815d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:19.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.140+0000 7f3457400640 1 -- 172.21.15.152:0/1692967121 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3458160390 con 0x7f345815d1e0 2024-04-20T11:36:19.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.140+0000 7f3457400640 1 -- 172.21.15.152:0/1692967121 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f3448001670 con 0x7f345815d1e0 2024-04-20T11:36:19.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.140+0000 7f3457400640 1 -- 172.21.15.152:0/1692967121 >> v1:172.21.15.152:6789/0 conn(0x7f345815d1e0 legacy=0x7f34580cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:19.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.140+0000 7f3457400640 1 --2- 172.21.15.152:0/1692967121 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f343c0039b0 0x7f343c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:19.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.140+0000 7f3457400640 1 --2- 172.21.15.152:0/1692967121 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f343c004800 0x7f343c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:19.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.140+0000 7f3457400640 1 --2- 172.21.15.152:0/1692967121 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f343c0051c0 0x7f343c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:19.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.140+0000 7f3457400640 1 -- 172.21.15.152:0/1692967121 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f34580726a0 con 0x7f343c0039b0 2024-04-20T11:36:19.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.140+0000 7f3457400640 1 -- 172.21.15.152:0/1692967121 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f34581600b0 con 0x7f343c0051c0 2024-04-20T11:36:19.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.140+0000 7f3457400640 1 -- 172.21.15.152:0/1692967121 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f343c00bf30 con 0x7f343c004800 2024-04-20T11:36:19.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.140+0000 7f345cc00640 1 --2- 172.21.15.152:0/1692967121 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3458071cd0 0x7f3458072160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:36:19.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.140+0000 7f3457e00640 1 --2- 172.21.15.152:0/1692967121 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f343c0039b0 0x7f343c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:19.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.141+0000 7f3457e00640 1 -- 172.21.15.152:0/1692967121 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f343c004800 msgr2=0x7f343c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:19.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.141+0000 7f3457e00640 1 --2- 172.21.15.152:0/1692967121 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f343c004800 0x7f343c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:19.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.141+0000 7f3457e00640 1 -- 172.21.15.152:0/1692967121 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f343c0051c0 msgr2=0x7f343c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:19.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.141+0000 7f3457e00640 1 --2- 172.21.15.152:0/1692967121 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f343c0051c0 0x7f343c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:19.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.141+0000 7f3457e00640 1 -- 172.21.15.152:0/1692967121 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f343c003520 con 0x7f343c0039b0 2024-04-20T11:36:19.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.142+0000 7f3457e00640 1 --2- 172.21.15.152:0/1692967121 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f343c0039b0 0x7f343c003dd0 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7f344800c240 tx=0x7f343c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67c6b1922c4ebce5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:19.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.142+0000 7f3457400640 1 -- 172.21.15.152:0/1692967121 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3448007d10 con 0x7f343c0039b0 2024-04-20T11:36:19.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.142+0000 7f3457400640 1 -- 172.21.15.152:0/1692967121 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3448030070 con 0x7f343c0039b0 2024-04-20T11:36:19.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.142+0000 7f3457400640 1 -- 172.21.15.152:0/1692967121 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f344802cd60 con 0x7f343c0039b0 2024-04-20T11:36:19.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.143+0000 7f345d600640 1 -- 172.21.15.152:0/1692967121 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f343c0039b0 msgr2=0x7f343c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:19.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.143+0000 7f345d600640 1 --2- 172.21.15.152:0/1692967121 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f343c0039b0 0x7f343c003dd0 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7f344800c240 tx=0x7f343c005830 comp rx=0 tx=0).stop 2024-04-20T11:36:19.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.143+0000 7f345cc00640 1 -- 172.21.15.152:0/1692967121 reap_dead start 2024-04-20T11:36:19.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.143+0000 7f345d600640 1 -- 172.21.15.152:0/1692967121 shutdown_connections 2024-04-20T11:36:19.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.143+0000 7f345d600640 1 -- 172.21.15.152:0/1692967121 >> 172.21.15.152:0/1692967121 conn(0x7f3458067ae0 msgr2=0x7f345806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:19.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.143+0000 7f345d600640 1 -- 172.21.15.152:0/1692967121 shutdown_connections 2024-04-20T11:36:19.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.143+0000 7f345d600640 1 -- 172.21.15.152:0/1692967121 wait complete. 2024-04-20T11:36:19.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.145+0000 7f345d600640 1 Processor -- start 2024-04-20T11:36:19.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.145+0000 7f345d600640 1 -- start start 2024-04-20T11:36:19.146 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.146+0000 7f345d600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f343c0039b0 0x7f3458150ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:19.146 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.146+0000 7f345d600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f34580b4040 0x7f3458153cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:19.146 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.146+0000 7f345d600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3458154200 0x7f34580b2230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:19.146 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.146+0000 7f345d600640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f3458072160 con 0x7f34580b4040 2024-04-20T11:36:19.146 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.146+0000 7f345d600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f34580ceba0 con 0x7f343c0039b0 2024-04-20T11:36:19.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.146+0000 7f345d600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f34580722e0 con 0x7f3458154200 2024-04-20T11:36:19.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.146+0000 7f344f400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f34580b4040 0x7f3458153cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:19.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.146+0000 7f344f400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f34580b4040 0x7f3458153cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:40270/0 (socket says 172.21.15.152:40270) 2024-04-20T11:36:19.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.146+0000 7f344f400640 1 -- 172.21.15.152:0/200349609 learned_addr learned my addr 172.21.15.152:0/200349609 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:19.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.146+0000 7f3457e00640 1 --2- 172.21.15.152:0/200349609 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f343c0039b0 0x7f3458150ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:19.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.147+0000 7f345cc00640 1 --2- 172.21.15.152:0/200349609 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3458154200 0x7f34580b2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:19.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.147+0000 7f344f400640 1 -- 172.21.15.152:0/200349609 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3458154200 msgr2=0x7f34580b2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:19.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.147+0000 7f344f400640 1 --2- 172.21.15.152:0/200349609 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3458154200 0x7f34580b2230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:19.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.147+0000 7f344f400640 1 -- 172.21.15.152:0/200349609 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f343c0039b0 msgr2=0x7f3458150ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:19.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.147+0000 7f344f400640 1 --2- 172.21.15.152:0/200349609 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f343c0039b0 0x7f3458150ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:19.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.147+0000 7f344f400640 1 -- 172.21.15.152:0/200349609 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34580b2fa0 con 0x7f34580b4040 2024-04-20T11:36:19.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.147+0000 7f344f400640 1 --2- 172.21.15.152:0/200349609 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f34580b4040 0x7f3458153cc0 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7f344000e8c0 tx=0x7f344000ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:19.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.147+0000 7f3454c00640 1 -- 172.21.15.152:0/200349609 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3440017070 con 0x7f34580b4040 2024-04-20T11:36:19.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.147+0000 7f3454c00640 1 -- 172.21.15.152:0/200349609 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f344000c940 con 0x7f34580b4040 2024-04-20T11:36:19.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.147+0000 7f345d600640 1 -- 172.21.15.152:0/200349609 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34580b3280 con 0x7f34580b4040 2024-04-20T11:36:19.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.147+0000 7f3454c00640 1 -- 172.21.15.152:0/200349609 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f344000cd10 con 0x7f34580b4040 2024-04-20T11:36:19.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.147+0000 7f345d600640 1 -- 172.21.15.152:0/200349609 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34580b3840 con 0x7f34580b4040 2024-04-20T11:36:19.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.149+0000 7f345d600640 1 -- 172.21.15.152:0/200349609 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f345815bce0 con 0x7f34580b4040 2024-04-20T11:36:19.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.150+0000 7f3454c00640 1 -- 172.21.15.152:0/200349609 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 22) ==== 91601+0+0 (secure 0 0 0) 0x7f344001a070 con 0x7f34580b4040 2024-04-20T11:36:19.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.151+0000 7f3454c00640 1 --2- 172.21.15.152:0/200349609 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f343406cf50 0x7f343406f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:19.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.151+0000 7f3454c00640 1 -- 172.21.15.152:0/200349609 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5709+0+0 (secure 0 0 0) 0x7f344008c500 con 0x7f34580b4040 2024-04-20T11:36:19.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.151+0000 7f3457e00640 1 --2- 172.21.15.152:0/200349609 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f343406cf50 0x7f343406f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:19.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.153+0000 7f3457e00640 1 --2- 172.21.15.152:0/200349609 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f343406cf50 0x7f343406f410 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f344800c240 tx=0x7f3448009030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:19.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.155+0000 7f3454c00640 1 -- 172.21.15.152:0/200349609 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f345815bce0 con 0x7f34580b4040 2024-04-20T11:36:19.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.351+0000 7f345d600640 1 -- 172.21.15.152:0/200349609 --> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f34580cfa10 con 0x7f343406cf50 2024-04-20T11:36:19.369 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.369+0000 7f3454c00640 1 -- 172.21.15.152:0/200349609 <== mgr.4109 v2:172.21.15.175:6836/4021250441 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252373 (secure 0 0 0) 0x7f34480829f0 con 0x7f343406cf50 2024-04-20T11:36:19.372 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:36:19.374 INFO:teuthology.orchestra.run.smithi152.stderr:dumped all 2024-04-20T11:36:19.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.375+0000 7f345d600640 1 -- 172.21.15.152:0/200349609 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f343406cf50 msgr2=0x7f343406f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:19.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.375+0000 7f345d600640 1 --2- 172.21.15.152:0/200349609 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f343406cf50 0x7f343406f410 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f344800c240 tx=0x7f3448009030 comp rx=0 tx=0).stop 2024-04-20T11:36:19.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.375+0000 7f345d600640 1 -- 172.21.15.152:0/200349609 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f34580b4040 msgr2=0x7f3458153cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:19.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.375+0000 7f345d600640 1 --2- 172.21.15.152:0/200349609 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f34580b4040 0x7f3458153cc0 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7f344000e8c0 tx=0x7f344000ed80 comp rx=0 tx=0).stop 2024-04-20T11:36:19.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.375+0000 7f345d600640 1 -- 172.21.15.152:0/200349609 shutdown_connections 2024-04-20T11:36:19.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.375+0000 7f345d600640 1 --2- 172.21.15.152:0/200349609 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f343406cf50 0x7f343406f410 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:19.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.375+0000 7f345d600640 1 --2- 172.21.15.152:0/200349609 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3458154200 0x7f34580b2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:19.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.375+0000 7f345d600640 1 --2- 172.21.15.152:0/200349609 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f34580b4040 0x7f3458153cc0 unknown :-1 s=CLOSED pgs=1985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:19.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.375+0000 7f345d600640 1 --2- 172.21.15.152:0/200349609 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f343c0039b0 0x7f3458150ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:19.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.375+0000 7f345d600640 1 -- 172.21.15.152:0/200349609 >> 172.21.15.152:0/200349609 conn(0x7f3458067ae0 msgr2=0x7f34580af360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:19.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.375+0000 7f345d600640 1 -- 172.21.15.152:0/200349609 shutdown_connections 2024-04-20T11:36:19.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.375+0000 7f345d600640 1 -- 172.21.15.152:0/200349609 wait complete. 2024-04-20T11:36:19.394 INFO:teuthology.orchestra.run.smithi152.stdout:{"pg_ready":true,"pg_map":{"version":192,"stamp":"2024-04-20T11:36:18.683630+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":224,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":167,"ondisk_log_size":167,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":264628,"kb_used_data":15176,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754710092,"statfs":{"total":773094113280,"available":772823134208,"internally_reserved":0,"allocated":15540224,"data_stored":7503300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12746,"internal_metadata":255315510},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.214259"},"pg_stats":[{"pgid":"23.3f","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.498594+0000","last_change":"2024-04-20T11:36:09.498245+0000","last_active":"2024-04-20T11:36:10.498594+0000","last_peered":"2024-04-20T11:36:10.498594+0000","last_clean":"2024-04-20T11:36:10.498594+0000","last_became_active":"2024-04-20T11:36:09.497956+0000","last_became_peered":"2024-04-20T11:36:09.497956+0000","last_unstale":"2024-04-20T11:36:10.498594+0000","last_undegraded":"2024-04-20T11:36:10.498594+0000","last_fullsized":"2024-04-20T11:36:10.498594+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:06:21.617876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.3e","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.674949+0000","last_change":"2024-04-20T11:36:09.511171+0000","last_active":"2024-04-20T11:36:10.674949+0000","last_peered":"2024-04-20T11:36:10.674949+0000","last_clean":"2024-04-20T11:36:10.674949+0000","last_became_active":"2024-04-20T11:36:09.510715+0000","last_became_peered":"2024-04-20T11:36:09.510715+0000","last_unstale":"2024-04-20T11:36:10.674949+0000","last_undegraded":"2024-04-20T11:36:10.674949+0000","last_fullsized":"2024-04-20T11:36:10.674949+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:56:28.407018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3d","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.505591+0000","last_change":"2024-04-20T11:36:09.517211+0000","last_active":"2024-04-20T11:36:11.505591+0000","last_peered":"2024-04-20T11:36:11.505591+0000","last_clean":"2024-04-20T11:36:11.505591+0000","last_became_active":"2024-04-20T11:36:09.516886+0000","last_became_peered":"2024-04-20T11:36:09.516886+0000","last_unstale":"2024-04-20T11:36:11.505591+0000","last_undegraded":"2024-04-20T11:36:11.505591+0000","last_fullsized":"2024-04-20T11:36:11.505591+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:11:59.531721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.3c","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.808485+0000","last_change":"2024-04-20T11:36:09.491911+0000","last_active":"2024-04-20T11:36:11.808485+0000","last_peered":"2024-04-20T11:36:11.808485+0000","last_clean":"2024-04-20T11:36:11.808485+0000","last_became_active":"2024-04-20T11:36:09.491136+0000","last_became_peered":"2024-04-20T11:36:09.491136+0000","last_unstale":"2024-04-20T11:36:11.808485+0000","last_undegraded":"2024-04-20T11:36:11.808485+0000","last_fullsized":"2024-04-20T11:36:11.808485+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:41:28.109593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3b","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.674463+0000","last_change":"2024-04-20T11:36:09.515606+0000","last_active":"2024-04-20T11:36:10.674463+0000","last_peered":"2024-04-20T11:36:10.674463+0000","last_clean":"2024-04-20T11:36:10.674463+0000","last_became_active":"2024-04-20T11:36:09.515282+0000","last_became_peered":"2024-04-20T11:36:09.515282+0000","last_unstale":"2024-04-20T11:36:10.674463+0000","last_undegraded":"2024-04-20T11:36:10.674463+0000","last_fullsized":"2024-04-20T11:36:10.674463+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:49:43.328289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3a","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.807103+0000","last_change":"2024-04-20T11:36:09.505767+0000","last_active":"2024-04-20T11:36:11.807103+0000","last_peered":"2024-04-20T11:36:11.807103+0000","last_clean":"2024-04-20T11:36:11.807103+0000","last_became_active":"2024-04-20T11:36:09.502268+0000","last_became_peered":"2024-04-20T11:36:09.502268+0000","last_unstale":"2024-04-20T11:36:11.807103+0000","last_undegraded":"2024-04-20T11:36:11.807103+0000","last_fullsized":"2024-04-20T11:36:11.807103+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:40:06.723507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.39","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.492148+0000","last_change":"2024-04-20T11:36:09.510744+0000","last_active":"2024-04-20T11:36:10.492148+0000","last_peered":"2024-04-20T11:36:10.492148+0000","last_clean":"2024-04-20T11:36:10.492148+0000","last_became_active":"2024-04-20T11:36:09.510385+0000","last_became_peered":"2024-04-20T11:36:09.510385+0000","last_unstale":"2024-04-20T11:36:10.492148+0000","last_undegraded":"2024-04-20T11:36:10.492148+0000","last_fullsized":"2024-04-20T11:36:10.492148+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:44:53.347896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.38","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.675349+0000","last_change":"2024-04-20T11:36:09.509705+0000","last_active":"2024-04-20T11:36:10.675349+0000","last_peered":"2024-04-20T11:36:10.675349+0000","last_clean":"2024-04-20T11:36:10.675349+0000","last_became_active":"2024-04-20T11:36:09.509111+0000","last_became_peered":"2024-04-20T11:36:09.509111+0000","last_unstale":"2024-04-20T11:36:10.675349+0000","last_undegraded":"2024-04-20T11:36:10.675349+0000","last_fullsized":"2024-04-20T11:36:10.675349+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:30:15.532693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.37","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.675032+0000","last_change":"2024-04-20T11:36:09.507670+0000","last_active":"2024-04-20T11:36:10.675032+0000","last_peered":"2024-04-20T11:36:10.675032+0000","last_clean":"2024-04-20T11:36:10.675032+0000","last_became_active":"2024-04-20T11:36:09.507308+0000","last_became_peered":"2024-04-20T11:36:09.507308+0000","last_unstale":"2024-04-20T11:36:10.675032+0000","last_undegraded":"2024-04-20T11:36:10.675032+0000","last_fullsized":"2024-04-20T11:36:10.675032+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:49:27.093205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.36","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.675225+0000","last_change":"2024-04-20T11:36:09.510613+0000","last_active":"2024-04-20T11:36:10.675225+0000","last_peered":"2024-04-20T11:36:10.675225+0000","last_clean":"2024-04-20T11:36:10.675225+0000","last_became_active":"2024-04-20T11:36:09.510280+0000","last_became_peered":"2024-04-20T11:36:09.510280+0000","last_unstale":"2024-04-20T11:36:10.675225+0000","last_undegraded":"2024-04-20T11:36:10.675225+0000","last_fullsized":"2024-04-20T11:36:10.675225+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:41:28.109593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.35","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.674568+0000","last_change":"2024-04-20T11:36:09.516333+0000","last_active":"2024-04-20T11:36:10.674568+0000","last_peered":"2024-04-20T11:36:10.674568+0000","last_clean":"2024-04-20T11:36:10.674568+0000","last_became_active":"2024-04-20T11:36:09.516038+0000","last_became_peered":"2024-04-20T11:36:09.516038+0000","last_unstale":"2024-04-20T11:36:10.674568+0000","last_undegraded":"2024-04-20T11:36:10.674568+0000","last_fullsized":"2024-04-20T11:36:10.674568+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:59:19.149913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.34","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.675001+0000","last_change":"2024-04-20T11:36:09.503669+0000","last_active":"2024-04-20T11:36:10.675001+0000","last_peered":"2024-04-20T11:36:10.675001+0000","last_clean":"2024-04-20T11:36:10.675001+0000","last_became_active":"2024-04-20T11:36:09.503351+0000","last_became_peered":"2024-04-20T11:36:09.503351+0000","last_unstale":"2024-04-20T11:36:10.675001+0000","last_undegraded":"2024-04-20T11:36:10.675001+0000","last_fullsized":"2024-04-20T11:36:10.675001+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:03:17.439635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.33","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.505133+0000","last_change":"2024-04-20T11:36:09.500004+0000","last_active":"2024-04-20T11:36:11.505133+0000","last_peered":"2024-04-20T11:36:11.505133+0000","last_clean":"2024-04-20T11:36:11.505133+0000","last_became_active":"2024-04-20T11:36:09.497333+0000","last_became_peered":"2024-04-20T11:36:09.497333+0000","last_unstale":"2024-04-20T11:36:11.505133+0000","last_undegraded":"2024-04-20T11:36:11.505133+0000","last_fullsized":"2024-04-20T11:36:11.505133+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:55:31.041430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.32","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.506115+0000","last_change":"2024-04-20T11:36:09.513751+0000","last_active":"2024-04-20T11:36:11.506115+0000","last_peered":"2024-04-20T11:36:11.506115+0000","last_clean":"2024-04-20T11:36:11.506115+0000","last_became_active":"2024-04-20T11:36:09.512773+0000","last_became_peered":"2024-04-20T11:36:09.512773+0000","last_unstale":"2024-04-20T11:36:11.506115+0000","last_undegraded":"2024-04-20T11:36:11.506115+0000","last_fullsized":"2024-04-20T11:36:11.506115+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:42:50.753207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.31","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.497845+0000","last_change":"2024-04-20T11:36:09.494578+0000","last_active":"2024-04-20T11:36:10.497845+0000","last_peered":"2024-04-20T11:36:10.497845+0000","last_clean":"2024-04-20T11:36:10.497845+0000","last_became_active":"2024-04-20T11:36:09.494216+0000","last_became_peered":"2024-04-20T11:36:09.494216+0000","last_unstale":"2024-04-20T11:36:10.497845+0000","last_undegraded":"2024-04-20T11:36:10.497845+0000","last_fullsized":"2024-04-20T11:36:10.497845+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:08:37.699106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.30","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.491449+0000","last_change":"2024-04-20T11:36:09.506793+0000","last_active":"2024-04-20T11:36:10.491449+0000","last_peered":"2024-04-20T11:36:10.491449+0000","last_clean":"2024-04-20T11:36:10.491449+0000","last_became_active":"2024-04-20T11:36:09.506217+0000","last_became_peered":"2024-04-20T11:36:09.506217+0000","last_unstale":"2024-04-20T11:36:10.491449+0000","last_undegraded":"2024-04-20T11:36:10.491449+0000","last_fullsized":"2024-04-20T11:36:10.491449+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:08:37.699106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.2f","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.807341+0000","last_change":"2024-04-20T11:36:09.502941+0000","last_active":"2024-04-20T11:36:11.807341+0000","last_peered":"2024-04-20T11:36:11.807341+0000","last_clean":"2024-04-20T11:36:11.807341+0000","last_became_active":"2024-04-20T11:36:09.501204+0000","last_became_peered":"2024-04-20T11:36:09.501204+0000","last_unstale":"2024-04-20T11:36:11.807341+0000","last_undegraded":"2024-04-20T11:36:11.807341+0000","last_fullsized":"2024-04-20T11:36:11.807341+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:41:49.723887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2e","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.496340+0000","last_change":"2024-04-20T11:36:09.497994+0000","last_active":"2024-04-20T11:36:10.496340+0000","last_peered":"2024-04-20T11:36:10.496340+0000","last_clean":"2024-04-20T11:36:10.496340+0000","last_became_active":"2024-04-20T11:36:09.497691+0000","last_became_peered":"2024-04-20T11:36:09.497691+0000","last_unstale":"2024-04-20T11:36:10.496340+0000","last_undegraded":"2024-04-20T11:36:10.496340+0000","last_fullsized":"2024-04-20T11:36:10.496340+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:40:28.853425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2d","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.807963+0000","last_change":"2024-04-20T11:36:09.517462+0000","last_active":"2024-04-20T11:36:11.807963+0000","last_peered":"2024-04-20T11:36:11.807963+0000","last_clean":"2024-04-20T11:36:11.807963+0000","last_became_active":"2024-04-20T11:36:09.517192+0000","last_became_peered":"2024-04-20T11:36:09.517192+0000","last_unstale":"2024-04-20T11:36:11.807963+0000","last_undegraded":"2024-04-20T11:36:11.807963+0000","last_fullsized":"2024-04-20T11:36:11.807963+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:33:01.539999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2c","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.505035+0000","last_change":"2024-04-20T11:36:09.513981+0000","last_active":"2024-04-20T11:36:11.505035+0000","last_peered":"2024-04-20T11:36:11.505035+0000","last_clean":"2024-04-20T11:36:11.505035+0000","last_became_active":"2024-04-20T11:36:09.513653+0000","last_became_peered":"2024-04-20T11:36:09.513653+0000","last_unstale":"2024-04-20T11:36:11.505035+0000","last_undegraded":"2024-04-20T11:36:11.505035+0000","last_fullsized":"2024-04-20T11:36:11.505035+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:14:26.387558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.2b","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.807325+0000","last_change":"2024-04-20T11:36:09.513386+0000","last_active":"2024-04-20T11:36:11.807325+0000","last_peered":"2024-04-20T11:36:11.807325+0000","last_clean":"2024-04-20T11:36:11.807325+0000","last_became_active":"2024-04-20T11:36:09.512484+0000","last_became_peered":"2024-04-20T11:36:09.512484+0000","last_unstale":"2024-04-20T11:36:11.807325+0000","last_undegraded":"2024-04-20T11:36:11.807325+0000","last_fullsized":"2024-04-20T11:36:11.807325+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:49:44.982942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2a","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.675443+0000","last_change":"2024-04-20T11:36:09.501933+0000","last_active":"2024-04-20T11:36:10.675443+0000","last_peered":"2024-04-20T11:36:10.675443+0000","last_clean":"2024-04-20T11:36:10.675443+0000","last_became_active":"2024-04-20T11:36:09.501168+0000","last_became_peered":"2024-04-20T11:36:09.501168+0000","last_unstale":"2024-04-20T11:36:10.675443+0000","last_undegraded":"2024-04-20T11:36:10.675443+0000","last_fullsized":"2024-04-20T11:36:10.675443+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:10:11.720624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.29","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.806672+0000","last_change":"2024-04-20T11:36:09.501305+0000","last_active":"2024-04-20T11:36:11.806672+0000","last_peered":"2024-04-20T11:36:11.806672+0000","last_clean":"2024-04-20T11:36:11.806672+0000","last_became_active":"2024-04-20T11:36:09.500714+0000","last_became_peered":"2024-04-20T11:36:09.500714+0000","last_unstale":"2024-04-20T11:36:11.806672+0000","last_undegraded":"2024-04-20T11:36:11.806672+0000","last_fullsized":"2024-04-20T11:36:11.806672+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:55:38.517484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.28","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.674679+0000","last_change":"2024-04-20T11:36:09.503752+0000","last_active":"2024-04-20T11:36:10.674679+0000","last_peered":"2024-04-20T11:36:10.674679+0000","last_clean":"2024-04-20T11:36:10.674679+0000","last_became_active":"2024-04-20T11:36:09.501356+0000","last_became_peered":"2024-04-20T11:36:09.501356+0000","last_unstale":"2024-04-20T11:36:10.674679+0000","last_undegraded":"2024-04-20T11:36:10.674679+0000","last_fullsized":"2024-04-20T11:36:10.674679+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:52:33.986991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.27","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.505645+0000","last_change":"2024-04-20T11:36:09.508345+0000","last_active":"2024-04-20T11:36:11.505645+0000","last_peered":"2024-04-20T11:36:11.505645+0000","last_clean":"2024-04-20T11:36:11.505645+0000","last_became_active":"2024-04-20T11:36:09.508006+0000","last_became_peered":"2024-04-20T11:36:09.508006+0000","last_unstale":"2024-04-20T11:36:11.505645+0000","last_undegraded":"2024-04-20T11:36:11.505645+0000","last_fullsized":"2024-04-20T11:36:11.505645+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:34:42.652012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.26","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.491806+0000","last_change":"2024-04-20T11:36:09.502314+0000","last_active":"2024-04-20T11:36:10.491806+0000","last_peered":"2024-04-20T11:36:10.491806+0000","last_clean":"2024-04-20T11:36:10.491806+0000","last_became_active":"2024-04-20T11:36:09.502005+0000","last_became_peered":"2024-04-20T11:36:09.502005+0000","last_unstale":"2024-04-20T11:36:10.491806+0000","last_undegraded":"2024-04-20T11:36:10.491806+0000","last_fullsized":"2024-04-20T11:36:10.491806+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:10:13.066071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.25","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.506008+0000","last_change":"2024-04-20T11:36:09.498214+0000","last_active":"2024-04-20T11:36:11.506008+0000","last_peered":"2024-04-20T11:36:11.506008+0000","last_clean":"2024-04-20T11:36:11.506008+0000","last_became_active":"2024-04-20T11:36:09.497877+0000","last_became_peered":"2024-04-20T11:36:09.497877+0000","last_unstale":"2024-04-20T11:36:11.506008+0000","last_undegraded":"2024-04-20T11:36:11.506008+0000","last_fullsized":"2024-04-20T11:36:11.506008+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:20:16.280679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.24","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.808688+0000","last_change":"2024-04-20T11:36:09.502782+0000","last_active":"2024-04-20T11:36:11.808688+0000","last_peered":"2024-04-20T11:36:11.808688+0000","last_clean":"2024-04-20T11:36:11.808688+0000","last_became_active":"2024-04-20T11:36:09.502509+0000","last_became_peered":"2024-04-20T11:36:09.502509+0000","last_unstale":"2024-04-20T11:36:11.808688+0000","last_undegraded":"2024-04-20T11:36:11.808688+0000","last_fullsized":"2024-04-20T11:36:11.808688+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:30:15.532693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.23","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.505461+0000","last_change":"2024-04-20T11:36:09.500622+0000","last_active":"2024-04-20T11:36:11.505461+0000","last_peered":"2024-04-20T11:36:11.505461+0000","last_clean":"2024-04-20T11:36:11.505461+0000","last_became_active":"2024-04-20T11:36:09.500341+0000","last_became_peered":"2024-04-20T11:36:09.500341+0000","last_unstale":"2024-04-20T11:36:11.505461+0000","last_undegraded":"2024-04-20T11:36:11.505461+0000","last_fullsized":"2024-04-20T11:36:11.505461+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:36:10.575443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.22","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.512516+0000","last_change":"2024-04-20T11:36:09.502504+0000","last_active":"2024-04-20T11:36:11.512516+0000","last_peered":"2024-04-20T11:36:11.512516+0000","last_clean":"2024-04-20T11:36:11.512516+0000","last_became_active":"2024-04-20T11:36:09.502201+0000","last_became_peered":"2024-04-20T11:36:09.502201+0000","last_unstale":"2024-04-20T11:36:11.512516+0000","last_undegraded":"2024-04-20T11:36:11.512516+0000","last_fullsized":"2024-04-20T11:36:11.512516+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:42:28.478688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.21","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.512231+0000","last_change":"2024-04-20T11:36:09.519295+0000","last_active":"2024-04-20T11:36:11.512231+0000","last_peered":"2024-04-20T11:36:11.512231+0000","last_clean":"2024-04-20T11:36:11.512231+0000","last_became_active":"2024-04-20T11:36:09.518233+0000","last_became_peered":"2024-04-20T11:36:09.518233+0000","last_unstale":"2024-04-20T11:36:11.512231+0000","last_undegraded":"2024-04-20T11:36:11.512231+0000","last_fullsized":"2024-04-20T11:36:11.512231+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:34:50.865938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.20","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.495715+0000","last_change":"2024-04-20T11:36:09.495789+0000","last_active":"2024-04-20T11:36:10.495715+0000","last_peered":"2024-04-20T11:36:10.495715+0000","last_clean":"2024-04-20T11:36:10.495715+0000","last_became_active":"2024-04-20T11:36:09.495472+0000","last_became_peered":"2024-04-20T11:36:09.495472+0000","last_unstale":"2024-04-20T11:36:10.495715+0000","last_undegraded":"2024-04-20T11:36:10.495715+0000","last_fullsized":"2024-04-20T11:36:10.495715+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:34:42.831349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.492044+0000","last_change":"2024-04-20T11:36:09.509822+0000","last_active":"2024-04-20T11:36:10.492044+0000","last_peered":"2024-04-20T11:36:10.492044+0000","last_clean":"2024-04-20T11:36:10.492044+0000","last_became_active":"2024-04-20T11:36:09.509521+0000","last_became_peered":"2024-04-20T11:36:09.509521+0000","last_unstale":"2024-04-20T11:36:10.492044+0000","last_undegraded":"2024-04-20T11:36:10.492044+0000","last_fullsized":"2024-04-20T11:36:10.492044+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:44:16.352910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1f","version":"141'4","reported_seq":18,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.498511+0000","last_change":"2024-04-20T11:36:08.483872+0000","last_active":"2024-04-20T11:36:10.498511+0000","last_peered":"2024-04-20T11:36:10.498511+0000","last_clean":"2024-04-20T11:36:10.498511+0000","last_became_active":"2024-04-20T11:36:08.483196+0000","last_became_peered":"2024-04-20T11:36:08.483196+0000","last_unstale":"2024-04-20T11:36:10.498511+0000","last_undegraded":"2024-04-20T11:36:10.498511+0000","last_fullsized":"2024-04-20T11:36:10.498511+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:37:38.832794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.513632+0000","last_change":"2024-04-20T11:36:09.505474+0000","last_active":"2024-04-20T11:36:11.513632+0000","last_peered":"2024-04-20T11:36:11.513632+0000","last_clean":"2024-04-20T11:36:11.513632+0000","last_became_active":"2024-04-20T11:36:09.502465+0000","last_became_peered":"2024-04-20T11:36:09.502465+0000","last_unstale":"2024-04-20T11:36:11.513632+0000","last_undegraded":"2024-04-20T11:36:11.513632+0000","last_fullsized":"2024-04-20T11:36:11.513632+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:51:23.507394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1e","version":"141'1","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.512968+0000","last_change":"2024-04-20T11:36:08.478064+0000","last_active":"2024-04-20T11:36:11.512968+0000","last_peered":"2024-04-20T11:36:11.512968+0000","last_clean":"2024-04-20T11:36:11.512968+0000","last_became_active":"2024-04-20T11:36:08.477788+0000","last_became_peered":"2024-04-20T11:36:08.477788+0000","last_unstale":"2024-04-20T11:36:11.512968+0000","last_undegraded":"2024-04-20T11:36:11.512968+0000","last_fullsized":"2024-04-20T11:36:11.512968+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:10:10.704042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.808302+0000","last_change":"2024-04-20T11:36:09.506372+0000","last_active":"2024-04-20T11:36:11.808302+0000","last_peered":"2024-04-20T11:36:11.808302+0000","last_clean":"2024-04-20T11:36:11.808302+0000","last_became_active":"2024-04-20T11:36:09.506107+0000","last_became_peered":"2024-04-20T11:36:09.506107+0000","last_unstale":"2024-04-20T11:36:11.808302+0000","last_undegraded":"2024-04-20T11:36:11.808302+0000","last_fullsized":"2024-04-20T11:36:11.808302+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:21:08.136339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1d","version":"141'2","reported_seq":16,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.808049+0000","last_change":"2024-04-20T11:36:08.479424+0000","last_active":"2024-04-20T11:36:11.808049+0000","last_peered":"2024-04-20T11:36:11.808049+0000","last_clean":"2024-04-20T11:36:11.808049+0000","last_became_active":"2024-04-20T11:36:08.479137+0000","last_became_peered":"2024-04-20T11:36:08.479137+0000","last_unstale":"2024-04-20T11:36:11.808049+0000","last_undegraded":"2024-04-20T11:36:11.808049+0000","last_fullsized":"2024-04-20T11:36:11.808049+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:25:33.273088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.506044+0000","last_change":"2024-04-20T11:36:09.498658+0000","last_active":"2024-04-20T11:36:11.506044+0000","last_peered":"2024-04-20T11:36:11.506044+0000","last_clean":"2024-04-20T11:36:11.506044+0000","last_became_active":"2024-04-20T11:36:09.498372+0000","last_became_peered":"2024-04-20T11:36:09.498372+0000","last_unstale":"2024-04-20T11:36:11.506044+0000","last_undegraded":"2024-04-20T11:36:11.506044+0000","last_fullsized":"2024-04-20T11:36:11.506044+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:14:54.539911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1c","version":"141'2","reported_seq":16,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.808844+0000","last_change":"2024-04-20T11:36:08.481345+0000","last_active":"2024-04-20T11:36:11.808844+0000","last_peered":"2024-04-20T11:36:11.808844+0000","last_clean":"2024-04-20T11:36:11.808844+0000","last_became_active":"2024-04-20T11:36:08.480807+0000","last_became_peered":"2024-04-20T11:36:08.480807+0000","last_unstale":"2024-04-20T11:36:11.808844+0000","last_undegraded":"2024-04-20T11:36:11.808844+0000","last_fullsized":"2024-04-20T11:36:11.808844+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:53:50.523715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.491980+0000","last_change":"2024-04-20T11:36:09.498794+0000","last_active":"2024-04-20T11:36:10.491980+0000","last_peered":"2024-04-20T11:36:10.491980+0000","last_clean":"2024-04-20T11:36:10.491980+0000","last_became_active":"2024-04-20T11:36:09.498063+0000","last_became_peered":"2024-04-20T11:36:09.498063+0000","last_unstale":"2024-04-20T11:36:10.491980+0000","last_undegraded":"2024-04-20T11:36:10.491980+0000","last_fullsized":"2024-04-20T11:36:10.491980+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:00:28.716804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1b","version":"141'1","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.505504+0000","last_change":"2024-04-20T11:36:08.473692+0000","last_active":"2024-04-20T11:36:11.505504+0000","last_peered":"2024-04-20T11:36:11.505504+0000","last_clean":"2024-04-20T11:36:11.505504+0000","last_became_active":"2024-04-20T11:36:08.473388+0000","last_became_peered":"2024-04-20T11:36:08.473388+0000","last_unstale":"2024-04-20T11:36:11.505504+0000","last_undegraded":"2024-04-20T11:36:11.505504+0000","last_fullsized":"2024-04-20T11:36:11.505504+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:09:00.631652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.807706+0000","last_change":"2024-04-20T11:36:09.496254+0000","last_active":"2024-04-20T11:36:11.807706+0000","last_peered":"2024-04-20T11:36:11.807706+0000","last_clean":"2024-04-20T11:36:11.807706+0000","last_became_active":"2024-04-20T11:36:09.492502+0000","last_became_peered":"2024-04-20T11:36:09.492502+0000","last_unstale":"2024-04-20T11:36:11.807706+0000","last_undegraded":"2024-04-20T11:36:11.807706+0000","last_fullsized":"2024-04-20T11:36:11.807706+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:49:27.093205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.1a","version":"141'1","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.808305+0000","last_change":"2024-04-20T11:36:08.472809+0000","last_active":"2024-04-20T11:36:11.808305+0000","last_peered":"2024-04-20T11:36:11.808305+0000","last_clean":"2024-04-20T11:36:11.808305+0000","last_became_active":"2024-04-20T11:36:08.472197+0000","last_became_peered":"2024-04-20T11:36:08.472197+0000","last_unstale":"2024-04-20T11:36:11.808305+0000","last_undegraded":"2024-04-20T11:36:11.808305+0000","last_fullsized":"2024-04-20T11:36:11.808305+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:08:24.707643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.807054+0000","last_change":"2024-04-20T11:36:09.483939+0000","last_active":"2024-04-20T11:36:11.807054+0000","last_peered":"2024-04-20T11:36:11.807054+0000","last_clean":"2024-04-20T11:36:11.807054+0000","last_became_active":"2024-04-20T11:36:09.483166+0000","last_became_peered":"2024-04-20T11:36:09.483166+0000","last_unstale":"2024-04-20T11:36:11.807054+0000","last_undegraded":"2024-04-20T11:36:11.807054+0000","last_fullsized":"2024-04-20T11:36:11.807054+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:03:17.439635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.674387+0000","last_change":"2024-04-20T11:36:08.481523+0000","last_active":"2024-04-20T11:36:10.674387+0000","last_peered":"2024-04-20T11:36:10.674387+0000","last_clean":"2024-04-20T11:36:10.674387+0000","last_became_active":"2024-04-20T11:36:08.481220+0000","last_became_peered":"2024-04-20T11:36:08.481220+0000","last_unstale":"2024-04-20T11:36:10.674387+0000","last_undegraded":"2024-04-20T11:36:10.674387+0000","last_fullsized":"2024-04-20T11:36:10.674387+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:27:21.500530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.806610+0000","last_change":"2024-04-20T11:36:09.499280+0000","last_active":"2024-04-20T11:36:11.806610+0000","last_peered":"2024-04-20T11:36:11.806610+0000","last_clean":"2024-04-20T11:36:11.806610+0000","last_became_active":"2024-04-20T11:36:09.499008+0000","last_became_peered":"2024-04-20T11:36:09.499008+0000","last_unstale":"2024-04-20T11:36:11.806610+0000","last_undegraded":"2024-04-20T11:36:11.806610+0000","last_fullsized":"2024-04-20T11:36:11.806610+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:35:54.753258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.491574+0000","last_change":"2024-04-20T11:36:08.459181+0000","last_active":"2024-04-20T11:36:10.491574+0000","last_peered":"2024-04-20T11:36:10.491574+0000","last_clean":"2024-04-20T11:36:10.491574+0000","last_became_active":"2024-04-20T11:36:08.455877+0000","last_became_peered":"2024-04-20T11:36:08.455877+0000","last_unstale":"2024-04-20T11:36:10.491574+0000","last_undegraded":"2024-04-20T11:36:10.491574+0000","last_fullsized":"2024-04-20T11:36:10.491574+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:03:08.918074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.496622+0000","last_change":"2024-04-20T11:36:09.498684+0000","last_active":"2024-04-20T11:36:10.496622+0000","last_peered":"2024-04-20T11:36:10.496622+0000","last_clean":"2024-04-20T11:36:10.496622+0000","last_became_active":"2024-04-20T11:36:09.498116+0000","last_became_peered":"2024-04-20T11:36:09.498116+0000","last_unstale":"2024-04-20T11:36:10.496622+0000","last_undegraded":"2024-04-20T11:36:10.496622+0000","last_fullsized":"2024-04-20T11:36:10.496622+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:06:21.617876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.807834+0000","last_change":"2024-04-20T11:36:08.472135+0000","last_active":"2024-04-20T11:36:11.807834+0000","last_peered":"2024-04-20T11:36:11.807834+0000","last_clean":"2024-04-20T11:36:11.807834+0000","last_became_active":"2024-04-20T11:36:08.471529+0000","last_became_peered":"2024-04-20T11:36:08.471529+0000","last_unstale":"2024-04-20T11:36:11.807834+0000","last_undegraded":"2024-04-20T11:36:11.807834+0000","last_fullsized":"2024-04-20T11:36:11.807834+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:29:38.601496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.496388+0000","last_change":"2024-04-20T11:36:09.497525+0000","last_active":"2024-04-20T11:36:10.496388+0000","last_peered":"2024-04-20T11:36:10.496388+0000","last_clean":"2024-04-20T11:36:10.496388+0000","last_became_active":"2024-04-20T11:36:09.497185+0000","last_became_peered":"2024-04-20T11:36:09.497185+0000","last_unstale":"2024-04-20T11:36:10.496388+0000","last_undegraded":"2024-04-20T11:36:10.496388+0000","last_fullsized":"2024-04-20T11:36:10.496388+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:39:12.611985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.15","version":"141'3","reported_seq":17,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.807992+0000","last_change":"2024-04-20T11:36:08.474633+0000","last_active":"2024-04-20T11:36:11.807992+0000","last_peered":"2024-04-20T11:36:11.807992+0000","last_clean":"2024-04-20T11:36:11.807992+0000","last_became_active":"2024-04-20T11:36:08.474310+0000","last_became_peered":"2024-04-20T11:36:08.474310+0000","last_unstale":"2024-04-20T11:36:11.807992+0000","last_undegraded":"2024-04-20T11:36:11.807992+0000","last_fullsized":"2024-04-20T11:36:11.807992+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:43:13.653616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.15","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.513740+0000","last_change":"2024-04-20T11:36:09.519775+0000","last_active":"2024-04-20T11:36:11.513740+0000","last_peered":"2024-04-20T11:36:11.513740+0000","last_clean":"2024-04-20T11:36:11.513740+0000","last_became_active":"2024-04-20T11:36:09.516787+0000","last_became_peered":"2024-04-20T11:36:09.516787+0000","last_unstale":"2024-04-20T11:36:11.513740+0000","last_undegraded":"2024-04-20T11:36:11.513740+0000","last_fullsized":"2024-04-20T11:36:11.513740+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:53:29.084025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.14","version":"141'6","reported_seq":20,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.512759+0000","last_change":"2024-04-20T11:36:08.481226+0000","last_active":"2024-04-20T11:36:11.512759+0000","last_peered":"2024-04-20T11:36:11.512759+0000","last_clean":"2024-04-20T11:36:11.512759+0000","last_became_active":"2024-04-20T11:36:08.480392+0000","last_became_peered":"2024-04-20T11:36:08.480392+0000","last_unstale":"2024-04-20T11:36:11.512759+0000","last_undegraded":"2024-04-20T11:36:11.512759+0000","last_fullsized":"2024-04-20T11:36:11.512759+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:07:03.390165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.495849+0000","last_change":"2024-04-20T11:36:09.505061+0000","last_active":"2024-04-20T11:36:10.495849+0000","last_peered":"2024-04-20T11:36:10.495849+0000","last_clean":"2024-04-20T11:36:10.495849+0000","last_became_active":"2024-04-20T11:36:09.504620+0000","last_became_peered":"2024-04-20T11:36:09.504620+0000","last_unstale":"2024-04-20T11:36:10.495849+0000","last_undegraded":"2024-04-20T11:36:10.495849+0000","last_fullsized":"2024-04-20T11:36:10.495849+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:16:01.895857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.13","version":"141'2","reported_seq":16,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.807629+0000","last_change":"2024-04-20T11:36:08.481075+0000","last_active":"2024-04-20T11:36:11.807629+0000","last_peered":"2024-04-20T11:36:11.807629+0000","last_clean":"2024-04-20T11:36:11.807629+0000","last_became_active":"2024-04-20T11:36:08.480423+0000","last_became_peered":"2024-04-20T11:36:08.480423+0000","last_unstale":"2024-04-20T11:36:11.807629+0000","last_undegraded":"2024-04-20T11:36:11.807629+0000","last_fullsized":"2024-04-20T11:36:11.807629+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:43:49.370831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.492241+0000","last_change":"2024-04-20T11:36:09.503953+0000","last_active":"2024-04-20T11:36:10.492241+0000","last_peered":"2024-04-20T11:36:10.492241+0000","last_clean":"2024-04-20T11:36:10.492241+0000","last_became_active":"2024-04-20T11:36:09.503682+0000","last_became_peered":"2024-04-20T11:36:09.503682+0000","last_unstale":"2024-04-20T11:36:10.492241+0000","last_undegraded":"2024-04-20T11:36:10.492241+0000","last_fullsized":"2024-04-20T11:36:10.492241+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:32:53.769485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.496025+0000","last_change":"2024-04-20T11:36:08.469986+0000","last_active":"2024-04-20T11:36:10.496025+0000","last_peered":"2024-04-20T11:36:10.496025+0000","last_clean":"2024-04-20T11:36:10.496025+0000","last_became_active":"2024-04-20T11:36:08.469638+0000","last_became_peered":"2024-04-20T11:36:08.469638+0000","last_unstale":"2024-04-20T11:36:10.496025+0000","last_undegraded":"2024-04-20T11:36:10.496025+0000","last_fullsized":"2024-04-20T11:36:10.496025+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:51:13.378758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.807021+0000","last_change":"2024-04-20T11:36:09.503606+0000","last_active":"2024-04-20T11:36:11.807021+0000","last_peered":"2024-04-20T11:36:11.807021+0000","last_clean":"2024-04-20T11:36:11.807021+0000","last_became_active":"2024-04-20T11:36:09.503261+0000","last_became_peered":"2024-04-20T11:36:09.503261+0000","last_unstale":"2024-04-20T11:36:11.807021+0000","last_undegraded":"2024-04-20T11:36:11.807021+0000","last_fullsized":"2024-04-20T11:36:11.807021+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:56:28.407018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.807199+0000","last_change":"2024-04-20T11:36:08.477055+0000","last_active":"2024-04-20T11:36:11.807199+0000","last_peered":"2024-04-20T11:36:11.807199+0000","last_clean":"2024-04-20T11:36:11.807199+0000","last_became_active":"2024-04-20T11:36:08.476730+0000","last_became_peered":"2024-04-20T11:36:08.476730+0000","last_unstale":"2024-04-20T11:36:11.807199+0000","last_undegraded":"2024-04-20T11:36:11.807199+0000","last_fullsized":"2024-04-20T11:36:11.807199+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:00:31.304085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.504738+0000","last_change":"2024-04-20T11:36:09.504805+0000","last_active":"2024-04-20T11:36:11.504738+0000","last_peered":"2024-04-20T11:36:11.504738+0000","last_clean":"2024-04-20T11:36:11.504738+0000","last_became_active":"2024-04-20T11:36:09.504541+0000","last_became_peered":"2024-04-20T11:36:09.504541+0000","last_unstale":"2024-04-20T11:36:11.504738+0000","last_undegraded":"2024-04-20T11:36:11.504738+0000","last_fullsized":"2024-04-20T11:36:11.504738+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:03:06.526803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.10","version":"141'8","reported_seq":22,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.511978+0000","last_change":"2024-04-20T11:36:08.476771+0000","last_active":"2024-04-20T11:36:11.511978+0000","last_peered":"2024-04-20T11:36:11.511978+0000","last_clean":"2024-04-20T11:36:11.511978+0000","last_became_active":"2024-04-20T11:36:08.476124+0000","last_became_peered":"2024-04-20T11:36:08.476124+0000","last_unstale":"2024-04-20T11:36:11.511978+0000","last_undegraded":"2024-04-20T11:36:11.511978+0000","last_fullsized":"2024-04-20T11:36:11.511978+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:41:27.093010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.809290+0000","last_change":"2024-04-20T11:36:09.500469+0000","last_active":"2024-04-20T11:36:11.809290+0000","last_peered":"2024-04-20T11:36:11.809290+0000","last_clean":"2024-04-20T11:36:11.809290+0000","last_became_active":"2024-04-20T11:36:09.495917+0000","last_became_peered":"2024-04-20T11:36:09.495917+0000","last_unstale":"2024-04-20T11:36:11.809290+0000","last_undegraded":"2024-04-20T11:36:11.809290+0000","last_fullsized":"2024-04-20T11:36:11.809290+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:07:04.406747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"141'1","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.505732+0000","last_change":"2024-04-20T11:36:08.479096+0000","last_active":"2024-04-20T11:36:11.505732+0000","last_peered":"2024-04-20T11:36:11.505732+0000","last_clean":"2024-04-20T11:36:11.505732+0000","last_became_active":"2024-04-20T11:36:08.478807+0000","last_became_peered":"2024-04-20T11:36:08.478807+0000","last_unstale":"2024-04-20T11:36:11.505732+0000","last_undegraded":"2024-04-20T11:36:11.505732+0000","last_fullsized":"2024-04-20T11:36:11.505732+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:25:58.656867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.496824+0000","last_change":"2024-04-20T11:36:09.510407+0000","last_active":"2024-04-20T11:36:10.496824+0000","last_peered":"2024-04-20T11:36:10.496824+0000","last_clean":"2024-04-20T11:36:10.496824+0000","last_became_active":"2024-04-20T11:36:09.510132+0000","last_became_peered":"2024-04-20T11:36:09.510132+0000","last_unstale":"2024-04-20T11:36:10.496824+0000","last_undegraded":"2024-04-20T11:36:10.496824+0000","last_fullsized":"2024-04-20T11:36:10.496824+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:13:57.374934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.e","version":"141'2","reported_seq":16,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.505227+0000","last_change":"2024-04-20T11:36:08.475925+0000","last_active":"2024-04-20T11:36:11.505227+0000","last_peered":"2024-04-20T11:36:11.505227+0000","last_clean":"2024-04-20T11:36:11.505227+0000","last_became_active":"2024-04-20T11:36:08.474739+0000","last_became_peered":"2024-04-20T11:36:08.474739+0000","last_unstale":"2024-04-20T11:36:11.505227+0000","last_undegraded":"2024-04-20T11:36:11.505227+0000","last_fullsized":"2024-04-20T11:36:11.505227+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:52:49.350436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.808772+0000","last_change":"2024-04-20T11:36:09.497659+0000","last_active":"2024-04-20T11:36:11.808772+0000","last_peered":"2024-04-20T11:36:11.808772+0000","last_clean":"2024-04-20T11:36:11.808772+0000","last_became_active":"2024-04-20T11:36:09.495859+0000","last_became_peered":"2024-04-20T11:36:09.495859+0000","last_unstale":"2024-04-20T11:36:11.808772+0000","last_undegraded":"2024-04-20T11:36:11.808772+0000","last_fullsized":"2024-04-20T11:36:11.808772+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:10:11.720624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.d","version":"141'1","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.498977+0000","last_change":"2024-04-20T11:36:08.484073+0000","last_active":"2024-04-20T11:36:10.498977+0000","last_peered":"2024-04-20T11:36:10.498977+0000","last_clean":"2024-04-20T11:36:10.498977+0000","last_became_active":"2024-04-20T11:36:08.483348+0000","last_became_peered":"2024-04-20T11:36:08.483348+0000","last_unstale":"2024-04-20T11:36:10.498977+0000","last_undegraded":"2024-04-20T11:36:10.498977+0000","last_fullsized":"2024-04-20T11:36:10.498977+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:58:31.453250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.491566+0000","last_change":"2024-04-20T11:36:09.515509+0000","last_active":"2024-04-20T11:36:10.491566+0000","last_peered":"2024-04-20T11:36:10.491566+0000","last_clean":"2024-04-20T11:36:10.491566+0000","last_became_active":"2024-04-20T11:36:09.515199+0000","last_became_peered":"2024-04-20T11:36:09.515199+0000","last_unstale":"2024-04-20T11:36:10.491566+0000","last_undegraded":"2024-04-20T11:36:10.491566+0000","last_fullsized":"2024-04-20T11:36:10.491566+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:06:21.617876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.498695+0000","last_change":"2024-04-20T11:36:08.481420+0000","last_active":"2024-04-20T11:36:10.498695+0000","last_peered":"2024-04-20T11:36:10.498695+0000","last_clean":"2024-04-20T11:36:10.498695+0000","last_became_active":"2024-04-20T11:36:08.481118+0000","last_became_peered":"2024-04-20T11:36:08.481118+0000","last_unstale":"2024-04-20T11:36:10.498695+0000","last_undegraded":"2024-04-20T11:36:10.498695+0000","last_fullsized":"2024-04-20T11:36:10.498695+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:27:32.804957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.504806+0000","last_change":"2024-04-20T11:36:09.504033+0000","last_active":"2024-04-20T11:36:11.504806+0000","last_peered":"2024-04-20T11:36:11.504806+0000","last_clean":"2024-04-20T11:36:11.504806+0000","last_became_active":"2024-04-20T11:36:09.503676+0000","last_became_peered":"2024-04-20T11:36:09.503676+0000","last_unstale":"2024-04-20T11:36:11.504806+0000","last_undegraded":"2024-04-20T11:36:11.504806+0000","last_fullsized":"2024-04-20T11:36:11.504806+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:56:53.061204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.b","version":"141'2","reported_seq":16,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.807844+0000","last_change":"2024-04-20T11:36:08.479935+0000","last_active":"2024-04-20T11:36:11.807844+0000","last_peered":"2024-04-20T11:36:11.807844+0000","last_clean":"2024-04-20T11:36:11.807844+0000","last_became_active":"2024-04-20T11:36:08.479361+0000","last_became_peered":"2024-04-20T11:36:08.479361+0000","last_unstale":"2024-04-20T11:36:11.807844+0000","last_undegraded":"2024-04-20T11:36:11.807844+0000","last_fullsized":"2024-04-20T11:36:11.807844+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:36:59.262204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.492424+0000","last_change":"2024-04-20T11:36:09.508916+0000","last_active":"2024-04-20T11:36:10.492424+0000","last_peered":"2024-04-20T11:36:10.492424+0000","last_clean":"2024-04-20T11:36:10.492424+0000","last_became_active":"2024-04-20T11:36:09.506508+0000","last_became_peered":"2024-04-20T11:36:09.506508+0000","last_unstale":"2024-04-20T11:36:10.492424+0000","last_undegraded":"2024-04-20T11:36:10.492424+0000","last_fullsized":"2024-04-20T11:36:10.492424+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:37:42.247415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.498687+0000","last_change":"2024-04-20T11:36:08.480015+0000","last_active":"2024-04-20T11:36:10.498687+0000","last_peered":"2024-04-20T11:36:10.498687+0000","last_clean":"2024-04-20T11:36:10.498687+0000","last_became_active":"2024-04-20T11:36:08.479699+0000","last_became_peered":"2024-04-20T11:36:08.479699+0000","last_unstale":"2024-04-20T11:36:10.498687+0000","last_undegraded":"2024-04-20T11:36:10.498687+0000","last_fullsized":"2024-04-20T11:36:10.498687+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:54:19.912464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.497824+0000","last_change":"2024-04-20T11:36:09.503816+0000","last_active":"2024-04-20T11:36:10.497824+0000","last_peered":"2024-04-20T11:36:10.497824+0000","last_clean":"2024-04-20T11:36:10.497824+0000","last_became_active":"2024-04-20T11:36:09.503166+0000","last_became_peered":"2024-04-20T11:36:09.503166+0000","last_unstale":"2024-04-20T11:36:10.497824+0000","last_undegraded":"2024-04-20T11:36:10.497824+0000","last_fullsized":"2024-04-20T11:36:10.497824+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:56:54.406631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.807386+0000","last_change":"2024-04-20T11:36:08.477340+0000","last_active":"2024-04-20T11:36:11.807386+0000","last_peered":"2024-04-20T11:36:11.807386+0000","last_clean":"2024-04-20T11:36:11.807386+0000","last_became_active":"2024-04-20T11:36:08.476853+0000","last_became_peered":"2024-04-20T11:36:08.476853+0000","last_unstale":"2024-04-20T11:36:11.807386+0000","last_undegraded":"2024-04-20T11:36:11.807386+0000","last_fullsized":"2024-04-20T11:36:11.807386+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:52:40.058619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.806778+0000","last_change":"2024-04-20T11:36:09.504017+0000","last_active":"2024-04-20T11:36:11.806778+0000","last_peered":"2024-04-20T11:36:11.806778+0000","last_clean":"2024-04-20T11:36:11.806778+0000","last_became_active":"2024-04-20T11:36:09.503623+0000","last_became_peered":"2024-04-20T11:36:09.503623+0000","last_unstale":"2024-04-20T11:36:11.806778+0000","last_undegraded":"2024-04-20T11:36:11.806778+0000","last_fullsized":"2024-04-20T11:36:11.806778+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:42:34.318459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.497950+0000","last_change":"2024-04-20T11:36:08.481304+0000","last_active":"2024-04-20T11:36:10.497950+0000","last_peered":"2024-04-20T11:36:10.497950+0000","last_clean":"2024-04-20T11:36:10.497950+0000","last_became_active":"2024-04-20T11:36:08.480957+0000","last_became_peered":"2024-04-20T11:36:08.480957+0000","last_unstale":"2024-04-20T11:36:10.497950+0000","last_undegraded":"2024-04-20T11:36:10.497950+0000","last_fullsized":"2024-04-20T11:36:10.497950+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:34:41.814767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.496549+0000","last_change":"2024-04-20T11:36:09.507310+0000","last_active":"2024-04-20T11:36:10.496549+0000","last_peered":"2024-04-20T11:36:10.496549+0000","last_clean":"2024-04-20T11:36:10.496549+0000","last_became_active":"2024-04-20T11:36:09.501972+0000","last_became_peered":"2024-04-20T11:36:09.501972+0000","last_unstale":"2024-04-20T11:36:10.496549+0000","last_undegraded":"2024-04-20T11:36:10.496549+0000","last_fullsized":"2024-04-20T11:36:10.496549+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:08:37.699106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.7","version":"141'2","reported_seq":16,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.496818+0000","last_change":"2024-04-20T11:36:08.482021+0000","last_active":"2024-04-20T11:36:10.496818+0000","last_peered":"2024-04-20T11:36:10.496818+0000","last_clean":"2024-04-20T11:36:10.496818+0000","last_became_active":"2024-04-20T11:36:08.481134+0000","last_became_peered":"2024-04-20T11:36:08.481134+0000","last_unstale":"2024-04-20T11:36:10.496818+0000","last_undegraded":"2024-04-20T11:36:10.496818+0000","last_fullsized":"2024-04-20T11:36:10.496818+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:34:49.849356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.491796+0000","last_change":"2024-04-20T11:36:09.503319+0000","last_active":"2024-04-20T11:36:10.491796+0000","last_peered":"2024-04-20T11:36:10.491796+0000","last_clean":"2024-04-20T11:36:10.491796+0000","last_became_active":"2024-04-20T11:36:09.503019+0000","last_became_peered":"2024-04-20T11:36:09.503019+0000","last_unstale":"2024-04-20T11:36:10.491796+0000","last_undegraded":"2024-04-20T11:36:10.491796+0000","last_fullsized":"2024-04-20T11:36:10.491796+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:13:57.374934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.6","version":"142'1","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.505736+0000","last_change":"2024-04-20T11:36:08.471585+0000","last_active":"2024-04-20T11:36:11.505736+0000","last_peered":"2024-04-20T11:36:11.505736+0000","last_clean":"2024-04-20T11:36:11.505736+0000","last_became_active":"2024-04-20T11:36:08.471286+0000","last_became_peered":"2024-04-20T11:36:08.471286+0000","last_unstale":"2024-04-20T11:36:11.505736+0000","last_undegraded":"2024-04-20T11:36:11.505736+0000","last_fullsized":"2024-04-20T11:36:11.505736+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:04:24.667809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.807695+0000","last_change":"2024-04-20T11:36:09.504584+0000","last_active":"2024-04-20T11:36:11.807695+0000","last_peered":"2024-04-20T11:36:11.807695+0000","last_clean":"2024-04-20T11:36:11.807695+0000","last_became_active":"2024-04-20T11:36:09.502500+0000","last_became_peered":"2024-04-20T11:36:09.502500+0000","last_unstale":"2024-04-20T11:36:11.807695+0000","last_undegraded":"2024-04-20T11:36:11.807695+0000","last_fullsized":"2024-04-20T11:36:11.807695+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:42:53.512494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.674832+0000","last_change":"2024-04-20T11:36:08.483684+0000","last_active":"2024-04-20T11:36:10.674832+0000","last_peered":"2024-04-20T11:36:10.674832+0000","last_clean":"2024-04-20T11:36:10.674832+0000","last_became_active":"2024-04-20T11:36:08.483405+0000","last_became_peered":"2024-04-20T11:36:08.483405+0000","last_unstale":"2024-04-20T11:36:10.674832+0000","last_undegraded":"2024-04-20T11:36:10.674832+0000","last_fullsized":"2024-04-20T11:36:10.674832+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:08:24.707643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.496468+0000","last_change":"2024-04-20T11:36:09.503075+0000","last_active":"2024-04-20T11:36:10.496468+0000","last_peered":"2024-04-20T11:36:10.496468+0000","last_clean":"2024-04-20T11:36:10.496468+0000","last_became_active":"2024-04-20T11:36:09.502712+0000","last_became_peered":"2024-04-20T11:36:09.502712+0000","last_unstale":"2024-04-20T11:36:10.496468+0000","last_undegraded":"2024-04-20T11:36:10.496468+0000","last_fullsized":"2024-04-20T11:36:10.496468+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:56:54.406631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.496193+0000","last_change":"2024-04-20T11:36:08.472571+0000","last_active":"2024-04-20T11:36:10.496193+0000","last_peered":"2024-04-20T11:36:10.496193+0000","last_clean":"2024-04-20T11:36:10.496193+0000","last_became_active":"2024-04-20T11:36:08.470365+0000","last_became_peered":"2024-04-20T11:36:08.470365+0000","last_unstale":"2024-04-20T11:36:10.496193+0000","last_undegraded":"2024-04-20T11:36:10.496193+0000","last_fullsized":"2024-04-20T11:36:10.496193+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:23:09.673966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.496372+0000","last_change":"2024-04-20T11:36:09.496179+0000","last_active":"2024-04-20T11:36:10.496372+0000","last_peered":"2024-04-20T11:36:10.496372+0000","last_clean":"2024-04-20T11:36:10.496372+0000","last_became_active":"2024-04-20T11:36:09.495921+0000","last_became_peered":"2024-04-20T11:36:09.495921+0000","last_unstale":"2024-04-20T11:36:10.496372+0000","last_undegraded":"2024-04-20T11:36:10.496372+0000","last_fullsized":"2024-04-20T11:36:10.496372+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:48:17.725873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.513030+0000","last_change":"2024-04-20T11:36:08.471003+0000","last_active":"2024-04-20T11:36:11.513030+0000","last_peered":"2024-04-20T11:36:11.513030+0000","last_clean":"2024-04-20T11:36:11.513030+0000","last_became_active":"2024-04-20T11:36:08.470689+0000","last_became_peered":"2024-04-20T11:36:08.470689+0000","last_unstale":"2024-04-20T11:36:11.513030+0000","last_undegraded":"2024-04-20T11:36:11.513030+0000","last_fullsized":"2024-04-20T11:36:11.513030+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:29:33.311708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.512894+0000","last_change":"2024-04-20T11:36:09.510186+0000","last_active":"2024-04-20T11:36:11.512894+0000","last_peered":"2024-04-20T11:36:11.512894+0000","last_clean":"2024-04-20T11:36:11.512894+0000","last_became_active":"2024-04-20T11:36:09.509860+0000","last_became_peered":"2024-04-20T11:36:09.509860+0000","last_unstale":"2024-04-20T11:36:11.512894+0000","last_undegraded":"2024-04-20T11:36:11.512894+0000","last_fullsized":"2024-04-20T11:36:11.512894+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:55:27.981008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.808267+0000","last_change":"2024-04-20T11:36:08.472251+0000","last_active":"2024-04-20T11:36:11.808267+0000","last_peered":"2024-04-20T11:36:11.808267+0000","last_clean":"2024-04-20T11:36:11.808267+0000","last_became_active":"2024-04-20T11:36:08.469968+0000","last_became_peered":"2024-04-20T11:36:08.469968+0000","last_unstale":"2024-04-20T11:36:11.808267+0000","last_undegraded":"2024-04-20T11:36:11.808267+0000","last_fullsized":"2024-04-20T11:36:11.808267+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:47:08.022520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.491347+0000","last_change":"2024-04-20T11:36:09.506539+0000","last_active":"2024-04-20T11:36:10.491347+0000","last_peered":"2024-04-20T11:36:10.491347+0000","last_clean":"2024-04-20T11:36:10.491347+0000","last_became_active":"2024-04-20T11:36:09.505746+0000","last_became_peered":"2024-04-20T11:36:09.505746+0000","last_unstale":"2024-04-20T11:36:10.491347+0000","last_undegraded":"2024-04-20T11:36:10.491347+0000","last_fullsized":"2024-04-20T11:36:10.491347+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:56:54.406631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.674650+0000","last_change":"2024-04-20T11:36:08.475717+0000","last_active":"2024-04-20T11:36:10.674650+0000","last_peered":"2024-04-20T11:36:10.674650+0000","last_clean":"2024-04-20T11:36:10.674650+0000","last_became_active":"2024-04-20T11:36:08.475397+0000","last_became_peered":"2024-04-20T11:36:08.475397+0000","last_unstale":"2024-04-20T11:36:10.674650+0000","last_undegraded":"2024-04-20T11:36:10.674650+0000","last_fullsized":"2024-04-20T11:36:10.674650+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:52:40.058619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.504842+0000","last_change":"2024-04-20T11:36:09.504191+0000","last_active":"2024-04-20T11:36:11.504842+0000","last_peered":"2024-04-20T11:36:11.504842+0000","last_clean":"2024-04-20T11:36:11.504842+0000","last_became_active":"2024-04-20T11:36:09.500083+0000","last_became_peered":"2024-04-20T11:36:09.500083+0000","last_unstale":"2024-04-20T11:36:11.504842+0000","last_undegraded":"2024-04-20T11:36:11.504842+0000","last_fullsized":"2024-04-20T11:36:11.504842+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:01:44.730919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.512190+0000","last_change":"2024-04-20T11:36:08.467047+0000","last_active":"2024-04-20T11:36:11.512190+0000","last_peered":"2024-04-20T11:36:11.512190+0000","last_clean":"2024-04-20T11:36:11.512190+0000","last_became_active":"2024-04-20T11:36:08.466681+0000","last_became_peered":"2024-04-20T11:36:08.466681+0000","last_unstale":"2024-04-20T11:36:11.512190+0000","last_undegraded":"2024-04-20T11:36:11.512190+0000","last_fullsized":"2024-04-20T11:36:11.512190+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:30:14.516111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.496672+0000","last_change":"2024-04-20T11:36:09.504230+0000","last_active":"2024-04-20T11:36:10.496672+0000","last_peered":"2024-04-20T11:36:10.496672+0000","last_clean":"2024-04-20T11:36:10.496672+0000","last_became_active":"2024-04-20T11:36:09.503909+0000","last_became_peered":"2024-04-20T11:36:09.503909+0000","last_unstale":"2024-04-20T11:36:10.496672+0000","last_undegraded":"2024-04-20T11:36:10.496672+0000","last_fullsized":"2024-04-20T11:36:10.496672+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:08.433765+0000","last_clean_scrub_stamp":"2024-04-20T11:36:08.433765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:10:13.066071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:11.513532+0000","last_change":"2024-04-20T11:36:08.473393+0000","last_active":"2024-04-20T11:36:11.513532+0000","last_peered":"2024-04-20T11:36:11.513532+0000","last_clean":"2024-04-20T11:36:11.513532+0000","last_became_active":"2024-04-20T11:36:08.473104+0000","last_became_peered":"2024-04-20T11:36:08.473104+0000","last_unstale":"2024-04-20T11:36:11.513532+0000","last_undegraded":"2024-04-20T11:36:11.513532+0000","last_fullsized":"2024-04-20T11:36:11.513532+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:07.417183+0000","last_clean_scrub_stamp":"2024-04-20T11:36:07.417183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:08:32.469884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"72'128","reported_seq":427,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-20T11:36:10.498104+0000","last_change":"2024-04-20T11:26:48.661897+0000","last_active":"2024-04-20T11:36:10.498104+0000","last_peered":"2024-04-20T11:36:10.498104+0000","last_clean":"2024-04-20T11:36:10.498104+0000","last_became_active":"2024-04-20T11:26:48.656535+0000","last_became_peered":"2024-04-20T11:26:48.656535+0000","last_unstale":"2024-04-20T11:36:10.498104+0000","last_undegraded":"2024-04-20T11:36:10.498104+0000","last_fullsized":"2024-04-20T11:36:10.498104+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:47.543004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:47.543004+0000","last_clean_scrub_stamp":"2024-04-20T11:26:47.543004+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:58:35.527105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":23,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":22,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574968,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33528,"kb_used_data":2348,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338312,"statfs":{"total":96636764160,"available":96602431488,"internally_reserved":0,"allocated":2404352,"data_stored":1382440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574965,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32932,"kb_used_data":1748,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338908,"statfs":{"total":96636764160,"available":96603041792,"internally_reserved":0,"allocated":1789952,"data_stored":789740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574967,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32932,"kb_used_data":1752,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338908,"statfs":{"total":96636764160,"available":96603041792,"internally_reserved":0,"allocated":1794048,"data_stored":790322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574965,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32940,"kb_used_data":1760,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338900,"statfs":{"total":96636764160,"available":96603033600,"internally_reserved":0,"allocated":1802240,"data_stored":791400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574966,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32928,"kb_used_data":1748,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338912,"statfs":{"total":96636764160,"available":96603045888,"internally_reserved":0,"allocated":1789952,"data_stored":789752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574967,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32928,"kb_used_data":1748,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338912,"statfs":{"total":96636764160,"available":96603045888,"internally_reserved":0,"allocated":1789952,"data_stored":789808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574965,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32928,"kb_used_data":1748,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338912,"statfs":{"total":96636764160,"available":96603045888,"internally_reserved":0,"allocated":1789952,"data_stored":789752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574966,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33512,"kb_used_data":2324,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338328,"statfs":{"total":96636764160,"available":96602447872,"internally_reserved":0,"allocated":2379776,"data_stored":1380086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T11:36:19.398 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-20T11:36:19.398 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-20T11:36:19.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.647+0000 7f6b49a00640 1 Processor -- start 2024-04-20T11:36:19.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.648+0000 7f6b49a00640 1 -- start start 2024-04-20T11:36:19.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.649+0000 7f6b49a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f6b4406f8f0 0x7f6b440d0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:19.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.649+0000 7f6b49a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b4406df50 0x7f6b4406e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:19.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.649+0000 7f6b49a00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b4415d6d0 con 0x7f6b440b3ea0 2024-04-20T11:36:19.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.649+0000 7f6b49a00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f6b44071d70 con 0x7f6b4406f8f0 2024-04-20T11:36:19.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.649+0000 7f6b49a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6b44072050 con 0x7f6b4406df50 2024-04-20T11:36:19.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.649+0000 7f6b42a00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f6b440b3ea0 0x7f6b440b4280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:44318/0 (socket says 172.21.15.152:44318) 2024-04-20T11:36:19.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.649+0000 7f6b42a00640 1 -- 172.21.15.152:0/47144109 learned_addr learned my addr 172.21.15.152:0/47144109 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:36:19.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.650+0000 7f6b42000640 1 --2- 172.21.15.152:0/47144109 >> v2:172.21.15.152:3300/0 conn(0x7f6b4406f8f0 0x7f6b440d0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:19.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.650+0000 7f6b43400640 1 --2- 172.21.15.152:0/47144109 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b4406df50 0x7f6b4406e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:19.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.650+0000 7f6b41600640 1 -- 172.21.15.152:0/47144109 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1064250466 0 0) 0x7f6b4415d6d0 con 0x7f6b440b3ea0 2024-04-20T11:36:19.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.650+0000 7f6b41600640 1 -- 172.21.15.152:0/47144109 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6b28003520 con 0x7f6b440b3ea0 2024-04-20T11:36:19.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.650+0000 7f6b42000640 1 -- 172.21.15.152:0/47144109 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b4406df50 msgr2=0x7f6b4406e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:19.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.650+0000 7f6b42000640 1 --2- 172.21.15.152:0/47144109 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b4406df50 0x7f6b4406e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:19.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.650+0000 7f6b42000640 1 -- 172.21.15.152:0/47144109 >> v1:172.21.15.152:6789/0 conn(0x7f6b440b3ea0 legacy=0x7f6b440b4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:19.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.650+0000 7f6b42000640 1 -- 172.21.15.152:0/47144109 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b44072330 con 0x7f6b4406f8f0 2024-04-20T11:36:19.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.650+0000 7f6b43400640 1 --2- 172.21.15.152:0/47144109 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b4406df50 0x7f6b4406e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:36:19.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.652+0000 7f6b42000640 1 --2- 172.21.15.152:0/47144109 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6b4406f8f0 0x7f6b440d0630 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c0097a0 tx=0x7f6b2c02b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=427ad63a1322e58e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:19.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.652+0000 7f6b41600640 1 -- 172.21.15.152:0/47144109 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b2c035020 con 0x7f6b4406f8f0 2024-04-20T11:36:19.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.652+0000 7f6b41600640 1 -- 172.21.15.152:0/47144109 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b2c031810 con 0x7f6b4406f8f0 2024-04-20T11:36:19.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.652+0000 7f6b41600640 1 -- 172.21.15.152:0/47144109 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b2c031be0 con 0x7f6b4406f8f0 2024-04-20T11:36:19.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.652+0000 7f6b49a00640 1 -- 172.21.15.152:0/47144109 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6b4406f8f0 msgr2=0x7f6b440d0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:19.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.652+0000 7f6b49a00640 1 --2- 172.21.15.152:0/47144109 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6b4406f8f0 0x7f6b440d0630 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c0097a0 tx=0x7f6b2c02b8b0 comp rx=0 tx=0).stop 2024-04-20T11:36:19.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.653+0000 7f6b49a00640 1 -- 172.21.15.152:0/47144109 shutdown_connections 2024-04-20T11:36:19.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.653+0000 7f6b49a00640 1 --2- 172.21.15.152:0/47144109 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b4406df50 0x7f6b4406e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:19.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.653+0000 7f6b49a00640 1 --2- 172.21.15.152:0/47144109 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6b4406f8f0 0x7f6b440d0630 unknown :-1 s=CLOSED pgs=1987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:19.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.653+0000 7f6b49a00640 1 -- 172.21.15.152:0/47144109 >> 172.21.15.152:0/47144109 conn(0x7f6b4406b980 msgr2=0x7f6b4406bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:19.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.653+0000 7f6b49a00640 1 -- 172.21.15.152:0/47144109 shutdown_connections 2024-04-20T11:36:19.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.653+0000 7f6b49a00640 1 -- 172.21.15.152:0/47144109 wait complete. 2024-04-20T11:36:19.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.654+0000 7f6b49a00640 1 Processor -- start 2024-04-20T11:36:19.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.655+0000 7f6b49a00640 1 -- start start 2024-04-20T11:36:19.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.655+0000 7f6b49a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b4406df50 0x7f6b4406ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:19.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.655+0000 7f6b49a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b4406f8f0 0x7f6b44161f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:19.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.656+0000 7f6b49a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6b440b3ea0 0x7f6b441597b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:19.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.656+0000 7f6b49a00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f6b440d0b70 con 0x7f6b440b3ea0 2024-04-20T11:36:19.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.656+0000 7f6b49a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6b440728f0 con 0x7f6b4406f8f0 2024-04-20T11:36:19.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.656+0000 7f6b49a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6b440d0cf0 con 0x7f6b4406df50 2024-04-20T11:36:19.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.656+0000 7f6b43400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6b440b3ea0 0x7f6b441597b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:19.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.656+0000 7f6b43400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6b440b3ea0 0x7f6b441597b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:40296/0 (socket says 172.21.15.152:40296) 2024-04-20T11:36:19.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.656+0000 7f6b43400640 1 -- 172.21.15.152:0/2441497423 learned_addr learned my addr 172.21.15.152:0/2441497423 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:19.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.656+0000 7f6b42a00640 1 --2- 172.21.15.152:0/2441497423 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b4406df50 0x7f6b4406ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:19.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.656+0000 7f6b43400640 1 -- 172.21.15.152:0/2441497423 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b4406df50 msgr2=0x7f6b4406ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:19.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.656+0000 7f6b43400640 1 --2- 172.21.15.152:0/2441497423 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b4406df50 0x7f6b4406ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:19.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.656+0000 7f6b43400640 1 -- 172.21.15.152:0/2441497423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b4406f8f0 msgr2=0x7f6b44161f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:19.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.656+0000 7f6b43400640 1 --2- 172.21.15.152:0/2441497423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b4406f8f0 0x7f6b44161f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:19.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.656+0000 7f6b43400640 1 -- 172.21.15.152:0/2441497423 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b441fa610 con 0x7f6b440b3ea0 2024-04-20T11:36:19.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.657+0000 7f6b43400640 1 --2- 172.21.15.152:0/2441497423 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6b440b3ea0 0x7f6b441597b0 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c007df0 tx=0x7f6b3c00d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:19.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.657+0000 7f6b3aa00640 1 -- 172.21.15.152:0/2441497423 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b3c01a070 con 0x7f6b440b3ea0 2024-04-20T11:36:19.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.657+0000 7f6b3aa00640 1 -- 172.21.15.152:0/2441497423 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b3c003990 con 0x7f6b440b3ea0 2024-04-20T11:36:19.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.657+0000 7f6b3aa00640 1 -- 172.21.15.152:0/2441497423 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b3c003d80 con 0x7f6b440b3ea0 2024-04-20T11:36:19.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.657+0000 7f6b49a00640 1 -- 172.21.15.152:0/2441497423 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b441fa8b0 con 0x7f6b440b3ea0 2024-04-20T11:36:19.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.657+0000 7f6b49a00640 1 -- 172.21.15.152:0/2441497423 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b441fae70 con 0x7f6b440b3ea0 2024-04-20T11:36:19.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.658+0000 7f6b49a00640 1 -- 172.21.15.152:0/2441497423 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b441fb070 con 0x7f6b440b3ea0 2024-04-20T11:36:19.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.659+0000 7f6b3aa00640 1 -- 172.21.15.152:0/2441497423 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 22) ==== 91601+0+0 (secure 0 0 0) 0x7f6b3c01d070 con 0x7f6b440b3ea0 2024-04-20T11:36:19.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.660+0000 7f6b3aa00640 1 --2- 172.21.15.152:0/2441497423 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f6b2006cf20 0x7f6b2006f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:19.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.660+0000 7f6b3aa00640 1 -- 172.21.15.152:0/2441497423 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5709+0+0 (secure 0 0 0) 0x7f6b3c08c980 con 0x7f6b440b3ea0 2024-04-20T11:36:19.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.661+0000 7f6b42a00640 1 --2- 172.21.15.152:0/2441497423 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f6b2006cf20 0x7f6b2006f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:19.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.662+0000 7f6b42a00640 1 --2- 172.21.15.152:0/2441497423 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f6b2006cf20 0x7f6b2006f3e0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f6b441eec80 tx=0x7f6b34009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:19.664 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.664+0000 7f6b3aa00640 1 -- 172.21.15.152:0/2441497423 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f6b441fb070 con 0x7f6b440b3ea0 2024-04-20T11:36:19.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.934+0000 7f6b49a00640 1 -- 172.21.15.152:0/2441497423 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f6b440689f0 con 0x7f6b440b3ea0 2024-04-20T11:36:19.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.938+0000 7f6b3aa00640 1 -- 172.21.15.152:0/2441497423 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v21) ==== 190+0+147 (secure 0 0 0) 0x7f6b3c010070 con 0x7f6b440b3ea0 2024-04-20T11:36:19.938 INFO:teuthology.orchestra.run.smithi152.stdout:[client.0] 2024-04-20T11:36:19.938 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQBDpiNmtsBWFhAArq9A7r33CcwgMmOU2pAGPw== 2024-04-20T11:36:19.938 INFO:teuthology.orchestra.run.smithi152.stdout: caps mds = "allow" 2024-04-20T11:36:19.938 INFO:teuthology.orchestra.run.smithi152.stdout: caps mon = "allow r" 2024-04-20T11:36:19.938 INFO:teuthology.orchestra.run.smithi152.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-20T11:36:19.938 INFO:teuthology.orchestra.run.smithi152.stderr:updated caps for client.0 2024-04-20T11:36:19.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.941+0000 7f6b49a00640 1 -- 172.21.15.152:0/2441497423 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f6b2006cf20 msgr2=0x7f6b2006f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:19.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.941+0000 7f6b49a00640 1 --2- 172.21.15.152:0/2441497423 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f6b2006cf20 0x7f6b2006f3e0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f6b441eec80 tx=0x7f6b34009040 comp rx=0 tx=0).stop 2024-04-20T11:36:19.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.941+0000 7f6b49a00640 1 -- 172.21.15.152:0/2441497423 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6b440b3ea0 msgr2=0x7f6b441597b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:19.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.941+0000 7f6b49a00640 1 --2- 172.21.15.152:0/2441497423 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6b440b3ea0 0x7f6b441597b0 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c007df0 tx=0x7f6b3c00d4a0 comp rx=0 tx=0).stop 2024-04-20T11:36:19.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.941+0000 7f6b49a00640 1 -- 172.21.15.152:0/2441497423 shutdown_connections 2024-04-20T11:36:19.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.941+0000 7f6b49a00640 1 --2- 172.21.15.152:0/2441497423 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f6b2006cf20 0x7f6b2006f3e0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:19.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.941+0000 7f6b49a00640 1 --2- 172.21.15.152:0/2441497423 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6b440b3ea0 0x7f6b441597b0 unknown :-1 s=CLOSED pgs=1988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:19.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.941+0000 7f6b49a00640 1 --2- 172.21.15.152:0/2441497423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b4406f8f0 0x7f6b44161f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:19.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.941+0000 7f6b49a00640 1 --2- 172.21.15.152:0/2441497423 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b4406df50 0x7f6b4406ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:19.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.941+0000 7f6b49a00640 1 -- 172.21.15.152:0/2441497423 >> 172.21.15.152:0/2441497423 conn(0x7f6b4406b980 msgr2=0x7f6b4415c920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:19.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.942+0000 7f6b49a00640 1 -- 172.21.15.152:0/2441497423 shutdown_connections 2024-04-20T11:36:19.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:19.942+0000 7f6b49a00640 1 -- 172.21.15.152:0/2441497423 wait complete. 2024-04-20T11:36:19.959 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:36:20.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:20.155+0000 7f5b2fe00640 1 Processor -- start 2024-04-20T11:36:20.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:20.156+0000 7f5b2fe00640 1 -- start start 2024-04-20T11:36:20.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:20.157+0000 7f5b2fe00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f5b2815e8e0 0x7f5b280d1740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:20.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:20.157+0000 7f5b2fe00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f5b280ba170 0x7f5b280d1c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:20.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:20.157+0000 7f5b2fe00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b280cfd30 con 0x7f5b280cf830 2024-04-20T11:36:20.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:20.157+0000 7f5b2fe00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f5b280b8430 con 0x7f5b2815e8e0 2024-04-20T11:36:20.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:20.157+0000 7f5b2fe00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f5b280b8680 con 0x7f5b280ba170 2024-04-20T11:36:20.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:20.158+0000 7f5b2d400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f5b2815e8e0 0x7f5b280d1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:20.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:20.158+0000 7f5b2ca00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f5b280cf830 0x7f5b28162450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:44330/0 (socket says 172.21.15.152:44330) 2024-04-20T11:36:20.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:20.158+0000 7f5b2ca00640 1 -- 172.21.15.152:0/543675903 learned_addr learned my addr 172.21.15.152:0/543675903 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:36:20.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:20.158+0000 7f5b2de00640 1 --2- 172.21.15.152:0/543675903 >> v2:172.21.15.175:3300/0 conn(0x7f5b280ba170 0x7f5b280d1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:20.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:20.158+0000 7f5b2d400640 1 -- 172.21.15.152:0/543675903 >> v2:172.21.15.175:3300/0 conn(0x7f5b280ba170 msgr2=0x7f5b280d1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:20.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:20.158+0000 7f5b2d400640 1 --2- 172.21.15.152:0/543675903 >> v2:172.21.15.175:3300/0 conn(0x7f5b280ba170 0x7f5b280d1c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:20.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:20.158+0000 7f5b2d400640 1 -- 172.21.15.152:0/543675903 >> v1:172.21.15.152:6789/0 conn(0x7f5b280cf830 legacy=0x7f5b28162450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:20.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:20.158+0000 7f5b2d400640 1 -- 172.21.15.152:0/543675903 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b280b8960 con 0x7f5b2815e8e0 2024-04-20T11:36:20.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:20.159+0000 7f5b2d400640 1 --2- 172.21.15.152:0/543675903 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5b2815e8e0 0x7f5b280d1740 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7f5b280c82f0 tx=0x7f5b1c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8d427465d702ed7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:20.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:20.160+0000 7f5b27e00640 1 -- 172.21.15.152:0/543675903 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b1c035020 con 0x7f5b2815e8e0 2024-04-20T11:36:20.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:20.160+0000 7f5b27e00640 1 -- 172.21.15.152:0/543675903 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b1c031830 con 0x7f5b2815e8e0 2024-04-20T11:36:20.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:20.160+0000 7f5b27e00640 1 -- 172.21.15.152:0/543675903 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b1c031c00 con 0x7f5b2815e8e0 2024-04-20T11:36:20.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:20.160+0000 7f5b2fe00640 1 -- 172.21.15.152:0/543675903 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5b2815e8e0 msgr2=0x7f5b280d1740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:20.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:20.160+0000 7f5b2fe00640 1 --2- 172.21.15.152:0/543675903 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5b2815e8e0 0x7f5b280d1740 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7f5b280c82f0 tx=0x7f5b1c02b8d0 comp rx=0 tx=0).stop 2024-04-20T11:36:20.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:20.160+0000 7f5b2fe00640 1 -- 172.21.15.152:0/543675903 shutdown_connections 2024-04-20T11:36:20.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:20.160+0000 7f5b2fe00640 1 --2- 172.21.15.152:0/543675903 >> v2:172.21.15.175:3300/0 conn(0x7f5b280ba170 0x7f5b280d1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:20.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:20.160+0000 7f5b2fe00640 1 --2- 172.21.15.152:0/543675903 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5b2815e8e0 0x7f5b280d1740 unknown :-1 s=CLOSED pgs=1990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:20.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:20.160+0000 7f5b2fe00640 1 -- 172.21.15.152:0/543675903 >> 172.21.15.152:0/543675903 conn(0x7f5b28067ae0 msgr2=0x7f5b28067ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:20.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:20.160+0000 7f5b2fe00640 1 -- 172.21.15.152:0/543675903 shutdown_connections 2024-04-20T11:36:20.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:20.161+0000 7f5b2fe00640 1 -- 172.21.15.152:0/543675903 wait complete. 2024-04-20T11:36:20.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:20.162+0000 7f5b2fe00640 1 Processor -- start 2024-04-20T11:36:20.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:20.162+0000 7f5b2fe00640 1 -- start start 2024-04-20T11:36:20.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:20.163+0000 7f5b2fe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5b280ba170 0x7f5b280ca2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:20.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:20.163+0000 7f5b2fe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5b280cf830 0x7f5b28148450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:20.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:20.163+0000 7f5b2fe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5b2815e8e0 0x7f5b28160ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:20.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:20.163+0000 7f5b2fe00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f5b280b3af0 con 0x7f5b280ba170 2024-04-20T11:36:20.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:20.163+0000 7f5b2fe00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5b2815ecc0 con 0x7f5b280cf830 2024-04-20T11:36:20.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:20.163+0000 7f5b2fe00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5b280b3c70 con 0x7f5b2815e8e0 2024-04-20T11:36:20.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:20.163+0000 7f5b2d400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5b280ba170 0x7f5b280ca2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:20.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:20.164+0000 7f5b2d400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5b280ba170 0x7f5b280ca2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:40302/0 (socket says 172.21.15.152:40302) 2024-04-20T11:36:20.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:20.164+0000 7f5b2d400640 1 -- 172.21.15.152:0/595961343 learned_addr learned my addr 172.21.15.152:0/595961343 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:20.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:20.164+0000 7f5b2d400640 1 -- 172.21.15.152:0/595961343 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5b2815e8e0 msgr2=0x7f5b28160ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:20.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:20.164+0000 7f5b2d400640 1 --2- 172.21.15.152:0/595961343 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5b2815e8e0 0x7f5b28160ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:20.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:20.164+0000 7f5b2d400640 1 -- 172.21.15.152:0/595961343 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5b280cf830 msgr2=0x7f5b28148450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:20.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:20.164+0000 7f5b2d400640 1 --2- 172.21.15.152:0/595961343 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5b280cf830 0x7f5b28148450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:20.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:20.164+0000 7f5b2d400640 1 -- 172.21.15.152:0/595961343 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b2815fe90 con 0x7f5b280ba170 2024-04-20T11:36:20.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:20.164+0000 7f5b2d400640 1 --2- 172.21.15.152:0/595961343 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5b280ba170 0x7f5b280ca2e0 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7f5b1c009890 tx=0x7f5b1c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:20.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:20.165+0000 7f5b25600640 1 -- 172.21.15.152:0/595961343 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b1c041070 con 0x7f5b280ba170 2024-04-20T11:36:20.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:20.165+0000 7f5b25600640 1 -- 172.21.15.152:0/595961343 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b1c03aa30 con 0x7f5b280ba170 2024-04-20T11:36:20.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:20.165+0000 7f5b2fe00640 1 -- 172.21.15.152:0/595961343 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b28160130 con 0x7f5b280ba170 2024-04-20T11:36:20.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:20.165+0000 7f5b25600640 1 -- 172.21.15.152:0/595961343 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b1c035040 con 0x7f5b280ba170 2024-04-20T11:36:20.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:20.165+0000 7f5b2fe00640 1 -- 172.21.15.152:0/595961343 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b281606f0 con 0x7f5b280ba170 2024-04-20T11:36:20.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:20.166+0000 7f5b2fe00640 1 -- 172.21.15.152:0/595961343 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b280689f0 con 0x7f5b280ba170 2024-04-20T11:36:20.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:20.167+0000 7f5b25600640 1 -- 172.21.15.152:0/595961343 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 22) ==== 91601+0+0 (secure 0 0 0) 0x7f5b1c038020 con 0x7f5b280ba170 2024-04-20T11:36:20.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:20.168+0000 7f5b25600640 1 --2- 172.21.15.152:0/595961343 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f5b0806cf20 0x7f5b0806f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:20.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:20.168+0000 7f5b25600640 1 -- 172.21.15.152:0/595961343 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5709+0+0 (secure 0 0 0) 0x7f5b1c0ae0c0 con 0x7f5b280ba170 2024-04-20T11:36:20.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:20.169+0000 7f5b2ca00640 1 --2- 172.21.15.152:0/595961343 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f5b0806cf20 0x7f5b0806f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:20.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:20.170+0000 7f5b2ca00640 1 --2- 172.21.15.152:0/595961343 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f5b0806cf20 0x7f5b0806f3e0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f5b14000f60 tx=0x7f5b14007a80 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:20.172 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:20.172+0000 7f5b25600640 1 -- 172.21.15.152:0/595961343 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f5b1c07bd20 con 0x7f5b280ba170 2024-04-20T11:36:20.412 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:20.412+0000 7f5b2fe00640 1 -- 172.21.15.152:0/595961343 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5b280cb020 con 0x7f5b280ba170 2024-04-20T11:36:20.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:20.413+0000 7f5b25600640 1 -- 172.21.15.152:0/595961343 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) ==== 94+0+5209 (secure 0 0 0) 0x7f5b1c07b620 con 0x7f5b280ba170 2024-04-20T11:36:20.413 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:36:20.413 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":96,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6275,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6834/721066448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":721066448},{"type":"v1","addr":"172.21.15.175:6834","nonce":721066448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":91},{"gid":9483,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6834/3026417634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":3026417634},{"type":"v1","addr":"172.21.15.152:6834","nonce":3026417634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":90},{"gid":9486,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/3399678659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6833","nonce":3399678659},{"type":"v1","addr":"172.21.15.152:6835","nonce":3399678659}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":91}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T11:36:08.919586+0000","modified":"2024-04-20T11:36:12.020458+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9501},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9501":{"gid":9501,"name":"b","rank":0,"incarnation":92,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6835/189289100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6833","nonce":189289100},{"type":"v1","addr":"172.21.15.175:6835","nonce":189289100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9501,"qdb_cluster":[9501]},"id":11}]} 2024-04-20T11:36:20.413 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 96 2024-04-20T11:36:20.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:20.416+0000 7f5b2fe00640 1 -- 172.21.15.152:0/595961343 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f5b0806cf20 msgr2=0x7f5b0806f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:20.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:20.416+0000 7f5b2fe00640 1 --2- 172.21.15.152:0/595961343 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f5b0806cf20 0x7f5b0806f3e0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f5b14000f60 tx=0x7f5b14007a80 comp rx=0 tx=0).stop 2024-04-20T11:36:20.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:20.416+0000 7f5b2fe00640 1 -- 172.21.15.152:0/595961343 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5b280ba170 msgr2=0x7f5b280ca2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:20.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:20.416+0000 7f5b2fe00640 1 --2- 172.21.15.152:0/595961343 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5b280ba170 0x7f5b280ca2e0 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7f5b1c009890 tx=0x7f5b1c0046c0 comp rx=0 tx=0).stop 2024-04-20T11:36:20.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:20.416+0000 7f5b2fe00640 1 -- 172.21.15.152:0/595961343 shutdown_connections 2024-04-20T11:36:20.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:20.416+0000 7f5b2fe00640 1 --2- 172.21.15.152:0/595961343 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f5b0806cf20 0x7f5b0806f3e0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:20.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:20.416+0000 7f5b2fe00640 1 --2- 172.21.15.152:0/595961343 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5b2815e8e0 0x7f5b28160ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:20.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:20.416+0000 7f5b2fe00640 1 --2- 172.21.15.152:0/595961343 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5b280cf830 0x7f5b28148450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:20.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:20.416+0000 7f5b2fe00640 1 --2- 172.21.15.152:0/595961343 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5b280ba170 0x7f5b280ca2e0 unknown :-1 s=CLOSED pgs=1991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:20.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:20.416+0000 7f5b2fe00640 1 -- 172.21.15.152:0/595961343 >> 172.21.15.152:0/595961343 conn(0x7f5b28067ae0 msgr2=0x7f5b280b7920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:20.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:20.417+0000 7f5b2fe00640 1 -- 172.21.15.152:0/595961343 shutdown_connections 2024-04-20T11:36:20.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:20.417+0000 7f5b2fe00640 1 -- 172.21.15.152:0/595961343 wait complete. 2024-04-20T11:36:20.838 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 96, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-20T11:36:08.919586+0000', 'modified': '2024-04-20T11:36:12.020458+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'snap_rstat': True, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9501}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9501': {'gid': 9501, 'name': 'b', 'rank': 0, 'incarnation': 92, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.175:6835/189289100', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6833', 'nonce': 189289100}, {'type': 'v1', 'addr': '172.21.15.175:6835', 'nonce': 189289100}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [23], 'metadata_pool': 22, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9501, 'qdb_cluster': [9501]} 2024-04-20T11:36:20.838 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-20T11:36:20.838 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-20T11:36:21.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.049+0000 7f7036800640 1 Processor -- start 2024-04-20T11:36:21.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.049+0000 7f7036800640 1 -- start start 2024-04-20T11:36:21.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.051+0000 7f7036800640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f70300d4ec0 0x7f70300d0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:21.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.051+0000 7f7036800640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70300d26a0 con 0x7f7030161d30 2024-04-20T11:36:21.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.051+0000 7f7036800640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70300d53c0 con 0x7f70300d2b80 2024-04-20T11:36:21.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.051+0000 7f7036800640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f70300d0540 con 0x7f70300d4ec0 2024-04-20T11:36:21.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.051+0000 7f7035400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f7030161d30 0x7f70300d15e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:44346/0 (socket says 172.21.15.152:44346) 2024-04-20T11:36:21.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.051+0000 7f7035400640 1 -- 172.21.15.152:0/1502695987 learned_addr learned my addr 172.21.15.152:0/1502695987 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:36:21.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.051+0000 7f702fe00640 1 -- 172.21.15.152:0/1502695987 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1958387588 0 0) 0x7f70300d26a0 con 0x7f7030161d30 2024-04-20T11:36:21.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.051+0000 7f702fe00640 1 -- 172.21.15.152:0/1502695987 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7018003520 con 0x7f7030161d30 2024-04-20T11:36:21.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.052+0000 7f7034a00640 1 --2- 172.21.15.152:0/1502695987 >> v2:172.21.15.175:3300/0 conn(0x7f70300d4ec0 0x7f70300d0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:21.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.052+0000 7f702fe00640 1 -- 172.21.15.152:0/1502695987 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3031483309 0 0) 0x7f7018003520 con 0x7f7030161d30 2024-04-20T11:36:21.054 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.052+0000 7f702fe00640 1 -- 172.21.15.152:0/1502695987 >> v2:172.21.15.175:3300/0 conn(0x7f70300d4ec0 msgr2=0x7f70300d0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:21.054 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.052+0000 7f702fe00640 1 --2- 172.21.15.152:0/1502695987 >> v2:172.21.15.175:3300/0 conn(0x7f70300d4ec0 0x7f70300d0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:21.054 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.052+0000 7f702fe00640 1 -- 172.21.15.152:0/1502695987 >> v1:172.21.15.175:6789/0 conn(0x7f70300d2b80 legacy=0x7f703006c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:21.054 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.052+0000 7f702fe00640 1 -- 172.21.15.152:0/1502695987 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70300cf520 con 0x7f7030161d30 2024-04-20T11:36:21.054 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.052+0000 7f702fe00640 1 -- 172.21.15.152:0/1502695987 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f7024001670 con 0x7f7030161d30 2024-04-20T11:36:21.054 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.052+0000 7f702fe00640 1 -- 172.21.15.152:0/1502695987 >> v1:172.21.15.152:6789/0 conn(0x7f7030161d30 legacy=0x7f70300d15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:21.054 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.052+0000 7f702fe00640 1 --2- 172.21.15.152:0/1502695987 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f70180039b0 0x7f7018003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:21.054 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.052+0000 7f702fe00640 1 --2- 172.21.15.152:0/1502695987 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7018004800 0x7f7018004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:21.054 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.052+0000 7f702fe00640 1 --2- 172.21.15.152:0/1502695987 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f70180051c0 0x7f701800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:21.054 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.052+0000 7f702fe00640 1 -- 172.21.15.152:0/1502695987 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f70300d0540 con 0x7f7018004800 2024-04-20T11:36:21.054 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.052+0000 7f702fe00640 1 -- 172.21.15.152:0/1502695987 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f701800c110 con 0x7f70180051c0 2024-04-20T11:36:21.054 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.052+0000 7f702fe00640 1 -- 172.21.15.152:0/1502695987 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f701800c310 con 0x7f70180039b0 2024-04-20T11:36:21.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.052+0000 7f7034a00640 1 --2- 172.21.15.152:0/1502695987 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7018004800 0x7f7018004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:21.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.053+0000 7f7035e00640 1 --2- 172.21.15.152:0/1502695987 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f70180051c0 0x7f701800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:21.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.053+0000 7f7035400640 1 --2- 172.21.15.152:0/1502695987 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f70180039b0 0x7f7018003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:21.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.053+0000 7f7034a00640 1 -- 172.21.15.152:0/1502695987 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f70180039b0 msgr2=0x7f7018003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:21.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.053+0000 7f7034a00640 1 --2- 172.21.15.152:0/1502695987 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f70180039b0 0x7f7018003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:21.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.053+0000 7f7034a00640 1 -- 172.21.15.152:0/1502695987 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f70180051c0 msgr2=0x7f701800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:21.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.053+0000 7f7034a00640 1 --2- 172.21.15.152:0/1502695987 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f70180051c0 0x7f701800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:21.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.053+0000 7f7034a00640 1 -- 172.21.15.152:0/1502695987 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7018003520 con 0x7f7018004800 2024-04-20T11:36:21.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.053+0000 7f7035e00640 1 -- 172.21.15.152:0/1502695987 reap_dead start 2024-04-20T11:36:21.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.053+0000 7f7035400640 1 --2- 172.21.15.152:0/1502695987 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f70180039b0 0x7f7018003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:36:21.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.054+0000 7f7034a00640 1 --2- 172.21.15.152:0/1502695987 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7018004800 0x7f7018004c80 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7f701c00ba10 tx=0x7f7018005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17d28f4c2231ebb4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:21.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.054+0000 7f702fe00640 1 -- 172.21.15.152:0/1502695987 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f701c00b5e0 con 0x7f7018004800 2024-04-20T11:36:21.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.054+0000 7f702fe00640 1 -- 172.21.15.152:0/1502695987 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f701c037070 con 0x7f7018004800 2024-04-20T11:36:21.057 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.054+0000 7f702fe00640 1 -- 172.21.15.152:0/1502695987 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f701c005db0 con 0x7f7018004800 2024-04-20T11:36:21.057 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.055+0000 7f7036800640 1 -- 172.21.15.152:0/1502695987 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7018004800 msgr2=0x7f7018004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:21.057 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.055+0000 7f7036800640 1 --2- 172.21.15.152:0/1502695987 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7018004800 0x7f7018004c80 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7f701c00ba10 tx=0x7f7018005830 comp rx=0 tx=0).stop 2024-04-20T11:36:21.057 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.055+0000 7f7036800640 1 -- 172.21.15.152:0/1502695987 shutdown_connections 2024-04-20T11:36:21.057 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.055+0000 7f7036800640 1 --2- 172.21.15.152:0/1502695987 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7018004800 0x7f7018004c80 unknown :-1 s=CLOSED pgs=1993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:21.057 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.055+0000 7f7036800640 1 -- 172.21.15.152:0/1502695987 >> 172.21.15.152:0/1502695987 conn(0x7f703008bfd0 msgr2=0x7f70300d45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:21.057 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.055+0000 7f7036800640 1 -- 172.21.15.152:0/1502695987 shutdown_connections 2024-04-20T11:36:21.057 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.055+0000 7f7036800640 1 -- 172.21.15.152:0/1502695987 wait complete. 2024-04-20T11:36:21.057 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.057+0000 7f7036800640 1 Processor -- start 2024-04-20T11:36:21.057 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.057+0000 7f7036800640 1 -- start start 2024-04-20T11:36:21.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.058+0000 7f7036800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7018004800 0x7f70300ccb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:21.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.058+0000 7f7036800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f70300b9f90 0x7f7030155ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:21.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.058+0000 7f7036800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f70300b9880 0x7f70300b26e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:21.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.058+0000 7f7036800640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f70300cf1b0 con 0x7f70300b9880 2024-04-20T11:36:21.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.058+0000 7f7036800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f70300cfb70 con 0x7f70300b9f90 2024-04-20T11:36:21.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.058+0000 7f7036800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7030162110 con 0x7f7018004800 2024-04-20T11:36:21.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.058+0000 7f7035e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f70300b9880 0x7f70300b26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:21.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.058+0000 7f7035e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f70300b9880 0x7f70300b26e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:40312/0 (socket says 172.21.15.152:40312) 2024-04-20T11:36:21.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.058+0000 7f7035e00640 1 -- 172.21.15.152:0/4047333504 learned_addr learned my addr 172.21.15.152:0/4047333504 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:21.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.058+0000 7f7035400640 1 --2- 172.21.15.152:0/4047333504 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7018004800 0x7f70300ccb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:21.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.059+0000 7f7034a00640 1 --2- 172.21.15.152:0/4047333504 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f70300b9f90 0x7f7030155ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:21.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.059+0000 7f7035e00640 1 -- 172.21.15.152:0/4047333504 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7018004800 msgr2=0x7f70300ccb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:21.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.059+0000 7f7035e00640 1 --2- 172.21.15.152:0/4047333504 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7018004800 0x7f70300ccb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:21.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.059+0000 7f7035e00640 1 -- 172.21.15.152:0/4047333504 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f70300b9f90 msgr2=0x7f7030155ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:21.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.059+0000 7f7035e00640 1 --2- 172.21.15.152:0/4047333504 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f70300b9f90 0x7f7030155ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:21.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.059+0000 7f7035e00640 1 -- 172.21.15.152:0/4047333504 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70301f4720 con 0x7f70300b9880 2024-04-20T11:36:21.061 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.059+0000 7f7035e00640 1 --2- 172.21.15.152:0/4047333504 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f70300b9880 0x7f70300b26e0 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7f70300d2120 tx=0x7f702800e970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:21.061 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.059+0000 7f702d600640 1 -- 172.21.15.152:0/4047333504 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7028019070 con 0x7f70300b9880 2024-04-20T11:36:21.061 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.059+0000 7f702d600640 1 -- 172.21.15.152:0/4047333504 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7028006d60 con 0x7f70300b9880 2024-04-20T11:36:21.061 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.059+0000 7f7036800640 1 -- 172.21.15.152:0/4047333504 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70301f4a00 con 0x7f70300b9880 2024-04-20T11:36:21.061 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.059+0000 7f702d600640 1 -- 172.21.15.152:0/4047333504 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7028003bb0 con 0x7f70300b9880 2024-04-20T11:36:21.061 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.060+0000 7f7036800640 1 -- 172.21.15.152:0/4047333504 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70301f4fc0 con 0x7f70300b9880 2024-04-20T11:36:21.061 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.062+0000 7f702d600640 1 -- 172.21.15.152:0/4047333504 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 22) ==== 91601+0+0 (secure 0 0 0) 0x7f702801c070 con 0x7f70300b9880 2024-04-20T11:36:21.062 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.063+0000 7f702d600640 1 --2- 172.21.15.152:0/4047333504 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f701006cf50 0x7f701006f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:21.062 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.063+0000 7f702d600640 1 -- 172.21.15.152:0/4047333504 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5709+0+0 (secure 0 0 0) 0x7f702808bca0 con 0x7f70300b9880 2024-04-20T11:36:21.062 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.063+0000 7f703aba0740 1 Processor -- start 2024-04-20T11:36:21.063 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.063+0000 7f703aba0740 1 -- start start 2024-04-20T11:36:21.063 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.063+0000 7f7035400640 1 --2- 172.21.15.152:0/4047333504 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f701006cf50 0x7f701006f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:21.065 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.064+0000 7f703aba0740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5597d7dc2670 0x5597d7d79ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:21.065 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.064+0000 7f703aba0740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5597d7d786f0 0x5597d7d7a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:21.065 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.064+0000 7f703aba0740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5597d7d74b90 0x5597d7da4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:21.065 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.064+0000 7f703aba0740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x5597d7ce09d0 con 0x5597d7d74b90 2024-04-20T11:36:21.065 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.064+0000 7f703aba0740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5597d7c87270 con 0x5597d7dc2670 2024-04-20T11:36:21.065 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.064+0000 7f703aba0740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x5597d7cdc990 con 0x5597d7d786f0 2024-04-20T11:36:21.066 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.064+0000 7f7035e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5597d7d74b90 0x5597d7da4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:21.066 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.065+0000 7f7035e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5597d7d74b90 0x5597d7da4980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:40316/0 (socket says 172.21.15.152:40316) 2024-04-20T11:36:21.066 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.065+0000 7f7035e00640 1 -- 172.21.15.152:0/1058228619 learned_addr learned my addr 172.21.15.152:0/1058228619 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:21.066 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.065+0000 7f7034a00640 1 --2- 172.21.15.152:0/1058228619 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5597d7d786f0 0x5597d7d7a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:21.066 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.065+0000 7f7035400640 1 --2- 172.21.15.152:0/1058228619 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5597d7dc2670 0x5597d7d79ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:21.066 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.065+0000 7f7035e00640 1 -- 172.21.15.152:0/1058228619 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5597d7d786f0 msgr2=0x5597d7d7a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:21.066 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.065+0000 7f7035e00640 1 --2- 172.21.15.152:0/1058228619 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5597d7d786f0 0x5597d7d7a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:21.066 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.065+0000 7f7035e00640 1 -- 172.21.15.152:0/1058228619 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5597d7dc2670 msgr2=0x5597d7d79ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:21.066 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.065+0000 7f7035e00640 1 --2- 172.21.15.152:0/1058228619 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5597d7dc2670 0x5597d7d79ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:21.066 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.065+0000 7f7035e00640 1 -- 172.21.15.152:0/1058228619 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597d79f32d0 con 0x5597d7d74b90 2024-04-20T11:36:21.066 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.065+0000 7f7035400640 1 --2- 172.21.15.152:0/1058228619 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5597d7dc2670 0x5597d7d79ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:36:21.067 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.065+0000 7f7035400640 1 --2- 172.21.15.152:0/4047333504 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f701006cf50 0x7f701006f410 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f70300b70e0 tx=0x7f7024009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:21.067 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.065+0000 7f7035e00640 1 --2- 172.21.15.152:0/1058228619 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5597d7d74b90 0x5597d7da4980 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7f7028055540 tx=0x7f70280590e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=546c7e7e6c18a943 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:21.067 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.065+0000 7f7021c00640 1 -- 172.21.15.152:0/1058228619 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7028061cc0 con 0x5597d7d74b90 2024-04-20T11:36:21.067 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.065+0000 7f7021c00640 1 -- 172.21.15.152:0/1058228619 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7028055750 con 0x5597d7d74b90 2024-04-20T11:36:21.067 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.066+0000 7f7021c00640 1 -- 172.21.15.152:0/1058228619 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f702805d220 con 0x5597d7d74b90 2024-04-20T11:36:21.071 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.066+0000 7f703aba0740 1 -- 172.21.15.152:0/1058228619 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5597d7d74b90 msgr2=0x5597d7da4980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:21.071 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.066+0000 7f703aba0740 1 --2- 172.21.15.152:0/1058228619 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5597d7d74b90 0x5597d7da4980 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7f7028055540 tx=0x7f70280590e0 comp rx=0 tx=0).stop 2024-04-20T11:36:21.071 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.066+0000 7f703aba0740 1 -- 172.21.15.152:0/1058228619 shutdown_connections 2024-04-20T11:36:21.071 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.066+0000 7f703aba0740 1 --2- 172.21.15.152:0/1058228619 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5597d7d74b90 0x5597d7da4980 unknown :-1 s=CLOSED pgs=1995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:21.071 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.066+0000 7f703aba0740 1 --2- 172.21.15.152:0/1058228619 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5597d7d786f0 0x5597d7d7a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:21.071 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.066+0000 7f703aba0740 1 --2- 172.21.15.152:0/1058228619 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5597d7dc2670 0x5597d7d79ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:21.071 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.066+0000 7f703aba0740 1 -- 172.21.15.152:0/1058228619 >> 172.21.15.152:0/1058228619 conn(0x5597d7d76e00 msgr2=0x5597d7dbf980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:21.071 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.066+0000 7f703aba0740 1 -- 172.21.15.152:0/1058228619 shutdown_connections 2024-04-20T11:36:21.071 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.066+0000 7f703aba0740 1 -- 172.21.15.152:0/1058228619 wait complete. 2024-04-20T11:36:21.071 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.067+0000 7f703aba0740 1 -- start start 2024-04-20T11:36:21.071 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.067+0000 7f703aba0740 1 Processor -- start 2024-04-20T11:36:21.071 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.068+0000 7f703aba0740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5597d7d786f0 0x5597d7dc5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:21.071 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.068+0000 7f703aba0740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5597d7dc2670 0x5597d7dc3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:21.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.068+0000 7f703aba0740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5597d7d97800 0x5597d7dbe950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:21.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.068+0000 7f703aba0740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x5597d7d78c00 con 0x5597d7dc2670 2024-04-20T11:36:21.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.068+0000 7f703aba0740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5597d7c9a5e0 con 0x5597d7d786f0 2024-04-20T11:36:21.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.068+0000 7f703aba0740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x5597d7d78ee0 con 0x5597d7d97800 2024-04-20T11:36:21.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.068+0000 7f703aba0740 10 client.0 _ll_register_callbacks cb 0x5597d79f2ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:36:21.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.069+0000 7f7034a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5597d7dc2670 0x5597d7dc3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:21.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.069+0000 7f7034a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5597d7dc2670 0x5597d7dc3300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:40330/0 (socket says 172.21.15.152:40330) 2024-04-20T11:36:21.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.069+0000 7f7034a00640 1 -- 172.21.15.152:0/2323294127 learned_addr learned my addr 172.21.15.152:0/2323294127 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:21.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.069+0000 7f7035400640 1 --2- 172.21.15.152:0/2323294127 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5597d7d786f0 0x5597d7dc5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:21.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.069+0000 7f7035e00640 1 --2- 172.21.15.152:0/2323294127 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5597d7d97800 0x5597d7dbe950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:21.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.069+0000 7f7034a00640 1 -- 172.21.15.152:0/2323294127 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5597d7d97800 msgr2=0x5597d7dbe950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:21.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.069+0000 7f7034a00640 1 --2- 172.21.15.152:0/2323294127 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5597d7d97800 0x5597d7dbe950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:21.073 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.069+0000 7f7034a00640 1 -- 172.21.15.152:0/2323294127 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5597d7d786f0 msgr2=0x5597d7dc5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:21.073 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.069+0000 7f7034a00640 1 --2- 172.21.15.152:0/2323294127 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5597d7d786f0 0x5597d7dc5fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:21.073 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.069+0000 7f7034a00640 1 -- 172.21.15.152:0/2323294127 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597d7abd6b0 con 0x5597d7dc2670 2024-04-20T11:36:21.073 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.069+0000 7f7035400640 1 --2- 172.21.15.152:0/2323294127 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5597d7d786f0 0x5597d7dc5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:36:21.073 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.069+0000 7f7034a00640 1 --2- 172.21.15.152:0/2323294127 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5597d7dc2670 0x5597d7dc3300 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7f701c034040 tx=0x7f701c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2800fc3d115cccf7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:21.073 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.069+0000 7f7016000640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-04-20T11:36:21.073 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.070+0000 7f7016000640 1 -- 172.21.15.152:0/2323294127 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f701c005e60 con 0x5597d7dc2670 2024-04-20T11:36:21.073 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.070+0000 7f703aba0740 1 -- 172.21.15.152:0/2323294127 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7028055750 con 0x5597d7dc2670 2024-04-20T11:36:21.073 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.070+0000 7f7016000640 1 -- 172.21.15.152:0/2323294127 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(142..142 src has 1..142) ==== 5709+0+0 (secure 0 0 0) 0x7f701c046af0 con 0x5597d7dc2670 2024-04-20T11:36:21.073 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.070+0000 7f7016000640 1 -- 172.21.15.152:0/2323294127 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=96) ==== 24+0+0 (secure 0 0 0) 0x7f701c04d050 con 0x5597d7dc2670 2024-04-20T11:36:21.073 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.070+0000 7f703aba0740 10 client.9660 fetch_fsmap learned FSMap version 96 2024-04-20T11:36:21.074 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.070+0000 7f703aba0740 1 -- 172.21.15.152:0/2323294127 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=96,monmap=2+}) -- 0x5597d7dc6db0 con 0x5597d7dc2670 2024-04-20T11:36:21.074 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.070+0000 7f7016000640 1 -- 172.21.15.152:0/2323294127 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7028055750 con 0x5597d7dc2670 2024-04-20T11:36:21.074 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.070+0000 7f7016000640 1 -- 172.21.15.152:0/2323294127 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 96) ==== 3495+0+0 (secure 0 0 0) 0x7f701c050020 con 0x5597d7dc2670 2024-04-20T11:36:21.074 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.070+0000 7f703aba0740 10 client.9660 fetch_fsmap finished waiting for FSMap version 96 2024-04-20T11:36:21.074 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.070+0000 7f703aba0740 10 client.9660 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-20T11:36:21.074 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.070+0000 7f703aba0740 20 client.9660 populate_metadata read hostname 'smithi152' 2024-04-20T11:36:21.074 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.070+0000 7f703aba0740 1 --2- 172.21.15.152:0/2323294127 >> [v2:172.21.15.175:6833/189289100,v1:172.21.15.175:6835/189289100] conn(0x5597d7d9bdd0 0x5597d7d9c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:21.074 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.070+0000 7f703aba0740 4 client.9660 mds_command: new command op to 9501 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:36:21.074 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.070+0000 7f703aba0740 1 -- 172.21.15.152:0/2323294127 --> [v2:172.21.15.175:6833/189289100,v1:172.21.15.175:6835/189289100] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5597d7cee3f0 con 0x5597d7d9bdd0 2024-04-20T11:36:21.074 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.071+0000 7f7035400640 1 --2- 172.21.15.152:0/2323294127 >> [v2:172.21.15.175:6833/189289100,v1:172.21.15.175:6835/189289100] conn(0x5597d7d9bdd0 0x5597d7d9c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:21.076 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.072+0000 7f7035400640 1 --2- 172.21.15.152:0/2323294127 >> [v2:172.21.15.175:6833/189289100,v1:172.21.15.175:6835/189289100] conn(0x5597d7d9bdd0 0x5597d7d9c1b0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c60dc0ba49c1b1 server_cookie=714ee4d59fe922af in_seq=0 out_seq=0 2024-04-20T11:36:21.076 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.072+0000 7f7016000640 10 client.9660 ms_handle_connect on v2:172.21.15.175:6833/189289100 2024-04-20T11:36:21.076 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.074+0000 7f7016000640 1 -- 172.21.15.152:0/2323294127 <== mds.0 v2:172.21.15.175:6833/189289100 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5597d7cee3f0 con 0x5597d7d9bdd0 2024-04-20T11:36:21.076 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.074+0000 7f7016000640 10 client.9660 handle_command_reply: tid=0 2024-04-20T11:36:21.076 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.074+0000 7f703aba0740 1 client.9660 shutdown 2024-04-20T11:36:21.076 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.075+0000 7f703aba0740 1 -- 172.21.15.152:0/2323294127 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5597d7dc2670 msgr2=0x5597d7dc3300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:21.077 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.075+0000 7f703aba0740 1 --2- 172.21.15.152:0/2323294127 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5597d7dc2670 0x5597d7dc3300 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7f701c034040 tx=0x7f701c0046c0 comp rx=0 tx=0).stop 2024-04-20T11:36:21.077 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.075+0000 7f703aba0740 1 -- 172.21.15.152:0/2323294127 shutdown_connections 2024-04-20T11:36:21.077 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.075+0000 7f703aba0740 1 --2- 172.21.15.152:0/2323294127 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5597d7d97800 0x5597d7dbe950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:21.077 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.075+0000 7f703aba0740 1 --2- 172.21.15.152:0/2323294127 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5597d7dc2670 0x5597d7dc3300 unknown :-1 s=CLOSED pgs=1996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:21.077 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.075+0000 7f703aba0740 1 --2- 172.21.15.152:0/2323294127 >> [v2:172.21.15.175:6833/189289100,v1:172.21.15.175:6835/189289100] conn(0x5597d7d9bdd0 0x5597d7d9c1b0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:21.077 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.075+0000 7f703aba0740 1 --2- 172.21.15.152:0/2323294127 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5597d7d786f0 0x5597d7dc5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:21.077 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.075+0000 7f7016000640 0 client.9660 ms_handle_reset on v2:172.21.15.175:6833/189289100 2024-04-20T11:36:21.077 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.075+0000 7f703aba0740 1 -- 172.21.15.152:0/2323294127 >> 172.21.15.152:0/2323294127 conn(0x5597d7d74b90 msgr2=0x5597d7d7a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:21.077 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.075+0000 7f703aba0740 1 -- 172.21.15.152:0/2323294127 shutdown_connections 2024-04-20T11:36:21.077 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.075+0000 7f703aba0740 1 -- 172.21.15.152:0/2323294127 wait complete. 2024-04-20T11:36:21.077 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.075+0000 7f703aba0740 20 client.9660 trim_cache size 0 max 16384 2024-04-20T11:36:21.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.094+0000 7f703aba0740 1 Processor -- start 2024-04-20T11:36:21.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.095+0000 7f703aba0740 1 -- start start 2024-04-20T11:36:21.096 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.095+0000 7f703aba0740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5597d7d786f0 0x5597d7d75800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:21.096 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.095+0000 7f703aba0740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5597d7d97800 0x5597d7dc3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:21.096 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.095+0000 7f703aba0740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5597d7dc2670 0x5597d7d79950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:21.096 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.095+0000 7f703aba0740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f70302004a0 con 0x5597d7dc2670 2024-04-20T11:36:21.096 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.095+0000 7f703aba0740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f70300cc170 con 0x5597d7d786f0 2024-04-20T11:36:21.096 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.095+0000 7f703aba0740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f70301ffe50 con 0x5597d7d97800 2024-04-20T11:36:21.096 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.096+0000 7f7035e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5597d7dc2670 0x5597d7d79950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:21.097 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.096+0000 7f7035e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5597d7dc2670 0x5597d7d79950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:40342/0 (socket says 172.21.15.152:40342) 2024-04-20T11:36:21.097 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.096+0000 7f7035e00640 1 -- 172.21.15.152:0/110774966 learned_addr learned my addr 172.21.15.152:0/110774966 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:21.097 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.096+0000 7f7034a00640 1 --2- 172.21.15.152:0/110774966 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5597d7d97800 0x5597d7dc3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:21.097 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.096+0000 7f7035e00640 1 -- 172.21.15.152:0/110774966 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5597d7d97800 msgr2=0x5597d7dc3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:21.097 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.096+0000 7f7035e00640 1 --2- 172.21.15.152:0/110774966 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5597d7d97800 0x5597d7dc3110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:21.097 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.096+0000 7f7035e00640 1 -- 172.21.15.152:0/110774966 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5597d7d786f0 msgr2=0x5597d7d75800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:21.097 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.096+0000 7f7035e00640 1 --2- 172.21.15.152:0/110774966 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5597d7d786f0 0x5597d7d75800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:21.097 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.096+0000 7f7035e00640 1 -- 172.21.15.152:0/110774966 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597d7d7a470 con 0x5597d7dc2670 2024-04-20T11:36:21.097 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.096+0000 7f7035e00640 1 --2- 172.21.15.152:0/110774966 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5597d7dc2670 0x5597d7d79950 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7f7028058de0 tx=0x7f7028005000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=effb826b92747ddf server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:21.097 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.097+0000 7f7021200640 1 -- 172.21.15.152:0/110774966 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7028055750 con 0x5597d7dc2670 2024-04-20T11:36:21.097 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.097+0000 7f7021200640 1 -- 172.21.15.152:0/110774966 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f702805d040 con 0x5597d7dc2670 2024-04-20T11:36:21.097 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.097+0000 7f7021200640 1 -- 172.21.15.152:0/110774966 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70280558f0 con 0x5597d7dc2670 2024-04-20T11:36:21.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.097+0000 7f703aba0740 1 -- 172.21.15.152:0/110774966 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5597d7dc2670 msgr2=0x5597d7d79950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:21.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.097+0000 7f703aba0740 1 --2- 172.21.15.152:0/110774966 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5597d7dc2670 0x5597d7d79950 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7f7028058de0 tx=0x7f7028005000 comp rx=0 tx=0).stop 2024-04-20T11:36:21.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.097+0000 7f703aba0740 1 -- 172.21.15.152:0/110774966 shutdown_connections 2024-04-20T11:36:21.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.097+0000 7f703aba0740 1 --2- 172.21.15.152:0/110774966 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5597d7dc2670 0x5597d7d79950 unknown :-1 s=CLOSED pgs=1997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:21.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.097+0000 7f703aba0740 1 --2- 172.21.15.152:0/110774966 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5597d7d97800 0x5597d7dc3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:21.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.097+0000 7f703aba0740 1 --2- 172.21.15.152:0/110774966 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5597d7d786f0 0x5597d7d75800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:21.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.097+0000 7f703aba0740 1 -- 172.21.15.152:0/110774966 >> 172.21.15.152:0/110774966 conn(0x5597d7d74b90 msgr2=0x5597d7d9de80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:21.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.097+0000 7f703aba0740 1 -- 172.21.15.152:0/110774966 shutdown_connections 2024-04-20T11:36:21.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.097+0000 7f703aba0740 1 -- 172.21.15.152:0/110774966 wait complete. 2024-04-20T11:36:21.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.098+0000 7f703aba0740 1 -- start start 2024-04-20T11:36:21.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.098+0000 7f703aba0740 1 Processor -- start 2024-04-20T11:36:21.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.099+0000 7f703aba0740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5597d7d786f0 0x5597d7d9d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:21.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.099+0000 7f703aba0740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5597d7d97800 0x5597d7d7a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:21.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.099+0000 7f703aba0740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5597d7dc2670 0x5597d7d7acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:21.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.099+0000 7f703aba0740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x5597d7dc7960 con 0x5597d7dc2670 2024-04-20T11:36:21.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.099+0000 7f703aba0740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5597d7dc7090 con 0x5597d7d786f0 2024-04-20T11:36:21.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.099+0000 7f703aba0740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x5597d7d9ecb0 con 0x5597d7d97800 2024-04-20T11:36:21.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.099+0000 7f703aba0740 10 client.0 _ll_register_callbacks cb 0x7f70180036f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:36:21.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.100+0000 7f7035e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5597d7dc2670 0x5597d7d7acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:21.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.100+0000 7f7035e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5597d7dc2670 0x5597d7d7acf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:40354/0 (socket says 172.21.15.152:40354) 2024-04-20T11:36:21.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.100+0000 7f7035e00640 1 -- 172.21.15.152:0/2929947677 learned_addr learned my addr 172.21.15.152:0/2929947677 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:21.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.100+0000 7f7035e00640 1 -- 172.21.15.152:0/2929947677 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5597d7d97800 msgr2=0x5597d7d7a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:21.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.100+0000 7f7035e00640 1 --2- 172.21.15.152:0/2929947677 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5597d7d97800 0x5597d7d7a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:21.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.100+0000 7f7035e00640 1 -- 172.21.15.152:0/2929947677 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5597d7d786f0 msgr2=0x5597d7d9d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:21.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.100+0000 7f7035400640 1 --2- 172.21.15.152:0/2929947677 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5597d7d786f0 0x5597d7d9d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:21.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.100+0000 7f7035e00640 1 --2- 172.21.15.152:0/2929947677 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5597d7d786f0 0x5597d7d9d220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:21.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.100+0000 7f7035e00640 1 -- 172.21.15.152:0/2929947677 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597d7da8d60 con 0x5597d7dc2670 2024-04-20T11:36:21.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.101+0000 7f7035e00640 1 --2- 172.21.15.152:0/2929947677 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5597d7dc2670 0x5597d7d7acf0 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7f7028059990 tx=0x7f7028005070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88c400c6550fe310 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:21.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.101+0000 7f7016000640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-04-20T11:36:21.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.101+0000 7f7016000640 1 -- 172.21.15.152:0/2929947677 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f702807d060 con 0x5597d7dc2670 2024-04-20T11:36:21.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.101+0000 7f7016000640 1 -- 172.21.15.152:0/2929947677 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(142..142 src has 1..142) ==== 5709+0+0 (secure 0 0 0) 0x7f7028058530 con 0x5597d7dc2670 2024-04-20T11:36:21.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.101+0000 7f703aba0740 1 -- 172.21.15.152:0/2929947677 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f702805d040 con 0x5597d7dc2670 2024-04-20T11:36:21.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.101+0000 7f7016000640 1 -- 172.21.15.152:0/2929947677 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=96) ==== 24+0+0 (secure 0 0 0) 0x7f7028058730 con 0x5597d7dc2670 2024-04-20T11:36:21.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.101+0000 7f703aba0740 10 client.9666 fetch_fsmap learned FSMap version 96 2024-04-20T11:36:21.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.101+0000 7f703aba0740 1 -- 172.21.15.152:0/2929947677 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=96,monmap=2+}) -- 0x5597d7e23ee0 con 0x5597d7dc2670 2024-04-20T11:36:21.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.101+0000 7f7016000640 1 -- 172.21.15.152:0/2929947677 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f702805d040 con 0x5597d7dc2670 2024-04-20T11:36:21.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.101+0000 7f7016000640 1 -- 172.21.15.152:0/2929947677 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 96) ==== 3495+0+0 (secure 0 0 0) 0x7f702806daf0 con 0x5597d7dc2670 2024-04-20T11:36:21.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.101+0000 7f703aba0740 10 client.9666 fetch_fsmap finished waiting for FSMap version 96 2024-04-20T11:36:21.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.101+0000 7f703aba0740 10 client.9666 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-20T11:36:21.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.101+0000 7f703aba0740 20 client.9666 populate_metadata read hostname 'smithi152' 2024-04-20T11:36:21.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.101+0000 7f703aba0740 1 --2- 172.21.15.152:0/2929947677 >> [v2:172.21.15.175:6833/189289100,v1:172.21.15.175:6835/189289100] conn(0x5597d7e235e0 0x5597d7dd9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:21.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.101+0000 7f703aba0740 4 client.9666 mds_command: new command op to 9501 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-20T11:36:21.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.101+0000 7f703aba0740 1 -- 172.21.15.152:0/2929947677 --> [v2:172.21.15.175:6833/189289100,v1:172.21.15.175:6835/189289100] -- command(tid 0: {"prefix": "status"}) -- 0x5597d7ce8260 con 0x5597d7e235e0 2024-04-20T11:36:21.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.102+0000 7f7035400640 1 --2- 172.21.15.152:0/2929947677 >> [v2:172.21.15.175:6833/189289100,v1:172.21.15.175:6835/189289100] conn(0x5597d7e235e0 0x5597d7dd9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:21.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.103+0000 7f7035400640 1 --2- 172.21.15.152:0/2929947677 >> [v2:172.21.15.175:6833/189289100,v1:172.21.15.175:6835/189289100] conn(0x5597d7e235e0 0x5597d7dd9740 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5b1b87cb6cab6d2 server_cookie=f9a6de808e1be4dc in_seq=0 out_seq=0 2024-04-20T11:36:21.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.103+0000 7f7016000640 10 client.9666 ms_handle_connect on v2:172.21.15.175:6833/189289100 2024-04-20T11:36:21.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.104+0000 7f7016000640 1 -- 172.21.15.152:0/2929947677 <== mds.0 v2:172.21.15.175:6833/189289100 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x5597d7ce8260 con 0x5597d7e235e0 2024-04-20T11:36:21.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.104+0000 7f7016000640 10 client.9666 handle_command_reply: tid=0 2024-04-20T11:36:21.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.104+0000 7f703aba0740 1 client.9666 shutdown 2024-04-20T11:36:21.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.104+0000 7f703aba0740 1 -- 172.21.15.152:0/2929947677 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5597d7dc2670 msgr2=0x5597d7d7acf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:21.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.104+0000 7f703aba0740 1 --2- 172.21.15.152:0/2929947677 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5597d7dc2670 0x5597d7d7acf0 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7f7028059990 tx=0x7f7028005070 comp rx=0 tx=0).stop 2024-04-20T11:36:21.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.104+0000 7f703aba0740 1 -- 172.21.15.152:0/2929947677 shutdown_connections 2024-04-20T11:36:21.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.104+0000 7f703aba0740 1 --2- 172.21.15.152:0/2929947677 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5597d7dc2670 0x5597d7d7acf0 unknown :-1 s=CLOSED pgs=1998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:21.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.104+0000 7f703aba0740 1 --2- 172.21.15.152:0/2929947677 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5597d7d97800 0x5597d7d7a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:21.107 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.104+0000 7f703aba0740 1 --2- 172.21.15.152:0/2929947677 >> [v2:172.21.15.175:6833/189289100,v1:172.21.15.175:6835/189289100] conn(0x5597d7e235e0 0x5597d7dd9740 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:21.107 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.104+0000 7f703aba0740 1 --2- 172.21.15.152:0/2929947677 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5597d7d786f0 0x5597d7d9d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:21.107 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.104+0000 7f7016000640 0 client.9666 ms_handle_reset on v2:172.21.15.175:6833/189289100 2024-04-20T11:36:21.107 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.104+0000 7f703aba0740 1 -- 172.21.15.152:0/2929947677 >> 172.21.15.152:0/2929947677 conn(0x5597d7d74b90 msgr2=0x5597d7d79ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:21.107 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.104+0000 7f703aba0740 1 -- 172.21.15.152:0/2929947677 shutdown_connections 2024-04-20T11:36:21.107 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.104+0000 7f703aba0740 1 -- 172.21.15.152:0/2929947677 wait complete. 2024-04-20T11:36:21.107 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.104+0000 7f703aba0740 20 client.9666 trim_cache size 0 max 16384 2024-04-20T11:36:21.107 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-04-20T11:36:21.107 INFO:teuthology.orchestra.run.smithi152.stdout: "cluster_fsid": "c8a72131-0d4f-42c7-892f-3f3af4d2bd3d", 2024-04-20T11:36:21.107 INFO:teuthology.orchestra.run.smithi152.stdout: "whoami": 0, 2024-04-20T11:36:21.108 INFO:teuthology.orchestra.run.smithi152.stdout: "id": 9501, 2024-04-20T11:36:21.108 INFO:teuthology.orchestra.run.smithi152.stdout: "want_state": "up:active", 2024-04-20T11:36:21.108 INFO:teuthology.orchestra.run.smithi152.stdout: "state": "up:active", 2024-04-20T11:36:21.108 INFO:teuthology.orchestra.run.smithi152.stdout: "fs_name": "cephfs", 2024-04-20T11:36:21.108 INFO:teuthology.orchestra.run.smithi152.stdout: "rank_uptime": 11.615321482000001, 2024-04-20T11:36:21.108 INFO:teuthology.orchestra.run.smithi152.stdout: "mdsmap_epoch": 96, 2024-04-20T11:36:21.108 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_epoch": 142, 2024-04-20T11:36:21.108 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_epoch_barrier": 142, 2024-04-20T11:36:21.108 INFO:teuthology.orchestra.run.smithi152.stdout: "uptime": 17.068508895000001 2024-04-20T11:36:21.108 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-04-20T11:36:21.109 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.105+0000 7f7036800640 1 -- 172.21.15.152:0/4047333504 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f701006cf50 msgr2=0x7f701006f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:21.109 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.105+0000 7f7036800640 1 --2- 172.21.15.152:0/4047333504 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f701006cf50 0x7f701006f410 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f70300b70e0 tx=0x7f7024009040 comp rx=0 tx=0).stop 2024-04-20T11:36:21.109 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.105+0000 7f7036800640 1 -- 172.21.15.152:0/4047333504 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f70300b9880 msgr2=0x7f70300b26e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:21.109 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.105+0000 7f7036800640 1 --2- 172.21.15.152:0/4047333504 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f70300b9880 0x7f70300b26e0 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7f70300d2120 tx=0x7f702800e970 comp rx=0 tx=0).stop 2024-04-20T11:36:21.109 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.105+0000 7f7036800640 1 -- 172.21.15.152:0/4047333504 shutdown_connections 2024-04-20T11:36:21.109 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.105+0000 7f7036800640 1 --2- 172.21.15.152:0/4047333504 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f701006cf50 0x7f701006f410 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:21.109 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.105+0000 7f7036800640 1 --2- 172.21.15.152:0/4047333504 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f70300b9880 0x7f70300b26e0 unknown :-1 s=CLOSED pgs=1994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:21.109 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.105+0000 7f7036800640 1 --2- 172.21.15.152:0/4047333504 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f70300b9f90 0x7f7030155ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:21.110 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.105+0000 7f7036800640 1 --2- 172.21.15.152:0/4047333504 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7018004800 0x7f70300ccb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:21.110 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.105+0000 7f7036800640 1 -- 172.21.15.152:0/4047333504 >> 172.21.15.152:0/4047333504 conn(0x7f703008bfd0 msgr2=0x7f70300c8240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:21.110 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.105+0000 7f7036800640 1 -- 172.21.15.152:0/4047333504 shutdown_connections 2024-04-20T11:36:21.110 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.105+0000 7f7036800640 1 -- 172.21.15.152:0/4047333504 wait complete. 2024-04-20T11:36:21.121 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-20T11:36:21.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.314+0000 7f2acca00640 1 Processor -- start 2024-04-20T11:36:21.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.314+0000 7f2acca00640 1 -- start start 2024-04-20T11:36:21.315 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.316+0000 7f2acca00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2ac815d1e0 0x7f2ac8160b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:21.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.316+0000 7f2acca00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f2ac815d5c0 0x7f2ac80cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:21.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.316+0000 7f2acca00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ac80ce8c0 con 0x7f2ac8071cd0 2024-04-20T11:36:21.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.316+0000 7f2acca00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f2ac81600b0 con 0x7f2ac815d5c0 2024-04-20T11:36:21.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.316+0000 7f2acca00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2ac8160390 con 0x7f2ac815d1e0 2024-04-20T11:36:21.318 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.317+0000 7f2ac6800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2ac815d1e0 0x7f2ac8160b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:21.318 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.317+0000 7f2ac6800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2ac815d1e0 0x7f2ac8160b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.152:36396/0 (socket says 172.21.15.152:36396) 2024-04-20T11:36:21.318 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.317+0000 7f2ac6800640 1 -- 172.21.15.152:0/3985280554 learned_addr learned my addr 172.21.15.152:0/3985280554 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:21.318 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.317+0000 7f2abfe00640 1 --2- 172.21.15.152:0/3985280554 >> v2:172.21.15.175:3300/0 conn(0x7f2ac815d5c0 0x7f2ac80cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:21.318 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.317+0000 7f2ac5e00640 1 -- 172.21.15.152:0/3985280554 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 398514216 0 0) 0x7f2ac80ce8c0 con 0x7f2ac8071cd0 2024-04-20T11:36:21.318 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.317+0000 7f2ac5e00640 1 -- 172.21.15.152:0/3985280554 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2aa8003520 con 0x7f2ac8071cd0 2024-04-20T11:36:21.318 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.318+0000 7f2ac6800640 1 -- 172.21.15.152:0/3985280554 >> v2:172.21.15.175:3300/0 conn(0x7f2ac815d5c0 msgr2=0x7f2ac80cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:21.318 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.318+0000 7f2ac6800640 1 --2- 172.21.15.152:0/3985280554 >> v2:172.21.15.175:3300/0 conn(0x7f2ac815d5c0 0x7f2ac80cdfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:21.319 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.318+0000 7f2ac6800640 1 -- 172.21.15.152:0/3985280554 >> v1:172.21.15.175:6789/0 conn(0x7f2ac8071cd0 legacy=0x7f2ac80ccd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:21.319 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.318+0000 7f2ac6800640 1 -- 172.21.15.152:0/3985280554 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ac8160670 con 0x7f2ac815d1e0 2024-04-20T11:36:21.319 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.319+0000 7f2ac6800640 1 --2- 172.21.15.152:0/3985280554 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2ac815d1e0 0x7f2ac8160b90 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2ab8009870 tx=0x7f2ab802b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae211a7ec62323de server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:21.319 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.320+0000 7f2ac5e00640 1 -- 172.21.15.152:0/3985280554 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ab8034070 con 0x7f2ac815d1e0 2024-04-20T11:36:21.320 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.320+0000 7f2ac5e00640 1 -- 172.21.15.152:0/3985280554 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ab8030840 con 0x7f2ac815d1e0 2024-04-20T11:36:21.320 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.320+0000 7f2ac5e00640 1 -- 172.21.15.152:0/3985280554 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ab8030c10 con 0x7f2ac815d1e0 2024-04-20T11:36:21.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.321+0000 7f2acca00640 1 -- 172.21.15.152:0/3985280554 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2ac815d1e0 msgr2=0x7f2ac8160b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:21.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.321+0000 7f2acca00640 1 --2- 172.21.15.152:0/3985280554 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2ac815d1e0 0x7f2ac8160b90 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2ab8009870 tx=0x7f2ab802b980 comp rx=0 tx=0).stop 2024-04-20T11:36:21.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.321+0000 7f2acca00640 1 -- 172.21.15.152:0/3985280554 shutdown_connections 2024-04-20T11:36:21.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.321+0000 7f2acca00640 1 --2- 172.21.15.152:0/3985280554 >> v2:172.21.15.175:3300/0 conn(0x7f2ac815d5c0 0x7f2ac80cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:21.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.321+0000 7f2acca00640 1 --2- 172.21.15.152:0/3985280554 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2ac815d1e0 0x7f2ac8160b90 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:21.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.321+0000 7f2acca00640 1 -- 172.21.15.152:0/3985280554 >> 172.21.15.152:0/3985280554 conn(0x7f2ac8067ae0 msgr2=0x7f2ac806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:21.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.321+0000 7f2acca00640 1 -- 172.21.15.152:0/3985280554 shutdown_connections 2024-04-20T11:36:21.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.321+0000 7f2acca00640 1 -- 172.21.15.152:0/3985280554 wait complete. 2024-04-20T11:36:21.323 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.323+0000 7f2acca00640 1 Processor -- start 2024-04-20T11:36:21.323 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.323+0000 7f2acca00640 1 -- start start 2024-04-20T11:36:21.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.324+0000 7f2acca00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2ac8071cd0 0x7f2ac8154c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:21.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.324+0000 7f2acca00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ac815d1e0 0x7f2ac80ae290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:21.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.324+0000 7f2acca00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ac815d5c0 0x7f2ac80ae7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:21.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.324+0000 7f2acca00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac80ce6a0 con 0x7f2ac815d5c0 2024-04-20T11:36:21.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.324+0000 7f2acca00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac80ce520 con 0x7f2ac815d1e0 2024-04-20T11:36:21.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.324+0000 7f2acca00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2ac8072160 con 0x7f2ac8071cd0 2024-04-20T11:36:21.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.325+0000 7f2ac7200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ac815d5c0 0x7f2ac80ae7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:21.326 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.325+0000 7f2ac7200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ac815d5c0 0x7f2ac80ae7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:40358/0 (socket says 172.21.15.152:40358) 2024-04-20T11:36:21.326 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.325+0000 7f2ac7200640 1 -- 172.21.15.152:0/1165532761 learned_addr learned my addr 172.21.15.152:0/1165532761 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:21.326 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.325+0000 7f2abfe00640 1 --2- 172.21.15.152:0/1165532761 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ac815d1e0 0x7f2ac80ae290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:21.326 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.325+0000 7f2ac6800640 1 --2- 172.21.15.152:0/1165532761 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2ac8071cd0 0x7f2ac8154c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:21.327 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.326+0000 7f2ac7200640 1 -- 172.21.15.152:0/1165532761 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2ac8071cd0 msgr2=0x7f2ac8154c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:21.327 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.326+0000 7f2ac7200640 1 --2- 172.21.15.152:0/1165532761 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2ac8071cd0 0x7f2ac8154c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:21.327 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.326+0000 7f2ac7200640 1 -- 172.21.15.152:0/1165532761 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ac815d1e0 msgr2=0x7f2ac80ae290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:21.327 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.326+0000 7f2ac7200640 1 --2- 172.21.15.152:0/1165532761 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ac815d1e0 0x7f2ac80ae290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:21.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.326+0000 7f2ac7200640 1 -- 172.21.15.152:0/1165532761 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ac80aee30 con 0x7f2ac815d5c0 2024-04-20T11:36:21.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.326+0000 7f2ac7200640 1 --2- 172.21.15.152:0/1165532761 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ac815d5c0 0x7f2ac80ae7d0 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7f2ac000c790 tx=0x7f2ac000cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:21.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.326+0000 7f2abea00640 1 -- 172.21.15.152:0/1165532761 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac0003b70 con 0x7f2ac815d5c0 2024-04-20T11:36:21.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.326+0000 7f2abea00640 1 -- 172.21.15.152:0/1165532761 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ac0003d10 con 0x7f2ac815d5c0 2024-04-20T11:36:21.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.326+0000 7f2abea00640 1 -- 172.21.15.152:0/1165532761 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac000aa50 con 0x7f2ac815d5c0 2024-04-20T11:36:21.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.327+0000 7f2acca00640 1 -- 172.21.15.152:0/1165532761 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ac80af110 con 0x7f2ac815d5c0 2024-04-20T11:36:21.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.327+0000 7f2acca00640 1 -- 172.21.15.152:0/1165532761 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ac815dc80 con 0x7f2ac815d5c0 2024-04-20T11:36:21.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.328+0000 7f2acca00640 1 -- 172.21.15.152:0/1165532761 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ac8075e60 con 0x7f2ac815d5c0 2024-04-20T11:36:21.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.329+0000 7f2abea00640 1 -- 172.21.15.152:0/1165532761 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 22) ==== 91601+0+0 (secure 0 0 0) 0x7f2ac000abf0 con 0x7f2ac815d5c0 2024-04-20T11:36:21.329 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.330+0000 7f2abea00640 1 --2- 172.21.15.152:0/1165532761 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f2aa406cf20 0x7f2aa406f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:21.330 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.330+0000 7f2abea00640 1 -- 172.21.15.152:0/1165532761 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5709+0+0 (secure 0 0 0) 0x7f2ac008c500 con 0x7f2ac815d5c0 2024-04-20T11:36:21.330 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.331+0000 7f2ac6800640 1 --2- 172.21.15.152:0/1165532761 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f2aa406cf20 0x7f2aa406f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:21.331 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.332+0000 7f2ac6800640 1 --2- 172.21.15.152:0/1165532761 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f2aa406cf20 0x7f2aa406f3e0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f2ab8036000 tx=0x7f2ab8008030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:21.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.334+0000 7f2abea00640 1 -- 172.21.15.152:0/1165532761 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f2ac00599f0 con 0x7f2ac815d5c0 2024-04-20T11:36:21.582 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.581+0000 7f2acca00640 1 -- 172.21.15.152:0/1165532761 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f2ac8072830 con 0x7f2ac815d5c0 2024-04-20T11:36:21.582 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.582+0000 7f2abea00640 1 -- 172.21.15.152:0/1165532761 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v96) ==== 53+0+73 (secure 0 0 0) 0x7f2ac005dcf0 con 0x7f2ac815d5c0 2024-04-20T11:36:21.583 INFO:teuthology.orchestra.run.smithi152.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-20T11:36:21.586 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.584+0000 7f2acca00640 1 -- 172.21.15.152:0/1165532761 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f2aa406cf20 msgr2=0x7f2aa406f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:21.586 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.584+0000 7f2acca00640 1 --2- 172.21.15.152:0/1165532761 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f2aa406cf20 0x7f2aa406f3e0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f2ab8036000 tx=0x7f2ab8008030 comp rx=0 tx=0).stop 2024-04-20T11:36:21.586 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.584+0000 7f2acca00640 1 -- 172.21.15.152:0/1165532761 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ac815d5c0 msgr2=0x7f2ac80ae7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:21.586 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.584+0000 7f2acca00640 1 --2- 172.21.15.152:0/1165532761 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ac815d5c0 0x7f2ac80ae7d0 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7f2ac000c790 tx=0x7f2ac000cc50 comp rx=0 tx=0).stop 2024-04-20T11:36:21.586 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.585+0000 7f2acca00640 1 -- 172.21.15.152:0/1165532761 shutdown_connections 2024-04-20T11:36:21.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.585+0000 7f2acca00640 1 --2- 172.21.15.152:0/1165532761 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f2aa406cf20 0x7f2aa406f3e0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:21.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.585+0000 7f2acca00640 1 --2- 172.21.15.152:0/1165532761 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ac815d5c0 0x7f2ac80ae7d0 unknown :-1 s=CLOSED pgs=1999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:21.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.585+0000 7f2acca00640 1 --2- 172.21.15.152:0/1165532761 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ac815d1e0 0x7f2ac80ae290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:21.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.585+0000 7f2acca00640 1 --2- 172.21.15.152:0/1165532761 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2ac8071cd0 0x7f2ac8154c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:21.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.585+0000 7f2acca00640 1 -- 172.21.15.152:0/1165532761 >> 172.21.15.152:0/1165532761 conn(0x7f2ac8067ae0 msgr2=0x7f2ac806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:21.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.585+0000 7f2acca00640 1 -- 172.21.15.152:0/1165532761 shutdown_connections 2024-04-20T11:36:21.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21.585+0000 7f2acca00640 1 -- 172.21.15.152:0/1165532761 wait complete. 2024-04-20T11:36:21.605 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-20T11:36:21.606 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-20T11:36:21.606 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi152.front.sepia.ceph.com 2024-04-20T11:36:21.606 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-20T11:36:21.606 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-20T11:36:21.606 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-20T11:36:21.606 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-20T11:36:21.606 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-20T11:36:21.606 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-20T11:36:21.607 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:36:21.607 DEBUG:teuthology.orchestra.run.smithi152:> ip addr 2024-04-20T11:36:21.659 INFO:teuthology.orchestra.run.smithi152.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-20T11:36:21.659 INFO:teuthology.orchestra.run.smithi152.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-20T11:36:21.659 INFO:teuthology.orchestra.run.smithi152.stdout: inet 127.0.0.1/8 scope host lo 2024-04-20T11:36:21.659 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:36:21.659 INFO:teuthology.orchestra.run.smithi152.stdout: inet6 ::1/128 scope host 2024-04-20T11:36:21.660 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:36:21.660 INFO:teuthology.orchestra.run.smithi152.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T11:36:21.660 INFO:teuthology.orchestra.run.smithi152.stdout: link/ether 0c:c4:7a:da:6c:44 brd ff:ff:ff:ff:ff:ff 2024-04-20T11:36:21.660 INFO:teuthology.orchestra.run.smithi152.stdout: altname enp7s0f0 2024-04-20T11:36:21.660 INFO:teuthology.orchestra.run.smithi152.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T11:36:21.660 INFO:teuthology.orchestra.run.smithi152.stdout: link/ether 0c:c4:7a:da:6c:45 brd ff:ff:ff:ff:ff:ff 2024-04-20T11:36:21.660 INFO:teuthology.orchestra.run.smithi152.stdout: altname enp7s0f1 2024-04-20T11:36:21.660 INFO:teuthology.orchestra.run.smithi152.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T11:36:21.660 INFO:teuthology.orchestra.run.smithi152.stdout: link/ether 0c:c4:7a:88:6d:e2 brd ff:ff:ff:ff:ff:ff 2024-04-20T11:36:21.660 INFO:teuthology.orchestra.run.smithi152.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-20T11:36:21.660 INFO:teuthology.orchestra.run.smithi152.stdout: link/ether 0c:c4:7a:88:6d:e3 brd ff:ff:ff:ff:ff:ff 2024-04-20T11:36:21.660 INFO:teuthology.orchestra.run.smithi152.stdout: inet 172.21.15.152/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-20T11:36:21.660 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft 42225sec preferred_lft 42225sec 2024-04-20T11:36:21.661 INFO:teuthology.orchestra.run.smithi152.stdout: inet6 fe80::ec4:7aff:fe88:6de3/64 scope link 2024-04-20T11:36:21.661 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:36:21.661 INFO:teuthology.orchestra.run.smithi152.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-20T11:36:21.661 INFO:teuthology.orchestra.run.smithi152.stdout: link/ether 06:3e:9a:d3:49:1e brd ff:ff:ff:ff:ff:ff 2024-04-20T11:36:21.661 INFO:teuthology.orchestra.run.smithi152.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-20T11:36:21.661 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:36:21.661 INFO:teuthology.orchestra.run.smithi152.stdout: inet6 fe80::a019:d8ff:fe70:1185/64 scope link 2024-04-20T11:36:21.661 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:36:21.661 INFO:teuthology.orchestra.run.smithi152.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-20T11:36:21.661 INFO:teuthology.orchestra.run.smithi152.stdout: link/ether 16:e5:08:78:20:47 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T11:36:21.661 INFO:teuthology.orchestra.run.smithi152.stdout: inet6 fe80::14e5:8ff:fe78:2047/64 scope link 2024-04-20T11:36:21.661 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:36:21.661 INFO:teuthology.orchestra.run.smithi152.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-20T11:36:21.661 INFO:teuthology.orchestra.run.smithi152.stdout: link/ether 06:3e:9a:d3:49:1e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T11:36:21.662 INFO:teuthology.orchestra.run.smithi152.stdout: inet6 fe80::43e:9aff:fed3:491e/64 scope link 2024-04-20T11:36:21.662 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:36:21.665 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:36:21.665 DEBUG:teuthology.orchestra.run.smithi152:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-20T11:36:21.743 INFO:teuthology.orchestra.run.smithi152.stdout:1 2024-04-20T11:36:21.747 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:36:21.747 DEBUG:teuthology.orchestra.run.smithi152:> route 2024-04-20T11:36:21.799 INFO:teuthology.orchestra.run.smithi152.stdout:Kernel IP routing table 2024-04-20T11:36:21.799 INFO:teuthology.orchestra.run.smithi152.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-20T11:36:21.799 INFO:teuthology.orchestra.run.smithi152.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-20T11:36:21.799 INFO:teuthology.orchestra.run.smithi152.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-20T11:36:21.799 INFO:teuthology.orchestra.run.smithi152.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-20T11:36:21.800 DEBUG:teuthology.orchestra.run.smithi152:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T11:36:21.800 DEBUG:teuthology.orchestra.run.smithi152:> set -e 2024-04-20T11:36:21.800 DEBUG:teuthology.orchestra.run.smithi152:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-20T11:36:21.800 DEBUG:teuthology.orchestra.run.smithi152:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-20T11:36:21.800 DEBUG:teuthology.orchestra.run.smithi152:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-20T11:36:21.800 DEBUG:teuthology.orchestra.run.smithi152:> ') 2024-04-20T11:36:21.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:21 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T11:36:22.003 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T11:36:22.014 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:36:22.014 DEBUG:teuthology.orchestra.run.smithi152:> ip netns list 2024-04-20T11:36:22.063 INFO:teuthology.orchestra.run.smithi152.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-20T11:36:22.063 INFO:teuthology.orchestra.run.smithi152.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-20T11:36:22.063 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-20T11:36:22.063 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:36:22.063 DEBUG:teuthology.orchestra.run.smithi152:> sudo ip link set brx.0 up 2024-04-20T11:36:22.144 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-20T11:36:22.144 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T11:36:22.144 DEBUG:teuthology.orchestra.run.smithi152:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-20T11:36:22.199 INFO:teuthology.orchestra.run.smithi152.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-20T11:36:22.199 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T11:36:22.199 DEBUG:teuthology.orchestra.run.smithi152:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-20T11:36:22.249 DEBUG:teuthology.orchestra.run.smithi152:> sudo modprobe fuse 2024-04-20T11:36:22.324 DEBUG:teuthology.orchestra.run.smithi152:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T11:36:22.377 INFO:teuthology.orchestra.run.smithi152.stdout:/proc 2024-04-20T11:36:22.378 INFO:teuthology.orchestra.run.smithi152.stdout:/sys 2024-04-20T11:36:22.378 INFO:teuthology.orchestra.run.smithi152.stdout:/dev 2024-04-20T11:36:22.378 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/security 2024-04-20T11:36:22.378 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/shm 2024-04-20T11:36:22.378 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/pts 2024-04-20T11:36:22.378 INFO:teuthology.orchestra.run.smithi152.stdout:/run 2024-04-20T11:36:22.378 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/cgroup 2024-04-20T11:36:22.378 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/pstore 2024-04-20T11:36:22.378 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/bpf 2024-04-20T11:36:22.378 INFO:teuthology.orchestra.run.smithi152.stdout:/ 2024-04-20T11:36:22.378 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/selinux 2024-04-20T11:36:22.378 INFO:teuthology.orchestra.run.smithi152.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T11:36:22.379 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/mqueue 2024-04-20T11:36:22.379 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/hugepages 2024-04-20T11:36:22.379 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/debug 2024-04-20T11:36:22.379 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/tracing 2024-04-20T11:36:22.379 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/fuse/connections 2024-04-20T11:36:22.379 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/config 2024-04-20T11:36:22.379 INFO:teuthology.orchestra.run.smithi152.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T11:36:22.379 INFO:teuthology.orchestra.run.smithi152.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T11:36:22.379 INFO:teuthology.orchestra.run.smithi152.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T11:36:22.379 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T11:36:22.379 INFO:teuthology.orchestra.run.smithi152.stdout:/run/user/1000 2024-04-20T11:36:22.379 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph 2024-04-20T11:36:22.379 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph/osd/ceph-0 2024-04-20T11:36:22.379 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph/osd/ceph-1 2024-04-20T11:36:22.380 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph/osd/ceph-2 2024-04-20T11:36:22.380 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph/osd/ceph-3 2024-04-20T11:36:22.380 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns 2024-04-20T11:36:22.380 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T11:36:22.380 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T11:36:22.380 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T11:36:22.380 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T11:36:22.381 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:36:22.381 DEBUG:teuthology.orchestra.run.smithi152:> ls /sys/fs/fuse/connections 2024-04-20T11:36:22.434 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-20T11:36:22.435 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-20T11:36:22.441 DEBUG:teuthology.orchestra.run.smithi152:> sudo modprobe fuse 2024-04-20T11:36:22.511 DEBUG:teuthology.orchestra.run.smithi152:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T11:36:22.550 INFO:teuthology.orchestra.run.smithi152.stdout:/proc 2024-04-20T11:36:22.550 INFO:teuthology.orchestra.run.smithi152.stdout:/sys 2024-04-20T11:36:22.550 INFO:teuthology.orchestra.run.smithi152.stdout:/dev 2024-04-20T11:36:22.550 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/security 2024-04-20T11:36:22.551 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/shm 2024-04-20T11:36:22.551 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/pts 2024-04-20T11:36:22.551 INFO:teuthology.orchestra.run.smithi152.stdout:/run 2024-04-20T11:36:22.551 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/cgroup 2024-04-20T11:36:22.551 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/pstore 2024-04-20T11:36:22.551 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/bpf 2024-04-20T11:36:22.551 INFO:teuthology.orchestra.run.smithi152.stdout:/ 2024-04-20T11:36:22.551 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/selinux 2024-04-20T11:36:22.551 INFO:teuthology.orchestra.run.smithi152.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T11:36:22.551 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/mqueue 2024-04-20T11:36:22.551 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/hugepages 2024-04-20T11:36:22.551 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/debug 2024-04-20T11:36:22.551 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/tracing 2024-04-20T11:36:22.551 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/fuse/connections 2024-04-20T11:36:22.552 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/config 2024-04-20T11:36:22.552 INFO:teuthology.orchestra.run.smithi152.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T11:36:22.552 INFO:teuthology.orchestra.run.smithi152.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T11:36:22.552 INFO:teuthology.orchestra.run.smithi152.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T11:36:22.552 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T11:36:22.552 INFO:teuthology.orchestra.run.smithi152.stdout:/run/user/1000 2024-04-20T11:36:22.552 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph 2024-04-20T11:36:22.552 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph/osd/ceph-0 2024-04-20T11:36:22.552 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph/osd/ceph-1 2024-04-20T11:36:22.552 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph/osd/ceph-2 2024-04-20T11:36:22.552 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph/osd/ceph-3 2024-04-20T11:36:22.552 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns 2024-04-20T11:36:22.552 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T11:36:22.552 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T11:36:22.553 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T11:36:22.553 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T11:36:22.553 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:36:22.554 DEBUG:teuthology.orchestra.run.smithi152:> ls /sys/fs/fuse/connections 2024-04-20T11:36:22.607 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi152.stderr:2024-04-20T11:36:22.607+0000 7fafe0cd3480 -1 init, newargv = 0x560d74f9be00 newargc=15 2024-04-20T11:36:22.608 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi152.stderr:ceph-fuse[68135]: starting ceph client 2024-04-20T11:36:22.619 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi152.stderr:ceph-fuse[68135]: starting fuse 2024-04-20T11:36:22.636 INFO:teuthology.orchestra.run.smithi152.stdout:42 2024-04-20T11:36:22.637 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-04-20T11:36:22.637 DEBUG:teuthology.orchestra.run.smithi152:> sudo stdin-killer -- python3 -c ' 2024-04-20T11:36:22.637 DEBUG:teuthology.orchestra.run.smithi152:> import glob 2024-04-20T11:36:22.637 DEBUG:teuthology.orchestra.run.smithi152:> import re 2024-04-20T11:36:22.637 DEBUG:teuthology.orchestra.run.smithi152:> import os 2024-04-20T11:36:22.637 DEBUG:teuthology.orchestra.run.smithi152:> import subprocess 2024-04-20T11:36:22.637 DEBUG:teuthology.orchestra.run.smithi152:> 2024-04-20T11:36:22.638 DEBUG:teuthology.orchestra.run.smithi152:> def _find_admin_socket(client_name): 2024-04-20T11:36:22.638 DEBUG:teuthology.orchestra.run.smithi152:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-20T11:36:22.638 DEBUG:teuthology.orchestra.run.smithi152:> files = glob.glob(asok_path) 2024-04-20T11:36:22.638 DEBUG:teuthology.orchestra.run.smithi152:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-20T11:36:22.638 DEBUG:teuthology.orchestra.run.smithi152:> 2024-04-20T11:36:22.638 DEBUG:teuthology.orchestra.run.smithi152:> # Given a non-glob path, it better be there 2024-04-20T11:36:22.638 DEBUG:teuthology.orchestra.run.smithi152:> if "*" not in asok_path: 2024-04-20T11:36:22.638 DEBUG:teuthology.orchestra.run.smithi152:> assert(len(files) == 1) 2024-04-20T11:36:22.638 DEBUG:teuthology.orchestra.run.smithi152:> return files[0] 2024-04-20T11:36:22.638 DEBUG:teuthology.orchestra.run.smithi152:> 2024-04-20T11:36:22.639 DEBUG:teuthology.orchestra.run.smithi152:> for f in files: 2024-04-20T11:36:22.639 DEBUG:teuthology.orchestra.run.smithi152:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-20T11:36:22.639 DEBUG:teuthology.orchestra.run.smithi152:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-20T11:36:22.639 DEBUG:teuthology.orchestra.run.smithi152:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-20T11:36:22.639 DEBUG:teuthology.orchestra.run.smithi152:> contents = proc_f.read() 2024-04-20T11:36:22.639 DEBUG:teuthology.orchestra.run.smithi152:> if mountpoint in contents: 2024-04-20T11:36:22.639 DEBUG:teuthology.orchestra.run.smithi152:> return f 2024-04-20T11:36:22.639 DEBUG:teuthology.orchestra.run.smithi152:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-20T11:36:22.639 DEBUG:teuthology.orchestra.run.smithi152:> 2024-04-20T11:36:22.639 DEBUG:teuthology.orchestra.run.smithi152:> print(_find_admin_socket("client.0")) 2024-04-20T11:36:22.639 DEBUG:teuthology.orchestra.run.smithi152:> ' 2024-04-20T11:36:22.818 INFO:teuthology.orchestra.run.smithi152.stdout:/var/run/ceph/ceph-client.0.68135.asok 2024-04-20T11:36:22.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T11:36:22.836 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.68135.asok 2024-04-20T11:36:22.836 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:36:22.836 DEBUG:teuthology.orchestra.run.smithi152:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.68135.asok status 2024-04-20T11:36:23.066 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-04-20T11:36:23.066 INFO:teuthology.orchestra.run.smithi152.stdout: "metadata": { 2024-04-20T11:36:23.066 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_sha1": "a8c84f5997950fd7d008204839253291d52e6eda", 2024-04-20T11:36:23.066 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", 2024-04-20T11:36:23.066 INFO:teuthology.orchestra.run.smithi152.stdout: "entity_id": "0", 2024-04-20T11:36:23.066 INFO:teuthology.orchestra.run.smithi152.stdout: "hostname": "smithi152", 2024-04-20T11:36:23.066 INFO:teuthology.orchestra.run.smithi152.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-20T11:36:23.067 INFO:teuthology.orchestra.run.smithi152.stdout: "pid": "68135", 2024-04-20T11:36:23.067 INFO:teuthology.orchestra.run.smithi152.stdout: "root": "/" 2024-04-20T11:36:23.067 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:36:23.067 INFO:teuthology.orchestra.run.smithi152.stdout: "dentry_count": 0, 2024-04-20T11:36:23.067 INFO:teuthology.orchestra.run.smithi152.stdout: "dentry_pinned_count": 0, 2024-04-20T11:36:23.067 INFO:teuthology.orchestra.run.smithi152.stdout: "id": 9675, 2024-04-20T11:36:23.067 INFO:teuthology.orchestra.run.smithi152.stdout: "inst": { 2024-04-20T11:36:23.067 INFO:teuthology.orchestra.run.smithi152.stdout: "name": { 2024-04-20T11:36:23.067 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "client", 2024-04-20T11:36:23.067 INFO:teuthology.orchestra.run.smithi152.stdout: "num": 9675 2024-04-20T11:36:23.067 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:36:23.067 INFO:teuthology.orchestra.run.smithi152.stdout: "addr": { 2024-04-20T11:36:23.068 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "v1", 2024-04-20T11:36:23.068 INFO:teuthology.orchestra.run.smithi152.stdout: "addr": "192.168.0.1:0", 2024-04-20T11:36:23.068 INFO:teuthology.orchestra.run.smithi152.stdout: "nonce": 2021436239 2024-04-20T11:36:23.068 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:36:23.068 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:36:23.068 INFO:teuthology.orchestra.run.smithi152.stdout: "addr": { 2024-04-20T11:36:23.068 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "v1", 2024-04-20T11:36:23.068 INFO:teuthology.orchestra.run.smithi152.stdout: "addr": "192.168.0.1:0", 2024-04-20T11:36:23.068 INFO:teuthology.orchestra.run.smithi152.stdout: "nonce": 2021436239 2024-04-20T11:36:23.068 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:36:23.068 INFO:teuthology.orchestra.run.smithi152.stdout: "inst_str": "client.9675 192.168.0.1:0/2021436239", 2024-04-20T11:36:23.068 INFO:teuthology.orchestra.run.smithi152.stdout: "addr_str": "192.168.0.1:0/2021436239", 2024-04-20T11:36:23.068 INFO:teuthology.orchestra.run.smithi152.stdout: "inode_count": 1, 2024-04-20T11:36:23.069 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_epoch": 96, 2024-04-20T11:36:23.069 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_epoch": 142, 2024-04-20T11:36:23.069 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_epoch_barrier": 0, 2024-04-20T11:36:23.069 INFO:teuthology.orchestra.run.smithi152.stdout: "blocklisted": false, 2024-04-20T11:36:23.069 INFO:teuthology.orchestra.run.smithi152.stdout: "fs_name": "cephfs" 2024-04-20T11:36:23.069 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-04-20T11:36:23.080 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:36:23.081 DEBUG:teuthology.orchestra.run.smithi152:> stat --file-system '--printf=%T 2024-04-20T11:36:23.081 DEBUG:teuthology.orchestra.run.smithi152:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-20T11:36:23.131 INFO:teuthology.orchestra.run.smithi152.stdout:fuseblk 2024-04-20T11:36:23.132 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-20T11:36:23.132 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:36:23.132 DEBUG:teuthology.orchestra.run.smithi152:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-20T11:36:23.212 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-20T11:36:23.422 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:23.421+0000 7f15a6000640 1 Processor -- start 2024-04-20T11:36:23.422 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:23.421+0000 7f15a6000640 1 -- start start 2024-04-20T11:36:23.424 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:23.423+0000 7f15a6000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f15a0072c70 0x7f15a00b35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:23.424 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:23.423+0000 7f15a6000640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15a015b340 con 0x7f15a0151ab0 2024-04-20T11:36:23.424 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:23.423+0000 7f15a6000640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15a015b620 con 0x7f15a00b3b00 2024-04-20T11:36:23.424 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:23.423+0000 7f15a6000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f15a015c100 con 0x7f15a0072c70 2024-04-20T11:36:23.424 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:23.423+0000 7f15a4c00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f15a0151ab0 0x7f15a0072560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:44360/0 (socket says 172.21.15.152:44360) 2024-04-20T11:36:23.424 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:23.423+0000 7f15a4c00640 1 -- 172.21.15.152:0/4102358383 learned_addr learned my addr 172.21.15.152:0/4102358383 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:36:23.424 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:23.423+0000 7f1597e00640 1 --2- 172.21.15.152:0/4102358383 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f15a0072c70 0x7f15a00b35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:23.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:23.424+0000 7f159fe00640 1 -- 172.21.15.152:0/4102358383 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2522135457 0 0) 0x7f15a015b340 con 0x7f15a0151ab0 2024-04-20T11:36:23.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:23.424+0000 7f159fe00640 1 -- 172.21.15.152:0/4102358383 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1584005730 con 0x7f15a0151ab0 2024-04-20T11:36:23.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:23.424+0000 7f159fe00640 1 -- 172.21.15.152:0/4102358383 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 866706384 0 0) 0x7f1584005730 con 0x7f15a0151ab0 2024-04-20T11:36:23.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:23.424+0000 7f159fe00640 1 -- 172.21.15.152:0/4102358383 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f15a0072c70 msgr2=0x7f15a00b35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:23.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:23.424+0000 7f159fe00640 1 --2- 172.21.15.152:0/4102358383 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f15a0072c70 0x7f15a00b35c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:23.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:23.424+0000 7f159fe00640 1 -- 172.21.15.152:0/4102358383 >> v1:172.21.15.175:6789/0 conn(0x7f15a00b3b00 legacy=0x7f15a015e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:23.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:23.424+0000 7f159fe00640 1 -- 172.21.15.152:0/4102358383 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15a015b900 con 0x7f15a0151ab0 2024-04-20T11:36:23.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:23.424+0000 7f159fe00640 1 -- 172.21.15.152:0/4102358383 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f1590001670 con 0x7f15a0151ab0 2024-04-20T11:36:23.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:23.424+0000 7f159fe00640 1 -- 172.21.15.152:0/4102358383 >> v1:172.21.15.152:6789/0 conn(0x7f15a0151ab0 legacy=0x7f15a0072560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:23.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:23.424+0000 7f159fe00640 1 --2- 172.21.15.152:0/4102358383 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f15a0072c70 0x7f1584005bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:23.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:23.424+0000 7f159fe00640 1 --2- 172.21.15.152:0/4102358383 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f15840061c0 0x7f1584006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:23.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:23.424+0000 7f159fe00640 1 --2- 172.21.15.152:0/4102358383 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1584006f50 0x7f15840073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:23.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:23.424+0000 7f159fe00640 1 -- 172.21.15.152:0/4102358383 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f15a015c100 con 0x7f15840061c0 2024-04-20T11:36:23.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:23.424+0000 7f159fe00640 1 -- 172.21.15.152:0/4102358383 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f158400dd10 con 0x7f1584006f50 2024-04-20T11:36:23.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:23.424+0000 7f159fe00640 1 -- 172.21.15.152:0/4102358383 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f158400df10 con 0x7f15a0072c70 2024-04-20T11:36:23.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:23.424+0000 7f1597e00640 1 --2- 172.21.15.152:0/4102358383 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f15840061c0 0x7f1584006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:23.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:23.425+0000 7f15a4c00640 1 --2- 172.21.15.152:0/4102358383 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f15a0072c70 0x7f1584005bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:23.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:23.425+0000 7f15a5600640 1 --2- 172.21.15.152:0/4102358383 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1584006f50 0x7f15840073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:23.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:23.425+0000 7f1597e00640 1 -- 172.21.15.152:0/4102358383 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f15a0072c70 msgr2=0x7f1584005bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:23.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:23.425+0000 7f1597e00640 1 --2- 172.21.15.152:0/4102358383 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f15a0072c70 0x7f1584005bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:23.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:23.425+0000 7f1597e00640 1 -- 172.21.15.152:0/4102358383 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1584006f50 msgr2=0x7f15840073d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:23.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:23.425+0000 7f1597e00640 1 --2- 172.21.15.152:0/4102358383 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1584006f50 0x7f15840073d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:23.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:23.425+0000 7f1597e00640 1 -- 172.21.15.152:0/4102358383 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1584005730 con 0x7f15840061c0 2024-04-20T11:36:23.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:23.426+0000 7f1597e00640 1 --2- 172.21.15.152:0/4102358383 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f15840061c0 0x7f1584006a10 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7f1588009ac0 tx=0x7f1584007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7bd1bd6a1ec7eef server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:23.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:23.426+0000 7f159fe00640 1 -- 172.21.15.152:0/4102358383 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1588039070 con 0x7f15840061c0 2024-04-20T11:36:23.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:23.426+0000 7f159fe00640 1 -- 172.21.15.152:0/4102358383 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f158802c600 con 0x7f15840061c0 2024-04-20T11:36:23.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:23.426+0000 7f159fe00640 1 -- 172.21.15.152:0/4102358383 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1588009690 con 0x7f15840061c0 2024-04-20T11:36:23.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:23.427+0000 7f15a6000640 1 -- 172.21.15.152:0/4102358383 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f15840061c0 msgr2=0x7f1584006a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:23.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:23.427+0000 7f15a6000640 1 --2- 172.21.15.152:0/4102358383 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f15840061c0 0x7f1584006a10 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7f1588009ac0 tx=0x7f1584007a40 comp rx=0 tx=0).stop 2024-04-20T11:36:23.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:23.427+0000 7f15a5600640 1 -- 172.21.15.152:0/4102358383 reap_dead start 2024-04-20T11:36:23.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:23.427+0000 7f15a6000640 1 -- 172.21.15.152:0/4102358383 shutdown_connections 2024-04-20T11:36:23.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:23.427+0000 7f15a6000640 1 -- 172.21.15.152:0/4102358383 >> 172.21.15.152:0/4102358383 conn(0x7f15a0067ae0 msgr2=0x7f15a006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:23.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:23.427+0000 7f15a6000640 1 -- 172.21.15.152:0/4102358383 shutdown_connections 2024-04-20T11:36:23.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:23.427+0000 7f15a6000640 1 -- 172.21.15.152:0/4102358383 wait complete. 2024-04-20T11:36:23.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:23.429+0000 7f15a6000640 1 Processor -- start 2024-04-20T11:36:23.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:23.429+0000 7f15a6000640 1 -- start start 2024-04-20T11:36:23.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:23.430+0000 7f15a6000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f15840061c0 0x7f15a0149cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:23.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:23.430+0000 7f15a6000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15a00b0000 0x7f15a00b0480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:23.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:23.430+0000 7f15a6000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f15a014a9d0 0x7f15a014ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:23.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:23.430+0000 7f15a6000640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f15a00b3f50 con 0x7f15a014a9d0 2024-04-20T11:36:23.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:23.430+0000 7f15a6000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f15a015bc70 con 0x7f15a00b0000 2024-04-20T11:36:23.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:23.430+0000 7f15a6000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f15a00b40d0 con 0x7f15840061c0 2024-04-20T11:36:23.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:23.430+0000 7f15a5600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f15a014a9d0 0x7f15a014ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:23.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:23.430+0000 7f15a5600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f15a014a9d0 0x7f15a014ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:40378/0 (socket says 172.21.15.152:40378) 2024-04-20T11:36:23.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:23.430+0000 7f15a5600640 1 -- 172.21.15.152:0/3653659850 learned_addr learned my addr 172.21.15.152:0/3653659850 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:23.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:23.430+0000 7f1597e00640 1 --2- 172.21.15.152:0/3653659850 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15a00b0000 0x7f15a00b0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:23.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:23.430+0000 7f15a4c00640 1 --2- 172.21.15.152:0/3653659850 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f15840061c0 0x7f15a0149cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:23.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:23.431+0000 7f15a5600640 1 -- 172.21.15.152:0/3653659850 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f15840061c0 msgr2=0x7f15a0149cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:23.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:23.431+0000 7f15a5600640 1 --2- 172.21.15.152:0/3653659850 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f15840061c0 0x7f15a0149cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:23.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:23.431+0000 7f15a5600640 1 -- 172.21.15.152:0/3653659850 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15a00b0000 msgr2=0x7f15a00b0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:23.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:23.431+0000 7f15a5600640 1 --2- 172.21.15.152:0/3653659850 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15a00b0000 0x7f15a00b0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:23.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:23.431+0000 7f15a5600640 1 -- 172.21.15.152:0/3653659850 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15a014b390 con 0x7f15a014a9d0 2024-04-20T11:36:23.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:23.431+0000 7f15a4c00640 1 --2- 172.21.15.152:0/3653659850 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f15840061c0 0x7f15a0149cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:36:23.433 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:23.431+0000 7f15a5600640 1 --2- 172.21.15.152:0/3653659850 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f15a014a9d0 0x7f15a014ae50 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7f15a00730f0 tx=0x7f159800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:23.433 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:23.431+0000 7f159d600640 1 -- 172.21.15.152:0/3653659850 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1598018070 con 0x7f15a014a9d0 2024-04-20T11:36:23.433 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:23.431+0000 7f159d600640 1 -- 172.21.15.152:0/3653659850 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1598003b70 con 0x7f15a014a9d0 2024-04-20T11:36:23.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:23.431+0000 7f159d600640 1 -- 172.21.15.152:0/3653659850 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1598005070 con 0x7f15a014a9d0 2024-04-20T11:36:23.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:23.431+0000 7f15a6000640 1 -- 172.21.15.152:0/3653659850 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15a014b650 con 0x7f15a014a9d0 2024-04-20T11:36:23.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:23.431+0000 7f15a6000640 1 -- 172.21.15.152:0/3653659850 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15a014bc10 con 0x7f15a014a9d0 2024-04-20T11:36:23.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:23.432+0000 7f15a6000640 1 -- 172.21.15.152:0/3653659850 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15a0075e60 con 0x7f15a014a9d0 2024-04-20T11:36:23.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:23.434+0000 7f159d600640 1 -- 172.21.15.152:0/3653659850 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 22) ==== 91601+0+0 (secure 0 0 0) 0x7f159801b070 con 0x7f15a014a9d0 2024-04-20T11:36:23.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:23.434+0000 7f159d600640 1 --2- 172.21.15.152:0/3653659850 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f157c06cf20 0x7f157c06f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:23.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:23.435+0000 7f159d600640 1 -- 172.21.15.152:0/3653659850 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5709+0+0 (secure 0 0 0) 0x7f159808c0c0 con 0x7f15a014a9d0 2024-04-20T11:36:23.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:23.435+0000 7f15a4c00640 1 --2- 172.21.15.152:0/3653659850 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f157c06cf20 0x7f157c06f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:23.437 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:23.436+0000 7f15a4c00640 1 --2- 172.21.15.152:0/3653659850 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f157c06cf20 0x7f157c06f3e0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f1590001eb0 tx=0x7f1590000ef0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:23.439 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:23.439+0000 7f159d600640 1 -- 172.21.15.152:0/3653659850 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f1598059af0 con 0x7f15a014a9d0 2024-04-20T11:36:23.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:23.680+0000 7f15a6000640 1 -- 172.21.15.152:0/3653659850 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f15a006e220 con 0x7f15a014a9d0 2024-04-20T11:36:23.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:23.681+0000 7f159d600640 1 -- 172.21.15.152:0/3653659850 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v96) ==== 53+0+73 (secure 0 0 0) 0x7f15980593f0 con 0x7f15a014a9d0 2024-04-20T11:36:23.682 INFO:teuthology.orchestra.run.smithi152.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-20T11:36:23.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:23.684+0000 7f15a6000640 1 -- 172.21.15.152:0/3653659850 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f157c06cf20 msgr2=0x7f157c06f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:23.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:23.684+0000 7f15a6000640 1 --2- 172.21.15.152:0/3653659850 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f157c06cf20 0x7f157c06f3e0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f1590001eb0 tx=0x7f1590000ef0 comp rx=0 tx=0).stop 2024-04-20T11:36:23.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:23.684+0000 7f15a6000640 1 -- 172.21.15.152:0/3653659850 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f15a014a9d0 msgr2=0x7f15a014ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:23.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:23.684+0000 7f15a6000640 1 --2- 172.21.15.152:0/3653659850 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f15a014a9d0 0x7f15a014ae50 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7f15a00730f0 tx=0x7f159800dc00 comp rx=0 tx=0).stop 2024-04-20T11:36:23.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:23.684+0000 7f15a6000640 1 -- 172.21.15.152:0/3653659850 shutdown_connections 2024-04-20T11:36:23.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:23.684+0000 7f15a6000640 1 --2- 172.21.15.152:0/3653659850 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f157c06cf20 0x7f157c06f3e0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:23.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:23.684+0000 7f15a6000640 1 --2- 172.21.15.152:0/3653659850 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f15a014a9d0 0x7f15a014ae50 unknown :-1 s=CLOSED pgs=2005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:23.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:23.684+0000 7f15a6000640 1 --2- 172.21.15.152:0/3653659850 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15a00b0000 0x7f15a00b0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:23.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:23.684+0000 7f15a6000640 1 --2- 172.21.15.152:0/3653659850 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f15840061c0 0x7f15a0149cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:23.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:23.684+0000 7f15a6000640 1 -- 172.21.15.152:0/3653659850 >> 172.21.15.152:0/3653659850 conn(0x7f15a0067ae0 msgr2=0x7f15a015a2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:23.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:23.685+0000 7f15a6000640 1 -- 172.21.15.152:0/3653659850 shutdown_connections 2024-04-20T11:36:23.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:23.685+0000 7f15a6000640 1 -- 172.21.15.152:0/3653659850 wait complete. 2024-04-20T11:36:23.704 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-20T11:36:23.704 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-20T11:36:23.705 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi175.front.sepia.ceph.com 2024-04-20T11:36:23.705 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-20T11:36:23.705 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-20T11:36:23.705 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-20T11:36:23.705 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-20T11:36:23.705 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-20T11:36:23.705 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-20T11:36:23.705 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:36:23.705 DEBUG:teuthology.orchestra.run.smithi175:> ip addr 2024-04-20T11:36:23.764 INFO:teuthology.orchestra.run.smithi175.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-20T11:36:23.764 INFO:teuthology.orchestra.run.smithi175.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-20T11:36:23.764 INFO:teuthology.orchestra.run.smithi175.stdout: inet 127.0.0.1/8 scope host lo 2024-04-20T11:36:23.764 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:36:23.764 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 ::1/128 scope host 2024-04-20T11:36:23.764 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:36:23.764 INFO:teuthology.orchestra.run.smithi175.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T11:36:23.764 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:d9:39:ee brd ff:ff:ff:ff:ff:ff 2024-04-20T11:36:23.764 INFO:teuthology.orchestra.run.smithi175.stdout: altname enp7s0f0 2024-04-20T11:36:23.765 INFO:teuthology.orchestra.run.smithi175.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T11:36:23.765 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:d9:39:ef brd ff:ff:ff:ff:ff:ff 2024-04-20T11:36:23.765 INFO:teuthology.orchestra.run.smithi175.stdout: altname enp7s0f1 2024-04-20T11:36:23.765 INFO:teuthology.orchestra.run.smithi175.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T11:36:23.765 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:88:71:f0 brd ff:ff:ff:ff:ff:ff 2024-04-20T11:36:23.765 INFO:teuthology.orchestra.run.smithi175.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-20T11:36:23.765 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:88:71:f1 brd ff:ff:ff:ff:ff:ff 2024-04-20T11:36:23.765 INFO:teuthology.orchestra.run.smithi175.stdout: inet 172.21.15.175/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-20T11:36:23.765 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft 42197sec preferred_lft 42197sec 2024-04-20T11:36:23.765 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::ec4:7aff:fe88:71f1/64 scope link 2024-04-20T11:36:23.765 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:36:23.765 INFO:teuthology.orchestra.run.smithi175.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-20T11:36:23.765 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 02:d6:c6:31:26:70 brd ff:ff:ff:ff:ff:ff 2024-04-20T11:36:23.765 INFO:teuthology.orchestra.run.smithi175.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-20T11:36:23.765 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:36:23.766 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::5091:d5ff:fee7:54b3/64 scope link 2024-04-20T11:36:23.766 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:36:23.766 INFO:teuthology.orchestra.run.smithi175.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-20T11:36:23.766 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 02:d6:c6:31:26:70 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-20T11:36:23.766 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::d6:c6ff:fe31:2670/64 scope link 2024-04-20T11:36:23.766 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:36:23.766 INFO:teuthology.orchestra.run.smithi175.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-20T11:36:23.766 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 9e:14:f3:e2:18:1e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-20T11:36:23.766 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::9c14:f3ff:fee2:181e/64 scope link 2024-04-20T11:36:23.766 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:36:23.767 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:36:23.767 DEBUG:teuthology.orchestra.run.smithi175:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-20T11:36:23.800 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:36:23.808 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:36:23.845 INFO:teuthology.orchestra.run.smithi175.stdout:1 2024-04-20T11:36:23.848 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:36:23.848 DEBUG:teuthology.orchestra.run.smithi175:> route 2024-04-20T11:36:23.931 INFO:teuthology.orchestra.run.smithi175.stdout:Kernel IP routing table 2024-04-20T11:36:23.931 INFO:teuthology.orchestra.run.smithi175.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-20T11:36:23.931 INFO:teuthology.orchestra.run.smithi175.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-20T11:36:23.931 INFO:teuthology.orchestra.run.smithi175.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-20T11:36:23.932 INFO:teuthology.orchestra.run.smithi175.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-20T11:36:23.932 DEBUG:teuthology.orchestra.run.smithi175:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T11:36:23.932 DEBUG:teuthology.orchestra.run.smithi175:> set -e 2024-04-20T11:36:23.932 DEBUG:teuthology.orchestra.run.smithi175:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-20T11:36:23.932 DEBUG:teuthology.orchestra.run.smithi175:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-20T11:36:23.932 DEBUG:teuthology.orchestra.run.smithi175:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-20T11:36:23.932 DEBUG:teuthology.orchestra.run.smithi175:> ') 2024-04-20T11:36:24.077 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:36:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T11:36:24.170 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:36:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T11:36:24.180 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:36:24.180 DEBUG:teuthology.orchestra.run.smithi175:> ip netns list 2024-04-20T11:36:24.226 INFO:teuthology.orchestra.run.smithi175.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-20T11:36:24.226 INFO:teuthology.orchestra.run.smithi175.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-20T11:36:24.226 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-20T11:36:24.226 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:36:24.226 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip link set brx.0 up 2024-04-20T11:36:24.352 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-04-20T11:36:24.353 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T11:36:24.353 DEBUG:teuthology.orchestra.run.smithi175:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-20T11:36:24.406 INFO:teuthology.orchestra.run.smithi175.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-20T11:36:24.406 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T11:36:24.406 DEBUG:teuthology.orchestra.run.smithi175:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-20T11:36:24.456 DEBUG:teuthology.orchestra.run.smithi175:> sudo modprobe fuse 2024-04-20T11:36:24.539 DEBUG:teuthology.orchestra.run.smithi175:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T11:36:24.593 INFO:teuthology.orchestra.run.smithi175.stdout:/proc 2024-04-20T11:36:24.593 INFO:teuthology.orchestra.run.smithi175.stdout:/sys 2024-04-20T11:36:24.593 INFO:teuthology.orchestra.run.smithi175.stdout:/dev 2024-04-20T11:36:24.593 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/security 2024-04-20T11:36:24.593 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/shm 2024-04-20T11:36:24.593 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/pts 2024-04-20T11:36:24.593 INFO:teuthology.orchestra.run.smithi175.stdout:/run 2024-04-20T11:36:24.593 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/cgroup 2024-04-20T11:36:24.593 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/pstore 2024-04-20T11:36:24.593 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/bpf 2024-04-20T11:36:24.593 INFO:teuthology.orchestra.run.smithi175.stdout:/ 2024-04-20T11:36:24.594 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/selinux 2024-04-20T11:36:24.594 INFO:teuthology.orchestra.run.smithi175.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T11:36:24.594 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/mqueue 2024-04-20T11:36:24.594 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/debug 2024-04-20T11:36:24.594 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/hugepages 2024-04-20T11:36:24.594 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/tracing 2024-04-20T11:36:24.594 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/fuse/connections 2024-04-20T11:36:24.594 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/config 2024-04-20T11:36:24.594 INFO:teuthology.orchestra.run.smithi175.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T11:36:24.594 INFO:teuthology.orchestra.run.smithi175.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T11:36:24.594 INFO:teuthology.orchestra.run.smithi175.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T11:36:24.594 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T11:36:24.594 INFO:teuthology.orchestra.run.smithi175.stdout:/run/user/1000 2024-04-20T11:36:24.595 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph 2024-04-20T11:36:24.595 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-4 2024-04-20T11:36:24.595 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-5 2024-04-20T11:36:24.595 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-6 2024-04-20T11:36:24.595 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-7 2024-04-20T11:36:24.595 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns 2024-04-20T11:36:24.595 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-20T11:36:24.595 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-20T11:36:24.595 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-20T11:36:24.595 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-20T11:36:24.596 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:36:24.596 DEBUG:teuthology.orchestra.run.smithi175:> ls /sys/fs/fuse/connections 2024-04-20T11:36:24.646 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-20T11:36:24.646 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-04-20T11:36:24.650 DEBUG:teuthology.orchestra.run.smithi175:> sudo modprobe fuse 2024-04-20T11:36:24.715 DEBUG:teuthology.orchestra.run.smithi175:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T11:36:24.757 INFO:teuthology.orchestra.run.smithi175.stdout:/proc 2024-04-20T11:36:24.757 INFO:teuthology.orchestra.run.smithi175.stdout:/sys 2024-04-20T11:36:24.757 INFO:teuthology.orchestra.run.smithi175.stdout:/dev 2024-04-20T11:36:24.757 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/security 2024-04-20T11:36:24.757 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/shm 2024-04-20T11:36:24.757 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/pts 2024-04-20T11:36:24.757 INFO:teuthology.orchestra.run.smithi175.stdout:/run 2024-04-20T11:36:24.757 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/cgroup 2024-04-20T11:36:24.758 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/pstore 2024-04-20T11:36:24.758 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/bpf 2024-04-20T11:36:24.758 INFO:teuthology.orchestra.run.smithi175.stdout:/ 2024-04-20T11:36:24.758 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/selinux 2024-04-20T11:36:24.758 INFO:teuthology.orchestra.run.smithi175.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T11:36:24.758 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/mqueue 2024-04-20T11:36:24.758 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/debug 2024-04-20T11:36:24.758 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/hugepages 2024-04-20T11:36:24.758 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/tracing 2024-04-20T11:36:24.758 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/fuse/connections 2024-04-20T11:36:24.758 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/config 2024-04-20T11:36:24.758 INFO:teuthology.orchestra.run.smithi175.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T11:36:24.758 INFO:teuthology.orchestra.run.smithi175.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T11:36:24.759 INFO:teuthology.orchestra.run.smithi175.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T11:36:24.759 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T11:36:24.759 INFO:teuthology.orchestra.run.smithi175.stdout:/run/user/1000 2024-04-20T11:36:24.759 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph 2024-04-20T11:36:24.759 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-4 2024-04-20T11:36:24.759 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-5 2024-04-20T11:36:24.759 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-6 2024-04-20T11:36:24.759 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-7 2024-04-20T11:36:24.759 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns 2024-04-20T11:36:24.759 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-20T11:36:24.759 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-20T11:36:24.759 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-20T11:36:24.759 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-20T11:36:24.760 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:36:24.760 DEBUG:teuthology.orchestra.run.smithi175:> ls /sys/fs/fuse/connections 2024-04-20T11:36:24.816 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi175.stderr:2024-04-20T11:36:24.816+0000 7fb315237480 -1 init, newargv = 0x55ff7fa115b0 newargc=15 2024-04-20T11:36:24.816 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi175.stderr:ceph-fuse[35847]: starting ceph client 2024-04-20T11:36:24.824 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi175.stderr:ceph-fuse[35847]: starting fuse 2024-04-20T11:36:24.844 INFO:teuthology.orchestra.run.smithi175.stdout:42 2024-04-20T11:36:24.844 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-04-20T11:36:24.845 DEBUG:teuthology.orchestra.run.smithi175:> sudo stdin-killer -- python3 -c ' 2024-04-20T11:36:24.845 DEBUG:teuthology.orchestra.run.smithi175:> import glob 2024-04-20T11:36:24.845 DEBUG:teuthology.orchestra.run.smithi175:> import re 2024-04-20T11:36:24.845 DEBUG:teuthology.orchestra.run.smithi175:> import os 2024-04-20T11:36:24.845 DEBUG:teuthology.orchestra.run.smithi175:> import subprocess 2024-04-20T11:36:24.845 DEBUG:teuthology.orchestra.run.smithi175:> 2024-04-20T11:36:24.845 DEBUG:teuthology.orchestra.run.smithi175:> def _find_admin_socket(client_name): 2024-04-20T11:36:24.845 DEBUG:teuthology.orchestra.run.smithi175:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-04-20T11:36:24.845 DEBUG:teuthology.orchestra.run.smithi175:> files = glob.glob(asok_path) 2024-04-20T11:36:24.845 DEBUG:teuthology.orchestra.run.smithi175:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-04-20T11:36:24.845 DEBUG:teuthology.orchestra.run.smithi175:> 2024-04-20T11:36:24.845 DEBUG:teuthology.orchestra.run.smithi175:> # Given a non-glob path, it better be there 2024-04-20T11:36:24.845 DEBUG:teuthology.orchestra.run.smithi175:> if "*" not in asok_path: 2024-04-20T11:36:24.845 DEBUG:teuthology.orchestra.run.smithi175:> assert(len(files) == 1) 2024-04-20T11:36:24.845 DEBUG:teuthology.orchestra.run.smithi175:> return files[0] 2024-04-20T11:36:24.846 DEBUG:teuthology.orchestra.run.smithi175:> 2024-04-20T11:36:24.846 DEBUG:teuthology.orchestra.run.smithi175:> for f in files: 2024-04-20T11:36:24.846 DEBUG:teuthology.orchestra.run.smithi175:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-20T11:36:24.846 DEBUG:teuthology.orchestra.run.smithi175:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-20T11:36:24.846 DEBUG:teuthology.orchestra.run.smithi175:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-20T11:36:24.846 DEBUG:teuthology.orchestra.run.smithi175:> contents = proc_f.read() 2024-04-20T11:36:24.846 DEBUG:teuthology.orchestra.run.smithi175:> if mountpoint in contents: 2024-04-20T11:36:24.846 DEBUG:teuthology.orchestra.run.smithi175:> return f 2024-04-20T11:36:24.846 DEBUG:teuthology.orchestra.run.smithi175:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-20T11:36:24.846 DEBUG:teuthology.orchestra.run.smithi175:> 2024-04-20T11:36:24.846 DEBUG:teuthology.orchestra.run.smithi175:> print(_find_admin_socket("client.1")) 2024-04-20T11:36:24.846 DEBUG:teuthology.orchestra.run.smithi175:> ' 2024-04-20T11:36:25.036 INFO:teuthology.orchestra.run.smithi175.stdout:/var/run/ceph/ceph-client.1.35847.asok 2024-04-20T11:36:25.040 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:36:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T11:36:25.052 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.35847.asok 2024-04-20T11:36:25.053 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:36:25.053 DEBUG:teuthology.orchestra.run.smithi175:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.35847.asok status 2024-04-20T11:36:25.279 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-04-20T11:36:25.279 INFO:teuthology.orchestra.run.smithi175.stdout: "metadata": { 2024-04-20T11:36:25.279 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_sha1": "a8c84f5997950fd7d008204839253291d52e6eda", 2024-04-20T11:36:25.279 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", 2024-04-20T11:36:25.279 INFO:teuthology.orchestra.run.smithi175.stdout: "entity_id": "1", 2024-04-20T11:36:25.279 INFO:teuthology.orchestra.run.smithi175.stdout: "hostname": "smithi175", 2024-04-20T11:36:25.279 INFO:teuthology.orchestra.run.smithi175.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-04-20T11:36:25.280 INFO:teuthology.orchestra.run.smithi175.stdout: "pid": "35847", 2024-04-20T11:36:25.280 INFO:teuthology.orchestra.run.smithi175.stdout: "root": "/" 2024-04-20T11:36:25.280 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-20T11:36:25.280 INFO:teuthology.orchestra.run.smithi175.stdout: "dentry_count": 0, 2024-04-20T11:36:25.280 INFO:teuthology.orchestra.run.smithi175.stdout: "dentry_pinned_count": 0, 2024-04-20T11:36:25.280 INFO:teuthology.orchestra.run.smithi175.stdout: "id": 6353, 2024-04-20T11:36:25.280 INFO:teuthology.orchestra.run.smithi175.stdout: "inst": { 2024-04-20T11:36:25.280 INFO:teuthology.orchestra.run.smithi175.stdout: "name": { 2024-04-20T11:36:25.280 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "client", 2024-04-20T11:36:25.280 INFO:teuthology.orchestra.run.smithi175.stdout: "num": 6353 2024-04-20T11:36:25.280 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-20T11:36:25.280 INFO:teuthology.orchestra.run.smithi175.stdout: "addr": { 2024-04-20T11:36:25.280 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "v1", 2024-04-20T11:36:25.280 INFO:teuthology.orchestra.run.smithi175.stdout: "addr": "192.168.0.1:0", 2024-04-20T11:36:25.281 INFO:teuthology.orchestra.run.smithi175.stdout: "nonce": 2505588120 2024-04-20T11:36:25.281 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-04-20T11:36:25.281 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-20T11:36:25.281 INFO:teuthology.orchestra.run.smithi175.stdout: "addr": { 2024-04-20T11:36:25.281 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "v1", 2024-04-20T11:36:25.281 INFO:teuthology.orchestra.run.smithi175.stdout: "addr": "192.168.0.1:0", 2024-04-20T11:36:25.281 INFO:teuthology.orchestra.run.smithi175.stdout: "nonce": 2505588120 2024-04-20T11:36:25.281 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-20T11:36:25.281 INFO:teuthology.orchestra.run.smithi175.stdout: "inst_str": "client.6353 192.168.0.1:0/2505588120", 2024-04-20T11:36:25.281 INFO:teuthology.orchestra.run.smithi175.stdout: "addr_str": "192.168.0.1:0/2505588120", 2024-04-20T11:36:25.281 INFO:teuthology.orchestra.run.smithi175.stdout: "inode_count": 1, 2024-04-20T11:36:25.281 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_epoch": 96, 2024-04-20T11:36:25.281 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_epoch": 142, 2024-04-20T11:36:25.281 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_epoch_barrier": 0, 2024-04-20T11:36:25.281 INFO:teuthology.orchestra.run.smithi175.stdout: "blocklisted": false, 2024-04-20T11:36:25.282 INFO:teuthology.orchestra.run.smithi175.stdout: "fs_name": "cephfs" 2024-04-20T11:36:25.282 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-04-20T11:36:25.294 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:36:25.294 DEBUG:teuthology.orchestra.run.smithi175:> stat --file-system '--printf=%T 2024-04-20T11:36:25.294 DEBUG:teuthology.orchestra.run.smithi175:> ' -- /home/ubuntu/cephtest/mnt.1 2024-04-20T11:36:25.346 INFO:teuthology.orchestra.run.smithi175.stdout:fuseblk 2024-04-20T11:36:25.346 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-04-20T11:36:25.346 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:36:25.347 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-20T11:36:25.429 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-04-20T11:36:25.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:25.648+0000 7f97d6600640 1 Processor -- start 2024-04-20T11:36:25.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:25.649+0000 7f97d6600640 1 -- start start 2024-04-20T11:36:25.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:25.650+0000 7f97d6600640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f97d00b88d0 0x7f97d0169520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:25.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:25.650+0000 7f97d6600640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97d00d40e0 con 0x7f97d015e910 2024-04-20T11:36:25.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:25.650+0000 7f97d6600640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97d00d43c0 con 0x7f97d00d22d0 2024-04-20T11:36:25.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:25.650+0000 7f97d6600640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f97d00d46a0 con 0x7f97d00b88d0 2024-04-20T11:36:25.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:25.650+0000 7f97cfe00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f97d00b88d0 0x7f97d0169520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:25.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:25.650+0000 7f97cf400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f97d015e910 0x7f97d00d3870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:44376/0 (socket says 172.21.15.152:44376) 2024-04-20T11:36:25.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:25.650+0000 7f97cf400640 1 -- 172.21.15.152:0/1481784400 learned_addr learned my addr 172.21.15.152:0/1481784400 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:36:25.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:25.651+0000 7f97cfe00640 1 -- 172.21.15.152:0/1481784400 >> v1:172.21.15.175:6789/0 conn(0x7f97d00d22d0 legacy=0x7f97d00cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:25.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:25.651+0000 7f97cfe00640 1 -- 172.21.15.152:0/1481784400 >> v1:172.21.15.152:6789/0 conn(0x7f97d015e910 legacy=0x7f97d00d3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:25.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:25.651+0000 7f97cfe00640 1 -- 172.21.15.152:0/1481784400 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97d00d4980 con 0x7f97d00b88d0 2024-04-20T11:36:25.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:25.651+0000 7f97cea00640 1 -- 172.21.15.152:0/1481784400 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2043667851 0 0) 0x7f97d00d40e0 con 0x7f97d015e910 2024-04-20T11:36:25.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:25.653+0000 7f97cfe00640 1 --2- 172.21.15.152:0/1481784400 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f97d00b88d0 0x7f97d0169520 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7f97c8009d90 tx=0x7f97c802bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff948ad2e3fdf24a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:25.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:25.653+0000 7f97cea00640 1 -- 172.21.15.152:0/1481784400 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97c8039070 con 0x7f97d00b88d0 2024-04-20T11:36:25.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:25.653+0000 7f97cea00640 1 -- 172.21.15.152:0/1481784400 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97c802cd00 con 0x7f97d00b88d0 2024-04-20T11:36:25.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:25.653+0000 7f97cea00640 1 -- 172.21.15.152:0/1481784400 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97c8034660 con 0x7f97d00b88d0 2024-04-20T11:36:25.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:25.654+0000 7f97d6600640 1 -- 172.21.15.152:0/1481784400 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f97d00b88d0 msgr2=0x7f97d0169520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:25.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:25.654+0000 7f97d6600640 1 --2- 172.21.15.152:0/1481784400 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f97d00b88d0 0x7f97d0169520 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7f97c8009d90 tx=0x7f97c802bf50 comp rx=0 tx=0).stop 2024-04-20T11:36:25.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:25.654+0000 7f97d6600640 1 -- 172.21.15.152:0/1481784400 shutdown_connections 2024-04-20T11:36:25.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:25.654+0000 7f97d6600640 1 --2- 172.21.15.152:0/1481784400 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f97d00b88d0 0x7f97d0169520 unknown :-1 s=CLOSED pgs=2008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:25.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:25.654+0000 7f97d6600640 1 -- 172.21.15.152:0/1481784400 >> 172.21.15.152:0/1481784400 conn(0x7f97d0067ae0 msgr2=0x7f97d0067ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:25.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:25.654+0000 7f97d6600640 1 -- 172.21.15.152:0/1481784400 shutdown_connections 2024-04-20T11:36:25.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:25.654+0000 7f97d6600640 1 -- 172.21.15.152:0/1481784400 wait complete. 2024-04-20T11:36:25.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:25.656+0000 7f97d6600640 1 Processor -- start 2024-04-20T11:36:25.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:25.656+0000 7f97d6600640 1 -- start start 2024-04-20T11:36:25.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:25.657+0000 7f97d6600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f97d00b88d0 0x7f97d014eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:25.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:25.657+0000 7f97d6600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f97d00d22d0 0x7f97d015ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:25.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:25.657+0000 7f97d6600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f97d015e910 0x7f97d014f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:25.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:25.657+0000 7f97d6600640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f97d00cfbe0 con 0x7f97d00d22d0 2024-04-20T11:36:25.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:25.657+0000 7f97d6600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f97d015ecf0 con 0x7f97d00b88d0 2024-04-20T11:36:25.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:25.657+0000 7f97d6600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f97d00cfd60 con 0x7f97d015e910 2024-04-20T11:36:25.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:25.657+0000 7f97c7e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f97d00d22d0 0x7f97d015ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:25.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:25.657+0000 7f97c7e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f97d00d22d0 0x7f97d015ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:40388/0 (socket says 172.21.15.152:40388) 2024-04-20T11:36:25.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:25.657+0000 7f97c7e00640 1 -- 172.21.15.152:0/349405921 learned_addr learned my addr 172.21.15.152:0/349405921 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:25.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:25.658+0000 7f97cf400640 1 --2- 172.21.15.152:0/349405921 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f97d00b88d0 0x7f97d014eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:25.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:25.658+0000 7f97cfe00640 1 --2- 172.21.15.152:0/349405921 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f97d015e910 0x7f97d014f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:25.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:25.658+0000 7f97c7e00640 1 -- 172.21.15.152:0/349405921 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f97d015e910 msgr2=0x7f97d014f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:25.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:25.658+0000 7f97c7e00640 1 --2- 172.21.15.152:0/349405921 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f97d015e910 0x7f97d014f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:25.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:25.658+0000 7f97c7e00640 1 -- 172.21.15.152:0/349405921 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f97d00b88d0 msgr2=0x7f97d014eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:25.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:25.658+0000 7f97c7e00640 1 --2- 172.21.15.152:0/349405921 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f97d00b88d0 0x7f97d014eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:25.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:25.658+0000 7f97c7e00640 1 -- 172.21.15.152:0/349405921 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97d00c9330 con 0x7f97d00d22d0 2024-04-20T11:36:25.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:25.658+0000 7f97cf400640 1 --2- 172.21.15.152:0/349405921 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f97d00b88d0 0x7f97d014eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:36:25.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:25.658+0000 7f97c7e00640 1 --2- 172.21.15.152:0/349405921 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f97d00d22d0 0x7f97d015ee70 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7f97b800c450 tx=0x7f97b800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:25.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:25.658+0000 7f97c7400640 1 -- 172.21.15.152:0/349405921 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b80052d0 con 0x7f97d00d22d0 2024-04-20T11:36:25.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:25.658+0000 7f97c7400640 1 -- 172.21.15.152:0/349405921 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97b8005470 con 0x7f97d00d22d0 2024-04-20T11:36:25.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:25.658+0000 7f97c7400640 1 -- 172.21.15.152:0/349405921 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b8012870 con 0x7f97d00d22d0 2024-04-20T11:36:25.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:25.658+0000 7f97d6600640 1 -- 172.21.15.152:0/349405921 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97d00c9610 con 0x7f97d00d22d0 2024-04-20T11:36:25.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:25.658+0000 7f97d6600640 1 -- 172.21.15.152:0/349405921 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97d00c9bd0 con 0x7f97d00d22d0 2024-04-20T11:36:25.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:25.660+0000 7f97d6600640 1 -- 172.21.15.152:0/349405921 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97d0068a60 con 0x7f97d00d22d0 2024-04-20T11:36:25.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:25.661+0000 7f97c7400640 1 -- 172.21.15.152:0/349405921 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 22) ==== 91601+0+0 (secure 0 0 0) 0x7f97b8012a10 con 0x7f97d00d22d0 2024-04-20T11:36:25.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:25.662+0000 7f97c7400640 1 --2- 172.21.15.152:0/349405921 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f97ac06cf50 0x7f97ac06f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:25.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:25.662+0000 7f97c7400640 1 -- 172.21.15.152:0/349405921 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5709+0+0 (secure 0 0 0) 0x7f97b808c2e0 con 0x7f97d00d22d0 2024-04-20T11:36:25.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:25.663+0000 7f97cf400640 1 --2- 172.21.15.152:0/349405921 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f97ac06cf50 0x7f97ac06f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:25.664 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:25.664+0000 7f97cf400640 1 --2- 172.21.15.152:0/349405921 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f97ac06cf50 0x7f97ac06f410 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f97c0000f60 tx=0x7f97c0009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:25.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:25.667+0000 7f97c7400640 1 -- 172.21.15.152:0/349405921 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f97b8059e60 con 0x7f97d00d22d0 2024-04-20T11:36:25.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:25.912+0000 7f97d6600640 1 -- 172.21.15.152:0/349405921 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f97d00cb9e0 con 0x7f97d00d22d0 2024-04-20T11:36:25.955 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:25.954+0000 7f97c7400640 1 -- 172.21.15.152:0/349405921 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v97) ==== 136+0+0 (secure 0 0 0) 0x7f97b8059760 con 0x7f97d00d22d0 2024-04-20T11:36:25.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:25.957+0000 7f97d6600640 1 -- 172.21.15.152:0/349405921 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f97ac06cf50 msgr2=0x7f97ac06f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:25.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:25.957+0000 7f97d6600640 1 --2- 172.21.15.152:0/349405921 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f97ac06cf50 0x7f97ac06f410 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f97c0000f60 tx=0x7f97c0009040 comp rx=0 tx=0).stop 2024-04-20T11:36:25.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:25.957+0000 7f97d6600640 1 -- 172.21.15.152:0/349405921 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f97d00d22d0 msgr2=0x7f97d015ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:25.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:25.957+0000 7f97d6600640 1 --2- 172.21.15.152:0/349405921 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f97d00d22d0 0x7f97d015ee70 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7f97b800c450 tx=0x7f97b800c910 comp rx=0 tx=0).stop 2024-04-20T11:36:25.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:25.957+0000 7f97d6600640 1 -- 172.21.15.152:0/349405921 shutdown_connections 2024-04-20T11:36:25.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:25.957+0000 7f97d6600640 1 --2- 172.21.15.152:0/349405921 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f97ac06cf50 0x7f97ac06f410 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:25.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:25.957+0000 7f97d6600640 1 --2- 172.21.15.152:0/349405921 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f97d015e910 0x7f97d014f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:25.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:25.957+0000 7f97d6600640 1 --2- 172.21.15.152:0/349405921 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f97d00d22d0 0x7f97d015ee70 unknown :-1 s=CLOSED pgs=2009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:25.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:25.957+0000 7f97d6600640 1 --2- 172.21.15.152:0/349405921 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f97d00b88d0 0x7f97d014eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:25.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:25.957+0000 7f97d6600640 1 -- 172.21.15.152:0/349405921 >> 172.21.15.152:0/349405921 conn(0x7f97d0067ae0 msgr2=0x7f97d00cf4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:25.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:25.957+0000 7f97d6600640 1 -- 172.21.15.152:0/349405921 shutdown_connections 2024-04-20T11:36:25.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:25.958+0000 7f97d6600640 1 -- 172.21.15.152:0/349405921 wait complete. 2024-04-20T11:36:25.978 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-04-20T11:36:26.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.174+0000 7f93e6400640 1 Processor -- start 2024-04-20T11:36:26.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.174+0000 7f93e6400640 1 -- start start 2024-04-20T11:36:26.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.175+0000 7f93e6400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f93e015c8b0 0x7f93e00cdbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:26.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.175+0000 7f93e6400640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f93e00d0df0 0x7f93e00d1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:26.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.175+0000 7f93e6400640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93e00ce4b0 con 0x7f93e00b1f70 2024-04-20T11:36:26.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.175+0000 7f93e6400640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f93e00b1cb0 con 0x7f93e015c8b0 2024-04-20T11:36:26.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.175+0000 7f93e6400640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f93e00ce1d0 con 0x7f93e00d0df0 2024-04-20T11:36:26.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.176+0000 7f93df400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f93e015c8b0 0x7f93e00cdbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:26.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.176+0000 7f93dfe00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f93e00b1f70 0x7f93e00b2350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:45308/0 (socket says 172.21.15.152:45308) 2024-04-20T11:36:26.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.176+0000 7f93dfe00640 1 -- 172.21.15.152:0/3826599993 learned_addr learned my addr 172.21.15.152:0/3826599993 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:36:26.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.176+0000 7f93df400640 1 -- 172.21.15.152:0/3826599993 >> v2:172.21.15.175:3300/0 conn(0x7f93e00d0df0 msgr2=0x7f93e00d1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:26.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.176+0000 7f93df400640 1 --2- 172.21.15.152:0/3826599993 >> v2:172.21.15.175:3300/0 conn(0x7f93e00d0df0 0x7f93e00d1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:26.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.176+0000 7f93df400640 1 -- 172.21.15.152:0/3826599993 >> v1:172.21.15.152:6789/0 conn(0x7f93e00b1f70 legacy=0x7f93e00b2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:26.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.176+0000 7f93df400640 1 -- 172.21.15.152:0/3826599993 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93e00d0310 con 0x7f93e015c8b0 2024-04-20T11:36:26.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.176+0000 7f93de000640 1 -- 172.21.15.152:0/3826599993 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1238659921 0 0) 0x7f93e00ce4b0 con 0x7f93e00b1f70 2024-04-20T11:36:26.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.177+0000 7f93df400640 1 --2- 172.21.15.152:0/3826599993 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f93e015c8b0 0x7f93e00cdbf0 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7f93e009c150 tx=0x7f93d002bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6524d8ab17c07ec server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:26.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.178+0000 7f93de000640 1 -- 172.21.15.152:0/3826599993 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93d0039070 con 0x7f93e015c8b0 2024-04-20T11:36:26.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.178+0000 7f93de000640 1 -- 172.21.15.152:0/3826599993 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93d002cb60 con 0x7f93e015c8b0 2024-04-20T11:36:26.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.178+0000 7f93de000640 1 -- 172.21.15.152:0/3826599993 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93d0034440 con 0x7f93e015c8b0 2024-04-20T11:36:26.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.178+0000 7f93e6400640 1 -- 172.21.15.152:0/3826599993 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f93e015c8b0 msgr2=0x7f93e00cdbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:26.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.178+0000 7f93e6400640 1 --2- 172.21.15.152:0/3826599993 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f93e015c8b0 0x7f93e00cdbf0 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7f93e009c150 tx=0x7f93d002bdb0 comp rx=0 tx=0).stop 2024-04-20T11:36:26.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.178+0000 7f93e6400640 1 -- 172.21.15.152:0/3826599993 shutdown_connections 2024-04-20T11:36:26.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.178+0000 7f93e6400640 1 --2- 172.21.15.152:0/3826599993 >> v2:172.21.15.175:3300/0 conn(0x7f93e00d0df0 0x7f93e00d1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:26.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.178+0000 7f93e6400640 1 --2- 172.21.15.152:0/3826599993 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f93e015c8b0 0x7f93e00cdbf0 unknown :-1 s=CLOSED pgs=2011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:26.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.178+0000 7f93e6400640 1 -- 172.21.15.152:0/3826599993 >> 172.21.15.152:0/3826599993 conn(0x7f93e00b4100 msgr2=0x7f93e00b4510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:26.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.178+0000 7f93e6400640 1 -- 172.21.15.152:0/3826599993 shutdown_connections 2024-04-20T11:36:26.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.179+0000 7f93e6400640 1 -- 172.21.15.152:0/3826599993 wait complete. 2024-04-20T11:36:26.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.180+0000 7f93e6400640 1 Processor -- start 2024-04-20T11:36:26.181 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.180+0000 7f93e6400640 1 -- start start 2024-04-20T11:36:26.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.181+0000 7f93e6400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f93e00b1f70 0x7f93e015de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:26.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.181+0000 7f93e6400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f93e00d0df0 0x7f93e0154ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:26.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.181+0000 7f93e6400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93e015c8b0 0x7f93e00c52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:26.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.181+0000 7f93e6400640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f93e015cc90 con 0x7f93e00d0df0 2024-04-20T11:36:26.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.181+0000 7f93e6400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f93e00d08d0 con 0x7f93e015c8b0 2024-04-20T11:36:26.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.181+0000 7f93e6400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f93e0160480 con 0x7f93e00b1f70 2024-04-20T11:36:26.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.182+0000 7f93dea00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f93e00d0df0 0x7f93e0154ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:26.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.182+0000 7f93dea00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f93e00d0df0 0x7f93e0154ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49482/0 (socket says 172.21.15.152:49482) 2024-04-20T11:36:26.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.182+0000 7f93dea00640 1 -- 172.21.15.152:0/94064903 learned_addr learned my addr 172.21.15.152:0/94064903 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:26.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.182+0000 7f93df400640 1 --2- 172.21.15.152:0/94064903 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f93e00b1f70 0x7f93e015de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:26.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.182+0000 7f93dfe00640 1 --2- 172.21.15.152:0/94064903 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93e015c8b0 0x7f93e00c52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:26.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.182+0000 7f93dea00640 1 -- 172.21.15.152:0/94064903 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f93e00b1f70 msgr2=0x7f93e015de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:26.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.182+0000 7f93dea00640 1 --2- 172.21.15.152:0/94064903 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f93e00b1f70 0x7f93e015de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:26.184 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.182+0000 7f93dea00640 1 -- 172.21.15.152:0/94064903 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93e015c8b0 msgr2=0x7f93e00c52c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:26.184 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.182+0000 7f93dea00640 1 --2- 172.21.15.152:0/94064903 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93e015c8b0 0x7f93e00c52c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:26.184 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.182+0000 7f93dea00640 1 -- 172.21.15.152:0/94064903 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93e00b7790 con 0x7f93e00d0df0 2024-04-20T11:36:26.184 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.182+0000 7f93dea00640 1 --2- 172.21.15.152:0/94064903 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f93e00d0df0 0x7f93e0154ad0 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x7f93c800b4a0 tx=0x7f93c800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:26.184 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.183+0000 7f93d7400640 1 -- 172.21.15.152:0/94064903 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c8014070 con 0x7f93e00d0df0 2024-04-20T11:36:26.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.183+0000 7f93d7400640 1 -- 172.21.15.152:0/94064903 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93c8010610 con 0x7f93e00d0df0 2024-04-20T11:36:26.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.183+0000 7f93d7400640 1 -- 172.21.15.152:0/94064903 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c8010a00 con 0x7f93e00d0df0 2024-04-20T11:36:26.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.183+0000 7f93e6400640 1 -- 172.21.15.152:0/94064903 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93e00b7a70 con 0x7f93e00d0df0 2024-04-20T11:36:26.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.183+0000 7f93e6400640 1 -- 172.21.15.152:0/94064903 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93e00b8030 con 0x7f93e00d0df0 2024-04-20T11:36:26.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.184+0000 7f93e6400640 1 -- 172.21.15.152:0/94064903 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93e0069100 con 0x7f93e00d0df0 2024-04-20T11:36:26.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.185+0000 7f93d7400640 1 -- 172.21.15.152:0/94064903 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 22) ==== 91601+0+0 (secure 0 0 0) 0x7f93c8017070 con 0x7f93e00d0df0 2024-04-20T11:36:26.187 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.186+0000 7f93d7400640 1 --2- 172.21.15.152:0/94064903 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f93bc06cf20 0x7f93bc06f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:26.187 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.187+0000 7f93d7400640 1 -- 172.21.15.152:0/94064903 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5709+0+0 (secure 0 0 0) 0x7f93c808c1b0 con 0x7f93e00d0df0 2024-04-20T11:36:26.188 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.187+0000 7f93df400640 1 --2- 172.21.15.152:0/94064903 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f93bc06cf20 0x7f93bc06f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:26.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.188+0000 7f93df400640 1 --2- 172.21.15.152:0/94064903 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f93bc06cf20 0x7f93bc06f3e0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f93d0036000 tx=0x7f93d0030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:26.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.191+0000 7f93d7400640 1 -- 172.21.15.152:0/94064903 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f93c8059c60 con 0x7f93e00d0df0 2024-04-20T11:36:26.418 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.417+0000 7f93e6400640 1 -- 172.21.15.152:0/94064903 --> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f93e015e3a0 con 0x7f93bc06cf20 2024-04-20T11:36:26.422 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.422+0000 7f93d7400640 1 -- 172.21.15.152:0/94064903 <== mgr.4109 v2:172.21.15.175:6836/4021250441 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f93e015e3a0 con 0x7f93bc06cf20 2024-04-20T11:36:26.422 INFO:teuthology.orchestra.run.smithi152.stdout:[ 2024-04-20T11:36:26.422 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:36:26.422 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "cephfs" 2024-04-20T11:36:26.422 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:36:26.422 INFO:teuthology.orchestra.run.smithi152.stdout:] 2024-04-20T11:36:26.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.424+0000 7f93e6400640 1 -- 172.21.15.152:0/94064903 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f93bc06cf20 msgr2=0x7f93bc06f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:26.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.424+0000 7f93e6400640 1 --2- 172.21.15.152:0/94064903 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f93bc06cf20 0x7f93bc06f3e0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f93d0036000 tx=0x7f93d0030000 comp rx=0 tx=0).stop 2024-04-20T11:36:26.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.424+0000 7f93e6400640 1 -- 172.21.15.152:0/94064903 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f93e00d0df0 msgr2=0x7f93e0154ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:26.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.424+0000 7f93e6400640 1 --2- 172.21.15.152:0/94064903 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f93e00d0df0 0x7f93e0154ad0 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x7f93c800b4a0 tx=0x7f93c800b960 comp rx=0 tx=0).stop 2024-04-20T11:36:26.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.424+0000 7f93e6400640 1 -- 172.21.15.152:0/94064903 shutdown_connections 2024-04-20T11:36:26.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.424+0000 7f93e6400640 1 --2- 172.21.15.152:0/94064903 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f93bc06cf20 0x7f93bc06f3e0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:26.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.424+0000 7f93e6400640 1 --2- 172.21.15.152:0/94064903 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93e015c8b0 0x7f93e00c52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:26.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.424+0000 7f93e6400640 1 --2- 172.21.15.152:0/94064903 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f93e00d0df0 0x7f93e0154ad0 unknown :-1 s=CLOSED pgs=2012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:26.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.424+0000 7f93e6400640 1 --2- 172.21.15.152:0/94064903 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f93e00b1f70 0x7f93e015de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:26.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.424+0000 7f93e6400640 1 -- 172.21.15.152:0/94064903 >> 172.21.15.152:0/94064903 conn(0x7f93e00b4100 msgr2=0x7f93e0074a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:26.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.424+0000 7f93e6400640 1 -- 172.21.15.152:0/94064903 shutdown_connections 2024-04-20T11:36:26.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.424+0000 7f93e6400640 1 -- 172.21.15.152:0/94064903 wait complete. 2024-04-20T11:36:26.444 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-20T11:36:26.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.651+0000 7f56ffe00640 1 Processor -- start 2024-04-20T11:36:26.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.651+0000 7f56ffe00640 1 -- start start 2024-04-20T11:36:26.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.653+0000 7f56ffe00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f5700163230 0x7f57000d2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:26.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.653+0000 7f56ffe00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f57000d33d0 0x7f57000cf470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:26.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.653+0000 7f56ffe00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57000d19c0 con 0x7f570015eb50 2024-04-20T11:36:26.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.653+0000 7f56ffe00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f5700162750 con 0x7f57000d33d0 2024-04-20T11:36:26.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.653+0000 7f56ffe00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f5700162a30 con 0x7f5700163230 2024-04-20T11:36:26.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.653+0000 7f56ff400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f57000d33d0 0x7f57000cf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:26.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.653+0000 7f56f7e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f5700163230 0x7f57000d2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:26.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.653+0000 7f56ff400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f57000d33d0 0x7f57000cf470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49490/0 (socket says 172.21.15.152:49490) 2024-04-20T11:36:26.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.653+0000 7f56ff400640 1 -- 172.21.15.152:0/2376630901 learned_addr learned my addr 172.21.15.152:0/2376630901 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:26.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.654+0000 7f56ff400640 1 -- 172.21.15.152:0/2376630901 >> v2:172.21.15.175:3300/0 conn(0x7f5700163230 msgr2=0x7f57000d2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:26.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.654+0000 7f56ff400640 1 --2- 172.21.15.152:0/2376630901 >> v2:172.21.15.175:3300/0 conn(0x7f5700163230 0x7f57000d2e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:26.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.654+0000 7f56ff400640 1 -- 172.21.15.152:0/2376630901 >> v1:172.21.15.175:6789/0 conn(0x7f570015eb50 legacy=0x7f57000d0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:26.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.654+0000 7f56ff400640 1 -- 172.21.15.152:0/2376630901 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5700162d10 con 0x7f57000d33d0 2024-04-20T11:36:26.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.655+0000 7f56ff400640 1 --2- 172.21.15.152:0/2376630901 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f57000d33d0 0x7f57000cf470 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7f570008df30 tx=0x7f56f802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=306be95c379c3d8f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:26.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.656+0000 7f56fe000640 1 -- 172.21.15.152:0/2376630901 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56f802ca80 con 0x7f57000d33d0 2024-04-20T11:36:26.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.656+0000 7f56fe000640 1 -- 172.21.15.152:0/2376630901 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56f802cc20 con 0x7f57000d33d0 2024-04-20T11:36:26.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.656+0000 7f56fe000640 1 -- 172.21.15.152:0/2376630901 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56f80343f0 con 0x7f57000d33d0 2024-04-20T11:36:26.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.656+0000 7f56ffe00640 1 -- 172.21.15.152:0/2376630901 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f57000d33d0 msgr2=0x7f57000cf470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:26.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.656+0000 7f56ffe00640 1 --2- 172.21.15.152:0/2376630901 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f57000d33d0 0x7f57000cf470 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7f570008df30 tx=0x7f56f802bee0 comp rx=0 tx=0).stop 2024-04-20T11:36:26.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.656+0000 7f56ffe00640 1 -- 172.21.15.152:0/2376630901 shutdown_connections 2024-04-20T11:36:26.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.656+0000 7f56ffe00640 1 --2- 172.21.15.152:0/2376630901 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f57000d33d0 0x7f57000cf470 unknown :-1 s=CLOSED pgs=2013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:26.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.656+0000 7f56ffe00640 1 --2- 172.21.15.152:0/2376630901 >> v2:172.21.15.175:3300/0 conn(0x7f5700163230 0x7f57000d2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:26.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.656+0000 7f56ffe00640 1 -- 172.21.15.152:0/2376630901 >> 172.21.15.152:0/2376630901 conn(0x7f570008bfd0 msgr2=0x7f57000d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:26.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.657+0000 7f56ffe00640 1 -- 172.21.15.152:0/2376630901 shutdown_connections 2024-04-20T11:36:26.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.657+0000 7f56ffe00640 1 -- 172.21.15.152:0/2376630901 wait complete. 2024-04-20T11:36:26.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.658+0000 7f56ffe00640 1 Processor -- start 2024-04-20T11:36:26.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.658+0000 7f56ffe00640 1 -- start start 2024-04-20T11:36:26.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.659+0000 7f56ffe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f57000d33d0 0x7f57000cc110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:26.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.659+0000 7f56ffe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f570015eb50 0x7f5700158b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:26.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.659+0000 7f56ffe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5700163230 0x7f5700156840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:26.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.659+0000 7f56ffe00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f57000d1610 con 0x7f570015eb50 2024-04-20T11:36:26.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.659+0000 7f56ffe00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f57000cf9b0 con 0x7f5700163230 2024-04-20T11:36:26.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.659+0000 7f56ffe00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f57000ce200 con 0x7f57000d33d0 2024-04-20T11:36:26.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.660+0000 7f56f7e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f570015eb50 0x7f5700158b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:26.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.660+0000 7f56f7e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f570015eb50 0x7f5700158b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49506/0 (socket says 172.21.15.152:49506) 2024-04-20T11:36:26.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.660+0000 7f56f7e00640 1 -- 172.21.15.152:0/761706629 learned_addr learned my addr 172.21.15.152:0/761706629 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:26.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.660+0000 7f56fea00640 1 --2- 172.21.15.152:0/761706629 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f57000d33d0 0x7f57000cc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:26.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.660+0000 7f56f7e00640 1 -- 172.21.15.152:0/761706629 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f57000d33d0 msgr2=0x7f57000cc110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:26.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.660+0000 7f56f7e00640 1 --2- 172.21.15.152:0/761706629 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f57000d33d0 0x7f57000cc110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:26.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.660+0000 7f56f7e00640 1 -- 172.21.15.152:0/761706629 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5700163230 msgr2=0x7f5700156840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:26.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.660+0000 7f56ff400640 1 --2- 172.21.15.152:0/761706629 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5700163230 0x7f5700156840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:26.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.660+0000 7f56f7e00640 1 --2- 172.21.15.152:0/761706629 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5700163230 0x7f5700156840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:26.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.660+0000 7f56f7e00640 1 -- 172.21.15.152:0/761706629 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57000cd6f0 con 0x7f570015eb50 2024-04-20T11:36:26.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.661+0000 7f56f7e00640 1 --2- 172.21.15.152:0/761706629 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f570015eb50 0x7f5700158b60 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7f56e800e8a0 tx=0x7f56e800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:26.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.661+0000 7f56f6a00640 1 -- 172.21.15.152:0/761706629 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56e8017070 con 0x7f570015eb50 2024-04-20T11:36:26.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.661+0000 7f56f6a00640 1 -- 172.21.15.152:0/761706629 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56e800c940 con 0x7f570015eb50 2024-04-20T11:36:26.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.661+0000 7f56ffe00640 1 -- 172.21.15.152:0/761706629 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57000cd9d0 con 0x7f570015eb50 2024-04-20T11:36:26.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.661+0000 7f56f6a00640 1 -- 172.21.15.152:0/761706629 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56e800ccf0 con 0x7f570015eb50 2024-04-20T11:36:26.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.661+0000 7f56ffe00640 1 -- 172.21.15.152:0/761706629 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57000b44f0 con 0x7f570015eb50 2024-04-20T11:36:26.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.662+0000 7f56ffe00640 1 -- 172.21.15.152:0/761706629 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5700075e60 con 0x7f570015eb50 2024-04-20T11:36:26.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.663+0000 7f56f6a00640 1 -- 172.21.15.152:0/761706629 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 22) ==== 91601+0+0 (secure 0 0 0) 0x7f56e801a070 con 0x7f570015eb50 2024-04-20T11:36:26.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.665+0000 7f56f6a00640 1 --2- 172.21.15.152:0/761706629 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f56dc06cf20 0x7f56dc06f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:26.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.665+0000 7f56f6a00640 1 -- 172.21.15.152:0/761706629 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5709+0+0 (secure 0 0 0) 0x7f56e808cae0 con 0x7f570015eb50 2024-04-20T11:36:26.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.665+0000 7f56fea00640 1 --2- 172.21.15.152:0/761706629 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f56dc06cf20 0x7f56dc06f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:26.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.666+0000 7f56fea00640 1 --2- 172.21.15.152:0/761706629 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f56dc06cf20 0x7f56dc06f3e0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f57000b9c80 tx=0x7f56f0009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:26.670 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.669+0000 7f56f6a00640 1 -- 172.21.15.152:0/761706629 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f56e805a590 con 0x7f570015eb50 2024-04-20T11:36:26.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.872+0000 7f56ffe00640 1 -- 172.21.15.152:0/761706629 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f570008c680 con 0x7f570015eb50 2024-04-20T11:36:26.873 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.873+0000 7f56f6a00640 1 -- 172.21.15.152:0/761706629 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v26)=0 v26) ==== 122+0+0 (secure 0 0 0) 0x7f56e8059e90 con 0x7f570015eb50 2024-04-20T11:36:26.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.875+0000 7f56ffe00640 1 -- 172.21.15.152:0/761706629 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f56dc06cf20 msgr2=0x7f56dc06f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:26.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.875+0000 7f56ffe00640 1 --2- 172.21.15.152:0/761706629 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f56dc06cf20 0x7f56dc06f3e0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f57000b9c80 tx=0x7f56f0009040 comp rx=0 tx=0).stop 2024-04-20T11:36:26.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.875+0000 7f56ffe00640 1 -- 172.21.15.152:0/761706629 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f570015eb50 msgr2=0x7f5700158b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:26.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.875+0000 7f56ffe00640 1 --2- 172.21.15.152:0/761706629 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f570015eb50 0x7f5700158b60 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7f56e800e8a0 tx=0x7f56e800ed60 comp rx=0 tx=0).stop 2024-04-20T11:36:26.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.875+0000 7f56ffe00640 1 -- 172.21.15.152:0/761706629 shutdown_connections 2024-04-20T11:36:26.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.875+0000 7f56ffe00640 1 --2- 172.21.15.152:0/761706629 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f56dc06cf20 0x7f56dc06f3e0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:26.877 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.875+0000 7f56ffe00640 1 --2- 172.21.15.152:0/761706629 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5700163230 0x7f5700156840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:26.877 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.875+0000 7f56ffe00640 1 --2- 172.21.15.152:0/761706629 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f570015eb50 0x7f5700158b60 unknown :-1 s=CLOSED pgs=2014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:26.877 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.875+0000 7f56ffe00640 1 --2- 172.21.15.152:0/761706629 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f57000d33d0 0x7f57000cc110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:26.877 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.875+0000 7f56ffe00640 1 -- 172.21.15.152:0/761706629 >> 172.21.15.152:0/761706629 conn(0x7f570008bfd0 msgr2=0x7f570015ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:26.877 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.876+0000 7f56ffe00640 1 -- 172.21.15.152:0/761706629 shutdown_connections 2024-04-20T11:36:26.877 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:26.876+0000 7f56ffe00640 1 -- 172.21.15.152:0/761706629 wait complete. 2024-04-20T11:36:26.894 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:36:26.894 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p ./volumes/_nogroup/subvol55180) 2024-04-20T11:36:26.996 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:36:26.996 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p ./volumes/subvol_grp75600/subvol41321) 2024-04-20T11:36:27.084 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol55180 2024-04-20T11:36:27.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.283+0000 7f494b200640 1 Processor -- start 2024-04-20T11:36:27.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.283+0000 7f494b200640 1 -- start start 2024-04-20T11:36:27.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.285+0000 7f494b200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f4944071cd0 0x7f49440720b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:27.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.285+0000 7f494b200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f4944160220 0x7f49441606b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:27.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.285+0000 7f494b200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f494406eb30 0x7f494406efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:27.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.285+0000 7f494b200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f494413f8e0 con 0x7f4944071cd0 2024-04-20T11:36:27.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.285+0000 7f494b200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f49440725f0 con 0x7f4944160220 2024-04-20T11:36:27.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.285+0000 7f494b200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4944160f90 con 0x7f494406eb30 2024-04-20T11:36:27.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.286+0000 7f4949e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f4944071cd0 0x7f49440720b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:27.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.286+0000 7f4949e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f4944071cd0 0x7f49440720b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49518/0 (socket says 172.21.15.152:49518) 2024-04-20T11:36:27.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.286+0000 7f4949e00640 1 -- 172.21.15.152:0/3146849596 learned_addr learned my addr 172.21.15.152:0/3146849596 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:27.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.286+0000 7f494a800640 1 --2- 172.21.15.152:0/3146849596 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f494406eb30 0x7f494406efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:27.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.286+0000 7f4949400640 1 --2- 172.21.15.152:0/3146849596 >> v2:172.21.15.175:3300/0 conn(0x7f4944160220 0x7f49441606b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:27.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.286+0000 7f4949e00640 1 -- 172.21.15.152:0/3146849596 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f494406eb30 msgr2=0x7f494406efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:27.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.286+0000 7f4949e00640 1 --2- 172.21.15.152:0/3146849596 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f494406eb30 0x7f494406efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:27.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.286+0000 7f4949e00640 1 -- 172.21.15.152:0/3146849596 >> v2:172.21.15.175:3300/0 conn(0x7f4944160220 msgr2=0x7f49441606b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:27.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.286+0000 7f4949e00640 1 --2- 172.21.15.152:0/3146849596 >> v2:172.21.15.175:3300/0 conn(0x7f4944160220 0x7f49441606b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:27.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.286+0000 7f4949e00640 1 -- 172.21.15.152:0/3146849596 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f494406dd90 con 0x7f4944071cd0 2024-04-20T11:36:27.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.288+0000 7f4949e00640 1 --2- 172.21.15.152:0/3146849596 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4944071cd0 0x7f49440720b0 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x7f49440cfd00 tx=0x7f493c02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=222111e8c9b9cd03 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:27.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.288+0000 7f4948a00640 1 -- 172.21.15.152:0/3146849596 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f493c02c9e0 con 0x7f4944071cd0 2024-04-20T11:36:27.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.288+0000 7f4948a00640 1 -- 172.21.15.152:0/3146849596 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f493c02cb80 con 0x7f4944071cd0 2024-04-20T11:36:27.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.288+0000 7f4948a00640 1 -- 172.21.15.152:0/3146849596 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f493c0343f0 con 0x7f4944071cd0 2024-04-20T11:36:27.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.289+0000 7f494b200640 1 -- 172.21.15.152:0/3146849596 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4944071cd0 msgr2=0x7f49440720b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:27.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.289+0000 7f494b200640 1 --2- 172.21.15.152:0/3146849596 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4944071cd0 0x7f49440720b0 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x7f49440cfd00 tx=0x7f493c02be40 comp rx=0 tx=0).stop 2024-04-20T11:36:27.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.289+0000 7f494b200640 1 -- 172.21.15.152:0/3146849596 shutdown_connections 2024-04-20T11:36:27.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.289+0000 7f494b200640 1 --2- 172.21.15.152:0/3146849596 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f494406eb30 0x7f494406efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:27.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.289+0000 7f494b200640 1 --2- 172.21.15.152:0/3146849596 >> v2:172.21.15.175:3300/0 conn(0x7f4944160220 0x7f49441606b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:27.291 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.289+0000 7f494b200640 1 --2- 172.21.15.152:0/3146849596 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4944071cd0 0x7f49440720b0 unknown :-1 s=CLOSED pgs=2015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:27.291 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.289+0000 7f494b200640 1 -- 172.21.15.152:0/3146849596 >> 172.21.15.152:0/3146849596 conn(0x7f4944067ae0 msgr2=0x7f494406b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:27.291 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.289+0000 7f494b200640 1 -- 172.21.15.152:0/3146849596 shutdown_connections 2024-04-20T11:36:27.291 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.289+0000 7f494b200640 1 -- 172.21.15.152:0/3146849596 wait complete. 2024-04-20T11:36:27.292 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.291+0000 7f494b200640 1 Processor -- start 2024-04-20T11:36:27.292 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.291+0000 7f494b200640 1 -- start start 2024-04-20T11:36:27.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.293+0000 7f494b200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f494406eb30 0x7f49441f8800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:27.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.293+0000 7f494b200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4944160220 0x7f4944150930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:27.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.293+0000 7f494b200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49440a9530 0x7f494415b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:27.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.293+0000 7f494b200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f494406e630 con 0x7f4944160220 2024-04-20T11:36:27.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.293+0000 7f494b200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f494406e350 con 0x7f49440a9530 2024-04-20T11:36:27.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.293+0000 7f494b200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4944160bf0 con 0x7f494406eb30 2024-04-20T11:36:27.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.293+0000 7f4949400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4944160220 0x7f4944150930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:27.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.293+0000 7f4949400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4944160220 0x7f4944150930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49520/0 (socket says 172.21.15.152:49520) 2024-04-20T11:36:27.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.293+0000 7f4949400640 1 -- 172.21.15.152:0/1852137774 learned_addr learned my addr 172.21.15.152:0/1852137774 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:27.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.293+0000 7f4949e00640 1 --2- 172.21.15.152:0/1852137774 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f494406eb30 0x7f49441f8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:27.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.293+0000 7f494a800640 1 --2- 172.21.15.152:0/1852137774 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49440a9530 0x7f494415b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:27.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.293+0000 7f4949400640 1 -- 172.21.15.152:0/1852137774 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f494406eb30 msgr2=0x7f49441f8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:27.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.293+0000 7f4949400640 1 --2- 172.21.15.152:0/1852137774 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f494406eb30 0x7f49441f8800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:27.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.293+0000 7f4949400640 1 -- 172.21.15.152:0/1852137774 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49440a9530 msgr2=0x7f494415b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:27.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.293+0000 7f4949400640 1 --2- 172.21.15.152:0/1852137774 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49440a9530 0x7f494415b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:27.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.293+0000 7f4949400640 1 -- 172.21.15.152:0/1852137774 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49440b5140 con 0x7f4944160220 2024-04-20T11:36:27.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.293+0000 7f4949e00640 1 --2- 172.21.15.152:0/1852137774 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f494406eb30 0x7f49441f8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:36:27.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.293+0000 7f494a800640 1 --2- 172.21.15.152:0/1852137774 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49440a9530 0x7f494415b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:36:27.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.294+0000 7f4949400640 1 --2- 172.21.15.152:0/1852137774 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4944160220 0x7f4944150930 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7f493400d5f0 tx=0x7f493400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:27.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.294+0000 7f493a000640 1 -- 172.21.15.152:0/1852137774 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49340125f0 con 0x7f4944160220 2024-04-20T11:36:27.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.294+0000 7f493a000640 1 -- 172.21.15.152:0/1852137774 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4934019070 con 0x7f4944160220 2024-04-20T11:36:27.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.294+0000 7f493a000640 1 -- 172.21.15.152:0/1852137774 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49340129d0 con 0x7f4944160220 2024-04-20T11:36:27.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.294+0000 7f494b200640 1 -- 172.21.15.152:0/1852137774 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49440a9a30 con 0x7f4944160220 2024-04-20T11:36:27.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.294+0000 7f494b200640 1 -- 172.21.15.152:0/1852137774 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4944152550 con 0x7f4944160220 2024-04-20T11:36:27.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.295+0000 7f494b200640 1 -- 172.21.15.152:0/1852137774 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4944075e60 con 0x7f4944160220 2024-04-20T11:36:27.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.296+0000 7f493a000640 1 -- 172.21.15.152:0/1852137774 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 22) ==== 91601+0+0 (secure 0 0 0) 0x7f4934012b70 con 0x7f4944160220 2024-04-20T11:36:27.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.297+0000 7f493a000640 1 --2- 172.21.15.152:0/1852137774 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f492806cf50 0x7f492806f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:27.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.297+0000 7f493a000640 1 -- 172.21.15.152:0/1852137774 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5709+0+0 (secure 0 0 0) 0x7f493408c9a0 con 0x7f4944160220 2024-04-20T11:36:27.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.298+0000 7f4949e00640 1 --2- 172.21.15.152:0/1852137774 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f492806cf50 0x7f492806f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:27.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.299+0000 7f4949e00640 1 --2- 172.21.15.152:0/1852137774 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f492806cf50 0x7f492806f410 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f493c035000 tx=0x7f493c030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:27.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.305+0000 7f493a000640 1 -- 172.21.15.152:0/1852137774 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f493405a520 con 0x7f4944160220 2024-04-20T11:36:27.532 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.532+0000 7f494b200640 1 -- 172.21.15.152:0/1852137774 --> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol55180", "target": ["mon-mgr", ""]}) -- 0x7f49440a9d10 con 0x7f492806cf50 2024-04-20T11:36:27.545 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:36:27.545+0000 7f8a5cc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:36:27.545 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:36:27.545+0000 7f8a5cc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:36:27.545 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:36:27.545+0000 7f8a5cc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:36:27.546 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:36:27.545+0000 7f8a5cc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:36:27.546 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:36:27.545+0000 7f8a5cc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:36:27.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.588+0000 7f493a000640 1 -- 172.21.15.152:0/1852137774 <== mgr.4109 v2:172.21.15.175:6836/4021250441 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+30 (secure 0 0 0) 0x7f49440a9d10 con 0x7f492806cf50 2024-04-20T11:36:27.589 INFO:teuthology.orchestra.run.smithi152.stdout:/volumes/_nogroup/subvol55180 2024-04-20T11:36:27.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.590+0000 7f494b200640 1 -- 172.21.15.152:0/1852137774 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f492806cf50 msgr2=0x7f492806f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:27.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.590+0000 7f494b200640 1 --2- 172.21.15.152:0/1852137774 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f492806cf50 0x7f492806f410 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f493c035000 tx=0x7f493c030000 comp rx=0 tx=0).stop 2024-04-20T11:36:27.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.590+0000 7f494b200640 1 -- 172.21.15.152:0/1852137774 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4944160220 msgr2=0x7f4944150930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:27.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.590+0000 7f494b200640 1 --2- 172.21.15.152:0/1852137774 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4944160220 0x7f4944150930 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7f493400d5f0 tx=0x7f493400dab0 comp rx=0 tx=0).stop 2024-04-20T11:36:27.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.590+0000 7f494b200640 1 -- 172.21.15.152:0/1852137774 shutdown_connections 2024-04-20T11:36:27.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.591+0000 7f494b200640 1 --2- 172.21.15.152:0/1852137774 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f492806cf50 0x7f492806f410 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:27.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.591+0000 7f494b200640 1 --2- 172.21.15.152:0/1852137774 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49440a9530 0x7f494415b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:27.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.591+0000 7f494b200640 1 --2- 172.21.15.152:0/1852137774 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4944160220 0x7f4944150930 unknown :-1 s=CLOSED pgs=2016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:27.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.591+0000 7f494b200640 1 --2- 172.21.15.152:0/1852137774 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f494406eb30 0x7f49441f8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:27.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.591+0000 7f494b200640 1 -- 172.21.15.152:0/1852137774 >> 172.21.15.152:0/1852137774 conn(0x7f4944067ae0 msgr2=0x7f494415f400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:27.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.591+0000 7f494b200640 1 -- 172.21.15.152:0/1852137774 shutdown_connections 2024-04-20T11:36:27.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.591+0000 7f494b200640 1 -- 172.21.15.152:0/1852137774 wait complete. 2024-04-20T11:36:27.611 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol41321 subvol_grp75600 2024-04-20T11:36:27.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.805+0000 7f49b9200640 1 Processor -- start 2024-04-20T11:36:27.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.806+0000 7f49b9200640 1 -- start start 2024-04-20T11:36:27.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.807+0000 7f49b9200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f49b40d06a0 0x7f49b40d1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:27.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.807+0000 7f49b9200640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49b415ec50 con 0x7f49b415e870 2024-04-20T11:36:27.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.807+0000 7f49b9200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49b40d22d0 con 0x7f49b40d0fa0 2024-04-20T11:36:27.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.807+0000 7f49b9200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f49b40d2ee0 con 0x7f49b40d06a0 2024-04-20T11:36:27.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.807+0000 7f49b3e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f49b40d06a0 0x7f49b40d1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:27.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.807+0000 7f49b3e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f49b40d06a0 0x7f49b40d1450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49530/0 (socket says 172.21.15.152:49530) 2024-04-20T11:36:27.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.807+0000 7f49b3e00640 1 -- 172.21.15.152:0/3489322527 learned_addr learned my addr 172.21.15.152:0/3489322527 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:27.809 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.808+0000 7f49b3e00640 1 -- 172.21.15.152:0/3489322527 >> v1:172.21.15.175:6789/0 conn(0x7f49b40d0fa0 legacy=0x7f49b40cff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:27.809 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.808+0000 7f49b3e00640 1 -- 172.21.15.152:0/3489322527 >> v1:172.21.15.152:6789/0 conn(0x7f49b415e870 legacy=0x7f49b40d3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:27.809 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.808+0000 7f49b3e00640 1 -- 172.21.15.152:0/3489322527 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49b40d25b0 con 0x7f49b40d06a0 2024-04-20T11:36:27.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.809+0000 7f49b3e00640 1 --2- 172.21.15.152:0/3489322527 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f49b40d06a0 0x7f49b40d1450 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7f49ac009e50 tx=0x7f49ac02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bab0766a43a4738 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:27.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.809+0000 7f49b2a00640 1 -- 172.21.15.152:0/3489322527 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49ac02ca80 con 0x7f49b40d06a0 2024-04-20T11:36:27.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.809+0000 7f49b2a00640 1 -- 172.21.15.152:0/3489322527 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49ac02cc20 con 0x7f49b40d06a0 2024-04-20T11:36:27.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.809+0000 7f49b2a00640 1 -- 172.21.15.152:0/3489322527 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49ac0343f0 con 0x7f49b40d06a0 2024-04-20T11:36:27.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.810+0000 7f49b9200640 1 -- 172.21.15.152:0/3489322527 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f49b40d06a0 msgr2=0x7f49b40d1450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:27.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.810+0000 7f49b9200640 1 --2- 172.21.15.152:0/3489322527 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f49b40d06a0 0x7f49b40d1450 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7f49ac009e50 tx=0x7f49ac02bee0 comp rx=0 tx=0).stop 2024-04-20T11:36:27.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.810+0000 7f49b9200640 1 -- 172.21.15.152:0/3489322527 shutdown_connections 2024-04-20T11:36:27.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.810+0000 7f49b9200640 1 --2- 172.21.15.152:0/3489322527 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f49b40d06a0 0x7f49b40d1450 unknown :-1 s=CLOSED pgs=2018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:27.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.810+0000 7f49b9200640 1 -- 172.21.15.152:0/3489322527 >> 172.21.15.152:0/3489322527 conn(0x7f49b408bfd0 msgr2=0x7f49b40d45d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:27.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.810+0000 7f49b9200640 1 -- 172.21.15.152:0/3489322527 shutdown_connections 2024-04-20T11:36:27.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.810+0000 7f49b9200640 1 -- 172.21.15.152:0/3489322527 wait complete. 2024-04-20T11:36:27.812 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.812+0000 7f49b9200640 1 Processor -- start 2024-04-20T11:36:27.812 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.812+0000 7f49b9200640 1 -- start start 2024-04-20T11:36:27.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.813+0000 7f49b9200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f49b40d06a0 0x7f49b4152c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:27.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.813+0000 7f49b9200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f49b40d0fa0 0x7f49b41f5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:27.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.813+0000 7f49b9200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49b415e870 0x7f49b41606e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:27.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.813+0000 7f49b9200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f49b406ca80 con 0x7f49b40d0fa0 2024-04-20T11:36:27.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.813+0000 7f49b9200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f49b40d2b70 con 0x7f49b415e870 2024-04-20T11:36:27.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.813+0000 7f49b9200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f49b406cc00 con 0x7f49b40d06a0 2024-04-20T11:36:27.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.813+0000 7f49abe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f49b40d0fa0 0x7f49b41f5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:27.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.813+0000 7f49abe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f49b40d0fa0 0x7f49b41f5470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49546/0 (socket says 172.21.15.152:49546) 2024-04-20T11:36:27.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.813+0000 7f49abe00640 1 -- 172.21.15.152:0/2505119945 learned_addr learned my addr 172.21.15.152:0/2505119945 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:27.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.814+0000 7f49b3e00640 1 --2- 172.21.15.152:0/2505119945 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49b415e870 0x7f49b41606e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:27.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.814+0000 7f49abe00640 1 -- 172.21.15.152:0/2505119945 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f49b40d06a0 msgr2=0x7f49b4152c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:27.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.814+0000 7f49b3400640 1 --2- 172.21.15.152:0/2505119945 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f49b40d06a0 0x7f49b4152c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:27.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.814+0000 7f49abe00640 1 --2- 172.21.15.152:0/2505119945 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f49b40d06a0 0x7f49b4152c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:27.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.814+0000 7f49abe00640 1 -- 172.21.15.152:0/2505119945 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49b415e870 msgr2=0x7f49b41606e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:27.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.814+0000 7f49abe00640 1 --2- 172.21.15.152:0/2505119945 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49b415e870 0x7f49b41606e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:27.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.814+0000 7f49abe00640 1 -- 172.21.15.152:0/2505119945 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49b40b8020 con 0x7f49b40d0fa0 2024-04-20T11:36:27.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.814+0000 7f49b3e00640 1 --2- 172.21.15.152:0/2505119945 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49b415e870 0x7f49b41606e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:36:27.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.814+0000 7f49b3400640 1 --2- 172.21.15.152:0/2505119945 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f49b40d06a0 0x7f49b4152c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:36:27.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.814+0000 7f49abe00640 1 --2- 172.21.15.152:0/2505119945 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f49b40d0fa0 0x7f49b41f5470 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 crypto rx=0x7f499c00c450 tx=0x7f499c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:27.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.814+0000 7f49ab400640 1 -- 172.21.15.152:0/2505119945 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f499c016020 con 0x7f49b40d0fa0 2024-04-20T11:36:27.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.814+0000 7f49ab400640 1 -- 172.21.15.152:0/2505119945 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f499c012830 con 0x7f49b40d0fa0 2024-04-20T11:36:27.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.814+0000 7f49ab400640 1 -- 172.21.15.152:0/2505119945 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f499c012c20 con 0x7f49b40d0fa0 2024-04-20T11:36:27.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.814+0000 7f49b9200640 1 -- 172.21.15.152:0/2505119945 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49b415f920 con 0x7f49b40d0fa0 2024-04-20T11:36:27.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.815+0000 7f49b9200640 1 -- 172.21.15.152:0/2505119945 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49b415fee0 con 0x7f49b40d0fa0 2024-04-20T11:36:27.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.816+0000 7f49b9200640 1 -- 172.21.15.152:0/2505119945 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49b4075e60 con 0x7f49b40d0fa0 2024-04-20T11:36:27.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.817+0000 7f49ab400640 1 -- 172.21.15.152:0/2505119945 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 22) ==== 91601+0+0 (secure 0 0 0) 0x7f499c019020 con 0x7f49b40d0fa0 2024-04-20T11:36:27.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.817+0000 7f49ab400640 1 --2- 172.21.15.152:0/2505119945 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f499006cf20 0x7f499006f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:27.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.818+0000 7f49ab400640 1 -- 172.21.15.152:0/2505119945 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5709+0+0 (secure 0 0 0) 0x7f499c08c490 con 0x7f49b40d0fa0 2024-04-20T11:36:27.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.818+0000 7f49b3400640 1 --2- 172.21.15.152:0/2505119945 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f499006cf20 0x7f499006f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:27.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.819+0000 7f49b3400640 1 --2- 172.21.15.152:0/2505119945 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f499006cf20 0x7f499006f3e0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f49a40015e0 tx=0x7f49a4009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:27.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:27.822+0000 7f49ab400640 1 -- 172.21.15.152:0/2505119945 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f499c059fd0 con 0x7f49b40d0fa0 2024-04-20T11:36:28.000 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:28.000+0000 7f49ab400640 1 -- 172.21.15.152:0/2505119945 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mgrmap(e 23) ==== 91652+0+0 (secure 0 0 0) 0x7f499c055f40 con 0x7f49b40d0fa0 2024-04-20T11:36:28.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:28.053+0000 7f49b9200640 1 -- 172.21.15.152:0/2505119945 --> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol41321", "group_name": "subvol_grp75600", "target": ["mon-mgr", ""]}) -- 0x7f49b4162390 con 0x7f499006cf20 2024-04-20T11:36:28.098 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:28.098+0000 7f49ab400640 1 -- 172.21.15.152:0/2505119945 <== mgr.4109 v2:172.21.15.175:6836/4021250441 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f49b4162390 con 0x7f499006cf20 2024-04-20T11:36:28.098 INFO:teuthology.orchestra.run.smithi152.stdout:/volumes/subvol_grp75600/subvol41321 2024-04-20T11:36:28.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:28.100+0000 7f49b9200640 1 -- 172.21.15.152:0/2505119945 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f499006cf20 msgr2=0x7f499006f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:28.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:28.100+0000 7f49b9200640 1 --2- 172.21.15.152:0/2505119945 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f499006cf20 0x7f499006f3e0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f49a40015e0 tx=0x7f49a4009040 comp rx=0 tx=0).stop 2024-04-20T11:36:28.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:28.100+0000 7f49b9200640 1 -- 172.21.15.152:0/2505119945 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f49b40d0fa0 msgr2=0x7f49b41f5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:28.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:28.100+0000 7f49b9200640 1 --2- 172.21.15.152:0/2505119945 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f49b40d0fa0 0x7f49b41f5470 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 crypto rx=0x7f499c00c450 tx=0x7f499c00c910 comp rx=0 tx=0).stop 2024-04-20T11:36:28.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:28.100+0000 7f49b9200640 1 -- 172.21.15.152:0/2505119945 shutdown_connections 2024-04-20T11:36:28.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:28.100+0000 7f49b9200640 1 --2- 172.21.15.152:0/2505119945 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f499006cf20 0x7f499006f3e0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:28.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:28.100+0000 7f49b9200640 1 --2- 172.21.15.152:0/2505119945 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49b415e870 0x7f49b41606e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:28.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:28.100+0000 7f49b9200640 1 --2- 172.21.15.152:0/2505119945 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f49b40d0fa0 0x7f49b41f5470 unknown :-1 s=CLOSED pgs=2019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:28.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:28.100+0000 7f49b9200640 1 --2- 172.21.15.152:0/2505119945 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f49b40d06a0 0x7f49b4152c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:28.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:28.100+0000 7f49b9200640 1 -- 172.21.15.152:0/2505119945 >> 172.21.15.152:0/2505119945 conn(0x7f49b408bfd0 msgr2=0x7f49b40b3fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:28.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:28.100+0000 7f49b9200640 1 -- 172.21.15.152:0/2505119945 shutdown_connections 2024-04-20T11:36:28.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:28.100+0000 7f49b9200640 1 -- 172.21.15.152:0/2505119945 wait complete. 2024-04-20T11:36:28.119 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol55180 2024-04-20T11:36:28.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:28.305+0000 7f56aee00640 1 Processor -- start 2024-04-20T11:36:28.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:28.305+0000 7f56aee00640 1 -- start start 2024-04-20T11:36:28.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:28.307+0000 7f56aee00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f56a815eb50 0x7f56a80d0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:28.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:28.307+0000 7f56aee00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56a80d19c0 con 0x7f56a8163230 2024-04-20T11:36:28.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:28.307+0000 7f56aee00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56a8162750 con 0x7f56a80d35a0 2024-04-20T11:36:28.308 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:28.307+0000 7f56aee00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f56a8162a30 con 0x7f56a815eb50 2024-04-20T11:36:28.308 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:28.307+0000 7f56ad000640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f56a8163230 0x7f56a80d2e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:45324/0 (socket says 172.21.15.152:45324) 2024-04-20T11:36:28.308 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:28.307+0000 7f56ad000640 1 -- 172.21.15.152:0/1155521241 learned_addr learned my addr 172.21.15.152:0/1155521241 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:36:28.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:28.308+0000 7f56ada00640 1 --2- 172.21.15.152:0/1155521241 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f56a815eb50 0x7f56a80d0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:28.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:28.308+0000 7f56a7e00640 1 -- 172.21.15.152:0/1155521241 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1318474365 0 0) 0x7f56a80d19c0 con 0x7f56a8163230 2024-04-20T11:36:28.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:28.308+0000 7f56a7e00640 1 -- 172.21.15.152:0/1155521241 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5690003520 con 0x7f56a8163230 2024-04-20T11:36:28.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:28.308+0000 7f56a7e00640 1 -- 172.21.15.152:0/1155521241 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1288333396 0 0) 0x7f5690003520 con 0x7f56a8163230 2024-04-20T11:36:28.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:28.308+0000 7f56a7e00640 1 -- 172.21.15.152:0/1155521241 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f56a815eb50 msgr2=0x7f56a80d0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:28.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:28.308+0000 7f56a7e00640 1 --2- 172.21.15.152:0/1155521241 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f56a815eb50 0x7f56a80d0f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:28.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:28.308+0000 7f56a7e00640 1 -- 172.21.15.152:0/1155521241 >> v1:172.21.15.175:6789/0 conn(0x7f56a80d35a0 legacy=0x7f56a80cf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:28.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:28.308+0000 7f56a7e00640 1 -- 172.21.15.152:0/1155521241 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56a8162d10 con 0x7f56a8163230 2024-04-20T11:36:28.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:28.308+0000 7f56a7e00640 1 -- 172.21.15.152:0/1155521241 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f56940017f0 con 0x7f56a8163230 2024-04-20T11:36:28.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:28.308+0000 7f56a7e00640 1 -- 172.21.15.152:0/1155521241 >> v1:172.21.15.152:6789/0 conn(0x7f56a8163230 legacy=0x7f56a80d2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:28.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:28.308+0000 7f56a7e00640 1 --2- 172.21.15.152:0/1155521241 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f56900039e0 0x7f5690003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:28.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:28.308+0000 7f56a7e00640 1 --2- 172.21.15.152:0/1155521241 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5690004800 0x7f5690004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:28.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:28.308+0000 7f56a7e00640 1 --2- 172.21.15.152:0/1155521241 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f56a815eb50 0x7f5690005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:28.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:28.308+0000 7f56a7e00640 1 -- 172.21.15.152:0/1155521241 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f56a8162a30 con 0x7f5690004800 2024-04-20T11:36:28.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:28.308+0000 7f56a7e00640 1 -- 172.21.15.152:0/1155521241 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f569000baf0 con 0x7f56900039e0 2024-04-20T11:36:28.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:28.308+0000 7f56a7e00640 1 -- 172.21.15.152:0/1155521241 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f569000bcf0 con 0x7f56a815eb50 2024-04-20T11:36:28.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:28.309+0000 7f56ad000640 1 --2- 172.21.15.152:0/1155521241 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5690004800 0x7f5690004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:28.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:28.309+0000 7f56ada00640 1 --2- 172.21.15.152:0/1155521241 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f56900039e0 0x7f5690003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:28.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:28.309+0000 7f56ad000640 1 -- 172.21.15.152:0/1155521241 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f56a815eb50 msgr2=0x7f5690005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:28.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:28.309+0000 7f56ad000640 1 --2- 172.21.15.152:0/1155521241 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f56a815eb50 0x7f5690005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:28.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:28.309+0000 7f56ad000640 1 -- 172.21.15.152:0/1155521241 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f56900039e0 msgr2=0x7f5690003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:28.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:28.309+0000 7f56ad000640 1 --2- 172.21.15.152:0/1155521241 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f56900039e0 0x7f5690003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:28.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:28.309+0000 7f56ad000640 1 -- 172.21.15.152:0/1155521241 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5690003520 con 0x7f5690004800 2024-04-20T11:36:28.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:28.309+0000 7f56ada00640 1 --2- 172.21.15.152:0/1155521241 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f56900039e0 0x7f5690003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:36:28.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:28.310+0000 7f56ad000640 1 --2- 172.21.15.152:0/1155521241 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5690004800 0x7f5690004c80 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7f569400c240 tx=0x7f5690005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e142b8ea41ecead6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:28.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:28.310+0000 7f56a7e00640 1 -- 172.21.15.152:0/1155521241 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5694007ce0 con 0x7f5690004800 2024-04-20T11:36:28.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:28.310+0000 7f56a7e00640 1 -- 172.21.15.152:0/1155521241 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5694030070 con 0x7f5690004800 2024-04-20T11:36:28.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:28.310+0000 7f56a7e00640 1 -- 172.21.15.152:0/1155521241 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56940074d0 con 0x7f5690004800 2024-04-20T11:36:28.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:28.311+0000 7f56aee00640 1 -- 172.21.15.152:0/1155521241 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5690004800 msgr2=0x7f5690004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:28.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:28.311+0000 7f56aee00640 1 --2- 172.21.15.152:0/1155521241 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5690004800 0x7f5690004c80 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7f569400c240 tx=0x7f5690005830 comp rx=0 tx=0).stop 2024-04-20T11:36:28.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:28.311+0000 7f56ae400640 1 -- 172.21.15.152:0/1155521241 reap_dead start 2024-04-20T11:36:28.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:28.311+0000 7f56aee00640 1 -- 172.21.15.152:0/1155521241 shutdown_connections 2024-04-20T11:36:28.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:28.311+0000 7f56aee00640 1 -- 172.21.15.152:0/1155521241 >> 172.21.15.152:0/1155521241 conn(0x7f56a808bfd0 msgr2=0x7f56a80d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:28.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:28.311+0000 7f56aee00640 1 -- 172.21.15.152:0/1155521241 shutdown_connections 2024-04-20T11:36:28.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:28.311+0000 7f56aee00640 1 -- 172.21.15.152:0/1155521241 wait complete. 2024-04-20T11:36:28.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:28.313+0000 7f56aee00640 1 Processor -- start 2024-04-20T11:36:28.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:28.313+0000 7f56aee00640 1 -- start start 2024-04-20T11:36:28.315 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:28.314+0000 7f56aee00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5690004800 0x7f56a8159a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:28.315 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:28.314+0000 7f56aee00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f56a815f3f0 0x7f56a815f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:28.315 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:28.314+0000 7f56aee00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f56a80b5f10 0x7f56a80b6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:28.315 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:28.314+0000 7f56aee00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f56a80d1080 con 0x7f56a80b5f10 2024-04-20T11:36:28.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:28.314+0000 7f56aee00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f56a80d0f00 con 0x7f5690004800 2024-04-20T11:36:28.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:28.314+0000 7f56aee00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f56a80d1200 con 0x7f56a815f3f0 2024-04-20T11:36:28.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:28.314+0000 7f56ae400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f56a80b5f10 0x7f56a80b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:28.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:28.314+0000 7f56ae400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f56a80b5f10 0x7f56a80b6390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49572/0 (socket says 172.21.15.152:49572) 2024-04-20T11:36:28.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:28.314+0000 7f56ae400640 1 -- 172.21.15.152:0/638556614 learned_addr learned my addr 172.21.15.152:0/638556614 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:28.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:28.315+0000 7f56ada00640 1 --2- 172.21.15.152:0/638556614 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5690004800 0x7f56a8159a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:28.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:28.315+0000 7f56ad000640 1 --2- 172.21.15.152:0/638556614 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f56a815f3f0 0x7f56a815f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:28.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:28.315+0000 7f56ae400640 1 -- 172.21.15.152:0/638556614 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f56a815f3f0 msgr2=0x7f56a815f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:28.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:28.315+0000 7f56ae400640 1 --2- 172.21.15.152:0/638556614 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f56a815f3f0 0x7f56a815f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:28.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:28.315+0000 7f56ae400640 1 -- 172.21.15.152:0/638556614 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5690004800 msgr2=0x7f56a8159a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:28.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:28.315+0000 7f56ae400640 1 --2- 172.21.15.152:0/638556614 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5690004800 0x7f56a8159a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:28.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:28.315+0000 7f56ae400640 1 -- 172.21.15.152:0/638556614 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56a80b6ad0 con 0x7f56a80b5f10 2024-04-20T11:36:28.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:28.315+0000 7f56ada00640 1 --2- 172.21.15.152:0/638556614 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5690004800 0x7f56a8159a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:36:28.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:28.315+0000 7f56ae400640 1 --2- 172.21.15.152:0/638556614 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f56a80b5f10 0x7f56a80b6390 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7f56a80ceb60 tx=0x7f56a000cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:28.317 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:28.315+0000 7f56a5600640 1 -- 172.21.15.152:0/638556614 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56a0005d40 con 0x7f56a80b5f10 2024-04-20T11:36:28.317 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:28.315+0000 7f56a5600640 1 -- 172.21.15.152:0/638556614 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56a000a610 con 0x7f56a80b5f10 2024-04-20T11:36:28.317 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:28.315+0000 7f56a5600640 1 -- 172.21.15.152:0/638556614 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56a000aa00 con 0x7f56a80b5f10 2024-04-20T11:36:28.317 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:28.315+0000 7f56aee00640 1 -- 172.21.15.152:0/638556614 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56a80b6db0 con 0x7f56a80b5f10 2024-04-20T11:36:28.318 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:28.315+0000 7f56aee00640 1 -- 172.21.15.152:0/638556614 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56a80b7410 con 0x7f56a80b5f10 2024-04-20T11:36:28.318 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:28.317+0000 7f56aee00640 1 -- 172.21.15.152:0/638556614 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56a80735a0 con 0x7f56a80b5f10 2024-04-20T11:36:28.318 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:28.318+0000 7f56a5600640 1 -- 172.21.15.152:0/638556614 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 23) ==== 91652+0+0 (secure 0 0 0) 0x7f56a0011070 con 0x7f56a80b5f10 2024-04-20T11:36:28.319 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:28.319+0000 7f56a5600640 1 --2- 172.21.15.152:0/638556614 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f568806cfb0 0x7f568806f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:28.319 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:28.319+0000 7f56a5600640 1 -- 172.21.15.152:0/638556614 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5709+0+0 (secure 0 0 0) 0x7f56a008b460 con 0x7f56a80b5f10 2024-04-20T11:36:28.319 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:28.319+0000 7f56ada00640 1 --2- 172.21.15.152:0/638556614 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f568806cfb0 0x7f568806f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:28.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:28.320+0000 7f56ada00640 1 --2- 172.21.15.152:0/638556614 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f568806cfb0 0x7f568806f470 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f569c009ac0 tx=0x7f569c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:28.324 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:28.323+0000 7f56a5600640 1 -- 172.21.15.152:0/638556614 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f56a80735a0 con 0x7f56a80b5f10 2024-04-20T11:36:28.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:28.553+0000 7f56aee00640 1 -- 172.21.15.152:0/638556614 --> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol55180", "target": ["mon-mgr", ""]}) -- 0x7f56a8075e60 con 0x7f568806cfb0 2024-04-20T11:36:28.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:28.564+0000 7f56a5600640 1 -- 172.21.15.152:0/638556614 <== mgr.4109 v2:172.21.15.175:6836/4021250441 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+30 (secure 0 0 0) 0x7f56a8075e60 con 0x7f568806cfb0 2024-04-20T11:36:28.565 INFO:teuthology.orchestra.run.smithi152.stdout:/volumes/_nogroup/subvol55180 2024-04-20T11:36:28.567 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:28.566+0000 7f56aee00640 1 -- 172.21.15.152:0/638556614 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f568806cfb0 msgr2=0x7f568806f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:28.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:28.566+0000 7f56aee00640 1 --2- 172.21.15.152:0/638556614 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f568806cfb0 0x7f568806f470 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f569c009ac0 tx=0x7f569c008040 comp rx=0 tx=0).stop 2024-04-20T11:36:28.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:28.566+0000 7f56aee00640 1 -- 172.21.15.152:0/638556614 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f56a80b5f10 msgr2=0x7f56a80b6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:28.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:28.566+0000 7f56aee00640 1 --2- 172.21.15.152:0/638556614 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f56a80b5f10 0x7f56a80b6390 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7f56a80ceb60 tx=0x7f56a000cb90 comp rx=0 tx=0).stop 2024-04-20T11:36:28.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:28.567+0000 7f56aee00640 1 -- 172.21.15.152:0/638556614 shutdown_connections 2024-04-20T11:36:28.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:28.567+0000 7f56aee00640 1 --2- 172.21.15.152:0/638556614 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f568806cfb0 0x7f568806f470 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:28.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:28.567+0000 7f56aee00640 1 --2- 172.21.15.152:0/638556614 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f56a80b5f10 0x7f56a80b6390 unknown :-1 s=CLOSED pgs=2022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:28.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:28.567+0000 7f56aee00640 1 --2- 172.21.15.152:0/638556614 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f56a815f3f0 0x7f56a815f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:28.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:28.567+0000 7f56aee00640 1 --2- 172.21.15.152:0/638556614 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5690004800 0x7f56a8159a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:28.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:28.567+0000 7f56aee00640 1 -- 172.21.15.152:0/638556614 >> 172.21.15.152:0/638556614 conn(0x7f56a808bfd0 msgr2=0x7f56a80b2bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:28.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:28.567+0000 7f56aee00640 1 -- 172.21.15.152:0/638556614 shutdown_connections 2024-04-20T11:36:28.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:28.567+0000 7f56aee00640 1 -- 172.21.15.152:0/638556614 wait complete. 2024-04-20T11:36:28.585 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:36:28.585 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_legacy/5e9677b37d03af36a84ac6e56f12dd21.meta) 2024-04-20T11:36:29.056 INFO:teuthology.orchestra.run.smithi152.stdout:[GLOBAL] 2024-04-20T11:36:29.056 INFO:teuthology.orchestra.run.smithi152.stdout:version = 1 2024-04-20T11:36:29.056 INFO:teuthology.orchestra.run.smithi152.stdout:type = subvolume 2024-04-20T11:36:29.057 INFO:teuthology.orchestra.run.smithi152.stdout:path = /volumes/_nogroup/subvol55180 2024-04-20T11:36:29.057 INFO:teuthology.orchestra.run.smithi152.stdout:state = complete 2024-04-20T11:36:29.057 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:36:29.063 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol41321 subvol_grp75600 2024-04-20T11:36:29.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:29.275+0000 7fa288a00640 1 Processor -- start 2024-04-20T11:36:29.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:29.275+0000 7fa288a00640 1 -- start start 2024-04-20T11:36:29.277 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:29.276+0000 7fa288a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa284144fa0 0x7fa2840b3470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:29.277 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:29.277+0000 7fa288a00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2840b3a50 con 0x7fa284145620 2024-04-20T11:36:29.277 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:29.277+0000 7fa288a00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2840b3d30 con 0x7fa284072530 2024-04-20T11:36:29.277 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:29.277+0000 7fa288a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa284145a80 con 0x7fa284144fa0 2024-04-20T11:36:29.277 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:29.277+0000 7fa279400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fa284145620 0x7fa284071e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:45340/0 (socket says 172.21.15.152:45340) 2024-04-20T11:36:29.277 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:29.277+0000 7fa279400640 1 -- 172.21.15.152:0/2492297578 learned_addr learned my addr 172.21.15.152:0/2492297578 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:36:29.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:29.277+0000 7fa281e00640 1 --2- 172.21.15.152:0/2492297578 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa284144fa0 0x7fa2840b3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:29.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:29.277+0000 7fa281400640 1 -- 172.21.15.152:0/2492297578 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2001307949 0 0) 0x7fa2840b3a50 con 0x7fa284145620 2024-04-20T11:36:29.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:29.277+0000 7fa281400640 1 -- 172.21.15.152:0/2492297578 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa268003520 con 0x7fa284145620 2024-04-20T11:36:29.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:29.278+0000 7fa281400640 1 -- 172.21.15.152:0/2492297578 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2122612850 0 0) 0x7fa268003520 con 0x7fa284145620 2024-04-20T11:36:29.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:29.278+0000 7fa281400640 1 -- 172.21.15.152:0/2492297578 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa284144fa0 msgr2=0x7fa2840b3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:29.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:29.278+0000 7fa281400640 1 --2- 172.21.15.152:0/2492297578 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa284144fa0 0x7fa2840b3470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:29.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:29.278+0000 7fa281400640 1 -- 172.21.15.152:0/2492297578 >> v1:172.21.15.175:6789/0 conn(0x7fa284072530 legacy=0x7fa2840af100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:29.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:29.278+0000 7fa281400640 1 -- 172.21.15.152:0/2492297578 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2840b4010 con 0x7fa284145620 2024-04-20T11:36:29.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:29.278+0000 7fa281400640 1 -- 172.21.15.152:0/2492297578 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7fa26c001670 con 0x7fa284145620 2024-04-20T11:36:29.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:29.278+0000 7fa281400640 1 -- 172.21.15.152:0/2492297578 >> v1:172.21.15.152:6789/0 conn(0x7fa284145620 legacy=0x7fa284071e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:29.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:29.278+0000 7fa281400640 1 --2- 172.21.15.152:0/2492297578 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa268003a50 0x7fa268003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:29.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:29.278+0000 7fa281400640 1 --2- 172.21.15.152:0/2492297578 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa284144fa0 0x7fa268004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:29.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:29.278+0000 7fa281400640 1 --2- 172.21.15.152:0/2492297578 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa268004e40 0x7fa2680052c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:29.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:29.278+0000 7fa281400640 1 -- 172.21.15.152:0/2492297578 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fa284145a80 con 0x7fa268003a50 2024-04-20T11:36:29.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:29.278+0000 7fa281400640 1 -- 172.21.15.152:0/2492297578 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa26800bd00 con 0x7fa268004e40 2024-04-20T11:36:29.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:29.278+0000 7fa281400640 1 -- 172.21.15.152:0/2492297578 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa26800bf00 con 0x7fa284144fa0 2024-04-20T11:36:29.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:29.278+0000 7fa281e00640 1 --2- 172.21.15.152:0/2492297578 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa268003a50 0x7fa268003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:29.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:29.279+0000 7fa281e00640 1 -- 172.21.15.152:0/2492297578 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa284144fa0 msgr2=0x7fa268004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:29.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:29.279+0000 7fa281e00640 1 --2- 172.21.15.152:0/2492297578 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa284144fa0 0x7fa268004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:29.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:29.279+0000 7fa281e00640 1 -- 172.21.15.152:0/2492297578 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa268004e40 msgr2=0x7fa2680052c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:29.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:29.279+0000 7fa281e00640 1 --2- 172.21.15.152:0/2492297578 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa268004e40 0x7fa2680052c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:29.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:29.279+0000 7fa281e00640 1 -- 172.21.15.152:0/2492297578 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa268003520 con 0x7fa268003a50 2024-04-20T11:36:29.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:29.280+0000 7fa281e00640 1 --2- 172.21.15.152:0/2492297578 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa268003a50 0x7fa268003e70 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 crypto rx=0x7fa27400ba30 tx=0x7fa268005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a43c65a06178f5e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:29.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:29.280+0000 7fa281400640 1 -- 172.21.15.152:0/2492297578 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa274036020 con 0x7fa268003a50 2024-04-20T11:36:29.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:29.280+0000 7fa281400640 1 -- 172.21.15.152:0/2492297578 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa274032680 con 0x7fa268003a50 2024-04-20T11:36:29.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:29.280+0000 7fa281400640 1 -- 172.21.15.152:0/2492297578 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa27400b600 con 0x7fa268003a50 2024-04-20T11:36:29.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:29.281+0000 7fa288a00640 1 -- 172.21.15.152:0/2492297578 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa268003a50 msgr2=0x7fa268003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:29.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:29.281+0000 7fa288a00640 1 --2- 172.21.15.152:0/2492297578 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa268003a50 0x7fa268003e70 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 crypto rx=0x7fa27400ba30 tx=0x7fa268005830 comp rx=0 tx=0).stop 2024-04-20T11:36:29.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:29.281+0000 7fa282800640 1 -- 172.21.15.152:0/2492297578 reap_dead start 2024-04-20T11:36:29.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:29.281+0000 7fa288a00640 1 -- 172.21.15.152:0/2492297578 shutdown_connections 2024-04-20T11:36:29.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:29.281+0000 7fa288a00640 1 -- 172.21.15.152:0/2492297578 >> 172.21.15.152:0/2492297578 conn(0x7fa28406b980 msgr2=0x7fa28406bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:29.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:29.281+0000 7fa288a00640 1 -- 172.21.15.152:0/2492297578 shutdown_connections 2024-04-20T11:36:29.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:29.281+0000 7fa288a00640 1 -- 172.21.15.152:0/2492297578 wait complete. 2024-04-20T11:36:29.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:29.282+0000 7fa288a00640 1 Processor -- start 2024-04-20T11:36:29.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:29.283+0000 7fa288a00640 1 -- start start 2024-04-20T11:36:29.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:29.283+0000 7fa288a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa268003a50 0x7fa284161b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:29.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:29.284+0000 7fa288a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa2840c5da0 0x7fa284140240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:29.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:29.284+0000 7fa288a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2840c76e0 0x7fa2840c7b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:29.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:29.284+0000 7fa288a00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fa2840b3470 con 0x7fa2840c5da0 2024-04-20T11:36:29.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:29.284+0000 7fa288a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa2840aeb90 con 0x7fa2840c76e0 2024-04-20T11:36:29.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:29.284+0000 7fa288a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa2840b35f0 con 0x7fa268003a50 2024-04-20T11:36:29.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:29.284+0000 7fa279400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa2840c5da0 0x7fa284140240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:29.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:29.284+0000 7fa279400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa2840c5da0 0x7fa284140240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49598/0 (socket says 172.21.15.152:49598) 2024-04-20T11:36:29.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:29.284+0000 7fa279400640 1 -- 172.21.15.152:0/351704121 learned_addr learned my addr 172.21.15.152:0/351704121 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:29.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:29.284+0000 7fa279400640 1 -- 172.21.15.152:0/351704121 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa268003a50 msgr2=0x7fa284161b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:29.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:29.284+0000 7fa279400640 1 --2- 172.21.15.152:0/351704121 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa268003a50 0x7fa284161b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:29.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:29.284+0000 7fa279400640 1 -- 172.21.15.152:0/351704121 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2840c76e0 msgr2=0x7fa2840c7b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:29.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:29.284+0000 7fa279400640 1 --2- 172.21.15.152:0/351704121 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2840c76e0 0x7fa2840c7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:29.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:29.284+0000 7fa279400640 1 -- 172.21.15.152:0/351704121 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2840c6520 con 0x7fa2840c5da0 2024-04-20T11:36:29.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:29.284+0000 7fa279400640 1 --2- 172.21.15.152:0/351704121 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa2840c5da0 0x7fa284140240 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7fa26c00c700 tx=0x7fa26c00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:29.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:29.285+0000 7fa27aa00640 1 -- 172.21.15.152:0/351704121 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa26c018070 con 0x7fa2840c5da0 2024-04-20T11:36:29.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:29.285+0000 7fa27aa00640 1 -- 172.21.15.152:0/351704121 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa26c0072f0 con 0x7fa2840c5da0 2024-04-20T11:36:29.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:29.285+0000 7fa27aa00640 1 -- 172.21.15.152:0/351704121 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa26c0047e0 con 0x7fa2840c5da0 2024-04-20T11:36:29.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:29.285+0000 7fa288a00640 1 -- 172.21.15.152:0/351704121 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2840c6800 con 0x7fa2840c5da0 2024-04-20T11:36:29.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:29.285+0000 7fa288a00640 1 -- 172.21.15.152:0/351704121 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2840c6220 con 0x7fa2840c5da0 2024-04-20T11:36:29.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:29.286+0000 7fa288a00640 1 -- 172.21.15.152:0/351704121 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa284068a60 con 0x7fa2840c5da0 2024-04-20T11:36:29.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:29.287+0000 7fa27aa00640 1 -- 172.21.15.152:0/351704121 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 23) ==== 91652+0+0 (secure 0 0 0) 0x7fa26c004980 con 0x7fa2840c5da0 2024-04-20T11:36:29.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:29.288+0000 7fa27aa00640 1 --2- 172.21.15.152:0/351704121 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fa26006cfb0 0x7fa26006f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:29.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:29.288+0000 7fa27aa00640 1 -- 172.21.15.152:0/351704121 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5709+0+0 (secure 0 0 0) 0x7fa26c08bb80 con 0x7fa2840c5da0 2024-04-20T11:36:29.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:29.289+0000 7fa281e00640 1 --2- 172.21.15.152:0/351704121 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fa26006cfb0 0x7fa26006f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:29.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:29.290+0000 7fa281e00640 1 --2- 172.21.15.152:0/351704121 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fa26006cfb0 0x7fa26006f470 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fa27400ba00 tx=0x7fa274032000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:29.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:29.292+0000 7fa27aa00640 1 -- 172.21.15.152:0/351704121 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fa26c0595e0 con 0x7fa2840c5da0 2024-04-20T11:36:29.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:29.519+0000 7fa288a00640 1 -- 172.21.15.152:0/351704121 --> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol41321", "group_name": "subvol_grp75600", "target": ["mon-mgr", ""]}) -- 0x7fa2840c74c0 con 0x7fa26006cfb0 2024-04-20T11:36:29.532 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:29.532+0000 7fa27aa00640 1 -- 172.21.15.152:0/351704121 <== mgr.4109 v2:172.21.15.175:6836/4021250441 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa2840c74c0 con 0x7fa26006cfb0 2024-04-20T11:36:29.532 INFO:teuthology.orchestra.run.smithi152.stdout:/volumes/subvol_grp75600/subvol41321 2024-04-20T11:36:29.535 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:29.534+0000 7fa288a00640 1 -- 172.21.15.152:0/351704121 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fa26006cfb0 msgr2=0x7fa26006f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:29.535 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:29.534+0000 7fa288a00640 1 --2- 172.21.15.152:0/351704121 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fa26006cfb0 0x7fa26006f470 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fa27400ba00 tx=0x7fa274032000 comp rx=0 tx=0).stop 2024-04-20T11:36:29.535 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:29.534+0000 7fa288a00640 1 -- 172.21.15.152:0/351704121 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa2840c5da0 msgr2=0x7fa284140240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:29.535 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:29.534+0000 7fa288a00640 1 --2- 172.21.15.152:0/351704121 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa2840c5da0 0x7fa284140240 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7fa26c00c700 tx=0x7fa26c00cbc0 comp rx=0 tx=0).stop 2024-04-20T11:36:29.535 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:29.534+0000 7fa288a00640 1 -- 172.21.15.152:0/351704121 shutdown_connections 2024-04-20T11:36:29.535 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:29.534+0000 7fa288a00640 1 --2- 172.21.15.152:0/351704121 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fa26006cfb0 0x7fa26006f470 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:29.535 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:29.534+0000 7fa288a00640 1 --2- 172.21.15.152:0/351704121 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2840c76e0 0x7fa2840c7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:29.535 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:29.534+0000 7fa288a00640 1 --2- 172.21.15.152:0/351704121 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa2840c5da0 0x7fa284140240 unknown :-1 s=CLOSED pgs=2025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:29.535 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:29.534+0000 7fa288a00640 1 --2- 172.21.15.152:0/351704121 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa268003a50 0x7fa284161b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:29.535 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:29.534+0000 7fa288a00640 1 -- 172.21.15.152:0/351704121 >> 172.21.15.152:0/351704121 conn(0x7fa28406b980 msgr2=0x7fa284146710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:29.536 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:29.534+0000 7fa288a00640 1 -- 172.21.15.152:0/351704121 shutdown_connections 2024-04-20T11:36:29.536 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:29.534+0000 7fa288a00640 1 -- 172.21.15.152:0/351704121 wait complete. 2024-04-20T11:36:29.554 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:36:29.554 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_legacy/afe8e28ddf306be9d0ff8385185156a6.meta) 2024-04-20T11:36:34.053 INFO:teuthology.orchestra.run.smithi152.stdout:[GLOBAL] 2024-04-20T11:36:34.053 INFO:teuthology.orchestra.run.smithi152.stdout:version = 1 2024-04-20T11:36:34.053 INFO:teuthology.orchestra.run.smithi152.stdout:type = subvolume 2024-04-20T11:36:34.053 INFO:teuthology.orchestra.run.smithi152.stdout:path = /volumes/subvol_grp75600/subvol41321 2024-04-20T11:36:34.053 INFO:teuthology.orchestra.run.smithi152.stdout:state = complete 2024-04-20T11:36:34.053 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:36:34.059 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol55180 2024-04-20T11:36:34.265 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.265+0000 7f4fba400640 1 Processor -- start 2024-04-20T11:36:34.266 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.265+0000 7f4fba400640 1 -- start start 2024-04-20T11:36:34.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.267+0000 7f4fba400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f4fb41624e0 0x7f4fb40b2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:34.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.267+0000 7f4fba400640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4fb40cde20 con 0x7f4fb40cebe0 2024-04-20T11:36:34.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.267+0000 7f4fba400640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4fb40ce100 con 0x7f4fb4162fd0 2024-04-20T11:36:34.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.267+0000 7f4fba400640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f4fb40ce3e0 con 0x7f4fb41624e0 2024-04-20T11:36:34.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.267+0000 7f4fb3400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f4fb41624e0 0x7f4fb40b2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:34.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.267+0000 7f4fb2a00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f4fb40cebe0 0x7f4fb41628c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:45344/0 (socket says 172.21.15.152:45344) 2024-04-20T11:36:34.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.267+0000 7f4fb3400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f4fb41624e0 0x7f4fb40b2630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49608/0 (socket says 172.21.15.152:49608) 2024-04-20T11:36:34.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.267+0000 7f4fb2a00640 1 -- 172.21.15.152:0/4193276189 learned_addr learned my addr 172.21.15.152:0/4193276189 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:36:34.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.268+0000 7f4fb3400640 1 -- 172.21.15.152:0/4193276189 >> v1:172.21.15.175:6789/0 conn(0x7f4fb4162fd0 legacy=0x7f4fb40cf0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:34.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.268+0000 7f4fb3400640 1 -- 172.21.15.152:0/4193276189 >> v1:172.21.15.152:6789/0 conn(0x7f4fb40cebe0 legacy=0x7f4fb41628c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:34.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.268+0000 7f4fb3400640 1 -- 172.21.15.152:0/4193276189 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fb40ce6c0 con 0x7f4fb41624e0 2024-04-20T11:36:34.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.269+0000 7f4fb3400640 1 --2- 172.21.15.152:0/4193276189 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4fb41624e0 0x7f4fb40b2630 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c0097a0 tx=0x7f4f9c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d91caadc6dbba86e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:34.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.269+0000 7f4fb2000640 1 -- 172.21.15.152:0/4193276189 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f9c035020 con 0x7f4fb41624e0 2024-04-20T11:36:34.270 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.269+0000 7f4fb2000640 1 -- 172.21.15.152:0/4193276189 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f9c031830 con 0x7f4fb41624e0 2024-04-20T11:36:34.270 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.269+0000 7f4fb2000640 1 -- 172.21.15.152:0/4193276189 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f9c031c00 con 0x7f4fb41624e0 2024-04-20T11:36:34.270 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.270+0000 7f4fba400640 1 -- 172.21.15.152:0/4193276189 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4fb41624e0 msgr2=0x7f4fb40b2630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:34.271 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.270+0000 7f4fba400640 1 --2- 172.21.15.152:0/4193276189 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4fb41624e0 0x7f4fb40b2630 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c0097a0 tx=0x7f4f9c02b8d0 comp rx=0 tx=0).stop 2024-04-20T11:36:34.271 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.270+0000 7f4fba400640 1 -- 172.21.15.152:0/4193276189 shutdown_connections 2024-04-20T11:36:34.271 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.270+0000 7f4fba400640 1 --2- 172.21.15.152:0/4193276189 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4fb41624e0 0x7f4fb40b2630 unknown :-1 s=CLOSED pgs=2027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:34.271 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.270+0000 7f4fba400640 1 -- 172.21.15.152:0/4193276189 >> 172.21.15.152:0/4193276189 conn(0x7f4fb409aa40 msgr2=0x7f4fb40b3130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:34.271 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.270+0000 7f4fba400640 1 -- 172.21.15.152:0/4193276189 shutdown_connections 2024-04-20T11:36:34.271 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.270+0000 7f4fba400640 1 -- 172.21.15.152:0/4193276189 wait complete. 2024-04-20T11:36:34.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.272+0000 7f4fba400640 1 Processor -- start 2024-04-20T11:36:34.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.272+0000 7f4fba400640 1 -- start start 2024-04-20T11:36:34.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.273+0000 7f4fba400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4fb40cebe0 0x7f4fb40cab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:34.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.273+0000 7f4fba400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4fb41624e0 0x7f4fb406f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:34.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.273+0000 7f4fba400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4fb4162fd0 0x7f4fb406fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:34.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.273+0000 7f4fba400640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb415ec20 con 0x7f4fb40cebe0 2024-04-20T11:36:34.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.273+0000 7f4fba400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb40b2e60 con 0x7f4fb4162fd0 2024-04-20T11:36:34.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.273+0000 7f4fba400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4fb415eda0 con 0x7f4fb41624e0 2024-04-20T11:36:34.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.273+0000 7f4fb3400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4fb40cebe0 0x7f4fb40cab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:34.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.273+0000 7f4fb3400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4fb40cebe0 0x7f4fb40cab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49620/0 (socket says 172.21.15.152:49620) 2024-04-20T11:36:34.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.273+0000 7f4fb3400640 1 -- 172.21.15.152:0/1760259621 learned_addr learned my addr 172.21.15.152:0/1760259621 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:34.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.273+0000 7f4fb3400640 1 -- 172.21.15.152:0/1760259621 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4fb41624e0 msgr2=0x7f4fb406f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:34.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.273+0000 7f4fb3400640 1 --2- 172.21.15.152:0/1760259621 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4fb41624e0 0x7f4fb406f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:34.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.273+0000 7f4fb3400640 1 -- 172.21.15.152:0/1760259621 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4fb4162fd0 msgr2=0x7f4fb406fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:34.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.273+0000 7f4fb3400640 1 --2- 172.21.15.152:0/1760259621 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4fb4162fd0 0x7f4fb406fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:34.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.273+0000 7f4fb3400640 1 -- 172.21.15.152:0/1760259621 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fb40c9620 con 0x7f4fb40cebe0 2024-04-20T11:36:34.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.274+0000 7f4fb3400640 1 --2- 172.21.15.152:0/1760259621 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4fb40cebe0 0x7f4fb40cab40 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c02bee0 tx=0x7f4f9c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:34.276 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.274+0000 7f4fab400640 1 -- 172.21.15.152:0/1760259621 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f9c041070 con 0x7f4fb40cebe0 2024-04-20T11:36:34.276 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.274+0000 7f4fab400640 1 -- 172.21.15.152:0/1760259621 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f9c03aa30 con 0x7f4fb40cebe0 2024-04-20T11:36:34.276 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.274+0000 7f4fab400640 1 -- 172.21.15.152:0/1760259621 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f9c035040 con 0x7f4fb40cebe0 2024-04-20T11:36:34.276 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.274+0000 7f4fba400640 1 -- 172.21.15.152:0/1760259621 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fb40c9900 con 0x7f4fb40cebe0 2024-04-20T11:36:34.276 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.274+0000 7f4fba400640 1 -- 172.21.15.152:0/1760259621 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fb40c9ec0 con 0x7f4fb40cebe0 2024-04-20T11:36:34.276 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.275+0000 7f4fba400640 1 -- 172.21.15.152:0/1760259621 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fb4069100 con 0x7f4fb40cebe0 2024-04-20T11:36:34.277 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.276+0000 7f4fab400640 1 -- 172.21.15.152:0/1760259621 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 23) ==== 91652+0+0 (secure 0 0 0) 0x7f4f9c038020 con 0x7f4fb40cebe0 2024-04-20T11:36:34.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.277+0000 7f4fab400640 1 --2- 172.21.15.152:0/1760259621 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4f9006cfb0 0x7f4f9006f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:34.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.277+0000 7f4fab400640 1 -- 172.21.15.152:0/1760259621 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5709+0+0 (secure 0 0 0) 0x7f4f9c0ae220 con 0x7f4fb40cebe0 2024-04-20T11:36:34.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.278+0000 7f4fb2a00640 1 --2- 172.21.15.152:0/1760259621 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4f9006cfb0 0x7f4f9006f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:34.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.279+0000 7f4fb2a00640 1 --2- 172.21.15.152:0/1760259621 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4f9006cfb0 0x7f4f9006f470 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f4fa4000f60 tx=0x7f4fa4007a80 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:34.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.282+0000 7f4fab400640 1 -- 172.21.15.152:0/1760259621 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f4f9c07bd80 con 0x7f4fb40cebe0 2024-04-20T11:36:34.512 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.512+0000 7f4fba400640 1 -- 172.21.15.152:0/1760259621 --> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol55180", "target": ["mon-mgr", ""]}) -- 0x7f4fb4159b00 con 0x7f4f9006cfb0 2024-04-20T11:36:34.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.553+0000 7f4fab400640 1 -- 172.21.15.152:0/1760259621 <== mgr.4109 v2:172.21.15.175:6836/4021250441 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4fb4159b00 con 0x7f4f9006cfb0 2024-04-20T11:36:34.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.556+0000 7f4fba400640 1 -- 172.21.15.152:0/1760259621 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4f9006cfb0 msgr2=0x7f4f9006f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:34.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.556+0000 7f4fba400640 1 --2- 172.21.15.152:0/1760259621 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4f9006cfb0 0x7f4f9006f470 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f4fa4000f60 tx=0x7f4fa4007a80 comp rx=0 tx=0).stop 2024-04-20T11:36:34.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.556+0000 7f4fba400640 1 -- 172.21.15.152:0/1760259621 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4fb40cebe0 msgr2=0x7f4fb40cab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:34.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.556+0000 7f4fba400640 1 --2- 172.21.15.152:0/1760259621 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4fb40cebe0 0x7f4fb40cab40 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c02bee0 tx=0x7f4f9c0046c0 comp rx=0 tx=0).stop 2024-04-20T11:36:34.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.556+0000 7f4fba400640 1 -- 172.21.15.152:0/1760259621 shutdown_connections 2024-04-20T11:36:34.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.556+0000 7f4fba400640 1 --2- 172.21.15.152:0/1760259621 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4f9006cfb0 0x7f4f9006f470 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:34.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.556+0000 7f4fba400640 1 --2- 172.21.15.152:0/1760259621 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4fb4162fd0 0x7f4fb406fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:34.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.556+0000 7f4fba400640 1 --2- 172.21.15.152:0/1760259621 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4fb41624e0 0x7f4fb406f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:34.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.556+0000 7f4fba400640 1 --2- 172.21.15.152:0/1760259621 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4fb40cebe0 0x7f4fb40cab40 unknown :-1 s=CLOSED pgs=2028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:34.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.556+0000 7f4fba400640 1 -- 172.21.15.152:0/1760259621 >> 172.21.15.152:0/1760259621 conn(0x7f4fb409aa40 msgr2=0x7f4fb415a020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:34.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.556+0000 7f4fba400640 1 -- 172.21.15.152:0/1760259621 shutdown_connections 2024-04-20T11:36:34.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.556+0000 7f4fba400640 1 -- 172.21.15.152:0/1760259621 wait complete. 2024-04-20T11:36:34.562 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:36:34.562+0000 7f8a60800640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:36:34.562 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:36:34.562+0000 7f8a60800640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:36:34.562 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:36:34.562+0000 7f8a60800640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:36:34.562 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:36:34.562+0000 7f8a60800640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:36:34.562 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:36:34.562+0000 7f8a60800640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:36:34.576 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol41321 subvol_grp75600 2024-04-20T11:36:34.585 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:36:34.585+0000 7f8a5fe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:36:34.585 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:36:34.585+0000 7f8a5fe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:36:34.585 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:36:34.585+0000 7f8a5fe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:36:34.585 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:36:34.585+0000 7f8a5fe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:36:34.585 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:36:34.585+0000 7f8a5fe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:36:34.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.779+0000 7f1628200640 1 Processor -- start 2024-04-20T11:36:34.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.779+0000 7f1628200640 1 -- start start 2024-04-20T11:36:34.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.781+0000 7f1628200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f1620145340 0x7f1620160db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:34.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.781+0000 7f1628200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f16200b47a0 0x7f16200ceaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:34.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.781+0000 7f1628200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16200b4f90 con 0x7f1620071e20 2024-04-20T11:36:34.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.781+0000 7f1628200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f1620145720 con 0x7f16200b47a0 2024-04-20T11:36:34.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.781+0000 7f1628200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f16200b4cb0 con 0x7f1620145340 2024-04-20T11:36:34.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.781+0000 7f1624e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f16200b47a0 0x7f16200ceaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:34.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.781+0000 7f1624e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f16200b47a0 0x7f16200ceaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49632/0 (socket says 172.21.15.152:49632) 2024-04-20T11:36:34.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.781+0000 7f1624e00640 1 -- 172.21.15.152:0/2432179350 learned_addr learned my addr 172.21.15.152:0/2432179350 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:34.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.782+0000 7f1624e00640 1 -- 172.21.15.152:0/2432179350 >> v2:172.21.15.175:3300/0 conn(0x7f1620145340 msgr2=0x7f1620160db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:34.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.782+0000 7f1624e00640 1 --2- 172.21.15.152:0/2432179350 >> v2:172.21.15.175:3300/0 conn(0x7f1620145340 0x7f1620160db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:34.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.782+0000 7f1624e00640 1 -- 172.21.15.152:0/2432179350 >> v1:172.21.15.175:6789/0 conn(0x7f1620071e20 legacy=0x7f162015cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:34.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.782+0000 7f1624e00640 1 -- 172.21.15.152:0/2432179350 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16201602d0 con 0x7f16200b47a0 2024-04-20T11:36:34.785 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.783+0000 7f1624e00640 1 --2- 172.21.15.152:0/2432179350 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f16200b47a0 0x7f16200ceaa0 secure :-1 s=READY pgs=2029 cs=0 l=1 rev1=1 crypto rx=0x7f1620073040 tx=0x7f160c02b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0f2d9a98b97fde7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:34.785 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.783+0000 7f161fe00640 1 -- 172.21.15.152:0/2432179350 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f160c038070 con 0x7f16200b47a0 2024-04-20T11:36:34.785 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.783+0000 7f161fe00640 1 -- 172.21.15.152:0/2432179350 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f160c02f050 con 0x7f16200b47a0 2024-04-20T11:36:34.785 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.783+0000 7f161fe00640 1 -- 172.21.15.152:0/2432179350 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f160c031970 con 0x7f16200b47a0 2024-04-20T11:36:34.785 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.784+0000 7f1628200640 1 -- 172.21.15.152:0/2432179350 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f16200b47a0 msgr2=0x7f16200ceaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:34.785 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.784+0000 7f1628200640 1 --2- 172.21.15.152:0/2432179350 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f16200b47a0 0x7f16200ceaa0 secure :-1 s=READY pgs=2029 cs=0 l=1 rev1=1 crypto rx=0x7f1620073040 tx=0x7f160c02b7f0 comp rx=0 tx=0).stop 2024-04-20T11:36:34.785 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.784+0000 7f1628200640 1 -- 172.21.15.152:0/2432179350 shutdown_connections 2024-04-20T11:36:34.785 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.784+0000 7f1628200640 1 --2- 172.21.15.152:0/2432179350 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f16200b47a0 0x7f16200ceaa0 unknown :-1 s=CLOSED pgs=2029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:34.785 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.784+0000 7f1628200640 1 --2- 172.21.15.152:0/2432179350 >> v2:172.21.15.175:3300/0 conn(0x7f1620145340 0x7f1620160db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:34.785 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.784+0000 7f1628200640 1 -- 172.21.15.152:0/2432179350 >> 172.21.15.152:0/2432179350 conn(0x7f162006b980 msgr2=0x7f162006bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:34.785 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.784+0000 7f1628200640 1 -- 172.21.15.152:0/2432179350 shutdown_connections 2024-04-20T11:36:34.785 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.784+0000 7f1628200640 1 -- 172.21.15.152:0/2432179350 wait complete. 2024-04-20T11:36:34.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.786+0000 7f1628200640 1 Processor -- start 2024-04-20T11:36:34.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.786+0000 7f1628200640 1 -- start start 2024-04-20T11:36:34.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.787+0000 7f1628200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1620071e20 0x7f1620139fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:34.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.787+0000 7f1628200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f16200b47a0 0x7f16201f8110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:34.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.787+0000 7f1628200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1620145340 0x7f162015e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:34.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.787+0000 7f1628200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f1620072260 con 0x7f1620145340 2024-04-20T11:36:34.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.787+0000 7f1628200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1620146010 con 0x7f1620071e20 2024-04-20T11:36:34.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.787+0000 7f1628200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f16200723e0 con 0x7f16200b47a0 2024-04-20T11:36:34.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.787+0000 7f1626200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1620145340 0x7f162015e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:34.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.787+0000 7f1626200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1620145340 0x7f162015e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49636/0 (socket says 172.21.15.152:49636) 2024-04-20T11:36:34.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.787+0000 7f1626200640 1 -- 172.21.15.152:0/4000231701 learned_addr learned my addr 172.21.15.152:0/4000231701 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:34.789 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.788+0000 7f1624e00640 1 --2- 172.21.15.152:0/4000231701 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f16200b47a0 0x7f16201f8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:34.789 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.788+0000 7f1625800640 1 --2- 172.21.15.152:0/4000231701 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1620071e20 0x7f1620139fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:34.790 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.788+0000 7f1626200640 1 -- 172.21.15.152:0/4000231701 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f16200b47a0 msgr2=0x7f16201f8110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:34.790 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.788+0000 7f1626200640 1 --2- 172.21.15.152:0/4000231701 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f16200b47a0 0x7f16201f8110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:34.790 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.788+0000 7f1626200640 1 -- 172.21.15.152:0/4000231701 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1620071e20 msgr2=0x7f1620139fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:34.790 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.788+0000 7f1626200640 1 --2- 172.21.15.152:0/4000231701 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1620071e20 0x7f1620139fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:34.790 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.788+0000 7f1626200640 1 -- 172.21.15.152:0/4000231701 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16201f8a60 con 0x7f1620145340 2024-04-20T11:36:34.790 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.788+0000 7f1624e00640 1 --2- 172.21.15.152:0/4000231701 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f16200b47a0 0x7f16201f8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:36:34.791 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.788+0000 7f1626200640 1 --2- 172.21.15.152:0/4000231701 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1620145340 0x7f162015e9f0 secure :-1 s=READY pgs=2030 cs=0 l=1 rev1=1 crypto rx=0x7f161800c7b0 tx=0x7f161800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:34.791 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.788+0000 7f161d600640 1 -- 172.21.15.152:0/4000231701 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1618017070 con 0x7f1620145340 2024-04-20T11:36:34.791 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.789+0000 7f161d600640 1 -- 172.21.15.152:0/4000231701 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1618003da0 con 0x7f1620145340 2024-04-20T11:36:34.791 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.789+0000 7f1628200640 1 -- 172.21.15.152:0/4000231701 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16201f8d40 con 0x7f1620145340 2024-04-20T11:36:34.791 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.789+0000 7f161d600640 1 -- 172.21.15.152:0/4000231701 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16180052b0 con 0x7f1620145340 2024-04-20T11:36:34.791 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.789+0000 7f1628200640 1 -- 172.21.15.152:0/4000231701 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f162013c170 con 0x7f1620145340 2024-04-20T11:36:34.791 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.790+0000 7f1628200640 1 -- 172.21.15.152:0/4000231701 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1620068a60 con 0x7f1620145340 2024-04-20T11:36:34.791 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.791+0000 7f161d600640 1 -- 172.21.15.152:0/4000231701 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 23) ==== 91652+0+0 (secure 0 0 0) 0x7f161801a070 con 0x7f1620145340 2024-04-20T11:36:34.792 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.792+0000 7f161d600640 1 --2- 172.21.15.152:0/4000231701 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f160006cfe0 0x7f160006f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:34.792 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.792+0000 7f161d600640 1 -- 172.21.15.152:0/4000231701 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5709+0+0 (secure 0 0 0) 0x7f161808bdb0 con 0x7f1620145340 2024-04-20T11:36:34.792 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.792+0000 7f1625800640 1 --2- 172.21.15.152:0/4000231701 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f160006cfe0 0x7f160006f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:34.794 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.794+0000 7f1625800640 1 --2- 172.21.15.152:0/4000231701 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f160006cfe0 0x7f160006f4a0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f161400b810 tx=0x7f1614008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:34.797 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:34.796+0000 7f161d600640 1 -- 172.21.15.152:0/4000231701 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f1618059810 con 0x7f1620145340 2024-04-20T11:36:35.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:35.028+0000 7f1628200640 1 -- 172.21.15.152:0/4000231701 --> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol41321", "group_name": "subvol_grp75600", "target": ["mon-mgr", ""]}) -- 0x7f16200b4580 con 0x7f160006cfe0 2024-04-20T11:36:35.075 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:35.075+0000 7f161d600640 1 -- 172.21.15.152:0/4000231701 <== mgr.4109 v2:172.21.15.175:6836/4021250441 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f16200b4580 con 0x7f160006cfe0 2024-04-20T11:36:35.079 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:35.078+0000 7f1628200640 1 -- 172.21.15.152:0/4000231701 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f160006cfe0 msgr2=0x7f160006f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:35.079 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:35.078+0000 7f1628200640 1 --2- 172.21.15.152:0/4000231701 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f160006cfe0 0x7f160006f4a0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f161400b810 tx=0x7f1614008040 comp rx=0 tx=0).stop 2024-04-20T11:36:35.079 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:35.078+0000 7f1628200640 1 -- 172.21.15.152:0/4000231701 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1620145340 msgr2=0x7f162015e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:35.079 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:35.078+0000 7f1628200640 1 --2- 172.21.15.152:0/4000231701 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1620145340 0x7f162015e9f0 secure :-1 s=READY pgs=2030 cs=0 l=1 rev1=1 crypto rx=0x7f161800c7b0 tx=0x7f161800cc70 comp rx=0 tx=0).stop 2024-04-20T11:36:35.079 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:35.078+0000 7f1628200640 1 -- 172.21.15.152:0/4000231701 shutdown_connections 2024-04-20T11:36:35.079 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:35.078+0000 7f1628200640 1 --2- 172.21.15.152:0/4000231701 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f160006cfe0 0x7f160006f4a0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:35.079 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:35.078+0000 7f1628200640 1 --2- 172.21.15.152:0/4000231701 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1620145340 0x7f162015e9f0 unknown :-1 s=CLOSED pgs=2030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:35.079 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:35.078+0000 7f1628200640 1 --2- 172.21.15.152:0/4000231701 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f16200b47a0 0x7f16201f8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:35.079 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:35.078+0000 7f1628200640 1 --2- 172.21.15.152:0/4000231701 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1620071e20 0x7f1620139fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:35.080 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:35.078+0000 7f1628200640 1 -- 172.21.15.152:0/4000231701 >> 172.21.15.152:0/4000231701 conn(0x7f162006b980 msgr2=0x7f162006de60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:35.080 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:35.078+0000 7f1628200640 1 -- 172.21.15.152:0/4000231701 shutdown_connections 2024-04-20T11:36:35.080 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:35.079+0000 7f1628200640 1 -- 172.21.15.152:0/4000231701 wait complete. 2024-04-20T11:36:35.098 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-04-20T11:36:35.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:35 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-20T11:36:35.211 INFO:teuthology.orchestra.run.smithi152.stdout:2 2024-04-20T11:36:35.212 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T11:36:35.221 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup rm cephfs subvol_grp75600 2024-04-20T11:36:35.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:35.426+0000 7f41b7e00640 1 Processor -- start 2024-04-20T11:36:35.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:35.426+0000 7f41b7e00640 1 -- start start 2024-04-20T11:36:35.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:35.428+0000 7f41b7e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f41b015b510 0x7f41b015fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:35.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:35.428+0000 7f41b7e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f41b00cfc40 0x7f41b00b3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:35.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:35.428+0000 7f41b7e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f41b00b2170 0x7f41b00b2600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:35.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:35.428+0000 7f41b7e00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f41b006f420 con 0x7f41b015b510 2024-04-20T11:36:35.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:35.428+0000 7f41b7e00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f41b00b4430 con 0x7f41b00cfc40 2024-04-20T11:36:35.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:35.428+0000 7f41b7e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f41b015f1c0 con 0x7f41b00b2170 2024-04-20T11:36:35.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:35.428+0000 7f41b5400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f41b015b510 0x7f41b015fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:35.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:35.428+0000 7f41b5400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f41b015b510 0x7f41b015fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49638/0 (socket says 172.21.15.152:49638) 2024-04-20T11:36:35.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:35.428+0000 7f41b5400640 1 -- 172.21.15.152:0/2203681388 learned_addr learned my addr 172.21.15.152:0/2203681388 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:35.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:35.429+0000 7f41b4a00640 1 --2- 172.21.15.152:0/2203681388 >> v2:172.21.15.175:3300/0 conn(0x7f41b00cfc40 0x7f41b00b3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:35.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:35.429+0000 7f41b5e00640 1 --2- 172.21.15.152:0/2203681388 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f41b00b2170 0x7f41b00b2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:35.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:35.429+0000 7f41b5400640 1 -- 172.21.15.152:0/2203681388 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f41b00b2170 msgr2=0x7f41b00b2600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:35.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:35.429+0000 7f41b5400640 1 --2- 172.21.15.152:0/2203681388 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f41b00b2170 0x7f41b00b2600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:35.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:35.429+0000 7f41b5400640 1 -- 172.21.15.152:0/2203681388 >> v2:172.21.15.175:3300/0 conn(0x7f41b00cfc40 msgr2=0x7f41b00b3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:35.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:35.429+0000 7f41b5400640 1 --2- 172.21.15.152:0/2203681388 >> v2:172.21.15.175:3300/0 conn(0x7f41b00cfc40 0x7f41b00b3ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:35.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:35.429+0000 7f41b5400640 1 -- 172.21.15.152:0/2203681388 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41b006f670 con 0x7f41b015b510 2024-04-20T11:36:35.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:35.430+0000 7f41b5400640 1 --2- 172.21.15.152:0/2203681388 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f41b015b510 0x7f41b015fc30 secure :-1 s=READY pgs=2031 cs=0 l=1 rev1=1 crypto rx=0x7f41b0072fa0 tx=0x7f41a002e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5dfb2522aecdd29 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:35.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:35.430+0000 7f41afe00640 1 -- 172.21.15.152:0/2203681388 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a002ec80 con 0x7f41b015b510 2024-04-20T11:36:35.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:35.430+0000 7f41afe00640 1 -- 172.21.15.152:0/2203681388 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41a002ee20 con 0x7f41b015b510 2024-04-20T11:36:35.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:35.430+0000 7f41afe00640 1 -- 172.21.15.152:0/2203681388 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a0036610 con 0x7f41b015b510 2024-04-20T11:36:35.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:35.431+0000 7f41b7e00640 1 -- 172.21.15.152:0/2203681388 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f41b015b510 msgr2=0x7f41b015fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:35.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:35.431+0000 7f41b7e00640 1 --2- 172.21.15.152:0/2203681388 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f41b015b510 0x7f41b015fc30 secure :-1 s=READY pgs=2031 cs=0 l=1 rev1=1 crypto rx=0x7f41b0072fa0 tx=0x7f41a002e0e0 comp rx=0 tx=0).stop 2024-04-20T11:36:35.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:35.431+0000 7f41b7e00640 1 -- 172.21.15.152:0/2203681388 shutdown_connections 2024-04-20T11:36:35.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:35.431+0000 7f41b7e00640 1 --2- 172.21.15.152:0/2203681388 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f41b00b2170 0x7f41b00b2600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:35.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:35.431+0000 7f41b7e00640 1 --2- 172.21.15.152:0/2203681388 >> v2:172.21.15.175:3300/0 conn(0x7f41b00cfc40 0x7f41b00b3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:35.433 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:35.431+0000 7f41b7e00640 1 --2- 172.21.15.152:0/2203681388 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f41b015b510 0x7f41b015fc30 unknown :-1 s=CLOSED pgs=2031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:35.433 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:35.431+0000 7f41b7e00640 1 -- 172.21.15.152:0/2203681388 >> 172.21.15.152:0/2203681388 conn(0x7f41b006b980 msgr2=0x7f41b006bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:35.433 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:35.431+0000 7f41b7e00640 1 -- 172.21.15.152:0/2203681388 shutdown_connections 2024-04-20T11:36:35.433 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:35.431+0000 7f41b7e00640 1 -- 172.21.15.152:0/2203681388 wait complete. 2024-04-20T11:36:35.433 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:35.433+0000 7f41b7e00640 1 Processor -- start 2024-04-20T11:36:35.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:35.433+0000 7f41b7e00640 1 -- start start 2024-04-20T11:36:35.436 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:35.434+0000 7f41b7e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f41b00b2170 0x7f41b01f0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:35.436 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:35.434+0000 7f41b7e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f41b00cfc40 0x7f41b014eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:35.436 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:35.434+0000 7f41b7e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41b015b510 0x7f41b014b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:35.436 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:35.434+0000 7f41b7e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f41b015b8f0 con 0x7f41b00cfc40 2024-04-20T11:36:35.436 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:35.434+0000 7f41b7e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f41b015f710 con 0x7f41b015b510 2024-04-20T11:36:35.436 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:35.434+0000 7f41b7e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f41b006f950 con 0x7f41b00b2170 2024-04-20T11:36:35.436 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:35.435+0000 7f41b4a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f41b00cfc40 0x7f41b014eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:35.436 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:35.435+0000 7f41b4a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f41b00cfc40 0x7f41b014eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49648/0 (socket says 172.21.15.152:49648) 2024-04-20T11:36:35.436 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:35.435+0000 7f41b4a00640 1 -- 172.21.15.152:0/1420144037 learned_addr learned my addr 172.21.15.152:0/1420144037 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:35.436 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:35.435+0000 7f41b5400640 1 --2- 172.21.15.152:0/1420144037 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f41b00b2170 0x7f41b01f0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:35.436 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:35.435+0000 7f41b5e00640 1 --2- 172.21.15.152:0/1420144037 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41b015b510 0x7f41b014b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:35.436 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:35.435+0000 7f41b4a00640 1 -- 172.21.15.152:0/1420144037 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f41b00b2170 msgr2=0x7f41b01f0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:35.436 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:35.435+0000 7f41b4a00640 1 --2- 172.21.15.152:0/1420144037 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f41b00b2170 0x7f41b01f0ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:35.437 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:35.435+0000 7f41b4a00640 1 -- 172.21.15.152:0/1420144037 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41b015b510 msgr2=0x7f41b014b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:35.437 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:35.435+0000 7f41b4a00640 1 --2- 172.21.15.152:0/1420144037 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41b015b510 0x7f41b014b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:35.437 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:35.435+0000 7f41b4a00640 1 -- 172.21.15.152:0/1420144037 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41b015e4c0 con 0x7f41b00cfc40 2024-04-20T11:36:35.437 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:35.435+0000 7f41b4a00640 1 --2- 172.21.15.152:0/1420144037 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f41b00cfc40 0x7f41b014eb40 secure :-1 s=READY pgs=2032 cs=0 l=1 rev1=1 crypto rx=0x7f419c00d800 tx=0x7f419c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:35.437 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:35.436+0000 7f41ad600640 1 -- 172.21.15.152:0/1420144037 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f419c00b6c0 con 0x7f41b00cfc40 2024-04-20T11:36:35.438 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:35.436+0000 7f41ad600640 1 -- 172.21.15.152:0/1420144037 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f419c019070 con 0x7f41b00cfc40 2024-04-20T11:36:35.438 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:35.436+0000 7f41ad600640 1 -- 172.21.15.152:0/1420144037 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f419c00ba80 con 0x7f41b00cfc40 2024-04-20T11:36:35.438 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:35.436+0000 7f41b7e00640 1 -- 172.21.15.152:0/1420144037 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41b014f920 con 0x7f41b00cfc40 2024-04-20T11:36:35.438 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:35.436+0000 7f41b7e00640 1 -- 172.21.15.152:0/1420144037 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41b01f4220 con 0x7f41b00cfc40 2024-04-20T11:36:35.438 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:35.437+0000 7f41b7e00640 1 -- 172.21.15.152:0/1420144037 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41b00689f0 con 0x7f41b00cfc40 2024-04-20T11:36:35.438 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:35.438+0000 7f41ad600640 1 -- 172.21.15.152:0/1420144037 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 23) ==== 91652+0+0 (secure 0 0 0) 0x7f419c00bc20 con 0x7f41b00cfc40 2024-04-20T11:36:35.440 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:35.439+0000 7f41ad600640 1 --2- 172.21.15.152:0/1420144037 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f419006cfb0 0x7f419006f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:35.440 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:35.440+0000 7f41ad600640 1 -- 172.21.15.152:0/1420144037 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5709+0+0 (secure 0 0 0) 0x7f419c08cb50 con 0x7f41b00cfc40 2024-04-20T11:36:35.440 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:35.440+0000 7f41b5400640 1 --2- 172.21.15.152:0/1420144037 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f419006cfb0 0x7f419006f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:35.442 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:35.441+0000 7f41b5400640 1 --2- 172.21.15.152:0/1420144037 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f419006cfb0 0x7f419006f470 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f41a0037000 tx=0x7f41a0032000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:35.444 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:35.444+0000 7f41ad600640 1 -- 172.21.15.152:0/1420144037 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f419c08ee20 con 0x7f41b00cfc40 2024-04-20T11:36:35.672 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:35.672+0000 7f41b7e00640 1 -- 172.21.15.152:0/1420144037 --> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup rm", "vol_name": "cephfs", "group_name": "subvol_grp75600", "target": ["mon-mgr", ""]}) -- 0x7f41b014b430 con 0x7f419006cfb0 2024-04-20T11:36:35.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:35.683+0000 7f41ad600640 1 -- 172.21.15.152:0/1420144037 <== mgr.4109 v2:172.21.15.175:6836/4021250441 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f41b014b430 con 0x7f419006cfb0 2024-04-20T11:36:35.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:35.685+0000 7f41b7e00640 1 -- 172.21.15.152:0/1420144037 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f419006cfb0 msgr2=0x7f419006f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:35.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:35.685+0000 7f41b7e00640 1 --2- 172.21.15.152:0/1420144037 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f419006cfb0 0x7f419006f470 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f41a0037000 tx=0x7f41a0032000 comp rx=0 tx=0).stop 2024-04-20T11:36:35.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:35.685+0000 7f41b7e00640 1 -- 172.21.15.152:0/1420144037 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f41b00cfc40 msgr2=0x7f41b014eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:35.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:35.685+0000 7f41b7e00640 1 --2- 172.21.15.152:0/1420144037 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f41b00cfc40 0x7f41b014eb40 secure :-1 s=READY pgs=2032 cs=0 l=1 rev1=1 crypto rx=0x7f419c00d800 tx=0x7f419c00dcc0 comp rx=0 tx=0).stop 2024-04-20T11:36:35.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:35.685+0000 7f41b7e00640 1 -- 172.21.15.152:0/1420144037 shutdown_connections 2024-04-20T11:36:35.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:35.685+0000 7f41b7e00640 1 --2- 172.21.15.152:0/1420144037 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f419006cfb0 0x7f419006f470 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:35.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:35.685+0000 7f41b7e00640 1 --2- 172.21.15.152:0/1420144037 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41b015b510 0x7f41b014b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:35.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:35.685+0000 7f41b7e00640 1 --2- 172.21.15.152:0/1420144037 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f41b00cfc40 0x7f41b014eb40 unknown :-1 s=CLOSED pgs=2032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:35.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:35.685+0000 7f41b7e00640 1 --2- 172.21.15.152:0/1420144037 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f41b00b2170 0x7f41b01f0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:35.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:35.685+0000 7f41b7e00640 1 -- 172.21.15.152:0/1420144037 >> 172.21.15.152:0/1420144037 conn(0x7f41b006b980 msgr2=0x7f41b00ce880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:35.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:35.685+0000 7f41b7e00640 1 -- 172.21.15.152:0/1420144037 shutdown_connections 2024-04-20T11:36:35.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:35.685+0000 7f41b7e00640 1 -- 172.21.15.152:0/1420144037 wait complete. 2024-04-20T11:36:35.704 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-20T11:36:35.705 DEBUG:teuthology.orchestra.run.smithi152:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-20T11:36:35.802 DEBUG:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-20T11:36:35.916 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-20T11:36:35.916 INFO:tasks.cephfs.mount:Terminating background process 2024-04-20T11:36:35.916 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-20T11:36:35.916 DEBUG:teuthology.orchestra.run.smithi152:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:36:35.973 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:36:36.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.176+0000 7fece0a00640 1 Processor -- start 2024-04-20T11:36:36.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.177+0000 7fece0a00640 1 -- start start 2024-04-20T11:36:36.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.178+0000 7fece0a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fecdc1603d0 0x7fecdc0b5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:36.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.178+0000 7fece0a00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fecdc0d3350 con 0x7fecdc15fcc0 2024-04-20T11:36:36.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.178+0000 7fece0a00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fecdc0d3630 con 0x7fecdc0b5480 2024-04-20T11:36:36.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.178+0000 7fece0a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fecdc0d3910 con 0x7fecdc1603d0 2024-04-20T11:36:36.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.178+0000 7fecdb400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fecdc15fcc0 0x7fecdc0d4e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:55506/0 (socket says 172.21.15.152:55506) 2024-04-20T11:36:36.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.178+0000 7fecdb400640 1 -- 172.21.15.152:0/1165551482 learned_addr learned my addr 172.21.15.152:0/1165551482 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:36:36.182 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.179+0000 7fecdaa00640 1 --2- 172.21.15.152:0/1165551482 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fecdc1603d0 0x7fecdc0b5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:36.182 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.179+0000 7fecda000640 1 -- 172.21.15.152:0/1165551482 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1954301157 0 0) 0x7fecdc0d3350 con 0x7fecdc15fcc0 2024-04-20T11:36:36.182 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.179+0000 7fecda000640 1 -- 172.21.15.152:0/1165551482 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fecc0003520 con 0x7fecdc15fcc0 2024-04-20T11:36:36.182 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.179+0000 7fecda000640 1 -- 172.21.15.152:0/1165551482 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 199217163 0 0) 0x7fecc0003520 con 0x7fecdc15fcc0 2024-04-20T11:36:36.182 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.179+0000 7fecda000640 1 -- 172.21.15.152:0/1165551482 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fecdc1603d0 msgr2=0x7fecdc0b5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:36.182 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.179+0000 7fecda000640 1 --2- 172.21.15.152:0/1165551482 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fecdc1603d0 0x7fecdc0b5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:36.182 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.179+0000 7fecda000640 1 -- 172.21.15.152:0/1165551482 >> v1:172.21.15.175:6789/0 conn(0x7fecdc0b5480 legacy=0x7fecdc0cf240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:36.182 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.179+0000 7fecda000640 1 -- 172.21.15.152:0/1165551482 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecdc0ced20 con 0x7fecdc15fcc0 2024-04-20T11:36:36.182 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.179+0000 7fecda000640 1 -- 172.21.15.152:0/1165551482 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7feccc0019f0 con 0x7fecdc15fcc0 2024-04-20T11:36:36.182 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.179+0000 7fecdaa00640 1 --2- 172.21.15.152:0/1165551482 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fecdc1603d0 0x7fecdc0b5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:36:36.182 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.179+0000 7fecda000640 1 -- 172.21.15.152:0/1165551482 >> v1:172.21.15.152:6789/0 conn(0x7fecdc15fcc0 legacy=0x7fecdc0d4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:36.182 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.179+0000 7fecda000640 1 --2- 172.21.15.152:0/1165551482 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fecc00039e0 0x7fecc0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:36.182 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.179+0000 7fecda000640 1 --2- 172.21.15.152:0/1165551482 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fecdc1603d0 0x7fecc0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:36.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.179+0000 7fecda000640 1 --2- 172.21.15.152:0/1165551482 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fecc0004d40 0x7fecc00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:36.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.180+0000 7fecda000640 1 -- 172.21.15.152:0/1165551482 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fecdc0d3910 con 0x7fecc0004d40 2024-04-20T11:36:36.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.180+0000 7fecda000640 1 -- 172.21.15.152:0/1165551482 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fecc000bad0 con 0x7fecc00039e0 2024-04-20T11:36:36.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.180+0000 7fecda000640 1 -- 172.21.15.152:0/1165551482 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fecc000bcd0 con 0x7fecdc1603d0 2024-04-20T11:36:36.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.180+0000 7fecdbe00640 1 --2- 172.21.15.152:0/1165551482 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fecc0004d40 0x7fecc00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:36.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.180+0000 7fecdaa00640 1 --2- 172.21.15.152:0/1165551482 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fecdc1603d0 0x7fecc0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:36.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.180+0000 7fecdb400640 1 --2- 172.21.15.152:0/1165551482 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fecc00039e0 0x7fecc0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:36.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.180+0000 7fecdbe00640 1 -- 172.21.15.152:0/1165551482 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fecdc1603d0 msgr2=0x7fecc0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:36.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.180+0000 7fecdbe00640 1 --2- 172.21.15.152:0/1165551482 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fecdc1603d0 0x7fecc0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:36.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.180+0000 7fecdbe00640 1 -- 172.21.15.152:0/1165551482 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fecc00039e0 msgr2=0x7fecc0003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:36.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.180+0000 7fecdbe00640 1 --2- 172.21.15.152:0/1165551482 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fecc00039e0 0x7fecc0003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:36.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.180+0000 7fecdbe00640 1 -- 172.21.15.152:0/1165551482 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecc0003520 con 0x7fecc0004d40 2024-04-20T11:36:36.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.181+0000 7fecdbe00640 1 --2- 172.21.15.152:0/1165551482 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fecc0004d40 0x7fecc00051c0 secure :-1 s=READY pgs=2034 cs=0 l=1 rev1=1 crypto rx=0x7fecd4001720 tx=0x7fecc0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c9d774979929b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:36.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.182+0000 7fecda000640 1 -- 172.21.15.152:0/1165551482 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecd4001370 con 0x7fecc0004d40 2024-04-20T11:36:36.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.182+0000 7fecda000640 1 -- 172.21.15.152:0/1165551482 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecd40052d0 con 0x7fecc0004d40 2024-04-20T11:36:36.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.182+0000 7fecda000640 1 -- 172.21.15.152:0/1165551482 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecd4003830 con 0x7fecc0004d40 2024-04-20T11:36:36.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.182+0000 7fece0a00640 1 -- 172.21.15.152:0/1165551482 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fecc0004d40 msgr2=0x7fecc00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:36.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.182+0000 7fece0a00640 1 --2- 172.21.15.152:0/1165551482 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fecc0004d40 0x7fecc00051c0 secure :-1 s=READY pgs=2034 cs=0 l=1 rev1=1 crypto rx=0x7fecd4001720 tx=0x7fecc0005830 comp rx=0 tx=0).stop 2024-04-20T11:36:36.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.182+0000 7fecdbe00640 1 -- 172.21.15.152:0/1165551482 reap_dead start 2024-04-20T11:36:36.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.182+0000 7fece0a00640 1 -- 172.21.15.152:0/1165551482 shutdown_connections 2024-04-20T11:36:36.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.182+0000 7fece0a00640 1 -- 172.21.15.152:0/1165551482 >> 172.21.15.152:0/1165551482 conn(0x7fecdc08bfd0 msgr2=0x7fecdc0d4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:36.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.183+0000 7fece0a00640 1 -- 172.21.15.152:0/1165551482 shutdown_connections 2024-04-20T11:36:36.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.183+0000 7fece0a00640 1 -- 172.21.15.152:0/1165551482 wait complete. 2024-04-20T11:36:36.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.184+0000 7fece0a00640 1 Processor -- start 2024-04-20T11:36:36.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.184+0000 7fece0a00640 1 -- start start 2024-04-20T11:36:36.187 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.185+0000 7fece0a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fecc0004d40 0x7fecdc158790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:36.188 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.185+0000 7fece0a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fecdc0b8030 0x7fecdc159e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:36.188 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.185+0000 7fece0a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fecdc157230 0x7fecdc14c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:36.188 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.185+0000 7fece0a00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fecdc0cfce0 con 0x7fecc0004d40 2024-04-20T11:36:36.188 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.185+0000 7fece0a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fecdc0cfb60 con 0x7fecdc0b8030 2024-04-20T11:36:36.188 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.185+0000 7fece0a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fecdc0cfe60 con 0x7fecdc157230 2024-04-20T11:36:36.188 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.186+0000 7fecdb400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fecc0004d40 0x7fecdc158790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:36.188 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.186+0000 7fecdb400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fecc0004d40 0x7fecdc158790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:57438/0 (socket says 172.21.15.152:57438) 2024-04-20T11:36:36.188 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.186+0000 7fecdb400640 1 -- 172.21.15.152:0/2772531476 learned_addr learned my addr 172.21.15.152:0/2772531476 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:36.188 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.186+0000 7fecdaa00640 1 --2- 172.21.15.152:0/2772531476 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fecdc0b8030 0x7fecdc159e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:36.188 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.186+0000 7fecdbe00640 1 --2- 172.21.15.152:0/2772531476 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fecdc157230 0x7fecdc14c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:36.188 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.186+0000 7fecdb400640 1 -- 172.21.15.152:0/2772531476 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fecdc157230 msgr2=0x7fecdc14c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:36.188 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.186+0000 7fecdb400640 1 --2- 172.21.15.152:0/2772531476 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fecdc157230 0x7fecdc14c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:36.188 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.186+0000 7fecdb400640 1 -- 172.21.15.152:0/2772531476 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fecdc0b8030 msgr2=0x7fecdc159e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:36.188 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.186+0000 7fecdb400640 1 --2- 172.21.15.152:0/2772531476 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fecdc0b8030 0x7fecdc159e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:36.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.186+0000 7fecdb400640 1 -- 172.21.15.152:0/2772531476 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecdc1576f0 con 0x7fecc0004d40 2024-04-20T11:36:36.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.186+0000 7fecdb400640 1 --2- 172.21.15.152:0/2772531476 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fecc0004d40 0x7fecdc158790 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 crypto rx=0x7feccc005fd0 tx=0x7feccc00e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:36.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.187+0000 7fecd3400640 1 -- 172.21.15.152:0/2772531476 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feccc00edf0 con 0x7fecc0004d40 2024-04-20T11:36:36.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.187+0000 7fecd3400640 1 -- 172.21.15.152:0/2772531476 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feccc018070 con 0x7fecc0004d40 2024-04-20T11:36:36.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.187+0000 7fecd3400640 1 -- 172.21.15.152:0/2772531476 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feccc013660 con 0x7fecc0004d40 2024-04-20T11:36:36.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.187+0000 7fece0a00640 1 -- 172.21.15.152:0/2772531476 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecdc1579d0 con 0x7fecc0004d40 2024-04-20T11:36:36.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.187+0000 7fece0a00640 1 -- 172.21.15.152:0/2772531476 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecdc158270 con 0x7fecc0004d40 2024-04-20T11:36:36.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.188+0000 7fece0a00640 1 -- 172.21.15.152:0/2772531476 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecdc157ba0 con 0x7fecc0004d40 2024-04-20T11:36:36.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.189+0000 7fecd3400640 1 -- 172.21.15.152:0/2772531476 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) ==== 91754+0+0 (secure 0 0 0) 0x7feccc013800 con 0x7fecc0004d40 2024-04-20T11:36:36.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.190+0000 7fecd3400640 1 --2- 172.21.15.152:0/2772531476 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fecb806d100 0x7fecb806f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:36.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.190+0000 7fecd3400640 1 -- 172.21.15.152:0/2772531476 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5709+0+0 (secure 0 0 0) 0x7feccc08ab80 con 0x7fecc0004d40 2024-04-20T11:36:36.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.191+0000 7fecdaa00640 1 --2- 172.21.15.152:0/2772531476 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fecb806d100 0x7fecb806f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:36.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.192+0000 7fecdaa00640 1 --2- 172.21.15.152:0/2772531476 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fecb806d100 0x7fecb806f5c0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fecdc1f7d30 tx=0x7fecc4008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:36.195 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.194+0000 7fecd3400640 1 -- 172.21.15.152:0/2772531476 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7feccc05a8a0 con 0x7fecc0004d40 2024-04-20T11:36:36.394 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.393+0000 7fece0a00640 1 -- 172.21.15.152:0/2772531476 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fecdc0ba4d0 con 0x7fecc0004d40 2024-04-20T11:36:36.396 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.396+0000 7fecd3400640 1 -- 172.21.15.152:0/2772531476 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v142) ==== 74+0+16731 (secure 0 0 0) 0x7feccc05a1a0 con 0x7fecc0004d40 2024-04-20T11:36:36.397 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:36:36.397 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":142,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:36:10.484514+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:36:06.702770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"141","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-04-20T11:36:07.911633+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:36:36.399 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.398+0000 7fece0a00640 1 -- 172.21.15.152:0/2772531476 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fecb806d100 msgr2=0x7fecb806f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:36.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.398+0000 7fece0a00640 1 --2- 172.21.15.152:0/2772531476 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fecb806d100 0x7fecb806f5c0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fecdc1f7d30 tx=0x7fecc4008040 comp rx=0 tx=0).stop 2024-04-20T11:36:36.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.398+0000 7fece0a00640 1 -- 172.21.15.152:0/2772531476 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fecc0004d40 msgr2=0x7fecdc158790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:36.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.398+0000 7fece0a00640 1 --2- 172.21.15.152:0/2772531476 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fecc0004d40 0x7fecdc158790 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 crypto rx=0x7feccc005fd0 tx=0x7feccc00e530 comp rx=0 tx=0).stop 2024-04-20T11:36:36.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.398+0000 7fece0a00640 1 -- 172.21.15.152:0/2772531476 shutdown_connections 2024-04-20T11:36:36.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.398+0000 7fece0a00640 1 --2- 172.21.15.152:0/2772531476 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fecb806d100 0x7fecb806f5c0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:36.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.398+0000 7fece0a00640 1 --2- 172.21.15.152:0/2772531476 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fecdc157230 0x7fecdc14c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:36.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.398+0000 7fece0a00640 1 --2- 172.21.15.152:0/2772531476 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fecdc0b8030 0x7fecdc159e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:36.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.398+0000 7fece0a00640 1 --2- 172.21.15.152:0/2772531476 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fecc0004d40 0x7fecdc158790 unknown :-1 s=CLOSED pgs=2035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:36.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.398+0000 7fece0a00640 1 -- 172.21.15.152:0/2772531476 >> 172.21.15.152:0/2772531476 conn(0x7fecdc08bfd0 msgr2=0x7fecdc06df10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:36.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.399+0000 7fece0a00640 1 -- 172.21.15.152:0/2772531476 shutdown_connections 2024-04-20T11:36:36.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.399+0000 7fece0a00640 1 -- 172.21.15.152:0/2772531476 wait complete. 2024-04-20T11:36:36.418 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T11:26:47.246571+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 22, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T11:36:06.702770+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '141', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 23, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T11:36:07.911633+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '142', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T11:36:36.419 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T11:36:36.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.618+0000 7f2882200640 1 Processor -- start 2024-04-20T11:36:36.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.618+0000 7f2882200640 1 -- start start 2024-04-20T11:36:36.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.620+0000 7f2882200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f287c15c590 0x7f287c06bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:36.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.620+0000 7f2882200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f287c06c2e0 0x7f287c0ae470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:36.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.620+0000 7f2882200640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f287c152830 con 0x7f287c152450 2024-04-20T11:36:36.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.620+0000 7f2882200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f287c0b39c0 con 0x7f287c15c590 2024-04-20T11:36:36.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.620+0000 7f2882200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f287c0b3ca0 con 0x7f287c06c2e0 2024-04-20T11:36:36.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.620+0000 7f2880e00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f287c152450 0x7f287c0b30c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:55510/0 (socket says 172.21.15.152:55510) 2024-04-20T11:36:36.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.620+0000 7f2880e00640 1 -- 172.21.15.152:0/1008645351 learned_addr learned my addr 172.21.15.152:0/1008645351 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:36:36.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.620+0000 7f287be00640 1 --2- 172.21.15.152:0/1008645351 >> v2:172.21.15.152:3300/0 conn(0x7f287c15c590 0x7f287c06bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:36.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.620+0000 7f2881800640 1 --2- 172.21.15.152:0/1008645351 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f287c06c2e0 0x7f287c0ae470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:36.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.620+0000 7f287b400640 1 -- 172.21.15.152:0/1008645351 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3868124109 0 0) 0x7f287c152830 con 0x7f287c152450 2024-04-20T11:36:36.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.621+0000 7f287be00640 1 -- 172.21.15.152:0/1008645351 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f287c06c2e0 msgr2=0x7f287c0ae470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:36.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.621+0000 7f287be00640 1 --2- 172.21.15.152:0/1008645351 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f287c06c2e0 0x7f287c0ae470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:36.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.621+0000 7f287be00640 1 -- 172.21.15.152:0/1008645351 >> v1:172.21.15.152:6789/0 conn(0x7f287c152450 legacy=0x7f287c0b30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:36.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.621+0000 7f287be00640 1 -- 172.21.15.152:0/1008645351 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f287c0b3f80 con 0x7f287c15c590 2024-04-20T11:36:36.623 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.622+0000 7f287be00640 1 --2- 172.21.15.152:0/1008645351 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f287c15c590 0x7f287c06bda0 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7f28680097a0 tx=0x7f286802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f267a9d49de29fb5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:36.623 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.622+0000 7f287b400640 1 -- 172.21.15.152:0/1008645351 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2868035020 con 0x7f287c15c590 2024-04-20T11:36:36.623 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.622+0000 7f287b400640 1 -- 172.21.15.152:0/1008645351 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2868031830 con 0x7f287c15c590 2024-04-20T11:36:36.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.622+0000 7f287b400640 1 -- 172.21.15.152:0/1008645351 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2868031c00 con 0x7f287c15c590 2024-04-20T11:36:36.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.622+0000 7f2882200640 1 -- 172.21.15.152:0/1008645351 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f287c15c590 msgr2=0x7f287c06bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:36.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.622+0000 7f2882200640 1 --2- 172.21.15.152:0/1008645351 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f287c15c590 0x7f287c06bda0 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7f28680097a0 tx=0x7f286802b8d0 comp rx=0 tx=0).stop 2024-04-20T11:36:36.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.623+0000 7f2882200640 1 -- 172.21.15.152:0/1008645351 shutdown_connections 2024-04-20T11:36:36.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.623+0000 7f2882200640 1 --2- 172.21.15.152:0/1008645351 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f287c06c2e0 0x7f287c0ae470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:36.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.623+0000 7f2882200640 1 --2- 172.21.15.152:0/1008645351 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f287c15c590 0x7f287c06bda0 unknown :-1 s=CLOSED pgs=2037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:36.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.623+0000 7f2882200640 1 -- 172.21.15.152:0/1008645351 >> 172.21.15.152:0/1008645351 conn(0x7f287c0b21e0 msgr2=0x7f287c0b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:36.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.623+0000 7f2882200640 1 -- 172.21.15.152:0/1008645351 shutdown_connections 2024-04-20T11:36:36.625 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.623+0000 7f2882200640 1 -- 172.21.15.152:0/1008645351 wait complete. 2024-04-20T11:36:36.625 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.624+0000 7f2882200640 1 Processor -- start 2024-04-20T11:36:36.625 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.625+0000 7f2882200640 1 -- start start 2024-04-20T11:36:36.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.625+0000 7f2882200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f287c06c2e0 0x7f287c1f4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:36.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.625+0000 7f2882200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f287c152450 0x7f287c14caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:36.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.625+0000 7f2882200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f287c15c590 0x7f287c1501a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:36.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.625+0000 7f2882200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f287c0ae9b0 con 0x7f287c15c590 2024-04-20T11:36:36.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.625+0000 7f2882200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f287c0adf00 con 0x7f287c06c2e0 2024-04-20T11:36:36.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.625+0000 7f2882200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f287c0aeb30 con 0x7f287c152450 2024-04-20T11:36:36.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.626+0000 7f2881800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f287c15c590 0x7f287c1501a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:36.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.626+0000 7f2881800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f287c15c590 0x7f287c1501a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:57450/0 (socket says 172.21.15.152:57450) 2024-04-20T11:36:36.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.626+0000 7f2881800640 1 -- 172.21.15.152:0/2482622982 learned_addr learned my addr 172.21.15.152:0/2482622982 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:36.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.626+0000 7f287be00640 1 --2- 172.21.15.152:0/2482622982 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f287c152450 0x7f287c14caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:36.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.626+0000 7f2880e00640 1 --2- 172.21.15.152:0/2482622982 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f287c06c2e0 0x7f287c1f4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:36.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.626+0000 7f2881800640 1 -- 172.21.15.152:0/2482622982 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f287c152450 msgr2=0x7f287c14caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:36.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.626+0000 7f2881800640 1 --2- 172.21.15.152:0/2482622982 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f287c152450 0x7f287c14caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:36.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.626+0000 7f2881800640 1 -- 172.21.15.152:0/2482622982 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f287c06c2e0 msgr2=0x7f287c1f4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:36.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.626+0000 7f2881800640 1 --2- 172.21.15.152:0/2482622982 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f287c06c2e0 0x7f287c1f4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:36.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.626+0000 7f2881800640 1 -- 172.21.15.152:0/2482622982 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f287c1f4400 con 0x7f287c15c590 2024-04-20T11:36:36.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.626+0000 7f2880e00640 1 --2- 172.21.15.152:0/2482622982 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f287c06c2e0 0x7f287c1f4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:36:36.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.627+0000 7f2881800640 1 --2- 172.21.15.152:0/2482622982 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f287c15c590 0x7f287c1501a0 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x7f2874007e40 tx=0x7f287400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:36.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.627+0000 7f2878c00640 1 -- 172.21.15.152:0/2482622982 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2874014620 con 0x7f287c15c590 2024-04-20T11:36:36.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.627+0000 7f2878c00640 1 -- 172.21.15.152:0/2482622982 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2874019070 con 0x7f287c15c590 2024-04-20T11:36:36.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.627+0000 7f2878c00640 1 -- 172.21.15.152:0/2482622982 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2874014a20 con 0x7f287c15c590 2024-04-20T11:36:36.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.627+0000 7f2882200640 1 -- 172.21.15.152:0/2482622982 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f287c150a80 con 0x7f287c15c590 2024-04-20T11:36:36.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.627+0000 7f2882200640 1 -- 172.21.15.152:0/2482622982 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f287c1f46c0 con 0x7f287c15c590 2024-04-20T11:36:36.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.628+0000 7f2882200640 1 -- 172.21.15.152:0/2482622982 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f287c075e60 con 0x7f287c15c590 2024-04-20T11:36:36.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.629+0000 7f2878c00640 1 -- 172.21.15.152:0/2482622982 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) ==== 91754+0+0 (secure 0 0 0) 0x7f2874014bc0 con 0x7f287c15c590 2024-04-20T11:36:36.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.630+0000 7f2878c00640 1 --2- 172.21.15.152:0/2482622982 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f285c06d0d0 0x7f285c06f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:36.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.630+0000 7f2878c00640 1 -- 172.21.15.152:0/2482622982 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5709+0+0 (secure 0 0 0) 0x7f287408d630 con 0x7f287c15c590 2024-04-20T11:36:36.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.631+0000 7f2880e00640 1 --2- 172.21.15.152:0/2482622982 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f285c06d0d0 0x7f285c06f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:36.632 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.632+0000 7f2880e00640 1 --2- 172.21.15.152:0/2482622982 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f285c06d0d0 0x7f285c06f590 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f2870000f60 tx=0x7f2870009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:36.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.635+0000 7f2878c00640 1 -- 172.21.15.152:0/2482622982 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f287405ab90 con 0x7f287c15c590 2024-04-20T11:36:36.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.834+0000 7f2882200640 1 -- 172.21.15.152:0/2482622982 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f287c155010 con 0x7f287c15c590 2024-04-20T11:36:36.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.835+0000 7f2878c00640 1 -- 172.21.15.152:0/2482622982 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v142) ==== 93+0+10 (secure 0 0 0) 0x7f287405a490 con 0x7f287c15c590 2024-04-20T11:36:36.836 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 1 2024-04-20T11:36:36.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.837+0000 7f2882200640 1 -- 172.21.15.152:0/2482622982 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f285c06d0d0 msgr2=0x7f285c06f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:36.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.837+0000 7f2882200640 1 --2- 172.21.15.152:0/2482622982 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f285c06d0d0 0x7f285c06f590 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f2870000f60 tx=0x7f2870009040 comp rx=0 tx=0).stop 2024-04-20T11:36:36.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.837+0000 7f2882200640 1 -- 172.21.15.152:0/2482622982 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f287c15c590 msgr2=0x7f287c1501a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:36.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.837+0000 7f2882200640 1 --2- 172.21.15.152:0/2482622982 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f287c15c590 0x7f287c1501a0 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x7f2874007e40 tx=0x7f287400e4b0 comp rx=0 tx=0).stop 2024-04-20T11:36:36.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.838+0000 7f2882200640 1 -- 172.21.15.152:0/2482622982 shutdown_connections 2024-04-20T11:36:36.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.838+0000 7f2882200640 1 --2- 172.21.15.152:0/2482622982 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f285c06d0d0 0x7f285c06f590 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:36.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.838+0000 7f2882200640 1 --2- 172.21.15.152:0/2482622982 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f287c15c590 0x7f287c1501a0 unknown :-1 s=CLOSED pgs=2038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:36.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.838+0000 7f2882200640 1 --2- 172.21.15.152:0/2482622982 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f287c152450 0x7f287c14caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:36.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.838+0000 7f2882200640 1 --2- 172.21.15.152:0/2482622982 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f287c06c2e0 0x7f287c1f4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:36.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.838+0000 7f2882200640 1 -- 172.21.15.152:0/2482622982 >> 172.21.15.152:0/2482622982 conn(0x7f287c0b21e0 msgr2=0x7f287c155350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:36.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.838+0000 7f2882200640 1 -- 172.21.15.152:0/2482622982 shutdown_connections 2024-04-20T11:36:36.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:36.838+0000 7f2882200640 1 -- 172.21.15.152:0/2482622982 wait complete. 2024-04-20T11:36:36.859 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T11:36:37.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.049+0000 7f0fbbc00640 1 Processor -- start 2024-04-20T11:36:37.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.050+0000 7f0fbbc00640 1 -- start start 2024-04-20T11:36:37.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.051+0000 7f0fbbc00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f0fb4161e50 0x7f0fb40b7890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:37.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.051+0000 7f0fbbc00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f0fb40d2860 0x7f0fb40ba180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:37.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.051+0000 7f0fbbc00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0fb40d2cd0 con 0x7f0fb40cf4f0 2024-04-20T11:36:37.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.051+0000 7f0fbbc00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f0fb4162230 con 0x7f0fb40d2860 2024-04-20T11:36:37.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.051+0000 7f0fbbc00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f0fb40b7f70 con 0x7f0fb4161e50 2024-04-20T11:36:37.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.052+0000 7f0fabe00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f0fb40d2860 0x7f0fb40ba180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:37.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.052+0000 7f0fabe00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f0fb40d2860 0x7f0fb40ba180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:57458/0 (socket says 172.21.15.152:57458) 2024-04-20T11:36:37.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.052+0000 7f0fb9c00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f0fb40cf4f0 0x7f0fb40cf920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:55516/0 (socket says 172.21.15.152:55516) 2024-04-20T11:36:37.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.052+0000 7f0fabe00640 1 -- 172.21.15.152:0/2335666892 learned_addr learned my addr 172.21.15.152:0/2335666892 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:37.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.052+0000 7f0fb9200640 1 --2- 172.21.15.152:0/2335666892 >> v2:172.21.15.175:3300/0 conn(0x7f0fb4161e50 0x7f0fb40b7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:37.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.052+0000 7f0fabe00640 1 -- 172.21.15.152:0/2335666892 >> v2:172.21.15.175:3300/0 conn(0x7f0fb4161e50 msgr2=0x7f0fb40b7890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:37.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.052+0000 7f0fabe00640 1 --2- 172.21.15.152:0/2335666892 >> v2:172.21.15.175:3300/0 conn(0x7f0fb4161e50 0x7f0fb40b7890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:37.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.052+0000 7f0fabe00640 1 -- 172.21.15.152:0/2335666892 >> v1:172.21.15.152:6789/0 conn(0x7f0fb40cf4f0 legacy=0x7f0fb40cf920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:37.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.052+0000 7f0fabe00640 1 -- 172.21.15.152:0/2335666892 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fb40b8250 con 0x7f0fb40d2860 2024-04-20T11:36:37.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.052+0000 7f0fb3e00640 1 -- 172.21.15.152:0/2335666892 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2257086175 0 0) 0x7f0fb40d2cd0 con 0x7f0fb40cf4f0 2024-04-20T11:36:37.054 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.053+0000 7f0fabe00640 1 --2- 172.21.15.152:0/2335666892 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0fb40d2860 0x7f0fb40ba180 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7f0fb40b6d50 tx=0x7f0fa002ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c1b8c1a78a9d6b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:37.054 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.054+0000 7f0fb3e00640 1 -- 172.21.15.152:0/2335666892 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fa002f850 con 0x7f0fb40d2860 2024-04-20T11:36:37.054 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.054+0000 7f0fb3e00640 1 -- 172.21.15.152:0/2335666892 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fa002f9f0 con 0x7f0fb40d2860 2024-04-20T11:36:37.054 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.054+0000 7f0fb3e00640 1 -- 172.21.15.152:0/2335666892 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fa002fdc0 con 0x7f0fb40d2860 2024-04-20T11:36:37.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.054+0000 7f0fbbc00640 1 -- 172.21.15.152:0/2335666892 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0fb40d2860 msgr2=0x7f0fb40ba180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:37.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.054+0000 7f0fbbc00640 1 --2- 172.21.15.152:0/2335666892 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0fb40d2860 0x7f0fb40ba180 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7f0fb40b6d50 tx=0x7f0fa002ba30 comp rx=0 tx=0).stop 2024-04-20T11:36:37.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.055+0000 7f0fbbc00640 1 -- 172.21.15.152:0/2335666892 shutdown_connections 2024-04-20T11:36:37.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.055+0000 7f0fbbc00640 1 --2- 172.21.15.152:0/2335666892 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0fb40d2860 0x7f0fb40ba180 unknown :-1 s=CLOSED pgs=2040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:37.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.055+0000 7f0fbbc00640 1 --2- 172.21.15.152:0/2335666892 >> v2:172.21.15.175:3300/0 conn(0x7f0fb4161e50 0x7f0fb40b7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:37.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.055+0000 7f0fbbc00640 1 -- 172.21.15.152:0/2335666892 >> 172.21.15.152:0/2335666892 conn(0x7f0fb4067ae0 msgr2=0x7f0fb4067ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:37.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.055+0000 7f0fbbc00640 1 -- 172.21.15.152:0/2335666892 shutdown_connections 2024-04-20T11:36:37.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.055+0000 7f0fbbc00640 1 -- 172.21.15.152:0/2335666892 wait complete. 2024-04-20T11:36:37.057 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.056+0000 7f0fbbc00640 1 Processor -- start 2024-04-20T11:36:37.057 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.056+0000 7f0fbbc00640 1 -- start start 2024-04-20T11:36:37.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.057+0000 7f0fbbc00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0fb40cf4f0 0x7f0fb4154ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:37.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.057+0000 7f0fbbc00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0fb40d2860 0x7f0fb40b5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:37.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.057+0000 7f0fbbc00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0fb4161e50 0x7f0fb40b5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:37.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.057+0000 7f0fbbc00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f0fb40b8af0 con 0x7f0fb4161e50 2024-04-20T11:36:37.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.057+0000 7f0fbbc00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0fb40b8810 con 0x7f0fb40cf4f0 2024-04-20T11:36:37.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.057+0000 7f0fbbc00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0fb40ba6c0 con 0x7f0fb40d2860 2024-04-20T11:36:37.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.058+0000 7f0fb9c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0fb4161e50 0x7f0fb40b5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:37.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.058+0000 7f0fb9c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0fb4161e50 0x7f0fb40b5990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:57466/0 (socket says 172.21.15.152:57466) 2024-04-20T11:36:37.061 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.058+0000 7f0fb9c00640 1 -- 172.21.15.152:0/436188107 learned_addr learned my addr 172.21.15.152:0/436188107 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:37.061 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.058+0000 7f0fabe00640 1 --2- 172.21.15.152:0/436188107 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0fb40d2860 0x7f0fb40b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:37.061 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.058+0000 7f0fb9200640 1 --2- 172.21.15.152:0/436188107 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0fb40cf4f0 0x7f0fb4154ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:37.061 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.058+0000 7f0fb9c00640 1 -- 172.21.15.152:0/436188107 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0fb40d2860 msgr2=0x7f0fb40b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:37.061 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.058+0000 7f0fb9c00640 1 --2- 172.21.15.152:0/436188107 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0fb40d2860 0x7f0fb40b5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:37.061 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.058+0000 7f0fb9c00640 1 -- 172.21.15.152:0/436188107 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0fb40cf4f0 msgr2=0x7f0fb4154ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:37.061 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.058+0000 7f0fb9c00640 1 --2- 172.21.15.152:0/436188107 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0fb40cf4f0 0x7f0fb4154ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:37.061 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.058+0000 7f0fb9c00640 1 -- 172.21.15.152:0/436188107 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fb4155400 con 0x7f0fb4161e50 2024-04-20T11:36:37.061 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.058+0000 7f0fb9200640 1 --2- 172.21.15.152:0/436188107 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0fb40cf4f0 0x7f0fb4154ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:36:37.061 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.058+0000 7f0fb9c00640 1 --2- 172.21.15.152:0/436188107 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0fb4161e50 0x7f0fb40b5990 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x7f0fac00c730 tx=0x7f0fac00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:37.061 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.059+0000 7f0fb1600640 1 -- 172.21.15.152:0/436188107 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fac018070 con 0x7f0fb4161e50 2024-04-20T11:36:37.061 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.059+0000 7f0fb1600640 1 -- 172.21.15.152:0/436188107 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fac00ce40 con 0x7f0fb4161e50 2024-04-20T11:36:37.061 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.059+0000 7f0fb1600640 1 -- 172.21.15.152:0/436188107 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fac00a860 con 0x7f0fb4161e50 2024-04-20T11:36:37.061 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.059+0000 7f0fbbc00640 1 -- 172.21.15.152:0/436188107 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fb41f52d0 con 0x7f0fb4161e50 2024-04-20T11:36:37.062 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.059+0000 7f0fbbc00640 1 -- 172.21.15.152:0/436188107 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fb41f58b0 con 0x7f0fb4161e50 2024-04-20T11:36:37.063 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.060+0000 7f0fbbc00640 1 -- 172.21.15.152:0/436188107 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fb4160b60 con 0x7f0fb4161e50 2024-04-20T11:36:37.063 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.061+0000 7f0fb1600640 1 -- 172.21.15.152:0/436188107 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) ==== 91754+0+0 (secure 0 0 0) 0x7f0fac01b070 con 0x7f0fb4161e50 2024-04-20T11:36:37.063 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.062+0000 7f0fb1600640 1 --2- 172.21.15.152:0/436188107 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f0f8806d0d0 0x7f0f8806f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:37.063 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.062+0000 7f0fb1600640 1 -- 172.21.15.152:0/436188107 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5709+0+0 (secure 0 0 0) 0x7f0fac089b30 con 0x7f0fb4161e50 2024-04-20T11:36:37.063 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.063+0000 7f0fb9200640 1 --2- 172.21.15.152:0/436188107 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f0f8806d0d0 0x7f0f8806f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:37.064 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.064+0000 7f0fb9200640 1 --2- 172.21.15.152:0/436188107 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f0f8806d0d0 0x7f0f8806f590 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f0f98008890 tx=0x7f0f98008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:37.067 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.067+0000 7f0fb1600640 1 -- 172.21.15.152:0/436188107 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f0fb4160b60 con 0x7f0fb4161e50 2024-04-20T11:36:37.265 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.264+0000 7f0fbbc00640 1 -- 172.21.15.152:0/436188107 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f0fb40689f0 con 0x7f0fb4161e50 2024-04-20T11:36:37.266 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.265+0000 7f0fb1600640 1 -- 172.21.15.152:0/436188107 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v142) ==== 104+0+11 (secure 0 0 0) 0x7f0fac059850 con 0x7f0fb4161e50 2024-04-20T11:36:37.266 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 32 2024-04-20T11:36:37.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.268+0000 7f0fbbc00640 1 -- 172.21.15.152:0/436188107 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f0f8806d0d0 msgr2=0x7f0f8806f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:37.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.268+0000 7f0fbbc00640 1 --2- 172.21.15.152:0/436188107 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f0f8806d0d0 0x7f0f8806f590 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f0f98008890 tx=0x7f0f98008040 comp rx=0 tx=0).stop 2024-04-20T11:36:37.270 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.268+0000 7f0fbbc00640 1 -- 172.21.15.152:0/436188107 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0fb4161e50 msgr2=0x7f0fb40b5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:37.270 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.268+0000 7f0fbbc00640 1 --2- 172.21.15.152:0/436188107 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0fb4161e50 0x7f0fb40b5990 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x7f0fac00c730 tx=0x7f0fac00cbf0 comp rx=0 tx=0).stop 2024-04-20T11:36:37.270 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.268+0000 7f0fbbc00640 1 -- 172.21.15.152:0/436188107 shutdown_connections 2024-04-20T11:36:37.270 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.268+0000 7f0fbbc00640 1 --2- 172.21.15.152:0/436188107 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f0f8806d0d0 0x7f0f8806f590 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:37.270 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.268+0000 7f0fbbc00640 1 --2- 172.21.15.152:0/436188107 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0fb4161e50 0x7f0fb40b5990 unknown :-1 s=CLOSED pgs=2041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:37.270 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.268+0000 7f0fbbc00640 1 --2- 172.21.15.152:0/436188107 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0fb40d2860 0x7f0fb40b5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:37.271 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.268+0000 7f0fbbc00640 1 --2- 172.21.15.152:0/436188107 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0fb40cf4f0 0x7f0fb4154ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:37.271 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.268+0000 7f0fbbc00640 1 -- 172.21.15.152:0/436188107 >> 172.21.15.152:0/436188107 conn(0x7f0fb4067ae0 msgr2=0x7f0fb40b42f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:37.271 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.268+0000 7f0fbbc00640 1 -- 172.21.15.152:0/436188107 shutdown_connections 2024-04-20T11:36:37.271 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.269+0000 7f0fbbc00640 1 -- 172.21.15.152:0/436188107 wait complete. 2024-04-20T11:36:37.289 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T11:36:37.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.495+0000 7fcbee000640 1 Processor -- start 2024-04-20T11:36:37.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.496+0000 7fcbee000640 1 -- start start 2024-04-20T11:36:37.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.497+0000 7fcbee000640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fcbe80cf5b0 0x7fcbe80d4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:37.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.497+0000 7fcbee000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fcbe8160230 0x7fcbe81606e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:37.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.497+0000 7fcbee000640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcbe80d2b70 con 0x7fcbe80ceeb0 2024-04-20T11:36:37.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.497+0000 7fcbee000640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fcbe80d5420 con 0x7fcbe8160230 2024-04-20T11:36:37.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.497+0000 7fcbee000640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fcbe80d2e50 con 0x7fcbe80cf5b0 2024-04-20T11:36:37.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.497+0000 7fcbed600640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fcbe8160230 0x7fcbe81606e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:37.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.497+0000 7fcbe7e00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fcbe80ceeb0 0x7fcbe80cf990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:55522/0 (socket says 172.21.15.152:55522) 2024-04-20T11:36:37.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.497+0000 7fcbe7e00640 1 -- 172.21.15.152:0/1413688004 learned_addr learned my addr 172.21.15.152:0/1413688004 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:36:37.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.498+0000 7fcbecc00640 1 --2- 172.21.15.152:0/1413688004 >> v2:172.21.15.175:3300/0 conn(0x7fcbe80cf5b0 0x7fcbe80d4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:37.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.498+0000 7fcbed600640 1 -- 172.21.15.152:0/1413688004 >> v2:172.21.15.175:3300/0 conn(0x7fcbe80cf5b0 msgr2=0x7fcbe80d4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:37.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.498+0000 7fcbed600640 1 --2- 172.21.15.152:0/1413688004 >> v2:172.21.15.175:3300/0 conn(0x7fcbe80cf5b0 0x7fcbe80d4ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:37.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.498+0000 7fcbed600640 1 -- 172.21.15.152:0/1413688004 >> v1:172.21.15.152:6789/0 conn(0x7fcbe80ceeb0 legacy=0x7fcbe80cf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:37.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.498+0000 7fcbed600640 1 -- 172.21.15.152:0/1413688004 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbe80d3130 con 0x7fcbe8160230 2024-04-20T11:36:37.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.498+0000 7fcbe7400640 1 -- 172.21.15.152:0/1413688004 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1252268376 0 0) 0x7fcbe80d2b70 con 0x7fcbe80ceeb0 2024-04-20T11:36:37.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.499+0000 7fcbed600640 1 --2- 172.21.15.152:0/1413688004 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcbe8160230 0x7fcbe81606e0 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x7fcbe808cc10 tx=0x7fcbe002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=549c8f3c128e78bb server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:37.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.500+0000 7fcbe7400640 1 -- 172.21.15.152:0/1413688004 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe0034070 con 0x7fcbe8160230 2024-04-20T11:36:37.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.500+0000 7fcbe7400640 1 -- 172.21.15.152:0/1413688004 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbe0030610 con 0x7fcbe8160230 2024-04-20T11:36:37.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.500+0000 7fcbe7400640 1 -- 172.21.15.152:0/1413688004 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe00309e0 con 0x7fcbe8160230 2024-04-20T11:36:37.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.500+0000 7fcbee000640 1 -- 172.21.15.152:0/1413688004 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcbe8160230 msgr2=0x7fcbe81606e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:37.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.500+0000 7fcbee000640 1 --2- 172.21.15.152:0/1413688004 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcbe8160230 0x7fcbe81606e0 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x7fcbe808cc10 tx=0x7fcbe002b960 comp rx=0 tx=0).stop 2024-04-20T11:36:37.502 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.500+0000 7fcbee000640 1 -- 172.21.15.152:0/1413688004 shutdown_connections 2024-04-20T11:36:37.502 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.500+0000 7fcbee000640 1 --2- 172.21.15.152:0/1413688004 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcbe8160230 0x7fcbe81606e0 unknown :-1 s=CLOSED pgs=2043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:37.502 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.500+0000 7fcbee000640 1 --2- 172.21.15.152:0/1413688004 >> v2:172.21.15.175:3300/0 conn(0x7fcbe80cf5b0 0x7fcbe80d4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:37.502 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.500+0000 7fcbee000640 1 -- 172.21.15.152:0/1413688004 >> 172.21.15.152:0/1413688004 conn(0x7fcbe808bfd0 msgr2=0x7fcbe80d4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:37.502 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.500+0000 7fcbee000640 1 -- 172.21.15.152:0/1413688004 shutdown_connections 2024-04-20T11:36:37.502 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.501+0000 7fcbee000640 1 -- 172.21.15.152:0/1413688004 wait complete. 2024-04-20T11:36:37.502 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.502+0000 7fcbee000640 1 Processor -- start 2024-04-20T11:36:37.503 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.502+0000 7fcbee000640 1 -- start start 2024-04-20T11:36:37.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.503+0000 7fcbee000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcbe80ceeb0 0x7fcbe80b97c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:37.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.503+0000 7fcbee000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcbe80cf5b0 0x7fcbe80c6d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:37.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.503+0000 7fcbee000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcbe8160230 0x7fcbe80c7240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:37.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.503+0000 7fcbee000640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fcbe80d3c10 con 0x7fcbe80ceeb0 2024-04-20T11:36:37.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.503+0000 7fcbee000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcbe80d36f0 con 0x7fcbe8160230 2024-04-20T11:36:37.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.503+0000 7fcbee000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fcbe80d2790 con 0x7fcbe80cf5b0 2024-04-20T11:36:37.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.503+0000 7fcbecc00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcbe80ceeb0 0x7fcbe80b97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:37.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.504+0000 7fcbecc00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcbe80ceeb0 0x7fcbe80b97c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:57476/0 (socket says 172.21.15.152:57476) 2024-04-20T11:36:37.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.504+0000 7fcbecc00640 1 -- 172.21.15.152:0/2554689927 learned_addr learned my addr 172.21.15.152:0/2554689927 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:37.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.504+0000 7fcbed600640 1 --2- 172.21.15.152:0/2554689927 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcbe8160230 0x7fcbe80c7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:37.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.504+0000 7fcbecc00640 1 -- 172.21.15.152:0/2554689927 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcbe80cf5b0 msgr2=0x7fcbe80c6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:37.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.504+0000 7fcbecc00640 1 --2- 172.21.15.152:0/2554689927 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcbe80cf5b0 0x7fcbe80c6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:37.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.504+0000 7fcbecc00640 1 -- 172.21.15.152:0/2554689927 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcbe8160230 msgr2=0x7fcbe80c7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:37.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.504+0000 7fcbecc00640 1 --2- 172.21.15.152:0/2554689927 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcbe8160230 0x7fcbe80c7240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:37.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.504+0000 7fcbecc00640 1 -- 172.21.15.152:0/2554689927 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbe80b7ff0 con 0x7fcbe80ceeb0 2024-04-20T11:36:37.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.504+0000 7fcbed600640 1 --2- 172.21.15.152:0/2554689927 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcbe8160230 0x7fcbe80c7240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:36:37.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.504+0000 7fcbecc00640 1 --2- 172.21.15.152:0/2554689927 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcbe80ceeb0 0x7fcbe80b97c0 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 crypto rx=0x7fcbd8007ef0 tx=0x7fcbd800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:37.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.504+0000 7fcbe4c00640 1 -- 172.21.15.152:0/2554689927 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbd8014620 con 0x7fcbe80ceeb0 2024-04-20T11:36:37.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.505+0000 7fcbe4c00640 1 -- 172.21.15.152:0/2554689927 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbd8019070 con 0x7fcbe80ceeb0 2024-04-20T11:36:37.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.505+0000 7fcbe4c00640 1 -- 172.21.15.152:0/2554689927 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbd8014a00 con 0x7fcbe80ceeb0 2024-04-20T11:36:37.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.505+0000 7fcbee000640 1 -- 172.21.15.152:0/2554689927 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbe80b82d0 con 0x7fcbe80ceeb0 2024-04-20T11:36:37.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.505+0000 7fcbee000640 1 -- 172.21.15.152:0/2554689927 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbe80b8890 con 0x7fcbe80ceeb0 2024-04-20T11:36:37.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.506+0000 7fcbee000640 1 -- 172.21.15.152:0/2554689927 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbe8075e60 con 0x7fcbe80ceeb0 2024-04-20T11:36:37.508 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.507+0000 7fcbe4c00640 1 -- 172.21.15.152:0/2554689927 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) ==== 91754+0+0 (secure 0 0 0) 0x7fcbd8014ba0 con 0x7fcbe80ceeb0 2024-04-20T11:36:37.508 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.508+0000 7fcbe4c00640 1 --2- 172.21.15.152:0/2554689927 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fcbc806d0d0 0x7fcbc806f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:37.508 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.508+0000 7fcbe4c00640 1 -- 172.21.15.152:0/2554689927 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5709+0+0 (secure 0 0 0) 0x7fcbd808d680 con 0x7fcbe80ceeb0 2024-04-20T11:36:37.509 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.508+0000 7fcbe7e00640 1 --2- 172.21.15.152:0/2554689927 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fcbc806d0d0 0x7fcbc806f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:37.510 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.509+0000 7fcbe7e00640 1 --2- 172.21.15.152:0/2554689927 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fcbc806d0d0 0x7fcbc806f590 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fcbe80cb0a0 tx=0x7fcbd4007910 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:37.512 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.512+0000 7fcbe4c00640 1 -- 172.21.15.152:0/2554689927 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fcbd805ab50 con 0x7fcbe80ceeb0 2024-04-20T11:36:37.716 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.715+0000 7fcbee000640 1 -- 172.21.15.152:0/2554689927 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fcbe80cf390 con 0x7fcbe80ceeb0 2024-04-20T11:36:37.717 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.716+0000 7fcbe4c00640 1 -- 172.21.15.152:0/2554689927 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v142) ==== 100+0+11 (secure 0 0 0) 0x7fcbd805a450 con 0x7fcbe80ceeb0 2024-04-20T11:36:37.717 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 64 2024-04-20T11:36:37.719 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.718+0000 7fcbee000640 1 -- 172.21.15.152:0/2554689927 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fcbc806d0d0 msgr2=0x7fcbc806f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:37.719 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.718+0000 7fcbee000640 1 --2- 172.21.15.152:0/2554689927 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fcbc806d0d0 0x7fcbc806f590 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fcbe80cb0a0 tx=0x7fcbd4007910 comp rx=0 tx=0).stop 2024-04-20T11:36:37.720 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.718+0000 7fcbee000640 1 -- 172.21.15.152:0/2554689927 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcbe80ceeb0 msgr2=0x7fcbe80b97c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:37.720 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.718+0000 7fcbee000640 1 --2- 172.21.15.152:0/2554689927 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcbe80ceeb0 0x7fcbe80b97c0 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 crypto rx=0x7fcbd8007ef0 tx=0x7fcbd800e4b0 comp rx=0 tx=0).stop 2024-04-20T11:36:37.720 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.719+0000 7fcbee000640 1 -- 172.21.15.152:0/2554689927 shutdown_connections 2024-04-20T11:36:37.720 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.719+0000 7fcbee000640 1 --2- 172.21.15.152:0/2554689927 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fcbc806d0d0 0x7fcbc806f590 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:37.720 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.719+0000 7fcbee000640 1 --2- 172.21.15.152:0/2554689927 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcbe8160230 0x7fcbe80c7240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:37.720 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.719+0000 7fcbee000640 1 --2- 172.21.15.152:0/2554689927 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcbe80cf5b0 0x7fcbe80c6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:37.720 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.719+0000 7fcbee000640 1 --2- 172.21.15.152:0/2554689927 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcbe80ceeb0 0x7fcbe80b97c0 unknown :-1 s=CLOSED pgs=2044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:37.720 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.719+0000 7fcbee000640 1 -- 172.21.15.152:0/2554689927 >> 172.21.15.152:0/2554689927 conn(0x7fcbe808bfd0 msgr2=0x7fcbe806caf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:37.720 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.719+0000 7fcbee000640 1 -- 172.21.15.152:0/2554689927 shutdown_connections 2024-04-20T11:36:37.720 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.719+0000 7fcbee000640 1 -- 172.21.15.152:0/2554689927 wait complete. 2024-04-20T11:36:37.739 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-20T11:36:37.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.942+0000 7f6e3ca00640 1 Processor -- start 2024-04-20T11:36:37.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.943+0000 7f6e3ca00640 1 -- start start 2024-04-20T11:36:37.945 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.944+0000 7f6e3ca00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f6e3815c7f0 0x7f6e38072b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:37.945 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.944+0000 7f6e3ca00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e380b4870 0x7f6e380ad2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:37.945 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.944+0000 7f6e3ca00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e380ce390 con 0x7f6e38073b60 2024-04-20T11:36:37.945 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.944+0000 7f6e3ca00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f6e38073ff0 con 0x7f6e3815c7f0 2024-04-20T11:36:37.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.944+0000 7f6e3ca00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6e380ce140 con 0x7f6e380b4870 2024-04-20T11:36:37.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.944+0000 7f6e37400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f6e3815c7f0 0x7f6e38072b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:37.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.944+0000 7f6e36a00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f6e38073b60 0x7f6e380b4160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:55530/0 (socket says 172.21.15.152:55530) 2024-04-20T11:36:37.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.944+0000 7f6e36a00640 1 -- 172.21.15.152:0/2743546072 learned_addr learned my addr 172.21.15.152:0/2743546072 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:36:37.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.945+0000 7f6e37e00640 1 --2- 172.21.15.152:0/2743546072 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e380b4870 0x7f6e380ad2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:37.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.945+0000 7f6e37400640 1 -- 172.21.15.152:0/2743546072 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e380b4870 msgr2=0x7f6e380ad2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:37.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.945+0000 7f6e37400640 1 --2- 172.21.15.152:0/2743546072 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e380b4870 0x7f6e380ad2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:37.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.945+0000 7f6e37400640 1 -- 172.21.15.152:0/2743546072 >> v1:172.21.15.152:6789/0 conn(0x7f6e38073b60 legacy=0x7f6e380b4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:37.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.945+0000 7f6e37400640 1 -- 172.21.15.152:0/2743546072 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e380ce670 con 0x7f6e3815c7f0 2024-04-20T11:36:37.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.945+0000 7f6e36000640 1 -- 172.21.15.152:0/2743546072 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2383684015 0 0) 0x7f6e380ce390 con 0x7f6e38073b60 2024-04-20T11:36:37.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.946+0000 7f6e37400640 1 --2- 172.21.15.152:0/2743546072 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6e3815c7f0 0x7f6e38072b70 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7f6e280097a0 tx=0x7f6e2802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2d80917bf1db42c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:37.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.946+0000 7f6e36000640 1 -- 172.21.15.152:0/2743546072 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e280315a0 con 0x7f6e3815c7f0 2024-04-20T11:36:37.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.946+0000 7f6e36000640 1 -- 172.21.15.152:0/2743546072 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e28037070 con 0x7f6e3815c7f0 2024-04-20T11:36:37.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.946+0000 7f6e36000640 1 -- 172.21.15.152:0/2743546072 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e28031960 con 0x7f6e3815c7f0 2024-04-20T11:36:37.948 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.947+0000 7f6e3ca00640 1 -- 172.21.15.152:0/2743546072 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6e3815c7f0 msgr2=0x7f6e38072b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:37.948 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.947+0000 7f6e3ca00640 1 --2- 172.21.15.152:0/2743546072 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6e3815c7f0 0x7f6e38072b70 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7f6e280097a0 tx=0x7f6e2802b840 comp rx=0 tx=0).stop 2024-04-20T11:36:37.948 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.947+0000 7f6e3ca00640 1 -- 172.21.15.152:0/2743546072 shutdown_connections 2024-04-20T11:36:37.948 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.947+0000 7f6e3ca00640 1 --2- 172.21.15.152:0/2743546072 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e380b4870 0x7f6e380ad2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:37.948 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.947+0000 7f6e3ca00640 1 --2- 172.21.15.152:0/2743546072 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6e3815c7f0 0x7f6e38072b70 unknown :-1 s=CLOSED pgs=2046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:37.948 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.947+0000 7f6e3ca00640 1 -- 172.21.15.152:0/2743546072 >> 172.21.15.152:0/2743546072 conn(0x7f6e380d0260 msgr2=0x7f6e380d0670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:37.948 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.947+0000 7f6e3ca00640 1 -- 172.21.15.152:0/2743546072 shutdown_connections 2024-04-20T11:36:37.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.947+0000 7f6e3ca00640 1 -- 172.21.15.152:0/2743546072 wait complete. 2024-04-20T11:36:37.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.949+0000 7f6e3ca00640 1 Processor -- start 2024-04-20T11:36:37.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.949+0000 7f6e3ca00640 1 -- start start 2024-04-20T11:36:37.951 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.950+0000 7f6e3ca00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6e38073b60 0x7f6e381e9120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:37.951 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.950+0000 7f6e3ca00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e380b4870 0x7f6e381f3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:37.951 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.950+0000 7f6e3ca00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e3809e030 0x7f6e38144b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:37.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.950+0000 7f6e3ca00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f6e380cef10 con 0x7f6e38073b60 2024-04-20T11:36:37.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.950+0000 7f6e3ca00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6e380cec30 con 0x7f6e3809e030 2024-04-20T11:36:37.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.950+0000 7f6e3ca00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6e380cf430 con 0x7f6e380b4870 2024-04-20T11:36:37.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.950+0000 7f6e37400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6e38073b60 0x7f6e381e9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:37.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.950+0000 7f6e37400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6e38073b60 0x7f6e381e9120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:57504/0 (socket says 172.21.15.152:57504) 2024-04-20T11:36:37.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.950+0000 7f6e37400640 1 -- 172.21.15.152:0/909806935 learned_addr learned my addr 172.21.15.152:0/909806935 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:37.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.951+0000 7f6e36a00640 1 --2- 172.21.15.152:0/909806935 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e380b4870 0x7f6e381f3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:37.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.951+0000 7f6e37400640 1 -- 172.21.15.152:0/909806935 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e380b4870 msgr2=0x7f6e381f3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:37.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.951+0000 7f6e37400640 1 --2- 172.21.15.152:0/909806935 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e380b4870 0x7f6e381f3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:37.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.951+0000 7f6e37e00640 1 --2- 172.21.15.152:0/909806935 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e3809e030 0x7f6e38144b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:37.953 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.951+0000 7f6e37400640 1 -- 172.21.15.152:0/909806935 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e3809e030 msgr2=0x7f6e38144b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:37.953 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.951+0000 7f6e37400640 1 --2- 172.21.15.152:0/909806935 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e3809e030 0x7f6e38144b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:37.953 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.951+0000 7f6e37400640 1 -- 172.21.15.152:0/909806935 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e381f4510 con 0x7f6e38073b60 2024-04-20T11:36:37.953 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.951+0000 7f6e36a00640 1 --2- 172.21.15.152:0/909806935 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e380b4870 0x7f6e381f3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:36:37.954 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.951+0000 7f6e37400640 1 --2- 172.21.15.152:0/909806935 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6e38073b60 0x7f6e381e9120 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 crypto rx=0x7f6e280097a0 tx=0x7f6e280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:37.954 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.952+0000 7f6e2f400640 1 -- 172.21.15.152:0/909806935 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e28004230 con 0x7f6e38073b60 2024-04-20T11:36:37.954 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.952+0000 7f6e2f400640 1 -- 172.21.15.152:0/909806935 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e280043f0 con 0x7f6e38073b60 2024-04-20T11:36:37.954 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.952+0000 7f6e2f400640 1 -- 172.21.15.152:0/909806935 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e28003850 con 0x7f6e38073b60 2024-04-20T11:36:37.954 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.952+0000 7f6e3ca00640 1 -- 172.21.15.152:0/909806935 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e381453e0 con 0x7f6e38073b60 2024-04-20T11:36:37.954 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.952+0000 7f6e3ca00640 1 -- 172.21.15.152:0/909806935 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e381459a0 con 0x7f6e38073b60 2024-04-20T11:36:37.954 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.953+0000 7f6e3ca00640 1 -- 172.21.15.152:0/909806935 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e38075e60 con 0x7f6e38073b60 2024-04-20T11:36:37.954 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.954+0000 7f6e2f400640 1 -- 172.21.15.152:0/909806935 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) ==== 91754+0+0 (secure 0 0 0) 0x7f6e280039f0 con 0x7f6e38073b60 2024-04-20T11:36:37.955 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.955+0000 7f6e2f400640 1 --2- 172.21.15.152:0/909806935 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f6e1806d100 0x7f6e1806f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:37.955 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.955+0000 7f6e2f400640 1 -- 172.21.15.152:0/909806935 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5709+0+0 (secure 0 0 0) 0x7f6e2802f030 con 0x7f6e38073b60 2024-04-20T11:36:37.955 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.955+0000 7f6e36a00640 1 --2- 172.21.15.152:0/909806935 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f6e1806d100 0x7f6e1806f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:37.957 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.957+0000 7f6e36a00640 1 --2- 172.21.15.152:0/909806935 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f6e1806d100 0x7f6e1806f5c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f6e380721b0 tx=0x7f6e24009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:37.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:37.959+0000 7f6e2f400640 1 -- 172.21.15.152:0/909806935 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f6e28041070 con 0x7f6e38073b60 2024-04-20T11:36:38.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:38.177+0000 7f6e3ca00640 1 -- 172.21.15.152:0/909806935 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f6e381450d0 con 0x7f6e38073b60 2024-04-20T11:36:38.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:38.178+0000 7f6e2f400640 1 -- 172.21.15.152:0/909806935 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v142) ==== 80+0+0 (secure 0 0 0) 0x7f6e2807c120 con 0x7f6e38073b60 2024-04-20T11:36:38.179 INFO:teuthology.orchestra.run.smithi152.stderr:listed 0 entries 2024-04-20T11:36:38.182 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:38.181+0000 7f6e3ca00640 1 -- 172.21.15.152:0/909806935 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f6e1806d100 msgr2=0x7f6e1806f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:38.182 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:38.181+0000 7f6e3ca00640 1 --2- 172.21.15.152:0/909806935 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f6e1806d100 0x7f6e1806f5c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f6e380721b0 tx=0x7f6e24009040 comp rx=0 tx=0).stop 2024-04-20T11:36:38.182 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:38.181+0000 7f6e3ca00640 1 -- 172.21.15.152:0/909806935 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6e38073b60 msgr2=0x7f6e381e9120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:38.182 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:38.181+0000 7f6e3ca00640 1 --2- 172.21.15.152:0/909806935 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6e38073b60 0x7f6e381e9120 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 crypto rx=0x7f6e280097a0 tx=0x7f6e280046c0 comp rx=0 tx=0).stop 2024-04-20T11:36:38.182 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:38.181+0000 7f6e3ca00640 1 -- 172.21.15.152:0/909806935 shutdown_connections 2024-04-20T11:36:38.182 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:38.181+0000 7f6e3ca00640 1 --2- 172.21.15.152:0/909806935 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f6e1806d100 0x7f6e1806f5c0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:38.182 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:38.181+0000 7f6e3ca00640 1 --2- 172.21.15.152:0/909806935 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e3809e030 0x7f6e38144b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:38.182 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:38.181+0000 7f6e3ca00640 1 --2- 172.21.15.152:0/909806935 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e380b4870 0x7f6e381f3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:38.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:38.181+0000 7f6e3ca00640 1 --2- 172.21.15.152:0/909806935 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6e38073b60 0x7f6e381e9120 unknown :-1 s=CLOSED pgs=2047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:38.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:38.181+0000 7f6e3ca00640 1 -- 172.21.15.152:0/909806935 >> 172.21.15.152:0/909806935 conn(0x7f6e380d0260 msgr2=0x7f6e38072b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:38.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:38.181+0000 7f6e3ca00640 1 -- 172.21.15.152:0/909806935 shutdown_connections 2024-04-20T11:36:38.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:38.181+0000 7f6e3ca00640 1 -- 172.21.15.152:0/909806935 wait complete. 2024-04-20T11:36:38.202 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi152.front.sepia.ceph.com... 2024-04-20T11:36:38.202 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:36:38.202 DEBUG:teuthology.orchestra.run.smithi152:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-20T11:36:38.275 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi152.stderr:ceph-fuse[68135]: fuse finished with error 0 and tester_r 0 2024-04-20T11:36:38.282 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi152.front.sepia.ceph.com 2024-04-20T11:36:38.282 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:36:38.282 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-20T11:36:38.424 INFO:tasks.cephfs.mount:Terminating background process 2024-04-20T11:36:38.425 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T11:36:38.425 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:36:38.482 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:36:38.693 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:38.692+0000 7f49b3800640 1 Processor -- start 2024-04-20T11:36:38.693 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:38.693+0000 7f49b3800640 1 -- start start 2024-04-20T11:36:38.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:38.694+0000 7f49b3800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f49ac15ed70 0x7f49ac0b94f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:38.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:38.694+0000 7f49b3800640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f49ac0b7fa0 0x7f49ac0d17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:38.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:38.694+0000 7f49b3800640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49ac0b8450 con 0x7f49ac0b9a30 2024-04-20T11:36:38.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:38.694+0000 7f49b3800640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f49ac0b9060 con 0x7f49ac0b7fa0 2024-04-20T11:36:38.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:38.694+0000 7f49b3800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f49ac0b8730 con 0x7f49ac15ed70 2024-04-20T11:36:38.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:38.694+0000 7f49abe00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f49ac0b9a30 0x7f49ac0b7890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:55544/0 (socket says 172.21.15.152:55544) 2024-04-20T11:36:38.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:38.694+0000 7f49abe00640 1 -- 172.21.15.152:0/1814528860 learned_addr learned my addr 172.21.15.152:0/1814528860 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:36:38.697 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:38.694+0000 7f49b1800640 1 --2- 172.21.15.152:0/1814528860 >> v2:172.21.15.175:3300/0 conn(0x7f49ac0b7fa0 0x7f49ac0d17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:38.697 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:38.694+0000 7f49b0e00640 1 --2- 172.21.15.152:0/1814528860 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f49ac15ed70 0x7f49ac0b94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:38.697 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:38.694+0000 7f49ab400640 1 -- 172.21.15.152:0/1814528860 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1418623505 0 0) 0x7f49ac0b8450 con 0x7f49ac0b9a30 2024-04-20T11:36:38.697 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:38.695+0000 7f49ab400640 1 -- 172.21.15.152:0/1814528860 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4990003520 con 0x7f49ac0b9a30 2024-04-20T11:36:38.697 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:38.695+0000 7f49ab400640 1 -- 172.21.15.152:0/1814528860 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1157458631 0 0) 0x7f4990003520 con 0x7f49ac0b9a30 2024-04-20T11:36:38.697 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:38.695+0000 7f49ab400640 1 -- 172.21.15.152:0/1814528860 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f49ac15ed70 msgr2=0x7f49ac0b94f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:38.697 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:38.695+0000 7f49ab400640 1 --2- 172.21.15.152:0/1814528860 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f49ac15ed70 0x7f49ac0b94f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:38.697 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:38.695+0000 7f49ab400640 1 -- 172.21.15.152:0/1814528860 >> v2:172.21.15.175:3300/0 conn(0x7f49ac0b7fa0 msgr2=0x7f49ac0d17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:38.697 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:38.695+0000 7f49ab400640 1 --2- 172.21.15.152:0/1814528860 >> v2:172.21.15.175:3300/0 conn(0x7f49ac0b7fa0 0x7f49ac0d17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:38.697 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:38.695+0000 7f49ab400640 1 -- 172.21.15.152:0/1814528860 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49ac0b8a10 con 0x7f49ac0b9a30 2024-04-20T11:36:38.697 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:38.695+0000 7f49ab400640 1 -- 172.21.15.152:0/1814528860 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f4994001670 con 0x7f49ac0b9a30 2024-04-20T11:36:38.697 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:38.695+0000 7f49ab400640 1 -- 172.21.15.152:0/1814528860 >> v1:172.21.15.152:6789/0 conn(0x7f49ac0b9a30 legacy=0x7f49ac0b7890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:38.697 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:38.695+0000 7f49ab400640 1 --2- 172.21.15.152:0/1814528860 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4990003a20 0x7f4990003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:38.698 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:38.695+0000 7f49ab400640 1 --2- 172.21.15.152:0/1814528860 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4990004800 0x7f4990004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:38.698 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:38.695+0000 7f49ab400640 1 --2- 172.21.15.152:0/1814528860 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f49ac15ed70 0x7f49900051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:38.698 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:38.695+0000 7f49ab400640 1 -- 172.21.15.152:0/1814528860 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f49ac0b9060 con 0x7f4990003a20 2024-04-20T11:36:38.698 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:38.695+0000 7f49ab400640 1 -- 172.21.15.152:0/1814528860 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f49ac0b8730 con 0x7f4990004800 2024-04-20T11:36:38.698 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:38.695+0000 7f49ab400640 1 -- 172.21.15.152:0/1814528860 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f499000b9c0 con 0x7f49ac15ed70 2024-04-20T11:36:38.698 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:38.695+0000 7f49b0e00640 1 --2- 172.21.15.152:0/1814528860 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4990003a20 0x7f4990003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:38.698 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:38.696+0000 7f49b0e00640 1 -- 172.21.15.152:0/1814528860 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f49ac15ed70 msgr2=0x7f49900051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:38.698 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:38.696+0000 7f49b0e00640 1 --2- 172.21.15.152:0/1814528860 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f49ac15ed70 0x7f49900051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:38.698 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:38.696+0000 7f49b0e00640 1 -- 172.21.15.152:0/1814528860 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4990004800 msgr2=0x7f4990004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:38.698 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:38.696+0000 7f49b0e00640 1 --2- 172.21.15.152:0/1814528860 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4990004800 0x7f4990004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:38.698 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:38.696+0000 7f49b0e00640 1 -- 172.21.15.152:0/1814528860 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4990003520 con 0x7f4990003a20 2024-04-20T11:36:38.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:38.697+0000 7f49b0e00640 1 --2- 172.21.15.152:0/1814528860 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4990003a20 0x7f4990003e40 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7f499c00e170 tx=0x7f4990005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaa6c9f2ad9033d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:38.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:38.697+0000 7f49ab400640 1 -- 172.21.15.152:0/1814528860 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f499c039070 con 0x7f4990003a20 2024-04-20T11:36:38.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:38.697+0000 7f49ab400640 1 -- 172.21.15.152:0/1814528860 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f499c02cc20 con 0x7f4990003a20 2024-04-20T11:36:38.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:38.697+0000 7f49ab400640 1 -- 172.21.15.152:0/1814528860 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f499c009c10 con 0x7f4990003a20 2024-04-20T11:36:38.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:38.698+0000 7f49b3800640 1 -- 172.21.15.152:0/1814528860 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4990003a20 msgr2=0x7f4990003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:38.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:38.698+0000 7f49b3800640 1 --2- 172.21.15.152:0/1814528860 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4990003a20 0x7f4990003e40 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7f499c00e170 tx=0x7f4990005830 comp rx=0 tx=0).stop 2024-04-20T11:36:38.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:38.698+0000 7f49b1800640 1 -- 172.21.15.152:0/1814528860 reap_dead start 2024-04-20T11:36:38.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:38.698+0000 7f49b3800640 1 -- 172.21.15.152:0/1814528860 shutdown_connections 2024-04-20T11:36:38.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:38.698+0000 7f49b3800640 1 -- 172.21.15.152:0/1814528860 >> 172.21.15.152:0/1814528860 conn(0x7f49ac067ae0 msgr2=0x7f49ac067ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:38.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:38.698+0000 7f49b3800640 1 -- 172.21.15.152:0/1814528860 shutdown_connections 2024-04-20T11:36:38.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:38.698+0000 7f49b3800640 1 -- 172.21.15.152:0/1814528860 wait complete. 2024-04-20T11:36:38.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:38.700+0000 7f49b3800640 1 Processor -- start 2024-04-20T11:36:38.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:38.700+0000 7f49b3800640 1 -- start start 2024-04-20T11:36:38.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:38.701+0000 7f49b3800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4990003a20 0x7f49ac14e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:38.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:38.701+0000 7f49b3800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f49ac14ec70 0x7f49ac14c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:38.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:38.701+0000 7f49b3800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49ac14ca80 0x7f49ac14cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:38.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:38.701+0000 7f49b3800640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f49ac0ba640 con 0x7f49ac14ec70 2024-04-20T11:36:38.703 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:38.701+0000 7f49b3800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f49ac0b94f0 con 0x7f49ac14ca80 2024-04-20T11:36:38.703 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:38.701+0000 7f49b3800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f49ac1f5f40 con 0x7f4990003a20 2024-04-20T11:36:38.703 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:38.701+0000 7f49abe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f49ac14ec70 0x7f49ac14c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:38.703 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:38.701+0000 7f49abe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f49ac14ec70 0x7f49ac14c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:57530/0 (socket says 172.21.15.152:57530) 2024-04-20T11:36:38.703 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:38.701+0000 7f49abe00640 1 -- 172.21.15.152:0/1343616897 learned_addr learned my addr 172.21.15.152:0/1343616897 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:38.703 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:38.701+0000 7f49b0e00640 1 --2- 172.21.15.152:0/1343616897 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4990003a20 0x7f49ac14e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:38.703 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:38.701+0000 7f49b1800640 1 --2- 172.21.15.152:0/1343616897 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49ac14ca80 0x7f49ac14cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:38.703 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:38.701+0000 7f49abe00640 1 -- 172.21.15.152:0/1343616897 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4990003a20 msgr2=0x7f49ac14e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:38.703 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:38.702+0000 7f49abe00640 1 --2- 172.21.15.152:0/1343616897 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4990003a20 0x7f49ac14e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:38.703 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:38.702+0000 7f49abe00640 1 -- 172.21.15.152:0/1343616897 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49ac14ca80 msgr2=0x7f49ac14cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:38.703 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:38.702+0000 7f49abe00640 1 --2- 172.21.15.152:0/1343616897 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49ac14ca80 0x7f49ac14cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:38.703 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:38.702+0000 7f49abe00640 1 -- 172.21.15.152:0/1343616897 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49ac1f57f0 con 0x7f49ac14ec70 2024-04-20T11:36:38.703 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:38.702+0000 7f49abe00640 1 --2- 172.21.15.152:0/1343616897 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f49ac14ec70 0x7f49ac14c540 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x7f499400c7d0 tx=0x7f499400cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:38.704 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:38.702+0000 7f49a8c00640 1 -- 172.21.15.152:0/1343616897 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4994018070 con 0x7f49ac14ec70 2024-04-20T11:36:38.704 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:38.702+0000 7f49a8c00640 1 -- 172.21.15.152:0/1343616897 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49940074e0 con 0x7f49ac14ec70 2024-04-20T11:36:38.704 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:38.702+0000 7f49a8c00640 1 -- 172.21.15.152:0/1343616897 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4994004a20 con 0x7f49ac14ec70 2024-04-20T11:36:38.704 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:38.702+0000 7f49b3800640 1 -- 172.21.15.152:0/1343616897 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49ac1f5c10 con 0x7f49ac14ec70 2024-04-20T11:36:38.704 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:38.702+0000 7f49b3800640 1 -- 172.21.15.152:0/1343616897 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49ac1f5510 con 0x7f49ac14ec70 2024-04-20T11:36:38.705 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:38.703+0000 7f49b3800640 1 -- 172.21.15.152:0/1343616897 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49ac0689f0 con 0x7f49ac14ec70 2024-04-20T11:36:38.705 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:38.704+0000 7f49a8c00640 1 -- 172.21.15.152:0/1343616897 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) ==== 91754+0+0 (secure 0 0 0) 0x7f4994004bc0 con 0x7f49ac14ec70 2024-04-20T11:36:38.706 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:38.705+0000 7f49a8c00640 1 --2- 172.21.15.152:0/1343616897 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f498806d100 0x7f498806f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:38.706 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:38.705+0000 7f49a8c00640 1 -- 172.21.15.152:0/1343616897 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5709+0+0 (secure 0 0 0) 0x7f499408c2c0 con 0x7f49ac14ec70 2024-04-20T11:36:38.706 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:38.706+0000 7f49b0e00640 1 --2- 172.21.15.152:0/1343616897 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f498806d100 0x7f498806f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:38.707 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:38.707+0000 7f49b0e00640 1 --2- 172.21.15.152:0/1343616897 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f498806d100 0x7f498806f5c0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f499c036000 tx=0x7f499c02e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:38.710 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:38.710+0000 7f49a8c00640 1 -- 172.21.15.152:0/1343616897 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f4994059760 con 0x7f49ac14ec70 2024-04-20T11:36:38.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:38.909+0000 7f49b3800640 1 -- 172.21.15.152:0/1343616897 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f49ac14d4a0 con 0x7f49ac14ec70 2024-04-20T11:36:38.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:38.912+0000 7f49a8c00640 1 -- 172.21.15.152:0/1343616897 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v142) ==== 74+0+16731 (secure 0 0 0) 0x7f4994059060 con 0x7f49ac14ec70 2024-04-20T11:36:38.913 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:36:38.914 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":142,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:36:10.484514+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:36:06.702770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"141","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-04-20T11:36:07.911633+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:36:38.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:38.915+0000 7f49b3800640 1 -- 172.21.15.152:0/1343616897 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f498806d100 msgr2=0x7f498806f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:38.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:38.915+0000 7f49b3800640 1 --2- 172.21.15.152:0/1343616897 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f498806d100 0x7f498806f5c0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f499c036000 tx=0x7f499c02e000 comp rx=0 tx=0).stop 2024-04-20T11:36:38.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:38.915+0000 7f49b3800640 1 -- 172.21.15.152:0/1343616897 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f49ac14ec70 msgr2=0x7f49ac14c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:38.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:38.915+0000 7f49b3800640 1 --2- 172.21.15.152:0/1343616897 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f49ac14ec70 0x7f49ac14c540 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x7f499400c7d0 tx=0x7f499400cc90 comp rx=0 tx=0).stop 2024-04-20T11:36:38.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:38.915+0000 7f49b3800640 1 -- 172.21.15.152:0/1343616897 shutdown_connections 2024-04-20T11:36:38.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:38.915+0000 7f49b3800640 1 --2- 172.21.15.152:0/1343616897 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f498806d100 0x7f498806f5c0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:38.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:38.915+0000 7f49b3800640 1 --2- 172.21.15.152:0/1343616897 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49ac14ca80 0x7f49ac14cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:38.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:38.915+0000 7f49b3800640 1 --2- 172.21.15.152:0/1343616897 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f49ac14ec70 0x7f49ac14c540 unknown :-1 s=CLOSED pgs=2050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:38.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:38.915+0000 7f49b3800640 1 --2- 172.21.15.152:0/1343616897 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4990003a20 0x7f49ac14e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:38.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:38.915+0000 7f49b3800640 1 -- 172.21.15.152:0/1343616897 >> 172.21.15.152:0/1343616897 conn(0x7f49ac067ae0 msgr2=0x7f49ac0d3cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:38.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:38.915+0000 7f49b3800640 1 -- 172.21.15.152:0/1343616897 shutdown_connections 2024-04-20T11:36:38.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:38.915+0000 7f49b3800640 1 -- 172.21.15.152:0/1343616897 wait complete. 2024-04-20T11:36:38.936 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T11:26:47.246571+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 22, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T11:36:06.702770+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '141', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 23, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T11:36:07.911633+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '142', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T11:36:38.936 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T11:36:39.136 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.136+0000 7f0f73000640 1 Processor -- start 2024-04-20T11:36:39.137 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.136+0000 7f0f73000640 1 -- start start 2024-04-20T11:36:39.139 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.138+0000 7f0f73000640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f0f6c146000 0x7f0f6c0b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:39.140 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.138+0000 7f0f73000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f0f6c0cd8c0 0x7f0f6c0b2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:39.140 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.138+0000 7f0f73000640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f6c1463e0 con 0x7f0f6c0b2170 2024-04-20T11:36:39.140 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.138+0000 7f0f73000640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f0f6c0cdca0 con 0x7f0f6c0cd8c0 2024-04-20T11:36:39.140 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.138+0000 7f0f73000640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f0f6c0b4750 con 0x7f0f6c146000 2024-04-20T11:36:39.140 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.138+0000 7f0f6b400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f0f6c0b2170 0x7f0f6c0cd1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:55552/0 (socket says 172.21.15.152:55552) 2024-04-20T11:36:39.140 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.138+0000 7f0f6b400640 1 -- 172.21.15.152:0/113658246 learned_addr learned my addr 172.21.15.152:0/113658246 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:36:39.140 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.138+0000 7f0f71000640 1 --2- 172.21.15.152:0/113658246 >> v2:172.21.15.152:3300/0 conn(0x7f0f6c0cd8c0 0x7f0f6c0b2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:39.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.138+0000 7f0f6be00640 1 --2- 172.21.15.152:0/113658246 >> v2:172.21.15.175:3300/0 conn(0x7f0f6c146000 0x7f0f6c0b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:39.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.139+0000 7f0f6aa00640 1 -- 172.21.15.152:0/113658246 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3309120368 0 0) 0x7f0f6c1463e0 con 0x7f0f6c0b2170 2024-04-20T11:36:39.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.139+0000 7f0f71000640 1 -- 172.21.15.152:0/113658246 >> v2:172.21.15.175:3300/0 conn(0x7f0f6c146000 msgr2=0x7f0f6c0b4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:39.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.139+0000 7f0f71000640 1 --2- 172.21.15.152:0/113658246 >> v2:172.21.15.175:3300/0 conn(0x7f0f6c146000 0x7f0f6c0b4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:39.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.139+0000 7f0f71000640 1 -- 172.21.15.152:0/113658246 >> v1:172.21.15.152:6789/0 conn(0x7f0f6c0b2170 legacy=0x7f0f6c0cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:39.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.139+0000 7f0f71000640 1 -- 172.21.15.152:0/113658246 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f6c0b3e20 con 0x7f0f6c0cd8c0 2024-04-20T11:36:39.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.140+0000 7f0f71000640 1 --2- 172.21.15.152:0/113658246 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0f6c0cd8c0 0x7f0f6c0b2620 secure :-1 s=READY pgs=2052 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c070190 tx=0x7f0f6402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0294d24ee6cab86 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:39.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.140+0000 7f0f6aa00640 1 -- 172.21.15.152:0/113658246 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f64039070 con 0x7f0f6c0cd8c0 2024-04-20T11:36:39.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.140+0000 7f0f6aa00640 1 -- 172.21.15.152:0/113658246 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f6402cbf0 con 0x7f0f6c0cd8c0 2024-04-20T11:36:39.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.140+0000 7f0f6aa00640 1 -- 172.21.15.152:0/113658246 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f64034440 con 0x7f0f6c0cd8c0 2024-04-20T11:36:39.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.141+0000 7f0f73000640 1 -- 172.21.15.152:0/113658246 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0f6c0cd8c0 msgr2=0x7f0f6c0b2620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:39.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.141+0000 7f0f73000640 1 --2- 172.21.15.152:0/113658246 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0f6c0cd8c0 0x7f0f6c0b2620 secure :-1 s=READY pgs=2052 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c070190 tx=0x7f0f6402be40 comp rx=0 tx=0).stop 2024-04-20T11:36:39.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.141+0000 7f0f73000640 1 -- 172.21.15.152:0/113658246 shutdown_connections 2024-04-20T11:36:39.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.141+0000 7f0f73000640 1 --2- 172.21.15.152:0/113658246 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0f6c0cd8c0 0x7f0f6c0b2620 unknown :-1 s=CLOSED pgs=2052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:39.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.141+0000 7f0f73000640 1 --2- 172.21.15.152:0/113658246 >> v2:172.21.15.175:3300/0 conn(0x7f0f6c146000 0x7f0f6c0b4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:39.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.141+0000 7f0f73000640 1 -- 172.21.15.152:0/113658246 >> 172.21.15.152:0/113658246 conn(0x7f0f6c06b980 msgr2=0x7f0f6c06bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:39.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.141+0000 7f0f73000640 1 -- 172.21.15.152:0/113658246 shutdown_connections 2024-04-20T11:36:39.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.141+0000 7f0f73000640 1 -- 172.21.15.152:0/113658246 wait complete. 2024-04-20T11:36:39.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.143+0000 7f0f73000640 1 Processor -- start 2024-04-20T11:36:39.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.143+0000 7f0f73000640 1 -- start start 2024-04-20T11:36:39.145 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.144+0000 7f0f73000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0f6c0b2170 0x7f0f6c141570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:39.145 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.144+0000 7f0f73000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0f6c0cd8c0 0x7f0f6c0ad8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:39.145 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.144+0000 7f0f73000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0f6c146000 0x7f0f6c1f7df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:39.145 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.144+0000 7f0f73000640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f0f6c15ccf0 con 0x7f0f6c146000 2024-04-20T11:36:39.145 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.144+0000 7f0f73000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0f6c0b43e0 con 0x7f0f6c0b2170 2024-04-20T11:36:39.145 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.144+0000 7f0f73000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0f6c15ce70 con 0x7f0f6c0cd8c0 2024-04-20T11:36:39.145 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.144+0000 7f0f71000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0f6c146000 0x7f0f6c1f7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:39.146 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.144+0000 7f0f71000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0f6c146000 0x7f0f6c1f7df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:57548/0 (socket says 172.21.15.152:57548) 2024-04-20T11:36:39.146 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.144+0000 7f0f71000640 1 -- 172.21.15.152:0/3888933407 learned_addr learned my addr 172.21.15.152:0/3888933407 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:39.146 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.144+0000 7f0f6b400640 1 --2- 172.21.15.152:0/3888933407 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0f6c0cd8c0 0x7f0f6c0ad8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:39.146 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.144+0000 7f0f6be00640 1 --2- 172.21.15.152:0/3888933407 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0f6c0b2170 0x7f0f6c141570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:39.146 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.144+0000 7f0f71000640 1 -- 172.21.15.152:0/3888933407 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0f6c0cd8c0 msgr2=0x7f0f6c0ad8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:39.146 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.144+0000 7f0f71000640 1 --2- 172.21.15.152:0/3888933407 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0f6c0cd8c0 0x7f0f6c0ad8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:39.146 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.144+0000 7f0f71000640 1 -- 172.21.15.152:0/3888933407 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0f6c0b2170 msgr2=0x7f0f6c141570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:39.146 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.144+0000 7f0f71000640 1 --2- 172.21.15.152:0/3888933407 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0f6c0b2170 0x7f0f6c141570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:39.146 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.144+0000 7f0f71000640 1 -- 172.21.15.152:0/3888933407 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f6c1f8330 con 0x7f0f6c146000 2024-04-20T11:36:39.146 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.144+0000 7f0f6b400640 1 --2- 172.21.15.152:0/3888933407 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0f6c0cd8c0 0x7f0f6c0ad8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:36:39.146 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.145+0000 7f0f71000640 1 --2- 172.21.15.152:0/3888933407 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0f6c146000 0x7f0f6c1f7df0 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x7f0f64037040 tx=0x7f0f64004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:39.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.145+0000 7f0f63e00640 1 -- 172.21.15.152:0/3888933407 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f64044020 con 0x7f0f6c146000 2024-04-20T11:36:39.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.145+0000 7f0f63e00640 1 -- 172.21.15.152:0/3888933407 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f6403a040 con 0x7f0f6c146000 2024-04-20T11:36:39.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.145+0000 7f0f63e00640 1 -- 172.21.15.152:0/3888933407 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f64039050 con 0x7f0f6c146000 2024-04-20T11:36:39.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.145+0000 7f0f73000640 1 -- 172.21.15.152:0/3888933407 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f6c1f8600 con 0x7f0f6c146000 2024-04-20T11:36:39.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.145+0000 7f0f73000640 1 -- 172.21.15.152:0/3888933407 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f6c1f8bc0 con 0x7f0f6c146000 2024-04-20T11:36:39.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.146+0000 7f0f73000640 1 -- 172.21.15.152:0/3888933407 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f6c0689f0 con 0x7f0f6c146000 2024-04-20T11:36:39.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.147+0000 7f0f63e00640 1 -- 172.21.15.152:0/3888933407 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) ==== 91754+0+0 (secure 0 0 0) 0x7f0f64003bd0 con 0x7f0f6c146000 2024-04-20T11:36:39.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.148+0000 7f0f63e00640 1 --2- 172.21.15.152:0/3888933407 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f0f4c06d0d0 0x7f0f4c06f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:39.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.148+0000 7f0f63e00640 1 -- 172.21.15.152:0/3888933407 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5709+0+0 (secure 0 0 0) 0x7f0f640af470 con 0x7f0f6c146000 2024-04-20T11:36:39.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.149+0000 7f0f6be00640 1 --2- 172.21.15.152:0/3888933407 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f0f4c06d0d0 0x7f0f4c06f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:39.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.150+0000 7f0f6be00640 1 --2- 172.21.15.152:0/3888933407 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f0f4c06d0d0 0x7f0f4c06f590 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f0f5c006fd0 tx=0x7f0f5c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:39.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.153+0000 7f0f63e00640 1 -- 172.21.15.152:0/3888933407 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f0f6404e060 con 0x7f0f6c146000 2024-04-20T11:36:39.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.356+0000 7f0f73000640 1 -- 172.21.15.152:0/3888933407 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0f6c145470 con 0x7f0f6c146000 2024-04-20T11:36:39.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.357+0000 7f0f63e00640 1 -- 172.21.15.152:0/3888933407 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v142) ==== 93+0+10 (secure 0 0 0) 0x7f0f6407c910 con 0x7f0f6c146000 2024-04-20T11:36:39.357 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 1 2024-04-20T11:36:39.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.359+0000 7f0f73000640 1 -- 172.21.15.152:0/3888933407 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f0f4c06d0d0 msgr2=0x7f0f4c06f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:39.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.359+0000 7f0f73000640 1 --2- 172.21.15.152:0/3888933407 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f0f4c06d0d0 0x7f0f4c06f590 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f0f5c006fd0 tx=0x7f0f5c008040 comp rx=0 tx=0).stop 2024-04-20T11:36:39.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.359+0000 7f0f73000640 1 -- 172.21.15.152:0/3888933407 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0f6c146000 msgr2=0x7f0f6c1f7df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:39.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.359+0000 7f0f73000640 1 --2- 172.21.15.152:0/3888933407 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0f6c146000 0x7f0f6c1f7df0 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x7f0f64037040 tx=0x7f0f64004a40 comp rx=0 tx=0).stop 2024-04-20T11:36:39.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.359+0000 7f0f73000640 1 -- 172.21.15.152:0/3888933407 shutdown_connections 2024-04-20T11:36:39.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.359+0000 7f0f73000640 1 --2- 172.21.15.152:0/3888933407 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f0f4c06d0d0 0x7f0f4c06f590 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:39.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.359+0000 7f0f73000640 1 --2- 172.21.15.152:0/3888933407 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0f6c146000 0x7f0f6c1f7df0 unknown :-1 s=CLOSED pgs=2053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:39.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.359+0000 7f0f73000640 1 --2- 172.21.15.152:0/3888933407 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0f6c0cd8c0 0x7f0f6c0ad8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:39.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.359+0000 7f0f73000640 1 --2- 172.21.15.152:0/3888933407 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0f6c0b2170 0x7f0f6c141570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:39.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.359+0000 7f0f73000640 1 -- 172.21.15.152:0/3888933407 >> 172.21.15.152:0/3888933407 conn(0x7f0f6c06b980 msgr2=0x7f0f6c06bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:39.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.359+0000 7f0f73000640 1 -- 172.21.15.152:0/3888933407 shutdown_connections 2024-04-20T11:36:39.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.360+0000 7f0f73000640 1 -- 172.21.15.152:0/3888933407 wait complete. 2024-04-20T11:36:39.380 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T11:36:39.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.588+0000 7f21bde00640 1 Processor -- start 2024-04-20T11:36:39.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.588+0000 7f21bde00640 1 -- start start 2024-04-20T11:36:39.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.590+0000 7f21bde00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f21b8160220 0x7f21b81606b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:39.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.590+0000 7f21bde00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f21b8141450 0x7f21b81418e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:39.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.590+0000 7f21bde00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21b806fb40 con 0x7f21b8071cd0 2024-04-20T11:36:39.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.590+0000 7f21bde00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f21b813f8e0 con 0x7f21b8160220 2024-04-20T11:36:39.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.590+0000 7f21bde00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f21b8160f90 con 0x7f21b8141450 2024-04-20T11:36:39.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.590+0000 7f21b7e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f21b8160220 0x7f21b81606b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:39.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.590+0000 7f21b7e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f21b8160220 0x7f21b81606b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:57562/0 (socket says 172.21.15.152:57562) 2024-04-20T11:36:39.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.590+0000 7f21b7e00640 1 -- 172.21.15.152:0/708428942 learned_addr learned my addr 172.21.15.152:0/708428942 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:39.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.590+0000 7f21bd400640 1 --2- 172.21.15.152:0/708428942 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f21b8141450 0x7f21b81418e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:39.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.591+0000 7f21b7e00640 1 -- 172.21.15.152:0/708428942 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f21b8141450 msgr2=0x7f21b81418e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:39.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.591+0000 7f21b7e00640 1 --2- 172.21.15.152:0/708428942 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f21b8141450 0x7f21b81418e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:39.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.591+0000 7f21b7e00640 1 -- 172.21.15.152:0/708428942 >> v1:172.21.15.175:6789/0 conn(0x7f21b8071cd0 legacy=0x7f21b80720b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:39.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.591+0000 7f21b7e00640 1 -- 172.21.15.152:0/708428942 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21b806dd90 con 0x7f21b8160220 2024-04-20T11:36:39.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.592+0000 7f21b7e00640 1 --2- 172.21.15.152:0/708428942 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f21b8160220 0x7f21b81606b0 secure :-1 s=READY pgs=2054 cs=0 l=1 rev1=1 crypto rx=0x7f21a40098a0 tx=0x7f21a402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99a57a29975af5a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:39.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.592+0000 7f21b7400640 1 -- 172.21.15.152:0/708428942 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21a4035020 con 0x7f21b8160220 2024-04-20T11:36:39.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.592+0000 7f21b7400640 1 -- 172.21.15.152:0/708428942 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21a4031640 con 0x7f21b8160220 2024-04-20T11:36:39.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.592+0000 7f21b7400640 1 -- 172.21.15.152:0/708428942 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21a402d040 con 0x7f21b8160220 2024-04-20T11:36:39.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.593+0000 7f21bde00640 1 -- 172.21.15.152:0/708428942 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f21b8160220 msgr2=0x7f21b81606b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:39.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.593+0000 7f21bde00640 1 --2- 172.21.15.152:0/708428942 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f21b8160220 0x7f21b81606b0 secure :-1 s=READY pgs=2054 cs=0 l=1 rev1=1 crypto rx=0x7f21a40098a0 tx=0x7f21a402b8f0 comp rx=0 tx=0).stop 2024-04-20T11:36:39.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.593+0000 7f21bde00640 1 -- 172.21.15.152:0/708428942 shutdown_connections 2024-04-20T11:36:39.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.593+0000 7f21bde00640 1 --2- 172.21.15.152:0/708428942 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f21b8141450 0x7f21b81418e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:39.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.593+0000 7f21bde00640 1 --2- 172.21.15.152:0/708428942 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f21b8160220 0x7f21b81606b0 unknown :-1 s=CLOSED pgs=2054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:39.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.593+0000 7f21bde00640 1 -- 172.21.15.152:0/708428942 >> 172.21.15.152:0/708428942 conn(0x7f21b8067ae0 msgr2=0x7f21b806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:39.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.593+0000 7f21bde00640 1 -- 172.21.15.152:0/708428942 shutdown_connections 2024-04-20T11:36:39.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.593+0000 7f21bde00640 1 -- 172.21.15.152:0/708428942 wait complete. 2024-04-20T11:36:39.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.595+0000 7f21bde00640 1 Processor -- start 2024-04-20T11:36:39.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.595+0000 7f21bde00640 1 -- start start 2024-04-20T11:36:39.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.596+0000 7f21bde00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f21b8071cd0 0x7f21b8156630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:39.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.596+0000 7f21bde00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f21b8141450 0x7f21b815a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:39.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.596+0000 7f21bde00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f21b8160220 0x7f21b80aa660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:39.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.596+0000 7f21bde00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f21b806e630 con 0x7f21b8160220 2024-04-20T11:36:39.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.596+0000 7f21bde00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f21b806e350 con 0x7f21b8071cd0 2024-04-20T11:36:39.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.596+0000 7f21bde00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f21b8160bf0 con 0x7f21b8141450 2024-04-20T11:36:39.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.596+0000 7f21bd400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f21b8160220 0x7f21b80aa660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:39.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.596+0000 7f21bd400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f21b8160220 0x7f21b80aa660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:57570/0 (socket says 172.21.15.152:57570) 2024-04-20T11:36:39.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.596+0000 7f21bd400640 1 -- 172.21.15.152:0/941810257 learned_addr learned my addr 172.21.15.152:0/941810257 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:39.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.596+0000 7f21b7e00640 1 --2- 172.21.15.152:0/941810257 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f21b8141450 0x7f21b815a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:39.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.597+0000 7f21bca00640 1 --2- 172.21.15.152:0/941810257 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f21b8071cd0 0x7f21b8156630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:39.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.597+0000 7f21bd400640 1 -- 172.21.15.152:0/941810257 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f21b8141450 msgr2=0x7f21b815a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:39.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.597+0000 7f21bd400640 1 --2- 172.21.15.152:0/941810257 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f21b8141450 0x7f21b815a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:39.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.597+0000 7f21bd400640 1 -- 172.21.15.152:0/941810257 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f21b8071cd0 msgr2=0x7f21b8156630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:39.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.597+0000 7f21bd400640 1 --2- 172.21.15.152:0/941810257 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f21b8071cd0 0x7f21b8156630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:39.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.597+0000 7f21bd400640 1 -- 172.21.15.152:0/941810257 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21b8159800 con 0x7f21b8160220 2024-04-20T11:36:39.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.597+0000 7f21bca00640 1 --2- 172.21.15.152:0/941810257 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f21b8071cd0 0x7f21b8156630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:36:39.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.597+0000 7f21bd400640 1 --2- 172.21.15.152:0/941810257 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f21b8160220 0x7f21b80aa660 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7f21b000efc0 tx=0x7f21b000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:39.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.597+0000 7f21b4c00640 1 -- 172.21.15.152:0/941810257 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b0018020 con 0x7f21b8160220 2024-04-20T11:36:39.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.597+0000 7f21b4c00640 1 -- 172.21.15.152:0/941810257 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21b0014480 con 0x7f21b8160220 2024-04-20T11:36:39.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.597+0000 7f21b4c00640 1 -- 172.21.15.152:0/941810257 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b0014870 con 0x7f21b8160220 2024-04-20T11:36:39.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.597+0000 7f21bde00640 1 -- 172.21.15.152:0/941810257 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21b8159ae0 con 0x7f21b8160220 2024-04-20T11:36:39.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.597+0000 7f21bde00640 1 -- 172.21.15.152:0/941810257 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21b815a0a0 con 0x7f21b8160220 2024-04-20T11:36:39.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.598+0000 7f21bde00640 1 -- 172.21.15.152:0/941810257 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21b8075e60 con 0x7f21b8160220 2024-04-20T11:36:39.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.600+0000 7f21b4c00640 1 -- 172.21.15.152:0/941810257 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) ==== 91754+0+0 (secure 0 0 0) 0x7f21b001b020 con 0x7f21b8160220 2024-04-20T11:36:39.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.600+0000 7f21b4c00640 1 --2- 172.21.15.152:0/941810257 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f219806d0d0 0x7f219806f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:39.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.601+0000 7f21b4c00640 1 -- 172.21.15.152:0/941810257 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5709+0+0 (secure 0 0 0) 0x7f21b008d690 con 0x7f21b8160220 2024-04-20T11:36:39.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.601+0000 7f21bca00640 1 --2- 172.21.15.152:0/941810257 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f219806d0d0 0x7f219806f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:39.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.602+0000 7f21bca00640 1 --2- 172.21.15.152:0/941810257 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f219806d0d0 0x7f219806f590 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f21ac000f60 tx=0x7f21ac009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:39.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.605+0000 7f21b4c00640 1 -- 172.21.15.152:0/941810257 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f21b0022030 con 0x7f21b8160220 2024-04-20T11:36:39.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.802+0000 7f21bde00640 1 -- 172.21.15.152:0/941810257 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f21b813f040 con 0x7f21b8160220 2024-04-20T11:36:39.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.803+0000 7f21b4c00640 1 -- 172.21.15.152:0/941810257 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v142) ==== 104+0+11 (secure 0 0 0) 0x7f21b0010040 con 0x7f21b8160220 2024-04-20T11:36:39.804 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 32 2024-04-20T11:36:39.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.806+0000 7f21bde00640 1 -- 172.21.15.152:0/941810257 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f219806d0d0 msgr2=0x7f219806f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:39.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.806+0000 7f21bde00640 1 --2- 172.21.15.152:0/941810257 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f219806d0d0 0x7f219806f590 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f21ac000f60 tx=0x7f21ac009040 comp rx=0 tx=0).stop 2024-04-20T11:36:39.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.806+0000 7f21bde00640 1 -- 172.21.15.152:0/941810257 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f21b8160220 msgr2=0x7f21b80aa660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:39.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.806+0000 7f21bde00640 1 --2- 172.21.15.152:0/941810257 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f21b8160220 0x7f21b80aa660 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7f21b000efc0 tx=0x7f21b000c490 comp rx=0 tx=0).stop 2024-04-20T11:36:39.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.806+0000 7f21bde00640 1 -- 172.21.15.152:0/941810257 shutdown_connections 2024-04-20T11:36:39.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.806+0000 7f21bde00640 1 --2- 172.21.15.152:0/941810257 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f219806d0d0 0x7f219806f590 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:39.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.806+0000 7f21bde00640 1 --2- 172.21.15.152:0/941810257 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f21b8160220 0x7f21b80aa660 unknown :-1 s=CLOSED pgs=2055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:39.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.806+0000 7f21bde00640 1 --2- 172.21.15.152:0/941810257 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f21b8141450 0x7f21b815a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:39.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.806+0000 7f21bde00640 1 --2- 172.21.15.152:0/941810257 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f21b8071cd0 0x7f21b8156630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:39.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.806+0000 7f21bde00640 1 -- 172.21.15.152:0/941810257 >> 172.21.15.152:0/941810257 conn(0x7f21b8067ae0 msgr2=0x7f21b815e030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:39.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.806+0000 7f21bde00640 1 -- 172.21.15.152:0/941810257 shutdown_connections 2024-04-20T11:36:39.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:39.807+0000 7f21bde00640 1 -- 172.21.15.152:0/941810257 wait complete. 2024-04-20T11:36:39.827 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T11:36:40.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.031+0000 7fea4d200640 1 Processor -- start 2024-04-20T11:36:40.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.032+0000 7fea4d200640 1 -- start start 2024-04-20T11:36:40.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.033+0000 7fea4d200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fea48146000 0x7fea480b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:40.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.033+0000 7fea4d200640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea481463e0 con 0x7fea480cd8c0 2024-04-20T11:36:40.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.033+0000 7fea4d200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea480b3e20 con 0x7fea480b2170 2024-04-20T11:36:40.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.033+0000 7fea4d200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fea480cdca0 con 0x7fea48146000 2024-04-20T11:36:40.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.034+0000 7fea46800640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fea480cd8c0 0x7fea4815ccf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:55566/0 (socket says 172.21.15.152:55566) 2024-04-20T11:36:40.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.034+0000 7fea46800640 1 -- 172.21.15.152:0/1799032355 learned_addr learned my addr 172.21.15.152:0/1799032355 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:36:40.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.034+0000 7fea45e00640 1 --2- 172.21.15.152:0/1799032355 >> v2:172.21.15.152:3300/0 conn(0x7fea48146000 0x7fea480b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:40.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.034+0000 7fea45400640 1 -- 172.21.15.152:0/1799032355 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1710566970 0 0) 0x7fea481463e0 con 0x7fea480cd8c0 2024-04-20T11:36:40.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.034+0000 7fea45e00640 1 -- 172.21.15.152:0/1799032355 >> v1:172.21.15.175:6789/0 conn(0x7fea480b2170 legacy=0x7fea480cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:40.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.034+0000 7fea45e00640 1 -- 172.21.15.152:0/1799032355 >> v1:172.21.15.152:6789/0 conn(0x7fea480cd8c0 legacy=0x7fea4815ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:40.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.034+0000 7fea45e00640 1 -- 172.21.15.152:0/1799032355 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea480b4100 con 0x7fea48146000 2024-04-20T11:36:40.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.035+0000 7fea45e00640 1 --2- 172.21.15.152:0/1799032355 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fea48146000 0x7fea480b4be0 secure :-1 s=READY pgs=2057 cs=0 l=1 rev1=1 crypto rx=0x7fea380097a0 tx=0x7fea3802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39d60ad9bf6d6fa4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:40.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.036+0000 7fea45400640 1 -- 172.21.15.152:0/1799032355 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea38035020 con 0x7fea48146000 2024-04-20T11:36:40.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.036+0000 7fea45400640 1 -- 172.21.15.152:0/1799032355 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea38031830 con 0x7fea48146000 2024-04-20T11:36:40.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.036+0000 7fea45400640 1 -- 172.21.15.152:0/1799032355 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea38031c00 con 0x7fea48146000 2024-04-20T11:36:40.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.036+0000 7fea4d200640 1 -- 172.21.15.152:0/1799032355 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fea48146000 msgr2=0x7fea480b4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:40.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.036+0000 7fea4d200640 1 --2- 172.21.15.152:0/1799032355 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fea48146000 0x7fea480b4be0 secure :-1 s=READY pgs=2057 cs=0 l=1 rev1=1 crypto rx=0x7fea380097a0 tx=0x7fea3802b8d0 comp rx=0 tx=0).stop 2024-04-20T11:36:40.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.036+0000 7fea4d200640 1 -- 172.21.15.152:0/1799032355 shutdown_connections 2024-04-20T11:36:40.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.037+0000 7fea4d200640 1 --2- 172.21.15.152:0/1799032355 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fea48146000 0x7fea480b4be0 unknown :-1 s=CLOSED pgs=2057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:40.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.037+0000 7fea4d200640 1 -- 172.21.15.152:0/1799032355 >> 172.21.15.152:0/1799032355 conn(0x7fea4806b980 msgr2=0x7fea4806bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:40.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.037+0000 7fea4d200640 1 -- 172.21.15.152:0/1799032355 shutdown_connections 2024-04-20T11:36:40.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.037+0000 7fea4d200640 1 -- 172.21.15.152:0/1799032355 wait complete. 2024-04-20T11:36:40.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.038+0000 7fea4d200640 1 Processor -- start 2024-04-20T11:36:40.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.039+0000 7fea4d200640 1 -- start start 2024-04-20T11:36:40.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.039+0000 7fea4d200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fea480b2170 0x7fea48140e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:40.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.039+0000 7fea4d200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fea480cd8c0 0x7fea480aeb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:40.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.039+0000 7fea4d200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fea48146000 0x7fea481e7a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:40.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.040+0000 7fea4d200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fea480b2620 con 0x7fea480b2170 2024-04-20T11:36:40.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.040+0000 7fea4d200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fea480b4750 con 0x7fea48146000 2024-04-20T11:36:40.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.040+0000 7fea4d200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fea480b27a0 con 0x7fea480cd8c0 2024-04-20T11:36:40.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.040+0000 7fea45e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fea480b2170 0x7fea48140e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:40.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.040+0000 7fea45e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fea480b2170 0x7fea48140e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:57588/0 (socket says 172.21.15.152:57588) 2024-04-20T11:36:40.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.040+0000 7fea45e00640 1 -- 172.21.15.152:0/3137554766 learned_addr learned my addr 172.21.15.152:0/3137554766 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:40.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.040+0000 7fea45e00640 1 -- 172.21.15.152:0/3137554766 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fea480cd8c0 msgr2=0x7fea480aeb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:40.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.040+0000 7fea45e00640 1 --2- 172.21.15.152:0/3137554766 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fea480cd8c0 0x7fea480aeb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:40.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.040+0000 7fea45e00640 1 -- 172.21.15.152:0/3137554766 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fea48146000 msgr2=0x7fea481e7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:40.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.040+0000 7fea45e00640 1 --2- 172.21.15.152:0/3137554766 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fea48146000 0x7fea481e7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:40.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.040+0000 7fea45e00640 1 -- 172.21.15.152:0/3137554766 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea481e4f90 con 0x7fea480b2170 2024-04-20T11:36:40.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.041+0000 7fea45e00640 1 --2- 172.21.15.152:0/3137554766 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fea480b2170 0x7fea48140e00 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7fea380097a0 tx=0x7fea380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:40.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.041+0000 7fea3ea00640 1 -- 172.21.15.152:0/3137554766 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea38041070 con 0x7fea480b2170 2024-04-20T11:36:40.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.041+0000 7fea3ea00640 1 -- 172.21.15.152:0/3137554766 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea38004230 con 0x7fea480b2170 2024-04-20T11:36:40.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.041+0000 7fea3ea00640 1 -- 172.21.15.152:0/3137554766 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea38035040 con 0x7fea480b2170 2024-04-20T11:36:40.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.041+0000 7fea4d200640 1 -- 172.21.15.152:0/3137554766 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea481e5270 con 0x7fea480b2170 2024-04-20T11:36:40.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.041+0000 7fea4d200640 1 -- 172.21.15.152:0/3137554766 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea48141340 con 0x7fea480b2170 2024-04-20T11:36:40.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.042+0000 7fea4d200640 1 -- 172.21.15.152:0/3137554766 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea480689f0 con 0x7fea480b2170 2024-04-20T11:36:40.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.043+0000 7fea3ea00640 1 -- 172.21.15.152:0/3137554766 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) ==== 91754+0+0 (secure 0 0 0) 0x7fea38038020 con 0x7fea480b2170 2024-04-20T11:36:40.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.044+0000 7fea3ea00640 1 --2- 172.21.15.152:0/3137554766 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fea2406d0d0 0x7fea2406f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:40.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.044+0000 7fea3ea00640 1 -- 172.21.15.152:0/3137554766 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5709+0+0 (secure 0 0 0) 0x7fea380ac090 con 0x7fea480b2170 2024-04-20T11:36:40.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.045+0000 7fea3d400640 1 --2- 172.21.15.152:0/3137554766 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fea2406d0d0 0x7fea2406f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:40.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.046+0000 7fea3d400640 1 --2- 172.21.15.152:0/3137554766 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fea2406d0d0 0x7fea2406f590 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fea30000f60 tx=0x7fea300079d0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:40.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.049+0000 7fea3ea00640 1 -- 172.21.15.152:0/3137554766 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fea3807bdb0 con 0x7fea480b2170 2024-04-20T11:36:40.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.247+0000 7fea4d200640 1 -- 172.21.15.152:0/3137554766 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fea4815c970 con 0x7fea480b2170 2024-04-20T11:36:40.249 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.248+0000 7fea3ea00640 1 -- 172.21.15.152:0/3137554766 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v142) ==== 100+0+11 (secure 0 0 0) 0x7fea3807b6b0 con 0x7fea480b2170 2024-04-20T11:36:40.249 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 64 2024-04-20T11:36:40.252 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.251+0000 7fea4d200640 1 -- 172.21.15.152:0/3137554766 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fea2406d0d0 msgr2=0x7fea2406f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:40.252 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.251+0000 7fea4d200640 1 --2- 172.21.15.152:0/3137554766 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fea2406d0d0 0x7fea2406f590 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fea30000f60 tx=0x7fea300079d0 comp rx=0 tx=0).stop 2024-04-20T11:36:40.252 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.251+0000 7fea4d200640 1 -- 172.21.15.152:0/3137554766 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fea480b2170 msgr2=0x7fea48140e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:40.252 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.251+0000 7fea4d200640 1 --2- 172.21.15.152:0/3137554766 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fea480b2170 0x7fea48140e00 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7fea380097a0 tx=0x7fea380046c0 comp rx=0 tx=0).stop 2024-04-20T11:36:40.252 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.251+0000 7fea4d200640 1 -- 172.21.15.152:0/3137554766 shutdown_connections 2024-04-20T11:36:40.252 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.251+0000 7fea4d200640 1 --2- 172.21.15.152:0/3137554766 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fea2406d0d0 0x7fea2406f590 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:40.252 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.251+0000 7fea4d200640 1 --2- 172.21.15.152:0/3137554766 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fea48146000 0x7fea481e7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:40.252 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.251+0000 7fea4d200640 1 --2- 172.21.15.152:0/3137554766 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fea480cd8c0 0x7fea480aeb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:40.252 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.251+0000 7fea4d200640 1 --2- 172.21.15.152:0/3137554766 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fea480b2170 0x7fea48140e00 unknown :-1 s=CLOSED pgs=2058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:40.253 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.251+0000 7fea4d200640 1 -- 172.21.15.152:0/3137554766 >> 172.21.15.152:0/3137554766 conn(0x7fea4806b980 msgr2=0x7fea480b1870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:40.253 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.251+0000 7fea4d200640 1 -- 172.21.15.152:0/3137554766 shutdown_connections 2024-04-20T11:36:40.253 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.251+0000 7fea4d200640 1 -- 172.21.15.152:0/3137554766 wait complete. 2024-04-20T11:36:40.271 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-20T11:36:40.473 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.472+0000 7fa208c00640 1 Processor -- start 2024-04-20T11:36:40.473 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.472+0000 7fa208c00640 1 -- start start 2024-04-20T11:36:40.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.474+0000 7fa208c00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fa20413f410 0x7fa20415d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:40.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.474+0000 7fa208c00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fa204141ce0 0x7fa2040d0a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:40.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.474+0000 7fa208c00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2041426a0 con 0x7fa204142160 2024-04-20T11:36:40.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.474+0000 7fa208c00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fa20415d060 con 0x7fa20413f410 2024-04-20T11:36:40.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.474+0000 7fa208c00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fa204142980 con 0x7fa204141ce0 2024-04-20T11:36:40.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.474+0000 7fa203400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fa20413f410 0x7fa20415d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:40.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.474+0000 7fa203400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fa20413f410 0x7fa20415d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:57600/0 (socket says 172.21.15.152:57600) 2024-04-20T11:36:40.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.474+0000 7fa203400640 1 -- 172.21.15.152:0/1178407444 learned_addr learned my addr 172.21.15.152:0/1178407444 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:40.476 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.474+0000 7fa202a00640 1 --2- 172.21.15.152:0/1178407444 >> v2:172.21.15.175:3300/0 conn(0x7fa204141ce0 0x7fa2040d0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:40.476 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.475+0000 7fa203400640 1 -- 172.21.15.152:0/1178407444 >> v2:172.21.15.175:3300/0 conn(0x7fa204141ce0 msgr2=0x7fa2040d0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:40.476 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.475+0000 7fa203400640 1 --2- 172.21.15.152:0/1178407444 >> v2:172.21.15.175:3300/0 conn(0x7fa204141ce0 0x7fa2040d0a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:40.476 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.475+0000 7fa203400640 1 -- 172.21.15.152:0/1178407444 >> v1:172.21.15.175:6789/0 conn(0x7fa204142160 legacy=0x7fa2041600d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:40.476 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.475+0000 7fa203400640 1 -- 172.21.15.152:0/1178407444 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa20415ccf0 con 0x7fa20413f410 2024-04-20T11:36:40.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.476+0000 7fa203400640 1 --2- 172.21.15.152:0/1178407444 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa20413f410 0x7fa20415d4f0 secure :-1 s=READY pgs=2059 cs=0 l=1 rev1=1 crypto rx=0x7fa2041611f0 tx=0x7fa1f402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f408f59b20491c42 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:40.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.476+0000 7fa202000640 1 -- 172.21.15.152:0/1178407444 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f4035020 con 0x7fa20413f410 2024-04-20T11:36:40.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.476+0000 7fa202000640 1 -- 172.21.15.152:0/1178407444 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1f4031870 con 0x7fa20413f410 2024-04-20T11:36:40.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.476+0000 7fa202000640 1 -- 172.21.15.152:0/1178407444 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f4031c40 con 0x7fa20413f410 2024-04-20T11:36:40.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.477+0000 7fa208c00640 1 -- 172.21.15.152:0/1178407444 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa20413f410 msgr2=0x7fa20415d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:40.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.477+0000 7fa208c00640 1 --2- 172.21.15.152:0/1178407444 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa20413f410 0x7fa20415d4f0 secure :-1 s=READY pgs=2059 cs=0 l=1 rev1=1 crypto rx=0x7fa2041611f0 tx=0x7fa1f402b910 comp rx=0 tx=0).stop 2024-04-20T11:36:40.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.477+0000 7fa208c00640 1 -- 172.21.15.152:0/1178407444 shutdown_connections 2024-04-20T11:36:40.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.477+0000 7fa208c00640 1 --2- 172.21.15.152:0/1178407444 >> v2:172.21.15.175:3300/0 conn(0x7fa204141ce0 0x7fa2040d0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:40.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.477+0000 7fa208c00640 1 --2- 172.21.15.152:0/1178407444 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa20413f410 0x7fa20415d4f0 unknown :-1 s=CLOSED pgs=2059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:40.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.477+0000 7fa208c00640 1 -- 172.21.15.152:0/1178407444 >> 172.21.15.152:0/1178407444 conn(0x7fa20406bda0 msgr2=0x7fa20406c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:40.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.477+0000 7fa208c00640 1 -- 172.21.15.152:0/1178407444 shutdown_connections 2024-04-20T11:36:40.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.477+0000 7fa208c00640 1 -- 172.21.15.152:0/1178407444 wait complete. 2024-04-20T11:36:40.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.479+0000 7fa208c00640 1 Processor -- start 2024-04-20T11:36:40.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.479+0000 7fa208c00640 1 -- start start 2024-04-20T11:36:40.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.480+0000 7fa208c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa20413f410 0x7fa20413a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:40.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.480+0000 7fa208c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa204141ce0 0x7fa204144410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:40.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.480+0000 7fa208c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa204142160 0x7fa2041585e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:40.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.480+0000 7fa208c00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fa20413f970 con 0x7fa204141ce0 2024-04-20T11:36:40.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.480+0000 7fa208c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa20413f7f0 con 0x7fa20413f410 2024-04-20T11:36:40.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.480+0000 7fa208c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa2040afbe0 con 0x7fa204142160 2024-04-20T11:36:40.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.480+0000 7fa202a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa204141ce0 0x7fa204144410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:40.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.480+0000 7fa202a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa204141ce0 0x7fa204144410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:57606/0 (socket says 172.21.15.152:57606) 2024-04-20T11:36:40.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.480+0000 7fa202a00640 1 -- 172.21.15.152:0/1337064053 learned_addr learned my addr 172.21.15.152:0/1337064053 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:40.482 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.481+0000 7fa202a00640 1 -- 172.21.15.152:0/1337064053 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa204142160 msgr2=0x7fa2041585e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:40.482 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.481+0000 7fa202a00640 1 --2- 172.21.15.152:0/1337064053 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa204142160 0x7fa2041585e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:40.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.481+0000 7fa202a00640 1 -- 172.21.15.152:0/1337064053 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa20413f410 msgr2=0x7fa20413a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:40.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.481+0000 7fa202a00640 1 --2- 172.21.15.152:0/1337064053 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa20413f410 0x7fa20413a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:40.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.481+0000 7fa202a00640 1 -- 172.21.15.152:0/1337064053 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa20413ae70 con 0x7fa204141ce0 2024-04-20T11:36:40.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.481+0000 7fa202a00640 1 --2- 172.21.15.152:0/1337064053 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa204141ce0 0x7fa204144410 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 crypto rx=0x7fa1f000d800 tx=0x7fa1f000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:40.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.482+0000 7fa1fb400640 1 -- 172.21.15.152:0/1337064053 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f000b6c0 con 0x7fa204141ce0 2024-04-20T11:36:40.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.482+0000 7fa1fb400640 1 -- 172.21.15.152:0/1337064053 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1f0019070 con 0x7fa204141ce0 2024-04-20T11:36:40.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.482+0000 7fa1fb400640 1 -- 172.21.15.152:0/1337064053 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f000ba80 con 0x7fa204141ce0 2024-04-20T11:36:40.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.482+0000 7fa208c00640 1 -- 172.21.15.152:0/1337064053 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa204157540 con 0x7fa204141ce0 2024-04-20T11:36:40.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.482+0000 7fa208c00640 1 -- 172.21.15.152:0/1337064053 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa204157b00 con 0x7fa204141ce0 2024-04-20T11:36:40.484 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.483+0000 7fa208c00640 1 -- 172.21.15.152:0/1337064053 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa204075e60 con 0x7fa204141ce0 2024-04-20T11:36:40.484 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.484+0000 7fa1fb400640 1 -- 172.21.15.152:0/1337064053 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) ==== 91754+0+0 (secure 0 0 0) 0x7fa1f000bc20 con 0x7fa204141ce0 2024-04-20T11:36:40.485 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.485+0000 7fa1fb400640 1 --2- 172.21.15.152:0/1337064053 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fa1e406d100 0x7fa1e406f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:40.485 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.485+0000 7fa1fb400640 1 -- 172.21.15.152:0/1337064053 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5709+0+0 (secure 0 0 0) 0x7fa1f008aab0 con 0x7fa204141ce0 2024-04-20T11:36:40.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.486+0000 7fa203400640 1 --2- 172.21.15.152:0/1337064053 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fa1e406d100 0x7fa1e406f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:40.487 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.487+0000 7fa203400640 1 --2- 172.21.15.152:0/1337064053 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fa1e406d100 0x7fa1e406f5c0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fa1f4036000 tx=0x7fa1f4030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:40.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.489+0000 7fa1fb400640 1 -- 172.21.15.152:0/1337064053 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fa1f001c360 con 0x7fa204141ce0 2024-04-20T11:36:40.707 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.707+0000 7fa208c00640 1 -- 172.21.15.152:0/1337064053 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fa20406f690 con 0x7fa204141ce0 2024-04-20T11:36:40.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.708+0000 7fa1fb400640 1 -- 172.21.15.152:0/1337064053 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v142) ==== 80+0+0 (secure 0 0 0) 0x7fa1f005a740 con 0x7fa204141ce0 2024-04-20T11:36:40.709 INFO:teuthology.orchestra.run.smithi152.stderr:listed 0 entries 2024-04-20T11:36:40.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.711+0000 7fa208c00640 1 -- 172.21.15.152:0/1337064053 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fa1e406d100 msgr2=0x7fa1e406f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:40.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.711+0000 7fa208c00640 1 --2- 172.21.15.152:0/1337064053 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fa1e406d100 0x7fa1e406f5c0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fa1f4036000 tx=0x7fa1f4030000 comp rx=0 tx=0).stop 2024-04-20T11:36:40.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.711+0000 7fa208c00640 1 -- 172.21.15.152:0/1337064053 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa204141ce0 msgr2=0x7fa204144410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:40.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.711+0000 7fa208c00640 1 --2- 172.21.15.152:0/1337064053 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa204141ce0 0x7fa204144410 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 crypto rx=0x7fa1f000d800 tx=0x7fa1f000dcc0 comp rx=0 tx=0).stop 2024-04-20T11:36:40.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.711+0000 7fa208c00640 1 -- 172.21.15.152:0/1337064053 shutdown_connections 2024-04-20T11:36:40.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.711+0000 7fa208c00640 1 --2- 172.21.15.152:0/1337064053 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fa1e406d100 0x7fa1e406f5c0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:40.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.711+0000 7fa208c00640 1 --2- 172.21.15.152:0/1337064053 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa204142160 0x7fa2041585e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:40.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.711+0000 7fa208c00640 1 --2- 172.21.15.152:0/1337064053 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa204141ce0 0x7fa204144410 unknown :-1 s=CLOSED pgs=2060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:40.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.711+0000 7fa208c00640 1 --2- 172.21.15.152:0/1337064053 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa20413f410 0x7fa20413a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:40.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.711+0000 7fa208c00640 1 -- 172.21.15.152:0/1337064053 >> 172.21.15.152:0/1337064053 conn(0x7fa20406bda0 msgr2=0x7fa2041405f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:40.713 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.711+0000 7fa208c00640 1 -- 172.21.15.152:0/1337064053 shutdown_connections 2024-04-20T11:36:40.713 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:40.711+0000 7fa208c00640 1 -- 172.21.15.152:0/1337064053 wait complete. 2024-04-20T11:36:40.731 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi175.front.sepia.ceph.com... 2024-04-20T11:36:40.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:36:40.732 DEBUG:teuthology.orchestra.run.smithi175:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-04-20T11:36:40.818 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi175.stderr:ceph-fuse[35847]: fuse finished with error 0 and tester_r 0 2024-04-20T11:36:40.824 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi175.front.sepia.ceph.com 2024-04-20T11:36:40.824 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:36:40.824 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-20T11:36:41.004 INFO:tasks.cephfs.mount:Terminating background process 2024-04-20T11:36:41.005 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-20T11:36:41.005 DEBUG:teuthology.orchestra.run.smithi152:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:36:41.063 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-20T11:36:41.063 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi152.front.sepia.ceph.com 2024-04-20T11:36:41.063 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:36:41.063 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-20T11:36:41.119 INFO:teuthology.orchestra.run.smithi152.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-20T11:36:41.120 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T11:36:41.120 INFO:tasks.cephfs.mount:Terminating background process 2024-04-20T11:36:41.120 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T11:36:41.120 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:36:41.177 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-20T11:36:41.177 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi175.front.sepia.ceph.com 2024-04-20T11:36:41.177 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:36:41.177 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-20T11:36:41.232 INFO:teuthology.orchestra.run.smithi175.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-20T11:36:41.233 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T11:36:41.233 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:36:41.445 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.444+0000 7f7ad6c00640 1 Processor -- start 2024-04-20T11:36:41.445 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.444+0000 7f7ad6c00640 1 -- start start 2024-04-20T11:36:41.446 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.445+0000 7f7ad6c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7ad00cd0d0 0x7f7ad00cd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:41.446 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.445+0000 7f7ad6c00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f7ad0071fe0 0x7f7ad0072470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:41.446 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.445+0000 7f7ad6c00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ad0146340 con 0x7f7ad0145f60 2024-04-20T11:36:41.446 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.445+0000 7f7ad6c00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f7ad006e9f0 con 0x7f7ad0071fe0 2024-04-20T11:36:41.446 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.445+0000 7f7ad6c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7ad0072d50 con 0x7f7ad00cd0d0 2024-04-20T11:36:41.447 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.446+0000 7f7ad4c00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f7ad0071fe0 0x7f7ad0072470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:41.447 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.446+0000 7f7acf400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7ad00cd0d0 0x7f7ad00cd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:41.447 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.446+0000 7f7ad4c00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f7ad0071fe0 0x7f7ad0072470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.152:34312/0 (socket says 172.21.15.152:34312) 2024-04-20T11:36:41.447 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.446+0000 7f7ad4c00640 1 -- 172.21.15.152:0/486980135 learned_addr learned my addr 172.21.15.152:0/486980135 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:41.448 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.447+0000 7f7acf400640 1 -- 172.21.15.152:0/486980135 >> v2:172.21.15.175:3300/0 conn(0x7f7ad0071fe0 msgr2=0x7f7ad0072470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:41.448 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.447+0000 7f7acf400640 1 --2- 172.21.15.152:0/486980135 >> v2:172.21.15.175:3300/0 conn(0x7f7ad0071fe0 0x7f7ad0072470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:41.448 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.447+0000 7f7acf400640 1 -- 172.21.15.152:0/486980135 >> v1:172.21.15.175:6789/0 conn(0x7f7ad0145f60 legacy=0x7f7ad006ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:41.448 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.447+0000 7f7acf400640 1 -- 172.21.15.152:0/486980135 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ad006e0c0 con 0x7f7ad00cd0d0 2024-04-20T11:36:41.449 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.448+0000 7f7acf400640 1 --2- 172.21.15.152:0/486980135 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7ad00cd0d0 0x7f7ad00cd550 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7ad006eb70 tx=0x7f7abc02be90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7be9b7a06ab1885 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:41.450 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.449+0000 7f7acea00640 1 -- 172.21.15.152:0/486980135 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7abc039070 con 0x7f7ad00cd0d0 2024-04-20T11:36:41.450 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.449+0000 7f7acea00640 1 -- 172.21.15.152:0/486980135 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7abc02cc40 con 0x7f7ad00cd0d0 2024-04-20T11:36:41.450 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.449+0000 7f7acea00640 1 -- 172.21.15.152:0/486980135 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7abc034440 con 0x7f7ad00cd0d0 2024-04-20T11:36:41.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.450+0000 7f7ad6c00640 1 -- 172.21.15.152:0/486980135 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7ad00cd0d0 msgr2=0x7f7ad00cd550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:41.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.450+0000 7f7ad6c00640 1 --2- 172.21.15.152:0/486980135 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7ad00cd0d0 0x7f7ad00cd550 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7ad006eb70 tx=0x7f7abc02be90 comp rx=0 tx=0).stop 2024-04-20T11:36:41.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.450+0000 7f7ad6c00640 1 -- 172.21.15.152:0/486980135 shutdown_connections 2024-04-20T11:36:41.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.450+0000 7f7ad6c00640 1 --2- 172.21.15.152:0/486980135 >> v2:172.21.15.175:3300/0 conn(0x7f7ad0071fe0 0x7f7ad0072470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:41.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.450+0000 7f7ad6c00640 1 --2- 172.21.15.152:0/486980135 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7ad00cd0d0 0x7f7ad00cd550 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:41.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.450+0000 7f7ad6c00640 1 -- 172.21.15.152:0/486980135 >> 172.21.15.152:0/486980135 conn(0x7f7ad0067cf0 msgr2=0x7f7ad006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:41.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.450+0000 7f7ad6c00640 1 -- 172.21.15.152:0/486980135 shutdown_connections 2024-04-20T11:36:41.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.450+0000 7f7ad6c00640 1 -- 172.21.15.152:0/486980135 wait complete. 2024-04-20T11:36:41.452 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.452+0000 7f7ad6c00640 1 Processor -- start 2024-04-20T11:36:41.452 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.452+0000 7f7ad6c00640 1 -- start start 2024-04-20T11:36:41.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.453+0000 7f7ad6c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ad0071fe0 0x7f7ad0132e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:41.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.453+0000 7f7ad6c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7ad00cd0d0 0x7f7ad0141500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:41.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.453+0000 7f7ad6c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7ad0145f60 0x7f7ad01e3c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:41.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.453+0000 7f7ad6c00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f7ad00729b0 con 0x7f7ad00cd0d0 2024-04-20T11:36:41.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.453+0000 7f7ad6c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7ad006e680 con 0x7f7ad0071fe0 2024-04-20T11:36:41.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.453+0000 7f7ad6c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7ad0072b30 con 0x7f7ad0145f60 2024-04-20T11:36:41.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.453+0000 7f7acf400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7ad00cd0d0 0x7f7ad0141500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:41.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.453+0000 7f7acf400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7ad00cd0d0 0x7f7ad0141500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:57622/0 (socket says 172.21.15.152:57622) 2024-04-20T11:36:41.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.453+0000 7f7acf400640 1 -- 172.21.15.152:0/67064125 learned_addr learned my addr 172.21.15.152:0/67064125 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:41.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.454+0000 7f7ad4c00640 1 --2- 172.21.15.152:0/67064125 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7ad0145f60 0x7f7ad01e3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:41.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.454+0000 7f7acf400640 1 -- 172.21.15.152:0/67064125 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7ad0145f60 msgr2=0x7f7ad01e3c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:41.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.454+0000 7f7acf400640 1 --2- 172.21.15.152:0/67064125 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7ad0145f60 0x7f7ad01e3c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:41.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.454+0000 7f7acfe00640 1 --2- 172.21.15.152:0/67064125 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ad0071fe0 0x7f7ad0132e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:41.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.454+0000 7f7acf400640 1 -- 172.21.15.152:0/67064125 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ad0071fe0 msgr2=0x7f7ad0132e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:41.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.454+0000 7f7acf400640 1 --2- 172.21.15.152:0/67064125 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ad0071fe0 0x7f7ad0132e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:41.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.454+0000 7f7acf400640 1 -- 172.21.15.152:0/67064125 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ad0141ee0 con 0x7f7ad00cd0d0 2024-04-20T11:36:41.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.454+0000 7f7acf400640 1 --2- 172.21.15.152:0/67064125 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7ad00cd0d0 0x7f7ad0141500 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 crypto rx=0x7f7abc037040 tx=0x7f7abc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:41.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.454+0000 7f7ac7e00640 1 -- 172.21.15.152:0/67064125 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7abc039050 con 0x7f7ad00cd0d0 2024-04-20T11:36:41.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.454+0000 7f7ac7e00640 1 -- 172.21.15.152:0/67064125 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7abc045070 con 0x7f7ad00cd0d0 2024-04-20T11:36:41.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.454+0000 7f7ac7e00640 1 -- 172.21.15.152:0/67064125 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7abc009450 con 0x7f7ad00cd0d0 2024-04-20T11:36:41.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.455+0000 7f7ad6c00640 1 -- 172.21.15.152:0/67064125 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ad01421c0 con 0x7f7ad00cd0d0 2024-04-20T11:36:41.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.455+0000 7f7ad6c00640 1 -- 172.21.15.152:0/67064125 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ad0141a40 con 0x7f7ad00cd0d0 2024-04-20T11:36:41.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.456+0000 7f7ad6c00640 1 -- 172.21.15.152:0/67064125 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ad00689f0 con 0x7f7ad00cd0d0 2024-04-20T11:36:41.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.457+0000 7f7ac7e00640 1 -- 172.21.15.152:0/67064125 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) ==== 91754+0+0 (secure 0 0 0) 0x7f7abc040690 con 0x7f7ad00cd0d0 2024-04-20T11:36:41.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.458+0000 7f7ac7e00640 1 --2- 172.21.15.152:0/67064125 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f7ab006d0d0 0x7f7ab006f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:41.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.458+0000 7f7ac7e00640 1 -- 172.21.15.152:0/67064125 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5709+0+0 (secure 0 0 0) 0x7f7abc0ad060 con 0x7f7ad00cd0d0 2024-04-20T11:36:41.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.459+0000 7f7acfe00640 1 --2- 172.21.15.152:0/67064125 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f7ab006d0d0 0x7f7ab006f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:41.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.460+0000 7f7acfe00640 1 --2- 172.21.15.152:0/67064125 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f7ab006d0d0 0x7f7ab006f590 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f7ac0001500 tx=0x7f7ac0009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:41.462 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.462+0000 7f7ac7e00640 1 -- 172.21.15.152:0/67064125 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f7abc07cd80 con 0x7f7ad00cd0d0 2024-04-20T11:36:41.707 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.706+0000 7f7ad6c00640 1 -- 172.21.15.152:0/67064125 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7ad00b1610 con 0x7f7ad00cd0d0 2024-04-20T11:36:41.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.708+0000 7f7ac7e00640 1 -- 172.21.15.152:0/67064125 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 97 v97) ==== 94+0+5209 (secure 0 0 0) 0x7f7abc07c680 con 0x7f7ad00cd0d0 2024-04-20T11:36:41.709 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:36:41.709 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":97,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6275,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6834/721066448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":721066448},{"type":"v1","addr":"172.21.15.175:6834","nonce":721066448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":91},{"gid":9483,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6834/3026417634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":3026417634},{"type":"v1","addr":"172.21.15.152:6834","nonce":3026417634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":90},{"gid":9486,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/3399678659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6833","nonce":3399678659},{"type":"v1","addr":"172.21.15.152:6835","nonce":3399678659}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":91}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T11:36:08.919586+0000","modified":"2024-04-20T11:36:12.020458+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9501},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9501":{"gid":9501,"name":"b","rank":0,"incarnation":92,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6835/189289100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6833","nonce":189289100},{"type":"v1","addr":"172.21.15.175:6835","nonce":189289100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9501,"qdb_cluster":[9501]},"id":11}]} 2024-04-20T11:36:41.709 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 97 2024-04-20T11:36:41.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.711+0000 7f7ad6c00640 1 -- 172.21.15.152:0/67064125 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f7ab006d0d0 msgr2=0x7f7ab006f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:41.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.711+0000 7f7ad6c00640 1 --2- 172.21.15.152:0/67064125 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f7ab006d0d0 0x7f7ab006f590 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f7ac0001500 tx=0x7f7ac0009040 comp rx=0 tx=0).stop 2024-04-20T11:36:41.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.711+0000 7f7ad6c00640 1 -- 172.21.15.152:0/67064125 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7ad00cd0d0 msgr2=0x7f7ad0141500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:41.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.711+0000 7f7ad6c00640 1 --2- 172.21.15.152:0/67064125 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7ad00cd0d0 0x7f7ad0141500 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 crypto rx=0x7f7abc037040 tx=0x7f7abc004a40 comp rx=0 tx=0).stop 2024-04-20T11:36:41.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.711+0000 7f7ad6c00640 1 -- 172.21.15.152:0/67064125 shutdown_connections 2024-04-20T11:36:41.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.711+0000 7f7ad6c00640 1 --2- 172.21.15.152:0/67064125 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f7ab006d0d0 0x7f7ab006f590 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:41.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.711+0000 7f7ad6c00640 1 --2- 172.21.15.152:0/67064125 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7ad0145f60 0x7f7ad01e3c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:41.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.711+0000 7f7ad6c00640 1 --2- 172.21.15.152:0/67064125 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7ad00cd0d0 0x7f7ad0141500 unknown :-1 s=CLOSED pgs=2061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:41.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.711+0000 7f7ad6c00640 1 --2- 172.21.15.152:0/67064125 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ad0071fe0 0x7f7ad0132e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:41.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.711+0000 7f7ad6c00640 1 -- 172.21.15.152:0/67064125 >> 172.21.15.152:0/67064125 conn(0x7f7ad0067cf0 msgr2=0x7f7ad006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:41.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.711+0000 7f7ad6c00640 1 -- 172.21.15.152:0/67064125 shutdown_connections 2024-04-20T11:36:41.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.711+0000 7f7ad6c00640 1 -- 172.21.15.152:0/67064125 wait complete. 2024-04-20T11:36:41.730 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:36:41.931 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.930+0000 7f2a08c00640 1 Processor -- start 2024-04-20T11:36:41.931 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.930+0000 7f2a08c00640 1 -- start start 2024-04-20T11:36:41.932 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.931+0000 7f2a08c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a04145f60 0x7f2a0406ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:41.932 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.931+0000 7f2a08c00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f2a040cd0d0 0x7f2a040cd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:41.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.932+0000 7f2a08c00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f2a04071fe0 0x7f2a04072470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:41.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.932+0000 7f2a08c00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f2a0406e9f0 con 0x7f2a04071fe0 2024-04-20T11:36:41.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.932+0000 7f2a08c00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f2a04146340 con 0x7f2a040cd0d0 2024-04-20T11:36:41.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.932+0000 7f2a08c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2a0406f3c0 con 0x7f2a04145f60 2024-04-20T11:36:41.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.932+0000 7f2a02800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f2a04071fe0 0x7f2a04072470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:41.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.932+0000 7f2a02800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f2a04071fe0 0x7f2a04072470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:57628/0 (socket says 172.21.15.152:57628) 2024-04-20T11:36:41.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.932+0000 7f2a02800640 1 -- 172.21.15.152:0/3342928922 learned_addr learned my addr 172.21.15.152:0/3342928922 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:41.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.932+0000 7f2a01400640 1 --2- 172.21.15.152:0/3342928922 >> v2:172.21.15.175:3300/0 conn(0x7f2a040cd0d0 0x7f2a040cd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:41.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.932+0000 7f2a01e00640 1 --2- 172.21.15.152:0/3342928922 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a04145f60 0x7f2a0406ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:41.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.932+0000 7f2a02800640 1 -- 172.21.15.152:0/3342928922 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a04145f60 msgr2=0x7f2a0406ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:41.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.933+0000 7f2a02800640 1 --2- 172.21.15.152:0/3342928922 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a04145f60 0x7f2a0406ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:41.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.933+0000 7f2a02800640 1 -- 172.21.15.152:0/3342928922 >> v2:172.21.15.175:3300/0 conn(0x7f2a040cd0d0 msgr2=0x7f2a040cd550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:41.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.933+0000 7f2a02800640 1 --2- 172.21.15.152:0/3342928922 >> v2:172.21.15.175:3300/0 conn(0x7f2a040cd0d0 0x7f2a040cd550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:41.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.933+0000 7f2a02800640 1 -- 172.21.15.152:0/3342928922 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a04072d50 con 0x7f2a04071fe0 2024-04-20T11:36:41.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.933+0000 7f2a01400640 1 --2- 172.21.15.152:0/3342928922 >> v2:172.21.15.175:3300/0 conn(0x7f2a040cd0d0 0x7f2a040cd550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:36:41.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.933+0000 7f2a01e00640 1 --2- 172.21.15.152:0/3342928922 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a04145f60 0x7f2a0406ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:36:41.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.934+0000 7f2a02800640 1 --2- 172.21.15.152:0/3342928922 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2a04071fe0 0x7f2a04072470 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 crypto rx=0x7f2a040b5890 tx=0x7f29fc02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ab3431893279dae server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:41.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.934+0000 7f2a00a00640 1 -- 172.21.15.152:0/3342928922 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29fc02ca70 con 0x7f2a04071fe0 2024-04-20T11:36:41.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.934+0000 7f2a00a00640 1 -- 172.21.15.152:0/3342928922 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29fc02cc10 con 0x7f2a04071fe0 2024-04-20T11:36:41.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.934+0000 7f2a00a00640 1 -- 172.21.15.152:0/3342928922 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29fc0343f0 con 0x7f2a04071fe0 2024-04-20T11:36:41.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.935+0000 7f2a08c00640 1 -- 172.21.15.152:0/3342928922 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2a04071fe0 msgr2=0x7f2a04072470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:41.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.935+0000 7f2a08c00640 1 --2- 172.21.15.152:0/3342928922 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2a04071fe0 0x7f2a04072470 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 crypto rx=0x7f2a040b5890 tx=0x7f29fc02bed0 comp rx=0 tx=0).stop 2024-04-20T11:36:41.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.935+0000 7f2a08c00640 1 -- 172.21.15.152:0/3342928922 shutdown_connections 2024-04-20T11:36:41.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.935+0000 7f2a08c00640 1 --2- 172.21.15.152:0/3342928922 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2a04071fe0 0x7f2a04072470 unknown :-1 s=CLOSED pgs=2062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:41.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.935+0000 7f2a08c00640 1 --2- 172.21.15.152:0/3342928922 >> v2:172.21.15.175:3300/0 conn(0x7f2a040cd0d0 0x7f2a040cd550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:41.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.935+0000 7f2a08c00640 1 --2- 172.21.15.152:0/3342928922 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a04145f60 0x7f2a0406ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:41.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.935+0000 7f2a08c00640 1 -- 172.21.15.152:0/3342928922 >> 172.21.15.152:0/3342928922 conn(0x7f2a04067cf0 msgr2=0x7f2a0406b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:41.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.935+0000 7f2a08c00640 1 -- 172.21.15.152:0/3342928922 shutdown_connections 2024-04-20T11:36:41.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.935+0000 7f2a08c00640 1 -- 172.21.15.152:0/3342928922 wait complete. 2024-04-20T11:36:41.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.937+0000 7f2a08c00640 1 Processor -- start 2024-04-20T11:36:41.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.937+0000 7f2a08c00640 1 -- start start 2024-04-20T11:36:41.939 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.938+0000 7f2a08c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2a04071fe0 0x7f2a041f4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:41.939 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.938+0000 7f2a08c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a040cd0d0 0x7f2a04141a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:41.939 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.938+0000 7f2a08c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a04145f60 0x7f2a0415e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:41.939 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.938+0000 7f2a08c00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f2a0406e680 con 0x7f2a04071fe0 2024-04-20T11:36:41.939 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.938+0000 7f2a08c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2a0406e3a0 con 0x7f2a040cd0d0 2024-04-20T11:36:41.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.938+0000 7f2a08c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2a040729b0 con 0x7f2a04145f60 2024-04-20T11:36:41.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.938+0000 7f2a01e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2a04071fe0 0x7f2a041f4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:41.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.938+0000 7f2a01e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2a04071fe0 0x7f2a041f4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:57638/0 (socket says 172.21.15.152:57638) 2024-04-20T11:36:41.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.938+0000 7f2a01e00640 1 -- 172.21.15.152:0/4018471393 learned_addr learned my addr 172.21.15.152:0/4018471393 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:41.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.938+0000 7f2a02800640 1 --2- 172.21.15.152:0/4018471393 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a04145f60 0x7f2a0415e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:41.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.938+0000 7f2a01e00640 1 -- 172.21.15.152:0/4018471393 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a04145f60 msgr2=0x7f2a0415e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:41.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.938+0000 7f2a01400640 1 --2- 172.21.15.152:0/4018471393 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a040cd0d0 0x7f2a04141a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:41.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.938+0000 7f2a01e00640 1 --2- 172.21.15.152:0/4018471393 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a04145f60 0x7f2a0415e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:41.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.938+0000 7f2a01e00640 1 -- 172.21.15.152:0/4018471393 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a040cd0d0 msgr2=0x7f2a04141a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:41.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.938+0000 7f2a01e00640 1 --2- 172.21.15.152:0/4018471393 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a040cd0d0 0x7f2a04141a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:41.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.938+0000 7f2a01e00640 1 -- 172.21.15.152:0/4018471393 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a0415eea0 con 0x7f2a04071fe0 2024-04-20T11:36:41.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.939+0000 7f2a01e00640 1 --2- 172.21.15.152:0/4018471393 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2a04071fe0 0x7f2a041f4370 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7f29f400c7f0 tx=0x7f29f400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:41.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.939+0000 7f29fa000640 1 -- 172.21.15.152:0/4018471393 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29f4015070 con 0x7f2a04071fe0 2024-04-20T11:36:41.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.939+0000 7f29fa000640 1 -- 172.21.15.152:0/4018471393 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29f4007820 con 0x7f2a04071fe0 2024-04-20T11:36:41.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.939+0000 7f29fa000640 1 -- 172.21.15.152:0/4018471393 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29f4007bd0 con 0x7f2a04071fe0 2024-04-20T11:36:41.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.939+0000 7f2a08c00640 1 -- 172.21.15.152:0/4018471393 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a040ada50 con 0x7f2a04071fe0 2024-04-20T11:36:41.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.939+0000 7f2a08c00640 1 -- 172.21.15.152:0/4018471393 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a040b36c0 con 0x7f2a04071fe0 2024-04-20T11:36:41.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.940+0000 7f2a08c00640 1 -- 172.21.15.152:0/4018471393 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a0406c740 con 0x7f2a04071fe0 2024-04-20T11:36:41.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.941+0000 7f29fa000640 1 -- 172.21.15.152:0/4018471393 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) ==== 91754+0+0 (secure 0 0 0) 0x7f29f4018070 con 0x7f2a04071fe0 2024-04-20T11:36:41.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.942+0000 7f29fa000640 1 --2- 172.21.15.152:0/4018471393 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f29e406d0d0 0x7f29e406f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:41.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.942+0000 7f29fa000640 1 -- 172.21.15.152:0/4018471393 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5709+0+0 (secure 0 0 0) 0x7f29f408d030 con 0x7f2a04071fe0 2024-04-20T11:36:41.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.943+0000 7f2a01400640 1 --2- 172.21.15.152:0/4018471393 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f29e406d0d0 0x7f29e406f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:41.945 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.944+0000 7f2a01400640 1 --2- 172.21.15.152:0/4018471393 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f29e406d0d0 0x7f29e406f590 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f29f0006fd0 tx=0x7f29f0008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:41.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:41.947+0000 7f29fa000640 1 -- 172.21.15.152:0/4018471393 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f2a0406c740 con 0x7f2a04071fe0 2024-04-20T11:36:42.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.148+0000 7f2a08c00640 1 -- 172.21.15.152:0/4018471393 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2a040689f0 con 0x7f2a04071fe0 2024-04-20T11:36:42.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.149+0000 7f29fa000640 1 -- 172.21.15.152:0/4018471393 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v142) ==== 74+0+16731 (secure 0 0 0) 0x7f29f4083e20 con 0x7f2a04071fe0 2024-04-20T11:36:42.150 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:36:42.150 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":142,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:36:10.484514+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:36:06.702770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"141","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-04-20T11:36:07.911633+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:36:42.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.151+0000 7f2a08c00640 1 -- 172.21.15.152:0/4018471393 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f29e406d0d0 msgr2=0x7f29e406f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:42.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.151+0000 7f2a08c00640 1 --2- 172.21.15.152:0/4018471393 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f29e406d0d0 0x7f29e406f590 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f29f0006fd0 tx=0x7f29f0008040 comp rx=0 tx=0).stop 2024-04-20T11:36:42.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.152+0000 7f2a08c00640 1 -- 172.21.15.152:0/4018471393 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2a04071fe0 msgr2=0x7f2a041f4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:42.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.152+0000 7f2a08c00640 1 --2- 172.21.15.152:0/4018471393 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2a04071fe0 0x7f2a041f4370 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7f29f400c7f0 tx=0x7f29f400ccb0 comp rx=0 tx=0).stop 2024-04-20T11:36:42.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.152+0000 7f2a08c00640 1 -- 172.21.15.152:0/4018471393 shutdown_connections 2024-04-20T11:36:42.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.152+0000 7f2a08c00640 1 --2- 172.21.15.152:0/4018471393 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f29e406d0d0 0x7f29e406f590 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:42.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.152+0000 7f2a08c00640 1 --2- 172.21.15.152:0/4018471393 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a04145f60 0x7f2a0415e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:42.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.152+0000 7f2a08c00640 1 --2- 172.21.15.152:0/4018471393 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a040cd0d0 0x7f2a04141a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:42.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.152+0000 7f2a08c00640 1 --2- 172.21.15.152:0/4018471393 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2a04071fe0 0x7f2a041f4370 unknown :-1 s=CLOSED pgs=2063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:42.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.152+0000 7f2a08c00640 1 -- 172.21.15.152:0/4018471393 >> 172.21.15.152:0/4018471393 conn(0x7f2a04067cf0 msgr2=0x7f2a0406f5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:42.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.152+0000 7f2a08c00640 1 -- 172.21.15.152:0/4018471393 shutdown_connections 2024-04-20T11:36:42.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.152+0000 7f2a08c00640 1 -- 172.21.15.152:0/4018471393 wait complete. 2024-04-20T11:36:42.169 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T11:26:47.246571+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 22, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T11:36:06.702770+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '141', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 23, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T11:36:07.911633+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '142', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T11:36:42.169 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T11:36:42.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.357+0000 7fccbd600640 1 Processor -- start 2024-04-20T11:36:42.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.358+0000 7fccbd600640 1 -- start start 2024-04-20T11:36:42.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.359+0000 7fccbd600640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fccb8152450 0x7fccb815c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:42.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.359+0000 7fccbd600640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fccb80b30c0 0x7fccb806bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:42.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.359+0000 7fccbd600640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fccb8152830 con 0x7fccb80b3520 2024-04-20T11:36:42.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.359+0000 7fccbd600640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fccb80b39c0 con 0x7fccb80b30c0 2024-04-20T11:36:42.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.359+0000 7fccbd600640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fccb80b3ca0 con 0x7fccb8152450 2024-04-20T11:36:42.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.359+0000 7fccaf400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fccb80b30c0 0x7fccb806bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:42.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.359+0000 7fccbcc00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fccb80b3520 0x7fccb815efa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:55574/0 (socket says 172.21.15.152:55574) 2024-04-20T11:36:42.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.359+0000 7fccbcc00640 1 -- 172.21.15.152:0/2907125765 learned_addr learned my addr 172.21.15.152:0/2907125765 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:36:42.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.359+0000 7fccb7e00640 1 --2- 172.21.15.152:0/2907125765 >> v2:172.21.15.175:3300/0 conn(0x7fccb8152450 0x7fccb815c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:42.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.360+0000 7fccaf400640 1 -- 172.21.15.152:0/2907125765 >> v2:172.21.15.175:3300/0 conn(0x7fccb8152450 msgr2=0x7fccb815c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:42.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.360+0000 7fccaf400640 1 --2- 172.21.15.152:0/2907125765 >> v2:172.21.15.175:3300/0 conn(0x7fccb8152450 0x7fccb815c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:42.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.360+0000 7fccaf400640 1 -- 172.21.15.152:0/2907125765 >> v1:172.21.15.152:6789/0 conn(0x7fccb80b3520 legacy=0x7fccb815efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:42.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.360+0000 7fccaf400640 1 -- 172.21.15.152:0/2907125765 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccb80b3f80 con 0x7fccb80b30c0 2024-04-20T11:36:42.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.360+0000 7fccb7400640 1 -- 172.21.15.152:0/2907125765 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1362846766 0 0) 0x7fccb8152830 con 0x7fccb80b3520 2024-04-20T11:36:42.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.361+0000 7fccaf400640 1 --2- 172.21.15.152:0/2907125765 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fccb80b30c0 0x7fccb806bda0 secure :-1 s=READY pgs=2065 cs=0 l=1 rev1=1 crypto rx=0x7fccb80d1250 tx=0x7fcca002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed94db50d9459e40 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:42.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.361+0000 7fccb7400640 1 -- 172.21.15.152:0/2907125765 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca0035020 con 0x7fccb80b30c0 2024-04-20T11:36:42.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.361+0000 7fccb7400640 1 -- 172.21.15.152:0/2907125765 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcca0031830 con 0x7fccb80b30c0 2024-04-20T11:36:42.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.361+0000 7fccb7400640 1 -- 172.21.15.152:0/2907125765 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca0031c00 con 0x7fccb80b30c0 2024-04-20T11:36:42.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.362+0000 7fccbd600640 1 -- 172.21.15.152:0/2907125765 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fccb80b30c0 msgr2=0x7fccb806bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:42.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.362+0000 7fccbd600640 1 --2- 172.21.15.152:0/2907125765 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fccb80b30c0 0x7fccb806bda0 secure :-1 s=READY pgs=2065 cs=0 l=1 rev1=1 crypto rx=0x7fccb80d1250 tx=0x7fcca002b8d0 comp rx=0 tx=0).stop 2024-04-20T11:36:42.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.362+0000 7fccbd600640 1 -- 172.21.15.152:0/2907125765 shutdown_connections 2024-04-20T11:36:42.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.362+0000 7fccbd600640 1 --2- 172.21.15.152:0/2907125765 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fccb80b30c0 0x7fccb806bda0 unknown :-1 s=CLOSED pgs=2065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:42.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.362+0000 7fccbd600640 1 --2- 172.21.15.152:0/2907125765 >> v2:172.21.15.175:3300/0 conn(0x7fccb8152450 0x7fccb815c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:42.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.362+0000 7fccbd600640 1 -- 172.21.15.152:0/2907125765 >> 172.21.15.152:0/2907125765 conn(0x7fccb80b21e0 msgr2=0x7fccb80b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:42.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.362+0000 7fccbd600640 1 -- 172.21.15.152:0/2907125765 shutdown_connections 2024-04-20T11:36:42.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.362+0000 7fccbd600640 1 -- 172.21.15.152:0/2907125765 wait complete. 2024-04-20T11:36:42.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.364+0000 7fccbd600640 1 Processor -- start 2024-04-20T11:36:42.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.364+0000 7fccbd600640 1 -- start start 2024-04-20T11:36:42.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.365+0000 7fccbd600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fccb80b30c0 0x7fccb80b0950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:42.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.365+0000 7fccbd600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fccb80b3520 0x7fccb81f2360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:42.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.365+0000 7fccbd600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fccb8152450 0x7fccb80cec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:42.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.365+0000 7fccbd600640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fccb80af480 con 0x7fccb80b30c0 2024-04-20T11:36:42.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.365+0000 7fccbd600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fccb80adf00 con 0x7fccb8152450 2024-04-20T11:36:42.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.365+0000 7fccbd600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fccb80af600 con 0x7fccb80b3520 2024-04-20T11:36:42.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.365+0000 7fccb7e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fccb80b30c0 0x7fccb80b0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:42.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.365+0000 7fccb7e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fccb80b30c0 0x7fccb80b0950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:57666/0 (socket says 172.21.15.152:57666) 2024-04-20T11:36:42.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.365+0000 7fccb7e00640 1 -- 172.21.15.152:0/4273981264 learned_addr learned my addr 172.21.15.152:0/4273981264 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:42.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.366+0000 7fccbcc00640 1 --2- 172.21.15.152:0/4273981264 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fccb8152450 0x7fccb80cec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:42.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.366+0000 7fccaf400640 1 --2- 172.21.15.152:0/4273981264 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fccb80b3520 0x7fccb81f2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:42.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.366+0000 7fccb7e00640 1 -- 172.21.15.152:0/4273981264 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fccb80b3520 msgr2=0x7fccb81f2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:42.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.366+0000 7fccb7e00640 1 --2- 172.21.15.152:0/4273981264 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fccb80b3520 0x7fccb81f2360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:42.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.366+0000 7fccb7e00640 1 -- 172.21.15.152:0/4273981264 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fccb8152450 msgr2=0x7fccb80cec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:42.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.366+0000 7fccb7e00640 1 --2- 172.21.15.152:0/4273981264 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fccb8152450 0x7fccb80cec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:42.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.366+0000 7fccb7e00640 1 -- 172.21.15.152:0/4273981264 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccb8155820 con 0x7fccb80b30c0 2024-04-20T11:36:42.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.366+0000 7fccb7e00640 1 --2- 172.21.15.152:0/4273981264 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fccb80b30c0 0x7fccb80b0950 secure :-1 s=READY pgs=2066 cs=0 l=1 rev1=1 crypto rx=0x7fcca800d800 tx=0x7fcca800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:42.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.366+0000 7fccb4c00640 1 -- 172.21.15.152:0/4273981264 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca800b6c0 con 0x7fccb80b30c0 2024-04-20T11:36:42.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.366+0000 7fccb4c00640 1 -- 172.21.15.152:0/4273981264 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcca8019070 con 0x7fccb80b30c0 2024-04-20T11:36:42.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.366+0000 7fccb4c00640 1 -- 172.21.15.152:0/4273981264 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca800ba80 con 0x7fccb80b30c0 2024-04-20T11:36:42.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.366+0000 7fccbd600640 1 -- 172.21.15.152:0/4273981264 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccb80cdbc0 con 0x7fccb80b30c0 2024-04-20T11:36:42.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.366+0000 7fccbd600640 1 -- 172.21.15.152:0/4273981264 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccb80ce180 con 0x7fccb80b30c0 2024-04-20T11:36:42.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.368+0000 7fccbd600640 1 -- 172.21.15.152:0/4273981264 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccb8075e60 con 0x7fccb80b30c0 2024-04-20T11:36:42.369 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.369+0000 7fccb4c00640 1 -- 172.21.15.152:0/4273981264 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) ==== 91754+0+0 (secure 0 0 0) 0x7fcca800bc20 con 0x7fccb80b30c0 2024-04-20T11:36:42.370 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.369+0000 7fccb4c00640 1 --2- 172.21.15.152:0/4273981264 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fcc9406d0d0 0x7fcc9406f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:42.370 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.370+0000 7fccb4c00640 1 -- 172.21.15.152:0/4273981264 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5709+0+0 (secure 0 0 0) 0x7fcca808aa30 con 0x7fccb80b30c0 2024-04-20T11:36:42.370 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.370+0000 7fccaf400640 1 --2- 172.21.15.152:0/4273981264 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fcc9406d0d0 0x7fcc9406f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:42.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.371+0000 7fccaf400640 1 --2- 172.21.15.152:0/4273981264 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fcc9406d0d0 0x7fcc9406f590 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fcca0036000 tx=0x7fcca0008030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:42.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.374+0000 7fccb4c00640 1 -- 172.21.15.152:0/4273981264 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fcca805a750 con 0x7fccb80b30c0 2024-04-20T11:36:42.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.572+0000 7fccbd600640 1 -- 172.21.15.152:0/4273981264 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fccb806f770 con 0x7fccb80b30c0 2024-04-20T11:36:42.574 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.573+0000 7fccb4c00640 1 -- 172.21.15.152:0/4273981264 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v142) ==== 93+0+10 (secure 0 0 0) 0x7fcca805a050 con 0x7fccb80b30c0 2024-04-20T11:36:42.574 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 1 2024-04-20T11:36:42.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.576+0000 7fccbd600640 1 -- 172.21.15.152:0/4273981264 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fcc9406d0d0 msgr2=0x7fcc9406f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:42.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.576+0000 7fccbd600640 1 --2- 172.21.15.152:0/4273981264 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fcc9406d0d0 0x7fcc9406f590 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fcca0036000 tx=0x7fcca0008030 comp rx=0 tx=0).stop 2024-04-20T11:36:42.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.576+0000 7fccbd600640 1 -- 172.21.15.152:0/4273981264 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fccb80b30c0 msgr2=0x7fccb80b0950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:42.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.576+0000 7fccbd600640 1 --2- 172.21.15.152:0/4273981264 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fccb80b30c0 0x7fccb80b0950 secure :-1 s=READY pgs=2066 cs=0 l=1 rev1=1 crypto rx=0x7fcca800d800 tx=0x7fcca800dcc0 comp rx=0 tx=0).stop 2024-04-20T11:36:42.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.576+0000 7fccbd600640 1 -- 172.21.15.152:0/4273981264 shutdown_connections 2024-04-20T11:36:42.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.576+0000 7fccbd600640 1 --2- 172.21.15.152:0/4273981264 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fcc9406d0d0 0x7fcc9406f590 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:42.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.576+0000 7fccbd600640 1 --2- 172.21.15.152:0/4273981264 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fccb8152450 0x7fccb80cec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:42.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.576+0000 7fccbd600640 1 --2- 172.21.15.152:0/4273981264 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fccb80b3520 0x7fccb81f2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:42.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.576+0000 7fccbd600640 1 --2- 172.21.15.152:0/4273981264 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fccb80b30c0 0x7fccb80b0950 unknown :-1 s=CLOSED pgs=2066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:42.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.576+0000 7fccbd600640 1 -- 172.21.15.152:0/4273981264 >> 172.21.15.152:0/4273981264 conn(0x7fccb80b21e0 msgr2=0x7fccb806bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:42.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.576+0000 7fccbd600640 1 -- 172.21.15.152:0/4273981264 shutdown_connections 2024-04-20T11:36:42.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.576+0000 7fccbd600640 1 -- 172.21.15.152:0/4273981264 wait complete. 2024-04-20T11:36:42.596 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T11:36:42.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.798+0000 7f0d7e000640 1 Processor -- start 2024-04-20T11:36:42.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.798+0000 7f0d7e000640 1 -- start start 2024-04-20T11:36:42.801 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.800+0000 7f0d7e000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f0d78071cd0 0x7f0d780720b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:42.801 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.800+0000 7f0d7e000640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f0d78160220 0x7f0d781606b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:42.801 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.800+0000 7f0d7e000640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d780725f0 con 0x7f0d7806eb30 2024-04-20T11:36:42.801 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.800+0000 7f0d7e000640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f0d7813f8e0 con 0x7f0d78071cd0 2024-04-20T11:36:42.801 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.800+0000 7f0d7e000640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f0d78160f90 con 0x7f0d78160220 2024-04-20T11:36:42.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.800+0000 7f0d7cc00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f0d78071cd0 0x7f0d780720b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:42.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.800+0000 7f0d7cc00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f0d78071cd0 0x7f0d780720b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:57670/0 (socket says 172.21.15.152:57670) 2024-04-20T11:36:42.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.800+0000 7f0d7cc00640 1 -- 172.21.15.152:0/3475720253 learned_addr learned my addr 172.21.15.152:0/3475720253 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:42.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.800+0000 7f0d77e00640 1 --2- 172.21.15.152:0/3475720253 >> v2:172.21.15.175:3300/0 conn(0x7f0d78160220 0x7f0d781606b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:42.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.801+0000 7f0d7cc00640 1 -- 172.21.15.152:0/3475720253 >> v2:172.21.15.175:3300/0 conn(0x7f0d78160220 msgr2=0x7f0d781606b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:42.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.801+0000 7f0d7cc00640 1 --2- 172.21.15.152:0/3475720253 >> v2:172.21.15.175:3300/0 conn(0x7f0d78160220 0x7f0d781606b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:42.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.801+0000 7f0d7cc00640 1 -- 172.21.15.152:0/3475720253 >> v1:172.21.15.175:6789/0 conn(0x7f0d7806eb30 legacy=0x7f0d7806efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:42.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.801+0000 7f0d7cc00640 1 -- 172.21.15.152:0/3475720253 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d7806dd90 con 0x7f0d78071cd0 2024-04-20T11:36:42.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.801+0000 7f0d77e00640 1 --2- 172.21.15.152:0/3475720253 >> v2:172.21.15.175:3300/0 conn(0x7f0d78160220 0x7f0d781606b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:36:42.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.802+0000 7f0d7cc00640 1 --2- 172.21.15.152:0/3475720253 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0d78071cd0 0x7f0d780720b0 secure :-1 s=READY pgs=2067 cs=0 l=1 rev1=1 crypto rx=0x7f0d780cfd00 tx=0x7f0d6c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55c9e3e21bf9e3ec server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:42.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.802+0000 7f0d77400640 1 -- 172.21.15.152:0/3475720253 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d6c02e070 con 0x7f0d78071cd0 2024-04-20T11:36:42.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.802+0000 7f0d77400640 1 -- 172.21.15.152:0/3475720253 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d6c037070 con 0x7f0d78071cd0 2024-04-20T11:36:42.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.802+0000 7f0d77400640 1 -- 172.21.15.152:0/3475720253 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d6c031a00 con 0x7f0d78071cd0 2024-04-20T11:36:42.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.803+0000 7f0d7e000640 1 -- 172.21.15.152:0/3475720253 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0d78071cd0 msgr2=0x7f0d780720b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:42.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.803+0000 7f0d7e000640 1 --2- 172.21.15.152:0/3475720253 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0d78071cd0 0x7f0d780720b0 secure :-1 s=READY pgs=2067 cs=0 l=1 rev1=1 crypto rx=0x7f0d780cfd00 tx=0x7f0d6c02b880 comp rx=0 tx=0).stop 2024-04-20T11:36:42.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.803+0000 7f0d7e000640 1 -- 172.21.15.152:0/3475720253 shutdown_connections 2024-04-20T11:36:42.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.803+0000 7f0d7e000640 1 --2- 172.21.15.152:0/3475720253 >> v2:172.21.15.175:3300/0 conn(0x7f0d78160220 0x7f0d781606b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:42.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.803+0000 7f0d7e000640 1 --2- 172.21.15.152:0/3475720253 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0d78071cd0 0x7f0d780720b0 unknown :-1 s=CLOSED pgs=2067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:42.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.803+0000 7f0d7e000640 1 -- 172.21.15.152:0/3475720253 >> 172.21.15.152:0/3475720253 conn(0x7f0d78067ae0 msgr2=0x7f0d7806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:42.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.803+0000 7f0d7e000640 1 -- 172.21.15.152:0/3475720253 shutdown_connections 2024-04-20T11:36:42.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.803+0000 7f0d7e000640 1 -- 172.21.15.152:0/3475720253 wait complete. 2024-04-20T11:36:42.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.805+0000 7f0d7e000640 1 Processor -- start 2024-04-20T11:36:42.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.805+0000 7f0d7e000640 1 -- start start 2024-04-20T11:36:42.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.807+0000 7f0d7e000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0d7806eb30 0x7f0d781377e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:42.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.807+0000 7f0d7e000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d78071cd0 0x7f0d781562e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:42.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.807+0000 7f0d7e000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0d78160220 0x7f0d78137d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:42.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.807+0000 7f0d7e000640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f0d7806e630 con 0x7f0d7806eb30 2024-04-20T11:36:42.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.807+0000 7f0d7e000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0d7806e350 con 0x7f0d78071cd0 2024-04-20T11:36:42.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.807+0000 7f0d7e000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0d78160bf0 con 0x7f0d78160220 2024-04-20T11:36:42.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.807+0000 7f0d7cc00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0d7806eb30 0x7f0d781377e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:42.809 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.807+0000 7f0d7cc00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0d7806eb30 0x7f0d781377e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:57684/0 (socket says 172.21.15.152:57684) 2024-04-20T11:36:42.809 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.807+0000 7f0d7cc00640 1 -- 172.21.15.152:0/3303221228 learned_addr learned my addr 172.21.15.152:0/3303221228 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:42.809 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.807+0000 7f0d7cc00640 1 -- 172.21.15.152:0/3303221228 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0d78160220 msgr2=0x7f0d78137d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:42.809 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.807+0000 7f0d7cc00640 1 --2- 172.21.15.152:0/3303221228 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0d78160220 0x7f0d78137d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:42.809 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.807+0000 7f0d77e00640 1 --2- 172.21.15.152:0/3303221228 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d78071cd0 0x7f0d781562e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:42.809 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.807+0000 7f0d7cc00640 1 -- 172.21.15.152:0/3303221228 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d78071cd0 msgr2=0x7f0d781562e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:42.809 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.807+0000 7f0d7cc00640 1 --2- 172.21.15.152:0/3303221228 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d78071cd0 0x7f0d781562e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:42.809 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.807+0000 7f0d7cc00640 1 -- 172.21.15.152:0/3303221228 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d78156820 con 0x7f0d7806eb30 2024-04-20T11:36:42.809 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.808+0000 7f0d7cc00640 1 --2- 172.21.15.152:0/3303221228 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0d7806eb30 0x7f0d781377e0 secure :-1 s=READY pgs=2068 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c000c00 tx=0x7f0d6c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:42.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.808+0000 7f0d74c00640 1 -- 172.21.15.152:0/3303221228 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d6c032880 con 0x7f0d7806eb30 2024-04-20T11:36:42.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.808+0000 7f0d74c00640 1 -- 172.21.15.152:0/3303221228 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d6c032a20 con 0x7f0d7806eb30 2024-04-20T11:36:42.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.808+0000 7f0d74c00640 1 -- 172.21.15.152:0/3303221228 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d6c032e10 con 0x7f0d7806eb30 2024-04-20T11:36:42.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.808+0000 7f0d7e000640 1 -- 172.21.15.152:0/3303221228 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d781382c0 con 0x7f0d7806eb30 2024-04-20T11:36:42.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.808+0000 7f0d7e000640 1 -- 172.21.15.152:0/3303221228 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d78137470 con 0x7f0d7806eb30 2024-04-20T11:36:42.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.809+0000 7f0d7e000640 1 -- 172.21.15.152:0/3303221228 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d780cf840 con 0x7f0d7806eb30 2024-04-20T11:36:42.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.810+0000 7f0d74c00640 1 -- 172.21.15.152:0/3303221228 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) ==== 91754+0+0 (secure 0 0 0) 0x7f0d6c04c060 con 0x7f0d7806eb30 2024-04-20T11:36:42.812 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.811+0000 7f0d74c00640 1 --2- 172.21.15.152:0/3303221228 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f0d5806d0d0 0x7f0d5806f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:42.812 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.811+0000 7f0d74c00640 1 -- 172.21.15.152:0/3303221228 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5709+0+0 (secure 0 0 0) 0x7f0d6c0ae6a0 con 0x7f0d7806eb30 2024-04-20T11:36:42.812 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.812+0000 7f0d77e00640 1 --2- 172.21.15.152:0/3303221228 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f0d5806d0d0 0x7f0d5806f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:42.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.813+0000 7f0d77e00640 1 --2- 172.21.15.152:0/3303221228 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f0d5806d0d0 0x7f0d5806f590 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f0d780af980 tx=0x7f0d64008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:42.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:42.816+0000 7f0d74c00640 1 -- 172.21.15.152:0/3303221228 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f0d780cf840 con 0x7f0d7806eb30 2024-04-20T11:36:43.017 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.016+0000 7f0d7e000640 1 -- 172.21.15.152:0/3303221228 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f0d78075e60 con 0x7f0d7806eb30 2024-04-20T11:36:43.018 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.017+0000 7f0d74c00640 1 -- 172.21.15.152:0/3303221228 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v142) ==== 104+0+11 (secure 0 0 0) 0x7f0d6c040070 con 0x7f0d7806eb30 2024-04-20T11:36:43.018 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 32 2024-04-20T11:36:43.021 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.020+0000 7f0d7e000640 1 -- 172.21.15.152:0/3303221228 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f0d5806d0d0 msgr2=0x7f0d5806f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:43.021 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.020+0000 7f0d7e000640 1 --2- 172.21.15.152:0/3303221228 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f0d5806d0d0 0x7f0d5806f590 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f0d780af980 tx=0x7f0d64008040 comp rx=0 tx=0).stop 2024-04-20T11:36:43.021 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.020+0000 7f0d7e000640 1 -- 172.21.15.152:0/3303221228 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0d7806eb30 msgr2=0x7f0d781377e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:43.021 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.020+0000 7f0d7e000640 1 --2- 172.21.15.152:0/3303221228 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0d7806eb30 0x7f0d781377e0 secure :-1 s=READY pgs=2068 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c000c00 tx=0x7f0d6c0046c0 comp rx=0 tx=0).stop 2024-04-20T11:36:43.021 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.020+0000 7f0d7e000640 1 -- 172.21.15.152:0/3303221228 shutdown_connections 2024-04-20T11:36:43.021 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.020+0000 7f0d7e000640 1 --2- 172.21.15.152:0/3303221228 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f0d5806d0d0 0x7f0d5806f590 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:43.022 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.020+0000 7f0d7e000640 1 --2- 172.21.15.152:0/3303221228 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0d78160220 0x7f0d78137d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:43.022 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.020+0000 7f0d7e000640 1 --2- 172.21.15.152:0/3303221228 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d78071cd0 0x7f0d781562e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:43.022 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.020+0000 7f0d7e000640 1 --2- 172.21.15.152:0/3303221228 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0d7806eb30 0x7f0d781377e0 unknown :-1 s=CLOSED pgs=2068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:43.022 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.020+0000 7f0d7e000640 1 -- 172.21.15.152:0/3303221228 >> 172.21.15.152:0/3303221228 conn(0x7f0d78067ae0 msgr2=0x7f0d7815cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:43.022 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.021+0000 7f0d7e000640 1 -- 172.21.15.152:0/3303221228 shutdown_connections 2024-04-20T11:36:43.022 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.021+0000 7f0d7e000640 1 -- 172.21.15.152:0/3303221228 wait complete. 2024-04-20T11:36:43.041 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T11:36:43.237 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.236+0000 7f8e87e00640 1 Processor -- start 2024-04-20T11:36:43.237 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.237+0000 7f8e87e00640 1 -- start start 2024-04-20T11:36:43.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.238+0000 7f8e87e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f8e880cf5b0 0x7f8e880d4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:43.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.238+0000 7f8e87e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8e88160230 0x7f8e881606e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:43.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.238+0000 7f8e87e00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e880d2b70 con 0x7f8e880ceeb0 2024-04-20T11:36:43.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.238+0000 7f8e87e00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f8e880d5420 con 0x7f8e880cf5b0 2024-04-20T11:36:43.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.238+0000 7f8e87e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8e880d2e50 con 0x7f8e88160230 2024-04-20T11:36:43.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.238+0000 7f8e86000640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f8e880ceeb0 0x7f8e880cf990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:55590/0 (socket says 172.21.15.152:55590) 2024-04-20T11:36:43.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.238+0000 7f8e86000640 1 -- 172.21.15.152:0/3733754369 learned_addr learned my addr 172.21.15.152:0/3733754369 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:36:43.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.238+0000 7f8e85600640 1 -- 172.21.15.152:0/3733754369 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3142230986 0 0) 0x7f8e880d2b70 con 0x7f8e880ceeb0 2024-04-20T11:36:43.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.238+0000 7f8e85600640 1 -- 172.21.15.152:0/3733754369 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8e6c003520 con 0x7f8e880ceeb0 2024-04-20T11:36:43.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.238+0000 7f8e86a00640 1 --2- 172.21.15.152:0/3733754369 >> v2:172.21.15.175:3300/0 conn(0x7f8e880cf5b0 0x7f8e880d4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:43.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.238+0000 7f8e87400640 1 --2- 172.21.15.152:0/3733754369 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8e88160230 0x7f8e881606e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:43.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.239+0000 7f8e85600640 1 -- 172.21.15.152:0/3733754369 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1990849324 0 0) 0x7f8e6c003520 con 0x7f8e880ceeb0 2024-04-20T11:36:43.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.239+0000 7f8e85600640 1 -- 172.21.15.152:0/3733754369 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8e88160230 msgr2=0x7f8e881606e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:43.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.239+0000 7f8e85600640 1 --2- 172.21.15.152:0/3733754369 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8e88160230 0x7f8e881606e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:43.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.239+0000 7f8e85600640 1 -- 172.21.15.152:0/3733754369 >> v2:172.21.15.175:3300/0 conn(0x7f8e880cf5b0 msgr2=0x7f8e880d4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:43.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.239+0000 7f8e85600640 1 --2- 172.21.15.152:0/3733754369 >> v2:172.21.15.175:3300/0 conn(0x7f8e880cf5b0 0x7f8e880d4ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:43.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.239+0000 7f8e85600640 1 -- 172.21.15.152:0/3733754369 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e880d3130 con 0x7f8e880ceeb0 2024-04-20T11:36:43.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.239+0000 7f8e85600640 1 -- 172.21.15.152:0/3733754369 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f8e700017f0 con 0x7f8e880ceeb0 2024-04-20T11:36:43.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.239+0000 7f8e85600640 1 -- 172.21.15.152:0/3733754369 >> v1:172.21.15.152:6789/0 conn(0x7f8e880ceeb0 legacy=0x7f8e880cf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:43.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.239+0000 7f8e85600640 1 --2- 172.21.15.152:0/3733754369 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8e6c0039b0 0x7f8e6c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:43.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.239+0000 7f8e85600640 1 --2- 172.21.15.152:0/3733754369 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e6c004800 0x7f8e6c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:43.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.239+0000 7f8e85600640 1 --2- 172.21.15.152:0/3733754369 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8e88160230 0x7f8e6c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:43.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.239+0000 7f8e85600640 1 -- 172.21.15.152:0/3733754369 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f8e880d5420 con 0x7f8e6c0039b0 2024-04-20T11:36:43.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.239+0000 7f8e85600640 1 -- 172.21.15.152:0/3733754369 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8e880d2e50 con 0x7f8e6c004800 2024-04-20T11:36:43.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.239+0000 7f8e85600640 1 -- 172.21.15.152:0/3733754369 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8e6c00bb30 con 0x7f8e88160230 2024-04-20T11:36:43.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.239+0000 7f8e86a00640 1 --2- 172.21.15.152:0/3733754369 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8e6c0039b0 0x7f8e6c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:43.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.239+0000 7f8e86a00640 1 -- 172.21.15.152:0/3733754369 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8e88160230 msgr2=0x7f8e6c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:43.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.239+0000 7f8e86a00640 1 --2- 172.21.15.152:0/3733754369 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8e88160230 0x7f8e6c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:43.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.239+0000 7f8e86a00640 1 -- 172.21.15.152:0/3733754369 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e6c004800 msgr2=0x7f8e6c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:43.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.239+0000 7f8e86a00640 1 --2- 172.21.15.152:0/3733754369 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e6c004800 0x7f8e6c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:43.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.240+0000 7f8e86a00640 1 -- 172.21.15.152:0/3733754369 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e6c003520 con 0x7f8e6c0039b0 2024-04-20T11:36:43.243 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.241+0000 7f8e86a00640 1 --2- 172.21.15.152:0/3733754369 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8e6c0039b0 0x7f8e6c003dd0 secure :-1 s=READY pgs=2070 cs=0 l=1 rev1=1 crypto rx=0x7f8e78007330 tx=0x7f8e6c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=541d41eb493335c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:43.243 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.241+0000 7f8e85600640 1 -- 172.21.15.152:0/3733754369 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e78039070 con 0x7f8e6c0039b0 2024-04-20T11:36:43.243 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.241+0000 7f8e85600640 1 -- 172.21.15.152:0/3733754369 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e7802ccb0 con 0x7f8e6c0039b0 2024-04-20T11:36:43.243 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.241+0000 7f8e85600640 1 -- 172.21.15.152:0/3733754369 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e78009be0 con 0x7f8e6c0039b0 2024-04-20T11:36:43.243 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.241+0000 7f8e87e00640 1 -- 172.21.15.152:0/3733754369 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8e6c0039b0 msgr2=0x7f8e6c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:43.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.241+0000 7f8e87e00640 1 --2- 172.21.15.152:0/3733754369 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8e6c0039b0 0x7f8e6c003dd0 secure :-1 s=READY pgs=2070 cs=0 l=1 rev1=1 crypto rx=0x7f8e78007330 tx=0x7f8e6c005830 comp rx=0 tx=0).stop 2024-04-20T11:36:43.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.241+0000 7f8e87400640 1 -- 172.21.15.152:0/3733754369 reap_dead start 2024-04-20T11:36:43.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.241+0000 7f8e87e00640 1 -- 172.21.15.152:0/3733754369 shutdown_connections 2024-04-20T11:36:43.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.241+0000 7f8e87e00640 1 -- 172.21.15.152:0/3733754369 >> 172.21.15.152:0/3733754369 conn(0x7f8e8808bfd0 msgr2=0x7f8e880d4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:43.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.241+0000 7f8e87e00640 1 -- 172.21.15.152:0/3733754369 shutdown_connections 2024-04-20T11:36:43.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.242+0000 7f8e87e00640 1 -- 172.21.15.152:0/3733754369 wait complete. 2024-04-20T11:36:43.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.243+0000 7f8e87e00640 1 Processor -- start 2024-04-20T11:36:43.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.243+0000 7f8e87e00640 1 -- start start 2024-04-20T11:36:43.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.244+0000 7f8e87e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8e6c0039b0 0x7f8e88156b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:43.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.244+0000 7f8e87e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8e881f7970 0x7f8e881f5da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:43.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.244+0000 7f8e87e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e881f7df0 0x7f8e881f62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:43.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.244+0000 7f8e87e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f8e880d3c10 con 0x7f8e881f7970 2024-04-20T11:36:43.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.244+0000 7f8e87e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8e880d36f0 con 0x7f8e881f7df0 2024-04-20T11:36:43.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.244+0000 7f8e87e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8e880d2790 con 0x7f8e6c0039b0 2024-04-20T11:36:43.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.244+0000 7f8e86000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8e881f7970 0x7f8e881f5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:43.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.245+0000 7f8e86000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8e881f7970 0x7f8e881f5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:57694/0 (socket says 172.21.15.152:57694) 2024-04-20T11:36:43.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.245+0000 7f8e86000640 1 -- 172.21.15.152:0/2049859554 learned_addr learned my addr 172.21.15.152:0/2049859554 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:43.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.245+0000 7f8e86000640 1 -- 172.21.15.152:0/2049859554 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8e6c0039b0 msgr2=0x7f8e88156b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:43.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.245+0000 7f8e86000640 1 --2- 172.21.15.152:0/2049859554 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8e6c0039b0 0x7f8e88156b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:43.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.245+0000 7f8e86000640 1 -- 172.21.15.152:0/2049859554 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e881f7df0 msgr2=0x7f8e881f62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:43.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.245+0000 7f8e86000640 1 --2- 172.21.15.152:0/2049859554 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e881f7df0 0x7f8e881f62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:43.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.245+0000 7f8e86000640 1 -- 172.21.15.152:0/2049859554 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e88157050 con 0x7f8e881f7970 2024-04-20T11:36:43.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.245+0000 7f8e86000640 1 --2- 172.21.15.152:0/2049859554 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8e881f7970 0x7f8e881f5da0 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 crypto rx=0x7f8e7000c800 tx=0x7f8e7000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:43.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.245+0000 7f8e7ea00640 1 -- 172.21.15.152:0/2049859554 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e70017070 con 0x7f8e881f7970 2024-04-20T11:36:43.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.245+0000 7f8e7ea00640 1 -- 172.21.15.152:0/2049859554 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e70007480 con 0x7f8e881f7970 2024-04-20T11:36:43.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.245+0000 7f8e7ea00640 1 -- 172.21.15.152:0/2049859554 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e700059c0 con 0x7f8e881f7970 2024-04-20T11:36:43.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.245+0000 7f8e87e00640 1 -- 172.21.15.152:0/2049859554 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e881f68d0 con 0x7f8e881f7970 2024-04-20T11:36:43.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.246+0000 7f8e87e00640 1 -- 172.21.15.152:0/2049859554 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e881f6e90 con 0x7f8e881f7970 2024-04-20T11:36:43.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.247+0000 7f8e87e00640 1 -- 172.21.15.152:0/2049859554 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e88075e60 con 0x7f8e881f7970 2024-04-20T11:36:43.249 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.248+0000 7f8e7ea00640 1 -- 172.21.15.152:0/2049859554 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) ==== 91754+0+0 (secure 0 0 0) 0x7f8e7001a070 con 0x7f8e881f7970 2024-04-20T11:36:43.249 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.249+0000 7f8e7ea00640 1 --2- 172.21.15.152:0/2049859554 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f8e6406d100 0x7f8e6406f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:43.249 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.249+0000 7f8e7ea00640 1 -- 172.21.15.152:0/2049859554 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5709+0+0 (secure 0 0 0) 0x7f8e70089c60 con 0x7f8e881f7970 2024-04-20T11:36:43.250 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.250+0000 7f8e86a00640 1 --2- 172.21.15.152:0/2049859554 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f8e6406d100 0x7f8e6406f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:43.251 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.251+0000 7f8e86a00640 1 --2- 172.21.15.152:0/2049859554 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f8e6406d100 0x7f8e6406f5c0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f8e78036000 tx=0x7f8e7802e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:43.254 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.254+0000 7f8e7ea00640 1 -- 172.21.15.152:0/2049859554 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f8e88157050 con 0x7f8e881f7970 2024-04-20T11:36:43.442 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.442+0000 7f8e87e00640 1 -- 172.21.15.152:0/2049859554 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f8e880cf390 con 0x7f8e881f7970 2024-04-20T11:36:43.443 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.443+0000 7f8e7ea00640 1 -- 172.21.15.152:0/2049859554 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v142) ==== 100+0+11 (secure 0 0 0) 0x7f8e70059980 con 0x7f8e881f7970 2024-04-20T11:36:43.443 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 64 2024-04-20T11:36:43.446 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.445+0000 7f8e87e00640 1 -- 172.21.15.152:0/2049859554 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f8e6406d100 msgr2=0x7f8e6406f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:43.446 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.445+0000 7f8e87e00640 1 --2- 172.21.15.152:0/2049859554 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f8e6406d100 0x7f8e6406f5c0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f8e78036000 tx=0x7f8e7802e000 comp rx=0 tx=0).stop 2024-04-20T11:36:43.446 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.445+0000 7f8e87e00640 1 -- 172.21.15.152:0/2049859554 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8e881f7970 msgr2=0x7f8e881f5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:43.446 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.445+0000 7f8e87e00640 1 --2- 172.21.15.152:0/2049859554 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8e881f7970 0x7f8e881f5da0 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 crypto rx=0x7f8e7000c800 tx=0x7f8e7000ccc0 comp rx=0 tx=0).stop 2024-04-20T11:36:43.446 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.445+0000 7f8e87e00640 1 -- 172.21.15.152:0/2049859554 shutdown_connections 2024-04-20T11:36:43.446 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.445+0000 7f8e87e00640 1 --2- 172.21.15.152:0/2049859554 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f8e6406d100 0x7f8e6406f5c0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:43.447 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.445+0000 7f8e87e00640 1 --2- 172.21.15.152:0/2049859554 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e881f7df0 0x7f8e881f62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:43.447 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.445+0000 7f8e87e00640 1 --2- 172.21.15.152:0/2049859554 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8e881f7970 0x7f8e881f5da0 unknown :-1 s=CLOSED pgs=2071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:43.447 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.445+0000 7f8e87e00640 1 --2- 172.21.15.152:0/2049859554 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8e6c0039b0 0x7f8e88156b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:43.447 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.445+0000 7f8e87e00640 1 -- 172.21.15.152:0/2049859554 >> 172.21.15.152:0/2049859554 conn(0x7f8e8808bfd0 msgr2=0x7f8e880d0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:43.447 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.446+0000 7f8e87e00640 1 -- 172.21.15.152:0/2049859554 shutdown_connections 2024-04-20T11:36:43.447 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.446+0000 7f8e87e00640 1 -- 172.21.15.152:0/2049859554 wait complete. 2024-04-20T11:36:43.464 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:36:43.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.655+0000 7f658d200640 1 Processor -- start 2024-04-20T11:36:43.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.655+0000 7f658d200640 1 -- start start 2024-04-20T11:36:43.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.657+0000 7f658d200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f6588160870 0x7f6588163c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:43.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.657+0000 7f658d200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f65880b3d80 0x7f65880b4210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:43.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.657+0000 7f658d200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f6588071db0 0x7f6588072200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:43.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.657+0000 7f658d200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f6588160c50 con 0x7f6588160870 2024-04-20T11:36:43.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.657+0000 7f658d200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f65881641b0 con 0x7f6588071db0 2024-04-20T11:36:43.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.657+0000 7f658d200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f658806e140 con 0x7f65880b3d80 2024-04-20T11:36:43.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.657+0000 7f6585e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f6588160870 0x7f6588163c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:43.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.657+0000 7f6585e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f6588160870 0x7f6588163c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:57708/0 (socket says 172.21.15.152:57708) 2024-04-20T11:36:43.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.657+0000 7f6585e00640 1 -- 172.21.15.152:0/2082330866 learned_addr learned my addr 172.21.15.152:0/2082330866 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:43.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.658+0000 7f6585400640 1 --2- 172.21.15.152:0/2082330866 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f65880b3d80 0x7f65880b4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:43.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.658+0000 7f6586800640 1 --2- 172.21.15.152:0/2082330866 >> v2:172.21.15.175:3300/0 conn(0x7f6588071db0 0x7f6588072200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:43.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.658+0000 7f6585e00640 1 -- 172.21.15.152:0/2082330866 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f65880b3d80 msgr2=0x7f65880b4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:43.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.658+0000 7f6585e00640 1 --2- 172.21.15.152:0/2082330866 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f65880b3d80 0x7f65880b4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:43.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.658+0000 7f6585e00640 1 -- 172.21.15.152:0/2082330866 >> v2:172.21.15.175:3300/0 conn(0x7f6588071db0 msgr2=0x7f6588072200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:43.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.658+0000 7f6585e00640 1 --2- 172.21.15.152:0/2082330866 >> v2:172.21.15.175:3300/0 conn(0x7f6588071db0 0x7f6588072200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:43.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.658+0000 7f6585e00640 1 -- 172.21.15.152:0/2082330866 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f658806e420 con 0x7f6588160870 2024-04-20T11:36:43.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.659+0000 7f6585e00640 1 --2- 172.21.15.152:0/2082330866 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6588160870 0x7f6588163c70 secure :-1 s=READY pgs=2072 cs=0 l=1 rev1=1 crypto rx=0x7f6588073200 tx=0x7f657802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cd9bc2ea23d865e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:43.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.660+0000 7f6584a00640 1 -- 172.21.15.152:0/2082330866 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6578035020 con 0x7f6588160870 2024-04-20T11:36:43.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.660+0000 7f6584a00640 1 -- 172.21.15.152:0/2082330866 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6578031870 con 0x7f6588160870 2024-04-20T11:36:43.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.660+0000 7f6584a00640 1 -- 172.21.15.152:0/2082330866 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6578031c40 con 0x7f6588160870 2024-04-20T11:36:43.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.660+0000 7f658d200640 1 -- 172.21.15.152:0/2082330866 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6588160870 msgr2=0x7f6588163c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:43.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.660+0000 7f658d200640 1 --2- 172.21.15.152:0/2082330866 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6588160870 0x7f6588163c70 secure :-1 s=READY pgs=2072 cs=0 l=1 rev1=1 crypto rx=0x7f6588073200 tx=0x7f657802b910 comp rx=0 tx=0).stop 2024-04-20T11:36:43.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.660+0000 7f658d200640 1 -- 172.21.15.152:0/2082330866 shutdown_connections 2024-04-20T11:36:43.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.660+0000 7f658d200640 1 --2- 172.21.15.152:0/2082330866 >> v2:172.21.15.175:3300/0 conn(0x7f6588071db0 0x7f6588072200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:43.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.660+0000 7f658d200640 1 --2- 172.21.15.152:0/2082330866 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f65880b3d80 0x7f65880b4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:43.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.660+0000 7f658d200640 1 --2- 172.21.15.152:0/2082330866 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6588160870 0x7f6588163c70 unknown :-1 s=CLOSED pgs=2072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:43.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.660+0000 7f658d200640 1 -- 172.21.15.152:0/2082330866 >> 172.21.15.152:0/2082330866 conn(0x7f6588067cf0 msgr2=0x7f658806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:43.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.661+0000 7f658d200640 1 -- 172.21.15.152:0/2082330866 shutdown_connections 2024-04-20T11:36:43.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.661+0000 7f658d200640 1 -- 172.21.15.152:0/2082330866 wait complete. 2024-04-20T11:36:43.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.663+0000 7f658d200640 1 Processor -- start 2024-04-20T11:36:43.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.663+0000 7f658d200640 1 -- start start 2024-04-20T11:36:43.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.664+0000 7f658d200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6588071db0 0x7f65880acc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:43.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.664+0000 7f658d200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f65880b3d80 0x7f65880ae4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:43.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.664+0000 7f658d200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6588160870 0x7f65880b1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:43.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.664+0000 7f658d200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f658806ecc0 con 0x7f6588160870 2024-04-20T11:36:43.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.664+0000 7f658d200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f658806e9e0 con 0x7f6588071db0 2024-04-20T11:36:43.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.664+0000 7f658d200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f658806de10 con 0x7f65880b3d80 2024-04-20T11:36:43.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.664+0000 7f6586800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6588160870 0x7f65880b1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:43.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.664+0000 7f6586800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6588160870 0x7f65880b1560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:57716/0 (socket says 172.21.15.152:57716) 2024-04-20T11:36:43.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.664+0000 7f6586800640 1 -- 172.21.15.152:0/879140977 learned_addr learned my addr 172.21.15.152:0/879140977 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:43.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.665+0000 7f6585400640 1 --2- 172.21.15.152:0/879140977 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f65880b3d80 0x7f65880ae4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:43.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.665+0000 7f6585e00640 1 --2- 172.21.15.152:0/879140977 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6588071db0 0x7f65880acc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:43.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.665+0000 7f6586800640 1 -- 172.21.15.152:0/879140977 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f65880b3d80 msgr2=0x7f65880ae4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:43.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.665+0000 7f6586800640 1 --2- 172.21.15.152:0/879140977 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f65880b3d80 0x7f65880ae4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:43.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.665+0000 7f6586800640 1 -- 172.21.15.152:0/879140977 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6588071db0 msgr2=0x7f65880acc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:43.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.665+0000 7f6586800640 1 --2- 172.21.15.152:0/879140977 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6588071db0 0x7f65880acc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:43.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.665+0000 7f6586800640 1 -- 172.21.15.152:0/879140977 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65880ad1b0 con 0x7f6588160870 2024-04-20T11:36:43.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.665+0000 7f6586800640 1 --2- 172.21.15.152:0/879140977 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6588160870 0x7f65880b1560 secure :-1 s=READY pgs=2073 cs=0 l=1 rev1=1 crypto rx=0x7f6580007e70 tx=0x7f658000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:43.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.665+0000 7f657e000640 1 -- 172.21.15.152:0/879140977 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6580014620 con 0x7f6588160870 2024-04-20T11:36:43.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.665+0000 7f657e000640 1 -- 172.21.15.152:0/879140977 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6580019070 con 0x7f6588160870 2024-04-20T11:36:43.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.665+0000 7f658d200640 1 -- 172.21.15.152:0/879140977 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65880ad430 con 0x7f6588160870 2024-04-20T11:36:43.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.665+0000 7f657e000640 1 -- 172.21.15.152:0/879140977 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6580014a20 con 0x7f6588160870 2024-04-20T11:36:43.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.666+0000 7f658d200640 1 -- 172.21.15.152:0/879140977 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65880ad9f0 con 0x7f6588160870 2024-04-20T11:36:43.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.667+0000 7f658d200640 1 -- 172.21.15.152:0/879140977 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f658806f1e0 con 0x7f6588160870 2024-04-20T11:36:43.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.668+0000 7f657e000640 1 -- 172.21.15.152:0/879140977 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) ==== 91754+0+0 (secure 0 0 0) 0x7f6580014bc0 con 0x7f6588160870 2024-04-20T11:36:43.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.669+0000 7f657e000640 1 --2- 172.21.15.152:0/879140977 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f656806d0d0 0x7f656806f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:43.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.669+0000 7f657e000640 1 -- 172.21.15.152:0/879140977 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5709+0+0 (secure 0 0 0) 0x7f658008d670 con 0x7f6588160870 2024-04-20T11:36:43.670 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.669+0000 7f6585e00640 1 --2- 172.21.15.152:0/879140977 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f656806d0d0 0x7f656806f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:43.671 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.671+0000 7f6585e00640 1 --2- 172.21.15.152:0/879140977 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f656806d0d0 0x7f656806f590 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f6578036000 tx=0x7f657803c040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:43.674 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.674+0000 7f657e000640 1 -- 172.21.15.152:0/879140977 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f658806f1e0 con 0x7f6588160870 2024-04-20T11:36:43.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.914+0000 7f658d200640 1 -- 172.21.15.152:0/879140977 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f65880689f0 con 0x7f6588160870 2024-04-20T11:36:43.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.915+0000 7f657e000640 1 -- 172.21.15.152:0/879140977 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 97 v97) ==== 94+0+5209 (secure 0 0 0) 0x7f65880ad1b0 con 0x7f6588160870 2024-04-20T11:36:43.916 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:36:43.916 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":97,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6275,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6834/721066448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":721066448},{"type":"v1","addr":"172.21.15.175:6834","nonce":721066448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":91},{"gid":9483,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6834/3026417634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":3026417634},{"type":"v1","addr":"172.21.15.152:6834","nonce":3026417634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":90},{"gid":9486,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/3399678659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6833","nonce":3399678659},{"type":"v1","addr":"172.21.15.152:6835","nonce":3399678659}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":91}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T11:36:08.919586+0000","modified":"2024-04-20T11:36:12.020458+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9501},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9501":{"gid":9501,"name":"b","rank":0,"incarnation":92,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6835/189289100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6833","nonce":189289100},{"type":"v1","addr":"172.21.15.175:6835","nonce":189289100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9501,"qdb_cluster":[9501]},"id":11}]} 2024-04-20T11:36:43.917 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 97 2024-04-20T11:36:43.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.918+0000 7f658d200640 1 -- 172.21.15.152:0/879140977 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f656806d0d0 msgr2=0x7f656806f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:43.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.918+0000 7f658d200640 1 --2- 172.21.15.152:0/879140977 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f656806d0d0 0x7f656806f590 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f6578036000 tx=0x7f657803c040 comp rx=0 tx=0).stop 2024-04-20T11:36:43.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.918+0000 7f658d200640 1 -- 172.21.15.152:0/879140977 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6588160870 msgr2=0x7f65880b1560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:43.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.918+0000 7f658d200640 1 --2- 172.21.15.152:0/879140977 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6588160870 0x7f65880b1560 secure :-1 s=READY pgs=2073 cs=0 l=1 rev1=1 crypto rx=0x7f6580007e70 tx=0x7f658000e4b0 comp rx=0 tx=0).stop 2024-04-20T11:36:43.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.918+0000 7f658d200640 1 -- 172.21.15.152:0/879140977 shutdown_connections 2024-04-20T11:36:43.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.918+0000 7f658d200640 1 --2- 172.21.15.152:0/879140977 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f656806d0d0 0x7f656806f590 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:43.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.918+0000 7f658d200640 1 --2- 172.21.15.152:0/879140977 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6588160870 0x7f65880b1560 unknown :-1 s=CLOSED pgs=2073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:43.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.918+0000 7f658d200640 1 --2- 172.21.15.152:0/879140977 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f65880b3d80 0x7f65880ae4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:43.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.918+0000 7f658d200640 1 --2- 172.21.15.152:0/879140977 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6588071db0 0x7f65880acc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:43.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.918+0000 7f658d200640 1 -- 172.21.15.152:0/879140977 >> 172.21.15.152:0/879140977 conn(0x7f6588067cf0 msgr2=0x7f6588161190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:43.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.919+0000 7f658d200640 1 -- 172.21.15.152:0/879140977 shutdown_connections 2024-04-20T11:36:43.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:43.919+0000 7f658d200640 1 -- 172.21.15.152:0/879140977 wait complete. 2024-04-20T11:36:43.938 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:36:44.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.141+0000 7f083cc00640 1 Processor -- start 2024-04-20T11:36:44.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.141+0000 7f083cc00640 1 -- start start 2024-04-20T11:36:44.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.143+0000 7f083cc00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f083815d1e0 0x7f0838160b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:44.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.143+0000 7f083cc00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f083815d5c0 0x7f08380cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:44.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.143+0000 7f083cc00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f0838071cd0 0x7f0838072160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:44.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.143+0000 7f083cc00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f08380726a0 con 0x7f0838071cd0 2024-04-20T11:36:44.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.143+0000 7f083cc00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f08380ce8c0 con 0x7f083815d5c0 2024-04-20T11:36:44.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.143+0000 7f083cc00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f08381600b0 con 0x7f083815d1e0 2024-04-20T11:36:44.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.143+0000 7f0837e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f0838071cd0 0x7f0838072160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:44.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.143+0000 7f0837e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f0838071cd0 0x7f0838072160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:57720/0 (socket says 172.21.15.152:57720) 2024-04-20T11:36:44.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.143+0000 7f0837e00640 1 -- 172.21.15.152:0/1661239352 learned_addr learned my addr 172.21.15.152:0/1661239352 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:44.145 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.143+0000 7f0837400640 1 --2- 172.21.15.152:0/1661239352 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f083815d1e0 0x7f0838160b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:44.145 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.143+0000 7f0837e00640 1 -- 172.21.15.152:0/1661239352 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f083815d1e0 msgr2=0x7f0838160b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:44.145 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.143+0000 7f082fe00640 1 --2- 172.21.15.152:0/1661239352 >> v2:172.21.15.175:3300/0 conn(0x7f083815d5c0 0x7f08380cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:44.145 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.143+0000 7f0837e00640 1 --2- 172.21.15.152:0/1661239352 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f083815d1e0 0x7f0838160b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:44.145 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.143+0000 7f0837e00640 1 -- 172.21.15.152:0/1661239352 >> v2:172.21.15.175:3300/0 conn(0x7f083815d5c0 msgr2=0x7f08380cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:44.145 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.143+0000 7f0837e00640 1 --2- 172.21.15.152:0/1661239352 >> v2:172.21.15.175:3300/0 conn(0x7f083815d5c0 0x7f08380cdfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:44.145 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.143+0000 7f0837e00640 1 -- 172.21.15.152:0/1661239352 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0838160390 con 0x7f0838071cd0 2024-04-20T11:36:44.146 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.145+0000 7f0837e00640 1 --2- 172.21.15.152:0/1661239352 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0838071cd0 0x7f0838072160 secure :-1 s=READY pgs=2074 cs=0 l=1 rev1=1 crypto rx=0x7f08380b5810 tx=0x7f083002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=660f4345cfd7b385 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:44.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.145+0000 7f0836a00640 1 -- 172.21.15.152:0/1661239352 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f083002ca70 con 0x7f0838071cd0 2024-04-20T11:36:44.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.145+0000 7f0836a00640 1 -- 172.21.15.152:0/1661239352 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f083002cc10 con 0x7f0838071cd0 2024-04-20T11:36:44.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.145+0000 7f0836a00640 1 -- 172.21.15.152:0/1661239352 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08300343f0 con 0x7f0838071cd0 2024-04-20T11:36:44.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.145+0000 7f083cc00640 1 -- 172.21.15.152:0/1661239352 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0838071cd0 msgr2=0x7f0838072160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:44.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.145+0000 7f083cc00640 1 --2- 172.21.15.152:0/1661239352 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0838071cd0 0x7f0838072160 secure :-1 s=READY pgs=2074 cs=0 l=1 rev1=1 crypto rx=0x7f08380b5810 tx=0x7f083002bed0 comp rx=0 tx=0).stop 2024-04-20T11:36:44.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.146+0000 7f083cc00640 1 -- 172.21.15.152:0/1661239352 shutdown_connections 2024-04-20T11:36:44.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.146+0000 7f083cc00640 1 --2- 172.21.15.152:0/1661239352 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0838071cd0 0x7f0838072160 unknown :-1 s=CLOSED pgs=2074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:44.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.146+0000 7f083cc00640 1 --2- 172.21.15.152:0/1661239352 >> v2:172.21.15.175:3300/0 conn(0x7f083815d5c0 0x7f08380cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:44.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.146+0000 7f083cc00640 1 --2- 172.21.15.152:0/1661239352 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f083815d1e0 0x7f0838160b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:44.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.146+0000 7f083cc00640 1 -- 172.21.15.152:0/1661239352 >> 172.21.15.152:0/1661239352 conn(0x7f0838067ae0 msgr2=0x7f083806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:44.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.146+0000 7f083cc00640 1 -- 172.21.15.152:0/1661239352 shutdown_connections 2024-04-20T11:36:44.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.146+0000 7f083cc00640 1 -- 172.21.15.152:0/1661239352 wait complete. 2024-04-20T11:36:44.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.147+0000 7f083cc00640 1 Processor -- start 2024-04-20T11:36:44.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.147+0000 7f083cc00640 1 -- start start 2024-04-20T11:36:44.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.148+0000 7f083cc00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0838071cd0 0x7f083814a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:44.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.148+0000 7f083cc00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f083815d1e0 0x7f0838152800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:44.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.148+0000 7f083cc00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f083815d5c0 0x7f0838152d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:44.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.148+0000 7f083cc00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f08380ceba0 con 0x7f083815d1e0 2024-04-20T11:36:44.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.148+0000 7f083cc00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f083815d9a0 con 0x7f0838071cd0 2024-04-20T11:36:44.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.148+0000 7f083cc00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f08380ce520 con 0x7f083815d5c0 2024-04-20T11:36:44.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.149+0000 7f082fe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f083815d1e0 0x7f0838152800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:44.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.149+0000 7f082fe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f083815d1e0 0x7f0838152800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:57734/0 (socket says 172.21.15.152:57734) 2024-04-20T11:36:44.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.149+0000 7f082fe00640 1 -- 172.21.15.152:0/3062159888 learned_addr learned my addr 172.21.15.152:0/3062159888 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:44.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.149+0000 7f082fe00640 1 -- 172.21.15.152:0/3062159888 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f083815d5c0 msgr2=0x7f0838152d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:44.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.149+0000 7f082fe00640 1 --2- 172.21.15.152:0/3062159888 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f083815d5c0 0x7f0838152d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:44.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.149+0000 7f082fe00640 1 -- 172.21.15.152:0/3062159888 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0838071cd0 msgr2=0x7f083814a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:44.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.149+0000 7f082fe00640 1 --2- 172.21.15.152:0/3062159888 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0838071cd0 0x7f083814a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:44.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.149+0000 7f082fe00640 1 -- 172.21.15.152:0/3062159888 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08380b3ce0 con 0x7f083815d1e0 2024-04-20T11:36:44.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.149+0000 7f082fe00640 1 --2- 172.21.15.152:0/3062159888 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f083815d1e0 0x7f0838152800 secure :-1 s=READY pgs=2075 cs=0 l=1 rev1=1 crypto rx=0x7f082000c820 tx=0x7f082000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:44.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.150+0000 7f082f400640 1 -- 172.21.15.152:0/3062159888 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0820015070 con 0x7f083815d1e0 2024-04-20T11:36:44.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.150+0000 7f082f400640 1 -- 172.21.15.152:0/3062159888 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0820004240 con 0x7f083815d1e0 2024-04-20T11:36:44.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.150+0000 7f082f400640 1 -- 172.21.15.152:0/3062159888 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f082000ce10 con 0x7f083815d1e0 2024-04-20T11:36:44.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.150+0000 7f083cc00640 1 -- 172.21.15.152:0/3062159888 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08380b3fc0 con 0x7f083815d1e0 2024-04-20T11:36:44.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.150+0000 7f083cc00640 1 -- 172.21.15.152:0/3062159888 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08380b4580 con 0x7f083815d1e0 2024-04-20T11:36:44.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.151+0000 7f083cc00640 1 -- 172.21.15.152:0/3062159888 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08380b47c0 con 0x7f083815d1e0 2024-04-20T11:36:44.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.152+0000 7f082f400640 1 -- 172.21.15.152:0/3062159888 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) ==== 91754+0+0 (secure 0 0 0) 0x7f0820007830 con 0x7f083815d1e0 2024-04-20T11:36:44.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.153+0000 7f082f400640 1 --2- 172.21.15.152:0/3062159888 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f081406d0d0 0x7f081406f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:44.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.153+0000 7f082f400640 1 -- 172.21.15.152:0/3062159888 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5709+0+0 (secure 0 0 0) 0x7f082008ce70 con 0x7f083815d1e0 2024-04-20T11:36:44.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.153+0000 7f0837400640 1 --2- 172.21.15.152:0/3062159888 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f081406d0d0 0x7f081406f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:44.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.155+0000 7f0837400640 1 --2- 172.21.15.152:0/3062159888 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f081406d0d0 0x7f081406f590 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f0828006b10 tx=0x7f0828008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:44.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.158+0000 7f082f400640 1 -- 172.21.15.152:0/3062159888 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f08380b47c0 con 0x7f083815d1e0 2024-04-20T11:36:44.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.357+0000 7f083cc00640 1 -- 172.21.15.152:0/3062159888 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0838075e60 con 0x7f083815d1e0 2024-04-20T11:36:44.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.360+0000 7f082f400640 1 -- 172.21.15.152:0/3062159888 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v142) ==== 74+0+16731 (secure 0 0 0) 0x7f082005a170 con 0x7f083815d1e0 2024-04-20T11:36:44.361 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:36:44.361 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":142,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:36:10.484514+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:36:06.702770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"141","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-04-20T11:36:07.911633+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:36:44.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.363+0000 7f083cc00640 1 -- 172.21.15.152:0/3062159888 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f081406d0d0 msgr2=0x7f081406f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:44.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.363+0000 7f083cc00640 1 --2- 172.21.15.152:0/3062159888 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f081406d0d0 0x7f081406f590 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f0828006b10 tx=0x7f0828008040 comp rx=0 tx=0).stop 2024-04-20T11:36:44.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.363+0000 7f083cc00640 1 -- 172.21.15.152:0/3062159888 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f083815d1e0 msgr2=0x7f0838152800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:44.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.363+0000 7f083cc00640 1 --2- 172.21.15.152:0/3062159888 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f083815d1e0 0x7f0838152800 secure :-1 s=READY pgs=2075 cs=0 l=1 rev1=1 crypto rx=0x7f082000c820 tx=0x7f082000cce0 comp rx=0 tx=0).stop 2024-04-20T11:36:44.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.363+0000 7f083cc00640 1 -- 172.21.15.152:0/3062159888 shutdown_connections 2024-04-20T11:36:44.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.363+0000 7f083cc00640 1 --2- 172.21.15.152:0/3062159888 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f081406d0d0 0x7f081406f590 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:44.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.363+0000 7f083cc00640 1 --2- 172.21.15.152:0/3062159888 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f083815d5c0 0x7f0838152d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:44.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.363+0000 7f083cc00640 1 --2- 172.21.15.152:0/3062159888 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f083815d1e0 0x7f0838152800 unknown :-1 s=CLOSED pgs=2075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:44.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.363+0000 7f083cc00640 1 --2- 172.21.15.152:0/3062159888 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0838071cd0 0x7f083814a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:44.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.363+0000 7f083cc00640 1 -- 172.21.15.152:0/3062159888 >> 172.21.15.152:0/3062159888 conn(0x7f0838067ae0 msgr2=0x7f083815a5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:44.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.363+0000 7f083cc00640 1 -- 172.21.15.152:0/3062159888 shutdown_connections 2024-04-20T11:36:44.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.363+0000 7f083cc00640 1 -- 172.21.15.152:0/3062159888 wait complete. 2024-04-20T11:36:44.383 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-20T11:36:44.383 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:36:44.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.580+0000 7fa1cd600640 1 Processor -- start 2024-04-20T11:36:44.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.580+0000 7fa1cd600640 1 -- start start 2024-04-20T11:36:44.582 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.582+0000 7fa1cd600640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fa1c80cd0d0 0x7fa1c80cd580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:44.582 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.582+0000 7fa1cd600640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fa1c8071fe0 0x7fa1c8072470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:44.582 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.582+0000 7fa1cd600640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1c8146340 con 0x7fa1c8145f60 2024-04-20T11:36:44.583 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.582+0000 7fa1cd600640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fa1c806e9f0 con 0x7fa1c8071fe0 2024-04-20T11:36:44.583 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.582+0000 7fa1cd600640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fa1c8072d50 con 0x7fa1c80cd0d0 2024-04-20T11:36:44.583 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.582+0000 7fa1c7400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fa1c8071fe0 0x7fa1c8072470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:44.583 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.582+0000 7fa1c7400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fa1c8071fe0 0x7fa1c8072470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:57738/0 (socket says 172.21.15.152:57738) 2024-04-20T11:36:44.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.582+0000 7fa1c7400640 1 -- 172.21.15.152:0/3917632889 learned_addr learned my addr 172.21.15.152:0/3917632889 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:44.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.582+0000 7fa1c6000640 1 --2- 172.21.15.152:0/3917632889 >> v2:172.21.15.175:3300/0 conn(0x7fa1c80cd0d0 0x7fa1c80cd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:44.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.583+0000 7fa1c7400640 1 -- 172.21.15.152:0/3917632889 >> v2:172.21.15.175:3300/0 conn(0x7fa1c80cd0d0 msgr2=0x7fa1c80cd580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:44.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.583+0000 7fa1c7400640 1 --2- 172.21.15.152:0/3917632889 >> v2:172.21.15.175:3300/0 conn(0x7fa1c80cd0d0 0x7fa1c80cd580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:44.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.583+0000 7fa1c7400640 1 -- 172.21.15.152:0/3917632889 >> v1:172.21.15.175:6789/0 conn(0x7fa1c8145f60 legacy=0x7fa1c806ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:44.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.583+0000 7fa1c7400640 1 -- 172.21.15.152:0/3917632889 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1c806e0c0 con 0x7fa1c8071fe0 2024-04-20T11:36:44.585 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.584+0000 7fa1c7400640 1 --2- 172.21.15.152:0/3917632889 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa1c8071fe0 0x7fa1c8072470 secure :-1 s=READY pgs=2076 cs=0 l=1 rev1=1 crypto rx=0x7fa1c806eb70 tx=0x7fa1c002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f39e61b077dacab server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:44.585 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.585+0000 7fa1c5600640 1 -- 172.21.15.152:0/3917632889 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c002ca70 con 0x7fa1c8071fe0 2024-04-20T11:36:44.585 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.585+0000 7fa1c5600640 1 -- 172.21.15.152:0/3917632889 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1c002cc10 con 0x7fa1c8071fe0 2024-04-20T11:36:44.585 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.585+0000 7fa1c5600640 1 -- 172.21.15.152:0/3917632889 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c00343f0 con 0x7fa1c8071fe0 2024-04-20T11:36:44.586 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.585+0000 7fa1cd600640 1 -- 172.21.15.152:0/3917632889 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa1c8071fe0 msgr2=0x7fa1c8072470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:44.586 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.585+0000 7fa1cd600640 1 --2- 172.21.15.152:0/3917632889 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa1c8071fe0 0x7fa1c8072470 secure :-1 s=READY pgs=2076 cs=0 l=1 rev1=1 crypto rx=0x7fa1c806eb70 tx=0x7fa1c002bed0 comp rx=0 tx=0).stop 2024-04-20T11:36:44.586 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.586+0000 7fa1cd600640 1 -- 172.21.15.152:0/3917632889 shutdown_connections 2024-04-20T11:36:44.586 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.586+0000 7fa1cd600640 1 --2- 172.21.15.152:0/3917632889 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa1c8071fe0 0x7fa1c8072470 unknown :-1 s=CLOSED pgs=2076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:44.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.586+0000 7fa1cd600640 1 --2- 172.21.15.152:0/3917632889 >> v2:172.21.15.175:3300/0 conn(0x7fa1c80cd0d0 0x7fa1c80cd580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:44.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.586+0000 7fa1cd600640 1 -- 172.21.15.152:0/3917632889 >> 172.21.15.152:0/3917632889 conn(0x7fa1c8067cf0 msgr2=0x7fa1c806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:44.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.586+0000 7fa1cd600640 1 -- 172.21.15.152:0/3917632889 shutdown_connections 2024-04-20T11:36:44.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.586+0000 7fa1cd600640 1 -- 172.21.15.152:0/3917632889 wait complete. 2024-04-20T11:36:44.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.588+0000 7fa1cd600640 1 Processor -- start 2024-04-20T11:36:44.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.588+0000 7fa1cd600640 1 -- start start 2024-04-20T11:36:44.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.589+0000 7fa1cd600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa1c8071fe0 0x7fa1c8158f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:44.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.589+0000 7fa1cd600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa1c80cd0d0 0x7fa1c80cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:44.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.589+0000 7fa1cd600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa1c8145f60 0x7fa1c80cfd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:44.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.589+0000 7fa1cd600640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fa1c80729b0 con 0x7fa1c8071fe0 2024-04-20T11:36:44.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.589+0000 7fa1cd600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa1c806e680 con 0x7fa1c80cd0d0 2024-04-20T11:36:44.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.589+0000 7fa1cd600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa1c8072b30 con 0x7fa1c8145f60 2024-04-20T11:36:44.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.589+0000 7fa1c6a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa1c8071fe0 0x7fa1c8158f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:44.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.589+0000 7fa1c6a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa1c8071fe0 0x7fa1c8158f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:57742/0 (socket says 172.21.15.152:57742) 2024-04-20T11:36:44.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.589+0000 7fa1c6a00640 1 -- 172.21.15.152:0/865309669 learned_addr learned my addr 172.21.15.152:0/865309669 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:44.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.589+0000 7fa1c6000640 1 --2- 172.21.15.152:0/865309669 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa1c80cd0d0 0x7fa1c80cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:44.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.589+0000 7fa1c6a00640 1 -- 172.21.15.152:0/865309669 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa1c8145f60 msgr2=0x7fa1c80cfd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:44.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.589+0000 7fa1c6a00640 1 --2- 172.21.15.152:0/865309669 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa1c8145f60 0x7fa1c80cfd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:44.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.589+0000 7fa1c6a00640 1 -- 172.21.15.152:0/865309669 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa1c80cd0d0 msgr2=0x7fa1c80cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:44.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.589+0000 7fa1c6a00640 1 --2- 172.21.15.152:0/865309669 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa1c80cd0d0 0x7fa1c80cf7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:44.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.589+0000 7fa1c6a00640 1 -- 172.21.15.152:0/865309669 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1c80abe40 con 0x7fa1c8071fe0 2024-04-20T11:36:44.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.590+0000 7fa1c6a00640 1 --2- 172.21.15.152:0/865309669 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa1c8071fe0 0x7fa1c8158f70 secure :-1 s=READY pgs=2077 cs=0 l=1 rev1=1 crypto rx=0x7fa1b000b3f0 tx=0x7fa1b000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:44.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.590+0000 7fa1bea00640 1 -- 172.21.15.152:0/865309669 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1b0015020 con 0x7fa1c8071fe0 2024-04-20T11:36:44.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.590+0000 7fa1bea00640 1 -- 172.21.15.152:0/865309669 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1b0011830 con 0x7fa1c8071fe0 2024-04-20T11:36:44.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.590+0000 7fa1cd600640 1 -- 172.21.15.152:0/865309669 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1c80ac120 con 0x7fa1c8071fe0 2024-04-20T11:36:44.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.590+0000 7fa1bea00640 1 -- 172.21.15.152:0/865309669 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1b0011c20 con 0x7fa1c8071fe0 2024-04-20T11:36:44.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.590+0000 7fa1cd600640 1 -- 172.21.15.152:0/865309669 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1c80ac6e0 con 0x7fa1c8071fe0 2024-04-20T11:36:44.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.591+0000 7fa1cd600640 1 -- 172.21.15.152:0/865309669 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1c80689f0 con 0x7fa1c8071fe0 2024-04-20T11:36:44.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.592+0000 7fa1bea00640 1 -- 172.21.15.152:0/865309669 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) ==== 91754+0+0 (secure 0 0 0) 0x7fa1b0018020 con 0x7fa1c8071fe0 2024-04-20T11:36:44.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.593+0000 7fa1bea00640 1 --2- 172.21.15.152:0/865309669 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fa1a406d0d0 0x7fa1a406f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:44.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.593+0000 7fa1bea00640 1 -- 172.21.15.152:0/865309669 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5709+0+0 (secure 0 0 0) 0x7fa1b008cab0 con 0x7fa1c8071fe0 2024-04-20T11:36:44.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.594+0000 7fa1c6000640 1 --2- 172.21.15.152:0/865309669 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fa1a406d0d0 0x7fa1a406f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:44.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.595+0000 7fa1c6000640 1 --2- 172.21.15.152:0/865309669 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fa1a406d0d0 0x7fa1a406f590 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fa1c80ad4f0 tx=0x7fa1b8008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:44.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.597+0000 7fa1bea00640 1 -- 172.21.15.152:0/865309669 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fa1b001f030 con 0x7fa1c8071fe0 2024-04-20T11:36:44.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.833+0000 7fa1cd600640 1 -- 172.21.15.152:0/865309669 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa1c806bc60 con 0x7fa1c8071fe0 2024-04-20T11:36:44.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.835+0000 7fa1bea00640 1 -- 172.21.15.152:0/865309669 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 97 v97) ==== 94+0+5209 (secure 0 0 0) 0x7fa1b00172c0 con 0x7fa1c8071fe0 2024-04-20T11:36:44.836 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:36:44.836 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":97,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6275,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6834/721066448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":721066448},{"type":"v1","addr":"172.21.15.175:6834","nonce":721066448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":91},{"gid":9483,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6834/3026417634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":3026417634},{"type":"v1","addr":"172.21.15.152:6834","nonce":3026417634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":90},{"gid":9486,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/3399678659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6833","nonce":3399678659},{"type":"v1","addr":"172.21.15.152:6835","nonce":3399678659}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":91}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T11:36:08.919586+0000","modified":"2024-04-20T11:36:12.020458+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9501},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9501":{"gid":9501,"name":"b","rank":0,"incarnation":92,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6835/189289100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6833","nonce":189289100},{"type":"v1","addr":"172.21.15.175:6835","nonce":189289100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9501,"qdb_cluster":[9501]},"id":11}]} 2024-04-20T11:36:44.836 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 97 2024-04-20T11:36:44.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.838+0000 7fa1cd600640 1 -- 172.21.15.152:0/865309669 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fa1a406d0d0 msgr2=0x7fa1a406f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:44.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.838+0000 7fa1cd600640 1 --2- 172.21.15.152:0/865309669 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fa1a406d0d0 0x7fa1a406f590 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fa1c80ad4f0 tx=0x7fa1b8008040 comp rx=0 tx=0).stop 2024-04-20T11:36:44.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.838+0000 7fa1cd600640 1 -- 172.21.15.152:0/865309669 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa1c8071fe0 msgr2=0x7fa1c8158f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:44.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.838+0000 7fa1cd600640 1 --2- 172.21.15.152:0/865309669 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa1c8071fe0 0x7fa1c8158f70 secure :-1 s=READY pgs=2077 cs=0 l=1 rev1=1 crypto rx=0x7fa1b000b3f0 tx=0x7fa1b000b8b0 comp rx=0 tx=0).stop 2024-04-20T11:36:44.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.838+0000 7fa1cd600640 1 -- 172.21.15.152:0/865309669 shutdown_connections 2024-04-20T11:36:44.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.838+0000 7fa1cd600640 1 --2- 172.21.15.152:0/865309669 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fa1a406d0d0 0x7fa1a406f590 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:44.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.838+0000 7fa1cd600640 1 --2- 172.21.15.152:0/865309669 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa1c8145f60 0x7fa1c80cfd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:44.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.838+0000 7fa1cd600640 1 --2- 172.21.15.152:0/865309669 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa1c80cd0d0 0x7fa1c80cf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:44.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.838+0000 7fa1cd600640 1 --2- 172.21.15.152:0/865309669 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa1c8071fe0 0x7fa1c8158f70 unknown :-1 s=CLOSED pgs=2077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:44.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.838+0000 7fa1cd600640 1 -- 172.21.15.152:0/865309669 >> 172.21.15.152:0/865309669 conn(0x7fa1c8067cf0 msgr2=0x7fa1c815ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:44.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.838+0000 7fa1cd600640 1 -- 172.21.15.152:0/865309669 shutdown_connections 2024-04-20T11:36:44.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:44.838+0000 7fa1cd600640 1 -- 172.21.15.152:0/865309669 wait complete. 2024-04-20T11:36:44.856 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:36:45.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.042+0000 7f2a86800640 1 Processor -- start 2024-04-20T11:36:45.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.042+0000 7f2a86800640 1 -- start start 2024-04-20T11:36:45.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.043+0000 7f2a86800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f2a8006f510 0x7f2a80140dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:45.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.043+0000 7f2a86800640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a800cf0a0 con 0x7f2a80141310 2024-04-20T11:36:45.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.043+0000 7f2a86800640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a800d00e0 con 0x7f2a8013e530 2024-04-20T11:36:45.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.043+0000 7f2a86800640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f2a80141b00 con 0x7f2a8006f510 2024-04-20T11:36:45.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.044+0000 7f2a85e00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f2a80141310 0x7f2a80071cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:55594/0 (socket says 172.21.15.152:55594) 2024-04-20T11:36:45.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.044+0000 7f2a85e00640 1 -- 172.21.15.152:0/3592942217 learned_addr learned my addr 172.21.15.152:0/3592942217 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:36:45.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.044+0000 7f2a84a00640 1 --2- 172.21.15.152:0/3592942217 >> v2:172.21.15.152:3300/0 conn(0x7f2a8006f510 0x7f2a80140dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:45.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.044+0000 7f2a84a00640 1 -- 172.21.15.152:0/3592942217 >> v1:172.21.15.175:6789/0 conn(0x7f2a8013e530 legacy=0x7f2a8006ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:45.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.044+0000 7f2a7fe00640 1 -- 172.21.15.152:0/3592942217 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1507657659 0 0) 0x7f2a78001490 con 0x7f2a80141310 2024-04-20T11:36:45.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.044+0000 7f2a84a00640 1 -- 172.21.15.152:0/3592942217 >> v1:172.21.15.152:6789/0 conn(0x7f2a80141310 legacy=0x7f2a80071cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:45.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.044+0000 7f2a84a00640 1 -- 172.21.15.152:0/3592942217 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a800cf320 con 0x7f2a8006f510 2024-04-20T11:36:45.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.045+0000 7f2a84a00640 1 --2- 172.21.15.152:0/3592942217 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2a8006f510 0x7f2a80140dd0 secure :-1 s=READY pgs=2079 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c0097a0 tx=0x7f2a6c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd022db34006df74 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:45.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.046+0000 7f2a7fe00640 1 -- 172.21.15.152:0/3592942217 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a6c035020 con 0x7f2a8006f510 2024-04-20T11:36:45.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.046+0000 7f2a7fe00640 1 -- 172.21.15.152:0/3592942217 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a6c031830 con 0x7f2a8006f510 2024-04-20T11:36:45.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.046+0000 7f2a7fe00640 1 -- 172.21.15.152:0/3592942217 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a6c031c00 con 0x7f2a8006f510 2024-04-20T11:36:45.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.046+0000 7f2a86800640 1 -- 172.21.15.152:0/3592942217 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2a8006f510 msgr2=0x7f2a80140dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:45.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.046+0000 7f2a86800640 1 --2- 172.21.15.152:0/3592942217 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2a8006f510 0x7f2a80140dd0 secure :-1 s=READY pgs=2079 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c0097a0 tx=0x7f2a6c02b8d0 comp rx=0 tx=0).stop 2024-04-20T11:36:45.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.046+0000 7f2a86800640 1 -- 172.21.15.152:0/3592942217 shutdown_connections 2024-04-20T11:36:45.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.046+0000 7f2a86800640 1 --2- 172.21.15.152:0/3592942217 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2a8006f510 0x7f2a80140dd0 unknown :-1 s=CLOSED pgs=2079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:45.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.046+0000 7f2a86800640 1 -- 172.21.15.152:0/3592942217 >> 172.21.15.152:0/3592942217 conn(0x7f2a80067ae0 msgr2=0x7f2a8006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:45.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.046+0000 7f2a86800640 1 -- 172.21.15.152:0/3592942217 shutdown_connections 2024-04-20T11:36:45.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.047+0000 7f2a86800640 1 -- 172.21.15.152:0/3592942217 wait complete. 2024-04-20T11:36:45.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.048+0000 7f2a86800640 1 Processor -- start 2024-04-20T11:36:45.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.048+0000 7f2a86800640 1 -- start start 2024-04-20T11:36:45.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.049+0000 7f2a86800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2a8006f510 0x7f2a801f8830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:45.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.049+0000 7f2a86800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a8013e530 0x7f2a800b29a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:45.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.049+0000 7f2a86800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a80141310 0x7f2a801f4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:45.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.049+0000 7f2a86800640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f2a800cf8e0 con 0x7f2a8006f510 2024-04-20T11:36:45.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.049+0000 7f2a86800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2a800cfc50 con 0x7f2a8013e530 2024-04-20T11:36:45.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.049+0000 7f2a86800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2a801417a0 con 0x7f2a80141310 2024-04-20T11:36:45.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.050+0000 7f2a85400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2a8006f510 0x7f2a801f8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:45.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.050+0000 7f2a85400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2a8006f510 0x7f2a801f8830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:57766/0 (socket says 172.21.15.152:57766) 2024-04-20T11:36:45.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.050+0000 7f2a85400640 1 -- 172.21.15.152:0/3086273574 learned_addr learned my addr 172.21.15.152:0/3086273574 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:45.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.050+0000 7f2a84a00640 1 --2- 172.21.15.152:0/3086273574 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a8013e530 0x7f2a800b29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:45.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.050+0000 7f2a85400640 1 -- 172.21.15.152:0/3086273574 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a80141310 msgr2=0x7f2a801f4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:45.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.050+0000 7f2a85400640 1 --2- 172.21.15.152:0/3086273574 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a80141310 0x7f2a801f4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:45.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.050+0000 7f2a85400640 1 -- 172.21.15.152:0/3086273574 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a8013e530 msgr2=0x7f2a800b29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:45.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.050+0000 7f2a85400640 1 --2- 172.21.15.152:0/3086273574 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a8013e530 0x7f2a800b29a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:45.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.050+0000 7f2a85400640 1 -- 172.21.15.152:0/3086273574 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a800b1b60 con 0x7f2a8006f510 2024-04-20T11:36:45.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.050+0000 7f2a85400640 1 --2- 172.21.15.152:0/3086273574 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2a8006f510 0x7f2a801f8830 secure :-1 s=READY pgs=2080 cs=0 l=1 rev1=1 crypto rx=0x7f2a7400c430 tx=0x7f2a7400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:45.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.051+0000 7f2a7d600640 1 -- 172.21.15.152:0/3086273574 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a74011790 con 0x7f2a8006f510 2024-04-20T11:36:45.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.051+0000 7f2a7d600640 1 -- 172.21.15.152:0/3086273574 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a74017070 con 0x7f2a8006f510 2024-04-20T11:36:45.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.051+0000 7f2a7d600640 1 -- 172.21.15.152:0/3086273574 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a74011bb0 con 0x7f2a8006f510 2024-04-20T11:36:45.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.051+0000 7f2a86800640 1 -- 172.21.15.152:0/3086273574 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a800b1e40 con 0x7f2a8006f510 2024-04-20T11:36:45.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.051+0000 7f2a86800640 1 -- 172.21.15.152:0/3086273574 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a80141d30 con 0x7f2a8006f510 2024-04-20T11:36:45.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.052+0000 7f2a86800640 1 -- 172.21.15.152:0/3086273574 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a80140920 con 0x7f2a8006f510 2024-04-20T11:36:45.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.053+0000 7f2a7d600640 1 -- 172.21.15.152:0/3086273574 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) ==== 91754+0+0 (secure 0 0 0) 0x7f2a7401c460 con 0x7f2a8006f510 2024-04-20T11:36:45.054 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.054+0000 7f2a7d600640 1 --2- 172.21.15.152:0/3086273574 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f2a6006d0d0 0x7f2a6006f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:45.054 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.054+0000 7f2a7d600640 1 -- 172.21.15.152:0/3086273574 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5709+0+0 (secure 0 0 0) 0x7f2a7408a3c0 con 0x7f2a8006f510 2024-04-20T11:36:45.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.054+0000 7f2a84a00640 1 --2- 172.21.15.152:0/3086273574 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f2a6006d0d0 0x7f2a6006f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:45.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.055+0000 7f2a84a00640 1 --2- 172.21.15.152:0/3086273574 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f2a6006d0d0 0x7f2a6006f590 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c036000 tx=0x7f2a6c03c040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:45.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.058+0000 7f2a7d600640 1 -- 172.21.15.152:0/3086273574 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f2a80140920 con 0x7f2a8006f510 2024-04-20T11:36:45.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.297+0000 7f2a86800640 1 -- 172.21.15.152:0/3086273574 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2a80141f30 con 0x7f2a8006f510 2024-04-20T11:36:45.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.298+0000 7f2a7d600640 1 -- 172.21.15.152:0/3086273574 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 97 v97) ==== 94+0+5209 (secure 0 0 0) 0x7f2a7405a0e0 con 0x7f2a8006f510 2024-04-20T11:36:45.299 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:36:45.299 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":97,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6275,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6834/721066448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":721066448},{"type":"v1","addr":"172.21.15.175:6834","nonce":721066448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":91},{"gid":9483,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6834/3026417634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":3026417634},{"type":"v1","addr":"172.21.15.152:6834","nonce":3026417634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":90},{"gid":9486,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/3399678659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6833","nonce":3399678659},{"type":"v1","addr":"172.21.15.152:6835","nonce":3399678659}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":91}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T11:36:08.919586+0000","modified":"2024-04-20T11:36:12.020458+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9501},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9501":{"gid":9501,"name":"b","rank":0,"incarnation":92,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6835/189289100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6833","nonce":189289100},{"type":"v1","addr":"172.21.15.175:6835","nonce":189289100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9501,"qdb_cluster":[9501]},"id":11}]} 2024-04-20T11:36:45.299 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 97 2024-04-20T11:36:45.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.300+0000 7f2a86800640 1 -- 172.21.15.152:0/3086273574 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f2a6006d0d0 msgr2=0x7f2a6006f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:45.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.300+0000 7f2a86800640 1 --2- 172.21.15.152:0/3086273574 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f2a6006d0d0 0x7f2a6006f590 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c036000 tx=0x7f2a6c03c040 comp rx=0 tx=0).stop 2024-04-20T11:36:45.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.300+0000 7f2a86800640 1 -- 172.21.15.152:0/3086273574 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2a8006f510 msgr2=0x7f2a801f8830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:45.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.300+0000 7f2a86800640 1 --2- 172.21.15.152:0/3086273574 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2a8006f510 0x7f2a801f8830 secure :-1 s=READY pgs=2080 cs=0 l=1 rev1=1 crypto rx=0x7f2a7400c430 tx=0x7f2a7400c8f0 comp rx=0 tx=0).stop 2024-04-20T11:36:45.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.301+0000 7f2a86800640 1 -- 172.21.15.152:0/3086273574 shutdown_connections 2024-04-20T11:36:45.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.301+0000 7f2a86800640 1 --2- 172.21.15.152:0/3086273574 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f2a6006d0d0 0x7f2a6006f590 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:45.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.301+0000 7f2a86800640 1 --2- 172.21.15.152:0/3086273574 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a80141310 0x7f2a801f4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:45.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.301+0000 7f2a86800640 1 --2- 172.21.15.152:0/3086273574 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a8013e530 0x7f2a800b29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:45.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.301+0000 7f2a86800640 1 --2- 172.21.15.152:0/3086273574 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2a8006f510 0x7f2a801f8830 unknown :-1 s=CLOSED pgs=2080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:45.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.301+0000 7f2a86800640 1 -- 172.21.15.152:0/3086273574 >> 172.21.15.152:0/3086273574 conn(0x7f2a80067ae0 msgr2=0x7f2a80072420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:45.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.301+0000 7f2a86800640 1 -- 172.21.15.152:0/3086273574 shutdown_connections 2024-04-20T11:36:45.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.301+0000 7f2a86800640 1 -- 172.21.15.152:0/3086273574 wait complete. 2024-04-20T11:36:45.320 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:36:45.513 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.513+0000 7feed1200640 1 Processor -- start 2024-04-20T11:36:45.514 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.513+0000 7feed1200640 1 -- start start 2024-04-20T11:36:45.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.514+0000 7feed1200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7feecc160b90 0x7feecc15d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:45.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.514+0000 7feed1200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7feecc071cd0 0x7feecc072120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:45.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.514+0000 7feed1200640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feecc072660 con 0x7feecc15d1e0 2024-04-20T11:36:45.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.514+0000 7feed1200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7feecc0ce8c0 con 0x7feecc160b90 2024-04-20T11:36:45.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.514+0000 7feed1200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7feecc1600b0 con 0x7feecc071cd0 2024-04-20T11:36:45.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.515+0000 7feec3e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7feecc160b90 0x7feecc15d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:45.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.515+0000 7feecb400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7feecc15d1e0 0x7feecc0cdfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:55608/0 (socket says 172.21.15.152:55608) 2024-04-20T11:36:45.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.515+0000 7feecb400640 1 -- 172.21.15.152:0/1685142942 learned_addr learned my addr 172.21.15.152:0/1685142942 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:36:45.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.515+0000 7feecbe00640 1 --2- 172.21.15.152:0/1685142942 >> v2:172.21.15.175:3300/0 conn(0x7feecc071cd0 0x7feecc072120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:45.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.515+0000 7feecaa00640 1 -- 172.21.15.152:0/1685142942 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3058695952 0 0) 0x7feebc001110 con 0x7feecc15d1e0 2024-04-20T11:36:45.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.515+0000 7feecaa00640 1 -- 172.21.15.152:0/1685142942 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feeb0003520 con 0x7feecc15d1e0 2024-04-20T11:36:45.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.515+0000 7feec3e00640 1 -- 172.21.15.152:0/1685142942 >> v2:172.21.15.175:3300/0 conn(0x7feecc071cd0 msgr2=0x7feecc072120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:45.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.515+0000 7feec3e00640 1 --2- 172.21.15.152:0/1685142942 >> v2:172.21.15.175:3300/0 conn(0x7feecc071cd0 0x7feecc072120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:45.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.515+0000 7feec3e00640 1 -- 172.21.15.152:0/1685142942 >> v1:172.21.15.152:6789/0 conn(0x7feecc15d1e0 legacy=0x7feecc0cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:45.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.515+0000 7feec3e00640 1 -- 172.21.15.152:0/1685142942 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feecc160390 con 0x7feecc160b90 2024-04-20T11:36:45.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.515+0000 7feecbe00640 1 --2- 172.21.15.152:0/1685142942 >> v2:172.21.15.175:3300/0 conn(0x7feecc071cd0 0x7feecc072120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:36:45.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.516+0000 7feec3e00640 1 --2- 172.21.15.152:0/1685142942 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feecc160b90 0x7feecc15d5c0 secure :-1 s=READY pgs=2082 cs=0 l=1 rev1=1 crypto rx=0x7feeb40097a0 tx=0x7feeb402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c26a6656a536ef6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:45.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.517+0000 7feecaa00640 1 -- 172.21.15.152:0/1685142942 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb4035020 con 0x7feecc160b90 2024-04-20T11:36:45.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.517+0000 7feecaa00640 1 -- 172.21.15.152:0/1685142942 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feeb4031830 con 0x7feecc160b90 2024-04-20T11:36:45.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.517+0000 7feecaa00640 1 -- 172.21.15.152:0/1685142942 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb4031c00 con 0x7feecc160b90 2024-04-20T11:36:45.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.517+0000 7feed1200640 1 -- 172.21.15.152:0/1685142942 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feecc160b90 msgr2=0x7feecc15d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:45.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.517+0000 7feed1200640 1 --2- 172.21.15.152:0/1685142942 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feecc160b90 0x7feecc15d5c0 secure :-1 s=READY pgs=2082 cs=0 l=1 rev1=1 crypto rx=0x7feeb40097a0 tx=0x7feeb402b8d0 comp rx=0 tx=0).stop 2024-04-20T11:36:45.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.518+0000 7feed1200640 1 -- 172.21.15.152:0/1685142942 shutdown_connections 2024-04-20T11:36:45.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.518+0000 7feed1200640 1 --2- 172.21.15.152:0/1685142942 >> v2:172.21.15.175:3300/0 conn(0x7feecc071cd0 0x7feecc072120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:45.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.518+0000 7feed1200640 1 --2- 172.21.15.152:0/1685142942 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feecc160b90 0x7feecc15d5c0 unknown :-1 s=CLOSED pgs=2082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:45.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.518+0000 7feed1200640 1 -- 172.21.15.152:0/1685142942 >> 172.21.15.152:0/1685142942 conn(0x7feecc067ae0 msgr2=0x7feecc06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:45.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.518+0000 7feed1200640 1 -- 172.21.15.152:0/1685142942 shutdown_connections 2024-04-20T11:36:45.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.518+0000 7feed1200640 1 -- 172.21.15.152:0/1685142942 wait complete. 2024-04-20T11:36:45.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.519+0000 7feed1200640 1 Processor -- start 2024-04-20T11:36:45.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.519+0000 7feed1200640 1 -- start start 2024-04-20T11:36:45.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.520+0000 7feed1200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feecc071cd0 0x7feecc1556e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:45.522 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.520+0000 7feed1200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feecc15d1e0 0x7feecc147ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:45.522 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.520+0000 7feed1200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feecc160b90 0x7feecc153730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:45.522 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.521+0000 7feed1200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7feecc0ccd70 con 0x7feecc15d1e0 2024-04-20T11:36:45.522 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.521+0000 7feed1200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7feecc0ceba0 con 0x7feecc071cd0 2024-04-20T11:36:45.522 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.521+0000 7feed1200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7feecc0ccef0 con 0x7feecc160b90 2024-04-20T11:36:45.522 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.521+0000 7feec3e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feecc15d1e0 0x7feecc147ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:45.522 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.521+0000 7feec3e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feecc15d1e0 0x7feecc147ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:57776/0 (socket says 172.21.15.152:57776) 2024-04-20T11:36:45.522 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.521+0000 7feec3e00640 1 -- 172.21.15.152:0/789802029 learned_addr learned my addr 172.21.15.152:0/789802029 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:45.522 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.521+0000 7feecb400640 1 --2- 172.21.15.152:0/789802029 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feecc071cd0 0x7feecc1556e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:45.522 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.521+0000 7feec3e00640 1 -- 172.21.15.152:0/789802029 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feecc160b90 msgr2=0x7feecc153730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:45.522 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.521+0000 7feec3e00640 1 --2- 172.21.15.152:0/789802029 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feecc160b90 0x7feecc153730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:45.522 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.521+0000 7feec3e00640 1 -- 172.21.15.152:0/789802029 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feecc071cd0 msgr2=0x7feecc1556e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:45.522 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.521+0000 7feec3e00640 1 --2- 172.21.15.152:0/789802029 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feecc071cd0 0x7feecc1556e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:45.523 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.521+0000 7feec3e00640 1 -- 172.21.15.152:0/789802029 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feecc155c20 con 0x7feecc15d1e0 2024-04-20T11:36:45.523 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.522+0000 7feec3e00640 1 --2- 172.21.15.152:0/789802029 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feecc15d1e0 0x7feecc147ab0 secure :-1 s=READY pgs=2083 cs=0 l=1 rev1=1 crypto rx=0x7feeb40098c0 tx=0x7feeb40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:45.523 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.522+0000 7feec3400640 1 -- 172.21.15.152:0/789802029 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb4040070 con 0x7feecc15d1e0 2024-04-20T11:36:45.524 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.522+0000 7feec3400640 1 -- 172.21.15.152:0/789802029 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feeb403e070 con 0x7feecc15d1e0 2024-04-20T11:36:45.524 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.522+0000 7feec3400640 1 -- 172.21.15.152:0/789802029 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb4035040 con 0x7feecc15d1e0 2024-04-20T11:36:45.524 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.522+0000 7feed1200640 1 -- 172.21.15.152:0/789802029 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feecc147ff0 con 0x7feecc15d1e0 2024-04-20T11:36:45.524 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.522+0000 7feed1200640 1 -- 172.21.15.152:0/789802029 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feecc156180 con 0x7feecc15d1e0 2024-04-20T11:36:45.524 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.523+0000 7feed1200640 1 -- 172.21.15.152:0/789802029 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feecc075e60 con 0x7feecc15d1e0 2024-04-20T11:36:45.524 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.524+0000 7feec3400640 1 -- 172.21.15.152:0/789802029 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) ==== 91754+0+0 (secure 0 0 0) 0x7feeb4038020 con 0x7feecc15d1e0 2024-04-20T11:36:45.525 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.525+0000 7feec3400640 1 --2- 172.21.15.152:0/789802029 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7feea806d0d0 0x7feea806f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:45.525 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.525+0000 7feec3400640 1 -- 172.21.15.152:0/789802029 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5709+0+0 (secure 0 0 0) 0x7feeb40ae600 con 0x7feecc15d1e0 2024-04-20T11:36:45.526 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.525+0000 7feecb400640 1 --2- 172.21.15.152:0/789802029 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7feea806d0d0 0x7feea806f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:45.527 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.527+0000 7feecb400640 1 --2- 172.21.15.152:0/789802029 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7feea806d0d0 0x7feea806f590 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7feebc0012b0 tx=0x7feebc009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:45.529 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.529+0000 7feec3400640 1 -- 172.21.15.152:0/789802029 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7feecc147ff0 con 0x7feecc15d1e0 2024-04-20T11:36:45.716 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.715+0000 7feed1200640 1 -- 172.21.15.152:0/789802029 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7feecc072840 con 0x7feecc15d1e0 2024-04-20T11:36:45.717 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.717+0000 7feec3400640 1 -- 172.21.15.152:0/789802029 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v142) ==== 74+0+16731 (secure 0 0 0) 0x7feecc155c20 con 0x7feecc15d1e0 2024-04-20T11:36:45.718 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:36:45.718 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":142,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:36:10.484514+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:36:06.702770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"141","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-04-20T11:36:07.911633+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:36:45.721 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.720+0000 7feed1200640 1 -- 172.21.15.152:0/789802029 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7feea806d0d0 msgr2=0x7feea806f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:45.721 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.720+0000 7feed1200640 1 --2- 172.21.15.152:0/789802029 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7feea806d0d0 0x7feea806f590 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7feebc0012b0 tx=0x7feebc009040 comp rx=0 tx=0).stop 2024-04-20T11:36:45.721 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.720+0000 7feed1200640 1 -- 172.21.15.152:0/789802029 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feecc15d1e0 msgr2=0x7feecc147ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:45.721 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.720+0000 7feed1200640 1 --2- 172.21.15.152:0/789802029 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feecc15d1e0 0x7feecc147ab0 secure :-1 s=READY pgs=2083 cs=0 l=1 rev1=1 crypto rx=0x7feeb40098c0 tx=0x7feeb40046c0 comp rx=0 tx=0).stop 2024-04-20T11:36:45.721 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.720+0000 7feed1200640 1 -- 172.21.15.152:0/789802029 shutdown_connections 2024-04-20T11:36:45.722 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.720+0000 7feed1200640 1 --2- 172.21.15.152:0/789802029 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7feea806d0d0 0x7feea806f590 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:45.722 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.720+0000 7feed1200640 1 --2- 172.21.15.152:0/789802029 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feecc160b90 0x7feecc153730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:45.722 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.720+0000 7feed1200640 1 --2- 172.21.15.152:0/789802029 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feecc15d1e0 0x7feecc147ab0 unknown :-1 s=CLOSED pgs=2083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:45.722 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.720+0000 7feed1200640 1 --2- 172.21.15.152:0/789802029 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feecc071cd0 0x7feecc1556e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:45.722 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.720+0000 7feed1200640 1 -- 172.21.15.152:0/789802029 >> 172.21.15.152:0/789802029 conn(0x7feecc067ae0 msgr2=0x7feecc0b1570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:45.722 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.721+0000 7feed1200640 1 -- 172.21.15.152:0/789802029 shutdown_connections 2024-04-20T11:36:45.722 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.721+0000 7feed1200640 1 -- 172.21.15.152:0/789802029 wait complete. 2024-04-20T11:36:45.740 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-20T11:36:45.938 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.937+0000 7f1c5da00640 1 Processor -- start 2024-04-20T11:36:45.938 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.937+0000 7f1c5da00640 1 -- start start 2024-04-20T11:36:45.939 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.939+0000 7f1c5da00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f1c5806fa50 0x7f1c580b3470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:45.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.939+0000 7f1c5da00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f1c5815fd60 0x7f1c580b39b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:45.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.939+0000 7f1c5da00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c580b4130 con 0x7f1c5815b2b0 2024-04-20T11:36:45.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.939+0000 7f1c5da00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f1c580b3ef0 con 0x7f1c5806fa50 2024-04-20T11:36:45.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.939+0000 7f1c5da00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f1c5815f8d0 con 0x7f1c5815fd60 2024-04-20T11:36:45.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.939+0000 7f1c56000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f1c5806fa50 0x7f1c580b3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:45.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.939+0000 7f1c56000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f1c5806fa50 0x7f1c580b3470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:46342/0 (socket says 172.21.15.152:46342) 2024-04-20T11:36:45.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.939+0000 7f1c56000640 1 -- 172.21.15.152:0/3860133813 learned_addr learned my addr 172.21.15.152:0/3860133813 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:45.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.939+0000 7f1c57400640 1 --2- 172.21.15.152:0/3860133813 >> v2:172.21.15.175:3300/0 conn(0x7f1c5815fd60 0x7f1c580b39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:45.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.940+0000 7f1c56000640 1 -- 172.21.15.152:0/3860133813 >> v2:172.21.15.175:3300/0 conn(0x7f1c5815fd60 msgr2=0x7f1c580b39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:45.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.940+0000 7f1c56000640 1 --2- 172.21.15.152:0/3860133813 >> v2:172.21.15.175:3300/0 conn(0x7f1c5815fd60 0x7f1c580b39b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:45.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.940+0000 7f1c56000640 1 -- 172.21.15.152:0/3860133813 >> v1:172.21.15.175:6789/0 conn(0x7f1c5815b2b0 legacy=0x7f1c5806f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:45.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.940+0000 7f1c56000640 1 -- 172.21.15.152:0/3860133813 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c5815f280 con 0x7f1c5806fa50 2024-04-20T11:36:45.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.941+0000 7f1c56000640 1 --2- 172.21.15.152:0/3860133813 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1c5806fa50 0x7f1c580b3470 secure :-1 s=READY pgs=2084 cs=0 l=1 rev1=1 crypto rx=0x7f1c580ccef0 tx=0x7f1c4402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=866d6c5ada15f717 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:45.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.941+0000 7f1c55600640 1 -- 172.21.15.152:0/3860133813 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c44035020 con 0x7f1c5806fa50 2024-04-20T11:36:45.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.941+0000 7f1c55600640 1 -- 172.21.15.152:0/3860133813 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c44031870 con 0x7f1c5806fa50 2024-04-20T11:36:45.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.941+0000 7f1c55600640 1 -- 172.21.15.152:0/3860133813 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c44031c40 con 0x7f1c5806fa50 2024-04-20T11:36:45.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.942+0000 7f1c5da00640 1 -- 172.21.15.152:0/3860133813 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1c5806fa50 msgr2=0x7f1c580b3470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:45.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.942+0000 7f1c5da00640 1 --2- 172.21.15.152:0/3860133813 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1c5806fa50 0x7f1c580b3470 secure :-1 s=READY pgs=2084 cs=0 l=1 rev1=1 crypto rx=0x7f1c580ccef0 tx=0x7f1c4402b910 comp rx=0 tx=0).stop 2024-04-20T11:36:45.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.942+0000 7f1c5da00640 1 -- 172.21.15.152:0/3860133813 shutdown_connections 2024-04-20T11:36:45.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.942+0000 7f1c5da00640 1 --2- 172.21.15.152:0/3860133813 >> v2:172.21.15.175:3300/0 conn(0x7f1c5815fd60 0x7f1c580b39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:45.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.942+0000 7f1c5da00640 1 --2- 172.21.15.152:0/3860133813 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1c5806fa50 0x7f1c580b3470 unknown :-1 s=CLOSED pgs=2084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:45.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.942+0000 7f1c5da00640 1 -- 172.21.15.152:0/3860133813 >> 172.21.15.152:0/3860133813 conn(0x7f1c58067cf0 msgr2=0x7f1c5806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:45.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.942+0000 7f1c5da00640 1 -- 172.21.15.152:0/3860133813 shutdown_connections 2024-04-20T11:36:45.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.942+0000 7f1c5da00640 1 -- 172.21.15.152:0/3860133813 wait complete. 2024-04-20T11:36:45.948 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.944+0000 7f1c5da00640 1 Processor -- start 2024-04-20T11:36:45.948 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.944+0000 7f1c5da00640 1 -- start start 2024-04-20T11:36:45.948 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.946+0000 7f1c5da00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c5806fa50 0x7f1c58152be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:45.948 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.946+0000 7f1c5da00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1c5815b2b0 0x7f1c581513b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:45.948 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.946+0000 7f1c5da00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1c5815fd60 0x7f1c5814b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:45.948 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.946+0000 7f1c5da00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f1c5815b810 con 0x7f1c5815fd60 2024-04-20T11:36:45.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.946+0000 7f1c5da00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1c5815b690 con 0x7f1c5806fa50 2024-04-20T11:36:45.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.946+0000 7f1c5da00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1c5815b990 con 0x7f1c5815b2b0 2024-04-20T11:36:45.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.946+0000 7f1c57400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1c5815fd60 0x7f1c5814b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:45.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.946+0000 7f1c57400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1c5815fd60 0x7f1c5814b0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:46356/0 (socket says 172.21.15.152:46356) 2024-04-20T11:36:45.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.946+0000 7f1c57400640 1 -- 172.21.15.152:0/4148409967 learned_addr learned my addr 172.21.15.152:0/4148409967 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:45.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.946+0000 7f1c56000640 1 --2- 172.21.15.152:0/4148409967 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1c5815b2b0 0x7f1c581513b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:45.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.946+0000 7f1c56a00640 1 --2- 172.21.15.152:0/4148409967 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c5806fa50 0x7f1c58152be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:45.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.946+0000 7f1c57400640 1 -- 172.21.15.152:0/4148409967 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1c5815b2b0 msgr2=0x7f1c581513b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:45.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.946+0000 7f1c57400640 1 --2- 172.21.15.152:0/4148409967 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1c5815b2b0 0x7f1c581513b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:45.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.946+0000 7f1c57400640 1 -- 172.21.15.152:0/4148409967 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c5806fa50 msgr2=0x7f1c58152be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:45.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.946+0000 7f1c57400640 1 --2- 172.21.15.152:0/4148409967 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c5806fa50 0x7f1c58152be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:45.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.946+0000 7f1c57400640 1 -- 172.21.15.152:0/4148409967 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c5814c120 con 0x7f1c5815fd60 2024-04-20T11:36:45.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.946+0000 7f1c56000640 1 --2- 172.21.15.152:0/4148409967 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1c5815b2b0 0x7f1c581513b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:36:45.950 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.947+0000 7f1c57400640 1 --2- 172.21.15.152:0/4148409967 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1c5815fd60 0x7f1c5814b0b0 secure :-1 s=READY pgs=2085 cs=0 l=1 rev1=1 crypto rx=0x7f1c50007e60 tx=0x7f1c5000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:45.950 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.947+0000 7f1c4ea00640 1 -- 172.21.15.152:0/4148409967 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c50014420 con 0x7f1c5815fd60 2024-04-20T11:36:45.950 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.947+0000 7f1c4ea00640 1 -- 172.21.15.152:0/4148409967 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c50019070 con 0x7f1c5815fd60 2024-04-20T11:36:45.950 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.947+0000 7f1c4ea00640 1 -- 172.21.15.152:0/4148409967 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c50014820 con 0x7f1c5815fd60 2024-04-20T11:36:45.950 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.947+0000 7f1c5da00640 1 -- 172.21.15.152:0/4148409967 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c58151b40 con 0x7f1c5815fd60 2024-04-20T11:36:45.950 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.947+0000 7f1c5da00640 1 -- 172.21.15.152:0/4148409967 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c58152100 con 0x7f1c5815fd60 2024-04-20T11:36:45.951 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.948+0000 7f1c5da00640 1 -- 172.21.15.152:0/4148409967 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c580cdaa0 con 0x7f1c5815fd60 2024-04-20T11:36:45.951 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.949+0000 7f1c4ea00640 1 -- 172.21.15.152:0/4148409967 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) ==== 91754+0+0 (secure 0 0 0) 0x7f1c500149c0 con 0x7f1c5815fd60 2024-04-20T11:36:45.951 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.950+0000 7f1c4ea00640 1 --2- 172.21.15.152:0/4148409967 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f1c3806d0d0 0x7f1c3806f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:45.951 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.950+0000 7f1c4ea00640 1 -- 172.21.15.152:0/4148409967 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5709+0+0 (secure 0 0 0) 0x7f1c5008ad90 con 0x7f1c5815fd60 2024-04-20T11:36:45.951 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.951+0000 7f1c56a00640 1 --2- 172.21.15.152:0/4148409967 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f1c3806d0d0 0x7f1c3806f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:45.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.952+0000 7f1c56a00640 1 --2- 172.21.15.152:0/4148409967 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f1c3806d0d0 0x7f1c3806f590 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f1c48003d90 tx=0x7f1c48009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:45.955 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:45.954+0000 7f1c4ea00640 1 -- 172.21.15.152:0/4148409967 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f1c580cdaa0 con 0x7f1c5815fd60 2024-04-20T11:36:46.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:46.177+0000 7f1c5da00640 1 -- 172.21.15.152:0/4148409967 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f1c580689f0 con 0x7f1c5815fd60 2024-04-20T11:36:46.178 INFO:tasks.ceph.mon.a.smithi152.stderr:2024-04-20T11:36:46.178+0000 7f8229800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-20T11:36:46.182 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:46.181+0000 7f1c4ea00640 1 -- 172.21.15.152:0/4148409967 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v98) ==== 161+0+0 (secure 0 0 0) 0x7f1c5005aab0 con 0x7f1c5815fd60 2024-04-20T11:36:46.182 INFO:teuthology.orchestra.run.smithi152.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-20T11:36:46.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:46.184+0000 7f1c5da00640 1 -- 172.21.15.152:0/4148409967 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f1c3806d0d0 msgr2=0x7f1c3806f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:46.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:46.184+0000 7f1c5da00640 1 --2- 172.21.15.152:0/4148409967 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f1c3806d0d0 0x7f1c3806f590 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f1c48003d90 tx=0x7f1c48009040 comp rx=0 tx=0).stop 2024-04-20T11:36:46.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:46.184+0000 7f1c5da00640 1 -- 172.21.15.152:0/4148409967 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1c5815fd60 msgr2=0x7f1c5814b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:46.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:46.184+0000 7f1c5da00640 1 --2- 172.21.15.152:0/4148409967 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1c5815fd60 0x7f1c5814b0b0 secure :-1 s=READY pgs=2085 cs=0 l=1 rev1=1 crypto rx=0x7f1c50007e60 tx=0x7f1c5000e4b0 comp rx=0 tx=0).stop 2024-04-20T11:36:46.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:46.184+0000 7f1c5da00640 1 -- 172.21.15.152:0/4148409967 shutdown_connections 2024-04-20T11:36:46.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:46.184+0000 7f1c5da00640 1 --2- 172.21.15.152:0/4148409967 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f1c3806d0d0 0x7f1c3806f590 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:46.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:46.184+0000 7f1c5da00640 1 --2- 172.21.15.152:0/4148409967 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1c5815fd60 0x7f1c5814b0b0 unknown :-1 s=CLOSED pgs=2085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:46.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:46.184+0000 7f1c5da00640 1 --2- 172.21.15.152:0/4148409967 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1c5815b2b0 0x7f1c581513b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:46.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:46.185+0000 7f1c5da00640 1 --2- 172.21.15.152:0/4148409967 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c5806fa50 0x7f1c58152be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:46.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:46.185+0000 7f1c5da00640 1 -- 172.21.15.152:0/4148409967 >> 172.21.15.152:0/4148409967 conn(0x7f1c58067cf0 msgr2=0x7f1c58072450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:46.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:46.185+0000 7f1c5da00640 1 -- 172.21.15.152:0/4148409967 shutdown_connections 2024-04-20T11:36:46.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:46.185+0000 7f1c5da00640 1 -- 172.21.15.152:0/4148409967 wait complete. 2024-04-20T11:36:46.204 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-20T11:36:46.305 INFO:tasks.ceph.mds.b.smithi175.stdout:starting mds.b at 2024-04-20T11:36:46.395 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:46.394+0000 7fef03200640 1 Processor -- start 2024-04-20T11:36:46.395 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:46.394+0000 7fef03200640 1 -- start start 2024-04-20T11:36:46.397 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:46.396+0000 7fef03200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fef04160220 0x7fef041606b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:46.397 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:46.396+0000 7fef03200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef04141450 0x7fef041418e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:46.397 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:46.396+0000 7fef03200640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef0406fb40 con 0x7fef04071cd0 2024-04-20T11:36:46.397 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:46.396+0000 7fef03200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fef0413f8e0 con 0x7fef04160220 2024-04-20T11:36:46.397 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:46.396+0000 7fef03200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fef04160f90 con 0x7fef04141450 2024-04-20T11:36:46.397 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:46.396+0000 7fef01e00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fef04071cd0 0x7fef040720b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:60288/0 (socket says 172.21.15.152:60288) 2024-04-20T11:36:46.397 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:46.396+0000 7fef01e00640 1 -- 172.21.15.152:0/1206668494 learned_addr learned my addr 172.21.15.152:0/1206668494 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:36:46.399 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:46.397+0000 7fef01400640 1 --2- 172.21.15.152:0/1206668494 >> v2:172.21.15.175:3300/0 conn(0x7fef04160220 0x7fef041606b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:46.399 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:46.397+0000 7fef02800640 1 --2- 172.21.15.152:0/1206668494 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef04141450 0x7fef041418e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:46.399 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:46.397+0000 7fef00a00640 1 -- 172.21.15.152:0/1206668494 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2880490816 0 0) 0x7fef0406fb40 con 0x7fef04071cd0 2024-04-20T11:36:46.399 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:46.397+0000 7fef00a00640 1 -- 172.21.15.152:0/1206668494 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feee8003520 con 0x7fef04071cd0 2024-04-20T11:36:46.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:46.397+0000 7fef00a00640 1 -- 172.21.15.152:0/1206668494 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 910774890 0 0) 0x7feee8003520 con 0x7fef04071cd0 2024-04-20T11:36:46.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:46.397+0000 7fef00a00640 1 -- 172.21.15.152:0/1206668494 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef04141450 msgr2=0x7fef041418e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:46.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:46.397+0000 7fef00a00640 1 --2- 172.21.15.152:0/1206668494 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef04141450 0x7fef041418e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:46.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:46.397+0000 7fef00a00640 1 -- 172.21.15.152:0/1206668494 >> v2:172.21.15.175:3300/0 conn(0x7fef04160220 msgr2=0x7fef041606b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:46.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:46.397+0000 7fef00a00640 1 --2- 172.21.15.152:0/1206668494 >> v2:172.21.15.175:3300/0 conn(0x7fef04160220 0x7fef041606b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:46.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:46.397+0000 7fef00a00640 1 -- 172.21.15.152:0/1206668494 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef0406dd90 con 0x7fef04071cd0 2024-04-20T11:36:46.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:46.397+0000 7fef00a00640 1 -- 172.21.15.152:0/1206668494 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7feef4001670 con 0x7fef04071cd0 2024-04-20T11:36:46.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:46.397+0000 7fef00a00640 1 -- 172.21.15.152:0/1206668494 >> v1:172.21.15.152:6789/0 conn(0x7fef04071cd0 legacy=0x7fef040720b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:46.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:46.397+0000 7fef00a00640 1 --2- 172.21.15.152:0/1206668494 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef04141450 0x7feee80039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:46.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:46.397+0000 7fef00a00640 1 --2- 172.21.15.152:0/1206668494 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feee8003f30 0x7feee8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:46.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:46.397+0000 7fef00a00640 1 --2- 172.21.15.152:0/1206668494 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feee8004d40 0x7feee80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:46.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:46.397+0000 7fef00a00640 1 -- 172.21.15.152:0/1206668494 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fef0413f8e0 con 0x7feee8003f30 2024-04-20T11:36:46.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:46.397+0000 7fef00a00640 1 -- 172.21.15.152:0/1206668494 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fef04160f90 con 0x7feee8004d40 2024-04-20T11:36:46.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:46.397+0000 7fef00a00640 1 -- 172.21.15.152:0/1206668494 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7feee800ba10 con 0x7fef04141450 2024-04-20T11:36:46.401 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:46.398+0000 7fef01400640 1 --2- 172.21.15.152:0/1206668494 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feee8003f30 0x7feee8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:46.401 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:46.398+0000 7fef01e00640 1 --2- 172.21.15.152:0/1206668494 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef04141450 0x7feee80039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:46.401 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:46.398+0000 7fef01400640 1 -- 172.21.15.152:0/1206668494 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef04141450 msgr2=0x7feee80039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:46.401 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:46.398+0000 7fef01400640 1 --2- 172.21.15.152:0/1206668494 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef04141450 0x7feee80039b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:46.401 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:46.398+0000 7fef01400640 1 -- 172.21.15.152:0/1206668494 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feee8004d40 msgr2=0x7feee80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:46.401 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:46.398+0000 7fef01400640 1 --2- 172.21.15.152:0/1206668494 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feee8004d40 0x7feee80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:46.401 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:46.398+0000 7fef01400640 1 -- 172.21.15.152:0/1206668494 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feee8003520 con 0x7feee8003f30 2024-04-20T11:36:46.401 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:46.399+0000 7fef01400640 1 --2- 172.21.15.152:0/1206668494 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feee8003f30 0x7feee8004800 secure :-1 s=READY pgs=2087 cs=0 l=1 rev1=1 crypto rx=0x7feef0009af0 tx=0x7feee8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40267c803cd6c566 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:46.402 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:46.399+0000 7fef00a00640 1 -- 172.21.15.152:0/1206668494 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef0039070 con 0x7feee8003f30 2024-04-20T11:36:46.402 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:46.399+0000 7fef00a00640 1 -- 172.21.15.152:0/1206668494 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feef002c710 con 0x7feee8003f30 2024-04-20T11:36:46.402 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:46.399+0000 7fef00a00640 1 -- 172.21.15.152:0/1206668494 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef0009690 con 0x7feee8003f30 2024-04-20T11:36:46.403 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:46.400+0000 7fef03200640 1 -- 172.21.15.152:0/1206668494 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feee8003f30 msgr2=0x7feee8004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:46.403 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:46.400+0000 7fef03200640 1 --2- 172.21.15.152:0/1206668494 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feee8003f30 0x7feee8004800 secure :-1 s=READY pgs=2087 cs=0 l=1 rev1=1 crypto rx=0x7feef0009af0 tx=0x7feee8005830 comp rx=0 tx=0).stop 2024-04-20T11:36:46.403 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:46.400+0000 7fef02800640 1 -- 172.21.15.152:0/1206668494 reap_dead start 2024-04-20T11:36:46.403 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:46.400+0000 7fef03200640 1 -- 172.21.15.152:0/1206668494 shutdown_connections 2024-04-20T11:36:46.403 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:46.400+0000 7fef03200640 1 -- 172.21.15.152:0/1206668494 >> 172.21.15.152:0/1206668494 conn(0x7fef04067ae0 msgr2=0x7fef0406b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:46.403 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:46.400+0000 7fef03200640 1 -- 172.21.15.152:0/1206668494 shutdown_connections 2024-04-20T11:36:46.403 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:46.400+0000 7fef03200640 1 -- 172.21.15.152:0/1206668494 wait complete. 2024-04-20T11:36:46.403 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:46.402+0000 7fef03200640 1 Processor -- start 2024-04-20T11:36:46.403 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:46.402+0000 7fef03200640 1 -- start start 2024-04-20T11:36:46.405 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:46.403+0000 7fef03200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feee8003f30 0x7fef041511b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:46.405 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:46.403+0000 7fef03200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef041e7000 0x7fef041e7480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:46.405 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:46.403+0000 7fef03200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef041f2100 0x7fef041f2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:46.405 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:46.403+0000 7fef03200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fef0406e630 con 0x7feee8003f30 2024-04-20T11:36:46.405 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:46.403+0000 7fef03200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fef0406e350 con 0x7fef041f2100 2024-04-20T11:36:46.405 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:46.403+0000 7fef03200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fef04160bf0 con 0x7fef041e7000 2024-04-20T11:36:46.405 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:46.403+0000 7fef01e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feee8003f30 0x7fef041511b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:46.406 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:46.403+0000 7fef01e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feee8003f30 0x7fef041511b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:46374/0 (socket says 172.21.15.152:46374) 2024-04-20T11:36:46.406 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:46.403+0000 7fef01e00640 1 -- 172.21.15.152:0/3083486297 learned_addr learned my addr 172.21.15.152:0/3083486297 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:46.406 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:46.403+0000 7fef01e00640 1 -- 172.21.15.152:0/3083486297 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef041e7000 msgr2=0x7fef041e7480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:46.406 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:46.403+0000 7fef01e00640 1 --2- 172.21.15.152:0/3083486297 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef041e7000 0x7fef041e7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:46.406 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:46.403+0000 7fef01e00640 1 -- 172.21.15.152:0/3083486297 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef041f2100 msgr2=0x7fef041f2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:46.406 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:46.403+0000 7fef01e00640 1 --2- 172.21.15.152:0/3083486297 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef041f2100 0x7fef041f2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:46.406 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:46.403+0000 7fef01e00640 1 -- 172.21.15.152:0/3083486297 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef041f2ac0 con 0x7feee8003f30 2024-04-20T11:36:46.406 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:46.404+0000 7fef01e00640 1 --2- 172.21.15.152:0/3083486297 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feee8003f30 0x7fef041511b0 secure :-1 s=READY pgs=2088 cs=0 l=1 rev1=1 crypto rx=0x7feef400d730 tx=0x7feef400dbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:46.406 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:46.404+0000 7feefa000640 1 -- 172.21.15.152:0/3083486297 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef4019070 con 0x7feee8003f30 2024-04-20T11:36:46.406 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:46.404+0000 7feefa000640 1 -- 172.21.15.152:0/3083486297 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feef40074e0 con 0x7feee8003f30 2024-04-20T11:36:46.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:46.404+0000 7feefa000640 1 -- 172.21.15.152:0/3083486297 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef40049e0 con 0x7feee8003f30 2024-04-20T11:36:46.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:46.404+0000 7fef03200640 1 -- 172.21.15.152:0/3083486297 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef041f34d0 con 0x7feee8003f30 2024-04-20T11:36:46.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:46.404+0000 7fef03200640 1 -- 172.21.15.152:0/3083486297 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef041f3a90 con 0x7feee8003f30 2024-04-20T11:36:46.408 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:46.405+0000 7fef03200640 1 -- 172.21.15.152:0/3083486297 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef04075e60 con 0x7feee8003f30 2024-04-20T11:36:46.408 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:46.406+0000 7feefa000640 1 -- 172.21.15.152:0/3083486297 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) ==== 91754+0+0 (secure 0 0 0) 0x7feef4004b80 con 0x7feee8003f30 2024-04-20T11:36:46.409 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:46.407+0000 7feefa000640 1 --2- 172.21.15.152:0/3083486297 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7feee406d100 0x7feee406f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:46.409 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:46.407+0000 7feefa000640 1 -- 172.21.15.152:0/3083486297 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5795+0+0 (secure 0 0 0) 0x7feef4089e60 con 0x7feee8003f30 2024-04-20T11:36:46.409 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:46.408+0000 7fef01400640 1 --2- 172.21.15.152:0/3083486297 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7feee406d100 0x7feee406f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:46.409 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:46.409+0000 7fef01400640 1 --2- 172.21.15.152:0/3083486297 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7feee406d100 0x7feee406f5c0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7feef0036000 tx=0x7feef002e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:46.412 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:46.412+0000 7feefa000640 1 -- 172.21.15.152:0/3083486297 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7feef4059b80 con 0x7feee8003f30 2024-04-20T11:36:46.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:46.642+0000 7fef03200640 1 -- 172.21.15.152:0/3083486297 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fef04072d30 con 0x7feee8003f30 2024-04-20T11:36:47.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:47.192+0000 7feefa000640 1 -- 172.21.15.152:0/3083486297 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v99) ==== 104+0+0 (secure 0 0 0) 0x7feef4059480 con 0x7feee8003f30 2024-04-20T11:36:47.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:47.195+0000 7fef03200640 1 -- 172.21.15.152:0/3083486297 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7feee406d100 msgr2=0x7feee406f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:47.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:47.195+0000 7fef03200640 1 --2- 172.21.15.152:0/3083486297 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7feee406d100 0x7feee406f5c0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7feef0036000 tx=0x7feef002e000 comp rx=0 tx=0).stop 2024-04-20T11:36:47.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:47.195+0000 7fef03200640 1 -- 172.21.15.152:0/3083486297 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feee8003f30 msgr2=0x7fef041511b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:47.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:47.195+0000 7fef03200640 1 --2- 172.21.15.152:0/3083486297 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feee8003f30 0x7fef041511b0 secure :-1 s=READY pgs=2088 cs=0 l=1 rev1=1 crypto rx=0x7feef400d730 tx=0x7feef400dbf0 comp rx=0 tx=0).stop 2024-04-20T11:36:47.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:47.195+0000 7fef03200640 1 -- 172.21.15.152:0/3083486297 shutdown_connections 2024-04-20T11:36:47.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:47.195+0000 7fef03200640 1 --2- 172.21.15.152:0/3083486297 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7feee406d100 0x7feee406f5c0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:47.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:47.195+0000 7fef03200640 1 --2- 172.21.15.152:0/3083486297 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef041f2100 0x7fef041f2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:47.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:47.195+0000 7fef03200640 1 --2- 172.21.15.152:0/3083486297 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef041e7000 0x7fef041e7480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:47.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:47.195+0000 7fef03200640 1 --2- 172.21.15.152:0/3083486297 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feee8003f30 0x7fef041511b0 unknown :-1 s=CLOSED pgs=2088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:47.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:47.195+0000 7fef03200640 1 -- 172.21.15.152:0/3083486297 >> 172.21.15.152:0/3083486297 conn(0x7fef04067ae0 msgr2=0x7fef0415e910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:47.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:47.195+0000 7fef03200640 1 -- 172.21.15.152:0/3083486297 shutdown_connections 2024-04-20T11:36:47.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:47.195+0000 7fef03200640 1 -- 172.21.15.152:0/3083486297 wait complete. 2024-04-20T11:36:47.217 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-20T11:36:47.423 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:47.423+0000 7f16e3e00640 1 Processor -- start 2024-04-20T11:36:47.424 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:47.423+0000 7f16e3e00640 1 -- start start 2024-04-20T11:36:47.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:47.424+0000 7f16e3e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f16e415ae50 0x7f16e415f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:47.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:47.425+0000 7f16e3e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f16e40cd2c0 0x7f16e40b3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:47.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:47.425+0000 7f16e3e00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16e415f820 con 0x7f16e4073790 2024-04-20T11:36:47.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:47.425+0000 7f16e3e00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f16e415b230 con 0x7f16e40cd2c0 2024-04-20T11:36:47.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:47.425+0000 7f16e3e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f16e4072360 con 0x7f16e415ae50 2024-04-20T11:36:47.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:47.425+0000 7f16dbe00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f16e4073790 0x7f16e40ccbb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:60298/0 (socket says 172.21.15.152:60298) 2024-04-20T11:36:47.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:47.425+0000 7f16dbe00640 1 -- 172.21.15.152:0/903627176 learned_addr learned my addr 172.21.15.152:0/903627176 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:36:47.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:47.425+0000 7f16e3400640 1 --2- 172.21.15.152:0/903627176 >> v2:172.21.15.152:3300/0 conn(0x7f16e40cd2c0 0x7f16e40b3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:47.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:47.425+0000 7f16e2a00640 1 --2- 172.21.15.152:0/903627176 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f16e415ae50 0x7f16e415f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:47.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:47.425+0000 7f16e2000640 1 -- 172.21.15.152:0/903627176 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3747588867 0 0) 0x7f16e415f820 con 0x7f16e4073790 2024-04-20T11:36:47.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:47.426+0000 7f16e2000640 1 -- 172.21.15.152:0/903627176 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f16c8003520 con 0x7f16e4073790 2024-04-20T11:36:47.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:47.426+0000 7f16e3400640 1 -- 172.21.15.152:0/903627176 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f16e415ae50 msgr2=0x7f16e415f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:47.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:47.426+0000 7f16e3400640 1 --2- 172.21.15.152:0/903627176 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f16e415ae50 0x7f16e415f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:47.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:47.426+0000 7f16e3400640 1 -- 172.21.15.152:0/903627176 >> v1:172.21.15.152:6789/0 conn(0x7f16e4073790 legacy=0x7f16e40ccbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:47.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:47.426+0000 7f16e3400640 1 -- 172.21.15.152:0/903627176 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16e4073c70 con 0x7f16e40cd2c0 2024-04-20T11:36:47.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:47.427+0000 7f16e3400640 1 --2- 172.21.15.152:0/903627176 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f16e40cd2c0 0x7f16e40b3430 secure :-1 s=READY pgs=2090 cs=0 l=1 rev1=1 crypto rx=0x7f16dc009d90 tx=0x7f16dc02be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=131ac4c4386dc38d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:47.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:47.427+0000 7f16e2000640 1 -- 172.21.15.152:0/903627176 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16dc039070 con 0x7f16e40cd2c0 2024-04-20T11:36:47.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:47.427+0000 7f16e2000640 1 -- 172.21.15.152:0/903627176 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16dc02cbc0 con 0x7f16e40cd2c0 2024-04-20T11:36:47.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:47.427+0000 7f16e2000640 1 -- 172.21.15.152:0/903627176 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16dc034440 con 0x7f16e40cd2c0 2024-04-20T11:36:47.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:47.428+0000 7f16e3e00640 1 -- 172.21.15.152:0/903627176 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f16e40cd2c0 msgr2=0x7f16e40b3430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:47.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:47.428+0000 7f16e3e00640 1 --2- 172.21.15.152:0/903627176 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f16e40cd2c0 0x7f16e40b3430 secure :-1 s=READY pgs=2090 cs=0 l=1 rev1=1 crypto rx=0x7f16dc009d90 tx=0x7f16dc02be10 comp rx=0 tx=0).stop 2024-04-20T11:36:47.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:47.428+0000 7f16e3e00640 1 -- 172.21.15.152:0/903627176 shutdown_connections 2024-04-20T11:36:47.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:47.428+0000 7f16e3e00640 1 --2- 172.21.15.152:0/903627176 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f16e40cd2c0 0x7f16e40b3430 unknown :-1 s=CLOSED pgs=2090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:47.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:47.428+0000 7f16e3e00640 1 --2- 172.21.15.152:0/903627176 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f16e415ae50 0x7f16e415f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:47.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:47.428+0000 7f16e3e00640 1 -- 172.21.15.152:0/903627176 >> 172.21.15.152:0/903627176 conn(0x7f16e406bda0 msgr2=0x7f16e406c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:47.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:47.428+0000 7f16e3e00640 1 -- 172.21.15.152:0/903627176 shutdown_connections 2024-04-20T11:36:47.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:47.428+0000 7f16e3e00640 1 -- 172.21.15.152:0/903627176 wait complete. 2024-04-20T11:36:47.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:47.429+0000 7f16e3e00640 1 Processor -- start 2024-04-20T11:36:47.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:47.430+0000 7f16e3e00640 1 -- start start 2024-04-20T11:36:47.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:47.430+0000 7f16e3e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f16e4073790 0x7f16e414a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:47.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:47.431+0000 7f16e3e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f16e40cd2c0 0x7f16e4149c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:47.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:47.431+0000 7f16e3e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f16e415ae50 0x7f16e40c93d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:47.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:47.431+0000 7f16e3e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f16e40b4f10 con 0x7f16e4073790 2024-04-20T11:36:47.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:47.431+0000 7f16e3e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f16e4071ff0 con 0x7f16e40cd2c0 2024-04-20T11:36:47.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:47.431+0000 7f16e3e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f16e406fb80 con 0x7f16e415ae50 2024-04-20T11:36:47.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:47.431+0000 7f16e2a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f16e4073790 0x7f16e414a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:47.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:47.431+0000 7f16e2a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f16e4073790 0x7f16e414a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:46390/0 (socket says 172.21.15.152:46390) 2024-04-20T11:36:47.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:47.431+0000 7f16e2a00640 1 -- 172.21.15.152:0/330732794 learned_addr learned my addr 172.21.15.152:0/330732794 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:47.433 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:47.431+0000 7f16e2a00640 1 -- 172.21.15.152:0/330732794 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f16e415ae50 msgr2=0x7f16e40c93d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:47.433 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:47.431+0000 7f16e2a00640 1 --2- 172.21.15.152:0/330732794 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f16e415ae50 0x7f16e40c93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:47.433 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:47.431+0000 7f16e2a00640 1 -- 172.21.15.152:0/330732794 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f16e40cd2c0 msgr2=0x7f16e4149c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:47.433 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:47.431+0000 7f16e2a00640 1 --2- 172.21.15.152:0/330732794 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f16e40cd2c0 0x7f16e4149c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:47.433 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:47.431+0000 7f16e2a00640 1 -- 172.21.15.152:0/330732794 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16e40c9910 con 0x7f16e4073790 2024-04-20T11:36:47.433 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:47.432+0000 7f16e2a00640 1 --2- 172.21.15.152:0/330732794 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f16e4073790 0x7f16e414a1b0 secure :-1 s=READY pgs=2091 cs=0 l=1 rev1=1 crypto rx=0x7f16d400e8a0 tx=0x7f16d400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:47.433 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:47.432+0000 7f16daa00640 1 -- 172.21.15.152:0/330732794 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16d4017070 con 0x7f16e4073790 2024-04-20T11:36:47.433 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:47.432+0000 7f16daa00640 1 -- 172.21.15.152:0/330732794 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16d400c940 con 0x7f16e4073790 2024-04-20T11:36:47.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:47.432+0000 7f16daa00640 1 -- 172.21.15.152:0/330732794 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16d400cd30 con 0x7f16e4073790 2024-04-20T11:36:47.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:47.432+0000 7f16e3e00640 1 -- 172.21.15.152:0/330732794 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16e415c630 con 0x7f16e4073790 2024-04-20T11:36:47.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:47.432+0000 7f16e3e00640 1 -- 172.21.15.152:0/330732794 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16e406f8a0 con 0x7f16e4073790 2024-04-20T11:36:47.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:47.433+0000 7f16e3e00640 1 -- 172.21.15.152:0/330732794 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16e4072510 con 0x7f16e4073790 2024-04-20T11:36:47.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:47.434+0000 7f16daa00640 1 -- 172.21.15.152:0/330732794 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) ==== 91754+0+0 (secure 0 0 0) 0x7f16d401a070 con 0x7f16e4073790 2024-04-20T11:36:47.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:47.435+0000 7f16daa00640 1 --2- 172.21.15.152:0/330732794 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f16c006d0d0 0x7f16c006f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:47.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:47.435+0000 7f16daa00640 1 -- 172.21.15.152:0/330732794 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5795+0+0 (secure 0 0 0) 0x7f16d408a410 con 0x7f16e4073790 2024-04-20T11:36:47.436 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:47.436+0000 7f16dbe00640 1 --2- 172.21.15.152:0/330732794 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f16c006d0d0 0x7f16c006f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:47.437 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:47.437+0000 7f16dbe00640 1 --2- 172.21.15.152:0/330732794 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f16c006d0d0 0x7f16c006f590 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f16cc000f60 tx=0x7f16cc0078f0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:47.440 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:47.439+0000 7f16daa00640 1 -- 172.21.15.152:0/330732794 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f16e4072510 con 0x7f16e4073790 2024-04-20T11:36:47.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:47.640+0000 7f16e3e00640 1 -- 172.21.15.152:0/330732794 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f16e40689f0 con 0x7f16e4073790 2024-04-20T11:36:48.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:48.195+0000 7f16daa00640 1 -- 172.21.15.152:0/330732794 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v144) ==== 181+0+0 (secure 0 0 0) 0x7f16d405a130 con 0x7f16e4073790 2024-04-20T11:36:48.196 INFO:teuthology.orchestra.run.smithi152.stderr:pool 'cephfs_metadata' removed 2024-04-20T11:36:48.199 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:48.199+0000 7f16e3e00640 1 -- 172.21.15.152:0/330732794 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f16c006d0d0 msgr2=0x7f16c006f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:48.199 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:48.199+0000 7f16e3e00640 1 --2- 172.21.15.152:0/330732794 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f16c006d0d0 0x7f16c006f590 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f16cc000f60 tx=0x7f16cc0078f0 comp rx=0 tx=0).stop 2024-04-20T11:36:48.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:48.199+0000 7f16e3e00640 1 -- 172.21.15.152:0/330732794 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f16e4073790 msgr2=0x7f16e414a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:48.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:48.199+0000 7f16e3e00640 1 --2- 172.21.15.152:0/330732794 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f16e4073790 0x7f16e414a1b0 secure :-1 s=READY pgs=2091 cs=0 l=1 rev1=1 crypto rx=0x7f16d400e8a0 tx=0x7f16d400ed60 comp rx=0 tx=0).stop 2024-04-20T11:36:48.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:48.200+0000 7f16e3e00640 1 -- 172.21.15.152:0/330732794 shutdown_connections 2024-04-20T11:36:48.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:48.200+0000 7f16e3e00640 1 --2- 172.21.15.152:0/330732794 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f16c006d0d0 0x7f16c006f590 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:48.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:48.200+0000 7f16e3e00640 1 --2- 172.21.15.152:0/330732794 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f16e415ae50 0x7f16e40c93d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:48.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:48.200+0000 7f16e3e00640 1 --2- 172.21.15.152:0/330732794 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f16e40cd2c0 0x7f16e4149c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:48.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:48.200+0000 7f16e3e00640 1 --2- 172.21.15.152:0/330732794 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f16e4073790 0x7f16e414a1b0 unknown :-1 s=CLOSED pgs=2091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:48.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:48.200+0000 7f16e3e00640 1 -- 172.21.15.152:0/330732794 >> 172.21.15.152:0/330732794 conn(0x7f16e406bda0 msgr2=0x7f16e40b3430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:48.201 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:48.201+0000 7f16e3e00640 1 -- 172.21.15.152:0/330732794 shutdown_connections 2024-04-20T11:36:48.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:48.202+0000 7f16e3e00640 1 -- 172.21.15.152:0/330732794 wait complete. 2024-04-20T11:36:48.233 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-20T11:36:48.418 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:48.417+0000 7f7ad3400640 1 Processor -- start 2024-04-20T11:36:48.418 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:48.418+0000 7f7ad3400640 1 -- start start 2024-04-20T11:36:48.420 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:48.419+0000 7f7ad3400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f7acc0cd8e0 0x7f7acc06e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:48.420 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:48.419+0000 7f7ad3400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7acc0b4d90 0x7f7acc15f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:48.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:48.419+0000 7f7ad3400640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7acc06eab0 con 0x7f7acc0cf220 2024-04-20T11:36:48.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:48.419+0000 7f7ad3400640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f7acc0d0240 con 0x7f7acc0cd8e0 2024-04-20T11:36:48.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:48.419+0000 7f7ad3400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7acc0cf6c0 con 0x7f7acc0b4d90 2024-04-20T11:36:48.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:48.419+0000 7f7ad0a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f7acc0cd8e0 0x7f7acc06e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:48.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:48.419+0000 7f7ad0a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f7acc0cd8e0 0x7f7acc06e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:46406/0 (socket says 172.21.15.152:46406) 2024-04-20T11:36:48.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:48.419+0000 7f7ad0a00640 1 -- 172.21.15.152:0/1328605221 learned_addr learned my addr 172.21.15.152:0/1328605221 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:48.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:48.420+0000 7f7ad1400640 1 --2- 172.21.15.152:0/1328605221 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7acc0b4d90 0x7f7acc15f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:48.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:48.420+0000 7f7ad0a00640 1 -- 172.21.15.152:0/1328605221 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7acc0b4d90 msgr2=0x7f7acc15f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:48.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:48.420+0000 7f7ad0a00640 1 --2- 172.21.15.152:0/1328605221 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7acc0b4d90 0x7f7acc15f410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:48.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:48.420+0000 7f7ad0a00640 1 -- 172.21.15.152:0/1328605221 >> v1:172.21.15.175:6789/0 conn(0x7f7acc0cf220 legacy=0x7f7acc0b4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:48.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:48.420+0000 7f7ad0a00640 1 -- 172.21.15.152:0/1328605221 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7acc0cf9a0 con 0x7f7acc0cd8e0 2024-04-20T11:36:48.423 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:48.421+0000 7f7ad0a00640 1 --2- 172.21.15.152:0/1328605221 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7acc0cd8e0 0x7f7acc06e570 secure :-1 s=READY pgs=2092 cs=0 l=1 rev1=1 crypto rx=0x7f7acc073680 tx=0x7f7ac002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e9a6c3f3e08de6f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:48.423 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:48.421+0000 7f7acb400640 1 -- 172.21.15.152:0/1328605221 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ac00317c0 con 0x7f7acc0cd8e0 2024-04-20T11:36:48.423 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:48.421+0000 7f7acb400640 1 -- 172.21.15.152:0/1328605221 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ac0037070 con 0x7f7acc0cd8e0 2024-04-20T11:36:48.423 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:48.421+0000 7f7acb400640 1 -- 172.21.15.152:0/1328605221 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ac0031b80 con 0x7f7acc0cd8e0 2024-04-20T11:36:48.423 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:48.422+0000 7f7ad3400640 1 -- 172.21.15.152:0/1328605221 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7acc0cd8e0 msgr2=0x7f7acc06e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:48.423 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:48.422+0000 7f7ad3400640 1 --2- 172.21.15.152:0/1328605221 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7acc0cd8e0 0x7f7acc06e570 secure :-1 s=READY pgs=2092 cs=0 l=1 rev1=1 crypto rx=0x7f7acc073680 tx=0x7f7ac002b860 comp rx=0 tx=0).stop 2024-04-20T11:36:48.423 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:48.422+0000 7f7ad3400640 1 -- 172.21.15.152:0/1328605221 shutdown_connections 2024-04-20T11:36:48.423 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:48.422+0000 7f7ad3400640 1 --2- 172.21.15.152:0/1328605221 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7acc0b4d90 0x7f7acc15f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:48.423 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:48.422+0000 7f7ad3400640 1 --2- 172.21.15.152:0/1328605221 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7acc0cd8e0 0x7f7acc06e570 unknown :-1 s=CLOSED pgs=2092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:48.423 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:48.422+0000 7f7ad3400640 1 -- 172.21.15.152:0/1328605221 >> 172.21.15.152:0/1328605221 conn(0x7f7acc06b980 msgr2=0x7f7acc06bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:48.423 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:48.422+0000 7f7ad3400640 1 -- 172.21.15.152:0/1328605221 shutdown_connections 2024-04-20T11:36:48.424 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:48.422+0000 7f7ad3400640 1 -- 172.21.15.152:0/1328605221 wait complete. 2024-04-20T11:36:48.424 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:48.424+0000 7f7ad3400640 1 Processor -- start 2024-04-20T11:36:48.424 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:48.424+0000 7f7ad3400640 1 -- start start 2024-04-20T11:36:48.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:48.425+0000 7f7ad3400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7acc0b4d90 0x7f7acc0cade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:48.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:48.425+0000 7f7ad3400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7acc0cd8e0 0x7f7acc0cb320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:48.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:48.425+0000 7f7ad3400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7acc0cf220 0x7f7acc1e88c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:48.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:48.425+0000 7f7ad3400640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f7acc0cdcc0 con 0x7f7acc0cf220 2024-04-20T11:36:48.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:48.425+0000 7f7ad3400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7acc0cff60 con 0x7f7acc0cd8e0 2024-04-20T11:36:48.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:48.425+0000 7f7ad3400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7acc144d70 con 0x7f7acc0b4d90 2024-04-20T11:36:48.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:48.425+0000 7f7ad1400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7acc0cf220 0x7f7acc1e88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:48.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:48.425+0000 7f7ad1400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7acc0cf220 0x7f7acc1e88c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:46416/0 (socket says 172.21.15.152:46416) 2024-04-20T11:36:48.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:48.425+0000 7f7ad1400640 1 -- 172.21.15.152:0/3376913063 learned_addr learned my addr 172.21.15.152:0/3376913063 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:48.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:48.425+0000 7f7acbe00640 1 --2- 172.21.15.152:0/3376913063 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7acc0cd8e0 0x7f7acc0cb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:48.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:48.425+0000 7f7ad0a00640 1 --2- 172.21.15.152:0/3376913063 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7acc0b4d90 0x7f7acc0cade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:48.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:48.426+0000 7f7ad1400640 1 -- 172.21.15.152:0/3376913063 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7acc0b4d90 msgr2=0x7f7acc0cade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:48.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:48.426+0000 7f7ad1400640 1 --2- 172.21.15.152:0/3376913063 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7acc0b4d90 0x7f7acc0cade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:48.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:48.426+0000 7f7ad1400640 1 -- 172.21.15.152:0/3376913063 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7acc0cd8e0 msgr2=0x7f7acc0cb320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:48.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:48.426+0000 7f7ad1400640 1 --2- 172.21.15.152:0/3376913063 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7acc0cd8e0 0x7f7acc0cb320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:48.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:48.426+0000 7f7ad1400640 1 -- 172.21.15.152:0/3376913063 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7acc1e7160 con 0x7f7acc0cf220 2024-04-20T11:36:48.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:48.426+0000 7f7ad0a00640 1 --2- 172.21.15.152:0/3376913063 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7acc0b4d90 0x7f7acc0cade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:36:48.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:48.426+0000 7f7ad1400640 1 --2- 172.21.15.152:0/3376913063 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7acc0cf220 0x7f7acc1e88c0 secure :-1 s=READY pgs=2093 cs=0 l=1 rev1=1 crypto rx=0x7f7ac400cc30 tx=0x7f7ac400a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:48.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:48.426+0000 7f7ac8c00640 1 -- 172.21.15.152:0/3376913063 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ac400ce40 con 0x7f7acc0cf220 2024-04-20T11:36:48.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:48.426+0000 7f7ac8c00640 1 -- 172.21.15.152:0/3376913063 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ac4003990 con 0x7f7acc0cf220 2024-04-20T11:36:48.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:48.426+0000 7f7ac8c00640 1 -- 172.21.15.152:0/3376913063 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ac4003d40 con 0x7f7acc0cf220 2024-04-20T11:36:48.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:48.426+0000 7f7ad3400640 1 -- 172.21.15.152:0/3376913063 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7acc1e7de0 con 0x7f7acc0cf220 2024-04-20T11:36:48.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:48.426+0000 7f7ad3400640 1 -- 172.21.15.152:0/3376913063 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7acc1e83a0 con 0x7f7acc0cf220 2024-04-20T11:36:48.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:48.427+0000 7f7ad3400640 1 -- 172.21.15.152:0/3376913063 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7acc068a60 con 0x7f7acc0cf220 2024-04-20T11:36:48.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:48.428+0000 7f7ac8c00640 1 -- 172.21.15.152:0/3376913063 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) ==== 91754+0+0 (secure 0 0 0) 0x7f7ac4003ee0 con 0x7f7acc0cf220 2024-04-20T11:36:48.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:48.429+0000 7f7ac8c00640 1 --2- 172.21.15.152:0/3376913063 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f7aac06d100 0x7f7aac06f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:48.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:48.429+0000 7f7ac8c00640 1 -- 172.21.15.152:0/3376913063 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5357+0+0 (secure 0 0 0) 0x7f7ac408d2b0 con 0x7f7acc0cf220 2024-04-20T11:36:48.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:48.430+0000 7f7ad0a00640 1 --2- 172.21.15.152:0/3376913063 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f7aac06d100 0x7f7aac06f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:48.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:48.432+0000 7f7ad0a00640 1 --2- 172.21.15.152:0/3376913063 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f7aac06d100 0x7f7aac06f5c0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f7ac0036000 tx=0x7f7ac0008030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:48.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:48.434+0000 7f7ac8c00640 1 -- 172.21.15.152:0/3376913063 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f7ac4014030 con 0x7f7acc0cf220 2024-04-20T11:36:48.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:48.636+0000 7f7ad3400640 1 -- 172.21.15.152:0/3376913063 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f7acc1e8570 con 0x7f7acc0cf220 2024-04-20T11:36:49.199 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.198+0000 7f7ac8c00640 1 -- 172.21.15.152:0/3376913063 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v145) ==== 169+0+0 (secure 0 0 0) 0x7f7ac405a710 con 0x7f7acc0cf220 2024-04-20T11:36:49.199 INFO:teuthology.orchestra.run.smithi152.stderr:pool 'cephfs_data' removed 2024-04-20T11:36:49.204 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.203+0000 7f7ad3400640 1 -- 172.21.15.152:0/3376913063 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f7aac06d100 msgr2=0x7f7aac06f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:49.204 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.203+0000 7f7ad3400640 1 --2- 172.21.15.152:0/3376913063 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f7aac06d100 0x7f7aac06f5c0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f7ac0036000 tx=0x7f7ac0008030 comp rx=0 tx=0).stop 2024-04-20T11:36:49.204 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.203+0000 7f7ad3400640 1 -- 172.21.15.152:0/3376913063 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7acc0cf220 msgr2=0x7f7acc1e88c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:49.204 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.203+0000 7f7ad3400640 1 --2- 172.21.15.152:0/3376913063 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7acc0cf220 0x7f7acc1e88c0 secure :-1 s=READY pgs=2093 cs=0 l=1 rev1=1 crypto rx=0x7f7ac400cc30 tx=0x7f7ac400a5c0 comp rx=0 tx=0).stop 2024-04-20T11:36:49.210 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.204+0000 7f7ad3400640 1 -- 172.21.15.152:0/3376913063 shutdown_connections 2024-04-20T11:36:49.210 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.204+0000 7f7ad3400640 1 --2- 172.21.15.152:0/3376913063 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f7aac06d100 0x7f7aac06f5c0 secure :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f7ac0036000 tx=0x7f7ac0008030 comp rx=0 tx=0).stop 2024-04-20T11:36:49.211 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.204+0000 7f7ad3400640 1 --2- 172.21.15.152:0/3376913063 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7acc0cf220 0x7f7acc1e88c0 unknown :-1 s=CLOSED pgs=2093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:49.211 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.204+0000 7f7ad3400640 1 --2- 172.21.15.152:0/3376913063 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7acc0cd8e0 0x7f7acc0cb320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:49.211 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.204+0000 7f7ad3400640 1 --2- 172.21.15.152:0/3376913063 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7acc0b4d90 0x7f7acc0cade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:49.211 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.204+0000 7f7ad3400640 1 -- 172.21.15.152:0/3376913063 >> 172.21.15.152:0/3376913063 conn(0x7f7acc06b980 msgr2=0x7f7acc06f4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:49.211 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.205+0000 7f7ad3400640 1 -- 172.21.15.152:0/3376913063 shutdown_connections 2024-04-20T11:36:49.211 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.205+0000 7f7ad3400640 1 -- 172.21.15.152:0/3376913063 wait complete. 2024-04-20T11:36:49.233 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-20T11:36:49.418 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.417+0000 7f4dd8e00640 1 Processor -- start 2024-04-20T11:36:49.418 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.417+0000 7f4dd8e00640 1 -- start start 2024-04-20T11:36:49.419 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.419+0000 7f4dd8e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4dd4160220 0x7f4dd41606b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:49.420 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.419+0000 7f4dd8e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f4dd4141450 0x7f4dd41418e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:49.420 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.419+0000 7f4dd8e00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4dd406fb40 con 0x7f4dd4071cd0 2024-04-20T11:36:49.420 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.419+0000 7f4dd8e00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f4dd413f8e0 con 0x7f4dd4141450 2024-04-20T11:36:49.420 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.419+0000 7f4dd8e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4dd4160f90 con 0x7f4dd4160220 2024-04-20T11:36:49.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.420+0000 7f4dcbe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4dd4160220 0x7f4dd41606b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:49.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.420+0000 7f4dcbe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4dd4160220 0x7f4dd41606b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.152:55854/0 (socket says 172.21.15.152:55854) 2024-04-20T11:36:49.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.420+0000 7f4dcbe00640 1 -- 172.21.15.152:0/2964946876 learned_addr learned my addr 172.21.15.152:0/2964946876 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:49.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.420+0000 7f4dd3e00640 1 --2- 172.21.15.152:0/2964946876 >> v2:172.21.15.175:3300/0 conn(0x7f4dd4141450 0x7f4dd41418e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:49.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.421+0000 7f4dcbe00640 1 -- 172.21.15.152:0/2964946876 >> v2:172.21.15.175:3300/0 conn(0x7f4dd4141450 msgr2=0x7f4dd41418e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:49.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.421+0000 7f4dcbe00640 1 --2- 172.21.15.152:0/2964946876 >> v2:172.21.15.175:3300/0 conn(0x7f4dd4141450 0x7f4dd41418e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:49.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.421+0000 7f4dcbe00640 1 -- 172.21.15.152:0/2964946876 >> v1:172.21.15.175:6789/0 conn(0x7f4dd4071cd0 legacy=0x7f4dd40720b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:49.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.421+0000 7f4dcbe00640 1 -- 172.21.15.152:0/2964946876 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dd406dd90 con 0x7f4dd4160220 2024-04-20T11:36:49.422 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.422+0000 7f4dcbe00640 1 --2- 172.21.15.152:0/2964946876 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4dd4160220 0x7f4dd41606b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc0098a0 tx=0x7f4dbc02b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=563dcbf28d3d364 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:49.423 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.422+0000 7f4dd2a00640 1 -- 172.21.15.152:0/2964946876 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dbc035020 con 0x7f4dd4160220 2024-04-20T11:36:49.423 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.422+0000 7f4dd2a00640 1 -- 172.21.15.152:0/2964946876 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4dbc031640 con 0x7f4dd4160220 2024-04-20T11:36:49.423 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.422+0000 7f4dd2a00640 1 -- 172.21.15.152:0/2964946876 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dbc02d040 con 0x7f4dd4160220 2024-04-20T11:36:49.424 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.423+0000 7f4dd8e00640 1 -- 172.21.15.152:0/2964946876 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4dd4160220 msgr2=0x7f4dd41606b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:49.424 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.423+0000 7f4dd8e00640 1 --2- 172.21.15.152:0/2964946876 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4dd4160220 0x7f4dd41606b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc0098a0 tx=0x7f4dbc02b8f0 comp rx=0 tx=0).stop 2024-04-20T11:36:49.424 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.423+0000 7f4dd8e00640 1 -- 172.21.15.152:0/2964946876 shutdown_connections 2024-04-20T11:36:49.424 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.423+0000 7f4dd8e00640 1 --2- 172.21.15.152:0/2964946876 >> v2:172.21.15.175:3300/0 conn(0x7f4dd4141450 0x7f4dd41418e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:49.424 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.423+0000 7f4dd8e00640 1 --2- 172.21.15.152:0/2964946876 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4dd4160220 0x7f4dd41606b0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:49.424 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.423+0000 7f4dd8e00640 1 -- 172.21.15.152:0/2964946876 >> 172.21.15.152:0/2964946876 conn(0x7f4dd4067ae0 msgr2=0x7f4dd406b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:49.424 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.423+0000 7f4dd8e00640 1 -- 172.21.15.152:0/2964946876 shutdown_connections 2024-04-20T11:36:49.424 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.423+0000 7f4dd8e00640 1 -- 172.21.15.152:0/2964946876 wait complete. 2024-04-20T11:36:49.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.425+0000 7f4dd8e00640 1 Processor -- start 2024-04-20T11:36:49.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.425+0000 7f4dd8e00640 1 -- start start 2024-04-20T11:36:49.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.426+0000 7f4dd8e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4dd4071cd0 0x7f4dd413dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:49.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.426+0000 7f4dd8e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4dd4141450 0x7f4dd40ce060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:49.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.426+0000 7f4dd8e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4dd4160220 0x7f4dd40b4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:49.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.426+0000 7f4dd8e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd406e630 con 0x7f4dd4071cd0 2024-04-20T11:36:49.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.426+0000 7f4dd8e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd406e350 con 0x7f4dd4141450 2024-04-20T11:36:49.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.426+0000 7f4dd8e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4dd4160bf0 con 0x7f4dd4160220 2024-04-20T11:36:49.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.427+0000 7f4dd3400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4dd4071cd0 0x7f4dd413dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:49.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.427+0000 7f4dd3400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4dd4071cd0 0x7f4dd413dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:46418/0 (socket says 172.21.15.152:46418) 2024-04-20T11:36:49.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.427+0000 7f4dd3400640 1 -- 172.21.15.152:0/268533657 learned_addr learned my addr 172.21.15.152:0/268533657 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:49.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.427+0000 7f4dd3e00640 1 --2- 172.21.15.152:0/268533657 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4dd4160220 0x7f4dd40b4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:49.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.427+0000 7f4dcbe00640 1 --2- 172.21.15.152:0/268533657 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4dd4141450 0x7f4dd40ce060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:49.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.427+0000 7f4dd3400640 1 -- 172.21.15.152:0/268533657 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4dd4160220 msgr2=0x7f4dd40b4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:49.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.427+0000 7f4dd3400640 1 --2- 172.21.15.152:0/268533657 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4dd4160220 0x7f4dd40b4aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:49.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.427+0000 7f4dd3400640 1 -- 172.21.15.152:0/268533657 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4dd4141450 msgr2=0x7f4dd40ce060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:49.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.427+0000 7f4dd3400640 1 --2- 172.21.15.152:0/268533657 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4dd4141450 0x7f4dd40ce060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:49.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.427+0000 7f4dd3400640 1 -- 172.21.15.152:0/268533657 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dd40b3a00 con 0x7f4dd4071cd0 2024-04-20T11:36:49.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.427+0000 7f4dcbe00640 1 --2- 172.21.15.152:0/268533657 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4dd4141450 0x7f4dd40ce060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:36:49.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.427+0000 7f4dd3e00640 1 --2- 172.21.15.152:0/268533657 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4dd4160220 0x7f4dd40b4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:36:49.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.428+0000 7f4dd3400640 1 --2- 172.21.15.152:0/268533657 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4dd4071cd0 0x7f4dd413dd30 secure :-1 s=READY pgs=2094 cs=0 l=1 rev1=1 crypto rx=0x7f4dc400c400 tx=0x7f4dc400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:49.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.428+0000 7f4dcb400640 1 -- 172.21.15.152:0/268533657 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc40052d0 con 0x7f4dd4071cd0 2024-04-20T11:36:49.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.428+0000 7f4dcb400640 1 -- 172.21.15.152:0/268533657 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4dc4005490 con 0x7f4dd4071cd0 2024-04-20T11:36:49.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.428+0000 7f4dd8e00640 1 -- 172.21.15.152:0/268533657 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4dd40b3ce0 con 0x7f4dd4071cd0 2024-04-20T11:36:49.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.428+0000 7f4dcb400640 1 -- 172.21.15.152:0/268533657 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc4007260 con 0x7f4dd4071cd0 2024-04-20T11:36:49.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.428+0000 7f4dd8e00640 1 -- 172.21.15.152:0/268533657 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4dd40b42a0 con 0x7f4dd4071cd0 2024-04-20T11:36:49.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.429+0000 7f4dd8e00640 1 -- 172.21.15.152:0/268533657 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4dd4075e60 con 0x7f4dd4071cd0 2024-04-20T11:36:49.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.430+0000 7f4dcb400640 1 -- 172.21.15.152:0/268533657 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) ==== 91754+0+0 (secure 0 0 0) 0x7f4dc4007400 con 0x7f4dd4071cd0 2024-04-20T11:36:49.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.431+0000 7f4dcb400640 1 --2- 172.21.15.152:0/268533657 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4db006d100 0x7f4db006f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:49.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.431+0000 7f4dcb400640 1 -- 172.21.15.152:0/268533657 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 4943+0+0 (secure 0 0 0) 0x7f4dc4089a60 con 0x7f4dd4071cd0 2024-04-20T11:36:49.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.432+0000 7f4dcbe00640 1 --2- 172.21.15.152:0/268533657 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4db006d100 0x7f4db006f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:49.433 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.433+0000 7f4dcbe00640 1 --2- 172.21.15.152:0/268533657 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4db006d100 0x7f4db006f5c0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc036000 tx=0x7f4dbc008030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:49.436 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.436+0000 7f4dcb400640 1 -- 172.21.15.152:0/268533657 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f4dc4059780 con 0x7f4dd4071cd0 2024-04-20T11:36:49.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.639+0000 7f4dd8e00640 1 -- 172.21.15.152:0/268533657 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f4dd4072d30 con 0x7f4dd4071cd0 2024-04-20T11:36:49.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.646+0000 7f4dcb400640 1 -- 172.21.15.152:0/268533657 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v27) ==== 104+0+0 (secure 0 0 0) 0x7f4dc4059080 con 0x7f4dd4071cd0 2024-04-20T11:36:49.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.649+0000 7f4dd8e00640 1 -- 172.21.15.152:0/268533657 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4db006d100 msgr2=0x7f4db006f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:49.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.649+0000 7f4dd8e00640 1 --2- 172.21.15.152:0/268533657 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4db006d100 0x7f4db006f5c0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc036000 tx=0x7f4dbc008030 comp rx=0 tx=0).stop 2024-04-20T11:36:49.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.649+0000 7f4dd8e00640 1 -- 172.21.15.152:0/268533657 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4dd4071cd0 msgr2=0x7f4dd413dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:49.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.649+0000 7f4dd8e00640 1 --2- 172.21.15.152:0/268533657 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4dd4071cd0 0x7f4dd413dd30 secure :-1 s=READY pgs=2094 cs=0 l=1 rev1=1 crypto rx=0x7f4dc400c400 tx=0x7f4dc400c8c0 comp rx=0 tx=0).stop 2024-04-20T11:36:49.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.649+0000 7f4dd8e00640 1 -- 172.21.15.152:0/268533657 shutdown_connections 2024-04-20T11:36:49.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.649+0000 7f4dd8e00640 1 --2- 172.21.15.152:0/268533657 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4db006d100 0x7f4db006f5c0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:49.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.649+0000 7f4dd8e00640 1 --2- 172.21.15.152:0/268533657 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4dd4160220 0x7f4dd40b4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:49.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.649+0000 7f4dd8e00640 1 --2- 172.21.15.152:0/268533657 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4dd4141450 0x7f4dd40ce060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:49.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.649+0000 7f4dd8e00640 1 --2- 172.21.15.152:0/268533657 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4dd4071cd0 0x7f4dd413dd30 unknown :-1 s=CLOSED pgs=2094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:49.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.649+0000 7f4dd8e00640 1 -- 172.21.15.152:0/268533657 >> 172.21.15.152:0/268533657 conn(0x7f4dd4067ae0 msgr2=0x7f4dd40b2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:49.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.650+0000 7f4dd8e00640 1 -- 172.21.15.152:0/268533657 shutdown_connections 2024-04-20T11:36:49.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.650+0000 7f4dd8e00640 1 -- 172.21.15.152:0/268533657 wait complete. 2024-04-20T11:36:49.668 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_legacy_to_v1' 2024-04-20T11:36:49.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.864+0000 7f211e200640 1 Processor -- start 2024-04-20T11:36:49.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.864+0000 7f211e200640 1 -- start start 2024-04-20T11:36:49.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.865+0000 7f211e200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f21180676c0 0x7f211815d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:49.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.865+0000 7f211e200640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21180cd680 con 0x7f21180ce3d0 2024-04-20T11:36:49.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.865+0000 7f211e200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f211815c8a0 con 0x7f21180ceab0 2024-04-20T11:36:49.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.865+0000 7f211e200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f21180cdf40 con 0x7f21180676c0 2024-04-20T11:36:49.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.866+0000 7f2117e00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f21180ce3d0 0x7f211806e580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:60304/0 (socket says 172.21.15.152:60304) 2024-04-20T11:36:49.867 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.866+0000 7f2117e00640 1 -- 172.21.15.152:0/3509671454 learned_addr learned my addr 172.21.15.152:0/3509671454 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:36:49.867 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.866+0000 7f211d800640 1 --2- 172.21.15.152:0/3509671454 >> v2:172.21.15.152:3300/0 conn(0x7f21180676c0 0x7f211815d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:49.867 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.866+0000 7f2117400640 1 -- 172.21.15.152:0/3509671454 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1149745823 0 0) 0x7f21180cd680 con 0x7f21180ce3d0 2024-04-20T11:36:49.867 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.866+0000 7f2117400640 1 -- 172.21.15.152:0/3509671454 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2100003520 con 0x7f21180ce3d0 2024-04-20T11:36:49.867 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.866+0000 7f211d800640 1 -- 172.21.15.152:0/3509671454 >> v1:172.21.15.175:6789/0 conn(0x7f21180ceab0 legacy=0x7f21180cee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:49.867 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.866+0000 7f211d800640 1 -- 172.21.15.152:0/3509671454 >> v1:172.21.15.152:6789/0 conn(0x7f21180ce3d0 legacy=0x7f211806e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:49.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.866+0000 7f211d800640 1 -- 172.21.15.152:0/3509671454 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21180cd8f0 con 0x7f21180676c0 2024-04-20T11:36:49.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.867+0000 7f211d800640 1 --2- 172.21.15.152:0/3509671454 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f21180676c0 0x7f211815d470 secure :-1 s=READY pgs=2096 cs=0 l=1 rev1=1 crypto rx=0x7f2110009d90 tx=0x7f211002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99938a0ee44dfe69 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:49.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.868+0000 7f2117400640 1 -- 172.21.15.152:0/3509671454 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2110039070 con 0x7f21180676c0 2024-04-20T11:36:49.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.868+0000 7f2117400640 1 -- 172.21.15.152:0/3509671454 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f211002cc60 con 0x7f21180676c0 2024-04-20T11:36:49.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.868+0000 7f2117400640 1 -- 172.21.15.152:0/3509671454 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2110034660 con 0x7f21180676c0 2024-04-20T11:36:49.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.868+0000 7f211e200640 1 -- 172.21.15.152:0/3509671454 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f21180676c0 msgr2=0x7f211815d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:49.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.868+0000 7f211e200640 1 --2- 172.21.15.152:0/3509671454 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f21180676c0 0x7f211815d470 secure :-1 s=READY pgs=2096 cs=0 l=1 rev1=1 crypto rx=0x7f2110009d90 tx=0x7f211002beb0 comp rx=0 tx=0).stop 2024-04-20T11:36:49.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.868+0000 7f211e200640 1 -- 172.21.15.152:0/3509671454 shutdown_connections 2024-04-20T11:36:49.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.868+0000 7f211e200640 1 --2- 172.21.15.152:0/3509671454 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f21180676c0 0x7f211815d470 unknown :-1 s=CLOSED pgs=2096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:49.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.868+0000 7f211e200640 1 -- 172.21.15.152:0/3509671454 >> 172.21.15.152:0/3509671454 conn(0x7f2118073370 msgr2=0x7f211806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:49.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.869+0000 7f211e200640 1 -- 172.21.15.152:0/3509671454 shutdown_connections 2024-04-20T11:36:49.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.869+0000 7f211e200640 1 -- 172.21.15.152:0/3509671454 wait complete. 2024-04-20T11:36:49.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.870+0000 7f211e200640 1 Processor -- start 2024-04-20T11:36:49.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.870+0000 7f211e200640 1 -- start start 2024-04-20T11:36:49.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.871+0000 7f211e200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f21180676c0 0x7f21181472e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:49.873 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.871+0000 7f211e200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f21180ce3d0 0x7f21180ae8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:49.873 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.871+0000 7f211e200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f21180ceab0 0x7f21180aee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:49.873 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.871+0000 7f211e200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f211806ec90 con 0x7f21180676c0 2024-04-20T11:36:49.873 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.871+0000 7f211e200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2118067ca0 con 0x7f21180ce3d0 2024-04-20T11:36:49.873 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.871+0000 7f211e200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f211806ee10 con 0x7f21180ceab0 2024-04-20T11:36:49.873 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.872+0000 7f211ce00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f21180676c0 0x7f21181472e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:49.873 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.872+0000 7f211ce00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f21180676c0 0x7f21181472e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:46434/0 (socket says 172.21.15.152:46434) 2024-04-20T11:36:49.873 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.872+0000 7f211ce00640 1 -- 172.21.15.152:0/372866382 learned_addr learned my addr 172.21.15.152:0/372866382 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:49.873 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.872+0000 7f211ce00640 1 -- 172.21.15.152:0/372866382 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f21180ceab0 msgr2=0x7f21180aee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:49.873 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.872+0000 7f211ce00640 1 --2- 172.21.15.152:0/372866382 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f21180ceab0 0x7f21180aee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:49.873 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.872+0000 7f211ce00640 1 -- 172.21.15.152:0/372866382 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f21180ce3d0 msgr2=0x7f21180ae8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:49.873 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.872+0000 7f211ce00640 1 --2- 172.21.15.152:0/372866382 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f21180ce3d0 0x7f21180ae8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:49.873 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.872+0000 7f211ce00640 1 -- 172.21.15.152:0/372866382 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2118147be0 con 0x7f21180676c0 2024-04-20T11:36:49.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.872+0000 7f211ce00640 1 --2- 172.21.15.152:0/372866382 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f21180676c0 0x7f21181472e0 secure :-1 s=READY pgs=2097 cs=0 l=1 rev1=1 crypto rx=0x7f210c00b3f0 tx=0x7f210c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:49.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.873+0000 7f2114c00640 1 -- 172.21.15.152:0/372866382 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f210c015020 con 0x7f21180676c0 2024-04-20T11:36:49.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.873+0000 7f2114c00640 1 -- 172.21.15.152:0/372866382 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f210c011830 con 0x7f21180676c0 2024-04-20T11:36:49.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.873+0000 7f211e200640 1 -- 172.21.15.152:0/372866382 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21180aacc0 con 0x7f21180676c0 2024-04-20T11:36:49.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.873+0000 7f2114c00640 1 -- 172.21.15.152:0/372866382 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f210c011c20 con 0x7f21180676c0 2024-04-20T11:36:49.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.873+0000 7f211e200640 1 -- 172.21.15.152:0/372866382 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2118148160 con 0x7f21180676c0 2024-04-20T11:36:49.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.874+0000 7f211e200640 1 -- 172.21.15.152:0/372866382 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2118075e60 con 0x7f21180676c0 2024-04-20T11:36:49.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.875+0000 7f2114c00640 1 -- 172.21.15.152:0/372866382 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) ==== 91754+0+0 (secure 0 0 0) 0x7f210c018020 con 0x7f21180676c0 2024-04-20T11:36:49.877 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.876+0000 7f2114c00640 1 --2- 172.21.15.152:0/372866382 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f20f806d0d0 0x7f20f806f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:49.877 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.876+0000 7f2114c00640 1 -- 172.21.15.152:0/372866382 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 4943+0+0 (secure 0 0 0) 0x7f210c08c7d0 con 0x7f21180676c0 2024-04-20T11:36:49.877 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.877+0000 7f2117e00640 1 --2- 172.21.15.152:0/372866382 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f20f806d0d0 0x7f20f806f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:49.878 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.878+0000 7f2117e00640 1 --2- 172.21.15.152:0/372866382 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f20f806d0d0 0x7f20f806f590 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f2104000f60 tx=0x7f21040078f0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:49.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:49.881+0000 7f2114c00640 1 -- 172.21.15.152:0/372866382 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f210c01f030 con 0x7f21180676c0 2024-04-20T11:36:50.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.154+0000 7f211e200640 1 -- 172.21.15.152:0/372866382 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_legacy_to_v1"]} v 0) -- 0x7f2118070370 con 0x7f21180676c0 2024-04-20T11:36:50.217 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.216+0000 7f2114c00640 1 -- 172.21.15.152:0/372866382 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_legacy_to_v1"]}]=0 v526) ==== 149+0+0 (secure 0 0 0) 0x7f210c0172c0 con 0x7f21180676c0 2024-04-20T11:36:50.221 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.219+0000 7f211e200640 1 -- 172.21.15.152:0/372866382 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f20f806d0d0 msgr2=0x7f20f806f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:50.221 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.219+0000 7f211e200640 1 --2- 172.21.15.152:0/372866382 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f20f806d0d0 0x7f20f806f590 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f2104000f60 tx=0x7f21040078f0 comp rx=0 tx=0).stop 2024-04-20T11:36:50.221 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.219+0000 7f211e200640 1 -- 172.21.15.152:0/372866382 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f21180676c0 msgr2=0x7f21181472e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:50.221 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.219+0000 7f211e200640 1 --2- 172.21.15.152:0/372866382 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f21180676c0 0x7f21181472e0 secure :-1 s=READY pgs=2097 cs=0 l=1 rev1=1 crypto rx=0x7f210c00b3f0 tx=0x7f210c00b8b0 comp rx=0 tx=0).stop 2024-04-20T11:36:50.221 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.220+0000 7f211e200640 1 -- 172.21.15.152:0/372866382 shutdown_connections 2024-04-20T11:36:50.222 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.220+0000 7f211e200640 1 --2- 172.21.15.152:0/372866382 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f20f806d0d0 0x7f20f806f590 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:50.222 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.220+0000 7f211e200640 1 --2- 172.21.15.152:0/372866382 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f21180ceab0 0x7f21180aee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:50.222 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.220+0000 7f211e200640 1 --2- 172.21.15.152:0/372866382 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f21180ce3d0 0x7f21180ae8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:50.222 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.220+0000 7f211e200640 1 --2- 172.21.15.152:0/372866382 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f21180676c0 0x7f21181472e0 unknown :-1 s=CLOSED pgs=2097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:50.222 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.220+0000 7f211e200640 1 -- 172.21.15.152:0/372866382 >> 172.21.15.152:0/372866382 conn(0x7f2118073370 msgr2=0x7f211806c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:50.222 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.220+0000 7f211e200640 1 -- 172.21.15.152:0/372866382 shutdown_connections 2024-04-20T11:36:50.222 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.220+0000 7f211e200640 1 -- 172.21.15.152:0/372866382 wait complete. 2024-04-20T11:36:50.241 INFO:tasks.cephfs_test_runner:poor man's upgrade test -- rather than going through a full upgrade cycle, ... ok 2024-04-20T11:36:50.241 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_upgrade_v1_to_v2 (tasks.cephfs.test_volumes.TestMisc) poor man's upgrade test -- theme continues... 2024-04-20T11:36:50.242 INFO:tasks.cephfs_test_runner:test_subvolume_upgrade_v1_to_v2 (tasks.cephfs.test_volumes.TestMisc) 2024-04-20T11:36:50.242 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:36:50.442 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.442+0000 7f686e400640 1 Processor -- start 2024-04-20T11:36:50.442 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.442+0000 7f686e400640 1 -- start start 2024-04-20T11:36:50.444 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.443+0000 7f686e400640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f6868160b90 0x7f686815d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:50.444 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.443+0000 7f686e400640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68680ce8c0 con 0x7f686815d1e0 2024-04-20T11:36:50.444 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.443+0000 7f686e400640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68681600b0 con 0x7f6868071cd0 2024-04-20T11:36:50.444 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.443+0000 7f686e400640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f6868160390 con 0x7f6868160b90 2024-04-20T11:36:50.445 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.444+0000 7f686d000640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f686815d1e0 0x7f68680cdfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:60318/0 (socket says 172.21.15.152:60318) 2024-04-20T11:36:50.445 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.444+0000 7f686d000640 1 -- 172.21.15.152:0/4172201768 learned_addr learned my addr 172.21.15.152:0/4172201768 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:36:50.445 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.444+0000 7f6867400640 1 -- 172.21.15.152:0/4172201768 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3016289966 0 0) 0x7f68680ce8c0 con 0x7f686815d1e0 2024-04-20T11:36:50.445 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.444+0000 7f6867400640 1 -- 172.21.15.152:0/4172201768 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f684c003520 con 0x7f686815d1e0 2024-04-20T11:36:50.445 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.444+0000 7f6867e00640 1 --2- 172.21.15.152:0/4172201768 >> v2:172.21.15.175:3300/0 conn(0x7f6868160b90 0x7f686815d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:50.447 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.445+0000 7f6867400640 1 -- 172.21.15.152:0/4172201768 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1746681438 0 0) 0x7f684c003520 con 0x7f686815d1e0 2024-04-20T11:36:50.447 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.445+0000 7f6867400640 1 -- 172.21.15.152:0/4172201768 >> v2:172.21.15.175:3300/0 conn(0x7f6868160b90 msgr2=0x7f686815d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:50.447 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.445+0000 7f6867400640 1 --2- 172.21.15.152:0/4172201768 >> v2:172.21.15.175:3300/0 conn(0x7f6868160b90 0x7f686815d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:50.447 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.445+0000 7f6867400640 1 -- 172.21.15.152:0/4172201768 >> v1:172.21.15.175:6789/0 conn(0x7f6868071cd0 legacy=0x7f6868072120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:50.447 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.445+0000 7f6867400640 1 -- 172.21.15.152:0/4172201768 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6868160670 con 0x7f686815d1e0 2024-04-20T11:36:50.447 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.445+0000 7f6867400640 1 -- 172.21.15.152:0/4172201768 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f6858001670 con 0x7f686815d1e0 2024-04-20T11:36:50.447 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.445+0000 7f6867e00640 1 --2- 172.21.15.152:0/4172201768 >> v2:172.21.15.175:3300/0 conn(0x7f6868160b90 0x7f686815d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:36:50.448 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.445+0000 7f6867400640 1 -- 172.21.15.152:0/4172201768 >> v1:172.21.15.152:6789/0 conn(0x7f686815d1e0 legacy=0x7f68680cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:50.448 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.445+0000 7f6867400640 1 --2- 172.21.15.152:0/4172201768 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f684c003a30 0x7f684c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:50.448 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.445+0000 7f6867400640 1 --2- 172.21.15.152:0/4172201768 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f684c004800 0x7f684c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:50.448 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.445+0000 7f6867400640 1 --2- 172.21.15.152:0/4172201768 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f684c0052e0 0x7f684c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:50.448 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.445+0000 7f6867400640 1 -- 172.21.15.152:0/4172201768 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f6868160390 con 0x7f684c004800 2024-04-20T11:36:50.448 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.445+0000 7f6867400640 1 -- 172.21.15.152:0/4172201768 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f684c00c190 con 0x7f684c0052e0 2024-04-20T11:36:50.448 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.445+0000 7f6867400640 1 -- 172.21.15.152:0/4172201768 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f684c00c390 con 0x7f684c003a30 2024-04-20T11:36:50.448 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.445+0000 7f6867e00640 1 --2- 172.21.15.152:0/4172201768 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f684c004800 0x7f684c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:50.448 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.446+0000 7f686d000640 1 --2- 172.21.15.152:0/4172201768 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f684c003a30 0x7f684c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:50.448 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.446+0000 7f686da00640 1 --2- 172.21.15.152:0/4172201768 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f684c0052e0 0x7f684c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:50.448 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.446+0000 7f6867e00640 1 -- 172.21.15.152:0/4172201768 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f684c003a30 msgr2=0x7f684c003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:50.448 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.446+0000 7f6867e00640 1 --2- 172.21.15.152:0/4172201768 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f684c003a30 0x7f684c003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:50.448 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.446+0000 7f6867e00640 1 -- 172.21.15.152:0/4172201768 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f684c0052e0 msgr2=0x7f684c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:50.448 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.446+0000 7f6867e00640 1 --2- 172.21.15.152:0/4172201768 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f684c0052e0 0x7f684c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:50.449 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.446+0000 7f6867e00640 1 -- 172.21.15.152:0/4172201768 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f684c003520 con 0x7f684c004800 2024-04-20T11:36:50.449 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.446+0000 7f686da00640 1 --2- 172.21.15.152:0/4172201768 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f684c0052e0 0x7f684c00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:36:50.449 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.446+0000 7f686da00640 1 -- 172.21.15.152:0/4172201768 reap_dead start 2024-04-20T11:36:50.449 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.447+0000 7f6867e00640 1 --2- 172.21.15.152:0/4172201768 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f684c004800 0x7f684c004c80 secure :-1 s=READY pgs=2099 cs=0 l=1 rev1=1 crypto rx=0x7f68500073b0 tx=0x7f684c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7e923b656ae34a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:50.450 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.447+0000 7f6867400640 1 -- 172.21.15.152:0/4172201768 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6850039070 con 0x7f684c004800 2024-04-20T11:36:50.450 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.447+0000 7f6867400640 1 -- 172.21.15.152:0/4172201768 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f685002cd30 con 0x7f684c004800 2024-04-20T11:36:50.450 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.447+0000 7f6867400640 1 -- 172.21.15.152:0/4172201768 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6850009c80 con 0x7f684c004800 2024-04-20T11:36:50.450 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.448+0000 7f686e400640 1 -- 172.21.15.152:0/4172201768 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f684c004800 msgr2=0x7f684c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:50.450 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.448+0000 7f686e400640 1 --2- 172.21.15.152:0/4172201768 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f684c004800 0x7f684c004c80 secure :-1 s=READY pgs=2099 cs=0 l=1 rev1=1 crypto rx=0x7f68500073b0 tx=0x7f684c005830 comp rx=0 tx=0).stop 2024-04-20T11:36:50.450 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.448+0000 7f686e400640 1 -- 172.21.15.152:0/4172201768 shutdown_connections 2024-04-20T11:36:50.450 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.448+0000 7f686e400640 1 --2- 172.21.15.152:0/4172201768 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f684c004800 0x7f684c004c80 unknown :-1 s=CLOSED pgs=2099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:50.450 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.448+0000 7f686e400640 1 -- 172.21.15.152:0/4172201768 >> 172.21.15.152:0/4172201768 conn(0x7f6868067ae0 msgr2=0x7f686806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:50.450 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.448+0000 7f686e400640 1 -- 172.21.15.152:0/4172201768 shutdown_connections 2024-04-20T11:36:50.450 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.448+0000 7f686e400640 1 -- 172.21.15.152:0/4172201768 wait complete. 2024-04-20T11:36:50.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.450+0000 7f686e400640 1 Processor -- start 2024-04-20T11:36:50.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.450+0000 7f686e400640 1 -- start start 2024-04-20T11:36:50.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.451+0000 7f686e400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f684c004800 0x7f6868152230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:50.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.451+0000 7f686e400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f68681f24b0 0x7f68680ac360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:50.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.451+0000 7f686e400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f68680ac8a0 0x7f68681e7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:50.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.451+0000 7f686e400640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f686815d740 con 0x7f68681f24b0 2024-04-20T11:36:50.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.451+0000 7f686e400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f686815d5c0 con 0x7f684c004800 2024-04-20T11:36:50.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.451+0000 7f686e400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f686815d8c0 con 0x7f68680ac8a0 2024-04-20T11:36:50.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.451+0000 7f6867e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f68681f24b0 0x7f68680ac360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:50.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.451+0000 7f6867e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f68681f24b0 0x7f68680ac360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:46438/0 (socket says 172.21.15.152:46438) 2024-04-20T11:36:50.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.451+0000 7f6867e00640 1 -- 172.21.15.152:0/917984204 learned_addr learned my addr 172.21.15.152:0/917984204 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:50.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.451+0000 7f686da00640 1 --2- 172.21.15.152:0/917984204 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f68680ac8a0 0x7f68681e7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:50.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.451+0000 7f6867e00640 1 -- 172.21.15.152:0/917984204 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f68680ac8a0 msgr2=0x7f68681e7cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:50.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.451+0000 7f6867e00640 1 --2- 172.21.15.152:0/917984204 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f68680ac8a0 0x7f68681e7cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:50.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.451+0000 7f686d000640 1 --2- 172.21.15.152:0/917984204 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f684c004800 0x7f6868152230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:50.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.451+0000 7f6867e00640 1 -- 172.21.15.152:0/917984204 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f684c004800 msgr2=0x7f6868152230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:50.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.452+0000 7f6867e00640 1 --2- 172.21.15.152:0/917984204 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f684c004800 0x7f6868152230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:50.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.452+0000 7f6867e00640 1 -- 172.21.15.152:0/917984204 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68680a9ae0 con 0x7f68681f24b0 2024-04-20T11:36:50.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.452+0000 7f6867e00640 1 --2- 172.21.15.152:0/917984204 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f68681f24b0 0x7f68680ac360 secure :-1 s=READY pgs=2100 cs=0 l=1 rev1=1 crypto rx=0x7f6850037040 tx=0x7f6850004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:50.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.452+0000 7f6864c00640 1 -- 172.21.15.152:0/917984204 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6850042030 con 0x7f68681f24b0 2024-04-20T11:36:50.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.452+0000 7f6864c00640 1 -- 172.21.15.152:0/917984204 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f685003a040 con 0x7f68681f24b0 2024-04-20T11:36:50.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.452+0000 7f686e400640 1 -- 172.21.15.152:0/917984204 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68680a9dc0 con 0x7f68681f24b0 2024-04-20T11:36:50.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.452+0000 7f6864c00640 1 -- 172.21.15.152:0/917984204 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6850039070 con 0x7f68681f24b0 2024-04-20T11:36:50.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.452+0000 7f686e400640 1 -- 172.21.15.152:0/917984204 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68680aa380 con 0x7f68681f24b0 2024-04-20T11:36:50.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.453+0000 7f686e400640 1 -- 172.21.15.152:0/917984204 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68680ce8c0 con 0x7f68681f24b0 2024-04-20T11:36:50.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.454+0000 7f6864c00640 1 -- 172.21.15.152:0/917984204 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) ==== 91754+0+0 (secure 0 0 0) 0x7f6850045070 con 0x7f68681f24b0 2024-04-20T11:36:50.456 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.455+0000 7f6864c00640 1 --2- 172.21.15.152:0/917984204 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f684406d100 0x7f684406f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:50.456 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.456+0000 7f6864c00640 1 -- 172.21.15.152:0/917984204 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 4943+0+0 (secure 0 0 0) 0x7f68500ae5c0 con 0x7f68681f24b0 2024-04-20T11:36:50.456 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.456+0000 7f686d000640 1 --2- 172.21.15.152:0/917984204 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f684406d100 0x7f684406f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:50.457 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.457+0000 7f686d000640 1 --2- 172.21.15.152:0/917984204 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f684406d100 0x7f684406f5c0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f68580017b0 tx=0x7f6858000ed0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:50.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.460+0000 7f6864c00640 1 -- 172.21.15.152:0/917984204 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f68680ce8c0 con 0x7f68681f24b0 2024-04-20T11:36:50.664 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.663+0000 7f686e400640 1 -- 172.21.15.152:0/917984204 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6868072120 con 0x7f68681f24b0 2024-04-20T11:36:50.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.665+0000 7f6864c00640 1 -- 172.21.15.152:0/917984204 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v145) ==== 74+0+13090 (secure 0 0 0) 0x7f6868072120 con 0x7f68681f24b0 2024-04-20T11:36:50.665 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:36:50.665 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":145,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:36:49.196418+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.175:6835/189289100":"2024-04-21T11:36:46.179764+0000","172.21.15.175:6833/189289100":"2024-04-21T11:36:46.179764+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:36:50.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.667+0000 7f686e400640 1 -- 172.21.15.152:0/917984204 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f684406d100 msgr2=0x7f684406f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:50.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.667+0000 7f686e400640 1 --2- 172.21.15.152:0/917984204 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f684406d100 0x7f684406f5c0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f68580017b0 tx=0x7f6858000ed0 comp rx=0 tx=0).stop 2024-04-20T11:36:50.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.667+0000 7f686e400640 1 -- 172.21.15.152:0/917984204 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f68681f24b0 msgr2=0x7f68680ac360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:50.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.667+0000 7f686e400640 1 --2- 172.21.15.152:0/917984204 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f68681f24b0 0x7f68680ac360 secure :-1 s=READY pgs=2100 cs=0 l=1 rev1=1 crypto rx=0x7f6850037040 tx=0x7f6850004ab0 comp rx=0 tx=0).stop 2024-04-20T11:36:50.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.668+0000 7f686e400640 1 -- 172.21.15.152:0/917984204 shutdown_connections 2024-04-20T11:36:50.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.668+0000 7f686e400640 1 --2- 172.21.15.152:0/917984204 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f684406d100 0x7f684406f5c0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:50.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.668+0000 7f686e400640 1 --2- 172.21.15.152:0/917984204 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f68680ac8a0 0x7f68681e7cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:50.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.668+0000 7f686e400640 1 --2- 172.21.15.152:0/917984204 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f68681f24b0 0x7f68680ac360 unknown :-1 s=CLOSED pgs=2100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:50.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.668+0000 7f686e400640 1 --2- 172.21.15.152:0/917984204 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f684c004800 0x7f6868152230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:50.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.668+0000 7f686e400640 1 -- 172.21.15.152:0/917984204 >> 172.21.15.152:0/917984204 conn(0x7f6868067ae0 msgr2=0x7f686806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:50.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.668+0000 7f686e400640 1 -- 172.21.15.152:0/917984204 shutdown_connections 2024-04-20T11:36:50.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.668+0000 7f686e400640 1 -- 172.21.15.152:0/917984204 wait complete. 2024-04-20T11:36:50.687 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T11:26:47.246571+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T11:36:50.687 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T11:36:50.894 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.894+0000 7fad4e600640 1 Processor -- start 2024-04-20T11:36:50.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.894+0000 7fad4e600640 1 -- start start 2024-04-20T11:36:50.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.896+0000 7fad4e600640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fad480b2880 0x7fad480b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:50.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.896+0000 7fad4e600640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad481463e0 con 0x7fad480cd1b0 2024-04-20T11:36:50.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.896+0000 7fad4e600640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad480b3e20 con 0x7fad48146000 2024-04-20T11:36:50.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.896+0000 7fad4e600640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fad480b4750 con 0x7fad480b2880 2024-04-20T11:36:50.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.896+0000 7fad47e00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fad480cd1b0 0x7fad480cd5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:60324/0 (socket says 172.21.15.152:60324) 2024-04-20T11:36:50.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.896+0000 7fad47e00640 1 -- 172.21.15.152:0/664620911 learned_addr learned my addr 172.21.15.152:0/664620911 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:36:50.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.896+0000 7fad46a00640 1 --2- 172.21.15.152:0/664620911 >> v2:172.21.15.175:3300/0 conn(0x7fad480b2880 0x7fad480b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:50.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.897+0000 7fad46000640 1 -- 172.21.15.152:0/664620911 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3644532033 0 0) 0x7fad481463e0 con 0x7fad480cd1b0 2024-04-20T11:36:50.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.897+0000 7fad46000640 1 -- 172.21.15.152:0/664620911 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fad2c003520 con 0x7fad480cd1b0 2024-04-20T11:36:50.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.897+0000 7fad46000640 1 -- 172.21.15.152:0/664620911 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2585889785 0 0) 0x7fad480b3e20 con 0x7fad48146000 2024-04-20T11:36:50.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.897+0000 7fad46000640 1 -- 172.21.15.152:0/664620911 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fad481463e0 con 0x7fad48146000 2024-04-20T11:36:50.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.897+0000 7fad46a00640 1 -- 172.21.15.152:0/664620911 >> v1:172.21.15.175:6789/0 conn(0x7fad48146000 legacy=0x7fad480b2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:50.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.897+0000 7fad46000640 1 -- 172.21.15.152:0/664620911 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2518965920 0 0) 0x7fad2c003520 con 0x7fad480cd1b0 2024-04-20T11:36:50.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.897+0000 7fad46a00640 1 -- 172.21.15.152:0/664620911 >> v1:172.21.15.152:6789/0 conn(0x7fad480cd1b0 legacy=0x7fad480cd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:50.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.897+0000 7fad46a00640 1 -- 172.21.15.152:0/664620911 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad480b4100 con 0x7fad480b2880 2024-04-20T11:36:50.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.898+0000 7fad46a00640 1 --2- 172.21.15.152:0/664620911 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad480b2880 0x7fad480b4be0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fad340097a0 tx=0x7fad3402b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71678c13449e34b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:50.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.899+0000 7fad46000640 1 -- 172.21.15.152:0/664620911 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad34037070 con 0x7fad480b2880 2024-04-20T11:36:50.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.899+0000 7fad46000640 1 -- 172.21.15.152:0/664620911 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad34030510 con 0x7fad480b2880 2024-04-20T11:36:50.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.899+0000 7fad46000640 1 -- 172.21.15.152:0/664620911 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad400019f0 con 0x7fad480b2880 2024-04-20T11:36:50.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.899+0000 7fad4e600640 1 -- 172.21.15.152:0/664620911 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad480b2880 msgr2=0x7fad480b4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:50.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.899+0000 7fad4e600640 1 --2- 172.21.15.152:0/664620911 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad480b2880 0x7fad480b4be0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fad340097a0 tx=0x7fad3402b790 comp rx=0 tx=0).stop 2024-04-20T11:36:50.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.900+0000 7fad4e600640 1 -- 172.21.15.152:0/664620911 shutdown_connections 2024-04-20T11:36:50.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.900+0000 7fad4e600640 1 --2- 172.21.15.152:0/664620911 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad480b2880 0x7fad480b4be0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:50.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.900+0000 7fad4e600640 1 -- 172.21.15.152:0/664620911 >> 172.21.15.152:0/664620911 conn(0x7fad4806b980 msgr2=0x7fad4806bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:50.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.900+0000 7fad4e600640 1 -- 172.21.15.152:0/664620911 shutdown_connections 2024-04-20T11:36:50.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.900+0000 7fad4e600640 1 -- 172.21.15.152:0/664620911 wait complete. 2024-04-20T11:36:50.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.901+0000 7fad4e600640 1 Processor -- start 2024-04-20T11:36:50.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.902+0000 7fad4e600640 1 -- start start 2024-04-20T11:36:50.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.902+0000 7fad4e600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fad480b2880 0x7fad4815bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:50.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.902+0000 7fad4e600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad480cd1b0 0x7fad4813fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:50.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.902+0000 7fad4e600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fad48146000 0x7fad4806e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:50.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.902+0000 7fad4e600640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fad4815ce70 con 0x7fad48146000 2024-04-20T11:36:50.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.902+0000 7fad4e600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fad4815ccf0 con 0x7fad480cd1b0 2024-04-20T11:36:50.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.902+0000 7fad4e600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fad4815cff0 con 0x7fad480b2880 2024-04-20T11:36:50.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.903+0000 7fad47e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fad48146000 0x7fad4806e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:50.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.903+0000 7fad47e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fad48146000 0x7fad4806e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:46454/0 (socket says 172.21.15.152:46454) 2024-04-20T11:36:50.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.903+0000 7fad47e00640 1 -- 172.21.15.152:0/3899153629 learned_addr learned my addr 172.21.15.152:0/3899153629 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:50.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.903+0000 7fad46a00640 1 --2- 172.21.15.152:0/3899153629 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad480cd1b0 0x7fad4813fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:50.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.903+0000 7fad47400640 1 --2- 172.21.15.152:0/3899153629 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fad480b2880 0x7fad4815bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:50.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.903+0000 7fad47e00640 1 -- 172.21.15.152:0/3899153629 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fad480b2880 msgr2=0x7fad4815bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:50.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.903+0000 7fad47e00640 1 --2- 172.21.15.152:0/3899153629 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fad480b2880 0x7fad4815bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:50.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.903+0000 7fad47e00640 1 -- 172.21.15.152:0/3899153629 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad480cd1b0 msgr2=0x7fad4813fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:50.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.903+0000 7fad47e00640 1 --2- 172.21.15.152:0/3899153629 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad480cd1b0 0x7fad4813fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:50.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.903+0000 7fad47e00640 1 -- 172.21.15.152:0/3899153629 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad4815c130 con 0x7fad48146000 2024-04-20T11:36:50.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.903+0000 7fad46a00640 1 --2- 172.21.15.152:0/3899153629 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad480cd1b0 0x7fad4813fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:36:50.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.904+0000 7fad47e00640 1 --2- 172.21.15.152:0/3899153629 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fad48146000 0x7fad4806e8d0 secure :-1 s=READY pgs=2102 cs=0 l=1 rev1=1 crypto rx=0x7fad40001fb0 tx=0x7fad4000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:50.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.904+0000 7fad3f400640 1 -- 172.21.15.152:0/3899153629 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad40017070 con 0x7fad48146000 2024-04-20T11:36:50.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.904+0000 7fad3f400640 1 -- 172.21.15.152:0/3899153629 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad40012400 con 0x7fad48146000 2024-04-20T11:36:50.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.904+0000 7fad4e600640 1 -- 172.21.15.152:0/3899153629 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad4813d280 con 0x7fad48146000 2024-04-20T11:36:50.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.904+0000 7fad3f400640 1 -- 172.21.15.152:0/3899153629 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad40012810 con 0x7fad48146000 2024-04-20T11:36:50.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.904+0000 7fad4e600640 1 -- 172.21.15.152:0/3899153629 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad4813d840 con 0x7fad48146000 2024-04-20T11:36:50.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.905+0000 7fad4e600640 1 -- 172.21.15.152:0/3899153629 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad480689f0 con 0x7fad48146000 2024-04-20T11:36:50.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.906+0000 7fad3f400640 1 -- 172.21.15.152:0/3899153629 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) ==== 91754+0+0 (secure 0 0 0) 0x7fad4001a070 con 0x7fad48146000 2024-04-20T11:36:50.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.908+0000 7fad3f400640 1 --2- 172.21.15.152:0/3899153629 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fad2806d100 0x7fad2806f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:50.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.908+0000 7fad3f400640 1 -- 172.21.15.152:0/3899153629 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 4943+0+0 (secure 0 0 0) 0x7fad4008a5e0 con 0x7fad48146000 2024-04-20T11:36:50.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.908+0000 7fad47400640 1 --2- 172.21.15.152:0/3899153629 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fad2806d100 0x7fad2806f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:50.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.910+0000 7fad47400640 1 --2- 172.21.15.152:0/3899153629 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fad2806d100 0x7fad2806f5c0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fad380011e0 tx=0x7fad38009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:50.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:50.912+0000 7fad3f400640 1 -- 172.21.15.152:0/3899153629 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fad4005a270 con 0x7fad48146000 2024-04-20T11:36:51.111 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:51.110+0000 7fad4e600640 1 -- 172.21.15.152:0/3899153629 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fad480b0d70 con 0x7fad48146000 2024-04-20T11:36:51.111 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:51.111+0000 7fad3f400640 1 -- 172.21.15.152:0/3899153629 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v145) ==== 93+0+10 (secure 0 0 0) 0x7fad40059b70 con 0x7fad48146000 2024-04-20T11:36:51.111 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 1 2024-04-20T11:36:51.114 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:51.113+0000 7fad4e600640 1 -- 172.21.15.152:0/3899153629 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fad2806d100 msgr2=0x7fad2806f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:51.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:51.113+0000 7fad4e600640 1 --2- 172.21.15.152:0/3899153629 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fad2806d100 0x7fad2806f5c0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fad380011e0 tx=0x7fad38009040 comp rx=0 tx=0).stop 2024-04-20T11:36:51.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:51.113+0000 7fad4e600640 1 -- 172.21.15.152:0/3899153629 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fad48146000 msgr2=0x7fad4806e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:51.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:51.113+0000 7fad4e600640 1 --2- 172.21.15.152:0/3899153629 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fad48146000 0x7fad4806e8d0 secure :-1 s=READY pgs=2102 cs=0 l=1 rev1=1 crypto rx=0x7fad40001fb0 tx=0x7fad4000a430 comp rx=0 tx=0).stop 2024-04-20T11:36:51.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:51.114+0000 7fad4e600640 1 -- 172.21.15.152:0/3899153629 shutdown_connections 2024-04-20T11:36:51.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:51.114+0000 7fad4e600640 1 --2- 172.21.15.152:0/3899153629 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fad2806d100 0x7fad2806f5c0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:51.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:51.114+0000 7fad4e600640 1 --2- 172.21.15.152:0/3899153629 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fad48146000 0x7fad4806e8d0 unknown :-1 s=CLOSED pgs=2102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:51.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:51.114+0000 7fad4e600640 1 --2- 172.21.15.152:0/3899153629 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad480cd1b0 0x7fad4813fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:51.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:51.114+0000 7fad4e600640 1 --2- 172.21.15.152:0/3899153629 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fad480b2880 0x7fad4815bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:51.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:51.114+0000 7fad4e600640 1 -- 172.21.15.152:0/3899153629 >> 172.21.15.152:0/3899153629 conn(0x7fad4806b980 msgr2=0x7fad48071db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:51.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:51.114+0000 7fad4e600640 1 -- 172.21.15.152:0/3899153629 shutdown_connections 2024-04-20T11:36:51.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:51.114+0000 7fad4e600640 1 -- 172.21.15.152:0/3899153629 wait complete. 2024-04-20T11:36:51.133 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_v1_to_v2' 2024-04-20T11:36:51.333 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:51.333+0000 7fe6f7000640 1 Processor -- start 2024-04-20T11:36:51.333 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:51.333+0000 7fe6f7000640 1 -- start start 2024-04-20T11:36:51.335 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:51.334+0000 7fe6f7000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fe6f00cd0d0 0x7fe6f00cd580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:51.335 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:51.334+0000 7fe6f7000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe6f0071fe0 0x7fe6f0072470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:51.335 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:51.334+0000 7fe6f7000640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6f0146340 con 0x7fe6f0145f60 2024-04-20T11:36:51.335 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:51.334+0000 7fe6f7000640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fe6f006e9f0 con 0x7fe6f00cd0d0 2024-04-20T11:36:51.335 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:51.334+0000 7fe6f7000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe6f0072d50 con 0x7fe6f0071fe0 2024-04-20T11:36:51.335 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:51.334+0000 7fe6ef400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fe6f00cd0d0 0x7fe6f00cd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:51.335 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:51.334+0000 7fe6efe00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fe6f0145f60 0x7fe6f006ee80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:60332/0 (socket says 172.21.15.152:60332) 2024-04-20T11:36:51.335 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:51.334+0000 7fe6efe00640 1 -- 172.21.15.152:0/600317747 learned_addr learned my addr 172.21.15.152:0/600317747 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:36:51.336 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:51.335+0000 7fe6f5000640 1 --2- 172.21.15.152:0/600317747 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe6f0071fe0 0x7fe6f0072470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:51.336 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:51.335+0000 7fe6ef400640 1 -- 172.21.15.152:0/600317747 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe6f0071fe0 msgr2=0x7fe6f0072470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:51.336 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:51.335+0000 7fe6ef400640 1 --2- 172.21.15.152:0/600317747 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe6f0071fe0 0x7fe6f0072470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:51.336 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:51.335+0000 7fe6ef400640 1 -- 172.21.15.152:0/600317747 >> v1:172.21.15.152:6789/0 conn(0x7fe6f0145f60 legacy=0x7fe6f006ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:51.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:51.335+0000 7fe6ef400640 1 -- 172.21.15.152:0/600317747 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6f006e0c0 con 0x7fe6f00cd0d0 2024-04-20T11:36:51.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:51.335+0000 7fe6f5000640 1 --2- 172.21.15.152:0/600317747 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe6f0071fe0 0x7fe6f0072470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:36:51.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:51.336+0000 7fe6ef400640 1 --2- 172.21.15.152:0/600317747 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe6f00cd0d0 0x7fe6f00cd580 secure :-1 s=READY pgs=2104 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc0097a0 tx=0x7fe6dc02b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=262ee95b8af9f16a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:51.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:51.336+0000 7fe6eea00640 1 -- 172.21.15.152:0/600317747 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6dc035020 con 0x7fe6f00cd0d0 2024-04-20T11:36:51.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:51.336+0000 7fe6eea00640 1 -- 172.21.15.152:0/600317747 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6dc031810 con 0x7fe6f00cd0d0 2024-04-20T11:36:51.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:51.336+0000 7fe6eea00640 1 -- 172.21.15.152:0/600317747 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6dc031be0 con 0x7fe6f00cd0d0 2024-04-20T11:36:51.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:51.337+0000 7fe6f7000640 1 -- 172.21.15.152:0/600317747 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe6f00cd0d0 msgr2=0x7fe6f00cd580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:51.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:51.337+0000 7fe6f7000640 1 --2- 172.21.15.152:0/600317747 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe6f00cd0d0 0x7fe6f00cd580 secure :-1 s=READY pgs=2104 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc0097a0 tx=0x7fe6dc02b8b0 comp rx=0 tx=0).stop 2024-04-20T11:36:51.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:51.337+0000 7fe6f7000640 1 -- 172.21.15.152:0/600317747 shutdown_connections 2024-04-20T11:36:51.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:51.337+0000 7fe6f7000640 1 --2- 172.21.15.152:0/600317747 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe6f0071fe0 0x7fe6f0072470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:51.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:51.337+0000 7fe6f7000640 1 --2- 172.21.15.152:0/600317747 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe6f00cd0d0 0x7fe6f00cd580 unknown :-1 s=CLOSED pgs=2104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:51.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:51.337+0000 7fe6f7000640 1 -- 172.21.15.152:0/600317747 >> 172.21.15.152:0/600317747 conn(0x7fe6f0067cf0 msgr2=0x7fe6f006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:51.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:51.337+0000 7fe6f7000640 1 -- 172.21.15.152:0/600317747 shutdown_connections 2024-04-20T11:36:51.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:51.337+0000 7fe6f7000640 1 -- 172.21.15.152:0/600317747 wait complete. 2024-04-20T11:36:51.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:51.339+0000 7fe6f7000640 1 Processor -- start 2024-04-20T11:36:51.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:51.339+0000 7fe6f7000640 1 -- start start 2024-04-20T11:36:51.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:51.340+0000 7fe6f7000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe6f0071fe0 0x7fe6f0157ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:51.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:51.340+0000 7fe6f7000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe6f00cd0d0 0x7fe6f01f60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:51.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:51.340+0000 7fe6f7000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe6f0145f60 0x7fe6f013fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:51.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:51.340+0000 7fe6f7000640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fe6f00729b0 con 0x7fe6f0071fe0 2024-04-20T11:36:51.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:51.340+0000 7fe6f7000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe6f006e680 con 0x7fe6f0145f60 2024-04-20T11:36:51.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:51.340+0000 7fe6f7000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe6f0072b30 con 0x7fe6f00cd0d0 2024-04-20T11:36:51.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:51.340+0000 7fe6efe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe6f0071fe0 0x7fe6f0157ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:51.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:51.340+0000 7fe6efe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe6f0071fe0 0x7fe6f0157ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:46468/0 (socket says 172.21.15.152:46468) 2024-04-20T11:36:51.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:51.340+0000 7fe6efe00640 1 -- 172.21.15.152:0/4001416661 learned_addr learned my addr 172.21.15.152:0/4001416661 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:51.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:51.340+0000 7fe6ef400640 1 --2- 172.21.15.152:0/4001416661 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe6f00cd0d0 0x7fe6f01f60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:51.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:51.340+0000 7fe6efe00640 1 -- 172.21.15.152:0/4001416661 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe6f00cd0d0 msgr2=0x7fe6f01f60d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:51.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:51.340+0000 7fe6efe00640 1 --2- 172.21.15.152:0/4001416661 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe6f00cd0d0 0x7fe6f01f60d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:51.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:51.340+0000 7fe6efe00640 1 -- 172.21.15.152:0/4001416661 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe6f0145f60 msgr2=0x7fe6f013fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:51.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:51.340+0000 7fe6f5000640 1 --2- 172.21.15.152:0/4001416661 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe6f0145f60 0x7fe6f013fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:51.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:51.341+0000 7fe6efe00640 1 --2- 172.21.15.152:0/4001416661 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe6f0145f60 0x7fe6f013fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:51.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:51.341+0000 7fe6efe00640 1 -- 172.21.15.152:0/4001416661 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6f013e080 con 0x7fe6f0071fe0 2024-04-20T11:36:51.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:51.341+0000 7fe6f5000640 1 --2- 172.21.15.152:0/4001416661 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe6f0145f60 0x7fe6f013fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:36:51.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:51.341+0000 7fe6efe00640 1 --2- 172.21.15.152:0/4001416661 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe6f0071fe0 0x7fe6f0157ae0 secure :-1 s=READY pgs=2105 cs=0 l=1 rev1=1 crypto rx=0x7fe6e000c3c0 tx=0x7fe6e000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:51.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:51.341+0000 7fe6e7e00640 1 -- 172.21.15.152:0/4001416661 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6e0017070 con 0x7fe6f0071fe0 2024-04-20T11:36:51.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:51.341+0000 7fe6e7e00640 1 -- 172.21.15.152:0/4001416661 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6e0005ce0 con 0x7fe6f0071fe0 2024-04-20T11:36:51.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:51.341+0000 7fe6e7e00640 1 -- 172.21.15.152:0/4001416661 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6e0007310 con 0x7fe6f0071fe0 2024-04-20T11:36:51.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:51.341+0000 7fe6f7000640 1 -- 172.21.15.152:0/4001416661 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6f01f6610 con 0x7fe6f0071fe0 2024-04-20T11:36:51.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:51.341+0000 7fe6f7000640 1 -- 172.21.15.152:0/4001416661 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6f013e600 con 0x7fe6f0071fe0 2024-04-20T11:36:51.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:51.342+0000 7fe6f7000640 1 -- 172.21.15.152:0/4001416661 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6f00689f0 con 0x7fe6f0071fe0 2024-04-20T11:36:51.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:51.343+0000 7fe6e7e00640 1 -- 172.21.15.152:0/4001416661 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) ==== 91754+0+0 (secure 0 0 0) 0x7fe6e001a070 con 0x7fe6f0071fe0 2024-04-20T11:36:51.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:51.344+0000 7fe6e7e00640 1 --2- 172.21.15.152:0/4001416661 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe6d006d0d0 0x7fe6d006f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:51.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:51.344+0000 7fe6e7e00640 1 -- 172.21.15.152:0/4001416661 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 4943+0+0 (secure 0 0 0) 0x7fe6e00898c0 con 0x7fe6f0071fe0 2024-04-20T11:36:51.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:51.345+0000 7fe6ef400640 1 --2- 172.21.15.152:0/4001416661 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe6d006d0d0 0x7fe6d006f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:51.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:51.346+0000 7fe6ef400640 1 --2- 172.21.15.152:0/4001416661 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe6d006d0d0 0x7fe6d006f590 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc036000 tx=0x7fe6dc008030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:51.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:51.349+0000 7fe6e7e00640 1 -- 172.21.15.152:0/4001416661 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe6e00595e0 con 0x7fe6f0071fe0 2024-04-20T11:36:51.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:51.617+0000 7fe6f7000640 1 -- 172.21.15.152:0/4001416661 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_v1_to_v2"]} v 0) -- 0x7fe6f006f840 con 0x7fe6f0071fe0 2024-04-20T11:36:52.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:52.231+0000 7fe6e7e00640 1 -- 172.21.15.152:0/4001416661 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_v1_to_v2"]}]=0 v528) ==== 148+0+0 (secure 0 0 0) 0x7fe6e0058ee0 con 0x7fe6f0071fe0 2024-04-20T11:36:52.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:52.234+0000 7fe6f7000640 1 -- 172.21.15.152:0/4001416661 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe6d006d0d0 msgr2=0x7fe6d006f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:52.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:52.234+0000 7fe6f7000640 1 --2- 172.21.15.152:0/4001416661 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe6d006d0d0 0x7fe6d006f590 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc036000 tx=0x7fe6dc008030 comp rx=0 tx=0).stop 2024-04-20T11:36:52.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:52.234+0000 7fe6f7000640 1 -- 172.21.15.152:0/4001416661 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe6f0071fe0 msgr2=0x7fe6f0157ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:52.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:52.234+0000 7fe6f7000640 1 --2- 172.21.15.152:0/4001416661 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe6f0071fe0 0x7fe6f0157ae0 secure :-1 s=READY pgs=2105 cs=0 l=1 rev1=1 crypto rx=0x7fe6e000c3c0 tx=0x7fe6e000c880 comp rx=0 tx=0).stop 2024-04-20T11:36:52.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:52.234+0000 7fe6f7000640 1 -- 172.21.15.152:0/4001416661 shutdown_connections 2024-04-20T11:36:52.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:52.234+0000 7fe6f7000640 1 --2- 172.21.15.152:0/4001416661 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe6d006d0d0 0x7fe6d006f590 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:52.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:52.234+0000 7fe6f7000640 1 --2- 172.21.15.152:0/4001416661 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe6f0145f60 0x7fe6f013fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:52.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:52.234+0000 7fe6f7000640 1 --2- 172.21.15.152:0/4001416661 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe6f00cd0d0 0x7fe6f01f60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:52.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:52.234+0000 7fe6f7000640 1 --2- 172.21.15.152:0/4001416661 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe6f0071fe0 0x7fe6f0157ae0 unknown :-1 s=CLOSED pgs=2105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:52.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:52.234+0000 7fe6f7000640 1 -- 172.21.15.152:0/4001416661 >> 172.21.15.152:0/4001416661 conn(0x7fe6f0067cf0 msgr2=0x7fe6f015bbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:52.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:52.234+0000 7fe6f7000640 1 -- 172.21.15.152:0/4001416661 shutdown_connections 2024-04-20T11:36:52.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:52.234+0000 7fe6f7000640 1 -- 172.21.15.152:0/4001416661 wait complete. 2024-04-20T11:36:52.256 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-20T11:36:52.456 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:52.456+0000 7fee2a600640 1 Processor -- start 2024-04-20T11:36:52.457 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:52.456+0000 7fee2a600640 1 -- start start 2024-04-20T11:36:52.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:52.457+0000 7fee2a600640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fee240d0fa0 0x7fee240b2530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:52.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:52.457+0000 7fee2a600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee2406c090 0x7fee2415e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:52.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:52.457+0000 7fee2a600640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee2415f3d0 con 0x7fee240b2a70 2024-04-20T11:36:52.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:52.457+0000 7fee2a600640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fee240d1380 con 0x7fee240d0fa0 2024-04-20T11:36:52.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:52.457+0000 7fee2a600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fee240b1e90 con 0x7fee2406c090 2024-04-20T11:36:52.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:52.458+0000 7fee22a00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fee240b2a70 0x7fee2406b980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:60336/0 (socket says 172.21.15.152:60336) 2024-04-20T11:36:52.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:52.458+0000 7fee22a00640 1 -- 172.21.15.152:0/3653241242 learned_addr learned my addr 172.21.15.152:0/3653241242 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:36:52.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:52.458+0000 7fee23400640 1 --2- 172.21.15.152:0/3653241242 >> v2:172.21.15.152:3300/0 conn(0x7fee240d0fa0 0x7fee240b2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:52.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:52.458+0000 7fee23e00640 1 --2- 172.21.15.152:0/3653241242 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee2406c090 0x7fee2415e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:52.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:52.458+0000 7fee22000640 1 -- 172.21.15.152:0/3653241242 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3743833140 0 0) 0x7fee2415f3d0 con 0x7fee240b2a70 2024-04-20T11:36:52.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:52.458+0000 7fee22000640 1 -- 172.21.15.152:0/3653241242 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fee08003520 con 0x7fee240b2a70 2024-04-20T11:36:52.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:52.458+0000 7fee23400640 1 -- 172.21.15.152:0/3653241242 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee2406c090 msgr2=0x7fee2415e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:52.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:52.458+0000 7fee23400640 1 --2- 172.21.15.152:0/3653241242 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee2406c090 0x7fee2415e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:52.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:52.458+0000 7fee23400640 1 -- 172.21.15.152:0/3653241242 >> v1:172.21.15.152:6789/0 conn(0x7fee240b2a70 legacy=0x7fee2406b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:52.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:52.458+0000 7fee23400640 1 -- 172.21.15.152:0/3653241242 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee2415f6b0 con 0x7fee240d0fa0 2024-04-20T11:36:52.461 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:52.460+0000 7fee23400640 1 --2- 172.21.15.152:0/3653241242 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fee240d0fa0 0x7fee240b2530 secure :-1 s=READY pgs=2107 cs=0 l=1 rev1=1 crypto rx=0x7fee140097a0 tx=0x7fee1402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1057016d3f09c5c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:52.461 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:52.460+0000 7fee22000640 1 -- 172.21.15.152:0/3653241242 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee140315a0 con 0x7fee240d0fa0 2024-04-20T11:36:52.461 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:52.460+0000 7fee22000640 1 -- 172.21.15.152:0/3653241242 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee14037070 con 0x7fee240d0fa0 2024-04-20T11:36:52.461 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:52.460+0000 7fee22000640 1 -- 172.21.15.152:0/3653241242 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee14031960 con 0x7fee240d0fa0 2024-04-20T11:36:52.461 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:52.460+0000 7fee2a600640 1 -- 172.21.15.152:0/3653241242 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fee240d0fa0 msgr2=0x7fee240b2530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:52.462 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:52.460+0000 7fee2a600640 1 --2- 172.21.15.152:0/3653241242 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fee240d0fa0 0x7fee240b2530 secure :-1 s=READY pgs=2107 cs=0 l=1 rev1=1 crypto rx=0x7fee140097a0 tx=0x7fee1402b840 comp rx=0 tx=0).stop 2024-04-20T11:36:52.462 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:52.461+0000 7fee2a600640 1 -- 172.21.15.152:0/3653241242 shutdown_connections 2024-04-20T11:36:52.462 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:52.461+0000 7fee2a600640 1 --2- 172.21.15.152:0/3653241242 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee2406c090 0x7fee2415e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:52.462 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:52.461+0000 7fee2a600640 1 --2- 172.21.15.152:0/3653241242 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fee240d0fa0 0x7fee240b2530 unknown :-1 s=CLOSED pgs=2107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:52.462 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:52.461+0000 7fee2a600640 1 -- 172.21.15.152:0/3653241242 >> 172.21.15.152:0/3653241242 conn(0x7fee2409aa40 msgr2=0x7fee240b3030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:52.462 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:52.461+0000 7fee2a600640 1 -- 172.21.15.152:0/3653241242 shutdown_connections 2024-04-20T11:36:52.462 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:52.461+0000 7fee2a600640 1 -- 172.21.15.152:0/3653241242 wait complete. 2024-04-20T11:36:52.463 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:52.462+0000 7fee2a600640 1 Processor -- start 2024-04-20T11:36:52.463 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:52.463+0000 7fee2a600640 1 -- start start 2024-04-20T11:36:52.464 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:52.463+0000 7fee2a600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee2406c090 0x7fee240c8780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:52.464 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:52.463+0000 7fee2a600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fee240b2a70 0x7fee240cd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:52.464 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:52.464+0000 7fee2a600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee240d0fa0 0x7fee24153990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:52.464 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:52.464+0000 7fee2a600640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fee24160190 con 0x7fee240b2a70 2024-04-20T11:36:52.465 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:52.464+0000 7fee2a600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fee2415fc70 con 0x7fee240d0fa0 2024-04-20T11:36:52.465 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:52.464+0000 7fee2a600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fee240ce5c0 con 0x7fee2406c090 2024-04-20T11:36:52.465 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:52.464+0000 7fee22a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fee240b2a70 0x7fee240cd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:52.465 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:52.464+0000 7fee22a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fee240b2a70 0x7fee240cd3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:46490/0 (socket says 172.21.15.152:46490) 2024-04-20T11:36:52.465 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:52.464+0000 7fee22a00640 1 -- 172.21.15.152:0/871738370 learned_addr learned my addr 172.21.15.152:0/871738370 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:52.466 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:52.464+0000 7fee22a00640 1 -- 172.21.15.152:0/871738370 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee2406c090 msgr2=0x7fee240c8780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:52.466 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:52.464+0000 7fee23400640 1 --2- 172.21.15.152:0/871738370 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee2406c090 0x7fee240c8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:52.466 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:52.464+0000 7fee23e00640 1 --2- 172.21.15.152:0/871738370 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee240d0fa0 0x7fee24153990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:52.466 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:52.464+0000 7fee22a00640 1 --2- 172.21.15.152:0/871738370 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee2406c090 0x7fee240c8780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:52.466 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:52.464+0000 7fee22a00640 1 -- 172.21.15.152:0/871738370 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee240d0fa0 msgr2=0x7fee24153990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:52.467 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:52.464+0000 7fee22a00640 1 --2- 172.21.15.152:0/871738370 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee240d0fa0 0x7fee24153990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:52.467 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:52.464+0000 7fee22a00640 1 -- 172.21.15.152:0/871738370 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee240c8e10 con 0x7fee240b2a70 2024-04-20T11:36:52.467 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:52.464+0000 7fee23400640 1 --2- 172.21.15.152:0/871738370 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee2406c090 0x7fee240c8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:36:52.467 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:52.464+0000 7fee23e00640 1 --2- 172.21.15.152:0/871738370 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee240d0fa0 0x7fee24153990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:36:52.467 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:52.465+0000 7fee22a00640 1 --2- 172.21.15.152:0/871738370 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fee240b2a70 0x7fee240cd3b0 secure :-1 s=READY pgs=2108 cs=0 l=1 rev1=1 crypto rx=0x7fee0c00cb70 tx=0x7fee0c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:52.467 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:52.465+0000 7fee1b400640 1 -- 172.21.15.152:0/871738370 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee0c017070 con 0x7fee240b2a70 2024-04-20T11:36:52.467 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:52.465+0000 7fee1b400640 1 -- 172.21.15.152:0/871738370 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee0c00ad40 con 0x7fee240b2a70 2024-04-20T11:36:52.467 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:52.465+0000 7fee1b400640 1 -- 172.21.15.152:0/871738370 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee0c012620 con 0x7fee240b2a70 2024-04-20T11:36:52.467 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:52.465+0000 7fee2a600640 1 -- 172.21.15.152:0/871738370 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee240c90f0 con 0x7fee240b2a70 2024-04-20T11:36:52.467 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:52.465+0000 7fee2a600640 1 -- 172.21.15.152:0/871738370 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee24160a90 con 0x7fee240b2a70 2024-04-20T11:36:52.468 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:52.466+0000 7fee2a600640 1 -- 172.21.15.152:0/871738370 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee241605a0 con 0x7fee240b2a70 2024-04-20T11:36:52.468 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:52.467+0000 7fee1b400640 1 -- 172.21.15.152:0/871738370 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) ==== 91754+0+0 (secure 0 0 0) 0x7fee0c01a070 con 0x7fee240b2a70 2024-04-20T11:36:52.468 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:52.468+0000 7fee1b400640 1 --2- 172.21.15.152:0/871738370 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fee0006d0d0 0x7fee0006f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:52.468 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:52.468+0000 7fee1b400640 1 -- 172.21.15.152:0/871738370 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 4943+0+0 (secure 0 0 0) 0x7fee0c08c940 con 0x7fee240b2a70 2024-04-20T11:36:52.469 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:52.469+0000 7fee23400640 1 --2- 172.21.15.152:0/871738370 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fee0006d0d0 0x7fee0006f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:52.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:52.470+0000 7fee23400640 1 --2- 172.21.15.152:0/871738370 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fee0006d0d0 0x7fee0006f590 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fee14036000 tx=0x7fee14008030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:52.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:52.472+0000 7fee1b400640 1 -- 172.21.15.152:0/871738370 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fee0c05a0e0 con 0x7fee240b2a70 2024-04-20T11:36:52.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:52.679+0000 7fee2a600640 1 -- 172.21.15.152:0/871738370 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fee240689f0 con 0x7fee240b2a70 2024-04-20T11:36:52.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:52.686+0000 7fee1b400640 1 -- 172.21.15.152:0/871738370 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v28)=0 v28) ==== 122+0+0 (secure 0 0 0) 0x7fee0c0599e0 con 0x7fee240b2a70 2024-04-20T11:36:52.691 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:52.689+0000 7fee2a600640 1 -- 172.21.15.152:0/871738370 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fee0006d0d0 msgr2=0x7fee0006f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:52.691 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:52.689+0000 7fee2a600640 1 --2- 172.21.15.152:0/871738370 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fee0006d0d0 0x7fee0006f590 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fee14036000 tx=0x7fee14008030 comp rx=0 tx=0).stop 2024-04-20T11:36:52.691 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:52.689+0000 7fee2a600640 1 -- 172.21.15.152:0/871738370 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fee240b2a70 msgr2=0x7fee240cd3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:52.691 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:52.689+0000 7fee2a600640 1 --2- 172.21.15.152:0/871738370 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fee240b2a70 0x7fee240cd3b0 secure :-1 s=READY pgs=2108 cs=0 l=1 rev1=1 crypto rx=0x7fee0c00cb70 tx=0x7fee0c00a430 comp rx=0 tx=0).stop 2024-04-20T11:36:52.691 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:52.689+0000 7fee2a600640 1 -- 172.21.15.152:0/871738370 shutdown_connections 2024-04-20T11:36:52.691 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:52.689+0000 7fee2a600640 1 --2- 172.21.15.152:0/871738370 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fee0006d0d0 0x7fee0006f590 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:52.691 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:52.689+0000 7fee2a600640 1 --2- 172.21.15.152:0/871738370 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee240d0fa0 0x7fee24153990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:52.691 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:52.689+0000 7fee2a600640 1 --2- 172.21.15.152:0/871738370 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fee240b2a70 0x7fee240cd3b0 unknown :-1 s=CLOSED pgs=2108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:52.691 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:52.689+0000 7fee2a600640 1 --2- 172.21.15.152:0/871738370 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee2406c090 0x7fee240c8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:52.692 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:52.689+0000 7fee2a600640 1 -- 172.21.15.152:0/871738370 >> 172.21.15.152:0/871738370 conn(0x7fee2409aa40 msgr2=0x7fee2415da60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:52.692 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:52.690+0000 7fee2a600640 1 -- 172.21.15.152:0/871738370 shutdown_connections 2024-04-20T11:36:52.692 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:52.690+0000 7fee2a600640 1 -- 172.21.15.152:0/871738370 wait complete. 2024-04-20T11:36:52.710 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-20T11:36:52.710 DEBUG:teuthology.orchestra.run.smithi152:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-20T11:36:52.814 DEBUG:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-20T11:36:52.932 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-20T11:36:52.933 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-20T11:36:52.933 DEBUG:teuthology.orchestra.run.smithi152:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:36:52.990 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-20T11:36:52.990 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T11:36:52.990 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:36:53.047 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-20T11:36:53.048 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-20T11:36:53.048 DEBUG:teuthology.orchestra.run.smithi152:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:36:53.105 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-20T11:36:53.106 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T11:36:53.106 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:36:53.161 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-20T11:36:53.162 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-20T11:36:53.162 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:36:53.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.374+0000 7f8fb7e00640 1 Processor -- start 2024-04-20T11:36:53.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.374+0000 7f8fb7e00640 1 -- start start 2024-04-20T11:36:53.377 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.376+0000 7f8fb7e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f8fb81603d0 0x7f8fb80b5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:53.377 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.376+0000 7f8fb7e00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8fb80d3350 con 0x7f8fb80b5480 2024-04-20T11:36:53.377 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.376+0000 7f8fb7e00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8fb80d3630 con 0x7f8fb815fcc0 2024-04-20T11:36:53.377 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.376+0000 7f8fb7e00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f8fb80d3910 con 0x7f8fb81603d0 2024-04-20T11:36:53.377 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.376+0000 7f8fb6000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f8fb81603d0 0x7f8fb80b5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:53.377 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.376+0000 7f8fb7400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f8fb80b5480 0x7f8fb80cf240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:60348/0 (socket says 172.21.15.152:60348) 2024-04-20T11:36:53.377 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.376+0000 7f8fb7400640 1 -- 172.21.15.152:0/951204978 learned_addr learned my addr 172.21.15.152:0/951204978 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:36:53.378 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.377+0000 7f8fb5600640 1 -- 172.21.15.152:0/951204978 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1094117221 0 0) 0x7f8fb80d3350 con 0x7f8fb80b5480 2024-04-20T11:36:53.378 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.377+0000 7f8fb5600640 1 -- 172.21.15.152:0/951204978 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8f9c003520 con 0x7f8fb80b5480 2024-04-20T11:36:53.378 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.377+0000 7f8fb6000640 1 -- 172.21.15.152:0/951204978 >> v1:172.21.15.175:6789/0 conn(0x7f8fb815fcc0 legacy=0x7f8fb80d4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:53.378 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.377+0000 7f8fb6000640 1 -- 172.21.15.152:0/951204978 >> v1:172.21.15.152:6789/0 conn(0x7f8fb80b5480 legacy=0x7f8fb80cf240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:53.378 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.377+0000 7f8fb6000640 1 -- 172.21.15.152:0/951204978 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fb80ced20 con 0x7f8fb81603d0 2024-04-20T11:36:53.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.378+0000 7f8fb6000640 1 --2- 172.21.15.152:0/951204978 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8fb81603d0 0x7f8fb80b5eb0 secure :-1 s=READY pgs=2110 cs=0 l=1 rev1=1 crypto rx=0x7f8fa4009790 tx=0x7f8fa402b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc6b3ce7772d7397 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:53.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.378+0000 7f8fb5600640 1 -- 172.21.15.152:0/951204978 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa4034070 con 0x7f8fb81603d0 2024-04-20T11:36:53.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.379+0000 7f8fb5600640 1 -- 172.21.15.152:0/951204978 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fa4030800 con 0x7f8fb81603d0 2024-04-20T11:36:53.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.379+0000 7f8fb5600640 1 -- 172.21.15.152:0/951204978 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa4030bd0 con 0x7f8fb81603d0 2024-04-20T11:36:53.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.379+0000 7f8fb7e00640 1 -- 172.21.15.152:0/951204978 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8fb81603d0 msgr2=0x7f8fb80b5eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:53.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.379+0000 7f8fb7e00640 1 --2- 172.21.15.152:0/951204978 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8fb81603d0 0x7f8fb80b5eb0 secure :-1 s=READY pgs=2110 cs=0 l=1 rev1=1 crypto rx=0x7f8fa4009790 tx=0x7f8fa402b940 comp rx=0 tx=0).stop 2024-04-20T11:36:53.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.379+0000 7f8fb7e00640 1 -- 172.21.15.152:0/951204978 shutdown_connections 2024-04-20T11:36:53.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.379+0000 7f8fb7e00640 1 --2- 172.21.15.152:0/951204978 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8fb81603d0 0x7f8fb80b5eb0 unknown :-1 s=CLOSED pgs=2110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:53.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.379+0000 7f8fb7e00640 1 -- 172.21.15.152:0/951204978 >> 172.21.15.152:0/951204978 conn(0x7f8fb808bfd0 msgr2=0x7f8fb80d4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:53.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.379+0000 7f8fb7e00640 1 -- 172.21.15.152:0/951204978 shutdown_connections 2024-04-20T11:36:53.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.379+0000 7f8fb7e00640 1 -- 172.21.15.152:0/951204978 wait complete. 2024-04-20T11:36:53.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.381+0000 7f8fb7e00640 1 Processor -- start 2024-04-20T11:36:53.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.381+0000 7f8fb7e00640 1 -- start start 2024-04-20T11:36:53.383 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.382+0000 7f8fb7e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8fb80b5480 0x7f8fb80ca740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:53.383 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.382+0000 7f8fb7e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8fb815fcc0 0x7f8fb80c82f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:53.383 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.382+0000 7f8fb7e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8fb81603d0 0x7f8fb80c8830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:53.383 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.382+0000 7f8fb7e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb80b6800 con 0x7f8fb81603d0 2024-04-20T11:36:53.383 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.382+0000 7f8fb7e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb80b6680 con 0x7f8fb80b5480 2024-04-20T11:36:53.383 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.382+0000 7f8fb7e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8fb80cf950 con 0x7f8fb815fcc0 2024-04-20T11:36:53.383 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.382+0000 7f8fb7400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8fb81603d0 0x7f8fb80c8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:53.383 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.383+0000 7f8fb7400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8fb81603d0 0x7f8fb80c8830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:46510/0 (socket says 172.21.15.152:46510) 2024-04-20T11:36:53.383 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.383+0000 7f8fb7400640 1 -- 172.21.15.152:0/685382882 learned_addr learned my addr 172.21.15.152:0/685382882 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:53.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.383+0000 7f8fb6000640 1 --2- 172.21.15.152:0/685382882 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8fb815fcc0 0x7f8fb80c82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:53.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.383+0000 7f8fb6a00640 1 --2- 172.21.15.152:0/685382882 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8fb80b5480 0x7f8fb80ca740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:53.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.383+0000 7f8fb7400640 1 -- 172.21.15.152:0/685382882 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8fb815fcc0 msgr2=0x7f8fb80c82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:53.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.383+0000 7f8fb7400640 1 --2- 172.21.15.152:0/685382882 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8fb815fcc0 0x7f8fb80c82f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:53.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.383+0000 7f8fb7400640 1 -- 172.21.15.152:0/685382882 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8fb80b5480 msgr2=0x7f8fb80ca740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:53.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.383+0000 7f8fb7400640 1 --2- 172.21.15.152:0/685382882 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8fb80b5480 0x7f8fb80ca740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:53.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.383+0000 7f8fb7400640 1 -- 172.21.15.152:0/685382882 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fb80c9f50 con 0x7f8fb81603d0 2024-04-20T11:36:53.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.383+0000 7f8fb6000640 1 --2- 172.21.15.152:0/685382882 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8fb815fcc0 0x7f8fb80c82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:36:53.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.383+0000 7f8fb7400640 1 --2- 172.21.15.152:0/685382882 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8fb81603d0 0x7f8fb80c8830 secure :-1 s=READY pgs=2111 cs=0 l=1 rev1=1 crypto rx=0x7f8fb000cb70 tx=0x7f8fb000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:53.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.384+0000 7f8faf400640 1 -- 172.21.15.152:0/685382882 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb0017070 con 0x7f8fb81603d0 2024-04-20T11:36:53.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.384+0000 7f8faf400640 1 -- 172.21.15.152:0/685382882 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fb000ad40 con 0x7f8fb81603d0 2024-04-20T11:36:53.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.384+0000 7f8faf400640 1 -- 172.21.15.152:0/685382882 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb0012620 con 0x7f8fb81603d0 2024-04-20T11:36:53.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.384+0000 7f8fb7e00640 1 -- 172.21.15.152:0/685382882 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fb80ca230 con 0x7f8fb81603d0 2024-04-20T11:36:53.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.384+0000 7f8fb7e00640 1 -- 172.21.15.152:0/685382882 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fb80cba80 con 0x7f8fb81603d0 2024-04-20T11:36:53.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.385+0000 7f8fb7e00640 1 -- 172.21.15.152:0/685382882 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fb8075e60 con 0x7f8fb81603d0 2024-04-20T11:36:53.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.386+0000 7f8faf400640 1 -- 172.21.15.152:0/685382882 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) ==== 91754+0+0 (secure 0 0 0) 0x7f8fb001a070 con 0x7f8fb81603d0 2024-04-20T11:36:53.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.387+0000 7f8faf400640 1 --2- 172.21.15.152:0/685382882 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f8f9806d0d0 0x7f8f9806f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:53.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.387+0000 7f8faf400640 1 -- 172.21.15.152:0/685382882 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 4943+0+0 (secure 0 0 0) 0x7f8fb008c960 con 0x7f8fb81603d0 2024-04-20T11:36:53.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.388+0000 7f8fb6a00640 1 --2- 172.21.15.152:0/685382882 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f8f9806d0d0 0x7f8f9806f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:53.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.389+0000 7f8fb6a00640 1 --2- 172.21.15.152:0/685382882 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f8f9806d0d0 0x7f8f9806f590 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f8fa80015a0 tx=0x7f8fa8009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:53.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.392+0000 7f8faf400640 1 -- 172.21.15.152:0/685382882 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f8fb005a100 con 0x7f8fb81603d0 2024-04-20T11:36:53.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.632+0000 7f8fb7e00640 1 -- 172.21.15.152:0/685382882 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8fb80d3060 con 0x7f8fb81603d0 2024-04-20T11:36:53.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.633+0000 7f8faf400640 1 -- 172.21.15.152:0/685382882 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) ==== 94+0+3878 (secure 0 0 0) 0x7f8fb0059a00 con 0x7f8fb81603d0 2024-04-20T11:36:53.634 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:36:53.634 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":99,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6275,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6834/721066448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":721066448},{"type":"v1","addr":"172.21.15.175:6834","nonce":721066448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":91},{"gid":7093,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1238181404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6833","nonce":1238181404},{"type":"v1","addr":"172.21.15.175:6835","nonce":1238181404}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":99},{"gid":9483,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6834/3026417634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":3026417634},{"type":"v1","addr":"172.21.15.152:6834","nonce":3026417634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":90},{"gid":9486,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/3399678659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6833","nonce":3399678659},{"type":"v1","addr":"172.21.15.152:6835","nonce":3399678659}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":91}],"filesystems":[]} 2024-04-20T11:36:53.634 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 99 2024-04-20T11:36:53.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.635+0000 7f8fb7e00640 1 -- 172.21.15.152:0/685382882 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f8f9806d0d0 msgr2=0x7f8f9806f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:53.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.635+0000 7f8fb7e00640 1 --2- 172.21.15.152:0/685382882 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f8f9806d0d0 0x7f8f9806f590 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f8fa80015a0 tx=0x7f8fa8009040 comp rx=0 tx=0).stop 2024-04-20T11:36:53.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.635+0000 7f8fb7e00640 1 -- 172.21.15.152:0/685382882 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8fb81603d0 msgr2=0x7f8fb80c8830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:53.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.635+0000 7f8fb7e00640 1 --2- 172.21.15.152:0/685382882 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8fb81603d0 0x7f8fb80c8830 secure :-1 s=READY pgs=2111 cs=0 l=1 rev1=1 crypto rx=0x7f8fb000cb70 tx=0x7f8fb000a430 comp rx=0 tx=0).stop 2024-04-20T11:36:53.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.636+0000 7f8fb7e00640 1 -- 172.21.15.152:0/685382882 shutdown_connections 2024-04-20T11:36:53.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.636+0000 7f8fb7e00640 1 --2- 172.21.15.152:0/685382882 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f8f9806d0d0 0x7f8f9806f590 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:53.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.636+0000 7f8fb7e00640 1 --2- 172.21.15.152:0/685382882 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8fb81603d0 0x7f8fb80c8830 unknown :-1 s=CLOSED pgs=2111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:53.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.636+0000 7f8fb7e00640 1 --2- 172.21.15.152:0/685382882 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8fb815fcc0 0x7f8fb80c82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:53.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.636+0000 7f8fb7e00640 1 --2- 172.21.15.152:0/685382882 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8fb80b5480 0x7f8fb80ca740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:53.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.636+0000 7f8fb7e00640 1 -- 172.21.15.152:0/685382882 >> 172.21.15.152:0/685382882 conn(0x7f8fb808bfd0 msgr2=0x7f8fb80d44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:53.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.636+0000 7f8fb7e00640 1 -- 172.21.15.152:0/685382882 shutdown_connections 2024-04-20T11:36:53.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.636+0000 7f8fb7e00640 1 -- 172.21.15.152:0/685382882 wait complete. 2024-04-20T11:36:53.654 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-20T11:36:53.655 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-20T11:36:53.655 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-20T11:36:53.655 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T11:36:53.655 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-20T11:36:53.655 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-20T11:36:53.655 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-20T11:36:53.656 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T11:36:53.656 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-20T11:36:53.656 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-20T11:36:53.656 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-20T11:36:53.656 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T11:36:53.656 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-20T11:36:53.656 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-20T11:36:53.656 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-20T11:36:53.657 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T11:36:53.674 INFO:tasks.ceph.mds.c:Stopped 2024-04-20T11:36:53.675 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-20T11:36:53.675 INFO:tasks.ceph.mds.a:Stopped 2024-04-20T11:36:53.675 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-20T11:36:53.679 INFO:tasks.ceph.mds.d:Stopped 2024-04-20T11:36:53.679 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-20T11:36:53.681 INFO:tasks.ceph.mds.b:Stopped 2024-04-20T11:36:53.681 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-20T11:36:53.722 INFO:tasks.ceph.mds.c:Started 2024-04-20T11:36:53.722 DEBUG:teuthology.parallel:result is None 2024-04-20T11:36:53.728 INFO:tasks.ceph.mds.a:Started 2024-04-20T11:36:53.728 DEBUG:teuthology.parallel:result is None 2024-04-20T11:36:53.729 INFO:tasks.ceph.mds.d:Started 2024-04-20T11:36:53.729 DEBUG:teuthology.parallel:result is None 2024-04-20T11:36:53.735 INFO:tasks.ceph.mds.b:Started 2024-04-20T11:36:53.735 DEBUG:teuthology.parallel:result is None 2024-04-20T11:36:53.735 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-20T11:36:53.872 INFO:tasks.ceph.mds.c.smithi152.stdout:starting mds.c at 2024-04-20T11:36:53.873 INFO:tasks.ceph.mds.a.smithi152.stdout:starting mds.a at 2024-04-20T11:36:53.883 INFO:tasks.ceph.mds.b.smithi175.stdout:starting mds.b at 2024-04-20T11:36:53.883 INFO:tasks.ceph.mds.d.smithi175.stdout:starting mds.d at 2024-04-20T11:36:53.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.914+0000 7fbc7f200640 1 Processor -- start 2024-04-20T11:36:53.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.915+0000 7fbc7f200640 1 -- start start 2024-04-20T11:36:53.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.916+0000 7fbc7f200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbc80161d30 0x7fbc800d0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:53.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.916+0000 7fbc7f200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fbc800d4ec0 0x7fbc800d15e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:53.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.916+0000 7fbc7f200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc800d1b20 con 0x7fbc800d2b80 2024-04-20T11:36:53.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.916+0000 7fbc7f200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fbc800d0540 con 0x7fbc800d4ec0 2024-04-20T11:36:53.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.916+0000 7fbc7f200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fbc800cfb70 con 0x7fbc80161d30 2024-04-20T11:36:53.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.916+0000 7fbc75400640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fbc800d4ec0 0x7fbc800d15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:53.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.916+0000 7fbc7de00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbc80161d30 0x7fbc800d0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:53.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.916+0000 7fbc75400640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fbc800d4ec0 0x7fbc800d15e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.152:42356/0 (socket says 172.21.15.152:42356) 2024-04-20T11:36:53.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.916+0000 7fbc75400640 1 -- 172.21.15.152:0/2601887563 learned_addr learned my addr 172.21.15.152:0/2601887563 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:53.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.916+0000 7fbc7e800640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7fbc800d2b80 0x7fbc8006c830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.152:52178/0 (socket says 172.21.15.152:52178) 2024-04-20T11:36:53.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.917+0000 7fbc75400640 1 -- 172.21.15.152:0/2601887563 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbc80161d30 msgr2=0x7fbc800d0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:53.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.917+0000 7fbc75400640 1 --2- 172.21.15.152:0/2601887563 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbc80161d30 0x7fbc800d0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:53.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.917+0000 7fbc75400640 1 -- 172.21.15.152:0/2601887563 >> v1:172.21.15.175:6789/0 conn(0x7fbc800d2b80 legacy=0x7fbc8006c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:53.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.917+0000 7fbc75400640 1 -- 172.21.15.152:0/2601887563 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc800d26a0 con 0x7fbc800d4ec0 2024-04-20T11:36:53.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.917+0000 7fbc7de00640 1 --2- 172.21.15.152:0/2601887563 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbc80161d30 0x7fbc800d0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:36:53.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.918+0000 7fbc75400640 1 --2- 172.21.15.152:0/2601887563 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbc800d4ec0 0x7fbc800d15e0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fbc68009850 tx=0x7fbc6802b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b93eeea8c066368 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:53.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.918+0000 7fbc7d400640 1 -- 172.21.15.152:0/2601887563 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc68004230 con 0x7fbc800d4ec0 2024-04-20T11:36:53.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.918+0000 7fbc7d400640 1 -- 172.21.15.152:0/2601887563 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc680043d0 con 0x7fbc800d4ec0 2024-04-20T11:36:53.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.918+0000 7fbc7d400640 1 -- 172.21.15.152:0/2601887563 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc68003810 con 0x7fbc800d4ec0 2024-04-20T11:36:53.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.919+0000 7fbc7f200640 1 -- 172.21.15.152:0/2601887563 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbc800d4ec0 msgr2=0x7fbc800d15e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:53.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.919+0000 7fbc7f200640 1 --2- 172.21.15.152:0/2601887563 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbc800d4ec0 0x7fbc800d15e0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fbc68009850 tx=0x7fbc6802b960 comp rx=0 tx=0).stop 2024-04-20T11:36:53.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.919+0000 7fbc7f200640 1 -- 172.21.15.152:0/2601887563 shutdown_connections 2024-04-20T11:36:53.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.919+0000 7fbc7f200640 1 --2- 172.21.15.152:0/2601887563 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbc800d4ec0 0x7fbc800d15e0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:53.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.919+0000 7fbc7f200640 1 --2- 172.21.15.152:0/2601887563 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbc80161d30 0x7fbc800d0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:53.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.919+0000 7fbc7f200640 1 -- 172.21.15.152:0/2601887563 >> 172.21.15.152:0/2601887563 conn(0x7fbc8008bfd0 msgr2=0x7fbc800d45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:53.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.919+0000 7fbc7f200640 1 -- 172.21.15.152:0/2601887563 shutdown_connections 2024-04-20T11:36:53.920 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:36:53.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.919+0000 7fbc7f200640 1 -- 172.21.15.152:0/2601887563 wait complete. 2024-04-20T11:36:53.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.921+0000 7fbc7f200640 1 Processor -- start 2024-04-20T11:36:53.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.921+0000 7fbc7f200640 1 -- start start 2024-04-20T11:36:53.922 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.922+0000 7fbc7f200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbc800d2b80 0x7fbc800cc000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:53.922 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.922+0000 7fbc7f200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc800d4ec0 0x7fbc800c9210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:53.922 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.922+0000 7fbc7f200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbc80161d30 0x7fbc801518a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:53.922 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.922+0000 7fbc7f200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fbc800cf800 con 0x7fbc800d4ec0 2024-04-20T11:36:53.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.922+0000 7fbc7f200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbc800cf520 con 0x7fbc800d2b80 2024-04-20T11:36:53.925 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:36:53.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.922+0000 7fbc7f200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fbc800cf1b0 con 0x7fbc80161d30 2024-04-20T11:36:53.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.922+0000 7fbc75400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc800d4ec0 0x7fbc800c9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:53.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.922+0000 7fbc75400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc800d4ec0 0x7fbc800c9210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:46566/0 (socket says 172.21.15.152:46566) 2024-04-20T11:36:53.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.922+0000 7fbc75400640 1 -- 172.21.15.152:0/1717582326 learned_addr learned my addr 172.21.15.152:0/1717582326 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:53.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.922+0000 7fbc7e800640 1 --2- 172.21.15.152:0/1717582326 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbc80161d30 0x7fbc801518a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:53.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.922+0000 7fbc7de00640 1 --2- 172.21.15.152:0/1717582326 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbc800d2b80 0x7fbc800cc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:53.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.923+0000 7fbc75400640 1 -- 172.21.15.152:0/1717582326 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbc80161d30 msgr2=0x7fbc801518a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:53.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.923+0000 7fbc75400640 1 --2- 172.21.15.152:0/1717582326 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbc80161d30 0x7fbc801518a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:53.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.923+0000 7fbc75400640 1 -- 172.21.15.152:0/1717582326 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbc800d2b80 msgr2=0x7fbc800cc000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:53.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.923+0000 7fbc75400640 1 --2- 172.21.15.152:0/1717582326 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbc800d2b80 0x7fbc800cc000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:53.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.923+0000 7fbc75400640 1 -- 172.21.15.152:0/1717582326 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc800cea40 con 0x7fbc800d4ec0 2024-04-20T11:36:53.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.923+0000 7fbc75400640 1 --2- 172.21.15.152:0/1717582326 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc800d4ec0 0x7fbc800c9210 secure :-1 s=READY pgs=2119 cs=0 l=1 rev1=1 crypto rx=0x7fbc680090b0 tx=0x7fbc680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:53.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.923+0000 7fbc76a00640 1 -- 172.21.15.152:0/1717582326 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc68048070 con 0x7fbc800d4ec0 2024-04-20T11:36:53.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.923+0000 7fbc76a00640 1 -- 172.21.15.152:0/1717582326 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc68003bd0 con 0x7fbc800d4ec0 2024-04-20T11:36:53.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.923+0000 7fbc76a00640 1 -- 172.21.15.152:0/1717582326 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc68006040 con 0x7fbc800d4ec0 2024-04-20T11:36:53.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.923+0000 7fbc7f200640 1 -- 172.21.15.152:0/1717582326 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc800cc690 con 0x7fbc800d4ec0 2024-04-20T11:36:53.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.923+0000 7fbc7f200640 1 -- 172.21.15.152:0/1717582326 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc800b42e0 con 0x7fbc800d4ec0 2024-04-20T11:36:53.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.925+0000 7fbc7f200640 1 -- 172.21.15.152:0/1717582326 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc80075e60 con 0x7fbc800d4ec0 2024-04-20T11:36:53.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.926+0000 7fbc76a00640 1 -- 172.21.15.152:0/1717582326 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) ==== 91754+0+0 (secure 0 0 0) 0x7fbc68004330 con 0x7fbc800d4ec0 2024-04-20T11:36:53.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.927+0000 7fbc76a00640 1 --2- 172.21.15.152:0/1717582326 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fbc5c06d0d0 0x7fbc5c06f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:53.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.927+0000 7fbc76a00640 1 -- 172.21.15.152:0/1717582326 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 4943+0+0 (secure 0 0 0) 0x7fbc680abe90 con 0x7fbc800d4ec0 2024-04-20T11:36:53.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.927+0000 7fbc7de00640 1 --2- 172.21.15.152:0/1717582326 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fbc5c06d0d0 0x7fbc5c06f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:53.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.928+0000 7fbc7de00640 1 --2- 172.21.15.152:0/1717582326 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fbc5c06d0d0 0x7fbc5c06f590 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fbc70009bd0 tx=0x7fbc7000b040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:53.932 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:53.931+0000 7fbc76a00640 1 -- 172.21.15.152:0/1717582326 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fbc68034020 con 0x7fbc800d4ec0 2024-04-20T11:36:54.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:54.146+0000 7fbc7f200640 1 -- 172.21.15.152:0/1717582326 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fbc80075050 con 0x7fbc800d4ec0 2024-04-20T11:36:55.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.268+0000 7fbc76a00640 1 -- 172.21.15.152:0/1717582326 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v150) ==== 97+0+0 (secure 0 0 0) 0x7fbc6807b560 con 0x7fbc800d4ec0 2024-04-20T11:36:55.269 INFO:teuthology.orchestra.run.smithi152.stderr: removed all blocklist entries 2024-04-20T11:36:55.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.272+0000 7fbc7f200640 1 -- 172.21.15.152:0/1717582326 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fbc5c06d0d0 msgr2=0x7fbc5c06f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:55.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.272+0000 7fbc7f200640 1 --2- 172.21.15.152:0/1717582326 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fbc5c06d0d0 0x7fbc5c06f590 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fbc70009bd0 tx=0x7fbc7000b040 comp rx=0 tx=0).stop 2024-04-20T11:36:55.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.272+0000 7fbc7f200640 1 -- 172.21.15.152:0/1717582326 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc800d4ec0 msgr2=0x7fbc800c9210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:55.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.272+0000 7fbc7f200640 1 --2- 172.21.15.152:0/1717582326 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc800d4ec0 0x7fbc800c9210 secure :-1 s=READY pgs=2119 cs=0 l=1 rev1=1 crypto rx=0x7fbc680090b0 tx=0x7fbc680046c0 comp rx=0 tx=0).stop 2024-04-20T11:36:55.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.272+0000 7fbc7f200640 1 -- 172.21.15.152:0/1717582326 shutdown_connections 2024-04-20T11:36:55.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.272+0000 7fbc7f200640 1 --2- 172.21.15.152:0/1717582326 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fbc5c06d0d0 0x7fbc5c06f590 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:55.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.272+0000 7fbc7f200640 1 --2- 172.21.15.152:0/1717582326 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbc80161d30 0x7fbc801518a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:55.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.272+0000 7fbc7f200640 1 --2- 172.21.15.152:0/1717582326 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc800d4ec0 0x7fbc800c9210 unknown :-1 s=CLOSED pgs=2119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:55.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.272+0000 7fbc7f200640 1 --2- 172.21.15.152:0/1717582326 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbc800d2b80 0x7fbc800cc000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:55.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.272+0000 7fbc7f200640 1 -- 172.21.15.152:0/1717582326 >> 172.21.15.152:0/1717582326 conn(0x7fbc8008bfd0 msgr2=0x7fbc800b35b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:55.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.272+0000 7fbc7f200640 1 -- 172.21.15.152:0/1717582326 shutdown_connections 2024-04-20T11:36:55.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.272+0000 7fbc7f200640 1 -- 172.21.15.152:0/1717582326 wait complete. 2024-04-20T11:36:55.294 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-20T11:36:55.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.486+0000 7fb93f000640 1 Processor -- start 2024-04-20T11:36:55.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.486+0000 7fb93f000640 1 -- start start 2024-04-20T11:36:55.488 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.488+0000 7fb93f000640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fb938162c50 0x7fb93806f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:55.488 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.488+0000 7fb93f000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fb93806f720 0x7fb938071cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:55.488 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.488+0000 7fb93f000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb938072210 0x7fb9380b3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:55.488 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.488+0000 7fb93f000640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fb938072660 con 0x7fb93806f720 2024-04-20T11:36:55.488 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.488+0000 7fb93f000640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fb93806ebe0 con 0x7fb938162c50 2024-04-20T11:36:55.489 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.488+0000 7fb93f000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb93806dcf0 con 0x7fb938072210 2024-04-20T11:36:55.489 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.488+0000 7fb93d200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fb93806f720 0x7fb938071cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:55.489 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.488+0000 7fb93d200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fb93806f720 0x7fb938071cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:46574/0 (socket says 172.21.15.152:46574) 2024-04-20T11:36:55.489 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.488+0000 7fb93d200640 1 -- 172.21.15.152:0/1337678990 learned_addr learned my addr 172.21.15.152:0/1337678990 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:55.489 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.488+0000 7fb93dc00640 1 --2- 172.21.15.152:0/1337678990 >> v2:172.21.15.175:3300/0 conn(0x7fb938162c50 0x7fb93806f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:55.489 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.488+0000 7fb93e600640 1 --2- 172.21.15.152:0/1337678990 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb938072210 0x7fb9380b3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:55.489 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.488+0000 7fb93d200640 1 -- 172.21.15.152:0/1337678990 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb938072210 msgr2=0x7fb9380b3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:55.489 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.488+0000 7fb93d200640 1 --2- 172.21.15.152:0/1337678990 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb938072210 0x7fb9380b3390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:55.489 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.488+0000 7fb93d200640 1 -- 172.21.15.152:0/1337678990 >> v2:172.21.15.175:3300/0 conn(0x7fb938162c50 msgr2=0x7fb93806f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:55.489 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.488+0000 7fb93d200640 1 --2- 172.21.15.152:0/1337678990 >> v2:172.21.15.175:3300/0 conn(0x7fb938162c50 0x7fb93806f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:55.489 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.488+0000 7fb93d200640 1 -- 172.21.15.152:0/1337678990 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb93806dfd0 con 0x7fb93806f720 2024-04-20T11:36:55.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.489+0000 7fb93d200640 1 --2- 172.21.15.152:0/1337678990 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb93806f720 0x7fb938071cd0 secure :-1 s=READY pgs=2120 cs=0 l=1 rev1=1 crypto rx=0x7fb93806c3b0 tx=0x7fb92402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1f84090facfa0a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:55.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.489+0000 7fb937e00640 1 -- 172.21.15.152:0/1337678990 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb924035020 con 0x7fb93806f720 2024-04-20T11:36:55.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.489+0000 7fb937e00640 1 -- 172.21.15.152:0/1337678990 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb924031870 con 0x7fb93806f720 2024-04-20T11:36:55.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.489+0000 7fb937e00640 1 -- 172.21.15.152:0/1337678990 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb924031c40 con 0x7fb93806f720 2024-04-20T11:36:55.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.490+0000 7fb93f000640 1 -- 172.21.15.152:0/1337678990 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb93806f720 msgr2=0x7fb938071cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:55.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.490+0000 7fb93f000640 1 --2- 172.21.15.152:0/1337678990 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb93806f720 0x7fb938071cd0 secure :-1 s=READY pgs=2120 cs=0 l=1 rev1=1 crypto rx=0x7fb93806c3b0 tx=0x7fb92402b910 comp rx=0 tx=0).stop 2024-04-20T11:36:55.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.490+0000 7fb93f000640 1 -- 172.21.15.152:0/1337678990 shutdown_connections 2024-04-20T11:36:55.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.490+0000 7fb93f000640 1 --2- 172.21.15.152:0/1337678990 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb938072210 0x7fb9380b3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:55.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.490+0000 7fb93f000640 1 --2- 172.21.15.152:0/1337678990 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb93806f720 0x7fb938071cd0 unknown :-1 s=CLOSED pgs=2120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:55.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.490+0000 7fb93f000640 1 --2- 172.21.15.152:0/1337678990 >> v2:172.21.15.175:3300/0 conn(0x7fb938162c50 0x7fb93806f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:55.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.490+0000 7fb93f000640 1 -- 172.21.15.152:0/1337678990 >> 172.21.15.152:0/1337678990 conn(0x7fb938067ae0 msgr2=0x7fb93806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:55.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.490+0000 7fb93f000640 1 -- 172.21.15.152:0/1337678990 shutdown_connections 2024-04-20T11:36:55.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.490+0000 7fb93f000640 1 -- 172.21.15.152:0/1337678990 wait complete. 2024-04-20T11:36:55.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.492+0000 7fb93f000640 1 Processor -- start 2024-04-20T11:36:55.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.492+0000 7fb93f000640 1 -- start start 2024-04-20T11:36:55.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.493+0000 7fb93f000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb93806f720 0x7fb9381fc340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:55.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.493+0000 7fb93f000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb938072210 0x7fb938157fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:55.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.493+0000 7fb93f000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb938162c50 0x7fb938158500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:55.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.493+0000 7fb93f000640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fb93806e870 con 0x7fb938162c50 2024-04-20T11:36:55.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.493+0000 7fb93f000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb93806e590 con 0x7fb93806f720 2024-04-20T11:36:55.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.493+0000 7fb93f000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb9380b38d0 con 0x7fb938072210 2024-04-20T11:36:55.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.493+0000 7fb93e600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb938162c50 0x7fb938158500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:55.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.493+0000 7fb93e600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb938162c50 0x7fb938158500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:46578/0 (socket says 172.21.15.152:46578) 2024-04-20T11:36:55.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.493+0000 7fb93e600640 1 -- 172.21.15.152:0/3696390563 learned_addr learned my addr 172.21.15.152:0/3696390563 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:55.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.494+0000 7fb93d200640 1 --2- 172.21.15.152:0/3696390563 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb938072210 0x7fb938157fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:55.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.494+0000 7fb93dc00640 1 --2- 172.21.15.152:0/3696390563 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb93806f720 0x7fb9381fc340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:55.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.494+0000 7fb93e600640 1 -- 172.21.15.152:0/3696390563 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb938072210 msgr2=0x7fb938157fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:55.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.494+0000 7fb93e600640 1 --2- 172.21.15.152:0/3696390563 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb938072210 0x7fb938157fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:55.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.494+0000 7fb93e600640 1 -- 172.21.15.152:0/3696390563 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb93806f720 msgr2=0x7fb9381fc340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:55.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.494+0000 7fb93e600640 1 --2- 172.21.15.152:0/3696390563 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb93806f720 0x7fb9381fc340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:55.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.494+0000 7fb93e600640 1 -- 172.21.15.152:0/3696390563 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9381fc880 con 0x7fb938162c50 2024-04-20T11:36:55.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.494+0000 7fb93dc00640 1 --2- 172.21.15.152:0/3696390563 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb93806f720 0x7fb9381fc340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:36:55.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.494+0000 7fb93e600640 1 --2- 172.21.15.152:0/3696390563 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb938162c50 0x7fb938158500 secure :-1 s=READY pgs=2121 cs=0 l=1 rev1=1 crypto rx=0x7fb930007e60 tx=0x7fb93000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:55.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.494+0000 7fb935600640 1 -- 172.21.15.152:0/3696390563 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb930014620 con 0x7fb938162c50 2024-04-20T11:36:55.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.494+0000 7fb935600640 1 -- 172.21.15.152:0/3696390563 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb930019070 con 0x7fb938162c50 2024-04-20T11:36:55.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.494+0000 7fb935600640 1 -- 172.21.15.152:0/3696390563 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb930014a20 con 0x7fb938162c50 2024-04-20T11:36:55.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.494+0000 7fb93f000640 1 -- 172.21.15.152:0/3696390563 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb938156a90 con 0x7fb938162c50 2024-04-20T11:36:55.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.494+0000 7fb93f000640 1 -- 172.21.15.152:0/3696390563 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb938157050 con 0x7fb938162c50 2024-04-20T11:36:55.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.496+0000 7fb93f000640 1 -- 172.21.15.152:0/3696390563 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb938075e60 con 0x7fb938162c50 2024-04-20T11:36:55.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.496+0000 7fb935600640 1 -- 172.21.15.152:0/3696390563 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) ==== 91754+0+0 (secure 0 0 0) 0x7fb930014bc0 con 0x7fb938162c50 2024-04-20T11:36:55.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.497+0000 7fb935600640 1 --2- 172.21.15.152:0/3696390563 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb91806d0d0 0x7fb91806f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:55.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.497+0000 7fb935600640 1 -- 172.21.15.152:0/3696390563 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 4857+0+0 (secure 0 0 0) 0x7fb93008ae00 con 0x7fb938162c50 2024-04-20T11:36:55.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.498+0000 7fb93dc00640 1 --2- 172.21.15.152:0/3696390563 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb91806d0d0 0x7fb91806f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:55.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.499+0000 7fb93dc00640 1 --2- 172.21.15.152:0/3696390563 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb91806d0d0 0x7fb91806f590 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fb92c006fd0 tx=0x7fb92c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:55.502 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.502+0000 7fb935600640 1 -- 172.21.15.152:0/3696390563 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fb93005ab20 con 0x7fb938162c50 2024-04-20T11:36:55.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.775+0000 7fb93f000640 1 -- 172.21.15.152:0/3696390563 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fb938165670 con 0x7fb938162c50 2024-04-20T11:36:55.776 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.775+0000 7fb935600640 1 -- 172.21.15.152:0/3696390563 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v21) ==== 80+0+5304 (secure 0 0 0) 0x7fb93005a420 con 0x7fb938162c50 2024-04-20T11:36:55.776 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:36:55.776 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-04-20T11:36:55.776 INFO:teuthology.orchestra.run.smithi152.stdout: "auth_dump": [ 2024-04-20T11:36:55.776 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:36:55.776 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "mds.a", 2024-04-20T11:36:55.776 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBCpiNmyFRJIBAAj2aIMXdb65nqRgKFDBLLzA==", 2024-04-20T11:36:55.777 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:36:55.777 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow", 2024-04-20T11:36:55.777 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow *", 2024-04-20T11:36:55.777 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow *", 2024-04-20T11:36:55.777 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:36:55.777 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:36:55.777 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:36:55.777 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:36:55.777 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "mds.b", 2024-04-20T11:36:55.777 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBCpiNmVPytOhAAzQkvNOJaNy+MeKETWhYLyg==", 2024-04-20T11:36:55.777 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:36:55.777 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow", 2024-04-20T11:36:55.777 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow *", 2024-04-20T11:36:55.777 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow *", 2024-04-20T11:36:55.778 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:36:55.778 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:36:55.778 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:36:55.778 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:36:55.778 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "mds.c", 2024-04-20T11:36:55.778 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBCpiNm5ViKLRAAg7VBQG4hFTLYUpJMpBAaVA==", 2024-04-20T11:36:55.778 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:36:55.778 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow", 2024-04-20T11:36:55.778 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow *", 2024-04-20T11:36:55.778 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow *", 2024-04-20T11:36:55.778 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:36:55.778 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:36:55.778 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:36:55.779 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:36:55.779 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "mds.d", 2024-04-20T11:36:55.779 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBDpiNmLWhaCxAA2e4qd/7+TKenJlWICIsvGg==", 2024-04-20T11:36:55.779 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:36:55.779 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow", 2024-04-20T11:36:55.779 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow *", 2024-04-20T11:36:55.779 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow *", 2024-04-20T11:36:55.779 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:36:55.779 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:36:55.779 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:36:55.779 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:36:55.779 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "osd.0", 2024-04-20T11:36:55.779 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBHpiNmpuhyDBAAsQXX9DNvWlo0UYh53lb+FA==", 2024-04-20T11:36:55.779 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:36:55.780 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow profile osd", 2024-04-20T11:36:55.780 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile osd", 2024-04-20T11:36:55.780 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:36:55.780 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:36:55.780 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:36:55.780 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:36:55.780 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "osd.1", 2024-04-20T11:36:55.780 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBKpiNmxBiLJhAAHon1adC8G2jbYoDcWdE9jA==", 2024-04-20T11:36:55.780 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:36:55.780 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow profile osd", 2024-04-20T11:36:55.780 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile osd", 2024-04-20T11:36:55.780 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:36:55.780 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:36:55.780 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:36:55.780 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:36:55.781 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "osd.2", 2024-04-20T11:36:55.781 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBNpiNmdn0zMhAAqwIO2kKahdSzTVh8/p3FCg==", 2024-04-20T11:36:55.781 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:36:55.781 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow profile osd", 2024-04-20T11:36:55.781 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile osd", 2024-04-20T11:36:55.781 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:36:55.781 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:36:55.781 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:36:55.781 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:36:55.781 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "osd.3", 2024-04-20T11:36:55.781 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBRpiNmTPCYAhAAHFNHX/xKn21jEFHDkHMjEw==", 2024-04-20T11:36:55.781 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:36:55.781 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow profile osd", 2024-04-20T11:36:55.782 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile osd", 2024-04-20T11:36:55.782 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:36:55.782 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:36:55.782 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:36:55.782 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:36:55.782 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "osd.4", 2024-04-20T11:36:55.782 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBXpiNmM8cVFxAAJl9Hu+MvlkEiMglutI89eg==", 2024-04-20T11:36:55.782 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:36:55.782 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow profile osd", 2024-04-20T11:36:55.782 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile osd", 2024-04-20T11:36:55.782 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:36:55.782 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:36:55.782 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:36:55.782 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:36:55.783 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "osd.5", 2024-04-20T11:36:55.783 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBapiNmnGxJMRAA90P0T+VsTA8LCTcexTVr7w==", 2024-04-20T11:36:55.783 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:36:55.783 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow profile osd", 2024-04-20T11:36:55.783 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile osd", 2024-04-20T11:36:55.783 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:36:55.783 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:36:55.783 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:36:55.783 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:36:55.783 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "osd.6", 2024-04-20T11:36:55.783 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBepiNmyLXWDxAAhtHD5h478mLovE5IejAFng==", 2024-04-20T11:36:55.783 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:36:55.783 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow profile osd", 2024-04-20T11:36:55.784 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile osd", 2024-04-20T11:36:55.784 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:36:55.784 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:36:55.784 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:36:55.784 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:36:55.784 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "osd.7", 2024-04-20T11:36:55.784 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBhpiNmB3jwHBAAqco9om6r2N/GqAV6rVKiaA==", 2024-04-20T11:36:55.784 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:36:55.784 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow profile osd", 2024-04-20T11:36:55.784 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile osd", 2024-04-20T11:36:55.784 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:36:55.784 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:36:55.784 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:36:55.784 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:36:55.784 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "client.0", 2024-04-20T11:36:55.785 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBDpiNmtsBWFhAArq9A7r33CcwgMmOU2pAGPw==", 2024-04-20T11:36:55.785 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:36:55.785 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow", 2024-04-20T11:36:55.785 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow r", 2024-04-20T11:36:55.785 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-20T11:36:55.785 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:36:55.785 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:36:55.785 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:36:55.785 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "client.1", 2024-04-20T11:36:55.785 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBDpiNm6aNnJxAAB431GSGytZ0IKWMh4DMalw==", 2024-04-20T11:36:55.785 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:36:55.785 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow", 2024-04-20T11:36:55.785 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow r", 2024-04-20T11:36:55.785 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow rw", 2024-04-20T11:36:55.786 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow rwx" 2024-04-20T11:36:55.786 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:36:55.786 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:36:55.786 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:36:55.786 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "client.2", 2024-04-20T11:36:55.786 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBDpiNm5QLtHRAAa2kHTDVdgSh/B4XGZaKa5Q==", 2024-04-20T11:36:55.786 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:36:55.786 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow", 2024-04-20T11:36:55.786 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow r", 2024-04-20T11:36:55.786 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow rw", 2024-04-20T11:36:55.786 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow rwx" 2024-04-20T11:36:55.786 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:36:55.786 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:36:55.786 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:36:55.787 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "client.3", 2024-04-20T11:36:55.787 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBDpiNmw95SMBAAdodzTuPp4WdKSEf+jI8Uuw==", 2024-04-20T11:36:55.787 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:36:55.787 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow", 2024-04-20T11:36:55.787 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow r", 2024-04-20T11:36:55.787 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow rw", 2024-04-20T11:36:55.787 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow rwx" 2024-04-20T11:36:55.787 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:36:55.787 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:36:55.787 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:36:55.787 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "client.admin", 2024-04-20T11:36:55.787 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBBpiNm3xz6HhAA82+N+trDW/AmBr3q8l4ECw==", 2024-04-20T11:36:55.787 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:36:55.787 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow *", 2024-04-20T11:36:55.788 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow *", 2024-04-20T11:36:55.788 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow *", 2024-04-20T11:36:55.788 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:36:55.788 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:36:55.788 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:36:55.788 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:36:55.788 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "mgr.x", 2024-04-20T11:36:55.788 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBCpiNmuhalFxAA9dTl/zSbUovCCZAanAipww==", 2024-04-20T11:36:55.788 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:36:55.788 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow *", 2024-04-20T11:36:55.788 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile mgr", 2024-04-20T11:36:55.788 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:36:55.788 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:36:55.789 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:36:55.789 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:36:55.789 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "mgr.y", 2024-04-20T11:36:55.789 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBCpiNmP1+5DRAAs6bVAEOr0i6FdPmSEyo/vQ==", 2024-04-20T11:36:55.789 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:36:55.789 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow *", 2024-04-20T11:36:55.789 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile mgr", 2024-04-20T11:36:55.789 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:36:55.789 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:36:55.789 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:36:55.789 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-04-20T11:36:55.789 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-04-20T11:36:55.790 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.778+0000 7fb93f000640 1 -- 172.21.15.152:0/3696390563 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb91806d0d0 msgr2=0x7fb91806f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:55.790 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.778+0000 7fb93f000640 1 --2- 172.21.15.152:0/3696390563 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb91806d0d0 0x7fb91806f590 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fb92c006fd0 tx=0x7fb92c008040 comp rx=0 tx=0).stop 2024-04-20T11:36:55.790 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.778+0000 7fb93f000640 1 -- 172.21.15.152:0/3696390563 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb938162c50 msgr2=0x7fb938158500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:55.790 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.778+0000 7fb93f000640 1 --2- 172.21.15.152:0/3696390563 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb938162c50 0x7fb938158500 secure :-1 s=READY pgs=2121 cs=0 l=1 rev1=1 crypto rx=0x7fb930007e60 tx=0x7fb93000e4b0 comp rx=0 tx=0).stop 2024-04-20T11:36:55.791 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.778+0000 7fb93f000640 1 -- 172.21.15.152:0/3696390563 shutdown_connections 2024-04-20T11:36:55.791 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.778+0000 7fb93f000640 1 --2- 172.21.15.152:0/3696390563 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb91806d0d0 0x7fb91806f590 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:55.791 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.778+0000 7fb93f000640 1 --2- 172.21.15.152:0/3696390563 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb938162c50 0x7fb938158500 unknown :-1 s=CLOSED pgs=2121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:55.791 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.778+0000 7fb93f000640 1 --2- 172.21.15.152:0/3696390563 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb938072210 0x7fb938157fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:55.791 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.778+0000 7fb93f000640 1 --2- 172.21.15.152:0/3696390563 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb93806f720 0x7fb9381fc340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:55.791 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.778+0000 7fb93f000640 1 -- 172.21.15.152:0/3696390563 >> 172.21.15.152:0/3696390563 conn(0x7fb938067ae0 msgr2=0x7fb9380b0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:55.791 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.778+0000 7fb93f000640 1 -- 172.21.15.152:0/3696390563 shutdown_connections 2024-04-20T11:36:55.791 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.778+0000 7fb93f000640 1 -- 172.21.15.152:0/3696390563 wait complete. 2024-04-20T11:36:55.796 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:36:55.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.987+0000 7f2572400640 1 Processor -- start 2024-04-20T11:36:55.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.987+0000 7f2572400640 1 -- start start 2024-04-20T11:36:55.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.987+0000 7f2572400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f256c13e530 0x7f256c06ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:55.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.987+0000 7f2572400640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f256c06f340 0x7f256c140dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:55.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.987+0000 7f2572400640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f256c0cf0a0 con 0x7f256c141310 2024-04-20T11:36:55.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.987+0000 7f2572400640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f256c141b00 con 0x7f256c06f340 2024-04-20T11:36:55.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.987+0000 7f2572400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f256c0cfc50 con 0x7f256c13e530 2024-04-20T11:36:55.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.988+0000 7f2571a00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f256c141310 0x7f256c15efd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:52140/0 (socket says 172.21.15.152:52140) 2024-04-20T11:36:55.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.988+0000 7f2571a00640 1 -- 172.21.15.152:0/4171004411 learned_addr learned my addr 172.21.15.152:0/4171004411 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:36:55.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.988+0000 7f2571000640 1 --2- 172.21.15.152:0/4171004411 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f256c13e530 0x7f256c06ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:55.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.988+0000 7f2563e00640 1 --2- 172.21.15.152:0/4171004411 >> v2:172.21.15.175:3300/0 conn(0x7f256c06f340 0x7f256c140dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:55.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.988+0000 7f256be00640 1 -- 172.21.15.152:0/4171004411 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 945062240 0 0) 0x7f2564001490 con 0x7f256c141310 2024-04-20T11:36:55.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.988+0000 7f256be00640 1 -- 172.21.15.152:0/4171004411 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2550003520 con 0x7f256c141310 2024-04-20T11:36:55.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.989+0000 7f256be00640 1 -- 172.21.15.152:0/4171004411 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 532747740 0 0) 0x7f2550003520 con 0x7f256c141310 2024-04-20T11:36:55.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.989+0000 7f256be00640 1 -- 172.21.15.152:0/4171004411 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f256c13e530 msgr2=0x7f256c06ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:55.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.989+0000 7f256be00640 1 --2- 172.21.15.152:0/4171004411 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f256c13e530 0x7f256c06ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:55.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.989+0000 7f256be00640 1 -- 172.21.15.152:0/4171004411 >> v2:172.21.15.175:3300/0 conn(0x7f256c06f340 msgr2=0x7f256c140dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:55.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.989+0000 7f256be00640 1 --2- 172.21.15.152:0/4171004411 >> v2:172.21.15.175:3300/0 conn(0x7f256c06f340 0x7f256c140dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:55.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.989+0000 7f256be00640 1 -- 172.21.15.152:0/4171004411 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f256c0d00e0 con 0x7f256c141310 2024-04-20T11:36:55.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.989+0000 7f256be00640 1 -- 172.21.15.152:0/4171004411 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f2564001bc0 con 0x7f256c141310 2024-04-20T11:36:55.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.989+0000 7f256be00640 1 -- 172.21.15.152:0/4171004411 >> v1:172.21.15.152:6789/0 conn(0x7f256c141310 legacy=0x7f256c15efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:55.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.989+0000 7f256be00640 1 --2- 172.21.15.152:0/4171004411 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25500039b0 0x7f2550003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:55.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.989+0000 7f256be00640 1 --2- 172.21.15.152:0/4171004411 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2550004800 0x7f2550004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:55.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.989+0000 7f256be00640 1 --2- 172.21.15.152:0/4171004411 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f256c13e530 0x7f25500051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:55.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.989+0000 7f256be00640 1 -- 172.21.15.152:0/4171004411 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f256c0cfc50 con 0x7f2550004800 2024-04-20T11:36:55.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.989+0000 7f256be00640 1 -- 172.21.15.152:0/4171004411 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f255000b9c0 con 0x7f25500039b0 2024-04-20T11:36:55.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.989+0000 7f256be00640 1 -- 172.21.15.152:0/4171004411 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f255000bb40 con 0x7f256c13e530 2024-04-20T11:36:55.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.989+0000 7f2563e00640 1 --2- 172.21.15.152:0/4171004411 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2550004800 0x7f2550004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:55.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.990+0000 7f2571000640 1 --2- 172.21.15.152:0/4171004411 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25500039b0 0x7f2550003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:55.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.990+0000 7f2571a00640 1 --2- 172.21.15.152:0/4171004411 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f256c13e530 0x7f25500051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:55.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.990+0000 7f2563e00640 1 -- 172.21.15.152:0/4171004411 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f256c13e530 msgr2=0x7f25500051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:55.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.990+0000 7f2563e00640 1 --2- 172.21.15.152:0/4171004411 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f256c13e530 0x7f25500051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:55.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.990+0000 7f2563e00640 1 -- 172.21.15.152:0/4171004411 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25500039b0 msgr2=0x7f2550003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:55.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.990+0000 7f2563e00640 1 --2- 172.21.15.152:0/4171004411 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25500039b0 0x7f2550003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:55.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.990+0000 7f2563e00640 1 -- 172.21.15.152:0/4171004411 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2550003520 con 0x7f2550004800 2024-04-20T11:36:55.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.991+0000 7f2563e00640 1 --2- 172.21.15.152:0/4171004411 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2550004800 0x7f2550004c80 secure :-1 s=READY pgs=2123 cs=0 l=1 rev1=1 crypto rx=0x7f2554009ac0 tx=0x7f2550005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3361173808f31630 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:55.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.991+0000 7f256be00640 1 -- 172.21.15.152:0/4171004411 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2554039070 con 0x7f2550004800 2024-04-20T11:36:55.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.991+0000 7f256be00640 1 -- 172.21.15.152:0/4171004411 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f255402c710 con 0x7f2550004800 2024-04-20T11:36:55.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.991+0000 7f256be00640 1 -- 172.21.15.152:0/4171004411 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2554009690 con 0x7f2550004800 2024-04-20T11:36:55.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.992+0000 7f2572400640 1 -- 172.21.15.152:0/4171004411 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2550004800 msgr2=0x7f2550004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:55.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.992+0000 7f2572400640 1 --2- 172.21.15.152:0/4171004411 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2550004800 0x7f2550004c80 secure :-1 s=READY pgs=2123 cs=0 l=1 rev1=1 crypto rx=0x7f2554009ac0 tx=0x7f2550005830 comp rx=0 tx=0).stop 2024-04-20T11:36:55.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.992+0000 7f2571a00640 1 -- 172.21.15.152:0/4171004411 reap_dead start 2024-04-20T11:36:55.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.992+0000 7f2572400640 1 -- 172.21.15.152:0/4171004411 shutdown_connections 2024-04-20T11:36:55.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.992+0000 7f2572400640 1 -- 172.21.15.152:0/4171004411 >> 172.21.15.152:0/4171004411 conn(0x7f256c067ae0 msgr2=0x7f256c06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:55.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.992+0000 7f2572400640 1 -- 172.21.15.152:0/4171004411 shutdown_connections 2024-04-20T11:36:55.995 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.992+0000 7f2572400640 1 -- 172.21.15.152:0/4171004411 wait complete. 2024-04-20T11:36:55.997 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.994+0000 7f2572400640 1 Processor -- start 2024-04-20T11:36:55.997 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.994+0000 7f2572400640 1 -- start start 2024-04-20T11:36:55.997 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.995+0000 7f2572400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2550004800 0x7f256c13ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:55.997 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.995+0000 7f2572400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f256c1588a0 0x7f256c158d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:55.997 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.995+0000 7f2572400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f256c152b70 0x7f256c152ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:55.997 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.995+0000 7f2572400640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f256c0cf600 con 0x7f256c1588a0 2024-04-20T11:36:55.997 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.995+0000 7f2572400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f256c072ce0 con 0x7f2550004800 2024-04-20T11:36:55.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.995+0000 7f2572400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f256c0cf8e0 con 0x7f256c152b70 2024-04-20T11:36:55.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.995+0000 7f2563e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f256c1588a0 0x7f256c158d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:55.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.995+0000 7f2563e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f256c1588a0 0x7f256c158d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:57042/0 (socket says 172.21.15.152:57042) 2024-04-20T11:36:55.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.995+0000 7f2563e00640 1 -- 172.21.15.152:0/624774899 learned_addr learned my addr 172.21.15.152:0/624774899 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:55.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.996+0000 7f2571000640 1 --2- 172.21.15.152:0/624774899 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2550004800 0x7f256c13ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:55.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.996+0000 7f2571a00640 1 --2- 172.21.15.152:0/624774899 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f256c152b70 0x7f256c152ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:55.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.996+0000 7f2563e00640 1 -- 172.21.15.152:0/624774899 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f256c152b70 msgr2=0x7f256c152ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:55.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.996+0000 7f2563e00640 1 --2- 172.21.15.152:0/624774899 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f256c152b70 0x7f256c152ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:55.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.996+0000 7f2563e00640 1 -- 172.21.15.152:0/624774899 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2550004800 msgr2=0x7f256c13ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:55.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.996+0000 7f2563e00640 1 --2- 172.21.15.152:0/624774899 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2550004800 0x7f256c13ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:55.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.996+0000 7f2563e00640 1 -- 172.21.15.152:0/624774899 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f256c0b1fa0 con 0x7f256c1588a0 2024-04-20T11:36:55.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.996+0000 7f2571000640 1 --2- 172.21.15.152:0/624774899 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2550004800 0x7f256c13ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:36:55.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.996+0000 7f2563e00640 1 --2- 172.21.15.152:0/624774899 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f256c1588a0 0x7f256c158d20 secure :-1 s=READY pgs=2124 cs=0 l=1 rev1=1 crypto rx=0x7f2554037040 tx=0x7f2554004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:55.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.996+0000 7f2569600640 1 -- 172.21.15.152:0/624774899 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2554042020 con 0x7f256c1588a0 2024-04-20T11:36:55.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.996+0000 7f2569600640 1 -- 172.21.15.152:0/624774899 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f255403a040 con 0x7f256c1588a0 2024-04-20T11:36:55.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.996+0000 7f2569600640 1 -- 172.21.15.152:0/624774899 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2554039070 con 0x7f256c1588a0 2024-04-20T11:36:55.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.996+0000 7f2572400640 1 -- 172.21.15.152:0/624774899 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f256c0b2240 con 0x7f256c1588a0 2024-04-20T11:36:55.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.997+0000 7f2572400640 1 -- 172.21.15.152:0/624774899 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f256c0b2800 con 0x7f256c1588a0 2024-04-20T11:36:55.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.998+0000 7f2572400640 1 -- 172.21.15.152:0/624774899 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f256c075e60 con 0x7f256c1588a0 2024-04-20T11:36:55.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:55.999+0000 7f2569600640 1 -- 172.21.15.152:0/624774899 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) ==== 91754+0+0 (secure 0 0 0) 0x7f2554045020 con 0x7f256c1588a0 2024-04-20T11:36:56.000 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.000+0000 7f2569600640 1 --2- 172.21.15.152:0/624774899 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f254806d100 0x7f254806f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:56.000 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.000+0000 7f2569600640 1 -- 172.21.15.152:0/624774899 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 4857+0+0 (secure 0 0 0) 0x7f25540ae0b0 con 0x7f256c1588a0 2024-04-20T11:36:56.001 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.000+0000 7f2571000640 1 --2- 172.21.15.152:0/624774899 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f254806d100 0x7f254806f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:56.002 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.001+0000 7f2571000640 1 --2- 172.21.15.152:0/624774899 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f254806d100 0x7f254806f5c0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f255c006ce0 tx=0x7f255c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:56.005 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.005+0000 7f2569600640 1 -- 172.21.15.152:0/624774899 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f255404c070 con 0x7f256c1588a0 2024-04-20T11:36:56.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.190+0000 7f2572400640 1 -- 172.21.15.152:0/624774899 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f256c15e9a0 con 0x7f256c1588a0 2024-04-20T11:36:56.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.192+0000 7f2569600640 1 -- 172.21.15.152:0/624774899 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v150) ==== 74+0+12961 (secure 0 0 0) 0x7f255407b810 con 0x7f256c1588a0 2024-04-20T11:36:56.193 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:36:56.193 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":150,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:36:55.266162+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:36:56.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.194+0000 7f2572400640 1 -- 172.21.15.152:0/624774899 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f254806d100 msgr2=0x7f254806f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:56.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.194+0000 7f2572400640 1 --2- 172.21.15.152:0/624774899 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f254806d100 0x7f254806f5c0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f255c006ce0 tx=0x7f255c008040 comp rx=0 tx=0).stop 2024-04-20T11:36:56.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.195+0000 7f2572400640 1 -- 172.21.15.152:0/624774899 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f256c1588a0 msgr2=0x7f256c158d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:56.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.195+0000 7f2572400640 1 --2- 172.21.15.152:0/624774899 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f256c1588a0 0x7f256c158d20 secure :-1 s=READY pgs=2124 cs=0 l=1 rev1=1 crypto rx=0x7f2554037040 tx=0x7f2554004730 comp rx=0 tx=0).stop 2024-04-20T11:36:56.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.195+0000 7f2572400640 1 -- 172.21.15.152:0/624774899 shutdown_connections 2024-04-20T11:36:56.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.195+0000 7f2572400640 1 --2- 172.21.15.152:0/624774899 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f254806d100 0x7f254806f5c0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:56.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.195+0000 7f2572400640 1 --2- 172.21.15.152:0/624774899 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f256c152b70 0x7f256c152ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:56.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.195+0000 7f2572400640 1 --2- 172.21.15.152:0/624774899 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f256c1588a0 0x7f256c158d20 unknown :-1 s=CLOSED pgs=2124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:56.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.195+0000 7f2572400640 1 --2- 172.21.15.152:0/624774899 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2550004800 0x7f256c13ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:56.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.195+0000 7f2572400640 1 -- 172.21.15.152:0/624774899 >> 172.21.15.152:0/624774899 conn(0x7f256c067ae0 msgr2=0x7f256c0aeec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:56.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.195+0000 7f2572400640 1 -- 172.21.15.152:0/624774899 shutdown_connections 2024-04-20T11:36:56.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.195+0000 7f2572400640 1 -- 172.21.15.152:0/624774899 wait complete. 2024-04-20T11:36:56.214 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T11:26:47.246571+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T11:36:56.214 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T11:36:56.405 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.405+0000 7f7b40200640 1 Processor -- start 2024-04-20T11:36:56.405 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.405+0000 7f7b40200640 1 -- start start 2024-04-20T11:36:56.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.407+0000 7f7b40200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f7b38146000 0x7f7b380b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:56.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.407+0000 7f7b40200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f7b380b2170 0x7f7b380b2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:56.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.407+0000 7f7b40200640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b381463e0 con 0x7f7b380cd1b0 2024-04-20T11:36:56.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.407+0000 7f7b40200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f7b380cdcc0 con 0x7f7b38146000 2024-04-20T11:36:56.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.407+0000 7f7b40200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f7b380b4750 con 0x7f7b380b2170 2024-04-20T11:36:56.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.407+0000 7f7b3d800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f7b38146000 0x7f7b380b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:56.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.407+0000 7f7b3e200640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f7b380cd1b0 0x7f7b380cd5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:52154/0 (socket says 172.21.15.152:52154) 2024-04-20T11:36:56.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.407+0000 7f7b3e200640 1 -- 172.21.15.152:0/1832847084 learned_addr learned my addr 172.21.15.152:0/1832847084 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:36:56.408 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.407+0000 7f7b3ce00640 1 --2- 172.21.15.152:0/1832847084 >> v2:172.21.15.175:3300/0 conn(0x7f7b380b2170 0x7f7b380b2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:56.408 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.407+0000 7f7b3d800640 1 -- 172.21.15.152:0/1832847084 >> v2:172.21.15.175:3300/0 conn(0x7f7b380b2170 msgr2=0x7f7b380b2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:56.408 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.407+0000 7f7b3d800640 1 --2- 172.21.15.152:0/1832847084 >> v2:172.21.15.175:3300/0 conn(0x7f7b380b2170 0x7f7b380b2620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:56.409 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.407+0000 7f7b3d800640 1 -- 172.21.15.152:0/1832847084 >> v1:172.21.15.152:6789/0 conn(0x7f7b380cd1b0 legacy=0x7f7b380cd5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:56.409 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.407+0000 7f7b3d800640 1 -- 172.21.15.152:0/1832847084 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b380b3e20 con 0x7f7b38146000 2024-04-20T11:36:56.409 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.408+0000 7f7b3d800640 1 --2- 172.21.15.152:0/1832847084 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7b38146000 0x7f7b380b4be0 secure :-1 s=READY pgs=2126 cs=0 l=1 rev1=1 crypto rx=0x7f7b380b3280 tx=0x7f7b2c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34c5e4bf054295a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:56.410 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.408+0000 7f7b37e00640 1 -- 172.21.15.152:0/1832847084 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b2c0315a0 con 0x7f7b38146000 2024-04-20T11:36:56.410 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.408+0000 7f7b37e00640 1 -- 172.21.15.152:0/1832847084 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b2c037070 con 0x7f7b38146000 2024-04-20T11:36:56.410 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.408+0000 7f7b37e00640 1 -- 172.21.15.152:0/1832847084 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b2c031960 con 0x7f7b38146000 2024-04-20T11:36:56.410 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.409+0000 7f7b40200640 1 -- 172.21.15.152:0/1832847084 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7b38146000 msgr2=0x7f7b380b4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:56.410 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.409+0000 7f7b40200640 1 --2- 172.21.15.152:0/1832847084 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7b38146000 0x7f7b380b4be0 secure :-1 s=READY pgs=2126 cs=0 l=1 rev1=1 crypto rx=0x7f7b380b3280 tx=0x7f7b2c02b840 comp rx=0 tx=0).stop 2024-04-20T11:36:56.410 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.409+0000 7f7b40200640 1 -- 172.21.15.152:0/1832847084 shutdown_connections 2024-04-20T11:36:56.410 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.409+0000 7f7b40200640 1 --2- 172.21.15.152:0/1832847084 >> v2:172.21.15.175:3300/0 conn(0x7f7b380b2170 0x7f7b380b2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:56.410 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.409+0000 7f7b40200640 1 --2- 172.21.15.152:0/1832847084 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7b38146000 0x7f7b380b4be0 unknown :-1 s=CLOSED pgs=2126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:56.411 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.409+0000 7f7b40200640 1 -- 172.21.15.152:0/1832847084 >> 172.21.15.152:0/1832847084 conn(0x7f7b3806b980 msgr2=0x7f7b3806bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:56.411 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.409+0000 7f7b40200640 1 -- 172.21.15.152:0/1832847084 shutdown_connections 2024-04-20T11:36:56.411 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.409+0000 7f7b40200640 1 -- 172.21.15.152:0/1832847084 wait complete. 2024-04-20T11:36:56.412 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.411+0000 7f7b40200640 1 Processor -- start 2024-04-20T11:36:56.412 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.411+0000 7f7b40200640 1 -- start start 2024-04-20T11:36:56.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.412+0000 7f7b40200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b380b2170 0x7f7b381327e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:56.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.412+0000 7f7b40200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7b380cd1b0 0x7f7b38130f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:56.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.412+0000 7f7b40200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7b38146000 0x7f7b380b10e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:56.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.412+0000 7f7b40200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f7b3815ccf0 con 0x7f7b38146000 2024-04-20T11:36:56.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.412+0000 7f7b40200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7b380b43e0 con 0x7f7b380b2170 2024-04-20T11:36:56.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.412+0000 7f7b40200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7b3815ce70 con 0x7f7b380cd1b0 2024-04-20T11:36:56.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.412+0000 7f7b3e200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7b38146000 0x7f7b380b10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:56.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.413+0000 7f7b3e200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7b38146000 0x7f7b380b10e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:57062/0 (socket says 172.21.15.152:57062) 2024-04-20T11:36:56.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.413+0000 7f7b3e200640 1 -- 172.21.15.152:0/372011096 learned_addr learned my addr 172.21.15.152:0/372011096 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:56.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.413+0000 7f7b3d800640 1 --2- 172.21.15.152:0/372011096 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b380b2170 0x7f7b381327e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:56.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.413+0000 7f7b3ce00640 1 --2- 172.21.15.152:0/372011096 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7b380cd1b0 0x7f7b38130f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:56.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.413+0000 7f7b3e200640 1 -- 172.21.15.152:0/372011096 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7b380cd1b0 msgr2=0x7f7b38130f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:56.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.413+0000 7f7b3e200640 1 --2- 172.21.15.152:0/372011096 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7b380cd1b0 0x7f7b38130f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:56.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.413+0000 7f7b3e200640 1 -- 172.21.15.152:0/372011096 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b380b2170 msgr2=0x7f7b381327e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:56.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.413+0000 7f7b3e200640 1 --2- 172.21.15.152:0/372011096 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b380b2170 0x7f7b381327e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:56.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.413+0000 7f7b3e200640 1 -- 172.21.15.152:0/372011096 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b38132d20 con 0x7f7b38146000 2024-04-20T11:36:56.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.413+0000 7f7b3d800640 1 --2- 172.21.15.152:0/372011096 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b380b2170 0x7f7b381327e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:36:56.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.413+0000 7f7b3e200640 1 --2- 172.21.15.152:0/372011096 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7b38146000 0x7f7b380b10e0 secure :-1 s=READY pgs=2127 cs=0 l=1 rev1=1 crypto rx=0x7f7b3000cbf0 tx=0x7f7b3000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:56.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.413+0000 7f7b35600640 1 -- 172.21.15.152:0/372011096 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b30017070 con 0x7f7b38146000 2024-04-20T11:36:56.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.414+0000 7f7b35600640 1 -- 172.21.15.152:0/372011096 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b3000ad40 con 0x7f7b38146000 2024-04-20T11:36:56.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.414+0000 7f7b40200640 1 -- 172.21.15.152:0/372011096 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b38131740 con 0x7f7b38146000 2024-04-20T11:36:56.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.414+0000 7f7b35600640 1 -- 172.21.15.152:0/372011096 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b30012600 con 0x7f7b38146000 2024-04-20T11:36:56.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.414+0000 7f7b40200640 1 -- 172.21.15.152:0/372011096 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b38131d00 con 0x7f7b38146000 2024-04-20T11:36:56.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.415+0000 7f7b40200640 1 -- 172.21.15.152:0/372011096 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b380689f0 con 0x7f7b38146000 2024-04-20T11:36:56.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.416+0000 7f7b35600640 1 -- 172.21.15.152:0/372011096 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) ==== 91754+0+0 (secure 0 0 0) 0x7f7b3001a070 con 0x7f7b38146000 2024-04-20T11:36:56.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.417+0000 7f7b35600640 1 --2- 172.21.15.152:0/372011096 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f7b1806d100 0x7f7b1806f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:56.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.417+0000 7f7b35600640 1 -- 172.21.15.152:0/372011096 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 4857+0+0 (secure 0 0 0) 0x7f7b3008c630 con 0x7f7b38146000 2024-04-20T11:36:56.418 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.418+0000 7f7b3d800640 1 --2- 172.21.15.152:0/372011096 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f7b1806d100 0x7f7b1806f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:56.419 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.419+0000 7f7b3d800640 1 --2- 172.21.15.152:0/372011096 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f7b1806d100 0x7f7b1806f5c0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f7b2c036000 tx=0x7f7b2c008030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:56.422 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.422+0000 7f7b35600640 1 -- 172.21.15.152:0/372011096 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f7b30059810 con 0x7f7b38146000 2024-04-20T11:36:56.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.622+0000 7f7b40200640 1 -- 172.21.15.152:0/372011096 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7b381457a0 con 0x7f7b38146000 2024-04-20T11:36:56.623 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.622+0000 7f7b35600640 1 -- 172.21.15.152:0/372011096 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v150) ==== 93+0+10 (secure 0 0 0) 0x7f7b3005db10 con 0x7f7b38146000 2024-04-20T11:36:56.623 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 1 2024-04-20T11:36:56.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.626+0000 7f7b40200640 1 -- 172.21.15.152:0/372011096 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f7b1806d100 msgr2=0x7f7b1806f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:56.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.626+0000 7f7b40200640 1 --2- 172.21.15.152:0/372011096 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f7b1806d100 0x7f7b1806f5c0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f7b2c036000 tx=0x7f7b2c008030 comp rx=0 tx=0).stop 2024-04-20T11:36:56.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.626+0000 7f7b40200640 1 -- 172.21.15.152:0/372011096 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7b38146000 msgr2=0x7f7b380b10e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:56.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.626+0000 7f7b40200640 1 --2- 172.21.15.152:0/372011096 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7b38146000 0x7f7b380b10e0 secure :-1 s=READY pgs=2127 cs=0 l=1 rev1=1 crypto rx=0x7f7b3000cbf0 tx=0x7f7b3000a430 comp rx=0 tx=0).stop 2024-04-20T11:36:56.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.626+0000 7f7b40200640 1 -- 172.21.15.152:0/372011096 shutdown_connections 2024-04-20T11:36:56.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.626+0000 7f7b40200640 1 --2- 172.21.15.152:0/372011096 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f7b1806d100 0x7f7b1806f5c0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:56.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.626+0000 7f7b40200640 1 --2- 172.21.15.152:0/372011096 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7b38146000 0x7f7b380b10e0 unknown :-1 s=CLOSED pgs=2127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:56.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.626+0000 7f7b40200640 1 --2- 172.21.15.152:0/372011096 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7b380cd1b0 0x7f7b38130f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:56.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.626+0000 7f7b40200640 1 --2- 172.21.15.152:0/372011096 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b380b2170 0x7f7b381327e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:56.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.626+0000 7f7b40200640 1 -- 172.21.15.152:0/372011096 >> 172.21.15.152:0/372011096 conn(0x7f7b3806b980 msgr2=0x7f7b38142860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:56.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.626+0000 7f7b40200640 1 -- 172.21.15.152:0/372011096 shutdown_connections 2024-04-20T11:36:56.627 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.626+0000 7f7b40200640 1 -- 172.21.15.152:0/372011096 wait complete. 2024-04-20T11:36:56.645 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-20T11:36:56.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.839+0000 7f9006800640 1 Processor -- start 2024-04-20T11:36:56.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.839+0000 7f9006800640 1 -- start start 2024-04-20T11:36:56.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.840+0000 7f9006800640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f900015c560 0x7f90000b3df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:56.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.840+0000 7f9006800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f90000b4330 0x7f90000d0100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:56.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.840+0000 7f9006800640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f900015fd20 con 0x7f90000d0640 2024-04-20T11:36:56.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.840+0000 7f9006800640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f900015ef60 con 0x7f900015c560 2024-04-20T11:36:56.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.840+0000 7f9006800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f900015f240 con 0x7f90000b4330 2024-04-20T11:36:56.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.840+0000 7f8fff400640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f900015c560 0x7f90000b3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:56.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.840+0000 7f8ffea00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f90000b4330 0x7f90000d0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:56.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.840+0000 7f8fff400640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f900015c560 0x7f90000b3df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.152:51440/0 (socket says 172.21.15.152:51440) 2024-04-20T11:36:56.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.840+0000 7f8fff400640 1 -- 172.21.15.152:0/2274933366 learned_addr learned my addr 172.21.15.152:0/2274933366 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:56.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.840+0000 7f8fffe00640 1 --1- 172.21.15.152:0/2274933366 >> v1:172.21.15.175:6789/0 conn(0x7f90000d0640 0x7f9000071db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.152:60242/0 (socket says 172.21.15.152:60242) 2024-04-20T11:36:56.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.841+0000 7f8ffea00640 1 -- 172.21.15.152:0/2274933366 >> v2:172.21.15.175:3300/0 conn(0x7f900015c560 msgr2=0x7f90000b3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:56.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.841+0000 7f8ffea00640 1 --2- 172.21.15.152:0/2274933366 >> v2:172.21.15.175:3300/0 conn(0x7f900015c560 0x7f90000b3df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:56.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.841+0000 7f8ffea00640 1 -- 172.21.15.152:0/2274933366 >> v1:172.21.15.175:6789/0 conn(0x7f90000d0640 legacy=0x7f9000071db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:56.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.841+0000 7f8ffea00640 1 -- 172.21.15.152:0/2274933366 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f900015f520 con 0x7f90000b4330 2024-04-20T11:36:56.843 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.843+0000 7f8ffea00640 1 --2- 172.21.15.152:0/2274933366 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f90000b4330 0x7f90000d0100 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8fec0097a0 tx=0x7f8fec02b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25bd3efe3ff0d96e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:56.844 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.843+0000 7f8ffe000640 1 -- 172.21.15.152:0/2274933366 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fec031780 con 0x7f90000b4330 2024-04-20T11:36:56.844 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.843+0000 7f8ffe000640 1 -- 172.21.15.152:0/2274933366 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fec037070 con 0x7f90000b4330 2024-04-20T11:36:56.844 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.843+0000 7f8ffe000640 1 -- 172.21.15.152:0/2274933366 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fec031b40 con 0x7f90000b4330 2024-04-20T11:36:56.844 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.844+0000 7f9006800640 1 -- 172.21.15.152:0/2274933366 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f90000b4330 msgr2=0x7f90000d0100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:56.845 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.844+0000 7f9006800640 1 --2- 172.21.15.152:0/2274933366 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f90000b4330 0x7f90000d0100 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8fec0097a0 tx=0x7f8fec02b820 comp rx=0 tx=0).stop 2024-04-20T11:36:56.845 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.844+0000 7f9006800640 1 -- 172.21.15.152:0/2274933366 shutdown_connections 2024-04-20T11:36:56.845 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.844+0000 7f9006800640 1 --2- 172.21.15.152:0/2274933366 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f90000b4330 0x7f90000d0100 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:56.845 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.844+0000 7f9006800640 1 --2- 172.21.15.152:0/2274933366 >> v2:172.21.15.175:3300/0 conn(0x7f900015c560 0x7f90000b3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:56.845 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.844+0000 7f9006800640 1 -- 172.21.15.152:0/2274933366 >> 172.21.15.152:0/2274933366 conn(0x7f9000067cf0 msgr2=0x7f900006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:56.845 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.844+0000 7f9006800640 1 -- 172.21.15.152:0/2274933366 shutdown_connections 2024-04-20T11:36:56.845 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.844+0000 7f9006800640 1 -- 172.21.15.152:0/2274933366 wait complete. 2024-04-20T11:36:56.846 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.846+0000 7f9006800640 1 Processor -- start 2024-04-20T11:36:56.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.846+0000 7f9006800640 1 -- start start 2024-04-20T11:36:56.848 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.847+0000 7f9006800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f90000b4330 0x7f9000152d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:56.848 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.847+0000 7f9006800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f90000d0640 0x7f900014cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:56.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.847+0000 7f9006800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f900015c560 0x7f9000150c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:56.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.847+0000 7f9006800640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f90000d0ab0 con 0x7f900015c560 2024-04-20T11:36:56.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.847+0000 7f9006800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f90000d0dc0 con 0x7f90000b4330 2024-04-20T11:36:56.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.847+0000 7f9006800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f90000aeaa0 con 0x7f90000d0640 2024-04-20T11:36:56.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.848+0000 7f8fffe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f900015c560 0x7f9000150c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:56.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.848+0000 7f8fffe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f900015c560 0x7f9000150c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:57066/0 (socket says 172.21.15.152:57066) 2024-04-20T11:36:56.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.848+0000 7f8fffe00640 1 -- 172.21.15.152:0/3030807805 learned_addr learned my addr 172.21.15.152:0/3030807805 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:56.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.848+0000 7f8fff400640 1 --2- 172.21.15.152:0/3030807805 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f90000b4330 0x7f9000152d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:56.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.848+0000 7f8ffea00640 1 --2- 172.21.15.152:0/3030807805 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f90000d0640 0x7f900014cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:56.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.848+0000 7f8fffe00640 1 -- 172.21.15.152:0/3030807805 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f90000d0640 msgr2=0x7f900014cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:56.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.848+0000 7f8fffe00640 1 --2- 172.21.15.152:0/3030807805 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f90000d0640 0x7f900014cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:56.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.848+0000 7f8fffe00640 1 -- 172.21.15.152:0/3030807805 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f90000b4330 msgr2=0x7f9000152d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:56.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.848+0000 7f8fffe00640 1 --2- 172.21.15.152:0/3030807805 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f90000b4330 0x7f9000152d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:56.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.848+0000 7f8fffe00640 1 -- 172.21.15.152:0/3030807805 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90000d00c0 con 0x7f900015c560 2024-04-20T11:36:56.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.849+0000 7f8fffe00640 1 --2- 172.21.15.152:0/3030807805 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f900015c560 0x7f9000150c80 secure :-1 s=READY pgs=2128 cs=0 l=1 rev1=1 crypto rx=0x7f8ff800c780 tx=0x7f8ff800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:56.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.849+0000 7f8ff7400640 1 -- 172.21.15.152:0/3030807805 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ff8017070 con 0x7f900015c560 2024-04-20T11:36:56.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.849+0000 7f9006800640 1 -- 172.21.15.152:0/3030807805 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90000cf820 con 0x7f900015c560 2024-04-20T11:36:56.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.849+0000 7f8ff7400640 1 -- 172.21.15.152:0/3030807805 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ff800a640 con 0x7f900015c560 2024-04-20T11:36:56.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.849+0000 7f8ff7400640 1 -- 172.21.15.152:0/3030807805 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ff800aa30 con 0x7f900015c560 2024-04-20T11:36:56.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.849+0000 7f9006800640 1 -- 172.21.15.152:0/3030807805 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f900015c9e0 con 0x7f900015c560 2024-04-20T11:36:56.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.850+0000 7f9006800640 1 -- 172.21.15.152:0/3030807805 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90000689f0 con 0x7f900015c560 2024-04-20T11:36:56.852 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.851+0000 7f8ff7400640 1 -- 172.21.15.152:0/3030807805 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) ==== 91754+0+0 (secure 0 0 0) 0x7f8ff801a070 con 0x7f900015c560 2024-04-20T11:36:56.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.852+0000 7f8ff7400640 1 --2- 172.21.15.152:0/3030807805 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f8fe006d100 0x7f8fe006f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:56.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.852+0000 7f8ff7400640 1 -- 172.21.15.152:0/3030807805 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 4857+0+0 (secure 0 0 0) 0x7f8ff808c150 con 0x7f900015c560 2024-04-20T11:36:56.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.853+0000 7f8fff400640 1 --2- 172.21.15.152:0/3030807805 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f8fe006d100 0x7f8fe006f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:56.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.854+0000 7f8fff400640 1 --2- 172.21.15.152:0/3030807805 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f8fe006d100 0x7f8fe006f5c0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f8ff0009b60 tx=0x7f8ff0008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:56.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:56.858+0000 7f8ff7400640 1 -- 172.21.15.152:0/3030807805 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f8ff8059150 con 0x7f900015c560 2024-04-20T11:36:57.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:57.059+0000 7f9006800640 1 -- 172.21.15.152:0/3030807805 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f900006bd00 con 0x7f900015c560 2024-04-20T11:36:57.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:57.060+0000 7f8ff7400640 1 -- 172.21.15.152:0/3030807805 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v150) ==== 84+0+65 (secure 0 0 0) 0x7f8ff805d450 con 0x7f900015c560 2024-04-20T11:36:57.060 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:36:57.060 INFO:teuthology.orchestra.run.smithi152.stdout:[ 2024-04-20T11:36:57.060 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:36:57.060 INFO:teuthology.orchestra.run.smithi152.stdout: "poolnum": 1, 2024-04-20T11:36:57.060 INFO:teuthology.orchestra.run.smithi152.stdout: "poolname": ".mgr" 2024-04-20T11:36:57.060 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:36:57.060 INFO:teuthology.orchestra.run.smithi152.stdout:] 2024-04-20T11:36:57.062 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:57.063+0000 7f9006800640 1 -- 172.21.15.152:0/3030807805 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f8fe006d100 msgr2=0x7f8fe006f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:57.062 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:57.063+0000 7f9006800640 1 --2- 172.21.15.152:0/3030807805 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f8fe006d100 0x7f8fe006f5c0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f8ff0009b60 tx=0x7f8ff0008040 comp rx=0 tx=0).stop 2024-04-20T11:36:57.063 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:57.063+0000 7f9006800640 1 -- 172.21.15.152:0/3030807805 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f900015c560 msgr2=0x7f9000150c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:57.063 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:57.063+0000 7f9006800640 1 --2- 172.21.15.152:0/3030807805 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f900015c560 0x7f9000150c80 secure :-1 s=READY pgs=2128 cs=0 l=1 rev1=1 crypto rx=0x7f8ff800c780 tx=0x7f8ff800cc40 comp rx=0 tx=0).stop 2024-04-20T11:36:57.063 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:57.063+0000 7f9006800640 1 -- 172.21.15.152:0/3030807805 shutdown_connections 2024-04-20T11:36:57.064 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:57.063+0000 7f9006800640 1 --2- 172.21.15.152:0/3030807805 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f8fe006d100 0x7f8fe006f5c0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:57.064 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:57.063+0000 7f9006800640 1 --2- 172.21.15.152:0/3030807805 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f900015c560 0x7f9000150c80 unknown :-1 s=CLOSED pgs=2128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:57.064 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:57.063+0000 7f9006800640 1 --2- 172.21.15.152:0/3030807805 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f90000d0640 0x7f900014cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:57.064 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:57.063+0000 7f9006800640 1 --2- 172.21.15.152:0/3030807805 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f90000b4330 0x7f9000152d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:57.064 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:57.063+0000 7f9006800640 1 -- 172.21.15.152:0/3030807805 >> 172.21.15.152:0/3030807805 conn(0x7f9000067cf0 msgr2=0x7f900015b720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:57.064 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:57.063+0000 7f9006800640 1 -- 172.21.15.152:0/3030807805 shutdown_connections 2024-04-20T11:36:57.064 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:57.063+0000 7f9006800640 1 -- 172.21.15.152:0/3030807805 wait complete. 2024-04-20T11:36:57.081 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-20T11:36:57.082 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-20T11:36:57.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:57.281+0000 7f2c9c800640 1 Processor -- start 2024-04-20T11:36:57.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:57.281+0000 7f2c9c800640 1 -- start start 2024-04-20T11:36:57.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:57.284+0000 7f2c9c800640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f2c940cd8e0 0x7f2c9406e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:57.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:57.284+0000 7f2c9c800640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c9406eab0 con 0x7f2c940cf220 2024-04-20T11:36:57.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:57.284+0000 7f2c9c800640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c940cf6c0 con 0x7f2c940b4d90 2024-04-20T11:36:57.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:57.284+0000 7f2c9c800640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f2c940d0240 con 0x7f2c940cd8e0 2024-04-20T11:36:57.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:57.285+0000 7f2c99400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f2c940cf220 0x7f2c940b4680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:52164/0 (socket says 172.21.15.152:52164) 2024-04-20T11:36:57.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:57.285+0000 7f2c99400640 1 -- 172.21.15.152:0/3731726161 learned_addr learned my addr 172.21.15.152:0/3731726161 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:36:57.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:57.285+0000 7f2c99e00640 1 --2- 172.21.15.152:0/3731726161 >> v2:172.21.15.175:3300/0 conn(0x7f2c940cd8e0 0x7f2c9406e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:57.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:57.285+0000 7f2c98a00640 1 -- 172.21.15.152:0/3731726161 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2051254082 0 0) 0x7f2c9406eab0 con 0x7f2c940cf220 2024-04-20T11:36:57.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:57.285+0000 7f2c98a00640 1 -- 172.21.15.152:0/3731726161 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2c7c003520 con 0x7f2c940cf220 2024-04-20T11:36:57.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:57.285+0000 7f2c98a00640 1 -- 172.21.15.152:0/3731726161 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 565907354 0 0) 0x7f2c7c003520 con 0x7f2c940cf220 2024-04-20T11:36:57.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:57.286+0000 7f2c98a00640 1 -- 172.21.15.152:0/3731726161 >> v2:172.21.15.175:3300/0 conn(0x7f2c940cd8e0 msgr2=0x7f2c9406e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:57.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:57.286+0000 7f2c98a00640 1 --2- 172.21.15.152:0/3731726161 >> v2:172.21.15.175:3300/0 conn(0x7f2c940cd8e0 0x7f2c9406e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:57.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:57.286+0000 7f2c98a00640 1 -- 172.21.15.152:0/3731726161 >> v1:172.21.15.175:6789/0 conn(0x7f2c940b4d90 legacy=0x7f2c9415f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:57.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:57.286+0000 7f2c98a00640 1 -- 172.21.15.152:0/3731726161 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c940cf9a0 con 0x7f2c940cf220 2024-04-20T11:36:57.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:57.286+0000 7f2c98a00640 1 -- 172.21.15.152:0/3731726161 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f2c84001670 con 0x7f2c940cf220 2024-04-20T11:36:57.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:57.286+0000 7f2c98a00640 1 -- 172.21.15.152:0/3731726161 >> v1:172.21.15.152:6789/0 conn(0x7f2c940cf220 legacy=0x7f2c940b4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:57.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:57.286+0000 7f2c98a00640 1 --2- 172.21.15.152:0/3731726161 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2c7c003a70 0x7f2c7c003e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:57.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:57.286+0000 7f2c98a00640 1 --2- 172.21.15.152:0/3731726161 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2c7c004800 0x7f2c7c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:57.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:57.286+0000 7f2c98a00640 1 --2- 172.21.15.152:0/3731726161 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c7c005310 0x7f2c7c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:57.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:57.286+0000 7f2c98a00640 1 -- 172.21.15.152:0/3731726161 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f2c940d0240 con 0x7f2c7c004800 2024-04-20T11:36:57.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:57.286+0000 7f2c98a00640 1 -- 172.21.15.152:0/3731726161 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2c7c00c250 con 0x7f2c7c005310 2024-04-20T11:36:57.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:57.286+0000 7f2c98a00640 1 -- 172.21.15.152:0/3731726161 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2c7c00c450 con 0x7f2c7c003a70 2024-04-20T11:36:57.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:57.286+0000 7f2c99400640 1 --2- 172.21.15.152:0/3731726161 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2c7c004800 0x7f2c7c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:57.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:57.286+0000 7f2c99400640 1 -- 172.21.15.152:0/3731726161 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2c7c003a70 msgr2=0x7f2c7c003e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:57.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:57.286+0000 7f2c99400640 1 --2- 172.21.15.152:0/3731726161 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2c7c003a70 0x7f2c7c003e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:57.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:57.286+0000 7f2c99400640 1 -- 172.21.15.152:0/3731726161 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c7c005310 msgr2=0x7f2c7c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:57.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:57.286+0000 7f2c99400640 1 --2- 172.21.15.152:0/3731726161 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c7c005310 0x7f2c7c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:57.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:57.286+0000 7f2c99400640 1 -- 172.21.15.152:0/3731726161 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c7c003520 con 0x7f2c7c004800 2024-04-20T11:36:57.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:57.286+0000 7f2c9a800640 1 -- 172.21.15.152:0/3731726161 reap_dead start 2024-04-20T11:36:57.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:57.288+0000 7f2c99400640 1 --2- 172.21.15.152:0/3731726161 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2c7c004800 0x7f2c7c004c80 secure :-1 s=READY pgs=2130 cs=0 l=1 rev1=1 crypto rx=0x7f2c8400c0e0 tx=0x7f2c7c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d67021475d8e04d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:57.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:57.288+0000 7f2c98a00640 1 -- 172.21.15.152:0/3731726161 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c84007c40 con 0x7f2c7c004800 2024-04-20T11:36:57.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:57.288+0000 7f2c98a00640 1 -- 172.21.15.152:0/3731726161 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c84030070 con 0x7f2c7c004800 2024-04-20T11:36:57.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:57.288+0000 7f2c98a00640 1 -- 172.21.15.152:0/3731726161 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c84005d70 con 0x7f2c7c004800 2024-04-20T11:36:57.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:57.288+0000 7f2c9c800640 1 -- 172.21.15.152:0/3731726161 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2c7c004800 msgr2=0x7f2c7c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:57.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:57.288+0000 7f2c9c800640 1 --2- 172.21.15.152:0/3731726161 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2c7c004800 0x7f2c7c004c80 secure :-1 s=READY pgs=2130 cs=0 l=1 rev1=1 crypto rx=0x7f2c8400c0e0 tx=0x7f2c7c005830 comp rx=0 tx=0).stop 2024-04-20T11:36:57.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:57.289+0000 7f2c9c800640 1 -- 172.21.15.152:0/3731726161 shutdown_connections 2024-04-20T11:36:57.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:57.289+0000 7f2c9c800640 1 --2- 172.21.15.152:0/3731726161 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2c7c004800 0x7f2c7c004c80 unknown :-1 s=CLOSED pgs=2130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:57.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:57.289+0000 7f2c9c800640 1 -- 172.21.15.152:0/3731726161 >> 172.21.15.152:0/3731726161 conn(0x7f2c9406b980 msgr2=0x7f2c9406bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:57.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:57.289+0000 7f2c9c800640 1 -- 172.21.15.152:0/3731726161 shutdown_connections 2024-04-20T11:36:57.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:57.289+0000 7f2c9c800640 1 -- 172.21.15.152:0/3731726161 wait complete. 2024-04-20T11:36:57.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:57.291+0000 7f2c9c800640 1 Processor -- start 2024-04-20T11:36:57.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:57.291+0000 7f2c9c800640 1 -- start start 2024-04-20T11:36:57.292 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:57.292+0000 7f2c9c800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c7c004800 0x7f2c940cbe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:57.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:57.292+0000 7f2c9c800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2c9415c540 0x7f2c941f4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:57.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:57.292+0000 7f2c9c800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2c941f4750 0x7f2c940b3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:57.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:57.292+0000 7f2c9c800640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f2c940cdcc0 con 0x7f2c9415c540 2024-04-20T11:36:57.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:57.292+0000 7f2c9c800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2c940cff60 con 0x7f2c7c004800 2024-04-20T11:36:57.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:57.292+0000 7f2c9c800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2c94144d70 con 0x7f2c941f4750 2024-04-20T11:36:57.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:57.292+0000 7f2c99400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2c9415c540 0x7f2c941f4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:57.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:57.292+0000 7f2c99400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2c9415c540 0x7f2c941f4210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:57082/0 (socket says 172.21.15.152:57082) 2024-04-20T11:36:57.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:57.292+0000 7f2c99400640 1 -- 172.21.15.152:0/292818599 learned_addr learned my addr 172.21.15.152:0/292818599 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:57.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:57.292+0000 7f2c99400640 1 -- 172.21.15.152:0/292818599 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2c941f4750 msgr2=0x7f2c940b3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:57.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:57.292+0000 7f2c99400640 1 --2- 172.21.15.152:0/292818599 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2c941f4750 0x7f2c940b3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:57.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:57.292+0000 7f2c99400640 1 -- 172.21.15.152:0/292818599 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c7c004800 msgr2=0x7f2c940cbe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:57.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:57.292+0000 7f2c99400640 1 --2- 172.21.15.152:0/292818599 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c7c004800 0x7f2c940cbe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:57.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:57.292+0000 7f2c99400640 1 -- 172.21.15.152:0/292818599 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c941f4d20 con 0x7f2c9415c540 2024-04-20T11:36:57.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:57.293+0000 7f2c99400640 1 --2- 172.21.15.152:0/292818599 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2c9415c540 0x7f2c941f4210 secure :-1 s=READY pgs=2131 cs=0 l=1 rev1=1 crypto rx=0x7f2c8402c460 tx=0x7f2c8402e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:57.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:57.293+0000 7f2c8e000640 1 -- 172.21.15.152:0/292818599 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c84046070 con 0x7f2c9415c540 2024-04-20T11:36:57.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:57.293+0000 7f2c8e000640 1 -- 172.21.15.152:0/292818599 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c84004bf0 con 0x7f2c9415c540 2024-04-20T11:36:57.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:57.293+0000 7f2c9c800640 1 -- 172.21.15.152:0/292818599 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c941f5000 con 0x7f2c9415c540 2024-04-20T11:36:57.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:57.293+0000 7f2c8e000640 1 -- 172.21.15.152:0/292818599 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c84000f50 con 0x7f2c9415c540 2024-04-20T11:36:57.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:57.293+0000 7f2c9c800640 1 -- 172.21.15.152:0/292818599 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c9415c020 con 0x7f2c9415c540 2024-04-20T11:36:57.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:57.294+0000 7f2c9c800640 1 -- 172.21.15.152:0/292818599 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c94068a60 con 0x7f2c9415c540 2024-04-20T11:36:57.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:57.295+0000 7f2c8e000640 1 -- 172.21.15.152:0/292818599 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) ==== 91754+0+0 (secure 0 0 0) 0x7f2c84007070 con 0x7f2c9415c540 2024-04-20T11:36:57.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:57.296+0000 7f2c8e000640 1 --2- 172.21.15.152:0/292818599 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f2c7806d0d0 0x7f2c7806f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:57.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:57.296+0000 7f2c8e000640 1 -- 172.21.15.152:0/292818599 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 4857+0+0 (secure 0 0 0) 0x7f2c840ad950 con 0x7f2c9415c540 2024-04-20T11:36:57.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:57.297+0000 7f2c99e00640 1 --2- 172.21.15.152:0/292818599 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f2c7806d0d0 0x7f2c7806f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:57.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:57.298+0000 7f2c99e00640 1 --2- 172.21.15.152:0/292818599 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f2c7806d0d0 0x7f2c7806f590 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f2c88006a90 tx=0x7f2c88006a20 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:57.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:57.300+0000 7f2c8e000640 1 -- 172.21.15.152:0/292818599 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f2c8407b550 con 0x7f2c9415c540 2024-04-20T11:36:57.502 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:57.503+0000 7f2c9c800640 1 -- 172.21.15.152:0/292818599 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f2c940afa10 con 0x7f2c9415c540 2024-04-20T11:36:58.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:58.298+0000 7f2c8e000640 1 -- 172.21.15.152:0/292818599 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v151) ==== 138+0+0 (secure 0 0 0) 0x7f2c8407ae50 con 0x7f2c9415c540 2024-04-20T11:36:58.298 INFO:teuthology.orchestra.run.smithi152.stderr:pool 'cephfs_metadata' created 2024-04-20T11:36:58.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:58.302+0000 7f2c9c800640 1 -- 172.21.15.152:0/292818599 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f2c7806d0d0 msgr2=0x7f2c7806f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:58.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:58.302+0000 7f2c9c800640 1 --2- 172.21.15.152:0/292818599 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f2c7806d0d0 0x7f2c7806f590 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f2c88006a90 tx=0x7f2c88006a20 comp rx=0 tx=0).stop 2024-04-20T11:36:58.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:58.302+0000 7f2c9c800640 1 -- 172.21.15.152:0/292818599 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2c9415c540 msgr2=0x7f2c941f4210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:58.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:58.302+0000 7f2c9c800640 1 --2- 172.21.15.152:0/292818599 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2c9415c540 0x7f2c941f4210 secure :-1 s=READY pgs=2131 cs=0 l=1 rev1=1 crypto rx=0x7f2c8402c460 tx=0x7f2c8402e070 comp rx=0 tx=0).stop 2024-04-20T11:36:58.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:58.302+0000 7f2c9c800640 1 -- 172.21.15.152:0/292818599 shutdown_connections 2024-04-20T11:36:58.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:58.302+0000 7f2c9c800640 1 --2- 172.21.15.152:0/292818599 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f2c7806d0d0 0x7f2c7806f590 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:58.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:58.302+0000 7f2c9c800640 1 --2- 172.21.15.152:0/292818599 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2c941f4750 0x7f2c940b3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:58.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:58.302+0000 7f2c9c800640 1 --2- 172.21.15.152:0/292818599 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2c9415c540 0x7f2c941f4210 unknown :-1 s=CLOSED pgs=2131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:58.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:58.302+0000 7f2c9c800640 1 --2- 172.21.15.152:0/292818599 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c7c004800 0x7f2c940cbe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:58.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:58.302+0000 7f2c9c800640 1 -- 172.21.15.152:0/292818599 >> 172.21.15.152:0/292818599 conn(0x7f2c9406b980 msgr2=0x7f2c94143d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:58.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:58.302+0000 7f2c9c800640 1 -- 172.21.15.152:0/292818599 shutdown_connections 2024-04-20T11:36:58.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:58.302+0000 7f2c9c800640 1 -- 172.21.15.152:0/292818599 wait complete. 2024-04-20T11:36:58.330 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-20T11:36:58.525 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:58.525+0000 7f7cd2800640 1 Processor -- start 2024-04-20T11:36:58.525 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:58.526+0000 7f7cd2800640 1 -- start start 2024-04-20T11:36:58.528 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:58.527+0000 7f7cd2800640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f7ccc0d0b80 0x7f7ccc160d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:58.528 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:58.527+0000 7f7cd2800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f7ccc0cd450 0x7f7ccc0723e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:58.528 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:58.527+0000 7f7cd2800640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ccc161290 con 0x7f7ccc0ccff0 2024-04-20T11:36:58.528 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:58.527+0000 7f7cd2800640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f7ccc145830 con 0x7f7ccc0cd450 2024-04-20T11:36:58.528 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:58.527+0000 7f7cd2800640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f7ccc06f070 con 0x7f7ccc0d0b80 2024-04-20T11:36:58.528 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:58.528+0000 7f7ccbe00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f7ccc0cd450 0x7f7ccc0723e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:58.528 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:58.528+0000 7f7ccbe00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f7ccc0cd450 0x7f7ccc0723e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:57094/0 (socket says 172.21.15.152:57094) 2024-04-20T11:36:58.528 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:58.528+0000 7f7ccbe00640 1 -- 172.21.15.152:0/1242449675 learned_addr learned my addr 172.21.15.152:0/1242449675 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:58.528 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:58.528+0000 7f7ccb400640 1 --2- 172.21.15.152:0/1242449675 >> v2:172.21.15.175:3300/0 conn(0x7f7ccc0d0b80 0x7f7ccc160d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:58.528 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:58.528+0000 7f7ccbe00640 1 -- 172.21.15.152:0/1242449675 >> v2:172.21.15.175:3300/0 conn(0x7f7ccc0d0b80 msgr2=0x7f7ccc160d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:58.529 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:58.528+0000 7f7ccbe00640 1 --2- 172.21.15.152:0/1242449675 >> v2:172.21.15.175:3300/0 conn(0x7f7ccc0d0b80 0x7f7ccc160d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:58.529 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:58.528+0000 7f7ccbe00640 1 -- 172.21.15.152:0/1242449675 >> v1:172.21.15.175:6789/0 conn(0x7f7ccc0ccff0 legacy=0x7f7ccc071cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:58.529 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:58.528+0000 7f7ccbe00640 1 -- 172.21.15.152:0/1242449675 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ccc06f500 con 0x7f7ccc0cd450 2024-04-20T11:36:58.529 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:58.528+0000 7f7ccb400640 1 --2- 172.21.15.152:0/1242449675 >> v2:172.21.15.175:3300/0 conn(0x7f7ccc0d0b80 0x7f7ccc160d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:36:58.529 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:58.529+0000 7f7ccbe00640 1 --2- 172.21.15.152:0/1242449675 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7ccc0cd450 0x7f7ccc0723e0 secure :-1 s=READY pgs=2132 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc0b3150 tx=0x7f7cc402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59883ab4176c7677 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:58.529 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:58.530+0000 7f7cca000640 1 -- 172.21.15.152:0/1242449675 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cc402ca80 con 0x7f7ccc0cd450 2024-04-20T11:36:58.530 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:58.530+0000 7f7cca000640 1 -- 172.21.15.152:0/1242449675 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cc402cc20 con 0x7f7ccc0cd450 2024-04-20T11:36:58.530 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:58.530+0000 7f7cca000640 1 -- 172.21.15.152:0/1242449675 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cc40343f0 con 0x7f7ccc0cd450 2024-04-20T11:36:58.530 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:58.530+0000 7f7cd2800640 1 -- 172.21.15.152:0/1242449675 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7ccc0cd450 msgr2=0x7f7ccc0723e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:58.530 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:58.530+0000 7f7cd2800640 1 --2- 172.21.15.152:0/1242449675 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7ccc0cd450 0x7f7ccc0723e0 secure :-1 s=READY pgs=2132 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc0b3150 tx=0x7f7cc402bee0 comp rx=0 tx=0).stop 2024-04-20T11:36:58.530 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:58.531+0000 7f7cd2800640 1 -- 172.21.15.152:0/1242449675 shutdown_connections 2024-04-20T11:36:58.531 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:58.531+0000 7f7cd2800640 1 --2- 172.21.15.152:0/1242449675 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7ccc0cd450 0x7f7ccc0723e0 unknown :-1 s=CLOSED pgs=2132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:58.531 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:58.531+0000 7f7cd2800640 1 --2- 172.21.15.152:0/1242449675 >> v2:172.21.15.175:3300/0 conn(0x7f7ccc0d0b80 0x7f7ccc160d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:58.531 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:58.531+0000 7f7cd2800640 1 -- 172.21.15.152:0/1242449675 >> 172.21.15.152:0/1242449675 conn(0x7f7ccc06b980 msgr2=0x7f7ccc06bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:58.531 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:58.531+0000 7f7cd2800640 1 -- 172.21.15.152:0/1242449675 shutdown_connections 2024-04-20T11:36:58.531 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:58.531+0000 7f7cd2800640 1 -- 172.21.15.152:0/1242449675 wait complete. 2024-04-20T11:36:58.532 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:58.532+0000 7f7cd2800640 1 Processor -- start 2024-04-20T11:36:58.532 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:58.533+0000 7f7cd2800640 1 -- start start 2024-04-20T11:36:58.533 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:58.533+0000 7f7cd2800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7ccc0ccff0 0x7f7ccc15bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:58.533 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:58.533+0000 7f7cd2800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7ccc0cd450 0x7f7ccc133710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:58.533 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:58.533+0000 7f7cd2800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ccc0d0b80 0x7f7ccc160050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:58.533 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:58.533+0000 7f7cd2800640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f7ccc06ea20 con 0x7f7ccc0cd450 2024-04-20T11:36:58.534 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:58.533+0000 7f7cd2800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7ccc06e740 con 0x7f7ccc0d0b80 2024-04-20T11:36:58.534 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:58.533+0000 7f7cd2800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7ccc06ed00 con 0x7f7ccc0ccff0 2024-04-20T11:36:58.534 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:58.534+0000 7f7ccaa00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7ccc0cd450 0x7f7ccc133710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:58.534 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:58.534+0000 7f7ccaa00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7ccc0cd450 0x7f7ccc133710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:57110/0 (socket says 172.21.15.152:57110) 2024-04-20T11:36:58.534 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:58.534+0000 7f7ccaa00640 1 -- 172.21.15.152:0/3126429941 learned_addr learned my addr 172.21.15.152:0/3126429941 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:58.535 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:58.534+0000 7f7ccb400640 1 --2- 172.21.15.152:0/3126429941 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7ccc0ccff0 0x7f7ccc15bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:58.535 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:58.534+0000 7f7ccaa00640 1 -- 172.21.15.152:0/3126429941 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7ccc0ccff0 msgr2=0x7f7ccc15bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:58.535 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:58.534+0000 7f7ccbe00640 1 --2- 172.21.15.152:0/3126429941 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ccc0d0b80 0x7f7ccc160050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:58.535 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:58.534+0000 7f7ccaa00640 1 --2- 172.21.15.152:0/3126429941 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7ccc0ccff0 0x7f7ccc15bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:58.535 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:58.534+0000 7f7ccaa00640 1 -- 172.21.15.152:0/3126429941 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ccc0d0b80 msgr2=0x7f7ccc160050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:58.535 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:58.534+0000 7f7ccaa00640 1 --2- 172.21.15.152:0/3126429941 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ccc0d0b80 0x7f7ccc160050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:58.536 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:58.534+0000 7f7ccaa00640 1 -- 172.21.15.152:0/3126429941 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ccc15db00 con 0x7f7ccc0cd450 2024-04-20T11:36:58.536 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:58.535+0000 7f7ccaa00640 1 --2- 172.21.15.152:0/3126429941 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7ccc0cd450 0x7f7ccc133710 secure :-1 s=READY pgs=2133 cs=0 l=1 rev1=1 crypto rx=0x7f7cb800c430 tx=0x7f7cb800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:58.536 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:58.535+0000 7f7cc3400640 1 -- 172.21.15.152:0/3126429941 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb8011790 con 0x7f7ccc0cd450 2024-04-20T11:36:58.536 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:58.535+0000 7f7cc3400640 1 -- 172.21.15.152:0/3126429941 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cb8017070 con 0x7f7ccc0cd450 2024-04-20T11:36:58.536 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:58.535+0000 7f7cc3400640 1 -- 172.21.15.152:0/3126429941 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb8011b70 con 0x7f7ccc0cd450 2024-04-20T11:36:58.536 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:58.535+0000 7f7cd2800640 1 -- 172.21.15.152:0/3126429941 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ccc15e5b0 con 0x7f7ccc0cd450 2024-04-20T11:36:58.536 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:58.535+0000 7f7cd2800640 1 -- 172.21.15.152:0/3126429941 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ccc15c530 con 0x7f7ccc0cd450 2024-04-20T11:36:58.536 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:58.536+0000 7f7cd2800640 1 -- 172.21.15.152:0/3126429941 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ccc15c760 con 0x7f7ccc0cd450 2024-04-20T11:36:58.537 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:58.538+0000 7f7cc3400640 1 -- 172.21.15.152:0/3126429941 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) ==== 91754+0+0 (secure 0 0 0) 0x7f7cb8011d10 con 0x7f7ccc0cd450 2024-04-20T11:36:58.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:58.538+0000 7f7cc3400640 1 --2- 172.21.15.152:0/3126429941 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f7cac06d0d0 0x7f7cac06f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:58.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:58.539+0000 7f7cc3400640 1 -- 172.21.15.152:0/3126429941 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5227+0+0 (secure 0 0 0) 0x7f7cb808cbc0 con 0x7f7ccc0cd450 2024-04-20T11:36:58.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:58.539+0000 7f7ccb400640 1 --2- 172.21.15.152:0/3126429941 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f7cac06d0d0 0x7f7cac06f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:58.540 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:58.540+0000 7f7ccb400640 1 --2- 172.21.15.152:0/3126429941 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f7cac06d0d0 0x7f7cac06f590 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc00b840 tx=0x7f7cbc008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:58.543 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:58.544+0000 7f7cc3400640 1 -- 172.21.15.152:0/3126429941 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f7ccc15c760 con 0x7f7ccc0cd450 2024-04-20T11:36:58.743 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:58.744+0000 7f7cd2800640 1 -- 172.21.15.152:0/3126429941 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f7ccc0689f0 con 0x7f7ccc0cd450 2024-04-20T11:36:59.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:59.316+0000 7f7cc3400640 1 -- 172.21.15.152:0/3126429941 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v152) ==== 170+0+0 (secure 0 0 0) 0x7f7cb805a130 con 0x7f7ccc0cd450 2024-04-20T11:36:59.317 INFO:teuthology.orchestra.run.smithi152.stderr:pool 'cephfs_data' created 2024-04-20T11:36:59.320 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:59.320+0000 7f7cd2800640 1 -- 172.21.15.152:0/3126429941 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f7cac06d0d0 msgr2=0x7f7cac06f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:59.320 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:59.320+0000 7f7cd2800640 1 --2- 172.21.15.152:0/3126429941 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f7cac06d0d0 0x7f7cac06f590 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc00b840 tx=0x7f7cbc008040 comp rx=0 tx=0).stop 2024-04-20T11:36:59.320 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:59.320+0000 7f7cd2800640 1 -- 172.21.15.152:0/3126429941 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7ccc0cd450 msgr2=0x7f7ccc133710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:59.320 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:59.320+0000 7f7cd2800640 1 --2- 172.21.15.152:0/3126429941 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7ccc0cd450 0x7f7ccc133710 secure :-1 s=READY pgs=2133 cs=0 l=1 rev1=1 crypto rx=0x7f7cb800c430 tx=0x7f7cb800c8f0 comp rx=0 tx=0).stop 2024-04-20T11:36:59.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:59.321+0000 7f7cd2800640 1 -- 172.21.15.152:0/3126429941 shutdown_connections 2024-04-20T11:36:59.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:59.321+0000 7f7cd2800640 1 --2- 172.21.15.152:0/3126429941 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f7cac06d0d0 0x7f7cac06f590 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:59.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:59.321+0000 7f7cd2800640 1 --2- 172.21.15.152:0/3126429941 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ccc0d0b80 0x7f7ccc160050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:59.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:59.321+0000 7f7cd2800640 1 --2- 172.21.15.152:0/3126429941 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7ccc0cd450 0x7f7ccc133710 unknown :-1 s=CLOSED pgs=2133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:59.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:59.321+0000 7f7cd2800640 1 --2- 172.21.15.152:0/3126429941 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7ccc0ccff0 0x7f7ccc15bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:59.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:59.321+0000 7f7cd2800640 1 -- 172.21.15.152:0/3126429941 >> 172.21.15.152:0/3126429941 conn(0x7f7ccc06b980 msgr2=0x7f7ccc0721b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:59.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:59.321+0000 7f7cd2800640 1 -- 172.21.15.152:0/3126429941 shutdown_connections 2024-04-20T11:36:59.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:59.322+0000 7f7cd2800640 1 -- 172.21.15.152:0/3126429941 wait complete. 2024-04-20T11:36:59.358 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-20T11:36:59.535 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:59.536+0000 7eff48e00640 1 Processor -- start 2024-04-20T11:36:59.535 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:59.536+0000 7eff48e00640 1 -- start start 2024-04-20T11:36:59.537 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:59.537+0000 7eff48e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7eff4415b510 0x7eff4415fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:59.537 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:59.537+0000 7eff48e00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff4415f1c0 con 0x7eff440b2880 2024-04-20T11:36:59.537 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:59.537+0000 7eff48e00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff4406f670 con 0x7eff440cfc40 2024-04-20T11:36:59.537 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:59.537+0000 7eff48e00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7eff4406f420 con 0x7eff4415b510 2024-04-20T11:36:59.537 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:59.538+0000 7eff41e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7eff4415b510 0x7eff4415fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:59.537 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:59.538+0000 7eff42800640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7eff440b2880 0x7eff440b4800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:52172/0 (socket says 172.21.15.152:52172) 2024-04-20T11:36:59.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:59.538+0000 7eff42800640 1 -- 172.21.15.152:0/1774075124 learned_addr learned my addr 172.21.15.152:0/1774075124 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:36:59.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:59.538+0000 7eff41e00640 1 -- 172.21.15.152:0/1774075124 >> v1:172.21.15.175:6789/0 conn(0x7eff440cfc40 legacy=0x7eff440b2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:36:59.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:59.538+0000 7eff41e00640 1 -- 172.21.15.152:0/1774075124 >> v1:172.21.15.152:6789/0 conn(0x7eff440b2880 legacy=0x7eff440b4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:59.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:59.538+0000 7eff41e00640 1 -- 172.21.15.152:0/1774075124 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff4415f430 con 0x7eff4415b510 2024-04-20T11:36:59.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:59.538+0000 7eff40a00640 1 -- 172.21.15.152:0/1774075124 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2031976094 0 0) 0x7eff4415f1c0 con 0x7eff440b2880 2024-04-20T11:36:59.539 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:59.539+0000 7eff41e00640 1 --2- 172.21.15.152:0/1774075124 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7eff4415b510 0x7eff4415fc30 secure :-1 s=READY pgs=2135 cs=0 l=1 rev1=1 crypto rx=0x7eff340097a0 tx=0x7eff3402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce49226d0dc51e98 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:59.539 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:59.540+0000 7eff40a00640 1 -- 172.21.15.152:0/1774075124 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff340315a0 con 0x7eff4415b510 2024-04-20T11:36:59.539 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:59.540+0000 7eff40a00640 1 -- 172.21.15.152:0/1774075124 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff34037070 con 0x7eff4415b510 2024-04-20T11:36:59.540 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:59.540+0000 7eff40a00640 1 -- 172.21.15.152:0/1774075124 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff34031960 con 0x7eff4415b510 2024-04-20T11:36:59.540 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:59.540+0000 7eff48e00640 1 -- 172.21.15.152:0/1774075124 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7eff4415b510 msgr2=0x7eff4415fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:59.540 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:59.540+0000 7eff48e00640 1 --2- 172.21.15.152:0/1774075124 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7eff4415b510 0x7eff4415fc30 secure :-1 s=READY pgs=2135 cs=0 l=1 rev1=1 crypto rx=0x7eff340097a0 tx=0x7eff3402b840 comp rx=0 tx=0).stop 2024-04-20T11:36:59.540 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:59.540+0000 7eff48e00640 1 -- 172.21.15.152:0/1774075124 shutdown_connections 2024-04-20T11:36:59.540 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:59.540+0000 7eff48e00640 1 --2- 172.21.15.152:0/1774075124 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7eff4415b510 0x7eff4415fc30 unknown :-1 s=CLOSED pgs=2135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:59.541 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:59.541+0000 7eff48e00640 1 -- 172.21.15.152:0/1774075124 >> 172.21.15.152:0/1774075124 conn(0x7eff4406b980 msgr2=0x7eff4406bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:59.541 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:59.541+0000 7eff48e00640 1 -- 172.21.15.152:0/1774075124 shutdown_connections 2024-04-20T11:36:59.541 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:59.541+0000 7eff48e00640 1 -- 172.21.15.152:0/1774075124 wait complete. 2024-04-20T11:36:59.542 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:59.542+0000 7eff48e00640 1 Processor -- start 2024-04-20T11:36:59.542 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:59.543+0000 7eff48e00640 1 -- start start 2024-04-20T11:36:59.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:59.543+0000 7eff48e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff440b2880 0x7eff4414d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:59.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:59.543+0000 7eff48e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7eff440cfc40 0x7eff44151a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:59.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:59.543+0000 7eff48e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7eff4415b510 0x7eff441532a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:59.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:59.543+0000 7eff48e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7eff4406f950 con 0x7eff440cfc40 2024-04-20T11:36:59.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:59.543+0000 7eff48e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7eff4415b8f0 con 0x7eff440b2880 2024-04-20T11:36:59.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:59.544+0000 7eff48e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7eff440b3ef0 con 0x7eff4415b510 2024-04-20T11:36:59.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:59.544+0000 7eff41400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7eff440cfc40 0x7eff44151a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:59.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:59.544+0000 7eff41400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7eff440cfc40 0x7eff44151a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:57130/0 (socket says 172.21.15.152:57130) 2024-04-20T11:36:59.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:59.544+0000 7eff41400640 1 -- 172.21.15.152:0/2388346077 learned_addr learned my addr 172.21.15.152:0/2388346077 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:36:59.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:59.544+0000 7eff41e00640 1 --2- 172.21.15.152:0/2388346077 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff440b2880 0x7eff4414d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:59.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:59.544+0000 7eff42800640 1 --2- 172.21.15.152:0/2388346077 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7eff4415b510 0x7eff441532a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:59.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:59.544+0000 7eff41400640 1 -- 172.21.15.152:0/2388346077 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7eff4415b510 msgr2=0x7eff441532a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:59.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:59.544+0000 7eff41400640 1 --2- 172.21.15.152:0/2388346077 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7eff4415b510 0x7eff441532a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:59.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:59.544+0000 7eff41400640 1 -- 172.21.15.152:0/2388346077 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff440b2880 msgr2=0x7eff4414d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:59.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:59.544+0000 7eff41400640 1 --2- 172.21.15.152:0/2388346077 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff440b2880 0x7eff4414d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:59.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:59.544+0000 7eff41400640 1 -- 172.21.15.152:0/2388346077 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff441522c0 con 0x7eff440cfc40 2024-04-20T11:36:59.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:59.545+0000 7eff41400640 1 --2- 172.21.15.152:0/2388346077 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7eff440cfc40 0x7eff44151a70 secure :-1 s=READY pgs=2136 cs=0 l=1 rev1=1 crypto rx=0x7eff3000c430 tx=0x7eff3000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:59.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:59.545+0000 7eff3a000640 1 -- 172.21.15.152:0/2388346077 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff30011790 con 0x7eff440cfc40 2024-04-20T11:36:59.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:59.545+0000 7eff3a000640 1 -- 172.21.15.152:0/2388346077 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff30017070 con 0x7eff440cfc40 2024-04-20T11:36:59.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:59.545+0000 7eff3a000640 1 -- 172.21.15.152:0/2388346077 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff30011b70 con 0x7eff440cfc40 2024-04-20T11:36:59.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:59.545+0000 7eff48e00640 1 -- 172.21.15.152:0/2388346077 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff441525a0 con 0x7eff440cfc40 2024-04-20T11:36:59.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:59.545+0000 7eff48e00640 1 -- 172.21.15.152:0/2388346077 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff44152b60 con 0x7eff440cfc40 2024-04-20T11:36:59.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:59.546+0000 7eff48e00640 1 -- 172.21.15.152:0/2388346077 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff440689f0 con 0x7eff440cfc40 2024-04-20T11:36:59.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:59.548+0000 7eff3a000640 1 -- 172.21.15.152:0/2388346077 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) ==== 91754+0+0 (secure 0 0 0) 0x7eff30011d10 con 0x7eff440cfc40 2024-04-20T11:36:59.548 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:59.549+0000 7eff3a000640 1 --2- 172.21.15.152:0/2388346077 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7eff2406d0d0 0x7eff2406f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:59.548 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:59.549+0000 7eff3a000640 1 -- 172.21.15.152:0/2388346077 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5609+0+0 (secure 0 0 0) 0x7eff3008cc10 con 0x7eff440cfc40 2024-04-20T11:36:59.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:59.549+0000 7eff41e00640 1 --2- 172.21.15.152:0/2388346077 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7eff2406d0d0 0x7eff2406f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:59.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:59.551+0000 7eff41e00640 1 --2- 172.21.15.152:0/2388346077 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7eff2406d0d0 0x7eff2406f590 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7eff34036000 tx=0x7eff3403d040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:59.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:59.554+0000 7eff3a000640 1 -- 172.21.15.152:0/2388346077 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7eff3005a110 con 0x7eff440cfc40 2024-04-20T11:36:59.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:36:59.788+0000 7eff48e00640 1 -- 172.21.15.152:0/2388346077 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7eff4406ecf0 con 0x7eff440cfc40 2024-04-20T11:37:00.331 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.330+0000 7eff3a000640 1 -- 172.21.15.152:0/2388346077 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 24 and data pool 25 v101) ==== 174+0+0 (secure 0 0 0) 0x7eff30059a10 con 0x7eff440cfc40 2024-04-20T11:37:00.331 INFO:teuthology.orchestra.run.smithi152.stderr:new fs with metadata pool 24 and data pool 25 2024-04-20T11:37:00.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.344+0000 7eff48e00640 1 -- 172.21.15.152:0/2388346077 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7eff2406d0d0 msgr2=0x7eff2406f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:00.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.344+0000 7eff48e00640 1 --2- 172.21.15.152:0/2388346077 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7eff2406d0d0 0x7eff2406f590 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7eff34036000 tx=0x7eff3403d040 comp rx=0 tx=0).stop 2024-04-20T11:37:00.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.344+0000 7eff48e00640 1 -- 172.21.15.152:0/2388346077 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7eff440cfc40 msgr2=0x7eff44151a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:00.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.344+0000 7eff48e00640 1 --2- 172.21.15.152:0/2388346077 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7eff440cfc40 0x7eff44151a70 secure :-1 s=READY pgs=2136 cs=0 l=1 rev1=1 crypto rx=0x7eff3000c430 tx=0x7eff3000c8f0 comp rx=0 tx=0).stop 2024-04-20T11:37:00.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.344+0000 7eff48e00640 1 -- 172.21.15.152:0/2388346077 shutdown_connections 2024-04-20T11:37:00.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.344+0000 7eff48e00640 1 --2- 172.21.15.152:0/2388346077 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7eff2406d0d0 0x7eff2406f590 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:00.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.344+0000 7eff48e00640 1 --2- 172.21.15.152:0/2388346077 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7eff4415b510 0x7eff441532a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:00.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.344+0000 7eff48e00640 1 --2- 172.21.15.152:0/2388346077 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7eff440cfc40 0x7eff44151a70 secure :-1 s=CLOSED pgs=2136 cs=0 l=1 rev1=1 crypto rx=0x7eff3000c430 tx=0x7eff3000c8f0 comp rx=0 tx=0).stop 2024-04-20T11:37:00.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.344+0000 7eff48e00640 1 --2- 172.21.15.152:0/2388346077 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff440b2880 0x7eff4414d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:00.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.344+0000 7eff48e00640 1 -- 172.21.15.152:0/2388346077 >> 172.21.15.152:0/2388346077 conn(0x7eff4406b980 msgr2=0x7eff44072750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:00.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.346+0000 7eff48e00640 1 -- 172.21.15.152:0/2388346077 shutdown_connections 2024-04-20T11:37:00.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.347+0000 7eff48e00640 1 -- 172.21.15.152:0/2388346077 wait complete. 2024-04-20T11:37:00.379 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:37:00.548 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.548+0000 7fed9de00640 1 Processor -- start 2024-04-20T11:37:00.548 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.548+0000 7fed9de00640 1 -- start start 2024-04-20T11:37:00.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.549+0000 7fed9de00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fed980cd0d0 0x7fed980cd580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:00.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.549+0000 7fed9de00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fed98071fe0 0x7fed98072470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:00.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.549+0000 7fed9de00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed98146340 con 0x7fed98145f60 2024-04-20T11:37:00.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.550+0000 7fed9de00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fed9806e9f0 con 0x7fed980cd0d0 2024-04-20T11:37:00.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.550+0000 7fed9de00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fed98072d50 con 0x7fed98071fe0 2024-04-20T11:37:00.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.550+0000 7fed8fe00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fed980cd0d0 0x7fed980cd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:00.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.550+0000 7fed96a00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fed98145f60 0x7fed9806ee80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:52178/0 (socket says 172.21.15.152:52178) 2024-04-20T11:37:00.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.550+0000 7fed96a00640 1 -- 172.21.15.152:0/1901778057 learned_addr learned my addr 172.21.15.152:0/1901778057 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:37:00.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.550+0000 7fed8fe00640 1 -- 172.21.15.152:0/1901778057 >> v2:172.21.15.175:3300/0 conn(0x7fed98071fe0 msgr2=0x7fed98072470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:00.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.550+0000 7fed8fe00640 1 --2- 172.21.15.152:0/1901778057 >> v2:172.21.15.175:3300/0 conn(0x7fed98071fe0 0x7fed98072470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:00.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.550+0000 7fed8fe00640 1 -- 172.21.15.152:0/1901778057 >> v1:172.21.15.152:6789/0 conn(0x7fed98145f60 legacy=0x7fed9806ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:00.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.550+0000 7fed8fe00640 1 -- 172.21.15.152:0/1901778057 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed9806e0c0 con 0x7fed980cd0d0 2024-04-20T11:37:00.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.551+0000 7fed8fe00640 1 --2- 172.21.15.152:0/1901778057 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fed980cd0d0 0x7fed980cd580 secure :-1 s=READY pgs=2138 cs=0 l=1 rev1=1 crypto rx=0x7fed9806eb70 tx=0x7fed8002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78403a442063a8e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:00.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.552+0000 7fed96000640 1 -- 172.21.15.152:0/1901778057 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed80035020 con 0x7fed980cd0d0 2024-04-20T11:37:00.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.552+0000 7fed96000640 1 -- 172.21.15.152:0/1901778057 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed80031830 con 0x7fed980cd0d0 2024-04-20T11:37:00.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.552+0000 7fed96000640 1 -- 172.21.15.152:0/1901778057 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed80031c00 con 0x7fed980cd0d0 2024-04-20T11:37:00.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.552+0000 7fed9de00640 1 -- 172.21.15.152:0/1901778057 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fed980cd0d0 msgr2=0x7fed980cd580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:00.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.552+0000 7fed9de00640 1 --2- 172.21.15.152:0/1901778057 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fed980cd0d0 0x7fed980cd580 secure :-1 s=READY pgs=2138 cs=0 l=1 rev1=1 crypto rx=0x7fed9806eb70 tx=0x7fed8002b8d0 comp rx=0 tx=0).stop 2024-04-20T11:37:00.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.553+0000 7fed9de00640 1 -- 172.21.15.152:0/1901778057 shutdown_connections 2024-04-20T11:37:00.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.553+0000 7fed9de00640 1 --2- 172.21.15.152:0/1901778057 >> v2:172.21.15.175:3300/0 conn(0x7fed98071fe0 0x7fed98072470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:00.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.553+0000 7fed9de00640 1 --2- 172.21.15.152:0/1901778057 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fed980cd0d0 0x7fed980cd580 unknown :-1 s=CLOSED pgs=2138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:00.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.553+0000 7fed9de00640 1 -- 172.21.15.152:0/1901778057 >> 172.21.15.152:0/1901778057 conn(0x7fed98067cf0 msgr2=0x7fed9806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:00.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.553+0000 7fed9de00640 1 -- 172.21.15.152:0/1901778057 shutdown_connections 2024-04-20T11:37:00.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.553+0000 7fed9de00640 1 -- 172.21.15.152:0/1901778057 wait complete. 2024-04-20T11:37:00.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.554+0000 7fed9de00640 1 Processor -- start 2024-04-20T11:37:00.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.555+0000 7fed9de00640 1 -- start start 2024-04-20T11:37:00.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.555+0000 7fed9de00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fed98071fe0 0x7fed980b43c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:00.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.555+0000 7fed9de00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed980cd0d0 0x7fed9815eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:00.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.555+0000 7fed9de00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fed98145f60 0x7fed9815efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:00.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.555+0000 7fed9de00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fed980729b0 con 0x7fed98145f60 2024-04-20T11:37:00.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.555+0000 7fed9de00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fed9806e680 con 0x7fed980cd0d0 2024-04-20T11:37:00.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.555+0000 7fed9de00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fed98072b30 con 0x7fed98071fe0 2024-04-20T11:37:00.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.556+0000 7fed97400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fed98145f60 0x7fed9815efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:00.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.556+0000 7fed97400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fed98145f60 0x7fed9815efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:57146/0 (socket says 172.21.15.152:57146) 2024-04-20T11:37:00.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.556+0000 7fed97400640 1 -- 172.21.15.152:0/4014940214 learned_addr learned my addr 172.21.15.152:0/4014940214 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:00.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.556+0000 7fed96a00640 1 --2- 172.21.15.152:0/4014940214 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fed98071fe0 0x7fed980b43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:00.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.556+0000 7fed8fe00640 1 --2- 172.21.15.152:0/4014940214 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed980cd0d0 0x7fed9815eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:00.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.556+0000 7fed97400640 1 -- 172.21.15.152:0/4014940214 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fed98071fe0 msgr2=0x7fed980b43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:00.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.556+0000 7fed97400640 1 --2- 172.21.15.152:0/4014940214 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fed98071fe0 0x7fed980b43c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:00.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.556+0000 7fed97400640 1 -- 172.21.15.152:0/4014940214 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed980cd0d0 msgr2=0x7fed9815eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:00.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.556+0000 7fed97400640 1 --2- 172.21.15.152:0/4014940214 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed980cd0d0 0x7fed9815eaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:00.558 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.556+0000 7fed97400640 1 -- 172.21.15.152:0/4014940214 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed981401c0 con 0x7fed98145f60 2024-04-20T11:37:00.558 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.556+0000 7fed96a00640 1 --2- 172.21.15.152:0/4014940214 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fed98071fe0 0x7fed980b43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:37:00.558 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.557+0000 7fed97400640 1 --2- 172.21.15.152:0/4014940214 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fed98145f60 0x7fed9815efe0 secure :-1 s=READY pgs=2139 cs=0 l=1 rev1=1 crypto rx=0x7fed9000b970 tx=0x7fed9000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:00.558 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.557+0000 7fed8f400640 1 -- 172.21.15.152:0/4014940214 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed90019070 con 0x7fed98145f60 2024-04-20T11:37:00.558 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.557+0000 7fed8f400640 1 -- 172.21.15.152:0/4014940214 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed9000cb60 con 0x7fed98145f60 2024-04-20T11:37:00.558 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.557+0000 7fed9de00640 1 -- 172.21.15.152:0/4014940214 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed98145250 con 0x7fed98145f60 2024-04-20T11:37:00.558 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.557+0000 7fed8f400640 1 -- 172.21.15.152:0/4014940214 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed90014440 con 0x7fed98145f60 2024-04-20T11:37:00.558 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.557+0000 7fed9de00640 1 -- 172.21.15.152:0/4014940214 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed981425c0 con 0x7fed98145f60 2024-04-20T11:37:00.558 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.558+0000 7fed9de00640 1 -- 172.21.15.152:0/4014940214 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed980689f0 con 0x7fed98145f60 2024-04-20T11:37:00.559 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.559+0000 7fed8f400640 1 -- 172.21.15.152:0/4014940214 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) ==== 91754+0+0 (secure 0 0 0) 0x7fed9001c070 con 0x7fed98145f60 2024-04-20T11:37:00.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.561+0000 7fed8f400640 1 --2- 172.21.15.152:0/4014940214 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fed7406d0d0 0x7fed7406f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:00.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.561+0000 7fed8f400640 1 -- 172.21.15.152:0/4014940214 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(153..153 src has 1..153) ==== 5709+0+0 (secure 0 0 0) 0x7fed9008cb40 con 0x7fed98145f60 2024-04-20T11:37:00.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.562+0000 7fed96a00640 1 --2- 172.21.15.152:0/4014940214 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fed7406d0d0 0x7fed7406f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:00.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.563+0000 7fed96a00640 1 --2- 172.21.15.152:0/4014940214 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fed7406d0d0 0x7fed7406f590 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fed98140800 tx=0x7fed88009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:00.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.566+0000 7fed8f400640 1 -- 172.21.15.152:0/4014940214 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fed90059f90 con 0x7fed98145f60 2024-04-20T11:37:00.765 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.765+0000 7fed9de00640 1 -- 172.21.15.152:0/4014940214 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fed9815a9b0 con 0x7fed98145f60 2024-04-20T11:37:00.766 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.767+0000 7fed8f400640 1 -- 172.21.15.152:0/4014940214 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v153) ==== 74+0+16752 (secure 0 0 0) 0x7fed90059890 con 0x7fed98145f60 2024-04-20T11:37:00.767 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:37:00.767 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":153,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:36:59.789865+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:36:57.520028+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"153","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2024-04-20T11:36:58.760637+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"153","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:37:00.769 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.769+0000 7fed9de00640 1 -- 172.21.15.152:0/4014940214 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fed7406d0d0 msgr2=0x7fed7406f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:00.769 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.769+0000 7fed9de00640 1 --2- 172.21.15.152:0/4014940214 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fed7406d0d0 0x7fed7406f590 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fed98140800 tx=0x7fed88009040 comp rx=0 tx=0).stop 2024-04-20T11:37:00.770 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.769+0000 7fed9de00640 1 -- 172.21.15.152:0/4014940214 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fed98145f60 msgr2=0x7fed9815efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:00.770 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.769+0000 7fed9de00640 1 --2- 172.21.15.152:0/4014940214 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fed98145f60 0x7fed9815efe0 secure :-1 s=READY pgs=2139 cs=0 l=1 rev1=1 crypto rx=0x7fed9000b970 tx=0x7fed9000be30 comp rx=0 tx=0).stop 2024-04-20T11:37:00.770 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.770+0000 7fed9de00640 1 -- 172.21.15.152:0/4014940214 shutdown_connections 2024-04-20T11:37:00.770 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.770+0000 7fed9de00640 1 --2- 172.21.15.152:0/4014940214 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fed7406d0d0 0x7fed7406f590 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:00.770 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.770+0000 7fed9de00640 1 --2- 172.21.15.152:0/4014940214 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fed98145f60 0x7fed9815efe0 unknown :-1 s=CLOSED pgs=2139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:00.770 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.770+0000 7fed9de00640 1 --2- 172.21.15.152:0/4014940214 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed980cd0d0 0x7fed9815eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:00.770 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.770+0000 7fed9de00640 1 --2- 172.21.15.152:0/4014940214 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fed98071fe0 0x7fed980b43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:00.770 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.770+0000 7fed9de00640 1 -- 172.21.15.152:0/4014940214 >> 172.21.15.152:0/4014940214 conn(0x7fed98067cf0 msgr2=0x7fed980b00e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:00.770 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.770+0000 7fed9de00640 1 -- 172.21.15.152:0/4014940214 shutdown_connections 2024-04-20T11:37:00.770 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.770+0000 7fed9de00640 1 -- 172.21.15.152:0/4014940214 wait complete. 2024-04-20T11:37:00.789 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:37:00.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.991+0000 7f481cc00640 1 Processor -- start 2024-04-20T11:37:00.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.991+0000 7f481cc00640 1 -- start start 2024-04-20T11:37:00.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.993+0000 7f481cc00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f4818160b90 0x7f481815d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:00.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.993+0000 7f481cc00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48180ce8c0 con 0x7f481815d1e0 2024-04-20T11:37:00.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.993+0000 7f481cc00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48181600b0 con 0x7f4818071cd0 2024-04-20T11:37:00.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.993+0000 7f481cc00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f4818160390 con 0x7f4818160b90 2024-04-20T11:37:00.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.993+0000 7f4817400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f481815d1e0 0x7f48180cdfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:52180/0 (socket says 172.21.15.152:52180) 2024-04-20T11:37:00.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.993+0000 7f4817400640 1 -- 172.21.15.152:0/3876148761 learned_addr learned my addr 172.21.15.152:0/3876148761 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:37:00.995 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.994+0000 7f4816a00640 1 --2- 172.21.15.152:0/3876148761 >> v2:172.21.15.175:3300/0 conn(0x7f4818160b90 0x7f481815d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:00.995 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.994+0000 7f4816000640 1 -- 172.21.15.152:0/3876148761 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3199883126 0 0) 0x7f48180ce8c0 con 0x7f481815d1e0 2024-04-20T11:37:00.995 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.994+0000 7f4816000640 1 -- 172.21.15.152:0/3876148761 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f47fc003520 con 0x7f481815d1e0 2024-04-20T11:37:00.995 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.994+0000 7f4816000640 1 -- 172.21.15.152:0/3876148761 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3102803665 0 0) 0x7f47fc003520 con 0x7f481815d1e0 2024-04-20T11:37:00.995 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.994+0000 7f4816000640 1 -- 172.21.15.152:0/3876148761 >> v2:172.21.15.175:3300/0 conn(0x7f4818160b90 msgr2=0x7f481815d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:00.995 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.994+0000 7f4816000640 1 --2- 172.21.15.152:0/3876148761 >> v2:172.21.15.175:3300/0 conn(0x7f4818160b90 0x7f481815d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:00.995 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.994+0000 7f4816000640 1 -- 172.21.15.152:0/3876148761 >> v1:172.21.15.175:6789/0 conn(0x7f4818071cd0 legacy=0x7f4818072120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:00.995 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.994+0000 7f4816000640 1 -- 172.21.15.152:0/3876148761 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4818160670 con 0x7f481815d1e0 2024-04-20T11:37:00.995 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.994+0000 7f4816000640 1 -- 172.21.15.152:0/3876148761 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f4808001670 con 0x7f481815d1e0 2024-04-20T11:37:00.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.994+0000 7f4816000640 1 -- 172.21.15.152:0/3876148761 >> v1:172.21.15.152:6789/0 conn(0x7f481815d1e0 legacy=0x7f48180cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:00.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.994+0000 7f4816000640 1 --2- 172.21.15.152:0/3876148761 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f47fc003a30 0x7f47fc003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:00.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.994+0000 7f4816000640 1 --2- 172.21.15.152:0/3876148761 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f47fc004800 0x7f47fc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:00.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.994+0000 7f4816000640 1 --2- 172.21.15.152:0/3876148761 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f47fc0051c0 0x7f47fc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:00.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.994+0000 7f4816000640 1 -- 172.21.15.152:0/3876148761 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f4818160390 con 0x7f47fc003a30 2024-04-20T11:37:00.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.994+0000 7f4816000640 1 -- 172.21.15.152:0/3876148761 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f47fc00c0f0 con 0x7f47fc004800 2024-04-20T11:37:00.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.994+0000 7f4816000640 1 -- 172.21.15.152:0/3876148761 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f47fc00c2f0 con 0x7f47fc0051c0 2024-04-20T11:37:00.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.994+0000 7f4817400640 1 --2- 172.21.15.152:0/3876148761 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f47fc003a30 0x7f47fc003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:00.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.995+0000 7f4817400640 1 -- 172.21.15.152:0/3876148761 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f47fc0051c0 msgr2=0x7f47fc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:00.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.995+0000 7f4817400640 1 --2- 172.21.15.152:0/3876148761 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f47fc0051c0 0x7f47fc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:00.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.995+0000 7f4817400640 1 -- 172.21.15.152:0/3876148761 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f47fc004800 msgr2=0x7f47fc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:00.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.995+0000 7f4817400640 1 --2- 172.21.15.152:0/3876148761 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f47fc004800 0x7f47fc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:00.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.995+0000 7f4817400640 1 -- 172.21.15.152:0/3876148761 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47fc003520 con 0x7f47fc003a30 2024-04-20T11:37:00.997 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.995+0000 7f4817e00640 1 -- 172.21.15.152:0/3876148761 reap_dead start 2024-04-20T11:37:00.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.996+0000 7f4817400640 1 --2- 172.21.15.152:0/3876148761 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f47fc003a30 0x7f47fc003e50 secure :-1 s=READY pgs=2141 cs=0 l=1 rev1=1 crypto rx=0x7f480800c110 tx=0x7f47fc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30f968234862fa4c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:00.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.996+0000 7f4816000640 1 -- 172.21.15.152:0/3876148761 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4808007c40 con 0x7f47fc003a30 2024-04-20T11:37:00.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.996+0000 7f4816000640 1 -- 172.21.15.152:0/3876148761 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4808030070 con 0x7f47fc003a30 2024-04-20T11:37:00.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.996+0000 7f4816000640 1 -- 172.21.15.152:0/3876148761 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4808005d70 con 0x7f47fc003a30 2024-04-20T11:37:00.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.997+0000 7f481cc00640 1 -- 172.21.15.152:0/3876148761 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f47fc003a30 msgr2=0x7f47fc003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:00.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.997+0000 7f481cc00640 1 --2- 172.21.15.152:0/3876148761 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f47fc003a30 0x7f47fc003e50 secure :-1 s=READY pgs=2141 cs=0 l=1 rev1=1 crypto rx=0x7f480800c110 tx=0x7f47fc005830 comp rx=0 tx=0).stop 2024-04-20T11:37:00.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.997+0000 7f481cc00640 1 -- 172.21.15.152:0/3876148761 shutdown_connections 2024-04-20T11:37:00.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.997+0000 7f481cc00640 1 --2- 172.21.15.152:0/3876148761 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f47fc003a30 0x7f47fc003e50 unknown :-1 s=CLOSED pgs=2141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:00.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.997+0000 7f481cc00640 1 -- 172.21.15.152:0/3876148761 >> 172.21.15.152:0/3876148761 conn(0x7f4818067ae0 msgr2=0x7f481806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:00.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.997+0000 7f481cc00640 1 -- 172.21.15.152:0/3876148761 shutdown_connections 2024-04-20T11:37:00.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.997+0000 7f481cc00640 1 -- 172.21.15.152:0/3876148761 wait complete. 2024-04-20T11:37:00.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.999+0000 7f481cc00640 1 Processor -- start 2024-04-20T11:37:00.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:00.999+0000 7f481cc00640 1 -- start start 2024-04-20T11:37:01.001 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:01.000+0000 7f481cc00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f47fc003a30 0x7f48181f4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:01.001 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:01.000+0000 7f481cc00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4818158bd0 0x7f4818159050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:01.001 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:01.000+0000 7f481cc00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f48181556c0 0x7f4818155b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:01.002 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:01.000+0000 7f481cc00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f481815d740 con 0x7f48181556c0 2024-04-20T11:37:01.002 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:01.000+0000 7f481cc00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f481815d5c0 con 0x7f4818158bd0 2024-04-20T11:37:01.002 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:01.000+0000 7f481cc00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f481815d8c0 con 0x7f47fc003a30 2024-04-20T11:37:01.002 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:01.000+0000 7f4817e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f48181556c0 0x7f4818155b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:01.002 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:01.000+0000 7f4817e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f48181556c0 0x7f4818155b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:57170/0 (socket says 172.21.15.152:57170) 2024-04-20T11:37:01.002 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:01.000+0000 7f4817e00640 1 -- 172.21.15.152:0/3672985300 learned_addr learned my addr 172.21.15.152:0/3672985300 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:01.002 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:01.000+0000 7f4817400640 1 --2- 172.21.15.152:0/3672985300 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f47fc003a30 0x7f48181f4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:01.002 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:01.000+0000 7f4816a00640 1 --2- 172.21.15.152:0/3672985300 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4818158bd0 0x7f4818159050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:01.002 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:01.000+0000 7f4817e00640 1 -- 172.21.15.152:0/3672985300 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f47fc003a30 msgr2=0x7f48181f4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:01.002 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:01.000+0000 7f4817e00640 1 --2- 172.21.15.152:0/3672985300 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f47fc003a30 0x7f48181f4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:01.002 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:01.000+0000 7f4817e00640 1 -- 172.21.15.152:0/3672985300 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4818158bd0 msgr2=0x7f4818159050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:01.002 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:01.000+0000 7f4817e00640 1 --2- 172.21.15.152:0/3672985300 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4818158bd0 0x7f4818159050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:01.003 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:01.001+0000 7f4817e00640 1 -- 172.21.15.152:0/3672985300 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4818157000 con 0x7f48181556c0 2024-04-20T11:37:01.003 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:01.001+0000 7f4817400640 1 --2- 172.21.15.152:0/3672985300 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f47fc003a30 0x7f48181f4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:37:01.003 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:01.001+0000 7f4816a00640 1 --2- 172.21.15.152:0/3672985300 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4818158bd0 0x7f4818159050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:37:01.003 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:01.001+0000 7f4817e00640 1 --2- 172.21.15.152:0/3672985300 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f48181556c0 0x7f4818155b40 secure :-1 s=READY pgs=2142 cs=0 l=1 rev1=1 crypto rx=0x7f481806fe80 tx=0x7f481000cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:01.003 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:01.001+0000 7f480f400640 1 -- 172.21.15.152:0/3672985300 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4810017070 con 0x7f48181556c0 2024-04-20T11:37:01.003 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:01.001+0000 7f480f400640 1 -- 172.21.15.152:0/3672985300 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4810005d60 con 0x7f48181556c0 2024-04-20T11:37:01.003 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:01.001+0000 7f481cc00640 1 -- 172.21.15.152:0/3672985300 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48181563f0 con 0x7f48181556c0 2024-04-20T11:37:01.003 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:01.001+0000 7f480f400640 1 -- 172.21.15.152:0/3672985300 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f481000a850 con 0x7f48181556c0 2024-04-20T11:37:01.003 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:01.001+0000 7f481cc00640 1 -- 172.21.15.152:0/3672985300 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48181569b0 con 0x7f48181556c0 2024-04-20T11:37:01.004 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:01.003+0000 7f481cc00640 1 -- 172.21.15.152:0/3672985300 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48180cf410 con 0x7f48181556c0 2024-04-20T11:37:01.004 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:01.004+0000 7f480f400640 1 -- 172.21.15.152:0/3672985300 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) ==== 91754+0+0 (secure 0 0 0) 0x7f481001a070 con 0x7f48181556c0 2024-04-20T11:37:01.004 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:01.005+0000 7f480f400640 1 --2- 172.21.15.152:0/3672985300 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f47f806d0d0 0x7f47f806f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:01.004 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:01.005+0000 7f480f400640 1 -- 172.21.15.152:0/3672985300 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(153..153 src has 1..153) ==== 5709+0+0 (secure 0 0 0) 0x7f481008bee0 con 0x7f48181556c0 2024-04-20T11:37:01.005 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:01.005+0000 7f4817400640 1 --2- 172.21.15.152:0/3672985300 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f47f806d0d0 0x7f47f806f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:01.006 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:01.006+0000 7f4817400640 1 --2- 172.21.15.152:0/3672985300 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f47f806d0d0 0x7f47f806f590 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f4808034000 tx=0x7f4808039040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:01.008 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:01.009+0000 7f480f400640 1 -- 172.21.15.152:0/3672985300 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f48180cf410 con 0x7f48181556c0 2024-04-20T11:37:01.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:01.196+0000 7f481cc00640 1 -- 172.21.15.152:0/3672985300 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f48180ce8c0 con 0x7f48181556c0 2024-04-20T11:37:01.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:01.198+0000 7f480f400640 1 -- 172.21.15.152:0/3672985300 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v153) ==== 74+0+16752 (secure 0 0 0) 0x7f48180ce8c0 con 0x7f48181556c0 2024-04-20T11:37:01.199 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:37:01.199 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":153,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:36:59.789865+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:36:57.520028+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"153","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2024-04-20T11:36:58.760637+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"153","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:37:01.201 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:01.201+0000 7f481cc00640 1 -- 172.21.15.152:0/3672985300 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f47f806d0d0 msgr2=0x7f47f806f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:01.201 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:01.201+0000 7f481cc00640 1 --2- 172.21.15.152:0/3672985300 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f47f806d0d0 0x7f47f806f590 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f4808034000 tx=0x7f4808039040 comp rx=0 tx=0).stop 2024-04-20T11:37:01.201 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:01.201+0000 7f481cc00640 1 -- 172.21.15.152:0/3672985300 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f48181556c0 msgr2=0x7f4818155b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:01.201 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:01.201+0000 7f481cc00640 1 --2- 172.21.15.152:0/3672985300 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f48181556c0 0x7f4818155b40 secure :-1 s=READY pgs=2142 cs=0 l=1 rev1=1 crypto rx=0x7f481806fe80 tx=0x7f481000cb90 comp rx=0 tx=0).stop 2024-04-20T11:37:01.201 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:01.201+0000 7f481cc00640 1 -- 172.21.15.152:0/3672985300 shutdown_connections 2024-04-20T11:37:01.201 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:01.201+0000 7f481cc00640 1 --2- 172.21.15.152:0/3672985300 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f47f806d0d0 0x7f47f806f590 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:01.201 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:01.201+0000 7f481cc00640 1 --2- 172.21.15.152:0/3672985300 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f48181556c0 0x7f4818155b40 unknown :-1 s=CLOSED pgs=2142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:01.201 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:01.201+0000 7f481cc00640 1 --2- 172.21.15.152:0/3672985300 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4818158bd0 0x7f4818159050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:01.201 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:01.201+0000 7f481cc00640 1 --2- 172.21.15.152:0/3672985300 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f47fc003a30 0x7f48181f4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:01.201 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:01.201+0000 7f481cc00640 1 -- 172.21.15.152:0/3672985300 >> 172.21.15.152:0/3672985300 conn(0x7f4818067ae0 msgr2=0x7f481815bc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:01.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:01.201+0000 7f481cc00640 1 -- 172.21.15.152:0/3672985300 shutdown_connections 2024-04-20T11:37:01.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:01.202+0000 7f481cc00640 1 -- 172.21.15.152:0/3672985300 wait complete. 2024-04-20T11:37:01.220 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-20T11:37:01.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:01.427+0000 7fb1eea00640 1 Processor -- start 2024-04-20T11:37:01.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:01.427+0000 7fb1eea00640 1 -- start start 2024-04-20T11:37:01.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:01.429+0000 7fb1eea00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fb1e8162bc0 0x7fb1e8167240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:01.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:01.429+0000 7fb1eea00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb1e806f720 0x7fb1e80b2170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:01.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:01.429+0000 7fb1eea00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fb1e80b26b0 0x7fb1e8071db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:01.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:01.429+0000 7fb1eea00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fb1e80ccb40 con 0x7fb1e8162bc0 2024-04-20T11:37:01.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:01.429+0000 7fb1eea00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fb1e80b3610 con 0x7fb1e80b26b0 2024-04-20T11:37:01.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:01.429+0000 7fb1eea00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb1e8167780 con 0x7fb1e806f720 2024-04-20T11:37:01.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:01.429+0000 7fb1e7e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fb1e8162bc0 0x7fb1e8167240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:01.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:01.429+0000 7fb1e7e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fb1e8162bc0 0x7fb1e8167240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:57172/0 (socket says 172.21.15.152:57172) 2024-04-20T11:37:01.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:01.429+0000 7fb1e7e00640 1 -- 172.21.15.152:0/2257881151 learned_addr learned my addr 172.21.15.152:0/2257881151 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:01.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:01.430+0000 7fb1eca00640 1 --2- 172.21.15.152:0/2257881151 >> v2:172.21.15.175:3300/0 conn(0x7fb1e80b26b0 0x7fb1e8071db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:01.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:01.430+0000 7fb1df400640 1 --2- 172.21.15.152:0/2257881151 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb1e806f720 0x7fb1e80b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:01.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:01.430+0000 7fb1e7e00640 1 -- 172.21.15.152:0/2257881151 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb1e806f720 msgr2=0x7fb1e80b2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:01.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:01.430+0000 7fb1e7e00640 1 --2- 172.21.15.152:0/2257881151 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb1e806f720 0x7fb1e80b2170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:01.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:01.430+0000 7fb1e7e00640 1 -- 172.21.15.152:0/2257881151 >> v2:172.21.15.175:3300/0 conn(0x7fb1e80b26b0 msgr2=0x7fb1e8071db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:01.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:01.430+0000 7fb1e7e00640 1 --2- 172.21.15.152:0/2257881151 >> v2:172.21.15.175:3300/0 conn(0x7fb1e80b26b0 0x7fb1e8071db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:01.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:01.430+0000 7fb1e7e00640 1 -- 172.21.15.152:0/2257881151 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1e806fc30 con 0x7fb1e8162bc0 2024-04-20T11:37:01.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:01.430+0000 7fb1df400640 1 --2- 172.21.15.152:0/2257881151 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb1e806f720 0x7fb1e80b2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:37:01.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:01.430+0000 7fb1eca00640 1 --2- 172.21.15.152:0/2257881151 >> v2:172.21.15.175:3300/0 conn(0x7fb1e80b26b0 0x7fb1e8071db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:37:01.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:01.431+0000 7fb1e7e00640 1 --2- 172.21.15.152:0/2257881151 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb1e8162bc0 0x7fb1e8167240 secure :-1 s=READY pgs=2143 cs=0 l=1 rev1=1 crypto rx=0x7fb1e8070160 tx=0x7fb1d802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c38e57c65e2c2be server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:01.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:01.431+0000 7fb1e7400640 1 -- 172.21.15.152:0/2257881151 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1d8035020 con 0x7fb1e8162bc0 2024-04-20T11:37:01.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:01.431+0000 7fb1e7400640 1 -- 172.21.15.152:0/2257881151 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1d8031870 con 0x7fb1e8162bc0 2024-04-20T11:37:01.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:01.432+0000 7fb1e7400640 1 -- 172.21.15.152:0/2257881151 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1d8031c40 con 0x7fb1e8162bc0 2024-04-20T11:37:01.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:01.432+0000 7fb1eea00640 1 -- 172.21.15.152:0/2257881151 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb1e8162bc0 msgr2=0x7fb1e8167240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:01.433 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:01.432+0000 7fb1eea00640 1 --2- 172.21.15.152:0/2257881151 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb1e8162bc0 0x7fb1e8167240 secure :-1 s=READY pgs=2143 cs=0 l=1 rev1=1 crypto rx=0x7fb1e8070160 tx=0x7fb1d802b910 comp rx=0 tx=0).stop 2024-04-20T11:37:01.433 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:01.432+0000 7fb1eea00640 1 -- 172.21.15.152:0/2257881151 shutdown_connections 2024-04-20T11:37:01.433 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:01.432+0000 7fb1eea00640 1 --2- 172.21.15.152:0/2257881151 >> v2:172.21.15.175:3300/0 conn(0x7fb1e80b26b0 0x7fb1e8071db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:01.433 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:01.432+0000 7fb1eea00640 1 --2- 172.21.15.152:0/2257881151 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb1e806f720 0x7fb1e80b2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:01.433 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:01.432+0000 7fb1eea00640 1 --2- 172.21.15.152:0/2257881151 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb1e8162bc0 0x7fb1e8167240 unknown :-1 s=CLOSED pgs=2143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:01.433 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:01.432+0000 7fb1eea00640 1 -- 172.21.15.152:0/2257881151 >> 172.21.15.152:0/2257881151 conn(0x7fb1e806b980 msgr2=0x7fb1e806bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:01.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:01.432+0000 7fb1eea00640 1 -- 172.21.15.152:0/2257881151 shutdown_connections 2024-04-20T11:37:01.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:01.432+0000 7fb1eea00640 1 -- 172.21.15.152:0/2257881151 wait complete. 2024-04-20T11:37:01.437 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:01.434+0000 7fb1eea00640 1 Processor -- start 2024-04-20T11:37:01.437 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:01.434+0000 7fb1eea00640 1 -- start start 2024-04-20T11:37:01.437 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:01.435+0000 7fb1eea00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1e806f720 0x7fb1e81f9250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:01.437 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:01.435+0000 7fb1eea00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb1e80b26b0 0x7fb1e81f5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:01.437 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:01.435+0000 7fb1eea00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb1e8162bc0 0x7fb1e8159b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:01.437 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:01.435+0000 7fb1eea00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fb1e80b4650 con 0x7fb1e8162bc0 2024-04-20T11:37:01.437 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:01.435+0000 7fb1eea00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb1e80b4370 con 0x7fb1e806f720 2024-04-20T11:37:01.437 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:01.435+0000 7fb1eea00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb1e8162fa0 con 0x7fb1e80b26b0 2024-04-20T11:37:01.438 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:01.436+0000 7fb1eca00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb1e8162bc0 0x7fb1e8159b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:01.438 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:01.436+0000 7fb1eca00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb1e8162bc0 0x7fb1e8159b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:57180/0 (socket says 172.21.15.152:57180) 2024-04-20T11:37:01.438 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:01.436+0000 7fb1eca00640 1 -- 172.21.15.152:0/2389350366 learned_addr learned my addr 172.21.15.152:0/2389350366 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:01.438 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:01.436+0000 7fb1e7e00640 1 --2- 172.21.15.152:0/2389350366 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1e806f720 0x7fb1e81f9250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:01.438 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:01.436+0000 7fb1df400640 1 --2- 172.21.15.152:0/2389350366 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb1e80b26b0 0x7fb1e81f5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:01.438 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:01.436+0000 7fb1eca00640 1 -- 172.21.15.152:0/2389350366 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb1e80b26b0 msgr2=0x7fb1e81f5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:01.438 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:01.436+0000 7fb1eca00640 1 --2- 172.21.15.152:0/2389350366 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb1e80b26b0 0x7fb1e81f5ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:01.438 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:01.436+0000 7fb1eca00640 1 -- 172.21.15.152:0/2389350366 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1e806f720 msgr2=0x7fb1e81f9250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:01.438 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:01.436+0000 7fb1eca00640 1 --2- 172.21.15.152:0/2389350366 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1e806f720 0x7fb1e81f9250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:01.438 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:01.436+0000 7fb1eca00640 1 -- 172.21.15.152:0/2389350366 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1e81f9790 con 0x7fb1e8162bc0 2024-04-20T11:37:01.438 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:01.436+0000 7fb1eca00640 1 --2- 172.21.15.152:0/2389350366 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb1e8162bc0 0x7fb1e8159b00 secure :-1 s=READY pgs=2144 cs=0 l=1 rev1=1 crypto rx=0x7fb1e000cef0 tx=0x7fb1e000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:01.438 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:01.437+0000 7fb1e4c00640 1 -- 172.21.15.152:0/2389350366 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e001a070 con 0x7fb1e8162bc0 2024-04-20T11:37:01.439 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:01.437+0000 7fb1e4c00640 1 -- 172.21.15.152:0/2389350366 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1e000d860 con 0x7fb1e8162bc0 2024-04-20T11:37:01.439 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:01.437+0000 7fb1e4c00640 1 -- 172.21.15.152:0/2389350366 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e000dc50 con 0x7fb1e8162bc0 2024-04-20T11:37:01.439 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:01.437+0000 7fb1eea00640 1 -- 172.21.15.152:0/2389350366 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1e8157ef0 con 0x7fb1e8162bc0 2024-04-20T11:37:01.439 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:01.437+0000 7fb1eea00640 1 -- 172.21.15.152:0/2389350366 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1e81584b0 con 0x7fb1e8162bc0 2024-04-20T11:37:01.440 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:01.438+0000 7fb1eea00640 1 -- 172.21.15.152:0/2389350366 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1e80b32d0 con 0x7fb1e8162bc0 2024-04-20T11:37:01.440 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:01.439+0000 7fb1e4c00640 1 -- 172.21.15.152:0/2389350366 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) ==== 91754+0+0 (secure 0 0 0) 0x7fb1e001d070 con 0x7fb1e8162bc0 2024-04-20T11:37:01.440 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:01.440+0000 7fb1e4c00640 1 --2- 172.21.15.152:0/2389350366 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb1c406d0d0 0x7fb1c406f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:01.440 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:01.440+0000 7fb1e4c00640 1 -- 172.21.15.152:0/2389350366 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(154..154 src has 1..154) ==== 5709+0+0 (secure 0 0 0) 0x7fb1e008d670 con 0x7fb1e8162bc0 2024-04-20T11:37:01.440 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:01.441+0000 7fb1e7e00640 1 --2- 172.21.15.152:0/2389350366 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb1c406d0d0 0x7fb1c406f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:01.441 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:01.442+0000 7fb1e7e00640 1 --2- 172.21.15.152:0/2389350366 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb1c406d0d0 0x7fb1c406f590 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fb1d8036000 tx=0x7fb1d803c040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:01.444 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:01.445+0000 7fb1e4c00640 1 -- 172.21.15.152:0/2389350366 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fb1e80b32d0 con 0x7fb1e8162bc0 2024-04-20T11:37:01.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:01.683+0000 7fb1eea00640 1 -- 172.21.15.152:0/2389350366 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fb1e80689f0 con 0x7fb1e8162bc0 2024-04-20T11:37:02.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:02.339+0000 7fb1e4c00640 1 -- 172.21.15.152:0/2389350366 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v104) ==== 118+0+0 (secure 0 0 0) 0x7fb1e81f9790 con 0x7fb1e8162bc0 2024-04-20T11:37:02.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:02.343+0000 7fb1eea00640 1 -- 172.21.15.152:0/2389350366 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb1c406d0d0 msgr2=0x7fb1c406f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:02.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:02.343+0000 7fb1eea00640 1 --2- 172.21.15.152:0/2389350366 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb1c406d0d0 0x7fb1c406f590 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fb1d8036000 tx=0x7fb1d803c040 comp rx=0 tx=0).stop 2024-04-20T11:37:02.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:02.343+0000 7fb1eea00640 1 -- 172.21.15.152:0/2389350366 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb1e8162bc0 msgr2=0x7fb1e8159b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:02.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:02.343+0000 7fb1eea00640 1 --2- 172.21.15.152:0/2389350366 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb1e8162bc0 0x7fb1e8159b00 secure :-1 s=READY pgs=2144 cs=0 l=1 rev1=1 crypto rx=0x7fb1e000cef0 tx=0x7fb1e000a710 comp rx=0 tx=0).stop 2024-04-20T11:37:02.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:02.343+0000 7fb1eea00640 1 -- 172.21.15.152:0/2389350366 shutdown_connections 2024-04-20T11:37:02.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:02.343+0000 7fb1eea00640 1 --2- 172.21.15.152:0/2389350366 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb1c406d0d0 0x7fb1c406f590 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:02.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:02.343+0000 7fb1eea00640 1 --2- 172.21.15.152:0/2389350366 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb1e8162bc0 0x7fb1e8159b00 unknown :-1 s=CLOSED pgs=2144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:02.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:02.343+0000 7fb1eea00640 1 --2- 172.21.15.152:0/2389350366 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb1e80b26b0 0x7fb1e81f5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:02.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:02.343+0000 7fb1eea00640 1 --2- 172.21.15.152:0/2389350366 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1e806f720 0x7fb1e81f9250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:02.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:02.343+0000 7fb1eea00640 1 -- 172.21.15.152:0/2389350366 >> 172.21.15.152:0/2389350366 conn(0x7fb1e806b980 msgr2=0x7fb1e80b05a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:02.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:02.344+0000 7fb1eea00640 1 -- 172.21.15.152:0/2389350366 shutdown_connections 2024-04-20T11:37:02.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:02.344+0000 7fb1eea00640 1 -- 172.21.15.152:0/2389350366 wait complete. 2024-04-20T11:37:02.367 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-20T11:37:02.367 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-20T11:37:02.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:02.573+0000 7fda35200640 1 Processor -- start 2024-04-20T11:37:02.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:02.573+0000 7fda35200640 1 -- start start 2024-04-20T11:37:02.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:02.575+0000 7fda35200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fda3015e5d0 0x7fda30162440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:02.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:02.575+0000 7fda35200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fda300b3fa0 0x7fda300af490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:02.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:02.575+0000 7fda35200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda300adc30 con 0x7fda3015e9b0 2024-04-20T11:37:02.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:02.575+0000 7fda35200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fda30162a70 con 0x7fda300b3fa0 2024-04-20T11:37:02.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:02.575+0000 7fda35200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fda30162d50 con 0x7fda3015e5d0 2024-04-20T11:37:02.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:02.575+0000 7fda2d400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fda300b3fa0 0x7fda300af490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:02.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:02.575+0000 7fda2d400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fda300b3fa0 0x7fda300af490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:57190/0 (socket says 172.21.15.152:57190) 2024-04-20T11:37:02.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:02.575+0000 7fda2d400640 1 -- 172.21.15.152:0/3825596155 learned_addr learned my addr 172.21.15.152:0/3825596155 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:02.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:02.575+0000 7fda2d400640 1 -- 172.21.15.152:0/3825596155 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fda3015e5d0 msgr2=0x7fda30162440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:02.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:02.575+0000 7fda2de00640 1 --2- 172.21.15.152:0/3825596155 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fda3015e5d0 0x7fda30162440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:02.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:02.576+0000 7fda2d400640 1 --2- 172.21.15.152:0/3825596155 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fda3015e5d0 0x7fda30162440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:02.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:02.576+0000 7fda2d400640 1 -- 172.21.15.152:0/3825596155 >> v1:172.21.15.175:6789/0 conn(0x7fda3015e9b0 legacy=0x7fda30071db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:02.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:02.576+0000 7fda2d400640 1 -- 172.21.15.152:0/3825596155 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda30163030 con 0x7fda300b3fa0 2024-04-20T11:37:02.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:02.577+0000 7fda2d400640 1 --2- 172.21.15.152:0/3825596155 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fda300b3fa0 0x7fda300af490 secure :-1 s=READY pgs=2145 cs=0 l=1 rev1=1 crypto rx=0x7fda1c009870 tx=0x7fda1c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ab9f6a342f8075d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:02.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:02.577+0000 7fda2ca00640 1 -- 172.21.15.152:0/3825596155 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda1c035020 con 0x7fda300b3fa0 2024-04-20T11:37:02.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:02.577+0000 7fda2ca00640 1 -- 172.21.15.152:0/3825596155 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda1c031640 con 0x7fda300b3fa0 2024-04-20T11:37:02.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:02.577+0000 7fda2ca00640 1 -- 172.21.15.152:0/3825596155 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda1c02d040 con 0x7fda300b3fa0 2024-04-20T11:37:02.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:02.578+0000 7fda35200640 1 -- 172.21.15.152:0/3825596155 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fda300b3fa0 msgr2=0x7fda300af490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:02.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:02.578+0000 7fda35200640 1 --2- 172.21.15.152:0/3825596155 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fda300b3fa0 0x7fda300af490 secure :-1 s=READY pgs=2145 cs=0 l=1 rev1=1 crypto rx=0x7fda1c009870 tx=0x7fda1c02b8f0 comp rx=0 tx=0).stop 2024-04-20T11:37:02.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:02.578+0000 7fda35200640 1 -- 172.21.15.152:0/3825596155 shutdown_connections 2024-04-20T11:37:02.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:02.578+0000 7fda35200640 1 --2- 172.21.15.152:0/3825596155 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fda300b3fa0 0x7fda300af490 unknown :-1 s=CLOSED pgs=2145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:02.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:02.578+0000 7fda35200640 1 --2- 172.21.15.152:0/3825596155 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fda3015e5d0 0x7fda30162440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:02.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:02.578+0000 7fda35200640 1 -- 172.21.15.152:0/3825596155 >> 172.21.15.152:0/3825596155 conn(0x7fda3006b980 msgr2=0x7fda3006bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:02.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:02.578+0000 7fda35200640 1 -- 172.21.15.152:0/3825596155 shutdown_connections 2024-04-20T11:37:02.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:02.578+0000 7fda35200640 1 -- 172.21.15.152:0/3825596155 wait complete. 2024-04-20T11:37:02.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:02.579+0000 7fda35200640 1 Processor -- start 2024-04-20T11:37:02.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:02.580+0000 7fda35200640 1 -- start start 2024-04-20T11:37:02.580 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:02.581+0000 7fda35200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda300b3fa0 0x7fda30158050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:02.580 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:02.581+0000 7fda35200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fda3015e5d0 0x7fda300ccec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:02.580 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:02.581+0000 7fda35200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fda3015e9b0 0x7fda301610c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:02.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:02.581+0000 7fda35200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fda3006e0a0 con 0x7fda3015e5d0 2024-04-20T11:37:02.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:02.581+0000 7fda35200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fda30163310 con 0x7fda300b3fa0 2024-04-20T11:37:02.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:02.581+0000 7fda35200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fda3006e220 con 0x7fda3015e9b0 2024-04-20T11:37:02.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:02.581+0000 7fda2d400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fda3015e5d0 0x7fda300ccec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:02.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:02.581+0000 7fda2d400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fda3015e5d0 0x7fda300ccec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:57192/0 (socket says 172.21.15.152:57192) 2024-04-20T11:37:02.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:02.581+0000 7fda2d400640 1 -- 172.21.15.152:0/2190517734 learned_addr learned my addr 172.21.15.152:0/2190517734 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:02.582 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:02.581+0000 7fda2d400640 1 -- 172.21.15.152:0/2190517734 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fda3015e9b0 msgr2=0x7fda301610c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:02.582 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:02.581+0000 7fda2d400640 1 --2- 172.21.15.152:0/2190517734 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fda3015e9b0 0x7fda301610c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:02.582 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:02.581+0000 7fda2d400640 1 -- 172.21.15.152:0/2190517734 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda300b3fa0 msgr2=0x7fda30158050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:02.582 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:02.581+0000 7fda2d400640 1 --2- 172.21.15.152:0/2190517734 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda300b3fa0 0x7fda30158050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:02.582 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:02.581+0000 7fda2d400640 1 -- 172.21.15.152:0/2190517734 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda3015fb30 con 0x7fda3015e5d0 2024-04-20T11:37:02.582 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:02.582+0000 7fda2d400640 1 --2- 172.21.15.152:0/2190517734 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fda3015e5d0 0x7fda300ccec0 secure :-1 s=READY pgs=2146 cs=0 l=1 rev1=1 crypto rx=0x7fda1c02b930 tx=0x7fda1c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:02.583 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:02.582+0000 7fda26000640 1 -- 172.21.15.152:0/2190517734 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda1c040070 con 0x7fda3015e5d0 2024-04-20T11:37:02.583 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:02.582+0000 7fda26000640 1 -- 172.21.15.152:0/2190517734 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda1c039c10 con 0x7fda3015e5d0 2024-04-20T11:37:02.583 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:02.582+0000 7fda26000640 1 -- 172.21.15.152:0/2190517734 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda1c035040 con 0x7fda3015e5d0 2024-04-20T11:37:02.583 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:02.582+0000 7fda35200640 1 -- 172.21.15.152:0/2190517734 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda3015fe10 con 0x7fda3015e5d0 2024-04-20T11:37:02.583 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:02.582+0000 7fda35200640 1 -- 172.21.15.152:0/2190517734 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda301603d0 con 0x7fda3015e5d0 2024-04-20T11:37:02.583 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:02.583+0000 7fda35200640 1 -- 172.21.15.152:0/2190517734 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda300689f0 con 0x7fda3015e5d0 2024-04-20T11:37:02.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:02.584+0000 7fda26000640 1 -- 172.21.15.152:0/2190517734 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) ==== 91754+0+0 (secure 0 0 0) 0x7fda1c04b060 con 0x7fda3015e5d0 2024-04-20T11:37:02.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:02.585+0000 7fda26000640 1 --2- 172.21.15.152:0/2190517734 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fda1006d0d0 0x7fda1006f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:02.585 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:02.585+0000 7fda26000640 1 -- 172.21.15.152:0/2190517734 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(154..154 src has 1..154) ==== 5709+0+0 (secure 0 0 0) 0x7fda1c0aea00 con 0x7fda3015e5d0 2024-04-20T11:37:02.585 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:02.586+0000 7fda2de00640 1 --2- 172.21.15.152:0/2190517734 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fda1006d0d0 0x7fda1006f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:02.586 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:02.587+0000 7fda2de00640 1 --2- 172.21.15.152:0/2190517734 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fda1006d0d0 0x7fda1006f590 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fda20006fd0 tx=0x7fda20008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:02.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:02.590+0000 7fda26000640 1 -- 172.21.15.152:0/2190517734 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fda1c02f300 con 0x7fda3015e5d0 2024-04-20T11:37:02.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:02.828+0000 7fda35200640 1 -- 172.21.15.152:0/2190517734 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fda3006fb00 con 0x7fda3015e5d0 2024-04-20T11:37:03.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:03.353+0000 7fda26000640 1 -- 172.21.15.152:0/2190517734 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v105) ==== 122+0+0 (secure 0 0 0) 0x7fda1c07be10 con 0x7fda3015e5d0 2024-04-20T11:37:03.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:03.356+0000 7fda35200640 1 -- 172.21.15.152:0/2190517734 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fda1006d0d0 msgr2=0x7fda1006f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:03.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:03.356+0000 7fda35200640 1 --2- 172.21.15.152:0/2190517734 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fda1006d0d0 0x7fda1006f590 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fda20006fd0 tx=0x7fda20008040 comp rx=0 tx=0).stop 2024-04-20T11:37:03.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:03.356+0000 7fda35200640 1 -- 172.21.15.152:0/2190517734 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fda3015e5d0 msgr2=0x7fda300ccec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:03.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:03.356+0000 7fda35200640 1 --2- 172.21.15.152:0/2190517734 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fda3015e5d0 0x7fda300ccec0 secure :-1 s=READY pgs=2146 cs=0 l=1 rev1=1 crypto rx=0x7fda1c02b930 tx=0x7fda1c0046c0 comp rx=0 tx=0).stop 2024-04-20T11:37:03.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:03.357+0000 7fda35200640 1 -- 172.21.15.152:0/2190517734 shutdown_connections 2024-04-20T11:37:03.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:03.357+0000 7fda35200640 1 --2- 172.21.15.152:0/2190517734 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fda1006d0d0 0x7fda1006f590 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:03.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:03.357+0000 7fda35200640 1 --2- 172.21.15.152:0/2190517734 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fda3015e9b0 0x7fda301610c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:03.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:03.357+0000 7fda35200640 1 --2- 172.21.15.152:0/2190517734 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fda3015e5d0 0x7fda300ccec0 unknown :-1 s=CLOSED pgs=2146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:03.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:03.357+0000 7fda35200640 1 --2- 172.21.15.152:0/2190517734 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda300b3fa0 0x7fda30158050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:03.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:03.357+0000 7fda35200640 1 -- 172.21.15.152:0/2190517734 >> 172.21.15.152:0/2190517734 conn(0x7fda3006b980 msgr2=0x7fda30159b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:03.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:03.357+0000 7fda35200640 1 -- 172.21.15.152:0/2190517734 shutdown_connections 2024-04-20T11:37:03.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:03.357+0000 7fda35200640 1 -- 172.21.15.152:0/2190517734 wait complete. 2024-04-20T11:37:03.378 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-20T11:37:03.378 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:37:03.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:03.581+0000 7fbac5400640 1 Processor -- start 2024-04-20T11:37:03.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:03.581+0000 7fbac5400640 1 -- start start 2024-04-20T11:37:03.582 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:03.582+0000 7fbac5400640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fbac015e8e0 0x7fbac00d1740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:03.582 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:03.582+0000 7fbac5400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbac00cf830 0x7fbac00d1c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:03.582 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:03.583+0000 7fbac5400640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbac00cfd30 con 0x7fbac00b3410 2024-04-20T11:37:03.582 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:03.583+0000 7fbac5400640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fbac00b8430 con 0x7fbac015e8e0 2024-04-20T11:37:03.583 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:03.583+0000 7fbac5400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fbac00b8680 con 0x7fbac00cf830 2024-04-20T11:37:03.583 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:03.583+0000 7fbabe800640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fbac015e8e0 0x7fbac00d1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:03.583 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:03.583+0000 7fbabde00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbac00cf830 0x7fbac00d1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:03.583 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:03.583+0000 7fbabf200640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7fbac00b3410 0x7fbac0162450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.152:60314/0 (socket says 172.21.15.152:60314) 2024-04-20T11:37:03.583 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:03.583+0000 7fbabf200640 1 -- 172.21.15.152:0/181023991 learned_addr learned my addr 172.21.15.152:0/181023991 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:37:03.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:03.584+0000 7fbabde00640 1 -- 172.21.15.152:0/181023991 >> v2:172.21.15.175:3300/0 conn(0x7fbac015e8e0 msgr2=0x7fbac00d1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:03.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:03.584+0000 7fbabde00640 1 --2- 172.21.15.152:0/181023991 >> v2:172.21.15.175:3300/0 conn(0x7fbac015e8e0 0x7fbac00d1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:03.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:03.584+0000 7fbabde00640 1 -- 172.21.15.152:0/181023991 >> v1:172.21.15.175:6789/0 conn(0x7fbac00b3410 legacy=0x7fbac0162450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:03.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:03.584+0000 7fbabde00640 1 -- 172.21.15.152:0/181023991 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbac00b8960 con 0x7fbac00cf830 2024-04-20T11:37:03.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:03.584+0000 7fbabe800640 1 --2- 172.21.15.152:0/181023991 >> v2:172.21.15.175:3300/0 conn(0x7fbac015e8e0 0x7fbac00d1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:37:03.585 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:03.586+0000 7fbabde00640 1 --2- 172.21.15.152:0/181023991 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbac00cf830 0x7fbac00d1c80 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fbab0009d90 tx=0x7fbab002be20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f75f693d549617b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:03.586 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:03.586+0000 7fbabd400640 1 -- 172.21.15.152:0/181023991 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbab0039070 con 0x7fbac00cf830 2024-04-20T11:37:03.586 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:03.586+0000 7fbabd400640 1 -- 172.21.15.152:0/181023991 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbab002cbd0 con 0x7fbac00cf830 2024-04-20T11:37:03.586 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:03.586+0000 7fbabd400640 1 -- 172.21.15.152:0/181023991 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbab0034440 con 0x7fbac00cf830 2024-04-20T11:37:03.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:03.587+0000 7fbac5400640 1 -- 172.21.15.152:0/181023991 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbac00cf830 msgr2=0x7fbac00d1c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:03.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:03.587+0000 7fbac5400640 1 --2- 172.21.15.152:0/181023991 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbac00cf830 0x7fbac00d1c80 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fbab0009d90 tx=0x7fbab002be20 comp rx=0 tx=0).stop 2024-04-20T11:37:03.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:03.587+0000 7fbac5400640 1 -- 172.21.15.152:0/181023991 shutdown_connections 2024-04-20T11:37:03.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:03.587+0000 7fbac5400640 1 --2- 172.21.15.152:0/181023991 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbac00cf830 0x7fbac00d1c80 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:03.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:03.587+0000 7fbac5400640 1 --2- 172.21.15.152:0/181023991 >> v2:172.21.15.175:3300/0 conn(0x7fbac015e8e0 0x7fbac00d1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:03.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:03.587+0000 7fbac5400640 1 -- 172.21.15.152:0/181023991 >> 172.21.15.152:0/181023991 conn(0x7fbac0067ae0 msgr2=0x7fbac0067ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:03.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:03.587+0000 7fbac5400640 1 -- 172.21.15.152:0/181023991 shutdown_connections 2024-04-20T11:37:03.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:03.587+0000 7fbac5400640 1 -- 172.21.15.152:0/181023991 wait complete. 2024-04-20T11:37:03.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:03.589+0000 7fbac5400640 1 Processor -- start 2024-04-20T11:37:03.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:03.589+0000 7fbac5400640 1 -- start start 2024-04-20T11:37:03.590 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:03.590+0000 7fbac5400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbac00b3410 0x7fbac0155570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:03.590 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:03.590+0000 7fbac5400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbac00cf830 0x7fbac01f58c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:03.590 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:03.590+0000 7fbac5400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbac015e8e0 0x7fbac0161250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:03.590 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:03.590+0000 7fbac5400640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fbac00b3af0 con 0x7fbac00cf830 2024-04-20T11:37:03.590 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:03.590+0000 7fbac5400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbac015ecc0 con 0x7fbac00b3410 2024-04-20T11:37:03.590 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:03.590+0000 7fbac5400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fbac00b3c70 con 0x7fbac015e8e0 2024-04-20T11:37:03.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:03.590+0000 7fbabde00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbac00cf830 0x7fbac01f58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:03.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:03.590+0000 7fbabde00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbac00cf830 0x7fbac01f58c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:57206/0 (socket says 172.21.15.152:57206) 2024-04-20T11:37:03.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:03.590+0000 7fbabde00640 1 -- 172.21.15.152:0/2768064490 learned_addr learned my addr 172.21.15.152:0/2768064490 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:03.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:03.590+0000 7fbabe800640 1 --2- 172.21.15.152:0/2768064490 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbac00b3410 0x7fbac0155570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:03.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:03.590+0000 7fbabde00640 1 -- 172.21.15.152:0/2768064490 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbac015e8e0 msgr2=0x7fbac0161250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:03.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:03.590+0000 7fbabf200640 1 --2- 172.21.15.152:0/2768064490 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbac015e8e0 0x7fbac0161250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:03.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:03.590+0000 7fbabde00640 1 --2- 172.21.15.152:0/2768064490 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbac015e8e0 0x7fbac0161250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:03.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:03.591+0000 7fbabde00640 1 -- 172.21.15.152:0/2768064490 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbac00b3410 msgr2=0x7fbac0155570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:03.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:03.591+0000 7fbabde00640 1 --2- 172.21.15.152:0/2768064490 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbac00b3410 0x7fbac0155570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:03.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:03.591+0000 7fbabde00640 1 -- 172.21.15.152:0/2768064490 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbac015efe0 con 0x7fbac00cf830 2024-04-20T11:37:03.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:03.591+0000 7fbabe800640 1 --2- 172.21.15.152:0/2768064490 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbac00b3410 0x7fbac0155570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:37:03.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:03.591+0000 7fbabde00640 1 --2- 172.21.15.152:0/2768064490 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbac00cf830 0x7fbac01f58c0 secure :-1 s=READY pgs=2147 cs=0 l=1 rev1=1 crypto rx=0x7fbab0009d90 tx=0x7fbab0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:03.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:03.591+0000 7fbab6a00640 1 -- 172.21.15.152:0/2768064490 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbab0039050 con 0x7fbac00cf830 2024-04-20T11:37:03.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:03.591+0000 7fbab6a00640 1 -- 172.21.15.152:0/2768064490 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbab0045070 con 0x7fbac00cf830 2024-04-20T11:37:03.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:03.591+0000 7fbac5400640 1 -- 172.21.15.152:0/2768064490 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbac015f2c0 con 0x7fbac00cf830 2024-04-20T11:37:03.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:03.591+0000 7fbab6a00640 1 -- 172.21.15.152:0/2768064490 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbab003e850 con 0x7fbac00cf830 2024-04-20T11:37:03.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:03.591+0000 7fbac5400640 1 -- 172.21.15.152:0/2768064490 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbac01f5e00 con 0x7fbac00cf830 2024-04-20T11:37:03.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:03.593+0000 7fbac5400640 1 -- 172.21.15.152:0/2768064490 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbac00ba4e0 con 0x7fbac00cf830 2024-04-20T11:37:03.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:03.594+0000 7fbab6a00640 1 -- 172.21.15.152:0/2768064490 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) ==== 91754+0+0 (secure 0 0 0) 0x7fbab003e9f0 con 0x7fbac00cf830 2024-04-20T11:37:03.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:03.594+0000 7fbab6a00640 1 --2- 172.21.15.152:0/2768064490 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fba9c06d0d0 0x7fba9c06f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:03.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:03.595+0000 7fbab6a00640 1 -- 172.21.15.152:0/2768064490 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(154..154 src has 1..154) ==== 5709+0+0 (secure 0 0 0) 0x7fbab00afb40 con 0x7fbac00cf830 2024-04-20T11:37:03.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:03.595+0000 7fbabe800640 1 --2- 172.21.15.152:0/2768064490 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fba9c06d0d0 0x7fba9c06f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:03.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:03.596+0000 7fbabe800640 1 --2- 172.21.15.152:0/2768064490 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fba9c06d0d0 0x7fba9c06f590 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fbac015c690 tx=0x7fbaa8008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:03.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:03.599+0000 7fbab6a00640 1 -- 172.21.15.152:0/2768064490 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fbac00ba4e0 con 0x7fbac00cf830 2024-04-20T11:37:03.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:03.830+0000 7fbac5400640 1 -- 172.21.15.152:0/2768064490 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbac00689f0 con 0x7fbac00cf830 2024-04-20T11:37:03.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:03.831+0000 7fbab6a00640 1 -- 172.21.15.152:0/2768064490 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) ==== 95+0+5220 (secure 0 0 0) 0x7fbab0081e20 con 0x7fbac00cf830 2024-04-20T11:37:03.831 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:37:03.831 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":105,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6458,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/1688411915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":1688411915},{"type":"v1","addr":"172.21.15.152:6835","nonce":1688411915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":100},{"gid":7141,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6834/599467453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":599467453},{"type":"v1","addr":"172.21.15.175:6834","nonce":599467453}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":100},{"gid":7144,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/691089259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6833","nonce":691089259},{"type":"v1","addr":"172.21.15.175:6835","nonce":691089259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":100}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T11:36:59.789831+0000","modified":"2024-04-20T11:37:02.829697+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10011},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10011":{"gid":10011,"name":"c","rank":0,"incarnation":101,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6833/1854325981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":1854325981},{"type":"v1","addr":"172.21.15.152:6833","nonce":1854325981}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10011,"qdb_cluster":[10011]},"id":12}]} 2024-04-20T11:37:03.831 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 105 2024-04-20T11:37:03.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:03.834+0000 7fbac5400640 1 -- 172.21.15.152:0/2768064490 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fba9c06d0d0 msgr2=0x7fba9c06f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:03.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:03.834+0000 7fbac5400640 1 --2- 172.21.15.152:0/2768064490 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fba9c06d0d0 0x7fba9c06f590 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fbac015c690 tx=0x7fbaa8008040 comp rx=0 tx=0).stop 2024-04-20T11:37:03.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:03.834+0000 7fbac5400640 1 -- 172.21.15.152:0/2768064490 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbac00cf830 msgr2=0x7fbac01f58c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:03.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:03.834+0000 7fbac5400640 1 --2- 172.21.15.152:0/2768064490 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbac00cf830 0x7fbac01f58c0 secure :-1 s=READY pgs=2147 cs=0 l=1 rev1=1 crypto rx=0x7fbab0009d90 tx=0x7fbab0004a40 comp rx=0 tx=0).stop 2024-04-20T11:37:03.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:03.834+0000 7fbac5400640 1 -- 172.21.15.152:0/2768064490 shutdown_connections 2024-04-20T11:37:03.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:03.834+0000 7fbac5400640 1 --2- 172.21.15.152:0/2768064490 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fba9c06d0d0 0x7fba9c06f590 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:03.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:03.834+0000 7fbac5400640 1 --2- 172.21.15.152:0/2768064490 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbac015e8e0 0x7fbac0161250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:03.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:03.834+0000 7fbac5400640 1 --2- 172.21.15.152:0/2768064490 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbac00cf830 0x7fbac01f58c0 unknown :-1 s=CLOSED pgs=2147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:03.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:03.834+0000 7fbac5400640 1 --2- 172.21.15.152:0/2768064490 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbac00b3410 0x7fbac0155570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:03.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:03.834+0000 7fbac5400640 1 -- 172.21.15.152:0/2768064490 >> 172.21.15.152:0/2768064490 conn(0x7fbac0067ae0 msgr2=0x7fbac00d2660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:03.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:03.834+0000 7fbac5400640 1 -- 172.21.15.152:0/2768064490 shutdown_connections 2024-04-20T11:37:03.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:03.834+0000 7fbac5400640 1 -- 172.21.15.152:0/2768064490 wait complete. 2024-04-20T11:37:03.853 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:37:04.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.050+0000 7fb593200640 1 Processor -- start 2024-04-20T11:37:04.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.050+0000 7fb593200640 1 -- start start 2024-04-20T11:37:04.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.051+0000 7fb593200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fb594147410 0x7fb5940afa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:04.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.052+0000 7fb593200640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5940aec90 con 0x7fb594071cd0 2024-04-20T11:37:04.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.052+0000 7fb593200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5940aef70 con 0x7fb59406e720 2024-04-20T11:37:04.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.052+0000 7fb593200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fb5940af250 con 0x7fb594147410 2024-04-20T11:37:04.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.052+0000 7fb591400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fb594071cd0 0x7fb59406e010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:52188/0 (socket says 172.21.15.152:52188) 2024-04-20T11:37:04.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.052+0000 7fb591400640 1 -- 172.21.15.152:0/3447198436 learned_addr learned my addr 172.21.15.152:0/3447198436 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:37:04.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.052+0000 7fb591e00640 1 --2- 172.21.15.152:0/3447198436 >> v2:172.21.15.152:3300/0 conn(0x7fb594147410 0x7fb5940afa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:04.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.052+0000 7fb590a00640 1 -- 172.21.15.152:0/3447198436 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2597098234 0 0) 0x7fb5940aec90 con 0x7fb594071cd0 2024-04-20T11:37:04.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.052+0000 7fb590a00640 1 -- 172.21.15.152:0/3447198436 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb578003520 con 0x7fb594071cd0 2024-04-20T11:37:04.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.053+0000 7fb591e00640 1 -- 172.21.15.152:0/3447198436 >> v1:172.21.15.175:6789/0 conn(0x7fb59406e720 legacy=0x7fb5940b3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:04.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.053+0000 7fb591e00640 1 -- 172.21.15.152:0/3447198436 >> v1:172.21.15.152:6789/0 conn(0x7fb594071cd0 legacy=0x7fb59406e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:04.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.053+0000 7fb591e00640 1 -- 172.21.15.152:0/3447198436 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5940af530 con 0x7fb594147410 2024-04-20T11:37:04.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.054+0000 7fb591e00640 1 --2- 172.21.15.152:0/3447198436 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb594147410 0x7fb5940afa50 secure :-1 s=READY pgs=2149 cs=0 l=1 rev1=1 crypto rx=0x7fb5840097a0 tx=0x7fb58402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55dd929d341458b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:04.054 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.054+0000 7fb590a00640 1 -- 172.21.15.152:0/3447198436 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb584035020 con 0x7fb594147410 2024-04-20T11:37:04.054 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.055+0000 7fb590a00640 1 -- 172.21.15.152:0/3447198436 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb584031830 con 0x7fb594147410 2024-04-20T11:37:04.054 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.055+0000 7fb590a00640 1 -- 172.21.15.152:0/3447198436 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb584031be0 con 0x7fb594147410 2024-04-20T11:37:04.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.055+0000 7fb593200640 1 -- 172.21.15.152:0/3447198436 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb594147410 msgr2=0x7fb5940afa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:04.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.055+0000 7fb593200640 1 --2- 172.21.15.152:0/3447198436 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb594147410 0x7fb5940afa50 secure :-1 s=READY pgs=2149 cs=0 l=1 rev1=1 crypto rx=0x7fb5840097a0 tx=0x7fb58402b8d0 comp rx=0 tx=0).stop 2024-04-20T11:37:04.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.055+0000 7fb593200640 1 -- 172.21.15.152:0/3447198436 shutdown_connections 2024-04-20T11:37:04.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.055+0000 7fb593200640 1 --2- 172.21.15.152:0/3447198436 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb594147410 0x7fb5940afa50 unknown :-1 s=CLOSED pgs=2149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:04.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.055+0000 7fb593200640 1 -- 172.21.15.152:0/3447198436 >> 172.21.15.152:0/3447198436 conn(0x7fb594067ae0 msgr2=0x7fb59406b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:04.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.055+0000 7fb593200640 1 -- 172.21.15.152:0/3447198436 shutdown_connections 2024-04-20T11:37:04.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.055+0000 7fb593200640 1 -- 172.21.15.152:0/3447198436 wait complete. 2024-04-20T11:37:04.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.057+0000 7fb593200640 1 Processor -- start 2024-04-20T11:37:04.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.057+0000 7fb593200640 1 -- start start 2024-04-20T11:37:04.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.058+0000 7fb593200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb59406e720 0x7fb59415dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:04.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.058+0000 7fb593200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb594071cd0 0x7fb594142f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:04.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.058+0000 7fb593200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb5940b2830 0x7fb5940b2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:04.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.058+0000 7fb593200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fb5940722b0 con 0x7fb5940b2830 2024-04-20T11:37:04.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.058+0000 7fb593200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb594072130 con 0x7fb59406e720 2024-04-20T11:37:04.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.058+0000 7fb593200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb594072430 con 0x7fb594071cd0 2024-04-20T11:37:04.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.058+0000 7fb592800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb5940b2830 0x7fb5940b2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:04.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.058+0000 7fb592800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb5940b2830 0x7fb5940b2ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:57212/0 (socket says 172.21.15.152:57212) 2024-04-20T11:37:04.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.058+0000 7fb592800640 1 -- 172.21.15.152:0/1782196184 learned_addr learned my addr 172.21.15.152:0/1782196184 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:04.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.059+0000 7fb591e00640 1 --2- 172.21.15.152:0/1782196184 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb59406e720 0x7fb59415dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:04.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.059+0000 7fb591400640 1 --2- 172.21.15.152:0/1782196184 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb594071cd0 0x7fb594142f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:04.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.059+0000 7fb592800640 1 -- 172.21.15.152:0/1782196184 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb594071cd0 msgr2=0x7fb594142f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:04.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.059+0000 7fb592800640 1 --2- 172.21.15.152:0/1782196184 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb594071cd0 0x7fb594142f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:04.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.059+0000 7fb592800640 1 -- 172.21.15.152:0/1782196184 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb59406e720 msgr2=0x7fb59415dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:04.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.059+0000 7fb592800640 1 --2- 172.21.15.152:0/1782196184 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb59406e720 0x7fb59415dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:04.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.059+0000 7fb592800640 1 -- 172.21.15.152:0/1782196184 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb59415e3e0 con 0x7fb5940b2830 2024-04-20T11:37:04.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.059+0000 7fb591400640 1 --2- 172.21.15.152:0/1782196184 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb594071cd0 0x7fb594142f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:37:04.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.059+0000 7fb592800640 1 --2- 172.21.15.152:0/1782196184 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb5940b2830 0x7fb5940b2ff0 secure :-1 s=READY pgs=2150 cs=0 l=1 rev1=1 crypto rx=0x7fb58c00c7b0 tx=0x7fb58c00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:04.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.060+0000 7fb58a000640 1 -- 172.21.15.152:0/1782196184 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb58c004b60 con 0x7fb5940b2830 2024-04-20T11:37:04.061 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.060+0000 7fb58a000640 1 -- 172.21.15.152:0/1782196184 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb58c004d00 con 0x7fb5940b2830 2024-04-20T11:37:04.061 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.060+0000 7fb58a000640 1 -- 172.21.15.152:0/1782196184 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb58c00aa90 con 0x7fb5940b2830 2024-04-20T11:37:04.061 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.060+0000 7fb593200640 1 -- 172.21.15.152:0/1782196184 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb59413d380 con 0x7fb5940b2830 2024-04-20T11:37:04.061 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.060+0000 7fb593200640 1 -- 172.21.15.152:0/1782196184 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb59413d940 con 0x7fb5940b2830 2024-04-20T11:37:04.061 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.061+0000 7fb593200640 1 -- 172.21.15.152:0/1782196184 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb594075e60 con 0x7fb5940b2830 2024-04-20T11:37:04.061 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.062+0000 7fb58a000640 1 -- 172.21.15.152:0/1782196184 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) ==== 91754+0+0 (secure 0 0 0) 0x7fb58c00ac30 con 0x7fb5940b2830 2024-04-20T11:37:04.063 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.063+0000 7fb58a000640 1 --2- 172.21.15.152:0/1782196184 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb57006d0d0 0x7fb57006f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:04.063 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.063+0000 7fb58a000640 1 -- 172.21.15.152:0/1782196184 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(154..154 src has 1..154) ==== 5709+0+0 (secure 0 0 0) 0x7fb58c08a730 con 0x7fb5940b2830 2024-04-20T11:37:04.063 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.064+0000 7fb591e00640 1 --2- 172.21.15.152:0/1782196184 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb57006d0d0 0x7fb57006f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:04.064 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.065+0000 7fb591e00640 1 --2- 172.21.15.152:0/1782196184 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb57006d0d0 0x7fb57006f590 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fb584036000 tx=0x7fb584008030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:04.067 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.068+0000 7fb58a000640 1 -- 172.21.15.152:0/1782196184 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fb58c020070 con 0x7fb5940b2830 2024-04-20T11:37:04.253 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.253+0000 7fb593200640 1 -- 172.21.15.152:0/1782196184 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb59413fb90 con 0x7fb5940b2830 2024-04-20T11:37:04.256 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.257+0000 7fb58a000640 1 -- 172.21.15.152:0/1782196184 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v154) ==== 74+0+16739 (secure 0 0 0) 0x7fb59415e3e0 con 0x7fb5940b2830 2024-04-20T11:37:04.257 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:37:04.257 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":154,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:37:01.327485+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:36:57.520028+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"153","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2024-04-20T11:36:58.760637+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"154","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:37:04.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.259+0000 7fb593200640 1 -- 172.21.15.152:0/1782196184 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb57006d0d0 msgr2=0x7fb57006f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:04.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.259+0000 7fb593200640 1 --2- 172.21.15.152:0/1782196184 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb57006d0d0 0x7fb57006f590 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fb584036000 tx=0x7fb584008030 comp rx=0 tx=0).stop 2024-04-20T11:37:04.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.259+0000 7fb593200640 1 -- 172.21.15.152:0/1782196184 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb5940b2830 msgr2=0x7fb5940b2ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:04.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.259+0000 7fb593200640 1 --2- 172.21.15.152:0/1782196184 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb5940b2830 0x7fb5940b2ff0 secure :-1 s=READY pgs=2150 cs=0 l=1 rev1=1 crypto rx=0x7fb58c00c7b0 tx=0x7fb58c00cc70 comp rx=0 tx=0).stop 2024-04-20T11:37:04.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.259+0000 7fb593200640 1 -- 172.21.15.152:0/1782196184 shutdown_connections 2024-04-20T11:37:04.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.259+0000 7fb593200640 1 --2- 172.21.15.152:0/1782196184 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb57006d0d0 0x7fb57006f590 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:04.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.259+0000 7fb593200640 1 --2- 172.21.15.152:0/1782196184 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb5940b2830 0x7fb5940b2ff0 unknown :-1 s=CLOSED pgs=2150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:04.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.259+0000 7fb593200640 1 --2- 172.21.15.152:0/1782196184 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb594071cd0 0x7fb594142f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:04.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.259+0000 7fb593200640 1 --2- 172.21.15.152:0/1782196184 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb59406e720 0x7fb59415dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:04.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.260+0000 7fb593200640 1 -- 172.21.15.152:0/1782196184 >> 172.21.15.152:0/1782196184 conn(0x7fb594067ae0 msgr2=0x7fb594160c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:04.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.260+0000 7fb593200640 1 -- 172.21.15.152:0/1782196184 shutdown_connections 2024-04-20T11:37:04.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.260+0000 7fb593200640 1 -- 172.21.15.152:0/1782196184 wait complete. 2024-04-20T11:37:04.277 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-20T11:37:04.277 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T11:37:04.469 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.469+0000 7f4edfe00640 1 Processor -- start 2024-04-20T11:37:04.469 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.469+0000 7f4edfe00640 1 -- start start 2024-04-20T11:37:04.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.471+0000 7f4edfe00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f4ee015fcc0 0x7f4ee00d4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:04.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.471+0000 7f4edfe00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f4ee01603d0 0x7f4ee00b5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:04.471 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.471+0000 7f4edfe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4ee00b5480 0x7f4ee00b63f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:04.471 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.471+0000 7f4edfe00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f4ee00d3350 con 0x7f4ee015fcc0 2024-04-20T11:37:04.471 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.471+0000 7f4edfe00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f4ee00d3630 con 0x7f4ee01603d0 2024-04-20T11:37:04.471 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.471+0000 7f4edfe00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4ee00d3910 con 0x7f4ee00b5480 2024-04-20T11:37:04.471 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.471+0000 7f4edea00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f4ee015fcc0 0x7f4ee00d4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:04.471 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.471+0000 7f4edea00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f4ee015fcc0 0x7f4ee00d4e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:57226/0 (socket says 172.21.15.152:57226) 2024-04-20T11:37:04.471 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.471+0000 7f4edea00640 1 -- 172.21.15.152:0/947351511 learned_addr learned my addr 172.21.15.152:0/947351511 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:04.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.471+0000 7f4ede000640 1 --2- 172.21.15.152:0/947351511 >> v2:172.21.15.175:3300/0 conn(0x7f4ee01603d0 0x7f4ee00b5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:04.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.472+0000 7f4edea00640 1 -- 172.21.15.152:0/947351511 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4ee00b5480 msgr2=0x7f4ee00b63f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:04.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.472+0000 7f4edf400640 1 --2- 172.21.15.152:0/947351511 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4ee00b5480 0x7f4ee00b63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:04.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.472+0000 7f4edea00640 1 --2- 172.21.15.152:0/947351511 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4ee00b5480 0x7f4ee00b63f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:04.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.472+0000 7f4edea00640 1 -- 172.21.15.152:0/947351511 >> v2:172.21.15.175:3300/0 conn(0x7f4ee01603d0 msgr2=0x7f4ee00b5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:04.473 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.472+0000 7f4edea00640 1 --2- 172.21.15.152:0/947351511 >> v2:172.21.15.175:3300/0 conn(0x7f4ee01603d0 0x7f4ee00b5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:04.473 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.472+0000 7f4edea00640 1 -- 172.21.15.152:0/947351511 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ee00ced20 con 0x7f4ee015fcc0 2024-04-20T11:37:04.473 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.472+0000 7f4edf400640 1 --2- 172.21.15.152:0/947351511 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4ee00b5480 0x7f4ee00b63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:37:04.473 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.473+0000 7f4edea00640 1 --2- 172.21.15.152:0/947351511 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ee015fcc0 0x7f4ee00d4e70 secure :-1 s=READY pgs=2151 cs=0 l=1 rev1=1 crypto rx=0x7f4ee008ca40 tx=0x7f4ed002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca7a9ef646cd354a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:04.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.473+0000 7f4edd600640 1 -- 172.21.15.152:0/947351511 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ed002ca70 con 0x7f4ee015fcc0 2024-04-20T11:37:04.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.473+0000 7f4edd600640 1 -- 172.21.15.152:0/947351511 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ed002cc10 con 0x7f4ee015fcc0 2024-04-20T11:37:04.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.473+0000 7f4edd600640 1 -- 172.21.15.152:0/947351511 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ed00343f0 con 0x7f4ee015fcc0 2024-04-20T11:37:04.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.474+0000 7f4edfe00640 1 -- 172.21.15.152:0/947351511 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ee015fcc0 msgr2=0x7f4ee00d4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:04.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.474+0000 7f4edfe00640 1 --2- 172.21.15.152:0/947351511 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ee015fcc0 0x7f4ee00d4e70 secure :-1 s=READY pgs=2151 cs=0 l=1 rev1=1 crypto rx=0x7f4ee008ca40 tx=0x7f4ed002bed0 comp rx=0 tx=0).stop 2024-04-20T11:37:04.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.474+0000 7f4edfe00640 1 -- 172.21.15.152:0/947351511 shutdown_connections 2024-04-20T11:37:04.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.474+0000 7f4edfe00640 1 --2- 172.21.15.152:0/947351511 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4ee00b5480 0x7f4ee00b63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:04.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.474+0000 7f4edfe00640 1 --2- 172.21.15.152:0/947351511 >> v2:172.21.15.175:3300/0 conn(0x7f4ee01603d0 0x7f4ee00b5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:04.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.474+0000 7f4edfe00640 1 --2- 172.21.15.152:0/947351511 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ee015fcc0 0x7f4ee00d4e70 unknown :-1 s=CLOSED pgs=2151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:04.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.474+0000 7f4edfe00640 1 -- 172.21.15.152:0/947351511 >> 172.21.15.152:0/947351511 conn(0x7f4ee008bfd0 msgr2=0x7f4ee00d4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:04.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.474+0000 7f4edfe00640 1 -- 172.21.15.152:0/947351511 shutdown_connections 2024-04-20T11:37:04.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.474+0000 7f4edfe00640 1 -- 172.21.15.152:0/947351511 wait complete. 2024-04-20T11:37:04.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.476+0000 7f4edfe00640 1 Processor -- start 2024-04-20T11:37:04.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.476+0000 7f4edfe00640 1 -- start start 2024-04-20T11:37:04.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.477+0000 7f4edfe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ee00b5480 0x7f4ee00ca4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:04.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.477+0000 7f4edfe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4ee015fcc0 0x7f4ee00cb620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:04.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.477+0000 7f4edfe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4ee01603d0 0x7f4ee00cbb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:04.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.477+0000 7f4edfe00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee0162650 con 0x7f4ee00b5480 2024-04-20T11:37:04.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.477+0000 7f4edfe00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee01624d0 con 0x7f4ee015fcc0 2024-04-20T11:37:04.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.477+0000 7f4edfe00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4ee01627d0 con 0x7f4ee01603d0 2024-04-20T11:37:04.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.477+0000 7f4edea00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ee00b5480 0x7f4ee00ca4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:04.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.477+0000 7f4edea00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ee00b5480 0x7f4ee00ca4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:57234/0 (socket says 172.21.15.152:57234) 2024-04-20T11:37:04.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.477+0000 7f4edea00640 1 -- 172.21.15.152:0/359649037 learned_addr learned my addr 172.21.15.152:0/359649037 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:04.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.478+0000 7f4edf400640 1 --2- 172.21.15.152:0/359649037 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4ee01603d0 0x7f4ee00cbb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:04.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.478+0000 7f4ede000640 1 --2- 172.21.15.152:0/359649037 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4ee015fcc0 0x7f4ee00cb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:04.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.478+0000 7f4edea00640 1 -- 172.21.15.152:0/359649037 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4ee01603d0 msgr2=0x7f4ee00cbb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:04.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.478+0000 7f4edea00640 1 --2- 172.21.15.152:0/359649037 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4ee01603d0 0x7f4ee00cbb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:04.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.478+0000 7f4edea00640 1 -- 172.21.15.152:0/359649037 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4ee015fcc0 msgr2=0x7f4ee00cb620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:04.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.478+0000 7f4edea00640 1 --2- 172.21.15.152:0/359649037 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4ee015fcc0 0x7f4ee00cb620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:04.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.478+0000 7f4edea00640 1 -- 172.21.15.152:0/359649037 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ee0158c50 con 0x7f4ee00b5480 2024-04-20T11:37:04.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.478+0000 7f4edf400640 1 --2- 172.21.15.152:0/359649037 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4ee01603d0 0x7f4ee00cbb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:37:04.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.478+0000 7f4edea00640 1 --2- 172.21.15.152:0/359649037 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ee00b5480 0x7f4ee00ca4c0 secure :-1 s=READY pgs=2152 cs=0 l=1 rev1=1 crypto rx=0x7f4ed0000f80 tx=0x7f4ed0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:04.480 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.478+0000 7f4ed6a00640 1 -- 172.21.15.152:0/359649037 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ed0043020 con 0x7f4ee00b5480 2024-04-20T11:37:04.480 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.478+0000 7f4ed6a00640 1 -- 172.21.15.152:0/359649037 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ed003e450 con 0x7f4ee00b5480 2024-04-20T11:37:04.480 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.478+0000 7f4ed6a00640 1 -- 172.21.15.152:0/359649037 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ed0009510 con 0x7f4ee00b5480 2024-04-20T11:37:04.480 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.478+0000 7f4edfe00640 1 -- 172.21.15.152:0/359649037 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ee00cc150 con 0x7f4ee00b5480 2024-04-20T11:37:04.480 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.479+0000 7f4edfe00640 1 -- 172.21.15.152:0/359649037 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ee00cca10 con 0x7f4ee00b5480 2024-04-20T11:37:04.480 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.480+0000 7f4edfe00640 1 -- 172.21.15.152:0/359649037 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ee00cc800 con 0x7f4ee00b5480 2024-04-20T11:37:04.480 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.481+0000 7f4ed6a00640 1 -- 172.21.15.152:0/359649037 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) ==== 91754+0+0 (secure 0 0 0) 0x7f4ed0046020 con 0x7f4ee00b5480 2024-04-20T11:37:04.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.482+0000 7f4ed6a00640 1 --2- 172.21.15.152:0/359649037 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4ebc06d0d0 0x7f4ebc06f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:04.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.482+0000 7f4ed6a00640 1 -- 172.21.15.152:0/359649037 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(154..154 src has 1..154) ==== 5709+0+0 (secure 0 0 0) 0x7f4ed00af5d0 con 0x7f4ee00b5480 2024-04-20T11:37:04.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.482+0000 7f4ede000640 1 --2- 172.21.15.152:0/359649037 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4ebc06d0d0 0x7f4ebc06f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:04.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.483+0000 7f4ede000640 1 --2- 172.21.15.152:0/359649037 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4ebc06d0d0 0x7f4ebc06f590 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f4ee0068f00 tx=0x7f4ec8008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:04.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.486+0000 7f4ed6a00640 1 -- 172.21.15.152:0/359649037 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f4ed004d030 con 0x7f4ee00b5480 2024-04-20T11:37:04.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.684+0000 7f4edfe00640 1 -- 172.21.15.152:0/359649037 --> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4ee00b6ce0 con 0x7f4ebc06d0d0 2024-04-20T11:37:04.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.702+0000 7f4ed6a00640 1 -- 172.21.15.152:0/359649037 <== mgr.4109 v2:172.21.15.175:6836/4021250441 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+240297 (secure 0 0 0) 0x7f4ee00b6ce0 con 0x7f4ebc06d0d0 2024-04-20T11:37:04.705 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:37:04.707 INFO:teuthology.orchestra.run.smithi152.stderr:dumped all 2024-04-20T11:37:04.708 INFO:tasks.ceph.mds.c.smithi152.stderr:2024-04-20T11:37:04.706+0000 7f68ca200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-20T11:37:04.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.707+0000 7f4edfe00640 1 -- 172.21.15.152:0/359649037 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4ebc06d0d0 msgr2=0x7f4ebc06f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:04.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.707+0000 7f4edfe00640 1 --2- 172.21.15.152:0/359649037 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4ebc06d0d0 0x7f4ebc06f590 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f4ee0068f00 tx=0x7f4ec8008040 comp rx=0 tx=0).stop 2024-04-20T11:37:04.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.707+0000 7f4edfe00640 1 -- 172.21.15.152:0/359649037 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ee00b5480 msgr2=0x7f4ee00ca4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:04.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.707+0000 7f4edfe00640 1 --2- 172.21.15.152:0/359649037 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ee00b5480 0x7f4ee00ca4c0 secure :-1 s=READY pgs=2152 cs=0 l=1 rev1=1 crypto rx=0x7f4ed0000f80 tx=0x7f4ed0004a40 comp rx=0 tx=0).stop 2024-04-20T11:37:04.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.708+0000 7f4edfe00640 1 -- 172.21.15.152:0/359649037 shutdown_connections 2024-04-20T11:37:04.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.708+0000 7f4edfe00640 1 --2- 172.21.15.152:0/359649037 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4ebc06d0d0 0x7f4ebc06f590 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:04.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.708+0000 7f4edfe00640 1 --2- 172.21.15.152:0/359649037 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4ee01603d0 0x7f4ee00cbb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:04.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.708+0000 7f4edfe00640 1 --2- 172.21.15.152:0/359649037 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4ee015fcc0 0x7f4ee00cb620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:04.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.708+0000 7f4edfe00640 1 --2- 172.21.15.152:0/359649037 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ee00b5480 0x7f4ee00ca4c0 unknown :-1 s=CLOSED pgs=2152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:04.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.708+0000 7f4edfe00640 1 -- 172.21.15.152:0/359649037 >> 172.21.15.152:0/359649037 conn(0x7f4ee008bfd0 msgr2=0x7f4ee006d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:04.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.708+0000 7f4edfe00640 1 -- 172.21.15.152:0/359649037 shutdown_connections 2024-04-20T11:37:04.710 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.708+0000 7f4edfe00640 1 -- 172.21.15.152:0/359649037 wait complete. 2024-04-20T11:37:04.726 INFO:teuthology.orchestra.run.smithi152.stdout:{"pg_ready":true,"pg_map":{"version":226,"stamp":"2024-04-20T11:37:02.699196+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":265460,"kb_used_data":15980,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754709260,"statfs":{"total":773094113280,"available":772822282240,"internally_reserved":0,"allocated":16363520,"data_stored":7862715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12746,"internal_metadata":255315510},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.395304"},"pg_stats":[{"pgid":"25.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:36:59.299355+0000","last_change":"2024-04-20T11:36:59.299355+0000","last_active":"2024-04-20T11:36:59.299355+0000","last_peered":"2024-04-20T11:36:59.299355+0000","last_clean":"2024-04-20T11:36:59.299355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:59.299355+0000","last_undegraded":"2024-04-20T11:36:59.299355+0000","last_fullsized":"2024-04-20T11:36:59.299355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:36:59.299355+0000","last_change":"2024-04-20T11:36:59.299355+0000","last_active":"2024-04-20T11:36:59.299355+0000","last_peered":"2024-04-20T11:36:59.299355+0000","last_clean":"2024-04-20T11:36:59.299355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:59.299355+0000","last_undegraded":"2024-04-20T11:36:59.299355+0000","last_fullsized":"2024-04-20T11:36:59.299355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:36:59.299355+0000","last_change":"2024-04-20T11:36:59.299355+0000","last_active":"2024-04-20T11:36:59.299355+0000","last_peered":"2024-04-20T11:36:59.299355+0000","last_clean":"2024-04-20T11:36:59.299355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:59.299355+0000","last_undegraded":"2024-04-20T11:36:59.299355+0000","last_fullsized":"2024-04-20T11:36:59.299355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:36:59.299355+0000","last_change":"2024-04-20T11:36:59.299355+0000","last_active":"2024-04-20T11:36:59.299355+0000","last_peered":"2024-04-20T11:36:59.299355+0000","last_clean":"2024-04-20T11:36:59.299355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:59.299355+0000","last_undegraded":"2024-04-20T11:36:59.299355+0000","last_fullsized":"2024-04-20T11:36:59.299355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:36:59.299355+0000","last_change":"2024-04-20T11:36:59.299355+0000","last_active":"2024-04-20T11:36:59.299355+0000","last_peered":"2024-04-20T11:36:59.299355+0000","last_clean":"2024-04-20T11:36:59.299355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:59.299355+0000","last_undegraded":"2024-04-20T11:36:59.299355+0000","last_fullsized":"2024-04-20T11:36:59.299355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:36:59.299355+0000","last_change":"2024-04-20T11:36:59.299355+0000","last_active":"2024-04-20T11:36:59.299355+0000","last_peered":"2024-04-20T11:36:59.299355+0000","last_clean":"2024-04-20T11:36:59.299355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:59.299355+0000","last_undegraded":"2024-04-20T11:36:59.299355+0000","last_fullsized":"2024-04-20T11:36:59.299355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:36:59.299355+0000","last_change":"2024-04-20T11:36:59.299355+0000","last_active":"2024-04-20T11:36:59.299355+0000","last_peered":"2024-04-20T11:36:59.299355+0000","last_clean":"2024-04-20T11:36:59.299355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:59.299355+0000","last_undegraded":"2024-04-20T11:36:59.299355+0000","last_fullsized":"2024-04-20T11:36:59.299355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:36:59.299355+0000","last_change":"2024-04-20T11:36:59.299355+0000","last_active":"2024-04-20T11:36:59.299355+0000","last_peered":"2024-04-20T11:36:59.299355+0000","last_clean":"2024-04-20T11:36:59.299355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:59.299355+0000","last_undegraded":"2024-04-20T11:36:59.299355+0000","last_fullsized":"2024-04-20T11:36:59.299355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:36:59.299355+0000","last_change":"2024-04-20T11:36:59.299355+0000","last_active":"2024-04-20T11:36:59.299355+0000","last_peered":"2024-04-20T11:36:59.299355+0000","last_clean":"2024-04-20T11:36:59.299355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:59.299355+0000","last_undegraded":"2024-04-20T11:36:59.299355+0000","last_fullsized":"2024-04-20T11:36:59.299355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:36:59.299355+0000","last_change":"2024-04-20T11:36:59.299355+0000","last_active":"2024-04-20T11:36:59.299355+0000","last_peered":"2024-04-20T11:36:59.299355+0000","last_clean":"2024-04-20T11:36:59.299355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:59.299355+0000","last_undegraded":"2024-04-20T11:36:59.299355+0000","last_fullsized":"2024-04-20T11:36:59.299355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:36:59.299355+0000","last_change":"2024-04-20T11:36:59.299355+0000","last_active":"2024-04-20T11:36:59.299355+0000","last_peered":"2024-04-20T11:36:59.299355+0000","last_clean":"2024-04-20T11:36:59.299355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:59.299355+0000","last_undegraded":"2024-04-20T11:36:59.299355+0000","last_fullsized":"2024-04-20T11:36:59.299355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:36:59.299355+0000","last_change":"2024-04-20T11:36:59.299355+0000","last_active":"2024-04-20T11:36:59.299355+0000","last_peered":"2024-04-20T11:36:59.299355+0000","last_clean":"2024-04-20T11:36:59.299355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:59.299355+0000","last_undegraded":"2024-04-20T11:36:59.299355+0000","last_fullsized":"2024-04-20T11:36:59.299355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:36:59.299355+0000","last_change":"2024-04-20T11:36:59.299355+0000","last_active":"2024-04-20T11:36:59.299355+0000","last_peered":"2024-04-20T11:36:59.299355+0000","last_clean":"2024-04-20T11:36:59.299355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:59.299355+0000","last_undegraded":"2024-04-20T11:36:59.299355+0000","last_fullsized":"2024-04-20T11:36:59.299355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:36:59.299355+0000","last_change":"2024-04-20T11:36:59.299355+0000","last_active":"2024-04-20T11:36:59.299355+0000","last_peered":"2024-04-20T11:36:59.299355+0000","last_clean":"2024-04-20T11:36:59.299355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:59.299355+0000","last_undegraded":"2024-04-20T11:36:59.299355+0000","last_fullsized":"2024-04-20T11:36:59.299355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:36:59.299355+0000","last_change":"2024-04-20T11:36:59.299355+0000","last_active":"2024-04-20T11:36:59.299355+0000","last_peered":"2024-04-20T11:36:59.299355+0000","last_clean":"2024-04-20T11:36:59.299355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:59.299355+0000","last_undegraded":"2024-04-20T11:36:59.299355+0000","last_fullsized":"2024-04-20T11:36:59.299355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:36:59.299355+0000","last_change":"2024-04-20T11:36:59.299355+0000","last_active":"2024-04-20T11:36:59.299355+0000","last_peered":"2024-04-20T11:36:59.299355+0000","last_clean":"2024-04-20T11:36:59.299355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:59.299355+0000","last_undegraded":"2024-04-20T11:36:59.299355+0000","last_fullsized":"2024-04-20T11:36:59.299355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:36:59.299355+0000","last_change":"2024-04-20T11:36:59.299355+0000","last_active":"2024-04-20T11:36:59.299355+0000","last_peered":"2024-04-20T11:36:59.299355+0000","last_clean":"2024-04-20T11:36:59.299355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:59.299355+0000","last_undegraded":"2024-04-20T11:36:59.299355+0000","last_fullsized":"2024-04-20T11:36:59.299355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:36:59.299355+0000","last_change":"2024-04-20T11:36:59.299355+0000","last_active":"2024-04-20T11:36:59.299355+0000","last_peered":"2024-04-20T11:36:59.299355+0000","last_clean":"2024-04-20T11:36:59.299355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:59.299355+0000","last_undegraded":"2024-04-20T11:36:59.299355+0000","last_fullsized":"2024-04-20T11:36:59.299355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:36:59.299355+0000","last_change":"2024-04-20T11:36:59.299355+0000","last_active":"2024-04-20T11:36:59.299355+0000","last_peered":"2024-04-20T11:36:59.299355+0000","last_clean":"2024-04-20T11:36:59.299355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:59.299355+0000","last_undegraded":"2024-04-20T11:36:59.299355+0000","last_fullsized":"2024-04-20T11:36:59.299355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:36:59.299355+0000","last_change":"2024-04-20T11:36:59.299355+0000","last_active":"2024-04-20T11:36:59.299355+0000","last_peered":"2024-04-20T11:36:59.299355+0000","last_clean":"2024-04-20T11:36:59.299355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:59.299355+0000","last_undegraded":"2024-04-20T11:36:59.299355+0000","last_fullsized":"2024-04-20T11:36:59.299355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:36:59.299355+0000","last_change":"2024-04-20T11:36:59.299355+0000","last_active":"2024-04-20T11:36:59.299355+0000","last_peered":"2024-04-20T11:36:59.299355+0000","last_clean":"2024-04-20T11:36:59.299355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:59.299355+0000","last_undegraded":"2024-04-20T11:36:59.299355+0000","last_fullsized":"2024-04-20T11:36:59.299355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:36:59.299355+0000","last_change":"2024-04-20T11:36:59.299355+0000","last_active":"2024-04-20T11:36:59.299355+0000","last_peered":"2024-04-20T11:36:59.299355+0000","last_clean":"2024-04-20T11:36:59.299355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:59.299355+0000","last_undegraded":"2024-04-20T11:36:59.299355+0000","last_fullsized":"2024-04-20T11:36:59.299355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:36:59.299355+0000","last_change":"2024-04-20T11:36:59.299355+0000","last_active":"2024-04-20T11:36:59.299355+0000","last_peered":"2024-04-20T11:36:59.299355+0000","last_clean":"2024-04-20T11:36:59.299355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:59.299355+0000","last_undegraded":"2024-04-20T11:36:59.299355+0000","last_fullsized":"2024-04-20T11:36:59.299355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:36:59.299355+0000","last_change":"2024-04-20T11:36:59.299355+0000","last_active":"2024-04-20T11:36:59.299355+0000","last_peered":"2024-04-20T11:36:59.299355+0000","last_clean":"2024-04-20T11:36:59.299355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:59.299355+0000","last_undegraded":"2024-04-20T11:36:59.299355+0000","last_fullsized":"2024-04-20T11:36:59.299355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:36:59.299355+0000","last_change":"2024-04-20T11:36:59.299355+0000","last_active":"2024-04-20T11:36:59.299355+0000","last_peered":"2024-04-20T11:36:59.299355+0000","last_clean":"2024-04-20T11:36:59.299355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:59.299355+0000","last_undegraded":"2024-04-20T11:36:59.299355+0000","last_fullsized":"2024-04-20T11:36:59.299355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:36:59.299355+0000","last_change":"2024-04-20T11:36:59.299355+0000","last_active":"2024-04-20T11:36:59.299355+0000","last_peered":"2024-04-20T11:36:59.299355+0000","last_clean":"2024-04-20T11:36:59.299355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:59.299355+0000","last_undegraded":"2024-04-20T11:36:59.299355+0000","last_fullsized":"2024-04-20T11:36:59.299355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:36:59.299355+0000","last_change":"2024-04-20T11:36:59.299355+0000","last_active":"2024-04-20T11:36:59.299355+0000","last_peered":"2024-04-20T11:36:59.299355+0000","last_clean":"2024-04-20T11:36:59.299355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:59.299355+0000","last_undegraded":"2024-04-20T11:36:59.299355+0000","last_fullsized":"2024-04-20T11:36:59.299355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:36:59.299355+0000","last_change":"2024-04-20T11:36:59.299355+0000","last_active":"2024-04-20T11:36:59.299355+0000","last_peered":"2024-04-20T11:36:59.299355+0000","last_clean":"2024-04-20T11:36:59.299355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:59.299355+0000","last_undegraded":"2024-04-20T11:36:59.299355+0000","last_fullsized":"2024-04-20T11:36:59.299355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:36:59.299355+0000","last_change":"2024-04-20T11:36:59.299355+0000","last_active":"2024-04-20T11:36:59.299355+0000","last_peered":"2024-04-20T11:36:59.299355+0000","last_clean":"2024-04-20T11:36:59.299355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:59.299355+0000","last_undegraded":"2024-04-20T11:36:59.299355+0000","last_fullsized":"2024-04-20T11:36:59.299355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:36:59.299355+0000","last_change":"2024-04-20T11:36:59.299355+0000","last_active":"2024-04-20T11:36:59.299355+0000","last_peered":"2024-04-20T11:36:59.299355+0000","last_clean":"2024-04-20T11:36:59.299355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:59.299355+0000","last_undegraded":"2024-04-20T11:36:59.299355+0000","last_fullsized":"2024-04-20T11:36:59.299355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:36:59.299355+0000","last_change":"2024-04-20T11:36:59.299355+0000","last_active":"2024-04-20T11:36:59.299355+0000","last_peered":"2024-04-20T11:36:59.299355+0000","last_clean":"2024-04-20T11:36:59.299355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:59.299355+0000","last_undegraded":"2024-04-20T11:36:59.299355+0000","last_fullsized":"2024-04-20T11:36:59.299355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:36:59.299355+0000","last_change":"2024-04-20T11:36:59.299355+0000","last_active":"2024-04-20T11:36:59.299355+0000","last_peered":"2024-04-20T11:36:59.299355+0000","last_clean":"2024-04-20T11:36:59.299355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:59.299355+0000","last_undegraded":"2024-04-20T11:36:59.299355+0000","last_fullsized":"2024-04-20T11:36:59.299355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:36:59.299355+0000","last_change":"2024-04-20T11:36:59.299355+0000","last_active":"2024-04-20T11:36:59.299355+0000","last_peered":"2024-04-20T11:36:59.299355+0000","last_clean":"2024-04-20T11:36:59.299355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:59.299355+0000","last_undegraded":"2024-04-20T11:36:59.299355+0000","last_fullsized":"2024-04-20T11:36:59.299355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1f","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-04-20T11:36:58.307880+0000","last_change":"2024-04-20T11:36:58.307880+0000","last_active":"2024-04-20T11:36:58.282949+0000","last_peered":"2024-04-20T11:36:58.282949+0000","last_clean":"2024-04-20T11:36:58.282949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:58.307880+0000","last_undegraded":"2024-04-20T11:36:58.307880+0000","last_fullsized":"2024-04-20T11:36:58.307880+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_clean_scrub_stamp":"2024-04-20T11:36:58.282949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:36:59.299355+0000","last_change":"2024-04-20T11:36:59.299355+0000","last_active":"2024-04-20T11:36:59.299355+0000","last_peered":"2024-04-20T11:36:59.299355+0000","last_clean":"2024-04-20T11:36:59.299355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:59.299355+0000","last_undegraded":"2024-04-20T11:36:59.299355+0000","last_fullsized":"2024-04-20T11:36:59.299355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1e","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-04-20T11:36:58.310504+0000","last_change":"2024-04-20T11:36:58.310504+0000","last_active":"2024-04-20T11:36:58.282949+0000","last_peered":"2024-04-20T11:36:58.282949+0000","last_clean":"2024-04-20T11:36:58.282949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:58.310504+0000","last_undegraded":"2024-04-20T11:36:58.310504+0000","last_fullsized":"2024-04-20T11:36:58.310504+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_clean_scrub_stamp":"2024-04-20T11:36:58.282949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:36:59.299355+0000","last_change":"2024-04-20T11:36:59.299355+0000","last_active":"2024-04-20T11:36:59.299355+0000","last_peered":"2024-04-20T11:36:59.299355+0000","last_clean":"2024-04-20T11:36:59.299355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:59.299355+0000","last_undegraded":"2024-04-20T11:36:59.299355+0000","last_fullsized":"2024-04-20T11:36:59.299355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1d","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-04-20T11:36:58.313383+0000","last_change":"2024-04-20T11:36:58.313383+0000","last_active":"2024-04-20T11:36:58.282949+0000","last_peered":"2024-04-20T11:36:58.282949+0000","last_clean":"2024-04-20T11:36:58.282949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:58.313383+0000","last_undegraded":"2024-04-20T11:36:58.313383+0000","last_fullsized":"2024-04-20T11:36:58.313383+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_clean_scrub_stamp":"2024-04-20T11:36:58.282949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:36:59.299355+0000","last_change":"2024-04-20T11:36:59.299355+0000","last_active":"2024-04-20T11:36:59.299355+0000","last_peered":"2024-04-20T11:36:59.299355+0000","last_clean":"2024-04-20T11:36:59.299355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:59.299355+0000","last_undegraded":"2024-04-20T11:36:59.299355+0000","last_fullsized":"2024-04-20T11:36:59.299355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1c","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-04-20T11:36:58.315474+0000","last_change":"2024-04-20T11:36:58.315474+0000","last_active":"2024-04-20T11:36:58.282949+0000","last_peered":"2024-04-20T11:36:58.282949+0000","last_clean":"2024-04-20T11:36:58.282949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:58.315474+0000","last_undegraded":"2024-04-20T11:36:58.315474+0000","last_fullsized":"2024-04-20T11:36:58.315474+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_clean_scrub_stamp":"2024-04-20T11:36:58.282949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:36:59.299355+0000","last_change":"2024-04-20T11:36:59.299355+0000","last_active":"2024-04-20T11:36:59.299355+0000","last_peered":"2024-04-20T11:36:59.299355+0000","last_clean":"2024-04-20T11:36:59.299355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:59.299355+0000","last_undegraded":"2024-04-20T11:36:59.299355+0000","last_fullsized":"2024-04-20T11:36:59.299355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1b","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-04-20T11:36:58.306812+0000","last_change":"2024-04-20T11:36:58.306812+0000","last_active":"2024-04-20T11:36:58.282949+0000","last_peered":"2024-04-20T11:36:58.282949+0000","last_clean":"2024-04-20T11:36:58.282949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:58.306812+0000","last_undegraded":"2024-04-20T11:36:58.306812+0000","last_fullsized":"2024-04-20T11:36:58.306812+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_clean_scrub_stamp":"2024-04-20T11:36:58.282949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:36:59.299355+0000","last_change":"2024-04-20T11:36:59.299355+0000","last_active":"2024-04-20T11:36:59.299355+0000","last_peered":"2024-04-20T11:36:59.299355+0000","last_clean":"2024-04-20T11:36:59.299355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:59.299355+0000","last_undegraded":"2024-04-20T11:36:59.299355+0000","last_fullsized":"2024-04-20T11:36:59.299355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1a","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-04-20T11:36:58.310130+0000","last_change":"2024-04-20T11:36:58.310130+0000","last_active":"2024-04-20T11:36:58.282949+0000","last_peered":"2024-04-20T11:36:58.282949+0000","last_clean":"2024-04-20T11:36:58.282949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:58.310130+0000","last_undegraded":"2024-04-20T11:36:58.310130+0000","last_fullsized":"2024-04-20T11:36:58.310130+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_clean_scrub_stamp":"2024-04-20T11:36:58.282949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:36:59.299355+0000","last_change":"2024-04-20T11:36:59.299355+0000","last_active":"2024-04-20T11:36:59.299355+0000","last_peered":"2024-04-20T11:36:59.299355+0000","last_clean":"2024-04-20T11:36:59.299355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:59.299355+0000","last_undegraded":"2024-04-20T11:36:59.299355+0000","last_fullsized":"2024-04-20T11:36:59.299355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-04-20T11:36:58.320064+0000","last_change":"2024-04-20T11:36:58.320064+0000","last_active":"2024-04-20T11:36:58.282949+0000","last_peered":"2024-04-20T11:36:58.282949+0000","last_clean":"2024-04-20T11:36:58.282949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:58.320064+0000","last_undegraded":"2024-04-20T11:36:58.320064+0000","last_fullsized":"2024-04-20T11:36:58.320064+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_clean_scrub_stamp":"2024-04-20T11:36:58.282949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:36:59.299355+0000","last_change":"2024-04-20T11:36:59.299355+0000","last_active":"2024-04-20T11:36:59.299355+0000","last_peered":"2024-04-20T11:36:59.299355+0000","last_clean":"2024-04-20T11:36:59.299355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:59.299355+0000","last_undegraded":"2024-04-20T11:36:59.299355+0000","last_fullsized":"2024-04-20T11:36:59.299355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-04-20T11:36:58.306666+0000","last_change":"2024-04-20T11:36:58.306666+0000","last_active":"2024-04-20T11:36:58.282949+0000","last_peered":"2024-04-20T11:36:58.282949+0000","last_clean":"2024-04-20T11:36:58.282949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:58.306666+0000","last_undegraded":"2024-04-20T11:36:58.306666+0000","last_fullsized":"2024-04-20T11:36:58.306666+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_clean_scrub_stamp":"2024-04-20T11:36:58.282949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:36:59.299355+0000","last_change":"2024-04-20T11:36:59.299355+0000","last_active":"2024-04-20T11:36:59.299355+0000","last_peered":"2024-04-20T11:36:59.299355+0000","last_clean":"2024-04-20T11:36:59.299355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:59.299355+0000","last_undegraded":"2024-04-20T11:36:59.299355+0000","last_fullsized":"2024-04-20T11:36:59.299355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-04-20T11:36:58.306570+0000","last_change":"2024-04-20T11:36:58.306570+0000","last_active":"2024-04-20T11:36:58.282949+0000","last_peered":"2024-04-20T11:36:58.282949+0000","last_clean":"2024-04-20T11:36:58.282949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:58.306570+0000","last_undegraded":"2024-04-20T11:36:58.306570+0000","last_fullsized":"2024-04-20T11:36:58.306570+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_clean_scrub_stamp":"2024-04-20T11:36:58.282949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:36:59.299355+0000","last_change":"2024-04-20T11:36:59.299355+0000","last_active":"2024-04-20T11:36:59.299355+0000","last_peered":"2024-04-20T11:36:59.299355+0000","last_clean":"2024-04-20T11:36:59.299355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:59.299355+0000","last_undegraded":"2024-04-20T11:36:59.299355+0000","last_fullsized":"2024-04-20T11:36:59.299355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.15","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-04-20T11:36:58.306640+0000","last_change":"2024-04-20T11:36:58.306640+0000","last_active":"2024-04-20T11:36:58.282949+0000","last_peered":"2024-04-20T11:36:58.282949+0000","last_clean":"2024-04-20T11:36:58.282949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:58.306640+0000","last_undegraded":"2024-04-20T11:36:58.306640+0000","last_fullsized":"2024-04-20T11:36:58.306640+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_clean_scrub_stamp":"2024-04-20T11:36:58.282949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:36:59.299355+0000","last_change":"2024-04-20T11:36:59.299355+0000","last_active":"2024-04-20T11:36:59.299355+0000","last_peered":"2024-04-20T11:36:59.299355+0000","last_clean":"2024-04-20T11:36:59.299355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:59.299355+0000","last_undegraded":"2024-04-20T11:36:59.299355+0000","last_fullsized":"2024-04-20T11:36:59.299355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.14","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-04-20T11:36:58.314567+0000","last_change":"2024-04-20T11:36:58.314567+0000","last_active":"2024-04-20T11:36:58.282949+0000","last_peered":"2024-04-20T11:36:58.282949+0000","last_clean":"2024-04-20T11:36:58.282949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:58.314567+0000","last_undegraded":"2024-04-20T11:36:58.314567+0000","last_fullsized":"2024-04-20T11:36:58.314567+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_clean_scrub_stamp":"2024-04-20T11:36:58.282949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:36:59.299355+0000","last_change":"2024-04-20T11:36:59.299355+0000","last_active":"2024-04-20T11:36:59.299355+0000","last_peered":"2024-04-20T11:36:59.299355+0000","last_clean":"2024-04-20T11:36:59.299355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:59.299355+0000","last_undegraded":"2024-04-20T11:36:59.299355+0000","last_fullsized":"2024-04-20T11:36:59.299355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.13","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-04-20T11:36:58.318384+0000","last_change":"2024-04-20T11:36:58.318384+0000","last_active":"2024-04-20T11:36:58.282949+0000","last_peered":"2024-04-20T11:36:58.282949+0000","last_clean":"2024-04-20T11:36:58.282949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:58.318384+0000","last_undegraded":"2024-04-20T11:36:58.318384+0000","last_fullsized":"2024-04-20T11:36:58.318384+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_clean_scrub_stamp":"2024-04-20T11:36:58.282949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:36:59.299355+0000","last_change":"2024-04-20T11:36:59.299355+0000","last_active":"2024-04-20T11:36:59.299355+0000","last_peered":"2024-04-20T11:36:59.299355+0000","last_clean":"2024-04-20T11:36:59.299355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:59.299355+0000","last_undegraded":"2024-04-20T11:36:59.299355+0000","last_fullsized":"2024-04-20T11:36:59.299355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-04-20T11:36:58.309610+0000","last_change":"2024-04-20T11:36:58.309610+0000","last_active":"2024-04-20T11:36:58.282949+0000","last_peered":"2024-04-20T11:36:58.282949+0000","last_clean":"2024-04-20T11:36:58.282949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:58.309610+0000","last_undegraded":"2024-04-20T11:36:58.309610+0000","last_fullsized":"2024-04-20T11:36:58.309610+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_clean_scrub_stamp":"2024-04-20T11:36:58.282949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:36:59.299355+0000","last_change":"2024-04-20T11:36:59.299355+0000","last_active":"2024-04-20T11:36:59.299355+0000","last_peered":"2024-04-20T11:36:59.299355+0000","last_clean":"2024-04-20T11:36:59.299355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:59.299355+0000","last_undegraded":"2024-04-20T11:36:59.299355+0000","last_fullsized":"2024-04-20T11:36:59.299355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-04-20T11:36:58.311837+0000","last_change":"2024-04-20T11:36:58.311837+0000","last_active":"2024-04-20T11:36:58.282949+0000","last_peered":"2024-04-20T11:36:58.282949+0000","last_clean":"2024-04-20T11:36:58.282949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:58.311837+0000","last_undegraded":"2024-04-20T11:36:58.311837+0000","last_fullsized":"2024-04-20T11:36:58.311837+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_clean_scrub_stamp":"2024-04-20T11:36:58.282949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:36:59.299355+0000","last_change":"2024-04-20T11:36:59.299355+0000","last_active":"2024-04-20T11:36:59.299355+0000","last_peered":"2024-04-20T11:36:59.299355+0000","last_clean":"2024-04-20T11:36:59.299355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:59.299355+0000","last_undegraded":"2024-04-20T11:36:59.299355+0000","last_fullsized":"2024-04-20T11:36:59.299355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.10","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-04-20T11:36:58.318369+0000","last_change":"2024-04-20T11:36:58.318369+0000","last_active":"2024-04-20T11:36:58.282949+0000","last_peered":"2024-04-20T11:36:58.282949+0000","last_clean":"2024-04-20T11:36:58.282949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:58.318369+0000","last_undegraded":"2024-04-20T11:36:58.318369+0000","last_fullsized":"2024-04-20T11:36:58.318369+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_clean_scrub_stamp":"2024-04-20T11:36:58.282949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:36:59.299355+0000","last_change":"2024-04-20T11:36:59.299355+0000","last_active":"2024-04-20T11:36:59.299355+0000","last_peered":"2024-04-20T11:36:59.299355+0000","last_clean":"2024-04-20T11:36:59.299355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:59.299355+0000","last_undegraded":"2024-04-20T11:36:59.299355+0000","last_fullsized":"2024-04-20T11:36:59.299355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.f","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-04-20T11:36:58.306676+0000","last_change":"2024-04-20T11:36:58.306676+0000","last_active":"2024-04-20T11:36:58.282949+0000","last_peered":"2024-04-20T11:36:58.282949+0000","last_clean":"2024-04-20T11:36:58.282949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:58.306676+0000","last_undegraded":"2024-04-20T11:36:58.306676+0000","last_fullsized":"2024-04-20T11:36:58.306676+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_clean_scrub_stamp":"2024-04-20T11:36:58.282949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:36:59.299355+0000","last_change":"2024-04-20T11:36:59.299355+0000","last_active":"2024-04-20T11:36:59.299355+0000","last_peered":"2024-04-20T11:36:59.299355+0000","last_clean":"2024-04-20T11:36:59.299355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:59.299355+0000","last_undegraded":"2024-04-20T11:36:59.299355+0000","last_fullsized":"2024-04-20T11:36:59.299355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.e","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-04-20T11:36:58.311188+0000","last_change":"2024-04-20T11:36:58.311188+0000","last_active":"2024-04-20T11:36:58.282949+0000","last_peered":"2024-04-20T11:36:58.282949+0000","last_clean":"2024-04-20T11:36:58.282949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:58.311188+0000","last_undegraded":"2024-04-20T11:36:58.311188+0000","last_fullsized":"2024-04-20T11:36:58.311188+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_clean_scrub_stamp":"2024-04-20T11:36:58.282949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:36:59.299355+0000","last_change":"2024-04-20T11:36:59.299355+0000","last_active":"2024-04-20T11:36:59.299355+0000","last_peered":"2024-04-20T11:36:59.299355+0000","last_clean":"2024-04-20T11:36:59.299355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:59.299355+0000","last_undegraded":"2024-04-20T11:36:59.299355+0000","last_fullsized":"2024-04-20T11:36:59.299355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.d","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-04-20T11:36:58.318543+0000","last_change":"2024-04-20T11:36:58.318543+0000","last_active":"2024-04-20T11:36:58.282949+0000","last_peered":"2024-04-20T11:36:58.282949+0000","last_clean":"2024-04-20T11:36:58.282949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:58.318543+0000","last_undegraded":"2024-04-20T11:36:58.318543+0000","last_fullsized":"2024-04-20T11:36:58.318543+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_clean_scrub_stamp":"2024-04-20T11:36:58.282949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:36:59.299355+0000","last_change":"2024-04-20T11:36:59.299355+0000","last_active":"2024-04-20T11:36:59.299355+0000","last_peered":"2024-04-20T11:36:59.299355+0000","last_clean":"2024-04-20T11:36:59.299355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:59.299355+0000","last_undegraded":"2024-04-20T11:36:59.299355+0000","last_fullsized":"2024-04-20T11:36:59.299355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-04-20T11:36:58.306765+0000","last_change":"2024-04-20T11:36:58.306765+0000","last_active":"2024-04-20T11:36:58.282949+0000","last_peered":"2024-04-20T11:36:58.282949+0000","last_clean":"2024-04-20T11:36:58.282949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:58.306765+0000","last_undegraded":"2024-04-20T11:36:58.306765+0000","last_fullsized":"2024-04-20T11:36:58.306765+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_clean_scrub_stamp":"2024-04-20T11:36:58.282949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:36:59.299355+0000","last_change":"2024-04-20T11:36:59.299355+0000","last_active":"2024-04-20T11:36:59.299355+0000","last_peered":"2024-04-20T11:36:59.299355+0000","last_clean":"2024-04-20T11:36:59.299355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:59.299355+0000","last_undegraded":"2024-04-20T11:36:59.299355+0000","last_fullsized":"2024-04-20T11:36:59.299355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.b","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-04-20T11:36:58.314265+0000","last_change":"2024-04-20T11:36:58.314265+0000","last_active":"2024-04-20T11:36:58.282949+0000","last_peered":"2024-04-20T11:36:58.282949+0000","last_clean":"2024-04-20T11:36:58.282949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:58.314265+0000","last_undegraded":"2024-04-20T11:36:58.314265+0000","last_fullsized":"2024-04-20T11:36:58.314265+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_clean_scrub_stamp":"2024-04-20T11:36:58.282949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:36:59.299355+0000","last_change":"2024-04-20T11:36:59.299355+0000","last_active":"2024-04-20T11:36:59.299355+0000","last_peered":"2024-04-20T11:36:59.299355+0000","last_clean":"2024-04-20T11:36:59.299355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:59.299355+0000","last_undegraded":"2024-04-20T11:36:59.299355+0000","last_fullsized":"2024-04-20T11:36:59.299355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-04-20T11:36:58.306545+0000","last_change":"2024-04-20T11:36:58.306545+0000","last_active":"2024-04-20T11:36:58.282949+0000","last_peered":"2024-04-20T11:36:58.282949+0000","last_clean":"2024-04-20T11:36:58.282949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:58.306545+0000","last_undegraded":"2024-04-20T11:36:58.306545+0000","last_fullsized":"2024-04-20T11:36:58.306545+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_clean_scrub_stamp":"2024-04-20T11:36:58.282949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:36:59.299355+0000","last_change":"2024-04-20T11:36:59.299355+0000","last_active":"2024-04-20T11:36:59.299355+0000","last_peered":"2024-04-20T11:36:59.299355+0000","last_clean":"2024-04-20T11:36:59.299355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:59.299355+0000","last_undegraded":"2024-04-20T11:36:59.299355+0000","last_fullsized":"2024-04-20T11:36:59.299355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-04-20T11:36:58.313283+0000","last_change":"2024-04-20T11:36:58.313283+0000","last_active":"2024-04-20T11:36:58.282949+0000","last_peered":"2024-04-20T11:36:58.282949+0000","last_clean":"2024-04-20T11:36:58.282949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:58.313283+0000","last_undegraded":"2024-04-20T11:36:58.313283+0000","last_fullsized":"2024-04-20T11:36:58.313283+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_clean_scrub_stamp":"2024-04-20T11:36:58.282949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:36:59.299355+0000","last_change":"2024-04-20T11:36:59.299355+0000","last_active":"2024-04-20T11:36:59.299355+0000","last_peered":"2024-04-20T11:36:59.299355+0000","last_clean":"2024-04-20T11:36:59.299355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:59.299355+0000","last_undegraded":"2024-04-20T11:36:59.299355+0000","last_fullsized":"2024-04-20T11:36:59.299355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-04-20T11:36:58.309993+0000","last_change":"2024-04-20T11:36:58.309993+0000","last_active":"2024-04-20T11:36:58.282949+0000","last_peered":"2024-04-20T11:36:58.282949+0000","last_clean":"2024-04-20T11:36:58.282949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:58.309993+0000","last_undegraded":"2024-04-20T11:36:58.309993+0000","last_fullsized":"2024-04-20T11:36:58.309993+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_clean_scrub_stamp":"2024-04-20T11:36:58.282949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:36:59.299355+0000","last_change":"2024-04-20T11:36:59.299355+0000","last_active":"2024-04-20T11:36:59.299355+0000","last_peered":"2024-04-20T11:36:59.299355+0000","last_clean":"2024-04-20T11:36:59.299355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:59.299355+0000","last_undegraded":"2024-04-20T11:36:59.299355+0000","last_fullsized":"2024-04-20T11:36:59.299355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.7","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-04-20T11:36:58.320920+0000","last_change":"2024-04-20T11:36:58.320920+0000","last_active":"2024-04-20T11:36:58.282949+0000","last_peered":"2024-04-20T11:36:58.282949+0000","last_clean":"2024-04-20T11:36:58.282949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:58.320920+0000","last_undegraded":"2024-04-20T11:36:58.320920+0000","last_fullsized":"2024-04-20T11:36:58.320920+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_clean_scrub_stamp":"2024-04-20T11:36:58.282949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:36:59.299355+0000","last_change":"2024-04-20T11:36:59.299355+0000","last_active":"2024-04-20T11:36:59.299355+0000","last_peered":"2024-04-20T11:36:59.299355+0000","last_clean":"2024-04-20T11:36:59.299355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:59.299355+0000","last_undegraded":"2024-04-20T11:36:59.299355+0000","last_fullsized":"2024-04-20T11:36:59.299355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-04-20T11:36:58.310090+0000","last_change":"2024-04-20T11:36:58.310090+0000","last_active":"2024-04-20T11:36:58.282949+0000","last_peered":"2024-04-20T11:36:58.282949+0000","last_clean":"2024-04-20T11:36:58.282949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:58.310090+0000","last_undegraded":"2024-04-20T11:36:58.310090+0000","last_fullsized":"2024-04-20T11:36:58.310090+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_clean_scrub_stamp":"2024-04-20T11:36:58.282949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:36:59.299355+0000","last_change":"2024-04-20T11:36:59.299355+0000","last_active":"2024-04-20T11:36:59.299355+0000","last_peered":"2024-04-20T11:36:59.299355+0000","last_clean":"2024-04-20T11:36:59.299355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:59.299355+0000","last_undegraded":"2024-04-20T11:36:59.299355+0000","last_fullsized":"2024-04-20T11:36:59.299355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-04-20T11:36:58.310075+0000","last_change":"2024-04-20T11:36:58.310075+0000","last_active":"2024-04-20T11:36:58.282949+0000","last_peered":"2024-04-20T11:36:58.282949+0000","last_clean":"2024-04-20T11:36:58.282949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:58.310075+0000","last_undegraded":"2024-04-20T11:36:58.310075+0000","last_fullsized":"2024-04-20T11:36:58.310075+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_clean_scrub_stamp":"2024-04-20T11:36:58.282949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:36:59.299355+0000","last_change":"2024-04-20T11:36:59.299355+0000","last_active":"2024-04-20T11:36:59.299355+0000","last_peered":"2024-04-20T11:36:59.299355+0000","last_clean":"2024-04-20T11:36:59.299355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:59.299355+0000","last_undegraded":"2024-04-20T11:36:59.299355+0000","last_fullsized":"2024-04-20T11:36:59.299355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-04-20T11:36:58.311031+0000","last_change":"2024-04-20T11:36:58.311031+0000","last_active":"2024-04-20T11:36:58.282949+0000","last_peered":"2024-04-20T11:36:58.282949+0000","last_clean":"2024-04-20T11:36:58.282949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:58.311031+0000","last_undegraded":"2024-04-20T11:36:58.311031+0000","last_fullsized":"2024-04-20T11:36:58.311031+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_clean_scrub_stamp":"2024-04-20T11:36:58.282949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:36:59.299355+0000","last_change":"2024-04-20T11:36:59.299355+0000","last_active":"2024-04-20T11:36:59.299355+0000","last_peered":"2024-04-20T11:36:59.299355+0000","last_clean":"2024-04-20T11:36:59.299355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:59.299355+0000","last_undegraded":"2024-04-20T11:36:59.299355+0000","last_fullsized":"2024-04-20T11:36:59.299355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-04-20T11:36:58.310583+0000","last_change":"2024-04-20T11:36:58.310583+0000","last_active":"2024-04-20T11:36:58.282949+0000","last_peered":"2024-04-20T11:36:58.282949+0000","last_clean":"2024-04-20T11:36:58.282949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:58.310583+0000","last_undegraded":"2024-04-20T11:36:58.310583+0000","last_fullsized":"2024-04-20T11:36:58.310583+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_clean_scrub_stamp":"2024-04-20T11:36:58.282949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:36:59.299355+0000","last_change":"2024-04-20T11:36:59.299355+0000","last_active":"2024-04-20T11:36:59.299355+0000","last_peered":"2024-04-20T11:36:59.299355+0000","last_clean":"2024-04-20T11:36:59.299355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:59.299355+0000","last_undegraded":"2024-04-20T11:36:59.299355+0000","last_fullsized":"2024-04-20T11:36:59.299355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-04-20T11:36:58.310218+0000","last_change":"2024-04-20T11:36:58.310218+0000","last_active":"2024-04-20T11:36:58.282949+0000","last_peered":"2024-04-20T11:36:58.282949+0000","last_clean":"2024-04-20T11:36:58.282949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:58.310218+0000","last_undegraded":"2024-04-20T11:36:58.310218+0000","last_fullsized":"2024-04-20T11:36:58.310218+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_clean_scrub_stamp":"2024-04-20T11:36:58.282949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:36:59.299355+0000","last_change":"2024-04-20T11:36:59.299355+0000","last_active":"2024-04-20T11:36:59.299355+0000","last_peered":"2024-04-20T11:36:59.299355+0000","last_clean":"2024-04-20T11:36:59.299355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:59.299355+0000","last_undegraded":"2024-04-20T11:36:59.299355+0000","last_fullsized":"2024-04-20T11:36:59.299355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-04-20T11:36:58.315537+0000","last_change":"2024-04-20T11:36:58.315537+0000","last_active":"2024-04-20T11:36:58.282949+0000","last_peered":"2024-04-20T11:36:58.282949+0000","last_clean":"2024-04-20T11:36:58.282949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:58.315537+0000","last_undegraded":"2024-04-20T11:36:58.315537+0000","last_fullsized":"2024-04-20T11:36:58.315537+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_clean_scrub_stamp":"2024-04-20T11:36:58.282949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:36:59.299355+0000","last_change":"2024-04-20T11:36:59.299355+0000","last_active":"2024-04-20T11:36:59.299355+0000","last_peered":"2024-04-20T11:36:59.299355+0000","last_clean":"2024-04-20T11:36:59.299355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:59.299355+0000","last_undegraded":"2024-04-20T11:36:59.299355+0000","last_fullsized":"2024-04-20T11:36:59.299355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-04-20T11:36:58.313184+0000","last_change":"2024-04-20T11:36:58.313184+0000","last_active":"2024-04-20T11:36:58.282949+0000","last_peered":"2024-04-20T11:36:58.282949+0000","last_clean":"2024-04-20T11:36:58.282949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:58.313184+0000","last_undegraded":"2024-04-20T11:36:58.313184+0000","last_fullsized":"2024-04-20T11:36:58.313184+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_clean_scrub_stamp":"2024-04-20T11:36:58.282949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T11:36:59.299355+0000","last_change":"2024-04-20T11:36:59.299355+0000","last_active":"2024-04-20T11:36:59.299355+0000","last_peered":"2024-04-20T11:36:59.299355+0000","last_clean":"2024-04-20T11:36:59.299355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:59.299355+0000","last_undegraded":"2024-04-20T11:36:59.299355+0000","last_fullsized":"2024-04-20T11:36:59.299355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-04-20T11:36:58.318587+0000","last_change":"2024-04-20T11:36:58.318587+0000","last_active":"2024-04-20T11:36:58.282949+0000","last_peered":"2024-04-20T11:36:58.282949+0000","last_clean":"2024-04-20T11:36:58.282949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T11:36:58.318587+0000","last_undegraded":"2024-04-20T11:36:58.318587+0000","last_fullsized":"2024-04-20T11:36:58.318587+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_clean_scrub_stamp":"2024-04-20T11:36:58.282949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"72'128","reported_seq":444,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-20T11:36:58.321856+0000","last_change":"2024-04-20T11:26:48.661897+0000","last_active":"2024-04-20T11:36:58.321856+0000","last_peered":"2024-04-20T11:36:58.321856+0000","last_clean":"2024-04-20T11:36:58.321856+0000","last_became_active":"2024-04-20T11:26:48.656535+0000","last_became_peered":"2024-04-20T11:26:48.656535+0000","last_unstale":"2024-04-20T11:36:58.321856+0000","last_undegraded":"2024-04-20T11:36:58.321856+0000","last_fullsized":"2024-04-20T11:36:58.321856+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:47.543004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:47.543004+0000","last_clean_scrub_stamp":"2024-04-20T11:26:47.543004+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:58:35.527105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":25,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":24,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574976,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33612,"kb_used_data":2428,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338228,"statfs":{"total":96636764160,"available":96602345472,"internally_reserved":0,"allocated":2486272,"data_stored":1423434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574973,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33044,"kb_used_data":1860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338796,"statfs":{"total":96636764160,"available":96602927104,"internally_reserved":0,"allocated":1904640,"data_stored":838883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574975,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33044,"kb_used_data":1848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338796,"statfs":{"total":96636764160,"available":96602927104,"internally_reserved":0,"allocated":1892352,"data_stored":833066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574973,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33036,"kb_used_data":1848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338804,"statfs":{"total":96636764160,"available":96602935296,"internally_reserved":0,"allocated":1892352,"data_stored":833066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574974,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33028,"kb_used_data":1848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338812,"statfs":{"total":96636764160,"available":96602943488,"internally_reserved":0,"allocated":1892352,"data_stored":833066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574975,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33028,"kb_used_data":1848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338812,"statfs":{"total":96636764160,"available":96602943488,"internally_reserved":0,"allocated":1892352,"data_stored":833066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574973,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33044,"kb_used_data":1860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338796,"statfs":{"total":96636764160,"available":96602927104,"internally_reserved":0,"allocated":1904640,"data_stored":838883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574974,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33624,"kb_used_data":2440,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338216,"statfs":{"total":96636764160,"available":96602333184,"internally_reserved":0,"allocated":2498560,"data_stored":1429251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T11:37:04.731 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T11:37:04.974 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.974+0000 7fe766a00640 1 Processor -- start 2024-04-20T11:37:04.974 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.975+0000 7fe766a00640 1 -- start start 2024-04-20T11:37:04.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.976+0000 7fe766a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe76015fcc0 0x7fe7600d4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:04.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.976+0000 7fe766a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fe7601603d0 0x7fe7600b5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:04.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.976+0000 7fe766a00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fe7600b5480 0x7fe7600b63f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:04.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.976+0000 7fe766a00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fe7600d3350 con 0x7fe7601603d0 2024-04-20T11:37:04.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.976+0000 7fe766a00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fe7600d3630 con 0x7fe7600b5480 2024-04-20T11:37:04.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.976+0000 7fe766a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe7600d3910 con 0x7fe76015fcc0 2024-04-20T11:37:04.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.976+0000 7fe764c00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fe7601603d0 0x7fe7600b5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:04.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.977+0000 7fe764c00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fe7601603d0 0x7fe7600b5eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:57242/0 (socket says 172.21.15.152:57242) 2024-04-20T11:37:04.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.977+0000 7fe764c00640 1 -- 172.21.15.152:0/3144898688 learned_addr learned my addr 172.21.15.152:0/3144898688 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:04.977 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.977+0000 7fe765600640 1 --2- 172.21.15.152:0/3144898688 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe76015fcc0 0x7fe7600d4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:04.977 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.977+0000 7fe764c00640 1 -- 172.21.15.152:0/3144898688 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe76015fcc0 msgr2=0x7fe7600d4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:04.977 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.977+0000 7fe766000640 1 --2- 172.21.15.152:0/3144898688 >> v2:172.21.15.175:3300/0 conn(0x7fe7600b5480 0x7fe7600b63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:04.977 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.977+0000 7fe764c00640 1 --2- 172.21.15.152:0/3144898688 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe76015fcc0 0x7fe7600d4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:04.978 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.977+0000 7fe764c00640 1 -- 172.21.15.152:0/3144898688 >> v2:172.21.15.175:3300/0 conn(0x7fe7600b5480 msgr2=0x7fe7600b63f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:04.978 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.977+0000 7fe764c00640 1 --2- 172.21.15.152:0/3144898688 >> v2:172.21.15.175:3300/0 conn(0x7fe7600b5480 0x7fe7600b63f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:04.978 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.977+0000 7fe764c00640 1 -- 172.21.15.152:0/3144898688 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7600ced20 con 0x7fe7601603d0 2024-04-20T11:37:04.978 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.977+0000 7fe765600640 1 --2- 172.21.15.152:0/3144898688 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe76015fcc0 0x7fe7600d4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:37:04.978 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.978+0000 7fe764c00640 1 --2- 172.21.15.152:0/3144898688 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe7601603d0 0x7fe7600b5eb0 secure :-1 s=READY pgs=2153 cs=0 l=1 rev1=1 crypto rx=0x7fe7600c8020 tx=0x7fe75002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c16c2205beadfcfb server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:04.979 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.978+0000 7fe75fe00640 1 -- 172.21.15.152:0/3144898688 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe75002d040 con 0x7fe7601603d0 2024-04-20T11:37:04.979 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.978+0000 7fe75fe00640 1 -- 172.21.15.152:0/3144898688 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe750004230 con 0x7fe7601603d0 2024-04-20T11:37:04.979 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.979+0000 7fe75fe00640 1 -- 172.21.15.152:0/3144898688 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe750003610 con 0x7fe7601603d0 2024-04-20T11:37:04.979 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.979+0000 7fe766a00640 1 -- 172.21.15.152:0/3144898688 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe7601603d0 msgr2=0x7fe7600b5eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:04.979 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.979+0000 7fe766a00640 1 --2- 172.21.15.152:0/3144898688 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe7601603d0 0x7fe7600b5eb0 secure :-1 s=READY pgs=2153 cs=0 l=1 rev1=1 crypto rx=0x7fe7600c8020 tx=0x7fe75002b8f0 comp rx=0 tx=0).stop 2024-04-20T11:37:04.979 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.979+0000 7fe766a00640 1 -- 172.21.15.152:0/3144898688 shutdown_connections 2024-04-20T11:37:04.979 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.979+0000 7fe766a00640 1 --2- 172.21.15.152:0/3144898688 >> v2:172.21.15.175:3300/0 conn(0x7fe7600b5480 0x7fe7600b63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:04.979 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.979+0000 7fe766a00640 1 --2- 172.21.15.152:0/3144898688 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe7601603d0 0x7fe7600b5eb0 unknown :-1 s=CLOSED pgs=2153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:04.979 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.979+0000 7fe766a00640 1 --2- 172.21.15.152:0/3144898688 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe76015fcc0 0x7fe7600d4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:04.980 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.979+0000 7fe766a00640 1 -- 172.21.15.152:0/3144898688 >> 172.21.15.152:0/3144898688 conn(0x7fe76008bfd0 msgr2=0x7fe7600d4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:04.980 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.979+0000 7fe766a00640 1 -- 172.21.15.152:0/3144898688 shutdown_connections 2024-04-20T11:37:04.980 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.979+0000 7fe766a00640 1 -- 172.21.15.152:0/3144898688 wait complete. 2024-04-20T11:37:04.980 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.981+0000 7fe766a00640 1 Processor -- start 2024-04-20T11:37:04.980 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.981+0000 7fe766a00640 1 -- start start 2024-04-20T11:37:04.982 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.982+0000 7fe766a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe7600b5480 0x7fe760159060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:04.982 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.982+0000 7fe766a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe76015fcc0 0x7fe76015d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:04.982 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.982+0000 7fe766a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7601603d0 0x7fe76015da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:04.982 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.982+0000 7fe766a00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fe760162650 con 0x7fe7600b5480 2024-04-20T11:37:04.982 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.982+0000 7fe766a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe7601624d0 con 0x7fe76015fcc0 2024-04-20T11:37:04.982 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.982+0000 7fe766a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe7601627d0 con 0x7fe7601603d0 2024-04-20T11:37:04.982 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.982+0000 7fe765600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe7600b5480 0x7fe760159060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:04.982 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.982+0000 7fe765600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe7600b5480 0x7fe760159060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:57246/0 (socket says 172.21.15.152:57246) 2024-04-20T11:37:04.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.982+0000 7fe765600640 1 -- 172.21.15.152:0/702854492 learned_addr learned my addr 172.21.15.152:0/702854492 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:04.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.983+0000 7fe764c00640 1 --2- 172.21.15.152:0/702854492 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe76015fcc0 0x7fe76015d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:04.984 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.983+0000 7fe765600640 1 -- 172.21.15.152:0/702854492 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7601603d0 msgr2=0x7fe76015da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:04.984 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.983+0000 7fe765600640 1 --2- 172.21.15.152:0/702854492 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7601603d0 0x7fe76015da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:04.984 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.983+0000 7fe765600640 1 -- 172.21.15.152:0/702854492 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe76015fcc0 msgr2=0x7fe76015d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:04.984 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.983+0000 7fe765600640 1 --2- 172.21.15.152:0/702854492 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe76015fcc0 0x7fe76015d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:04.984 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.983+0000 7fe765600640 1 -- 172.21.15.152:0/702854492 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7600b1160 con 0x7fe7600b5480 2024-04-20T11:37:04.984 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.983+0000 7fe765600640 1 --2- 172.21.15.152:0/702854492 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe7600b5480 0x7fe760159060 secure :-1 s=READY pgs=2154 cs=0 l=1 rev1=1 crypto rx=0x7fe75400c7f0 tx=0x7fe75400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:04.984 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.984+0000 7fe75d600640 1 -- 172.21.15.152:0/702854492 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe754015070 con 0x7fe7600b5480 2024-04-20T11:37:04.984 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.984+0000 7fe75d600640 1 -- 172.21.15.152:0/702854492 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe754007a30 con 0x7fe7600b5480 2024-04-20T11:37:04.984 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.984+0000 7fe75d600640 1 -- 172.21.15.152:0/702854492 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe754007e00 con 0x7fe7600b5480 2024-04-20T11:37:04.984 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.984+0000 7fe766a00640 1 -- 172.21.15.152:0/702854492 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7600cc200 con 0x7fe7600b5480 2024-04-20T11:37:04.984 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.984+0000 7fe766a00640 1 -- 172.21.15.152:0/702854492 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7600b1400 con 0x7fe7600b5480 2024-04-20T11:37:04.985 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.985+0000 7fe766a00640 1 -- 172.21.15.152:0/702854492 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe760075e60 con 0x7fe7600b5480 2024-04-20T11:37:04.985 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.986+0000 7fe75d600640 1 -- 172.21.15.152:0/702854492 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) ==== 91754+0+0 (secure 0 0 0) 0x7fe754018070 con 0x7fe7600b5480 2024-04-20T11:37:04.986 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.987+0000 7fe75d600640 1 --2- 172.21.15.152:0/702854492 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe74406d100 0x7fe74406f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:04.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.987+0000 7fe75d600640 1 -- 172.21.15.152:0/702854492 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(154..154 src has 1..154) ==== 5709+0+0 (secure 0 0 0) 0x7fe75408ce60 con 0x7fe7600b5480 2024-04-20T11:37:04.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.988+0000 7fe764c00640 1 --2- 172.21.15.152:0/702854492 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe74406d100 0x7fe74406f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:04.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.989+0000 7fe764c00640 1 --2- 172.21.15.152:0/702854492 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe74406d100 0x7fe74406f5c0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fe750035000 tx=0x7fe750033040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:04.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:04.991+0000 7fe75d600640 1 -- 172.21.15.152:0/702854492 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe75405a1f0 con 0x7fe7600b5480 2024-04-20T11:37:05.187 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:05.188+0000 7fe766a00640 1 -- 172.21.15.152:0/702854492 --> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe7600b6ce0 con 0x7fe74406d100 2024-04-20T11:37:05.206 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:05.206+0000 7fe75d600640 1 -- 172.21.15.152:0/702854492 <== mgr.4109 v2:172.21.15.175:6836/4021250441 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252485 (secure 0 0 0) 0x7fe7600b6ce0 con 0x7fe74406d100 2024-04-20T11:37:05.209 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:37:05.210 INFO:teuthology.orchestra.run.smithi152.stderr:dumped all 2024-04-20T11:37:05.212 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:05.211+0000 7fe766a00640 1 -- 172.21.15.152:0/702854492 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe74406d100 msgr2=0x7fe74406f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:05.212 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:05.211+0000 7fe766a00640 1 --2- 172.21.15.152:0/702854492 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe74406d100 0x7fe74406f5c0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fe750035000 tx=0x7fe750033040 comp rx=0 tx=0).stop 2024-04-20T11:37:05.212 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:05.211+0000 7fe766a00640 1 -- 172.21.15.152:0/702854492 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe7600b5480 msgr2=0x7fe760159060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:05.213 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:05.211+0000 7fe766a00640 1 --2- 172.21.15.152:0/702854492 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe7600b5480 0x7fe760159060 secure :-1 s=READY pgs=2154 cs=0 l=1 rev1=1 crypto rx=0x7fe75400c7f0 tx=0x7fe75400cbf0 comp rx=0 tx=0).stop 2024-04-20T11:37:05.213 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:05.212+0000 7fe766a00640 1 -- 172.21.15.152:0/702854492 shutdown_connections 2024-04-20T11:37:05.213 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:05.212+0000 7fe766a00640 1 --2- 172.21.15.152:0/702854492 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe74406d100 0x7fe74406f5c0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:05.213 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:05.212+0000 7fe766a00640 1 --2- 172.21.15.152:0/702854492 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7601603d0 0x7fe76015da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:05.213 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:05.212+0000 7fe766a00640 1 --2- 172.21.15.152:0/702854492 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe76015fcc0 0x7fe76015d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:05.213 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:05.212+0000 7fe766a00640 1 --2- 172.21.15.152:0/702854492 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe7600b5480 0x7fe760159060 unknown :-1 s=CLOSED pgs=2154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:05.213 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:05.212+0000 7fe766a00640 1 -- 172.21.15.152:0/702854492 >> 172.21.15.152:0/702854492 conn(0x7fe76008bfd0 msgr2=0x7fe7600caef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:05.213 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:05.212+0000 7fe766a00640 1 -- 172.21.15.152:0/702854492 shutdown_connections 2024-04-20T11:37:05.213 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:05.212+0000 7fe766a00640 1 -- 172.21.15.152:0/702854492 wait complete. 2024-04-20T11:37:05.231 INFO:teuthology.orchestra.run.smithi152.stdout:{"pg_ready":true,"pg_map":{"version":227,"stamp":"2024-04-20T11:37:04.701404+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":224,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":167,"ondisk_log_size":167,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":189,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":265860,"kb_used_data":16392,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754708860,"statfs":{"total":773094113280,"available":772821872640,"internally_reserved":0,"allocated":16785408,"data_stored":8051420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12746,"internal_metadata":255315510},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.003591"},"pg_stats":[{"pgid":"25.3f","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-20T11:37:01.815845+0000","last_change":"2024-04-20T11:37:00.354545+0000","last_active":"2024-04-20T11:37:01.815845+0000","last_peered":"2024-04-20T11:37:01.815845+0000","last_clean":"2024-04-20T11:37:01.815845+0000","last_became_active":"2024-04-20T11:37:00.349388+0000","last_became_peered":"2024-04-20T11:37:00.349388+0000","last_unstale":"2024-04-20T11:37:01.815845+0000","last_undegraded":"2024-04-20T11:37:01.815845+0000","last_fullsized":"2024-04-20T11:37:01.815845+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:36:36.288679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.3e","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-20T11:37:01.814101+0000","last_change":"2024-04-20T11:37:00.351199+0000","last_active":"2024-04-20T11:37:01.814101+0000","last_peered":"2024-04-20T11:37:01.814101+0000","last_clean":"2024-04-20T11:37:01.814101+0000","last_became_active":"2024-04-20T11:37:00.350892+0000","last_became_peered":"2024-04-20T11:37:00.350892+0000","last_unstale":"2024-04-20T11:37:01.814101+0000","last_undegraded":"2024-04-20T11:37:01.814101+0000","last_fullsized":"2024-04-20T11:37:01.814101+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:29:03.042475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3d","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-20T11:37:01.340965+0000","last_change":"2024-04-20T11:37:00.350461+0000","last_active":"2024-04-20T11:37:01.340965+0000","last_peered":"2024-04-20T11:37:01.340965+0000","last_clean":"2024-04-20T11:37:01.340965+0000","last_became_active":"2024-04-20T11:37:00.350142+0000","last_became_peered":"2024-04-20T11:37:00.350142+0000","last_unstale":"2024-04-20T11:37:01.340965+0000","last_undegraded":"2024-04-20T11:37:01.340965+0000","last_fullsized":"2024-04-20T11:37:01.340965+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:31:48.289723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3c","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-20T11:37:01.813710+0000","last_change":"2024-04-20T11:37:00.348056+0000","last_active":"2024-04-20T11:37:01.813710+0000","last_peered":"2024-04-20T11:37:01.813710+0000","last_clean":"2024-04-20T11:37:01.813710+0000","last_became_active":"2024-04-20T11:37:00.347782+0000","last_became_peered":"2024-04-20T11:37:00.347782+0000","last_unstale":"2024-04-20T11:37:01.813710+0000","last_undegraded":"2024-04-20T11:37:01.813710+0000","last_fullsized":"2024-04-20T11:37:01.813710+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:42:00.785692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3b","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-20T11:37:01.342941+0000","last_change":"2024-04-20T11:37:00.350923+0000","last_active":"2024-04-20T11:37:01.342941+0000","last_peered":"2024-04-20T11:37:01.342941+0000","last_clean":"2024-04-20T11:37:01.342941+0000","last_became_active":"2024-04-20T11:37:00.350576+0000","last_became_peered":"2024-04-20T11:37:00.350576+0000","last_unstale":"2024-04-20T11:37:01.342941+0000","last_undegraded":"2024-04-20T11:37:01.342941+0000","last_fullsized":"2024-04-20T11:37:01.342941+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:56:18.846599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.3a","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-20T11:37:01.340091+0000","last_change":"2024-04-20T11:37:00.353424+0000","last_active":"2024-04-20T11:37:01.340091+0000","last_peered":"2024-04-20T11:37:01.340091+0000","last_clean":"2024-04-20T11:37:01.340091+0000","last_became_active":"2024-04-20T11:37:00.353059+0000","last_became_peered":"2024-04-20T11:37:00.353059+0000","last_unstale":"2024-04-20T11:37:01.340091+0000","last_undegraded":"2024-04-20T11:37:01.340091+0000","last_fullsized":"2024-04-20T11:37:01.340091+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:16:42.328148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.39","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-20T11:37:01.812207+0000","last_change":"2024-04-20T11:37:00.344020+0000","last_active":"2024-04-20T11:37:01.812207+0000","last_peered":"2024-04-20T11:37:01.812207+0000","last_clean":"2024-04-20T11:37:01.812207+0000","last_became_active":"2024-04-20T11:37:00.343709+0000","last_became_peered":"2024-04-20T11:37:00.343709+0000","last_unstale":"2024-04-20T11:37:01.812207+0000","last_undegraded":"2024-04-20T11:37:01.812207+0000","last_fullsized":"2024-04-20T11:37:01.812207+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:44:30.731560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.38","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-20T11:37:01.341424+0000","last_change":"2024-04-20T11:37:00.346019+0000","last_active":"2024-04-20T11:37:01.341424+0000","last_peered":"2024-04-20T11:37:01.341424+0000","last_clean":"2024-04-20T11:37:01.341424+0000","last_became_active":"2024-04-20T11:37:00.345740+0000","last_became_peered":"2024-04-20T11:37:00.345740+0000","last_unstale":"2024-04-20T11:37:01.341424+0000","last_undegraded":"2024-04-20T11:37:01.341424+0000","last_fullsized":"2024-04-20T11:37:01.341424+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:45:52.140285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.37","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-20T11:37:01.345886+0000","last_change":"2024-04-20T11:37:00.355024+0000","last_active":"2024-04-20T11:37:01.345886+0000","last_peered":"2024-04-20T11:37:01.345886+0000","last_clean":"2024-04-20T11:37:01.345886+0000","last_became_active":"2024-04-20T11:37:00.353918+0000","last_became_peered":"2024-04-20T11:37:00.353918+0000","last_unstale":"2024-04-20T11:37:01.345886+0000","last_undegraded":"2024-04-20T11:37:01.345886+0000","last_fullsized":"2024-04-20T11:37:01.345886+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:55:11.794637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.36","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-20T11:37:01.363107+0000","last_change":"2024-04-20T11:37:00.336959+0000","last_active":"2024-04-20T11:37:01.363107+0000","last_peered":"2024-04-20T11:37:01.363107+0000","last_clean":"2024-04-20T11:37:01.363107+0000","last_became_active":"2024-04-20T11:37:00.336488+0000","last_became_peered":"2024-04-20T11:37:00.336488+0000","last_unstale":"2024-04-20T11:37:01.363107+0000","last_undegraded":"2024-04-20T11:37:01.363107+0000","last_fullsized":"2024-04-20T11:37:01.363107+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:32:45.451319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.35","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-20T11:37:01.814630+0000","last_change":"2024-04-20T11:37:00.341794+0000","last_active":"2024-04-20T11:37:01.814630+0000","last_peered":"2024-04-20T11:37:01.814630+0000","last_clean":"2024-04-20T11:37:01.814630+0000","last_became_active":"2024-04-20T11:37:00.341355+0000","last_became_peered":"2024-04-20T11:37:00.341355+0000","last_unstale":"2024-04-20T11:37:01.814630+0000","last_undegraded":"2024-04-20T11:37:01.814630+0000","last_fullsized":"2024-04-20T11:37:01.814630+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:06:49.478364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.34","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-20T11:37:01.813949+0000","last_change":"2024-04-20T11:37:00.353940+0000","last_active":"2024-04-20T11:37:01.813949+0000","last_peered":"2024-04-20T11:37:01.813949+0000","last_clean":"2024-04-20T11:37:01.813949+0000","last_became_active":"2024-04-20T11:37:00.353492+0000","last_became_peered":"2024-04-20T11:37:00.353492+0000","last_unstale":"2024-04-20T11:37:01.813949+0000","last_undegraded":"2024-04-20T11:37:01.813949+0000","last_fullsized":"2024-04-20T11:37:01.813949+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:30:33.225757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.33","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-20T11:37:01.811884+0000","last_change":"2024-04-20T11:37:00.345463+0000","last_active":"2024-04-20T11:37:01.811884+0000","last_peered":"2024-04-20T11:37:01.811884+0000","last_clean":"2024-04-20T11:37:01.811884+0000","last_became_active":"2024-04-20T11:37:00.345152+0000","last_became_peered":"2024-04-20T11:37:00.345152+0000","last_unstale":"2024-04-20T11:37:01.811884+0000","last_undegraded":"2024-04-20T11:37:01.811884+0000","last_fullsized":"2024-04-20T11:37:01.811884+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:45:44.213487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.32","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-20T11:37:01.345053+0000","last_change":"2024-04-20T11:37:00.350091+0000","last_active":"2024-04-20T11:37:01.345053+0000","last_peered":"2024-04-20T11:37:01.345053+0000","last_clean":"2024-04-20T11:37:01.345053+0000","last_became_active":"2024-04-20T11:37:00.346654+0000","last_became_peered":"2024-04-20T11:37:00.346654+0000","last_unstale":"2024-04-20T11:37:01.345053+0000","last_undegraded":"2024-04-20T11:37:01.345053+0000","last_fullsized":"2024-04-20T11:37:01.345053+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:52:14.372985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.31","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-20T11:37:01.812822+0000","last_change":"2024-04-20T11:37:00.346282+0000","last_active":"2024-04-20T11:37:01.812822+0000","last_peered":"2024-04-20T11:37:01.812822+0000","last_clean":"2024-04-20T11:37:01.812822+0000","last_became_active":"2024-04-20T11:37:00.345949+0000","last_became_peered":"2024-04-20T11:37:00.345949+0000","last_unstale":"2024-04-20T11:37:01.812822+0000","last_undegraded":"2024-04-20T11:37:01.812822+0000","last_fullsized":"2024-04-20T11:37:01.812822+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:38:33.113005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.30","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-20T11:37:01.341576+0000","last_change":"2024-04-20T11:37:00.349771+0000","last_active":"2024-04-20T11:37:01.341576+0000","last_peered":"2024-04-20T11:37:01.341576+0000","last_clean":"2024-04-20T11:37:01.341576+0000","last_became_active":"2024-04-20T11:37:00.349484+0000","last_became_peered":"2024-04-20T11:37:00.349484+0000","last_unstale":"2024-04-20T11:37:01.341576+0000","last_undegraded":"2024-04-20T11:37:01.341576+0000","last_fullsized":"2024-04-20T11:37:01.341576+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:24:05.653365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2f","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-20T11:37:01.343831+0000","last_change":"2024-04-20T11:37:00.352253+0000","last_active":"2024-04-20T11:37:01.343831+0000","last_peered":"2024-04-20T11:37:01.343831+0000","last_clean":"2024-04-20T11:37:01.343831+0000","last_became_active":"2024-04-20T11:37:00.351790+0000","last_became_peered":"2024-04-20T11:37:00.351790+0000","last_unstale":"2024-04-20T11:37:01.343831+0000","last_undegraded":"2024-04-20T11:37:01.343831+0000","last_fullsized":"2024-04-20T11:37:01.343831+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:55:11.794637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2e","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-20T11:37:01.815627+0000","last_change":"2024-04-20T11:37:00.350069+0000","last_active":"2024-04-20T11:37:01.815627+0000","last_peered":"2024-04-20T11:37:01.815627+0000","last_clean":"2024-04-20T11:37:01.815627+0000","last_became_active":"2024-04-20T11:37:00.349753+0000","last_became_peered":"2024-04-20T11:37:00.349753+0000","last_unstale":"2024-04-20T11:37:01.815627+0000","last_undegraded":"2024-04-20T11:37:01.815627+0000","last_fullsized":"2024-04-20T11:37:01.815627+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:22:31.839715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2d","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-20T11:37:01.363132+0000","last_change":"2024-04-20T11:37:00.357825+0000","last_active":"2024-04-20T11:37:01.363132+0000","last_peered":"2024-04-20T11:37:01.363132+0000","last_clean":"2024-04-20T11:37:01.363132+0000","last_became_active":"2024-04-20T11:37:00.357509+0000","last_became_peered":"2024-04-20T11:37:00.357509+0000","last_unstale":"2024-04-20T11:37:01.363132+0000","last_undegraded":"2024-04-20T11:37:01.363132+0000","last_fullsized":"2024-04-20T11:37:01.363132+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:29:39.732245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.2c","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-20T11:37:01.343347+0000","last_change":"2024-04-20T11:37:00.352438+0000","last_active":"2024-04-20T11:37:01.343347+0000","last_peered":"2024-04-20T11:37:01.343347+0000","last_clean":"2024-04-20T11:37:01.343347+0000","last_became_active":"2024-04-20T11:37:00.351511+0000","last_became_peered":"2024-04-20T11:37:00.351511+0000","last_unstale":"2024-04-20T11:37:01.343347+0000","last_undegraded":"2024-04-20T11:37:01.343347+0000","last_fullsized":"2024-04-20T11:37:01.343347+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:54:19.949616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2b","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-20T11:37:01.343082+0000","last_change":"2024-04-20T11:37:00.349531+0000","last_active":"2024-04-20T11:37:01.343082+0000","last_peered":"2024-04-20T11:37:01.343082+0000","last_clean":"2024-04-20T11:37:01.343082+0000","last_became_active":"2024-04-20T11:37:00.349224+0000","last_became_peered":"2024-04-20T11:37:00.349224+0000","last_unstale":"2024-04-20T11:37:01.343082+0000","last_undegraded":"2024-04-20T11:37:01.343082+0000","last_fullsized":"2024-04-20T11:37:01.343082+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:52:14.372985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2a","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-20T11:37:01.362738+0000","last_change":"2024-04-20T11:37:00.347362+0000","last_active":"2024-04-20T11:37:01.362738+0000","last_peered":"2024-04-20T11:37:01.362738+0000","last_clean":"2024-04-20T11:37:01.362738+0000","last_became_active":"2024-04-20T11:37:00.347036+0000","last_became_peered":"2024-04-20T11:37:00.347036+0000","last_unstale":"2024-04-20T11:37:01.362738+0000","last_undegraded":"2024-04-20T11:37:01.362738+0000","last_fullsized":"2024-04-20T11:37:01.362738+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:12:17.343090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.29","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-20T11:37:01.339816+0000","last_change":"2024-04-20T11:37:00.344279+0000","last_active":"2024-04-20T11:37:01.339816+0000","last_peered":"2024-04-20T11:37:01.339816+0000","last_clean":"2024-04-20T11:37:01.339816+0000","last_became_active":"2024-04-20T11:37:00.343991+0000","last_became_peered":"2024-04-20T11:37:00.343991+0000","last_unstale":"2024-04-20T11:37:01.339816+0000","last_undegraded":"2024-04-20T11:37:01.339816+0000","last_fullsized":"2024-04-20T11:37:01.339816+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:38:33.113005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.28","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-20T11:37:01.341749+0000","last_change":"2024-04-20T11:37:00.340227+0000","last_active":"2024-04-20T11:37:01.341749+0000","last_peered":"2024-04-20T11:37:01.341749+0000","last_clean":"2024-04-20T11:37:01.341749+0000","last_became_active":"2024-04-20T11:37:00.339960+0000","last_became_peered":"2024-04-20T11:37:00.339960+0000","last_unstale":"2024-04-20T11:37:01.341749+0000","last_undegraded":"2024-04-20T11:37:01.341749+0000","last_fullsized":"2024-04-20T11:37:01.341749+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:16:19.317472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.27","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-20T11:37:01.813619+0000","last_change":"2024-04-20T11:37:00.351663+0000","last_active":"2024-04-20T11:37:01.813619+0000","last_peered":"2024-04-20T11:37:01.813619+0000","last_clean":"2024-04-20T11:37:01.813619+0000","last_became_active":"2024-04-20T11:37:00.347191+0000","last_became_peered":"2024-04-20T11:37:00.347191+0000","last_unstale":"2024-04-20T11:37:01.813619+0000","last_undegraded":"2024-04-20T11:37:01.813619+0000","last_fullsized":"2024-04-20T11:37:01.813619+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:16:42.328148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.26","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-20T11:37:01.814726+0000","last_change":"2024-04-20T11:37:00.345844+0000","last_active":"2024-04-20T11:37:01.814726+0000","last_peered":"2024-04-20T11:37:01.814726+0000","last_clean":"2024-04-20T11:37:01.814726+0000","last_became_active":"2024-04-20T11:37:00.345063+0000","last_became_peered":"2024-04-20T11:37:00.345063+0000","last_unstale":"2024-04-20T11:37:01.814726+0000","last_undegraded":"2024-04-20T11:37:01.814726+0000","last_fullsized":"2024-04-20T11:37:01.814726+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:11:35.697161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.25","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-20T11:37:01.341190+0000","last_change":"2024-04-20T11:37:00.348960+0000","last_active":"2024-04-20T11:37:01.341190+0000","last_peered":"2024-04-20T11:37:01.341190+0000","last_clean":"2024-04-20T11:37:01.341190+0000","last_became_active":"2024-04-20T11:37:00.348668+0000","last_became_peered":"2024-04-20T11:37:00.348668+0000","last_unstale":"2024-04-20T11:37:01.341190+0000","last_undegraded":"2024-04-20T11:37:01.341190+0000","last_fullsized":"2024-04-20T11:37:01.341190+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:29:03.042475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.24","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-20T11:37:01.343414+0000","last_change":"2024-04-20T11:37:00.341627+0000","last_active":"2024-04-20T11:37:01.343414+0000","last_peered":"2024-04-20T11:37:01.343414+0000","last_clean":"2024-04-20T11:37:01.343414+0000","last_became_active":"2024-04-20T11:37:00.341303+0000","last_became_peered":"2024-04-20T11:37:00.341303+0000","last_unstale":"2024-04-20T11:37:01.343414+0000","last_undegraded":"2024-04-20T11:37:01.343414+0000","last_fullsized":"2024-04-20T11:37:01.343414+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:04:00.800247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.23","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-20T11:37:01.812329+0000","last_change":"2024-04-20T11:37:00.358714+0000","last_active":"2024-04-20T11:37:01.812329+0000","last_peered":"2024-04-20T11:37:01.812329+0000","last_clean":"2024-04-20T11:37:01.812329+0000","last_became_active":"2024-04-20T11:37:00.358357+0000","last_became_peered":"2024-04-20T11:37:00.358357+0000","last_unstale":"2024-04-20T11:37:01.812329+0000","last_undegraded":"2024-04-20T11:37:01.812329+0000","last_fullsized":"2024-04-20T11:37:01.812329+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:33:44.635076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.22","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-20T11:37:01.345123+0000","last_change":"2024-04-20T11:37:00.358046+0000","last_active":"2024-04-20T11:37:01.345123+0000","last_peered":"2024-04-20T11:37:01.345123+0000","last_clean":"2024-04-20T11:37:01.345123+0000","last_became_active":"2024-04-20T11:37:00.356394+0000","last_became_peered":"2024-04-20T11:37:00.356394+0000","last_unstale":"2024-04-20T11:37:01.345123+0000","last_undegraded":"2024-04-20T11:37:01.345123+0000","last_fullsized":"2024-04-20T11:37:01.345123+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:54:19.949616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.21","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-20T11:37:01.340415+0000","last_change":"2024-04-20T11:37:00.345001+0000","last_active":"2024-04-20T11:37:01.340415+0000","last_peered":"2024-04-20T11:37:01.340415+0000","last_clean":"2024-04-20T11:37:01.340415+0000","last_became_active":"2024-04-20T11:37:00.344682+0000","last_became_peered":"2024-04-20T11:37:00.344682+0000","last_unstale":"2024-04-20T11:37:01.340415+0000","last_undegraded":"2024-04-20T11:37:01.340415+0000","last_fullsized":"2024-04-20T11:37:01.340415+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:30:33.225757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.20","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-20T11:37:01.341748+0000","last_change":"2024-04-20T11:37:00.348188+0000","last_active":"2024-04-20T11:37:01.341748+0000","last_peered":"2024-04-20T11:37:01.341748+0000","last_clean":"2024-04-20T11:37:01.341748+0000","last_became_active":"2024-04-20T11:37:00.347908+0000","last_became_peered":"2024-04-20T11:37:00.347908+0000","last_unstale":"2024-04-20T11:37:01.341748+0000","last_undegraded":"2024-04-20T11:37:01.341748+0000","last_fullsized":"2024-04-20T11:37:01.341748+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:24:01.556138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-20T11:37:01.342380+0000","last_change":"2024-04-20T11:37:00.353401+0000","last_active":"2024-04-20T11:37:01.342380+0000","last_peered":"2024-04-20T11:37:01.342380+0000","last_clean":"2024-04-20T11:37:01.342380+0000","last_became_active":"2024-04-20T11:37:00.353045+0000","last_became_peered":"2024-04-20T11:37:00.353045+0000","last_unstale":"2024-04-20T11:37:01.342380+0000","last_undegraded":"2024-04-20T11:37:01.342380+0000","last_fullsized":"2024-04-20T11:37:01.342380+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:11:08.307839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1f","version":"153'4","reported_seq":18,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-20T11:37:01.343822+0000","last_change":"2024-04-20T11:36:59.347129+0000","last_active":"2024-04-20T11:37:01.343822+0000","last_peered":"2024-04-20T11:37:01.343822+0000","last_clean":"2024-04-20T11:37:01.343822+0000","last_became_active":"2024-04-20T11:36:59.346843+0000","last_became_peered":"2024-04-20T11:36:59.346843+0000","last_unstale":"2024-04-20T11:37:01.343822+0000","last_undegraded":"2024-04-20T11:37:01.343822+0000","last_fullsized":"2024-04-20T11:37:01.343822+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_clean_scrub_stamp":"2024-04-20T11:36:58.282949+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:04:07.288819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-20T11:37:01.363655+0000","last_change":"2024-04-20T11:37:00.336729+0000","last_active":"2024-04-20T11:37:01.363655+0000","last_peered":"2024-04-20T11:37:01.363655+0000","last_clean":"2024-04-20T11:37:01.363655+0000","last_became_active":"2024-04-20T11:37:00.336416+0000","last_became_peered":"2024-04-20T11:37:00.336416+0000","last_unstale":"2024-04-20T11:37:01.363655+0000","last_undegraded":"2024-04-20T11:37:01.363655+0000","last_fullsized":"2024-04-20T11:37:01.363655+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:04:16.130051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1e","version":"153'1","reported_seq":15,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-20T11:37:01.345687+0000","last_change":"2024-04-20T11:36:59.352160+0000","last_active":"2024-04-20T11:37:01.345687+0000","last_peered":"2024-04-20T11:37:01.345687+0000","last_clean":"2024-04-20T11:37:01.345687+0000","last_became_active":"2024-04-20T11:36:59.349645+0000","last_became_peered":"2024-04-20T11:36:59.349645+0000","last_unstale":"2024-04-20T11:37:01.345687+0000","last_undegraded":"2024-04-20T11:37:01.345687+0000","last_fullsized":"2024-04-20T11:37:01.345687+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_clean_scrub_stamp":"2024-04-20T11:36:58.282949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:07:54.255931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-20T11:37:01.812040+0000","last_change":"2024-04-20T11:37:00.345312+0000","last_active":"2024-04-20T11:37:01.812040+0000","last_peered":"2024-04-20T11:37:01.812040+0000","last_clean":"2024-04-20T11:37:01.812040+0000","last_became_active":"2024-04-20T11:37:00.344969+0000","last_became_peered":"2024-04-20T11:37:00.344969+0000","last_unstale":"2024-04-20T11:37:01.812040+0000","last_undegraded":"2024-04-20T11:37:01.812040+0000","last_fullsized":"2024-04-20T11:37:01.812040+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:48:22.589718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1d","version":"153'2","reported_seq":16,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-20T11:37:01.341018+0000","last_change":"2024-04-20T11:36:59.348811+0000","last_active":"2024-04-20T11:37:01.341018+0000","last_peered":"2024-04-20T11:37:01.341018+0000","last_clean":"2024-04-20T11:37:01.341018+0000","last_became_active":"2024-04-20T11:36:59.348310+0000","last_became_peered":"2024-04-20T11:36:59.348310+0000","last_unstale":"2024-04-20T11:37:01.341018+0000","last_undegraded":"2024-04-20T11:37:01.341018+0000","last_fullsized":"2024-04-20T11:37:01.341018+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_clean_scrub_stamp":"2024-04-20T11:36:58.282949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:09:27.548291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-20T11:37:01.345776+0000","last_change":"2024-04-20T11:37:00.349300+0000","last_active":"2024-04-20T11:37:01.345776+0000","last_peered":"2024-04-20T11:37:01.345776+0000","last_clean":"2024-04-20T11:37:01.345776+0000","last_became_active":"2024-04-20T11:37:00.347002+0000","last_became_peered":"2024-04-20T11:37:00.347002+0000","last_unstale":"2024-04-20T11:37:01.345776+0000","last_undegraded":"2024-04-20T11:37:01.345776+0000","last_fullsized":"2024-04-20T11:37:01.345776+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:41:40.228890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1c","version":"153'2","reported_seq":16,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-20T11:37:01.814030+0000","last_change":"2024-04-20T11:36:59.354551+0000","last_active":"2024-04-20T11:37:01.814030+0000","last_peered":"2024-04-20T11:37:01.814030+0000","last_clean":"2024-04-20T11:37:01.814030+0000","last_became_active":"2024-04-20T11:36:59.353866+0000","last_became_peered":"2024-04-20T11:36:59.353866+0000","last_unstale":"2024-04-20T11:37:01.814030+0000","last_undegraded":"2024-04-20T11:37:01.814030+0000","last_fullsized":"2024-04-20T11:37:01.814030+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_clean_scrub_stamp":"2024-04-20T11:36:58.282949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:38:29.698560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-20T11:37:01.363490+0000","last_change":"2024-04-20T11:37:00.341365+0000","last_active":"2024-04-20T11:37:01.363490+0000","last_peered":"2024-04-20T11:37:01.363490+0000","last_clean":"2024-04-20T11:37:01.363490+0000","last_became_active":"2024-04-20T11:37:00.341103+0000","last_became_peered":"2024-04-20T11:37:00.341103+0000","last_unstale":"2024-04-20T11:37:01.363490+0000","last_undegraded":"2024-04-20T11:37:01.363490+0000","last_fullsized":"2024-04-20T11:37:01.363490+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:02:35.630307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1b","version":"153'1","reported_seq":15,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-20T11:37:01.343197+0000","last_change":"2024-04-20T11:36:59.343792+0000","last_active":"2024-04-20T11:37:01.343197+0000","last_peered":"2024-04-20T11:37:01.343197+0000","last_clean":"2024-04-20T11:37:01.343197+0000","last_became_active":"2024-04-20T11:36:59.343477+0000","last_became_peered":"2024-04-20T11:36:59.343477+0000","last_unstale":"2024-04-20T11:37:01.343197+0000","last_undegraded":"2024-04-20T11:37:01.343197+0000","last_fullsized":"2024-04-20T11:37:01.343197+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_clean_scrub_stamp":"2024-04-20T11:36:58.282949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:00:08.999097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-20T11:37:01.342751+0000","last_change":"2024-04-20T11:37:00.352697+0000","last_active":"2024-04-20T11:37:01.342751+0000","last_peered":"2024-04-20T11:37:01.342751+0000","last_clean":"2024-04-20T11:37:01.342751+0000","last_became_active":"2024-04-20T11:37:00.352425+0000","last_became_peered":"2024-04-20T11:37:00.352425+0000","last_unstale":"2024-04-20T11:37:01.342751+0000","last_undegraded":"2024-04-20T11:37:01.342751+0000","last_fullsized":"2024-04-20T11:37:01.342751+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:09:55.938321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1a","version":"153'1","reported_seq":15,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-20T11:37:01.341840+0000","last_change":"2024-04-20T11:36:59.340764+0000","last_active":"2024-04-20T11:37:01.341840+0000","last_peered":"2024-04-20T11:37:01.341840+0000","last_clean":"2024-04-20T11:37:01.341840+0000","last_became_active":"2024-04-20T11:36:59.339578+0000","last_became_peered":"2024-04-20T11:36:59.339578+0000","last_unstale":"2024-04-20T11:37:01.341840+0000","last_undegraded":"2024-04-20T11:37:01.341840+0000","last_fullsized":"2024-04-20T11:37:01.341840+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_clean_scrub_stamp":"2024-04-20T11:36:58.282949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:17:32.151650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-20T11:37:01.812647+0000","last_change":"2024-04-20T11:37:00.344905+0000","last_active":"2024-04-20T11:37:01.812647+0000","last_peered":"2024-04-20T11:37:01.812647+0000","last_clean":"2024-04-20T11:37:01.812647+0000","last_became_active":"2024-04-20T11:37:00.342855+0000","last_became_peered":"2024-04-20T11:37:00.342855+0000","last_unstale":"2024-04-20T11:37:01.812647+0000","last_undegraded":"2024-04-20T11:37:01.812647+0000","last_fullsized":"2024-04-20T11:37:01.812647+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:15:24.090590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-20T11:37:01.812759+0000","last_change":"2024-04-20T11:36:59.354909+0000","last_active":"2024-04-20T11:37:01.812759+0000","last_peered":"2024-04-20T11:37:01.812759+0000","last_clean":"2024-04-20T11:37:01.812759+0000","last_became_active":"2024-04-20T11:36:59.353605+0000","last_became_peered":"2024-04-20T11:36:59.353605+0000","last_unstale":"2024-04-20T11:37:01.812759+0000","last_undegraded":"2024-04-20T11:37:01.812759+0000","last_fullsized":"2024-04-20T11:37:01.812759+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_clean_scrub_stamp":"2024-04-20T11:36:58.282949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:05:22.859522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-20T11:37:01.341261+0000","last_change":"2024-04-20T11:37:00.346050+0000","last_active":"2024-04-20T11:37:01.341261+0000","last_peered":"2024-04-20T11:37:01.341261+0000","last_clean":"2024-04-20T11:37:01.341261+0000","last_became_active":"2024-04-20T11:37:00.345765+0000","last_became_peered":"2024-04-20T11:37:00.345765+0000","last_unstale":"2024-04-20T11:37:01.341261+0000","last_undegraded":"2024-04-20T11:37:01.341261+0000","last_fullsized":"2024-04-20T11:37:01.341261+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:29:15.657930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-20T11:37:01.815017+0000","last_change":"2024-04-20T11:36:59.349365+0000","last_active":"2024-04-20T11:37:01.815017+0000","last_peered":"2024-04-20T11:37:01.815017+0000","last_clean":"2024-04-20T11:37:01.815017+0000","last_became_active":"2024-04-20T11:36:59.349012+0000","last_became_peered":"2024-04-20T11:36:59.349012+0000","last_unstale":"2024-04-20T11:37:01.815017+0000","last_undegraded":"2024-04-20T11:37:01.815017+0000","last_fullsized":"2024-04-20T11:37:01.815017+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_clean_scrub_stamp":"2024-04-20T11:36:58.282949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:33:43.618669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-20T11:37:01.342227+0000","last_change":"2024-04-20T11:37:00.341289+0000","last_active":"2024-04-20T11:37:01.342227+0000","last_peered":"2024-04-20T11:37:01.342227+0000","last_clean":"2024-04-20T11:37:01.342227+0000","last_became_active":"2024-04-20T11:37:00.340960+0000","last_became_peered":"2024-04-20T11:37:00.340960+0000","last_unstale":"2024-04-20T11:37:01.342227+0000","last_undegraded":"2024-04-20T11:37:01.342227+0000","last_fullsized":"2024-04-20T11:37:01.342227+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:50:53.626641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-20T11:37:01.343610+0000","last_change":"2024-04-20T11:36:59.352196+0000","last_active":"2024-04-20T11:37:01.343610+0000","last_peered":"2024-04-20T11:37:01.343610+0000","last_clean":"2024-04-20T11:37:01.343610+0000","last_became_active":"2024-04-20T11:36:59.351662+0000","last_became_peered":"2024-04-20T11:36:59.351662+0000","last_unstale":"2024-04-20T11:37:01.343610+0000","last_undegraded":"2024-04-20T11:37:01.343610+0000","last_fullsized":"2024-04-20T11:37:01.343610+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_clean_scrub_stamp":"2024-04-20T11:36:58.282949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:57:18.256202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-20T11:37:01.812504+0000","last_change":"2024-04-20T11:37:00.350231+0000","last_active":"2024-04-20T11:37:01.812504+0000","last_peered":"2024-04-20T11:37:01.812504+0000","last_clean":"2024-04-20T11:37:01.812504+0000","last_became_active":"2024-04-20T11:37:00.349928+0000","last_became_peered":"2024-04-20T11:37:00.349928+0000","last_unstale":"2024-04-20T11:37:01.812504+0000","last_undegraded":"2024-04-20T11:37:01.812504+0000","last_fullsized":"2024-04-20T11:37:01.812504+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:55:28.984886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.15","version":"153'3","reported_seq":17,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-20T11:37:01.343477+0000","last_change":"2024-04-20T11:36:59.341339+0000","last_active":"2024-04-20T11:37:01.343477+0000","last_peered":"2024-04-20T11:37:01.343477+0000","last_clean":"2024-04-20T11:37:01.343477+0000","last_became_active":"2024-04-20T11:36:59.341045+0000","last_became_peered":"2024-04-20T11:36:59.341045+0000","last_unstale":"2024-04-20T11:37:01.343477+0000","last_undegraded":"2024-04-20T11:37:01.343477+0000","last_fullsized":"2024-04-20T11:37:01.343477+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_clean_scrub_stamp":"2024-04-20T11:36:58.282949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:53:23.836175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-20T11:37:01.363096+0000","last_change":"2024-04-20T11:37:00.345926+0000","last_active":"2024-04-20T11:37:01.363096+0000","last_peered":"2024-04-20T11:37:01.363096+0000","last_clean":"2024-04-20T11:37:01.363096+0000","last_became_active":"2024-04-20T11:37:00.345606+0000","last_became_peered":"2024-04-20T11:37:00.345606+0000","last_unstale":"2024-04-20T11:37:01.363096+0000","last_undegraded":"2024-04-20T11:37:01.363096+0000","last_fullsized":"2024-04-20T11:37:01.363096+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:30:47.028621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.14","version":"153'6","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-20T11:37:01.340391+0000","last_change":"2024-04-20T11:36:59.347973+0000","last_active":"2024-04-20T11:37:01.340391+0000","last_peered":"2024-04-20T11:37:01.340391+0000","last_clean":"2024-04-20T11:37:01.340391+0000","last_became_active":"2024-04-20T11:36:59.347696+0000","last_became_peered":"2024-04-20T11:36:59.347696+0000","last_unstale":"2024-04-20T11:37:01.340391+0000","last_undegraded":"2024-04-20T11:37:01.340391+0000","last_fullsized":"2024-04-20T11:37:01.340391+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_clean_scrub_stamp":"2024-04-20T11:36:58.282949+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:49:07.575057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-20T11:37:01.340242+0000","last_change":"2024-04-20T11:37:00.344267+0000","last_active":"2024-04-20T11:37:01.340242+0000","last_peered":"2024-04-20T11:37:01.340242+0000","last_clean":"2024-04-20T11:37:01.340242+0000","last_became_active":"2024-04-20T11:37:00.343972+0000","last_became_peered":"2024-04-20T11:37:00.343972+0000","last_unstale":"2024-04-20T11:37:01.340242+0000","last_undegraded":"2024-04-20T11:37:01.340242+0000","last_fullsized":"2024-04-20T11:37:01.340242+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:42:00.785692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.13","version":"153'2","reported_seq":16,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-20T11:37:01.814128+0000","last_change":"2024-04-20T11:36:59.351813+0000","last_active":"2024-04-20T11:37:01.814128+0000","last_peered":"2024-04-20T11:37:01.814128+0000","last_clean":"2024-04-20T11:37:01.814128+0000","last_became_active":"2024-04-20T11:36:59.351545+0000","last_became_peered":"2024-04-20T11:36:59.351545+0000","last_unstale":"2024-04-20T11:37:01.814128+0000","last_undegraded":"2024-04-20T11:37:01.814128+0000","last_fullsized":"2024-04-20T11:37:01.814128+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_clean_scrub_stamp":"2024-04-20T11:36:58.282949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:51:28.142827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-20T11:37:01.812916+0000","last_change":"2024-04-20T11:37:00.341804+0000","last_active":"2024-04-20T11:37:01.812916+0000","last_peered":"2024-04-20T11:37:01.812916+0000","last_clean":"2024-04-20T11:37:01.812916+0000","last_became_active":"2024-04-20T11:37:00.341266+0000","last_became_peered":"2024-04-20T11:37:00.341266+0000","last_unstale":"2024-04-20T11:37:01.812916+0000","last_undegraded":"2024-04-20T11:37:01.812916+0000","last_fullsized":"2024-04-20T11:37:01.812916+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:49:11.661094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-20T11:37:01.363049+0000","last_change":"2024-04-20T11:36:59.345039+0000","last_active":"2024-04-20T11:37:01.363049+0000","last_peered":"2024-04-20T11:37:01.363049+0000","last_clean":"2024-04-20T11:37:01.363049+0000","last_became_active":"2024-04-20T11:36:59.344670+0000","last_became_peered":"2024-04-20T11:36:59.344670+0000","last_unstale":"2024-04-20T11:37:01.363049+0000","last_undegraded":"2024-04-20T11:37:01.363049+0000","last_fullsized":"2024-04-20T11:37:01.363049+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_clean_scrub_stamp":"2024-04-20T11:36:58.282949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:40:18.271898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-20T11:37:01.344681+0000","last_change":"2024-04-20T11:37:00.348521+0000","last_active":"2024-04-20T11:37:01.344681+0000","last_peered":"2024-04-20T11:37:01.344681+0000","last_clean":"2024-04-20T11:37:01.344681+0000","last_became_active":"2024-04-20T11:37:00.346307+0000","last_became_peered":"2024-04-20T11:37:00.346307+0000","last_unstale":"2024-04-20T11:37:01.344681+0000","last_undegraded":"2024-04-20T11:37:01.344681+0000","last_fullsized":"2024-04-20T11:37:01.344681+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:56:18.846599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-20T11:37:01.341401+0000","last_change":"2024-04-20T11:36:59.352698+0000","last_active":"2024-04-20T11:37:01.341401+0000","last_peered":"2024-04-20T11:37:01.341401+0000","last_clean":"2024-04-20T11:37:01.341401+0000","last_became_active":"2024-04-20T11:36:59.352278+0000","last_became_peered":"2024-04-20T11:36:59.352278+0000","last_unstale":"2024-04-20T11:37:01.341401+0000","last_undegraded":"2024-04-20T11:37:01.341401+0000","last_fullsized":"2024-04-20T11:37:01.341401+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_clean_scrub_stamp":"2024-04-20T11:36:58.282949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:19:32.299032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-20T11:37:01.362844+0000","last_change":"2024-04-20T11:37:00.349971+0000","last_active":"2024-04-20T11:37:01.362844+0000","last_peered":"2024-04-20T11:37:01.362844+0000","last_clean":"2024-04-20T11:37:01.362844+0000","last_became_active":"2024-04-20T11:37:00.348010+0000","last_became_peered":"2024-04-20T11:37:00.348010+0000","last_unstale":"2024-04-20T11:37:01.362844+0000","last_undegraded":"2024-04-20T11:37:01.362844+0000","last_fullsized":"2024-04-20T11:37:01.362844+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:54:20.660104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.10","version":"153'8","reported_seq":22,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-20T11:37:01.813052+0000","last_change":"2024-04-20T11:36:59.354209+0000","last_active":"2024-04-20T11:37:01.813052+0000","last_peered":"2024-04-20T11:37:01.813052+0000","last_clean":"2024-04-20T11:37:01.813052+0000","last_became_active":"2024-04-20T11:36:59.353912+0000","last_became_peered":"2024-04-20T11:36:59.353912+0000","last_unstale":"2024-04-20T11:37:01.813052+0000","last_undegraded":"2024-04-20T11:37:01.813052+0000","last_fullsized":"2024-04-20T11:37:01.813052+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_clean_scrub_stamp":"2024-04-20T11:36:58.282949+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:11:34.680755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-20T11:37:01.363222+0000","last_change":"2024-04-20T11:37:00.347301+0000","last_active":"2024-04-20T11:37:01.363222+0000","last_peered":"2024-04-20T11:37:01.363222+0000","last_clean":"2024-04-20T11:37:01.363222+0000","last_became_active":"2024-04-20T11:37:00.347000+0000","last_became_peered":"2024-04-20T11:37:00.347000+0000","last_unstale":"2024-04-20T11:37:01.363222+0000","last_undegraded":"2024-04-20T11:37:01.363222+0000","last_fullsized":"2024-04-20T11:37:01.363222+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:07:29.209433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.f","version":"153'1","reported_seq":15,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-20T11:37:01.343889+0000","last_change":"2024-04-20T11:36:59.346412+0000","last_active":"2024-04-20T11:37:01.343889+0000","last_peered":"2024-04-20T11:37:01.343889+0000","last_clean":"2024-04-20T11:37:01.343889+0000","last_became_active":"2024-04-20T11:36:59.346122+0000","last_became_peered":"2024-04-20T11:36:59.346122+0000","last_unstale":"2024-04-20T11:37:01.343889+0000","last_undegraded":"2024-04-20T11:37:01.343889+0000","last_fullsized":"2024-04-20T11:37:01.343889+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_clean_scrub_stamp":"2024-04-20T11:36:58.282949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:50:16.942389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-20T11:37:01.345957+0000","last_change":"2024-04-20T11:37:00.358949+0000","last_active":"2024-04-20T11:37:01.345957+0000","last_peered":"2024-04-20T11:37:01.345957+0000","last_clean":"2024-04-20T11:37:01.345957+0000","last_became_active":"2024-04-20T11:37:00.357814+0000","last_became_peered":"2024-04-20T11:37:00.357814+0000","last_unstale":"2024-04-20T11:37:01.345957+0000","last_undegraded":"2024-04-20T11:37:01.345957+0000","last_fullsized":"2024-04-20T11:37:01.345957+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:28:24.687129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.e","version":"153'2","reported_seq":16,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-20T11:37:01.342313+0000","last_change":"2024-04-20T11:36:59.349055+0000","last_active":"2024-04-20T11:37:01.342313+0000","last_peered":"2024-04-20T11:37:01.342313+0000","last_clean":"2024-04-20T11:37:01.342313+0000","last_became_active":"2024-04-20T11:36:59.347689+0000","last_became_peered":"2024-04-20T11:36:59.347689+0000","last_unstale":"2024-04-20T11:37:01.342313+0000","last_undegraded":"2024-04-20T11:37:01.342313+0000","last_fullsized":"2024-04-20T11:37:01.342313+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_clean_scrub_stamp":"2024-04-20T11:36:58.282949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:40:18.271898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-20T11:37:01.345276+0000","last_change":"2024-04-20T11:37:00.354027+0000","last_active":"2024-04-20T11:37:01.345276+0000","last_peered":"2024-04-20T11:37:01.345276+0000","last_clean":"2024-04-20T11:37:01.345276+0000","last_became_active":"2024-04-20T11:37:00.353638+0000","last_became_peered":"2024-04-20T11:37:00.353638+0000","last_unstale":"2024-04-20T11:37:01.345276+0000","last_undegraded":"2024-04-20T11:37:01.345276+0000","last_fullsized":"2024-04-20T11:37:01.345276+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:04:00.800247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.d","version":"153'1","reported_seq":15,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-20T11:37:01.815419+0000","last_change":"2024-04-20T11:36:59.350389+0000","last_active":"2024-04-20T11:37:01.815419+0000","last_peered":"2024-04-20T11:37:01.815419+0000","last_clean":"2024-04-20T11:37:01.815419+0000","last_became_active":"2024-04-20T11:36:59.350090+0000","last_became_peered":"2024-04-20T11:36:59.350090+0000","last_unstale":"2024-04-20T11:37:01.815419+0000","last_undegraded":"2024-04-20T11:37:01.815419+0000","last_fullsized":"2024-04-20T11:37:01.815419+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_clean_scrub_stamp":"2024-04-20T11:36:58.282949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:19:32.299032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-20T11:37:01.813991+0000","last_change":"2024-04-20T11:37:00.348902+0000","last_active":"2024-04-20T11:37:01.813991+0000","last_peered":"2024-04-20T11:37:01.813991+0000","last_clean":"2024-04-20T11:37:01.813991+0000","last_became_active":"2024-04-20T11:37:00.348568+0000","last_became_peered":"2024-04-20T11:37:00.348568+0000","last_unstale":"2024-04-20T11:37:01.813991+0000","last_undegraded":"2024-04-20T11:37:01.813991+0000","last_fullsized":"2024-04-20T11:37:01.813991+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:31:48.289723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-20T11:37:01.812997+0000","last_change":"2024-04-20T11:36:59.351498+0000","last_active":"2024-04-20T11:37:01.812997+0000","last_peered":"2024-04-20T11:37:01.812997+0000","last_clean":"2024-04-20T11:37:01.812997+0000","last_became_active":"2024-04-20T11:36:59.351176+0000","last_became_peered":"2024-04-20T11:36:59.351176+0000","last_unstale":"2024-04-20T11:37:01.812997+0000","last_undegraded":"2024-04-20T11:37:01.812997+0000","last_fullsized":"2024-04-20T11:37:01.812997+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_clean_scrub_stamp":"2024-04-20T11:36:58.282949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:45:43.197080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-20T11:37:01.362974+0000","last_change":"2024-04-20T11:37:00.342943+0000","last_active":"2024-04-20T11:37:01.362974+0000","last_peered":"2024-04-20T11:37:01.362974+0000","last_clean":"2024-04-20T11:37:01.362974+0000","last_became_active":"2024-04-20T11:37:00.342651+0000","last_became_peered":"2024-04-20T11:37:00.342651+0000","last_unstale":"2024-04-20T11:37:01.362974+0000","last_undegraded":"2024-04-20T11:37:01.362974+0000","last_fullsized":"2024-04-20T11:37:01.362974+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:17:27.370526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.b","version":"153'2","reported_seq":16,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-20T11:37:01.340389+0000","last_change":"2024-04-20T11:36:59.342468+0000","last_active":"2024-04-20T11:37:01.340389+0000","last_peered":"2024-04-20T11:37:01.340389+0000","last_clean":"2024-04-20T11:37:01.340389+0000","last_became_active":"2024-04-20T11:36:59.342117+0000","last_became_peered":"2024-04-20T11:36:59.342117+0000","last_unstale":"2024-04-20T11:37:01.340389+0000","last_undegraded":"2024-04-20T11:37:01.340389+0000","last_fullsized":"2024-04-20T11:37:01.340389+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_clean_scrub_stamp":"2024-04-20T11:36:58.282949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:40:02.461169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-20T11:37:01.813323+0000","last_change":"2024-04-20T11:37:00.349252+0000","last_active":"2024-04-20T11:37:01.813323+0000","last_peered":"2024-04-20T11:37:01.813323+0000","last_clean":"2024-04-20T11:37:01.813323+0000","last_became_active":"2024-04-20T11:37:00.347087+0000","last_became_peered":"2024-04-20T11:37:00.347087+0000","last_unstale":"2024-04-20T11:37:01.813323+0000","last_undegraded":"2024-04-20T11:37:01.813323+0000","last_fullsized":"2024-04-20T11:37:01.813323+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:13:41.480168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-20T11:37:01.342634+0000","last_change":"2024-04-20T11:36:59.352352+0000","last_active":"2024-04-20T11:37:01.342634+0000","last_peered":"2024-04-20T11:37:01.342634+0000","last_clean":"2024-04-20T11:37:01.342634+0000","last_became_active":"2024-04-20T11:36:59.352048+0000","last_became_peered":"2024-04-20T11:36:59.352048+0000","last_unstale":"2024-04-20T11:37:01.342634+0000","last_undegraded":"2024-04-20T11:37:01.342634+0000","last_fullsized":"2024-04-20T11:37:01.342634+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_clean_scrub_stamp":"2024-04-20T11:36:58.282949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:42:17.958777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-20T11:37:01.811393+0000","last_change":"2024-04-20T11:37:00.342310+0000","last_active":"2024-04-20T11:37:01.811393+0000","last_peered":"2024-04-20T11:37:01.811393+0000","last_clean":"2024-04-20T11:37:01.811393+0000","last_became_active":"2024-04-20T11:37:00.341991+0000","last_became_peered":"2024-04-20T11:37:00.341991+0000","last_unstale":"2024-04-20T11:37:01.811393+0000","last_undegraded":"2024-04-20T11:37:01.811393+0000","last_fullsized":"2024-04-20T11:37:01.811393+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:01:19.582395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-20T11:37:01.340265+0000","last_change":"2024-04-20T11:36:59.341401+0000","last_active":"2024-04-20T11:37:01.340265+0000","last_peered":"2024-04-20T11:37:01.340265+0000","last_clean":"2024-04-20T11:37:01.340265+0000","last_became_active":"2024-04-20T11:36:59.341140+0000","last_became_peered":"2024-04-20T11:36:59.341140+0000","last_unstale":"2024-04-20T11:37:01.340265+0000","last_undegraded":"2024-04-20T11:37:01.340265+0000","last_fullsized":"2024-04-20T11:37:01.340265+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_clean_scrub_stamp":"2024-04-20T11:36:58.282949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:57:44.255815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-20T11:37:01.342384+0000","last_change":"2024-04-20T11:37:00.339552+0000","last_active":"2024-04-20T11:37:01.342384+0000","last_peered":"2024-04-20T11:37:01.342384+0000","last_clean":"2024-04-20T11:37:01.342384+0000","last_became_active":"2024-04-20T11:37:00.339168+0000","last_became_peered":"2024-04-20T11:37:00.339168+0000","last_unstale":"2024-04-20T11:37:01.342384+0000","last_undegraded":"2024-04-20T11:37:01.342384+0000","last_fullsized":"2024-04-20T11:37:01.342384+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:35:41.731529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-20T11:37:01.344587+0000","last_change":"2024-04-20T11:36:59.352410+0000","last_active":"2024-04-20T11:37:01.344587+0000","last_peered":"2024-04-20T11:37:01.344587+0000","last_clean":"2024-04-20T11:37:01.344587+0000","last_became_active":"2024-04-20T11:36:59.352084+0000","last_became_peered":"2024-04-20T11:36:59.352084+0000","last_unstale":"2024-04-20T11:37:01.344587+0000","last_undegraded":"2024-04-20T11:37:01.344587+0000","last_fullsized":"2024-04-20T11:37:01.344587+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_clean_scrub_stamp":"2024-04-20T11:36:58.282949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:11:01.569808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-20T11:37:01.343649+0000","last_change":"2024-04-20T11:37:00.338888+0000","last_active":"2024-04-20T11:37:01.343649+0000","last_peered":"2024-04-20T11:37:01.343649+0000","last_clean":"2024-04-20T11:37:01.343649+0000","last_became_active":"2024-04-20T11:37:00.338565+0000","last_became_peered":"2024-04-20T11:37:00.338565+0000","last_unstale":"2024-04-20T11:37:01.343649+0000","last_undegraded":"2024-04-20T11:37:01.343649+0000","last_fullsized":"2024-04-20T11:37:01.343649+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:41:40.228890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"153'2","reported_seq":16,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-20T11:37:01.815748+0000","last_change":"2024-04-20T11:36:59.352710+0000","last_active":"2024-04-20T11:37:01.815748+0000","last_peered":"2024-04-20T11:37:01.815748+0000","last_clean":"2024-04-20T11:37:01.815748+0000","last_became_active":"2024-04-20T11:36:59.351394+0000","last_became_peered":"2024-04-20T11:36:59.351394+0000","last_unstale":"2024-04-20T11:37:01.815748+0000","last_undegraded":"2024-04-20T11:37:01.815748+0000","last_fullsized":"2024-04-20T11:37:01.815748+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_clean_scrub_stamp":"2024-04-20T11:36:58.282949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:17:32.151650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-20T11:37:01.363685+0000","last_change":"2024-04-20T11:37:00.335465+0000","last_active":"2024-04-20T11:37:01.363685+0000","last_peered":"2024-04-20T11:37:01.363685+0000","last_clean":"2024-04-20T11:37:01.363685+0000","last_became_active":"2024-04-20T11:37:00.330242+0000","last_became_peered":"2024-04-20T11:37:00.330242+0000","last_unstale":"2024-04-20T11:37:01.363685+0000","last_undegraded":"2024-04-20T11:37:01.363685+0000","last_fullsized":"2024-04-20T11:37:01.363685+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:45:42.805857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.6","version":"154'1","reported_seq":15,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-20T11:37:01.363264+0000","last_change":"2024-04-20T11:36:59.342161+0000","last_active":"2024-04-20T11:37:01.363264+0000","last_peered":"2024-04-20T11:37:01.363264+0000","last_clean":"2024-04-20T11:37:01.363264+0000","last_became_active":"2024-04-20T11:36:59.341904+0000","last_became_peered":"2024-04-20T11:36:59.341904+0000","last_unstale":"2024-04-20T11:37:01.363264+0000","last_undegraded":"2024-04-20T11:37:01.363264+0000","last_fullsized":"2024-04-20T11:37:01.363264+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_clean_scrub_stamp":"2024-04-20T11:36:58.282949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:24:45.505159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-20T11:37:01.342739+0000","last_change":"2024-04-20T11:37:00.336987+0000","last_active":"2024-04-20T11:37:01.342739+0000","last_peered":"2024-04-20T11:37:01.342739+0000","last_clean":"2024-04-20T11:37:01.342739+0000","last_became_active":"2024-04-20T11:37:00.336698+0000","last_became_peered":"2024-04-20T11:37:00.336698+0000","last_unstale":"2024-04-20T11:37:01.342739+0000","last_undegraded":"2024-04-20T11:37:01.342739+0000","last_fullsized":"2024-04-20T11:37:01.342739+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:42:57.369741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-20T11:37:01.363295+0000","last_change":"2024-04-20T11:36:59.345579+0000","last_active":"2024-04-20T11:37:01.363295+0000","last_peered":"2024-04-20T11:37:01.363295+0000","last_clean":"2024-04-20T11:37:01.363295+0000","last_became_active":"2024-04-20T11:36:59.345301+0000","last_became_peered":"2024-04-20T11:36:59.345301+0000","last_unstale":"2024-04-20T11:37:01.363295+0000","last_undegraded":"2024-04-20T11:37:01.363295+0000","last_fullsized":"2024-04-20T11:37:01.363295+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_clean_scrub_stamp":"2024-04-20T11:36:58.282949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:50:01.416972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-20T11:37:01.342358+0000","last_change":"2024-04-20T11:37:00.349182+0000","last_active":"2024-04-20T11:37:01.342358+0000","last_peered":"2024-04-20T11:37:01.342358+0000","last_clean":"2024-04-20T11:37:01.342358+0000","last_became_active":"2024-04-20T11:37:00.348647+0000","last_became_peered":"2024-04-20T11:37:00.348647+0000","last_unstale":"2024-04-20T11:37:01.342358+0000","last_undegraded":"2024-04-20T11:37:01.342358+0000","last_fullsized":"2024-04-20T11:37:01.342358+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:17:39.579767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-20T11:37:01.342789+0000","last_change":"2024-04-20T11:36:59.349739+0000","last_active":"2024-04-20T11:37:01.342789+0000","last_peered":"2024-04-20T11:37:01.342789+0000","last_clean":"2024-04-20T11:37:01.342789+0000","last_became_active":"2024-04-20T11:36:59.348498+0000","last_became_peered":"2024-04-20T11:36:59.348498+0000","last_unstale":"2024-04-20T11:37:01.342789+0000","last_undegraded":"2024-04-20T11:37:01.342789+0000","last_fullsized":"2024-04-20T11:37:01.342789+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_clean_scrub_stamp":"2024-04-20T11:36:58.282949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:24:45.505159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-20T11:37:01.815288+0000","last_change":"2024-04-20T11:37:00.357683+0000","last_active":"2024-04-20T11:37:01.815288+0000","last_peered":"2024-04-20T11:37:01.815288+0000","last_clean":"2024-04-20T11:37:01.815288+0000","last_became_active":"2024-04-20T11:37:00.357388+0000","last_became_peered":"2024-04-20T11:37:00.357388+0000","last_unstale":"2024-04-20T11:37:01.815288+0000","last_undegraded":"2024-04-20T11:37:01.815288+0000","last_fullsized":"2024-04-20T11:37:01.815288+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:49:06.804585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-20T11:37:01.345105+0000","last_change":"2024-04-20T11:36:59.347468+0000","last_active":"2024-04-20T11:37:01.345105+0000","last_peered":"2024-04-20T11:37:01.345105+0000","last_clean":"2024-04-20T11:37:01.345105+0000","last_became_active":"2024-04-20T11:36:59.347181+0000","last_became_peered":"2024-04-20T11:36:59.347181+0000","last_unstale":"2024-04-20T11:37:01.345105+0000","last_undegraded":"2024-04-20T11:37:01.345105+0000","last_fullsized":"2024-04-20T11:37:01.345105+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_clean_scrub_stamp":"2024-04-20T11:36:58.282949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:09:23.335650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-20T11:37:01.813987+0000","last_change":"2024-04-20T11:37:00.348745+0000","last_active":"2024-04-20T11:37:01.813987+0000","last_peered":"2024-04-20T11:37:01.813987+0000","last_clean":"2024-04-20T11:37:01.813987+0000","last_became_active":"2024-04-20T11:37:00.348469+0000","last_became_peered":"2024-04-20T11:37:00.348469+0000","last_unstale":"2024-04-20T11:37:01.813987+0000","last_undegraded":"2024-04-20T11:37:01.813987+0000","last_fullsized":"2024-04-20T11:37:01.813987+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:27:51.374548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-20T11:37:01.345235+0000","last_change":"2024-04-20T11:36:59.346814+0000","last_active":"2024-04-20T11:37:01.345235+0000","last_peered":"2024-04-20T11:37:01.345235+0000","last_clean":"2024-04-20T11:37:01.345235+0000","last_became_active":"2024-04-20T11:36:59.346372+0000","last_became_peered":"2024-04-20T11:36:59.346372+0000","last_unstale":"2024-04-20T11:37:01.345235+0000","last_undegraded":"2024-04-20T11:37:01.345235+0000","last_fullsized":"2024-04-20T11:37:01.345235+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_clean_scrub_stamp":"2024-04-20T11:36:58.282949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:31:05.381877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-20T11:37:01.811790+0000","last_change":"2024-04-20T11:37:00.344618+0000","last_active":"2024-04-20T11:37:01.811790+0000","last_peered":"2024-04-20T11:37:01.811790+0000","last_clean":"2024-04-20T11:37:01.811790+0000","last_became_active":"2024-04-20T11:37:00.344300+0000","last_became_peered":"2024-04-20T11:37:00.344300+0000","last_unstale":"2024-04-20T11:37:01.811790+0000","last_undegraded":"2024-04-20T11:37:01.811790+0000","last_fullsized":"2024-04-20T11:37:01.811790+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:45:07.218500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-20T11:37:01.813118+0000","last_change":"2024-04-20T11:36:59.350732+0000","last_active":"2024-04-20T11:37:01.813118+0000","last_peered":"2024-04-20T11:37:01.813118+0000","last_clean":"2024-04-20T11:37:01.813118+0000","last_became_active":"2024-04-20T11:36:59.347179+0000","last_became_peered":"2024-04-20T11:36:59.347179+0000","last_unstale":"2024-04-20T11:37:01.813118+0000","last_undegraded":"2024-04-20T11:37:01.813118+0000","last_fullsized":"2024-04-20T11:37:01.813118+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_clean_scrub_stamp":"2024-04-20T11:36:58.282949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:59:22.319016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-20T11:37:01.342536+0000","last_change":"2024-04-20T11:37:00.351714+0000","last_active":"2024-04-20T11:37:01.342536+0000","last_peered":"2024-04-20T11:37:01.342536+0000","last_clean":"2024-04-20T11:37:01.342536+0000","last_became_active":"2024-04-20T11:37:00.351367+0000","last_became_peered":"2024-04-20T11:37:00.351367+0000","last_unstale":"2024-04-20T11:37:01.342536+0000","last_undegraded":"2024-04-20T11:37:01.342536+0000","last_fullsized":"2024-04-20T11:37:01.342536+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:43:19.344278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-20T11:37:01.339654+0000","last_change":"2024-04-20T11:36:59.333484+0000","last_active":"2024-04-20T11:37:01.339654+0000","last_peered":"2024-04-20T11:37:01.339654+0000","last_clean":"2024-04-20T11:37:01.339654+0000","last_became_active":"2024-04-20T11:36:59.333118+0000","last_became_peered":"2024-04-20T11:36:59.333118+0000","last_unstale":"2024-04-20T11:37:01.339654+0000","last_undegraded":"2024-04-20T11:37:01.339654+0000","last_fullsized":"2024-04-20T11:37:01.339654+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_clean_scrub_stamp":"2024-04-20T11:36:58.282949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:41:18.702609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-20T11:37:01.362544+0000","last_change":"2024-04-20T11:37:00.346411+0000","last_active":"2024-04-20T11:37:01.362544+0000","last_peered":"2024-04-20T11:37:01.362544+0000","last_clean":"2024-04-20T11:37:01.362544+0000","last_became_active":"2024-04-20T11:37:00.345955+0000","last_became_peered":"2024-04-20T11:37:00.345955+0000","last_unstale":"2024-04-20T11:37:01.362544+0000","last_undegraded":"2024-04-20T11:37:01.362544+0000","last_fullsized":"2024-04-20T11:37:01.362544+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:59.299355+0000","last_clean_scrub_stamp":"2024-04-20T11:36:59.299355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:09:55.938321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-20T11:37:01.811859+0000","last_change":"2024-04-20T11:36:59.354783+0000","last_active":"2024-04-20T11:37:01.811859+0000","last_peered":"2024-04-20T11:37:01.811859+0000","last_clean":"2024-04-20T11:37:01.811859+0000","last_became_active":"2024-04-20T11:36:59.354252+0000","last_became_peered":"2024-04-20T11:36:59.354252+0000","last_unstale":"2024-04-20T11:37:01.811859+0000","last_undegraded":"2024-04-20T11:37:01.811859+0000","last_fullsized":"2024-04-20T11:37:01.811859+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:36:58.282949+0000","last_clean_scrub_stamp":"2024-04-20T11:36:58.282949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:15:11.879393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"72'128","reported_seq":450,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-20T11:37:01.813360+0000","last_change":"2024-04-20T11:26:48.661897+0000","last_active":"2024-04-20T11:37:01.813360+0000","last_peered":"2024-04-20T11:37:01.813360+0000","last_clean":"2024-04-20T11:37:01.813360+0000","last_became_active":"2024-04-20T11:26:48.656535+0000","last_became_peered":"2024-04-20T11:26:48.656535+0000","last_unstale":"2024-04-20T11:37:01.813360+0000","last_undegraded":"2024-04-20T11:37:01.813360+0000","last_fullsized":"2024-04-20T11:37:01.813360+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:47.543004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:47.543004+0000","last_clean_scrub_stamp":"2024-04-20T11:26:47.543004+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:58:35.527105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":25,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":24,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574977,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33664,"kb_used_data":2484,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338176,"statfs":{"total":96636764160,"available":96602292224,"internally_reserved":0,"allocated":2543616,"data_stored":1448705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574974,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33080,"kb_used_data":1896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338760,"statfs":{"total":96636764160,"available":96602890240,"internally_reserved":0,"allocated":1941504,"data_stored":858233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574976,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33084,"kb_used_data":1896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338756,"statfs":{"total":96636764160,"available":96602886144,"internally_reserved":0,"allocated":1941504,"data_stored":858233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574974,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33092,"kb_used_data":1912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338748,"statfs":{"total":96636764160,"available":96602877952,"internally_reserved":0,"allocated":1957888,"data_stored":860427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574975,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33088,"kb_used_data":1908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338752,"statfs":{"total":96636764160,"available":96602882048,"internally_reserved":0,"allocated":1953792,"data_stored":858427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574976,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33088,"kb_used_data":1908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338752,"statfs":{"total":96636764160,"available":96602882048,"internally_reserved":0,"allocated":1953792,"data_stored":858927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574974,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33096,"kb_used_data":1908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338744,"statfs":{"total":96636764160,"available":96602873856,"internally_reserved":0,"allocated":1953792,"data_stored":859845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574975,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33668,"kb_used_data":2480,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338172,"statfs":{"total":96636764160,"available":96602288128,"internally_reserved":0,"allocated":2539520,"data_stored":1448623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T11:37:05.235 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-20T11:37:05.235 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-20T11:37:05.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:05.478+0000 7f72a7200640 1 Processor -- start 2024-04-20T11:37:05.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:05.478+0000 7f72a7200640 1 -- start start 2024-04-20T11:37:05.480 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:05.480+0000 7f72a7200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f72a80d1690 0x7f72a80cfcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:05.480 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:05.480+0000 7f72a7200640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72a80d54d0 con 0x7f72a8161c20 2024-04-20T11:37:05.480 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:05.480+0000 7f72a7200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72a80d0400 con 0x7f72a80d5060 2024-04-20T11:37:05.480 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:05.480+0000 7f72a7200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f72a80d2160 con 0x7f72a80d1690 2024-04-20T11:37:05.480 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:05.481+0000 7f72a6800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f72a80d1690 0x7f72a80cfcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:05.480 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:05.481+0000 7f72a5e00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f72a8161c20 0x7f72a80d2e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:52196/0 (socket says 172.21.15.152:52196) 2024-04-20T11:37:05.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:05.481+0000 7f72a5e00640 1 -- 172.21.15.152:0/3788384713 learned_addr learned my addr 172.21.15.152:0/3788384713 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:37:05.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:05.481+0000 7f72a6800640 1 -- 172.21.15.152:0/3788384713 >> v1:172.21.15.175:6789/0 conn(0x7f72a80d5060 legacy=0x7f72a80d0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:05.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:05.481+0000 7f72a6800640 1 -- 172.21.15.152:0/3788384713 >> v1:172.21.15.152:6789/0 conn(0x7f72a8161c20 legacy=0x7f72a80d2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:05.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:05.481+0000 7f72a6800640 1 -- 172.21.15.152:0/3788384713 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72a80d23b0 con 0x7f72a80d1690 2024-04-20T11:37:05.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:05.481+0000 7f72a4a00640 1 -- 172.21.15.152:0/3788384713 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1243018237 0 0) 0x7f72a80d54d0 con 0x7f72a8161c20 2024-04-20T11:37:05.482 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:05.482+0000 7f72a6800640 1 --2- 172.21.15.152:0/3788384713 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f72a80d1690 0x7f72a80cfcb0 secure :-1 s=READY pgs=2156 cs=0 l=1 rev1=1 crypto rx=0x7f72a00097a0 tx=0x7f72a002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=818b849d5732fcce server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:05.482 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:05.483+0000 7f72a4a00640 1 -- 172.21.15.152:0/3788384713 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a00315a0 con 0x7f72a80d1690 2024-04-20T11:37:05.482 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:05.483+0000 7f72a4a00640 1 -- 172.21.15.152:0/3788384713 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72a0037070 con 0x7f72a80d1690 2024-04-20T11:37:05.482 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:05.483+0000 7f72a4a00640 1 -- 172.21.15.152:0/3788384713 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a0031960 con 0x7f72a80d1690 2024-04-20T11:37:05.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:05.483+0000 7f72a7200640 1 -- 172.21.15.152:0/3788384713 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f72a80d1690 msgr2=0x7f72a80cfcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:05.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:05.483+0000 7f72a7200640 1 --2- 172.21.15.152:0/3788384713 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f72a80d1690 0x7f72a80cfcb0 secure :-1 s=READY pgs=2156 cs=0 l=1 rev1=1 crypto rx=0x7f72a00097a0 tx=0x7f72a002b840 comp rx=0 tx=0).stop 2024-04-20T11:37:05.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:05.483+0000 7f72a7200640 1 -- 172.21.15.152:0/3788384713 shutdown_connections 2024-04-20T11:37:05.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:05.483+0000 7f72a7200640 1 --2- 172.21.15.152:0/3788384713 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f72a80d1690 0x7f72a80cfcb0 unknown :-1 s=CLOSED pgs=2156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:05.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:05.483+0000 7f72a7200640 1 -- 172.21.15.152:0/3788384713 >> 172.21.15.152:0/3788384713 conn(0x7f72a808bfd0 msgr2=0x7f72a80d4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:05.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:05.483+0000 7f72a7200640 1 -- 172.21.15.152:0/3788384713 shutdown_connections 2024-04-20T11:37:05.484 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:05.483+0000 7f72a7200640 1 -- 172.21.15.152:0/3788384713 wait complete. 2024-04-20T11:37:05.484 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:05.485+0000 7f72a7200640 1 Processor -- start 2024-04-20T11:37:05.484 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:05.485+0000 7f72a7200640 1 -- start start 2024-04-20T11:37:05.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:05.486+0000 7f72a7200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f72a80d1690 0x7f72a809ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:05.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:05.486+0000 7f72a7200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72a80d5060 0x7f72a81f46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:05.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:05.486+0000 7f72a7200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f72a8161c20 0x7f72a80b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:05.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:05.486+0000 7f72a7200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f72a80d35a0 con 0x7f72a80d1690 2024-04-20T11:37:05.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:05.486+0000 7f72a7200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f72a80d2a00 con 0x7f72a80d5060 2024-04-20T11:37:05.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:05.486+0000 7f72a7200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f72a8162000 con 0x7f72a8161c20 2024-04-20T11:37:05.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:05.486+0000 7f72a5e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f72a80d1690 0x7f72a809ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:05.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:05.486+0000 7f72a5e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f72a80d1690 0x7f72a809ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:57254/0 (socket says 172.21.15.152:57254) 2024-04-20T11:37:05.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:05.486+0000 7f72a5e00640 1 -- 172.21.15.152:0/710867887 learned_addr learned my addr 172.21.15.152:0/710867887 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:05.487 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:05.487+0000 7f72a6800640 1 --2- 172.21.15.152:0/710867887 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f72a8161c20 0x7f72a80b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:05.487 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:05.487+0000 7f72a5e00640 1 -- 172.21.15.152:0/710867887 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f72a8161c20 msgr2=0x7f72a80b4d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:05.487 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:05.487+0000 7f72a5e00640 1 --2- 172.21.15.152:0/710867887 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f72a8161c20 0x7f72a80b4d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:05.488 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:05.487+0000 7f72a5e00640 1 -- 172.21.15.152:0/710867887 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72a80d5060 msgr2=0x7f72a81f46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:05.488 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:05.487+0000 7f72a5e00640 1 --2- 172.21.15.152:0/710867887 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72a80d5060 0x7f72a81f46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:05.488 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:05.487+0000 7f72a5e00640 1 -- 172.21.15.152:0/710867887 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72a815f110 con 0x7f72a80d1690 2024-04-20T11:37:05.488 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:05.487+0000 7f72a5e00640 1 --2- 172.21.15.152:0/710867887 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f72a80d1690 0x7f72a809ec30 secure :-1 s=READY pgs=2157 cs=0 l=1 rev1=1 crypto rx=0x7f729800c890 tx=0x7f729800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:05.488 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:05.487+0000 7f729e000640 1 -- 172.21.15.152:0/710867887 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7298018070 con 0x7f72a80d1690 2024-04-20T11:37:05.488 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:05.487+0000 7f729e000640 1 -- 172.21.15.152:0/710867887 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f729800a640 con 0x7f72a80d1690 2024-04-20T11:37:05.488 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:05.487+0000 7f729e000640 1 -- 172.21.15.152:0/710867887 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f729800aa30 con 0x7f72a80d1690 2024-04-20T11:37:05.488 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:05.487+0000 7f72a7200640 1 -- 172.21.15.152:0/710867887 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72a809fe20 con 0x7f72a80d1690 2024-04-20T11:37:05.488 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:05.487+0000 7f72a7200640 1 -- 172.21.15.152:0/710867887 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72a809f5f0 con 0x7f72a80d1690 2024-04-20T11:37:05.488 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:05.488+0000 7f72a7200640 1 -- 172.21.15.152:0/710867887 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72a80a05d0 con 0x7f72a80d1690 2024-04-20T11:37:05.489 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:05.489+0000 7f729e000640 1 -- 172.21.15.152:0/710867887 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) ==== 91754+0+0 (secure 0 0 0) 0x7f729801b070 con 0x7f72a80d1690 2024-04-20T11:37:05.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:05.490+0000 7f729e000640 1 --2- 172.21.15.152:0/710867887 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f728806d0d0 0x7f728806f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:05.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:05.490+0000 7f729e000640 1 -- 172.21.15.152:0/710867887 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(154..154 src has 1..154) ==== 5709+0+0 (secure 0 0 0) 0x7f729808c550 con 0x7f72a80d1690 2024-04-20T11:37:05.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:05.491+0000 7f72a5400640 1 --2- 172.21.15.152:0/710867887 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f728806d0d0 0x7f728806f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:05.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:05.492+0000 7f72a5400640 1 --2- 172.21.15.152:0/710867887 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f728806d0d0 0x7f728806f590 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f72a806e4c0 tx=0x7f72940079b0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:05.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:05.495+0000 7f729e000640 1 -- 172.21.15.152:0/710867887 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f7298059960 con 0x7f72a80d1690 2024-04-20T11:37:05.766 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:05.766+0000 7f72a7200640 1 -- 172.21.15.152:0/710867887 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f72a80d1c80 con 0x7f72a80d1690 2024-04-20T11:37:05.771 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:05.772+0000 7f729e000640 1 -- 172.21.15.152:0/710867887 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v22) ==== 190+0+147 (secure 0 0 0) 0x7f7298059260 con 0x7f72a80d1690 2024-04-20T11:37:05.771 INFO:teuthology.orchestra.run.smithi152.stdout:[client.0] 2024-04-20T11:37:05.772 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQBDpiNmtsBWFhAArq9A7r33CcwgMmOU2pAGPw== 2024-04-20T11:37:05.772 INFO:teuthology.orchestra.run.smithi152.stdout: caps mds = "allow" 2024-04-20T11:37:05.772 INFO:teuthology.orchestra.run.smithi152.stdout: caps mon = "allow r" 2024-04-20T11:37:05.772 INFO:teuthology.orchestra.run.smithi152.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-20T11:37:05.772 INFO:teuthology.orchestra.run.smithi152.stderr:updated caps for client.0 2024-04-20T11:37:05.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:05.775+0000 7f72a7200640 1 -- 172.21.15.152:0/710867887 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f728806d0d0 msgr2=0x7f728806f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:05.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:05.775+0000 7f72a7200640 1 --2- 172.21.15.152:0/710867887 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f728806d0d0 0x7f728806f590 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f72a806e4c0 tx=0x7f72940079b0 comp rx=0 tx=0).stop 2024-04-20T11:37:05.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:05.775+0000 7f72a7200640 1 -- 172.21.15.152:0/710867887 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f72a80d1690 msgr2=0x7f72a809ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:05.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:05.775+0000 7f72a7200640 1 --2- 172.21.15.152:0/710867887 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f72a80d1690 0x7f72a809ec30 secure :-1 s=READY pgs=2157 cs=0 l=1 rev1=1 crypto rx=0x7f729800c890 tx=0x7f729800cd50 comp rx=0 tx=0).stop 2024-04-20T11:37:05.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:05.775+0000 7f72a7200640 1 -- 172.21.15.152:0/710867887 shutdown_connections 2024-04-20T11:37:05.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:05.775+0000 7f72a7200640 1 --2- 172.21.15.152:0/710867887 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f728806d0d0 0x7f728806f590 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:05.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:05.775+0000 7f72a7200640 1 --2- 172.21.15.152:0/710867887 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f72a8161c20 0x7f72a80b4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:05.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:05.775+0000 7f72a7200640 1 --2- 172.21.15.152:0/710867887 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72a80d5060 0x7f72a81f46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:05.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:05.775+0000 7f72a7200640 1 --2- 172.21.15.152:0/710867887 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f72a80d1690 0x7f72a809ec30 unknown :-1 s=CLOSED pgs=2157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:05.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:05.775+0000 7f72a7200640 1 -- 172.21.15.152:0/710867887 >> 172.21.15.152:0/710867887 conn(0x7f72a808bfd0 msgr2=0x7f72a80b8d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:05.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:05.775+0000 7f72a7200640 1 -- 172.21.15.152:0/710867887 shutdown_connections 2024-04-20T11:37:05.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:05.775+0000 7f72a7200640 1 -- 172.21.15.152:0/710867887 wait complete. 2024-04-20T11:37:05.792 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:37:05.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:05.994+0000 7f742f000640 1 Processor -- start 2024-04-20T11:37:05.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:05.994+0000 7f742f000640 1 -- start start 2024-04-20T11:37:05.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:05.996+0000 7f742f000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f742815fcc0 0x7f74280d4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:05.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:05.996+0000 7f742f000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74281603d0 0x7f74280b5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:05.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:05.996+0000 7f742f000640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74280d3350 con 0x7f74280b5480 2024-04-20T11:37:05.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:05.996+0000 7f742f000640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f74280d3630 con 0x7f742815fcc0 2024-04-20T11:37:05.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:05.996+0000 7f742f000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f74280d3910 con 0x7f74281603d0 2024-04-20T11:37:05.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:05.996+0000 7f742dc00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f742815fcc0 0x7f74280d4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:05.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:05.996+0000 7f742dc00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f742815fcc0 0x7f74280d4e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:59176/0 (socket says 172.21.15.152:59176) 2024-04-20T11:37:05.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:05.996+0000 7f742dc00640 1 -- 172.21.15.152:0/3804874737 learned_addr learned my addr 172.21.15.152:0/3804874737 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:05.997 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:05.997+0000 7f742d200640 1 --2- 172.21.15.152:0/3804874737 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74281603d0 0x7f74280b5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:05.997 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:05.997+0000 7f742dc00640 1 -- 172.21.15.152:0/3804874737 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74281603d0 msgr2=0x7f74280b5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:05.997 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:05.997+0000 7f742dc00640 1 --2- 172.21.15.152:0/3804874737 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74281603d0 0x7f74280b5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:05.997 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:05.997+0000 7f742dc00640 1 -- 172.21.15.152:0/3804874737 >> v1:172.21.15.175:6789/0 conn(0x7f74280b5480 legacy=0x7f74281624d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:05.997 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:05.997+0000 7f742dc00640 1 -- 172.21.15.152:0/3804874737 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74280ced20 con 0x7f742815fcc0 2024-04-20T11:37:05.997 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:05.997+0000 7f742d200640 1 --2- 172.21.15.152:0/3804874737 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74281603d0 0x7f74280b5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:37:05.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:05.998+0000 7f742dc00640 1 --2- 172.21.15.152:0/3804874737 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f742815fcc0 0x7f74280d4e70 secure :-1 s=READY pgs=2158 cs=0 l=1 rev1=1 crypto rx=0x7f741c009870 tx=0x7f741c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3be1c327562060a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:05.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:05.998+0000 7f7427e00640 1 -- 172.21.15.152:0/3804874737 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f741c035020 con 0x7f742815fcc0 2024-04-20T11:37:05.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:05.999+0000 7f7427e00640 1 -- 172.21.15.152:0/3804874737 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f741c031640 con 0x7f742815fcc0 2024-04-20T11:37:05.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:05.999+0000 7f7427e00640 1 -- 172.21.15.152:0/3804874737 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f741c02d040 con 0x7f742815fcc0 2024-04-20T11:37:05.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:05.999+0000 7f742f000640 1 -- 172.21.15.152:0/3804874737 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f742815fcc0 msgr2=0x7f74280d4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:05.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:05.999+0000 7f742f000640 1 --2- 172.21.15.152:0/3804874737 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f742815fcc0 0x7f74280d4e70 secure :-1 s=READY pgs=2158 cs=0 l=1 rev1=1 crypto rx=0x7f741c009870 tx=0x7f741c02b8f0 comp rx=0 tx=0).stop 2024-04-20T11:37:05.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.000+0000 7f742f000640 1 -- 172.21.15.152:0/3804874737 shutdown_connections 2024-04-20T11:37:05.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.000+0000 7f742f000640 1 --2- 172.21.15.152:0/3804874737 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74281603d0 0x7f74280b5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:05.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.000+0000 7f742f000640 1 --2- 172.21.15.152:0/3804874737 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f742815fcc0 0x7f74280d4e70 unknown :-1 s=CLOSED pgs=2158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:06.000 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.000+0000 7f742f000640 1 -- 172.21.15.152:0/3804874737 >> 172.21.15.152:0/3804874737 conn(0x7f742808bfd0 msgr2=0x7f74280d4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:06.000 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.000+0000 7f742f000640 1 -- 172.21.15.152:0/3804874737 shutdown_connections 2024-04-20T11:37:06.000 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.000+0000 7f742f000640 1 -- 172.21.15.152:0/3804874737 wait complete. 2024-04-20T11:37:06.001 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.001+0000 7f742f000640 1 Processor -- start 2024-04-20T11:37:06.001 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.002+0000 7f742f000640 1 -- start start 2024-04-20T11:37:06.003 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.002+0000 7f742f000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74280b5480 0x7f74281f7390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:06.003 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.002+0000 7f742f000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f742815fcc0 0x7f74281f5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:06.003 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.002+0000 7f742f000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f74281603d0 0x7f74281f4520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:06.003 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.002+0000 7f742f000640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f74280b6570 con 0x7f74281603d0 2024-04-20T11:37:06.003 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.002+0000 7f742f000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f74280b63f0 con 0x7f742815fcc0 2024-04-20T11:37:06.003 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.002+0000 7f742f000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f74280b66f0 con 0x7f74280b5480 2024-04-20T11:37:06.003 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.003+0000 7f742e600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f74281603d0 0x7f74281f4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:06.003 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.003+0000 7f742e600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f74281603d0 0x7f74281f4520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:59186/0 (socket says 172.21.15.152:59186) 2024-04-20T11:37:06.003 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.003+0000 7f742e600640 1 -- 172.21.15.152:0/1010185575 learned_addr learned my addr 172.21.15.152:0/1010185575 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:06.003 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.003+0000 7f742d200640 1 --2- 172.21.15.152:0/1010185575 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f742815fcc0 0x7f74281f5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:06.003 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.003+0000 7f742dc00640 1 --2- 172.21.15.152:0/1010185575 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74280b5480 0x7f74281f7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:06.004 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.003+0000 7f742e600640 1 -- 172.21.15.152:0/1010185575 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74280b5480 msgr2=0x7f74281f7390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:06.004 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.003+0000 7f742e600640 1 --2- 172.21.15.152:0/1010185575 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74280b5480 0x7f74281f7390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:06.004 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.003+0000 7f742e600640 1 -- 172.21.15.152:0/1010185575 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f742815fcc0 msgr2=0x7f74281f5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:06.004 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.003+0000 7f742e600640 1 --2- 172.21.15.152:0/1010185575 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f742815fcc0 0x7f74281f5460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:06.004 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.003+0000 7f742e600640 1 -- 172.21.15.152:0/1010185575 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74281f5180 con 0x7f74281603d0 2024-04-20T11:37:06.004 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.004+0000 7f742e600640 1 --2- 172.21.15.152:0/1010185575 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f74281603d0 0x7f74281f4520 secure :-1 s=READY pgs=2159 cs=0 l=1 rev1=1 crypto rx=0x7f742000b440 tx=0x7f742000b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:06.005 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.004+0000 7f7425600640 1 -- 172.21.15.152:0/1010185575 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7420015020 con 0x7f74281603d0 2024-04-20T11:37:06.005 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.004+0000 7f7425600640 1 -- 172.21.15.152:0/1010185575 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7420011830 con 0x7f74281603d0 2024-04-20T11:37:06.005 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.004+0000 7f742f000640 1 -- 172.21.15.152:0/1010185575 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7428155a40 con 0x7f74281603d0 2024-04-20T11:37:06.005 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.004+0000 7f7425600640 1 -- 172.21.15.152:0/1010185575 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7420011c20 con 0x7f74281603d0 2024-04-20T11:37:06.006 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.004+0000 7f742f000640 1 -- 172.21.15.152:0/1010185575 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7428156000 con 0x7f74281603d0 2024-04-20T11:37:06.006 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.005+0000 7f742f000640 1 -- 172.21.15.152:0/1010185575 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7428075e60 con 0x7f74281603d0 2024-04-20T11:37:06.006 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.006+0000 7f7425600640 1 -- 172.21.15.152:0/1010185575 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) ==== 91754+0+0 (secure 0 0 0) 0x7f7420018020 con 0x7f74281603d0 2024-04-20T11:37:06.007 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.008+0000 7f7425600640 1 --2- 172.21.15.152:0/1010185575 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f740c06d0d0 0x7f740c06f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:06.007 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.008+0000 7f7425600640 1 -- 172.21.15.152:0/1010185575 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(154..154 src has 1..154) ==== 5709+0+0 (secure 0 0 0) 0x7f742008c900 con 0x7f74281603d0 2024-04-20T11:37:06.007 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.008+0000 7f742dc00640 1 --2- 172.21.15.152:0/1010185575 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f740c06d0d0 0x7f740c06f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:06.009 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.009+0000 7f742dc00640 1 --2- 172.21.15.152:0/1010185575 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f740c06d0d0 0x7f740c06f590 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f741c036000 tx=0x7f741c008030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:06.011 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.011+0000 7f7425600640 1 -- 172.21.15.152:0/1010185575 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f742001f030 con 0x7f74281603d0 2024-04-20T11:37:06.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.242+0000 7f742f000640 1 -- 172.21.15.152:0/1010185575 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f74280b6ce0 con 0x7f74281603d0 2024-04-20T11:37:06.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.243+0000 7f7425600640 1 -- 172.21.15.152:0/1010185575 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) ==== 95+0+5220 (secure 0 0 0) 0x7f74200172c0 con 0x7f74281603d0 2024-04-20T11:37:06.243 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:37:06.243 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":105,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6458,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/1688411915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":1688411915},{"type":"v1","addr":"172.21.15.152:6835","nonce":1688411915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":100},{"gid":7141,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6834/599467453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":599467453},{"type":"v1","addr":"172.21.15.175:6834","nonce":599467453}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":100},{"gid":7144,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/691089259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6833","nonce":691089259},{"type":"v1","addr":"172.21.15.175:6835","nonce":691089259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":100}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T11:36:59.789831+0000","modified":"2024-04-20T11:37:02.829697+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10011},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10011":{"gid":10011,"name":"c","rank":0,"incarnation":101,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6833/1854325981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":1854325981},{"type":"v1","addr":"172.21.15.152:6833","nonce":1854325981}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10011,"qdb_cluster":[10011]},"id":12}]} 2024-04-20T11:37:06.243 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 105 2024-04-20T11:37:06.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.245+0000 7f742f000640 1 -- 172.21.15.152:0/1010185575 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f740c06d0d0 msgr2=0x7f740c06f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:06.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.245+0000 7f742f000640 1 --2- 172.21.15.152:0/1010185575 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f740c06d0d0 0x7f740c06f590 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f741c036000 tx=0x7f741c008030 comp rx=0 tx=0).stop 2024-04-20T11:37:06.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.245+0000 7f742f000640 1 -- 172.21.15.152:0/1010185575 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f74281603d0 msgr2=0x7f74281f4520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:06.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.245+0000 7f742f000640 1 --2- 172.21.15.152:0/1010185575 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f74281603d0 0x7f74281f4520 secure :-1 s=READY pgs=2159 cs=0 l=1 rev1=1 crypto rx=0x7f742000b440 tx=0x7f742000b900 comp rx=0 tx=0).stop 2024-04-20T11:37:06.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.246+0000 7f742f000640 1 -- 172.21.15.152:0/1010185575 shutdown_connections 2024-04-20T11:37:06.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.246+0000 7f742f000640 1 --2- 172.21.15.152:0/1010185575 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f740c06d0d0 0x7f740c06f590 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:06.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.246+0000 7f742f000640 1 --2- 172.21.15.152:0/1010185575 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f74281603d0 0x7f74281f4520 unknown :-1 s=CLOSED pgs=2159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:06.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.246+0000 7f742f000640 1 --2- 172.21.15.152:0/1010185575 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f742815fcc0 0x7f74281f5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:06.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.246+0000 7f742f000640 1 --2- 172.21.15.152:0/1010185575 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74280b5480 0x7f74281f7390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:06.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.246+0000 7f742f000640 1 -- 172.21.15.152:0/1010185575 >> 172.21.15.152:0/1010185575 conn(0x7f742808bfd0 msgr2=0x7f74280d0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:06.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.246+0000 7f742f000640 1 -- 172.21.15.152:0/1010185575 shutdown_connections 2024-04-20T11:37:06.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.246+0000 7f742f000640 1 -- 172.21.15.152:0/1010185575 wait complete. 2024-04-20T11:37:06.666 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 105, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-20T11:36:59.789831+0000', 'modified': '2024-04-20T11:37:02.829697+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'snap_rstat': True, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10011}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10011': {'gid': 10011, 'name': 'c', 'rank': 0, 'incarnation': 101, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.152:6833/1854325981', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6832', 'nonce': 1854325981}, {'type': 'v1', 'addr': '172.21.15.152:6833', 'nonce': 1854325981}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [25], 'metadata_pool': 24, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10011, 'qdb_cluster': [10011]} 2024-04-20T11:37:06.666 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-20T11:37:06.667 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-20T11:37:06.879 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.880+0000 7f4ee9400640 1 Processor -- start 2024-04-20T11:37:06.879 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.880+0000 7f4ee9400640 1 -- start start 2024-04-20T11:37:06.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.881+0000 7f4ee9400640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f4ee415c7f0 0x7f4ee4072b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:06.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.881+0000 7f4ee9400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4ee4073b60 0x7f4ee40ad2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:06.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.881+0000 7f4ee9400640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ee4073fc0 con 0x7f4ee40b4160 2024-04-20T11:37:06.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.881+0000 7f4ee9400640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f4ee40ce140 con 0x7f4ee415c7f0 2024-04-20T11:37:06.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.881+0000 7f4ee9400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4ee40ce390 con 0x7f4ee4073b60 2024-04-20T11:37:06.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.882+0000 7f4edb400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4ee4073b60 0x7f4ee40ad2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:06.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.882+0000 7f4ee3e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f4ee415c7f0 0x7f4ee4072b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:06.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.882+0000 7f4edb400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4ee4073b60 0x7f4ee40ad2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.152:47044/0 (socket says 172.21.15.152:47044) 2024-04-20T11:37:06.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.882+0000 7f4edb400640 1 -- 172.21.15.152:0/902072118 learned_addr learned my addr 172.21.15.152:0/902072118 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:06.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.883+0000 7f4edb400640 1 -- 172.21.15.152:0/902072118 >> v2:172.21.15.175:3300/0 conn(0x7f4ee415c7f0 msgr2=0x7f4ee4072b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:06.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.883+0000 7f4edb400640 1 --2- 172.21.15.152:0/902072118 >> v2:172.21.15.175:3300/0 conn(0x7f4ee415c7f0 0x7f4ee4072b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:06.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.883+0000 7f4edb400640 1 -- 172.21.15.152:0/902072118 >> v1:172.21.15.175:6789/0 conn(0x7f4ee40b4160 legacy=0x7f4ee40cf430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:06.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.883+0000 7f4edb400640 1 -- 172.21.15.152:0/902072118 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ee40ce670 con 0x7f4ee4073b60 2024-04-20T11:37:06.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.884+0000 7f4edb400640 1 --2- 172.21.15.152:0/902072118 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4ee4073b60 0x7f4ee40ad2b0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc009870 tx=0x7f4ecc02b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15e0c31656cd4019 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:06.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.885+0000 7f4ee3400640 1 -- 172.21.15.152:0/902072118 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ecc035020 con 0x7f4ee4073b60 2024-04-20T11:37:06.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.885+0000 7f4ee3400640 1 -- 172.21.15.152:0/902072118 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ecc031640 con 0x7f4ee4073b60 2024-04-20T11:37:06.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.885+0000 7f4ee3400640 1 -- 172.21.15.152:0/902072118 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ecc02d040 con 0x7f4ee4073b60 2024-04-20T11:37:06.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.885+0000 7f4ee9400640 1 -- 172.21.15.152:0/902072118 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4ee4073b60 msgr2=0x7f4ee40ad2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:06.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.885+0000 7f4ee9400640 1 --2- 172.21.15.152:0/902072118 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4ee4073b60 0x7f4ee40ad2b0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc009870 tx=0x7f4ecc02b8f0 comp rx=0 tx=0).stop 2024-04-20T11:37:06.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.885+0000 7f4ee9400640 1 -- 172.21.15.152:0/902072118 shutdown_connections 2024-04-20T11:37:06.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.886+0000 7f4ee9400640 1 --2- 172.21.15.152:0/902072118 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4ee4073b60 0x7f4ee40ad2b0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:06.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.886+0000 7f4ee9400640 1 --2- 172.21.15.152:0/902072118 >> v2:172.21.15.175:3300/0 conn(0x7f4ee415c7f0 0x7f4ee4072b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:06.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.886+0000 7f4ee9400640 1 -- 172.21.15.152:0/902072118 >> 172.21.15.152:0/902072118 conn(0x7f4ee40d0260 msgr2=0x7f4ee40d0670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:06.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.886+0000 7f4ee9400640 1 -- 172.21.15.152:0/902072118 shutdown_connections 2024-04-20T11:37:06.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.886+0000 7f4ee9400640 1 -- 172.21.15.152:0/902072118 wait complete. 2024-04-20T11:37:06.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.888+0000 7f4ee9400640 1 Processor -- start 2024-04-20T11:37:06.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.888+0000 7f4ee9400640 1 -- start start 2024-04-20T11:37:06.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.889+0000 7f4ee9400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4ee4073b60 0x7f4ee41e49c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:06.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.889+0000 7f4ee9400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4ee40b4160 0x7f4ee4158720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:06.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.889+0000 7f4ee9400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ee414ddd0 0x7f4ee4159f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:06.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.889+0000 7f4ee9400640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee40cef10 con 0x7f4ee414ddd0 2024-04-20T11:37:06.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.889+0000 7f4ee9400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee40cec30 con 0x7f4ee4073b60 2024-04-20T11:37:06.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.889+0000 7f4ee9400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4ee40b4590 con 0x7f4ee40b4160 2024-04-20T11:37:06.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.889+0000 7f4ee8a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ee414ddd0 0x7f4ee4159f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:06.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.889+0000 7f4ee8a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ee414ddd0 0x7f4ee4159f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:59188/0 (socket says 172.21.15.152:59188) 2024-04-20T11:37:06.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.889+0000 7f4ee8a00640 1 -- 172.21.15.152:0/1688075748 learned_addr learned my addr 172.21.15.152:0/1688075748 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:06.891 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.890+0000 7f4edb400640 1 --2- 172.21.15.152:0/1688075748 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4ee40b4160 0x7f4ee4158720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:06.891 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.890+0000 7f4ee3e00640 1 --2- 172.21.15.152:0/1688075748 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4ee4073b60 0x7f4ee41e49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:06.891 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.890+0000 7f4ee8a00640 1 -- 172.21.15.152:0/1688075748 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4ee40b4160 msgr2=0x7f4ee4158720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:06.891 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.890+0000 7f4ee8a00640 1 --2- 172.21.15.152:0/1688075748 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4ee40b4160 0x7f4ee4158720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:06.891 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.890+0000 7f4ee8a00640 1 -- 172.21.15.152:0/1688075748 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4ee4073b60 msgr2=0x7f4ee41e49c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:06.891 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.890+0000 7f4ee8a00640 1 --2- 172.21.15.152:0/1688075748 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4ee4073b60 0x7f4ee41e49c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:06.891 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.890+0000 7f4ee8a00640 1 -- 172.21.15.152:0/1688075748 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ee4158c60 con 0x7f4ee414ddd0 2024-04-20T11:37:06.891 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.890+0000 7f4ee8a00640 1 --2- 172.21.15.152:0/1688075748 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ee414ddd0 0x7f4ee4159f80 secure :-1 s=READY pgs=2160 cs=0 l=1 rev1=1 crypto rx=0x7f4edc00c7d0 tx=0x7f4edc00cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:06.891 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.890+0000 7f4ee0c00640 1 -- 172.21.15.152:0/1688075748 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4edc003b70 con 0x7f4ee414ddd0 2024-04-20T11:37:06.891 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.891+0000 7f4ee0c00640 1 -- 172.21.15.152:0/1688075748 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4edc003d30 con 0x7f4ee414ddd0 2024-04-20T11:37:06.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.891+0000 7f4ee9400640 1 -- 172.21.15.152:0/1688075748 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ee4158ee0 con 0x7f4ee414ddd0 2024-04-20T11:37:06.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.891+0000 7f4ee0c00640 1 -- 172.21.15.152:0/1688075748 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4edc00aa90 con 0x7f4ee414ddd0 2024-04-20T11:37:06.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.891+0000 7f4ee9400640 1 -- 172.21.15.152:0/1688075748 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ee41594a0 con 0x7f4ee414ddd0 2024-04-20T11:37:06.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.893+0000 7f4ee0c00640 1 -- 172.21.15.152:0/1688075748 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) ==== 91754+0+0 (secure 0 0 0) 0x7f4edc00ac30 con 0x7f4ee414ddd0 2024-04-20T11:37:06.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.894+0000 7f4ee0c00640 1 --2- 172.21.15.152:0/1688075748 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4ec006d0d0 0x7f4ec006f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:06.894 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.894+0000 7f4ee0c00640 1 -- 172.21.15.152:0/1688075748 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(154..154 src has 1..154) ==== 5709+0+0 (secure 0 0 0) 0x7f4edc08cde0 con 0x7f4ee414ddd0 2024-04-20T11:37:06.894 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.895+0000 7f4eed7a2740 1 Processor -- start 2024-04-20T11:37:06.894 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.895+0000 7f4eed7a2740 1 -- start start 2024-04-20T11:37:06.894 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.895+0000 7f4ee3e00640 1 --2- 172.21.15.152:0/1688075748 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4ec006d0d0 0x7f4ec006f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:06.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.895+0000 7f4eed7a2740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a427bce670 0x55a427b85ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:06.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.896+0000 7f4eed7a2740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a427b846f0 0x55a427b861e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:06.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.896+0000 7f4eed7a2740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a427b80b90 0x55a427bb0980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:06.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.896+0000 7f4eed7a2740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55a427aec9d0 con 0x55a427b846f0 2024-04-20T11:37:06.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.896+0000 7f4eed7a2740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55a427a93270 con 0x55a427bce670 2024-04-20T11:37:06.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.896+0000 7f4eed7a2740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x55a427ae8990 con 0x55a427b80b90 2024-04-20T11:37:06.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.896+0000 7f4edb400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a427b846f0 0x55a427b861e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:06.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.896+0000 7f4edb400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a427b846f0 0x55a427b861e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:59192/0 (socket says 172.21.15.152:59192) 2024-04-20T11:37:06.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.896+0000 7f4edb400640 1 -- 172.21.15.152:0/2494606332 learned_addr learned my addr 172.21.15.152:0/2494606332 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:06.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.896+0000 7f4ee8a00640 1 --2- 172.21.15.152:0/2494606332 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a427b80b90 0x55a427bb0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:06.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.896+0000 7f4ee3e00640 1 --2- 172.21.15.152:0/2494606332 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a427bce670 0x55a427b85ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:06.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.896+0000 7f4edb400640 1 -- 172.21.15.152:0/2494606332 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a427b80b90 msgr2=0x55a427bb0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:06.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.896+0000 7f4edb400640 1 --2- 172.21.15.152:0/2494606332 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a427b80b90 0x55a427bb0980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:06.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.896+0000 7f4edb400640 1 -- 172.21.15.152:0/2494606332 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a427bce670 msgr2=0x55a427b85ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:06.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.896+0000 7f4edb400640 1 --2- 172.21.15.152:0/2494606332 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a427bce670 0x55a427b85ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:06.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.896+0000 7f4ee3e00640 1 --2- 172.21.15.152:0/1688075748 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4ec006d0d0 0x7f4ec006f590 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f4ed4009a30 tx=0x7f4ed4008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:06.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.896+0000 7f4edb400640 1 -- 172.21.15.152:0/2494606332 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4277ff2d0 con 0x55a427b846f0 2024-04-20T11:37:06.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.897+0000 7f4edb400640 1 --2- 172.21.15.152:0/2494606332 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a427b846f0 0x55a427b861e0 secure :-1 s=READY pgs=2161 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc02b930 tx=0x7f4ecc008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=190e569294725933 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:06.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.897+0000 7f4ed3e00640 1 -- 172.21.15.152:0/2494606332 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ecc035040 con 0x55a427b846f0 2024-04-20T11:37:06.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.897+0000 7f4ed3e00640 1 -- 172.21.15.152:0/2494606332 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ecc0317d0 con 0x55a427b846f0 2024-04-20T11:37:06.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.897+0000 7f4ed3e00640 1 -- 172.21.15.152:0/2494606332 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ecc031b80 con 0x55a427b846f0 2024-04-20T11:37:06.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.898+0000 7f4eed7a2740 1 -- 172.21.15.152:0/2494606332 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a427b846f0 msgr2=0x55a427b861e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:06.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.898+0000 7f4eed7a2740 1 --2- 172.21.15.152:0/2494606332 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a427b846f0 0x55a427b861e0 secure :-1 s=READY pgs=2161 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc02b930 tx=0x7f4ecc008030 comp rx=0 tx=0).stop 2024-04-20T11:37:06.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.898+0000 7f4eed7a2740 1 -- 172.21.15.152:0/2494606332 shutdown_connections 2024-04-20T11:37:06.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.898+0000 7f4eed7a2740 1 --2- 172.21.15.152:0/2494606332 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a427b80b90 0x55a427bb0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:06.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.898+0000 7f4eed7a2740 1 --2- 172.21.15.152:0/2494606332 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a427b846f0 0x55a427b861e0 unknown :-1 s=CLOSED pgs=2161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:06.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.898+0000 7f4eed7a2740 1 --2- 172.21.15.152:0/2494606332 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a427bce670 0x55a427b85ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:06.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.898+0000 7f4eed7a2740 1 -- 172.21.15.152:0/2494606332 >> 172.21.15.152:0/2494606332 conn(0x55a427b82e00 msgr2=0x55a427bcb980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:06.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.898+0000 7f4eed7a2740 1 -- 172.21.15.152:0/2494606332 shutdown_connections 2024-04-20T11:37:06.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.898+0000 7f4eed7a2740 1 -- 172.21.15.152:0/2494606332 wait complete. 2024-04-20T11:37:06.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.899+0000 7f4eed7a2740 1 -- start start 2024-04-20T11:37:06.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.899+0000 7f4eed7a2740 1 Processor -- start 2024-04-20T11:37:06.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.900+0000 7f4eed7a2740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a427b846f0 0x55a427ba6de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:06.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.900+0000 7f4eed7a2740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a427bce670 0x55a427bd2b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:06.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.900+0000 7f4eed7a2740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a427bd30c0 0x55a427bad410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:06.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.900+0000 7f4eed7a2740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55a427b84c00 con 0x55a427b846f0 2024-04-20T11:37:06.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.900+0000 7f4eed7a2740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55a427aa65e0 con 0x55a427bd30c0 2024-04-20T11:37:06.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.900+0000 7f4eed7a2740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x55a427b84ee0 con 0x55a427bce670 2024-04-20T11:37:06.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.900+0000 7f4eed7a2740 10 client.0 _ll_register_callbacks cb 0x55a4277feac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:37:06.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.900+0000 7f4ee3e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a427b846f0 0x55a427ba6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:06.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.900+0000 7f4ee3e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a427b846f0 0x55a427ba6de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:59196/0 (socket says 172.21.15.152:59196) 2024-04-20T11:37:06.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.900+0000 7f4ee3e00640 1 -- 172.21.15.152:0/2091249773 learned_addr learned my addr 172.21.15.152:0/2091249773 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:06.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.900+0000 7f4ee3e00640 1 -- 172.21.15.152:0/2091249773 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a427bce670 msgr2=0x55a427bd2b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:06.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.901+0000 7f4ee3e00640 1 --2- 172.21.15.152:0/2091249773 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a427bce670 0x55a427bd2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:06.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.901+0000 7f4ee3e00640 1 -- 172.21.15.152:0/2091249773 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a427bd30c0 msgr2=0x55a427bad410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:06.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.901+0000 7f4ee3e00640 1 --2- 172.21.15.152:0/2091249773 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a427bd30c0 0x55a427bad410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:06.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.901+0000 7f4ee3e00640 1 -- 172.21.15.152:0/2091249773 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4278c96b0 con 0x55a427b846f0 2024-04-20T11:37:06.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.901+0000 7f4ee3e00640 1 --2- 172.21.15.152:0/2091249773 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a427b846f0 0x55a427ba6de0 secure :-1 s=READY pgs=2162 cs=0 l=1 rev1=1 crypto rx=0x7f4ed40156c0 tx=0x7f4ed40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b7a5630eb7a57c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:06.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.901+0000 7f4ed0c00640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-04-20T11:37:06.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.901+0000 7f4ed0c00640 1 -- 172.21.15.152:0/2091249773 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ed4025070 con 0x55a427b846f0 2024-04-20T11:37:06.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.901+0000 7f4ed0c00640 1 -- 172.21.15.152:0/2091249773 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(154..154 src has 1..154) ==== 5709+0+0 (secure 0 0 0) 0x7f4ed40146b0 con 0x55a427b846f0 2024-04-20T11:37:06.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.901+0000 7f4eed7a2740 1 -- 172.21.15.152:0/2091249773 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ecc0317d0 con 0x55a427b846f0 2024-04-20T11:37:06.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.902+0000 7f4ed0c00640 1 -- 172.21.15.152:0/2091249773 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=105) ==== 24+0+0 (secure 0 0 0) 0x7f4ed4030050 con 0x55a427b846f0 2024-04-20T11:37:06.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.902+0000 7f4eed7a2740 10 client.10152 fetch_fsmap learned FSMap version 105 2024-04-20T11:37:06.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.902+0000 7f4eed7a2740 1 -- 172.21.15.152:0/2091249773 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=105,monmap=2+}) -- 0x55a427bdbd50 con 0x55a427b846f0 2024-04-20T11:37:06.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.902+0000 7f4ed0c00640 1 -- 172.21.15.152:0/2091249773 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ecc0317d0 con 0x55a427b846f0 2024-04-20T11:37:06.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.902+0000 7f4ed0c00640 1 -- 172.21.15.152:0/2091249773 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 105) ==== 3495+0+0 (secure 0 0 0) 0x7f4ed4033020 con 0x55a427b846f0 2024-04-20T11:37:06.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.902+0000 7f4eed7a2740 10 client.10152 fetch_fsmap finished waiting for FSMap version 105 2024-04-20T11:37:06.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.902+0000 7f4eed7a2740 10 client.10152 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-20T11:37:06.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.902+0000 7f4eed7a2740 20 client.10152 populate_metadata read hostname 'smithi152' 2024-04-20T11:37:06.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.902+0000 7f4eed7a2740 1 --2- 172.21.15.152:0/2091249773 >> [v2:172.21.15.152:6832/1854325981,v1:172.21.15.152:6833/1854325981] conn(0x55a427ba9000 0x55a427bd1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:06.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.902+0000 7f4eed7a2740 4 client.10152 mds_command: new command op to 10011 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:37:06.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.902+0000 7f4eed7a2740 1 -- 172.21.15.152:0/2091249773 --> [v2:172.21.15.152:6832/1854325981,v1:172.21.15.152:6833/1854325981] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a427afa3f0 con 0x55a427ba9000 2024-04-20T11:37:06.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.902+0000 7f4edb400640 1 --2- 172.21.15.152:0/2091249773 >> [v2:172.21.15.152:6832/1854325981,v1:172.21.15.152:6833/1854325981] conn(0x55a427ba9000 0x55a427bd1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:06.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.903+0000 7f4edb400640 1 --2- 172.21.15.152:0/2091249773 >> [v2:172.21.15.152:6832/1854325981,v1:172.21.15.152:6833/1854325981] conn(0x55a427ba9000 0x55a427bd1cd0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f555dbd75a54a095 server_cookie=a983d0f37409bbf5 in_seq=0 out_seq=0 2024-04-20T11:37:06.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.903+0000 7f4ed0c00640 10 client.10152 ms_handle_connect on v2:172.21.15.152:6832/1854325981 2024-04-20T11:37:06.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.904+0000 7f4ed0c00640 1 -- 172.21.15.152:0/2091249773 <== mds.0 v2:172.21.15.152:6832/1854325981 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a427afa3f0 con 0x55a427ba9000 2024-04-20T11:37:06.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.904+0000 7f4ed0c00640 10 client.10152 handle_command_reply: tid=0 2024-04-20T11:37:06.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.904+0000 7f4eed7a2740 1 client.10152 shutdown 2024-04-20T11:37:06.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.904+0000 7f4eed7a2740 1 -- 172.21.15.152:0/2091249773 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a427b846f0 msgr2=0x55a427ba6de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:06.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.904+0000 7f4eed7a2740 1 --2- 172.21.15.152:0/2091249773 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a427b846f0 0x55a427ba6de0 secure :-1 s=READY pgs=2162 cs=0 l=1 rev1=1 crypto rx=0x7f4ed40156c0 tx=0x7f4ed40046c0 comp rx=0 tx=0).stop 2024-04-20T11:37:06.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.904+0000 7f4eed7a2740 1 -- 172.21.15.152:0/2091249773 shutdown_connections 2024-04-20T11:37:06.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.904+0000 7f4eed7a2740 1 --2- 172.21.15.152:0/2091249773 >> [v2:172.21.15.152:6832/1854325981,v1:172.21.15.152:6833/1854325981] conn(0x55a427ba9000 0x55a427bd1cd0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:06.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.904+0000 7f4eed7a2740 1 --2- 172.21.15.152:0/2091249773 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a427bd30c0 0x55a427bad410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:06.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.904+0000 7f4eed7a2740 1 --2- 172.21.15.152:0/2091249773 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a427bce670 0x55a427bd2b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:06.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.904+0000 7f4eed7a2740 1 --2- 172.21.15.152:0/2091249773 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a427b846f0 0x55a427ba6de0 unknown :-1 s=CLOSED pgs=2162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:06.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.904+0000 7f4ed0c00640 0 client.10152 ms_handle_reset on v2:172.21.15.152:6832/1854325981 2024-04-20T11:37:06.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.904+0000 7f4eed7a2740 1 -- 172.21.15.152:0/2091249773 >> 172.21.15.152:0/2091249773 conn(0x55a427b80b90 msgr2=0x55a427b867b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:06.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.904+0000 7f4eed7a2740 1 -- 172.21.15.152:0/2091249773 shutdown_connections 2024-04-20T11:37:06.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.904+0000 7f4eed7a2740 1 -- 172.21.15.152:0/2091249773 wait complete. 2024-04-20T11:37:06.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.905+0000 7f4eed7a2740 20 client.10152 trim_cache size 0 max 16384 2024-04-20T11:37:06.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.924+0000 7f4eed7a2740 1 Processor -- start 2024-04-20T11:37:06.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.924+0000 7f4eed7a2740 1 -- start start 2024-04-20T11:37:06.925 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.925+0000 7f4eed7a2740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a427b846f0 0x55a427bad0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:06.925 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.925+0000 7f4eed7a2740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a427bce670 0x55a427b81070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:06.925 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.925+0000 7f4eed7a2740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a427bd30c0 0x55a427ba7920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:06.925 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.925+0000 7f4eed7a2740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee41fb980 con 0x55a427bce670 2024-04-20T11:37:06.925 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.925+0000 7f4eed7a2740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee40ae6a0 con 0x55a427bd30c0 2024-04-20T11:37:06.925 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.925+0000 7f4eed7a2740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4ee40aecf0 con 0x55a427b846f0 2024-04-20T11:37:06.925 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.925+0000 7f4edb400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a427bce670 0x55a427b81070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:06.925 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.925+0000 7f4edb400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a427bce670 0x55a427b81070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:59198/0 (socket says 172.21.15.152:59198) 2024-04-20T11:37:06.925 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.925+0000 7f4edb400640 1 -- 172.21.15.152:0/2161967253 learned_addr learned my addr 172.21.15.152:0/2161967253 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:06.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.926+0000 7f4edb400640 1 -- 172.21.15.152:0/2161967253 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a427b846f0 msgr2=0x55a427bad0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:06.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.926+0000 7f4edb400640 1 --2- 172.21.15.152:0/2161967253 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a427b846f0 0x55a427bad0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:06.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.926+0000 7f4edb400640 1 -- 172.21.15.152:0/2161967253 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a427bd30c0 msgr2=0x55a427ba7920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:06.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.926+0000 7f4edb400640 1 --2- 172.21.15.152:0/2161967253 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a427bd30c0 0x55a427ba7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:06.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.926+0000 7f4edb400640 1 -- 172.21.15.152:0/2161967253 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a427bcb3a0 con 0x55a427bce670 2024-04-20T11:37:06.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.926+0000 7f4edb400640 1 --2- 172.21.15.152:0/2161967253 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a427bce670 0x55a427b81070 secure :-1 s=READY pgs=2163 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc02b930 tx=0x7f4ecc030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1133e21c05b40364 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:06.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.926+0000 7f4ed3400640 1 -- 172.21.15.152:0/2161967253 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ecc054070 con 0x55a427bce670 2024-04-20T11:37:06.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.926+0000 7f4ed3400640 1 -- 172.21.15.152:0/2161967253 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ecc004460 con 0x55a427bce670 2024-04-20T11:37:06.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.926+0000 7f4ed3400640 1 -- 172.21.15.152:0/2161967253 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ecc035040 con 0x55a427bce670 2024-04-20T11:37:06.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.927+0000 7f4eed7a2740 1 -- 172.21.15.152:0/2161967253 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a427bce670 msgr2=0x55a427b81070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:06.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.927+0000 7f4eed7a2740 1 --2- 172.21.15.152:0/2161967253 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a427bce670 0x55a427b81070 secure :-1 s=READY pgs=2163 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc02b930 tx=0x7f4ecc030000 comp rx=0 tx=0).stop 2024-04-20T11:37:06.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.927+0000 7f4eed7a2740 1 -- 172.21.15.152:0/2161967253 shutdown_connections 2024-04-20T11:37:06.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.927+0000 7f4eed7a2740 1 --2- 172.21.15.152:0/2161967253 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a427bd30c0 0x55a427ba7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:06.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.927+0000 7f4eed7a2740 1 --2- 172.21.15.152:0/2161967253 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a427bce670 0x55a427b81070 unknown :-1 s=CLOSED pgs=2163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:06.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.927+0000 7f4eed7a2740 1 --2- 172.21.15.152:0/2161967253 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a427b846f0 0x55a427bad0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:06.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.927+0000 7f4eed7a2740 1 -- 172.21.15.152:0/2161967253 >> 172.21.15.152:0/2161967253 conn(0x55a427b80b90 msgr2=0x55a427ba6bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:06.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.927+0000 7f4eed7a2740 1 -- 172.21.15.152:0/2161967253 shutdown_connections 2024-04-20T11:37:06.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.927+0000 7f4eed7a2740 1 -- 172.21.15.152:0/2161967253 wait complete. 2024-04-20T11:37:06.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.928+0000 7f4eed7a2740 1 -- start start 2024-04-20T11:37:06.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.928+0000 7f4eed7a2740 1 Processor -- start 2024-04-20T11:37:06.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.929+0000 7f4eed7a2740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a427b846f0 0x55a427befbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:06.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.929+0000 7f4eed7a2740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a427bce670 0x55a427c45fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:06.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.929+0000 7f4eed7a2740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a427bd30c0 0x55a427bb43a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:06.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.929+0000 7f4eed7a2740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55a427ba8e80 con 0x55a427bce670 2024-04-20T11:37:06.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.929+0000 7f4eed7a2740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55a427bd3dc0 con 0x55a427bd30c0 2024-04-20T11:37:06.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.929+0000 7f4eed7a2740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x55a427ba9440 con 0x55a427b846f0 2024-04-20T11:37:06.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.929+0000 7f4eed7a2740 10 client.0 _ll_register_callbacks cb 0x7f4ed4009680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:37:06.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.929+0000 7f4edb400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a427bce670 0x55a427c45fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:06.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.929+0000 7f4edb400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a427bce670 0x55a427c45fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:59214/0 (socket says 172.21.15.152:59214) 2024-04-20T11:37:06.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.929+0000 7f4edb400640 1 -- 172.21.15.152:0/573303557 learned_addr learned my addr 172.21.15.152:0/573303557 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:06.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.930+0000 7f4ee3e00640 1 --2- 172.21.15.152:0/573303557 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a427b846f0 0x55a427befbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:06.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.930+0000 7f4edb400640 1 -- 172.21.15.152:0/573303557 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a427b846f0 msgr2=0x55a427befbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:06.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.930+0000 7f4edb400640 1 --2- 172.21.15.152:0/573303557 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a427b846f0 0x55a427befbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:06.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.930+0000 7f4edb400640 1 -- 172.21.15.152:0/573303557 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a427bd30c0 msgr2=0x55a427bb43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:06.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.930+0000 7f4edb400640 1 --2- 172.21.15.152:0/573303557 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a427bd30c0 0x55a427bb43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:06.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.930+0000 7f4edb400640 1 -- 172.21.15.152:0/573303557 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a427b85a60 con 0x55a427bce670 2024-04-20T11:37:06.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.930+0000 7f4ee3e00640 1 --2- 172.21.15.152:0/573303557 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a427b846f0 0x55a427befbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:37:06.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.930+0000 7f4edb400640 1 --2- 172.21.15.152:0/573303557 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a427bce670 0x55a427c45fb0 secure :-1 s=READY pgs=2164 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc004650 tx=0x7f4ecc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86997cd7f342f40c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:06.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.930+0000 7f4ed0c00640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-04-20T11:37:06.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.930+0000 7f4ed0c00640 1 -- 172.21.15.152:0/573303557 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ecc054070 con 0x55a427bce670 2024-04-20T11:37:06.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.930+0000 7f4ed0c00640 1 -- 172.21.15.152:0/573303557 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(154..154 src has 1..154) ==== 5709+0+0 (secure 0 0 0) 0x7f4ecc031b60 con 0x55a427bce670 2024-04-20T11:37:06.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.930+0000 7f4eed7a2740 1 -- 172.21.15.152:0/573303557 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ecc004460 con 0x55a427bce670 2024-04-20T11:37:06.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.931+0000 7f4ed0c00640 1 -- 172.21.15.152:0/573303557 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=105) ==== 24+0+0 (secure 0 0 0) 0x7f4ecc035040 con 0x55a427bce670 2024-04-20T11:37:06.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.931+0000 7f4eed7a2740 10 client.10158 fetch_fsmap learned FSMap version 105 2024-04-20T11:37:06.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.931+0000 7f4eed7a2740 1 -- 172.21.15.152:0/573303557 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=105,monmap=2+}) -- 0x55a427ba54f0 con 0x55a427bce670 2024-04-20T11:37:06.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.931+0000 7f4ed0c00640 1 -- 172.21.15.152:0/573303557 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ecc004460 con 0x55a427bce670 2024-04-20T11:37:06.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.931+0000 7f4ed0c00640 1 -- 172.21.15.152:0/573303557 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 105) ==== 3495+0+0 (secure 0 0 0) 0x7f4ecc04f0d0 con 0x55a427bce670 2024-04-20T11:37:06.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.931+0000 7f4eed7a2740 10 client.10158 fetch_fsmap finished waiting for FSMap version 105 2024-04-20T11:37:06.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.931+0000 7f4eed7a2740 10 client.10158 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-20T11:37:06.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.931+0000 7f4eed7a2740 20 client.10158 populate_metadata read hostname 'smithi152' 2024-04-20T11:37:06.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.931+0000 7f4eed7a2740 1 --2- 172.21.15.152:0/573303557 >> [v2:172.21.15.152:6832/1854325981,v1:172.21.15.152:6833/1854325981] conn(0x55a427c4a2a0 0x55a427c4a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:06.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.931+0000 7f4eed7a2740 4 client.10158 mds_command: new command op to 10011 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-20T11:37:06.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.931+0000 7f4eed7a2740 1 -- 172.21.15.152:0/573303557 --> [v2:172.21.15.152:6832/1854325981,v1:172.21.15.152:6833/1854325981] -- command(tid 0: {"prefix": "status"}) -- 0x55a427af4260 con 0x55a427c4a2a0 2024-04-20T11:37:06.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.931+0000 7f4ee3e00640 1 --2- 172.21.15.152:0/573303557 >> [v2:172.21.15.152:6832/1854325981,v1:172.21.15.152:6833/1854325981] conn(0x55a427c4a2a0 0x55a427c4a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:06.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.932+0000 7f4ee3e00640 1 --2- 172.21.15.152:0/573303557 >> [v2:172.21.15.152:6832/1854325981,v1:172.21.15.152:6833/1854325981] conn(0x55a427c4a2a0 0x55a427c4a680 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e589be61d640aa44 server_cookie=f22507dbf1bda41c in_seq=0 out_seq=0 2024-04-20T11:37:06.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.932+0000 7f4ed0c00640 10 client.10158 ms_handle_connect on v2:172.21.15.152:6832/1854325981 2024-04-20T11:37:06.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.932+0000 7f4ed0c00640 1 -- 172.21.15.152:0/573303557 <== mds.0 v2:172.21.15.152:6832/1854325981 1 ==== command_reply(tid 0: 0 ) ==== 8+0+329 (crc 0 0 0) 0x55a427af4260 con 0x55a427c4a2a0 2024-04-20T11:37:06.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.932+0000 7f4ed0c00640 10 client.10158 handle_command_reply: tid=0 2024-04-20T11:37:06.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.932+0000 7f4eed7a2740 1 client.10158 shutdown 2024-04-20T11:37:06.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.932+0000 7f4eed7a2740 1 -- 172.21.15.152:0/573303557 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a427bce670 msgr2=0x55a427c45fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:06.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.932+0000 7f4eed7a2740 1 --2- 172.21.15.152:0/573303557 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a427bce670 0x55a427c45fb0 secure :-1 s=READY pgs=2164 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc004650 tx=0x7f4ecc004730 comp rx=0 tx=0).stop 2024-04-20T11:37:06.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.932+0000 7f4eed7a2740 1 -- 172.21.15.152:0/573303557 shutdown_connections 2024-04-20T11:37:06.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.932+0000 7f4eed7a2740 1 --2- 172.21.15.152:0/573303557 >> [v2:172.21.15.152:6832/1854325981,v1:172.21.15.152:6833/1854325981] conn(0x55a427c4a2a0 0x55a427c4a680 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:06.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.932+0000 7f4eed7a2740 1 --2- 172.21.15.152:0/573303557 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a427bd30c0 0x55a427bb43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:06.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.932+0000 7f4eed7a2740 1 --2- 172.21.15.152:0/573303557 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a427bce670 0x55a427c45fb0 unknown :-1 s=CLOSED pgs=2164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:06.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.932+0000 7f4eed7a2740 1 --2- 172.21.15.152:0/573303557 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a427b846f0 0x55a427befbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:06.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.932+0000 7f4ed0c00640 0 client.10158 ms_handle_reset on v2:172.21.15.152:6832/1854325981 2024-04-20T11:37:06.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.932+0000 7f4eed7a2740 1 -- 172.21.15.152:0/573303557 >> 172.21.15.152:0/573303557 conn(0x55a427b80b90 msgr2=0x55a427b80fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:06.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.933+0000 7f4eed7a2740 1 -- 172.21.15.152:0/573303557 shutdown_connections 2024-04-20T11:37:06.938 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.933+0000 7f4eed7a2740 1 -- 172.21.15.152:0/573303557 wait complete. 2024-04-20T11:37:06.938 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.933+0000 7f4eed7a2740 20 client.10158 trim_cache size 0 max 16384 2024-04-20T11:37:06.938 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-04-20T11:37:06.938 INFO:teuthology.orchestra.run.smithi152.stdout: "cluster_fsid": "c8a72131-0d4f-42c7-892f-3f3af4d2bd3d", 2024-04-20T11:37:06.938 INFO:teuthology.orchestra.run.smithi152.stdout: "whoami": 0, 2024-04-20T11:37:06.938 INFO:teuthology.orchestra.run.smithi152.stdout: "id": 10011, 2024-04-20T11:37:06.938 INFO:teuthology.orchestra.run.smithi152.stdout: "want_state": "up:active", 2024-04-20T11:37:06.938 INFO:teuthology.orchestra.run.smithi152.stdout: "state": "up:active", 2024-04-20T11:37:06.938 INFO:teuthology.orchestra.run.smithi152.stdout: "fs_name": "cephfs", 2024-04-20T11:37:06.939 INFO:teuthology.orchestra.run.smithi152.stdout: "rank_uptime": 6.6028163720000004, 2024-04-20T11:37:06.939 INFO:teuthology.orchestra.run.smithi152.stdout: "mdsmap_epoch": 105, 2024-04-20T11:37:06.939 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_epoch": 154, 2024-04-20T11:37:06.939 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_epoch_barrier": 154, 2024-04-20T11:37:06.939 INFO:teuthology.orchestra.run.smithi152.stdout: "uptime": 13.055782441 2024-04-20T11:37:06.939 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-04-20T11:37:06.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.933+0000 7f4ee9400640 1 -- 172.21.15.152:0/1688075748 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4ec006d0d0 msgr2=0x7f4ec006f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:06.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.933+0000 7f4ee9400640 1 --2- 172.21.15.152:0/1688075748 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4ec006d0d0 0x7f4ec006f590 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f4ed4009a30 tx=0x7f4ed4008040 comp rx=0 tx=0).stop 2024-04-20T11:37:06.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.933+0000 7f4ee9400640 1 -- 172.21.15.152:0/1688075748 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ee414ddd0 msgr2=0x7f4ee4159f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:06.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.933+0000 7f4ee9400640 1 --2- 172.21.15.152:0/1688075748 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ee414ddd0 0x7f4ee4159f80 secure :-1 s=READY pgs=2160 cs=0 l=1 rev1=1 crypto rx=0x7f4edc00c7d0 tx=0x7f4edc00cc90 comp rx=0 tx=0).stop 2024-04-20T11:37:06.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.933+0000 7f4ee9400640 1 -- 172.21.15.152:0/1688075748 shutdown_connections 2024-04-20T11:37:06.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.933+0000 7f4ee9400640 1 --2- 172.21.15.152:0/1688075748 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4ec006d0d0 0x7f4ec006f590 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:06.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.933+0000 7f4ee9400640 1 --2- 172.21.15.152:0/1688075748 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ee414ddd0 0x7f4ee4159f80 unknown :-1 s=CLOSED pgs=2160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:06.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.933+0000 7f4ee9400640 1 --2- 172.21.15.152:0/1688075748 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4ee40b4160 0x7f4ee4158720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:06.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.933+0000 7f4ee9400640 1 --2- 172.21.15.152:0/1688075748 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4ee4073b60 0x7f4ee41e49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:06.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.933+0000 7f4ee9400640 1 -- 172.21.15.152:0/1688075748 >> 172.21.15.152:0/1688075748 conn(0x7f4ee40d0260 msgr2=0x7f4ee40b0b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:06.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.934+0000 7f4ee9400640 1 -- 172.21.15.152:0/1688075748 shutdown_connections 2024-04-20T11:37:06.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:06.934+0000 7f4ee9400640 1 -- 172.21.15.152:0/1688075748 wait complete. 2024-04-20T11:37:06.950 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-20T11:37:07.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:07.144+0000 7efed3200640 1 Processor -- start 2024-04-20T11:37:07.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:07.145+0000 7efed3200640 1 -- start start 2024-04-20T11:37:07.146 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:07.146+0000 7efed3200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7efed40afa50 0x7efed406e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:07.146 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:07.146+0000 7efed3200640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efed40aec90 con 0x7efed40723e0 2024-04-20T11:37:07.146 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:07.146+0000 7efed3200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efed40aef70 con 0x7efed4147410 2024-04-20T11:37:07.146 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:07.146+0000 7efed3200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7efed40af250 con 0x7efed40afa50 2024-04-20T11:37:07.146 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:07.147+0000 7efed2800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7efed40afa50 0x7efed406e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:07.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:07.147+0000 7efecbe00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7efed40723e0 0x7efed406e010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:43914/0 (socket says 172.21.15.152:43914) 2024-04-20T11:37:07.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:07.147+0000 7efed2800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7efed40afa50 0x7efed406e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:59226/0 (socket says 172.21.15.152:59226) 2024-04-20T11:37:07.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:07.147+0000 7efecbe00640 1 -- 172.21.15.152:0/2450125357 learned_addr learned my addr 172.21.15.152:0/2450125357 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:37:07.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:07.147+0000 7efed2800640 1 -- 172.21.15.152:0/2450125357 >> v1:172.21.15.175:6789/0 conn(0x7efed4147410 legacy=0x7efed4071cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:07.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:07.147+0000 7efed2800640 1 -- 172.21.15.152:0/2450125357 >> v1:172.21.15.152:6789/0 conn(0x7efed40723e0 legacy=0x7efed406e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:07.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:07.147+0000 7efed1400640 1 -- 172.21.15.152:0/2450125357 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2206140263 0 0) 0x7efed40aec90 con 0x7efed40723e0 2024-04-20T11:37:07.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:07.147+0000 7efed2800640 1 -- 172.21.15.152:0/2450125357 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efed40af530 con 0x7efed40afa50 2024-04-20T11:37:07.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:07.148+0000 7efed2800640 1 --2- 172.21.15.152:0/2450125357 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efed40afa50 0x7efed406e720 secure :-1 s=READY pgs=2166 cs=0 l=1 rev1=1 crypto rx=0x7efecc009d90 tx=0x7efecc02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32d5c33514af43c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:07.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:07.149+0000 7efed1400640 1 -- 172.21.15.152:0/2450125357 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efecc039070 con 0x7efed40afa50 2024-04-20T11:37:07.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:07.149+0000 7efed1400640 1 -- 172.21.15.152:0/2450125357 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efecc02cc60 con 0x7efed40afa50 2024-04-20T11:37:07.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:07.149+0000 7efed1400640 1 -- 172.21.15.152:0/2450125357 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efecc034660 con 0x7efed40afa50 2024-04-20T11:37:07.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:07.149+0000 7efed3200640 1 -- 172.21.15.152:0/2450125357 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efed40afa50 msgr2=0x7efed406e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:07.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:07.149+0000 7efed3200640 1 --2- 172.21.15.152:0/2450125357 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efed40afa50 0x7efed406e720 secure :-1 s=READY pgs=2166 cs=0 l=1 rev1=1 crypto rx=0x7efecc009d90 tx=0x7efecc02beb0 comp rx=0 tx=0).stop 2024-04-20T11:37:07.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:07.149+0000 7efed3200640 1 -- 172.21.15.152:0/2450125357 shutdown_connections 2024-04-20T11:37:07.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:07.149+0000 7efed3200640 1 --2- 172.21.15.152:0/2450125357 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efed40afa50 0x7efed406e720 unknown :-1 s=CLOSED pgs=2166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:07.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:07.149+0000 7efed3200640 1 -- 172.21.15.152:0/2450125357 >> 172.21.15.152:0/2450125357 conn(0x7efed4067ae0 msgr2=0x7efed406b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:07.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:07.149+0000 7efed3200640 1 -- 172.21.15.152:0/2450125357 shutdown_connections 2024-04-20T11:37:07.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:07.150+0000 7efed3200640 1 -- 172.21.15.152:0/2450125357 wait complete. 2024-04-20T11:37:07.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:07.151+0000 7efed3200640 1 Processor -- start 2024-04-20T11:37:07.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:07.151+0000 7efed3200640 1 -- start start 2024-04-20T11:37:07.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:07.152+0000 7efed3200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efed40723e0 0x7efed415d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:07.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:07.152+0000 7efed3200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efed40afa50 0x7efed413db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:07.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:07.152+0000 7efed3200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efed4147410 0x7efed413e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:07.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:07.152+0000 7efed3200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7efed40b3510 con 0x7efed4147410 2024-04-20T11:37:07.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:07.152+0000 7efed3200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7efed40b3390 con 0x7efed40723e0 2024-04-20T11:37:07.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:07.152+0000 7efed3200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7efed40b3690 con 0x7efed40afa50 2024-04-20T11:37:07.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:07.152+0000 7efed2800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efed4147410 0x7efed413e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:07.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:07.153+0000 7efed2800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efed4147410 0x7efed413e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:59230/0 (socket says 172.21.15.152:59230) 2024-04-20T11:37:07.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:07.153+0000 7efed2800640 1 -- 172.21.15.152:0/3135596672 learned_addr learned my addr 172.21.15.152:0/3135596672 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:07.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:07.153+0000 7efecbe00640 1 --2- 172.21.15.152:0/3135596672 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efed40afa50 0x7efed413db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:07.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:07.153+0000 7efed1e00640 1 --2- 172.21.15.152:0/3135596672 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efed40723e0 0x7efed415d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:07.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:07.153+0000 7efed2800640 1 -- 172.21.15.152:0/3135596672 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efed40afa50 msgr2=0x7efed413db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:07.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:07.153+0000 7efed2800640 1 --2- 172.21.15.152:0/3135596672 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efed40afa50 0x7efed413db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:07.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:07.153+0000 7efed2800640 1 -- 172.21.15.152:0/3135596672 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efed40723e0 msgr2=0x7efed415d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:07.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:07.153+0000 7efed2800640 1 --2- 172.21.15.152:0/3135596672 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efed40723e0 0x7efed415d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:07.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:07.153+0000 7efed2800640 1 -- 172.21.15.152:0/3135596672 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efed4143ee0 con 0x7efed4147410 2024-04-20T11:37:07.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:07.153+0000 7efecbe00640 1 --2- 172.21.15.152:0/3135596672 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efed40afa50 0x7efed413db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:37:07.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:07.153+0000 7efed1e00640 1 --2- 172.21.15.152:0/3135596672 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efed40723e0 0x7efed415d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:37:07.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:07.153+0000 7efed2800640 1 --2- 172.21.15.152:0/3135596672 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efed4147410 0x7efed413e0a0 secure :-1 s=READY pgs=2167 cs=0 l=1 rev1=1 crypto rx=0x7efecc009d90 tx=0x7efecc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:07.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:07.154+0000 7efeca000640 1 -- 172.21.15.152:0/3135596672 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efecc039050 con 0x7efed4147410 2024-04-20T11:37:07.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:07.154+0000 7efeca000640 1 -- 172.21.15.152:0/3135596672 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efecc03a040 con 0x7efed4147410 2024-04-20T11:37:07.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:07.154+0000 7efeca000640 1 -- 172.21.15.152:0/3135596672 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efecc03e610 con 0x7efed4147410 2024-04-20T11:37:07.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:07.154+0000 7efed3200640 1 -- 172.21.15.152:0/3135596672 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efed41441c0 con 0x7efed4147410 2024-04-20T11:37:07.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:07.154+0000 7efed3200640 1 -- 172.21.15.152:0/3135596672 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efed415e2e0 con 0x7efed4147410 2024-04-20T11:37:07.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:07.155+0000 7efed3200640 1 -- 172.21.15.152:0/3135596672 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efed41446e0 con 0x7efed4147410 2024-04-20T11:37:07.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:07.156+0000 7efeca000640 1 -- 172.21.15.152:0/3135596672 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) ==== 91754+0+0 (secure 0 0 0) 0x7efecc047030 con 0x7efed4147410 2024-04-20T11:37:07.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:07.157+0000 7efeca000640 1 --2- 172.21.15.152:0/3135596672 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7efeb006d100 0x7efeb006f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:07.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:07.157+0000 7efeca000640 1 -- 172.21.15.152:0/3135596672 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(154..154 src has 1..154) ==== 5709+0+0 (secure 0 0 0) 0x7efecc0aeeb0 con 0x7efed4147410 2024-04-20T11:37:07.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:07.157+0000 7efed1e00640 1 --2- 172.21.15.152:0/3135596672 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7efeb006d100 0x7efeb006f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:07.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:07.158+0000 7efed1e00640 1 --2- 172.21.15.152:0/3135596672 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7efeb006d100 0x7efeb006f5c0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7efed413cac0 tx=0x7efec4009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:07.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:07.161+0000 7efeca000640 1 -- 172.21.15.152:0/3135596672 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7efecc07c410 con 0x7efed4147410 2024-04-20T11:37:07.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:07.406+0000 7efed3200640 1 -- 172.21.15.152:0/3135596672 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7efed4075e60 con 0x7efed4147410 2024-04-20T11:37:07.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:07.406+0000 7efeca000640 1 -- 172.21.15.152:0/3135596672 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v105) ==== 53+0+73 (secure 0 0 0) 0x7efecc07bd10 con 0x7efed4147410 2024-04-20T11:37:07.413 INFO:teuthology.orchestra.run.smithi152.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-20T11:37:07.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:07.409+0000 7efed3200640 1 -- 172.21.15.152:0/3135596672 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7efeb006d100 msgr2=0x7efeb006f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:07.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:07.409+0000 7efed3200640 1 --2- 172.21.15.152:0/3135596672 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7efeb006d100 0x7efeb006f5c0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7efed413cac0 tx=0x7efec4009040 comp rx=0 tx=0).stop 2024-04-20T11:37:07.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:07.409+0000 7efed3200640 1 -- 172.21.15.152:0/3135596672 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efed4147410 msgr2=0x7efed413e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:07.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:07.409+0000 7efed3200640 1 --2- 172.21.15.152:0/3135596672 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efed4147410 0x7efed413e0a0 secure :-1 s=READY pgs=2167 cs=0 l=1 rev1=1 crypto rx=0x7efecc009d90 tx=0x7efecc004a40 comp rx=0 tx=0).stop 2024-04-20T11:37:07.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:07.409+0000 7efed3200640 1 -- 172.21.15.152:0/3135596672 shutdown_connections 2024-04-20T11:37:07.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:07.409+0000 7efed3200640 1 --2- 172.21.15.152:0/3135596672 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7efeb006d100 0x7efeb006f5c0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:07.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:07.409+0000 7efed3200640 1 --2- 172.21.15.152:0/3135596672 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efed4147410 0x7efed413e0a0 unknown :-1 s=CLOSED pgs=2167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:07.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:07.409+0000 7efed3200640 1 --2- 172.21.15.152:0/3135596672 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efed40afa50 0x7efed413db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:07.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:07.409+0000 7efed3200640 1 --2- 172.21.15.152:0/3135596672 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efed40723e0 0x7efed415d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:07.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:07.409+0000 7efed3200640 1 -- 172.21.15.152:0/3135596672 >> 172.21.15.152:0/3135596672 conn(0x7efed4067ae0 msgr2=0x7efed4140cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:07.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:07.409+0000 7efed3200640 1 -- 172.21.15.152:0/3135596672 shutdown_connections 2024-04-20T11:37:07.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:07.409+0000 7efed3200640 1 -- 172.21.15.152:0/3135596672 wait complete. 2024-04-20T11:37:07.429 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-20T11:37:07.429 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-20T11:37:07.429 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi152.front.sepia.ceph.com 2024-04-20T11:37:07.429 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-20T11:37:07.430 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-20T11:37:07.430 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-20T11:37:07.430 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-20T11:37:07.430 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-20T11:37:07.430 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-20T11:37:07.430 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:37:07.430 DEBUG:teuthology.orchestra.run.smithi152:> ip addr 2024-04-20T11:37:07.491 INFO:teuthology.orchestra.run.smithi152.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-20T11:37:07.492 INFO:teuthology.orchestra.run.smithi152.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-20T11:37:07.492 INFO:teuthology.orchestra.run.smithi152.stdout: inet 127.0.0.1/8 scope host lo 2024-04-20T11:37:07.492 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:37:07.493 INFO:teuthology.orchestra.run.smithi152.stdout: inet6 ::1/128 scope host 2024-04-20T11:37:07.493 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:37:07.493 INFO:teuthology.orchestra.run.smithi152.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T11:37:07.493 INFO:teuthology.orchestra.run.smithi152.stdout: link/ether 0c:c4:7a:da:6c:44 brd ff:ff:ff:ff:ff:ff 2024-04-20T11:37:07.493 INFO:teuthology.orchestra.run.smithi152.stdout: altname enp7s0f0 2024-04-20T11:37:07.494 INFO:teuthology.orchestra.run.smithi152.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T11:37:07.494 INFO:teuthology.orchestra.run.smithi152.stdout: link/ether 0c:c4:7a:da:6c:45 brd ff:ff:ff:ff:ff:ff 2024-04-20T11:37:07.494 INFO:teuthology.orchestra.run.smithi152.stdout: altname enp7s0f1 2024-04-20T11:37:07.495 INFO:teuthology.orchestra.run.smithi152.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T11:37:07.495 INFO:teuthology.orchestra.run.smithi152.stdout: link/ether 0c:c4:7a:88:6d:e2 brd ff:ff:ff:ff:ff:ff 2024-04-20T11:37:07.495 INFO:teuthology.orchestra.run.smithi152.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-20T11:37:07.495 INFO:teuthology.orchestra.run.smithi152.stdout: link/ether 0c:c4:7a:88:6d:e3 brd ff:ff:ff:ff:ff:ff 2024-04-20T11:37:07.495 INFO:teuthology.orchestra.run.smithi152.stdout: inet 172.21.15.152/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-20T11:37:07.496 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft 42179sec preferred_lft 42179sec 2024-04-20T11:37:07.496 INFO:teuthology.orchestra.run.smithi152.stdout: inet6 fe80::ec4:7aff:fe88:6de3/64 scope link 2024-04-20T11:37:07.496 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:37:07.496 INFO:teuthology.orchestra.run.smithi152.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-20T11:37:07.496 INFO:teuthology.orchestra.run.smithi152.stdout: link/ether 06:3e:9a:d3:49:1e brd ff:ff:ff:ff:ff:ff 2024-04-20T11:37:07.496 INFO:teuthology.orchestra.run.smithi152.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-20T11:37:07.496 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:37:07.496 INFO:teuthology.orchestra.run.smithi152.stdout: inet6 fe80::a019:d8ff:fe70:1185/64 scope link 2024-04-20T11:37:07.496 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:37:07.498 INFO:teuthology.orchestra.run.smithi152.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-20T11:37:07.498 INFO:teuthology.orchestra.run.smithi152.stdout: link/ether 16:e5:08:78:20:47 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T11:37:07.498 INFO:teuthology.orchestra.run.smithi152.stdout: inet6 fe80::14e5:8ff:fe78:2047/64 scope link 2024-04-20T11:37:07.498 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:37:07.499 INFO:teuthology.orchestra.run.smithi152.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-20T11:37:07.500 INFO:teuthology.orchestra.run.smithi152.stdout: link/ether 06:3e:9a:d3:49:1e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T11:37:07.501 INFO:teuthology.orchestra.run.smithi152.stdout: inet6 fe80::43e:9aff:fed3:491e/64 scope link 2024-04-20T11:37:07.501 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:37:07.501 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:37:07.501 DEBUG:teuthology.orchestra.run.smithi152:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-20T11:37:07.580 INFO:teuthology.orchestra.run.smithi152.stdout:1 2024-04-20T11:37:07.584 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:37:07.585 DEBUG:teuthology.orchestra.run.smithi152:> route 2024-04-20T11:37:07.639 INFO:teuthology.orchestra.run.smithi152.stdout:Kernel IP routing table 2024-04-20T11:37:07.639 INFO:teuthology.orchestra.run.smithi152.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-20T11:37:07.640 INFO:teuthology.orchestra.run.smithi152.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-20T11:37:07.640 INFO:teuthology.orchestra.run.smithi152.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-20T11:37:07.640 INFO:teuthology.orchestra.run.smithi152.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-20T11:37:07.641 DEBUG:teuthology.orchestra.run.smithi152:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T11:37:07.641 DEBUG:teuthology.orchestra.run.smithi152:> set -e 2024-04-20T11:37:07.641 DEBUG:teuthology.orchestra.run.smithi152:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-20T11:37:07.641 DEBUG:teuthology.orchestra.run.smithi152:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-20T11:37:07.641 DEBUG:teuthology.orchestra.run.smithi152:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-20T11:37:07.641 DEBUG:teuthology.orchestra.run.smithi152:> ') 2024-04-20T11:37:07.748 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T11:37:07.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T11:37:07.852 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:37:07.852 DEBUG:teuthology.orchestra.run.smithi152:> ip netns list 2024-04-20T11:37:07.905 INFO:teuthology.orchestra.run.smithi152.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-20T11:37:07.905 INFO:teuthology.orchestra.run.smithi152.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-20T11:37:07.905 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-20T11:37:07.905 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:37:07.905 DEBUG:teuthology.orchestra.run.smithi152:> sudo ip link set brx.0 up 2024-04-20T11:37:07.989 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-20T11:37:07.989 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T11:37:07.989 DEBUG:teuthology.orchestra.run.smithi152:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-20T11:37:08.044 INFO:teuthology.orchestra.run.smithi152.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-20T11:37:08.044 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T11:37:08.044 DEBUG:teuthology.orchestra.run.smithi152:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-20T11:37:08.098 DEBUG:teuthology.orchestra.run.smithi152:> sudo modprobe fuse 2024-04-20T11:37:08.185 DEBUG:teuthology.orchestra.run.smithi152:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T11:37:08.238 INFO:teuthology.orchestra.run.smithi152.stdout:/proc 2024-04-20T11:37:08.238 INFO:teuthology.orchestra.run.smithi152.stdout:/sys 2024-04-20T11:37:08.238 INFO:teuthology.orchestra.run.smithi152.stdout:/dev 2024-04-20T11:37:08.238 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/security 2024-04-20T11:37:08.238 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/shm 2024-04-20T11:37:08.238 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/pts 2024-04-20T11:37:08.238 INFO:teuthology.orchestra.run.smithi152.stdout:/run 2024-04-20T11:37:08.238 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/cgroup 2024-04-20T11:37:08.238 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/pstore 2024-04-20T11:37:08.238 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/bpf 2024-04-20T11:37:08.238 INFO:teuthology.orchestra.run.smithi152.stdout:/ 2024-04-20T11:37:08.239 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/selinux 2024-04-20T11:37:08.239 INFO:teuthology.orchestra.run.smithi152.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T11:37:08.239 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/mqueue 2024-04-20T11:37:08.239 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/hugepages 2024-04-20T11:37:08.239 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/debug 2024-04-20T11:37:08.239 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/tracing 2024-04-20T11:37:08.239 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/fuse/connections 2024-04-20T11:37:08.239 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/config 2024-04-20T11:37:08.239 INFO:teuthology.orchestra.run.smithi152.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T11:37:08.239 INFO:teuthology.orchestra.run.smithi152.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T11:37:08.239 INFO:teuthology.orchestra.run.smithi152.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T11:37:08.239 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T11:37:08.239 INFO:teuthology.orchestra.run.smithi152.stdout:/run/user/1000 2024-04-20T11:37:08.239 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph 2024-04-20T11:37:08.240 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph/osd/ceph-0 2024-04-20T11:37:08.240 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph/osd/ceph-1 2024-04-20T11:37:08.240 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph/osd/ceph-2 2024-04-20T11:37:08.240 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph/osd/ceph-3 2024-04-20T11:37:08.240 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns 2024-04-20T11:37:08.240 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T11:37:08.240 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T11:37:08.240 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T11:37:08.240 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T11:37:08.241 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:37:08.241 DEBUG:teuthology.orchestra.run.smithi152:> ls /sys/fs/fuse/connections 2024-04-20T11:37:08.294 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-20T11:37:08.294 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-20T11:37:08.299 DEBUG:teuthology.orchestra.run.smithi152:> sudo modprobe fuse 2024-04-20T11:37:08.377 DEBUG:teuthology.orchestra.run.smithi152:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T11:37:08.419 INFO:teuthology.orchestra.run.smithi152.stdout:/proc 2024-04-20T11:37:08.419 INFO:teuthology.orchestra.run.smithi152.stdout:/sys 2024-04-20T11:37:08.419 INFO:teuthology.orchestra.run.smithi152.stdout:/dev 2024-04-20T11:37:08.419 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/security 2024-04-20T11:37:08.419 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/shm 2024-04-20T11:37:08.419 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/pts 2024-04-20T11:37:08.419 INFO:teuthology.orchestra.run.smithi152.stdout:/run 2024-04-20T11:37:08.419 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/cgroup 2024-04-20T11:37:08.419 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/pstore 2024-04-20T11:37:08.420 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/bpf 2024-04-20T11:37:08.420 INFO:teuthology.orchestra.run.smithi152.stdout:/ 2024-04-20T11:37:08.420 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/selinux 2024-04-20T11:37:08.420 INFO:teuthology.orchestra.run.smithi152.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T11:37:08.420 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/mqueue 2024-04-20T11:37:08.420 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/hugepages 2024-04-20T11:37:08.420 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/debug 2024-04-20T11:37:08.420 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/tracing 2024-04-20T11:37:08.420 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/fuse/connections 2024-04-20T11:37:08.420 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/config 2024-04-20T11:37:08.420 INFO:teuthology.orchestra.run.smithi152.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T11:37:08.420 INFO:teuthology.orchestra.run.smithi152.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T11:37:08.420 INFO:teuthology.orchestra.run.smithi152.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T11:37:08.420 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T11:37:08.421 INFO:teuthology.orchestra.run.smithi152.stdout:/run/user/1000 2024-04-20T11:37:08.421 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph 2024-04-20T11:37:08.421 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph/osd/ceph-0 2024-04-20T11:37:08.421 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph/osd/ceph-1 2024-04-20T11:37:08.421 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph/osd/ceph-2 2024-04-20T11:37:08.421 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph/osd/ceph-3 2024-04-20T11:37:08.421 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns 2024-04-20T11:37:08.421 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T11:37:08.421 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T11:37:08.421 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T11:37:08.421 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T11:37:08.422 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:37:08.422 DEBUG:teuthology.orchestra.run.smithi152:> ls /sys/fs/fuse/connections 2024-04-20T11:37:08.475 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi152.stderr:2024-04-20T11:37:08.475+0000 7fcb7d437480 -1 init, newargv = 0x55ab8e62c500 newargc=15 2024-04-20T11:37:08.475 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi152.stderr:ceph-fuse[71523]: starting ceph client 2024-04-20T11:37:08.484 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi152.stderr:ceph-fuse[71523]: starting fuse 2024-04-20T11:37:08.504 INFO:teuthology.orchestra.run.smithi152.stdout:42 2024-04-20T11:37:08.504 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-04-20T11:37:08.504 DEBUG:teuthology.orchestra.run.smithi152:> sudo stdin-killer -- python3 -c ' 2024-04-20T11:37:08.504 DEBUG:teuthology.orchestra.run.smithi152:> import glob 2024-04-20T11:37:08.504 DEBUG:teuthology.orchestra.run.smithi152:> import re 2024-04-20T11:37:08.504 DEBUG:teuthology.orchestra.run.smithi152:> import os 2024-04-20T11:37:08.504 DEBUG:teuthology.orchestra.run.smithi152:> import subprocess 2024-04-20T11:37:08.504 DEBUG:teuthology.orchestra.run.smithi152:> 2024-04-20T11:37:08.505 DEBUG:teuthology.orchestra.run.smithi152:> def _find_admin_socket(client_name): 2024-04-20T11:37:08.505 DEBUG:teuthology.orchestra.run.smithi152:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-20T11:37:08.505 DEBUG:teuthology.orchestra.run.smithi152:> files = glob.glob(asok_path) 2024-04-20T11:37:08.505 DEBUG:teuthology.orchestra.run.smithi152:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-20T11:37:08.505 DEBUG:teuthology.orchestra.run.smithi152:> 2024-04-20T11:37:08.505 DEBUG:teuthology.orchestra.run.smithi152:> # Given a non-glob path, it better be there 2024-04-20T11:37:08.505 DEBUG:teuthology.orchestra.run.smithi152:> if "*" not in asok_path: 2024-04-20T11:37:08.505 DEBUG:teuthology.orchestra.run.smithi152:> assert(len(files) == 1) 2024-04-20T11:37:08.505 DEBUG:teuthology.orchestra.run.smithi152:> return files[0] 2024-04-20T11:37:08.505 DEBUG:teuthology.orchestra.run.smithi152:> 2024-04-20T11:37:08.505 DEBUG:teuthology.orchestra.run.smithi152:> for f in files: 2024-04-20T11:37:08.505 DEBUG:teuthology.orchestra.run.smithi152:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-20T11:37:08.505 DEBUG:teuthology.orchestra.run.smithi152:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-20T11:37:08.505 DEBUG:teuthology.orchestra.run.smithi152:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-20T11:37:08.505 DEBUG:teuthology.orchestra.run.smithi152:> contents = proc_f.read() 2024-04-20T11:37:08.505 DEBUG:teuthology.orchestra.run.smithi152:> if mountpoint in contents: 2024-04-20T11:37:08.506 DEBUG:teuthology.orchestra.run.smithi152:> return f 2024-04-20T11:37:08.506 DEBUG:teuthology.orchestra.run.smithi152:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-20T11:37:08.506 DEBUG:teuthology.orchestra.run.smithi152:> 2024-04-20T11:37:08.506 DEBUG:teuthology.orchestra.run.smithi152:> print(_find_admin_socket("client.0")) 2024-04-20T11:37:08.506 DEBUG:teuthology.orchestra.run.smithi152:> ' 2024-04-20T11:37:08.704 INFO:teuthology.orchestra.run.smithi152.stdout:/var/run/ceph/ceph-client.0.71523.asok 2024-04-20T11:37:08.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T11:37:08.722 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.71523.asok 2024-04-20T11:37:08.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:37:08.722 DEBUG:teuthology.orchestra.run.smithi152:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.71523.asok status 2024-04-20T11:37:08.948 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-04-20T11:37:08.949 INFO:teuthology.orchestra.run.smithi152.stdout: "metadata": { 2024-04-20T11:37:08.949 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_sha1": "a8c84f5997950fd7d008204839253291d52e6eda", 2024-04-20T11:37:08.949 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", 2024-04-20T11:37:08.949 INFO:teuthology.orchestra.run.smithi152.stdout: "entity_id": "0", 2024-04-20T11:37:08.949 INFO:teuthology.orchestra.run.smithi152.stdout: "hostname": "smithi152", 2024-04-20T11:37:08.949 INFO:teuthology.orchestra.run.smithi152.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-20T11:37:08.949 INFO:teuthology.orchestra.run.smithi152.stdout: "pid": "71523", 2024-04-20T11:37:08.950 INFO:teuthology.orchestra.run.smithi152.stdout: "root": "/" 2024-04-20T11:37:08.950 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:37:08.950 INFO:teuthology.orchestra.run.smithi152.stdout: "dentry_count": 0, 2024-04-20T11:37:08.950 INFO:teuthology.orchestra.run.smithi152.stdout: "dentry_pinned_count": 0, 2024-04-20T11:37:08.950 INFO:teuthology.orchestra.run.smithi152.stdout: "id": 10173, 2024-04-20T11:37:08.950 INFO:teuthology.orchestra.run.smithi152.stdout: "inst": { 2024-04-20T11:37:08.950 INFO:teuthology.orchestra.run.smithi152.stdout: "name": { 2024-04-20T11:37:08.950 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "client", 2024-04-20T11:37:08.951 INFO:teuthology.orchestra.run.smithi152.stdout: "num": 10173 2024-04-20T11:37:08.951 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:37:08.951 INFO:teuthology.orchestra.run.smithi152.stdout: "addr": { 2024-04-20T11:37:08.951 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "v1", 2024-04-20T11:37:08.951 INFO:teuthology.orchestra.run.smithi152.stdout: "addr": "192.168.0.1:0", 2024-04-20T11:37:08.951 INFO:teuthology.orchestra.run.smithi152.stdout: "nonce": 3635310914 2024-04-20T11:37:08.951 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:37:08.951 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:37:08.951 INFO:teuthology.orchestra.run.smithi152.stdout: "addr": { 2024-04-20T11:37:08.952 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "v1", 2024-04-20T11:37:08.952 INFO:teuthology.orchestra.run.smithi152.stdout: "addr": "192.168.0.1:0", 2024-04-20T11:37:08.952 INFO:teuthology.orchestra.run.smithi152.stdout: "nonce": 3635310914 2024-04-20T11:37:08.952 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:37:08.952 INFO:teuthology.orchestra.run.smithi152.stdout: "inst_str": "client.10173 192.168.0.1:0/3635310914", 2024-04-20T11:37:08.952 INFO:teuthology.orchestra.run.smithi152.stdout: "addr_str": "192.168.0.1:0/3635310914", 2024-04-20T11:37:08.952 INFO:teuthology.orchestra.run.smithi152.stdout: "inode_count": 1, 2024-04-20T11:37:08.952 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_epoch": 105, 2024-04-20T11:37:08.952 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_epoch": 154, 2024-04-20T11:37:08.953 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_epoch_barrier": 0, 2024-04-20T11:37:08.953 INFO:teuthology.orchestra.run.smithi152.stdout: "blocklisted": false, 2024-04-20T11:37:08.953 INFO:teuthology.orchestra.run.smithi152.stdout: "fs_name": "cephfs" 2024-04-20T11:37:08.953 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-04-20T11:37:08.964 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:37:08.964 DEBUG:teuthology.orchestra.run.smithi152:> stat --file-system '--printf=%T 2024-04-20T11:37:08.964 DEBUG:teuthology.orchestra.run.smithi152:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-20T11:37:09.017 INFO:teuthology.orchestra.run.smithi152.stdout:fuseblk 2024-04-20T11:37:09.017 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-20T11:37:09.017 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:37:09.018 DEBUG:teuthology.orchestra.run.smithi152:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-20T11:37:09.102 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-20T11:37:09.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:09.303+0000 7fd1bfa00640 1 Processor -- start 2024-04-20T11:37:09.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:09.303+0000 7fd1bfa00640 1 -- start start 2024-04-20T11:37:09.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:09.305+0000 7fd1bfa00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd1b80b4500 0x7fd1b80d0100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:09.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:09.305+0000 7fd1bfa00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1b815fd20 con 0x7fd1b80d0640 2024-04-20T11:37:09.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:09.305+0000 7fd1bfa00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1b815ef60 con 0x7fd1b815c560 2024-04-20T11:37:09.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:09.305+0000 7fd1bfa00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd1b815f240 con 0x7fd1b80b4500 2024-04-20T11:37:09.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:09.305+0000 7fd1bda00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fd1b80d0640 0x7fd1b80ada90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:43928/0 (socket says 172.21.15.152:43928) 2024-04-20T11:37:09.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:09.305+0000 7fd1bda00640 1 -- 172.21.15.152:0/743939284 learned_addr learned my addr 172.21.15.152:0/743939284 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:37:09.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:09.305+0000 7fd1b7e00640 1 --2- 172.21.15.152:0/743939284 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd1b80b4500 0x7fd1b80d0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:09.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:09.306+0000 7fd1b7400640 1 -- 172.21.15.152:0/743939284 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 687593818 0 0) 0x7fd1b815fd20 con 0x7fd1b80d0640 2024-04-20T11:37:09.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:09.306+0000 7fd1b7400640 1 -- 172.21.15.152:0/743939284 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd19c003520 con 0x7fd1b80d0640 2024-04-20T11:37:09.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:09.306+0000 7fd1b7400640 1 -- 172.21.15.152:0/743939284 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2214598857 0 0) 0x7fd19c003520 con 0x7fd1b80d0640 2024-04-20T11:37:09.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:09.306+0000 7fd1b7400640 1 -- 172.21.15.152:0/743939284 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd1b80b4500 msgr2=0x7fd1b80d0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:09.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:09.306+0000 7fd1b7400640 1 --2- 172.21.15.152:0/743939284 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd1b80b4500 0x7fd1b80d0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:09.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:09.306+0000 7fd1b7400640 1 -- 172.21.15.152:0/743939284 >> v1:172.21.15.175:6789/0 conn(0x7fd1b815c560 legacy=0x7fd1b80b3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:09.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:09.306+0000 7fd1b7400640 1 -- 172.21.15.152:0/743939284 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1b815f520 con 0x7fd1b80d0640 2024-04-20T11:37:09.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:09.306+0000 7fd1b7400640 1 -- 172.21.15.152:0/743939284 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7fd1b00019f0 con 0x7fd1b80d0640 2024-04-20T11:37:09.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:09.306+0000 7fd1b7400640 1 -- 172.21.15.152:0/743939284 >> v1:172.21.15.152:6789/0 conn(0x7fd1b80d0640 legacy=0x7fd1b80ada90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:09.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:09.306+0000 7fd1b7400640 1 --2- 172.21.15.152:0/743939284 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd1b80b4500 0x7fd19c003a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:09.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:09.306+0000 7fd1b7400640 1 --2- 172.21.15.152:0/743939284 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd19c003fd0 0x7fd19c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:09.308 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:09.306+0000 7fd1b7400640 1 --2- 172.21.15.152:0/743939284 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd19c004d40 0x7fd19c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:09.308 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:09.306+0000 7fd1b7400640 1 -- 172.21.15.152:0/743939284 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fd1b815f240 con 0x7fd19c004d40 2024-04-20T11:37:09.308 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:09.306+0000 7fd1b7400640 1 -- 172.21.15.152:0/743939284 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd19c00bbb0 con 0x7fd19c003fd0 2024-04-20T11:37:09.308 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:09.306+0000 7fd1b7400640 1 -- 172.21.15.152:0/743939284 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd19c00bdb0 con 0x7fd1b80b4500 2024-04-20T11:37:09.308 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:09.306+0000 7fd1bda00640 1 --2- 172.21.15.152:0/743939284 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd19c004d40 0x7fd19c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:09.308 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:09.307+0000 7fd1bd000640 1 --2- 172.21.15.152:0/743939284 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd1b80b4500 0x7fd19c003a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:09.308 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:09.307+0000 7fd1b7e00640 1 --2- 172.21.15.152:0/743939284 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd19c003fd0 0x7fd19c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:09.308 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:09.307+0000 7fd1bda00640 1 -- 172.21.15.152:0/743939284 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd1b80b4500 msgr2=0x7fd19c003a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:09.308 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:09.307+0000 7fd1bda00640 1 --2- 172.21.15.152:0/743939284 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd1b80b4500 0x7fd19c003a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:09.308 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:09.307+0000 7fd1bda00640 1 -- 172.21.15.152:0/743939284 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd19c003fd0 msgr2=0x7fd19c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:09.308 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:09.307+0000 7fd1bda00640 1 --2- 172.21.15.152:0/743939284 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd19c003fd0 0x7fd19c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:09.308 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:09.307+0000 7fd1bda00640 1 -- 172.21.15.152:0/743939284 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd19c003520 con 0x7fd19c004d40 2024-04-20T11:37:09.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:09.308+0000 7fd1bda00640 1 --2- 172.21.15.152:0/743939284 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd19c004d40 0x7fd19c0051c0 secure :-1 s=READY pgs=2171 cs=0 l=1 rev1=1 crypto rx=0x7fd1b000a840 tx=0x7fd19c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff8a153e86cd3978 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:09.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:09.308+0000 7fd1b7400640 1 -- 172.21.15.152:0/743939284 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b000a460 con 0x7fd19c004d40 2024-04-20T11:37:09.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:09.308+0000 7fd1b7400640 1 -- 172.21.15.152:0/743939284 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1b0034650 con 0x7fd19c004d40 2024-04-20T11:37:09.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:09.308+0000 7fd1b7400640 1 -- 172.21.15.152:0/743939284 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b0034a80 con 0x7fd19c004d40 2024-04-20T11:37:09.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:09.309+0000 7fd1bfa00640 1 -- 172.21.15.152:0/743939284 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd19c004d40 msgr2=0x7fd19c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:09.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:09.309+0000 7fd1bfa00640 1 --2- 172.21.15.152:0/743939284 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd19c004d40 0x7fd19c0051c0 secure :-1 s=READY pgs=2171 cs=0 l=1 rev1=1 crypto rx=0x7fd1b000a840 tx=0x7fd19c005830 comp rx=0 tx=0).stop 2024-04-20T11:37:09.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:09.309+0000 7fd1bda00640 1 -- 172.21.15.152:0/743939284 reap_dead start 2024-04-20T11:37:09.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:09.309+0000 7fd1bfa00640 1 -- 172.21.15.152:0/743939284 shutdown_connections 2024-04-20T11:37:09.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:09.309+0000 7fd1bfa00640 1 -- 172.21.15.152:0/743939284 >> 172.21.15.152:0/743939284 conn(0x7fd1b8067cf0 msgr2=0x7fd1b806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:09.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:09.309+0000 7fd1bfa00640 1 -- 172.21.15.152:0/743939284 shutdown_connections 2024-04-20T11:37:09.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:09.309+0000 7fd1bfa00640 1 -- 172.21.15.152:0/743939284 wait complete. 2024-04-20T11:37:09.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:09.311+0000 7fd1bfa00640 1 Processor -- start 2024-04-20T11:37:09.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:09.311+0000 7fd1bfa00640 1 -- start start 2024-04-20T11:37:09.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:09.312+0000 7fd1bfa00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd19c004d40 0x7fd1b8157c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:09.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:09.312+0000 7fd1bfa00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd1b8147dd0 0x7fd1b8160ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:09.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:09.312+0000 7fd1bfa00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd1b8159da0 0x7fd1b815a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:09.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:09.312+0000 7fd1bfa00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fd1b80d0aa0 con 0x7fd1b8159da0 2024-04-20T11:37:09.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:09.312+0000 7fd1bfa00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd1b80d0dc0 con 0x7fd1b8147dd0 2024-04-20T11:37:09.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:09.312+0000 7fd1bfa00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd1b80d0100 con 0x7fd19c004d40 2024-04-20T11:37:09.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:09.312+0000 7fd1bda00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd1b8159da0 0x7fd1b815a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:09.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:09.312+0000 7fd1bda00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd1b8159da0 0x7fd1b815a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:59240/0 (socket says 172.21.15.152:59240) 2024-04-20T11:37:09.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:09.312+0000 7fd1bda00640 1 -- 172.21.15.152:0/3308768658 learned_addr learned my addr 172.21.15.152:0/3308768658 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:09.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:09.312+0000 7fd1bd000640 1 --2- 172.21.15.152:0/3308768658 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd19c004d40 0x7fd1b8157c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:09.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:09.313+0000 7fd1b7e00640 1 --2- 172.21.15.152:0/3308768658 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd1b8147dd0 0x7fd1b8160ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:09.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:09.313+0000 7fd1bda00640 1 -- 172.21.15.152:0/3308768658 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd19c004d40 msgr2=0x7fd1b8157c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:09.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:09.313+0000 7fd1bda00640 1 --2- 172.21.15.152:0/3308768658 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd19c004d40 0x7fd1b8157c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:09.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:09.313+0000 7fd1bda00640 1 -- 172.21.15.152:0/3308768658 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd1b8147dd0 msgr2=0x7fd1b8160ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:09.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:09.313+0000 7fd1bda00640 1 --2- 172.21.15.152:0/3308768658 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd1b8147dd0 0x7fd1b8160ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:09.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:09.313+0000 7fd1bda00640 1 -- 172.21.15.152:0/3308768658 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1b81610e0 con 0x7fd1b8159da0 2024-04-20T11:37:09.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:09.313+0000 7fd1b7e00640 1 --2- 172.21.15.152:0/3308768658 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd1b8147dd0 0x7fd1b8160ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:37:09.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:09.313+0000 7fd1bd000640 1 --2- 172.21.15.152:0/3308768658 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd19c004d40 0x7fd1b8157c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:37:09.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:09.313+0000 7fd1bda00640 1 --2- 172.21.15.152:0/3308768658 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd1b8159da0 0x7fd1b815a220 secure :-1 s=READY pgs=2172 cs=0 l=1 rev1=1 crypto rx=0x7fd1b002cc50 tx=0x7fd1b815f6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:09.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:09.313+0000 7fd1b4c00640 1 -- 172.21.15.152:0/3308768658 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b0045070 con 0x7fd1b8159da0 2024-04-20T11:37:09.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:09.313+0000 7fd1b4c00640 1 -- 172.21.15.152:0/3308768658 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1b003a040 con 0x7fd1b8159da0 2024-04-20T11:37:09.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:09.313+0000 7fd1bfa00640 1 -- 172.21.15.152:0/3308768658 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1b8161380 con 0x7fd1b8159da0 2024-04-20T11:37:09.315 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:09.313+0000 7fd1b4c00640 1 -- 172.21.15.152:0/3308768658 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b0034c20 con 0x7fd1b8159da0 2024-04-20T11:37:09.315 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:09.313+0000 7fd1bfa00640 1 -- 172.21.15.152:0/3308768658 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1b8161940 con 0x7fd1b8159da0 2024-04-20T11:37:09.315 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:09.315+0000 7fd1bfa00640 1 -- 172.21.15.152:0/3308768658 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1b815a760 con 0x7fd1b8159da0 2024-04-20T11:37:09.315 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:09.316+0000 7fd1b4c00640 1 -- 172.21.15.152:0/3308768658 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) ==== 91754+0+0 (secure 0 0 0) 0x7fd1b0048070 con 0x7fd1b8159da0 2024-04-20T11:37:09.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:09.317+0000 7fd1b4c00640 1 --2- 172.21.15.152:0/3308768658 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd19406d0d0 0x7fd19406f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:09.317 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:09.317+0000 7fd1b4c00640 1 -- 172.21.15.152:0/3308768658 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(154..154 src has 1..154) ==== 5709+0+0 (secure 0 0 0) 0x7fd1b00ae660 con 0x7fd1b8159da0 2024-04-20T11:37:09.317 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:09.318+0000 7fd1bd000640 1 --2- 172.21.15.152:0/3308768658 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd19406d0d0 0x7fd19406f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:09.318 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:09.319+0000 7fd1bd000640 1 --2- 172.21.15.152:0/3308768658 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd19406d0d0 0x7fd19406f590 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fd1a00016b0 tx=0x7fd1a0009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:09.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:09.322+0000 7fd1b4c00640 1 -- 172.21.15.152:0/3308768658 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fd1b007be30 con 0x7fd1b8159da0 2024-04-20T11:37:09.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:09.561+0000 7fd1bfa00640 1 -- 172.21.15.152:0/3308768658 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fd1b80689f0 con 0x7fd1b8159da0 2024-04-20T11:37:09.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:09.562+0000 7fd1b4c00640 1 -- 172.21.15.152:0/3308768658 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v105) ==== 53+0+73 (secure 0 0 0) 0x7fd1b007b730 con 0x7fd1b8159da0 2024-04-20T11:37:09.561 INFO:teuthology.orchestra.run.smithi152.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-20T11:37:09.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:09.564+0000 7fd1bfa00640 1 -- 172.21.15.152:0/3308768658 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd19406d0d0 msgr2=0x7fd19406f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:09.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:09.564+0000 7fd1bfa00640 1 --2- 172.21.15.152:0/3308768658 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd19406d0d0 0x7fd19406f590 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fd1a00016b0 tx=0x7fd1a0009040 comp rx=0 tx=0).stop 2024-04-20T11:37:09.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:09.564+0000 7fd1bfa00640 1 -- 172.21.15.152:0/3308768658 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd1b8159da0 msgr2=0x7fd1b815a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:09.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:09.564+0000 7fd1bfa00640 1 --2- 172.21.15.152:0/3308768658 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd1b8159da0 0x7fd1b815a220 secure :-1 s=READY pgs=2172 cs=0 l=1 rev1=1 crypto rx=0x7fd1b002cc50 tx=0x7fd1b815f6f0 comp rx=0 tx=0).stop 2024-04-20T11:37:09.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:09.564+0000 7fd1bfa00640 1 -- 172.21.15.152:0/3308768658 shutdown_connections 2024-04-20T11:37:09.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:09.564+0000 7fd1bfa00640 1 --2- 172.21.15.152:0/3308768658 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd19406d0d0 0x7fd19406f590 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:09.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:09.564+0000 7fd1bfa00640 1 --2- 172.21.15.152:0/3308768658 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd1b8159da0 0x7fd1b815a220 unknown :-1 s=CLOSED pgs=2172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:09.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:09.564+0000 7fd1bfa00640 1 --2- 172.21.15.152:0/3308768658 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd1b8147dd0 0x7fd1b8160ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:09.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:09.564+0000 7fd1bfa00640 1 --2- 172.21.15.152:0/3308768658 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd19c004d40 0x7fd1b8157c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:09.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:09.564+0000 7fd1bfa00640 1 -- 172.21.15.152:0/3308768658 >> 172.21.15.152:0/3308768658 conn(0x7fd1b8067cf0 msgr2=0x7fd1b80af300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:09.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:09.565+0000 7fd1bfa00640 1 -- 172.21.15.152:0/3308768658 shutdown_connections 2024-04-20T11:37:09.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:09.565+0000 7fd1bfa00640 1 -- 172.21.15.152:0/3308768658 wait complete. 2024-04-20T11:37:09.583 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-20T11:37:09.583 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-20T11:37:09.583 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi175.front.sepia.ceph.com 2024-04-20T11:37:09.583 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-20T11:37:09.583 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-20T11:37:09.583 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-20T11:37:09.583 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-20T11:37:09.583 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-20T11:37:09.583 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-20T11:37:09.583 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:37:09.584 DEBUG:teuthology.orchestra.run.smithi175:> ip addr 2024-04-20T11:37:09.639 INFO:teuthology.orchestra.run.smithi175.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-20T11:37:09.639 INFO:teuthology.orchestra.run.smithi175.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-20T11:37:09.639 INFO:teuthology.orchestra.run.smithi175.stdout: inet 127.0.0.1/8 scope host lo 2024-04-20T11:37:09.639 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:37:09.639 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 ::1/128 scope host 2024-04-20T11:37:09.639 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:37:09.640 INFO:teuthology.orchestra.run.smithi175.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T11:37:09.640 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:d9:39:ee brd ff:ff:ff:ff:ff:ff 2024-04-20T11:37:09.640 INFO:teuthology.orchestra.run.smithi175.stdout: altname enp7s0f0 2024-04-20T11:37:09.640 INFO:teuthology.orchestra.run.smithi175.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T11:37:09.640 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:d9:39:ef brd ff:ff:ff:ff:ff:ff 2024-04-20T11:37:09.640 INFO:teuthology.orchestra.run.smithi175.stdout: altname enp7s0f1 2024-04-20T11:37:09.640 INFO:teuthology.orchestra.run.smithi175.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T11:37:09.640 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:88:71:f0 brd ff:ff:ff:ff:ff:ff 2024-04-20T11:37:09.640 INFO:teuthology.orchestra.run.smithi175.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-20T11:37:09.640 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:88:71:f1 brd ff:ff:ff:ff:ff:ff 2024-04-20T11:37:09.640 INFO:teuthology.orchestra.run.smithi175.stdout: inet 172.21.15.175/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-20T11:37:09.640 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft 42151sec preferred_lft 42151sec 2024-04-20T11:37:09.640 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::ec4:7aff:fe88:71f1/64 scope link 2024-04-20T11:37:09.640 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:37:09.641 INFO:teuthology.orchestra.run.smithi175.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-20T11:37:09.641 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 02:d6:c6:31:26:70 brd ff:ff:ff:ff:ff:ff 2024-04-20T11:37:09.641 INFO:teuthology.orchestra.run.smithi175.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-20T11:37:09.641 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:37:09.641 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::5091:d5ff:fee7:54b3/64 scope link 2024-04-20T11:37:09.641 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:37:09.641 INFO:teuthology.orchestra.run.smithi175.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-20T11:37:09.641 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 02:d6:c6:31:26:70 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-20T11:37:09.641 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::d6:c6ff:fe31:2670/64 scope link 2024-04-20T11:37:09.641 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:37:09.641 INFO:teuthology.orchestra.run.smithi175.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-20T11:37:09.641 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 9e:14:f3:e2:18:1e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-20T11:37:09.641 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::9c14:f3ff:fee2:181e/64 scope link 2024-04-20T11:37:09.641 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:37:09.642 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:37:09.642 DEBUG:teuthology.orchestra.run.smithi175:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-20T11:37:09.717 INFO:teuthology.orchestra.run.smithi175.stdout:1 2024-04-20T11:37:09.721 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:37:09.721 DEBUG:teuthology.orchestra.run.smithi175:> route 2024-04-20T11:37:09.773 INFO:teuthology.orchestra.run.smithi175.stdout:Kernel IP routing table 2024-04-20T11:37:09.774 INFO:teuthology.orchestra.run.smithi175.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-20T11:37:09.774 INFO:teuthology.orchestra.run.smithi175.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-20T11:37:09.774 INFO:teuthology.orchestra.run.smithi175.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-20T11:37:09.774 INFO:teuthology.orchestra.run.smithi175.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-20T11:37:09.774 DEBUG:teuthology.orchestra.run.smithi175:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T11:37:09.774 DEBUG:teuthology.orchestra.run.smithi175:> set -e 2024-04-20T11:37:09.774 DEBUG:teuthology.orchestra.run.smithi175:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-20T11:37:09.774 DEBUG:teuthology.orchestra.run.smithi175:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-20T11:37:09.774 DEBUG:teuthology.orchestra.run.smithi175:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-20T11:37:09.775 DEBUG:teuthology.orchestra.run.smithi175:> ') 2024-04-20T11:37:09.876 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:37:09 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T11:37:09.971 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:37:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T11:37:09.980 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:37:09.981 DEBUG:teuthology.orchestra.run.smithi175:> ip netns list 2024-04-20T11:37:10.027 INFO:teuthology.orchestra.run.smithi175.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-20T11:37:10.027 INFO:teuthology.orchestra.run.smithi175.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-20T11:37:10.028 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-20T11:37:10.028 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:37:10.028 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip link set brx.0 up 2024-04-20T11:37:10.156 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-04-20T11:37:10.156 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T11:37:10.156 DEBUG:teuthology.orchestra.run.smithi175:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-20T11:37:10.207 INFO:teuthology.orchestra.run.smithi175.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-20T11:37:10.207 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T11:37:10.207 DEBUG:teuthology.orchestra.run.smithi175:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-20T11:37:10.261 DEBUG:teuthology.orchestra.run.smithi175:> sudo modprobe fuse 2024-04-20T11:37:10.339 DEBUG:teuthology.orchestra.run.smithi175:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T11:37:10.393 INFO:teuthology.orchestra.run.smithi175.stdout:/proc 2024-04-20T11:37:10.393 INFO:teuthology.orchestra.run.smithi175.stdout:/sys 2024-04-20T11:37:10.393 INFO:teuthology.orchestra.run.smithi175.stdout:/dev 2024-04-20T11:37:10.393 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/security 2024-04-20T11:37:10.393 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/shm 2024-04-20T11:37:10.393 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/pts 2024-04-20T11:37:10.393 INFO:teuthology.orchestra.run.smithi175.stdout:/run 2024-04-20T11:37:10.393 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/cgroup 2024-04-20T11:37:10.393 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/pstore 2024-04-20T11:37:10.393 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/bpf 2024-04-20T11:37:10.393 INFO:teuthology.orchestra.run.smithi175.stdout:/ 2024-04-20T11:37:10.394 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/selinux 2024-04-20T11:37:10.394 INFO:teuthology.orchestra.run.smithi175.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T11:37:10.394 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/mqueue 2024-04-20T11:37:10.394 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/debug 2024-04-20T11:37:10.394 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/hugepages 2024-04-20T11:37:10.394 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/tracing 2024-04-20T11:37:10.394 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/fuse/connections 2024-04-20T11:37:10.394 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/config 2024-04-20T11:37:10.394 INFO:teuthology.orchestra.run.smithi175.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T11:37:10.394 INFO:teuthology.orchestra.run.smithi175.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T11:37:10.394 INFO:teuthology.orchestra.run.smithi175.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T11:37:10.394 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T11:37:10.394 INFO:teuthology.orchestra.run.smithi175.stdout:/run/user/1000 2024-04-20T11:37:10.394 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph 2024-04-20T11:37:10.394 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-4 2024-04-20T11:37:10.395 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-5 2024-04-20T11:37:10.395 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-6 2024-04-20T11:37:10.395 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-7 2024-04-20T11:37:10.395 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns 2024-04-20T11:37:10.395 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-20T11:37:10.395 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-20T11:37:10.395 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-20T11:37:10.395 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-20T11:37:10.396 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:37:10.396 DEBUG:teuthology.orchestra.run.smithi175:> ls /sys/fs/fuse/connections 2024-04-20T11:37:10.447 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-20T11:37:10.447 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-04-20T11:37:10.452 DEBUG:teuthology.orchestra.run.smithi175:> sudo modprobe fuse 2024-04-20T11:37:10.522 DEBUG:teuthology.orchestra.run.smithi175:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T11:37:10.563 INFO:teuthology.orchestra.run.smithi175.stdout:/proc 2024-04-20T11:37:10.563 INFO:teuthology.orchestra.run.smithi175.stdout:/sys 2024-04-20T11:37:10.563 INFO:teuthology.orchestra.run.smithi175.stdout:/dev 2024-04-20T11:37:10.563 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/security 2024-04-20T11:37:10.563 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/shm 2024-04-20T11:37:10.563 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/pts 2024-04-20T11:37:10.563 INFO:teuthology.orchestra.run.smithi175.stdout:/run 2024-04-20T11:37:10.563 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/cgroup 2024-04-20T11:37:10.563 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/pstore 2024-04-20T11:37:10.563 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/bpf 2024-04-20T11:37:10.563 INFO:teuthology.orchestra.run.smithi175.stdout:/ 2024-04-20T11:37:10.564 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/selinux 2024-04-20T11:37:10.564 INFO:teuthology.orchestra.run.smithi175.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T11:37:10.564 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/mqueue 2024-04-20T11:37:10.564 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/debug 2024-04-20T11:37:10.564 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/hugepages 2024-04-20T11:37:10.564 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/tracing 2024-04-20T11:37:10.564 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/fuse/connections 2024-04-20T11:37:10.564 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/config 2024-04-20T11:37:10.564 INFO:teuthology.orchestra.run.smithi175.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T11:37:10.564 INFO:teuthology.orchestra.run.smithi175.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T11:37:10.564 INFO:teuthology.orchestra.run.smithi175.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T11:37:10.564 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T11:37:10.564 INFO:teuthology.orchestra.run.smithi175.stdout:/run/user/1000 2024-04-20T11:37:10.564 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph 2024-04-20T11:37:10.565 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-4 2024-04-20T11:37:10.565 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-5 2024-04-20T11:37:10.565 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-6 2024-04-20T11:37:10.565 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-7 2024-04-20T11:37:10.565 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns 2024-04-20T11:37:10.565 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-20T11:37:10.565 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-20T11:37:10.565 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-20T11:37:10.565 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-20T11:37:10.566 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:37:10.566 DEBUG:teuthology.orchestra.run.smithi175:> ls /sys/fs/fuse/connections 2024-04-20T11:37:10.626 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi175.stderr:2024-04-20T11:37:10.626+0000 7f75e69e6480 -1 init, newargv = 0x5639b63cc090 newargc=15 2024-04-20T11:37:10.626 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi175.stderr:ceph-fuse[36532]: starting ceph client 2024-04-20T11:37:10.637 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi175.stderr:ceph-fuse[36532]: starting fuse 2024-04-20T11:37:10.648 INFO:teuthology.orchestra.run.smithi175.stdout:42 2024-04-20T11:37:10.648 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-04-20T11:37:10.648 DEBUG:teuthology.orchestra.run.smithi175:> sudo stdin-killer -- python3 -c ' 2024-04-20T11:37:10.649 DEBUG:teuthology.orchestra.run.smithi175:> import glob 2024-04-20T11:37:10.649 DEBUG:teuthology.orchestra.run.smithi175:> import re 2024-04-20T11:37:10.649 DEBUG:teuthology.orchestra.run.smithi175:> import os 2024-04-20T11:37:10.649 DEBUG:teuthology.orchestra.run.smithi175:> import subprocess 2024-04-20T11:37:10.649 DEBUG:teuthology.orchestra.run.smithi175:> 2024-04-20T11:37:10.649 DEBUG:teuthology.orchestra.run.smithi175:> def _find_admin_socket(client_name): 2024-04-20T11:37:10.649 DEBUG:teuthology.orchestra.run.smithi175:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-04-20T11:37:10.649 DEBUG:teuthology.orchestra.run.smithi175:> files = glob.glob(asok_path) 2024-04-20T11:37:10.649 DEBUG:teuthology.orchestra.run.smithi175:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-04-20T11:37:10.649 DEBUG:teuthology.orchestra.run.smithi175:> 2024-04-20T11:37:10.649 DEBUG:teuthology.orchestra.run.smithi175:> # Given a non-glob path, it better be there 2024-04-20T11:37:10.649 DEBUG:teuthology.orchestra.run.smithi175:> if "*" not in asok_path: 2024-04-20T11:37:10.649 DEBUG:teuthology.orchestra.run.smithi175:> assert(len(files) == 1) 2024-04-20T11:37:10.649 DEBUG:teuthology.orchestra.run.smithi175:> return files[0] 2024-04-20T11:37:10.649 DEBUG:teuthology.orchestra.run.smithi175:> 2024-04-20T11:37:10.649 DEBUG:teuthology.orchestra.run.smithi175:> for f in files: 2024-04-20T11:37:10.650 DEBUG:teuthology.orchestra.run.smithi175:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-20T11:37:10.650 DEBUG:teuthology.orchestra.run.smithi175:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-20T11:37:10.650 DEBUG:teuthology.orchestra.run.smithi175:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-20T11:37:10.650 DEBUG:teuthology.orchestra.run.smithi175:> contents = proc_f.read() 2024-04-20T11:37:10.650 DEBUG:teuthology.orchestra.run.smithi175:> if mountpoint in contents: 2024-04-20T11:37:10.650 DEBUG:teuthology.orchestra.run.smithi175:> return f 2024-04-20T11:37:10.650 DEBUG:teuthology.orchestra.run.smithi175:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-20T11:37:10.650 DEBUG:teuthology.orchestra.run.smithi175:> 2024-04-20T11:37:10.650 DEBUG:teuthology.orchestra.run.smithi175:> print(_find_admin_socket("client.1")) 2024-04-20T11:37:10.650 DEBUG:teuthology.orchestra.run.smithi175:> ' 2024-04-20T11:37:10.829 INFO:teuthology.orchestra.run.smithi175.stdout:/var/run/ceph/ceph-client.1.36532.asok 2024-04-20T11:37:10.833 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:37:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T11:37:10.845 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.36532.asok 2024-04-20T11:37:10.845 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:37:10.845 DEBUG:teuthology.orchestra.run.smithi175:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.36532.asok status 2024-04-20T11:37:11.071 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-04-20T11:37:11.071 INFO:teuthology.orchestra.run.smithi175.stdout: "metadata": { 2024-04-20T11:37:11.071 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_sha1": "a8c84f5997950fd7d008204839253291d52e6eda", 2024-04-20T11:37:11.071 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", 2024-04-20T11:37:11.072 INFO:teuthology.orchestra.run.smithi175.stdout: "entity_id": "1", 2024-04-20T11:37:11.072 INFO:teuthology.orchestra.run.smithi175.stdout: "hostname": "smithi175", 2024-04-20T11:37:11.072 INFO:teuthology.orchestra.run.smithi175.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-04-20T11:37:11.072 INFO:teuthology.orchestra.run.smithi175.stdout: "pid": "36532", 2024-04-20T11:37:11.072 INFO:teuthology.orchestra.run.smithi175.stdout: "root": "/" 2024-04-20T11:37:11.072 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-20T11:37:11.072 INFO:teuthology.orchestra.run.smithi175.stdout: "dentry_count": 0, 2024-04-20T11:37:11.072 INFO:teuthology.orchestra.run.smithi175.stdout: "dentry_pinned_count": 0, 2024-04-20T11:37:11.072 INFO:teuthology.orchestra.run.smithi175.stdout: "id": 6512, 2024-04-20T11:37:11.072 INFO:teuthology.orchestra.run.smithi175.stdout: "inst": { 2024-04-20T11:37:11.072 INFO:teuthology.orchestra.run.smithi175.stdout: "name": { 2024-04-20T11:37:11.072 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "client", 2024-04-20T11:37:11.072 INFO:teuthology.orchestra.run.smithi175.stdout: "num": 6512 2024-04-20T11:37:11.072 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-20T11:37:11.073 INFO:teuthology.orchestra.run.smithi175.stdout: "addr": { 2024-04-20T11:37:11.073 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "v1", 2024-04-20T11:37:11.073 INFO:teuthology.orchestra.run.smithi175.stdout: "addr": "192.168.0.1:0", 2024-04-20T11:37:11.073 INFO:teuthology.orchestra.run.smithi175.stdout: "nonce": 2507674889 2024-04-20T11:37:11.073 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-04-20T11:37:11.073 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-20T11:37:11.073 INFO:teuthology.orchestra.run.smithi175.stdout: "addr": { 2024-04-20T11:37:11.073 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "v1", 2024-04-20T11:37:11.073 INFO:teuthology.orchestra.run.smithi175.stdout: "addr": "192.168.0.1:0", 2024-04-20T11:37:11.073 INFO:teuthology.orchestra.run.smithi175.stdout: "nonce": 2507674889 2024-04-20T11:37:11.073 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-20T11:37:11.073 INFO:teuthology.orchestra.run.smithi175.stdout: "inst_str": "client.6512 192.168.0.1:0/2507674889", 2024-04-20T11:37:11.073 INFO:teuthology.orchestra.run.smithi175.stdout: "addr_str": "192.168.0.1:0/2507674889", 2024-04-20T11:37:11.073 INFO:teuthology.orchestra.run.smithi175.stdout: "inode_count": 1, 2024-04-20T11:37:11.073 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_epoch": 105, 2024-04-20T11:37:11.074 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_epoch": 154, 2024-04-20T11:37:11.074 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_epoch_barrier": 0, 2024-04-20T11:37:11.074 INFO:teuthology.orchestra.run.smithi175.stdout: "blocklisted": false, 2024-04-20T11:37:11.074 INFO:teuthology.orchestra.run.smithi175.stdout: "fs_name": "cephfs" 2024-04-20T11:37:11.074 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-04-20T11:37:11.085 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:37:11.086 DEBUG:teuthology.orchestra.run.smithi175:> stat --file-system '--printf=%T 2024-04-20T11:37:11.086 DEBUG:teuthology.orchestra.run.smithi175:> ' -- /home/ubuntu/cephtest/mnt.1 2024-04-20T11:37:11.140 INFO:teuthology.orchestra.run.smithi175.stdout:fuseblk 2024-04-20T11:37:11.140 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-04-20T11:37:11.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:37:11.141 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-20T11:37:11.226 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-04-20T11:37:11.442 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:11.442+0000 7f670fa00640 1 Processor -- start 2024-04-20T11:37:11.442 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:11.442+0000 7f670fa00640 1 -- start start 2024-04-20T11:37:11.444 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:11.444+0000 7f670fa00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f6708144f80 0x7f670806eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:11.444 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:11.444+0000 7f670fa00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f670815f1a0 con 0x7f670806f420 2024-04-20T11:37:11.444 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:11.444+0000 7f670fa00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f670815f480 con 0x7f67080cd930 2024-04-20T11:37:11.444 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:11.444+0000 7f670fa00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f670815f760 con 0x7f6708144f80 2024-04-20T11:37:11.444 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:11.444+0000 7f670d000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f6708144f80 0x7f670806eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:11.444 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:11.444+0000 7f670d000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f6708144f80 0x7f670806eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:59256/0 (socket says 172.21.15.152:59256) 2024-04-20T11:37:11.444 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:11.445+0000 7f670d000640 1 -- 172.21.15.152:0/3632750764 learned_addr learned my addr 172.21.15.152:0/3632750764 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:11.445 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:11.445+0000 7f670d000640 1 -- 172.21.15.152:0/3632750764 >> v1:172.21.15.175:6789/0 conn(0x7f67080cd930 legacy=0x7f6708071db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:11.445 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:11.445+0000 7f670d000640 1 -- 172.21.15.152:0/3632750764 >> v1:172.21.15.152:6789/0 conn(0x7f670806f420 legacy=0x7f67080cd220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:11.445 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:11.445+0000 7f670d000640 1 -- 172.21.15.152:0/3632750764 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f670815fa40 con 0x7f6708144f80 2024-04-20T11:37:11.445 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:11.446+0000 7f670d000640 1 --2- 172.21.15.152:0/3632750764 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6708144f80 0x7f670806eee0 secure :-1 s=READY pgs=2174 cs=0 l=1 rev1=1 crypto rx=0x7f66f800bb10 tx=0x7f66f802e170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4001cb876bd6c3f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:11.446 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:11.446+0000 7f6707e00640 1 -- 172.21.15.152:0/3632750764 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f803b070 con 0x7f6708144f80 2024-04-20T11:37:11.446 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:11.446+0000 7f6707e00640 1 -- 172.21.15.152:0/3632750764 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66f8036440 con 0x7f6708144f80 2024-04-20T11:37:11.446 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:11.447+0000 7f6707e00640 1 -- 172.21.15.152:0/3632750764 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f8036810 con 0x7f6708144f80 2024-04-20T11:37:11.447 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:11.447+0000 7f670fa00640 1 -- 172.21.15.152:0/3632750764 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6708144f80 msgr2=0x7f670806eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:11.447 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:11.447+0000 7f670fa00640 1 --2- 172.21.15.152:0/3632750764 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6708144f80 0x7f670806eee0 secure :-1 s=READY pgs=2174 cs=0 l=1 rev1=1 crypto rx=0x7f66f800bb10 tx=0x7f66f802e170 comp rx=0 tx=0).stop 2024-04-20T11:37:11.447 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:11.447+0000 7f670fa00640 1 -- 172.21.15.152:0/3632750764 shutdown_connections 2024-04-20T11:37:11.447 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:11.447+0000 7f670fa00640 1 --2- 172.21.15.152:0/3632750764 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6708144f80 0x7f670806eee0 unknown :-1 s=CLOSED pgs=2174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:11.447 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:11.447+0000 7f670fa00640 1 -- 172.21.15.152:0/3632750764 >> 172.21.15.152:0/3632750764 conn(0x7f6708067cf0 msgr2=0x7f670806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:11.447 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:11.447+0000 7f670fa00640 1 -- 172.21.15.152:0/3632750764 shutdown_connections 2024-04-20T11:37:11.447 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:11.447+0000 7f670fa00640 1 -- 172.21.15.152:0/3632750764 wait complete. 2024-04-20T11:37:11.448 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:11.449+0000 7f670fa00640 1 Processor -- start 2024-04-20T11:37:11.448 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:11.449+0000 7f670fa00640 1 -- start start 2024-04-20T11:37:11.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:11.450+0000 7f670fa00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f670806f420 0x7f67080a9710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:11.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:11.450+0000 7f670fa00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f67080cd930 0x7f67080af990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:11.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:11.450+0000 7f670fa00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6708144f80 0x7f67080afed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:11.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:11.450+0000 7f670fa00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f6708072640 con 0x7f6708144f80 2024-04-20T11:37:11.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:11.450+0000 7f670fa00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f67080724c0 con 0x7f670806f420 2024-04-20T11:37:11.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:11.450+0000 7f670fa00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f67080727c0 con 0x7f67080cd930 2024-04-20T11:37:11.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:11.450+0000 7f670da00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6708144f80 0x7f67080afed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:11.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:11.450+0000 7f670da00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6708144f80 0x7f67080afed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:59270/0 (socket says 172.21.15.152:59270) 2024-04-20T11:37:11.452 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:11.450+0000 7f670da00640 1 -- 172.21.15.152:0/948394367 learned_addr learned my addr 172.21.15.152:0/948394367 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:11.452 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:11.451+0000 7f670d000640 1 --2- 172.21.15.152:0/948394367 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f670806f420 0x7f67080a9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:11.452 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:11.451+0000 7f66ffe00640 1 --2- 172.21.15.152:0/948394367 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f67080cd930 0x7f67080af990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:11.452 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:11.451+0000 7f670da00640 1 -- 172.21.15.152:0/948394367 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f67080cd930 msgr2=0x7f67080af990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:11.452 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:11.451+0000 7f670da00640 1 --2- 172.21.15.152:0/948394367 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f67080cd930 0x7f67080af990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:11.452 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:11.451+0000 7f670da00640 1 -- 172.21.15.152:0/948394367 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f670806f420 msgr2=0x7f67080a9710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:11.452 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:11.451+0000 7f670da00640 1 --2- 172.21.15.152:0/948394367 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f670806f420 0x7f67080a9710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:11.452 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:11.451+0000 7f670da00640 1 -- 172.21.15.152:0/948394367 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67081e6e00 con 0x7f6708144f80 2024-04-20T11:37:11.452 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:11.451+0000 7f670d000640 1 --2- 172.21.15.152:0/948394367 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f670806f420 0x7f67080a9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:37:11.452 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:11.451+0000 7f66ffe00640 1 --2- 172.21.15.152:0/948394367 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f67080cd930 0x7f67080af990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:37:11.452 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:11.451+0000 7f670da00640 1 --2- 172.21.15.152:0/948394367 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6708144f80 0x7f67080afed0 secure :-1 s=READY pgs=2175 cs=0 l=1 rev1=1 crypto rx=0x7f670000c780 tx=0x7f670000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:11.452 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:11.451+0000 7f6705600640 1 -- 172.21.15.152:0/948394367 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6700017070 con 0x7f6708144f80 2024-04-20T11:37:11.452 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:11.451+0000 7f6705600640 1 -- 172.21.15.152:0/948394367 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6700003da0 con 0x7f6708144f80 2024-04-20T11:37:11.452 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:11.451+0000 7f6705600640 1 -- 172.21.15.152:0/948394367 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67000052b0 con 0x7f6708144f80 2024-04-20T11:37:11.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:11.452+0000 7f670fa00640 1 -- 172.21.15.152:0/948394367 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67081e70e0 con 0x7f6708144f80 2024-04-20T11:37:11.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:11.452+0000 7f670fa00640 1 -- 172.21.15.152:0/948394367 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67081e76a0 con 0x7f6708144f80 2024-04-20T11:37:11.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:11.453+0000 7f670fa00640 1 -- 172.21.15.152:0/948394367 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6708068a60 con 0x7f6708144f80 2024-04-20T11:37:11.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:11.454+0000 7f6705600640 1 -- 172.21.15.152:0/948394367 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) ==== 91754+0+0 (secure 0 0 0) 0x7f6700005450 con 0x7f6708144f80 2024-04-20T11:37:11.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:11.455+0000 7f6705600640 1 --2- 172.21.15.152:0/948394367 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f66e406d0d0 0x7f66e406f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:11.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:11.455+0000 7f6705600640 1 -- 172.21.15.152:0/948394367 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(154..154 src has 1..154) ==== 5709+0+0 (secure 0 0 0) 0x7f670008c400 con 0x7f6708144f80 2024-04-20T11:37:11.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:11.455+0000 7f670d000640 1 --2- 172.21.15.152:0/948394367 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f66e406d0d0 0x7f66e406f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:11.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:11.457+0000 7f670d000640 1 --2- 172.21.15.152:0/948394367 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f66e406d0d0 0x7f66e406f590 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f66f8038000 tx=0x7f66f8008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:11.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:11.459+0000 7f6705600640 1 -- 172.21.15.152:0/948394367 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f67000598a0 con 0x7f6708144f80 2024-04-20T11:37:11.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:11.691+0000 7f670fa00640 1 -- 172.21.15.152:0/948394367 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f670806bd00 con 0x7f6708144f80 2024-04-20T11:37:12.420 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:12.420+0000 7f6705600640 1 -- 172.21.15.152:0/948394367 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v106) ==== 136+0+0 (secure 0 0 0) 0x7f67000591a0 con 0x7f6708144f80 2024-04-20T11:37:12.424 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:12.424+0000 7f670fa00640 1 -- 172.21.15.152:0/948394367 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f66e406d0d0 msgr2=0x7f66e406f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:12.424 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:12.424+0000 7f670fa00640 1 --2- 172.21.15.152:0/948394367 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f66e406d0d0 0x7f66e406f590 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f66f8038000 tx=0x7f66f8008040 comp rx=0 tx=0).stop 2024-04-20T11:37:12.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:12.424+0000 7f670fa00640 1 -- 172.21.15.152:0/948394367 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6708144f80 msgr2=0x7f67080afed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:12.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:12.424+0000 7f670fa00640 1 --2- 172.21.15.152:0/948394367 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6708144f80 0x7f67080afed0 secure :-1 s=READY pgs=2175 cs=0 l=1 rev1=1 crypto rx=0x7f670000c780 tx=0x7f670000cc40 comp rx=0 tx=0).stop 2024-04-20T11:37:12.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:12.425+0000 7f670fa00640 1 -- 172.21.15.152:0/948394367 shutdown_connections 2024-04-20T11:37:12.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:12.425+0000 7f670fa00640 1 --2- 172.21.15.152:0/948394367 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f66e406d0d0 0x7f66e406f590 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:12.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:12.425+0000 7f670fa00640 1 --2- 172.21.15.152:0/948394367 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6708144f80 0x7f67080afed0 unknown :-1 s=CLOSED pgs=2175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:12.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:12.425+0000 7f670fa00640 1 --2- 172.21.15.152:0/948394367 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f67080cd930 0x7f67080af990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:12.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:12.425+0000 7f670fa00640 1 --2- 172.21.15.152:0/948394367 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f670806f420 0x7f67080a9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:12.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:12.425+0000 7f670fa00640 1 -- 172.21.15.152:0/948394367 >> 172.21.15.152:0/948394367 conn(0x7f6708067cf0 msgr2=0x7f670815dab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:12.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:12.425+0000 7f670fa00640 1 -- 172.21.15.152:0/948394367 shutdown_connections 2024-04-20T11:37:12.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:12.425+0000 7f670fa00640 1 -- 172.21.15.152:0/948394367 wait complete. 2024-04-20T11:37:12.448 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-04-20T11:37:12.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:12.652+0000 7f6b3fe00640 1 Processor -- start 2024-04-20T11:37:12.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:12.653+0000 7f6b3fe00640 1 -- start start 2024-04-20T11:37:12.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:12.654+0000 7f6b3fe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b40161d30 0x7f6b400d0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:12.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:12.654+0000 7f6b3fe00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f6b400d2b80 0x7f6b400d3030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:12.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:12.654+0000 7f6b3fe00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b400d26a0 con 0x7f6b400d4ec0 2024-04-20T11:37:12.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:12.654+0000 7f6b3fe00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f6b400d0540 con 0x7f6b400d2b80 2024-04-20T11:37:12.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:12.654+0000 7f6b3fe00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6b400cfb70 con 0x7f6b40161d30 2024-04-20T11:37:12.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:12.654+0000 7f6b3e000640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f6b400d4ec0 0x7f6b400d15e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:43946/0 (socket says 172.21.15.152:43946) 2024-04-20T11:37:12.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:12.654+0000 7f6b3e000640 1 -- 172.21.15.152:0/223944321 learned_addr learned my addr 172.21.15.152:0/223944321 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:37:12.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:12.655+0000 7f6b3ea00640 1 --2- 172.21.15.152:0/223944321 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b40161d30 0x7f6b400d0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:12.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:12.655+0000 7f6b3f400640 1 --2- 172.21.15.152:0/223944321 >> v2:172.21.15.175:3300/0 conn(0x7f6b400d2b80 0x7f6b400d3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:12.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:12.655+0000 7f6b3d600640 1 -- 172.21.15.152:0/223944321 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1237613115 0 0) 0x7f6b400d26a0 con 0x7f6b400d4ec0 2024-04-20T11:37:12.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:12.655+0000 7f6b3d600640 1 -- 172.21.15.152:0/223944321 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6b20003520 con 0x7f6b400d4ec0 2024-04-20T11:37:12.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:12.655+0000 7f6b3d600640 1 -- 172.21.15.152:0/223944321 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4177805349 0 0) 0x7f6b20003520 con 0x7f6b400d4ec0 2024-04-20T11:37:12.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:12.655+0000 7f6b3d600640 1 -- 172.21.15.152:0/223944321 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b40161d30 msgr2=0x7f6b400d0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:12.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:12.655+0000 7f6b3d600640 1 --2- 172.21.15.152:0/223944321 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b40161d30 0x7f6b400d0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:12.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:12.655+0000 7f6b3d600640 1 -- 172.21.15.152:0/223944321 >> v2:172.21.15.175:3300/0 conn(0x7f6b400d2b80 msgr2=0x7f6b400d3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:12.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:12.655+0000 7f6b3d600640 1 --2- 172.21.15.152:0/223944321 >> v2:172.21.15.175:3300/0 conn(0x7f6b400d2b80 0x7f6b400d3030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:12.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:12.655+0000 7f6b3d600640 1 -- 172.21.15.152:0/223944321 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b400d53c0 con 0x7f6b400d4ec0 2024-04-20T11:37:12.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:12.655+0000 7f6b3d600640 1 -- 172.21.15.152:0/223944321 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f6b28001b70 con 0x7f6b400d4ec0 2024-04-20T11:37:12.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:12.655+0000 7f6b3d600640 1 -- 172.21.15.152:0/223944321 >> v1:172.21.15.152:6789/0 conn(0x7f6b400d4ec0 legacy=0x7f6b400d15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:12.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:12.655+0000 7f6b3d600640 1 --2- 172.21.15.152:0/223944321 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6b200039b0 0x7f6b20003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:12.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:12.655+0000 7f6b3d600640 1 --2- 172.21.15.152:0/223944321 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b40161d30 0x7f6b20004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:12.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:12.656+0000 7f6b3d600640 1 --2- 172.21.15.152:0/223944321 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b20004d40 0x7f6b200051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:12.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:12.656+0000 7f6b3d600640 1 -- 172.21.15.152:0/223944321 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f6b400d0540 con 0x7f6b200039b0 2024-04-20T11:37:12.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:12.656+0000 7f6b3d600640 1 -- 172.21.15.152:0/223944321 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6b400cfb70 con 0x7f6b20004d40 2024-04-20T11:37:12.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:12.656+0000 7f6b3d600640 1 -- 172.21.15.152:0/223944321 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6b2000b9c0 con 0x7f6b40161d30 2024-04-20T11:37:12.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:12.656+0000 7f6b3ea00640 1 --2- 172.21.15.152:0/223944321 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6b200039b0 0x7f6b20003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:12.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:12.656+0000 7f6b3ea00640 1 -- 172.21.15.152:0/223944321 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b40161d30 msgr2=0x7f6b20004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:12.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:12.656+0000 7f6b3ea00640 1 --2- 172.21.15.152:0/223944321 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b40161d30 0x7f6b20004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:12.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:12.656+0000 7f6b3ea00640 1 -- 172.21.15.152:0/223944321 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b20004d40 msgr2=0x7f6b200051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:12.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:12.656+0000 7f6b3ea00640 1 --2- 172.21.15.152:0/223944321 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b20004d40 0x7f6b200051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:12.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:12.656+0000 7f6b3ea00640 1 -- 172.21.15.152:0/223944321 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b20003520 con 0x7f6b200039b0 2024-04-20T11:37:12.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:12.657+0000 7f6b3ea00640 1 --2- 172.21.15.152:0/223944321 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6b200039b0 0x7f6b20003dd0 secure :-1 s=READY pgs=2177 cs=0 l=1 rev1=1 crypto rx=0x7f6b30009ac0 tx=0x7f6b20005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cbc0761a621b7de server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:12.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:12.657+0000 7f6b3d600640 1 -- 172.21.15.152:0/223944321 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b30039070 con 0x7f6b200039b0 2024-04-20T11:37:12.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:12.657+0000 7f6b3d600640 1 -- 172.21.15.152:0/223944321 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b3002c710 con 0x7f6b200039b0 2024-04-20T11:37:12.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:12.657+0000 7f6b3d600640 1 -- 172.21.15.152:0/223944321 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b30009690 con 0x7f6b200039b0 2024-04-20T11:37:12.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:12.658+0000 7f6b3fe00640 1 -- 172.21.15.152:0/223944321 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6b200039b0 msgr2=0x7f6b20003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:12.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:12.658+0000 7f6b3fe00640 1 --2- 172.21.15.152:0/223944321 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6b200039b0 0x7f6b20003dd0 secure :-1 s=READY pgs=2177 cs=0 l=1 rev1=1 crypto rx=0x7f6b30009ac0 tx=0x7f6b20005830 comp rx=0 tx=0).stop 2024-04-20T11:37:12.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:12.658+0000 7f6b3f400640 1 -- 172.21.15.152:0/223944321 reap_dead start 2024-04-20T11:37:12.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:12.658+0000 7f6b3fe00640 1 -- 172.21.15.152:0/223944321 shutdown_connections 2024-04-20T11:37:12.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:12.658+0000 7f6b3fe00640 1 -- 172.21.15.152:0/223944321 >> 172.21.15.152:0/223944321 conn(0x7f6b4008bfd0 msgr2=0x7f6b400d45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:12.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:12.658+0000 7f6b3fe00640 1 -- 172.21.15.152:0/223944321 shutdown_connections 2024-04-20T11:37:12.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:12.658+0000 7f6b3fe00640 1 -- 172.21.15.152:0/223944321 wait complete. 2024-04-20T11:37:12.664 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:12.660+0000 7f6b3fe00640 1 Processor -- start 2024-04-20T11:37:12.664 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:12.660+0000 7f6b3fe00640 1 -- start start 2024-04-20T11:37:12.664 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:12.661+0000 7f6b3fe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b200039b0 0x7f6b400b9d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:12.664 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:12.661+0000 7f6b3fe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6b40159d90 0x7f6b4015a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:12.664 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:12.661+0000 7f6b3fe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b400b2480 0x7f6b400b2930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:12.664 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:12.661+0000 7f6b3fe00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f6b400cf1b0 con 0x7f6b40159d90 2024-04-20T11:37:12.664 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:12.661+0000 7f6b3fe00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6b400cf800 con 0x7f6b400b2480 2024-04-20T11:37:12.664 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:12.661+0000 7f6b3fe00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6b40162110 con 0x7f6b200039b0 2024-04-20T11:37:12.664 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:12.661+0000 7f6b3e000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6b40159d90 0x7f6b4015a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:12.664 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:12.662+0000 7f6b3e000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6b40159d90 0x7f6b4015a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:59284/0 (socket says 172.21.15.152:59284) 2024-04-20T11:37:12.664 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:12.662+0000 7f6b3e000640 1 -- 172.21.15.152:0/669461508 learned_addr learned my addr 172.21.15.152:0/669461508 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:12.664 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:12.662+0000 7f6b3ea00640 1 --2- 172.21.15.152:0/669461508 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b200039b0 0x7f6b400b9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:12.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:12.662+0000 7f6b3f400640 1 --2- 172.21.15.152:0/669461508 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b400b2480 0x7f6b400b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:12.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:12.662+0000 7f6b3e000640 1 -- 172.21.15.152:0/669461508 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b200039b0 msgr2=0x7f6b400b9d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:12.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:12.662+0000 7f6b3e000640 1 --2- 172.21.15.152:0/669461508 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b200039b0 0x7f6b400b9d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:12.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:12.662+0000 7f6b3e000640 1 -- 172.21.15.152:0/669461508 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b400b2480 msgr2=0x7f6b400b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:12.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:12.662+0000 7f6b3e000640 1 --2- 172.21.15.152:0/669461508 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b400b2480 0x7f6b400b2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:12.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:12.662+0000 7f6b3e000640 1 -- 172.21.15.152:0/669461508 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b400ba2b0 con 0x7f6b40159d90 2024-04-20T11:37:12.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:12.662+0000 7f6b3ea00640 1 --2- 172.21.15.152:0/669461508 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b200039b0 0x7f6b400b9d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:37:12.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:12.662+0000 7f6b3e000640 1 --2- 172.21.15.152:0/669461508 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6b40159d90 0x7f6b4015a210 secure :-1 s=READY pgs=2178 cs=0 l=1 rev1=1 crypto rx=0x7f6b2800ca70 tx=0x7f6b2800cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:12.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:12.663+0000 7f6b36a00640 1 -- 172.21.15.152:0/669461508 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b2800a7c0 con 0x7f6b40159d90 2024-04-20T11:37:12.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:12.663+0000 7f6b36a00640 1 -- 172.21.15.152:0/669461508 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b2800a960 con 0x7f6b40159d90 2024-04-20T11:37:12.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:12.663+0000 7f6b3fe00640 1 -- 172.21.15.152:0/669461508 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b401f4290 con 0x7f6b40159d90 2024-04-20T11:37:12.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:12.663+0000 7f6b36a00640 1 -- 172.21.15.152:0/669461508 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b2800ad30 con 0x7f6b40159d90 2024-04-20T11:37:12.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:12.663+0000 7f6b3fe00640 1 -- 172.21.15.152:0/669461508 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b401f4850 con 0x7f6b40159d90 2024-04-20T11:37:12.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:12.664+0000 7f6b3fe00640 1 -- 172.21.15.152:0/669461508 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b40075e60 con 0x7f6b40159d90 2024-04-20T11:37:12.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:12.665+0000 7f6b36a00640 1 -- 172.21.15.152:0/669461508 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) ==== 91754+0+0 (secure 0 0 0) 0x7f6b28007070 con 0x7f6b40159d90 2024-04-20T11:37:12.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:12.666+0000 7f6b36a00640 1 --2- 172.21.15.152:0/669461508 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f6b1c06d0d0 0x7f6b1c06f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:12.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:12.666+0000 7f6b36a00640 1 -- 172.21.15.152:0/669461508 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(154..154 src has 1..154) ==== 5709+0+0 (secure 0 0 0) 0x7f6b2808a7a0 con 0x7f6b40159d90 2024-04-20T11:37:12.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:12.667+0000 7f6b3ea00640 1 --2- 172.21.15.152:0/669461508 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f6b1c06d0d0 0x7f6b1c06f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:12.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:12.668+0000 7f6b3ea00640 1 --2- 172.21.15.152:0/669461508 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f6b1c06d0d0 0x7f6b1c06f590 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f6b30036000 tx=0x7f6b30030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:12.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:12.670+0000 7f6b36a00640 1 -- 172.21.15.152:0/669461508 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f6b2805a4c0 con 0x7f6b40159d90 2024-04-20T11:37:12.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:12.899+0000 7f6b3fe00640 1 -- 172.21.15.152:0/669461508 --> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f6b400d1d70 con 0x7f6b1c06d0d0 2024-04-20T11:37:12.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:12.903+0000 7f6b36a00640 1 -- 172.21.15.152:0/669461508 <== mgr.4109 v2:172.21.15.175:6836/4021250441 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f6b400d1d70 con 0x7f6b1c06d0d0 2024-04-20T11:37:12.903 INFO:teuthology.orchestra.run.smithi152.stdout:[ 2024-04-20T11:37:12.903 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:37:12.903 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "cephfs" 2024-04-20T11:37:12.903 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:37:12.903 INFO:teuthology.orchestra.run.smithi152.stdout:] 2024-04-20T11:37:12.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:12.905+0000 7f6b3fe00640 1 -- 172.21.15.152:0/669461508 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f6b1c06d0d0 msgr2=0x7f6b1c06f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:12.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:12.905+0000 7f6b3fe00640 1 --2- 172.21.15.152:0/669461508 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f6b1c06d0d0 0x7f6b1c06f590 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f6b30036000 tx=0x7f6b30030000 comp rx=0 tx=0).stop 2024-04-20T11:37:12.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:12.905+0000 7f6b3fe00640 1 -- 172.21.15.152:0/669461508 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6b40159d90 msgr2=0x7f6b4015a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:12.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:12.905+0000 7f6b3fe00640 1 --2- 172.21.15.152:0/669461508 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6b40159d90 0x7f6b4015a210 secure :-1 s=READY pgs=2178 cs=0 l=1 rev1=1 crypto rx=0x7f6b2800ca70 tx=0x7f6b2800cf30 comp rx=0 tx=0).stop 2024-04-20T11:37:12.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:12.905+0000 7f6b3fe00640 1 -- 172.21.15.152:0/669461508 shutdown_connections 2024-04-20T11:37:12.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:12.905+0000 7f6b3fe00640 1 --2- 172.21.15.152:0/669461508 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f6b1c06d0d0 0x7f6b1c06f590 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:12.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:12.905+0000 7f6b3fe00640 1 --2- 172.21.15.152:0/669461508 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b400b2480 0x7f6b400b2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:12.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:12.905+0000 7f6b3fe00640 1 --2- 172.21.15.152:0/669461508 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6b40159d90 0x7f6b4015a210 unknown :-1 s=CLOSED pgs=2178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:12.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:12.905+0000 7f6b3fe00640 1 --2- 172.21.15.152:0/669461508 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b200039b0 0x7f6b400b9d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:12.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:12.905+0000 7f6b3fe00640 1 -- 172.21.15.152:0/669461508 >> 172.21.15.152:0/669461508 conn(0x7f6b4008bfd0 msgr2=0x7f6b400b4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:12.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:12.905+0000 7f6b3fe00640 1 -- 172.21.15.152:0/669461508 shutdown_connections 2024-04-20T11:37:12.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:12.905+0000 7f6b3fe00640 1 -- 172.21.15.152:0/669461508 wait complete. 2024-04-20T11:37:12.923 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-20T11:37:13.122 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:13.123+0000 7f87b1400640 1 Processor -- start 2024-04-20T11:37:13.123 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:13.123+0000 7f87b1400640 1 -- start start 2024-04-20T11:37:13.124 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:13.125+0000 7f87b1400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f87ac0afba0 0x7f87ac162440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:13.124 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:13.125+0000 7f87b1400640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f87ac0b3fa0 0x7f87ac0ae110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:13.124 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:13.125+0000 7f87b1400640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87ac0adc30 con 0x7f87ac15e5d0 2024-04-20T11:37:13.124 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:13.125+0000 7f87b1400640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f87ac162a70 con 0x7f87ac0afba0 2024-04-20T11:37:13.124 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:13.125+0000 7f87b1400640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f87ac162d50 con 0x7f87ac0b3fa0 2024-04-20T11:37:13.125 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:13.125+0000 7f87a9e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f87ac0afba0 0x7f87ac162440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:13.126 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:13.125+0000 7f87a9e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f87ac0afba0 0x7f87ac162440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:59296/0 (socket says 172.21.15.152:59296) 2024-04-20T11:37:13.126 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:13.125+0000 7f87a9e00640 1 -- 172.21.15.152:0/3103293357 learned_addr learned my addr 172.21.15.152:0/3103293357 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:13.126 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:13.125+0000 7f87ab200640 1 --2- 172.21.15.152:0/3103293357 >> v2:172.21.15.175:3300/0 conn(0x7f87ac0b3fa0 0x7f87ac0ae110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:13.126 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:13.126+0000 7f87a9e00640 1 -- 172.21.15.152:0/3103293357 >> v2:172.21.15.175:3300/0 conn(0x7f87ac0b3fa0 msgr2=0x7f87ac0ae110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:13.126 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:13.126+0000 7f87a9e00640 1 --2- 172.21.15.152:0/3103293357 >> v2:172.21.15.175:3300/0 conn(0x7f87ac0b3fa0 0x7f87ac0ae110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:13.126 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:13.126+0000 7f87a9e00640 1 -- 172.21.15.152:0/3103293357 >> v1:172.21.15.175:6789/0 conn(0x7f87ac15e5d0 legacy=0x7f87ac0af490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:13.126 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:13.126+0000 7f87a9e00640 1 -- 172.21.15.152:0/3103293357 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87ac163030 con 0x7f87ac0afba0 2024-04-20T11:37:13.126 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:13.126+0000 7f87ab200640 1 --2- 172.21.15.152:0/3103293357 >> v2:172.21.15.175:3300/0 conn(0x7f87ac0b3fa0 0x7f87ac0ae110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:37:13.127 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:13.127+0000 7f87a9e00640 1 --2- 172.21.15.152:0/3103293357 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87ac0afba0 0x7f87ac162440 secure :-1 s=READY pgs=2179 cs=0 l=1 rev1=1 crypto rx=0x7f87ac0b5130 tx=0x7f879802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d5a48ca2f5909cc server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:13.127 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:13.127+0000 7f87a9400640 1 -- 172.21.15.152:0/3103293357 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8798035020 con 0x7f87ac0afba0 2024-04-20T11:37:13.127 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:13.127+0000 7f87a9400640 1 -- 172.21.15.152:0/3103293357 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8798031870 con 0x7f87ac0afba0 2024-04-20T11:37:13.127 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:13.127+0000 7f87a9400640 1 -- 172.21.15.152:0/3103293357 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8798031c40 con 0x7f87ac0afba0 2024-04-20T11:37:13.128 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:13.128+0000 7f87b1400640 1 -- 172.21.15.152:0/3103293357 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87ac0afba0 msgr2=0x7f87ac162440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:13.128 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:13.128+0000 7f87b1400640 1 --2- 172.21.15.152:0/3103293357 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87ac0afba0 0x7f87ac162440 secure :-1 s=READY pgs=2179 cs=0 l=1 rev1=1 crypto rx=0x7f87ac0b5130 tx=0x7f879802b910 comp rx=0 tx=0).stop 2024-04-20T11:37:13.128 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:13.128+0000 7f87b1400640 1 -- 172.21.15.152:0/3103293357 shutdown_connections 2024-04-20T11:37:13.128 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:13.128+0000 7f87b1400640 1 --2- 172.21.15.152:0/3103293357 >> v2:172.21.15.175:3300/0 conn(0x7f87ac0b3fa0 0x7f87ac0ae110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:13.128 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:13.128+0000 7f87b1400640 1 --2- 172.21.15.152:0/3103293357 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87ac0afba0 0x7f87ac162440 unknown :-1 s=CLOSED pgs=2179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:13.128 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:13.128+0000 7f87b1400640 1 -- 172.21.15.152:0/3103293357 >> 172.21.15.152:0/3103293357 conn(0x7f87ac06b980 msgr2=0x7f87ac06bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:13.128 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:13.128+0000 7f87b1400640 1 -- 172.21.15.152:0/3103293357 shutdown_connections 2024-04-20T11:37:13.128 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:13.128+0000 7f87b1400640 1 -- 172.21.15.152:0/3103293357 wait complete. 2024-04-20T11:37:13.130 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:13.131+0000 7f87b1400640 1 Processor -- start 2024-04-20T11:37:13.130 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:13.131+0000 7f87b1400640 1 -- start start 2024-04-20T11:37:13.132 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:13.132+0000 7f87b1400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f87ac0afba0 0x7f87ac160070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:13.132 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:13.132+0000 7f87b1400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87ac0b3fa0 0x7f87ac1605b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:13.132 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:13.132+0000 7f87b1400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87ac15e5d0 0x7f87ac1561b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:13.132 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:13.132+0000 7f87b1400640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f87ac06e0a0 con 0x7f87ac15e5d0 2024-04-20T11:37:13.133 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:13.132+0000 7f87b1400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f87ac163310 con 0x7f87ac0b3fa0 2024-04-20T11:37:13.133 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:13.132+0000 7f87b1400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f87ac06e220 con 0x7f87ac0afba0 2024-04-20T11:37:13.133 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:13.132+0000 7f87ab200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87ac15e5d0 0x7f87ac1561b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:13.133 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:13.132+0000 7f87ab200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87ac15e5d0 0x7f87ac1561b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:59300/0 (socket says 172.21.15.152:59300) 2024-04-20T11:37:13.133 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:13.132+0000 7f87ab200640 1 -- 172.21.15.152:0/3467918326 learned_addr learned my addr 172.21.15.152:0/3467918326 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:13.133 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:13.133+0000 7f87a9e00640 1 --2- 172.21.15.152:0/3467918326 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87ac0b3fa0 0x7f87ac1605b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:13.133 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:13.133+0000 7f87aa800640 1 --2- 172.21.15.152:0/3467918326 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f87ac0afba0 0x7f87ac160070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:13.133 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:13.133+0000 7f87ab200640 1 -- 172.21.15.152:0/3467918326 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f87ac0afba0 msgr2=0x7f87ac160070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:13.133 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:13.133+0000 7f87ab200640 1 --2- 172.21.15.152:0/3467918326 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f87ac0afba0 0x7f87ac160070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:13.133 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:13.133+0000 7f87ab200640 1 -- 172.21.15.152:0/3467918326 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87ac0b3fa0 msgr2=0x7f87ac1605b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:13.133 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:13.133+0000 7f87ab200640 1 --2- 172.21.15.152:0/3467918326 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87ac0b3fa0 0x7f87ac1605b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:13.133 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:13.133+0000 7f87ab200640 1 -- 172.21.15.152:0/3467918326 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87ac0ccf60 con 0x7f87ac15e5d0 2024-04-20T11:37:13.134 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:13.133+0000 7f87ab200640 1 --2- 172.21.15.152:0/3467918326 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87ac15e5d0 0x7f87ac1561b0 secure :-1 s=READY pgs=2180 cs=0 l=1 rev1=1 crypto rx=0x7f87a400cef0 tx=0x7f87a400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:13.134 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:13.133+0000 7f87a2a00640 1 -- 172.21.15.152:0/3467918326 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87a401a070 con 0x7f87ac15e5d0 2024-04-20T11:37:13.135 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:13.133+0000 7f87a2a00640 1 -- 172.21.15.152:0/3467918326 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87a400d860 con 0x7f87ac15e5d0 2024-04-20T11:37:13.135 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:13.133+0000 7f87a2a00640 1 -- 172.21.15.152:0/3467918326 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87a400dc50 con 0x7f87ac15e5d0 2024-04-20T11:37:13.135 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:13.134+0000 7f87b1400640 1 -- 172.21.15.152:0/3467918326 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87ac1612b0 con 0x7f87ac15e5d0 2024-04-20T11:37:13.135 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:13.134+0000 7f87b1400640 1 -- 172.21.15.152:0/3467918326 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87ac161870 con 0x7f87ac15e5d0 2024-04-20T11:37:13.135 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:13.135+0000 7f87b1400640 1 -- 172.21.15.152:0/3467918326 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87ac0689f0 con 0x7f87ac15e5d0 2024-04-20T11:37:13.135 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:13.136+0000 7f87a2a00640 1 -- 172.21.15.152:0/3467918326 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) ==== 91754+0+0 (secure 0 0 0) 0x7f87a401d070 con 0x7f87ac15e5d0 2024-04-20T11:37:13.136 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:13.137+0000 7f87a2a00640 1 --2- 172.21.15.152:0/3467918326 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f878c06d0d0 0x7f878c06f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:13.136 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:13.137+0000 7f87a2a00640 1 -- 172.21.15.152:0/3467918326 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(154..154 src has 1..154) ==== 5709+0+0 (secure 0 0 0) 0x7f87a408d670 con 0x7f87ac15e5d0 2024-04-20T11:37:13.137 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:13.137+0000 7f87aa800640 1 --2- 172.21.15.152:0/3467918326 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f878c06d0d0 0x7f878c06f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:13.138 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:13.139+0000 7f87aa800640 1 --2- 172.21.15.152:0/3467918326 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f878c06d0d0 0x7f878c06f590 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f879c006fd0 tx=0x7f879c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:13.140 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:13.141+0000 7f87a2a00640 1 -- 172.21.15.152:0/3467918326 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f87a405ac00 con 0x7f87ac15e5d0 2024-04-20T11:37:13.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:13.338+0000 7f87b1400640 1 -- 172.21.15.152:0/3467918326 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f87ac160fd0 con 0x7f87ac15e5d0 2024-04-20T11:37:13.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:13.339+0000 7f87a2a00640 1 -- 172.21.15.152:0/3467918326 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v28)=0 v28) ==== 122+0+0 (secure 0 0 0) 0x7f87ac160fd0 con 0x7f87ac15e5d0 2024-04-20T11:37:13.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:13.342+0000 7f87b1400640 1 -- 172.21.15.152:0/3467918326 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f878c06d0d0 msgr2=0x7f878c06f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:13.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:13.342+0000 7f87b1400640 1 --2- 172.21.15.152:0/3467918326 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f878c06d0d0 0x7f878c06f590 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f879c006fd0 tx=0x7f879c008040 comp rx=0 tx=0).stop 2024-04-20T11:37:13.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:13.342+0000 7f87b1400640 1 -- 172.21.15.152:0/3467918326 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87ac15e5d0 msgr2=0x7f87ac1561b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:13.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:13.342+0000 7f87b1400640 1 --2- 172.21.15.152:0/3467918326 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87ac15e5d0 0x7f87ac1561b0 secure :-1 s=READY pgs=2180 cs=0 l=1 rev1=1 crypto rx=0x7f87a400cef0 tx=0x7f87a400a710 comp rx=0 tx=0).stop 2024-04-20T11:37:13.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:13.342+0000 7f87b1400640 1 -- 172.21.15.152:0/3467918326 shutdown_connections 2024-04-20T11:37:13.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:13.342+0000 7f87b1400640 1 --2- 172.21.15.152:0/3467918326 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f878c06d0d0 0x7f878c06f590 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:13.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:13.342+0000 7f87b1400640 1 --2- 172.21.15.152:0/3467918326 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87ac15e5d0 0x7f87ac1561b0 unknown :-1 s=CLOSED pgs=2180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:13.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:13.342+0000 7f87b1400640 1 --2- 172.21.15.152:0/3467918326 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87ac0b3fa0 0x7f87ac1605b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:13.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:13.342+0000 7f87b1400640 1 --2- 172.21.15.152:0/3467918326 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f87ac0afba0 0x7f87ac160070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:13.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:13.342+0000 7f87b1400640 1 -- 172.21.15.152:0/3467918326 >> 172.21.15.152:0/3467918326 conn(0x7f87ac06b980 msgr2=0x7f87ac158c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:13.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:13.342+0000 7f87b1400640 1 -- 172.21.15.152:0/3467918326 shutdown_connections 2024-04-20T11:37:13.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:13.342+0000 7f87b1400640 1 -- 172.21.15.152:0/3467918326 wait complete. 2024-04-20T11:37:13.361 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:37:13.361 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/_nogroup/subvol90290/97eba81a-1987-479d-93eb-7666caa1afa9) 2024-04-20T11:37:13.460 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:37:13.460 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2024-04-20T11:37:13.510 INFO:teuthology.orchestra.run.smithi152.stdout:cephfs_data 2024-04-20T11:37:13.511 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:37:13.511 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data volumes/_nogroup/subvol90290/97eba81a-1987-479d-93eb-7666caa1afa9) 2024-04-20T11:37:13.595 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-20T11:37:13.595 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/home/ubuntu/cephtest/mnt.0/volumes/_nogroup/subvol90290/.meta 2024-04-20T11:37:13.680 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:37:13.680 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/subvol_grp58840/subvol90151/a34d7ecf-a8f8-4ee8-bfd4-b979dfcfaf2f) 2024-04-20T11:37:13.820 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:37:13.820 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2024-04-20T11:37:13.875 INFO:teuthology.orchestra.run.smithi152.stdout:cephfs_data 2024-04-20T11:37:13.875 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:37:13.875 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data volumes/subvol_grp58840/subvol90151/a34d7ecf-a8f8-4ee8-bfd4-b979dfcfaf2f) 2024-04-20T11:37:13.964 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-20T11:37:13.964 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/home/ubuntu/cephtest/mnt.0/volumes/subvol_grp58840/subvol90151/.meta 2024-04-20T11:37:14.052 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol90290 2024-04-20T11:37:14.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.306+0000 7fd596e00640 1 Processor -- start 2024-04-20T11:37:14.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.306+0000 7fd596e00640 1 -- start start 2024-04-20T11:37:14.308 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.308+0000 7fd596e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fd590160220 0x7fd5901606b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:14.308 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.308+0000 7fd596e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd590141450 0x7fd5901418e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:14.308 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.308+0000 7fd596e00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd59006fb40 con 0x7fd590071cd0 2024-04-20T11:37:14.308 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.308+0000 7fd596e00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fd59013f8e0 con 0x7fd590160220 2024-04-20T11:37:14.308 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.308+0000 7fd596e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd590160f90 con 0x7fd590141450 2024-04-20T11:37:14.308 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.308+0000 7fd595000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fd590160220 0x7fd5901606b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:14.308 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.308+0000 7fd595a00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fd590071cd0 0x7fd5900720b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:43956/0 (socket says 172.21.15.152:43956) 2024-04-20T11:37:14.308 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.308+0000 7fd595a00640 1 -- 172.21.15.152:0/4065957787 learned_addr learned my addr 172.21.15.152:0/4065957787 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:37:14.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.309+0000 7fd596400640 1 --2- 172.21.15.152:0/4065957787 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd590141450 0x7fd5901418e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:14.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.309+0000 7fd595000640 1 -- 172.21.15.152:0/4065957787 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd590141450 msgr2=0x7fd5901418e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:14.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.309+0000 7fd595000640 1 --2- 172.21.15.152:0/4065957787 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd590141450 0x7fd5901418e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:14.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.309+0000 7fd595000640 1 -- 172.21.15.152:0/4065957787 >> v1:172.21.15.152:6789/0 conn(0x7fd590071cd0 legacy=0x7fd5900720b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:14.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.309+0000 7fd595000640 1 -- 172.21.15.152:0/4065957787 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd59006dd90 con 0x7fd590160220 2024-04-20T11:37:14.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.309+0000 7fd58fe00640 1 -- 172.21.15.152:0/4065957787 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 196061854 0 0) 0x7fd59006fb40 con 0x7fd590071cd0 2024-04-20T11:37:14.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.310+0000 7fd595000640 1 --2- 172.21.15.152:0/4065957787 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd590160220 0x7fd5901606b0 secure :-1 s=READY pgs=2182 cs=0 l=1 rev1=1 crypto rx=0x7fd57c0097d0 tx=0x7fd57c02b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6564f12576959597 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:14.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.310+0000 7fd58fe00640 1 -- 172.21.15.152:0/4065957787 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd57c031770 con 0x7fd590160220 2024-04-20T11:37:14.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.311+0000 7fd58fe00640 1 -- 172.21.15.152:0/4065957787 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd57c037070 con 0x7fd590160220 2024-04-20T11:37:14.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.311+0000 7fd58fe00640 1 -- 172.21.15.152:0/4065957787 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd57c031b30 con 0x7fd590160220 2024-04-20T11:37:14.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.311+0000 7fd596e00640 1 -- 172.21.15.152:0/4065957787 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd590160220 msgr2=0x7fd5901606b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:14.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.311+0000 7fd596e00640 1 --2- 172.21.15.152:0/4065957787 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd590160220 0x7fd5901606b0 secure :-1 s=READY pgs=2182 cs=0 l=1 rev1=1 crypto rx=0x7fd57c0097d0 tx=0x7fd57c02b810 comp rx=0 tx=0).stop 2024-04-20T11:37:14.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.311+0000 7fd596e00640 1 -- 172.21.15.152:0/4065957787 shutdown_connections 2024-04-20T11:37:14.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.311+0000 7fd596e00640 1 --2- 172.21.15.152:0/4065957787 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd590141450 0x7fd5901418e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:14.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.311+0000 7fd596e00640 1 --2- 172.21.15.152:0/4065957787 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd590160220 0x7fd5901606b0 unknown :-1 s=CLOSED pgs=2182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:14.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.311+0000 7fd596e00640 1 -- 172.21.15.152:0/4065957787 >> 172.21.15.152:0/4065957787 conn(0x7fd590067ae0 msgr2=0x7fd59006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:14.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.311+0000 7fd596e00640 1 -- 172.21.15.152:0/4065957787 shutdown_connections 2024-04-20T11:37:14.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.311+0000 7fd596e00640 1 -- 172.21.15.152:0/4065957787 wait complete. 2024-04-20T11:37:14.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.313+0000 7fd596e00640 1 Processor -- start 2024-04-20T11:37:14.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.313+0000 7fd596e00640 1 -- start start 2024-04-20T11:37:14.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.314+0000 7fd596e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd590071cd0 0x7fd590132a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:14.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.314+0000 7fd596e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd590141450 0x7fd59015ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:14.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.314+0000 7fd596e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd590160220 0x7fd59015f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:14.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.314+0000 7fd596e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fd59006e630 con 0x7fd590071cd0 2024-04-20T11:37:14.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.314+0000 7fd596e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd59006e350 con 0x7fd590141450 2024-04-20T11:37:14.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.314+0000 7fd596e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd590160bf0 con 0x7fd590160220 2024-04-20T11:37:14.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.314+0000 7fd595a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd590071cd0 0x7fd590132a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:14.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.314+0000 7fd595a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd590071cd0 0x7fd590132a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:59322/0 (socket says 172.21.15.152:59322) 2024-04-20T11:37:14.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.314+0000 7fd595a00640 1 -- 172.21.15.152:0/3554338834 learned_addr learned my addr 172.21.15.152:0/3554338834 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:14.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.315+0000 7fd595000640 1 --2- 172.21.15.152:0/3554338834 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd590141450 0x7fd59015ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:14.317 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.315+0000 7fd596400640 1 --2- 172.21.15.152:0/3554338834 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd590160220 0x7fd59015f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:14.317 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.315+0000 7fd595a00640 1 -- 172.21.15.152:0/3554338834 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd590160220 msgr2=0x7fd59015f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:14.317 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.315+0000 7fd595a00640 1 --2- 172.21.15.152:0/3554338834 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd590160220 0x7fd59015f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:14.317 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.315+0000 7fd595a00640 1 -- 172.21.15.152:0/3554338834 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd590141450 msgr2=0x7fd59015ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:14.317 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.315+0000 7fd595a00640 1 --2- 172.21.15.152:0/3554338834 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd590141450 0x7fd59015ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:14.317 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.315+0000 7fd595a00640 1 -- 172.21.15.152:0/3554338834 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5900aa460 con 0x7fd590071cd0 2024-04-20T11:37:14.317 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.315+0000 7fd595000640 1 --2- 172.21.15.152:0/3554338834 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd590141450 0x7fd59015ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:37:14.317 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.315+0000 7fd595a00640 1 --2- 172.21.15.152:0/3554338834 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd590071cd0 0x7fd590132a80 secure :-1 s=READY pgs=2183 cs=0 l=1 rev1=1 crypto rx=0x7fd58400c4d0 tx=0x7fd58400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:14.317 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.315+0000 7fd58d600640 1 -- 172.21.15.152:0/3554338834 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd584018070 con 0x7fd590071cd0 2024-04-20T11:37:14.317 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.315+0000 7fd58d600640 1 -- 172.21.15.152:0/3554338834 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd584007500 con 0x7fd590071cd0 2024-04-20T11:37:14.317 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.315+0000 7fd596e00640 1 -- 172.21.15.152:0/3554338834 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5900aa740 con 0x7fd590071cd0 2024-04-20T11:37:14.317 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.315+0000 7fd58d600640 1 -- 172.21.15.152:0/3554338834 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd584004a20 con 0x7fd590071cd0 2024-04-20T11:37:14.318 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.316+0000 7fd596e00640 1 -- 172.21.15.152:0/3554338834 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5900aad00 con 0x7fd590071cd0 2024-04-20T11:37:14.318 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.317+0000 7fd596e00640 1 -- 172.21.15.152:0/3554338834 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd590075e60 con 0x7fd590071cd0 2024-04-20T11:37:14.318 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.318+0000 7fd58d600640 1 -- 172.21.15.152:0/3554338834 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) ==== 91754+0+0 (secure 0 0 0) 0x7fd58401b070 con 0x7fd590071cd0 2024-04-20T11:37:14.319 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.319+0000 7fd58d600640 1 --2- 172.21.15.152:0/3554338834 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd57006d0d0 0x7fd57006f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:14.319 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.319+0000 7fd58d600640 1 -- 172.21.15.152:0/3554338834 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(154..154 src has 1..154) ==== 5709+0+0 (secure 0 0 0) 0x7fd58408c190 con 0x7fd590071cd0 2024-04-20T11:37:14.319 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.319+0000 7fd595000640 1 --2- 172.21.15.152:0/3554338834 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd57006d0d0 0x7fd57006f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:14.320 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.321+0000 7fd595000640 1 --2- 172.21.15.152:0/3554338834 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd57006d0d0 0x7fd57006f590 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fd57c036000 tx=0x7fd57c008030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:14.323 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.323+0000 7fd58d600640 1 -- 172.21.15.152:0/3554338834 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fd5840595a0 con 0x7fd590071cd0 2024-04-20T11:37:14.558 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.558+0000 7fd596e00640 1 -- 172.21.15.152:0/3554338834 --> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol90290", "target": ["mon-mgr", ""]}) -- 0x7fd59006c820 con 0x7fd57006d0d0 2024-04-20T11:37:14.570 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:37:14.570+0000 7f8a5cc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:37:14.571 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:37:14.570+0000 7f8a5cc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:37:14.571 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:37:14.570+0000 7f8a5cc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:37:14.571 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:37:14.570+0000 7f8a5cc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:37:14.571 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:37:14.570+0000 7f8a5cc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:37:14.692 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.692+0000 7fd58d600640 1 -- 172.21.15.152:0/3554338834 <== mgr.4109 v2:172.21.15.175:6836/4021250441 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+67 (secure 0 0 0) 0x7fd59006c820 con 0x7fd57006d0d0 2024-04-20T11:37:14.692 INFO:teuthology.orchestra.run.smithi152.stdout:/volumes/_nogroup/subvol90290/97eba81a-1987-479d-93eb-7666caa1afa9 2024-04-20T11:37:14.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.696+0000 7fd596e00640 1 -- 172.21.15.152:0/3554338834 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd57006d0d0 msgr2=0x7fd57006f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:14.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.696+0000 7fd596e00640 1 --2- 172.21.15.152:0/3554338834 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd57006d0d0 0x7fd57006f590 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fd57c036000 tx=0x7fd57c008030 comp rx=0 tx=0).stop 2024-04-20T11:37:14.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.696+0000 7fd596e00640 1 -- 172.21.15.152:0/3554338834 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd590071cd0 msgr2=0x7fd590132a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:14.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.696+0000 7fd596e00640 1 --2- 172.21.15.152:0/3554338834 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd590071cd0 0x7fd590132a80 secure :-1 s=READY pgs=2183 cs=0 l=1 rev1=1 crypto rx=0x7fd58400c4d0 tx=0x7fd58400c990 comp rx=0 tx=0).stop 2024-04-20T11:37:14.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.696+0000 7fd596e00640 1 -- 172.21.15.152:0/3554338834 shutdown_connections 2024-04-20T11:37:14.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.696+0000 7fd596e00640 1 --2- 172.21.15.152:0/3554338834 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd57006d0d0 0x7fd57006f590 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:14.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.696+0000 7fd596e00640 1 --2- 172.21.15.152:0/3554338834 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd590160220 0x7fd59015f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:14.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.696+0000 7fd596e00640 1 --2- 172.21.15.152:0/3554338834 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd590141450 0x7fd59015ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:14.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.696+0000 7fd596e00640 1 --2- 172.21.15.152:0/3554338834 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd590071cd0 0x7fd590132a80 unknown :-1 s=CLOSED pgs=2183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:14.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.696+0000 7fd596e00640 1 -- 172.21.15.152:0/3554338834 >> 172.21.15.152:0/3554338834 conn(0x7fd590067ae0 msgr2=0x7fd59015dd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:14.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.696+0000 7fd596e00640 1 -- 172.21.15.152:0/3554338834 shutdown_connections 2024-04-20T11:37:14.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.696+0000 7fd596e00640 1 -- 172.21.15.152:0/3554338834 wait complete. 2024-04-20T11:37:14.718 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol90151 subvol_grp58840 2024-04-20T11:37:14.925 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.925+0000 7f139be00640 1 Processor -- start 2024-04-20T11:37:14.925 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.925+0000 7f139be00640 1 -- start start 2024-04-20T11:37:14.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.927+0000 7f139be00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f139c0d05f0 0x7f139c0d0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:14.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.927+0000 7f139be00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f139c15d8d0 con 0x7f139c1569d0 2024-04-20T11:37:14.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.927+0000 7f139be00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f139c15d310 con 0x7f139c0ad760 2024-04-20T11:37:14.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.927+0000 7f139be00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f139c0d0f80 con 0x7f139c0d05f0 2024-04-20T11:37:14.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.927+0000 7f139aa00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f139c1569d0 0x7f139c06e510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:43966/0 (socket says 172.21.15.152:43966) 2024-04-20T11:37:14.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.927+0000 7f139aa00640 1 -- 172.21.15.152:0/172601162 learned_addr learned my addr 172.21.15.152:0/172601162 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:37:14.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.927+0000 7f139b400640 1 --2- 172.21.15.152:0/172601162 >> v2:172.21.15.152:3300/0 conn(0x7f139c0d05f0 0x7f139c0d0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:14.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.927+0000 7f1399600640 1 -- 172.21.15.152:0/172601162 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3059329527 0 0) 0x7f139c15d8d0 con 0x7f139c1569d0 2024-04-20T11:37:14.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.928+0000 7f1399600640 1 -- 172.21.15.152:0/172601162 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1380003520 con 0x7f139c1569d0 2024-04-20T11:37:14.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.928+0000 7f139b400640 1 -- 172.21.15.152:0/172601162 >> v1:172.21.15.175:6789/0 conn(0x7f139c0ad760 legacy=0x7f139c15ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:14.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.928+0000 7f139b400640 1 -- 172.21.15.152:0/172601162 >> v1:172.21.15.152:6789/0 conn(0x7f139c1569d0 legacy=0x7f139c06e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:14.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.928+0000 7f139b400640 1 -- 172.21.15.152:0/172601162 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f139c15d5f0 con 0x7f139c0d05f0 2024-04-20T11:37:14.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.929+0000 7f139b400640 1 --2- 172.21.15.152:0/172601162 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f139c0d05f0 0x7f139c0d0a40 secure :-1 s=READY pgs=2185 cs=0 l=1 rev1=1 crypto rx=0x7f1394009d90 tx=0x7f139402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3069f8e639373a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:14.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.929+0000 7f1399600640 1 -- 172.21.15.152:0/172601162 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1394039070 con 0x7f139c0d05f0 2024-04-20T11:37:14.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.929+0000 7f1399600640 1 -- 172.21.15.152:0/172601162 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f139402cc60 con 0x7f139c0d05f0 2024-04-20T11:37:14.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.929+0000 7f1399600640 1 -- 172.21.15.152:0/172601162 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1394034660 con 0x7f139c0d05f0 2024-04-20T11:37:14.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.930+0000 7f139be00640 1 -- 172.21.15.152:0/172601162 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f139c0d05f0 msgr2=0x7f139c0d0a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:14.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.930+0000 7f139be00640 1 --2- 172.21.15.152:0/172601162 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f139c0d05f0 0x7f139c0d0a40 secure :-1 s=READY pgs=2185 cs=0 l=1 rev1=1 crypto rx=0x7f1394009d90 tx=0x7f139402beb0 comp rx=0 tx=0).stop 2024-04-20T11:37:14.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.930+0000 7f139be00640 1 -- 172.21.15.152:0/172601162 shutdown_connections 2024-04-20T11:37:14.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.930+0000 7f139be00640 1 --2- 172.21.15.152:0/172601162 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f139c0d05f0 0x7f139c0d0a40 unknown :-1 s=CLOSED pgs=2185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:14.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.930+0000 7f139be00640 1 -- 172.21.15.152:0/172601162 >> 172.21.15.152:0/172601162 conn(0x7f139c0b21e0 msgr2=0x7f139c0b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:14.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.930+0000 7f139be00640 1 -- 172.21.15.152:0/172601162 shutdown_connections 2024-04-20T11:37:14.931 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.930+0000 7f139be00640 1 -- 172.21.15.152:0/172601162 wait complete. 2024-04-20T11:37:14.931 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.932+0000 7f139be00640 1 Processor -- start 2024-04-20T11:37:14.931 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.932+0000 7f139be00640 1 -- start start 2024-04-20T11:37:14.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.933+0000 7f139be00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f139c0ad760 0x7f139c15b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:14.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.933+0000 7f139be00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f139c0d05f0 0x7f139c1f6d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:14.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.933+0000 7f139be00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f139c1569d0 0x7f139c1f72c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:14.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.933+0000 7f139be00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f139c156db0 con 0x7f139c1569d0 2024-04-20T11:37:14.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.933+0000 7f139be00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f139c06dff0 con 0x7f139c0d05f0 2024-04-20T11:37:14.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.933+0000 7f139be00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f139c0b30c0 con 0x7f139c0ad760 2024-04-20T11:37:14.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.933+0000 7f139b400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f139c1569d0 0x7f139c1f72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:14.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.933+0000 7f139b400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f139c1569d0 0x7f139c1f72c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:59342/0 (socket says 172.21.15.152:59342) 2024-04-20T11:37:14.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.933+0000 7f139b400640 1 -- 172.21.15.152:0/3412424736 learned_addr learned my addr 172.21.15.152:0/3412424736 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:14.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.933+0000 7f139a000640 1 --2- 172.21.15.152:0/3412424736 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f139c0d05f0 0x7f139c1f6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:14.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.933+0000 7f139b400640 1 -- 172.21.15.152:0/3412424736 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f139c0ad760 msgr2=0x7f139c15b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:14.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.933+0000 7f139b400640 1 --2- 172.21.15.152:0/3412424736 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f139c0ad760 0x7f139c15b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:14.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.933+0000 7f139b400640 1 -- 172.21.15.152:0/3412424736 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f139c0d05f0 msgr2=0x7f139c1f6d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:14.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.933+0000 7f139b400640 1 --2- 172.21.15.152:0/3412424736 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f139c0d05f0 0x7f139c1f6d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:14.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.933+0000 7f139b400640 1 -- 172.21.15.152:0/3412424736 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f139c15c640 con 0x7f139c1569d0 2024-04-20T11:37:14.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.933+0000 7f139a000640 1 --2- 172.21.15.152:0/3412424736 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f139c0d05f0 0x7f139c1f6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:37:14.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.934+0000 7f139b400640 1 --2- 172.21.15.152:0/3412424736 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f139c1569d0 0x7f139c1f72c0 secure :-1 s=READY pgs=2186 cs=0 l=1 rev1=1 crypto rx=0x7f1394009d90 tx=0x7f1394004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:14.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.934+0000 7f1392a00640 1 -- 172.21.15.152:0/3412424736 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1394039050 con 0x7f139c1569d0 2024-04-20T11:37:14.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.934+0000 7f1392a00640 1 -- 172.21.15.152:0/3412424736 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1394045070 con 0x7f139c1569d0 2024-04-20T11:37:14.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.934+0000 7f1392a00640 1 -- 172.21.15.152:0/3412424736 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1394009450 con 0x7f139c1569d0 2024-04-20T11:37:14.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.934+0000 7f139be00640 1 -- 172.21.15.152:0/3412424736 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f139c157180 con 0x7f139c1569d0 2024-04-20T11:37:14.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.934+0000 7f139be00640 1 -- 172.21.15.152:0/3412424736 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f139c15bd80 con 0x7f139c1569d0 2024-04-20T11:37:14.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.935+0000 7f139be00640 1 -- 172.21.15.152:0/3412424736 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f139c15c810 con 0x7f139c1569d0 2024-04-20T11:37:14.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.936+0000 7f1392a00640 1 -- 172.21.15.152:0/3412424736 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) ==== 91754+0+0 (secure 0 0 0) 0x7f139403e850 con 0x7f139c1569d0 2024-04-20T11:37:14.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.937+0000 7f1392a00640 1 --2- 172.21.15.152:0/3412424736 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f137c06d0d0 0x7f137c06f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:14.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.937+0000 7f1392a00640 1 -- 172.21.15.152:0/3412424736 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(154..154 src has 1..154) ==== 5709+0+0 (secure 0 0 0) 0x7f13940afc70 con 0x7f139c1569d0 2024-04-20T11:37:14.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.938+0000 7f139aa00640 1 --2- 172.21.15.152:0/3412424736 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f137c06d0d0 0x7f137c06f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:14.939 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.939+0000 7f139aa00640 1 --2- 172.21.15.152:0/3412424736 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f137c06d0d0 0x7f137c06f590 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f138c000f60 tx=0x7f138c0078f0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:14.948 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:14.949+0000 7f1392a00640 1 -- 172.21.15.152:0/3412424736 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f139407d1d0 con 0x7f139c1569d0 2024-04-20T11:37:15.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:15.180+0000 7f139be00640 1 -- 172.21.15.152:0/3412424736 --> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol90151", "group_name": "subvol_grp58840", "target": ["mon-mgr", ""]}) -- 0x7f139c0b44c0 con 0x7f137c06d0d0 2024-04-20T11:37:15.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:15.294+0000 7f1392a00640 1 -- 172.21.15.152:0/3412424736 <== mgr.4109 v2:172.21.15.175:6836/4021250441 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+74 (secure 0 0 0) 0x7f138c005890 con 0x7f137c06d0d0 2024-04-20T11:37:15.294 INFO:teuthology.orchestra.run.smithi152.stdout:/volumes/subvol_grp58840/subvol90151/a34d7ecf-a8f8-4ee8-bfd4-b979dfcfaf2f 2024-04-20T11:37:15.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:15.297+0000 7f139be00640 1 -- 172.21.15.152:0/3412424736 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f137c06d0d0 msgr2=0x7f137c06f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:15.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:15.297+0000 7f139be00640 1 --2- 172.21.15.152:0/3412424736 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f137c06d0d0 0x7f137c06f590 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f138c000f60 tx=0x7f138c0078f0 comp rx=0 tx=0).stop 2024-04-20T11:37:15.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:15.298+0000 7f139be00640 1 -- 172.21.15.152:0/3412424736 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f139c1569d0 msgr2=0x7f139c1f72c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:15.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:15.298+0000 7f139be00640 1 --2- 172.21.15.152:0/3412424736 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f139c1569d0 0x7f139c1f72c0 secure :-1 s=READY pgs=2186 cs=0 l=1 rev1=1 crypto rx=0x7f1394009d90 tx=0x7f1394004a40 comp rx=0 tx=0).stop 2024-04-20T11:37:15.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:15.298+0000 7f139be00640 1 -- 172.21.15.152:0/3412424736 shutdown_connections 2024-04-20T11:37:15.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:15.298+0000 7f139be00640 1 --2- 172.21.15.152:0/3412424736 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f137c06d0d0 0x7f137c06f590 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:15.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:15.298+0000 7f139be00640 1 --2- 172.21.15.152:0/3412424736 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f139c1569d0 0x7f139c1f72c0 unknown :-1 s=CLOSED pgs=2186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:15.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:15.298+0000 7f139be00640 1 --2- 172.21.15.152:0/3412424736 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f139c0d05f0 0x7f139c1f6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:15.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:15.298+0000 7f139be00640 1 --2- 172.21.15.152:0/3412424736 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f139c0ad760 0x7f139c15b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:15.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:15.298+0000 7f139be00640 1 -- 172.21.15.152:0/3412424736 >> 172.21.15.152:0/3412424736 conn(0x7f139c0b21e0 msgr2=0x7f139c06e510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:15.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:15.298+0000 7f139be00640 1 -- 172.21.15.152:0/3412424736 shutdown_connections 2024-04-20T11:37:15.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:15.298+0000 7f139be00640 1 -- 172.21.15.152:0/3412424736 wait complete. 2024-04-20T11:37:15.319 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:37:15.319 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_nogroup/subvol90290/.meta) 2024-04-20T11:37:15.399 INFO:teuthology.orchestra.run.smithi152.stdout:[GLOBAL] 2024-04-20T11:37:15.399 INFO:teuthology.orchestra.run.smithi152.stdout:version = 2 2024-04-20T11:37:15.399 INFO:teuthology.orchestra.run.smithi152.stdout:type = subvolume 2024-04-20T11:37:15.399 INFO:teuthology.orchestra.run.smithi152.stdout:path = /volumes/_nogroup/subvol90290/97eba81a-1987-479d-93eb-7666caa1afa9 2024-04-20T11:37:15.399 INFO:teuthology.orchestra.run.smithi152.stdout:state = complete 2024-04-20T11:37:15.399 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:37:15.403 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:37:15.403 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/subvol_grp58840/subvol90151/.meta) 2024-04-20T11:37:15.485 INFO:teuthology.orchestra.run.smithi152.stdout:[GLOBAL] 2024-04-20T11:37:15.485 INFO:teuthology.orchestra.run.smithi152.stdout:version = 2 2024-04-20T11:37:15.486 INFO:teuthology.orchestra.run.smithi152.stdout:type = subvolume 2024-04-20T11:37:15.486 INFO:teuthology.orchestra.run.smithi152.stdout:path = /volumes/subvol_grp58840/subvol90151/a34d7ecf-a8f8-4ee8-bfd4-b979dfcfaf2f 2024-04-20T11:37:15.486 INFO:teuthology.orchestra.run.smithi152.stdout:state = complete 2024-04-20T11:37:15.486 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:37:15.490 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol90290 2024-04-20T11:37:15.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:15.686+0000 7fdd77c00640 1 Processor -- start 2024-04-20T11:37:15.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:15.686+0000 7fdd77c00640 1 -- start start 2024-04-20T11:37:15.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:15.687+0000 7fdd77c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd70162bc0 0x7fdd70167240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:15.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:15.688+0000 7fdd77c00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fdd700b2880 0x7fdd7006e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:15.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:15.688+0000 7fdd77c00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd70167780 con 0x7fdd7006f720 2024-04-20T11:37:15.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:15.688+0000 7fdd77c00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fdd700ccb40 con 0x7fdd700b2880 2024-04-20T11:37:15.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:15.688+0000 7fdd77c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fdd700b3610 con 0x7fdd70162bc0 2024-04-20T11:37:15.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:15.688+0000 7fdd6fe00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fdd7006f720 0x7fdd700b2170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:43976/0 (socket says 172.21.15.152:43976) 2024-04-20T11:37:15.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:15.688+0000 7fdd6fe00640 1 -- 172.21.15.152:0/895821845 learned_addr learned my addr 172.21.15.152:0/895821845 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:37:15.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:15.688+0000 7fdd75c00640 1 --2- 172.21.15.152:0/895821845 >> v2:172.21.15.152:3300/0 conn(0x7fdd700b2880 0x7fdd7006e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:15.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:15.688+0000 7fdd75200640 1 --2- 172.21.15.152:0/895821845 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd70162bc0 0x7fdd70167240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:15.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:15.688+0000 7fdd6f400640 1 -- 172.21.15.152:0/895821845 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1553192084 0 0) 0x7fdd5c001110 con 0x7fdd7006f720 2024-04-20T11:37:15.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:15.688+0000 7fdd75c00640 1 -- 172.21.15.152:0/895821845 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd70162bc0 msgr2=0x7fdd70167240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:15.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:15.688+0000 7fdd75c00640 1 --2- 172.21.15.152:0/895821845 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd70162bc0 0x7fdd70167240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:15.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:15.688+0000 7fdd75c00640 1 -- 172.21.15.152:0/895821845 >> v1:172.21.15.152:6789/0 conn(0x7fdd7006f720 legacy=0x7fdd700b2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:15.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:15.688+0000 7fdd75c00640 1 -- 172.21.15.152:0/895821845 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd7006fc30 con 0x7fdd700b2880 2024-04-20T11:37:15.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:15.690+0000 7fdd75c00640 1 --2- 172.21.15.152:0/895821845 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdd700b2880 0x7fdd7006e010 secure :-1 s=READY pgs=2188 cs=0 l=1 rev1=1 crypto rx=0x7fdd68009d90 tx=0x7fdd6802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca1f41fa6bcfa1ba server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:15.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:15.690+0000 7fdd6f400640 1 -- 172.21.15.152:0/895821845 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd68039070 con 0x7fdd700b2880 2024-04-20T11:37:15.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:15.690+0000 7fdd6f400640 1 -- 172.21.15.152:0/895821845 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd6802cc60 con 0x7fdd700b2880 2024-04-20T11:37:15.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:15.690+0000 7fdd6f400640 1 -- 172.21.15.152:0/895821845 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd68034660 con 0x7fdd700b2880 2024-04-20T11:37:15.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:15.690+0000 7fdd77c00640 1 -- 172.21.15.152:0/895821845 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdd700b2880 msgr2=0x7fdd7006e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:15.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:15.690+0000 7fdd77c00640 1 --2- 172.21.15.152:0/895821845 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdd700b2880 0x7fdd7006e010 secure :-1 s=READY pgs=2188 cs=0 l=1 rev1=1 crypto rx=0x7fdd68009d90 tx=0x7fdd6802beb0 comp rx=0 tx=0).stop 2024-04-20T11:37:15.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:15.691+0000 7fdd77c00640 1 -- 172.21.15.152:0/895821845 shutdown_connections 2024-04-20T11:37:15.691 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:15.691+0000 7fdd77c00640 1 --2- 172.21.15.152:0/895821845 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdd700b2880 0x7fdd7006e010 unknown :-1 s=CLOSED pgs=2188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:15.691 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:15.691+0000 7fdd77c00640 1 --2- 172.21.15.152:0/895821845 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd70162bc0 0x7fdd70167240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:15.691 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:15.691+0000 7fdd77c00640 1 -- 172.21.15.152:0/895821845 >> 172.21.15.152:0/895821845 conn(0x7fdd7006b980 msgr2=0x7fdd7006bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:15.691 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:15.691+0000 7fdd77c00640 1 -- 172.21.15.152:0/895821845 shutdown_connections 2024-04-20T11:37:15.691 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:15.691+0000 7fdd77c00640 1 -- 172.21.15.152:0/895821845 wait complete. 2024-04-20T11:37:15.692 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:15.692+0000 7fdd77c00640 1 Processor -- start 2024-04-20T11:37:15.692 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:15.693+0000 7fdd77c00640 1 -- start start 2024-04-20T11:37:15.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:15.693+0000 7fdd77c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdd7006f720 0x7fdd701496c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:15.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:15.693+0000 7fdd77c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd700b2880 0x7fdd70153390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:15.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:15.693+0000 7fdd77c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd70162bc0 0x7fdd70155f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:15.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:15.693+0000 7fdd77c00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fdd700b4650 con 0x7fdd7006f720 2024-04-20T11:37:15.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:15.693+0000 7fdd77c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdd700b4370 con 0x7fdd70162bc0 2024-04-20T11:37:15.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:15.693+0000 7fdd77c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fdd70162fa0 con 0x7fdd700b2880 2024-04-20T11:37:15.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:15.694+0000 7fdd75200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdd7006f720 0x7fdd701496c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:15.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:15.694+0000 7fdd75200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdd7006f720 0x7fdd701496c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:59350/0 (socket says 172.21.15.152:59350) 2024-04-20T11:37:15.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:15.694+0000 7fdd75200640 1 -- 172.21.15.152:0/519904943 learned_addr learned my addr 172.21.15.152:0/519904943 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:15.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:15.694+0000 7fdd75c00640 1 --2- 172.21.15.152:0/519904943 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd70162bc0 0x7fdd70155f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:15.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:15.694+0000 7fdd6fe00640 1 --2- 172.21.15.152:0/519904943 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd700b2880 0x7fdd70153390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:15.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:15.694+0000 7fdd75200640 1 -- 172.21.15.152:0/519904943 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd700b2880 msgr2=0x7fdd70153390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:15.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:15.694+0000 7fdd75200640 1 --2- 172.21.15.152:0/519904943 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd700b2880 0x7fdd70153390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:15.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:15.694+0000 7fdd75200640 1 -- 172.21.15.152:0/519904943 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd70162bc0 msgr2=0x7fdd70155f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:15.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:15.694+0000 7fdd75200640 1 --2- 172.21.15.152:0/519904943 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd70162bc0 0x7fdd70155f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:15.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:15.694+0000 7fdd75200640 1 -- 172.21.15.152:0/519904943 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd701538d0 con 0x7fdd7006f720 2024-04-20T11:37:15.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:15.694+0000 7fdd75c00640 1 --2- 172.21.15.152:0/519904943 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd70162bc0 0x7fdd70155f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:37:15.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:15.695+0000 7fdd75200640 1 --2- 172.21.15.152:0/519904943 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdd7006f720 0x7fdd701496c0 secure :-1 s=READY pgs=2189 cs=0 l=1 rev1=1 crypto rx=0x7fdd6400c7f0 tx=0x7fdd6400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:15.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:15.695+0000 7fdd6cc00640 1 -- 172.21.15.152:0/519904943 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd64015070 con 0x7fdd7006f720 2024-04-20T11:37:15.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:15.695+0000 7fdd6cc00640 1 -- 172.21.15.152:0/519904943 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd64004220 con 0x7fdd7006f720 2024-04-20T11:37:15.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:15.695+0000 7fdd6cc00640 1 -- 172.21.15.152:0/519904943 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd64007620 con 0x7fdd7006f720 2024-04-20T11:37:15.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:15.695+0000 7fdd77c00640 1 -- 172.21.15.152:0/519904943 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd70155480 con 0x7fdd7006f720 2024-04-20T11:37:15.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:15.695+0000 7fdd77c00640 1 -- 172.21.15.152:0/519904943 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd70154be0 con 0x7fdd7006f720 2024-04-20T11:37:15.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:15.696+0000 7fdd77c00640 1 -- 172.21.15.152:0/519904943 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd700689f0 con 0x7fdd7006f720 2024-04-20T11:37:15.697 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:15.697+0000 7fdd6cc00640 1 -- 172.21.15.152:0/519904943 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) ==== 91754+0+0 (secure 0 0 0) 0x7fdd640079d0 con 0x7fdd7006f720 2024-04-20T11:37:15.698 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:15.698+0000 7fdd6cc00640 1 --2- 172.21.15.152:0/519904943 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fdd5006d0d0 0x7fdd5006f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:15.698 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:15.698+0000 7fdd6cc00640 1 -- 172.21.15.152:0/519904943 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(154..154 src has 1..154) ==== 5709+0+0 (secure 0 0 0) 0x7fdd6408d280 con 0x7fdd7006f720 2024-04-20T11:37:15.698 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:15.699+0000 7fdd6fe00640 1 --2- 172.21.15.152:0/519904943 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fdd5006d0d0 0x7fdd5006f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:15.699 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:15.700+0000 7fdd6fe00640 1 --2- 172.21.15.152:0/519904943 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fdd5006d0d0 0x7fdd5006f590 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fdd701fc0d0 tx=0x7fdd5c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:15.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:15.702+0000 7fdd6cc00640 1 -- 172.21.15.152:0/519904943 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fdd6405a580 con 0x7fdd7006f720 2024-04-20T11:37:15.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:15.780+0000 7fdd6cc00640 1 -- 172.21.15.152:0/519904943 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mgrmap(e 25) ==== 91805+0+0 (secure 0 0 0) 0x7fdd640564f0 con 0x7fdd7006f720 2024-04-20T11:37:15.938 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:15.939+0000 7fdd77c00640 1 -- 172.21.15.152:0/519904943 --> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol90290", "target": ["mon-mgr", ""]}) -- 0x7fdd7006e7b0 con 0x7fdd5006d0d0 2024-04-20T11:37:15.975 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:15.975+0000 7fdd6cc00640 1 -- 172.21.15.152:0/519904943 <== mgr.4109 v2:172.21.15.175:6836/4021250441 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fdd7006e7b0 con 0x7fdd5006d0d0 2024-04-20T11:37:15.978 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:15.978+0000 7fdd77c00640 1 -- 172.21.15.152:0/519904943 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fdd5006d0d0 msgr2=0x7fdd5006f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:15.978 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:15.978+0000 7fdd77c00640 1 --2- 172.21.15.152:0/519904943 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fdd5006d0d0 0x7fdd5006f590 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fdd701fc0d0 tx=0x7fdd5c009040 comp rx=0 tx=0).stop 2024-04-20T11:37:15.978 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:15.978+0000 7fdd77c00640 1 -- 172.21.15.152:0/519904943 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdd7006f720 msgr2=0x7fdd701496c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:15.978 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:15.978+0000 7fdd77c00640 1 --2- 172.21.15.152:0/519904943 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdd7006f720 0x7fdd701496c0 secure :-1 s=READY pgs=2189 cs=0 l=1 rev1=1 crypto rx=0x7fdd6400c7f0 tx=0x7fdd6400ccb0 comp rx=0 tx=0).stop 2024-04-20T11:37:15.978 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:15.978+0000 7fdd77c00640 1 -- 172.21.15.152:0/519904943 shutdown_connections 2024-04-20T11:37:15.978 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:15.978+0000 7fdd77c00640 1 --2- 172.21.15.152:0/519904943 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fdd5006d0d0 0x7fdd5006f590 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:15.978 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:15.978+0000 7fdd77c00640 1 --2- 172.21.15.152:0/519904943 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd70162bc0 0x7fdd70155f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:15.978 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:15.978+0000 7fdd77c00640 1 --2- 172.21.15.152:0/519904943 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd700b2880 0x7fdd70153390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:15.979 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:15.978+0000 7fdd77c00640 1 --2- 172.21.15.152:0/519904943 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdd7006f720 0x7fdd701496c0 unknown :-1 s=CLOSED pgs=2189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:15.979 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:15.978+0000 7fdd77c00640 1 -- 172.21.15.152:0/519904943 >> 172.21.15.152:0/519904943 conn(0x7fdd7006b980 msgr2=0x7fdd700b1c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:15.979 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:15.978+0000 7fdd77c00640 1 -- 172.21.15.152:0/519904943 shutdown_connections 2024-04-20T11:37:15.979 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:15.978+0000 7fdd77c00640 1 -- 172.21.15.152:0/519904943 wait complete. 2024-04-20T11:37:15.991 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:37:15.991+0000 7f8a5fe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:37:15.991 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:37:15.991+0000 7f8a5fe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:37:15.991 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:37:15.991+0000 7f8a5fe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:37:15.991 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:37:15.991+0000 7f8a5fe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:37:15.991 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:37:15.991+0000 7f8a5fe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:37:15.998 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol90151 subvol_grp58840 2024-04-20T11:37:16.014 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:37:16.014+0000 7f8a60800640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:37:16.014 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:37:16.014+0000 7f8a60800640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:37:16.014 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:37:16.014+0000 7f8a60800640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:37:16.014 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:37:16.014+0000 7f8a60800640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:37:16.014 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:37:16.014+0000 7f8a60800640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:37:16.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:16.191+0000 7fce0ba00640 1 Processor -- start 2024-04-20T11:37:16.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:16.192+0000 7fce0ba00640 1 -- start start 2024-04-20T11:37:16.193 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:16.193+0000 7fce0ba00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fce040b2330 0x7fce040b48d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:16.193 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:16.193+0000 7fce0ba00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fce040b2710 0x7fce0415cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:16.193 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:16.193+0000 7fce0ba00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce0415c4b0 con 0x7fce0415b9a0 2024-04-20T11:37:16.193 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:16.193+0000 7fce0ba00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fce040d1400 con 0x7fce040b2710 2024-04-20T11:37:16.193 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:16.193+0000 7fce0ba00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fce0415c790 con 0x7fce040b2330 2024-04-20T11:37:16.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:16.194+0000 7fce09a00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fce0415b9a0 0x7fce040d1890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:34878/0 (socket says 172.21.15.152:34878) 2024-04-20T11:37:16.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:16.194+0000 7fce09a00640 1 -- 172.21.15.152:0/4288144167 learned_addr learned my addr 172.21.15.152:0/4288144167 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:37:16.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:16.194+0000 7fce09000640 1 --2- 172.21.15.152:0/4288144167 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fce040b2330 0x7fce040b48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:16.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:16.194+0000 7fce03e00640 1 --2- 172.21.15.152:0/4288144167 >> v2:172.21.15.175:3300/0 conn(0x7fce040b2710 0x7fce0415cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:16.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:16.194+0000 7fce03400640 1 -- 172.21.15.152:0/4288144167 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3604297101 0 0) 0x7fce0415c4b0 con 0x7fce0415b9a0 2024-04-20T11:37:16.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:16.194+0000 7fce03400640 1 -- 172.21.15.152:0/4288144167 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcde4005730 con 0x7fce0415b9a0 2024-04-20T11:37:16.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:16.195+0000 7fce03400640 1 -- 172.21.15.152:0/4288144167 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3690087130 0 0) 0x7fcde4005730 con 0x7fce0415b9a0 2024-04-20T11:37:16.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:16.195+0000 7fce03400640 1 -- 172.21.15.152:0/4288144167 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fce040b2330 msgr2=0x7fce040b48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:16.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:16.195+0000 7fce03400640 1 --2- 172.21.15.152:0/4288144167 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fce040b2330 0x7fce040b48d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:16.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:16.195+0000 7fce03400640 1 -- 172.21.15.152:0/4288144167 >> v2:172.21.15.175:3300/0 conn(0x7fce040b2710 msgr2=0x7fce0415cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:16.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:16.195+0000 7fce03400640 1 --2- 172.21.15.152:0/4288144167 >> v2:172.21.15.175:3300/0 conn(0x7fce040b2710 0x7fce0415cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:16.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:16.195+0000 7fce03400640 1 -- 172.21.15.152:0/4288144167 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce0415ca70 con 0x7fce0415b9a0 2024-04-20T11:37:16.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:16.195+0000 7fce03400640 1 -- 172.21.15.152:0/4288144167 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7fcdfc001670 con 0x7fce0415b9a0 2024-04-20T11:37:16.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:16.195+0000 7fce03400640 1 -- 172.21.15.152:0/4288144167 >> v1:172.21.15.152:6789/0 conn(0x7fce0415b9a0 legacy=0x7fce040d1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:16.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:16.195+0000 7fce03400640 1 --2- 172.21.15.152:0/4288144167 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcde4005c30 0x7fcde4006050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:16.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:16.195+0000 7fce03400640 1 --2- 172.21.15.152:0/4288144167 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fce040b2330 0x7fcde4006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:16.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:16.195+0000 7fce03400640 1 --2- 172.21.15.152:0/4288144167 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcde4006f50 0x7fcde40073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:16.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:16.195+0000 7fce03400640 1 -- 172.21.15.152:0/4288144167 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fce040d1400 con 0x7fcde4005c30 2024-04-20T11:37:16.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:16.195+0000 7fce03400640 1 -- 172.21.15.152:0/4288144167 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fce0415c790 con 0x7fcde4006f50 2024-04-20T11:37:16.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:16.195+0000 7fce03400640 1 -- 172.21.15.152:0/4288144167 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fcde400dbd0 con 0x7fce040b2330 2024-04-20T11:37:16.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:16.195+0000 7fce09000640 1 --2- 172.21.15.152:0/4288144167 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcde4005c30 0x7fcde4006050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:16.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:16.196+0000 7fce03e00640 1 --2- 172.21.15.152:0/4288144167 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fce040b2330 0x7fcde4006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:16.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:16.196+0000 7fce09a00640 1 --2- 172.21.15.152:0/4288144167 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcde4006f50 0x7fcde40073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:16.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:16.196+0000 7fce09000640 1 -- 172.21.15.152:0/4288144167 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fce040b2330 msgr2=0x7fcde4006a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:16.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:16.196+0000 7fce09000640 1 --2- 172.21.15.152:0/4288144167 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fce040b2330 0x7fcde4006a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:16.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:16.196+0000 7fce09000640 1 -- 172.21.15.152:0/4288144167 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcde4006f50 msgr2=0x7fcde40073d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:16.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:16.196+0000 7fce09000640 1 --2- 172.21.15.152:0/4288144167 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcde4006f50 0x7fcde40073d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:16.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:16.196+0000 7fce09000640 1 -- 172.21.15.152:0/4288144167 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcde4005730 con 0x7fcde4005c30 2024-04-20T11:37:16.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:16.196+0000 7fce09a00640 1 --2- 172.21.15.152:0/4288144167 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcde4006f50 0x7fcde40073d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:37:16.199 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:16.197+0000 7fce09000640 1 --2- 172.21.15.152:0/4288144167 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcde4005c30 0x7fcde4006050 secure :-1 s=READY pgs=2191 cs=0 l=1 rev1=1 crypto rx=0x7fcdf4009ac0 tx=0x7fcde4007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba5a41d38dc90dfa server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:16.199 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:16.197+0000 7fce03400640 1 -- 172.21.15.152:0/4288144167 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf4039070 con 0x7fcde4005c30 2024-04-20T11:37:16.199 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:16.197+0000 7fce03400640 1 -- 172.21.15.152:0/4288144167 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdf402c6b0 con 0x7fcde4005c30 2024-04-20T11:37:16.199 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:16.197+0000 7fce03400640 1 -- 172.21.15.152:0/4288144167 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf4009690 con 0x7fcde4005c30 2024-04-20T11:37:16.199 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:16.198+0000 7fce0ba00640 1 -- 172.21.15.152:0/4288144167 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcde4005c30 msgr2=0x7fcde4006050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:16.199 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:16.198+0000 7fce0ba00640 1 --2- 172.21.15.152:0/4288144167 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcde4005c30 0x7fcde4006050 secure :-1 s=READY pgs=2191 cs=0 l=1 rev1=1 crypto rx=0x7fcdf4009ac0 tx=0x7fcde4007a40 comp rx=0 tx=0).stop 2024-04-20T11:37:16.199 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:16.198+0000 7fce09a00640 1 -- 172.21.15.152:0/4288144167 reap_dead start 2024-04-20T11:37:16.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:16.198+0000 7fce0ba00640 1 -- 172.21.15.152:0/4288144167 shutdown_connections 2024-04-20T11:37:16.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:16.198+0000 7fce0ba00640 1 -- 172.21.15.152:0/4288144167 >> 172.21.15.152:0/4288144167 conn(0x7fce040b7130 msgr2=0x7fce0409aa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:16.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:16.198+0000 7fce0ba00640 1 -- 172.21.15.152:0/4288144167 shutdown_connections 2024-04-20T11:37:16.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:16.198+0000 7fce0ba00640 1 -- 172.21.15.152:0/4288144167 wait complete. 2024-04-20T11:37:16.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:16.200+0000 7fce0ba00640 1 Processor -- start 2024-04-20T11:37:16.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:16.200+0000 7fce0ba00640 1 -- start start 2024-04-20T11:37:16.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:16.201+0000 7fce0ba00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcde4005c30 0x7fce040b03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:16.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:16.201+0000 7fce0ba00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fce041e3920 0x7fce040cd000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:16.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:16.201+0000 7fce0ba00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce04142bc0 0x7fce04143070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:16.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:16.201+0000 7fce0ba00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fce0415be30 con 0x7fce041e3920 2024-04-20T11:37:16.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:16.201+0000 7fce0ba00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fce040b2b90 con 0x7fce04142bc0 2024-04-20T11:37:16.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:16.201+0000 7fce0ba00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fce0415bfb0 con 0x7fcde4005c30 2024-04-20T11:37:16.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:16.201+0000 7fce03e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fce041e3920 0x7fce040cd000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:16.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:16.201+0000 7fce03e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fce041e3920 0x7fce040cd000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:33990/0 (socket says 172.21.15.152:33990) 2024-04-20T11:37:16.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:16.201+0000 7fce03e00640 1 -- 172.21.15.152:0/1316684136 learned_addr learned my addr 172.21.15.152:0/1316684136 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:16.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:16.201+0000 7fce09a00640 1 --2- 172.21.15.152:0/1316684136 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce04142bc0 0x7fce04143070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:16.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:16.201+0000 7fce09000640 1 --2- 172.21.15.152:0/1316684136 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcde4005c30 0x7fce040b03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:16.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:16.201+0000 7fce03e00640 1 -- 172.21.15.152:0/1316684136 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcde4005c30 msgr2=0x7fce040b03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:16.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:16.201+0000 7fce03e00640 1 --2- 172.21.15.152:0/1316684136 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcde4005c30 0x7fce040b03d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:16.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:16.201+0000 7fce03e00640 1 -- 172.21.15.152:0/1316684136 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce04142bc0 msgr2=0x7fce04143070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:16.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:16.201+0000 7fce03e00640 1 --2- 172.21.15.152:0/1316684136 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce04142bc0 0x7fce04143070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:16.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:16.201+0000 7fce03e00640 1 -- 172.21.15.152:0/1316684136 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce04140cd0 con 0x7fce041e3920 2024-04-20T11:37:16.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:16.202+0000 7fce03e00640 1 --2- 172.21.15.152:0/1316684136 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fce041e3920 0x7fce040cd000 secure :-1 s=READY pgs=2192 cs=0 l=1 rev1=1 crypto rx=0x7fcdec00f3d0 tx=0x7fcdec00f890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:16.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:16.202+0000 7fce00c00640 1 -- 172.21.15.152:0/1316684136 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdec016620 con 0x7fce041e3920 2024-04-20T11:37:16.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:16.202+0000 7fce00c00640 1 -- 172.21.15.152:0/1316684136 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdec01b070 con 0x7fce041e3920 2024-04-20T11:37:16.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:16.202+0000 7fce00c00640 1 -- 172.21.15.152:0/1316684136 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdec016a20 con 0x7fce041e3920 2024-04-20T11:37:16.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:16.202+0000 7fce0ba00640 1 -- 172.21.15.152:0/1316684136 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce040b0b00 con 0x7fce041e3920 2024-04-20T11:37:16.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:16.202+0000 7fce0ba00640 1 -- 172.21.15.152:0/1316684136 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce040b10c0 con 0x7fce041e3920 2024-04-20T11:37:16.204 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:16.203+0000 7fce0ba00640 1 -- 172.21.15.152:0/1316684136 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce04068a60 con 0x7fce041e3920 2024-04-20T11:37:16.204 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:16.204+0000 7fce00c00640 1 -- 172.21.15.152:0/1316684136 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 91805+0+0 (secure 0 0 0) 0x7fcdec005d10 con 0x7fce041e3920 2024-04-20T11:37:16.205 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:16.206+0000 7fce00c00640 1 --2- 172.21.15.152:0/1316684136 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fcde006d160 0x7fcde006f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:16.205 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:16.206+0000 7fce00c00640 1 -- 172.21.15.152:0/1316684136 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(154..154 src has 1..154) ==== 5709+0+0 (secure 0 0 0) 0x7fcdec08dc20 con 0x7fce041e3920 2024-04-20T11:37:16.206 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:16.206+0000 7fce09000640 1 --2- 172.21.15.152:0/1316684136 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fcde006d160 0x7fcde006f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:16.207 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:16.208+0000 7fce09000640 1 --2- 172.21.15.152:0/1316684136 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fcde006d160 0x7fcde006f620 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fcdf4036000 tx=0x7fcdf4030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:16.210 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:16.211+0000 7fce00c00640 1 -- 172.21.15.152:0/1316684136 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fcdec025020 con 0x7fce041e3920 2024-04-20T11:37:16.446 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:16.447+0000 7fce0ba00640 1 -- 172.21.15.152:0/1316684136 --> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol90151", "group_name": "subvol_grp58840", "target": ["mon-mgr", ""]}) -- 0x7fce040b1300 con 0x7fcde006d160 2024-04-20T11:37:17.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:17.807+0000 7fce00c00640 1 -- 172.21.15.152:0/1316684136 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mgrmap(e 26) ==== 91907+0+0 (secure 0 0 0) 0x7fcdec056f90 con 0x7fce041e3920 2024-04-20T11:37:23.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:23.908+0000 7fce00c00640 1 -- 172.21.15.152:0/1316684136 <== mgr.4109 v2:172.21.15.175:6836/4021250441 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fce040b1300 con 0x7fcde006d160 2024-04-20T11:37:23.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:23.912+0000 7fce0ba00640 1 -- 172.21.15.152:0/1316684136 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fcde006d160 msgr2=0x7fcde006f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:23.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:23.912+0000 7fce0ba00640 1 --2- 172.21.15.152:0/1316684136 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fcde006d160 0x7fcde006f620 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fcdf4036000 tx=0x7fcdf4030000 comp rx=0 tx=0).stop 2024-04-20T11:37:23.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:23.912+0000 7fce0ba00640 1 -- 172.21.15.152:0/1316684136 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fce041e3920 msgr2=0x7fce040cd000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:23.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:23.912+0000 7fce0ba00640 1 --2- 172.21.15.152:0/1316684136 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fce041e3920 0x7fce040cd000 secure :-1 s=READY pgs=2192 cs=0 l=1 rev1=1 crypto rx=0x7fcdec00f3d0 tx=0x7fcdec00f890 comp rx=0 tx=0).stop 2024-04-20T11:37:23.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:23.912+0000 7fce0ba00640 1 -- 172.21.15.152:0/1316684136 shutdown_connections 2024-04-20T11:37:23.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:23.912+0000 7fce0ba00640 1 --2- 172.21.15.152:0/1316684136 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fcde006d160 0x7fcde006f620 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:23.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:23.912+0000 7fce0ba00640 1 --2- 172.21.15.152:0/1316684136 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce04142bc0 0x7fce04143070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:23.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:23.912+0000 7fce0ba00640 1 --2- 172.21.15.152:0/1316684136 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fce041e3920 0x7fce040cd000 unknown :-1 s=CLOSED pgs=2192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:23.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:23.912+0000 7fce0ba00640 1 --2- 172.21.15.152:0/1316684136 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcde4005c30 0x7fce040b03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:23.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:23.912+0000 7fce0ba00640 1 -- 172.21.15.152:0/1316684136 >> 172.21.15.152:0/1316684136 conn(0x7fce040b7130 msgr2=0x7fce041e2d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:23.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:23.912+0000 7fce0ba00640 1 -- 172.21.15.152:0/1316684136 shutdown_connections 2024-04-20T11:37:23.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:23.912+0000 7fce0ba00640 1 -- 172.21.15.152:0/1316684136 wait complete. 2024-04-20T11:37:23.937 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-04-20T11:37:24.013 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:37:24.018 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:37:24.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:24 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-20T11:37:24.050 INFO:teuthology.orchestra.run.smithi152.stdout:2 2024-04-20T11:37:24.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T11:37:24.060 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-20T11:37:24.060 DEBUG:teuthology.orchestra.run.smithi152:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-20T11:37:24.202 DEBUG:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-20T11:37:24.328 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-20T11:37:24.328 INFO:tasks.cephfs.mount:Terminating background process 2024-04-20T11:37:24.328 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-20T11:37:24.328 DEBUG:teuthology.orchestra.run.smithi152:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:37:24.386 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:37:24.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:24.588+0000 7f1d4a800640 1 Processor -- start 2024-04-20T11:37:24.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:24.588+0000 7f1d4a800640 1 -- start start 2024-04-20T11:37:24.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:24.589+0000 7f1d4a800640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f1d440ba520 0x7f1d440d28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:24.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:24.589+0000 7f1d4a800640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d440b80b0 con 0x7f1d440d2e30 2024-04-20T11:37:24.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:24.589+0000 7f1d4a800640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d440b8330 con 0x7f1d4415eb80 2024-04-20T11:37:24.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:24.589+0000 7f1d4a800640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f1d440b8610 con 0x7f1d440ba520 2024-04-20T11:37:24.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:24.590+0000 7f1d43e00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f1d440d2e30 0x7f1d440d1810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:34894/0 (socket says 172.21.15.152:34894) 2024-04-20T11:37:24.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:24.590+0000 7f1d43e00640 1 -- 172.21.15.152:0/1931519759 learned_addr learned my addr 172.21.15.152:0/1931519759 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:37:24.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:24.590+0000 7f1d42a00640 1 --2- 172.21.15.152:0/1931519759 >> v2:172.21.15.175:3300/0 conn(0x7f1d440ba520 0x7f1d440d28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:24.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:24.590+0000 7f1d42000640 1 -- 172.21.15.152:0/1931519759 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 36422142 0 0) 0x7f1d3c001490 con 0x7f1d440d2e30 2024-04-20T11:37:24.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:24.590+0000 7f1d42000640 1 -- 172.21.15.152:0/1931519759 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1d28003520 con 0x7f1d440d2e30 2024-04-20T11:37:24.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:24.591+0000 7f1d42000640 1 -- 172.21.15.152:0/1931519759 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1286080470 0 0) 0x7f1d28003520 con 0x7f1d440d2e30 2024-04-20T11:37:24.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:24.591+0000 7f1d42000640 1 -- 172.21.15.152:0/1931519759 >> v2:172.21.15.175:3300/0 conn(0x7f1d440ba520 msgr2=0x7f1d440d28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:24.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:24.591+0000 7f1d42000640 1 --2- 172.21.15.152:0/1931519759 >> v2:172.21.15.175:3300/0 conn(0x7f1d440ba520 0x7f1d440d28f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:24.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:24.591+0000 7f1d42000640 1 -- 172.21.15.152:0/1931519759 >> v1:172.21.15.175:6789/0 conn(0x7f1d4415eb80 legacy=0x7f1d440b79a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:24.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:24.591+0000 7f1d42000640 1 -- 172.21.15.152:0/1931519759 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d440b88f0 con 0x7f1d440d2e30 2024-04-20T11:37:24.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:24.591+0000 7f1d42000640 1 -- 172.21.15.152:0/1931519759 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f1d3c001bc0 con 0x7f1d440d2e30 2024-04-20T11:37:24.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:24.591+0000 7f1d42000640 1 -- 172.21.15.152:0/1931519759 >> v1:172.21.15.152:6789/0 conn(0x7f1d440d2e30 legacy=0x7f1d440d1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:24.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:24.591+0000 7f1d42000640 1 --2- 172.21.15.152:0/1931519759 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d28003a70 0x7f1d28003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:24.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:24.591+0000 7f1d42000640 1 --2- 172.21.15.152:0/1931519759 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d28004800 0x7f1d28004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:24.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:24.591+0000 7f1d42000640 1 --2- 172.21.15.152:0/1931519759 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1d280053c0 0x7f1d2800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:24.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:24.591+0000 7f1d42000640 1 -- 172.21.15.152:0/1931519759 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f1d440b8610 con 0x7f1d28003a70 2024-04-20T11:37:24.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:24.591+0000 7f1d42000640 1 -- 172.21.15.152:0/1931519759 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1d2800c250 con 0x7f1d28004800 2024-04-20T11:37:24.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:24.591+0000 7f1d42000640 1 -- 172.21.15.152:0/1931519759 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1d2800c450 con 0x7f1d280053c0 2024-04-20T11:37:24.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:24.591+0000 7f1d42a00640 1 --2- 172.21.15.152:0/1931519759 >> v2:172.21.15.175:3300/0 conn(0x7f1d440ba520 0x7f1d440d28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:37:24.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:24.591+0000 7f1d43400640 1 --2- 172.21.15.152:0/1931519759 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d28003a70 0x7f1d28003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:24.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:24.591+0000 7f1d43400640 1 -- 172.21.15.152:0/1931519759 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1d280053c0 msgr2=0x7f1d2800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:24.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:24.591+0000 7f1d43400640 1 --2- 172.21.15.152:0/1931519759 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1d280053c0 0x7f1d2800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:24.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:24.591+0000 7f1d43400640 1 -- 172.21.15.152:0/1931519759 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d28004800 msgr2=0x7f1d28004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:24.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:24.591+0000 7f1d43400640 1 --2- 172.21.15.152:0/1931519759 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d28004800 0x7f1d28004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:24.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:24.591+0000 7f1d43400640 1 -- 172.21.15.152:0/1931519759 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d28003520 con 0x7f1d28003a70 2024-04-20T11:37:24.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:24.592+0000 7f1d43e00640 1 -- 172.21.15.152:0/1931519759 reap_dead start 2024-04-20T11:37:24.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:24.593+0000 7f1d43400640 1 --2- 172.21.15.152:0/1931519759 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d28003a70 0x7f1d28003e70 secure :-1 s=READY pgs=2194 cs=0 l=1 rev1=1 crypto rx=0x7f1d34003db0 tx=0x7f1d28005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9582c5bca2ff02e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:24.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:24.593+0000 7f1d42000640 1 -- 172.21.15.152:0/1931519759 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d34037070 con 0x7f1d28003a70 2024-04-20T11:37:24.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:24.593+0000 7f1d42000640 1 -- 172.21.15.152:0/1931519759 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d34032600 con 0x7f1d28003a70 2024-04-20T11:37:24.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:24.593+0000 7f1d42000640 1 -- 172.21.15.152:0/1931519759 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d34003980 con 0x7f1d28003a70 2024-04-20T11:37:24.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:24.593+0000 7f1d4a800640 1 -- 172.21.15.152:0/1931519759 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d28003a70 msgr2=0x7f1d28003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:24.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:24.593+0000 7f1d4a800640 1 --2- 172.21.15.152:0/1931519759 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d28003a70 0x7f1d28003e70 secure :-1 s=READY pgs=2194 cs=0 l=1 rev1=1 crypto rx=0x7f1d34003db0 tx=0x7f1d28005830 comp rx=0 tx=0).stop 2024-04-20T11:37:24.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:24.594+0000 7f1d4a800640 1 -- 172.21.15.152:0/1931519759 shutdown_connections 2024-04-20T11:37:24.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:24.594+0000 7f1d4a800640 1 --2- 172.21.15.152:0/1931519759 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d28003a70 0x7f1d28003e70 unknown :-1 s=CLOSED pgs=2194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:24.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:24.594+0000 7f1d4a800640 1 -- 172.21.15.152:0/1931519759 >> 172.21.15.152:0/1931519759 conn(0x7f1d44067ae0 msgr2=0x7f1d44067ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:24.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:24.594+0000 7f1d4a800640 1 -- 172.21.15.152:0/1931519759 shutdown_connections 2024-04-20T11:37:24.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:24.594+0000 7f1d4a800640 1 -- 172.21.15.152:0/1931519759 wait complete. 2024-04-20T11:37:24.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:24.595+0000 7f1d4a800640 1 Processor -- start 2024-04-20T11:37:24.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:24.596+0000 7f1d4a800640 1 -- start start 2024-04-20T11:37:24.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:24.596+0000 7f1d4a800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d28003a70 0x7f1d441f4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:24.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:24.596+0000 7f1d4a800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1d44154c00 0x7f1d44155080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:24.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:24.597+0000 7f1d4a800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d44160180 0x7f1d44160600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:24.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:24.597+0000 7f1d4a800640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f1d440d2a70 con 0x7f1d44160180 2024-04-20T11:37:24.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:24.597+0000 7f1d4a800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1d440d28f0 con 0x7f1d28003a70 2024-04-20T11:37:24.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:24.597+0000 7f1d4a800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1d440d2bf0 con 0x7f1d44154c00 2024-04-20T11:37:24.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:24.597+0000 7f1d43e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d44160180 0x7f1d44160600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:24.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:24.597+0000 7f1d43e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d44160180 0x7f1d44160600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:34022/0 (socket says 172.21.15.152:34022) 2024-04-20T11:37:24.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:24.597+0000 7f1d43e00640 1 -- 172.21.15.152:0/3250451407 learned_addr learned my addr 172.21.15.152:0/3250451407 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:24.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:24.597+0000 7f1d43e00640 1 -- 172.21.15.152:0/3250451407 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1d44154c00 msgr2=0x7f1d44155080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:24.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:24.597+0000 7f1d43e00640 1 --2- 172.21.15.152:0/3250451407 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1d44154c00 0x7f1d44155080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:24.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:24.597+0000 7f1d43400640 1 --2- 172.21.15.152:0/3250451407 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d28003a70 0x7f1d441f4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:24.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:24.597+0000 7f1d43e00640 1 -- 172.21.15.152:0/3250451407 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d28003a70 msgr2=0x7f1d441f4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:24.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:24.597+0000 7f1d43e00640 1 --2- 172.21.15.152:0/3250451407 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d28003a70 0x7f1d441f4290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:24.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:24.597+0000 7f1d43e00640 1 -- 172.21.15.152:0/3250451407 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d44160f50 con 0x7f1d44160180 2024-04-20T11:37:24.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:24.598+0000 7f1d43e00640 1 --2- 172.21.15.152:0/3250451407 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d44160180 0x7f1d44160600 secure :-1 s=READY pgs=2195 cs=0 l=1 rev1=1 crypto rx=0x7f1d440d1230 tx=0x7f1d3c00cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:24.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:24.598+0000 7f1d3b400640 1 -- 172.21.15.152:0/3250451407 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d3c004d60 con 0x7f1d44160180 2024-04-20T11:37:24.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:24.598+0000 7f1d3b400640 1 -- 172.21.15.152:0/3250451407 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d3c00a5b0 con 0x7f1d44160180 2024-04-20T11:37:24.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:24.598+0000 7f1d3b400640 1 -- 172.21.15.152:0/3250451407 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d3c00a980 con 0x7f1d44160180 2024-04-20T11:37:24.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:24.598+0000 7f1d4a800640 1 -- 172.21.15.152:0/3250451407 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d44161230 con 0x7f1d44160180 2024-04-20T11:37:24.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:24.598+0000 7f1d4a800640 1 -- 172.21.15.152:0/3250451407 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d441617f0 con 0x7f1d44160180 2024-04-20T11:37:24.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:24.599+0000 7f1d4a800640 1 -- 172.21.15.152:0/3250451407 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d440689f0 con 0x7f1d44160180 2024-04-20T11:37:24.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:24.600+0000 7f1d3b400640 1 -- 172.21.15.152:0/3250451407 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 26) ==== 91907+0+0 (secure 0 0 0) 0x7f1d3c00ab20 con 0x7f1d44160180 2024-04-20T11:37:24.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:24.601+0000 7f1d3b400640 1 --2- 172.21.15.152:0/3250451407 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f1d2406d280 0x7f1d2406f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:24.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:24.601+0000 7f1d3b400640 1 -- 172.21.15.152:0/3250451407 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(155..155 src has 1..155) ==== 5767+0+0 (secure 0 0 0) 0x7f1d3c0896e0 con 0x7f1d44160180 2024-04-20T11:37:24.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:24.602+0000 7f1d43400640 1 --2- 172.21.15.152:0/3250451407 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f1d2406d280 0x7f1d2406f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:24.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:24.603+0000 7f1d43400640 1 --2- 172.21.15.152:0/3250451407 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f1d2406d280 0x7f1d2406f740 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f1d34034000 tx=0x7f1d34009030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:24.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:24.606+0000 7f1d3b400640 1 -- 172.21.15.152:0/3250451407 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f1d3c01a030 con 0x7f1d44160180 2024-04-20T11:37:24.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:24.807+0000 7f1d4a800640 1 -- 172.21.15.152:0/3250451407 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1d440d20f0 con 0x7f1d44160180 2024-04-20T11:37:24.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:24.810+0000 7f1d3b400640 1 -- 172.21.15.152:0/3250451407 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v155) ==== 74+0+16781 (secure 0 0 0) 0x7f1d440b80b0 con 0x7f1d44160180 2024-04-20T11:37:24.810 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:37:24.811 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":155,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:37:23.824055+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:36:57.520028+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"153","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2024-04-20T11:36:58.760637+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"154","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"25.13","mappings":[{"from":0,"to":1}]},{"pgid":"25.15","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:37:24.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:24.813+0000 7f1d4a800640 1 -- 172.21.15.152:0/3250451407 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f1d2406d280 msgr2=0x7f1d2406f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:24.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:24.813+0000 7f1d4a800640 1 --2- 172.21.15.152:0/3250451407 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f1d2406d280 0x7f1d2406f740 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f1d34034000 tx=0x7f1d34009030 comp rx=0 tx=0).stop 2024-04-20T11:37:24.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:24.813+0000 7f1d4a800640 1 -- 172.21.15.152:0/3250451407 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d44160180 msgr2=0x7f1d44160600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:24.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:24.813+0000 7f1d4a800640 1 --2- 172.21.15.152:0/3250451407 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d44160180 0x7f1d44160600 secure :-1 s=READY pgs=2195 cs=0 l=1 rev1=1 crypto rx=0x7f1d440d1230 tx=0x7f1d3c00cc00 comp rx=0 tx=0).stop 2024-04-20T11:37:24.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:24.813+0000 7f1d4a800640 1 -- 172.21.15.152:0/3250451407 shutdown_connections 2024-04-20T11:37:24.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:24.813+0000 7f1d4a800640 1 --2- 172.21.15.152:0/3250451407 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f1d2406d280 0x7f1d2406f740 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:24.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:24.813+0000 7f1d4a800640 1 --2- 172.21.15.152:0/3250451407 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d44160180 0x7f1d44160600 unknown :-1 s=CLOSED pgs=2195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:24.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:24.813+0000 7f1d4a800640 1 --2- 172.21.15.152:0/3250451407 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1d44154c00 0x7f1d44155080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:24.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:24.813+0000 7f1d4a800640 1 --2- 172.21.15.152:0/3250451407 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d28003a70 0x7f1d441f4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:24.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:24.813+0000 7f1d4a800640 1 -- 172.21.15.152:0/3250451407 >> 172.21.15.152:0/3250451407 conn(0x7f1d44067ae0 msgr2=0x7f1d440d4aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:24.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:24.813+0000 7f1d4a800640 1 -- 172.21.15.152:0/3250451407 shutdown_connections 2024-04-20T11:37:24.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:24.813+0000 7f1d4a800640 1 -- 172.21.15.152:0/3250451407 wait complete. 2024-04-20T11:37:24.834 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T11:26:47.246571+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 24, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T11:36:57.520028+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '153', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 25, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T11:36:58.760637+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '154', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T11:37:24.834 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T11:37:25.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.028+0000 7fc527e00640 1 Processor -- start 2024-04-20T11:37:25.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.028+0000 7fc527e00640 1 -- start start 2024-04-20T11:37:25.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.030+0000 7fc527e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc528156c00 0x7fc528160a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:25.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.030+0000 7fc527e00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc528160fd0 con 0x7fc52806c4b0 2024-04-20T11:37:25.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.030+0000 7fc527e00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc52815d5c0 con 0x7fc52815ca40 2024-04-20T11:37:25.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.030+0000 7fc527e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc52815d8a0 con 0x7fc528156c00 2024-04-20T11:37:25.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.031+0000 7fc527400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fc52806c4b0 0x7fc5280b30c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:34898/0 (socket says 172.21.15.152:34898) 2024-04-20T11:37:25.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.031+0000 7fc527400640 1 -- 172.21.15.152:0/2900903736 learned_addr learned my addr 172.21.15.152:0/2900903736 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:37:25.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.031+0000 7fc526a00640 1 --2- 172.21.15.152:0/2900903736 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc528156c00 0x7fc528160a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:25.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.031+0000 7fc525600640 1 -- 172.21.15.152:0/2900903736 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2090836108 0 0) 0x7fc528160fd0 con 0x7fc52806c4b0 2024-04-20T11:37:25.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.031+0000 7fc525600640 1 -- 172.21.15.152:0/2900903736 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc50c003520 con 0x7fc52806c4b0 2024-04-20T11:37:25.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.032+0000 7fc525600640 1 -- 172.21.15.152:0/2900903736 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2101622298 0 0) 0x7fc50c003520 con 0x7fc52806c4b0 2024-04-20T11:37:25.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.032+0000 7fc525600640 1 -- 172.21.15.152:0/2900903736 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc528156c00 msgr2=0x7fc528160a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:25.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.032+0000 7fc525600640 1 --2- 172.21.15.152:0/2900903736 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc528156c00 0x7fc528160a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:25.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.032+0000 7fc525600640 1 -- 172.21.15.152:0/2900903736 >> v1:172.21.15.175:6789/0 conn(0x7fc52815ca40 legacy=0x7fc52806bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:25.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.032+0000 7fc525600640 1 -- 172.21.15.152:0/2900903736 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc528160290 con 0x7fc52806c4b0 2024-04-20T11:37:25.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.032+0000 7fc525600640 1 -- 172.21.15.152:0/2900903736 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7fc520001a10 con 0x7fc52806c4b0 2024-04-20T11:37:25.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.032+0000 7fc525600640 1 -- 172.21.15.152:0/2900903736 >> v1:172.21.15.152:6789/0 conn(0x7fc52806c4b0 legacy=0x7fc5280b30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:25.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.032+0000 7fc525600640 1 --2- 172.21.15.152:0/2900903736 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc528156c00 0x7fc50c0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:25.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.032+0000 7fc525600640 1 --2- 172.21.15.152:0/2900903736 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc50c003fb0 0x7fc50c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:25.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.032+0000 7fc525600640 1 --2- 172.21.15.152:0/2900903736 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc50c004d40 0x7fc50c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:25.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.032+0000 7fc525600640 1 -- 172.21.15.152:0/2900903736 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fc52815d8a0 con 0x7fc50c004d40 2024-04-20T11:37:25.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.032+0000 7fc525600640 1 -- 172.21.15.152:0/2900903736 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc50c00bb00 con 0x7fc50c003fb0 2024-04-20T11:37:25.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.032+0000 7fc525600640 1 -- 172.21.15.152:0/2900903736 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc50c00bd00 con 0x7fc528156c00 2024-04-20T11:37:25.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.032+0000 7fc527400640 1 --2- 172.21.15.152:0/2900903736 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc50c004d40 0x7fc50c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:25.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.033+0000 7fc527400640 1 -- 172.21.15.152:0/2900903736 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc528156c00 msgr2=0x7fc50c0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:25.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.033+0000 7fc527400640 1 --2- 172.21.15.152:0/2900903736 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc528156c00 0x7fc50c0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:25.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.033+0000 7fc527400640 1 -- 172.21.15.152:0/2900903736 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc50c003fb0 msgr2=0x7fc50c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:25.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.033+0000 7fc526000640 1 --2- 172.21.15.152:0/2900903736 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc50c003fb0 0x7fc50c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:25.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.033+0000 7fc527400640 1 --2- 172.21.15.152:0/2900903736 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc50c003fb0 0x7fc50c004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:25.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.033+0000 7fc527400640 1 -- 172.21.15.152:0/2900903736 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc50c003520 con 0x7fc50c004d40 2024-04-20T11:37:25.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.034+0000 7fc527400640 1 --2- 172.21.15.152:0/2900903736 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc50c004d40 0x7fc50c0051c0 secure :-1 s=READY pgs=2197 cs=0 l=1 rev1=1 crypto rx=0x7fc5281611a0 tx=0x7fc50c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a0d2de4c64ddcc2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:25.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.034+0000 7fc525600640 1 -- 172.21.15.152:0/2900903736 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc52000a3b0 con 0x7fc50c004d40 2024-04-20T11:37:25.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.034+0000 7fc525600640 1 -- 172.21.15.152:0/2900903736 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc520034630 con 0x7fc50c004d40 2024-04-20T11:37:25.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.034+0000 7fc525600640 1 -- 172.21.15.152:0/2900903736 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc520034a60 con 0x7fc50c004d40 2024-04-20T11:37:25.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.035+0000 7fc527e00640 1 -- 172.21.15.152:0/2900903736 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc50c004d40 msgr2=0x7fc50c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:25.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.035+0000 7fc527e00640 1 --2- 172.21.15.152:0/2900903736 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc50c004d40 0x7fc50c0051c0 secure :-1 s=READY pgs=2197 cs=0 l=1 rev1=1 crypto rx=0x7fc5281611a0 tx=0x7fc50c005830 comp rx=0 tx=0).stop 2024-04-20T11:37:25.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.035+0000 7fc527400640 1 -- 172.21.15.152:0/2900903736 reap_dead start 2024-04-20T11:37:25.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.035+0000 7fc527e00640 1 -- 172.21.15.152:0/2900903736 shutdown_connections 2024-04-20T11:37:25.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.035+0000 7fc527e00640 1 -- 172.21.15.152:0/2900903736 >> 172.21.15.152:0/2900903736 conn(0x7fc5280b21e0 msgr2=0x7fc5280b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:25.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.035+0000 7fc527e00640 1 -- 172.21.15.152:0/2900903736 shutdown_connections 2024-04-20T11:37:25.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.035+0000 7fc527e00640 1 -- 172.21.15.152:0/2900903736 wait complete. 2024-04-20T11:37:25.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.036+0000 7fc527e00640 1 Processor -- start 2024-04-20T11:37:25.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.037+0000 7fc527e00640 1 -- start start 2024-04-20T11:37:25.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.037+0000 7fc527e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc50c004d40 0x7fc52809a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:25.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.038+0000 7fc527e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc5281f4e20 0x7fc528154800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:25.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.038+0000 7fc527e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc528154d40 0x7fc528141060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:25.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.038+0000 7fc527e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fc528160c10 con 0x7fc5281f4e20 2024-04-20T11:37:25.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.038+0000 7fc527e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc528160a90 con 0x7fc528154d40 2024-04-20T11:37:25.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.038+0000 7fc527e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc528160d90 con 0x7fc50c004d40 2024-04-20T11:37:25.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.038+0000 7fc526000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc5281f4e20 0x7fc528154800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:25.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.038+0000 7fc526000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc5281f4e20 0x7fc528154800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:34042/0 (socket says 172.21.15.152:34042) 2024-04-20T11:37:25.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.038+0000 7fc526000640 1 -- 172.21.15.152:0/3595012121 learned_addr learned my addr 172.21.15.152:0/3595012121 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:25.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.038+0000 7fc527400640 1 --2- 172.21.15.152:0/3595012121 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc528154d40 0x7fc528141060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:25.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.038+0000 7fc526000640 1 -- 172.21.15.152:0/3595012121 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc50c004d40 msgr2=0x7fc52809a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:25.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.038+0000 7fc526000640 1 --2- 172.21.15.152:0/3595012121 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc50c004d40 0x7fc52809a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:25.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.038+0000 7fc526000640 1 -- 172.21.15.152:0/3595012121 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc528154d40 msgr2=0x7fc528141060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:25.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.038+0000 7fc526000640 1 --2- 172.21.15.152:0/3595012121 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc528154d40 0x7fc528141060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:25.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.038+0000 7fc526000640 1 -- 172.21.15.152:0/3595012121 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc528153760 con 0x7fc5281f4e20 2024-04-20T11:37:25.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.038+0000 7fc527400640 1 --2- 172.21.15.152:0/3595012121 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc528154d40 0x7fc528141060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:37:25.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.039+0000 7fc526000640 1 --2- 172.21.15.152:0/3595012121 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc5281f4e20 0x7fc528154800 secure :-1 s=READY pgs=2198 cs=0 l=1 rev1=1 crypto rx=0x7fc510001eb0 tx=0x7fc51000d9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:25.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.039+0000 7fc51ea00640 1 -- 172.21.15.152:0/3595012121 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc510005480 con 0x7fc5281f4e20 2024-04-20T11:37:25.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.039+0000 7fc51ea00640 1 -- 172.21.15.152:0/3595012121 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc510017070 con 0x7fc5281f4e20 2024-04-20T11:37:25.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.039+0000 7fc51ea00640 1 -- 172.21.15.152:0/3595012121 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc510003810 con 0x7fc5281f4e20 2024-04-20T11:37:25.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.039+0000 7fc527e00640 1 -- 172.21.15.152:0/3595012121 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc528153a40 con 0x7fc5281f4e20 2024-04-20T11:37:25.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.039+0000 7fc527e00640 1 -- 172.21.15.152:0/3595012121 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc528154000 con 0x7fc5281f4e20 2024-04-20T11:37:25.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.040+0000 7fc527e00640 1 -- 172.21.15.152:0/3595012121 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc528075e60 con 0x7fc5281f4e20 2024-04-20T11:37:25.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.041+0000 7fc51ea00640 1 -- 172.21.15.152:0/3595012121 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 26) ==== 91907+0+0 (secure 0 0 0) 0x7fc5100039b0 con 0x7fc5281f4e20 2024-04-20T11:37:25.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.043+0000 7fc51ea00640 1 --2- 172.21.15.152:0/3595012121 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fc50406d280 0x7fc50406f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:25.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.043+0000 7fc51ea00640 1 -- 172.21.15.152:0/3595012121 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5767+0+0 (secure 0 0 0) 0x7fc51008a110 con 0x7fc5281f4e20 2024-04-20T11:37:25.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.043+0000 7fc526a00640 1 --2- 172.21.15.152:0/3595012121 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fc50406d280 0x7fc50406f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:25.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.045+0000 7fc526a00640 1 --2- 172.21.15.152:0/3595012121 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fc50406d280 0x7fc50406f740 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fc518009a40 tx=0x7fc518009500 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:25.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.048+0000 7fc51ea00640 1 -- 172.21.15.152:0/3595012121 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fc51005a140 con 0x7fc5281f4e20 2024-04-20T11:37:25.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.248+0000 7fc527e00640 1 -- 172.21.15.152:0/3595012121 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc52806f390 con 0x7fc5281f4e20 2024-04-20T11:37:25.249 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.249+0000 7fc51ea00640 1 -- 172.21.15.152:0/3595012121 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v156) ==== 93+0+10 (secure 0 0 0) 0x7fc510059a40 con 0x7fc5281f4e20 2024-04-20T11:37:25.249 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 1 2024-04-20T11:37:25.252 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.252+0000 7fc527e00640 1 -- 172.21.15.152:0/3595012121 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fc50406d280 msgr2=0x7fc50406f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:25.252 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.252+0000 7fc527e00640 1 --2- 172.21.15.152:0/3595012121 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fc50406d280 0x7fc50406f740 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fc518009a40 tx=0x7fc518009500 comp rx=0 tx=0).stop 2024-04-20T11:37:25.252 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.252+0000 7fc527e00640 1 -- 172.21.15.152:0/3595012121 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc5281f4e20 msgr2=0x7fc528154800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:25.252 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.252+0000 7fc527e00640 1 --2- 172.21.15.152:0/3595012121 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc5281f4e20 0x7fc528154800 secure :-1 s=READY pgs=2198 cs=0 l=1 rev1=1 crypto rx=0x7fc510001eb0 tx=0x7fc51000d9d0 comp rx=0 tx=0).stop 2024-04-20T11:37:25.252 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.252+0000 7fc527e00640 1 -- 172.21.15.152:0/3595012121 shutdown_connections 2024-04-20T11:37:25.252 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.252+0000 7fc527e00640 1 --2- 172.21.15.152:0/3595012121 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fc50406d280 0x7fc50406f740 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:25.252 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.252+0000 7fc527e00640 1 --2- 172.21.15.152:0/3595012121 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc528154d40 0x7fc528141060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:25.252 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.252+0000 7fc527e00640 1 --2- 172.21.15.152:0/3595012121 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc5281f4e20 0x7fc528154800 unknown :-1 s=CLOSED pgs=2198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:25.253 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.252+0000 7fc527e00640 1 --2- 172.21.15.152:0/3595012121 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc50c004d40 0x7fc52809a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:25.253 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.252+0000 7fc527e00640 1 -- 172.21.15.152:0/3595012121 >> 172.21.15.152:0/3595012121 conn(0x7fc5280b21e0 msgr2=0x7fc52815c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:25.253 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.252+0000 7fc527e00640 1 -- 172.21.15.152:0/3595012121 shutdown_connections 2024-04-20T11:37:25.253 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.252+0000 7fc527e00640 1 -- 172.21.15.152:0/3595012121 wait complete. 2024-04-20T11:37:25.270 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T11:37:25.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.475+0000 7f7d20600640 1 Processor -- start 2024-04-20T11:37:25.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.475+0000 7f7d20600640 1 -- start start 2024-04-20T11:37:25.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.477+0000 7f7d20600640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f7d180cd8c0 0x7f7d180b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:25.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.477+0000 7f7d20600640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d181463e0 con 0x7f7d18146000 2024-04-20T11:37:25.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.477+0000 7f7d20600640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d180b3e20 con 0x7f7d180b2880 2024-04-20T11:37:25.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.477+0000 7f7d20600640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f7d180b4750 con 0x7f7d180cd8c0 2024-04-20T11:37:25.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.477+0000 7f7d1e600640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f7d180cd8c0 0x7f7d180b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:25.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.477+0000 7f7d1dc00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f7d18146000 0x7f7d180b2170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:34902/0 (socket says 172.21.15.152:34902) 2024-04-20T11:37:25.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.477+0000 7f7d1dc00640 1 -- 172.21.15.152:0/2050640206 learned_addr learned my addr 172.21.15.152:0/2050640206 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:37:25.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.478+0000 7f7d1e600640 1 -- 172.21.15.152:0/2050640206 >> v1:172.21.15.175:6789/0 conn(0x7f7d180b2880 legacy=0x7f7d180cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:25.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.478+0000 7f7d1e600640 1 -- 172.21.15.152:0/2050640206 >> v1:172.21.15.152:6789/0 conn(0x7f7d18146000 legacy=0x7f7d180b2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:25.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.478+0000 7f7d1e600640 1 -- 172.21.15.152:0/2050640206 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d180b4100 con 0x7f7d180cd8c0 2024-04-20T11:37:25.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.479+0000 7f7d1e600640 1 --2- 172.21.15.152:0/2050640206 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7d180cd8c0 0x7f7d180b4be0 secure :-1 s=READY pgs=2200 cs=0 l=1 rev1=1 crypto rx=0x7f7d10009d90 tx=0x7f7d1002bc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41f380ce2b4defcb server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:25.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.479+0000 7f7d17e00640 1 -- 172.21.15.152:0/2050640206 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d10039070 con 0x7f7d180cd8c0 2024-04-20T11:37:25.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.479+0000 7f7d17e00640 1 -- 172.21.15.152:0/2050640206 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d1002c990 con 0x7f7d180cd8c0 2024-04-20T11:37:25.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.479+0000 7f7d17e00640 1 -- 172.21.15.152:0/2050640206 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d1002cd60 con 0x7f7d180cd8c0 2024-04-20T11:37:25.480 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.480+0000 7f7d20600640 1 -- 172.21.15.152:0/2050640206 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7d180cd8c0 msgr2=0x7f7d180b4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:25.480 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.480+0000 7f7d20600640 1 --2- 172.21.15.152:0/2050640206 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7d180cd8c0 0x7f7d180b4be0 secure :-1 s=READY pgs=2200 cs=0 l=1 rev1=1 crypto rx=0x7f7d10009d90 tx=0x7f7d1002bc60 comp rx=0 tx=0).stop 2024-04-20T11:37:25.480 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.480+0000 7f7d20600640 1 -- 172.21.15.152:0/2050640206 shutdown_connections 2024-04-20T11:37:25.480 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.480+0000 7f7d20600640 1 --2- 172.21.15.152:0/2050640206 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7d180cd8c0 0x7f7d180b4be0 unknown :-1 s=CLOSED pgs=2200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:25.480 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.480+0000 7f7d20600640 1 -- 172.21.15.152:0/2050640206 >> 172.21.15.152:0/2050640206 conn(0x7f7d1806b980 msgr2=0x7f7d1806bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:25.480 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.480+0000 7f7d20600640 1 -- 172.21.15.152:0/2050640206 shutdown_connections 2024-04-20T11:37:25.480 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.480+0000 7f7d20600640 1 -- 172.21.15.152:0/2050640206 wait complete. 2024-04-20T11:37:25.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.482+0000 7f7d20600640 1 Processor -- start 2024-04-20T11:37:25.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.482+0000 7f7d20600640 1 -- start start 2024-04-20T11:37:25.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.483+0000 7f7d20600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7d180b2880 0x7f7d181f28f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:25.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.483+0000 7f7d20600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d180cd8c0 0x7f7d1815b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:25.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.483+0000 7f7d20600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7d18146000 0x7f7d1813c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:25.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.483+0000 7f7d20600640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f7d1815ce70 con 0x7f7d180b2880 2024-04-20T11:37:25.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.483+0000 7f7d20600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7d1815ccf0 con 0x7f7d180cd8c0 2024-04-20T11:37:25.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.483+0000 7f7d20600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7d1815cff0 con 0x7f7d18146000 2024-04-20T11:37:25.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.483+0000 7f7d1dc00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7d180b2880 0x7f7d181f28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:25.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.484+0000 7f7d1dc00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7d180b2880 0x7f7d181f28f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:34062/0 (socket says 172.21.15.152:34062) 2024-04-20T11:37:25.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.484+0000 7f7d1dc00640 1 -- 172.21.15.152:0/2682337582 learned_addr learned my addr 172.21.15.152:0/2682337582 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:25.485 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.484+0000 7f7d1d200640 1 --2- 172.21.15.152:0/2682337582 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d180cd8c0 0x7f7d1815b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:25.485 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.484+0000 7f7d1dc00640 1 -- 172.21.15.152:0/2682337582 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7d18146000 msgr2=0x7f7d1813c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:25.485 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.484+0000 7f7d1dc00640 1 --2- 172.21.15.152:0/2682337582 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7d18146000 0x7f7d1813c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:25.485 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.484+0000 7f7d1dc00640 1 -- 172.21.15.152:0/2682337582 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d180cd8c0 msgr2=0x7f7d1815b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:25.485 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.484+0000 7f7d1dc00640 1 --2- 172.21.15.152:0/2682337582 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d180cd8c0 0x7f7d1815b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:25.485 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.484+0000 7f7d1dc00640 1 -- 172.21.15.152:0/2682337582 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d181f2e30 con 0x7f7d180b2880 2024-04-20T11:37:25.485 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.484+0000 7f7d1d200640 1 --2- 172.21.15.152:0/2682337582 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d180cd8c0 0x7f7d1815b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:37:25.485 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.484+0000 7f7d1dc00640 1 --2- 172.21.15.152:0/2682337582 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7d180b2880 0x7f7d181f28f0 secure :-1 s=READY pgs=2201 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c00c780 tx=0x7f7d0c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:25.485 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.485+0000 7f7d15600640 1 -- 172.21.15.152:0/2682337582 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d0c018070 con 0x7f7d180b2880 2024-04-20T11:37:25.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.485+0000 7f7d15600640 1 -- 172.21.15.152:0/2682337582 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d0c0072d0 con 0x7f7d180b2880 2024-04-20T11:37:25.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.485+0000 7f7d20600640 1 -- 172.21.15.152:0/2682337582 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d1813e620 con 0x7f7d180b2880 2024-04-20T11:37:25.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.485+0000 7f7d15600640 1 -- 172.21.15.152:0/2682337582 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d0c0047e0 con 0x7f7d180b2880 2024-04-20T11:37:25.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.485+0000 7f7d20600640 1 -- 172.21.15.152:0/2682337582 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d1813ebe0 con 0x7f7d180b2880 2024-04-20T11:37:25.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.486+0000 7f7d20600640 1 -- 172.21.15.152:0/2682337582 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d180689f0 con 0x7f7d180b2880 2024-04-20T11:37:25.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.487+0000 7f7d15600640 1 -- 172.21.15.152:0/2682337582 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 26) ==== 91907+0+0 (secure 0 0 0) 0x7f7d0c01b070 con 0x7f7d180b2880 2024-04-20T11:37:25.487 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.488+0000 7f7d15600640 1 --2- 172.21.15.152:0/2682337582 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f7cf806d280 0x7f7cf806f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:25.487 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.488+0000 7f7d15600640 1 -- 172.21.15.152:0/2682337582 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5767+0+0 (secure 0 0 0) 0x7f7d0c089730 con 0x7f7d180b2880 2024-04-20T11:37:25.488 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.489+0000 7f7d1d200640 1 --2- 172.21.15.152:0/2682337582 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f7cf806d280 0x7f7cf806f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:25.489 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.490+0000 7f7d1d200640 1 --2- 172.21.15.152:0/2682337582 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f7cf806d280 0x7f7cf806f740 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f7d180cce10 tx=0x7f7d04009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:25.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.493+0000 7f7d15600640 1 -- 172.21.15.152:0/2682337582 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f7d0c059820 con 0x7f7d180b2880 2024-04-20T11:37:25.691 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.692+0000 7f7d20600640 1 -- 172.21.15.152:0/2682337582 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7d1815c9f0 con 0x7f7d180b2880 2024-04-20T11:37:25.692 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.692+0000 7f7d15600640 1 -- 172.21.15.152:0/2682337582 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v156) ==== 104+0+11 (secure 0 0 0) 0x7f7d0c059120 con 0x7f7d180b2880 2024-04-20T11:37:25.692 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 32 2024-04-20T11:37:25.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.695+0000 7f7d20600640 1 -- 172.21.15.152:0/2682337582 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f7cf806d280 msgr2=0x7f7cf806f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:25.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.695+0000 7f7d20600640 1 --2- 172.21.15.152:0/2682337582 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f7cf806d280 0x7f7cf806f740 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f7d180cce10 tx=0x7f7d04009040 comp rx=0 tx=0).stop 2024-04-20T11:37:25.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.696+0000 7f7d20600640 1 -- 172.21.15.152:0/2682337582 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7d180b2880 msgr2=0x7f7d181f28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:25.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.696+0000 7f7d20600640 1 --2- 172.21.15.152:0/2682337582 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7d180b2880 0x7f7d181f28f0 secure :-1 s=READY pgs=2201 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c00c780 tx=0x7f7d0c00cc40 comp rx=0 tx=0).stop 2024-04-20T11:37:25.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.696+0000 7f7d20600640 1 -- 172.21.15.152:0/2682337582 shutdown_connections 2024-04-20T11:37:25.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.696+0000 7f7d20600640 1 --2- 172.21.15.152:0/2682337582 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f7cf806d280 0x7f7cf806f740 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:25.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.696+0000 7f7d20600640 1 --2- 172.21.15.152:0/2682337582 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7d18146000 0x7f7d1813c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:25.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.696+0000 7f7d20600640 1 --2- 172.21.15.152:0/2682337582 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d180cd8c0 0x7f7d1815b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:25.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.696+0000 7f7d20600640 1 --2- 172.21.15.152:0/2682337582 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7d180b2880 0x7f7d181f28f0 unknown :-1 s=CLOSED pgs=2201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:25.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.696+0000 7f7d20600640 1 -- 172.21.15.152:0/2682337582 >> 172.21.15.152:0/2682337582 conn(0x7f7d1806b980 msgr2=0x7f7d181445c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:25.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.696+0000 7f7d20600640 1 -- 172.21.15.152:0/2682337582 shutdown_connections 2024-04-20T11:37:25.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.696+0000 7f7d20600640 1 -- 172.21.15.152:0/2682337582 wait complete. 2024-04-20T11:37:25.715 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T11:37:25.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.921+0000 7f273b200640 1 Processor -- start 2024-04-20T11:37:25.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.921+0000 7f273b200640 1 -- start start 2024-04-20T11:37:25.922 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.922+0000 7f273b200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f273c15e640 0x7f273c06dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:25.922 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.922+0000 7f273b200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f273c15ea20 0x7f273c06bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:25.922 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.922+0000 7f273b200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f273c0b5070 con 0x7f273c06f410 2024-04-20T11:37:25.922 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.922+0000 7f273b200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f273c06e1a0 con 0x7f273c15e640 2024-04-20T11:37:25.922 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.922+0000 7f273b200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f273c06e480 con 0x7f273c15ea20 2024-04-20T11:37:25.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.922+0000 7f2739e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f273c15e640 0x7f273c06dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:25.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.923+0000 7f2739e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f273c15e640 0x7f273c06dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36206/0 (socket says 172.21.15.152:36206) 2024-04-20T11:37:25.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.923+0000 7f2739e00640 1 -- 172.21.15.152:0/2523580395 learned_addr learned my addr 172.21.15.152:0/2523580395 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:25.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.923+0000 7f2739e00640 1 -- 172.21.15.152:0/2523580395 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f273c15ea20 msgr2=0x7f273c06bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:25.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.923+0000 7f2739e00640 1 --2- 172.21.15.152:0/2523580395 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f273c15ea20 0x7f273c06bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:25.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.923+0000 7f2739e00640 1 -- 172.21.15.152:0/2523580395 >> v1:172.21.15.175:6789/0 conn(0x7f273c06f410 legacy=0x7f273c0b3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:25.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.923+0000 7f2739e00640 1 -- 172.21.15.152:0/2523580395 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f273c06e760 con 0x7f273c15e640 2024-04-20T11:37:25.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.924+0000 7f2739e00640 1 --2- 172.21.15.152:0/2523580395 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f273c15e640 0x7f273c06dbc0 secure :-1 s=READY pgs=2202 cs=0 l=1 rev1=1 crypto rx=0x7f272c009870 tx=0x7f272c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bedd057a45d5ade server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:25.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.924+0000 7f2738a00640 1 -- 172.21.15.152:0/2523580395 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f272c035020 con 0x7f273c15e640 2024-04-20T11:37:25.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.924+0000 7f2738a00640 1 -- 172.21.15.152:0/2523580395 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f272c031640 con 0x7f273c15e640 2024-04-20T11:37:25.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.924+0000 7f2738a00640 1 -- 172.21.15.152:0/2523580395 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f272c02d040 con 0x7f273c15e640 2024-04-20T11:37:25.925 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.925+0000 7f273b200640 1 -- 172.21.15.152:0/2523580395 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f273c15e640 msgr2=0x7f273c06dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:25.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.925+0000 7f273b200640 1 --2- 172.21.15.152:0/2523580395 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f273c15e640 0x7f273c06dbc0 secure :-1 s=READY pgs=2202 cs=0 l=1 rev1=1 crypto rx=0x7f272c009870 tx=0x7f272c02b8f0 comp rx=0 tx=0).stop 2024-04-20T11:37:25.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.925+0000 7f273b200640 1 -- 172.21.15.152:0/2523580395 shutdown_connections 2024-04-20T11:37:25.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.925+0000 7f273b200640 1 --2- 172.21.15.152:0/2523580395 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f273c15ea20 0x7f273c06bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:25.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.925+0000 7f273b200640 1 --2- 172.21.15.152:0/2523580395 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f273c15e640 0x7f273c06dbc0 unknown :-1 s=CLOSED pgs=2202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:25.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.925+0000 7f273b200640 1 -- 172.21.15.152:0/2523580395 >> 172.21.15.152:0/2523580395 conn(0x7f273c0b21e0 msgr2=0x7f273c0b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:25.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.925+0000 7f273b200640 1 -- 172.21.15.152:0/2523580395 shutdown_connections 2024-04-20T11:37:25.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.925+0000 7f273b200640 1 -- 172.21.15.152:0/2523580395 wait complete. 2024-04-20T11:37:25.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.927+0000 7f273b200640 1 Processor -- start 2024-04-20T11:37:25.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.927+0000 7f273b200640 1 -- start start 2024-04-20T11:37:25.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.928+0000 7f273b200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f273c06f410 0x7f273c14baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:25.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.928+0000 7f273b200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f273c15e640 0x7f273c1544b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:25.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.928+0000 7f273b200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f273c1549f0 0x7f273c0b1200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:25.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.928+0000 7f273b200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f273c0ae5d0 con 0x7f273c06f410 2024-04-20T11:37:25.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.928+0000 7f273b200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f273c0ad0a0 con 0x7f273c15e640 2024-04-20T11:37:25.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.928+0000 7f273b200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f273c0ae750 con 0x7f273c1549f0 2024-04-20T11:37:25.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.928+0000 7f2739e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f273c06f410 0x7f273c14baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:25.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.929+0000 7f2739e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f273c06f410 0x7f273c14baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36222/0 (socket says 172.21.15.152:36222) 2024-04-20T11:37:25.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.929+0000 7f2739e00640 1 -- 172.21.15.152:0/1965832916 learned_addr learned my addr 172.21.15.152:0/1965832916 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:25.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.929+0000 7f2739400640 1 --2- 172.21.15.152:0/1965832916 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f273c15e640 0x7f273c1544b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:25.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.929+0000 7f273a800640 1 --2- 172.21.15.152:0/1965832916 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f273c1549f0 0x7f273c0b1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:25.931 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.929+0000 7f2739e00640 1 -- 172.21.15.152:0/1965832916 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f273c1549f0 msgr2=0x7f273c0b1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:25.931 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.929+0000 7f2739e00640 1 --2- 172.21.15.152:0/1965832916 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f273c1549f0 0x7f273c0b1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:25.931 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.929+0000 7f2739e00640 1 -- 172.21.15.152:0/1965832916 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f273c15e640 msgr2=0x7f273c1544b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:25.931 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.929+0000 7f2739e00640 1 --2- 172.21.15.152:0/1965832916 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f273c15e640 0x7f273c1544b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:25.931 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.929+0000 7f2739e00640 1 -- 172.21.15.152:0/1965832916 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f273c154f70 con 0x7f273c06f410 2024-04-20T11:37:25.931 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.929+0000 7f2739e00640 1 --2- 172.21.15.152:0/1965832916 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f273c06f410 0x7f273c14baf0 secure :-1 s=READY pgs=2203 cs=0 l=1 rev1=1 crypto rx=0x7f272c02b930 tx=0x7f272c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:25.931 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.930+0000 7f2732000640 1 -- 172.21.15.152:0/1965832916 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f272c03ac60 con 0x7f273c06f410 2024-04-20T11:37:25.931 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.930+0000 7f2732000640 1 -- 172.21.15.152:0/1965832916 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f272c03ae20 con 0x7f273c06f410 2024-04-20T11:37:25.931 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.930+0000 7f2732000640 1 -- 172.21.15.152:0/1965832916 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f272c044630 con 0x7f273c06f410 2024-04-20T11:37:25.931 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.930+0000 7f273b200640 1 -- 172.21.15.152:0/1965832916 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f273c155250 con 0x7f273c06f410 2024-04-20T11:37:25.931 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.930+0000 7f273b200640 1 -- 172.21.15.152:0/1965832916 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f273c155810 con 0x7f273c06f410 2024-04-20T11:37:25.932 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.931+0000 7f273b200640 1 -- 172.21.15.152:0/1965832916 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f273c0aec90 con 0x7f273c06f410 2024-04-20T11:37:25.932 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.932+0000 7f2732000640 1 -- 172.21.15.152:0/1965832916 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 26) ==== 91907+0+0 (secure 0 0 0) 0x7f272c04b060 con 0x7f273c06f410 2024-04-20T11:37:25.932 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.933+0000 7f2732000640 1 --2- 172.21.15.152:0/1965832916 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f271806d280 0x7f271806f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:25.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.933+0000 7f2732000640 1 -- 172.21.15.152:0/1965832916 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5767+0+0 (secure 0 0 0) 0x7f272c0abfc0 con 0x7f273c06f410 2024-04-20T11:37:25.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.933+0000 7f2739400640 1 --2- 172.21.15.152:0/1965832916 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f271806d280 0x7f271806f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:25.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.935+0000 7f2739400640 1 --2- 172.21.15.152:0/1965832916 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f271806d280 0x7f271806f740 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f273c1f7ae0 tx=0x7f2724009060 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:25.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:25.938+0000 7f2732000640 1 -- 172.21.15.152:0/1965832916 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f272c03f070 con 0x7f273c06f410 2024-04-20T11:37:26.135 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:26.136+0000 7f273b200640 1 -- 172.21.15.152:0/1965832916 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f273c0b4840 con 0x7f273c06f410 2024-04-20T11:37:26.136 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:26.137+0000 7f2732000640 1 -- 172.21.15.152:0/1965832916 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v156) ==== 100+0+11 (secure 0 0 0) 0x7f272c07bff0 con 0x7f273c06f410 2024-04-20T11:37:26.136 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 64 2024-04-20T11:37:26.139 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:26.139+0000 7f273b200640 1 -- 172.21.15.152:0/1965832916 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f271806d280 msgr2=0x7f271806f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:26.140 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:26.139+0000 7f273b200640 1 --2- 172.21.15.152:0/1965832916 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f271806d280 0x7f271806f740 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f273c1f7ae0 tx=0x7f2724009060 comp rx=0 tx=0).stop 2024-04-20T11:37:26.140 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:26.139+0000 7f273b200640 1 -- 172.21.15.152:0/1965832916 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f273c06f410 msgr2=0x7f273c14baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:26.140 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:26.139+0000 7f273b200640 1 --2- 172.21.15.152:0/1965832916 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f273c06f410 0x7f273c14baf0 secure :-1 s=READY pgs=2203 cs=0 l=1 rev1=1 crypto rx=0x7f272c02b930 tx=0x7f272c0046c0 comp rx=0 tx=0).stop 2024-04-20T11:37:26.140 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:26.139+0000 7f273b200640 1 -- 172.21.15.152:0/1965832916 shutdown_connections 2024-04-20T11:37:26.140 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:26.139+0000 7f273b200640 1 --2- 172.21.15.152:0/1965832916 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f271806d280 0x7f271806f740 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:26.140 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:26.139+0000 7f273b200640 1 --2- 172.21.15.152:0/1965832916 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f273c1549f0 0x7f273c0b1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:26.140 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:26.139+0000 7f273b200640 1 --2- 172.21.15.152:0/1965832916 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f273c15e640 0x7f273c1544b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:26.140 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:26.139+0000 7f273b200640 1 --2- 172.21.15.152:0/1965832916 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f273c06f410 0x7f273c14baf0 unknown :-1 s=CLOSED pgs=2203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:26.140 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:26.139+0000 7f273b200640 1 -- 172.21.15.152:0/1965832916 >> 172.21.15.152:0/1965832916 conn(0x7f273c0b21e0 msgr2=0x7f273c0ad5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:26.140 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:26.139+0000 7f273b200640 1 -- 172.21.15.152:0/1965832916 shutdown_connections 2024-04-20T11:37:26.140 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:26.140+0000 7f273b200640 1 -- 172.21.15.152:0/1965832916 wait complete. 2024-04-20T11:37:26.158 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-20T11:37:26.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:26.362+0000 7fe3a7800640 1 Processor -- start 2024-04-20T11:37:26.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:26.362+0000 7fe3a7800640 1 -- start start 2024-04-20T11:37:26.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:26.364+0000 7fe3a7800640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fe3a0145f00 0x7fe3a00cd0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:26.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:26.364+0000 7fe3a7800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe3a00cd610 0x7fe3a006e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:26.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:26.364+0000 7fe3a7800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fe3a006e900 0x7fe3a01601c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:26.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:26.364+0000 7fe3a7800640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fe3a00cf9f0 con 0x7fe3a006e900 2024-04-20T11:37:26.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:26.364+0000 7fe3a7800640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fe3a00cda50 con 0x7fe3a0145f00 2024-04-20T11:37:26.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:26.364+0000 7fe3a7800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe3a006f110 con 0x7fe3a00cd610 2024-04-20T11:37:26.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:26.364+0000 7fe3a5800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fe3a006e900 0x7fe3a01601c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:26.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:26.364+0000 7fe3a5800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fe3a006e900 0x7fe3a01601c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36232/0 (socket says 172.21.15.152:36232) 2024-04-20T11:37:26.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:26.364+0000 7fe3a5800640 1 -- 172.21.15.152:0/3182148970 learned_addr learned my addr 172.21.15.152:0/3182148970 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:26.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:26.364+0000 7fe3a4e00640 1 --2- 172.21.15.152:0/3182148970 >> v2:172.21.15.175:3300/0 conn(0x7fe3a0145f00 0x7fe3a00cd0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:26.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:26.364+0000 7fe39fe00640 1 --2- 172.21.15.152:0/3182148970 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe3a00cd610 0x7fe3a006e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:26.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:26.364+0000 7fe3a5800640 1 -- 172.21.15.152:0/3182148970 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe3a00cd610 msgr2=0x7fe3a006e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:26.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:26.365+0000 7fe3a5800640 1 --2- 172.21.15.152:0/3182148970 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe3a00cd610 0x7fe3a006e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:26.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:26.365+0000 7fe3a5800640 1 -- 172.21.15.152:0/3182148970 >> v2:172.21.15.175:3300/0 conn(0x7fe3a0145f00 msgr2=0x7fe3a00cd0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:26.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:26.365+0000 7fe3a5800640 1 --2- 172.21.15.152:0/3182148970 >> v2:172.21.15.175:3300/0 conn(0x7fe3a0145f00 0x7fe3a00cd0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:26.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:26.365+0000 7fe3a5800640 1 -- 172.21.15.152:0/3182148970 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3a006f3f0 con 0x7fe3a006e900 2024-04-20T11:37:26.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:26.365+0000 7fe39fe00640 1 --2- 172.21.15.152:0/3182148970 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe3a00cd610 0x7fe3a006e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:37:26.366 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:26.366+0000 7fe3a5800640 1 --2- 172.21.15.152:0/3182148970 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe3a006e900 0x7fe3a01601c0 secure :-1 s=READY pgs=2204 cs=0 l=1 rev1=1 crypto rx=0x7fe3a006c840 tx=0x7fe39802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4863af8defe43ef server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:26.366 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:26.366+0000 7fe39f400640 1 -- 172.21.15.152:0/3182148970 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe39802ca80 con 0x7fe3a006e900 2024-04-20T11:37:26.366 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:26.366+0000 7fe39f400640 1 -- 172.21.15.152:0/3182148970 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe39802cc20 con 0x7fe3a006e900 2024-04-20T11:37:26.366 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:26.366+0000 7fe39f400640 1 -- 172.21.15.152:0/3182148970 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3980343f0 con 0x7fe3a006e900 2024-04-20T11:37:26.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:26.367+0000 7fe3a7800640 1 -- 172.21.15.152:0/3182148970 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe3a006e900 msgr2=0x7fe3a01601c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:26.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:26.367+0000 7fe3a7800640 1 --2- 172.21.15.152:0/3182148970 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe3a006e900 0x7fe3a01601c0 secure :-1 s=READY pgs=2204 cs=0 l=1 rev1=1 crypto rx=0x7fe3a006c840 tx=0x7fe39802bee0 comp rx=0 tx=0).stop 2024-04-20T11:37:26.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:26.367+0000 7fe3a7800640 1 -- 172.21.15.152:0/3182148970 shutdown_connections 2024-04-20T11:37:26.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:26.367+0000 7fe3a7800640 1 --2- 172.21.15.152:0/3182148970 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe3a006e900 0x7fe3a01601c0 unknown :-1 s=CLOSED pgs=2204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:26.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:26.367+0000 7fe3a7800640 1 --2- 172.21.15.152:0/3182148970 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe3a00cd610 0x7fe3a006e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:26.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:26.367+0000 7fe3a7800640 1 --2- 172.21.15.152:0/3182148970 >> v2:172.21.15.175:3300/0 conn(0x7fe3a0145f00 0x7fe3a00cd0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:26.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:26.367+0000 7fe3a7800640 1 -- 172.21.15.152:0/3182148970 >> 172.21.15.152:0/3182148970 conn(0x7fe3a0067cf0 msgr2=0x7fe3a006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:26.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:26.367+0000 7fe3a7800640 1 -- 172.21.15.152:0/3182148970 shutdown_connections 2024-04-20T11:37:26.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:26.367+0000 7fe3a7800640 1 -- 172.21.15.152:0/3182148970 wait complete. 2024-04-20T11:37:26.369 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:26.369+0000 7fe3a7800640 1 Processor -- start 2024-04-20T11:37:26.369 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:26.369+0000 7fe3a7800640 1 -- start start 2024-04-20T11:37:26.370 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:26.370+0000 7fe3a7800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe3a006e900 0x7fe3a01f3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:26.370 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:26.370+0000 7fe3a7800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe3a00cd610 0x7fe3a00cb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:26.370 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:26.370+0000 7fe3a7800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe3a0145f00 0x7fe3a00cb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:26.370 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:26.370+0000 7fe3a7800640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a01462e0 con 0x7fe3a006e900 2024-04-20T11:37:26.370 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:26.370+0000 7fe3a7800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a00cf680 con 0x7fe3a0145f00 2024-04-20T11:37:26.370 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:26.370+0000 7fe3a7800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe3a006ed90 con 0x7fe3a00cd610 2024-04-20T11:37:26.371 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:26.370+0000 7fe3a4e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe3a006e900 0x7fe3a01f3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:26.371 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:26.370+0000 7fe3a4e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe3a006e900 0x7fe3a01f3810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36248/0 (socket says 172.21.15.152:36248) 2024-04-20T11:37:26.371 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:26.370+0000 7fe3a4e00640 1 -- 172.21.15.152:0/392083365 learned_addr learned my addr 172.21.15.152:0/392083365 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:26.371 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:26.370+0000 7fe3a5800640 1 --2- 172.21.15.152:0/392083365 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe3a0145f00 0x7fe3a00cb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:26.371 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:26.370+0000 7fe3a4e00640 1 -- 172.21.15.152:0/392083365 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe3a00cd610 msgr2=0x7fe3a00cb1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:26.371 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:26.370+0000 7fe3a4e00640 1 --2- 172.21.15.152:0/392083365 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe3a00cd610 0x7fe3a00cb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:26.371 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:26.370+0000 7fe3a4e00640 1 -- 172.21.15.152:0/392083365 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe3a0145f00 msgr2=0x7fe3a00cb6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:26.371 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:26.370+0000 7fe3a4e00640 1 --2- 172.21.15.152:0/392083365 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe3a0145f00 0x7fe3a00cb6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:26.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:26.370+0000 7fe3a4e00640 1 -- 172.21.15.152:0/392083365 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3a015ee80 con 0x7fe3a006e900 2024-04-20T11:37:26.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:26.370+0000 7fe3a5800640 1 --2- 172.21.15.152:0/392083365 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe3a0145f00 0x7fe3a00cb6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:37:26.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:26.371+0000 7fe3a4e00640 1 --2- 172.21.15.152:0/392083365 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe3a006e900 0x7fe3a01f3810 secure :-1 s=READY pgs=2205 cs=0 l=1 rev1=1 crypto rx=0x7fe39400d5f0 tx=0x7fe39400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:26.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:26.371+0000 7fe39cc00640 1 -- 172.21.15.152:0/392083365 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3940125f0 con 0x7fe3a006e900 2024-04-20T11:37:26.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:26.371+0000 7fe39cc00640 1 -- 172.21.15.152:0/392083365 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe394019070 con 0x7fe3a006e900 2024-04-20T11:37:26.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:26.371+0000 7fe39cc00640 1 -- 172.21.15.152:0/392083365 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3940129d0 con 0x7fe3a006e900 2024-04-20T11:37:26.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:26.371+0000 7fe3a7800640 1 -- 172.21.15.152:0/392083365 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3a00cbcf0 con 0x7fe3a006e900 2024-04-20T11:37:26.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:26.371+0000 7fe3a7800640 1 -- 172.21.15.152:0/392083365 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3a015f400 con 0x7fe3a006e900 2024-04-20T11:37:26.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:26.372+0000 7fe3a7800640 1 -- 172.21.15.152:0/392083365 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3a00689f0 con 0x7fe3a006e900 2024-04-20T11:37:26.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:26.373+0000 7fe39cc00640 1 -- 172.21.15.152:0/392083365 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 26) ==== 91907+0+0 (secure 0 0 0) 0x7fe394012b70 con 0x7fe3a006e900 2024-04-20T11:37:26.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:26.374+0000 7fe39cc00640 1 --2- 172.21.15.152:0/392083365 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe38006d280 0x7fe38006f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:26.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:26.374+0000 7fe39cc00640 1 -- 172.21.15.152:0/392083365 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5767+0+0 (secure 0 0 0) 0x7fe39408a680 con 0x7fe3a006e900 2024-04-20T11:37:26.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:26.375+0000 7fe39fe00640 1 --2- 172.21.15.152:0/392083365 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe38006d280 0x7fe38006f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:26.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:26.376+0000 7fe39fe00640 1 --2- 172.21.15.152:0/392083365 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe38006d280 0x7fe38006f740 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fe38c006fd0 tx=0x7fe38c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:26.378 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:26.379+0000 7fe39cc00640 1 -- 172.21.15.152:0/392083365 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe39405a6b0 con 0x7fe3a006e900 2024-04-20T11:37:26.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:26.596+0000 7fe3a7800640 1 -- 172.21.15.152:0/392083365 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe3a015da80 con 0x7fe3a006e900 2024-04-20T11:37:26.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:26.596+0000 7fe39cc00640 1 -- 172.21.15.152:0/392083365 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v156) ==== 80+0+0 (secure 0 0 0) 0x7fe394059fb0 con 0x7fe3a006e900 2024-04-20T11:37:26.596 INFO:teuthology.orchestra.run.smithi152.stderr:listed 0 entries 2024-04-20T11:37:26.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:26.599+0000 7fe3a7800640 1 -- 172.21.15.152:0/392083365 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe38006d280 msgr2=0x7fe38006f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:26.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:26.599+0000 7fe3a7800640 1 --2- 172.21.15.152:0/392083365 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe38006d280 0x7fe38006f740 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fe38c006fd0 tx=0x7fe38c008040 comp rx=0 tx=0).stop 2024-04-20T11:37:26.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:26.599+0000 7fe3a7800640 1 -- 172.21.15.152:0/392083365 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe3a006e900 msgr2=0x7fe3a01f3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:26.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:26.599+0000 7fe3a7800640 1 --2- 172.21.15.152:0/392083365 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe3a006e900 0x7fe3a01f3810 secure :-1 s=READY pgs=2205 cs=0 l=1 rev1=1 crypto rx=0x7fe39400d5f0 tx=0x7fe39400dab0 comp rx=0 tx=0).stop 2024-04-20T11:37:26.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:26.599+0000 7fe3a7800640 1 -- 172.21.15.152:0/392083365 shutdown_connections 2024-04-20T11:37:26.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:26.599+0000 7fe3a7800640 1 --2- 172.21.15.152:0/392083365 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe38006d280 0x7fe38006f740 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:26.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:26.599+0000 7fe3a7800640 1 --2- 172.21.15.152:0/392083365 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe3a0145f00 0x7fe3a00cb6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:26.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:26.599+0000 7fe3a7800640 1 --2- 172.21.15.152:0/392083365 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe3a00cd610 0x7fe3a00cb1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:26.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:26.599+0000 7fe3a7800640 1 --2- 172.21.15.152:0/392083365 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe3a006e900 0x7fe3a01f3810 unknown :-1 s=CLOSED pgs=2205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:26.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:26.599+0000 7fe3a7800640 1 -- 172.21.15.152:0/392083365 >> 172.21.15.152:0/392083365 conn(0x7fe3a0067cf0 msgr2=0x7fe3a00b5050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:26.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:26.600+0000 7fe3a7800640 1 -- 172.21.15.152:0/392083365 shutdown_connections 2024-04-20T11:37:26.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:26.600+0000 7fe3a7800640 1 -- 172.21.15.152:0/392083365 wait complete. 2024-04-20T11:37:26.617 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi152.front.sepia.ceph.com... 2024-04-20T11:37:26.617 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:37:26.617 DEBUG:teuthology.orchestra.run.smithi152:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-20T11:37:26.691 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi152.stderr:ceph-fuse[71523]: fuse finished with error 0 and tester_r 0 2024-04-20T11:37:26.696 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi152.front.sepia.ceph.com 2024-04-20T11:37:26.696 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:37:26.696 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-20T11:37:26.893 INFO:tasks.cephfs.mount:Terminating background process 2024-04-20T11:37:26.893 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T11:37:26.893 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:37:26.945 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:37:27.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.150+0000 7fe144200640 1 Processor -- start 2024-04-20T11:37:27.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.150+0000 7fe144200640 1 -- start start 2024-04-20T11:37:27.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.151+0000 7fe144200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe13c15b5e0 0x7fe13c06eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:27.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.152+0000 7fe144200640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe13c0b5770 con 0x7fe13c0b4c50 2024-04-20T11:37:27.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.152+0000 7fe144200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe13c06e060 con 0x7fe13c06fa50 2024-04-20T11:37:27.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.152+0000 7fe144200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe13c06de10 con 0x7fe13c15b5e0 2024-04-20T11:37:27.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.152+0000 7fe140e00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fe13c0b4c50 0x7fe13c06f340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:40768/0 (socket says 172.21.15.152:40768) 2024-04-20T11:37:27.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.152+0000 7fe140e00640 1 -- 172.21.15.152:0/2379391841 learned_addr learned my addr 172.21.15.152:0/2379391841 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:37:27.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.152+0000 7fe141800640 1 --2- 172.21.15.152:0/2379391841 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe13c15b5e0 0x7fe13c06eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:27.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.152+0000 7fe13be00640 1 -- 172.21.15.152:0/2379391841 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3912688129 0 0) 0x7fe13c0b5770 con 0x7fe13c0b4c50 2024-04-20T11:37:27.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.152+0000 7fe13be00640 1 -- 172.21.15.152:0/2379391841 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe124003520 con 0x7fe13c0b4c50 2024-04-20T11:37:27.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.153+0000 7fe13be00640 1 -- 172.21.15.152:0/2379391841 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1205418130 0 0) 0x7fe124003520 con 0x7fe13c0b4c50 2024-04-20T11:37:27.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.153+0000 7fe13be00640 1 -- 172.21.15.152:0/2379391841 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe13c15b5e0 msgr2=0x7fe13c06eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:27.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.153+0000 7fe13be00640 1 --2- 172.21.15.152:0/2379391841 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe13c15b5e0 0x7fe13c06eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:27.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.153+0000 7fe13be00640 1 -- 172.21.15.152:0/2379391841 >> v1:172.21.15.175:6789/0 conn(0x7fe13c06fa50 legacy=0x7fe13c15cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:27.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.153+0000 7fe13be00640 1 -- 172.21.15.152:0/2379391841 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe13c06e340 con 0x7fe13c0b4c50 2024-04-20T11:37:27.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.153+0000 7fe13be00640 1 -- 172.21.15.152:0/2379391841 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7fe128001690 con 0x7fe13c0b4c50 2024-04-20T11:37:27.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.153+0000 7fe13be00640 1 -- 172.21.15.152:0/2379391841 >> v1:172.21.15.152:6789/0 conn(0x7fe13c0b4c50 legacy=0x7fe13c06f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:27.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.153+0000 7fe13be00640 1 --2- 172.21.15.152:0/2379391841 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe124003a50 0x7fe124003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:27.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.153+0000 7fe13be00640 1 --2- 172.21.15.152:0/2379391841 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe13c15b5e0 0x7fe124004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:27.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.153+0000 7fe13be00640 1 --2- 172.21.15.152:0/2379391841 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe124004d40 0x7fe1240051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:27.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.153+0000 7fe13be00640 1 -- 172.21.15.152:0/2379391841 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fe13c06de10 con 0x7fe124003a50 2024-04-20T11:37:27.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.153+0000 7fe13be00640 1 -- 172.21.15.152:0/2379391841 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe12400bb50 con 0x7fe124004d40 2024-04-20T11:37:27.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.153+0000 7fe13be00640 1 -- 172.21.15.152:0/2379391841 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe12400bd20 con 0x7fe13c15b5e0 2024-04-20T11:37:27.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.153+0000 7fe141800640 1 --2- 172.21.15.152:0/2379391841 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe124003a50 0x7fe124003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:27.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.154+0000 7fe141800640 1 -- 172.21.15.152:0/2379391841 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe13c15b5e0 msgr2=0x7fe124004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:27.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.154+0000 7fe141800640 1 --2- 172.21.15.152:0/2379391841 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe13c15b5e0 0x7fe124004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:27.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.154+0000 7fe141800640 1 -- 172.21.15.152:0/2379391841 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe124004d40 msgr2=0x7fe1240051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:27.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.154+0000 7fe142200640 1 --2- 172.21.15.152:0/2379391841 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe124004d40 0x7fe1240051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:27.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.154+0000 7fe141800640 1 --2- 172.21.15.152:0/2379391841 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe124004d40 0x7fe1240051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:27.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.154+0000 7fe141800640 1 -- 172.21.15.152:0/2379391841 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe124003520 con 0x7fe124003a50 2024-04-20T11:37:27.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.155+0000 7fe141800640 1 --2- 172.21.15.152:0/2379391841 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe124003a50 0x7fe124003e70 secure :-1 s=READY pgs=2207 cs=0 l=1 rev1=1 crypto rx=0x7fe13000bd60 tx=0x7fe124005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7833e4f4cd2b515d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:27.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.155+0000 7fe13be00640 1 -- 172.21.15.152:0/2379391841 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe13000b930 con 0x7fe124003a50 2024-04-20T11:37:27.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.155+0000 7fe13be00640 1 -- 172.21.15.152:0/2379391841 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1300306e0 con 0x7fe124003a50 2024-04-20T11:37:27.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.155+0000 7fe13be00640 1 -- 172.21.15.152:0/2379391841 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe130030ab0 con 0x7fe124003a50 2024-04-20T11:37:27.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.156+0000 7fe144200640 1 -- 172.21.15.152:0/2379391841 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe124003a50 msgr2=0x7fe124003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:27.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.156+0000 7fe144200640 1 --2- 172.21.15.152:0/2379391841 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe124003a50 0x7fe124003e70 secure :-1 s=READY pgs=2207 cs=0 l=1 rev1=1 crypto rx=0x7fe13000bd60 tx=0x7fe124005830 comp rx=0 tx=0).stop 2024-04-20T11:37:27.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.156+0000 7fe142200640 1 -- 172.21.15.152:0/2379391841 reap_dead start 2024-04-20T11:37:27.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.156+0000 7fe144200640 1 -- 172.21.15.152:0/2379391841 shutdown_connections 2024-04-20T11:37:27.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.156+0000 7fe144200640 1 -- 172.21.15.152:0/2379391841 >> 172.21.15.152:0/2379391841 conn(0x7fe13c067cf0 msgr2=0x7fe13c06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:27.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.156+0000 7fe144200640 1 -- 172.21.15.152:0/2379391841 shutdown_connections 2024-04-20T11:37:27.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.156+0000 7fe144200640 1 -- 172.21.15.152:0/2379391841 wait complete. 2024-04-20T11:37:27.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.158+0000 7fe144200640 1 Processor -- start 2024-04-20T11:37:27.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.158+0000 7fe144200640 1 -- start start 2024-04-20T11:37:27.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.159+0000 7fe144200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe124003a50 0x7fe13c1e4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:27.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.159+0000 7fe144200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe13c0ae8c0 0x7fe13c1595c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:27.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.159+0000 7fe144200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe13c155ae0 0x7fe13c155f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:27.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.159+0000 7fe144200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fe13c0b42d0 con 0x7fe13c0ae8c0 2024-04-20T11:37:27.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.159+0000 7fe144200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe13c15c9d0 con 0x7fe13c155ae0 2024-04-20T11:37:27.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.159+0000 7fe144200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe13c06eb40 con 0x7fe124003a50 2024-04-20T11:37:27.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.159+0000 7fe140e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe13c0ae8c0 0x7fe13c1595c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:27.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.159+0000 7fe140e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe13c0ae8c0 0x7fe13c1595c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36254/0 (socket says 172.21.15.152:36254) 2024-04-20T11:37:27.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.159+0000 7fe140e00640 1 -- 172.21.15.152:0/672073132 learned_addr learned my addr 172.21.15.152:0/672073132 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:27.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.159+0000 7fe141800640 1 --2- 172.21.15.152:0/672073132 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe124003a50 0x7fe13c1e4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:27.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.160+0000 7fe142200640 1 --2- 172.21.15.152:0/672073132 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe13c155ae0 0x7fe13c155f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:27.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.160+0000 7fe140e00640 1 -- 172.21.15.152:0/672073132 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe124003a50 msgr2=0x7fe13c1e4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:27.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.160+0000 7fe140e00640 1 --2- 172.21.15.152:0/672073132 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe124003a50 0x7fe13c1e4850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:27.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.160+0000 7fe140e00640 1 -- 172.21.15.152:0/672073132 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe13c155ae0 msgr2=0x7fe13c155f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:27.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.160+0000 7fe140e00640 1 --2- 172.21.15.152:0/672073132 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe13c155ae0 0x7fe13c155f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:27.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.160+0000 7fe140e00640 1 -- 172.21.15.152:0/672073132 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe13c1e50c0 con 0x7fe13c0ae8c0 2024-04-20T11:37:27.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.160+0000 7fe141800640 1 --2- 172.21.15.152:0/672073132 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe124003a50 0x7fe13c1e4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:37:27.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.160+0000 7fe142200640 1 --2- 172.21.15.152:0/672073132 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe13c155ae0 0x7fe13c155f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:37:27.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.160+0000 7fe140e00640 1 --2- 172.21.15.152:0/672073132 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe13c0ae8c0 0x7fe13c1595c0 secure :-1 s=READY pgs=2208 cs=0 l=1 rev1=1 crypto rx=0x7fe12800c730 tx=0x7fe12800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:27.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.160+0000 7fe139600640 1 -- 172.21.15.152:0/672073132 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe128018070 con 0x7fe13c0ae8c0 2024-04-20T11:37:27.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.160+0000 7fe139600640 1 -- 172.21.15.152:0/672073132 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1280072d0 con 0x7fe13c0ae8c0 2024-04-20T11:37:27.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.160+0000 7fe139600640 1 -- 172.21.15.152:0/672073132 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1280047e0 con 0x7fe13c0ae8c0 2024-04-20T11:37:27.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.160+0000 7fe144200640 1 -- 172.21.15.152:0/672073132 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe13c1e53a0 con 0x7fe13c0ae8c0 2024-04-20T11:37:27.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.160+0000 7fe144200640 1 -- 172.21.15.152:0/672073132 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe13c1e5960 con 0x7fe13c0ae8c0 2024-04-20T11:37:27.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.161+0000 7fe144200640 1 -- 172.21.15.152:0/672073132 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe13c0689f0 con 0x7fe13c0ae8c0 2024-04-20T11:37:27.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.162+0000 7fe139600640 1 -- 172.21.15.152:0/672073132 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 26) ==== 91907+0+0 (secure 0 0 0) 0x7fe12801b070 con 0x7fe13c0ae8c0 2024-04-20T11:37:27.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.163+0000 7fe139600640 1 --2- 172.21.15.152:0/672073132 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe11c06d280 0x7fe11c06f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:27.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.164+0000 7fe139600640 1 -- 172.21.15.152:0/672073132 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5767+0+0 (secure 0 0 0) 0x7fe128089980 con 0x7fe13c0ae8c0 2024-04-20T11:37:27.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.164+0000 7fe141800640 1 --2- 172.21.15.152:0/672073132 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe11c06d280 0x7fe11c06f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:27.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.165+0000 7fe141800640 1 --2- 172.21.15.152:0/672073132 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe11c06d280 0x7fe11c06f740 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fe130039000 tx=0x7fe130034000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:27.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.170+0000 7fe139600640 1 -- 172.21.15.152:0/672073132 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe1280599b0 con 0x7fe13c0ae8c0 2024-04-20T11:37:27.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.373+0000 7fe144200640 1 -- 172.21.15.152:0/672073132 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe13c0cd190 con 0x7fe13c0ae8c0 2024-04-20T11:37:27.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.376+0000 7fe139600640 1 -- 172.21.15.152:0/672073132 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v156) ==== 74+0+16781 (secure 0 0 0) 0x7fe1280592b0 con 0x7fe13c0ae8c0 2024-04-20T11:37:27.376 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:37:27.376 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":156,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:37:24.837045+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:36:57.520028+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"153","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2024-04-20T11:36:58.760637+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"154","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"25.13","mappings":[{"from":0,"to":1}]},{"pgid":"25.15","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:37:27.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.379+0000 7fe144200640 1 -- 172.21.15.152:0/672073132 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe11c06d280 msgr2=0x7fe11c06f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:27.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.379+0000 7fe144200640 1 --2- 172.21.15.152:0/672073132 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe11c06d280 0x7fe11c06f740 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fe130039000 tx=0x7fe130034000 comp rx=0 tx=0).stop 2024-04-20T11:37:27.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.379+0000 7fe144200640 1 -- 172.21.15.152:0/672073132 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe13c0ae8c0 msgr2=0x7fe13c1595c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:27.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.379+0000 7fe144200640 1 --2- 172.21.15.152:0/672073132 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe13c0ae8c0 0x7fe13c1595c0 secure :-1 s=READY pgs=2208 cs=0 l=1 rev1=1 crypto rx=0x7fe12800c730 tx=0x7fe12800cbf0 comp rx=0 tx=0).stop 2024-04-20T11:37:27.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.379+0000 7fe144200640 1 -- 172.21.15.152:0/672073132 shutdown_connections 2024-04-20T11:37:27.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.379+0000 7fe144200640 1 --2- 172.21.15.152:0/672073132 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe11c06d280 0x7fe11c06f740 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:27.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.379+0000 7fe144200640 1 --2- 172.21.15.152:0/672073132 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe13c155ae0 0x7fe13c155f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:27.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.379+0000 7fe144200640 1 --2- 172.21.15.152:0/672073132 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe13c0ae8c0 0x7fe13c1595c0 unknown :-1 s=CLOSED pgs=2208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:27.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.379+0000 7fe144200640 1 --2- 172.21.15.152:0/672073132 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe124003a50 0x7fe13c1e4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:27.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.379+0000 7fe144200640 1 -- 172.21.15.152:0/672073132 >> 172.21.15.152:0/672073132 conn(0x7fe13c067cf0 msgr2=0x7fe13c0b2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:27.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.379+0000 7fe144200640 1 -- 172.21.15.152:0/672073132 shutdown_connections 2024-04-20T11:37:27.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.379+0000 7fe144200640 1 -- 172.21.15.152:0/672073132 wait complete. 2024-04-20T11:37:27.398 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T11:26:47.246571+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 24, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T11:36:57.520028+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '153', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 25, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T11:36:58.760637+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '154', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T11:37:27.398 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T11:37:27.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.606+0000 7efc62200640 1 Processor -- start 2024-04-20T11:37:27.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.606+0000 7efc62200640 1 -- start start 2024-04-20T11:37:27.607 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.607+0000 7efc62200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7efc5c15e5d0 0x7efc5c162440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:27.607 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.607+0000 7efc62200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efc5c0b3fa0 0x7efc5c0af490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:27.607 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.607+0000 7efc62200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc5c0adc30 con 0x7efc5c15e9b0 2024-04-20T11:37:27.607 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.607+0000 7efc62200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7efc5c162a70 con 0x7efc5c15e5d0 2024-04-20T11:37:27.608 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.607+0000 7efc62200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7efc5c162d50 con 0x7efc5c0b3fa0 2024-04-20T11:37:27.608 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.608+0000 7efc5b400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7efc5c15e5d0 0x7efc5c162440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:27.608 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.608+0000 7efc5b400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7efc5c15e5d0 0x7efc5c162440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36264/0 (socket says 172.21.15.152:36264) 2024-04-20T11:37:27.608 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.608+0000 7efc5b400640 1 -- 172.21.15.152:0/643081168 learned_addr learned my addr 172.21.15.152:0/643081168 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:27.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.608+0000 7efc5b400640 1 -- 172.21.15.152:0/643081168 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efc5c0b3fa0 msgr2=0x7efc5c0af490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:27.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.608+0000 7efc5b400640 1 --2- 172.21.15.152:0/643081168 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efc5c0b3fa0 0x7efc5c0af490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:27.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.608+0000 7efc5b400640 1 -- 172.21.15.152:0/643081168 >> v1:172.21.15.175:6789/0 conn(0x7efc5c15e9b0 legacy=0x7efc5c071db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:27.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.608+0000 7efc5b400640 1 -- 172.21.15.152:0/643081168 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc5c163030 con 0x7efc5c15e5d0 2024-04-20T11:37:27.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.609+0000 7efc5b400640 1 --2- 172.21.15.152:0/643081168 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efc5c15e5d0 0x7efc5c162440 secure :-1 s=READY pgs=2209 cs=0 l=1 rev1=1 crypto rx=0x7efc5c06ff00 tx=0x7efc4c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5eb355d524c477f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:27.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.610+0000 7efc5aa00640 1 -- 172.21.15.152:0/643081168 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc4c035020 con 0x7efc5c15e5d0 2024-04-20T11:37:27.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.610+0000 7efc5aa00640 1 -- 172.21.15.152:0/643081168 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc4c031640 con 0x7efc5c15e5d0 2024-04-20T11:37:27.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.610+0000 7efc5aa00640 1 -- 172.21.15.152:0/643081168 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc4c02d040 con 0x7efc5c15e5d0 2024-04-20T11:37:27.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.610+0000 7efc62200640 1 -- 172.21.15.152:0/643081168 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efc5c15e5d0 msgr2=0x7efc5c162440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:27.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.610+0000 7efc62200640 1 --2- 172.21.15.152:0/643081168 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efc5c15e5d0 0x7efc5c162440 secure :-1 s=READY pgs=2209 cs=0 l=1 rev1=1 crypto rx=0x7efc5c06ff00 tx=0x7efc4c02b8f0 comp rx=0 tx=0).stop 2024-04-20T11:37:27.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.611+0000 7efc62200640 1 -- 172.21.15.152:0/643081168 shutdown_connections 2024-04-20T11:37:27.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.611+0000 7efc62200640 1 --2- 172.21.15.152:0/643081168 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efc5c0b3fa0 0x7efc5c0af490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:27.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.611+0000 7efc62200640 1 --2- 172.21.15.152:0/643081168 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efc5c15e5d0 0x7efc5c162440 unknown :-1 s=CLOSED pgs=2209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:27.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.611+0000 7efc62200640 1 -- 172.21.15.152:0/643081168 >> 172.21.15.152:0/643081168 conn(0x7efc5c06b980 msgr2=0x7efc5c06bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:27.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.611+0000 7efc62200640 1 -- 172.21.15.152:0/643081168 shutdown_connections 2024-04-20T11:37:27.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.611+0000 7efc62200640 1 -- 172.21.15.152:0/643081168 wait complete. 2024-04-20T11:37:27.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.612+0000 7efc62200640 1 Processor -- start 2024-04-20T11:37:27.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.613+0000 7efc62200640 1 -- start start 2024-04-20T11:37:27.614 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.613+0000 7efc62200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efc5c0b3fa0 0x7efc5c14fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:27.614 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.613+0000 7efc62200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efc5c15e5d0 0x7efc5c161820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:27.614 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.613+0000 7efc62200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efc5c15e9b0 0x7efc5c0ae110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:27.614 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.613+0000 7efc62200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7efc5c06e0a0 con 0x7efc5c15e5d0 2024-04-20T11:37:27.614 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.613+0000 7efc62200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7efc5c163310 con 0x7efc5c0b3fa0 2024-04-20T11:37:27.614 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.613+0000 7efc62200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7efc5c06e220 con 0x7efc5c15e9b0 2024-04-20T11:37:27.614 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.614+0000 7efc52a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efc5c15e5d0 0x7efc5c161820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:27.614 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.614+0000 7efc52a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efc5c15e5d0 0x7efc5c161820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36278/0 (socket says 172.21.15.152:36278) 2024-04-20T11:37:27.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.614+0000 7efc52a00640 1 -- 172.21.15.152:0/3506814276 learned_addr learned my addr 172.21.15.152:0/3506814276 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:27.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.614+0000 7efc5b400640 1 --2- 172.21.15.152:0/3506814276 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efc5c0b3fa0 0x7efc5c14fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:27.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.614+0000 7efc5be00640 1 --2- 172.21.15.152:0/3506814276 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efc5c15e9b0 0x7efc5c0ae110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:27.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.614+0000 7efc52a00640 1 -- 172.21.15.152:0/3506814276 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efc5c15e9b0 msgr2=0x7efc5c0ae110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:27.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.614+0000 7efc52a00640 1 --2- 172.21.15.152:0/3506814276 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efc5c15e9b0 0x7efc5c0ae110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:27.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.614+0000 7efc52a00640 1 -- 172.21.15.152:0/3506814276 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efc5c0b3fa0 msgr2=0x7efc5c14fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:27.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.614+0000 7efc52a00640 1 --2- 172.21.15.152:0/3506814276 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efc5c0b3fa0 0x7efc5c14fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:27.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.614+0000 7efc52a00640 1 -- 172.21.15.152:0/3506814276 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc5c15f360 con 0x7efc5c15e5d0 2024-04-20T11:37:27.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.614+0000 7efc5be00640 1 --2- 172.21.15.152:0/3506814276 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efc5c15e9b0 0x7efc5c0ae110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:37:27.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.614+0000 7efc5b400640 1 --2- 172.21.15.152:0/3506814276 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efc5c0b3fa0 0x7efc5c14fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:37:27.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.615+0000 7efc52a00640 1 --2- 172.21.15.152:0/3506814276 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efc5c15e5d0 0x7efc5c161820 secure :-1 s=READY pgs=2210 cs=0 l=1 rev1=1 crypto rx=0x7efc4400b4d0 tx=0x7efc4400b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:27.616 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.615+0000 7efc53e00640 1 -- 172.21.15.152:0/3506814276 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc44014070 con 0x7efc5c15e5d0 2024-04-20T11:37:27.616 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.615+0000 7efc53e00640 1 -- 172.21.15.152:0/3506814276 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc44010870 con 0x7efc5c15e5d0 2024-04-20T11:37:27.616 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.615+0000 7efc53e00640 1 -- 172.21.15.152:0/3506814276 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc44010c60 con 0x7efc5c15e5d0 2024-04-20T11:37:27.616 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.615+0000 7efc62200640 1 -- 172.21.15.152:0/3506814276 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc5c15f640 con 0x7efc5c15e5d0 2024-04-20T11:37:27.616 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.615+0000 7efc62200640 1 -- 172.21.15.152:0/3506814276 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc5c15fc00 con 0x7efc5c15e5d0 2024-04-20T11:37:27.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.616+0000 7efc62200640 1 -- 172.21.15.152:0/3506814276 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc5c0689f0 con 0x7efc5c15e5d0 2024-04-20T11:37:27.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.617+0000 7efc53e00640 1 -- 172.21.15.152:0/3506814276 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 26) ==== 91907+0+0 (secure 0 0 0) 0x7efc44017070 con 0x7efc5c15e5d0 2024-04-20T11:37:27.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.618+0000 7efc53e00640 1 --2- 172.21.15.152:0/3506814276 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7efc3806d280 0x7efc3806f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:27.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.618+0000 7efc53e00640 1 -- 172.21.15.152:0/3506814276 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5767+0+0 (secure 0 0 0) 0x7efc4408a0b0 con 0x7efc5c15e5d0 2024-04-20T11:37:27.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.619+0000 7efc5b400640 1 --2- 172.21.15.152:0/3506814276 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7efc3806d280 0x7efc3806f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:27.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.620+0000 7efc5b400640 1 --2- 172.21.15.152:0/3506814276 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7efc3806d280 0x7efc3806f740 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7efc4c036000 tx=0x7efc4c03c040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:27.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.622+0000 7efc53e00640 1 -- 172.21.15.152:0/3506814276 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7efc4405a0e0 con 0x7efc5c15e5d0 2024-04-20T11:37:27.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.824+0000 7efc62200640 1 -- 172.21.15.152:0/3506814276 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7efc5c0b1c60 con 0x7efc5c15e5d0 2024-04-20T11:37:27.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.825+0000 7efc53e00640 1 -- 172.21.15.152:0/3506814276 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v156) ==== 93+0+10 (secure 0 0 0) 0x7efc440599e0 con 0x7efc5c15e5d0 2024-04-20T11:37:27.825 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 1 2024-04-20T11:37:27.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.828+0000 7efc62200640 1 -- 172.21.15.152:0/3506814276 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7efc3806d280 msgr2=0x7efc3806f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:27.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.828+0000 7efc62200640 1 --2- 172.21.15.152:0/3506814276 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7efc3806d280 0x7efc3806f740 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7efc4c036000 tx=0x7efc4c03c040 comp rx=0 tx=0).stop 2024-04-20T11:37:27.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.828+0000 7efc62200640 1 -- 172.21.15.152:0/3506814276 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efc5c15e5d0 msgr2=0x7efc5c161820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:27.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.828+0000 7efc62200640 1 --2- 172.21.15.152:0/3506814276 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efc5c15e5d0 0x7efc5c161820 secure :-1 s=READY pgs=2210 cs=0 l=1 rev1=1 crypto rx=0x7efc4400b4d0 tx=0x7efc4400b990 comp rx=0 tx=0).stop 2024-04-20T11:37:27.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.828+0000 7efc62200640 1 -- 172.21.15.152:0/3506814276 shutdown_connections 2024-04-20T11:37:27.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.828+0000 7efc62200640 1 --2- 172.21.15.152:0/3506814276 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7efc3806d280 0x7efc3806f740 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:27.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.828+0000 7efc62200640 1 --2- 172.21.15.152:0/3506814276 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efc5c15e9b0 0x7efc5c0ae110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:27.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.828+0000 7efc62200640 1 --2- 172.21.15.152:0/3506814276 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efc5c15e5d0 0x7efc5c161820 unknown :-1 s=CLOSED pgs=2210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:27.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.828+0000 7efc62200640 1 --2- 172.21.15.152:0/3506814276 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efc5c0b3fa0 0x7efc5c14fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:27.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.828+0000 7efc62200640 1 -- 172.21.15.152:0/3506814276 >> 172.21.15.152:0/3506814276 conn(0x7efc5c06b980 msgr2=0x7efc5c0b2180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:27.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.828+0000 7efc62200640 1 -- 172.21.15.152:0/3506814276 shutdown_connections 2024-04-20T11:37:27.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:27.828+0000 7efc62200640 1 -- 172.21.15.152:0/3506814276 wait complete. 2024-04-20T11:37:27.846 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T11:37:28.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.045+0000 7f26aec00640 1 Processor -- start 2024-04-20T11:37:28.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.046+0000 7f26aec00640 1 -- start start 2024-04-20T11:37:28.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.047+0000 7f26aec00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f26a80ba430 0x7f26a80b7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:28.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.047+0000 7f26aec00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26a815ec70 con 0x7f26a80b7cc0 2024-04-20T11:37:28.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.047+0000 7f26aec00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26a80d2ef0 con 0x7f26a815e890 2024-04-20T11:37:28.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.047+0000 7f26aec00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f26a80b8e20 con 0x7f26a80ba430 2024-04-20T11:37:28.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.047+0000 7f26a7400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f26a80ba430 0x7f26a80b7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:28.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.047+0000 7f26acc00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f26a80b7cc0 0x7f26a80d1790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:40778/0 (socket says 172.21.15.152:40778) 2024-04-20T11:37:28.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.047+0000 7f26acc00640 1 -- 172.21.15.152:0/1013581801 learned_addr learned my addr 172.21.15.152:0/1013581801 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:37:28.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.048+0000 7f26a7400640 1 -- 172.21.15.152:0/1013581801 >> v1:172.21.15.175:6789/0 conn(0x7f26a815e890 legacy=0x7f26a80d27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:28.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.048+0000 7f26a7400640 1 -- 172.21.15.152:0/1013581801 >> v1:172.21.15.152:6789/0 conn(0x7f26a80b7cc0 legacy=0x7f26a80d1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:28.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.048+0000 7f26a7400640 1 -- 172.21.15.152:0/1013581801 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26a80b8210 con 0x7f26a80ba430 2024-04-20T11:37:28.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.048+0000 7f26a6a00640 1 -- 172.21.15.152:0/1013581801 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2195273918 0 0) 0x7f26a815ec70 con 0x7f26a80b7cc0 2024-04-20T11:37:28.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.049+0000 7f26a7400640 1 --2- 172.21.15.152:0/1013581801 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f26a80ba430 0x7f26a80b7780 secure :-1 s=READY pgs=2212 cs=0 l=1 rev1=1 crypto rx=0x7f26a80b8fa0 tx=0x7f269002bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58a742c3b784fa31 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:28.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.049+0000 7f26a6a00640 1 -- 172.21.15.152:0/1013581801 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2690039070 con 0x7f26a80ba430 2024-04-20T11:37:28.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.049+0000 7f26a6a00640 1 -- 172.21.15.152:0/1013581801 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f269002cce0 con 0x7f26a80ba430 2024-04-20T11:37:28.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.049+0000 7f26a6a00640 1 -- 172.21.15.152:0/1013581801 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2690034660 con 0x7f26a80ba430 2024-04-20T11:37:28.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.050+0000 7f26aec00640 1 -- 172.21.15.152:0/1013581801 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f26a80ba430 msgr2=0x7f26a80b7780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:28.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.050+0000 7f26aec00640 1 --2- 172.21.15.152:0/1013581801 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f26a80ba430 0x7f26a80b7780 secure :-1 s=READY pgs=2212 cs=0 l=1 rev1=1 crypto rx=0x7f26a80b8fa0 tx=0x7f269002bf30 comp rx=0 tx=0).stop 2024-04-20T11:37:28.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.050+0000 7f26aec00640 1 -- 172.21.15.152:0/1013581801 shutdown_connections 2024-04-20T11:37:28.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.050+0000 7f26aec00640 1 --2- 172.21.15.152:0/1013581801 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f26a80ba430 0x7f26a80b7780 unknown :-1 s=CLOSED pgs=2212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:28.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.050+0000 7f26aec00640 1 -- 172.21.15.152:0/1013581801 >> 172.21.15.152:0/1013581801 conn(0x7f26a8067ae0 msgr2=0x7f26a8067ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:28.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.050+0000 7f26aec00640 1 -- 172.21.15.152:0/1013581801 shutdown_connections 2024-04-20T11:37:28.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.050+0000 7f26aec00640 1 -- 172.21.15.152:0/1013581801 wait complete. 2024-04-20T11:37:28.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.052+0000 7f26aec00640 1 Processor -- start 2024-04-20T11:37:28.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.052+0000 7f26aec00640 1 -- start start 2024-04-20T11:37:28.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.053+0000 7f26aec00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f26a80b7cc0 0x7f26a8146900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:28.054 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.053+0000 7f26aec00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26a80ba430 0x7f26a80ca580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:28.054 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.053+0000 7f26aec00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f26a815e890 0x7f26a80caac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:28.054 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.053+0000 7f26aec00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f26a80b8ab0 con 0x7f26a80b7cc0 2024-04-20T11:37:28.054 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.053+0000 7f26aec00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f26a80b87d0 con 0x7f26a80ba430 2024-04-20T11:37:28.054 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.053+0000 7f26aec00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f26a80d1ea0 con 0x7f26a815e890 2024-04-20T11:37:28.054 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.053+0000 7f26a7e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f26a80b7cc0 0x7f26a8146900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:28.054 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.053+0000 7f26a7e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f26a80b7cc0 0x7f26a8146900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36294/0 (socket says 172.21.15.152:36294) 2024-04-20T11:37:28.054 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.053+0000 7f26a7e00640 1 -- 172.21.15.152:0/1631172381 learned_addr learned my addr 172.21.15.152:0/1631172381 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:28.054 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.054+0000 7f26a7400640 1 --2- 172.21.15.152:0/1631172381 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26a80ba430 0x7f26a80ca580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:28.054 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.054+0000 7f26acc00640 1 --2- 172.21.15.152:0/1631172381 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f26a815e890 0x7f26a80caac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:28.054 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.054+0000 7f26a7e00640 1 -- 172.21.15.152:0/1631172381 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f26a815e890 msgr2=0x7f26a80caac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:28.054 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.054+0000 7f26a7e00640 1 --2- 172.21.15.152:0/1631172381 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f26a815e890 0x7f26a80caac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:28.054 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.054+0000 7f26a7e00640 1 -- 172.21.15.152:0/1631172381 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26a80ba430 msgr2=0x7f26a80ca580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:28.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.054+0000 7f26a7e00640 1 --2- 172.21.15.152:0/1631172381 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26a80ba430 0x7f26a80ca580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:28.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.054+0000 7f26a7e00640 1 -- 172.21.15.152:0/1631172381 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26a80c94e0 con 0x7f26a80b7cc0 2024-04-20T11:37:28.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.054+0000 7f26a7e00640 1 --2- 172.21.15.152:0/1631172381 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f26a80b7cc0 0x7f26a8146900 secure :-1 s=READY pgs=2213 cs=0 l=1 rev1=1 crypto rx=0x7f269800b420 tx=0x7f269800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:28.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.054+0000 7f269fe00640 1 -- 172.21.15.152:0/1631172381 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2698011830 con 0x7f26a80b7cc0 2024-04-20T11:37:28.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.055+0000 7f269fe00640 1 -- 172.21.15.152:0/1631172381 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2698016070 con 0x7f26a80b7cc0 2024-04-20T11:37:28.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.055+0000 7f269fe00640 1 -- 172.21.15.152:0/1631172381 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2698011c10 con 0x7f26a80b7cc0 2024-04-20T11:37:28.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.055+0000 7f26aec00640 1 -- 172.21.15.152:0/1631172381 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26a80c97c0 con 0x7f26a80b7cc0 2024-04-20T11:37:28.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.055+0000 7f26aec00640 1 -- 172.21.15.152:0/1631172381 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26a80c9d80 con 0x7f26a80b7cc0 2024-04-20T11:37:28.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.056+0000 7f26aec00640 1 -- 172.21.15.152:0/1631172381 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26a80689f0 con 0x7f26a80b7cc0 2024-04-20T11:37:28.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.057+0000 7f269fe00640 1 -- 172.21.15.152:0/1631172381 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 26) ==== 91907+0+0 (secure 0 0 0) 0x7f269801b460 con 0x7f26a80b7cc0 2024-04-20T11:37:28.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.058+0000 7f269fe00640 1 --2- 172.21.15.152:0/1631172381 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f268406d280 0x7f268406f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:28.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.058+0000 7f269fe00640 1 -- 172.21.15.152:0/1631172381 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5767+0+0 (secure 0 0 0) 0x7f2698089d90 con 0x7f26a80b7cc0 2024-04-20T11:37:28.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.059+0000 7f26a7400640 1 --2- 172.21.15.152:0/1631172381 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f268406d280 0x7f268406f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:28.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.060+0000 7f26a7400640 1 --2- 172.21.15.152:0/1631172381 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f268406d280 0x7f268406f740 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f2690036000 tx=0x7f2690030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:28.062 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.063+0000 7f269fe00640 1 -- 172.21.15.152:0/1631172381 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f2698020020 con 0x7f26a80b7cc0 2024-04-20T11:37:28.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.262+0000 7f26aec00640 1 -- 172.21.15.152:0/1631172381 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f26a806bbf0 con 0x7f26a80b7cc0 2024-04-20T11:37:28.262 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.263+0000 7f269fe00640 1 -- 172.21.15.152:0/1631172381 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v156) ==== 104+0+11 (secure 0 0 0) 0x7f2698059dc0 con 0x7f26a80b7cc0 2024-04-20T11:37:28.262 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 32 2024-04-20T11:37:28.265 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.265+0000 7f26aec00640 1 -- 172.21.15.152:0/1631172381 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f268406d280 msgr2=0x7f268406f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:28.266 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.265+0000 7f26aec00640 1 --2- 172.21.15.152:0/1631172381 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f268406d280 0x7f268406f740 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f2690036000 tx=0x7f2690030000 comp rx=0 tx=0).stop 2024-04-20T11:37:28.266 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.265+0000 7f26aec00640 1 -- 172.21.15.152:0/1631172381 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f26a80b7cc0 msgr2=0x7f26a8146900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:28.266 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.265+0000 7f26aec00640 1 --2- 172.21.15.152:0/1631172381 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f26a80b7cc0 0x7f26a8146900 secure :-1 s=READY pgs=2213 cs=0 l=1 rev1=1 crypto rx=0x7f269800b420 tx=0x7f269800b8e0 comp rx=0 tx=0).stop 2024-04-20T11:37:28.266 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.266+0000 7f26aec00640 1 -- 172.21.15.152:0/1631172381 shutdown_connections 2024-04-20T11:37:28.266 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.266+0000 7f26aec00640 1 --2- 172.21.15.152:0/1631172381 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f268406d280 0x7f268406f740 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:28.266 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.266+0000 7f26aec00640 1 --2- 172.21.15.152:0/1631172381 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f26a815e890 0x7f26a80caac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:28.266 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.266+0000 7f26aec00640 1 --2- 172.21.15.152:0/1631172381 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26a80ba430 0x7f26a80ca580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:28.266 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.266+0000 7f26aec00640 1 --2- 172.21.15.152:0/1631172381 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f26a80b7cc0 0x7f26a8146900 unknown :-1 s=CLOSED pgs=2213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:28.266 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.266+0000 7f26aec00640 1 -- 172.21.15.152:0/1631172381 >> 172.21.15.152:0/1631172381 conn(0x7f26a8067ae0 msgr2=0x7f26a80b55c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:28.266 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.266+0000 7f26aec00640 1 -- 172.21.15.152:0/1631172381 shutdown_connections 2024-04-20T11:37:28.266 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.266+0000 7f26aec00640 1 -- 172.21.15.152:0/1631172381 wait complete. 2024-04-20T11:37:28.284 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T11:37:28.488 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.488+0000 7fde83e00640 1 Processor -- start 2024-04-20T11:37:28.488 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.488+0000 7fde83e00640 1 -- start start 2024-04-20T11:37:28.489 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.489+0000 7fde83e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fde8415ca40 0x7fde8415cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:28.489 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.490+0000 7fde83e00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde8415d5c0 con 0x7fde84156c00 2024-04-20T11:37:28.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.490+0000 7fde83e00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde8415d8a0 con 0x7fde8406bda0 2024-04-20T11:37:28.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.490+0000 7fde83e00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fde84160290 con 0x7fde8415ca40 2024-04-20T11:37:28.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.490+0000 7fde82a00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fde84156c00 0x7fde84160a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:40780/0 (socket says 172.21.15.152:40780) 2024-04-20T11:37:28.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.490+0000 7fde82a00640 1 -- 172.21.15.152:0/1563532540 learned_addr learned my addr 172.21.15.152:0/1563532540 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:37:28.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.490+0000 7fde82000640 1 --2- 172.21.15.152:0/1563532540 >> v2:172.21.15.152:3300/0 conn(0x7fde8415ca40 0x7fde8415cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:28.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.490+0000 7fde81600640 1 -- 172.21.15.152:0/1563532540 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 362566931 0 0) 0x7fde8415d5c0 con 0x7fde84156c00 2024-04-20T11:37:28.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.491+0000 7fde81600640 1 -- 172.21.15.152:0/1563532540 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fde68003520 con 0x7fde84156c00 2024-04-20T11:37:28.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.491+0000 7fde82000640 1 -- 172.21.15.152:0/1563532540 >> v1:172.21.15.175:6789/0 conn(0x7fde8406bda0 legacy=0x7fde840b30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:28.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.491+0000 7fde82000640 1 -- 172.21.15.152:0/1563532540 >> v1:172.21.15.152:6789/0 conn(0x7fde84156c00 legacy=0x7fde84160a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:28.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.491+0000 7fde82000640 1 -- 172.21.15.152:0/1563532540 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde840ad2e0 con 0x7fde8415ca40 2024-04-20T11:37:28.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.492+0000 7fde82000640 1 --2- 172.21.15.152:0/1563532540 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fde8415ca40 0x7fde8415cec0 secure :-1 s=READY pgs=2215 cs=0 l=1 rev1=1 crypto rx=0x7fde6c0097a0 tx=0x7fde6c02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=510dcf7b7f2b85f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:28.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.492+0000 7fde81600640 1 -- 172.21.15.152:0/1563532540 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde6c034070 con 0x7fde8415ca40 2024-04-20T11:37:28.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.492+0000 7fde81600640 1 -- 172.21.15.152:0/1563532540 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde6c030610 con 0x7fde8415ca40 2024-04-20T11:37:28.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.492+0000 7fde81600640 1 -- 172.21.15.152:0/1563532540 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde6c0309e0 con 0x7fde8415ca40 2024-04-20T11:37:28.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.493+0000 7fde83e00640 1 -- 172.21.15.152:0/1563532540 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fde8415ca40 msgr2=0x7fde8415cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:28.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.493+0000 7fde83e00640 1 --2- 172.21.15.152:0/1563532540 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fde8415ca40 0x7fde8415cec0 secure :-1 s=READY pgs=2215 cs=0 l=1 rev1=1 crypto rx=0x7fde6c0097a0 tx=0x7fde6c02b960 comp rx=0 tx=0).stop 2024-04-20T11:37:28.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.493+0000 7fde83e00640 1 -- 172.21.15.152:0/1563532540 shutdown_connections 2024-04-20T11:37:28.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.493+0000 7fde83e00640 1 --2- 172.21.15.152:0/1563532540 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fde8415ca40 0x7fde8415cec0 unknown :-1 s=CLOSED pgs=2215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:28.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.493+0000 7fde83e00640 1 -- 172.21.15.152:0/1563532540 >> 172.21.15.152:0/1563532540 conn(0x7fde840b21e0 msgr2=0x7fde840b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:28.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.493+0000 7fde83e00640 1 -- 172.21.15.152:0/1563532540 shutdown_connections 2024-04-20T11:37:28.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.493+0000 7fde83e00640 1 -- 172.21.15.152:0/1563532540 wait complete. 2024-04-20T11:37:28.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.495+0000 7fde83e00640 1 Processor -- start 2024-04-20T11:37:28.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.495+0000 7fde83e00640 1 -- start start 2024-04-20T11:37:28.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.496+0000 7fde83e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fde8406bda0 0x7fde84145a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:28.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.496+0000 7fde83e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fde84156c00 0x7fde8415e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:28.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.496+0000 7fde83e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde8415ca40 0x7fde8415e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:28.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.496+0000 7fde83e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fde840b3b60 con 0x7fde84156c00 2024-04-20T11:37:28.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.496+0000 7fde83e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fde840b39e0 con 0x7fde8415ca40 2024-04-20T11:37:28.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.496+0000 7fde83e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fde840b3ce0 con 0x7fde8406bda0 2024-04-20T11:37:28.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.496+0000 7fde82000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fde84156c00 0x7fde8415e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:28.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.496+0000 7fde82000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fde84156c00 0x7fde8415e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36312/0 (socket says 172.21.15.152:36312) 2024-04-20T11:37:28.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.496+0000 7fde82000640 1 -- 172.21.15.152:0/724690120 learned_addr learned my addr 172.21.15.152:0/724690120 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:28.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.496+0000 7fde82000640 1 -- 172.21.15.152:0/724690120 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fde8406bda0 msgr2=0x7fde84145a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:28.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.496+0000 7fde82000640 1 --2- 172.21.15.152:0/724690120 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fde8406bda0 0x7fde84145a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:28.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.496+0000 7fde82000640 1 -- 172.21.15.152:0/724690120 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde8415ca40 msgr2=0x7fde8415e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:28.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.496+0000 7fde82000640 1 --2- 172.21.15.152:0/724690120 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde8415ca40 0x7fde8415e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:28.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.496+0000 7fde82000640 1 -- 172.21.15.152:0/724690120 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde8414d240 con 0x7fde84156c00 2024-04-20T11:37:28.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.497+0000 7fde82000640 1 --2- 172.21.15.152:0/724690120 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fde84156c00 0x7fde8415e370 secure :-1 s=READY pgs=2216 cs=0 l=1 rev1=1 crypto rx=0x7fde6c0098c0 tx=0x7fde6c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:28.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.497+0000 7fde7aa00640 1 -- 172.21.15.152:0/724690120 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde6c034040 con 0x7fde84156c00 2024-04-20T11:37:28.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.497+0000 7fde7aa00640 1 -- 172.21.15.152:0/724690120 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde6c03aaa0 con 0x7fde84156c00 2024-04-20T11:37:28.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.497+0000 7fde83e00640 1 -- 172.21.15.152:0/724690120 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde8414ea10 con 0x7fde84156c00 2024-04-20T11:37:28.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.497+0000 7fde7aa00640 1 -- 172.21.15.152:0/724690120 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde6c0443f0 con 0x7fde84156c00 2024-04-20T11:37:28.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.497+0000 7fde83e00640 1 -- 172.21.15.152:0/724690120 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde84157510 con 0x7fde84156c00 2024-04-20T11:37:28.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.499+0000 7fde83e00640 1 -- 172.21.15.152:0/724690120 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde84075e60 con 0x7fde84156c00 2024-04-20T11:37:28.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.500+0000 7fde7aa00640 1 -- 172.21.15.152:0/724690120 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 26) ==== 91907+0+0 (secure 0 0 0) 0x7fde6c04b060 con 0x7fde84156c00 2024-04-20T11:37:28.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.500+0000 7fde7aa00640 1 --2- 172.21.15.152:0/724690120 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fde6006d2b0 0x7fde6006f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:28.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.501+0000 7fde7aa00640 1 -- 172.21.15.152:0/724690120 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5767+0+0 (secure 0 0 0) 0x7fde6c0ac090 con 0x7fde84156c00 2024-04-20T11:37:28.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.501+0000 7fde82a00640 1 --2- 172.21.15.152:0/724690120 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fde6006d2b0 0x7fde6006f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:28.502 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.502+0000 7fde82a00640 1 --2- 172.21.15.152:0/724690120 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fde6006d2b0 0x7fde6006f770 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fde8415d070 tx=0x7fde74001690 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:28.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.505+0000 7fde7aa00640 1 -- 172.21.15.152:0/724690120 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fde8414d240 con 0x7fde84156c00 2024-04-20T11:37:28.704 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.704+0000 7fde83e00640 1 -- 172.21.15.152:0/724690120 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fde8415ad10 con 0x7fde84156c00 2024-04-20T11:37:28.705 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.705+0000 7fde7aa00640 1 -- 172.21.15.152:0/724690120 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v156) ==== 100+0+11 (secure 0 0 0) 0x7fde6c03f070 con 0x7fde84156c00 2024-04-20T11:37:28.705 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 64 2024-04-20T11:37:28.707 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.708+0000 7fde83e00640 1 -- 172.21.15.152:0/724690120 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fde6006d2b0 msgr2=0x7fde6006f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:28.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.708+0000 7fde83e00640 1 --2- 172.21.15.152:0/724690120 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fde6006d2b0 0x7fde6006f770 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fde8415d070 tx=0x7fde74001690 comp rx=0 tx=0).stop 2024-04-20T11:37:28.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.708+0000 7fde83e00640 1 -- 172.21.15.152:0/724690120 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fde84156c00 msgr2=0x7fde8415e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:28.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.708+0000 7fde83e00640 1 --2- 172.21.15.152:0/724690120 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fde84156c00 0x7fde8415e370 secure :-1 s=READY pgs=2216 cs=0 l=1 rev1=1 crypto rx=0x7fde6c0098c0 tx=0x7fde6c0046c0 comp rx=0 tx=0).stop 2024-04-20T11:37:28.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.708+0000 7fde83e00640 1 -- 172.21.15.152:0/724690120 shutdown_connections 2024-04-20T11:37:28.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.708+0000 7fde83e00640 1 --2- 172.21.15.152:0/724690120 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fde6006d2b0 0x7fde6006f770 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:28.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.708+0000 7fde83e00640 1 --2- 172.21.15.152:0/724690120 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde8415ca40 0x7fde8415e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:28.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.708+0000 7fde83e00640 1 --2- 172.21.15.152:0/724690120 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fde84156c00 0x7fde8415e370 unknown :-1 s=CLOSED pgs=2216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:28.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.708+0000 7fde83e00640 1 --2- 172.21.15.152:0/724690120 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fde8406bda0 0x7fde84145a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:28.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.708+0000 7fde83e00640 1 -- 172.21.15.152:0/724690120 >> 172.21.15.152:0/724690120 conn(0x7fde840b21e0 msgr2=0x7fde8415f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:28.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.708+0000 7fde83e00640 1 -- 172.21.15.152:0/724690120 shutdown_connections 2024-04-20T11:37:28.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.708+0000 7fde83e00640 1 -- 172.21.15.152:0/724690120 wait complete. 2024-04-20T11:37:28.728 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-20T11:37:28.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.933+0000 7f049e400640 1 Processor -- start 2024-04-20T11:37:28.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.934+0000 7f049e400640 1 -- start start 2024-04-20T11:37:28.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.935+0000 7f049e400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f0498145f00 0x7f04980cd0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:28.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.935+0000 7f049e400640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f049806ead0 0x7f04980cfe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:28.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.935+0000 7f049e400640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f049806f110 con 0x7f04980cd610 2024-04-20T11:37:28.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.935+0000 7f049e400640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f04980cda80 con 0x7f0498145f00 2024-04-20T11:37:28.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.935+0000 7f049e400640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f04980cf9f0 con 0x7f049806ead0 2024-04-20T11:37:28.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.936+0000 7f0497400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f0498145f00 0x7f04980cd0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:28.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.936+0000 7f0497400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f0498145f00 0x7f04980cd0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36320/0 (socket says 172.21.15.152:36320) 2024-04-20T11:37:28.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.936+0000 7f0497400640 1 -- 172.21.15.152:0/2094549530 learned_addr learned my addr 172.21.15.152:0/2094549530 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:28.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.936+0000 7f048fe00640 1 --1- 172.21.15.152:0/2094549530 >> v1:172.21.15.152:6789/0 conn(0x7f04980cd610 0x7f049806e3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:40782/0 (socket says 172.21.15.152:40782) 2024-04-20T11:37:28.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.936+0000 7f0497e00640 1 --2- 172.21.15.152:0/2094549530 >> v2:172.21.15.175:3300/0 conn(0x7f049806ead0 0x7f04980cfe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:28.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.936+0000 7f0497400640 1 -- 172.21.15.152:0/2094549530 >> v2:172.21.15.175:3300/0 conn(0x7f049806ead0 msgr2=0x7f04980cfe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:28.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.936+0000 7f0497400640 1 --2- 172.21.15.152:0/2094549530 >> v2:172.21.15.175:3300/0 conn(0x7f049806ead0 0x7f04980cfe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:28.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.936+0000 7f0497400640 1 -- 172.21.15.152:0/2094549530 >> v1:172.21.15.152:6789/0 conn(0x7f04980cd610 legacy=0x7f049806e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:28.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.936+0000 7f0497400640 1 -- 172.21.15.152:0/2094549530 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f049806f3f0 con 0x7f0498145f00 2024-04-20T11:37:28.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.937+0000 7f0497400640 1 --2- 172.21.15.152:0/2094549530 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0498145f00 0x7f04980cd0d0 secure :-1 s=READY pgs=2218 cs=0 l=1 rev1=1 crypto rx=0x7f049806c7b0 tx=0x7f048802b9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5996ffb82abdf42 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:28.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.938+0000 7f0496a00640 1 -- 172.21.15.152:0/2094549530 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0488034070 con 0x7f0498145f00 2024-04-20T11:37:28.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.938+0000 7f0496a00640 1 -- 172.21.15.152:0/2094549530 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0488030870 con 0x7f0498145f00 2024-04-20T11:37:28.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.938+0000 7f0496a00640 1 -- 172.21.15.152:0/2094549530 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0488030c40 con 0x7f0498145f00 2024-04-20T11:37:28.938 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.938+0000 7f049e400640 1 -- 172.21.15.152:0/2094549530 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0498145f00 msgr2=0x7f04980cd0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:28.938 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.938+0000 7f049e400640 1 --2- 172.21.15.152:0/2094549530 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0498145f00 0x7f04980cd0d0 secure :-1 s=READY pgs=2218 cs=0 l=1 rev1=1 crypto rx=0x7f049806c7b0 tx=0x7f048802b9b0 comp rx=0 tx=0).stop 2024-04-20T11:37:28.938 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.938+0000 7f049e400640 1 -- 172.21.15.152:0/2094549530 shutdown_connections 2024-04-20T11:37:28.938 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.938+0000 7f049e400640 1 --2- 172.21.15.152:0/2094549530 >> v2:172.21.15.175:3300/0 conn(0x7f049806ead0 0x7f04980cfe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:28.938 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.938+0000 7f049e400640 1 --2- 172.21.15.152:0/2094549530 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0498145f00 0x7f04980cd0d0 unknown :-1 s=CLOSED pgs=2218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:28.939 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.938+0000 7f049e400640 1 -- 172.21.15.152:0/2094549530 >> 172.21.15.152:0/2094549530 conn(0x7f0498067cf0 msgr2=0x7f049806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:28.939 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.939+0000 7f049e400640 1 -- 172.21.15.152:0/2094549530 shutdown_connections 2024-04-20T11:37:28.939 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.939+0000 7f049e400640 1 -- 172.21.15.152:0/2094549530 wait complete. 2024-04-20T11:37:28.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.940+0000 7f049e400640 1 Processor -- start 2024-04-20T11:37:28.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.940+0000 7f049e400640 1 -- start start 2024-04-20T11:37:28.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.941+0000 7f049e400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f049806ead0 0x7f049815b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:28.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.941+0000 7f049e400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f04980cd610 0x7f04981f4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:28.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.941+0000 7f049e400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0498145f00 0x7f04980c9b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:28.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.941+0000 7f049e400640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f04981462e0 con 0x7f0498145f00 2024-04-20T11:37:28.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.941+0000 7f049e400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f04980cf680 con 0x7f04980cd610 2024-04-20T11:37:28.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.941+0000 7f049e400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f04980d03c0 con 0x7f049806ead0 2024-04-20T11:37:28.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.942+0000 7f0497e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0498145f00 0x7f04980c9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:28.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.942+0000 7f0497e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0498145f00 0x7f04980c9b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36324/0 (socket says 172.21.15.152:36324) 2024-04-20T11:37:28.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.942+0000 7f0497e00640 1 -- 172.21.15.152:0/1960764663 learned_addr learned my addr 172.21.15.152:0/1960764663 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:28.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.942+0000 7f048fe00640 1 --2- 172.21.15.152:0/1960764663 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f04980cd610 0x7f04981f4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:28.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.942+0000 7f0497400640 1 --2- 172.21.15.152:0/1960764663 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f049806ead0 0x7f049815b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:28.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.942+0000 7f0497e00640 1 -- 172.21.15.152:0/1960764663 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f049806ead0 msgr2=0x7f049815b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:28.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.942+0000 7f0497e00640 1 --2- 172.21.15.152:0/1960764663 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f049806ead0 0x7f049815b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:28.944 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.942+0000 7f0497e00640 1 -- 172.21.15.152:0/1960764663 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f04980cd610 msgr2=0x7f04981f4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:28.944 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.942+0000 7f0497e00640 1 --2- 172.21.15.152:0/1960764663 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f04980cd610 0x7f04981f4a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:28.944 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.942+0000 7f0497e00640 1 -- 172.21.15.152:0/1960764663 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04981432c0 con 0x7f0498145f00 2024-04-20T11:37:28.944 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.942+0000 7f0497400640 1 --2- 172.21.15.152:0/1960764663 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f049806ead0 0x7f049815b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:37:28.944 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.943+0000 7f0497e00640 1 --2- 172.21.15.152:0/1960764663 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0498145f00 0x7f04980c9b20 secure :-1 s=READY pgs=2219 cs=0 l=1 rev1=1 crypto rx=0x7f049000cd70 tx=0x7f049000a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:28.944 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.943+0000 7f048f400640 1 -- 172.21.15.152:0/1960764663 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0490003b90 con 0x7f0498145f00 2024-04-20T11:37:28.944 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.943+0000 7f048f400640 1 -- 172.21.15.152:0/1960764663 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0490003d30 con 0x7f0498145f00 2024-04-20T11:37:28.944 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.943+0000 7f048f400640 1 -- 172.21.15.152:0/1960764663 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0490004240 con 0x7f0498145f00 2024-04-20T11:37:28.944 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.943+0000 7f049e400640 1 -- 172.21.15.152:0/1960764663 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04981435a0 con 0x7f0498145f00 2024-04-20T11:37:28.944 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.943+0000 7f049e400640 1 -- 172.21.15.152:0/1960764663 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0498143b60 con 0x7f0498145f00 2024-04-20T11:37:28.945 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.944+0000 7f049e400640 1 -- 172.21.15.152:0/1960764663 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04980689f0 con 0x7f0498145f00 2024-04-20T11:37:28.945 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.945+0000 7f048f400640 1 -- 172.21.15.152:0/1960764663 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 26) ==== 91907+0+0 (secure 0 0 0) 0x7f04900043e0 con 0x7f0498145f00 2024-04-20T11:37:28.945 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.946+0000 7f048f400640 1 --2- 172.21.15.152:0/1960764663 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f047406d2b0 0x7f047406f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:28.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.946+0000 7f048f400640 1 -- 172.21.15.152:0/1960764663 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5767+0+0 (secure 0 0 0) 0x7f049008adf0 con 0x7f0498145f00 2024-04-20T11:37:28.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.947+0000 7f0497400640 1 --2- 172.21.15.152:0/1960764663 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f047406d2b0 0x7f047406f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:28.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.948+0000 7f0497400640 1 --2- 172.21.15.152:0/1960764663 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f047406d2b0 0x7f047406f770 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f0488036000 tx=0x7f048803b040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:28.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:28.950+0000 7f048f400640 1 -- 172.21.15.152:0/1960764663 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f049005ae20 con 0x7f0498145f00 2024-04-20T11:37:29.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:29.164+0000 7f049e400640 1 -- 172.21.15.152:0/1960764663 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f0498143d30 con 0x7f0498145f00 2024-04-20T11:37:29.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:29.165+0000 7f048f400640 1 -- 172.21.15.152:0/1960764663 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v156) ==== 80+0+0 (secure 0 0 0) 0x7f049005a720 con 0x7f0498145f00 2024-04-20T11:37:29.165 INFO:teuthology.orchestra.run.smithi152.stderr:listed 0 entries 2024-04-20T11:37:29.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:29.167+0000 7f049e400640 1 -- 172.21.15.152:0/1960764663 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f047406d2b0 msgr2=0x7f047406f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:29.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:29.167+0000 7f049e400640 1 --2- 172.21.15.152:0/1960764663 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f047406d2b0 0x7f047406f770 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f0488036000 tx=0x7f048803b040 comp rx=0 tx=0).stop 2024-04-20T11:37:29.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:29.168+0000 7f049e400640 1 -- 172.21.15.152:0/1960764663 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0498145f00 msgr2=0x7f04980c9b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:29.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:29.168+0000 7f049e400640 1 --2- 172.21.15.152:0/1960764663 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0498145f00 0x7f04980c9b20 secure :-1 s=READY pgs=2219 cs=0 l=1 rev1=1 crypto rx=0x7f049000cd70 tx=0x7f049000a660 comp rx=0 tx=0).stop 2024-04-20T11:37:29.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:29.168+0000 7f049e400640 1 -- 172.21.15.152:0/1960764663 shutdown_connections 2024-04-20T11:37:29.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:29.168+0000 7f049e400640 1 --2- 172.21.15.152:0/1960764663 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f047406d2b0 0x7f047406f770 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:29.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:29.168+0000 7f049e400640 1 --2- 172.21.15.152:0/1960764663 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0498145f00 0x7f04980c9b20 unknown :-1 s=CLOSED pgs=2219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:29.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:29.168+0000 7f049e400640 1 --2- 172.21.15.152:0/1960764663 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f04980cd610 0x7f04981f4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:29.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:29.168+0000 7f049e400640 1 --2- 172.21.15.152:0/1960764663 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f049806ead0 0x7f049815b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:29.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:29.168+0000 7f049e400640 1 -- 172.21.15.152:0/1960764663 >> 172.21.15.152:0/1960764663 conn(0x7f0498067cf0 msgr2=0x7f049815ccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:29.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:29.168+0000 7f049e400640 1 -- 172.21.15.152:0/1960764663 shutdown_connections 2024-04-20T11:37:29.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:29.168+0000 7f049e400640 1 -- 172.21.15.152:0/1960764663 wait complete. 2024-04-20T11:37:29.186 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi175.front.sepia.ceph.com... 2024-04-20T11:37:29.187 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:37:29.187 DEBUG:teuthology.orchestra.run.smithi175:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-04-20T11:37:29.271 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi175.stderr:ceph-fuse[36532]: fuse finished with error 0 and tester_r 0 2024-04-20T11:37:29.277 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi175.front.sepia.ceph.com 2024-04-20T11:37:29.278 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:37:29.278 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-20T11:37:29.447 INFO:tasks.cephfs.mount:Terminating background process 2024-04-20T11:37:29.447 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-20T11:37:29.447 DEBUG:teuthology.orchestra.run.smithi152:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:37:29.508 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-20T11:37:29.508 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi152.front.sepia.ceph.com 2024-04-20T11:37:29.508 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:37:29.508 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-20T11:37:29.558 INFO:teuthology.orchestra.run.smithi152.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-20T11:37:29.559 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T11:37:29.559 INFO:tasks.cephfs.mount:Terminating background process 2024-04-20T11:37:29.560 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T11:37:29.560 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:37:29.616 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-20T11:37:29.616 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi175.front.sepia.ceph.com 2024-04-20T11:37:29.617 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:37:29.617 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-20T11:37:29.669 INFO:teuthology.orchestra.run.smithi175.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-20T11:37:29.670 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T11:37:29.671 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:37:29.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:29.884+0000 7fb0bdc00640 1 Processor -- start 2024-04-20T11:37:29.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:29.885+0000 7fb0bdc00640 1 -- start start 2024-04-20T11:37:29.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:29.887+0000 7fb0bdc00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb0b806b980 0x7fb0b80b4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:29.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:29.887+0000 7fb0bdc00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0b806f5d0 con 0x7fb0b806ee10 2024-04-20T11:37:29.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:29.887+0000 7fb0bdc00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0b806f8b0 con 0x7fb0b81601b0 2024-04-20T11:37:29.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:29.887+0000 7fb0bdc00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb0b806fb90 con 0x7fb0b806b980 2024-04-20T11:37:29.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:29.887+0000 7fb0b7400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fb0b806ee10 0x7fb0b806bd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:40790/0 (socket says 172.21.15.152:40790) 2024-04-20T11:37:29.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:29.887+0000 7fb0b7400640 1 -- 172.21.15.152:0/1313789387 learned_addr learned my addr 172.21.15.152:0/1313789387 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:37:29.888 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:29.887+0000 7fb0b7e00640 1 --2- 172.21.15.152:0/1313789387 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb0b806b980 0x7fb0b80b4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:29.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:29.887+0000 7fb0b6a00640 1 -- 172.21.15.152:0/1313789387 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3624036579 0 0) 0x7fb0b806f5d0 con 0x7fb0b806ee10 2024-04-20T11:37:29.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:29.887+0000 7fb0b6a00640 1 -- 172.21.15.152:0/1313789387 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb09c003520 con 0x7fb0b806ee10 2024-04-20T11:37:29.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:29.888+0000 7fb0b6a00640 1 -- 172.21.15.152:0/1313789387 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4172178682 0 0) 0x7fb09c003520 con 0x7fb0b806ee10 2024-04-20T11:37:29.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:29.888+0000 7fb0b6a00640 1 -- 172.21.15.152:0/1313789387 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb0b806b980 msgr2=0x7fb0b80b4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:29.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:29.888+0000 7fb0b6a00640 1 --2- 172.21.15.152:0/1313789387 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb0b806b980 0x7fb0b80b4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:29.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:29.888+0000 7fb0b6a00640 1 -- 172.21.15.152:0/1313789387 >> v1:172.21.15.175:6789/0 conn(0x7fb0b81601b0 legacy=0x7fb0b8160640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:29.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:29.888+0000 7fb0b6a00640 1 -- 172.21.15.152:0/1313789387 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0b806fe70 con 0x7fb0b806ee10 2024-04-20T11:37:29.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:29.888+0000 7fb0b6a00640 1 -- 172.21.15.152:0/1313789387 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7fb0a4001670 con 0x7fb0b806ee10 2024-04-20T11:37:29.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:29.888+0000 7fb0b6a00640 1 -- 172.21.15.152:0/1313789387 >> v1:172.21.15.152:6789/0 conn(0x7fb0b806ee10 legacy=0x7fb0b806bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:29.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:29.888+0000 7fb0b6a00640 1 --2- 172.21.15.152:0/1313789387 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb09c0039e0 0x7fb09c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:29.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:29.888+0000 7fb0b6a00640 1 --2- 172.21.15.152:0/1313789387 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb0b806b980 0x7fb09c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:29.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:29.888+0000 7fb0b6a00640 1 --2- 172.21.15.152:0/1313789387 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb09c004d40 0x7fb09c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:29.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:29.888+0000 7fb0b6a00640 1 -- 172.21.15.152:0/1313789387 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b806fb90 con 0x7fb09c0039e0 2024-04-20T11:37:29.890 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:29.888+0000 7fb0b6a00640 1 -- 172.21.15.152:0/1313789387 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb09c00bad0 con 0x7fb09c004d40 2024-04-20T11:37:29.890 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:29.888+0000 7fb0b6a00640 1 -- 172.21.15.152:0/1313789387 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb09c00bcd0 con 0x7fb0b806b980 2024-04-20T11:37:29.890 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:29.888+0000 7fb0b7e00640 1 --2- 172.21.15.152:0/1313789387 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb09c0039e0 0x7fb09c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:29.890 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:29.889+0000 7fb0b7400640 1 --2- 172.21.15.152:0/1313789387 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb0b806b980 0x7fb09c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:29.890 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:29.889+0000 7fb0b7e00640 1 -- 172.21.15.152:0/1313789387 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb0b806b980 msgr2=0x7fb09c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:29.890 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:29.889+0000 7fb0b7e00640 1 --2- 172.21.15.152:0/1313789387 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb0b806b980 0x7fb09c004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:29.890 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:29.889+0000 7fb0b7e00640 1 -- 172.21.15.152:0/1313789387 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb09c004d40 msgr2=0x7fb09c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:29.890 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:29.889+0000 7fb0b7e00640 1 --2- 172.21.15.152:0/1313789387 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb09c004d40 0x7fb09c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:29.890 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:29.889+0000 7fb0b7e00640 1 -- 172.21.15.152:0/1313789387 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb09c003520 con 0x7fb09c0039e0 2024-04-20T11:37:29.891 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:29.890+0000 7fb0b7e00640 1 --2- 172.21.15.152:0/1313789387 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb09c0039e0 0x7fb09c003e00 secure :-1 s=READY pgs=2221 cs=0 l=1 rev1=1 crypto rx=0x7fb0a8009a90 tx=0x7fb09c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c62f9c35a6da8f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:29.891 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:29.890+0000 7fb0b6a00640 1 -- 172.21.15.152:0/1313789387 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a8039070 con 0x7fb09c0039e0 2024-04-20T11:37:29.891 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:29.890+0000 7fb0b6a00640 1 -- 172.21.15.152:0/1313789387 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0a802c680 con 0x7fb09c0039e0 2024-04-20T11:37:29.891 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:29.890+0000 7fb0b6a00640 1 -- 172.21.15.152:0/1313789387 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a8009690 con 0x7fb09c0039e0 2024-04-20T11:37:29.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:29.891+0000 7fb0bdc00640 1 -- 172.21.15.152:0/1313789387 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb09c0039e0 msgr2=0x7fb09c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:29.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:29.891+0000 7fb0bdc00640 1 --2- 172.21.15.152:0/1313789387 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb09c0039e0 0x7fb09c003e00 secure :-1 s=READY pgs=2221 cs=0 l=1 rev1=1 crypto rx=0x7fb0a8009a90 tx=0x7fb09c005830 comp rx=0 tx=0).stop 2024-04-20T11:37:29.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:29.891+0000 7fb0bd200640 1 -- 172.21.15.152:0/1313789387 reap_dead start 2024-04-20T11:37:29.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:29.891+0000 7fb0bdc00640 1 -- 172.21.15.152:0/1313789387 shutdown_connections 2024-04-20T11:37:29.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:29.891+0000 7fb0bdc00640 1 -- 172.21.15.152:0/1313789387 >> 172.21.15.152:0/1313789387 conn(0x7fb0b80732b0 msgr2=0x7fb0b809aa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:29.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:29.891+0000 7fb0bdc00640 1 -- 172.21.15.152:0/1313789387 shutdown_connections 2024-04-20T11:37:29.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:29.891+0000 7fb0bdc00640 1 -- 172.21.15.152:0/1313789387 wait complete. 2024-04-20T11:37:29.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:29.893+0000 7fb0bdc00640 1 Processor -- start 2024-04-20T11:37:29.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:29.893+0000 7fb0bdc00640 1 -- start start 2024-04-20T11:37:29.894 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:29.894+0000 7fb0bdc00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb09c0039e0 0x7fb0b8142880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:29.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:29.894+0000 7fb0bdc00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb0b81450f0 0x7fb0b8143f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:29.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:29.894+0000 7fb0bdc00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb0b815d580 0x7fb0b815d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:29.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:29.894+0000 7fb0bdc00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b8162000 con 0x7fb0b815d580 2024-04-20T11:37:29.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:29.894+0000 7fb0bdc00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b806f250 con 0x7fb09c0039e0 2024-04-20T11:37:29.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:29.894+0000 7fb0bdc00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb0b8162180 con 0x7fb0b81450f0 2024-04-20T11:37:29.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:29.894+0000 7fb0bd200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb0b815d580 0x7fb0b815d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:29.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:29.895+0000 7fb0bd200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb0b815d580 0x7fb0b815d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36342/0 (socket says 172.21.15.152:36342) 2024-04-20T11:37:29.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:29.895+0000 7fb0bd200640 1 -- 172.21.15.152:0/1560083235 learned_addr learned my addr 172.21.15.152:0/1560083235 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:29.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:29.895+0000 7fb0b7400640 1 --2- 172.21.15.152:0/1560083235 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb0b81450f0 0x7fb0b8143f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:29.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:29.895+0000 7fb0b7e00640 1 --2- 172.21.15.152:0/1560083235 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb09c0039e0 0x7fb0b8142880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:29.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:29.895+0000 7fb0bd200640 1 -- 172.21.15.152:0/1560083235 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb0b81450f0 msgr2=0x7fb0b8143f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:29.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:29.895+0000 7fb0bd200640 1 --2- 172.21.15.152:0/1560083235 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb0b81450f0 0x7fb0b8143f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:29.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:29.895+0000 7fb0bd200640 1 -- 172.21.15.152:0/1560083235 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb09c0039e0 msgr2=0x7fb0b8142880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:29.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:29.895+0000 7fb0bd200640 1 --2- 172.21.15.152:0/1560083235 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb09c0039e0 0x7fb0b8142880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:29.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:29.895+0000 7fb0bd200640 1 -- 172.21.15.152:0/1560083235 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0b815df20 con 0x7fb0b815d580 2024-04-20T11:37:29.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:29.895+0000 7fb0bd200640 1 --2- 172.21.15.152:0/1560083235 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb0b815d580 0x7fb0b815d9e0 secure :-1 s=READY pgs=2222 cs=0 l=1 rev1=1 crypto rx=0x7fb0b8160590 tx=0x7fb0b000cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:29.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:29.896+0000 7fb0afe00640 1 -- 172.21.15.152:0/1560083235 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b0017070 con 0x7fb0b815d580 2024-04-20T11:37:29.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:29.896+0000 7fb0afe00640 1 -- 172.21.15.152:0/1560083235 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0b0005d60 con 0x7fb0b815d580 2024-04-20T11:37:29.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:29.896+0000 7fb0afe00640 1 -- 172.21.15.152:0/1560083235 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b000a850 con 0x7fb0b815d580 2024-04-20T11:37:29.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:29.896+0000 7fb0bdc00640 1 -- 172.21.15.152:0/1560083235 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0b8142ef0 con 0x7fb0b815d580 2024-04-20T11:37:29.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:29.896+0000 7fb0bdc00640 1 -- 172.21.15.152:0/1560083235 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0b81434b0 con 0x7fb0b815d580 2024-04-20T11:37:29.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:29.897+0000 7fb0bdc00640 1 -- 172.21.15.152:0/1560083235 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0b8075e60 con 0x7fb0b815d580 2024-04-20T11:37:29.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:29.898+0000 7fb0afe00640 1 -- 172.21.15.152:0/1560083235 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 26) ==== 91907+0+0 (secure 0 0 0) 0x7fb0b000a9f0 con 0x7fb0b815d580 2024-04-20T11:37:29.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:29.899+0000 7fb0afe00640 1 --2- 172.21.15.152:0/1560083235 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb09806d2b0 0x7fb09806f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:29.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:29.899+0000 7fb0afe00640 1 -- 172.21.15.152:0/1560083235 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5767+0+0 (secure 0 0 0) 0x7fb0b0089400 con 0x7fb0b815d580 2024-04-20T11:37:29.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:29.900+0000 7fb0b7e00640 1 --2- 172.21.15.152:0/1560083235 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb09806d2b0 0x7fb09806f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:29.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:29.901+0000 7fb0b7e00640 1 --2- 172.21.15.152:0/1560083235 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb09806d2b0 0x7fb09806f770 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fb0a8036000 tx=0x7fb0a8030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:29.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:29.904+0000 7fb0afe00640 1 -- 172.21.15.152:0/1560083235 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fb0b0059430 con 0x7fb0b815d580 2024-04-20T11:37:30.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.144+0000 7fb0bdc00640 1 -- 172.21.15.152:0/1560083235 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb0b806dd50 con 0x7fb0b815d580 2024-04-20T11:37:30.145 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.145+0000 7fb0afe00640 1 -- 172.21.15.152:0/1560083235 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 106 v106) ==== 95+0+5220 (secure 0 0 0) 0x7fb0b0058d30 con 0x7fb0b815d580 2024-04-20T11:37:30.145 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:37:30.145 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":106,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6458,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/1688411915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":1688411915},{"type":"v1","addr":"172.21.15.152:6835","nonce":1688411915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":100},{"gid":7141,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6834/599467453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":599467453},{"type":"v1","addr":"172.21.15.175:6834","nonce":599467453}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":100},{"gid":7144,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/691089259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6833","nonce":691089259},{"type":"v1","addr":"172.21.15.175:6835","nonce":691089259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":100}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T11:36:59.789831+0000","modified":"2024-04-20T11:37:02.829697+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10011},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10011":{"gid":10011,"name":"c","rank":0,"incarnation":101,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6833/1854325981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":1854325981},{"type":"v1","addr":"172.21.15.152:6833","nonce":1854325981}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10011,"qdb_cluster":[10011]},"id":12}]} 2024-04-20T11:37:30.145 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 106 2024-04-20T11:37:30.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.148+0000 7fb0bdc00640 1 -- 172.21.15.152:0/1560083235 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb09806d2b0 msgr2=0x7fb09806f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:30.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.148+0000 7fb0bdc00640 1 --2- 172.21.15.152:0/1560083235 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb09806d2b0 0x7fb09806f770 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fb0a8036000 tx=0x7fb0a8030000 comp rx=0 tx=0).stop 2024-04-20T11:37:30.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.148+0000 7fb0bdc00640 1 -- 172.21.15.152:0/1560083235 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb0b815d580 msgr2=0x7fb0b815d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:30.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.148+0000 7fb0bdc00640 1 --2- 172.21.15.152:0/1560083235 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb0b815d580 0x7fb0b815d9e0 secure :-1 s=READY pgs=2222 cs=0 l=1 rev1=1 crypto rx=0x7fb0b8160590 tx=0x7fb0b000cb90 comp rx=0 tx=0).stop 2024-04-20T11:37:30.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.148+0000 7fb0bdc00640 1 -- 172.21.15.152:0/1560083235 shutdown_connections 2024-04-20T11:37:30.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.148+0000 7fb0bdc00640 1 --2- 172.21.15.152:0/1560083235 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb09806d2b0 0x7fb09806f770 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:30.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.148+0000 7fb0bdc00640 1 --2- 172.21.15.152:0/1560083235 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb0b815d580 0x7fb0b815d9e0 unknown :-1 s=CLOSED pgs=2222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:30.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.148+0000 7fb0bdc00640 1 --2- 172.21.15.152:0/1560083235 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb0b81450f0 0x7fb0b8143f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:30.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.148+0000 7fb0bdc00640 1 --2- 172.21.15.152:0/1560083235 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb09c0039e0 0x7fb0b8142880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:30.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.148+0000 7fb0bdc00640 1 -- 172.21.15.152:0/1560083235 >> 172.21.15.152:0/1560083235 conn(0x7fb0b80732b0 msgr2=0x7fb0b80b0340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:30.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.148+0000 7fb0bdc00640 1 -- 172.21.15.152:0/1560083235 shutdown_connections 2024-04-20T11:37:30.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.148+0000 7fb0bdc00640 1 -- 172.21.15.152:0/1560083235 wait complete. 2024-04-20T11:37:30.167 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:37:30.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.362+0000 7f814dc00640 1 Processor -- start 2024-04-20T11:37:30.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.362+0000 7f814dc00640 1 -- start start 2024-04-20T11:37:30.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.364+0000 7f814dc00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f8148147410 0x7f81480afa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:30.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.364+0000 7f814dc00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f814806e720 0x7f8148072130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:30.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.364+0000 7f814dc00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81480aec90 con 0x7f8148071cd0 2024-04-20T11:37:30.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.364+0000 7f814dc00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f8148072670 con 0x7f8148147410 2024-04-20T11:37:30.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.364+0000 7f814dc00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f81480aef70 con 0x7f814806e720 2024-04-20T11:37:30.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.364+0000 7f8147e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f8148147410 0x7f81480afa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:30.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.364+0000 7f8147e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f8148147410 0x7f81480afa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36358/0 (socket says 172.21.15.152:36358) 2024-04-20T11:37:30.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.364+0000 7f8147e00640 1 -- 172.21.15.152:0/1431805455 learned_addr learned my addr 172.21.15.152:0/1431805455 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:30.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.365+0000 7f814d200640 1 --2- 172.21.15.152:0/1431805455 >> v2:172.21.15.175:3300/0 conn(0x7f814806e720 0x7f8148072130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:30.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.365+0000 7f8147e00640 1 -- 172.21.15.152:0/1431805455 >> v2:172.21.15.175:3300/0 conn(0x7f814806e720 msgr2=0x7f8148072130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:30.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.365+0000 7f8147e00640 1 --2- 172.21.15.152:0/1431805455 >> v2:172.21.15.175:3300/0 conn(0x7f814806e720 0x7f8148072130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:30.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.365+0000 7f8147e00640 1 -- 172.21.15.152:0/1431805455 >> v1:172.21.15.152:6789/0 conn(0x7f8148071cd0 legacy=0x7f814806e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:30.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.365+0000 7f8147e00640 1 -- 172.21.15.152:0/1431805455 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81480af250 con 0x7f8148147410 2024-04-20T11:37:30.366 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.366+0000 7f8147e00640 1 --2- 172.21.15.152:0/1431805455 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8148147410 0x7f81480afa50 secure :-1 s=READY pgs=2224 cs=0 l=1 rev1=1 crypto rx=0x7f8148148310 tx=0x7f813802bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88af5148b947f034 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:30.366 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.366+0000 7f8146a00640 1 -- 172.21.15.152:0/1431805455 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f813802cb10 con 0x7f8148147410 2024-04-20T11:37:30.366 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.366+0000 7f8146a00640 1 -- 172.21.15.152:0/1431805455 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f813802ccb0 con 0x7f8148147410 2024-04-20T11:37:30.366 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.366+0000 7f8146a00640 1 -- 172.21.15.152:0/1431805455 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f813802ce60 con 0x7f8148147410 2024-04-20T11:37:30.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.367+0000 7f814dc00640 1 -- 172.21.15.152:0/1431805455 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8148147410 msgr2=0x7f81480afa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:30.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.367+0000 7f814dc00640 1 --2- 172.21.15.152:0/1431805455 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8148147410 0x7f81480afa50 secure :-1 s=READY pgs=2224 cs=0 l=1 rev1=1 crypto rx=0x7f8148148310 tx=0x7f813802bf70 comp rx=0 tx=0).stop 2024-04-20T11:37:30.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.367+0000 7f814dc00640 1 -- 172.21.15.152:0/1431805455 shutdown_connections 2024-04-20T11:37:30.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.367+0000 7f814dc00640 1 --2- 172.21.15.152:0/1431805455 >> v2:172.21.15.175:3300/0 conn(0x7f814806e720 0x7f8148072130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:30.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.367+0000 7f814dc00640 1 --2- 172.21.15.152:0/1431805455 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8148147410 0x7f81480afa50 unknown :-1 s=CLOSED pgs=2224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:30.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.367+0000 7f814dc00640 1 -- 172.21.15.152:0/1431805455 >> 172.21.15.152:0/1431805455 conn(0x7f8148067ae0 msgr2=0x7f814806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:30.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.367+0000 7f814dc00640 1 -- 172.21.15.152:0/1431805455 shutdown_connections 2024-04-20T11:37:30.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.367+0000 7f814dc00640 1 -- 172.21.15.152:0/1431805455 wait complete. 2024-04-20T11:37:30.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.369+0000 7f814dc00640 1 Processor -- start 2024-04-20T11:37:30.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.369+0000 7f814dc00640 1 -- start start 2024-04-20T11:37:30.370 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.370+0000 7f814dc00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f814806e720 0x7f81480b4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:30.370 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.370+0000 7f814dc00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8148071cd0 0x7f814813aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:30.370 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.370+0000 7f814dc00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8148147410 0x7f814813afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:30.370 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.370+0000 7f814dc00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f81480b3390 con 0x7f8148071cd0 2024-04-20T11:37:30.370 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.370+0000 7f814dc00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f81480ae980 con 0x7f814806e720 2024-04-20T11:37:30.370 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.370+0000 7f814dc00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f81480b3510 con 0x7f8148147410 2024-04-20T11:37:30.370 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.370+0000 7f8147400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8148071cd0 0x7f814813aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:30.370 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.370+0000 7f8147400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8148071cd0 0x7f814813aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36360/0 (socket says 172.21.15.152:36360) 2024-04-20T11:37:30.370 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.370+0000 7f8147400640 1 -- 172.21.15.152:0/4035261765 learned_addr learned my addr 172.21.15.152:0/4035261765 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:30.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.371+0000 7f8147e00640 1 --2- 172.21.15.152:0/4035261765 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f814806e720 0x7f81480b4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:30.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.371+0000 7f8147400640 1 -- 172.21.15.152:0/4035261765 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8148147410 msgr2=0x7f814813afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:30.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.371+0000 7f8147400640 1 --2- 172.21.15.152:0/4035261765 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8148147410 0x7f814813afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:30.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.371+0000 7f8147400640 1 -- 172.21.15.152:0/4035261765 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f814806e720 msgr2=0x7f81480b4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:30.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.371+0000 7f8147400640 1 --2- 172.21.15.152:0/4035261765 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f814806e720 0x7f81480b4cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:30.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.371+0000 7f8147400640 1 -- 172.21.15.152:0/4035261765 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8148161710 con 0x7f8148071cd0 2024-04-20T11:37:30.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.371+0000 7f8147400640 1 --2- 172.21.15.152:0/4035261765 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8148071cd0 0x7f814813aaa0 secure :-1 s=READY pgs=2225 cs=0 l=1 rev1=1 crypto rx=0x7f813000c400 tx=0x7f813000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:30.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.371+0000 7f813fe00640 1 -- 172.21.15.152:0/4035261765 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81300052d0 con 0x7f8148071cd0 2024-04-20T11:37:30.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.371+0000 7f813fe00640 1 -- 172.21.15.152:0/4035261765 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8130005490 con 0x7f8148071cd0 2024-04-20T11:37:30.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.371+0000 7f813fe00640 1 -- 172.21.15.152:0/4035261765 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8130007260 con 0x7f8148071cd0 2024-04-20T11:37:30.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.371+0000 7f814dc00640 1 -- 172.21.15.152:0/4035261765 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81481f57f0 con 0x7f8148071cd0 2024-04-20T11:37:30.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.371+0000 7f814dc00640 1 -- 172.21.15.152:0/4035261765 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81481f6030 con 0x7f8148071cd0 2024-04-20T11:37:30.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.372+0000 7f814dc00640 1 -- 172.21.15.152:0/4035261765 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8148075e60 con 0x7f8148071cd0 2024-04-20T11:37:30.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.373+0000 7f813fe00640 1 -- 172.21.15.152:0/4035261765 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 26) ==== 91907+0+0 (secure 0 0 0) 0x7f8130007400 con 0x7f8148071cd0 2024-04-20T11:37:30.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.374+0000 7f813fe00640 1 --2- 172.21.15.152:0/4035261765 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f812406d280 0x7f812406f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:30.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.374+0000 7f813fe00640 1 -- 172.21.15.152:0/4035261765 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5767+0+0 (secure 0 0 0) 0x7f8130089870 con 0x7f8148071cd0 2024-04-20T11:37:30.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.375+0000 7f8147e00640 1 --2- 172.21.15.152:0/4035261765 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f812406d280 0x7f812406f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:30.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.376+0000 7f8147e00640 1 --2- 172.21.15.152:0/4035261765 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f812406d280 0x7f812406f740 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f8138035000 tx=0x7f8138030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:30.378 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.379+0000 7f813fe00640 1 -- 172.21.15.152:0/4035261765 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f81481f57f0 con 0x7f8148071cd0 2024-04-20T11:37:30.582 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.583+0000 7f814dc00640 1 -- 172.21.15.152:0/4035261765 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f81480b11c0 con 0x7f8148071cd0 2024-04-20T11:37:30.585 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.586+0000 7f813fe00640 1 -- 172.21.15.152:0/4035261765 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v156) ==== 74+0+16781 (secure 0 0 0) 0x7f81300598a0 con 0x7f8148071cd0 2024-04-20T11:37:30.586 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:37:30.586 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":156,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:37:24.837045+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:36:57.520028+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"153","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2024-04-20T11:36:58.760637+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"154","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"25.13","mappings":[{"from":0,"to":1}]},{"pgid":"25.15","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:37:30.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.589+0000 7f814dc00640 1 -- 172.21.15.152:0/4035261765 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f812406d280 msgr2=0x7f812406f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:30.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.589+0000 7f814dc00640 1 --2- 172.21.15.152:0/4035261765 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f812406d280 0x7f812406f740 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f8138035000 tx=0x7f8138030000 comp rx=0 tx=0).stop 2024-04-20T11:37:30.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.589+0000 7f814dc00640 1 -- 172.21.15.152:0/4035261765 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8148071cd0 msgr2=0x7f814813aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:30.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.589+0000 7f814dc00640 1 --2- 172.21.15.152:0/4035261765 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8148071cd0 0x7f814813aaa0 secure :-1 s=READY pgs=2225 cs=0 l=1 rev1=1 crypto rx=0x7f813000c400 tx=0x7f813000c8c0 comp rx=0 tx=0).stop 2024-04-20T11:37:30.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.589+0000 7f814dc00640 1 -- 172.21.15.152:0/4035261765 shutdown_connections 2024-04-20T11:37:30.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.589+0000 7f814dc00640 1 --2- 172.21.15.152:0/4035261765 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f812406d280 0x7f812406f740 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:30.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.589+0000 7f814dc00640 1 --2- 172.21.15.152:0/4035261765 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8148147410 0x7f814813afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:30.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.589+0000 7f814dc00640 1 --2- 172.21.15.152:0/4035261765 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8148071cd0 0x7f814813aaa0 unknown :-1 s=CLOSED pgs=2225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:30.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.589+0000 7f814dc00640 1 --2- 172.21.15.152:0/4035261765 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f814806e720 0x7f81480b4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:30.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.589+0000 7f814dc00640 1 -- 172.21.15.152:0/4035261765 >> 172.21.15.152:0/4035261765 conn(0x7f8148067ae0 msgr2=0x7f81481422a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:30.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.589+0000 7f814dc00640 1 -- 172.21.15.152:0/4035261765 shutdown_connections 2024-04-20T11:37:30.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.589+0000 7f814dc00640 1 -- 172.21.15.152:0/4035261765 wait complete. 2024-04-20T11:37:30.607 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T11:26:47.246571+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 24, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T11:36:57.520028+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '153', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 25, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T11:36:58.760637+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '154', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T11:37:30.608 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T11:37:30.812 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.812+0000 7f1001400640 1 Processor -- start 2024-04-20T11:37:30.812 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.812+0000 7f1001400640 1 -- start start 2024-04-20T11:37:30.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.814+0000 7f1001400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0ffc0b2880 0x7f0ffc0b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:30.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.814+0000 7f1001400640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ffc1463e0 con 0x7f0ffc0cd1b0 2024-04-20T11:37:30.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.814+0000 7f1001400640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ffc0b3e20 con 0x7f0ffc146000 2024-04-20T11:37:30.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.814+0000 7f1001400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0ffc0b4750 con 0x7f0ffc0b2880 2024-04-20T11:37:30.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.814+0000 7f0ffb200640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f0ffc0cd1b0 0x7f0ffc0cd600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:40810/0 (socket says 172.21.15.152:40810) 2024-04-20T11:37:30.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.814+0000 7f0ffb200640 1 -- 172.21.15.152:0/1784516022 learned_addr learned my addr 172.21.15.152:0/1784516022 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:37:30.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.814+0000 7f0ff9e00640 1 --2- 172.21.15.152:0/1784516022 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0ffc0b2880 0x7f0ffc0b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:30.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.815+0000 7f0ff9400640 1 -- 172.21.15.152:0/1784516022 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4168120109 0 0) 0x7f0ffc1463e0 con 0x7f0ffc0cd1b0 2024-04-20T11:37:30.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.815+0000 7f0ff9400640 1 -- 172.21.15.152:0/1784516022 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0fe0003520 con 0x7f0ffc0cd1b0 2024-04-20T11:37:30.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.815+0000 7f0ff9400640 1 -- 172.21.15.152:0/1784516022 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 276577684 0 0) 0x7f0ffc0b3e20 con 0x7f0ffc146000 2024-04-20T11:37:30.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.815+0000 7f0ff9400640 1 -- 172.21.15.152:0/1784516022 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0ffc1463e0 con 0x7f0ffc146000 2024-04-20T11:37:30.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.815+0000 7f0ff9400640 1 -- 172.21.15.152:0/1784516022 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3841272597 0 0) 0x7f0fe0003520 con 0x7f0ffc0cd1b0 2024-04-20T11:37:30.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.815+0000 7f0ff9e00640 1 -- 172.21.15.152:0/1784516022 >> v1:172.21.15.175:6789/0 conn(0x7f0ffc146000 legacy=0x7f0ffc0b2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:30.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.815+0000 7f0ff9e00640 1 -- 172.21.15.152:0/1784516022 >> v1:172.21.15.152:6789/0 conn(0x7f0ffc0cd1b0 legacy=0x7f0ffc0cd600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:30.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.815+0000 7f0ff9e00640 1 -- 172.21.15.152:0/1784516022 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ffc0b4100 con 0x7f0ffc0b2880 2024-04-20T11:37:30.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.817+0000 7f0ff9e00640 1 --2- 172.21.15.152:0/1784516022 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0ffc0b2880 0x7f0ffc0b4be0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0fe80097d0 tx=0x7f0fe802b770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13648c0a0eb9ee6f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:30.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.817+0000 7f0ff9400640 1 -- 172.21.15.152:0/1784516022 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fe8037070 con 0x7f0ffc0b2880 2024-04-20T11:37:30.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.817+0000 7f0ff9400640 1 -- 172.21.15.152:0/1784516022 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fe8030510 con 0x7f0ffc0b2880 2024-04-20T11:37:30.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.817+0000 7f0ff9400640 1 -- 172.21.15.152:0/1784516022 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff40019f0 con 0x7f0ffc0b2880 2024-04-20T11:37:30.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.818+0000 7f1001400640 1 -- 172.21.15.152:0/1784516022 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0ffc0b2880 msgr2=0x7f0ffc0b4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:30.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.818+0000 7f1001400640 1 --2- 172.21.15.152:0/1784516022 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0ffc0b2880 0x7f0ffc0b4be0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0fe80097d0 tx=0x7f0fe802b770 comp rx=0 tx=0).stop 2024-04-20T11:37:30.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.818+0000 7f1001400640 1 -- 172.21.15.152:0/1784516022 shutdown_connections 2024-04-20T11:37:30.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.818+0000 7f1001400640 1 --2- 172.21.15.152:0/1784516022 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0ffc0b2880 0x7f0ffc0b4be0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:30.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.818+0000 7f1001400640 1 -- 172.21.15.152:0/1784516022 >> 172.21.15.152:0/1784516022 conn(0x7f0ffc06b980 msgr2=0x7f0ffc06bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:30.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.818+0000 7f1001400640 1 -- 172.21.15.152:0/1784516022 shutdown_connections 2024-04-20T11:37:30.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.818+0000 7f1001400640 1 -- 172.21.15.152:0/1784516022 wait complete. 2024-04-20T11:37:30.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.820+0000 7f1001400640 1 Processor -- start 2024-04-20T11:37:30.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.820+0000 7f1001400640 1 -- start start 2024-04-20T11:37:30.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.821+0000 7f1001400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0ffc0b2880 0x7f0ffc1e6be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:30.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.822+0000 7f1001400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ffc0cd1b0 0x7f0ffc0b11d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:30.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.822+0000 7f1001400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0ffc146000 0x7f0ffc0b1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:30.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.822+0000 7f1001400640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f0ffc15ce70 con 0x7f0ffc0b2880 2024-04-20T11:37:30.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.822+0000 7f1001400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0ffc15ccf0 con 0x7f0ffc0cd1b0 2024-04-20T11:37:30.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.822+0000 7f1001400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0ffc15cff0 con 0x7f0ffc146000 2024-04-20T11:37:30.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.822+0000 7f0ffa800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0ffc0b2880 0x7f0ffc1e6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:30.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.822+0000 7f0ffa800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0ffc0b2880 0x7f0ffc1e6be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36376/0 (socket says 172.21.15.152:36376) 2024-04-20T11:37:30.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.822+0000 7f0ffa800640 1 -- 172.21.15.152:0/2168751646 learned_addr learned my addr 172.21.15.152:0/2168751646 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:30.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.822+0000 7f0ffb200640 1 --2- 172.21.15.152:0/2168751646 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0ffc146000 0x7f0ffc0b1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:30.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.822+0000 7f0ff9e00640 1 --2- 172.21.15.152:0/2168751646 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ffc0cd1b0 0x7f0ffc0b11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:30.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.822+0000 7f0ffa800640 1 -- 172.21.15.152:0/2168751646 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0ffc146000 msgr2=0x7f0ffc0b1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:30.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.823+0000 7f0ffa800640 1 --2- 172.21.15.152:0/2168751646 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0ffc146000 0x7f0ffc0b1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:30.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.823+0000 7f0ffa800640 1 -- 172.21.15.152:0/2168751646 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ffc0cd1b0 msgr2=0x7f0ffc0b11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:30.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.823+0000 7f0ffa800640 1 --2- 172.21.15.152:0/2168751646 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ffc0cd1b0 0x7f0ffc0b11d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:30.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.823+0000 7f0ffa800640 1 -- 172.21.15.152:0/2168751646 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ffc1e8740 con 0x7f0ffc0b2880 2024-04-20T11:37:30.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.823+0000 7f0ffa800640 1 --2- 172.21.15.152:0/2168751646 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0ffc0b2880 0x7f0ffc1e6be0 secure :-1 s=READY pgs=2227 cs=0 l=1 rev1=1 crypto rx=0x7f0fec00c430 tx=0x7f0fec00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:30.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.823+0000 7f0ff2a00640 1 -- 172.21.15.152:0/2168751646 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fec017070 con 0x7f0ffc0b2880 2024-04-20T11:37:30.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.823+0000 7f0ff2a00640 1 -- 172.21.15.152:0/2168751646 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fec0052f0 con 0x7f0ffc0b2880 2024-04-20T11:37:30.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.823+0000 7f0ff2a00640 1 -- 172.21.15.152:0/2168751646 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fec003830 con 0x7f0ffc0b2880 2024-04-20T11:37:30.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.823+0000 7f1001400640 1 -- 172.21.15.152:0/2168751646 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ffc1e7120 con 0x7f0ffc0b2880 2024-04-20T11:37:30.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.823+0000 7f1001400640 1 -- 172.21.15.152:0/2168751646 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ffc1e7980 con 0x7f0ffc0b2880 2024-04-20T11:37:30.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.825+0000 7f1001400640 1 -- 172.21.15.152:0/2168751646 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ffc0689f0 con 0x7f0ffc0b2880 2024-04-20T11:37:30.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.825+0000 7f0ff2a00640 1 -- 172.21.15.152:0/2168751646 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 26) ==== 91907+0+0 (secure 0 0 0) 0x7f0fec01a070 con 0x7f0ffc0b2880 2024-04-20T11:37:30.826 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.826+0000 7f0ff2a00640 1 --2- 172.21.15.152:0/2168751646 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f0fdc06d2b0 0x7f0fdc06f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:30.826 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.827+0000 7f0ff2a00640 1 -- 172.21.15.152:0/2168751646 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5767+0+0 (secure 0 0 0) 0x7f0fec089720 con 0x7f0ffc0b2880 2024-04-20T11:37:30.826 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.827+0000 7f0ff9e00640 1 --2- 172.21.15.152:0/2168751646 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f0fdc06d2b0 0x7f0fdc06f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:30.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.828+0000 7f0ff9e00640 1 --2- 172.21.15.152:0/2168751646 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f0fdc06d2b0 0x7f0fdc06f770 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f0fe8034000 tx=0x7f0fe8008030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:30.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:30.831+0000 7f0ff2a00640 1 -- 172.21.15.152:0/2168751646 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f0fec059750 con 0x7f0ffc0b2880 2024-04-20T11:37:31.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.032+0000 7f1001400640 1 -- 172.21.15.152:0/2168751646 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0ffc0b2e00 con 0x7f0ffc0b2880 2024-04-20T11:37:31.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.033+0000 7f0ff2a00640 1 -- 172.21.15.152:0/2168751646 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v156) ==== 93+0+10 (secure 0 0 0) 0x7f0fec059050 con 0x7f0ffc0b2880 2024-04-20T11:37:31.033 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 1 2024-04-20T11:37:31.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.035+0000 7f1001400640 1 -- 172.21.15.152:0/2168751646 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f0fdc06d2b0 msgr2=0x7f0fdc06f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:31.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.035+0000 7f1001400640 1 --2- 172.21.15.152:0/2168751646 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f0fdc06d2b0 0x7f0fdc06f770 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f0fe8034000 tx=0x7f0fe8008030 comp rx=0 tx=0).stop 2024-04-20T11:37:31.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.035+0000 7f1001400640 1 -- 172.21.15.152:0/2168751646 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0ffc0b2880 msgr2=0x7f0ffc1e6be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:31.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.035+0000 7f1001400640 1 --2- 172.21.15.152:0/2168751646 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0ffc0b2880 0x7f0ffc1e6be0 secure :-1 s=READY pgs=2227 cs=0 l=1 rev1=1 crypto rx=0x7f0fec00c430 tx=0x7f0fec00c8f0 comp rx=0 tx=0).stop 2024-04-20T11:37:31.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.035+0000 7f1001400640 1 -- 172.21.15.152:0/2168751646 shutdown_connections 2024-04-20T11:37:31.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.035+0000 7f1001400640 1 --2- 172.21.15.152:0/2168751646 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f0fdc06d2b0 0x7f0fdc06f770 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:31.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.035+0000 7f1001400640 1 --2- 172.21.15.152:0/2168751646 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0ffc146000 0x7f0ffc0b1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:31.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.035+0000 7f1001400640 1 --2- 172.21.15.152:0/2168751646 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ffc0cd1b0 0x7f0ffc0b11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:31.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.035+0000 7f1001400640 1 --2- 172.21.15.152:0/2168751646 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0ffc0b2880 0x7f0ffc1e6be0 unknown :-1 s=CLOSED pgs=2227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:31.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.035+0000 7f1001400640 1 -- 172.21.15.152:0/2168751646 >> 172.21.15.152:0/2168751646 conn(0x7f0ffc06b980 msgr2=0x7f0ffc0d0430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:31.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.036+0000 7f1001400640 1 -- 172.21.15.152:0/2168751646 shutdown_connections 2024-04-20T11:37:31.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.036+0000 7f1001400640 1 -- 172.21.15.152:0/2168751646 wait complete. 2024-04-20T11:37:31.054 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T11:37:31.251 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.251+0000 7f2df4000640 1 Processor -- start 2024-04-20T11:37:31.251 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.251+0000 7f2df4000640 1 -- start start 2024-04-20T11:37:31.253 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.253+0000 7f2df4000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f2dec15b510 0x7f2dec15fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:31.253 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.253+0000 7f2df4000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2dec0b2880 0x7f2dec0b3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:31.253 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.253+0000 7f2df4000640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2dec15f1c0 con 0x7f2dec0cfc40 2024-04-20T11:37:31.253 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.253+0000 7f2df4000640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f2dec06f420 con 0x7f2dec15b510 2024-04-20T11:37:31.253 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.253+0000 7f2df4000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2dec0b4430 con 0x7f2dec0b2880 2024-04-20T11:37:31.253 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.253+0000 7f2df1600640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f2dec15b510 0x7f2dec15fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:31.254 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.254+0000 7f2df1600640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f2dec15b510 0x7f2dec15fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36388/0 (socket says 172.21.15.152:36388) 2024-04-20T11:37:31.254 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.254+0000 7f2df1600640 1 -- 172.21.15.152:0/2726931359 learned_addr learned my addr 172.21.15.152:0/2726931359 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:31.254 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.254+0000 7f2df1600640 1 -- 172.21.15.152:0/2726931359 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2dec0b2880 msgr2=0x7f2dec0b3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:31.254 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.254+0000 7f2df1600640 1 --2- 172.21.15.152:0/2726931359 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2dec0b2880 0x7f2dec0b3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:31.254 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.254+0000 7f2df1600640 1 -- 172.21.15.152:0/2726931359 >> v1:172.21.15.175:6789/0 conn(0x7f2dec0cfc40 legacy=0x7f2dec0b2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:31.254 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.254+0000 7f2df1600640 1 -- 172.21.15.152:0/2726931359 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dec06f670 con 0x7f2dec15b510 2024-04-20T11:37:31.255 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.255+0000 7f2df1600640 1 --2- 172.21.15.152:0/2726931359 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2dec15b510 0x7f2dec15fc30 secure :-1 s=READY pgs=2228 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc009870 tx=0x7f2ddc02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91fe7ff8ef0fca46 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:31.256 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.255+0000 7f2debe00640 1 -- 172.21.15.152:0/2726931359 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ddc035020 con 0x7f2dec15b510 2024-04-20T11:37:31.256 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.255+0000 7f2debe00640 1 -- 172.21.15.152:0/2726931359 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ddc031640 con 0x7f2dec15b510 2024-04-20T11:37:31.256 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.255+0000 7f2debe00640 1 -- 172.21.15.152:0/2726931359 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ddc02d040 con 0x7f2dec15b510 2024-04-20T11:37:31.256 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.256+0000 7f2df4000640 1 -- 172.21.15.152:0/2726931359 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2dec15b510 msgr2=0x7f2dec15fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:31.256 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.256+0000 7f2df4000640 1 --2- 172.21.15.152:0/2726931359 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2dec15b510 0x7f2dec15fc30 secure :-1 s=READY pgs=2228 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc009870 tx=0x7f2ddc02b8f0 comp rx=0 tx=0).stop 2024-04-20T11:37:31.256 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.256+0000 7f2df4000640 1 -- 172.21.15.152:0/2726931359 shutdown_connections 2024-04-20T11:37:31.256 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.256+0000 7f2df4000640 1 --2- 172.21.15.152:0/2726931359 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2dec0b2880 0x7f2dec0b3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:31.256 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.256+0000 7f2df4000640 1 --2- 172.21.15.152:0/2726931359 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2dec15b510 0x7f2dec15fc30 unknown :-1 s=CLOSED pgs=2228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:31.256 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.256+0000 7f2df4000640 1 -- 172.21.15.152:0/2726931359 >> 172.21.15.152:0/2726931359 conn(0x7f2dec06b980 msgr2=0x7f2dec06bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:31.256 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.256+0000 7f2df4000640 1 -- 172.21.15.152:0/2726931359 shutdown_connections 2024-04-20T11:37:31.256 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.256+0000 7f2df4000640 1 -- 172.21.15.152:0/2726931359 wait complete. 2024-04-20T11:37:31.257 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.258+0000 7f2df4000640 1 Processor -- start 2024-04-20T11:37:31.257 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.258+0000 7f2df4000640 1 -- start start 2024-04-20T11:37:31.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.259+0000 7f2df4000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2dec0b2880 0x7f2dec152aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:31.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.259+0000 7f2df4000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2dec0cfc40 0x7f2dec159c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:31.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.259+0000 7f2df4000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2dec15b510 0x7f2dec15bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:31.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.259+0000 7f2df4000640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f2dec15b8f0 con 0x7f2dec15b510 2024-04-20T11:37:31.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.259+0000 7f2df4000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2dec15f710 con 0x7f2dec0b2880 2024-04-20T11:37:31.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.259+0000 7f2df4000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2dec06f950 con 0x7f2dec0cfc40 2024-04-20T11:37:31.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.259+0000 7f2df2000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2dec15b510 0x7f2dec15bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:31.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.259+0000 7f2df2000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2dec15b510 0x7f2dec15bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36390/0 (socket says 172.21.15.152:36390) 2024-04-20T11:37:31.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.259+0000 7f2df2000640 1 -- 172.21.15.152:0/115257500 learned_addr learned my addr 172.21.15.152:0/115257500 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:31.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.260+0000 7f2df0c00640 1 --2- 172.21.15.152:0/115257500 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2dec0cfc40 0x7f2dec159c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:31.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.260+0000 7f2df1600640 1 --2- 172.21.15.152:0/115257500 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2dec0b2880 0x7f2dec152aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:31.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.260+0000 7f2df2000640 1 -- 172.21.15.152:0/115257500 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2dec0cfc40 msgr2=0x7f2dec159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:31.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.260+0000 7f2df2000640 1 --2- 172.21.15.152:0/115257500 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2dec0cfc40 0x7f2dec159c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:31.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.260+0000 7f2df2000640 1 -- 172.21.15.152:0/115257500 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2dec0b2880 msgr2=0x7f2dec152aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:31.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.260+0000 7f2df2000640 1 --2- 172.21.15.152:0/115257500 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2dec0b2880 0x7f2dec152aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:31.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.260+0000 7f2df2000640 1 -- 172.21.15.152:0/115257500 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dec152fe0 con 0x7f2dec15b510 2024-04-20T11:37:31.262 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.260+0000 7f2df2000640 1 --2- 172.21.15.152:0/115257500 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2dec15b510 0x7f2dec15bae0 secure :-1 s=READY pgs=2229 cs=0 l=1 rev1=1 crypto rx=0x7f2de400b970 tx=0x7f2de400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:31.262 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.261+0000 7f2de9600640 1 -- 172.21.15.152:0/115257500 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de4019070 con 0x7f2dec15b510 2024-04-20T11:37:31.262 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.261+0000 7f2de9600640 1 -- 172.21.15.152:0/115257500 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2de400cb60 con 0x7f2dec15b510 2024-04-20T11:37:31.262 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.261+0000 7f2de9600640 1 -- 172.21.15.152:0/115257500 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de4014440 con 0x7f2dec15b510 2024-04-20T11:37:31.262 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.261+0000 7f2df4000640 1 -- 172.21.15.152:0/115257500 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2dec1f88f0 con 0x7f2dec15b510 2024-04-20T11:37:31.262 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.261+0000 7f2df4000640 1 -- 172.21.15.152:0/115257500 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2dec1f8eb0 con 0x7f2dec15b510 2024-04-20T11:37:31.262 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.262+0000 7f2df4000640 1 -- 172.21.15.152:0/115257500 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2dec0689f0 con 0x7f2dec15b510 2024-04-20T11:37:31.262 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.263+0000 7f2de9600640 1 -- 172.21.15.152:0/115257500 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 26) ==== 91907+0+0 (secure 0 0 0) 0x7f2de401c070 con 0x7f2dec15b510 2024-04-20T11:37:31.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.264+0000 7f2de9600640 1 --2- 172.21.15.152:0/115257500 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f2dd006d2b0 0x7f2dd006f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:31.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.264+0000 7f2de9600640 1 -- 172.21.15.152:0/115257500 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5767+0+0 (secure 0 0 0) 0x7f2de408a000 con 0x7f2dec15b510 2024-04-20T11:37:31.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.265+0000 7f2df1600640 1 --2- 172.21.15.152:0/115257500 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f2dd006d2b0 0x7f2dd006f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:31.265 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.266+0000 7f2df1600640 1 --2- 172.21.15.152:0/115257500 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f2dd006d2b0 0x7f2dd006f770 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc036000 tx=0x7f2ddc03c040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:31.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.268+0000 7f2de9600640 1 -- 172.21.15.152:0/115257500 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f2de405a030 con 0x7f2dec15b510 2024-04-20T11:37:31.473 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.473+0000 7f2df4000640 1 -- 172.21.15.152:0/115257500 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f2dec06ecf0 con 0x7f2dec15b510 2024-04-20T11:37:31.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.474+0000 7f2de9600640 1 -- 172.21.15.152:0/115257500 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v156) ==== 104+0+11 (secure 0 0 0) 0x7f2de4059930 con 0x7f2dec15b510 2024-04-20T11:37:31.474 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 32 2024-04-20T11:37:31.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.477+0000 7f2df4000640 1 -- 172.21.15.152:0/115257500 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f2dd006d2b0 msgr2=0x7f2dd006f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:31.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.477+0000 7f2df4000640 1 --2- 172.21.15.152:0/115257500 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f2dd006d2b0 0x7f2dd006f770 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc036000 tx=0x7f2ddc03c040 comp rx=0 tx=0).stop 2024-04-20T11:37:31.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.477+0000 7f2df4000640 1 -- 172.21.15.152:0/115257500 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2dec15b510 msgr2=0x7f2dec15bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:31.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.477+0000 7f2df4000640 1 --2- 172.21.15.152:0/115257500 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2dec15b510 0x7f2dec15bae0 secure :-1 s=READY pgs=2229 cs=0 l=1 rev1=1 crypto rx=0x7f2de400b970 tx=0x7f2de400be30 comp rx=0 tx=0).stop 2024-04-20T11:37:31.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.477+0000 7f2df4000640 1 -- 172.21.15.152:0/115257500 shutdown_connections 2024-04-20T11:37:31.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.477+0000 7f2df4000640 1 --2- 172.21.15.152:0/115257500 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f2dd006d2b0 0x7f2dd006f770 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:31.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.477+0000 7f2df4000640 1 --2- 172.21.15.152:0/115257500 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2dec15b510 0x7f2dec15bae0 unknown :-1 s=CLOSED pgs=2229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:31.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.477+0000 7f2df4000640 1 --2- 172.21.15.152:0/115257500 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2dec0cfc40 0x7f2dec159c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:31.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.477+0000 7f2df4000640 1 --2- 172.21.15.152:0/115257500 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2dec0b2880 0x7f2dec152aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:31.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.477+0000 7f2df4000640 1 -- 172.21.15.152:0/115257500 >> 172.21.15.152:0/115257500 conn(0x7f2dec06b980 msgr2=0x7f2dec0cf1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:31.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.477+0000 7f2df4000640 1 -- 172.21.15.152:0/115257500 shutdown_connections 2024-04-20T11:37:31.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.477+0000 7f2df4000640 1 -- 172.21.15.152:0/115257500 wait complete. 2024-04-20T11:37:31.496 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T11:37:31.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.703+0000 7f83f0c00640 1 Processor -- start 2024-04-20T11:37:31.703 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.703+0000 7f83f0c00640 1 -- start start 2024-04-20T11:37:31.705 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.705+0000 7f83f0c00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f83ec0cfc40 0x7f83ec15fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:31.705 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.705+0000 7f83f0c00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83ec15f1c0 con 0x7f83ec15b510 2024-04-20T11:37:31.705 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.705+0000 7f83f0c00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83ec06f670 con 0x7f83ec0b2880 2024-04-20T11:37:31.705 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.705+0000 7f83f0c00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f83ec06f420 con 0x7f83ec0cfc40 2024-04-20T11:37:31.705 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.705+0000 7f83e9e00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f83ec15b510 0x7f83ec0b2170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:40812/0 (socket says 172.21.15.152:40812) 2024-04-20T11:37:31.705 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.705+0000 7f83e9e00640 1 -- 172.21.15.152:0/984667164 learned_addr learned my addr 172.21.15.152:0/984667164 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:37:31.705 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.705+0000 7f83e9400640 1 --2- 172.21.15.152:0/984667164 >> v2:172.21.15.152:3300/0 conn(0x7f83ec0cfc40 0x7f83ec15fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:31.706 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.706+0000 7f83e8a00640 1 -- 172.21.15.152:0/984667164 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 988667464 0 0) 0x7f83ec15f1c0 con 0x7f83ec15b510 2024-04-20T11:37:31.706 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.706+0000 7f83e8a00640 1 -- 172.21.15.152:0/984667164 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f83d0003520 con 0x7f83ec15b510 2024-04-20T11:37:31.706 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.706+0000 7f83e9400640 1 -- 172.21.15.152:0/984667164 >> v1:172.21.15.175:6789/0 conn(0x7f83ec0b2880 legacy=0x7f83ec0b4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:31.706 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.706+0000 7f83e9400640 1 -- 172.21.15.152:0/984667164 >> v1:172.21.15.152:6789/0 conn(0x7f83ec15b510 legacy=0x7f83ec0b2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:31.706 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.706+0000 7f83e9400640 1 -- 172.21.15.152:0/984667164 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83ec15f430 con 0x7f83ec0cfc40 2024-04-20T11:37:31.707 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.707+0000 7f83e9400640 1 --2- 172.21.15.152:0/984667164 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f83ec0cfc40 0x7f83ec15fc30 secure :-1 s=READY pgs=2231 cs=0 l=1 rev1=1 crypto rx=0x7f83d80097a0 tx=0x7f83d802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c4813bfeef350f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:31.707 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.707+0000 7f83e8a00640 1 -- 172.21.15.152:0/984667164 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d80315a0 con 0x7f83ec0cfc40 2024-04-20T11:37:31.707 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.707+0000 7f83e8a00640 1 -- 172.21.15.152:0/984667164 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83d8037070 con 0x7f83ec0cfc40 2024-04-20T11:37:31.707 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.707+0000 7f83e8a00640 1 -- 172.21.15.152:0/984667164 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d8031960 con 0x7f83ec0cfc40 2024-04-20T11:37:31.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.708+0000 7f83f0c00640 1 -- 172.21.15.152:0/984667164 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f83ec0cfc40 msgr2=0x7f83ec15fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:31.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.708+0000 7f83f0c00640 1 --2- 172.21.15.152:0/984667164 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f83ec0cfc40 0x7f83ec15fc30 secure :-1 s=READY pgs=2231 cs=0 l=1 rev1=1 crypto rx=0x7f83d80097a0 tx=0x7f83d802b840 comp rx=0 tx=0).stop 2024-04-20T11:37:31.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.708+0000 7f83f0c00640 1 -- 172.21.15.152:0/984667164 shutdown_connections 2024-04-20T11:37:31.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.708+0000 7f83f0c00640 1 --2- 172.21.15.152:0/984667164 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f83ec0cfc40 0x7f83ec15fc30 unknown :-1 s=CLOSED pgs=2231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:31.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.708+0000 7f83f0c00640 1 -- 172.21.15.152:0/984667164 >> 172.21.15.152:0/984667164 conn(0x7f83ec06b980 msgr2=0x7f83ec06bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:31.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.708+0000 7f83f0c00640 1 -- 172.21.15.152:0/984667164 shutdown_connections 2024-04-20T11:37:31.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.708+0000 7f83f0c00640 1 -- 172.21.15.152:0/984667164 wait complete. 2024-04-20T11:37:31.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.710+0000 7f83f0c00640 1 Processor -- start 2024-04-20T11:37:31.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.710+0000 7f83f0c00640 1 -- start start 2024-04-20T11:37:31.711 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.711+0000 7f83f0c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f83ec0b2880 0x7f83ec0b2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:31.711 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.711+0000 7f83f0c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f83ec0cfc40 0x7f83ec0b3410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:31.711 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.711+0000 7f83f0c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f83ec15b510 0x7f83ec15ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:31.711 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.711+0000 7f83f0c00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f83ec06f950 con 0x7f83ec0b2880 2024-04-20T11:37:31.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.711+0000 7f83f0c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f83ec15b8f0 con 0x7f83ec0cfc40 2024-04-20T11:37:31.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.711+0000 7f83f0c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f83ec0b3ef0 con 0x7f83ec15b510 2024-04-20T11:37:31.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.711+0000 7f83e9e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f83ec0b2880 0x7f83ec0b2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:31.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.711+0000 7f83e9e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f83ec0b2880 0x7f83ec0b2ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36400/0 (socket says 172.21.15.152:36400) 2024-04-20T11:37:31.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.711+0000 7f83e9e00640 1 -- 172.21.15.152:0/424941872 learned_addr learned my addr 172.21.15.152:0/424941872 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:31.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.712+0000 7f83e9400640 1 --2- 172.21.15.152:0/424941872 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f83ec0cfc40 0x7f83ec0b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:31.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.712+0000 7f83ea800640 1 --2- 172.21.15.152:0/424941872 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f83ec15b510 0x7f83ec15ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:31.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.712+0000 7f83e9e00640 1 -- 172.21.15.152:0/424941872 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f83ec15b510 msgr2=0x7f83ec15ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:31.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.712+0000 7f83e9e00640 1 --2- 172.21.15.152:0/424941872 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f83ec15b510 0x7f83ec15ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:31.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.712+0000 7f83e9e00640 1 -- 172.21.15.152:0/424941872 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f83ec0cfc40 msgr2=0x7f83ec0b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:31.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.712+0000 7f83e9e00640 1 --2- 172.21.15.152:0/424941872 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f83ec0cfc40 0x7f83ec0b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:31.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.712+0000 7f83e9e00640 1 -- 172.21.15.152:0/424941872 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83ec15c730 con 0x7f83ec0b2880 2024-04-20T11:37:31.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.712+0000 7f83e9400640 1 --2- 172.21.15.152:0/424941872 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f83ec0cfc40 0x7f83ec0b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:37:31.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.712+0000 7f83e9e00640 1 --2- 172.21.15.152:0/424941872 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f83ec0b2880 0x7f83ec0b2ed0 secure :-1 s=READY pgs=2232 cs=0 l=1 rev1=1 crypto rx=0x7f83dc00c3c0 tx=0x7f83dc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:31.713 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.712+0000 7f83e2000640 1 -- 172.21.15.152:0/424941872 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83dc017070 con 0x7f83ec0b2880 2024-04-20T11:37:31.714 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.712+0000 7f83e2000640 1 -- 172.21.15.152:0/424941872 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83dc005ce0 con 0x7f83ec0b2880 2024-04-20T11:37:31.714 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.712+0000 7f83f0c00640 1 -- 172.21.15.152:0/424941872 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83ec15ca10 con 0x7f83ec0b2880 2024-04-20T11:37:31.714 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.712+0000 7f83e2000640 1 -- 172.21.15.152:0/424941872 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83dc0072f0 con 0x7f83ec0b2880 2024-04-20T11:37:31.714 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.713+0000 7f83f0c00640 1 -- 172.21.15.152:0/424941872 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83ec15cfd0 con 0x7f83ec0b2880 2024-04-20T11:37:31.714 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.714+0000 7f83f0c00640 1 -- 172.21.15.152:0/424941872 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83ec0689f0 con 0x7f83ec0b2880 2024-04-20T11:37:31.714 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.715+0000 7f83e2000640 1 -- 172.21.15.152:0/424941872 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 26) ==== 91907+0+0 (secure 0 0 0) 0x7f83dc01a070 con 0x7f83ec0b2880 2024-04-20T11:37:31.715 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.715+0000 7f83e2000640 1 --2- 172.21.15.152:0/424941872 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f83cc06d280 0x7f83cc06f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:31.715 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.716+0000 7f83e2000640 1 -- 172.21.15.152:0/424941872 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5767+0+0 (secure 0 0 0) 0x7f83dc0896e0 con 0x7f83ec0b2880 2024-04-20T11:37:31.716 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.716+0000 7f83e9400640 1 --2- 172.21.15.152:0/424941872 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f83cc06d280 0x7f83cc06f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:31.717 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.717+0000 7f83e9400640 1 --2- 172.21.15.152:0/424941872 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f83cc06d280 0x7f83cc06f740 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f83d8036000 tx=0x7f83d8008030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:31.719 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.720+0000 7f83e2000640 1 -- 172.21.15.152:0/424941872 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f83dc059710 con 0x7f83ec0b2880 2024-04-20T11:37:31.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.920+0000 7f83f0c00640 1 -- 172.21.15.152:0/424941872 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f83ec157740 con 0x7f83ec0b2880 2024-04-20T11:37:31.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.921+0000 7f83e2000640 1 -- 172.21.15.152:0/424941872 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v156) ==== 100+0+11 (secure 0 0 0) 0x7f83dc059010 con 0x7f83ec0b2880 2024-04-20T11:37:31.921 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 64 2024-04-20T11:37:31.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.923+0000 7f83f0c00640 1 -- 172.21.15.152:0/424941872 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f83cc06d280 msgr2=0x7f83cc06f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:31.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.923+0000 7f83f0c00640 1 --2- 172.21.15.152:0/424941872 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f83cc06d280 0x7f83cc06f740 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f83d8036000 tx=0x7f83d8008030 comp rx=0 tx=0).stop 2024-04-20T11:37:31.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.924+0000 7f83f0c00640 1 -- 172.21.15.152:0/424941872 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f83ec0b2880 msgr2=0x7f83ec0b2ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:31.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.924+0000 7f83f0c00640 1 --2- 172.21.15.152:0/424941872 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f83ec0b2880 0x7f83ec0b2ed0 secure :-1 s=READY pgs=2232 cs=0 l=1 rev1=1 crypto rx=0x7f83dc00c3c0 tx=0x7f83dc00c880 comp rx=0 tx=0).stop 2024-04-20T11:37:31.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.924+0000 7f83f0c00640 1 -- 172.21.15.152:0/424941872 shutdown_connections 2024-04-20T11:37:31.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.924+0000 7f83f0c00640 1 --2- 172.21.15.152:0/424941872 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f83cc06d280 0x7f83cc06f740 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:31.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.924+0000 7f83f0c00640 1 --2- 172.21.15.152:0/424941872 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f83ec15b510 0x7f83ec15ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:31.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.924+0000 7f83f0c00640 1 --2- 172.21.15.152:0/424941872 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f83ec0cfc40 0x7f83ec0b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:31.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.924+0000 7f83f0c00640 1 --2- 172.21.15.152:0/424941872 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f83ec0b2880 0x7f83ec0b2ed0 unknown :-1 s=CLOSED pgs=2232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:31.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.924+0000 7f83f0c00640 1 -- 172.21.15.152:0/424941872 >> 172.21.15.152:0/424941872 conn(0x7f83ec06b980 msgr2=0x7f83ec0ce290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:31.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.924+0000 7f83f0c00640 1 -- 172.21.15.152:0/424941872 shutdown_connections 2024-04-20T11:37:31.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:31.924+0000 7f83f0c00640 1 -- 172.21.15.152:0/424941872 wait complete. 2024-04-20T11:37:31.942 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:37:32.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.147+0000 7f7bb2c00640 1 Processor -- start 2024-04-20T11:37:32.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.148+0000 7f7bb2c00640 1 -- start start 2024-04-20T11:37:32.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.149+0000 7f7bb2c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7bac0cd0d0 0x7f7bac0cd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:32.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.149+0000 7f7bb2c00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f7bac071fe0 0x7f7bac072470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:32.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.149+0000 7f7bb2c00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7bac146340 con 0x7f7bac145f60 2024-04-20T11:37:32.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.149+0000 7f7bb2c00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f7bac06e9f0 con 0x7f7bac071fe0 2024-04-20T11:37:32.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.149+0000 7f7bb2c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7bac072d50 con 0x7f7bac0cd0d0 2024-04-20T11:37:32.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.150+0000 7f7babe00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f7bac145f60 0x7f7bac06ee80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:40818/0 (socket says 172.21.15.152:40818) 2024-04-20T11:37:32.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.150+0000 7f7babe00640 1 -- 172.21.15.152:0/1176794506 learned_addr learned my addr 172.21.15.152:0/1176794506 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:37:32.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.150+0000 7f7ba3400640 1 --2- 172.21.15.152:0/1176794506 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7bac0cd0d0 0x7f7bac0cd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:32.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.150+0000 7f7bab400640 1 -- 172.21.15.152:0/1176794506 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2602757752 0 0) 0x7f7bac146340 con 0x7f7bac145f60 2024-04-20T11:37:32.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.150+0000 7f7bab400640 1 -- 172.21.15.152:0/1176794506 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7b90003520 con 0x7f7bac145f60 2024-04-20T11:37:32.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.150+0000 7f7bb0c00640 1 --2- 172.21.15.152:0/1176794506 >> v2:172.21.15.175:3300/0 conn(0x7f7bac071fe0 0x7f7bac072470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:32.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.150+0000 7f7bab400640 1 -- 172.21.15.152:0/1176794506 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2122410802 0 0) 0x7f7b90003520 con 0x7f7bac145f60 2024-04-20T11:37:32.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.150+0000 7f7bab400640 1 -- 172.21.15.152:0/1176794506 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7bac0cd0d0 msgr2=0x7f7bac0cd550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:32.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.150+0000 7f7bab400640 1 --2- 172.21.15.152:0/1176794506 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7bac0cd0d0 0x7f7bac0cd550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:32.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.151+0000 7f7bab400640 1 -- 172.21.15.152:0/1176794506 >> v2:172.21.15.175:3300/0 conn(0x7f7bac071fe0 msgr2=0x7f7bac072470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:32.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.151+0000 7f7bab400640 1 --2- 172.21.15.152:0/1176794506 >> v2:172.21.15.175:3300/0 conn(0x7f7bac071fe0 0x7f7bac072470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:32.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.151+0000 7f7bab400640 1 -- 172.21.15.152:0/1176794506 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bac06e0c0 con 0x7f7bac145f60 2024-04-20T11:37:32.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.151+0000 7f7bab400640 1 -- 172.21.15.152:0/1176794506 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f7b9c001670 con 0x7f7bac145f60 2024-04-20T11:37:32.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.151+0000 7f7bab400640 1 -- 172.21.15.152:0/1176794506 >> v1:172.21.15.152:6789/0 conn(0x7f7bac145f60 legacy=0x7f7bac06ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:32.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.151+0000 7f7bab400640 1 --2- 172.21.15.152:0/1176794506 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7bac0cd0d0 0x7f7b90003a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:32.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.151+0000 7f7bab400640 1 --2- 172.21.15.152:0/1176794506 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b90003fc0 0x7f7b90004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:32.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.151+0000 7f7bab400640 1 --2- 172.21.15.152:0/1176794506 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7b90004d40 0x7f7b900051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:32.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.151+0000 7f7bab400640 1 -- 172.21.15.152:0/1176794506 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f7bac06e9f0 con 0x7f7b90004d40 2024-04-20T11:37:32.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.151+0000 7f7bab400640 1 -- 172.21.15.152:0/1176794506 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7bac072d50 con 0x7f7b90003fc0 2024-04-20T11:37:32.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.151+0000 7f7bab400640 1 -- 172.21.15.152:0/1176794506 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7b9000baa0 con 0x7f7bac0cd0d0 2024-04-20T11:37:32.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.151+0000 7f7bb0c00640 1 --2- 172.21.15.152:0/1176794506 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7b90004d40 0x7f7b900051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:32.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.151+0000 7f7ba3400640 1 --2- 172.21.15.152:0/1176794506 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b90003fc0 0x7f7b90004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:32.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.151+0000 7f7babe00640 1 --2- 172.21.15.152:0/1176794506 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7bac0cd0d0 0x7f7b90003a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:32.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.152+0000 7f7bb0c00640 1 -- 172.21.15.152:0/1176794506 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7bac0cd0d0 msgr2=0x7f7b90003a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:32.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.152+0000 7f7bb0c00640 1 --2- 172.21.15.152:0/1176794506 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7bac0cd0d0 0x7f7b90003a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:32.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.152+0000 7f7bb0c00640 1 -- 172.21.15.152:0/1176794506 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b90003fc0 msgr2=0x7f7b90004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:32.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.152+0000 7f7bb0c00640 1 --2- 172.21.15.152:0/1176794506 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b90003fc0 0x7f7b90004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:32.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.152+0000 7f7bb0c00640 1 -- 172.21.15.152:0/1176794506 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b90003520 con 0x7f7b90004d40 2024-04-20T11:37:32.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.152+0000 7f7babe00640 1 --2- 172.21.15.152:0/1176794506 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7bac0cd0d0 0x7f7b90003a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:37:32.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.153+0000 7f7bb0c00640 1 --2- 172.21.15.152:0/1176794506 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7b90004d40 0x7f7b900051c0 secure :-1 s=READY pgs=2234 cs=0 l=1 rev1=1 crypto rx=0x7f7ba4007360 tx=0x7f7b90005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14bb05dcb2101fcd server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:32.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.153+0000 7f7bab400640 1 -- 172.21.15.152:0/1176794506 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ba4039070 con 0x7f7b90004d40 2024-04-20T11:37:32.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.153+0000 7f7bab400640 1 -- 172.21.15.152:0/1176794506 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ba402ccf0 con 0x7f7b90004d40 2024-04-20T11:37:32.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.153+0000 7f7bab400640 1 -- 172.21.15.152:0/1176794506 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ba4009be0 con 0x7f7b90004d40 2024-04-20T11:37:32.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.154+0000 7f7bb2c00640 1 -- 172.21.15.152:0/1176794506 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7b90004d40 msgr2=0x7f7b900051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:32.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.154+0000 7f7bb2c00640 1 --2- 172.21.15.152:0/1176794506 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7b90004d40 0x7f7b900051c0 secure :-1 s=READY pgs=2234 cs=0 l=1 rev1=1 crypto rx=0x7f7ba4007360 tx=0x7f7b90005830 comp rx=0 tx=0).stop 2024-04-20T11:37:32.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.154+0000 7f7bb0c00640 1 -- 172.21.15.152:0/1176794506 reap_dead start 2024-04-20T11:37:32.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.154+0000 7f7bb2c00640 1 -- 172.21.15.152:0/1176794506 shutdown_connections 2024-04-20T11:37:32.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.154+0000 7f7bb2c00640 1 -- 172.21.15.152:0/1176794506 >> 172.21.15.152:0/1176794506 conn(0x7f7bac067cf0 msgr2=0x7f7bac06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:32.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.154+0000 7f7bb2c00640 1 -- 172.21.15.152:0/1176794506 shutdown_connections 2024-04-20T11:37:32.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.154+0000 7f7bb2c00640 1 -- 172.21.15.152:0/1176794506 wait complete. 2024-04-20T11:37:32.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.156+0000 7f7bb2c00640 1 Processor -- start 2024-04-20T11:37:32.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.156+0000 7f7bb2c00640 1 -- start start 2024-04-20T11:37:32.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.157+0000 7f7bb2c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7b90004d40 0x7f7bac140f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:32.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.157+0000 7f7bb2c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7bac142800 0x7f7bac0b4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:32.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.157+0000 7f7bb2c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7bac0b47d0 0x7f7bac143e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:32.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.157+0000 7f7bb2c00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f7bac0729b0 con 0x7f7b90004d40 2024-04-20T11:37:32.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.157+0000 7f7bb2c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7bac06e680 con 0x7f7bac0b47d0 2024-04-20T11:37:32.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.157+0000 7f7bb2c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7bac072b30 con 0x7f7bac142800 2024-04-20T11:37:32.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.157+0000 7f7babe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7b90004d40 0x7f7bac140f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:32.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.157+0000 7f7babe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7b90004d40 0x7f7bac140f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36410/0 (socket says 172.21.15.152:36410) 2024-04-20T11:37:32.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.157+0000 7f7babe00640 1 -- 172.21.15.152:0/2330982591 learned_addr learned my addr 172.21.15.152:0/2330982591 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:32.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.157+0000 7f7babe00640 1 -- 172.21.15.152:0/2330982591 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7bac142800 msgr2=0x7f7bac0b4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:32.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.157+0000 7f7babe00640 1 --2- 172.21.15.152:0/2330982591 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7bac142800 0x7f7bac0b4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:32.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.157+0000 7f7babe00640 1 -- 172.21.15.152:0/2330982591 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7bac0b47d0 msgr2=0x7f7bac143e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:32.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.157+0000 7f7babe00640 1 --2- 172.21.15.152:0/2330982591 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7bac0b47d0 0x7f7bac143e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:32.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.157+0000 7f7babe00640 1 -- 172.21.15.152:0/2330982591 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bac1414c0 con 0x7f7b90004d40 2024-04-20T11:37:32.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.158+0000 7f7babe00640 1 --2- 172.21.15.152:0/2330982591 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7b90004d40 0x7f7bac140f80 secure :-1 s=READY pgs=2235 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c00dae0 tx=0x7f7b9c007e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:32.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.158+0000 7f7ba8c00640 1 -- 172.21.15.152:0/2330982591 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b9c019070 con 0x7f7b90004d40 2024-04-20T11:37:32.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.158+0000 7f7ba8c00640 1 -- 172.21.15.152:0/2330982591 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b9c0074e0 con 0x7f7b90004d40 2024-04-20T11:37:32.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.158+0000 7f7ba8c00640 1 -- 172.21.15.152:0/2330982591 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b9c0047e0 con 0x7f7b90004d40 2024-04-20T11:37:32.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.158+0000 7f7bb2c00640 1 -- 172.21.15.152:0/2330982591 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bac141760 con 0x7f7b90004d40 2024-04-20T11:37:32.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.158+0000 7f7bb2c00640 1 -- 172.21.15.152:0/2330982591 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bac141d20 con 0x7f7b90004d40 2024-04-20T11:37:32.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.159+0000 7f7bb2c00640 1 -- 172.21.15.152:0/2330982591 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bac0689f0 con 0x7f7b90004d40 2024-04-20T11:37:32.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.160+0000 7f7ba8c00640 1 -- 172.21.15.152:0/2330982591 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 26) ==== 91907+0+0 (secure 0 0 0) 0x7f7b9c004980 con 0x7f7b90004d40 2024-04-20T11:37:32.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.161+0000 7f7ba8c00640 1 --2- 172.21.15.152:0/2330982591 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f7b8806d2b0 0x7f7b8806f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:32.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.161+0000 7f7ba8c00640 1 -- 172.21.15.152:0/2330982591 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5767+0+0 (secure 0 0 0) 0x7f7b9c089d70 con 0x7f7b90004d40 2024-04-20T11:37:32.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.162+0000 7f7ba3400640 1 --2- 172.21.15.152:0/2330982591 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f7b8806d2b0 0x7f7b8806f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:32.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.163+0000 7f7ba3400640 1 --2- 172.21.15.152:0/2330982591 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f7b8806d2b0 0x7f7b8806f770 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f7bac0b5320 tx=0x7f7b94008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:32.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.166+0000 7f7ba8c00640 1 -- 172.21.15.152:0/2330982591 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f7bac1414c0 con 0x7f7b90004d40 2024-04-20T11:37:32.409 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.410+0000 7f7bb2c00640 1 -- 172.21.15.152:0/2330982591 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7bac0d0a20 con 0x7f7b90004d40 2024-04-20T11:37:32.411 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.411+0000 7f7ba8c00640 1 -- 172.21.15.152:0/2330982591 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 106 v106) ==== 95+0+5220 (secure 0 0 0) 0x7f7b9c059da0 con 0x7f7b90004d40 2024-04-20T11:37:32.411 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:37:32.411 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":106,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6458,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/1688411915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":1688411915},{"type":"v1","addr":"172.21.15.152:6835","nonce":1688411915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":100},{"gid":7141,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6834/599467453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":599467453},{"type":"v1","addr":"172.21.15.175:6834","nonce":599467453}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":100},{"gid":7144,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/691089259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6833","nonce":691089259},{"type":"v1","addr":"172.21.15.175:6835","nonce":691089259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":100}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T11:36:59.789831+0000","modified":"2024-04-20T11:37:02.829697+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10011},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10011":{"gid":10011,"name":"c","rank":0,"incarnation":101,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6833/1854325981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":1854325981},{"type":"v1","addr":"172.21.15.152:6833","nonce":1854325981}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10011,"qdb_cluster":[10011]},"id":12}]} 2024-04-20T11:37:32.411 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 106 2024-04-20T11:37:32.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.414+0000 7f7bb2c00640 1 -- 172.21.15.152:0/2330982591 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f7b8806d2b0 msgr2=0x7f7b8806f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:32.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.414+0000 7f7bb2c00640 1 --2- 172.21.15.152:0/2330982591 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f7b8806d2b0 0x7f7b8806f770 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f7bac0b5320 tx=0x7f7b94008040 comp rx=0 tx=0).stop 2024-04-20T11:37:32.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.414+0000 7f7bb2c00640 1 -- 172.21.15.152:0/2330982591 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7b90004d40 msgr2=0x7f7bac140f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:32.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.414+0000 7f7bb2c00640 1 --2- 172.21.15.152:0/2330982591 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7b90004d40 0x7f7bac140f80 secure :-1 s=READY pgs=2235 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c00dae0 tx=0x7f7b9c007e30 comp rx=0 tx=0).stop 2024-04-20T11:37:32.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.414+0000 7f7bb2c00640 1 -- 172.21.15.152:0/2330982591 shutdown_connections 2024-04-20T11:37:32.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.414+0000 7f7bb2c00640 1 --2- 172.21.15.152:0/2330982591 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f7b8806d2b0 0x7f7b8806f770 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:32.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.414+0000 7f7bb2c00640 1 --2- 172.21.15.152:0/2330982591 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7bac0b47d0 0x7f7bac143e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:32.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.414+0000 7f7bb2c00640 1 --2- 172.21.15.152:0/2330982591 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7bac142800 0x7f7bac0b4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:32.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.414+0000 7f7bb2c00640 1 --2- 172.21.15.152:0/2330982591 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7b90004d40 0x7f7bac140f80 unknown :-1 s=CLOSED pgs=2235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:32.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.414+0000 7f7bb2c00640 1 -- 172.21.15.152:0/2330982591 >> 172.21.15.152:0/2330982591 conn(0x7f7bac067cf0 msgr2=0x7f7bac1601a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:32.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.414+0000 7f7bb2c00640 1 -- 172.21.15.152:0/2330982591 shutdown_connections 2024-04-20T11:37:32.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.414+0000 7f7bb2c00640 1 -- 172.21.15.152:0/2330982591 wait complete. 2024-04-20T11:37:32.432 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:37:32.625 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.625+0000 7f541be00640 1 Processor -- start 2024-04-20T11:37:32.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.626+0000 7f541be00640 1 -- start start 2024-04-20T11:37:32.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.627+0000 7f541be00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f541c16b6a0 0x7f541c16da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:32.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.627+0000 7f541be00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f541c0b5510 con 0x7f541c06ea20 2024-04-20T11:37:32.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.627+0000 7f541be00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f541c06df40 con 0x7f541c071cd0 2024-04-20T11:37:32.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.627+0000 7f541be00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f541c0ceec0 con 0x7f541c16b6a0 2024-04-20T11:37:32.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.627+0000 7f5412000640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f541c06ea20 0x7f541c0b4a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:40820/0 (socket says 172.21.15.152:40820) 2024-04-20T11:37:32.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.628+0000 7f5412000640 1 -- 172.21.15.152:0/1956345869 learned_addr learned my addr 172.21.15.152:0/1956345869 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:37:32.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.628+0000 7f541b400640 1 --2- 172.21.15.152:0/1956345869 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f541c16b6a0 0x7f541c16da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:32.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.628+0000 7f541a000640 1 -- 172.21.15.152:0/1956345869 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2965700704 0 0) 0x7f541c0b5510 con 0x7f541c06ea20 2024-04-20T11:37:32.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.628+0000 7f541a000640 1 -- 172.21.15.152:0/1956345869 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5400003520 con 0x7f541c06ea20 2024-04-20T11:37:32.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.628+0000 7f541a000640 1 -- 172.21.15.152:0/1956345869 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1107773842 0 0) 0x7f5400003520 con 0x7f541c06ea20 2024-04-20T11:37:32.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.628+0000 7f541a000640 1 -- 172.21.15.152:0/1956345869 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f541c16b6a0 msgr2=0x7f541c16da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:32.632 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.628+0000 7f541a000640 1 --2- 172.21.15.152:0/1956345869 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f541c16b6a0 0x7f541c16da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:32.632 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.628+0000 7f541a000640 1 -- 172.21.15.152:0/1956345869 >> v1:172.21.15.175:6789/0 conn(0x7f541c071cd0 legacy=0x7f541c0720b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:32.632 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.628+0000 7f541a000640 1 -- 172.21.15.152:0/1956345869 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f541c06e220 con 0x7f541c06ea20 2024-04-20T11:37:32.632 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.628+0000 7f541a000640 1 -- 172.21.15.152:0/1956345869 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f5404001670 con 0x7f541c06ea20 2024-04-20T11:37:32.632 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.628+0000 7f541a000640 1 -- 172.21.15.152:0/1956345869 >> v1:172.21.15.152:6789/0 conn(0x7f541c06ea20 legacy=0x7f541c0b4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:32.632 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.628+0000 7f541a000640 1 --2- 172.21.15.152:0/1956345869 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f541c16b6a0 0x7f54000039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:32.632 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.628+0000 7f541a000640 1 --2- 172.21.15.152:0/1956345869 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5400004010 0x7f5400004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:32.632 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.628+0000 7f541a000640 1 --2- 172.21.15.152:0/1956345869 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5400004d40 0x7f54000051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:32.632 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.628+0000 7f541a000640 1 -- 172.21.15.152:0/1956345869 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f541c0ceec0 con 0x7f5400004010 2024-04-20T11:37:32.632 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.628+0000 7f541a000640 1 -- 172.21.15.152:0/1956345869 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f540000bb80 con 0x7f5400004d40 2024-04-20T11:37:32.632 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.628+0000 7f541a000640 1 -- 172.21.15.152:0/1956345869 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f540000bd80 con 0x7f541c16b6a0 2024-04-20T11:37:32.632 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.629+0000 7f5412000640 1 --2- 172.21.15.152:0/1956345869 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5400004010 0x7f5400004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:32.632 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.629+0000 7f541b400640 1 --2- 172.21.15.152:0/1956345869 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5400004d40 0x7f54000051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:32.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.629+0000 7f5412000640 1 -- 172.21.15.152:0/1956345869 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f541c16b6a0 msgr2=0x7f54000039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:32.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.629+0000 7f5412000640 1 --2- 172.21.15.152:0/1956345869 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f541c16b6a0 0x7f54000039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:32.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.629+0000 7f5412000640 1 -- 172.21.15.152:0/1956345869 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5400004d40 msgr2=0x7f54000051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:32.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.629+0000 7f5412000640 1 --2- 172.21.15.152:0/1956345869 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5400004d40 0x7f54000051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:32.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.629+0000 7f5412000640 1 -- 172.21.15.152:0/1956345869 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5400003520 con 0x7f5400004010 2024-04-20T11:37:32.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.630+0000 7f5412000640 1 --2- 172.21.15.152:0/1956345869 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5400004010 0x7f5400004800 secure :-1 s=READY pgs=2237 cs=0 l=1 rev1=1 crypto rx=0x7f540400c240 tx=0x7f5400005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=399e9357cd3d5dd7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:32.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.630+0000 7f541a000640 1 -- 172.21.15.152:0/1956345869 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5404007cf0 con 0x7f5400004010 2024-04-20T11:37:32.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.631+0000 7f541a000640 1 -- 172.21.15.152:0/1956345869 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5404030070 con 0x7f5400004010 2024-04-20T11:37:32.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.631+0000 7f541a000640 1 -- 172.21.15.152:0/1956345869 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5404005d70 con 0x7f5400004010 2024-04-20T11:37:32.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.631+0000 7f541be00640 1 -- 172.21.15.152:0/1956345869 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5400004010 msgr2=0x7f5400004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:32.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.631+0000 7f541be00640 1 --2- 172.21.15.152:0/1956345869 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5400004010 0x7f5400004800 secure :-1 s=READY pgs=2237 cs=0 l=1 rev1=1 crypto rx=0x7f540400c240 tx=0x7f5400005830 comp rx=0 tx=0).stop 2024-04-20T11:37:32.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.631+0000 7f541b400640 1 -- 172.21.15.152:0/1956345869 reap_dead start 2024-04-20T11:37:32.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.631+0000 7f541be00640 1 -- 172.21.15.152:0/1956345869 shutdown_connections 2024-04-20T11:37:32.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.631+0000 7f541be00640 1 -- 172.21.15.152:0/1956345869 >> 172.21.15.152:0/1956345869 conn(0x7f541c067ae0 msgr2=0x7f541c06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:32.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.631+0000 7f541be00640 1 -- 172.21.15.152:0/1956345869 shutdown_connections 2024-04-20T11:37:32.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.631+0000 7f541be00640 1 -- 172.21.15.152:0/1956345869 wait complete. 2024-04-20T11:37:32.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.633+0000 7f541be00640 1 Processor -- start 2024-04-20T11:37:32.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.633+0000 7f541be00640 1 -- start start 2024-04-20T11:37:32.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.634+0000 7f541be00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5400004010 0x7f541c160490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:32.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.634+0000 7f541be00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f541c144210 0x7f541c144670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:32.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.634+0000 7f541be00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f541c144bb0 0x7f541c15c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:32.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.634+0000 7f541be00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f541c163e60 con 0x7f541c144210 2024-04-20T11:37:32.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.634+0000 7f541be00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f541c06dcf0 con 0x7f541c144bb0 2024-04-20T11:37:32.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.634+0000 7f541be00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f541c0b4180 con 0x7f5400004010 2024-04-20T11:37:32.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.634+0000 7f5412000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f541c144210 0x7f541c144670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:32.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.634+0000 7f5412000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f541c144210 0x7f541c144670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36430/0 (socket says 172.21.15.152:36430) 2024-04-20T11:37:32.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.634+0000 7f5412000640 1 -- 172.21.15.152:0/2966598361 learned_addr learned my addr 172.21.15.152:0/2966598361 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:32.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.635+0000 7f541aa00640 1 --2- 172.21.15.152:0/2966598361 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5400004010 0x7f541c160490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:32.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.635+0000 7f541b400640 1 --2- 172.21.15.152:0/2966598361 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f541c144bb0 0x7f541c15c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:32.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.635+0000 7f5412000640 1 -- 172.21.15.152:0/2966598361 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5400004010 msgr2=0x7f541c160490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:32.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.635+0000 7f5412000640 1 --2- 172.21.15.152:0/2966598361 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5400004010 0x7f541c160490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:32.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.635+0000 7f5412000640 1 -- 172.21.15.152:0/2966598361 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f541c144bb0 msgr2=0x7f541c15c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:32.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.635+0000 7f5412000640 1 --2- 172.21.15.152:0/2966598361 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f541c144bb0 0x7f541c15c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:32.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.635+0000 7f5412000640 1 -- 172.21.15.152:0/2966598361 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f541c158b70 con 0x7f541c144210 2024-04-20T11:37:32.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.635+0000 7f5412000640 1 --2- 172.21.15.152:0/2966598361 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f541c144210 0x7f541c144670 secure :-1 s=READY pgs=2238 cs=0 l=1 rev1=1 crypto rx=0x7f540402c540 tx=0x7f541c06e3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:32.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.636+0000 7f5413400640 1 -- 172.21.15.152:0/2966598361 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5404045070 con 0x7f541c144210 2024-04-20T11:37:32.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.636+0000 7f5413400640 1 -- 172.21.15.152:0/2966598361 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5404000ed0 con 0x7f541c144210 2024-04-20T11:37:32.638 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.636+0000 7f541be00640 1 -- 172.21.15.152:0/2966598361 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f541c158e10 con 0x7f541c144210 2024-04-20T11:37:32.638 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.636+0000 7f5413400640 1 -- 172.21.15.152:0/2966598361 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5404004e10 con 0x7f541c144210 2024-04-20T11:37:32.638 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.636+0000 7f541be00640 1 -- 172.21.15.152:0/2966598361 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f541c1593d0 con 0x7f541c144210 2024-04-20T11:37:32.638 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.637+0000 7f541be00640 1 -- 172.21.15.152:0/2966598361 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f541c075e60 con 0x7f541c144210 2024-04-20T11:37:32.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.638+0000 7f5413400640 1 -- 172.21.15.152:0/2966598361 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 26) ==== 91907+0+0 (secure 0 0 0) 0x7f5404007070 con 0x7f541c144210 2024-04-20T11:37:32.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.639+0000 7f5413400640 1 --2- 172.21.15.152:0/2966598361 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f53f806d280 0x7f53f806f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:32.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.639+0000 7f5413400640 1 -- 172.21.15.152:0/2966598361 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5767+0+0 (secure 0 0 0) 0x7f54040ab490 con 0x7f541c144210 2024-04-20T11:37:32.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.640+0000 7f541aa00640 1 --2- 172.21.15.152:0/2966598361 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f53f806d280 0x7f53f806f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:32.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.641+0000 7f541aa00640 1 --2- 172.21.15.152:0/2966598361 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f53f806d280 0x7f53f806f740 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f540c000f20 tx=0x7f540c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:32.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.644+0000 7f5413400640 1 -- 172.21.15.152:0/2966598361 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f540407ade0 con 0x7f541c144210 2024-04-20T11:37:32.845 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.845+0000 7f541be00640 1 -- 172.21.15.152:0/2966598361 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f541c072530 con 0x7f541c144210 2024-04-20T11:37:32.846 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.847+0000 7f5413400640 1 -- 172.21.15.152:0/2966598361 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v156) ==== 74+0+16781 (secure 0 0 0) 0x7f540407f0e0 con 0x7f541c144210 2024-04-20T11:37:32.847 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:37:32.847 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":156,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:37:24.837045+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:36:57.520028+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"153","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2024-04-20T11:36:58.760637+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"154","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"25.13","mappings":[{"from":0,"to":1}]},{"pgid":"25.15","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:37:32.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.849+0000 7f541be00640 1 -- 172.21.15.152:0/2966598361 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f53f806d280 msgr2=0x7f53f806f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:32.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.849+0000 7f541be00640 1 --2- 172.21.15.152:0/2966598361 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f53f806d280 0x7f53f806f740 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f540c000f20 tx=0x7f540c009040 comp rx=0 tx=0).stop 2024-04-20T11:37:32.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.849+0000 7f541be00640 1 -- 172.21.15.152:0/2966598361 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f541c144210 msgr2=0x7f541c144670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:32.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.849+0000 7f541be00640 1 --2- 172.21.15.152:0/2966598361 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f541c144210 0x7f541c144670 secure :-1 s=READY pgs=2238 cs=0 l=1 rev1=1 crypto rx=0x7f540402c540 tx=0x7f541c06e3f0 comp rx=0 tx=0).stop 2024-04-20T11:37:32.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.850+0000 7f541be00640 1 -- 172.21.15.152:0/2966598361 shutdown_connections 2024-04-20T11:37:32.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.850+0000 7f541be00640 1 --2- 172.21.15.152:0/2966598361 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f53f806d280 0x7f53f806f740 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:32.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.850+0000 7f541be00640 1 --2- 172.21.15.152:0/2966598361 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f541c144bb0 0x7f541c15c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:32.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.850+0000 7f541be00640 1 --2- 172.21.15.152:0/2966598361 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f541c144210 0x7f541c144670 unknown :-1 s=CLOSED pgs=2238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:32.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.850+0000 7f541be00640 1 --2- 172.21.15.152:0/2966598361 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5400004010 0x7f541c160490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:32.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.850+0000 7f541be00640 1 -- 172.21.15.152:0/2966598361 >> 172.21.15.152:0/2966598361 conn(0x7f541c067ae0 msgr2=0x7f541c15aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:32.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.850+0000 7f541be00640 1 -- 172.21.15.152:0/2966598361 shutdown_connections 2024-04-20T11:37:32.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:32.850+0000 7f541be00640 1 -- 172.21.15.152:0/2966598361 wait complete. 2024-04-20T11:37:32.870 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-20T11:37:32.870 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:37:33.064 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.064+0000 7f72daa00640 1 Processor -- start 2024-04-20T11:37:33.064 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.064+0000 7f72daa00640 1 -- start start 2024-04-20T11:37:33.065 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.066+0000 7f72daa00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f72d406fa50 0x7f72d415cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:33.065 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.066+0000 7f72daa00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72d40b5770 con 0x7f72d415b5e0 2024-04-20T11:37:33.065 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.066+0000 7f72daa00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72d406e060 con 0x7f72d406eb40 2024-04-20T11:37:33.066 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.066+0000 7f72daa00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f72d406de10 con 0x7f72d406fa50 2024-04-20T11:37:33.066 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.066+0000 7f72d3e00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f72d415b5e0 0x7f72d40b4c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:40828/0 (socket says 172.21.15.152:40828) 2024-04-20T11:37:33.066 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.066+0000 7f72d3e00640 1 -- 172.21.15.152:0/1268636229 learned_addr learned my addr 172.21.15.152:0/1268636229 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:37:33.068 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.066+0000 7f72d8a00640 1 --2- 172.21.15.152:0/1268636229 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f72d406fa50 0x7f72d415cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:33.068 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.066+0000 7f72d3400640 1 -- 172.21.15.152:0/1268636229 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1311268425 0 0) 0x7f72d40b5770 con 0x7f72d415b5e0 2024-04-20T11:37:33.068 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.067+0000 7f72d3400640 1 -- 172.21.15.152:0/1268636229 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f72b8003520 con 0x7f72d415b5e0 2024-04-20T11:37:33.068 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.067+0000 7f72d3400640 1 -- 172.21.15.152:0/1268636229 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2295732924 0 0) 0x7f72b8003520 con 0x7f72d415b5e0 2024-04-20T11:37:33.068 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.067+0000 7f72d3400640 1 -- 172.21.15.152:0/1268636229 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f72d406fa50 msgr2=0x7f72d415cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:33.068 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.067+0000 7f72d3400640 1 --2- 172.21.15.152:0/1268636229 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f72d406fa50 0x7f72d415cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:33.069 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.067+0000 7f72d3400640 1 -- 172.21.15.152:0/1268636229 >> v1:172.21.15.175:6789/0 conn(0x7f72d406eb40 legacy=0x7f72d406f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:33.069 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.067+0000 7f72d3400640 1 -- 172.21.15.152:0/1268636229 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72d406e340 con 0x7f72d415b5e0 2024-04-20T11:37:33.069 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.067+0000 7f72d3400640 1 -- 172.21.15.152:0/1268636229 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f72c4001690 con 0x7f72d415b5e0 2024-04-20T11:37:33.069 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.067+0000 7f72d3400640 1 -- 172.21.15.152:0/1268636229 >> v1:172.21.15.152:6789/0 conn(0x7f72d415b5e0 legacy=0x7f72d40b4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:33.069 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.067+0000 7f72d3400640 1 --2- 172.21.15.152:0/1268636229 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f72b8003a50 0x7f72b8003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:33.069 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.067+0000 7f72d3400640 1 --2- 172.21.15.152:0/1268636229 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72b8004800 0x7f72b8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:33.069 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.067+0000 7f72d3400640 1 --2- 172.21.15.152:0/1268636229 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f72d406fa50 0x7f72b80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:33.069 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.067+0000 7f72d3400640 1 -- 172.21.15.152:0/1268636229 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f72d406de10 con 0x7f72b8003a50 2024-04-20T11:37:33.069 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.067+0000 7f72d3400640 1 -- 172.21.15.152:0/1268636229 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f72b800dd00 con 0x7f72b8004800 2024-04-20T11:37:33.069 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.067+0000 7f72d3400640 1 -- 172.21.15.152:0/1268636229 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f72b800df00 con 0x7f72d406fa50 2024-04-20T11:37:33.069 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.067+0000 7f72d3e00640 1 --2- 172.21.15.152:0/1268636229 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f72b8003a50 0x7f72b8003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:33.069 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.068+0000 7f72d8a00640 1 --2- 172.21.15.152:0/1268636229 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f72d406fa50 0x7f72b80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:33.069 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.068+0000 7f72cbe00640 1 --2- 172.21.15.152:0/1268636229 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72b8004800 0x7f72b8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:33.070 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.068+0000 7f72d3e00640 1 -- 172.21.15.152:0/1268636229 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f72d406fa50 msgr2=0x7f72b80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:33.070 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.068+0000 7f72d3e00640 1 --2- 172.21.15.152:0/1268636229 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f72d406fa50 0x7f72b80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:33.070 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.068+0000 7f72d3e00640 1 -- 172.21.15.152:0/1268636229 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72b8004800 msgr2=0x7f72b8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:33.070 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.068+0000 7f72d3e00640 1 --2- 172.21.15.152:0/1268636229 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72b8004800 0x7f72b8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:33.070 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.068+0000 7f72d3e00640 1 -- 172.21.15.152:0/1268636229 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72b8003520 con 0x7f72b8003a50 2024-04-20T11:37:33.071 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.069+0000 7f72d3e00640 1 --2- 172.21.15.152:0/1268636229 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f72b8003a50 0x7f72b8003e70 secure :-1 s=READY pgs=2240 cs=0 l=1 rev1=1 crypto rx=0x7f72c400c210 tx=0x7f72b8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d400597b760d2c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:33.071 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.069+0000 7f72d3400640 1 -- 172.21.15.152:0/1268636229 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c4007d30 con 0x7f72b8003a50 2024-04-20T11:37:33.071 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.069+0000 7f72d3400640 1 -- 172.21.15.152:0/1268636229 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72c4030070 con 0x7f72b8003a50 2024-04-20T11:37:33.071 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.069+0000 7f72d3400640 1 -- 172.21.15.152:0/1268636229 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c4005d10 con 0x7f72b8003a50 2024-04-20T11:37:33.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.070+0000 7f72daa00640 1 -- 172.21.15.152:0/1268636229 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f72b8003a50 msgr2=0x7f72b8003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:33.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.070+0000 7f72daa00640 1 --2- 172.21.15.152:0/1268636229 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f72b8003a50 0x7f72b8003e70 secure :-1 s=READY pgs=2240 cs=0 l=1 rev1=1 crypto rx=0x7f72c400c210 tx=0x7f72b8005830 comp rx=0 tx=0).stop 2024-04-20T11:37:33.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.070+0000 7f72d8a00640 1 -- 172.21.15.152:0/1268636229 reap_dead start 2024-04-20T11:37:33.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.070+0000 7f72daa00640 1 -- 172.21.15.152:0/1268636229 shutdown_connections 2024-04-20T11:37:33.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.070+0000 7f72daa00640 1 -- 172.21.15.152:0/1268636229 >> 172.21.15.152:0/1268636229 conn(0x7f72d4067cf0 msgr2=0x7f72d406b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:33.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.070+0000 7f72daa00640 1 -- 172.21.15.152:0/1268636229 shutdown_connections 2024-04-20T11:37:33.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.070+0000 7f72daa00640 1 -- 172.21.15.152:0/1268636229 wait complete. 2024-04-20T11:37:33.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.072+0000 7f72daa00640 1 Processor -- start 2024-04-20T11:37:33.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.072+0000 7f72daa00640 1 -- start start 2024-04-20T11:37:33.074 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.073+0000 7f72daa00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f72b8003a50 0x7f72d40aeba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:33.074 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.073+0000 7f72daa00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f72d41557f0 0x7f72d40cbbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:33.074 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.073+0000 7f72daa00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72d415d460 0x7f72d415d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:33.074 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.073+0000 7f72daa00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f72d40b42d0 con 0x7f72d41557f0 2024-04-20T11:37:33.074 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.073+0000 7f72daa00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f72d415c9d0 con 0x7f72d415d460 2024-04-20T11:37:33.074 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.073+0000 7f72daa00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f72d40b5360 con 0x7f72b8003a50 2024-04-20T11:37:33.074 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.073+0000 7f72cbe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f72d41557f0 0x7f72d40cbbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:33.074 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.073+0000 7f72cbe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f72d41557f0 0x7f72d40cbbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36452/0 (socket says 172.21.15.152:36452) 2024-04-20T11:37:33.074 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.073+0000 7f72cbe00640 1 -- 172.21.15.152:0/537254057 learned_addr learned my addr 172.21.15.152:0/537254057 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:33.075 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.074+0000 7f72d3e00640 1 --2- 172.21.15.152:0/537254057 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f72b8003a50 0x7f72d40aeba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:33.075 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.074+0000 7f72d8a00640 1 --2- 172.21.15.152:0/537254057 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72d415d460 0x7f72d415d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:33.075 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.074+0000 7f72cbe00640 1 -- 172.21.15.152:0/537254057 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f72b8003a50 msgr2=0x7f72d40aeba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:33.075 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.074+0000 7f72cbe00640 1 --2- 172.21.15.152:0/537254057 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f72b8003a50 0x7f72d40aeba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:33.075 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.074+0000 7f72cbe00640 1 -- 172.21.15.152:0/537254057 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72d415d460 msgr2=0x7f72d415d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:33.075 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.074+0000 7f72cbe00640 1 --2- 172.21.15.152:0/537254057 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72d415d460 0x7f72d415d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:33.075 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.074+0000 7f72cbe00640 1 -- 172.21.15.152:0/537254057 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72d4155d90 con 0x7f72d41557f0 2024-04-20T11:37:33.075 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.074+0000 7f72cbe00640 1 --2- 172.21.15.152:0/537254057 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f72d41557f0 0x7f72d40cbbc0 secure :-1 s=READY pgs=2241 cs=0 l=1 rev1=1 crypto rx=0x7f72bc00e4d0 tx=0x7f72bc00e990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:33.075 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.074+0000 7f72d0c00640 1 -- 172.21.15.152:0/537254057 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72bc0052e0 con 0x7f72d41557f0 2024-04-20T11:37:33.075 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.074+0000 7f72d0c00640 1 -- 172.21.15.152:0/537254057 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72bc018070 con 0x7f72d41557f0 2024-04-20T11:37:33.075 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.074+0000 7f72d0c00640 1 -- 172.21.15.152:0/537254057 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72bc003810 con 0x7f72d41557f0 2024-04-20T11:37:33.075 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.074+0000 7f72daa00640 1 -- 172.21.15.152:0/537254057 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72d4156070 con 0x7f72d41557f0 2024-04-20T11:37:33.075 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.075+0000 7f72daa00640 1 -- 172.21.15.152:0/537254057 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72d4156630 con 0x7f72d41557f0 2024-04-20T11:37:33.076 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.076+0000 7f72daa00640 1 -- 172.21.15.152:0/537254057 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72d40689f0 con 0x7f72d41557f0 2024-04-20T11:37:33.076 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.077+0000 7f72d0c00640 1 -- 172.21.15.152:0/537254057 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 26) ==== 91907+0+0 (secure 0 0 0) 0x7f72bc0039b0 con 0x7f72d41557f0 2024-04-20T11:37:33.077 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.078+0000 7f72d0c00640 1 --2- 172.21.15.152:0/537254057 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f72b006d280 0x7f72b006f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:33.077 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.078+0000 7f72d0c00640 1 -- 172.21.15.152:0/537254057 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5767+0+0 (secure 0 0 0) 0x7f72bc089fd0 con 0x7f72d41557f0 2024-04-20T11:37:33.078 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.078+0000 7f72d3e00640 1 --2- 172.21.15.152:0/537254057 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f72b006d280 0x7f72b006f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:33.079 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.079+0000 7f72d3e00640 1 --2- 172.21.15.152:0/537254057 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f72b006d280 0x7f72b006f740 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f72c4034000 tx=0x7f72c4007000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:33.081 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.082+0000 7f72d0c00640 1 -- 172.21.15.152:0/537254057 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f72bc05a000 con 0x7f72d41557f0 2024-04-20T11:37:33.324 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.325+0000 7f72daa00640 1 -- 172.21.15.152:0/537254057 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f72d40afc90 con 0x7f72d41557f0 2024-04-20T11:37:33.423 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.326+0000 7f72d0c00640 1 -- 172.21.15.152:0/537254057 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 106 v106) ==== 95+0+5220 (secure 0 0 0) 0x7f72bc059900 con 0x7f72d41557f0 2024-04-20T11:37:33.423 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:37:33.423 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":106,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6458,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/1688411915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":1688411915},{"type":"v1","addr":"172.21.15.152:6835","nonce":1688411915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":100},{"gid":7141,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6834/599467453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":599467453},{"type":"v1","addr":"172.21.15.175:6834","nonce":599467453}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":100},{"gid":7144,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/691089259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6833","nonce":691089259},{"type":"v1","addr":"172.21.15.175:6835","nonce":691089259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":100}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T11:36:59.789831+0000","modified":"2024-04-20T11:37:02.829697+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10011},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10011":{"gid":10011,"name":"c","rank":0,"incarnation":101,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6833/1854325981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":1854325981},{"type":"v1","addr":"172.21.15.152:6833","nonce":1854325981}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10011,"qdb_cluster":[10011]},"id":12}]} 2024-04-20T11:37:33.425 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 106 2024-04-20T11:37:33.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.329+0000 7f72daa00640 1 -- 172.21.15.152:0/537254057 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f72b006d280 msgr2=0x7f72b006f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:33.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.329+0000 7f72daa00640 1 --2- 172.21.15.152:0/537254057 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f72b006d280 0x7f72b006f740 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f72c4034000 tx=0x7f72c4007000 comp rx=0 tx=0).stop 2024-04-20T11:37:33.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.329+0000 7f72daa00640 1 -- 172.21.15.152:0/537254057 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f72d41557f0 msgr2=0x7f72d40cbbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:33.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.329+0000 7f72daa00640 1 --2- 172.21.15.152:0/537254057 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f72d41557f0 0x7f72d40cbbc0 secure :-1 s=READY pgs=2241 cs=0 l=1 rev1=1 crypto rx=0x7f72bc00e4d0 tx=0x7f72bc00e990 comp rx=0 tx=0).stop 2024-04-20T11:37:33.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.329+0000 7f72daa00640 1 -- 172.21.15.152:0/537254057 shutdown_connections 2024-04-20T11:37:33.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.329+0000 7f72daa00640 1 --2- 172.21.15.152:0/537254057 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f72b006d280 0x7f72b006f740 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:33.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.329+0000 7f72daa00640 1 --2- 172.21.15.152:0/537254057 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72d415d460 0x7f72d415d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:33.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.329+0000 7f72daa00640 1 --2- 172.21.15.152:0/537254057 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f72d41557f0 0x7f72d40cbbc0 unknown :-1 s=CLOSED pgs=2241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:33.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.329+0000 7f72daa00640 1 --2- 172.21.15.152:0/537254057 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f72b8003a50 0x7f72d40aeba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:33.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.329+0000 7f72daa00640 1 -- 172.21.15.152:0/537254057 >> 172.21.15.152:0/537254057 conn(0x7f72d4067cf0 msgr2=0x7f72d4071ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:33.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.329+0000 7f72daa00640 1 -- 172.21.15.152:0/537254057 shutdown_connections 2024-04-20T11:37:33.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.329+0000 7f72daa00640 1 -- 172.21.15.152:0/537254057 wait complete. 2024-04-20T11:37:33.426 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:37:33.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.635+0000 7fbfd3c00640 1 Processor -- start 2024-04-20T11:37:33.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.636+0000 7fbfd3c00640 1 -- start start 2024-04-20T11:37:33.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.637+0000 7fbfd3c00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fbfcc0d0c90 0x7fbfcc15d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:33.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.637+0000 7fbfd3c00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fbfcc0cd030 0x7fbfcc0cd480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:33.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.637+0000 7fbfd3c00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbfcc09adc0 con 0x7fbfcc15c970 2024-04-20T11:37:33.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.637+0000 7fbfd3c00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fbfcc15cd50 con 0x7fbfcc0d0c90 2024-04-20T11:37:33.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.637+0000 7fbfd3c00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fbfcc15de20 con 0x7fbfcc0cd030 2024-04-20T11:37:33.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.638+0000 7fbfcbe00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fbfcc0d0c90 0x7fbfcc15d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:33.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.638+0000 7fbfcbe00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fbfcc0d0c90 0x7fbfcc15d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36468/0 (socket says 172.21.15.152:36468) 2024-04-20T11:37:33.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.638+0000 7fbfcbe00640 1 -- 172.21.15.152:0/2084359114 learned_addr learned my addr 172.21.15.152:0/2084359114 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:33.638 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.638+0000 7fbfd1c00640 1 --2- 172.21.15.152:0/2084359114 >> v2:172.21.15.175:3300/0 conn(0x7fbfcc0cd030 0x7fbfcc0cd480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:33.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.638+0000 7fbfcbe00640 1 -- 172.21.15.152:0/2084359114 >> v2:172.21.15.175:3300/0 conn(0x7fbfcc0cd030 msgr2=0x7fbfcc0cd480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:33.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.638+0000 7fbfcbe00640 1 --2- 172.21.15.152:0/2084359114 >> v2:172.21.15.175:3300/0 conn(0x7fbfcc0cd030 0x7fbfcc0cd480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:33.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.638+0000 7fbfcbe00640 1 -- 172.21.15.152:0/2084359114 >> v1:172.21.15.175:6789/0 conn(0x7fbfcc15c970 legacy=0x7fbfcc0d0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:33.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.638+0000 7fbfcbe00640 1 -- 172.21.15.152:0/2084359114 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfcc09b0a0 con 0x7fbfcc0d0c90 2024-04-20T11:37:33.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.638+0000 7fbfd1c00640 1 --2- 172.21.15.152:0/2084359114 >> v2:172.21.15.175:3300/0 conn(0x7fbfcc0cd030 0x7fbfcc0cd480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:37:33.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.639+0000 7fbfcbe00640 1 --2- 172.21.15.152:0/2084359114 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbfcc0d0c90 0x7fbfcc15d8e0 secure :-1 s=READY pgs=2242 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc0ccde0 tx=0x7fbfb802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96b6d98d22306fe5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:33.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.639+0000 7fbfcb400640 1 -- 172.21.15.152:0/2084359114 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb8035020 con 0x7fbfcc0d0c90 2024-04-20T11:37:33.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.640+0000 7fbfcb400640 1 -- 172.21.15.152:0/2084359114 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfb8031870 con 0x7fbfcc0d0c90 2024-04-20T11:37:33.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.640+0000 7fbfcb400640 1 -- 172.21.15.152:0/2084359114 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb8031c40 con 0x7fbfcc0d0c90 2024-04-20T11:37:33.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.640+0000 7fbfd3c00640 1 -- 172.21.15.152:0/2084359114 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbfcc0d0c90 msgr2=0x7fbfcc15d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:33.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.640+0000 7fbfd3c00640 1 --2- 172.21.15.152:0/2084359114 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbfcc0d0c90 0x7fbfcc15d8e0 secure :-1 s=READY pgs=2242 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc0ccde0 tx=0x7fbfb802b910 comp rx=0 tx=0).stop 2024-04-20T11:37:33.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.640+0000 7fbfd3c00640 1 -- 172.21.15.152:0/2084359114 shutdown_connections 2024-04-20T11:37:33.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.640+0000 7fbfd3c00640 1 --2- 172.21.15.152:0/2084359114 >> v2:172.21.15.175:3300/0 conn(0x7fbfcc0cd030 0x7fbfcc0cd480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:33.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.640+0000 7fbfd3c00640 1 --2- 172.21.15.152:0/2084359114 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbfcc0d0c90 0x7fbfcc15d8e0 unknown :-1 s=CLOSED pgs=2242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:33.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.640+0000 7fbfd3c00640 1 -- 172.21.15.152:0/2084359114 >> 172.21.15.152:0/2084359114 conn(0x7fbfcc0b4050 msgr2=0x7fbfcc0b4460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:33.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.640+0000 7fbfd3c00640 1 -- 172.21.15.152:0/2084359114 shutdown_connections 2024-04-20T11:37:33.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.640+0000 7fbfd3c00640 1 -- 172.21.15.152:0/2084359114 wait complete. 2024-04-20T11:37:33.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.642+0000 7fbfd3c00640 1 Processor -- start 2024-04-20T11:37:33.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.642+0000 7fbfd3c00640 1 -- start start 2024-04-20T11:37:33.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.643+0000 7fbfd3c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbfcc0cd030 0x7fbfcc0c6a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:33.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.643+0000 7fbfd3c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbfcc0d0c90 0x7fbfcc0c8600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:33.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.643+0000 7fbfd3c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbfcc15c970 0x7fbfcc0c8b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:33.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.643+0000 7fbfd3c00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fbfcc09b940 con 0x7fbfcc15c970 2024-04-20T11:37:33.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.643+0000 7fbfd3c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbfcc09b660 con 0x7fbfcc0d0c90 2024-04-20T11:37:33.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.643+0000 7fbfd3c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fbfcc09aa40 con 0x7fbfcc0cd030 2024-04-20T11:37:33.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.643+0000 7fbfd1c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbfcc15c970 0x7fbfcc0c8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:33.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.644+0000 7fbfd1c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbfcc15c970 0x7fbfcc0c8b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36476/0 (socket says 172.21.15.152:36476) 2024-04-20T11:37:33.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.644+0000 7fbfd1c00640 1 -- 172.21.15.152:0/843775000 learned_addr learned my addr 172.21.15.152:0/843775000 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:33.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.644+0000 7fbfcbe00640 1 --2- 172.21.15.152:0/843775000 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbfcc0d0c90 0x7fbfcc0c8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:33.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.644+0000 7fbfd1200640 1 --2- 172.21.15.152:0/843775000 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbfcc0cd030 0x7fbfcc0c6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:33.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.644+0000 7fbfd1c00640 1 -- 172.21.15.152:0/843775000 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbfcc0cd030 msgr2=0x7fbfcc0c6a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:33.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.644+0000 7fbfd1c00640 1 --2- 172.21.15.152:0/843775000 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbfcc0cd030 0x7fbfcc0c6a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:33.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.644+0000 7fbfd1c00640 1 -- 172.21.15.152:0/843775000 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbfcc0d0c90 msgr2=0x7fbfcc0c8600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:33.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.644+0000 7fbfd1c00640 1 --2- 172.21.15.152:0/843775000 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbfcc0d0c90 0x7fbfcc0c8600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:33.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.644+0000 7fbfd1c00640 1 -- 172.21.15.152:0/843775000 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfcc1f4ce0 con 0x7fbfcc15c970 2024-04-20T11:37:33.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.644+0000 7fbfd1c00640 1 --2- 172.21.15.152:0/843775000 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbfcc15c970 0x7fbfcc0c8b40 secure :-1 s=READY pgs=2243 cs=0 l=1 rev1=1 crypto rx=0x7fbfc400cef0 tx=0x7fbfc400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:33.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.645+0000 7fbfc8c00640 1 -- 172.21.15.152:0/843775000 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc401a070 con 0x7fbfcc15c970 2024-04-20T11:37:33.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.645+0000 7fbfc8c00640 1 -- 172.21.15.152:0/843775000 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfc400d860 con 0x7fbfcc15c970 2024-04-20T11:37:33.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.645+0000 7fbfc8c00640 1 -- 172.21.15.152:0/843775000 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc400dc50 con 0x7fbfcc15c970 2024-04-20T11:37:33.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.645+0000 7fbfd3c00640 1 -- 172.21.15.152:0/843775000 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfcc1e3dc0 con 0x7fbfcc15c970 2024-04-20T11:37:33.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.645+0000 7fbfd3c00640 1 -- 172.21.15.152:0/843775000 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfcc1f4f70 con 0x7fbfcc15c970 2024-04-20T11:37:33.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.646+0000 7fbfd3c00640 1 -- 172.21.15.152:0/843775000 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfcc06c6a0 con 0x7fbfcc15c970 2024-04-20T11:37:33.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.647+0000 7fbfc8c00640 1 -- 172.21.15.152:0/843775000 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 26) ==== 91907+0+0 (secure 0 0 0) 0x7fbfc401d070 con 0x7fbfcc15c970 2024-04-20T11:37:33.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.648+0000 7fbfc8c00640 1 --2- 172.21.15.152:0/843775000 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fbfac06d2b0 0x7fbfac06f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:33.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.648+0000 7fbfc8c00640 1 -- 172.21.15.152:0/843775000 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5767+0+0 (secure 0 0 0) 0x7fbfc408ab60 con 0x7fbfcc15c970 2024-04-20T11:37:33.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.649+0000 7fbfd1200640 1 --2- 172.21.15.152:0/843775000 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fbfac06d2b0 0x7fbfac06f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:33.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.650+0000 7fbfd1200640 1 --2- 172.21.15.152:0/843775000 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fbfac06d2b0 0x7fbfac06f770 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fbfc000a730 tx=0x7fbfc0008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:33.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.653+0000 7fbfc8c00640 1 -- 172.21.15.152:0/843775000 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fbfc405ab90 con 0x7fbfcc15c970 2024-04-20T11:37:33.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.897+0000 7fbfd3c00640 1 -- 172.21.15.152:0/843775000 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbfcc069100 con 0x7fbfcc15c970 2024-04-20T11:37:33.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.898+0000 7fbfc8c00640 1 -- 172.21.15.152:0/843775000 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 106 v106) ==== 95+0+5220 (secure 0 0 0) 0x7fbfc405a490 con 0x7fbfcc15c970 2024-04-20T11:37:33.898 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:37:33.898 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":106,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6458,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/1688411915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":1688411915},{"type":"v1","addr":"172.21.15.152:6835","nonce":1688411915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":100},{"gid":7141,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6834/599467453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":599467453},{"type":"v1","addr":"172.21.15.175:6834","nonce":599467453}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":100},{"gid":7144,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/691089259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6833","nonce":691089259},{"type":"v1","addr":"172.21.15.175:6835","nonce":691089259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":100}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T11:36:59.789831+0000","modified":"2024-04-20T11:37:02.829697+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10011},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10011":{"gid":10011,"name":"c","rank":0,"incarnation":101,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6833/1854325981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":1854325981},{"type":"v1","addr":"172.21.15.152:6833","nonce":1854325981}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10011,"qdb_cluster":[10011]},"id":12}]} 2024-04-20T11:37:33.898 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 106 2024-04-20T11:37:33.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.900+0000 7fbfd3c00640 1 -- 172.21.15.152:0/843775000 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fbfac06d2b0 msgr2=0x7fbfac06f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:33.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.900+0000 7fbfd3c00640 1 --2- 172.21.15.152:0/843775000 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fbfac06d2b0 0x7fbfac06f770 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fbfc000a730 tx=0x7fbfc0008040 comp rx=0 tx=0).stop 2024-04-20T11:37:33.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.900+0000 7fbfd3c00640 1 -- 172.21.15.152:0/843775000 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbfcc15c970 msgr2=0x7fbfcc0c8b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:33.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.900+0000 7fbfd3c00640 1 --2- 172.21.15.152:0/843775000 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbfcc15c970 0x7fbfcc0c8b40 secure :-1 s=READY pgs=2243 cs=0 l=1 rev1=1 crypto rx=0x7fbfc400cef0 tx=0x7fbfc400a710 comp rx=0 tx=0).stop 2024-04-20T11:37:33.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.901+0000 7fbfd3c00640 1 -- 172.21.15.152:0/843775000 shutdown_connections 2024-04-20T11:37:33.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.901+0000 7fbfd3c00640 1 --2- 172.21.15.152:0/843775000 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fbfac06d2b0 0x7fbfac06f770 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:33.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.901+0000 7fbfd3c00640 1 --2- 172.21.15.152:0/843775000 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbfcc15c970 0x7fbfcc0c8b40 unknown :-1 s=CLOSED pgs=2243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:33.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.901+0000 7fbfd3c00640 1 --2- 172.21.15.152:0/843775000 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbfcc0d0c90 0x7fbfcc0c8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:33.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.901+0000 7fbfd3c00640 1 --2- 172.21.15.152:0/843775000 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbfcc0cd030 0x7fbfcc0c6a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:33.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.901+0000 7fbfd3c00640 1 -- 172.21.15.152:0/843775000 >> 172.21.15.152:0/843775000 conn(0x7fbfcc0b4050 msgr2=0x7fbfcc0b5da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:33.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.901+0000 7fbfd3c00640 1 -- 172.21.15.152:0/843775000 shutdown_connections 2024-04-20T11:37:33.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:33.901+0000 7fbfd3c00640 1 -- 172.21.15.152:0/843775000 wait complete. 2024-04-20T11:37:33.921 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:37:34.116 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.116+0000 7f3f08c00640 1 Processor -- start 2024-04-20T11:37:34.116 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.116+0000 7f3f08c00640 1 -- start start 2024-04-20T11:37:34.118 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.117+0000 7f3f08c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3f04071cd0 0x7f3f040720b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:34.118 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.118+0000 7f3f08c00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f3f0406ea20 0x7f3f040b4a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:34.118 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.118+0000 7f3f08c00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f040725f0 con 0x7f3f040b4f90 2024-04-20T11:37:34.118 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.118+0000 7f3f08c00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f3f0406dcf0 con 0x7f3f0406ea20 2024-04-20T11:37:34.118 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.118+0000 7f3f08c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f3f040b5510 con 0x7f3f04071cd0 2024-04-20T11:37:34.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.118+0000 7f3f02a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f3f0406ea20 0x7f3f040b4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:34.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.118+0000 7f3f02a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f3f0406ea20 0x7f3f040b4a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36484/0 (socket says 172.21.15.152:36484) 2024-04-20T11:37:34.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.118+0000 7f3f02a00640 1 -- 172.21.15.152:0/916058900 learned_addr learned my addr 172.21.15.152:0/916058900 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:34.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.118+0000 7f3f02a00640 1 -- 172.21.15.152:0/916058900 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3f04071cd0 msgr2=0x7f3f040720b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:34.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.118+0000 7f3f02a00640 1 --2- 172.21.15.152:0/916058900 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3f04071cd0 0x7f3f040720b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:34.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.118+0000 7f3f02a00640 1 -- 172.21.15.152:0/916058900 >> v1:172.21.15.152:6789/0 conn(0x7f3f040b4f90 legacy=0x7f3f040cdeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:34.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.118+0000 7f3f02a00640 1 -- 172.21.15.152:0/916058900 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f0406df40 con 0x7f3f0406ea20 2024-04-20T11:37:34.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.119+0000 7f3f02a00640 1 --2- 172.21.15.152:0/916058900 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3f0406ea20 0x7f3f040b4a50 secure :-1 s=READY pgs=2245 cs=0 l=1 rev1=1 crypto rx=0x7f3eec009870 tx=0x7f3eec02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca9146bfe53613c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:34.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.120+0000 7f3f02000640 1 -- 172.21.15.152:0/916058900 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eec034070 con 0x7f3f0406ea20 2024-04-20T11:37:34.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.120+0000 7f3f02000640 1 -- 172.21.15.152:0/916058900 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3eec030840 con 0x7f3f0406ea20 2024-04-20T11:37:34.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.120+0000 7f3f02000640 1 -- 172.21.15.152:0/916058900 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eec030c10 con 0x7f3f0406ea20 2024-04-20T11:37:34.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.120+0000 7f3f08c00640 1 -- 172.21.15.152:0/916058900 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3f0406ea20 msgr2=0x7f3f040b4a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:34.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.120+0000 7f3f08c00640 1 --2- 172.21.15.152:0/916058900 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3f0406ea20 0x7f3f040b4a50 secure :-1 s=READY pgs=2245 cs=0 l=1 rev1=1 crypto rx=0x7f3eec009870 tx=0x7f3eec02b980 comp rx=0 tx=0).stop 2024-04-20T11:37:34.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.120+0000 7f3f08c00640 1 -- 172.21.15.152:0/916058900 shutdown_connections 2024-04-20T11:37:34.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.121+0000 7f3f08c00640 1 --2- 172.21.15.152:0/916058900 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3f0406ea20 0x7f3f040b4a50 unknown :-1 s=CLOSED pgs=2245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:34.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.121+0000 7f3f08c00640 1 --2- 172.21.15.152:0/916058900 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3f04071cd0 0x7f3f040720b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:34.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.121+0000 7f3f08c00640 1 -- 172.21.15.152:0/916058900 >> 172.21.15.152:0/916058900 conn(0x7f3f04067ae0 msgr2=0x7f3f0406b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:34.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.121+0000 7f3f08c00640 1 -- 172.21.15.152:0/916058900 shutdown_connections 2024-04-20T11:37:34.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.121+0000 7f3f08c00640 1 -- 172.21.15.152:0/916058900 wait complete. 2024-04-20T11:37:34.123 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.123+0000 7f3f08c00640 1 Processor -- start 2024-04-20T11:37:34.123 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.123+0000 7f3f08c00640 1 -- start start 2024-04-20T11:37:34.125 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.124+0000 7f3f08c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3f0406ea20 0x7f3f04150450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:34.125 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.124+0000 7f3f08c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3f04071cd0 0x7f3f040ceec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:34.125 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.124+0000 7f3f08c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3f040b4f90 0x7f3f040cf400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:34.125 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.124+0000 7f3f08c00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f3f04163e60 con 0x7f3f040b4f90 2024-04-20T11:37:34.125 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.124+0000 7f3f08c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3f0406e500 con 0x7f3f04071cd0 2024-04-20T11:37:34.125 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.124+0000 7f3f08c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f3f040b4180 con 0x7f3f0406ea20 2024-04-20T11:37:34.125 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.124+0000 7f3f03e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3f040b4f90 0x7f3f040cf400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:34.125 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.124+0000 7f3f03e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3f040b4f90 0x7f3f040cf400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36486/0 (socket says 172.21.15.152:36486) 2024-04-20T11:37:34.125 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.124+0000 7f3f03e00640 1 -- 172.21.15.152:0/1659826798 learned_addr learned my addr 172.21.15.152:0/1659826798 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:34.125 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.124+0000 7f3f02a00640 1 --2- 172.21.15.152:0/1659826798 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3f04071cd0 0x7f3f040ceec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:34.125 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.124+0000 7f3f03400640 1 --2- 172.21.15.152:0/1659826798 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3f0406ea20 0x7f3f04150450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:34.126 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.124+0000 7f3f03e00640 1 -- 172.21.15.152:0/1659826798 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3f0406ea20 msgr2=0x7f3f04150450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:34.126 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.124+0000 7f3f03e00640 1 --2- 172.21.15.152:0/1659826798 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3f0406ea20 0x7f3f04150450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:34.126 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.124+0000 7f3f03e00640 1 -- 172.21.15.152:0/1659826798 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3f04071cd0 msgr2=0x7f3f040ceec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:34.126 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.124+0000 7f3f03e00640 1 --2- 172.21.15.152:0/1659826798 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3f04071cd0 0x7f3f040ceec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:34.126 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.125+0000 7f3f03e00640 1 -- 172.21.15.152:0/1659826798 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f04150990 con 0x7f3f040b4f90 2024-04-20T11:37:34.126 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.125+0000 7f3f03e00640 1 --2- 172.21.15.152:0/1659826798 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3f040b4f90 0x7f3f040cf400 secure :-1 s=READY pgs=2246 cs=0 l=1 rev1=1 crypto rx=0x7f3efc00c680 tx=0x7f3efc00cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:34.126 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.125+0000 7f3efb400640 1 -- 172.21.15.152:0/1659826798 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3efc003b70 con 0x7f3f040b4f90 2024-04-20T11:37:34.126 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.125+0000 7f3efb400640 1 -- 172.21.15.152:0/1659826798 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3efc003d10 con 0x7f3f040b4f90 2024-04-20T11:37:34.126 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.125+0000 7f3efb400640 1 -- 172.21.15.152:0/1659826798 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3efc005310 con 0x7f3f040b4f90 2024-04-20T11:37:34.126 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.125+0000 7f3f08c00640 1 -- 172.21.15.152:0/1659826798 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f041ecdd0 con 0x7f3f040b4f90 2024-04-20T11:37:34.126 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.125+0000 7f3f08c00640 1 -- 172.21.15.152:0/1659826798 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f041ed390 con 0x7f3f040b4f90 2024-04-20T11:37:34.127 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.127+0000 7f3f08c00640 1 -- 172.21.15.152:0/1659826798 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f04075e60 con 0x7f3f040b4f90 2024-04-20T11:37:34.127 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.128+0000 7f3efb400640 1 -- 172.21.15.152:0/1659826798 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 26) ==== 91907+0+0 (secure 0 0 0) 0x7f3efc0054b0 con 0x7f3f040b4f90 2024-04-20T11:37:34.128 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.128+0000 7f3efb400640 1 --2- 172.21.15.152:0/1659826798 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f3ee006d280 0x7f3ee006f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:34.128 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.129+0000 7f3efb400640 1 -- 172.21.15.152:0/1659826798 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5767+0+0 (secure 0 0 0) 0x7f3efc08a110 con 0x7f3f040b4f90 2024-04-20T11:37:34.128 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.129+0000 7f3f03400640 1 --2- 172.21.15.152:0/1659826798 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f3ee006d280 0x7f3ee006f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:34.130 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.131+0000 7f3f03400640 1 --2- 172.21.15.152:0/1659826798 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f3ee006d280 0x7f3ee006f740 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f3ef400a810 tx=0x7f3ef4008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:34.133 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.134+0000 7f3efb400640 1 -- 172.21.15.152:0/1659826798 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f3f04150990 con 0x7f3f040b4f90 2024-04-20T11:37:34.335 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.335+0000 7f3f08c00640 1 -- 172.21.15.152:0/1659826798 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3f040b4820 con 0x7f3f040b4f90 2024-04-20T11:37:34.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.337+0000 7f3efb400640 1 -- 172.21.15.152:0/1659826798 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v156) ==== 74+0+16781 (secure 0 0 0) 0x7f3efc05a140 con 0x7f3f040b4f90 2024-04-20T11:37:34.337 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:37:34.338 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":156,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:37:24.837045+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:36:57.520028+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"153","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2024-04-20T11:36:58.760637+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"154","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"25.13","mappings":[{"from":0,"to":1}]},{"pgid":"25.15","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:37:34.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.340+0000 7f3f08c00640 1 -- 172.21.15.152:0/1659826798 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f3ee006d280 msgr2=0x7f3ee006f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:34.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.340+0000 7f3f08c00640 1 --2- 172.21.15.152:0/1659826798 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f3ee006d280 0x7f3ee006f740 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f3ef400a810 tx=0x7f3ef4008040 comp rx=0 tx=0).stop 2024-04-20T11:37:34.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.340+0000 7f3f08c00640 1 -- 172.21.15.152:0/1659826798 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3f040b4f90 msgr2=0x7f3f040cf400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:34.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.340+0000 7f3f08c00640 1 --2- 172.21.15.152:0/1659826798 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3f040b4f90 0x7f3f040cf400 secure :-1 s=READY pgs=2246 cs=0 l=1 rev1=1 crypto rx=0x7f3efc00c680 tx=0x7f3efc00cb40 comp rx=0 tx=0).stop 2024-04-20T11:37:34.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.340+0000 7f3f08c00640 1 -- 172.21.15.152:0/1659826798 shutdown_connections 2024-04-20T11:37:34.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.340+0000 7f3f08c00640 1 --2- 172.21.15.152:0/1659826798 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f3ee006d280 0x7f3ee006f740 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:34.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.340+0000 7f3f08c00640 1 --2- 172.21.15.152:0/1659826798 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3f040b4f90 0x7f3f040cf400 unknown :-1 s=CLOSED pgs=2246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:34.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.340+0000 7f3f08c00640 1 --2- 172.21.15.152:0/1659826798 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3f04071cd0 0x7f3f040ceec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:34.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.340+0000 7f3f08c00640 1 --2- 172.21.15.152:0/1659826798 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3f0406ea20 0x7f3f04150450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:34.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.340+0000 7f3f08c00640 1 -- 172.21.15.152:0/1659826798 >> 172.21.15.152:0/1659826798 conn(0x7f3f04067ae0 msgr2=0x7f3f040cc7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:34.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.341+0000 7f3f08c00640 1 -- 172.21.15.152:0/1659826798 shutdown_connections 2024-04-20T11:37:34.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.341+0000 7f3f08c00640 1 -- 172.21.15.152:0/1659826798 wait complete. 2024-04-20T11:37:34.361 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-20T11:37:34.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.562+0000 7f09ed600640 1 Processor -- start 2024-04-20T11:37:34.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.563+0000 7f09ed600640 1 -- start start 2024-04-20T11:37:34.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.564+0000 7f09ed600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f09e8161d30 0x7f09e80d0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:34.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.564+0000 7f09ed600640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f09e80d4ec0 0x7f09e80d15e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:34.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.564+0000 7f09ed600640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09e80d1b20 con 0x7f09e80d2b80 2024-04-20T11:37:34.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.564+0000 7f09ed600640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f09e80d0540 con 0x7f09e80d4ec0 2024-04-20T11:37:34.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.564+0000 7f09ed600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f09e80cfb70 con 0x7f09e8161d30 2024-04-20T11:37:34.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.565+0000 7f09ecc00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f09e80d2b80 0x7f09e806c830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:40850/0 (socket says 172.21.15.152:40850) 2024-04-20T11:37:34.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.565+0000 7f09ecc00640 1 -- 172.21.15.152:0/426560546 learned_addr learned my addr 172.21.15.152:0/426560546 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:37:34.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.565+0000 7f09e7e00640 1 --2- 172.21.15.152:0/426560546 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f09e8161d30 0x7f09e80d0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:34.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.565+0000 7f09e7400640 1 --2- 172.21.15.152:0/426560546 >> v2:172.21.15.175:3300/0 conn(0x7f09e80d4ec0 0x7f09e80d15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:34.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.565+0000 7f09e6a00640 1 -- 172.21.15.152:0/426560546 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 592068870 0 0) 0x7f09e80d1b20 con 0x7f09e80d2b80 2024-04-20T11:37:34.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.565+0000 7f09e6a00640 1 -- 172.21.15.152:0/426560546 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f09c8003520 con 0x7f09e80d2b80 2024-04-20T11:37:34.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.566+0000 7f09e6a00640 1 -- 172.21.15.152:0/426560546 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1807323576 0 0) 0x7f09c8003520 con 0x7f09e80d2b80 2024-04-20T11:37:34.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.566+0000 7f09e7e00640 1 -- 172.21.15.152:0/426560546 >> v2:172.21.15.175:3300/0 conn(0x7f09e80d4ec0 msgr2=0x7f09e80d15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:34.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.566+0000 7f09e7e00640 1 --2- 172.21.15.152:0/426560546 >> v2:172.21.15.175:3300/0 conn(0x7f09e80d4ec0 0x7f09e80d15e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:34.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.566+0000 7f09e7e00640 1 -- 172.21.15.152:0/426560546 >> v1:172.21.15.152:6789/0 conn(0x7f09e80d2b80 legacy=0x7f09e806c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:34.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.566+0000 7f09e7e00640 1 -- 172.21.15.152:0/426560546 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09e80d26a0 con 0x7f09e8161d30 2024-04-20T11:37:34.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.567+0000 7f09e7e00640 1 --2- 172.21.15.152:0/426560546 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f09e8161d30 0x7f09e80d0000 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f09d80097a0 tx=0x7f09d802b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ebb886b390752e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:34.567 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.568+0000 7f09e6a00640 1 -- 172.21.15.152:0/426560546 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d8037070 con 0x7f09e8161d30 2024-04-20T11:37:34.567 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.568+0000 7f09e6a00640 1 -- 172.21.15.152:0/426560546 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09d8030510 con 0x7f09e8161d30 2024-04-20T11:37:34.567 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.568+0000 7f09e6a00640 1 -- 172.21.15.152:0/426560546 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09e0001670 con 0x7f09e8161d30 2024-04-20T11:37:34.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.568+0000 7f09ed600640 1 -- 172.21.15.152:0/426560546 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f09e8161d30 msgr2=0x7f09e80d0000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:34.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.568+0000 7f09ed600640 1 --2- 172.21.15.152:0/426560546 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f09e8161d30 0x7f09e80d0000 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f09d80097a0 tx=0x7f09d802b790 comp rx=0 tx=0).stop 2024-04-20T11:37:34.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.569+0000 7f09ed600640 1 -- 172.21.15.152:0/426560546 shutdown_connections 2024-04-20T11:37:34.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.569+0000 7f09ed600640 1 --2- 172.21.15.152:0/426560546 >> v2:172.21.15.175:3300/0 conn(0x7f09e80d4ec0 0x7f09e80d15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:34.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.569+0000 7f09ed600640 1 --2- 172.21.15.152:0/426560546 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f09e8161d30 0x7f09e80d0000 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:34.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.569+0000 7f09ed600640 1 -- 172.21.15.152:0/426560546 >> 172.21.15.152:0/426560546 conn(0x7f09e808bfd0 msgr2=0x7f09e80d45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:34.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.569+0000 7f09ed600640 1 -- 172.21.15.152:0/426560546 shutdown_connections 2024-04-20T11:37:34.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.569+0000 7f09ed600640 1 -- 172.21.15.152:0/426560546 wait complete. 2024-04-20T11:37:34.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.570+0000 7f09ed600640 1 Processor -- start 2024-04-20T11:37:34.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.571+0000 7f09ed600640 1 -- start start 2024-04-20T11:37:34.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.571+0000 7f09ed600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09e80d2b80 0x7f09e8154910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:34.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.571+0000 7f09ed600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f09e80d4ec0 0x7f09e815e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:34.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.571+0000 7f09ed600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f09e8161d30 0x7f09e815e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:34.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.571+0000 7f09ed600640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f09e80cf800 con 0x7f09e80d4ec0 2024-04-20T11:37:34.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.571+0000 7f09ed600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f09e80cf520 con 0x7f09e80d2b80 2024-04-20T11:37:34.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.571+0000 7f09ed600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f09e80cf1b0 con 0x7f09e8161d30 2024-04-20T11:37:34.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.572+0000 7f09e7400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f09e80d4ec0 0x7f09e815e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:34.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.572+0000 7f09e7400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f09e80d4ec0 0x7f09e815e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36488/0 (socket says 172.21.15.152:36488) 2024-04-20T11:37:34.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.572+0000 7f09e7400640 1 -- 172.21.15.152:0/1136455497 learned_addr learned my addr 172.21.15.152:0/1136455497 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:34.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.572+0000 7f09ecc00640 1 --2- 172.21.15.152:0/1136455497 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f09e8161d30 0x7f09e815e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:34.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.572+0000 7f09e7e00640 1 --2- 172.21.15.152:0/1136455497 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09e80d2b80 0x7f09e8154910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:34.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.572+0000 7f09e7400640 1 -- 172.21.15.152:0/1136455497 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f09e8161d30 msgr2=0x7f09e815e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:34.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.572+0000 7f09e7400640 1 --2- 172.21.15.152:0/1136455497 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f09e8161d30 0x7f09e815e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:34.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.572+0000 7f09e7400640 1 -- 172.21.15.152:0/1136455497 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09e80d2b80 msgr2=0x7f09e8154910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:34.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.572+0000 7f09e7400640 1 --2- 172.21.15.152:0/1136455497 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09e80d2b80 0x7f09e8154910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:34.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.572+0000 7f09e7400640 1 -- 172.21.15.152:0/1136455497 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09e8154e50 con 0x7f09e80d4ec0 2024-04-20T11:37:34.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.572+0000 7f09ecc00640 1 --2- 172.21.15.152:0/1136455497 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f09e8161d30 0x7f09e815e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:37:34.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.573+0000 7f09e7400640 1 --2- 172.21.15.152:0/1136455497 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f09e80d4ec0 0x7f09e815e4a0 secure :-1 s=READY pgs=2248 cs=0 l=1 rev1=1 crypto rx=0x7f09d000edb0 tx=0x7f09d000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:34.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.573+0000 7f09dfe00640 1 -- 172.21.15.152:0/1136455497 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d0018020 con 0x7f09e80d4ec0 2024-04-20T11:37:34.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.573+0000 7f09dfe00640 1 -- 172.21.15.152:0/1136455497 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09d0014480 con 0x7f09e80d4ec0 2024-04-20T11:37:34.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.573+0000 7f09ed600640 1 -- 172.21.15.152:0/1136455497 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09e809e4f0 con 0x7f09e80d4ec0 2024-04-20T11:37:34.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.573+0000 7f09dfe00640 1 -- 172.21.15.152:0/1136455497 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d0014870 con 0x7f09e80d4ec0 2024-04-20T11:37:34.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.573+0000 7f09ed600640 1 -- 172.21.15.152:0/1136455497 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09e809eab0 con 0x7f09e80d4ec0 2024-04-20T11:37:34.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.574+0000 7f09ed600640 1 -- 172.21.15.152:0/1136455497 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09e8075e60 con 0x7f09e80d4ec0 2024-04-20T11:37:34.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.575+0000 7f09dfe00640 1 -- 172.21.15.152:0/1136455497 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 26) ==== 91907+0+0 (secure 0 0 0) 0x7f09d001b020 con 0x7f09e80d4ec0 2024-04-20T11:37:34.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.576+0000 7f09dfe00640 1 --2- 172.21.15.152:0/1136455497 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f09c406d280 0x7f09c406f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:34.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.576+0000 7f09dfe00640 1 -- 172.21.15.152:0/1136455497 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5767+0+0 (secure 0 0 0) 0x7f09d008af00 con 0x7f09e80d4ec0 2024-04-20T11:37:34.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.577+0000 7f09e7e00640 1 --2- 172.21.15.152:0/1136455497 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f09c406d280 0x7f09c406f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:34.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.578+0000 7f09e7e00640 1 --2- 172.21.15.152:0/1136455497 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f09c406d280 0x7f09c406f740 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f09d8034000 tx=0x7f09d802f000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:34.580 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.581+0000 7f09dfe00640 1 -- 172.21.15.152:0/1136455497 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f09d0022030 con 0x7f09e80d4ec0 2024-04-20T11:37:34.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.818+0000 7f09ed600640 1 -- 172.21.15.152:0/1136455497 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f09e80d1ab0 con 0x7f09e80d4ec0 2024-04-20T11:37:34.819 INFO:tasks.ceph.mon.a.smithi152.stderr:2024-04-20T11:37:34.819+0000 7f8229800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-20T11:37:34.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.823+0000 7f09dfe00640 1 -- 172.21.15.152:0/1136455497 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v107) ==== 161+0+0 (secure 0 0 0) 0x7f09d0010040 con 0x7f09e80d4ec0 2024-04-20T11:37:34.823 INFO:teuthology.orchestra.run.smithi152.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-20T11:37:34.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.827+0000 7f09ed600640 1 -- 172.21.15.152:0/1136455497 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f09c406d280 msgr2=0x7f09c406f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:34.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.827+0000 7f09ed600640 1 --2- 172.21.15.152:0/1136455497 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f09c406d280 0x7f09c406f740 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f09d8034000 tx=0x7f09d802f000 comp rx=0 tx=0).stop 2024-04-20T11:37:34.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.828+0000 7f09ed600640 1 -- 172.21.15.152:0/1136455497 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f09e80d4ec0 msgr2=0x7f09e815e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:34.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.828+0000 7f09ed600640 1 --2- 172.21.15.152:0/1136455497 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f09e80d4ec0 0x7f09e815e4a0 secure :-1 s=READY pgs=2248 cs=0 l=1 rev1=1 crypto rx=0x7f09d000edb0 tx=0x7f09d000c490 comp rx=0 tx=0).stop 2024-04-20T11:37:34.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.828+0000 7f09ed600640 1 -- 172.21.15.152:0/1136455497 shutdown_connections 2024-04-20T11:37:34.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.828+0000 7f09ed600640 1 --2- 172.21.15.152:0/1136455497 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f09c406d280 0x7f09c406f740 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:34.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.828+0000 7f09ed600640 1 --2- 172.21.15.152:0/1136455497 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f09e8161d30 0x7f09e815e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:34.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.828+0000 7f09ed600640 1 --2- 172.21.15.152:0/1136455497 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f09e80d4ec0 0x7f09e815e4a0 secure :-1 s=CLOSED pgs=2248 cs=0 l=1 rev1=1 crypto rx=0x7f09d000edb0 tx=0x7f09d000c490 comp rx=0 tx=0).stop 2024-04-20T11:37:34.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.828+0000 7f09ed600640 1 --2- 172.21.15.152:0/1136455497 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09e80d2b80 0x7f09e8154910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:34.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.828+0000 7f09ed600640 1 -- 172.21.15.152:0/1136455497 >> 172.21.15.152:0/1136455497 conn(0x7f09e808bfd0 msgr2=0x7f09e80ca790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:34.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.829+0000 7f09ed600640 1 -- 172.21.15.152:0/1136455497 shutdown_connections 2024-04-20T11:37:34.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:34.829+0000 7f09ed600640 1 -- 172.21.15.152:0/1136455497 wait complete. 2024-04-20T11:37:34.857 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-20T11:37:34.907 INFO:tasks.ceph.mds.c.smithi152.stdout:starting mds.c at 2024-04-20T11:37:35.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:35.044+0000 7f1604400640 1 Processor -- start 2024-04-20T11:37:35.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:35.044+0000 7f1604400640 1 -- start start 2024-04-20T11:37:35.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:35.045+0000 7f1604400640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f15fc0b2880 0x7f15fc0b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:35.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:35.045+0000 7f1604400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f15fc0cd1b0 0x7f15fc0cd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:35.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:35.045+0000 7f1604400640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15fc1463e0 con 0x7f15fc146000 2024-04-20T11:37:35.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:35.046+0000 7f1604400640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f15fc0b4750 con 0x7f15fc0cd1b0 2024-04-20T11:37:35.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:35.046+0000 7f1604400640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f15fc0b3e20 con 0x7f15fc0b2880 2024-04-20T11:37:35.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:35.046+0000 7f1602400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f15fc0cd1b0 0x7f15fc0cd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:35.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:35.046+0000 7f1601a00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f15fc146000 0x7f15fc0b2170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:40868/0 (socket says 172.21.15.152:40868) 2024-04-20T11:37:35.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:35.046+0000 7f1601a00640 1 -- 172.21.15.152:0/2766701716 learned_addr learned my addr 172.21.15.152:0/2766701716 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:37:35.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:35.046+0000 7f1602400640 1 -- 172.21.15.152:0/2766701716 >> v2:172.21.15.175:3300/0 conn(0x7f15fc0b2880 msgr2=0x7f15fc0b4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:35.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:35.046+0000 7f1601000640 1 --2- 172.21.15.152:0/2766701716 >> v2:172.21.15.175:3300/0 conn(0x7f15fc0b2880 0x7f15fc0b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:35.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:35.046+0000 7f1602400640 1 --2- 172.21.15.152:0/2766701716 >> v2:172.21.15.175:3300/0 conn(0x7f15fc0b2880 0x7f15fc0b4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:35.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:35.046+0000 7f15fbe00640 1 -- 172.21.15.152:0/2766701716 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2923377613 0 0) 0x7f15fc1463e0 con 0x7f15fc146000 2024-04-20T11:37:35.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:35.046+0000 7f1602400640 1 -- 172.21.15.152:0/2766701716 >> v1:172.21.15.152:6789/0 conn(0x7f15fc146000 legacy=0x7f15fc0b2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:35.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:35.046+0000 7f1602400640 1 -- 172.21.15.152:0/2766701716 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15fc0b4100 con 0x7f15fc0cd1b0 2024-04-20T11:37:35.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:35.046+0000 7f1601000640 1 --2- 172.21.15.152:0/2766701716 >> v2:172.21.15.175:3300/0 conn(0x7f15fc0b2880 0x7f15fc0b4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:37:35.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:35.048+0000 7f1602400640 1 --2- 172.21.15.152:0/2766701716 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f15fc0cd1b0 0x7f15fc0cd5f0 secure :-1 s=READY pgs=2253 cs=0 l=1 rev1=1 crypto rx=0x7f15fc0b48d0 tx=0x7f15f402be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46199ce0deaf88e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:35.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:35.048+0000 7f15fbe00640 1 -- 172.21.15.152:0/2766701716 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15f4039070 con 0x7f15fc0cd1b0 2024-04-20T11:37:35.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:35.048+0000 7f15fbe00640 1 -- 172.21.15.152:0/2766701716 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15f402cbd0 con 0x7f15fc0cd1b0 2024-04-20T11:37:35.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:35.048+0000 7f15fbe00640 1 -- 172.21.15.152:0/2766701716 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15f4034440 con 0x7f15fc0cd1b0 2024-04-20T11:37:35.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:35.048+0000 7f1604400640 1 -- 172.21.15.152:0/2766701716 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f15fc0cd1b0 msgr2=0x7f15fc0cd5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:35.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:35.048+0000 7f1604400640 1 --2- 172.21.15.152:0/2766701716 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f15fc0cd1b0 0x7f15fc0cd5f0 secure :-1 s=READY pgs=2253 cs=0 l=1 rev1=1 crypto rx=0x7f15fc0b48d0 tx=0x7f15f402be20 comp rx=0 tx=0).stop 2024-04-20T11:37:35.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:35.049+0000 7f1604400640 1 -- 172.21.15.152:0/2766701716 shutdown_connections 2024-04-20T11:37:35.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:35.049+0000 7f1604400640 1 --2- 172.21.15.152:0/2766701716 >> v2:172.21.15.175:3300/0 conn(0x7f15fc0b2880 0x7f15fc0b4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:35.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:35.049+0000 7f1604400640 1 --2- 172.21.15.152:0/2766701716 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f15fc0cd1b0 0x7f15fc0cd5f0 unknown :-1 s=CLOSED pgs=2253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:35.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:35.049+0000 7f1604400640 1 -- 172.21.15.152:0/2766701716 >> 172.21.15.152:0/2766701716 conn(0x7f15fc06b980 msgr2=0x7f15fc06bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:35.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:35.049+0000 7f1604400640 1 -- 172.21.15.152:0/2766701716 shutdown_connections 2024-04-20T11:37:35.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:35.049+0000 7f1604400640 1 -- 172.21.15.152:0/2766701716 wait complete. 2024-04-20T11:37:35.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:35.050+0000 7f1604400640 1 Processor -- start 2024-04-20T11:37:35.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:35.051+0000 7f1604400640 1 -- start start 2024-04-20T11:37:35.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:35.052+0000 7f1604400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f15fc0b2880 0x7f15fc15bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:35.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:35.052+0000 7f1604400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15fc0cd1b0 0x7f15fc143860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:35.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:35.052+0000 7f1604400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f15fc146000 0x7f15fc143da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:35.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:35.052+0000 7f1604400640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f15fc0cdb30 con 0x7f15fc0b2880 2024-04-20T11:37:35.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:35.052+0000 7f1604400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f15fc15ccf0 con 0x7f15fc0cd1b0 2024-04-20T11:37:35.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:35.052+0000 7f1604400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f15fc1f8cf0 con 0x7f15fc146000 2024-04-20T11:37:35.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:35.052+0000 7f1601a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f15fc0b2880 0x7f15fc15bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:35.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:35.052+0000 7f1601a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f15fc0b2880 0x7f15fc15bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36534/0 (socket says 172.21.15.152:36534) 2024-04-20T11:37:35.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:35.052+0000 7f1601a00640 1 -- 172.21.15.152:0/3260066277 learned_addr learned my addr 172.21.15.152:0/3260066277 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:35.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:35.052+0000 7f1601000640 1 --2- 172.21.15.152:0/3260066277 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15fc0cd1b0 0x7f15fc143860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:35.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:35.052+0000 7f1601a00640 1 -- 172.21.15.152:0/3260066277 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f15fc146000 msgr2=0x7f15fc143da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:35.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:35.052+0000 7f1602400640 1 --2- 172.21.15.152:0/3260066277 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f15fc146000 0x7f15fc143da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:35.054 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:35.052+0000 7f1601a00640 1 --2- 172.21.15.152:0/3260066277 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f15fc146000 0x7f15fc143da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:35.054 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:35.052+0000 7f1601a00640 1 -- 172.21.15.152:0/3260066277 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15fc0cd1b0 msgr2=0x7f15fc143860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:35.054 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:35.052+0000 7f1601a00640 1 --2- 172.21.15.152:0/3260066277 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15fc0cd1b0 0x7f15fc143860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:35.054 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:35.052+0000 7f1601a00640 1 -- 172.21.15.152:0/3260066277 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15fc13f6e0 con 0x7f15fc0b2880 2024-04-20T11:37:35.054 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:35.053+0000 7f1601a00640 1 --2- 172.21.15.152:0/3260066277 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f15fc0b2880 0x7f15fc15bfb0 secure :-1 s=READY pgs=2254 cs=0 l=1 rev1=1 crypto rx=0x7f15f000c430 tx=0x7f15f000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:35.054 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:35.053+0000 7f15f9600640 1 -- 172.21.15.152:0/3260066277 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15f00074e0 con 0x7f15fc0b2880 2024-04-20T11:37:35.054 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:35.053+0000 7f15f9600640 1 -- 172.21.15.152:0/3260066277 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15f00047e0 con 0x7f15fc0b2880 2024-04-20T11:37:35.054 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:35.053+0000 7f15f9600640 1 -- 172.21.15.152:0/3260066277 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15f0004bd0 con 0x7f15fc0b2880 2024-04-20T11:37:35.054 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:35.053+0000 7f1604400640 1 -- 172.21.15.152:0/3260066277 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15fc1f8a10 con 0x7f15fc0b2880 2024-04-20T11:37:35.054 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:35.053+0000 7f1604400640 1 -- 172.21.15.152:0/3260066277 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15fc13d040 con 0x7f15fc0b2880 2024-04-20T11:37:35.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:35.054+0000 7f1604400640 1 -- 172.21.15.152:0/3260066277 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15fc0ce990 con 0x7f15fc0b2880 2024-04-20T11:37:35.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:35.056+0000 7f15f9600640 1 -- 172.21.15.152:0/3260066277 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 26) ==== 91907+0+0 (secure 0 0 0) 0x7f15f0011070 con 0x7f15fc0b2880 2024-04-20T11:37:35.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:35.056+0000 7f15f9600640 1 --2- 172.21.15.152:0/3260066277 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f15dc06d280 0x7f15dc06f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:35.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:35.057+0000 7f15f9600640 1 -- 172.21.15.152:0/3260066277 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(157..157 src has 1..157) ==== 5853+0+0 (secure 0 0 0) 0x7f15f0089f40 con 0x7f15fc0b2880 2024-04-20T11:37:35.057 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:35.057+0000 7f1601000640 1 --2- 172.21.15.152:0/3260066277 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f15dc06d280 0x7f15dc06f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:35.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:35.059+0000 7f1601000640 1 --2- 172.21.15.152:0/3260066277 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f15dc06d280 0x7f15dc06f740 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f15fc0af4d0 tx=0x7f15e8008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:35.061 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:35.061+0000 7f15f9600640 1 -- 172.21.15.152:0/3260066277 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f15fc0ce990 con 0x7f15fc0b2880 2024-04-20T11:37:35.292 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:35.292+0000 7f1604400640 1 -- 172.21.15.152:0/3260066277 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f15fc0689f0 con 0x7f15fc0b2880 2024-04-20T11:37:35.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:35.915+0000 7f15f9600640 1 -- 172.21.15.152:0/3260066277 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v108) ==== 104+0+0 (secure 0 0 0) 0x7f15fc13f6e0 con 0x7f15fc0b2880 2024-04-20T11:37:35.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:35.919+0000 7f1604400640 1 -- 172.21.15.152:0/3260066277 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f15dc06d280 msgr2=0x7f15dc06f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:35.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:35.919+0000 7f1604400640 1 --2- 172.21.15.152:0/3260066277 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f15dc06d280 0x7f15dc06f740 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f15fc0af4d0 tx=0x7f15e8008040 comp rx=0 tx=0).stop 2024-04-20T11:37:35.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:35.919+0000 7f1604400640 1 -- 172.21.15.152:0/3260066277 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f15fc0b2880 msgr2=0x7f15fc15bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:35.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:35.919+0000 7f1604400640 1 --2- 172.21.15.152:0/3260066277 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f15fc0b2880 0x7f15fc15bfb0 secure :-1 s=READY pgs=2254 cs=0 l=1 rev1=1 crypto rx=0x7f15f000c430 tx=0x7f15f000c8f0 comp rx=0 tx=0).stop 2024-04-20T11:37:35.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:35.919+0000 7f1604400640 1 -- 172.21.15.152:0/3260066277 shutdown_connections 2024-04-20T11:37:35.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:35.919+0000 7f1604400640 1 --2- 172.21.15.152:0/3260066277 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f15dc06d280 0x7f15dc06f740 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:35.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:35.919+0000 7f1604400640 1 --2- 172.21.15.152:0/3260066277 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f15fc146000 0x7f15fc143da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:35.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:35.919+0000 7f1604400640 1 --2- 172.21.15.152:0/3260066277 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15fc0cd1b0 0x7f15fc143860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:35.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:35.919+0000 7f1604400640 1 --2- 172.21.15.152:0/3260066277 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f15fc0b2880 0x7f15fc15bfb0 unknown :-1 s=CLOSED pgs=2254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:35.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:35.919+0000 7f1604400640 1 -- 172.21.15.152:0/3260066277 >> 172.21.15.152:0/3260066277 conn(0x7f15fc06b980 msgr2=0x7f15fc06bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:35.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:35.919+0000 7f1604400640 1 -- 172.21.15.152:0/3260066277 shutdown_connections 2024-04-20T11:37:35.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:35.920+0000 7f1604400640 1 -- 172.21.15.152:0/3260066277 wait complete. 2024-04-20T11:37:35.943 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-20T11:37:36.146 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:36.147+0000 7f1559600640 1 Processor -- start 2024-04-20T11:37:36.146 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:36.147+0000 7f1559600640 1 -- start start 2024-04-20T11:37:36.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:36.148+0000 7f1559600640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f15540ce1e0 0x7f155415cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:36.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:36.148+0000 7f1559600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f155415d1b0 0x7f1554071db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:36.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:36.148+0000 7f1559600640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f155415d710 con 0x7f155415a450 2024-04-20T11:37:36.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:36.148+0000 7f1559600640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f15540cd320 con 0x7f15540ce1e0 2024-04-20T11:37:36.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:36.148+0000 7f1559600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f155415d9f0 con 0x7f155415d1b0 2024-04-20T11:37:36.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:36.149+0000 7f154a000640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f15540ce1e0 0x7f155415cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:36.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:36.149+0000 7f1553400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f155415d1b0 0x7f1554071db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:36.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:36.149+0000 7f154a000640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f15540ce1e0 0x7f155415cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.152:60354/0 (socket says 172.21.15.152:60354) 2024-04-20T11:37:36.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:36.149+0000 7f154a000640 1 -- 172.21.15.152:0/1471862073 learned_addr learned my addr 172.21.15.152:0/1471862073 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:36.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:36.150+0000 7f1553400640 1 -- 172.21.15.152:0/1471862073 >> v2:172.21.15.175:3300/0 conn(0x7f15540ce1e0 msgr2=0x7f155415cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:36.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:36.150+0000 7f1553400640 1 --2- 172.21.15.152:0/1471862073 >> v2:172.21.15.175:3300/0 conn(0x7f15540ce1e0 0x7f155415cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:36.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:36.150+0000 7f1553400640 1 -- 172.21.15.152:0/1471862073 >> v1:172.21.15.175:6789/0 conn(0x7f155415a450 legacy=0x7f15540cd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:36.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:36.150+0000 7f1553400640 1 -- 172.21.15.152:0/1471862073 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15540ccfb0 con 0x7f155415d1b0 2024-04-20T11:37:36.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:36.150+0000 7f154a000640 1 --2- 172.21.15.152:0/1471862073 >> v2:172.21.15.175:3300/0 conn(0x7f15540ce1e0 0x7f155415cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:37:36.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:36.152+0000 7f1553400640 1 --2- 172.21.15.152:0/1471862073 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f155415d1b0 0x7f1554071db0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f154c0097a0 tx=0x7f154c02b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af3cc636c101f031 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:36.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:36.152+0000 7f1552000640 1 -- 172.21.15.152:0/1471862073 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f154c034070 con 0x7f155415d1b0 2024-04-20T11:37:36.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:36.152+0000 7f1552000640 1 -- 172.21.15.152:0/1471862073 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f154c030610 con 0x7f155415d1b0 2024-04-20T11:37:36.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:36.152+0000 7f1552000640 1 -- 172.21.15.152:0/1471862073 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f154c0309e0 con 0x7f155415d1b0 2024-04-20T11:37:36.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:36.153+0000 7f1559600640 1 -- 172.21.15.152:0/1471862073 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f155415d1b0 msgr2=0x7f1554071db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:36.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:36.153+0000 7f1559600640 1 --2- 172.21.15.152:0/1471862073 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f155415d1b0 0x7f1554071db0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f154c0097a0 tx=0x7f154c02b960 comp rx=0 tx=0).stop 2024-04-20T11:37:36.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:36.153+0000 7f1559600640 1 -- 172.21.15.152:0/1471862073 shutdown_connections 2024-04-20T11:37:36.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:36.153+0000 7f1559600640 1 --2- 172.21.15.152:0/1471862073 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f155415d1b0 0x7f1554071db0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:36.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:36.153+0000 7f1559600640 1 --2- 172.21.15.152:0/1471862073 >> v2:172.21.15.175:3300/0 conn(0x7f15540ce1e0 0x7f155415cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:36.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:36.153+0000 7f1559600640 1 -- 172.21.15.152:0/1471862073 >> 172.21.15.152:0/1471862073 conn(0x7f1554067cf0 msgr2=0x7f155406b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:36.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:36.153+0000 7f1559600640 1 -- 172.21.15.152:0/1471862073 shutdown_connections 2024-04-20T11:37:36.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:36.153+0000 7f1559600640 1 -- 172.21.15.152:0/1471862073 wait complete. 2024-04-20T11:37:36.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:36.155+0000 7f1559600640 1 Processor -- start 2024-04-20T11:37:36.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:36.155+0000 7f1559600640 1 -- start start 2024-04-20T11:37:36.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:36.156+0000 7f1559600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15540ce1e0 0x7f1554142fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:36.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:36.156+0000 7f1559600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f155415a450 0x7f1554138ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:36.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:36.156+0000 7f1559600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f155415d1b0 0x7f155415ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:36.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:36.156+0000 7f1559600640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f1554072700 con 0x7f155415a450 2024-04-20T11:37:36.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:36.156+0000 7f1559600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1554072580 con 0x7f15540ce1e0 2024-04-20T11:37:36.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:36.156+0000 7f1559600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f155406ddd0 con 0x7f155415d1b0 2024-04-20T11:37:36.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:36.156+0000 7f154a000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f155415a450 0x7f1554138ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:36.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:36.156+0000 7f154a000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f155415a450 0x7f1554138ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:59710/0 (socket says 172.21.15.152:59710) 2024-04-20T11:37:36.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:36.156+0000 7f154a000640 1 -- 172.21.15.152:0/54153579 learned_addr learned my addr 172.21.15.152:0/54153579 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:36.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:36.156+0000 7f1553400640 1 --2- 172.21.15.152:0/54153579 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f155415d1b0 0x7f155415ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:36.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:36.156+0000 7f1552a00640 1 --2- 172.21.15.152:0/54153579 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15540ce1e0 0x7f1554142fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:36.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:36.157+0000 7f154a000640 1 -- 172.21.15.152:0/54153579 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f155415d1b0 msgr2=0x7f155415ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:36.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:36.157+0000 7f154a000640 1 --2- 172.21.15.152:0/54153579 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f155415d1b0 0x7f155415ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:36.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:36.157+0000 7f154a000640 1 -- 172.21.15.152:0/54153579 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15540ce1e0 msgr2=0x7f1554142fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:36.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:36.157+0000 7f154a000640 1 --2- 172.21.15.152:0/54153579 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15540ce1e0 0x7f1554142fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:36.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:36.157+0000 7f154a000640 1 -- 172.21.15.152:0/54153579 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1554144890 con 0x7f155415a450 2024-04-20T11:37:36.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:36.157+0000 7f1552a00640 1 --2- 172.21.15.152:0/54153579 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15540ce1e0 0x7f1554142fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:37:36.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:36.157+0000 7f154a000640 1 --2- 172.21.15.152:0/54153579 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f155415a450 0x7f1554138ec0 secure :-1 s=READY pgs=2255 cs=0 l=1 rev1=1 crypto rx=0x7f153c00e9d0 tx=0x7f153c00ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:36.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:36.157+0000 7f154b400640 1 -- 172.21.15.152:0/54153579 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f153c017070 con 0x7f155415a450 2024-04-20T11:37:36.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:36.157+0000 7f154b400640 1 -- 172.21.15.152:0/54153579 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f153c00ca60 con 0x7f155415a450 2024-04-20T11:37:36.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:36.157+0000 7f154b400640 1 -- 172.21.15.152:0/54153579 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f153c00ce10 con 0x7f155415a450 2024-04-20T11:37:36.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:36.157+0000 7f1559600640 1 -- 172.21.15.152:0/54153579 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15541437f0 con 0x7f155415a450 2024-04-20T11:37:36.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:36.157+0000 7f1559600640 1 -- 172.21.15.152:0/54153579 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1554143db0 con 0x7f155415a450 2024-04-20T11:37:36.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:36.158+0000 7f1559600640 1 -- 172.21.15.152:0/54153579 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1554068320 con 0x7f155415a450 2024-04-20T11:37:36.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:36.160+0000 7f154b400640 1 -- 172.21.15.152:0/54153579 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 26) ==== 91907+0+0 (secure 0 0 0) 0x7f153c01a070 con 0x7f155415a450 2024-04-20T11:37:36.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:36.160+0000 7f154b400640 1 --2- 172.21.15.152:0/54153579 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f153006d280 0x7f153006f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:36.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:36.161+0000 7f154b400640 1 -- 172.21.15.152:0/54153579 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(157..157 src has 1..157) ==== 5853+0+0 (secure 0 0 0) 0x7f153c08a630 con 0x7f155415a450 2024-04-20T11:37:36.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:36.161+0000 7f1552a00640 1 --2- 172.21.15.152:0/54153579 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f153006d280 0x7f153006f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:36.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:36.162+0000 7f1552a00640 1 --2- 172.21.15.152:0/54153579 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f153006d280 0x7f153006f740 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f1544001d70 tx=0x7f1544009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:36.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:36.165+0000 7f154b400640 1 -- 172.21.15.152:0/54153579 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f1554068320 con 0x7f155415a450 2024-04-20T11:37:36.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:36.349+0000 7f1559600640 1 -- 172.21.15.152:0/54153579 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f1554072f10 con 0x7f155415a450 2024-04-20T11:37:36.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:36.943+0000 7f154b400640 1 -- 172.21.15.152:0/54153579 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v158) ==== 181+0+0 (secure 0 0 0) 0x7f153c05a660 con 0x7f155415a450 2024-04-20T11:37:36.943 INFO:teuthology.orchestra.run.smithi152.stderr:pool 'cephfs_metadata' removed 2024-04-20T11:37:36.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:36.947+0000 7f1559600640 1 -- 172.21.15.152:0/54153579 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f153006d280 msgr2=0x7f153006f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:36.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:36.947+0000 7f1559600640 1 --2- 172.21.15.152:0/54153579 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f153006d280 0x7f153006f740 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f1544001d70 tx=0x7f1544009040 comp rx=0 tx=0).stop 2024-04-20T11:37:36.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:36.947+0000 7f1559600640 1 -- 172.21.15.152:0/54153579 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f155415a450 msgr2=0x7f1554138ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:36.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:36.947+0000 7f1559600640 1 --2- 172.21.15.152:0/54153579 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f155415a450 0x7f1554138ec0 secure :-1 s=READY pgs=2255 cs=0 l=1 rev1=1 crypto rx=0x7f153c00e9d0 tx=0x7f153c00ee90 comp rx=0 tx=0).stop 2024-04-20T11:37:36.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:36.947+0000 7f1559600640 1 -- 172.21.15.152:0/54153579 shutdown_connections 2024-04-20T11:37:36.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:36.947+0000 7f1559600640 1 --2- 172.21.15.152:0/54153579 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f153006d280 0x7f153006f740 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:36.948 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:36.947+0000 7f1559600640 1 --2- 172.21.15.152:0/54153579 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f155415d1b0 0x7f155415ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:36.948 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:36.947+0000 7f1559600640 1 --2- 172.21.15.152:0/54153579 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f155415a450 0x7f1554138ec0 unknown :-1 s=CLOSED pgs=2255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:36.948 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:36.947+0000 7f1559600640 1 --2- 172.21.15.152:0/54153579 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15540ce1e0 0x7f1554142fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:36.948 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:36.947+0000 7f1559600640 1 -- 172.21.15.152:0/54153579 >> 172.21.15.152:0/54153579 conn(0x7f1554067cf0 msgr2=0x7f1554141de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:36.948 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:36.948+0000 7f1559600640 1 -- 172.21.15.152:0/54153579 shutdown_connections 2024-04-20T11:37:36.948 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:36.948+0000 7f1559600640 1 -- 172.21.15.152:0/54153579 wait complete. 2024-04-20T11:37:36.971 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-20T11:37:37.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:37.169+0000 7f7856e00640 1 Processor -- start 2024-04-20T11:37:37.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:37.169+0000 7f7856e00640 1 -- start start 2024-04-20T11:37:37.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:37.170+0000 7f7856e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f785006fd30 0x7f78500725e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:37.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:37.171+0000 7f7856e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f785015b800 0x7f785015be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:37.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:37.171+0000 7f7856e00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f785015bc90 con 0x7f78500d06a0 2024-04-20T11:37:37.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:37.171+0000 7f7856e00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f7850072ce0 con 0x7f785015b800 2024-04-20T11:37:37.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:37.171+0000 7f7856e00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f78500b3890 con 0x7f785006fd30 2024-04-20T11:37:37.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:37.171+0000 7f7855000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f785015b800 0x7f785015be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:37.172 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:37.171+0000 7f7856400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f78500d06a0 0x7f78500b4650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:41820/0 (socket says 172.21.15.152:41820) 2024-04-20T11:37:37.172 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:37.171+0000 7f7856400640 1 -- 172.21.15.152:0/356507103 learned_addr learned my addr 172.21.15.152:0/356507103 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:37:37.172 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:37.171+0000 7f7855a00640 1 --2- 172.21.15.152:0/356507103 >> v2:172.21.15.175:3300/0 conn(0x7f785006fd30 0x7f78500725e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:37.172 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:37.171+0000 7f7855000640 1 -- 172.21.15.152:0/356507103 >> v2:172.21.15.175:3300/0 conn(0x7f785006fd30 msgr2=0x7f78500725e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:37.172 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:37.171+0000 7f7855000640 1 --2- 172.21.15.152:0/356507103 >> v2:172.21.15.175:3300/0 conn(0x7f785006fd30 0x7f78500725e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:37.172 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:37.171+0000 7f7855000640 1 -- 172.21.15.152:0/356507103 >> v1:172.21.15.152:6789/0 conn(0x7f78500d06a0 legacy=0x7f78500b4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:37.172 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:37.171+0000 7f7855000640 1 -- 172.21.15.152:0/356507103 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78500b3b70 con 0x7f785015b800 2024-04-20T11:37:37.172 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:37.171+0000 7f7855a00640 1 --2- 172.21.15.152:0/356507103 >> v2:172.21.15.175:3300/0 conn(0x7f785006fd30 0x7f78500725e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:37:37.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:37.173+0000 7f7855000640 1 --2- 172.21.15.152:0/356507103 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f785015b800 0x7f785015be90 secure :-1 s=READY pgs=2257 cs=0 l=1 rev1=1 crypto rx=0x7f785006f4e0 tx=0x7f783c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f23e23cbd24b8796 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:37.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:37.173+0000 7f784fe00640 1 -- 172.21.15.152:0/356507103 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f783c035020 con 0x7f785015b800 2024-04-20T11:37:37.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:37.173+0000 7f784fe00640 1 -- 172.21.15.152:0/356507103 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f783c031830 con 0x7f785015b800 2024-04-20T11:37:37.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:37.173+0000 7f784fe00640 1 -- 172.21.15.152:0/356507103 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f783c031c00 con 0x7f785015b800 2024-04-20T11:37:37.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:37.173+0000 7f7856e00640 1 -- 172.21.15.152:0/356507103 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f785015b800 msgr2=0x7f785015be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:37.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:37.173+0000 7f7856e00640 1 --2- 172.21.15.152:0/356507103 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f785015b800 0x7f785015be90 secure :-1 s=READY pgs=2257 cs=0 l=1 rev1=1 crypto rx=0x7f785006f4e0 tx=0x7f783c02b8d0 comp rx=0 tx=0).stop 2024-04-20T11:37:37.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:37.174+0000 7f7856e00640 1 -- 172.21.15.152:0/356507103 shutdown_connections 2024-04-20T11:37:37.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:37.174+0000 7f7856e00640 1 --2- 172.21.15.152:0/356507103 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f785015b800 0x7f785015be90 unknown :-1 s=CLOSED pgs=2257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:37.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:37.174+0000 7f7856e00640 1 --2- 172.21.15.152:0/356507103 >> v2:172.21.15.175:3300/0 conn(0x7f785006fd30 0x7f78500725e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:37.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:37.174+0000 7f7856e00640 1 -- 172.21.15.152:0/356507103 >> 172.21.15.152:0/356507103 conn(0x7f78500b21e0 msgr2=0x7f78500b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:37.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:37.174+0000 7f7856e00640 1 -- 172.21.15.152:0/356507103 shutdown_connections 2024-04-20T11:37:37.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:37.174+0000 7f7856e00640 1 -- 172.21.15.152:0/356507103 wait complete. 2024-04-20T11:37:37.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:37.175+0000 7f7856e00640 1 Processor -- start 2024-04-20T11:37:37.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:37.176+0000 7f7856e00640 1 -- start start 2024-04-20T11:37:37.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:37.176+0000 7f7856e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f785006fd30 0x7f7850150800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:37.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:37.177+0000 7f7856e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78500d06a0 0x7f78500ceb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:37.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:37.177+0000 7f7856e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f785015b800 0x7f78500cf080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:37.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:37.177+0000 7f7856e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f785015c3d0 con 0x7f785015b800 2024-04-20T11:37:37.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:37.177+0000 7f7856e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f78500b4130 con 0x7f78500d06a0 2024-04-20T11:37:37.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:37.177+0000 7f7856e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f785015dec0 con 0x7f785006fd30 2024-04-20T11:37:37.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:37.177+0000 7f7856400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f785015b800 0x7f78500cf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:37.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:37.177+0000 7f7856400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f785015b800 0x7f78500cf080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:59720/0 (socket says 172.21.15.152:59720) 2024-04-20T11:37:37.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:37.177+0000 7f7856400640 1 -- 172.21.15.152:0/1393945148 learned_addr learned my addr 172.21.15.152:0/1393945148 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:37.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:37.177+0000 7f7855a00640 1 --2- 172.21.15.152:0/1393945148 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f785006fd30 0x7f7850150800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:37.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:37.177+0000 7f7856400640 1 -- 172.21.15.152:0/1393945148 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f785006fd30 msgr2=0x7f7850150800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:37.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:37.177+0000 7f7856400640 1 --2- 172.21.15.152:0/1393945148 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f785006fd30 0x7f7850150800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:37.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:37.177+0000 7f7855000640 1 --2- 172.21.15.152:0/1393945148 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78500d06a0 0x7f78500ceb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:37.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:37.177+0000 7f7856400640 1 -- 172.21.15.152:0/1393945148 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78500d06a0 msgr2=0x7f78500ceb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:37.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:37.177+0000 7f7856400640 1 --2- 172.21.15.152:0/1393945148 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78500d06a0 0x7f78500ceb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:37.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:37.177+0000 7f7856400640 1 -- 172.21.15.152:0/1393945148 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78500cf690 con 0x7f785015b800 2024-04-20T11:37:37.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:37.178+0000 7f7856400640 1 --2- 172.21.15.152:0/1393945148 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f785015b800 0x7f78500cf080 secure :-1 s=READY pgs=2258 cs=0 l=1 rev1=1 crypto rx=0x7f784800cbc0 tx=0x7f784800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:37.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:37.178+0000 7f784d600640 1 -- 172.21.15.152:0/1393945148 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7848017070 con 0x7f785015b800 2024-04-20T11:37:37.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:37.178+0000 7f784d600640 1 -- 172.21.15.152:0/1393945148 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f784800ad40 con 0x7f785015b800 2024-04-20T11:37:37.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:37.178+0000 7f784d600640 1 -- 172.21.15.152:0/1393945148 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7848012600 con 0x7f785015b800 2024-04-20T11:37:37.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:37.178+0000 7f7856e00640 1 -- 172.21.15.152:0/1393945148 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78501f4300 con 0x7f785015b800 2024-04-20T11:37:37.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:37.178+0000 7f7856e00640 1 -- 172.21.15.152:0/1393945148 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78500cc8e0 con 0x7f785015b800 2024-04-20T11:37:37.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:37.179+0000 7f7856e00640 1 -- 172.21.15.152:0/1393945148 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78501f4130 con 0x7f785015b800 2024-04-20T11:37:37.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:37.180+0000 7f784d600640 1 -- 172.21.15.152:0/1393945148 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 26) ==== 91907+0+0 (secure 0 0 0) 0x7f784801a070 con 0x7f785015b800 2024-04-20T11:37:37.181 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:37.181+0000 7f784d600640 1 --2- 172.21.15.152:0/1393945148 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f783006d2b0 0x7f783006f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:37.181 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:37.182+0000 7f784d600640 1 -- 172.21.15.152:0/1393945148 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 5415+0+0 (secure 0 0 0) 0x7f7848089f60 con 0x7f785015b800 2024-04-20T11:37:37.181 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:37.182+0000 7f7855a00640 1 --2- 172.21.15.152:0/1393945148 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f783006d2b0 0x7f783006f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:37.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:37.183+0000 7f7855a00640 1 --2- 172.21.15.152:0/1393945148 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f783006d2b0 0x7f783006f770 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f78501603b0 tx=0x7f7844008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:37.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:37.186+0000 7f784d600640 1 -- 172.21.15.152:0/1393945148 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f785015bc90 con 0x7f785015b800 2024-04-20T11:37:37.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:37.386+0000 7f7856e00640 1 -- 172.21.15.152:0/1393945148 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f7850075e60 con 0x7f785015b800 2024-04-20T11:37:37.955 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:37.955+0000 7f784d600640 1 -- 172.21.15.152:0/1393945148 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v159) ==== 169+0+0 (secure 0 0 0) 0x7f7848059940 con 0x7f785015b800 2024-04-20T11:37:37.956 INFO:teuthology.orchestra.run.smithi152.stderr:pool 'cephfs_data' removed 2024-04-20T11:37:37.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:37.961+0000 7f7856e00640 1 -- 172.21.15.152:0/1393945148 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f783006d2b0 msgr2=0x7f783006f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:37.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:37.961+0000 7f7856e00640 1 --2- 172.21.15.152:0/1393945148 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f783006d2b0 0x7f783006f770 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f78501603b0 tx=0x7f7844008040 comp rx=0 tx=0).stop 2024-04-20T11:37:37.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:37.961+0000 7f7856e00640 1 -- 172.21.15.152:0/1393945148 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f785015b800 msgr2=0x7f78500cf080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:37.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:37.961+0000 7f7856e00640 1 --2- 172.21.15.152:0/1393945148 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f785015b800 0x7f78500cf080 secure :-1 s=READY pgs=2258 cs=0 l=1 rev1=1 crypto rx=0x7f784800cbc0 tx=0x7f784800a430 comp rx=0 tx=0).stop 2024-04-20T11:37:37.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:37.962+0000 7f7856e00640 1 -- 172.21.15.152:0/1393945148 shutdown_connections 2024-04-20T11:37:37.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:37.962+0000 7f7856e00640 1 --2- 172.21.15.152:0/1393945148 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f783006d2b0 0x7f783006f770 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:37.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:37.962+0000 7f7856e00640 1 --2- 172.21.15.152:0/1393945148 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f785015b800 0x7f78500cf080 unknown :-1 s=CLOSED pgs=2258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:37.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:37.962+0000 7f7856e00640 1 --2- 172.21.15.152:0/1393945148 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78500d06a0 0x7f78500ceb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:37.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:37.962+0000 7f7856e00640 1 --2- 172.21.15.152:0/1393945148 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f785006fd30 0x7f7850150800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:37.963 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:37.962+0000 7f7856e00640 1 -- 172.21.15.152:0/1393945148 >> 172.21.15.152:0/1393945148 conn(0x7f78500b21e0 msgr2=0x7f785015d060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:37.963 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:37.964+0000 7f7856e00640 1 -- 172.21.15.152:0/1393945148 shutdown_connections 2024-04-20T11:37:37.964 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:37.964+0000 7f7856e00640 1 -- 172.21.15.152:0/1393945148 wait complete. 2024-04-20T11:37:37.991 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-20T11:37:38.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.180+0000 7f4f3c000640 1 Processor -- start 2024-04-20T11:37:38.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.180+0000 7f4f3c000640 1 -- start start 2024-04-20T11:37:38.182 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.182+0000 7f4f3c000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4f34146000 0x7f4f340b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:38.182 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.182+0000 7f4f3c000640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f4f340b2170 0x7f4f340b25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:38.182 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.182+0000 7f4f3c000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f4f340cd1b0 0x7f4f340cd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:38.182 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.182+0000 7f4f3c000640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f4f340b4750 con 0x7f4f340cd1b0 2024-04-20T11:37:38.182 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.182+0000 7f4f3c000640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f4f341463e0 con 0x7f4f340b2170 2024-04-20T11:37:38.182 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.182+0000 7f4f3c000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4f340b3e20 con 0x7f4f34146000 2024-04-20T11:37:38.182 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.182+0000 7f4f3a000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f4f340cd1b0 0x7f4f340cd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:38.182 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.182+0000 7f4f3a000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f4f340cd1b0 0x7f4f340cd5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:59724/0 (socket says 172.21.15.152:59724) 2024-04-20T11:37:38.182 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.182+0000 7f4f3a000640 1 -- 172.21.15.152:0/1332918148 learned_addr learned my addr 172.21.15.152:0/1332918148 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:38.182 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.182+0000 7f4f38c00640 1 --2- 172.21.15.152:0/1332918148 >> v2:172.21.15.175:3300/0 conn(0x7f4f340b2170 0x7f4f340b25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:38.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.182+0000 7f4f39600640 1 --2- 172.21.15.152:0/1332918148 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4f34146000 0x7f4f340b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:38.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.183+0000 7f4f3a000640 1 -- 172.21.15.152:0/1332918148 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4f34146000 msgr2=0x7f4f340b4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:38.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.183+0000 7f4f3a000640 1 --2- 172.21.15.152:0/1332918148 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4f34146000 0x7f4f340b4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:38.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.183+0000 7f4f3a000640 1 -- 172.21.15.152:0/1332918148 >> v2:172.21.15.175:3300/0 conn(0x7f4f340b2170 msgr2=0x7f4f340b25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:38.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.183+0000 7f4f3a000640 1 --2- 172.21.15.152:0/1332918148 >> v2:172.21.15.175:3300/0 conn(0x7f4f340b2170 0x7f4f340b25f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:38.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.183+0000 7f4f3a000640 1 -- 172.21.15.152:0/1332918148 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f340b4100 con 0x7f4f340cd1b0 2024-04-20T11:37:38.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.183+0000 7f4f38c00640 1 --2- 172.21.15.152:0/1332918148 >> v2:172.21.15.175:3300/0 conn(0x7f4f340b2170 0x7f4f340b25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:37:38.184 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.184+0000 7f4f3a000640 1 --2- 172.21.15.152:0/1332918148 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4f340cd1b0 0x7f4f340cd5c0 secure :-1 s=READY pgs=2259 cs=0 l=1 rev1=1 crypto rx=0x7f4f3406ffb0 tx=0x7f4f2c02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd24c13cbb8585f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:38.184 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.185+0000 7f4f33e00640 1 -- 172.21.15.152:0/1332918148 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f2c02c9e0 con 0x7f4f340cd1b0 2024-04-20T11:37:38.184 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.185+0000 7f4f33e00640 1 -- 172.21.15.152:0/1332918148 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f2c02cb80 con 0x7f4f340cd1b0 2024-04-20T11:37:38.184 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.185+0000 7f4f33e00640 1 -- 172.21.15.152:0/1332918148 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f2c0343f0 con 0x7f4f340cd1b0 2024-04-20T11:37:38.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.185+0000 7f4f3c000640 1 -- 172.21.15.152:0/1332918148 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4f340cd1b0 msgr2=0x7f4f340cd5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:38.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.185+0000 7f4f3c000640 1 --2- 172.21.15.152:0/1332918148 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4f340cd1b0 0x7f4f340cd5c0 secure :-1 s=READY pgs=2259 cs=0 l=1 rev1=1 crypto rx=0x7f4f3406ffb0 tx=0x7f4f2c02be40 comp rx=0 tx=0).stop 2024-04-20T11:37:38.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.185+0000 7f4f3c000640 1 -- 172.21.15.152:0/1332918148 shutdown_connections 2024-04-20T11:37:38.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.185+0000 7f4f3c000640 1 --2- 172.21.15.152:0/1332918148 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4f340cd1b0 0x7f4f340cd5c0 unknown :-1 s=CLOSED pgs=2259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:38.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.185+0000 7f4f3c000640 1 --2- 172.21.15.152:0/1332918148 >> v2:172.21.15.175:3300/0 conn(0x7f4f340b2170 0x7f4f340b25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:38.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.185+0000 7f4f3c000640 1 --2- 172.21.15.152:0/1332918148 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4f34146000 0x7f4f340b4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:38.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.185+0000 7f4f3c000640 1 -- 172.21.15.152:0/1332918148 >> 172.21.15.152:0/1332918148 conn(0x7f4f3406b980 msgr2=0x7f4f3406bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:38.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.186+0000 7f4f3c000640 1 -- 172.21.15.152:0/1332918148 shutdown_connections 2024-04-20T11:37:38.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.186+0000 7f4f3c000640 1 -- 172.21.15.152:0/1332918148 wait complete. 2024-04-20T11:37:38.187 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.187+0000 7f4f3c000640 1 Processor -- start 2024-04-20T11:37:38.187 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.188+0000 7f4f3c000640 1 -- start start 2024-04-20T11:37:38.188 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.188+0000 7f4f3c000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4f340b2170 0x7f4f3413b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:38.188 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.188+0000 7f4f3c000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4f340cd1b0 0x7f4f3415f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:38.188 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.188+0000 7f4f3c000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4f34146000 0x7f4f3415f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:38.188 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.189+0000 7f4f3c000640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f4f340cdc80 con 0x7f4f34146000 2024-04-20T11:37:38.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.189+0000 7f4f3c000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4f340cdb00 con 0x7f4f340b2170 2024-04-20T11:37:38.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.189+0000 7f4f3c000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4f3415ccf0 con 0x7f4f340cd1b0 2024-04-20T11:37:38.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.189+0000 7f4f3a000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4f34146000 0x7f4f3415f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:38.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.189+0000 7f4f3a000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4f34146000 0x7f4f3415f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:59732/0 (socket says 172.21.15.152:59732) 2024-04-20T11:37:38.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.189+0000 7f4f3a000640 1 -- 172.21.15.152:0/3986326221 learned_addr learned my addr 172.21.15.152:0/3986326221 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:38.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.189+0000 7f4f38c00640 1 --2- 172.21.15.152:0/3986326221 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4f340cd1b0 0x7f4f3415f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:38.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.189+0000 7f4f39600640 1 --2- 172.21.15.152:0/3986326221 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4f340b2170 0x7f4f3413b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:38.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.189+0000 7f4f3a000640 1 -- 172.21.15.152:0/3986326221 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4f340cd1b0 msgr2=0x7f4f3415f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:38.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.189+0000 7f4f3a000640 1 --2- 172.21.15.152:0/3986326221 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4f340cd1b0 0x7f4f3415f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:38.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.189+0000 7f4f3a000640 1 -- 172.21.15.152:0/3986326221 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4f340b2170 msgr2=0x7f4f3413b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:38.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.189+0000 7f4f3a000640 1 --2- 172.21.15.152:0/3986326221 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4f340b2170 0x7f4f3413b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:38.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.189+0000 7f4f3a000640 1 -- 172.21.15.152:0/3986326221 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f3413f470 con 0x7f4f34146000 2024-04-20T11:37:38.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.190+0000 7f4f3a000640 1 --2- 172.21.15.152:0/3986326221 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4f34146000 0x7f4f3415f5a0 secure :-1 s=READY pgs=2260 cs=0 l=1 rev1=1 crypto rx=0x7f4f2c036040 tx=0x7f4f2c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:38.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.190+0000 7f4f31600640 1 -- 172.21.15.152:0/3986326221 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f2c043020 con 0x7f4f34146000 2024-04-20T11:37:38.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.190+0000 7f4f31600640 1 -- 172.21.15.152:0/3986326221 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f2c03e450 con 0x7f4f34146000 2024-04-20T11:37:38.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.190+0000 7f4f31600640 1 -- 172.21.15.152:0/3986326221 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f2c009510 con 0x7f4f34146000 2024-04-20T11:37:38.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.190+0000 7f4f3c000640 1 -- 172.21.15.152:0/3986326221 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f3413b970 con 0x7f4f34146000 2024-04-20T11:37:38.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.190+0000 7f4f3c000640 1 -- 172.21.15.152:0/3986326221 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f341f57e0 con 0x7f4f34146000 2024-04-20T11:37:38.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.191+0000 7f4f3c000640 1 -- 172.21.15.152:0/3986326221 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f340689f0 con 0x7f4f34146000 2024-04-20T11:37:38.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.192+0000 7f4f31600640 1 -- 172.21.15.152:0/3986326221 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 26) ==== 91907+0+0 (secure 0 0 0) 0x7f4f2c046020 con 0x7f4f34146000 2024-04-20T11:37:38.193 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.193+0000 7f4f31600640 1 --2- 172.21.15.152:0/3986326221 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4f1406d280 0x7f4f1406f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:38.193 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.193+0000 7f4f31600640 1 -- 172.21.15.152:0/3986326221 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(159..159 src has 1..159) ==== 4943+0+0 (secure 0 0 0) 0x7f4f2c0acb40 con 0x7f4f34146000 2024-04-20T11:37:38.193 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.194+0000 7f4f39600640 1 --2- 172.21.15.152:0/3986326221 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4f1406d280 0x7f4f1406f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:38.195 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.195+0000 7f4f39600640 1 --2- 172.21.15.152:0/3986326221 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4f1406d280 0x7f4f1406f740 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f4f28006fd0 tx=0x7f4f28008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:38.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.198+0000 7f4f31600640 1 -- 172.21.15.152:0/3986326221 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f4f2c04d030 con 0x7f4f34146000 2024-04-20T11:37:38.399 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.400+0000 7f4f3c000640 1 -- 172.21.15.152:0/3986326221 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f4f340b2e90 con 0x7f4f34146000 2024-04-20T11:37:38.404 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.405+0000 7f4f31600640 1 -- 172.21.15.152:0/3986326221 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v29) ==== 104+0+0 (secure 0 0 0) 0x7f4f3413f470 con 0x7f4f34146000 2024-04-20T11:37:38.408 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.407+0000 7f4f3c000640 1 -- 172.21.15.152:0/3986326221 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4f1406d280 msgr2=0x7f4f1406f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:38.408 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.407+0000 7f4f3c000640 1 --2- 172.21.15.152:0/3986326221 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4f1406d280 0x7f4f1406f740 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f4f28006fd0 tx=0x7f4f28008040 comp rx=0 tx=0).stop 2024-04-20T11:37:38.408 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.407+0000 7f4f3c000640 1 -- 172.21.15.152:0/3986326221 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4f34146000 msgr2=0x7f4f3415f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:38.408 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.407+0000 7f4f3c000640 1 --2- 172.21.15.152:0/3986326221 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4f34146000 0x7f4f3415f5a0 secure :-1 s=READY pgs=2260 cs=0 l=1 rev1=1 crypto rx=0x7f4f2c036040 tx=0x7f4f2c004a40 comp rx=0 tx=0).stop 2024-04-20T11:37:38.408 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.408+0000 7f4f3c000640 1 -- 172.21.15.152:0/3986326221 shutdown_connections 2024-04-20T11:37:38.408 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.408+0000 7f4f3c000640 1 --2- 172.21.15.152:0/3986326221 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4f1406d280 0x7f4f1406f740 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:38.408 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.408+0000 7f4f3c000640 1 --2- 172.21.15.152:0/3986326221 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4f34146000 0x7f4f3415f5a0 unknown :-1 s=CLOSED pgs=2260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:38.408 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.408+0000 7f4f3c000640 1 --2- 172.21.15.152:0/3986326221 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4f340cd1b0 0x7f4f3415f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:38.408 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.408+0000 7f4f3c000640 1 --2- 172.21.15.152:0/3986326221 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4f340b2170 0x7f4f3413b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:38.408 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.408+0000 7f4f3c000640 1 -- 172.21.15.152:0/3986326221 >> 172.21.15.152:0/3986326221 conn(0x7f4f3406b980 msgr2=0x7f4f3406bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:38.408 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.408+0000 7f4f3c000640 1 -- 172.21.15.152:0/3986326221 shutdown_connections 2024-04-20T11:37:38.408 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.408+0000 7f4f3c000640 1 -- 172.21.15.152:0/3986326221 wait complete. 2024-04-20T11:37:38.427 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_v1_to_v2' 2024-04-20T11:37:38.623 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.623+0000 7f8475e00640 1 Processor -- start 2024-04-20T11:37:38.623 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.623+0000 7f8475e00640 1 -- start start 2024-04-20T11:37:38.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.625+0000 7f8475e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f847013e530 0x7f847006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:38.625 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.625+0000 7f8475e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f84701414e0 0x7f8470071cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:38.625 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.625+0000 7f8475e00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84700cf0a0 con 0x7f847006f340 2024-04-20T11:37:38.625 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.625+0000 7f8475e00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f8470141b00 con 0x7f84701414e0 2024-04-20T11:37:38.625 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.625+0000 7f8475e00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f84700cfc50 con 0x7f847013e530 2024-04-20T11:37:38.625 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.625+0000 7f8475400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f84701414e0 0x7f8470071cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:38.625 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.625+0000 7f846fe00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f847006f340 0x7f8470140dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:41830/0 (socket says 172.21.15.152:41830) 2024-04-20T11:37:38.625 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.625+0000 7f846fe00640 1 -- 172.21.15.152:0/4183552749 learned_addr learned my addr 172.21.15.152:0/4183552749 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:37:38.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.626+0000 7f8474a00640 1 --2- 172.21.15.152:0/4183552749 >> v2:172.21.15.175:3300/0 conn(0x7f847013e530 0x7f847006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:38.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.626+0000 7f8475400640 1 -- 172.21.15.152:0/4183552749 >> v2:172.21.15.175:3300/0 conn(0x7f847013e530 msgr2=0x7f847006ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:38.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.626+0000 7f8475400640 1 --2- 172.21.15.152:0/4183552749 >> v2:172.21.15.175:3300/0 conn(0x7f847013e530 0x7f847006ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:38.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.626+0000 7f8475400640 1 -- 172.21.15.152:0/4183552749 >> v1:172.21.15.152:6789/0 conn(0x7f847006f340 legacy=0x7f8470140dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:38.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.626+0000 7f8475400640 1 -- 172.21.15.152:0/4183552749 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84700d00e0 con 0x7f84701414e0 2024-04-20T11:37:38.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.626+0000 7f8474a00640 1 --2- 172.21.15.152:0/4183552749 >> v2:172.21.15.175:3300/0 conn(0x7f847013e530 0x7f847006ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:37:38.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.626+0000 7f846f400640 1 -- 172.21.15.152:0/4183552749 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1513927492 0 0) 0x7f845c001110 con 0x7f847006f340 2024-04-20T11:37:38.627 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.627+0000 7f8475400640 1 --2- 172.21.15.152:0/4183552749 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f84701414e0 0x7f8470071cd0 secure :-1 s=READY pgs=2262 cs=0 l=1 rev1=1 crypto rx=0x7f8470070110 tx=0x7f846802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e055cb3fbfd1f71a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:38.627 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.627+0000 7f846f400640 1 -- 172.21.15.152:0/4183552749 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8468039070 con 0x7f84701414e0 2024-04-20T11:37:38.627 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.627+0000 7f846f400640 1 -- 172.21.15.152:0/4183552749 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f846802cc80 con 0x7f84701414e0 2024-04-20T11:37:38.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.627+0000 7f846f400640 1 -- 172.21.15.152:0/4183552749 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8468034660 con 0x7f84701414e0 2024-04-20T11:37:38.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.628+0000 7f8475e00640 1 -- 172.21.15.152:0/4183552749 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f84701414e0 msgr2=0x7f8470071cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:38.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.628+0000 7f8475e00640 1 --2- 172.21.15.152:0/4183552749 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f84701414e0 0x7f8470071cd0 secure :-1 s=READY pgs=2262 cs=0 l=1 rev1=1 crypto rx=0x7f8470070110 tx=0x7f846802bed0 comp rx=0 tx=0).stop 2024-04-20T11:37:38.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.628+0000 7f8475e00640 1 -- 172.21.15.152:0/4183552749 shutdown_connections 2024-04-20T11:37:38.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.628+0000 7f8475e00640 1 --2- 172.21.15.152:0/4183552749 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f84701414e0 0x7f8470071cd0 unknown :-1 s=CLOSED pgs=2262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:38.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.628+0000 7f8475e00640 1 --2- 172.21.15.152:0/4183552749 >> v2:172.21.15.175:3300/0 conn(0x7f847013e530 0x7f847006ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:38.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.628+0000 7f8475e00640 1 -- 172.21.15.152:0/4183552749 >> 172.21.15.152:0/4183552749 conn(0x7f8470067ae0 msgr2=0x7f847006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:38.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.628+0000 7f8475e00640 1 -- 172.21.15.152:0/4183552749 shutdown_connections 2024-04-20T11:37:38.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.628+0000 7f8475e00640 1 -- 172.21.15.152:0/4183552749 wait complete. 2024-04-20T11:37:38.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.630+0000 7f8475e00640 1 Processor -- start 2024-04-20T11:37:38.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.630+0000 7f8475e00640 1 -- start start 2024-04-20T11:37:38.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.631+0000 7f8475e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f847006f340 0x7f84701f3e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:38.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.631+0000 7f8475e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f847013e530 0x7f8470141d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:38.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.631+0000 7f8475e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f84701414e0 0x7f847015d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:38.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.631+0000 7f8475e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f84700cf8e0 con 0x7f847006f340 2024-04-20T11:37:38.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.631+0000 7f8475e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f84700cf600 con 0x7f84701414e0 2024-04-20T11:37:38.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.631+0000 7f8475e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8470072210 con 0x7f847013e530 2024-04-20T11:37:38.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.631+0000 7f8474a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f847006f340 0x7f84701f3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:38.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.631+0000 7f8474a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f847006f340 0x7f84701f3e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:59746/0 (socket says 172.21.15.152:59746) 2024-04-20T11:37:38.632 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.631+0000 7f8474a00640 1 -- 172.21.15.152:0/1633801495 learned_addr learned my addr 172.21.15.152:0/1633801495 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:38.632 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.631+0000 7f8474a00640 1 -- 172.21.15.152:0/1633801495 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f847013e530 msgr2=0x7f8470141d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:38.632 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.631+0000 7f8474a00640 1 --2- 172.21.15.152:0/1633801495 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f847013e530 0x7f8470141d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:38.632 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.631+0000 7f8474a00640 1 -- 172.21.15.152:0/1633801495 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f84701414e0 msgr2=0x7f847015d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:38.632 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.631+0000 7f8474a00640 1 --2- 172.21.15.152:0/1633801495 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f84701414e0 0x7f847015d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:38.632 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.631+0000 7f8474a00640 1 -- 172.21.15.152:0/1633801495 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84700b1cb0 con 0x7f847006f340 2024-04-20T11:37:38.632 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.632+0000 7f8474a00640 1 --2- 172.21.15.152:0/1633801495 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f847006f340 0x7f84701f3e10 secure :-1 s=READY pgs=2263 cs=0 l=1 rev1=1 crypto rx=0x7f846400c7f0 tx=0x7f846400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:38.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.632+0000 7f846cc00640 1 -- 172.21.15.152:0/1633801495 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8464015070 con 0x7f847006f340 2024-04-20T11:37:38.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.632+0000 7f846cc00640 1 -- 172.21.15.152:0/1633801495 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8464007a30 con 0x7f847006f340 2024-04-20T11:37:38.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.632+0000 7f846cc00640 1 -- 172.21.15.152:0/1633801495 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8464007e00 con 0x7f847006f340 2024-04-20T11:37:38.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.632+0000 7f8475e00640 1 -- 172.21.15.152:0/1633801495 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84700b1f90 con 0x7f847006f340 2024-04-20T11:37:38.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.632+0000 7f8475e00640 1 -- 172.21.15.152:0/1633801495 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84700b2550 con 0x7f847006f340 2024-04-20T11:37:38.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.633+0000 7f8475e00640 1 -- 172.21.15.152:0/1633801495 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8470075e60 con 0x7f847006f340 2024-04-20T11:37:38.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.634+0000 7f846cc00640 1 -- 172.21.15.152:0/1633801495 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 26) ==== 91907+0+0 (secure 0 0 0) 0x7f8464018070 con 0x7f847006f340 2024-04-20T11:37:38.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.635+0000 7f846cc00640 1 --2- 172.21.15.152:0/1633801495 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f845006d280 0x7f845006f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:38.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.635+0000 7f846cc00640 1 -- 172.21.15.152:0/1633801495 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(159..159 src has 1..159) ==== 4943+0+0 (secure 0 0 0) 0x7f846408a2d0 con 0x7f847006f340 2024-04-20T11:37:38.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.636+0000 7f846fe00640 1 --2- 172.21.15.152:0/1633801495 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f845006d280 0x7f845006f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:38.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.637+0000 7f846fe00640 1 --2- 172.21.15.152:0/1633801495 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f845006d280 0x7f845006f740 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f8470138550 tx=0x7f845c001330 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:38.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.640+0000 7f846cc00640 1 -- 172.21.15.152:0/1633801495 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f846405a300 con 0x7f847006f340 2024-04-20T11:37:38.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.918+0000 7f8475e00640 1 -- 172.21.15.152:0/1633801495 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_v1_to_v2"]} v 0) -- 0x7f84700b5570 con 0x7f847006f340 2024-04-20T11:37:38.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.958+0000 7f846cc00640 1 -- 172.21.15.152:0/1633801495 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_v1_to_v2"]}]=0 v571) ==== 145+0+0 (secure 0 0 0) 0x7f8464059c00 con 0x7f847006f340 2024-04-20T11:37:38.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.961+0000 7f8475e00640 1 -- 172.21.15.152:0/1633801495 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f845006d280 msgr2=0x7f845006f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:38.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.961+0000 7f8475e00640 1 --2- 172.21.15.152:0/1633801495 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f845006d280 0x7f845006f740 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f8470138550 tx=0x7f845c001330 comp rx=0 tx=0).stop 2024-04-20T11:37:38.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.961+0000 7f8475e00640 1 -- 172.21.15.152:0/1633801495 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f847006f340 msgr2=0x7f84701f3e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:38.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.961+0000 7f8475e00640 1 --2- 172.21.15.152:0/1633801495 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f847006f340 0x7f84701f3e10 secure :-1 s=READY pgs=2263 cs=0 l=1 rev1=1 crypto rx=0x7f846400c7f0 tx=0x7f846400cbf0 comp rx=0 tx=0).stop 2024-04-20T11:37:38.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.962+0000 7f8475e00640 1 -- 172.21.15.152:0/1633801495 shutdown_connections 2024-04-20T11:37:38.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.962+0000 7f8475e00640 1 --2- 172.21.15.152:0/1633801495 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f845006d280 0x7f845006f740 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:38.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.962+0000 7f8475e00640 1 --2- 172.21.15.152:0/1633801495 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f84701414e0 0x7f847015d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:38.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.962+0000 7f8475e00640 1 --2- 172.21.15.152:0/1633801495 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f847013e530 0x7f8470141d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:38.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.962+0000 7f8475e00640 1 --2- 172.21.15.152:0/1633801495 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f847006f340 0x7f84701f3e10 unknown :-1 s=CLOSED pgs=2263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:38.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.962+0000 7f8475e00640 1 -- 172.21.15.152:0/1633801495 >> 172.21.15.152:0/1633801495 conn(0x7f8470067ae0 msgr2=0x7f84700b3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:38.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.962+0000 7f8475e00640 1 -- 172.21.15.152:0/1633801495 shutdown_connections 2024-04-20T11:37:38.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:38.962+0000 7f8475e00640 1 -- 172.21.15.152:0/1633801495 wait complete. 2024-04-20T11:37:38.979 INFO:tasks.cephfs_test_runner:poor man's upgrade test -- theme continues... ... ok 2024-04-20T11:37:38.980 INFO:tasks.cephfs_test_runner:Starting test: test_unparseable_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc) Validate unparseable text .meta file on legacy subvol root doesn't break the system 2024-04-20T11:37:38.980 INFO:tasks.cephfs_test_runner:test_unparseable_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc) 2024-04-20T11:37:38.980 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:37:39.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.173+0000 7fef37a00640 1 Processor -- start 2024-04-20T11:37:39.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.174+0000 7fef37a00640 1 -- start start 2024-04-20T11:37:39.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.175+0000 7fef37a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fef3015eae0 0x7fef300d2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:39.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.175+0000 7fef37a00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fef300d1f60 0x7fef300b7a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:39.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.175+0000 7fef37a00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef300d3190 con 0x7fef300ba210 2024-04-20T11:37:39.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.175+0000 7fef37a00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fef3015eec0 con 0x7fef3015eae0 2024-04-20T11:37:39.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.175+0000 7fef37a00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fef300d3410 con 0x7fef300d1f60 2024-04-20T11:37:39.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.176+0000 7fef35000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fef3015eae0 0x7fef300d2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:39.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.176+0000 7fef35000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fef3015eae0 0x7fef300d2c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:59750/0 (socket says 172.21.15.152:59750) 2024-04-20T11:37:39.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.176+0000 7fef35000640 1 -- 172.21.15.152:0/84378418 learned_addr learned my addr 172.21.15.152:0/84378418 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:39.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.176+0000 7fef35a00640 1 --2- 172.21.15.152:0/84378418 >> v2:172.21.15.175:3300/0 conn(0x7fef300d1f60 0x7fef300b7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:39.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.176+0000 7fef35000640 1 -- 172.21.15.152:0/84378418 >> v2:172.21.15.175:3300/0 conn(0x7fef300d1f60 msgr2=0x7fef300b7a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:39.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.176+0000 7fef35000640 1 --2- 172.21.15.152:0/84378418 >> v2:172.21.15.175:3300/0 conn(0x7fef300d1f60 0x7fef300b7a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:39.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.176+0000 7fef35000640 1 -- 172.21.15.152:0/84378418 >> v1:172.21.15.175:6789/0 conn(0x7fef300ba210 legacy=0x7fef300d1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:39.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.176+0000 7fef35000640 1 -- 172.21.15.152:0/84378418 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef300d36f0 con 0x7fef3015eae0 2024-04-20T11:37:39.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.177+0000 7fef35000640 1 --2- 172.21.15.152:0/84378418 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef3015eae0 0x7fef300d2c50 secure :-1 s=READY pgs=2264 cs=0 l=1 rev1=1 crypto rx=0x7fef300b9fb0 tx=0x7fef2002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38675079ac0c6e6c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:39.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.178+0000 7fef2f400640 1 -- 172.21.15.152:0/84378418 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef20035020 con 0x7fef3015eae0 2024-04-20T11:37:39.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.178+0000 7fef2f400640 1 -- 172.21.15.152:0/84378418 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef20031870 con 0x7fef3015eae0 2024-04-20T11:37:39.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.178+0000 7fef2f400640 1 -- 172.21.15.152:0/84378418 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef20031c40 con 0x7fef3015eae0 2024-04-20T11:37:39.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.178+0000 7fef37a00640 1 -- 172.21.15.152:0/84378418 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef3015eae0 msgr2=0x7fef300d2c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:39.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.178+0000 7fef37a00640 1 --2- 172.21.15.152:0/84378418 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef3015eae0 0x7fef300d2c50 secure :-1 s=READY pgs=2264 cs=0 l=1 rev1=1 crypto rx=0x7fef300b9fb0 tx=0x7fef2002b910 comp rx=0 tx=0).stop 2024-04-20T11:37:39.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.179+0000 7fef37a00640 1 -- 172.21.15.152:0/84378418 shutdown_connections 2024-04-20T11:37:39.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.179+0000 7fef37a00640 1 --2- 172.21.15.152:0/84378418 >> v2:172.21.15.175:3300/0 conn(0x7fef300d1f60 0x7fef300b7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:39.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.179+0000 7fef37a00640 1 --2- 172.21.15.152:0/84378418 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef3015eae0 0x7fef300d2c50 unknown :-1 s=CLOSED pgs=2264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:39.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.179+0000 7fef37a00640 1 -- 172.21.15.152:0/84378418 >> 172.21.15.152:0/84378418 conn(0x7fef3006d260 msgr2=0x7fef3006d670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:39.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.179+0000 7fef37a00640 1 -- 172.21.15.152:0/84378418 shutdown_connections 2024-04-20T11:37:39.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.179+0000 7fef37a00640 1 -- 172.21.15.152:0/84378418 wait complete. 2024-04-20T11:37:39.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.180+0000 7fef37a00640 1 Processor -- start 2024-04-20T11:37:39.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.181+0000 7fef37a00640 1 -- start start 2024-04-20T11:37:39.181 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.181+0000 7fef37a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef300ba210 0x7fef3014c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:39.181 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.181+0000 7fef37a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef300d1f60 0x7fef3014ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:39.181 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.181+0000 7fef37a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef3015eae0 0x7fef30150070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:39.181 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.181+0000 7fef37a00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fef300d3f90 con 0x7fef300d1f60 2024-04-20T11:37:39.181 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.181+0000 7fef37a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fef300d3cb0 con 0x7fef300ba210 2024-04-20T11:37:39.182 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.181+0000 7fef37a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fef300d44b0 con 0x7fef3015eae0 2024-04-20T11:37:39.182 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.182+0000 7fef2fe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef300d1f60 0x7fef3014ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:39.182 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.182+0000 7fef2fe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef300d1f60 0x7fef3014ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:59758/0 (socket says 172.21.15.152:59758) 2024-04-20T11:37:39.182 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.182+0000 7fef2fe00640 1 -- 172.21.15.152:0/302199657 learned_addr learned my addr 172.21.15.152:0/302199657 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:39.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.182+0000 7fef35000640 1 --2- 172.21.15.152:0/302199657 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef300ba210 0x7fef3014c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:39.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.182+0000 7fef2fe00640 1 -- 172.21.15.152:0/302199657 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef3015eae0 msgr2=0x7fef30150070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:39.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.182+0000 7fef2fe00640 1 --2- 172.21.15.152:0/302199657 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef3015eae0 0x7fef30150070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:39.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.182+0000 7fef2fe00640 1 -- 172.21.15.152:0/302199657 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef300ba210 msgr2=0x7fef3014c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:39.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.182+0000 7fef2fe00640 1 --2- 172.21.15.152:0/302199657 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef300ba210 0x7fef3014c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:39.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.182+0000 7fef2fe00640 1 -- 172.21.15.152:0/302199657 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef3014fe50 con 0x7fef300d1f60 2024-04-20T11:37:39.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.183+0000 7fef2fe00640 1 --2- 172.21.15.152:0/302199657 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef300d1f60 0x7fef3014ab50 secure :-1 s=READY pgs=2265 cs=0 l=1 rev1=1 crypto rx=0x7fef1800c7a0 tx=0x7fef1800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:39.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.183+0000 7fef2cc00640 1 -- 172.21.15.152:0/302199657 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef18017070 con 0x7fef300d1f60 2024-04-20T11:37:39.184 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.183+0000 7fef2cc00640 1 -- 172.21.15.152:0/302199657 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef18003da0 con 0x7fef300d1f60 2024-04-20T11:37:39.184 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.183+0000 7fef2cc00640 1 -- 172.21.15.152:0/302199657 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef180052b0 con 0x7fef300d1f60 2024-04-20T11:37:39.184 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.183+0000 7fef37a00640 1 -- 172.21.15.152:0/302199657 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef3014c7b0 con 0x7fef300d1f60 2024-04-20T11:37:39.184 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.183+0000 7fef37a00640 1 -- 172.21.15.152:0/302199657 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef3014cd70 con 0x7fef300d1f60 2024-04-20T11:37:39.184 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.184+0000 7fef37a00640 1 -- 172.21.15.152:0/302199657 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef300b7070 con 0x7fef300d1f60 2024-04-20T11:37:39.184 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.185+0000 7fef2cc00640 1 -- 172.21.15.152:0/302199657 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 26) ==== 91907+0+0 (secure 0 0 0) 0x7fef18005450 con 0x7fef300d1f60 2024-04-20T11:37:39.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.186+0000 7fef2cc00640 1 --2- 172.21.15.152:0/302199657 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fef0c06d280 0x7fef0c06f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:39.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.186+0000 7fef2cc00640 1 -- 172.21.15.152:0/302199657 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(159..159 src has 1..159) ==== 4943+0+0 (secure 0 0 0) 0x7fef180898e0 con 0x7fef300d1f60 2024-04-20T11:37:39.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.187+0000 7fef35000640 1 --2- 172.21.15.152:0/302199657 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fef0c06d280 0x7fef0c06f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:39.187 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.188+0000 7fef35000640 1 --2- 172.21.15.152:0/302199657 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fef0c06d280 0x7fef0c06f740 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fef20036000 tx=0x7fef2003c040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:39.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.191+0000 7fef2cc00640 1 -- 172.21.15.152:0/302199657 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fef300b7070 con 0x7fef300d1f60 2024-04-20T11:37:39.378 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.379+0000 7fef37a00640 1 -- 172.21.15.152:0/302199657 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fef30073be0 con 0x7fef300d1f60 2024-04-20T11:37:39.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.381+0000 7fef2cc00640 1 -- 172.21.15.152:0/302199657 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v159) ==== 74+0+13092 (secure 0 0 0) 0x7fef30073be0 con 0x7fef300d1f60 2024-04-20T11:37:39.381 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:37:39.381 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":159,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:37:37.944168+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.152:6833/1854325981":"2024-04-21T11:37:34.819848+0000","172.21.15.152:6832/1854325981":"2024-04-21T11:37:34.819848+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:37:39.384 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.384+0000 7fef37a00640 1 -- 172.21.15.152:0/302199657 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fef0c06d280 msgr2=0x7fef0c06f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:39.384 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.384+0000 7fef37a00640 1 --2- 172.21.15.152:0/302199657 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fef0c06d280 0x7fef0c06f740 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fef20036000 tx=0x7fef2003c040 comp rx=0 tx=0).stop 2024-04-20T11:37:39.384 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.384+0000 7fef37a00640 1 -- 172.21.15.152:0/302199657 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef300d1f60 msgr2=0x7fef3014ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:39.384 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.384+0000 7fef37a00640 1 --2- 172.21.15.152:0/302199657 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef300d1f60 0x7fef3014ab50 secure :-1 s=READY pgs=2265 cs=0 l=1 rev1=1 crypto rx=0x7fef1800c7a0 tx=0x7fef1800cc60 comp rx=0 tx=0).stop 2024-04-20T11:37:39.384 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.384+0000 7fef37a00640 1 -- 172.21.15.152:0/302199657 shutdown_connections 2024-04-20T11:37:39.384 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.384+0000 7fef37a00640 1 --2- 172.21.15.152:0/302199657 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fef0c06d280 0x7fef0c06f740 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:39.384 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.384+0000 7fef37a00640 1 --2- 172.21.15.152:0/302199657 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef3015eae0 0x7fef30150070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:39.384 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.384+0000 7fef37a00640 1 --2- 172.21.15.152:0/302199657 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef300d1f60 0x7fef3014ab50 unknown :-1 s=CLOSED pgs=2265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:39.384 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.384+0000 7fef37a00640 1 --2- 172.21.15.152:0/302199657 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef300ba210 0x7fef3014c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:39.384 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.384+0000 7fef37a00640 1 -- 172.21.15.152:0/302199657 >> 172.21.15.152:0/302199657 conn(0x7fef3006d260 msgr2=0x7fef30161060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:39.384 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.384+0000 7fef37a00640 1 -- 172.21.15.152:0/302199657 shutdown_connections 2024-04-20T11:37:39.384 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.384+0000 7fef37a00640 1 -- 172.21.15.152:0/302199657 wait complete. 2024-04-20T11:37:39.400 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T11:26:47.246571+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T11:37:39.401 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T11:37:39.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.592+0000 7fe51ca00640 1 Processor -- start 2024-04-20T11:37:39.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.592+0000 7fe51ca00640 1 -- start start 2024-04-20T11:37:39.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.594+0000 7fe51ca00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fe5180b2880 0x7fe5180b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:39.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.594+0000 7fe51ca00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5181463e0 con 0x7fe518146000 2024-04-20T11:37:39.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.594+0000 7fe51ca00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5180b3e20 con 0x7fe5180cd1b0 2024-04-20T11:37:39.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.594+0000 7fe51ca00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fe5180b4750 con 0x7fe5180b2880 2024-04-20T11:37:39.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.595+0000 7fe50d400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fe5180b2880 0x7fe5180b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:39.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.595+0000 7fe50d400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fe5180b2880 0x7fe5180b4be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:59772/0 (socket says 172.21.15.152:59772) 2024-04-20T11:37:39.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.595+0000 7fe50d400640 1 -- 172.21.15.152:0/1498612495 learned_addr learned my addr 172.21.15.152:0/1498612495 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:39.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.595+0000 7fe515e00640 1 --1- 172.21.15.152:0/1498612495 >> v1:172.21.15.152:6789/0 conn(0x7fe518146000 0x7fe5180b2170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:41832/0 (socket says 172.21.15.152:41832) 2024-04-20T11:37:39.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.595+0000 7fe50d400640 1 -- 172.21.15.152:0/1498612495 >> v1:172.21.15.175:6789/0 conn(0x7fe5180cd1b0 legacy=0x7fe5180cd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:39.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.595+0000 7fe50d400640 1 -- 172.21.15.152:0/1498612495 >> v1:172.21.15.152:6789/0 conn(0x7fe518146000 legacy=0x7fe5180b2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:39.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.595+0000 7fe50d400640 1 -- 172.21.15.152:0/1498612495 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5180b4100 con 0x7fe5180b2880 2024-04-20T11:37:39.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.596+0000 7fe50d400640 1 --2- 172.21.15.152:0/1498612495 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe5180b2880 0x7fe5180b4be0 secure :-1 s=READY pgs=2267 cs=0 l=1 rev1=1 crypto rx=0x7fe500009870 tx=0x7fe50002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2b55f61183f8d40 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:39.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.597+0000 7fe515400640 1 -- 172.21.15.152:0/1498612495 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe500034070 con 0x7fe5180b2880 2024-04-20T11:37:39.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.597+0000 7fe515400640 1 -- 172.21.15.152:0/1498612495 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe500030860 con 0x7fe5180b2880 2024-04-20T11:37:39.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.597+0000 7fe515400640 1 -- 172.21.15.152:0/1498612495 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe500030c30 con 0x7fe5180b2880 2024-04-20T11:37:39.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.597+0000 7fe51ca00640 1 -- 172.21.15.152:0/1498612495 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe5180b2880 msgr2=0x7fe5180b4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:39.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.597+0000 7fe51ca00640 1 --2- 172.21.15.152:0/1498612495 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe5180b2880 0x7fe5180b4be0 secure :-1 s=READY pgs=2267 cs=0 l=1 rev1=1 crypto rx=0x7fe500009870 tx=0x7fe50002b9a0 comp rx=0 tx=0).stop 2024-04-20T11:37:39.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.597+0000 7fe51ca00640 1 -- 172.21.15.152:0/1498612495 shutdown_connections 2024-04-20T11:37:39.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.597+0000 7fe51ca00640 1 --2- 172.21.15.152:0/1498612495 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe5180b2880 0x7fe5180b4be0 unknown :-1 s=CLOSED pgs=2267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:39.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.597+0000 7fe51ca00640 1 -- 172.21.15.152:0/1498612495 >> 172.21.15.152:0/1498612495 conn(0x7fe51806b980 msgr2=0x7fe51806bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:39.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.597+0000 7fe51ca00640 1 -- 172.21.15.152:0/1498612495 shutdown_connections 2024-04-20T11:37:39.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.597+0000 7fe51ca00640 1 -- 172.21.15.152:0/1498612495 wait complete. 2024-04-20T11:37:39.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.599+0000 7fe51ca00640 1 Processor -- start 2024-04-20T11:37:39.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.599+0000 7fe51ca00640 1 -- start start 2024-04-20T11:37:39.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.600+0000 7fe51ca00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe5180b2880 0x7fe51815ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:39.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.600+0000 7fe51ca00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe5180cd1b0 0x7fe518143a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:39.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.600+0000 7fe51ca00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe518146000 0x7fe518143fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:39.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.600+0000 7fe51ca00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fe51815ce70 con 0x7fe5180cd1b0 2024-04-20T11:37:39.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.600+0000 7fe51ca00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe51815ccf0 con 0x7fe518146000 2024-04-20T11:37:39.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.600+0000 7fe51ca00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe51815cff0 con 0x7fe5180b2880 2024-04-20T11:37:39.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.600+0000 7fe50d400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe5180cd1b0 0x7fe518143a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:39.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.600+0000 7fe50d400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe5180cd1b0 0x7fe518143a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:59776/0 (socket says 172.21.15.152:59776) 2024-04-20T11:37:39.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.600+0000 7fe50d400640 1 -- 172.21.15.152:0/2903292694 learned_addr learned my addr 172.21.15.152:0/2903292694 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:39.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.601+0000 7fe516800640 1 --2- 172.21.15.152:0/2903292694 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe518146000 0x7fe518143fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:39.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.601+0000 7fe515e00640 1 --2- 172.21.15.152:0/2903292694 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe5180b2880 0x7fe51815ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:39.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.601+0000 7fe50d400640 1 -- 172.21.15.152:0/2903292694 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe5180b2880 msgr2=0x7fe51815ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:39.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.601+0000 7fe50d400640 1 --2- 172.21.15.152:0/2903292694 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe5180b2880 0x7fe51815ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:39.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.601+0000 7fe50d400640 1 -- 172.21.15.152:0/2903292694 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe518146000 msgr2=0x7fe518143fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:39.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.601+0000 7fe50d400640 1 --2- 172.21.15.152:0/2903292694 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe518146000 0x7fe518143fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:39.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.601+0000 7fe50d400640 1 -- 172.21.15.152:0/2903292694 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe51815f320 con 0x7fe5180cd1b0 2024-04-20T11:37:39.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.601+0000 7fe515e00640 1 --2- 172.21.15.152:0/2903292694 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe5180b2880 0x7fe51815ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:37:39.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.601+0000 7fe50d400640 1 --2- 172.21.15.152:0/2903292694 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe5180cd1b0 0x7fe518143a90 secure :-1 s=READY pgs=2268 cs=0 l=1 rev1=1 crypto rx=0x7fe500000c00 tx=0x7fe500004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:39.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.602+0000 7fe50ea00640 1 -- 172.21.15.152:0/2903292694 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe500048070 con 0x7fe5180cd1b0 2024-04-20T11:37:39.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.602+0000 7fe50ea00640 1 -- 172.21.15.152:0/2903292694 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe500039d90 con 0x7fe5180cd1b0 2024-04-20T11:37:39.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.602+0000 7fe50ea00640 1 -- 172.21.15.152:0/2903292694 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe500034040 con 0x7fe5180cd1b0 2024-04-20T11:37:39.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.602+0000 7fe51ca00640 1 -- 172.21.15.152:0/2903292694 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe51813eee0 con 0x7fe5180cd1b0 2024-04-20T11:37:39.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.602+0000 7fe51ca00640 1 -- 172.21.15.152:0/2903292694 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe518140f60 con 0x7fe5180cd1b0 2024-04-20T11:37:39.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.603+0000 7fe51ca00640 1 -- 172.21.15.152:0/2903292694 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5180689f0 con 0x7fe5180cd1b0 2024-04-20T11:37:39.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.604+0000 7fe50ea00640 1 -- 172.21.15.152:0/2903292694 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 26) ==== 91907+0+0 (secure 0 0 0) 0x7fe50004b070 con 0x7fe5180cd1b0 2024-04-20T11:37:39.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.605+0000 7fe50ea00640 1 --2- 172.21.15.152:0/2903292694 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe4f406d280 0x7fe4f406f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:39.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.605+0000 7fe50ea00640 1 -- 172.21.15.152:0/2903292694 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(159..159 src has 1..159) ==== 4943+0+0 (secure 0 0 0) 0x7fe5000ac410 con 0x7fe5180cd1b0 2024-04-20T11:37:39.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.605+0000 7fe515e00640 1 --2- 172.21.15.152:0/2903292694 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe4f406d280 0x7fe4f406f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:39.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.606+0000 7fe515e00640 1 --2- 172.21.15.152:0/2903292694 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe4f406d280 0x7fe4f406f740 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fe508007fd0 tx=0x7fe508009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:39.608 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.609+0000 7fe50ea00640 1 -- 172.21.15.152:0/2903292694 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe50007c440 con 0x7fe5180cd1b0 2024-04-20T11:37:39.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.812+0000 7fe51ca00640 1 -- 172.21.15.152:0/2903292694 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe51806cd90 con 0x7fe5180cd1b0 2024-04-20T11:37:39.812 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.813+0000 7fe50ea00640 1 -- 172.21.15.152:0/2903292694 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v159) ==== 93+0+10 (secure 0 0 0) 0x7fe50007bd40 con 0x7fe5180cd1b0 2024-04-20T11:37:39.812 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 1 2024-04-20T11:37:39.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.815+0000 7fe51ca00640 1 -- 172.21.15.152:0/2903292694 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe4f406d280 msgr2=0x7fe4f406f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:39.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.815+0000 7fe51ca00640 1 --2- 172.21.15.152:0/2903292694 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe4f406d280 0x7fe4f406f740 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fe508007fd0 tx=0x7fe508009040 comp rx=0 tx=0).stop 2024-04-20T11:37:39.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.815+0000 7fe51ca00640 1 -- 172.21.15.152:0/2903292694 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe5180cd1b0 msgr2=0x7fe518143a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:39.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.815+0000 7fe51ca00640 1 --2- 172.21.15.152:0/2903292694 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe5180cd1b0 0x7fe518143a90 secure :-1 s=READY pgs=2268 cs=0 l=1 rev1=1 crypto rx=0x7fe500000c00 tx=0x7fe500004730 comp rx=0 tx=0).stop 2024-04-20T11:37:39.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.815+0000 7fe51ca00640 1 -- 172.21.15.152:0/2903292694 shutdown_connections 2024-04-20T11:37:39.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.815+0000 7fe51ca00640 1 --2- 172.21.15.152:0/2903292694 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe4f406d280 0x7fe4f406f740 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:39.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.815+0000 7fe51ca00640 1 --2- 172.21.15.152:0/2903292694 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe518146000 0x7fe518143fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:39.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.815+0000 7fe51ca00640 1 --2- 172.21.15.152:0/2903292694 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe5180cd1b0 0x7fe518143a90 unknown :-1 s=CLOSED pgs=2268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:39.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.815+0000 7fe51ca00640 1 --2- 172.21.15.152:0/2903292694 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe5180b2880 0x7fe51815ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:39.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.815+0000 7fe51ca00640 1 -- 172.21.15.152:0/2903292694 >> 172.21.15.152:0/2903292694 conn(0x7fe51806b980 msgr2=0x7fe51806bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:39.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.815+0000 7fe51ca00640 1 -- 172.21.15.152:0/2903292694 shutdown_connections 2024-04-20T11:37:39.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:39.816+0000 7fe51ca00640 1 -- 172.21.15.152:0/2903292694 wait complete. 2024-04-20T11:37:39.834 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_unparseable_metafile_on_legacy_to_v1_upgrade' 2024-04-20T11:37:40.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:40.036+0000 7eff18a00640 1 Processor -- start 2024-04-20T11:37:40.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:40.036+0000 7eff18a00640 1 -- start start 2024-04-20T11:37:40.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:40.038+0000 7eff18a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7eff1415d1e0 0x7eff14160b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:40.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:40.038+0000 7eff18a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7eff14071cd0 0x7eff14072160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:40.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:40.038+0000 7eff18a00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff140ce8c0 con 0x7eff1415d5c0 2024-04-20T11:37:40.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:40.038+0000 7eff18a00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7eff140726a0 con 0x7eff14071cd0 2024-04-20T11:37:40.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:40.038+0000 7eff18a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7eff141600b0 con 0x7eff1415d1e0 2024-04-20T11:37:40.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:40.038+0000 7eff13200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7eff14071cd0 0x7eff14072160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:40.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:40.038+0000 7eff13200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7eff14071cd0 0x7eff14072160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:59790/0 (socket says 172.21.15.152:59790) 2024-04-20T11:37:40.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:40.038+0000 7eff13200640 1 -- 172.21.15.152:0/3835674292 learned_addr learned my addr 172.21.15.152:0/3835674292 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:40.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:40.038+0000 7eff12800640 1 --2- 172.21.15.152:0/3835674292 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7eff1415d1e0 0x7eff14160b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:40.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:40.039+0000 7eff13200640 1 -- 172.21.15.152:0/3835674292 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7eff1415d1e0 msgr2=0x7eff14160b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:40.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:40.039+0000 7eff13200640 1 --2- 172.21.15.152:0/3835674292 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7eff1415d1e0 0x7eff14160b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:40.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:40.039+0000 7eff13200640 1 -- 172.21.15.152:0/3835674292 >> v1:172.21.15.175:6789/0 conn(0x7eff1415d5c0 legacy=0x7eff140cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:40.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:40.039+0000 7eff13200640 1 -- 172.21.15.152:0/3835674292 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff14160390 con 0x7eff14071cd0 2024-04-20T11:37:40.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:40.040+0000 7eff13200640 1 --2- 172.21.15.152:0/3835674292 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7eff14071cd0 0x7eff14072160 secure :-1 s=READY pgs=2269 cs=0 l=1 rev1=1 crypto rx=0x7eff0c009e50 tx=0x7eff0c02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4538df5d1cbe757c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:40.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:40.040+0000 7eff11400640 1 -- 172.21.15.152:0/3835674292 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff0c02cb00 con 0x7eff14071cd0 2024-04-20T11:37:40.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:40.040+0000 7eff11400640 1 -- 172.21.15.152:0/3835674292 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff0c02cca0 con 0x7eff14071cd0 2024-04-20T11:37:40.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:40.040+0000 7eff11400640 1 -- 172.21.15.152:0/3835674292 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff0c034610 con 0x7eff14071cd0 2024-04-20T11:37:40.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:40.041+0000 7eff18a00640 1 -- 172.21.15.152:0/3835674292 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7eff14071cd0 msgr2=0x7eff14072160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:40.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:40.041+0000 7eff18a00640 1 --2- 172.21.15.152:0/3835674292 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7eff14071cd0 0x7eff14072160 secure :-1 s=READY pgs=2269 cs=0 l=1 rev1=1 crypto rx=0x7eff0c009e50 tx=0x7eff0c02bf60 comp rx=0 tx=0).stop 2024-04-20T11:37:40.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:40.041+0000 7eff18a00640 1 -- 172.21.15.152:0/3835674292 shutdown_connections 2024-04-20T11:37:40.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:40.041+0000 7eff18a00640 1 --2- 172.21.15.152:0/3835674292 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7eff14071cd0 0x7eff14072160 unknown :-1 s=CLOSED pgs=2269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:40.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:40.041+0000 7eff18a00640 1 --2- 172.21.15.152:0/3835674292 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7eff1415d1e0 0x7eff14160b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:40.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:40.041+0000 7eff18a00640 1 -- 172.21.15.152:0/3835674292 >> 172.21.15.152:0/3835674292 conn(0x7eff14067ae0 msgr2=0x7eff1406b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:40.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:40.041+0000 7eff18a00640 1 -- 172.21.15.152:0/3835674292 shutdown_connections 2024-04-20T11:37:40.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:40.041+0000 7eff18a00640 1 -- 172.21.15.152:0/3835674292 wait complete. 2024-04-20T11:37:40.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:40.042+0000 7eff18a00640 1 Processor -- start 2024-04-20T11:37:40.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:40.043+0000 7eff18a00640 1 -- start start 2024-04-20T11:37:40.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:40.043+0000 7eff18a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff14071cd0 0x7eff141f4570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:40.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:40.044+0000 7eff18a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7eff1415d1e0 0x7eff141598f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:40.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:40.044+0000 7eff18a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7eff1415d5c0 0x7eff141583e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:40.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:40.044+0000 7eff18a00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7eff140ccd70 con 0x7eff1415d1e0 2024-04-20T11:37:40.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:40.044+0000 7eff18a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7eff140ceba0 con 0x7eff14071cd0 2024-04-20T11:37:40.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:40.044+0000 7eff18a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7eff140ccef0 con 0x7eff1415d5c0 2024-04-20T11:37:40.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:40.044+0000 7eff11e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7eff1415d1e0 0x7eff141598f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:40.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:40.044+0000 7eff11e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7eff1415d1e0 0x7eff141598f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:59798/0 (socket says 172.21.15.152:59798) 2024-04-20T11:37:40.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:40.044+0000 7eff11e00640 1 -- 172.21.15.152:0/4241387565 learned_addr learned my addr 172.21.15.152:0/4241387565 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:40.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:40.044+0000 7eff13200640 1 --2- 172.21.15.152:0/4241387565 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7eff1415d5c0 0x7eff141583e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:40.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:40.044+0000 7eff12800640 1 --2- 172.21.15.152:0/4241387565 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff14071cd0 0x7eff141f4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:40.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:40.044+0000 7eff11e00640 1 -- 172.21.15.152:0/4241387565 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7eff1415d5c0 msgr2=0x7eff141583e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:40.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:40.044+0000 7eff11e00640 1 --2- 172.21.15.152:0/4241387565 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7eff1415d5c0 0x7eff141583e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:40.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:40.044+0000 7eff11e00640 1 -- 172.21.15.152:0/4241387565 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff14071cd0 msgr2=0x7eff141f4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:40.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:40.044+0000 7eff11e00640 1 --2- 172.21.15.152:0/4241387565 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff14071cd0 0x7eff141f4570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:40.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:40.044+0000 7eff11e00640 1 -- 172.21.15.152:0/4241387565 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff141f7890 con 0x7eff1415d1e0 2024-04-20T11:37:40.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:40.045+0000 7eff11e00640 1 --2- 172.21.15.152:0/4241387565 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7eff1415d1e0 0x7eff141598f0 secure :-1 s=READY pgs=2270 cs=0 l=1 rev1=1 crypto rx=0x7eff0000c400 tx=0x7eff0000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:40.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:40.045+0000 7eff0aa00640 1 -- 172.21.15.152:0/4241387565 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff00017070 con 0x7eff1415d1e0 2024-04-20T11:37:40.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:40.045+0000 7eff0aa00640 1 -- 172.21.15.152:0/4241387565 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff000052d0 con 0x7eff1415d1e0 2024-04-20T11:37:40.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:40.045+0000 7eff0aa00640 1 -- 172.21.15.152:0/4241387565 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff00003830 con 0x7eff1415d1e0 2024-04-20T11:37:40.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:40.045+0000 7eff18a00640 1 -- 172.21.15.152:0/4241387565 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff141e5f60 con 0x7eff1415d1e0 2024-04-20T11:37:40.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:40.045+0000 7eff18a00640 1 -- 172.21.15.152:0/4241387565 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff141e6200 con 0x7eff1415d1e0 2024-04-20T11:37:40.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:40.046+0000 7eff18a00640 1 -- 172.21.15.152:0/4241387565 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff14075e60 con 0x7eff1415d1e0 2024-04-20T11:37:40.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:40.047+0000 7eff0aa00640 1 -- 172.21.15.152:0/4241387565 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 26) ==== 91907+0+0 (secure 0 0 0) 0x7eff0001a070 con 0x7eff1415d1e0 2024-04-20T11:37:40.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:40.048+0000 7eff0aa00640 1 --2- 172.21.15.152:0/4241387565 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7efef406d280 0x7efef406f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:40.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:40.048+0000 7eff0aa00640 1 -- 172.21.15.152:0/4241387565 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(159..159 src has 1..159) ==== 4943+0+0 (secure 0 0 0) 0x7eff00089630 con 0x7eff1415d1e0 2024-04-20T11:37:40.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:40.049+0000 7eff12800640 1 --2- 172.21.15.152:0/4241387565 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7efef406d280 0x7efef406f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:40.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:40.050+0000 7eff12800640 1 --2- 172.21.15.152:0/4241387565 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7efef406d280 0x7efef406f740 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7eff04006fd0 tx=0x7eff04008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:40.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:40.053+0000 7eff0aa00640 1 -- 172.21.15.152:0/4241387565 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7eff141f7890 con 0x7eff1415d1e0 2024-04-20T11:37:40.331 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:40.331+0000 7eff18a00640 1 -- 172.21.15.152:0/4241387565 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_unparseable_metafile_on_legacy_to_v1_upgrade"]} v 0) -- 0x7eff14072c90 con 0x7eff1415d1e0 2024-04-20T11:37:40.974 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:40.974+0000 7eff0aa00640 1 -- 172.21.15.152:0/4241387565 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_unparseable_metafile_on_legacy_to_v1_upgrade"]}]=0 v573) ==== 166+0+0 (secure 0 0 0) 0x7eff00059660 con 0x7eff1415d1e0 2024-04-20T11:37:40.978 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:40.978+0000 7eff18a00640 1 -- 172.21.15.152:0/4241387565 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7efef406d280 msgr2=0x7efef406f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:40.978 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:40.978+0000 7eff18a00640 1 --2- 172.21.15.152:0/4241387565 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7efef406d280 0x7efef406f740 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7eff04006fd0 tx=0x7eff04008040 comp rx=0 tx=0).stop 2024-04-20T11:37:40.979 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:40.978+0000 7eff18a00640 1 -- 172.21.15.152:0/4241387565 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7eff1415d1e0 msgr2=0x7eff141598f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:40.979 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:40.978+0000 7eff18a00640 1 --2- 172.21.15.152:0/4241387565 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7eff1415d1e0 0x7eff141598f0 secure :-1 s=READY pgs=2270 cs=0 l=1 rev1=1 crypto rx=0x7eff0000c400 tx=0x7eff0000c8c0 comp rx=0 tx=0).stop 2024-04-20T11:37:40.979 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:40.979+0000 7eff18a00640 1 -- 172.21.15.152:0/4241387565 shutdown_connections 2024-04-20T11:37:40.979 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:40.979+0000 7eff18a00640 1 --2- 172.21.15.152:0/4241387565 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7efef406d280 0x7efef406f740 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:40.979 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:40.979+0000 7eff18a00640 1 --2- 172.21.15.152:0/4241387565 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7eff1415d5c0 0x7eff141583e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:40.979 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:40.979+0000 7eff18a00640 1 --2- 172.21.15.152:0/4241387565 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7eff1415d1e0 0x7eff141598f0 unknown :-1 s=CLOSED pgs=2270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:40.979 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:40.979+0000 7eff18a00640 1 --2- 172.21.15.152:0/4241387565 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff14071cd0 0x7eff141f4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:40.979 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:40.979+0000 7eff18a00640 1 -- 172.21.15.152:0/4241387565 >> 172.21.15.152:0/4241387565 conn(0x7eff14067ae0 msgr2=0x7eff1415bb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:40.979 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:40.979+0000 7eff18a00640 1 -- 172.21.15.152:0/4241387565 shutdown_connections 2024-04-20T11:37:40.979 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:40.979+0000 7eff18a00640 1 -- 172.21.15.152:0/4241387565 wait complete. 2024-04-20T11:37:40.999 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-20T11:37:41.205 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:41.205+0000 7f6e16400640 1 Processor -- start 2024-04-20T11:37:41.205 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:41.205+0000 7f6e16400640 1 -- start start 2024-04-20T11:37:41.207 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:41.207+0000 7f6e16400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f6e10071cd0 0x7f6e100720b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:41.207 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:41.207+0000 7f6e16400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e10160220 0x7f6e101606b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:41.207 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:41.207+0000 7f6e16400640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e100725f0 con 0x7f6e1006eb30 2024-04-20T11:37:41.207 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:41.207+0000 7f6e16400640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f6e1013f8e0 con 0x7f6e10071cd0 2024-04-20T11:37:41.207 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:41.207+0000 7f6e16400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6e10160f90 con 0x7f6e10160220 2024-04-20T11:37:41.207 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:41.207+0000 7f6e15000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f6e10071cd0 0x7f6e100720b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:41.207 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:41.207+0000 7f6e15a00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f6e1006eb30 0x7f6e1006efc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:41834/0 (socket says 172.21.15.152:41834) 2024-04-20T11:37:41.207 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:41.207+0000 7f6e15a00640 1 -- 172.21.15.152:0/2097050021 learned_addr learned my addr 172.21.15.152:0/2097050021 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:37:41.208 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:41.208+0000 7f6e15000640 1 -- 172.21.15.152:0/2097050021 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e10160220 msgr2=0x7f6e101606b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:41.208 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:41.208+0000 7f6e15000640 1 --2- 172.21.15.152:0/2097050021 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e10160220 0x7f6e101606b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:41.208 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:41.208+0000 7f6e15000640 1 -- 172.21.15.152:0/2097050021 >> v1:172.21.15.152:6789/0 conn(0x7f6e1006eb30 legacy=0x7f6e1006efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:41.208 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:41.208+0000 7f6e15000640 1 -- 172.21.15.152:0/2097050021 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e1006dd90 con 0x7f6e10071cd0 2024-04-20T11:37:41.208 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:41.208+0000 7f6e0fe00640 1 -- 172.21.15.152:0/2097050021 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 605955591 0 0) 0x7f6e100725f0 con 0x7f6e1006eb30 2024-04-20T11:37:41.209 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:41.209+0000 7f6e15000640 1 --2- 172.21.15.152:0/2097050021 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6e10071cd0 0x7f6e100720b0 secure :-1 s=READY pgs=2272 cs=0 l=1 rev1=1 crypto rx=0x7f6e000097a0 tx=0x7f6e0002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cef2e7441eb77b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:41.209 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:41.209+0000 7f6e0fe00640 1 -- 172.21.15.152:0/2097050021 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e000315a0 con 0x7f6e10071cd0 2024-04-20T11:37:41.209 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:41.209+0000 7f6e0fe00640 1 -- 172.21.15.152:0/2097050021 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e00037070 con 0x7f6e10071cd0 2024-04-20T11:37:41.209 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:41.209+0000 7f6e0fe00640 1 -- 172.21.15.152:0/2097050021 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e00031960 con 0x7f6e10071cd0 2024-04-20T11:37:41.210 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:41.210+0000 7f6e16400640 1 -- 172.21.15.152:0/2097050021 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6e10071cd0 msgr2=0x7f6e100720b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:41.210 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:41.210+0000 7f6e16400640 1 --2- 172.21.15.152:0/2097050021 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6e10071cd0 0x7f6e100720b0 secure :-1 s=READY pgs=2272 cs=0 l=1 rev1=1 crypto rx=0x7f6e000097a0 tx=0x7f6e0002b840 comp rx=0 tx=0).stop 2024-04-20T11:37:41.210 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:41.210+0000 7f6e16400640 1 -- 172.21.15.152:0/2097050021 shutdown_connections 2024-04-20T11:37:41.210 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:41.210+0000 7f6e16400640 1 --2- 172.21.15.152:0/2097050021 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e10160220 0x7f6e101606b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:41.210 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:41.210+0000 7f6e16400640 1 --2- 172.21.15.152:0/2097050021 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6e10071cd0 0x7f6e100720b0 unknown :-1 s=CLOSED pgs=2272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:41.210 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:41.210+0000 7f6e16400640 1 -- 172.21.15.152:0/2097050021 >> 172.21.15.152:0/2097050021 conn(0x7f6e10067ae0 msgr2=0x7f6e1006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:41.210 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:41.210+0000 7f6e16400640 1 -- 172.21.15.152:0/2097050021 shutdown_connections 2024-04-20T11:37:41.210 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:41.210+0000 7f6e16400640 1 -- 172.21.15.152:0/2097050021 wait complete. 2024-04-20T11:37:41.211 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:41.212+0000 7f6e16400640 1 Processor -- start 2024-04-20T11:37:41.211 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:41.212+0000 7f6e16400640 1 -- start start 2024-04-20T11:37:41.213 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:41.213+0000 7f6e16400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e1006eb30 0x7f6e101f4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:41.213 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:41.213+0000 7f6e16400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6e10071cd0 0x7f6e101f7c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:41.213 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:41.213+0000 7f6e16400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e10160220 0x7f6e101f81c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:41.213 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:41.213+0000 7f6e16400640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f6e1006e630 con 0x7f6e10071cd0 2024-04-20T11:37:41.213 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:41.213+0000 7f6e16400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6e1006e350 con 0x7f6e1006eb30 2024-04-20T11:37:41.213 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:41.213+0000 7f6e16400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6e10160bf0 con 0x7f6e10160220 2024-04-20T11:37:41.213 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:41.213+0000 7f6e07e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6e10071cd0 0x7f6e101f7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:41.213 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:41.213+0000 7f6e07e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6e10071cd0 0x7f6e101f7c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:59814/0 (socket says 172.21.15.152:59814) 2024-04-20T11:37:41.213 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:41.213+0000 7f6e07e00640 1 -- 172.21.15.152:0/3194529983 learned_addr learned my addr 172.21.15.152:0/3194529983 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:41.213 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:41.213+0000 7f6e15a00640 1 --2- 172.21.15.152:0/3194529983 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e10160220 0x7f6e101f81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:41.213 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:41.213+0000 7f6e15000640 1 --2- 172.21.15.152:0/3194529983 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e1006eb30 0x7f6e101f4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:41.213 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:41.213+0000 7f6e07e00640 1 -- 172.21.15.152:0/3194529983 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e10160220 msgr2=0x7f6e101f81c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:41.213 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:41.213+0000 7f6e07e00640 1 --2- 172.21.15.152:0/3194529983 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e10160220 0x7f6e101f81c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:41.214 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:41.213+0000 7f6e07e00640 1 -- 172.21.15.152:0/3194529983 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e1006eb30 msgr2=0x7f6e101f4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:41.214 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:41.213+0000 7f6e07e00640 1 --2- 172.21.15.152:0/3194529983 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e1006eb30 0x7f6e101f4710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:41.214 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:41.213+0000 7f6e07e00640 1 -- 172.21.15.152:0/3194529983 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e100ac040 con 0x7f6e10071cd0 2024-04-20T11:37:41.214 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:41.214+0000 7f6e07e00640 1 --2- 172.21.15.152:0/3194529983 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6e10071cd0 0x7f6e101f7c80 secure :-1 s=READY pgs=2273 cs=0 l=1 rev1=1 crypto rx=0x7f6df800b410 tx=0x7f6df800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:41.215 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:41.214+0000 7f6e0d600640 1 -- 172.21.15.152:0/3194529983 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df8015020 con 0x7f6e10071cd0 2024-04-20T11:37:41.215 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:41.214+0000 7f6e0d600640 1 -- 172.21.15.152:0/3194529983 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6df8011850 con 0x7f6e10071cd0 2024-04-20T11:37:41.215 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:41.214+0000 7f6e0d600640 1 -- 172.21.15.152:0/3194529983 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df8011c40 con 0x7f6e10071cd0 2024-04-20T11:37:41.215 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:41.214+0000 7f6e16400640 1 -- 172.21.15.152:0/3194529983 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e100ac320 con 0x7f6e10071cd0 2024-04-20T11:37:41.215 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:41.214+0000 7f6e16400640 1 -- 172.21.15.152:0/3194529983 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e100ac8e0 con 0x7f6e10071cd0 2024-04-20T11:37:41.215 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:41.215+0000 7f6e16400640 1 -- 172.21.15.152:0/3194529983 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e10075e60 con 0x7f6e10071cd0 2024-04-20T11:37:41.216 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:41.216+0000 7f6e0d600640 1 -- 172.21.15.152:0/3194529983 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 26) ==== 91907+0+0 (secure 0 0 0) 0x7f6df8018020 con 0x7f6e10071cd0 2024-04-20T11:37:41.217 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:41.217+0000 7f6e0d600640 1 --2- 172.21.15.152:0/3194529983 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f6dec06d280 0x7f6dec06f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:41.217 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:41.217+0000 7f6e0d600640 1 -- 172.21.15.152:0/3194529983 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(159..159 src has 1..159) ==== 4943+0+0 (secure 0 0 0) 0x7f6df808a270 con 0x7f6e10071cd0 2024-04-20T11:37:41.217 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:41.218+0000 7f6e15000640 1 --2- 172.21.15.152:0/3194529983 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f6dec06d280 0x7f6dec06f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:41.218 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:41.219+0000 7f6e15000640 1 --2- 172.21.15.152:0/3194529983 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f6dec06d280 0x7f6dec06f740 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f6e00036000 tx=0x7f6e00008030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:41.221 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:41.221+0000 7f6e0d600640 1 -- 172.21.15.152:0/3194529983 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f6df801f020 con 0x7f6e10071cd0 2024-04-20T11:37:41.423 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:41.423+0000 7f6e16400640 1 -- 172.21.15.152:0/3194529983 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f6e100acab0 con 0x7f6e10071cd0 2024-04-20T11:37:41.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:41.429+0000 7f6e0d600640 1 -- 172.21.15.152:0/3194529983 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v30)=0 v30) ==== 122+0+0 (secure 0 0 0) 0x7f6df80172c0 con 0x7f6e10071cd0 2024-04-20T11:37:41.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:41.432+0000 7f6e16400640 1 -- 172.21.15.152:0/3194529983 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f6dec06d280 msgr2=0x7f6dec06f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:41.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:41.432+0000 7f6e16400640 1 --2- 172.21.15.152:0/3194529983 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f6dec06d280 0x7f6dec06f740 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f6e00036000 tx=0x7f6e00008030 comp rx=0 tx=0).stop 2024-04-20T11:37:41.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:41.432+0000 7f6e16400640 1 -- 172.21.15.152:0/3194529983 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6e10071cd0 msgr2=0x7f6e101f7c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:41.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:41.432+0000 7f6e16400640 1 --2- 172.21.15.152:0/3194529983 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6e10071cd0 0x7f6e101f7c80 secure :-1 s=READY pgs=2273 cs=0 l=1 rev1=1 crypto rx=0x7f6df800b410 tx=0x7f6df800b8d0 comp rx=0 tx=0).stop 2024-04-20T11:37:41.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:41.432+0000 7f6e16400640 1 -- 172.21.15.152:0/3194529983 shutdown_connections 2024-04-20T11:37:41.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:41.432+0000 7f6e16400640 1 --2- 172.21.15.152:0/3194529983 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f6dec06d280 0x7f6dec06f740 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:41.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:41.432+0000 7f6e16400640 1 --2- 172.21.15.152:0/3194529983 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e10160220 0x7f6e101f81c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:41.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:41.432+0000 7f6e16400640 1 --2- 172.21.15.152:0/3194529983 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6e10071cd0 0x7f6e101f7c80 unknown :-1 s=CLOSED pgs=2273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:41.433 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:41.432+0000 7f6e16400640 1 --2- 172.21.15.152:0/3194529983 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e1006eb30 0x7f6e101f4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:41.433 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:41.432+0000 7f6e16400640 1 -- 172.21.15.152:0/3194529983 >> 172.21.15.152:0/3194529983 conn(0x7f6e10067ae0 msgr2=0x7f6e1013ed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:41.433 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:41.432+0000 7f6e16400640 1 -- 172.21.15.152:0/3194529983 shutdown_connections 2024-04-20T11:37:41.433 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:41.432+0000 7f6e16400640 1 -- 172.21.15.152:0/3194529983 wait complete. 2024-04-20T11:37:41.450 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-20T11:37:41.450 DEBUG:teuthology.orchestra.run.smithi152:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-20T11:37:41.551 DEBUG:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-20T11:37:41.681 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-20T11:37:41.682 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-20T11:37:41.682 DEBUG:teuthology.orchestra.run.smithi152:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:37:41.739 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-20T11:37:41.739 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T11:37:41.739 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:37:41.796 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-20T11:37:41.796 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-20T11:37:41.797 DEBUG:teuthology.orchestra.run.smithi152:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:37:41.854 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-20T11:37:41.854 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T11:37:41.854 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:37:41.912 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-20T11:37:41.912 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-20T11:37:41.912 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:37:42.122 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.123+0000 7f365a600640 1 Processor -- start 2024-04-20T11:37:42.123 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.123+0000 7f365a600640 1 -- start start 2024-04-20T11:37:42.124 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.125+0000 7f365a600640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f365415b5e0 0x7f365406eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:42.125 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.125+0000 7f365a600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f36540b4c50 0x7f36540b50d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:42.125 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.125+0000 7f365a600640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f365406f340 0x7f365406f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:42.125 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.125+0000 7f365a600640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f365406fcd0 con 0x7f365415b5e0 2024-04-20T11:37:42.125 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.125+0000 7f365a600640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f365406de10 con 0x7f365406f340 2024-04-20T11:37:42.125 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.125+0000 7f365a600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f36540b5770 con 0x7f36540b4c50 2024-04-20T11:37:42.125 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.125+0000 7f3653400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f365415b5e0 0x7f365406eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:42.125 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.125+0000 7f3653400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f365415b5e0 0x7f365406eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:59826/0 (socket says 172.21.15.152:59826) 2024-04-20T11:37:42.125 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.125+0000 7f3653400640 1 -- 172.21.15.152:0/3534055871 learned_addr learned my addr 172.21.15.152:0/3534055871 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:42.126 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.125+0000 7f3653400640 1 -- 172.21.15.152:0/3534055871 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f36540b4c50 msgr2=0x7f36540b50d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:42.126 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.125+0000 7f3653400640 1 --2- 172.21.15.152:0/3534055871 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f36540b4c50 0x7f36540b50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:42.126 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.125+0000 7f3653400640 1 -- 172.21.15.152:0/3534055871 >> v2:172.21.15.175:3300/0 conn(0x7f365406f340 msgr2=0x7f365406f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:42.126 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.125+0000 7f3653400640 1 --2- 172.21.15.152:0/3534055871 >> v2:172.21.15.175:3300/0 conn(0x7f365406f340 0x7f365406f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:42.126 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.125+0000 7f3653400640 1 -- 172.21.15.152:0/3534055871 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f365406e060 con 0x7f365415b5e0 2024-04-20T11:37:42.127 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.127+0000 7f3653400640 1 --2- 172.21.15.152:0/3534055871 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f365415b5e0 0x7f365406eb40 secure :-1 s=READY pgs=2274 cs=0 l=1 rev1=1 crypto rx=0x7f36540b0090 tx=0x7f364402b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4f5ec4d27047c11 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:42.127 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.127+0000 7f3652a00640 1 -- 172.21.15.152:0/3534055871 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3644035020 con 0x7f365415b5e0 2024-04-20T11:37:42.127 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.127+0000 7f3652a00640 1 -- 172.21.15.152:0/3534055871 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3644031880 con 0x7f365415b5e0 2024-04-20T11:37:42.127 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.127+0000 7f3652a00640 1 -- 172.21.15.152:0/3534055871 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3644031c50 con 0x7f365415b5e0 2024-04-20T11:37:42.127 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.127+0000 7f365a600640 1 -- 172.21.15.152:0/3534055871 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f365415b5e0 msgr2=0x7f365406eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:42.128 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.127+0000 7f365a600640 1 --2- 172.21.15.152:0/3534055871 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f365415b5e0 0x7f365406eb40 secure :-1 s=READY pgs=2274 cs=0 l=1 rev1=1 crypto rx=0x7f36540b0090 tx=0x7f364402b920 comp rx=0 tx=0).stop 2024-04-20T11:37:42.128 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.128+0000 7f365a600640 1 -- 172.21.15.152:0/3534055871 shutdown_connections 2024-04-20T11:37:42.128 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.128+0000 7f365a600640 1 --2- 172.21.15.152:0/3534055871 >> v2:172.21.15.175:3300/0 conn(0x7f365406f340 0x7f365406f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:42.128 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.128+0000 7f365a600640 1 --2- 172.21.15.152:0/3534055871 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f36540b4c50 0x7f36540b50d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:42.128 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.128+0000 7f365a600640 1 --2- 172.21.15.152:0/3534055871 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f365415b5e0 0x7f365406eb40 unknown :-1 s=CLOSED pgs=2274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:42.128 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.128+0000 7f365a600640 1 -- 172.21.15.152:0/3534055871 >> 172.21.15.152:0/3534055871 conn(0x7f3654067cf0 msgr2=0x7f365406b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:42.128 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.128+0000 7f365a600640 1 -- 172.21.15.152:0/3534055871 shutdown_connections 2024-04-20T11:37:42.128 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.128+0000 7f365a600640 1 -- 172.21.15.152:0/3534055871 wait complete. 2024-04-20T11:37:42.129 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.129+0000 7f365a600640 1 Processor -- start 2024-04-20T11:37:42.129 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.130+0000 7f365a600640 1 -- start start 2024-04-20T11:37:42.131 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.130+0000 7f365a600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f365406f340 0x7f365415a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:42.131 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.130+0000 7f365a600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f36540b4c50 0x7f3654154080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:42.132 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.131+0000 7f365a600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f365415b5e0 0x7f3654143e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:42.132 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.131+0000 7f365a600640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f36540b42d0 con 0x7f36540b4c50 2024-04-20T11:37:42.132 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.131+0000 7f365a600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f365406e620 con 0x7f365406f340 2024-04-20T11:37:42.132 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.131+0000 7f365a600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f365415cd70 con 0x7f365415b5e0 2024-04-20T11:37:42.132 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.131+0000 7f364aa00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f36540b4c50 0x7f3654154080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:42.132 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.131+0000 7f364aa00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f36540b4c50 0x7f3654154080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:59832/0 (socket says 172.21.15.152:59832) 2024-04-20T11:37:42.132 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.131+0000 7f364aa00640 1 -- 172.21.15.152:0/3759267278 learned_addr learned my addr 172.21.15.152:0/3759267278 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:42.132 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.131+0000 7f364aa00640 1 -- 172.21.15.152:0/3759267278 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f365415b5e0 msgr2=0x7f3654143e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:42.132 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.131+0000 7f364aa00640 1 --2- 172.21.15.152:0/3759267278 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f365415b5e0 0x7f3654143e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:42.132 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.131+0000 7f364aa00640 1 -- 172.21.15.152:0/3759267278 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f365406f340 msgr2=0x7f365415a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:42.132 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.131+0000 7f3653400640 1 --2- 172.21.15.152:0/3759267278 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f365406f340 0x7f365415a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:42.132 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.131+0000 7f364aa00640 1 --2- 172.21.15.152:0/3759267278 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f365406f340 0x7f365415a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:42.132 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.131+0000 7f364aa00640 1 -- 172.21.15.152:0/3759267278 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3654159380 con 0x7f36540b4c50 2024-04-20T11:37:42.133 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.132+0000 7f364aa00640 1 --2- 172.21.15.152:0/3759267278 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f36540b4c50 0x7f3654154080 secure :-1 s=READY pgs=2275 cs=0 l=1 rev1=1 crypto rx=0x7f363c00b4a0 tx=0x7f363c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:42.133 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.132+0000 7f364be00640 1 -- 172.21.15.152:0/3759267278 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f363c014070 con 0x7f36540b4c50 2024-04-20T11:37:42.133 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.132+0000 7f364be00640 1 -- 172.21.15.152:0/3759267278 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f363c010610 con 0x7f36540b4c50 2024-04-20T11:37:42.133 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.132+0000 7f364be00640 1 -- 172.21.15.152:0/3759267278 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f363c010a00 con 0x7f36540b4c50 2024-04-20T11:37:42.133 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.132+0000 7f365a600640 1 -- 172.21.15.152:0/3759267278 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3654159660 con 0x7f36540b4c50 2024-04-20T11:37:42.133 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.132+0000 7f365a600640 1 -- 172.21.15.152:0/3759267278 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3654159c20 con 0x7f36540b4c50 2024-04-20T11:37:42.133 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.133+0000 7f365a600640 1 -- 172.21.15.152:0/3759267278 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36540689f0 con 0x7f36540b4c50 2024-04-20T11:37:42.134 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.134+0000 7f364be00640 1 -- 172.21.15.152:0/3759267278 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 26) ==== 91907+0+0 (secure 0 0 0) 0x7f363c017070 con 0x7f36540b4c50 2024-04-20T11:37:42.135 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.135+0000 7f364be00640 1 --2- 172.21.15.152:0/3759267278 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f363006d280 0x7f363006f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:42.135 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.135+0000 7f364be00640 1 -- 172.21.15.152:0/3759267278 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(159..159 src has 1..159) ==== 4943+0+0 (secure 0 0 0) 0x7f363c089d30 con 0x7f36540b4c50 2024-04-20T11:37:42.135 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.136+0000 7f3653400640 1 --2- 172.21.15.152:0/3759267278 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f363006d280 0x7f363006f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:42.136 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.137+0000 7f3653400640 1 --2- 172.21.15.152:0/3759267278 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f363006d280 0x7f363006f740 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f3644036000 tx=0x7f364403c040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:42.139 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.140+0000 7f364be00640 1 -- 172.21.15.152:0/3759267278 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f363c059d60 con 0x7f36540b4c50 2024-04-20T11:37:42.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.377+0000 7f365a600640 1 -- 172.21.15.152:0/3759267278 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3654158700 con 0x7f36540b4c50 2024-04-20T11:37:42.377 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.378+0000 7f364be00640 1 -- 172.21.15.152:0/3759267278 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 108 v108) ==== 95+0+3881 (secure 0 0 0) 0x7f363c059660 con 0x7f36540b4c50 2024-04-20T11:37:42.377 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:37:42.377 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":108,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6458,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/1688411915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":1688411915},{"type":"v1","addr":"172.21.15.152:6835","nonce":1688411915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":100},{"gid":7141,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6834/599467453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":599467453},{"type":"v1","addr":"172.21.15.175:6834","nonce":599467453}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":100},{"gid":7144,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/691089259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6833","nonce":691089259},{"type":"v1","addr":"172.21.15.175:6835","nonce":691089259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":100},{"gid":10404,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/1869089243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":1869089243},{"type":"v1","addr":"172.21.15.152:6833","nonce":1869089243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":108}],"filesystems":[]} 2024-04-20T11:37:42.378 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 108 2024-04-20T11:37:42.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.380+0000 7f365a600640 1 -- 172.21.15.152:0/3759267278 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f363006d280 msgr2=0x7f363006f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:42.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.380+0000 7f365a600640 1 --2- 172.21.15.152:0/3759267278 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f363006d280 0x7f363006f740 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f3644036000 tx=0x7f364403c040 comp rx=0 tx=0).stop 2024-04-20T11:37:42.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.380+0000 7f365a600640 1 -- 172.21.15.152:0/3759267278 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f36540b4c50 msgr2=0x7f3654154080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:42.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.380+0000 7f365a600640 1 --2- 172.21.15.152:0/3759267278 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f36540b4c50 0x7f3654154080 secure :-1 s=READY pgs=2275 cs=0 l=1 rev1=1 crypto rx=0x7f363c00b4a0 tx=0x7f363c00b960 comp rx=0 tx=0).stop 2024-04-20T11:37:42.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.380+0000 7f365a600640 1 -- 172.21.15.152:0/3759267278 shutdown_connections 2024-04-20T11:37:42.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.380+0000 7f365a600640 1 --2- 172.21.15.152:0/3759267278 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f363006d280 0x7f363006f740 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:42.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.380+0000 7f365a600640 1 --2- 172.21.15.152:0/3759267278 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f365415b5e0 0x7f3654143e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:42.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.380+0000 7f365a600640 1 --2- 172.21.15.152:0/3759267278 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f36540b4c50 0x7f3654154080 unknown :-1 s=CLOSED pgs=2275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:42.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.380+0000 7f365a600640 1 --2- 172.21.15.152:0/3759267278 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f365406f340 0x7f365415a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:42.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.380+0000 7f365a600640 1 -- 172.21.15.152:0/3759267278 >> 172.21.15.152:0/3759267278 conn(0x7f3654067cf0 msgr2=0x7f365415e830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:42.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.381+0000 7f365a600640 1 -- 172.21.15.152:0/3759267278 shutdown_connections 2024-04-20T11:37:42.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.381+0000 7f365a600640 1 -- 172.21.15.152:0/3759267278 wait complete. 2024-04-20T11:37:42.399 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-20T11:37:42.399 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-20T11:37:42.399 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-20T11:37:42.399 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T11:37:42.399 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-20T11:37:42.399 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-20T11:37:42.400 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-20T11:37:42.400 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T11:37:42.400 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-20T11:37:42.400 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-20T11:37:42.400 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-20T11:37:42.400 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T11:37:42.400 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-20T11:37:42.401 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-20T11:37:42.401 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-20T11:37:42.401 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T11:37:42.415 INFO:tasks.ceph.mds.a:Stopped 2024-04-20T11:37:42.415 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-20T11:37:42.417 INFO:tasks.ceph.mds.c:Stopped 2024-04-20T11:37:42.418 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-20T11:37:42.432 INFO:tasks.ceph.mds.d:Stopped 2024-04-20T11:37:42.433 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-20T11:37:42.437 INFO:tasks.ceph.mds.b:Stopped 2024-04-20T11:37:42.437 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-20T11:37:42.461 INFO:tasks.ceph.mds.a:Started 2024-04-20T11:37:42.462 DEBUG:teuthology.parallel:result is None 2024-04-20T11:37:42.465 INFO:tasks.ceph.mds.c:Started 2024-04-20T11:37:42.465 DEBUG:teuthology.parallel:result is None 2024-04-20T11:37:42.482 INFO:tasks.ceph.mds.d:Started 2024-04-20T11:37:42.483 DEBUG:teuthology.parallel:result is None 2024-04-20T11:37:42.489 INFO:tasks.ceph.mds.b:Started 2024-04-20T11:37:42.489 DEBUG:teuthology.parallel:result is None 2024-04-20T11:37:42.489 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-20T11:37:42.608 INFO:tasks.ceph.mds.c.smithi152.stdout:starting mds.c at 2024-04-20T11:37:42.609 INFO:tasks.ceph.mds.a.smithi152.stdout:starting mds.a at 2024-04-20T11:37:42.643 INFO:tasks.ceph.mds.d.smithi175.stdout:starting mds.d at 2024-04-20T11:37:42.649 INFO:tasks.ceph.mds.b.smithi175.stdout:starting mds.b at 2024-04-20T11:37:42.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.664+0000 7f1d2aa00640 1 Processor -- start 2024-04-20T11:37:42.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.664+0000 7f1d2aa00640 1 -- start start 2024-04-20T11:37:42.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.665+0000 7f1d2aa00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f1d2415e8a0 0x7f1d24162ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:42.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.665+0000 7f1d2aa00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f1d240b3a00 0x7f1d2406cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:42.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.665+0000 7f1d2aa00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1d240d2b70 0x7f1d240d3000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:42.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.665+0000 7f1d2aa00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f1d240b3e40 con 0x7f1d2415e8a0 2024-04-20T11:37:42.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.665+0000 7f1d2aa00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f1d240d4eb0 con 0x7f1d240b3a00 2024-04-20T11:37:42.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.665+0000 7f1d2aa00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1d240d5190 con 0x7f1d240d2b70 2024-04-20T11:37:42.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.665+0000 7f1d29600640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f1d2415e8a0 0x7f1d24162ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:42.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.665+0000 7f1d29600640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f1d2415e8a0 0x7f1d24162ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:59868/0 (socket says 172.21.15.152:59868) 2024-04-20T11:37:42.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.665+0000 7f1d29600640 1 -- 172.21.15.152:0/4223702581 learned_addr learned my addr 172.21.15.152:0/4223702581 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:42.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.666+0000 7f1d28c00640 1 --2- 172.21.15.152:0/4223702581 >> v2:172.21.15.175:3300/0 conn(0x7f1d240b3a00 0x7f1d2406cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:42.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.666+0000 7f1d2a000640 1 --2- 172.21.15.152:0/4223702581 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1d240d2b70 0x7f1d240d3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:42.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.666+0000 7f1d29600640 1 -- 172.21.15.152:0/4223702581 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1d240d2b70 msgr2=0x7f1d240d3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:42.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.666+0000 7f1d29600640 1 --2- 172.21.15.152:0/4223702581 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1d240d2b70 0x7f1d240d3000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:42.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.666+0000 7f1d29600640 1 -- 172.21.15.152:0/4223702581 >> v2:172.21.15.175:3300/0 conn(0x7f1d240b3a00 msgr2=0x7f1d2406cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:42.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.666+0000 7f1d29600640 1 --2- 172.21.15.152:0/4223702581 >> v2:172.21.15.175:3300/0 conn(0x7f1d240b3a00 0x7f1d2406cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:42.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.666+0000 7f1d29600640 1 -- 172.21.15.152:0/4223702581 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d240cf340 con 0x7f1d2415e8a0 2024-04-20T11:37:42.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.666+0000 7f1d28c00640 1 --2- 172.21.15.152:0/4223702581 >> v2:172.21.15.175:3300/0 conn(0x7f1d240b3a00 0x7f1d2406cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:37:42.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.667+0000 7f1d29600640 1 --2- 172.21.15.152:0/4223702581 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d2415e8a0 0x7f1d24162ef0 secure :-1 s=READY pgs=2281 cs=0 l=1 rev1=1 crypto rx=0x7f1d2408dee0 tx=0x7f1d1802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=879f2423a380f36b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:42.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.667+0000 7f1d23e00640 1 -- 172.21.15.152:0/4223702581 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d18034070 con 0x7f1d2415e8a0 2024-04-20T11:37:42.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.667+0000 7f1d23e00640 1 -- 172.21.15.152:0/4223702581 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d18030860 con 0x7f1d2415e8a0 2024-04-20T11:37:42.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.667+0000 7f1d23e00640 1 -- 172.21.15.152:0/4223702581 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d18030c30 con 0x7f1d2415e8a0 2024-04-20T11:37:42.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.668+0000 7f1d2aa00640 1 -- 172.21.15.152:0/4223702581 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d2415e8a0 msgr2=0x7f1d24162ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:42.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.668+0000 7f1d2aa00640 1 --2- 172.21.15.152:0/4223702581 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d2415e8a0 0x7f1d24162ef0 secure :-1 s=READY pgs=2281 cs=0 l=1 rev1=1 crypto rx=0x7f1d2408dee0 tx=0x7f1d1802b9a0 comp rx=0 tx=0).stop 2024-04-20T11:37:42.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.668+0000 7f1d2aa00640 1 -- 172.21.15.152:0/4223702581 shutdown_connections 2024-04-20T11:37:42.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.668+0000 7f1d2aa00640 1 --2- 172.21.15.152:0/4223702581 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1d240d2b70 0x7f1d240d3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:42.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.668+0000 7f1d2aa00640 1 --2- 172.21.15.152:0/4223702581 >> v2:172.21.15.175:3300/0 conn(0x7f1d240b3a00 0x7f1d2406cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:42.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.668+0000 7f1d2aa00640 1 --2- 172.21.15.152:0/4223702581 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d2415e8a0 0x7f1d24162ef0 unknown :-1 s=CLOSED pgs=2281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:42.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.668+0000 7f1d2aa00640 1 -- 172.21.15.152:0/4223702581 >> 172.21.15.152:0/4223702581 conn(0x7f1d2408bfd0 msgr2=0x7f1d240d4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:42.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.668+0000 7f1d2aa00640 1 -- 172.21.15.152:0/4223702581 shutdown_connections 2024-04-20T11:37:42.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.668+0000 7f1d2aa00640 1 -- 172.21.15.152:0/4223702581 wait complete. 2024-04-20T11:37:42.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.670+0000 7f1d2aa00640 1 Processor -- start 2024-04-20T11:37:42.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.670+0000 7f1d2aa00640 1 -- start start 2024-04-20T11:37:42.672 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.671+0000 7f1d2aa00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1d240b3a00 0x7f1d240b95a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:42.672 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.671+0000 7f1d2aa00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d240d2b70 0x7f1d240b6530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:42.672 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.671+0000 7f1d2aa00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d2415e8a0 0x7f1d2415b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:42.672 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.671+0000 7f1d2aa00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f1d240d5470 con 0x7f1d2415e8a0 2024-04-20T11:37:42.672 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.671+0000 7f1d2aa00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1d2406d080 con 0x7f1d240d2b70 2024-04-20T11:37:42.672 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.671+0000 7f1d2aa00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1d2415ec80 con 0x7f1d240b3a00 2024-04-20T11:37:42.672 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.671+0000 7f1d29600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1d240b3a00 0x7f1d240b95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:42.672 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.671+0000 7f1d2a000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d2415e8a0 0x7f1d2415b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:42.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.671+0000 7f1d29600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1d240b3a00 0x7f1d240b95a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.152:50746/0 (socket says 172.21.15.152:50746) 2024-04-20T11:37:42.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.671+0000 7f1d29600640 1 -- 172.21.15.152:0/3232466618 learned_addr learned my addr 172.21.15.152:0/3232466618 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:42.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.671+0000 7f1d28c00640 1 --2- 172.21.15.152:0/3232466618 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d240d2b70 0x7f1d240b6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:42.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.672+0000 7f1d29600640 1 -- 172.21.15.152:0/3232466618 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d240d2b70 msgr2=0x7f1d240b6530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:42.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.672+0000 7f1d29600640 1 --2- 172.21.15.152:0/3232466618 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d240d2b70 0x7f1d240b6530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:42.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.672+0000 7f1d29600640 1 -- 172.21.15.152:0/3232466618 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d2415e8a0 msgr2=0x7f1d2415b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:42.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.672+0000 7f1d29600640 1 --2- 172.21.15.152:0/3232466618 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d2415e8a0 0x7f1d2415b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:42.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.672+0000 7f1d29600640 1 -- 172.21.15.152:0/3232466618 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d240b5170 con 0x7f1d240b3a00 2024-04-20T11:37:42.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.672+0000 7f1d2a000640 1 --2- 172.21.15.152:0/3232466618 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d2415e8a0 0x7f1d2415b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:37:42.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.672+0000 7f1d29600640 1 --2- 172.21.15.152:0/3232466618 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1d240b3a00 0x7f1d240b95a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1d1802b9e0 tx=0x7f1d18004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:42.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.672+0000 7f1d21600640 1 -- 172.21.15.152:0/3232466618 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d18048070 con 0x7f1d240b3a00 2024-04-20T11:37:42.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.672+0000 7f1d21600640 1 -- 172.21.15.152:0/3232466618 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d18030660 con 0x7f1d240b3a00 2024-04-20T11:37:42.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.672+0000 7f1d2aa00640 1 -- 172.21.15.152:0/3232466618 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d240b5450 con 0x7f1d240b3a00 2024-04-20T11:37:42.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.672+0000 7f1d21600640 1 -- 172.21.15.152:0/3232466618 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d18034040 con 0x7f1d240b3a00 2024-04-20T11:37:42.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.673+0000 7f1d2aa00640 1 -- 172.21.15.152:0/3232466618 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d240b5a10 con 0x7f1d240b3a00 2024-04-20T11:37:42.674 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.674+0000 7f1d2aa00640 1 -- 172.21.15.152:0/3232466618 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d24075e60 con 0x7f1d240b3a00 2024-04-20T11:37:42.674 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.675+0000 7f1d21600640 1 -- 172.21.15.152:0/3232466618 <== mon.2 v2:172.21.15.175:3301/0 4 ==== mgrmap(e 26) ==== 91907+0+0 (secure 0 0 0) 0x7f1d1804b070 con 0x7f1d240b3a00 2024-04-20T11:37:42.675 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.676+0000 7f1d21600640 1 --2- 172.21.15.152:0/3232466618 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f1d0406d280 0x7f1d0406f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:42.675 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.676+0000 7f1d21600640 1 -- 172.21.15.152:0/3232466618 <== mon.2 v2:172.21.15.175:3301/0 5 ==== osd_map(159..159 src has 1..159) ==== 4943+0+0 (secure 0 0 0) 0x7f1d180ac2a0 con 0x7f1d240b3a00 2024-04-20T11:37:42.676 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.676+0000 7f1d28c00640 1 --2- 172.21.15.152:0/3232466618 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f1d0406d280 0x7f1d0406f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:42.676 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.677+0000 7f1d28c00640 1 --2- 172.21.15.152:0/3232466618 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f1d0406d280 0x7f1d0406f740 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f1d240ba0a0 tx=0x7f1d10008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:42.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.682+0000 7f1d21600640 1 -- 172.21.15.152:0/3232466618 <== mon.2 v2:172.21.15.175:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f1d1807bc80 con 0x7f1d240b3a00 2024-04-20T11:37:42.888 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:42.889+0000 7f1d2aa00640 1 -- 172.21.15.152:0/3232466618 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f1d240d1c80 con 0x7f1d240b3a00 2024-04-20T11:37:43.006 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.006+0000 7f1d21600640 1 -- 172.21.15.152:0/3232466618 <== mon.2 v2:172.21.15.175:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v161) ==== 97+0+0 (secure 0 0 0) 0x7f1d1807ff80 con 0x7f1d240b3a00 2024-04-20T11:37:43.006 INFO:teuthology.orchestra.run.smithi152.stderr: removed all blocklist entries 2024-04-20T11:37:43.009 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.010+0000 7f1d2aa00640 1 -- 172.21.15.152:0/3232466618 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f1d0406d280 msgr2=0x7f1d0406f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:43.009 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.010+0000 7f1d2aa00640 1 --2- 172.21.15.152:0/3232466618 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f1d0406d280 0x7f1d0406f740 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f1d240ba0a0 tx=0x7f1d10008040 comp rx=0 tx=0).stop 2024-04-20T11:37:43.010 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.010+0000 7f1d2aa00640 1 -- 172.21.15.152:0/3232466618 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1d240b3a00 msgr2=0x7f1d240b95a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:43.010 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.010+0000 7f1d2aa00640 1 --2- 172.21.15.152:0/3232466618 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1d240b3a00 0x7f1d240b95a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1d1802b9e0 tx=0x7f1d18004730 comp rx=0 tx=0).stop 2024-04-20T11:37:43.010 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.010+0000 7f1d2aa00640 1 -- 172.21.15.152:0/3232466618 shutdown_connections 2024-04-20T11:37:43.010 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.010+0000 7f1d2aa00640 1 --2- 172.21.15.152:0/3232466618 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f1d0406d280 0x7f1d0406f740 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:43.010 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.010+0000 7f1d2aa00640 1 --2- 172.21.15.152:0/3232466618 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d2415e8a0 0x7f1d2415b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:43.010 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.010+0000 7f1d2aa00640 1 --2- 172.21.15.152:0/3232466618 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d240d2b70 0x7f1d240b6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:43.011 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.010+0000 7f1d2aa00640 1 --2- 172.21.15.152:0/3232466618 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1d240b3a00 0x7f1d240b95a0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:43.011 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.010+0000 7f1d2aa00640 1 -- 172.21.15.152:0/3232466618 >> 172.21.15.152:0/3232466618 conn(0x7f1d2408bfd0 msgr2=0x7f1d240d4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:43.011 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.011+0000 7f1d2aa00640 1 -- 172.21.15.152:0/3232466618 shutdown_connections 2024-04-20T11:37:43.011 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.011+0000 7f1d2aa00640 1 -- 172.21.15.152:0/3232466618 wait complete. 2024-04-20T11:37:43.028 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-20T11:37:43.218 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.219+0000 7ffbb9600640 1 Processor -- start 2024-04-20T11:37:43.219 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.219+0000 7ffbb9600640 1 -- start start 2024-04-20T11:37:43.221 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.221+0000 7ffbb9600640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7ffbb406f510 0x7ffbb40cf030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:43.221 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.221+0000 7ffbb9600640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffbb40cfe20 con 0x7ffbb4144d40 2024-04-20T11:37:43.221 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.221+0000 7ffbb9600640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffbb415f1e0 con 0x7ffbb415fca0 2024-04-20T11:37:43.221 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.221+0000 7ffbb9600640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7ffbb415f4c0 con 0x7ffbb406f510 2024-04-20T11:37:43.221 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.221+0000 7ffbb3e00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7ffbb4144d40 0x7ffbb406ee00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:41852/0 (socket says 172.21.15.152:41852) 2024-04-20T11:37:43.221 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.221+0000 7ffbb3e00640 1 -- 172.21.15.152:0/539755479 learned_addr learned my addr 172.21.15.152:0/539755479 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:37:43.221 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.221+0000 7ffbb3400640 1 --2- 172.21.15.152:0/539755479 >> v2:172.21.15.152:3300/0 conn(0x7ffbb406f510 0x7ffbb40cf030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:43.222 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.222+0000 7ffbb2a00640 1 -- 172.21.15.152:0/539755479 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4272439137 0 0) 0x7ffbb40cfe20 con 0x7ffbb4144d40 2024-04-20T11:37:43.222 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.222+0000 7ffbb2a00640 1 -- 172.21.15.152:0/539755479 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffb98005730 con 0x7ffbb4144d40 2024-04-20T11:37:43.222 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.222+0000 7ffbb3400640 1 -- 172.21.15.152:0/539755479 >> v1:172.21.15.175:6789/0 conn(0x7ffbb415fca0 legacy=0x7ffbb40cf570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:43.222 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.222+0000 7ffbb3400640 1 -- 172.21.15.152:0/539755479 >> v1:172.21.15.152:6789/0 conn(0x7ffbb4144d40 legacy=0x7ffbb406ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:43.222 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.222+0000 7ffbb3400640 1 -- 172.21.15.152:0/539755479 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbb4145120 con 0x7ffbb406f510 2024-04-20T11:37:43.223 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.223+0000 7ffbb3400640 1 --2- 172.21.15.152:0/539755479 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffbb406f510 0x7ffbb40cf030 secure :-1 s=READY pgs=2283 cs=0 l=1 rev1=1 crypto rx=0x7ffba40097a0 tx=0x7ffba402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a431ad36e20583a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:43.223 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.223+0000 7ffbb2a00640 1 -- 172.21.15.152:0/539755479 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffba40315a0 con 0x7ffbb406f510 2024-04-20T11:37:43.223 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.223+0000 7ffbb2a00640 1 -- 172.21.15.152:0/539755479 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffba4037070 con 0x7ffbb406f510 2024-04-20T11:37:43.224 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.223+0000 7ffbb2a00640 1 -- 172.21.15.152:0/539755479 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffba4031960 con 0x7ffbb406f510 2024-04-20T11:37:43.224 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.224+0000 7ffbb9600640 1 -- 172.21.15.152:0/539755479 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffbb406f510 msgr2=0x7ffbb40cf030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:43.224 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.224+0000 7ffbb9600640 1 --2- 172.21.15.152:0/539755479 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffbb406f510 0x7ffbb40cf030 secure :-1 s=READY pgs=2283 cs=0 l=1 rev1=1 crypto rx=0x7ffba40097a0 tx=0x7ffba402b840 comp rx=0 tx=0).stop 2024-04-20T11:37:43.224 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.224+0000 7ffbb9600640 1 -- 172.21.15.152:0/539755479 shutdown_connections 2024-04-20T11:37:43.224 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.224+0000 7ffbb9600640 1 --2- 172.21.15.152:0/539755479 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffbb406f510 0x7ffbb40cf030 unknown :-1 s=CLOSED pgs=2283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:43.224 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.224+0000 7ffbb9600640 1 -- 172.21.15.152:0/539755479 >> 172.21.15.152:0/539755479 conn(0x7ffbb4067ae0 msgr2=0x7ffbb406b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:43.224 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.224+0000 7ffbb9600640 1 -- 172.21.15.152:0/539755479 shutdown_connections 2024-04-20T11:37:43.224 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.224+0000 7ffbb9600640 1 -- 172.21.15.152:0/539755479 wait complete. 2024-04-20T11:37:43.225 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.226+0000 7ffbb9600640 1 Processor -- start 2024-04-20T11:37:43.225 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.226+0000 7ffbb9600640 1 -- start start 2024-04-20T11:37:43.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.227+0000 7ffbb9600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffbb406f510 0x7ffbb415a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:43.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.227+0000 7ffbb9600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffbb4144d40 0x7ffbb41eff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:43.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.227+0000 7ffbb9600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffbb415fca0 0x7ffbb4142a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:43.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.227+0000 7ffbb9600640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7ffbb406e080 con 0x7ffbb406f510 2024-04-20T11:37:43.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.227+0000 7ffbb9600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ffbb406df00 con 0x7ffbb415fca0 2024-04-20T11:37:43.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.227+0000 7ffbb9600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ffbb406e200 con 0x7ffbb4144d40 2024-04-20T11:37:43.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.227+0000 7ffbb3e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffbb406f510 0x7ffbb415a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:43.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.227+0000 7ffbb3e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffbb406f510 0x7ffbb415a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:59904/0 (socket says 172.21.15.152:59904) 2024-04-20T11:37:43.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.227+0000 7ffbb3e00640 1 -- 172.21.15.152:0/2637772847 learned_addr learned my addr 172.21.15.152:0/2637772847 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:43.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.227+0000 7ffbb3e00640 1 -- 172.21.15.152:0/2637772847 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffbb4144d40 msgr2=0x7ffbb41eff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:43.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.227+0000 7ffbb8c00640 1 --2- 172.21.15.152:0/2637772847 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffbb415fca0 0x7ffbb4142a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:43.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.227+0000 7ffbb3e00640 1 --2- 172.21.15.152:0/2637772847 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffbb4144d40 0x7ffbb41eff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:43.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.227+0000 7ffbb3e00640 1 -- 172.21.15.152:0/2637772847 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffbb415fca0 msgr2=0x7ffbb4142a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:43.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.228+0000 7ffbb3e00640 1 --2- 172.21.15.152:0/2637772847 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffbb415fca0 0x7ffbb4142a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:43.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.228+0000 7ffbb3e00640 1 -- 172.21.15.152:0/2637772847 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbb40cf8c0 con 0x7ffbb406f510 2024-04-20T11:37:43.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.228+0000 7ffbb8c00640 1 --2- 172.21.15.152:0/2637772847 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffbb415fca0 0x7ffbb4142a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:37:43.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.228+0000 7ffbb3e00640 1 --2- 172.21.15.152:0/2637772847 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffbb406f510 0x7ffbb415a1a0 secure :-1 s=READY pgs=2284 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c00c430 tx=0x7ffb9c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:43.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.228+0000 7ffbabe00640 1 -- 172.21.15.152:0/2637772847 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb9c017070 con 0x7ffbb406f510 2024-04-20T11:37:43.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.228+0000 7ffbabe00640 1 -- 172.21.15.152:0/2637772847 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb9c005ce0 con 0x7ffbb406f510 2024-04-20T11:37:43.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.228+0000 7ffbabe00640 1 -- 172.21.15.152:0/2637772847 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb9c0072f0 con 0x7ffbb406f510 2024-04-20T11:37:43.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.228+0000 7ffbb9600640 1 -- 172.21.15.152:0/2637772847 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbb415aa10 con 0x7ffbb406f510 2024-04-20T11:37:43.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.228+0000 7ffbb9600640 1 -- 172.21.15.152:0/2637772847 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbb413ca90 con 0x7ffbb406f510 2024-04-20T11:37:43.230 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.229+0000 7ffbb9600640 1 -- 172.21.15.152:0/2637772847 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbb4075e60 con 0x7ffbb406f510 2024-04-20T11:37:43.230 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.230+0000 7ffbabe00640 1 -- 172.21.15.152:0/2637772847 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 26) ==== 91907+0+0 (secure 0 0 0) 0x7ffb9c01a070 con 0x7ffbb406f510 2024-04-20T11:37:43.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.231+0000 7ffbabe00640 1 --2- 172.21.15.152:0/2637772847 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7ffb9006d280 0x7ffb9006f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:43.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.231+0000 7ffbabe00640 1 -- 172.21.15.152:0/2637772847 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(163..163 src has 1..163) ==== 4857+0+0 (secure 0 0 0) 0x7ffb9c089780 con 0x7ffbb406f510 2024-04-20T11:37:43.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.232+0000 7ffbb3400640 1 --2- 172.21.15.152:0/2637772847 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7ffb9006d280 0x7ffb9006f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:43.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.233+0000 7ffbb3400640 1 --2- 172.21.15.152:0/2637772847 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7ffb9006d280 0x7ffb9006f740 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7ffba4036000 tx=0x7ffba4030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:43.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.236+0000 7ffbabe00640 1 -- 172.21.15.152:0/2637772847 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ffb9c059870 con 0x7ffbb406f510 2024-04-20T11:37:43.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.491+0000 7ffbb9600640 1 -- 172.21.15.152:0/2637772847 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7ffbb4141aa0 con 0x7ffbb406f510 2024-04-20T11:37:43.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.492+0000 7ffbabe00640 1 -- 172.21.15.152:0/2637772847 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v22) ==== 80+0+5304 (secure 0 0 0) 0x7ffb9c059170 con 0x7ffbb406f510 2024-04-20T11:37:43.492 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:37:43.492 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-04-20T11:37:43.492 INFO:teuthology.orchestra.run.smithi152.stdout: "auth_dump": [ 2024-04-20T11:37:43.493 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:37:43.493 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "mds.a", 2024-04-20T11:37:43.493 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBCpiNmyFRJIBAAj2aIMXdb65nqRgKFDBLLzA==", 2024-04-20T11:37:43.493 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:37:43.493 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow", 2024-04-20T11:37:43.493 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow *", 2024-04-20T11:37:43.493 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow *", 2024-04-20T11:37:43.493 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:37:43.493 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:37:43.493 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:37:43.493 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:37:43.493 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "mds.b", 2024-04-20T11:37:43.493 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBCpiNmVPytOhAAzQkvNOJaNy+MeKETWhYLyg==", 2024-04-20T11:37:43.493 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:37:43.494 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow", 2024-04-20T11:37:43.494 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow *", 2024-04-20T11:37:43.494 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow *", 2024-04-20T11:37:43.494 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:37:43.494 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:37:43.494 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:37:43.494 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:37:43.494 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "mds.c", 2024-04-20T11:37:43.494 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBCpiNm5ViKLRAAg7VBQG4hFTLYUpJMpBAaVA==", 2024-04-20T11:37:43.494 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:37:43.494 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow", 2024-04-20T11:37:43.494 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow *", 2024-04-20T11:37:43.494 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow *", 2024-04-20T11:37:43.494 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:37:43.494 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:37:43.495 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:37:43.495 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:37:43.495 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "mds.d", 2024-04-20T11:37:43.495 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBDpiNmLWhaCxAA2e4qd/7+TKenJlWICIsvGg==", 2024-04-20T11:37:43.495 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:37:43.495 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow", 2024-04-20T11:37:43.495 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow *", 2024-04-20T11:37:43.495 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow *", 2024-04-20T11:37:43.495 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:37:43.495 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:37:43.495 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:37:43.495 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:37:43.495 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "osd.0", 2024-04-20T11:37:43.495 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBHpiNmpuhyDBAAsQXX9DNvWlo0UYh53lb+FA==", 2024-04-20T11:37:43.495 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:37:43.496 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow profile osd", 2024-04-20T11:37:43.496 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile osd", 2024-04-20T11:37:43.496 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:37:43.496 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:37:43.496 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:37:43.496 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:37:43.496 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "osd.1", 2024-04-20T11:37:43.496 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBKpiNmxBiLJhAAHon1adC8G2jbYoDcWdE9jA==", 2024-04-20T11:37:43.496 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:37:43.496 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow profile osd", 2024-04-20T11:37:43.496 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile osd", 2024-04-20T11:37:43.496 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:37:43.496 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:37:43.496 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:37:43.497 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:37:43.497 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "osd.2", 2024-04-20T11:37:43.497 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBNpiNmdn0zMhAAqwIO2kKahdSzTVh8/p3FCg==", 2024-04-20T11:37:43.497 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:37:43.497 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow profile osd", 2024-04-20T11:37:43.497 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile osd", 2024-04-20T11:37:43.497 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:37:43.497 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:37:43.497 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:37:43.497 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:37:43.497 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "osd.3", 2024-04-20T11:37:43.497 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBRpiNmTPCYAhAAHFNHX/xKn21jEFHDkHMjEw==", 2024-04-20T11:37:43.497 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:37:43.497 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow profile osd", 2024-04-20T11:37:43.497 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile osd", 2024-04-20T11:37:43.498 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:37:43.498 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:37:43.498 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:37:43.498 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:37:43.498 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "osd.4", 2024-04-20T11:37:43.498 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBXpiNmM8cVFxAAJl9Hu+MvlkEiMglutI89eg==", 2024-04-20T11:37:43.498 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:37:43.498 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow profile osd", 2024-04-20T11:37:43.498 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile osd", 2024-04-20T11:37:43.498 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:37:43.498 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:37:43.498 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:37:43.498 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:37:43.498 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "osd.5", 2024-04-20T11:37:43.499 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBapiNmnGxJMRAA90P0T+VsTA8LCTcexTVr7w==", 2024-04-20T11:37:43.499 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:37:43.499 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow profile osd", 2024-04-20T11:37:43.499 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile osd", 2024-04-20T11:37:43.499 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:37:43.499 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:37:43.499 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:37:43.499 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:37:43.499 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "osd.6", 2024-04-20T11:37:43.499 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBepiNmyLXWDxAAhtHD5h478mLovE5IejAFng==", 2024-04-20T11:37:43.499 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:37:43.499 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow profile osd", 2024-04-20T11:37:43.499 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile osd", 2024-04-20T11:37:43.499 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:37:43.499 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:37:43.500 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:37:43.500 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:37:43.500 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "osd.7", 2024-04-20T11:37:43.500 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBhpiNmB3jwHBAAqco9om6r2N/GqAV6rVKiaA==", 2024-04-20T11:37:43.500 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:37:43.500 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow profile osd", 2024-04-20T11:37:43.500 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile osd", 2024-04-20T11:37:43.500 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:37:43.500 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:37:43.500 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:37:43.500 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:37:43.500 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "client.0", 2024-04-20T11:37:43.500 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBDpiNmtsBWFhAArq9A7r33CcwgMmOU2pAGPw==", 2024-04-20T11:37:43.500 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:37:43.500 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow", 2024-04-20T11:37:43.501 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow r", 2024-04-20T11:37:43.501 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-20T11:37:43.501 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:37:43.501 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:37:43.501 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:37:43.501 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "client.1", 2024-04-20T11:37:43.501 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBDpiNm6aNnJxAAB431GSGytZ0IKWMh4DMalw==", 2024-04-20T11:37:43.501 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:37:43.501 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow", 2024-04-20T11:37:43.501 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow r", 2024-04-20T11:37:43.501 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow rw", 2024-04-20T11:37:43.501 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow rwx" 2024-04-20T11:37:43.501 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:37:43.501 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:37:43.501 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:37:43.502 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "client.2", 2024-04-20T11:37:43.502 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBDpiNm5QLtHRAAa2kHTDVdgSh/B4XGZaKa5Q==", 2024-04-20T11:37:43.502 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:37:43.502 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow", 2024-04-20T11:37:43.502 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow r", 2024-04-20T11:37:43.502 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow rw", 2024-04-20T11:37:43.502 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow rwx" 2024-04-20T11:37:43.502 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:37:43.502 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:37:43.502 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:37:43.502 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "client.3", 2024-04-20T11:37:43.502 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBDpiNmw95SMBAAdodzTuPp4WdKSEf+jI8Uuw==", 2024-04-20T11:37:43.502 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:37:43.502 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow", 2024-04-20T11:37:43.502 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow r", 2024-04-20T11:37:43.503 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow rw", 2024-04-20T11:37:43.503 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow rwx" 2024-04-20T11:37:43.503 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:37:43.503 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:37:43.503 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:37:43.503 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "client.admin", 2024-04-20T11:37:43.503 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBBpiNm3xz6HhAA82+N+trDW/AmBr3q8l4ECw==", 2024-04-20T11:37:43.503 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:37:43.503 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow *", 2024-04-20T11:37:43.503 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr": "allow *", 2024-04-20T11:37:43.503 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow *", 2024-04-20T11:37:43.503 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:37:43.503 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:37:43.503 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:37:43.503 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:37:43.504 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "mgr.x", 2024-04-20T11:37:43.504 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBCpiNmuhalFxAA9dTl/zSbUovCCZAanAipww==", 2024-04-20T11:37:43.504 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:37:43.504 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow *", 2024-04-20T11:37:43.504 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile mgr", 2024-04-20T11:37:43.504 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:37:43.504 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:37:43.504 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:37:43.504 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:37:43.504 INFO:teuthology.orchestra.run.smithi152.stdout: "entity": "mgr.y", 2024-04-20T11:37:43.504 INFO:teuthology.orchestra.run.smithi152.stdout: "key": "AQBCpiNmP1+5DRAAs6bVAEOr0i6FdPmSEyo/vQ==", 2024-04-20T11:37:43.504 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": { 2024-04-20T11:37:43.504 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": "allow *", 2024-04-20T11:37:43.504 INFO:teuthology.orchestra.run.smithi152.stdout: "mon": "allow profile mgr", 2024-04-20T11:37:43.504 INFO:teuthology.orchestra.run.smithi152.stdout: "osd": "allow *" 2024-04-20T11:37:43.505 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:37:43.505 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:37:43.505 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-04-20T11:37:43.505 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-04-20T11:37:43.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.495+0000 7ffbb9600640 1 -- 172.21.15.152:0/2637772847 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7ffb9006d280 msgr2=0x7ffb9006f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:43.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.495+0000 7ffbb9600640 1 --2- 172.21.15.152:0/2637772847 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7ffb9006d280 0x7ffb9006f740 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7ffba4036000 tx=0x7ffba4030000 comp rx=0 tx=0).stop 2024-04-20T11:37:43.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.495+0000 7ffbb9600640 1 -- 172.21.15.152:0/2637772847 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffbb406f510 msgr2=0x7ffbb415a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:43.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.495+0000 7ffbb9600640 1 --2- 172.21.15.152:0/2637772847 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffbb406f510 0x7ffbb415a1a0 secure :-1 s=READY pgs=2284 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c00c430 tx=0x7ffb9c00c8f0 comp rx=0 tx=0).stop 2024-04-20T11:37:43.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.495+0000 7ffbb9600640 1 -- 172.21.15.152:0/2637772847 shutdown_connections 2024-04-20T11:37:43.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.495+0000 7ffbb9600640 1 --2- 172.21.15.152:0/2637772847 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7ffb9006d280 0x7ffb9006f740 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:43.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.495+0000 7ffbb9600640 1 --2- 172.21.15.152:0/2637772847 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffbb415fca0 0x7ffbb4142a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:43.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.495+0000 7ffbb9600640 1 --2- 172.21.15.152:0/2637772847 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffbb4144d40 0x7ffbb41eff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:43.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.495+0000 7ffbb9600640 1 --2- 172.21.15.152:0/2637772847 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffbb406f510 0x7ffbb415a1a0 unknown :-1 s=CLOSED pgs=2284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:43.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.495+0000 7ffbb9600640 1 -- 172.21.15.152:0/2637772847 >> 172.21.15.152:0/2637772847 conn(0x7ffbb4067ae0 msgr2=0x7ffbb415e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:43.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.495+0000 7ffbb9600640 1 -- 172.21.15.152:0/2637772847 shutdown_connections 2024-04-20T11:37:43.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.495+0000 7ffbb9600640 1 -- 172.21.15.152:0/2637772847 wait complete. 2024-04-20T11:37:43.514 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:37:43.714 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.714+0000 7feca9200640 1 Processor -- start 2024-04-20T11:37:43.714 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.714+0000 7feca9200640 1 -- start start 2024-04-20T11:37:43.715 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.716+0000 7feca9200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7feca4147410 0x7feca40afa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:43.716 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.716+0000 7feca9200640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feca40aec90 con 0x7feca406e720 2024-04-20T11:37:43.716 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.716+0000 7feca9200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feca40aef70 con 0x7feca4071cd0 2024-04-20T11:37:43.716 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.716+0000 7feca9200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7feca40af250 con 0x7feca4147410 2024-04-20T11:37:43.716 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.716+0000 7feca3e00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7feca406e720 0x7feca40b3390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:41858/0 (socket says 172.21.15.152:41858) 2024-04-20T11:37:43.716 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.716+0000 7feca3e00640 1 -- 172.21.15.152:0/656120718 learned_addr learned my addr 172.21.15.152:0/656120718 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:37:43.718 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.717+0000 7feca3400640 1 --2- 172.21.15.152:0/656120718 >> v2:172.21.15.175:3300/0 conn(0x7feca4147410 0x7feca40afa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:43.718 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.717+0000 7feca2000640 1 -- 172.21.15.152:0/656120718 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 869334304 0 0) 0x7feca40aec90 con 0x7feca406e720 2024-04-20T11:37:43.718 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.717+0000 7feca2000640 1 -- 172.21.15.152:0/656120718 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fec88003520 con 0x7feca406e720 2024-04-20T11:37:43.718 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.717+0000 7feca2000640 1 -- 172.21.15.152:0/656120718 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1207203301 0 0) 0x7fec88003520 con 0x7feca406e720 2024-04-20T11:37:43.718 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.717+0000 7feca2000640 1 -- 172.21.15.152:0/656120718 >> v2:172.21.15.175:3300/0 conn(0x7feca4147410 msgr2=0x7feca40afa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:43.718 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.717+0000 7feca2000640 1 --2- 172.21.15.152:0/656120718 >> v2:172.21.15.175:3300/0 conn(0x7feca4147410 0x7feca40afa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:43.718 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.717+0000 7feca2000640 1 -- 172.21.15.152:0/656120718 >> v1:172.21.15.175:6789/0 conn(0x7feca4071cd0 legacy=0x7feca406e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:43.718 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.717+0000 7feca2000640 1 -- 172.21.15.152:0/656120718 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feca40af530 con 0x7feca406e720 2024-04-20T11:37:43.718 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.717+0000 7feca2000640 1 -- 172.21.15.152:0/656120718 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7fec9c0019f0 con 0x7feca406e720 2024-04-20T11:37:43.718 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.717+0000 7feca2000640 1 -- 172.21.15.152:0/656120718 >> v1:172.21.15.152:6789/0 conn(0x7feca406e720 legacy=0x7feca40b3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:43.718 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.717+0000 7feca2000640 1 --2- 172.21.15.152:0/656120718 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fec880039b0 0x7fec88003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:43.718 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.717+0000 7feca2000640 1 --2- 172.21.15.152:0/656120718 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fec88004800 0x7fec88004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:43.718 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.717+0000 7feca2000640 1 --2- 172.21.15.152:0/656120718 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fec880051c0 0x7fec8800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:43.718 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.717+0000 7feca2000640 1 -- 172.21.15.152:0/656120718 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7feca40af250 con 0x7fec880039b0 2024-04-20T11:37:43.719 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.717+0000 7feca2000640 1 -- 172.21.15.152:0/656120718 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fec8800c060 con 0x7fec88004800 2024-04-20T11:37:43.719 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.717+0000 7feca2000640 1 -- 172.21.15.152:0/656120718 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fec8800c260 con 0x7fec880051c0 2024-04-20T11:37:43.719 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.718+0000 7feca3400640 1 --2- 172.21.15.152:0/656120718 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fec880039b0 0x7fec88003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:43.719 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.718+0000 7feca3400640 1 -- 172.21.15.152:0/656120718 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fec880051c0 msgr2=0x7fec8800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:43.719 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.718+0000 7feca3400640 1 --2- 172.21.15.152:0/656120718 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fec880051c0 0x7fec8800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:43.719 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.718+0000 7feca3400640 1 -- 172.21.15.152:0/656120718 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fec88004800 msgr2=0x7fec88004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:43.719 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.718+0000 7feca3400640 1 --2- 172.21.15.152:0/656120718 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fec88004800 0x7fec88004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:43.719 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.718+0000 7feca3400640 1 -- 172.21.15.152:0/656120718 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec88003520 con 0x7fec880039b0 2024-04-20T11:37:43.719 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.718+0000 7feca3e00640 1 -- 172.21.15.152:0/656120718 reap_dead start 2024-04-20T11:37:43.720 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.719+0000 7feca3400640 1 --2- 172.21.15.152:0/656120718 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fec880039b0 0x7fec88003dd0 secure :-1 s=READY pgs=2286 cs=0 l=1 rev1=1 crypto rx=0x7fec94009ac0 tx=0x7fec88005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ea1530cc824069b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:43.720 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.719+0000 7feca2000640 1 -- 172.21.15.152:0/656120718 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec94039070 con 0x7fec880039b0 2024-04-20T11:37:43.720 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.719+0000 7feca2000640 1 -- 172.21.15.152:0/656120718 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec9402c6b0 con 0x7fec880039b0 2024-04-20T11:37:43.720 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.719+0000 7feca2000640 1 -- 172.21.15.152:0/656120718 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec94009690 con 0x7fec880039b0 2024-04-20T11:37:43.720 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.720+0000 7feca9200640 1 -- 172.21.15.152:0/656120718 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fec880039b0 msgr2=0x7fec88003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:43.721 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.720+0000 7feca9200640 1 --2- 172.21.15.152:0/656120718 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fec880039b0 0x7fec88003dd0 secure :-1 s=READY pgs=2286 cs=0 l=1 rev1=1 crypto rx=0x7fec94009ac0 tx=0x7fec88005830 comp rx=0 tx=0).stop 2024-04-20T11:37:43.721 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.720+0000 7feca9200640 1 -- 172.21.15.152:0/656120718 shutdown_connections 2024-04-20T11:37:43.721 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.720+0000 7feca9200640 1 --2- 172.21.15.152:0/656120718 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fec880039b0 0x7fec88003dd0 unknown :-1 s=CLOSED pgs=2286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:43.721 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.720+0000 7feca9200640 1 -- 172.21.15.152:0/656120718 >> 172.21.15.152:0/656120718 conn(0x7feca4067ae0 msgr2=0x7feca406b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:43.721 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.720+0000 7feca9200640 1 -- 172.21.15.152:0/656120718 shutdown_connections 2024-04-20T11:37:43.721 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.720+0000 7feca9200640 1 -- 172.21.15.152:0/656120718 wait complete. 2024-04-20T11:37:43.721 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.722+0000 7feca9200640 1 Processor -- start 2024-04-20T11:37:43.721 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.722+0000 7feca9200640 1 -- start start 2024-04-20T11:37:43.723 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.723+0000 7feca9200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fec880039b0 0x7feca4161140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:43.723 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.723+0000 7feca9200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feca413ea40 0x7feca413eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:43.723 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.723+0000 7feca9200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feca413c0a0 0x7feca413c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:43.723 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.723+0000 7feca9200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7feca40afbd0 con 0x7feca413ea40 2024-04-20T11:37:43.723 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.723+0000 7feca9200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7feca40afa50 con 0x7feca413c0a0 2024-04-20T11:37:43.723 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.723+0000 7feca9200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7feca40afd50 con 0x7fec880039b0 2024-04-20T11:37:43.723 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.723+0000 7feca2a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feca413ea40 0x7feca413eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:43.723 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.723+0000 7feca2a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feca413ea40 0x7feca413eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:59924/0 (socket says 172.21.15.152:59924) 2024-04-20T11:37:43.723 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.723+0000 7feca2a00640 1 -- 172.21.15.152:0/1648454007 learned_addr learned my addr 172.21.15.152:0/1648454007 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:43.724 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.724+0000 7feca3e00640 1 --2- 172.21.15.152:0/1648454007 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feca413c0a0 0x7feca413c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:43.724 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.724+0000 7feca3400640 1 --2- 172.21.15.152:0/1648454007 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fec880039b0 0x7feca4161140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:43.725 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.724+0000 7feca2a00640 1 -- 172.21.15.152:0/1648454007 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fec880039b0 msgr2=0x7feca4161140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:43.725 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.724+0000 7feca2a00640 1 --2- 172.21.15.152:0/1648454007 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fec880039b0 0x7feca4161140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:43.725 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.724+0000 7feca2a00640 1 -- 172.21.15.152:0/1648454007 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feca413c0a0 msgr2=0x7feca413c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:43.725 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.724+0000 7feca2a00640 1 --2- 172.21.15.152:0/1648454007 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feca413c0a0 0x7feca413c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:43.725 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.724+0000 7feca2a00640 1 -- 172.21.15.152:0/1648454007 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feca4161680 con 0x7feca413ea40 2024-04-20T11:37:43.725 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.724+0000 7feca3e00640 1 --2- 172.21.15.152:0/1648454007 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feca413c0a0 0x7feca413c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:37:43.725 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.724+0000 7feca2a00640 1 --2- 172.21.15.152:0/1648454007 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feca413ea40 0x7feca413eec0 secure :-1 s=READY pgs=2287 cs=0 l=1 rev1=1 crypto rx=0x7fec9000c500 tx=0x7fec9000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:43.725 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.724+0000 7fec9b400640 1 -- 172.21.15.152:0/1648454007 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec900052d0 con 0x7feca413ea40 2024-04-20T11:37:43.725 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.724+0000 7fec9b400640 1 -- 172.21.15.152:0/1648454007 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec90005470 con 0x7feca413ea40 2024-04-20T11:37:43.725 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.724+0000 7fec9b400640 1 -- 172.21.15.152:0/1648454007 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec90012850 con 0x7feca413ea40 2024-04-20T11:37:43.725 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.725+0000 7feca9200640 1 -- 172.21.15.152:0/1648454007 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feca41600a0 con 0x7feca413ea40 2024-04-20T11:37:43.725 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.725+0000 7feca9200640 1 -- 172.21.15.152:0/1648454007 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feca4160660 con 0x7feca413ea40 2024-04-20T11:37:43.726 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.726+0000 7feca9200640 1 -- 172.21.15.152:0/1648454007 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feca4075e60 con 0x7feca413ea40 2024-04-20T11:37:43.726 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.727+0000 7fec9b400640 1 -- 172.21.15.152:0/1648454007 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 26) ==== 91907+0+0 (secure 0 0 0) 0x7fec900129f0 con 0x7feca413ea40 2024-04-20T11:37:43.727 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.728+0000 7fec9b400640 1 --2- 172.21.15.152:0/1648454007 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fec8406d2b0 0x7fec8406f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:43.727 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.728+0000 7fec9b400640 1 -- 172.21.15.152:0/1648454007 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(163..163 src has 1..163) ==== 4857+0+0 (secure 0 0 0) 0x7fec9008a020 con 0x7feca413ea40 2024-04-20T11:37:43.728 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.729+0000 7feca3400640 1 --2- 172.21.15.152:0/1648454007 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fec8406d2b0 0x7fec8406f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:43.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.730+0000 7feca3400640 1 --2- 172.21.15.152:0/1648454007 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fec8406d2b0 0x7fec8406f770 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fec94036000 tx=0x7fec94030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:43.731 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.732+0000 7fec9b400640 1 -- 172.21.15.152:0/1648454007 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fec9005a050 con 0x7feca413ea40 2024-04-20T11:37:43.938 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.938+0000 7feca9200640 1 -- 172.21.15.152:0/1648454007 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7feca4072540 con 0x7feca413ea40 2024-04-20T11:37:43.939 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.940+0000 7fec9b400640 1 -- 172.21.15.152:0/1648454007 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v163) ==== 74+0+12961 (secure 0 0 0) 0x7fec90059950 con 0x7feca413ea40 2024-04-20T11:37:43.939 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:37:43.940 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":163,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:37:43.011404+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:37:43.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.943+0000 7feca9200640 1 -- 172.21.15.152:0/1648454007 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fec8406d2b0 msgr2=0x7fec8406f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:43.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.943+0000 7feca9200640 1 --2- 172.21.15.152:0/1648454007 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fec8406d2b0 0x7fec8406f770 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fec94036000 tx=0x7fec94030000 comp rx=0 tx=0).stop 2024-04-20T11:37:43.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.943+0000 7feca9200640 1 -- 172.21.15.152:0/1648454007 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feca413ea40 msgr2=0x7feca413eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:43.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.943+0000 7feca9200640 1 --2- 172.21.15.152:0/1648454007 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feca413ea40 0x7feca413eec0 secure :-1 s=READY pgs=2287 cs=0 l=1 rev1=1 crypto rx=0x7fec9000c500 tx=0x7fec9000c9c0 comp rx=0 tx=0).stop 2024-04-20T11:37:43.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.943+0000 7feca9200640 1 -- 172.21.15.152:0/1648454007 shutdown_connections 2024-04-20T11:37:43.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.943+0000 7feca9200640 1 --2- 172.21.15.152:0/1648454007 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fec8406d2b0 0x7fec8406f770 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:43.944 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.943+0000 7feca9200640 1 --2- 172.21.15.152:0/1648454007 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feca413c0a0 0x7feca413c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:43.944 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.943+0000 7feca9200640 1 --2- 172.21.15.152:0/1648454007 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feca413ea40 0x7feca413eec0 unknown :-1 s=CLOSED pgs=2287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:43.944 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.943+0000 7feca9200640 1 --2- 172.21.15.152:0/1648454007 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fec880039b0 0x7feca4161140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:43.944 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.943+0000 7feca9200640 1 -- 172.21.15.152:0/1648454007 >> 172.21.15.152:0/1648454007 conn(0x7feca4067ae0 msgr2=0x7feca40b34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:43.944 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.943+0000 7feca9200640 1 -- 172.21.15.152:0/1648454007 shutdown_connections 2024-04-20T11:37:43.944 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:43.943+0000 7feca9200640 1 -- 172.21.15.152:0/1648454007 wait complete. 2024-04-20T11:37:43.963 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T11:26:47.246571+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T11:37:43.964 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T11:37:44.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.158+0000 7fdbf4600640 1 Processor -- start 2024-04-20T11:37:44.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.158+0000 7fdbf4600640 1 -- start start 2024-04-20T11:37:44.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.159+0000 7fdbf4600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdbec0b2880 0x7fdbec0b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:44.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.159+0000 7fdbf4600640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fdbec0cd1b0 0x7fdbec0cd600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:44.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.159+0000 7fdbf4600640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdbec1463e0 con 0x7fdbec146000 2024-04-20T11:37:44.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.159+0000 7fdbf4600640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fdbec0b4750 con 0x7fdbec0cd1b0 2024-04-20T11:37:44.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.160+0000 7fdbf4600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fdbec0b3e20 con 0x7fdbec0b2880 2024-04-20T11:37:44.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.160+0000 7fdbf2600640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fdbec0cd1b0 0x7fdbec0cd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:44.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.160+0000 7fdbf2600640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fdbec0cd1b0 0x7fdbec0cd600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:59932/0 (socket says 172.21.15.152:59932) 2024-04-20T11:37:44.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.160+0000 7fdbf2600640 1 -- 172.21.15.152:0/2028819489 learned_addr learned my addr 172.21.15.152:0/2028819489 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:44.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.160+0000 7fdbf1200640 1 --2- 172.21.15.152:0/2028819489 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdbec0b2880 0x7fdbec0b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:44.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.160+0000 7fdbf2600640 1 -- 172.21.15.152:0/2028819489 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdbec0b2880 msgr2=0x7fdbec0b4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:44.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.160+0000 7fdbf2600640 1 --2- 172.21.15.152:0/2028819489 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdbec0b2880 0x7fdbec0b4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:44.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.160+0000 7fdbf2600640 1 -- 172.21.15.152:0/2028819489 >> v1:172.21.15.175:6789/0 conn(0x7fdbec146000 legacy=0x7fdbec0b2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:44.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.160+0000 7fdbf2600640 1 -- 172.21.15.152:0/2028819489 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbec0b4100 con 0x7fdbec0cd1b0 2024-04-20T11:37:44.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.162+0000 7fdbf2600640 1 --2- 172.21.15.152:0/2028819489 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdbec0cd1b0 0x7fdbec0cd600 secure :-1 s=READY pgs=2288 cs=0 l=1 rev1=1 crypto rx=0x7fdbe4009e50 tx=0x7fdbe402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e0352d080585adc server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:44.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.162+0000 7fdbebe00640 1 -- 172.21.15.152:0/2028819489 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe402c9e0 con 0x7fdbec0cd1b0 2024-04-20T11:37:44.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.162+0000 7fdbebe00640 1 -- 172.21.15.152:0/2028819489 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbe402cb80 con 0x7fdbec0cd1b0 2024-04-20T11:37:44.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.162+0000 7fdbebe00640 1 -- 172.21.15.152:0/2028819489 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe40343f0 con 0x7fdbec0cd1b0 2024-04-20T11:37:44.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.163+0000 7fdbf4600640 1 -- 172.21.15.152:0/2028819489 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdbec0cd1b0 msgr2=0x7fdbec0cd600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:44.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.163+0000 7fdbf4600640 1 --2- 172.21.15.152:0/2028819489 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdbec0cd1b0 0x7fdbec0cd600 secure :-1 s=READY pgs=2288 cs=0 l=1 rev1=1 crypto rx=0x7fdbe4009e50 tx=0x7fdbe402be40 comp rx=0 tx=0).stop 2024-04-20T11:37:44.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.163+0000 7fdbf4600640 1 -- 172.21.15.152:0/2028819489 shutdown_connections 2024-04-20T11:37:44.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.163+0000 7fdbf4600640 1 --2- 172.21.15.152:0/2028819489 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdbec0cd1b0 0x7fdbec0cd600 unknown :-1 s=CLOSED pgs=2288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:44.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.163+0000 7fdbf4600640 1 --2- 172.21.15.152:0/2028819489 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdbec0b2880 0x7fdbec0b4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:44.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.163+0000 7fdbf4600640 1 -- 172.21.15.152:0/2028819489 >> 172.21.15.152:0/2028819489 conn(0x7fdbec06b980 msgr2=0x7fdbec06bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:44.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.163+0000 7fdbf4600640 1 -- 172.21.15.152:0/2028819489 shutdown_connections 2024-04-20T11:37:44.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.163+0000 7fdbf4600640 1 -- 172.21.15.152:0/2028819489 wait complete. 2024-04-20T11:37:44.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.165+0000 7fdbf4600640 1 Processor -- start 2024-04-20T11:37:44.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.165+0000 7fdbf4600640 1 -- start start 2024-04-20T11:37:44.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.166+0000 7fdbf4600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdbec0b2880 0x7fdbec144900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:44.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.166+0000 7fdbf4600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdbec0cd1b0 0x7fdbec144250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:44.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.166+0000 7fdbf4600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdbec146000 0x7fdbec1e4f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:44.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.166+0000 7fdbf4600640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fdbec0cdcc0 con 0x7fdbec0cd1b0 2024-04-20T11:37:44.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.166+0000 7fdbf4600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdbec0cdb40 con 0x7fdbec146000 2024-04-20T11:37:44.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.166+0000 7fdbf4600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fdbec15ccf0 con 0x7fdbec0b2880 2024-04-20T11:37:44.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.167+0000 7fdbf1200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdbec0cd1b0 0x7fdbec144250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:44.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.167+0000 7fdbf1200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdbec0cd1b0 0x7fdbec144250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:59936/0 (socket says 172.21.15.152:59936) 2024-04-20T11:37:44.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.167+0000 7fdbf1200640 1 -- 172.21.15.152:0/2470029311 learned_addr learned my addr 172.21.15.152:0/2470029311 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:44.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.167+0000 7fdbf1c00640 1 --2- 172.21.15.152:0/2470029311 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdbec0b2880 0x7fdbec144900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:44.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.167+0000 7fdbf2600640 1 --2- 172.21.15.152:0/2470029311 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdbec146000 0x7fdbec1e4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:44.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.167+0000 7fdbf1200640 1 -- 172.21.15.152:0/2470029311 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdbec0b2880 msgr2=0x7fdbec144900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:44.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.167+0000 7fdbf1200640 1 --2- 172.21.15.152:0/2470029311 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdbec0b2880 0x7fdbec144900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:44.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.167+0000 7fdbf1200640 1 -- 172.21.15.152:0/2470029311 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdbec146000 msgr2=0x7fdbec1e4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:44.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.167+0000 7fdbf1200640 1 --2- 172.21.15.152:0/2470029311 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdbec146000 0x7fdbec1e4f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:44.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.167+0000 7fdbf1200640 1 -- 172.21.15.152:0/2470029311 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbec1f5c10 con 0x7fdbec0cd1b0 2024-04-20T11:37:44.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.167+0000 7fdbf1200640 1 --2- 172.21.15.152:0/2470029311 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdbec0cd1b0 0x7fdbec144250 secure :-1 s=READY pgs=2289 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc00d800 tx=0x7fdbdc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:44.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.168+0000 7fdbe9600640 1 -- 172.21.15.152:0/2470029311 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbdc00b6c0 con 0x7fdbec0cd1b0 2024-04-20T11:37:44.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.168+0000 7fdbe9600640 1 -- 172.21.15.152:0/2470029311 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbdc019070 con 0x7fdbec0cd1b0 2024-04-20T11:37:44.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.168+0000 7fdbe9600640 1 -- 172.21.15.152:0/2470029311 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbdc00ba80 con 0x7fdbec0cd1b0 2024-04-20T11:37:44.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.168+0000 7fdbf4600640 1 -- 172.21.15.152:0/2470029311 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbec15e500 con 0x7fdbec0cd1b0 2024-04-20T11:37:44.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.168+0000 7fdbf4600640 1 -- 172.21.15.152:0/2470029311 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbec1f6190 con 0x7fdbec0cd1b0 2024-04-20T11:37:44.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.169+0000 7fdbf4600640 1 -- 172.21.15.152:0/2470029311 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbec0689f0 con 0x7fdbec0cd1b0 2024-04-20T11:37:44.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.170+0000 7fdbe9600640 1 -- 172.21.15.152:0/2470029311 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 26) ==== 91907+0+0 (secure 0 0 0) 0x7fdbdc00bc20 con 0x7fdbec0cd1b0 2024-04-20T11:37:44.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.172+0000 7fdbe9600640 1 --2- 172.21.15.152:0/2470029311 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fdbd006d280 0x7fdbd006f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:44.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.172+0000 7fdbe9600640 1 -- 172.21.15.152:0/2470029311 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(163..163 src has 1..163) ==== 4857+0+0 (secure 0 0 0) 0x7fdbdc08a7c0 con 0x7fdbec0cd1b0 2024-04-20T11:37:44.172 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.172+0000 7fdbf1c00640 1 --2- 172.21.15.152:0/2470029311 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fdbd006d280 0x7fdbd006f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:44.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.173+0000 7fdbf1c00640 1 --2- 172.21.15.152:0/2470029311 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fdbd006d280 0x7fdbd006f740 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fdbe0006fd0 tx=0x7fdbe0008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:44.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.176+0000 7fdbe9600640 1 -- 172.21.15.152:0/2470029311 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fdbdc05a7f0 con 0x7fdbec0cd1b0 2024-04-20T11:37:44.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.376+0000 7fdbf4600640 1 -- 172.21.15.152:0/2470029311 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fdbec06cd90 con 0x7fdbec0cd1b0 2024-04-20T11:37:44.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.376+0000 7fdbe9600640 1 -- 172.21.15.152:0/2470029311 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v163) ==== 93+0+10 (secure 0 0 0) 0x7fdbdc05a0f0 con 0x7fdbec0cd1b0 2024-04-20T11:37:44.376 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 1 2024-04-20T11:37:44.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.379+0000 7fdbf4600640 1 -- 172.21.15.152:0/2470029311 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fdbd006d280 msgr2=0x7fdbd006f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:44.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.379+0000 7fdbf4600640 1 --2- 172.21.15.152:0/2470029311 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fdbd006d280 0x7fdbd006f740 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fdbe0006fd0 tx=0x7fdbe0008040 comp rx=0 tx=0).stop 2024-04-20T11:37:44.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.379+0000 7fdbf4600640 1 -- 172.21.15.152:0/2470029311 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdbec0cd1b0 msgr2=0x7fdbec144250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:44.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.379+0000 7fdbf4600640 1 --2- 172.21.15.152:0/2470029311 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdbec0cd1b0 0x7fdbec144250 secure :-1 s=READY pgs=2289 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc00d800 tx=0x7fdbdc00dcc0 comp rx=0 tx=0).stop 2024-04-20T11:37:44.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.379+0000 7fdbf4600640 1 -- 172.21.15.152:0/2470029311 shutdown_connections 2024-04-20T11:37:44.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.379+0000 7fdbf4600640 1 --2- 172.21.15.152:0/2470029311 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fdbd006d280 0x7fdbd006f740 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:44.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.379+0000 7fdbf4600640 1 --2- 172.21.15.152:0/2470029311 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdbec146000 0x7fdbec1e4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:44.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.379+0000 7fdbf4600640 1 --2- 172.21.15.152:0/2470029311 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdbec0cd1b0 0x7fdbec144250 unknown :-1 s=CLOSED pgs=2289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:44.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.379+0000 7fdbf4600640 1 --2- 172.21.15.152:0/2470029311 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdbec0b2880 0x7fdbec144900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:44.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.379+0000 7fdbf4600640 1 -- 172.21.15.152:0/2470029311 >> 172.21.15.152:0/2470029311 conn(0x7fdbec06b980 msgr2=0x7fdbec0b1f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:44.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.379+0000 7fdbf4600640 1 -- 172.21.15.152:0/2470029311 shutdown_connections 2024-04-20T11:37:44.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.379+0000 7fdbf4600640 1 -- 172.21.15.152:0/2470029311 wait complete. 2024-04-20T11:37:44.399 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-20T11:37:44.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.597+0000 7f2fea800640 1 Processor -- start 2024-04-20T11:37:44.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.597+0000 7f2fea800640 1 -- start start 2024-04-20T11:37:44.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.599+0000 7f2fea800640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f2fe415e6c0 0x7f2fe415f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:44.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.599+0000 7f2fea800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f2fe40d4ce0 0x7f2fe40d2b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:44.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.599+0000 7f2fea800640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2fe41601e0 con 0x7f2fe40d3060 2024-04-20T11:37:44.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.599+0000 7f2fea800640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f2fe406caf0 con 0x7f2fe40d4ce0 2024-04-20T11:37:44.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.599+0000 7f2fea800640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f2fe4160480 con 0x7f2fe415e6c0 2024-04-20T11:37:44.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.599+0000 7f2fe8a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f2fe40d4ce0 0x7f2fe40d2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:44.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.599+0000 7f2fe8a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f2fe40d4ce0 0x7f2fe40d2b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:59944/0 (socket says 172.21.15.152:59944) 2024-04-20T11:37:44.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.599+0000 7f2fe8a00640 1 -- 172.21.15.152:0/3116443303 learned_addr learned my addr 172.21.15.152:0/3116443303 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:44.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.599+0000 7f2fe9e00640 1 --1- 172.21.15.152:0/3116443303 >> v1:172.21.15.152:6789/0 conn(0x7f2fe40d3060 0x7f2fe4162340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:41872/0 (socket says 172.21.15.152:41872) 2024-04-20T11:37:44.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.599+0000 7f2fe9400640 1 --2- 172.21.15.152:0/3116443303 >> v2:172.21.15.175:3300/0 conn(0x7f2fe415e6c0 0x7f2fe415f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:44.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.600+0000 7f2fe8a00640 1 -- 172.21.15.152:0/3116443303 >> v2:172.21.15.175:3300/0 conn(0x7f2fe415e6c0 msgr2=0x7f2fe415f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:44.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.600+0000 7f2fe8a00640 1 --2- 172.21.15.152:0/3116443303 >> v2:172.21.15.175:3300/0 conn(0x7f2fe415e6c0 0x7f2fe415f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:44.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.600+0000 7f2fe8a00640 1 -- 172.21.15.152:0/3116443303 >> v1:172.21.15.152:6789/0 conn(0x7f2fe40d3060 legacy=0x7f2fe4162340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:44.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.600+0000 7f2fe8a00640 1 -- 172.21.15.152:0/3116443303 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fe4160760 con 0x7f2fe40d4ce0 2024-04-20T11:37:44.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.600+0000 7f2fe9400640 1 --2- 172.21.15.152:0/3116443303 >> v2:172.21.15.175:3300/0 conn(0x7f2fe415e6c0 0x7f2fe415f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:37:44.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.601+0000 7f2fe8a00640 1 --2- 172.21.15.152:0/3116443303 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2fe40d4ce0 0x7f2fe40d2b20 secure :-1 s=READY pgs=2291 cs=0 l=1 rev1=1 crypto rx=0x7f2fe408cdd0 tx=0x7f2fd002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0cb4ac42209b1c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:44.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.601+0000 7f2fe3e00640 1 -- 172.21.15.152:0/3116443303 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fd002d040 con 0x7f2fe40d4ce0 2024-04-20T11:37:44.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.601+0000 7f2fe3e00640 1 -- 172.21.15.152:0/3116443303 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fd0004230 con 0x7f2fe40d4ce0 2024-04-20T11:37:44.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.601+0000 7f2fe3e00640 1 -- 172.21.15.152:0/3116443303 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fd0003610 con 0x7f2fe40d4ce0 2024-04-20T11:37:44.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.602+0000 7f2fea800640 1 -- 172.21.15.152:0/3116443303 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2fe40d4ce0 msgr2=0x7f2fe40d2b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:44.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.602+0000 7f2fea800640 1 --2- 172.21.15.152:0/3116443303 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2fe40d4ce0 0x7f2fe40d2b20 secure :-1 s=READY pgs=2291 cs=0 l=1 rev1=1 crypto rx=0x7f2fe408cdd0 tx=0x7f2fd002b8f0 comp rx=0 tx=0).stop 2024-04-20T11:37:44.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.602+0000 7f2fea800640 1 -- 172.21.15.152:0/3116443303 shutdown_connections 2024-04-20T11:37:44.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.602+0000 7f2fea800640 1 --2- 172.21.15.152:0/3116443303 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2fe40d4ce0 0x7f2fe40d2b20 unknown :-1 s=CLOSED pgs=2291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:44.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.602+0000 7f2fea800640 1 --2- 172.21.15.152:0/3116443303 >> v2:172.21.15.175:3300/0 conn(0x7f2fe415e6c0 0x7f2fe415f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:44.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.602+0000 7f2fea800640 1 -- 172.21.15.152:0/3116443303 >> 172.21.15.152:0/3116443303 conn(0x7f2fe408bfd0 msgr2=0x7f2fe40d45d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:44.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.602+0000 7f2fea800640 1 -- 172.21.15.152:0/3116443303 shutdown_connections 2024-04-20T11:37:44.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.602+0000 7f2fea800640 1 -- 172.21.15.152:0/3116443303 wait complete. 2024-04-20T11:37:44.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.603+0000 7f2fea800640 1 Processor -- start 2024-04-20T11:37:44.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.604+0000 7f2fea800640 1 -- start start 2024-04-20T11:37:44.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.605+0000 7f2fea800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2fe40d3060 0x7f2fe40cb980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:44.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.605+0000 7f2fea800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2fe40d4ce0 0x7f2fe4152120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:44.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.605+0000 7f2fea800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2fe415e6c0 0x7f2fe415c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:44.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.605+0000 7f2fea800640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f2fe40d5120 con 0x7f2fe40d3060 2024-04-20T11:37:44.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.605+0000 7f2fea800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2fe40d5450 con 0x7f2fe415e6c0 2024-04-20T11:37:44.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.605+0000 7f2fea800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2fe4161f70 con 0x7f2fe40d4ce0 2024-04-20T11:37:44.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.605+0000 7f2fe9400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2fe40d3060 0x7f2fe40cb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:44.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.605+0000 7f2fe9400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2fe40d3060 0x7f2fe40cb980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:59946/0 (socket says 172.21.15.152:59946) 2024-04-20T11:37:44.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.605+0000 7f2fe9400640 1 -- 172.21.15.152:0/2192636616 learned_addr learned my addr 172.21.15.152:0/2192636616 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:44.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.605+0000 7f2fe8a00640 1 --2- 172.21.15.152:0/2192636616 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2fe40d4ce0 0x7f2fe4152120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:44.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.605+0000 7f2fe9400640 1 -- 172.21.15.152:0/2192636616 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2fe40d4ce0 msgr2=0x7f2fe4152120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:44.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.605+0000 7f2fe9e00640 1 --2- 172.21.15.152:0/2192636616 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2fe415e6c0 0x7f2fe415c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:44.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.605+0000 7f2fe9400640 1 --2- 172.21.15.152:0/2192636616 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2fe40d4ce0 0x7f2fe4152120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:44.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.605+0000 7f2fe9400640 1 -- 172.21.15.152:0/2192636616 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2fe415e6c0 msgr2=0x7f2fe415c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:44.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.605+0000 7f2fe9400640 1 --2- 172.21.15.152:0/2192636616 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2fe415e6c0 0x7f2fe415c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:44.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.605+0000 7f2fe9400640 1 -- 172.21.15.152:0/2192636616 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fe40cbec0 con 0x7f2fe40d3060 2024-04-20T11:37:44.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.606+0000 7f2fe9400640 1 --2- 172.21.15.152:0/2192636616 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2fe40d3060 0x7f2fe40cb980 secure :-1 s=READY pgs=2292 cs=0 l=1 rev1=1 crypto rx=0x7f2fd800c820 tx=0x7f2fd800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:44.607 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.606+0000 7f2fe1600640 1 -- 172.21.15.152:0/2192636616 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fd8015070 con 0x7f2fe40d3060 2024-04-20T11:37:44.607 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.606+0000 7f2fe1600640 1 -- 172.21.15.152:0/2192636616 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fd8007a30 con 0x7f2fe40d3060 2024-04-20T11:37:44.607 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.606+0000 7f2fe1600640 1 -- 172.21.15.152:0/2192636616 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fd8007de0 con 0x7f2fe40d3060 2024-04-20T11:37:44.607 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.606+0000 7f2fea800640 1 -- 172.21.15.152:0/2192636616 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fe40c8240 con 0x7f2fe40d3060 2024-04-20T11:37:44.607 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.606+0000 7f2fea800640 1 -- 172.21.15.152:0/2192636616 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fe40c8800 con 0x7f2fe40d3060 2024-04-20T11:37:44.607 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.607+0000 7f2fea800640 1 -- 172.21.15.152:0/2192636616 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fe4075e60 con 0x7f2fe40d3060 2024-04-20T11:37:44.608 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.608+0000 7f2fe1600640 1 -- 172.21.15.152:0/2192636616 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 26) ==== 91907+0+0 (secure 0 0 0) 0x7f2fd8018070 con 0x7f2fe40d3060 2024-04-20T11:37:44.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.609+0000 7f2fe1600640 1 --2- 172.21.15.152:0/2192636616 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f2fc406d2b0 0x7f2fc406f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:44.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.609+0000 7f2fe1600640 1 -- 172.21.15.152:0/2192636616 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(163..163 src has 1..163) ==== 4857+0+0 (secure 0 0 0) 0x7f2fd808a1c0 con 0x7f2fe40d3060 2024-04-20T11:37:44.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.610+0000 7f2fe8a00640 1 --2- 172.21.15.152:0/2192636616 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f2fc406d2b0 0x7f2fc406f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:44.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.611+0000 7f2fe8a00640 1 --2- 172.21.15.152:0/2192636616 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f2fc406d2b0 0x7f2fc406f770 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f2fd0035000 tx=0x7f2fd0008030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:44.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.614+0000 7f2fe1600640 1 -- 172.21.15.152:0/2192636616 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f2fd805a160 con 0x7f2fe40d3060 2024-04-20T11:37:44.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.807+0000 7f2fea800640 1 -- 172.21.15.152:0/2192636616 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f2fe40d1f00 con 0x7f2fe40d3060 2024-04-20T11:37:44.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.808+0000 7f2fe1600640 1 -- 172.21.15.152:0/2192636616 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v163) ==== 84+0+65 (secure 0 0 0) 0x7f2fd8059a60 con 0x7f2fe40d3060 2024-04-20T11:37:44.808 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:37:44.808 INFO:teuthology.orchestra.run.smithi152.stdout:[ 2024-04-20T11:37:44.808 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:37:44.808 INFO:teuthology.orchestra.run.smithi152.stdout: "poolnum": 1, 2024-04-20T11:37:44.808 INFO:teuthology.orchestra.run.smithi152.stdout: "poolname": ".mgr" 2024-04-20T11:37:44.808 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:37:44.808 INFO:teuthology.orchestra.run.smithi152.stdout:] 2024-04-20T11:37:44.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.810+0000 7f2fea800640 1 -- 172.21.15.152:0/2192636616 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f2fc406d2b0 msgr2=0x7f2fc406f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:44.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.810+0000 7f2fea800640 1 --2- 172.21.15.152:0/2192636616 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f2fc406d2b0 0x7f2fc406f770 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f2fd0035000 tx=0x7f2fd0008030 comp rx=0 tx=0).stop 2024-04-20T11:37:44.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.810+0000 7f2fea800640 1 -- 172.21.15.152:0/2192636616 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2fe40d3060 msgr2=0x7f2fe40cb980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:44.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.810+0000 7f2fea800640 1 --2- 172.21.15.152:0/2192636616 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2fe40d3060 0x7f2fe40cb980 secure :-1 s=READY pgs=2292 cs=0 l=1 rev1=1 crypto rx=0x7f2fd800c820 tx=0x7f2fd800cce0 comp rx=0 tx=0).stop 2024-04-20T11:37:44.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.810+0000 7f2fea800640 1 -- 172.21.15.152:0/2192636616 shutdown_connections 2024-04-20T11:37:44.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.810+0000 7f2fea800640 1 --2- 172.21.15.152:0/2192636616 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f2fc406d2b0 0x7f2fc406f770 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:44.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.810+0000 7f2fea800640 1 --2- 172.21.15.152:0/2192636616 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2fe415e6c0 0x7f2fe415c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:44.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.810+0000 7f2fea800640 1 --2- 172.21.15.152:0/2192636616 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2fe40d4ce0 0x7f2fe4152120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:44.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.810+0000 7f2fea800640 1 --2- 172.21.15.152:0/2192636616 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2fe40d3060 0x7f2fe40cb980 unknown :-1 s=CLOSED pgs=2292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:44.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.810+0000 7f2fea800640 1 -- 172.21.15.152:0/2192636616 >> 172.21.15.152:0/2192636616 conn(0x7f2fe408bfd0 msgr2=0x7f2fe415a1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:44.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.811+0000 7f2fea800640 1 -- 172.21.15.152:0/2192636616 shutdown_connections 2024-04-20T11:37:44.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:44.811+0000 7f2fea800640 1 -- 172.21.15.152:0/2192636616 wait complete. 2024-04-20T11:37:44.829 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-20T11:37:44.829 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-20T11:37:45.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:45.028+0000 7feb6e800640 1 Processor -- start 2024-04-20T11:37:45.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:45.028+0000 7feb6e800640 1 -- start start 2024-04-20T11:37:45.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:45.030+0000 7feb6e800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7feb68072530 0x7feb68160db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:45.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:45.030+0000 7feb6e800640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7feb680b47a0 0x7feb680ceaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:45.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:45.030+0000 7feb6e800640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb680b4f90 con 0x7feb68145340 2024-04-20T11:37:45.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:45.030+0000 7feb6e800640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7feb68145720 con 0x7feb68072530 2024-04-20T11:37:45.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:45.030+0000 7feb6e800640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7feb680b4cb0 con 0x7feb680b47a0 2024-04-20T11:37:45.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:45.031+0000 7feb5fe00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7feb68072530 0x7feb68160db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:45.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:45.031+0000 7feb5fe00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7feb68072530 0x7feb68160db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:59958/0 (socket says 172.21.15.152:59958) 2024-04-20T11:37:45.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:45.031+0000 7feb5fe00640 1 -- 172.21.15.152:0/2356176072 learned_addr learned my addr 172.21.15.152:0/2356176072 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:45.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:45.031+0000 7feb67e00640 1 --2- 172.21.15.152:0/2356176072 >> v2:172.21.15.175:3300/0 conn(0x7feb680b47a0 0x7feb680ceaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:45.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:45.031+0000 7feb5fe00640 1 -- 172.21.15.152:0/2356176072 >> v2:172.21.15.175:3300/0 conn(0x7feb680b47a0 msgr2=0x7feb680ceaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:45.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:45.031+0000 7feb5fe00640 1 --2- 172.21.15.152:0/2356176072 >> v2:172.21.15.175:3300/0 conn(0x7feb680b47a0 0x7feb680ceaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:45.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:45.031+0000 7feb5fe00640 1 -- 172.21.15.152:0/2356176072 >> v1:172.21.15.175:6789/0 conn(0x7feb68145340 legacy=0x7feb68071e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:45.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:45.031+0000 7feb5fe00640 1 -- 172.21.15.152:0/2356176072 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb681602d0 con 0x7feb68072530 2024-04-20T11:37:45.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:45.031+0000 7feb67e00640 1 --2- 172.21.15.152:0/2356176072 >> v2:172.21.15.175:3300/0 conn(0x7feb680b47a0 0x7feb680ceaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:37:45.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:45.032+0000 7feb5fe00640 1 --2- 172.21.15.152:0/2356176072 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feb68072530 0x7feb68160db0 secure :-1 s=READY pgs=2293 cs=0 l=1 rev1=1 crypto rx=0x7feb68072df0 tx=0x7feb5002b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ccfcee58af2943a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:45.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:45.033+0000 7feb66a00640 1 -- 172.21.15.152:0/2356176072 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb50038070 con 0x7feb68072530 2024-04-20T11:37:45.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:45.033+0000 7feb66a00640 1 -- 172.21.15.152:0/2356176072 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb5002f050 con 0x7feb68072530 2024-04-20T11:37:45.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:45.033+0000 7feb66a00640 1 -- 172.21.15.152:0/2356176072 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb50031970 con 0x7feb68072530 2024-04-20T11:37:45.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:45.033+0000 7feb6e800640 1 -- 172.21.15.152:0/2356176072 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feb68072530 msgr2=0x7feb68160db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:45.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:45.033+0000 7feb6e800640 1 --2- 172.21.15.152:0/2356176072 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feb68072530 0x7feb68160db0 secure :-1 s=READY pgs=2293 cs=0 l=1 rev1=1 crypto rx=0x7feb68072df0 tx=0x7feb5002b7f0 comp rx=0 tx=0).stop 2024-04-20T11:37:45.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:45.033+0000 7feb6e800640 1 -- 172.21.15.152:0/2356176072 shutdown_connections 2024-04-20T11:37:45.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:45.034+0000 7feb6e800640 1 --2- 172.21.15.152:0/2356176072 >> v2:172.21.15.175:3300/0 conn(0x7feb680b47a0 0x7feb680ceaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:45.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:45.034+0000 7feb6e800640 1 --2- 172.21.15.152:0/2356176072 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feb68072530 0x7feb68160db0 unknown :-1 s=CLOSED pgs=2293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:45.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:45.034+0000 7feb6e800640 1 -- 172.21.15.152:0/2356176072 >> 172.21.15.152:0/2356176072 conn(0x7feb6806b980 msgr2=0x7feb6806bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:45.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:45.034+0000 7feb6e800640 1 -- 172.21.15.152:0/2356176072 shutdown_connections 2024-04-20T11:37:45.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:45.034+0000 7feb6e800640 1 -- 172.21.15.152:0/2356176072 wait complete. 2024-04-20T11:37:45.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:45.036+0000 7feb6e800640 1 Processor -- start 2024-04-20T11:37:45.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:45.036+0000 7feb6e800640 1 -- start start 2024-04-20T11:37:45.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:45.037+0000 7feb6e800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feb68072530 0x7feb6813b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:45.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:45.037+0000 7feb6e800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feb680b47a0 0x7feb6813ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:45.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:45.037+0000 7feb6e800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feb68145340 0x7feb6815e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:45.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:45.037+0000 7feb6e800640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7feb6815cbd0 con 0x7feb680b47a0 2024-04-20T11:37:45.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:45.037+0000 7feb6e800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7feb68146010 con 0x7feb68145340 2024-04-20T11:37:45.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:45.037+0000 7feb6e800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7feb6815cd50 con 0x7feb68072530 2024-04-20T11:37:45.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:45.037+0000 7feb5fe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feb680b47a0 0x7feb6813ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:45.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:45.037+0000 7feb5fe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feb680b47a0 0x7feb6813ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:59960/0 (socket says 172.21.15.152:59960) 2024-04-20T11:37:45.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:45.037+0000 7feb5fe00640 1 -- 172.21.15.152:0/3790248180 learned_addr learned my addr 172.21.15.152:0/3790248180 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:45.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:45.038+0000 7feb67400640 1 --2- 172.21.15.152:0/3790248180 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feb68072530 0x7feb6813b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:45.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:45.038+0000 7feb67e00640 1 --2- 172.21.15.152:0/3790248180 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feb68145340 0x7feb6815e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:45.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:45.038+0000 7feb5fe00640 1 -- 172.21.15.152:0/3790248180 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feb68072530 msgr2=0x7feb6813b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:45.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:45.038+0000 7feb5fe00640 1 --2- 172.21.15.152:0/3790248180 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feb68072530 0x7feb6813b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:45.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:45.038+0000 7feb5fe00640 1 -- 172.21.15.152:0/3790248180 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feb68145340 msgr2=0x7feb6815e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:45.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:45.038+0000 7feb5fe00640 1 --2- 172.21.15.152:0/3790248180 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feb68145340 0x7feb6815e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:45.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:45.038+0000 7feb5fe00640 1 -- 172.21.15.152:0/3790248180 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb6815c090 con 0x7feb680b47a0 2024-04-20T11:37:45.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:45.038+0000 7feb5fe00640 1 --2- 172.21.15.152:0/3790248180 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feb680b47a0 0x7feb6813ba10 secure :-1 s=READY pgs=2294 cs=0 l=1 rev1=1 crypto rx=0x7feb50000c00 tx=0x7feb500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:45.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:45.038+0000 7feb5f400640 1 -- 172.21.15.152:0/3790248180 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb50042070 con 0x7feb680b47a0 2024-04-20T11:37:45.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:45.039+0000 7feb5f400640 1 -- 172.21.15.152:0/3790248180 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb5003d870 con 0x7feb680b47a0 2024-04-20T11:37:45.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:45.039+0000 7feb6e800640 1 -- 172.21.15.152:0/3790248180 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb6813c040 con 0x7feb680b47a0 2024-04-20T11:37:45.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:45.039+0000 7feb5f400640 1 -- 172.21.15.152:0/3790248180 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb50038040 con 0x7feb680b47a0 2024-04-20T11:37:45.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:45.039+0000 7feb6e800640 1 -- 172.21.15.152:0/3790248180 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb6813c600 con 0x7feb680b47a0 2024-04-20T11:37:45.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:45.040+0000 7feb6e800640 1 -- 172.21.15.152:0/3790248180 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb68068a60 con 0x7feb680b47a0 2024-04-20T11:37:45.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:45.041+0000 7feb5f400640 1 -- 172.21.15.152:0/3790248180 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 26) ==== 91907+0+0 (secure 0 0 0) 0x7feb50045070 con 0x7feb680b47a0 2024-04-20T11:37:45.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:45.042+0000 7feb5f400640 1 --2- 172.21.15.152:0/3790248180 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7feb4406d280 0x7feb4406f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:45.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:45.042+0000 7feb5f400640 1 -- 172.21.15.152:0/3790248180 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(163..163 src has 1..163) ==== 4857+0+0 (secure 0 0 0) 0x7feb500ac0f0 con 0x7feb680b47a0 2024-04-20T11:37:45.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:45.042+0000 7feb67400640 1 --2- 172.21.15.152:0/3790248180 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7feb4406d280 0x7feb4406f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:45.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:45.043+0000 7feb67400640 1 --2- 172.21.15.152:0/3790248180 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7feb4406d280 0x7feb4406f740 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7feb58006fd0 tx=0x7feb58008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:45.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:45.046+0000 7feb5f400640 1 -- 172.21.15.152:0/3790248180 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7feb5007c240 con 0x7feb680b47a0 2024-04-20T11:37:45.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:45.232+0000 7feb6e800640 1 -- 172.21.15.152:0/3790248180 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7feb680b4580 con 0x7feb680b47a0 2024-04-20T11:37:46.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:46.036+0000 7feb5f400640 1 -- 172.21.15.152:0/3790248180 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v164) ==== 138+0+0 (secure 0 0 0) 0x7feb5007bb40 con 0x7feb680b47a0 2024-04-20T11:37:46.036 INFO:teuthology.orchestra.run.smithi152.stderr:pool 'cephfs_metadata' created 2024-04-20T11:37:46.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:46.040+0000 7feb6e800640 1 -- 172.21.15.152:0/3790248180 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7feb4406d280 msgr2=0x7feb4406f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:46.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:46.040+0000 7feb6e800640 1 --2- 172.21.15.152:0/3790248180 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7feb4406d280 0x7feb4406f740 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7feb58006fd0 tx=0x7feb58008040 comp rx=0 tx=0).stop 2024-04-20T11:37:46.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:46.040+0000 7feb6e800640 1 -- 172.21.15.152:0/3790248180 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feb680b47a0 msgr2=0x7feb6813ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:46.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:46.040+0000 7feb6e800640 1 --2- 172.21.15.152:0/3790248180 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feb680b47a0 0x7feb6813ba10 secure :-1 s=READY pgs=2294 cs=0 l=1 rev1=1 crypto rx=0x7feb50000c00 tx=0x7feb500046c0 comp rx=0 tx=0).stop 2024-04-20T11:37:46.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:46.040+0000 7feb6e800640 1 -- 172.21.15.152:0/3790248180 shutdown_connections 2024-04-20T11:37:46.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:46.040+0000 7feb6e800640 1 --2- 172.21.15.152:0/3790248180 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7feb4406d280 0x7feb4406f740 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:46.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:46.040+0000 7feb6e800640 1 --2- 172.21.15.152:0/3790248180 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feb68145340 0x7feb6815e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:46.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:46.040+0000 7feb6e800640 1 --2- 172.21.15.152:0/3790248180 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feb680b47a0 0x7feb6813ba10 unknown :-1 s=CLOSED pgs=2294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:46.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:46.040+0000 7feb6e800640 1 --2- 172.21.15.152:0/3790248180 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feb68072530 0x7feb6813b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:46.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:46.040+0000 7feb6e800640 1 -- 172.21.15.152:0/3790248180 >> 172.21.15.152:0/3790248180 conn(0x7feb6806b980 msgr2=0x7feb680ad9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:46.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:46.041+0000 7feb6e800640 1 -- 172.21.15.152:0/3790248180 shutdown_connections 2024-04-20T11:37:46.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:46.041+0000 7feb6e800640 1 -- 172.21.15.152:0/3790248180 wait complete. 2024-04-20T11:37:46.069 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-20T11:37:46.266 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:46.266+0000 7fe5c9a00640 1 Processor -- start 2024-04-20T11:37:46.266 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:46.266+0000 7fe5c9a00640 1 -- start start 2024-04-20T11:37:46.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:46.268+0000 7fe5c9a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe5c40d0b80 0x7fe5c4160d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:46.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:46.268+0000 7fe5c9a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fe5c40ccff0 0x7fe5c40cd450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:46.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:46.268+0000 7fe5c9a00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5c4161290 con 0x7fe5c4071cd0 2024-04-20T11:37:46.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:46.268+0000 7fe5c9a00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fe5c40cd990 con 0x7fe5c40ccff0 2024-04-20T11:37:46.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:46.268+0000 7fe5c9a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe5c4145830 con 0x7fe5c40d0b80 2024-04-20T11:37:46.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:46.269+0000 7fe5ba000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fe5c40ccff0 0x7fe5c40cd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:46.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:46.269+0000 7fe5ba000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fe5c40ccff0 0x7fe5c40cd450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35034/0 (socket says 172.21.15.152:35034) 2024-04-20T11:37:46.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:46.269+0000 7fe5ba000640 1 -- 172.21.15.152:0/2310860879 learned_addr learned my addr 172.21.15.152:0/2310860879 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:46.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:46.269+0000 7fe5ba000640 1 -- 172.21.15.152:0/2310860879 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe5c40d0b80 msgr2=0x7fe5c4160d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:46.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:46.269+0000 7fe5ba000640 1 --2- 172.21.15.152:0/2310860879 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe5c40d0b80 0x7fe5c4160d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:46.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:46.269+0000 7fe5ba000640 1 -- 172.21.15.152:0/2310860879 >> v1:172.21.15.175:6789/0 conn(0x7fe5c4071cd0 legacy=0x7fe5c4072160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:46.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:46.269+0000 7fe5ba000640 1 -- 172.21.15.152:0/2310860879 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5c406f500 con 0x7fe5c40ccff0 2024-04-20T11:37:46.270 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:46.270+0000 7fe5ba000640 1 --2- 172.21.15.152:0/2310860879 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe5c40ccff0 0x7fe5c40cd450 secure :-1 s=READY pgs=2295 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac009870 tx=0x7fe5ac02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e34e1bb2ace4e77 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:46.270 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:46.271+0000 7fe5c2000640 1 -- 172.21.15.152:0/2310860879 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5ac035020 con 0x7fe5c40ccff0 2024-04-20T11:37:46.270 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:46.271+0000 7fe5c2000640 1 -- 172.21.15.152:0/2310860879 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5ac031640 con 0x7fe5c40ccff0 2024-04-20T11:37:46.270 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:46.271+0000 7fe5c2000640 1 -- 172.21.15.152:0/2310860879 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5ac02d040 con 0x7fe5c40ccff0 2024-04-20T11:37:46.271 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:46.271+0000 7fe5c9a00640 1 -- 172.21.15.152:0/2310860879 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe5c40ccff0 msgr2=0x7fe5c40cd450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:46.271 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:46.271+0000 7fe5c9a00640 1 --2- 172.21.15.152:0/2310860879 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe5c40ccff0 0x7fe5c40cd450 secure :-1 s=READY pgs=2295 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac009870 tx=0x7fe5ac02b8f0 comp rx=0 tx=0).stop 2024-04-20T11:37:46.271 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:46.272+0000 7fe5c9a00640 1 -- 172.21.15.152:0/2310860879 shutdown_connections 2024-04-20T11:37:46.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:46.272+0000 7fe5c9a00640 1 --2- 172.21.15.152:0/2310860879 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe5c40ccff0 0x7fe5c40cd450 unknown :-1 s=CLOSED pgs=2295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:46.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:46.272+0000 7fe5c9a00640 1 --2- 172.21.15.152:0/2310860879 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe5c40d0b80 0x7fe5c4160d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:46.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:46.272+0000 7fe5c9a00640 1 -- 172.21.15.152:0/2310860879 >> 172.21.15.152:0/2310860879 conn(0x7fe5c406b980 msgr2=0x7fe5c406bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:46.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:46.272+0000 7fe5c9a00640 1 -- 172.21.15.152:0/2310860879 shutdown_connections 2024-04-20T11:37:46.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:46.272+0000 7fe5c9a00640 1 -- 172.21.15.152:0/2310860879 wait complete. 2024-04-20T11:37:46.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:46.273+0000 7fe5c9a00640 1 Processor -- start 2024-04-20T11:37:46.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:46.273+0000 7fe5c9a00640 1 -- start start 2024-04-20T11:37:46.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:46.274+0000 7fe5c9a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe5c4071cd0 0x7fe5c415b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:46.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:46.274+0000 7fe5c9a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe5c40ccff0 0x7fe5c413b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:46.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:46.274+0000 7fe5c9a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe5c40d0b80 0x7fe5c413ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:46.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:46.274+0000 7fe5c9a00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c406e740 con 0x7fe5c40d0b80 2024-04-20T11:37:46.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:46.274+0000 7fe5c9a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c406f070 con 0x7fe5c4071cd0 2024-04-20T11:37:46.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:46.274+0000 7fe5c9a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe5c406ea20 con 0x7fe5c40ccff0 2024-04-20T11:37:46.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:46.275+0000 7fe5c3400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe5c40d0b80 0x7fe5c413ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:46.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:46.275+0000 7fe5c3400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe5c40d0b80 0x7fe5c413ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35044/0 (socket says 172.21.15.152:35044) 2024-04-20T11:37:46.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:46.275+0000 7fe5c3400640 1 -- 172.21.15.152:0/2903577491 learned_addr learned my addr 172.21.15.152:0/2903577491 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:46.276 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:46.275+0000 7fe5c2a00640 1 --2- 172.21.15.152:0/2903577491 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe5c4071cd0 0x7fe5c415b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:46.276 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:46.275+0000 7fe5ba000640 1 --2- 172.21.15.152:0/2903577491 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe5c40ccff0 0x7fe5c413b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:46.276 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:46.275+0000 7fe5c3400640 1 -- 172.21.15.152:0/2903577491 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe5c40ccff0 msgr2=0x7fe5c413b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:46.276 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:46.275+0000 7fe5c3400640 1 --2- 172.21.15.152:0/2903577491 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe5c40ccff0 0x7fe5c413b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:46.276 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:46.275+0000 7fe5c3400640 1 -- 172.21.15.152:0/2903577491 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe5c4071cd0 msgr2=0x7fe5c415b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:46.276 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:46.275+0000 7fe5c3400640 1 --2- 172.21.15.152:0/2903577491 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe5c4071cd0 0x7fe5c415b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:46.276 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:46.275+0000 7fe5c3400640 1 -- 172.21.15.152:0/2903577491 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5c41f2cd0 con 0x7fe5c40d0b80 2024-04-20T11:37:46.276 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:46.275+0000 7fe5c2a00640 1 --2- 172.21.15.152:0/2903577491 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe5c4071cd0 0x7fe5c415b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:37:46.276 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:46.276+0000 7fe5c3400640 1 --2- 172.21.15.152:0/2903577491 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe5c40d0b80 0x7fe5c413ccd0 secure :-1 s=READY pgs=2296 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc00c410 tx=0x7fe5bc00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:46.276 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:46.276+0000 7fe5bb400640 1 -- 172.21.15.152:0/2903577491 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5bc011790 con 0x7fe5c40d0b80 2024-04-20T11:37:46.277 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:46.276+0000 7fe5bb400640 1 -- 172.21.15.152:0/2903577491 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5bc017070 con 0x7fe5c40d0b80 2024-04-20T11:37:46.277 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:46.276+0000 7fe5bb400640 1 -- 172.21.15.152:0/2903577491 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5bc011bb0 con 0x7fe5c40d0b80 2024-04-20T11:37:46.277 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:46.276+0000 7fe5c9a00640 1 -- 172.21.15.152:0/2903577491 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5c41f2fb0 con 0x7fe5c40d0b80 2024-04-20T11:37:46.277 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:46.276+0000 7fe5c9a00640 1 -- 172.21.15.152:0/2903577491 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5c41f3570 con 0x7fe5c40d0b80 2024-04-20T11:37:46.277 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:46.277+0000 7fe5c9a00640 1 -- 172.21.15.152:0/2903577491 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5c415c940 con 0x7fe5c40d0b80 2024-04-20T11:37:46.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:46.279+0000 7fe5bb400640 1 -- 172.21.15.152:0/2903577491 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 26) ==== 91907+0+0 (secure 0 0 0) 0x7fe5bc007020 con 0x7fe5c40d0b80 2024-04-20T11:37:46.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:46.279+0000 7fe5bb400640 1 --2- 172.21.15.152:0/2903577491 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe5a006d280 0x7fe5a006f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:46.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:46.280+0000 7fe5bb400640 1 -- 172.21.15.152:0/2903577491 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(164..164 src has 1..164) ==== 5227+0+0 (secure 0 0 0) 0x7fe5bc08a8d0 con 0x7fe5c40d0b80 2024-04-20T11:37:46.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:46.280+0000 7fe5c2a00640 1 --2- 172.21.15.152:0/2903577491 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe5a006d280 0x7fe5a006f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:46.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:46.281+0000 7fe5c2a00640 1 --2- 172.21.15.152:0/2903577491 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe5a006d280 0x7fe5a006f740 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fe5c41e6e80 tx=0x7fe5b40073e0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:46.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:46.284+0000 7fe5bb400640 1 -- 172.21.15.152:0/2903577491 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe5c415c940 con 0x7fe5c40d0b80 2024-04-20T11:37:46.485 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:46.485+0000 7fe5c9a00640 1 -- 172.21.15.152:0/2903577491 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fe5c40689f0 con 0x7fe5c40d0b80 2024-04-20T11:37:47.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:47.043+0000 7fe5bb400640 1 -- 172.21.15.152:0/2903577491 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v165) ==== 170+0+0 (secure 0 0 0) 0x7fe5bc021020 con 0x7fe5c40d0b80 2024-04-20T11:37:47.043 INFO:teuthology.orchestra.run.smithi152.stderr:pool 'cephfs_data' created 2024-04-20T11:37:47.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:47.047+0000 7fe5c9a00640 1 -- 172.21.15.152:0/2903577491 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe5a006d280 msgr2=0x7fe5a006f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:47.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:47.047+0000 7fe5c9a00640 1 --2- 172.21.15.152:0/2903577491 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe5a006d280 0x7fe5a006f740 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fe5c41e6e80 tx=0x7fe5b40073e0 comp rx=0 tx=0).stop 2024-04-20T11:37:47.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:47.047+0000 7fe5c9a00640 1 -- 172.21.15.152:0/2903577491 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe5c40d0b80 msgr2=0x7fe5c413ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:47.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:47.047+0000 7fe5c9a00640 1 --2- 172.21.15.152:0/2903577491 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe5c40d0b80 0x7fe5c413ccd0 secure :-1 s=READY pgs=2296 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc00c410 tx=0x7fe5bc00c8d0 comp rx=0 tx=0).stop 2024-04-20T11:37:47.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:47.047+0000 7fe5c9a00640 1 -- 172.21.15.152:0/2903577491 shutdown_connections 2024-04-20T11:37:47.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:47.047+0000 7fe5c9a00640 1 --2- 172.21.15.152:0/2903577491 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe5a006d280 0x7fe5a006f740 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:47.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:47.047+0000 7fe5c9a00640 1 --2- 172.21.15.152:0/2903577491 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe5c40d0b80 0x7fe5c413ccd0 unknown :-1 s=CLOSED pgs=2296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:47.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:47.047+0000 7fe5c9a00640 1 --2- 172.21.15.152:0/2903577491 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe5c40ccff0 0x7fe5c413b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:47.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:47.047+0000 7fe5c9a00640 1 --2- 172.21.15.152:0/2903577491 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe5c4071cd0 0x7fe5c415b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:47.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:47.047+0000 7fe5c9a00640 1 -- 172.21.15.152:0/2903577491 >> 172.21.15.152:0/2903577491 conn(0x7fe5c406b980 msgr2=0x7fe5c40aff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:47.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:47.048+0000 7fe5c9a00640 1 -- 172.21.15.152:0/2903577491 shutdown_connections 2024-04-20T11:37:47.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:47.048+0000 7fe5c9a00640 1 -- 172.21.15.152:0/2903577491 wait complete. 2024-04-20T11:37:47.079 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-20T11:37:47.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:47.259+0000 7ff745c00640 1 Processor -- start 2024-04-20T11:37:47.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:47.259+0000 7ff745c00640 1 -- start start 2024-04-20T11:37:47.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:47.261+0000 7ff745c00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7ff7400cd8c0 0x7ff7400b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:47.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:47.261+0000 7ff745c00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7401463e0 con 0x7ff7400b2880 2024-04-20T11:37:47.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:47.261+0000 7ff745c00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7400b3e20 con 0x7ff740146000 2024-04-20T11:37:47.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:47.261+0000 7ff745c00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7ff7400b4750 con 0x7ff7400cd8c0 2024-04-20T11:37:47.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:47.261+0000 7ff73f400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7ff7400cd8c0 0x7ff7400b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:47.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:47.261+0000 7ff73e000640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7ff7400b2880 0x7ff7400cd1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:58434/0 (socket says 172.21.15.152:58434) 2024-04-20T11:37:47.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:47.261+0000 7ff73e000640 1 -- 172.21.15.152:0/1940269782 learned_addr learned my addr 172.21.15.152:0/1940269782 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:37:47.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:47.261+0000 7ff73f400640 1 -- 172.21.15.152:0/1940269782 >> v1:172.21.15.175:6789/0 conn(0x7ff740146000 legacy=0x7ff7400b2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:47.262 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:47.261+0000 7ff73f400640 1 -- 172.21.15.152:0/1940269782 >> v1:172.21.15.152:6789/0 conn(0x7ff7400b2880 legacy=0x7ff7400cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:47.262 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:47.262+0000 7ff73f400640 1 -- 172.21.15.152:0/1940269782 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7400b4100 con 0x7ff7400cd8c0 2024-04-20T11:37:47.262 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:47.262+0000 7ff73d600640 1 -- 172.21.15.152:0/1940269782 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2064463602 0 0) 0x7ff7401463e0 con 0x7ff7400b2880 2024-04-20T11:37:47.262 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:47.263+0000 7ff73f400640 1 --2- 172.21.15.152:0/1940269782 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff7400cd8c0 0x7ff7400b4be0 secure :-1 s=READY pgs=2298 cs=0 l=1 rev1=1 crypto rx=0x7ff7380097a0 tx=0x7ff73802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1c374bfe740c308 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:47.262 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:47.263+0000 7ff73d600640 1 -- 172.21.15.152:0/1940269782 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7380315a0 con 0x7ff7400cd8c0 2024-04-20T11:37:47.263 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:47.263+0000 7ff73d600640 1 -- 172.21.15.152:0/1940269782 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff738037070 con 0x7ff7400cd8c0 2024-04-20T11:37:47.263 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:47.263+0000 7ff73d600640 1 -- 172.21.15.152:0/1940269782 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff738031960 con 0x7ff7400cd8c0 2024-04-20T11:37:47.263 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:47.263+0000 7ff745c00640 1 -- 172.21.15.152:0/1940269782 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff7400cd8c0 msgr2=0x7ff7400b4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:47.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:47.263+0000 7ff745c00640 1 --2- 172.21.15.152:0/1940269782 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff7400cd8c0 0x7ff7400b4be0 secure :-1 s=READY pgs=2298 cs=0 l=1 rev1=1 crypto rx=0x7ff7380097a0 tx=0x7ff73802b840 comp rx=0 tx=0).stop 2024-04-20T11:37:47.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:47.264+0000 7ff745c00640 1 -- 172.21.15.152:0/1940269782 shutdown_connections 2024-04-20T11:37:47.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:47.264+0000 7ff745c00640 1 --2- 172.21.15.152:0/1940269782 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff7400cd8c0 0x7ff7400b4be0 unknown :-1 s=CLOSED pgs=2298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:47.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:47.264+0000 7ff745c00640 1 -- 172.21.15.152:0/1940269782 >> 172.21.15.152:0/1940269782 conn(0x7ff74006b980 msgr2=0x7ff74006bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:47.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:47.264+0000 7ff745c00640 1 -- 172.21.15.152:0/1940269782 shutdown_connections 2024-04-20T11:37:47.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:47.264+0000 7ff745c00640 1 -- 172.21.15.152:0/1940269782 wait complete. 2024-04-20T11:37:47.265 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:47.265+0000 7ff745c00640 1 Processor -- start 2024-04-20T11:37:47.265 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:47.266+0000 7ff745c00640 1 -- start start 2024-04-20T11:37:47.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:47.266+0000 7ff745c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7400b2880 0x7ff74015b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:47.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:47.266+0000 7ff745c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff7400cd8c0 0x7ff74013cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:47.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:47.266+0000 7ff745c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff740146000 0x7ff74013d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:47.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:47.267+0000 7ff745c00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7ff74015ce70 con 0x7ff740146000 2024-04-20T11:37:47.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:47.267+0000 7ff745c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff74015ccf0 con 0x7ff7400b2880 2024-04-20T11:37:47.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:47.267+0000 7ff745c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff74015cff0 con 0x7ff7400cd8c0 2024-04-20T11:37:47.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:47.267+0000 7ff73f400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff740146000 0x7ff74013d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:47.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:47.267+0000 7ff73f400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff740146000 0x7ff74013d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35062/0 (socket says 172.21.15.152:35062) 2024-04-20T11:37:47.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:47.267+0000 7ff73f400640 1 -- 172.21.15.152:0/1039724882 learned_addr learned my addr 172.21.15.152:0/1039724882 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:47.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:47.267+0000 7ff73ea00640 1 --2- 172.21.15.152:0/1039724882 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7400b2880 0x7ff74015b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:47.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:47.267+0000 7ff73e000640 1 --2- 172.21.15.152:0/1039724882 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff7400cd8c0 0x7ff74013cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:47.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:47.267+0000 7ff73f400640 1 -- 172.21.15.152:0/1039724882 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff7400cd8c0 msgr2=0x7ff74013cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:47.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:47.267+0000 7ff73f400640 1 --2- 172.21.15.152:0/1039724882 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff7400cd8c0 0x7ff74013cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:47.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:47.267+0000 7ff73f400640 1 -- 172.21.15.152:0/1039724882 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7400b2880 msgr2=0x7ff74015b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:47.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:47.267+0000 7ff73f400640 1 --2- 172.21.15.152:0/1039724882 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7400b2880 0x7ff74015b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:47.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:47.267+0000 7ff73f400640 1 -- 172.21.15.152:0/1039724882 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff74013bf80 con 0x7ff740146000 2024-04-20T11:37:47.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:47.267+0000 7ff73ea00640 1 --2- 172.21.15.152:0/1039724882 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7400b2880 0x7ff74015b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:37:47.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:47.267+0000 7ff73e000640 1 --2- 172.21.15.152:0/1039724882 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff7400cd8c0 0x7ff74013cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:37:47.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:47.268+0000 7ff73f400640 1 --2- 172.21.15.152:0/1039724882 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff740146000 0x7ff74013d240 secure :-1 s=READY pgs=2299 cs=0 l=1 rev1=1 crypto rx=0x7ff7380097a0 tx=0x7ff7380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:47.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:47.268+0000 7ff736a00640 1 -- 172.21.15.152:0/1039724882 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff738032840 con 0x7ff740146000 2024-04-20T11:37:47.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:47.268+0000 7ff745c00640 1 -- 172.21.15.152:0/1039724882 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff74006f4e0 con 0x7ff740146000 2024-04-20T11:37:47.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:47.268+0000 7ff736a00640 1 -- 172.21.15.152:0/1039724882 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7380329e0 con 0x7ff740146000 2024-04-20T11:37:47.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:47.268+0000 7ff736a00640 1 -- 172.21.15.152:0/1039724882 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff738032db0 con 0x7ff740146000 2024-04-20T11:37:47.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:47.268+0000 7ff745c00640 1 -- 172.21.15.152:0/1039724882 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff74006faa0 con 0x7ff740146000 2024-04-20T11:37:47.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:47.269+0000 7ff745c00640 1 -- 172.21.15.152:0/1039724882 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff74006f2d0 con 0x7ff740146000 2024-04-20T11:37:47.270 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:47.271+0000 7ff736a00640 1 -- 172.21.15.152:0/1039724882 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 26) ==== 91907+0+0 (secure 0 0 0) 0x7ff73804e470 con 0x7ff740146000 2024-04-20T11:37:47.271 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:47.272+0000 7ff736a00640 1 --2- 172.21.15.152:0/1039724882 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7ff71c06d280 0x7ff71c06f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:47.271 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:47.272+0000 7ff736a00640 1 -- 172.21.15.152:0/1039724882 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(165..165 src has 1..165) ==== 5609+0+0 (secure 0 0 0) 0x7ff73802f030 con 0x7ff740146000 2024-04-20T11:37:47.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:47.273+0000 7ff73ea00640 1 --2- 172.21.15.152:0/1039724882 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7ff71c06d280 0x7ff71c06f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:47.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:47.274+0000 7ff73ea00640 1 --2- 172.21.15.152:0/1039724882 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7ff71c06d280 0x7ff71c06f740 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7ff74015c870 tx=0x7ff7300073c0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:47.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:47.276+0000 7ff736a00640 1 -- 172.21.15.152:0/1039724882 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ff738040070 con 0x7ff740146000 2024-04-20T11:37:47.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:47.505+0000 7ff745c00640 1 -- 172.21.15.152:0/1039724882 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7ff7400689f0 con 0x7ff740146000 2024-04-20T11:37:48.091 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.091+0000 7ff736a00640 1 -- 172.21.15.152:0/1039724882 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 26 and data pool 27 v111) ==== 174+0+0 (secure 0 0 0) 0x7ff73807bec0 con 0x7ff740146000 2024-04-20T11:37:48.091 INFO:teuthology.orchestra.run.smithi152.stderr:new fs with metadata pool 26 and data pool 27 2024-04-20T11:37:48.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.099+0000 7ff745c00640 1 -- 172.21.15.152:0/1039724882 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7ff71c06d280 msgr2=0x7ff71c06f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:48.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.099+0000 7ff745c00640 1 --2- 172.21.15.152:0/1039724882 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7ff71c06d280 0x7ff71c06f740 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7ff74015c870 tx=0x7ff7300073c0 comp rx=0 tx=0).stop 2024-04-20T11:37:48.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.099+0000 7ff745c00640 1 -- 172.21.15.152:0/1039724882 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff740146000 msgr2=0x7ff74013d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:48.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.099+0000 7ff745c00640 1 --2- 172.21.15.152:0/1039724882 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff740146000 0x7ff74013d240 secure :-1 s=READY pgs=2299 cs=0 l=1 rev1=1 crypto rx=0x7ff7380097a0 tx=0x7ff7380046c0 comp rx=0 tx=0).stop 2024-04-20T11:37:48.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.100+0000 7ff745c00640 1 -- 172.21.15.152:0/1039724882 shutdown_connections 2024-04-20T11:37:48.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.100+0000 7ff745c00640 1 --2- 172.21.15.152:0/1039724882 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7ff71c06d280 0x7ff71c06f740 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:48.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.100+0000 7ff745c00640 1 --2- 172.21.15.152:0/1039724882 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff740146000 0x7ff74013d240 unknown :-1 s=CLOSED pgs=2299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:48.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.100+0000 7ff745c00640 1 --2- 172.21.15.152:0/1039724882 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff7400cd8c0 0x7ff74013cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:48.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.100+0000 7ff745c00640 1 --2- 172.21.15.152:0/1039724882 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7400b2880 0x7ff74015b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:48.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.100+0000 7ff745c00640 1 -- 172.21.15.152:0/1039724882 >> 172.21.15.152:0/1039724882 conn(0x7ff74006b980 msgr2=0x7ff7400b4be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:48.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.102+0000 7ff745c00640 1 -- 172.21.15.152:0/1039724882 shutdown_connections 2024-04-20T11:37:48.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.102+0000 7ff745c00640 1 -- 172.21.15.152:0/1039724882 wait complete. 2024-04-20T11:37:48.138 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:37:48.308 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.307+0000 7fd18ac00640 1 Processor -- start 2024-04-20T11:37:48.308 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.307+0000 7fd18ac00640 1 -- start start 2024-04-20T11:37:48.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.309+0000 7fd18ac00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd18415e890 0x7fd184162cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:48.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.309+0000 7fd18ac00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fd1840ae1e0 0x7fd1840afe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:48.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.309+0000 7fd18ac00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1840b55a0 con 0x7fd1840add80 2024-04-20T11:37:48.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.309+0000 7fd18ac00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fd1840b5880 con 0x7fd1840ae1e0 2024-04-20T11:37:48.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.309+0000 7fd18ac00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd1840b5b60 con 0x7fd18415e890 2024-04-20T11:37:48.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.309+0000 7fd18a200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fd1840ae1e0 0x7fd1840afe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:48.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.309+0000 7fd18a200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fd1840ae1e0 0x7fd1840afe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.152:43560/0 (socket says 172.21.15.152:43560) 2024-04-20T11:37:48.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.309+0000 7fd189800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd18415e890 0x7fd184162cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:48.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.309+0000 7fd18a200640 1 -- 172.21.15.152:0/1895016978 learned_addr learned my addr 172.21.15.152:0/1895016978 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:48.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.310+0000 7fd18a200640 1 -- 172.21.15.152:0/1895016978 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd18415e890 msgr2=0x7fd184162cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:48.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.310+0000 7fd18a200640 1 --2- 172.21.15.152:0/1895016978 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd18415e890 0x7fd184162cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:48.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.310+0000 7fd18a200640 1 -- 172.21.15.152:0/1895016978 >> v1:172.21.15.175:6789/0 conn(0x7fd1840add80 legacy=0x7fd1840af750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:48.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.310+0000 7fd18a200640 1 -- 172.21.15.152:0/1895016978 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1840b5e40 con 0x7fd1840ae1e0 2024-04-20T11:37:48.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.311+0000 7fd18a200640 1 --2- 172.21.15.152:0/1895016978 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd1840ae1e0 0x7fd1840afe60 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fd17c009e50 tx=0x7fd17c02bed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1bf3da97dd8f722 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:48.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.312+0000 7fd183e00640 1 -- 172.21.15.152:0/1895016978 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd17c02ca70 con 0x7fd1840ae1e0 2024-04-20T11:37:48.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.312+0000 7fd183e00640 1 -- 172.21.15.152:0/1895016978 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd17c02cc10 con 0x7fd1840ae1e0 2024-04-20T11:37:48.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.312+0000 7fd183e00640 1 -- 172.21.15.152:0/1895016978 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd17c0343f0 con 0x7fd1840ae1e0 2024-04-20T11:37:48.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.312+0000 7fd18ac00640 1 -- 172.21.15.152:0/1895016978 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd1840ae1e0 msgr2=0x7fd1840afe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:48.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.312+0000 7fd18ac00640 1 --2- 172.21.15.152:0/1895016978 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd1840ae1e0 0x7fd1840afe60 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fd17c009e50 tx=0x7fd17c02bed0 comp rx=0 tx=0).stop 2024-04-20T11:37:48.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.313+0000 7fd18ac00640 1 -- 172.21.15.152:0/1895016978 shutdown_connections 2024-04-20T11:37:48.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.313+0000 7fd18ac00640 1 --2- 172.21.15.152:0/1895016978 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd1840ae1e0 0x7fd1840afe60 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:48.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.313+0000 7fd18ac00640 1 --2- 172.21.15.152:0/1895016978 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd18415e890 0x7fd184162cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:48.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.313+0000 7fd18ac00640 1 -- 172.21.15.152:0/1895016978 >> 172.21.15.152:0/1895016978 conn(0x7fd184073340 msgr2=0x7fd18406b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:48.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.313+0000 7fd18ac00640 1 -- 172.21.15.152:0/1895016978 shutdown_connections 2024-04-20T11:37:48.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.313+0000 7fd18ac00640 1 -- 172.21.15.152:0/1895016978 wait complete. 2024-04-20T11:37:48.315 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.314+0000 7fd18ac00640 1 Processor -- start 2024-04-20T11:37:48.315 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.315+0000 7fd18ac00640 1 -- start start 2024-04-20T11:37:48.317 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.315+0000 7fd18ac00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd1840add80 0x7fd1841464f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:48.317 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.315+0000 7fd18ac00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd1840ae1e0 0x7fd1841540a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:48.317 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.316+0000 7fd18ac00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd18415e890 0x7fd1841614f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:48.317 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.316+0000 7fd18ac00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fd1840b5250 con 0x7fd1840ae1e0 2024-04-20T11:37:48.317 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.316+0000 7fd18ac00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd1840ae610 con 0x7fd1840add80 2024-04-20T11:37:48.317 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.316+0000 7fd18ac00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd1840b6640 con 0x7fd18415e890 2024-04-20T11:37:48.317 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.316+0000 7fd188e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd1840ae1e0 0x7fd1841540a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:48.317 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.316+0000 7fd188e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd1840ae1e0 0x7fd1841540a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35070/0 (socket says 172.21.15.152:35070) 2024-04-20T11:37:48.317 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.316+0000 7fd188e00640 1 -- 172.21.15.152:0/2013289413 learned_addr learned my addr 172.21.15.152:0/2013289413 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:48.317 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.316+0000 7fd189800640 1 --2- 172.21.15.152:0/2013289413 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd1840add80 0x7fd1841464f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:48.318 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.316+0000 7fd18a200640 1 --2- 172.21.15.152:0/2013289413 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd18415e890 0x7fd1841614f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:48.318 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.316+0000 7fd188e00640 1 -- 172.21.15.152:0/2013289413 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd18415e890 msgr2=0x7fd1841614f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:48.318 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.316+0000 7fd188e00640 1 --2- 172.21.15.152:0/2013289413 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd18415e890 0x7fd1841614f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:48.318 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.316+0000 7fd188e00640 1 -- 172.21.15.152:0/2013289413 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd1840add80 msgr2=0x7fd1841464f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:48.318 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.316+0000 7fd188e00640 1 --2- 172.21.15.152:0/2013289413 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd1840add80 0x7fd1841464f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:48.318 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.316+0000 7fd188e00640 1 -- 172.21.15.152:0/2013289413 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd184160450 con 0x7fd1840ae1e0 2024-04-20T11:37:48.318 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.317+0000 7fd188e00640 1 --2- 172.21.15.152:0/2013289413 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd1840ae1e0 0x7fd1841540a0 secure :-1 s=READY pgs=2300 cs=0 l=1 rev1=1 crypto rx=0x7fd17000c400 tx=0x7fd17000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:48.319 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.317+0000 7fd181600640 1 -- 172.21.15.152:0/2013289413 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1700052d0 con 0x7fd1840ae1e0 2024-04-20T11:37:48.319 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.317+0000 7fd181600640 1 -- 172.21.15.152:0/2013289413 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd170005490 con 0x7fd1840ae1e0 2024-04-20T11:37:48.319 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.317+0000 7fd181600640 1 -- 172.21.15.152:0/2013289413 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd170007260 con 0x7fd1840ae1e0 2024-04-20T11:37:48.319 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.317+0000 7fd18ac00640 1 -- 172.21.15.152:0/2013289413 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd184160730 con 0x7fd1840ae1e0 2024-04-20T11:37:48.319 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.317+0000 7fd18ac00640 1 -- 172.21.15.152:0/2013289413 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd184160cf0 con 0x7fd1840ae1e0 2024-04-20T11:37:48.319 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.318+0000 7fd18ac00640 1 -- 172.21.15.152:0/2013289413 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd184075e60 con 0x7fd1840ae1e0 2024-04-20T11:37:48.320 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.319+0000 7fd181600640 1 -- 172.21.15.152:0/2013289413 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 26) ==== 91907+0+0 (secure 0 0 0) 0x7fd170007400 con 0x7fd1840ae1e0 2024-04-20T11:37:48.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.321+0000 7fd181600640 1 --2- 172.21.15.152:0/2013289413 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd16406d2b0 0x7fd16406f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:48.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.321+0000 7fd181600640 1 -- 172.21.15.152:0/2013289413 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(166..166 src has 1..166) ==== 5709+0+0 (secure 0 0 0) 0x7fd170089880 con 0x7fd1840ae1e0 2024-04-20T11:37:48.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.321+0000 7fd189800640 1 --2- 172.21.15.152:0/2013289413 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd16406d2b0 0x7fd16406f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:48.323 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.323+0000 7fd189800640 1 --2- 172.21.15.152:0/2013289413 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd16406d2b0 0x7fd16406f770 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fd178009970 tx=0x7fd178008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:48.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.325+0000 7fd181600640 1 -- 172.21.15.152:0/2013289413 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fd170059820 con 0x7fd1840ae1e0 2024-04-20T11:37:48.525 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.524+0000 7fd18ac00640 1 -- 172.21.15.152:0/2013289413 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd1840d09b0 con 0x7fd1840ae1e0 2024-04-20T11:37:48.528 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.527+0000 7fd181600640 1 -- 172.21.15.152:0/2013289413 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v166) ==== 74+0+16756 (secure 0 0 0) 0x7fd170059120 con 0x7fd1840ae1e0 2024-04-20T11:37:48.528 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:37:48.528 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":166,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:37:47.506816+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:37:45.247938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"166","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2024-04-20T11:37:46.503171+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"166","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:37:48.531 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.530+0000 7fd18ac00640 1 -- 172.21.15.152:0/2013289413 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd16406d2b0 msgr2=0x7fd16406f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:48.531 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.530+0000 7fd18ac00640 1 --2- 172.21.15.152:0/2013289413 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd16406d2b0 0x7fd16406f770 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fd178009970 tx=0x7fd178008040 comp rx=0 tx=0).stop 2024-04-20T11:37:48.531 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.530+0000 7fd18ac00640 1 -- 172.21.15.152:0/2013289413 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd1840ae1e0 msgr2=0x7fd1841540a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:48.531 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.530+0000 7fd18ac00640 1 --2- 172.21.15.152:0/2013289413 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd1840ae1e0 0x7fd1841540a0 secure :-1 s=READY pgs=2300 cs=0 l=1 rev1=1 crypto rx=0x7fd17000c400 tx=0x7fd17000c8c0 comp rx=0 tx=0).stop 2024-04-20T11:37:48.531 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.530+0000 7fd18ac00640 1 -- 172.21.15.152:0/2013289413 shutdown_connections 2024-04-20T11:37:48.531 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.530+0000 7fd18ac00640 1 --2- 172.21.15.152:0/2013289413 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd16406d2b0 0x7fd16406f770 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:48.531 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.530+0000 7fd18ac00640 1 --2- 172.21.15.152:0/2013289413 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd18415e890 0x7fd1841614f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:48.531 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.530+0000 7fd18ac00640 1 --2- 172.21.15.152:0/2013289413 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd1840ae1e0 0x7fd1841540a0 unknown :-1 s=CLOSED pgs=2300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:48.531 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.530+0000 7fd18ac00640 1 --2- 172.21.15.152:0/2013289413 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd1840add80 0x7fd1841464f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:48.531 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.530+0000 7fd18ac00640 1 -- 172.21.15.152:0/2013289413 >> 172.21.15.152:0/2013289413 conn(0x7fd184073340 msgr2=0x7fd1840b0940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:48.532 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.530+0000 7fd18ac00640 1 -- 172.21.15.152:0/2013289413 shutdown_connections 2024-04-20T11:37:48.532 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.530+0000 7fd18ac00640 1 -- 172.21.15.152:0/2013289413 wait complete. 2024-04-20T11:37:48.549 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:37:48.733 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.732+0000 7f4c7f200640 1 Processor -- start 2024-04-20T11:37:48.733 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.733+0000 7f4c7f200640 1 -- start start 2024-04-20T11:37:48.734 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.734+0000 7f4c7f200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f4c80072c70 0x7f4c800b35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:48.734 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.734+0000 7f4c7f200640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c8015b340 con 0x7f4c800b3b00 2024-04-20T11:37:48.734 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.734+0000 7f4c7f200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c8015b620 con 0x7f4c80151ab0 2024-04-20T11:37:48.735 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.734+0000 7f4c7f200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f4c8015c100 con 0x7f4c80072c70 2024-04-20T11:37:48.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.734+0000 7f4c7e800640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f4c800b3b00 0x7f4c8015e720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:58446/0 (socket says 172.21.15.152:58446) 2024-04-20T11:37:48.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.734+0000 7f4c7e800640 1 -- 172.21.15.152:0/3584884394 learned_addr learned my addr 172.21.15.152:0/3584884394 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:37:48.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.734+0000 7f4c7d400640 1 --2- 172.21.15.152:0/3584884394 >> v2:172.21.15.175:3300/0 conn(0x7f4c80072c70 0x7f4c800b35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:48.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.735+0000 7f4c7ca00640 1 -- 172.21.15.152:0/3584884394 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2860390338 0 0) 0x7f4c8015b340 con 0x7f4c800b3b00 2024-04-20T11:37:48.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.735+0000 7f4c7ca00640 1 -- 172.21.15.152:0/3584884394 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4c64003440 con 0x7f4c800b3b00 2024-04-20T11:37:48.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.735+0000 7f4c7ca00640 1 -- 172.21.15.152:0/3584884394 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1851542931 0 0) 0x7f4c64003440 con 0x7f4c800b3b00 2024-04-20T11:37:48.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.735+0000 7f4c7ca00640 1 -- 172.21.15.152:0/3584884394 >> v2:172.21.15.175:3300/0 conn(0x7f4c80072c70 msgr2=0x7f4c800b35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:48.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.735+0000 7f4c7ca00640 1 --2- 172.21.15.152:0/3584884394 >> v2:172.21.15.175:3300/0 conn(0x7f4c80072c70 0x7f4c800b35c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:48.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.735+0000 7f4c7ca00640 1 -- 172.21.15.152:0/3584884394 >> v1:172.21.15.175:6789/0 conn(0x7f4c80151ab0 legacy=0x7f4c80072560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:48.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.735+0000 7f4c7ca00640 1 -- 172.21.15.152:0/3584884394 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c8015b900 con 0x7f4c800b3b00 2024-04-20T11:37:48.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.735+0000 7f4c7ca00640 1 -- 172.21.15.152:0/3584884394 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f4c78001670 con 0x7f4c800b3b00 2024-04-20T11:37:48.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.735+0000 7f4c7ca00640 1 -- 172.21.15.152:0/3584884394 >> v1:172.21.15.152:6789/0 conn(0x7f4c800b3b00 legacy=0x7f4c8015e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:48.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.735+0000 7f4c7ca00640 1 --2- 172.21.15.152:0/3584884394 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4c64003aa0 0x7f4c64003ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:48.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.735+0000 7f4c7ca00640 1 --2- 172.21.15.152:0/3584884394 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c64008850 0x7f4c6400acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:48.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.735+0000 7f4c7ca00640 1 --2- 172.21.15.152:0/3584884394 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4c6400b4c0 0x7f4c6400d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:48.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.735+0000 7f4c7ca00640 1 -- 172.21.15.152:0/3584884394 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f4c8015c100 con 0x7f4c64008850 2024-04-20T11:37:48.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.735+0000 7f4c7ca00640 1 -- 172.21.15.152:0/3584884394 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4c6400e260 con 0x7f4c6400b4c0 2024-04-20T11:37:48.738 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.735+0000 7f4c7ca00640 1 -- 172.21.15.152:0/3584884394 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4c6400e460 con 0x7f4c64003aa0 2024-04-20T11:37:48.738 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.735+0000 7f4c7d400640 1 --2- 172.21.15.152:0/3584884394 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c64008850 0x7f4c6400acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:48.738 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.736+0000 7f4c7de00640 1 --2- 172.21.15.152:0/3584884394 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4c64003aa0 0x7f4c64003ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:48.738 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.736+0000 7f4c7d400640 1 -- 172.21.15.152:0/3584884394 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4c64003aa0 msgr2=0x7f4c64003ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:48.738 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.736+0000 7f4c7d400640 1 --2- 172.21.15.152:0/3584884394 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4c64003aa0 0x7f4c64003ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:48.738 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.736+0000 7f4c7e800640 1 --2- 172.21.15.152:0/3584884394 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4c6400b4c0 0x7f4c6400d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:48.738 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.736+0000 7f4c7d400640 1 -- 172.21.15.152:0/3584884394 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4c6400b4c0 msgr2=0x7f4c6400d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:48.738 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.736+0000 7f4c7d400640 1 --2- 172.21.15.152:0/3584884394 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4c6400b4c0 0x7f4c6400d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:48.738 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.736+0000 7f4c7d400640 1 -- 172.21.15.152:0/3584884394 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c64003440 con 0x7f4c64008850 2024-04-20T11:37:48.738 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.736+0000 7f4c7e800640 1 -- 172.21.15.152:0/3584884394 reap_dead start 2024-04-20T11:37:48.739 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.737+0000 7f4c7d400640 1 --2- 172.21.15.152:0/3584884394 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c64008850 0x7f4c6400acc0 secure :-1 s=READY pgs=2302 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c009ac0 tx=0x7f4c64003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5f3b005cb87c228 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:48.739 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.737+0000 7f4c7ca00640 1 -- 172.21.15.152:0/3584884394 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c6c039070 con 0x7f4c64008850 2024-04-20T11:37:48.739 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.737+0000 7f4c7ca00640 1 -- 172.21.15.152:0/3584884394 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c6c02c6b0 con 0x7f4c64008850 2024-04-20T11:37:48.739 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.737+0000 7f4c7ca00640 1 -- 172.21.15.152:0/3584884394 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c6c009690 con 0x7f4c64008850 2024-04-20T11:37:48.740 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.738+0000 7f4c7f200640 1 -- 172.21.15.152:0/3584884394 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c64008850 msgr2=0x7f4c6400acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:48.740 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.738+0000 7f4c7f200640 1 --2- 172.21.15.152:0/3584884394 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c64008850 0x7f4c6400acc0 secure :-1 s=READY pgs=2302 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c009ac0 tx=0x7f4c64003730 comp rx=0 tx=0).stop 2024-04-20T11:37:48.740 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.738+0000 7f4c7f200640 1 -- 172.21.15.152:0/3584884394 shutdown_connections 2024-04-20T11:37:48.740 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.738+0000 7f4c7f200640 1 --2- 172.21.15.152:0/3584884394 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c64008850 0x7f4c6400acc0 unknown :-1 s=CLOSED pgs=2302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:48.740 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.738+0000 7f4c7f200640 1 -- 172.21.15.152:0/3584884394 >> 172.21.15.152:0/3584884394 conn(0x7f4c80067ae0 msgr2=0x7f4c8006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:48.740 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.738+0000 7f4c7f200640 1 -- 172.21.15.152:0/3584884394 shutdown_connections 2024-04-20T11:37:48.740 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.738+0000 7f4c7f200640 1 -- 172.21.15.152:0/3584884394 wait complete. 2024-04-20T11:37:48.740 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.740+0000 7f4c7f200640 1 Processor -- start 2024-04-20T11:37:48.741 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.740+0000 7f4c7f200640 1 -- start start 2024-04-20T11:37:48.743 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.741+0000 7f4c7f200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c64008850 0x7f4c80140bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:48.743 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.741+0000 7f4c7f200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4c8015d400 0x7f4c8015d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:48.743 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.741+0000 7f4c7f200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4c800adff0 0x7f4c800ae470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:48.743 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.741+0000 7f4c7f200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f4c800b3f30 con 0x7f4c64008850 2024-04-20T11:37:48.743 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.741+0000 7f4c7f200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4c8015bc70 con 0x7f4c800adff0 2024-04-20T11:37:48.743 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.741+0000 7f4c7f200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4c800b40b0 con 0x7f4c8015d400 2024-04-20T11:37:48.743 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.741+0000 7f4c7de00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c64008850 0x7f4c80140bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:48.743 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.741+0000 7f4c7de00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c64008850 0x7f4c80140bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35078/0 (socket says 172.21.15.152:35078) 2024-04-20T11:37:48.743 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.741+0000 7f4c7de00640 1 -- 172.21.15.152:0/934320040 learned_addr learned my addr 172.21.15.152:0/934320040 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:48.743 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.741+0000 7f4c7d400640 1 --2- 172.21.15.152:0/934320040 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4c8015d400 0x7f4c8015d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:48.743 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.741+0000 7f4c7de00640 1 -- 172.21.15.152:0/934320040 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4c8015d400 msgr2=0x7f4c8015d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:48.743 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.741+0000 7f4c7de00640 1 --2- 172.21.15.152:0/934320040 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4c8015d400 0x7f4c8015d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:48.743 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.741+0000 7f4c7de00640 1 -- 172.21.15.152:0/934320040 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4c800adff0 msgr2=0x7f4c800ae470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:48.744 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.741+0000 7f4c7e800640 1 --2- 172.21.15.152:0/934320040 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4c800adff0 0x7f4c800ae470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:48.744 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.742+0000 7f4c7de00640 1 --2- 172.21.15.152:0/934320040 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4c800adff0 0x7f4c800ae470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:48.744 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.742+0000 7f4c7de00640 1 -- 172.21.15.152:0/934320040 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c801467f0 con 0x7f4c64008850 2024-04-20T11:37:48.744 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.742+0000 7f4c7de00640 1 --2- 172.21.15.152:0/934320040 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c64008850 0x7f4c80140bd0 secure :-1 s=READY pgs=2303 cs=0 l=1 rev1=1 crypto rx=0x7f4c7000e920 tx=0x7f4c7000ede0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:48.744 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.742+0000 7f4c76000640 1 -- 172.21.15.152:0/934320040 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c70003b80 con 0x7f4c64008850 2024-04-20T11:37:48.744 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.742+0000 7f4c76000640 1 -- 172.21.15.152:0/934320040 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c70018070 con 0x7f4c64008850 2024-04-20T11:37:48.744 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.742+0000 7f4c76000640 1 -- 172.21.15.152:0/934320040 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c700050d0 con 0x7f4c64008850 2024-04-20T11:37:48.744 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.742+0000 7f4c7f200640 1 -- 172.21.15.152:0/934320040 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c800aebd0 con 0x7f4c64008850 2024-04-20T11:37:48.744 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.742+0000 7f4c7f200640 1 -- 172.21.15.152:0/934320040 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c800af190 con 0x7f4c64008850 2024-04-20T11:37:48.745 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.743+0000 7f4c7f200640 1 -- 172.21.15.152:0/934320040 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c80075e60 con 0x7f4c64008850 2024-04-20T11:37:48.745 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.744+0000 7f4c76000640 1 -- 172.21.15.152:0/934320040 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 26) ==== 91907+0+0 (secure 0 0 0) 0x7f4c70005270 con 0x7f4c64008850 2024-04-20T11:37:48.745 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.745+0000 7f4c76000640 1 --2- 172.21.15.152:0/934320040 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4c6006d2b0 0x7f4c6006f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:48.746 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.745+0000 7f4c76000640 1 -- 172.21.15.152:0/934320040 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(166..166 src has 1..166) ==== 5709+0+0 (secure 0 0 0) 0x7f4c7008a370 con 0x7f4c64008850 2024-04-20T11:37:48.746 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.746+0000 7f4c7d400640 1 --2- 172.21.15.152:0/934320040 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4c6006d2b0 0x7f4c6006f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:48.747 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.747+0000 7f4c7d400640 1 --2- 172.21.15.152:0/934320040 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4c6006d2b0 0x7f4c6006f770 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c036000 tx=0x7f4c6c030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:48.749 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.749+0000 7f4c76000640 1 -- 172.21.15.152:0/934320040 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f4c7001a310 con 0x7f4c64008850 2024-04-20T11:37:48.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.936+0000 7f4c7f200640 1 -- 172.21.15.152:0/934320040 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4c8006f4c0 con 0x7f4c64008850 2024-04-20T11:37:48.938 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.937+0000 7f4c76000640 1 -- 172.21.15.152:0/934320040 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v166) ==== 74+0+16756 (secure 0 0 0) 0x7f4c7005a3a0 con 0x7f4c64008850 2024-04-20T11:37:48.938 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:37:48.938 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":166,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:37:47.506816+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:37:45.247938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"166","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2024-04-20T11:37:46.503171+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"166","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:37:48.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.939+0000 7f4c7f200640 1 -- 172.21.15.152:0/934320040 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4c6006d2b0 msgr2=0x7f4c6006f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:48.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.939+0000 7f4c7f200640 1 --2- 172.21.15.152:0/934320040 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4c6006d2b0 0x7f4c6006f770 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c036000 tx=0x7f4c6c030000 comp rx=0 tx=0).stop 2024-04-20T11:37:48.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.939+0000 7f4c7f200640 1 -- 172.21.15.152:0/934320040 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c64008850 msgr2=0x7f4c80140bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:48.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.939+0000 7f4c7f200640 1 --2- 172.21.15.152:0/934320040 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c64008850 0x7f4c80140bd0 secure :-1 s=READY pgs=2303 cs=0 l=1 rev1=1 crypto rx=0x7f4c7000e920 tx=0x7f4c7000ede0 comp rx=0 tx=0).stop 2024-04-20T11:37:48.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.939+0000 7f4c7f200640 1 -- 172.21.15.152:0/934320040 shutdown_connections 2024-04-20T11:37:48.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.939+0000 7f4c7f200640 1 --2- 172.21.15.152:0/934320040 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4c6006d2b0 0x7f4c6006f770 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:48.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.940+0000 7f4c7f200640 1 --2- 172.21.15.152:0/934320040 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4c800adff0 0x7f4c800ae470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:48.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.940+0000 7f4c7f200640 1 --2- 172.21.15.152:0/934320040 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4c8015d400 0x7f4c8015d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:48.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.940+0000 7f4c7f200640 1 --2- 172.21.15.152:0/934320040 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c64008850 0x7f4c80140bd0 unknown :-1 s=CLOSED pgs=2303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:48.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.940+0000 7f4c7f200640 1 -- 172.21.15.152:0/934320040 >> 172.21.15.152:0/934320040 conn(0x7f4c80067ae0 msgr2=0x7f4c800b4880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:48.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.940+0000 7f4c7f200640 1 -- 172.21.15.152:0/934320040 shutdown_connections 2024-04-20T11:37:48.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:48.940+0000 7f4c7f200640 1 -- 172.21.15.152:0/934320040 wait complete. 2024-04-20T11:37:48.959 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-20T11:37:49.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:49.163+0000 7f395c400640 1 Processor -- start 2024-04-20T11:37:49.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:49.163+0000 7f395c400640 1 -- start start 2024-04-20T11:37:49.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:49.165+0000 7f395c400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f39540b2880 0x7f39540b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:49.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:49.165+0000 7f395c400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f39540cd1b0 0x7f39540cd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:49.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:49.165+0000 7f395c400640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39541463e0 con 0x7f3954146000 2024-04-20T11:37:49.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:49.165+0000 7f395c400640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f39540b4750 con 0x7f39540b2880 2024-04-20T11:37:49.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:49.165+0000 7f395c400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f39540b3e20 con 0x7f39540cd1b0 2024-04-20T11:37:49.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:49.166+0000 7f3959000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f39540b2880 0x7f39540b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:49.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:49.166+0000 7f3959000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f39540b2880 0x7f39540b4be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35086/0 (socket says 172.21.15.152:35086) 2024-04-20T11:37:49.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:49.166+0000 7f3959000640 1 -- 172.21.15.152:0/2894227132 learned_addr learned my addr 172.21.15.152:0/2894227132 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:49.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:49.166+0000 7f3959000640 1 -- 172.21.15.152:0/2894227132 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f39540cd1b0 msgr2=0x7f39540cd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:49.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:49.166+0000 7f3959000640 1 --2- 172.21.15.152:0/2894227132 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f39540cd1b0 0x7f39540cd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:49.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:49.166+0000 7f3959000640 1 -- 172.21.15.152:0/2894227132 >> v1:172.21.15.175:6789/0 conn(0x7f3954146000 legacy=0x7f39540b2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:49.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:49.166+0000 7f3959000640 1 -- 172.21.15.152:0/2894227132 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39540b4100 con 0x7f39540b2880 2024-04-20T11:37:49.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:49.167+0000 7f3959000640 1 --2- 172.21.15.152:0/2894227132 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f39540b2880 0x7f39540b4be0 secure :-1 s=READY pgs=2304 cs=0 l=1 rev1=1 crypto rx=0x7f3940009870 tx=0x7f394002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3804e246b16bde6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:49.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:49.168+0000 7f3953e00640 1 -- 172.21.15.152:0/2894227132 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3940035020 con 0x7f39540b2880 2024-04-20T11:37:49.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:49.168+0000 7f3953e00640 1 -- 172.21.15.152:0/2894227132 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3940031640 con 0x7f39540b2880 2024-04-20T11:37:49.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:49.168+0000 7f3953e00640 1 -- 172.21.15.152:0/2894227132 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f394002d040 con 0x7f39540b2880 2024-04-20T11:37:49.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:49.168+0000 7f395c400640 1 -- 172.21.15.152:0/2894227132 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f39540b2880 msgr2=0x7f39540b4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:49.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:49.168+0000 7f395c400640 1 --2- 172.21.15.152:0/2894227132 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f39540b2880 0x7f39540b4be0 secure :-1 s=READY pgs=2304 cs=0 l=1 rev1=1 crypto rx=0x7f3940009870 tx=0x7f394002b8f0 comp rx=0 tx=0).stop 2024-04-20T11:37:49.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:49.168+0000 7f395c400640 1 -- 172.21.15.152:0/2894227132 shutdown_connections 2024-04-20T11:37:49.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:49.169+0000 7f395c400640 1 --2- 172.21.15.152:0/2894227132 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f39540cd1b0 0x7f39540cd5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:49.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:49.169+0000 7f395c400640 1 --2- 172.21.15.152:0/2894227132 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f39540b2880 0x7f39540b4be0 unknown :-1 s=CLOSED pgs=2304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:49.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:49.169+0000 7f395c400640 1 -- 172.21.15.152:0/2894227132 >> 172.21.15.152:0/2894227132 conn(0x7f395406b980 msgr2=0x7f395406bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:49.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:49.169+0000 7f395c400640 1 -- 172.21.15.152:0/2894227132 shutdown_connections 2024-04-20T11:37:49.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:49.169+0000 7f395c400640 1 -- 172.21.15.152:0/2894227132 wait complete. 2024-04-20T11:37:49.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:49.170+0000 7f395c400640 1 Processor -- start 2024-04-20T11:37:49.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:49.170+0000 7f395c400640 1 -- start start 2024-04-20T11:37:49.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:49.171+0000 7f395c400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39540b2880 0x7f39541444a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:49.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:49.171+0000 7f395c400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f39540cd1b0 0x7f395415efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:49.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:49.171+0000 7f395c400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3954146000 0x7f395415f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:49.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:49.171+0000 7f395c400640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f39540cdb30 con 0x7f3954146000 2024-04-20T11:37:49.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:49.171+0000 7f395c400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f395415ccf0 con 0x7f39540b2880 2024-04-20T11:37:49.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:49.171+0000 7f395c400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f39541f4990 con 0x7f39540cd1b0 2024-04-20T11:37:49.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:49.172+0000 7f395a400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3954146000 0x7f395415f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:49.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:49.172+0000 7f395a400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3954146000 0x7f395415f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35088/0 (socket says 172.21.15.152:35088) 2024-04-20T11:37:49.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:49.172+0000 7f395a400640 1 -- 172.21.15.152:0/100577463 learned_addr learned my addr 172.21.15.152:0/100577463 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:49.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:49.172+0000 7f3959a00640 1 --2- 172.21.15.152:0/100577463 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39540b2880 0x7f39541444a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:49.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:49.172+0000 7f3959000640 1 --2- 172.21.15.152:0/100577463 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f39540cd1b0 0x7f395415efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:49.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:49.172+0000 7f395a400640 1 -- 172.21.15.152:0/100577463 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f39540cd1b0 msgr2=0x7f395415efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:49.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:49.172+0000 7f395a400640 1 --2- 172.21.15.152:0/100577463 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f39540cd1b0 0x7f395415efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:49.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:49.172+0000 7f395a400640 1 -- 172.21.15.152:0/100577463 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39540b2880 msgr2=0x7f39541444a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:49.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:49.172+0000 7f395a400640 1 --2- 172.21.15.152:0/100577463 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39540b2880 0x7f39541444a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:49.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:49.172+0000 7f395a400640 1 -- 172.21.15.152:0/100577463 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39541f4c70 con 0x7f3954146000 2024-04-20T11:37:49.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:49.172+0000 7f3959000640 1 --2- 172.21.15.152:0/100577463 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f39540cd1b0 0x7f395415efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:37:49.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:49.173+0000 7f395a400640 1 --2- 172.21.15.152:0/100577463 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3954146000 0x7f395415f4e0 secure :-1 s=READY pgs=2305 cs=0 l=1 rev1=1 crypto rx=0x7f394c00b970 tx=0x7f394c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:49.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:49.173+0000 7f3951600640 1 -- 172.21.15.152:0/100577463 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f394c019070 con 0x7f3954146000 2024-04-20T11:37:49.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:49.173+0000 7f3951600640 1 -- 172.21.15.152:0/100577463 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f394c00cb60 con 0x7f3954146000 2024-04-20T11:37:49.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:49.173+0000 7f3951600640 1 -- 172.21.15.152:0/100577463 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f394c014440 con 0x7f3954146000 2024-04-20T11:37:49.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:49.173+0000 7f395c400640 1 -- 172.21.15.152:0/100577463 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39541f4f50 con 0x7f3954146000 2024-04-20T11:37:49.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:49.173+0000 7f395c400640 1 -- 172.21.15.152:0/100577463 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39541449e0 con 0x7f3954146000 2024-04-20T11:37:49.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:49.174+0000 7f395c400640 1 -- 172.21.15.152:0/100577463 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39540689f0 con 0x7f3954146000 2024-04-20T11:37:49.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:49.175+0000 7f3951600640 1 -- 172.21.15.152:0/100577463 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 26) ==== 91907+0+0 (secure 0 0 0) 0x7f394c01c070 con 0x7f3954146000 2024-04-20T11:37:49.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:49.176+0000 7f3951600640 1 --2- 172.21.15.152:0/100577463 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f393406d280 0x7f393406f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:49.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:49.176+0000 7f3951600640 1 -- 172.21.15.152:0/100577463 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 5709+0+0 (secure 0 0 0) 0x7f394c08a040 con 0x7f3954146000 2024-04-20T11:37:49.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:49.177+0000 7f3959a00640 1 --2- 172.21.15.152:0/100577463 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f393406d280 0x7f393406f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:49.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:49.178+0000 7f3959a00640 1 --2- 172.21.15.152:0/100577463 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f393406d280 0x7f393406f740 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f39540d0b60 tx=0x7f3948009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:49.181 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:49.181+0000 7f3951600640 1 -- 172.21.15.152:0/100577463 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f394c05a070 con 0x7f3954146000 2024-04-20T11:37:49.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:49.429+0000 7f395c400640 1 -- 172.21.15.152:0/100577463 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f39540b2f10 con 0x7f3954146000 2024-04-20T11:37:50.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:50.105+0000 7f3951600640 1 -- 172.21.15.152:0/100577463 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v114) ==== 118+0+0 (secure 0 0 0) 0x7f394c059970 con 0x7f3954146000 2024-04-20T11:37:50.111 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:50.109+0000 7f395c400640 1 -- 172.21.15.152:0/100577463 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f393406d280 msgr2=0x7f393406f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:50.111 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:50.109+0000 7f395c400640 1 --2- 172.21.15.152:0/100577463 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f393406d280 0x7f393406f740 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f39540d0b60 tx=0x7f3948009040 comp rx=0 tx=0).stop 2024-04-20T11:37:50.111 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:50.109+0000 7f395c400640 1 -- 172.21.15.152:0/100577463 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3954146000 msgr2=0x7f395415f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:50.111 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:50.109+0000 7f395c400640 1 --2- 172.21.15.152:0/100577463 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3954146000 0x7f395415f4e0 secure :-1 s=READY pgs=2305 cs=0 l=1 rev1=1 crypto rx=0x7f394c00b970 tx=0x7f394c00be30 comp rx=0 tx=0).stop 2024-04-20T11:37:50.111 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:50.110+0000 7f395c400640 1 -- 172.21.15.152:0/100577463 shutdown_connections 2024-04-20T11:37:50.111 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:50.110+0000 7f395c400640 1 --2- 172.21.15.152:0/100577463 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f393406d280 0x7f393406f740 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:50.111 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:50.110+0000 7f395c400640 1 --2- 172.21.15.152:0/100577463 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3954146000 0x7f395415f4e0 unknown :-1 s=CLOSED pgs=2305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:50.111 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:50.110+0000 7f395c400640 1 --2- 172.21.15.152:0/100577463 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f39540cd1b0 0x7f395415efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:50.111 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:50.110+0000 7f395c400640 1 --2- 172.21.15.152:0/100577463 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39540b2880 0x7f39541444a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:50.111 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:50.110+0000 7f395c400640 1 -- 172.21.15.152:0/100577463 >> 172.21.15.152:0/100577463 conn(0x7f395406b980 msgr2=0x7f395406bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:50.111 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:50.110+0000 7f395c400640 1 -- 172.21.15.152:0/100577463 shutdown_connections 2024-04-20T11:37:50.111 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:50.110+0000 7f395c400640 1 -- 172.21.15.152:0/100577463 wait complete. 2024-04-20T11:37:50.134 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-20T11:37:50.134 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-20T11:37:50.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:50.343+0000 7fd445a00640 1 Processor -- start 2024-04-20T11:37:50.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:50.343+0000 7fd445a00640 1 -- start start 2024-04-20T11:37:50.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:50.345+0000 7fd445a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd44015eb50 0x7fd4400d0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:50.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:50.345+0000 7fd445a00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fd440163230 0x7fd4400d2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:50.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:50.345+0000 7fd445a00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4400d19c0 con 0x7fd4400d1440 2024-04-20T11:37:50.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:50.345+0000 7fd445a00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fd440162750 con 0x7fd440163230 2024-04-20T11:37:50.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:50.345+0000 7fd445a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd440162a30 con 0x7fd44015eb50 2024-04-20T11:37:50.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:50.345+0000 7fd445000640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fd4400d1440 0x7fd44015ef30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:58458/0 (socket says 172.21.15.152:58458) 2024-04-20T11:37:50.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:50.345+0000 7fd445000640 1 -- 172.21.15.152:0/585311151 learned_addr learned my addr 172.21.15.152:0/585311151 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:37:50.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:50.346+0000 7fd43fe00640 1 --2- 172.21.15.152:0/585311151 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd44015eb50 0x7fd4400d0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:50.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:50.346+0000 7fd43f400640 1 --2- 172.21.15.152:0/585311151 >> v2:172.21.15.175:3300/0 conn(0x7fd440163230 0x7fd4400d2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:50.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:50.346+0000 7fd43ea00640 1 -- 172.21.15.152:0/585311151 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1034234487 0 0) 0x7fd4400d19c0 con 0x7fd4400d1440 2024-04-20T11:37:50.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:50.346+0000 7fd43ea00640 1 -- 172.21.15.152:0/585311151 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd424003520 con 0x7fd4400d1440 2024-04-20T11:37:50.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:50.346+0000 7fd43ea00640 1 -- 172.21.15.152:0/585311151 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2905329496 0 0) 0x7fd424003520 con 0x7fd4400d1440 2024-04-20T11:37:50.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:50.346+0000 7fd43ea00640 1 -- 172.21.15.152:0/585311151 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd44015eb50 msgr2=0x7fd4400d0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:50.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:50.346+0000 7fd43ea00640 1 --2- 172.21.15.152:0/585311151 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd44015eb50 0x7fd4400d0f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:50.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:50.346+0000 7fd43ea00640 1 -- 172.21.15.152:0/585311151 >> v2:172.21.15.175:3300/0 conn(0x7fd440163230 msgr2=0x7fd4400d2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:50.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:50.346+0000 7fd43ea00640 1 --2- 172.21.15.152:0/585311151 >> v2:172.21.15.175:3300/0 conn(0x7fd440163230 0x7fd4400d2e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:50.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:50.346+0000 7fd43ea00640 1 -- 172.21.15.152:0/585311151 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd440162d10 con 0x7fd4400d1440 2024-04-20T11:37:50.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:50.346+0000 7fd43ea00640 1 -- 172.21.15.152:0/585311151 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7fd4380019f0 con 0x7fd4400d1440 2024-04-20T11:37:50.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:50.346+0000 7fd43ea00640 1 -- 172.21.15.152:0/585311151 >> v1:172.21.15.152:6789/0 conn(0x7fd4400d1440 legacy=0x7fd44015ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:50.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:50.346+0000 7fd43ea00640 1 --2- 172.21.15.152:0/585311151 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd4240039b0 0x7fd424003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:50.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:50.346+0000 7fd43ea00640 1 --2- 172.21.15.152:0/585311151 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd424004800 0x7fd424004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:50.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:50.346+0000 7fd43ea00640 1 --2- 172.21.15.152:0/585311151 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd44015eb50 0x7fd4240051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:50.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:50.346+0000 7fd43ea00640 1 -- 172.21.15.152:0/585311151 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fd440162750 con 0x7fd4240039b0 2024-04-20T11:37:50.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:50.346+0000 7fd43ea00640 1 -- 172.21.15.152:0/585311151 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd440162a30 con 0x7fd424004800 2024-04-20T11:37:50.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:50.346+0000 7fd43ea00640 1 -- 172.21.15.152:0/585311151 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd42400b9c0 con 0x7fd44015eb50 2024-04-20T11:37:50.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:50.347+0000 7fd43fe00640 1 --2- 172.21.15.152:0/585311151 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd4240039b0 0x7fd424003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:50.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:50.347+0000 7fd43f400640 1 --2- 172.21.15.152:0/585311151 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd424004800 0x7fd424004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:50.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:50.347+0000 7fd43fe00640 1 -- 172.21.15.152:0/585311151 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd44015eb50 msgr2=0x7fd4240051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:50.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:50.347+0000 7fd43fe00640 1 --2- 172.21.15.152:0/585311151 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd44015eb50 0x7fd4240051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:50.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:50.347+0000 7fd43fe00640 1 -- 172.21.15.152:0/585311151 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd424004800 msgr2=0x7fd424004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:50.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:50.347+0000 7fd43fe00640 1 --2- 172.21.15.152:0/585311151 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd424004800 0x7fd424004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:50.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:50.347+0000 7fd43fe00640 1 -- 172.21.15.152:0/585311151 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd424003520 con 0x7fd4240039b0 2024-04-20T11:37:50.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:50.348+0000 7fd43fe00640 1 --2- 172.21.15.152:0/585311151 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd4240039b0 0x7fd424003dd0 secure :-1 s=READY pgs=2307 cs=0 l=1 rev1=1 crypto rx=0x7fd430009af0 tx=0x7fd424005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac0ef708e4bfc148 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:50.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:50.349+0000 7fd43ea00640 1 -- 172.21.15.152:0/585311151 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd430039070 con 0x7fd4240039b0 2024-04-20T11:37:50.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:50.349+0000 7fd43ea00640 1 -- 172.21.15.152:0/585311151 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd43002c710 con 0x7fd4240039b0 2024-04-20T11:37:50.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:50.349+0000 7fd43ea00640 1 -- 172.21.15.152:0/585311151 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd430009690 con 0x7fd4240039b0 2024-04-20T11:37:50.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:50.349+0000 7fd445a00640 1 -- 172.21.15.152:0/585311151 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd4240039b0 msgr2=0x7fd424003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:50.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:50.349+0000 7fd445a00640 1 --2- 172.21.15.152:0/585311151 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd4240039b0 0x7fd424003dd0 secure :-1 s=READY pgs=2307 cs=0 l=1 rev1=1 crypto rx=0x7fd430009af0 tx=0x7fd424005830 comp rx=0 tx=0).stop 2024-04-20T11:37:50.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:50.349+0000 7fd445000640 1 -- 172.21.15.152:0/585311151 reap_dead start 2024-04-20T11:37:50.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:50.349+0000 7fd445a00640 1 -- 172.21.15.152:0/585311151 shutdown_connections 2024-04-20T11:37:50.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:50.349+0000 7fd445a00640 1 -- 172.21.15.152:0/585311151 >> 172.21.15.152:0/585311151 conn(0x7fd44008bfd0 msgr2=0x7fd4400d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:50.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:50.349+0000 7fd445a00640 1 -- 172.21.15.152:0/585311151 shutdown_connections 2024-04-20T11:37:50.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:50.349+0000 7fd445a00640 1 -- 172.21.15.152:0/585311151 wait complete. 2024-04-20T11:37:50.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:50.351+0000 7fd445a00640 1 Processor -- start 2024-04-20T11:37:50.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:50.351+0000 7fd445a00640 1 -- start start 2024-04-20T11:37:50.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:50.352+0000 7fd445a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd4240039b0 0x7fd44015f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:50.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:50.352+0000 7fd445a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd4400b5c40 0x7fd440158f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:50.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:50.352+0000 7fd445a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4400b6510 0x7fd4400b6970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:50.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:50.352+0000 7fd445a00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fd4400d1080 con 0x7fd4400b5c40 2024-04-20T11:37:50.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:50.352+0000 7fd445a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd4400d0f00 con 0x7fd4400b6510 2024-04-20T11:37:50.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:50.352+0000 7fd445a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd4400d1200 con 0x7fd4240039b0 2024-04-20T11:37:50.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:50.352+0000 7fd43f400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd4400b5c40 0x7fd440158f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:50.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:50.353+0000 7fd43f400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd4400b5c40 0x7fd440158f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35106/0 (socket says 172.21.15.152:35106) 2024-04-20T11:37:50.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:50.353+0000 7fd43f400640 1 -- 172.21.15.152:0/696837185 learned_addr learned my addr 172.21.15.152:0/696837185 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:50.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:50.353+0000 7fd43fe00640 1 --2- 172.21.15.152:0/696837185 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd4240039b0 0x7fd44015f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:50.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:50.353+0000 7fd445000640 1 --2- 172.21.15.152:0/696837185 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4400b6510 0x7fd4400b6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:50.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:50.353+0000 7fd43f400640 1 -- 172.21.15.152:0/696837185 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd4240039b0 msgr2=0x7fd44015f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:50.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:50.353+0000 7fd43f400640 1 --2- 172.21.15.152:0/696837185 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd4240039b0 0x7fd44015f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:50.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:50.353+0000 7fd43f400640 1 -- 172.21.15.152:0/696837185 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4400b6510 msgr2=0x7fd4400b6970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:50.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:50.353+0000 7fd43f400640 1 --2- 172.21.15.152:0/696837185 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4400b6510 0x7fd4400b6970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:50.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:50.353+0000 7fd43f400640 1 -- 172.21.15.152:0/696837185 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4400b8a30 con 0x7fd4400b5c40 2024-04-20T11:37:50.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:50.353+0000 7fd445000640 1 --2- 172.21.15.152:0/696837185 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4400b6510 0x7fd4400b6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:37:50.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:50.353+0000 7fd43f400640 1 --2- 172.21.15.152:0/696837185 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd4400b5c40 0x7fd440158f40 secure :-1 s=READY pgs=2308 cs=0 l=1 rev1=1 crypto rx=0x7fd42c010930 tx=0x7fd42c010df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:50.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:50.354+0000 7fd437e00640 1 -- 172.21.15.152:0/696837185 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd42c00f040 con 0x7fd4400b5c40 2024-04-20T11:37:50.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:50.354+0000 7fd437e00640 1 -- 172.21.15.152:0/696837185 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd42c00ec20 con 0x7fd4400b5c40 2024-04-20T11:37:50.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:50.354+0000 7fd437e00640 1 -- 172.21.15.152:0/696837185 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd42c01c400 con 0x7fd4400b5c40 2024-04-20T11:37:50.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:50.354+0000 7fd445a00640 1 -- 172.21.15.152:0/696837185 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4400b8d10 con 0x7fd4400b5c40 2024-04-20T11:37:50.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:50.354+0000 7fd445a00640 1 -- 172.21.15.152:0/696837185 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4400b7260 con 0x7fd4400b5c40 2024-04-20T11:37:50.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:50.355+0000 7fd445a00640 1 -- 172.21.15.152:0/696837185 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd440075e60 con 0x7fd4400b5c40 2024-04-20T11:37:50.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:50.356+0000 7fd437e00640 1 -- 172.21.15.152:0/696837185 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 26) ==== 91907+0+0 (secure 0 0 0) 0x7fd42c01c5a0 con 0x7fd4400b5c40 2024-04-20T11:37:50.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:50.357+0000 7fd437e00640 1 --2- 172.21.15.152:0/696837185 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd42006d280 0x7fd42006f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:50.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:50.357+0000 7fd437e00640 1 -- 172.21.15.152:0/696837185 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 5709+0+0 (secure 0 0 0) 0x7fd42c08a970 con 0x7fd4400b5c40 2024-04-20T11:37:50.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:50.358+0000 7fd43fe00640 1 --2- 172.21.15.152:0/696837185 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd42006d280 0x7fd42006f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:50.359 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:50.359+0000 7fd43fe00640 1 --2- 172.21.15.152:0/696837185 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd42006d280 0x7fd42006f740 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fd430036000 tx=0x7fd430030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:50.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:50.362+0000 7fd437e00640 1 -- 172.21.15.152:0/696837185 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fd42c018030 con 0x7fd4400b5c40 2024-04-20T11:37:50.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:50.602+0000 7fd445a00640 1 -- 172.21.15.152:0/696837185 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fd44008c680 con 0x7fd4400b5c40 2024-04-20T11:37:51.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.118+0000 7fd437e00640 1 -- 172.21.15.152:0/696837185 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v115) ==== 122+0+0 (secure 0 0 0) 0x7fd42c05a410 con 0x7fd4400b5c40 2024-04-20T11:37:51.124 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.123+0000 7fd445a00640 1 -- 172.21.15.152:0/696837185 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd42006d280 msgr2=0x7fd42006f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:51.124 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.123+0000 7fd445a00640 1 --2- 172.21.15.152:0/696837185 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd42006d280 0x7fd42006f740 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fd430036000 tx=0x7fd430030000 comp rx=0 tx=0).stop 2024-04-20T11:37:51.124 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.123+0000 7fd445a00640 1 -- 172.21.15.152:0/696837185 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd4400b5c40 msgr2=0x7fd440158f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:51.124 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.123+0000 7fd445a00640 1 --2- 172.21.15.152:0/696837185 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd4400b5c40 0x7fd440158f40 secure :-1 s=READY pgs=2308 cs=0 l=1 rev1=1 crypto rx=0x7fd42c010930 tx=0x7fd42c010df0 comp rx=0 tx=0).stop 2024-04-20T11:37:51.124 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.123+0000 7fd445a00640 1 -- 172.21.15.152:0/696837185 shutdown_connections 2024-04-20T11:37:51.124 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.123+0000 7fd445a00640 1 --2- 172.21.15.152:0/696837185 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd42006d280 0x7fd42006f740 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:51.124 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.123+0000 7fd445a00640 1 --2- 172.21.15.152:0/696837185 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4400b6510 0x7fd4400b6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:51.124 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.123+0000 7fd445a00640 1 --2- 172.21.15.152:0/696837185 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd4400b5c40 0x7fd440158f40 unknown :-1 s=CLOSED pgs=2308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:51.124 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.123+0000 7fd445a00640 1 --2- 172.21.15.152:0/696837185 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd4240039b0 0x7fd44015f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:51.125 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.124+0000 7fd445a00640 1 -- 172.21.15.152:0/696837185 >> 172.21.15.152:0/696837185 conn(0x7fd44008bfd0 msgr2=0x7fd4400cfd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:51.125 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.124+0000 7fd445a00640 1 -- 172.21.15.152:0/696837185 shutdown_connections 2024-04-20T11:37:51.125 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.124+0000 7fd445a00640 1 -- 172.21.15.152:0/696837185 wait complete. 2024-04-20T11:37:51.145 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-20T11:37:51.145 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:37:51.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.350+0000 7f58ae800640 1 Processor -- start 2024-04-20T11:37:51.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.350+0000 7f58ae800640 1 -- start start 2024-04-20T11:37:51.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.352+0000 7f58ae800640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f58a815fcc0 0x7f58a80d4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:51.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.352+0000 7f58ae800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f58a81603d0 0x7f58a80b5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:51.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.352+0000 7f58ae800640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58a80d3350 con 0x7f58a80b5480 2024-04-20T11:37:51.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.352+0000 7f58ae800640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f58a80d3630 con 0x7f58a815fcc0 2024-04-20T11:37:51.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.352+0000 7f58ae800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f58a80d3910 con 0x7f58a81603d0 2024-04-20T11:37:51.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.352+0000 7f58ade00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f58a80b5480 0x7f58a81624d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:58460/0 (socket says 172.21.15.152:58460) 2024-04-20T11:37:51.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.352+0000 7f58ade00640 1 -- 172.21.15.152:0/1717876584 learned_addr learned my addr 172.21.15.152:0/1717876584 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:37:51.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.352+0000 7f58ad400640 1 --2- 172.21.15.152:0/1717876584 >> v2:172.21.15.175:3300/0 conn(0x7f58a815fcc0 0x7f58a80d4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:51.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.352+0000 7f58aca00640 1 --2- 172.21.15.152:0/1717876584 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f58a81603d0 0x7f58a80b5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:51.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.353+0000 7f58a7e00640 1 -- 172.21.15.152:0/1717876584 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3974323962 0 0) 0x7f58a80d3350 con 0x7f58a80b5480 2024-04-20T11:37:51.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.353+0000 7f58a7e00640 1 -- 172.21.15.152:0/1717876584 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f588c003520 con 0x7f58a80b5480 2024-04-20T11:37:51.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.353+0000 7f58a7e00640 1 -- 172.21.15.152:0/1717876584 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 305676079 0 0) 0x7f588c003520 con 0x7f58a80b5480 2024-04-20T11:37:51.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.353+0000 7f58a7e00640 1 -- 172.21.15.152:0/1717876584 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f58a81603d0 msgr2=0x7f58a80b5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:51.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.353+0000 7f58a7e00640 1 --2- 172.21.15.152:0/1717876584 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f58a81603d0 0x7f58a80b5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:51.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.353+0000 7f58a7e00640 1 -- 172.21.15.152:0/1717876584 >> v2:172.21.15.175:3300/0 conn(0x7f58a815fcc0 msgr2=0x7f58a80d4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:51.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.353+0000 7f58a7e00640 1 --2- 172.21.15.152:0/1717876584 >> v2:172.21.15.175:3300/0 conn(0x7f58a815fcc0 0x7f58a80d4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:51.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.353+0000 7f58a7e00640 1 -- 172.21.15.152:0/1717876584 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58a80ced20 con 0x7f58a80b5480 2024-04-20T11:37:51.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.353+0000 7f58a7e00640 1 -- 172.21.15.152:0/1717876584 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f58a0001670 con 0x7f58a80b5480 2024-04-20T11:37:51.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.353+0000 7f58a7e00640 1 -- 172.21.15.152:0/1717876584 >> v1:172.21.15.152:6789/0 conn(0x7f58a80b5480 legacy=0x7f58a81624d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:51.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.353+0000 7f58a7e00640 1 --2- 172.21.15.152:0/1717876584 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f588c0039b0 0x7f588c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:51.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.353+0000 7f58a7e00640 1 --2- 172.21.15.152:0/1717876584 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f588c004800 0x7f588c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:51.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.353+0000 7f58a7e00640 1 --2- 172.21.15.152:0/1717876584 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f58a81603d0 0x7f588c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:51.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.353+0000 7f58a7e00640 1 -- 172.21.15.152:0/1717876584 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f58a80d3630 con 0x7f588c0039b0 2024-04-20T11:37:51.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.353+0000 7f58a7e00640 1 -- 172.21.15.152:0/1717876584 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f58a80d3910 con 0x7f588c004800 2024-04-20T11:37:51.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.353+0000 7f58a7e00640 1 -- 172.21.15.152:0/1717876584 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f588c00b9c0 con 0x7f58a81603d0 2024-04-20T11:37:51.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.354+0000 7f58ad400640 1 --2- 172.21.15.152:0/1717876584 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f588c0039b0 0x7f588c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:51.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.354+0000 7f58aca00640 1 --2- 172.21.15.152:0/1717876584 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f588c004800 0x7f588c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:51.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.354+0000 7f58ad400640 1 -- 172.21.15.152:0/1717876584 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f58a81603d0 msgr2=0x7f588c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:51.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.354+0000 7f58ade00640 1 --2- 172.21.15.152:0/1717876584 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f58a81603d0 0x7f588c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:51.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.354+0000 7f58ad400640 1 --2- 172.21.15.152:0/1717876584 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f58a81603d0 0x7f588c0051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:51.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.354+0000 7f58ad400640 1 -- 172.21.15.152:0/1717876584 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f588c004800 msgr2=0x7f588c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:51.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.354+0000 7f58ad400640 1 --2- 172.21.15.152:0/1717876584 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f588c004800 0x7f588c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:51.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.354+0000 7f58ad400640 1 -- 172.21.15.152:0/1717876584 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f588c003520 con 0x7f588c0039b0 2024-04-20T11:37:51.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.354+0000 7f58aca00640 1 --2- 172.21.15.152:0/1717876584 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f588c004800 0x7f588c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:37:51.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.355+0000 7f58ad400640 1 --2- 172.21.15.152:0/1717876584 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f588c0039b0 0x7f588c003dd0 secure :-1 s=READY pgs=2310 cs=0 l=1 rev1=1 crypto rx=0x7f5898009ac0 tx=0x7f588c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83bea53e23acd09a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:51.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.355+0000 7f58a7e00640 1 -- 172.21.15.152:0/1717876584 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5898039070 con 0x7f588c0039b0 2024-04-20T11:37:51.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.355+0000 7f58a7e00640 1 -- 172.21.15.152:0/1717876584 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f589802c6d0 con 0x7f588c0039b0 2024-04-20T11:37:51.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.355+0000 7f58a7e00640 1 -- 172.21.15.152:0/1717876584 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5898009690 con 0x7f588c0039b0 2024-04-20T11:37:51.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.356+0000 7f58ae800640 1 -- 172.21.15.152:0/1717876584 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f588c0039b0 msgr2=0x7f588c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:51.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.356+0000 7f58ae800640 1 --2- 172.21.15.152:0/1717876584 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f588c0039b0 0x7f588c003dd0 secure :-1 s=READY pgs=2310 cs=0 l=1 rev1=1 crypto rx=0x7f5898009ac0 tx=0x7f588c005830 comp rx=0 tx=0).stop 2024-04-20T11:37:51.359 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.356+0000 7f58ade00640 1 -- 172.21.15.152:0/1717876584 reap_dead start 2024-04-20T11:37:51.359 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.356+0000 7f58ae800640 1 -- 172.21.15.152:0/1717876584 shutdown_connections 2024-04-20T11:37:51.359 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.356+0000 7f58ae800640 1 -- 172.21.15.152:0/1717876584 >> 172.21.15.152:0/1717876584 conn(0x7f58a808bfd0 msgr2=0x7f58a80d4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:51.359 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.356+0000 7f58ae800640 1 -- 172.21.15.152:0/1717876584 shutdown_connections 2024-04-20T11:37:51.359 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.356+0000 7f58ae800640 1 -- 172.21.15.152:0/1717876584 wait complete. 2024-04-20T11:37:51.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.358+0000 7f58ae800640 1 Processor -- start 2024-04-20T11:37:51.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.358+0000 7f58ae800640 1 -- start start 2024-04-20T11:37:51.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.359+0000 7f58ae800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f588c0039b0 0x7f58a81619f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:51.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.359+0000 7f58ae800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f58a80cc0a0 0x7f58a815bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:51.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.359+0000 7f58ae800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f58a815c4a0 0x7f58a81ea1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:51.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.359+0000 7f58ae800640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f58a8162d60 con 0x7f58a80cc0a0 2024-04-20T11:37:51.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.359+0000 7f58ae800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f58a8162be0 con 0x7f58a815c4a0 2024-04-20T11:37:51.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.359+0000 7f58ae800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f58a8162ee0 con 0x7f588c0039b0 2024-04-20T11:37:51.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.359+0000 7f58aca00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f58a80cc0a0 0x7f58a815bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:51.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.359+0000 7f58aca00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f58a80cc0a0 0x7f58a815bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35120/0 (socket says 172.21.15.152:35120) 2024-04-20T11:37:51.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.359+0000 7f58aca00640 1 -- 172.21.15.152:0/929784235 learned_addr learned my addr 172.21.15.152:0/929784235 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:51.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.360+0000 7f58ad400640 1 --2- 172.21.15.152:0/929784235 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f588c0039b0 0x7f58a81619f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:51.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.360+0000 7f58ade00640 1 --2- 172.21.15.152:0/929784235 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f58a815c4a0 0x7f58a81ea1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:51.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.360+0000 7f58aca00640 1 -- 172.21.15.152:0/929784235 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f588c0039b0 msgr2=0x7f58a81619f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:51.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.360+0000 7f58aca00640 1 --2- 172.21.15.152:0/929784235 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f588c0039b0 0x7f58a81619f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:51.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.360+0000 7f58aca00640 1 -- 172.21.15.152:0/929784235 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f58a815c4a0 msgr2=0x7f58a81ea1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:51.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.360+0000 7f58aca00640 1 --2- 172.21.15.152:0/929784235 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f58a815c4a0 0x7f58a81ea1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:51.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.360+0000 7f58aca00640 1 -- 172.21.15.152:0/929784235 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58a80b82d0 con 0x7f58a80cc0a0 2024-04-20T11:37:51.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.360+0000 7f58ade00640 1 --2- 172.21.15.152:0/929784235 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f58a815c4a0 0x7f58a81ea1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:37:51.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.360+0000 7f58aca00640 1 --2- 172.21.15.152:0/929784235 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f58a80cc0a0 0x7f58a815bf60 secure :-1 s=READY pgs=2311 cs=0 l=1 rev1=1 crypto rx=0x7f589400f380 tx=0x7f589400f840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:51.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.360+0000 7f58a5600640 1 -- 172.21.15.152:0/929784235 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5894003b90 con 0x7f58a80cc0a0 2024-04-20T11:37:51.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.360+0000 7f58a5600640 1 -- 172.21.15.152:0/929784235 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5894003d30 con 0x7f58a80cc0a0 2024-04-20T11:37:51.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.360+0000 7f58a5600640 1 -- 172.21.15.152:0/929784235 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5894004260 con 0x7f58a80cc0a0 2024-04-20T11:37:51.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.360+0000 7f58ae800640 1 -- 172.21.15.152:0/929784235 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58a80b85b0 con 0x7f58a80cc0a0 2024-04-20T11:37:51.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.361+0000 7f58ae800640 1 -- 172.21.15.152:0/929784235 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58a80b8b70 con 0x7f58a80cc0a0 2024-04-20T11:37:51.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.362+0000 7f58ae800640 1 -- 172.21.15.152:0/929784235 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58a8075e60 con 0x7f58a80cc0a0 2024-04-20T11:37:51.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.363+0000 7f58a5600640 1 -- 172.21.15.152:0/929784235 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 26) ==== 91907+0+0 (secure 0 0 0) 0x7f5894004400 con 0x7f58a80cc0a0 2024-04-20T11:37:51.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.364+0000 7f58a5600640 1 --2- 172.21.15.152:0/929784235 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f588806d280 0x7f588806f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:51.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.364+0000 7f58a5600640 1 -- 172.21.15.152:0/929784235 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 5709+0+0 (secure 0 0 0) 0x7f589408a990 con 0x7f58a80cc0a0 2024-04-20T11:37:51.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.364+0000 7f58ad400640 1 --2- 172.21.15.152:0/929784235 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f588806d280 0x7f588806f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:51.366 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.366+0000 7f58ad400640 1 --2- 172.21.15.152:0/929784235 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f588806d280 0x7f588806f740 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f5898036000 tx=0x7f5898030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:51.369 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.368+0000 7f58a5600640 1 -- 172.21.15.152:0/929784235 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f5894012070 con 0x7f58a80cc0a0 2024-04-20T11:37:51.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.604+0000 7f58ae800640 1 -- 172.21.15.152:0/929784235 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f58a80ba4d0 con 0x7f58a80cc0a0 2024-04-20T11:37:51.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.605+0000 7f58a5600640 1 -- 172.21.15.152:0/929784235 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 115 v115) ==== 95+0+5215 (secure 0 0 0) 0x7f589405a9c0 con 0x7f58a80cc0a0 2024-04-20T11:37:51.606 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:37:51.606 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":115,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6605,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6834/4020670267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6833","nonce":4020670267},{"type":"v1","addr":"172.21.15.152:6834","nonce":4020670267}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":109},{"gid":6614,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/352716681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":352716681},{"type":"v1","addr":"172.21.15.175:6833","nonce":352716681}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":110},{"gid":6617,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/2512108447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":2512108447},{"type":"v1","addr":"172.21.15.175:6835","nonce":2512108447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":110}],"filesystems":[{"mdsmap":{"epoch":115,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T11:37:47.506735+0000","modified":"2024-04-20T11:37:50.605372+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7339},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7339":{"gid":7339,"name":"c","rank":0,"incarnation":111,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6835/951802292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":951802292},{"type":"v1","addr":"172.21.15.152:6835","nonce":951802292}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7339,"qdb_cluster":[7339]},"id":13}]} 2024-04-20T11:37:51.606 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 115 2024-04-20T11:37:51.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.608+0000 7f58ae800640 1 -- 172.21.15.152:0/929784235 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f588806d280 msgr2=0x7f588806f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:51.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.608+0000 7f58ae800640 1 --2- 172.21.15.152:0/929784235 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f588806d280 0x7f588806f740 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f5898036000 tx=0x7f5898030000 comp rx=0 tx=0).stop 2024-04-20T11:37:51.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.608+0000 7f58ae800640 1 -- 172.21.15.152:0/929784235 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f58a80cc0a0 msgr2=0x7f58a815bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:51.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.608+0000 7f58ae800640 1 --2- 172.21.15.152:0/929784235 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f58a80cc0a0 0x7f58a815bf60 secure :-1 s=READY pgs=2311 cs=0 l=1 rev1=1 crypto rx=0x7f589400f380 tx=0x7f589400f840 comp rx=0 tx=0).stop 2024-04-20T11:37:51.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.608+0000 7f58ae800640 1 -- 172.21.15.152:0/929784235 shutdown_connections 2024-04-20T11:37:51.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.608+0000 7f58ae800640 1 --2- 172.21.15.152:0/929784235 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f588806d280 0x7f588806f740 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:51.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.608+0000 7f58ae800640 1 --2- 172.21.15.152:0/929784235 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f58a815c4a0 0x7f58a81ea1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:51.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.608+0000 7f58ae800640 1 --2- 172.21.15.152:0/929784235 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f58a80cc0a0 0x7f58a815bf60 unknown :-1 s=CLOSED pgs=2311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:51.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.608+0000 7f58ae800640 1 --2- 172.21.15.152:0/929784235 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f588c0039b0 0x7f58a81619f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:51.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.608+0000 7f58ae800640 1 -- 172.21.15.152:0/929784235 >> 172.21.15.152:0/929784235 conn(0x7f58a808bfd0 msgr2=0x7f58a806dd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:51.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.608+0000 7f58ae800640 1 -- 172.21.15.152:0/929784235 shutdown_connections 2024-04-20T11:37:51.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.608+0000 7f58ae800640 1 -- 172.21.15.152:0/929784235 wait complete. 2024-04-20T11:37:51.626 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:37:51.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.814+0000 7fad9ce00640 1 Processor -- start 2024-04-20T11:37:51.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.814+0000 7fad9ce00640 1 -- start start 2024-04-20T11:37:51.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.816+0000 7fad9ce00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fad9815eb80 0x7fad980ccbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:51.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.816+0000 7fad9ce00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fad980b17d0 0x7fad9806cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:51.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.816+0000 7fad9ce00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad980d1af0 con 0x7fad9806d360 2024-04-20T11:37:51.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.816+0000 7fad9ce00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fad9809ae50 con 0x7fad980b17d0 2024-04-20T11:37:51.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.816+0000 7fad9ce00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fad980d1250 con 0x7fad9815eb80 2024-04-20T11:37:51.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.817+0000 7fad8d400640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fad980b17d0 0x7fad9806cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:51.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.817+0000 7fad95e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fad9815eb80 0x7fad980ccbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:51.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.817+0000 7fad8d400640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fad980b17d0 0x7fad9806cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.152:43688/0 (socket says 172.21.15.152:43688) 2024-04-20T11:37:51.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.817+0000 7fad8d400640 1 -- 172.21.15.152:0/3800786397 learned_addr learned my addr 172.21.15.152:0/3800786397 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:51.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.817+0000 7fad8d400640 1 -- 172.21.15.152:0/3800786397 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fad9815eb80 msgr2=0x7fad980ccbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:51.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.817+0000 7fad8d400640 1 --2- 172.21.15.152:0/3800786397 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fad9815eb80 0x7fad980ccbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:51.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.817+0000 7fad8d400640 1 -- 172.21.15.152:0/3800786397 >> v1:172.21.15.175:6789/0 conn(0x7fad9806d360 legacy=0x7fad9809b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:51.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.817+0000 7fad8d400640 1 -- 172.21.15.152:0/3800786397 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad980d1530 con 0x7fad980b17d0 2024-04-20T11:37:51.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.819+0000 7fad8d400640 1 --2- 172.21.15.152:0/3800786397 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad980b17d0 0x7fad9806cca0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fad80009870 tx=0x7fad8002b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5664b1a2803c0f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:51.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.819+0000 7fad95400640 1 -- 172.21.15.152:0/3800786397 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad80034070 con 0x7fad980b17d0 2024-04-20T11:37:51.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.819+0000 7fad95400640 1 -- 172.21.15.152:0/3800786397 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad80030840 con 0x7fad980b17d0 2024-04-20T11:37:51.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.820+0000 7fad95400640 1 -- 172.21.15.152:0/3800786397 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad80030c10 con 0x7fad980b17d0 2024-04-20T11:37:51.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.820+0000 7fad9ce00640 1 -- 172.21.15.152:0/3800786397 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad980b17d0 msgr2=0x7fad9806cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:51.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.820+0000 7fad9ce00640 1 --2- 172.21.15.152:0/3800786397 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad980b17d0 0x7fad9806cca0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fad80009870 tx=0x7fad8002b980 comp rx=0 tx=0).stop 2024-04-20T11:37:51.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.820+0000 7fad9ce00640 1 -- 172.21.15.152:0/3800786397 shutdown_connections 2024-04-20T11:37:51.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.820+0000 7fad9ce00640 1 --2- 172.21.15.152:0/3800786397 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad980b17d0 0x7fad9806cca0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:51.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.820+0000 7fad9ce00640 1 --2- 172.21.15.152:0/3800786397 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fad9815eb80 0x7fad980ccbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:51.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.820+0000 7fad9ce00640 1 -- 172.21.15.152:0/3800786397 >> 172.21.15.152:0/3800786397 conn(0x7fad980b4080 msgr2=0x7fad980b4490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:51.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.820+0000 7fad9ce00640 1 -- 172.21.15.152:0/3800786397 shutdown_connections 2024-04-20T11:37:51.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.820+0000 7fad9ce00640 1 -- 172.21.15.152:0/3800786397 wait complete. 2024-04-20T11:37:51.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.822+0000 7fad9ce00640 1 Processor -- start 2024-04-20T11:37:51.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.822+0000 7fad9ce00640 1 -- start start 2024-04-20T11:37:51.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.823+0000 7fad9ce00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fad9806d360 0x7fad980b59f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:51.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.823+0000 7fad9ce00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fad980b17d0 0x7fad9814a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:51.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.823+0000 7fad9ce00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad9815eb80 0x7fad980c7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:51.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.823+0000 7fad9ce00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fad9815ef60 con 0x7fad980b17d0 2024-04-20T11:37:51.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.823+0000 7fad9ce00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fad9809aae0 con 0x7fad9815eb80 2024-04-20T11:37:51.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.823+0000 7fad9ce00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fad9806d7f0 con 0x7fad9806d360 2024-04-20T11:37:51.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.823+0000 7fad8d400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fad980b17d0 0x7fad9814a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:51.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.823+0000 7fad8d400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fad980b17d0 0x7fad9814a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35128/0 (socket says 172.21.15.152:35128) 2024-04-20T11:37:51.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.823+0000 7fad8d400640 1 -- 172.21.15.152:0/3791607102 learned_addr learned my addr 172.21.15.152:0/3791607102 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:51.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.824+0000 7fad95e00640 1 --2- 172.21.15.152:0/3791607102 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fad9806d360 0x7fad980b59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:51.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.824+0000 7fad96800640 1 --2- 172.21.15.152:0/3791607102 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad9815eb80 0x7fad980c7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:51.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.824+0000 7fad8d400640 1 -- 172.21.15.152:0/3791607102 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fad9806d360 msgr2=0x7fad980b59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:51.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.824+0000 7fad8d400640 1 --2- 172.21.15.152:0/3791607102 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fad9806d360 0x7fad980b59f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:51.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.824+0000 7fad8d400640 1 -- 172.21.15.152:0/3791607102 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad9815eb80 msgr2=0x7fad980c7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:51.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.824+0000 7fad8d400640 1 --2- 172.21.15.152:0/3791607102 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad9815eb80 0x7fad980c7cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:51.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.824+0000 7fad8d400640 1 -- 172.21.15.152:0/3791607102 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad981f4d30 con 0x7fad980b17d0 2024-04-20T11:37:51.826 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.824+0000 7fad8d400640 1 --2- 172.21.15.152:0/3791607102 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fad980b17d0 0x7fad9814a560 secure :-1 s=READY pgs=2312 cs=0 l=1 rev1=1 crypto rx=0x7fad80000c00 tx=0x7fad80004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:51.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.824+0000 7fad8ea00640 1 -- 172.21.15.152:0/3791607102 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad80048070 con 0x7fad980b17d0 2024-04-20T11:37:51.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.824+0000 7fad8ea00640 1 -- 172.21.15.152:0/3791607102 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad80030640 con 0x7fad980b17d0 2024-04-20T11:37:51.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.825+0000 7fad8ea00640 1 -- 172.21.15.152:0/3791607102 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad80034040 con 0x7fad980b17d0 2024-04-20T11:37:51.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.825+0000 7fad9ce00640 1 -- 172.21.15.152:0/3791607102 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad9814aaa0 con 0x7fad980b17d0 2024-04-20T11:37:51.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.825+0000 7fad9ce00640 1 -- 172.21.15.152:0/3791607102 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad9814d6e0 con 0x7fad980b17d0 2024-04-20T11:37:51.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.826+0000 7fad9ce00640 1 -- 172.21.15.152:0/3791607102 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad980689f0 con 0x7fad980b17d0 2024-04-20T11:37:51.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.827+0000 7fad8ea00640 1 -- 172.21.15.152:0/3791607102 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 26) ==== 91907+0+0 (secure 0 0 0) 0x7fad8004b070 con 0x7fad980b17d0 2024-04-20T11:37:51.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.828+0000 7fad8ea00640 1 --2- 172.21.15.152:0/3791607102 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fad7406d280 0x7fad7406f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:51.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.828+0000 7fad8ea00640 1 -- 172.21.15.152:0/3791607102 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 5709+0+0 (secure 0 0 0) 0x7fad800ac210 con 0x7fad980b17d0 2024-04-20T11:37:51.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.829+0000 7fad95e00640 1 --2- 172.21.15.152:0/3791607102 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fad7406d280 0x7fad7406f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:51.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.830+0000 7fad95e00640 1 --2- 172.21.15.152:0/3791607102 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fad7406d280 0x7fad7406f740 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fad88009a30 tx=0x7fad88008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:51.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:51.832+0000 7fad8ea00640 1 -- 172.21.15.152:0/3791607102 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fad8007c240 con 0x7fad980b17d0 2024-04-20T11:37:52.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.035+0000 7fad9ce00640 1 -- 172.21.15.152:0/3791607102 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fad98074f20 con 0x7fad980b17d0 2024-04-20T11:37:52.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.037+0000 7fad8ea00640 1 -- 172.21.15.152:0/3791607102 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v167) ==== 74+0+16743 (secure 0 0 0) 0x7fad8007bb40 con 0x7fad980b17d0 2024-04-20T11:37:52.038 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:37:52.038 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":167,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:37:49.091768+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:37:45.247938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"166","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2024-04-20T11:37:46.503171+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"167","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:37:52.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.040+0000 7fad9ce00640 1 -- 172.21.15.152:0/3791607102 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fad7406d280 msgr2=0x7fad7406f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:52.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.040+0000 7fad9ce00640 1 --2- 172.21.15.152:0/3791607102 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fad7406d280 0x7fad7406f740 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fad88009a30 tx=0x7fad88008040 comp rx=0 tx=0).stop 2024-04-20T11:37:52.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.040+0000 7fad9ce00640 1 -- 172.21.15.152:0/3791607102 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fad980b17d0 msgr2=0x7fad9814a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:52.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.040+0000 7fad9ce00640 1 --2- 172.21.15.152:0/3791607102 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fad980b17d0 0x7fad9814a560 secure :-1 s=READY pgs=2312 cs=0 l=1 rev1=1 crypto rx=0x7fad80000c00 tx=0x7fad80004730 comp rx=0 tx=0).stop 2024-04-20T11:37:52.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.040+0000 7fad9ce00640 1 -- 172.21.15.152:0/3791607102 shutdown_connections 2024-04-20T11:37:52.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.040+0000 7fad9ce00640 1 --2- 172.21.15.152:0/3791607102 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fad7406d280 0x7fad7406f740 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:52.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.040+0000 7fad9ce00640 1 --2- 172.21.15.152:0/3791607102 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad9815eb80 0x7fad980c7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:52.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.040+0000 7fad9ce00640 1 --2- 172.21.15.152:0/3791607102 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fad980b17d0 0x7fad9814a560 unknown :-1 s=CLOSED pgs=2312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:52.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.040+0000 7fad9ce00640 1 --2- 172.21.15.152:0/3791607102 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fad9806d360 0x7fad980b59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:52.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.040+0000 7fad9ce00640 1 -- 172.21.15.152:0/3791607102 >> 172.21.15.152:0/3791607102 conn(0x7fad980b4080 msgr2=0x7fad980b2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:52.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.041+0000 7fad9ce00640 1 -- 172.21.15.152:0/3791607102 shutdown_connections 2024-04-20T11:37:52.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.041+0000 7fad9ce00640 1 -- 172.21.15.152:0/3791607102 wait complete. 2024-04-20T11:37:52.059 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-20T11:37:52.059 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T11:37:52.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.258+0000 7f4aed200640 1 Processor -- start 2024-04-20T11:37:52.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.258+0000 7f4aed200640 1 -- start start 2024-04-20T11:37:52.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.260+0000 7f4aed200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f4ae80b65c0 0x7f4ae8169800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:52.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.260+0000 7f4aed200640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ae80d3350 con 0x7f4ae815fcc0 2024-04-20T11:37:52.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.260+0000 7f4aed200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ae80d3630 con 0x7f4ae81603d0 2024-04-20T11:37:52.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.260+0000 7f4aed200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f4ae80d3910 con 0x7f4ae80b65c0 2024-04-20T11:37:52.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.260+0000 7f4ae7e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f4ae80b65c0 0x7f4ae8169800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:52.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.260+0000 7f4ae7400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f4ae815fcc0 0x7f4ae80d4e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:58470/0 (socket says 172.21.15.152:58470) 2024-04-20T11:37:52.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.260+0000 7f4ae7400640 1 -- 172.21.15.152:0/832401130 learned_addr learned my addr 172.21.15.152:0/832401130 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:37:52.262 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.261+0000 7f4ae7e00640 1 -- 172.21.15.152:0/832401130 >> v1:172.21.15.175:6789/0 conn(0x7f4ae81603d0 legacy=0x7f4ae80b5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:52.262 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.261+0000 7f4ae7e00640 1 -- 172.21.15.152:0/832401130 >> v1:172.21.15.152:6789/0 conn(0x7f4ae815fcc0 legacy=0x7f4ae80d4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:52.262 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.261+0000 7f4ae7e00640 1 -- 172.21.15.152:0/832401130 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ae80ced20 con 0x7f4ae80b65c0 2024-04-20T11:37:52.262 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.261+0000 7f4ae6000640 1 -- 172.21.15.152:0/832401130 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3352054688 0 0) 0x7f4ae80d3350 con 0x7f4ae815fcc0 2024-04-20T11:37:52.263 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.262+0000 7f4ae7e00640 1 --2- 172.21.15.152:0/832401130 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ae80b65c0 0x7f4ae8169800 secure :-1 s=READY pgs=2314 cs=0 l=1 rev1=1 crypto rx=0x7f4ae00097a0 tx=0x7f4ae002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ea5fa046aec8e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:52.263 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.262+0000 7f4ae6000640 1 -- 172.21.15.152:0/832401130 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae0035020 con 0x7f4ae80b65c0 2024-04-20T11:37:52.263 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.262+0000 7f4ae6000640 1 -- 172.21.15.152:0/832401130 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ae0031830 con 0x7f4ae80b65c0 2024-04-20T11:37:52.263 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.262+0000 7f4ae6000640 1 -- 172.21.15.152:0/832401130 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae0031c00 con 0x7f4ae80b65c0 2024-04-20T11:37:52.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.263+0000 7f4aed200640 1 -- 172.21.15.152:0/832401130 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ae80b65c0 msgr2=0x7f4ae8169800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:52.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.263+0000 7f4aed200640 1 --2- 172.21.15.152:0/832401130 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ae80b65c0 0x7f4ae8169800 secure :-1 s=READY pgs=2314 cs=0 l=1 rev1=1 crypto rx=0x7f4ae00097a0 tx=0x7f4ae002b8d0 comp rx=0 tx=0).stop 2024-04-20T11:37:52.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.263+0000 7f4aed200640 1 -- 172.21.15.152:0/832401130 shutdown_connections 2024-04-20T11:37:52.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.263+0000 7f4aed200640 1 --2- 172.21.15.152:0/832401130 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ae80b65c0 0x7f4ae8169800 unknown :-1 s=CLOSED pgs=2314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:52.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.263+0000 7f4aed200640 1 -- 172.21.15.152:0/832401130 >> 172.21.15.152:0/832401130 conn(0x7f4ae808bfd0 msgr2=0x7f4ae80d4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:52.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.263+0000 7f4aed200640 1 -- 172.21.15.152:0/832401130 shutdown_connections 2024-04-20T11:37:52.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.263+0000 7f4aed200640 1 -- 172.21.15.152:0/832401130 wait complete. 2024-04-20T11:37:52.266 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.265+0000 7f4aed200640 1 Processor -- start 2024-04-20T11:37:52.266 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.265+0000 7f4aed200640 1 -- start start 2024-04-20T11:37:52.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.266+0000 7f4aed200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4ae80b65c0 0x7f4ae80cb620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:52.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.266+0000 7f4aed200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ae815fcc0 0x7f4ae8153a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:52.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.267+0000 7f4aed200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4ae81603d0 0x7f4ae81579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:52.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.267+0000 7f4aed200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae8169d40 con 0x7f4ae815fcc0 2024-04-20T11:37:52.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.267+0000 7f4aed200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae80b5710 con 0x7f4ae80b65c0 2024-04-20T11:37:52.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.267+0000 7f4aed200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4ae8169ec0 con 0x7f4ae81603d0 2024-04-20T11:37:52.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.267+0000 7f4ae6a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ae815fcc0 0x7f4ae8153a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:52.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.267+0000 7f4ae6a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ae815fcc0 0x7f4ae8153a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35142/0 (socket says 172.21.15.152:35142) 2024-04-20T11:37:52.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.267+0000 7f4ae6a00640 1 -- 172.21.15.152:0/3115322807 learned_addr learned my addr 172.21.15.152:0/3115322807 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:52.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.267+0000 7f4ae6a00640 1 -- 172.21.15.152:0/3115322807 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4ae81603d0 msgr2=0x7f4ae81579a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:52.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.267+0000 7f4ae7400640 1 --2- 172.21.15.152:0/3115322807 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4ae80b65c0 0x7f4ae80cb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:52.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.267+0000 7f4ae6a00640 1 --2- 172.21.15.152:0/3115322807 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4ae81603d0 0x7f4ae81579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:52.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.267+0000 7f4ae6a00640 1 -- 172.21.15.152:0/3115322807 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4ae80b65c0 msgr2=0x7f4ae80cb620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:52.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.267+0000 7f4ae6a00640 1 --2- 172.21.15.152:0/3115322807 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4ae80b65c0 0x7f4ae80cb620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:52.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.267+0000 7f4ae6a00640 1 -- 172.21.15.152:0/3115322807 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ae81587d0 con 0x7f4ae815fcc0 2024-04-20T11:37:52.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.268+0000 7f4ae6a00640 1 --2- 172.21.15.152:0/3115322807 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ae815fcc0 0x7f4ae8153a50 secure :-1 s=READY pgs=2315 cs=0 l=1 rev1=1 crypto rx=0x7f4ad000c7b0 tx=0x7f4ad000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:52.270 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.268+0000 7f4adf400640 1 -- 172.21.15.152:0/3115322807 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad0016020 con 0x7f4ae815fcc0 2024-04-20T11:37:52.270 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.268+0000 7f4adf400640 1 -- 172.21.15.152:0/3115322807 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ad000aa50 con 0x7f4ae815fcc0 2024-04-20T11:37:52.270 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.268+0000 7f4adf400640 1 -- 172.21.15.152:0/3115322807 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad000ae40 con 0x7f4ae815fcc0 2024-04-20T11:37:52.270 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.268+0000 7f4aed200640 1 -- 172.21.15.152:0/3115322807 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ae81562c0 con 0x7f4ae815fcc0 2024-04-20T11:37:52.270 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.268+0000 7f4aed200640 1 -- 172.21.15.152:0/3115322807 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ae8156880 con 0x7f4ae815fcc0 2024-04-20T11:37:52.270 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.269+0000 7f4aed200640 1 -- 172.21.15.152:0/3115322807 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ae8075e60 con 0x7f4ae815fcc0 2024-04-20T11:37:52.271 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.270+0000 7f4adf400640 1 -- 172.21.15.152:0/3115322807 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 26) ==== 91907+0+0 (secure 0 0 0) 0x7f4ad0019020 con 0x7f4ae815fcc0 2024-04-20T11:37:52.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.271+0000 7f4adf400640 1 --2- 172.21.15.152:0/3115322807 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4ac406d280 0x7f4ac406f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:52.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.271+0000 7f4adf400640 1 -- 172.21.15.152:0/3115322807 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 5709+0+0 (secure 0 0 0) 0x7f4ad008a5d0 con 0x7f4ae815fcc0 2024-04-20T11:37:52.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.272+0000 7f4ae7400640 1 --2- 172.21.15.152:0/3115322807 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4ac406d280 0x7f4ac406f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:52.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.273+0000 7f4ae7400640 1 --2- 172.21.15.152:0/3115322807 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4ac406d280 0x7f4ac406f740 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f4ad8007fd0 tx=0x7f4ad8009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:52.277 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.277+0000 7f4adf400640 1 -- 172.21.15.152:0/3115322807 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f4ae81587d0 con 0x7f4ae815fcc0 2024-04-20T11:37:52.471 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.470+0000 7f4aed200640 1 -- 172.21.15.152:0/3115322807 --> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4ae80b6ce0 con 0x7f4ac406d280 2024-04-20T11:37:52.489 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.489+0000 7f4adf400640 1 -- 172.21.15.152:0/3115322807 <== mgr.4109 v2:172.21.15.175:6836/4021250441 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252421 (secure 0 0 0) 0x7f4ae80b6ce0 con 0x7f4ac406d280 2024-04-20T11:37:52.493 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:37:52.494 INFO:teuthology.orchestra.run.smithi152.stderr:dumped all 2024-04-20T11:37:52.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.494+0000 7f4aed200640 1 -- 172.21.15.152:0/3115322807 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4ac406d280 msgr2=0x7f4ac406f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:52.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.494+0000 7f4aed200640 1 --2- 172.21.15.152:0/3115322807 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4ac406d280 0x7f4ac406f740 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f4ad8007fd0 tx=0x7f4ad8009040 comp rx=0 tx=0).stop 2024-04-20T11:37:52.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.494+0000 7f4aed200640 1 -- 172.21.15.152:0/3115322807 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ae815fcc0 msgr2=0x7f4ae8153a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:52.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.494+0000 7f4aed200640 1 --2- 172.21.15.152:0/3115322807 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ae815fcc0 0x7f4ae8153a50 secure :-1 s=READY pgs=2315 cs=0 l=1 rev1=1 crypto rx=0x7f4ad000c7b0 tx=0x7f4ad000cc70 comp rx=0 tx=0).stop 2024-04-20T11:37:52.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.494+0000 7f4aed200640 1 -- 172.21.15.152:0/3115322807 shutdown_connections 2024-04-20T11:37:52.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.494+0000 7f4aed200640 1 --2- 172.21.15.152:0/3115322807 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4ac406d280 0x7f4ac406f740 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:52.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.494+0000 7f4aed200640 1 --2- 172.21.15.152:0/3115322807 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4ae81603d0 0x7f4ae81579a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:52.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.494+0000 7f4aed200640 1 --2- 172.21.15.152:0/3115322807 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ae815fcc0 0x7f4ae8153a50 unknown :-1 s=CLOSED pgs=2315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:52.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.494+0000 7f4aed200640 1 --2- 172.21.15.152:0/3115322807 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4ae80b65c0 0x7f4ae80cb620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:52.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.494+0000 7f4aed200640 1 -- 172.21.15.152:0/3115322807 >> 172.21.15.152:0/3115322807 conn(0x7f4ae808bfd0 msgr2=0x7f4ae806caf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:52.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.495+0000 7f4aed200640 1 -- 172.21.15.152:0/3115322807 shutdown_connections 2024-04-20T11:37:52.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.495+0000 7f4aed200640 1 -- 172.21.15.152:0/3115322807 wait complete. 2024-04-20T11:37:52.514 INFO:teuthology.orchestra.run.smithi152.stdout:{"pg_ready":true,"pg_map":{"version":263,"stamp":"2024-04-20T11:37:50.718864+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":223,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":166,"ondisk_log_size":166,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":118,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":271216,"kb_used_data":17576,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754703504,"statfs":{"total":773094113280,"available":772816388096,"internally_reserved":0,"allocated":17997824,"data_stored":8590278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12746,"internal_metadata":259575350},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.680182"},"pg_stats":[{"pgid":"27.3f","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.112861+0000","last_change":"2024-04-20T11:37:48.110014+0000","last_active":"2024-04-20T11:37:48.112861+0000","last_peered":"2024-04-20T11:37:48.112861+0000","last_clean":"2024-04-20T11:37:48.112861+0000","last_became_active":"2024-04-20T11:37:48.109604+0000","last_became_peered":"2024-04-20T11:37:48.109604+0000","last_unstale":"2024-04-20T11:37:48.112861+0000","last_undegraded":"2024-04-20T11:37:48.112861+0000","last_fullsized":"2024-04-20T11:37:48.112861+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:58:28.810390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.3e","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.113219+0000","last_change":"2024-04-20T11:37:48.111899+0000","last_active":"2024-04-20T11:37:48.113219+0000","last_peered":"2024-04-20T11:37:48.113219+0000","last_clean":"2024-04-20T11:37:48.113219+0000","last_became_active":"2024-04-20T11:37:48.111494+0000","last_became_peered":"2024-04-20T11:37:48.111494+0000","last_unstale":"2024-04-20T11:37:48.113219+0000","last_undegraded":"2024-04-20T11:37:48.113219+0000","last_fullsized":"2024-04-20T11:37:48.113219+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:16:11.827279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.3d","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.117372+0000","last_change":"2024-04-20T11:37:48.113842+0000","last_active":"2024-04-20T11:37:48.117372+0000","last_peered":"2024-04-20T11:37:48.117372+0000","last_clean":"2024-04-20T11:37:48.117372+0000","last_became_active":"2024-04-20T11:37:48.111221+0000","last_became_peered":"2024-04-20T11:37:48.111221+0000","last_unstale":"2024-04-20T11:37:48.117372+0000","last_undegraded":"2024-04-20T11:37:48.117372+0000","last_fullsized":"2024-04-20T11:37:48.117372+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:16:11.827279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.3c","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.118106+0000","last_change":"2024-04-20T11:37:48.114560+0000","last_active":"2024-04-20T11:37:48.118106+0000","last_peered":"2024-04-20T11:37:48.118106+0000","last_clean":"2024-04-20T11:37:48.118106+0000","last_became_active":"2024-04-20T11:37:48.113882+0000","last_became_peered":"2024-04-20T11:37:48.113882+0000","last_unstale":"2024-04-20T11:37:48.118106+0000","last_undegraded":"2024-04-20T11:37:48.118106+0000","last_fullsized":"2024-04-20T11:37:48.118106+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:37:02.492137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3b","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.122803+0000","last_change":"2024-04-20T11:37:48.120358+0000","last_active":"2024-04-20T11:37:48.122803+0000","last_peered":"2024-04-20T11:37:48.122803+0000","last_clean":"2024-04-20T11:37:48.122803+0000","last_became_active":"2024-04-20T11:37:48.119698+0000","last_became_peered":"2024-04-20T11:37:48.119698+0000","last_unstale":"2024-04-20T11:37:48.122803+0000","last_undegraded":"2024-04-20T11:37:48.122803+0000","last_fullsized":"2024-04-20T11:37:48.122803+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:05:03.866740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3a","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.117265+0000","last_change":"2024-04-20T11:37:48.111495+0000","last_active":"2024-04-20T11:37:48.117265+0000","last_peered":"2024-04-20T11:37:48.117265+0000","last_clean":"2024-04-20T11:37:48.117265+0000","last_became_active":"2024-04-20T11:37:48.111156+0000","last_became_peered":"2024-04-20T11:37:48.111156+0000","last_unstale":"2024-04-20T11:37:48.117265+0000","last_undegraded":"2024-04-20T11:37:48.117265+0000","last_fullsized":"2024-04-20T11:37:48.117265+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:33:33.188008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.39","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.112757+0000","last_change":"2024-04-20T11:37:48.110685+0000","last_active":"2024-04-20T11:37:48.112757+0000","last_peered":"2024-04-20T11:37:48.112757+0000","last_clean":"2024-04-20T11:37:48.112757+0000","last_became_active":"2024-04-20T11:37:48.110239+0000","last_became_peered":"2024-04-20T11:37:48.110239+0000","last_unstale":"2024-04-20T11:37:48.112757+0000","last_undegraded":"2024-04-20T11:37:48.112757+0000","last_fullsized":"2024-04-20T11:37:48.112757+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:18:55.243430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.38","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.113216+0000","last_change":"2024-04-20T11:37:48.110355+0000","last_active":"2024-04-20T11:37:48.113216+0000","last_peered":"2024-04-20T11:37:48.113216+0000","last_clean":"2024-04-20T11:37:48.113216+0000","last_became_active":"2024-04-20T11:37:48.107599+0000","last_became_peered":"2024-04-20T11:37:48.107599+0000","last_unstale":"2024-04-20T11:37:48.113216+0000","last_undegraded":"2024-04-20T11:37:48.113216+0000","last_fullsized":"2024-04-20T11:37:48.113216+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:31:34.765310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.37","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.111581+0000","last_change":"2024-04-20T11:37:48.108719+0000","last_active":"2024-04-20T11:37:48.111581+0000","last_peered":"2024-04-20T11:37:48.111581+0000","last_clean":"2024-04-20T11:37:48.111581+0000","last_became_active":"2024-04-20T11:37:48.108425+0000","last_became_peered":"2024-04-20T11:37:48.108425+0000","last_unstale":"2024-04-20T11:37:48.111581+0000","last_undegraded":"2024-04-20T11:37:48.111581+0000","last_fullsized":"2024-04-20T11:37:48.111581+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:49:10.326407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.36","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.118530+0000","last_change":"2024-04-20T11:37:48.103567+0000","last_active":"2024-04-20T11:37:48.118530+0000","last_peered":"2024-04-20T11:37:48.118530+0000","last_clean":"2024-04-20T11:37:48.118530+0000","last_became_active":"2024-04-20T11:37:48.103296+0000","last_became_peered":"2024-04-20T11:37:48.103296+0000","last_unstale":"2024-04-20T11:37:48.118530+0000","last_undegraded":"2024-04-20T11:37:48.118530+0000","last_fullsized":"2024-04-20T11:37:48.118530+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:55:14.028926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.35","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.117353+0000","last_change":"2024-04-20T11:37:48.113400+0000","last_active":"2024-04-20T11:37:48.117353+0000","last_peered":"2024-04-20T11:37:48.117353+0000","last_clean":"2024-04-20T11:37:48.117353+0000","last_became_active":"2024-04-20T11:37:48.113136+0000","last_became_peered":"2024-04-20T11:37:48.113136+0000","last_unstale":"2024-04-20T11:37:48.117353+0000","last_undegraded":"2024-04-20T11:37:48.117353+0000","last_fullsized":"2024-04-20T11:37:48.117353+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:56:16.721575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.34","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.120878+0000","last_change":"2024-04-20T11:37:48.117085+0000","last_active":"2024-04-20T11:37:48.120878+0000","last_peered":"2024-04-20T11:37:48.120878+0000","last_clean":"2024-04-20T11:37:48.120878+0000","last_became_active":"2024-04-20T11:37:48.116688+0000","last_became_peered":"2024-04-20T11:37:48.116688+0000","last_unstale":"2024-04-20T11:37:48.120878+0000","last_undegraded":"2024-04-20T11:37:48.120878+0000","last_fullsized":"2024-04-20T11:37:48.120878+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:51:41.363330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.33","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.113104+0000","last_change":"2024-04-20T11:37:48.107803+0000","last_active":"2024-04-20T11:37:48.113104+0000","last_peered":"2024-04-20T11:37:48.113104+0000","last_clean":"2024-04-20T11:37:48.113104+0000","last_became_active":"2024-04-20T11:37:48.107441+0000","last_became_peered":"2024-04-20T11:37:48.107441+0000","last_unstale":"2024-04-20T11:37:48.113104+0000","last_undegraded":"2024-04-20T11:37:48.113104+0000","last_fullsized":"2024-04-20T11:37:48.113104+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:24:53.390054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.32","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.117828+0000","last_change":"2024-04-20T11:37:48.116007+0000","last_active":"2024-04-20T11:37:48.117828+0000","last_peered":"2024-04-20T11:37:48.117828+0000","last_clean":"2024-04-20T11:37:48.117828+0000","last_became_active":"2024-04-20T11:37:48.114524+0000","last_became_peered":"2024-04-20T11:37:48.114524+0000","last_unstale":"2024-04-20T11:37:48.117828+0000","last_undegraded":"2024-04-20T11:37:48.117828+0000","last_fullsized":"2024-04-20T11:37:48.117828+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:37:42.226969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.31","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.105699+0000","last_change":"2024-04-20T11:37:48.102631+0000","last_active":"2024-04-20T11:37:48.105699+0000","last_peered":"2024-04-20T11:37:48.105699+0000","last_clean":"2024-04-20T11:37:48.105699+0000","last_became_active":"2024-04-20T11:37:48.101097+0000","last_became_peered":"2024-04-20T11:37:48.101097+0000","last_unstale":"2024-04-20T11:37:48.105699+0000","last_undegraded":"2024-04-20T11:37:48.105699+0000","last_fullsized":"2024-04-20T11:37:48.105699+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:56:04.982500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.30","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.118913+0000","last_change":"2024-04-20T11:37:48.108421+0000","last_active":"2024-04-20T11:37:48.118913+0000","last_peered":"2024-04-20T11:37:48.118913+0000","last_clean":"2024-04-20T11:37:48.118913+0000","last_became_active":"2024-04-20T11:37:48.108116+0000","last_became_peered":"2024-04-20T11:37:48.108116+0000","last_unstale":"2024-04-20T11:37:48.118913+0000","last_undegraded":"2024-04-20T11:37:48.118913+0000","last_fullsized":"2024-04-20T11:37:48.118913+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:05:47.730738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2f","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.112839+0000","last_change":"2024-04-20T11:37:48.109128+0000","last_active":"2024-04-20T11:37:48.112839+0000","last_peered":"2024-04-20T11:37:48.112839+0000","last_clean":"2024-04-20T11:37:48.112839+0000","last_became_active":"2024-04-20T11:37:48.108798+0000","last_became_peered":"2024-04-20T11:37:48.108798+0000","last_unstale":"2024-04-20T11:37:48.112839+0000","last_undegraded":"2024-04-20T11:37:48.112839+0000","last_fullsized":"2024-04-20T11:37:48.112839+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:23:09.091466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.2e","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.118557+0000","last_change":"2024-04-20T11:37:48.105543+0000","last_active":"2024-04-20T11:37:48.118557+0000","last_peered":"2024-04-20T11:37:48.118557+0000","last_clean":"2024-04-20T11:37:48.118557+0000","last_became_active":"2024-04-20T11:37:48.100271+0000","last_became_peered":"2024-04-20T11:37:48.100271+0000","last_unstale":"2024-04-20T11:37:48.118557+0000","last_undegraded":"2024-04-20T11:37:48.118557+0000","last_fullsized":"2024-04-20T11:37:48.118557+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:59:49.660352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2d","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.118652+0000","last_change":"2024-04-20T11:37:48.106550+0000","last_active":"2024-04-20T11:37:48.118652+0000","last_peered":"2024-04-20T11:37:48.118652+0000","last_clean":"2024-04-20T11:37:48.118652+0000","last_became_active":"2024-04-20T11:37:48.105926+0000","last_became_peered":"2024-04-20T11:37:48.105926+0000","last_unstale":"2024-04-20T11:37:48.118652+0000","last_undegraded":"2024-04-20T11:37:48.118652+0000","last_fullsized":"2024-04-20T11:37:48.118652+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:23:08.343514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2c","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.118209+0000","last_change":"2024-04-20T11:37:48.112417+0000","last_active":"2024-04-20T11:37:48.118209+0000","last_peered":"2024-04-20T11:37:48.118209+0000","last_clean":"2024-04-20T11:37:48.118209+0000","last_became_active":"2024-04-20T11:37:48.112060+0000","last_became_peered":"2024-04-20T11:37:48.112060+0000","last_unstale":"2024-04-20T11:37:48.118209+0000","last_undegraded":"2024-04-20T11:37:48.118209+0000","last_fullsized":"2024-04-20T11:37:48.118209+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:08:16.946122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.2b","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.116690+0000","last_change":"2024-04-20T11:37:48.109694+0000","last_active":"2024-04-20T11:37:48.116690+0000","last_peered":"2024-04-20T11:37:48.116690+0000","last_clean":"2024-04-20T11:37:48.116690+0000","last_became_active":"2024-04-20T11:37:48.109417+0000","last_became_peered":"2024-04-20T11:37:48.109417+0000","last_unstale":"2024-04-20T11:37:48.116690+0000","last_undegraded":"2024-04-20T11:37:48.116690+0000","last_fullsized":"2024-04-20T11:37:48.116690+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:25:24.946294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2a","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.122191+0000","last_change":"2024-04-20T11:37:48.121180+0000","last_active":"2024-04-20T11:37:48.122191+0000","last_peered":"2024-04-20T11:37:48.122191+0000","last_clean":"2024-04-20T11:37:48.122191+0000","last_became_active":"2024-04-20T11:37:48.120321+0000","last_became_peered":"2024-04-20T11:37:48.120321+0000","last_unstale":"2024-04-20T11:37:48.122191+0000","last_undegraded":"2024-04-20T11:37:48.122191+0000","last_fullsized":"2024-04-20T11:37:48.122191+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:31:47.943569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.29","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.111394+0000","last_change":"2024-04-20T11:37:48.108383+0000","last_active":"2024-04-20T11:37:48.111394+0000","last_peered":"2024-04-20T11:37:48.111394+0000","last_clean":"2024-04-20T11:37:48.111394+0000","last_became_active":"2024-04-20T11:37:48.107974+0000","last_became_peered":"2024-04-20T11:37:48.107974+0000","last_unstale":"2024-04-20T11:37:48.111394+0000","last_undegraded":"2024-04-20T11:37:48.111394+0000","last_fullsized":"2024-04-20T11:37:48.111394+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:08:16.946122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.28","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.117571+0000","last_change":"2024-04-20T11:37:48.111530+0000","last_active":"2024-04-20T11:37:48.117571+0000","last_peered":"2024-04-20T11:37:48.117571+0000","last_clean":"2024-04-20T11:37:48.117571+0000","last_became_active":"2024-04-20T11:37:48.111259+0000","last_became_peered":"2024-04-20T11:37:48.111259+0000","last_unstale":"2024-04-20T11:37:48.117571+0000","last_undegraded":"2024-04-20T11:37:48.117571+0000","last_fullsized":"2024-04-20T11:37:48.117571+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:24:54.986907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.27","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.111560+0000","last_change":"2024-04-20T11:37:48.104944+0000","last_active":"2024-04-20T11:37:48.111560+0000","last_peered":"2024-04-20T11:37:48.111560+0000","last_clean":"2024-04-20T11:37:48.111560+0000","last_became_active":"2024-04-20T11:37:48.099879+0000","last_became_peered":"2024-04-20T11:37:48.099879+0000","last_unstale":"2024-04-20T11:37:48.111560+0000","last_undegraded":"2024-04-20T11:37:48.111560+0000","last_fullsized":"2024-04-20T11:37:48.111560+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:30:03.394619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.26","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.113627+0000","last_change":"2024-04-20T11:37:48.110830+0000","last_active":"2024-04-20T11:37:48.113627+0000","last_peered":"2024-04-20T11:37:48.113627+0000","last_clean":"2024-04-20T11:37:48.113627+0000","last_became_active":"2024-04-20T11:37:48.110535+0000","last_became_peered":"2024-04-20T11:37:48.110535+0000","last_unstale":"2024-04-20T11:37:48.113627+0000","last_undegraded":"2024-04-20T11:37:48.113627+0000","last_fullsized":"2024-04-20T11:37:48.113627+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:29:13.486540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.25","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.120994+0000","last_change":"2024-04-20T11:37:48.118234+0000","last_active":"2024-04-20T11:37:48.120994+0000","last_peered":"2024-04-20T11:37:48.120994+0000","last_clean":"2024-04-20T11:37:48.120994+0000","last_became_active":"2024-04-20T11:37:48.117890+0000","last_became_peered":"2024-04-20T11:37:48.117890+0000","last_unstale":"2024-04-20T11:37:48.120994+0000","last_undegraded":"2024-04-20T11:37:48.120994+0000","last_fullsized":"2024-04-20T11:37:48.120994+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:18:27.316456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.24","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.109493+0000","last_change":"2024-04-20T11:37:48.106896+0000","last_active":"2024-04-20T11:37:48.109493+0000","last_peered":"2024-04-20T11:37:48.109493+0000","last_clean":"2024-04-20T11:37:48.109493+0000","last_became_active":"2024-04-20T11:37:48.106600+0000","last_became_peered":"2024-04-20T11:37:48.106600+0000","last_unstale":"2024-04-20T11:37:48.109493+0000","last_undegraded":"2024-04-20T11:37:48.109493+0000","last_fullsized":"2024-04-20T11:37:48.109493+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:49:10.326407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.23","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.119856+0000","last_change":"2024-04-20T11:37:48.116816+0000","last_active":"2024-04-20T11:37:48.119856+0000","last_peered":"2024-04-20T11:37:48.119856+0000","last_clean":"2024-04-20T11:37:48.119856+0000","last_became_active":"2024-04-20T11:37:48.116504+0000","last_became_peered":"2024-04-20T11:37:48.116504+0000","last_unstale":"2024-04-20T11:37:48.119856+0000","last_undegraded":"2024-04-20T11:37:48.119856+0000","last_fullsized":"2024-04-20T11:37:48.119856+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:17:07.054161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.22","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.120067+0000","last_change":"2024-04-20T11:37:48.117391+0000","last_active":"2024-04-20T11:37:48.120067+0000","last_peered":"2024-04-20T11:37:48.120067+0000","last_clean":"2024-04-20T11:37:48.120067+0000","last_became_active":"2024-04-20T11:37:48.113649+0000","last_became_peered":"2024-04-20T11:37:48.113649+0000","last_unstale":"2024-04-20T11:37:48.120067+0000","last_undegraded":"2024-04-20T11:37:48.120067+0000","last_fullsized":"2024-04-20T11:37:48.120067+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:49:59.397783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.21","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.105927+0000","last_change":"2024-04-20T11:37:48.098202+0000","last_active":"2024-04-20T11:37:48.105927+0000","last_peered":"2024-04-20T11:37:48.105927+0000","last_clean":"2024-04-20T11:37:48.105927+0000","last_became_active":"2024-04-20T11:37:48.097719+0000","last_became_peered":"2024-04-20T11:37:48.097719+0000","last_unstale":"2024-04-20T11:37:48.105927+0000","last_undegraded":"2024-04-20T11:37:48.105927+0000","last_fullsized":"2024-04-20T11:37:48.105927+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:45:55.333086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.20","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.120168+0000","last_change":"2024-04-20T11:37:48.119020+0000","last_active":"2024-04-20T11:37:48.120168+0000","last_peered":"2024-04-20T11:37:48.120168+0000","last_clean":"2024-04-20T11:37:48.120168+0000","last_became_active":"2024-04-20T11:37:48.117155+0000","last_became_peered":"2024-04-20T11:37:48.117155+0000","last_unstale":"2024-04-20T11:37:48.120168+0000","last_undegraded":"2024-04-20T11:37:48.120168+0000","last_fullsized":"2024-04-20T11:37:48.120168+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:38:32.481887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.1e","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.103374+0000","last_change":"2024-04-20T11:37:48.096757+0000","last_active":"2024-04-20T11:37:48.103374+0000","last_peered":"2024-04-20T11:37:48.103374+0000","last_clean":"2024-04-20T11:37:48.103374+0000","last_became_active":"2024-04-20T11:37:48.092696+0000","last_became_peered":"2024-04-20T11:37:48.092696+0000","last_unstale":"2024-04-20T11:37:48.103374+0000","last_undegraded":"2024-04-20T11:37:48.103374+0000","last_fullsized":"2024-04-20T11:37:48.103374+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:31:20.962446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.1f","version":"166'4","reported_seq":16,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.139951+0000","last_change":"2024-04-20T11:37:47.078653+0000","last_active":"2024-04-20T11:37:48.139951+0000","last_peered":"2024-04-20T11:37:48.139951+0000","last_clean":"2024-04-20T11:37:48.139951+0000","last_became_active":"2024-04-20T11:37:47.078374+0000","last_became_peered":"2024-04-20T11:37:47.078374+0000","last_unstale":"2024-04-20T11:37:48.139951+0000","last_undegraded":"2024-04-20T11:37:48.139951+0000","last_fullsized":"2024-04-20T11:37:48.139951+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_clean_scrub_stamp":"2024-04-20T11:37:46.019877+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:46:38.860806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1f","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.120816+0000","last_change":"2024-04-20T11:37:48.119678+0000","last_active":"2024-04-20T11:37:48.120816+0000","last_peered":"2024-04-20T11:37:48.120816+0000","last_clean":"2024-04-20T11:37:48.120816+0000","last_became_active":"2024-04-20T11:37:48.119306+0000","last_became_peered":"2024-04-20T11:37:48.119306+0000","last_unstale":"2024-04-20T11:37:48.120816+0000","last_undegraded":"2024-04-20T11:37:48.120816+0000","last_fullsized":"2024-04-20T11:37:48.120816+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:46:30.542546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.1e","version":"166'1","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.139549+0000","last_change":"2024-04-20T11:37:47.079218+0000","last_active":"2024-04-20T11:37:48.139549+0000","last_peered":"2024-04-20T11:37:48.139549+0000","last_clean":"2024-04-20T11:37:48.139549+0000","last_became_active":"2024-04-20T11:37:47.078908+0000","last_became_peered":"2024-04-20T11:37:47.078908+0000","last_unstale":"2024-04-20T11:37:48.139549+0000","last_undegraded":"2024-04-20T11:37:48.139549+0000","last_fullsized":"2024-04-20T11:37:48.139549+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_clean_scrub_stamp":"2024-04-20T11:37:46.019877+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:17:29.048669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1c","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.115559+0000","last_change":"2024-04-20T11:37:48.112060+0000","last_active":"2024-04-20T11:37:48.115559+0000","last_peered":"2024-04-20T11:37:48.115559+0000","last_clean":"2024-04-20T11:37:48.115559+0000","last_became_active":"2024-04-20T11:37:48.111752+0000","last_became_peered":"2024-04-20T11:37:48.111752+0000","last_unstale":"2024-04-20T11:37:48.115559+0000","last_undegraded":"2024-04-20T11:37:48.115559+0000","last_fullsized":"2024-04-20T11:37:48.115559+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:28:58.720468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1d","version":"166'2","reported_seq":14,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.132462+0000","last_change":"2024-04-20T11:37:47.082172+0000","last_active":"2024-04-20T11:37:48.132462+0000","last_peered":"2024-04-20T11:37:48.132462+0000","last_clean":"2024-04-20T11:37:48.132462+0000","last_became_active":"2024-04-20T11:37:47.081688+0000","last_became_peered":"2024-04-20T11:37:47.081688+0000","last_unstale":"2024-04-20T11:37:48.132462+0000","last_undegraded":"2024-04-20T11:37:48.132462+0000","last_fullsized":"2024-04-20T11:37:48.132462+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_clean_scrub_stamp":"2024-04-20T11:37:46.019877+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:31:19.946278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1d","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.107141+0000","last_change":"2024-04-20T11:37:48.098935+0000","last_active":"2024-04-20T11:37:48.107141+0000","last_peered":"2024-04-20T11:37:48.107141+0000","last_clean":"2024-04-20T11:37:48.107141+0000","last_became_active":"2024-04-20T11:37:48.098623+0000","last_became_peered":"2024-04-20T11:37:48.098623+0000","last_unstale":"2024-04-20T11:37:48.107141+0000","last_undegraded":"2024-04-20T11:37:48.107141+0000","last_fullsized":"2024-04-20T11:37:48.107141+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:42:28.063123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1c","version":"166'2","reported_seq":14,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.137128+0000","last_change":"2024-04-20T11:37:47.088265+0000","last_active":"2024-04-20T11:37:48.137128+0000","last_peered":"2024-04-20T11:37:48.137128+0000","last_clean":"2024-04-20T11:37:48.137128+0000","last_became_active":"2024-04-20T11:37:47.087899+0000","last_became_peered":"2024-04-20T11:37:47.087899+0000","last_unstale":"2024-04-20T11:37:48.137128+0000","last_undegraded":"2024-04-20T11:37:48.137128+0000","last_fullsized":"2024-04-20T11:37:48.137128+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_clean_scrub_stamp":"2024-04-20T11:37:46.019877+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:12:20.796103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1b","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.117593+0000","last_change":"2024-04-20T11:37:48.112994+0000","last_active":"2024-04-20T11:37:48.117593+0000","last_peered":"2024-04-20T11:37:48.117593+0000","last_clean":"2024-04-20T11:37:48.117593+0000","last_became_active":"2024-04-20T11:37:48.112719+0000","last_became_peered":"2024-04-20T11:37:48.112719+0000","last_unstale":"2024-04-20T11:37:48.117593+0000","last_undegraded":"2024-04-20T11:37:48.117593+0000","last_fullsized":"2024-04-20T11:37:48.117593+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:09:49.168773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1a","version":"166'1","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.132922+0000","last_change":"2024-04-20T11:37:47.077871+0000","last_active":"2024-04-20T11:37:48.132922+0000","last_peered":"2024-04-20T11:37:48.132922+0000","last_clean":"2024-04-20T11:37:48.132922+0000","last_became_active":"2024-04-20T11:37:47.077568+0000","last_became_peered":"2024-04-20T11:37:47.077568+0000","last_unstale":"2024-04-20T11:37:48.132922+0000","last_undegraded":"2024-04-20T11:37:48.132922+0000","last_fullsized":"2024-04-20T11:37:48.132922+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_clean_scrub_stamp":"2024-04-20T11:37:46.019877+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:13:04.063611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.118574+0000","last_change":"2024-04-20T11:37:48.115194+0000","last_active":"2024-04-20T11:37:48.118574+0000","last_peered":"2024-04-20T11:37:48.118574+0000","last_clean":"2024-04-20T11:37:48.118574+0000","last_became_active":"2024-04-20T11:37:48.114053+0000","last_became_peered":"2024-04-20T11:37:48.114053+0000","last_unstale":"2024-04-20T11:37:48.118574+0000","last_undegraded":"2024-04-20T11:37:48.118574+0000","last_fullsized":"2024-04-20T11:37:48.118574+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:03:23.366996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":12,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.104088+0000","last_change":"2024-04-20T11:37:47.077239+0000","last_active":"2024-04-20T11:37:48.104088+0000","last_peered":"2024-04-20T11:37:48.104088+0000","last_clean":"2024-04-20T11:37:48.104088+0000","last_became_active":"2024-04-20T11:37:47.076919+0000","last_became_peered":"2024-04-20T11:37:47.076919+0000","last_unstale":"2024-04-20T11:37:48.104088+0000","last_undegraded":"2024-04-20T11:37:48.104088+0000","last_fullsized":"2024-04-20T11:37:48.104088+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_clean_scrub_stamp":"2024-04-20T11:37:46.019877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:18:26.300288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.116795+0000","last_change":"2024-04-20T11:37:48.108749+0000","last_active":"2024-04-20T11:37:48.116795+0000","last_peered":"2024-04-20T11:37:48.116795+0000","last_clean":"2024-04-20T11:37:48.116795+0000","last_became_active":"2024-04-20T11:37:48.108426+0000","last_became_peered":"2024-04-20T11:37:48.108426+0000","last_unstale":"2024-04-20T11:37:48.116795+0000","last_undegraded":"2024-04-20T11:37:48.116795+0000","last_fullsized":"2024-04-20T11:37:48.116795+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:32:24.547538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":12,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.094892+0000","last_change":"2024-04-20T11:37:47.081824+0000","last_active":"2024-04-20T11:37:48.094892+0000","last_peered":"2024-04-20T11:37:48.094892+0000","last_clean":"2024-04-20T11:37:48.094892+0000","last_became_active":"2024-04-20T11:37:47.081004+0000","last_became_peered":"2024-04-20T11:37:47.081004+0000","last_unstale":"2024-04-20T11:37:48.094892+0000","last_undegraded":"2024-04-20T11:37:48.094892+0000","last_fullsized":"2024-04-20T11:37:48.094892+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_clean_scrub_stamp":"2024-04-20T11:37:46.019877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:18:14.091047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.118048+0000","last_change":"2024-04-20T11:37:48.113664+0000","last_active":"2024-04-20T11:37:48.118048+0000","last_peered":"2024-04-20T11:37:48.118048+0000","last_clean":"2024-04-20T11:37:48.118048+0000","last_became_active":"2024-04-20T11:37:48.113388+0000","last_became_peered":"2024-04-20T11:37:48.113388+0000","last_unstale":"2024-04-20T11:37:48.118048+0000","last_undegraded":"2024-04-20T11:37:48.118048+0000","last_fullsized":"2024-04-20T11:37:48.118048+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:16:17.021486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":12,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.104881+0000","last_change":"2024-04-20T11:37:47.079085+0000","last_active":"2024-04-20T11:37:48.104881+0000","last_peered":"2024-04-20T11:37:48.104881+0000","last_clean":"2024-04-20T11:37:48.104881+0000","last_became_active":"2024-04-20T11:37:47.078764+0000","last_became_peered":"2024-04-20T11:37:47.078764+0000","last_unstale":"2024-04-20T11:37:48.104881+0000","last_undegraded":"2024-04-20T11:37:48.104881+0000","last_fullsized":"2024-04-20T11:37:48.104881+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_clean_scrub_stamp":"2024-04-20T11:37:46.019877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:11:55.028361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.123282+0000","last_change":"2024-04-20T11:37:48.118132+0000","last_active":"2024-04-20T11:37:48.123282+0000","last_peered":"2024-04-20T11:37:48.123282+0000","last_clean":"2024-04-20T11:37:48.123282+0000","last_became_active":"2024-04-20T11:37:48.117030+0000","last_became_peered":"2024-04-20T11:37:48.117030+0000","last_unstale":"2024-04-20T11:37:48.123282+0000","last_undegraded":"2024-04-20T11:37:48.123282+0000","last_fullsized":"2024-04-20T11:37:48.123282+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:07:37.215053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":12,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.106328+0000","last_change":"2024-04-20T11:37:47.076623+0000","last_active":"2024-04-20T11:37:48.106328+0000","last_peered":"2024-04-20T11:37:48.106328+0000","last_clean":"2024-04-20T11:37:48.106328+0000","last_became_active":"2024-04-20T11:37:47.075990+0000","last_became_peered":"2024-04-20T11:37:47.075990+0000","last_unstale":"2024-04-20T11:37:48.106328+0000","last_undegraded":"2024-04-20T11:37:48.106328+0000","last_fullsized":"2024-04-20T11:37:48.106328+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_clean_scrub_stamp":"2024-04-20T11:37:46.019877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:30:26.452766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.14","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.118128+0000","last_change":"2024-04-20T11:37:48.114787+0000","last_active":"2024-04-20T11:37:48.118128+0000","last_peered":"2024-04-20T11:37:48.118128+0000","last_clean":"2024-04-20T11:37:48.118128+0000","last_became_active":"2024-04-20T11:37:48.114272+0000","last_became_peered":"2024-04-20T11:37:48.114272+0000","last_unstale":"2024-04-20T11:37:48.118128+0000","last_undegraded":"2024-04-20T11:37:48.118128+0000","last_fullsized":"2024-04-20T11:37:48.118128+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:55:21.709133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.15","version":"166'3","reported_seq":15,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.134358+0000","last_change":"2024-04-20T11:37:47.073262+0000","last_active":"2024-04-20T11:37:48.134358+0000","last_peered":"2024-04-20T11:37:48.134358+0000","last_clean":"2024-04-20T11:37:48.134358+0000","last_became_active":"2024-04-20T11:37:47.072961+0000","last_became_peered":"2024-04-20T11:37:47.072961+0000","last_unstale":"2024-04-20T11:37:48.134358+0000","last_undegraded":"2024-04-20T11:37:48.134358+0000","last_fullsized":"2024-04-20T11:37:48.134358+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_clean_scrub_stamp":"2024-04-20T11:37:46.019877+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:37:01.475969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.15","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.107560+0000","last_change":"2024-04-20T11:37:48.105449+0000","last_active":"2024-04-20T11:37:48.107560+0000","last_peered":"2024-04-20T11:37:48.107560+0000","last_clean":"2024-04-20T11:37:48.107560+0000","last_became_active":"2024-04-20T11:37:48.104338+0000","last_became_peered":"2024-04-20T11:37:48.104338+0000","last_unstale":"2024-04-20T11:37:48.107560+0000","last_undegraded":"2024-04-20T11:37:48.107560+0000","last_fullsized":"2024-04-20T11:37:48.107560+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:04:41.081124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.14","version":"166'6","reported_seq":18,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.140046+0000","last_change":"2024-04-20T11:37:47.082031+0000","last_active":"2024-04-20T11:37:48.140046+0000","last_peered":"2024-04-20T11:37:48.140046+0000","last_clean":"2024-04-20T11:37:48.140046+0000","last_became_active":"2024-04-20T11:37:47.081736+0000","last_became_peered":"2024-04-20T11:37:47.081736+0000","last_unstale":"2024-04-20T11:37:48.140046+0000","last_undegraded":"2024-04-20T11:37:48.140046+0000","last_fullsized":"2024-04-20T11:37:48.140046+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_clean_scrub_stamp":"2024-04-20T11:37:46.019877+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:37:06.748530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.117287+0000","last_change":"2024-04-20T11:37:48.112035+0000","last_active":"2024-04-20T11:37:48.117287+0000","last_peered":"2024-04-20T11:37:48.117287+0000","last_clean":"2024-04-20T11:37:48.117287+0000","last_became_active":"2024-04-20T11:37:48.111552+0000","last_became_peered":"2024-04-20T11:37:48.111552+0000","last_unstale":"2024-04-20T11:37:48.117287+0000","last_undegraded":"2024-04-20T11:37:48.117287+0000","last_fullsized":"2024-04-20T11:37:48.117287+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:30:27.468934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.13","version":"166'2","reported_seq":14,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.134700+0000","last_change":"2024-04-20T11:37:47.081340+0000","last_active":"2024-04-20T11:37:48.134700+0000","last_peered":"2024-04-20T11:37:48.134700+0000","last_clean":"2024-04-20T11:37:48.134700+0000","last_became_active":"2024-04-20T11:37:47.081021+0000","last_became_peered":"2024-04-20T11:37:47.081021+0000","last_unstale":"2024-04-20T11:37:48.134700+0000","last_undegraded":"2024-04-20T11:37:48.134700+0000","last_fullsized":"2024-04-20T11:37:48.134700+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_clean_scrub_stamp":"2024-04-20T11:37:46.019877+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:55:07.380625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.13","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.096781+0000","last_change":"2024-04-20T11:37:48.092651+0000","last_active":"2024-04-20T11:37:48.096781+0000","last_peered":"2024-04-20T11:37:48.096781+0000","last_clean":"2024-04-20T11:37:48.096781+0000","last_became_active":"2024-04-20T11:37:48.091945+0000","last_became_peered":"2024-04-20T11:37:48.091945+0000","last_unstale":"2024-04-20T11:37:48.096781+0000","last_undegraded":"2024-04-20T11:37:48.096781+0000","last_fullsized":"2024-04-20T11:37:48.096781+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:15:54.010830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":12,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.100961+0000","last_change":"2024-04-20T11:37:47.083406+0000","last_active":"2024-04-20T11:37:48.100961+0000","last_peered":"2024-04-20T11:37:48.100961+0000","last_clean":"2024-04-20T11:37:48.100961+0000","last_became_active":"2024-04-20T11:37:47.083070+0000","last_became_peered":"2024-04-20T11:37:47.083070+0000","last_unstale":"2024-04-20T11:37:48.100961+0000","last_undegraded":"2024-04-20T11:37:48.100961+0000","last_fullsized":"2024-04-20T11:37:48.100961+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_clean_scrub_stamp":"2024-04-20T11:37:46.019877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:31:33.749142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.10","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.122505+0000","last_change":"2024-04-20T11:37:48.116671+0000","last_active":"2024-04-20T11:37:48.122505+0000","last_peered":"2024-04-20T11:37:48.122505+0000","last_clean":"2024-04-20T11:37:48.122505+0000","last_became_active":"2024-04-20T11:37:48.113019+0000","last_became_peered":"2024-04-20T11:37:48.113019+0000","last_unstale":"2024-04-20T11:37:48.122505+0000","last_undegraded":"2024-04-20T11:37:48.122505+0000","last_fullsized":"2024-04-20T11:37:48.122505+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:49:59.397783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":12,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.084683+0000","last_change":"2024-04-20T11:37:47.078540+0000","last_active":"2024-04-20T11:37:48.084683+0000","last_peered":"2024-04-20T11:37:48.084683+0000","last_clean":"2024-04-20T11:37:48.084683+0000","last_became_active":"2024-04-20T11:37:47.077798+0000","last_became_peered":"2024-04-20T11:37:47.077798+0000","last_unstale":"2024-04-20T11:37:48.084683+0000","last_undegraded":"2024-04-20T11:37:48.084683+0000","last_fullsized":"2024-04-20T11:37:48.084683+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_clean_scrub_stamp":"2024-04-20T11:37:46.019877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:09:48.152605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.112300+0000","last_change":"2024-04-20T11:37:48.109165+0000","last_active":"2024-04-20T11:37:48.112300+0000","last_peered":"2024-04-20T11:37:48.112300+0000","last_clean":"2024-04-20T11:37:48.112300+0000","last_became_active":"2024-04-20T11:37:48.108827+0000","last_became_peered":"2024-04-20T11:37:48.108827+0000","last_unstale":"2024-04-20T11:37:48.112300+0000","last_undegraded":"2024-04-20T11:37:48.112300+0000","last_fullsized":"2024-04-20T11:37:48.112300+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:36:33.053650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.10","version":"166'8","reported_seq":20,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.137294+0000","last_change":"2024-04-20T11:37:47.078339+0000","last_active":"2024-04-20T11:37:48.137294+0000","last_peered":"2024-04-20T11:37:48.137294+0000","last_clean":"2024-04-20T11:37:48.137294+0000","last_became_active":"2024-04-20T11:37:47.078042+0000","last_became_peered":"2024-04-20T11:37:47.078042+0000","last_unstale":"2024-04-20T11:37:48.137294+0000","last_undegraded":"2024-04-20T11:37:48.137294+0000","last_fullsized":"2024-04-20T11:37:48.137294+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_clean_scrub_stamp":"2024-04-20T11:37:46.019877+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:42:47.506213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.e","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.109630+0000","last_change":"2024-04-20T11:37:48.105092+0000","last_active":"2024-04-20T11:37:48.109630+0000","last_peered":"2024-04-20T11:37:48.109630+0000","last_clean":"2024-04-20T11:37:48.109630+0000","last_became_active":"2024-04-20T11:37:48.103166+0000","last_became_peered":"2024-04-20T11:37:48.103166+0000","last_unstale":"2024-04-20T11:37:48.109630+0000","last_undegraded":"2024-04-20T11:37:48.109630+0000","last_fullsized":"2024-04-20T11:37:48.109630+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:38:56.071315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.f","version":"166'1","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.132870+0000","last_change":"2024-04-20T11:37:47.082946+0000","last_active":"2024-04-20T11:37:48.132870+0000","last_peered":"2024-04-20T11:37:48.132870+0000","last_clean":"2024-04-20T11:37:48.132870+0000","last_became_active":"2024-04-20T11:37:47.082275+0000","last_became_peered":"2024-04-20T11:37:47.082275+0000","last_unstale":"2024-04-20T11:37:48.132870+0000","last_undegraded":"2024-04-20T11:37:48.132870+0000","last_fullsized":"2024-04-20T11:37:48.132870+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_clean_scrub_stamp":"2024-04-20T11:37:46.019877+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:43:44.090262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.f","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.108871+0000","last_change":"2024-04-20T11:37:48.099908+0000","last_active":"2024-04-20T11:37:48.108871+0000","last_peered":"2024-04-20T11:37:48.108871+0000","last_clean":"2024-04-20T11:37:48.108871+0000","last_became_active":"2024-04-20T11:37:48.099598+0000","last_became_peered":"2024-04-20T11:37:48.099598+0000","last_unstale":"2024-04-20T11:37:48.108871+0000","last_undegraded":"2024-04-20T11:37:48.108871+0000","last_fullsized":"2024-04-20T11:37:48.108871+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:32:36.026412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.e","version":"166'2","reported_seq":14,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.139497+0000","last_change":"2024-04-20T11:37:47.078579+0000","last_active":"2024-04-20T11:37:48.139497+0000","last_peered":"2024-04-20T11:37:48.139497+0000","last_clean":"2024-04-20T11:37:48.139497+0000","last_became_active":"2024-04-20T11:37:47.078177+0000","last_became_peered":"2024-04-20T11:37:47.078177+0000","last_unstale":"2024-04-20T11:37:48.139497+0000","last_undegraded":"2024-04-20T11:37:48.139497+0000","last_fullsized":"2024-04-20T11:37:48.139497+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_clean_scrub_stamp":"2024-04-20T11:37:46.019877+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:42:47.506213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.099543+0000","last_change":"2024-04-20T11:37:48.092965+0000","last_active":"2024-04-20T11:37:48.099543+0000","last_peered":"2024-04-20T11:37:48.099543+0000","last_clean":"2024-04-20T11:37:48.099543+0000","last_became_active":"2024-04-20T11:37:48.091532+0000","last_became_peered":"2024-04-20T11:37:48.091532+0000","last_unstale":"2024-04-20T11:37:48.099543+0000","last_undegraded":"2024-04-20T11:37:48.099543+0000","last_fullsized":"2024-04-20T11:37:48.099543+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:38:56.071315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.d","version":"166'1","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.135579+0000","last_change":"2024-04-20T11:37:47.079127+0000","last_active":"2024-04-20T11:37:48.135579+0000","last_peered":"2024-04-20T11:37:48.135579+0000","last_clean":"2024-04-20T11:37:48.135579+0000","last_became_active":"2024-04-20T11:37:47.078846+0000","last_became_peered":"2024-04-20T11:37:47.078846+0000","last_unstale":"2024-04-20T11:37:48.135579+0000","last_undegraded":"2024-04-20T11:37:48.135579+0000","last_fullsized":"2024-04-20T11:37:48.135579+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_clean_scrub_stamp":"2024-04-20T11:37:46.019877+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:55:20.692965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.d","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.118673+0000","last_change":"2024-04-20T11:37:48.106804+0000","last_active":"2024-04-20T11:37:48.118673+0000","last_peered":"2024-04-20T11:37:48.118673+0000","last_clean":"2024-04-20T11:37:48.118673+0000","last_became_active":"2024-04-20T11:37:48.106320+0000","last_became_peered":"2024-04-20T11:37:48.106320+0000","last_unstale":"2024-04-20T11:37:48.118673+0000","last_undegraded":"2024-04-20T11:37:48.118673+0000","last_fullsized":"2024-04-20T11:37:48.118673+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:29:23.037410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":12,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.082232+0000","last_change":"2024-04-20T11:37:47.081678+0000","last_active":"2024-04-20T11:37:48.082232+0000","last_peered":"2024-04-20T11:37:48.082232+0000","last_clean":"2024-04-20T11:37:48.082232+0000","last_became_active":"2024-04-20T11:37:47.080366+0000","last_became_peered":"2024-04-20T11:37:47.080366+0000","last_unstale":"2024-04-20T11:37:48.082232+0000","last_undegraded":"2024-04-20T11:37:48.082232+0000","last_fullsized":"2024-04-20T11:37:48.082232+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_clean_scrub_stamp":"2024-04-20T11:37:46.019877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:32:35.010244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.119885+0000","last_change":"2024-04-20T11:37:48.113125+0000","last_active":"2024-04-20T11:37:48.119885+0000","last_peered":"2024-04-20T11:37:48.119885+0000","last_clean":"2024-04-20T11:37:48.119885+0000","last_became_active":"2024-04-20T11:37:48.112842+0000","last_became_peered":"2024-04-20T11:37:48.112842+0000","last_unstale":"2024-04-20T11:37:48.119885+0000","last_undegraded":"2024-04-20T11:37:48.119885+0000","last_fullsized":"2024-04-20T11:37:48.119885+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:56:04.982500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.b","version":"166'2","reported_seq":14,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.134377+0000","last_change":"2024-04-20T11:37:47.074786+0000","last_active":"2024-04-20T11:37:48.134377+0000","last_peered":"2024-04-20T11:37:48.134377+0000","last_clean":"2024-04-20T11:37:48.134377+0000","last_became_active":"2024-04-20T11:37:47.074387+0000","last_became_peered":"2024-04-20T11:37:47.074387+0000","last_unstale":"2024-04-20T11:37:48.134377+0000","last_undegraded":"2024-04-20T11:37:48.134377+0000","last_fullsized":"2024-04-20T11:37:48.134377+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_clean_scrub_stamp":"2024-04-20T11:37:46.019877+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:16:16.005318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.b","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.122926+0000","last_change":"2024-04-20T11:37:48.116958+0000","last_active":"2024-04-20T11:37:48.122926+0000","last_peered":"2024-04-20T11:37:48.122926+0000","last_clean":"2024-04-20T11:37:48.122926+0000","last_became_active":"2024-04-20T11:37:48.114749+0000","last_became_peered":"2024-04-20T11:37:48.114749+0000","last_unstale":"2024-04-20T11:37:48.122926+0000","last_undegraded":"2024-04-20T11:37:48.122926+0000","last_fullsized":"2024-04-20T11:37:48.122926+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:14:29.216857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":12,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.098293+0000","last_change":"2024-04-20T11:37:47.080682+0000","last_active":"2024-04-20T11:37:48.098293+0000","last_peered":"2024-04-20T11:37:48.098293+0000","last_clean":"2024-04-20T11:37:48.098293+0000","last_became_active":"2024-04-20T11:37:47.080348+0000","last_became_peered":"2024-04-20T11:37:47.080348+0000","last_unstale":"2024-04-20T11:37:48.098293+0000","last_undegraded":"2024-04-20T11:37:48.098293+0000","last_fullsized":"2024-04-20T11:37:48.098293+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_clean_scrub_stamp":"2024-04-20T11:37:46.019877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:29:22.021242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.112733+0000","last_change":"2024-04-20T11:37:48.110493+0000","last_active":"2024-04-20T11:37:48.112733+0000","last_peered":"2024-04-20T11:37:48.112733+0000","last_clean":"2024-04-20T11:37:48.112733+0000","last_became_active":"2024-04-20T11:37:48.110178+0000","last_became_peered":"2024-04-20T11:37:48.110178+0000","last_unstale":"2024-04-20T11:37:48.112733+0000","last_undegraded":"2024-04-20T11:37:48.112733+0000","last_fullsized":"2024-04-20T11:37:48.112733+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:56:16.721575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":12,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.098076+0000","last_change":"2024-04-20T11:37:47.083902+0000","last_active":"2024-04-20T11:37:48.098076+0000","last_peered":"2024-04-20T11:37:48.098076+0000","last_clean":"2024-04-20T11:37:48.098076+0000","last_became_active":"2024-04-20T11:37:47.083619+0000","last_became_peered":"2024-04-20T11:37:47.083619+0000","last_unstale":"2024-04-20T11:37:48.098076+0000","last_undegraded":"2024-04-20T11:37:48.098076+0000","last_fullsized":"2024-04-20T11:37:48.098076+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_clean_scrub_stamp":"2024-04-20T11:37:46.019877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:23:07.327346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.117696+0000","last_change":"2024-04-20T11:37:48.114675+0000","last_active":"2024-04-20T11:37:48.117696+0000","last_peered":"2024-04-20T11:37:48.117696+0000","last_clean":"2024-04-20T11:37:48.117696+0000","last_became_active":"2024-04-20T11:37:48.114212+0000","last_became_peered":"2024-04-20T11:37:48.114212+0000","last_unstale":"2024-04-20T11:37:48.117696+0000","last_undegraded":"2024-04-20T11:37:48.117696+0000","last_fullsized":"2024-04-20T11:37:48.117696+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:03:23.366996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":12,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.107470+0000","last_change":"2024-04-20T11:37:47.078603+0000","last_active":"2024-04-20T11:37:48.107470+0000","last_peered":"2024-04-20T11:37:48.107470+0000","last_clean":"2024-04-20T11:37:48.107470+0000","last_became_active":"2024-04-20T11:37:47.078301+0000","last_became_peered":"2024-04-20T11:37:47.078301+0000","last_unstale":"2024-04-20T11:37:48.107470+0000","last_undegraded":"2024-04-20T11:37:48.107470+0000","last_fullsized":"2024-04-20T11:37:48.107470+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_clean_scrub_stamp":"2024-04-20T11:37:46.019877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:18:14.091047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.121810+0000","last_change":"2024-04-20T11:37:48.119108+0000","last_active":"2024-04-20T11:37:48.121810+0000","last_peered":"2024-04-20T11:37:48.121810+0000","last_clean":"2024-04-20T11:37:48.121810+0000","last_became_active":"2024-04-20T11:37:48.116567+0000","last_became_peered":"2024-04-20T11:37:48.116567+0000","last_unstale":"2024-04-20T11:37:48.121810+0000","last_undegraded":"2024-04-20T11:37:48.121810+0000","last_fullsized":"2024-04-20T11:37:48.121810+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:11:56.044528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.7","version":"166'2","reported_seq":14,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.131150+0000","last_change":"2024-04-20T11:37:47.070199+0000","last_active":"2024-04-20T11:37:48.131150+0000","last_peered":"2024-04-20T11:37:48.131150+0000","last_clean":"2024-04-20T11:37:48.131150+0000","last_became_active":"2024-04-20T11:37:47.069218+0000","last_became_peered":"2024-04-20T11:37:47.069218+0000","last_unstale":"2024-04-20T11:37:48.131150+0000","last_undegraded":"2024-04-20T11:37:48.131150+0000","last_fullsized":"2024-04-20T11:37:48.131150+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_clean_scrub_stamp":"2024-04-20T11:37:46.019877+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:30:02.378451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.7","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.116933+0000","last_change":"2024-04-20T11:37:48.114848+0000","last_active":"2024-04-20T11:37:48.116933+0000","last_peered":"2024-04-20T11:37:48.116933+0000","last_clean":"2024-04-20T11:37:48.116933+0000","last_became_active":"2024-04-20T11:37:48.114533+0000","last_became_peered":"2024-04-20T11:37:48.114533+0000","last_unstale":"2024-04-20T11:37:48.116933+0000","last_undegraded":"2024-04-20T11:37:48.116933+0000","last_fullsized":"2024-04-20T11:37:48.116933+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:56:30.317215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.6","version":"0'0","reported_seq":12,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.102188+0000","last_change":"2024-04-20T11:37:47.083881+0000","last_active":"2024-04-20T11:37:48.102188+0000","last_peered":"2024-04-20T11:37:48.102188+0000","last_clean":"2024-04-20T11:37:48.102188+0000","last_became_active":"2024-04-20T11:37:47.082907+0000","last_became_peered":"2024-04-20T11:37:47.082907+0000","last_unstale":"2024-04-20T11:37:48.102188+0000","last_undegraded":"2024-04-20T11:37:48.102188+0000","last_fullsized":"2024-04-20T11:37:48.102188+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_clean_scrub_stamp":"2024-04-20T11:37:46.019877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:29:03.542441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.109344+0000","last_change":"2024-04-20T11:37:48.102559+0000","last_active":"2024-04-20T11:37:48.109344+0000","last_peered":"2024-04-20T11:37:48.109344+0000","last_clean":"2024-04-20T11:37:48.109344+0000","last_became_active":"2024-04-20T11:37:48.102261+0000","last_became_peered":"2024-04-20T11:37:48.102261+0000","last_unstale":"2024-04-20T11:37:48.109344+0000","last_undegraded":"2024-04-20T11:37:48.109344+0000","last_fullsized":"2024-04-20T11:37:48.109344+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:29:50.779164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":12,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.105296+0000","last_change":"2024-04-20T11:37:47.077943+0000","last_active":"2024-04-20T11:37:48.105296+0000","last_peered":"2024-04-20T11:37:48.105296+0000","last_clean":"2024-04-20T11:37:48.105296+0000","last_became_active":"2024-04-20T11:37:47.077677+0000","last_became_peered":"2024-04-20T11:37:47.077677+0000","last_unstale":"2024-04-20T11:37:48.105296+0000","last_undegraded":"2024-04-20T11:37:48.105296+0000","last_fullsized":"2024-04-20T11:37:48.105296+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_clean_scrub_stamp":"2024-04-20T11:37:46.019877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:10:42.658843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.117841+0000","last_change":"2024-04-20T11:37:48.115266+0000","last_active":"2024-04-20T11:37:48.117841+0000","last_peered":"2024-04-20T11:37:48.117841+0000","last_clean":"2024-04-20T11:37:48.117841+0000","last_became_active":"2024-04-20T11:37:48.114254+0000","last_became_peered":"2024-04-20T11:37:48.114254+0000","last_unstale":"2024-04-20T11:37:48.117841+0000","last_undegraded":"2024-04-20T11:37:48.117841+0000","last_fullsized":"2024-04-20T11:37:48.117841+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:05:03.866740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":12,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.082642+0000","last_change":"2024-04-20T11:37:47.064262+0000","last_active":"2024-04-20T11:37:48.082642+0000","last_peered":"2024-04-20T11:37:48.082642+0000","last_clean":"2024-04-20T11:37:48.082642+0000","last_became_active":"2024-04-20T11:37:47.063707+0000","last_became_peered":"2024-04-20T11:37:47.063707+0000","last_unstale":"2024-04-20T11:37:48.082642+0000","last_undegraded":"2024-04-20T11:37:48.082642+0000","last_fullsized":"2024-04-20T11:37:48.082642+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_clean_scrub_stamp":"2024-04-20T11:37:46.019877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:29:49.762996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.116147+0000","last_change":"2024-04-20T11:37:48.113860+0000","last_active":"2024-04-20T11:37:48.116147+0000","last_peered":"2024-04-20T11:37:48.116147+0000","last_clean":"2024-04-20T11:37:48.116147+0000","last_became_active":"2024-04-20T11:37:48.113551+0000","last_became_peered":"2024-04-20T11:37:48.113551+0000","last_unstale":"2024-04-20T11:37:48.116147+0000","last_undegraded":"2024-04-20T11:37:48.116147+0000","last_fullsized":"2024-04-20T11:37:48.116147+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:44:31.574362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":12,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.099785+0000","last_change":"2024-04-20T11:37:47.081822+0000","last_active":"2024-04-20T11:37:48.099785+0000","last_peered":"2024-04-20T11:37:48.099785+0000","last_clean":"2024-04-20T11:37:48.099785+0000","last_became_active":"2024-04-20T11:37:47.081527+0000","last_became_peered":"2024-04-20T11:37:47.081527+0000","last_unstale":"2024-04-20T11:37:48.099785+0000","last_undegraded":"2024-04-20T11:37:48.099785+0000","last_fullsized":"2024-04-20T11:37:48.099785+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_clean_scrub_stamp":"2024-04-20T11:37:46.019877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:19:32.710430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.122470+0000","last_change":"2024-04-20T11:37:48.121392+0000","last_active":"2024-04-20T11:37:48.122470+0000","last_peered":"2024-04-20T11:37:48.122470+0000","last_clean":"2024-04-20T11:37:48.122470+0000","last_became_active":"2024-04-20T11:37:48.121034+0000","last_became_peered":"2024-04-20T11:37:48.121034+0000","last_unstale":"2024-04-20T11:37:48.122470+0000","last_undegraded":"2024-04-20T11:37:48.122470+0000","last_fullsized":"2024-04-20T11:37:48.122470+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:43:45.106430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":12,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.107066+0000","last_change":"2024-04-20T11:37:47.077310+0000","last_active":"2024-04-20T11:37:48.107066+0000","last_peered":"2024-04-20T11:37:48.107066+0000","last_clean":"2024-04-20T11:37:48.107066+0000","last_became_active":"2024-04-20T11:37:47.076693+0000","last_became_peered":"2024-04-20T11:37:47.076693+0000","last_unstale":"2024-04-20T11:37:48.107066+0000","last_undegraded":"2024-04-20T11:37:48.107066+0000","last_fullsized":"2024-04-20T11:37:48.107066+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_clean_scrub_stamp":"2024-04-20T11:37:46.019877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:55:07.380625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.112363+0000","last_change":"2024-04-20T11:37:48.107949+0000","last_active":"2024-04-20T11:37:48.112363+0000","last_peered":"2024-04-20T11:37:48.112363+0000","last_clean":"2024-04-20T11:37:48.112363+0000","last_became_active":"2024-04-20T11:37:48.107636+0000","last_became_peered":"2024-04-20T11:37:48.107636+0000","last_unstale":"2024-04-20T11:37:48.112363+0000","last_undegraded":"2024-04-20T11:37:48.112363+0000","last_fullsized":"2024-04-20T11:37:48.112363+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:45:18.468249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":12,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.103717+0000","last_change":"2024-04-20T11:37:47.077826+0000","last_active":"2024-04-20T11:37:48.103717+0000","last_peered":"2024-04-20T11:37:48.103717+0000","last_clean":"2024-04-20T11:37:48.103717+0000","last_became_active":"2024-04-20T11:37:47.077577+0000","last_became_peered":"2024-04-20T11:37:47.077577+0000","last_unstale":"2024-04-20T11:37:48.103717+0000","last_undegraded":"2024-04-20T11:37:48.103717+0000","last_fullsized":"2024-04-20T11:37:48.103717+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_clean_scrub_stamp":"2024-04-20T11:37:46.019877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:33:32.171840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.112203+0000","last_change":"2024-04-20T11:37:48.110376+0000","last_active":"2024-04-20T11:37:48.112203+0000","last_peered":"2024-04-20T11:37:48.112203+0000","last_clean":"2024-04-20T11:37:48.112203+0000","last_became_active":"2024-04-20T11:37:48.109316+0000","last_became_peered":"2024-04-20T11:37:48.109316+0000","last_unstale":"2024-04-20T11:37:48.112203+0000","last_undegraded":"2024-04-20T11:37:48.112203+0000","last_fullsized":"2024-04-20T11:37:48.112203+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:45:18.468249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":12,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.100742+0000","last_change":"2024-04-20T11:37:47.079771+0000","last_active":"2024-04-20T11:37:48.100742+0000","last_peered":"2024-04-20T11:37:48.100742+0000","last_clean":"2024-04-20T11:37:48.100742+0000","last_became_active":"2024-04-20T11:37:47.079431+0000","last_became_peered":"2024-04-20T11:37:47.079431+0000","last_unstale":"2024-04-20T11:37:48.100742+0000","last_undegraded":"2024-04-20T11:37:48.100742+0000","last_fullsized":"2024-04-20T11:37:48.100742+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_clean_scrub_stamp":"2024-04-20T11:37:46.019877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:23:18.560236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1a","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.104878+0000","last_change":"2024-04-20T11:37:48.102464+0000","last_active":"2024-04-20T11:37:48.104878+0000","last_peered":"2024-04-20T11:37:48.104878+0000","last_clean":"2024-04-20T11:37:48.104878+0000","last_became_active":"2024-04-20T11:37:48.101872+0000","last_became_peered":"2024-04-20T11:37:48.101872+0000","last_unstale":"2024-04-20T11:37:48.104878+0000","last_undegraded":"2024-04-20T11:37:48.104878+0000","last_fullsized":"2024-04-20T11:37:48.104878+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:05:47.730738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1b","version":"166'1","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.136903+0000","last_change":"2024-04-20T11:37:47.082880+0000","last_active":"2024-04-20T11:37:48.136903+0000","last_peered":"2024-04-20T11:37:48.136903+0000","last_clean":"2024-04-20T11:37:48.136903+0000","last_became_active":"2024-04-20T11:37:47.082283+0000","last_became_peered":"2024-04-20T11:37:47.082283+0000","last_unstale":"2024-04-20T11:37:48.136903+0000","last_undegraded":"2024-04-20T11:37:48.136903+0000","last_fullsized":"2024-04-20T11:37:48.136903+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_clean_scrub_stamp":"2024-04-20T11:37:46.019877+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:37:23.009200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"72'128","reported_seq":471,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.103043+0000","last_change":"2024-04-20T11:26:48.661897+0000","last_active":"2024-04-20T11:37:48.103043+0000","last_peered":"2024-04-20T11:37:48.103043+0000","last_clean":"2024-04-20T11:37:48.103043+0000","last_became_active":"2024-04-20T11:26:48.656535+0000","last_became_peered":"2024-04-20T11:26:48.656535+0000","last_unstale":"2024-04-20T11:37:48.103043+0000","last_undegraded":"2024-04-20T11:37:48.103043+0000","last_fullsized":"2024-04-20T11:37:48.103043+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:47.543004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:47.543004+0000","last_clean_scrub_stamp":"2024-04-20T11:26:47.543004+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:58:35.527105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":27,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574986,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33828,"kb_used_data":2640,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338012,"statfs":{"total":96636764160,"available":96602124288,"internally_reserved":0,"allocated":2703360,"data_stored":1519271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574983,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33232,"kb_used_data":2052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338608,"statfs":{"total":96636764160,"available":96602734592,"internally_reserved":0,"allocated":2101248,"data_stored":927873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574985,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33236,"kb_used_data":2052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338604,"statfs":{"total":96636764160,"available":96602730496,"internally_reserved":0,"allocated":2101248,"data_stored":927873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574983,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33256,"kb_used_data":2068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338584,"statfs":{"total":96636764160,"available":96602710016,"internally_reserved":0,"allocated":2117632,"data_stored":929007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574984,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33248,"kb_used_data":2060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338592,"statfs":{"total":96636764160,"available":96602718208,"internally_reserved":0,"allocated":2109440,"data_stored":927473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574985,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33220,"kb_used_data":2036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338620,"statfs":{"total":96636764160,"available":96602746880,"internally_reserved":0,"allocated":2084864,"data_stored":920516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574983,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33220,"kb_used_data":2036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338620,"statfs":{"total":96636764160,"available":96602746880,"internally_reserved":0,"allocated":2084864,"data_stored":920516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574984,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37976,"kb_used_data":2632,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333864,"statfs":{"total":96636764160,"available":96597876736,"internally_reserved":0,"allocated":2695168,"data_stored":1517749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T11:37:52.519 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T11:37:52.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.753+0000 7f9159800640 1 Processor -- start 2024-04-20T11:37:52.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.753+0000 7f9159800640 1 -- start start 2024-04-20T11:37:52.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.755+0000 7f9159800640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f915406fa50 0x7f915415cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:52.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.755+0000 7f9159800640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91540b5770 con 0x7f915415b5e0 2024-04-20T11:37:52.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.755+0000 7f9159800640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f915406e060 con 0x7f915406eb40 2024-04-20T11:37:52.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.755+0000 7f9159800640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f915406de10 con 0x7f915406fa50 2024-04-20T11:37:52.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.755+0000 7f9152a00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f915415b5e0 0x7f91540b4c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:58486/0 (socket says 172.21.15.152:58486) 2024-04-20T11:37:52.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.755+0000 7f9152a00640 1 -- 172.21.15.152:0/649509104 learned_addr learned my addr 172.21.15.152:0/649509104 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:37:52.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.755+0000 7f9153400640 1 --2- 172.21.15.152:0/649509104 >> v2:172.21.15.175:3300/0 conn(0x7f915406fa50 0x7f915415cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:52.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.755+0000 7f9151600640 1 -- 172.21.15.152:0/649509104 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4051661148 0 0) 0x7f91540b5770 con 0x7f915415b5e0 2024-04-20T11:37:52.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.755+0000 7f9151600640 1 -- 172.21.15.152:0/649509104 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9138003520 con 0x7f915415b5e0 2024-04-20T11:37:52.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.756+0000 7f9151600640 1 -- 172.21.15.152:0/649509104 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3428421639 0 0) 0x7f9138003520 con 0x7f915415b5e0 2024-04-20T11:37:52.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.756+0000 7f9151600640 1 -- 172.21.15.152:0/649509104 >> v2:172.21.15.175:3300/0 conn(0x7f915406fa50 msgr2=0x7f915415cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:52.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.756+0000 7f9151600640 1 --2- 172.21.15.152:0/649509104 >> v2:172.21.15.175:3300/0 conn(0x7f915406fa50 0x7f915415cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:52.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.756+0000 7f9151600640 1 -- 172.21.15.152:0/649509104 >> v1:172.21.15.175:6789/0 conn(0x7f915406eb40 legacy=0x7f915406f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:52.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.756+0000 7f9151600640 1 -- 172.21.15.152:0/649509104 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f915406e340 con 0x7f915415b5e0 2024-04-20T11:37:52.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.756+0000 7f9151600640 1 -- 172.21.15.152:0/649509104 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f9144001670 con 0x7f915415b5e0 2024-04-20T11:37:52.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.756+0000 7f9151600640 1 -- 172.21.15.152:0/649509104 >> v1:172.21.15.152:6789/0 conn(0x7f915415b5e0 legacy=0x7f91540b4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:52.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.756+0000 7f9151600640 1 --2- 172.21.15.152:0/649509104 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9138003a20 0x7f9138003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:52.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.756+0000 7f9151600640 1 --2- 172.21.15.152:0/649509104 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9138004800 0x7f9138004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:52.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.756+0000 7f9151600640 1 --2- 172.21.15.152:0/649509104 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f91380051c0 0x7f913800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:52.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.756+0000 7f9151600640 1 -- 172.21.15.152:0/649509104 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f915406de10 con 0x7f9138003a20 2024-04-20T11:37:52.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.756+0000 7f9151600640 1 -- 172.21.15.152:0/649509104 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f913800c0f0 con 0x7f91380051c0 2024-04-20T11:37:52.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.756+0000 7f9151600640 1 -- 172.21.15.152:0/649509104 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f913800c2f0 con 0x7f9138004800 2024-04-20T11:37:52.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.756+0000 7f9152a00640 1 --2- 172.21.15.152:0/649509104 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9138003a20 0x7f9138003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:52.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.757+0000 7f9152a00640 1 -- 172.21.15.152:0/649509104 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9138004800 msgr2=0x7f9138004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:52.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.757+0000 7f9152000640 1 --2- 172.21.15.152:0/649509104 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9138004800 0x7f9138004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:52.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.757+0000 7f9153400640 1 --2- 172.21.15.152:0/649509104 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f91380051c0 0x7f913800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:52.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.757+0000 7f9152a00640 1 --2- 172.21.15.152:0/649509104 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9138004800 0x7f9138004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:52.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.757+0000 7f9152a00640 1 -- 172.21.15.152:0/649509104 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f91380051c0 msgr2=0x7f913800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:52.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.757+0000 7f9152a00640 1 --2- 172.21.15.152:0/649509104 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f91380051c0 0x7f913800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:52.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.757+0000 7f9152a00640 1 -- 172.21.15.152:0/649509104 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9138003520 con 0x7f9138003a20 2024-04-20T11:37:52.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.757+0000 7f9153400640 1 --2- 172.21.15.152:0/649509104 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f91380051c0 0x7f913800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:37:52.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.757+0000 7f9153400640 1 -- 172.21.15.152:0/649509104 reap_dead start 2024-04-20T11:37:52.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.758+0000 7f9152a00640 1 --2- 172.21.15.152:0/649509104 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9138003a20 0x7f9138003e60 secure :-1 s=READY pgs=2317 cs=0 l=1 rev1=1 crypto rx=0x7f914400c210 tx=0x7f9138005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93946bb01c0493cc server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:52.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.758+0000 7f9151600640 1 -- 172.21.15.152:0/649509104 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9144007cf0 con 0x7f9138003a20 2024-04-20T11:37:52.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.758+0000 7f9151600640 1 -- 172.21.15.152:0/649509104 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9144030070 con 0x7f9138003a20 2024-04-20T11:37:52.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.758+0000 7f9151600640 1 -- 172.21.15.152:0/649509104 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9144005d70 con 0x7f9138003a20 2024-04-20T11:37:52.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.759+0000 7f9159800640 1 -- 172.21.15.152:0/649509104 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9138003a20 msgr2=0x7f9138003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:52.762 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.759+0000 7f9159800640 1 --2- 172.21.15.152:0/649509104 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9138003a20 0x7f9138003e60 secure :-1 s=READY pgs=2317 cs=0 l=1 rev1=1 crypto rx=0x7f914400c210 tx=0x7f9138005830 comp rx=0 tx=0).stop 2024-04-20T11:37:52.762 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.759+0000 7f9159800640 1 -- 172.21.15.152:0/649509104 shutdown_connections 2024-04-20T11:37:52.762 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.759+0000 7f9159800640 1 --2- 172.21.15.152:0/649509104 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9138003a20 0x7f9138003e60 unknown :-1 s=CLOSED pgs=2317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:52.762 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.759+0000 7f9159800640 1 -- 172.21.15.152:0/649509104 >> 172.21.15.152:0/649509104 conn(0x7f9154067cf0 msgr2=0x7f915406b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:52.762 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.759+0000 7f9159800640 1 -- 172.21.15.152:0/649509104 shutdown_connections 2024-04-20T11:37:52.762 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.759+0000 7f9159800640 1 -- 172.21.15.152:0/649509104 wait complete. 2024-04-20T11:37:52.762 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.761+0000 7f9159800640 1 Processor -- start 2024-04-20T11:37:52.762 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.761+0000 7f9159800640 1 -- start start 2024-04-20T11:37:52.764 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.762+0000 7f9159800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9138003a20 0x7f9154147220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:52.764 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.762+0000 7f9159800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f915413fc20 0x7f9154146850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:52.764 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.762+0000 7f9159800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9154146d90 0x7f91540cf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:52.764 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.762+0000 7f9159800640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f915406f340 con 0x7f9138003a20 2024-04-20T11:37:52.764 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.762+0000 7f9159800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f91540b42d0 con 0x7f915413fc20 2024-04-20T11:37:52.764 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.762+0000 7f9159800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f915406f4c0 con 0x7f9154146d90 2024-04-20T11:37:52.764 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.762+0000 7f9152a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9138003a20 0x7f9154147220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:52.764 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.762+0000 7f9152a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9138003a20 0x7f9154147220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35160/0 (socket says 172.21.15.152:35160) 2024-04-20T11:37:52.764 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.762+0000 7f9152a00640 1 -- 172.21.15.152:0/516414256 learned_addr learned my addr 172.21.15.152:0/516414256 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:52.764 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.762+0000 7f9152a00640 1 -- 172.21.15.152:0/516414256 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9154146d90 msgr2=0x7f91540cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:52.764 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.762+0000 7f9152a00640 1 --2- 172.21.15.152:0/516414256 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9154146d90 0x7f91540cf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:52.764 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.762+0000 7f9152a00640 1 -- 172.21.15.152:0/516414256 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f915413fc20 msgr2=0x7f9154146850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:52.765 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.762+0000 7f9152a00640 1 --2- 172.21.15.152:0/516414256 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f915413fc20 0x7f9154146850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:52.765 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.762+0000 7f9152a00640 1 -- 172.21.15.152:0/516414256 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91541f0510 con 0x7f9138003a20 2024-04-20T11:37:52.765 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.763+0000 7f9152a00640 1 --2- 172.21.15.152:0/516414256 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9138003a20 0x7f9154147220 secure :-1 s=READY pgs=2318 cs=0 l=1 rev1=1 crypto rx=0x7f914400c210 tx=0x7f915406e230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:52.765 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.763+0000 7f914aa00640 1 -- 172.21.15.152:0/516414256 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9144007cf0 con 0x7f9138003a20 2024-04-20T11:37:52.765 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.763+0000 7f914aa00640 1 -- 172.21.15.152:0/516414256 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9144004bf0 con 0x7f9138003a20 2024-04-20T11:37:52.765 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.763+0000 7f914aa00640 1 -- 172.21.15.152:0/516414256 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9144001c30 con 0x7f9138003a20 2024-04-20T11:37:52.765 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.763+0000 7f9159800640 1 -- 172.21.15.152:0/516414256 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91541f07f0 con 0x7f9138003a20 2024-04-20T11:37:52.765 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.763+0000 7f9159800640 1 -- 172.21.15.152:0/516414256 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91541f0db0 con 0x7f9138003a20 2024-04-20T11:37:52.765 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.764+0000 7f9159800640 1 -- 172.21.15.152:0/516414256 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91540689f0 con 0x7f9138003a20 2024-04-20T11:37:52.766 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.765+0000 7f914aa00640 1 -- 172.21.15.152:0/516414256 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 26) ==== 91907+0+0 (secure 0 0 0) 0x7f9144048020 con 0x7f9138003a20 2024-04-20T11:37:52.767 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.766+0000 7f914aa00640 1 --2- 172.21.15.152:0/516414256 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f913406d2b0 0x7f913406f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:52.767 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.766+0000 7f914aa00640 1 -- 172.21.15.152:0/516414256 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 5709+0+0 (secure 0 0 0) 0x7f91440ab460 con 0x7f9138003a20 2024-04-20T11:37:52.767 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.767+0000 7f9152000640 1 --2- 172.21.15.152:0/516414256 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f913406d2b0 0x7f913406f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:52.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.768+0000 7f9152000640 1 --2- 172.21.15.152:0/516414256 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f913406d2b0 0x7f913406f770 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f9140000f60 tx=0x7f9140009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:52.771 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.771+0000 7f914aa00640 1 -- 172.21.15.152:0/516414256 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f914407b490 con 0x7f9138003a20 2024-04-20T11:37:52.963 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.962+0000 7f9159800640 1 -- 172.21.15.152:0/516414256 --> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f915406cd90 con 0x7f913406d2b0 2024-04-20T11:37:52.980 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.980+0000 7f914aa00640 1 -- 172.21.15.152:0/516414256 <== mgr.4109 v2:172.21.15.175:6836/4021250441 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252421 (secure 0 0 0) 0x7f915406cd90 con 0x7f913406d2b0 2024-04-20T11:37:52.984 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:37:52.985 INFO:teuthology.orchestra.run.smithi152.stderr:dumped all 2024-04-20T11:37:52.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.984+0000 7f9159800640 1 -- 172.21.15.152:0/516414256 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f913406d2b0 msgr2=0x7f913406f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:52.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.984+0000 7f9159800640 1 --2- 172.21.15.152:0/516414256 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f913406d2b0 0x7f913406f770 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f9140000f60 tx=0x7f9140009040 comp rx=0 tx=0).stop 2024-04-20T11:37:52.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.984+0000 7f9159800640 1 -- 172.21.15.152:0/516414256 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9138003a20 msgr2=0x7f9154147220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:52.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.984+0000 7f9159800640 1 --2- 172.21.15.152:0/516414256 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9138003a20 0x7f9154147220 secure :-1 s=READY pgs=2318 cs=0 l=1 rev1=1 crypto rx=0x7f914400c210 tx=0x7f915406e230 comp rx=0 tx=0).stop 2024-04-20T11:37:52.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.984+0000 7f9159800640 1 -- 172.21.15.152:0/516414256 shutdown_connections 2024-04-20T11:37:52.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.984+0000 7f9159800640 1 --2- 172.21.15.152:0/516414256 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f913406d2b0 0x7f913406f770 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:52.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.984+0000 7f9159800640 1 --2- 172.21.15.152:0/516414256 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9154146d90 0x7f91540cf2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:52.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.985+0000 7f9159800640 1 --2- 172.21.15.152:0/516414256 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f915413fc20 0x7f9154146850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:52.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.985+0000 7f9159800640 1 --2- 172.21.15.152:0/516414256 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9138003a20 0x7f9154147220 unknown :-1 s=CLOSED pgs=2318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:52.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.985+0000 7f9159800640 1 -- 172.21.15.152:0/516414256 >> 172.21.15.152:0/516414256 conn(0x7f9154067cf0 msgr2=0x7f91540b00c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:52.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.985+0000 7f9159800640 1 -- 172.21.15.152:0/516414256 shutdown_connections 2024-04-20T11:37:52.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:52.985+0000 7f9159800640 1 -- 172.21.15.152:0/516414256 wait complete. 2024-04-20T11:37:53.004 INFO:teuthology.orchestra.run.smithi152.stdout:{"pg_ready":true,"pg_map":{"version":264,"stamp":"2024-04-20T11:37:52.719450+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":223,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":166,"ondisk_log_size":166,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":118,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":271216,"kb_used_data":17576,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754703504,"statfs":{"total":773094113280,"available":772816388096,"internally_reserved":0,"allocated":17997824,"data_stored":8590278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12746,"internal_metadata":259575350},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002223"},"pg_stats":[{"pgid":"27.3f","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.112861+0000","last_change":"2024-04-20T11:37:48.110014+0000","last_active":"2024-04-20T11:37:48.112861+0000","last_peered":"2024-04-20T11:37:48.112861+0000","last_clean":"2024-04-20T11:37:48.112861+0000","last_became_active":"2024-04-20T11:37:48.109604+0000","last_became_peered":"2024-04-20T11:37:48.109604+0000","last_unstale":"2024-04-20T11:37:48.112861+0000","last_undegraded":"2024-04-20T11:37:48.112861+0000","last_fullsized":"2024-04-20T11:37:48.112861+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:58:28.810390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.3e","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.113219+0000","last_change":"2024-04-20T11:37:48.111899+0000","last_active":"2024-04-20T11:37:48.113219+0000","last_peered":"2024-04-20T11:37:48.113219+0000","last_clean":"2024-04-20T11:37:48.113219+0000","last_became_active":"2024-04-20T11:37:48.111494+0000","last_became_peered":"2024-04-20T11:37:48.111494+0000","last_unstale":"2024-04-20T11:37:48.113219+0000","last_undegraded":"2024-04-20T11:37:48.113219+0000","last_fullsized":"2024-04-20T11:37:48.113219+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:16:11.827279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.3d","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.117372+0000","last_change":"2024-04-20T11:37:48.113842+0000","last_active":"2024-04-20T11:37:48.117372+0000","last_peered":"2024-04-20T11:37:48.117372+0000","last_clean":"2024-04-20T11:37:48.117372+0000","last_became_active":"2024-04-20T11:37:48.111221+0000","last_became_peered":"2024-04-20T11:37:48.111221+0000","last_unstale":"2024-04-20T11:37:48.117372+0000","last_undegraded":"2024-04-20T11:37:48.117372+0000","last_fullsized":"2024-04-20T11:37:48.117372+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:16:11.827279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.3c","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.118106+0000","last_change":"2024-04-20T11:37:48.114560+0000","last_active":"2024-04-20T11:37:48.118106+0000","last_peered":"2024-04-20T11:37:48.118106+0000","last_clean":"2024-04-20T11:37:48.118106+0000","last_became_active":"2024-04-20T11:37:48.113882+0000","last_became_peered":"2024-04-20T11:37:48.113882+0000","last_unstale":"2024-04-20T11:37:48.118106+0000","last_undegraded":"2024-04-20T11:37:48.118106+0000","last_fullsized":"2024-04-20T11:37:48.118106+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:37:02.492137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3b","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.122803+0000","last_change":"2024-04-20T11:37:48.120358+0000","last_active":"2024-04-20T11:37:48.122803+0000","last_peered":"2024-04-20T11:37:48.122803+0000","last_clean":"2024-04-20T11:37:48.122803+0000","last_became_active":"2024-04-20T11:37:48.119698+0000","last_became_peered":"2024-04-20T11:37:48.119698+0000","last_unstale":"2024-04-20T11:37:48.122803+0000","last_undegraded":"2024-04-20T11:37:48.122803+0000","last_fullsized":"2024-04-20T11:37:48.122803+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:05:03.866740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3a","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.117265+0000","last_change":"2024-04-20T11:37:48.111495+0000","last_active":"2024-04-20T11:37:48.117265+0000","last_peered":"2024-04-20T11:37:48.117265+0000","last_clean":"2024-04-20T11:37:48.117265+0000","last_became_active":"2024-04-20T11:37:48.111156+0000","last_became_peered":"2024-04-20T11:37:48.111156+0000","last_unstale":"2024-04-20T11:37:48.117265+0000","last_undegraded":"2024-04-20T11:37:48.117265+0000","last_fullsized":"2024-04-20T11:37:48.117265+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:33:33.188008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.39","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.112757+0000","last_change":"2024-04-20T11:37:48.110685+0000","last_active":"2024-04-20T11:37:48.112757+0000","last_peered":"2024-04-20T11:37:48.112757+0000","last_clean":"2024-04-20T11:37:48.112757+0000","last_became_active":"2024-04-20T11:37:48.110239+0000","last_became_peered":"2024-04-20T11:37:48.110239+0000","last_unstale":"2024-04-20T11:37:48.112757+0000","last_undegraded":"2024-04-20T11:37:48.112757+0000","last_fullsized":"2024-04-20T11:37:48.112757+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:18:55.243430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.38","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.113216+0000","last_change":"2024-04-20T11:37:48.110355+0000","last_active":"2024-04-20T11:37:48.113216+0000","last_peered":"2024-04-20T11:37:48.113216+0000","last_clean":"2024-04-20T11:37:48.113216+0000","last_became_active":"2024-04-20T11:37:48.107599+0000","last_became_peered":"2024-04-20T11:37:48.107599+0000","last_unstale":"2024-04-20T11:37:48.113216+0000","last_undegraded":"2024-04-20T11:37:48.113216+0000","last_fullsized":"2024-04-20T11:37:48.113216+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:31:34.765310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.37","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.111581+0000","last_change":"2024-04-20T11:37:48.108719+0000","last_active":"2024-04-20T11:37:48.111581+0000","last_peered":"2024-04-20T11:37:48.111581+0000","last_clean":"2024-04-20T11:37:48.111581+0000","last_became_active":"2024-04-20T11:37:48.108425+0000","last_became_peered":"2024-04-20T11:37:48.108425+0000","last_unstale":"2024-04-20T11:37:48.111581+0000","last_undegraded":"2024-04-20T11:37:48.111581+0000","last_fullsized":"2024-04-20T11:37:48.111581+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:49:10.326407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.36","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.118530+0000","last_change":"2024-04-20T11:37:48.103567+0000","last_active":"2024-04-20T11:37:48.118530+0000","last_peered":"2024-04-20T11:37:48.118530+0000","last_clean":"2024-04-20T11:37:48.118530+0000","last_became_active":"2024-04-20T11:37:48.103296+0000","last_became_peered":"2024-04-20T11:37:48.103296+0000","last_unstale":"2024-04-20T11:37:48.118530+0000","last_undegraded":"2024-04-20T11:37:48.118530+0000","last_fullsized":"2024-04-20T11:37:48.118530+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:55:14.028926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.35","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.117353+0000","last_change":"2024-04-20T11:37:48.113400+0000","last_active":"2024-04-20T11:37:48.117353+0000","last_peered":"2024-04-20T11:37:48.117353+0000","last_clean":"2024-04-20T11:37:48.117353+0000","last_became_active":"2024-04-20T11:37:48.113136+0000","last_became_peered":"2024-04-20T11:37:48.113136+0000","last_unstale":"2024-04-20T11:37:48.117353+0000","last_undegraded":"2024-04-20T11:37:48.117353+0000","last_fullsized":"2024-04-20T11:37:48.117353+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:56:16.721575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.34","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.120878+0000","last_change":"2024-04-20T11:37:48.117085+0000","last_active":"2024-04-20T11:37:48.120878+0000","last_peered":"2024-04-20T11:37:48.120878+0000","last_clean":"2024-04-20T11:37:48.120878+0000","last_became_active":"2024-04-20T11:37:48.116688+0000","last_became_peered":"2024-04-20T11:37:48.116688+0000","last_unstale":"2024-04-20T11:37:48.120878+0000","last_undegraded":"2024-04-20T11:37:48.120878+0000","last_fullsized":"2024-04-20T11:37:48.120878+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:51:41.363330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.33","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.113104+0000","last_change":"2024-04-20T11:37:48.107803+0000","last_active":"2024-04-20T11:37:48.113104+0000","last_peered":"2024-04-20T11:37:48.113104+0000","last_clean":"2024-04-20T11:37:48.113104+0000","last_became_active":"2024-04-20T11:37:48.107441+0000","last_became_peered":"2024-04-20T11:37:48.107441+0000","last_unstale":"2024-04-20T11:37:48.113104+0000","last_undegraded":"2024-04-20T11:37:48.113104+0000","last_fullsized":"2024-04-20T11:37:48.113104+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:24:53.390054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.32","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.117828+0000","last_change":"2024-04-20T11:37:48.116007+0000","last_active":"2024-04-20T11:37:48.117828+0000","last_peered":"2024-04-20T11:37:48.117828+0000","last_clean":"2024-04-20T11:37:48.117828+0000","last_became_active":"2024-04-20T11:37:48.114524+0000","last_became_peered":"2024-04-20T11:37:48.114524+0000","last_unstale":"2024-04-20T11:37:48.117828+0000","last_undegraded":"2024-04-20T11:37:48.117828+0000","last_fullsized":"2024-04-20T11:37:48.117828+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:37:42.226969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.31","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.105699+0000","last_change":"2024-04-20T11:37:48.102631+0000","last_active":"2024-04-20T11:37:48.105699+0000","last_peered":"2024-04-20T11:37:48.105699+0000","last_clean":"2024-04-20T11:37:48.105699+0000","last_became_active":"2024-04-20T11:37:48.101097+0000","last_became_peered":"2024-04-20T11:37:48.101097+0000","last_unstale":"2024-04-20T11:37:48.105699+0000","last_undegraded":"2024-04-20T11:37:48.105699+0000","last_fullsized":"2024-04-20T11:37:48.105699+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:56:04.982500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.30","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.118913+0000","last_change":"2024-04-20T11:37:48.108421+0000","last_active":"2024-04-20T11:37:48.118913+0000","last_peered":"2024-04-20T11:37:48.118913+0000","last_clean":"2024-04-20T11:37:48.118913+0000","last_became_active":"2024-04-20T11:37:48.108116+0000","last_became_peered":"2024-04-20T11:37:48.108116+0000","last_unstale":"2024-04-20T11:37:48.118913+0000","last_undegraded":"2024-04-20T11:37:48.118913+0000","last_fullsized":"2024-04-20T11:37:48.118913+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:05:47.730738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2f","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.112839+0000","last_change":"2024-04-20T11:37:48.109128+0000","last_active":"2024-04-20T11:37:48.112839+0000","last_peered":"2024-04-20T11:37:48.112839+0000","last_clean":"2024-04-20T11:37:48.112839+0000","last_became_active":"2024-04-20T11:37:48.108798+0000","last_became_peered":"2024-04-20T11:37:48.108798+0000","last_unstale":"2024-04-20T11:37:48.112839+0000","last_undegraded":"2024-04-20T11:37:48.112839+0000","last_fullsized":"2024-04-20T11:37:48.112839+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:23:09.091466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.2e","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.118557+0000","last_change":"2024-04-20T11:37:48.105543+0000","last_active":"2024-04-20T11:37:48.118557+0000","last_peered":"2024-04-20T11:37:48.118557+0000","last_clean":"2024-04-20T11:37:48.118557+0000","last_became_active":"2024-04-20T11:37:48.100271+0000","last_became_peered":"2024-04-20T11:37:48.100271+0000","last_unstale":"2024-04-20T11:37:48.118557+0000","last_undegraded":"2024-04-20T11:37:48.118557+0000","last_fullsized":"2024-04-20T11:37:48.118557+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:59:49.660352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2d","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.118652+0000","last_change":"2024-04-20T11:37:48.106550+0000","last_active":"2024-04-20T11:37:48.118652+0000","last_peered":"2024-04-20T11:37:48.118652+0000","last_clean":"2024-04-20T11:37:48.118652+0000","last_became_active":"2024-04-20T11:37:48.105926+0000","last_became_peered":"2024-04-20T11:37:48.105926+0000","last_unstale":"2024-04-20T11:37:48.118652+0000","last_undegraded":"2024-04-20T11:37:48.118652+0000","last_fullsized":"2024-04-20T11:37:48.118652+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:23:08.343514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2c","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.118209+0000","last_change":"2024-04-20T11:37:48.112417+0000","last_active":"2024-04-20T11:37:48.118209+0000","last_peered":"2024-04-20T11:37:48.118209+0000","last_clean":"2024-04-20T11:37:48.118209+0000","last_became_active":"2024-04-20T11:37:48.112060+0000","last_became_peered":"2024-04-20T11:37:48.112060+0000","last_unstale":"2024-04-20T11:37:48.118209+0000","last_undegraded":"2024-04-20T11:37:48.118209+0000","last_fullsized":"2024-04-20T11:37:48.118209+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:08:16.946122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.2b","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.116690+0000","last_change":"2024-04-20T11:37:48.109694+0000","last_active":"2024-04-20T11:37:48.116690+0000","last_peered":"2024-04-20T11:37:48.116690+0000","last_clean":"2024-04-20T11:37:48.116690+0000","last_became_active":"2024-04-20T11:37:48.109417+0000","last_became_peered":"2024-04-20T11:37:48.109417+0000","last_unstale":"2024-04-20T11:37:48.116690+0000","last_undegraded":"2024-04-20T11:37:48.116690+0000","last_fullsized":"2024-04-20T11:37:48.116690+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:25:24.946294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2a","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.122191+0000","last_change":"2024-04-20T11:37:48.121180+0000","last_active":"2024-04-20T11:37:48.122191+0000","last_peered":"2024-04-20T11:37:48.122191+0000","last_clean":"2024-04-20T11:37:48.122191+0000","last_became_active":"2024-04-20T11:37:48.120321+0000","last_became_peered":"2024-04-20T11:37:48.120321+0000","last_unstale":"2024-04-20T11:37:48.122191+0000","last_undegraded":"2024-04-20T11:37:48.122191+0000","last_fullsized":"2024-04-20T11:37:48.122191+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:31:47.943569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.29","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.111394+0000","last_change":"2024-04-20T11:37:48.108383+0000","last_active":"2024-04-20T11:37:48.111394+0000","last_peered":"2024-04-20T11:37:48.111394+0000","last_clean":"2024-04-20T11:37:48.111394+0000","last_became_active":"2024-04-20T11:37:48.107974+0000","last_became_peered":"2024-04-20T11:37:48.107974+0000","last_unstale":"2024-04-20T11:37:48.111394+0000","last_undegraded":"2024-04-20T11:37:48.111394+0000","last_fullsized":"2024-04-20T11:37:48.111394+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:08:16.946122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.28","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.117571+0000","last_change":"2024-04-20T11:37:48.111530+0000","last_active":"2024-04-20T11:37:48.117571+0000","last_peered":"2024-04-20T11:37:48.117571+0000","last_clean":"2024-04-20T11:37:48.117571+0000","last_became_active":"2024-04-20T11:37:48.111259+0000","last_became_peered":"2024-04-20T11:37:48.111259+0000","last_unstale":"2024-04-20T11:37:48.117571+0000","last_undegraded":"2024-04-20T11:37:48.117571+0000","last_fullsized":"2024-04-20T11:37:48.117571+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:24:54.986907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.27","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.111560+0000","last_change":"2024-04-20T11:37:48.104944+0000","last_active":"2024-04-20T11:37:48.111560+0000","last_peered":"2024-04-20T11:37:48.111560+0000","last_clean":"2024-04-20T11:37:48.111560+0000","last_became_active":"2024-04-20T11:37:48.099879+0000","last_became_peered":"2024-04-20T11:37:48.099879+0000","last_unstale":"2024-04-20T11:37:48.111560+0000","last_undegraded":"2024-04-20T11:37:48.111560+0000","last_fullsized":"2024-04-20T11:37:48.111560+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:30:03.394619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.26","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.113627+0000","last_change":"2024-04-20T11:37:48.110830+0000","last_active":"2024-04-20T11:37:48.113627+0000","last_peered":"2024-04-20T11:37:48.113627+0000","last_clean":"2024-04-20T11:37:48.113627+0000","last_became_active":"2024-04-20T11:37:48.110535+0000","last_became_peered":"2024-04-20T11:37:48.110535+0000","last_unstale":"2024-04-20T11:37:48.113627+0000","last_undegraded":"2024-04-20T11:37:48.113627+0000","last_fullsized":"2024-04-20T11:37:48.113627+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:29:13.486540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.25","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.120994+0000","last_change":"2024-04-20T11:37:48.118234+0000","last_active":"2024-04-20T11:37:48.120994+0000","last_peered":"2024-04-20T11:37:48.120994+0000","last_clean":"2024-04-20T11:37:48.120994+0000","last_became_active":"2024-04-20T11:37:48.117890+0000","last_became_peered":"2024-04-20T11:37:48.117890+0000","last_unstale":"2024-04-20T11:37:48.120994+0000","last_undegraded":"2024-04-20T11:37:48.120994+0000","last_fullsized":"2024-04-20T11:37:48.120994+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:18:27.316456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.24","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.109493+0000","last_change":"2024-04-20T11:37:48.106896+0000","last_active":"2024-04-20T11:37:48.109493+0000","last_peered":"2024-04-20T11:37:48.109493+0000","last_clean":"2024-04-20T11:37:48.109493+0000","last_became_active":"2024-04-20T11:37:48.106600+0000","last_became_peered":"2024-04-20T11:37:48.106600+0000","last_unstale":"2024-04-20T11:37:48.109493+0000","last_undegraded":"2024-04-20T11:37:48.109493+0000","last_fullsized":"2024-04-20T11:37:48.109493+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:49:10.326407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.23","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.119856+0000","last_change":"2024-04-20T11:37:48.116816+0000","last_active":"2024-04-20T11:37:48.119856+0000","last_peered":"2024-04-20T11:37:48.119856+0000","last_clean":"2024-04-20T11:37:48.119856+0000","last_became_active":"2024-04-20T11:37:48.116504+0000","last_became_peered":"2024-04-20T11:37:48.116504+0000","last_unstale":"2024-04-20T11:37:48.119856+0000","last_undegraded":"2024-04-20T11:37:48.119856+0000","last_fullsized":"2024-04-20T11:37:48.119856+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:17:07.054161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.22","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.120067+0000","last_change":"2024-04-20T11:37:48.117391+0000","last_active":"2024-04-20T11:37:48.120067+0000","last_peered":"2024-04-20T11:37:48.120067+0000","last_clean":"2024-04-20T11:37:48.120067+0000","last_became_active":"2024-04-20T11:37:48.113649+0000","last_became_peered":"2024-04-20T11:37:48.113649+0000","last_unstale":"2024-04-20T11:37:48.120067+0000","last_undegraded":"2024-04-20T11:37:48.120067+0000","last_fullsized":"2024-04-20T11:37:48.120067+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:49:59.397783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.21","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.105927+0000","last_change":"2024-04-20T11:37:48.098202+0000","last_active":"2024-04-20T11:37:48.105927+0000","last_peered":"2024-04-20T11:37:48.105927+0000","last_clean":"2024-04-20T11:37:48.105927+0000","last_became_active":"2024-04-20T11:37:48.097719+0000","last_became_peered":"2024-04-20T11:37:48.097719+0000","last_unstale":"2024-04-20T11:37:48.105927+0000","last_undegraded":"2024-04-20T11:37:48.105927+0000","last_fullsized":"2024-04-20T11:37:48.105927+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:45:55.333086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.20","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.120168+0000","last_change":"2024-04-20T11:37:48.119020+0000","last_active":"2024-04-20T11:37:48.120168+0000","last_peered":"2024-04-20T11:37:48.120168+0000","last_clean":"2024-04-20T11:37:48.120168+0000","last_became_active":"2024-04-20T11:37:48.117155+0000","last_became_peered":"2024-04-20T11:37:48.117155+0000","last_unstale":"2024-04-20T11:37:48.120168+0000","last_undegraded":"2024-04-20T11:37:48.120168+0000","last_fullsized":"2024-04-20T11:37:48.120168+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:38:32.481887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.1e","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.103374+0000","last_change":"2024-04-20T11:37:48.096757+0000","last_active":"2024-04-20T11:37:48.103374+0000","last_peered":"2024-04-20T11:37:48.103374+0000","last_clean":"2024-04-20T11:37:48.103374+0000","last_became_active":"2024-04-20T11:37:48.092696+0000","last_became_peered":"2024-04-20T11:37:48.092696+0000","last_unstale":"2024-04-20T11:37:48.103374+0000","last_undegraded":"2024-04-20T11:37:48.103374+0000","last_fullsized":"2024-04-20T11:37:48.103374+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:31:20.962446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.1f","version":"166'4","reported_seq":16,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.139951+0000","last_change":"2024-04-20T11:37:47.078653+0000","last_active":"2024-04-20T11:37:48.139951+0000","last_peered":"2024-04-20T11:37:48.139951+0000","last_clean":"2024-04-20T11:37:48.139951+0000","last_became_active":"2024-04-20T11:37:47.078374+0000","last_became_peered":"2024-04-20T11:37:47.078374+0000","last_unstale":"2024-04-20T11:37:48.139951+0000","last_undegraded":"2024-04-20T11:37:48.139951+0000","last_fullsized":"2024-04-20T11:37:48.139951+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_clean_scrub_stamp":"2024-04-20T11:37:46.019877+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:46:38.860806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1f","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.120816+0000","last_change":"2024-04-20T11:37:48.119678+0000","last_active":"2024-04-20T11:37:48.120816+0000","last_peered":"2024-04-20T11:37:48.120816+0000","last_clean":"2024-04-20T11:37:48.120816+0000","last_became_active":"2024-04-20T11:37:48.119306+0000","last_became_peered":"2024-04-20T11:37:48.119306+0000","last_unstale":"2024-04-20T11:37:48.120816+0000","last_undegraded":"2024-04-20T11:37:48.120816+0000","last_fullsized":"2024-04-20T11:37:48.120816+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:46:30.542546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.1e","version":"166'1","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.139549+0000","last_change":"2024-04-20T11:37:47.079218+0000","last_active":"2024-04-20T11:37:48.139549+0000","last_peered":"2024-04-20T11:37:48.139549+0000","last_clean":"2024-04-20T11:37:48.139549+0000","last_became_active":"2024-04-20T11:37:47.078908+0000","last_became_peered":"2024-04-20T11:37:47.078908+0000","last_unstale":"2024-04-20T11:37:48.139549+0000","last_undegraded":"2024-04-20T11:37:48.139549+0000","last_fullsized":"2024-04-20T11:37:48.139549+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_clean_scrub_stamp":"2024-04-20T11:37:46.019877+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:17:29.048669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1c","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.115559+0000","last_change":"2024-04-20T11:37:48.112060+0000","last_active":"2024-04-20T11:37:48.115559+0000","last_peered":"2024-04-20T11:37:48.115559+0000","last_clean":"2024-04-20T11:37:48.115559+0000","last_became_active":"2024-04-20T11:37:48.111752+0000","last_became_peered":"2024-04-20T11:37:48.111752+0000","last_unstale":"2024-04-20T11:37:48.115559+0000","last_undegraded":"2024-04-20T11:37:48.115559+0000","last_fullsized":"2024-04-20T11:37:48.115559+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:28:58.720468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1d","version":"166'2","reported_seq":14,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.132462+0000","last_change":"2024-04-20T11:37:47.082172+0000","last_active":"2024-04-20T11:37:48.132462+0000","last_peered":"2024-04-20T11:37:48.132462+0000","last_clean":"2024-04-20T11:37:48.132462+0000","last_became_active":"2024-04-20T11:37:47.081688+0000","last_became_peered":"2024-04-20T11:37:47.081688+0000","last_unstale":"2024-04-20T11:37:48.132462+0000","last_undegraded":"2024-04-20T11:37:48.132462+0000","last_fullsized":"2024-04-20T11:37:48.132462+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_clean_scrub_stamp":"2024-04-20T11:37:46.019877+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:31:19.946278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1d","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.107141+0000","last_change":"2024-04-20T11:37:48.098935+0000","last_active":"2024-04-20T11:37:48.107141+0000","last_peered":"2024-04-20T11:37:48.107141+0000","last_clean":"2024-04-20T11:37:48.107141+0000","last_became_active":"2024-04-20T11:37:48.098623+0000","last_became_peered":"2024-04-20T11:37:48.098623+0000","last_unstale":"2024-04-20T11:37:48.107141+0000","last_undegraded":"2024-04-20T11:37:48.107141+0000","last_fullsized":"2024-04-20T11:37:48.107141+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:42:28.063123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1c","version":"166'2","reported_seq":14,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.137128+0000","last_change":"2024-04-20T11:37:47.088265+0000","last_active":"2024-04-20T11:37:48.137128+0000","last_peered":"2024-04-20T11:37:48.137128+0000","last_clean":"2024-04-20T11:37:48.137128+0000","last_became_active":"2024-04-20T11:37:47.087899+0000","last_became_peered":"2024-04-20T11:37:47.087899+0000","last_unstale":"2024-04-20T11:37:48.137128+0000","last_undegraded":"2024-04-20T11:37:48.137128+0000","last_fullsized":"2024-04-20T11:37:48.137128+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_clean_scrub_stamp":"2024-04-20T11:37:46.019877+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:12:20.796103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1b","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.117593+0000","last_change":"2024-04-20T11:37:48.112994+0000","last_active":"2024-04-20T11:37:48.117593+0000","last_peered":"2024-04-20T11:37:48.117593+0000","last_clean":"2024-04-20T11:37:48.117593+0000","last_became_active":"2024-04-20T11:37:48.112719+0000","last_became_peered":"2024-04-20T11:37:48.112719+0000","last_unstale":"2024-04-20T11:37:48.117593+0000","last_undegraded":"2024-04-20T11:37:48.117593+0000","last_fullsized":"2024-04-20T11:37:48.117593+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:09:49.168773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1a","version":"166'1","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.132922+0000","last_change":"2024-04-20T11:37:47.077871+0000","last_active":"2024-04-20T11:37:48.132922+0000","last_peered":"2024-04-20T11:37:48.132922+0000","last_clean":"2024-04-20T11:37:48.132922+0000","last_became_active":"2024-04-20T11:37:47.077568+0000","last_became_peered":"2024-04-20T11:37:47.077568+0000","last_unstale":"2024-04-20T11:37:48.132922+0000","last_undegraded":"2024-04-20T11:37:48.132922+0000","last_fullsized":"2024-04-20T11:37:48.132922+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_clean_scrub_stamp":"2024-04-20T11:37:46.019877+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:13:04.063611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.118574+0000","last_change":"2024-04-20T11:37:48.115194+0000","last_active":"2024-04-20T11:37:48.118574+0000","last_peered":"2024-04-20T11:37:48.118574+0000","last_clean":"2024-04-20T11:37:48.118574+0000","last_became_active":"2024-04-20T11:37:48.114053+0000","last_became_peered":"2024-04-20T11:37:48.114053+0000","last_unstale":"2024-04-20T11:37:48.118574+0000","last_undegraded":"2024-04-20T11:37:48.118574+0000","last_fullsized":"2024-04-20T11:37:48.118574+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:03:23.366996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":12,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.104088+0000","last_change":"2024-04-20T11:37:47.077239+0000","last_active":"2024-04-20T11:37:48.104088+0000","last_peered":"2024-04-20T11:37:48.104088+0000","last_clean":"2024-04-20T11:37:48.104088+0000","last_became_active":"2024-04-20T11:37:47.076919+0000","last_became_peered":"2024-04-20T11:37:47.076919+0000","last_unstale":"2024-04-20T11:37:48.104088+0000","last_undegraded":"2024-04-20T11:37:48.104088+0000","last_fullsized":"2024-04-20T11:37:48.104088+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_clean_scrub_stamp":"2024-04-20T11:37:46.019877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:18:26.300288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.116795+0000","last_change":"2024-04-20T11:37:48.108749+0000","last_active":"2024-04-20T11:37:48.116795+0000","last_peered":"2024-04-20T11:37:48.116795+0000","last_clean":"2024-04-20T11:37:48.116795+0000","last_became_active":"2024-04-20T11:37:48.108426+0000","last_became_peered":"2024-04-20T11:37:48.108426+0000","last_unstale":"2024-04-20T11:37:48.116795+0000","last_undegraded":"2024-04-20T11:37:48.116795+0000","last_fullsized":"2024-04-20T11:37:48.116795+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:32:24.547538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":12,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.094892+0000","last_change":"2024-04-20T11:37:47.081824+0000","last_active":"2024-04-20T11:37:48.094892+0000","last_peered":"2024-04-20T11:37:48.094892+0000","last_clean":"2024-04-20T11:37:48.094892+0000","last_became_active":"2024-04-20T11:37:47.081004+0000","last_became_peered":"2024-04-20T11:37:47.081004+0000","last_unstale":"2024-04-20T11:37:48.094892+0000","last_undegraded":"2024-04-20T11:37:48.094892+0000","last_fullsized":"2024-04-20T11:37:48.094892+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_clean_scrub_stamp":"2024-04-20T11:37:46.019877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:18:14.091047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.118048+0000","last_change":"2024-04-20T11:37:48.113664+0000","last_active":"2024-04-20T11:37:48.118048+0000","last_peered":"2024-04-20T11:37:48.118048+0000","last_clean":"2024-04-20T11:37:48.118048+0000","last_became_active":"2024-04-20T11:37:48.113388+0000","last_became_peered":"2024-04-20T11:37:48.113388+0000","last_unstale":"2024-04-20T11:37:48.118048+0000","last_undegraded":"2024-04-20T11:37:48.118048+0000","last_fullsized":"2024-04-20T11:37:48.118048+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:16:17.021486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":12,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.104881+0000","last_change":"2024-04-20T11:37:47.079085+0000","last_active":"2024-04-20T11:37:48.104881+0000","last_peered":"2024-04-20T11:37:48.104881+0000","last_clean":"2024-04-20T11:37:48.104881+0000","last_became_active":"2024-04-20T11:37:47.078764+0000","last_became_peered":"2024-04-20T11:37:47.078764+0000","last_unstale":"2024-04-20T11:37:48.104881+0000","last_undegraded":"2024-04-20T11:37:48.104881+0000","last_fullsized":"2024-04-20T11:37:48.104881+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_clean_scrub_stamp":"2024-04-20T11:37:46.019877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:11:55.028361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.123282+0000","last_change":"2024-04-20T11:37:48.118132+0000","last_active":"2024-04-20T11:37:48.123282+0000","last_peered":"2024-04-20T11:37:48.123282+0000","last_clean":"2024-04-20T11:37:48.123282+0000","last_became_active":"2024-04-20T11:37:48.117030+0000","last_became_peered":"2024-04-20T11:37:48.117030+0000","last_unstale":"2024-04-20T11:37:48.123282+0000","last_undegraded":"2024-04-20T11:37:48.123282+0000","last_fullsized":"2024-04-20T11:37:48.123282+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:07:37.215053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":12,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.106328+0000","last_change":"2024-04-20T11:37:47.076623+0000","last_active":"2024-04-20T11:37:48.106328+0000","last_peered":"2024-04-20T11:37:48.106328+0000","last_clean":"2024-04-20T11:37:48.106328+0000","last_became_active":"2024-04-20T11:37:47.075990+0000","last_became_peered":"2024-04-20T11:37:47.075990+0000","last_unstale":"2024-04-20T11:37:48.106328+0000","last_undegraded":"2024-04-20T11:37:48.106328+0000","last_fullsized":"2024-04-20T11:37:48.106328+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_clean_scrub_stamp":"2024-04-20T11:37:46.019877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:30:26.452766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.14","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.118128+0000","last_change":"2024-04-20T11:37:48.114787+0000","last_active":"2024-04-20T11:37:48.118128+0000","last_peered":"2024-04-20T11:37:48.118128+0000","last_clean":"2024-04-20T11:37:48.118128+0000","last_became_active":"2024-04-20T11:37:48.114272+0000","last_became_peered":"2024-04-20T11:37:48.114272+0000","last_unstale":"2024-04-20T11:37:48.118128+0000","last_undegraded":"2024-04-20T11:37:48.118128+0000","last_fullsized":"2024-04-20T11:37:48.118128+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:55:21.709133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.15","version":"166'3","reported_seq":15,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.134358+0000","last_change":"2024-04-20T11:37:47.073262+0000","last_active":"2024-04-20T11:37:48.134358+0000","last_peered":"2024-04-20T11:37:48.134358+0000","last_clean":"2024-04-20T11:37:48.134358+0000","last_became_active":"2024-04-20T11:37:47.072961+0000","last_became_peered":"2024-04-20T11:37:47.072961+0000","last_unstale":"2024-04-20T11:37:48.134358+0000","last_undegraded":"2024-04-20T11:37:48.134358+0000","last_fullsized":"2024-04-20T11:37:48.134358+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_clean_scrub_stamp":"2024-04-20T11:37:46.019877+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:37:01.475969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.15","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.107560+0000","last_change":"2024-04-20T11:37:48.105449+0000","last_active":"2024-04-20T11:37:48.107560+0000","last_peered":"2024-04-20T11:37:48.107560+0000","last_clean":"2024-04-20T11:37:48.107560+0000","last_became_active":"2024-04-20T11:37:48.104338+0000","last_became_peered":"2024-04-20T11:37:48.104338+0000","last_unstale":"2024-04-20T11:37:48.107560+0000","last_undegraded":"2024-04-20T11:37:48.107560+0000","last_fullsized":"2024-04-20T11:37:48.107560+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:04:41.081124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.14","version":"166'6","reported_seq":18,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.140046+0000","last_change":"2024-04-20T11:37:47.082031+0000","last_active":"2024-04-20T11:37:48.140046+0000","last_peered":"2024-04-20T11:37:48.140046+0000","last_clean":"2024-04-20T11:37:48.140046+0000","last_became_active":"2024-04-20T11:37:47.081736+0000","last_became_peered":"2024-04-20T11:37:47.081736+0000","last_unstale":"2024-04-20T11:37:48.140046+0000","last_undegraded":"2024-04-20T11:37:48.140046+0000","last_fullsized":"2024-04-20T11:37:48.140046+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_clean_scrub_stamp":"2024-04-20T11:37:46.019877+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:37:06.748530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.117287+0000","last_change":"2024-04-20T11:37:48.112035+0000","last_active":"2024-04-20T11:37:48.117287+0000","last_peered":"2024-04-20T11:37:48.117287+0000","last_clean":"2024-04-20T11:37:48.117287+0000","last_became_active":"2024-04-20T11:37:48.111552+0000","last_became_peered":"2024-04-20T11:37:48.111552+0000","last_unstale":"2024-04-20T11:37:48.117287+0000","last_undegraded":"2024-04-20T11:37:48.117287+0000","last_fullsized":"2024-04-20T11:37:48.117287+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:30:27.468934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.13","version":"166'2","reported_seq":14,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.134700+0000","last_change":"2024-04-20T11:37:47.081340+0000","last_active":"2024-04-20T11:37:48.134700+0000","last_peered":"2024-04-20T11:37:48.134700+0000","last_clean":"2024-04-20T11:37:48.134700+0000","last_became_active":"2024-04-20T11:37:47.081021+0000","last_became_peered":"2024-04-20T11:37:47.081021+0000","last_unstale":"2024-04-20T11:37:48.134700+0000","last_undegraded":"2024-04-20T11:37:48.134700+0000","last_fullsized":"2024-04-20T11:37:48.134700+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_clean_scrub_stamp":"2024-04-20T11:37:46.019877+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:55:07.380625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.13","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.096781+0000","last_change":"2024-04-20T11:37:48.092651+0000","last_active":"2024-04-20T11:37:48.096781+0000","last_peered":"2024-04-20T11:37:48.096781+0000","last_clean":"2024-04-20T11:37:48.096781+0000","last_became_active":"2024-04-20T11:37:48.091945+0000","last_became_peered":"2024-04-20T11:37:48.091945+0000","last_unstale":"2024-04-20T11:37:48.096781+0000","last_undegraded":"2024-04-20T11:37:48.096781+0000","last_fullsized":"2024-04-20T11:37:48.096781+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:15:54.010830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":12,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.100961+0000","last_change":"2024-04-20T11:37:47.083406+0000","last_active":"2024-04-20T11:37:48.100961+0000","last_peered":"2024-04-20T11:37:48.100961+0000","last_clean":"2024-04-20T11:37:48.100961+0000","last_became_active":"2024-04-20T11:37:47.083070+0000","last_became_peered":"2024-04-20T11:37:47.083070+0000","last_unstale":"2024-04-20T11:37:48.100961+0000","last_undegraded":"2024-04-20T11:37:48.100961+0000","last_fullsized":"2024-04-20T11:37:48.100961+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_clean_scrub_stamp":"2024-04-20T11:37:46.019877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:31:33.749142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.10","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.122505+0000","last_change":"2024-04-20T11:37:48.116671+0000","last_active":"2024-04-20T11:37:48.122505+0000","last_peered":"2024-04-20T11:37:48.122505+0000","last_clean":"2024-04-20T11:37:48.122505+0000","last_became_active":"2024-04-20T11:37:48.113019+0000","last_became_peered":"2024-04-20T11:37:48.113019+0000","last_unstale":"2024-04-20T11:37:48.122505+0000","last_undegraded":"2024-04-20T11:37:48.122505+0000","last_fullsized":"2024-04-20T11:37:48.122505+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:49:59.397783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":12,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.084683+0000","last_change":"2024-04-20T11:37:47.078540+0000","last_active":"2024-04-20T11:37:48.084683+0000","last_peered":"2024-04-20T11:37:48.084683+0000","last_clean":"2024-04-20T11:37:48.084683+0000","last_became_active":"2024-04-20T11:37:47.077798+0000","last_became_peered":"2024-04-20T11:37:47.077798+0000","last_unstale":"2024-04-20T11:37:48.084683+0000","last_undegraded":"2024-04-20T11:37:48.084683+0000","last_fullsized":"2024-04-20T11:37:48.084683+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_clean_scrub_stamp":"2024-04-20T11:37:46.019877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:09:48.152605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.112300+0000","last_change":"2024-04-20T11:37:48.109165+0000","last_active":"2024-04-20T11:37:48.112300+0000","last_peered":"2024-04-20T11:37:48.112300+0000","last_clean":"2024-04-20T11:37:48.112300+0000","last_became_active":"2024-04-20T11:37:48.108827+0000","last_became_peered":"2024-04-20T11:37:48.108827+0000","last_unstale":"2024-04-20T11:37:48.112300+0000","last_undegraded":"2024-04-20T11:37:48.112300+0000","last_fullsized":"2024-04-20T11:37:48.112300+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:36:33.053650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.10","version":"166'8","reported_seq":20,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.137294+0000","last_change":"2024-04-20T11:37:47.078339+0000","last_active":"2024-04-20T11:37:48.137294+0000","last_peered":"2024-04-20T11:37:48.137294+0000","last_clean":"2024-04-20T11:37:48.137294+0000","last_became_active":"2024-04-20T11:37:47.078042+0000","last_became_peered":"2024-04-20T11:37:47.078042+0000","last_unstale":"2024-04-20T11:37:48.137294+0000","last_undegraded":"2024-04-20T11:37:48.137294+0000","last_fullsized":"2024-04-20T11:37:48.137294+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_clean_scrub_stamp":"2024-04-20T11:37:46.019877+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:42:47.506213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.e","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.109630+0000","last_change":"2024-04-20T11:37:48.105092+0000","last_active":"2024-04-20T11:37:48.109630+0000","last_peered":"2024-04-20T11:37:48.109630+0000","last_clean":"2024-04-20T11:37:48.109630+0000","last_became_active":"2024-04-20T11:37:48.103166+0000","last_became_peered":"2024-04-20T11:37:48.103166+0000","last_unstale":"2024-04-20T11:37:48.109630+0000","last_undegraded":"2024-04-20T11:37:48.109630+0000","last_fullsized":"2024-04-20T11:37:48.109630+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:38:56.071315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.f","version":"166'1","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.132870+0000","last_change":"2024-04-20T11:37:47.082946+0000","last_active":"2024-04-20T11:37:48.132870+0000","last_peered":"2024-04-20T11:37:48.132870+0000","last_clean":"2024-04-20T11:37:48.132870+0000","last_became_active":"2024-04-20T11:37:47.082275+0000","last_became_peered":"2024-04-20T11:37:47.082275+0000","last_unstale":"2024-04-20T11:37:48.132870+0000","last_undegraded":"2024-04-20T11:37:48.132870+0000","last_fullsized":"2024-04-20T11:37:48.132870+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_clean_scrub_stamp":"2024-04-20T11:37:46.019877+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:43:44.090262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.f","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.108871+0000","last_change":"2024-04-20T11:37:48.099908+0000","last_active":"2024-04-20T11:37:48.108871+0000","last_peered":"2024-04-20T11:37:48.108871+0000","last_clean":"2024-04-20T11:37:48.108871+0000","last_became_active":"2024-04-20T11:37:48.099598+0000","last_became_peered":"2024-04-20T11:37:48.099598+0000","last_unstale":"2024-04-20T11:37:48.108871+0000","last_undegraded":"2024-04-20T11:37:48.108871+0000","last_fullsized":"2024-04-20T11:37:48.108871+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:32:36.026412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.e","version":"166'2","reported_seq":14,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.139497+0000","last_change":"2024-04-20T11:37:47.078579+0000","last_active":"2024-04-20T11:37:48.139497+0000","last_peered":"2024-04-20T11:37:48.139497+0000","last_clean":"2024-04-20T11:37:48.139497+0000","last_became_active":"2024-04-20T11:37:47.078177+0000","last_became_peered":"2024-04-20T11:37:47.078177+0000","last_unstale":"2024-04-20T11:37:48.139497+0000","last_undegraded":"2024-04-20T11:37:48.139497+0000","last_fullsized":"2024-04-20T11:37:48.139497+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_clean_scrub_stamp":"2024-04-20T11:37:46.019877+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:42:47.506213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.099543+0000","last_change":"2024-04-20T11:37:48.092965+0000","last_active":"2024-04-20T11:37:48.099543+0000","last_peered":"2024-04-20T11:37:48.099543+0000","last_clean":"2024-04-20T11:37:48.099543+0000","last_became_active":"2024-04-20T11:37:48.091532+0000","last_became_peered":"2024-04-20T11:37:48.091532+0000","last_unstale":"2024-04-20T11:37:48.099543+0000","last_undegraded":"2024-04-20T11:37:48.099543+0000","last_fullsized":"2024-04-20T11:37:48.099543+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:38:56.071315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.d","version":"166'1","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.135579+0000","last_change":"2024-04-20T11:37:47.079127+0000","last_active":"2024-04-20T11:37:48.135579+0000","last_peered":"2024-04-20T11:37:48.135579+0000","last_clean":"2024-04-20T11:37:48.135579+0000","last_became_active":"2024-04-20T11:37:47.078846+0000","last_became_peered":"2024-04-20T11:37:47.078846+0000","last_unstale":"2024-04-20T11:37:48.135579+0000","last_undegraded":"2024-04-20T11:37:48.135579+0000","last_fullsized":"2024-04-20T11:37:48.135579+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_clean_scrub_stamp":"2024-04-20T11:37:46.019877+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:55:20.692965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.d","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.118673+0000","last_change":"2024-04-20T11:37:48.106804+0000","last_active":"2024-04-20T11:37:48.118673+0000","last_peered":"2024-04-20T11:37:48.118673+0000","last_clean":"2024-04-20T11:37:48.118673+0000","last_became_active":"2024-04-20T11:37:48.106320+0000","last_became_peered":"2024-04-20T11:37:48.106320+0000","last_unstale":"2024-04-20T11:37:48.118673+0000","last_undegraded":"2024-04-20T11:37:48.118673+0000","last_fullsized":"2024-04-20T11:37:48.118673+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:29:23.037410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":12,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.082232+0000","last_change":"2024-04-20T11:37:47.081678+0000","last_active":"2024-04-20T11:37:48.082232+0000","last_peered":"2024-04-20T11:37:48.082232+0000","last_clean":"2024-04-20T11:37:48.082232+0000","last_became_active":"2024-04-20T11:37:47.080366+0000","last_became_peered":"2024-04-20T11:37:47.080366+0000","last_unstale":"2024-04-20T11:37:48.082232+0000","last_undegraded":"2024-04-20T11:37:48.082232+0000","last_fullsized":"2024-04-20T11:37:48.082232+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_clean_scrub_stamp":"2024-04-20T11:37:46.019877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:32:35.010244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.119885+0000","last_change":"2024-04-20T11:37:48.113125+0000","last_active":"2024-04-20T11:37:48.119885+0000","last_peered":"2024-04-20T11:37:48.119885+0000","last_clean":"2024-04-20T11:37:48.119885+0000","last_became_active":"2024-04-20T11:37:48.112842+0000","last_became_peered":"2024-04-20T11:37:48.112842+0000","last_unstale":"2024-04-20T11:37:48.119885+0000","last_undegraded":"2024-04-20T11:37:48.119885+0000","last_fullsized":"2024-04-20T11:37:48.119885+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:56:04.982500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.b","version":"166'2","reported_seq":14,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.134377+0000","last_change":"2024-04-20T11:37:47.074786+0000","last_active":"2024-04-20T11:37:48.134377+0000","last_peered":"2024-04-20T11:37:48.134377+0000","last_clean":"2024-04-20T11:37:48.134377+0000","last_became_active":"2024-04-20T11:37:47.074387+0000","last_became_peered":"2024-04-20T11:37:47.074387+0000","last_unstale":"2024-04-20T11:37:48.134377+0000","last_undegraded":"2024-04-20T11:37:48.134377+0000","last_fullsized":"2024-04-20T11:37:48.134377+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_clean_scrub_stamp":"2024-04-20T11:37:46.019877+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:16:16.005318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.b","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.122926+0000","last_change":"2024-04-20T11:37:48.116958+0000","last_active":"2024-04-20T11:37:48.122926+0000","last_peered":"2024-04-20T11:37:48.122926+0000","last_clean":"2024-04-20T11:37:48.122926+0000","last_became_active":"2024-04-20T11:37:48.114749+0000","last_became_peered":"2024-04-20T11:37:48.114749+0000","last_unstale":"2024-04-20T11:37:48.122926+0000","last_undegraded":"2024-04-20T11:37:48.122926+0000","last_fullsized":"2024-04-20T11:37:48.122926+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:14:29.216857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":12,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.098293+0000","last_change":"2024-04-20T11:37:47.080682+0000","last_active":"2024-04-20T11:37:48.098293+0000","last_peered":"2024-04-20T11:37:48.098293+0000","last_clean":"2024-04-20T11:37:48.098293+0000","last_became_active":"2024-04-20T11:37:47.080348+0000","last_became_peered":"2024-04-20T11:37:47.080348+0000","last_unstale":"2024-04-20T11:37:48.098293+0000","last_undegraded":"2024-04-20T11:37:48.098293+0000","last_fullsized":"2024-04-20T11:37:48.098293+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_clean_scrub_stamp":"2024-04-20T11:37:46.019877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:29:22.021242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.112733+0000","last_change":"2024-04-20T11:37:48.110493+0000","last_active":"2024-04-20T11:37:48.112733+0000","last_peered":"2024-04-20T11:37:48.112733+0000","last_clean":"2024-04-20T11:37:48.112733+0000","last_became_active":"2024-04-20T11:37:48.110178+0000","last_became_peered":"2024-04-20T11:37:48.110178+0000","last_unstale":"2024-04-20T11:37:48.112733+0000","last_undegraded":"2024-04-20T11:37:48.112733+0000","last_fullsized":"2024-04-20T11:37:48.112733+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:56:16.721575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":12,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.098076+0000","last_change":"2024-04-20T11:37:47.083902+0000","last_active":"2024-04-20T11:37:48.098076+0000","last_peered":"2024-04-20T11:37:48.098076+0000","last_clean":"2024-04-20T11:37:48.098076+0000","last_became_active":"2024-04-20T11:37:47.083619+0000","last_became_peered":"2024-04-20T11:37:47.083619+0000","last_unstale":"2024-04-20T11:37:48.098076+0000","last_undegraded":"2024-04-20T11:37:48.098076+0000","last_fullsized":"2024-04-20T11:37:48.098076+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_clean_scrub_stamp":"2024-04-20T11:37:46.019877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:23:07.327346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.117696+0000","last_change":"2024-04-20T11:37:48.114675+0000","last_active":"2024-04-20T11:37:48.117696+0000","last_peered":"2024-04-20T11:37:48.117696+0000","last_clean":"2024-04-20T11:37:48.117696+0000","last_became_active":"2024-04-20T11:37:48.114212+0000","last_became_peered":"2024-04-20T11:37:48.114212+0000","last_unstale":"2024-04-20T11:37:48.117696+0000","last_undegraded":"2024-04-20T11:37:48.117696+0000","last_fullsized":"2024-04-20T11:37:48.117696+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:03:23.366996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":12,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.107470+0000","last_change":"2024-04-20T11:37:47.078603+0000","last_active":"2024-04-20T11:37:48.107470+0000","last_peered":"2024-04-20T11:37:48.107470+0000","last_clean":"2024-04-20T11:37:48.107470+0000","last_became_active":"2024-04-20T11:37:47.078301+0000","last_became_peered":"2024-04-20T11:37:47.078301+0000","last_unstale":"2024-04-20T11:37:48.107470+0000","last_undegraded":"2024-04-20T11:37:48.107470+0000","last_fullsized":"2024-04-20T11:37:48.107470+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_clean_scrub_stamp":"2024-04-20T11:37:46.019877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:18:14.091047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.121810+0000","last_change":"2024-04-20T11:37:48.119108+0000","last_active":"2024-04-20T11:37:48.121810+0000","last_peered":"2024-04-20T11:37:48.121810+0000","last_clean":"2024-04-20T11:37:48.121810+0000","last_became_active":"2024-04-20T11:37:48.116567+0000","last_became_peered":"2024-04-20T11:37:48.116567+0000","last_unstale":"2024-04-20T11:37:48.121810+0000","last_undegraded":"2024-04-20T11:37:48.121810+0000","last_fullsized":"2024-04-20T11:37:48.121810+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:11:56.044528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.7","version":"166'2","reported_seq":14,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.131150+0000","last_change":"2024-04-20T11:37:47.070199+0000","last_active":"2024-04-20T11:37:48.131150+0000","last_peered":"2024-04-20T11:37:48.131150+0000","last_clean":"2024-04-20T11:37:48.131150+0000","last_became_active":"2024-04-20T11:37:47.069218+0000","last_became_peered":"2024-04-20T11:37:47.069218+0000","last_unstale":"2024-04-20T11:37:48.131150+0000","last_undegraded":"2024-04-20T11:37:48.131150+0000","last_fullsized":"2024-04-20T11:37:48.131150+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_clean_scrub_stamp":"2024-04-20T11:37:46.019877+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:30:02.378451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.7","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.116933+0000","last_change":"2024-04-20T11:37:48.114848+0000","last_active":"2024-04-20T11:37:48.116933+0000","last_peered":"2024-04-20T11:37:48.116933+0000","last_clean":"2024-04-20T11:37:48.116933+0000","last_became_active":"2024-04-20T11:37:48.114533+0000","last_became_peered":"2024-04-20T11:37:48.114533+0000","last_unstale":"2024-04-20T11:37:48.116933+0000","last_undegraded":"2024-04-20T11:37:48.116933+0000","last_fullsized":"2024-04-20T11:37:48.116933+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:56:30.317215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.6","version":"0'0","reported_seq":12,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.102188+0000","last_change":"2024-04-20T11:37:47.083881+0000","last_active":"2024-04-20T11:37:48.102188+0000","last_peered":"2024-04-20T11:37:48.102188+0000","last_clean":"2024-04-20T11:37:48.102188+0000","last_became_active":"2024-04-20T11:37:47.082907+0000","last_became_peered":"2024-04-20T11:37:47.082907+0000","last_unstale":"2024-04-20T11:37:48.102188+0000","last_undegraded":"2024-04-20T11:37:48.102188+0000","last_fullsized":"2024-04-20T11:37:48.102188+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_clean_scrub_stamp":"2024-04-20T11:37:46.019877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:29:03.542441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.109344+0000","last_change":"2024-04-20T11:37:48.102559+0000","last_active":"2024-04-20T11:37:48.109344+0000","last_peered":"2024-04-20T11:37:48.109344+0000","last_clean":"2024-04-20T11:37:48.109344+0000","last_became_active":"2024-04-20T11:37:48.102261+0000","last_became_peered":"2024-04-20T11:37:48.102261+0000","last_unstale":"2024-04-20T11:37:48.109344+0000","last_undegraded":"2024-04-20T11:37:48.109344+0000","last_fullsized":"2024-04-20T11:37:48.109344+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:29:50.779164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":12,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.105296+0000","last_change":"2024-04-20T11:37:47.077943+0000","last_active":"2024-04-20T11:37:48.105296+0000","last_peered":"2024-04-20T11:37:48.105296+0000","last_clean":"2024-04-20T11:37:48.105296+0000","last_became_active":"2024-04-20T11:37:47.077677+0000","last_became_peered":"2024-04-20T11:37:47.077677+0000","last_unstale":"2024-04-20T11:37:48.105296+0000","last_undegraded":"2024-04-20T11:37:48.105296+0000","last_fullsized":"2024-04-20T11:37:48.105296+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_clean_scrub_stamp":"2024-04-20T11:37:46.019877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:10:42.658843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.117841+0000","last_change":"2024-04-20T11:37:48.115266+0000","last_active":"2024-04-20T11:37:48.117841+0000","last_peered":"2024-04-20T11:37:48.117841+0000","last_clean":"2024-04-20T11:37:48.117841+0000","last_became_active":"2024-04-20T11:37:48.114254+0000","last_became_peered":"2024-04-20T11:37:48.114254+0000","last_unstale":"2024-04-20T11:37:48.117841+0000","last_undegraded":"2024-04-20T11:37:48.117841+0000","last_fullsized":"2024-04-20T11:37:48.117841+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:05:03.866740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":12,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.082642+0000","last_change":"2024-04-20T11:37:47.064262+0000","last_active":"2024-04-20T11:37:48.082642+0000","last_peered":"2024-04-20T11:37:48.082642+0000","last_clean":"2024-04-20T11:37:48.082642+0000","last_became_active":"2024-04-20T11:37:47.063707+0000","last_became_peered":"2024-04-20T11:37:47.063707+0000","last_unstale":"2024-04-20T11:37:48.082642+0000","last_undegraded":"2024-04-20T11:37:48.082642+0000","last_fullsized":"2024-04-20T11:37:48.082642+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_clean_scrub_stamp":"2024-04-20T11:37:46.019877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:29:49.762996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.116147+0000","last_change":"2024-04-20T11:37:48.113860+0000","last_active":"2024-04-20T11:37:48.116147+0000","last_peered":"2024-04-20T11:37:48.116147+0000","last_clean":"2024-04-20T11:37:48.116147+0000","last_became_active":"2024-04-20T11:37:48.113551+0000","last_became_peered":"2024-04-20T11:37:48.113551+0000","last_unstale":"2024-04-20T11:37:48.116147+0000","last_undegraded":"2024-04-20T11:37:48.116147+0000","last_fullsized":"2024-04-20T11:37:48.116147+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:44:31.574362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":12,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.099785+0000","last_change":"2024-04-20T11:37:47.081822+0000","last_active":"2024-04-20T11:37:48.099785+0000","last_peered":"2024-04-20T11:37:48.099785+0000","last_clean":"2024-04-20T11:37:48.099785+0000","last_became_active":"2024-04-20T11:37:47.081527+0000","last_became_peered":"2024-04-20T11:37:47.081527+0000","last_unstale":"2024-04-20T11:37:48.099785+0000","last_undegraded":"2024-04-20T11:37:48.099785+0000","last_fullsized":"2024-04-20T11:37:48.099785+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_clean_scrub_stamp":"2024-04-20T11:37:46.019877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:19:32.710430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.122470+0000","last_change":"2024-04-20T11:37:48.121392+0000","last_active":"2024-04-20T11:37:48.122470+0000","last_peered":"2024-04-20T11:37:48.122470+0000","last_clean":"2024-04-20T11:37:48.122470+0000","last_became_active":"2024-04-20T11:37:48.121034+0000","last_became_peered":"2024-04-20T11:37:48.121034+0000","last_unstale":"2024-04-20T11:37:48.122470+0000","last_undegraded":"2024-04-20T11:37:48.122470+0000","last_fullsized":"2024-04-20T11:37:48.122470+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:43:45.106430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":12,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.107066+0000","last_change":"2024-04-20T11:37:47.077310+0000","last_active":"2024-04-20T11:37:48.107066+0000","last_peered":"2024-04-20T11:37:48.107066+0000","last_clean":"2024-04-20T11:37:48.107066+0000","last_became_active":"2024-04-20T11:37:47.076693+0000","last_became_peered":"2024-04-20T11:37:47.076693+0000","last_unstale":"2024-04-20T11:37:48.107066+0000","last_undegraded":"2024-04-20T11:37:48.107066+0000","last_fullsized":"2024-04-20T11:37:48.107066+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_clean_scrub_stamp":"2024-04-20T11:37:46.019877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:55:07.380625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.112363+0000","last_change":"2024-04-20T11:37:48.107949+0000","last_active":"2024-04-20T11:37:48.112363+0000","last_peered":"2024-04-20T11:37:48.112363+0000","last_clean":"2024-04-20T11:37:48.112363+0000","last_became_active":"2024-04-20T11:37:48.107636+0000","last_became_peered":"2024-04-20T11:37:48.107636+0000","last_unstale":"2024-04-20T11:37:48.112363+0000","last_undegraded":"2024-04-20T11:37:48.112363+0000","last_fullsized":"2024-04-20T11:37:48.112363+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:45:18.468249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":12,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.103717+0000","last_change":"2024-04-20T11:37:47.077826+0000","last_active":"2024-04-20T11:37:48.103717+0000","last_peered":"2024-04-20T11:37:48.103717+0000","last_clean":"2024-04-20T11:37:48.103717+0000","last_became_active":"2024-04-20T11:37:47.077577+0000","last_became_peered":"2024-04-20T11:37:47.077577+0000","last_unstale":"2024-04-20T11:37:48.103717+0000","last_undegraded":"2024-04-20T11:37:48.103717+0000","last_fullsized":"2024-04-20T11:37:48.103717+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_clean_scrub_stamp":"2024-04-20T11:37:46.019877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:33:32.171840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.112203+0000","last_change":"2024-04-20T11:37:48.110376+0000","last_active":"2024-04-20T11:37:48.112203+0000","last_peered":"2024-04-20T11:37:48.112203+0000","last_clean":"2024-04-20T11:37:48.112203+0000","last_became_active":"2024-04-20T11:37:48.109316+0000","last_became_peered":"2024-04-20T11:37:48.109316+0000","last_unstale":"2024-04-20T11:37:48.112203+0000","last_undegraded":"2024-04-20T11:37:48.112203+0000","last_fullsized":"2024-04-20T11:37:48.112203+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:45:18.468249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":12,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.100742+0000","last_change":"2024-04-20T11:37:47.079771+0000","last_active":"2024-04-20T11:37:48.100742+0000","last_peered":"2024-04-20T11:37:48.100742+0000","last_clean":"2024-04-20T11:37:48.100742+0000","last_became_active":"2024-04-20T11:37:47.079431+0000","last_became_peered":"2024-04-20T11:37:47.079431+0000","last_unstale":"2024-04-20T11:37:48.100742+0000","last_undegraded":"2024-04-20T11:37:48.100742+0000","last_fullsized":"2024-04-20T11:37:48.100742+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_clean_scrub_stamp":"2024-04-20T11:37:46.019877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:23:18.560236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1a","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.104878+0000","last_change":"2024-04-20T11:37:48.102464+0000","last_active":"2024-04-20T11:37:48.104878+0000","last_peered":"2024-04-20T11:37:48.104878+0000","last_clean":"2024-04-20T11:37:48.104878+0000","last_became_active":"2024-04-20T11:37:48.101872+0000","last_became_peered":"2024-04-20T11:37:48.101872+0000","last_unstale":"2024-04-20T11:37:48.104878+0000","last_undegraded":"2024-04-20T11:37:48.104878+0000","last_fullsized":"2024-04-20T11:37:48.104878+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:47.036044+0000","last_clean_scrub_stamp":"2024-04-20T11:37:47.036044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:05:47.730738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1b","version":"166'1","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.136903+0000","last_change":"2024-04-20T11:37:47.082880+0000","last_active":"2024-04-20T11:37:48.136903+0000","last_peered":"2024-04-20T11:37:48.136903+0000","last_clean":"2024-04-20T11:37:48.136903+0000","last_became_active":"2024-04-20T11:37:47.082283+0000","last_became_peered":"2024-04-20T11:37:47.082283+0000","last_unstale":"2024-04-20T11:37:48.136903+0000","last_undegraded":"2024-04-20T11:37:48.136903+0000","last_fullsized":"2024-04-20T11:37:48.136903+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:37:46.019877+0000","last_clean_scrub_stamp":"2024-04-20T11:37:46.019877+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:37:23.009200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"72'128","reported_seq":471,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-04-20T11:37:48.103043+0000","last_change":"2024-04-20T11:26:48.661897+0000","last_active":"2024-04-20T11:37:48.103043+0000","last_peered":"2024-04-20T11:37:48.103043+0000","last_clean":"2024-04-20T11:37:48.103043+0000","last_became_active":"2024-04-20T11:26:48.656535+0000","last_became_peered":"2024-04-20T11:26:48.656535+0000","last_unstale":"2024-04-20T11:37:48.103043+0000","last_undegraded":"2024-04-20T11:37:48.103043+0000","last_fullsized":"2024-04-20T11:37:48.103043+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:47.543004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:47.543004+0000","last_clean_scrub_stamp":"2024-04-20T11:26:47.543004+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:58:35.527105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":27,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574986,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33828,"kb_used_data":2640,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338012,"statfs":{"total":96636764160,"available":96602124288,"internally_reserved":0,"allocated":2703360,"data_stored":1519271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574983,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33232,"kb_used_data":2052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338608,"statfs":{"total":96636764160,"available":96602734592,"internally_reserved":0,"allocated":2101248,"data_stored":927873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574985,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33236,"kb_used_data":2052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338604,"statfs":{"total":96636764160,"available":96602730496,"internally_reserved":0,"allocated":2101248,"data_stored":927873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574983,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33256,"kb_used_data":2068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338584,"statfs":{"total":96636764160,"available":96602710016,"internally_reserved":0,"allocated":2117632,"data_stored":929007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574984,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33248,"kb_used_data":2060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338592,"statfs":{"total":96636764160,"available":96602718208,"internally_reserved":0,"allocated":2109440,"data_stored":927473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574985,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33220,"kb_used_data":2036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338620,"statfs":{"total":96636764160,"available":96602746880,"internally_reserved":0,"allocated":2084864,"data_stored":920516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574983,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33220,"kb_used_data":2036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338620,"statfs":{"total":96636764160,"available":96602746880,"internally_reserved":0,"allocated":2084864,"data_stored":920516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574984,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37976,"kb_used_data":2632,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333864,"statfs":{"total":96636764160,"available":96597876736,"internally_reserved":0,"allocated":2695168,"data_stored":1517749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T11:37:53.008 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-20T11:37:53.008 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-20T11:37:53.266 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.265+0000 7f3fa7a00640 1 Processor -- start 2024-04-20T11:37:53.266 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.266+0000 7f3fa7a00640 1 -- start start 2024-04-20T11:37:53.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.267+0000 7f3fa7a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3fa00d0b80 0x7f3fa0160d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:53.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.267+0000 7f3fa7a00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f3fa00cd450 0x7f3fa00723e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:53.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.267+0000 7f3fa7a00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3fa0161290 con 0x7f3fa00ccff0 2024-04-20T11:37:53.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.267+0000 7f3fa7a00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f3fa0145830 con 0x7f3fa00cd450 2024-04-20T11:37:53.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.267+0000 7f3fa7a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f3fa006f070 con 0x7f3fa00d0b80 2024-04-20T11:37:53.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.268+0000 7f3fa5a00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f3fa00cd450 0x7f3fa00723e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:53.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.268+0000 7f3fa5000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3fa00d0b80 0x7f3fa0160d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:53.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.268+0000 7f3fa5000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3fa00d0b80 0x7f3fa0160d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.152:36538/0 (socket says 172.21.15.152:36538) 2024-04-20T11:37:53.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.268+0000 7f3fa5000640 1 -- 172.21.15.152:0/2893155929 learned_addr learned my addr 172.21.15.152:0/2893155929 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:53.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.269+0000 7f3fa5000640 1 -- 172.21.15.152:0/2893155929 >> v2:172.21.15.175:3300/0 conn(0x7f3fa00cd450 msgr2=0x7f3fa00723e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:53.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.269+0000 7f3fa5000640 1 --2- 172.21.15.152:0/2893155929 >> v2:172.21.15.175:3300/0 conn(0x7f3fa00cd450 0x7f3fa00723e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:53.270 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.269+0000 7f3fa5000640 1 -- 172.21.15.152:0/2893155929 >> v1:172.21.15.175:6789/0 conn(0x7f3fa00ccff0 legacy=0x7f3fa0071cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:53.270 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.269+0000 7f3fa5000640 1 -- 172.21.15.152:0/2893155929 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fa006f500 con 0x7f3fa00d0b80 2024-04-20T11:37:53.270 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.269+0000 7f3f9f400640 1 -- 172.21.15.152:0/2893155929 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2071675943 0 0) 0x7f3fa0161290 con 0x7f3fa00ccff0 2024-04-20T11:37:53.271 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.270+0000 7f3fa5000640 1 --2- 172.21.15.152:0/2893155929 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3fa00d0b80 0x7f3fa0160d50 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3f90009870 tx=0x7f3f9002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=113d8d10fafacd40 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:53.271 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.271+0000 7f3f9f400640 1 -- 172.21.15.152:0/2893155929 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f90034070 con 0x7f3fa00d0b80 2024-04-20T11:37:53.271 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.271+0000 7f3f9f400640 1 -- 172.21.15.152:0/2893155929 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f90030840 con 0x7f3fa00d0b80 2024-04-20T11:37:53.271 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.271+0000 7f3f9f400640 1 -- 172.21.15.152:0/2893155929 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f90030c10 con 0x7f3fa00d0b80 2024-04-20T11:37:53.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.271+0000 7f3fa7a00640 1 -- 172.21.15.152:0/2893155929 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3fa00d0b80 msgr2=0x7f3fa0160d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:53.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.271+0000 7f3fa7a00640 1 --2- 172.21.15.152:0/2893155929 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3fa00d0b80 0x7f3fa0160d50 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3f90009870 tx=0x7f3f9002b980 comp rx=0 tx=0).stop 2024-04-20T11:37:53.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.271+0000 7f3fa7a00640 1 -- 172.21.15.152:0/2893155929 shutdown_connections 2024-04-20T11:37:53.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.271+0000 7f3fa7a00640 1 --2- 172.21.15.152:0/2893155929 >> v2:172.21.15.175:3300/0 conn(0x7f3fa00cd450 0x7f3fa00723e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:53.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.271+0000 7f3fa7a00640 1 --2- 172.21.15.152:0/2893155929 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3fa00d0b80 0x7f3fa0160d50 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:53.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.271+0000 7f3fa7a00640 1 -- 172.21.15.152:0/2893155929 >> 172.21.15.152:0/2893155929 conn(0x7f3fa006b980 msgr2=0x7f3fa006bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:53.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.271+0000 7f3fa7a00640 1 -- 172.21.15.152:0/2893155929 shutdown_connections 2024-04-20T11:37:53.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.271+0000 7f3fa7a00640 1 -- 172.21.15.152:0/2893155929 wait complete. 2024-04-20T11:37:53.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.273+0000 7f3fa7a00640 1 Processor -- start 2024-04-20T11:37:53.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.273+0000 7f3fa7a00640 1 -- start start 2024-04-20T11:37:53.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.274+0000 7f3fa7a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3fa00ccff0 0x7f3fa015c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:53.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.274+0000 7f3fa7a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3fa00cd450 0x7f3fa00b00b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:53.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.274+0000 7f3fa7a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3fa00d0b80 0x7f3fa00b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:53.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.274+0000 7f3fa7a00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f3fa006ea20 con 0x7f3fa00cd450 2024-04-20T11:37:53.276 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.274+0000 7f3fa7a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3fa006e740 con 0x7f3fa00ccff0 2024-04-20T11:37:53.276 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.274+0000 7f3fa7a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f3fa006ed00 con 0x7f3fa00d0b80 2024-04-20T11:37:53.276 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.274+0000 7f3f9fe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3fa00cd450 0x7f3fa00b00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:53.276 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.275+0000 7f3f9fe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3fa00cd450 0x7f3fa00b00b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35174/0 (socket says 172.21.15.152:35174) 2024-04-20T11:37:53.276 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.275+0000 7f3f9fe00640 1 -- 172.21.15.152:0/1343679900 learned_addr learned my addr 172.21.15.152:0/1343679900 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:53.276 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.275+0000 7f3fa5000640 1 --2- 172.21.15.152:0/1343679900 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3fa00ccff0 0x7f3fa015c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:53.276 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.275+0000 7f3fa5a00640 1 --2- 172.21.15.152:0/1343679900 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3fa00d0b80 0x7f3fa00b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:53.276 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.275+0000 7f3f9fe00640 1 -- 172.21.15.152:0/1343679900 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3fa00d0b80 msgr2=0x7f3fa00b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:53.276 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.275+0000 7f3f9fe00640 1 --2- 172.21.15.152:0/1343679900 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3fa00d0b80 0x7f3fa00b05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:53.276 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.275+0000 7f3f9fe00640 1 -- 172.21.15.152:0/1343679900 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3fa00ccff0 msgr2=0x7f3fa015c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:53.276 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.275+0000 7f3f9fe00640 1 --2- 172.21.15.152:0/1343679900 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3fa00ccff0 0x7f3fa015c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:53.276 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.275+0000 7f3f9fe00640 1 -- 172.21.15.152:0/1343679900 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fa00cb780 con 0x7f3fa00cd450 2024-04-20T11:37:53.277 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.275+0000 7f3f9fe00640 1 --2- 172.21.15.152:0/1343679900 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3fa00cd450 0x7f3fa00b00b0 secure :-1 s=READY pgs=2319 cs=0 l=1 rev1=1 crypto rx=0x7f3f8800c890 tx=0x7f3f8800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:53.277 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.275+0000 7f3f9cc00640 1 -- 172.21.15.152:0/1343679900 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f88017070 con 0x7f3fa00cd450 2024-04-20T11:37:53.277 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.276+0000 7f3f9cc00640 1 -- 172.21.15.152:0/1343679900 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f88003dc0 con 0x7f3fa00cd450 2024-04-20T11:37:53.277 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.276+0000 7f3f9cc00640 1 -- 172.21.15.152:0/1343679900 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f880052b0 con 0x7f3fa00cd450 2024-04-20T11:37:53.277 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.276+0000 7f3fa7a00640 1 -- 172.21.15.152:0/1343679900 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fa00ca6e0 con 0x7f3fa00cd450 2024-04-20T11:37:53.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.276+0000 7f3fa7a00640 1 -- 172.21.15.152:0/1343679900 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fa00caca0 con 0x7f3fa00cd450 2024-04-20T11:37:53.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.277+0000 7f3fa7a00640 1 -- 172.21.15.152:0/1343679900 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fa00689f0 con 0x7f3fa00cd450 2024-04-20T11:37:53.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.278+0000 7f3f9cc00640 1 -- 172.21.15.152:0/1343679900 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 26) ==== 91907+0+0 (secure 0 0 0) 0x7f3f8801a070 con 0x7f3fa00cd450 2024-04-20T11:37:53.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.279+0000 7f3f9cc00640 1 --2- 172.21.15.152:0/1343679900 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f3f7c06d280 0x7f3f7c06f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:53.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.280+0000 7f3f9cc00640 1 -- 172.21.15.152:0/1343679900 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 5709+0+0 (secure 0 0 0) 0x7f3f88089a90 con 0x7f3fa00cd450 2024-04-20T11:37:53.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.280+0000 7f3fa5000640 1 --2- 172.21.15.152:0/1343679900 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f3f7c06d280 0x7f3f7c06f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:53.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.281+0000 7f3fa5000640 1 --2- 172.21.15.152:0/1343679900 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f3f7c06d280 0x7f3f7c06f740 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f3f90036000 tx=0x7f3f90008030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:53.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.284+0000 7f3f9cc00640 1 -- 172.21.15.152:0/1343679900 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f3f88059a40 con 0x7f3fa00cd450 2024-04-20T11:37:53.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.564+0000 7f3fa7a00640 1 -- 172.21.15.152:0/1343679900 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f3fa0143bb0 con 0x7f3fa00cd450 2024-04-20T11:37:53.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.568+0000 7f3f9cc00640 1 -- 172.21.15.152:0/1343679900 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v23) ==== 190+0+147 (secure 0 0 0) 0x7f3f88059340 con 0x7f3fa00cd450 2024-04-20T11:37:53.569 INFO:teuthology.orchestra.run.smithi152.stdout:[client.0] 2024-04-20T11:37:53.569 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQBDpiNmtsBWFhAArq9A7r33CcwgMmOU2pAGPw== 2024-04-20T11:37:53.569 INFO:teuthology.orchestra.run.smithi152.stdout: caps mds = "allow" 2024-04-20T11:37:53.569 INFO:teuthology.orchestra.run.smithi152.stdout: caps mon = "allow r" 2024-04-20T11:37:53.569 INFO:teuthology.orchestra.run.smithi152.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-20T11:37:53.569 INFO:teuthology.orchestra.run.smithi152.stderr:updated caps for client.0 2024-04-20T11:37:53.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.571+0000 7f3fa7a00640 1 -- 172.21.15.152:0/1343679900 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f3f7c06d280 msgr2=0x7f3f7c06f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:53.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.571+0000 7f3fa7a00640 1 --2- 172.21.15.152:0/1343679900 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f3f7c06d280 0x7f3f7c06f740 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f3f90036000 tx=0x7f3f90008030 comp rx=0 tx=0).stop 2024-04-20T11:37:53.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.571+0000 7f3fa7a00640 1 -- 172.21.15.152:0/1343679900 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3fa00cd450 msgr2=0x7f3fa00b00b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:53.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.571+0000 7f3fa7a00640 1 --2- 172.21.15.152:0/1343679900 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3fa00cd450 0x7f3fa00b00b0 secure :-1 s=READY pgs=2319 cs=0 l=1 rev1=1 crypto rx=0x7f3f8800c890 tx=0x7f3f8800cd50 comp rx=0 tx=0).stop 2024-04-20T11:37:53.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.571+0000 7f3fa7a00640 1 -- 172.21.15.152:0/1343679900 shutdown_connections 2024-04-20T11:37:53.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.571+0000 7f3fa7a00640 1 --2- 172.21.15.152:0/1343679900 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f3f7c06d280 0x7f3f7c06f740 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:53.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.571+0000 7f3fa7a00640 1 --2- 172.21.15.152:0/1343679900 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3fa00d0b80 0x7f3fa00b05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:53.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.571+0000 7f3fa7a00640 1 --2- 172.21.15.152:0/1343679900 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3fa00cd450 0x7f3fa00b00b0 unknown :-1 s=CLOSED pgs=2319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:53.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.571+0000 7f3fa7a00640 1 --2- 172.21.15.152:0/1343679900 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3fa00ccff0 0x7f3fa015c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:53.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.571+0000 7f3fa7a00640 1 -- 172.21.15.152:0/1343679900 >> 172.21.15.152:0/1343679900 conn(0x7f3fa006b980 msgr2=0x7f3fa01440d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:53.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.571+0000 7f3fa7a00640 1 -- 172.21.15.152:0/1343679900 shutdown_connections 2024-04-20T11:37:53.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.572+0000 7f3fa7a00640 1 -- 172.21.15.152:0/1343679900 wait complete. 2024-04-20T11:37:53.592 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:37:53.791 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.790+0000 7f3266800640 1 Processor -- start 2024-04-20T11:37:53.791 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.790+0000 7f3266800640 1 -- start start 2024-04-20T11:37:53.792 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.792+0000 7f3266800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f32600b3d80 0x7f32600b4210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:53.792 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.792+0000 7f3266800640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f32601643b0 0x7f3260164800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:53.792 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.792+0000 7f3266800640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f326006e140 con 0x7f3260160870 2024-04-20T11:37:53.793 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.792+0000 7f3266800640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f3260160c50 con 0x7f32601643b0 2024-04-20T11:37:53.793 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.792+0000 7f3266800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f326006e420 con 0x7f32600b3d80 2024-04-20T11:37:53.793 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.792+0000 7f325f400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f3260160870 0x7f3260163c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:58496/0 (socket says 172.21.15.152:58496) 2024-04-20T11:37:53.793 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.792+0000 7f325f400640 1 -- 172.21.15.152:0/1991423194 learned_addr learned my addr 172.21.15.152:0/1991423194 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:37:53.795 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.792+0000 7f325fe00640 1 --2- 172.21.15.152:0/1991423194 >> v2:172.21.15.175:3300/0 conn(0x7f32601643b0 0x7f3260164800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:53.795 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.792+0000 7f3256a00640 1 --2- 172.21.15.152:0/1991423194 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f32600b3d80 0x7f32600b4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:53.795 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.792+0000 7f325ea00640 1 -- 172.21.15.152:0/1991423194 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3699925523 0 0) 0x7f326006e140 con 0x7f3260160870 2024-04-20T11:37:53.795 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.792+0000 7f325ea00640 1 -- 172.21.15.152:0/1991423194 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3244003520 con 0x7f3260160870 2024-04-20T11:37:53.795 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.793+0000 7f325ea00640 1 -- 172.21.15.152:0/1991423194 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2380800661 0 0) 0x7f3244003520 con 0x7f3260160870 2024-04-20T11:37:53.795 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.793+0000 7f325ea00640 1 -- 172.21.15.152:0/1991423194 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f32600b3d80 msgr2=0x7f32600b4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:53.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.793+0000 7f325ea00640 1 --2- 172.21.15.152:0/1991423194 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f32600b3d80 0x7f32600b4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:53.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.793+0000 7f325ea00640 1 -- 172.21.15.152:0/1991423194 >> v2:172.21.15.175:3300/0 conn(0x7f32601643b0 msgr2=0x7f3260164800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:53.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.793+0000 7f325ea00640 1 --2- 172.21.15.152:0/1991423194 >> v2:172.21.15.175:3300/0 conn(0x7f32601643b0 0x7f3260164800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:53.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.793+0000 7f325ea00640 1 -- 172.21.15.152:0/1991423194 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f326006e700 con 0x7f3260160870 2024-04-20T11:37:53.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.793+0000 7f325ea00640 1 -- 172.21.15.152:0/1991423194 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f3250001670 con 0x7f3260160870 2024-04-20T11:37:53.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.793+0000 7f325fe00640 1 --2- 172.21.15.152:0/1991423194 >> v2:172.21.15.175:3300/0 conn(0x7f32601643b0 0x7f3260164800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:37:53.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.793+0000 7f325ea00640 1 -- 172.21.15.152:0/1991423194 >> v1:172.21.15.152:6789/0 conn(0x7f3260160870 legacy=0x7f3260163c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:53.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.793+0000 7f325ea00640 1 --2- 172.21.15.152:0/1991423194 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f32600b3d80 0x7f3244003a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:53.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.793+0000 7f325ea00640 1 --2- 172.21.15.152:0/1991423194 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3244004050 0x7f3244004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:53.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.793+0000 7f325ea00640 1 --2- 172.21.15.152:0/1991423194 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3244004d40 0x7f32440051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:53.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.793+0000 7f325ea00640 1 -- 172.21.15.152:0/1991423194 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f3260160c50 con 0x7f3244004d40 2024-04-20T11:37:53.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.793+0000 7f325ea00640 1 -- 172.21.15.152:0/1991423194 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f326006e420 con 0x7f3244004050 2024-04-20T11:37:53.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.793+0000 7f325ea00640 1 -- 172.21.15.152:0/1991423194 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f324400bb50 con 0x7f32600b3d80 2024-04-20T11:37:53.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.793+0000 7f325fe00640 1 --2- 172.21.15.152:0/1991423194 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3244004d40 0x7f32440051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:53.797 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.794+0000 7f325f400640 1 --2- 172.21.15.152:0/1991423194 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f32600b3d80 0x7f3244003a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:53.797 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.794+0000 7f3256a00640 1 --2- 172.21.15.152:0/1991423194 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3244004050 0x7f3244004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:53.797 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.794+0000 7f325fe00640 1 -- 172.21.15.152:0/1991423194 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f32600b3d80 msgr2=0x7f3244003a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:53.797 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.794+0000 7f325fe00640 1 --2- 172.21.15.152:0/1991423194 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f32600b3d80 0x7f3244003a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:53.797 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.794+0000 7f325fe00640 1 -- 172.21.15.152:0/1991423194 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3244004050 msgr2=0x7f3244004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:53.797 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.794+0000 7f325fe00640 1 --2- 172.21.15.152:0/1991423194 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3244004050 0x7f3244004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:53.797 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.794+0000 7f325fe00640 1 -- 172.21.15.152:0/1991423194 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3244003520 con 0x7f3244004d40 2024-04-20T11:37:53.797 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.795+0000 7f325fe00640 1 --2- 172.21.15.152:0/1991423194 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3244004d40 0x7f32440051c0 secure :-1 s=READY pgs=2321 cs=0 l=1 rev1=1 crypto rx=0x7f32580073b0 tx=0x7f3244005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82690dc8f46de6c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:53.798 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.795+0000 7f325ea00640 1 -- 172.21.15.152:0/1991423194 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3258039070 con 0x7f3244004d40 2024-04-20T11:37:53.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.795+0000 7f325ea00640 1 -- 172.21.15.152:0/1991423194 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f325802cd90 con 0x7f3244004d40 2024-04-20T11:37:53.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.795+0000 7f325ea00640 1 -- 172.21.15.152:0/1991423194 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3258009c80 con 0x7f3244004d40 2024-04-20T11:37:53.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.796+0000 7f3266800640 1 -- 172.21.15.152:0/1991423194 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3244004d40 msgr2=0x7f32440051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:53.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.796+0000 7f3266800640 1 --2- 172.21.15.152:0/1991423194 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3244004d40 0x7f32440051c0 secure :-1 s=READY pgs=2321 cs=0 l=1 rev1=1 crypto rx=0x7f32580073b0 tx=0x7f3244005830 comp rx=0 tx=0).stop 2024-04-20T11:37:53.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.796+0000 7f325fe00640 1 -- 172.21.15.152:0/1991423194 reap_dead start 2024-04-20T11:37:53.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.796+0000 7f3266800640 1 -- 172.21.15.152:0/1991423194 shutdown_connections 2024-04-20T11:37:53.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.796+0000 7f3266800640 1 -- 172.21.15.152:0/1991423194 >> 172.21.15.152:0/1991423194 conn(0x7f3260067cf0 msgr2=0x7f326006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:53.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.796+0000 7f3266800640 1 -- 172.21.15.152:0/1991423194 shutdown_connections 2024-04-20T11:37:53.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.796+0000 7f3266800640 1 -- 172.21.15.152:0/1991423194 wait complete. 2024-04-20T11:37:53.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.798+0000 7f3266800640 1 Processor -- start 2024-04-20T11:37:53.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.798+0000 7f3266800640 1 -- start start 2024-04-20T11:37:53.801 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.799+0000 7f3266800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3244004d40 0x7f32600b14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:53.801 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.799+0000 7f3266800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f32600cea40 0x7f32600cf910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:53.801 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.799+0000 7f3266800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32601fbaa0 0x7f32601fbf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:53.801 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.799+0000 7f3266800640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f326006de10 con 0x7f32600cea40 2024-04-20T11:37:53.801 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.799+0000 7f3266800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f326006ecc0 con 0x7f32601fbaa0 2024-04-20T11:37:53.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.799+0000 7f3266800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f326006f1e0 con 0x7f3244004d40 2024-04-20T11:37:53.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.799+0000 7f3256a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f32600cea40 0x7f32600cf910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:53.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.799+0000 7f3256a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f32600cea40 0x7f32600cf910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35204/0 (socket says 172.21.15.152:35204) 2024-04-20T11:37:53.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.799+0000 7f3256a00640 1 -- 172.21.15.152:0/3854630786 learned_addr learned my addr 172.21.15.152:0/3854630786 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:53.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.799+0000 7f325f400640 1 --2- 172.21.15.152:0/3854630786 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3244004d40 0x7f32600b14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:53.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.799+0000 7f3256a00640 1 -- 172.21.15.152:0/3854630786 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3244004d40 msgr2=0x7f32600b14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:53.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.799+0000 7f3256a00640 1 --2- 172.21.15.152:0/3854630786 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3244004d40 0x7f32600b14a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:53.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.799+0000 7f3256a00640 1 -- 172.21.15.152:0/3854630786 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32601fbaa0 msgr2=0x7f32601fbf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:53.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.799+0000 7f325fe00640 1 --2- 172.21.15.152:0/3854630786 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32601fbaa0 0x7f32601fbf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:53.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.799+0000 7f3256a00640 1 --2- 172.21.15.152:0/3854630786 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32601fbaa0 0x7f32601fbf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:53.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.799+0000 7f3256a00640 1 -- 172.21.15.152:0/3854630786 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32600b0400 con 0x7f32600cea40 2024-04-20T11:37:53.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.799+0000 7f325fe00640 1 --2- 172.21.15.152:0/3854630786 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32601fbaa0 0x7f32601fbf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:37:53.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.800+0000 7f3256a00640 1 --2- 172.21.15.152:0/3854630786 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f32600cea40 0x7f32600cf910 secure :-1 s=READY pgs=2322 cs=0 l=1 rev1=1 crypto rx=0x7f324800f560 tx=0x7f324800fa20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:53.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.800+0000 7f3257e00640 1 -- 172.21.15.152:0/3854630786 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3248018020 con 0x7f32600cea40 2024-04-20T11:37:53.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.800+0000 7f3257e00640 1 -- 172.21.15.152:0/3854630786 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3248014870 con 0x7f32600cea40 2024-04-20T11:37:53.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.800+0000 7f3266800640 1 -- 172.21.15.152:0/3854630786 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32600b06e0 con 0x7f32600cea40 2024-04-20T11:37:53.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.800+0000 7f3257e00640 1 -- 172.21.15.152:0/3854630786 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3248014c60 con 0x7f32600cea40 2024-04-20T11:37:53.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.800+0000 7f3266800640 1 -- 172.21.15.152:0/3854630786 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32600b0ca0 con 0x7f32600cea40 2024-04-20T11:37:53.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.801+0000 7f3266800640 1 -- 172.21.15.152:0/3854630786 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32600689f0 con 0x7f32600cea40 2024-04-20T11:37:53.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.802+0000 7f3257e00640 1 -- 172.21.15.152:0/3854630786 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 26) ==== 91907+0+0 (secure 0 0 0) 0x7f324801b020 con 0x7f32600cea40 2024-04-20T11:37:53.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.803+0000 7f3257e00640 1 --2- 172.21.15.152:0/3854630786 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f323c06d280 0x7f323c06f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:53.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.803+0000 7f3257e00640 1 -- 172.21.15.152:0/3854630786 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 5709+0+0 (secure 0 0 0) 0x7f324808a8a0 con 0x7f32600cea40 2024-04-20T11:37:53.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.804+0000 7f325f400640 1 --2- 172.21.15.152:0/3854630786 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f323c06d280 0x7f323c06f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:53.805 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.805+0000 7f325f400640 1 --2- 172.21.15.152:0/3854630786 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f323c06d280 0x7f323c06f740 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f3250001eb0 tx=0x7f3250000ef0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:53.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:53.807+0000 7f3257e00640 1 -- 172.21.15.152:0/3854630786 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f3248022020 con 0x7f32600cea40 2024-04-20T11:37:54.020 INFO:tasks.ceph.mds.c.smithi152.stderr:2024-04-20T11:37:54.019+0000 7f9fd5600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-20T11:37:54.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.029+0000 7f3266800640 1 -- 172.21.15.152:0/3854630786 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f32601654e0 con 0x7f32600cea40 2024-04-20T11:37:54.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.030+0000 7f3257e00640 1 -- 172.21.15.152:0/3854630786 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 115 v115) ==== 95+0+5215 (secure 0 0 0) 0x7f3248024300 con 0x7f32600cea40 2024-04-20T11:37:54.031 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:37:54.031 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":115,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6605,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6834/4020670267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6833","nonce":4020670267},{"type":"v1","addr":"172.21.15.152:6834","nonce":4020670267}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":109},{"gid":6614,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/352716681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":352716681},{"type":"v1","addr":"172.21.15.175:6833","nonce":352716681}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":110},{"gid":6617,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/2512108447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":2512108447},{"type":"v1","addr":"172.21.15.175:6835","nonce":2512108447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":110}],"filesystems":[{"mdsmap":{"epoch":115,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T11:37:47.506735+0000","modified":"2024-04-20T11:37:50.605372+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7339},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7339":{"gid":7339,"name":"c","rank":0,"incarnation":111,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6835/951802292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":951802292},{"type":"v1","addr":"172.21.15.152:6835","nonce":951802292}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7339,"qdb_cluster":[7339]},"id":13}]} 2024-04-20T11:37:54.031 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 115 2024-04-20T11:37:54.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.032+0000 7f3266800640 1 -- 172.21.15.152:0/3854630786 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f323c06d280 msgr2=0x7f323c06f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:54.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.032+0000 7f3266800640 1 --2- 172.21.15.152:0/3854630786 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f323c06d280 0x7f323c06f740 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f3250001eb0 tx=0x7f3250000ef0 comp rx=0 tx=0).stop 2024-04-20T11:37:54.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.032+0000 7f3266800640 1 -- 172.21.15.152:0/3854630786 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f32600cea40 msgr2=0x7f32600cf910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:54.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.032+0000 7f3266800640 1 --2- 172.21.15.152:0/3854630786 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f32600cea40 0x7f32600cf910 secure :-1 s=READY pgs=2322 cs=0 l=1 rev1=1 crypto rx=0x7f324800f560 tx=0x7f324800fa20 comp rx=0 tx=0).stop 2024-04-20T11:37:54.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.033+0000 7f3266800640 1 -- 172.21.15.152:0/3854630786 shutdown_connections 2024-04-20T11:37:54.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.033+0000 7f3266800640 1 --2- 172.21.15.152:0/3854630786 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f323c06d280 0x7f323c06f740 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:54.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.033+0000 7f3266800640 1 --2- 172.21.15.152:0/3854630786 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32601fbaa0 0x7f32601fbf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:54.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.033+0000 7f3266800640 1 --2- 172.21.15.152:0/3854630786 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f32600cea40 0x7f32600cf910 unknown :-1 s=CLOSED pgs=2322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:54.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.033+0000 7f3266800640 1 --2- 172.21.15.152:0/3854630786 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3244004d40 0x7f32600b14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:54.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.033+0000 7f3266800640 1 -- 172.21.15.152:0/3854630786 >> 172.21.15.152:0/3854630786 conn(0x7f3260067cf0 msgr2=0x7f3260166010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:54.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.033+0000 7f3266800640 1 -- 172.21.15.152:0/3854630786 shutdown_connections 2024-04-20T11:37:54.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.033+0000 7f3266800640 1 -- 172.21.15.152:0/3854630786 wait complete. 2024-04-20T11:37:54.152 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:37:54.160 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:37:54.455 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 115, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-20T11:37:47.506735+0000', 'modified': '2024-04-20T11:37:50.605372+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'snap_rstat': True, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7339}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7339': {'gid': 7339, 'name': 'c', 'rank': 0, 'incarnation': 111, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.152:6835/951802292', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6832', 'nonce': 951802292}, {'type': 'v1', 'addr': '172.21.15.152:6835', 'nonce': 951802292}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [27], 'metadata_pool': 26, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7339, 'qdb_cluster': [7339]} 2024-04-20T11:37:54.456 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-20T11:37:54.456 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-20T11:37:54.671 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.670+0000 7f622be00640 1 Processor -- start 2024-04-20T11:37:54.671 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.671+0000 7f622be00640 1 -- start start 2024-04-20T11:37:54.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.672+0000 7f622be00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f62240ce9a0 0x7f62240d0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:54.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.672+0000 7f622be00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f622409b710 0x7f622406d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:54.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.672+0000 7f622be00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62240d0e10 con 0x7f622406d6b0 2024-04-20T11:37:54.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.672+0000 7f622be00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f62240d1a20 con 0x7f622409b710 2024-04-20T11:37:54.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.672+0000 7f622be00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f622406db00 con 0x7f62240ce9a0 2024-04-20T11:37:54.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.673+0000 7f6228a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f622409b710 0x7f622406d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:54.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.673+0000 7f6229e00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f622406d6b0 0x7f622415e1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:58498/0 (socket says 172.21.15.152:58498) 2024-04-20T11:37:54.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.673+0000 7f6229e00640 1 -- 172.21.15.152:0/2857224882 learned_addr learned my addr 172.21.15.152:0/2857224882 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:37:54.674 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.673+0000 7f6229400640 1 --2- 172.21.15.152:0/2857224882 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f62240ce9a0 0x7f62240d0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:54.674 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.673+0000 7f6228a00640 1 -- 172.21.15.152:0/2857224882 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f62240ce9a0 msgr2=0x7f62240d0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:54.674 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.673+0000 7f6228a00640 1 --2- 172.21.15.152:0/2857224882 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f62240ce9a0 0x7f62240d0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:54.674 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.673+0000 7f6228a00640 1 -- 172.21.15.152:0/2857224882 >> v1:172.21.15.152:6789/0 conn(0x7f622406d6b0 legacy=0x7f622415e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:54.675 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.673+0000 7f6228a00640 1 -- 172.21.15.152:0/2857224882 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62240d10f0 con 0x7f622409b710 2024-04-20T11:37:54.675 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.673+0000 7f6223e00640 1 -- 172.21.15.152:0/2857224882 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1422803306 0 0) 0x7f62240d0e10 con 0x7f622406d6b0 2024-04-20T11:37:54.676 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.674+0000 7f6228a00640 1 --2- 172.21.15.152:0/2857224882 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f622409b710 0x7f622406d170 secure :-1 s=READY pgs=2324 cs=0 l=1 rev1=1 crypto rx=0x7f62180097a0 tx=0x7f621802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13a7277bb78633a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:54.676 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.674+0000 7f6223e00640 1 -- 172.21.15.152:0/2857224882 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62180315a0 con 0x7f622409b710 2024-04-20T11:37:54.676 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.675+0000 7f6223e00640 1 -- 172.21.15.152:0/2857224882 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6218037070 con 0x7f622409b710 2024-04-20T11:37:54.676 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.675+0000 7f6223e00640 1 -- 172.21.15.152:0/2857224882 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6218031960 con 0x7f622409b710 2024-04-20T11:37:54.676 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.675+0000 7f622be00640 1 -- 172.21.15.152:0/2857224882 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f622409b710 msgr2=0x7f622406d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:54.676 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.675+0000 7f622be00640 1 --2- 172.21.15.152:0/2857224882 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f622409b710 0x7f622406d170 secure :-1 s=READY pgs=2324 cs=0 l=1 rev1=1 crypto rx=0x7f62180097a0 tx=0x7f621802b840 comp rx=0 tx=0).stop 2024-04-20T11:37:54.676 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.675+0000 7f622be00640 1 -- 172.21.15.152:0/2857224882 shutdown_connections 2024-04-20T11:37:54.676 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.675+0000 7f622be00640 1 --2- 172.21.15.152:0/2857224882 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f622409b710 0x7f622406d170 unknown :-1 s=CLOSED pgs=2324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:54.676 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.675+0000 7f622be00640 1 --2- 172.21.15.152:0/2857224882 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f62240ce9a0 0x7f62240d0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:54.677 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.675+0000 7f622be00640 1 -- 172.21.15.152:0/2857224882 >> 172.21.15.152:0/2857224882 conn(0x7f62240b4040 msgr2=0x7f62240b4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:54.677 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.675+0000 7f622be00640 1 -- 172.21.15.152:0/2857224882 shutdown_connections 2024-04-20T11:37:54.677 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.675+0000 7f622be00640 1 -- 172.21.15.152:0/2857224882 wait complete. 2024-04-20T11:37:54.678 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.677+0000 7f622be00640 1 Processor -- start 2024-04-20T11:37:54.678 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.677+0000 7f622be00640 1 -- start start 2024-04-20T11:37:54.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.678+0000 7f622be00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f622406d6b0 0x7f62240cb180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:54.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.678+0000 7f622be00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f622409b710 0x7f6224158920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:54.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.678+0000 7f622be00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f62240ce9a0 0x7f62240c9620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:54.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.678+0000 7f622be00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f62240ced80 con 0x7f62240ce9a0 2024-04-20T11:37:54.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.678+0000 7f622be00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f62240d16b0 con 0x7f622409b710 2024-04-20T11:37:54.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.678+0000 7f622be00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f622409bb50 con 0x7f622406d6b0 2024-04-20T11:37:54.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.679+0000 7f6229e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f62240ce9a0 0x7f62240c9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:54.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.679+0000 7f6229e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f62240ce9a0 0x7f62240c9620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35234/0 (socket says 172.21.15.152:35234) 2024-04-20T11:37:54.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.679+0000 7f6229e00640 1 -- 172.21.15.152:0/77449319 learned_addr learned my addr 172.21.15.152:0/77449319 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:54.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.679+0000 7f6228a00640 1 --2- 172.21.15.152:0/77449319 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f622409b710 0x7f6224158920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:54.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.679+0000 7f6229400640 1 --2- 172.21.15.152:0/77449319 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f622406d6b0 0x7f62240cb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:54.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.679+0000 7f6229e00640 1 -- 172.21.15.152:0/77449319 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f622406d6b0 msgr2=0x7f62240cb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:54.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.679+0000 7f6229e00640 1 --2- 172.21.15.152:0/77449319 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f622406d6b0 0x7f62240cb180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:54.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.679+0000 7f6229e00640 1 -- 172.21.15.152:0/77449319 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f622409b710 msgr2=0x7f6224158920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:54.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.679+0000 7f6229e00640 1 --2- 172.21.15.152:0/77449319 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f622409b710 0x7f6224158920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:54.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.679+0000 7f6229e00640 1 -- 172.21.15.152:0/77449319 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62240ca0e0 con 0x7f62240ce9a0 2024-04-20T11:37:54.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.679+0000 7f6229e00640 1 --2- 172.21.15.152:0/77449319 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f62240ce9a0 0x7f62240c9620 secure :-1 s=READY pgs=2325 cs=0 l=1 rev1=1 crypto rx=0x7f621c00c700 tx=0x7f621c00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:54.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.680+0000 7f6221600640 1 -- 172.21.15.152:0/77449319 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f621c018070 con 0x7f62240ce9a0 2024-04-20T11:37:54.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.680+0000 7f6221600640 1 -- 172.21.15.152:0/77449319 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f621c00ce10 con 0x7f62240ce9a0 2024-04-20T11:37:54.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.680+0000 7f6221600640 1 -- 172.21.15.152:0/77449319 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f621c00a860 con 0x7f62240ce9a0 2024-04-20T11:37:54.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.680+0000 7f622be00640 1 -- 172.21.15.152:0/77449319 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62240ca3c0 con 0x7f62240ce9a0 2024-04-20T11:37:54.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.680+0000 7f622be00640 1 -- 172.21.15.152:0/77449319 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62240cac60 con 0x7f62240ce9a0 2024-04-20T11:37:54.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.682+0000 7f6221600640 1 -- 172.21.15.152:0/77449319 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 26) ==== 91907+0+0 (secure 0 0 0) 0x7f621c01b070 con 0x7f62240ce9a0 2024-04-20T11:37:54.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.683+0000 7f6221600640 1 --2- 172.21.15.152:0/77449319 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f620806d280 0x7f620806f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:54.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.683+0000 7f6221600640 1 -- 172.21.15.152:0/77449319 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 5709+0+0 (secure 0 0 0) 0x7f621c089840 con 0x7f62240ce9a0 2024-04-20T11:37:54.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.684+0000 7f622ebff740 1 Processor -- start 2024-04-20T11:37:54.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.684+0000 7f6229400640 1 --2- 172.21.15.152:0/77449319 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f620806d280 0x7f620806f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:54.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.684+0000 7f622ebff740 1 -- start start 2024-04-20T11:37:54.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.685+0000 7f622ebff740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562eedfba460 0x562eedf71a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:54.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.685+0000 7f622ebff740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562eedf704e0 0x562eedf71fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:54.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.685+0000 7f622ebff740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x562eedf6c930 0x562eedf9c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:54.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.685+0000 7f622ebff740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x562eeded89d0 con 0x562eedf704e0 2024-04-20T11:37:54.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.685+0000 7f622ebff740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x562eede7f270 con 0x562eedfba460 2024-04-20T11:37:54.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.685+0000 7f622ebff740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x562eeded4990 con 0x562eedf6c930 2024-04-20T11:37:54.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.685+0000 7f6228a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562eedf704e0 0x562eedf71fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:54.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.685+0000 7f6229400640 1 --2- 172.21.15.152:0/77449319 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f620806d280 0x7f620806f740 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f6214006fd0 tx=0x7f6214008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:54.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.685+0000 7f6228a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562eedf704e0 0x562eedf71fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35244/0 (socket says 172.21.15.152:35244) 2024-04-20T11:37:54.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.685+0000 7f6228a00640 1 -- 172.21.15.152:0/3551282453 learned_addr learned my addr 172.21.15.152:0/3551282453 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:54.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.685+0000 7f6229e00640 1 --2- 172.21.15.152:0/3551282453 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x562eedf6c930 0x562eedf9c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:54.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.685+0000 7f6229400640 1 --2- 172.21.15.152:0/3551282453 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562eedfba460 0x562eedf71a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:54.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.685+0000 7f6228a00640 1 -- 172.21.15.152:0/3551282453 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x562eedf6c930 msgr2=0x562eedf9c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:54.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.685+0000 7f6228a00640 1 --2- 172.21.15.152:0/3551282453 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x562eedf6c930 0x562eedf9c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:54.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.685+0000 7f6228a00640 1 -- 172.21.15.152:0/3551282453 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562eedfba460 msgr2=0x562eedf71a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:54.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.685+0000 7f6228a00640 1 --2- 172.21.15.152:0/3551282453 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562eedfba460 0x562eedf71a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:54.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.685+0000 7f6228a00640 1 -- 172.21.15.152:0/3551282453 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562eedbeb2d0 con 0x562eedf704e0 2024-04-20T11:37:54.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.686+0000 7f6228a00640 1 --2- 172.21.15.152:0/3551282453 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562eedf704e0 0x562eedf71fd0 secure :-1 s=READY pgs=2326 cs=0 l=1 rev1=1 crypto rx=0x7f6218034040 tx=0x7f6218030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=816f7c5076d92f6f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:54.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.686+0000 7f6211c00640 1 -- 172.21.15.152:0/3551282453 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6218031860 con 0x562eedf704e0 2024-04-20T11:37:54.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.686+0000 7f6211c00640 1 -- 172.21.15.152:0/3551282453 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6218037040 con 0x562eedf704e0 2024-04-20T11:37:54.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.686+0000 7f6211c00640 1 -- 172.21.15.152:0/3551282453 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6218031c00 con 0x562eedf704e0 2024-04-20T11:37:54.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.687+0000 7f622ebff740 1 -- 172.21.15.152:0/3551282453 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562eedf704e0 msgr2=0x562eedf71fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:54.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.687+0000 7f622ebff740 1 --2- 172.21.15.152:0/3551282453 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562eedf704e0 0x562eedf71fd0 secure :-1 s=READY pgs=2326 cs=0 l=1 rev1=1 crypto rx=0x7f6218034040 tx=0x7f6218030000 comp rx=0 tx=0).stop 2024-04-20T11:37:54.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.687+0000 7f622ebff740 1 -- 172.21.15.152:0/3551282453 shutdown_connections 2024-04-20T11:37:54.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.687+0000 7f622ebff740 1 --2- 172.21.15.152:0/3551282453 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x562eedf6c930 0x562eedf9c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:54.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.687+0000 7f622ebff740 1 --2- 172.21.15.152:0/3551282453 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562eedf704e0 0x562eedf71fd0 unknown :-1 s=CLOSED pgs=2326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:54.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.687+0000 7f622ebff740 1 --2- 172.21.15.152:0/3551282453 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562eedfba460 0x562eedf71a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:54.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.687+0000 7f622ebff740 1 -- 172.21.15.152:0/3551282453 >> 172.21.15.152:0/3551282453 conn(0x562eedf6ebf0 msgr2=0x562eedfb7770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:54.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.687+0000 7f622ebff740 1 -- 172.21.15.152:0/3551282453 shutdown_connections 2024-04-20T11:37:54.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.687+0000 7f622ebff740 1 -- 172.21.15.152:0/3551282453 wait complete. 2024-04-20T11:37:54.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.688+0000 7f622ebff740 1 -- start start 2024-04-20T11:37:54.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.688+0000 7f622ebff740 1 Processor -- start 2024-04-20T11:37:54.693 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.689+0000 7f622ebff740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562eedf704e0 0x562eedf93940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:54.693 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.689+0000 7f622ebff740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x562eedfba460 0x562eedf916a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:54.693 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.689+0000 7f622ebff740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562eedf99d70 0x562eedf91be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:54.693 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.689+0000 7f622ebff740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x562eedf709f0 con 0x562eedf99d70 2024-04-20T11:37:54.693 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.689+0000 7f622ebff740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x562eede925e0 con 0x562eedf704e0 2024-04-20T11:37:54.693 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.689+0000 7f622ebff740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x562eedf70cd0 con 0x562eedfba460 2024-04-20T11:37:54.693 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.689+0000 7f622ebff740 10 client.0 _ll_register_callbacks cb 0x562eedbeaac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:37:54.693 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.689+0000 7f6229e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562eedf99d70 0x562eedf91be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:54.693 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.689+0000 7f6229e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562eedf99d70 0x562eedf91be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35254/0 (socket says 172.21.15.152:35254) 2024-04-20T11:37:54.693 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.689+0000 7f6229e00640 1 -- 172.21.15.152:0/3802233858 learned_addr learned my addr 172.21.15.152:0/3802233858 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:54.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.690+0000 7f6229400640 1 --2- 172.21.15.152:0/3802233858 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562eedf704e0 0x562eedf93940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:54.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.690+0000 7f6228a00640 1 --2- 172.21.15.152:0/3802233858 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x562eedfba460 0x562eedf916a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:54.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.690+0000 7f6229e00640 1 -- 172.21.15.152:0/3802233858 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x562eedfba460 msgr2=0x562eedf916a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:54.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.690+0000 7f6229e00640 1 --2- 172.21.15.152:0/3802233858 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x562eedfba460 0x562eedf916a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:54.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.690+0000 7f6229e00640 1 -- 172.21.15.152:0/3802233858 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562eedf704e0 msgr2=0x562eedf93940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:54.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.690+0000 7f6229e00640 1 --2- 172.21.15.152:0/3802233858 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562eedf704e0 0x562eedf93940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:54.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.690+0000 7f6229e00640 1 -- 172.21.15.152:0/3802233858 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562eedcb56b0 con 0x562eedf99d70 2024-04-20T11:37:54.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.690+0000 7f6228a00640 1 --2- 172.21.15.152:0/3802233858 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x562eedfba460 0x562eedf916a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:37:54.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.690+0000 7f6229400640 1 --2- 172.21.15.152:0/3802233858 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562eedf704e0 0x562eedf93940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:37:54.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.690+0000 7f6229e00640 1 --2- 172.21.15.152:0/3802233858 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562eedf99d70 0x562eedf91be0 secure :-1 s=READY pgs=2327 cs=0 l=1 rev1=1 crypto rx=0x7f621c00a430 tx=0x7f621c0599b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=293180f13d4b4637 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:54.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.690+0000 7f6206000640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-04-20T11:37:54.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.690+0000 7f6206000640 1 -- 172.21.15.152:0/3802233858 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f621c06dd80 con 0x562eedf99d70 2024-04-20T11:37:54.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.690+0000 7f6206000640 1 -- 172.21.15.152:0/3802233858 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(167..167 src has 1..167) ==== 5709+0+0 (secure 0 0 0) 0x7f621c0585c0 con 0x562eedf99d70 2024-04-20T11:37:54.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.690+0000 7f622ebff740 1 -- 172.21.15.152:0/3802233858 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6218037040 con 0x562eedf99d70 2024-04-20T11:37:54.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.691+0000 7f6206000640 1 -- 172.21.15.152:0/3802233858 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=115) ==== 24+0+0 (secure 0 0 0) 0x7f621c0587c0 con 0x562eedf99d70 2024-04-20T11:37:54.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.691+0000 7f622ebff740 10 client.10632 fetch_fsmap learned FSMap version 115 2024-04-20T11:37:54.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.691+0000 7f622ebff740 1 -- 172.21.15.152:0/3802233858 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=115,monmap=2+}) -- 0x562eedfbcfa0 con 0x562eedf99d70 2024-04-20T11:37:54.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.691+0000 7f6206000640 1 -- 172.21.15.152:0/3802233858 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6218037040 con 0x562eedf99d70 2024-04-20T11:37:54.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.691+0000 7f6206000640 1 -- 172.21.15.152:0/3802233858 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 115) ==== 3495+0+0 (secure 0 0 0) 0x7f621c054ec0 con 0x562eedf99d70 2024-04-20T11:37:54.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.691+0000 7f622ebff740 10 client.10632 fetch_fsmap finished waiting for FSMap version 115 2024-04-20T11:37:54.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.691+0000 7f622ebff740 10 client.10632 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-20T11:37:54.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.691+0000 7f622ebff740 20 client.10632 populate_metadata read hostname 'smithi152' 2024-04-20T11:37:54.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.691+0000 7f622ebff740 1 --2- 172.21.15.152:0/3802233858 >> [v2:172.21.15.152:6832/951802292,v1:172.21.15.152:6835/951802292] conn(0x562eedf94dc0 0x562eedfbe680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:54.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.691+0000 7f622ebff740 4 client.10632 mds_command: new command op to 7339 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:37:54.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.691+0000 7f622ebff740 1 -- 172.21.15.152:0/3802233858 --> [v2:172.21.15.152:6832/951802292,v1:172.21.15.152:6835/951802292] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562eedee63f0 con 0x562eedf94dc0 2024-04-20T11:37:54.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.691+0000 7f6229400640 1 --2- 172.21.15.152:0/3802233858 >> [v2:172.21.15.152:6832/951802292,v1:172.21.15.152:6835/951802292] conn(0x562eedf94dc0 0x562eedfbe680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:54.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.692+0000 7f6229400640 1 --2- 172.21.15.152:0/3802233858 >> [v2:172.21.15.152:6832/951802292,v1:172.21.15.152:6835/951802292] conn(0x562eedf94dc0 0x562eedfbe680 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb95d65d6605a3ba server_cookie=5958c5f640f7bf00 in_seq=0 out_seq=0 2024-04-20T11:37:54.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.692+0000 7f6206000640 10 client.10632 ms_handle_connect on v2:172.21.15.152:6832/951802292 2024-04-20T11:37:54.697 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.693+0000 7f6206000640 1 -- 172.21.15.152:0/3802233858 <== mds.0 v2:172.21.15.152:6832/951802292 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562eedee63f0 con 0x562eedf94dc0 2024-04-20T11:37:54.697 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.693+0000 7f6206000640 10 client.10632 handle_command_reply: tid=0 2024-04-20T11:37:54.697 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.693+0000 7f622ebff740 1 client.10632 shutdown 2024-04-20T11:37:54.697 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.693+0000 7f622ebff740 1 -- 172.21.15.152:0/3802233858 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562eedf99d70 msgr2=0x562eedf91be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:54.697 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.693+0000 7f622ebff740 1 --2- 172.21.15.152:0/3802233858 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562eedf99d70 0x562eedf91be0 secure :-1 s=READY pgs=2327 cs=0 l=1 rev1=1 crypto rx=0x7f621c00a430 tx=0x7f621c0599b0 comp rx=0 tx=0).stop 2024-04-20T11:37:54.697 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.693+0000 7f622ebff740 1 -- 172.21.15.152:0/3802233858 shutdown_connections 2024-04-20T11:37:54.697 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.693+0000 7f622ebff740 1 --2- 172.21.15.152:0/3802233858 >> [v2:172.21.15.152:6832/951802292,v1:172.21.15.152:6835/951802292] conn(0x562eedf94dc0 0x562eedfbe680 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:54.697 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.693+0000 7f622ebff740 1 --2- 172.21.15.152:0/3802233858 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562eedf99d70 0x562eedf91be0 unknown :-1 s=CLOSED pgs=2327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:54.697 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.693+0000 7f622ebff740 1 --2- 172.21.15.152:0/3802233858 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x562eedfba460 0x562eedf916a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:54.697 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.693+0000 7f622ebff740 1 --2- 172.21.15.152:0/3802233858 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562eedf704e0 0x562eedf93940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:54.698 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.693+0000 7f6206000640 0 client.10632 ms_handle_reset on v2:172.21.15.152:6832/951802292 2024-04-20T11:37:54.698 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.693+0000 7f622ebff740 1 -- 172.21.15.152:0/3802233858 >> 172.21.15.152:0/3802233858 conn(0x562eedf6c930 msgr2=0x562eedf6d520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:54.698 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.693+0000 7f622ebff740 1 -- 172.21.15.152:0/3802233858 shutdown_connections 2024-04-20T11:37:54.698 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.694+0000 7f622ebff740 1 -- 172.21.15.152:0/3802233858 wait complete. 2024-04-20T11:37:54.698 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.694+0000 7f622ebff740 20 client.10632 trim_cache size 0 max 16384 2024-04-20T11:37:54.714 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.713+0000 7f622ebff740 1 Processor -- start 2024-04-20T11:37:54.714 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.713+0000 7f622ebff740 1 -- start start 2024-04-20T11:37:54.715 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.714+0000 7f622ebff740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x562eedf704e0 0x562eedf72790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:54.715 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.714+0000 7f622ebff740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562eedf99d70 0x562eee0359e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:54.715 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.714+0000 7f622ebff740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562eedfba460 0x562eee035f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:54.715 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.714+0000 7f622ebff740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f62240c6f60 con 0x562eedfba460 2024-04-20T11:37:54.715 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.714+0000 7f622ebff740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f62240c62c0 con 0x562eedf99d70 2024-04-20T11:37:54.715 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.714+0000 7f622ebff740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f62240c6910 con 0x562eedf704e0 2024-04-20T11:37:54.717 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.715+0000 7f6229e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562eedfba460 0x562eee035f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:54.717 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.715+0000 7f6229e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562eedfba460 0x562eee035f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35262/0 (socket says 172.21.15.152:35262) 2024-04-20T11:37:54.717 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.715+0000 7f6229e00640 1 -- 172.21.15.152:0/2245660075 learned_addr learned my addr 172.21.15.152:0/2245660075 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:54.717 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.715+0000 7f6229e00640 1 -- 172.21.15.152:0/2245660075 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x562eedf704e0 msgr2=0x562eedf72790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:54.717 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.715+0000 7f6229400640 1 --2- 172.21.15.152:0/2245660075 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x562eedf704e0 0x562eedf72790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:54.717 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.715+0000 7f6228a00640 1 --2- 172.21.15.152:0/2245660075 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562eedf99d70 0x562eee0359e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:54.717 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.715+0000 7f6229e00640 1 --2- 172.21.15.152:0/2245660075 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x562eedf704e0 0x562eedf72790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:54.717 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.715+0000 7f6229e00640 1 -- 172.21.15.152:0/2245660075 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562eedf99d70 msgr2=0x562eee0359e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:54.717 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.715+0000 7f6229e00640 1 --2- 172.21.15.152:0/2245660075 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562eedf99d70 0x562eee0359e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:54.718 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.715+0000 7f6229e00640 1 -- 172.21.15.152:0/2245660075 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562eedfb6590 con 0x562eedfba460 2024-04-20T11:37:54.718 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.715+0000 7f6229e00640 1 --2- 172.21.15.152:0/2245660075 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562eedfba460 0x562eee035f20 secure :-1 s=READY pgs=2328 cs=0 l=1 rev1=1 crypto rx=0x7f621c089e30 tx=0x7f621c009070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d2bd9fadf3ae051 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:54.718 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.716+0000 7f6211200640 1 -- 172.21.15.152:0/2245660075 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f621c058ad0 con 0x562eedfba460 2024-04-20T11:37:54.718 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.716+0000 7f6211200640 1 -- 172.21.15.152:0/2245660075 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f621c089a40 con 0x562eedfba460 2024-04-20T11:37:54.718 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.716+0000 7f6211200640 1 -- 172.21.15.152:0/2245660075 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f621c06db80 con 0x562eedfba460 2024-04-20T11:37:54.718 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.716+0000 7f622ebff740 1 -- 172.21.15.152:0/2245660075 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562eedfba460 msgr2=0x562eee035f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:54.718 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.716+0000 7f622ebff740 1 --2- 172.21.15.152:0/2245660075 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562eedfba460 0x562eee035f20 secure :-1 s=READY pgs=2328 cs=0 l=1 rev1=1 crypto rx=0x7f621c089e30 tx=0x7f621c009070 comp rx=0 tx=0).stop 2024-04-20T11:37:54.718 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.716+0000 7f622ebff740 1 -- 172.21.15.152:0/2245660075 shutdown_connections 2024-04-20T11:37:54.718 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.716+0000 7f622ebff740 1 --2- 172.21.15.152:0/2245660075 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562eedfba460 0x562eee035f20 unknown :-1 s=CLOSED pgs=2328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:54.718 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.716+0000 7f622ebff740 1 --2- 172.21.15.152:0/2245660075 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562eedf99d70 0x562eee0359e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:54.718 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.716+0000 7f622ebff740 1 --2- 172.21.15.152:0/2245660075 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x562eedf704e0 0x562eedf72790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:54.718 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.716+0000 7f622ebff740 1 -- 172.21.15.152:0/2245660075 >> 172.21.15.152:0/2245660075 conn(0x562eedf6c930 msgr2=0x562eedf6d250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:54.718 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.716+0000 7f622ebff740 1 -- 172.21.15.152:0/2245660075 shutdown_connections 2024-04-20T11:37:54.718 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.716+0000 7f622ebff740 1 -- 172.21.15.152:0/2245660075 wait complete. 2024-04-20T11:37:54.721 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.717+0000 7f622ebff740 1 -- start start 2024-04-20T11:37:54.722 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.717+0000 7f622ebff740 1 Processor -- start 2024-04-20T11:37:54.722 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.718+0000 7f622ebff740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562eedf704e0 0x562eee024030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:54.722 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.718+0000 7f622ebff740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x562eedf99d70 0x562eedf902a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:54.722 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.718+0000 7f622ebff740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562eedfba460 0x562eedfbe590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:54.722 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.718+0000 7f622ebff740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x562eedf94c40 con 0x562eedf704e0 2024-04-20T11:37:54.722 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.718+0000 7f622ebff740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x562eedf92440 con 0x562eedfba460 2024-04-20T11:37:54.722 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.718+0000 7f622ebff740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x562eedf95290 con 0x562eedf99d70 2024-04-20T11:37:54.722 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.718+0000 7f622ebff740 10 client.0 _ll_register_callbacks cb 0x7f621c089d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:37:54.722 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.719+0000 7f6229400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562eedf704e0 0x562eee024030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:54.722 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.719+0000 7f6229400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562eedf704e0 0x562eee024030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35274/0 (socket says 172.21.15.152:35274) 2024-04-20T11:37:54.722 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.719+0000 7f6229400640 1 -- 172.21.15.152:0/524161067 learned_addr learned my addr 172.21.15.152:0/524161067 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:54.722 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.719+0000 7f6228a00640 1 --2- 172.21.15.152:0/524161067 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x562eedf99d70 0x562eedf902a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:54.722 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.719+0000 7f6229e00640 1 --2- 172.21.15.152:0/524161067 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562eedfba460 0x562eedfbe590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:54.723 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.719+0000 7f6229400640 1 -- 172.21.15.152:0/524161067 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x562eedf99d70 msgr2=0x562eedf902a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:54.723 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.719+0000 7f6229400640 1 --2- 172.21.15.152:0/524161067 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x562eedf99d70 0x562eedf902a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:54.723 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.719+0000 7f6229400640 1 -- 172.21.15.152:0/524161067 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562eedfba460 msgr2=0x562eedfbe590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:54.723 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.719+0000 7f6229400640 1 --2- 172.21.15.152:0/524161067 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562eedfba460 0x562eedfbe590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:54.723 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.719+0000 7f6229400640 1 -- 172.21.15.152:0/524161067 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562eedfd2190 con 0x562eedf704e0 2024-04-20T11:37:54.723 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.719+0000 7f6228a00640 1 --2- 172.21.15.152:0/524161067 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x562eedf99d70 0x562eedf902a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:37:54.723 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.719+0000 7f6229e00640 1 --2- 172.21.15.152:0/524161067 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562eedfba460 0x562eedfbe590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:37:54.723 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.719+0000 7f6229400640 1 --2- 172.21.15.152:0/524161067 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562eedf704e0 0x562eee024030 secure :-1 s=READY pgs=2329 cs=0 l=1 rev1=1 crypto rx=0x7f621400e900 tx=0x7f6214004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d68e0af434adb4bf server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:54.723 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.719+0000 7f6206000640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-04-20T11:37:54.723 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.720+0000 7f6206000640 1 -- 172.21.15.152:0/524161067 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f621400f040 con 0x562eedf704e0 2024-04-20T11:37:54.723 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.720+0000 7f622ebff740 1 -- 172.21.15.152:0/524161067 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f621c089a40 con 0x562eedf704e0 2024-04-20T11:37:54.723 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.720+0000 7f6206000640 1 -- 172.21.15.152:0/524161067 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(167..167 src has 1..167) ==== 5709+0+0 (secure 0 0 0) 0x7f62140262b0 con 0x562eedf704e0 2024-04-20T11:37:54.723 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.720+0000 7f6206000640 1 -- 172.21.15.152:0/524161067 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=115) ==== 24+0+0 (secure 0 0 0) 0x7f6214036050 con 0x562eedf704e0 2024-04-20T11:37:54.723 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.720+0000 7f622ebff740 10 client.10638 fetch_fsmap learned FSMap version 115 2024-04-20T11:37:54.724 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.720+0000 7f622ebff740 1 -- 172.21.15.152:0/524161067 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=115,monmap=2+}) -- 0x562eedf9f310 con 0x562eedf704e0 2024-04-20T11:37:54.724 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.720+0000 7f6206000640 1 -- 172.21.15.152:0/524161067 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f621c089a40 con 0x562eedf704e0 2024-04-20T11:37:54.724 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.721+0000 7f6206000640 1 -- 172.21.15.152:0/524161067 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 115) ==== 3495+0+0 (secure 0 0 0) 0x7f6214039020 con 0x562eedf704e0 2024-04-20T11:37:54.724 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.721+0000 7f622ebff740 10 client.10638 fetch_fsmap finished waiting for FSMap version 115 2024-04-20T11:37:54.724 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.721+0000 7f622ebff740 10 client.10638 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-20T11:37:54.724 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.721+0000 7f622ebff740 20 client.10638 populate_metadata read hostname 'smithi152' 2024-04-20T11:37:54.724 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.721+0000 7f622ebff740 1 --2- 172.21.15.152:0/524161067 >> [v2:172.21.15.152:6832/951802292,v1:172.21.15.152:6835/951802292] conn(0x562eedfdb850 0x562eee0461d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:54.724 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.721+0000 7f622ebff740 4 client.10638 mds_command: new command op to 7339 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-20T11:37:54.724 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.721+0000 7f622ebff740 1 -- 172.21.15.152:0/524161067 --> [v2:172.21.15.152:6832/951802292,v1:172.21.15.152:6835/951802292] -- command(tid 0: {"prefix": "status"}) -- 0x562eedee0260 con 0x562eedfdb850 2024-04-20T11:37:54.724 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.721+0000 7f6228a00640 1 --2- 172.21.15.152:0/524161067 >> [v2:172.21.15.152:6832/951802292,v1:172.21.15.152:6835/951802292] conn(0x562eedfdb850 0x562eee0461d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:54.725 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.721+0000 7f6228a00640 1 --2- 172.21.15.152:0/524161067 >> [v2:172.21.15.152:6832/951802292,v1:172.21.15.152:6835/951802292] conn(0x562eedfdb850 0x562eee0461d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7086563044f17730 server_cookie=19690fb01b66363d in_seq=0 out_seq=0 2024-04-20T11:37:54.725 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.721+0000 7f6206000640 10 client.10638 ms_handle_connect on v2:172.21.15.152:6832/951802292 2024-04-20T11:37:54.726 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.722+0000 7f6206000640 1 -- 172.21.15.152:0/524161067 <== mds.0 v2:172.21.15.152:6832/951802292 1 ==== command_reply(tid 0: 0 ) ==== 8+0+328 (crc 0 0 0) 0x562eedee0260 con 0x562eedfdb850 2024-04-20T11:37:54.726 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.722+0000 7f6206000640 10 client.10638 handle_command_reply: tid=0 2024-04-20T11:37:54.726 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.722+0000 7f622ebff740 1 client.10638 shutdown 2024-04-20T11:37:54.726 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.722+0000 7f622ebff740 1 -- 172.21.15.152:0/524161067 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562eedf704e0 msgr2=0x562eee024030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:54.726 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.722+0000 7f622ebff740 1 --2- 172.21.15.152:0/524161067 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562eedf704e0 0x562eee024030 secure :-1 s=READY pgs=2329 cs=0 l=1 rev1=1 crypto rx=0x7f621400e900 tx=0x7f6214004730 comp rx=0 tx=0).stop 2024-04-20T11:37:54.726 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.722+0000 7f622ebff740 1 -- 172.21.15.152:0/524161067 shutdown_connections 2024-04-20T11:37:54.726 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.722+0000 7f622ebff740 1 --2- 172.21.15.152:0/524161067 >> [v2:172.21.15.152:6832/951802292,v1:172.21.15.152:6835/951802292] conn(0x562eedfdb850 0x562eee0461d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:54.726 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.722+0000 7f622ebff740 1 --2- 172.21.15.152:0/524161067 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562eedfba460 0x562eedfbe590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:54.726 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.722+0000 7f622ebff740 1 --2- 172.21.15.152:0/524161067 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x562eedf99d70 0x562eedf902a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:54.726 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.722+0000 7f622ebff740 1 --2- 172.21.15.152:0/524161067 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562eedf704e0 0x562eee024030 unknown :-1 s=CLOSED pgs=2329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:54.726 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.722+0000 7f6206000640 0 client.10638 ms_handle_reset on v2:172.21.15.152:6832/951802292 2024-04-20T11:37:54.726 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.722+0000 7f622ebff740 1 -- 172.21.15.152:0/524161067 >> 172.21.15.152:0/524161067 conn(0x562eedf6c930 msgr2=0x562eedfb9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:54.726 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.722+0000 7f622ebff740 1 -- 172.21.15.152:0/524161067 shutdown_connections 2024-04-20T11:37:54.727 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.722+0000 7f622ebff740 1 -- 172.21.15.152:0/524161067 wait complete. 2024-04-20T11:37:54.727 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.722+0000 7f622ebff740 20 client.10638 trim_cache size 0 max 16384 2024-04-20T11:37:54.727 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-04-20T11:37:54.727 INFO:teuthology.orchestra.run.smithi152.stdout: "cluster_fsid": "c8a72131-0d4f-42c7-892f-3f3af4d2bd3d", 2024-04-20T11:37:54.727 INFO:teuthology.orchestra.run.smithi152.stdout: "whoami": 0, 2024-04-20T11:37:54.727 INFO:teuthology.orchestra.run.smithi152.stdout: "id": 7339, 2024-04-20T11:37:54.727 INFO:teuthology.orchestra.run.smithi152.stdout: "want_state": "up:active", 2024-04-20T11:37:54.727 INFO:teuthology.orchestra.run.smithi152.stdout: "state": "up:active", 2024-04-20T11:37:54.727 INFO:teuthology.orchestra.run.smithi152.stdout: "fs_name": "cephfs", 2024-04-20T11:37:54.727 INFO:teuthology.orchestra.run.smithi152.stdout: "rank_uptime": 6.6172873909999996, 2024-04-20T11:37:54.727 INFO:teuthology.orchestra.run.smithi152.stdout: "mdsmap_epoch": 115, 2024-04-20T11:37:54.728 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_epoch": 167, 2024-04-20T11:37:54.728 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_epoch_barrier": 167, 2024-04-20T11:37:54.728 INFO:teuthology.orchestra.run.smithi152.stdout: "uptime": 12.110022073 2024-04-20T11:37:54.728 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-04-20T11:37:54.728 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.723+0000 7f622be00640 1 -- 172.21.15.152:0/77449319 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f620806d280 msgr2=0x7f620806f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:54.728 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.723+0000 7f622be00640 1 --2- 172.21.15.152:0/77449319 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f620806d280 0x7f620806f740 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f6214006fd0 tx=0x7f6214008040 comp rx=0 tx=0).stop 2024-04-20T11:37:54.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.723+0000 7f622be00640 1 -- 172.21.15.152:0/77449319 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f62240ce9a0 msgr2=0x7f62240c9620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:54.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.723+0000 7f622be00640 1 --2- 172.21.15.152:0/77449319 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f62240ce9a0 0x7f62240c9620 secure :-1 s=READY pgs=2325 cs=0 l=1 rev1=1 crypto rx=0x7f621c00c700 tx=0x7f621c00cbc0 comp rx=0 tx=0).stop 2024-04-20T11:37:54.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.723+0000 7f622be00640 1 -- 172.21.15.152:0/77449319 shutdown_connections 2024-04-20T11:37:54.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.723+0000 7f622be00640 1 --2- 172.21.15.152:0/77449319 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f620806d280 0x7f620806f740 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:54.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.723+0000 7f622be00640 1 --2- 172.21.15.152:0/77449319 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f62240ce9a0 0x7f62240c9620 unknown :-1 s=CLOSED pgs=2325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:54.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.723+0000 7f622be00640 1 --2- 172.21.15.152:0/77449319 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f622409b710 0x7f6224158920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:54.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.723+0000 7f622be00640 1 --2- 172.21.15.152:0/77449319 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f622406d6b0 0x7f62240cb180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:54.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.723+0000 7f622be00640 1 -- 172.21.15.152:0/77449319 >> 172.21.15.152:0/77449319 conn(0x7f62240b4040 msgr2=0x7f6224071840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:54.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.723+0000 7f622be00640 1 -- 172.21.15.152:0/77449319 shutdown_connections 2024-04-20T11:37:54.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.723+0000 7f622be00640 1 -- 172.21.15.152:0/77449319 wait complete. 2024-04-20T11:37:54.742 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-20T11:37:54.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.941+0000 7f3ac9e00640 1 Processor -- start 2024-04-20T11:37:54.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.941+0000 7f3ac9e00640 1 -- start start 2024-04-20T11:37:54.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.942+0000 7f3ac9e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ac40cd0d0 0x7f3ac40cd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:54.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.942+0000 7f3ac9e00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ac4146340 con 0x7f3ac4145f60 2024-04-20T11:37:54.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.942+0000 7f3ac9e00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ac4072d50 con 0x7f3ac4071fe0 2024-04-20T11:37:54.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.942+0000 7f3ac9e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f3ac406e9f0 con 0x7f3ac40cd0d0 2024-04-20T11:37:54.945 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.943+0000 7f3ac2a00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f3ac4145f60 0x7f3ac406ee80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:58506/0 (socket says 172.21.15.152:58506) 2024-04-20T11:37:54.945 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.943+0000 7f3ac2a00640 1 -- 172.21.15.152:0/3232011832 learned_addr learned my addr 172.21.15.152:0/3232011832 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:37:54.945 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.943+0000 7f3ac2000640 1 --2- 172.21.15.152:0/3232011832 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ac40cd0d0 0x7f3ac40cd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:54.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.943+0000 7f3ac1600640 1 -- 172.21.15.152:0/3232011832 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2909394438 0 0) 0x7f3ac4146340 con 0x7f3ac4145f60 2024-04-20T11:37:54.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.943+0000 7f3ac1600640 1 -- 172.21.15.152:0/3232011832 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3aa8003520 con 0x7f3ac4145f60 2024-04-20T11:37:54.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.944+0000 7f3ac1600640 1 -- 172.21.15.152:0/3232011832 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3627408438 0 0) 0x7f3aa8003520 con 0x7f3ac4145f60 2024-04-20T11:37:54.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.944+0000 7f3ac1600640 1 -- 172.21.15.152:0/3232011832 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ac40cd0d0 msgr2=0x7f3ac40cd550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:54.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.944+0000 7f3ac1600640 1 --2- 172.21.15.152:0/3232011832 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ac40cd0d0 0x7f3ac40cd550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:54.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.944+0000 7f3ac1600640 1 -- 172.21.15.152:0/3232011832 >> v1:172.21.15.175:6789/0 conn(0x7f3ac4071fe0 legacy=0x7f3ac4072470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:54.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.944+0000 7f3ac1600640 1 -- 172.21.15.152:0/3232011832 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ac406e0c0 con 0x7f3ac4145f60 2024-04-20T11:37:54.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.944+0000 7f3ac1600640 1 -- 172.21.15.152:0/3232011832 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f3ab4001670 con 0x7f3ac4145f60 2024-04-20T11:37:54.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.944+0000 7f3ac1600640 1 -- 172.21.15.152:0/3232011832 >> v1:172.21.15.152:6789/0 conn(0x7f3ac4145f60 legacy=0x7f3ac406ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:54.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.944+0000 7f3ac1600640 1 --2- 172.21.15.152:0/3232011832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3aa8003a50 0x7f3aa8003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:54.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.944+0000 7f3ac1600640 1 --2- 172.21.15.152:0/3232011832 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ac40cd0d0 0x7f3aa8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:54.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.944+0000 7f3ac1600640 1 --2- 172.21.15.152:0/3232011832 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3aa8004d40 0x7f3aa80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:54.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.944+0000 7f3ac1600640 1 -- 172.21.15.152:0/3232011832 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f3ac406e9f0 con 0x7f3aa8004d40 2024-04-20T11:37:54.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.944+0000 7f3ac1600640 1 -- 172.21.15.152:0/3232011832 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa800bad0 con 0x7f3aa8003a50 2024-04-20T11:37:54.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.944+0000 7f3ac1600640 1 -- 172.21.15.152:0/3232011832 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f3aa800bcd0 con 0x7f3ac40cd0d0 2024-04-20T11:37:54.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.944+0000 7f3ac3400640 1 --2- 172.21.15.152:0/3232011832 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3aa8004d40 0x7f3aa80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:54.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.944+0000 7f3ac2a00640 1 --2- 172.21.15.152:0/3232011832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3aa8003a50 0x7f3aa8003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:54.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.944+0000 7f3ac2000640 1 --2- 172.21.15.152:0/3232011832 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ac40cd0d0 0x7f3aa8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:54.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.944+0000 7f3ac3400640 1 -- 172.21.15.152:0/3232011832 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ac40cd0d0 msgr2=0x7f3aa8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:54.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.944+0000 7f3ac3400640 1 --2- 172.21.15.152:0/3232011832 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ac40cd0d0 0x7f3aa8004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:54.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.944+0000 7f3ac3400640 1 -- 172.21.15.152:0/3232011832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3aa8003a50 msgr2=0x7f3aa8003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:54.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.944+0000 7f3ac3400640 1 --2- 172.21.15.152:0/3232011832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3aa8003a50 0x7f3aa8003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:54.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.945+0000 7f3ac3400640 1 -- 172.21.15.152:0/3232011832 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3aa8003520 con 0x7f3aa8004d40 2024-04-20T11:37:54.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.946+0000 7f3ac3400640 1 --2- 172.21.15.152:0/3232011832 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3aa8004d40 0x7f3aa80051c0 secure :-1 s=READY pgs=2331 cs=0 l=1 rev1=1 crypto rx=0x7f3abc001e20 tx=0x7f3aa8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b8fe8f03bccd55b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:54.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.946+0000 7f3ac1600640 1 -- 172.21.15.152:0/3232011832 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3abc001a70 con 0x7f3aa8004d40 2024-04-20T11:37:54.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.946+0000 7f3ac1600640 1 -- 172.21.15.152:0/3232011832 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3abc003b70 con 0x7f3aa8004d40 2024-04-20T11:37:54.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.946+0000 7f3ac1600640 1 -- 172.21.15.152:0/3232011832 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3abc0050d0 con 0x7f3aa8004d40 2024-04-20T11:37:54.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.946+0000 7f3ac9e00640 1 -- 172.21.15.152:0/3232011832 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3aa8004d40 msgr2=0x7f3aa80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:54.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.946+0000 7f3ac9e00640 1 --2- 172.21.15.152:0/3232011832 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3aa8004d40 0x7f3aa80051c0 secure :-1 s=READY pgs=2331 cs=0 l=1 rev1=1 crypto rx=0x7f3abc001e20 tx=0x7f3aa8005830 comp rx=0 tx=0).stop 2024-04-20T11:37:54.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.946+0000 7f3ac3400640 1 -- 172.21.15.152:0/3232011832 reap_dead start 2024-04-20T11:37:54.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.946+0000 7f3ac9e00640 1 -- 172.21.15.152:0/3232011832 shutdown_connections 2024-04-20T11:37:54.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.946+0000 7f3ac9e00640 1 -- 172.21.15.152:0/3232011832 >> 172.21.15.152:0/3232011832 conn(0x7f3ac4067cf0 msgr2=0x7f3ac406b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:54.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.946+0000 7f3ac9e00640 1 -- 172.21.15.152:0/3232011832 shutdown_connections 2024-04-20T11:37:54.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.947+0000 7f3ac9e00640 1 -- 172.21.15.152:0/3232011832 wait complete. 2024-04-20T11:37:54.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.948+0000 7f3ac9e00640 1 Processor -- start 2024-04-20T11:37:54.950 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.948+0000 7f3ac9e00640 1 -- start start 2024-04-20T11:37:54.951 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.949+0000 7f3ac9e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3aa8004d40 0x7f3ac41f5c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:54.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.949+0000 7f3ac9e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ac40c8030 0x7f3ac4144800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:54.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.949+0000 7f3ac9e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ac40c84b0 0x7f3ac4144d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:54.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.949+0000 7f3ac9e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f3ac40cd550 con 0x7f3aa8004d40 2024-04-20T11:37:54.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.949+0000 7f3ac9e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3ac406e680 con 0x7f3ac40c84b0 2024-04-20T11:37:54.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.949+0000 7f3ac9e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f3ac40cd6d0 con 0x7f3ac40c8030 2024-04-20T11:37:54.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.950+0000 7f3ac2a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3aa8004d40 0x7f3ac41f5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:54.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.950+0000 7f3ac2a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3aa8004d40 0x7f3ac41f5c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35296/0 (socket says 172.21.15.152:35296) 2024-04-20T11:37:54.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.950+0000 7f3ac2a00640 1 -- 172.21.15.152:0/788732836 learned_addr learned my addr 172.21.15.152:0/788732836 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:54.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.950+0000 7f3ac2000640 1 --2- 172.21.15.152:0/788732836 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ac40c8030 0x7f3ac4144800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:54.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.950+0000 7f3ac3400640 1 --2- 172.21.15.152:0/788732836 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ac40c84b0 0x7f3ac4144d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:54.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.950+0000 7f3ac2a00640 1 -- 172.21.15.152:0/788732836 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ac40c8030 msgr2=0x7f3ac4144800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:54.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.950+0000 7f3ac2a00640 1 --2- 172.21.15.152:0/788732836 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ac40c8030 0x7f3ac4144800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:54.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.950+0000 7f3ac2a00640 1 -- 172.21.15.152:0/788732836 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ac40c84b0 msgr2=0x7f3ac4144d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:54.953 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.950+0000 7f3ac2a00640 1 --2- 172.21.15.152:0/788732836 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ac40c84b0 0x7f3ac4144d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:54.953 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.950+0000 7f3ac2a00640 1 -- 172.21.15.152:0/788732836 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ac4145280 con 0x7f3aa8004d40 2024-04-20T11:37:54.953 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.950+0000 7f3ac2a00640 1 --2- 172.21.15.152:0/788732836 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3aa8004d40 0x7f3ac41f5c90 secure :-1 s=READY pgs=2332 cs=0 l=1 rev1=1 crypto rx=0x7f3ab400e730 tx=0x7f3ab400ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:54.953 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.951+0000 7f3abb400640 1 -- 172.21.15.152:0/788732836 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ab400ee40 con 0x7f3aa8004d40 2024-04-20T11:37:54.953 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.951+0000 7f3abb400640 1 -- 172.21.15.152:0/788732836 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ab4018070 con 0x7f3aa8004d40 2024-04-20T11:37:54.953 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.951+0000 7f3ac9e00640 1 -- 172.21.15.152:0/788732836 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ac40c89a0 con 0x7f3aa8004d40 2024-04-20T11:37:54.953 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.951+0000 7f3abb400640 1 -- 172.21.15.152:0/788732836 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ab400cb10 con 0x7f3aa8004d40 2024-04-20T11:37:54.953 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.951+0000 7f3ac9e00640 1 -- 172.21.15.152:0/788732836 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ac40c8f60 con 0x7f3aa8004d40 2024-04-20T11:37:54.953 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.952+0000 7f3ac9e00640 1 -- 172.21.15.152:0/788732836 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ac40689f0 con 0x7f3aa8004d40 2024-04-20T11:37:54.953 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.953+0000 7f3abb400640 1 -- 172.21.15.152:0/788732836 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 26) ==== 91907+0+0 (secure 0 0 0) 0x7f3ab400ccb0 con 0x7f3aa8004d40 2024-04-20T11:37:54.954 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.954+0000 7f3abb400640 1 --2- 172.21.15.152:0/788732836 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f3aa406d280 0x7f3aa406f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:54.954 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.954+0000 7f3abb400640 1 -- 172.21.15.152:0/788732836 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 5709+0+0 (secure 0 0 0) 0x7f3ab408a170 con 0x7f3aa8004d40 2024-04-20T11:37:54.955 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.955+0000 7f3ac2000640 1 --2- 172.21.15.152:0/788732836 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f3aa406d280 0x7f3aa406f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:54.956 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.956+0000 7f3ac2000640 1 --2- 172.21.15.152:0/788732836 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f3aa406d280 0x7f3aa406f740 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f3ac41e9520 tx=0x7f3ab0008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:54.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:54.959+0000 7f3abb400640 1 -- 172.21.15.152:0/788732836 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f3ab405a1a0 con 0x7f3aa8004d40 2024-04-20T11:37:55.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:55.202+0000 7f3ac9e00640 1 -- 172.21.15.152:0/788732836 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f3ac40ca8c0 con 0x7f3aa8004d40 2024-04-20T11:37:55.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:55.202+0000 7f3abb400640 1 -- 172.21.15.152:0/788732836 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v115) ==== 53+0+73 (secure 0 0 0) 0x7f3ab4059aa0 con 0x7f3aa8004d40 2024-04-20T11:37:55.203 INFO:teuthology.orchestra.run.smithi152.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-20T11:37:55.206 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:55.205+0000 7f3ac9e00640 1 -- 172.21.15.152:0/788732836 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f3aa406d280 msgr2=0x7f3aa406f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:55.206 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:55.205+0000 7f3ac9e00640 1 --2- 172.21.15.152:0/788732836 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f3aa406d280 0x7f3aa406f740 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f3ac41e9520 tx=0x7f3ab0008040 comp rx=0 tx=0).stop 2024-04-20T11:37:55.206 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:55.205+0000 7f3ac9e00640 1 -- 172.21.15.152:0/788732836 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3aa8004d40 msgr2=0x7f3ac41f5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:55.206 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:55.205+0000 7f3ac9e00640 1 --2- 172.21.15.152:0/788732836 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3aa8004d40 0x7f3ac41f5c90 secure :-1 s=READY pgs=2332 cs=0 l=1 rev1=1 crypto rx=0x7f3ab400e730 tx=0x7f3ab400ebf0 comp rx=0 tx=0).stop 2024-04-20T11:37:55.206 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:55.205+0000 7f3ac9e00640 1 -- 172.21.15.152:0/788732836 shutdown_connections 2024-04-20T11:37:55.206 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:55.205+0000 7f3ac9e00640 1 --2- 172.21.15.152:0/788732836 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f3aa406d280 0x7f3aa406f740 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:55.206 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:55.205+0000 7f3ac9e00640 1 --2- 172.21.15.152:0/788732836 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ac40c84b0 0x7f3ac4144d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:55.206 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:55.205+0000 7f3ac9e00640 1 --2- 172.21.15.152:0/788732836 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ac40c8030 0x7f3ac4144800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:55.206 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:55.205+0000 7f3ac9e00640 1 --2- 172.21.15.152:0/788732836 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3aa8004d40 0x7f3ac41f5c90 unknown :-1 s=CLOSED pgs=2332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:55.206 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:55.205+0000 7f3ac9e00640 1 -- 172.21.15.152:0/788732836 >> 172.21.15.152:0/788732836 conn(0x7f3ac4067cf0 msgr2=0x7f3ac41601a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:55.206 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:55.205+0000 7f3ac9e00640 1 -- 172.21.15.152:0/788732836 shutdown_connections 2024-04-20T11:37:55.207 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:55.205+0000 7f3ac9e00640 1 -- 172.21.15.152:0/788732836 wait complete. 2024-04-20T11:37:55.225 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-20T11:37:55.225 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-20T11:37:55.225 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi152.front.sepia.ceph.com 2024-04-20T11:37:55.225 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-20T11:37:55.225 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-20T11:37:55.225 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-20T11:37:55.225 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-20T11:37:55.226 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-20T11:37:55.226 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-20T11:37:55.226 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:37:55.226 DEBUG:teuthology.orchestra.run.smithi152:> ip addr 2024-04-20T11:37:55.276 INFO:teuthology.orchestra.run.smithi152.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-20T11:37:55.276 INFO:teuthology.orchestra.run.smithi152.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-20T11:37:55.276 INFO:teuthology.orchestra.run.smithi152.stdout: inet 127.0.0.1/8 scope host lo 2024-04-20T11:37:55.276 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:37:55.276 INFO:teuthology.orchestra.run.smithi152.stdout: inet6 ::1/128 scope host 2024-04-20T11:37:55.276 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:37:55.276 INFO:teuthology.orchestra.run.smithi152.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T11:37:55.277 INFO:teuthology.orchestra.run.smithi152.stdout: link/ether 0c:c4:7a:da:6c:44 brd ff:ff:ff:ff:ff:ff 2024-04-20T11:37:55.277 INFO:teuthology.orchestra.run.smithi152.stdout: altname enp7s0f0 2024-04-20T11:37:55.277 INFO:teuthology.orchestra.run.smithi152.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T11:37:55.277 INFO:teuthology.orchestra.run.smithi152.stdout: link/ether 0c:c4:7a:da:6c:45 brd ff:ff:ff:ff:ff:ff 2024-04-20T11:37:55.277 INFO:teuthology.orchestra.run.smithi152.stdout: altname enp7s0f1 2024-04-20T11:37:55.277 INFO:teuthology.orchestra.run.smithi152.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T11:37:55.277 INFO:teuthology.orchestra.run.smithi152.stdout: link/ether 0c:c4:7a:88:6d:e2 brd ff:ff:ff:ff:ff:ff 2024-04-20T11:37:55.277 INFO:teuthology.orchestra.run.smithi152.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-20T11:37:55.277 INFO:teuthology.orchestra.run.smithi152.stdout: link/ether 0c:c4:7a:88:6d:e3 brd ff:ff:ff:ff:ff:ff 2024-04-20T11:37:55.277 INFO:teuthology.orchestra.run.smithi152.stdout: inet 172.21.15.152/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-20T11:37:55.277 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft 42131sec preferred_lft 42131sec 2024-04-20T11:37:55.277 INFO:teuthology.orchestra.run.smithi152.stdout: inet6 fe80::ec4:7aff:fe88:6de3/64 scope link 2024-04-20T11:37:55.277 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:37:55.277 INFO:teuthology.orchestra.run.smithi152.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-20T11:37:55.277 INFO:teuthology.orchestra.run.smithi152.stdout: link/ether 06:3e:9a:d3:49:1e brd ff:ff:ff:ff:ff:ff 2024-04-20T11:37:55.278 INFO:teuthology.orchestra.run.smithi152.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-20T11:37:55.278 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:37:55.278 INFO:teuthology.orchestra.run.smithi152.stdout: inet6 fe80::a019:d8ff:fe70:1185/64 scope link 2024-04-20T11:37:55.278 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:37:55.278 INFO:teuthology.orchestra.run.smithi152.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-20T11:37:55.278 INFO:teuthology.orchestra.run.smithi152.stdout: link/ether 16:e5:08:78:20:47 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T11:37:55.278 INFO:teuthology.orchestra.run.smithi152.stdout: inet6 fe80::14e5:8ff:fe78:2047/64 scope link 2024-04-20T11:37:55.278 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:37:55.278 INFO:teuthology.orchestra.run.smithi152.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-20T11:37:55.278 INFO:teuthology.orchestra.run.smithi152.stdout: link/ether 06:3e:9a:d3:49:1e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T11:37:55.278 INFO:teuthology.orchestra.run.smithi152.stdout: inet6 fe80::43e:9aff:fed3:491e/64 scope link 2024-04-20T11:37:55.278 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:37:55.279 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:37:55.279 DEBUG:teuthology.orchestra.run.smithi152:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-20T11:37:55.356 INFO:teuthology.orchestra.run.smithi152.stdout:1 2024-04-20T11:37:55.361 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:37:55.361 DEBUG:teuthology.orchestra.run.smithi152:> route 2024-04-20T11:37:55.414 INFO:teuthology.orchestra.run.smithi152.stdout:Kernel IP routing table 2024-04-20T11:37:55.415 INFO:teuthology.orchestra.run.smithi152.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-20T11:37:55.415 INFO:teuthology.orchestra.run.smithi152.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-20T11:37:55.415 INFO:teuthology.orchestra.run.smithi152.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-20T11:37:55.415 INFO:teuthology.orchestra.run.smithi152.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-20T11:37:55.415 DEBUG:teuthology.orchestra.run.smithi152:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T11:37:55.416 DEBUG:teuthology.orchestra.run.smithi152:> set -e 2024-04-20T11:37:55.416 DEBUG:teuthology.orchestra.run.smithi152:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-20T11:37:55.416 DEBUG:teuthology.orchestra.run.smithi152:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-20T11:37:55.416 DEBUG:teuthology.orchestra.run.smithi152:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-20T11:37:55.416 DEBUG:teuthology.orchestra.run.smithi152:> ') 2024-04-20T11:37:55.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T11:37:55.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T11:37:55.627 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:37:55.628 DEBUG:teuthology.orchestra.run.smithi152:> ip netns list 2024-04-20T11:37:55.675 INFO:teuthology.orchestra.run.smithi152.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-20T11:37:55.676 INFO:teuthology.orchestra.run.smithi152.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-20T11:37:55.677 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-20T11:37:55.677 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:37:55.677 DEBUG:teuthology.orchestra.run.smithi152:> sudo ip link set brx.0 up 2024-04-20T11:37:55.759 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-20T11:37:55.759 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T11:37:55.759 DEBUG:teuthology.orchestra.run.smithi152:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-20T11:37:55.812 INFO:teuthology.orchestra.run.smithi152.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-20T11:37:55.812 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T11:37:55.812 DEBUG:teuthology.orchestra.run.smithi152:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-20T11:37:55.863 DEBUG:teuthology.orchestra.run.smithi152:> sudo modprobe fuse 2024-04-20T11:37:55.946 DEBUG:teuthology.orchestra.run.smithi152:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T11:37:55.997 INFO:teuthology.orchestra.run.smithi152.stdout:/proc 2024-04-20T11:37:55.997 INFO:teuthology.orchestra.run.smithi152.stdout:/sys 2024-04-20T11:37:55.997 INFO:teuthology.orchestra.run.smithi152.stdout:/dev 2024-04-20T11:37:55.997 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/security 2024-04-20T11:37:55.997 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/shm 2024-04-20T11:37:55.997 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/pts 2024-04-20T11:37:55.997 INFO:teuthology.orchestra.run.smithi152.stdout:/run 2024-04-20T11:37:55.998 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/cgroup 2024-04-20T11:37:55.998 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/pstore 2024-04-20T11:37:55.998 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/bpf 2024-04-20T11:37:55.998 INFO:teuthology.orchestra.run.smithi152.stdout:/ 2024-04-20T11:37:55.998 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/selinux 2024-04-20T11:37:55.998 INFO:teuthology.orchestra.run.smithi152.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T11:37:55.998 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/mqueue 2024-04-20T11:37:55.998 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/hugepages 2024-04-20T11:37:55.998 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/debug 2024-04-20T11:37:55.998 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/tracing 2024-04-20T11:37:55.998 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/fuse/connections 2024-04-20T11:37:55.998 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/config 2024-04-20T11:37:55.998 INFO:teuthology.orchestra.run.smithi152.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T11:37:55.999 INFO:teuthology.orchestra.run.smithi152.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T11:37:55.999 INFO:teuthology.orchestra.run.smithi152.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T11:37:55.999 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T11:37:55.999 INFO:teuthology.orchestra.run.smithi152.stdout:/run/user/1000 2024-04-20T11:37:55.999 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph 2024-04-20T11:37:55.999 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph/osd/ceph-0 2024-04-20T11:37:55.999 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph/osd/ceph-1 2024-04-20T11:37:55.999 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph/osd/ceph-2 2024-04-20T11:37:55.999 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph/osd/ceph-3 2024-04-20T11:37:55.999 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns 2024-04-20T11:37:55.999 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T11:37:55.999 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T11:37:55.999 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T11:37:55.999 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T11:37:56.000 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:37:56.000 DEBUG:teuthology.orchestra.run.smithi152:> ls /sys/fs/fuse/connections 2024-04-20T11:37:56.056 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-20T11:37:56.056 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-20T11:37:56.063 DEBUG:teuthology.orchestra.run.smithi152:> sudo modprobe fuse 2024-04-20T11:37:56.138 DEBUG:teuthology.orchestra.run.smithi152:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T11:37:56.177 INFO:teuthology.orchestra.run.smithi152.stdout:/proc 2024-04-20T11:37:56.178 INFO:teuthology.orchestra.run.smithi152.stdout:/sys 2024-04-20T11:37:56.178 INFO:teuthology.orchestra.run.smithi152.stdout:/dev 2024-04-20T11:37:56.178 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/security 2024-04-20T11:37:56.178 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/shm 2024-04-20T11:37:56.178 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/pts 2024-04-20T11:37:56.178 INFO:teuthology.orchestra.run.smithi152.stdout:/run 2024-04-20T11:37:56.178 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/cgroup 2024-04-20T11:37:56.178 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/pstore 2024-04-20T11:37:56.178 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/bpf 2024-04-20T11:37:56.178 INFO:teuthology.orchestra.run.smithi152.stdout:/ 2024-04-20T11:37:56.178 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/selinux 2024-04-20T11:37:56.178 INFO:teuthology.orchestra.run.smithi152.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T11:37:56.178 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/mqueue 2024-04-20T11:37:56.179 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/hugepages 2024-04-20T11:37:56.179 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/debug 2024-04-20T11:37:56.179 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/tracing 2024-04-20T11:37:56.179 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/fuse/connections 2024-04-20T11:37:56.179 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/config 2024-04-20T11:37:56.179 INFO:teuthology.orchestra.run.smithi152.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T11:37:56.179 INFO:teuthology.orchestra.run.smithi152.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T11:37:56.179 INFO:teuthology.orchestra.run.smithi152.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T11:37:56.179 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T11:37:56.179 INFO:teuthology.orchestra.run.smithi152.stdout:/run/user/1000 2024-04-20T11:37:56.179 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph 2024-04-20T11:37:56.179 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph/osd/ceph-0 2024-04-20T11:37:56.179 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph/osd/ceph-1 2024-04-20T11:37:56.179 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph/osd/ceph-2 2024-04-20T11:37:56.179 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph/osd/ceph-3 2024-04-20T11:37:56.180 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns 2024-04-20T11:37:56.180 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T11:37:56.180 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T11:37:56.180 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T11:37:56.180 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T11:37:56.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:37:56.181 DEBUG:teuthology.orchestra.run.smithi152:> ls /sys/fs/fuse/connections 2024-04-20T11:37:56.233 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi152.stderr:2024-04-20T11:37:56.232+0000 7f8447a42480 -1 init, newargv = 0x561865305580 newargc=15 2024-04-20T11:37:56.233 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi152.stderr:ceph-fuse[74912]: starting ceph client 2024-04-20T11:37:56.242 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi152.stderr:ceph-fuse[74912]: starting fuse 2024-04-20T11:37:56.260 INFO:teuthology.orchestra.run.smithi152.stdout:42 2024-04-20T11:37:56.261 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-04-20T11:37:56.261 DEBUG:teuthology.orchestra.run.smithi152:> sudo stdin-killer -- python3 -c ' 2024-04-20T11:37:56.261 DEBUG:teuthology.orchestra.run.smithi152:> import glob 2024-04-20T11:37:56.261 DEBUG:teuthology.orchestra.run.smithi152:> import re 2024-04-20T11:37:56.261 DEBUG:teuthology.orchestra.run.smithi152:> import os 2024-04-20T11:37:56.262 DEBUG:teuthology.orchestra.run.smithi152:> import subprocess 2024-04-20T11:37:56.262 DEBUG:teuthology.orchestra.run.smithi152:> 2024-04-20T11:37:56.262 DEBUG:teuthology.orchestra.run.smithi152:> def _find_admin_socket(client_name): 2024-04-20T11:37:56.262 DEBUG:teuthology.orchestra.run.smithi152:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-20T11:37:56.262 DEBUG:teuthology.orchestra.run.smithi152:> files = glob.glob(asok_path) 2024-04-20T11:37:56.262 DEBUG:teuthology.orchestra.run.smithi152:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-20T11:37:56.262 DEBUG:teuthology.orchestra.run.smithi152:> 2024-04-20T11:37:56.262 DEBUG:teuthology.orchestra.run.smithi152:> # Given a non-glob path, it better be there 2024-04-20T11:37:56.262 DEBUG:teuthology.orchestra.run.smithi152:> if "*" not in asok_path: 2024-04-20T11:37:56.262 DEBUG:teuthology.orchestra.run.smithi152:> assert(len(files) == 1) 2024-04-20T11:37:56.262 DEBUG:teuthology.orchestra.run.smithi152:> return files[0] 2024-04-20T11:37:56.263 DEBUG:teuthology.orchestra.run.smithi152:> 2024-04-20T11:37:56.263 DEBUG:teuthology.orchestra.run.smithi152:> for f in files: 2024-04-20T11:37:56.263 DEBUG:teuthology.orchestra.run.smithi152:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-20T11:37:56.263 DEBUG:teuthology.orchestra.run.smithi152:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-20T11:37:56.263 DEBUG:teuthology.orchestra.run.smithi152:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-20T11:37:56.263 DEBUG:teuthology.orchestra.run.smithi152:> contents = proc_f.read() 2024-04-20T11:37:56.263 DEBUG:teuthology.orchestra.run.smithi152:> if mountpoint in contents: 2024-04-20T11:37:56.263 DEBUG:teuthology.orchestra.run.smithi152:> return f 2024-04-20T11:37:56.263 DEBUG:teuthology.orchestra.run.smithi152:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-20T11:37:56.263 DEBUG:teuthology.orchestra.run.smithi152:> 2024-04-20T11:37:56.264 DEBUG:teuthology.orchestra.run.smithi152:> print(_find_admin_socket("client.0")) 2024-04-20T11:37:56.264 DEBUG:teuthology.orchestra.run.smithi152:> ' 2024-04-20T11:37:56.457 INFO:teuthology.orchestra.run.smithi152.stdout:/var/run/ceph/ceph-client.0.74912.asok 2024-04-20T11:37:56.461 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T11:37:56.474 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.74912.asok 2024-04-20T11:37:56.474 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:37:56.474 DEBUG:teuthology.orchestra.run.smithi152:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.74912.asok status 2024-04-20T11:37:56.702 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-04-20T11:37:56.703 INFO:teuthology.orchestra.run.smithi152.stdout: "metadata": { 2024-04-20T11:37:56.703 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_sha1": "a8c84f5997950fd7d008204839253291d52e6eda", 2024-04-20T11:37:56.703 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", 2024-04-20T11:37:56.703 INFO:teuthology.orchestra.run.smithi152.stdout: "entity_id": "0", 2024-04-20T11:37:56.703 INFO:teuthology.orchestra.run.smithi152.stdout: "hostname": "smithi152", 2024-04-20T11:37:56.703 INFO:teuthology.orchestra.run.smithi152.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-20T11:37:56.703 INFO:teuthology.orchestra.run.smithi152.stdout: "pid": "74912", 2024-04-20T11:37:56.703 INFO:teuthology.orchestra.run.smithi152.stdout: "root": "/" 2024-04-20T11:37:56.703 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:37:56.703 INFO:teuthology.orchestra.run.smithi152.stdout: "dentry_count": 0, 2024-04-20T11:37:56.703 INFO:teuthology.orchestra.run.smithi152.stdout: "dentry_pinned_count": 0, 2024-04-20T11:37:56.704 INFO:teuthology.orchestra.run.smithi152.stdout: "id": 10650, 2024-04-20T11:37:56.704 INFO:teuthology.orchestra.run.smithi152.stdout: "inst": { 2024-04-20T11:37:56.704 INFO:teuthology.orchestra.run.smithi152.stdout: "name": { 2024-04-20T11:37:56.704 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "client", 2024-04-20T11:37:56.704 INFO:teuthology.orchestra.run.smithi152.stdout: "num": 10650 2024-04-20T11:37:56.704 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:37:56.704 INFO:teuthology.orchestra.run.smithi152.stdout: "addr": { 2024-04-20T11:37:56.704 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "v1", 2024-04-20T11:37:56.704 INFO:teuthology.orchestra.run.smithi152.stdout: "addr": "192.168.0.1:0", 2024-04-20T11:37:56.704 INFO:teuthology.orchestra.run.smithi152.stdout: "nonce": 1128512104 2024-04-20T11:37:56.704 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:37:56.704 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:37:56.704 INFO:teuthology.orchestra.run.smithi152.stdout: "addr": { 2024-04-20T11:37:56.704 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "v1", 2024-04-20T11:37:56.705 INFO:teuthology.orchestra.run.smithi152.stdout: "addr": "192.168.0.1:0", 2024-04-20T11:37:56.705 INFO:teuthology.orchestra.run.smithi152.stdout: "nonce": 1128512104 2024-04-20T11:37:56.705 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-20T11:37:56.705 INFO:teuthology.orchestra.run.smithi152.stdout: "inst_str": "client.10650 192.168.0.1:0/1128512104", 2024-04-20T11:37:56.705 INFO:teuthology.orchestra.run.smithi152.stdout: "addr_str": "192.168.0.1:0/1128512104", 2024-04-20T11:37:56.705 INFO:teuthology.orchestra.run.smithi152.stdout: "inode_count": 1, 2024-04-20T11:37:56.705 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_epoch": 115, 2024-04-20T11:37:56.705 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_epoch": 167, 2024-04-20T11:37:56.705 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_epoch_barrier": 0, 2024-04-20T11:37:56.705 INFO:teuthology.orchestra.run.smithi152.stdout: "blocklisted": false, 2024-04-20T11:37:56.705 INFO:teuthology.orchestra.run.smithi152.stdout: "fs_name": "cephfs" 2024-04-20T11:37:56.705 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-04-20T11:37:56.717 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:37:56.717 DEBUG:teuthology.orchestra.run.smithi152:> stat --file-system '--printf=%T 2024-04-20T11:37:56.717 DEBUG:teuthology.orchestra.run.smithi152:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-20T11:37:56.772 INFO:teuthology.orchestra.run.smithi152.stdout:fuseblk 2024-04-20T11:37:56.772 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-20T11:37:56.772 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:37:56.772 DEBUG:teuthology.orchestra.run.smithi152:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-20T11:37:56.855 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-20T11:37:57.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:57.058+0000 7f712f200640 1 Processor -- start 2024-04-20T11:37:57.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:57.058+0000 7f712f200640 1 -- start start 2024-04-20T11:37:57.061 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:57.060+0000 7f712f200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f713015d1e0 0x7f7130160b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:57.061 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:57.060+0000 7f712f200640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71300ce8c0 con 0x7f7130071cd0 2024-04-20T11:37:57.061 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:57.060+0000 7f712f200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71301600b0 con 0x7f713015d5c0 2024-04-20T11:37:57.061 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:57.060+0000 7f712f200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f7130160390 con 0x7f713015d1e0 2024-04-20T11:37:57.061 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:57.060+0000 7f712e800640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f7130071cd0 0x7f7130072120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:34694/0 (socket says 172.21.15.152:34694) 2024-04-20T11:37:57.061 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:57.060+0000 7f712e800640 1 -- 172.21.15.152:0/2971863915 learned_addr learned my addr 172.21.15.152:0/2971863915 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:37:57.063 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:57.061+0000 7f712de00640 1 --2- 172.21.15.152:0/2971863915 >> v2:172.21.15.175:3300/0 conn(0x7f713015d1e0 0x7f7130160b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:57.063 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:57.061+0000 7f712d400640 1 -- 172.21.15.152:0/2971863915 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 49012579 0 0) 0x7f71300ce8c0 con 0x7f7130071cd0 2024-04-20T11:37:57.063 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:57.061+0000 7f712d400640 1 -- 172.21.15.152:0/2971863915 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7114003520 con 0x7f7130071cd0 2024-04-20T11:37:57.063 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:57.061+0000 7f712d400640 1 -- 172.21.15.152:0/2971863915 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1550543613 0 0) 0x7f7114003520 con 0x7f7130071cd0 2024-04-20T11:37:57.063 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:57.061+0000 7f712d400640 1 -- 172.21.15.152:0/2971863915 >> v2:172.21.15.175:3300/0 conn(0x7f713015d1e0 msgr2=0x7f7130160b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:57.063 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:57.061+0000 7f712d400640 1 --2- 172.21.15.152:0/2971863915 >> v2:172.21.15.175:3300/0 conn(0x7f713015d1e0 0x7f7130160b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:57.063 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:57.061+0000 7f712d400640 1 -- 172.21.15.152:0/2971863915 >> v1:172.21.15.175:6789/0 conn(0x7f713015d5c0 legacy=0x7f71300cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:57.064 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:57.061+0000 7f712d400640 1 -- 172.21.15.152:0/2971863915 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7130160670 con 0x7f7130071cd0 2024-04-20T11:37:57.064 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:57.061+0000 7f712d400640 1 -- 172.21.15.152:0/2971863915 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f71280019f0 con 0x7f7130071cd0 2024-04-20T11:37:57.064 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:57.061+0000 7f712d400640 1 -- 172.21.15.152:0/2971863915 >> v1:172.21.15.152:6789/0 conn(0x7f7130071cd0 legacy=0x7f7130072120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:57.064 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:57.061+0000 7f712d400640 1 --2- 172.21.15.152:0/2971863915 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f71140039b0 0x7f7114003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:57.064 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:57.061+0000 7f712d400640 1 --2- 172.21.15.152:0/2971863915 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7114004800 0x7f7114004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:57.064 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:57.061+0000 7f712d400640 1 --2- 172.21.15.152:0/2971863915 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f71140051c0 0x7f711400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:57.064 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:57.062+0000 7f712d400640 1 -- 172.21.15.152:0/2971863915 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f7130160390 con 0x7f71140039b0 2024-04-20T11:37:57.064 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:57.062+0000 7f712d400640 1 -- 172.21.15.152:0/2971863915 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f711400c060 con 0x7f7114004800 2024-04-20T11:37:57.064 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:57.062+0000 7f712d400640 1 -- 172.21.15.152:0/2971863915 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f711400c260 con 0x7f71140051c0 2024-04-20T11:37:57.064 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:57.062+0000 7f712de00640 1 --2- 172.21.15.152:0/2971863915 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f71140039b0 0x7f7114003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:57.064 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:57.062+0000 7f712de00640 1 -- 172.21.15.152:0/2971863915 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f71140051c0 msgr2=0x7f711400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:57.064 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:57.062+0000 7f712de00640 1 --2- 172.21.15.152:0/2971863915 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f71140051c0 0x7f711400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:57.064 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:57.062+0000 7f712de00640 1 -- 172.21.15.152:0/2971863915 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7114004800 msgr2=0x7f7114004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:57.064 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:57.062+0000 7f712de00640 1 --2- 172.21.15.152:0/2971863915 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7114004800 0x7f7114004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:57.065 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:57.062+0000 7f712de00640 1 -- 172.21.15.152:0/2971863915 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7114003520 con 0x7f71140039b0 2024-04-20T11:37:57.065 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:57.062+0000 7f712e800640 1 -- 172.21.15.152:0/2971863915 reap_dead start 2024-04-20T11:37:57.066 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:57.063+0000 7f712de00640 1 --2- 172.21.15.152:0/2971863915 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f71140039b0 0x7f7114003dd0 secure :-1 s=READY pgs=2336 cs=0 l=1 rev1=1 crypto rx=0x7f7120009ac0 tx=0x7f7114005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1565154de91f6a25 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:57.066 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:57.063+0000 7f712d400640 1 -- 172.21.15.152:0/2971863915 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7120039070 con 0x7f71140039b0 2024-04-20T11:37:57.066 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:57.064+0000 7f712d400640 1 -- 172.21.15.152:0/2971863915 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f712002c6b0 con 0x7f71140039b0 2024-04-20T11:37:57.066 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:57.064+0000 7f712d400640 1 -- 172.21.15.152:0/2971863915 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7120009690 con 0x7f71140039b0 2024-04-20T11:37:57.066 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:57.064+0000 7f712f200640 1 -- 172.21.15.152:0/2971863915 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f71140039b0 msgr2=0x7f7114003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:57.066 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:57.064+0000 7f712f200640 1 --2- 172.21.15.152:0/2971863915 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f71140039b0 0x7f7114003dd0 secure :-1 s=READY pgs=2336 cs=0 l=1 rev1=1 crypto rx=0x7f7120009ac0 tx=0x7f7114005830 comp rx=0 tx=0).stop 2024-04-20T11:37:57.066 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:57.064+0000 7f712f200640 1 -- 172.21.15.152:0/2971863915 shutdown_connections 2024-04-20T11:37:57.066 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:57.064+0000 7f712f200640 1 --2- 172.21.15.152:0/2971863915 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f71140039b0 0x7f7114003dd0 unknown :-1 s=CLOSED pgs=2336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:57.067 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:57.064+0000 7f712f200640 1 -- 172.21.15.152:0/2971863915 >> 172.21.15.152:0/2971863915 conn(0x7f7130067ae0 msgr2=0x7f713006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:57.067 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:57.064+0000 7f712f200640 1 -- 172.21.15.152:0/2971863915 shutdown_connections 2024-04-20T11:37:57.067 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:57.064+0000 7f712f200640 1 -- 172.21.15.152:0/2971863915 wait complete. 2024-04-20T11:37:57.069 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:57.066+0000 7f712f200640 1 Processor -- start 2024-04-20T11:37:57.069 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:57.066+0000 7f712f200640 1 -- start start 2024-04-20T11:37:57.069 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:57.067+0000 7f712f200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71140039b0 0x7f71301e6840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:57.070 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:57.067+0000 7f712f200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7130154a10 0x7f71301558a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:57.070 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:57.067+0000 7f712f200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f71301517d0 0x7f7130151c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:57.070 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:57.067+0000 7f712f200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f7130160d10 con 0x7f71301517d0 2024-04-20T11:37:57.070 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:57.067+0000 7f712f200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7130160b90 con 0x7f71140039b0 2024-04-20T11:37:57.070 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:57.067+0000 7f712f200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7130160e90 con 0x7f7130154a10 2024-04-20T11:37:57.070 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:57.067+0000 7f712e800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f71301517d0 0x7f7130151c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:57.070 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:57.067+0000 7f712e800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f71301517d0 0x7f7130151c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:55738/0 (socket says 172.21.15.152:55738) 2024-04-20T11:37:57.070 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:57.067+0000 7f712e800640 1 -- 172.21.15.152:0/2031742822 learned_addr learned my addr 172.21.15.152:0/2031742822 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:57.071 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:57.068+0000 7f7127e00640 1 --2- 172.21.15.152:0/2031742822 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7130154a10 0x7f71301558a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:57.071 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:57.068+0000 7f712de00640 1 --2- 172.21.15.152:0/2031742822 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71140039b0 0x7f71301e6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:57.071 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:57.068+0000 7f712e800640 1 -- 172.21.15.152:0/2031742822 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7130154a10 msgr2=0x7f71301558a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:57.071 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:57.068+0000 7f712e800640 1 --2- 172.21.15.152:0/2031742822 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7130154a10 0x7f71301558a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:57.071 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:57.068+0000 7f712e800640 1 -- 172.21.15.152:0/2031742822 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71140039b0 msgr2=0x7f71301e6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:57.071 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:57.068+0000 7f712e800640 1 --2- 172.21.15.152:0/2031742822 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71140039b0 0x7f71301e6840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:57.071 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:57.068+0000 7f712e800640 1 -- 172.21.15.152:0/2031742822 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71301521a0 con 0x7f71301517d0 2024-04-20T11:37:57.071 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:57.068+0000 7f712e800640 1 --2- 172.21.15.152:0/2031742822 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f71301517d0 0x7f7130151c60 secure :-1 s=READY pgs=2337 cs=0 l=1 rev1=1 crypto rx=0x7f713015d9a0 tx=0x7f712800cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:57.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:57.068+0000 7f7126000640 1 -- 172.21.15.152:0/2031742822 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f712800ce50 con 0x7f71301517d0 2024-04-20T11:37:57.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:57.068+0000 7f7126000640 1 -- 172.21.15.152:0/2031742822 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f712800a610 con 0x7f71301517d0 2024-04-20T11:37:57.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:57.068+0000 7f7126000640 1 -- 172.21.15.152:0/2031742822 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f712800aa00 con 0x7f71301517d0 2024-04-20T11:37:57.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:57.068+0000 7f712f200640 1 -- 172.21.15.152:0/2031742822 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71301e7190 con 0x7f71301517d0 2024-04-20T11:37:57.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:57.069+0000 7f712f200640 1 -- 172.21.15.152:0/2031742822 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71301e7750 con 0x7f71301517d0 2024-04-20T11:37:57.073 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:57.070+0000 7f712f200640 1 -- 172.21.15.152:0/2031742822 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7130075e60 con 0x7f71301517d0 2024-04-20T11:37:57.073 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:57.071+0000 7f7126000640 1 -- 172.21.15.152:0/2031742822 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 26) ==== 91907+0+0 (secure 0 0 0) 0x7f7128011070 con 0x7f71301517d0 2024-04-20T11:37:57.074 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:57.072+0000 7f7126000640 1 --2- 172.21.15.152:0/2031742822 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f710c06d280 0x7f710c06f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:57.074 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:57.072+0000 7f7126000640 1 -- 172.21.15.152:0/2031742822 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 5709+0+0 (secure 0 0 0) 0x7f7128089100 con 0x7f71301517d0 2024-04-20T11:37:57.074 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:57.072+0000 7f712de00640 1 --2- 172.21.15.152:0/2031742822 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f710c06d280 0x7f710c06f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:57.074 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:57.074+0000 7f712de00640 1 --2- 172.21.15.152:0/2031742822 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f710c06d280 0x7f710c06f740 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f7120036000 tx=0x7f7120030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:57.077 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:57.076+0000 7f7126000640 1 -- 172.21.15.152:0/2031742822 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f7128059130 con 0x7f71301517d0 2024-04-20T11:37:57.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:57.321+0000 7f712f200640 1 -- 172.21.15.152:0/2031742822 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f71300af7b0 con 0x7f71301517d0 2024-04-20T11:37:57.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:57.321+0000 7f7126000640 1 -- 172.21.15.152:0/2031742822 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v115) ==== 53+0+73 (secure 0 0 0) 0x7f7128058a30 con 0x7f71301517d0 2024-04-20T11:37:57.322 INFO:teuthology.orchestra.run.smithi152.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-20T11:37:57.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:57.324+0000 7f712f200640 1 -- 172.21.15.152:0/2031742822 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f710c06d280 msgr2=0x7f710c06f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:57.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:57.324+0000 7f712f200640 1 --2- 172.21.15.152:0/2031742822 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f710c06d280 0x7f710c06f740 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f7120036000 tx=0x7f7120030000 comp rx=0 tx=0).stop 2024-04-20T11:37:57.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:57.324+0000 7f712f200640 1 -- 172.21.15.152:0/2031742822 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f71301517d0 msgr2=0x7f7130151c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:57.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:57.324+0000 7f712f200640 1 --2- 172.21.15.152:0/2031742822 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f71301517d0 0x7f7130151c60 secure :-1 s=READY pgs=2337 cs=0 l=1 rev1=1 crypto rx=0x7f713015d9a0 tx=0x7f712800cc00 comp rx=0 tx=0).stop 2024-04-20T11:37:57.326 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:57.324+0000 7f712f200640 1 -- 172.21.15.152:0/2031742822 shutdown_connections 2024-04-20T11:37:57.326 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:57.324+0000 7f712f200640 1 --2- 172.21.15.152:0/2031742822 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f710c06d280 0x7f710c06f740 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:57.326 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:57.324+0000 7f712f200640 1 --2- 172.21.15.152:0/2031742822 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f71301517d0 0x7f7130151c60 unknown :-1 s=CLOSED pgs=2337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:57.326 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:57.324+0000 7f712f200640 1 --2- 172.21.15.152:0/2031742822 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7130154a10 0x7f71301558a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:57.326 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:57.324+0000 7f712f200640 1 --2- 172.21.15.152:0/2031742822 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71140039b0 0x7f71301e6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:57.326 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:57.324+0000 7f712f200640 1 -- 172.21.15.152:0/2031742822 >> 172.21.15.152:0/2031742822 conn(0x7f7130067ae0 msgr2=0x7f713006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:57.326 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:57.324+0000 7f712f200640 1 -- 172.21.15.152:0/2031742822 shutdown_connections 2024-04-20T11:37:57.326 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:57.324+0000 7f712f200640 1 -- 172.21.15.152:0/2031742822 wait complete. 2024-04-20T11:37:57.344 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-20T11:37:57.344 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-20T11:37:57.344 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi175.front.sepia.ceph.com 2024-04-20T11:37:57.344 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-20T11:37:57.344 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-20T11:37:57.344 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-20T11:37:57.344 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-20T11:37:57.344 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-20T11:37:57.344 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-20T11:37:57.344 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:37:57.345 DEBUG:teuthology.orchestra.run.smithi175:> ip addr 2024-04-20T11:37:57.403 INFO:teuthology.orchestra.run.smithi175.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-20T11:37:57.403 INFO:teuthology.orchestra.run.smithi175.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-20T11:37:57.403 INFO:teuthology.orchestra.run.smithi175.stdout: inet 127.0.0.1/8 scope host lo 2024-04-20T11:37:57.404 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:37:57.404 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 ::1/128 scope host 2024-04-20T11:37:57.404 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:37:57.404 INFO:teuthology.orchestra.run.smithi175.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T11:37:57.404 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:d9:39:ee brd ff:ff:ff:ff:ff:ff 2024-04-20T11:37:57.404 INFO:teuthology.orchestra.run.smithi175.stdout: altname enp7s0f0 2024-04-20T11:37:57.404 INFO:teuthology.orchestra.run.smithi175.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T11:37:57.404 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:d9:39:ef brd ff:ff:ff:ff:ff:ff 2024-04-20T11:37:57.404 INFO:teuthology.orchestra.run.smithi175.stdout: altname enp7s0f1 2024-04-20T11:37:57.404 INFO:teuthology.orchestra.run.smithi175.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T11:37:57.404 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:88:71:f0 brd ff:ff:ff:ff:ff:ff 2024-04-20T11:37:57.404 INFO:teuthology.orchestra.run.smithi175.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-20T11:37:57.405 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:88:71:f1 brd ff:ff:ff:ff:ff:ff 2024-04-20T11:37:57.405 INFO:teuthology.orchestra.run.smithi175.stdout: inet 172.21.15.175/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-20T11:37:57.405 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft 42104sec preferred_lft 42104sec 2024-04-20T11:37:57.405 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::ec4:7aff:fe88:71f1/64 scope link 2024-04-20T11:37:57.405 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:37:57.405 INFO:teuthology.orchestra.run.smithi175.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-20T11:37:57.405 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 02:d6:c6:31:26:70 brd ff:ff:ff:ff:ff:ff 2024-04-20T11:37:57.405 INFO:teuthology.orchestra.run.smithi175.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-20T11:37:57.405 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:37:57.405 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::5091:d5ff:fee7:54b3/64 scope link 2024-04-20T11:37:57.405 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:37:57.405 INFO:teuthology.orchestra.run.smithi175.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-20T11:37:57.405 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 02:d6:c6:31:26:70 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-20T11:37:57.405 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::d6:c6ff:fe31:2670/64 scope link 2024-04-20T11:37:57.406 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:37:57.406 INFO:teuthology.orchestra.run.smithi175.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-20T11:37:57.406 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 9e:14:f3:e2:18:1e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-20T11:37:57.406 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::9c14:f3ff:fee2:181e/64 scope link 2024-04-20T11:37:57.406 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:37:57.407 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:37:57.407 DEBUG:teuthology.orchestra.run.smithi175:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-20T11:37:57.492 INFO:teuthology.orchestra.run.smithi175.stdout:1 2024-04-20T11:37:57.497 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:37:57.497 DEBUG:teuthology.orchestra.run.smithi175:> route 2024-04-20T11:37:57.550 INFO:teuthology.orchestra.run.smithi175.stdout:Kernel IP routing table 2024-04-20T11:37:57.550 INFO:teuthology.orchestra.run.smithi175.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-20T11:37:57.550 INFO:teuthology.orchestra.run.smithi175.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-20T11:37:57.550 INFO:teuthology.orchestra.run.smithi175.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-20T11:37:57.550 INFO:teuthology.orchestra.run.smithi175.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-20T11:37:57.551 DEBUG:teuthology.orchestra.run.smithi175:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T11:37:57.551 DEBUG:teuthology.orchestra.run.smithi175:> set -e 2024-04-20T11:37:57.552 DEBUG:teuthology.orchestra.run.smithi175:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-20T11:37:57.552 DEBUG:teuthology.orchestra.run.smithi175:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-20T11:37:57.552 DEBUG:teuthology.orchestra.run.smithi175:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-20T11:37:57.552 DEBUG:teuthology.orchestra.run.smithi175:> ') 2024-04-20T11:37:57.656 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:37:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T11:37:57.751 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:37:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T11:37:57.760 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:37:57.760 DEBUG:teuthology.orchestra.run.smithi175:> ip netns list 2024-04-20T11:37:57.803 INFO:teuthology.orchestra.run.smithi175.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-20T11:37:57.803 INFO:teuthology.orchestra.run.smithi175.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-20T11:37:57.803 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-20T11:37:57.804 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:37:57.804 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip link set brx.0 up 2024-04-20T11:37:57.930 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-04-20T11:37:57.930 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T11:37:57.931 DEBUG:teuthology.orchestra.run.smithi175:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-20T11:37:57.984 INFO:teuthology.orchestra.run.smithi175.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-20T11:37:57.984 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T11:37:57.984 DEBUG:teuthology.orchestra.run.smithi175:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-20T11:37:58.035 DEBUG:teuthology.orchestra.run.smithi175:> sudo modprobe fuse 2024-04-20T11:37:58.112 DEBUG:teuthology.orchestra.run.smithi175:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T11:37:58.162 INFO:teuthology.orchestra.run.smithi175.stdout:/proc 2024-04-20T11:37:58.162 INFO:teuthology.orchestra.run.smithi175.stdout:/sys 2024-04-20T11:37:58.162 INFO:teuthology.orchestra.run.smithi175.stdout:/dev 2024-04-20T11:37:58.162 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/security 2024-04-20T11:37:58.163 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/shm 2024-04-20T11:37:58.163 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/pts 2024-04-20T11:37:58.163 INFO:teuthology.orchestra.run.smithi175.stdout:/run 2024-04-20T11:37:58.163 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/cgroup 2024-04-20T11:37:58.163 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/pstore 2024-04-20T11:37:58.163 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/bpf 2024-04-20T11:37:58.163 INFO:teuthology.orchestra.run.smithi175.stdout:/ 2024-04-20T11:37:58.163 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/selinux 2024-04-20T11:37:58.163 INFO:teuthology.orchestra.run.smithi175.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T11:37:58.163 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/mqueue 2024-04-20T11:37:58.163 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/debug 2024-04-20T11:37:58.163 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/hugepages 2024-04-20T11:37:58.163 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/tracing 2024-04-20T11:37:58.163 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/fuse/connections 2024-04-20T11:37:58.164 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/config 2024-04-20T11:37:58.164 INFO:teuthology.orchestra.run.smithi175.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T11:37:58.164 INFO:teuthology.orchestra.run.smithi175.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T11:37:58.164 INFO:teuthology.orchestra.run.smithi175.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T11:37:58.164 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T11:37:58.164 INFO:teuthology.orchestra.run.smithi175.stdout:/run/user/1000 2024-04-20T11:37:58.164 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph 2024-04-20T11:37:58.164 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-4 2024-04-20T11:37:58.164 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-5 2024-04-20T11:37:58.164 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-6 2024-04-20T11:37:58.164 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-7 2024-04-20T11:37:58.164 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns 2024-04-20T11:37:58.164 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-20T11:37:58.164 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-20T11:37:58.165 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-20T11:37:58.165 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-20T11:37:58.165 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:37:58.166 DEBUG:teuthology.orchestra.run.smithi175:> ls /sys/fs/fuse/connections 2024-04-20T11:37:58.212 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-20T11:37:58.212 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-04-20T11:37:58.259 DEBUG:teuthology.orchestra.run.smithi175:> sudo modprobe fuse 2024-04-20T11:37:58.329 DEBUG:teuthology.orchestra.run.smithi175:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T11:37:58.371 INFO:teuthology.orchestra.run.smithi175.stdout:/proc 2024-04-20T11:37:58.371 INFO:teuthology.orchestra.run.smithi175.stdout:/sys 2024-04-20T11:37:58.371 INFO:teuthology.orchestra.run.smithi175.stdout:/dev 2024-04-20T11:37:58.371 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/security 2024-04-20T11:37:58.371 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/shm 2024-04-20T11:37:58.371 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/pts 2024-04-20T11:37:58.371 INFO:teuthology.orchestra.run.smithi175.stdout:/run 2024-04-20T11:37:58.371 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/cgroup 2024-04-20T11:37:58.371 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/pstore 2024-04-20T11:37:58.372 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/bpf 2024-04-20T11:37:58.372 INFO:teuthology.orchestra.run.smithi175.stdout:/ 2024-04-20T11:37:58.372 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/selinux 2024-04-20T11:37:58.372 INFO:teuthology.orchestra.run.smithi175.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T11:37:58.372 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/mqueue 2024-04-20T11:37:58.372 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/debug 2024-04-20T11:37:58.372 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/hugepages 2024-04-20T11:37:58.372 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/tracing 2024-04-20T11:37:58.372 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/fuse/connections 2024-04-20T11:37:58.372 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/config 2024-04-20T11:37:58.372 INFO:teuthology.orchestra.run.smithi175.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T11:37:58.372 INFO:teuthology.orchestra.run.smithi175.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T11:37:58.372 INFO:teuthology.orchestra.run.smithi175.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T11:37:58.373 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T11:37:58.373 INFO:teuthology.orchestra.run.smithi175.stdout:/run/user/1000 2024-04-20T11:37:58.373 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph 2024-04-20T11:37:58.373 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-4 2024-04-20T11:37:58.373 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-5 2024-04-20T11:37:58.373 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-6 2024-04-20T11:37:58.373 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-7 2024-04-20T11:37:58.373 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns 2024-04-20T11:37:58.373 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-20T11:37:58.373 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-20T11:37:58.373 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-20T11:37:58.373 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-20T11:37:58.374 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:37:58.374 DEBUG:teuthology.orchestra.run.smithi175:> ls /sys/fs/fuse/connections 2024-04-20T11:37:58.435 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi175.stderr:2024-04-20T11:37:58.436+0000 7f8c30a0d480 -1 init, newargv = 0x558ab0f708d0 newargc=15 2024-04-20T11:37:58.435 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi175.stderr:ceph-fuse[37216]: starting ceph client 2024-04-20T11:37:58.444 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi175.stderr:ceph-fuse[37216]: starting fuse 2024-04-20T11:37:58.459 INFO:teuthology.orchestra.run.smithi175.stdout:42 2024-04-20T11:37:58.459 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-04-20T11:37:58.460 DEBUG:teuthology.orchestra.run.smithi175:> sudo stdin-killer -- python3 -c ' 2024-04-20T11:37:58.460 DEBUG:teuthology.orchestra.run.smithi175:> import glob 2024-04-20T11:37:58.460 DEBUG:teuthology.orchestra.run.smithi175:> import re 2024-04-20T11:37:58.460 DEBUG:teuthology.orchestra.run.smithi175:> import os 2024-04-20T11:37:58.460 DEBUG:teuthology.orchestra.run.smithi175:> import subprocess 2024-04-20T11:37:58.460 DEBUG:teuthology.orchestra.run.smithi175:> 2024-04-20T11:37:58.460 DEBUG:teuthology.orchestra.run.smithi175:> def _find_admin_socket(client_name): 2024-04-20T11:37:58.460 DEBUG:teuthology.orchestra.run.smithi175:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-04-20T11:37:58.460 DEBUG:teuthology.orchestra.run.smithi175:> files = glob.glob(asok_path) 2024-04-20T11:37:58.460 DEBUG:teuthology.orchestra.run.smithi175:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-04-20T11:37:58.460 DEBUG:teuthology.orchestra.run.smithi175:> 2024-04-20T11:37:58.460 DEBUG:teuthology.orchestra.run.smithi175:> # Given a non-glob path, it better be there 2024-04-20T11:37:58.460 DEBUG:teuthology.orchestra.run.smithi175:> if "*" not in asok_path: 2024-04-20T11:37:58.460 DEBUG:teuthology.orchestra.run.smithi175:> assert(len(files) == 1) 2024-04-20T11:37:58.461 DEBUG:teuthology.orchestra.run.smithi175:> return files[0] 2024-04-20T11:37:58.461 DEBUG:teuthology.orchestra.run.smithi175:> 2024-04-20T11:37:58.461 DEBUG:teuthology.orchestra.run.smithi175:> for f in files: 2024-04-20T11:37:58.461 DEBUG:teuthology.orchestra.run.smithi175:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-20T11:37:58.461 DEBUG:teuthology.orchestra.run.smithi175:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-20T11:37:58.461 DEBUG:teuthology.orchestra.run.smithi175:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-20T11:37:58.461 DEBUG:teuthology.orchestra.run.smithi175:> contents = proc_f.read() 2024-04-20T11:37:58.461 DEBUG:teuthology.orchestra.run.smithi175:> if mountpoint in contents: 2024-04-20T11:37:58.461 DEBUG:teuthology.orchestra.run.smithi175:> return f 2024-04-20T11:37:58.461 DEBUG:teuthology.orchestra.run.smithi175:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-20T11:37:58.461 DEBUG:teuthology.orchestra.run.smithi175:> 2024-04-20T11:37:58.461 DEBUG:teuthology.orchestra.run.smithi175:> print(_find_admin_socket("client.1")) 2024-04-20T11:37:58.461 DEBUG:teuthology.orchestra.run.smithi175:> ' 2024-04-20T11:37:58.646 INFO:teuthology.orchestra.run.smithi175.stdout:/var/run/ceph/ceph-client.1.37216.asok 2024-04-20T11:37:58.651 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T11:37:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T11:37:58.662 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.37216.asok 2024-04-20T11:37:58.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:37:58.663 DEBUG:teuthology.orchestra.run.smithi175:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.37216.asok status 2024-04-20T11:37:58.888 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-04-20T11:37:58.888 INFO:teuthology.orchestra.run.smithi175.stdout: "metadata": { 2024-04-20T11:37:58.888 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_sha1": "a8c84f5997950fd7d008204839253291d52e6eda", 2024-04-20T11:37:58.888 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", 2024-04-20T11:37:58.888 INFO:teuthology.orchestra.run.smithi175.stdout: "entity_id": "1", 2024-04-20T11:37:58.888 INFO:teuthology.orchestra.run.smithi175.stdout: "hostname": "smithi175", 2024-04-20T11:37:58.888 INFO:teuthology.orchestra.run.smithi175.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-04-20T11:37:58.888 INFO:teuthology.orchestra.run.smithi175.stdout: "pid": "37216", 2024-04-20T11:37:58.888 INFO:teuthology.orchestra.run.smithi175.stdout: "root": "/" 2024-04-20T11:37:58.889 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-20T11:37:58.889 INFO:teuthology.orchestra.run.smithi175.stdout: "dentry_count": 0, 2024-04-20T11:37:58.889 INFO:teuthology.orchestra.run.smithi175.stdout: "dentry_pinned_count": 0, 2024-04-20T11:37:58.889 INFO:teuthology.orchestra.run.smithi175.stdout: "id": 7438, 2024-04-20T11:37:58.889 INFO:teuthology.orchestra.run.smithi175.stdout: "inst": { 2024-04-20T11:37:58.889 INFO:teuthology.orchestra.run.smithi175.stdout: "name": { 2024-04-20T11:37:58.889 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "client", 2024-04-20T11:37:58.889 INFO:teuthology.orchestra.run.smithi175.stdout: "num": 7438 2024-04-20T11:37:58.889 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-20T11:37:58.889 INFO:teuthology.orchestra.run.smithi175.stdout: "addr": { 2024-04-20T11:37:58.889 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "v1", 2024-04-20T11:37:58.889 INFO:teuthology.orchestra.run.smithi175.stdout: "addr": "192.168.0.1:0", 2024-04-20T11:37:58.889 INFO:teuthology.orchestra.run.smithi175.stdout: "nonce": 384034471 2024-04-20T11:37:58.889 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-04-20T11:37:58.890 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-20T11:37:58.890 INFO:teuthology.orchestra.run.smithi175.stdout: "addr": { 2024-04-20T11:37:58.890 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "v1", 2024-04-20T11:37:58.890 INFO:teuthology.orchestra.run.smithi175.stdout: "addr": "192.168.0.1:0", 2024-04-20T11:37:58.890 INFO:teuthology.orchestra.run.smithi175.stdout: "nonce": 384034471 2024-04-20T11:37:58.890 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-20T11:37:58.890 INFO:teuthology.orchestra.run.smithi175.stdout: "inst_str": "client.7438 192.168.0.1:0/384034471", 2024-04-20T11:37:58.890 INFO:teuthology.orchestra.run.smithi175.stdout: "addr_str": "192.168.0.1:0/384034471", 2024-04-20T11:37:58.890 INFO:teuthology.orchestra.run.smithi175.stdout: "inode_count": 1, 2024-04-20T11:37:58.890 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_epoch": 115, 2024-04-20T11:37:58.890 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_epoch": 167, 2024-04-20T11:37:58.890 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_epoch_barrier": 0, 2024-04-20T11:37:58.890 INFO:teuthology.orchestra.run.smithi175.stdout: "blocklisted": false, 2024-04-20T11:37:58.890 INFO:teuthology.orchestra.run.smithi175.stdout: "fs_name": "cephfs" 2024-04-20T11:37:58.890 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-04-20T11:37:58.902 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:37:58.902 DEBUG:teuthology.orchestra.run.smithi175:> stat --file-system '--printf=%T 2024-04-20T11:37:58.902 DEBUG:teuthology.orchestra.run.smithi175:> ' -- /home/ubuntu/cephtest/mnt.1 2024-04-20T11:37:58.951 INFO:teuthology.orchestra.run.smithi175.stdout:fuseblk 2024-04-20T11:37:58.951 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-04-20T11:37:58.951 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:37:58.952 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-20T11:37:59.031 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-04-20T11:37:59.253 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:59.247+0000 7f744d800640 1 Processor -- start 2024-04-20T11:37:59.253 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:59.248+0000 7f744d800640 1 -- start start 2024-04-20T11:37:59.253 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:59.249+0000 7f744d800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74480d18e0 0x7f74480d3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:59.253 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:59.249+0000 7f744d800640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7448162b10 con 0x7f744815e900 2024-04-20T11:37:59.253 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:59.249+0000 7f744d800640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7448162df0 con 0x7f7448162610 2024-04-20T11:37:59.253 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:59.249+0000 7f744d800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f74480d4130 con 0x7f74480d18e0 2024-04-20T11:37:59.254 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:59.249+0000 7f7446a00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f744815e900 0x7f74480b7af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:34708/0 (socket says 172.21.15.152:34708) 2024-04-20T11:37:59.254 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:59.249+0000 7f7446a00640 1 -- 172.21.15.152:0/603169082 learned_addr learned my addr 172.21.15.152:0/603169082 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:37:59.254 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:59.250+0000 7f7446000640 1 -- 172.21.15.152:0/603169082 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 128844180 0 0) 0x7f7448162b10 con 0x7f744815e900 2024-04-20T11:37:59.254 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:59.250+0000 7f7446000640 1 -- 172.21.15.152:0/603169082 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f742c003520 con 0x7f744815e900 2024-04-20T11:37:59.254 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:59.250+0000 7f7447400640 1 --2- 172.21.15.152:0/603169082 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74480d18e0 0x7f74480d3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:59.254 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:59.250+0000 7f7446000640 1 -- 172.21.15.152:0/603169082 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2528153912 0 0) 0x7f742c003520 con 0x7f744815e900 2024-04-20T11:37:59.254 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:59.250+0000 7f7446000640 1 -- 172.21.15.152:0/603169082 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74480d18e0 msgr2=0x7f74480d3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:59.254 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:59.250+0000 7f7446000640 1 --2- 172.21.15.152:0/603169082 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74480d18e0 0x7f74480d3bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:59.254 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:59.250+0000 7f7446000640 1 -- 172.21.15.152:0/603169082 >> v1:172.21.15.175:6789/0 conn(0x7f7448162610 legacy=0x7f74480d11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:59.254 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:59.250+0000 7f7446000640 1 -- 172.21.15.152:0/603169082 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74481630d0 con 0x7f744815e900 2024-04-20T11:37:59.254 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:59.250+0000 7f7446000640 1 -- 172.21.15.152:0/603169082 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f7438001690 con 0x7f744815e900 2024-04-20T11:37:59.254 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:59.250+0000 7f7446000640 1 -- 172.21.15.152:0/603169082 >> v1:172.21.15.152:6789/0 conn(0x7f744815e900 legacy=0x7f74480b7af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:59.254 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:59.250+0000 7f7446000640 1 --2- 172.21.15.152:0/603169082 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f742c003a50 0x7f742c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:59.254 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:59.250+0000 7f7446000640 1 --2- 172.21.15.152:0/603169082 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f742c004800 0x7f742c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:59.255 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:59.250+0000 7f7446000640 1 --2- 172.21.15.152:0/603169082 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74480d18e0 0x7f742c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:59.255 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:59.250+0000 7f7446000640 1 -- 172.21.15.152:0/603169082 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f74480d4130 con 0x7f742c003a50 2024-04-20T11:37:59.255 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:59.250+0000 7f7446000640 1 -- 172.21.15.152:0/603169082 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f742c005520 con 0x7f742c004800 2024-04-20T11:37:59.255 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:59.250+0000 7f7446000640 1 -- 172.21.15.152:0/603169082 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f742c00bf00 con 0x7f74480d18e0 2024-04-20T11:37:59.255 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:59.250+0000 7f7446a00640 1 --2- 172.21.15.152:0/603169082 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f742c003a50 0x7f742c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:59.255 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:59.251+0000 7f743fe00640 1 --2- 172.21.15.152:0/603169082 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f742c004800 0x7f742c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:59.255 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:59.251+0000 7f7447400640 1 --2- 172.21.15.152:0/603169082 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74480d18e0 0x7f742c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:59.255 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:59.251+0000 7f7446a00640 1 -- 172.21.15.152:0/603169082 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74480d18e0 msgr2=0x7f742c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:59.255 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:59.251+0000 7f7446a00640 1 --2- 172.21.15.152:0/603169082 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74480d18e0 0x7f742c00b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:59.255 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:59.251+0000 7f7446a00640 1 -- 172.21.15.152:0/603169082 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f742c004800 msgr2=0x7f742c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:37:59.255 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:59.251+0000 7f7446a00640 1 --2- 172.21.15.152:0/603169082 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f742c004800 0x7f742c004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:59.255 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:59.251+0000 7f7446a00640 1 -- 172.21.15.152:0/603169082 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f742c003520 con 0x7f742c003a50 2024-04-20T11:37:59.255 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:59.252+0000 7f7446a00640 1 --2- 172.21.15.152:0/603169082 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f742c003a50 0x7f742c003e70 secure :-1 s=READY pgs=2339 cs=0 l=1 rev1=1 crypto rx=0x7f743800c0e0 tx=0x7f742c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28aca0e3183b57a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:59.255 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:59.252+0000 7f7446000640 1 -- 172.21.15.152:0/603169082 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7438007c40 con 0x7f742c003a50 2024-04-20T11:37:59.256 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:59.252+0000 7f7446000640 1 -- 172.21.15.152:0/603169082 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7438030070 con 0x7f742c003a50 2024-04-20T11:37:59.256 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:59.252+0000 7f7446000640 1 -- 172.21.15.152:0/603169082 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7438005d10 con 0x7f742c003a50 2024-04-20T11:37:59.258 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:59.253+0000 7f744d800640 1 -- 172.21.15.152:0/603169082 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f742c003a50 msgr2=0x7f742c003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:59.258 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:59.253+0000 7f744d800640 1 --2- 172.21.15.152:0/603169082 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f742c003a50 0x7f742c003e70 secure :-1 s=READY pgs=2339 cs=0 l=1 rev1=1 crypto rx=0x7f743800c0e0 tx=0x7f742c005830 comp rx=0 tx=0).stop 2024-04-20T11:37:59.258 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:59.253+0000 7f7447400640 1 -- 172.21.15.152:0/603169082 reap_dead start 2024-04-20T11:37:59.258 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:59.253+0000 7f744d800640 1 -- 172.21.15.152:0/603169082 shutdown_connections 2024-04-20T11:37:59.258 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:59.253+0000 7f744d800640 1 -- 172.21.15.152:0/603169082 >> 172.21.15.152:0/603169082 conn(0x7f74480676c0 msgr2=0x7f7448067ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:59.258 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:59.253+0000 7f744d800640 1 -- 172.21.15.152:0/603169082 shutdown_connections 2024-04-20T11:37:59.258 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:59.253+0000 7f744d800640 1 -- 172.21.15.152:0/603169082 wait complete. 2024-04-20T11:37:59.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:59.255+0000 7f744d800640 1 Processor -- start 2024-04-20T11:37:59.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:59.255+0000 7f744d800640 1 -- start start 2024-04-20T11:37:59.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:59.256+0000 7f744d800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f742c003a50 0x7f74480c7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:59.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:59.256+0000 7f744d800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74481618c0 0x7f74480cbbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:59.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:59.256+0000 7f744d800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f74480c8aa0 0x7f74480c8f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:59.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:59.256+0000 7f744d800640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f74480d4560 con 0x7f74480c8aa0 2024-04-20T11:37:59.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:59.256+0000 7f744d800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f74480b8200 con 0x7f742c003a50 2024-04-20T11:37:59.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:59.256+0000 7f744d800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f744815ece0 con 0x7f74481618c0 2024-04-20T11:37:59.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:59.256+0000 7f7447400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f74480c8aa0 0x7f74480c8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:59.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:59.256+0000 7f7447400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f74480c8aa0 0x7f74480c8f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:55746/0 (socket says 172.21.15.152:55746) 2024-04-20T11:37:59.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:59.256+0000 7f7447400640 1 -- 172.21.15.152:0/1593957976 learned_addr learned my addr 172.21.15.152:0/1593957976 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:37:59.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:59.256+0000 7f7447400640 1 -- 172.21.15.152:0/1593957976 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74481618c0 msgr2=0x7f74480cbbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:59.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:59.256+0000 7f7447400640 1 --2- 172.21.15.152:0/1593957976 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74481618c0 0x7f74480cbbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:59.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:59.256+0000 7f7447400640 1 -- 172.21.15.152:0/1593957976 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f742c003a50 msgr2=0x7f74480c7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:59.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:59.256+0000 7f7447400640 1 --2- 172.21.15.152:0/1593957976 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f742c003a50 0x7f74480c7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:59.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:59.256+0000 7f7447400640 1 -- 172.21.15.152:0/1593957976 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7448153950 con 0x7f74480c8aa0 2024-04-20T11:37:59.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:59.257+0000 7f7447400640 1 --2- 172.21.15.152:0/1593957976 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f74480c8aa0 0x7f74480c8f00 secure :-1 s=READY pgs=2340 cs=0 l=1 rev1=1 crypto rx=0x7f74480d5920 tx=0x7f7440009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:59.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:59.257+0000 7f743ea00640 1 -- 172.21.15.152:0/1593957976 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7440019070 con 0x7f74480c8aa0 2024-04-20T11:37:59.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:59.257+0000 7f743ea00640 1 -- 172.21.15.152:0/1593957976 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7440007cf0 con 0x7f74480c8aa0 2024-04-20T11:37:59.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:59.257+0000 7f743ea00640 1 -- 172.21.15.152:0/1593957976 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7440014640 con 0x7f74480c8aa0 2024-04-20T11:37:59.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:59.257+0000 7f744d800640 1 -- 172.21.15.152:0/1593957976 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7448153c30 con 0x7f74480c8aa0 2024-04-20T11:37:59.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:59.257+0000 7f744d800640 1 -- 172.21.15.152:0/1593957976 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7448161d40 con 0x7f74480c8aa0 2024-04-20T11:37:59.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:59.258+0000 7f744d800640 1 -- 172.21.15.152:0/1593957976 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74480c8390 con 0x7f74480c8aa0 2024-04-20T11:37:59.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:59.260+0000 7f743ea00640 1 -- 172.21.15.152:0/1593957976 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 26) ==== 91907+0+0 (secure 0 0 0) 0x7f744001c070 con 0x7f74480c8aa0 2024-04-20T11:37:59.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:59.261+0000 7f743ea00640 1 --2- 172.21.15.152:0/1593957976 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f742406d2b0 0x7f742406f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:59.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:59.261+0000 7f743ea00640 1 -- 172.21.15.152:0/1593957976 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 5709+0+0 (secure 0 0 0) 0x7f7440089c80 con 0x7f74480c8aa0 2024-04-20T11:37:59.262 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:59.261+0000 7f7446a00640 1 --2- 172.21.15.152:0/1593957976 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f742406d2b0 0x7f742406f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:59.263 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:59.263+0000 7f7446a00640 1 --2- 172.21.15.152:0/1593957976 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f742406d2b0 0x7f742406f770 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f7438034000 tx=0x7f7438039040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:59.266 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:59.266+0000 7f743ea00640 1 -- 172.21.15.152:0/1593957976 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f7440059cb0 con 0x7f74480c8aa0 2024-04-20T11:37:59.509 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:37:59.508+0000 7f744d800640 1 -- 172.21.15.152:0/1593957976 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f7448068a60 con 0x7f74480c8aa0 2024-04-20T11:38:00.146 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.146+0000 7f743ea00640 1 -- 172.21.15.152:0/1593957976 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v116) ==== 136+0+0 (secure 0 0 0) 0x7f74400595b0 con 0x7f74480c8aa0 2024-04-20T11:38:00.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.149+0000 7f744d800640 1 -- 172.21.15.152:0/1593957976 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f742406d2b0 msgr2=0x7f742406f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:00.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.150+0000 7f744d800640 1 --2- 172.21.15.152:0/1593957976 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f742406d2b0 0x7f742406f770 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f7438034000 tx=0x7f7438039040 comp rx=0 tx=0).stop 2024-04-20T11:38:00.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.150+0000 7f744d800640 1 -- 172.21.15.152:0/1593957976 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f74480c8aa0 msgr2=0x7f74480c8f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:00.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.150+0000 7f744d800640 1 --2- 172.21.15.152:0/1593957976 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f74480c8aa0 0x7f74480c8f00 secure :-1 s=READY pgs=2340 cs=0 l=1 rev1=1 crypto rx=0x7f74480d5920 tx=0x7f7440009cd0 comp rx=0 tx=0).stop 2024-04-20T11:38:00.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.150+0000 7f744d800640 1 -- 172.21.15.152:0/1593957976 shutdown_connections 2024-04-20T11:38:00.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.150+0000 7f744d800640 1 --2- 172.21.15.152:0/1593957976 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f742406d2b0 0x7f742406f770 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:00.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.150+0000 7f744d800640 1 --2- 172.21.15.152:0/1593957976 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f74480c8aa0 0x7f74480c8f00 unknown :-1 s=CLOSED pgs=2340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:00.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.150+0000 7f744d800640 1 --2- 172.21.15.152:0/1593957976 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74481618c0 0x7f74480cbbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:00.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.150+0000 7f744d800640 1 --2- 172.21.15.152:0/1593957976 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f742c003a50 0x7f74480c7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:00.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.150+0000 7f744d800640 1 -- 172.21.15.152:0/1593957976 >> 172.21.15.152:0/1593957976 conn(0x7f74480676c0 msgr2=0x7f7448159110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:00.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.150+0000 7f744d800640 1 -- 172.21.15.152:0/1593957976 shutdown_connections 2024-04-20T11:38:00.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.150+0000 7f744d800640 1 -- 172.21.15.152:0/1593957976 wait complete. 2024-04-20T11:38:00.175 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-04-20T11:38:00.377 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.376+0000 7f4458a00640 1 Processor -- start 2024-04-20T11:38:00.377 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.376+0000 7f4458a00640 1 -- start start 2024-04-20T11:38:00.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.378+0000 7f4458a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f445406fa50 0x7f44540b3470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:00.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.378+0000 7f4458a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f445415fd60 0x7f44540b39b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:00.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.378+0000 7f4458a00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44540b4130 con 0x7f445415b2b0 2024-04-20T11:38:00.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.378+0000 7f4458a00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f44540b3ef0 con 0x7f445415fd60 2024-04-20T11:38:00.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.378+0000 7f4458a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f445415f8d0 con 0x7f445406fa50 2024-04-20T11:38:00.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.378+0000 7f4451e00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f445415b2b0 0x7f445406f340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:34718/0 (socket says 172.21.15.152:34718) 2024-04-20T11:38:00.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.378+0000 7f4451e00640 1 -- 172.21.15.152:0/3560385271 learned_addr learned my addr 172.21.15.152:0/3560385271 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:38:00.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.378+0000 7f4452800640 1 --2- 172.21.15.152:0/3560385271 >> v2:172.21.15.152:3300/0 conn(0x7f445415fd60 0x7f44540b39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:00.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.379+0000 7f4451400640 1 --2- 172.21.15.152:0/3560385271 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f445406fa50 0x7f44540b3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:00.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.379+0000 7f4452800640 1 -- 172.21.15.152:0/3560385271 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f445406fa50 msgr2=0x7f44540b3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:00.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.379+0000 7f4452800640 1 --2- 172.21.15.152:0/3560385271 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f445406fa50 0x7f44540b3470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:00.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.379+0000 7f4452800640 1 -- 172.21.15.152:0/3560385271 >> v1:172.21.15.152:6789/0 conn(0x7f445415b2b0 legacy=0x7f445406f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:00.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.379+0000 7f4452800640 1 -- 172.21.15.152:0/3560385271 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f445415f280 con 0x7f445415fd60 2024-04-20T11:38:00.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.379+0000 7f4450a00640 1 -- 172.21.15.152:0/3560385271 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2518354129 0 0) 0x7f44540b4130 con 0x7f445415b2b0 2024-04-20T11:38:00.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.380+0000 7f4452800640 1 --2- 172.21.15.152:0/3560385271 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f445415fd60 0x7f44540b39b0 secure :-1 s=READY pgs=2342 cs=0 l=1 rev1=1 crypto rx=0x7f444c0097a0 tx=0x7f444c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8ee527ea91e44b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:00.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.380+0000 7f4450a00640 1 -- 172.21.15.152:0/3560385271 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f444c035020 con 0x7f445415fd60 2024-04-20T11:38:00.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.380+0000 7f4450a00640 1 -- 172.21.15.152:0/3560385271 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f444c031830 con 0x7f445415fd60 2024-04-20T11:38:00.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.381+0000 7f4450a00640 1 -- 172.21.15.152:0/3560385271 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f444c031c00 con 0x7f445415fd60 2024-04-20T11:38:00.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.381+0000 7f4458a00640 1 -- 172.21.15.152:0/3560385271 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f445415fd60 msgr2=0x7f44540b39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:00.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.381+0000 7f4458a00640 1 --2- 172.21.15.152:0/3560385271 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f445415fd60 0x7f44540b39b0 secure :-1 s=READY pgs=2342 cs=0 l=1 rev1=1 crypto rx=0x7f444c0097a0 tx=0x7f444c02b8d0 comp rx=0 tx=0).stop 2024-04-20T11:38:00.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.381+0000 7f4458a00640 1 -- 172.21.15.152:0/3560385271 shutdown_connections 2024-04-20T11:38:00.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.381+0000 7f4458a00640 1 --2- 172.21.15.152:0/3560385271 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f445406fa50 0x7f44540b3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:00.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.381+0000 7f4458a00640 1 --2- 172.21.15.152:0/3560385271 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f445415fd60 0x7f44540b39b0 unknown :-1 s=CLOSED pgs=2342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:00.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.381+0000 7f4458a00640 1 -- 172.21.15.152:0/3560385271 >> 172.21.15.152:0/3560385271 conn(0x7f4454067cf0 msgr2=0x7f445406b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:00.383 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.381+0000 7f4458a00640 1 -- 172.21.15.152:0/3560385271 shutdown_connections 2024-04-20T11:38:00.383 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.381+0000 7f4458a00640 1 -- 172.21.15.152:0/3560385271 wait complete. 2024-04-20T11:38:00.383 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.383+0000 7f4458a00640 1 Processor -- start 2024-04-20T11:38:00.384 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.383+0000 7f4458a00640 1 -- start start 2024-04-20T11:38:00.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.384+0000 7f4458a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f445406fa50 0x7f44541f43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:00.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.384+0000 7f4458a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f445415b2b0 0x7f44541523e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:00.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.384+0000 7f4458a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f445415fd60 0x7f44540ae7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:00.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.384+0000 7f4458a00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f445415b810 con 0x7f445415fd60 2024-04-20T11:38:00.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.384+0000 7f4458a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f445415b690 con 0x7f445406fa50 2024-04-20T11:38:00.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.384+0000 7f4458a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f445415b990 con 0x7f445415b2b0 2024-04-20T11:38:00.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.384+0000 7f4452800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f445415fd60 0x7f44540ae7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:00.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.384+0000 7f4452800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f445415fd60 0x7f44540ae7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:55764/0 (socket says 172.21.15.152:55764) 2024-04-20T11:38:00.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.384+0000 7f4452800640 1 -- 172.21.15.152:0/3913957534 learned_addr learned my addr 172.21.15.152:0/3913957534 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:38:00.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.385+0000 7f4451e00640 1 --2- 172.21.15.152:0/3913957534 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f445406fa50 0x7f44541f43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:00.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.385+0000 7f4451400640 1 --2- 172.21.15.152:0/3913957534 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f445415b2b0 0x7f44541523e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:00.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.385+0000 7f4452800640 1 -- 172.21.15.152:0/3913957534 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f445415b2b0 msgr2=0x7f44541523e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:00.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.385+0000 7f4452800640 1 --2- 172.21.15.152:0/3913957534 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f445415b2b0 0x7f44541523e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:00.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.385+0000 7f4452800640 1 -- 172.21.15.152:0/3913957534 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f445406fa50 msgr2=0x7f44541f43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:00.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.385+0000 7f4452800640 1 --2- 172.21.15.152:0/3913957534 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f445406fa50 0x7f44541f43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:00.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.385+0000 7f4452800640 1 -- 172.21.15.152:0/3913957534 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44540af1b0 con 0x7f445415fd60 2024-04-20T11:38:00.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.385+0000 7f4451e00640 1 --2- 172.21.15.152:0/3913957534 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f445406fa50 0x7f44541f43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:38:00.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.385+0000 7f4452800640 1 --2- 172.21.15.152:0/3913957534 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f445415fd60 0x7f44540ae7a0 secure :-1 s=READY pgs=2343 cs=0 l=1 rev1=1 crypto rx=0x7f444c034040 tx=0x7f444c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:00.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.386+0000 7f444a000640 1 -- 172.21.15.152:0/3913957534 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f444c040070 con 0x7f445415fd60 2024-04-20T11:38:00.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.386+0000 7f444a000640 1 -- 172.21.15.152:0/3913957534 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f444c0398e0 con 0x7f445415fd60 2024-04-20T11:38:00.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.386+0000 7f444a000640 1 -- 172.21.15.152:0/3913957534 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f444c035040 con 0x7f445415fd60 2024-04-20T11:38:00.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.386+0000 7f4458a00640 1 -- 172.21.15.152:0/3913957534 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f445415bc00 con 0x7f445415fd60 2024-04-20T11:38:00.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.386+0000 7f4458a00640 1 -- 172.21.15.152:0/3913957534 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f445415c1c0 con 0x7f445415fd60 2024-04-20T11:38:00.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.387+0000 7f4458a00640 1 -- 172.21.15.152:0/3913957534 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44540d0f40 con 0x7f445415fd60 2024-04-20T11:38:00.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.388+0000 7f444a000640 1 -- 172.21.15.152:0/3913957534 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 26) ==== 91907+0+0 (secure 0 0 0) 0x7f444c04b060 con 0x7f445415fd60 2024-04-20T11:38:00.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.389+0000 7f444a000640 1 --2- 172.21.15.152:0/3913957534 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f443006d280 0x7f443006f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:00.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.389+0000 7f444a000640 1 -- 172.21.15.152:0/3913957534 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 5709+0+0 (secure 0 0 0) 0x7f444c0abb90 con 0x7f445415fd60 2024-04-20T11:38:00.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.390+0000 7f4451e00640 1 --2- 172.21.15.152:0/3913957534 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f443006d280 0x7f443006f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:00.391 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.391+0000 7f4451e00640 1 --2- 172.21.15.152:0/3913957534 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f443006d280 0x7f443006f740 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f44440012e0 tx=0x7f4444009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:00.394 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.394+0000 7f444a000640 1 -- 172.21.15.152:0/3913957534 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f44540d0f40 con 0x7f445415fd60 2024-04-20T11:38:00.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.620+0000 7f4458a00640 1 -- 172.21.15.152:0/3913957534 --> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f445406eef0 con 0x7f443006d280 2024-04-20T11:38:00.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.625+0000 7f444a000640 1 -- 172.21.15.152:0/3913957534 <== mgr.4109 v2:172.21.15.175:6836/4021250441 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f445406eef0 con 0x7f443006d280 2024-04-20T11:38:00.625 INFO:teuthology.orchestra.run.smithi152.stdout:[ 2024-04-20T11:38:00.625 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-04-20T11:38:00.625 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "cephfs" 2024-04-20T11:38:00.625 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-20T11:38:00.625 INFO:teuthology.orchestra.run.smithi152.stdout:] 2024-04-20T11:38:00.627 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.626+0000 7f4458a00640 1 -- 172.21.15.152:0/3913957534 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f443006d280 msgr2=0x7f443006f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:00.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.626+0000 7f4458a00640 1 --2- 172.21.15.152:0/3913957534 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f443006d280 0x7f443006f740 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f44440012e0 tx=0x7f4444009040 comp rx=0 tx=0).stop 2024-04-20T11:38:00.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.627+0000 7f4458a00640 1 -- 172.21.15.152:0/3913957534 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f445415fd60 msgr2=0x7f44540ae7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:00.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.627+0000 7f4458a00640 1 --2- 172.21.15.152:0/3913957534 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f445415fd60 0x7f44540ae7a0 secure :-1 s=READY pgs=2343 cs=0 l=1 rev1=1 crypto rx=0x7f444c034040 tx=0x7f444c0046c0 comp rx=0 tx=0).stop 2024-04-20T11:38:00.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.627+0000 7f4458a00640 1 -- 172.21.15.152:0/3913957534 shutdown_connections 2024-04-20T11:38:00.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.627+0000 7f4458a00640 1 --2- 172.21.15.152:0/3913957534 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f443006d280 0x7f443006f740 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:00.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.627+0000 7f4458a00640 1 --2- 172.21.15.152:0/3913957534 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f445415fd60 0x7f44540ae7a0 unknown :-1 s=CLOSED pgs=2343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:00.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.627+0000 7f4458a00640 1 --2- 172.21.15.152:0/3913957534 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f445415b2b0 0x7f44541523e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:00.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.627+0000 7f4458a00640 1 --2- 172.21.15.152:0/3913957534 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f445406fa50 0x7f44541f43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:00.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.627+0000 7f4458a00640 1 -- 172.21.15.152:0/3913957534 >> 172.21.15.152:0/3913957534 conn(0x7f4454067cf0 msgr2=0x7f44540ce230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:00.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.627+0000 7f4458a00640 1 -- 172.21.15.152:0/3913957534 shutdown_connections 2024-04-20T11:38:00.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.627+0000 7f4458a00640 1 -- 172.21.15.152:0/3913957534 wait complete. 2024-04-20T11:38:00.646 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-20T11:38:00.845 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.844+0000 7f6f37e00640 1 Processor -- start 2024-04-20T11:38:00.845 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.845+0000 7f6f37e00640 1 -- start start 2024-04-20T11:38:00.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.846+0000 7f6f37e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f6f3815eb50 0x7f6f380d0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:00.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.846+0000 7f6f37e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6f380d35a0 0x7f6f380cf470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:00.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.846+0000 7f6f37e00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f380d19c0 con 0x7f6f38163230 2024-04-20T11:38:00.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.846+0000 7f6f37e00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f6f38162750 con 0x7f6f3815eb50 2024-04-20T11:38:00.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.846+0000 7f6f37e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6f38162a30 con 0x7f6f380d35a0 2024-04-20T11:38:00.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.846+0000 7f6f36a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f6f3815eb50 0x7f6f380d0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:00.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.846+0000 7f6f36a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f6f3815eb50 0x7f6f380d0f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:55768/0 (socket says 172.21.15.152:55768) 2024-04-20T11:38:00.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.846+0000 7f6f36a00640 1 -- 172.21.15.152:0/1188876419 learned_addr learned my addr 172.21.15.152:0/1188876419 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:38:00.848 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.847+0000 7f6f37400640 1 --2- 172.21.15.152:0/1188876419 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6f380d35a0 0x7f6f380cf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:00.848 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.847+0000 7f6f36a00640 1 -- 172.21.15.152:0/1188876419 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6f380d35a0 msgr2=0x7f6f380cf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:00.848 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.847+0000 7f6f36a00640 1 --2- 172.21.15.152:0/1188876419 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6f380d35a0 0x7f6f380cf470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:00.848 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.847+0000 7f6f36a00640 1 -- 172.21.15.152:0/1188876419 >> v1:172.21.15.152:6789/0 conn(0x7f6f38163230 legacy=0x7f6f380d2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:00.848 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.847+0000 7f6f36a00640 1 -- 172.21.15.152:0/1188876419 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f38162d10 con 0x7f6f3815eb50 2024-04-20T11:38:00.848 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.847+0000 7f6f35600640 1 -- 172.21.15.152:0/1188876419 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 974079282 0 0) 0x7f6f380d19c0 con 0x7f6f38163230 2024-04-20T11:38:00.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.848+0000 7f6f36a00640 1 --2- 172.21.15.152:0/1188876419 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6f3815eb50 0x7f6f380d0f00 secure :-1 s=READY pgs=2345 cs=0 l=1 rev1=1 crypto rx=0x7f6f28009e50 tx=0x7f6f2802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1b68d25e9025487 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:00.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.848+0000 7f6f35600640 1 -- 172.21.15.152:0/1188876419 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f2802ca70 con 0x7f6f3815eb50 2024-04-20T11:38:00.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.848+0000 7f6f35600640 1 -- 172.21.15.152:0/1188876419 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f2802cc10 con 0x7f6f3815eb50 2024-04-20T11:38:00.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.848+0000 7f6f35600640 1 -- 172.21.15.152:0/1188876419 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f280343f0 con 0x7f6f3815eb50 2024-04-20T11:38:00.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.849+0000 7f6f37e00640 1 -- 172.21.15.152:0/1188876419 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6f3815eb50 msgr2=0x7f6f380d0f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:00.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.849+0000 7f6f37e00640 1 --2- 172.21.15.152:0/1188876419 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6f3815eb50 0x7f6f380d0f00 secure :-1 s=READY pgs=2345 cs=0 l=1 rev1=1 crypto rx=0x7f6f28009e50 tx=0x7f6f2802bed0 comp rx=0 tx=0).stop 2024-04-20T11:38:00.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.849+0000 7f6f37e00640 1 -- 172.21.15.152:0/1188876419 shutdown_connections 2024-04-20T11:38:00.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.849+0000 7f6f37e00640 1 --2- 172.21.15.152:0/1188876419 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6f380d35a0 0x7f6f380cf470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:00.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.849+0000 7f6f37e00640 1 --2- 172.21.15.152:0/1188876419 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6f3815eb50 0x7f6f380d0f00 unknown :-1 s=CLOSED pgs=2345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:00.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.849+0000 7f6f37e00640 1 -- 172.21.15.152:0/1188876419 >> 172.21.15.152:0/1188876419 conn(0x7f6f3808bfd0 msgr2=0x7f6f380d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:00.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.849+0000 7f6f37e00640 1 -- 172.21.15.152:0/1188876419 shutdown_connections 2024-04-20T11:38:00.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.849+0000 7f6f37e00640 1 -- 172.21.15.152:0/1188876419 wait complete. 2024-04-20T11:38:00.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.851+0000 7f6f37e00640 1 Processor -- start 2024-04-20T11:38:00.852 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.851+0000 7f6f37e00640 1 -- start start 2024-04-20T11:38:00.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.852+0000 7f6f37e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6f380d35a0 0x7f6f38159240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:00.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.852+0000 7f6f37e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6f3815eb50 0x7f6f38153a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:00.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.852+0000 7f6f37e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6f38163230 0x7f6f38153f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:00.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.852+0000 7f6f37e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f6f380d15c0 con 0x7f6f380d35a0 2024-04-20T11:38:00.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.852+0000 7f6f37e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6f380d1440 con 0x7f6f38163230 2024-04-20T11:38:00.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.852+0000 7f6f37e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6f380d1740 con 0x7f6f3815eb50 2024-04-20T11:38:00.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.852+0000 7f6f36a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6f380d35a0 0x7f6f38159240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:00.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.852+0000 7f6f36a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6f380d35a0 0x7f6f38159240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:55778/0 (socket says 172.21.15.152:55778) 2024-04-20T11:38:00.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.852+0000 7f6f36a00640 1 -- 172.21.15.152:0/2255067895 learned_addr learned my addr 172.21.15.152:0/2255067895 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:38:00.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.853+0000 7f6f36000640 1 --2- 172.21.15.152:0/2255067895 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6f3815eb50 0x7f6f38153a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:00.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.853+0000 7f6f36a00640 1 -- 172.21.15.152:0/2255067895 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6f3815eb50 msgr2=0x7f6f38153a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:00.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.853+0000 7f6f36a00640 1 --2- 172.21.15.152:0/2255067895 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6f3815eb50 0x7f6f38153a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:00.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.853+0000 7f6f36a00640 1 -- 172.21.15.152:0/2255067895 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6f38163230 msgr2=0x7f6f38153f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:00.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.853+0000 7f6f36a00640 1 --2- 172.21.15.152:0/2255067895 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6f38163230 0x7f6f38153f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:00.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.853+0000 7f6f36a00640 1 -- 172.21.15.152:0/2255067895 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f381f8930 con 0x7f6f380d35a0 2024-04-20T11:38:00.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.853+0000 7f6f36a00640 1 --2- 172.21.15.152:0/2255067895 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6f380d35a0 0x7f6f38159240 secure :-1 s=READY pgs=2346 cs=0 l=1 rev1=1 crypto rx=0x7f6f28036040 tx=0x7f6f28004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:00.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.853+0000 7f6f2ea00640 1 -- 172.21.15.152:0/2255067895 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f28043020 con 0x7f6f380d35a0 2024-04-20T11:38:00.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.854+0000 7f6f2ea00640 1 -- 172.21.15.152:0/2255067895 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f2803e450 con 0x7f6f380d35a0 2024-04-20T11:38:00.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.854+0000 7f6f2ea00640 1 -- 172.21.15.152:0/2255067895 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f28009510 con 0x7f6f380d35a0 2024-04-20T11:38:00.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.854+0000 7f6f37e00640 1 -- 172.21.15.152:0/2255067895 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f380b40f0 con 0x7f6f380d35a0 2024-04-20T11:38:00.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.854+0000 7f6f37e00640 1 -- 172.21.15.152:0/2255067895 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f380b46b0 con 0x7f6f380d35a0 2024-04-20T11:38:00.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.855+0000 7f6f37e00640 1 -- 172.21.15.152:0/2255067895 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f38075e60 con 0x7f6f380d35a0 2024-04-20T11:38:00.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.856+0000 7f6f2ea00640 1 -- 172.21.15.152:0/2255067895 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 26) ==== 91907+0+0 (secure 0 0 0) 0x7f6f28046020 con 0x7f6f380d35a0 2024-04-20T11:38:00.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.857+0000 7f6f2ea00640 1 --2- 172.21.15.152:0/2255067895 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f6f1406d280 0x7f6f1406f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:00.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.857+0000 7f6f2ea00640 1 -- 172.21.15.152:0/2255067895 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 5709+0+0 (secure 0 0 0) 0x7f6f280acb10 con 0x7f6f380d35a0 2024-04-20T11:38:00.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.858+0000 7f6f36000640 1 --2- 172.21.15.152:0/2255067895 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f6f1406d280 0x7f6f1406f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:00.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.859+0000 7f6f36000640 1 --2- 172.21.15.152:0/2255067895 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f6f1406d280 0x7f6f1406f740 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f6f20007fd0 tx=0x7f6f20009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:00.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:00.861+0000 7f6f2ea00640 1 -- 172.21.15.152:0/2255067895 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f6f2804d030 con 0x7f6f380d35a0 2024-04-20T11:38:01.062 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:01.061+0000 7f6f37e00640 1 -- 172.21.15.152:0/2255067895 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f6f3808c680 con 0x7f6f380d35a0 2024-04-20T11:38:01.063 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:01.062+0000 7f6f2ea00640 1 -- 172.21.15.152:0/2255067895 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v30)=0 v30) ==== 122+0+0 (secure 0 0 0) 0x7f6f2807cb40 con 0x7f6f380d35a0 2024-04-20T11:38:01.066 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:01.065+0000 7f6f37e00640 1 -- 172.21.15.152:0/2255067895 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f6f1406d280 msgr2=0x7f6f1406f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:01.066 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:01.065+0000 7f6f37e00640 1 --2- 172.21.15.152:0/2255067895 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f6f1406d280 0x7f6f1406f740 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f6f20007fd0 tx=0x7f6f20009040 comp rx=0 tx=0).stop 2024-04-20T11:38:01.066 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:01.065+0000 7f6f37e00640 1 -- 172.21.15.152:0/2255067895 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6f380d35a0 msgr2=0x7f6f38159240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:01.067 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:01.065+0000 7f6f37e00640 1 --2- 172.21.15.152:0/2255067895 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6f380d35a0 0x7f6f38159240 secure :-1 s=READY pgs=2346 cs=0 l=1 rev1=1 crypto rx=0x7f6f28036040 tx=0x7f6f28004a40 comp rx=0 tx=0).stop 2024-04-20T11:38:01.067 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:01.065+0000 7f6f37e00640 1 -- 172.21.15.152:0/2255067895 shutdown_connections 2024-04-20T11:38:01.067 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:01.065+0000 7f6f37e00640 1 --2- 172.21.15.152:0/2255067895 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f6f1406d280 0x7f6f1406f740 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:01.067 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:01.065+0000 7f6f37e00640 1 --2- 172.21.15.152:0/2255067895 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6f38163230 0x7f6f38153f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:01.067 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:01.065+0000 7f6f37e00640 1 --2- 172.21.15.152:0/2255067895 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6f3815eb50 0x7f6f38153a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:01.067 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:01.065+0000 7f6f37e00640 1 --2- 172.21.15.152:0/2255067895 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6f380d35a0 0x7f6f38159240 unknown :-1 s=CLOSED pgs=2346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:01.067 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:01.066+0000 7f6f37e00640 1 -- 172.21.15.152:0/2255067895 >> 172.21.15.152:0/2255067895 conn(0x7f6f3808bfd0 msgr2=0x7f6f3806caf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:01.067 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:01.066+0000 7f6f37e00640 1 -- 172.21.15.152:0/2255067895 shutdown_connections 2024-04-20T11:38:01.067 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:01.066+0000 7f6f37e00640 1 -- 172.21.15.152:0/2255067895 wait complete. 2024-04-20T11:38:01.087 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:38:01.087 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p ./volumes/subvol_grp73080/subvol90420) 2024-04-20T11:38:01.185 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:38:01.185 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2024-04-20T11:38:01.236 INFO:teuthology.orchestra.run.smithi152.stdout:cephfs_data 2024-04-20T11:38:01.236 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:38:01.237 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data ./volumes/subvol_grp73080/subvol90420) 2024-04-20T11:38:01.319 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-20T11:38:01.319 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/home/ubuntu/cephtest/mnt.0/./volumes/subvol_grp73080/subvol90420/.meta 2024-04-20T11:38:01.403 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol90420 subvol_grp73080 2024-04-20T11:38:01.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:01.660+0000 7fdfa2000640 1 Processor -- start 2024-04-20T11:38:01.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:01.661+0000 7fdfa2000640 1 -- start start 2024-04-20T11:38:01.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:01.662+0000 7fdfa2000640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fdf9c162ab0 0x7fdf9c06dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:01.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:01.662+0000 7fdfa2000640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf9c0ad0a0 con 0x7fdf9c06e1b0 2024-04-20T11:38:01.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:01.662+0000 7fdfa2000640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf9c06e6a0 con 0x7fdf9c0d0d00 2024-04-20T11:38:01.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:01.662+0000 7fdfa2000640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fdf9c06ec60 con 0x7fdf9c162ab0 2024-04-20T11:38:01.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:01.662+0000 7fdf9be00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fdf9c06e1b0 0x7fdf9c0d05f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:34742/0 (socket says 172.21.15.152:34742) 2024-04-20T11:38:01.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:01.663+0000 7fdf9be00640 1 -- 172.21.15.152:0/553083365 learned_addr learned my addr 172.21.15.152:0/553083365 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:38:01.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:01.663+0000 7fdfa0c00640 1 --2- 172.21.15.152:0/553083365 >> v2:172.21.15.175:3300/0 conn(0x7fdf9c162ab0 0x7fdf9c06dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:01.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:01.663+0000 7fdf9b400640 1 -- 172.21.15.152:0/553083365 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 620356993 0 0) 0x7fdf9c0ad0a0 con 0x7fdf9c06e1b0 2024-04-20T11:38:01.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:01.663+0000 7fdf9b400640 1 -- 172.21.15.152:0/553083365 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdf84003520 con 0x7fdf9c06e1b0 2024-04-20T11:38:01.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:01.663+0000 7fdf9b400640 1 -- 172.21.15.152:0/553083365 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 618987173 0 0) 0x7fdf84003520 con 0x7fdf9c06e1b0 2024-04-20T11:38:01.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:01.663+0000 7fdf9b400640 1 -- 172.21.15.152:0/553083365 >> v2:172.21.15.175:3300/0 conn(0x7fdf9c162ab0 msgr2=0x7fdf9c06dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:01.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:01.663+0000 7fdf9b400640 1 --2- 172.21.15.152:0/553083365 >> v2:172.21.15.175:3300/0 conn(0x7fdf9c162ab0 0x7fdf9c06dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:01.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:01.663+0000 7fdf9b400640 1 -- 172.21.15.152:0/553083365 >> v1:172.21.15.175:6789/0 conn(0x7fdf9c0d0d00 legacy=0x7fdf9c0b4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:01.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:01.663+0000 7fdf9b400640 1 -- 172.21.15.152:0/553083365 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf9c06e980 con 0x7fdf9c06e1b0 2024-04-20T11:38:01.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:01.663+0000 7fdf9b400640 1 -- 172.21.15.152:0/553083365 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7fdf88001670 con 0x7fdf9c06e1b0 2024-04-20T11:38:01.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:01.663+0000 7fdf9b400640 1 -- 172.21.15.152:0/553083365 >> v1:172.21.15.152:6789/0 conn(0x7fdf9c06e1b0 legacy=0x7fdf9c0d05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:01.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:01.664+0000 7fdf9b400640 1 --2- 172.21.15.152:0/553083365 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdf840039b0 0x7fdf84003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:01.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:01.664+0000 7fdf9b400640 1 --2- 172.21.15.152:0/553083365 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf84004800 0x7fdf84004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:01.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:01.664+0000 7fdf9b400640 1 --2- 172.21.15.152:0/553083365 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdf840051c0 0x7fdf8400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:01.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:01.664+0000 7fdf9b400640 1 -- 172.21.15.152:0/553083365 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fdf9c06ec60 con 0x7fdf840051c0 2024-04-20T11:38:01.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:01.664+0000 7fdf9b400640 1 -- 172.21.15.152:0/553083365 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdf8400c160 con 0x7fdf84004800 2024-04-20T11:38:01.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:01.664+0000 7fdf9b400640 1 -- 172.21.15.152:0/553083365 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fdf8400c360 con 0x7fdf840039b0 2024-04-20T11:38:01.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:01.664+0000 7fdfa1600640 1 --2- 172.21.15.152:0/553083365 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdf840051c0 0x7fdf8400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:01.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:01.664+0000 7fdfa0c00640 1 --2- 172.21.15.152:0/553083365 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdf840039b0 0x7fdf84003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:01.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:01.664+0000 7fdf9be00640 1 --2- 172.21.15.152:0/553083365 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf84004800 0x7fdf84004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:01.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:01.664+0000 7fdfa1600640 1 -- 172.21.15.152:0/553083365 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdf840039b0 msgr2=0x7fdf84003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:01.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:01.664+0000 7fdfa1600640 1 --2- 172.21.15.152:0/553083365 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdf840039b0 0x7fdf84003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:01.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:01.664+0000 7fdfa1600640 1 -- 172.21.15.152:0/553083365 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf84004800 msgr2=0x7fdf84004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:01.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:01.664+0000 7fdfa1600640 1 --2- 172.21.15.152:0/553083365 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf84004800 0x7fdf84004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:01.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:01.664+0000 7fdfa1600640 1 -- 172.21.15.152:0/553083365 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf84003520 con 0x7fdf840051c0 2024-04-20T11:38:01.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:01.665+0000 7fdfa1600640 1 -- 172.21.15.152:0/553083365 reap_dead start 2024-04-20T11:38:01.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:01.665+0000 7fdfa1600640 1 --2- 172.21.15.152:0/553083365 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdf840051c0 0x7fdf8400b9c0 secure :-1 s=READY pgs=2348 cs=0 l=1 rev1=1 crypto rx=0x7fdf94001e20 tx=0x7fdf84005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13c0f809f4b526ce server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:01.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:01.666+0000 7fdf9b400640 1 -- 172.21.15.152:0/553083365 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf94001a70 con 0x7fdf840051c0 2024-04-20T11:38:01.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:01.666+0000 7fdf9b400640 1 -- 172.21.15.152:0/553083365 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf94003b70 con 0x7fdf840051c0 2024-04-20T11:38:01.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:01.666+0000 7fdf9b400640 1 -- 172.21.15.152:0/553083365 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf940050d0 con 0x7fdf840051c0 2024-04-20T11:38:01.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:01.666+0000 7fdfa2000640 1 -- 172.21.15.152:0/553083365 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdf840051c0 msgr2=0x7fdf8400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:01.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:01.666+0000 7fdfa2000640 1 --2- 172.21.15.152:0/553083365 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdf840051c0 0x7fdf8400b9c0 secure :-1 s=READY pgs=2348 cs=0 l=1 rev1=1 crypto rx=0x7fdf94001e20 tx=0x7fdf84005830 comp rx=0 tx=0).stop 2024-04-20T11:38:01.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:01.667+0000 7fdfa2000640 1 -- 172.21.15.152:0/553083365 shutdown_connections 2024-04-20T11:38:01.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:01.667+0000 7fdfa2000640 1 --2- 172.21.15.152:0/553083365 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdf840051c0 0x7fdf8400b9c0 unknown :-1 s=CLOSED pgs=2348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:01.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:01.667+0000 7fdfa2000640 1 -- 172.21.15.152:0/553083365 >> 172.21.15.152:0/553083365 conn(0x7fdf9c0b21e0 msgr2=0x7fdf9c0b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:01.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:01.667+0000 7fdfa2000640 1 -- 172.21.15.152:0/553083365 shutdown_connections 2024-04-20T11:38:01.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:01.667+0000 7fdfa2000640 1 -- 172.21.15.152:0/553083365 wait complete. 2024-04-20T11:38:01.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:01.668+0000 7fdfa2000640 1 Processor -- start 2024-04-20T11:38:01.670 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:01.669+0000 7fdfa2000640 1 -- start start 2024-04-20T11:38:01.671 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:01.669+0000 7fdfa2000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdf840051c0 0x7fdf9c15ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:01.671 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:01.669+0000 7fdfa2000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf9c0cecd0 0x7fdf9c0b07d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:01.671 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:01.670+0000 7fdfa2000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdf9c0cf150 0x7fdf9c0b0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:01.671 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:01.670+0000 7fdfa2000640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fdf9c06f6e0 con 0x7fdf840051c0 2024-04-20T11:38:01.671 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:01.670+0000 7fdfa2000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdf9c162e90 con 0x7fdf9c0cecd0 2024-04-20T11:38:01.671 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:01.670+0000 7fdfa2000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fdf9c06f860 con 0x7fdf9c0cf150 2024-04-20T11:38:01.671 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:01.670+0000 7fdfa0c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdf840051c0 0x7fdf9c15ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:01.671 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:01.670+0000 7fdfa0c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdf840051c0 0x7fdf9c15ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:55796/0 (socket says 172.21.15.152:55796) 2024-04-20T11:38:01.671 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:01.670+0000 7fdfa0c00640 1 -- 172.21.15.152:0/1222426800 learned_addr learned my addr 172.21.15.152:0/1222426800 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:38:01.671 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:01.670+0000 7fdfa0c00640 1 -- 172.21.15.152:0/1222426800 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdf9c0cf150 msgr2=0x7fdf9c0b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:01.671 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:01.670+0000 7fdfa0c00640 1 --2- 172.21.15.152:0/1222426800 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdf9c0cf150 0x7fdf9c0b0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:01.672 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:01.670+0000 7fdfa0c00640 1 -- 172.21.15.152:0/1222426800 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf9c0cecd0 msgr2=0x7fdf9c0b07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:01.672 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:01.670+0000 7fdfa0c00640 1 --2- 172.21.15.152:0/1222426800 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf9c0cecd0 0x7fdf9c0b07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:01.672 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:01.670+0000 7fdfa0c00640 1 -- 172.21.15.152:0/1222426800 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf9c09aff0 con 0x7fdf840051c0 2024-04-20T11:38:01.672 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:01.671+0000 7fdfa0c00640 1 --2- 172.21.15.152:0/1222426800 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdf840051c0 0x7fdf9c15ae60 secure :-1 s=READY pgs=2349 cs=0 l=1 rev1=1 crypto rx=0x7fdf90010860 tx=0x7fdf90010d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:01.672 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:01.671+0000 7fdf98c00640 1 -- 172.21.15.152:0/1222426800 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf9000f040 con 0x7fdf840051c0 2024-04-20T11:38:01.672 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:01.671+0000 7fdf98c00640 1 -- 172.21.15.152:0/1222426800 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf9000eb60 con 0x7fdf840051c0 2024-04-20T11:38:01.672 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:01.671+0000 7fdf98c00640 1 -- 172.21.15.152:0/1222426800 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf9001c400 con 0x7fdf840051c0 2024-04-20T11:38:01.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:01.671+0000 7fdfa2000640 1 -- 172.21.15.152:0/1222426800 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf9c0cd4b0 con 0x7fdf840051c0 2024-04-20T11:38:01.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:01.671+0000 7fdfa2000640 1 -- 172.21.15.152:0/1222426800 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf9c0cda70 con 0x7fdf840051c0 2024-04-20T11:38:01.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:01.672+0000 7fdfa2000640 1 -- 172.21.15.152:0/1222426800 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf9c075e60 con 0x7fdf840051c0 2024-04-20T11:38:01.674 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:01.673+0000 7fdf98c00640 1 -- 172.21.15.152:0/1222426800 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 26) ==== 91907+0+0 (secure 0 0 0) 0x7fdf9001c5a0 con 0x7fdf840051c0 2024-04-20T11:38:01.675 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:01.674+0000 7fdf98c00640 1 --2- 172.21.15.152:0/1222426800 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fdf7c06d280 0x7fdf7c06f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:01.675 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:01.674+0000 7fdf98c00640 1 -- 172.21.15.152:0/1222426800 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 5709+0+0 (secure 0 0 0) 0x7fdf9008aa10 con 0x7fdf840051c0 2024-04-20T11:38:01.675 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:01.675+0000 7fdf9be00640 1 --2- 172.21.15.152:0/1222426800 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fdf7c06d280 0x7fdf7c06f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:01.676 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:01.676+0000 7fdf9be00640 1 --2- 172.21.15.152:0/1222426800 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fdf7c06d280 0x7fdf7c06f740 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fdf88001920 tx=0x7fdf88009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:01.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:01.678+0000 7fdf98c00640 1 -- 172.21.15.152:0/1222426800 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fdf9c09aff0 con 0x7fdf840051c0 2024-04-20T11:38:01.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:01.909+0000 7fdfa2000640 1 -- 172.21.15.152:0/1222426800 --> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol90420", "group_name": "subvol_grp73080", "target": ["mon-mgr", ""]}) -- 0x7fdf9c06cc40 con 0x7fdf7c06d280 2024-04-20T11:38:01.924 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:38:01.925+0000 7f8a5cc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:38:01.924 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:38:01.925+0000 7f8a5cc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:38:01.924 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:38:01.925+0000 7f8a5cc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:38:01.924 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:38:01.925+0000 7f8a5cc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:38:01.924 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:38:01.925+0000 7f8a5cc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:38:02.003 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.003+0000 7fdf98c00640 1 -- 172.21.15.152:0/1222426800 <== mgr.4109 v2:172.21.15.175:6836/4021250441 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fdf9c06cc40 con 0x7fdf7c06d280 2024-04-20T11:38:02.003 INFO:teuthology.orchestra.run.smithi152.stdout:/volumes/subvol_grp73080/subvol90420 2024-04-20T11:38:02.007 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.006+0000 7fdfa2000640 1 -- 172.21.15.152:0/1222426800 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fdf7c06d280 msgr2=0x7fdf7c06f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:02.007 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.006+0000 7fdfa2000640 1 --2- 172.21.15.152:0/1222426800 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fdf7c06d280 0x7fdf7c06f740 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fdf88001920 tx=0x7fdf88009040 comp rx=0 tx=0).stop 2024-04-20T11:38:02.007 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.006+0000 7fdfa2000640 1 -- 172.21.15.152:0/1222426800 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdf840051c0 msgr2=0x7fdf9c15ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:02.007 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.006+0000 7fdfa2000640 1 --2- 172.21.15.152:0/1222426800 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdf840051c0 0x7fdf9c15ae60 secure :-1 s=READY pgs=2349 cs=0 l=1 rev1=1 crypto rx=0x7fdf90010860 tx=0x7fdf90010d20 comp rx=0 tx=0).stop 2024-04-20T11:38:02.007 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.006+0000 7fdfa2000640 1 -- 172.21.15.152:0/1222426800 shutdown_connections 2024-04-20T11:38:02.007 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.006+0000 7fdfa2000640 1 --2- 172.21.15.152:0/1222426800 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fdf7c06d280 0x7fdf7c06f740 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:02.007 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.006+0000 7fdfa2000640 1 --2- 172.21.15.152:0/1222426800 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdf9c0cf150 0x7fdf9c0b0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:02.007 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.006+0000 7fdfa2000640 1 --2- 172.21.15.152:0/1222426800 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf9c0cecd0 0x7fdf9c0b07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:02.007 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.006+0000 7fdfa2000640 1 --2- 172.21.15.152:0/1222426800 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdf840051c0 0x7fdf9c15ae60 unknown :-1 s=CLOSED pgs=2349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:02.007 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.006+0000 7fdfa2000640 1 -- 172.21.15.152:0/1222426800 >> 172.21.15.152:0/1222426800 conn(0x7fdf9c0b21e0 msgr2=0x7fdf9c071c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:02.007 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.006+0000 7fdfa2000640 1 -- 172.21.15.152:0/1222426800 shutdown_connections 2024-04-20T11:38:02.007 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.006+0000 7fdfa2000640 1 -- 172.21.15.152:0/1222426800 wait complete. 2024-04-20T11:38:02.027 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol90420 subvol_grp73080 2024-04-20T11:38:02.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.235+0000 7fbec3000640 1 Processor -- start 2024-04-20T11:38:02.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.235+0000 7fbec3000640 1 -- start start 2024-04-20T11:38:02.237 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.236+0000 7fbec3000640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fbebc0ad760 0x7fbebc15ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:02.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.236+0000 7fbec3000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fbebc0d05f0 0x7fbebc0d0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:02.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.237+0000 7fbec3000640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbebc15d8d0 con 0x7fbebc1569d0 2024-04-20T11:38:02.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.237+0000 7fbec3000640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fbebc0d0f80 con 0x7fbebc0d05f0 2024-04-20T11:38:02.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.237+0000 7fbec3000640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fbebc15d310 con 0x7fbebc0ad760 2024-04-20T11:38:02.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.237+0000 7fbec2600640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fbebc0d05f0 0x7fbebc0d0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:02.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.237+0000 7fbec2600640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fbebc0d05f0 0x7fbebc0d0a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:55810/0 (socket says 172.21.15.152:55810) 2024-04-20T11:38:02.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.237+0000 7fbec2600640 1 -- 172.21.15.152:0/405020526 learned_addr learned my addr 172.21.15.152:0/405020526 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:38:02.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.237+0000 7fbec1200640 1 --2- 172.21.15.152:0/405020526 >> v2:172.21.15.175:3300/0 conn(0x7fbebc0ad760 0x7fbebc15ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:02.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.237+0000 7fbec2600640 1 -- 172.21.15.152:0/405020526 >> v2:172.21.15.175:3300/0 conn(0x7fbebc0ad760 msgr2=0x7fbebc15ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:02.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.237+0000 7fbec2600640 1 --2- 172.21.15.152:0/405020526 >> v2:172.21.15.175:3300/0 conn(0x7fbebc0ad760 0x7fbebc15ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:02.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.237+0000 7fbec2600640 1 -- 172.21.15.152:0/405020526 >> v1:172.21.15.175:6789/0 conn(0x7fbebc1569d0 legacy=0x7fbebc06e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:02.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.237+0000 7fbec2600640 1 -- 172.21.15.152:0/405020526 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbebc15d5f0 con 0x7fbebc0d05f0 2024-04-20T11:38:02.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.239+0000 7fbec2600640 1 --2- 172.21.15.152:0/405020526 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbebc0d05f0 0x7fbebc0d0a40 secure :-1 s=READY pgs=2351 cs=0 l=1 rev1=1 crypto rx=0x7fbebc0b2e40 tx=0x7fbeb402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c017384de889446d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:02.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.239+0000 7fbebbe00640 1 -- 172.21.15.152:0/405020526 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeb402ca80 con 0x7fbebc0d05f0 2024-04-20T11:38:02.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.239+0000 7fbebbe00640 1 -- 172.21.15.152:0/405020526 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbeb402cc20 con 0x7fbebc0d05f0 2024-04-20T11:38:02.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.239+0000 7fbebbe00640 1 -- 172.21.15.152:0/405020526 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeb40343f0 con 0x7fbebc0d05f0 2024-04-20T11:38:02.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.240+0000 7fbec3000640 1 -- 172.21.15.152:0/405020526 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbebc0d05f0 msgr2=0x7fbebc0d0a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:02.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.240+0000 7fbec3000640 1 --2- 172.21.15.152:0/405020526 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbebc0d05f0 0x7fbebc0d0a40 secure :-1 s=READY pgs=2351 cs=0 l=1 rev1=1 crypto rx=0x7fbebc0b2e40 tx=0x7fbeb402bee0 comp rx=0 tx=0).stop 2024-04-20T11:38:02.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.240+0000 7fbec3000640 1 -- 172.21.15.152:0/405020526 shutdown_connections 2024-04-20T11:38:02.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.240+0000 7fbec3000640 1 --2- 172.21.15.152:0/405020526 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbebc0d05f0 0x7fbebc0d0a40 unknown :-1 s=CLOSED pgs=2351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:02.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.240+0000 7fbec3000640 1 --2- 172.21.15.152:0/405020526 >> v2:172.21.15.175:3300/0 conn(0x7fbebc0ad760 0x7fbebc15ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:02.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.240+0000 7fbec3000640 1 -- 172.21.15.152:0/405020526 >> 172.21.15.152:0/405020526 conn(0x7fbebc0b21e0 msgr2=0x7fbebc0b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:02.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.240+0000 7fbec3000640 1 -- 172.21.15.152:0/405020526 shutdown_connections 2024-04-20T11:38:02.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.240+0000 7fbec3000640 1 -- 172.21.15.152:0/405020526 wait complete. 2024-04-20T11:38:02.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.241+0000 7fbec3000640 1 Processor -- start 2024-04-20T11:38:02.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.242+0000 7fbec3000640 1 -- start start 2024-04-20T11:38:02.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.242+0000 7fbec3000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbebc0ad760 0x7fbebc156f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:02.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.243+0000 7fbec3000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbebc0d05f0 0x7fbebc0b3120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:02.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.243+0000 7fbec3000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbebc1569d0 0x7fbebc0b4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:02.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.243+0000 7fbec3000640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fbebc156db0 con 0x7fbebc1569d0 2024-04-20T11:38:02.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.243+0000 7fbec3000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbebc06dff0 con 0x7fbebc0d05f0 2024-04-20T11:38:02.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.243+0000 7fbec3000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fbebc15cf50 con 0x7fbebc0ad760 2024-04-20T11:38:02.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.243+0000 7fbec2600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbebc1569d0 0x7fbebc0b4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:02.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.243+0000 7fbec2600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbebc1569d0 0x7fbebc0b4a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:55820/0 (socket says 172.21.15.152:55820) 2024-04-20T11:38:02.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.243+0000 7fbec2600640 1 -- 172.21.15.152:0/1143795293 learned_addr learned my addr 172.21.15.152:0/1143795293 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:38:02.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.243+0000 7fbec2600640 1 -- 172.21.15.152:0/1143795293 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbebc0ad760 msgr2=0x7fbebc156f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:02.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.243+0000 7fbec2600640 1 --2- 172.21.15.152:0/1143795293 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbebc0ad760 0x7fbebc156f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:02.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.243+0000 7fbec2600640 1 -- 172.21.15.152:0/1143795293 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbebc0d05f0 msgr2=0x7fbebc0b3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:02.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.243+0000 7fbec1200640 1 --2- 172.21.15.152:0/1143795293 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbebc0d05f0 0x7fbebc0b3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:02.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.243+0000 7fbec2600640 1 --2- 172.21.15.152:0/1143795293 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbebc0d05f0 0x7fbebc0b3120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:02.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.243+0000 7fbec2600640 1 -- 172.21.15.152:0/1143795293 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbebc0b3a80 con 0x7fbebc1569d0 2024-04-20T11:38:02.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.244+0000 7fbec2600640 1 --2- 172.21.15.152:0/1143795293 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbebc1569d0 0x7fbebc0b4a60 secure :-1 s=READY pgs=2352 cs=0 l=1 rev1=1 crypto rx=0x7fbeb4036040 tx=0x7fbeb4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:02.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.244+0000 7fbeb9600640 1 -- 172.21.15.152:0/1143795293 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeb4043020 con 0x7fbebc1569d0 2024-04-20T11:38:02.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.244+0000 7fbeb9600640 1 -- 172.21.15.152:0/1143795293 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbeb403e450 con 0x7fbebc1569d0 2024-04-20T11:38:02.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.244+0000 7fbeb9600640 1 -- 172.21.15.152:0/1143795293 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeb4009510 con 0x7fbebc1569d0 2024-04-20T11:38:02.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.244+0000 7fbec3000640 1 -- 172.21.15.152:0/1143795293 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbebc157470 con 0x7fbebc1569d0 2024-04-20T11:38:02.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.244+0000 7fbec3000640 1 -- 172.21.15.152:0/1143795293 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbebc0b3fd0 con 0x7fbebc1569d0 2024-04-20T11:38:02.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.245+0000 7fbec3000640 1 -- 172.21.15.152:0/1143795293 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbebc070550 con 0x7fbebc1569d0 2024-04-20T11:38:02.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.246+0000 7fbeb9600640 1 -- 172.21.15.152:0/1143795293 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 26) ==== 91907+0+0 (secure 0 0 0) 0x7fbeb4046020 con 0x7fbebc1569d0 2024-04-20T11:38:02.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.248+0000 7fbeb9600640 1 --2- 172.21.15.152:0/1143795293 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fbe9c06d280 0x7fbe9c06f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:02.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.248+0000 7fbeb9600640 1 -- 172.21.15.152:0/1143795293 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 5709+0+0 (secure 0 0 0) 0x7fbeb40acb60 con 0x7fbebc1569d0 2024-04-20T11:38:02.249 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.249+0000 7fbec1c00640 1 --2- 172.21.15.152:0/1143795293 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fbe9c06d280 0x7fbe9c06f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:02.250 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.250+0000 7fbec1c00640 1 --2- 172.21.15.152:0/1143795293 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fbe9c06d280 0x7fbe9c06f740 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fbeb0009650 tx=0x7fbeb0006ba0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:02.253 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.252+0000 7fbeb9600640 1 -- 172.21.15.152:0/1143795293 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fbebc070550 con 0x7fbebc1569d0 2024-04-20T11:38:02.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.482+0000 7fbec3000640 1 -- 172.21.15.152:0/1143795293 --> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol90420", "group_name": "subvol_grp73080", "target": ["mon-mgr", ""]}) -- 0x7fbebc075e60 con 0x7fbe9c06d280 2024-04-20T11:38:02.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.493+0000 7fbeb9600640 1 -- 172.21.15.152:0/1143795293 <== mgr.4109 v2:172.21.15.175:6836/4021250441 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fbebc075e60 con 0x7fbe9c06d280 2024-04-20T11:38:02.494 INFO:teuthology.orchestra.run.smithi152.stdout:/volumes/subvol_grp73080/subvol90420 2024-04-20T11:38:02.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.496+0000 7fbec3000640 1 -- 172.21.15.152:0/1143795293 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fbe9c06d280 msgr2=0x7fbe9c06f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:02.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.496+0000 7fbec3000640 1 --2- 172.21.15.152:0/1143795293 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fbe9c06d280 0x7fbe9c06f740 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fbeb0009650 tx=0x7fbeb0006ba0 comp rx=0 tx=0).stop 2024-04-20T11:38:02.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.496+0000 7fbec3000640 1 -- 172.21.15.152:0/1143795293 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbebc1569d0 msgr2=0x7fbebc0b4a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:02.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.496+0000 7fbec3000640 1 --2- 172.21.15.152:0/1143795293 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbebc1569d0 0x7fbebc0b4a60 secure :-1 s=READY pgs=2352 cs=0 l=1 rev1=1 crypto rx=0x7fbeb4036040 tx=0x7fbeb4004a40 comp rx=0 tx=0).stop 2024-04-20T11:38:02.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.496+0000 7fbec3000640 1 -- 172.21.15.152:0/1143795293 shutdown_connections 2024-04-20T11:38:02.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.496+0000 7fbec3000640 1 --2- 172.21.15.152:0/1143795293 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fbe9c06d280 0x7fbe9c06f740 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:02.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.496+0000 7fbec3000640 1 --2- 172.21.15.152:0/1143795293 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbebc1569d0 0x7fbebc0b4a60 unknown :-1 s=CLOSED pgs=2352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:02.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.496+0000 7fbec3000640 1 --2- 172.21.15.152:0/1143795293 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbebc0d05f0 0x7fbebc0b3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:02.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.496+0000 7fbec3000640 1 --2- 172.21.15.152:0/1143795293 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbebc0ad760 0x7fbebc156f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:02.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.496+0000 7fbec3000640 1 -- 172.21.15.152:0/1143795293 >> 172.21.15.152:0/1143795293 conn(0x7fbebc0b21e0 msgr2=0x7fbebc0afd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:02.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.497+0000 7fbec3000640 1 -- 172.21.15.152:0/1143795293 shutdown_connections 2024-04-20T11:38:02.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.497+0000 7fbec3000640 1 -- 172.21.15.152:0/1143795293 wait complete. 2024-04-20T11:38:02.516 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:38:02.516 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_legacy/5321fc0339b8bb33b91a55c4dd7eb271.meta) 2024-04-20T11:38:02.630 INFO:teuthology.orchestra.run.smithi152.stdout:[GLOBAL] 2024-04-20T11:38:02.630 INFO:teuthology.orchestra.run.smithi152.stdout:version = 1 2024-04-20T11:38:02.630 INFO:teuthology.orchestra.run.smithi152.stdout:type = subvolume 2024-04-20T11:38:02.630 INFO:teuthology.orchestra.run.smithi152.stdout:path = /volumes/subvol_grp73080/subvol90420 2024-04-20T11:38:02.630 INFO:teuthology.orchestra.run.smithi152.stdout:state = complete 2024-04-20T11:38:02.630 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:38:02.636 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol90420 subvol_grp73080 2024-04-20T11:38:02.844 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.843+0000 7f47f7200640 1 Processor -- start 2024-04-20T11:38:02.844 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.844+0000 7f47f7200640 1 -- start start 2024-04-20T11:38:02.846 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.845+0000 7f47f7200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f47f80d0fa0 0x7f47f80d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:02.846 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.845+0000 7f47f7200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f47f80cff90 0x7f47f80d0420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:02.846 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.845+0000 7f47f7200640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47f815ec50 con 0x7f47f815e870 2024-04-20T11:38:02.846 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.845+0000 7f47f7200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f47f80d1960 con 0x7f47f80cff90 2024-04-20T11:38:02.846 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.845+0000 7f47f7200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f47f80d2ee0 con 0x7f47f80d0fa0 2024-04-20T11:38:02.846 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.845+0000 7f47f6800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f47f80cff90 0x7f47f80d0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:02.846 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.845+0000 7f47f6800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f47f80cff90 0x7f47f80d0420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:55832/0 (socket says 172.21.15.152:55832) 2024-04-20T11:38:02.846 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.845+0000 7f47f6800640 1 -- 172.21.15.152:0/2748109403 learned_addr learned my addr 172.21.15.152:0/2748109403 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:38:02.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.845+0000 7f47f5e00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f47f815e870 0x7f47f80d3370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:34752/0 (socket says 172.21.15.152:34752) 2024-04-20T11:38:02.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.846+0000 7f47f5400640 1 --2- 172.21.15.152:0/2748109403 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f47f80d0fa0 0x7f47f80d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:02.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.846+0000 7f47f6800640 1 -- 172.21.15.152:0/2748109403 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f47f80d0fa0 msgr2=0x7f47f80d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:02.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.846+0000 7f47f6800640 1 --2- 172.21.15.152:0/2748109403 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f47f80d0fa0 0x7f47f80d1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:02.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.846+0000 7f47f6800640 1 -- 172.21.15.152:0/2748109403 >> v1:172.21.15.152:6789/0 conn(0x7f47f815e870 legacy=0x7f47f80d3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:02.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.846+0000 7f47f6800640 1 -- 172.21.15.152:0/2748109403 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47f80d22d0 con 0x7f47f80cff90 2024-04-20T11:38:02.848 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.846+0000 7f47f4a00640 1 -- 172.21.15.152:0/2748109403 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1833800066 0 0) 0x7f47f815ec50 con 0x7f47f815e870 2024-04-20T11:38:02.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.847+0000 7f47f6800640 1 --2- 172.21.15.152:0/2748109403 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f47f80cff90 0x7f47f80d0420 secure :-1 s=READY pgs=2354 cs=0 l=1 rev1=1 crypto rx=0x7f47f0009870 tx=0x7f47f002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=317d6f9efe615706 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:02.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.847+0000 7f47f4a00640 1 -- 172.21.15.152:0/2748109403 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f0035020 con 0x7f47f80cff90 2024-04-20T11:38:02.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.847+0000 7f47f4a00640 1 -- 172.21.15.152:0/2748109403 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47f0031640 con 0x7f47f80cff90 2024-04-20T11:38:02.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.848+0000 7f47f4a00640 1 -- 172.21.15.152:0/2748109403 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f002d040 con 0x7f47f80cff90 2024-04-20T11:38:02.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.848+0000 7f47f7200640 1 -- 172.21.15.152:0/2748109403 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f47f80cff90 msgr2=0x7f47f80d0420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:02.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.848+0000 7f47f7200640 1 --2- 172.21.15.152:0/2748109403 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f47f80cff90 0x7f47f80d0420 secure :-1 s=READY pgs=2354 cs=0 l=1 rev1=1 crypto rx=0x7f47f0009870 tx=0x7f47f002b8f0 comp rx=0 tx=0).stop 2024-04-20T11:38:02.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.848+0000 7f47f7200640 1 -- 172.21.15.152:0/2748109403 shutdown_connections 2024-04-20T11:38:02.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.848+0000 7f47f7200640 1 --2- 172.21.15.152:0/2748109403 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f47f80d0fa0 0x7f47f80d1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:02.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.848+0000 7f47f7200640 1 --2- 172.21.15.152:0/2748109403 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f47f80cff90 0x7f47f80d0420 unknown :-1 s=CLOSED pgs=2354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:02.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.848+0000 7f47f7200640 1 -- 172.21.15.152:0/2748109403 >> 172.21.15.152:0/2748109403 conn(0x7f47f808bfd0 msgr2=0x7f47f80d45d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:02.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.848+0000 7f47f7200640 1 -- 172.21.15.152:0/2748109403 shutdown_connections 2024-04-20T11:38:02.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.848+0000 7f47f7200640 1 -- 172.21.15.152:0/2748109403 wait complete. 2024-04-20T11:38:02.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.850+0000 7f47f7200640 1 Processor -- start 2024-04-20T11:38:02.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.850+0000 7f47f7200640 1 -- start start 2024-04-20T11:38:02.852 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.851+0000 7f47f7200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f47f80cff90 0x7f47f80b1140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:02.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.851+0000 7f47f7200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f47f80d0fa0 0x7f47f8152820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:02.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.851+0000 7f47f7200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f47f815e870 0x7f47f8161010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:02.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.851+0000 7f47f7200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f47f80d2b70 con 0x7f47f80d0fa0 2024-04-20T11:38:02.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.851+0000 7f47f7200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f47f80d2890 con 0x7f47f815e870 2024-04-20T11:38:02.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.851+0000 7f47f7200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f47f80d0960 con 0x7f47f80cff90 2024-04-20T11:38:02.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.851+0000 7f47f5400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f47f80d0fa0 0x7f47f8152820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:02.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.851+0000 7f47f5400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f47f80d0fa0 0x7f47f8152820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:55846/0 (socket says 172.21.15.152:55846) 2024-04-20T11:38:02.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.851+0000 7f47f5400640 1 -- 172.21.15.152:0/934059849 learned_addr learned my addr 172.21.15.152:0/934059849 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:38:02.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.852+0000 7f47f5e00640 1 --2- 172.21.15.152:0/934059849 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f47f80cff90 0x7f47f80b1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:02.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.852+0000 7f47f5400640 1 -- 172.21.15.152:0/934059849 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f47f80cff90 msgr2=0x7f47f80b1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:02.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.852+0000 7f47f5400640 1 --2- 172.21.15.152:0/934059849 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f47f80cff90 0x7f47f80b1140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:02.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.852+0000 7f47f5400640 1 -- 172.21.15.152:0/934059849 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f47f815e870 msgr2=0x7f47f8161010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:02.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.852+0000 7f47f5400640 1 --2- 172.21.15.152:0/934059849 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f47f815e870 0x7f47f8161010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:02.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.852+0000 7f47f5400640 1 -- 172.21.15.152:0/934059849 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47f80ccdf0 con 0x7f47f80d0fa0 2024-04-20T11:38:02.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.852+0000 7f47f5400640 1 --2- 172.21.15.152:0/934059849 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f47f80d0fa0 0x7f47f8152820 secure :-1 s=READY pgs=2355 cs=0 l=1 rev1=1 crypto rx=0x7f47e400d7d0 tx=0x7f47e400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:02.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.852+0000 7f47ee000640 1 -- 172.21.15.152:0/934059849 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e4018020 con 0x7f47f80d0fa0 2024-04-20T11:38:02.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.853+0000 7f47ee000640 1 -- 172.21.15.152:0/934059849 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47e40096c0 con 0x7f47f80d0fa0 2024-04-20T11:38:02.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.853+0000 7f47ee000640 1 -- 172.21.15.152:0/934059849 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e4009ab0 con 0x7f47f80d0fa0 2024-04-20T11:38:02.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.853+0000 7f47f7200640 1 -- 172.21.15.152:0/934059849 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47f80cd0d0 con 0x7f47f80d0fa0 2024-04-20T11:38:02.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.853+0000 7f47f7200640 1 -- 172.21.15.152:0/934059849 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47f80cd690 con 0x7f47f80d0fa0 2024-04-20T11:38:02.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.854+0000 7f47f7200640 1 -- 172.21.15.152:0/934059849 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47f8075e60 con 0x7f47f80d0fa0 2024-04-20T11:38:02.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.855+0000 7f47ee000640 1 -- 172.21.15.152:0/934059849 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 26) ==== 91907+0+0 (secure 0 0 0) 0x7f47e401b020 con 0x7f47f80d0fa0 2024-04-20T11:38:02.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.856+0000 7f47ee000640 1 --2- 172.21.15.152:0/934059849 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f47d806d280 0x7f47d806f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:02.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.856+0000 7f47ee000640 1 -- 172.21.15.152:0/934059849 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 5709+0+0 (secure 0 0 0) 0x7f47e408a930 con 0x7f47f80d0fa0 2024-04-20T11:38:02.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.857+0000 7f47f5e00640 1 --2- 172.21.15.152:0/934059849 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f47d806d280 0x7f47d806f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:02.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.858+0000 7f47f5e00640 1 --2- 172.21.15.152:0/934059849 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f47d806d280 0x7f47d806f740 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f47e80012e0 tx=0x7f47e8005eb0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:02.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:02.860+0000 7f47ee000640 1 -- 172.21.15.152:0/934059849 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f47e4022020 con 0x7f47f80d0fa0 2024-04-20T11:38:03.090 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:03.089+0000 7f47f7200640 1 -- 172.21.15.152:0/934059849 --> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol90420", "group_name": "subvol_grp73080", "target": ["mon-mgr", ""]}) -- 0x7f47f80ccba0 con 0x7f47d806d280 2024-04-20T11:38:03.125 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:03.124+0000 7f47ee000640 1 -- 172.21.15.152:0/934059849 <== mgr.4109 v2:172.21.15.175:6836/4021250441 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f47f80ccba0 con 0x7f47d806d280 2024-04-20T11:38:03.128 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:03.126+0000 7f47f7200640 1 -- 172.21.15.152:0/934059849 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f47d806d280 msgr2=0x7f47d806f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:03.128 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:03.127+0000 7f47f7200640 1 --2- 172.21.15.152:0/934059849 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f47d806d280 0x7f47d806f740 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f47e80012e0 tx=0x7f47e8005eb0 comp rx=0 tx=0).stop 2024-04-20T11:38:03.128 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:03.127+0000 7f47f7200640 1 -- 172.21.15.152:0/934059849 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f47f80d0fa0 msgr2=0x7f47f8152820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:03.128 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:03.127+0000 7f47f7200640 1 --2- 172.21.15.152:0/934059849 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f47f80d0fa0 0x7f47f8152820 secure :-1 s=READY pgs=2355 cs=0 l=1 rev1=1 crypto rx=0x7f47e400d7d0 tx=0x7f47e400dc90 comp rx=0 tx=0).stop 2024-04-20T11:38:03.128 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:03.127+0000 7f47f7200640 1 -- 172.21.15.152:0/934059849 shutdown_connections 2024-04-20T11:38:03.128 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:03.127+0000 7f47f7200640 1 --2- 172.21.15.152:0/934059849 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f47d806d280 0x7f47d806f740 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:03.128 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:03.127+0000 7f47f7200640 1 --2- 172.21.15.152:0/934059849 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f47f815e870 0x7f47f8161010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:03.128 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:03.127+0000 7f47f7200640 1 --2- 172.21.15.152:0/934059849 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f47f80d0fa0 0x7f47f8152820 unknown :-1 s=CLOSED pgs=2355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:03.128 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:03.127+0000 7f47f7200640 1 --2- 172.21.15.152:0/934059849 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f47f80cff90 0x7f47f80b1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:03.128 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:03.127+0000 7f47f7200640 1 -- 172.21.15.152:0/934059849 >> 172.21.15.152:0/934059849 conn(0x7f47f808bfd0 msgr2=0x7f47f8162ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:03.128 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:03.127+0000 7f47f7200640 1 -- 172.21.15.152:0/934059849 shutdown_connections 2024-04-20T11:38:03.128 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:03.127+0000 7f47f7200640 1 -- 172.21.15.152:0/934059849 wait complete. 2024-04-20T11:38:03.138 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:38:03.139+0000 7f8a5f400640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:38:03.138 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:38:03.139+0000 7f8a5f400640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:38:03.138 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:38:03.139+0000 7f8a5f400640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:38:03.138 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:38:03.139+0000 7f8a5f400640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:38:03.138 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:38:03.139+0000 7f8a5f400640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:38:03.148 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-04-20T11:38:03.162 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:38:03.163+0000 7f8a5fe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:38:03.162 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:38:03.163+0000 7f8a5fe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:38:03.162 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:38:03.163+0000 7f8a5fe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:38:03.162 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:38:03.163+0000 7f8a5fe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:38:03.162 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T11:38:03.163+0000 7f8a5fe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:38:03.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:03 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-20T11:38:03.250 INFO:teuthology.orchestra.run.smithi152.stdout:2 2024-04-20T11:38:03.250 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T11:38:03.259 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup rm cephfs subvol_grp73080 2024-04-20T11:38:03.446 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:03.445+0000 7f20af000640 1 Processor -- start 2024-04-20T11:38:03.446 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:03.445+0000 7f20af000640 1 -- start start 2024-04-20T11:38:03.447 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:03.447+0000 7f20af000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f20a8146000 0x7f20a80b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:03.447 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:03.447+0000 7f20af000640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f20a80cd8c0 0x7f20a80b2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:03.447 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:03.447+0000 7f20af000640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20a81463e0 con 0x7f20a80b2170 2024-04-20T11:38:03.447 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:03.447+0000 7f20af000640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f20a80cdca0 con 0x7f20a8146000 2024-04-20T11:38:03.448 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:03.447+0000 7f20af000640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f20a80b4750 con 0x7f20a80cd8c0 2024-04-20T11:38:03.448 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:03.447+0000 7f20a7e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f20a8146000 0x7f20a80b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:03.449 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:03.447+0000 7f20a7e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f20a8146000 0x7f20a80b4be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:55848/0 (socket says 172.21.15.152:55848) 2024-04-20T11:38:03.449 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:03.447+0000 7f20a7e00640 1 -- 172.21.15.152:0/521160432 learned_addr learned my addr 172.21.15.152:0/521160432 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:38:03.449 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:03.448+0000 7f20ad000640 1 --2- 172.21.15.152:0/521160432 >> v2:172.21.15.175:3300/0 conn(0x7f20a80cd8c0 0x7f20a80b2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:03.449 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:03.448+0000 7f20a7e00640 1 -- 172.21.15.152:0/521160432 >> v2:172.21.15.175:3300/0 conn(0x7f20a80cd8c0 msgr2=0x7f20a80b2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:03.449 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:03.448+0000 7f20a7e00640 1 --2- 172.21.15.152:0/521160432 >> v2:172.21.15.175:3300/0 conn(0x7f20a80cd8c0 0x7f20a80b2620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:03.449 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:03.448+0000 7f20a7e00640 1 -- 172.21.15.152:0/521160432 >> v1:172.21.15.175:6789/0 conn(0x7f20a80b2170 legacy=0x7f20a80cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:03.449 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:03.448+0000 7f20a7e00640 1 -- 172.21.15.152:0/521160432 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20a80b3e20 con 0x7f20a8146000 2024-04-20T11:38:03.450 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:03.449+0000 7f20a7e00640 1 --2- 172.21.15.152:0/521160432 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f20a8146000 0x7f20a80b4be0 secure :-1 s=READY pgs=2356 cs=0 l=1 rev1=1 crypto rx=0x7f20a81601b0 tx=0x7f209802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4af2103588822a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:03.450 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:03.449+0000 7f20a7400640 1 -- 172.21.15.152:0/521160432 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f209802e070 con 0x7f20a8146000 2024-04-20T11:38:03.450 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:03.449+0000 7f20a7400640 1 -- 172.21.15.152:0/521160432 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2098037070 con 0x7f20a8146000 2024-04-20T11:38:03.450 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:03.449+0000 7f20a7400640 1 -- 172.21.15.152:0/521160432 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2098031a00 con 0x7f20a8146000 2024-04-20T11:38:03.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:03.450+0000 7f20af000640 1 -- 172.21.15.152:0/521160432 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f20a8146000 msgr2=0x7f20a80b4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:03.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:03.450+0000 7f20af000640 1 --2- 172.21.15.152:0/521160432 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f20a8146000 0x7f20a80b4be0 secure :-1 s=READY pgs=2356 cs=0 l=1 rev1=1 crypto rx=0x7f20a81601b0 tx=0x7f209802b880 comp rx=0 tx=0).stop 2024-04-20T11:38:03.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:03.450+0000 7f20af000640 1 -- 172.21.15.152:0/521160432 shutdown_connections 2024-04-20T11:38:03.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:03.450+0000 7f20af000640 1 --2- 172.21.15.152:0/521160432 >> v2:172.21.15.175:3300/0 conn(0x7f20a80cd8c0 0x7f20a80b2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:03.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:03.450+0000 7f20af000640 1 --2- 172.21.15.152:0/521160432 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f20a8146000 0x7f20a80b4be0 unknown :-1 s=CLOSED pgs=2356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:03.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:03.450+0000 7f20af000640 1 -- 172.21.15.152:0/521160432 >> 172.21.15.152:0/521160432 conn(0x7f20a806b980 msgr2=0x7f20a806bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:03.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:03.450+0000 7f20af000640 1 -- 172.21.15.152:0/521160432 shutdown_connections 2024-04-20T11:38:03.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:03.450+0000 7f20af000640 1 -- 172.21.15.152:0/521160432 wait complete. 2024-04-20T11:38:03.452 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:03.452+0000 7f20af000640 1 Processor -- start 2024-04-20T11:38:03.452 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:03.452+0000 7f20af000640 1 -- start start 2024-04-20T11:38:03.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:03.453+0000 7f20af000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f20a80b2170 0x7f20a81ee360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:03.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:03.453+0000 7f20af000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f20a80cd8c0 0x7f20a81414d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:03.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:03.453+0000 7f20af000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20a8146000 0x7f20a815f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:03.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:03.453+0000 7f20af000640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f20a815ccf0 con 0x7f20a80cd8c0 2024-04-20T11:38:03.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:03.453+0000 7f20af000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f20a80b43e0 con 0x7f20a8146000 2024-04-20T11:38:03.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:03.453+0000 7f20af000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f20a815ce70 con 0x7f20a80b2170 2024-04-20T11:38:03.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:03.453+0000 7f209fe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f20a80cd8c0 0x7f20a81414d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:03.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:03.453+0000 7f209fe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f20a80cd8c0 0x7f20a81414d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:55864/0 (socket says 172.21.15.152:55864) 2024-04-20T11:38:03.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:03.453+0000 7f209fe00640 1 -- 172.21.15.152:0/2021504721 learned_addr learned my addr 172.21.15.152:0/2021504721 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:38:03.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:03.453+0000 7f209fe00640 1 -- 172.21.15.152:0/2021504721 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f20a80b2170 msgr2=0x7f20a81ee360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:03.456 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:03.453+0000 7f20a7e00640 1 --2- 172.21.15.152:0/2021504721 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f20a80b2170 0x7f20a81ee360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:03.456 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:03.454+0000 7f209fe00640 1 --2- 172.21.15.152:0/2021504721 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f20a80b2170 0x7f20a81ee360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:03.456 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:03.454+0000 7f20ad000640 1 --2- 172.21.15.152:0/2021504721 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20a8146000 0x7f20a815f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:03.456 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:03.454+0000 7f209fe00640 1 -- 172.21.15.152:0/2021504721 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20a8146000 msgr2=0x7f20a815f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:03.456 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:03.454+0000 7f209fe00640 1 --2- 172.21.15.152:0/2021504721 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20a8146000 0x7f20a815f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:03.456 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:03.454+0000 7f209fe00640 1 -- 172.21.15.152:0/2021504721 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20a81f8900 con 0x7f20a80cd8c0 2024-04-20T11:38:03.456 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:03.454+0000 7f20a7e00640 1 --2- 172.21.15.152:0/2021504721 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f20a80b2170 0x7f20a81ee360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:38:03.456 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:03.454+0000 7f209fe00640 1 --2- 172.21.15.152:0/2021504721 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f20a80cd8c0 0x7f20a81414d0 secure :-1 s=READY pgs=2357 cs=0 l=1 rev1=1 crypto rx=0x7f209000c430 tx=0x7f209000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:03.456 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:03.454+0000 7f20a4c00640 1 -- 172.21.15.152:0/2021504721 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2090011790 con 0x7f20a80cd8c0 2024-04-20T11:38:03.456 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:03.454+0000 7f20a4c00640 1 -- 172.21.15.152:0/2021504721 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2090017070 con 0x7f20a80cd8c0 2024-04-20T11:38:03.456 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:03.454+0000 7f20a4c00640 1 -- 172.21.15.152:0/2021504721 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2090011b70 con 0x7f20a80cd8c0 2024-04-20T11:38:03.456 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:03.454+0000 7f20af000640 1 -- 172.21.15.152:0/2021504721 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20a81f7b40 con 0x7f20a80cd8c0 2024-04-20T11:38:03.456 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:03.454+0000 7f20af000640 1 -- 172.21.15.152:0/2021504721 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20a81f8100 con 0x7f20a80cd8c0 2024-04-20T11:38:03.457 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:03.455+0000 7f20af000640 1 -- 172.21.15.152:0/2021504721 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20a80689f0 con 0x7f20a80cd8c0 2024-04-20T11:38:03.457 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:03.456+0000 7f20a4c00640 1 -- 172.21.15.152:0/2021504721 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 26) ==== 91907+0+0 (secure 0 0 0) 0x7f2090011d10 con 0x7f20a80cd8c0 2024-04-20T11:38:03.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:03.457+0000 7f20a4c00640 1 --2- 172.21.15.152:0/2021504721 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f208406d280 0x7f208406f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:03.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:03.457+0000 7f20a4c00640 1 -- 172.21.15.152:0/2021504721 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 5709+0+0 (secure 0 0 0) 0x7f209008a170 con 0x7f20a80cd8c0 2024-04-20T11:38:03.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:03.458+0000 7f20a7e00640 1 --2- 172.21.15.152:0/2021504721 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f208406d280 0x7f208406f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:03.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:03.459+0000 7f20a7e00640 1 --2- 172.21.15.152:0/2021504721 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f208406d280 0x7f208406f740 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f2098036000 tx=0x7f2098008030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:03.462 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:03.461+0000 7f20a4c00640 1 -- 172.21.15.152:0/2021504721 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f209005a1a0 con 0x7f20a80cd8c0 2024-04-20T11:38:03.691 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:03.690+0000 7f20af000640 1 -- 172.21.15.152:0/2021504721 --> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup rm", "vol_name": "cephfs", "group_name": "subvol_grp73080", "target": ["mon-mgr", ""]}) -- 0x7f20a81ee8a0 con 0x7f208406d280 2024-04-20T11:38:03.703 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:03.702+0000 7f20a4c00640 1 -- 172.21.15.152:0/2021504721 <== mgr.4109 v2:172.21.15.175:6836/4021250441 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f20a81ee8a0 con 0x7f208406d280 2024-04-20T11:38:03.705 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:03.704+0000 7f20af000640 1 -- 172.21.15.152:0/2021504721 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f208406d280 msgr2=0x7f208406f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:03.705 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:03.704+0000 7f20af000640 1 --2- 172.21.15.152:0/2021504721 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f208406d280 0x7f208406f740 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f2098036000 tx=0x7f2098008030 comp rx=0 tx=0).stop 2024-04-20T11:38:03.705 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:03.704+0000 7f20af000640 1 -- 172.21.15.152:0/2021504721 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f20a80cd8c0 msgr2=0x7f20a81414d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:03.706 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:03.704+0000 7f20af000640 1 --2- 172.21.15.152:0/2021504721 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f20a80cd8c0 0x7f20a81414d0 secure :-1 s=READY pgs=2357 cs=0 l=1 rev1=1 crypto rx=0x7f209000c430 tx=0x7f209000c8f0 comp rx=0 tx=0).stop 2024-04-20T11:38:03.706 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:03.705+0000 7f20af000640 1 -- 172.21.15.152:0/2021504721 shutdown_connections 2024-04-20T11:38:03.706 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:03.705+0000 7f20af000640 1 --2- 172.21.15.152:0/2021504721 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f208406d280 0x7f208406f740 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:03.706 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:03.705+0000 7f20af000640 1 --2- 172.21.15.152:0/2021504721 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20a8146000 0x7f20a815f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:03.706 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:03.705+0000 7f20af000640 1 --2- 172.21.15.152:0/2021504721 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f20a80cd8c0 0x7f20a81414d0 unknown :-1 s=CLOSED pgs=2357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:03.706 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:03.705+0000 7f20af000640 1 --2- 172.21.15.152:0/2021504721 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f20a80b2170 0x7f20a81ee360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:03.706 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:03.705+0000 7f20af000640 1 -- 172.21.15.152:0/2021504721 >> 172.21.15.152:0/2021504721 conn(0x7f20a806b980 msgr2=0x7f20a806bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:03.706 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:03.705+0000 7f20af000640 1 -- 172.21.15.152:0/2021504721 shutdown_connections 2024-04-20T11:38:03.706 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:03.705+0000 7f20af000640 1 -- 172.21.15.152:0/2021504721 wait complete. 2024-04-20T11:38:03.724 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-20T11:38:03.724 DEBUG:teuthology.orchestra.run.smithi152:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-20T11:38:03.825 DEBUG:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-20T11:38:03.936 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-20T11:38:03.936 INFO:tasks.cephfs.mount:Terminating background process 2024-04-20T11:38:03.937 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-20T11:38:03.937 DEBUG:teuthology.orchestra.run.smithi152:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:38:03.996 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:38:04.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.246+0000 7f18e7e00640 1 Processor -- start 2024-04-20T11:38:04.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.247+0000 7f18e7e00640 1 -- start start 2024-04-20T11:38:04.249 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.248+0000 7f18e7e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f18e81414e0 0x7f18e8166360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:04.249 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.248+0000 7f18e7e00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18e80cf0a0 con 0x7f18e813e530 2024-04-20T11:38:04.249 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.248+0000 7f18e7e00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18e80d00e0 con 0x7f18e806f510 2024-04-20T11:38:04.249 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.248+0000 7f18e7e00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f18e8141b00 con 0x7f18e81414e0 2024-04-20T11:38:04.249 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.248+0000 7f18e7400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f18e81414e0 0x7f18e8166360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:04.249 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.248+0000 7f18e6a00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f18e813e530 0x7f18e806ee00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:34760/0 (socket says 172.21.15.152:34760) 2024-04-20T11:38:04.249 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.248+0000 7f18e6a00640 1 -- 172.21.15.152:0/3111406210 learned_addr learned my addr 172.21.15.152:0/3111406210 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:38:04.250 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.249+0000 7f18e7400640 1 -- 172.21.15.152:0/3111406210 >> v1:172.21.15.175:6789/0 conn(0x7f18e806f510 legacy=0x7f18e8140dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:04.250 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.249+0000 7f18e7400640 1 -- 172.21.15.152:0/3111406210 >> v1:172.21.15.152:6789/0 conn(0x7f18e813e530 legacy=0x7f18e806ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:04.250 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.249+0000 7f18e7400640 1 -- 172.21.15.152:0/3111406210 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18e80cf320 con 0x7f18e81414e0 2024-04-20T11:38:04.250 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.250+0000 7f18e7400640 1 --2- 172.21.15.152:0/3111406210 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f18e81414e0 0x7f18e8166360 secure :-1 s=READY pgs=2359 cs=0 l=1 rev1=1 crypto rx=0x7f18e00097a0 tx=0x7f18e002dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=827deada2795c269 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:04.251 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.250+0000 7f18e5600640 1 -- 172.21.15.152:0/3111406210 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18e002f8a0 con 0x7f18e81414e0 2024-04-20T11:38:04.251 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.250+0000 7f18e5600640 1 -- 172.21.15.152:0/3111406210 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18e002fa40 con 0x7f18e81414e0 2024-04-20T11:38:04.251 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.250+0000 7f18e5600640 1 -- 172.21.15.152:0/3111406210 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18e002fe10 con 0x7f18e81414e0 2024-04-20T11:38:04.252 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.251+0000 7f18e7e00640 1 -- 172.21.15.152:0/3111406210 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f18e81414e0 msgr2=0x7f18e8166360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:04.252 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.251+0000 7f18e7e00640 1 --2- 172.21.15.152:0/3111406210 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f18e81414e0 0x7f18e8166360 secure :-1 s=READY pgs=2359 cs=0 l=1 rev1=1 crypto rx=0x7f18e00097a0 tx=0x7f18e002dae0 comp rx=0 tx=0).stop 2024-04-20T11:38:04.252 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.251+0000 7f18e7e00640 1 -- 172.21.15.152:0/3111406210 shutdown_connections 2024-04-20T11:38:04.252 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.251+0000 7f18e7e00640 1 --2- 172.21.15.152:0/3111406210 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f18e81414e0 0x7f18e8166360 unknown :-1 s=CLOSED pgs=2359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:04.252 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.251+0000 7f18e7e00640 1 -- 172.21.15.152:0/3111406210 >> 172.21.15.152:0/3111406210 conn(0x7f18e8067ae0 msgr2=0x7f18e806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:04.252 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.251+0000 7f18e7e00640 1 -- 172.21.15.152:0/3111406210 shutdown_connections 2024-04-20T11:38:04.252 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.251+0000 7f18e7e00640 1 -- 172.21.15.152:0/3111406210 wait complete. 2024-04-20T11:38:04.254 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.253+0000 7f18e7e00640 1 Processor -- start 2024-04-20T11:38:04.254 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.254+0000 7f18e7e00640 1 -- start start 2024-04-20T11:38:04.256 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.255+0000 7f18e7e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18e806f510 0x7f18e80b1910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:04.256 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.255+0000 7f18e7e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f18e813e530 0x7f18e8156290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:04.256 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.255+0000 7f18e7e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f18e81414e0 0x7f18e812fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:04.256 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.255+0000 7f18e7e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f18e8072ce0 con 0x7f18e813e530 2024-04-20T11:38:04.257 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.255+0000 7f18e7e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f18e80cfc50 con 0x7f18e806f510 2024-04-20T11:38:04.257 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.255+0000 7f18e7e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f18e80cf8e0 con 0x7f18e81414e0 2024-04-20T11:38:04.257 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.255+0000 7f18e6000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f18e813e530 0x7f18e8156290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:04.257 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.255+0000 7f18e6000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f18e813e530 0x7f18e8156290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:55882/0 (socket says 172.21.15.152:55882) 2024-04-20T11:38:04.257 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.255+0000 7f18e6000640 1 -- 172.21.15.152:0/1020094828 learned_addr learned my addr 172.21.15.152:0/1020094828 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:38:04.257 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.255+0000 7f18e6a00640 1 --2- 172.21.15.152:0/1020094828 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18e806f510 0x7f18e80b1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:04.257 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.256+0000 7f18e7400640 1 --2- 172.21.15.152:0/1020094828 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f18e81414e0 0x7f18e812fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:04.257 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.256+0000 7f18e6000640 1 -- 172.21.15.152:0/1020094828 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f18e81414e0 msgr2=0x7f18e812fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:04.257 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.256+0000 7f18e6000640 1 --2- 172.21.15.152:0/1020094828 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f18e81414e0 0x7f18e812fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:04.257 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.256+0000 7f18e6000640 1 -- 172.21.15.152:0/1020094828 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18e806f510 msgr2=0x7f18e80b1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:04.257 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.256+0000 7f18e6000640 1 --2- 172.21.15.152:0/1020094828 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18e806f510 0x7f18e80b1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:04.257 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.256+0000 7f18e6000640 1 -- 172.21.15.152:0/1020094828 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18e812da30 con 0x7f18e813e530 2024-04-20T11:38:04.257 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.256+0000 7f18e7400640 1 --2- 172.21.15.152:0/1020094828 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f18e81414e0 0x7f18e812fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:38:04.257 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.256+0000 7f18e6000640 1 --2- 172.21.15.152:0/1020094828 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f18e813e530 0x7f18e8156290 secure :-1 s=READY pgs=2360 cs=0 l=1 rev1=1 crypto rx=0x7f18d400b420 tx=0x7f18d400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:04.258 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.256+0000 7f18dea00640 1 -- 172.21.15.152:0/1020094828 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18d4011830 con 0x7f18e813e530 2024-04-20T11:38:04.258 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.256+0000 7f18dea00640 1 -- 172.21.15.152:0/1020094828 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18d4016070 con 0x7f18e813e530 2024-04-20T11:38:04.258 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.256+0000 7f18dea00640 1 -- 172.21.15.152:0/1020094828 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18d4011c10 con 0x7f18e813e530 2024-04-20T11:38:04.258 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.256+0000 7f18e7e00640 1 -- 172.21.15.152:0/1020094828 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18e80b1fb0 con 0x7f18e813e530 2024-04-20T11:38:04.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.257+0000 7f18e7e00640 1 -- 172.21.15.152:0/1020094828 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18e80b2570 con 0x7f18e813e530 2024-04-20T11:38:04.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.258+0000 7f18e7e00640 1 -- 172.21.15.152:0/1020094828 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18e8075e60 con 0x7f18e813e530 2024-04-20T11:38:04.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.259+0000 7f18dea00640 1 -- 172.21.15.152:0/1020094828 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 27) ==== 92060+0+0 (secure 0 0 0) 0x7f18d401b460 con 0x7f18e813e530 2024-04-20T11:38:04.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.260+0000 7f18dea00640 1 --2- 172.21.15.152:0/1020094828 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f18c806d430 0x7f18c806f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:04.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.260+0000 7f18dea00640 1 -- 172.21.15.152:0/1020094828 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 5709+0+0 (secure 0 0 0) 0x7f18d4089fc0 con 0x7f18e813e530 2024-04-20T11:38:04.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.261+0000 7f18e6a00640 1 --2- 172.21.15.152:0/1020094828 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f18c806d430 0x7f18c806f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:04.262 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.262+0000 7f18e6a00640 1 --2- 172.21.15.152:0/1020094828 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f18c806d430 0x7f18c806f8f0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f18d80012e0 tx=0x7f18d8009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:04.265 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.265+0000 7f18dea00640 1 -- 172.21.15.152:0/1020094828 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f18d4020020 con 0x7f18e813e530 2024-04-20T11:38:04.465 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.465+0000 7f18e7e00640 1 -- 172.21.15.152:0/1020094828 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f18e80b3490 con 0x7f18e813e530 2024-04-20T11:38:04.467 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.467+0000 7f18dea00640 1 -- 172.21.15.152:0/1020094828 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v167) ==== 74+0+16743 (secure 0 0 0) 0x7f18d4059e40 con 0x7f18e813e530 2024-04-20T11:38:04.468 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:38:04.468 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":167,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:37:49.091768+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:37:45.247938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"166","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2024-04-20T11:37:46.503171+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"167","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:38:04.471 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.470+0000 7f18e7e00640 1 -- 172.21.15.152:0/1020094828 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f18c806d430 msgr2=0x7f18c806f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:04.471 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.470+0000 7f18e7e00640 1 --2- 172.21.15.152:0/1020094828 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f18c806d430 0x7f18c806f8f0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f18d80012e0 tx=0x7f18d8009040 comp rx=0 tx=0).stop 2024-04-20T11:38:04.471 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.470+0000 7f18e7e00640 1 -- 172.21.15.152:0/1020094828 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f18e813e530 msgr2=0x7f18e8156290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:04.471 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.470+0000 7f18e7e00640 1 --2- 172.21.15.152:0/1020094828 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f18e813e530 0x7f18e8156290 secure :-1 s=READY pgs=2360 cs=0 l=1 rev1=1 crypto rx=0x7f18d400b420 tx=0x7f18d400b8e0 comp rx=0 tx=0).stop 2024-04-20T11:38:04.471 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.470+0000 7f18e7e00640 1 -- 172.21.15.152:0/1020094828 shutdown_connections 2024-04-20T11:38:04.471 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.470+0000 7f18e7e00640 1 --2- 172.21.15.152:0/1020094828 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f18c806d430 0x7f18c806f8f0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:04.471 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.470+0000 7f18e7e00640 1 --2- 172.21.15.152:0/1020094828 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f18e81414e0 0x7f18e812fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:04.471 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.470+0000 7f18e7e00640 1 --2- 172.21.15.152:0/1020094828 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f18e813e530 0x7f18e8156290 unknown :-1 s=CLOSED pgs=2360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:04.471 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.470+0000 7f18e7e00640 1 --2- 172.21.15.152:0/1020094828 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18e806f510 0x7f18e80b1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:04.471 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.470+0000 7f18e7e00640 1 -- 172.21.15.152:0/1020094828 >> 172.21.15.152:0/1020094828 conn(0x7f18e8067ae0 msgr2=0x7f18e8159570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:04.471 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.470+0000 7f18e7e00640 1 -- 172.21.15.152:0/1020094828 shutdown_connections 2024-04-20T11:38:04.471 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.470+0000 7f18e7e00640 1 -- 172.21.15.152:0/1020094828 wait complete. 2024-04-20T11:38:04.490 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T11:26:47.246571+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 26, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T11:37:45.247938+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '166', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 27, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T11:37:46.503171+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '167', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T11:38:04.490 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T11:38:04.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.685+0000 7fb9d3200640 1 Processor -- start 2024-04-20T11:38:04.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.685+0000 7fb9d3200640 1 -- start start 2024-04-20T11:38:04.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.686+0000 7fb9d3200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fb9cc15b510 0x7fb9cc15fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:04.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.687+0000 7fb9d3200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fb9cc0cfc40 0x7fb9cc0b3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:04.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.687+0000 7fb9d3200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9cc15f1c0 con 0x7fb9cc0b2170 2024-04-20T11:38:04.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.687+0000 7fb9d3200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fb9cc06f420 con 0x7fb9cc0cfc40 2024-04-20T11:38:04.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.687+0000 7fb9d3200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fb9cc0b4430 con 0x7fb9cc15b510 2024-04-20T11:38:04.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.687+0000 7fb9c3400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fb9cc0cfc40 0x7fb9cc0b3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:04.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.687+0000 7fb9c3400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fb9cc0cfc40 0x7fb9cc0b3ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:55884/0 (socket says 172.21.15.152:55884) 2024-04-20T11:38:04.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.687+0000 7fb9c3400640 1 -- 172.21.15.152:0/1900662553 learned_addr learned my addr 172.21.15.152:0/1900662553 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:38:04.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.687+0000 7fb9c3400640 1 -- 172.21.15.152:0/1900662553 >> v2:172.21.15.175:3300/0 conn(0x7fb9cc15b510 msgr2=0x7fb9cc15fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:04.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.687+0000 7fb9c3400640 1 --2- 172.21.15.152:0/1900662553 >> v2:172.21.15.175:3300/0 conn(0x7fb9cc15b510 0x7fb9cc15fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:04.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.687+0000 7fb9c3400640 1 -- 172.21.15.152:0/1900662553 >> v1:172.21.15.175:6789/0 conn(0x7fb9cc0b2170 legacy=0x7fb9cc0b4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:04.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.688+0000 7fb9c3400640 1 -- 172.21.15.152:0/1900662553 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9cc06f670 con 0x7fb9cc0cfc40 2024-04-20T11:38:04.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.689+0000 7fb9c3400640 1 --2- 172.21.15.152:0/1900662553 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb9cc0cfc40 0x7fb9cc0b3ef0 secure :-1 s=READY pgs=2361 cs=0 l=1 rev1=1 crypto rx=0x7fb9cc06c520 tx=0x7fb9b402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6d4339aafc2a8a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:04.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.689+0000 7fb9cb400640 1 -- 172.21.15.152:0/1900662553 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b402e070 con 0x7fb9cc0cfc40 2024-04-20T11:38:04.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.689+0000 7fb9cb400640 1 -- 172.21.15.152:0/1900662553 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9b4037070 con 0x7fb9cc0cfc40 2024-04-20T11:38:04.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.689+0000 7fb9cb400640 1 -- 172.21.15.152:0/1900662553 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b4031a00 con 0x7fb9cc0cfc40 2024-04-20T11:38:04.691 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.690+0000 7fb9d3200640 1 -- 172.21.15.152:0/1900662553 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb9cc0cfc40 msgr2=0x7fb9cc0b3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:04.691 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.690+0000 7fb9d3200640 1 --2- 172.21.15.152:0/1900662553 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb9cc0cfc40 0x7fb9cc0b3ef0 secure :-1 s=READY pgs=2361 cs=0 l=1 rev1=1 crypto rx=0x7fb9cc06c520 tx=0x7fb9b402b880 comp rx=0 tx=0).stop 2024-04-20T11:38:04.691 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.690+0000 7fb9d3200640 1 -- 172.21.15.152:0/1900662553 shutdown_connections 2024-04-20T11:38:04.691 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.690+0000 7fb9d3200640 1 --2- 172.21.15.152:0/1900662553 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb9cc0cfc40 0x7fb9cc0b3ef0 unknown :-1 s=CLOSED pgs=2361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:04.691 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.690+0000 7fb9d3200640 1 --2- 172.21.15.152:0/1900662553 >> v2:172.21.15.175:3300/0 conn(0x7fb9cc15b510 0x7fb9cc15fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:04.691 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.690+0000 7fb9d3200640 1 -- 172.21.15.152:0/1900662553 >> 172.21.15.152:0/1900662553 conn(0x7fb9cc06b980 msgr2=0x7fb9cc06bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:04.691 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.690+0000 7fb9d3200640 1 -- 172.21.15.152:0/1900662553 shutdown_connections 2024-04-20T11:38:04.691 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.690+0000 7fb9d3200640 1 -- 172.21.15.152:0/1900662553 wait complete. 2024-04-20T11:38:04.692 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.692+0000 7fb9d3200640 1 Processor -- start 2024-04-20T11:38:04.692 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.692+0000 7fb9d3200640 1 -- start start 2024-04-20T11:38:04.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.693+0000 7fb9d3200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb9cc0b2170 0x7fb9cc149030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:04.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.693+0000 7fb9d3200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9cc0cfc40 0x7fb9cc15d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:04.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.693+0000 7fb9d3200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb9cc15b510 0x7fb9cc157ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:04.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.693+0000 7fb9d3200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fb9cc15b8f0 con 0x7fb9cc0b2170 2024-04-20T11:38:04.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.693+0000 7fb9d3200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb9cc15f710 con 0x7fb9cc0cfc40 2024-04-20T11:38:04.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.693+0000 7fb9d3200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb9cc06f950 con 0x7fb9cc15b510 2024-04-20T11:38:04.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.693+0000 7fb9cbe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb9cc0b2170 0x7fb9cc149030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:04.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.693+0000 7fb9cbe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb9cc0b2170 0x7fb9cc149030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:55886/0 (socket says 172.21.15.152:55886) 2024-04-20T11:38:04.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.693+0000 7fb9cbe00640 1 -- 172.21.15.152:0/2502086687 learned_addr learned my addr 172.21.15.152:0/2502086687 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:38:04.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.694+0000 7fb9c3400640 1 --2- 172.21.15.152:0/2502086687 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9cc0cfc40 0x7fb9cc15d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:04.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.694+0000 7fb9cbe00640 1 -- 172.21.15.152:0/2502086687 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb9cc15b510 msgr2=0x7fb9cc157ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:04.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.694+0000 7fb9cbe00640 1 --2- 172.21.15.152:0/2502086687 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb9cc15b510 0x7fb9cc157ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:04.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.694+0000 7fb9cbe00640 1 -- 172.21.15.152:0/2502086687 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9cc0cfc40 msgr2=0x7fb9cc15d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:04.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.694+0000 7fb9cbe00640 1 --2- 172.21.15.152:0/2502086687 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9cc0cfc40 0x7fb9cc15d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:04.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.694+0000 7fb9cbe00640 1 -- 172.21.15.152:0/2502086687 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9cc15e360 con 0x7fb9cc0b2170 2024-04-20T11:38:04.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.694+0000 7fb9cbe00640 1 --2- 172.21.15.152:0/2502086687 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb9cc0b2170 0x7fb9cc149030 secure :-1 s=READY pgs=2362 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc00ba50 tx=0x7fb9bc00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:04.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.694+0000 7fb9c8c00640 1 -- 172.21.15.152:0/2502086687 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9bc019070 con 0x7fb9cc0b2170 2024-04-20T11:38:04.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.694+0000 7fb9c8c00640 1 -- 172.21.15.152:0/2502086687 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9bc00cc00 con 0x7fb9cc0b2170 2024-04-20T11:38:04.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.694+0000 7fb9c8c00640 1 -- 172.21.15.152:0/2502086687 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9bc014440 con 0x7fb9cc0b2170 2024-04-20T11:38:04.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.694+0000 7fb9d3200640 1 -- 172.21.15.152:0/2502086687 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9cc15e640 con 0x7fb9cc0b2170 2024-04-20T11:38:04.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.694+0000 7fb9d3200640 1 -- 172.21.15.152:0/2502086687 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9cc149570 con 0x7fb9cc0b2170 2024-04-20T11:38:04.697 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.696+0000 7fb9d3200640 1 -- 172.21.15.152:0/2502086687 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9cc0689f0 con 0x7fb9cc0b2170 2024-04-20T11:38:04.697 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.697+0000 7fb9c8c00640 1 -- 172.21.15.152:0/2502086687 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 27) ==== 92060+0+0 (secure 0 0 0) 0x7fb9bc01c070 con 0x7fb9cc0b2170 2024-04-20T11:38:04.698 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.697+0000 7fb9c8c00640 1 --2- 172.21.15.152:0/2502086687 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb9a806d430 0x7fb9a806f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:04.698 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.698+0000 7fb9c8c00640 1 -- 172.21.15.152:0/2502086687 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 5709+0+0 (secure 0 0 0) 0x7fb9bc08a490 con 0x7fb9cc0b2170 2024-04-20T11:38:04.698 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.698+0000 7fb9c3400640 1 --2- 172.21.15.152:0/2502086687 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb9a806d430 0x7fb9a806f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:04.699 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.699+0000 7fb9c3400640 1 --2- 172.21.15.152:0/2502086687 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb9a806d430 0x7fb9a806f8f0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fb9b4036000 tx=0x7fb9b403d040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:04.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.702+0000 7fb9c8c00640 1 -- 172.21.15.152:0/2502086687 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fb9bc05a120 con 0x7fb9cc0b2170 2024-04-20T11:38:04.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.902+0000 7fb9d3200640 1 -- 172.21.15.152:0/2502086687 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb9cc0cf9e0 con 0x7fb9cc0b2170 2024-04-20T11:38:04.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.903+0000 7fb9c8c00640 1 -- 172.21.15.152:0/2502086687 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v167) ==== 93+0+10 (secure 0 0 0) 0x7fb9bc059a20 con 0x7fb9cc0b2170 2024-04-20T11:38:04.904 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 1 2024-04-20T11:38:04.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.906+0000 7fb9d3200640 1 -- 172.21.15.152:0/2502086687 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb9a806d430 msgr2=0x7fb9a806f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:04.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.906+0000 7fb9d3200640 1 --2- 172.21.15.152:0/2502086687 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb9a806d430 0x7fb9a806f8f0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fb9b4036000 tx=0x7fb9b403d040 comp rx=0 tx=0).stop 2024-04-20T11:38:04.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.906+0000 7fb9d3200640 1 -- 172.21.15.152:0/2502086687 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb9cc0b2170 msgr2=0x7fb9cc149030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:04.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.906+0000 7fb9d3200640 1 --2- 172.21.15.152:0/2502086687 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb9cc0b2170 0x7fb9cc149030 secure :-1 s=READY pgs=2362 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc00ba50 tx=0x7fb9bc00bf10 comp rx=0 tx=0).stop 2024-04-20T11:38:04.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.906+0000 7fb9d3200640 1 -- 172.21.15.152:0/2502086687 shutdown_connections 2024-04-20T11:38:04.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.906+0000 7fb9d3200640 1 --2- 172.21.15.152:0/2502086687 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb9a806d430 0x7fb9a806f8f0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:04.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.906+0000 7fb9d3200640 1 --2- 172.21.15.152:0/2502086687 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb9cc15b510 0x7fb9cc157ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:04.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.906+0000 7fb9d3200640 1 --2- 172.21.15.152:0/2502086687 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9cc0cfc40 0x7fb9cc15d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:04.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.906+0000 7fb9d3200640 1 --2- 172.21.15.152:0/2502086687 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb9cc0b2170 0x7fb9cc149030 unknown :-1 s=CLOSED pgs=2362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:04.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.906+0000 7fb9d3200640 1 -- 172.21.15.152:0/2502086687 >> 172.21.15.152:0/2502086687 conn(0x7fb9cc06b980 msgr2=0x7fb9cc153410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:04.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.906+0000 7fb9d3200640 1 -- 172.21.15.152:0/2502086687 shutdown_connections 2024-04-20T11:38:04.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:04.906+0000 7fb9d3200640 1 -- 172.21.15.152:0/2502086687 wait complete. 2024-04-20T11:38:04.926 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T11:38:05.129 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.128+0000 7f64a1c00640 1 Processor -- start 2024-04-20T11:38:05.129 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.128+0000 7f64a1c00640 1 -- start start 2024-04-20T11:38:05.131 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.130+0000 7f64a1c00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f649c15b510 0x7f649c15fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:05.131 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.130+0000 7f64a1c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f649c0b2880 0x7f649c0b3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:05.131 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.130+0000 7f64a1c00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f649c15f1c0 con 0x7f649c0cfc40 2024-04-20T11:38:05.131 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.130+0000 7f64a1c00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f649c06f420 con 0x7f649c15b510 2024-04-20T11:38:05.131 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.130+0000 7f64a1c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f649c0b4430 con 0x7f649c0b2880 2024-04-20T11:38:05.131 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.131+0000 7f649a000640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f649c0cfc40 0x7f649c0b2170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:34766/0 (socket says 172.21.15.152:34766) 2024-04-20T11:38:05.131 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.131+0000 7f649a000640 1 -- 172.21.15.152:0/2371070821 learned_addr learned my addr 172.21.15.152:0/2371070821 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:38:05.133 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.131+0000 7f649aa00640 1 --2- 172.21.15.152:0/2371070821 >> v2:172.21.15.175:3300/0 conn(0x7f649c15b510 0x7f649c15fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:05.133 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.131+0000 7f649b400640 1 --2- 172.21.15.152:0/2371070821 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f649c0b2880 0x7f649c0b3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:05.133 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.131+0000 7f6499600640 1 -- 172.21.15.152:0/2371070821 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 367600686 0 0) 0x7f649c15f1c0 con 0x7f649c0cfc40 2024-04-20T11:38:05.133 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.131+0000 7f6499600640 1 -- 172.21.15.152:0/2371070821 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f647c005730 con 0x7f649c0cfc40 2024-04-20T11:38:05.133 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.131+0000 7f6499600640 1 -- 172.21.15.152:0/2371070821 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2864382986 0 0) 0x7f647c005730 con 0x7f649c0cfc40 2024-04-20T11:38:05.133 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.132+0000 7f6499600640 1 -- 172.21.15.152:0/2371070821 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f649c0b2880 msgr2=0x7f649c0b3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:05.133 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.132+0000 7f6499600640 1 --2- 172.21.15.152:0/2371070821 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f649c0b2880 0x7f649c0b3ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:05.134 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.132+0000 7f6499600640 1 -- 172.21.15.152:0/2371070821 >> v2:172.21.15.175:3300/0 conn(0x7f649c15b510 msgr2=0x7f649c15fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:05.134 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.132+0000 7f6499600640 1 --2- 172.21.15.152:0/2371070821 >> v2:172.21.15.175:3300/0 conn(0x7f649c15b510 0x7f649c15fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:05.134 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.132+0000 7f6499600640 1 -- 172.21.15.152:0/2371070821 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f649c06f670 con 0x7f649c0cfc40 2024-04-20T11:38:05.134 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.132+0000 7f6499600640 1 -- 172.21.15.152:0/2371070821 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f64840019f0 con 0x7f649c0cfc40 2024-04-20T11:38:05.134 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.132+0000 7f649b400640 1 --2- 172.21.15.152:0/2371070821 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f649c0b2880 0x7f649c0b3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:38:05.134 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.132+0000 7f6499600640 1 -- 172.21.15.152:0/2371070821 >> v1:172.21.15.152:6789/0 conn(0x7f649c0cfc40 legacy=0x7f649c0b2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:05.134 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.132+0000 7f6499600640 1 --2- 172.21.15.152:0/2371070821 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f649c0b2880 0x7f647c005bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:05.134 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.132+0000 7f6499600640 1 --2- 172.21.15.152:0/2371070821 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f647c006100 0x7f647c006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:05.134 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.132+0000 7f6499600640 1 --2- 172.21.15.152:0/2371070821 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f647c006f50 0x7f647c0073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:05.134 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.132+0000 7f6499600640 1 -- 172.21.15.152:0/2371070821 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f649c06f420 con 0x7f647c006f50 2024-04-20T11:38:05.134 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.132+0000 7f6499600640 1 -- 172.21.15.152:0/2371070821 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f649c0b4430 con 0x7f647c006100 2024-04-20T11:38:05.134 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.132+0000 7f6499600640 1 -- 172.21.15.152:0/2371070821 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f647c00dc00 con 0x7f649c0b2880 2024-04-20T11:38:05.134 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.132+0000 7f649b400640 1 --2- 172.21.15.152:0/2371070821 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f647c006f50 0x7f647c0073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:05.134 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.132+0000 7f649aa00640 1 --2- 172.21.15.152:0/2371070821 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f649c0b2880 0x7f647c005bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:05.135 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.132+0000 7f649a000640 1 --2- 172.21.15.152:0/2371070821 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f647c006100 0x7f647c006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:05.135 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.132+0000 7f649b400640 1 -- 172.21.15.152:0/2371070821 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f649c0b2880 msgr2=0x7f647c005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:05.135 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.132+0000 7f649b400640 1 --2- 172.21.15.152:0/2371070821 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f649c0b2880 0x7f647c005bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:05.135 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.132+0000 7f649b400640 1 -- 172.21.15.152:0/2371070821 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f647c006100 msgr2=0x7f647c006a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:05.135 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.132+0000 7f649b400640 1 --2- 172.21.15.152:0/2371070821 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f647c006100 0x7f647c006a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:05.135 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.133+0000 7f649b400640 1 -- 172.21.15.152:0/2371070821 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f647c005730 con 0x7f647c006f50 2024-04-20T11:38:05.136 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.134+0000 7f649b400640 1 --2- 172.21.15.152:0/2371070821 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f647c006f50 0x7f647c0073d0 secure :-1 s=READY pgs=2364 cs=0 l=1 rev1=1 crypto rx=0x7f64940098a0 tx=0x7f647c007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=169e89c3aa537783 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:05.136 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.134+0000 7f6499600640 1 -- 172.21.15.152:0/2371070821 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f649400f040 con 0x7f647c006f50 2024-04-20T11:38:05.136 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.134+0000 7f6499600640 1 -- 172.21.15.152:0/2371070821 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6494004230 con 0x7f647c006f50 2024-04-20T11:38:05.136 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.134+0000 7f6499600640 1 -- 172.21.15.152:0/2371070821 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6494003610 con 0x7f647c006f50 2024-04-20T11:38:05.136 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.134+0000 7f64a1c00640 1 -- 172.21.15.152:0/2371070821 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f647c006f50 msgr2=0x7f647c0073d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:05.136 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.134+0000 7f64a1c00640 1 --2- 172.21.15.152:0/2371070821 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f647c006f50 0x7f647c0073d0 secure :-1 s=READY pgs=2364 cs=0 l=1 rev1=1 crypto rx=0x7f64940098a0 tx=0x7f647c007a40 comp rx=0 tx=0).stop 2024-04-20T11:38:05.136 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.134+0000 7f649b400640 1 -- 172.21.15.152:0/2371070821 reap_dead start 2024-04-20T11:38:05.136 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.134+0000 7f64a1c00640 1 -- 172.21.15.152:0/2371070821 shutdown_connections 2024-04-20T11:38:05.137 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.134+0000 7f64a1c00640 1 -- 172.21.15.152:0/2371070821 >> 172.21.15.152:0/2371070821 conn(0x7f649c06b980 msgr2=0x7f649c06bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:05.137 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.135+0000 7f64a1c00640 1 -- 172.21.15.152:0/2371070821 shutdown_connections 2024-04-20T11:38:05.137 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.135+0000 7f64a1c00640 1 -- 172.21.15.152:0/2371070821 wait complete. 2024-04-20T11:38:05.137 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.136+0000 7f64a1c00640 1 Processor -- start 2024-04-20T11:38:05.137 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.137+0000 7f64a1c00640 1 -- start start 2024-04-20T11:38:05.138 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.137+0000 7f64a1c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f649c142380 0x7f649c15d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:05.138 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.137+0000 7f64a1c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f649c15de20 0x7f649c072370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:05.138 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.137+0000 7f64a1c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f649c0728b0 0x7f649c144a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:05.138 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.137+0000 7f64a1c00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f649c15b8f0 con 0x7f649c15de20 2024-04-20T11:38:05.138 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.137+0000 7f64a1c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f649c15f710 con 0x7f649c0728b0 2024-04-20T11:38:05.138 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.137+0000 7f64a1c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f649c06f950 con 0x7f649c142380 2024-04-20T11:38:05.140 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.138+0000 7f649a000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f649c15de20 0x7f649c072370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:05.140 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.138+0000 7f649a000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f649c15de20 0x7f649c072370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:55904/0 (socket says 172.21.15.152:55904) 2024-04-20T11:38:05.140 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.138+0000 7f649a000640 1 -- 172.21.15.152:0/1893447155 learned_addr learned my addr 172.21.15.152:0/1893447155 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:38:05.140 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.138+0000 7f649aa00640 1 --2- 172.21.15.152:0/1893447155 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f649c142380 0x7f649c15d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:05.140 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.138+0000 7f649b400640 1 --2- 172.21.15.152:0/1893447155 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f649c0728b0 0x7f649c144a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:05.140 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.138+0000 7f649a000640 1 -- 172.21.15.152:0/1893447155 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f649c142380 msgr2=0x7f649c15d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:05.140 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.138+0000 7f649a000640 1 --2- 172.21.15.152:0/1893447155 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f649c142380 0x7f649c15d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:05.140 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.138+0000 7f649a000640 1 -- 172.21.15.152:0/1893447155 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f649c0728b0 msgr2=0x7f649c144a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:05.140 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.138+0000 7f649a000640 1 --2- 172.21.15.152:0/1893447155 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f649c0728b0 0x7f649c144a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:05.140 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.139+0000 7f649a000640 1 -- 172.21.15.152:0/1893447155 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f649c142760 con 0x7f649c15de20 2024-04-20T11:38:05.140 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.139+0000 7f649a000640 1 --2- 172.21.15.152:0/1893447155 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f649c15de20 0x7f649c072370 secure :-1 s=READY pgs=2365 cs=0 l=1 rev1=1 crypto rx=0x7f6484001b50 tx=0x7f648400b560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:05.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.139+0000 7f6492a00640 1 -- 172.21.15.152:0/1893447155 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6484019070 con 0x7f649c15de20 2024-04-20T11:38:05.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.139+0000 7f6492a00640 1 -- 172.21.15.152:0/1893447155 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64840072d0 con 0x7f649c15de20 2024-04-20T11:38:05.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.139+0000 7f6492a00640 1 -- 172.21.15.152:0/1893447155 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64840076a0 con 0x7f649c15de20 2024-04-20T11:38:05.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.139+0000 7f64a1c00640 1 -- 172.21.15.152:0/1893447155 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f649c0cf280 con 0x7f649c15de20 2024-04-20T11:38:05.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.139+0000 7f64a1c00640 1 -- 172.21.15.152:0/1893447155 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f649c15e750 con 0x7f649c15de20 2024-04-20T11:38:05.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.140+0000 7f64a1c00640 1 -- 172.21.15.152:0/1893447155 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f649c0689f0 con 0x7f649c15de20 2024-04-20T11:38:05.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.141+0000 7f6492a00640 1 -- 172.21.15.152:0/1893447155 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 27) ==== 92060+0+0 (secure 0 0 0) 0x7f6484007840 con 0x7f649c15de20 2024-04-20T11:38:05.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.142+0000 7f6492a00640 1 --2- 172.21.15.152:0/1893447155 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f647806d460 0x7f647806f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:05.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.142+0000 7f6492a00640 1 -- 172.21.15.152:0/1893447155 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 5709+0+0 (secure 0 0 0) 0x7f648408ad00 con 0x7f649c15de20 2024-04-20T11:38:05.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.143+0000 7f649aa00640 1 --2- 172.21.15.152:0/1893447155 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f647806d460 0x7f647806f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:05.145 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.144+0000 7f649aa00640 1 --2- 172.21.15.152:0/1893447155 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f647806d460 0x7f647806f920 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f648c009a90 tx=0x7f648c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:05.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.148+0000 7f6492a00640 1 -- 172.21.15.152:0/1893447155 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f648405a930 con 0x7f649c15de20 2024-04-20T11:38:05.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.353+0000 7f64a1c00640 1 -- 172.21.15.152:0/1893447155 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f649c06ecf0 con 0x7f649c15de20 2024-04-20T11:38:05.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.354+0000 7f6492a00640 1 -- 172.21.15.152:0/1893447155 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v167) ==== 104+0+11 (secure 0 0 0) 0x7f648405a230 con 0x7f649c15de20 2024-04-20T11:38:05.355 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 32 2024-04-20T11:38:05.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.357+0000 7f64a1c00640 1 -- 172.21.15.152:0/1893447155 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f647806d460 msgr2=0x7f647806f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:05.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.357+0000 7f64a1c00640 1 --2- 172.21.15.152:0/1893447155 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f647806d460 0x7f647806f920 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f648c009a90 tx=0x7f648c008040 comp rx=0 tx=0).stop 2024-04-20T11:38:05.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.357+0000 7f64a1c00640 1 -- 172.21.15.152:0/1893447155 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f649c15de20 msgr2=0x7f649c072370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:05.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.357+0000 7f64a1c00640 1 --2- 172.21.15.152:0/1893447155 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f649c15de20 0x7f649c072370 secure :-1 s=READY pgs=2365 cs=0 l=1 rev1=1 crypto rx=0x7f6484001b50 tx=0x7f648400b560 comp rx=0 tx=0).stop 2024-04-20T11:38:05.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.357+0000 7f64a1c00640 1 -- 172.21.15.152:0/1893447155 shutdown_connections 2024-04-20T11:38:05.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.357+0000 7f64a1c00640 1 --2- 172.21.15.152:0/1893447155 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f647806d460 0x7f647806f920 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:05.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.357+0000 7f64a1c00640 1 --2- 172.21.15.152:0/1893447155 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f649c0728b0 0x7f649c144a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:05.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.357+0000 7f64a1c00640 1 --2- 172.21.15.152:0/1893447155 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f649c15de20 0x7f649c072370 unknown :-1 s=CLOSED pgs=2365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:05.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.357+0000 7f64a1c00640 1 --2- 172.21.15.152:0/1893447155 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f649c142380 0x7f649c15d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:05.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.357+0000 7f64a1c00640 1 -- 172.21.15.152:0/1893447155 >> 172.21.15.152:0/1893447155 conn(0x7f649c06b980 msgr2=0x7f649c0b01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:05.359 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.357+0000 7f64a1c00640 1 -- 172.21.15.152:0/1893447155 shutdown_connections 2024-04-20T11:38:05.359 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.357+0000 7f64a1c00640 1 -- 172.21.15.152:0/1893447155 wait complete. 2024-04-20T11:38:05.378 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T11:38:05.582 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.582+0000 7f3f41e00640 1 Processor -- start 2024-04-20T11:38:05.583 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.582+0000 7f3f41e00640 1 -- start start 2024-04-20T11:38:05.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.583+0000 7f3f41e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f3f3c162bd0 0x7f3f3c06f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:05.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.583+0000 7f3f41e00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f3c0b46a0 con 0x7f3f3c0b4890 2024-04-20T11:38:05.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.583+0000 7f3f41e00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f3c06dc60 con 0x7f3f3c06c4b0 2024-04-20T11:38:05.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.583+0000 7f3f41e00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f3f3c06e870 con 0x7f3f3c162bd0 2024-04-20T11:38:05.585 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.584+0000 7f3f40a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f3f3c162bd0 0x7f3f3c06f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:05.585 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.584+0000 7f3f3be00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f3f3c0b4890 0x7f3f3c06bda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:34770/0 (socket says 172.21.15.152:34770) 2024-04-20T11:38:05.585 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.584+0000 7f3f40a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f3f3c162bd0 0x7f3f3c06f410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:55918/0 (socket says 172.21.15.152:55918) 2024-04-20T11:38:05.585 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.584+0000 7f3f3be00640 1 -- 172.21.15.152:0/1601123534 learned_addr learned my addr 172.21.15.152:0/1601123534 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:38:05.585 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.584+0000 7f3f40a00640 1 -- 172.21.15.152:0/1601123534 >> v1:172.21.15.175:6789/0 conn(0x7f3f3c06c4b0 legacy=0x7f3f3c16c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:05.585 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.585+0000 7f3f40a00640 1 -- 172.21.15.152:0/1601123534 >> v1:172.21.15.152:6789/0 conn(0x7f3f3c0b4890 legacy=0x7f3f3c06bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:05.585 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.585+0000 7f3f40a00640 1 -- 172.21.15.152:0/1601123534 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f3c06df40 con 0x7f3f3c162bd0 2024-04-20T11:38:05.586 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.586+0000 7f3f40a00640 1 --2- 172.21.15.152:0/1601123534 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3f3c162bd0 0x7f3f3c06f410 secure :-1 s=READY pgs=2367 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c0097a0 tx=0x7f3f2c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa5a97339991cb51 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:05.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.586+0000 7f3f3b400640 1 -- 172.21.15.152:0/1601123534 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f2c035020 con 0x7f3f3c162bd0 2024-04-20T11:38:05.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.586+0000 7f3f3b400640 1 -- 172.21.15.152:0/1601123534 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f2c031830 con 0x7f3f3c162bd0 2024-04-20T11:38:05.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.586+0000 7f3f3b400640 1 -- 172.21.15.152:0/1601123534 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f2c031c00 con 0x7f3f3c162bd0 2024-04-20T11:38:05.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.587+0000 7f3f41e00640 1 -- 172.21.15.152:0/1601123534 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3f3c162bd0 msgr2=0x7f3f3c06f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:05.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.587+0000 7f3f41e00640 1 --2- 172.21.15.152:0/1601123534 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3f3c162bd0 0x7f3f3c06f410 secure :-1 s=READY pgs=2367 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c0097a0 tx=0x7f3f2c02b8d0 comp rx=0 tx=0).stop 2024-04-20T11:38:05.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.587+0000 7f3f41e00640 1 -- 172.21.15.152:0/1601123534 shutdown_connections 2024-04-20T11:38:05.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.587+0000 7f3f41e00640 1 --2- 172.21.15.152:0/1601123534 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3f3c162bd0 0x7f3f3c06f410 unknown :-1 s=CLOSED pgs=2367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:05.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.587+0000 7f3f41e00640 1 -- 172.21.15.152:0/1601123534 >> 172.21.15.152:0/1601123534 conn(0x7f3f3c0b21e0 msgr2=0x7f3f3c0b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:05.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.587+0000 7f3f41e00640 1 -- 172.21.15.152:0/1601123534 shutdown_connections 2024-04-20T11:38:05.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.587+0000 7f3f41e00640 1 -- 172.21.15.152:0/1601123534 wait complete. 2024-04-20T11:38:05.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.589+0000 7f3f41e00640 1 Processor -- start 2024-04-20T11:38:05.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.590+0000 7f3f41e00640 1 -- start start 2024-04-20T11:38:05.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.591+0000 7f3f41e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3f3c06c4b0 0x7f3f3c15eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:05.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.591+0000 7f3f41e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3f3c0b4890 0x7f3f3c161a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:05.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.591+0000 7f3f41e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3f3c162bd0 0x7f3f3c161f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:05.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.591+0000 7f3f41e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f3f3c162fb0 con 0x7f3f3c06c4b0 2024-04-20T11:38:05.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.591+0000 7f3f41e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3f3c06e500 con 0x7f3f3c0b4890 2024-04-20T11:38:05.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.591+0000 7f3f41e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f3f3c06f950 con 0x7f3f3c162bd0 2024-04-20T11:38:05.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.591+0000 7f3f40a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3f3c06c4b0 0x7f3f3c15eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:05.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.591+0000 7f3f40a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3f3c06c4b0 0x7f3f3c15eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:55920/0 (socket says 172.21.15.152:55920) 2024-04-20T11:38:05.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.591+0000 7f3f40a00640 1 -- 172.21.15.152:0/3515682051 learned_addr learned my addr 172.21.15.152:0/3515682051 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:38:05.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.592+0000 7f3f40a00640 1 -- 172.21.15.152:0/3515682051 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3f3c162bd0 msgr2=0x7f3f3c161f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:05.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.592+0000 7f3f40a00640 1 --2- 172.21.15.152:0/3515682051 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3f3c162bd0 0x7f3f3c161f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:05.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.592+0000 7f3f40a00640 1 -- 172.21.15.152:0/3515682051 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3f3c0b4890 msgr2=0x7f3f3c161a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:05.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.592+0000 7f3f3be00640 1 --2- 172.21.15.152:0/3515682051 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3f3c0b4890 0x7f3f3c161a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:05.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.592+0000 7f3f40a00640 1 --2- 172.21.15.152:0/3515682051 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3f3c0b4890 0x7f3f3c161a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:05.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.592+0000 7f3f40a00640 1 -- 172.21.15.152:0/3515682051 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f3c09ae50 con 0x7f3f3c06c4b0 2024-04-20T11:38:05.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.592+0000 7f3f40a00640 1 --2- 172.21.15.152:0/3515682051 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3f3c06c4b0 0x7f3f3c15eb30 secure :-1 s=READY pgs=2368 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c000c00 tx=0x7f3f2c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:05.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.592+0000 7f3f38c00640 1 -- 172.21.15.152:0/3515682051 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f2c03aa70 con 0x7f3f3c06c4b0 2024-04-20T11:38:05.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.592+0000 7f3f38c00640 1 -- 172.21.15.152:0/3515682051 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f2c03ac10 con 0x7f3f3c06c4b0 2024-04-20T11:38:05.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.592+0000 7f3f41e00640 1 -- 172.21.15.152:0/3515682051 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f3c160f70 con 0x7f3f3c06c4b0 2024-04-20T11:38:05.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.592+0000 7f3f38c00640 1 -- 172.21.15.152:0/3515682051 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f2c0443f0 con 0x7f3f3c06c4b0 2024-04-20T11:38:05.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.593+0000 7f3f41e00640 1 -- 172.21.15.152:0/3515682051 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f3c15f070 con 0x7f3f3c06c4b0 2024-04-20T11:38:05.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.594+0000 7f3f41e00640 1 -- 172.21.15.152:0/3515682051 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f3c075e60 con 0x7f3f3c06c4b0 2024-04-20T11:38:05.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.595+0000 7f3f38c00640 1 -- 172.21.15.152:0/3515682051 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 27) ==== 92060+0+0 (secure 0 0 0) 0x7f3f2c04b060 con 0x7f3f3c06c4b0 2024-04-20T11:38:05.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.596+0000 7f3f38c00640 1 --2- 172.21.15.152:0/3515682051 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f3f1c06d430 0x7f3f1c06f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:05.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.596+0000 7f3f38c00640 1 -- 172.21.15.152:0/3515682051 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 5709+0+0 (secure 0 0 0) 0x7f3f2c0ac200 con 0x7f3f3c06c4b0 2024-04-20T11:38:05.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.597+0000 7f3f3be00640 1 --2- 172.21.15.152:0/3515682051 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f3f1c06d430 0x7f3f1c06f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:05.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.598+0000 7f3f3be00640 1 --2- 172.21.15.152:0/3515682051 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f3f1c06d430 0x7f3f1c06f8f0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c0ae2f0 tx=0x7f3f28009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:05.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.600+0000 7f3f38c00640 1 -- 172.21.15.152:0/3515682051 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f3f2c0b2030 con 0x7f3f3c06c4b0 2024-04-20T11:38:05.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.805+0000 7f3f41e00640 1 -- 172.21.15.152:0/3515682051 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f3f3c166520 con 0x7f3f3c06c4b0 2024-04-20T11:38:05.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.807+0000 7f3f38c00640 1 -- 172.21.15.152:0/3515682051 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v167) ==== 100+0+11 (secure 0 0 0) 0x7f3f2c03f070 con 0x7f3f3c06c4b0 2024-04-20T11:38:05.807 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 64 2024-04-20T11:38:05.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.809+0000 7f3f41e00640 1 -- 172.21.15.152:0/3515682051 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f3f1c06d430 msgr2=0x7f3f1c06f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:05.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.809+0000 7f3f41e00640 1 --2- 172.21.15.152:0/3515682051 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f3f1c06d430 0x7f3f1c06f8f0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c0ae2f0 tx=0x7f3f28009040 comp rx=0 tx=0).stop 2024-04-20T11:38:05.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.809+0000 7f3f41e00640 1 -- 172.21.15.152:0/3515682051 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3f3c06c4b0 msgr2=0x7f3f3c15eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:05.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.809+0000 7f3f41e00640 1 --2- 172.21.15.152:0/3515682051 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3f3c06c4b0 0x7f3f3c15eb30 secure :-1 s=READY pgs=2368 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c000c00 tx=0x7f3f2c0046c0 comp rx=0 tx=0).stop 2024-04-20T11:38:05.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.809+0000 7f3f41e00640 1 -- 172.21.15.152:0/3515682051 shutdown_connections 2024-04-20T11:38:05.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.809+0000 7f3f41e00640 1 --2- 172.21.15.152:0/3515682051 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f3f1c06d430 0x7f3f1c06f8f0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:05.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.809+0000 7f3f41e00640 1 --2- 172.21.15.152:0/3515682051 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3f3c162bd0 0x7f3f3c161f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:05.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.809+0000 7f3f41e00640 1 --2- 172.21.15.152:0/3515682051 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3f3c0b4890 0x7f3f3c161a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:05.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.810+0000 7f3f41e00640 1 --2- 172.21.15.152:0/3515682051 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3f3c06c4b0 0x7f3f3c15eb30 unknown :-1 s=CLOSED pgs=2368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:05.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.810+0000 7f3f41e00640 1 -- 172.21.15.152:0/3515682051 >> 172.21.15.152:0/3515682051 conn(0x7f3f3c0b21e0 msgr2=0x7f3f3c165250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:05.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.810+0000 7f3f41e00640 1 -- 172.21.15.152:0/3515682051 shutdown_connections 2024-04-20T11:38:05.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:05.810+0000 7f3f41e00640 1 -- 172.21.15.152:0/3515682051 wait complete. 2024-04-20T11:38:05.830 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-20T11:38:06.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.040+0000 7fb2e9e00640 1 Processor -- start 2024-04-20T11:38:06.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.040+0000 7fb2e9e00640 1 -- start start 2024-04-20T11:38:06.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.042+0000 7fb2e9e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fb2e4161d30 0x7fb2e40d0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:06.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.042+0000 7fb2e9e00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2e40d26a0 con 0x7fb2e40d2b80 2024-04-20T11:38:06.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.042+0000 7fb2e9e00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2e40d53c0 con 0x7fb2e40d4ec0 2024-04-20T11:38:06.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.042+0000 7fb2e9e00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fb2e40d0540 con 0x7fb2e4161d30 2024-04-20T11:38:06.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.042+0000 7fb2e9400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fb2e40d2b80 0x7fb2e406c830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:34350/0 (socket says 172.21.15.152:34350) 2024-04-20T11:38:06.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.042+0000 7fb2e9400640 1 -- 172.21.15.152:0/3200566605 learned_addr learned my addr 172.21.15.152:0/3200566605 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:38:06.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.043+0000 7fb2e8a00640 1 --2- 172.21.15.152:0/3200566605 >> v2:172.21.15.175:3300/0 conn(0x7fb2e4161d30 0x7fb2e40d0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:06.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.043+0000 7fb2e3e00640 1 -- 172.21.15.152:0/3200566605 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3065977537 0 0) 0x7fb2e40d26a0 con 0x7fb2e40d2b80 2024-04-20T11:38:06.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.043+0000 7fb2e3e00640 1 -- 172.21.15.152:0/3200566605 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb2c8003520 con 0x7fb2e40d2b80 2024-04-20T11:38:06.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.043+0000 7fb2e3e00640 1 -- 172.21.15.152:0/3200566605 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 505379548 0 0) 0x7fb2c8003520 con 0x7fb2e40d2b80 2024-04-20T11:38:06.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.043+0000 7fb2e3e00640 1 -- 172.21.15.152:0/3200566605 >> v2:172.21.15.175:3300/0 conn(0x7fb2e4161d30 msgr2=0x7fb2e40d0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:06.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.043+0000 7fb2e3e00640 1 --2- 172.21.15.152:0/3200566605 >> v2:172.21.15.175:3300/0 conn(0x7fb2e4161d30 0x7fb2e40d0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:06.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.043+0000 7fb2e3e00640 1 -- 172.21.15.152:0/3200566605 >> v1:172.21.15.175:6789/0 conn(0x7fb2e40d4ec0 legacy=0x7fb2e40d15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:06.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.043+0000 7fb2e3e00640 1 -- 172.21.15.152:0/3200566605 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2e40cf520 con 0x7fb2e40d2b80 2024-04-20T11:38:06.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.043+0000 7fb2e3e00640 1 -- 172.21.15.152:0/3200566605 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7fb2dc0019f0 con 0x7fb2e40d2b80 2024-04-20T11:38:06.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.043+0000 7fb2e3e00640 1 -- 172.21.15.152:0/3200566605 >> v1:172.21.15.152:6789/0 conn(0x7fb2e40d2b80 legacy=0x7fb2e406c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:06.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.043+0000 7fb2e3e00640 1 --2- 172.21.15.152:0/3200566605 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb2c8003a00 0x7fb2c8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:06.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.043+0000 7fb2e3e00640 1 --2- 172.21.15.152:0/3200566605 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb2c8004800 0x7fb2c8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:06.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.043+0000 7fb2e3e00640 1 --2- 172.21.15.152:0/3200566605 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb2c80051c0 0x7fb2c800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:06.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.043+0000 7fb2e3e00640 1 -- 172.21.15.152:0/3200566605 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e40d0540 con 0x7fb2c80051c0 2024-04-20T11:38:06.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.043+0000 7fb2e3e00640 1 -- 172.21.15.152:0/3200566605 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c800c0c0 con 0x7fb2c8003a00 2024-04-20T11:38:06.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.043+0000 7fb2e3e00640 1 -- 172.21.15.152:0/3200566605 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb2c800c2c0 con 0x7fb2c8004800 2024-04-20T11:38:06.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.044+0000 7fb2e9400640 1 --2- 172.21.15.152:0/3200566605 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb2c80051c0 0x7fb2c800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:06.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.044+0000 7fb2e8a00640 1 --2- 172.21.15.152:0/3200566605 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb2c8003a00 0x7fb2c8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:06.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.044+0000 7fb2dbe00640 1 --2- 172.21.15.152:0/3200566605 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb2c8004800 0x7fb2c8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:06.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.044+0000 7fb2e9400640 1 -- 172.21.15.152:0/3200566605 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb2c8004800 msgr2=0x7fb2c8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:06.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.044+0000 7fb2e9400640 1 --2- 172.21.15.152:0/3200566605 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb2c8004800 0x7fb2c8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:06.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.044+0000 7fb2e9400640 1 -- 172.21.15.152:0/3200566605 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb2c8003a00 msgr2=0x7fb2c8003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:06.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.044+0000 7fb2e9400640 1 --2- 172.21.15.152:0/3200566605 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb2c8003a00 0x7fb2c8003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:06.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.044+0000 7fb2e9400640 1 -- 172.21.15.152:0/3200566605 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2c8003520 con 0x7fb2c80051c0 2024-04-20T11:38:06.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.044+0000 7fb2dbe00640 1 --2- 172.21.15.152:0/3200566605 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb2c8004800 0x7fb2c8004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:38:06.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.045+0000 7fb2e9400640 1 -- 172.21.15.152:0/3200566605 reap_dead start 2024-04-20T11:38:06.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.045+0000 7fb2e9400640 1 --2- 172.21.15.152:0/3200566605 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb2c80051c0 0x7fb2c800b9c0 secure :-1 s=READY pgs=2370 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc00a770 tx=0x7fb2c8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d84b321f0089af2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:06.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.046+0000 7fb2e3e00640 1 -- 172.21.15.152:0/3200566605 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc00a390 con 0x7fb2c80051c0 2024-04-20T11:38:06.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.046+0000 7fb2e3e00640 1 -- 172.21.15.152:0/3200566605 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2dc034650 con 0x7fb2c80051c0 2024-04-20T11:38:06.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.046+0000 7fb2e3e00640 1 -- 172.21.15.152:0/3200566605 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc034a40 con 0x7fb2c80051c0 2024-04-20T11:38:06.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.046+0000 7fb2e9e00640 1 -- 172.21.15.152:0/3200566605 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb2c80051c0 msgr2=0x7fb2c800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:06.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.046+0000 7fb2e9e00640 1 --2- 172.21.15.152:0/3200566605 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb2c80051c0 0x7fb2c800b9c0 secure :-1 s=READY pgs=2370 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc00a770 tx=0x7fb2c8005830 comp rx=0 tx=0).stop 2024-04-20T11:38:06.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.046+0000 7fb2e9e00640 1 -- 172.21.15.152:0/3200566605 shutdown_connections 2024-04-20T11:38:06.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.046+0000 7fb2e9e00640 1 --2- 172.21.15.152:0/3200566605 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb2c80051c0 0x7fb2c800b9c0 unknown :-1 s=CLOSED pgs=2370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:06.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.046+0000 7fb2e9e00640 1 -- 172.21.15.152:0/3200566605 >> 172.21.15.152:0/3200566605 conn(0x7fb2e408bfd0 msgr2=0x7fb2e40d45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:06.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.046+0000 7fb2e9e00640 1 -- 172.21.15.152:0/3200566605 shutdown_connections 2024-04-20T11:38:06.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.046+0000 7fb2e9e00640 1 -- 172.21.15.152:0/3200566605 wait complete. 2024-04-20T11:38:06.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.048+0000 7fb2e9e00640 1 Processor -- start 2024-04-20T11:38:06.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.048+0000 7fb2e9e00640 1 -- start start 2024-04-20T11:38:06.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.049+0000 7fb2e9e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb2c80051c0 0x7fb2e415cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:06.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.049+0000 7fb2e9e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb2e41f87d0 0x7fb2e415a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:06.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.049+0000 7fb2e9e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb2e414d0e0 0x7fb2e414d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:06.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.049+0000 7fb2e9e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e40cf1b0 con 0x7fb2e414d0e0 2024-04-20T11:38:06.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.049+0000 7fb2e9e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e40cfb70 con 0x7fb2e41f87d0 2024-04-20T11:38:06.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.049+0000 7fb2e9e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb2e4162110 con 0x7fb2c80051c0 2024-04-20T11:38:06.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.050+0000 7fb2e9400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb2e414d0e0 0x7fb2e414d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:06.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.050+0000 7fb2e9400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb2e414d0e0 0x7fb2e414d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:37270/0 (socket says 172.21.15.152:37270) 2024-04-20T11:38:06.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.050+0000 7fb2e9400640 1 -- 172.21.15.152:0/1621924309 learned_addr learned my addr 172.21.15.152:0/1621924309 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:38:06.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.050+0000 7fb2e8a00640 1 --2- 172.21.15.152:0/1621924309 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb2c80051c0 0x7fb2e415cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:06.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.050+0000 7fb2dbe00640 1 --2- 172.21.15.152:0/1621924309 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb2e41f87d0 0x7fb2e415a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:06.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.050+0000 7fb2e9400640 1 -- 172.21.15.152:0/1621924309 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb2c80051c0 msgr2=0x7fb2e415cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:06.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.050+0000 7fb2e9400640 1 --2- 172.21.15.152:0/1621924309 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb2c80051c0 0x7fb2e415cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:06.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.050+0000 7fb2e9400640 1 -- 172.21.15.152:0/1621924309 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb2e41f87d0 msgr2=0x7fb2e415a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:06.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.050+0000 7fb2e9400640 1 --2- 172.21.15.152:0/1621924309 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb2e41f87d0 0x7fb2e415a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:06.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.050+0000 7fb2e9400640 1 -- 172.21.15.152:0/1621924309 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2e415d5e0 con 0x7fb2e414d0e0 2024-04-20T11:38:06.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.050+0000 7fb2e9400640 1 --2- 172.21.15.152:0/1621924309 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb2e414d0e0 0x7fb2e414d560 secure :-1 s=READY pgs=2371 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc02cb80 tx=0x7fb2e40cfae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:06.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.051+0000 7fb2e1600640 1 -- 172.21.15.152:0/1621924309 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc001ee0 con 0x7fb2e414d0e0 2024-04-20T11:38:06.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.051+0000 7fb2e1600640 1 -- 172.21.15.152:0/1621924309 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2dc03a040 con 0x7fb2e414d0e0 2024-04-20T11:38:06.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.051+0000 7fb2e9e00640 1 -- 172.21.15.152:0/1621924309 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2e415d8c0 con 0x7fb2e414d0e0 2024-04-20T11:38:06.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.051+0000 7fb2e1600640 1 -- 172.21.15.152:0/1621924309 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc0019b0 con 0x7fb2e414d0e0 2024-04-20T11:38:06.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.051+0000 7fb2e9e00640 1 -- 172.21.15.152:0/1621924309 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2e415de80 con 0x7fb2e414d0e0 2024-04-20T11:38:06.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.052+0000 7fb2e9e00640 1 -- 172.21.15.152:0/1621924309 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2e4075e60 con 0x7fb2e414d0e0 2024-04-20T11:38:06.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.053+0000 7fb2e1600640 1 -- 172.21.15.152:0/1621924309 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 27) ==== 92060+0+0 (secure 0 0 0) 0x7fb2dc047020 con 0x7fb2e414d0e0 2024-04-20T11:38:06.054 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.054+0000 7fb2e1600640 1 --2- 172.21.15.152:0/1621924309 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb2c006d430 0x7fb2c006f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:06.054 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.054+0000 7fb2e1600640 1 -- 172.21.15.152:0/1621924309 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 5709+0+0 (secure 0 0 0) 0x7fb2dc0ac030 con 0x7fb2e414d0e0 2024-04-20T11:38:06.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.054+0000 7fb2e8a00640 1 --2- 172.21.15.152:0/1621924309 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb2c006d430 0x7fb2c006f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:06.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.056+0000 7fb2e8a00640 1 --2- 172.21.15.152:0/1621924309 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb2c006d430 0x7fb2c006f8f0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fb2d4009990 tx=0x7fb2d4008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:06.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.058+0000 7fb2e1600640 1 -- 172.21.15.152:0/1621924309 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fb2dc07bf10 con 0x7fb2e414d0e0 2024-04-20T11:38:06.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.273+0000 7fb2e9e00640 1 -- 172.21.15.152:0/1621924309 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb2e415ff80 con 0x7fb2e414d0e0 2024-04-20T11:38:06.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.274+0000 7fb2e1600640 1 -- 172.21.15.152:0/1621924309 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v167) ==== 80+0+0 (secure 0 0 0) 0x7fb2dc07b810 con 0x7fb2e414d0e0 2024-04-20T11:38:06.274 INFO:teuthology.orchestra.run.smithi152.stderr:listed 0 entries 2024-04-20T11:38:06.277 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.276+0000 7fb2e9e00640 1 -- 172.21.15.152:0/1621924309 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb2c006d430 msgr2=0x7fb2c006f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:06.277 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.276+0000 7fb2e9e00640 1 --2- 172.21.15.152:0/1621924309 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb2c006d430 0x7fb2c006f8f0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fb2d4009990 tx=0x7fb2d4008040 comp rx=0 tx=0).stop 2024-04-20T11:38:06.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.276+0000 7fb2e9e00640 1 -- 172.21.15.152:0/1621924309 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb2e414d0e0 msgr2=0x7fb2e414d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:06.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.276+0000 7fb2e9e00640 1 --2- 172.21.15.152:0/1621924309 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb2e414d0e0 0x7fb2e414d560 secure :-1 s=READY pgs=2371 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc02cb80 tx=0x7fb2e40cfae0 comp rx=0 tx=0).stop 2024-04-20T11:38:06.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.277+0000 7fb2e9e00640 1 -- 172.21.15.152:0/1621924309 shutdown_connections 2024-04-20T11:38:06.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.277+0000 7fb2e9e00640 1 --2- 172.21.15.152:0/1621924309 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fb2c006d430 0x7fb2c006f8f0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:06.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.277+0000 7fb2e9e00640 1 --2- 172.21.15.152:0/1621924309 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb2e414d0e0 0x7fb2e414d560 unknown :-1 s=CLOSED pgs=2371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:06.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.277+0000 7fb2e9e00640 1 --2- 172.21.15.152:0/1621924309 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb2e41f87d0 0x7fb2e415a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:06.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.277+0000 7fb2e9e00640 1 --2- 172.21.15.152:0/1621924309 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb2c80051c0 0x7fb2e415cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:06.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.277+0000 7fb2e9e00640 1 -- 172.21.15.152:0/1621924309 >> 172.21.15.152:0/1621924309 conn(0x7fb2e408bfd0 msgr2=0x7fb2e40b31e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:06.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.277+0000 7fb2e9e00640 1 -- 172.21.15.152:0/1621924309 shutdown_connections 2024-04-20T11:38:06.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.277+0000 7fb2e9e00640 1 -- 172.21.15.152:0/1621924309 wait complete. 2024-04-20T11:38:06.297 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi152.front.sepia.ceph.com... 2024-04-20T11:38:06.297 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:38:06.297 DEBUG:teuthology.orchestra.run.smithi152:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-20T11:38:06.371 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi152.stderr:ceph-fuse[74912]: fuse finished with error 0 and tester_r 0 2024-04-20T11:38:06.375 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi152.front.sepia.ceph.com 2024-04-20T11:38:06.375 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:38:06.375 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-20T11:38:06.574 INFO:tasks.cephfs.mount:Terminating background process 2024-04-20T11:38:06.574 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T11:38:06.574 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:38:06.627 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:38:06.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.831+0000 7fb00f200640 1 Processor -- start 2024-04-20T11:38:06.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.831+0000 7fb00f200640 1 -- start start 2024-04-20T11:38:06.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.832+0000 7fb00f200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb01015d1e0 0x7fb010160b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:06.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.832+0000 7fb00f200640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0100ce8c0 con 0x7fb01015d5c0 2024-04-20T11:38:06.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.833+0000 7fb00f200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0101600b0 con 0x7fb010071cd0 2024-04-20T11:38:06.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.833+0000 7fb00f200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb010160390 con 0x7fb01015d1e0 2024-04-20T11:38:06.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.833+0000 7fb00d400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fb01015d5c0 0x7fb0100cdfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:34362/0 (socket says 172.21.15.152:34362) 2024-04-20T11:38:06.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.833+0000 7fb00d400640 1 -- 172.21.15.152:0/1125100052 learned_addr learned my addr 172.21.15.152:0/1125100052 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:38:06.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.833+0000 7fb00de00640 1 --2- 172.21.15.152:0/1125100052 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb01015d1e0 0x7fb010160b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:06.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.833+0000 7fb00ca00640 1 -- 172.21.15.152:0/1125100052 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2348607299 0 0) 0x7fb0100ce8c0 con 0x7fb01015d5c0 2024-04-20T11:38:06.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.833+0000 7fb00ca00640 1 -- 172.21.15.152:0/1125100052 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faff4003520 con 0x7fb01015d5c0 2024-04-20T11:38:06.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.834+0000 7fb00ca00640 1 -- 172.21.15.152:0/1125100052 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 723133594 0 0) 0x7faff4003520 con 0x7fb01015d5c0 2024-04-20T11:38:06.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.834+0000 7fb00ca00640 1 -- 172.21.15.152:0/1125100052 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb01015d1e0 msgr2=0x7fb010160b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:06.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.834+0000 7fb00ca00640 1 --2- 172.21.15.152:0/1125100052 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb01015d1e0 0x7fb010160b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:06.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.834+0000 7fb00ca00640 1 -- 172.21.15.152:0/1125100052 >> v1:172.21.15.175:6789/0 conn(0x7fb010071cd0 legacy=0x7fb0100ccd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:06.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.834+0000 7fb00ca00640 1 -- 172.21.15.152:0/1125100052 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb010160670 con 0x7fb01015d5c0 2024-04-20T11:38:06.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.834+0000 7fb00ca00640 1 -- 172.21.15.152:0/1125100052 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7faffc001670 con 0x7fb01015d5c0 2024-04-20T11:38:06.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.834+0000 7fb00ca00640 1 -- 172.21.15.152:0/1125100052 >> v1:172.21.15.152:6789/0 conn(0x7fb01015d5c0 legacy=0x7fb0100cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:06.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.834+0000 7fb00ca00640 1 --2- 172.21.15.152:0/1125100052 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faff40039e0 0x7faff4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:06.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.834+0000 7fb00ca00640 1 --2- 172.21.15.152:0/1125100052 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb01015d1e0 0x7faff4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:06.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.834+0000 7fb00ca00640 1 --2- 172.21.15.152:0/1125100052 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faff4004d40 0x7faff40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:06.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.834+0000 7fb00ca00640 1 -- 172.21.15.152:0/1125100052 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fb010160390 con 0x7faff40039e0 2024-04-20T11:38:06.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.834+0000 7fb00ca00640 1 -- 172.21.15.152:0/1125100052 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7faff400baa0 con 0x7faff4004d40 2024-04-20T11:38:06.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.834+0000 7fb00ca00640 1 -- 172.21.15.152:0/1125100052 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7faff400bca0 con 0x7fb01015d1e0 2024-04-20T11:38:06.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.834+0000 7fb00de00640 1 --2- 172.21.15.152:0/1125100052 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faff40039e0 0x7faff4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:06.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.835+0000 7fb00d400640 1 --2- 172.21.15.152:0/1125100052 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb01015d1e0 0x7faff4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:06.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.835+0000 7fb00de00640 1 -- 172.21.15.152:0/1125100052 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb01015d1e0 msgr2=0x7faff4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:06.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.835+0000 7fb00de00640 1 --2- 172.21.15.152:0/1125100052 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb01015d1e0 0x7faff4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:06.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.835+0000 7fb00de00640 1 -- 172.21.15.152:0/1125100052 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faff4004d40 msgr2=0x7faff40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:06.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.835+0000 7fb00de00640 1 --2- 172.21.15.152:0/1125100052 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faff4004d40 0x7faff40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:06.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.835+0000 7fb00de00640 1 -- 172.21.15.152:0/1125100052 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faff4003520 con 0x7faff40039e0 2024-04-20T11:38:06.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.835+0000 7fb00d400640 1 --2- 172.21.15.152:0/1125100052 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb01015d1e0 0x7faff4004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:38:06.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.836+0000 7fb00de00640 1 --2- 172.21.15.152:0/1125100052 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faff40039e0 0x7faff4003e00 secure :-1 s=READY pgs=2373 cs=0 l=1 rev1=1 crypto rx=0x7fb000009a90 tx=0x7faff4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a7a8fb95efbe21f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:06.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.836+0000 7fb00ca00640 1 -- 172.21.15.152:0/1125100052 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb000039070 con 0x7faff40039e0 2024-04-20T11:38:06.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.836+0000 7fb00ca00640 1 -- 172.21.15.152:0/1125100052 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb00002c680 con 0x7faff40039e0 2024-04-20T11:38:06.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.836+0000 7fb00ca00640 1 -- 172.21.15.152:0/1125100052 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb000009690 con 0x7faff40039e0 2024-04-20T11:38:06.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.837+0000 7fb00f200640 1 -- 172.21.15.152:0/1125100052 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faff40039e0 msgr2=0x7faff4003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:06.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.837+0000 7fb00f200640 1 --2- 172.21.15.152:0/1125100052 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faff40039e0 0x7faff4003e00 secure :-1 s=READY pgs=2373 cs=0 l=1 rev1=1 crypto rx=0x7fb000009a90 tx=0x7faff4005830 comp rx=0 tx=0).stop 2024-04-20T11:38:06.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.837+0000 7fb00e800640 1 -- 172.21.15.152:0/1125100052 reap_dead start 2024-04-20T11:38:06.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.837+0000 7fb00f200640 1 -- 172.21.15.152:0/1125100052 shutdown_connections 2024-04-20T11:38:06.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.837+0000 7fb00f200640 1 -- 172.21.15.152:0/1125100052 >> 172.21.15.152:0/1125100052 conn(0x7fb010067ae0 msgr2=0x7fb01006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:06.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.837+0000 7fb00f200640 1 -- 172.21.15.152:0/1125100052 shutdown_connections 2024-04-20T11:38:06.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.837+0000 7fb00f200640 1 -- 172.21.15.152:0/1125100052 wait complete. 2024-04-20T11:38:06.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.838+0000 7fb00f200640 1 Processor -- start 2024-04-20T11:38:06.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.839+0000 7fb00f200640 1 -- start start 2024-04-20T11:38:06.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.839+0000 7fb00f200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faff40039e0 0x7fb010153ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:06.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.840+0000 7fb00f200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb010153ff0 0x7fb0101e6e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:06.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.840+0000 7fb00f200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb0101e73c0 0x7fb01014b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:06.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.840+0000 7fb00f200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fb010160d10 con 0x7fb010153ff0 2024-04-20T11:38:06.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.840+0000 7fb00f200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb010160b90 con 0x7fb0101e73c0 2024-04-20T11:38:06.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.840+0000 7fb00f200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb010160e90 con 0x7faff40039e0 2024-04-20T11:38:06.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.840+0000 7fb00d400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb010153ff0 0x7fb0101e6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:06.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.840+0000 7fb00d400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb010153ff0 0x7fb0101e6e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:37282/0 (socket says 172.21.15.152:37282) 2024-04-20T11:38:06.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.840+0000 7fb00d400640 1 -- 172.21.15.152:0/1057980885 learned_addr learned my addr 172.21.15.152:0/1057980885 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:38:06.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.840+0000 7fb00de00640 1 --2- 172.21.15.152:0/1057980885 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faff40039e0 0x7fb010153ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:06.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.840+0000 7fb00e800640 1 --2- 172.21.15.152:0/1057980885 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb0101e73c0 0x7fb01014b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:06.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.840+0000 7fb00d400640 1 -- 172.21.15.152:0/1057980885 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faff40039e0 msgr2=0x7fb010153ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:06.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.840+0000 7fb00d400640 1 --2- 172.21.15.152:0/1057980885 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faff40039e0 0x7fb010153ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:06.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.840+0000 7fb00d400640 1 -- 172.21.15.152:0/1057980885 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb0101e73c0 msgr2=0x7fb01014b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:06.843 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.840+0000 7fb00d400640 1 --2- 172.21.15.152:0/1057980885 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb0101e73c0 0x7fb01014b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:06.843 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.840+0000 7fb00d400640 1 -- 172.21.15.152:0/1057980885 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0101e7a60 con 0x7fb010153ff0 2024-04-20T11:38:06.843 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.840+0000 7fb00e800640 1 --2- 172.21.15.152:0/1057980885 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb0101e73c0 0x7fb01014b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:38:06.843 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.841+0000 7fb00d400640 1 --2- 172.21.15.152:0/1057980885 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb010153ff0 0x7fb0101e6e80 secure :-1 s=READY pgs=2374 cs=0 l=1 rev1=1 crypto rx=0x7faffc00d810 tx=0x7faffc00dcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:06.843 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.841+0000 7fb006000640 1 -- 172.21.15.152:0/1057980885 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faffc019070 con 0x7fb010153ff0 2024-04-20T11:38:06.843 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.841+0000 7fb006000640 1 -- 172.21.15.152:0/1057980885 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faffc0074e0 con 0x7fb010153ff0 2024-04-20T11:38:06.843 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.841+0000 7fb00f200640 1 -- 172.21.15.152:0/1057980885 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0101e7d40 con 0x7fb010153ff0 2024-04-20T11:38:06.843 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.841+0000 7fb006000640 1 -- 172.21.15.152:0/1057980885 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faffc004a00 con 0x7fb010153ff0 2024-04-20T11:38:06.843 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.841+0000 7fb00f200640 1 -- 172.21.15.152:0/1057980885 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0101e8300 con 0x7fb010153ff0 2024-04-20T11:38:06.843 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.842+0000 7fb00f200640 1 -- 172.21.15.152:0/1057980885 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb01006ef10 con 0x7fb010153ff0 2024-04-20T11:38:06.844 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.843+0000 7fb006000640 1 -- 172.21.15.152:0/1057980885 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 27) ==== 92060+0+0 (secure 0 0 0) 0x7faffc01c070 con 0x7fb010153ff0 2024-04-20T11:38:06.845 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.845+0000 7fb006000640 1 --2- 172.21.15.152:0/1057980885 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7faff006d460 0x7faff006f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:06.845 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.845+0000 7fb006000640 1 -- 172.21.15.152:0/1057980885 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 5709+0+0 (secure 0 0 0) 0x7faffc08a330 con 0x7fb010153ff0 2024-04-20T11:38:06.846 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.845+0000 7fb00de00640 1 --2- 172.21.15.152:0/1057980885 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7faff006d460 0x7faff006f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:06.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.846+0000 7fb00de00640 1 --2- 172.21.15.152:0/1057980885 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7faff006d460 0x7faff006f920 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fb000036000 tx=0x7fb000030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:06.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:06.849+0000 7fb006000640 1 -- 172.21.15.152:0/1057980885 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fb01006ef10 con 0x7fb010153ff0 2024-04-20T11:38:07.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.052+0000 7fb00f200640 1 -- 172.21.15.152:0/1057980885 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb010075e60 con 0x7fb010153ff0 2024-04-20T11:38:07.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.054+0000 7fb006000640 1 -- 172.21.15.152:0/1057980885 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v167) ==== 74+0+16743 (secure 0 0 0) 0x7faffc059ed0 con 0x7fb010153ff0 2024-04-20T11:38:07.055 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:38:07.055 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":167,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:37:49.091768+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:37:45.247938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"166","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2024-04-20T11:37:46.503171+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"167","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:38:07.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.057+0000 7fb00f200640 1 -- 172.21.15.152:0/1057980885 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7faff006d460 msgr2=0x7faff006f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:07.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.057+0000 7fb00f200640 1 --2- 172.21.15.152:0/1057980885 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7faff006d460 0x7faff006f920 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fb000036000 tx=0x7fb000030000 comp rx=0 tx=0).stop 2024-04-20T11:38:07.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.057+0000 7fb00f200640 1 -- 172.21.15.152:0/1057980885 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb010153ff0 msgr2=0x7fb0101e6e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:07.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.057+0000 7fb00f200640 1 --2- 172.21.15.152:0/1057980885 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb010153ff0 0x7fb0101e6e80 secure :-1 s=READY pgs=2374 cs=0 l=1 rev1=1 crypto rx=0x7faffc00d810 tx=0x7faffc00dcd0 comp rx=0 tx=0).stop 2024-04-20T11:38:07.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.057+0000 7fb00f200640 1 -- 172.21.15.152:0/1057980885 shutdown_connections 2024-04-20T11:38:07.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.057+0000 7fb00f200640 1 --2- 172.21.15.152:0/1057980885 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7faff006d460 0x7faff006f920 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:07.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.057+0000 7fb00f200640 1 --2- 172.21.15.152:0/1057980885 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb0101e73c0 0x7fb01014b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:07.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.057+0000 7fb00f200640 1 --2- 172.21.15.152:0/1057980885 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb010153ff0 0x7fb0101e6e80 unknown :-1 s=CLOSED pgs=2374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:07.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.057+0000 7fb00f200640 1 --2- 172.21.15.152:0/1057980885 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faff40039e0 0x7fb010153ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:07.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.057+0000 7fb00f200640 1 -- 172.21.15.152:0/1057980885 >> 172.21.15.152:0/1057980885 conn(0x7fb010067ae0 msgr2=0x7fb01015e2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:07.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.057+0000 7fb00f200640 1 -- 172.21.15.152:0/1057980885 shutdown_connections 2024-04-20T11:38:07.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.057+0000 7fb00f200640 1 -- 172.21.15.152:0/1057980885 wait complete. 2024-04-20T11:38:07.077 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T11:26:47.246571+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 26, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T11:37:45.247938+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '166', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 27, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T11:37:46.503171+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '167', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T11:38:07.078 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T11:38:07.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.283+0000 7f2bafa00640 1 Processor -- start 2024-04-20T11:38:07.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.283+0000 7f2bafa00640 1 -- start start 2024-04-20T11:38:07.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.285+0000 7f2bafa00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2ba815c050 0x7f2ba8071db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:07.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.285+0000 7f2bafa00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f2ba80722f0 0x7f2ba80b3c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:07.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.285+0000 7f2bafa00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ba815b6a0 con 0x7f2ba80b4160 2024-04-20T11:38:07.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.285+0000 7f2bafa00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f2ba80b45d0 con 0x7f2ba80722f0 2024-04-20T11:38:07.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.285+0000 7f2bafa00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2ba8155810 con 0x7f2ba815c050 2024-04-20T11:38:07.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.286+0000 7f2bad000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2ba815c050 0x7f2ba8071db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:07.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.286+0000 7f2ba7e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f2ba80722f0 0x7f2ba80b3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:07.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.286+0000 7f2bad000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2ba815c050 0x7f2ba8071db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.152:49594/0 (socket says 172.21.15.152:49594) 2024-04-20T11:38:07.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.286+0000 7f2bad000640 1 -- 172.21.15.152:0/2233058682 learned_addr learned my addr 172.21.15.152:0/2233058682 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:38:07.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.286+0000 7f2bada00640 1 --1- 172.21.15.152:0/2233058682 >> v1:172.21.15.175:6789/0 conn(0x7f2ba80b4160 0x7f2ba815e8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.152:48936/0 (socket says 172.21.15.152:48936) 2024-04-20T11:38:07.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.286+0000 7f2bad000640 1 -- 172.21.15.152:0/2233058682 >> v2:172.21.15.175:3300/0 conn(0x7f2ba80722f0 msgr2=0x7f2ba80b3c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:07.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.287+0000 7f2bad000640 1 --2- 172.21.15.152:0/2233058682 >> v2:172.21.15.175:3300/0 conn(0x7f2ba80722f0 0x7f2ba80b3c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:07.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.287+0000 7f2bad000640 1 -- 172.21.15.152:0/2233058682 >> v1:172.21.15.175:6789/0 conn(0x7f2ba80b4160 legacy=0x7f2ba815e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:07.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.287+0000 7f2bad000640 1 -- 172.21.15.152:0/2233058682 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ba80b4800 con 0x7f2ba815c050 2024-04-20T11:38:07.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.288+0000 7f2bad000640 1 --2- 172.21.15.152:0/2233058682 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2ba815c050 0x7f2ba8071db0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2b9c00bb10 tx=0x7f2b9c02e0c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8520db6d4bca889b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:07.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.288+0000 7f2ba7400640 1 -- 172.21.15.152:0/2233058682 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b9c03b070 con 0x7f2ba815c050 2024-04-20T11:38:07.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.289+0000 7f2ba7400640 1 -- 172.21.15.152:0/2233058682 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b9c036440 con 0x7f2ba815c050 2024-04-20T11:38:07.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.289+0000 7f2ba7400640 1 -- 172.21.15.152:0/2233058682 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b9c036810 con 0x7f2ba815c050 2024-04-20T11:38:07.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.289+0000 7f2bafa00640 1 -- 172.21.15.152:0/2233058682 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2ba815c050 msgr2=0x7f2ba8071db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:07.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.289+0000 7f2bafa00640 1 --2- 172.21.15.152:0/2233058682 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2ba815c050 0x7f2ba8071db0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2b9c00bb10 tx=0x7f2b9c02e0c0 comp rx=0 tx=0).stop 2024-04-20T11:38:07.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.289+0000 7f2bafa00640 1 -- 172.21.15.152:0/2233058682 shutdown_connections 2024-04-20T11:38:07.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.289+0000 7f2bafa00640 1 --2- 172.21.15.152:0/2233058682 >> v2:172.21.15.175:3300/0 conn(0x7f2ba80722f0 0x7f2ba80b3c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:07.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.289+0000 7f2bafa00640 1 --2- 172.21.15.152:0/2233058682 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2ba815c050 0x7f2ba8071db0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:07.291 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.289+0000 7f2bafa00640 1 -- 172.21.15.152:0/2233058682 >> 172.21.15.152:0/2233058682 conn(0x7f2ba806b980 msgr2=0x7f2ba806bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:07.291 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.289+0000 7f2bafa00640 1 -- 172.21.15.152:0/2233058682 shutdown_connections 2024-04-20T11:38:07.291 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.290+0000 7f2bafa00640 1 -- 172.21.15.152:0/2233058682 wait complete. 2024-04-20T11:38:07.292 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.291+0000 7f2bafa00640 1 Processor -- start 2024-04-20T11:38:07.292 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.292+0000 7f2bafa00640 1 -- start start 2024-04-20T11:38:07.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.293+0000 7f2bafa00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ba80722f0 0x7f2ba81f3de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:07.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.293+0000 7f2bafa00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2ba80b4160 0x7f2ba8159dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:07.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.293+0000 7f2bafa00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ba815c050 0x7f2ba815a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:07.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.293+0000 7f2bafa00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f2ba80b4dc0 con 0x7f2ba80722f0 2024-04-20T11:38:07.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.293+0000 7f2bafa00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2ba806dc50 con 0x7f2ba815c050 2024-04-20T11:38:07.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.293+0000 7f2bafa00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2ba806dea0 con 0x7f2ba80b4160 2024-04-20T11:38:07.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.293+0000 7f2bad000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ba80722f0 0x7f2ba81f3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:07.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.293+0000 7f2bad000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ba80722f0 0x7f2ba81f3de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:37290/0 (socket says 172.21.15.152:37290) 2024-04-20T11:38:07.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.293+0000 7f2bad000640 1 -- 172.21.15.152:0/771793422 learned_addr learned my addr 172.21.15.152:0/771793422 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:38:07.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.293+0000 7f2bad000640 1 -- 172.21.15.152:0/771793422 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2ba80b4160 msgr2=0x7f2ba8159dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:07.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.293+0000 7f2bad000640 1 --2- 172.21.15.152:0/771793422 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2ba80b4160 0x7f2ba8159dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:07.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.293+0000 7f2bad000640 1 -- 172.21.15.152:0/771793422 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ba815c050 msgr2=0x7f2ba815a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:07.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.293+0000 7f2bad000640 1 --2- 172.21.15.152:0/771793422 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ba815c050 0x7f2ba815a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:07.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.293+0000 7f2bad000640 1 -- 172.21.15.152:0/771793422 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ba80aae30 con 0x7f2ba80722f0 2024-04-20T11:38:07.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.294+0000 7f2bad000640 1 --2- 172.21.15.152:0/771793422 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ba80722f0 0x7f2ba81f3de0 secure :-1 s=READY pgs=2375 cs=0 l=1 rev1=1 crypto rx=0x7f2b9c02e100 tx=0x7f2b9c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:07.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.294+0000 7f2ba4c00640 1 -- 172.21.15.152:0/771793422 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b9c03b030 con 0x7f2ba80722f0 2024-04-20T11:38:07.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.294+0000 7f2ba4c00640 1 -- 172.21.15.152:0/771793422 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b9c03d050 con 0x7f2ba80722f0 2024-04-20T11:38:07.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.294+0000 7f2bafa00640 1 -- 172.21.15.152:0/771793422 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ba80ab110 con 0x7f2ba80722f0 2024-04-20T11:38:07.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.294+0000 7f2ba4c00640 1 -- 172.21.15.152:0/771793422 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b9c034870 con 0x7f2ba80722f0 2024-04-20T11:38:07.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.294+0000 7f2bafa00640 1 -- 172.21.15.152:0/771793422 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ba8151b50 con 0x7f2ba80722f0 2024-04-20T11:38:07.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.295+0000 7f2bafa00640 1 -- 172.21.15.152:0/771793422 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ba80689f0 con 0x7f2ba80722f0 2024-04-20T11:38:07.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.296+0000 7f2ba4c00640 1 -- 172.21.15.152:0/771793422 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 27) ==== 92060+0+0 (secure 0 0 0) 0x7f2b9c048060 con 0x7f2ba80722f0 2024-04-20T11:38:07.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.297+0000 7f2ba4c00640 1 --2- 172.21.15.152:0/771793422 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f2b8806d430 0x7f2b8806f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:07.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.297+0000 7f2ba4c00640 1 -- 172.21.15.152:0/771793422 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 5709+0+0 (secure 0 0 0) 0x7f2b9c0abb60 con 0x7f2ba80722f0 2024-04-20T11:38:07.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.298+0000 7f2ba7e00640 1 --2- 172.21.15.152:0/771793422 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f2b8806d430 0x7f2b8806f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:07.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.299+0000 7f2ba7e00640 1 --2- 172.21.15.152:0/771793422 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f2b8806d430 0x7f2b8806f8f0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f2b94006a90 tx=0x7f2b94006a20 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:07.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.302+0000 7f2ba4c00640 1 -- 172.21.15.152:0/771793422 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f2b9c07b920 con 0x7f2ba80722f0 2024-04-20T11:38:07.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.499+0000 7f2bafa00640 1 -- 172.21.15.152:0/771793422 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2ba8072a90 con 0x7f2ba80722f0 2024-04-20T11:38:07.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.500+0000 7f2ba4c00640 1 -- 172.21.15.152:0/771793422 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v167) ==== 93+0+10 (secure 0 0 0) 0x7f2b9c07b220 con 0x7f2ba80722f0 2024-04-20T11:38:07.501 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 1 2024-04-20T11:38:07.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.503+0000 7f2bafa00640 1 -- 172.21.15.152:0/771793422 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f2b8806d430 msgr2=0x7f2b8806f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:07.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.503+0000 7f2bafa00640 1 --2- 172.21.15.152:0/771793422 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f2b8806d430 0x7f2b8806f8f0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f2b94006a90 tx=0x7f2b94006a20 comp rx=0 tx=0).stop 2024-04-20T11:38:07.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.503+0000 7f2bafa00640 1 -- 172.21.15.152:0/771793422 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ba80722f0 msgr2=0x7f2ba81f3de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:07.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.503+0000 7f2bafa00640 1 --2- 172.21.15.152:0/771793422 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ba80722f0 0x7f2ba81f3de0 secure :-1 s=READY pgs=2375 cs=0 l=1 rev1=1 crypto rx=0x7f2b9c02e100 tx=0x7f2b9c0046c0 comp rx=0 tx=0).stop 2024-04-20T11:38:07.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.503+0000 7f2bafa00640 1 -- 172.21.15.152:0/771793422 shutdown_connections 2024-04-20T11:38:07.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.503+0000 7f2bafa00640 1 --2- 172.21.15.152:0/771793422 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f2b8806d430 0x7f2b8806f8f0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:07.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.503+0000 7f2bafa00640 1 --2- 172.21.15.152:0/771793422 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ba815c050 0x7f2ba815a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:07.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.503+0000 7f2bafa00640 1 --2- 172.21.15.152:0/771793422 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2ba80b4160 0x7f2ba8159dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:07.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.503+0000 7f2bafa00640 1 --2- 172.21.15.152:0/771793422 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ba80722f0 0x7f2ba81f3de0 unknown :-1 s=CLOSED pgs=2375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:07.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.503+0000 7f2bafa00640 1 -- 172.21.15.152:0/771793422 >> 172.21.15.152:0/771793422 conn(0x7f2ba806b980 msgr2=0x7f2ba80b21c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:07.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.503+0000 7f2bafa00640 1 -- 172.21.15.152:0/771793422 shutdown_connections 2024-04-20T11:38:07.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.503+0000 7f2bafa00640 1 -- 172.21.15.152:0/771793422 wait complete. 2024-04-20T11:38:07.521 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T11:38:07.726 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.725+0000 7f1f8c400640 1 Processor -- start 2024-04-20T11:38:07.726 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.726+0000 7f1f8c400640 1 -- start start 2024-04-20T11:38:07.728 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.727+0000 7f1f8c400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1f840b2880 0x7f1f8416d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:07.728 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.727+0000 7f1f8c400640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f8406fc30 con 0x7f1f8406f720 2024-04-20T11:38:07.728 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.727+0000 7f1f8c400640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f840b4090 con 0x7f1f84162bc0 2024-04-20T11:38:07.728 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.727+0000 7f1f8c400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1f840ccb40 con 0x7f1f840b2880 2024-04-20T11:38:07.728 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.728+0000 7f1f89000640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f1f8406f720 0x7f1f840b2170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:34368/0 (socket says 172.21.15.152:34368) 2024-04-20T11:38:07.728 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.728+0000 7f1f89000640 1 -- 172.21.15.152:0/484938093 learned_addr learned my addr 172.21.15.152:0/484938093 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:38:07.730 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.728+0000 7f1f8a400640 1 --2- 172.21.15.152:0/484938093 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1f840b2880 0x7f1f8416d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:07.731 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.728+0000 7f1f83e00640 1 -- 172.21.15.152:0/484938093 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1046606717 0 0) 0x7f1f8406fc30 con 0x7f1f8406f720 2024-04-20T11:38:07.731 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.728+0000 7f1f83e00640 1 -- 172.21.15.152:0/484938093 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1f6c003520 con 0x7f1f8406f720 2024-04-20T11:38:07.731 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.728+0000 7f1f83e00640 1 -- 172.21.15.152:0/484938093 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2392055450 0 0) 0x7f1f6c003520 con 0x7f1f8406f720 2024-04-20T11:38:07.731 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.729+0000 7f1f83e00640 1 -- 172.21.15.152:0/484938093 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1f840b2880 msgr2=0x7f1f8416d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:07.731 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.729+0000 7f1f83e00640 1 --2- 172.21.15.152:0/484938093 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1f840b2880 0x7f1f8416d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:07.731 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.729+0000 7f1f83e00640 1 -- 172.21.15.152:0/484938093 >> v1:172.21.15.175:6789/0 conn(0x7f1f84162bc0 legacy=0x7f1f84167240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:07.731 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.729+0000 7f1f83e00640 1 -- 172.21.15.152:0/484938093 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f840b4370 con 0x7f1f8406f720 2024-04-20T11:38:07.731 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.729+0000 7f1f83e00640 1 -- 172.21.15.152:0/484938093 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f1f74001690 con 0x7f1f8406f720 2024-04-20T11:38:07.731 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.729+0000 7f1f83e00640 1 -- 172.21.15.152:0/484938093 >> v1:172.21.15.152:6789/0 conn(0x7f1f8406f720 legacy=0x7f1f840b2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:07.731 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.729+0000 7f1f83e00640 1 --2- 172.21.15.152:0/484938093 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1f6c0039e0 0x7f1f6c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:07.731 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.729+0000 7f1f83e00640 1 --2- 172.21.15.152:0/484938093 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1f6c004800 0x7f1f6c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:07.731 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.729+0000 7f1f83e00640 1 --2- 172.21.15.152:0/484938093 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f6c0051c0 0x7f1f6c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:07.731 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.729+0000 7f1f83e00640 1 -- 172.21.15.152:0/484938093 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f1f840ccb40 con 0x7f1f6c004800 2024-04-20T11:38:07.731 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.729+0000 7f1f83e00640 1 -- 172.21.15.152:0/484938093 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1f6c00c090 con 0x7f1f6c0051c0 2024-04-20T11:38:07.732 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.729+0000 7f1f83e00640 1 -- 172.21.15.152:0/484938093 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1f6c00c290 con 0x7f1f6c0039e0 2024-04-20T11:38:07.732 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.729+0000 7f1f8a400640 1 --2- 172.21.15.152:0/484938093 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1f840b2880 0x7f1f8416d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:38:07.732 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.729+0000 7f1f89000640 1 --2- 172.21.15.152:0/484938093 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1f6c004800 0x7f1f6c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:07.732 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.729+0000 7f1f89000640 1 -- 172.21.15.152:0/484938093 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1f6c0039e0 msgr2=0x7f1f6c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:07.732 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.729+0000 7f1f89a00640 1 --2- 172.21.15.152:0/484938093 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1f6c0039e0 0x7f1f6c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:07.732 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.730+0000 7f1f8a400640 1 --2- 172.21.15.152:0/484938093 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f6c0051c0 0x7f1f6c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:07.732 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.730+0000 7f1f89000640 1 --2- 172.21.15.152:0/484938093 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1f6c0039e0 0x7f1f6c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:07.732 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.730+0000 7f1f89000640 1 -- 172.21.15.152:0/484938093 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f6c0051c0 msgr2=0x7f1f6c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:07.732 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.730+0000 7f1f89000640 1 --2- 172.21.15.152:0/484938093 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f6c0051c0 0x7f1f6c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:07.732 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.730+0000 7f1f89000640 1 -- 172.21.15.152:0/484938093 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f6c003520 con 0x7f1f6c004800 2024-04-20T11:38:07.732 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.730+0000 7f1f89a00640 1 --2- 172.21.15.152:0/484938093 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1f6c0039e0 0x7f1f6c003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:38:07.732 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.730+0000 7f1f8a400640 1 --2- 172.21.15.152:0/484938093 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f6c0051c0 0x7f1f6c00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:38:07.733 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.731+0000 7f1f89000640 1 --2- 172.21.15.152:0/484938093 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1f6c004800 0x7f1f6c004c80 secure :-1 s=READY pgs=2377 cs=0 l=1 rev1=1 crypto rx=0x7f1f840b2f70 tx=0x7f1f6c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23d4b44533bc12ff server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:07.733 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.731+0000 7f1f83e00640 1 -- 172.21.15.152:0/484938093 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f74007d10 con 0x7f1f6c004800 2024-04-20T11:38:07.734 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.731+0000 7f1f83e00640 1 -- 172.21.15.152:0/484938093 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f74030070 con 0x7f1f6c004800 2024-04-20T11:38:07.734 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.731+0000 7f1f83e00640 1 -- 172.21.15.152:0/484938093 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f74005d30 con 0x7f1f6c004800 2024-04-20T11:38:07.734 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.732+0000 7f1f8c400640 1 -- 172.21.15.152:0/484938093 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1f6c004800 msgr2=0x7f1f6c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:07.734 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.732+0000 7f1f8c400640 1 --2- 172.21.15.152:0/484938093 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1f6c004800 0x7f1f6c004c80 secure :-1 s=READY pgs=2377 cs=0 l=1 rev1=1 crypto rx=0x7f1f840b2f70 tx=0x7f1f6c005830 comp rx=0 tx=0).stop 2024-04-20T11:38:07.734 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.732+0000 7f1f8a400640 1 -- 172.21.15.152:0/484938093 reap_dead start 2024-04-20T11:38:07.734 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.732+0000 7f1f8c400640 1 -- 172.21.15.152:0/484938093 shutdown_connections 2024-04-20T11:38:07.734 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.732+0000 7f1f8c400640 1 -- 172.21.15.152:0/484938093 >> 172.21.15.152:0/484938093 conn(0x7f1f8406b980 msgr2=0x7f1f8406bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:07.734 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.732+0000 7f1f8c400640 1 -- 172.21.15.152:0/484938093 shutdown_connections 2024-04-20T11:38:07.734 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.732+0000 7f1f8c400640 1 -- 172.21.15.152:0/484938093 wait complete. 2024-04-20T11:38:07.735 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.734+0000 7f1f8c400640 1 Processor -- start 2024-04-20T11:38:07.735 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.734+0000 7f1f8c400640 1 -- start start 2024-04-20T11:38:07.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.735+0000 7f1f8c400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1f6c004800 0x7f1f8415ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:07.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.735+0000 7f1f8c400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1f8415ef60 0x7f1f841f92d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:07.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.735+0000 7f1f8c400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f841f9810 0x7f1f841f9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:07.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.735+0000 7f1f8c400640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f1f84162fa0 con 0x7f1f6c004800 2024-04-20T11:38:07.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.735+0000 7f1f8c400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1f840b3610 con 0x7f1f841f9810 2024-04-20T11:38:07.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.735+0000 7f1f8c400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1f84163120 con 0x7f1f8415ef60 2024-04-20T11:38:07.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.735+0000 7f1f89a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1f6c004800 0x7f1f8415ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:07.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.735+0000 7f1f89a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1f6c004800 0x7f1f8415ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:37314/0 (socket says 172.21.15.152:37314) 2024-04-20T11:38:07.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.735+0000 7f1f89a00640 1 -- 172.21.15.152:0/1539042979 learned_addr learned my addr 172.21.15.152:0/1539042979 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:38:07.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.735+0000 7f1f89a00640 1 -- 172.21.15.152:0/1539042979 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1f8415ef60 msgr2=0x7f1f841f92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:07.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.735+0000 7f1f89a00640 1 --2- 172.21.15.152:0/1539042979 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1f8415ef60 0x7f1f841f92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:07.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.735+0000 7f1f89a00640 1 -- 172.21.15.152:0/1539042979 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f841f9810 msgr2=0x7f1f841f9ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:07.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.735+0000 7f1f89a00640 1 --2- 172.21.15.152:0/1539042979 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f841f9810 0x7f1f841f9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:07.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.735+0000 7f1f89a00640 1 -- 172.21.15.152:0/1539042979 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f84156320 con 0x7f1f6c004800 2024-04-20T11:38:07.738 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.736+0000 7f1f89a00640 1 --2- 172.21.15.152:0/1539042979 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1f6c004800 0x7f1f8415ea20 secure :-1 s=READY pgs=2378 cs=0 l=1 rev1=1 crypto rx=0x7f1f7800d7a0 tx=0x7f1f7800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:07.738 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.736+0000 7f1f81600640 1 -- 172.21.15.152:0/1539042979 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f78005070 con 0x7f1f6c004800 2024-04-20T11:38:07.738 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.736+0000 7f1f81600640 1 -- 172.21.15.152:0/1539042979 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f78005210 con 0x7f1f6c004800 2024-04-20T11:38:07.738 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.736+0000 7f1f81600640 1 -- 172.21.15.152:0/1539042979 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f78007030 con 0x7f1f6c004800 2024-04-20T11:38:07.738 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.736+0000 7f1f8c400640 1 -- 172.21.15.152:0/1539042979 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f8415f470 con 0x7f1f6c004800 2024-04-20T11:38:07.738 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.736+0000 7f1f8c400640 1 -- 172.21.15.152:0/1539042979 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f8415fa30 con 0x7f1f6c004800 2024-04-20T11:38:07.738 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.737+0000 7f1f8c400640 1 -- 172.21.15.152:0/1539042979 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f8406f0b0 con 0x7f1f6c004800 2024-04-20T11:38:07.739 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.738+0000 7f1f81600640 1 -- 172.21.15.152:0/1539042979 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 27) ==== 92060+0+0 (secure 0 0 0) 0x7f1f7801a030 con 0x7f1f6c004800 2024-04-20T11:38:07.740 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.740+0000 7f1f81600640 1 --2- 172.21.15.152:0/1539042979 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f1f6806d460 0x7f1f6806f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:07.740 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.740+0000 7f1f81600640 1 -- 172.21.15.152:0/1539042979 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 5709+0+0 (secure 0 0 0) 0x7f1f78089bf0 con 0x7f1f6c004800 2024-04-20T11:38:07.740 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.740+0000 7f1f89000640 1 --2- 172.21.15.152:0/1539042979 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f1f6806d460 0x7f1f6806f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:07.742 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.742+0000 7f1f89000640 1 --2- 172.21.15.152:0/1539042979 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f1f6806d460 0x7f1f6806f920 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f1f840b2f70 tx=0x7f1f74039040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:07.744 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.744+0000 7f1f81600640 1 -- 172.21.15.152:0/1539042979 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f1f8406f0b0 con 0x7f1f6c004800 2024-04-20T11:38:07.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.942+0000 7f1f8c400640 1 -- 172.21.15.152:0/1539042979 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f1f840689f0 con 0x7f1f6c004800 2024-04-20T11:38:07.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.942+0000 7f1f81600640 1 -- 172.21.15.152:0/1539042979 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v167) ==== 104+0+11 (secure 0 0 0) 0x7f1f78021020 con 0x7f1f6c004800 2024-04-20T11:38:07.943 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 32 2024-04-20T11:38:07.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.945+0000 7f1f8c400640 1 -- 172.21.15.152:0/1539042979 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f1f6806d460 msgr2=0x7f1f6806f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:07.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.945+0000 7f1f8c400640 1 --2- 172.21.15.152:0/1539042979 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f1f6806d460 0x7f1f6806f920 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f1f840b2f70 tx=0x7f1f74039040 comp rx=0 tx=0).stop 2024-04-20T11:38:07.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.945+0000 7f1f8c400640 1 -- 172.21.15.152:0/1539042979 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1f6c004800 msgr2=0x7f1f8415ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:07.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.945+0000 7f1f8c400640 1 --2- 172.21.15.152:0/1539042979 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1f6c004800 0x7f1f8415ea20 secure :-1 s=READY pgs=2378 cs=0 l=1 rev1=1 crypto rx=0x7f1f7800d7a0 tx=0x7f1f7800dc60 comp rx=0 tx=0).stop 2024-04-20T11:38:07.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.945+0000 7f1f8c400640 1 -- 172.21.15.152:0/1539042979 shutdown_connections 2024-04-20T11:38:07.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.945+0000 7f1f8c400640 1 --2- 172.21.15.152:0/1539042979 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f1f6806d460 0x7f1f6806f920 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:07.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.945+0000 7f1f8c400640 1 --2- 172.21.15.152:0/1539042979 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f841f9810 0x7f1f841f9ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:07.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.945+0000 7f1f8c400640 1 --2- 172.21.15.152:0/1539042979 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1f8415ef60 0x7f1f841f92d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:07.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.945+0000 7f1f8c400640 1 --2- 172.21.15.152:0/1539042979 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1f6c004800 0x7f1f8415ea20 unknown :-1 s=CLOSED pgs=2378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:07.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.945+0000 7f1f8c400640 1 -- 172.21.15.152:0/1539042979 >> 172.21.15.152:0/1539042979 conn(0x7f1f8406b980 msgr2=0x7f1f840b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:07.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.945+0000 7f1f8c400640 1 -- 172.21.15.152:0/1539042979 shutdown_connections 2024-04-20T11:38:07.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:07.945+0000 7f1f8c400640 1 -- 172.21.15.152:0/1539042979 wait complete. 2024-04-20T11:38:07.964 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T11:38:08.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.169+0000 7f4c63200640 1 Processor -- start 2024-04-20T11:38:08.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.169+0000 7f4c63200640 1 -- start start 2024-04-20T11:38:08.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.171+0000 7f4c63200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f4c6415ca40 0x7f4c6415cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:08.172 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.171+0000 7f4c63200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4c6406bda0 0x7f4c6406c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:08.172 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.171+0000 7f4c63200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c6415d5c0 con 0x7f4c64156c00 2024-04-20T11:38:08.172 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.171+0000 7f4c63200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f4c6415d8a0 con 0x7f4c6415ca40 2024-04-20T11:38:08.172 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.171+0000 7f4c63200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4c64160290 con 0x7f4c6406bda0 2024-04-20T11:38:08.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.171+0000 7f4c62800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4c6406bda0 0x7f4c6406c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:08.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.172+0000 7f4c61400640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f4c6415ca40 0x7f4c6415cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:08.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.172+0000 7f4c62800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4c6406bda0 0x7f4c6406c230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.152:49648/0 (socket says 172.21.15.152:49648) 2024-04-20T11:38:08.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.172+0000 7f4c61e00640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7f4c64156c00 0x7f4c64160a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.152:48962/0 (socket says 172.21.15.152:48962) 2024-04-20T11:38:08.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.172+0000 7f4c62800640 1 -- 172.21.15.152:0/2190466854 learned_addr learned my addr 172.21.15.152:0/2190466854 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:38:08.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.172+0000 7f4c62800640 1 -- 172.21.15.152:0/2190466854 >> v2:172.21.15.175:3300/0 conn(0x7f4c6415ca40 msgr2=0x7f4c6415cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:08.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.172+0000 7f4c62800640 1 --2- 172.21.15.152:0/2190466854 >> v2:172.21.15.175:3300/0 conn(0x7f4c6415ca40 0x7f4c6415cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:08.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.172+0000 7f4c62800640 1 -- 172.21.15.152:0/2190466854 >> v1:172.21.15.175:6789/0 conn(0x7f4c64156c00 legacy=0x7f4c64160a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:08.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.172+0000 7f4c62800640 1 -- 172.21.15.152:0/2190466854 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c640ad2e0 con 0x7f4c6406bda0 2024-04-20T11:38:08.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.173+0000 7f4c61400640 1 --2- 172.21.15.152:0/2190466854 >> v2:172.21.15.175:3300/0 conn(0x7f4c6415ca40 0x7f4c6415cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:38:08.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.174+0000 7f4c62800640 1 --2- 172.21.15.152:0/2190466854 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4c6406bda0 0x7f4c6406c230 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c009e50 tx=0x7f4c5c02bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2228251a6ad2a90 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:08.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.175+0000 7f4c60a00640 1 -- 172.21.15.152:0/2190466854 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c5c02cb00 con 0x7f4c6406bda0 2024-04-20T11:38:08.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.175+0000 7f4c60a00640 1 -- 172.21.15.152:0/2190466854 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c5c02cca0 con 0x7f4c6406bda0 2024-04-20T11:38:08.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.175+0000 7f4c60a00640 1 -- 172.21.15.152:0/2190466854 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c5c034610 con 0x7f4c6406bda0 2024-04-20T11:38:08.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.175+0000 7f4c63200640 1 -- 172.21.15.152:0/2190466854 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4c6406bda0 msgr2=0x7f4c6406c230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:08.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.175+0000 7f4c63200640 1 --2- 172.21.15.152:0/2190466854 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4c6406bda0 0x7f4c6406c230 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c009e50 tx=0x7f4c5c02bf60 comp rx=0 tx=0).stop 2024-04-20T11:38:08.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.176+0000 7f4c63200640 1 -- 172.21.15.152:0/2190466854 shutdown_connections 2024-04-20T11:38:08.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.176+0000 7f4c63200640 1 --2- 172.21.15.152:0/2190466854 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4c6406bda0 0x7f4c6406c230 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:08.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.176+0000 7f4c63200640 1 --2- 172.21.15.152:0/2190466854 >> v2:172.21.15.175:3300/0 conn(0x7f4c6415ca40 0x7f4c6415cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:08.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.176+0000 7f4c63200640 1 -- 172.21.15.152:0/2190466854 >> 172.21.15.152:0/2190466854 conn(0x7f4c640b21e0 msgr2=0x7f4c640b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:08.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.176+0000 7f4c63200640 1 -- 172.21.15.152:0/2190466854 shutdown_connections 2024-04-20T11:38:08.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.176+0000 7f4c63200640 1 -- 172.21.15.152:0/2190466854 wait complete. 2024-04-20T11:38:08.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.178+0000 7f4c63200640 1 Processor -- start 2024-04-20T11:38:08.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.178+0000 7f4c63200640 1 -- start start 2024-04-20T11:38:08.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.179+0000 7f4c63200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4c6406bda0 0x7f4c641f5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:08.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.179+0000 7f4c63200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c64156c00 0x7f4c6414b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:08.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.179+0000 7f4c63200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4c6415ca40 0x7f4c6415e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:08.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.179+0000 7f4c63200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f4c640b34d0 con 0x7f4c64156c00 2024-04-20T11:38:08.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.179+0000 7f4c63200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4c640b3350 con 0x7f4c6415ca40 2024-04-20T11:38:08.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.179+0000 7f4c63200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4c640b3650 con 0x7f4c6406bda0 2024-04-20T11:38:08.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.179+0000 7f4c61400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c64156c00 0x7f4c6414b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:08.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.180+0000 7f4c61400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c64156c00 0x7f4c6414b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:37328/0 (socket says 172.21.15.152:37328) 2024-04-20T11:38:08.181 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.180+0000 7f4c61400640 1 -- 172.21.15.152:0/897986590 learned_addr learned my addr 172.21.15.152:0/897986590 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:38:08.181 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.180+0000 7f4c61e00640 1 --2- 172.21.15.152:0/897986590 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4c6406bda0 0x7f4c641f5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:08.182 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.180+0000 7f4c62800640 1 --2- 172.21.15.152:0/897986590 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4c6415ca40 0x7f4c6415e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:08.182 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.180+0000 7f4c61400640 1 -- 172.21.15.152:0/897986590 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4c6406bda0 msgr2=0x7f4c641f5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:08.182 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.180+0000 7f4c61400640 1 --2- 172.21.15.152:0/897986590 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4c6406bda0 0x7f4c641f5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:08.182 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.180+0000 7f4c61400640 1 -- 172.21.15.152:0/897986590 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4c6415ca40 msgr2=0x7f4c6415e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:08.182 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.180+0000 7f4c61400640 1 --2- 172.21.15.152:0/897986590 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4c6415ca40 0x7f4c6415e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:08.182 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.180+0000 7f4c61400640 1 -- 172.21.15.152:0/897986590 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c6414bbe0 con 0x7f4c64156c00 2024-04-20T11:38:08.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.180+0000 7f4c62800640 1 --2- 172.21.15.152:0/897986590 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4c6415ca40 0x7f4c6415e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:38:08.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.180+0000 7f4c61400640 1 --2- 172.21.15.152:0/897986590 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c64156c00 0x7f4c6414b6a0 secure :-1 s=READY pgs=2379 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c00e960 tx=0x7f4c4c00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:08.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.181+0000 7f4c5a000640 1 -- 172.21.15.152:0/897986590 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c4c017070 con 0x7f4c64156c00 2024-04-20T11:38:08.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.181+0000 7f4c5a000640 1 -- 172.21.15.152:0/897986590 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c4c00ca00 con 0x7f4c64156c00 2024-04-20T11:38:08.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.181+0000 7f4c5a000640 1 -- 172.21.15.152:0/897986590 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c4c00cdd0 con 0x7f4c64156c00 2024-04-20T11:38:08.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.181+0000 7f4c63200640 1 -- 172.21.15.152:0/897986590 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c6415e610 con 0x7f4c64156c00 2024-04-20T11:38:08.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.181+0000 7f4c63200640 1 -- 172.21.15.152:0/897986590 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c64150df0 con 0x7f4c64156c00 2024-04-20T11:38:08.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.182+0000 7f4c63200640 1 -- 172.21.15.152:0/897986590 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c64075e60 con 0x7f4c64156c00 2024-04-20T11:38:08.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.183+0000 7f4c5a000640 1 -- 172.21.15.152:0/897986590 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 27) ==== 92060+0+0 (secure 0 0 0) 0x7f4c4c01a070 con 0x7f4c64156c00 2024-04-20T11:38:08.184 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.184+0000 7f4c5a000640 1 --2- 172.21.15.152:0/897986590 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4c4006d430 0x7f4c4006f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:08.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.184+0000 7f4c5a000640 1 -- 172.21.15.152:0/897986590 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 5709+0+0 (secure 0 0 0) 0x7f4c4c08a600 con 0x7f4c64156c00 2024-04-20T11:38:08.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.185+0000 7f4c61e00640 1 --2- 172.21.15.152:0/897986590 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4c4006d430 0x7f4c4006f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:08.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.186+0000 7f4c61e00640 1 --2- 172.21.15.152:0/897986590 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4c4006d430 0x7f4c4006f8f0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f4c54000f60 tx=0x7f4c54007cc0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:08.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.188+0000 7f4c5a000640 1 -- 172.21.15.152:0/897986590 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f4c6414bbe0 con 0x7f4c64156c00 2024-04-20T11:38:08.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.390+0000 7f4c63200640 1 -- 172.21.15.152:0/897986590 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f4c6406f390 con 0x7f4c64156c00 2024-04-20T11:38:08.391 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.391+0000 7f4c5a000640 1 -- 172.21.15.152:0/897986590 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v167) ==== 100+0+11 (secure 0 0 0) 0x7f4c4c05a230 con 0x7f4c64156c00 2024-04-20T11:38:08.391 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 64 2024-04-20T11:38:08.394 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.393+0000 7f4c63200640 1 -- 172.21.15.152:0/897986590 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4c4006d430 msgr2=0x7f4c4006f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:08.394 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.393+0000 7f4c63200640 1 --2- 172.21.15.152:0/897986590 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4c4006d430 0x7f4c4006f8f0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f4c54000f60 tx=0x7f4c54007cc0 comp rx=0 tx=0).stop 2024-04-20T11:38:08.394 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.393+0000 7f4c63200640 1 -- 172.21.15.152:0/897986590 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c64156c00 msgr2=0x7f4c6414b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:08.395 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.393+0000 7f4c63200640 1 --2- 172.21.15.152:0/897986590 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c64156c00 0x7f4c6414b6a0 secure :-1 s=READY pgs=2379 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c00e960 tx=0x7f4c4c00ee20 comp rx=0 tx=0).stop 2024-04-20T11:38:08.395 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.393+0000 7f4c63200640 1 -- 172.21.15.152:0/897986590 shutdown_connections 2024-04-20T11:38:08.395 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.393+0000 7f4c63200640 1 --2- 172.21.15.152:0/897986590 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4c4006d430 0x7f4c4006f8f0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:08.395 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.393+0000 7f4c63200640 1 --2- 172.21.15.152:0/897986590 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4c6415ca40 0x7f4c6415e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:08.395 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.393+0000 7f4c63200640 1 --2- 172.21.15.152:0/897986590 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c64156c00 0x7f4c6414b6a0 unknown :-1 s=CLOSED pgs=2379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:08.395 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.393+0000 7f4c63200640 1 --2- 172.21.15.152:0/897986590 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4c6406bda0 0x7f4c641f5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:08.395 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.393+0000 7f4c63200640 1 -- 172.21.15.152:0/897986590 >> 172.21.15.152:0/897986590 conn(0x7f4c640b21e0 msgr2=0x7f4c641558b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:08.395 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.394+0000 7f4c63200640 1 -- 172.21.15.152:0/897986590 shutdown_connections 2024-04-20T11:38:08.395 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.394+0000 7f4c63200640 1 -- 172.21.15.152:0/897986590 wait complete. 2024-04-20T11:38:08.414 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-20T11:38:08.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.616+0000 7fbd9b600640 1 Processor -- start 2024-04-20T11:38:08.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.617+0000 7fbd9b600640 1 -- start start 2024-04-20T11:38:08.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.618+0000 7fbd9b600640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fbd940cd7e0 0x7fbd9406e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:08.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.618+0000 7fbd9b600640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd9406f110 con 0x7fbd9406e900 2024-04-20T11:38:08.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.618+0000 7fbd9b600640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd9406f3f0 con 0x7fbd94145f00 2024-04-20T11:38:08.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.618+0000 7fbd9b600640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fbd940cf9f0 con 0x7fbd940cd7e0 2024-04-20T11:38:08.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.619+0000 7fbd99600640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fbd9406e900 0x7fbd940cfe80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:34376/0 (socket says 172.21.15.152:34376) 2024-04-20T11:38:08.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.619+0000 7fbd99600640 1 -- 172.21.15.152:0/589354869 learned_addr learned my addr 172.21.15.152:0/589354869 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:38:08.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.619+0000 7fbd93400640 1 -- 172.21.15.152:0/589354869 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1370897959 0 0) 0x7fbd9406f110 con 0x7fbd9406e900 2024-04-20T11:38:08.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.619+0000 7fbd93400640 1 -- 172.21.15.152:0/589354869 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbd7c003520 con 0x7fbd9406e900 2024-04-20T11:38:08.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.619+0000 7fbd93e00640 1 --2- 172.21.15.152:0/589354869 >> v2:172.21.15.175:3300/0 conn(0x7fbd940cd7e0 0x7fbd9406e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:08.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.620+0000 7fbd93400640 1 -- 172.21.15.152:0/589354869 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 69522440 0 0) 0x7fbd7c003520 con 0x7fbd9406e900 2024-04-20T11:38:08.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.620+0000 7fbd93400640 1 -- 172.21.15.152:0/589354869 >> v2:172.21.15.175:3300/0 conn(0x7fbd940cd7e0 msgr2=0x7fbd9406e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:08.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.620+0000 7fbd93400640 1 --2- 172.21.15.152:0/589354869 >> v2:172.21.15.175:3300/0 conn(0x7fbd940cd7e0 0x7fbd9406e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:08.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.620+0000 7fbd93400640 1 -- 172.21.15.152:0/589354869 >> v1:172.21.15.175:6789/0 conn(0x7fbd94145f00 legacy=0x7fbd940cd0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:08.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.620+0000 7fbd93400640 1 -- 172.21.15.152:0/589354869 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd940cf3a0 con 0x7fbd9406e900 2024-04-20T11:38:08.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.620+0000 7fbd93400640 1 -- 172.21.15.152:0/589354869 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7fbd8c0019f0 con 0x7fbd9406e900 2024-04-20T11:38:08.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.620+0000 7fbd93400640 1 -- 172.21.15.152:0/589354869 >> v1:172.21.15.152:6789/0 conn(0x7fbd9406e900 legacy=0x7fbd940cfe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:08.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.620+0000 7fbd93400640 1 --2- 172.21.15.152:0/589354869 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd7c0039b0 0x7fbd7c003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:08.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.620+0000 7fbd93400640 1 --2- 172.21.15.152:0/589354869 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd7c004800 0x7fbd7c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:08.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.620+0000 7fbd93400640 1 --2- 172.21.15.152:0/589354869 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd7c005210 0x7fbd7c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:08.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.620+0000 7fbd93400640 1 -- 172.21.15.152:0/589354869 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fbd940cf9f0 con 0x7fbd7c004800 2024-04-20T11:38:08.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.620+0000 7fbd93400640 1 -- 172.21.15.152:0/589354869 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbd7c00c140 con 0x7fbd7c0039b0 2024-04-20T11:38:08.623 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.620+0000 7fbd93400640 1 -- 172.21.15.152:0/589354869 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fbd7c00c340 con 0x7fbd7c005210 2024-04-20T11:38:08.623 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.620+0000 7fbd93e00640 1 --2- 172.21.15.152:0/589354869 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd7c004800 0x7fbd7c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:08.623 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.620+0000 7fbd98c00640 1 --2- 172.21.15.152:0/589354869 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd7c0039b0 0x7fbd7c003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:08.623 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.620+0000 7fbd93e00640 1 -- 172.21.15.152:0/589354869 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd7c005210 msgr2=0x7fbd7c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:08.623 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.620+0000 7fbd93e00640 1 --2- 172.21.15.152:0/589354869 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd7c005210 0x7fbd7c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:08.623 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.621+0000 7fbd93e00640 1 -- 172.21.15.152:0/589354869 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd7c0039b0 msgr2=0x7fbd7c003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:08.623 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.621+0000 7fbd93e00640 1 --2- 172.21.15.152:0/589354869 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd7c0039b0 0x7fbd7c003df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:08.623 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.621+0000 7fbd93e00640 1 -- 172.21.15.152:0/589354869 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd7c003520 con 0x7fbd7c004800 2024-04-20T11:38:08.623 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.621+0000 7fbd99600640 1 -- 172.21.15.152:0/589354869 reap_dead start 2024-04-20T11:38:08.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.622+0000 7fbd93e00640 1 --2- 172.21.15.152:0/589354869 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd7c004800 0x7fbd7c004c80 secure :-1 s=READY pgs=2381 cs=0 l=1 rev1=1 crypto rx=0x7fbd8000ba30 tx=0x7fbd7c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbabb6d6be35fee8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:08.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.622+0000 7fbd93400640 1 -- 172.21.15.152:0/589354869 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd80036020 con 0x7fbd7c004800 2024-04-20T11:38:08.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.622+0000 7fbd93400640 1 -- 172.21.15.152:0/589354869 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd80032640 con 0x7fbd7c004800 2024-04-20T11:38:08.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.622+0000 7fbd93400640 1 -- 172.21.15.152:0/589354869 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd8000b600 con 0x7fbd7c004800 2024-04-20T11:38:08.625 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.622+0000 7fbd9b600640 1 -- 172.21.15.152:0/589354869 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd7c004800 msgr2=0x7fbd7c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:08.625 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.622+0000 7fbd9b600640 1 --2- 172.21.15.152:0/589354869 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd7c004800 0x7fbd7c004c80 secure :-1 s=READY pgs=2381 cs=0 l=1 rev1=1 crypto rx=0x7fbd8000ba30 tx=0x7fbd7c005830 comp rx=0 tx=0).stop 2024-04-20T11:38:08.625 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.623+0000 7fbd9b600640 1 -- 172.21.15.152:0/589354869 shutdown_connections 2024-04-20T11:38:08.625 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.623+0000 7fbd9b600640 1 --2- 172.21.15.152:0/589354869 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd7c004800 0x7fbd7c004c80 unknown :-1 s=CLOSED pgs=2381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:08.625 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.623+0000 7fbd9b600640 1 -- 172.21.15.152:0/589354869 >> 172.21.15.152:0/589354869 conn(0x7fbd94067cf0 msgr2=0x7fbd9406b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:08.625 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.623+0000 7fbd9b600640 1 -- 172.21.15.152:0/589354869 shutdown_connections 2024-04-20T11:38:08.625 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.623+0000 7fbd9b600640 1 -- 172.21.15.152:0/589354869 wait complete. 2024-04-20T11:38:08.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.624+0000 7fbd9b600640 1 Processor -- start 2024-04-20T11:38:08.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.625+0000 7fbd9b600640 1 -- start start 2024-04-20T11:38:08.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.625+0000 7fbd9b600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd7c004800 0x7fbd94153af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:08.627 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.626+0000 7fbd9b600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd94154030 0x7fbd9413c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:08.627 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.626+0000 7fbd9b600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd9413c9f0 0x7fbd94139b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:08.627 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.626+0000 7fbd9b600640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fbd9406ed50 con 0x7fbd9413c9f0 2024-04-20T11:38:08.627 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.626+0000 7fbd9b600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbd941462e0 con 0x7fbd7c004800 2024-04-20T11:38:08.627 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.626+0000 7fbd9b600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fbd9406eed0 con 0x7fbd94154030 2024-04-20T11:38:08.627 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.626+0000 7fbd99600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd9413c9f0 0x7fbd94139b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:08.627 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.626+0000 7fbd99600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd9413c9f0 0x7fbd94139b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:37346/0 (socket says 172.21.15.152:37346) 2024-04-20T11:38:08.627 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.626+0000 7fbd99600640 1 -- 172.21.15.152:0/261785524 learned_addr learned my addr 172.21.15.152:0/261785524 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:38:08.627 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.626+0000 7fbd99600640 1 -- 172.21.15.152:0/261785524 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd94154030 msgr2=0x7fbd9413c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:08.627 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.626+0000 7fbd99600640 1 --2- 172.21.15.152:0/261785524 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd94154030 0x7fbd9413c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:08.627 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.626+0000 7fbd99600640 1 -- 172.21.15.152:0/261785524 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd7c004800 msgr2=0x7fbd94153af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:08.627 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.626+0000 7fbd99600640 1 --2- 172.21.15.152:0/261785524 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd7c004800 0x7fbd94153af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:08.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.626+0000 7fbd99600640 1 -- 172.21.15.152:0/261785524 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd9413f0f0 con 0x7fbd9413c9f0 2024-04-20T11:38:08.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.627+0000 7fbd99600640 1 --2- 172.21.15.152:0/261785524 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd9413c9f0 0x7fbd94139b00 secure :-1 s=READY pgs=2382 cs=0 l=1 rev1=1 crypto rx=0x7fbd94071b70 tx=0x7fbd8c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:08.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.627+0000 7fbd90c00640 1 -- 172.21.15.152:0/261785524 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd8c00ce70 con 0x7fbd9413c9f0 2024-04-20T11:38:08.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.627+0000 7fbd90c00640 1 -- 172.21.15.152:0/261785524 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd8c00a640 con 0x7fbd9413c9f0 2024-04-20T11:38:08.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.627+0000 7fbd90c00640 1 -- 172.21.15.152:0/261785524 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd8c00aa30 con 0x7fbd9413c9f0 2024-04-20T11:38:08.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.627+0000 7fbd9b600640 1 -- 172.21.15.152:0/261785524 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd9413f360 con 0x7fbd9413c9f0 2024-04-20T11:38:08.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.627+0000 7fbd9b600640 1 -- 172.21.15.152:0/261785524 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd9413f920 con 0x7fbd9413c9f0 2024-04-20T11:38:08.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.628+0000 7fbd9b600640 1 -- 172.21.15.152:0/261785524 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd940689f0 con 0x7fbd9413c9f0 2024-04-20T11:38:08.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.629+0000 7fbd90c00640 1 -- 172.21.15.152:0/261785524 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 27) ==== 92060+0+0 (secure 0 0 0) 0x7fbd8c00abd0 con 0x7fbd9413c9f0 2024-04-20T11:38:08.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.630+0000 7fbd90c00640 1 --2- 172.21.15.152:0/261785524 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fbd7406d460 0x7fbd7406f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:08.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.630+0000 7fbd90c00640 1 -- 172.21.15.152:0/261785524 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 5709+0+0 (secure 0 0 0) 0x7fbd8c089c00 con 0x7fbd9413c9f0 2024-04-20T11:38:08.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.631+0000 7fbd98c00640 1 --2- 172.21.15.152:0/261785524 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fbd7406d460 0x7fbd7406f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:08.632 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.632+0000 7fbd98c00640 1 --2- 172.21.15.152:0/261785524 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fbd7406d460 0x7fbd7406f920 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fbd88000c00 tx=0x7fbd88009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:08.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.635+0000 7fbd90c00640 1 -- 172.21.15.152:0/261785524 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fbd8c01a030 con 0x7fbd9413c9f0 2024-04-20T11:38:08.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.851+0000 7fbd9b600640 1 -- 172.21.15.152:0/261785524 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fbd940b3d10 con 0x7fbd9413c9f0 2024-04-20T11:38:08.852 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.851+0000 7fbd90c00640 1 -- 172.21.15.152:0/261785524 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v167) ==== 80+0+0 (secure 0 0 0) 0x7fbd8c059800 con 0x7fbd9413c9f0 2024-04-20T11:38:08.852 INFO:teuthology.orchestra.run.smithi152.stderr:listed 0 entries 2024-04-20T11:38:08.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.854+0000 7fbd9b600640 1 -- 172.21.15.152:0/261785524 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fbd7406d460 msgr2=0x7fbd7406f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:08.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.854+0000 7fbd9b600640 1 --2- 172.21.15.152:0/261785524 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fbd7406d460 0x7fbd7406f920 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fbd88000c00 tx=0x7fbd88009040 comp rx=0 tx=0).stop 2024-04-20T11:38:08.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.854+0000 7fbd9b600640 1 -- 172.21.15.152:0/261785524 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd9413c9f0 msgr2=0x7fbd94139b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:08.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.854+0000 7fbd9b600640 1 --2- 172.21.15.152:0/261785524 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd9413c9f0 0x7fbd94139b00 secure :-1 s=READY pgs=2382 cs=0 l=1 rev1=1 crypto rx=0x7fbd94071b70 tx=0x7fbd8c00cc20 comp rx=0 tx=0).stop 2024-04-20T11:38:08.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.854+0000 7fbd9b600640 1 -- 172.21.15.152:0/261785524 shutdown_connections 2024-04-20T11:38:08.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.854+0000 7fbd9b600640 1 --2- 172.21.15.152:0/261785524 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fbd7406d460 0x7fbd7406f920 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:08.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.854+0000 7fbd9b600640 1 --2- 172.21.15.152:0/261785524 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd9413c9f0 0x7fbd94139b00 unknown :-1 s=CLOSED pgs=2382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:08.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.854+0000 7fbd9b600640 1 --2- 172.21.15.152:0/261785524 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd94154030 0x7fbd9413c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:08.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.854+0000 7fbd9b600640 1 --2- 172.21.15.152:0/261785524 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd7c004800 0x7fbd94153af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:08.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.854+0000 7fbd9b600640 1 -- 172.21.15.152:0/261785524 >> 172.21.15.152:0/261785524 conn(0x7fbd94067cf0 msgr2=0x7fbd940b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:08.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.854+0000 7fbd9b600640 1 -- 172.21.15.152:0/261785524 shutdown_connections 2024-04-20T11:38:08.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:08.855+0000 7fbd9b600640 1 -- 172.21.15.152:0/261785524 wait complete. 2024-04-20T11:38:08.875 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi175.front.sepia.ceph.com... 2024-04-20T11:38:08.875 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:38:08.875 DEBUG:teuthology.orchestra.run.smithi175:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-04-20T11:38:08.961 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi175.stderr:ceph-fuse[37216]: fuse finished with error 0 and tester_r 0 2024-04-20T11:38:08.966 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi175.front.sepia.ceph.com 2024-04-20T11:38:08.966 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:38:08.966 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-20T11:38:09.034 INFO:tasks.cephfs.mount:Terminating background process 2024-04-20T11:38:09.034 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-20T11:38:09.034 DEBUG:teuthology.orchestra.run.smithi152:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:38:09.091 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-20T11:38:09.091 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi152.front.sepia.ceph.com 2024-04-20T11:38:09.091 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:38:09.091 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-20T11:38:09.140 INFO:teuthology.orchestra.run.smithi152.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-20T11:38:09.141 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T11:38:09.141 INFO:tasks.cephfs.mount:Terminating background process 2024-04-20T11:38:09.142 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T11:38:09.142 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:38:09.201 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-20T11:38:09.201 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi175.front.sepia.ceph.com 2024-04-20T11:38:09.201 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:38:09.201 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-20T11:38:09.253 INFO:teuthology.orchestra.run.smithi175.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-20T11:38:09.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T11:38:09.254 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:38:09.465 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.464+0000 7f33b9e00640 1 Processor -- start 2024-04-20T11:38:09.465 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.465+0000 7f33b9e00640 1 -- start start 2024-04-20T11:38:09.467 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.466+0000 7f33b9e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f33b40723e0 0x7f33b40afa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:09.467 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.466+0000 7f33b9e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33b406e010 0x7f33b406e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:09.467 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.466+0000 7f33b9e00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33b40aec90 con 0x7f33b4147410 2024-04-20T11:38:09.467 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.466+0000 7f33b9e00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f33b40aef70 con 0x7f33b40723e0 2024-04-20T11:38:09.467 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.466+0000 7f33b9e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f33b40af250 con 0x7f33b406e010 2024-04-20T11:38:09.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.467+0000 7f33b9400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33b406e010 0x7f33b406e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:09.473 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.467+0000 7f33b9400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33b406e010 0x7f33b406e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.152:49700/0 (socket says 172.21.15.152:49700) 2024-04-20T11:38:09.473 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.467+0000 7f33b9400640 1 -- 172.21.15.152:0/3106587167 learned_addr learned my addr 172.21.15.152:0/3106587167 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:38:09.473 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.467+0000 7f33b3e00640 1 --2- 172.21.15.152:0/3106587167 >> v2:172.21.15.175:3300/0 conn(0x7f33b40723e0 0x7f33b40afa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:09.473 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.468+0000 7f33b9400640 1 -- 172.21.15.152:0/3106587167 >> v2:172.21.15.175:3300/0 conn(0x7f33b40723e0 msgr2=0x7f33b40afa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:09.473 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.468+0000 7f33b9400640 1 --2- 172.21.15.152:0/3106587167 >> v2:172.21.15.175:3300/0 conn(0x7f33b40723e0 0x7f33b40afa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:09.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.468+0000 7f33b9400640 1 -- 172.21.15.152:0/3106587167 >> v1:172.21.15.175:6789/0 conn(0x7f33b4147410 legacy=0x7f33b4071cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:09.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.468+0000 7f33b9400640 1 -- 172.21.15.152:0/3106587167 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33b40af530 con 0x7f33b406e010 2024-04-20T11:38:09.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.469+0000 7f33b9400640 1 --2- 172.21.15.152:0/3106587167 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33b406e010 0x7f33b406e4a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f33ac009e50 tx=0x7f33ac02bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a49a8810a857760 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:09.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.470+0000 7f33b3400640 1 -- 172.21.15.152:0/3106587167 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33ac02ca70 con 0x7f33b406e010 2024-04-20T11:38:09.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.470+0000 7f33b3400640 1 -- 172.21.15.152:0/3106587167 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33ac02cc10 con 0x7f33b406e010 2024-04-20T11:38:09.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.470+0000 7f33b3400640 1 -- 172.21.15.152:0/3106587167 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33ac0343f0 con 0x7f33b406e010 2024-04-20T11:38:09.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.470+0000 7f33b9e00640 1 -- 172.21.15.152:0/3106587167 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33b406e010 msgr2=0x7f33b406e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:09.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.470+0000 7f33b9e00640 1 --2- 172.21.15.152:0/3106587167 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33b406e010 0x7f33b406e4a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f33ac009e50 tx=0x7f33ac02bed0 comp rx=0 tx=0).stop 2024-04-20T11:38:09.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.471+0000 7f33b9e00640 1 -- 172.21.15.152:0/3106587167 shutdown_connections 2024-04-20T11:38:09.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.471+0000 7f33b9e00640 1 --2- 172.21.15.152:0/3106587167 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33b406e010 0x7f33b406e4a0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:09.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.471+0000 7f33b9e00640 1 --2- 172.21.15.152:0/3106587167 >> v2:172.21.15.175:3300/0 conn(0x7f33b40723e0 0x7f33b40afa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:09.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.471+0000 7f33b9e00640 1 -- 172.21.15.152:0/3106587167 >> 172.21.15.152:0/3106587167 conn(0x7f33b4067ae0 msgr2=0x7f33b406b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:09.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.471+0000 7f33b9e00640 1 -- 172.21.15.152:0/3106587167 shutdown_connections 2024-04-20T11:38:09.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.471+0000 7f33b9e00640 1 -- 172.21.15.152:0/3106587167 wait complete. 2024-04-20T11:38:09.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.473+0000 7f33b9e00640 1 Processor -- start 2024-04-20T11:38:09.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.473+0000 7f33b9e00640 1 -- start start 2024-04-20T11:38:09.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.474+0000 7f33b9e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33b406e010 0x7f33b413ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:09.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.474+0000 7f33b9e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33b40723e0 0x7f33b40b0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:09.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.474+0000 7f33b9e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f33b4147410 0x7f33b40b10d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:09.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.474+0000 7f33b9e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f33b406e9e0 con 0x7f33b4147410 2024-04-20T11:38:09.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.474+0000 7f33b9e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f33b40b3390 con 0x7f33b40723e0 2024-04-20T11:38:09.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.474+0000 7f33b9e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f33b40af8a0 con 0x7f33b406e010 2024-04-20T11:38:09.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.475+0000 7f33b9400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f33b4147410 0x7f33b40b10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:09.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.475+0000 7f33b9400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f33b4147410 0x7f33b40b10d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:37356/0 (socket says 172.21.15.152:37356) 2024-04-20T11:38:09.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.475+0000 7f33b9400640 1 -- 172.21.15.152:0/1485923980 learned_addr learned my addr 172.21.15.152:0/1485923980 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:38:09.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.475+0000 7f33b3e00640 1 --2- 172.21.15.152:0/1485923980 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33b40723e0 0x7f33b40b0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:09.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.475+0000 7f33b8a00640 1 --2- 172.21.15.152:0/1485923980 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33b406e010 0x7f33b413ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:09.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.475+0000 7f33b9400640 1 -- 172.21.15.152:0/1485923980 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33b406e010 msgr2=0x7f33b413ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:09.480 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.475+0000 7f33b9400640 1 --2- 172.21.15.152:0/1485923980 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33b406e010 0x7f33b413ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:09.480 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.475+0000 7f33b9400640 1 -- 172.21.15.152:0/1485923980 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33b40723e0 msgr2=0x7f33b40b0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:09.480 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.475+0000 7f33b9400640 1 --2- 172.21.15.152:0/1485923980 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33b40723e0 0x7f33b40b0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:09.480 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.475+0000 7f33b9400640 1 -- 172.21.15.152:0/1485923980 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33b40afaf0 con 0x7f33b4147410 2024-04-20T11:38:09.480 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.476+0000 7f33b9400640 1 --2- 172.21.15.152:0/1485923980 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f33b4147410 0x7f33b40b10d0 secure :-1 s=READY pgs=2383 cs=0 l=1 rev1=1 crypto rx=0x7f33ac036040 tx=0x7f33ac004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:09.480 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.476+0000 7f33b0c00640 1 -- 172.21.15.152:0/1485923980 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33ac043020 con 0x7f33b4147410 2024-04-20T11:38:09.480 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.476+0000 7f33b0c00640 1 -- 172.21.15.152:0/1485923980 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33ac03e450 con 0x7f33b4147410 2024-04-20T11:38:09.480 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.476+0000 7f33b9e00640 1 -- 172.21.15.152:0/1485923980 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33b40afdd0 con 0x7f33b4147410 2024-04-20T11:38:09.480 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.476+0000 7f33b0c00640 1 -- 172.21.15.152:0/1485923980 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33ac009510 con 0x7f33b4147410 2024-04-20T11:38:09.480 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.476+0000 7f33b9e00640 1 -- 172.21.15.152:0/1485923980 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33b40b0390 con 0x7f33b4147410 2024-04-20T11:38:09.480 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.477+0000 7f33b9e00640 1 -- 172.21.15.152:0/1485923980 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33b4075e60 con 0x7f33b4147410 2024-04-20T11:38:09.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.479+0000 7f33b0c00640 1 -- 172.21.15.152:0/1485923980 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 27) ==== 92060+0+0 (secure 0 0 0) 0x7f33ac005d10 con 0x7f33b4147410 2024-04-20T11:38:09.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.480+0000 7f33b0c00640 1 --2- 172.21.15.152:0/1485923980 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f339006d430 0x7f339006f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:09.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.480+0000 7f33b0c00640 1 -- 172.21.15.152:0/1485923980 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 5709+0+0 (secure 0 0 0) 0x7f33ac0acfa0 con 0x7f33b4147410 2024-04-20T11:38:09.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.480+0000 7f33b8a00640 1 --2- 172.21.15.152:0/1485923980 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f339006d430 0x7f339006f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:09.482 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.482+0000 7f33b8a00640 1 --2- 172.21.15.152:0/1485923980 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f339006d430 0x7f339006f8f0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f33a4006fd0 tx=0x7f33a4008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:09.485 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.485+0000 7f33b0c00640 1 -- 172.21.15.152:0/1485923980 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f33ac07ce20 con 0x7f33b4147410 2024-04-20T11:38:09.723 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.722+0000 7f33b9e00640 1 -- 172.21.15.152:0/1485923980 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f33b406ccb0 con 0x7f33b4147410 2024-04-20T11:38:09.724 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.723+0000 7f33b0c00640 1 -- 172.21.15.152:0/1485923980 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 116 v116) ==== 95+0+5215 (secure 0 0 0) 0x7f33ac07c720 con 0x7f33b4147410 2024-04-20T11:38:09.724 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:38:09.724 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":116,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6605,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6834/4020670267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6833","nonce":4020670267},{"type":"v1","addr":"172.21.15.152:6834","nonce":4020670267}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":109},{"gid":6614,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/352716681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":352716681},{"type":"v1","addr":"172.21.15.175:6833","nonce":352716681}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":110},{"gid":6617,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/2512108447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":2512108447},{"type":"v1","addr":"172.21.15.175:6835","nonce":2512108447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":110}],"filesystems":[{"mdsmap":{"epoch":115,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T11:37:47.506735+0000","modified":"2024-04-20T11:37:50.605372+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7339},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7339":{"gid":7339,"name":"c","rank":0,"incarnation":111,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6835/951802292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":951802292},{"type":"v1","addr":"172.21.15.152:6835","nonce":951802292}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7339,"qdb_cluster":[7339]},"id":13}]} 2024-04-20T11:38:09.724 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 116 2024-04-20T11:38:09.726 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.726+0000 7f33b9e00640 1 -- 172.21.15.152:0/1485923980 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f339006d430 msgr2=0x7f339006f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:09.727 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.726+0000 7f33b9e00640 1 --2- 172.21.15.152:0/1485923980 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f339006d430 0x7f339006f8f0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f33a4006fd0 tx=0x7f33a4008040 comp rx=0 tx=0).stop 2024-04-20T11:38:09.727 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.726+0000 7f33b9e00640 1 -- 172.21.15.152:0/1485923980 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f33b4147410 msgr2=0x7f33b40b10d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:09.727 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.726+0000 7f33b9e00640 1 --2- 172.21.15.152:0/1485923980 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f33b4147410 0x7f33b40b10d0 secure :-1 s=READY pgs=2383 cs=0 l=1 rev1=1 crypto rx=0x7f33ac036040 tx=0x7f33ac004a40 comp rx=0 tx=0).stop 2024-04-20T11:38:09.727 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.726+0000 7f33b9e00640 1 -- 172.21.15.152:0/1485923980 shutdown_connections 2024-04-20T11:38:09.727 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.726+0000 7f33b9e00640 1 --2- 172.21.15.152:0/1485923980 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f339006d430 0x7f339006f8f0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:09.727 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.726+0000 7f33b9e00640 1 --2- 172.21.15.152:0/1485923980 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f33b4147410 0x7f33b40b10d0 unknown :-1 s=CLOSED pgs=2383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:09.727 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.726+0000 7f33b9e00640 1 --2- 172.21.15.152:0/1485923980 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33b40723e0 0x7f33b40b0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:09.727 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.726+0000 7f33b9e00640 1 --2- 172.21.15.152:0/1485923980 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33b406e010 0x7f33b413ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:09.727 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.726+0000 7f33b9e00640 1 -- 172.21.15.152:0/1485923980 >> 172.21.15.152:0/1485923980 conn(0x7f33b4067ae0 msgr2=0x7f33b406b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:09.728 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.726+0000 7f33b9e00640 1 -- 172.21.15.152:0/1485923980 shutdown_connections 2024-04-20T11:38:09.728 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.726+0000 7f33b9e00640 1 -- 172.21.15.152:0/1485923980 wait complete. 2024-04-20T11:38:09.745 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:38:09.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.946+0000 7fe8c5c00640 1 Processor -- start 2024-04-20T11:38:09.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.946+0000 7fe8c5c00640 1 -- start start 2024-04-20T11:38:09.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.948+0000 7fe8c5c00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fe8c013e530 0x7fe8c006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:09.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.948+0000 7fe8c5c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8c01414e0 0x7fe8c0071cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:09.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.948+0000 7fe8c5c00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8c00cf0a0 con 0x7fe8c006f340 2024-04-20T11:38:09.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.948+0000 7fe8c5c00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fe8c0141b00 con 0x7fe8c013e530 2024-04-20T11:38:09.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.948+0000 7fe8c5c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe8c00cfc50 con 0x7fe8c01414e0 2024-04-20T11:38:09.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.948+0000 7fe8bf400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fe8c006f340 0x7fe8c0140dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:34390/0 (socket says 172.21.15.152:34390) 2024-04-20T11:38:09.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.948+0000 7fe8bf400640 1 -- 172.21.15.152:0/3064888815 learned_addr learned my addr 172.21.15.152:0/3064888815 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:38:09.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.948+0000 7fe8bfe00640 1 --2- 172.21.15.152:0/3064888815 >> v2:172.21.15.152:3300/0 conn(0x7fe8c013e530 0x7fe8c006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:09.950 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.949+0000 7fe8c5200640 1 --2- 172.21.15.152:0/3064888815 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8c01414e0 0x7fe8c0071cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:09.950 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.949+0000 7fe8bfe00640 1 -- 172.21.15.152:0/3064888815 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8c01414e0 msgr2=0x7fe8c0071cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:09.950 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.949+0000 7fe8bfe00640 1 --2- 172.21.15.152:0/3064888815 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8c01414e0 0x7fe8c0071cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:09.950 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.949+0000 7fe8bea00640 1 -- 172.21.15.152:0/3064888815 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1846440245 0 0) 0x7fe8ac001110 con 0x7fe8c006f340 2024-04-20T11:38:09.950 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.949+0000 7fe8bfe00640 1 -- 172.21.15.152:0/3064888815 >> v1:172.21.15.152:6789/0 conn(0x7fe8c006f340 legacy=0x7fe8c0140dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:09.950 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.949+0000 7fe8bfe00640 1 -- 172.21.15.152:0/3064888815 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8c00d00e0 con 0x7fe8c013e530 2024-04-20T11:38:09.951 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.950+0000 7fe8bfe00640 1 --2- 172.21.15.152:0/3064888815 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe8c013e530 0x7fe8c006ee00 secure :-1 s=READY pgs=2385 cs=0 l=1 rev1=1 crypto rx=0x7fe8b00097a0 tx=0x7fe8b002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=124d44b2d47dc92b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:09.951 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.951+0000 7fe8bea00640 1 -- 172.21.15.152:0/3064888815 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b0035020 con 0x7fe8c013e530 2024-04-20T11:38:09.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.951+0000 7fe8bea00640 1 -- 172.21.15.152:0/3064888815 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8b0031830 con 0x7fe8c013e530 2024-04-20T11:38:09.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.951+0000 7fe8bea00640 1 -- 172.21.15.152:0/3064888815 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b0031c00 con 0x7fe8c013e530 2024-04-20T11:38:09.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.952+0000 7fe8c5c00640 1 -- 172.21.15.152:0/3064888815 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe8c013e530 msgr2=0x7fe8c006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:09.953 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.952+0000 7fe8c5c00640 1 --2- 172.21.15.152:0/3064888815 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe8c013e530 0x7fe8c006ee00 secure :-1 s=READY pgs=2385 cs=0 l=1 rev1=1 crypto rx=0x7fe8b00097a0 tx=0x7fe8b002b8d0 comp rx=0 tx=0).stop 2024-04-20T11:38:09.953 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.952+0000 7fe8c5c00640 1 -- 172.21.15.152:0/3064888815 shutdown_connections 2024-04-20T11:38:09.953 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.952+0000 7fe8c5c00640 1 --2- 172.21.15.152:0/3064888815 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8c01414e0 0x7fe8c0071cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:09.953 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.952+0000 7fe8c5c00640 1 --2- 172.21.15.152:0/3064888815 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe8c013e530 0x7fe8c006ee00 unknown :-1 s=CLOSED pgs=2385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:09.953 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.952+0000 7fe8c5c00640 1 -- 172.21.15.152:0/3064888815 >> 172.21.15.152:0/3064888815 conn(0x7fe8c0067ae0 msgr2=0x7fe8c006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:09.953 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.952+0000 7fe8c5c00640 1 -- 172.21.15.152:0/3064888815 shutdown_connections 2024-04-20T11:38:09.953 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.952+0000 7fe8c5c00640 1 -- 172.21.15.152:0/3064888815 wait complete. 2024-04-20T11:38:09.954 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.954+0000 7fe8c5c00640 1 Processor -- start 2024-04-20T11:38:09.954 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.954+0000 7fe8c5c00640 1 -- start start 2024-04-20T11:38:09.957 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.955+0000 7fe8c5c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8c006f340 0x7fe8c0151ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:09.957 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.955+0000 7fe8c5c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8c013e530 0x7fe8c01e8160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:09.957 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.955+0000 7fe8c5c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe8c01414e0 0x7fe8c013d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:09.957 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.955+0000 7fe8c5c00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c00cf8e0 con 0x7fe8c01414e0 2024-04-20T11:38:09.957 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.955+0000 7fe8c5c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c00cf600 con 0x7fe8c006f340 2024-04-20T11:38:09.957 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.955+0000 7fe8c5c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe8c0072210 con 0x7fe8c013e530 2024-04-20T11:38:09.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.955+0000 7fe8c5200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe8c01414e0 0x7fe8c013d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:09.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.955+0000 7fe8c5200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe8c01414e0 0x7fe8c013d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:37380/0 (socket says 172.21.15.152:37380) 2024-04-20T11:38:09.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.955+0000 7fe8c5200640 1 -- 172.21.15.152:0/35211558 learned_addr learned my addr 172.21.15.152:0/35211558 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:38:09.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.956+0000 7fe8bf400640 1 --2- 172.21.15.152:0/35211558 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8c013e530 0x7fe8c01e8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:09.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.956+0000 7fe8bfe00640 1 --2- 172.21.15.152:0/35211558 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8c006f340 0x7fe8c0151ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:09.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.956+0000 7fe8c5200640 1 -- 172.21.15.152:0/35211558 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8c013e530 msgr2=0x7fe8c01e8160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:09.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.956+0000 7fe8c5200640 1 --2- 172.21.15.152:0/35211558 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8c013e530 0x7fe8c01e8160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:09.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.956+0000 7fe8c5200640 1 -- 172.21.15.152:0/35211558 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8c006f340 msgr2=0x7fe8c0151ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:09.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.956+0000 7fe8c5200640 1 --2- 172.21.15.152:0/35211558 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8c006f340 0x7fe8c0151ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:09.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.956+0000 7fe8c5200640 1 -- 172.21.15.152:0/35211558 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8c00b4610 con 0x7fe8c01414e0 2024-04-20T11:38:09.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.956+0000 7fe8bf400640 1 --2- 172.21.15.152:0/35211558 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8c013e530 0x7fe8c01e8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:38:09.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.956+0000 7fe8bfe00640 1 --2- 172.21.15.152:0/35211558 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8c006f340 0x7fe8c0151ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:38:09.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.956+0000 7fe8c5200640 1 --2- 172.21.15.152:0/35211558 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe8c01414e0 0x7fe8c013d770 secure :-1 s=READY pgs=2386 cs=0 l=1 rev1=1 crypto rx=0x7fe8b8007e90 tx=0x7fe8b800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:09.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.956+0000 7fe8b7e00640 1 -- 172.21.15.152:0/35211558 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b8014420 con 0x7fe8c01414e0 2024-04-20T11:38:09.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.956+0000 7fe8b7e00640 1 -- 172.21.15.152:0/35211558 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8b8019070 con 0x7fe8c01414e0 2024-04-20T11:38:09.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.956+0000 7fe8b7e00640 1 -- 172.21.15.152:0/35211558 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b8014800 con 0x7fe8c01414e0 2024-04-20T11:38:09.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.957+0000 7fe8c5c00640 1 -- 172.21.15.152:0/35211558 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8c0151ff0 con 0x7fe8c01414e0 2024-04-20T11:38:09.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.957+0000 7fe8c5c00640 1 -- 172.21.15.152:0/35211558 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8c0150fd0 con 0x7fe8c01414e0 2024-04-20T11:38:09.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.958+0000 7fe8c5c00640 1 -- 172.21.15.152:0/35211558 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8c0075e60 con 0x7fe8c01414e0 2024-04-20T11:38:09.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.959+0000 7fe8b7e00640 1 -- 172.21.15.152:0/35211558 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 27) ==== 92060+0+0 (secure 0 0 0) 0x7fe8b80149a0 con 0x7fe8c01414e0 2024-04-20T11:38:09.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.960+0000 7fe8b7e00640 1 --2- 172.21.15.152:0/35211558 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe8a006d430 0x7fe8a006f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:09.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.960+0000 7fe8b7e00640 1 -- 172.21.15.152:0/35211558 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 5709+0+0 (secure 0 0 0) 0x7fe8b808ad50 con 0x7fe8c01414e0 2024-04-20T11:38:09.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.961+0000 7fe8bfe00640 1 --2- 172.21.15.152:0/35211558 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe8a006d430 0x7fe8a006f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:09.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.962+0000 7fe8bfe00640 1 --2- 172.21.15.152:0/35211558 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe8a006d430 0x7fe8a006f8f0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fe8b0036000 tx=0x7fe8b0008030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:09.965 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:09.964+0000 7fe8b7e00640 1 -- 172.21.15.152:0/35211558 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe8b805abd0 con 0x7fe8c01414e0 2024-04-20T11:38:10.172 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.171+0000 7fe8c5c00640 1 -- 172.21.15.152:0/35211558 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe8c006e450 con 0x7fe8c01414e0 2024-04-20T11:38:10.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.174+0000 7fe8b7e00640 1 -- 172.21.15.152:0/35211558 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v167) ==== 74+0+16743 (secure 0 0 0) 0x7fe8b805a4d0 con 0x7fe8c01414e0 2024-04-20T11:38:10.175 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:38:10.176 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":167,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:37:49.091768+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:37:45.247938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"166","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2024-04-20T11:37:46.503171+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"167","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:38:10.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.177+0000 7fe8c5c00640 1 -- 172.21.15.152:0/35211558 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe8a006d430 msgr2=0x7fe8a006f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:10.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.177+0000 7fe8c5c00640 1 --2- 172.21.15.152:0/35211558 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe8a006d430 0x7fe8a006f8f0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fe8b0036000 tx=0x7fe8b0008030 comp rx=0 tx=0).stop 2024-04-20T11:38:10.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.177+0000 7fe8c5c00640 1 -- 172.21.15.152:0/35211558 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe8c01414e0 msgr2=0x7fe8c013d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:10.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.177+0000 7fe8c5c00640 1 --2- 172.21.15.152:0/35211558 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe8c01414e0 0x7fe8c013d770 secure :-1 s=READY pgs=2386 cs=0 l=1 rev1=1 crypto rx=0x7fe8b8007e90 tx=0x7fe8b800e4b0 comp rx=0 tx=0).stop 2024-04-20T11:38:10.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.178+0000 7fe8c5c00640 1 -- 172.21.15.152:0/35211558 shutdown_connections 2024-04-20T11:38:10.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.178+0000 7fe8c5c00640 1 --2- 172.21.15.152:0/35211558 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe8a006d430 0x7fe8a006f8f0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:10.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.178+0000 7fe8c5c00640 1 --2- 172.21.15.152:0/35211558 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe8c01414e0 0x7fe8c013d770 unknown :-1 s=CLOSED pgs=2386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:10.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.178+0000 7fe8c5c00640 1 --2- 172.21.15.152:0/35211558 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8c013e530 0x7fe8c01e8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:10.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.178+0000 7fe8c5c00640 1 --2- 172.21.15.152:0/35211558 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8c006f340 0x7fe8c0151ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:10.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.178+0000 7fe8c5c00640 1 -- 172.21.15.152:0/35211558 >> 172.21.15.152:0/35211558 conn(0x7fe8c0067ae0 msgr2=0x7fe8c015c7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:10.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.178+0000 7fe8c5c00640 1 -- 172.21.15.152:0/35211558 shutdown_connections 2024-04-20T11:38:10.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.178+0000 7fe8c5c00640 1 -- 172.21.15.152:0/35211558 wait complete. 2024-04-20T11:38:10.197 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T11:26:47.246571+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 26, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T11:37:45.247938+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '166', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 27, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T11:37:46.503171+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '167', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T11:38:10.198 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T11:38:10.402 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.401+0000 7f74af600640 1 Processor -- start 2024-04-20T11:38:10.402 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.401+0000 7f74af600640 1 -- start start 2024-04-20T11:38:10.404 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.403+0000 7f74af600640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f74a8146000 0x7f74a80b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:10.404 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.403+0000 7f74af600640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f74a80b2170 0x7f74a80b2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:10.404 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.403+0000 7f74af600640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74a81463e0 con 0x7f74a80cd1b0 2024-04-20T11:38:10.404 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.403+0000 7f74af600640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f74a80cdcc0 con 0x7f74a8146000 2024-04-20T11:38:10.404 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.403+0000 7f74af600640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f74a80b4750 con 0x7f74a80b2170 2024-04-20T11:38:10.404 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.403+0000 7f74acc00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f74a8146000 0x7f74a80b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:10.404 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.403+0000 7f74ad600640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f74a80cd1b0 0x7f74a80cd5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:34398/0 (socket says 172.21.15.152:34398) 2024-04-20T11:38:10.404 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.403+0000 7f74ad600640 1 -- 172.21.15.152:0/1560612046 learned_addr learned my addr 172.21.15.152:0/1560612046 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:38:10.404 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.403+0000 7f74a7e00640 1 --2- 172.21.15.152:0/1560612046 >> v2:172.21.15.175:3300/0 conn(0x7f74a80b2170 0x7f74a80b2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:10.405 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.404+0000 7f74a7400640 1 -- 172.21.15.152:0/1560612046 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2984260804 0 0) 0x7f74a81463e0 con 0x7f74a80cd1b0 2024-04-20T11:38:10.405 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.404+0000 7f74acc00640 1 -- 172.21.15.152:0/1560612046 >> v2:172.21.15.175:3300/0 conn(0x7f74a80b2170 msgr2=0x7f74a80b2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:10.405 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.404+0000 7f74acc00640 1 --2- 172.21.15.152:0/1560612046 >> v2:172.21.15.175:3300/0 conn(0x7f74a80b2170 0x7f74a80b2620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:10.405 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.404+0000 7f74acc00640 1 -- 172.21.15.152:0/1560612046 >> v1:172.21.15.152:6789/0 conn(0x7f74a80cd1b0 legacy=0x7f74a80cd5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:10.405 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.404+0000 7f74acc00640 1 -- 172.21.15.152:0/1560612046 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74a80b3e20 con 0x7f74a8146000 2024-04-20T11:38:10.406 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.405+0000 7f74acc00640 1 --2- 172.21.15.152:0/1560612046 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f74a8146000 0x7f74a80b4be0 secure :-1 s=READY pgs=2388 cs=0 l=1 rev1=1 crypto rx=0x7f74a80b3280 tx=0x7f749802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52f9430c095de12d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:10.406 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.405+0000 7f74a7400640 1 -- 172.21.15.152:0/1560612046 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7498039070 con 0x7f74a8146000 2024-04-20T11:38:10.406 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.405+0000 7f74a7400640 1 -- 172.21.15.152:0/1560612046 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f749802cbf0 con 0x7f74a8146000 2024-04-20T11:38:10.406 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.405+0000 7f74a7400640 1 -- 172.21.15.152:0/1560612046 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7498034440 con 0x7f74a8146000 2024-04-20T11:38:10.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.406+0000 7f74af600640 1 -- 172.21.15.152:0/1560612046 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f74a8146000 msgr2=0x7f74a80b4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:10.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.406+0000 7f74af600640 1 --2- 172.21.15.152:0/1560612046 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f74a8146000 0x7f74a80b4be0 secure :-1 s=READY pgs=2388 cs=0 l=1 rev1=1 crypto rx=0x7f74a80b3280 tx=0x7f749802be40 comp rx=0 tx=0).stop 2024-04-20T11:38:10.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.406+0000 7f74af600640 1 -- 172.21.15.152:0/1560612046 shutdown_connections 2024-04-20T11:38:10.408 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.406+0000 7f74af600640 1 --2- 172.21.15.152:0/1560612046 >> v2:172.21.15.175:3300/0 conn(0x7f74a80b2170 0x7f74a80b2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:10.408 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.406+0000 7f74af600640 1 --2- 172.21.15.152:0/1560612046 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f74a8146000 0x7f74a80b4be0 unknown :-1 s=CLOSED pgs=2388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:10.408 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.406+0000 7f74af600640 1 -- 172.21.15.152:0/1560612046 >> 172.21.15.152:0/1560612046 conn(0x7f74a806b980 msgr2=0x7f74a806bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:10.408 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.406+0000 7f74af600640 1 -- 172.21.15.152:0/1560612046 shutdown_connections 2024-04-20T11:38:10.408 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.406+0000 7f74af600640 1 -- 172.21.15.152:0/1560612046 wait complete. 2024-04-20T11:38:10.408 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.408+0000 7f74af600640 1 Processor -- start 2024-04-20T11:38:10.409 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.408+0000 7f74af600640 1 -- start start 2024-04-20T11:38:10.410 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.409+0000 7f74af600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f74a80b2170 0x7f74a80c71d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:10.411 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.409+0000 7f74af600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74a80cd1b0 0x7f74a813f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:10.411 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.409+0000 7f74af600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74a8146000 0x7f74a813fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:10.411 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.409+0000 7f74af600640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f74a815ccf0 con 0x7f74a80b2170 2024-04-20T11:38:10.411 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.409+0000 7f74af600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f74a80b43e0 con 0x7f74a80cd1b0 2024-04-20T11:38:10.411 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.409+0000 7f74af600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f74a815ce70 con 0x7f74a8146000 2024-04-20T11:38:10.411 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.409+0000 7f74acc00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f74a80b2170 0x7f74a80c71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:10.411 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.409+0000 7f74acc00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f74a80b2170 0x7f74a80c71d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:37406/0 (socket says 172.21.15.152:37406) 2024-04-20T11:38:10.411 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.410+0000 7f74acc00640 1 -- 172.21.15.152:0/3280542137 learned_addr learned my addr 172.21.15.152:0/3280542137 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:38:10.411 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.410+0000 7f74acc00640 1 -- 172.21.15.152:0/3280542137 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74a8146000 msgr2=0x7f74a813fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:10.411 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.410+0000 7f74a7e00640 1 --2- 172.21.15.152:0/3280542137 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74a80cd1b0 0x7f74a813f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:10.411 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.410+0000 7f74acc00640 1 --2- 172.21.15.152:0/3280542137 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74a8146000 0x7f74a813fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:10.411 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.410+0000 7f74acc00640 1 -- 172.21.15.152:0/3280542137 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74a80cd1b0 msgr2=0x7f74a813f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:10.412 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.410+0000 7f74acc00640 1 --2- 172.21.15.152:0/3280542137 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74a80cd1b0 0x7f74a813f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:10.412 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.410+0000 7f74acc00640 1 -- 172.21.15.152:0/3280542137 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74a8140490 con 0x7f74a80b2170 2024-04-20T11:38:10.412 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.410+0000 7f74acc00640 1 --2- 172.21.15.152:0/3280542137 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f74a80b2170 0x7f74a80c71d0 secure :-1 s=READY pgs=2389 cs=0 l=1 rev1=1 crypto rx=0x7f7498037040 tx=0x7f7498004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:10.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.411+0000 7f74a4c00640 1 -- 172.21.15.152:0/3280542137 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7498039050 con 0x7f74a80b2170 2024-04-20T11:38:10.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.411+0000 7f74a4c00640 1 -- 172.21.15.152:0/3280542137 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f749803a040 con 0x7f74a80b2170 2024-04-20T11:38:10.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.411+0000 7f74af600640 1 -- 172.21.15.152:0/3280542137 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74a8140770 con 0x7f74a80b2170 2024-04-20T11:38:10.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.411+0000 7f74a4c00640 1 -- 172.21.15.152:0/3280542137 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f749803e610 con 0x7f74a80b2170 2024-04-20T11:38:10.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.411+0000 7f74af600640 1 -- 172.21.15.152:0/3280542137 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74a8140d30 con 0x7f74a80b2170 2024-04-20T11:38:10.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.412+0000 7f74af600640 1 -- 172.21.15.152:0/3280542137 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74a80689f0 con 0x7f74a80b2170 2024-04-20T11:38:10.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.413+0000 7f74a4c00640 1 -- 172.21.15.152:0/3280542137 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 27) ==== 92060+0+0 (secure 0 0 0) 0x7f7498047020 con 0x7f74a80b2170 2024-04-20T11:38:10.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.414+0000 7f74a4c00640 1 --2- 172.21.15.152:0/3280542137 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f748406d430 0x7f748406f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:10.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.414+0000 7f74a4c00640 1 -- 172.21.15.152:0/3280542137 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 5709+0+0 (secure 0 0 0) 0x7f74980ac8b0 con 0x7f74a80b2170 2024-04-20T11:38:10.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.414+0000 7f74a7e00640 1 --2- 172.21.15.152:0/3280542137 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f748406d430 0x7f748406f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:10.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.416+0000 7f74a7e00640 1 --2- 172.21.15.152:0/3280542137 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f748406d430 0x7f748406f8f0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f74a815e380 tx=0x7f7490008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:10.419 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.418+0000 7f74a4c00640 1 -- 172.21.15.152:0/3280542137 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f749804e020 con 0x7f74a80b2170 2024-04-20T11:38:10.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.620+0000 7f74af600640 1 -- 172.21.15.152:0/3280542137 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f74a815dc60 con 0x7f74a80b2170 2024-04-20T11:38:10.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.621+0000 7f74a4c00640 1 -- 172.21.15.152:0/3280542137 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v167) ==== 93+0+10 (secure 0 0 0) 0x7f749807c730 con 0x7f74a80b2170 2024-04-20T11:38:10.622 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 1 2024-04-20T11:38:10.625 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.624+0000 7f74af600640 1 -- 172.21.15.152:0/3280542137 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f748406d430 msgr2=0x7f748406f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:10.625 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.624+0000 7f74af600640 1 --2- 172.21.15.152:0/3280542137 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f748406d430 0x7f748406f8f0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f74a815e380 tx=0x7f7490008040 comp rx=0 tx=0).stop 2024-04-20T11:38:10.625 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.624+0000 7f74af600640 1 -- 172.21.15.152:0/3280542137 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f74a80b2170 msgr2=0x7f74a80c71d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:10.625 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.624+0000 7f74af600640 1 --2- 172.21.15.152:0/3280542137 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f74a80b2170 0x7f74a80c71d0 secure :-1 s=READY pgs=2389 cs=0 l=1 rev1=1 crypto rx=0x7f7498037040 tx=0x7f7498004a40 comp rx=0 tx=0).stop 2024-04-20T11:38:10.625 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.624+0000 7f74af600640 1 -- 172.21.15.152:0/3280542137 shutdown_connections 2024-04-20T11:38:10.625 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.624+0000 7f74af600640 1 --2- 172.21.15.152:0/3280542137 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f748406d430 0x7f748406f8f0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:10.625 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.624+0000 7f74af600640 1 --2- 172.21.15.152:0/3280542137 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74a8146000 0x7f74a813fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:10.625 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.624+0000 7f74af600640 1 --2- 172.21.15.152:0/3280542137 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74a80cd1b0 0x7f74a813f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:10.625 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.624+0000 7f74af600640 1 --2- 172.21.15.152:0/3280542137 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f74a80b2170 0x7f74a80c71d0 unknown :-1 s=CLOSED pgs=2389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:10.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.624+0000 7f74af600640 1 -- 172.21.15.152:0/3280542137 >> 172.21.15.152:0/3280542137 conn(0x7f74a806b980 msgr2=0x7f74a80cf9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:10.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.624+0000 7f74af600640 1 -- 172.21.15.152:0/3280542137 shutdown_connections 2024-04-20T11:38:10.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.624+0000 7f74af600640 1 -- 172.21.15.152:0/3280542137 wait complete. 2024-04-20T11:38:10.646 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T11:38:10.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.853+0000 7f2558000640 1 Processor -- start 2024-04-20T11:38:10.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.853+0000 7f2558000640 1 -- start start 2024-04-20T11:38:10.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.855+0000 7f2558000640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f25500b3fa0 0x7f2550162440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:10.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.855+0000 7f2558000640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25500adc30 con 0x7f255015e5d0 2024-04-20T11:38:10.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.855+0000 7f2558000640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2550162a70 con 0x7f25500afba0 2024-04-20T11:38:10.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.855+0000 7f2558000640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f2550162d50 con 0x7f25500b3fa0 2024-04-20T11:38:10.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.855+0000 7f2555600640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f255015e5d0 0x7f25500af490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:34402/0 (socket says 172.21.15.152:34402) 2024-04-20T11:38:10.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.855+0000 7f2555600640 1 -- 172.21.15.152:0/2070866701 learned_addr learned my addr 172.21.15.152:0/2070866701 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:38:10.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.855+0000 7f2556000640 1 --2- 172.21.15.152:0/2070866701 >> v2:172.21.15.175:3300/0 conn(0x7f25500b3fa0 0x7f2550162440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:10.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.856+0000 7f254fe00640 1 -- 172.21.15.152:0/2070866701 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2880296195 0 0) 0x7f25500adc30 con 0x7f255015e5d0 2024-04-20T11:38:10.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.856+0000 7f254fe00640 1 -- 172.21.15.152:0/2070866701 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2538003520 con 0x7f255015e5d0 2024-04-20T11:38:10.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.856+0000 7f254fe00640 1 -- 172.21.15.152:0/2070866701 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1658860597 0 0) 0x7f2538003520 con 0x7f255015e5d0 2024-04-20T11:38:10.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.856+0000 7f254fe00640 1 -- 172.21.15.152:0/2070866701 >> v2:172.21.15.175:3300/0 conn(0x7f25500b3fa0 msgr2=0x7f2550162440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:10.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.856+0000 7f254fe00640 1 --2- 172.21.15.152:0/2070866701 >> v2:172.21.15.175:3300/0 conn(0x7f25500b3fa0 0x7f2550162440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:10.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.856+0000 7f254fe00640 1 -- 172.21.15.152:0/2070866701 >> v1:172.21.15.175:6789/0 conn(0x7f25500afba0 legacy=0x7f25500ae110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:10.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.856+0000 7f254fe00640 1 -- 172.21.15.152:0/2070866701 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2550163030 con 0x7f255015e5d0 2024-04-20T11:38:10.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.856+0000 7f254fe00640 1 -- 172.21.15.152:0/2070866701 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f2544001670 con 0x7f255015e5d0 2024-04-20T11:38:10.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.856+0000 7f254fe00640 1 -- 172.21.15.152:0/2070866701 >> v1:172.21.15.152:6789/0 conn(0x7f255015e5d0 legacy=0x7f25500af490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:10.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.856+0000 7f254fe00640 1 --2- 172.21.15.152:0/2070866701 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2538003a00 0x7f2538003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:10.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.856+0000 7f254fe00640 1 --2- 172.21.15.152:0/2070866701 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2538004800 0x7f2538004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:10.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.856+0000 7f254fe00640 1 --2- 172.21.15.152:0/2070866701 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f25380051c0 0x7f253800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:10.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.856+0000 7f254fe00640 1 -- 172.21.15.152:0/2070866701 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f2550162d50 con 0x7f25380051c0 2024-04-20T11:38:10.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.856+0000 7f254fe00640 1 -- 172.21.15.152:0/2070866701 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f253800c0c0 con 0x7f2538004800 2024-04-20T11:38:10.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.856+0000 7f254fe00640 1 -- 172.21.15.152:0/2070866701 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f253800c2c0 con 0x7f2538003a00 2024-04-20T11:38:10.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.856+0000 7f2556000640 1 --2- 172.21.15.152:0/2070866701 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f25380051c0 0x7f253800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:10.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.857+0000 7f2554c00640 1 --2- 172.21.15.152:0/2070866701 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2538004800 0x7f2538004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:10.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.857+0000 7f2555600640 1 --2- 172.21.15.152:0/2070866701 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2538003a00 0x7f2538003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:10.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.857+0000 7f2556000640 1 -- 172.21.15.152:0/2070866701 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2538003a00 msgr2=0x7f2538003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:10.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.857+0000 7f2556000640 1 --2- 172.21.15.152:0/2070866701 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2538003a00 0x7f2538003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:10.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.857+0000 7f2556000640 1 -- 172.21.15.152:0/2070866701 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2538004800 msgr2=0x7f2538004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:10.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.857+0000 7f2556000640 1 --2- 172.21.15.152:0/2070866701 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2538004800 0x7f2538004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:10.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.857+0000 7f2556000640 1 -- 172.21.15.152:0/2070866701 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2538003520 con 0x7f25380051c0 2024-04-20T11:38:10.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.857+0000 7f2555600640 1 --2- 172.21.15.152:0/2070866701 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2538003a00 0x7f2538003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:38:10.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.858+0000 7f2556000640 1 -- 172.21.15.152:0/2070866701 reap_dead start 2024-04-20T11:38:10.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.858+0000 7f2556000640 1 --2- 172.21.15.152:0/2070866701 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f25380051c0 0x7f253800b9c0 secure :-1 s=READY pgs=2391 cs=0 l=1 rev1=1 crypto rx=0x7f2548009fc0 tx=0x7f2538005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac2b5643f1ae798 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:10.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.858+0000 7f254fe00640 1 -- 172.21.15.152:0/2070866701 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2548009c10 con 0x7f25380051c0 2024-04-20T11:38:10.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.858+0000 7f254fe00640 1 -- 172.21.15.152:0/2070866701 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f254802cca0 con 0x7f25380051c0 2024-04-20T11:38:10.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.858+0000 7f254fe00640 1 -- 172.21.15.152:0/2070866701 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2548003bd0 con 0x7f25380051c0 2024-04-20T11:38:10.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.859+0000 7f2558000640 1 -- 172.21.15.152:0/2070866701 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f25380051c0 msgr2=0x7f253800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:10.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.859+0000 7f2558000640 1 --2- 172.21.15.152:0/2070866701 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f25380051c0 0x7f253800b9c0 secure :-1 s=READY pgs=2391 cs=0 l=1 rev1=1 crypto rx=0x7f2548009fc0 tx=0x7f2538005830 comp rx=0 tx=0).stop 2024-04-20T11:38:10.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.859+0000 7f2558000640 1 -- 172.21.15.152:0/2070866701 shutdown_connections 2024-04-20T11:38:10.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.859+0000 7f2558000640 1 --2- 172.21.15.152:0/2070866701 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f25380051c0 0x7f253800b9c0 unknown :-1 s=CLOSED pgs=2391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:10.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.859+0000 7f2558000640 1 -- 172.21.15.152:0/2070866701 >> 172.21.15.152:0/2070866701 conn(0x7f255006b980 msgr2=0x7f255006bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:10.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.859+0000 7f2558000640 1 -- 172.21.15.152:0/2070866701 shutdown_connections 2024-04-20T11:38:10.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.859+0000 7f2558000640 1 -- 172.21.15.152:0/2070866701 wait complete. 2024-04-20T11:38:10.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.861+0000 7f2558000640 1 Processor -- start 2024-04-20T11:38:10.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.861+0000 7f2558000640 1 -- start start 2024-04-20T11:38:10.863 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.862+0000 7f2558000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f25380051c0 0x7f25500b0d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:10.863 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.862+0000 7f2558000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2550147e90 0x7f255015cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:10.863 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.862+0000 7f2558000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f255015d150 0x7f25501f43c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:10.863 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.862+0000 7f2558000640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f255006e0a0 con 0x7f25380051c0 2024-04-20T11:38:10.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.862+0000 7f2558000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2550163310 con 0x7f255015d150 2024-04-20T11:38:10.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.862+0000 7f2558000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f255006e220 con 0x7f2550147e90 2024-04-20T11:38:10.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.862+0000 7f2555600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f25380051c0 0x7f25500b0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:10.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.862+0000 7f2555600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f25380051c0 0x7f25500b0d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:37412/0 (socket says 172.21.15.152:37412) 2024-04-20T11:38:10.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.862+0000 7f2555600640 1 -- 172.21.15.152:0/2788375676 learned_addr learned my addr 172.21.15.152:0/2788375676 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:38:10.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.863+0000 7f2554c00640 1 --2- 172.21.15.152:0/2788375676 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2550147e90 0x7f255015cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:10.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.863+0000 7f2556000640 1 --2- 172.21.15.152:0/2788375676 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f255015d150 0x7f25501f43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:10.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.863+0000 7f2555600640 1 -- 172.21.15.152:0/2788375676 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2550147e90 msgr2=0x7f255015cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:10.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.863+0000 7f2555600640 1 --2- 172.21.15.152:0/2788375676 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2550147e90 0x7f255015cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:10.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.863+0000 7f2555600640 1 -- 172.21.15.152:0/2788375676 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f255015d150 msgr2=0x7f25501f43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:10.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.863+0000 7f2555600640 1 --2- 172.21.15.152:0/2788375676 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f255015d150 0x7f25501f43c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:10.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.863+0000 7f2555600640 1 -- 172.21.15.152:0/2788375676 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25501f50f0 con 0x7f25380051c0 2024-04-20T11:38:10.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.863+0000 7f2555600640 1 --2- 172.21.15.152:0/2788375676 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f25380051c0 0x7f25500b0d90 secure :-1 s=READY pgs=2392 cs=0 l=1 rev1=1 crypto rx=0x7f254400d7a0 tx=0x7f254400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:10.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.863+0000 7f254d600640 1 -- 172.21.15.152:0/2788375676 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2544019070 con 0x7f25380051c0 2024-04-20T11:38:10.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.863+0000 7f254d600640 1 -- 172.21.15.152:0/2788375676 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25440074e0 con 0x7f25380051c0 2024-04-20T11:38:10.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.863+0000 7f2558000640 1 -- 172.21.15.152:0/2788375676 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25500b02b0 con 0x7f25380051c0 2024-04-20T11:38:10.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.863+0000 7f254d600640 1 -- 172.21.15.152:0/2788375676 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2544004a00 con 0x7f25380051c0 2024-04-20T11:38:10.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.864+0000 7f2558000640 1 -- 172.21.15.152:0/2788375676 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2550148310 con 0x7f25380051c0 2024-04-20T11:38:10.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.865+0000 7f2558000640 1 -- 172.21.15.152:0/2788375676 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25500689f0 con 0x7f25380051c0 2024-04-20T11:38:10.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.866+0000 7f254d600640 1 -- 172.21.15.152:0/2788375676 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 27) ==== 92060+0+0 (secure 0 0 0) 0x7f254401c070 con 0x7f25380051c0 2024-04-20T11:38:10.867 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.867+0000 7f254d600640 1 --2- 172.21.15.152:0/2788375676 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f253006d430 0x7f253006f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:10.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.867+0000 7f254d600640 1 -- 172.21.15.152:0/2788375676 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 5709+0+0 (secure 0 0 0) 0x7f254408a1c0 con 0x7f25380051c0 2024-04-20T11:38:10.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.868+0000 7f2554c00640 1 --2- 172.21.15.152:0/2788375676 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f253006d430 0x7f253006f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:10.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.869+0000 7f2554c00640 1 --2- 172.21.15.152:0/2788375676 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f253006d430 0x7f253006f8f0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f25500ce830 tx=0x7f253c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:10.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:10.872+0000 7f254d600640 1 -- 172.21.15.152:0/2788375676 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f2544059dc0 con 0x7f25380051c0 2024-04-20T11:38:11.075 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.074+0000 7f2558000640 1 -- 172.21.15.152:0/2788375676 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f255015d530 con 0x7f25380051c0 2024-04-20T11:38:11.076 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.075+0000 7f254d600640 1 -- 172.21.15.152:0/2788375676 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v167) ==== 104+0+11 (secure 0 0 0) 0x7f25440596c0 con 0x7f25380051c0 2024-04-20T11:38:11.076 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 32 2024-04-20T11:38:11.079 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.079+0000 7f2558000640 1 -- 172.21.15.152:0/2788375676 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f253006d430 msgr2=0x7f253006f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:11.079 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.079+0000 7f2558000640 1 --2- 172.21.15.152:0/2788375676 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f253006d430 0x7f253006f8f0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f25500ce830 tx=0x7f253c009040 comp rx=0 tx=0).stop 2024-04-20T11:38:11.079 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.079+0000 7f2558000640 1 -- 172.21.15.152:0/2788375676 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f25380051c0 msgr2=0x7f25500b0d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:11.080 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.079+0000 7f2558000640 1 --2- 172.21.15.152:0/2788375676 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f25380051c0 0x7f25500b0d90 secure :-1 s=READY pgs=2392 cs=0 l=1 rev1=1 crypto rx=0x7f254400d7a0 tx=0x7f254400dc60 comp rx=0 tx=0).stop 2024-04-20T11:38:11.080 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.079+0000 7f2558000640 1 -- 172.21.15.152:0/2788375676 shutdown_connections 2024-04-20T11:38:11.080 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.079+0000 7f2558000640 1 --2- 172.21.15.152:0/2788375676 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f253006d430 0x7f253006f8f0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:11.080 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.079+0000 7f2558000640 1 --2- 172.21.15.152:0/2788375676 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f255015d150 0x7f25501f43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:11.080 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.079+0000 7f2558000640 1 --2- 172.21.15.152:0/2788375676 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2550147e90 0x7f255015cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:11.080 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.079+0000 7f2558000640 1 --2- 172.21.15.152:0/2788375676 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f25380051c0 0x7f25500b0d90 unknown :-1 s=CLOSED pgs=2392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:11.080 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.079+0000 7f2558000640 1 -- 172.21.15.152:0/2788375676 >> 172.21.15.152:0/2788375676 conn(0x7f255006b980 msgr2=0x7f25500ae260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:11.081 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.079+0000 7f2558000640 1 -- 172.21.15.152:0/2788375676 shutdown_connections 2024-04-20T11:38:11.081 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.080+0000 7f2558000640 1 -- 172.21.15.152:0/2788375676 wait complete. 2024-04-20T11:38:11.099 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T11:38:11.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.301+0000 7fe75fc00640 1 Processor -- start 2024-04-20T11:38:11.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.301+0000 7fe75fc00640 1 -- start start 2024-04-20T11:38:11.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.302+0000 7fe75fc00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fe75815b510 0x7fe75815fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:11.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.302+0000 7fe75fc00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fe7580cfc40 0x7fe7580b3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:11.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.303+0000 7fe75fc00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe75815f1c0 con 0x7fe7580b2170 2024-04-20T11:38:11.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.303+0000 7fe75fc00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fe75806f420 con 0x7fe75815b510 2024-04-20T11:38:11.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.303+0000 7fe75fc00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fe7580b4430 con 0x7fe7580cfc40 2024-04-20T11:38:11.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.303+0000 7fe75d200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fe75815b510 0x7fe75815fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:11.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.303+0000 7fe75d200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fe75815b510 0x7fe75815fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:37416/0 (socket says 172.21.15.152:37416) 2024-04-20T11:38:11.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.303+0000 7fe75d200640 1 -- 172.21.15.152:0/3335158840 learned_addr learned my addr 172.21.15.152:0/3335158840 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:38:11.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.303+0000 7fe75d200640 1 -- 172.21.15.152:0/3335158840 >> v2:172.21.15.175:3300/0 conn(0x7fe7580cfc40 msgr2=0x7fe7580b3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:11.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.303+0000 7fe75d200640 1 --2- 172.21.15.152:0/3335158840 >> v2:172.21.15.175:3300/0 conn(0x7fe7580cfc40 0x7fe7580b3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:11.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.303+0000 7fe75d200640 1 -- 172.21.15.152:0/3335158840 >> v1:172.21.15.175:6789/0 conn(0x7fe7580b2170 legacy=0x7fe7580b4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:11.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.303+0000 7fe75d200640 1 -- 172.21.15.152:0/3335158840 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe75806f670 con 0x7fe75815b510 2024-04-20T11:38:11.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.304+0000 7fe75d200640 1 --2- 172.21.15.152:0/3335158840 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe75815b510 0x7fe75815fc30 secure :-1 s=READY pgs=2393 cs=0 l=1 rev1=1 crypto rx=0x7fe758072fa0 tx=0x7fe74c02da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e85cc2d2a296326 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:11.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.305+0000 7fe757400640 1 -- 172.21.15.152:0/3335158840 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe74c02f850 con 0x7fe75815b510 2024-04-20T11:38:11.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.305+0000 7fe757400640 1 -- 172.21.15.152:0/3335158840 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe74c02f9f0 con 0x7fe75815b510 2024-04-20T11:38:11.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.305+0000 7fe757400640 1 -- 172.21.15.152:0/3335158840 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe74c02fdc0 con 0x7fe75815b510 2024-04-20T11:38:11.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.305+0000 7fe75fc00640 1 -- 172.21.15.152:0/3335158840 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe75815b510 msgr2=0x7fe75815fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:11.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.305+0000 7fe75fc00640 1 --2- 172.21.15.152:0/3335158840 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe75815b510 0x7fe75815fc30 secure :-1 s=READY pgs=2393 cs=0 l=1 rev1=1 crypto rx=0x7fe758072fa0 tx=0x7fe74c02da90 comp rx=0 tx=0).stop 2024-04-20T11:38:11.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.306+0000 7fe75fc00640 1 -- 172.21.15.152:0/3335158840 shutdown_connections 2024-04-20T11:38:11.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.306+0000 7fe75fc00640 1 --2- 172.21.15.152:0/3335158840 >> v2:172.21.15.175:3300/0 conn(0x7fe7580cfc40 0x7fe7580b3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:11.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.306+0000 7fe75fc00640 1 --2- 172.21.15.152:0/3335158840 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe75815b510 0x7fe75815fc30 unknown :-1 s=CLOSED pgs=2393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:11.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.306+0000 7fe75fc00640 1 -- 172.21.15.152:0/3335158840 >> 172.21.15.152:0/3335158840 conn(0x7fe75806b980 msgr2=0x7fe75806bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:11.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.306+0000 7fe75fc00640 1 -- 172.21.15.152:0/3335158840 shutdown_connections 2024-04-20T11:38:11.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.306+0000 7fe75fc00640 1 -- 172.21.15.152:0/3335158840 wait complete. 2024-04-20T11:38:11.308 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.307+0000 7fe75fc00640 1 Processor -- start 2024-04-20T11:38:11.308 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.308+0000 7fe75fc00640 1 -- start start 2024-04-20T11:38:11.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.309+0000 7fe75fc00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe7580b2170 0x7fe758152230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:11.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.309+0000 7fe75fc00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7580cfc40 0x7fe75814a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:11.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.309+0000 7fe75fc00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe75815b510 0x7fe758141ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:11.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.309+0000 7fe75fc00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fe75815b8f0 con 0x7fe7580b2170 2024-04-20T11:38:11.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.309+0000 7fe75fc00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe75815f710 con 0x7fe7580cfc40 2024-04-20T11:38:11.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.309+0000 7fe75fc00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe75806f950 con 0x7fe75815b510 2024-04-20T11:38:11.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.309+0000 7fe75d200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe7580b2170 0x7fe758152230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:11.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.309+0000 7fe75d200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe7580b2170 0x7fe758152230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:37422/0 (socket says 172.21.15.152:37422) 2024-04-20T11:38:11.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.309+0000 7fe75d200640 1 -- 172.21.15.152:0/2869427576 learned_addr learned my addr 172.21.15.152:0/2869427576 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:38:11.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.309+0000 7fe75d200640 1 -- 172.21.15.152:0/2869427576 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe75815b510 msgr2=0x7fe758141ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:11.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.309+0000 7fe75d200640 1 --2- 172.21.15.152:0/2869427576 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe75815b510 0x7fe758141ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:11.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.309+0000 7fe75d200640 1 -- 172.21.15.152:0/2869427576 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7580cfc40 msgr2=0x7fe75814a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:11.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.309+0000 7fe75d200640 1 --2- 172.21.15.152:0/2869427576 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7580cfc40 0x7fe75814a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:11.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.309+0000 7fe75d200640 1 -- 172.21.15.152:0/2869427576 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7581496a0 con 0x7fe7580b2170 2024-04-20T11:38:11.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.310+0000 7fe75d200640 1 --2- 172.21.15.152:0/2869427576 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe7580b2170 0x7fe758152230 secure :-1 s=READY pgs=2394 cs=0 l=1 rev1=1 crypto rx=0x7fe74c02e0a0 tx=0x7fe74c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:11.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.310+0000 7fe754c00640 1 -- 172.21.15.152:0/2869427576 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe74c049070 con 0x7fe7580b2170 2024-04-20T11:38:11.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.310+0000 7fe754c00640 1 -- 172.21.15.152:0/2869427576 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe74c039c40 con 0x7fe7580b2170 2024-04-20T11:38:11.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.310+0000 7fe754c00640 1 -- 172.21.15.152:0/2869427576 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe74c030420 con 0x7fe7580b2170 2024-04-20T11:38:11.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.310+0000 7fe75fc00640 1 -- 172.21.15.152:0/2869427576 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe758149980 con 0x7fe7580b2170 2024-04-20T11:38:11.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.310+0000 7fe75fc00640 1 -- 172.21.15.152:0/2869427576 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe758149f40 con 0x7fe7580b2170 2024-04-20T11:38:11.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.311+0000 7fe75fc00640 1 -- 172.21.15.152:0/2869427576 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7580689f0 con 0x7fe7580b2170 2024-04-20T11:38:11.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.312+0000 7fe754c00640 1 -- 172.21.15.152:0/2869427576 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 27) ==== 92060+0+0 (secure 0 0 0) 0x7fe74c043390 con 0x7fe7580b2170 2024-04-20T11:38:11.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.313+0000 7fe754c00640 1 --2- 172.21.15.152:0/2869427576 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe73806d430 0x7fe73806f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:11.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.313+0000 7fe754c00640 1 -- 172.21.15.152:0/2869427576 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 5709+0+0 (secure 0 0 0) 0x7fe74c0ae420 con 0x7fe7580b2170 2024-04-20T11:38:11.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.314+0000 7fe757e00640 1 --2- 172.21.15.152:0/2869427576 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe73806d430 0x7fe73806f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:11.315 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.315+0000 7fe757e00640 1 --2- 172.21.15.152:0/2869427576 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe73806d430 0x7fe73806f8f0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fe75815ef80 tx=0x7fe744006c90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:11.318 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.317+0000 7fe754c00640 1 -- 172.21.15.152:0/2869427576 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe74c07e090 con 0x7fe7580b2170 2024-04-20T11:38:11.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.517+0000 7fe75fc00640 1 -- 172.21.15.152:0/2869427576 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fe75806ecf0 con 0x7fe7580b2170 2024-04-20T11:38:11.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.518+0000 7fe754c00640 1 -- 172.21.15.152:0/2869427576 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v167) ==== 100+0+11 (secure 0 0 0) 0x7fe74c07d990 con 0x7fe7580b2170 2024-04-20T11:38:11.519 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 64 2024-04-20T11:38:11.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.520+0000 7fe75fc00640 1 -- 172.21.15.152:0/2869427576 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe73806d430 msgr2=0x7fe73806f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:11.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.520+0000 7fe75fc00640 1 --2- 172.21.15.152:0/2869427576 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe73806d430 0x7fe73806f8f0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fe75815ef80 tx=0x7fe744006c90 comp rx=0 tx=0).stop 2024-04-20T11:38:11.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.520+0000 7fe75fc00640 1 -- 172.21.15.152:0/2869427576 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe7580b2170 msgr2=0x7fe758152230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:11.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.520+0000 7fe75fc00640 1 --2- 172.21.15.152:0/2869427576 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe7580b2170 0x7fe758152230 secure :-1 s=READY pgs=2394 cs=0 l=1 rev1=1 crypto rx=0x7fe74c02e0a0 tx=0x7fe74c004730 comp rx=0 tx=0).stop 2024-04-20T11:38:11.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.521+0000 7fe75fc00640 1 -- 172.21.15.152:0/2869427576 shutdown_connections 2024-04-20T11:38:11.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.521+0000 7fe75fc00640 1 --2- 172.21.15.152:0/2869427576 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe73806d430 0x7fe73806f8f0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:11.522 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.521+0000 7fe75fc00640 1 --2- 172.21.15.152:0/2869427576 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe75815b510 0x7fe758141ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:11.522 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.521+0000 7fe75fc00640 1 --2- 172.21.15.152:0/2869427576 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7580cfc40 0x7fe75814a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:11.522 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.521+0000 7fe75fc00640 1 --2- 172.21.15.152:0/2869427576 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe7580b2170 0x7fe758152230 unknown :-1 s=CLOSED pgs=2394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:11.522 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.521+0000 7fe75fc00640 1 -- 172.21.15.152:0/2869427576 >> 172.21.15.152:0/2869427576 conn(0x7fe75806b980 msgr2=0x7fe75814bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:11.522 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.521+0000 7fe75fc00640 1 -- 172.21.15.152:0/2869427576 shutdown_connections 2024-04-20T11:38:11.522 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.521+0000 7fe75fc00640 1 -- 172.21.15.152:0/2869427576 wait complete. 2024-04-20T11:38:11.541 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:38:11.738 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.736+0000 7fddf1000640 1 Processor -- start 2024-04-20T11:38:11.738 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.736+0000 7fddf1000640 1 -- start start 2024-04-20T11:38:11.738 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.738+0000 7fddf1000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fddec0ce9c0 0x7fddec0cc9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:11.739 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.738+0000 7fddf1000640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fddec09b150 0x7fddec0d14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:11.739 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.738+0000 7fddf1000640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fddec162270 con 0x7fddec0d1040 2024-04-20T11:38:11.739 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.738+0000 7fddf1000640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fddec0b1d00 con 0x7fddec0ce9c0 2024-04-20T11:38:11.739 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.738+0000 7fddf1000640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fddec0b1fe0 con 0x7fddec09b150 2024-04-20T11:38:11.740 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.738+0000 7fddeb400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fddec0ce9c0 0x7fddec0cc9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:11.740 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.738+0000 7fdde2a00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fddec0d1040 0x7fddec09aa40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:34410/0 (socket says 172.21.15.152:34410) 2024-04-20T11:38:11.740 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.738+0000 7fdde2a00640 1 -- 172.21.15.152:0/3471116812 learned_addr learned my addr 172.21.15.152:0/3471116812 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:38:11.740 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.739+0000 7fddebe00640 1 --2- 172.21.15.152:0/3471116812 >> v2:172.21.15.175:3300/0 conn(0x7fddec09b150 0x7fddec0d14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:11.740 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.739+0000 7fddeb400640 1 -- 172.21.15.152:0/3471116812 >> v2:172.21.15.175:3300/0 conn(0x7fddec09b150 msgr2=0x7fddec0d14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:11.740 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.739+0000 7fddeb400640 1 --2- 172.21.15.152:0/3471116812 >> v2:172.21.15.175:3300/0 conn(0x7fddec09b150 0x7fddec0d14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:11.740 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.739+0000 7fddeb400640 1 -- 172.21.15.152:0/3471116812 >> v1:172.21.15.152:6789/0 conn(0x7fddec0d1040 legacy=0x7fddec09aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:11.740 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.739+0000 7fddeb400640 1 -- 172.21.15.152:0/3471116812 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddec0b22c0 con 0x7fddec0ce9c0 2024-04-20T11:38:11.740 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.739+0000 7fddeaa00640 1 -- 172.21.15.152:0/3471116812 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1884979780 0 0) 0x7fddec162270 con 0x7fddec0d1040 2024-04-20T11:38:11.741 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.741+0000 7fddeb400640 1 --2- 172.21.15.152:0/3471116812 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fddec0ce9c0 0x7fddec0cc9f0 secure :-1 s=READY pgs=2396 cs=0 l=1 rev1=1 crypto rx=0x7fddec0cedd0 tx=0x7fdddc02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=250f4b930a18bace server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:11.741 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.741+0000 7fddeaa00640 1 -- 172.21.15.152:0/3471116812 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdddc034070 con 0x7fddec0ce9c0 2024-04-20T11:38:11.741 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.741+0000 7fddeaa00640 1 -- 172.21.15.152:0/3471116812 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdddc030610 con 0x7fddec0ce9c0 2024-04-20T11:38:11.742 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.741+0000 7fddeaa00640 1 -- 172.21.15.152:0/3471116812 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdddc0309e0 con 0x7fddec0ce9c0 2024-04-20T11:38:11.742 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.742+0000 7fddf1000640 1 -- 172.21.15.152:0/3471116812 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fddec0ce9c0 msgr2=0x7fddec0cc9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:11.742 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.742+0000 7fddf1000640 1 --2- 172.21.15.152:0/3471116812 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fddec0ce9c0 0x7fddec0cc9f0 secure :-1 s=READY pgs=2396 cs=0 l=1 rev1=1 crypto rx=0x7fddec0cedd0 tx=0x7fdddc02b960 comp rx=0 tx=0).stop 2024-04-20T11:38:11.743 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.742+0000 7fddf1000640 1 -- 172.21.15.152:0/3471116812 shutdown_connections 2024-04-20T11:38:11.743 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.742+0000 7fddf1000640 1 --2- 172.21.15.152:0/3471116812 >> v2:172.21.15.175:3300/0 conn(0x7fddec09b150 0x7fddec0d14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:11.743 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.742+0000 7fddf1000640 1 --2- 172.21.15.152:0/3471116812 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fddec0ce9c0 0x7fddec0cc9f0 unknown :-1 s=CLOSED pgs=2396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:11.743 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.742+0000 7fddf1000640 1 -- 172.21.15.152:0/3471116812 >> 172.21.15.152:0/3471116812 conn(0x7fddec0b3e90 msgr2=0x7fddec0b42a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:11.743 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.742+0000 7fddf1000640 1 -- 172.21.15.152:0/3471116812 shutdown_connections 2024-04-20T11:38:11.743 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.742+0000 7fddf1000640 1 -- 172.21.15.152:0/3471116812 wait complete. 2024-04-20T11:38:11.744 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.744+0000 7fddf1000640 1 Processor -- start 2024-04-20T11:38:11.744 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.744+0000 7fddf1000640 1 -- start start 2024-04-20T11:38:11.746 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.745+0000 7fddf1000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fddec09b150 0x7fddec0c7da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:11.746 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.745+0000 7fddf1000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fddec0ce9c0 0x7fddec1f8980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:11.746 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.745+0000 7fddf1000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fddec0d1040 0x7fddec15f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:11.746 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.745+0000 7fddf1000640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fddec06cd20 con 0x7fddec0ce9c0 2024-04-20T11:38:11.746 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.745+0000 7fddf1000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fddec0b2880 con 0x7fddec09b150 2024-04-20T11:38:11.747 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.745+0000 7fddf1000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fddec0b2da0 con 0x7fddec0d1040 2024-04-20T11:38:11.747 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.745+0000 7fdde2a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fddec0ce9c0 0x7fddec1f8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:11.747 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.745+0000 7fdde2a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fddec0ce9c0 0x7fddec1f8980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:37434/0 (socket says 172.21.15.152:37434) 2024-04-20T11:38:11.747 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.745+0000 7fdde2a00640 1 -- 172.21.15.152:0/2330954540 learned_addr learned my addr 172.21.15.152:0/2330954540 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:38:11.747 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.745+0000 7fddeb400640 1 --2- 172.21.15.152:0/2330954540 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fddec09b150 0x7fddec0c7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:11.747 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.745+0000 7fdde2a00640 1 -- 172.21.15.152:0/2330954540 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fddec0d1040 msgr2=0x7fddec15f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:11.747 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.745+0000 7fdde2a00640 1 --2- 172.21.15.152:0/2330954540 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fddec0d1040 0x7fddec15f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:11.747 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.745+0000 7fdde2a00640 1 -- 172.21.15.152:0/2330954540 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fddec09b150 msgr2=0x7fddec0c7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:11.747 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.745+0000 7fdde2a00640 1 --2- 172.21.15.152:0/2330954540 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fddec09b150 0x7fddec0c7da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:11.747 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.745+0000 7fdde2a00640 1 -- 172.21.15.152:0/2330954540 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddec1f5950 con 0x7fddec0ce9c0 2024-04-20T11:38:11.747 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.746+0000 7fdde2a00640 1 --2- 172.21.15.152:0/2330954540 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fddec0ce9c0 0x7fddec1f8980 secure :-1 s=READY pgs=2397 cs=0 l=1 rev1=1 crypto rx=0x7fddd400c460 tx=0x7fddd400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:11.748 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.746+0000 7fdde3e00640 1 -- 172.21.15.152:0/2330954540 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd4018070 con 0x7fddec0ce9c0 2024-04-20T11:38:11.748 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.746+0000 7fdde3e00640 1 -- 172.21.15.152:0/2330954540 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddd4007500 con 0x7fddec0ce9c0 2024-04-20T11:38:11.748 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.746+0000 7fdde3e00640 1 -- 172.21.15.152:0/2330954540 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd4004a20 con 0x7fddec0ce9c0 2024-04-20T11:38:11.748 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.746+0000 7fddf1000640 1 -- 172.21.15.152:0/2330954540 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddec1f4b90 con 0x7fddec0ce9c0 2024-04-20T11:38:11.748 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.746+0000 7fddf1000640 1 -- 172.21.15.152:0/2330954540 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddec1f5150 con 0x7fddec0ce9c0 2024-04-20T11:38:11.748 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.747+0000 7fddf1000640 1 -- 172.21.15.152:0/2330954540 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddec071620 con 0x7fddec0ce9c0 2024-04-20T11:38:11.749 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.748+0000 7fdde3e00640 1 -- 172.21.15.152:0/2330954540 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 27) ==== 92060+0+0 (secure 0 0 0) 0x7fddd401b070 con 0x7fddec0ce9c0 2024-04-20T11:38:11.749 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.749+0000 7fdde3e00640 1 --2- 172.21.15.152:0/2330954540 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fddc806d430 0x7fddc806f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:11.750 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.749+0000 7fdde3e00640 1 -- 172.21.15.152:0/2330954540 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 5709+0+0 (secure 0 0 0) 0x7fddd40899d0 con 0x7fddec0ce9c0 2024-04-20T11:38:11.750 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.750+0000 7fddeb400640 1 --2- 172.21.15.152:0/2330954540 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fddc806d430 0x7fddc806f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:11.751 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.751+0000 7fddeb400640 1 --2- 172.21.15.152:0/2330954540 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fddc806d430 0x7fddc806f8f0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fdddc036000 tx=0x7fdddc008030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:11.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.754+0000 7fdde3e00640 1 -- 172.21.15.152:0/2330954540 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fddec071620 con 0x7fddec0ce9c0 2024-04-20T11:38:11.974 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.974+0000 7fddf1000640 1 -- 172.21.15.152:0/2330954540 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fddec075e60 con 0x7fddec0ce9c0 2024-04-20T11:38:11.975 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.975+0000 7fdde3e00640 1 -- 172.21.15.152:0/2330954540 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 116 v116) ==== 95+0+5215 (secure 0 0 0) 0x7fddd4059600 con 0x7fddec0ce9c0 2024-04-20T11:38:11.975 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:38:11.976 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":116,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6605,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6834/4020670267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6833","nonce":4020670267},{"type":"v1","addr":"172.21.15.152:6834","nonce":4020670267}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":109},{"gid":6614,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/352716681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":352716681},{"type":"v1","addr":"172.21.15.175:6833","nonce":352716681}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":110},{"gid":6617,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/2512108447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":2512108447},{"type":"v1","addr":"172.21.15.175:6835","nonce":2512108447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":110}],"filesystems":[{"mdsmap":{"epoch":115,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T11:37:47.506735+0000","modified":"2024-04-20T11:37:50.605372+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7339},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7339":{"gid":7339,"name":"c","rank":0,"incarnation":111,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6835/951802292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":951802292},{"type":"v1","addr":"172.21.15.152:6835","nonce":951802292}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7339,"qdb_cluster":[7339]},"id":13}]} 2024-04-20T11:38:11.976 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 116 2024-04-20T11:38:11.978 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.977+0000 7fddf1000640 1 -- 172.21.15.152:0/2330954540 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fddc806d430 msgr2=0x7fddc806f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:11.978 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.977+0000 7fddf1000640 1 --2- 172.21.15.152:0/2330954540 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fddc806d430 0x7fddc806f8f0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fdddc036000 tx=0x7fdddc008030 comp rx=0 tx=0).stop 2024-04-20T11:38:11.978 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.977+0000 7fddf1000640 1 -- 172.21.15.152:0/2330954540 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fddec0ce9c0 msgr2=0x7fddec1f8980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:11.979 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.977+0000 7fddf1000640 1 --2- 172.21.15.152:0/2330954540 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fddec0ce9c0 0x7fddec1f8980 secure :-1 s=READY pgs=2397 cs=0 l=1 rev1=1 crypto rx=0x7fddd400c460 tx=0x7fddd400c920 comp rx=0 tx=0).stop 2024-04-20T11:38:11.979 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.977+0000 7fddf1000640 1 -- 172.21.15.152:0/2330954540 shutdown_connections 2024-04-20T11:38:11.979 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.977+0000 7fddf1000640 1 --2- 172.21.15.152:0/2330954540 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fddc806d430 0x7fddc806f8f0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:11.979 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.977+0000 7fddf1000640 1 --2- 172.21.15.152:0/2330954540 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fddec0d1040 0x7fddec15f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:11.979 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.977+0000 7fddf1000640 1 --2- 172.21.15.152:0/2330954540 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fddec0ce9c0 0x7fddec1f8980 unknown :-1 s=CLOSED pgs=2397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:11.979 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.977+0000 7fddf1000640 1 --2- 172.21.15.152:0/2330954540 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fddec09b150 0x7fddec0c7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:11.979 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.978+0000 7fddf1000640 1 -- 172.21.15.152:0/2330954540 >> 172.21.15.152:0/2330954540 conn(0x7fddec0b3e90 msgr2=0x7fddec070f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:11.979 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.978+0000 7fddf1000640 1 -- 172.21.15.152:0/2330954540 shutdown_connections 2024-04-20T11:38:11.979 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:11.978+0000 7fddf1000640 1 -- 172.21.15.152:0/2330954540 wait complete. 2024-04-20T11:38:11.997 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:38:12.194 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.193+0000 7efd1ea00640 1 Processor -- start 2024-04-20T11:38:12.194 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.194+0000 7efd1ea00640 1 -- start start 2024-04-20T11:38:12.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.195+0000 7efd1ea00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efd18145f60 0x7efd1806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:12.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.195+0000 7efd1ea00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd18146340 con 0x7efd180cd0d0 2024-04-20T11:38:12.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.195+0000 7efd1ea00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd1806f3c0 con 0x7efd180726f0 2024-04-20T11:38:12.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.195+0000 7efd1ea00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7efd1806e9f0 con 0x7efd18145f60 2024-04-20T11:38:12.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.196+0000 7efd17400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7efd180cd0d0 0x7efd18071fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:34414/0 (socket says 172.21.15.152:34414) 2024-04-20T11:38:12.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.196+0000 7efd17400640 1 -- 172.21.15.152:0/907686955 learned_addr learned my addr 172.21.15.152:0/907686955 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:38:12.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.196+0000 7efd17e00640 1 --2- 172.21.15.152:0/907686955 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efd18145f60 0x7efd1806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:12.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.196+0000 7efd16a00640 1 -- 172.21.15.152:0/907686955 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3612508488 0 0) 0x7efd18146340 con 0x7efd180cd0d0 2024-04-20T11:38:12.199 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.196+0000 7efd16a00640 1 -- 172.21.15.152:0/907686955 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efcfc003520 con 0x7efd180cd0d0 2024-04-20T11:38:12.199 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.196+0000 7efd16a00640 1 -- 172.21.15.152:0/907686955 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3948624281 0 0) 0x7efcfc003520 con 0x7efd180cd0d0 2024-04-20T11:38:12.199 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.196+0000 7efd16a00640 1 -- 172.21.15.152:0/907686955 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efd18145f60 msgr2=0x7efd1806ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:12.199 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.196+0000 7efd16a00640 1 --2- 172.21.15.152:0/907686955 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efd18145f60 0x7efd1806ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:12.199 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.196+0000 7efd16a00640 1 -- 172.21.15.152:0/907686955 >> v1:172.21.15.175:6789/0 conn(0x7efd180726f0 legacy=0x7efd180cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:12.199 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.197+0000 7efd16a00640 1 -- 172.21.15.152:0/907686955 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd18072d50 con 0x7efd180cd0d0 2024-04-20T11:38:12.199 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.197+0000 7efd16a00640 1 -- 172.21.15.152:0/907686955 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7efd04001670 con 0x7efd180cd0d0 2024-04-20T11:38:12.199 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.197+0000 7efd16a00640 1 -- 172.21.15.152:0/907686955 >> v1:172.21.15.152:6789/0 conn(0x7efd180cd0d0 legacy=0x7efd18071fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:12.199 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.197+0000 7efd16a00640 1 --2- 172.21.15.152:0/907686955 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efcfc003a50 0x7efcfc003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:12.199 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.197+0000 7efd16a00640 1 --2- 172.21.15.152:0/907686955 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efd18145f60 0x7efcfc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:12.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.197+0000 7efd16a00640 1 --2- 172.21.15.152:0/907686955 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efcfc004d40 0x7efcfc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:12.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.197+0000 7efd16a00640 1 -- 172.21.15.152:0/907686955 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7efd1806e9f0 con 0x7efcfc003a50 2024-04-20T11:38:12.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.197+0000 7efd16a00640 1 -- 172.21.15.152:0/907686955 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7efcfc00bad0 con 0x7efcfc004d40 2024-04-20T11:38:12.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.197+0000 7efd16a00640 1 -- 172.21.15.152:0/907686955 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7efcfc00bcd0 con 0x7efd18145f60 2024-04-20T11:38:12.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.197+0000 7efd17e00640 1 --2- 172.21.15.152:0/907686955 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efcfc003a50 0x7efcfc003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:12.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.197+0000 7efd17e00640 1 -- 172.21.15.152:0/907686955 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efd18145f60 msgr2=0x7efcfc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:12.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.197+0000 7efd17e00640 1 --2- 172.21.15.152:0/907686955 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efd18145f60 0x7efcfc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:12.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.197+0000 7efd17e00640 1 -- 172.21.15.152:0/907686955 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efcfc004d40 msgr2=0x7efcfc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:12.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.197+0000 7efd17e00640 1 --2- 172.21.15.152:0/907686955 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efcfc004d40 0x7efcfc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:12.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.197+0000 7efd17e00640 1 -- 172.21.15.152:0/907686955 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcfc003520 con 0x7efcfc003a50 2024-04-20T11:38:12.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.199+0000 7efd17e00640 1 --2- 172.21.15.152:0/907686955 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efcfc003a50 0x7efcfc003e70 secure :-1 s=READY pgs=2399 cs=0 l=1 rev1=1 crypto rx=0x7efd08009a90 tx=0x7efcfc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e44065cfeeba58be server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:12.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.199+0000 7efd16a00640 1 -- 172.21.15.152:0/907686955 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd08039070 con 0x7efcfc003a50 2024-04-20T11:38:12.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.199+0000 7efd16a00640 1 -- 172.21.15.152:0/907686955 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd0802c600 con 0x7efcfc003a50 2024-04-20T11:38:12.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.199+0000 7efd16a00640 1 -- 172.21.15.152:0/907686955 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd08009690 con 0x7efcfc003a50 2024-04-20T11:38:12.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.199+0000 7efd1ea00640 1 -- 172.21.15.152:0/907686955 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efcfc003a50 msgr2=0x7efcfc003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:12.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.199+0000 7efd1ea00640 1 --2- 172.21.15.152:0/907686955 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efcfc003a50 0x7efcfc003e70 secure :-1 s=READY pgs=2399 cs=0 l=1 rev1=1 crypto rx=0x7efd08009a90 tx=0x7efcfc005830 comp rx=0 tx=0).stop 2024-04-20T11:38:12.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.199+0000 7efd1ca00640 1 -- 172.21.15.152:0/907686955 reap_dead start 2024-04-20T11:38:12.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.199+0000 7efd1ea00640 1 -- 172.21.15.152:0/907686955 shutdown_connections 2024-04-20T11:38:12.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.199+0000 7efd1ea00640 1 -- 172.21.15.152:0/907686955 >> 172.21.15.152:0/907686955 conn(0x7efd18067cf0 msgr2=0x7efd1806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:12.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.200+0000 7efd1ea00640 1 -- 172.21.15.152:0/907686955 shutdown_connections 2024-04-20T11:38:12.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.200+0000 7efd1ea00640 1 -- 172.21.15.152:0/907686955 wait complete. 2024-04-20T11:38:12.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.201+0000 7efd1ea00640 1 Processor -- start 2024-04-20T11:38:12.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.201+0000 7efd1ea00640 1 -- start start 2024-04-20T11:38:12.205 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.202+0000 7efd1ea00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efcfc003a50 0x7efd180af020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:12.205 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.202+0000 7efd1ea00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efd1813ce90 0x7efd180cff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:12.205 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.202+0000 7efd1ea00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efd180d04c0 0x7efd181f4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:12.205 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.202+0000 7efd1ea00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7efd1806e680 con 0x7efd1813ce90 2024-04-20T11:38:12.205 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.202+0000 7efd1ea00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7efd1806e3a0 con 0x7efcfc003a50 2024-04-20T11:38:12.205 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.202+0000 7efd1ea00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7efd1806ee80 con 0x7efd180d04c0 2024-04-20T11:38:12.205 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.203+0000 7efd17400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efd1813ce90 0x7efd180cff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:12.205 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.203+0000 7efd17400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efd1813ce90 0x7efd180cff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:37450/0 (socket says 172.21.15.152:37450) 2024-04-20T11:38:12.205 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.203+0000 7efd17400640 1 -- 172.21.15.152:0/1184061696 learned_addr learned my addr 172.21.15.152:0/1184061696 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:38:12.206 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.203+0000 7efd17e00640 1 --2- 172.21.15.152:0/1184061696 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efcfc003a50 0x7efd180af020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:12.206 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.203+0000 7efd1ca00640 1 --2- 172.21.15.152:0/1184061696 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efd180d04c0 0x7efd181f4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:12.206 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.203+0000 7efd17400640 1 -- 172.21.15.152:0/1184061696 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efd180d04c0 msgr2=0x7efd181f4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:12.206 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.203+0000 7efd17400640 1 --2- 172.21.15.152:0/1184061696 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efd180d04c0 0x7efd181f4c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:12.206 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.203+0000 7efd17400640 1 -- 172.21.15.152:0/1184061696 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efcfc003a50 msgr2=0x7efd180af020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:12.206 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.203+0000 7efd17400640 1 --2- 172.21.15.152:0/1184061696 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efcfc003a50 0x7efd180af020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:12.206 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.203+0000 7efd17400640 1 -- 172.21.15.152:0/1184061696 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd180af560 con 0x7efd1813ce90 2024-04-20T11:38:12.206 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.203+0000 7efd17400640 1 --2- 172.21.15.152:0/1184061696 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efd1813ce90 0x7efd180cff80 secure :-1 s=READY pgs=2400 cs=0 l=1 rev1=1 crypto rx=0x7efd0400c700 tx=0x7efd0400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:12.206 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.204+0000 7efd0fe00640 1 -- 172.21.15.152:0/1184061696 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd04018070 con 0x7efd1813ce90 2024-04-20T11:38:12.206 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.204+0000 7efd0fe00640 1 -- 172.21.15.152:0/1184061696 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd04007500 con 0x7efd1813ce90 2024-04-20T11:38:12.206 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.204+0000 7efd0fe00640 1 -- 172.21.15.152:0/1184061696 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd04004a20 con 0x7efd1813ce90 2024-04-20T11:38:12.207 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.204+0000 7efd1ea00640 1 -- 172.21.15.152:0/1184061696 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd18142730 con 0x7efd1813ce90 2024-04-20T11:38:12.207 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.204+0000 7efd1ea00640 1 -- 172.21.15.152:0/1184061696 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd18142cf0 con 0x7efd1813ce90 2024-04-20T11:38:12.207 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.205+0000 7efd1ea00640 1 -- 172.21.15.152:0/1184061696 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd180689f0 con 0x7efd1813ce90 2024-04-20T11:38:12.207 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.206+0000 7efd0fe00640 1 -- 172.21.15.152:0/1184061696 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 27) ==== 92060+0+0 (secure 0 0 0) 0x7efd04004bc0 con 0x7efd1813ce90 2024-04-20T11:38:12.208 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.207+0000 7efd0fe00640 1 --2- 172.21.15.152:0/1184061696 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7efcf806d430 0x7efcf806f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:12.208 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.207+0000 7efd0fe00640 1 -- 172.21.15.152:0/1184061696 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 5709+0+0 (secure 0 0 0) 0x7efd04089e20 con 0x7efd1813ce90 2024-04-20T11:38:12.208 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.208+0000 7efd17e00640 1 --2- 172.21.15.152:0/1184061696 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7efcf806d430 0x7efcf806f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:12.209 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.209+0000 7efd17e00640 1 --2- 172.21.15.152:0/1184061696 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7efcf806d430 0x7efcf806f8f0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7efd08036000 tx=0x7efd08030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:12.212 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.211+0000 7efd0fe00640 1 -- 172.21.15.152:0/1184061696 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7efd180af560 con 0x7efd1813ce90 2024-04-20T11:38:12.411 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.411+0000 7efd1ea00640 1 -- 172.21.15.152:0/1184061696 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efd180b4900 con 0x7efd1813ce90 2024-04-20T11:38:12.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.412+0000 7efd0fe00640 1 -- 172.21.15.152:0/1184061696 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v167) ==== 74+0+16743 (secure 0 0 0) 0x7efd040599c0 con 0x7efd1813ce90 2024-04-20T11:38:12.414 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:38:12.414 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":167,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:37:49.091768+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:37:45.247938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"166","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2024-04-20T11:37:46.503171+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"167","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:38:12.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.415+0000 7efd1ea00640 1 -- 172.21.15.152:0/1184061696 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7efcf806d430 msgr2=0x7efcf806f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:12.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.415+0000 7efd1ea00640 1 --2- 172.21.15.152:0/1184061696 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7efcf806d430 0x7efcf806f8f0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7efd08036000 tx=0x7efd08030000 comp rx=0 tx=0).stop 2024-04-20T11:38:12.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.415+0000 7efd1ea00640 1 -- 172.21.15.152:0/1184061696 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efd1813ce90 msgr2=0x7efd180cff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:12.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.415+0000 7efd1ea00640 1 --2- 172.21.15.152:0/1184061696 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efd1813ce90 0x7efd180cff80 secure :-1 s=READY pgs=2400 cs=0 l=1 rev1=1 crypto rx=0x7efd0400c700 tx=0x7efd0400cbc0 comp rx=0 tx=0).stop 2024-04-20T11:38:12.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.415+0000 7efd1ea00640 1 -- 172.21.15.152:0/1184061696 shutdown_connections 2024-04-20T11:38:12.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.415+0000 7efd1ea00640 1 --2- 172.21.15.152:0/1184061696 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7efcf806d430 0x7efcf806f8f0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:12.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.415+0000 7efd1ea00640 1 --2- 172.21.15.152:0/1184061696 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efd180d04c0 0x7efd181f4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:12.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.415+0000 7efd1ea00640 1 --2- 172.21.15.152:0/1184061696 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efd1813ce90 0x7efd180cff80 unknown :-1 s=CLOSED pgs=2400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:12.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.415+0000 7efd1ea00640 1 --2- 172.21.15.152:0/1184061696 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efcfc003a50 0x7efd180af020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:12.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.415+0000 7efd1ea00640 1 -- 172.21.15.152:0/1184061696 >> 172.21.15.152:0/1184061696 conn(0x7efd18067cf0 msgr2=0x7efd18141150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:12.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.416+0000 7efd1ea00640 1 -- 172.21.15.152:0/1184061696 shutdown_connections 2024-04-20T11:38:12.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.416+0000 7efd1ea00640 1 -- 172.21.15.152:0/1184061696 wait complete. 2024-04-20T11:38:12.435 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-20T11:38:12.436 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:38:12.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.619+0000 7f63b1600640 1 Processor -- start 2024-04-20T11:38:12.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.620+0000 7f63b1600640 1 -- start start 2024-04-20T11:38:12.623 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.621+0000 7f63b1600640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f63ac13e530 0x7f63ac06ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:12.623 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.621+0000 7f63b1600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f63ac06f340 0x7f63ac140dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:12.623 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.621+0000 7f63b1600640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f63ac141310 0x7f63ac15efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:12.623 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.621+0000 7f63b1600640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f63ac141b00 con 0x7f63ac141310 2024-04-20T11:38:12.623 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.621+0000 7f63b1600640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f63ac0cfc50 con 0x7f63ac13e530 2024-04-20T11:38:12.623 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.621+0000 7f63b1600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f63ac0cf0a0 con 0x7f63ac06f340 2024-04-20T11:38:12.623 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.622+0000 7f63b0c00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f63ac141310 0x7f63ac15efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:12.623 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.622+0000 7f63b0c00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f63ac141310 0x7f63ac15efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:37466/0 (socket says 172.21.15.152:37466) 2024-04-20T11:38:12.623 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.622+0000 7f63b0c00640 1 -- 172.21.15.152:0/2427340507 learned_addr learned my addr 172.21.15.152:0/2427340507 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:38:12.623 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.622+0000 7f63abe00640 1 --2- 172.21.15.152:0/2427340507 >> v2:172.21.15.175:3300/0 conn(0x7f63ac13e530 0x7f63ac06ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:12.623 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.622+0000 7f63a3400640 1 --2- 172.21.15.152:0/2427340507 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f63ac06f340 0x7f63ac140dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:12.623 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.622+0000 7f63b0c00640 1 -- 172.21.15.152:0/2427340507 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f63ac06f340 msgr2=0x7f63ac140dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:12.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.622+0000 7f63b0c00640 1 --2- 172.21.15.152:0/2427340507 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f63ac06f340 0x7f63ac140dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:12.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.622+0000 7f63b0c00640 1 -- 172.21.15.152:0/2427340507 >> v2:172.21.15.175:3300/0 conn(0x7f63ac13e530 msgr2=0x7f63ac06ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:12.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.622+0000 7f63b0c00640 1 --2- 172.21.15.152:0/2427340507 >> v2:172.21.15.175:3300/0 conn(0x7f63ac13e530 0x7f63ac06ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:12.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.622+0000 7f63b0c00640 1 -- 172.21.15.152:0/2427340507 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63ac0d00e0 con 0x7f63ac141310 2024-04-20T11:38:12.625 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.623+0000 7f63b0c00640 1 --2- 172.21.15.152:0/2427340507 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f63ac141310 0x7f63ac15efd0 secure :-1 s=READY pgs=2401 cs=0 l=1 rev1=1 crypto rx=0x7f63ac070110 tx=0x7f63a402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43b72160c9055373 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:12.625 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.624+0000 7f63ab400640 1 -- 172.21.15.152:0/2427340507 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a402ca70 con 0x7f63ac141310 2024-04-20T11:38:12.625 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.624+0000 7f63ab400640 1 -- 172.21.15.152:0/2427340507 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63a402cc10 con 0x7f63ac141310 2024-04-20T11:38:12.625 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.624+0000 7f63ab400640 1 -- 172.21.15.152:0/2427340507 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a40343f0 con 0x7f63ac141310 2024-04-20T11:38:12.625 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.624+0000 7f63b1600640 1 -- 172.21.15.152:0/2427340507 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f63ac141310 msgr2=0x7f63ac15efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:12.625 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.624+0000 7f63b1600640 1 --2- 172.21.15.152:0/2427340507 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f63ac141310 0x7f63ac15efd0 secure :-1 s=READY pgs=2401 cs=0 l=1 rev1=1 crypto rx=0x7f63ac070110 tx=0x7f63a402bed0 comp rx=0 tx=0).stop 2024-04-20T11:38:12.625 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.624+0000 7f63b1600640 1 -- 172.21.15.152:0/2427340507 shutdown_connections 2024-04-20T11:38:12.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.624+0000 7f63b1600640 1 --2- 172.21.15.152:0/2427340507 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f63ac141310 0x7f63ac15efd0 unknown :-1 s=CLOSED pgs=2401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:12.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.624+0000 7f63b1600640 1 --2- 172.21.15.152:0/2427340507 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f63ac06f340 0x7f63ac140dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:12.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.624+0000 7f63b1600640 1 --2- 172.21.15.152:0/2427340507 >> v2:172.21.15.175:3300/0 conn(0x7f63ac13e530 0x7f63ac06ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:12.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.624+0000 7f63b1600640 1 -- 172.21.15.152:0/2427340507 >> 172.21.15.152:0/2427340507 conn(0x7f63ac067ae0 msgr2=0x7f63ac06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:12.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.625+0000 7f63b1600640 1 -- 172.21.15.152:0/2427340507 shutdown_connections 2024-04-20T11:38:12.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.625+0000 7f63b1600640 1 -- 172.21.15.152:0/2427340507 wait complete. 2024-04-20T11:38:12.627 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.626+0000 7f63b1600640 1 Processor -- start 2024-04-20T11:38:12.627 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.626+0000 7f63b1600640 1 -- start start 2024-04-20T11:38:12.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.627+0000 7f63b1600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63ac06f340 0x7f63ac0b2c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:12.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.627+0000 7f63b1600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f63ac13e530 0x7f63ac0aa8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:12.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.627+0000 7f63b1600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f63ac141310 0x7f63ac13d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:12.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.627+0000 7f63b1600640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f63ac0cf8e0 con 0x7f63ac141310 2024-04-20T11:38:12.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.627+0000 7f63b1600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f63ac0cf600 con 0x7f63ac06f340 2024-04-20T11:38:12.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.627+0000 7f63b1600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f63ac1417a0 con 0x7f63ac13e530 2024-04-20T11:38:12.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.628+0000 7f63b0c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f63ac141310 0x7f63ac13d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:12.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.628+0000 7f63b0c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f63ac141310 0x7f63ac13d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:37482/0 (socket says 172.21.15.152:37482) 2024-04-20T11:38:12.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.628+0000 7f63b0c00640 1 -- 172.21.15.152:0/3381733753 learned_addr learned my addr 172.21.15.152:0/3381733753 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:38:12.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.628+0000 7f63a3400640 1 --2- 172.21.15.152:0/3381733753 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f63ac13e530 0x7f63ac0aa8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:12.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.628+0000 7f63b0c00640 1 -- 172.21.15.152:0/3381733753 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f63ac13e530 msgr2=0x7f63ac0aa8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:12.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.628+0000 7f63b0c00640 1 --2- 172.21.15.152:0/3381733753 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f63ac13e530 0x7f63ac0aa8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:12.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.628+0000 7f63b0c00640 1 -- 172.21.15.152:0/3381733753 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63ac06f340 msgr2=0x7f63ac0b2c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:12.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.628+0000 7f63b0c00640 1 --2- 172.21.15.152:0/3381733753 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63ac06f340 0x7f63ac0b2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:12.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.628+0000 7f63b0c00640 1 -- 172.21.15.152:0/3381733753 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63ac0b1b80 con 0x7f63ac141310 2024-04-20T11:38:12.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.628+0000 7f63b0c00640 1 --2- 172.21.15.152:0/3381733753 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f63ac141310 0x7f63ac13d2b0 secure :-1 s=READY pgs=2402 cs=0 l=1 rev1=1 crypto rx=0x7f63a4009f40 tx=0x7f63a4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:12.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.629+0000 7f63a8c00640 1 -- 172.21.15.152:0/3381733753 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a4044020 con 0x7f63ac141310 2024-04-20T11:38:12.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.629+0000 7f63a8c00640 1 -- 172.21.15.152:0/3381733753 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63a4040690 con 0x7f63ac141310 2024-04-20T11:38:12.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.629+0000 7f63b1600640 1 -- 172.21.15.152:0/3381733753 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63ac0b1ec0 con 0x7f63ac141310 2024-04-20T11:38:12.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.629+0000 7f63a8c00640 1 -- 172.21.15.152:0/3381733753 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a4009510 con 0x7f63ac141310 2024-04-20T11:38:12.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.629+0000 7f63b1600640 1 -- 172.21.15.152:0/3381733753 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63ac0b2700 con 0x7f63ac141310 2024-04-20T11:38:12.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.630+0000 7f63b1600640 1 -- 172.21.15.152:0/3381733753 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63ac075e60 con 0x7f63ac141310 2024-04-20T11:38:12.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.631+0000 7f63a8c00640 1 -- 172.21.15.152:0/3381733753 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 27) ==== 92060+0+0 (secure 0 0 0) 0x7f63a4047020 con 0x7f63ac141310 2024-04-20T11:38:12.632 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.632+0000 7f63a8c00640 1 --2- 172.21.15.152:0/3381733753 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f638806d430 0x7f638806f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:12.632 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.632+0000 7f63a8c00640 1 -- 172.21.15.152:0/3381733753 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 5709+0+0 (secure 0 0 0) 0x7f63a40ad0e0 con 0x7f63ac141310 2024-04-20T11:38:12.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.632+0000 7f63abe00640 1 --2- 172.21.15.152:0/3381733753 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f638806d430 0x7f638806f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:12.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.634+0000 7f63abe00640 1 --2- 172.21.15.152:0/3381733753 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f638806d430 0x7f638806f8f0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f639c0098f0 tx=0x7f639c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:12.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.636+0000 7f63a8c00640 1 -- 172.21.15.152:0/3381733753 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f63a404e030 con 0x7f63ac141310 2024-04-20T11:38:12.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.881+0000 7f63b1600640 1 -- 172.21.15.152:0/3381733753 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f63ac15b360 con 0x7f63ac141310 2024-04-20T11:38:12.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.882+0000 7f63a8c00640 1 -- 172.21.15.152:0/3381733753 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 116 v116) ==== 95+0+5215 (secure 0 0 0) 0x7f63ac0b1ec0 con 0x7f63ac141310 2024-04-20T11:38:12.883 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:38:12.883 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":116,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6605,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6834/4020670267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6833","nonce":4020670267},{"type":"v1","addr":"172.21.15.152:6834","nonce":4020670267}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":109},{"gid":6614,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/352716681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":352716681},{"type":"v1","addr":"172.21.15.175:6833","nonce":352716681}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":110},{"gid":6617,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/2512108447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":2512108447},{"type":"v1","addr":"172.21.15.175:6835","nonce":2512108447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":110}],"filesystems":[{"mdsmap":{"epoch":115,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T11:37:47.506735+0000","modified":"2024-04-20T11:37:50.605372+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7339},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7339":{"gid":7339,"name":"c","rank":0,"incarnation":111,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6835/951802292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":951802292},{"type":"v1","addr":"172.21.15.152:6835","nonce":951802292}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7339,"qdb_cluster":[7339]},"id":13}]} 2024-04-20T11:38:12.884 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 116 2024-04-20T11:38:12.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.885+0000 7f63b1600640 1 -- 172.21.15.152:0/3381733753 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f638806d430 msgr2=0x7f638806f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:12.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.885+0000 7f63b1600640 1 --2- 172.21.15.152:0/3381733753 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f638806d430 0x7f638806f8f0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f639c0098f0 tx=0x7f639c008040 comp rx=0 tx=0).stop 2024-04-20T11:38:12.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.886+0000 7f63b1600640 1 -- 172.21.15.152:0/3381733753 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f63ac141310 msgr2=0x7f63ac13d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:12.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.886+0000 7f63b1600640 1 --2- 172.21.15.152:0/3381733753 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f63ac141310 0x7f63ac13d2b0 secure :-1 s=READY pgs=2402 cs=0 l=1 rev1=1 crypto rx=0x7f63a4009f40 tx=0x7f63a4004a40 comp rx=0 tx=0).stop 2024-04-20T11:38:12.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.886+0000 7f63b1600640 1 -- 172.21.15.152:0/3381733753 shutdown_connections 2024-04-20T11:38:12.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.886+0000 7f63b1600640 1 --2- 172.21.15.152:0/3381733753 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f638806d430 0x7f638806f8f0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:12.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.886+0000 7f63b1600640 1 --2- 172.21.15.152:0/3381733753 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f63ac141310 0x7f63ac13d2b0 unknown :-1 s=CLOSED pgs=2402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:12.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.886+0000 7f63b1600640 1 --2- 172.21.15.152:0/3381733753 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f63ac13e530 0x7f63ac0aa8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:12.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.886+0000 7f63b1600640 1 --2- 172.21.15.152:0/3381733753 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63ac06f340 0x7f63ac0b2c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:12.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.886+0000 7f63b1600640 1 -- 172.21.15.152:0/3381733753 >> 172.21.15.152:0/3381733753 conn(0x7f63ac067ae0 msgr2=0x7f63ac0b4c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:12.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.886+0000 7f63b1600640 1 -- 172.21.15.152:0/3381733753 shutdown_connections 2024-04-20T11:38:12.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:12.886+0000 7f63b1600640 1 -- 172.21.15.152:0/3381733753 wait complete. 2024-04-20T11:38:12.906 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:38:13.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.105+0000 7f7b97200640 1 Processor -- start 2024-04-20T11:38:13.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.106+0000 7f7b97200640 1 -- start start 2024-04-20T11:38:13.108 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.107+0000 7f7b97200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f7b90145f60 0x7f7b9006ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:13.108 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.107+0000 7f7b97200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f7b900cd0d0 0x7f7b900cd580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:13.108 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.107+0000 7f7b97200640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b90146340 con 0x7f7b90071fe0 2024-04-20T11:38:13.108 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.107+0000 7f7b97200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f7b9006e9f0 con 0x7f7b90145f60 2024-04-20T11:38:13.108 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.107+0000 7f7b97200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f7b9006f3c0 con 0x7f7b900cd0d0 2024-04-20T11:38:13.109 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.108+0000 7f7b8fe00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f7b90145f60 0x7f7b9006ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:13.109 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.108+0000 7f7b8fe00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f7b90145f60 0x7f7b9006ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:37496/0 (socket says 172.21.15.152:37496) 2024-04-20T11:38:13.109 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.108+0000 7f7b8fe00640 1 -- 172.21.15.152:0/832060694 learned_addr learned my addr 172.21.15.152:0/832060694 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:38:13.109 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.108+0000 7f7b87400640 1 --2- 172.21.15.152:0/832060694 >> v2:172.21.15.175:3300/0 conn(0x7f7b900cd0d0 0x7f7b900cd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:13.109 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.108+0000 7f7b8fe00640 1 -- 172.21.15.152:0/832060694 >> v2:172.21.15.175:3300/0 conn(0x7f7b900cd0d0 msgr2=0x7f7b900cd580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:13.109 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.108+0000 7f7b8fe00640 1 --2- 172.21.15.152:0/832060694 >> v2:172.21.15.175:3300/0 conn(0x7f7b900cd0d0 0x7f7b900cd580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:13.110 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.108+0000 7f7b8fe00640 1 -- 172.21.15.152:0/832060694 >> v1:172.21.15.152:6789/0 conn(0x7f7b90071fe0 legacy=0x7f7b90072470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:13.110 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.108+0000 7f7b8fe00640 1 -- 172.21.15.152:0/832060694 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b90072d50 con 0x7f7b90145f60 2024-04-20T11:38:13.110 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.108+0000 7f7b87400640 1 --2- 172.21.15.152:0/832060694 >> v2:172.21.15.175:3300/0 conn(0x7f7b900cd0d0 0x7f7b900cd580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:38:13.110 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.109+0000 7f7b8fe00640 1 --2- 172.21.15.152:0/832060694 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7b90145f60 0x7f7b9006ee80 secure :-1 s=READY pgs=2404 cs=0 l=1 rev1=1 crypto rx=0x7f7b9006eb70 tx=0x7f7b8002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ffa52c856b9f081 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:13.110 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.110+0000 7f7b8f400640 1 -- 172.21.15.152:0/832060694 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b80034070 con 0x7f7b90145f60 2024-04-20T11:38:13.110 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.110+0000 7f7b8f400640 1 -- 172.21.15.152:0/832060694 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b80030860 con 0x7f7b90145f60 2024-04-20T11:38:13.111 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.110+0000 7f7b8f400640 1 -- 172.21.15.152:0/832060694 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b80030c30 con 0x7f7b90145f60 2024-04-20T11:38:13.111 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.111+0000 7f7b97200640 1 -- 172.21.15.152:0/832060694 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7b90145f60 msgr2=0x7f7b9006ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:13.112 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.111+0000 7f7b97200640 1 --2- 172.21.15.152:0/832060694 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7b90145f60 0x7f7b9006ee80 secure :-1 s=READY pgs=2404 cs=0 l=1 rev1=1 crypto rx=0x7f7b9006eb70 tx=0x7f7b8002b9a0 comp rx=0 tx=0).stop 2024-04-20T11:38:13.112 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.111+0000 7f7b97200640 1 -- 172.21.15.152:0/832060694 shutdown_connections 2024-04-20T11:38:13.112 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.111+0000 7f7b97200640 1 --2- 172.21.15.152:0/832060694 >> v2:172.21.15.175:3300/0 conn(0x7f7b900cd0d0 0x7f7b900cd580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:13.112 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.111+0000 7f7b97200640 1 --2- 172.21.15.152:0/832060694 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7b90145f60 0x7f7b9006ee80 unknown :-1 s=CLOSED pgs=2404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:13.112 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.111+0000 7f7b97200640 1 -- 172.21.15.152:0/832060694 >> 172.21.15.152:0/832060694 conn(0x7f7b90067cf0 msgr2=0x7f7b9006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:13.112 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.111+0000 7f7b97200640 1 -- 172.21.15.152:0/832060694 shutdown_connections 2024-04-20T11:38:13.112 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.111+0000 7f7b97200640 1 -- 172.21.15.152:0/832060694 wait complete. 2024-04-20T11:38:13.113 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.113+0000 7f7b97200640 1 Processor -- start 2024-04-20T11:38:13.113 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.113+0000 7f7b97200640 1 -- start start 2024-04-20T11:38:13.114 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.114+0000 7f7b97200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7b90071fe0 0x7f7b901e4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:13.114 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.114+0000 7f7b97200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b900cd0d0 0x7f7b90144f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:13.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.114+0000 7f7b97200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7b90145f60 0x7f7b9015f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:13.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.114+0000 7f7b97200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f7b9006e680 con 0x7f7b90145f60 2024-04-20T11:38:13.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.114+0000 7f7b97200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7b9006e3a0 con 0x7f7b900cd0d0 2024-04-20T11:38:13.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.114+0000 7f7b97200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7b900cf290 con 0x7f7b90071fe0 2024-04-20T11:38:13.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.114+0000 7f7b95200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7b90145f60 0x7f7b9015f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:13.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.114+0000 7f7b95200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7b90145f60 0x7f7b9015f8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:37498/0 (socket says 172.21.15.152:37498) 2024-04-20T11:38:13.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.114+0000 7f7b95200640 1 -- 172.21.15.152:0/225547764 learned_addr learned my addr 172.21.15.152:0/225547764 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:38:13.116 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.114+0000 7f7b87400640 1 --2- 172.21.15.152:0/225547764 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b900cd0d0 0x7f7b90144f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:13.116 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.114+0000 7f7b8fe00640 1 --2- 172.21.15.152:0/225547764 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7b90071fe0 0x7f7b901e4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:13.116 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.114+0000 7f7b95200640 1 -- 172.21.15.152:0/225547764 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7b90071fe0 msgr2=0x7f7b901e4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:13.116 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.114+0000 7f7b95200640 1 --2- 172.21.15.152:0/225547764 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7b90071fe0 0x7f7b901e4eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:13.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.114+0000 7f7b95200640 1 -- 172.21.15.152:0/225547764 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b900cd0d0 msgr2=0x7f7b90144f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:13.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.114+0000 7f7b95200640 1 --2- 172.21.15.152:0/225547764 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b900cd0d0 0x7f7b90144f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:13.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.114+0000 7f7b95200640 1 -- 172.21.15.152:0/225547764 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b901e55b0 con 0x7f7b90145f60 2024-04-20T11:38:13.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.114+0000 7f7b8fe00640 1 --2- 172.21.15.152:0/225547764 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7b90071fe0 0x7f7b901e4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:38:13.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.115+0000 7f7b95200640 1 --2- 172.21.15.152:0/225547764 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7b90145f60 0x7f7b9015f8c0 secure :-1 s=READY pgs=2405 cs=0 l=1 rev1=1 crypto rx=0x7f7b8800c7d0 tx=0x7f7b8800cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:13.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.115+0000 7f7b8cc00640 1 -- 172.21.15.152:0/225547764 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b88003b70 con 0x7f7b90145f60 2024-04-20T11:38:13.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.115+0000 7f7b8cc00640 1 -- 172.21.15.152:0/225547764 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b88003d30 con 0x7f7b90145f60 2024-04-20T11:38:13.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.115+0000 7f7b97200640 1 -- 172.21.15.152:0/225547764 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b901e5890 con 0x7f7b90145f60 2024-04-20T11:38:13.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.115+0000 7f7b8cc00640 1 -- 172.21.15.152:0/225547764 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b8800aa90 con 0x7f7b90145f60 2024-04-20T11:38:13.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.115+0000 7f7b97200640 1 -- 172.21.15.152:0/225547764 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b901f46d0 con 0x7f7b90145f60 2024-04-20T11:38:13.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.117+0000 7f7b97200640 1 -- 172.21.15.152:0/225547764 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b900689f0 con 0x7f7b90145f60 2024-04-20T11:38:13.118 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.118+0000 7f7b8cc00640 1 -- 172.21.15.152:0/225547764 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 27) ==== 92060+0+0 (secure 0 0 0) 0x7f7b8800ac30 con 0x7f7b90145f60 2024-04-20T11:38:13.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.118+0000 7f7b8cc00640 1 --2- 172.21.15.152:0/225547764 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f7b6c06d430 0x7f7b6c06f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:13.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.119+0000 7f7b8cc00640 1 -- 172.21.15.152:0/225547764 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 5709+0+0 (secure 0 0 0) 0x7f7b8808a680 con 0x7f7b90145f60 2024-04-20T11:38:13.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.119+0000 7f7b8fe00640 1 --2- 172.21.15.152:0/225547764 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f7b6c06d430 0x7f7b6c06f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:13.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.120+0000 7f7b8fe00640 1 --2- 172.21.15.152:0/225547764 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f7b6c06d430 0x7f7b6c06f8f0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f7b80036000 tx=0x7f7b8003b040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:13.124 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.123+0000 7f7b8cc00640 1 -- 172.21.15.152:0/225547764 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f7b8805a310 con 0x7f7b90145f60 2024-04-20T11:38:13.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.350+0000 7f7b97200640 1 -- 172.21.15.152:0/225547764 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7b9006c6e0 con 0x7f7b90145f60 2024-04-20T11:38:13.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.351+0000 7f7b8cc00640 1 -- 172.21.15.152:0/225547764 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 116 v116) ==== 95+0+5215 (secure 0 0 0) 0x7f7b88059c10 con 0x7f7b90145f60 2024-04-20T11:38:13.352 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:38:13.352 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":116,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6605,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6834/4020670267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6833","nonce":4020670267},{"type":"v1","addr":"172.21.15.152:6834","nonce":4020670267}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":109},{"gid":6614,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/352716681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":352716681},{"type":"v1","addr":"172.21.15.175:6833","nonce":352716681}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":110},{"gid":6617,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/2512108447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":2512108447},{"type":"v1","addr":"172.21.15.175:6835","nonce":2512108447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":110}],"filesystems":[{"mdsmap":{"epoch":115,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T11:37:47.506735+0000","modified":"2024-04-20T11:37:50.605372+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7339},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7339":{"gid":7339,"name":"c","rank":0,"incarnation":111,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6835/951802292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":951802292},{"type":"v1","addr":"172.21.15.152:6835","nonce":951802292}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7339,"qdb_cluster":[7339]},"id":13}]} 2024-04-20T11:38:13.352 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 116 2024-04-20T11:38:13.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.353+0000 7f7b97200640 1 -- 172.21.15.152:0/225547764 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f7b6c06d430 msgr2=0x7f7b6c06f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:13.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.353+0000 7f7b97200640 1 --2- 172.21.15.152:0/225547764 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f7b6c06d430 0x7f7b6c06f8f0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f7b80036000 tx=0x7f7b8003b040 comp rx=0 tx=0).stop 2024-04-20T11:38:13.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.353+0000 7f7b97200640 1 -- 172.21.15.152:0/225547764 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7b90145f60 msgr2=0x7f7b9015f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:13.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.353+0000 7f7b97200640 1 --2- 172.21.15.152:0/225547764 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7b90145f60 0x7f7b9015f8c0 secure :-1 s=READY pgs=2405 cs=0 l=1 rev1=1 crypto rx=0x7f7b8800c7d0 tx=0x7f7b8800cc90 comp rx=0 tx=0).stop 2024-04-20T11:38:13.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.353+0000 7f7b97200640 1 -- 172.21.15.152:0/225547764 shutdown_connections 2024-04-20T11:38:13.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.353+0000 7f7b97200640 1 --2- 172.21.15.152:0/225547764 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f7b6c06d430 0x7f7b6c06f8f0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:13.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.353+0000 7f7b97200640 1 --2- 172.21.15.152:0/225547764 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7b90145f60 0x7f7b9015f8c0 unknown :-1 s=CLOSED pgs=2405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:13.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.353+0000 7f7b97200640 1 --2- 172.21.15.152:0/225547764 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b900cd0d0 0x7f7b90144f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:13.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.353+0000 7f7b97200640 1 --2- 172.21.15.152:0/225547764 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7b90071fe0 0x7f7b901e4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:13.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.353+0000 7f7b97200640 1 -- 172.21.15.152:0/225547764 >> 172.21.15.152:0/225547764 conn(0x7f7b90067cf0 msgr2=0x7f7b900cf5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:13.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.354+0000 7f7b97200640 1 -- 172.21.15.152:0/225547764 shutdown_connections 2024-04-20T11:38:13.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.354+0000 7f7b97200640 1 -- 172.21.15.152:0/225547764 wait complete. 2024-04-20T11:38:13.372 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:38:13.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.565+0000 7f822d200640 1 Processor -- start 2024-04-20T11:38:13.567 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.566+0000 7f822d200640 1 -- start start 2024-04-20T11:38:13.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.567+0000 7f822d200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f822815eb80 0x7f82280b79a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:13.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.567+0000 7f822d200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f82280ba520 0x7f82280d28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:13.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.567+0000 7f822d200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f82280d2e30 0x7f82280d1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:13.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.567+0000 7f822d200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f82280b8330 con 0x7f82280d2e30 2024-04-20T11:38:13.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.567+0000 7f822d200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f82280b8610 con 0x7f822815eb80 2024-04-20T11:38:13.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.567+0000 7f822d200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f82280b88f0 con 0x7f82280ba520 2024-04-20T11:38:13.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.567+0000 7f8226800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f82280d2e30 0x7f82280d1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:13.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.567+0000 7f8226800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f82280d2e30 0x7f82280d1810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:37510/0 (socket says 172.21.15.152:37510) 2024-04-20T11:38:13.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.567+0000 7f8226800640 1 -- 172.21.15.152:0/2674950691 learned_addr learned my addr 172.21.15.152:0/2674950691 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:38:13.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.568+0000 7f8225e00640 1 --2- 172.21.15.152:0/2674950691 >> v2:172.21.15.175:3300/0 conn(0x7f822815eb80 0x7f82280b79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:13.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.568+0000 7f8225400640 1 --2- 172.21.15.152:0/2674950691 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f82280ba520 0x7f82280d28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:13.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.568+0000 7f8226800640 1 -- 172.21.15.152:0/2674950691 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f82280ba520 msgr2=0x7f82280d28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:13.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.568+0000 7f8226800640 1 --2- 172.21.15.152:0/2674950691 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f82280ba520 0x7f82280d28f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:13.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.568+0000 7f8226800640 1 -- 172.21.15.152:0/2674950691 >> v2:172.21.15.175:3300/0 conn(0x7f822815eb80 msgr2=0x7f82280b79a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:13.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.568+0000 7f8226800640 1 --2- 172.21.15.152:0/2674950691 >> v2:172.21.15.175:3300/0 conn(0x7f822815eb80 0x7f82280b79a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:13.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.568+0000 7f8226800640 1 -- 172.21.15.152:0/2674950691 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82280b8bd0 con 0x7f82280d2e30 2024-04-20T11:38:13.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.568+0000 7f8225400640 1 --2- 172.21.15.152:0/2674950691 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f82280ba520 0x7f82280d28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:38:13.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.568+0000 7f8225e00640 1 --2- 172.21.15.152:0/2674950691 >> v2:172.21.15.175:3300/0 conn(0x7f822815eb80 0x7f82280b79a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:38:13.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.569+0000 7f8226800640 1 --2- 172.21.15.152:0/2674950691 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f82280d2e30 0x7f82280d1810 secure :-1 s=READY pgs=2406 cs=0 l=1 rev1=1 crypto rx=0x7f82280b6b00 tx=0x7f822002be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e30cc02a2808c52c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:13.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.569+0000 7f8224a00640 1 -- 172.21.15.152:0/2674950691 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f822002c9f0 con 0x7f82280d2e30 2024-04-20T11:38:13.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.569+0000 7f8224a00640 1 -- 172.21.15.152:0/2674950691 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f822002cb90 con 0x7f82280d2e30 2024-04-20T11:38:13.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.569+0000 7f8224a00640 1 -- 172.21.15.152:0/2674950691 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82200343f0 con 0x7f82280d2e30 2024-04-20T11:38:13.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.570+0000 7f822d200640 1 -- 172.21.15.152:0/2674950691 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f82280d2e30 msgr2=0x7f82280d1810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:13.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.570+0000 7f822d200640 1 --2- 172.21.15.152:0/2674950691 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f82280d2e30 0x7f82280d1810 secure :-1 s=READY pgs=2406 cs=0 l=1 rev1=1 crypto rx=0x7f82280b6b00 tx=0x7f822002be50 comp rx=0 tx=0).stop 2024-04-20T11:38:13.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.570+0000 7f822d200640 1 -- 172.21.15.152:0/2674950691 shutdown_connections 2024-04-20T11:38:13.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.570+0000 7f822d200640 1 --2- 172.21.15.152:0/2674950691 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f82280d2e30 0x7f82280d1810 unknown :-1 s=CLOSED pgs=2406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:13.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.570+0000 7f822d200640 1 --2- 172.21.15.152:0/2674950691 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f82280ba520 0x7f82280d28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:13.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.570+0000 7f822d200640 1 --2- 172.21.15.152:0/2674950691 >> v2:172.21.15.175:3300/0 conn(0x7f822815eb80 0x7f82280b79a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:13.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.570+0000 7f822d200640 1 -- 172.21.15.152:0/2674950691 >> 172.21.15.152:0/2674950691 conn(0x7f8228067ae0 msgr2=0x7f8228067ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:13.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.570+0000 7f822d200640 1 -- 172.21.15.152:0/2674950691 shutdown_connections 2024-04-20T11:38:13.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.570+0000 7f822d200640 1 -- 172.21.15.152:0/2674950691 wait complete. 2024-04-20T11:38:13.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.572+0000 7f822d200640 1 Processor -- start 2024-04-20T11:38:13.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.572+0000 7f822d200640 1 -- start start 2024-04-20T11:38:13.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.573+0000 7f822d200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f82280ba520 0x7f8228154b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:13.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.573+0000 7f822d200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f82280d2e30 0x7f82281f7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:13.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.573+0000 7f822d200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f822815eb80 0x7f82281f8210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:13.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.573+0000 7f822d200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f82280d1d50 con 0x7f82280d2e30 2024-04-20T11:38:13.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.573+0000 7f822d200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f82280b80f0 con 0x7f822815eb80 2024-04-20T11:38:13.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.573+0000 7f822d200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f82280d1ed0 con 0x7f82280ba520 2024-04-20T11:38:13.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.573+0000 7f8225400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f82280d2e30 0x7f82281f7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:13.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.573+0000 7f8225400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f82280d2e30 0x7f82281f7cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:37516/0 (socket says 172.21.15.152:37516) 2024-04-20T11:38:13.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.573+0000 7f8225400640 1 -- 172.21.15.152:0/2029587278 learned_addr learned my addr 172.21.15.152:0/2029587278 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:38:13.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.573+0000 7f8225400640 1 -- 172.21.15.152:0/2029587278 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f82280ba520 msgr2=0x7f8228154b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:13.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.573+0000 7f8225400640 1 --2- 172.21.15.152:0/2029587278 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f82280ba520 0x7f8228154b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:13.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.573+0000 7f8225400640 1 -- 172.21.15.152:0/2029587278 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f822815eb80 msgr2=0x7f82281f8210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:13.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.573+0000 7f8225400640 1 --2- 172.21.15.152:0/2029587278 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f822815eb80 0x7f82281f8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:13.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.573+0000 7f8225400640 1 -- 172.21.15.152:0/2029587278 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8228160050 con 0x7f82280d2e30 2024-04-20T11:38:13.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.574+0000 7f8225400640 1 --2- 172.21.15.152:0/2029587278 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f82280d2e30 0x7f82281f7cd0 secure :-1 s=READY pgs=2407 cs=0 l=1 rev1=1 crypto rx=0x7f821000c7c0 tx=0x7f821000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:13.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.574+0000 7f821e000640 1 -- 172.21.15.152:0/2029587278 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8210015070 con 0x7f82280d2e30 2024-04-20T11:38:13.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.574+0000 7f821e000640 1 -- 172.21.15.152:0/2029587278 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8210007a30 con 0x7f82280d2e30 2024-04-20T11:38:13.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.574+0000 7f821e000640 1 -- 172.21.15.152:0/2029587278 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8210007e00 con 0x7f82280d2e30 2024-04-20T11:38:13.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.574+0000 7f822d200640 1 -- 172.21.15.152:0/2029587278 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8228144d50 con 0x7f82280d2e30 2024-04-20T11:38:13.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.574+0000 7f822d200640 1 -- 172.21.15.152:0/2029587278 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8228157a60 con 0x7f82280d2e30 2024-04-20T11:38:13.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.575+0000 7f822d200640 1 -- 172.21.15.152:0/2029587278 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82280689f0 con 0x7f82280d2e30 2024-04-20T11:38:13.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.576+0000 7f821e000640 1 -- 172.21.15.152:0/2029587278 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 27) ==== 92060+0+0 (secure 0 0 0) 0x7f8210018070 con 0x7f82280d2e30 2024-04-20T11:38:13.580 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.577+0000 7f821e000640 1 --2- 172.21.15.152:0/2029587278 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f820406d430 0x7f820406f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:13.580 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.577+0000 7f821e000640 1 -- 172.21.15.152:0/2029587278 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 5709+0+0 (secure 0 0 0) 0x7f821008a6e0 con 0x7f82280d2e30 2024-04-20T11:38:13.580 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.578+0000 7f8225e00640 1 --2- 172.21.15.152:0/2029587278 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f820406d430 0x7f820406f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:13.580 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.579+0000 7f8225e00640 1 --2- 172.21.15.152:0/2029587278 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f820406d430 0x7f820406f8f0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f8218006fd0 tx=0x7f8218008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:13.582 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.581+0000 7f821e000640 1 -- 172.21.15.152:0/2029587278 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f821005a310 con 0x7f82280d2e30 2024-04-20T11:38:13.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.782+0000 7f822d200640 1 -- 172.21.15.152:0/2029587278 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f82280d5660 con 0x7f82280d2e30 2024-04-20T11:38:13.785 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.785+0000 7f821e000640 1 -- 172.21.15.152:0/2029587278 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v167) ==== 74+0+16743 (secure 0 0 0) 0x7f8210059c10 con 0x7f82280d2e30 2024-04-20T11:38:13.786 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:38:13.786 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":167,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:37:49.091768+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:37:45.247938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"166","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2024-04-20T11:37:46.503171+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"167","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:38:13.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.787+0000 7f822d200640 1 -- 172.21.15.152:0/2029587278 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f820406d430 msgr2=0x7f820406f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:13.789 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.787+0000 7f822d200640 1 --2- 172.21.15.152:0/2029587278 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f820406d430 0x7f820406f8f0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f8218006fd0 tx=0x7f8218008040 comp rx=0 tx=0).stop 2024-04-20T11:38:13.789 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.787+0000 7f822d200640 1 -- 172.21.15.152:0/2029587278 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f82280d2e30 msgr2=0x7f82281f7cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:13.789 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.787+0000 7f822d200640 1 --2- 172.21.15.152:0/2029587278 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f82280d2e30 0x7f82281f7cd0 secure :-1 s=READY pgs=2407 cs=0 l=1 rev1=1 crypto rx=0x7f821000c7c0 tx=0x7f821000cc80 comp rx=0 tx=0).stop 2024-04-20T11:38:13.789 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.788+0000 7f822d200640 1 -- 172.21.15.152:0/2029587278 shutdown_connections 2024-04-20T11:38:13.789 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.788+0000 7f822d200640 1 --2- 172.21.15.152:0/2029587278 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f820406d430 0x7f820406f8f0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:13.789 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.788+0000 7f822d200640 1 --2- 172.21.15.152:0/2029587278 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f822815eb80 0x7f82281f8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:13.789 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.788+0000 7f822d200640 1 --2- 172.21.15.152:0/2029587278 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f82280d2e30 0x7f82281f7cd0 unknown :-1 s=CLOSED pgs=2407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:13.789 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.788+0000 7f822d200640 1 --2- 172.21.15.152:0/2029587278 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f82280ba520 0x7f8228154b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:13.789 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.788+0000 7f822d200640 1 -- 172.21.15.152:0/2029587278 >> 172.21.15.152:0/2029587278 conn(0x7f8228067ae0 msgr2=0x7f82280d4b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:13.789 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.788+0000 7f822d200640 1 -- 172.21.15.152:0/2029587278 shutdown_connections 2024-04-20T11:38:13.789 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:13.788+0000 7f822d200640 1 -- 172.21.15.152:0/2029587278 wait complete. 2024-04-20T11:38:13.808 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-20T11:38:14.016 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.016+0000 7fa4bf200640 1 Processor -- start 2024-04-20T11:38:14.017 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.016+0000 7fa4bf200640 1 -- start start 2024-04-20T11:38:14.018 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.017+0000 7fa4bf200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa4c00d4ec0 0x7fa4c00d0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:14.018 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.017+0000 7fa4bf200640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4c00d26a0 con 0x7fa4c0161d30 2024-04-20T11:38:14.018 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.017+0000 7fa4bf200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4c00d53c0 con 0x7fa4c00d2b80 2024-04-20T11:38:14.018 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.017+0000 7fa4bf200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa4c00d0540 con 0x7fa4c00d4ec0 2024-04-20T11:38:14.018 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.017+0000 7fa4bde00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fa4c0161d30 0x7fa4c00d15e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:34426/0 (socket says 172.21.15.152:34426) 2024-04-20T11:38:14.018 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.017+0000 7fa4bde00640 1 -- 172.21.15.152:0/809271107 learned_addr learned my addr 172.21.15.152:0/809271107 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:38:14.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.018+0000 7fa4bca00640 1 -- 172.21.15.152:0/809271107 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2863676879 0 0) 0x7fa4c00d26a0 con 0x7fa4c0161d30 2024-04-20T11:38:14.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.018+0000 7fa4bd400640 1 --2- 172.21.15.152:0/809271107 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa4c00d4ec0 0x7fa4c00d0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:14.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.018+0000 7fa4bca00640 1 -- 172.21.15.152:0/809271107 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa4a0003520 con 0x7fa4c0161d30 2024-04-20T11:38:14.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.018+0000 7fa4bca00640 1 -- 172.21.15.152:0/809271107 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 895195264 0 0) 0x7fa4a0003520 con 0x7fa4c0161d30 2024-04-20T11:38:14.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.018+0000 7fa4bca00640 1 -- 172.21.15.152:0/809271107 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa4c00d4ec0 msgr2=0x7fa4c00d0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:14.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.018+0000 7fa4bca00640 1 --2- 172.21.15.152:0/809271107 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa4c00d4ec0 0x7fa4c00d0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:14.021 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.018+0000 7fa4bca00640 1 -- 172.21.15.152:0/809271107 >> v1:172.21.15.175:6789/0 conn(0x7fa4c00d2b80 legacy=0x7fa4c006c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:14.021 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.018+0000 7fa4bca00640 1 -- 172.21.15.152:0/809271107 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4c00cf520 con 0x7fa4c0161d30 2024-04-20T11:38:14.021 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.018+0000 7fa4bca00640 1 -- 172.21.15.152:0/809271107 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7fa4b0001670 con 0x7fa4c0161d30 2024-04-20T11:38:14.021 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.018+0000 7fa4bd400640 1 --2- 172.21.15.152:0/809271107 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa4c00d4ec0 0x7fa4c00d0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:38:14.021 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.018+0000 7fa4bca00640 1 -- 172.21.15.152:0/809271107 >> v1:172.21.15.152:6789/0 conn(0x7fa4c0161d30 legacy=0x7fa4c00d15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:14.021 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.018+0000 7fa4bca00640 1 --2- 172.21.15.152:0/809271107 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa4a00039e0 0x7fa4a0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:14.021 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.018+0000 7fa4bca00640 1 --2- 172.21.15.152:0/809271107 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa4a0004800 0x7fa4a0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:14.021 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.018+0000 7fa4bca00640 1 --2- 172.21.15.152:0/809271107 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa4c00d4ec0 0x7fa4a00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:14.021 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.019+0000 7fa4bca00640 1 -- 172.21.15.152:0/809271107 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fa4c00d0540 con 0x7fa4a00039e0 2024-04-20T11:38:14.021 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.019+0000 7fa4bca00640 1 -- 172.21.15.152:0/809271107 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a000bac0 con 0x7fa4a0004800 2024-04-20T11:38:14.021 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.019+0000 7fa4bca00640 1 -- 172.21.15.152:0/809271107 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa4a000bcc0 con 0x7fa4c00d4ec0 2024-04-20T11:38:14.021 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.019+0000 7fa4bde00640 1 --2- 172.21.15.152:0/809271107 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa4a00039e0 0x7fa4a0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:14.021 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.019+0000 7fa4bde00640 1 -- 172.21.15.152:0/809271107 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa4c00d4ec0 msgr2=0x7fa4a00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:14.021 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.019+0000 7fa4bde00640 1 --2- 172.21.15.152:0/809271107 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa4c00d4ec0 0x7fa4a00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:14.021 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.019+0000 7fa4bde00640 1 -- 172.21.15.152:0/809271107 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa4a0004800 msgr2=0x7fa4a0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:14.022 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.019+0000 7fa4bd400640 1 --2- 172.21.15.152:0/809271107 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa4a0004800 0x7fa4a0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:14.022 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.019+0000 7fa4bde00640 1 --2- 172.21.15.152:0/809271107 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa4a0004800 0x7fa4a0004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:14.022 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.019+0000 7fa4bde00640 1 -- 172.21.15.152:0/809271107 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4a0003520 con 0x7fa4a00039e0 2024-04-20T11:38:14.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.020+0000 7fa4bde00640 1 --2- 172.21.15.152:0/809271107 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa4a00039e0 0x7fa4a0003e00 secure :-1 s=READY pgs=2409 cs=0 l=1 rev1=1 crypto rx=0x7fa4b000c240 tx=0x7fa4a0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23c9d2ce9f374333 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:14.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.021+0000 7fa4bca00640 1 -- 172.21.15.152:0/809271107 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4b0007d10 con 0x7fa4a00039e0 2024-04-20T11:38:14.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.021+0000 7fa4bca00640 1 -- 172.21.15.152:0/809271107 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4b0030070 con 0x7fa4a00039e0 2024-04-20T11:38:14.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.021+0000 7fa4bca00640 1 -- 172.21.15.152:0/809271107 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4b002cd60 con 0x7fa4a00039e0 2024-04-20T11:38:14.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.021+0000 7fa4bf200640 1 -- 172.21.15.152:0/809271107 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa4a00039e0 msgr2=0x7fa4a0003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:14.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.021+0000 7fa4bf200640 1 --2- 172.21.15.152:0/809271107 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa4a00039e0 0x7fa4a0003e00 secure :-1 s=READY pgs=2409 cs=0 l=1 rev1=1 crypto rx=0x7fa4b000c240 tx=0x7fa4a0005830 comp rx=0 tx=0).stop 2024-04-20T11:38:14.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.021+0000 7fa4be800640 1 -- 172.21.15.152:0/809271107 reap_dead start 2024-04-20T11:38:14.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.021+0000 7fa4bf200640 1 -- 172.21.15.152:0/809271107 shutdown_connections 2024-04-20T11:38:14.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.021+0000 7fa4bf200640 1 -- 172.21.15.152:0/809271107 >> 172.21.15.152:0/809271107 conn(0x7fa4c008bfd0 msgr2=0x7fa4c00d45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:14.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.021+0000 7fa4bf200640 1 -- 172.21.15.152:0/809271107 shutdown_connections 2024-04-20T11:38:14.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.021+0000 7fa4bf200640 1 -- 172.21.15.152:0/809271107 wait complete. 2024-04-20T11:38:14.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.023+0000 7fa4bf200640 1 Processor -- start 2024-04-20T11:38:14.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.023+0000 7fa4bf200640 1 -- start start 2024-04-20T11:38:14.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.024+0000 7fa4bf200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa4a00039e0 0x7fa4c01f5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:14.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.024+0000 7fa4bf200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa4c00cd4f0 0x7fa4c00cd930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:14.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.024+0000 7fa4bf200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa4c00b7c60 0x7fa4c00b80e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:14.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.024+0000 7fa4bf200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fa4c00cf1b0 con 0x7fa4c00b7c60 2024-04-20T11:38:14.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.024+0000 7fa4bf200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa4c00cfb70 con 0x7fa4c00cd4f0 2024-04-20T11:38:14.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.024+0000 7fa4bf200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa4c0162110 con 0x7fa4a00039e0 2024-04-20T11:38:14.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.024+0000 7fa4be800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa4c00b7c60 0x7fa4c00b80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:14.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.024+0000 7fa4be800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa4c00b7c60 0x7fa4c00b80e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:37544/0 (socket says 172.21.15.152:37544) 2024-04-20T11:38:14.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.024+0000 7fa4be800640 1 -- 172.21.15.152:0/4277385822 learned_addr learned my addr 172.21.15.152:0/4277385822 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:38:14.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.025+0000 7fa4bde00640 1 --2- 172.21.15.152:0/4277385822 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa4a00039e0 0x7fa4c01f5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:14.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.025+0000 7fa4bd400640 1 --2- 172.21.15.152:0/4277385822 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa4c00cd4f0 0x7fa4c00cd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:14.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.025+0000 7fa4be800640 1 -- 172.21.15.152:0/4277385822 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa4a00039e0 msgr2=0x7fa4c01f5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:14.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.025+0000 7fa4be800640 1 --2- 172.21.15.152:0/4277385822 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa4a00039e0 0x7fa4c01f5ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:14.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.025+0000 7fa4be800640 1 -- 172.21.15.152:0/4277385822 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa4c00cd4f0 msgr2=0x7fa4c00cd930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:14.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.025+0000 7fa4be800640 1 --2- 172.21.15.152:0/4277385822 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa4c00cd4f0 0x7fa4c00cd930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:14.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.025+0000 7fa4be800640 1 -- 172.21.15.152:0/4277385822 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4c00cb670 con 0x7fa4c00b7c60 2024-04-20T11:38:14.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.025+0000 7fa4be800640 1 --2- 172.21.15.152:0/4277385822 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa4c00b7c60 0x7fa4c00b80e0 secure :-1 s=READY pgs=2410 cs=0 l=1 rev1=1 crypto rx=0x7fa4c00ba650 tx=0x7fa4b800c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:14.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.026+0000 7fa4b6000640 1 -- 172.21.15.152:0/4277385822 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4b80074e0 con 0x7fa4c00b7c60 2024-04-20T11:38:14.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.026+0000 7fa4b6000640 1 -- 172.21.15.152:0/4277385822 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4b8004020 con 0x7fa4c00b7c60 2024-04-20T11:38:14.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.026+0000 7fa4b6000640 1 -- 172.21.15.152:0/4277385822 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4b8004410 con 0x7fa4c00b7c60 2024-04-20T11:38:14.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.026+0000 7fa4bf200640 1 -- 172.21.15.152:0/4277385822 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4c00b8a00 con 0x7fa4c00b7c60 2024-04-20T11:38:14.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.026+0000 7fa4bf200640 1 -- 172.21.15.152:0/4277385822 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4c00cbc10 con 0x7fa4c00b7c60 2024-04-20T11:38:14.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.027+0000 7fa4bf200640 1 -- 172.21.15.152:0/4277385822 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4c0075e60 con 0x7fa4c00b7c60 2024-04-20T11:38:14.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.028+0000 7fa4b6000640 1 -- 172.21.15.152:0/4277385822 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 27) ==== 92060+0+0 (secure 0 0 0) 0x7fa4b80057c0 con 0x7fa4c00b7c60 2024-04-20T11:38:14.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.029+0000 7fa4b6000640 1 --2- 172.21.15.152:0/4277385822 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fa49c06d460 0x7fa49c06f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:14.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.029+0000 7fa4b6000640 1 -- 172.21.15.152:0/4277385822 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 5709+0+0 (secure 0 0 0) 0x7fa4b80898b0 con 0x7fa4c00b7c60 2024-04-20T11:38:14.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.030+0000 7fa4bde00640 1 --2- 172.21.15.152:0/4277385822 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fa49c06d460 0x7fa49c06f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:14.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.031+0000 7fa4bde00640 1 --2- 172.21.15.152:0/4277385822 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fa49c06d460 0x7fa49c06f920 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fa4b0034000 tx=0x7fa4b0009030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:14.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.033+0000 7fa4b6000640 1 -- 172.21.15.152:0/4277385822 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fa4b8059540 con 0x7fa4c00b7c60 2024-04-20T11:38:14.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.263+0000 7fa4bf200640 1 -- 172.21.15.152:0/4277385822 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fa4c0160ea0 con 0x7fa4c00b7c60 2024-04-20T11:38:14.265 INFO:tasks.ceph.mon.a.smithi152.stderr:2024-04-20T11:38:14.265+0000 7f8229800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-20T11:38:14.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.269+0000 7fa4b6000640 1 -- 172.21.15.152:0/4277385822 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v117) ==== 161+0+0 (secure 0 0 0) 0x7fa4b8058e40 con 0x7fa4c00b7c60 2024-04-20T11:38:14.269 INFO:teuthology.orchestra.run.smithi152.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-20T11:38:14.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.271+0000 7fa4bf200640 1 -- 172.21.15.152:0/4277385822 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fa49c06d460 msgr2=0x7fa49c06f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:14.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.271+0000 7fa4bf200640 1 --2- 172.21.15.152:0/4277385822 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fa49c06d460 0x7fa49c06f920 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fa4b0034000 tx=0x7fa4b0009030 comp rx=0 tx=0).stop 2024-04-20T11:38:14.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.271+0000 7fa4bf200640 1 -- 172.21.15.152:0/4277385822 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa4c00b7c60 msgr2=0x7fa4c00b80e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:14.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.271+0000 7fa4bf200640 1 --2- 172.21.15.152:0/4277385822 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa4c00b7c60 0x7fa4c00b80e0 secure :-1 s=READY pgs=2410 cs=0 l=1 rev1=1 crypto rx=0x7fa4c00ba650 tx=0x7fa4b800c760 comp rx=0 tx=0).stop 2024-04-20T11:38:14.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.271+0000 7fa4bf200640 1 -- 172.21.15.152:0/4277385822 shutdown_connections 2024-04-20T11:38:14.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.271+0000 7fa4bf200640 1 --2- 172.21.15.152:0/4277385822 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fa49c06d460 0x7fa49c06f920 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:14.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.271+0000 7fa4bf200640 1 --2- 172.21.15.152:0/4277385822 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa4c00b7c60 0x7fa4c00b80e0 unknown :-1 s=CLOSED pgs=2410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:14.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.271+0000 7fa4bf200640 1 --2- 172.21.15.152:0/4277385822 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa4c00cd4f0 0x7fa4c00cd930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:14.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.271+0000 7fa4bf200640 1 --2- 172.21.15.152:0/4277385822 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa4a00039e0 0x7fa4c01f5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:14.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.271+0000 7fa4bf200640 1 -- 172.21.15.152:0/4277385822 >> 172.21.15.152:0/4277385822 conn(0x7fa4c008bfd0 msgr2=0x7fa4c00ca690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:14.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.272+0000 7fa4bf200640 1 -- 172.21.15.152:0/4277385822 shutdown_connections 2024-04-20T11:38:14.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.272+0000 7fa4bf200640 1 -- 172.21.15.152:0/4277385822 wait complete. 2024-04-20T11:38:14.295 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-20T11:38:14.374 INFO:tasks.ceph.mds.c.smithi152.stdout:starting mds.c at 2024-04-20T11:38:14.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.483+0000 7f22c3200640 1 Processor -- start 2024-04-20T11:38:14.484 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.483+0000 7f22c3200640 1 -- start start 2024-04-20T11:38:14.485 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.484+0000 7f22c3200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f22bc071cd0 0x7f22bc0720b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:14.485 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.484+0000 7f22c3200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f22bc160220 0x7f22bc160680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:14.485 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.484+0000 7f22c3200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22bc0725f0 con 0x7f22bc06eb30 2024-04-20T11:38:14.485 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.484+0000 7f22c3200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f22bc13f8e0 con 0x7f22bc160220 2024-04-20T11:38:14.485 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.484+0000 7f22c3200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f22bc160f90 con 0x7f22bc071cd0 2024-04-20T11:38:14.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.485+0000 7f22c1400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f22bc160220 0x7f22bc160680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:14.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.485+0000 7f22c1400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f22bc160220 0x7f22bc160680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:37560/0 (socket says 172.21.15.152:37560) 2024-04-20T11:38:14.487 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.485+0000 7f22c1400640 1 -- 172.21.15.152:0/2538301665 learned_addr learned my addr 172.21.15.152:0/2538301665 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:38:14.487 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.485+0000 7f22c1400640 1 -- 172.21.15.152:0/2538301665 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f22bc071cd0 msgr2=0x7f22bc0720b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:14.487 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.485+0000 7f22c1e00640 1 --2- 172.21.15.152:0/2538301665 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f22bc071cd0 0x7f22bc0720b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:14.487 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.485+0000 7f22c1400640 1 --2- 172.21.15.152:0/2538301665 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f22bc071cd0 0x7f22bc0720b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:14.487 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.485+0000 7f22c1400640 1 -- 172.21.15.152:0/2538301665 >> v1:172.21.15.175:6789/0 conn(0x7f22bc06eb30 legacy=0x7f22bc06efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:14.487 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.485+0000 7f22c1400640 1 -- 172.21.15.152:0/2538301665 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22bc06dd90 con 0x7f22bc160220 2024-04-20T11:38:14.488 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.486+0000 7f22c1400640 1 --2- 172.21.15.152:0/2538301665 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f22bc160220 0x7f22bc160680 secure :-1 s=READY pgs=2413 cs=0 l=1 rev1=1 crypto rx=0x7f22ac009870 tx=0x7f22ac02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=528912c7003c9dc8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:14.488 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.487+0000 7f22c0a00640 1 -- 172.21.15.152:0/2538301665 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22ac035020 con 0x7f22bc160220 2024-04-20T11:38:14.489 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.487+0000 7f22c0a00640 1 -- 172.21.15.152:0/2538301665 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22ac031640 con 0x7f22bc160220 2024-04-20T11:38:14.489 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.487+0000 7f22c0a00640 1 -- 172.21.15.152:0/2538301665 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22ac02d040 con 0x7f22bc160220 2024-04-20T11:38:14.489 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.487+0000 7f22c3200640 1 -- 172.21.15.152:0/2538301665 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f22bc160220 msgr2=0x7f22bc160680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:14.489 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.487+0000 7f22c3200640 1 --2- 172.21.15.152:0/2538301665 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f22bc160220 0x7f22bc160680 secure :-1 s=READY pgs=2413 cs=0 l=1 rev1=1 crypto rx=0x7f22ac009870 tx=0x7f22ac02b8f0 comp rx=0 tx=0).stop 2024-04-20T11:38:14.489 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.487+0000 7f22c3200640 1 -- 172.21.15.152:0/2538301665 shutdown_connections 2024-04-20T11:38:14.489 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.487+0000 7f22c3200640 1 --2- 172.21.15.152:0/2538301665 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f22bc160220 0x7f22bc160680 unknown :-1 s=CLOSED pgs=2413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:14.489 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.487+0000 7f22c3200640 1 --2- 172.21.15.152:0/2538301665 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f22bc071cd0 0x7f22bc0720b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:14.489 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.487+0000 7f22c3200640 1 -- 172.21.15.152:0/2538301665 >> 172.21.15.152:0/2538301665 conn(0x7f22bc067ae0 msgr2=0x7f22bc06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:14.489 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.487+0000 7f22c3200640 1 -- 172.21.15.152:0/2538301665 shutdown_connections 2024-04-20T11:38:14.489 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.487+0000 7f22c3200640 1 -- 172.21.15.152:0/2538301665 wait complete. 2024-04-20T11:38:14.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.489+0000 7f22c3200640 1 Processor -- start 2024-04-20T11:38:14.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.489+0000 7f22c3200640 1 -- start start 2024-04-20T11:38:14.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.490+0000 7f22c3200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22bc06eb30 0x7f22bc139e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:14.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.490+0000 7f22c3200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f22bc071cd0 0x7f22bc155e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:14.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.490+0000 7f22c3200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f22bc160220 0x7f22bc13a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:14.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.490+0000 7f22c3200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f22bc06e630 con 0x7f22bc071cd0 2024-04-20T11:38:14.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.490+0000 7f22c3200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f22bc06e350 con 0x7f22bc06eb30 2024-04-20T11:38:14.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.490+0000 7f22c3200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f22bc13fb90 con 0x7f22bc160220 2024-04-20T11:38:14.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.490+0000 7f22c1400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f22bc071cd0 0x7f22bc155e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:14.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.491+0000 7f22c1400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f22bc071cd0 0x7f22bc155e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:37566/0 (socket says 172.21.15.152:37566) 2024-04-20T11:38:14.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.491+0000 7f22c1400640 1 -- 172.21.15.152:0/1786791837 learned_addr learned my addr 172.21.15.152:0/1786791837 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:38:14.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.491+0000 7f22c1e00640 1 --2- 172.21.15.152:0/1786791837 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22bc06eb30 0x7f22bc139e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:14.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.491+0000 7f22c2800640 1 --2- 172.21.15.152:0/1786791837 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f22bc160220 0x7f22bc13a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:14.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.491+0000 7f22c1400640 1 -- 172.21.15.152:0/1786791837 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f22bc160220 msgr2=0x7f22bc13a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:14.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.491+0000 7f22c1400640 1 --2- 172.21.15.152:0/1786791837 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f22bc160220 0x7f22bc13a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:14.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.491+0000 7f22c1400640 1 -- 172.21.15.152:0/1786791837 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22bc06eb30 msgr2=0x7f22bc139e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:14.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.491+0000 7f22c1400640 1 --2- 172.21.15.152:0/1786791837 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22bc06eb30 0x7f22bc139e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:14.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.491+0000 7f22c1400640 1 -- 172.21.15.152:0/1786791837 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22bc139090 con 0x7f22bc071cd0 2024-04-20T11:38:14.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.491+0000 7f22c1400640 1 --2- 172.21.15.152:0/1786791837 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f22bc071cd0 0x7f22bc155e10 secure :-1 s=READY pgs=2414 cs=0 l=1 rev1=1 crypto rx=0x7f22ac000c00 tx=0x7f22ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:14.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.492+0000 7f22b6000640 1 -- 172.21.15.152:0/1786791837 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22ac040070 con 0x7f22bc071cd0 2024-04-20T11:38:14.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.492+0000 7f22b6000640 1 -- 172.21.15.152:0/1786791837 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22ac039c10 con 0x7f22bc071cd0 2024-04-20T11:38:14.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.492+0000 7f22b6000640 1 -- 172.21.15.152:0/1786791837 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22ac035040 con 0x7f22bc071cd0 2024-04-20T11:38:14.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.492+0000 7f22c3200640 1 -- 172.21.15.152:0/1786791837 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22bc139370 con 0x7f22bc071cd0 2024-04-20T11:38:14.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.492+0000 7f22c3200640 1 -- 172.21.15.152:0/1786791837 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22bc139930 con 0x7f22bc071cd0 2024-04-20T11:38:14.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.493+0000 7f22c3200640 1 -- 172.21.15.152:0/1786791837 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22bc075e60 con 0x7f22bc071cd0 2024-04-20T11:38:14.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.494+0000 7f22b6000640 1 -- 172.21.15.152:0/1786791837 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 27) ==== 92060+0+0 (secure 0 0 0) 0x7f22ac04b060 con 0x7f22bc071cd0 2024-04-20T11:38:14.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.495+0000 7f22b6000640 1 --2- 172.21.15.152:0/1786791837 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f22a006d460 0x7f22a006f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:14.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.495+0000 7f22b6000640 1 -- 172.21.15.152:0/1786791837 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(168..168 src has 1..168) ==== 5795+0+0 (secure 0 0 0) 0x7f22ac0abfe0 con 0x7f22bc071cd0 2024-04-20T11:38:14.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.495+0000 7f22c1e00640 1 --2- 172.21.15.152:0/1786791837 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f22a006d460 0x7f22a006f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:14.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.496+0000 7f22c1e00640 1 --2- 172.21.15.152:0/1786791837 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f22a006d460 0x7f22a006f920 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f22bc1570a0 tx=0x7f22b0008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:14.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.499+0000 7f22b6000640 1 -- 172.21.15.152:0/1786791837 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f22ac02f300 con 0x7f22bc071cd0 2024-04-20T11:38:14.738 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:14.738+0000 7f22c3200640 1 -- 172.21.15.152:0/1786791837 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f22bc068e90 con 0x7f22bc071cd0 2024-04-20T11:38:15.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:15.284+0000 7f22b6000640 1 -- 172.21.15.152:0/1786791837 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v118) ==== 104+0+0 (secure 0 0 0) 0x7f22ac07be60 con 0x7f22bc071cd0 2024-04-20T11:38:15.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:15.287+0000 7f22c3200640 1 -- 172.21.15.152:0/1786791837 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f22a006d460 msgr2=0x7f22a006f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:15.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:15.287+0000 7f22c3200640 1 --2- 172.21.15.152:0/1786791837 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f22a006d460 0x7f22a006f920 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f22bc1570a0 tx=0x7f22b0008040 comp rx=0 tx=0).stop 2024-04-20T11:38:15.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:15.288+0000 7f22c3200640 1 -- 172.21.15.152:0/1786791837 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f22bc071cd0 msgr2=0x7f22bc155e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:15.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:15.288+0000 7f22c3200640 1 --2- 172.21.15.152:0/1786791837 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f22bc071cd0 0x7f22bc155e10 secure :-1 s=READY pgs=2414 cs=0 l=1 rev1=1 crypto rx=0x7f22ac000c00 tx=0x7f22ac0046c0 comp rx=0 tx=0).stop 2024-04-20T11:38:15.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:15.288+0000 7f22c3200640 1 -- 172.21.15.152:0/1786791837 shutdown_connections 2024-04-20T11:38:15.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:15.288+0000 7f22c3200640 1 --2- 172.21.15.152:0/1786791837 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f22a006d460 0x7f22a006f920 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:15.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:15.288+0000 7f22c3200640 1 --2- 172.21.15.152:0/1786791837 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f22bc160220 0x7f22bc13a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:15.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:15.288+0000 7f22c3200640 1 --2- 172.21.15.152:0/1786791837 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f22bc071cd0 0x7f22bc155e10 unknown :-1 s=CLOSED pgs=2414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:15.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:15.288+0000 7f22c3200640 1 --2- 172.21.15.152:0/1786791837 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22bc06eb30 0x7f22bc139e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:15.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:15.288+0000 7f22c3200640 1 -- 172.21.15.152:0/1786791837 >> 172.21.15.152:0/1786791837 conn(0x7f22bc067ae0 msgr2=0x7f22bc06efc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:15.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:15.288+0000 7f22c3200640 1 -- 172.21.15.152:0/1786791837 shutdown_connections 2024-04-20T11:38:15.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:15.288+0000 7f22c3200640 1 -- 172.21.15.152:0/1786791837 wait complete. 2024-04-20T11:38:15.310 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-20T11:38:15.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:15.505+0000 7faf37c00640 1 Processor -- start 2024-04-20T11:38:15.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:15.505+0000 7faf37c00640 1 -- start start 2024-04-20T11:38:15.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:15.507+0000 7faf37c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faf30071db0 0x7faf30072220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:15.508 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:15.507+0000 7faf37c00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf3015d710 con 0x7faf300ce1e0 2024-04-20T11:38:15.508 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:15.507+0000 7faf37c00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf3015d9f0 con 0x7faf3015a450 2024-04-20T11:38:15.508 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:15.507+0000 7faf37c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7faf30072760 con 0x7faf30071db0 2024-04-20T11:38:15.508 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:15.507+0000 7faf2fe00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7faf300ce1e0 0x7faf3015cc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:34428/0 (socket says 172.21.15.152:34428) 2024-04-20T11:38:15.508 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:15.507+0000 7faf2fe00640 1 -- 172.21.15.152:0/1454663671 learned_addr learned my addr 172.21.15.152:0/1454663671 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:38:15.510 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:15.507+0000 7faf35c00640 1 --2- 172.21.15.152:0/1454663671 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faf30071db0 0x7faf30072220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:15.510 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:15.508+0000 7faf2f400640 1 -- 172.21.15.152:0/1454663671 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3205275873 0 0) 0x7faf3015d710 con 0x7faf300ce1e0 2024-04-20T11:38:15.510 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:15.508+0000 7faf2f400640 1 -- 172.21.15.152:0/1454663671 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faf18003520 con 0x7faf300ce1e0 2024-04-20T11:38:15.510 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:15.508+0000 7faf2f400640 1 -- 172.21.15.152:0/1454663671 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3489618085 0 0) 0x7faf18003520 con 0x7faf300ce1e0 2024-04-20T11:38:15.510 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:15.508+0000 7faf2f400640 1 -- 172.21.15.152:0/1454663671 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faf30071db0 msgr2=0x7faf30072220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:15.510 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:15.508+0000 7faf2f400640 1 --2- 172.21.15.152:0/1454663671 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faf30071db0 0x7faf30072220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:15.510 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:15.508+0000 7faf2f400640 1 -- 172.21.15.152:0/1454663671 >> v1:172.21.15.175:6789/0 conn(0x7faf3015a450 legacy=0x7faf300cd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:15.510 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:15.508+0000 7faf2f400640 1 -- 172.21.15.152:0/1454663671 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf300ccfb0 con 0x7faf300ce1e0 2024-04-20T11:38:15.510 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:15.508+0000 7faf2f400640 1 -- 172.21.15.152:0/1454663671 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7faf1c001670 con 0x7faf300ce1e0 2024-04-20T11:38:15.511 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:15.508+0000 7faf2f400640 1 -- 172.21.15.152:0/1454663671 >> v1:172.21.15.152:6789/0 conn(0x7faf300ce1e0 legacy=0x7faf3015cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:15.511 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:15.508+0000 7faf2f400640 1 --2- 172.21.15.152:0/1454663671 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faf30071db0 0x7faf18003a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:15.511 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:15.508+0000 7faf2f400640 1 --2- 172.21.15.152:0/1454663671 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faf18004020 0x7faf18004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:15.511 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:15.508+0000 7faf2f400640 1 --2- 172.21.15.152:0/1454663671 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faf18004d40 0x7faf180051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:15.511 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:15.508+0000 7faf2f400640 1 -- 172.21.15.152:0/1454663671 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7faf30072760 con 0x7faf18004020 2024-04-20T11:38:15.511 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:15.508+0000 7faf2f400640 1 -- 172.21.15.152:0/1454663671 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7faf1800bc10 con 0x7faf18004d40 2024-04-20T11:38:15.511 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:15.508+0000 7faf2f400640 1 -- 172.21.15.152:0/1454663671 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7faf1800be10 con 0x7faf30071db0 2024-04-20T11:38:15.511 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:15.509+0000 7faf2fe00640 1 --2- 172.21.15.152:0/1454663671 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faf18004020 0x7faf18004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:15.511 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:15.509+0000 7faf35200640 1 --2- 172.21.15.152:0/1454663671 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faf30071db0 0x7faf18003a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:15.511 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:15.509+0000 7faf35c00640 1 --2- 172.21.15.152:0/1454663671 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faf18004d40 0x7faf180051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:15.511 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:15.509+0000 7faf2fe00640 1 -- 172.21.15.152:0/1454663671 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faf30071db0 msgr2=0x7faf18003a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:15.511 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:15.509+0000 7faf2fe00640 1 --2- 172.21.15.152:0/1454663671 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faf30071db0 0x7faf18003a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:15.511 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:15.509+0000 7faf2fe00640 1 -- 172.21.15.152:0/1454663671 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faf18004d40 msgr2=0x7faf180051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:15.511 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:15.509+0000 7faf2fe00640 1 --2- 172.21.15.152:0/1454663671 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faf18004d40 0x7faf180051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:15.512 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:15.509+0000 7faf2fe00640 1 -- 172.21.15.152:0/1454663671 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf18003520 con 0x7faf18004020 2024-04-20T11:38:15.513 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:15.510+0000 7faf2fe00640 1 --2- 172.21.15.152:0/1454663671 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faf18004020 0x7faf18004800 secure :-1 s=READY pgs=2416 cs=0 l=1 rev1=1 crypto rx=0x7faf1c00c190 tx=0x7faf18005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f865bb05dc33b1a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:15.513 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:15.510+0000 7faf2f400640 1 -- 172.21.15.152:0/1454663671 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf1c007c60 con 0x7faf18004020 2024-04-20T11:38:15.513 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:15.510+0000 7faf2f400640 1 -- 172.21.15.152:0/1454663671 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf1c030070 con 0x7faf18004020 2024-04-20T11:38:15.513 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:15.510+0000 7faf2f400640 1 -- 172.21.15.152:0/1454663671 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf1c005db0 con 0x7faf18004020 2024-04-20T11:38:15.513 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:15.511+0000 7faf37c00640 1 -- 172.21.15.152:0/1454663671 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faf18004020 msgr2=0x7faf18004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:15.513 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:15.511+0000 7faf37c00640 1 --2- 172.21.15.152:0/1454663671 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faf18004020 0x7faf18004800 secure :-1 s=READY pgs=2416 cs=0 l=1 rev1=1 crypto rx=0x7faf1c00c190 tx=0x7faf18005830 comp rx=0 tx=0).stop 2024-04-20T11:38:15.513 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:15.511+0000 7faf35c00640 1 -- 172.21.15.152:0/1454663671 reap_dead start 2024-04-20T11:38:15.513 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:15.511+0000 7faf37c00640 1 -- 172.21.15.152:0/1454663671 shutdown_connections 2024-04-20T11:38:15.513 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:15.511+0000 7faf37c00640 1 -- 172.21.15.152:0/1454663671 >> 172.21.15.152:0/1454663671 conn(0x7faf30067cf0 msgr2=0x7faf3006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:15.513 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:15.511+0000 7faf37c00640 1 -- 172.21.15.152:0/1454663671 shutdown_connections 2024-04-20T11:38:15.513 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:15.511+0000 7faf37c00640 1 -- 172.21.15.152:0/1454663671 wait complete. 2024-04-20T11:38:15.514 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:15.513+0000 7faf37c00640 1 Processor -- start 2024-04-20T11:38:15.514 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:15.513+0000 7faf37c00640 1 -- start start 2024-04-20T11:38:15.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:15.514+0000 7faf37c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faf18004020 0x7faf30144f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:15.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:15.514+0000 7faf37c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faf30141900 0x7faf300a9510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:15.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:15.514+0000 7faf37c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faf30141d60 0x7faf300b0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:15.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:15.514+0000 7faf37c00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7faf30072220 con 0x7faf30141d60 2024-04-20T11:38:15.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:15.514+0000 7faf37c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7faf300cd320 con 0x7faf18004020 2024-04-20T11:38:15.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:15.514+0000 7faf37c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7faf300723a0 con 0x7faf30141900 2024-04-20T11:38:15.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:15.515+0000 7faf35c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faf30141d60 0x7faf300b0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:15.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:15.515+0000 7faf35c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faf30141d60 0x7faf300b0c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:37592/0 (socket says 172.21.15.152:37592) 2024-04-20T11:38:15.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:15.515+0000 7faf35c00640 1 -- 172.21.15.152:0/74823163 learned_addr learned my addr 172.21.15.152:0/74823163 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:38:15.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:15.515+0000 7faf2fe00640 1 --2- 172.21.15.152:0/74823163 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faf30141900 0x7faf300a9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:15.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:15.515+0000 7faf35200640 1 --2- 172.21.15.152:0/74823163 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faf18004020 0x7faf30144f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:15.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:15.515+0000 7faf35c00640 1 -- 172.21.15.152:0/74823163 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faf30141900 msgr2=0x7faf300a9510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:15.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:15.515+0000 7faf35c00640 1 --2- 172.21.15.152:0/74823163 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faf30141900 0x7faf300a9510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:15.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:15.515+0000 7faf35c00640 1 -- 172.21.15.152:0/74823163 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faf18004020 msgr2=0x7faf30144f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:15.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:15.515+0000 7faf35c00640 1 --2- 172.21.15.152:0/74823163 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faf18004020 0x7faf30144f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:15.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:15.515+0000 7faf35c00640 1 -- 172.21.15.152:0/74823163 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf30145450 con 0x7faf30141d60 2024-04-20T11:38:15.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:15.516+0000 7faf35c00640 1 --2- 172.21.15.152:0/74823163 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faf30141d60 0x7faf300b0c60 secure :-1 s=READY pgs=2417 cs=0 l=1 rev1=1 crypto rx=0x7faf30071d60 tx=0x7faf280115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:15.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:15.516+0000 7faf2cc00640 1 -- 172.21.15.152:0/74823163 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf28011e10 con 0x7faf30141d60 2024-04-20T11:38:15.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:15.516+0000 7faf2cc00640 1 -- 172.21.15.152:0/74823163 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf2801b070 con 0x7faf30141d60 2024-04-20T11:38:15.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:15.516+0000 7faf2cc00640 1 -- 172.21.15.152:0/74823163 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf28003990 con 0x7faf30141d60 2024-04-20T11:38:15.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:15.516+0000 7faf37c00640 1 -- 172.21.15.152:0/74823163 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf300b1ae0 con 0x7faf30141d60 2024-04-20T11:38:15.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:15.516+0000 7faf37c00640 1 -- 172.21.15.152:0/74823163 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf300b08e0 con 0x7faf30141d60 2024-04-20T11:38:15.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:15.517+0000 7faf37c00640 1 -- 172.21.15.152:0/74823163 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf30068a60 con 0x7faf30141d60 2024-04-20T11:38:15.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:15.519+0000 7faf2cc00640 1 -- 172.21.15.152:0/74823163 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 27) ==== 92060+0+0 (secure 0 0 0) 0x7faf28005d10 con 0x7faf30141d60 2024-04-20T11:38:15.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:15.519+0000 7faf2cc00640 1 --2- 172.21.15.152:0/74823163 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7faf1006d430 0x7faf1006f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:15.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:15.520+0000 7faf2cc00640 1 -- 172.21.15.152:0/74823163 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(168..168 src has 1..168) ==== 5795+0+0 (secure 0 0 0) 0x7faf2808aad0 con 0x7faf30141d60 2024-04-20T11:38:15.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:15.520+0000 7faf35200640 1 --2- 172.21.15.152:0/74823163 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7faf1006d430 0x7faf1006f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:15.522 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:15.521+0000 7faf35200640 1 --2- 172.21.15.152:0/74823163 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7faf1006d430 0x7faf1006f8f0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7faf24007fd0 tx=0x7faf24009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:15.525 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:15.524+0000 7faf2cc00640 1 -- 172.21.15.152:0/74823163 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7faf28025020 con 0x7faf30141d60 2024-04-20T11:38:15.726 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:15.725+0000 7faf37c00640 1 -- 172.21.15.152:0/74823163 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7faf300cf3e0 con 0x7faf30141d60 2024-04-20T11:38:16.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:16.306+0000 7faf2cc00640 1 -- 172.21.15.152:0/74823163 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v169) ==== 181+0+0 (secure 0 0 0) 0x7faf28014350 con 0x7faf30141d60 2024-04-20T11:38:16.307 INFO:teuthology.orchestra.run.smithi152.stderr:pool 'cephfs_metadata' removed 2024-04-20T11:38:16.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:16.309+0000 7faf37c00640 1 -- 172.21.15.152:0/74823163 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7faf1006d430 msgr2=0x7faf1006f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:16.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:16.309+0000 7faf37c00640 1 --2- 172.21.15.152:0/74823163 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7faf1006d430 0x7faf1006f8f0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7faf24007fd0 tx=0x7faf24009040 comp rx=0 tx=0).stop 2024-04-20T11:38:16.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:16.309+0000 7faf37c00640 1 -- 172.21.15.152:0/74823163 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faf30141d60 msgr2=0x7faf300b0c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:16.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:16.309+0000 7faf37c00640 1 --2- 172.21.15.152:0/74823163 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faf30141d60 0x7faf300b0c60 secure :-1 s=READY pgs=2417 cs=0 l=1 rev1=1 crypto rx=0x7faf30071d60 tx=0x7faf280115a0 comp rx=0 tx=0).stop 2024-04-20T11:38:16.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:16.309+0000 7faf37c00640 1 -- 172.21.15.152:0/74823163 shutdown_connections 2024-04-20T11:38:16.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:16.310+0000 7faf37c00640 1 --2- 172.21.15.152:0/74823163 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7faf1006d430 0x7faf1006f8f0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:16.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:16.310+0000 7faf37c00640 1 --2- 172.21.15.152:0/74823163 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faf30141d60 0x7faf300b0c60 unknown :-1 s=CLOSED pgs=2417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:16.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:16.310+0000 7faf37c00640 1 --2- 172.21.15.152:0/74823163 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faf30141900 0x7faf300a9510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:16.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:16.310+0000 7faf37c00640 1 --2- 172.21.15.152:0/74823163 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faf18004020 0x7faf30144f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:16.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:16.310+0000 7faf37c00640 1 -- 172.21.15.152:0/74823163 >> 172.21.15.152:0/74823163 conn(0x7faf30067cf0 msgr2=0x7faf300ce5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:16.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:16.310+0000 7faf37c00640 1 -- 172.21.15.152:0/74823163 shutdown_connections 2024-04-20T11:38:16.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:16.310+0000 7faf37c00640 1 -- 172.21.15.152:0/74823163 wait complete. 2024-04-20T11:38:16.338 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-20T11:38:16.531 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:16.530+0000 7fc9d5e00640 1 Processor -- start 2024-04-20T11:38:16.531 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:16.531+0000 7fc9d5e00640 1 -- start start 2024-04-20T11:38:16.533 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:16.532+0000 7fc9d5e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fc9d00cd8e0 0x7fc9d006e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:16.533 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:16.532+0000 7fc9d5e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fc9d00cf220 0x7fc9d00b4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:16.533 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:16.532+0000 7fc9d5e00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9d006eab0 con 0x7fc9d0144d70 2024-04-20T11:38:16.533 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:16.532+0000 7fc9d5e00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fc9d00d0240 con 0x7fc9d00cf220 2024-04-20T11:38:16.533 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:16.532+0000 7fc9d5e00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fc9d00cf6c0 con 0x7fc9d00cd8e0 2024-04-20T11:38:16.533 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:16.532+0000 7fc9c6000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fc9d00cf220 0x7fc9d00b4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:16.533 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:16.532+0000 7fc9c6000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fc9d00cf220 0x7fc9d00b4680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54628/0 (socket says 172.21.15.152:54628) 2024-04-20T11:38:16.533 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:16.532+0000 7fc9c6000640 1 -- 172.21.15.152:0/1354597374 learned_addr learned my addr 172.21.15.152:0/1354597374 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:38:16.533 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:16.532+0000 7fc9cf400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fc9d0144d70 0x7fc9d015f410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:43996/0 (socket says 172.21.15.152:43996) 2024-04-20T11:38:16.534 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:16.533+0000 7fc9cea00640 1 --2- 172.21.15.152:0/1354597374 >> v2:172.21.15.175:3300/0 conn(0x7fc9d00cd8e0 0x7fc9d006e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:16.534 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:16.533+0000 7fc9c6000640 1 -- 172.21.15.152:0/1354597374 >> v2:172.21.15.175:3300/0 conn(0x7fc9d00cd8e0 msgr2=0x7fc9d006e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:16.535 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:16.533+0000 7fc9c6000640 1 --2- 172.21.15.152:0/1354597374 >> v2:172.21.15.175:3300/0 conn(0x7fc9d00cd8e0 0x7fc9d006e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:16.535 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:16.533+0000 7fc9c6000640 1 -- 172.21.15.152:0/1354597374 >> v1:172.21.15.152:6789/0 conn(0x7fc9d0144d70 legacy=0x7fc9d015f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:16.535 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:16.533+0000 7fc9c6000640 1 -- 172.21.15.152:0/1354597374 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9d00cf9a0 con 0x7fc9d00cf220 2024-04-20T11:38:16.535 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:16.533+0000 7fc9ce000640 1 -- 172.21.15.152:0/1354597374 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3554594782 0 0) 0x7fc9d006eab0 con 0x7fc9d0144d70 2024-04-20T11:38:16.535 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:16.534+0000 7fc9c6000640 1 --2- 172.21.15.152:0/1354597374 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc9d00cf220 0x7fc9d00b4680 secure :-1 s=READY pgs=2419 cs=0 l=1 rev1=1 crypto rx=0x7fc9d0073680 tx=0x7fc9b802e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc270ecadfdffc76 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:16.536 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:16.534+0000 7fc9ce000640 1 -- 172.21.15.152:0/1354597374 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9b803b070 con 0x7fc9d00cf220 2024-04-20T11:38:16.536 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:16.534+0000 7fc9ce000640 1 -- 172.21.15.152:0/1354597374 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9b8036440 con 0x7fc9d00cf220 2024-04-20T11:38:16.536 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:16.534+0000 7fc9ce000640 1 -- 172.21.15.152:0/1354597374 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9b8036810 con 0x7fc9d00cf220 2024-04-20T11:38:16.536 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:16.535+0000 7fc9d5e00640 1 -- 172.21.15.152:0/1354597374 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc9d00cf220 msgr2=0x7fc9d00b4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:16.536 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:16.535+0000 7fc9d5e00640 1 --2- 172.21.15.152:0/1354597374 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc9d00cf220 0x7fc9d00b4680 secure :-1 s=READY pgs=2419 cs=0 l=1 rev1=1 crypto rx=0x7fc9d0073680 tx=0x7fc9b802e0e0 comp rx=0 tx=0).stop 2024-04-20T11:38:16.536 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:16.535+0000 7fc9d5e00640 1 -- 172.21.15.152:0/1354597374 shutdown_connections 2024-04-20T11:38:16.536 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:16.535+0000 7fc9d5e00640 1 --2- 172.21.15.152:0/1354597374 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc9d00cf220 0x7fc9d00b4680 unknown :-1 s=CLOSED pgs=2419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:16.536 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:16.535+0000 7fc9d5e00640 1 --2- 172.21.15.152:0/1354597374 >> v2:172.21.15.175:3300/0 conn(0x7fc9d00cd8e0 0x7fc9d006e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:16.537 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:16.535+0000 7fc9d5e00640 1 -- 172.21.15.152:0/1354597374 >> 172.21.15.152:0/1354597374 conn(0x7fc9d006b980 msgr2=0x7fc9d006bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:16.537 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:16.535+0000 7fc9d5e00640 1 -- 172.21.15.152:0/1354597374 shutdown_connections 2024-04-20T11:38:16.537 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:16.535+0000 7fc9d5e00640 1 -- 172.21.15.152:0/1354597374 wait complete. 2024-04-20T11:38:16.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:16.537+0000 7fc9d5e00640 1 Processor -- start 2024-04-20T11:38:16.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:16.537+0000 7fc9d5e00640 1 -- start start 2024-04-20T11:38:16.539 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:16.538+0000 7fc9d5e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc9d00cd8e0 0x7fc9d00b00e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:16.539 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:16.538+0000 7fc9d5e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc9d00cf220 0x7fc9d01f4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:16.539 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:16.538+0000 7fc9d5e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc9d0144d70 0x7fc9d00b37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:16.539 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:16.538+0000 7fc9d5e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d00cdcc0 con 0x7fc9d00cd8e0 2024-04-20T11:38:16.539 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:16.538+0000 7fc9d5e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d00cff60 con 0x7fc9d0144d70 2024-04-20T11:38:16.539 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:16.538+0000 7fc9d5e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc9d01447b0 con 0x7fc9d00cf220 2024-04-20T11:38:16.539 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:16.539+0000 7fc9cea00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc9d00cd8e0 0x7fc9d00b00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:16.539 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:16.539+0000 7fc9cea00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc9d00cd8e0 0x7fc9d00b00e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54638/0 (socket says 172.21.15.152:54638) 2024-04-20T11:38:16.540 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:16.539+0000 7fc9cea00640 1 -- 172.21.15.152:0/1486453807 learned_addr learned my addr 172.21.15.152:0/1486453807 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:38:16.541 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:16.539+0000 7fc9c6000640 1 --2- 172.21.15.152:0/1486453807 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc9d00cf220 0x7fc9d01f4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:16.541 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:16.539+0000 7fc9cea00640 1 -- 172.21.15.152:0/1486453807 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc9d00cf220 msgr2=0x7fc9d01f4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:16.541 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:16.539+0000 7fc9cea00640 1 --2- 172.21.15.152:0/1486453807 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc9d00cf220 0x7fc9d01f4ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:16.541 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:16.539+0000 7fc9cea00640 1 -- 172.21.15.152:0/1486453807 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc9d0144d70 msgr2=0x7fc9d00b37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:16.541 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:16.539+0000 7fc9cf400640 1 --2- 172.21.15.152:0/1486453807 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc9d0144d70 0x7fc9d00b37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:16.541 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:16.539+0000 7fc9cea00640 1 --2- 172.21.15.152:0/1486453807 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc9d0144d70 0x7fc9d00b37c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:16.541 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:16.539+0000 7fc9cea00640 1 -- 172.21.15.152:0/1486453807 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9d00b06c0 con 0x7fc9d00cd8e0 2024-04-20T11:38:16.541 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:16.539+0000 7fc9cea00640 1 --2- 172.21.15.152:0/1486453807 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc9d00cd8e0 0x7fc9d00b00e0 secure :-1 s=READY pgs=2420 cs=0 l=1 rev1=1 crypto rx=0x7fc9c000d620 tx=0x7fc9c000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:16.541 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:16.540+0000 7fc9c7e00640 1 -- 172.21.15.152:0/1486453807 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c000b620 con 0x7fc9d00cd8e0 2024-04-20T11:38:16.541 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:16.540+0000 7fc9c7e00640 1 -- 172.21.15.152:0/1486453807 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9c0019070 con 0x7fc9d00cd8e0 2024-04-20T11:38:16.541 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:16.540+0000 7fc9c7e00640 1 -- 172.21.15.152:0/1486453807 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c000ba00 con 0x7fc9d00cd8e0 2024-04-20T11:38:16.542 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:16.540+0000 7fc9d5e00640 1 -- 172.21.15.152:0/1486453807 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9d00b09a0 con 0x7fc9d00cd8e0 2024-04-20T11:38:16.542 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:16.540+0000 7fc9d5e00640 1 -- 172.21.15.152:0/1486453807 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9d00b0f60 con 0x7fc9d00cd8e0 2024-04-20T11:38:16.542 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:16.541+0000 7fc9d5e00640 1 -- 172.21.15.152:0/1486453807 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9d0068a60 con 0x7fc9d00cd8e0 2024-04-20T11:38:16.542 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:16.542+0000 7fc9c7e00640 1 -- 172.21.15.152:0/1486453807 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 27) ==== 92060+0+0 (secure 0 0 0) 0x7fc9c000bba0 con 0x7fc9d00cd8e0 2024-04-20T11:38:16.543 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:16.543+0000 7fc9c7e00640 1 --2- 172.21.15.152:0/1486453807 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fc9ac06d430 0x7fc9ac06f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:16.543 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:16.543+0000 7fc9c7e00640 1 -- 172.21.15.152:0/1486453807 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(169..169 src has 1..169) ==== 5357+0+0 (secure 0 0 0) 0x7fc9c008a8f0 con 0x7fc9d00cd8e0 2024-04-20T11:38:16.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:16.543+0000 7fc9c6000640 1 --2- 172.21.15.152:0/1486453807 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fc9ac06d430 0x7fc9ac06f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:16.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:16.545+0000 7fc9c6000640 1 --2- 172.21.15.152:0/1486453807 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fc9ac06d430 0x7fc9ac06f8f0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fc9b8038000 tx=0x7fc9b8008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:16.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:16.547+0000 7fc9c7e00640 1 -- 172.21.15.152:0/1486453807 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fc9c005a770 con 0x7fc9d00cd8e0 2024-04-20T11:38:16.751 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:16.750+0000 7fc9d5e00640 1 -- 172.21.15.152:0/1486453807 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fc9d00aec40 con 0x7fc9d00cd8e0 2024-04-20T11:38:17.317 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.316+0000 7fc9c7e00640 1 -- 172.21.15.152:0/1486453807 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v170) ==== 169+0+0 (secure 0 0 0) 0x7fc9c005a070 con 0x7fc9d00cd8e0 2024-04-20T11:38:17.317 INFO:teuthology.orchestra.run.smithi152.stderr:pool 'cephfs_data' removed 2024-04-20T11:38:17.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.320+0000 7fc9d5e00640 1 -- 172.21.15.152:0/1486453807 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fc9ac06d430 msgr2=0x7fc9ac06f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:17.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.320+0000 7fc9d5e00640 1 --2- 172.21.15.152:0/1486453807 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fc9ac06d430 0x7fc9ac06f8f0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fc9b8038000 tx=0x7fc9b8008040 comp rx=0 tx=0).stop 2024-04-20T11:38:17.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.320+0000 7fc9d5e00640 1 -- 172.21.15.152:0/1486453807 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc9d00cd8e0 msgr2=0x7fc9d00b00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:17.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.320+0000 7fc9d5e00640 1 --2- 172.21.15.152:0/1486453807 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc9d00cd8e0 0x7fc9d00b00e0 secure :-1 s=READY pgs=2420 cs=0 l=1 rev1=1 crypto rx=0x7fc9c000d620 tx=0x7fc9c000dae0 comp rx=0 tx=0).stop 2024-04-20T11:38:17.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.320+0000 7fc9d5e00640 1 -- 172.21.15.152:0/1486453807 shutdown_connections 2024-04-20T11:38:17.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.320+0000 7fc9d5e00640 1 --2- 172.21.15.152:0/1486453807 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fc9ac06d430 0x7fc9ac06f8f0 secure :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fc9b8038000 tx=0x7fc9b8008040 comp rx=0 tx=0).stop 2024-04-20T11:38:17.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.320+0000 7fc9d5e00640 1 --2- 172.21.15.152:0/1486453807 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc9d0144d70 0x7fc9d00b37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:17.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.320+0000 7fc9d5e00640 1 --2- 172.21.15.152:0/1486453807 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc9d00cf220 0x7fc9d01f4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:17.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.320+0000 7fc9d5e00640 1 --2- 172.21.15.152:0/1486453807 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc9d00cd8e0 0x7fc9d00b00e0 unknown :-1 s=CLOSED pgs=2420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:17.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.320+0000 7fc9d5e00640 1 -- 172.21.15.152:0/1486453807 >> 172.21.15.152:0/1486453807 conn(0x7fc9d006b980 msgr2=0x7fc9d01e4090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:17.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.320+0000 7fc9d5e00640 1 -- 172.21.15.152:0/1486453807 shutdown_connections 2024-04-20T11:38:17.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.321+0000 7fc9d5e00640 1 -- 172.21.15.152:0/1486453807 wait complete. 2024-04-20T11:38:17.351 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-20T11:38:17.540 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.539+0000 7f04c3200640 1 Processor -- start 2024-04-20T11:38:17.540 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.539+0000 7f04c3200640 1 -- start start 2024-04-20T11:38:17.541 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.541+0000 7f04c3200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f04bc160220 0x7f04bc1606b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:17.541 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.541+0000 7f04c3200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f04bc141450 0x7f04bc1418e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:17.541 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.541+0000 7f04c3200640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04bc06fb40 con 0x7f04bc071cd0 2024-04-20T11:38:17.542 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.541+0000 7f04c3200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f04bc13f8e0 con 0x7f04bc160220 2024-04-20T11:38:17.542 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.541+0000 7f04c3200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f04bc160f90 con 0x7f04bc141450 2024-04-20T11:38:17.542 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.541+0000 7f04c1e00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f04bc071cd0 0x7f04bc0720b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:43998/0 (socket says 172.21.15.152:43998) 2024-04-20T11:38:17.542 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.541+0000 7f04c1e00640 1 -- 172.21.15.152:0/1700694847 learned_addr learned my addr 172.21.15.152:0/1700694847 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:38:17.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.541+0000 7f04c1400640 1 --2- 172.21.15.152:0/1700694847 >> v2:172.21.15.175:3300/0 conn(0x7f04bc160220 0x7f04bc1606b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:17.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.541+0000 7f04c2800640 1 --2- 172.21.15.152:0/1700694847 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f04bc141450 0x7f04bc1418e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:17.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.541+0000 7f04c0a00640 1 -- 172.21.15.152:0/1700694847 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 711118214 0 0) 0x7f04bc06fb40 con 0x7f04bc071cd0 2024-04-20T11:38:17.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.542+0000 7f04c0a00640 1 -- 172.21.15.152:0/1700694847 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f04a4003520 con 0x7f04bc071cd0 2024-04-20T11:38:17.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.542+0000 7f04c0a00640 1 -- 172.21.15.152:0/1700694847 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1945197588 0 0) 0x7f04a4003520 con 0x7f04bc071cd0 2024-04-20T11:38:17.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.542+0000 7f04c0a00640 1 -- 172.21.15.152:0/1700694847 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f04bc141450 msgr2=0x7f04bc1418e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:17.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.542+0000 7f04c0a00640 1 --2- 172.21.15.152:0/1700694847 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f04bc141450 0x7f04bc1418e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:17.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.542+0000 7f04c0a00640 1 -- 172.21.15.152:0/1700694847 >> v2:172.21.15.175:3300/0 conn(0x7f04bc160220 msgr2=0x7f04bc1606b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:17.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.542+0000 7f04c0a00640 1 --2- 172.21.15.152:0/1700694847 >> v2:172.21.15.175:3300/0 conn(0x7f04bc160220 0x7f04bc1606b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:17.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.542+0000 7f04c0a00640 1 -- 172.21.15.152:0/1700694847 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04bc06dd90 con 0x7f04bc071cd0 2024-04-20T11:38:17.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.542+0000 7f04c0a00640 1 -- 172.21.15.152:0/1700694847 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f04b0001670 con 0x7f04bc071cd0 2024-04-20T11:38:17.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.542+0000 7f04c0a00640 1 -- 172.21.15.152:0/1700694847 >> v1:172.21.15.152:6789/0 conn(0x7f04bc071cd0 legacy=0x7f04bc0720b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:17.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.542+0000 7f04c0a00640 1 --2- 172.21.15.152:0/1700694847 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f04a40039b0 0x7f04a4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:17.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.542+0000 7f04c0a00640 1 --2- 172.21.15.152:0/1700694847 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f04bc141450 0x7f04a4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:17.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.542+0000 7f04c0a00640 1 --2- 172.21.15.152:0/1700694847 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f04a4004d40 0x7f04a40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:17.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.542+0000 7f04c0a00640 1 -- 172.21.15.152:0/1700694847 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f04bc13f8e0 con 0x7f04a4004d40 2024-04-20T11:38:17.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.542+0000 7f04c0a00640 1 -- 172.21.15.152:0/1700694847 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f04bc160f90 con 0x7f04a40039b0 2024-04-20T11:38:17.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.542+0000 7f04c0a00640 1 -- 172.21.15.152:0/1700694847 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f04a400b9f0 con 0x7f04bc141450 2024-04-20T11:38:17.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.542+0000 7f04c2800640 1 --2- 172.21.15.152:0/1700694847 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f04a4004d40 0x7f04a40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:17.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.543+0000 7f04c1e00640 1 --2- 172.21.15.152:0/1700694847 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f04a40039b0 0x7f04a4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:17.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.543+0000 7f04c1400640 1 --2- 172.21.15.152:0/1700694847 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f04bc141450 0x7f04a4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:17.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.543+0000 7f04c2800640 1 -- 172.21.15.152:0/1700694847 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f04bc141450 msgr2=0x7f04a4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:17.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.543+0000 7f04c2800640 1 --2- 172.21.15.152:0/1700694847 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f04bc141450 0x7f04a4004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:17.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.543+0000 7f04c2800640 1 -- 172.21.15.152:0/1700694847 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f04a40039b0 msgr2=0x7f04a4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:17.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.543+0000 7f04c2800640 1 --2- 172.21.15.152:0/1700694847 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f04a40039b0 0x7f04a4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:17.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.543+0000 7f04c2800640 1 -- 172.21.15.152:0/1700694847 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04a4003520 con 0x7f04a4004d40 2024-04-20T11:38:17.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.544+0000 7f04c2800640 1 --2- 172.21.15.152:0/1700694847 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f04a4004d40 0x7f04a40051c0 secure :-1 s=READY pgs=2422 cs=0 l=1 rev1=1 crypto rx=0x7f04b8009fc0 tx=0x7f04a4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=556bb2fa4db94a21 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:17.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.544+0000 7f04c0a00640 1 -- 172.21.15.152:0/1700694847 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b8039070 con 0x7f04a4004d40 2024-04-20T11:38:17.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.544+0000 7f04c0a00640 1 -- 172.21.15.152:0/1700694847 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04b802cc20 con 0x7f04a4004d40 2024-04-20T11:38:17.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.544+0000 7f04c0a00640 1 -- 172.21.15.152:0/1700694847 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b8009c10 con 0x7f04a4004d40 2024-04-20T11:38:17.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.545+0000 7f04c3200640 1 -- 172.21.15.152:0/1700694847 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f04a4004d40 msgr2=0x7f04a40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:17.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.545+0000 7f04c3200640 1 --2- 172.21.15.152:0/1700694847 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f04a4004d40 0x7f04a40051c0 secure :-1 s=READY pgs=2422 cs=0 l=1 rev1=1 crypto rx=0x7f04b8009fc0 tx=0x7f04a4005830 comp rx=0 tx=0).stop 2024-04-20T11:38:17.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.545+0000 7f04c2800640 1 -- 172.21.15.152:0/1700694847 reap_dead start 2024-04-20T11:38:17.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.545+0000 7f04c3200640 1 -- 172.21.15.152:0/1700694847 shutdown_connections 2024-04-20T11:38:17.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.545+0000 7f04c3200640 1 -- 172.21.15.152:0/1700694847 >> 172.21.15.152:0/1700694847 conn(0x7f04bc067ae0 msgr2=0x7f04bc06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:17.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.545+0000 7f04c3200640 1 -- 172.21.15.152:0/1700694847 shutdown_connections 2024-04-20T11:38:17.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.545+0000 7f04c3200640 1 -- 172.21.15.152:0/1700694847 wait complete. 2024-04-20T11:38:17.548 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.547+0000 7f04c3200640 1 Processor -- start 2024-04-20T11:38:17.548 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.547+0000 7f04c3200640 1 -- start start 2024-04-20T11:38:17.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.548+0000 7f04c3200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f04bc0b13d0 0x7f04bc13dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:17.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.548+0000 7f04c3200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f04bc0b17b0 0x7f04bc13e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:17.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.548+0000 7f04c3200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f04bc1e75f0 0x7f04bc1e7a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:17.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.548+0000 7f04c3200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f04bc06e630 con 0x7f04bc0b17b0 2024-04-20T11:38:17.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.548+0000 7f04c3200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f04bc06e350 con 0x7f04bc0b13d0 2024-04-20T11:38:17.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.548+0000 7f04c3200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f04bc160bf0 con 0x7f04bc1e75f0 2024-04-20T11:38:17.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.548+0000 7f04c1400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f04bc0b17b0 0x7f04bc13e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:17.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.548+0000 7f04c1400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f04bc0b17b0 0x7f04bc13e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54648/0 (socket says 172.21.15.152:54648) 2024-04-20T11:38:17.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.548+0000 7f04c1400640 1 -- 172.21.15.152:0/1843744607 learned_addr learned my addr 172.21.15.152:0/1843744607 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:38:17.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.548+0000 7f04c1e00640 1 --2- 172.21.15.152:0/1843744607 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f04bc0b13d0 0x7f04bc13dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:17.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.549+0000 7f04c2800640 1 --2- 172.21.15.152:0/1843744607 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f04bc1e75f0 0x7f04bc1e7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:17.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.549+0000 7f04c1400640 1 -- 172.21.15.152:0/1843744607 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f04bc1e75f0 msgr2=0x7f04bc1e7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:17.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.549+0000 7f04c1400640 1 --2- 172.21.15.152:0/1843744607 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f04bc1e75f0 0x7f04bc1e7a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:17.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.549+0000 7f04c1400640 1 -- 172.21.15.152:0/1843744607 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f04bc0b13d0 msgr2=0x7f04bc13dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:17.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.549+0000 7f04c1400640 1 --2- 172.21.15.152:0/1843744607 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f04bc0b13d0 0x7f04bc13dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:17.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.549+0000 7f04c1400640 1 -- 172.21.15.152:0/1843744607 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04bc1e7fb0 con 0x7f04bc0b17b0 2024-04-20T11:38:17.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.549+0000 7f04c2800640 1 --2- 172.21.15.152:0/1843744607 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f04bc1e75f0 0x7f04bc1e7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:38:17.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.549+0000 7f04c1400640 1 --2- 172.21.15.152:0/1843744607 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f04bc0b17b0 0x7f04bc13e290 secure :-1 s=READY pgs=2423 cs=0 l=1 rev1=1 crypto rx=0x7f04ac010810 tx=0x7f04ac010cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:17.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.549+0000 7f04b6000640 1 -- 172.21.15.152:0/1843744607 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04ac00f040 con 0x7f04bc0b17b0 2024-04-20T11:38:17.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.549+0000 7f04b6000640 1 -- 172.21.15.152:0/1843744607 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04ac00eb00 con 0x7f04bc0b17b0 2024-04-20T11:38:17.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.549+0000 7f04b6000640 1 -- 172.21.15.152:0/1843744607 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04ac01c400 con 0x7f04bc0b17b0 2024-04-20T11:38:17.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.549+0000 7f04c3200640 1 -- 172.21.15.152:0/1843744607 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04bc1e60f0 con 0x7f04bc0b17b0 2024-04-20T11:38:17.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.550+0000 7f04c3200640 1 -- 172.21.15.152:0/1843744607 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04bc1e66b0 con 0x7f04bc0b17b0 2024-04-20T11:38:17.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.551+0000 7f04c3200640 1 -- 172.21.15.152:0/1843744607 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04bc075e60 con 0x7f04bc0b17b0 2024-04-20T11:38:17.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.552+0000 7f04b6000640 1 -- 172.21.15.152:0/1843744607 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 27) ==== 92060+0+0 (secure 0 0 0) 0x7f04ac01c5a0 con 0x7f04bc0b17b0 2024-04-20T11:38:17.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.553+0000 7f04b6000640 1 --2- 172.21.15.152:0/1843744607 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f04a006d430 0x7f04a006f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:17.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.553+0000 7f04b6000640 1 -- 172.21.15.152:0/1843744607 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 4943+0+0 (secure 0 0 0) 0x7f04ac08ad60 con 0x7f04bc0b17b0 2024-04-20T11:38:17.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.553+0000 7f04c1e00640 1 --2- 172.21.15.152:0/1843744607 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f04a006d430 0x7f04a006f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:17.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.554+0000 7f04c1e00640 1 --2- 172.21.15.152:0/1843744607 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f04a006d430 0x7f04a006f8f0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f04b0001050 tx=0x7f04b0009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:17.558 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.557+0000 7f04b6000640 1 -- 172.21.15.152:0/1843744607 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f04ac018030 con 0x7f04bc0b17b0 2024-04-20T11:38:17.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.756+0000 7f04c3200640 1 -- 172.21.15.152:0/1843744607 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f04bc072d30 con 0x7f04bc0b17b0 2024-04-20T11:38:17.764 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.764+0000 7f04b6000640 1 -- 172.21.15.152:0/1843744607 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v31) ==== 104+0+0 (secure 0 0 0) 0x7f04ac05a340 con 0x7f04bc0b17b0 2024-04-20T11:38:17.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.766+0000 7f04c3200640 1 -- 172.21.15.152:0/1843744607 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f04a006d430 msgr2=0x7f04a006f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:17.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.766+0000 7f04c3200640 1 --2- 172.21.15.152:0/1843744607 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f04a006d430 0x7f04a006f8f0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f04b0001050 tx=0x7f04b0009040 comp rx=0 tx=0).stop 2024-04-20T11:38:17.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.766+0000 7f04c3200640 1 -- 172.21.15.152:0/1843744607 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f04bc0b17b0 msgr2=0x7f04bc13e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:17.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.766+0000 7f04c3200640 1 --2- 172.21.15.152:0/1843744607 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f04bc0b17b0 0x7f04bc13e290 secure :-1 s=READY pgs=2423 cs=0 l=1 rev1=1 crypto rx=0x7f04ac010810 tx=0x7f04ac010cd0 comp rx=0 tx=0).stop 2024-04-20T11:38:17.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.767+0000 7f04c3200640 1 -- 172.21.15.152:0/1843744607 shutdown_connections 2024-04-20T11:38:17.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.767+0000 7f04c3200640 1 --2- 172.21.15.152:0/1843744607 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f04a006d430 0x7f04a006f8f0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:17.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.767+0000 7f04c3200640 1 --2- 172.21.15.152:0/1843744607 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f04bc1e75f0 0x7f04bc1e7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:17.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.767+0000 7f04c3200640 1 --2- 172.21.15.152:0/1843744607 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f04bc0b17b0 0x7f04bc13e290 unknown :-1 s=CLOSED pgs=2423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:17.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.767+0000 7f04c3200640 1 --2- 172.21.15.152:0/1843744607 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f04bc0b13d0 0x7f04bc13dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:17.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.767+0000 7f04c3200640 1 -- 172.21.15.152:0/1843744607 >> 172.21.15.152:0/1843744607 conn(0x7f04bc067ae0 msgr2=0x7f04bc15d280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:17.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.767+0000 7f04c3200640 1 -- 172.21.15.152:0/1843744607 shutdown_connections 2024-04-20T11:38:17.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.767+0000 7f04c3200640 1 -- 172.21.15.152:0/1843744607 wait complete. 2024-04-20T11:38:17.787 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_unparseable_metafile_on_legacy_to_v1_upgrade' 2024-04-20T11:38:17.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.988+0000 7f18d2800640 1 Processor -- start 2024-04-20T11:38:17.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.988+0000 7f18d2800640 1 -- start start 2024-04-20T11:38:17.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.990+0000 7f18d2800640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f18cc1603d0 0x7f18cc0b5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:17.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.990+0000 7f18d2800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f18cc0b5480 0x7f18cc0b63f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:17.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.990+0000 7f18d2800640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18cc0d3350 con 0x7f18cc15fcc0 2024-04-20T11:38:17.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.990+0000 7f18d2800640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f18cc0d3630 con 0x7f18cc1603d0 2024-04-20T11:38:17.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.990+0000 7f18d2800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f18cc0d3910 con 0x7f18cc0b5480 2024-04-20T11:38:17.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.990+0000 7f18d1400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f18cc15fcc0 0x7f18cc0d4e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:44002/0 (socket says 172.21.15.152:44002) 2024-04-20T11:38:17.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.990+0000 7f18d1400640 1 -- 172.21.15.152:0/3156064683 learned_addr learned my addr 172.21.15.152:0/3156064683 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:38:17.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.991+0000 7f18d1e00640 1 --2- 172.21.15.152:0/3156064683 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f18cc0b5480 0x7f18cc0b63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:17.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.991+0000 7f18d0a00640 1 --2- 172.21.15.152:0/3156064683 >> v2:172.21.15.175:3300/0 conn(0x7f18cc1603d0 0x7f18cc0b5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:17.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.991+0000 7f18cbe00640 1 -- 172.21.15.152:0/3156064683 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1394154743 0 0) 0x7f18cc0d3350 con 0x7f18cc15fcc0 2024-04-20T11:38:17.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.991+0000 7f18cbe00640 1 -- 172.21.15.152:0/3156064683 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f18b4003520 con 0x7f18cc15fcc0 2024-04-20T11:38:17.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.991+0000 7f18cbe00640 1 -- 172.21.15.152:0/3156064683 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2635749596 0 0) 0x7f18b4003520 con 0x7f18cc15fcc0 2024-04-20T11:38:17.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.991+0000 7f18cbe00640 1 -- 172.21.15.152:0/3156064683 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f18cc0b5480 msgr2=0x7f18cc0b63f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:17.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.991+0000 7f18cbe00640 1 --2- 172.21.15.152:0/3156064683 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f18cc0b5480 0x7f18cc0b63f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:17.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.991+0000 7f18cbe00640 1 -- 172.21.15.152:0/3156064683 >> v2:172.21.15.175:3300/0 conn(0x7f18cc1603d0 msgr2=0x7f18cc0b5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:17.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.991+0000 7f18cbe00640 1 --2- 172.21.15.152:0/3156064683 >> v2:172.21.15.175:3300/0 conn(0x7f18cc1603d0 0x7f18cc0b5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:17.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.991+0000 7f18cbe00640 1 -- 172.21.15.152:0/3156064683 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18cc0ced20 con 0x7f18cc15fcc0 2024-04-20T11:38:17.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.991+0000 7f18cbe00640 1 -- 172.21.15.152:0/3156064683 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f18bc001670 con 0x7f18cc15fcc0 2024-04-20T11:38:17.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.991+0000 7f18cbe00640 1 -- 172.21.15.152:0/3156064683 >> v1:172.21.15.152:6789/0 conn(0x7f18cc15fcc0 legacy=0x7f18cc0d4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:17.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.991+0000 7f18cbe00640 1 --2- 172.21.15.152:0/3156064683 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f18cc0b5480 0x7f18b40039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:17.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.991+0000 7f18cbe00640 1 --2- 172.21.15.152:0/3156064683 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18b4003f30 0x7f18b4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:17.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.991+0000 7f18cbe00640 1 --2- 172.21.15.152:0/3156064683 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f18b4004d40 0x7f18b40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:17.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.991+0000 7f18cbe00640 1 -- 172.21.15.152:0/3156064683 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f18cc0d3630 con 0x7f18b4004d40 2024-04-20T11:38:17.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.991+0000 7f18cbe00640 1 -- 172.21.15.152:0/3156064683 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f18cc0d3910 con 0x7f18b4003f30 2024-04-20T11:38:17.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.991+0000 7f18cbe00640 1 -- 172.21.15.152:0/3156064683 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f18b400b9c0 con 0x7f18cc0b5480 2024-04-20T11:38:17.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.992+0000 7f18d1e00640 1 --2- 172.21.15.152:0/3156064683 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f18b4004d40 0x7f18b40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:17.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.992+0000 7f18d0a00640 1 --2- 172.21.15.152:0/3156064683 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18b4003f30 0x7f18b4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:17.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.992+0000 7f18d1400640 1 --2- 172.21.15.152:0/3156064683 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f18cc0b5480 0x7f18b40039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:17.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.992+0000 7f18d1e00640 1 -- 172.21.15.152:0/3156064683 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f18cc0b5480 msgr2=0x7f18b40039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:17.995 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.992+0000 7f18d1e00640 1 --2- 172.21.15.152:0/3156064683 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f18cc0b5480 0x7f18b40039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:17.995 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.992+0000 7f18d1e00640 1 -- 172.21.15.152:0/3156064683 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18b4003f30 msgr2=0x7f18b4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:17.995 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.992+0000 7f18d1e00640 1 --2- 172.21.15.152:0/3156064683 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18b4003f30 0x7f18b4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:17.995 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.992+0000 7f18d1e00640 1 -- 172.21.15.152:0/3156064683 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18b4003520 con 0x7f18b4004d40 2024-04-20T11:38:17.995 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.992+0000 7f18d1400640 1 --2- 172.21.15.152:0/3156064683 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f18cc0b5480 0x7f18b40039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:38:17.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.993+0000 7f18d1e00640 1 --2- 172.21.15.152:0/3156064683 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f18b4004d40 0x7f18b40051c0 secure :-1 s=READY pgs=2425 cs=0 l=1 rev1=1 crypto rx=0x7f18c4009a90 tx=0x7f18b4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4da26c2b0e4e9ba9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:17.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.993+0000 7f18cbe00640 1 -- 172.21.15.152:0/3156064683 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c4039070 con 0x7f18b4004d40 2024-04-20T11:38:17.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.994+0000 7f18cbe00640 1 -- 172.21.15.152:0/3156064683 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18c402c670 con 0x7f18b4004d40 2024-04-20T11:38:17.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.994+0000 7f18cbe00640 1 -- 172.21.15.152:0/3156064683 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c4009690 con 0x7f18b4004d40 2024-04-20T11:38:17.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.994+0000 7f18d2800640 1 -- 172.21.15.152:0/3156064683 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f18b4004d40 msgr2=0x7f18b40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:17.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.994+0000 7f18d2800640 1 --2- 172.21.15.152:0/3156064683 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f18b4004d40 0x7f18b40051c0 secure :-1 s=READY pgs=2425 cs=0 l=1 rev1=1 crypto rx=0x7f18c4009a90 tx=0x7f18b4005830 comp rx=0 tx=0).stop 2024-04-20T11:38:17.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.994+0000 7f18d1e00640 1 -- 172.21.15.152:0/3156064683 reap_dead start 2024-04-20T11:38:17.997 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.994+0000 7f18d2800640 1 -- 172.21.15.152:0/3156064683 shutdown_connections 2024-04-20T11:38:17.997 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.994+0000 7f18d2800640 1 -- 172.21.15.152:0/3156064683 >> 172.21.15.152:0/3156064683 conn(0x7f18cc08bfd0 msgr2=0x7f18cc0d4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:17.997 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.994+0000 7f18d2800640 1 -- 172.21.15.152:0/3156064683 shutdown_connections 2024-04-20T11:38:17.997 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.994+0000 7f18d2800640 1 -- 172.21.15.152:0/3156064683 wait complete. 2024-04-20T11:38:17.997 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.996+0000 7f18d2800640 1 Processor -- start 2024-04-20T11:38:17.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.996+0000 7f18d2800640 1 -- start start 2024-04-20T11:38:17.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.997+0000 7f18d2800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f18b4004d40 0x7f18cc0b9590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:17.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.997+0000 7f18d2800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18cc0b6f00 0x7f18cc06cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:17.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.997+0000 7f18d2800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f18cc156670 0x7f18cc156ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:17.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.997+0000 7f18d2800640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f18cc0b67c0 con 0x7f18cc156670 2024-04-20T11:38:17.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.997+0000 7f18d2800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f18cc0b6640 con 0x7f18cc0b6f00 2024-04-20T11:38:17.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.997+0000 7f18d2800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f18cc0cf240 con 0x7f18b4004d40 2024-04-20T11:38:18.000 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.997+0000 7f18d1e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f18cc156670 0x7f18cc156ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:18.000 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.997+0000 7f18d1e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f18cc156670 0x7f18cc156ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54662/0 (socket says 172.21.15.152:54662) 2024-04-20T11:38:18.000 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.998+0000 7f18d1e00640 1 -- 172.21.15.152:0/2415821700 learned_addr learned my addr 172.21.15.152:0/2415821700 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:38:18.000 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.998+0000 7f18d0a00640 1 --2- 172.21.15.152:0/2415821700 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18cc0b6f00 0x7f18cc06cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:18.000 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.998+0000 7f18d1400640 1 --2- 172.21.15.152:0/2415821700 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f18b4004d40 0x7f18cc0b9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:18.001 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.998+0000 7f18d1e00640 1 -- 172.21.15.152:0/2415821700 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f18b4004d40 msgr2=0x7f18cc0b9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:18.001 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.998+0000 7f18d1e00640 1 --2- 172.21.15.152:0/2415821700 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f18b4004d40 0x7f18cc0b9590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:18.001 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.998+0000 7f18d1e00640 1 -- 172.21.15.152:0/2415821700 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18cc0b6f00 msgr2=0x7f18cc06cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:18.001 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.998+0000 7f18d1e00640 1 --2- 172.21.15.152:0/2415821700 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18cc0b6f00 0x7f18cc06cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:18.001 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.998+0000 7f18d1e00640 1 -- 172.21.15.152:0/2415821700 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18cc157010 con 0x7f18cc156670 2024-04-20T11:38:18.001 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.998+0000 7f18d1400640 1 --2- 172.21.15.152:0/2415821700 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f18b4004d40 0x7f18cc0b9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:38:18.001 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.998+0000 7f18d0a00640 1 --2- 172.21.15.152:0/2415821700 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18cc0b6f00 0x7f18cc06cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:38:18.001 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.998+0000 7f18d1e00640 1 --2- 172.21.15.152:0/2415821700 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f18cc156670 0x7f18cc156ad0 secure :-1 s=READY pgs=2426 cs=0 l=1 rev1=1 crypto rx=0x7f18c4009bb0 tx=0x7f18c4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:18.001 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.998+0000 7f18c9600640 1 -- 172.21.15.152:0/2415821700 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c4039070 con 0x7f18cc156670 2024-04-20T11:38:18.001 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.999+0000 7f18c9600640 1 -- 172.21.15.152:0/2415821700 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18c4043070 con 0x7f18cc156670 2024-04-20T11:38:18.001 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.999+0000 7f18c9600640 1 -- 172.21.15.152:0/2415821700 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c4009690 con 0x7f18cc156670 2024-04-20T11:38:18.001 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.999+0000 7f18d2800640 1 -- 172.21.15.152:0/2415821700 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18cc0b74c0 con 0x7f18cc156670 2024-04-20T11:38:18.001 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:17.999+0000 7f18d2800640 1 -- 172.21.15.152:0/2415821700 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18cc0b7a80 con 0x7f18cc156670 2024-04-20T11:38:18.002 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:18.000+0000 7f18d2800640 1 -- 172.21.15.152:0/2415821700 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18cc075e60 con 0x7f18cc156670 2024-04-20T11:38:18.002 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:18.001+0000 7f18c9600640 1 -- 172.21.15.152:0/2415821700 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 27) ==== 92060+0+0 (secure 0 0 0) 0x7f18c4034a90 con 0x7f18cc156670 2024-04-20T11:38:18.002 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:18.002+0000 7f18c9600640 1 --2- 172.21.15.152:0/2415821700 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f18b006d430 0x7f18b006f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:18.003 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:18.002+0000 7f18c9600640 1 -- 172.21.15.152:0/2415821700 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 4943+0+0 (secure 0 0 0) 0x7f18c40ab1d0 con 0x7f18cc156670 2024-04-20T11:38:18.003 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:18.003+0000 7f18d1400640 1 --2- 172.21.15.152:0/2415821700 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f18b006d430 0x7f18b006f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:18.004 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:18.004+0000 7f18d1400640 1 --2- 172.21.15.152:0/2415821700 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f18b006d430 0x7f18b006f8f0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f18bc007b70 tx=0x7f18bc009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:18.006 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:18.006+0000 7f18c9600640 1 -- 172.21.15.152:0/2415821700 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f18c407b050 con 0x7f18cc156670 2024-04-20T11:38:18.262 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:18.261+0000 7f18d2800640 1 -- 172.21.15.152:0/2415821700 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_unparseable_metafile_on_legacy_to_v1_upgrade"]} v 0) -- 0x7f18cc0b6ce0 con 0x7f18cc156670 2024-04-20T11:38:18.320 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:18.320+0000 7f18c9600640 1 -- 172.21.15.152:0/2415821700 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_unparseable_metafile_on_legacy_to_v1_upgrade"]}]=0 v606) ==== 163+0+0 (secure 0 0 0) 0x7f18c407a950 con 0x7f18cc156670 2024-04-20T11:38:18.323 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:18.322+0000 7f18d2800640 1 -- 172.21.15.152:0/2415821700 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f18b006d430 msgr2=0x7f18b006f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:18.324 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:18.322+0000 7f18d2800640 1 --2- 172.21.15.152:0/2415821700 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f18b006d430 0x7f18b006f8f0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f18bc007b70 tx=0x7f18bc009040 comp rx=0 tx=0).stop 2024-04-20T11:38:18.324 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:18.322+0000 7f18d2800640 1 -- 172.21.15.152:0/2415821700 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f18cc156670 msgr2=0x7f18cc156ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:18.324 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:18.322+0000 7f18d2800640 1 --2- 172.21.15.152:0/2415821700 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f18cc156670 0x7f18cc156ad0 secure :-1 s=READY pgs=2426 cs=0 l=1 rev1=1 crypto rx=0x7f18c4009bb0 tx=0x7f18c4004730 comp rx=0 tx=0).stop 2024-04-20T11:38:18.324 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:18.323+0000 7f18d2800640 1 -- 172.21.15.152:0/2415821700 shutdown_connections 2024-04-20T11:38:18.324 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:18.323+0000 7f18d2800640 1 --2- 172.21.15.152:0/2415821700 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f18b006d430 0x7f18b006f8f0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:18.324 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:18.323+0000 7f18d2800640 1 --2- 172.21.15.152:0/2415821700 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f18cc156670 0x7f18cc156ad0 unknown :-1 s=CLOSED pgs=2426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:18.324 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:18.323+0000 7f18d2800640 1 --2- 172.21.15.152:0/2415821700 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18cc0b6f00 0x7f18cc06cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:18.324 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:18.323+0000 7f18d2800640 1 --2- 172.21.15.152:0/2415821700 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f18b4004d40 0x7f18cc0b9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:18.324 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:18.323+0000 7f18d2800640 1 -- 172.21.15.152:0/2415821700 >> 172.21.15.152:0/2415821700 conn(0x7f18cc08bfd0 msgr2=0x7f18cc0d3b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:18.324 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:18.323+0000 7f18d2800640 1 -- 172.21.15.152:0/2415821700 shutdown_connections 2024-04-20T11:38:18.324 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:18.323+0000 7f18d2800640 1 -- 172.21.15.152:0/2415821700 wait complete. 2024-04-20T11:38:18.341 INFO:tasks.cephfs_test_runner:Validate unparseable text .meta file on legacy subvol root doesn't break the system ... ok 2024-04-20T11:38:18.341 INFO:tasks.cephfs_test_runner: 2024-04-20T11:38:18.342 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-20T11:38:18.342 INFO:tasks.cephfs_test_runner:Ran 11 tests in 640.050s 2024-04-20T11:38:18.342 INFO:tasks.cephfs_test_runner: 2024-04-20T11:38:18.342 INFO:tasks.cephfs_test_runner:OK 2024-04-20T11:38:18.342 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-04-20T11:38:18.353 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-04-20T11:38:18.360 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-04-20T11:38:18.360 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-20T11:38:18.360 DEBUG:teuthology.orchestra.run.smithi152:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:38:18.409 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-20T11:38:18.409 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-20T11:38:18.409 DEBUG:teuthology.orchestra.run.smithi152:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:38:18.505 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-20T11:38:18.505 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T11:38:18.505 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:38:18.563 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-20T11:38:18.563 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T11:38:18.563 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:38:18.616 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-20T11:38:18.616 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:38:18.616 DEBUG:teuthology.orchestra.run.smithi152:> ip netns list 2024-04-20T11:38:18.673 INFO:teuthology.orchestra.run.smithi152.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-20T11:38:18.674 INFO:teuthology.orchestra.run.smithi152.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-20T11:38:18.675 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:38:18.675 DEBUG:teuthology.orchestra.run.smithi152:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T11:38:18.762 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:38:18.762 DEBUG:teuthology.orchestra.run.smithi152:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T11:38:18.834 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:38:18.834 DEBUG:teuthology.orchestra.run.smithi152:> sudo ip link delete ceph-brx 2024-04-20T11:38:18.933 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:38:18.933 DEBUG:teuthology.orchestra.run.smithi175:> ip netns list 2024-04-20T11:38:18.990 INFO:teuthology.orchestra.run.smithi175.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-20T11:38:18.990 INFO:teuthology.orchestra.run.smithi175.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-20T11:38:18.991 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:38:18.991 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-20T11:38:19.079 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:38:19.079 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-20T11:38:19.157 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:38:19.157 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip link delete ceph-brx 2024-04-20T11:38:19.272 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-20T11:38:19.281 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-20T11:38:19.282 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T11:38:19.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.494+0000 7fa216000640 1 Processor -- start 2024-04-20T11:38:19.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.494+0000 7fa216000640 1 -- start start 2024-04-20T11:38:19.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.496+0000 7fa216000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fa210147410 0x7fa2100afa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:19.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.496+0000 7fa216000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa21006e720 0x7fa210072130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:19.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.496+0000 7fa216000640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2100aec90 con 0x7fa210071cd0 2024-04-20T11:38:19.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.496+0000 7fa216000640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fa210072670 con 0x7fa210147410 2024-04-20T11:38:19.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.496+0000 7fa216000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa2100aef70 con 0x7fa21006e720 2024-04-20T11:38:19.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.497+0000 7fa207e00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fa210071cd0 0x7fa21006e010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:44018/0 (socket says 172.21.15.152:44018) 2024-04-20T11:38:19.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.497+0000 7fa207e00640 1 -- 172.21.15.152:0/3173228114 learned_addr learned my addr 172.21.15.152:0/3173228114 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:38:19.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.497+0000 7fa214c00640 1 --2- 172.21.15.152:0/3173228114 >> v2:172.21.15.152:3300/0 conn(0x7fa210147410 0x7fa2100afa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:19.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.497+0000 7fa215600640 1 --2- 172.21.15.152:0/3173228114 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa21006e720 0x7fa210072130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:19.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.497+0000 7fa20fe00640 1 -- 172.21.15.152:0/3173228114 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1298265130 0 0) 0x7fa2100aec90 con 0x7fa210071cd0 2024-04-20T11:38:19.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.497+0000 7fa214c00640 1 -- 172.21.15.152:0/3173228114 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa21006e720 msgr2=0x7fa210072130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:19.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.497+0000 7fa214c00640 1 --2- 172.21.15.152:0/3173228114 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa21006e720 0x7fa210072130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:19.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.497+0000 7fa214c00640 1 -- 172.21.15.152:0/3173228114 >> v1:172.21.15.152:6789/0 conn(0x7fa210071cd0 legacy=0x7fa21006e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:19.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.497+0000 7fa214c00640 1 -- 172.21.15.152:0/3173228114 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2100af250 con 0x7fa210147410 2024-04-20T11:38:19.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.498+0000 7fa214c00640 1 --2- 172.21.15.152:0/3173228114 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa210147410 0x7fa2100afa50 secure :-1 s=READY pgs=2428 cs=0 l=1 rev1=1 crypto rx=0x7fa2000097a0 tx=0x7fa20002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccfe2259cd0d5ec7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:19.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.498+0000 7fa20fe00640 1 -- 172.21.15.152:0/3173228114 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa200035020 con 0x7fa210147410 2024-04-20T11:38:19.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.498+0000 7fa20fe00640 1 -- 172.21.15.152:0/3173228114 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa200031830 con 0x7fa210147410 2024-04-20T11:38:19.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.498+0000 7fa20fe00640 1 -- 172.21.15.152:0/3173228114 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa200031c00 con 0x7fa210147410 2024-04-20T11:38:19.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.499+0000 7fa216000640 1 -- 172.21.15.152:0/3173228114 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa210147410 msgr2=0x7fa2100afa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:19.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.499+0000 7fa216000640 1 --2- 172.21.15.152:0/3173228114 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa210147410 0x7fa2100afa50 secure :-1 s=READY pgs=2428 cs=0 l=1 rev1=1 crypto rx=0x7fa2000097a0 tx=0x7fa20002b8d0 comp rx=0 tx=0).stop 2024-04-20T11:38:19.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.499+0000 7fa216000640 1 -- 172.21.15.152:0/3173228114 shutdown_connections 2024-04-20T11:38:19.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.499+0000 7fa216000640 1 --2- 172.21.15.152:0/3173228114 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa21006e720 0x7fa210072130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:19.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.499+0000 7fa216000640 1 --2- 172.21.15.152:0/3173228114 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa210147410 0x7fa2100afa50 unknown :-1 s=CLOSED pgs=2428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:19.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.499+0000 7fa216000640 1 -- 172.21.15.152:0/3173228114 >> 172.21.15.152:0/3173228114 conn(0x7fa210067ae0 msgr2=0x7fa21006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:19.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.499+0000 7fa216000640 1 -- 172.21.15.152:0/3173228114 shutdown_connections 2024-04-20T11:38:19.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.499+0000 7fa216000640 1 -- 172.21.15.152:0/3173228114 wait complete. 2024-04-20T11:38:19.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.501+0000 7fa216000640 1 Processor -- start 2024-04-20T11:38:19.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.501+0000 7fa216000640 1 -- start start 2024-04-20T11:38:19.503 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.502+0000 7fa216000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa21006e720 0x7fa21015d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:19.503 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.502+0000 7fa216000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa210071cd0 0x7fa21013d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:19.503 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.502+0000 7fa216000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa210147410 0x7fa21013de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:19.503 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.502+0000 7fa216000640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fa2100b3390 con 0x7fa21006e720 2024-04-20T11:38:19.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.502+0000 7fa216000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa2100ae980 con 0x7fa210071cd0 2024-04-20T11:38:19.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.502+0000 7fa216000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa2100b3510 con 0x7fa210147410 2024-04-20T11:38:19.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.502+0000 7fa214c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa21006e720 0x7fa21015d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:19.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.502+0000 7fa214c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa21006e720 0x7fa21015d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54674/0 (socket says 172.21.15.152:54674) 2024-04-20T11:38:19.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.502+0000 7fa214c00640 1 -- 172.21.15.152:0/1930229426 learned_addr learned my addr 172.21.15.152:0/1930229426 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:38:19.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.503+0000 7fa214c00640 1 -- 172.21.15.152:0/1930229426 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa210147410 msgr2=0x7fa21013de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:19.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.503+0000 7fa214c00640 1 --2- 172.21.15.152:0/1930229426 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa210147410 0x7fa21013de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:19.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.503+0000 7fa214c00640 1 -- 172.21.15.152:0/1930229426 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa210071cd0 msgr2=0x7fa21013d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:19.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.503+0000 7fa214c00640 1 --2- 172.21.15.152:0/1930229426 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa210071cd0 0x7fa21013d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:19.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.503+0000 7fa214c00640 1 -- 172.21.15.152:0/1930229426 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa210160650 con 0x7fa21006e720 2024-04-20T11:38:19.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.503+0000 7fa214c00640 1 --2- 172.21.15.152:0/1930229426 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa21006e720 0x7fa21015d830 secure :-1 s=READY pgs=2429 cs=0 l=1 rev1=1 crypto rx=0x7fa2000097a0 tx=0x7fa2000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:19.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.503+0000 7fa20d600640 1 -- 172.21.15.152:0/1930229426 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa200035040 con 0x7fa21006e720 2024-04-20T11:38:19.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.503+0000 7fa20d600640 1 -- 172.21.15.152:0/1930229426 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa200004230 con 0x7fa21006e720 2024-04-20T11:38:19.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.503+0000 7fa20d600640 1 -- 172.21.15.152:0/1930229426 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa20003a700 con 0x7fa21006e720 2024-04-20T11:38:19.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.503+0000 7fa216000640 1 -- 172.21.15.152:0/1930229426 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa210160930 con 0x7fa21006e720 2024-04-20T11:38:19.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.504+0000 7fa216000640 1 -- 172.21.15.152:0/1930229426 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa210160ef0 con 0x7fa21006e720 2024-04-20T11:38:19.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.504+0000 7fa216000640 1 -- 172.21.15.152:0/1930229426 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa210075e60 con 0x7fa21006e720 2024-04-20T11:38:19.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.505+0000 7fa20d600640 1 -- 172.21.15.152:0/1930229426 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 27) ==== 92060+0+0 (secure 0 0 0) 0x7fa200038020 con 0x7fa21006e720 2024-04-20T11:38:19.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.506+0000 7fa20d600640 1 --2- 172.21.15.152:0/1930229426 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fa1ec06d430 0x7fa1ec06f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:19.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.506+0000 7fa20d600640 1 -- 172.21.15.152:0/1930229426 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 4943+0+0 (secure 0 0 0) 0x7fa2000ac390 con 0x7fa21006e720 2024-04-20T11:38:19.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.507+0000 7fa207e00640 1 --2- 172.21.15.152:0/1930229426 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fa1ec06d430 0x7fa1ec06f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:19.509 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.508+0000 7fa207e00640 1 --2- 172.21.15.152:0/1930229426 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fa1ec06d430 0x7fa1ec06f8f0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fa1f8007fd0 tx=0x7fa1f8007920 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:19.512 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.512+0000 7fa20d600640 1 -- 172.21.15.152:0/1930229426 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fa20007c210 con 0x7fa21006e720 2024-04-20T11:38:19.707 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.706+0000 7fa216000640 1 -- 172.21.15.152:0/1930229426 --> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa210141b80 con 0x7fa1ec06d430 2024-04-20T11:38:19.711 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.709+0000 7fa20d600640 1 -- 172.21.15.152:0/1930229426 <== mgr.4109 v2:172.21.15.175:6836/4021250441 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13498 (secure 0 0 0) 0x7fa210141b80 con 0x7fa1ec06d430 2024-04-20T11:38:19.711 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:38:19.712 INFO:teuthology.orchestra.run.smithi152.stderr:dumped all 2024-04-20T11:38:19.713 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.711+0000 7fa216000640 1 -- 172.21.15.152:0/1930229426 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fa1ec06d430 msgr2=0x7fa1ec06f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:19.713 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.711+0000 7fa216000640 1 --2- 172.21.15.152:0/1930229426 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fa1ec06d430 0x7fa1ec06f8f0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fa1f8007fd0 tx=0x7fa1f8007920 comp rx=0 tx=0).stop 2024-04-20T11:38:19.713 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.711+0000 7fa216000640 1 -- 172.21.15.152:0/1930229426 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa21006e720 msgr2=0x7fa21015d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:19.713 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.711+0000 7fa216000640 1 --2- 172.21.15.152:0/1930229426 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa21006e720 0x7fa21015d830 secure :-1 s=READY pgs=2429 cs=0 l=1 rev1=1 crypto rx=0x7fa2000097a0 tx=0x7fa2000046c0 comp rx=0 tx=0).stop 2024-04-20T11:38:19.714 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.711+0000 7fa216000640 1 -- 172.21.15.152:0/1930229426 shutdown_connections 2024-04-20T11:38:19.714 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.711+0000 7fa216000640 1 --2- 172.21.15.152:0/1930229426 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fa1ec06d430 0x7fa1ec06f8f0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:19.714 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.711+0000 7fa216000640 1 --2- 172.21.15.152:0/1930229426 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa210147410 0x7fa21013de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:19.714 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.711+0000 7fa216000640 1 --2- 172.21.15.152:0/1930229426 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa210071cd0 0x7fa21013d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:19.714 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.711+0000 7fa216000640 1 --2- 172.21.15.152:0/1930229426 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa21006e720 0x7fa21015d830 unknown :-1 s=CLOSED pgs=2429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:19.714 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.711+0000 7fa216000640 1 -- 172.21.15.152:0/1930229426 >> 172.21.15.152:0/1930229426 conn(0x7fa210067ae0 msgr2=0x7fa2101420a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:19.714 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.712+0000 7fa216000640 1 -- 172.21.15.152:0/1930229426 shutdown_connections 2024-04-20T11:38:19.714 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.712+0000 7fa216000640 1 -- 172.21.15.152:0/1930229426 wait complete. 2024-04-20T11:38:19.731 INFO:teuthology.orchestra.run.smithi152.stdout:{"pg_ready":true,"pg_map":{"version":280,"stamp":"2024-04-20T11:38:18.729457+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":148,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":4,"kb":754974720,"kb_used":271556,"kb_used_data":17944,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754703164,"statfs":{"total":773094113280,"available":772816039936,"internally_reserved":0,"allocated":18374656,"data_stored":8773961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12746,"internal_metadata":259575350},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-58692,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":-4,"num_read_kb":-4,"num_write":-89,"num_write_kb":-91,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001834"},"pg_stats":[{"pgid":"1.0","version":"72'128","reported_seq":478,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-20T11:38:18.192054+0000","last_change":"2024-04-20T11:26:48.661897+0000","last_active":"2024-04-20T11:38:18.192054+0000","last_peered":"2024-04-20T11:38:18.192054+0000","last_clean":"2024-04-20T11:38:18.192054+0000","last_became_active":"2024-04-20T11:26:48.656535+0000","last_became_peered":"2024-04-20T11:26:48.656535+0000","last_unstale":"2024-04-20T11:38:18.192054+0000","last_undegraded":"2024-04-20T11:38:18.192054+0000","last_fullsized":"2024-04-20T11:38:18.192054+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:47.543004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:47.543004+0000","last_clean_scrub_stamp":"2024-04-20T11:26:47.543004+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:58:35.527105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574992,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33868,"kb_used_data":2684,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337972,"statfs":{"total":96636764160,"available":96602083328,"internally_reserved":0,"allocated":2748416,"data_stored":1541608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574989,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33284,"kb_used_data":2104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338556,"statfs":{"total":96636764160,"available":96602681344,"internally_reserved":0,"allocated":2154496,"data_stored":951240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574991,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33284,"kb_used_data":2104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338556,"statfs":{"total":96636764160,"available":96602681344,"internally_reserved":0,"allocated":2154496,"data_stored":951240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574989,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33268,"kb_used_data":2088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338572,"statfs":{"total":96636764160,"available":96602697728,"internally_reserved":0,"allocated":2138112,"data_stored":945675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574990,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33288,"kb_used_data":2104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338552,"statfs":{"total":96636764160,"available":96602677248,"internally_reserved":0,"allocated":2154496,"data_stored":951240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574991,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33268,"kb_used_data":2088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338572,"statfs":{"total":96636764160,"available":96602697728,"internally_reserved":0,"allocated":2138112,"data_stored":945675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574989,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33288,"kb_used_data":2104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338552,"statfs":{"total":96636764160,"available":96602677248,"internally_reserved":0,"allocated":2154496,"data_stored":951240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574990,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38008,"kb_used_data":2668,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333832,"statfs":{"total":96636764160,"available":96597843968,"internally_reserved":0,"allocated":2732032,"data_stored":1536043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T11:38:19.732 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T11:38:19.932 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.931+0000 7f242cc00640 1 Processor -- start 2024-04-20T11:38:19.932 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.931+0000 7f242cc00640 1 -- start start 2024-04-20T11:38:19.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.933+0000 7f242cc00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f242815d1e0 0x7f2428160b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:19.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.933+0000 7f242cc00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f242815d5c0 0x7f24280cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:19.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.933+0000 7f242cc00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24280ce8c0 con 0x7f2428071cd0 2024-04-20T11:38:19.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.933+0000 7f242cc00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f24281600b0 con 0x7f242815d1e0 2024-04-20T11:38:19.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.933+0000 7f242cc00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f2428160390 con 0x7f242815d5c0 2024-04-20T11:38:19.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.933+0000 7f2427400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f242815d1e0 0x7f2428160b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:19.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.933+0000 7f2427400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f242815d1e0 0x7f2428160b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54680/0 (socket says 172.21.15.152:54680) 2024-04-20T11:38:19.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.933+0000 7f2427400640 1 -- 172.21.15.152:0/1405088867 learned_addr learned my addr 172.21.15.152:0/1405088867 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:38:19.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.934+0000 7f2427400640 1 -- 172.21.15.152:0/1405088867 >> v2:172.21.15.175:3300/0 conn(0x7f242815d5c0 msgr2=0x7f24280cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:19.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.934+0000 7f2427400640 1 --2- 172.21.15.152:0/1405088867 >> v2:172.21.15.175:3300/0 conn(0x7f242815d5c0 0x7f24280cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:19.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.934+0000 7f2427400640 1 -- 172.21.15.152:0/1405088867 >> v1:172.21.15.175:6789/0 conn(0x7f2428071cd0 legacy=0x7f2428072120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:19.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.934+0000 7f2427400640 1 -- 172.21.15.152:0/1405088867 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2428160670 con 0x7f242815d1e0 2024-04-20T11:38:19.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.935+0000 7f2427400640 1 --2- 172.21.15.152:0/1405088867 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f242815d1e0 0x7f2428160b90 secure :-1 s=READY pgs=2430 cs=0 l=1 rev1=1 crypto rx=0x7f2418009870 tx=0x7f241802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90e0e95ff871f5e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:19.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.935+0000 7f2426000640 1 -- 172.21.15.152:0/1405088867 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2418035020 con 0x7f242815d1e0 2024-04-20T11:38:19.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.935+0000 7f2426000640 1 -- 172.21.15.152:0/1405088867 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2418031870 con 0x7f242815d1e0 2024-04-20T11:38:19.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.935+0000 7f2426000640 1 -- 172.21.15.152:0/1405088867 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2418031c40 con 0x7f242815d1e0 2024-04-20T11:38:19.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.936+0000 7f242cc00640 1 -- 172.21.15.152:0/1405088867 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f242815d1e0 msgr2=0x7f2428160b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:19.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.936+0000 7f242cc00640 1 --2- 172.21.15.152:0/1405088867 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f242815d1e0 0x7f2428160b90 secure :-1 s=READY pgs=2430 cs=0 l=1 rev1=1 crypto rx=0x7f2418009870 tx=0x7f241802b910 comp rx=0 tx=0).stop 2024-04-20T11:38:19.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.936+0000 7f242cc00640 1 -- 172.21.15.152:0/1405088867 shutdown_connections 2024-04-20T11:38:19.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.936+0000 7f242cc00640 1 --2- 172.21.15.152:0/1405088867 >> v2:172.21.15.175:3300/0 conn(0x7f242815d5c0 0x7f24280cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:19.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.936+0000 7f242cc00640 1 --2- 172.21.15.152:0/1405088867 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f242815d1e0 0x7f2428160b90 unknown :-1 s=CLOSED pgs=2430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:19.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.936+0000 7f242cc00640 1 -- 172.21.15.152:0/1405088867 >> 172.21.15.152:0/1405088867 conn(0x7f2428067ae0 msgr2=0x7f242806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:19.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.936+0000 7f242cc00640 1 -- 172.21.15.152:0/1405088867 shutdown_connections 2024-04-20T11:38:19.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.936+0000 7f242cc00640 1 -- 172.21.15.152:0/1405088867 wait complete. 2024-04-20T11:38:19.938 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.938+0000 7f242cc00640 1 Processor -- start 2024-04-20T11:38:19.938 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.938+0000 7f242cc00640 1 -- start start 2024-04-20T11:38:19.939 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.939+0000 7f242cc00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2428071cd0 0x7f2428156530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:19.939 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.939+0000 7f242cc00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f242815d1e0 0x7f2428147cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:19.939 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.939+0000 7f242cc00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f242815d5c0 0x7f24281589b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:19.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.939+0000 7f242cc00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f24280ce6a0 con 0x7f242815d1e0 2024-04-20T11:38:19.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.939+0000 7f242cc00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f24280ce520 con 0x7f2428071cd0 2024-04-20T11:38:19.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.939+0000 7f242cc00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f24280ccd70 con 0x7f242815d5c0 2024-04-20T11:38:19.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.939+0000 7f2426a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f242815d1e0 0x7f2428147cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:19.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.939+0000 7f2426a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f242815d1e0 0x7f2428147cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54682/0 (socket says 172.21.15.152:54682) 2024-04-20T11:38:19.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.939+0000 7f2426a00640 1 -- 172.21.15.152:0/3855534644 learned_addr learned my addr 172.21.15.152:0/3855534644 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:38:19.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.939+0000 7f2427400640 1 --2- 172.21.15.152:0/3855534644 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2428071cd0 0x7f2428156530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:19.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.939+0000 7f2426a00640 1 -- 172.21.15.152:0/3855534644 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f242815d5c0 msgr2=0x7f24281589b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:19.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.939+0000 7f2426a00640 1 --2- 172.21.15.152:0/3855534644 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f242815d5c0 0x7f24281589b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:19.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.939+0000 7f2426a00640 1 -- 172.21.15.152:0/3855534644 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2428071cd0 msgr2=0x7f2428156530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:19.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.939+0000 7f2426a00640 1 --2- 172.21.15.152:0/3855534644 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2428071cd0 0x7f2428156530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:19.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.939+0000 7f2426a00640 1 -- 172.21.15.152:0/3855534644 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2428148210 con 0x7f242815d1e0 2024-04-20T11:38:19.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.940+0000 7f2426a00640 1 --2- 172.21.15.152:0/3855534644 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f242815d1e0 0x7f2428147cd0 secure :-1 s=READY pgs=2431 cs=0 l=1 rev1=1 crypto rx=0x7f241400b440 tx=0x7f241400b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:19.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.940+0000 7f241f400640 1 -- 172.21.15.152:0/3855534644 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2414015020 con 0x7f242815d1e0 2024-04-20T11:38:19.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.940+0000 7f241f400640 1 -- 172.21.15.152:0/3855534644 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2414011890 con 0x7f242815d1e0 2024-04-20T11:38:19.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.940+0000 7f242cc00640 1 -- 172.21.15.152:0/3855534644 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24280af0f0 con 0x7f242815d1e0 2024-04-20T11:38:19.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.940+0000 7f241f400640 1 -- 172.21.15.152:0/3855534644 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2414011c80 con 0x7f242815d1e0 2024-04-20T11:38:19.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.940+0000 7f242cc00640 1 -- 172.21.15.152:0/3855534644 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2428153ae0 con 0x7f242815d1e0 2024-04-20T11:38:19.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.941+0000 7f242cc00640 1 -- 172.21.15.152:0/3855534644 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2428075e60 con 0x7f242815d1e0 2024-04-20T11:38:19.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.942+0000 7f241f400640 1 -- 172.21.15.152:0/3855534644 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 27) ==== 92060+0+0 (secure 0 0 0) 0x7f2414018020 con 0x7f242815d1e0 2024-04-20T11:38:19.944 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.943+0000 7f241f400640 1 --2- 172.21.15.152:0/3855534644 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f240806d430 0x7f240806f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:19.944 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.943+0000 7f241f400640 1 -- 172.21.15.152:0/3855534644 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 4943+0+0 (secure 0 0 0) 0x7f241408a5b0 con 0x7f242815d1e0 2024-04-20T11:38:19.945 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.944+0000 7f2427400640 1 --2- 172.21.15.152:0/3855534644 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f240806d430 0x7f240806f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:19.945 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.945+0000 7f2427400640 1 --2- 172.21.15.152:0/3855534644 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f240806d430 0x7f240806f8f0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f2418036000 tx=0x7f241803c040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:19.948 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:19.947+0000 7f241f400640 1 -- 172.21.15.152:0/3855534644 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f241401f020 con 0x7f242815d1e0 2024-04-20T11:38:20.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.142+0000 7f242cc00640 1 -- 172.21.15.152:0/3855534644 --> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2428072c90 con 0x7f240806d430 2024-04-20T11:38:20.146 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.145+0000 7f241f400640 1 -- 172.21.15.152:0/3855534644 <== mgr.4109 v2:172.21.15.175:6836/4021250441 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13498 (secure 0 0 0) 0x7f2428072c90 con 0x7f240806d430 2024-04-20T11:38:20.146 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:38:20.146 INFO:teuthology.orchestra.run.smithi152.stderr:dumped all 2024-04-20T11:38:20.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.147+0000 7f242cc00640 1 -- 172.21.15.152:0/3855534644 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f240806d430 msgr2=0x7f240806f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:20.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.147+0000 7f242cc00640 1 --2- 172.21.15.152:0/3855534644 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f240806d430 0x7f240806f8f0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f2418036000 tx=0x7f241803c040 comp rx=0 tx=0).stop 2024-04-20T11:38:20.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.147+0000 7f242cc00640 1 -- 172.21.15.152:0/3855534644 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f242815d1e0 msgr2=0x7f2428147cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:20.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.147+0000 7f242cc00640 1 --2- 172.21.15.152:0/3855534644 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f242815d1e0 0x7f2428147cd0 secure :-1 s=READY pgs=2431 cs=0 l=1 rev1=1 crypto rx=0x7f241400b440 tx=0x7f241400b900 comp rx=0 tx=0).stop 2024-04-20T11:38:20.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.147+0000 7f242cc00640 1 -- 172.21.15.152:0/3855534644 shutdown_connections 2024-04-20T11:38:20.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.147+0000 7f242cc00640 1 --2- 172.21.15.152:0/3855534644 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f240806d430 0x7f240806f8f0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:20.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.148+0000 7f242cc00640 1 --2- 172.21.15.152:0/3855534644 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f242815d5c0 0x7f24281589b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:20.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.148+0000 7f242cc00640 1 --2- 172.21.15.152:0/3855534644 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f242815d1e0 0x7f2428147cd0 unknown :-1 s=CLOSED pgs=2431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:20.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.148+0000 7f242cc00640 1 --2- 172.21.15.152:0/3855534644 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2428071cd0 0x7f2428156530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:20.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.148+0000 7f242cc00640 1 -- 172.21.15.152:0/3855534644 >> 172.21.15.152:0/3855534644 conn(0x7f2428067ae0 msgr2=0x7f24280b14e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:20.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.148+0000 7f242cc00640 1 -- 172.21.15.152:0/3855534644 shutdown_connections 2024-04-20T11:38:20.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.148+0000 7f242cc00640 1 -- 172.21.15.152:0/3855534644 wait complete. 2024-04-20T11:38:20.167 INFO:teuthology.orchestra.run.smithi152.stdout:{"pg_ready":true,"pg_map":{"version":280,"stamp":"2024-04-20T11:38:18.729457+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":148,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":4,"kb":754974720,"kb_used":271556,"kb_used_data":17944,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754703164,"statfs":{"total":773094113280,"available":772816039936,"internally_reserved":0,"allocated":18374656,"data_stored":8773961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12746,"internal_metadata":259575350},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-58692,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":-4,"num_read_kb":-4,"num_write":-89,"num_write_kb":-91,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001834"},"pg_stats":[{"pgid":"1.0","version":"72'128","reported_seq":478,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-20T11:38:18.192054+0000","last_change":"2024-04-20T11:26:48.661897+0000","last_active":"2024-04-20T11:38:18.192054+0000","last_peered":"2024-04-20T11:38:18.192054+0000","last_clean":"2024-04-20T11:38:18.192054+0000","last_became_active":"2024-04-20T11:26:48.656535+0000","last_became_peered":"2024-04-20T11:26:48.656535+0000","last_unstale":"2024-04-20T11:38:18.192054+0000","last_undegraded":"2024-04-20T11:38:18.192054+0000","last_fullsized":"2024-04-20T11:38:18.192054+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:47.543004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:47.543004+0000","last_clean_scrub_stamp":"2024-04-20T11:26:47.543004+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:58:35.527105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574992,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33868,"kb_used_data":2684,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337972,"statfs":{"total":96636764160,"available":96602083328,"internally_reserved":0,"allocated":2748416,"data_stored":1541608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574989,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33284,"kb_used_data":2104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338556,"statfs":{"total":96636764160,"available":96602681344,"internally_reserved":0,"allocated":2154496,"data_stored":951240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574991,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33284,"kb_used_data":2104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338556,"statfs":{"total":96636764160,"available":96602681344,"internally_reserved":0,"allocated":2154496,"data_stored":951240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574989,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33268,"kb_used_data":2088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338572,"statfs":{"total":96636764160,"available":96602697728,"internally_reserved":0,"allocated":2138112,"data_stored":945675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574990,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33288,"kb_used_data":2104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338552,"statfs":{"total":96636764160,"available":96602677248,"internally_reserved":0,"allocated":2154496,"data_stored":951240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574991,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33268,"kb_used_data":2088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338572,"statfs":{"total":96636764160,"available":96602697728,"internally_reserved":0,"allocated":2138112,"data_stored":945675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574989,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33288,"kb_used_data":2104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338552,"statfs":{"total":96636764160,"available":96602677248,"internally_reserved":0,"allocated":2154496,"data_stored":951240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574990,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38008,"kb_used_data":2668,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333832,"statfs":{"total":96636764160,"available":96597843968,"internally_reserved":0,"allocated":2732032,"data_stored":1536043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T11:38:20.167 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-20T11:38:20.167 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T11:38:20.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.354+0000 7f7c91200640 1 Processor -- start 2024-04-20T11:38:20.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.354+0000 7f7c91200640 1 -- start start 2024-04-20T11:38:20.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.355+0000 7f7c91200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f7c8c160b90 0x7f7c8c15d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:20.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.356+0000 7f7c91200640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c8c0ce8c0 con 0x7f7c8c071cd0 2024-04-20T11:38:20.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.356+0000 7f7c91200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c8c1600b0 con 0x7f7c8c15d1e0 2024-04-20T11:38:20.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.356+0000 7f7c91200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f7c8c160390 con 0x7f7c8c160b90 2024-04-20T11:38:20.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.356+0000 7f7c8aa00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f7c8c160b90 0x7f7c8c15d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:20.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.356+0000 7f7c8aa00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f7c8c160b90 0x7f7c8c15d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54692/0 (socket says 172.21.15.152:54692) 2024-04-20T11:38:20.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.356+0000 7f7c8aa00640 1 -- 172.21.15.152:0/1626541963 learned_addr learned my addr 172.21.15.152:0/1626541963 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:38:20.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.356+0000 7f7c8aa00640 1 -- 172.21.15.152:0/1626541963 >> v1:172.21.15.175:6789/0 conn(0x7f7c8c15d1e0 legacy=0x7f7c8c0cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:20.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.356+0000 7f7c8aa00640 1 -- 172.21.15.152:0/1626541963 >> v1:172.21.15.152:6789/0 conn(0x7f7c8c071cd0 legacy=0x7f7c8c072120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:20.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.356+0000 7f7c8aa00640 1 -- 172.21.15.152:0/1626541963 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c8c160670 con 0x7f7c8c160b90 2024-04-20T11:38:20.359 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.357+0000 7f7c8aa00640 1 --2- 172.21.15.152:0/1626541963 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7c8c160b90 0x7f7c8c15d5c0 secure :-1 s=READY pgs=2433 cs=0 l=1 rev1=1 crypto rx=0x7f7c780098a0 tx=0x7f7c7802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=938c2c04610d69b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:20.359 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.358+0000 7f7c8a000640 1 -- 172.21.15.152:0/1626541963 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c78034070 con 0x7f7c8c160b90 2024-04-20T11:38:20.359 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.358+0000 7f7c8a000640 1 -- 172.21.15.152:0/1626541963 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c78030840 con 0x7f7c8c160b90 2024-04-20T11:38:20.359 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.358+0000 7f7c8a000640 1 -- 172.21.15.152:0/1626541963 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c78030c10 con 0x7f7c8c160b90 2024-04-20T11:38:20.359 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.358+0000 7f7c91200640 1 -- 172.21.15.152:0/1626541963 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7c8c160b90 msgr2=0x7f7c8c15d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:20.359 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.358+0000 7f7c91200640 1 --2- 172.21.15.152:0/1626541963 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7c8c160b90 0x7f7c8c15d5c0 secure :-1 s=READY pgs=2433 cs=0 l=1 rev1=1 crypto rx=0x7f7c780098a0 tx=0x7f7c7802b980 comp rx=0 tx=0).stop 2024-04-20T11:38:20.359 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.358+0000 7f7c91200640 1 -- 172.21.15.152:0/1626541963 shutdown_connections 2024-04-20T11:38:20.359 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.358+0000 7f7c91200640 1 --2- 172.21.15.152:0/1626541963 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7c8c160b90 0x7f7c8c15d5c0 unknown :-1 s=CLOSED pgs=2433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:20.359 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.358+0000 7f7c91200640 1 -- 172.21.15.152:0/1626541963 >> 172.21.15.152:0/1626541963 conn(0x7f7c8c067ae0 msgr2=0x7f7c8c06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:20.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.358+0000 7f7c91200640 1 -- 172.21.15.152:0/1626541963 shutdown_connections 2024-04-20T11:38:20.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.358+0000 7f7c91200640 1 -- 172.21.15.152:0/1626541963 wait complete. 2024-04-20T11:38:20.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.360+0000 7f7c91200640 1 Processor -- start 2024-04-20T11:38:20.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.360+0000 7f7c91200640 1 -- start start 2024-04-20T11:38:20.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.361+0000 7f7c91200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c8c071cd0 0x7f7c8c14cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:20.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.361+0000 7f7c91200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7c8c15d1e0 0x7f7c8c1591b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:20.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.361+0000 7f7c91200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7c8c160b90 0x7f7c8c1596f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:20.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.361+0000 7f7c91200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f7c8c0ccef0 con 0x7f7c8c160b90 2024-04-20T11:38:20.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.361+0000 7f7c91200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7c8c0ccd70 con 0x7f7c8c071cd0 2024-04-20T11:38:20.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.361+0000 7f7c91200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7c8c0cd070 con 0x7f7c8c15d1e0 2024-04-20T11:38:20.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.361+0000 7f7c8be00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7c8c160b90 0x7f7c8c1596f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:20.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.361+0000 7f7c8be00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7c8c160b90 0x7f7c8c1596f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54704/0 (socket says 172.21.15.152:54704) 2024-04-20T11:38:20.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.361+0000 7f7c8be00640 1 -- 172.21.15.152:0/3135978086 learned_addr learned my addr 172.21.15.152:0/3135978086 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:38:20.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.362+0000 7f7c8aa00640 1 --2- 172.21.15.152:0/3135978086 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7c8c15d1e0 0x7f7c8c1591b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:20.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.362+0000 7f7c8b400640 1 --2- 172.21.15.152:0/3135978086 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c8c071cd0 0x7f7c8c14cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:20.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.362+0000 7f7c8be00640 1 -- 172.21.15.152:0/3135978086 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7c8c15d1e0 msgr2=0x7f7c8c1591b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:20.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.362+0000 7f7c8be00640 1 --2- 172.21.15.152:0/3135978086 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7c8c15d1e0 0x7f7c8c1591b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:20.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.362+0000 7f7c8be00640 1 -- 172.21.15.152:0/3135978086 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c8c071cd0 msgr2=0x7f7c8c14cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:20.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.362+0000 7f7c8be00640 1 --2- 172.21.15.152:0/3135978086 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c8c071cd0 0x7f7c8c14cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:20.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.362+0000 7f7c8be00640 1 -- 172.21.15.152:0/3135978086 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c8c159d20 con 0x7f7c8c160b90 2024-04-20T11:38:20.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.362+0000 7f7c8b400640 1 --2- 172.21.15.152:0/3135978086 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c8c071cd0 0x7f7c8c14cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:38:20.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.362+0000 7f7c8be00640 1 --2- 172.21.15.152:0/3135978086 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7c8c160b90 0x7f7c8c1596f0 secure :-1 s=READY pgs=2434 cs=0 l=1 rev1=1 crypto rx=0x7f7c8400c7a0 tx=0x7f7c8400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:20.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.362+0000 7f7c83400640 1 -- 172.21.15.152:0/3135978086 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c84003b70 con 0x7f7c8c160b90 2024-04-20T11:38:20.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.362+0000 7f7c83400640 1 -- 172.21.15.152:0/3135978086 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c84003d30 con 0x7f7c8c160b90 2024-04-20T11:38:20.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.362+0000 7f7c83400640 1 -- 172.21.15.152:0/3135978086 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c8400aa90 con 0x7f7c8c160b90 2024-04-20T11:38:20.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.362+0000 7f7c91200640 1 -- 172.21.15.152:0/3135978086 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c8c1f61a0 con 0x7f7c8c160b90 2024-04-20T11:38:20.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.363+0000 7f7c91200640 1 -- 172.21.15.152:0/3135978086 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c8c1f6760 con 0x7f7c8c160b90 2024-04-20T11:38:20.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.364+0000 7f7c91200640 1 -- 172.21.15.152:0/3135978086 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c8c06ef10 con 0x7f7c8c160b90 2024-04-20T11:38:20.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.365+0000 7f7c83400640 1 -- 172.21.15.152:0/3135978086 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 27) ==== 92060+0+0 (secure 0 0 0) 0x7f7c8400ac30 con 0x7f7c8c160b90 2024-04-20T11:38:20.366 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.365+0000 7f7c83400640 1 --2- 172.21.15.152:0/3135978086 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f7c6c06d430 0x7f7c6c06f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:20.366 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.366+0000 7f7c83400640 1 -- 172.21.15.152:0/3135978086 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 4943+0+0 (secure 0 0 0) 0x7f7c8408a700 con 0x7f7c8c160b90 2024-04-20T11:38:20.366 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.366+0000 7f7c8b400640 1 --2- 172.21.15.152:0/3135978086 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f7c6c06d430 0x7f7c6c06f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:20.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.367+0000 7f7c8b400640 1 --2- 172.21.15.152:0/3135978086 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f7c6c06d430 0x7f7c6c06f8f0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f7c7c007fd0 tx=0x7f7c7c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:20.370 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.370+0000 7f7c83400640 1 -- 172.21.15.152:0/3135978086 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f7c8c06ef10 con 0x7f7c8c160b90 2024-04-20T11:38:20.567 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.566+0000 7f7c91200640 1 -- 172.21.15.152:0/3135978086 --> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7c8c0cf790 con 0x7f7c6c06d430 2024-04-20T11:38:20.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.569+0000 7f7c83400640 1 -- 172.21.15.152:0/3135978086 <== mgr.4109 v2:172.21.15.175:6836/4021250441 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13498 (secure 0 0 0) 0x7f7c8c0cf790 con 0x7f7c6c06d430 2024-04-20T11:38:20.570 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:38:20.570 INFO:teuthology.orchestra.run.smithi152.stderr:dumped all 2024-04-20T11:38:20.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.571+0000 7f7c91200640 1 -- 172.21.15.152:0/3135978086 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f7c6c06d430 msgr2=0x7f7c6c06f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:20.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.571+0000 7f7c91200640 1 --2- 172.21.15.152:0/3135978086 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f7c6c06d430 0x7f7c6c06f8f0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f7c7c007fd0 tx=0x7f7c7c009040 comp rx=0 tx=0).stop 2024-04-20T11:38:20.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.571+0000 7f7c91200640 1 -- 172.21.15.152:0/3135978086 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7c8c160b90 msgr2=0x7f7c8c1596f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:20.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.571+0000 7f7c91200640 1 --2- 172.21.15.152:0/3135978086 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7c8c160b90 0x7f7c8c1596f0 secure :-1 s=READY pgs=2434 cs=0 l=1 rev1=1 crypto rx=0x7f7c8400c7a0 tx=0x7f7c8400cc60 comp rx=0 tx=0).stop 2024-04-20T11:38:20.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.572+0000 7f7c91200640 1 -- 172.21.15.152:0/3135978086 shutdown_connections 2024-04-20T11:38:20.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.572+0000 7f7c91200640 1 --2- 172.21.15.152:0/3135978086 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f7c6c06d430 0x7f7c6c06f8f0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:20.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.572+0000 7f7c91200640 1 --2- 172.21.15.152:0/3135978086 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7c8c160b90 0x7f7c8c1596f0 unknown :-1 s=CLOSED pgs=2434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:20.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.572+0000 7f7c91200640 1 --2- 172.21.15.152:0/3135978086 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7c8c15d1e0 0x7f7c8c1591b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:20.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.572+0000 7f7c91200640 1 --2- 172.21.15.152:0/3135978086 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c8c071cd0 0x7f7c8c14cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:20.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.572+0000 7f7c91200640 1 -- 172.21.15.152:0/3135978086 >> 172.21.15.152:0/3135978086 conn(0x7f7c8c067ae0 msgr2=0x7f7c8c06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:20.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.572+0000 7f7c91200640 1 -- 172.21.15.152:0/3135978086 shutdown_connections 2024-04-20T11:38:20.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.572+0000 7f7c91200640 1 -- 172.21.15.152:0/3135978086 wait complete. 2024-04-20T11:38:20.592 INFO:teuthology.orchestra.run.smithi152.stdout:{"pg_ready":true,"pg_map":{"version":280,"stamp":"2024-04-20T11:38:18.729457+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":148,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":4,"kb":754974720,"kb_used":271556,"kb_used_data":17944,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754703164,"statfs":{"total":773094113280,"available":772816039936,"internally_reserved":0,"allocated":18374656,"data_stored":8773961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12746,"internal_metadata":259575350},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-58692,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":-4,"num_read_kb":-4,"num_write":-89,"num_write_kb":-91,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001834"},"pg_stats":[{"pgid":"1.0","version":"72'128","reported_seq":478,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-20T11:38:18.192054+0000","last_change":"2024-04-20T11:26:48.661897+0000","last_active":"2024-04-20T11:38:18.192054+0000","last_peered":"2024-04-20T11:38:18.192054+0000","last_clean":"2024-04-20T11:38:18.192054+0000","last_became_active":"2024-04-20T11:26:48.656535+0000","last_became_peered":"2024-04-20T11:26:48.656535+0000","last_unstale":"2024-04-20T11:38:18.192054+0000","last_undegraded":"2024-04-20T11:38:18.192054+0000","last_fullsized":"2024-04-20T11:38:18.192054+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:47.543004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:47.543004+0000","last_clean_scrub_stamp":"2024-04-20T11:26:47.543004+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:58:35.527105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574992,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33868,"kb_used_data":2684,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337972,"statfs":{"total":96636764160,"available":96602083328,"internally_reserved":0,"allocated":2748416,"data_stored":1541608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574989,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33284,"kb_used_data":2104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338556,"statfs":{"total":96636764160,"available":96602681344,"internally_reserved":0,"allocated":2154496,"data_stored":951240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574991,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33284,"kb_used_data":2104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338556,"statfs":{"total":96636764160,"available":96602681344,"internally_reserved":0,"allocated":2154496,"data_stored":951240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574989,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33268,"kb_used_data":2088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338572,"statfs":{"total":96636764160,"available":96602697728,"internally_reserved":0,"allocated":2138112,"data_stored":945675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574990,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33288,"kb_used_data":2104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338552,"statfs":{"total":96636764160,"available":96602677248,"internally_reserved":0,"allocated":2154496,"data_stored":951240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574991,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33268,"kb_used_data":2088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338572,"statfs":{"total":96636764160,"available":96602697728,"internally_reserved":0,"allocated":2138112,"data_stored":945675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574989,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33288,"kb_used_data":2104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338552,"statfs":{"total":96636764160,"available":96602677248,"internally_reserved":0,"allocated":2154496,"data_stored":951240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574990,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38008,"kb_used_data":2668,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333832,"statfs":{"total":96636764160,"available":96597843968,"internally_reserved":0,"allocated":2732032,"data_stored":1536043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T11:38:20.593 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:38:20.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.798+0000 7f4b51600640 1 Processor -- start 2024-04-20T11:38:20.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.799+0000 7f4b51600640 1 -- start start 2024-04-20T11:38:20.801 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.800+0000 7f4b51600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4b4c15d1e0 0x7f4b4c160b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:20.801 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.800+0000 7f4b51600640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f4b4c071cd0 0x7f4b4c072160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:20.801 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.800+0000 7f4b51600640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b4c0ce8c0 con 0x7f4b4c15d5c0 2024-04-20T11:38:20.801 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.800+0000 7f4b51600640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f4b4c0726a0 con 0x7f4b4c071cd0 2024-04-20T11:38:20.801 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.800+0000 7f4b51600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4b4c1600b0 con 0x7f4b4c15d1e0 2024-04-20T11:38:20.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.801+0000 7f4b50c00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f4b4c071cd0 0x7f4b4c072160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:20.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.801+0000 7f4b50c00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f4b4c071cd0 0x7f4b4c072160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54714/0 (socket says 172.21.15.152:54714) 2024-04-20T11:38:20.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.801+0000 7f4b50c00640 1 -- 172.21.15.152:0/2329134612 learned_addr learned my addr 172.21.15.152:0/2329134612 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:38:20.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.801+0000 7f4b4be00640 1 --2- 172.21.15.152:0/2329134612 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4b4c15d1e0 0x7f4b4c160b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:20.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.801+0000 7f4b50c00640 1 -- 172.21.15.152:0/2329134612 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4b4c15d1e0 msgr2=0x7f4b4c160b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:20.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.801+0000 7f4b50c00640 1 --2- 172.21.15.152:0/2329134612 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4b4c15d1e0 0x7f4b4c160b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:20.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.801+0000 7f4b50c00640 1 -- 172.21.15.152:0/2329134612 >> v1:172.21.15.175:6789/0 conn(0x7f4b4c15d5c0 legacy=0x7f4b4c0cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:20.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.801+0000 7f4b50c00640 1 -- 172.21.15.152:0/2329134612 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b4c160390 con 0x7f4b4c071cd0 2024-04-20T11:38:20.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.803+0000 7f4b50c00640 1 --2- 172.21.15.152:0/2329134612 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b4c071cd0 0x7f4b4c072160 secure :-1 s=READY pgs=2435 cs=0 l=1 rev1=1 crypto rx=0x7f4b44009e50 tx=0x7f4b4402bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=964df482d6aa1c96 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:20.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.803+0000 7f4b4b400640 1 -- 172.21.15.152:0/2329134612 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b4402cb00 con 0x7f4b4c071cd0 2024-04-20T11:38:20.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.803+0000 7f4b4b400640 1 -- 172.21.15.152:0/2329134612 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b4402cca0 con 0x7f4b4c071cd0 2024-04-20T11:38:20.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.803+0000 7f4b4b400640 1 -- 172.21.15.152:0/2329134612 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b44034610 con 0x7f4b4c071cd0 2024-04-20T11:38:20.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.804+0000 7f4b51600640 1 -- 172.21.15.152:0/2329134612 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b4c071cd0 msgr2=0x7f4b4c072160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:20.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.804+0000 7f4b51600640 1 --2- 172.21.15.152:0/2329134612 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b4c071cd0 0x7f4b4c072160 secure :-1 s=READY pgs=2435 cs=0 l=1 rev1=1 crypto rx=0x7f4b44009e50 tx=0x7f4b4402bf60 comp rx=0 tx=0).stop 2024-04-20T11:38:20.805 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.804+0000 7f4b51600640 1 -- 172.21.15.152:0/2329134612 shutdown_connections 2024-04-20T11:38:20.805 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.804+0000 7f4b51600640 1 --2- 172.21.15.152:0/2329134612 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b4c071cd0 0x7f4b4c072160 unknown :-1 s=CLOSED pgs=2435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:20.805 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.804+0000 7f4b51600640 1 --2- 172.21.15.152:0/2329134612 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4b4c15d1e0 0x7f4b4c160b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:20.805 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.804+0000 7f4b51600640 1 -- 172.21.15.152:0/2329134612 >> 172.21.15.152:0/2329134612 conn(0x7f4b4c067ae0 msgr2=0x7f4b4c06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:20.805 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.804+0000 7f4b51600640 1 -- 172.21.15.152:0/2329134612 shutdown_connections 2024-04-20T11:38:20.805 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.804+0000 7f4b51600640 1 -- 172.21.15.152:0/2329134612 wait complete. 2024-04-20T11:38:20.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.805+0000 7f4b51600640 1 Processor -- start 2024-04-20T11:38:20.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.806+0000 7f4b51600640 1 -- start start 2024-04-20T11:38:20.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.806+0000 7f4b51600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4b4c071cd0 0x7f4b4c0aeec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:20.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.806+0000 7f4b51600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b4c15d1e0 0x7f4b4c14a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:20.809 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.806+0000 7f4b51600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4b4c15d5c0 0x7f4b4c1f49d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:20.809 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.807+0000 7f4b51600640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f4b4c0ccd70 con 0x7f4b4c15d1e0 2024-04-20T11:38:20.809 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.807+0000 7f4b51600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4b4c0ceba0 con 0x7f4b4c15d5c0 2024-04-20T11:38:20.809 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.807+0000 7f4b51600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4b4c0ccef0 con 0x7f4b4c071cd0 2024-04-20T11:38:20.809 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.807+0000 7f4b43400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b4c15d1e0 0x7f4b4c14a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:20.809 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.807+0000 7f4b43400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b4c15d1e0 0x7f4b4c14a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54728/0 (socket says 172.21.15.152:54728) 2024-04-20T11:38:20.809 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.807+0000 7f4b43400640 1 -- 172.21.15.152:0/2291223277 learned_addr learned my addr 172.21.15.152:0/2291223277 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:38:20.809 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.807+0000 7f4b4be00640 1 --2- 172.21.15.152:0/2291223277 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4b4c071cd0 0x7f4b4c0aeec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:20.809 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.807+0000 7f4b43400640 1 -- 172.21.15.152:0/2291223277 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4b4c071cd0 msgr2=0x7f4b4c0aeec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:20.809 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.807+0000 7f4b43400640 1 --2- 172.21.15.152:0/2291223277 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4b4c071cd0 0x7f4b4c0aeec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:20.809 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.807+0000 7f4b50c00640 1 --2- 172.21.15.152:0/2291223277 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4b4c15d5c0 0x7f4b4c1f49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:20.809 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.807+0000 7f4b43400640 1 -- 172.21.15.152:0/2291223277 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4b4c15d5c0 msgr2=0x7f4b4c1f49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:20.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.807+0000 7f4b43400640 1 --2- 172.21.15.152:0/2291223277 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4b4c15d5c0 0x7f4b4c1f49d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:20.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.807+0000 7f4b43400640 1 -- 172.21.15.152:0/2291223277 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b4c0ade20 con 0x7f4b4c15d1e0 2024-04-20T11:38:20.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.807+0000 7f4b4be00640 1 --2- 172.21.15.152:0/2291223277 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4b4c071cd0 0x7f4b4c0aeec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:38:20.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.808+0000 7f4b43400640 1 --2- 172.21.15.152:0/2291223277 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b4c15d1e0 0x7f4b4c14a5b0 secure :-1 s=READY pgs=2436 cs=0 l=1 rev1=1 crypto rx=0x7f4b3400c3c0 tx=0x7f4b3400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:20.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.808+0000 7f4b48c00640 1 -- 172.21.15.152:0/2291223277 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b340052d0 con 0x7f4b4c15d1e0 2024-04-20T11:38:20.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.808+0000 7f4b48c00640 1 -- 172.21.15.152:0/2291223277 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b34005490 con 0x7f4b4c15d1e0 2024-04-20T11:38:20.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.808+0000 7f4b48c00640 1 -- 172.21.15.152:0/2291223277 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b34007260 con 0x7f4b4c15d1e0 2024-04-20T11:38:20.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.808+0000 7f4b51600640 1 -- 172.21.15.152:0/2291223277 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b4c0ae100 con 0x7f4b4c15d1e0 2024-04-20T11:38:20.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.808+0000 7f4b51600640 1 -- 172.21.15.152:0/2291223277 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b4c0ae6c0 con 0x7f4b4c15d1e0 2024-04-20T11:38:20.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.809+0000 7f4b51600640 1 -- 172.21.15.152:0/2291223277 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b4c075e60 con 0x7f4b4c15d1e0 2024-04-20T11:38:20.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.810+0000 7f4b48c00640 1 -- 172.21.15.152:0/2291223277 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 27) ==== 92060+0+0 (secure 0 0 0) 0x7f4b34007400 con 0x7f4b4c15d1e0 2024-04-20T11:38:20.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.811+0000 7f4b48c00640 1 --2- 172.21.15.152:0/2291223277 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4b2806d430 0x7f4b2806f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:20.812 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.811+0000 7f4b48c00640 1 -- 172.21.15.152:0/2291223277 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 4943+0+0 (secure 0 0 0) 0x7f4b34089be0 con 0x7f4b4c15d1e0 2024-04-20T11:38:20.812 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.812+0000 7f4b4be00640 1 --2- 172.21.15.152:0/2291223277 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4b2806d430 0x7f4b2806f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:20.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.813+0000 7f4b4be00640 1 --2- 172.21.15.152:0/2291223277 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4b2806d430 0x7f4b2806f8f0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c008890 tx=0x7f4b3c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:20.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:20.815+0000 7f4b48c00640 1 -- 172.21.15.152:0/2291223277 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f4b34059870 con 0x7f4b4c15d1e0 2024-04-20T11:38:21.016 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:21.016+0000 7f4b51600640 1 -- 172.21.15.152:0/2291223277 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4b4c0b1e00 con 0x7f4b4c15d1e0 2024-04-20T11:38:21.018 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:21.017+0000 7f4b48c00640 1 -- 172.21.15.152:0/2291223277 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v170) ==== 74+0+13090 (secure 0 0 0) 0x7f4b34059170 con 0x7f4b4c15d1e0 2024-04-20T11:38:21.018 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:38:21.018 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":170,"fsid":"c8a72131-0d4f-42c7-892f-3f3af4d2bd3d","created":"2024-04-20T11:26:33.958222+0000","modified":"2024-04-20T11:38:17.308329+0000","last_up_change":"2024-04-20T11:26:46.520256+0000","last_in_change":"2024-04-20T11:26:37.321214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:26:47.246571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c34f4e30-d57a-4adb-aeed-19ea76f925d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6825","nonce":1121306353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6827","nonce":1121306353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6831","nonce":1121306353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1121306353},{"type":"v1","addr":"172.21.15.152:6829","nonce":1121306353}]},"public_addr":"172.21.15.152:6825/1121306353","cluster_addr":"172.21.15.152:6827/1121306353","heartbeat_back_addr":"172.21.15.152:6831/1121306353","heartbeat_front_addr":"172.21.15.152:6829/1121306353","state":["exists","up"]},{"osd":1,"uuid":"b518a138-bfc3-43bd-8f4d-e0ab01bf7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6809","nonce":1552206819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6811","nonce":1552206819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6815","nonce":1552206819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1552206819},{"type":"v1","addr":"172.21.15.152:6813","nonce":1552206819}]},"public_addr":"172.21.15.152:6809/1552206819","cluster_addr":"172.21.15.152:6811/1552206819","heartbeat_back_addr":"172.21.15.152:6815/1552206819","heartbeat_front_addr":"172.21.15.152:6813/1552206819","state":["exists","up"]},{"osd":2,"uuid":"f3d523ee-a814-4537-92cc-87a84bb60958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6801","nonce":2519167656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6803","nonce":2519167656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6807","nonce":2519167656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2519167656},{"type":"v1","addr":"172.21.15.152:6805","nonce":2519167656}]},"public_addr":"172.21.15.152:6801/2519167656","cluster_addr":"172.21.15.152:6803/2519167656","heartbeat_back_addr":"172.21.15.152:6807/2519167656","heartbeat_front_addr":"172.21.15.152:6805/2519167656","state":["exists","up"]},{"osd":3,"uuid":"27143d57-518d-4081-9ac5-c49ee82adf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6817","nonce":3995926281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6819","nonce":3995926281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6823","nonce":3995926281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3995926281},{"type":"v1","addr":"172.21.15.152:6821","nonce":3995926281}]},"public_addr":"172.21.15.152:6817/3995926281","cluster_addr":"172.21.15.152:6819/3995926281","heartbeat_back_addr":"172.21.15.152:6823/3995926281","heartbeat_front_addr":"172.21.15.152:6821/3995926281","state":["exists","up"]},{"osd":4,"uuid":"d431d09f-6287-457c-bb3f-7f521756de61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6802","nonce":1235215467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6805","nonce":1235215467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6811","nonce":1235215467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1235215467},{"type":"v1","addr":"172.21.15.175:6808","nonce":1235215467}]},"public_addr":"172.21.15.175:6802/1235215467","cluster_addr":"172.21.15.175:6805/1235215467","heartbeat_back_addr":"172.21.15.175:6811/1235215467","heartbeat_front_addr":"172.21.15.175:6808/1235215467","state":["exists","up"]},{"osd":5,"uuid":"38e483ba-3b23-4492-8c5c-b3af28240a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6804","nonce":3465937300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6810","nonce":3465937300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6815","nonce":3465937300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3465937300},{"type":"v1","addr":"172.21.15.175:6813","nonce":3465937300}]},"public_addr":"172.21.15.175:6804/3465937300","cluster_addr":"172.21.15.175:6810/3465937300","heartbeat_back_addr":"172.21.15.175:6815/3465937300","heartbeat_front_addr":"172.21.15.175:6813/3465937300","state":["exists","up"]},{"osd":6,"uuid":"5271d8ec-ba0e-4387-8f4b-113ef144f40e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6822","nonce":2659150371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6826","nonce":2659150371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6831","nonce":2659150371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2659150371},{"type":"v1","addr":"172.21.15.175:6829","nonce":2659150371}]},"public_addr":"172.21.15.175:6822/2659150371","cluster_addr":"172.21.15.175:6826/2659150371","heartbeat_back_addr":"172.21.15.175:6831/2659150371","heartbeat_front_addr":"172.21.15.175:6829/2659150371","state":["exists","up"]},{"osd":7,"uuid":"f2e6b6f3-b14b-43fa-a2f3-c253a43accfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6817","nonce":3116669518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6819","nonce":3116669518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6827","nonce":3116669518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":3116669518},{"type":"v1","addr":"172.21.15.175:6823","nonce":3116669518}]},"public_addr":"172.21.15.175:6817/3116669518","cluster_addr":"172.21.15.175:6819/3116669518","heartbeat_back_addr":"172.21.15.175:6827/3116669518","heartbeat_front_addr":"172.21.15.175:6823/3116669518","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.786153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.722516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.852730+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.747337+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.470178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.687637+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.756282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:26:42.666442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.152:6835/951802292":"2024-04-21T11:38:14.266154+0000","172.21.15.152:6832/951802292":"2024-04-21T11:38:14.266154+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:38:21.021 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:21.020+0000 7f4b51600640 1 -- 172.21.15.152:0/2291223277 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4b2806d430 msgr2=0x7f4b2806f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:21.021 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:21.020+0000 7f4b51600640 1 --2- 172.21.15.152:0/2291223277 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4b2806d430 0x7f4b2806f8f0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c008890 tx=0x7f4b3c008040 comp rx=0 tx=0).stop 2024-04-20T11:38:21.021 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:21.020+0000 7f4b51600640 1 -- 172.21.15.152:0/2291223277 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b4c15d1e0 msgr2=0x7f4b4c14a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:21.021 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:21.020+0000 7f4b51600640 1 --2- 172.21.15.152:0/2291223277 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b4c15d1e0 0x7f4b4c14a5b0 secure :-1 s=READY pgs=2436 cs=0 l=1 rev1=1 crypto rx=0x7f4b3400c3c0 tx=0x7f4b3400c880 comp rx=0 tx=0).stop 2024-04-20T11:38:21.021 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:21.020+0000 7f4b51600640 1 -- 172.21.15.152:0/2291223277 shutdown_connections 2024-04-20T11:38:21.021 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:21.020+0000 7f4b51600640 1 --2- 172.21.15.152:0/2291223277 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f4b2806d430 0x7f4b2806f8f0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:21.021 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:21.020+0000 7f4b51600640 1 --2- 172.21.15.152:0/2291223277 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4b4c15d5c0 0x7f4b4c1f49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:21.021 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:21.020+0000 7f4b51600640 1 --2- 172.21.15.152:0/2291223277 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b4c15d1e0 0x7f4b4c14a5b0 unknown :-1 s=CLOSED pgs=2436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:21.021 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:21.020+0000 7f4b51600640 1 --2- 172.21.15.152:0/2291223277 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4b4c071cd0 0x7f4b4c0aeec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:21.022 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:21.020+0000 7f4b51600640 1 -- 172.21.15.152:0/2291223277 >> 172.21.15.152:0/2291223277 conn(0x7f4b4c067ae0 msgr2=0x7f4b4c155ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:21.022 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:21.020+0000 7f4b51600640 1 -- 172.21.15.152:0/2291223277 shutdown_connections 2024-04-20T11:38:21.022 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:21.020+0000 7f4b51600640 1 -- 172.21.15.152:0/2291223277 wait complete. 2024-04-20T11:38:22.041 INFO:tasks.ceph:Scrubbing osd.0 2024-04-20T11:38:22.041 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-04-20T11:38:22.256 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.256+0000 7f08b3200640 1 Processor -- start 2024-04-20T11:38:22.256 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.256+0000 7f08b3200640 1 -- start start 2024-04-20T11:38:22.258 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.258+0000 7f08b3200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f08ac15e870 0x7f08ac0d3370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:22.258 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.258+0000 7f08b3200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f08ac0d0fa0 0x7f08ac0d1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:22.258 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.258+0000 7f08b3200640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08ac15ec50 con 0x7f08ac0cff90 2024-04-20T11:38:22.258 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.258+0000 7f08b3200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f08ac0d2ee0 con 0x7f08ac15e870 2024-04-20T11:38:22.258 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.258+0000 7f08b3200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f08ac0d38b0 con 0x7f08ac0d0fa0 2024-04-20T11:38:22.258 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.258+0000 7f08b1e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f08ac15e870 0x7f08ac0d3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:22.258 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.258+0000 7f08b1e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f08ac15e870 0x7f08ac0d3370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54744/0 (socket says 172.21.15.152:54744) 2024-04-20T11:38:22.258 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.258+0000 7f08b1e00640 1 -- 172.21.15.152:0/4105197917 learned_addr learned my addr 172.21.15.152:0/4105197917 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:38:22.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.258+0000 7f08b1400640 1 --2- 172.21.15.152:0/4105197917 >> v2:172.21.15.175:3300/0 conn(0x7f08ac0d0fa0 0x7f08ac0d1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:22.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.258+0000 7f08b1e00640 1 -- 172.21.15.152:0/4105197917 >> v2:172.21.15.175:3300/0 conn(0x7f08ac0d0fa0 msgr2=0x7f08ac0d1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:22.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.258+0000 7f08b1e00640 1 --2- 172.21.15.152:0/4105197917 >> v2:172.21.15.175:3300/0 conn(0x7f08ac0d0fa0 0x7f08ac0d1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:22.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.258+0000 7f08b1e00640 1 -- 172.21.15.152:0/4105197917 >> v1:172.21.15.152:6789/0 conn(0x7f08ac0cff90 legacy=0x7f08ac0d0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:22.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.258+0000 7f08b1e00640 1 -- 172.21.15.152:0/4105197917 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08ac0d22d0 con 0x7f08ac15e870 2024-04-20T11:38:22.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.258+0000 7f08b0a00640 1 -- 172.21.15.152:0/4105197917 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3291959772 0 0) 0x7f08ac15ec50 con 0x7f08ac0cff90 2024-04-20T11:38:22.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.259+0000 7f08b1e00640 1 --2- 172.21.15.152:0/4105197917 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f08ac15e870 0x7f08ac0d3370 secure :-1 s=READY pgs=2438 cs=0 l=1 rev1=1 crypto rx=0x7f08ac0b7190 tx=0x7f08a402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32ab31b03fea65cb server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:22.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.259+0000 7f08b0a00640 1 -- 172.21.15.152:0/4105197917 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a402ca80 con 0x7f08ac15e870 2024-04-20T11:38:22.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.259+0000 7f08b0a00640 1 -- 172.21.15.152:0/4105197917 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08a402cc20 con 0x7f08ac15e870 2024-04-20T11:38:22.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.259+0000 7f08b0a00640 1 -- 172.21.15.152:0/4105197917 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a40343f0 con 0x7f08ac15e870 2024-04-20T11:38:22.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.260+0000 7f08b3200640 1 -- 172.21.15.152:0/4105197917 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f08ac15e870 msgr2=0x7f08ac0d3370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:22.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.260+0000 7f08b3200640 1 --2- 172.21.15.152:0/4105197917 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f08ac15e870 0x7f08ac0d3370 secure :-1 s=READY pgs=2438 cs=0 l=1 rev1=1 crypto rx=0x7f08ac0b7190 tx=0x7f08a402bee0 comp rx=0 tx=0).stop 2024-04-20T11:38:22.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.260+0000 7f08b3200640 1 -- 172.21.15.152:0/4105197917 shutdown_connections 2024-04-20T11:38:22.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.260+0000 7f08b3200640 1 --2- 172.21.15.152:0/4105197917 >> v2:172.21.15.175:3300/0 conn(0x7f08ac0d0fa0 0x7f08ac0d1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:22.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.260+0000 7f08b3200640 1 --2- 172.21.15.152:0/4105197917 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f08ac15e870 0x7f08ac0d3370 unknown :-1 s=CLOSED pgs=2438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:22.262 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.260+0000 7f08b3200640 1 -- 172.21.15.152:0/4105197917 >> 172.21.15.152:0/4105197917 conn(0x7f08ac08bfd0 msgr2=0x7f08ac0d45d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:22.262 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.260+0000 7f08b3200640 1 -- 172.21.15.152:0/4105197917 shutdown_connections 2024-04-20T11:38:22.262 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.260+0000 7f08b3200640 1 -- 172.21.15.152:0/4105197917 wait complete. 2024-04-20T11:38:22.262 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.262+0000 7f08b3200640 1 Processor -- start 2024-04-20T11:38:22.262 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.262+0000 7f08b3200640 1 -- start start 2024-04-20T11:38:22.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.263+0000 7f08b3200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f08ac0cff90 0x7f08ac155400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:22.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.263+0000 7f08b3200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f08ac0d0fa0 0x7f08ac160080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:22.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.263+0000 7f08b3200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f08ac15e870 0x7f08ac1564c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:22.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.263+0000 7f08b3200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f08ac0d2b70 con 0x7f08ac0cff90 2024-04-20T11:38:22.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.263+0000 7f08b3200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f08ac0d2890 con 0x7f08ac15e870 2024-04-20T11:38:22.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.263+0000 7f08b3200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f08ac06ca80 con 0x7f08ac0d0fa0 2024-04-20T11:38:22.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.263+0000 7f08b1e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f08ac0cff90 0x7f08ac155400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:22.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.263+0000 7f08b1e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f08ac0cff90 0x7f08ac155400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54754/0 (socket says 172.21.15.152:54754) 2024-04-20T11:38:22.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.263+0000 7f08b1e00640 1 -- 172.21.15.152:0/3135169408 learned_addr learned my addr 172.21.15.152:0/3135169408 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:38:22.265 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.264+0000 7f08b1e00640 1 -- 172.21.15.152:0/3135169408 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f08ac0d0fa0 msgr2=0x7f08ac160080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:22.265 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.264+0000 7f08b1e00640 1 --2- 172.21.15.152:0/3135169408 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f08ac0d0fa0 0x7f08ac160080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:22.265 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.264+0000 7f08b1e00640 1 -- 172.21.15.152:0/3135169408 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f08ac15e870 msgr2=0x7f08ac1564c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:22.265 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.264+0000 7f08b1e00640 1 --2- 172.21.15.152:0/3135169408 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f08ac15e870 0x7f08ac1564c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:22.265 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.264+0000 7f08b1e00640 1 -- 172.21.15.152:0/3135169408 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08ac155940 con 0x7f08ac0cff90 2024-04-20T11:38:22.266 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.264+0000 7f08b1e00640 1 --2- 172.21.15.152:0/3135169408 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f08ac0cff90 0x7f08ac155400 secure :-1 s=READY pgs=2439 cs=0 l=1 rev1=1 crypto rx=0x7f08a4000f80 tx=0x7f08a4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:22.266 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.264+0000 7f08a2000640 1 -- 172.21.15.152:0/3135169408 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a4043020 con 0x7f08ac0cff90 2024-04-20T11:38:22.266 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.264+0000 7f08a2000640 1 -- 172.21.15.152:0/3135169408 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08a403e450 con 0x7f08ac0cff90 2024-04-20T11:38:22.266 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.264+0000 7f08a2000640 1 -- 172.21.15.152:0/3135169408 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a4009510 con 0x7f08ac0cff90 2024-04-20T11:38:22.266 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.264+0000 7f08b3200640 1 -- 172.21.15.152:0/3135169408 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08ac1f4be0 con 0x7f08ac0cff90 2024-04-20T11:38:22.266 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.264+0000 7f08b3200640 1 -- 172.21.15.152:0/3135169408 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08ac1619b0 con 0x7f08ac0cff90 2024-04-20T11:38:22.266 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.265+0000 7f08b3200640 1 -- 172.21.15.152:0/3135169408 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f08ac0ced50 con 0x7f08ac0cff90 2024-04-20T11:38:22.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.267+0000 7f08a2000640 1 -- 172.21.15.152:0/3135169408 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 27) ==== 92060+0+0 (secure 0 0 0) 0x7f08a4046020 con 0x7f08ac0cff90 2024-04-20T11:38:22.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.268+0000 7f08a2000640 1 --2- 172.21.15.152:0/3135169408 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f089006d430 0x7f089006f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:22.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.268+0000 7f08a2000640 1 -- 172.21.15.152:0/3135169408 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 4943+0+0 (secure 0 0 0) 0x7f08a40ace70 con 0x7f08ac0cff90 2024-04-20T11:38:22.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.268+0000 7f08a2000640 1 --2- 172.21.15.152:0/3135169408 >> [v2:172.21.15.152:6824/1121306353,v1:172.21.15.152:6825/1121306353] conn(0x7f08900732f0 0x7f0890075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:22.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.268+0000 7f08a2000640 1 -- 172.21.15.152:0/3135169408 --> [v2:172.21.15.152:6824/1121306353,v1:172.21.15.152:6825/1121306353] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f08a404ee50 con 0x7f08900732f0 2024-04-20T11:38:22.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.268+0000 7f08a2000640 1 -- 172.21.15.152:0/3135169408 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_get_version_reply(handle=1 version=170) ==== 24+0+0 (secure 0 0 0) 0x7f08a40b6050 con 0x7f08ac0cff90 2024-04-20T11:38:22.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.268+0000 7f08b2800640 1 --2- 172.21.15.152:0/3135169408 >> [v2:172.21.15.152:6824/1121306353,v1:172.21.15.152:6825/1121306353] conn(0x7f08900732f0 0x7f0890075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:22.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.268+0000 7f08b1400640 1 --2- 172.21.15.152:0/3135169408 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f089006d430 0x7f089006f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:22.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.269+0000 7f08b2800640 1 --2- 172.21.15.152:0/3135169408 >> [v2:172.21.15.152:6824/1121306353,v1:172.21.15.152:6825/1121306353] conn(0x7f08900732f0 0x7f0890075710 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:22.270 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.269+0000 7f08b1400640 1 --2- 172.21.15.152:0/3135169408 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f089006d430 0x7f089006f8f0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f089c006e20 tx=0x7f089c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:22.270 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.270+0000 7f08a2000640 1 -- 172.21.15.152:0/3135169408 <== osd.0 v2:172.21.15.152:6824/1121306353 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f08a404ee50 con 0x7f08900732f0 2024-04-20T11:38:22.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.284+0000 7f08b3200640 1 -- 172.21.15.152:0/3135169408 --> [v2:172.21.15.152:6824/1121306353,v1:172.21.15.152:6825/1121306353] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f08ac075e60 con 0x7f08900732f0 2024-04-20T11:38:22.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.284+0000 7f08a2000640 1 -- 172.21.15.152:0/3135169408 <== osd.0 v2:172.21.15.152:6824/1121306353 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f08ac075e60 con 0x7f08900732f0 2024-04-20T11:38:22.286 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-04-20T11:38:22.286 INFO:teuthology.orchestra.run.smithi152.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-20T11:38:22.286 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-04-20T11:38:22.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.285+0000 7f08b3200640 1 -- 172.21.15.152:0/3135169408 >> [v2:172.21.15.152:6824/1121306353,v1:172.21.15.152:6825/1121306353] conn(0x7f08900732f0 msgr2=0x7f0890075710 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:22.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.285+0000 7f08b3200640 1 --2- 172.21.15.152:0/3135169408 >> [v2:172.21.15.152:6824/1121306353,v1:172.21.15.152:6825/1121306353] conn(0x7f08900732f0 0x7f0890075710 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:22.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.285+0000 7f08b3200640 1 -- 172.21.15.152:0/3135169408 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f089006d430 msgr2=0x7f089006f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:22.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.285+0000 7f08b3200640 1 --2- 172.21.15.152:0/3135169408 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f089006d430 0x7f089006f8f0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f089c006e20 tx=0x7f089c008040 comp rx=0 tx=0).stop 2024-04-20T11:38:22.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.285+0000 7f08b3200640 1 -- 172.21.15.152:0/3135169408 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f08ac0cff90 msgr2=0x7f08ac155400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:22.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.285+0000 7f08b3200640 1 --2- 172.21.15.152:0/3135169408 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f08ac0cff90 0x7f08ac155400 secure :-1 s=READY pgs=2439 cs=0 l=1 rev1=1 crypto rx=0x7f08a4000f80 tx=0x7f08a4004a40 comp rx=0 tx=0).stop 2024-04-20T11:38:22.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.285+0000 7f08b2800640 1 -- 172.21.15.152:0/3135169408 reap_dead start 2024-04-20T11:38:22.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.285+0000 7f08b3200640 1 -- 172.21.15.152:0/3135169408 shutdown_connections 2024-04-20T11:38:22.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.285+0000 7f08b3200640 1 -- 172.21.15.152:0/3135169408 >> 172.21.15.152:0/3135169408 conn(0x7f08ac08bfd0 msgr2=0x7f08ac0b4190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:22.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.285+0000 7f08b3200640 1 -- 172.21.15.152:0/3135169408 shutdown_connections 2024-04-20T11:38:22.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.286+0000 7f08b3200640 1 -- 172.21.15.152:0/3135169408 wait complete. 2024-04-20T11:38:22.305 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-04-20T11:38:22.502 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.502+0000 7f8c29800640 1 Processor -- start 2024-04-20T11:38:22.502 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.502+0000 7f8c29800640 1 -- start start 2024-04-20T11:38:22.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.504+0000 7f8c29800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8c24159320 0x7f8c2415bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:22.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.504+0000 7f8c29800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f8c2406e0f0 0x7f8c2415d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:22.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.504+0000 7f8c29800640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c240cfef0 con 0x7f8c240d0e60 2024-04-20T11:38:22.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.504+0000 7f8c29800640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f8c2415c130 con 0x7f8c2406e0f0 2024-04-20T11:38:22.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.504+0000 7f8c29800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8c2415c3d0 con 0x7f8c24159320 2024-04-20T11:38:22.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.504+0000 7f8c28e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f8c2406e0f0 0x7f8c2415d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:22.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.504+0000 7f8c28e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f8c2406e0f0 0x7f8c2415d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54758/0 (socket says 172.21.15.152:54758) 2024-04-20T11:38:22.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.504+0000 7f8c28e00640 1 -- 172.21.15.152:0/520327701 learned_addr learned my addr 172.21.15.152:0/520327701 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:38:22.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.504+0000 7f8c23e00640 1 --2- 172.21.15.152:0/520327701 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8c24159320 0x7f8c2415bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:22.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.504+0000 7f8c28e00640 1 -- 172.21.15.152:0/520327701 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8c24159320 msgr2=0x7f8c2415bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:22.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.504+0000 7f8c28e00640 1 --2- 172.21.15.152:0/520327701 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8c24159320 0x7f8c2415bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:22.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.504+0000 7f8c28e00640 1 -- 172.21.15.152:0/520327701 >> v1:172.21.15.175:6789/0 conn(0x7f8c240d0e60 legacy=0x7f8c2406d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:22.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.504+0000 7f8c28e00640 1 -- 172.21.15.152:0/520327701 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c2415c6b0 con 0x7f8c2406e0f0 2024-04-20T11:38:22.508 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.505+0000 7f8c28e00640 1 --2- 172.21.15.152:0/520327701 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c2406e0f0 0x7f8c2415d450 secure :-1 s=READY pgs=2440 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c009e50 tx=0x7f8c1c02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39d1e79ea0340f92 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:22.513 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.505+0000 7f8c23400640 1 -- 172.21.15.152:0/520327701 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c1c02c9e0 con 0x7f8c2406e0f0 2024-04-20T11:38:22.513 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.505+0000 7f8c23400640 1 -- 172.21.15.152:0/520327701 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c1c02cb80 con 0x7f8c2406e0f0 2024-04-20T11:38:22.514 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.505+0000 7f8c23400640 1 -- 172.21.15.152:0/520327701 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c1c0343f0 con 0x7f8c2406e0f0 2024-04-20T11:38:22.514 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.506+0000 7f8c29800640 1 -- 172.21.15.152:0/520327701 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c2406e0f0 msgr2=0x7f8c2415d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:22.514 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.506+0000 7f8c29800640 1 --2- 172.21.15.152:0/520327701 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c2406e0f0 0x7f8c2415d450 secure :-1 s=READY pgs=2440 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c009e50 tx=0x7f8c1c02be40 comp rx=0 tx=0).stop 2024-04-20T11:38:22.514 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.506+0000 7f8c29800640 1 -- 172.21.15.152:0/520327701 shutdown_connections 2024-04-20T11:38:22.514 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.506+0000 7f8c29800640 1 --2- 172.21.15.152:0/520327701 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c2406e0f0 0x7f8c2415d450 unknown :-1 s=CLOSED pgs=2440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:22.514 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.506+0000 7f8c29800640 1 --2- 172.21.15.152:0/520327701 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8c24159320 0x7f8c2415bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:22.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.506+0000 7f8c29800640 1 -- 172.21.15.152:0/520327701 >> 172.21.15.152:0/520327701 conn(0x7f8c240d0260 msgr2=0x7f8c240d0670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:22.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.506+0000 7f8c29800640 1 -- 172.21.15.152:0/520327701 shutdown_connections 2024-04-20T11:38:22.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.506+0000 7f8c29800640 1 -- 172.21.15.152:0/520327701 wait complete. 2024-04-20T11:38:22.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.508+0000 7f8c29800640 1 Processor -- start 2024-04-20T11:38:22.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.508+0000 7f8c29800640 1 -- start start 2024-04-20T11:38:22.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.509+0000 7f8c29800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8c2406e0f0 0x7f8c2414a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:22.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.509+0000 7f8c29800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8c240d0e60 0x7f8c240ae1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:22.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.509+0000 7f8c29800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c24159320 0x7f8c240ae710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:22.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.509+0000 7f8c29800640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f8c2415d990 con 0x7f8c24159320 2024-04-20T11:38:22.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.509+0000 7f8c29800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8c2415cf30 con 0x7f8c2406e0f0 2024-04-20T11:38:22.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.509+0000 7f8c29800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8c2415db10 con 0x7f8c240d0e60 2024-04-20T11:38:22.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.509+0000 7f8c28e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c24159320 0x7f8c240ae710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:22.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.509+0000 7f8c28e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c24159320 0x7f8c240ae710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54766/0 (socket says 172.21.15.152:54766) 2024-04-20T11:38:22.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.509+0000 7f8c28e00640 1 -- 172.21.15.152:0/4019807343 learned_addr learned my addr 172.21.15.152:0/4019807343 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:38:22.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.510+0000 7f8c28e00640 1 -- 172.21.15.152:0/4019807343 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8c240d0e60 msgr2=0x7f8c240ae1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:22.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.510+0000 7f8c28e00640 1 --2- 172.21.15.152:0/4019807343 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8c240d0e60 0x7f8c240ae1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:22.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.510+0000 7f8c28e00640 1 -- 172.21.15.152:0/4019807343 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8c2406e0f0 msgr2=0x7f8c2414a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:22.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.510+0000 7f8c23e00640 1 --2- 172.21.15.152:0/4019807343 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8c2406e0f0 0x7f8c2414a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:22.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.510+0000 7f8c28e00640 1 --2- 172.21.15.152:0/4019807343 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8c2406e0f0 0x7f8c2414a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:22.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.510+0000 7f8c28e00640 1 -- 172.21.15.152:0/4019807343 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c2414b0b0 con 0x7f8c24159320 2024-04-20T11:38:22.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.510+0000 7f8c28e00640 1 --2- 172.21.15.152:0/4019807343 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c24159320 0x7f8c240ae710 secure :-1 s=READY pgs=2441 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c036040 tx=0x7f8c1c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:22.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.510+0000 7f8c20c00640 1 -- 172.21.15.152:0/4019807343 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c1c043020 con 0x7f8c24159320 2024-04-20T11:38:22.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.510+0000 7f8c20c00640 1 -- 172.21.15.152:0/4019807343 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c1c03e450 con 0x7f8c24159320 2024-04-20T11:38:22.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.510+0000 7f8c20c00640 1 -- 172.21.15.152:0/4019807343 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c1c009510 con 0x7f8c24159320 2024-04-20T11:38:22.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.510+0000 7f8c29800640 1 -- 172.21.15.152:0/4019807343 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c2414b390 con 0x7f8c24159320 2024-04-20T11:38:22.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.511+0000 7f8c29800640 1 -- 172.21.15.152:0/4019807343 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c2414b950 con 0x7f8c24159320 2024-04-20T11:38:22.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.512+0000 7f8c29800640 1 -- 172.21.15.152:0/4019807343 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c24075e60 con 0x7f8c24159320 2024-04-20T11:38:22.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.513+0000 7f8c20c00640 1 -- 172.21.15.152:0/4019807343 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 27) ==== 92060+0+0 (secure 0 0 0) 0x7f8c1c046020 con 0x7f8c24159320 2024-04-20T11:38:22.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.514+0000 7f8c20c00640 1 --2- 172.21.15.152:0/4019807343 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f8c0006d430 0x7f8c0006f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:22.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.514+0000 7f8c20c00640 1 -- 172.21.15.152:0/4019807343 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 4943+0+0 (secure 0 0 0) 0x7f8c1c0acda0 con 0x7f8c24159320 2024-04-20T11:38:22.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.514+0000 7f8c23e00640 1 --2- 172.21.15.152:0/4019807343 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f8c0006d430 0x7f8c0006f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:22.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.515+0000 7f8c23e00640 1 --2- 172.21.15.152:0/4019807343 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f8c0006d430 0x7f8c0006f8f0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f8c14006fd0 tx=0x7f8c14008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:22.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.518+0000 7f8c20c00640 1 -- 172.21.15.152:0/4019807343 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f8c1c04d030 con 0x7f8c24159320 2024-04-20T11:38:22.716 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.716+0000 7f8c29800640 1 -- 172.21.15.152:0/4019807343 --> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7f8c2406db10 con 0x7f8c0006d430 2024-04-20T11:38:22.717 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.717+0000 7f8c20c00640 1 -- 172.21.15.152:0/4019807343 <== mgr.4109 v2:172.21.15.175:6836/4021250441 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f8c2406db10 con 0x7f8c0006d430 2024-04-20T11:38:22.717 INFO:teuthology.orchestra.run.smithi152.stderr:instructed osd(s) 0 to deep-scrub 2024-04-20T11:38:22.721 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.720+0000 7f8c29800640 1 -- 172.21.15.152:0/4019807343 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f8c0006d430 msgr2=0x7f8c0006f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:22.721 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.720+0000 7f8c29800640 1 --2- 172.21.15.152:0/4019807343 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f8c0006d430 0x7f8c0006f8f0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f8c14006fd0 tx=0x7f8c14008040 comp rx=0 tx=0).stop 2024-04-20T11:38:22.721 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.720+0000 7f8c29800640 1 -- 172.21.15.152:0/4019807343 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c24159320 msgr2=0x7f8c240ae710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:22.721 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.720+0000 7f8c29800640 1 --2- 172.21.15.152:0/4019807343 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c24159320 0x7f8c240ae710 secure :-1 s=READY pgs=2441 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c036040 tx=0x7f8c1c004a40 comp rx=0 tx=0).stop 2024-04-20T11:38:22.721 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.720+0000 7f8c29800640 1 -- 172.21.15.152:0/4019807343 shutdown_connections 2024-04-20T11:38:22.721 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.720+0000 7f8c29800640 1 --2- 172.21.15.152:0/4019807343 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f8c0006d430 0x7f8c0006f8f0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:22.721 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.720+0000 7f8c29800640 1 --2- 172.21.15.152:0/4019807343 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c24159320 0x7f8c240ae710 unknown :-1 s=CLOSED pgs=2441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:22.721 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.720+0000 7f8c29800640 1 --2- 172.21.15.152:0/4019807343 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8c240d0e60 0x7f8c240ae1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:22.721 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.720+0000 7f8c29800640 1 --2- 172.21.15.152:0/4019807343 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8c2406e0f0 0x7f8c2414a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:22.721 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.720+0000 7f8c29800640 1 -- 172.21.15.152:0/4019807343 >> 172.21.15.152:0/4019807343 conn(0x7f8c240d0260 msgr2=0x7f8c24157e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:22.721 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.720+0000 7f8c29800640 1 -- 172.21.15.152:0/4019807343 shutdown_connections 2024-04-20T11:38:22.721 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.720+0000 7f8c29800640 1 -- 172.21.15.152:0/4019807343 wait complete. 2024-04-20T11:38:22.739 INFO:tasks.ceph:Scrubbing osd.1 2024-04-20T11:38:22.739 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-04-20T11:38:22.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.943+0000 7f549fe00640 1 Processor -- start 2024-04-20T11:38:22.944 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.943+0000 7f549fe00640 1 -- start start 2024-04-20T11:38:22.945 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.944+0000 7f549fe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f54a00ad760 0x7f54a015ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:22.945 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.944+0000 7f549fe00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f54a00d05f0 0x7f54a00d0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:22.945 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.944+0000 7f549fe00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54a015d8d0 con 0x7f54a01569d0 2024-04-20T11:38:22.945 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.944+0000 7f549fe00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f54a015d310 con 0x7f54a00d05f0 2024-04-20T11:38:22.945 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.944+0000 7f549fe00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f54a015d5f0 con 0x7f54a00ad760 2024-04-20T11:38:22.948 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.945+0000 7f549ea00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f54a01569d0 0x7f54a006e510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:44032/0 (socket says 172.21.15.152:44032) 2024-04-20T11:38:22.948 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.945+0000 7f549ea00640 1 -- 172.21.15.152:0/2855049578 learned_addr learned my addr 172.21.15.152:0/2855049578 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:38:22.948 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.945+0000 7f5496000640 1 --2- 172.21.15.152:0/2855049578 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f54a00ad760 0x7f54a015ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:22.948 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.945+0000 7f549f400640 1 --2- 172.21.15.152:0/2855049578 >> v2:172.21.15.175:3300/0 conn(0x7f54a00d05f0 0x7f54a00d0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:22.948 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.946+0000 7f549e000640 1 -- 172.21.15.152:0/2855049578 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1437485821 0 0) 0x7f54a015d8d0 con 0x7f54a01569d0 2024-04-20T11:38:22.948 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.946+0000 7f549e000640 1 -- 172.21.15.152:0/2855049578 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5480003520 con 0x7f54a01569d0 2024-04-20T11:38:22.948 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.946+0000 7f549e000640 1 -- 172.21.15.152:0/2855049578 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2331801688 0 0) 0x7f5480003520 con 0x7f54a01569d0 2024-04-20T11:38:22.948 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.946+0000 7f549e000640 1 -- 172.21.15.152:0/2855049578 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f54a00ad760 msgr2=0x7f54a015ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:22.948 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.946+0000 7f549e000640 1 --2- 172.21.15.152:0/2855049578 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f54a00ad760 0x7f54a015ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:22.948 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.946+0000 7f549e000640 1 -- 172.21.15.152:0/2855049578 >> v2:172.21.15.175:3300/0 conn(0x7f54a00d05f0 msgr2=0x7f54a00d0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:22.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.946+0000 7f549e000640 1 --2- 172.21.15.152:0/2855049578 >> v2:172.21.15.175:3300/0 conn(0x7f54a00d05f0 0x7f54a00d0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:22.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.946+0000 7f549e000640 1 -- 172.21.15.152:0/2855049578 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54a006dd10 con 0x7f54a01569d0 2024-04-20T11:38:22.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.946+0000 7f549e000640 1 -- 172.21.15.152:0/2855049578 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f5490001670 con 0x7f54a01569d0 2024-04-20T11:38:22.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.946+0000 7f549e000640 1 -- 172.21.15.152:0/2855049578 >> v1:172.21.15.152:6789/0 conn(0x7f54a01569d0 legacy=0x7f54a006e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:22.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.946+0000 7f549e000640 1 --2- 172.21.15.152:0/2855049578 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f54a00ad760 0x7f54800039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:22.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.946+0000 7f549e000640 1 --2- 172.21.15.152:0/2855049578 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5480003f30 0x7f5480004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:22.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.946+0000 7f549e000640 1 --2- 172.21.15.152:0/2855049578 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5480004d40 0x7f54800051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:22.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.946+0000 7f549e000640 1 -- 172.21.15.152:0/2855049578 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f54a015d310 con 0x7f5480004d40 2024-04-20T11:38:22.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.946+0000 7f549e000640 1 -- 172.21.15.152:0/2855049578 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f54a015d5f0 con 0x7f5480003f30 2024-04-20T11:38:22.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.946+0000 7f549e000640 1 -- 172.21.15.152:0/2855049578 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f548000b9c0 con 0x7f54a00ad760 2024-04-20T11:38:22.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.947+0000 7f549f400640 1 --2- 172.21.15.152:0/2855049578 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5480004d40 0x7f54800051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:22.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.947+0000 7f549ea00640 1 --2- 172.21.15.152:0/2855049578 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f54a00ad760 0x7f54800039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:22.950 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.947+0000 7f5496000640 1 --2- 172.21.15.152:0/2855049578 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5480003f30 0x7f5480004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:22.950 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.947+0000 7f549f400640 1 -- 172.21.15.152:0/2855049578 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f54a00ad760 msgr2=0x7f54800039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:22.950 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.947+0000 7f549f400640 1 --2- 172.21.15.152:0/2855049578 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f54a00ad760 0x7f54800039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:22.950 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.947+0000 7f549f400640 1 -- 172.21.15.152:0/2855049578 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5480003f30 msgr2=0x7f5480004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:22.950 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.947+0000 7f549f400640 1 --2- 172.21.15.152:0/2855049578 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5480003f30 0x7f5480004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:22.950 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.947+0000 7f549f400640 1 -- 172.21.15.152:0/2855049578 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5480003520 con 0x7f5480004d40 2024-04-20T11:38:22.951 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.948+0000 7f549f400640 1 --2- 172.21.15.152:0/2855049578 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5480004d40 0x7f54800051c0 secure :-1 s=READY pgs=2443 cs=0 l=1 rev1=1 crypto rx=0x7f5498009fc0 tx=0x7f5480005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60c81ecc619cf67c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:22.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.948+0000 7f549e000640 1 -- 172.21.15.152:0/2855049578 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5498039070 con 0x7f5480004d40 2024-04-20T11:38:22.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.949+0000 7f549e000640 1 -- 172.21.15.152:0/2855049578 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f549802cc40 con 0x7f5480004d40 2024-04-20T11:38:22.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.949+0000 7f549e000640 1 -- 172.21.15.152:0/2855049578 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5498009c10 con 0x7f5480004d40 2024-04-20T11:38:22.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.949+0000 7f549fe00640 1 -- 172.21.15.152:0/2855049578 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5480004d40 msgr2=0x7f54800051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:22.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.949+0000 7f549fe00640 1 --2- 172.21.15.152:0/2855049578 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5480004d40 0x7f54800051c0 secure :-1 s=READY pgs=2443 cs=0 l=1 rev1=1 crypto rx=0x7f5498009fc0 tx=0x7f5480005830 comp rx=0 tx=0).stop 2024-04-20T11:38:22.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.949+0000 7f549f400640 1 -- 172.21.15.152:0/2855049578 reap_dead start 2024-04-20T11:38:22.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.949+0000 7f549fe00640 1 -- 172.21.15.152:0/2855049578 shutdown_connections 2024-04-20T11:38:22.953 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.949+0000 7f549fe00640 1 -- 172.21.15.152:0/2855049578 >> 172.21.15.152:0/2855049578 conn(0x7f54a00b21e0 msgr2=0x7f54a00b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:22.953 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.949+0000 7f549fe00640 1 -- 172.21.15.152:0/2855049578 shutdown_connections 2024-04-20T11:38:22.953 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.949+0000 7f549fe00640 1 -- 172.21.15.152:0/2855049578 wait complete. 2024-04-20T11:38:22.953 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.951+0000 7f549fe00640 1 Processor -- start 2024-04-20T11:38:22.953 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.951+0000 7f549fe00640 1 -- start start 2024-04-20T11:38:22.956 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.952+0000 7f549fe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f54a01f4760 0x7f54a01f4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:22.957 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.952+0000 7f549fe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54a015b5f0 0x7f54a015ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:22.957 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.952+0000 7f549fe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f54a009d630 0x7f54a009dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:22.957 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.952+0000 7f549fe00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f54a015ca10 con 0x7f54a009d630 2024-04-20T11:38:22.957 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.952+0000 7f549fe00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f54a0156db0 con 0x7f54a015b5f0 2024-04-20T11:38:22.957 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.952+0000 7f549fe00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f54a015cb90 con 0x7f54a01f4760 2024-04-20T11:38:22.957 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.952+0000 7f549f400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f54a009d630 0x7f54a009dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:22.957 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.952+0000 7f549f400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f54a009d630 0x7f54a009dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54780/0 (socket says 172.21.15.152:54780) 2024-04-20T11:38:22.957 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.952+0000 7f549f400640 1 -- 172.21.15.152:0/2071615912 learned_addr learned my addr 172.21.15.152:0/2071615912 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:38:22.957 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.953+0000 7f549ea00640 1 --2- 172.21.15.152:0/2071615912 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f54a01f4760 0x7f54a01f4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:22.957 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.953+0000 7f5496000640 1 --2- 172.21.15.152:0/2071615912 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54a015b5f0 0x7f54a015ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:22.957 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.953+0000 7f549f400640 1 -- 172.21.15.152:0/2071615912 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f54a01f4760 msgr2=0x7f54a01f4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:22.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.953+0000 7f549f400640 1 --2- 172.21.15.152:0/2071615912 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f54a01f4760 0x7f54a01f4b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:22.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.953+0000 7f549f400640 1 -- 172.21.15.152:0/2071615912 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54a015b5f0 msgr2=0x7f54a015ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:22.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.953+0000 7f549f400640 1 --2- 172.21.15.152:0/2071615912 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54a015b5f0 0x7f54a015ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:22.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.953+0000 7f549f400640 1 -- 172.21.15.152:0/2071615912 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54a01570d0 con 0x7f54a009d630 2024-04-20T11:38:22.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.953+0000 7f5496000640 1 --2- 172.21.15.152:0/2071615912 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54a015b5f0 0x7f54a015ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:38:22.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.953+0000 7f549f400640 1 --2- 172.21.15.152:0/2071615912 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f54a009d630 0x7f54a009dab0 secure :-1 s=READY pgs=2444 cs=0 l=1 rev1=1 crypto rx=0x7f5498037040 tx=0x7f5498009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:22.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.953+0000 7f5497400640 1 -- 172.21.15.152:0/2071615912 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5498039030 con 0x7f54a009d630 2024-04-20T11:38:22.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.953+0000 7f5497400640 1 -- 172.21.15.152:0/2071615912 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f549803a040 con 0x7f54a009d630 2024-04-20T11:38:22.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.953+0000 7f5497400640 1 -- 172.21.15.152:0/2071615912 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5498009c10 con 0x7f54a009d630 2024-04-20T11:38:22.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.954+0000 7f549fe00640 1 -- 172.21.15.152:0/2071615912 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54a0157370 con 0x7f54a009d630 2024-04-20T11:38:22.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.954+0000 7f549fe00640 1 -- 172.21.15.152:0/2071615912 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54a0157930 con 0x7f54a009d630 2024-04-20T11:38:22.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.954+0000 7f549fe00640 1 -- 172.21.15.152:0/2071615912 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f54a0075e60 con 0x7f54a009d630 2024-04-20T11:38:22.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.956+0000 7f5497400640 1 -- 172.21.15.152:0/2071615912 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 27) ==== 92060+0+0 (secure 0 0 0) 0x7f5498045020 con 0x7f54a009d630 2024-04-20T11:38:22.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.957+0000 7f5497400640 1 --2- 172.21.15.152:0/2071615912 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f547c06d460 0x7f547c06f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:22.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.957+0000 7f5497400640 1 -- 172.21.15.152:0/2071615912 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 4943+0+0 (secure 0 0 0) 0x7f54980abec0 con 0x7f54a009d630 2024-04-20T11:38:22.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.957+0000 7f5497400640 1 --2- 172.21.15.152:0/2071615912 >> [v2:172.21.15.152:6808/1552206819,v1:172.21.15.152:6809/1552206819] conn(0x7f547c073320 0x7f547c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:22.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.957+0000 7f5497400640 1 -- 172.21.15.152:0/2071615912 --> [v2:172.21.15.152:6808/1552206819,v1:172.21.15.152:6809/1552206819] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f549802cc40 con 0x7f547c073320 2024-04-20T11:38:22.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.957+0000 7f5497400640 1 -- 172.21.15.152:0/2071615912 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_get_version_reply(handle=1 version=170) ==== 24+0+0 (secure 0 0 0) 0x7f54980b5050 con 0x7f54a009d630 2024-04-20T11:38:22.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.957+0000 7f5496000640 1 --2- 172.21.15.152:0/2071615912 >> [v2:172.21.15.152:6808/1552206819,v1:172.21.15.152:6809/1552206819] conn(0x7f547c073320 0x7f547c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:22.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.957+0000 7f549ea00640 1 --2- 172.21.15.152:0/2071615912 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f547c06d460 0x7f547c06f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:22.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.958+0000 7f5496000640 1 --2- 172.21.15.152:0/2071615912 >> [v2:172.21.15.152:6808/1552206819,v1:172.21.15.152:6809/1552206819] conn(0x7f547c073320 0x7f547c075740 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:22.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.958+0000 7f549ea00640 1 --2- 172.21.15.152:0/2071615912 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f547c06d460 0x7f547c06f920 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f5490004840 tx=0x7f5490001c90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:22.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.959+0000 7f5497400640 1 -- 172.21.15.152:0/2071615912 <== osd.1 v2:172.21.15.152:6808/1552206819 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f549802cc40 con 0x7f547c073320 2024-04-20T11:38:22.974 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.973+0000 7f549fe00640 1 -- 172.21.15.152:0/2071615912 --> [v2:172.21.15.152:6808/1552206819,v1:172.21.15.152:6809/1552206819] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f54a0160ea0 con 0x7f547c073320 2024-04-20T11:38:22.974 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.973+0000 7f5497400640 1 -- 172.21.15.152:0/2071615912 <== osd.1 v2:172.21.15.152:6808/1552206819 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f54a0160ea0 con 0x7f547c073320 2024-04-20T11:38:22.974 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-04-20T11:38:22.974 INFO:teuthology.orchestra.run.smithi152.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-20T11:38:22.974 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-04-20T11:38:22.975 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.974+0000 7f549fe00640 1 -- 172.21.15.152:0/2071615912 >> [v2:172.21.15.152:6808/1552206819,v1:172.21.15.152:6809/1552206819] conn(0x7f547c073320 msgr2=0x7f547c075740 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:22.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.974+0000 7f549fe00640 1 --2- 172.21.15.152:0/2071615912 >> [v2:172.21.15.152:6808/1552206819,v1:172.21.15.152:6809/1552206819] conn(0x7f547c073320 0x7f547c075740 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:22.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.974+0000 7f549fe00640 1 -- 172.21.15.152:0/2071615912 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f547c06d460 msgr2=0x7f547c06f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:22.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.974+0000 7f549fe00640 1 --2- 172.21.15.152:0/2071615912 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f547c06d460 0x7f547c06f920 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f5490004840 tx=0x7f5490001c90 comp rx=0 tx=0).stop 2024-04-20T11:38:22.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.974+0000 7f549fe00640 1 -- 172.21.15.152:0/2071615912 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f54a009d630 msgr2=0x7f54a009dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:22.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.974+0000 7f549fe00640 1 --2- 172.21.15.152:0/2071615912 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f54a009d630 0x7f54a009dab0 secure :-1 s=READY pgs=2444 cs=0 l=1 rev1=1 crypto rx=0x7f5498037040 tx=0x7f5498009750 comp rx=0 tx=0).stop 2024-04-20T11:38:22.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.974+0000 7f549f400640 1 -- 172.21.15.152:0/2071615912 reap_dead start 2024-04-20T11:38:22.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.974+0000 7f549fe00640 1 -- 172.21.15.152:0/2071615912 shutdown_connections 2024-04-20T11:38:22.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.974+0000 7f549fe00640 1 -- 172.21.15.152:0/2071615912 >> 172.21.15.152:0/2071615912 conn(0x7f54a00b21e0 msgr2=0x7f54a0071c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:22.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.974+0000 7f549fe00640 1 -- 172.21.15.152:0/2071615912 shutdown_connections 2024-04-20T11:38:22.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:22.974+0000 7f549fe00640 1 -- 172.21.15.152:0/2071615912 wait complete. 2024-04-20T11:38:22.994 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-04-20T11:38:23.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.198+0000 7f0ae3200640 1 Processor -- start 2024-04-20T11:38:23.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.198+0000 7f0ae3200640 1 -- start start 2024-04-20T11:38:23.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.200+0000 7f0ae3200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f0adc145f00 0x7f0adc0cd0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:23.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.200+0000 7f0ae3200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0adc06ead0 0x7f0adc0cfe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:23.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.200+0000 7f0ae3200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0adc06f110 con 0x7f0adc0cd610 2024-04-20T11:38:23.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.200+0000 7f0ae3200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f0adc0cda80 con 0x7f0adc145f00 2024-04-20T11:38:23.201 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.200+0000 7f0ae3200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0adc0cf9f0 con 0x7f0adc06ead0 2024-04-20T11:38:23.201 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.200+0000 7f0adbe00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f0adc145f00 0x7f0adc0cd0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:23.201 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.200+0000 7f0adbe00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f0adc145f00 0x7f0adc0cd0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54786/0 (socket says 172.21.15.152:54786) 2024-04-20T11:38:23.201 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.200+0000 7f0adbe00640 1 -- 172.21.15.152:0/2306114004 learned_addr learned my addr 172.21.15.152:0/2306114004 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:38:23.201 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.200+0000 7f0adbe00640 1 -- 172.21.15.152:0/2306114004 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0adc06ead0 msgr2=0x7f0adc0cfe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:23.201 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.200+0000 7f0ae1200640 1 --2- 172.21.15.152:0/2306114004 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0adc06ead0 0x7f0adc0cfe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:23.201 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.200+0000 7f0adbe00640 1 --2- 172.21.15.152:0/2306114004 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0adc06ead0 0x7f0adc0cfe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:23.201 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.200+0000 7f0adbe00640 1 -- 172.21.15.152:0/2306114004 >> v1:172.21.15.175:6789/0 conn(0x7f0adc0cd610 legacy=0x7f0adc06e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:23.201 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.200+0000 7f0adbe00640 1 -- 172.21.15.152:0/2306114004 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0adc06f3f0 con 0x7f0adc145f00 2024-04-20T11:38:23.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.201+0000 7f0adbe00640 1 --2- 172.21.15.152:0/2306114004 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0adc145f00 0x7f0adc0cd0d0 secure :-1 s=READY pgs=2445 cs=0 l=1 rev1=1 crypto rx=0x7f0acc009870 tx=0x7f0acc02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b0c1171fe6bc4f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:23.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.201+0000 7f0adaa00640 1 -- 172.21.15.152:0/2306114004 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0acc035020 con 0x7f0adc145f00 2024-04-20T11:38:23.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.201+0000 7f0adaa00640 1 -- 172.21.15.152:0/2306114004 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0acc031640 con 0x7f0adc145f00 2024-04-20T11:38:23.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.201+0000 7f0adaa00640 1 -- 172.21.15.152:0/2306114004 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0acc02d040 con 0x7f0adc145f00 2024-04-20T11:38:23.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.202+0000 7f0ae3200640 1 -- 172.21.15.152:0/2306114004 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0adc145f00 msgr2=0x7f0adc0cd0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:23.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.202+0000 7f0ae3200640 1 --2- 172.21.15.152:0/2306114004 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0adc145f00 0x7f0adc0cd0d0 secure :-1 s=READY pgs=2445 cs=0 l=1 rev1=1 crypto rx=0x7f0acc009870 tx=0x7f0acc02b8f0 comp rx=0 tx=0).stop 2024-04-20T11:38:23.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.202+0000 7f0ae3200640 1 -- 172.21.15.152:0/2306114004 shutdown_connections 2024-04-20T11:38:23.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.202+0000 7f0ae3200640 1 --2- 172.21.15.152:0/2306114004 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0adc06ead0 0x7f0adc0cfe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:23.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.202+0000 7f0ae3200640 1 --2- 172.21.15.152:0/2306114004 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0adc145f00 0x7f0adc0cd0d0 unknown :-1 s=CLOSED pgs=2445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:23.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.202+0000 7f0ae3200640 1 -- 172.21.15.152:0/2306114004 >> 172.21.15.152:0/2306114004 conn(0x7f0adc067cf0 msgr2=0x7f0adc06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:23.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.202+0000 7f0ae3200640 1 -- 172.21.15.152:0/2306114004 shutdown_connections 2024-04-20T11:38:23.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.202+0000 7f0ae3200640 1 -- 172.21.15.152:0/2306114004 wait complete. 2024-04-20T11:38:23.204 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.203+0000 7f0ae3200640 1 Processor -- start 2024-04-20T11:38:23.204 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.204+0000 7f0ae3200640 1 -- start start 2024-04-20T11:38:23.206 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.204+0000 7f0ae3200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0adc06ead0 0x7f0adc1f57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:23.206 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.206+0000 7f0ae3200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0adc0cd610 0x7f0adc142690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:23.206 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.206+0000 7f0ae3200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0adc145f00 0x7f0adc144590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:23.206 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.206+0000 7f0ae3200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f0adc1462e0 con 0x7f0adc0cd610 2024-04-20T11:38:23.206 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.206+0000 7f0ae3200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0adc0cf680 con 0x7f0adc06ead0 2024-04-20T11:38:23.206 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.206+0000 7f0ae3200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0adc0d03c0 con 0x7f0adc145f00 2024-04-20T11:38:23.206 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.206+0000 7f0adb400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0adc0cd610 0x7f0adc142690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:23.206 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.206+0000 7f0adb400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0adc0cd610 0x7f0adc142690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54802/0 (socket says 172.21.15.152:54802) 2024-04-20T11:38:23.206 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.206+0000 7f0adb400640 1 -- 172.21.15.152:0/1073070749 learned_addr learned my addr 172.21.15.152:0/1073070749 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:38:23.206 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.206+0000 7f0adb400640 1 -- 172.21.15.152:0/1073070749 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0adc145f00 msgr2=0x7f0adc144590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:23.207 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.206+0000 7f0adb400640 1 --2- 172.21.15.152:0/1073070749 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0adc145f00 0x7f0adc144590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:23.207 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.206+0000 7f0adb400640 1 -- 172.21.15.152:0/1073070749 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0adc06ead0 msgr2=0x7f0adc1f57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:23.207 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.206+0000 7f0adb400640 1 --2- 172.21.15.152:0/1073070749 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0adc06ead0 0x7f0adc1f57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:23.207 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.206+0000 7f0adb400640 1 -- 172.21.15.152:0/1073070749 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0adc142bd0 con 0x7f0adc0cd610 2024-04-20T11:38:23.207 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.206+0000 7f0adb400640 1 --2- 172.21.15.152:0/1073070749 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0adc0cd610 0x7f0adc142690 secure :-1 s=READY pgs=2446 cs=0 l=1 rev1=1 crypto rx=0x7f0ac800c430 tx=0x7f0ac800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:23.208 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.206+0000 7f0ad3e00640 1 -- 172.21.15.152:0/1073070749 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ac8011790 con 0x7f0adc0cd610 2024-04-20T11:38:23.208 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.206+0000 7f0ad3e00640 1 -- 172.21.15.152:0/1073070749 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ac8017070 con 0x7f0adc0cd610 2024-04-20T11:38:23.208 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.206+0000 7f0ad3e00640 1 -- 172.21.15.152:0/1073070749 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ac8011bb0 con 0x7f0adc0cd610 2024-04-20T11:38:23.208 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.206+0000 7f0ae3200640 1 -- 172.21.15.152:0/1073070749 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0adc13d910 con 0x7f0adc0cd610 2024-04-20T11:38:23.208 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.206+0000 7f0ae3200640 1 -- 172.21.15.152:0/1073070749 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0adc13ded0 con 0x7f0adc0cd610 2024-04-20T11:38:23.208 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.207+0000 7f0ae3200640 1 -- 172.21.15.152:0/1073070749 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0adc0b38a0 con 0x7f0adc0cd610 2024-04-20T11:38:23.208 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.208+0000 7f0ad3e00640 1 -- 172.21.15.152:0/1073070749 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 27) ==== 92060+0+0 (secure 0 0 0) 0x7f0ac801c460 con 0x7f0adc0cd610 2024-04-20T11:38:23.209 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.209+0000 7f0ad3e00640 1 --2- 172.21.15.152:0/1073070749 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f0abc06d430 0x7f0abc06f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:23.209 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.209+0000 7f0ad3e00640 1 -- 172.21.15.152:0/1073070749 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 4943+0+0 (secure 0 0 0) 0x7f0ac808a420 con 0x7f0adc0cd610 2024-04-20T11:38:23.210 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.210+0000 7f0adbe00640 1 --2- 172.21.15.152:0/1073070749 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f0abc06d430 0x7f0abc06f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:23.211 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.211+0000 7f0adbe00640 1 --2- 172.21.15.152:0/1073070749 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f0abc06d430 0x7f0abc06f8f0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f0acc036000 tx=0x7f0acc030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:23.214 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.215+0000 7f0ad3e00640 1 -- 172.21.15.152:0/1073070749 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f0ac805a2a0 con 0x7f0adc0cd610 2024-04-20T11:38:23.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.417+0000 7f0ae3200640 1 -- 172.21.15.152:0/1073070749 --> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7f0adc0689f0 con 0x7f0abc06d430 2024-04-20T11:38:23.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.417+0000 7f0ad3e00640 1 -- 172.21.15.152:0/1073070749 <== mgr.4109 v2:172.21.15.175:6836/4021250441 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f0adc0689f0 con 0x7f0abc06d430 2024-04-20T11:38:23.417 INFO:teuthology.orchestra.run.smithi152.stderr:instructed osd(s) 1 to deep-scrub 2024-04-20T11:38:23.420 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.419+0000 7f0ae3200640 1 -- 172.21.15.152:0/1073070749 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f0abc06d430 msgr2=0x7f0abc06f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:23.420 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.419+0000 7f0ae3200640 1 --2- 172.21.15.152:0/1073070749 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f0abc06d430 0x7f0abc06f8f0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f0acc036000 tx=0x7f0acc030000 comp rx=0 tx=0).stop 2024-04-20T11:38:23.420 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.419+0000 7f0ae3200640 1 -- 172.21.15.152:0/1073070749 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0adc0cd610 msgr2=0x7f0adc142690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:23.420 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.419+0000 7f0ae3200640 1 --2- 172.21.15.152:0/1073070749 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0adc0cd610 0x7f0adc142690 secure :-1 s=READY pgs=2446 cs=0 l=1 rev1=1 crypto rx=0x7f0ac800c430 tx=0x7f0ac800c8f0 comp rx=0 tx=0).stop 2024-04-20T11:38:23.420 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.419+0000 7f0ae3200640 1 -- 172.21.15.152:0/1073070749 shutdown_connections 2024-04-20T11:38:23.420 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.419+0000 7f0ae3200640 1 --2- 172.21.15.152:0/1073070749 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f0abc06d430 0x7f0abc06f8f0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:23.420 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.419+0000 7f0ae3200640 1 --2- 172.21.15.152:0/1073070749 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0adc145f00 0x7f0adc144590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:23.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.419+0000 7f0ae3200640 1 --2- 172.21.15.152:0/1073070749 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0adc0cd610 0x7f0adc142690 unknown :-1 s=CLOSED pgs=2446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:23.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.419+0000 7f0ae3200640 1 --2- 172.21.15.152:0/1073070749 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0adc06ead0 0x7f0adc1f57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:23.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.419+0000 7f0ae3200640 1 -- 172.21.15.152:0/1073070749 >> 172.21.15.152:0/1073070749 conn(0x7f0adc067cf0 msgr2=0x7f0adc15db10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:23.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.420+0000 7f0ae3200640 1 -- 172.21.15.152:0/1073070749 shutdown_connections 2024-04-20T11:38:23.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.420+0000 7f0ae3200640 1 -- 172.21.15.152:0/1073070749 wait complete. 2024-04-20T11:38:23.440 INFO:tasks.ceph:Scrubbing osd.2 2024-04-20T11:38:23.440 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-04-20T11:38:23.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.643+0000 7fe20ce00640 1 Processor -- start 2024-04-20T11:38:23.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.643+0000 7fe20ce00640 1 -- start start 2024-04-20T11:38:23.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.646+0000 7fe20ce00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fe208145340 0x7fe208160db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:23.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.646+0000 7fe20ce00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2080b4f90 con 0x7fe2080b47a0 2024-04-20T11:38:23.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.646+0000 7fe20ce00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2080b4cb0 con 0x7fe208072530 2024-04-20T11:38:23.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.646+0000 7fe20ce00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fe208145720 con 0x7fe208145340 2024-04-20T11:38:23.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.646+0000 7fe205e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fe208145340 0x7fe208160db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:23.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.646+0000 7fe205e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fe208145340 0x7fe208160db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54816/0 (socket says 172.21.15.152:54816) 2024-04-20T11:38:23.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.646+0000 7fe205e00640 1 -- 172.21.15.152:0/770914884 learned_addr learned my addr 172.21.15.152:0/770914884 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:38:23.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.646+0000 7fe205e00640 1 -- 172.21.15.152:0/770914884 >> v1:172.21.15.175:6789/0 conn(0x7fe208072530 legacy=0x7fe20815cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:23.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.646+0000 7fe205e00640 1 -- 172.21.15.152:0/770914884 >> v1:172.21.15.152:6789/0 conn(0x7fe2080b47a0 legacy=0x7fe208071e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:23.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.646+0000 7fe205e00640 1 -- 172.21.15.152:0/770914884 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2081602d0 con 0x7fe208145340 2024-04-20T11:38:23.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.648+0000 7fe205e00640 1 --2- 172.21.15.152:0/770914884 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe208145340 0x7fe208160db0 secure :-1 s=READY pgs=2448 cs=0 l=1 rev1=1 crypto rx=0x7fe1f8009e50 tx=0x7fe1f802be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=412239f1578c5c12 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:23.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.648+0000 7fe204a00640 1 -- 172.21.15.152:0/770914884 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1f802c9f0 con 0x7fe208145340 2024-04-20T11:38:23.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.648+0000 7fe204a00640 1 -- 172.21.15.152:0/770914884 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1f802cb90 con 0x7fe208145340 2024-04-20T11:38:23.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.648+0000 7fe204a00640 1 -- 172.21.15.152:0/770914884 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1f80343f0 con 0x7fe208145340 2024-04-20T11:38:23.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.648+0000 7fe20ce00640 1 -- 172.21.15.152:0/770914884 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe208145340 msgr2=0x7fe208160db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:23.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.648+0000 7fe20ce00640 1 --2- 172.21.15.152:0/770914884 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe208145340 0x7fe208160db0 secure :-1 s=READY pgs=2448 cs=0 l=1 rev1=1 crypto rx=0x7fe1f8009e50 tx=0x7fe1f802be50 comp rx=0 tx=0).stop 2024-04-20T11:38:23.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.648+0000 7fe20ce00640 1 -- 172.21.15.152:0/770914884 shutdown_connections 2024-04-20T11:38:23.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.648+0000 7fe20ce00640 1 --2- 172.21.15.152:0/770914884 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe208145340 0x7fe208160db0 unknown :-1 s=CLOSED pgs=2448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:23.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.648+0000 7fe20ce00640 1 -- 172.21.15.152:0/770914884 >> 172.21.15.152:0/770914884 conn(0x7fe20806b980 msgr2=0x7fe20806bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:23.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.649+0000 7fe20ce00640 1 -- 172.21.15.152:0/770914884 shutdown_connections 2024-04-20T11:38:23.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.649+0000 7fe20ce00640 1 -- 172.21.15.152:0/770914884 wait complete. 2024-04-20T11:38:23.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.650+0000 7fe20ce00640 1 Processor -- start 2024-04-20T11:38:23.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.650+0000 7fe20ce00640 1 -- start start 2024-04-20T11:38:23.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.651+0000 7fe20ce00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe208072530 0x7fe20815bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:23.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.651+0000 7fe20ce00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe2080b47a0 0x7fe20806ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:23.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.651+0000 7fe20ce00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe208145340 0x7fe20815cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:23.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.651+0000 7fe20ce00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fe2080ceaa0 con 0x7fe2080b47a0 2024-04-20T11:38:23.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.651+0000 7fe20ce00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe208146010 con 0x7fe208145340 2024-04-20T11:38:23.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.651+0000 7fe20ce00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe2080cec20 con 0x7fe208072530 2024-04-20T11:38:23.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.652+0000 7fe205400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe2080b47a0 0x7fe20806ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:23.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.652+0000 7fe205400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe2080b47a0 0x7fe20806ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54822/0 (socket says 172.21.15.152:54822) 2024-04-20T11:38:23.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.652+0000 7fe205400640 1 -- 172.21.15.152:0/1666736649 learned_addr learned my addr 172.21.15.152:0/1666736649 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:38:23.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.652+0000 7fe205400640 1 -- 172.21.15.152:0/1666736649 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe208072530 msgr2=0x7fe20815bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:23.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.652+0000 7fe205400640 1 --2- 172.21.15.152:0/1666736649 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe208072530 0x7fe20815bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:23.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.652+0000 7fe205400640 1 -- 172.21.15.152:0/1666736649 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe208145340 msgr2=0x7fe20815cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:23.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.652+0000 7fe205400640 1 --2- 172.21.15.152:0/1666736649 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe208145340 0x7fe20815cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:23.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.652+0000 7fe205400640 1 -- 172.21.15.152:0/1666736649 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe20815c110 con 0x7fe2080b47a0 2024-04-20T11:38:23.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.652+0000 7fe205400640 1 --2- 172.21.15.152:0/1666736649 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe2080b47a0 0x7fe20806ecc0 secure :-1 s=READY pgs=2449 cs=0 l=1 rev1=1 crypto rx=0x7fe1f000c400 tx=0x7fe1f000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:23.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.653+0000 7fe1fe000640 1 -- 172.21.15.152:0/1666736649 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1f00052d0 con 0x7fe2080b47a0 2024-04-20T11:38:23.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.653+0000 7fe1fe000640 1 -- 172.21.15.152:0/1666736649 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1f0005490 con 0x7fe2080b47a0 2024-04-20T11:38:23.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.653+0000 7fe1fe000640 1 -- 172.21.15.152:0/1666736649 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1f0007260 con 0x7fe2080b47a0 2024-04-20T11:38:23.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.653+0000 7fe20ce00640 1 -- 172.21.15.152:0/1666736649 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe20806df00 con 0x7fe2080b47a0 2024-04-20T11:38:23.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.653+0000 7fe20ce00640 1 -- 172.21.15.152:0/1666736649 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe20806e4c0 con 0x7fe2080b47a0 2024-04-20T11:38:23.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.654+0000 7fe20ce00640 1 -- 172.21.15.152:0/1666736649 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe208068a60 con 0x7fe2080b47a0 2024-04-20T11:38:23.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.655+0000 7fe1fe000640 1 -- 172.21.15.152:0/1666736649 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 27) ==== 92060+0+0 (secure 0 0 0) 0x7fe1f0007400 con 0x7fe2080b47a0 2024-04-20T11:38:23.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.656+0000 7fe1fe000640 1 --2- 172.21.15.152:0/1666736649 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe1e406d430 0x7fe1e406f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:23.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.656+0000 7fe1fe000640 1 -- 172.21.15.152:0/1666736649 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 4943+0+0 (secure 0 0 0) 0x7fe1f0089d00 con 0x7fe2080b47a0 2024-04-20T11:38:23.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.656+0000 7fe1fe000640 1 --2- 172.21.15.152:0/1666736649 >> [v2:172.21.15.152:6800/2519167656,v1:172.21.15.152:6801/2519167656] conn(0x7fe1e40732f0 0x7fe1e4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:23.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.656+0000 7fe1fe000640 1 -- 172.21.15.152:0/1666736649 --> [v2:172.21.15.152:6800/2519167656,v1:172.21.15.152:6801/2519167656] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe1f0009e50 con 0x7fe1e40732f0 2024-04-20T11:38:23.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.656+0000 7fe1fe000640 1 -- 172.21.15.152:0/1666736649 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_get_version_reply(handle=1 version=170) ==== 24+0+0 (secure 0 0 0) 0x7fe1f0093050 con 0x7fe2080b47a0 2024-04-20T11:38:23.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.657+0000 7fe206800640 1 --2- 172.21.15.152:0/1666736649 >> [v2:172.21.15.152:6800/2519167656,v1:172.21.15.152:6801/2519167656] conn(0x7fe1e40732f0 0x7fe1e4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:23.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.657+0000 7fe205e00640 1 --2- 172.21.15.152:0/1666736649 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe1e406d430 0x7fe1e406f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:23.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.657+0000 7fe206800640 1 --2- 172.21.15.152:0/1666736649 >> [v2:172.21.15.152:6800/2519167656,v1:172.21.15.152:6801/2519167656] conn(0x7fe1e40732f0 0x7fe1e4075710 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:23.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.658+0000 7fe205e00640 1 --2- 172.21.15.152:0/1666736649 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe1e406d430 0x7fe1e406f8f0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fe1f8035000 tx=0x7fe1f802e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:23.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.658+0000 7fe1fe000640 1 -- 172.21.15.152:0/1666736649 <== osd.2 v2:172.21.15.152:6800/2519167656 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe1f0009e50 con 0x7fe1e40732f0 2024-04-20T11:38:23.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.673+0000 7fe20ce00640 1 -- 172.21.15.152:0/1666736649 --> [v2:172.21.15.152:6800/2519167656,v1:172.21.15.152:6801/2519167656] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fe2080b4580 con 0x7fe1e40732f0 2024-04-20T11:38:23.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.674+0000 7fe1fe000640 1 -- 172.21.15.152:0/1666736649 <== osd.2 v2:172.21.15.152:6800/2519167656 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fe2080b4580 con 0x7fe1e40732f0 2024-04-20T11:38:23.674 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-04-20T11:38:23.674 INFO:teuthology.orchestra.run.smithi152.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-20T11:38:23.674 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-04-20T11:38:23.675 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.674+0000 7fe20ce00640 1 -- 172.21.15.152:0/1666736649 >> [v2:172.21.15.152:6800/2519167656,v1:172.21.15.152:6801/2519167656] conn(0x7fe1e40732f0 msgr2=0x7fe1e4075710 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:23.675 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.674+0000 7fe20ce00640 1 --2- 172.21.15.152:0/1666736649 >> [v2:172.21.15.152:6800/2519167656,v1:172.21.15.152:6801/2519167656] conn(0x7fe1e40732f0 0x7fe1e4075710 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:23.675 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.674+0000 7fe20ce00640 1 -- 172.21.15.152:0/1666736649 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe1e406d430 msgr2=0x7fe1e406f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:23.675 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.674+0000 7fe20ce00640 1 --2- 172.21.15.152:0/1666736649 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe1e406d430 0x7fe1e406f8f0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fe1f8035000 tx=0x7fe1f802e000 comp rx=0 tx=0).stop 2024-04-20T11:38:23.675 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.674+0000 7fe20ce00640 1 -- 172.21.15.152:0/1666736649 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe2080b47a0 msgr2=0x7fe20806ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:23.675 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.674+0000 7fe20ce00640 1 --2- 172.21.15.152:0/1666736649 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe2080b47a0 0x7fe20806ecc0 secure :-1 s=READY pgs=2449 cs=0 l=1 rev1=1 crypto rx=0x7fe1f000c400 tx=0x7fe1f000c8c0 comp rx=0 tx=0).stop 2024-04-20T11:38:23.675 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.674+0000 7fe206800640 1 -- 172.21.15.152:0/1666736649 reap_dead start 2024-04-20T11:38:23.675 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.674+0000 7fe20ce00640 1 -- 172.21.15.152:0/1666736649 shutdown_connections 2024-04-20T11:38:23.675 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.674+0000 7fe20ce00640 1 -- 172.21.15.152:0/1666736649 >> 172.21.15.152:0/1666736649 conn(0x7fe20806b980 msgr2=0x7fe2081413c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:23.675 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.674+0000 7fe20ce00640 1 -- 172.21.15.152:0/1666736649 shutdown_connections 2024-04-20T11:38:23.675 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.674+0000 7fe20ce00640 1 -- 172.21.15.152:0/1666736649 wait complete. 2024-04-20T11:38:23.692 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-04-20T11:38:23.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.893+0000 7f134fe00640 1 Processor -- start 2024-04-20T11:38:23.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.893+0000 7f134fe00640 1 -- start start 2024-04-20T11:38:23.894 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.894+0000 7f134fe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f135015c7f0 0x7f1350072b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:23.894 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.894+0000 7f134fe00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f13500b4870 0x7f13500ad2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:23.894 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.895+0000 7f134fe00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1350073fc0 con 0x7f1350073b60 2024-04-20T11:38:23.894 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.895+0000 7f134fe00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f13500ce140 con 0x7f13500b4870 2024-04-20T11:38:23.894 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.895+0000 7f134fe00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f13500ce390 con 0x7f135015c7f0 2024-04-20T11:38:23.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.895+0000 7f134ea00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f135015c7f0 0x7f1350072b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:23.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.895+0000 7f134ea00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f135015c7f0 0x7f1350072b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.152:52738/0 (socket says 172.21.15.152:52738) 2024-04-20T11:38:23.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.895+0000 7f134ea00640 1 -- 172.21.15.152:0/3731183241 learned_addr learned my addr 172.21.15.152:0/3731183241 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:38:23.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.896+0000 7f134f400640 1 --2- 172.21.15.152:0/3731183241 >> v2:172.21.15.175:3300/0 conn(0x7f13500b4870 0x7f13500ad2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:23.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.896+0000 7f134ea00640 1 -- 172.21.15.152:0/3731183241 >> v2:172.21.15.175:3300/0 conn(0x7f13500b4870 msgr2=0x7f13500ad2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:23.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.896+0000 7f134ea00640 1 --2- 172.21.15.152:0/3731183241 >> v2:172.21.15.175:3300/0 conn(0x7f13500b4870 0x7f13500ad2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:23.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.896+0000 7f134ea00640 1 -- 172.21.15.152:0/3731183241 >> v1:172.21.15.175:6789/0 conn(0x7f1350073b60 legacy=0x7f13500b4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:23.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.896+0000 7f134ea00640 1 -- 172.21.15.152:0/3731183241 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13500ce670 con 0x7f135015c7f0 2024-04-20T11:38:23.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.898+0000 7f134ea00640 1 --2- 172.21.15.152:0/3731183241 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f135015c7f0 0x7f1350072b70 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f1340009870 tx=0x7f134002b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=891fd16c6b5f6710 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:23.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.898+0000 7f134e000640 1 -- 172.21.15.152:0/3731183241 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1340035020 con 0x7f135015c7f0 2024-04-20T11:38:23.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.898+0000 7f134e000640 1 -- 172.21.15.152:0/3731183241 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1340031640 con 0x7f135015c7f0 2024-04-20T11:38:23.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.898+0000 7f134e000640 1 -- 172.21.15.152:0/3731183241 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f134002d040 con 0x7f135015c7f0 2024-04-20T11:38:23.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.899+0000 7f134fe00640 1 -- 172.21.15.152:0/3731183241 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f135015c7f0 msgr2=0x7f1350072b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:23.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.899+0000 7f134fe00640 1 --2- 172.21.15.152:0/3731183241 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f135015c7f0 0x7f1350072b70 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f1340009870 tx=0x7f134002b8f0 comp rx=0 tx=0).stop 2024-04-20T11:38:23.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.899+0000 7f134fe00640 1 -- 172.21.15.152:0/3731183241 shutdown_connections 2024-04-20T11:38:23.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.899+0000 7f134fe00640 1 --2- 172.21.15.152:0/3731183241 >> v2:172.21.15.175:3300/0 conn(0x7f13500b4870 0x7f13500ad2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:23.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.899+0000 7f134fe00640 1 --2- 172.21.15.152:0/3731183241 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f135015c7f0 0x7f1350072b70 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:23.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.899+0000 7f134fe00640 1 -- 172.21.15.152:0/3731183241 >> 172.21.15.152:0/3731183241 conn(0x7f13500d0260 msgr2=0x7f13500d0670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:23.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.899+0000 7f134fe00640 1 -- 172.21.15.152:0/3731183241 shutdown_connections 2024-04-20T11:38:23.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.899+0000 7f134fe00640 1 -- 172.21.15.152:0/3731183241 wait complete. 2024-04-20T11:38:23.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.901+0000 7f134fe00640 1 Processor -- start 2024-04-20T11:38:23.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.901+0000 7f134fe00640 1 -- start start 2024-04-20T11:38:23.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.902+0000 7f134fe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1350073b60 0x7f1350150950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:23.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.902+0000 7f134fe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f13500b4870 0x7f13501e7240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:23.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.902+0000 7f134fe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f13501e7780 0x7f1350158700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:23.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.902+0000 7f134fe00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f13500cef10 con 0x7f13500b4870 2024-04-20T11:38:23.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.902+0000 7f134fe00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f13500cec30 con 0x7f1350073b60 2024-04-20T11:38:23.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.902+0000 7f134fe00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f13500cf430 con 0x7f13501e7780 2024-04-20T11:38:23.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.902+0000 7f1346000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f13500b4870 0x7f13501e7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:23.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.902+0000 7f1346000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f13500b4870 0x7f13501e7240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54836/0 (socket says 172.21.15.152:54836) 2024-04-20T11:38:23.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.902+0000 7f1346000640 1 -- 172.21.15.152:0/4213560547 learned_addr learned my addr 172.21.15.152:0/4213560547 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:38:23.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.903+0000 7f134ea00640 1 --2- 172.21.15.152:0/4213560547 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1350073b60 0x7f1350150950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:23.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.903+0000 7f134f400640 1 --2- 172.21.15.152:0/4213560547 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f13501e7780 0x7f1350158700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:23.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.903+0000 7f1346000640 1 -- 172.21.15.152:0/4213560547 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f13501e7780 msgr2=0x7f1350158700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:23.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.903+0000 7f1346000640 1 --2- 172.21.15.152:0/4213560547 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f13501e7780 0x7f1350158700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:23.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.903+0000 7f1346000640 1 -- 172.21.15.152:0/4213560547 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1350073b60 msgr2=0x7f1350150950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:23.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.903+0000 7f1346000640 1 --2- 172.21.15.152:0/4213560547 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1350073b60 0x7f1350150950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:23.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.903+0000 7f1346000640 1 -- 172.21.15.152:0/4213560547 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13501e7c20 con 0x7f13500b4870 2024-04-20T11:38:23.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.903+0000 7f134f400640 1 --2- 172.21.15.152:0/4213560547 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f13501e7780 0x7f1350158700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:38:23.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.903+0000 7f1346000640 1 --2- 172.21.15.152:0/4213560547 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f13500b4870 0x7f13501e7240 secure :-1 s=READY pgs=2450 cs=0 l=1 rev1=1 crypto rx=0x7f133800c400 tx=0x7f133800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:23.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.903+0000 7f1347400640 1 -- 172.21.15.152:0/4213560547 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13380052d0 con 0x7f13500b4870 2024-04-20T11:38:23.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.903+0000 7f1347400640 1 -- 172.21.15.152:0/4213560547 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1338005490 con 0x7f13500b4870 2024-04-20T11:38:23.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.903+0000 7f134fe00640 1 -- 172.21.15.152:0/4213560547 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13501e7f00 con 0x7f13500b4870 2024-04-20T11:38:23.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.903+0000 7f1347400640 1 -- 172.21.15.152:0/4213560547 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1338007260 con 0x7f13500b4870 2024-04-20T11:38:23.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.904+0000 7f134fe00640 1 -- 172.21.15.152:0/4213560547 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13501e84c0 con 0x7f13500b4870 2024-04-20T11:38:23.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.905+0000 7f134fe00640 1 -- 172.21.15.152:0/4213560547 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13501e88a0 con 0x7f13500b4870 2024-04-20T11:38:23.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.906+0000 7f1347400640 1 -- 172.21.15.152:0/4213560547 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 27) ==== 92060+0+0 (secure 0 0 0) 0x7f1338007400 con 0x7f13500b4870 2024-04-20T11:38:23.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.907+0000 7f1347400640 1 --2- 172.21.15.152:0/4213560547 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f132c06d460 0x7f132c06f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:23.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.907+0000 7f1347400640 1 -- 172.21.15.152:0/4213560547 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 4943+0+0 (secure 0 0 0) 0x7f1338089c20 con 0x7f13500b4870 2024-04-20T11:38:23.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.908+0000 7f134ea00640 1 --2- 172.21.15.152:0/4213560547 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f132c06d460 0x7f132c06f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:23.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.909+0000 7f134ea00640 1 --2- 172.21.15.152:0/4213560547 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f132c06d460 0x7f132c06f920 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f1340036000 tx=0x7f134003c040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:23.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:23.912+0000 7f1347400640 1 -- 172.21.15.152:0/4213560547 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f13501e88a0 con 0x7f13500b4870 2024-04-20T11:38:24.110 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.111+0000 7f134fe00640 1 -- 172.21.15.152:0/4213560547 --> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7f1350075e60 con 0x7f132c06d460 2024-04-20T11:38:24.111 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.112+0000 7f1347400640 1 -- 172.21.15.152:0/4213560547 <== mgr.4109 v2:172.21.15.175:6836/4021250441 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f1350075e60 con 0x7f132c06d460 2024-04-20T11:38:24.112 INFO:teuthology.orchestra.run.smithi152.stderr:instructed osd(s) 2 to deep-scrub 2024-04-20T11:38:24.114 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.114+0000 7f134fe00640 1 -- 172.21.15.152:0/4213560547 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f132c06d460 msgr2=0x7f132c06f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:24.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.114+0000 7f134fe00640 1 --2- 172.21.15.152:0/4213560547 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f132c06d460 0x7f132c06f920 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f1340036000 tx=0x7f134003c040 comp rx=0 tx=0).stop 2024-04-20T11:38:24.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.114+0000 7f134fe00640 1 -- 172.21.15.152:0/4213560547 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f13500b4870 msgr2=0x7f13501e7240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:24.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.114+0000 7f134fe00640 1 --2- 172.21.15.152:0/4213560547 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f13500b4870 0x7f13501e7240 secure :-1 s=READY pgs=2450 cs=0 l=1 rev1=1 crypto rx=0x7f133800c400 tx=0x7f133800c8c0 comp rx=0 tx=0).stop 2024-04-20T11:38:24.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.114+0000 7f134fe00640 1 -- 172.21.15.152:0/4213560547 shutdown_connections 2024-04-20T11:38:24.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.114+0000 7f134fe00640 1 --2- 172.21.15.152:0/4213560547 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f132c06d460 0x7f132c06f920 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:24.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.114+0000 7f134fe00640 1 --2- 172.21.15.152:0/4213560547 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f13501e7780 0x7f1350158700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:24.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.114+0000 7f134fe00640 1 --2- 172.21.15.152:0/4213560547 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f13500b4870 0x7f13501e7240 unknown :-1 s=CLOSED pgs=2450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:24.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.114+0000 7f134fe00640 1 --2- 172.21.15.152:0/4213560547 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1350073b60 0x7f1350150950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:24.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.114+0000 7f134fe00640 1 -- 172.21.15.152:0/4213560547 >> 172.21.15.152:0/4213560547 conn(0x7f13500d0260 msgr2=0x7f13500cf860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:24.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.115+0000 7f134fe00640 1 -- 172.21.15.152:0/4213560547 shutdown_connections 2024-04-20T11:38:24.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.115+0000 7f134fe00640 1 -- 172.21.15.152:0/4213560547 wait complete. 2024-04-20T11:38:24.133 INFO:tasks.ceph:Scrubbing osd.3 2024-04-20T11:38:24.134 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-04-20T11:38:24.282 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:38:24.290 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:38:24.331 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.331+0000 7fd810a00640 1 Processor -- start 2024-04-20T11:38:24.331 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.331+0000 7fd810a00640 1 -- start start 2024-04-20T11:38:24.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.333+0000 7fd810a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd80c1624f0 0x7fd80c0b5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:24.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.333+0000 7fd810a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fd80c0ccf20 0x7fd80c06eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:24.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.333+0000 7fd810a00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd80c0b42a0 con 0x7fd80c1628d0 2024-04-20T11:38:24.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.333+0000 7fd810a00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fd80c0b55a0 con 0x7fd80c0ccf20 2024-04-20T11:38:24.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.333+0000 7fd810a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd80c06f400 con 0x7fd80c1624f0 2024-04-20T11:38:24.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.334+0000 7fd809400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fd80c0ccf20 0x7fd80c06eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:24.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.334+0000 7fd809400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fd80c0ccf20 0x7fd80c06eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54850/0 (socket says 172.21.15.152:54850) 2024-04-20T11:38:24.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.334+0000 7fd809400640 1 -- 172.21.15.152:0/3677639303 learned_addr learned my addr 172.21.15.152:0/3677639303 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:38:24.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.334+0000 7fd80a800640 1 --1- 172.21.15.152:0/3677639303 >> v1:172.21.15.152:6789/0 conn(0x7fd80c1628d0 0x7fd80c06b980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:44050/0 (socket says 172.21.15.152:44050) 2024-04-20T11:38:24.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.334+0000 7fd809e00640 1 --2- 172.21.15.152:0/3677639303 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd80c1624f0 0x7fd80c0b5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:24.335 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.334+0000 7fd809400640 1 -- 172.21.15.152:0/3677639303 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd80c1624f0 msgr2=0x7fd80c0b5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:24.335 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.334+0000 7fd809400640 1 --2- 172.21.15.152:0/3677639303 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd80c1624f0 0x7fd80c0b5060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:24.335 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.334+0000 7fd809400640 1 -- 172.21.15.152:0/3677639303 >> v1:172.21.15.152:6789/0 conn(0x7fd80c1628d0 legacy=0x7fd80c06b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:24.335 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.334+0000 7fd809400640 1 -- 172.21.15.152:0/3677639303 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd80c0b4580 con 0x7fd80c0ccf20 2024-04-20T11:38:24.335 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.334+0000 7fd809e00640 1 --2- 172.21.15.152:0/3677639303 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd80c1624f0 0x7fd80c0b5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:38:24.336 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.335+0000 7fd809400640 1 --2- 172.21.15.152:0/3677639303 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd80c0ccf20 0x7fd80c06eec0 secure :-1 s=READY pgs=2452 cs=0 l=1 rev1=1 crypto rx=0x7fd80c06f580 tx=0x7fd7f802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7882ca53c3593c24 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:24.336 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.336+0000 7fd808a00640 1 -- 172.21.15.152:0/3677639303 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7f8034070 con 0x7fd80c0ccf20 2024-04-20T11:38:24.336 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.336+0000 7fd808a00640 1 -- 172.21.15.152:0/3677639303 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7f8030840 con 0x7fd80c0ccf20 2024-04-20T11:38:24.336 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.336+0000 7fd808a00640 1 -- 172.21.15.152:0/3677639303 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7f8030c10 con 0x7fd80c0ccf20 2024-04-20T11:38:24.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.336+0000 7fd810a00640 1 -- 172.21.15.152:0/3677639303 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd80c0ccf20 msgr2=0x7fd80c06eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:24.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.336+0000 7fd810a00640 1 --2- 172.21.15.152:0/3677639303 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd80c0ccf20 0x7fd80c06eec0 secure :-1 s=READY pgs=2452 cs=0 l=1 rev1=1 crypto rx=0x7fd80c06f580 tx=0x7fd7f802b980 comp rx=0 tx=0).stop 2024-04-20T11:38:24.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.337+0000 7fd810a00640 1 -- 172.21.15.152:0/3677639303 shutdown_connections 2024-04-20T11:38:24.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.337+0000 7fd810a00640 1 --2- 172.21.15.152:0/3677639303 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd80c0ccf20 0x7fd80c06eec0 unknown :-1 s=CLOSED pgs=2452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:24.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.337+0000 7fd810a00640 1 --2- 172.21.15.152:0/3677639303 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd80c1624f0 0x7fd80c0b5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:24.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.337+0000 7fd810a00640 1 -- 172.21.15.152:0/3677639303 >> 172.21.15.152:0/3677639303 conn(0x7fd80c0cd780 msgr2=0x7fd80c0cdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:24.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.337+0000 7fd810a00640 1 -- 172.21.15.152:0/3677639303 shutdown_connections 2024-04-20T11:38:24.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.337+0000 7fd810a00640 1 -- 172.21.15.152:0/3677639303 wait complete. 2024-04-20T11:38:24.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.339+0000 7fd810a00640 1 Processor -- start 2024-04-20T11:38:24.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.340+0000 7fd810a00640 1 -- start start 2024-04-20T11:38:24.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.341+0000 7fd810a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd80c0ccf20 0x7fd80c143310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:24.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.341+0000 7fd810a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd80c1624f0 0x7fd80c142990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:24.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.341+0000 7fd810a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd80c1628d0 0x7fd80c1528d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:24.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.341+0000 7fd810a00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fd80c0cd380 con 0x7fd80c0ccf20 2024-04-20T11:38:24.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.341+0000 7fd810a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd80c0b4b40 con 0x7fd80c1628d0 2024-04-20T11:38:24.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.341+0000 7fd810a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd80c0b3f70 con 0x7fd80c1624f0 2024-04-20T11:38:24.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.341+0000 7fd809e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd80c0ccf20 0x7fd80c143310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:24.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.341+0000 7fd809e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd80c0ccf20 0x7fd80c143310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54860/0 (socket says 172.21.15.152:54860) 2024-04-20T11:38:24.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.341+0000 7fd809e00640 1 -- 172.21.15.152:0/271340145 learned_addr learned my addr 172.21.15.152:0/271340145 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:38:24.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.342+0000 7fd809400640 1 --2- 172.21.15.152:0/271340145 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd80c1624f0 0x7fd80c142990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:24.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.342+0000 7fd80a800640 1 --2- 172.21.15.152:0/271340145 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd80c1628d0 0x7fd80c1528d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:24.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.342+0000 7fd809e00640 1 -- 172.21.15.152:0/271340145 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd80c1624f0 msgr2=0x7fd80c142990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:24.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.342+0000 7fd809e00640 1 --2- 172.21.15.152:0/271340145 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd80c1624f0 0x7fd80c142990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:24.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.342+0000 7fd809e00640 1 -- 172.21.15.152:0/271340145 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd80c1628d0 msgr2=0x7fd80c1528d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:24.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.342+0000 7fd809e00640 1 --2- 172.21.15.152:0/271340145 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd80c1628d0 0x7fd80c1528d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:24.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.342+0000 7fd809e00640 1 -- 172.21.15.152:0/271340145 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd80c160030 con 0x7fd80c0ccf20 2024-04-20T11:38:24.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.342+0000 7fd809e00640 1 --2- 172.21.15.152:0/271340145 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd80c0ccf20 0x7fd80c143310 secure :-1 s=READY pgs=2453 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc00e810 tx=0x7fd7fc00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:24.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.343+0000 7fd802000640 1 -- 172.21.15.152:0/271340145 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7fc00f040 con 0x7fd80c0ccf20 2024-04-20T11:38:24.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.343+0000 7fd802000640 1 -- 172.21.15.152:0/271340145 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7fc004030 con 0x7fd80c0ccf20 2024-04-20T11:38:24.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.343+0000 7fd802000640 1 -- 172.21.15.152:0/271340145 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7fc004400 con 0x7fd80c0ccf20 2024-04-20T11:38:24.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.343+0000 7fd810a00640 1 -- 172.21.15.152:0/271340145 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd80c160310 con 0x7fd80c0ccf20 2024-04-20T11:38:24.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.343+0000 7fd810a00640 1 -- 172.21.15.152:0/271340145 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd80c1608d0 con 0x7fd80c0ccf20 2024-04-20T11:38:24.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.344+0000 7fd810a00640 1 -- 172.21.15.152:0/271340145 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd80c068a60 con 0x7fd80c0ccf20 2024-04-20T11:38:24.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.345+0000 7fd802000640 1 -- 172.21.15.152:0/271340145 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 27) ==== 92060+0+0 (secure 0 0 0) 0x7fd7fc00c6d0 con 0x7fd80c0ccf20 2024-04-20T11:38:24.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.346+0000 7fd802000640 1 --2- 172.21.15.152:0/271340145 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd7ec06d430 0x7fd7ec06f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:24.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.346+0000 7fd802000640 1 -- 172.21.15.152:0/271340145 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 4943+0+0 (secure 0 0 0) 0x7fd7fc08a4b0 con 0x7fd80c0ccf20 2024-04-20T11:38:24.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.346+0000 7fd802000640 1 --2- 172.21.15.152:0/271340145 >> [v2:172.21.15.152:6816/3995926281,v1:172.21.15.152:6817/3995926281] conn(0x7fd7ec0732f0 0x7fd7ec075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:24.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.346+0000 7fd802000640 1 -- 172.21.15.152:0/271340145 --> [v2:172.21.15.152:6816/3995926281,v1:172.21.15.152:6817/3995926281] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd7ec075dc0 con 0x7fd7ec0732f0 2024-04-20T11:38:24.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.346+0000 7fd802000640 1 -- 172.21.15.152:0/271340145 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_get_version_reply(handle=1 version=170) ==== 24+0+0 (secure 0 0 0) 0x7fd7fc092070 con 0x7fd80c0ccf20 2024-04-20T11:38:24.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.346+0000 7fd809400640 1 --2- 172.21.15.152:0/271340145 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd7ec06d430 0x7fd7ec06f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:24.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.346+0000 7fd80a800640 1 --2- 172.21.15.152:0/271340145 >> [v2:172.21.15.152:6816/3995926281,v1:172.21.15.152:6817/3995926281] conn(0x7fd7ec0732f0 0x7fd7ec075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:24.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.347+0000 7fd80a800640 1 --2- 172.21.15.152:0/271340145 >> [v2:172.21.15.152:6816/3995926281,v1:172.21.15.152:6817/3995926281] conn(0x7fd7ec0732f0 0x7fd7ec075710 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:24.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.347+0000 7fd809400640 1 --2- 172.21.15.152:0/271340145 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd7ec06d430 0x7fd7ec06f8f0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fd7f8036000 tx=0x7fd7f8008030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:24.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.348+0000 7fd802000640 1 -- 172.21.15.152:0/271340145 <== osd.3 v2:172.21.15.152:6816/3995926281 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd7ec075dc0 con 0x7fd7ec0732f0 2024-04-20T11:38:24.366 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.367+0000 7fd810a00640 1 -- 172.21.15.152:0/271340145 --> [v2:172.21.15.152:6816/3995926281,v1:172.21.15.152:6817/3995926281] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fd80c075040 con 0x7fd7ec0732f0 2024-04-20T11:38:24.366 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.367+0000 7fd802000640 1 -- 172.21.15.152:0/271340145 <== osd.3 v2:172.21.15.152:6816/3995926281 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fd80c075040 con 0x7fd7ec0732f0 2024-04-20T11:38:24.367 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-04-20T11:38:24.367 INFO:teuthology.orchestra.run.smithi152.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-20T11:38:24.367 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-04-20T11:38:24.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.367+0000 7fd810a00640 1 -- 172.21.15.152:0/271340145 >> [v2:172.21.15.152:6816/3995926281,v1:172.21.15.152:6817/3995926281] conn(0x7fd7ec0732f0 msgr2=0x7fd7ec075710 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:24.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.367+0000 7fd810a00640 1 --2- 172.21.15.152:0/271340145 >> [v2:172.21.15.152:6816/3995926281,v1:172.21.15.152:6817/3995926281] conn(0x7fd7ec0732f0 0x7fd7ec075710 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:24.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.368+0000 7fd810a00640 1 -- 172.21.15.152:0/271340145 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd7ec06d430 msgr2=0x7fd7ec06f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:24.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.368+0000 7fd810a00640 1 --2- 172.21.15.152:0/271340145 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fd7ec06d430 0x7fd7ec06f8f0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fd7f8036000 tx=0x7fd7f8008030 comp rx=0 tx=0).stop 2024-04-20T11:38:24.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.368+0000 7fd810a00640 1 -- 172.21.15.152:0/271340145 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd80c0ccf20 msgr2=0x7fd80c143310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:24.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.368+0000 7fd810a00640 1 --2- 172.21.15.152:0/271340145 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd80c0ccf20 0x7fd80c143310 secure :-1 s=READY pgs=2453 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc00e810 tx=0x7fd7fc00ecd0 comp rx=0 tx=0).stop 2024-04-20T11:38:24.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.368+0000 7fd80a800640 1 -- 172.21.15.152:0/271340145 reap_dead start 2024-04-20T11:38:24.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.368+0000 7fd810a00640 1 -- 172.21.15.152:0/271340145 shutdown_connections 2024-04-20T11:38:24.369 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.368+0000 7fd810a00640 1 -- 172.21.15.152:0/271340145 >> 172.21.15.152:0/271340145 conn(0x7fd80c0cd780 msgr2=0x7fd80c0cdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:24.369 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.368+0000 7fd810a00640 1 -- 172.21.15.152:0/271340145 shutdown_connections 2024-04-20T11:38:24.369 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.368+0000 7fd810a00640 1 -- 172.21.15.152:0/271340145 wait complete. 2024-04-20T11:38:24.385 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-04-20T11:38:24.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.652+0000 7f1351800640 1 Processor -- start 2024-04-20T11:38:24.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.653+0000 7f1351800640 1 -- start start 2024-04-20T11:38:24.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.654+0000 7f1351800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f134c15c7f0 0x7f134c072b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:24.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.654+0000 7f1351800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f134c073b60 0x7f134c0ad2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:24.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.655+0000 7f1351800640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f134c073fc0 con 0x7f134c0b4160 2024-04-20T11:38:24.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.655+0000 7f1351800640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f134c0ce140 con 0x7f134c15c7f0 2024-04-20T11:38:24.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.655+0000 7f1351800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f134c0ce390 con 0x7f134c073b60 2024-04-20T11:38:24.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.655+0000 7f134be00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f134c15c7f0 0x7f134c072b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:24.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.655+0000 7f1350e00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f134c0b4160 0x7f134c0cf430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:44056/0 (socket says 172.21.15.152:44056) 2024-04-20T11:38:24.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.655+0000 7f1350e00640 1 -- 172.21.15.152:0/1338753388 learned_addr learned my addr 172.21.15.152:0/1338753388 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:38:24.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.655+0000 7f1343400640 1 --2- 172.21.15.152:0/1338753388 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f134c073b60 0x7f134c0ad2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:24.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.655+0000 7f134be00640 1 -- 172.21.15.152:0/1338753388 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f134c073b60 msgr2=0x7f134c0ad2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:24.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.655+0000 7f134be00640 1 --2- 172.21.15.152:0/1338753388 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f134c073b60 0x7f134c0ad2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:24.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.655+0000 7f134be00640 1 -- 172.21.15.152:0/1338753388 >> v1:172.21.15.152:6789/0 conn(0x7f134c0b4160 legacy=0x7f134c0cf430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:24.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.655+0000 7f134be00640 1 -- 172.21.15.152:0/1338753388 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f134c0ce670 con 0x7f134c15c7f0 2024-04-20T11:38:24.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.655+0000 7f134b400640 1 -- 172.21.15.152:0/1338753388 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1600958354 0 0) 0x7f134c073fc0 con 0x7f134c0b4160 2024-04-20T11:38:24.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.656+0000 7f134be00640 1 --2- 172.21.15.152:0/1338753388 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f134c15c7f0 0x7f134c072b70 secure :-1 s=READY pgs=2455 cs=0 l=1 rev1=1 crypto rx=0x7f133c0097a0 tx=0x7f133c02b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61552a6e375639e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:24.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.657+0000 7f134b400640 1 -- 172.21.15.152:0/1338753388 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f133c031780 con 0x7f134c15c7f0 2024-04-20T11:38:24.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.657+0000 7f134b400640 1 -- 172.21.15.152:0/1338753388 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f133c037070 con 0x7f134c15c7f0 2024-04-20T11:38:24.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.657+0000 7f134b400640 1 -- 172.21.15.152:0/1338753388 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f133c031b40 con 0x7f134c15c7f0 2024-04-20T11:38:24.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.657+0000 7f1351800640 1 -- 172.21.15.152:0/1338753388 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f134c15c7f0 msgr2=0x7f134c072b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:24.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.657+0000 7f1351800640 1 --2- 172.21.15.152:0/1338753388 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f134c15c7f0 0x7f134c072b70 secure :-1 s=READY pgs=2455 cs=0 l=1 rev1=1 crypto rx=0x7f133c0097a0 tx=0x7f133c02b820 comp rx=0 tx=0).stop 2024-04-20T11:38:24.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.657+0000 7f1351800640 1 -- 172.21.15.152:0/1338753388 shutdown_connections 2024-04-20T11:38:24.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.657+0000 7f1351800640 1 --2- 172.21.15.152:0/1338753388 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f134c073b60 0x7f134c0ad2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:24.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.657+0000 7f1351800640 1 --2- 172.21.15.152:0/1338753388 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f134c15c7f0 0x7f134c072b70 unknown :-1 s=CLOSED pgs=2455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:24.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.657+0000 7f1351800640 1 -- 172.21.15.152:0/1338753388 >> 172.21.15.152:0/1338753388 conn(0x7f134c0d0260 msgr2=0x7f134c0d0670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:24.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.658+0000 7f1351800640 1 -- 172.21.15.152:0/1338753388 shutdown_connections 2024-04-20T11:38:24.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.658+0000 7f1351800640 1 -- 172.21.15.152:0/1338753388 wait complete. 2024-04-20T11:38:24.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.659+0000 7f1351800640 1 Processor -- start 2024-04-20T11:38:24.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.659+0000 7f1351800640 1 -- start start 2024-04-20T11:38:24.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.660+0000 7f1351800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f134c073b60 0x7f134c1f4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:24.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.660+0000 7f1351800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f134c0b4160 0x7f134c14ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:24.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.660+0000 7f1351800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f134c150b80 0x7f134c158700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:24.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.660+0000 7f1351800640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f134c0cef10 con 0x7f134c0b4160 2024-04-20T11:38:24.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.660+0000 7f1351800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f134c0cec30 con 0x7f134c073b60 2024-04-20T11:38:24.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.660+0000 7f1351800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f134c0b4590 con 0x7f134c150b80 2024-04-20T11:38:24.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.661+0000 7f1343400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f134c0b4160 0x7f134c14ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:24.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.661+0000 7f134be00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f134c073b60 0x7f134c1f4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:24.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.661+0000 7f1350e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f134c150b80 0x7f134c158700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:24.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.661+0000 7f1343400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f134c0b4160 0x7f134c14ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54882/0 (socket says 172.21.15.152:54882) 2024-04-20T11:38:24.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.661+0000 7f1343400640 1 -- 172.21.15.152:0/1172097427 learned_addr learned my addr 172.21.15.152:0/1172097427 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:38:24.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.661+0000 7f1343400640 1 -- 172.21.15.152:0/1172097427 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f134c150b80 msgr2=0x7f134c158700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:24.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.661+0000 7f1343400640 1 --2- 172.21.15.152:0/1172097427 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f134c150b80 0x7f134c158700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:24.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.661+0000 7f1343400640 1 -- 172.21.15.152:0/1172097427 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f134c073b60 msgr2=0x7f134c1f4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:24.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.661+0000 7f1343400640 1 --2- 172.21.15.152:0/1172097427 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f134c073b60 0x7f134c1f4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:24.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.661+0000 7f1343400640 1 -- 172.21.15.152:0/1172097427 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f134c1f5180 con 0x7f134c0b4160 2024-04-20T11:38:24.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.661+0000 7f1350e00640 1 --2- 172.21.15.152:0/1172097427 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f134c150b80 0x7f134c158700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:38:24.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.662+0000 7f1343400640 1 --2- 172.21.15.152:0/1172097427 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f134c0b4160 0x7f134c14ddd0 secure :-1 s=READY pgs=2456 cs=0 l=1 rev1=1 crypto rx=0x7f133400c7f0 tx=0x7f133400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:24.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.662+0000 7f1348c00640 1 -- 172.21.15.152:0/1172097427 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1334015070 con 0x7f134c0b4160 2024-04-20T11:38:24.664 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.662+0000 7f1348c00640 1 -- 172.21.15.152:0/1172097427 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1334004240 con 0x7f134c0b4160 2024-04-20T11:38:24.664 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.662+0000 7f1348c00640 1 -- 172.21.15.152:0/1172097427 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1334007620 con 0x7f134c0b4160 2024-04-20T11:38:24.664 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.662+0000 7f1351800640 1 -- 172.21.15.152:0/1172097427 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f134c0b1bd0 con 0x7f134c0b4160 2024-04-20T11:38:24.664 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.662+0000 7f1351800640 1 -- 172.21.15.152:0/1172097427 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f134c14fdc0 con 0x7f134c0b4160 2024-04-20T11:38:24.664 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.663+0000 7f1351800640 1 -- 172.21.15.152:0/1172097427 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f134c075e60 con 0x7f134c0b4160 2024-04-20T11:38:24.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.665+0000 7f1348c00640 1 -- 172.21.15.152:0/1172097427 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 27) ==== 92060+0+0 (secure 0 0 0) 0x7f13340079d0 con 0x7f134c0b4160 2024-04-20T11:38:24.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.665+0000 7f1348c00640 1 --2- 172.21.15.152:0/1172097427 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f132806d430 0x7f132806f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:24.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.666+0000 7f1348c00640 1 -- 172.21.15.152:0/1172097427 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 4943+0+0 (secure 0 0 0) 0x7f133408a720 con 0x7f134c0b4160 2024-04-20T11:38:24.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.666+0000 7f134be00640 1 --2- 172.21.15.152:0/1172097427 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f132806d430 0x7f132806f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:24.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.667+0000 7f134be00640 1 --2- 172.21.15.152:0/1172097427 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f132806d430 0x7f132806f8f0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f133c036000 tx=0x7f133c008030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:24.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.673+0000 7f1348c00640 1 -- 172.21.15.152:0/1172097427 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f134c1f5180 con 0x7f134c0b4160 2024-04-20T11:38:24.878 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.878+0000 7f1351800640 1 -- 172.21.15.152:0/1172097427 --> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7f134c0b0af0 con 0x7f132806d430 2024-04-20T11:38:24.879 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.879+0000 7f1348c00640 1 -- 172.21.15.152:0/1172097427 <== mgr.4109 v2:172.21.15.175:6836/4021250441 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f134c0b0af0 con 0x7f132806d430 2024-04-20T11:38:24.879 INFO:teuthology.orchestra.run.smithi152.stderr:instructed osd(s) 3 to deep-scrub 2024-04-20T11:38:24.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.882+0000 7f1351800640 1 -- 172.21.15.152:0/1172097427 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f132806d430 msgr2=0x7f132806f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:24.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.882+0000 7f1351800640 1 --2- 172.21.15.152:0/1172097427 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f132806d430 0x7f132806f8f0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f133c036000 tx=0x7f133c008030 comp rx=0 tx=0).stop 2024-04-20T11:38:24.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.882+0000 7f1351800640 1 -- 172.21.15.152:0/1172097427 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f134c0b4160 msgr2=0x7f134c14ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:24.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.882+0000 7f1351800640 1 --2- 172.21.15.152:0/1172097427 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f134c0b4160 0x7f134c14ddd0 secure :-1 s=READY pgs=2456 cs=0 l=1 rev1=1 crypto rx=0x7f133400c7f0 tx=0x7f133400ccb0 comp rx=0 tx=0).stop 2024-04-20T11:38:24.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.882+0000 7f1351800640 1 -- 172.21.15.152:0/1172097427 shutdown_connections 2024-04-20T11:38:24.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.882+0000 7f1351800640 1 --2- 172.21.15.152:0/1172097427 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f132806d430 0x7f132806f8f0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:24.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.882+0000 7f1351800640 1 --2- 172.21.15.152:0/1172097427 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f134c150b80 0x7f134c158700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:24.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.882+0000 7f1351800640 1 --2- 172.21.15.152:0/1172097427 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f134c0b4160 0x7f134c14ddd0 unknown :-1 s=CLOSED pgs=2456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:24.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.882+0000 7f1351800640 1 --2- 172.21.15.152:0/1172097427 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f134c073b60 0x7f134c1f4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:24.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.882+0000 7f1351800640 1 -- 172.21.15.152:0/1172097427 >> 172.21.15.152:0/1172097427 conn(0x7f134c0d0260 msgr2=0x7f134c0af2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:24.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.883+0000 7f1351800640 1 -- 172.21.15.152:0/1172097427 shutdown_connections 2024-04-20T11:38:24.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:24.883+0000 7f1351800640 1 -- 172.21.15.152:0/1172097427 wait complete. 2024-04-20T11:38:24.902 INFO:tasks.ceph:Scrubbing osd.4 2024-04-20T11:38:24.902 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-04-20T11:38:25.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.116+0000 7f0b56200640 1 Processor -- start 2024-04-20T11:38:25.116 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.116+0000 7f0b56200640 1 -- start start 2024-04-20T11:38:25.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.117+0000 7f0b56200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b501569d0 0x7f0b500ad760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:25.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.117+0000 7f0b56200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f0b5006e970 0x7f0b500d05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:25.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.117+0000 7f0b56200640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b5015d8d0 con 0x7f0b5006e510 2024-04-20T11:38:25.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.117+0000 7f0b56200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f0b5015d310 con 0x7f0b5006e970 2024-04-20T11:38:25.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.117+0000 7f0b56200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0b5015d5f0 con 0x7f0b501569d0 2024-04-20T11:38:25.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.117+0000 7f0b4fe00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f0b5006e510 0x7f0b5015ca10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:44060/0 (socket says 172.21.15.152:44060) 2024-04-20T11:38:25.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.117+0000 7f0b4fe00640 1 -- 172.21.15.152:0/12272482 learned_addr learned my addr 172.21.15.152:0/12272482 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:38:25.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.118+0000 7f0b54e00640 1 --2- 172.21.15.152:0/12272482 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b501569d0 0x7f0b500ad760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:25.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.118+0000 7f0b4f400640 1 -- 172.21.15.152:0/12272482 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2827618676 0 0) 0x7f0b5015d8d0 con 0x7f0b5006e510 2024-04-20T11:38:25.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.118+0000 7f0b55800640 1 --2- 172.21.15.152:0/12272482 >> v2:172.21.15.175:3300/0 conn(0x7f0b5006e970 0x7f0b500d05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:25.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.118+0000 7f0b4f400640 1 -- 172.21.15.152:0/12272482 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0b34003520 con 0x7f0b5006e510 2024-04-20T11:38:25.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.118+0000 7f0b4f400640 1 -- 172.21.15.152:0/12272482 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1167163518 0 0) 0x7f0b34003520 con 0x7f0b5006e510 2024-04-20T11:38:25.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.118+0000 7f0b4f400640 1 -- 172.21.15.152:0/12272482 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b501569d0 msgr2=0x7f0b500ad760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:25.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.118+0000 7f0b4f400640 1 --2- 172.21.15.152:0/12272482 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b501569d0 0x7f0b500ad760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:25.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.118+0000 7f0b4f400640 1 -- 172.21.15.152:0/12272482 >> v2:172.21.15.175:3300/0 conn(0x7f0b5006e970 msgr2=0x7f0b500d05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:25.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.118+0000 7f0b4f400640 1 --2- 172.21.15.152:0/12272482 >> v2:172.21.15.175:3300/0 conn(0x7f0b5006e970 0x7f0b500d05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:25.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.118+0000 7f0b4f400640 1 -- 172.21.15.152:0/12272482 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b5006dd10 con 0x7f0b5006e510 2024-04-20T11:38:25.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.118+0000 7f0b4f400640 1 -- 172.21.15.152:0/12272482 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f0b3c001690 con 0x7f0b5006e510 2024-04-20T11:38:25.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.118+0000 7f0b4f400640 1 -- 172.21.15.152:0/12272482 >> v1:172.21.15.152:6789/0 conn(0x7f0b5006e510 legacy=0x7f0b5015ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:25.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.118+0000 7f0b4f400640 1 --2- 172.21.15.152:0/12272482 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0b340039b0 0x7f0b34003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:25.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.118+0000 7f0b4f400640 1 --2- 172.21.15.152:0/12272482 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b501569d0 0x7f0b34004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:25.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.118+0000 7f0b4f400640 1 --2- 172.21.15.152:0/12272482 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b34004d40 0x7f0b340051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:25.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.118+0000 7f0b4f400640 1 -- 172.21.15.152:0/12272482 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f0b5015d310 con 0x7f0b340039b0 2024-04-20T11:38:25.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.118+0000 7f0b4f400640 1 -- 172.21.15.152:0/12272482 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0b5015d5f0 con 0x7f0b34004d40 2024-04-20T11:38:25.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.118+0000 7f0b4f400640 1 -- 172.21.15.152:0/12272482 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0b3400bb20 con 0x7f0b501569d0 2024-04-20T11:38:25.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.119+0000 7f0b54e00640 1 --2- 172.21.15.152:0/12272482 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0b340039b0 0x7f0b34003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:25.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.119+0000 7f0b4fe00640 1 --2- 172.21.15.152:0/12272482 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b501569d0 0x7f0b34004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:25.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.119+0000 7f0b55800640 1 --2- 172.21.15.152:0/12272482 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b34004d40 0x7f0b340051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:25.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.119+0000 7f0b54e00640 1 -- 172.21.15.152:0/12272482 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b501569d0 msgr2=0x7f0b34004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:25.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.119+0000 7f0b54e00640 1 --2- 172.21.15.152:0/12272482 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b501569d0 0x7f0b34004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:25.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.119+0000 7f0b54e00640 1 -- 172.21.15.152:0/12272482 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b34004d40 msgr2=0x7f0b340051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:25.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.119+0000 7f0b54e00640 1 --2- 172.21.15.152:0/12272482 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b34004d40 0x7f0b340051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:25.122 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.119+0000 7f0b54e00640 1 -- 172.21.15.152:0/12272482 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b34003520 con 0x7f0b340039b0 2024-04-20T11:38:25.122 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.121+0000 7f0b54e00640 1 --2- 172.21.15.152:0/12272482 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0b340039b0 0x7f0b34003dd0 secure :-1 s=READY pgs=2458 cs=0 l=1 rev1=1 crypto rx=0x7f0b4000ba30 tx=0x7f0b34005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=966e9eb0173054b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:25.122 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.121+0000 7f0b4f400640 1 -- 172.21.15.152:0/12272482 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b40036020 con 0x7f0b340039b0 2024-04-20T11:38:25.122 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.121+0000 7f0b4f400640 1 -- 172.21.15.152:0/12272482 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b40032640 con 0x7f0b340039b0 2024-04-20T11:38:25.123 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.121+0000 7f0b4f400640 1 -- 172.21.15.152:0/12272482 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b4000b600 con 0x7f0b340039b0 2024-04-20T11:38:25.124 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.121+0000 7f0b56200640 1 -- 172.21.15.152:0/12272482 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0b340039b0 msgr2=0x7f0b34003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:25.124 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.121+0000 7f0b56200640 1 --2- 172.21.15.152:0/12272482 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0b340039b0 0x7f0b34003dd0 secure :-1 s=READY pgs=2458 cs=0 l=1 rev1=1 crypto rx=0x7f0b4000ba30 tx=0x7f0b34005830 comp rx=0 tx=0).stop 2024-04-20T11:38:25.124 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.121+0000 7f0b55800640 1 -- 172.21.15.152:0/12272482 reap_dead start 2024-04-20T11:38:25.124 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.122+0000 7f0b56200640 1 -- 172.21.15.152:0/12272482 shutdown_connections 2024-04-20T11:38:25.124 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.122+0000 7f0b56200640 1 -- 172.21.15.152:0/12272482 >> 172.21.15.152:0/12272482 conn(0x7f0b500b21e0 msgr2=0x7f0b500b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:25.124 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.122+0000 7f0b56200640 1 -- 172.21.15.152:0/12272482 shutdown_connections 2024-04-20T11:38:25.124 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.122+0000 7f0b56200640 1 -- 172.21.15.152:0/12272482 wait complete. 2024-04-20T11:38:25.124 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.123+0000 7f0b56200640 1 Processor -- start 2024-04-20T11:38:25.124 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.124+0000 7f0b56200640 1 -- start start 2024-04-20T11:38:25.126 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.124+0000 7f0b56200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0b340039b0 0x7f0b501f4150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:25.126 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.124+0000 7f0b56200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b500b1020 0x7f0b500affc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:25.126 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.124+0000 7f0b56200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b500b1460 0x7f0b500b0500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:25.126 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.125+0000 7f0b56200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f0b500ad760 con 0x7f0b340039b0 2024-04-20T11:38:25.126 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.125+0000 7f0b56200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0b50156db0 con 0x7f0b500b1020 2024-04-20T11:38:25.127 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.125+0000 7f0b56200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0b500ad8e0 con 0x7f0b500b1460 2024-04-20T11:38:25.127 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.125+0000 7f0b54e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0b340039b0 0x7f0b501f4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:25.127 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.125+0000 7f0b54e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0b340039b0 0x7f0b501f4150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54904/0 (socket says 172.21.15.152:54904) 2024-04-20T11:38:25.127 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.125+0000 7f0b54e00640 1 -- 172.21.15.152:0/2155339581 learned_addr learned my addr 172.21.15.152:0/2155339581 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:38:25.127 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.125+0000 7f0b54e00640 1 -- 172.21.15.152:0/2155339581 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b500b1460 msgr2=0x7f0b500b0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:25.127 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.125+0000 7f0b54e00640 1 --2- 172.21.15.152:0/2155339581 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b500b1460 0x7f0b500b0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:25.127 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.125+0000 7f0b54e00640 1 -- 172.21.15.152:0/2155339581 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b500b1020 msgr2=0x7f0b500affc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:25.127 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.125+0000 7f0b54e00640 1 --2- 172.21.15.152:0/2155339581 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b500b1020 0x7f0b500affc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:25.127 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.125+0000 7f0b54e00640 1 -- 172.21.15.152:0/2155339581 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b501f4690 con 0x7f0b340039b0 2024-04-20T11:38:25.127 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.126+0000 7f0b54e00640 1 --2- 172.21.15.152:0/2155339581 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0b340039b0 0x7f0b501f4150 secure :-1 s=READY pgs=2459 cs=0 l=1 rev1=1 crypto rx=0x7f0b4000bb50 tx=0x7f0b40004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:25.127 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.126+0000 7f0b4cc00640 1 -- 172.21.15.152:0/2155339581 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b40041070 con 0x7f0b340039b0 2024-04-20T11:38:25.127 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.126+0000 7f0b4cc00640 1 -- 172.21.15.152:0/2155339581 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b40030a50 con 0x7f0b340039b0 2024-04-20T11:38:25.128 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.126+0000 7f0b56200640 1 -- 172.21.15.152:0/2155339581 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b50153700 con 0x7f0b340039b0 2024-04-20T11:38:25.128 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.126+0000 7f0b4cc00640 1 -- 172.21.15.152:0/2155339581 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b40036040 con 0x7f0b340039b0 2024-04-20T11:38:25.128 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.126+0000 7f0b56200640 1 -- 172.21.15.152:0/2155339581 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b50153cc0 con 0x7f0b340039b0 2024-04-20T11:38:25.128 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.127+0000 7f0b56200640 1 -- 172.21.15.152:0/2155339581 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0b50075e60 con 0x7f0b340039b0 2024-04-20T11:38:25.129 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.128+0000 7f0b4cc00640 1 -- 172.21.15.152:0/2155339581 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 27) ==== 92060+0+0 (secure 0 0 0) 0x7f0b40039020 con 0x7f0b340039b0 2024-04-20T11:38:25.129 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.129+0000 7f0b4cc00640 1 --2- 172.21.15.152:0/2155339581 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f0b3006d430 0x7f0b3006f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:25.129 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.129+0000 7f0b4cc00640 1 -- 172.21.15.152:0/2155339581 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 4943+0+0 (secure 0 0 0) 0x7f0b400ac3d0 con 0x7f0b340039b0 2024-04-20T11:38:25.130 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.129+0000 7f0b4cc00640 1 --2- 172.21.15.152:0/2155339581 >> [v2:172.21.15.175:6800/1235215467,v1:172.21.15.175:6802/1235215467] conn(0x7f0b300732f0 0x7f0b30075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:25.130 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.129+0000 7f0b4cc00640 1 -- 172.21.15.152:0/2155339581 --> [v2:172.21.15.175:6800/1235215467,v1:172.21.15.175:6802/1235215467] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0b4004de50 con 0x7f0b300732f0 2024-04-20T11:38:25.130 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.129+0000 7f0b4cc00640 1 -- 172.21.15.152:0/2155339581 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_get_version_reply(handle=1 version=170) ==== 24+0+0 (secure 0 0 0) 0x7f0b400b32a0 con 0x7f0b340039b0 2024-04-20T11:38:25.130 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.130+0000 7f0b4fe00640 1 --2- 172.21.15.152:0/2155339581 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f0b3006d430 0x7f0b3006f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:25.130 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.130+0000 7f0b55800640 1 --2- 172.21.15.152:0/2155339581 >> [v2:172.21.15.175:6800/1235215467,v1:172.21.15.175:6802/1235215467] conn(0x7f0b300732f0 0x7f0b30075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:25.131 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.131+0000 7f0b4fe00640 1 --2- 172.21.15.152:0/2155339581 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f0b3006d430 0x7f0b3006f8f0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c001eb0 tx=0x7f0b3c001dc0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:25.131 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.131+0000 7f0b55800640 1 --2- 172.21.15.152:0/2155339581 >> [v2:172.21.15.175:6800/1235215467,v1:172.21.15.175:6802/1235215467] conn(0x7f0b300732f0 0x7f0b30075710 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:25.133 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.133+0000 7f0b4cc00640 1 -- 172.21.15.152:0/2155339581 <== osd.4 v2:172.21.15.175:6800/1235215467 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0b4004de50 con 0x7f0b300732f0 2024-04-20T11:38:25.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.148+0000 7f0b56200640 1 -- 172.21.15.152:0/2155339581 --> [v2:172.21.15.175:6800/1235215467,v1:172.21.15.175:6802/1235215467] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f0b5006f440 con 0x7f0b300732f0 2024-04-20T11:38:25.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.149+0000 7f0b4cc00640 1 -- 172.21.15.152:0/2155339581 <== osd.4 v2:172.21.15.175:6800/1235215467 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f0b5006f440 con 0x7f0b300732f0 2024-04-20T11:38:25.149 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-04-20T11:38:25.149 INFO:teuthology.orchestra.run.smithi152.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-20T11:38:25.149 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-04-20T11:38:25.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.149+0000 7f0b56200640 1 -- 172.21.15.152:0/2155339581 >> [v2:172.21.15.175:6800/1235215467,v1:172.21.15.175:6802/1235215467] conn(0x7f0b300732f0 msgr2=0x7f0b30075710 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:25.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.149+0000 7f0b56200640 1 --2- 172.21.15.152:0/2155339581 >> [v2:172.21.15.175:6800/1235215467,v1:172.21.15.175:6802/1235215467] conn(0x7f0b300732f0 0x7f0b30075710 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:25.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.150+0000 7f0b56200640 1 -- 172.21.15.152:0/2155339581 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f0b3006d430 msgr2=0x7f0b3006f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:25.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.150+0000 7f0b56200640 1 --2- 172.21.15.152:0/2155339581 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f0b3006d430 0x7f0b3006f8f0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c001eb0 tx=0x7f0b3c001dc0 comp rx=0 tx=0).stop 2024-04-20T11:38:25.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.150+0000 7f0b56200640 1 -- 172.21.15.152:0/2155339581 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0b340039b0 msgr2=0x7f0b501f4150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:25.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.150+0000 7f0b56200640 1 --2- 172.21.15.152:0/2155339581 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0b340039b0 0x7f0b501f4150 secure :-1 s=READY pgs=2459 cs=0 l=1 rev1=1 crypto rx=0x7f0b4000bb50 tx=0x7f0b40004730 comp rx=0 tx=0).stop 2024-04-20T11:38:25.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.150+0000 7f0b55800640 1 -- 172.21.15.152:0/2155339581 reap_dead start 2024-04-20T11:38:25.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.150+0000 7f0b56200640 1 -- 172.21.15.152:0/2155339581 shutdown_connections 2024-04-20T11:38:25.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.150+0000 7f0b56200640 1 -- 172.21.15.152:0/2155339581 >> 172.21.15.152:0/2155339581 conn(0x7f0b500b21e0 msgr2=0x7f0b5015ef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:25.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.150+0000 7f0b56200640 1 -- 172.21.15.152:0/2155339581 shutdown_connections 2024-04-20T11:38:25.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.150+0000 7f0b56200640 1 -- 172.21.15.152:0/2155339581 wait complete. 2024-04-20T11:38:25.169 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-04-20T11:38:25.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.365+0000 7f346dc00640 1 Processor -- start 2024-04-20T11:38:25.366 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.366+0000 7f346dc00640 1 -- start start 2024-04-20T11:38:25.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.367+0000 7f346dc00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3468145f00 0x7f34680cd0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:25.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.367+0000 7f346dc00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f34680cd610 0x7f346806e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:25.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.367+0000 7f346dc00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f346806f110 con 0x7f346806e900 2024-04-20T11:38:25.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.367+0000 7f346dc00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f34680cf9f0 con 0x7f34680cd610 2024-04-20T11:38:25.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.367+0000 7f346dc00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f34680cda50 con 0x7f3468145f00 2024-04-20T11:38:25.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.368+0000 7f345fe00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f34680cd610 0x7f346806e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:25.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.368+0000 7f3467400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f346806e900 0x7f34681601c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:44072/0 (socket says 172.21.15.152:44072) 2024-04-20T11:38:25.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.368+0000 7f3467400640 1 -- 172.21.15.152:0/2241716040 learned_addr learned my addr 172.21.15.152:0/2241716040 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:38:25.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.368+0000 7f3466a00640 1 --2- 172.21.15.152:0/2241716040 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3468145f00 0x7f34680cd0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:25.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.368+0000 7f345fe00640 1 -- 172.21.15.152:0/2241716040 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3468145f00 msgr2=0x7f34680cd0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:25.369 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.368+0000 7f345fe00640 1 --2- 172.21.15.152:0/2241716040 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3468145f00 0x7f34680cd0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:25.369 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.368+0000 7f345fe00640 1 -- 172.21.15.152:0/2241716040 >> v1:172.21.15.152:6789/0 conn(0x7f346806e900 legacy=0x7f34681601c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:25.369 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.368+0000 7f345fe00640 1 -- 172.21.15.152:0/2241716040 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f346806f3f0 con 0x7f34680cd610 2024-04-20T11:38:25.370 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.369+0000 7f345fe00640 1 --2- 172.21.15.152:0/2241716040 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f34680cd610 0x7f346806e3c0 secure :-1 s=READY pgs=2461 cs=0 l=1 rev1=1 crypto rx=0x7f34500097a0 tx=0x7f345002b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ea98e48e0d44645 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:25.370 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.369+0000 7f3466000640 1 -- 172.21.15.152:0/2241716040 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3450035020 con 0x7f34680cd610 2024-04-20T11:38:25.370 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.370+0000 7f3466000640 1 -- 172.21.15.152:0/2241716040 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3450031810 con 0x7f34680cd610 2024-04-20T11:38:25.370 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.370+0000 7f3466000640 1 -- 172.21.15.152:0/2241716040 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3450031be0 con 0x7f34680cd610 2024-04-20T11:38:25.370 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.370+0000 7f346dc00640 1 -- 172.21.15.152:0/2241716040 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f34680cd610 msgr2=0x7f346806e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:25.370 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.370+0000 7f346dc00640 1 --2- 172.21.15.152:0/2241716040 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f34680cd610 0x7f346806e3c0 secure :-1 s=READY pgs=2461 cs=0 l=1 rev1=1 crypto rx=0x7f34500097a0 tx=0x7f345002b8b0 comp rx=0 tx=0).stop 2024-04-20T11:38:25.371 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.370+0000 7f346dc00640 1 -- 172.21.15.152:0/2241716040 shutdown_connections 2024-04-20T11:38:25.371 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.370+0000 7f346dc00640 1 --2- 172.21.15.152:0/2241716040 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f34680cd610 0x7f346806e3c0 unknown :-1 s=CLOSED pgs=2461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:25.371 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.370+0000 7f346dc00640 1 --2- 172.21.15.152:0/2241716040 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3468145f00 0x7f34680cd0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:25.371 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.370+0000 7f346dc00640 1 -- 172.21.15.152:0/2241716040 >> 172.21.15.152:0/2241716040 conn(0x7f3468067cf0 msgr2=0x7f346806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:25.371 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.370+0000 7f346dc00640 1 -- 172.21.15.152:0/2241716040 shutdown_connections 2024-04-20T11:38:25.371 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.370+0000 7f346dc00640 1 -- 172.21.15.152:0/2241716040 wait complete. 2024-04-20T11:38:25.371 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.372+0000 7f346dc00640 1 Processor -- start 2024-04-20T11:38:25.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.372+0000 7f346dc00640 1 -- start start 2024-04-20T11:38:25.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.373+0000 7f346dc00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f346806e900 0x7f34680c5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:25.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.373+0000 7f346dc00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f34680cd610 0x7f34680c59b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:25.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.373+0000 7f346dc00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3468145f00 0x7f3468142bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:25.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.373+0000 7f346dc00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f34681462e0 con 0x7f34680cd610 2024-04-20T11:38:25.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.373+0000 7f346dc00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f34680cf680 con 0x7f3468145f00 2024-04-20T11:38:25.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.373+0000 7f346dc00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f346806ed90 con 0x7f346806e900 2024-04-20T11:38:25.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.373+0000 7f345fe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f34680cd610 0x7f34680c59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:25.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.373+0000 7f345fe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f34680cd610 0x7f34680c59b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54928/0 (socket says 172.21.15.152:54928) 2024-04-20T11:38:25.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.374+0000 7f345fe00640 1 -- 172.21.15.152:0/2352567227 learned_addr learned my addr 172.21.15.152:0/2352567227 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:38:25.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.374+0000 7f3466a00640 1 --2- 172.21.15.152:0/2352567227 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f346806e900 0x7f34680c5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:25.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.374+0000 7f345fe00640 1 -- 172.21.15.152:0/2352567227 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f346806e900 msgr2=0x7f34680c5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:25.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.374+0000 7f345fe00640 1 --2- 172.21.15.152:0/2352567227 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f346806e900 0x7f34680c5470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:25.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.374+0000 7f345fe00640 1 -- 172.21.15.152:0/2352567227 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3468145f00 msgr2=0x7f3468142bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:25.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.374+0000 7f345fe00640 1 --2- 172.21.15.152:0/2352567227 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3468145f00 0x7f3468142bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:25.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.374+0000 7f345fe00640 1 -- 172.21.15.152:0/2352567227 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34680c6230 con 0x7f34680cd610 2024-04-20T11:38:25.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.374+0000 7f345fe00640 1 --2- 172.21.15.152:0/2352567227 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f34680cd610 0x7f34680c59b0 secure :-1 s=READY pgs=2462 cs=0 l=1 rev1=1 crypto rx=0x7f3450009890 tx=0x7f34500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:25.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.375+0000 7f345ea00640 1 -- 172.21.15.152:0/2352567227 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3450041070 con 0x7f34680cd610 2024-04-20T11:38:25.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.375+0000 7f345ea00640 1 -- 172.21.15.152:0/2352567227 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f345003f070 con 0x7f34680cd610 2024-04-20T11:38:25.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.375+0000 7f345ea00640 1 -- 172.21.15.152:0/2352567227 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3450035040 con 0x7f34680cd610 2024-04-20T11:38:25.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.375+0000 7f346dc00640 1 -- 172.21.15.152:0/2352567227 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34680c6510 con 0x7f34680cd610 2024-04-20T11:38:25.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.375+0000 7f346dc00640 1 -- 172.21.15.152:0/2352567227 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34680c6ad0 con 0x7f34680cd610 2024-04-20T11:38:25.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.376+0000 7f346dc00640 1 -- 172.21.15.152:0/2352567227 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34680689f0 con 0x7f34680cd610 2024-04-20T11:38:25.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.377+0000 7f345ea00640 1 -- 172.21.15.152:0/2352567227 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 27) ==== 92060+0+0 (secure 0 0 0) 0x7f3450038020 con 0x7f34680cd610 2024-04-20T11:38:25.378 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.378+0000 7f345ea00640 1 --2- 172.21.15.152:0/2352567227 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f344406d430 0x7f344406f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:25.378 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.378+0000 7f345ea00640 1 -- 172.21.15.152:0/2352567227 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 4943+0+0 (secure 0 0 0) 0x7f34500ac400 con 0x7f34680cd610 2024-04-20T11:38:25.378 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.379+0000 7f3466a00640 1 --2- 172.21.15.152:0/2352567227 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f344406d430 0x7f344406f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:25.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.380+0000 7f3466a00640 1 --2- 172.21.15.152:0/2352567227 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f344406d430 0x7f344406f8f0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f3458006fd0 tx=0x7f3458008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:25.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.382+0000 7f345ea00640 1 -- 172.21.15.152:0/2352567227 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f345007c280 con 0x7f34680cd610 2024-04-20T11:38:25.582 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.583+0000 7f346dc00640 1 -- 172.21.15.152:0/2352567227 --> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7f3468072a50 con 0x7f344406d430 2024-04-20T11:38:25.583 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.584+0000 7f345ea00640 1 -- 172.21.15.152:0/2352567227 <== mgr.4109 v2:172.21.15.175:6836/4021250441 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f3468072a50 con 0x7f344406d430 2024-04-20T11:38:25.583 INFO:teuthology.orchestra.run.smithi152.stderr:instructed osd(s) 4 to deep-scrub 2024-04-20T11:38:25.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.587+0000 7f346dc00640 1 -- 172.21.15.152:0/2352567227 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f344406d430 msgr2=0x7f344406f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:25.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.587+0000 7f346dc00640 1 --2- 172.21.15.152:0/2352567227 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f344406d430 0x7f344406f8f0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f3458006fd0 tx=0x7f3458008040 comp rx=0 tx=0).stop 2024-04-20T11:38:25.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.587+0000 7f346dc00640 1 -- 172.21.15.152:0/2352567227 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f34680cd610 msgr2=0x7f34680c59b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:25.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.587+0000 7f346dc00640 1 --2- 172.21.15.152:0/2352567227 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f34680cd610 0x7f34680c59b0 secure :-1 s=READY pgs=2462 cs=0 l=1 rev1=1 crypto rx=0x7f3450009890 tx=0x7f34500046c0 comp rx=0 tx=0).stop 2024-04-20T11:38:25.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.587+0000 7f346dc00640 1 -- 172.21.15.152:0/2352567227 shutdown_connections 2024-04-20T11:38:25.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.587+0000 7f346dc00640 1 --2- 172.21.15.152:0/2352567227 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f344406d430 0x7f344406f8f0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:25.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.587+0000 7f346dc00640 1 --2- 172.21.15.152:0/2352567227 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3468145f00 0x7f3468142bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:25.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.587+0000 7f346dc00640 1 --2- 172.21.15.152:0/2352567227 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f34680cd610 0x7f34680c59b0 unknown :-1 s=CLOSED pgs=2462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:25.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.587+0000 7f346dc00640 1 --2- 172.21.15.152:0/2352567227 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f346806e900 0x7f34680c5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:25.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.587+0000 7f346dc00640 1 -- 172.21.15.152:0/2352567227 >> 172.21.15.152:0/2352567227 conn(0x7f3468067cf0 msgr2=0x7f34680d0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:25.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.587+0000 7f346dc00640 1 -- 172.21.15.152:0/2352567227 shutdown_connections 2024-04-20T11:38:25.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.587+0000 7f346dc00640 1 -- 172.21.15.152:0/2352567227 wait complete. 2024-04-20T11:38:25.606 INFO:tasks.ceph:Scrubbing osd.5 2024-04-20T11:38:25.606 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-04-20T11:38:25.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.816+0000 7f1e5a200640 1 Processor -- start 2024-04-20T11:38:25.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.817+0000 7f1e5a200640 1 -- start start 2024-04-20T11:38:25.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.818+0000 7f1e5a200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f1e540d0930 0x7f1e5406fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:25.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.818+0000 7f1e5a200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1e540702d0 0x7f1e540ccdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:25.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.818+0000 7f1e5a200640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e5409aae0 con 0x7f1e5415c6f0 2024-04-20T11:38:25.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.818+0000 7f1e5a200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f1e5409adc0 con 0x7f1e540d0930 2024-04-20T11:38:25.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.818+0000 7f1e5a200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1e5409b0a0 con 0x7f1e540702d0 2024-04-20T11:38:25.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.819+0000 7f1e58e00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f1e5415c6f0 0x7f1e5409b8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:48268/0 (socket says 172.21.15.152:48268) 2024-04-20T11:38:25.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.819+0000 7f1e58e00640 1 -- 172.21.15.152:0/3388679544 learned_addr learned my addr 172.21.15.152:0/3388679544 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:38:25.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.819+0000 7f1e53400640 1 -- 172.21.15.152:0/3388679544 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2434739163 0 0) 0x7f1e5409aae0 con 0x7f1e5415c6f0 2024-04-20T11:38:25.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.819+0000 7f1e53400640 1 -- 172.21.15.152:0/3388679544 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1e38003520 con 0x7f1e5415c6f0 2024-04-20T11:38:25.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.819+0000 7f1e53e00640 1 --2- 172.21.15.152:0/3388679544 >> v2:172.21.15.175:3300/0 conn(0x7f1e540d0930 0x7f1e5406fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:25.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.819+0000 7f1e59800640 1 --2- 172.21.15.152:0/3388679544 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1e540702d0 0x7f1e540ccdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:25.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.819+0000 7f1e53400640 1 -- 172.21.15.152:0/3388679544 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2819464460 0 0) 0x7f1e38003520 con 0x7f1e5415c6f0 2024-04-20T11:38:25.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.819+0000 7f1e53400640 1 -- 172.21.15.152:0/3388679544 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1e540702d0 msgr2=0x7f1e540ccdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:25.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.819+0000 7f1e53400640 1 --2- 172.21.15.152:0/3388679544 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1e540702d0 0x7f1e540ccdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:25.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.819+0000 7f1e53400640 1 -- 172.21.15.152:0/3388679544 >> v2:172.21.15.175:3300/0 conn(0x7f1e540d0930 msgr2=0x7f1e5406fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:25.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.819+0000 7f1e53400640 1 --2- 172.21.15.152:0/3388679544 >> v2:172.21.15.175:3300/0 conn(0x7f1e540d0930 0x7f1e5406fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:25.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.819+0000 7f1e53400640 1 -- 172.21.15.152:0/3388679544 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e5409b380 con 0x7f1e5415c6f0 2024-04-20T11:38:25.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.819+0000 7f1e53400640 1 -- 172.21.15.152:0/3388679544 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f1e44001670 con 0x7f1e5415c6f0 2024-04-20T11:38:25.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.820+0000 7f1e53400640 1 -- 172.21.15.152:0/3388679544 >> v1:172.21.15.152:6789/0 conn(0x7f1e5415c6f0 legacy=0x7f1e5409b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:25.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.820+0000 7f1e53400640 1 --2- 172.21.15.152:0/3388679544 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1e380039b0 0x7f1e38003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:25.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.820+0000 7f1e53400640 1 --2- 172.21.15.152:0/3388679544 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1e38004800 0x7f1e38004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:25.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.820+0000 7f1e53400640 1 --2- 172.21.15.152:0/3388679544 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1e540702d0 0x7f1e380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:25.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.820+0000 7f1e53400640 1 -- 172.21.15.152:0/3388679544 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f1e5409adc0 con 0x7f1e38004800 2024-04-20T11:38:25.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.820+0000 7f1e53400640 1 -- 172.21.15.152:0/3388679544 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1e5409b0a0 con 0x7f1e380039b0 2024-04-20T11:38:25.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.820+0000 7f1e53400640 1 -- 172.21.15.152:0/3388679544 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1e3800bac0 con 0x7f1e540702d0 2024-04-20T11:38:25.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.820+0000 7f1e53e00640 1 --2- 172.21.15.152:0/3388679544 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1e38004800 0x7f1e38004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:25.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.820+0000 7f1e59800640 1 --2- 172.21.15.152:0/3388679544 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1e540702d0 0x7f1e380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:25.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.820+0000 7f1e58e00640 1 --2- 172.21.15.152:0/3388679544 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1e380039b0 0x7f1e38003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:25.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.820+0000 7f1e53e00640 1 -- 172.21.15.152:0/3388679544 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1e540702d0 msgr2=0x7f1e380051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:25.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.820+0000 7f1e53e00640 1 --2- 172.21.15.152:0/3388679544 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1e540702d0 0x7f1e380051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:25.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.820+0000 7f1e53e00640 1 -- 172.21.15.152:0/3388679544 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1e380039b0 msgr2=0x7f1e38003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:25.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.820+0000 7f1e53e00640 1 --2- 172.21.15.152:0/3388679544 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1e380039b0 0x7f1e38003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:25.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.820+0000 7f1e53e00640 1 -- 172.21.15.152:0/3388679544 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e38003520 con 0x7f1e38004800 2024-04-20T11:38:25.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.820+0000 7f1e59800640 1 --2- 172.21.15.152:0/3388679544 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1e540702d0 0x7f1e380051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:38:25.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.821+0000 7f1e53e00640 1 --2- 172.21.15.152:0/3388679544 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1e38004800 0x7f1e38004c80 secure :-1 s=READY pgs=2464 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c00ba00 tx=0x7f1e38005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8dd94d8133afd60 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:25.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.822+0000 7f1e53400640 1 -- 172.21.15.152:0/3388679544 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e3c036020 con 0x7f1e38004800 2024-04-20T11:38:25.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.822+0000 7f1e53400640 1 -- 172.21.15.152:0/3388679544 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e3c032630 con 0x7f1e38004800 2024-04-20T11:38:25.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.822+0000 7f1e53400640 1 -- 172.21.15.152:0/3388679544 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e3c00b600 con 0x7f1e38004800 2024-04-20T11:38:25.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.822+0000 7f1e5a200640 1 -- 172.21.15.152:0/3388679544 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1e38004800 msgr2=0x7f1e38004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:25.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.822+0000 7f1e5a200640 1 --2- 172.21.15.152:0/3388679544 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1e38004800 0x7f1e38004c80 secure :-1 s=READY pgs=2464 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c00ba00 tx=0x7f1e38005830 comp rx=0 tx=0).stop 2024-04-20T11:38:25.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.822+0000 7f1e59800640 1 -- 172.21.15.152:0/3388679544 reap_dead start 2024-04-20T11:38:25.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.822+0000 7f1e5a200640 1 -- 172.21.15.152:0/3388679544 shutdown_connections 2024-04-20T11:38:25.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.822+0000 7f1e5a200640 1 -- 172.21.15.152:0/3388679544 >> 172.21.15.152:0/3388679544 conn(0x7f1e540d2350 msgr2=0x7f1e540d2760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:25.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.823+0000 7f1e5a200640 1 -- 172.21.15.152:0/3388679544 shutdown_connections 2024-04-20T11:38:25.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.823+0000 7f1e5a200640 1 -- 172.21.15.152:0/3388679544 wait complete. 2024-04-20T11:38:25.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.824+0000 7f1e5a200640 1 Processor -- start 2024-04-20T11:38:25.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.824+0000 7f1e5a200640 1 -- start start 2024-04-20T11:38:25.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.825+0000 7f1e5a200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1e38004800 0x7f1e540b5d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:25.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.825+0000 7f1e5a200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1e5406e190 0x7f1e540b6760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:25.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.825+0000 7f1e5a200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1e5406e610 0x7f1e541593c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:25.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.825+0000 7f1e5a200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f1e540d1300 con 0x7f1e5406e190 2024-04-20T11:38:25.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.825+0000 7f1e5a200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1e540d1180 con 0x7f1e5406e610 2024-04-20T11:38:25.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.825+0000 7f1e5a200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1e54070760 con 0x7f1e38004800 2024-04-20T11:38:25.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.826+0000 7f1e53e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1e5406e190 0x7f1e540b6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:25.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.826+0000 7f1e53e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1e5406e190 0x7f1e540b6760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52164/0 (socket says 172.21.15.152:52164) 2024-04-20T11:38:25.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.826+0000 7f1e53e00640 1 -- 172.21.15.152:0/2895193932 learned_addr learned my addr 172.21.15.152:0/2895193932 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:38:25.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.826+0000 7f1e58e00640 1 --2- 172.21.15.152:0/2895193932 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1e38004800 0x7f1e540b5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:25.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.826+0000 7f1e53e00640 1 -- 172.21.15.152:0/2895193932 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1e38004800 msgr2=0x7f1e540b5d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:25.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.826+0000 7f1e53e00640 1 --2- 172.21.15.152:0/2895193932 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1e38004800 0x7f1e540b5d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:25.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.826+0000 7f1e53e00640 1 -- 172.21.15.152:0/2895193932 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1e5406e610 msgr2=0x7f1e541593c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:25.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.826+0000 7f1e53e00640 1 --2- 172.21.15.152:0/2895193932 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1e5406e610 0x7f1e541593c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:25.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.826+0000 7f1e53e00640 1 -- 172.21.15.152:0/2895193932 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e540b4f70 con 0x7f1e5406e190 2024-04-20T11:38:25.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.826+0000 7f1e58e00640 1 --2- 172.21.15.152:0/2895193932 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1e38004800 0x7f1e540b5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:38:25.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.826+0000 7f1e53e00640 1 --2- 172.21.15.152:0/2895193932 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1e5406e190 0x7f1e540b6760 secure :-1 s=READY pgs=2465 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c00bb20 tx=0x7f1e3c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:25.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.827+0000 7f1e50c00640 1 -- 172.21.15.152:0/2895193932 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e3c041070 con 0x7f1e5406e190 2024-04-20T11:38:25.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.827+0000 7f1e50c00640 1 -- 172.21.15.152:0/2895193932 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e3c030a40 con 0x7f1e5406e190 2024-04-20T11:38:25.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.827+0000 7f1e50c00640 1 -- 172.21.15.152:0/2895193932 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e3c036040 con 0x7f1e5406e190 2024-04-20T11:38:25.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.827+0000 7f1e5a200640 1 -- 172.21.15.152:0/2895193932 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e540b71b0 con 0x7f1e5406e190 2024-04-20T11:38:25.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.827+0000 7f1e5a200640 1 -- 172.21.15.152:0/2895193932 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e540b5510 con 0x7f1e5406e190 2024-04-20T11:38:25.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.828+0000 7f1e5a200640 1 -- 172.21.15.152:0/2895193932 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1e54075e60 con 0x7f1e5406e190 2024-04-20T11:38:25.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.829+0000 7f1e50c00640 1 -- 172.21.15.152:0/2895193932 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 27) ==== 92060+0+0 (secure 0 0 0) 0x7f1e3c039020 con 0x7f1e5406e190 2024-04-20T11:38:25.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.830+0000 7f1e50c00640 1 --2- 172.21.15.152:0/2895193932 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f1e3006d430 0x7f1e3006f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:25.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.830+0000 7f1e50c00640 1 -- 172.21.15.152:0/2895193932 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 4943+0+0 (secure 0 0 0) 0x7f1e3c0ac720 con 0x7f1e5406e190 2024-04-20T11:38:25.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.830+0000 7f1e50c00640 1 --2- 172.21.15.152:0/2895193932 >> [v2:172.21.15.175:6801/3465937300,v1:172.21.15.175:6804/3465937300] conn(0x7f1e300732f0 0x7f1e30075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:25.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.830+0000 7f1e50c00640 1 -- 172.21.15.152:0/2895193932 --> [v2:172.21.15.175:6801/3465937300,v1:172.21.15.175:6804/3465937300] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1e3c03ce50 con 0x7f1e300732f0 2024-04-20T11:38:25.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.830+0000 7f1e50c00640 1 -- 172.21.15.152:0/2895193932 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_get_version_reply(handle=1 version=170) ==== 24+0+0 (secure 0 0 0) 0x7f1e3c0b32a0 con 0x7f1e5406e190 2024-04-20T11:38:25.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.831+0000 7f1e58e00640 1 --2- 172.21.15.152:0/2895193932 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f1e3006d430 0x7f1e3006f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:25.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.831+0000 7f1e59800640 1 --2- 172.21.15.152:0/2895193932 >> [v2:172.21.15.175:6801/3465937300,v1:172.21.15.175:6804/3465937300] conn(0x7f1e300732f0 0x7f1e30075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:25.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.832+0000 7f1e58e00640 1 --2- 172.21.15.152:0/2895193932 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f1e3006d430 0x7f1e3006f8f0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f1e44000c00 tx=0x7f1e44001bc0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:25.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.832+0000 7f1e59800640 1 --2- 172.21.15.152:0/2895193932 >> [v2:172.21.15.175:6801/3465937300,v1:172.21.15.175:6804/3465937300] conn(0x7f1e300732f0 0x7f1e30075710 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:25.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.834+0000 7f1e50c00640 1 -- 172.21.15.152:0/2895193932 <== osd.5 v2:172.21.15.175:6801/3465937300 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1e3c03ce50 con 0x7f1e300732f0 2024-04-20T11:38:25.848 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.849+0000 7f1e5a200640 1 -- 172.21.15.152:0/2895193932 --> [v2:172.21.15.175:6801/3465937300,v1:172.21.15.175:6804/3465937300] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f1e540b82d0 con 0x7f1e300732f0 2024-04-20T11:38:25.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.850+0000 7f1e50c00640 1 -- 172.21.15.152:0/2895193932 <== osd.5 v2:172.21.15.175:6801/3465937300 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f1e540b82d0 con 0x7f1e300732f0 2024-04-20T11:38:25.850 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-04-20T11:38:25.850 INFO:teuthology.orchestra.run.smithi152.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-20T11:38:25.850 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-04-20T11:38:25.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.850+0000 7f1e5a200640 1 -- 172.21.15.152:0/2895193932 >> [v2:172.21.15.175:6801/3465937300,v1:172.21.15.175:6804/3465937300] conn(0x7f1e300732f0 msgr2=0x7f1e30075710 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:25.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.850+0000 7f1e5a200640 1 --2- 172.21.15.152:0/2895193932 >> [v2:172.21.15.175:6801/3465937300,v1:172.21.15.175:6804/3465937300] conn(0x7f1e300732f0 0x7f1e30075710 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:25.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.850+0000 7f1e5a200640 1 -- 172.21.15.152:0/2895193932 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f1e3006d430 msgr2=0x7f1e3006f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:25.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.850+0000 7f1e5a200640 1 --2- 172.21.15.152:0/2895193932 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f1e3006d430 0x7f1e3006f8f0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f1e44000c00 tx=0x7f1e44001bc0 comp rx=0 tx=0).stop 2024-04-20T11:38:25.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.850+0000 7f1e5a200640 1 -- 172.21.15.152:0/2895193932 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1e5406e190 msgr2=0x7f1e540b6760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:25.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.850+0000 7f1e5a200640 1 --2- 172.21.15.152:0/2895193932 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1e5406e190 0x7f1e540b6760 secure :-1 s=READY pgs=2465 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c00bb20 tx=0x7f1e3c004730 comp rx=0 tx=0).stop 2024-04-20T11:38:25.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.851+0000 7f1e59800640 1 -- 172.21.15.152:0/2895193932 reap_dead start 2024-04-20T11:38:25.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.851+0000 7f1e5a200640 1 -- 172.21.15.152:0/2895193932 shutdown_connections 2024-04-20T11:38:25.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.851+0000 7f1e5a200640 1 -- 172.21.15.152:0/2895193932 >> 172.21.15.152:0/2895193932 conn(0x7f1e540d2350 msgr2=0x7f1e5415b080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:25.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.851+0000 7f1e5a200640 1 -- 172.21.15.152:0/2895193932 shutdown_connections 2024-04-20T11:38:25.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:25.851+0000 7f1e5a200640 1 -- 172.21.15.152:0/2895193932 wait complete. 2024-04-20T11:38:25.868 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-04-20T11:38:26.074 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.074+0000 7f7a7b200640 1 Processor -- start 2024-04-20T11:38:26.074 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.075+0000 7f7a7b200640 1 -- start start 2024-04-20T11:38:26.076 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.076+0000 7f7a7b200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7a7c161d30 0x7f7a7c0d0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:26.076 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.076+0000 7f7a7b200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f7a7c0d2b80 0x7f7a7c0d3030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:26.076 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.076+0000 7f7a7b200640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a7c0d26a0 con 0x7f7a7c0d4ec0 2024-04-20T11:38:26.076 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.076+0000 7f7a7b200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f7a7c0d0540 con 0x7f7a7c0d2b80 2024-04-20T11:38:26.076 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.076+0000 7f7a7b200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7a7c0cfb70 con 0x7f7a7c161d30 2024-04-20T11:38:26.076 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.076+0000 7f7a79400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f7a7c0d4ec0 0x7f7a7c0d15e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:48280/0 (socket says 172.21.15.152:48280) 2024-04-20T11:38:26.077 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.076+0000 7f7a79400640 1 -- 172.21.15.152:0/1277044584 learned_addr learned my addr 172.21.15.152:0/1277044584 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:38:26.078 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.077+0000 7f7a79e00640 1 --2- 172.21.15.152:0/1277044584 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7a7c161d30 0x7f7a7c0d0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:26.079 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.077+0000 7f7a7a800640 1 --2- 172.21.15.152:0/1277044584 >> v2:172.21.15.175:3300/0 conn(0x7f7a7c0d2b80 0x7f7a7c0d3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:26.079 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.077+0000 7f7a78a00640 1 -- 172.21.15.152:0/1277044584 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3894787897 0 0) 0x7f7a7c0d26a0 con 0x7f7a7c0d4ec0 2024-04-20T11:38:26.079 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.077+0000 7f7a78a00640 1 -- 172.21.15.152:0/1277044584 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7a60003520 con 0x7f7a7c0d4ec0 2024-04-20T11:38:26.079 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.077+0000 7f7a78a00640 1 -- 172.21.15.152:0/1277044584 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3184099239 0 0) 0x7f7a60003520 con 0x7f7a7c0d4ec0 2024-04-20T11:38:26.079 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.077+0000 7f7a78a00640 1 -- 172.21.15.152:0/1277044584 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7a7c161d30 msgr2=0x7f7a7c0d0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:26.079 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.077+0000 7f7a78a00640 1 --2- 172.21.15.152:0/1277044584 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7a7c161d30 0x7f7a7c0d0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:26.079 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.077+0000 7f7a78a00640 1 -- 172.21.15.152:0/1277044584 >> v2:172.21.15.175:3300/0 conn(0x7f7a7c0d2b80 msgr2=0x7f7a7c0d3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:26.079 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.077+0000 7f7a78a00640 1 --2- 172.21.15.152:0/1277044584 >> v2:172.21.15.175:3300/0 conn(0x7f7a7c0d2b80 0x7f7a7c0d3030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:26.079 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.077+0000 7f7a78a00640 1 -- 172.21.15.152:0/1277044584 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a7c0d53c0 con 0x7f7a7c0d4ec0 2024-04-20T11:38:26.079 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.077+0000 7f7a78a00640 1 -- 172.21.15.152:0/1277044584 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f7a640017f0 con 0x7f7a7c0d4ec0 2024-04-20T11:38:26.080 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.077+0000 7f7a78a00640 1 -- 172.21.15.152:0/1277044584 >> v1:172.21.15.152:6789/0 conn(0x7f7a7c0d4ec0 legacy=0x7f7a7c0d15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:26.080 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.077+0000 7f7a78a00640 1 --2- 172.21.15.152:0/1277044584 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7a600039b0 0x7f7a60003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:26.080 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.078+0000 7f7a78a00640 1 --2- 172.21.15.152:0/1277044584 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a60004800 0x7f7a60004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:26.080 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.078+0000 7f7a78a00640 1 --2- 172.21.15.152:0/1277044584 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7a7c161d30 0x7f7a600051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:26.080 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.078+0000 7f7a78a00640 1 -- 172.21.15.152:0/1277044584 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f7a7c0d0540 con 0x7f7a600039b0 2024-04-20T11:38:26.080 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.078+0000 7f7a78a00640 1 -- 172.21.15.152:0/1277044584 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7a7c0cfb70 con 0x7f7a60004800 2024-04-20T11:38:26.080 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.078+0000 7f7a78a00640 1 -- 172.21.15.152:0/1277044584 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7a6000b9c0 con 0x7f7a7c161d30 2024-04-20T11:38:26.080 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.078+0000 7f7a79e00640 1 --2- 172.21.15.152:0/1277044584 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7a600039b0 0x7f7a60003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:26.080 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.078+0000 7f7a79e00640 1 -- 172.21.15.152:0/1277044584 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7a7c161d30 msgr2=0x7f7a600051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:26.080 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.078+0000 7f7a79e00640 1 --2- 172.21.15.152:0/1277044584 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7a7c161d30 0x7f7a600051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:26.080 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.078+0000 7f7a79e00640 1 -- 172.21.15.152:0/1277044584 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a60004800 msgr2=0x7f7a60004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:26.080 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.078+0000 7f7a79e00640 1 --2- 172.21.15.152:0/1277044584 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a60004800 0x7f7a60004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:26.081 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.078+0000 7f7a79e00640 1 -- 172.21.15.152:0/1277044584 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a60003520 con 0x7f7a600039b0 2024-04-20T11:38:26.082 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.079+0000 7f7a79e00640 1 --2- 172.21.15.152:0/1277044584 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7a600039b0 0x7f7a60003dd0 secure :-1 s=READY pgs=2467 cs=0 l=1 rev1=1 crypto rx=0x7f7a6c009ac0 tx=0x7f7a60005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b81bdc04f9a6f27c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:26.082 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.080+0000 7f7a78a00640 1 -- 172.21.15.152:0/1277044584 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a6c039070 con 0x7f7a600039b0 2024-04-20T11:38:26.082 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.080+0000 7f7a78a00640 1 -- 172.21.15.152:0/1277044584 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a6c02c710 con 0x7f7a600039b0 2024-04-20T11:38:26.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.080+0000 7f7a78a00640 1 -- 172.21.15.152:0/1277044584 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a6c009690 con 0x7f7a600039b0 2024-04-20T11:38:26.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.080+0000 7f7a7b200640 1 -- 172.21.15.152:0/1277044584 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7a600039b0 msgr2=0x7f7a60003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:26.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.080+0000 7f7a7b200640 1 --2- 172.21.15.152:0/1277044584 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7a600039b0 0x7f7a60003dd0 secure :-1 s=READY pgs=2467 cs=0 l=1 rev1=1 crypto rx=0x7f7a6c009ac0 tx=0x7f7a60005830 comp rx=0 tx=0).stop 2024-04-20T11:38:26.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.080+0000 7f7a7a800640 1 -- 172.21.15.152:0/1277044584 reap_dead start 2024-04-20T11:38:26.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.080+0000 7f7a7b200640 1 -- 172.21.15.152:0/1277044584 shutdown_connections 2024-04-20T11:38:26.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.081+0000 7f7a7b200640 1 -- 172.21.15.152:0/1277044584 >> 172.21.15.152:0/1277044584 conn(0x7f7a7c08bfd0 msgr2=0x7f7a7c0d45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:26.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.081+0000 7f7a7b200640 1 -- 172.21.15.152:0/1277044584 shutdown_connections 2024-04-20T11:38:26.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.081+0000 7f7a7b200640 1 -- 172.21.15.152:0/1277044584 wait complete. 2024-04-20T11:38:26.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.082+0000 7f7a7b200640 1 Processor -- start 2024-04-20T11:38:26.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.082+0000 7f7a7b200640 1 -- start start 2024-04-20T11:38:26.085 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.083+0000 7f7a7b200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7a600039b0 0x7f7a7c15bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:26.085 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.083+0000 7f7a7b200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a7c15a0d0 0x7f7a7c15f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:26.085 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.083+0000 7f7a7b200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7a7c15faa0 0x7f7a7c1f4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:26.085 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.083+0000 7f7a7b200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f7a7c0cf1b0 con 0x7f7a600039b0 2024-04-20T11:38:26.085 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.083+0000 7f7a7b200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7a7c0cf800 con 0x7f7a7c15a0d0 2024-04-20T11:38:26.085 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.083+0000 7f7a7b200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7a7c162110 con 0x7f7a7c15faa0 2024-04-20T11:38:26.086 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.084+0000 7f7a79e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7a600039b0 0x7f7a7c15bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:26.086 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.084+0000 7f7a79e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7a600039b0 0x7f7a7c15bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52182/0 (socket says 172.21.15.152:52182) 2024-04-20T11:38:26.086 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.084+0000 7f7a79e00640 1 -- 172.21.15.152:0/3805915834 learned_addr learned my addr 172.21.15.152:0/3805915834 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:38:26.086 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.084+0000 7f7a7a800640 1 --2- 172.21.15.152:0/3805915834 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7a7c15faa0 0x7f7a7c1f4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:26.086 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.084+0000 7f7a79e00640 1 -- 172.21.15.152:0/3805915834 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7a7c15faa0 msgr2=0x7f7a7c1f4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:26.086 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.084+0000 7f7a79e00640 1 --2- 172.21.15.152:0/3805915834 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7a7c15faa0 0x7f7a7c1f4390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:26.086 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.084+0000 7f7a79e00640 1 -- 172.21.15.152:0/3805915834 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a7c15a0d0 msgr2=0x7f7a7c15f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:26.086 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.084+0000 7f7a79e00640 1 --2- 172.21.15.152:0/3805915834 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a7c15a0d0 0x7f7a7c15f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:26.086 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.084+0000 7f7a79e00640 1 -- 172.21.15.152:0/3805915834 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a7c1f5380 con 0x7f7a600039b0 2024-04-20T11:38:26.086 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.084+0000 7f7a79e00640 1 --2- 172.21.15.152:0/3805915834 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7a600039b0 0x7f7a7c15bb30 secure :-1 s=READY pgs=2468 cs=0 l=1 rev1=1 crypto rx=0x7f7a6c037040 tx=0x7f7a6c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:26.086 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.085+0000 7f7a72000640 1 -- 172.21.15.152:0/3805915834 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a6c042020 con 0x7f7a600039b0 2024-04-20T11:38:26.086 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.085+0000 7f7a72000640 1 -- 172.21.15.152:0/3805915834 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a6c03a040 con 0x7f7a600039b0 2024-04-20T11:38:26.086 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.085+0000 7f7a72000640 1 -- 172.21.15.152:0/3805915834 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a6c039070 con 0x7f7a600039b0 2024-04-20T11:38:26.087 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.085+0000 7f7a7b200640 1 -- 172.21.15.152:0/3805915834 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a7c1f50a0 con 0x7f7a600039b0 2024-04-20T11:38:26.087 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.085+0000 7f7a7b200640 1 -- 172.21.15.152:0/3805915834 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a7c1598d0 con 0x7f7a600039b0 2024-04-20T11:38:26.087 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.086+0000 7f7a7b200640 1 -- 172.21.15.152:0/3805915834 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a7c15a510 con 0x7f7a600039b0 2024-04-20T11:38:26.087 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.087+0000 7f7a72000640 1 -- 172.21.15.152:0/3805915834 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 27) ==== 92060+0+0 (secure 0 0 0) 0x7f7a6c045020 con 0x7f7a600039b0 2024-04-20T11:38:26.088 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.088+0000 7f7a72000640 1 --2- 172.21.15.152:0/3805915834 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f7a5806d460 0x7f7a5806f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:26.088 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.088+0000 7f7a72000640 1 -- 172.21.15.152:0/3805915834 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 4943+0+0 (secure 0 0 0) 0x7f7a6c0abc90 con 0x7f7a600039b0 2024-04-20T11:38:26.088 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.089+0000 7f7a79400640 1 --2- 172.21.15.152:0/3805915834 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f7a5806d460 0x7f7a5806f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:26.089 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.090+0000 7f7a79400640 1 --2- 172.21.15.152:0/3805915834 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f7a5806d460 0x7f7a5806f920 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f7a640010e0 tx=0x7f7a64001c70 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:26.092 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.093+0000 7f7a72000640 1 -- 172.21.15.152:0/3805915834 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f7a6c04c030 con 0x7f7a600039b0 2024-04-20T11:38:26.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.297+0000 7f7a7b200640 1 -- 172.21.15.152:0/3805915834 --> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7f7a7c075e60 con 0x7f7a5806d460 2024-04-20T11:38:26.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.298+0000 7f7a72000640 1 -- 172.21.15.152:0/3805915834 <== mgr.4109 v2:172.21.15.175:6836/4021250441 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f7a7c075e60 con 0x7f7a5806d460 2024-04-20T11:38:26.298 INFO:teuthology.orchestra.run.smithi152.stderr:instructed osd(s) 5 to deep-scrub 2024-04-20T11:38:26.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.301+0000 7f7a7b200640 1 -- 172.21.15.152:0/3805915834 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f7a5806d460 msgr2=0x7f7a5806f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:26.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.301+0000 7f7a7b200640 1 --2- 172.21.15.152:0/3805915834 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f7a5806d460 0x7f7a5806f920 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f7a640010e0 tx=0x7f7a64001c70 comp rx=0 tx=0).stop 2024-04-20T11:38:26.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.301+0000 7f7a7b200640 1 -- 172.21.15.152:0/3805915834 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7a600039b0 msgr2=0x7f7a7c15bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:26.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.301+0000 7f7a7b200640 1 --2- 172.21.15.152:0/3805915834 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7a600039b0 0x7f7a7c15bb30 secure :-1 s=READY pgs=2468 cs=0 l=1 rev1=1 crypto rx=0x7f7a6c037040 tx=0x7f7a6c004730 comp rx=0 tx=0).stop 2024-04-20T11:38:26.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.301+0000 7f7a7b200640 1 -- 172.21.15.152:0/3805915834 shutdown_connections 2024-04-20T11:38:26.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.301+0000 7f7a7b200640 1 --2- 172.21.15.152:0/3805915834 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f7a5806d460 0x7f7a5806f920 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:26.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.301+0000 7f7a7b200640 1 --2- 172.21.15.152:0/3805915834 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7a7c15faa0 0x7f7a7c1f4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:26.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.301+0000 7f7a7b200640 1 --2- 172.21.15.152:0/3805915834 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a7c15a0d0 0x7f7a7c15f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:26.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.301+0000 7f7a7b200640 1 --2- 172.21.15.152:0/3805915834 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7a600039b0 0x7f7a7c15bb30 unknown :-1 s=CLOSED pgs=2468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:26.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.301+0000 7f7a7b200640 1 -- 172.21.15.152:0/3805915834 >> 172.21.15.152:0/3805915834 conn(0x7f7a7c08bfd0 msgr2=0x7f7a7c06d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:26.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.301+0000 7f7a7b200640 1 -- 172.21.15.152:0/3805915834 shutdown_connections 2024-04-20T11:38:26.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.301+0000 7f7a7b200640 1 -- 172.21.15.152:0/3805915834 wait complete. 2024-04-20T11:38:26.320 INFO:tasks.ceph:Scrubbing osd.6 2024-04-20T11:38:26.320 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-04-20T11:38:26.529 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.529+0000 7f812e400640 1 Processor -- start 2024-04-20T11:38:26.530 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.530+0000 7f812e400640 1 -- start start 2024-04-20T11:38:26.532 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.531+0000 7f812e400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f812815e870 0x7f81280d3370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:26.532 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.531+0000 7f812e400640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f81280d06a0 0x7f81280d1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:26.532 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.531+0000 7f812e400640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f812815ec50 con 0x7f81280d0fa0 2024-04-20T11:38:26.532 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.531+0000 7f812e400640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f81280d2ee0 con 0x7f812815e870 2024-04-20T11:38:26.532 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.531+0000 7f812e400640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f81280d38b0 con 0x7f81280d06a0 2024-04-20T11:38:26.532 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.531+0000 7f812d000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f812815e870 0x7f81280d3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:26.532 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.532+0000 7f811fe00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f81280d0fa0 0x7f81280cff90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:48284/0 (socket says 172.21.15.152:48284) 2024-04-20T11:38:26.532 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.532+0000 7f811fe00640 1 -- 172.21.15.152:0/2177138146 learned_addr learned my addr 172.21.15.152:0/2177138146 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:38:26.532 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.532+0000 7f812da00640 1 --2- 172.21.15.152:0/2177138146 >> v2:172.21.15.175:3300/0 conn(0x7f81280d06a0 0x7f81280d1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:26.532 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.532+0000 7f812d000640 1 -- 172.21.15.152:0/2177138146 >> v2:172.21.15.175:3300/0 conn(0x7f81280d06a0 msgr2=0x7f81280d1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:26.533 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.532+0000 7f812d000640 1 --2- 172.21.15.152:0/2177138146 >> v2:172.21.15.175:3300/0 conn(0x7f81280d06a0 0x7f81280d1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:26.533 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.532+0000 7f812d000640 1 -- 172.21.15.152:0/2177138146 >> v1:172.21.15.152:6789/0 conn(0x7f81280d0fa0 legacy=0x7f81280cff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:26.533 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.532+0000 7f8127e00640 1 -- 172.21.15.152:0/2177138146 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2494515020 0 0) 0x7f812815ec50 con 0x7f81280d0fa0 2024-04-20T11:38:26.533 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.532+0000 7f812d000640 1 -- 172.21.15.152:0/2177138146 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81280d22d0 con 0x7f812815e870 2024-04-20T11:38:26.534 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.533+0000 7f812d000640 1 --2- 172.21.15.152:0/2177138146 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f812815e870 0x7f81280d3370 secure :-1 s=READY pgs=2470 cs=0 l=1 rev1=1 crypto rx=0x7f812808cae0 tx=0x7f811802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e478b1314d9307a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:26.534 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.534+0000 7f8127e00640 1 -- 172.21.15.152:0/2177138146 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81180315a0 con 0x7f812815e870 2024-04-20T11:38:26.534 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.534+0000 7f8127e00640 1 -- 172.21.15.152:0/2177138146 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8118037070 con 0x7f812815e870 2024-04-20T11:38:26.534 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.534+0000 7f8127e00640 1 -- 172.21.15.152:0/2177138146 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8118031960 con 0x7f812815e870 2024-04-20T11:38:26.535 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.534+0000 7f812e400640 1 -- 172.21.15.152:0/2177138146 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f812815e870 msgr2=0x7f81280d3370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:26.535 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.534+0000 7f812e400640 1 --2- 172.21.15.152:0/2177138146 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f812815e870 0x7f81280d3370 secure :-1 s=READY pgs=2470 cs=0 l=1 rev1=1 crypto rx=0x7f812808cae0 tx=0x7f811802b840 comp rx=0 tx=0).stop 2024-04-20T11:38:26.535 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.534+0000 7f812e400640 1 -- 172.21.15.152:0/2177138146 shutdown_connections 2024-04-20T11:38:26.535 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.534+0000 7f812e400640 1 --2- 172.21.15.152:0/2177138146 >> v2:172.21.15.175:3300/0 conn(0x7f81280d06a0 0x7f81280d1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:26.535 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.534+0000 7f812e400640 1 --2- 172.21.15.152:0/2177138146 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f812815e870 0x7f81280d3370 unknown :-1 s=CLOSED pgs=2470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:26.535 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.534+0000 7f812e400640 1 -- 172.21.15.152:0/2177138146 >> 172.21.15.152:0/2177138146 conn(0x7f812808bfd0 msgr2=0x7f81280d45d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:26.535 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.535+0000 7f812e400640 1 -- 172.21.15.152:0/2177138146 shutdown_connections 2024-04-20T11:38:26.535 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.535+0000 7f812e400640 1 -- 172.21.15.152:0/2177138146 wait complete. 2024-04-20T11:38:26.536 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.536+0000 7f812e400640 1 Processor -- start 2024-04-20T11:38:26.536 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.536+0000 7f812e400640 1 -- start start 2024-04-20T11:38:26.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.537+0000 7f812e400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f81280d06a0 0x7f81280cc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:26.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.537+0000 7f812e400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f81280d0fa0 0x7f8128150900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:26.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.537+0000 7f812e400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f812815e870 0x7f8128160d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:26.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.537+0000 7f812e400640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f81280d2b70 con 0x7f81280d0fa0 2024-04-20T11:38:26.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.537+0000 7f812e400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f81280d2890 con 0x7f812815e870 2024-04-20T11:38:26.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.537+0000 7f812e400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f812806ca80 con 0x7f81280d06a0 2024-04-20T11:38:26.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.538+0000 7f811fe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f81280d0fa0 0x7f8128150900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:26.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.538+0000 7f811fe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f81280d0fa0 0x7f8128150900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52202/0 (socket says 172.21.15.152:52202) 2024-04-20T11:38:26.539 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.538+0000 7f811fe00640 1 -- 172.21.15.152:0/1488104765 learned_addr learned my addr 172.21.15.152:0/1488104765 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:38:26.539 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.538+0000 7f812d000640 1 --2- 172.21.15.152:0/1488104765 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f81280d06a0 0x7f81280cc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:26.539 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.538+0000 7f811fe00640 1 -- 172.21.15.152:0/1488104765 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f81280d06a0 msgr2=0x7f81280cc0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:26.539 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.538+0000 7f811fe00640 1 --2- 172.21.15.152:0/1488104765 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f81280d06a0 0x7f81280cc0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:26.539 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.538+0000 7f811fe00640 1 -- 172.21.15.152:0/1488104765 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f812815e870 msgr2=0x7f8128160d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:26.539 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.538+0000 7f811fe00640 1 --2- 172.21.15.152:0/1488104765 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f812815e870 0x7f8128160d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:26.539 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.538+0000 7f811fe00640 1 -- 172.21.15.152:0/1488104765 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81281617e0 con 0x7f81280d0fa0 2024-04-20T11:38:26.539 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.538+0000 7f811fe00640 1 --2- 172.21.15.152:0/1488104765 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f81280d0fa0 0x7f8128150900 secure :-1 s=READY pgs=2471 cs=0 l=1 rev1=1 crypto rx=0x7f811000c4d0 tx=0x7f811000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:26.540 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.539+0000 7f8125600640 1 -- 172.21.15.152:0/1488104765 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8110007480 con 0x7f81280d0fa0 2024-04-20T11:38:26.540 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.539+0000 7f8125600640 1 -- 172.21.15.152:0/1488104765 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81100057a0 con 0x7f81280d0fa0 2024-04-20T11:38:26.540 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.539+0000 7f812e400640 1 -- 172.21.15.152:0/1488104765 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8128161ac0 con 0x7f81280d0fa0 2024-04-20T11:38:26.540 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.539+0000 7f8125600640 1 -- 172.21.15.152:0/1488104765 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8110007620 con 0x7f81280d0fa0 2024-04-20T11:38:26.541 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.539+0000 7f812e400640 1 -- 172.21.15.152:0/1488104765 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f812815aed0 con 0x7f81280d0fa0 2024-04-20T11:38:26.541 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.540+0000 7f812e400640 1 -- 172.21.15.152:0/1488104765 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f81280cbc90 con 0x7f81280d0fa0 2024-04-20T11:38:26.541 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.541+0000 7f8125600640 1 -- 172.21.15.152:0/1488104765 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 27) ==== 92060+0+0 (secure 0 0 0) 0x7f8110005b50 con 0x7f81280d0fa0 2024-04-20T11:38:26.542 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.542+0000 7f8125600640 1 --2- 172.21.15.152:0/1488104765 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f810406d430 0x7f810406f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:26.542 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.542+0000 7f8125600640 1 -- 172.21.15.152:0/1488104765 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 4943+0+0 (secure 0 0 0) 0x7f811008a410 con 0x7f81280d0fa0 2024-04-20T11:38:26.542 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.542+0000 7f8125600640 1 --2- 172.21.15.152:0/1488104765 >> [v2:172.21.15.175:6820/2659150371,v1:172.21.15.175:6822/2659150371] conn(0x7f81040732f0 0x7f8104075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:26.542 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.542+0000 7f8125600640 1 -- 172.21.15.152:0/1488104765 --> [v2:172.21.15.175:6820/2659150371,v1:172.21.15.175:6822/2659150371] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8104075dc0 con 0x7f81040732f0 2024-04-20T11:38:26.542 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.542+0000 7f8125600640 1 -- 172.21.15.152:0/1488104765 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_get_version_reply(handle=1 version=170) ==== 24+0+0 (secure 0 0 0) 0x7f81100912a0 con 0x7f81280d0fa0 2024-04-20T11:38:26.543 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.543+0000 7f812d000640 1 --2- 172.21.15.152:0/1488104765 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f810406d430 0x7f810406f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:26.543 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.543+0000 7f812da00640 1 --2- 172.21.15.152:0/1488104765 >> [v2:172.21.15.175:6820/2659150371,v1:172.21.15.175:6822/2659150371] conn(0x7f81040732f0 0x7f8104075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:26.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.544+0000 7f812da00640 1 --2- 172.21.15.152:0/1488104765 >> [v2:172.21.15.175:6820/2659150371,v1:172.21.15.175:6822/2659150371] conn(0x7f81040732f0 0x7f8104075710 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:26.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.545+0000 7f812d000640 1 --2- 172.21.15.152:0/1488104765 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f810406d430 0x7f810406f8f0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f8118036000 tx=0x7f8118008030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:26.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.547+0000 7f8125600640 1 -- 172.21.15.152:0/1488104765 <== osd.6 v2:172.21.15.175:6820/2659150371 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8104075dc0 con 0x7f81040732f0 2024-04-20T11:38:26.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.561+0000 7f812e400640 1 -- 172.21.15.152:0/1488104765 --> [v2:172.21.15.175:6820/2659150371,v1:172.21.15.175:6822/2659150371] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f81280cb6a0 con 0x7f81040732f0 2024-04-20T11:38:26.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.562+0000 7f8125600640 1 -- 172.21.15.152:0/1488104765 <== osd.6 v2:172.21.15.175:6820/2659150371 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f81280cb6a0 con 0x7f81040732f0 2024-04-20T11:38:26.562 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-04-20T11:38:26.562 INFO:teuthology.orchestra.run.smithi152.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-20T11:38:26.562 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-04-20T11:38:26.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.563+0000 7f812e400640 1 -- 172.21.15.152:0/1488104765 >> [v2:172.21.15.175:6820/2659150371,v1:172.21.15.175:6822/2659150371] conn(0x7f81040732f0 msgr2=0x7f8104075710 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:26.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.563+0000 7f812e400640 1 --2- 172.21.15.152:0/1488104765 >> [v2:172.21.15.175:6820/2659150371,v1:172.21.15.175:6822/2659150371] conn(0x7f81040732f0 0x7f8104075710 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:26.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.563+0000 7f812e400640 1 -- 172.21.15.152:0/1488104765 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f810406d430 msgr2=0x7f810406f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:26.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.563+0000 7f812e400640 1 --2- 172.21.15.152:0/1488104765 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f810406d430 0x7f810406f8f0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f8118036000 tx=0x7f8118008030 comp rx=0 tx=0).stop 2024-04-20T11:38:26.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.563+0000 7f812e400640 1 -- 172.21.15.152:0/1488104765 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f81280d0fa0 msgr2=0x7f8128150900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:26.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.563+0000 7f812e400640 1 --2- 172.21.15.152:0/1488104765 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f81280d0fa0 0x7f8128150900 secure :-1 s=READY pgs=2471 cs=0 l=1 rev1=1 crypto rx=0x7f811000c4d0 tx=0x7f811000c990 comp rx=0 tx=0).stop 2024-04-20T11:38:26.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.563+0000 7f812da00640 1 -- 172.21.15.152:0/1488104765 reap_dead start 2024-04-20T11:38:26.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.563+0000 7f812e400640 1 -- 172.21.15.152:0/1488104765 shutdown_connections 2024-04-20T11:38:26.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.563+0000 7f812e400640 1 -- 172.21.15.152:0/1488104765 >> 172.21.15.152:0/1488104765 conn(0x7f812808bfd0 msgr2=0x7f81280b58e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:26.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.563+0000 7f812e400640 1 -- 172.21.15.152:0/1488104765 shutdown_connections 2024-04-20T11:38:26.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.563+0000 7f812e400640 1 -- 172.21.15.152:0/1488104765 wait complete. 2024-04-20T11:38:26.579 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-04-20T11:38:26.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.780+0000 7f2427a00640 1 Processor -- start 2024-04-20T11:38:26.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.780+0000 7f2427a00640 1 -- start start 2024-04-20T11:38:26.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.782+0000 7f2427a00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f24200cd7e0 0x7f242006e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:26.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.782+0000 7f2427a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f242006e900 0x7f24200cfe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:26.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.782+0000 7f2427a00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f242006f110 con 0x7f2420145f00 2024-04-20T11:38:26.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.782+0000 7f2427a00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f24200cf9f0 con 0x7f24200cd7e0 2024-04-20T11:38:26.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.782+0000 7f2427a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f242006f3f0 con 0x7f242006e900 2024-04-20T11:38:26.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.782+0000 7f2425000640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f2420145f00 0x7f24200cd0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:48294/0 (socket says 172.21.15.152:48294) 2024-04-20T11:38:26.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.782+0000 7f2425000640 1 -- 172.21.15.152:0/3840686273 learned_addr learned my addr 172.21.15.152:0/3840686273 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:38:26.784 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.783+0000 7f2425a00640 1 --2- 172.21.15.152:0/3840686273 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f242006e900 0x7f24200cfe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:26.784 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.783+0000 7f2417e00640 1 --2- 172.21.15.152:0/3840686273 >> v2:172.21.15.175:3300/0 conn(0x7f24200cd7e0 0x7f242006e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:26.785 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.783+0000 7f241fe00640 1 -- 172.21.15.152:0/3840686273 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 214177108 0 0) 0x7f242006f110 con 0x7f2420145f00 2024-04-20T11:38:26.785 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.783+0000 7f241fe00640 1 -- 172.21.15.152:0/3840686273 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2404003520 con 0x7f2420145f00 2024-04-20T11:38:26.785 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.783+0000 7f241fe00640 1 -- 172.21.15.152:0/3840686273 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3091207461 0 0) 0x7f2404003520 con 0x7f2420145f00 2024-04-20T11:38:26.785 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.783+0000 7f241fe00640 1 -- 172.21.15.152:0/3840686273 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f242006e900 msgr2=0x7f24200cfe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:26.785 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.783+0000 7f241fe00640 1 --2- 172.21.15.152:0/3840686273 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f242006e900 0x7f24200cfe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:26.785 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.783+0000 7f241fe00640 1 -- 172.21.15.152:0/3840686273 >> v2:172.21.15.175:3300/0 conn(0x7f24200cd7e0 msgr2=0x7f242006e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:26.785 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.783+0000 7f241fe00640 1 --2- 172.21.15.152:0/3840686273 >> v2:172.21.15.175:3300/0 conn(0x7f24200cd7e0 0x7f242006e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:26.785 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.783+0000 7f241fe00640 1 -- 172.21.15.152:0/3840686273 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24200cf3a0 con 0x7f2420145f00 2024-04-20T11:38:26.785 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.783+0000 7f241fe00640 1 -- 172.21.15.152:0/3840686273 <== mon.0 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2462599104 0 0) 0x7f2410001670 con 0x7f2420145f00 2024-04-20T11:38:26.785 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.783+0000 7f2417e00640 1 --2- 172.21.15.152:0/3840686273 >> v2:172.21.15.175:3300/0 conn(0x7f24200cd7e0 0x7f242006e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:38:26.785 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.783+0000 7f241fe00640 1 -- 172.21.15.152:0/3840686273 >> v1:172.21.15.152:6789/0 conn(0x7f2420145f00 legacy=0x7f24200cd0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:26.785 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.783+0000 7f241fe00640 1 --2- 172.21.15.152:0/3840686273 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f24040039b0 0x7f2404003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:26.785 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.784+0000 7f241fe00640 1 --2- 172.21.15.152:0/3840686273 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2404004800 0x7f2404004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:26.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.784+0000 7f241fe00640 1 --2- 172.21.15.152:0/3840686273 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f242006e900 0x7f24040051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:26.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.784+0000 7f241fe00640 1 -- 172.21.15.152:0/3840686273 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f24200cf9f0 con 0x7f2404004800 2024-04-20T11:38:26.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.784+0000 7f241fe00640 1 -- 172.21.15.152:0/3840686273 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f242006f3f0 con 0x7f24040039b0 2024-04-20T11:38:26.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.784+0000 7f241fe00640 1 -- 172.21.15.152:0/3840686273 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f240400b9c0 con 0x7f242006e900 2024-04-20T11:38:26.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.784+0000 7f2417e00640 1 --2- 172.21.15.152:0/3840686273 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2404004800 0x7f2404004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:26.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.784+0000 7f2425000640 1 --2- 172.21.15.152:0/3840686273 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f24040039b0 0x7f2404003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:26.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.784+0000 7f2417e00640 1 -- 172.21.15.152:0/3840686273 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f242006e900 msgr2=0x7f24040051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:26.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.784+0000 7f2417e00640 1 --2- 172.21.15.152:0/3840686273 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f242006e900 0x7f24040051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:26.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.784+0000 7f2417e00640 1 -- 172.21.15.152:0/3840686273 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f24040039b0 msgr2=0x7f2404003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:26.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.784+0000 7f2417e00640 1 --2- 172.21.15.152:0/3840686273 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f24040039b0 0x7f2404003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:26.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.784+0000 7f2417e00640 1 -- 172.21.15.152:0/3840686273 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2404003520 con 0x7f2404004800 2024-04-20T11:38:26.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.785+0000 7f2417e00640 1 --2- 172.21.15.152:0/3840686273 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2404004800 0x7f2404004c80 secure :-1 s=READY pgs=2473 cs=0 l=1 rev1=1 crypto rx=0x7f24080098a0 tx=0x7f2404005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c9cbaf6d8fccc2d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:26.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.786+0000 7f241fe00640 1 -- 172.21.15.152:0/3840686273 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f240800f040 con 0x7f2404004800 2024-04-20T11:38:26.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.786+0000 7f241fe00640 1 -- 172.21.15.152:0/3840686273 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2408004230 con 0x7f2404004800 2024-04-20T11:38:26.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.786+0000 7f241fe00640 1 -- 172.21.15.152:0/3840686273 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2408003610 con 0x7f2404004800 2024-04-20T11:38:26.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.786+0000 7f2427a00640 1 -- 172.21.15.152:0/3840686273 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2404004800 msgr2=0x7f2404004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:26.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.786+0000 7f2427a00640 1 --2- 172.21.15.152:0/3840686273 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2404004800 0x7f2404004c80 secure :-1 s=READY pgs=2473 cs=0 l=1 rev1=1 crypto rx=0x7f24080098a0 tx=0x7f2404005830 comp rx=0 tx=0).stop 2024-04-20T11:38:26.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.786+0000 7f2425a00640 1 -- 172.21.15.152:0/3840686273 reap_dead start 2024-04-20T11:38:26.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.786+0000 7f2427a00640 1 -- 172.21.15.152:0/3840686273 shutdown_connections 2024-04-20T11:38:26.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.786+0000 7f2427a00640 1 -- 172.21.15.152:0/3840686273 >> 172.21.15.152:0/3840686273 conn(0x7f2420067cf0 msgr2=0x7f242006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:26.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.786+0000 7f2427a00640 1 -- 172.21.15.152:0/3840686273 shutdown_connections 2024-04-20T11:38:26.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.786+0000 7f2427a00640 1 -- 172.21.15.152:0/3840686273 wait complete. 2024-04-20T11:38:26.789 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.788+0000 7f2427a00640 1 Processor -- start 2024-04-20T11:38:26.789 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.788+0000 7f2427a00640 1 -- start start 2024-04-20T11:38:26.789 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.789+0000 7f2427a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2404004800 0x7f2420139f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:26.789 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.789+0000 7f2427a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f242013c330 0x7f242013c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:26.789 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.789+0000 7f2427a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2420144870 0x7f2420144cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:26.789 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.789+0000 7f2427a00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f242006ed90 con 0x7f2404004800 2024-04-20T11:38:26.790 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.789+0000 7f2427a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f24201462e0 con 0x7f242013c330 2024-04-20T11:38:26.790 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.789+0000 7f2427a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f24200cfe80 con 0x7f2420144870 2024-04-20T11:38:26.790 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.790+0000 7f2425000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2404004800 0x7f2420139f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:26.790 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.790+0000 7f2425000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2404004800 0x7f2420139f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52216/0 (socket says 172.21.15.152:52216) 2024-04-20T11:38:26.790 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.790+0000 7f2425000640 1 -- 172.21.15.152:0/4099062182 learned_addr learned my addr 172.21.15.152:0/4099062182 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:38:26.791 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.790+0000 7f2425000640 1 -- 172.21.15.152:0/4099062182 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2420144870 msgr2=0x7f2420144cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:26.791 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.790+0000 7f2425000640 1 --2- 172.21.15.152:0/4099062182 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2420144870 0x7f2420144cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:26.791 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.790+0000 7f2425000640 1 -- 172.21.15.152:0/4099062182 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f242013c330 msgr2=0x7f242013c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:26.791 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.790+0000 7f2425000640 1 --2- 172.21.15.152:0/4099062182 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f242013c330 0x7f242013c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:26.791 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.790+0000 7f2425000640 1 -- 172.21.15.152:0/4099062182 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f242013cea0 con 0x7f2404004800 2024-04-20T11:38:26.791 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.790+0000 7f2425000640 1 --2- 172.21.15.152:0/4099062182 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2404004800 0x7f2420139f10 secure :-1 s=READY pgs=2474 cs=0 l=1 rev1=1 crypto rx=0x7f241000d850 tx=0x7f241000dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:26.791 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.791+0000 7f241d600640 1 -- 172.21.15.152:0/4099062182 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2410019070 con 0x7f2404004800 2024-04-20T11:38:26.791 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.791+0000 7f241d600640 1 -- 172.21.15.152:0/4099062182 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24100074e0 con 0x7f2404004800 2024-04-20T11:38:26.791 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.791+0000 7f2427a00640 1 -- 172.21.15.152:0/4099062182 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f242013d180 con 0x7f2404004800 2024-04-20T11:38:26.791 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.791+0000 7f241d600640 1 -- 172.21.15.152:0/4099062182 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24100049e0 con 0x7f2404004800 2024-04-20T11:38:26.792 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.791+0000 7f2427a00640 1 -- 172.21.15.152:0/4099062182 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24201425b0 con 0x7f2404004800 2024-04-20T11:38:26.792 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.792+0000 7f2427a00640 1 -- 172.21.15.152:0/4099062182 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24200689f0 con 0x7f2404004800 2024-04-20T11:38:26.792 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.793+0000 7f241d600640 1 -- 172.21.15.152:0/4099062182 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 27) ==== 92060+0+0 (secure 0 0 0) 0x7f241001c070 con 0x7f2404004800 2024-04-20T11:38:26.793 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.794+0000 7f241d600640 1 --2- 172.21.15.152:0/4099062182 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f23fc06d460 0x7f23fc06f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:26.793 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.794+0000 7f241d600640 1 -- 172.21.15.152:0/4099062182 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 4943+0+0 (secure 0 0 0) 0x7f241008a0f0 con 0x7f2404004800 2024-04-20T11:38:26.794 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.794+0000 7f2417e00640 1 --2- 172.21.15.152:0/4099062182 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f23fc06d460 0x7f23fc06f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:26.795 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.796+0000 7f2417e00640 1 --2- 172.21.15.152:0/4099062182 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f23fc06d460 0x7f23fc06f920 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f2408035000 tx=0x7f2408030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:26.797 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:26.798+0000 7f241d600640 1 -- 172.21.15.152:0/4099062182 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f2410059d20 con 0x7f2404004800 2024-04-20T11:38:26.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.000+0000 7f2427a00640 1 -- 172.21.15.152:0/4099062182 --> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7f24201413d0 con 0x7f23fc06d460 2024-04-20T11:38:27.000 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.001+0000 7f241d600640 1 -- 172.21.15.152:0/4099062182 <== mgr.4109 v2:172.21.15.175:6836/4021250441 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f24201413d0 con 0x7f23fc06d460 2024-04-20T11:38:27.001 INFO:teuthology.orchestra.run.smithi152.stderr:instructed osd(s) 6 to deep-scrub 2024-04-20T11:38:27.004 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.004+0000 7f2427a00640 1 -- 172.21.15.152:0/4099062182 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f23fc06d460 msgr2=0x7f23fc06f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:27.004 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.004+0000 7f2427a00640 1 --2- 172.21.15.152:0/4099062182 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f23fc06d460 0x7f23fc06f920 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f2408035000 tx=0x7f2408030000 comp rx=0 tx=0).stop 2024-04-20T11:38:27.004 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.004+0000 7f2427a00640 1 -- 172.21.15.152:0/4099062182 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2404004800 msgr2=0x7f2420139f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:27.004 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.004+0000 7f2427a00640 1 --2- 172.21.15.152:0/4099062182 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2404004800 0x7f2420139f10 secure :-1 s=READY pgs=2474 cs=0 l=1 rev1=1 crypto rx=0x7f241000d850 tx=0x7f241000dd10 comp rx=0 tx=0).stop 2024-04-20T11:38:27.004 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.004+0000 7f2427a00640 1 -- 172.21.15.152:0/4099062182 shutdown_connections 2024-04-20T11:38:27.004 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.004+0000 7f2427a00640 1 --2- 172.21.15.152:0/4099062182 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f23fc06d460 0x7f23fc06f920 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:27.004 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.004+0000 7f2427a00640 1 --2- 172.21.15.152:0/4099062182 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2420144870 0x7f2420144cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:27.004 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.004+0000 7f2427a00640 1 --2- 172.21.15.152:0/4099062182 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f242013c330 0x7f242013c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:27.004 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.004+0000 7f2427a00640 1 --2- 172.21.15.152:0/4099062182 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2404004800 0x7f2420139f10 unknown :-1 s=CLOSED pgs=2474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:27.004 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.004+0000 7f2427a00640 1 -- 172.21.15.152:0/4099062182 >> 172.21.15.152:0/4099062182 conn(0x7f2420067cf0 msgr2=0x7f2420140310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:27.004 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.004+0000 7f2427a00640 1 -- 172.21.15.152:0/4099062182 shutdown_connections 2024-04-20T11:38:27.005 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.004+0000 7f2427a00640 1 -- 172.21.15.152:0/4099062182 wait complete. 2024-04-20T11:38:27.022 INFO:tasks.ceph:Scrubbing osd.7 2024-04-20T11:38:27.022 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-04-20T11:38:27.227 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.227+0000 7f0cf9400640 1 Processor -- start 2024-04-20T11:38:27.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.228+0000 7f0cf9400640 1 -- start start 2024-04-20T11:38:27.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.229+0000 7f0cf9400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f0cf40d22d0 0x7f0cf40cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:27.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.229+0000 7f0cf9400640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f0cf40cfa10 0x7f0cf40b78c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:27.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.229+0000 7f0cf9400640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0cf40d40e0 con 0x7f0cf415e910 2024-04-20T11:38:27.230 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.229+0000 7f0cf9400640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f0cf40d43c0 con 0x7f0cf40d22d0 2024-04-20T11:38:27.230 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.229+0000 7f0cf9400640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f0cf40d46a0 con 0x7f0cf40cfa10 2024-04-20T11:38:27.230 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.229+0000 7f0cf2800640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f0cf415e910 0x7f0cf40d3870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:48310/0 (socket says 172.21.15.152:48310) 2024-04-20T11:38:27.230 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.229+0000 7f0cf2800640 1 -- 172.21.15.152:0/2372262906 learned_addr learned my addr 172.21.15.152:0/2372262906 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:38:27.230 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.230+0000 7f0ce9e00640 1 --2- 172.21.15.152:0/2372262906 >> v2:172.21.15.152:3300/0 conn(0x7f0cf40d22d0 0x7f0cf40cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:27.230 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.230+0000 7f0cf1e00640 1 -- 172.21.15.152:0/2372262906 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2601952292 0 0) 0x7f0cf40d40e0 con 0x7f0cf415e910 2024-04-20T11:38:27.230 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.230+0000 7f0cf3200640 1 --2- 172.21.15.152:0/2372262906 >> v2:172.21.15.175:3300/0 conn(0x7f0cf40cfa10 0x7f0cf40b78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:27.230 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.230+0000 7f0cf1e00640 1 -- 172.21.15.152:0/2372262906 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0cd8003520 con 0x7f0cf415e910 2024-04-20T11:38:27.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.230+0000 7f0ce9e00640 1 -- 172.21.15.152:0/2372262906 >> v2:172.21.15.175:3300/0 conn(0x7f0cf40cfa10 msgr2=0x7f0cf40b78c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:27.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.230+0000 7f0ce9e00640 1 --2- 172.21.15.152:0/2372262906 >> v2:172.21.15.175:3300/0 conn(0x7f0cf40cfa10 0x7f0cf40b78c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:27.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.230+0000 7f0ce9e00640 1 -- 172.21.15.152:0/2372262906 >> v1:172.21.15.152:6789/0 conn(0x7f0cf415e910 legacy=0x7f0cf40d3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:27.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.230+0000 7f0cf1e00640 1 -- 172.21.15.152:0/2372262906 <== mon.0 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3573183908 0 0) 0x7f0cd8003520 con 0x7f0cf415e910 2024-04-20T11:38:27.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.230+0000 7f0ce9e00640 1 -- 172.21.15.152:0/2372262906 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cf40d4980 con 0x7f0cf40d22d0 2024-04-20T11:38:27.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.231+0000 7f0ce9e00640 1 --2- 172.21.15.152:0/2372262906 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0cf40d22d0 0x7f0cf40cf4d0 secure :-1 s=READY pgs=2476 cs=0 l=1 rev1=1 crypto rx=0x7f0cf40d2740 tx=0x7f0cdc02b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d8b9143f5b036cd server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:27.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.232+0000 7f0cf1e00640 1 -- 172.21.15.152:0/2372262906 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cdc038070 con 0x7f0cf40d22d0 2024-04-20T11:38:27.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.232+0000 7f0cf1e00640 1 -- 172.21.15.152:0/2372262906 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cdc033440 con 0x7f0cf40d22d0 2024-04-20T11:38:27.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.232+0000 7f0cf1e00640 1 -- 172.21.15.152:0/2372262906 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce40019f0 con 0x7f0cf40d22d0 2024-04-20T11:38:27.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.232+0000 7f0cf9400640 1 -- 172.21.15.152:0/2372262906 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0cf40d22d0 msgr2=0x7f0cf40cf4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:27.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.232+0000 7f0cf9400640 1 --2- 172.21.15.152:0/2372262906 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0cf40d22d0 0x7f0cf40cf4d0 secure :-1 s=READY pgs=2476 cs=0 l=1 rev1=1 crypto rx=0x7f0cf40d2740 tx=0x7f0cdc02b7a0 comp rx=0 tx=0).stop 2024-04-20T11:38:27.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.232+0000 7f0cf9400640 1 -- 172.21.15.152:0/2372262906 shutdown_connections 2024-04-20T11:38:27.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.232+0000 7f0cf9400640 1 --2- 172.21.15.152:0/2372262906 >> v2:172.21.15.175:3300/0 conn(0x7f0cf40cfa10 0x7f0cf40b78c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:27.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.232+0000 7f0cf9400640 1 --2- 172.21.15.152:0/2372262906 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0cf40d22d0 0x7f0cf40cf4d0 unknown :-1 s=CLOSED pgs=2476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:27.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.232+0000 7f0cf9400640 1 -- 172.21.15.152:0/2372262906 >> 172.21.15.152:0/2372262906 conn(0x7f0cf4067ae0 msgr2=0x7f0cf4067ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:27.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.232+0000 7f0cf9400640 1 -- 172.21.15.152:0/2372262906 shutdown_connections 2024-04-20T11:38:27.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.233+0000 7f0cf9400640 1 -- 172.21.15.152:0/2372262906 wait complete. 2024-04-20T11:38:27.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.234+0000 7f0cf9400640 1 Processor -- start 2024-04-20T11:38:27.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.234+0000 7f0cf9400640 1 -- start start 2024-04-20T11:38:27.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.235+0000 7f0cf9400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0cf40cfa10 0x7f0cf4152130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:27.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.235+0000 7f0cf9400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0cf40d22d0 0x7f0cf40cd080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:27.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.235+0000 7f0cf9400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0cf415e910 0x7f0cf40cd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:27.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.235+0000 7f0cf9400640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf415ecf0 con 0x7f0cf40cfa10 2024-04-20T11:38:27.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.235+0000 7f0cf9400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf40cfe20 con 0x7f0cf415e910 2024-04-20T11:38:27.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.235+0000 7f0cf9400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0cf40b7e00 con 0x7f0cf40d22d0 2024-04-20T11:38:27.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.236+0000 7f0cf2800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0cf40cfa10 0x7f0cf4152130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:27.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.236+0000 7f0cf2800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0cf40cfa10 0x7f0cf4152130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52232/0 (socket says 172.21.15.152:52232) 2024-04-20T11:38:27.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.236+0000 7f0cf2800640 1 -- 172.21.15.152:0/1078434975 learned_addr learned my addr 172.21.15.152:0/1078434975 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:38:27.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.236+0000 7f0ce9e00640 1 --2- 172.21.15.152:0/1078434975 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0cf40d22d0 0x7f0cf40cd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:27.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.236+0000 7f0cf2800640 1 -- 172.21.15.152:0/1078434975 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0cf40d22d0 msgr2=0x7f0cf40cd080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:27.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.236+0000 7f0cf2800640 1 --2- 172.21.15.152:0/1078434975 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0cf40d22d0 0x7f0cf40cd080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:27.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.236+0000 7f0cf2800640 1 -- 172.21.15.152:0/1078434975 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0cf415e910 msgr2=0x7f0cf40cd5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:27.237 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.236+0000 7f0cf2800640 1 --2- 172.21.15.152:0/1078434975 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0cf415e910 0x7f0cf40cd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:27.237 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.236+0000 7f0cf2800640 1 -- 172.21.15.152:0/1078434975 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cf40ca250 con 0x7f0cf40cfa10 2024-04-20T11:38:27.237 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.236+0000 7f0cf2800640 1 --2- 172.21.15.152:0/1078434975 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0cf40cfa10 0x7f0cf4152130 secure :-1 s=READY pgs=2477 cs=0 l=1 rev1=1 crypto rx=0x7f0ce40018a0 tx=0x7f0ce400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:27.237 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.237+0000 7f0ceb400640 1 -- 172.21.15.152:0/1078434975 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce4012400 con 0x7f0cf40cfa10 2024-04-20T11:38:27.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.237+0000 7f0ceb400640 1 -- 172.21.15.152:0/1078434975 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ce40125c0 con 0x7f0cf40cfa10 2024-04-20T11:38:27.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.237+0000 7f0ceb400640 1 -- 172.21.15.152:0/1078434975 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce40129b0 con 0x7f0cf40cfa10 2024-04-20T11:38:27.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.237+0000 7f0cf9400640 1 -- 172.21.15.152:0/1078434975 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cf40ca530 con 0x7f0cf40cfa10 2024-04-20T11:38:27.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.237+0000 7f0cf9400640 1 -- 172.21.15.152:0/1078434975 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cf40caaf0 con 0x7f0cf40cfa10 2024-04-20T11:38:27.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.238+0000 7f0cf9400640 1 -- 172.21.15.152:0/1078434975 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0cf4152670 con 0x7f0cf40cfa10 2024-04-20T11:38:27.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.239+0000 7f0ceb400640 1 -- 172.21.15.152:0/1078434975 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 27) ==== 92060+0+0 (secure 0 0 0) 0x7f0ce4012b50 con 0x7f0cf40cfa10 2024-04-20T11:38:27.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.240+0000 7f0ceb400640 1 --2- 172.21.15.152:0/1078434975 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f0cd006d430 0x7f0cd006f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:27.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.240+0000 7f0ceb400640 1 -- 172.21.15.152:0/1078434975 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 4943+0+0 (secure 0 0 0) 0x7f0ce408afd0 con 0x7f0cf40cfa10 2024-04-20T11:38:27.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.240+0000 7f0ceb400640 1 --2- 172.21.15.152:0/1078434975 >> [v2:172.21.15.175:6816/3116669518,v1:172.21.15.175:6817/3116669518] conn(0x7f0cd00732f0 0x7f0cd0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:27.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.240+0000 7f0ceb400640 1 -- 172.21.15.152:0/1078434975 --> [v2:172.21.15.175:6816/3116669518,v1:172.21.15.175:6817/3116669518] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0cd0075dc0 con 0x7f0cd00732f0 2024-04-20T11:38:27.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.240+0000 7f0ceb400640 1 -- 172.21.15.152:0/1078434975 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_get_version_reply(handle=1 version=170) ==== 24+0+0 (secure 0 0 0) 0x7f0ce4094050 con 0x7f0cf40cfa10 2024-04-20T11:38:27.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.241+0000 7f0ce9e00640 1 --2- 172.21.15.152:0/1078434975 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f0cd006d430 0x7f0cd006f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:27.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.241+0000 7f0cf3200640 1 --2- 172.21.15.152:0/1078434975 >> [v2:172.21.15.175:6816/3116669518,v1:172.21.15.175:6817/3116669518] conn(0x7f0cd00732f0 0x7f0cd0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:27.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.242+0000 7f0ce9e00640 1 --2- 172.21.15.152:0/1078434975 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f0cd006d430 0x7f0cd006f8f0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc035000 tx=0x7f0cdc02f000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:27.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.242+0000 7f0cf3200640 1 --2- 172.21.15.152:0/1078434975 >> [v2:172.21.15.175:6816/3116669518,v1:172.21.15.175:6817/3116669518] conn(0x7f0cd00732f0 0x7f0cd0075710 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:27.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.244+0000 7f0ceb400640 1 -- 172.21.15.152:0/1078434975 <== osd.7 v2:172.21.15.175:6816/3116669518 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0cd0075dc0 con 0x7f0cd00732f0 2024-04-20T11:38:27.258 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.259+0000 7f0cf9400640 1 -- 172.21.15.152:0/1078434975 --> [v2:172.21.15.175:6816/3116669518,v1:172.21.15.175:6817/3116669518] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f0cf4068a60 con 0x7f0cd00732f0 2024-04-20T11:38:27.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.260+0000 7f0ceb400640 1 -- 172.21.15.152:0/1078434975 <== osd.7 v2:172.21.15.175:6816/3116669518 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f0cf4068a60 con 0x7f0cd00732f0 2024-04-20T11:38:27.259 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-04-20T11:38:27.260 INFO:teuthology.orchestra.run.smithi152.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-20T11:38:27.260 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-04-20T11:38:27.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.260+0000 7f0cf9400640 1 -- 172.21.15.152:0/1078434975 >> [v2:172.21.15.175:6816/3116669518,v1:172.21.15.175:6817/3116669518] conn(0x7f0cd00732f0 msgr2=0x7f0cd0075710 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:27.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.260+0000 7f0cf9400640 1 --2- 172.21.15.152:0/1078434975 >> [v2:172.21.15.175:6816/3116669518,v1:172.21.15.175:6817/3116669518] conn(0x7f0cd00732f0 0x7f0cd0075710 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:27.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.260+0000 7f0cf9400640 1 -- 172.21.15.152:0/1078434975 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f0cd006d430 msgr2=0x7f0cd006f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:27.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.260+0000 7f0cf9400640 1 --2- 172.21.15.152:0/1078434975 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f0cd006d430 0x7f0cd006f8f0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc035000 tx=0x7f0cdc02f000 comp rx=0 tx=0).stop 2024-04-20T11:38:27.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.260+0000 7f0cf9400640 1 -- 172.21.15.152:0/1078434975 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0cf40cfa10 msgr2=0x7f0cf4152130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:27.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.260+0000 7f0cf9400640 1 --2- 172.21.15.152:0/1078434975 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0cf40cfa10 0x7f0cf4152130 secure :-1 s=READY pgs=2477 cs=0 l=1 rev1=1 crypto rx=0x7f0ce40018a0 tx=0x7f0ce400a430 comp rx=0 tx=0).stop 2024-04-20T11:38:27.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.260+0000 7f0cf3200640 1 -- 172.21.15.152:0/1078434975 reap_dead start 2024-04-20T11:38:27.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.261+0000 7f0cf9400640 1 -- 172.21.15.152:0/1078434975 shutdown_connections 2024-04-20T11:38:27.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.261+0000 7f0cf9400640 1 -- 172.21.15.152:0/1078434975 >> 172.21.15.152:0/1078434975 conn(0x7f0cf4067ae0 msgr2=0x7f0cf4067ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:27.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.261+0000 7f0cf9400640 1 -- 172.21.15.152:0/1078434975 shutdown_connections 2024-04-20T11:38:27.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.261+0000 7f0cf9400640 1 -- 172.21.15.152:0/1078434975 wait complete. 2024-04-20T11:38:27.278 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-04-20T11:38:27.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.472+0000 7fed99c00640 1 Processor -- start 2024-04-20T11:38:27.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.473+0000 7fed99c00640 1 -- start start 2024-04-20T11:38:27.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.474+0000 7fed99c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fed940d0e60 0x7fed9406d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:27.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.474+0000 7fed99c00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fed9406df20 0x7fed9415d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:27.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.474+0000 7fed99c00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed940cfef0 con 0x7fed94159320 2024-04-20T11:38:27.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.474+0000 7fed99c00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fed9406e390 con 0x7fed9406df20 2024-04-20T11:38:27.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.474+0000 7fed99c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fed9415c3d0 con 0x7fed940d0e60 2024-04-20T11:38:27.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.474+0000 7fed93e00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fed94159320 0x7fed9415bbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:48316/0 (socket says 172.21.15.152:48316) 2024-04-20T11:38:27.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.474+0000 7fed93e00640 1 -- 172.21.15.152:0/4266734953 learned_addr learned my addr 172.21.15.152:0/4266734953 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:38:27.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.474+0000 7fed99200640 1 --2- 172.21.15.152:0/4266734953 >> v2:172.21.15.152:3300/0 conn(0x7fed9406df20 0x7fed9415d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:27.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.475+0000 7fed8be00640 1 --2- 172.21.15.152:0/4266734953 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fed940d0e60 0x7fed9406d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:27.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.475+0000 7fed93400640 1 -- 172.21.15.152:0/4266734953 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4009649382 0 0) 0x7fed940cfef0 con 0x7fed94159320 2024-04-20T11:38:27.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.475+0000 7fed93400640 1 -- 172.21.15.152:0/4266734953 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fed78005730 con 0x7fed94159320 2024-04-20T11:38:27.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.475+0000 7fed99200640 1 -- 172.21.15.152:0/4266734953 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fed940d0e60 msgr2=0x7fed9406d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:27.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.475+0000 7fed99200640 1 --2- 172.21.15.152:0/4266734953 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fed940d0e60 0x7fed9406d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:27.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.475+0000 7fed99200640 1 -- 172.21.15.152:0/4266734953 >> v1:172.21.15.152:6789/0 conn(0x7fed94159320 legacy=0x7fed9415bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:27.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.475+0000 7fed99200640 1 -- 172.21.15.152:0/4266734953 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed9415c6b0 con 0x7fed9406df20 2024-04-20T11:38:27.476 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.476+0000 7fed99200640 1 --2- 172.21.15.152:0/4266734953 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fed9406df20 0x7fed9415d450 secure :-1 s=READY pgs=2479 cs=0 l=1 rev1=1 crypto rx=0x7fed8c009d90 tx=0x7fed8c02be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=753ad2f1fcf49072 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:27.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.476+0000 7fed93400640 1 -- 172.21.15.152:0/4266734953 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed8c039070 con 0x7fed9406df20 2024-04-20T11:38:27.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.476+0000 7fed93400640 1 -- 172.21.15.152:0/4266734953 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed8c02cbd0 con 0x7fed9406df20 2024-04-20T11:38:27.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.476+0000 7fed93400640 1 -- 172.21.15.152:0/4266734953 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed8c034440 con 0x7fed9406df20 2024-04-20T11:38:27.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.477+0000 7fed99c00640 1 -- 172.21.15.152:0/4266734953 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fed9406df20 msgr2=0x7fed9415d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:27.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.477+0000 7fed99c00640 1 --2- 172.21.15.152:0/4266734953 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fed9406df20 0x7fed9415d450 secure :-1 s=READY pgs=2479 cs=0 l=1 rev1=1 crypto rx=0x7fed8c009d90 tx=0x7fed8c02be20 comp rx=0 tx=0).stop 2024-04-20T11:38:27.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.477+0000 7fed99c00640 1 -- 172.21.15.152:0/4266734953 shutdown_connections 2024-04-20T11:38:27.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.477+0000 7fed99c00640 1 --2- 172.21.15.152:0/4266734953 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fed940d0e60 0x7fed9406d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:27.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.477+0000 7fed99c00640 1 --2- 172.21.15.152:0/4266734953 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fed9406df20 0x7fed9415d450 unknown :-1 s=CLOSED pgs=2479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:27.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.477+0000 7fed99c00640 1 -- 172.21.15.152:0/4266734953 >> 172.21.15.152:0/4266734953 conn(0x7fed940d0260 msgr2=0x7fed940d0670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:27.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.477+0000 7fed99c00640 1 -- 172.21.15.152:0/4266734953 shutdown_connections 2024-04-20T11:38:27.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.477+0000 7fed99c00640 1 -- 172.21.15.152:0/4266734953 wait complete. 2024-04-20T11:38:27.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.479+0000 7fed99c00640 1 Processor -- start 2024-04-20T11:38:27.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.479+0000 7fed99c00640 1 -- start start 2024-04-20T11:38:27.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.480+0000 7fed99c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fed9406df20 0x7fed941f7070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:27.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.480+0000 7fed99c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fed940d0e60 0x7fed941f16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:27.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.480+0000 7fed99c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed94159320 0x7fed94156fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:27.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.480+0000 7fed99c00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fed9415d990 con 0x7fed9406df20 2024-04-20T11:38:27.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.480+0000 7fed99c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fed9415cf30 con 0x7fed94159320 2024-04-20T11:38:27.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.480+0000 7fed99c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fed9415db10 con 0x7fed940d0e60 2024-04-20T11:38:27.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.480+0000 7fed93e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fed9406df20 0x7fed941f7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:27.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.480+0000 7fed93e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fed9406df20 0x7fed941f7070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52250/0 (socket says 172.21.15.152:52250) 2024-04-20T11:38:27.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.480+0000 7fed93e00640 1 -- 172.21.15.152:0/3469386520 learned_addr learned my addr 172.21.15.152:0/3469386520 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:38:27.482 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.481+0000 7fed8be00640 1 --2- 172.21.15.152:0/3469386520 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fed940d0e60 0x7fed941f16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:27.482 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.481+0000 7fed93e00640 1 -- 172.21.15.152:0/3469386520 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fed940d0e60 msgr2=0x7fed941f16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:27.482 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.481+0000 7fed99200640 1 --2- 172.21.15.152:0/3469386520 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed94159320 0x7fed94156fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:27.482 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.481+0000 7fed93e00640 1 --2- 172.21.15.152:0/3469386520 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fed940d0e60 0x7fed941f16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:27.482 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.481+0000 7fed93e00640 1 -- 172.21.15.152:0/3469386520 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed94159320 msgr2=0x7fed94156fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:27.482 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.481+0000 7fed93e00640 1 --2- 172.21.15.152:0/3469386520 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed94159320 0x7fed94156fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:27.482 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.481+0000 7fed93e00640 1 -- 172.21.15.152:0/3469386520 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed941f5fd0 con 0x7fed9406df20 2024-04-20T11:38:27.482 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.481+0000 7fed8be00640 1 --2- 172.21.15.152:0/3469386520 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fed940d0e60 0x7fed941f16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:38:27.482 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.481+0000 7fed93e00640 1 --2- 172.21.15.152:0/3469386520 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fed9406df20 0x7fed941f7070 secure :-1 s=READY pgs=2480 cs=0 l=1 rev1=1 crypto rx=0x7fed8400c3c0 tx=0x7fed8400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:27.482 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.481+0000 7fed90c00640 1 -- 172.21.15.152:0/3469386520 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed84017070 con 0x7fed9406df20 2024-04-20T11:38:27.482 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.481+0000 7fed90c00640 1 -- 172.21.15.152:0/3469386520 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed84005ce0 con 0x7fed9406df20 2024-04-20T11:38:27.482 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.481+0000 7fed99c00640 1 -- 172.21.15.152:0/3469386520 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed941f62b0 con 0x7fed9406df20 2024-04-20T11:38:27.482 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.481+0000 7fed90c00640 1 -- 172.21.15.152:0/3469386520 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed840072f0 con 0x7fed9406df20 2024-04-20T11:38:27.482 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.482+0000 7fed99c00640 1 -- 172.21.15.152:0/3469386520 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed941f6870 con 0x7fed9406df20 2024-04-20T11:38:27.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.483+0000 7fed99c00640 1 -- 172.21.15.152:0/3469386520 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed94075e60 con 0x7fed9406df20 2024-04-20T11:38:27.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.484+0000 7fed90c00640 1 -- 172.21.15.152:0/3469386520 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 27) ==== 92060+0+0 (secure 0 0 0) 0x7fed8401a070 con 0x7fed9406df20 2024-04-20T11:38:27.484 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.485+0000 7fed90c00640 1 --2- 172.21.15.152:0/3469386520 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fed7006d460 0x7fed7006f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:27.484 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.485+0000 7fed90c00640 1 -- 172.21.15.152:0/3469386520 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 4943+0+0 (secure 0 0 0) 0x7fed84089b30 con 0x7fed9406df20 2024-04-20T11:38:27.484 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.485+0000 7fed8be00640 1 --2- 172.21.15.152:0/3469386520 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fed7006d460 0x7fed7006f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:27.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.486+0000 7fed8be00640 1 --2- 172.21.15.152:0/3469386520 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fed7006d460 0x7fed7006f920 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fed9415c2d0 tx=0x7fed7c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:27.489 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.489+0000 7fed90c00640 1 -- 172.21.15.152:0/3469386520 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fed84059700 con 0x7fed9406df20 2024-04-20T11:38:27.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.690+0000 7fed99c00640 1 -- 172.21.15.152:0/3469386520 --> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7fed9406ef90 con 0x7fed7006d460 2024-04-20T11:38:27.691 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.691+0000 7fed90c00640 1 -- 172.21.15.152:0/3469386520 <== mgr.4109 v2:172.21.15.175:6836/4021250441 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fed9406ef90 con 0x7fed7006d460 2024-04-20T11:38:27.691 INFO:teuthology.orchestra.run.smithi152.stderr:instructed osd(s) 7 to deep-scrub 2024-04-20T11:38:27.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.694+0000 7fed99c00640 1 -- 172.21.15.152:0/3469386520 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fed7006d460 msgr2=0x7fed7006f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:27.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.694+0000 7fed99c00640 1 --2- 172.21.15.152:0/3469386520 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fed7006d460 0x7fed7006f920 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fed9415c2d0 tx=0x7fed7c008040 comp rx=0 tx=0).stop 2024-04-20T11:38:27.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.694+0000 7fed99c00640 1 -- 172.21.15.152:0/3469386520 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fed9406df20 msgr2=0x7fed941f7070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:27.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.694+0000 7fed99c00640 1 --2- 172.21.15.152:0/3469386520 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fed9406df20 0x7fed941f7070 secure :-1 s=READY pgs=2480 cs=0 l=1 rev1=1 crypto rx=0x7fed8400c3c0 tx=0x7fed8400c880 comp rx=0 tx=0).stop 2024-04-20T11:38:27.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.694+0000 7fed99c00640 1 -- 172.21.15.152:0/3469386520 shutdown_connections 2024-04-20T11:38:27.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.694+0000 7fed99c00640 1 --2- 172.21.15.152:0/3469386520 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fed7006d460 0x7fed7006f920 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:27.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.694+0000 7fed99c00640 1 --2- 172.21.15.152:0/3469386520 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed94159320 0x7fed94156fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:27.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.694+0000 7fed99c00640 1 --2- 172.21.15.152:0/3469386520 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fed940d0e60 0x7fed941f16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:27.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.694+0000 7fed99c00640 1 --2- 172.21.15.152:0/3469386520 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fed9406df20 0x7fed941f7070 unknown :-1 s=CLOSED pgs=2480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:27.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.694+0000 7fed99c00640 1 -- 172.21.15.152:0/3469386520 >> 172.21.15.152:0/3469386520 conn(0x7fed940d0260 msgr2=0x7fed940b4170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:27.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.694+0000 7fed99c00640 1 -- 172.21.15.152:0/3469386520 shutdown_connections 2024-04-20T11:38:27.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.695+0000 7fed99c00640 1 -- 172.21.15.152:0/3469386520 wait complete. 2024-04-20T11:38:27.713 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T11:38:27.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.906+0000 7f594da00640 1 Processor -- start 2024-04-20T11:38:27.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.906+0000 7f594da00640 1 -- start start 2024-04-20T11:38:27.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.907+0000 7f594da00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f594806f510 0x7f5948140dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:27.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.907+0000 7f594da00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f5948141310 0x7f5948071cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:27.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.907+0000 7f594da00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59480cf0a0 con 0x7f594813e530 2024-04-20T11:38:27.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.907+0000 7f594da00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f5948141b00 con 0x7f5948141310 2024-04-20T11:38:27.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.907+0000 7f594da00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f59480cfc50 con 0x7f594806f510 2024-04-20T11:38:27.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.908+0000 7f594d000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f5948141310 0x7f5948071cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:27.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.908+0000 7f594d000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f5948141310 0x7f5948071cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52258/0 (socket says 172.21.15.152:52258) 2024-04-20T11:38:27.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.908+0000 7f594d000640 1 -- 172.21.15.152:0/2064824288 learned_addr learned my addr 172.21.15.152:0/2064824288 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:38:27.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.908+0000 7f593f400640 1 --2- 172.21.15.152:0/2064824288 >> v2:172.21.15.175:3300/0 conn(0x7f594806f510 0x7f5948140dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:27.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.908+0000 7f594d000640 1 -- 172.21.15.152:0/2064824288 >> v2:172.21.15.175:3300/0 conn(0x7f594806f510 msgr2=0x7f5948140dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:27.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.908+0000 7f594d000640 1 --2- 172.21.15.152:0/2064824288 >> v2:172.21.15.175:3300/0 conn(0x7f594806f510 0x7f5948140dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:27.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.908+0000 7f594d000640 1 -- 172.21.15.152:0/2064824288 >> v1:172.21.15.175:6789/0 conn(0x7f594813e530 legacy=0x7f594806ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:27.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.908+0000 7f594d000640 1 -- 172.21.15.152:0/2064824288 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59480d00e0 con 0x7f5948141310 2024-04-20T11:38:27.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.910+0000 7f594d000640 1 --2- 172.21.15.152:0/2064824288 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5948141310 0x7f5948071cd0 secure :-1 s=READY pgs=2481 cs=0 l=1 rev1=1 crypto rx=0x7f594806feb0 tx=0x7f594002e0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91046af72e05b0b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:27.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.910+0000 7f5947400640 1 -- 172.21.15.152:0/2064824288 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f594002ea90 con 0x7f5948141310 2024-04-20T11:38:27.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.910+0000 7f5947400640 1 -- 172.21.15.152:0/2064824288 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f594002ec30 con 0x7f5948141310 2024-04-20T11:38:27.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.910+0000 7f5947400640 1 -- 172.21.15.152:0/2064824288 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5940009600 con 0x7f5948141310 2024-04-20T11:38:27.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.911+0000 7f594da00640 1 -- 172.21.15.152:0/2064824288 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5948141310 msgr2=0x7f5948071cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:27.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.911+0000 7f594da00640 1 --2- 172.21.15.152:0/2064824288 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5948141310 0x7f5948071cd0 secure :-1 s=READY pgs=2481 cs=0 l=1 rev1=1 crypto rx=0x7f594806feb0 tx=0x7f594002e0f0 comp rx=0 tx=0).stop 2024-04-20T11:38:27.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.911+0000 7f594da00640 1 -- 172.21.15.152:0/2064824288 shutdown_connections 2024-04-20T11:38:27.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.911+0000 7f594da00640 1 --2- 172.21.15.152:0/2064824288 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5948141310 0x7f5948071cd0 unknown :-1 s=CLOSED pgs=2481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:27.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.911+0000 7f594da00640 1 --2- 172.21.15.152:0/2064824288 >> v2:172.21.15.175:3300/0 conn(0x7f594806f510 0x7f5948140dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:27.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.911+0000 7f594da00640 1 -- 172.21.15.152:0/2064824288 >> 172.21.15.152:0/2064824288 conn(0x7f5948067ae0 msgr2=0x7f594806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:27.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.911+0000 7f594da00640 1 -- 172.21.15.152:0/2064824288 shutdown_connections 2024-04-20T11:38:27.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.911+0000 7f594da00640 1 -- 172.21.15.152:0/2064824288 wait complete. 2024-04-20T11:38:27.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.913+0000 7f594da00640 1 Processor -- start 2024-04-20T11:38:27.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.913+0000 7f594da00640 1 -- start start 2024-04-20T11:38:27.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.914+0000 7f594da00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f594806f510 0x7f5948142980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:27.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.914+0000 7f594da00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f594813e530 0x7f59481f37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:27.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.914+0000 7f594da00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5948141310 0x7f59481f3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:27.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.914+0000 7f594da00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f59480cf8e0 con 0x7f594806f510 2024-04-20T11:38:27.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.914+0000 7f594da00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f59480cf600 con 0x7f594813e530 2024-04-20T11:38:27.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.914+0000 7f594da00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f59481417a0 con 0x7f5948141310 2024-04-20T11:38:27.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.914+0000 7f5947e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f594806f510 0x7f5948142980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:27.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.914+0000 7f5947e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f594806f510 0x7f5948142980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52262/0 (socket says 172.21.15.152:52262) 2024-04-20T11:38:27.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.914+0000 7f5947e00640 1 -- 172.21.15.152:0/4078413838 learned_addr learned my addr 172.21.15.152:0/4078413838 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:38:27.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.915+0000 7f593f400640 1 --2- 172.21.15.152:0/4078413838 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f594813e530 0x7f59481f37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:27.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.915+0000 7f5947e00640 1 -- 172.21.15.152:0/4078413838 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5948141310 msgr2=0x7f59481f3ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:27.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.915+0000 7f5947e00640 1 --2- 172.21.15.152:0/4078413838 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5948141310 0x7f59481f3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:27.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.915+0000 7f5947e00640 1 -- 172.21.15.152:0/4078413838 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f594813e530 msgr2=0x7f59481f37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:27.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.915+0000 7f5947e00640 1 --2- 172.21.15.152:0/4078413838 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f594813e530 0x7f59481f37a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:27.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.915+0000 7f5947e00640 1 -- 172.21.15.152:0/4078413838 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5948156580 con 0x7f594806f510 2024-04-20T11:38:27.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.915+0000 7f5947e00640 1 --2- 172.21.15.152:0/4078413838 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f594806f510 0x7f5948142980 secure :-1 s=READY pgs=2482 cs=0 l=1 rev1=1 crypto rx=0x7f593800c430 tx=0x7f593800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:27.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.915+0000 7f5944c00640 1 -- 172.21.15.152:0/4078413838 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5938011790 con 0x7f594806f510 2024-04-20T11:38:27.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.915+0000 7f5944c00640 1 -- 172.21.15.152:0/4078413838 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5938017070 con 0x7f594806f510 2024-04-20T11:38:27.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.915+0000 7f594da00640 1 -- 172.21.15.152:0/4078413838 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5948156860 con 0x7f594806f510 2024-04-20T11:38:27.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.915+0000 7f5944c00640 1 -- 172.21.15.152:0/4078413838 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5938011b70 con 0x7f594806f510 2024-04-20T11:38:27.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.915+0000 7f594da00640 1 -- 172.21.15.152:0/4078413838 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5948156e20 con 0x7f594806f510 2024-04-20T11:38:27.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.916+0000 7f594da00640 1 -- 172.21.15.152:0/4078413838 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5948075e60 con 0x7f594806f510 2024-04-20T11:38:27.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.917+0000 7f5944c00640 1 -- 172.21.15.152:0/4078413838 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 27) ==== 92060+0+0 (secure 0 0 0) 0x7f5938011d10 con 0x7f594806f510 2024-04-20T11:38:27.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.918+0000 7f5944c00640 1 --2- 172.21.15.152:0/4078413838 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f592406d430 0x7f592406f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:27.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.918+0000 7f5944c00640 1 -- 172.21.15.152:0/4078413838 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 4943+0+0 (secure 0 0 0) 0x7f593808a3a0 con 0x7f594806f510 2024-04-20T11:38:27.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.919+0000 7f593f400640 1 --2- 172.21.15.152:0/4078413838 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f592406d430 0x7f592406f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:27.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.920+0000 7f593f400640 1 --2- 172.21.15.152:0/4078413838 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f592406d430 0x7f592406f8f0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f5948159130 tx=0x7f5930008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:27.922 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:27.923+0000 7f5944c00640 1 -- 172.21.15.152:0/4078413838 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f593805a280 con 0x7f594806f510 2024-04-20T11:38:28.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:28.102+0000 7f594da00640 1 -- 172.21.15.152:0/4078413838 --> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f59480b2f30 con 0x7f592406d430 2024-04-20T11:38:28.104 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:38:28.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:28.104+0000 7f5944c00640 1 -- 172.21.15.152:0/4078413838 <== mgr.4109 v2:172.21.15.175:6836/4021250441 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13422 (secure 0 0 0) 0x7f59480b2f30 con 0x7f592406d430 2024-04-20T11:38:28.105 INFO:teuthology.orchestra.run.smithi152.stderr:dumped all 2024-04-20T11:38:28.107 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:28.107+0000 7f594da00640 1 -- 172.21.15.152:0/4078413838 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f592406d430 msgr2=0x7f592406f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:28.107 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:28.107+0000 7f594da00640 1 --2- 172.21.15.152:0/4078413838 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f592406d430 0x7f592406f8f0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f5948159130 tx=0x7f5930008040 comp rx=0 tx=0).stop 2024-04-20T11:38:28.107 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:28.107+0000 7f594da00640 1 -- 172.21.15.152:0/4078413838 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f594806f510 msgr2=0x7f5948142980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:28.107 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:28.107+0000 7f594da00640 1 --2- 172.21.15.152:0/4078413838 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f594806f510 0x7f5948142980 secure :-1 s=READY pgs=2482 cs=0 l=1 rev1=1 crypto rx=0x7f593800c430 tx=0x7f593800c8f0 comp rx=0 tx=0).stop 2024-04-20T11:38:28.107 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:28.107+0000 7f594da00640 1 -- 172.21.15.152:0/4078413838 shutdown_connections 2024-04-20T11:38:28.107 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:28.107+0000 7f594da00640 1 --2- 172.21.15.152:0/4078413838 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f592406d430 0x7f592406f8f0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:28.107 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:28.107+0000 7f594da00640 1 --2- 172.21.15.152:0/4078413838 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5948141310 0x7f59481f3ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:28.107 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:28.107+0000 7f594da00640 1 --2- 172.21.15.152:0/4078413838 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f594813e530 0x7f59481f37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:28.107 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:28.107+0000 7f594da00640 1 --2- 172.21.15.152:0/4078413838 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f594806f510 0x7f5948142980 unknown :-1 s=CLOSED pgs=2482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:28.108 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:28.107+0000 7f594da00640 1 -- 172.21.15.152:0/4078413838 >> 172.21.15.152:0/4078413838 conn(0x7f5948067ae0 msgr2=0x7f59480b4740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:28.108 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:28.107+0000 7f594da00640 1 -- 172.21.15.152:0/4078413838 shutdown_connections 2024-04-20T11:38:28.108 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:28.107+0000 7f594da00640 1 -- 172.21.15.152:0/4078413838 wait complete. 2024-04-20T11:38:28.126 INFO:teuthology.orchestra.run.smithi152.stdout:{"pg_ready":true,"pg_map":{"version":284,"stamp":"2024-04-20T11:38:26.731788+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":271612,"kb_used_data":17992,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754703108,"statfs":{"total":773094113280,"available":772815982592,"internally_reserved":0,"allocated":18423808,"data_stored":8790656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12746,"internal_metadata":259575350},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":-2,"num_read_kb":-2,"num_write":-4,"num_write_kb":-2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002943"},"pg_stats":[{"pgid":"1.0","version":"72'128","reported_seq":478,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-20T11:38:18.192054+0000","last_change":"2024-04-20T11:26:48.661897+0000","last_active":"2024-04-20T11:38:18.192054+0000","last_peered":"2024-04-20T11:38:18.192054+0000","last_clean":"2024-04-20T11:38:18.192054+0000","last_became_active":"2024-04-20T11:26:48.656535+0000","last_became_peered":"2024-04-20T11:26:48.656535+0000","last_unstale":"2024-04-20T11:38:18.192054+0000","last_undegraded":"2024-04-20T11:38:18.192054+0000","last_fullsized":"2024-04-20T11:38:18.192054+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:26:47.543004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:26:47.543004+0000","last_clean_scrub_stamp":"2024-04-20T11:26:47.543004+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:58:35.527105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574993,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33868,"kb_used_data":2684,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337972,"statfs":{"total":96636764160,"available":96602083328,"internally_reserved":0,"allocated":2748416,"data_stored":1541608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574990,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33284,"kb_used_data":2104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338556,"statfs":{"total":96636764160,"available":96602681344,"internally_reserved":0,"allocated":2154496,"data_stored":951240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574992,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33284,"kb_used_data":2104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338556,"statfs":{"total":96636764160,"available":96602681344,"internally_reserved":0,"allocated":2154496,"data_stored":951240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574990,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33288,"kb_used_data":2104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338552,"statfs":{"total":96636764160,"available":96602677248,"internally_reserved":0,"allocated":2154496,"data_stored":951240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574991,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33288,"kb_used_data":2104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338552,"statfs":{"total":96636764160,"available":96602677248,"internally_reserved":0,"allocated":2154496,"data_stored":951240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574992,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33288,"kb_used_data":2104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338552,"statfs":{"total":96636764160,"available":96602677248,"internally_reserved":0,"allocated":2154496,"data_stored":951240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574990,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33288,"kb_used_data":2104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338552,"statfs":{"total":96636764160,"available":96602677248,"internally_reserved":0,"allocated":2154496,"data_stored":951240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574991,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38024,"kb_used_data":2684,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333816,"statfs":{"total":96636764160,"available":96597827584,"internally_reserved":0,"allocated":2748416,"data_stored":1541608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T11:38:28.129 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-04-20T11:26:47.543004+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=11, tm_min=26, tm_sec=47, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=11, tm_min=38, tm_sec=21, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T11:38:28.129 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-20T11:38:48.130 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T11:38:48.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.343+0000 7f9d09400640 1 Processor -- start 2024-04-20T11:38:48.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.343+0000 7f9d09400640 1 -- start start 2024-04-20T11:38:48.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.344+0000 7f9d09400640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f9d040ba520 0x7f9d040d28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:48.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.344+0000 7f9d09400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d040d2e30 0x7f9d040d1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:48.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.344+0000 7f9d09400640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d040b80b0 con 0x7f9d0415eb80 2024-04-20T11:38:48.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.344+0000 7f9d09400640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f9d040b8330 con 0x7f9d040ba520 2024-04-20T11:38:48.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.344+0000 7f9d09400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f9d040b8610 con 0x7f9d040d2e30 2024-04-20T11:38:48.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.345+0000 7f9d03200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d040d2e30 0x7f9d040d1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:48.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.345+0000 7f9d03200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d040d2e30 0x7f9d040d1810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.152:36718/0 (socket says 172.21.15.152:36718) 2024-04-20T11:38:48.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.345+0000 7f9d01e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f9d040ba520 0x7f9d040d28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:48.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.345+0000 7f9d03200640 1 -- 172.21.15.152:0/1547824523 learned_addr learned my addr 172.21.15.152:0/1547824523 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:38:48.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.346+0000 7f9d01e00640 1 -- 172.21.15.152:0/1547824523 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d040d2e30 msgr2=0x7f9d040d1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:48.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.346+0000 7f9d01e00640 1 --2- 172.21.15.152:0/1547824523 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d040d2e30 0x7f9d040d1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:48.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.346+0000 7f9d01e00640 1 -- 172.21.15.152:0/1547824523 >> v1:172.21.15.175:6789/0 conn(0x7f9d0415eb80 legacy=0x7f9d040b79a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:48.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.346+0000 7f9d01e00640 1 -- 172.21.15.152:0/1547824523 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d040b88f0 con 0x7f9d040ba520 2024-04-20T11:38:48.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.346+0000 7f9d03200640 1 --2- 172.21.15.152:0/1547824523 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d040d2e30 0x7f9d040d1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:38:48.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.347+0000 7f9d01e00640 1 --2- 172.21.15.152:0/1547824523 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d040ba520 0x7f9d040d28f0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f9cf4009d90 tx=0x7f9cf402be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=948d8357db20635c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:48.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.348+0000 7f9d01400640 1 -- 172.21.15.152:0/1547824523 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cf4039070 con 0x7f9d040ba520 2024-04-20T11:38:48.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.348+0000 7f9d01400640 1 -- 172.21.15.152:0/1547824523 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cf402cbc0 con 0x7f9d040ba520 2024-04-20T11:38:48.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.348+0000 7f9d01400640 1 -- 172.21.15.152:0/1547824523 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cf4034440 con 0x7f9d040ba520 2024-04-20T11:38:48.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.348+0000 7f9d09400640 1 -- 172.21.15.152:0/1547824523 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d040ba520 msgr2=0x7f9d040d28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:48.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.348+0000 7f9d09400640 1 --2- 172.21.15.152:0/1547824523 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d040ba520 0x7f9d040d28f0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f9cf4009d90 tx=0x7f9cf402be10 comp rx=0 tx=0).stop 2024-04-20T11:38:48.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.349+0000 7f9d09400640 1 -- 172.21.15.152:0/1547824523 shutdown_connections 2024-04-20T11:38:48.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.349+0000 7f9d09400640 1 --2- 172.21.15.152:0/1547824523 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d040d2e30 0x7f9d040d1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:48.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.349+0000 7f9d09400640 1 --2- 172.21.15.152:0/1547824523 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d040ba520 0x7f9d040d28f0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:48.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.349+0000 7f9d09400640 1 -- 172.21.15.152:0/1547824523 >> 172.21.15.152:0/1547824523 conn(0x7f9d04067ae0 msgr2=0x7f9d04067ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:48.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.349+0000 7f9d09400640 1 -- 172.21.15.152:0/1547824523 shutdown_connections 2024-04-20T11:38:48.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.349+0000 7f9d09400640 1 -- 172.21.15.152:0/1547824523 wait complete. 2024-04-20T11:38:48.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.351+0000 7f9d09400640 1 Processor -- start 2024-04-20T11:38:48.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.351+0000 7f9d09400640 1 -- start start 2024-04-20T11:38:48.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.352+0000 7f9d09400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d040ba520 0x7f9d041f4250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:48.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.352+0000 7f9d09400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d040d2e30 0x7f9d040ca6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:48.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.352+0000 7f9d09400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d0415eb80 0x7f9d040cac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:48.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.352+0000 7f9d09400640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f9d040d1ed0 con 0x7f9d0415eb80 2024-04-20T11:38:48.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.352+0000 7f9d09400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9d040d1d50 con 0x7f9d040ba520 2024-04-20T11:38:48.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.352+0000 7f9d09400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f9d040d2050 con 0x7f9d040d2e30 2024-04-20T11:38:48.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.353+0000 7f9d03200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d0415eb80 0x7f9d040cac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:48.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.353+0000 7f9d03200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d0415eb80 0x7f9d040cac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:47160/0 (socket says 172.21.15.152:47160) 2024-04-20T11:38:48.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.353+0000 7f9d03200640 1 -- 172.21.15.152:0/3489149852 learned_addr learned my addr 172.21.15.152:0/3489149852 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:38:48.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.353+0000 7f9d01e00640 1 --2- 172.21.15.152:0/3489149852 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d040d2e30 0x7f9d040ca6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:48.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.353+0000 7f9d02800640 1 --2- 172.21.15.152:0/3489149852 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d040ba520 0x7f9d041f4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:48.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.353+0000 7f9d03200640 1 -- 172.21.15.152:0/3489149852 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d040d2e30 msgr2=0x7f9d040ca6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:48.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.353+0000 7f9d03200640 1 --2- 172.21.15.152:0/3489149852 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d040d2e30 0x7f9d040ca6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:48.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.353+0000 7f9d03200640 1 -- 172.21.15.152:0/3489149852 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d040ba520 msgr2=0x7f9d041f4250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:48.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.353+0000 7f9d03200640 1 --2- 172.21.15.152:0/3489149852 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d040ba520 0x7f9d041f4250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:48.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.353+0000 7f9d03200640 1 -- 172.21.15.152:0/3489149852 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d041566d0 con 0x7f9d0415eb80 2024-04-20T11:38:48.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.354+0000 7f9d03200640 1 --2- 172.21.15.152:0/3489149852 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d0415eb80 0x7f9d040cac20 secure :-1 s=READY pgs=2483 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc006e00 tx=0x7f9cfc00ee10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:48.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.354+0000 7f9cfaa00640 1 -- 172.21.15.152:0/3489149852 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cfc004030 con 0x7f9d0415eb80 2024-04-20T11:38:48.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.354+0000 7f9cfaa00640 1 -- 172.21.15.152:0/3489149852 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cfc0041d0 con 0x7f9d0415eb80 2024-04-20T11:38:48.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.354+0000 7f9d09400640 1 -- 172.21.15.152:0/3489149852 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d0415aab0 con 0x7f9d0415eb80 2024-04-20T11:38:48.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.354+0000 7f9cfaa00640 1 -- 172.21.15.152:0/3489149852 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cfc00c9a0 con 0x7f9d0415eb80 2024-04-20T11:38:48.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.354+0000 7f9d09400640 1 -- 172.21.15.152:0/3489149852 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d0415b350 con 0x7f9d0415eb80 2024-04-20T11:38:48.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.355+0000 7f9d09400640 1 -- 172.21.15.152:0/3489149852 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d040689f0 con 0x7f9d0415eb80 2024-04-20T11:38:48.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.356+0000 7f9cfaa00640 1 -- 172.21.15.152:0/3489149852 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 27) ==== 92060+0+0 (secure 0 0 0) 0x7f9cfc00cb40 con 0x7f9d0415eb80 2024-04-20T11:38:48.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.358+0000 7f9cfaa00640 1 --2- 172.21.15.152:0/3489149852 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f9ce006d460 0x7f9ce006f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:48.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.358+0000 7f9cfaa00640 1 -- 172.21.15.152:0/3489149852 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 4943+0+0 (secure 0 0 0) 0x7f9cfc08d380 con 0x7f9d0415eb80 2024-04-20T11:38:48.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.358+0000 7f9d02800640 1 --2- 172.21.15.152:0/3489149852 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f9ce006d460 0x7f9ce006f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:48.359 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.360+0000 7f9d02800640 1 --2- 172.21.15.152:0/3489149852 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f9ce006d460 0x7f9ce006f920 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f9cec007fd0 tx=0x7f9cec009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:48.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.363+0000 7f9cfaa00640 1 -- 172.21.15.152:0/3489149852 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f9cfc017020 con 0x7f9d0415eb80 2024-04-20T11:38:48.559 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.559+0000 7f9d09400640 1 -- 172.21.15.152:0/3489149852 --> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9d040d5660 con 0x7f9ce006d460 2024-04-20T11:38:48.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.562+0000 7f9cfaa00640 1 -- 172.21.15.152:0/3489149852 <== mgr.4109 v2:172.21.15.175:6836/4021250441 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13380 (secure 0 0 0) 0x7f9d040d5660 con 0x7f9ce006d460 2024-04-20T11:38:48.562 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:38:48.562 INFO:teuthology.orchestra.run.smithi152.stderr:dumped all 2024-04-20T11:38:48.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.564+0000 7f9d09400640 1 -- 172.21.15.152:0/3489149852 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f9ce006d460 msgr2=0x7f9ce006f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:48.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.564+0000 7f9d09400640 1 --2- 172.21.15.152:0/3489149852 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f9ce006d460 0x7f9ce006f920 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f9cec007fd0 tx=0x7f9cec009040 comp rx=0 tx=0).stop 2024-04-20T11:38:48.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.564+0000 7f9d09400640 1 -- 172.21.15.152:0/3489149852 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d0415eb80 msgr2=0x7f9d040cac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:48.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.564+0000 7f9d09400640 1 --2- 172.21.15.152:0/3489149852 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d0415eb80 0x7f9d040cac20 secure :-1 s=READY pgs=2483 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc006e00 tx=0x7f9cfc00ee10 comp rx=0 tx=0).stop 2024-04-20T11:38:48.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.564+0000 7f9d09400640 1 -- 172.21.15.152:0/3489149852 shutdown_connections 2024-04-20T11:38:48.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.564+0000 7f9d09400640 1 --2- 172.21.15.152:0/3489149852 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f9ce006d460 0x7f9ce006f920 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:48.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.564+0000 7f9d09400640 1 --2- 172.21.15.152:0/3489149852 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d0415eb80 0x7f9d040cac20 unknown :-1 s=CLOSED pgs=2483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:48.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.564+0000 7f9d09400640 1 --2- 172.21.15.152:0/3489149852 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d040d2e30 0x7f9d040ca6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:48.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.564+0000 7f9d09400640 1 --2- 172.21.15.152:0/3489149852 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d040ba520 0x7f9d041f4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:48.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.564+0000 7f9d09400640 1 -- 172.21.15.152:0/3489149852 >> 172.21.15.152:0/3489149852 conn(0x7f9d04067ae0 msgr2=0x7f9d040c8550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:48.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.565+0000 7f9d09400640 1 -- 172.21.15.152:0/3489149852 shutdown_connections 2024-04-20T11:38:48.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.565+0000 7f9d09400640 1 -- 172.21.15.152:0/3489149852 wait complete. 2024-04-20T11:38:48.582 INFO:teuthology.orchestra.run.smithi152.stdout:{"pg_ready":true,"pg_map":{"version":294,"stamp":"2024-04-20T11:38:46.738069+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":130,"ondisk_log_size":130,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":271600,"kb_used_data":17992,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754703120,"statfs":{"total":773094113280,"available":772815994880,"internally_reserved":0,"allocated":18423808,"data_stored":8790656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12746,"internal_metadata":259575350},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004078"},"pg_stats":[{"pgid":"1.0","version":"170'130","reported_seq":489,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-20T11:38:28.440711+0000","last_change":"2024-04-20T11:38:28.440781+0000","last_active":"2024-04-20T11:38:28.440711+0000","last_peered":"2024-04-20T11:38:28.440711+0000","last_clean":"2024-04-20T11:38:28.440711+0000","last_became_active":"2024-04-20T11:26:48.656535+0000","last_became_peered":"2024-04-20T11:26:48.656535+0000","last_unstale":"2024-04-20T11:38:28.440711+0000","last_undegraded":"2024-04-20T11:38:28.440711+0000","last_fullsized":"2024-04-20T11:38:28.440711+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"170'130","last_scrub_stamp":"2024-04-20T11:38:28.440520+0000","last_deep_scrub":"170'130","last_deep_scrub_stamp":"2024-04-20T11:38:28.440520+0000","last_clean_scrub_stamp":"2024-04-20T11:38:28.440520+0000","objects_scrubbed":2,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:29:41.154010+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":130,"ondisk_log_size":130,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574997,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33860,"kb_used_data":2684,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337980,"statfs":{"total":96636764160,"available":96602091520,"internally_reserved":0,"allocated":2748416,"data_stored":1541608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574994,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33284,"kb_used_data":2104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338556,"statfs":{"total":96636764160,"available":96602681344,"internally_reserved":0,"allocated":2154496,"data_stored":951240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574996,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33284,"kb_used_data":2104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338556,"statfs":{"total":96636764160,"available":96602681344,"internally_reserved":0,"allocated":2154496,"data_stored":951240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574994,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33288,"kb_used_data":2104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338552,"statfs":{"total":96636764160,"available":96602677248,"internally_reserved":0,"allocated":2154496,"data_stored":951240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574995,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33288,"kb_used_data":2104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338552,"statfs":{"total":96636764160,"available":96602677248,"internally_reserved":0,"allocated":2154496,"data_stored":951240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574996,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33288,"kb_used_data":2104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338552,"statfs":{"total":96636764160,"available":96602677248,"internally_reserved":0,"allocated":2154496,"data_stored":951240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574994,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33288,"kb_used_data":2104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338552,"statfs":{"total":96636764160,"available":96602677248,"internally_reserved":0,"allocated":2154496,"data_stored":951240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574995,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38020,"kb_used_data":2684,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333820,"statfs":{"total":96636764160,"available":96597831680,"internally_reserved":0,"allocated":2748416,"data_stored":1541608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T11:38:48.583 DEBUG:teuthology.orchestra.run.smithi152:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-04-20T11:38:48.776 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.776+0000 7f2fb8400640 1 Processor -- start 2024-04-20T11:38:48.776 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.776+0000 7f2fb8400640 1 -- start start 2024-04-20T11:38:48.777 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.778+0000 7f2fb8400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2fb015b510 0x7f2fb015fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:48.777 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.778+0000 7f2fb8400640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f2fb00cfc40 0x7f2fb00b3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:48.778 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.778+0000 7f2fb8400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f2fb00b2170 0x7f2fb00b2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:48.778 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.778+0000 7f2fb8400640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7f2fb006f420 con 0x7f2fb00b2170 2024-04-20T11:38:48.778 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.778+0000 7f2fb8400640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f2fb00b4430 con 0x7f2fb00cfc40 2024-04-20T11:38:48.778 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.778+0000 7f2fb8400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2fb015f1c0 con 0x7f2fb015b510 2024-04-20T11:38:48.779 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.778+0000 7f2fb6400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f2fb00b2170 0x7f2fb00b2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:48.779 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.778+0000 7f2fb6400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f2fb00b2170 0x7f2fb00b2620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:47172/0 (socket says 172.21.15.152:47172) 2024-04-20T11:38:48.779 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.778+0000 7f2fb6400640 1 -- 172.21.15.152:0/2074633410 learned_addr learned my addr 172.21.15.152:0/2074633410 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:38:48.779 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.779+0000 7f2fb5a00640 1 --2- 172.21.15.152:0/2074633410 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2fb015b510 0x7f2fb015fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:48.779 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.779+0000 7f2fb5000640 1 --2- 172.21.15.152:0/2074633410 >> v2:172.21.15.175:3300/0 conn(0x7f2fb00cfc40 0x7f2fb00b3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:48.779 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.779+0000 7f2fb6400640 1 -- 172.21.15.152:0/2074633410 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2fb015b510 msgr2=0x7f2fb015fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:48.779 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.779+0000 7f2fb6400640 1 --2- 172.21.15.152:0/2074633410 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2fb015b510 0x7f2fb015fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:48.779 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.779+0000 7f2fb6400640 1 -- 172.21.15.152:0/2074633410 >> v2:172.21.15.175:3300/0 conn(0x7f2fb00cfc40 msgr2=0x7f2fb00b3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:48.779 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.779+0000 7f2fb6400640 1 --2- 172.21.15.152:0/2074633410 >> v2:172.21.15.175:3300/0 conn(0x7f2fb00cfc40 0x7f2fb00b3ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:48.779 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.779+0000 7f2fb6400640 1 -- 172.21.15.152:0/2074633410 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fb006f670 con 0x7f2fb00b2170 2024-04-20T11:38:48.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.780+0000 7f2fb6400640 1 --2- 172.21.15.152:0/2074633410 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2fb00b2170 0x7f2fb00b2620 secure :-1 s=READY pgs=2484 cs=0 l=1 rev1=1 crypto rx=0x7f2fb0072fd0 tx=0x7f2fa802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dd193e9dd45dfe1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:48.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.780+0000 7f2fafe00640 1 -- 172.21.15.152:0/2074633410 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fa8035020 con 0x7f2fb00b2170 2024-04-20T11:38:48.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.780+0000 7f2fafe00640 1 -- 172.21.15.152:0/2074633410 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fa8031640 con 0x7f2fb00b2170 2024-04-20T11:38:48.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.780+0000 7f2fafe00640 1 -- 172.21.15.152:0/2074633410 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fa802d040 con 0x7f2fb00b2170 2024-04-20T11:38:48.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.781+0000 7f2fb8400640 1 -- 172.21.15.152:0/2074633410 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2fb00b2170 msgr2=0x7f2fb00b2620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:48.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.781+0000 7f2fb8400640 1 --2- 172.21.15.152:0/2074633410 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2fb00b2170 0x7f2fb00b2620 secure :-1 s=READY pgs=2484 cs=0 l=1 rev1=1 crypto rx=0x7f2fb0072fd0 tx=0x7f2fa802b8f0 comp rx=0 tx=0).stop 2024-04-20T11:38:48.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.781+0000 7f2fb8400640 1 -- 172.21.15.152:0/2074633410 shutdown_connections 2024-04-20T11:38:48.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.781+0000 7f2fb8400640 1 --2- 172.21.15.152:0/2074633410 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2fb00b2170 0x7f2fb00b2620 unknown :-1 s=CLOSED pgs=2484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:48.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.781+0000 7f2fb8400640 1 --2- 172.21.15.152:0/2074633410 >> v2:172.21.15.175:3300/0 conn(0x7f2fb00cfc40 0x7f2fb00b3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:48.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.781+0000 7f2fb8400640 1 --2- 172.21.15.152:0/2074633410 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2fb015b510 0x7f2fb015fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:48.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.781+0000 7f2fb8400640 1 -- 172.21.15.152:0/2074633410 >> 172.21.15.152:0/2074633410 conn(0x7f2fb006b980 msgr2=0x7f2fb006bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:48.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.781+0000 7f2fb8400640 1 -- 172.21.15.152:0/2074633410 shutdown_connections 2024-04-20T11:38:48.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.781+0000 7f2fb8400640 1 -- 172.21.15.152:0/2074633410 wait complete. 2024-04-20T11:38:48.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.783+0000 7f2fb8400640 1 Processor -- start 2024-04-20T11:38:48.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.783+0000 7f2fb8400640 1 -- start start 2024-04-20T11:38:48.785 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.784+0000 7f2fb8400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2fb00b2170 0x7f2fb015d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:48.785 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.784+0000 7f2fb8400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2fb00cfc40 0x7f2fb015c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:48.785 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.784+0000 7f2fb8400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2fb015b510 0x7f2fb0157750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:48.785 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.784+0000 7f2fb8400640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb015b8f0 con 0x7f2fb00b2170 2024-04-20T11:38:48.785 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.784+0000 7f2fb8400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb015f710 con 0x7f2fb015b510 2024-04-20T11:38:48.785 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.784+0000 7f2fb8400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2fb006f950 con 0x7f2fb00cfc40 2024-04-20T11:38:48.785 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.784+0000 7f2fb5a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2fb00b2170 0x7f2fb015d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:48.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.784+0000 7f2fb5a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2fb00b2170 0x7f2fb015d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:47178/0 (socket says 172.21.15.152:47178) 2024-04-20T11:38:48.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.784+0000 7f2fb5a00640 1 -- 172.21.15.152:0/3529555268 learned_addr learned my addr 172.21.15.152:0/3529555268 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:38:48.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.785+0000 7f2fb6400640 1 --2- 172.21.15.152:0/3529555268 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2fb015b510 0x7f2fb0157750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:48.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.785+0000 7f2fb5a00640 1 -- 172.21.15.152:0/3529555268 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2fb00cfc40 msgr2=0x7f2fb015c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:48.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.785+0000 7f2fb5000640 1 --2- 172.21.15.152:0/3529555268 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2fb00cfc40 0x7f2fb015c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:48.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.785+0000 7f2fb5a00640 1 --2- 172.21.15.152:0/3529555268 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2fb00cfc40 0x7f2fb015c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:48.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.785+0000 7f2fb5a00640 1 -- 172.21.15.152:0/3529555268 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2fb015b510 msgr2=0x7f2fb0157750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:48.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.785+0000 7f2fb5a00640 1 --2- 172.21.15.152:0/3529555268 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2fb015b510 0x7f2fb0157750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:48.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.785+0000 7f2fb5a00640 1 -- 172.21.15.152:0/3529555268 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fb014c440 con 0x7f2fb00b2170 2024-04-20T11:38:48.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.785+0000 7f2fb6400640 1 --2- 172.21.15.152:0/3529555268 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2fb015b510 0x7f2fb0157750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:38:48.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.785+0000 7f2fb5000640 1 --2- 172.21.15.152:0/3529555268 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2fb00cfc40 0x7f2fb015c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:38:48.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.785+0000 7f2fb5a00640 1 --2- 172.21.15.152:0/3529555268 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2fb00b2170 0x7f2fb015d8d0 secure :-1 s=READY pgs=2485 cs=0 l=1 rev1=1 crypto rx=0x7f2fa4007e40 tx=0x7f2fa400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:48.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.785+0000 7f2fad600640 1 -- 172.21.15.152:0/3529555268 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fa4014620 con 0x7f2fb00b2170 2024-04-20T11:38:48.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.785+0000 7f2fad600640 1 -- 172.21.15.152:0/3529555268 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fa4019070 con 0x7f2fb00b2170 2024-04-20T11:38:48.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.785+0000 7f2fb8400640 1 -- 172.21.15.152:0/3529555268 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fb014c720 con 0x7f2fb00b2170 2024-04-20T11:38:48.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.785+0000 7f2fad600640 1 -- 172.21.15.152:0/3529555268 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fa4014a00 con 0x7f2fb00b2170 2024-04-20T11:38:48.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.786+0000 7f2fb8400640 1 -- 172.21.15.152:0/3529555268 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fb014cce0 con 0x7f2fb00b2170 2024-04-20T11:38:48.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.787+0000 7f2fb8400640 1 -- 172.21.15.152:0/3529555268 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fb00689f0 con 0x7f2fb00b2170 2024-04-20T11:38:48.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.788+0000 7f2fad600640 1 -- 172.21.15.152:0/3529555268 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 27) ==== 92060+0+0 (secure 0 0 0) 0x7f2fa4014ba0 con 0x7f2fb00b2170 2024-04-20T11:38:48.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.789+0000 7f2fad600640 1 --2- 172.21.15.152:0/3529555268 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f2f9006d460 0x7f2f9006f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:48.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.789+0000 7f2fad600640 1 -- 172.21.15.152:0/3529555268 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 4943+0+0 (secure 0 0 0) 0x7f2fa408ad40 con 0x7f2fb00b2170 2024-04-20T11:38:48.789 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.789+0000 7f2fb5000640 1 --2- 172.21.15.152:0/3529555268 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f2f9006d460 0x7f2f9006f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:48.790 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.790+0000 7f2fb5000640 1 --2- 172.21.15.152:0/3529555268 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f2f9006d460 0x7f2f9006f920 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f2fb0072900 tx=0x7f2f9c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:48.792 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.793+0000 7f2fad600640 1 -- 172.21.15.152:0/3529555268 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f2fa405ac20 con 0x7f2fb00b2170 2024-04-20T11:38:48.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:48.995+0000 7f2fb8400640 1 -- 172.21.15.152:0/3529555268 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7f2fb006ecf0 con 0x7f2fb00b2170 2024-04-20T11:38:49.002 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:49.002+0000 7f2fad600640 1 -- 172.21.15.152:0/3529555268 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v32)=0 v32) ==== 123+0+0 (secure 0 0 0) 0x7f2fa405a520 con 0x7f2fb00b2170 2024-04-20T11:38:49.003 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:49.003+0000 7f2fad600640 1 -- 172.21.15.152:0/3529555268 <== mon.0 v2:172.21.15.152:3300/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f2fa405e8d0 con 0x7f2fb00b2170 2024-04-20T11:38:49.005 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:49.005+0000 7f2fb8400640 1 -- 172.21.15.152:0/3529555268 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f2f9006d460 msgr2=0x7f2f9006f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:49.006 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:49.005+0000 7f2fb8400640 1 --2- 172.21.15.152:0/3529555268 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f2f9006d460 0x7f2f9006f920 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f2fb0072900 tx=0x7f2f9c008040 comp rx=0 tx=0).stop 2024-04-20T11:38:49.006 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:49.005+0000 7f2fb8400640 1 -- 172.21.15.152:0/3529555268 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2fb00b2170 msgr2=0x7f2fb015d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:49.006 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:49.005+0000 7f2fb8400640 1 --2- 172.21.15.152:0/3529555268 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2fb00b2170 0x7f2fb015d8d0 secure :-1 s=READY pgs=2485 cs=0 l=1 rev1=1 crypto rx=0x7f2fa4007e40 tx=0x7f2fa400e4b0 comp rx=0 tx=0).stop 2024-04-20T11:38:49.006 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:49.006+0000 7f2fb8400640 1 -- 172.21.15.152:0/3529555268 shutdown_connections 2024-04-20T11:38:49.006 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:49.006+0000 7f2fb8400640 1 --2- 172.21.15.152:0/3529555268 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7f2f9006d460 0x7f2f9006f920 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:49.006 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:49.006+0000 7f2fb8400640 1 --2- 172.21.15.152:0/3529555268 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2fb015b510 0x7f2fb0157750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:49.006 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:49.006+0000 7f2fb8400640 1 --2- 172.21.15.152:0/3529555268 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2fb00cfc40 0x7f2fb015c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:49.006 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:49.006+0000 7f2fb8400640 1 --2- 172.21.15.152:0/3529555268 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2fb00b2170 0x7f2fb015d8d0 unknown :-1 s=CLOSED pgs=2485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:49.006 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:49.006+0000 7f2fb8400640 1 -- 172.21.15.152:0/3529555268 >> 172.21.15.152:0/3529555268 conn(0x7f2fb006b980 msgr2=0x7f2fb00d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:49.006 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:49.006+0000 7f2fb8400640 1 -- 172.21.15.152:0/3529555268 shutdown_connections 2024-04-20T11:38:49.006 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:49.006+0000 7f2fb8400640 1 -- 172.21.15.152:0/3529555268 wait complete. 2024-04-20T11:38:49.026 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-20T11:38:49.026 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:38:49.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:49.229+0000 7fe657200640 1 Processor -- start 2024-04-20T11:38:49.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:49.229+0000 7fe657200640 1 -- start start 2024-04-20T11:38:49.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:49.231+0000 7fe657200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fe65815fcc0 0x7fe6580d4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:49.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:49.231+0000 7fe657200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fe6581603d0 0x7fe6580b5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:49.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:49.231+0000 7fe657200640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6580d3350 con 0x7fe6580b5480 2024-04-20T11:38:49.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:49.231+0000 7fe657200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 -- 0x7fe6580d3630 con 0x7fe6581603d0 2024-04-20T11:38:49.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:49.231+0000 7fe657200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fe6580d3910 con 0x7fe65815fcc0 2024-04-20T11:38:49.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:49.231+0000 7fe656800640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fe6580b5480 0x7fe6581624d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:33680/0 (socket says 172.21.15.152:33680) 2024-04-20T11:38:49.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:49.231+0000 7fe656800640 1 -- 172.21.15.152:0/2989977964 learned_addr learned my addr 172.21.15.152:0/2989977964 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-04-20T11:38:49.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:49.231+0000 7fe64d400640 1 --2- 172.21.15.152:0/2989977964 >> v2:172.21.15.152:3300/0 conn(0x7fe6581603d0 0x7fe6580b5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:49.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:49.232+0000 7fe655e00640 1 --2- 172.21.15.152:0/2989977964 >> v2:172.21.15.175:3300/0 conn(0x7fe65815fcc0 0x7fe6580d4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:49.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:49.232+0000 7fe655400640 1 -- 172.21.15.152:0/2989977964 <== mon.0 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3082013107 0 0) 0x7fe6580d3350 con 0x7fe6580b5480 2024-04-20T11:38:49.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:49.232+0000 7fe655400640 1 -- 172.21.15.152:0/2989977964 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe63c003520 con 0x7fe6580b5480 2024-04-20T11:38:49.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:49.232+0000 7fe64d400640 1 -- 172.21.15.152:0/2989977964 >> v2:172.21.15.175:3300/0 conn(0x7fe65815fcc0 msgr2=0x7fe6580d4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:38:49.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:49.232+0000 7fe64d400640 1 --2- 172.21.15.152:0/2989977964 >> v2:172.21.15.175:3300/0 conn(0x7fe65815fcc0 0x7fe6580d4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:49.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:49.232+0000 7fe64d400640 1 -- 172.21.15.152:0/2989977964 >> v1:172.21.15.152:6789/0 conn(0x7fe6580b5480 legacy=0x7fe6581624d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:49.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:49.232+0000 7fe64d400640 1 -- 172.21.15.152:0/2989977964 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6580ced20 con 0x7fe6581603d0 2024-04-20T11:38:49.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:49.233+0000 7fe64d400640 1 --2- 172.21.15.152:0/2989977964 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe6581603d0 0x7fe6580b5eb0 secure :-1 s=READY pgs=2487 cs=0 l=1 rev1=1 crypto rx=0x7fe65808cb80 tx=0x7fe64802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a08a1c6f14cd44 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:49.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:49.234+0000 7fe655400640 1 -- 172.21.15.152:0/2989977964 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe648035020 con 0x7fe6581603d0 2024-04-20T11:38:49.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:49.234+0000 7fe655400640 1 -- 172.21.15.152:0/2989977964 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fe648031830 con 0x7fe6581603d0 2024-04-20T11:38:49.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:49.234+0000 7fe655400640 1 -- 172.21.15.152:0/2989977964 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe648031c20 con 0x7fe6581603d0 2024-04-20T11:38:49.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:49.234+0000 7fe657200640 1 -- 172.21.15.152:0/2989977964 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe6581603d0 msgr2=0x7fe6580b5eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:49.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:49.234+0000 7fe657200640 1 --2- 172.21.15.152:0/2989977964 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe6581603d0 0x7fe6580b5eb0 secure :-1 s=READY pgs=2487 cs=0 l=1 rev1=1 crypto rx=0x7fe65808cb80 tx=0x7fe64802b8d0 comp rx=0 tx=0).stop 2024-04-20T11:38:49.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:49.235+0000 7fe657200640 1 -- 172.21.15.152:0/2989977964 shutdown_connections 2024-04-20T11:38:49.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:49.235+0000 7fe657200640 1 --2- 172.21.15.152:0/2989977964 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe6581603d0 0x7fe6580b5eb0 unknown :-1 s=CLOSED pgs=2487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:49.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:49.235+0000 7fe657200640 1 --2- 172.21.15.152:0/2989977964 >> v2:172.21.15.175:3300/0 conn(0x7fe65815fcc0 0x7fe6580d4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:49.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:49.235+0000 7fe657200640 1 -- 172.21.15.152:0/2989977964 >> 172.21.15.152:0/2989977964 conn(0x7fe65808bfd0 msgr2=0x7fe6580d4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:49.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:49.235+0000 7fe657200640 1 -- 172.21.15.152:0/2989977964 shutdown_connections 2024-04-20T11:38:49.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:49.235+0000 7fe657200640 1 -- 172.21.15.152:0/2989977964 wait complete. 2024-04-20T11:38:49.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:49.236+0000 7fe657200640 1 Processor -- start 2024-04-20T11:38:49.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:49.237+0000 7fe657200640 1 -- start start 2024-04-20T11:38:49.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:49.237+0000 7fe657200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe6580b5480 0x7fe6581615c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:49.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:49.238+0000 7fe657200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe65815fcc0 0x7fe6580ca490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:49.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:49.238+0000 7fe657200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe6581603d0 0x7fe658154330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:49.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:49.238+0000 7fe657200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fe6580b6570 con 0x7fe6580b5480 2024-04-20T11:38:49.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:49.238+0000 7fe657200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe6580b63f0 con 0x7fe65815fcc0 2024-04-20T11:38:49.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:49.238+0000 7fe657200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe6580b66f0 con 0x7fe6581603d0 2024-04-20T11:38:49.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:49.238+0000 7fe655e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe6580b5480 0x7fe6581615c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:49.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:49.238+0000 7fe655e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe6580b5480 0x7fe6581615c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:47188/0 (socket says 172.21.15.152:47188) 2024-04-20T11:38:49.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:49.238+0000 7fe655e00640 1 -- 172.21.15.152:0/440750930 learned_addr learned my addr 172.21.15.152:0/440750930 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-20T11:38:49.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:49.238+0000 7fe655e00640 1 -- 172.21.15.152:0/440750930 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe6581603d0 msgr2=0x7fe658154330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:49.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:49.238+0000 7fe655e00640 1 --2- 172.21.15.152:0/440750930 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe6581603d0 0x7fe658154330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:49.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:49.238+0000 7fe655e00640 1 -- 172.21.15.152:0/440750930 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe65815fcc0 msgr2=0x7fe6580ca490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:49.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:49.238+0000 7fe655e00640 1 --2- 172.21.15.152:0/440750930 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe65815fcc0 0x7fe6580ca490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:49.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:49.238+0000 7fe655e00640 1 -- 172.21.15.152:0/440750930 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe65809e490 con 0x7fe6580b5480 2024-04-20T11:38:49.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:49.239+0000 7fe655e00640 1 --2- 172.21.15.152:0/440750930 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe6580b5480 0x7fe6581615c0 secure :-1 s=READY pgs=2488 cs=0 l=1 rev1=1 crypto rx=0x7fe64000d800 tx=0x7fe64000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:49.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:49.239+0000 7fe64ea00640 1 -- 172.21.15.152:0/440750930 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe64000b6c0 con 0x7fe6580b5480 2024-04-20T11:38:49.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:49.239+0000 7fe64ea00640 1 -- 172.21.15.152:0/440750930 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fe640019070 con 0x7fe6580b5480 2024-04-20T11:38:49.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:49.239+0000 7fe657200640 1 -- 172.21.15.152:0/440750930 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe65809e770 con 0x7fe6580b5480 2024-04-20T11:38:49.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:49.239+0000 7fe64ea00640 1 -- 172.21.15.152:0/440750930 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe64000baa0 con 0x7fe6580b5480 2024-04-20T11:38:49.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:49.239+0000 7fe657200640 1 -- 172.21.15.152:0/440750930 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe65809ed30 con 0x7fe6580b5480 2024-04-20T11:38:49.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:49.240+0000 7fe657200640 1 -- 172.21.15.152:0/440750930 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe658075e60 con 0x7fe6580b5480 2024-04-20T11:38:49.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:49.241+0000 7fe64ea00640 1 -- 172.21.15.152:0/440750930 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 27) ==== 92060+0+0 (secure 0 0 0) 0x7fe64000bc40 con 0x7fe6580b5480 2024-04-20T11:38:49.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:49.242+0000 7fe64ea00640 1 --2- 172.21.15.152:0/440750930 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe63406d430 0x7fe63406f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:49.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:49.242+0000 7fe64ea00640 1 -- 172.21.15.152:0/440750930 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 4943+0+0 (secure 0 0 0) 0x7fe64008aab0 con 0x7fe6580b5480 2024-04-20T11:38:49.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:49.243+0000 7fe64d400640 1 --2- 172.21.15.152:0/440750930 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe63406d430 0x7fe63406f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:49.243 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:49.244+0000 7fe64d400640 1 --2- 172.21.15.152:0/440750930 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe63406d430 0x7fe63406f8f0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fe648036000 tx=0x7fe648030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:49.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:49.247+0000 7fe64ea00640 1 -- 172.21.15.152:0/440750930 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe64005a870 con 0x7fe6580b5480 2024-04-20T11:38:49.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:49.479+0000 7fe657200640 1 -- 172.21.15.152:0/440750930 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe6580ba4d0 con 0x7fe6580b5480 2024-04-20T11:38:49.480 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:49.480+0000 7fe64ea00640 1 -- 172.21.15.152:0/440750930 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 118 v118) ==== 95+0+3884 (secure 0 0 0) 0x7fe64005a170 con 0x7fe6580b5480 2024-04-20T11:38:49.480 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-20T11:38:49.480 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":118,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6605,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6834/4020670267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6833","nonce":4020670267},{"type":"v1","addr":"172.21.15.152:6834","nonce":4020670267}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":109},{"gid":6614,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/352716681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":352716681},{"type":"v1","addr":"172.21.15.175:6833","nonce":352716681}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":110},{"gid":6617,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/2512108447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":2512108447},{"type":"v1","addr":"172.21.15.175:6835","nonce":2512108447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":110},{"gid":10869,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/1105404863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":1105404863},{"type":"v1","addr":"172.21.15.152:6835","nonce":1105404863}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":118}],"filesystems":[]} 2024-04-20T11:38:49.480 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 118 2024-04-20T11:38:49.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:49.483+0000 7fe657200640 1 -- 172.21.15.152:0/440750930 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe63406d430 msgr2=0x7fe63406f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:49.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:49.483+0000 7fe657200640 1 --2- 172.21.15.152:0/440750930 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe63406d430 0x7fe63406f8f0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fe648036000 tx=0x7fe648030000 comp rx=0 tx=0).stop 2024-04-20T11:38:49.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:49.483+0000 7fe657200640 1 -- 172.21.15.152:0/440750930 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe6580b5480 msgr2=0x7fe6581615c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:49.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:49.483+0000 7fe657200640 1 --2- 172.21.15.152:0/440750930 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe6580b5480 0x7fe6581615c0 secure :-1 s=READY pgs=2488 cs=0 l=1 rev1=1 crypto rx=0x7fe64000d800 tx=0x7fe64000dcc0 comp rx=0 tx=0).stop 2024-04-20T11:38:49.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:49.483+0000 7fe657200640 1 -- 172.21.15.152:0/440750930 shutdown_connections 2024-04-20T11:38:49.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:49.483+0000 7fe657200640 1 --2- 172.21.15.152:0/440750930 >> [v2:172.21.15.175:6836/4021250441,v1:172.21.15.175:6837/4021250441] conn(0x7fe63406d430 0x7fe63406f8f0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:49.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:49.483+0000 7fe657200640 1 --2- 172.21.15.152:0/440750930 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe6581603d0 0x7fe658154330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:49.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:49.483+0000 7fe657200640 1 --2- 172.21.15.152:0/440750930 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe65815fcc0 0x7fe6580ca490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:49.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:49.483+0000 7fe657200640 1 --2- 172.21.15.152:0/440750930 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe6580b5480 0x7fe6581615c0 unknown :-1 s=CLOSED pgs=2488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:49.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:49.483+0000 7fe657200640 1 -- 172.21.15.152:0/440750930 >> 172.21.15.152:0/440750930 conn(0x7fe65808bfd0 msgr2=0x7fe65806caf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:49.484 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:49.483+0000 7fe657200640 1 -- 172.21.15.152:0/440750930 shutdown_connections 2024-04-20T11:38:49.484 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-20T11:38:49.483+0000 7fe657200640 1 -- 172.21.15.152:0/440750930 wait complete. 2024-04-20T11:38:49.502 DEBUG:tasks.cephfs.filesystem:already dead... 2024-04-20T11:38:49.502 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-20T11:38:49.502 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-20T11:38:49.503 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T11:38:49.519 INFO:tasks.ceph.mds.a:Stopped 2024-04-20T11:38:49.519 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-20T11:38:49.519 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T11:38:49.584 INFO:tasks.ceph.mds.c:Stopped 2024-04-20T11:38:49.584 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-20T11:38:49.584 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T11:38:49.611 INFO:tasks.ceph.mds.b:Stopped 2024-04-20T11:38:49.611 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-20T11:38:49.611 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T11:38:49.678 INFO:tasks.ceph.mds.d:Stopped 2024-04-20T11:38:49.679 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-20T11:38:49.679 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-20T11:38:49.679 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T11:38:49.735 INFO:tasks.ceph.osd.0:Stopped 2024-04-20T11:38:49.735 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-20T11:38:49.735 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T11:38:49.809 INFO:tasks.ceph.osd.1:Stopped 2024-04-20T11:38:49.810 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-20T11:38:49.810 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T11:38:49.887 INFO:tasks.ceph.osd.2:Stopped 2024-04-20T11:38:49.887 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-04-20T11:38:49.887 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T11:38:49.961 INFO:tasks.ceph.osd.3:Stopped 2024-04-20T11:38:49.962 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-04-20T11:38:49.962 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T11:38:50.034 INFO:tasks.ceph.osd.4:Stopped 2024-04-20T11:38:50.034 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-04-20T11:38:50.034 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T11:38:50.128 INFO:tasks.ceph.osd.5:Stopped 2024-04-20T11:38:50.128 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-04-20T11:38:50.128 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T11:38:50.217 INFO:tasks.ceph.osd.6:Stopped 2024-04-20T11:38:50.217 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-04-20T11:38:50.217 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T11:38:54.406 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:38:54.414 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:38:55.412 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~0s 2024-04-20T11:38:56.319 INFO:tasks.ceph.osd.7:Stopped 2024-04-20T11:38:56.319 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-20T11:38:56.320 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-04-20T11:38:56.320 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T11:39:00.816 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~0s 2024-04-20T11:39:00.816 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 has been restored 2024-04-20T11:39:02.421 INFO:tasks.ceph.mgr.y:Stopped 2024-04-20T11:39:02.421 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-04-20T11:39:02.422 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T11:39:02.511 INFO:tasks.ceph.mgr.x:Stopped 2024-04-20T11:39:02.511 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-20T11:39:02.511 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-20T11:39:02.511 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T11:39:02.566 INFO:tasks.ceph.mon.a:Stopped 2024-04-20T11:39:02.566 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-20T11:39:02.566 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T11:39:02.593 INFO:tasks.ceph.mon.b:Stopped 2024-04-20T11:39:02.594 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-20T11:39:02.594 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T11:39:02.679 INFO:tasks.ceph.mon.c:Stopped 2024-04-20T11:39:02.680 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-20T11:39:02.680 DEBUG:teuthology.orchestra.run.smithi152:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | head -n 1 2024-04-20T11:39:02.788 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi152.front.sepia.ceph.com 2024-04-20T11:39:02.788 DEBUG:teuthology.orchestra.run.smithi152:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-04-20T11:39:03.023 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi152.front.sepia.ceph.com 2024-04-20T11:39:03.024 DEBUG:teuthology.orchestra.run.smithi152:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-04-20T11:39:03.148 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi152.front.sepia.ceph.com 2024-04-20T11:39:03.148 DEBUG:teuthology.orchestra.run.smithi152:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-04-20T11:39:03.253 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi152.front.sepia.ceph.com 2024-04-20T11:39:03.253 DEBUG:teuthology.orchestra.run.smithi152:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-04-20T11:39:03.388 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi175.front.sepia.ceph.com 2024-04-20T11:39:03.388 DEBUG:teuthology.orchestra.run.smithi175:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-04-20T11:39:03.564 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi175.front.sepia.ceph.com 2024-04-20T11:39:03.564 DEBUG:teuthology.orchestra.run.smithi175:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-04-20T11:39:03.692 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi175.front.sepia.ceph.com 2024-04-20T11:39:03.692 DEBUG:teuthology.orchestra.run.smithi175:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-04-20T11:39:03.839 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi175.front.sepia.ceph.com 2024-04-20T11:39:03.839 DEBUG:teuthology.orchestra.run.smithi175:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-04-20T11:39:03.979 INFO:tasks.ceph:Archiving mon data... 2024-04-20T11:39:03.981 DEBUG:teuthology.misc:Transferring archived files from smithi152:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7664528/data/mon.a.tgz 2024-04-20T11:39:03.981 DEBUG:teuthology.orchestra.run.smithi152:> mktemp 2024-04-20T11:39:04.037 INFO:teuthology.orchestra.run.smithi152.stdout:/tmp/tmp.LQ5xmS7yd6 2024-04-20T11:39:04.037 DEBUG:teuthology.orchestra.run.smithi152:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.LQ5xmS7yd6 2024-04-20T11:39:04.341 DEBUG:teuthology.orchestra.run.smithi152:> sudo chmod 0666 /tmp/tmp.LQ5xmS7yd6 2024-04-20T11:39:04.484 DEBUG:teuthology.orchestra.remote:smithi152:/tmp/tmp.LQ5xmS7yd6 is 968KB 2024-04-20T11:39:04.604 DEBUG:teuthology.orchestra.run.smithi152:> rm -fr /tmp/tmp.LQ5xmS7yd6 2024-04-20T11:39:04.662 DEBUG:teuthology.misc:Transferring archived files from smithi175:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7664528/data/mon.b.tgz 2024-04-20T11:39:04.662 DEBUG:teuthology.orchestra.run.smithi175:> mktemp 2024-04-20T11:39:04.719 INFO:teuthology.orchestra.run.smithi175.stdout:/tmp/tmp.1JOFbbCmwx 2024-04-20T11:39:04.719 DEBUG:teuthology.orchestra.run.smithi175:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.1JOFbbCmwx 2024-04-20T11:39:04.996 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod 0666 /tmp/tmp.1JOFbbCmwx 2024-04-20T11:39:05.132 DEBUG:teuthology.orchestra.remote:smithi175:/tmp/tmp.1JOFbbCmwx is 969KB 2024-04-20T11:39:05.246 DEBUG:teuthology.orchestra.run.smithi175:> rm -fr /tmp/tmp.1JOFbbCmwx 2024-04-20T11:39:05.304 DEBUG:teuthology.misc:Transferring archived files from smithi175:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7664528/data/mon.c.tgz 2024-04-20T11:39:05.304 DEBUG:teuthology.orchestra.run.smithi175:> mktemp 2024-04-20T11:39:05.356 INFO:teuthology.orchestra.run.smithi175.stdout:/tmp/tmp.HniN6hGiwF 2024-04-20T11:39:05.356 DEBUG:teuthology.orchestra.run.smithi175:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.HniN6hGiwF 2024-04-20T11:39:05.620 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod 0666 /tmp/tmp.HniN6hGiwF 2024-04-20T11:39:05.742 DEBUG:teuthology.orchestra.remote:smithi175:/tmp/tmp.HniN6hGiwF is 969KB 2024-04-20T11:39:05.817 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y has been restored 2024-04-20T11:39:05.858 DEBUG:teuthology.orchestra.run.smithi175:> rm -fr /tmp/tmp.HniN6hGiwF 2024-04-20T11:39:05.914 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-20T11:39:05.914 DEBUG:teuthology.orchestra.run.smithi152:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-20T11:39:05.922 DEBUG:teuthology.orchestra.run.smithi175:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-20T11:39:06.060 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-20T11:39:06.117 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-20T11:39:06.117 INFO:tasks.ceph:Archiving crash dumps... 2024-04-20T11:39:06.119 DEBUG:teuthology.misc:Transferring archived files from smithi152:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7664528/remote/smithi152/crash 2024-04-20T11:39:06.120 DEBUG:teuthology.orchestra.run.smithi152:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-20T11:39:06.207 DEBUG:teuthology.misc:Transferring archived files from smithi175:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7664528/remote/smithi175/crash 2024-04-20T11:39:06.208 DEBUG:teuthology.orchestra.run.smithi175:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-20T11:39:06.293 INFO:tasks.ceph:Shutting down logrotate 2024-04-20T11:39:06.294 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:39:06.301 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:39:06.429 DEBUG:teuthology.orchestra.run.smithi152:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-20T11:39:06.517 DEBUG:teuthology.orchestra.run.smithi175:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-20T11:39:06.607 INFO:tasks.ceph:Compressing logs... 2024-04-20T11:39:06.607 DEBUG:teuthology.orchestra.run.smithi152:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-20T11:39:06.614 DEBUG:teuthology.orchestra.run.smithi175:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-20T11:39:06.689 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28153.log 2024-04-20T11:39:06.690 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75458.log 2024-04-20T11:39:06.691 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.28153.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35413.log 2024-04-20T11:39:06.691 INFO:teuthology.orchestra.run.smithi152.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.28153.log.gz 2024-04-20T11:39:06.692 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.75458.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72745.log 2024-04-20T11:39:06.692 INFO:teuthology.orchestra.run.smithi152.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.75458.log.gz 2024-04-20T11:39:06.693 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.35413.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40127.log 2024-04-20T11:39:06.693 INFO:teuthology.orchestra.run.smithi152.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35413.log.gz 2024-04-20T11:39:06.694 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.31493.log 2024-04-20T11:39:06.695 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.72745.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36800.log 2024-04-20T11:39:06.695 INFO:teuthology.orchestra.run.smithi152.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.72745.log.gz 2024-04-20T11:39:06.696 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.32167.log 2024-04-20T11:39:06.696 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-client.1.31493.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.37216.log 2024-04-20T11:39:06.697 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.40127.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28632.log 2024-04-20T11:39:06.697 INFO:teuthology.orchestra.run.smithi152.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.40127.log.gz 2024-04-20T11:39:06.697 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.36800.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.64036.log 2024-04-20T11:39:06.697 INFO:teuthology.orchestra.run.smithi152.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.36800.log.gz 2024-04-20T11:39:06.700 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.28632.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66251.log 2024-04-20T11:39:06.700 INFO:teuthology.orchestra.run.smithi152.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.28632.log.gz 2024-04-20T11:39:06.700 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.0.64036.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59751.log 2024-04-20T11:39:06.700 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.66251.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33794.log 2024-04-20T11:39:06.701 INFO:teuthology.orchestra.run.smithi152.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.66251.log.gz 2024-04-20T11:39:06.701 INFO:teuthology.orchestra.run.smithi152.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.0.64036.log.gz 2024-04-20T11:39:06.701 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-client.1.32167.log: 90.5% -- replaced with /var/log/ceph/ceph-client.1.31493.log.gz 2024-04-20T11:39:06.701 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22411.log 2024-04-20T11:39:06.701 INFO:teuthology.orchestra.run.smithi175.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.1.32167.log.gz 2024-04-20T11:39:06.702 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-client.1.37216.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22366.log 2024-04-20T11:39:06.702 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-client.admin.22411.log: 89.3% -- replaced with /var/log/ceph/ceph-client.1.37216.log.gz 2024-04-20T11:39:06.702 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-20T11:39:06.702 INFO:teuthology.orchestra.run.smithi175.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.22411.log.gz 2024-04-20T11:39:06.702 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-client.admin.22366.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.29062.log 2024-04-20T11:39:06.702 INFO:teuthology.orchestra.run.smithi175.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.22366.log.gz 2024-04-20T11:39:06.702 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.33724.log 2024-04-20T11:39:06.704 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-client.3.29062.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.28538.log 2024-04-20T11:39:06.704 INFO:teuthology.orchestra.run.smithi175.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.3.29062.log.gz 2024-04-20T11:39:06.704 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-client.1.33724.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-20T11:39:06.704 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-client.1.28538.log: 92.8% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-20T11:39:06.704 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22531.log 2024-04-20T11:39:06.707 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.59751.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.59751.log.gz 2024-04-20T11:39:06.707 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36299.log 2024-04-20T11:39:06.707 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.33794.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38063.log 2024-04-20T11:39:06.707 INFO:teuthology.orchestra.run.smithi152.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.33794.log.gz 2024-04-20T11:39:06.707 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.36299.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44190.log 2024-04-20T11:39:06.707 INFO:teuthology.orchestra.run.smithi152.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.36299.log.gz 2024-04-20T11:39:06.707 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.38063.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71137.log 2024-04-20T11:39:06.707 INFO:teuthology.orchestra.run.smithi152.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.38063.log.gz 2024-04-20T11:39:06.707 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.44190.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35497.log 2024-04-20T11:39:06.708 INFO:teuthology.orchestra.run.smithi152.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44190.log.gz 2024-04-20T11:39:06.708 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.71137.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58178.log 2024-04-20T11:39:06.708 INFO:teuthology.orchestra.run.smithi152.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.71137.log.gz 2024-04-20T11:39:06.708 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.35497.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62289.log 2024-04-20T11:39:06.708 INFO:teuthology.orchestra.run.smithi152.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.35497.log.gz 2024-04-20T11:39:06.711 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.58178.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50172.log 2024-04-20T11:39:06.711 INFO:teuthology.orchestra.run.smithi152.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.58178.log.gz 2024-04-20T11:39:06.711 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.62289.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64589.log 2024-04-20T11:39:06.711 INFO:teuthology.orchestra.run.smithi152.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.62289.log.gz 2024-04-20T11:39:06.711 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.50172.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53220.log 2024-04-20T11:39:06.712 INFO:teuthology.orchestra.run.smithi152.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50172.log.gz 2024-04-20T11:39:06.712 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.64589.log: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.64589.log.gz 2024-04-20T11:39:06.712 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50862.log 2024-04-20T11:39:06.713 INFO:teuthology.orchestra.run.smithi175.stderr: 92.3% -- replaced with /var/log/ceph/ceph-client.1.33724.log.gz 2024-04-20T11:39:06.713 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph.audit.log: 90.7% -- replaced with /var/log/ceph/ceph-client.1.28538.log.gz 2024-04-20T11:39:06.713 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-04-20T11:39:06.713 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-client.admin.22531.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.22531.log.gz 2024-04-20T11:39:06.713 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-04-20T11:39:06.713 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.36532.log 2024-04-20T11:39:06.713 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.29798.log 2024-04-20T11:39:06.713 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-client.1.36532.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-04-20T11:39:06.713 INFO:teuthology.orchestra.run.smithi175.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.1.36532.log.gz 2024-04-20T11:39:06.713 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-client.1.29798.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-04-20T11:39:06.713 INFO:teuthology.orchestra.run.smithi175.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.1.29798.log.gz 2024-04-20T11:39:06.713 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22571.log 2024-04-20T11:39:06.713 INFO:teuthology.orchestra.run.smithi175.stderr: 93.6% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-20T11:39:06.714 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-osd.5.log: /var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22280.log 2024-04-20T11:39:06.715 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-client.admin.22571.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.30460.log 2024-04-20T11:39:06.715 INFO:teuthology.orchestra.run.smithi175.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.22571.log.gz 2024-04-20T11:39:06.716 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-client.admin.22280.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.22280.log.gz 2024-04-20T11:39:06.716 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-04-20T11:39:06.718 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.53220.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69174.log 2024-04-20T11:39:06.718 INFO:teuthology.orchestra.run.smithi152.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.53220.log.gz 2024-04-20T11:39:06.718 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.50862.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45722.log 2024-04-20T11:39:06.718 INFO:teuthology.orchestra.run.smithi152.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50862.log.gz 2024-04-20T11:39:06.718 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.69174.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40411.log 2024-04-20T11:39:06.718 INFO:teuthology.orchestra.run.smithi152.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.69174.log.gz 2024-04-20T11:39:06.718 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.45722.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35921.log 2024-04-20T11:39:06.718 INFO:teuthology.orchestra.run.smithi152.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45722.log.gz 2024-04-20T11:39:06.718 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.40411.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.47498.log 2024-04-20T11:39:06.719 INFO:teuthology.orchestra.run.smithi152.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.40411.log.gz 2024-04-20T11:39:06.719 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.35921.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57092.log 2024-04-20T11:39:06.719 INFO:teuthology.orchestra.run.smithi152.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.35921.log.gz 2024-04-20T11:39:06.719 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.0.47498.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69543.log 2024-04-20T11:39:06.720 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-client.1.30460.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-04-20T11:39:06.721 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.34437.log 2024-04-20T11:39:06.721 INFO:teuthology.orchestra.run.smithi175.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.1.30460.log.gz 2024-04-20T11:39:06.721 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-04-20T11:39:06.722 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.57092.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67329.log 2024-04-20T11:39:06.722 INFO:teuthology.orchestra.run.smithi152.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.0.47498.log.gz 2024-04-20T11:39:06.722 INFO:teuthology.orchestra.run.smithi152.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.57092.log.gz 2024-04-20T11:39:06.722 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.69543.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49664.log 2024-04-20T11:39:06.722 INFO:teuthology.orchestra.run.smithi152.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.69543.log.gz 2024-04-20T11:39:06.723 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.67329.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40885.log 2024-04-20T11:39:06.723 INFO:teuthology.orchestra.run.smithi152.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.67329.log.gz 2024-04-20T11:39:06.723 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.49664.log: gzip 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49664.log.gz -5 2024-04-20T11:39:06.723 INFO:teuthology.orchestra.run.smithi152.stderr: --verbose -- /var/log/ceph/ceph-client.admin.42281.log 2024-04-20T11:39:06.723 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-client.1.34437.log: /var/log/ceph/ceph-osd.6.log: 89.9% -- replaced with /var/log/ceph/ceph-client.1.34437.log.gz 2024-04-20T11:39:06.724 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22320.log 2024-04-20T11:39:06.724 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.40885.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.40885.log.gz 2024-04-20T11:39:06.724 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50522.log 2024-04-20T11:39:06.724 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.42281.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68644.log 2024-04-20T11:39:06.724 INFO:teuthology.orchestra.run.smithi152.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42281.log.gz 2024-04-20T11:39:06.725 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.50522.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50522.log.gzgzip 2024-04-20T11:39:06.725 INFO:teuthology.orchestra.run.smithi152.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.68928.log 2024-04-20T11:39:06.726 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-client.admin.22320.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22451.log 2024-04-20T11:39:06.727 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.68644.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69300.log 2024-04-20T11:39:06.727 INFO:teuthology.orchestra.run.smithi152.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.68644.log.gz 2024-04-20T11:39:06.728 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.68928.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74466.log 2024-04-20T11:39:06.728 INFO:teuthology.orchestra.run.smithi152.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.68928.log.gz 2024-04-20T11:39:06.729 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-client.admin.22451.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.22320.log.gzgzip 2024-04-20T11:39:06.729 INFO:teuthology.orchestra.run.smithi175.stderr: -5 --verbose -- /var/log/ceph/ceph-client.1.35123.log 2024-04-20T11:39:06.730 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.69300.log: gzip -5 --verbose -- 88.4% /var/log/ceph/ceph-client.admin.36233.log -- replaced with /var/log/ceph/ceph-client.admin.69300.log.gz 2024-04-20T11:39:06.730 INFO:teuthology.orchestra.run.smithi152.stderr: 2024-04-20T11:39:06.730 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.74466.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33894.log 2024-04-20T11:39:06.730 INFO:teuthology.orchestra.run.smithi152.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.74466.log.gz 2024-04-20T11:39:06.731 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.36233.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38189.log 2024-04-20T11:39:06.731 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.33894.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.36233.log.gz 2024-04-20T11:39:06.732 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.33043.log 2024-04-20T11:39:06.733 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41903.log 2024-04-20T11:39:06.733 INFO:teuthology.orchestra.run.smithi152.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.33894.log.gz 2024-04-20T11:39:06.733 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.38189.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49496.log 2024-04-20T11:39:06.733 INFO:teuthology.orchestra.run.smithi152.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.38189.log.gz 2024-04-20T11:39:06.734 INFO:teuthology.orchestra.run.smithi175.stderr: 88.8%/var/log/ceph/ceph-client.1.35123.log: -- replaced with /var/log/ceph/ceph-client.admin.22451.log.gz 2024-04-20T11:39:06.734 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.41903.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29496.log 2024-04-20T11:39:06.734 INFO:teuthology.orchestra.run.smithi152.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41903.log.gz 2024-04-20T11:39:06.735 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.49496.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49496.log.gz 2024-04-20T11:39:06.736 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40717.log 2024-04-20T11:39:06.736 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-client.1.33043.log: 90.9% -- replaced with /var/log/ceph/ceph-client.1.35123.log.gz 2024-04-20T11:39:06.736 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.29496.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36107.log 2024-04-20T11:39:06.736 INFO:teuthology.orchestra.run.smithi152.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.29496.log.gz 2024-04-20T11:39:06.737 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.40717.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40717.log.gz 2024-04-20T11:39:06.737 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46944.log 2024-04-20T11:39:06.738 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.36107.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51573.log 2024-04-20T11:39:06.738 INFO:teuthology.orchestra.run.smithi152.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.36107.log.gz 2024-04-20T11:39:06.739 INFO:teuthology.orchestra.run.smithi175.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.1.33043.log.gz 2024-04-20T11:39:06.739 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-04-20T11:39:06.739 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.46944.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53484.log 2024-04-20T11:39:06.739 INFO:teuthology.orchestra.run.smithi152.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46944.log.gz 2024-04-20T11:39:06.740 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-04-20T11:39:06.740 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.51573.log: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.51573.log.gz 2024-04-20T11:39:06.741 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39476.log 2024-04-20T11:39:06.741 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.35847.log 2024-04-20T11:39:06.742 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.53484.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75537.log 2024-04-20T11:39:06.742 INFO:teuthology.orchestra.run.smithi152.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.53484.log.gz 2024-04-20T11:39:06.743 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.39476.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32726.log 2024-04-20T11:39:06.743 INFO:teuthology.orchestra.run.smithi152.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.39476.log.gz 2024-04-20T11:39:06.744 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22491.log 2024-04-20T11:39:06.744 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.75537.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.75537.log.gz 2024-04-20T11:39:06.744 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70755.log 2024-04-20T11:39:06.745 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.32726.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.32726.log.gz 2024-04-20T11:39:06.745 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59104.log 2024-04-20T11:39:06.747 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.70755.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69132.log 2024-04-20T11:39:06.747 INFO:teuthology.orchestra.run.smithi152.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.70755.log.gz 2024-04-20T11:39:06.747 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.59104.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.59104.log.gz 2024-04-20T11:39:06.747 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.71523.log 2024-04-20T11:39:06.748 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-client.1.35847.log: /var/log/ceph/ceph-client.admin.22491.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.22491.log.gz 2024-04-20T11:39:06.749 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.69132.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50736.log 2024-04-20T11:39:06.749 INFO:teuthology.orchestra.run.smithi152.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.69132.log.gz 2024-04-20T11:39:06.750 INFO:teuthology.orchestra.run.smithi175.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.1.35847.log.gz 2024-04-20T11:39:06.750 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.0.71523.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42617.log 2024-04-20T11:39:06.750 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.50736.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72670.log 2024-04-20T11:39:06.751 INFO:teuthology.orchestra.run.smithi152.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.50736.log.gz 2024-04-20T11:39:06.751 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.42617.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36958.log 2024-04-20T11:39:06.751 INFO:teuthology.orchestra.run.smithi152.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42617.log.gz 2024-04-20T11:39:06.752 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.72670.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41011.log 2024-04-20T11:39:06.753 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.36958.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.72670.log.gz 2024-04-20T11:39:06.753 INFO:teuthology.orchestra.run.smithi152.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.36958.log.gz 2024-04-20T11:39:06.754 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68807.log 2024-04-20T11:39:06.754 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.41011.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35455.log 2024-04-20T11:39:06.755 INFO:teuthology.orchestra.run.smithi152.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41011.log.gz 2024-04-20T11:39:06.756 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.68807.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74424.log 2024-04-20T11:39:06.756 INFO:teuthology.orchestra.run.smithi152.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.68807.log.gz 2024-04-20T11:39:06.757 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.35455.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67245.log 2024-04-20T11:39:06.757 INFO:teuthology.orchestra.run.smithi152.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.35455.log.gz 2024-04-20T11:39:06.758 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.74424.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74210.log 2024-04-20T11:39:06.758 INFO:teuthology.orchestra.run.smithi152.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.74424.log.gz 2024-04-20T11:39:06.759 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.67245.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65612.log 2024-04-20T11:39:06.759 INFO:teuthology.orchestra.run.smithi152.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.67245.log.gz 2024-04-20T11:39:06.759 INFO:teuthology.orchestra.run.smithi152.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.0.71523.log.gz 2024-04-20T11:39:06.760 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.74210.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42197.log 2024-04-20T11:39:06.760 INFO:teuthology.orchestra.run.smithi152.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.74210.log.gz 2024-04-20T11:39:06.761 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.65612.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37652.log 2024-04-20T11:39:06.761 INFO:teuthology.orchestra.run.smithi152.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.65612.log.gz 2024-04-20T11:39:06.762 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.42197.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32085.log 2024-04-20T11:39:06.762 INFO:teuthology.orchestra.run.smithi152.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42197.log.gz 2024-04-20T11:39:06.763 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.37652.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41732.log 2024-04-20T11:39:06.763 INFO:teuthology.orchestra.run.smithi152.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.37652.log.gz 2024-04-20T11:39:06.764 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.32085.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47220.log 2024-04-20T11:39:06.764 INFO:teuthology.orchestra.run.smithi152.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.32085.log.gz 2024-04-20T11:39:06.765 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.41732.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70448.log 2024-04-20T11:39:06.765 INFO:teuthology.orchestra.run.smithi152.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41732.log.gz 2024-04-20T11:39:06.765 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.47220.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35539.log 2024-04-20T11:39:06.766 INFO:teuthology.orchestra.run.smithi152.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47220.log.gz 2024-04-20T11:39:06.767 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.70448.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37406.log 2024-04-20T11:39:06.767 INFO:teuthology.orchestra.run.smithi152.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.70448.log.gz 2024-04-20T11:39:06.768 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.35539.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33534.log 2024-04-20T11:39:06.768 INFO:teuthology.orchestra.run.smithi152.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.35539.log.gz 2024-04-20T11:39:06.769 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.37406.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44939.log 2024-04-20T11:39:06.769 INFO:teuthology.orchestra.run.smithi152.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.37406.log.gz 2024-04-20T11:39:06.770 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.33534.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29231.log 2024-04-20T11:39:06.770 INFO:teuthology.orchestra.run.smithi152.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.33534.log.gz 2024-04-20T11:39:06.771 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.44939.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59987.log 2024-04-20T11:39:06.771 INFO:teuthology.orchestra.run.smithi152.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.44939.log.gz 2024-04-20T11:39:06.772 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.29231.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35709.log 2024-04-20T11:39:06.772 INFO:teuthology.orchestra.run.smithi152.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.29231.log.gz 2024-04-20T11:39:06.773 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.59987.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.59987.log.gz 2024-04-20T11:39:06.773 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52308.log 2024-04-20T11:39:06.774 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.35709.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44274.log 2024-04-20T11:39:06.774 INFO:teuthology.orchestra.run.smithi152.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.35709.log.gz 2024-04-20T11:39:06.775 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.52308.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30498.log 2024-04-20T11:39:06.775 INFO:teuthology.orchestra.run.smithi152.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.52308.log.gz 2024-04-20T11:39:06.776 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.44274.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63027.log 2024-04-20T11:39:06.776 INFO:teuthology.orchestra.run.smithi152.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44274.log.gz 2024-04-20T11:39:06.777 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.30498.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62121.log 2024-04-20T11:39:06.777 INFO:teuthology.orchestra.run.smithi152.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.30498.log.gz 2024-04-20T11:39:06.778 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.63027.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41317.log 2024-04-20T11:39:06.778 INFO:teuthology.orchestra.run.smithi152.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.63027.log.gz 2024-04-20T11:39:06.779 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.62121.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32589.log 2024-04-20T11:39:06.779 INFO:teuthology.orchestra.run.smithi152.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.62121.log.gz 2024-04-20T11:39:06.780 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.41317.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41317.log.gz 2024-04-20T11:39:06.780 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68376.log 2024-04-20T11:39:06.781 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.32589.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37727.log 2024-04-20T11:39:06.782 INFO:teuthology.orchestra.run.smithi152.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.32589.log.gz 2024-04-20T11:39:06.782 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.68376.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.74912.log 2024-04-20T11:39:06.782 INFO:teuthology.orchestra.run.smithi152.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.68376.log.gz 2024-04-20T11:39:06.783 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.37727.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34098.log 2024-04-20T11:39:06.784 INFO:teuthology.orchestra.run.smithi152.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.37727.log.gz 2024-04-20T11:39:06.784 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.0.74912.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59227.log 2024-04-20T11:39:06.785 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.34098.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69375.log 2024-04-20T11:39:06.786 INFO:teuthology.orchestra.run.smithi152.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34098.log.gz 2024-04-20T11:39:06.786 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.59227.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58677.log 2024-04-20T11:39:06.787 INFO:teuthology.orchestra.run.smithi152.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.59227.log.gz 2024-04-20T11:39:06.787 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.69375.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.31450.log 2024-04-20T11:39:06.788 INFO:teuthology.orchestra.run.smithi152.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.69375.log.gz 2024-04-20T11:39:06.789 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.58677.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.58677.log.gz 2024-04-20T11:39:06.789 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33408.log 2024-04-20T11:39:06.790 INFO:teuthology.orchestra.run.smithi152.stderr: 92.2% -- replaced with /var/log/ceph/ceph-client.0.74912.log.gz 2024-04-20T11:39:06.790 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.2.31450.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66125.log 2024-04-20T11:39:06.791 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.33408.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68747.log 2024-04-20T11:39:06.791 INFO:teuthology.orchestra.run.smithi152.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.33408.log.gz 2024-04-20T11:39:06.791 INFO:teuthology.orchestra.run.smithi152.stderr: 91.9% -- replaced with /var/log/ceph/ceph-client.2.31450.log.gz 2024-04-20T11:39:06.792 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.66125.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28897.log 2024-04-20T11:39:06.792 INFO:teuthology.orchestra.run.smithi152.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.66125.log.gz 2024-04-20T11:39:06.793 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.68747.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59625.log 2024-04-20T11:39:06.793 INFO:teuthology.orchestra.run.smithi152.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.68747.log.gz 2024-04-20T11:39:06.794 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.28897.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41861.log 2024-04-20T11:39:06.794 INFO:teuthology.orchestra.run.smithi152.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.28897.log.gz 2024-04-20T11:39:06.795 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.59625.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32379.log 2024-04-20T11:39:06.795 INFO:teuthology.orchestra.run.smithi152.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.59625.log.gz 2024-04-20T11:39:06.796 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.41861.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61138.log 2024-04-20T11:39:06.796 INFO:teuthology.orchestra.run.smithi152.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.41861.log.gz 2024-04-20T11:39:06.797 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.32379.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40353.log 2024-04-20T11:39:06.797 INFO:teuthology.orchestra.run.smithi152.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.32379.log.gz 2024-04-20T11:39:06.798 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.61138.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63758.log 2024-04-20T11:39:06.798 INFO:teuthology.orchestra.run.smithi152.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.61138.log.gz 2024-04-20T11:39:06.799 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.40353.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70131.log 2024-04-20T11:39:06.799 INFO:teuthology.orchestra.run.smithi152.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.40353.log.gz 2024-04-20T11:39:06.800 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.63758.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42772.log 2024-04-20T11:39:06.800 INFO:teuthology.orchestra.run.smithi152.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.63758.log.gz 2024-04-20T11:39:06.801 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.70131.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-20T11:39:06.801 INFO:teuthology.orchestra.run.smithi152.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.70131.log.gz 2024-04-20T11:39:06.801 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.42772.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72628.log 2024-04-20T11:39:06.802 INFO:teuthology.orchestra.run.smithi152.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42772.log.gz 2024-04-20T11:39:06.802 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58635.log 2024-04-20T11:39:06.804 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.72628.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77688.log 2024-04-20T11:39:06.804 INFO:teuthology.orchestra.run.smithi152.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.72628.log.gz 2024-04-20T11:39:06.805 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.58635.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61794.log 2024-04-20T11:39:06.805 INFO:teuthology.orchestra.run.smithi152.stderr: 87.0% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-20T11:39:06.805 INFO:teuthology.orchestra.run.smithi152.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.58635.log.gz 2024-04-20T11:39:06.806 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.77688.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31959.log 2024-04-20T11:39:06.806 INFO:teuthology.orchestra.run.smithi152.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.77688.log.gz 2024-04-20T11:39:06.807 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.61794.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.61794.log.gz 2024-04-20T11:39:06.807 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32127.log 2024-04-20T11:39:06.807 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.31959.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36716.log 2024-04-20T11:39:06.808 INFO:teuthology.orchestra.run.smithi152.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.31959.log.gz 2024-04-20T11:39:06.809 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.32127.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53125.log 2024-04-20T11:39:06.809 INFO:teuthology.orchestra.run.smithi152.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.32127.log.gz 2024-04-20T11:39:06.810 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.36716.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.36716.log.gz 2024-04-20T11:39:06.810 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71095.log 2024-04-20T11:39:06.811 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.53125.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49538.log 2024-04-20T11:39:06.811 INFO:teuthology.orchestra.run.smithi152.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53125.log.gz 2024-04-20T11:39:06.812 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.71095.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66564.log 2024-04-20T11:39:06.812 INFO:teuthology.orchestra.run.smithi152.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.71095.log.gz 2024-04-20T11:39:06.813 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.49538.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64466.log 2024-04-20T11:39:06.813 INFO:teuthology.orchestra.run.smithi152.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.49538.log.gz 2024-04-20T11:39:06.814 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.66564.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45302.log 2024-04-20T11:39:06.814 INFO:teuthology.orchestra.run.smithi152.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.66564.log.gz 2024-04-20T11:39:06.816 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.64466.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44981.log 2024-04-20T11:39:06.816 INFO:teuthology.orchestra.run.smithi152.stderr: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.64466.log.gz 2024-04-20T11:39:06.816 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.45302.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69879.log 2024-04-20T11:39:06.816 INFO:teuthology.orchestra.run.smithi152.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45302.log.gz 2024-04-20T11:39:06.817 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.44981.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75825.log 2024-04-20T11:39:06.818 INFO:teuthology.orchestra.run.smithi152.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.44981.log.gz 2024-04-20T11:39:06.819 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.69879.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64259.log 2024-04-20T11:39:06.819 INFO:teuthology.orchestra.run.smithi152.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.69879.log.gz 2024-04-20T11:39:06.820 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.75825.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57424.log 2024-04-20T11:39:06.820 INFO:teuthology.orchestra.run.smithi152.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.75825.log.gz 2024-04-20T11:39:06.821 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.64259.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44650.log 2024-04-20T11:39:06.821 INFO:teuthology.orchestra.run.smithi152.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.64259.log.gz 2024-04-20T11:39:06.821 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.57424.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35163.log 2024-04-20T11:39:06.822 INFO:teuthology.orchestra.run.smithi152.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.57424.log.gz 2024-04-20T11:39:06.822 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.44650.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55128.log 2024-04-20T11:39:06.823 INFO:teuthology.orchestra.run.smithi152.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44650.log.gz 2024-04-20T11:39:06.824 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.35163.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49580.log 2024-04-20T11:39:06.824 INFO:teuthology.orchestra.run.smithi152.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.35163.log.gz 2024-04-20T11:39:06.824 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.55128.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29233.log 2024-04-20T11:39:06.825 INFO:teuthology.orchestra.run.smithi152.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.55128.log.gz 2024-04-20T11:39:06.825 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.49580.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42113.log 2024-04-20T11:39:06.826 INFO:teuthology.orchestra.run.smithi152.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49580.log.gz 2024-04-20T11:39:06.827 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.29233.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28338.log 2024-04-20T11:39:06.827 INFO:teuthology.orchestra.run.smithi152.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.29233.log.gz 2024-04-20T11:39:06.828 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.42113.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48173.log 2024-04-20T11:39:06.828 INFO:teuthology.orchestra.run.smithi152.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42113.log.gz 2024-04-20T11:39:06.829 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.28338.log: gzip 89.6% -5 -- replaced with /var/log/ceph/ceph-client.admin.28338.log.gz 2024-04-20T11:39:06.829 INFO:teuthology.orchestra.run.smithi152.stderr: --verbose -- /var/log/ceph/ceph-client.admin.50341.log 2024-04-20T11:39:06.830 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.48173.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48173.log.gz 2024-04-20T11:39:06.831 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46375.log 2024-04-20T11:39:06.833 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.50341.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35581.log 2024-04-20T11:39:06.833 INFO:teuthology.orchestra.run.smithi152.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.50341.log.gz 2024-04-20T11:39:06.833 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.46375.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71179.log 2024-04-20T11:39:06.833 INFO:teuthology.orchestra.run.smithi152.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.46375.log.gz 2024-04-20T11:39:06.835 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.35581.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.35581.log.gz 2024-04-20T11:39:06.835 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63292.log 2024-04-20T11:39:06.835 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.71179.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34182.log 2024-04-20T11:39:06.837 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.63292.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.71179.log.gz 2024-04-20T11:39:06.837 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40269.log 2024-04-20T11:39:06.837 INFO:teuthology.orchestra.run.smithi152.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.63292.log.gz 2024-04-20T11:39:06.837 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.34182.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34182.log.gz 2024-04-20T11:39:06.837 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30102.log 2024-04-20T11:39:06.840 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.40269.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36065.log 2024-04-20T11:39:06.840 INFO:teuthology.orchestra.run.smithi152.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.40269.log.gz 2024-04-20T11:39:06.840 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.30102.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48755.log 2024-04-20T11:39:06.840 INFO:teuthology.orchestra.run.smithi152.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.30102.log.gz 2024-04-20T11:39:06.840 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.36065.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.36065.log.gz 2024-04-20T11:39:06.841 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41095.log 2024-04-20T11:39:06.843 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.48755.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33366.log 2024-04-20T11:39:06.843 INFO:teuthology.orchestra.run.smithi152.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48755.log.gz 2024-04-20T11:39:06.844 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.41095.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32852.log 2024-04-20T11:39:06.844 INFO:teuthology.orchestra.run.smithi152.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.41095.log.gz 2024-04-20T11:39:06.844 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.33366.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65897.log 2024-04-20T11:39:06.844 INFO:teuthology.orchestra.run.smithi152.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.33366.log.gz 2024-04-20T11:39:06.846 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.32852.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64425.log 2024-04-20T11:39:06.847 INFO:teuthology.orchestra.run.smithi152.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.32852.log.gz 2024-04-20T11:39:06.847 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.65897.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70797.log 2024-04-20T11:39:06.847 INFO:teuthology.orchestra.run.smithi152.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.65897.log.gz 2024-04-20T11:39:06.847 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.64425.log: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.64425.log.gz 2024-04-20T11:39:06.847 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52476.log 2024-04-20T11:39:06.849 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.70797.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-04-20T11:39:06.849 INFO:teuthology.orchestra.run.smithi152.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.70797.log.gz 2024-04-20T11:39:06.849 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.52476.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31597.log 2024-04-20T11:39:06.849 INFO:teuthology.orchestra.run.smithi152.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.52476.log.gz 2024-04-20T11:39:06.850 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35041.log 2024-04-20T11:39:06.851 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.31597.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.31597.log.gz 2024-04-20T11:39:06.851 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47637.log 2024-04-20T11:39:06.851 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.35041.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29462.log 2024-04-20T11:39:06.853 INFO:teuthology.orchestra.run.smithi152.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.35041.log.gz 2024-04-20T11:39:06.853 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.47637.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66989.log 2024-04-20T11:39:06.853 INFO:teuthology.orchestra.run.smithi152.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47637.log.gz 2024-04-20T11:39:06.854 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.29462.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35083.log 2024-04-20T11:39:06.854 INFO:teuthology.orchestra.run.smithi152.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.29462.log.gz 2024-04-20T11:39:06.855 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.66989.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44813.log 2024-04-20T11:39:06.855 INFO:teuthology.orchestra.run.smithi152.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.66989.log.gz 2024-04-20T11:39:06.855 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.35083.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28896.log 2024-04-20T11:39:06.856 INFO:teuthology.orchestra.run.smithi152.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.35083.log.gz 2024-04-20T11:39:06.857 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.44813.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57176.log 2024-04-20T11:39:06.857 INFO:teuthology.orchestra.run.smithi152.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44813.log.gz 2024-04-20T11:39:06.858 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.28896.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67203.log 2024-04-20T11:39:06.858 INFO:teuthology.orchestra.run.smithi152.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.28896.log.gz 2024-04-20T11:39:06.863 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.57176.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72256.log 2024-04-20T11:39:06.863 INFO:teuthology.orchestra.run.smithi152.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.57176.log.gz 2024-04-20T11:39:06.863 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.67203.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35371.log 2024-04-20T11:39:06.863 INFO:teuthology.orchestra.run.smithi152.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.67203.log.gz 2024-04-20T11:39:06.864 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.72256.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52873.log 2024-04-20T11:39:06.864 INFO:teuthology.orchestra.run.smithi152.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.72256.log.gz 2024-04-20T11:39:06.864 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.35371.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69417.log 2024-04-20T11:39:06.864 INFO:teuthology.orchestra.run.smithi152.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.35371.log.gz 2024-04-20T11:39:06.864 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.52873.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40001.log 2024-04-20T11:39:06.864 INFO:teuthology.orchestra.run.smithi152.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.52873.log.gz 2024-04-20T11:39:06.865 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.69417.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32547.log 2024-04-20T11:39:06.865 INFO:teuthology.orchestra.run.smithi152.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.69417.log.gz 2024-04-20T11:39:06.865 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.40001.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64301.log 2024-04-20T11:39:06.866 INFO:teuthology.orchestra.run.smithi152.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.40001.log.gz 2024-04-20T11:39:06.866 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.32547.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73081.log 2024-04-20T11:39:06.866 INFO:teuthology.orchestra.run.smithi152.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.32547.log.gz 2024-04-20T11:39:06.867 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.64301.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.30923.log 2024-04-20T11:39:06.867 INFO:teuthology.orchestra.run.smithi152.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.64301.log.gz 2024-04-20T11:39:06.868 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.73081.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77730.log 2024-04-20T11:39:06.869 INFO:teuthology.orchestra.run.smithi152.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.73081.log.gz 2024-04-20T11:39:06.869 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.0.30923.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53568.log 2024-04-20T11:39:06.870 INFO:teuthology.orchestra.run.smithi152.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.0.30923.log.gz 2024-04-20T11:39:06.871 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.77730.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47112.log 2024-04-20T11:39:06.871 INFO:teuthology.orchestra.run.smithi152.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.77730.log.gz 2024-04-20T11:39:06.871 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.53568.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55086.log 2024-04-20T11:39:06.872 INFO:teuthology.orchestra.run.smithi152.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.53568.log.gz 2024-04-20T11:39:06.872 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.47112.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60488.log 2024-04-20T11:39:06.872 INFO:teuthology.orchestra.run.smithi152.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47112.log.gz 2024-04-20T11:39:06.874 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.55086.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34854.log 2024-04-20T11:39:06.874 INFO:teuthology.orchestra.run.smithi152.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.55086.log.gz 2024-04-20T11:39:06.874 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.60488.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73352.log 2024-04-20T11:39:06.875 INFO:teuthology.orchestra.run.smithi152.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.60488.log.gz 2024-04-20T11:39:06.875 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.34854.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68499.log 2024-04-20T11:39:06.876 INFO:teuthology.orchestra.run.smithi152.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.34854.log.gz 2024-04-20T11:39:06.877 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.73352.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61953.log 2024-04-20T11:39:06.877 INFO:teuthology.orchestra.run.smithi152.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.73352.log.gz 2024-04-20T11:39:06.878 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.68499.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77478.log 2024-04-20T11:39:06.878 INFO:teuthology.orchestra.run.smithi152.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.68499.log.gz 2024-04-20T11:39:06.879 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.61953.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43283.log 2024-04-20T11:39:06.879 INFO:teuthology.orchestra.run.smithi152.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.61953.log.gz 2024-04-20T11:39:06.880 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.77478.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66823.log 2024-04-20T11:39:06.880 INFO:teuthology.orchestra.run.smithi152.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.77478.log.gz 2024-04-20T11:39:06.881 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.43283.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43283.log.gz 2024-04-20T11:39:06.881 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48131.log 2024-04-20T11:39:06.882 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.66823.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73903.log 2024-04-20T11:39:06.882 INFO:teuthology.orchestra.run.smithi152.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.66823.log.gz 2024-04-20T11:39:06.883 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.48131.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70885.log 2024-04-20T11:39:06.884 INFO:teuthology.orchestra.run.smithi152.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.48131.log.gz 2024-04-20T11:39:06.885 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.73903.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39518.log 2024-04-20T11:39:06.885 INFO:teuthology.orchestra.run.smithi152.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.73903.log.gz 2024-04-20T11:39:06.886 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.70885.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.70885.log.gz 2024-04-20T11:39:06.886 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73165.log 2024-04-20T11:39:06.889 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.39518.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31749.log 2024-04-20T11:39:06.889 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.73165.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61911.log 2024-04-20T11:39:06.889 INFO:teuthology.orchestra.run.smithi152.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.39518.log.gz 2024-04-20T11:39:06.889 INFO:teuthology.orchestra.run.smithi152.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.73165.log.gz 2024-04-20T11:39:06.890 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.31749.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62163.log 2024-04-20T11:39:06.890 INFO:teuthology.orchestra.run.smithi152.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.31749.log.gz 2024-04-20T11:39:06.892 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.61911.log: gzip 88.7% -- replaced with /var/log/ceph/ceph-client.admin.61911.log.gz -5 2024-04-20T11:39:06.892 INFO:teuthology.orchestra.run.smithi152.stderr: --verbose -- /var/log/ceph/ceph-client.admin.45867.log 2024-04-20T11:39:06.892 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.62163.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62349.log 2024-04-20T11:39:06.892 INFO:teuthology.orchestra.run.smithi152.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.62163.log.gz 2024-04-20T11:39:06.893 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.45867.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49370.log 2024-04-20T11:39:06.893 INFO:teuthology.orchestra.run.smithi152.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45867.log.gz 2024-04-20T11:39:06.895 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.62349.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65122.log 2024-04-20T11:39:06.895 INFO:teuthology.orchestra.run.smithi152.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.62349.log.gz 2024-04-20T11:39:06.895 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.49370.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.49370.log.gz 2024-04-20T11:39:06.895 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44566.log 2024-04-20T11:39:06.895 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.65122.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70574.log 2024-04-20T11:39:06.895 INFO:teuthology.orchestra.run.smithi152.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.65122.log.gz 2024-04-20T11:39:06.897 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.44566.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61626.log 2024-04-20T11:39:06.897 INFO:teuthology.orchestra.run.smithi152.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44566.log.gz 2024-04-20T11:39:06.898 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.70574.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58346.log 2024-04-20T11:39:06.898 INFO:teuthology.orchestra.run.smithi152.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.70574.log.gz 2024-04-20T11:39:06.899 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.61626.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75293.log 2024-04-20T11:39:06.900 INFO:teuthology.orchestra.run.smithi152.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.61626.log.gz 2024-04-20T11:39:06.900 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.58346.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43455.log 2024-04-20T11:39:06.900 INFO:teuthology.orchestra.run.smithi152.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.58346.log.gz 2024-04-20T11:39:06.901 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.75293.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44716.log 2024-04-20T11:39:06.902 INFO:teuthology.orchestra.run.smithi152.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.75293.log.gz 2024-04-20T11:39:06.902 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.43455.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45951.log 2024-04-20T11:39:06.902 INFO:teuthology.orchestra.run.smithi152.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43455.log.gz 2024-04-20T11:39:06.903 INFO:teuthology.orchestra.run.smithi175.stderr: 91.9% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-04-20T11:39:06.906 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.44716.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46119.log 2024-04-20T11:39:06.906 INFO:teuthology.orchestra.run.smithi152.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44716.log.gz 2024-04-20T11:39:06.906 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.45951.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67287.log 2024-04-20T11:39:06.906 INFO:teuthology.orchestra.run.smithi152.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45951.log.gz 2024-04-20T11:39:06.907 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.46119.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69627.log 2024-04-20T11:39:06.907 INFO:teuthology.orchestra.run.smithi152.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46119.log.gz 2024-04-20T11:39:06.907 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.67287.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65696.log 2024-04-20T11:39:06.907 INFO:teuthology.orchestra.run.smithi152.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.67287.log.gz 2024-04-20T11:39:06.909 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.69627.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-20T11:39:06.909 INFO:teuthology.orchestra.run.smithi152.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.69627.log.gz 2024-04-20T11:39:06.910 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.65696.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29555.log 2024-04-20T11:39:06.910 INFO:teuthology.orchestra.run.smithi152.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.65696.log.gz 2024-04-20T11:39:06.910 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77436.log 2024-04-20T11:39:06.912 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.29555.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62434.log 2024-04-20T11:39:06.913 INFO:teuthology.orchestra.run.smithi152.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.29555.log.gz 2024-04-20T11:39:06.913 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.77436.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.77436.log.gz 2024-04-20T11:39:06.913 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62079.log 2024-04-20T11:39:06.913 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.62434.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64507.log 2024-04-20T11:39:06.913 INFO:teuthology.orchestra.run.smithi152.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.62434.log.gz 2024-04-20T11:39:06.915 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.62079.log: 89.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-20T11:39:06.915 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose 87.8% -- -- replaced with /var/log/ceph/ceph-client.admin.62079.log.gz /var/log/ceph/ceph-client.admin.72829.log 2024-04-20T11:39:06.915 INFO:teuthology.orchestra.run.smithi152.stderr: 2024-04-20T11:39:06.915 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.64507.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37937.log 2024-04-20T11:39:06.916 INFO:teuthology.orchestra.run.smithi152.stderr: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.64507.log.gz 2024-04-20T11:39:06.916 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.72829.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66167.log 2024-04-20T11:39:06.916 INFO:teuthology.orchestra.run.smithi152.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.72829.log.gz 2024-04-20T11:39:06.917 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.37937.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42239.log 2024-04-20T11:39:06.917 INFO:teuthology.orchestra.run.smithi152.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.37937.log.gz 2024-04-20T11:39:06.918 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.66167.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77394.log 2024-04-20T11:39:06.918 INFO:teuthology.orchestra.run.smithi152.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.66167.log.gz 2024-04-20T11:39:06.919 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.42239.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29559.log 2024-04-20T11:39:06.919 INFO:teuthology.orchestra.run.smithi152.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42239.log.gz 2024-04-20T11:39:06.920 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.77394.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51903.log 2024-04-20T11:39:06.920 INFO:teuthology.orchestra.run.smithi152.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.77394.log.gz 2024-04-20T11:39:06.921 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.29559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27942.log 2024-04-20T11:39:06.921 INFO:teuthology.orchestra.run.smithi152.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.29559.log.gz 2024-04-20T11:39:06.922 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.51903.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70660.log 2024-04-20T11:39:06.922 INFO:teuthology.orchestra.run.smithi152.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.51903.log.gz 2024-04-20T11:39:06.923 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.27942.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46460.log 2024-04-20T11:39:06.923 INFO:teuthology.orchestra.run.smithi152.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.27942.log.gz 2024-04-20T11:39:06.925 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.70660.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.70660.log.gz 2024-04-20T11:39:06.925 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65450.log 2024-04-20T11:39:06.925 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.46460.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40579.log 2024-04-20T11:39:06.926 INFO:teuthology.orchestra.run.smithi152.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.46460.log.gz 2024-04-20T11:39:06.927 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.65450.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57647.log 2024-04-20T11:39:06.927 INFO:teuthology.orchestra.run.smithi152.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.65450.log.gz 2024-04-20T11:39:06.928 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.40579.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74568.log 2024-04-20T11:39:06.928 INFO:teuthology.orchestra.run.smithi152.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.40579.log.gz 2024-04-20T11:39:06.929 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.57647.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34602.log 2024-04-20T11:39:06.929 INFO:teuthology.orchestra.run.smithi152.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.57647.log.gz 2024-04-20T11:39:06.930 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.74568.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75621.log 2024-04-20T11:39:06.931 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.34602.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.74568.log.gz 2024-04-20T11:39:06.931 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62644.log 2024-04-20T11:39:06.932 INFO:teuthology.orchestra.run.smithi152.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.34602.log.gz 2024-04-20T11:39:06.933 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.75621.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41690.log 2024-04-20T11:39:06.933 INFO:teuthology.orchestra.run.smithi152.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.75621.log.gz 2024-04-20T11:39:06.933 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.62644.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54615.log 2024-04-20T11:39:06.934 INFO:teuthology.orchestra.run.smithi152.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.62644.log.gz 2024-04-20T11:39:06.935 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.41690.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76236.log 2024-04-20T11:39:06.935 INFO:teuthology.orchestra.run.smithi152.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.41690.log.gz 2024-04-20T11:39:06.936 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.54615.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65738.log 2024-04-20T11:39:06.936 INFO:teuthology.orchestra.run.smithi152.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.54615.log.gz 2024-04-20T11:39:06.937 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.76236.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52191.log 2024-04-20T11:39:06.937 INFO:teuthology.orchestra.run.smithi152.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.76236.log.gz 2024-04-20T11:39:06.938 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.65738.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32337.log 2024-04-20T11:39:06.938 INFO:teuthology.orchestra.run.smithi152.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.65738.log.gz 2024-04-20T11:39:06.939 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.52191.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59271.log 2024-04-20T11:39:06.939 INFO:teuthology.orchestra.run.smithi152.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.52191.log.gz 2024-04-20T11:39:06.940 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.32337.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33626.log 2024-04-20T11:39:06.940 INFO:teuthology.orchestra.run.smithi152.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.32337.log.gz 2024-04-20T11:39:06.941 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.59271.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41377.log 2024-04-20T11:39:06.941 INFO:teuthology.orchestra.run.smithi152.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.59271.log.gz 2024-04-20T11:39:06.942 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.33626.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33752.log 2024-04-20T11:39:06.943 INFO:teuthology.orchestra.run.smithi152.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.33626.log.gz 2024-04-20T11:39:06.943 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.41377.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42934.log 2024-04-20T11:39:06.944 INFO:teuthology.orchestra.run.smithi152.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41377.log.gz 2024-04-20T11:39:06.944 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.33752.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35623.log 2024-04-20T11:39:06.945 INFO:teuthology.orchestra.run.smithi152.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.33752.log.gz 2024-04-20T11:39:06.945 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.42934.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29491.log 2024-04-20T11:39:06.946 INFO:teuthology.orchestra.run.smithi152.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42934.log.gz 2024-04-20T11:39:06.946 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.35623.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43581.log 2024-04-20T11:39:06.947 INFO:teuthology.orchestra.run.smithi152.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35623.log.gz 2024-04-20T11:39:06.948 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.29491.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51660.log 2024-04-20T11:39:06.948 INFO:teuthology.orchestra.run.smithi152.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.29491.log.gz 2024-04-20T11:39:06.949 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.43581.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50091.log 2024-04-20T11:39:06.949 INFO:teuthology.orchestra.run.smithi152.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.43581.log.gz 2024-04-20T11:39:06.950 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.51660.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47721.log 2024-04-20T11:39:06.950 INFO:teuthology.orchestra.run.smithi152.stderr: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.51660.log.gz 2024-04-20T11:39:06.951 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.50091.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39392.log 2024-04-20T11:39:06.951 INFO:teuthology.orchestra.run.smithi152.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50091.log.gz 2024-04-20T11:39:06.952 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.47721.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48047.log 2024-04-20T11:39:06.952 INFO:teuthology.orchestra.run.smithi152.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.47721.log.gz 2024-04-20T11:39:06.953 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.39392.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49748.log 2024-04-20T11:39:06.954 INFO:teuthology.orchestra.run.smithi152.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.39392.log.gz 2024-04-20T11:39:06.954 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.48047.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47154.log 2024-04-20T11:39:06.955 INFO:teuthology.orchestra.run.smithi152.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48047.log.gz 2024-04-20T11:39:06.956 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.49748.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69669.log 2024-04-20T11:39:06.956 INFO:teuthology.orchestra.run.smithi152.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.49748.log.gz 2024-04-20T11:39:06.956 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.47154.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59835.log 2024-04-20T11:39:06.957 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.69669.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.47154.log.gz 2024-04-20T11:39:06.958 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48545.log 2024-04-20T11:39:06.958 INFO:teuthology.orchestra.run.smithi152.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.69669.log.gz 2024-04-20T11:39:06.959 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.59835.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74075.log 2024-04-20T11:39:06.959 INFO:teuthology.orchestra.run.smithi152.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.59835.log.gz 2024-04-20T11:39:06.960 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.48545.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53708.log 2024-04-20T11:39:06.960 INFO:teuthology.orchestra.run.smithi152.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48545.log.gz 2024-04-20T11:39:06.961 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.74075.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28464.log 2024-04-20T11:39:06.961 INFO:teuthology.orchestra.run.smithi152.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.74075.log.gz 2024-04-20T11:39:06.962 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.53708.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74634.log 2024-04-20T11:39:06.962 INFO:teuthology.orchestra.run.smithi152.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.53708.log.gz 2024-04-20T11:39:06.963 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.28464.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52149.log 2024-04-20T11:39:06.963 INFO:teuthology.orchestra.run.smithi152.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.28464.log.gz 2024-04-20T11:39:06.964 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.74634.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73520.log 2024-04-20T11:39:06.964 INFO:teuthology.orchestra.run.smithi152.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.74634.log.gz 2024-04-20T11:39:06.965 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.52149.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61282.log 2024-04-20T11:39:06.966 INFO:teuthology.orchestra.run.smithi152.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52149.log.gz 2024-04-20T11:39:06.966 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.73520.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29587.log 2024-04-20T11:39:06.967 INFO:teuthology.orchestra.run.smithi152.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.73520.log.gz 2024-04-20T11:39:06.968 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.61282.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34308.log 2024-04-20T11:39:06.968 INFO:teuthology.orchestra.run.smithi152.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.61282.log.gz 2024-04-20T11:39:06.969 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.29587.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51489.log 2024-04-20T11:39:06.969 INFO:teuthology.orchestra.run.smithi152.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.29587.log.gz 2024-04-20T11:39:06.970 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.34308.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68274.log 2024-04-20T11:39:06.970 INFO:teuthology.orchestra.run.smithi152.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.34308.log.gz 2024-04-20T11:39:06.971 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.51489.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71662.log 2024-04-20T11:39:06.971 INFO:teuthology.orchestra.run.smithi152.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51489.log.gz 2024-04-20T11:39:06.972 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.68274.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76998.log 2024-04-20T11:39:06.972 INFO:teuthology.orchestra.run.smithi152.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.68274.log.gz 2024-04-20T11:39:06.973 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.71662.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44372.log 2024-04-20T11:39:06.973 INFO:teuthology.orchestra.run.smithi152.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.71662.log.gz 2024-04-20T11:39:06.974 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.76998.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.16552.log 2024-04-20T11:39:06.974 INFO:teuthology.orchestra.run.smithi152.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.76998.log.gz 2024-04-20T11:39:06.975 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.44372.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62392.log 2024-04-20T11:39:06.976 INFO:teuthology.orchestra.run.smithi152.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44372.log.gz 2024-04-20T11:39:06.976 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph.tmp-client.admin.16552.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53838.log 2024-04-20T11:39:06.977 INFO:teuthology.orchestra.run.smithi152.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.16552.log.gz 2024-04-20T11:39:06.978 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.62392.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-04-20T11:39:06.978 INFO:teuthology.orchestra.run.smithi152.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.62392.log.gz 2024-04-20T11:39:06.979 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.53838.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76956.log 2024-04-20T11:39:06.979 INFO:teuthology.orchestra.run.smithi152.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.53838.log.gz 2024-04-20T11:39:06.979 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49874.log 2024-04-20T11:39:06.981 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.76956.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41606.log 2024-04-20T11:39:06.981 INFO:teuthology.orchestra.run.smithi152.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.76956.log.gz 2024-04-20T11:39:06.982 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.49874.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35244.log 2024-04-20T11:39:06.982 INFO:teuthology.orchestra.run.smithi152.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49874.log.gz 2024-04-20T11:39:06.983 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.41606.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52788.log 2024-04-20T11:39:06.983 INFO:teuthology.orchestra.run.smithi152.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.41606.log.gz 2024-04-20T11:39:06.985 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.35244.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72382.log 2024-04-20T11:39:06.985 INFO:teuthology.orchestra.run.smithi152.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.35244.log.gz 2024-04-20T11:39:06.986 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.52788.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66396.log 2024-04-20T11:39:06.986 INFO:teuthology.orchestra.run.smithi152.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52788.log.gz 2024-04-20T11:39:06.987 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.72382.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76914.log 2024-04-20T11:39:06.987 INFO:teuthology.orchestra.run.smithi152.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.72382.log.gz 2024-04-20T11:39:06.988 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.66396.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76194.log 2024-04-20T11:39:06.989 INFO:teuthology.orchestra.run.smithi152.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.66396.log.gz 2024-04-20T11:39:06.990 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.76914.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31707.log 2024-04-20T11:39:06.990 INFO:teuthology.orchestra.run.smithi152.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.76914.log.gz 2024-04-20T11:39:06.991 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.76194.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77310.log 2024-04-20T11:39:06.991 INFO:teuthology.orchestra.run.smithi152.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.76194.log.gz 2024-04-20T11:39:06.992 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.31707.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66648.log 2024-04-20T11:39:06.992 INFO:teuthology.orchestra.run.smithi152.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.31707.log.gz 2024-04-20T11:39:06.993 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.77310.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50385.log 2024-04-20T11:39:06.994 INFO:teuthology.orchestra.run.smithi152.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.77310.log.gz 2024-04-20T11:39:06.995 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.66648.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61752.log 2024-04-20T11:39:06.995 INFO:teuthology.orchestra.run.smithi152.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.66648.log.gz 2024-04-20T11:39:06.996 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.50385.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67665.log 2024-04-20T11:39:06.996 INFO:teuthology.orchestra.run.smithi152.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.50385.log.gz 2024-04-20T11:39:06.997 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.61752.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38441.log 2024-04-20T11:39:06.997 INFO:teuthology.orchestra.run.smithi152.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.61752.log.gz 2024-04-20T11:39:06.998 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.67665.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65981.log 2024-04-20T11:39:06.999 INFO:teuthology.orchestra.run.smithi152.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.67665.log.gz 2024-04-20T11:39:07.000 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.38441.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73123.log 2024-04-20T11:39:07.000 INFO:teuthology.orchestra.run.smithi152.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.38441.log.gz 2024-04-20T11:39:07.001 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.65981.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53442.log 2024-04-20T11:39:07.001 INFO:teuthology.orchestra.run.smithi152.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.65981.log.gz 2024-04-20T11:39:07.002 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.73123.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74126.log 2024-04-20T11:39:07.003 INFO:teuthology.orchestra.run.smithi152.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.73123.log.gz 2024-04-20T11:39:07.004 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.53442.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27984.log 2024-04-20T11:39:07.004 INFO:teuthology.orchestra.run.smithi152.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.53442.log.gz 2024-04-20T11:39:07.005 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.74126.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61464.log 2024-04-20T11:39:07.005 INFO:teuthology.orchestra.run.smithi152.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.74126.log.gz 2024-04-20T11:39:07.006 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.27984.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39012.log 2024-04-20T11:39:07.006 INFO:teuthology.orchestra.run.smithi152.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.27984.log.gz 2024-04-20T11:39:07.007 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.61464.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37159.log 2024-04-20T11:39:07.007 INFO:teuthology.orchestra.run.smithi152.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.61464.log.gz 2024-04-20T11:39:07.008 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.39012.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68886.log 2024-04-20T11:39:07.009 INFO:teuthology.orchestra.run.smithi152.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.39012.log.gz 2024-04-20T11:39:07.010 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.37159.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74382.log 2024-04-20T11:39:07.010 INFO:teuthology.orchestra.run.smithi152.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.37159.log.gz 2024-04-20T11:39:07.011 INFO:teuthology.orchestra.run.smithi175.stderr: 92.7% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-04-20T11:39:07.011 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.68886.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39098.log 2024-04-20T11:39:07.011 INFO:teuthology.orchestra.run.smithi152.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.68886.log.gz 2024-04-20T11:39:07.012 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.74382.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58304.log 2024-04-20T11:39:07.012 INFO:teuthology.orchestra.run.smithi152.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.74382.log.gz 2024-04-20T11:39:07.013 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.39098.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54006.log 2024-04-20T11:39:07.013 INFO:teuthology.orchestra.run.smithi152.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.39098.log.gz 2024-04-20T11:39:07.015 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.58304.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76026.log 2024-04-20T11:39:07.015 INFO:teuthology.orchestra.run.smithi152.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.58304.log.gz 2024-04-20T11:39:07.016 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.54006.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45825.log 2024-04-20T11:39:07.016 INFO:teuthology.orchestra.run.smithi152.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.54006.log.gz 2024-04-20T11:39:07.017 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.76026.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52560.log 2024-04-20T11:39:07.017 INFO:teuthology.orchestra.run.smithi152.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.76026.log.gz 2024-04-20T11:39:07.018 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.45825.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45680.log 2024-04-20T11:39:07.019 INFO:teuthology.orchestra.run.smithi152.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45825.log.gz 2024-04-20T11:39:07.020 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.52560.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75337.log 2024-04-20T11:39:07.020 INFO:teuthology.orchestra.run.smithi152.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52560.log.gz 2024-04-20T11:39:07.021 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.45680.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57935.log 2024-04-20T11:39:07.021 INFO:teuthology.orchestra.run.smithi152.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45680.log.gz 2024-04-20T11:39:07.022 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.75337.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72997.log 2024-04-20T11:39:07.022 INFO:teuthology.orchestra.run.smithi152.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.75337.log.gz 2024-04-20T11:39:07.023 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.57935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69963.log 2024-04-20T11:39:07.024 INFO:teuthology.orchestra.run.smithi152.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.57935.log.gz 2024-04-20T11:39:07.025 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.72997.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51945.log 2024-04-20T11:39:07.025 INFO:teuthology.orchestra.run.smithi152.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.72997.log.gz 2024-04-20T11:39:07.026 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.69963.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53083.log 2024-04-20T11:39:07.026 INFO:teuthology.orchestra.run.smithi152.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.69963.log.gz 2024-04-20T11:39:07.027 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.51945.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45554.log 2024-04-20T11:39:07.027 INFO:teuthology.orchestra.run.smithi152.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51945.log.gz 2024-04-20T11:39:07.028 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.53083.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69711.log 2024-04-20T11:39:07.028 INFO:teuthology.orchestra.run.smithi152.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.53083.log.gz 2024-04-20T11:39:07.030 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.45554.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46586.log 2024-04-20T11:39:07.030 INFO:teuthology.orchestra.run.smithi152.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45554.log.gz 2024-04-20T11:39:07.030 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.69711.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72502.log 2024-04-20T11:39:07.031 INFO:teuthology.orchestra.run.smithi152.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.69711.log.gz 2024-04-20T11:39:07.032 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.46586.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76675.log 2024-04-20T11:39:07.032 INFO:teuthology.orchestra.run.smithi152.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46586.log.gz 2024-04-20T11:39:07.033 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.72502.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61240.log 2024-04-20T11:39:07.033 INFO:teuthology.orchestra.run.smithi152.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.72502.log.gz 2024-04-20T11:39:07.034 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.76675.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73987.log 2024-04-20T11:39:07.034 INFO:teuthology.orchestra.run.smithi152.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.76675.log.gz 2024-04-20T11:39:07.035 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.61240.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75398.log 2024-04-20T11:39:07.036 INFO:teuthology.orchestra.run.smithi152.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.61240.log.gz 2024-04-20T11:39:07.037 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.73987.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43018.log 2024-04-20T11:39:07.037 INFO:teuthology.orchestra.run.smithi152.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.73987.log.gz 2024-04-20T11:39:07.038 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.75398.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34812.log 2024-04-20T11:39:07.038 INFO:teuthology.orchestra.run.smithi152.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.75398.log.gz 2024-04-20T11:39:07.039 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.43018.log: gzip -5 --verbose -- 88.5% /var/log/ceph/ceph-client.admin.33324.log 2024-04-20T11:39:07.039 INFO:teuthology.orchestra.run.smithi152.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43018.log.gz 2024-04-20T11:39:07.040 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.34812.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46294.log 2024-04-20T11:39:07.040 INFO:teuthology.orchestra.run.smithi152.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.34812.log.gz 2024-04-20T11:39:07.041 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.33324.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34266.log 2024-04-20T11:39:07.041 INFO:teuthology.orchestra.run.smithi152.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.33324.log.gz 2024-04-20T11:39:07.043 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.46294.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30228.log 2024-04-20T11:39:07.043 INFO:teuthology.orchestra.run.smithi152.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46294.log.gz 2024-04-20T11:39:07.044 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.34266.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-04-20T11:39:07.044 INFO:teuthology.orchestra.run.smithi152.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34266.log.gz 2024-04-20T11:39:07.045 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.30228.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52728.log 2024-04-20T11:39:07.045 INFO:teuthology.orchestra.run.smithi152.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.30228.log.gz 2024-04-20T11:39:07.046 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40453.log 2024-04-20T11:39:07.048 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.52728.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49454.log 2024-04-20T11:39:07.048 INFO:teuthology.orchestra.run.smithi152.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52728.log.gz 2024-04-20T11:39:07.049 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.40453.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58761.log 2024-04-20T11:39:07.049 INFO:teuthology.orchestra.run.smithi152.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.40453.log.gz 2024-04-20T11:39:07.050 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.49454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53041.log 2024-04-20T11:39:07.050 INFO:teuthology.orchestra.run.smithi152.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49454.log.gz 2024-04-20T11:39:07.052 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.58761.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42449.log 2024-04-20T11:39:07.052 INFO:teuthology.orchestra.run.smithi152.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.58761.log.gz 2024-04-20T11:39:07.053 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.53041.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51531.log 2024-04-20T11:39:07.053 INFO:teuthology.orchestra.run.smithi152.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.53041.log.gz 2024-04-20T11:39:07.054 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.42449.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66065.log 2024-04-20T11:39:07.054 INFO:teuthology.orchestra.run.smithi152.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.42449.log.gz 2024-04-20T11:39:07.056 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.51531.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40227.log 2024-04-20T11:39:07.056 INFO:teuthology.orchestra.run.smithi152.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51531.log.gz 2024-04-20T11:39:07.057 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.66065.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68705.log 2024-04-20T11:39:07.057 INFO:teuthology.orchestra.run.smithi152.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.66065.log.gz 2024-04-20T11:39:07.058 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.40227.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75951.log 2024-04-20T11:39:07.058 INFO:teuthology.orchestra.run.smithi152.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.40227.log.gz 2024-04-20T11:39:07.060 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.68705.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57134.log 2024-04-20T11:39:07.060 INFO:teuthology.orchestra.run.smithi152.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.68705.log.gz 2024-04-20T11:39:07.061 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.75951.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55739.log 2024-04-20T11:39:07.061 INFO:teuthology.orchestra.run.smithi152.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.75951.log.gz 2024-04-20T11:39:07.062 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.57134.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74298.log 2024-04-20T11:39:07.062 INFO:teuthology.orchestra.run.smithi152.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.57134.log.gz 2024-04-20T11:39:07.064 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.55739.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67581.log 2024-04-20T11:39:07.064 INFO:teuthology.orchestra.run.smithi152.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.55739.log.gz 2024-04-20T11:39:07.065 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.74298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69054.log 2024-04-20T11:39:07.065 INFO:teuthology.orchestra.run.smithi152.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.74298.log.gz 2024-04-20T11:39:07.066 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.67581.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51700.log 2024-04-20T11:39:07.066 INFO:teuthology.orchestra.run.smithi152.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.67581.log.gz 2024-04-20T11:39:07.068 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.69054.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60872.log 2024-04-20T11:39:07.068 INFO:teuthology.orchestra.run.smithi152.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.69054.log.gz 2024-04-20T11:39:07.069 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.51700.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73818.log 2024-04-20T11:39:07.069 INFO:teuthology.orchestra.run.smithi152.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.51700.log.gz 2024-04-20T11:39:07.070 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.60872.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35837.log 2024-04-20T11:39:07.070 INFO:teuthology.orchestra.run.smithi152.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.60872.log.gz 2024-04-20T11:39:07.071 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.73818.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42155.log 2024-04-20T11:39:07.072 INFO:teuthology.orchestra.run.smithi152.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.73818.log.gz 2024-04-20T11:39:07.073 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.35837.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75579.log 2024-04-20T11:39:07.073 INFO:teuthology.orchestra.run.smithi152.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35837.log.gz 2024-04-20T11:39:07.074 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.42155.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36842.log 2024-04-20T11:39:07.074 INFO:teuthology.orchestra.run.smithi152.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.42155.log.gz 2024-04-20T11:39:07.075 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.75579.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62686.log 2024-04-20T11:39:07.076 INFO:teuthology.orchestra.run.smithi152.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.75579.log.gz 2024-04-20T11:39:07.076 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.36842.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75705.log 2024-04-20T11:39:07.077 INFO:teuthology.orchestra.run.smithi152.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36842.log.gz 2024-04-20T11:39:07.077 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.62686.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67031.log 2024-04-20T11:39:07.078 INFO:teuthology.orchestra.run.smithi152.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.62686.log.gz 2024-04-20T11:39:07.079 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.75705.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49286.log 2024-04-20T11:39:07.079 INFO:teuthology.orchestra.run.smithi152.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.75705.log.gz 2024-04-20T11:39:07.080 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.67031.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64876.log 2024-04-20T11:39:07.080 INFO:teuthology.orchestra.run.smithi152.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.67031.log.gz 2024-04-20T11:39:07.081 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.49286.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76498.log 2024-04-20T11:39:07.082 INFO:teuthology.orchestra.run.smithi152.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.49286.log.gz 2024-04-20T11:39:07.083 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.64876.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42491.log 2024-04-20T11:39:07.083 INFO:teuthology.orchestra.run.smithi152.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.64876.log.gz 2024-04-20T11:39:07.084 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.76498.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47821.log 2024-04-20T11:39:07.084 INFO:teuthology.orchestra.run.smithi152.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.76498.log.gz 2024-04-20T11:39:07.085 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.42491.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40759.log 2024-04-20T11:39:07.085 INFO:teuthology.orchestra.run.smithi152.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42491.log.gz 2024-04-20T11:39:07.086 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.47821.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62602.log 2024-04-20T11:39:07.087 INFO:teuthology.orchestra.run.smithi152.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47821.log.gz 2024-04-20T11:39:07.088 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.40759.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57466.log 2024-04-20T11:39:07.088 INFO:teuthology.orchestra.run.smithi152.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.40759.log.gz 2024-04-20T11:39:07.089 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.62602.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28111.log 2024-04-20T11:39:07.089 INFO:teuthology.orchestra.run.smithi152.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.62602.log.gz 2024-04-20T11:39:07.090 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.57466.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58136.log 2024-04-20T11:39:07.090 INFO:teuthology.orchestra.run.smithi152.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.57466.log.gz 2024-04-20T11:39:07.091 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.28111.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46077.log 2024-04-20T11:39:07.092 INFO:teuthology.orchestra.run.smithi152.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.28111.log.gz 2024-04-20T11:39:07.092 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.58136.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-04-20T11:39:07.093 INFO:teuthology.orchestra.run.smithi152.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.58136.log.gz 2024-04-20T11:39:07.094 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.46077.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.44009.log 2024-04-20T11:39:07.094 INFO:teuthology.orchestra.run.smithi152.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46077.log.gz 2024-04-20T11:39:07.095 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74340.log 2024-04-20T11:39:07.096 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.0.44009.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51819.log 2024-04-20T11:39:07.098 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.74340.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-04-20T11:39:07.098 INFO:teuthology.orchestra.run.smithi152.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.74340.log.gz 2024-04-20T11:39:07.099 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.51819.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31555.log 2024-04-20T11:39:07.099 INFO:teuthology.orchestra.run.smithi152.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51819.log.gz 2024-04-20T11:39:07.102 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-mgr.y.log: /var/log/ceph/ceph-client.admin.31555.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.31555.log.gz 2024-04-20T11:39:07.102 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52686.log 2024-04-20T11:39:07.104 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28674.log 2024-04-20T11:39:07.105 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.52686.log: 92.4% -- replaced with /var/log/ceph/ceph-client.0.44009.log.gz 2024-04-20T11:39:07.105 INFO:teuthology.orchestra.run.smithi152.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52686.log.gz 2024-04-20T11:39:07.106 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.28674.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.28674.log.gz 2024-04-20T11:39:07.107 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53922.log 2024-04-20T11:39:07.108 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40043.log 2024-04-20T11:39:07.110 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.53922.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.39862.log 2024-04-20T11:39:07.110 INFO:teuthology.orchestra.run.smithi152.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.53922.log.gz 2024-04-20T11:39:07.111 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.40043.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.40043.log.gz 2024-04-20T11:39:07.113 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.0.39862.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66438.log 2024-04-20T11:39:07.113 INFO:teuthology.orchestra.run.smithi152.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.0.39862.log.gz 2024-04-20T11:39:07.114 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47070.log 2024-04-20T11:39:07.116 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.66438.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65324.log 2024-04-20T11:39:07.116 INFO:teuthology.orchestra.run.smithi152.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.66438.log.gz 2024-04-20T11:39:07.118 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.47070.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.47070.log.gz 2024-04-20T11:39:07.118 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.65324.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.65324.log.gz 2024-04-20T11:39:07.119 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36191.log 2024-04-20T11:39:07.121 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37484.log 2024-04-20T11:39:07.122 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.36191.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39182.log 2024-04-20T11:39:07.122 INFO:teuthology.orchestra.run.smithi152.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.36191.log.gz 2024-04-20T11:39:07.123 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.37484.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.37484.log.gz 2024-04-20T11:39:07.124 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.39182.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.39182.log.gz 2024-04-20T11:39:07.125 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40495.log 2024-04-20T11:39:07.126 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50970.log 2024-04-20T11:39:07.127 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.40495.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45638.log 2024-04-20T11:39:07.128 INFO:teuthology.orchestra.run.smithi152.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.40495.log.gz 2024-04-20T11:39:07.129 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.50970.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.50970.log.gz 2024-04-20T11:39:07.130 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.45638.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45638.log.gz 2024-04-20T11:39:07.131 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41275.log 2024-04-20T11:39:07.132 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60956.log 2024-04-20T11:39:07.133 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.41275.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31027.log 2024-04-20T11:39:07.133 INFO:teuthology.orchestra.run.smithi152.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.41275.log.gz 2024-04-20T11:39:07.135 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.60956.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.60956.log.gz 2024-04-20T11:39:07.136 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.31027.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.31027.log.gz 2024-04-20T11:39:07.137 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28886.log 2024-04-20T11:39:07.137 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43060.log 2024-04-20T11:39:07.139 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.28886.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48215.log 2024-04-20T11:39:07.139 INFO:teuthology.orchestra.run.smithi152.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.28886.log.gz 2024-04-20T11:39:07.140 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.43060.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43060.log.gz 2024-04-20T11:39:07.142 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.48215.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48215.log.gz 2024-04-20T11:39:07.142 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45512.log 2024-04-20T11:39:07.143 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73436.log 2024-04-20T11:39:07.145 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.45512.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28292.log 2024-04-20T11:39:07.145 INFO:teuthology.orchestra.run.smithi152.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45512.log.gz 2024-04-20T11:39:07.146 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.73436.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.73436.log.gz 2024-04-20T11:39:07.148 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.28292.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.28292.log.gz 2024-04-20T11:39:07.148 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73478.log 2024-04-20T11:39:07.150 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77250.log 2024-04-20T11:39:07.151 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.73478.log: 88.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76110.log 2024-04-20T11:39:07.152 INFO:teuthology.orchestra.run.smithi152.stderr: -- replaced with /var/log/ceph/ceph-client.admin.73478.log.gz 2024-04-20T11:39:07.153 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.77250.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.77250.log.gz 2024-04-20T11:39:07.154 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.76110.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.76110.log.gz 2024-04-20T11:39:07.154 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-04-20T11:39:07.156 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27655.log 2024-04-20T11:39:07.157 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64384.log 2024-04-20T11:39:07.158 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.27655.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.27655.log.gz 2024-04-20T11:39:07.160 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.64384.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.64384.log.gz 2024-04-20T11:39:07.160 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73039.log 2024-04-20T11:39:07.160 INFO:teuthology.orchestra.run.smithi175.stderr: 91.4% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-04-20T11:39:07.162 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61995.log 2024-04-20T11:39:07.163 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.73039.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.73039.log.gz 2024-04-20T11:39:07.164 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.61995.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.61995.log.gz 2024-04-20T11:39:07.165 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73394.log 2024-04-20T11:39:07.166 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55518.log 2024-04-20T11:39:07.168 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.73394.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.73394.log.gz 2024-04-20T11:39:07.169 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.55518.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.55518.log.gz 2024-04-20T11:39:07.170 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57893.log 2024-04-20T11:39:07.171 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61668.log 2024-04-20T11:39:07.173 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.57893.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.57893.log.gz 2024-04-20T11:39:07.174 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.61668.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.61668.log.gz 2024-04-20T11:39:07.174 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73945.log 2024-04-20T11:39:07.178 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.73945.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.73945.log.gz 2024-04-20T11:39:07.178 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51861.log 2024-04-20T11:39:07.180 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33710.log 2024-04-20T11:39:07.181 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.51861.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51861.log.gz 2024-04-20T11:39:07.182 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.33710.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.33710.log.gz 2024-04-20T11:39:07.183 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49244.log 2024-04-20T11:39:07.184 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49790.log 2024-04-20T11:39:07.186 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.49244.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49244.log.gz 2024-04-20T11:39:07.187 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.49790.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49790.log.gz 2024-04-20T11:39:07.187 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29224.log 2024-04-20T11:39:07.189 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.68135.log 2024-04-20T11:39:07.190 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.29224.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.29224.log.gz 2024-04-20T11:39:07.193 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.0.68135.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72871.log 2024-04-20T11:39:07.196 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45344.log 2024-04-20T11:39:07.196 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.72871.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40969.log 2024-04-20T11:39:07.196 INFO:teuthology.orchestra.run.smithi152.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.72871.log.gz 2024-04-20T11:39:07.197 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.45344.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45344.log.gz 2024-04-20T11:39:07.198 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34938.log 2024-04-20T11:39:07.199 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.40969.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.40969.log.gz 2024-04-20T11:39:07.203 INFO:teuthology.orchestra.run.smithi152.stderr: 92.3% -- replaced with /var/log/ceph/ceph-client.0.68135.log.gz 2024-04-20T11:39:07.205 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.34938.log: /var/log/ceph/ceph-client.admin.77124.log 2024-04-20T11:39:07.205 INFO:teuthology.orchestra.run.smithi152.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.34938.log.gz 2024-04-20T11:39:07.207 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.77124.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.77124.log.gz 2024-04-20T11:39:07.207 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50610.log 2024-04-20T11:39:07.209 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65246.log 2024-04-20T11:39:07.210 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.50610.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.50610.log.gz 2024-04-20T11:39:07.211 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.65246.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.65246.log.gz 2024-04-20T11:39:07.212 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77082.log 2024-04-20T11:39:07.214 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65855.log 2024-04-20T11:39:07.215 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.77082.log: gzip -5 89.1% --verbose -- replaced with /var/log/ceph/ceph-client.admin.77082.log.gz -- 2024-04-20T11:39:07.215 INFO:teuthology.orchestra.run.smithi152.stderr: /var/log/ceph/ceph-client.admin.66023.log 2024-04-20T11:39:07.217 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.65855.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.65855.log.gz 2024-04-20T11:39:07.217 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.66023.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36758.log 2024-04-20T11:39:07.218 INFO:teuthology.orchestra.run.smithi152.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.66023.log.gz 2024-04-20T11:39:07.220 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.36758.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.36758.log.gz 2024-04-20T11:39:07.220 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48377.log 2024-04-20T11:39:07.222 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69258.log 2024-04-20T11:39:07.223 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.48377.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48377.log.gz 2024-04-20T11:39:07.224 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.69258.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.69258.log.gz 2024-04-20T11:39:07.225 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47947.log 2024-04-20T11:39:07.227 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45470.log 2024-04-20T11:39:07.228 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.47947.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47947.log.gz 2024-04-20T11:39:07.229 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.45470.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45470.log.gz 2024-04-20T11:39:07.230 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29213.log 2024-04-20T11:39:07.232 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77562.log 2024-04-20T11:39:07.233 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.29213.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.29213.log.gz 2024-04-20T11:39:07.234 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38884.log 2024-04-20T11:39:07.235 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.77562.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.77562.log.gz 2024-04-20T11:39:07.237 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5/var/log/ceph/ceph-client.admin.38884.log: --verbose -- /var/log/ceph/ceph-client.admin.38718.log 2024-04-20T11:39:07.237 INFO:teuthology.orchestra.run.smithi152.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.38884.log.gz 2024-04-20T11:39:07.239 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.36577.log 2024-04-20T11:39:07.240 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.38718.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.38718.log.gz 2024-04-20T11:39:07.240 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46716.log 2024-04-20T11:39:07.243 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.0.36577.log: /var/log/ceph/ceph-client.admin.46716.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28590.log 2024-04-20T11:39:07.243 INFO:teuthology.orchestra.run.smithi152.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46716.log.gz 2024-04-20T11:39:07.245 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45059.log 2024-04-20T11:39:07.247 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.28590.log: 92.3%/var/log/ceph/ceph-client.admin.45059.log: -- replaced with /var/log/ceph/ceph-client.0.36577.log.gz 87.9% -- replaced with /var/log/ceph/ceph-client.admin.28590.log.gz 2024-04-20T11:39:07.247 INFO:teuthology.orchestra.run.smithi152.stderr: 2024-04-20T11:39:07.248 INFO:teuthology.orchestra.run.smithi152.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45059.log.gz 2024-04-20T11:39:07.248 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64958.log 2024-04-20T11:39:07.250 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70348.log 2024-04-20T11:39:07.251 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.64958.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.64958.log.gz 2024-04-20T11:39:07.252 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.70348.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.70348.log.gz 2024-04-20T11:39:07.253 INFO:teuthology.orchestra.run.smithi175.stderr: 91.3% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-04-20T11:39:07.253 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50011.log 2024-04-20T11:39:07.254 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50480.log 2024-04-20T11:39:07.256 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.50011.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.50011.log.gz 2024-04-20T11:39:07.257 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.50480.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50480.log.gz 2024-04-20T11:39:07.258 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38273.log 2024-04-20T11:39:07.258 INFO:teuthology.orchestra.run.smithi175.stderr: 93.0% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-04-20T11:39:07.259 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52602.log 2024-04-20T11:39:07.260 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.38273.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.38273.log.gz 2024-04-20T11:39:07.261 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54918.log 2024-04-20T11:39:07.263 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.52602.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48893.log 2024-04-20T11:39:07.264 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.54918.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.52602.log.gz 2024-04-20T11:39:07.264 INFO:teuthology.orchestra.run.smithi152.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.54918.log.gz 2024-04-20T11:39:07.266 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.48893.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48293.log 2024-04-20T11:39:07.268 INFO:teuthology.orchestra.run.smithi152.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48893.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75135.log 2024-04-20T11:39:07.268 INFO:teuthology.orchestra.run.smithi152.stderr: 2024-04-20T11:39:07.270 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.48293.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48293.log.gz 2024-04-20T11:39:07.271 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.75135.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58430.log 2024-04-20T11:39:07.271 INFO:teuthology.orchestra.run.smithi152.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.75135.log.gz 2024-04-20T11:39:07.273 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.58430.log: -- /var/log/ceph/ceph-client.admin.34560.log 2024-04-20T11:39:07.273 INFO:teuthology.orchestra.run.smithi152.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.58430.log.gz 2024-04-20T11:39:07.274 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77166.log 2024-04-20T11:39:07.275 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.34560.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.34560.log.gz 2024-04-20T11:39:07.276 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5/var/log/ceph/ceph-client.admin.77166.log: --verbose -- /var/log/ceph/ceph-client.admin.67749.log 2024-04-20T11:39:07.277 INFO:teuthology.orchestra.run.smithi152.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.77166.log.gz 2024-04-20T11:39:07.278 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34686.log 2024-04-20T11:39:07.280 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.67749.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.67749.log.gz 2024-04-20T11:39:07.280 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68583.log 2024-04-20T11:39:07.282 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.34686.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.34686.log.gz 2024-04-20T11:39:07.282 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42976.log 2024-04-20T11:39:07.284 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.68583.log: 92.3% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-04-20T11:39:07.284 INFO:teuthology.orchestra.run.smithi152.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.68583.log.gz 2024-04-20T11:39:07.284 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64835.log 2024-04-20T11:39:07.285 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.42976.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42976.log.gz 2024-04-20T11:39:07.286 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47905.log 2024-04-20T11:39:07.287 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.64835.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30312.log 2024-04-20T11:39:07.287 INFO:teuthology.orchestra.run.smithi152.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.64835.log.gz 2024-04-20T11:39:07.288 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.47905.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47905.log.gz 2024-04-20T11:39:07.289 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43413.log 2024-04-20T11:39:07.289 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.30312.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.30312.log.gz 2024-04-20T11:39:07.290 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34350.log 2024-04-20T11:39:07.291 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.43413.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58262.log 2024-04-20T11:39:07.291 INFO:teuthology.orchestra.run.smithi152.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43413.log.gz 2024-04-20T11:39:07.293 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.34350.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.34350.log.gz 2024-04-20T11:39:07.293 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.58262.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42029.log 2024-04-20T11:39:07.294 INFO:teuthology.orchestra.run.smithi152.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.58262.log.gz 2024-04-20T11:39:07.294 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38543.log 2024-04-20T11:39:07.296 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.42029.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45909.log 2024-04-20T11:39:07.296 INFO:teuthology.orchestra.run.smithi152.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42029.log.gz 2024-04-20T11:39:07.298 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.38543.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.38543.log.gz 2024-04-20T11:39:07.298 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27765.log 2024-04-20T11:39:07.299 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.45909.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.45909.log.gz 2024-04-20T11:39:07.299 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45386.log 2024-04-20T11:39:07.301 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.27765.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69795.log 2024-04-20T11:39:07.301 INFO:teuthology.orchestra.run.smithi152.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.27765.log.gz 2024-04-20T11:39:07.302 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.45386.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45386.log.gz 2024-04-20T11:39:07.303 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.69795.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57508.log 2024-04-20T11:39:07.304 INFO:teuthology.orchestra.run.smithi152.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.69795.log.gz 2024-04-20T11:39:07.304 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37280.log 2024-04-20T11:39:07.306 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.57508.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52233.log 2024-04-20T11:39:07.306 INFO:teuthology.orchestra.run.smithi152.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.57508.log.gz 2024-04-20T11:39:07.307 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.37280.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.37280.log.gz 2024-04-20T11:39:07.308 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28890.log 2024-04-20T11:39:07.309 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.52233.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35795.log 2024-04-20T11:39:07.309 INFO:teuthology.orchestra.run.smithi152.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.52233.log.gz 2024-04-20T11:39:07.310 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.28890.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73268.log 2024-04-20T11:39:07.310 INFO:teuthology.orchestra.run.smithi152.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.28890.log.gz 2024-04-20T11:39:07.312 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.35795.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51387.log 2024-04-20T11:39:07.312 INFO:teuthology.orchestra.run.smithi152.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.35795.log.gz 2024-04-20T11:39:07.313 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.73268.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.73268.log.gz 2024-04-20T11:39:07.313 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64548.log 2024-04-20T11:39:07.314 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.51387.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51387.log.gz 2024-04-20T11:39:07.314 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29234.log 2024-04-20T11:39:07.316 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.64548.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35329.log 2024-04-20T11:39:07.316 INFO:teuthology.orchestra.run.smithi152.stderr: 85.5% -- replaced with /var/log/ceph/ceph-client.admin.64548.log.gz 2024-04-20T11:39:07.317 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.29234.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.29234.log.gz 2024-04-20T11:39:07.318 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43241.log 2024-04-20T11:39:07.318 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.35329.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35329.log.gz 2024-04-20T11:39:07.319 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52350.log 2024-04-20T11:39:07.321 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.43241.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77812.log 2024-04-20T11:39:07.321 INFO:teuthology.orchestra.run.smithi152.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43241.log.gz 2024-04-20T11:39:07.322 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.52350.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52350.log.gz 2024-04-20T11:39:07.322 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45227.log 2024-04-20T11:39:07.324 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.77812.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.77812.log.gz 2024-04-20T11:39:07.324 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52518.log 2024-04-20T11:39:07.325 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.45227.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.45227.log.gz 2024-04-20T11:39:07.325 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60914.log 2024-04-20T11:39:07.327 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.52518.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76278.log 2024-04-20T11:39:07.327 INFO:teuthology.orchestra.run.smithi152.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52518.log.gz 2024-04-20T11:39:07.328 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.60914.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30270.log 2024-04-20T11:39:07.328 INFO:teuthology.orchestra.run.smithi152.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.60914.log.gz 2024-04-20T11:39:07.329 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.76278.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.76278.log.gz 2024-04-20T11:39:07.330 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70839.log 2024-04-20T11:39:07.330 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.30270.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.30270.log.gz 2024-04-20T11:39:07.331 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63380.log 2024-04-20T11:39:07.333 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.70839.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41137.log 2024-04-20T11:39:07.333 INFO:teuthology.orchestra.run.smithi152.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.70839.log.gz 2024-04-20T11:39:07.334 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.63380.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.63380.log.gz 2024-04-20T11:39:07.335 INFO:teuthology.orchestra.run.smithi152.stderr:gzip/var/log/ceph/ceph-client.admin.41137.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.32211.log 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41137.log.gz 2024-04-20T11:39:07.336 INFO:teuthology.orchestra.run.smithi152.stderr: 2024-04-20T11:39:07.337 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72141.log 2024-04-20T11:39:07.339 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.32211.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69501.log 2024-04-20T11:39:07.339 INFO:teuthology.orchestra.run.smithi152.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.32211.log.gz 2024-04-20T11:39:07.340 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.72141.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.72141.log.gz 2024-04-20T11:39:07.340 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46800.log 2024-04-20T11:39:07.342 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.69501.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40537.log 2024-04-20T11:39:07.342 INFO:teuthology.orchestra.run.smithi152.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.69501.log.gz 2024-04-20T11:39:07.343 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.46800.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63566.log 2024-04-20T11:39:07.343 INFO:teuthology.orchestra.run.smithi152.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46800.log.gz 2024-04-20T11:39:07.344 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.40537.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38315.log 2024-04-20T11:39:07.344 INFO:teuthology.orchestra.run.smithi152.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.40537.log.gz 2024-04-20T11:39:07.346 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.63566.log: gzip -5 --verbose -- 89.0% -- replaced with /var/log/ceph/ceph-client.admin.63566.log.gz 2024-04-20T11:39:07.346 INFO:teuthology.orchestra.run.smithi152.stderr: /var/log/ceph/ceph-client.admin.43665.log 2024-04-20T11:39:07.347 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.38315.log: gzip 89.0% -- replaced with /var/log/ceph/ceph-client.admin.38315.log.gz -5 2024-04-20T11:39:07.348 INFO:teuthology.orchestra.run.smithi152.stderr: --verbose -- /var/log/ceph/ceph-client.admin.57689.log 2024-04-20T11:39:07.349 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.43665.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-04-20T11:39:07.349 INFO:teuthology.orchestra.run.smithi152.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.43665.log.gz 2024-04-20T11:39:07.350 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.57689.log: 88.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67371.log 2024-04-20T11:39:07.350 INFO:teuthology.orchestra.run.smithi152.stderr: -- replaced with /var/log/ceph/ceph-client.admin.57689.log.gz 2024-04-20T11:39:07.353 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-mon.a.log: /var/log/ceph/ceph-client.admin.67371.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.67371.log.gz 2024-04-20T11:39:07.353 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28197.log 2024-04-20T11:39:07.355 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45780.log 2024-04-20T11:39:07.356 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.28197.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.28197.log.gz 2024-04-20T11:39:07.357 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28548.log 2024-04-20T11:39:07.357 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.45780.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.45780.log.gz 2024-04-20T11:39:07.358 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48461.log 2024-04-20T11:39:07.360 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.28548.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.28548.log.gz 2024-04-20T11:39:07.360 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46860.log 2024-04-20T11:39:07.361 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.48461.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.48461.log.gz 2024-04-20T11:39:07.363 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.46860.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.46860.log.gz 2024-04-20T11:39:07.364 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32684.log 2024-04-20T11:39:07.365 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37526.log 2024-04-20T11:39:07.366 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.32684.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.32684.log.gz 2024-04-20T11:39:07.367 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35286.log 2024-04-20T11:39:07.368 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.37526.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.37526.log.gz 2024-04-20T11:39:07.369 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76320.log 2024-04-20T11:39:07.370 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.35286.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.35286.log.gz 2024-04-20T11:39:07.371 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35753.log 2024-04-20T11:39:07.372 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.76320.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.76320.log.gz 2024-04-20T11:39:07.373 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54048.log 2024-04-20T11:39:07.374 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.35753.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.35753.log.gz 2024-04-20T11:39:07.374 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63334.log 2024-04-20T11:39:07.376 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.54048.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.54048.log.gz 2024-04-20T11:39:07.376 INFO:teuthology.orchestra.run.smithi152.stderr:gzip/var/log/ceph/ceph-client.admin.63334.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.39056.log 2024-04-20T11:39:07.377 INFO:teuthology.orchestra.run.smithi152.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.63334.log.gz 2024-04-20T11:39:07.379 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.39056.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.39056.log.gz 2024-04-20T11:39:07.380 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67073.log 2024-04-20T11:39:07.381 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49202.log 2024-04-20T11:39:07.383 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.67073.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.67073.log.gz 2024-04-20T11:39:07.383 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32295.log 2024-04-20T11:39:07.384 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.49202.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49202.log.gz 2024-04-20T11:39:07.385 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67539.log 2024-04-20T11:39:07.386 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.32295.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.32295.log.gz 2024-04-20T11:39:07.387 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65492.log 2024-04-20T11:39:07.388 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.67539.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.67539.log.gz 2024-04-20T11:39:07.389 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51987.log 2024-04-20T11:39:07.390 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.65492.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.65492.log.gz 2024-04-20T11:39:07.391 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73657.log 2024-04-20T11:39:07.392 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.51987.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51987.log.gz 2024-04-20T11:39:07.393 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47679.log 2024-04-20T11:39:07.394 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.73657.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.73657.log.gz 2024-04-20T11:39:07.395 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.47679.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.47679.log.gz 2024-04-20T11:39:07.396 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59667.log 2024-04-20T11:39:07.397 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64630.log 2024-04-20T11:39:07.399 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.59667.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.59667.log.gz 2024-04-20T11:39:07.400 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.64630.log: -- /var/log/ceph/ceph-client.admin.32768.log 2024-04-20T11:39:07.400 INFO:teuthology.orchestra.run.smithi152.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.64630.log.gz 2024-04-20T11:39:07.401 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70532.log 2024-04-20T11:39:07.403 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.32768.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.32768.log.gz 2024-04-20T11:39:07.403 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27861.log 2024-04-20T11:39:07.404 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.70532.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.70532.log.gz 2024-04-20T11:39:07.405 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28894.log/var/log/ceph/ceph-client.admin.27861.log: 2024-04-20T11:39:07.406 INFO:teuthology.orchestra.run.smithi152.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.27861.log.gz 2024-04-20T11:39:07.407 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34476.log 2024-04-20T11:39:07.408 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.28894.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.28894.log.gz 2024-04-20T11:39:07.409 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54198.log 2024-04-20T11:39:07.410 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.34476.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34476.log.gz 2024-04-20T11:39:07.411 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58584.log 2024-04-20T11:39:07.412 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.54198.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.54198.log.gz 2024-04-20T11:39:07.413 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66606.log 2024-04-20T11:39:07.414 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.58584.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.58584.log.gz 2024-04-20T11:39:07.415 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72298.log 2024-04-20T11:39:07.416 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.66606.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.66606.log.gz 2024-04-20T11:39:07.417 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.72298.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.72298.log.gz 2024-04-20T11:39:07.418 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53880.log 2024-04-20T11:39:07.419 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67455.log 2024-04-20T11:39:07.421 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.53880.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.53880.log.gz 2024-04-20T11:39:07.421 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61548.log 2024-04-20T11:39:07.422 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.67455.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.67455.log.gz 2024-04-20T11:39:07.423 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59495.log 2024-04-20T11:39:07.424 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.61548.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.61548.log.gz 2024-04-20T11:39:07.425 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55621.log/var/log/ceph/ceph-client.admin.59495.log: 2024-04-20T11:39:07.426 INFO:teuthology.orchestra.run.smithi152.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.59495.log.gz 2024-04-20T11:39:07.427 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49916.log 2024-04-20T11:39:07.428 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.55621.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55621.log.gz 2024-04-20T11:39:07.429 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69216.log 2024-04-20T11:39:07.430 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.49916.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49916.log.gz 2024-04-20T11:39:07.431 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66311.log 2024-04-20T11:39:07.432 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.69216.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.69216.log.gz 2024-04-20T11:39:07.433 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64217.log 2024-04-20T11:39:07.434 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.66311.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.66311.log.gz 2024-04-20T11:39:07.435 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.64217.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.64217.log.gz 2024-04-20T11:39:07.436 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37811.log 2024-04-20T11:39:07.438 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67707.log 2024-04-20T11:39:07.439 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.37811.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37811.log.gz 2024-04-20T11:39:07.440 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37979.log 2024-04-20T11:39:07.441 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.67707.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.67707.log.gz 2024-04-20T11:39:07.441 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40311.log 2024-04-20T11:39:07.442 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.37979.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37979.log.gz 2024-04-20T11:39:07.443 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70005.log 2024-04-20T11:39:07.444 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.40311.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40311.log.gz 2024-04-20T11:39:07.445 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37364.log 2024-04-20T11:39:07.446 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.70005.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.70005.log.gz 2024-04-20T11:39:07.447 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58929.log 2024-04-20T11:39:07.448 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.37364.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.37364.log.gz 2024-04-20T11:39:07.449 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55476.log 2024-04-20T11:39:07.450 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.58929.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.58929.log.gz 2024-04-20T11:39:07.451 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22710.log 2024-04-20T11:39:07.452 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.55476.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.55476.log.gz 2024-04-20T11:39:07.453 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48713.log/var/log/ceph/ceph-client.admin.22710.log: 2024-04-20T11:39:07.453 INFO:teuthology.orchestra.run.smithi152.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.22710.log.gz 2024-04-20T11:39:07.454 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58388.log 2024-04-20T11:39:07.456 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.48713.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48713.log.gz 2024-04-20T11:39:07.457 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.58388.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.58388.log.gz 2024-04-20T11:39:07.458 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61380.log 2024-04-20T11:39:07.459 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31833.log 2024-04-20T11:39:07.460 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.61380.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.61380.log.gz 2024-04-20T11:39:07.461 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42071.log 2024-04-20T11:39:07.462 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.31833.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.31833.log.gz 2024-04-20T11:39:07.463 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47863.log 2024-04-20T11:39:07.464 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.42071.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42071.log.gz 2024-04-20T11:39:07.465 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.47863.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47863.log.gz 2024-04-20T11:39:07.466 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75783.log 2024-04-20T11:39:07.467 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29934.log 2024-04-20T11:39:07.468 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.75783.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.75783.log.gz 2024-04-20T11:39:07.469 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70927.log 2024-04-20T11:39:07.470 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.29934.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.29934.log.gz 2024-04-20T11:39:07.471 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57731.log 2024-04-20T11:39:07.472 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.70927.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.70927.log.gz 2024-04-20T11:39:07.473 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29557.log 2024-04-20T11:39:07.474 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.57731.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.57731.log.gz 2024-04-20T11:39:07.475 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.29557.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.29557.log.gz 2024-04-20T11:39:07.476 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46986.log 2024-04-20T11:39:07.477 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47989.log 2024-04-20T11:39:07.479 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.46986.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46986.log.gz 2024-04-20T11:39:07.479 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67623.log 2024-04-20T11:39:07.480 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.47989.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47989.log.gz 2024-04-20T11:39:07.481 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23078.log/var/log/ceph/ceph-client.admin.67623.log: 2024-04-20T11:39:07.482 INFO:teuthology.orchestra.run.smithi152.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.67623.log.gz 2024-04-20T11:39:07.483 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36149.log 2024-04-20T11:39:07.484 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.23078.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.23078.log.gz 2024-04-20T11:39:07.485 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.36149.log: /var/log/ceph/ceph-client.admin.70173.log 2024-04-20T11:39:07.485 INFO:teuthology.orchestra.run.smithi152.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.36149.log.gz 2024-04-20T11:39:07.486 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75867.log 2024-04-20T11:39:07.488 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.70173.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.70173.log.gz 2024-04-20T11:39:07.488 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42323.log 2024-04-20T11:39:07.489 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.75867.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.75867.log.gz 2024-04-20T11:39:07.490 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76591.log 2024-04-20T11:39:07.492 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.42323.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.42323.log.gz 2024-04-20T11:39:07.492 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34224.log 2024-04-20T11:39:07.493 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.76591.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.76591.log.gz 2024-04-20T11:39:07.494 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46902.log 2024-04-20T11:39:07.496 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.34224.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.34224.log.gz 2024-04-20T11:39:07.496 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.46902.log: -- /var/log/ceph/ceph-client.admin.71977.log 2024-04-20T11:39:07.496 INFO:teuthology.orchestra.run.smithi152.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46902.log.gz 2024-04-20T11:39:07.497 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38021.log 2024-04-20T11:39:07.499 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.71977.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.71977.log.gz 2024-04-20T11:39:07.500 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57851.log 2024-04-20T11:39:07.500 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.38021.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.38021.log.gz 2024-04-20T11:39:07.501 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33240.log 2024-04-20T11:39:07.503 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.57851.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.57851.log.gz 2024-04-20T11:39:07.504 INFO:teuthology.orchestra.run.smithi152.stderr:gzip/var/log/ceph/ceph-client.admin.33240.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.28898.log 2024-04-20T11:39:07.504 INFO:teuthology.orchestra.run.smithi152.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.33240.log.gz 2024-04-20T11:39:07.505 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60830.log 2024-04-20T11:39:07.506 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.28898.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.28898.log.gz 2024-04-20T11:39:07.508 INFO:teuthology.orchestra.run.smithi152.stderr:gzip/var/log/ceph/ceph-client.admin.60830.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.70969.log 2024-04-20T11:39:07.508 INFO:teuthology.orchestra.run.smithi152.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.60830.log.gz 2024-04-20T11:39:07.509 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59313.log 2024-04-20T11:39:07.510 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.70969.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.70969.log.gz 2024-04-20T11:39:07.511 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.59313.log: /var/log/ceph/ceph-client.admin.45101.log 2024-04-20T11:39:07.512 INFO:teuthology.orchestra.run.smithi152.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.59313.log.gz 2024-04-20T11:39:07.512 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59537.log 2024-04-20T11:39:07.514 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.45101.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.45101.log.gz 2024-04-20T11:39:07.515 INFO:teuthology.orchestra.run.smithi152.stderr:gzip/var/log/ceph/ceph-client.admin.59537.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.28241.log 2024-04-20T11:39:07.515 INFO:teuthology.orchestra.run.smithi152.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.59537.log.gz 2024-04-20T11:39:07.516 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63422.log 2024-04-20T11:39:07.518 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.28241.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.28241.log.gz 2024-04-20T11:39:07.519 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.63422.log: /var/log/ceph/ceph-client.admin.71011.log 2024-04-20T11:39:07.519 INFO:teuthology.orchestra.run.smithi152.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.63422.log.gz 2024-04-20T11:39:07.520 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59444.log 2024-04-20T11:39:07.521 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.71011.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.71011.log.gz 2024-04-20T11:39:07.523 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5/var/log/ceph/ceph-client.admin.59444.log: --verbose -- /var/log/ceph/ceph-client.admin.41945.log 2024-04-20T11:39:07.523 INFO:teuthology.orchestra.run.smithi152.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.59444.log.gz 2024-04-20T11:39:07.524 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46502.log 2024-04-20T11:39:07.525 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.41945.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41945.log.gz 2024-04-20T11:39:07.526 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42575.log 2024-04-20T11:39:07.526 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.46502.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46502.log.gz 2024-04-20T11:39:07.527 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62247.log 2024-04-20T11:39:07.528 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.42575.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.42575.log.gz 2024-04-20T11:39:07.529 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.62247.log: /var/log/ceph/ceph-client.admin.61710.log 2024-04-20T11:39:07.530 INFO:teuthology.orchestra.run.smithi152.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.62247.log.gz 2024-04-20T11:39:07.531 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41233.log 2024-04-20T11:39:07.532 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.61710.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.61710.log.gz 2024-04-20T11:39:07.533 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38231.log 2024-04-20T11:39:07.534 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.41233.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41233.log.gz 2024-04-20T11:39:07.535 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72544.log 2024-04-20T11:39:07.536 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.38231.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.38231.log.gz 2024-04-20T11:39:07.537 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.72544.log: -- /var/log/ceph/ceph-client.admin.39224.log 2024-04-20T11:39:07.537 INFO:teuthology.orchestra.run.smithi152.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.72544.log.gz 2024-04-20T11:39:07.538 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64343.log 2024-04-20T11:39:07.540 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.39224.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.39224.log.gz 2024-04-20T11:39:07.541 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29976.log/var/log/ceph/ceph-client.admin.64343.log: 2024-04-20T11:39:07.541 INFO:teuthology.orchestra.run.smithi152.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.64343.log.gz 2024-04-20T11:39:07.542 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73310.log 2024-04-20T11:39:07.543 INFO:teuthology.orchestra.run.smithi175.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-04-20T11:39:07.544 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.29976.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.29976.log.gz 2024-04-20T11:39:07.544 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66743.log 2024-04-20T11:39:07.545 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.73310.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.73310.log.gz 2024-04-20T11:39:07.546 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76152.log 2024-04-20T11:39:07.547 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.66743.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.66743.log.gz 2024-04-20T11:39:07.548 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72340.log 2024-04-20T11:39:07.548 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.76152.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.76152.log.gz 2024-04-20T11:39:07.550 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53400.log 2024-04-20T11:39:07.550 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.72340.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.72340.log.gz 2024-04-20T11:39:07.551 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.53400.log: /var/log/ceph/ceph-client.admin.49160.log 2024-04-20T11:39:07.552 INFO:teuthology.orchestra.run.smithi152.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.53400.log.gz 2024-04-20T11:39:07.553 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50299.log 2024-04-20T11:39:07.554 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.49160.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49160.log.gz 2024-04-20T11:39:07.555 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.50299.log: -- /var/log/ceph/ceph-client.admin.28716.log 2024-04-20T11:39:07.556 INFO:teuthology.orchestra.run.smithi152.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50299.log.gz 2024-04-20T11:39:07.557 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65163.log 2024-04-20T11:39:07.558 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.28716.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.28716.log.gz 2024-04-20T11:39:07.559 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59877.log 2024-04-20T11:39:07.560 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.65163.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.65163.log.gz 2024-04-20T11:39:07.560 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62983.log 2024-04-20T11:39:07.562 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.59877.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.59877.log.gz 2024-04-20T11:39:07.562 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57773.log 2024-04-20T11:39:07.563 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.62983.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.62983.log.gz 2024-04-20T11:39:07.564 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58845.log 2024-04-20T11:39:07.565 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.57773.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.57773.log.gz 2024-04-20T11:39:07.566 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75663.log 2024-04-20T11:39:07.567 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.58845.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.58845.log.gz 2024-04-20T11:39:07.569 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.75663.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38968.log 2024-04-20T11:39:07.569 INFO:teuthology.orchestra.run.smithi152.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.75663.log.gz 2024-04-20T11:39:07.570 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33978.log 2024-04-20T11:39:07.570 INFO:teuthology.orchestra.run.smithi175.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-04-20T11:39:07.571 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.38968.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.38968.log.gz 2024-04-20T11:39:07.571 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48977.log 2024-04-20T11:39:07.573 INFO:teuthology.orchestra.run.smithi175.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-04-20T11:39:07.574 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.33978.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.33978.log.gz 2024-04-20T11:39:07.574 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.48977.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64175.log 2024-04-20T11:39:07.574 INFO:teuthology.orchestra.run.smithi152.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48977.log.gz 2024-04-20T11:39:07.576 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.64175.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.64175.log.gz 2024-04-20T11:39:07.577 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50429.log 2024-04-20T11:39:07.578 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59355.log 2024-04-20T11:39:07.580 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.50429.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50429.log.gz 2024-04-20T11:39:07.580 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.54476.log 2024-04-20T11:39:07.581 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.59355.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.59355.log.gz 2024-04-20T11:39:07.582 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46214.log 2024-04-20T11:39:07.584 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.0.54476.log: /var/log/ceph/ceph-client.admin.46214.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46214.log.gz 2024-04-20T11:39:07.585 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38800.log 2024-04-20T11:39:07.588 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.38800.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49118.log 2024-04-20T11:39:07.589 INFO:teuthology.orchestra.run.smithi152.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.38800.log.gz 2024-04-20T11:39:07.590 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75093.log 2024-04-20T11:39:07.593 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.49118.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49118.log.gz 2024-04-20T11:39:07.594 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.75093.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34770.log 2024-04-20T11:39:07.595 INFO:teuthology.orchestra.run.smithi152.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.75093.log.gz 2024-04-20T11:39:07.597 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51447.log 2024-04-20T11:39:07.599 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.34770.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34770.log.gz 2024-04-20T11:39:07.602 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.51447.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58803.log 2024-04-20T11:39:07.603 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40801.log 89.1% -- replaced with /var/log/ceph/ceph-client.admin.51447.log.gz 2024-04-20T11:39:07.603 INFO:teuthology.orchestra.run.smithi152.stderr: 2024-04-20T11:39:07.605 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.58803.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.58803.log.gz 2024-04-20T11:39:07.606 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38926.log 2024-04-20T11:39:07.611 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.38926.log: /var/log/ceph/ceph-client.admin.40801.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.40801.log.gz 2024-04-20T11:39:07.611 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43146.log 2024-04-20T11:39:07.612 INFO:teuthology.orchestra.run.smithi152.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.38926.log.gz 2024-04-20T11:39:07.614 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28884.log 2024-04-20T11:39:07.618 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.28884.log: /var/log/ceph/ceph-client.admin.43146.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43146.log.gz 2024-04-20T11:39:07.618 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41191.log 2024-04-20T11:39:07.619 INFO:teuthology.orchestra.run.smithi152.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.28884.log.gz 2024-04-20T11:39:07.621 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76633.log 2024-04-20T11:39:07.624 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.76633.log: /var/log/ceph/ceph-client.admin.41191.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59024.log 2024-04-20T11:39:07.626 INFO:teuthology.orchestra.run.smithi152.stderr: 88.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52392.log 2024-04-20T11:39:07.626 INFO:teuthology.orchestra.run.smithi152.stderr: -- replaced with /var/log/ceph/ceph-client.admin.41191.log.gz 2024-04-20T11:39:07.628 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.52392.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.76633.log.gz 2024-04-20T11:39:07.630 INFO:teuthology.orchestra.run.smithi152.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52392.log.gz 2024-04-20T11:39:07.631 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40675.log 2024-04-20T11:39:07.631 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.59024.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.59024.log.gz 2024-04-20T11:39:07.633 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.40675.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44608.log 2024-04-20T11:39:07.636 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.44608.log: gzip -5 --verbose 88.2% -- replaced with /var/log/ceph/ceph-client.admin.40675.log.gz 2024-04-20T11:39:07.636 INFO:teuthology.orchestra.run.smithi152.stderr: -- /var/log/ceph/ceph-client.admin.29211.log 2024-04-20T11:39:07.637 INFO:teuthology.orchestra.run.smithi152.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44608.log.gz 2024-04-20T11:39:07.639 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63069.log 2024-04-20T11:39:07.643 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.29211.log: /var/log/ceph/ceph-client.admin.63069.log: 87.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66354.log 2024-04-20T11:39:07.643 INFO:teuthology.orchestra.run.smithi152.stderr: -- replaced with /var/log/ceph/ceph-client.admin.29211.log.gz 2024-04-20T11:39:07.644 INFO:teuthology.orchestra.run.smithi152.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.63069.log.gz 2024-04-20T11:39:07.645 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.66354.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.66354.log.gz 2024-04-20T11:39:07.646 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71788.log 2024-04-20T11:39:07.649 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.71788.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31917.log 2024-04-20T11:39:07.649 INFO:teuthology.orchestra.run.smithi152.stderr: 92.9% -- replaced with /var/log/ceph/ceph-client.0.54476.log.gz 2024-04-20T11:39:07.649 INFO:teuthology.orchestra.run.smithi152.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.71788.log.gz 2024-04-20T11:39:07.652 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58220.log 2024-04-20T11:39:07.653 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.31917.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76872.log 2024-04-20T11:39:07.655 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.58220.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.58220.log.gz 2024-04-20T11:39:07.655 INFO:teuthology.orchestra.run.smithi152.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.31917.log.gz 2024-04-20T11:39:07.657 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48503.log 2024-04-20T11:39:07.658 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.76872.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.76872.log.gz 2024-04-20T11:39:07.658 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69837.log 2024-04-20T11:39:07.659 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.48503.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48503.log.gz 2024-04-20T11:39:07.660 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.69837.log: /var/log/ceph/ceph-client.admin.50214.log 2024-04-20T11:39:07.661 INFO:teuthology.orchestra.run.smithi152.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.69837.log.gz 2024-04-20T11:39:07.662 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60446.log 2024-04-20T11:39:07.664 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.50214.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50214.log.gz 2024-04-20T11:39:07.664 INFO:teuthology.orchestra.run.smithi152.stderr:gzip/var/log/ceph/ceph-client.admin.60446.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.72424.log 2024-04-20T11:39:07.664 INFO:teuthology.orchestra.run.smithi152.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.60446.log.gz 2024-04-20T11:39:07.665 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76068.log 2024-04-20T11:39:07.667 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.72424.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.72424.log.gz 2024-04-20T11:39:07.668 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.76068.log: /var/log/ceph/ceph-client.admin.46544.log 2024-04-20T11:39:07.668 INFO:teuthology.orchestra.run.smithi175.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-04-20T11:39:07.669 INFO:teuthology.orchestra.run.smithi152.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.76068.log.gz 2024-04-20T11:39:07.669 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64999.log 2024-04-20T11:39:07.671 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.46544.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46544.log.gz 2024-04-20T11:39:07.671 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36023.log 2024-04-20T11:39:07.672 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.64999.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.64999.log.gz 93.8% 2024-04-20T11:39:07.672 INFO:teuthology.orchestra.run.smithi152.stderr: -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-04-20T11:39:07.673 INFO:teuthology.orchestra.run.smithi175.stderr: 2024-04-20T11:39:07.673 INFO:teuthology.orchestra.run.smithi175.stderr:real 0m1.009s 2024-04-20T11:39:07.673 INFO:teuthology.orchestra.run.smithi175.stderr:user 0m4.939s 2024-04-20T11:39:07.673 INFO:teuthology.orchestra.run.smithi175.stderr:sys 0m0.513s 2024-04-20T11:39:07.675 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72955.log 2024-04-20T11:39:07.675 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.36023.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.36023.log.gz 2024-04-20T11:39:07.675 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33936.log 2024-04-20T11:39:07.676 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.72955.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32994.log 2024-04-20T11:39:07.676 INFO:teuthology.orchestra.run.smithi152.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.72955.log.gz 2024-04-20T11:39:07.677 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.33936.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52434.log 2024-04-20T11:39:07.678 INFO:teuthology.orchestra.run.smithi152.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.33936.log.gz 2024-04-20T11:39:07.678 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.32994.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.32994.log.gz 2024-04-20T11:39:07.679 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33852.log 2024-04-20T11:39:07.679 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.52434.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52434.log.gz 2024-04-20T11:39:07.680 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39308.log 2024-04-20T11:39:07.681 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.33852.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35879.log 2024-04-20T11:39:07.681 INFO:teuthology.orchestra.run.smithi152.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.33852.log.gz 2024-04-20T11:39:07.682 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.39308.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.39308.log.gz 2024-04-20T11:39:07.684 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.35879.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.35879.log.gz 2024-04-20T11:39:07.684 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49706.log 2024-04-20T11:39:07.685 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59920.log 2024-04-20T11:39:07.687 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.49706.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45993.log 2024-04-20T11:39:07.687 INFO:teuthology.orchestra.run.smithi152.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49706.log.gz 2024-04-20T11:39:07.688 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.59920.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.60265.log 2024-04-20T11:39:07.689 INFO:teuthology.orchestra.run.smithi152.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.59920.log.gz 2024-04-20T11:39:07.689 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.45993.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45993.log.gz 2024-04-20T11:39:07.692 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.0.60265.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65408.log 2024-04-20T11:39:07.694 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44445.log 2024-04-20T11:39:07.694 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.65408.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37075.log 2024-04-20T11:39:07.695 INFO:teuthology.orchestra.run.smithi152.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.65408.log.gz 2024-04-20T11:39:07.695 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37014.log 2024-04-20T11:39:07.698 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.44445.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44445.log.gz 2024-04-20T11:39:07.698 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.37014.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65654.log 2024-04-20T11:39:07.699 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.37075.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.37075.log.gz 2024-04-20T11:39:07.699 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53612.log 2024-04-20T11:39:07.700 INFO:teuthology.orchestra.run.smithi152.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.37014.log.gz 2024-04-20T11:39:07.702 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.65654.log: /var/log/ceph/ceph-client.admin.53612.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.65654.log.gz 2024-04-20T11:39:07.702 INFO:teuthology.orchestra.run.smithi152.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53612.log.gz 2024-04-20T11:39:07.703 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54960.log 2024-04-20T11:39:07.704 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52957.log 2024-04-20T11:39:07.706 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.54960.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54960.log.gz 2024-04-20T11:39:07.706 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40085.log 2024-04-20T11:39:07.707 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.52957.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52957.log.gz 2024-04-20T11:39:07.708 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42365.log 2024-04-20T11:39:07.708 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.40085.log: 92.3% -- replaced with /var/log/ceph/ceph-client.0.60265.log.gz 2024-04-20T11:39:07.709 INFO:teuthology.orchestra.run.smithi152.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.40085.log.gz 2024-04-20T11:39:07.709 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76404.log 2024-04-20T11:39:07.710 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.42365.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22975.log 2024-04-20T11:39:07.711 INFO:teuthology.orchestra.run.smithi152.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42365.log.gz 2024-04-20T11:39:07.712 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.76404.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33668.log 2024-04-20T11:39:07.712 INFO:teuthology.orchestra.run.smithi152.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.76404.log.gz 2024-04-20T11:39:07.713 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.22975.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55434.log 2024-04-20T11:39:07.713 INFO:teuthology.orchestra.run.smithi152.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.22975.log.gz 2024-04-20T11:39:07.714 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.33668.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38399.log 2024-04-20T11:39:07.715 INFO:teuthology.orchestra.run.smithi152.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.33668.log.gz 2024-04-20T11:39:07.716 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.55434.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30018.log 2024-04-20T11:39:07.716 INFO:teuthology.orchestra.run.smithi152.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.55434.log.gz 2024-04-20T11:39:07.717 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.38399.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44897.log 2024-04-20T11:39:07.717 INFO:teuthology.orchestra.run.smithi152.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.38399.log.gz 2024-04-20T11:39:07.718 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.30018.log: 89.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41053.log 2024-04-20T11:39:07.718 INFO:teuthology.orchestra.run.smithi152.stderr: -- replaced with /var/log/ceph/ceph-client.admin.30018.log.gz 2024-04-20T11:39:07.720 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.44897.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76362.log 2024-04-20T11:39:07.720 INFO:teuthology.orchestra.run.smithi152.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44897.log.gz 2024-04-20T11:39:07.721 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.41053.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41053.log.gz 2024-04-20T11:39:07.721 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50904.log 2024-04-20T11:39:07.722 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.76362.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.76362.log.gz 2024-04-20T11:39:07.722 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41420.log 2024-04-20T11:39:07.724 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.50904.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29808.log 2024-04-20T11:39:07.724 INFO:teuthology.orchestra.run.smithi152.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.50904.log.gz 2024-04-20T11:39:07.725 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.41420.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.41420.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60998.log 2024-04-20T11:39:07.725 INFO:teuthology.orchestra.run.smithi152.stderr: 2024-04-20T11:39:07.726 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.29808.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.29808.log.gz 2024-04-20T11:39:07.726 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67497.log 2024-04-20T11:39:07.728 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.60998.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63482.log 2024-04-20T11:39:07.728 INFO:teuthology.orchestra.run.smithi152.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.60998.log.gz 2024-04-20T11:39:07.729 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.67497.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72913.log 2024-04-20T11:39:07.729 INFO:teuthology.orchestra.run.smithi152.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.67497.log.gz 2024-04-20T11:39:07.730 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.63482.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34434.log 2024-04-20T11:39:07.730 INFO:teuthology.orchestra.run.smithi152.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.63482.log.gz 2024-04-20T11:39:07.732 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.72913.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.72913.log.gz 2024-04-20T11:39:07.732 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5/var/log/ceph/ceph-client.admin.34434.log: --verbose -- /var/log/ceph/ceph-client.admin.34998.log 2024-04-20T11:39:07.733 INFO:teuthology.orchestra.run.smithi152.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34434.log.gz 2024-04-20T11:39:07.733 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68541.log 2024-04-20T11:39:07.734 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.34998.log: 93.0% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-04-20T11:39:07.735 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39584.log 2024-04-20T11:39:07.735 INFO:teuthology.orchestra.run.smithi152.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34998.log.gz 2024-04-20T11:39:07.736 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.68541.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39434.log 2024-04-20T11:39:07.736 INFO:teuthology.orchestra.run.smithi152.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.68541.log.gz 2024-04-20T11:39:07.737 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.39584.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52644.log 2024-04-20T11:39:07.737 INFO:teuthology.orchestra.run.smithi152.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.39584.log.gz 2024-04-20T11:39:07.739 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.39434.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65081.log 2024-04-20T11:39:07.739 INFO:teuthology.orchestra.run.smithi152.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.39434.log.gz 2024-04-20T11:39:07.740 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.52644.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63524.log 2024-04-20T11:39:07.740 INFO:teuthology.orchestra.run.smithi152.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52644.log.gz 2024-04-20T11:39:07.741 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.65081.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57321.log 2024-04-20T11:39:07.741 INFO:teuthology.orchestra.run.smithi152.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.65081.log.gz 2024-04-20T11:39:07.742 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.63524.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35665.log 2024-04-20T11:39:07.743 INFO:teuthology.orchestra.run.smithi152.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.63524.log.gz 2024-04-20T11:39:07.744 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.57321.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73225.log 2024-04-20T11:39:07.744 INFO:teuthology.orchestra.run.smithi152.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.57321.log.gz 2024-04-20T11:39:07.744 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.35665.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43539.log 2024-04-20T11:39:07.745 INFO:teuthology.orchestra.run.smithi152.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.35665.log.gz 2024-04-20T11:39:07.746 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.73225.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38842.log 2024-04-20T11:39:07.746 INFO:teuthology.orchestra.run.smithi152.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.73225.log.gz 2024-04-20T11:39:07.747 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.43539.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59185.log 2024-04-20T11:39:07.747 INFO:teuthology.orchestra.run.smithi152.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43539.log.gz 2024-04-20T11:39:07.748 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.38842.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39350.log 2024-04-20T11:39:07.748 INFO:teuthology.orchestra.run.smithi152.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.38842.log.gz 2024-04-20T11:39:07.750 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.59185.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59399.log 2024-04-20T11:39:07.750 INFO:teuthology.orchestra.run.smithi152.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.59185.log.gz 2024-04-20T11:39:07.751 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.39350.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47028.log 2024-04-20T11:39:07.751 INFO:teuthology.orchestra.run.smithi152.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.39350.log.gz 2024-04-20T11:39:07.752 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.59399.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54741.log 2024-04-20T11:39:07.752 INFO:teuthology.orchestra.run.smithi152.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.59399.log.gz 2024-04-20T11:39:07.753 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.47028.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77770.log 2024-04-20T11:39:07.753 INFO:teuthology.orchestra.run.smithi152.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.47028.log.gz 2024-04-20T11:39:07.755 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.54741.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63155.log 2024-04-20T11:39:07.755 INFO:teuthology.orchestra.run.smithi152.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.54741.log.gz 2024-04-20T11:39:07.755 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.77770.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43497.log 2024-04-20T11:39:07.756 INFO:teuthology.orchestra.run.smithi152.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.77770.log.gz 2024-04-20T11:39:07.757 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.63155.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62941.log 2024-04-20T11:39:07.757 INFO:teuthology.orchestra.run.smithi152.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.63155.log.gz 2024-04-20T11:39:07.758 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.43497.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77604.log 2024-04-20T11:39:07.758 INFO:teuthology.orchestra.run.smithi152.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43497.log.gz 2024-04-20T11:39:07.759 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.62941.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41648.log 2024-04-20T11:39:07.759 INFO:teuthology.orchestra.run.smithi152.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.62941.log.gz 2024-04-20T11:39:07.761 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.77604.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30186.log 2024-04-20T11:39:07.761 INFO:teuthology.orchestra.run.smithi152.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.77604.log.gz 2024-04-20T11:39:07.762 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.41648.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28027.log 2024-04-20T11:39:07.762 INFO:teuthology.orchestra.run.smithi152.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.41648.log.gz 2024-04-20T11:39:07.763 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.30186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72586.log 2024-04-20T11:39:07.763 INFO:teuthology.orchestra.run.smithi152.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.30186.log.gz 2024-04-20T11:39:07.764 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.28027.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66946.log 2024-04-20T11:39:07.764 INFO:teuthology.orchestra.run.smithi152.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.28027.log.gz 2024-04-20T11:39:07.765 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.72586.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71245.log 2024-04-20T11:39:07.766 INFO:teuthology.orchestra.run.smithi152.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.72586.log.gz 2024-04-20T11:39:07.766 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.66946.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39266.log 2024-04-20T11:39:07.767 INFO:teuthology.orchestra.run.smithi152.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.66946.log.gz 2024-04-20T11:39:07.767 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.71245.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63692.log 2024-04-20T11:39:07.768 INFO:teuthology.orchestra.run.smithi152.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.71245.log.gz 2024-04-20T11:39:07.769 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.39266.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57237.log 2024-04-20T11:39:07.769 INFO:teuthology.orchestra.run.smithi152.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.39266.log.gz 2024-04-20T11:39:07.770 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.63692.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28422.log 2024-04-20T11:39:07.771 INFO:teuthology.orchestra.run.smithi152.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.63692.log.gz 2024-04-20T11:39:07.772 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.57237.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63199.log 2024-04-20T11:39:07.772 INFO:teuthology.orchestra.run.smithi152.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.57237.log.gz 2024-04-20T11:39:07.773 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.28422.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35981.log 2024-04-20T11:39:07.773 INFO:teuthology.orchestra.run.smithi152.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.28422.log.gz 2024-04-20T11:39:07.774 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.63199.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28380.log 2024-04-20T11:39:07.774 INFO:teuthology.orchestra.run.smithi152.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.63199.log.gz 2024-04-20T11:39:07.775 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.35981.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48587.log 2024-04-20T11:39:07.776 INFO:teuthology.orchestra.run.smithi152.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.35981.log.gz 2024-04-20T11:39:07.777 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.28380.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65939.log 2024-04-20T11:39:07.777 INFO:teuthology.orchestra.run.smithi152.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.28380.log.gz 2024-04-20T11:39:07.778 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.48587.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58472.log 2024-04-20T11:39:07.778 INFO:teuthology.orchestra.run.smithi152.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48587.log.gz 2024-04-20T11:39:07.779 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.65939.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29227.log 2024-04-20T11:39:07.779 INFO:teuthology.orchestra.run.smithi152.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.65939.log.gz 2024-04-20T11:39:07.780 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.58472.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46758.log 2024-04-20T11:39:07.780 INFO:teuthology.orchestra.run.smithi152.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.58472.log.gz 2024-04-20T11:39:07.782 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.29227.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74168.log 2024-04-20T11:39:07.782 INFO:teuthology.orchestra.run.smithi152.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.29227.log.gz 2024-04-20T11:39:07.783 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.46758.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76549.log 2024-04-20T11:39:07.783 INFO:teuthology.orchestra.run.smithi152.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.46758.log.gz 2024-04-20T11:39:07.784 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.74168.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44855.log 2024-04-20T11:39:07.784 INFO:teuthology.orchestra.run.smithi152.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.74168.log.gz 2024-04-20T11:39:07.785 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.76549.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54876.log 2024-04-20T11:39:07.786 INFO:teuthology.orchestra.run.smithi152.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.76549.log.gz 2024-04-20T11:39:07.786 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.44855.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47763.log 2024-04-20T11:39:07.786 INFO:teuthology.orchestra.run.smithi152.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44855.log.gz 2024-04-20T11:39:07.787 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.54876.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39140.log 2024-04-20T11:39:07.788 INFO:teuthology.orchestra.run.smithi152.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.54876.log.gz 2024-04-20T11:39:07.789 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.47763.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.47763.log.gz 2024-04-20T11:39:07.789 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64794.log 2024-04-20T11:39:07.790 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.39140.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.39140.log.gz 2024-04-20T11:39:07.790 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61094.log 2024-04-20T11:39:07.792 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.64794.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61180.log 2024-04-20T11:39:07.792 INFO:teuthology.orchestra.run.smithi152.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.64794.log.gz 2024-04-20T11:39:07.792 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.61094.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67791.log 2024-04-20T11:39:07.793 INFO:teuthology.orchestra.run.smithi152.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.61094.log.gz 2024-04-20T11:39:07.794 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.61180.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68334.log 2024-04-20T11:39:07.794 INFO:teuthology.orchestra.run.smithi152.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.61180.log.gz 2024-04-20T11:39:07.795 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.67791.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62037.log 2024-04-20T11:39:07.796 INFO:teuthology.orchestra.run.smithi152.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.67791.log.gz 2024-04-20T11:39:07.796 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.68334.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52915.log 2024-04-20T11:39:07.797 INFO:teuthology.orchestra.run.smithi152.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.68334.log.gz 2024-04-20T11:39:07.798 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.62037.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53964.log 2024-04-20T11:39:07.798 INFO:teuthology.orchestra.run.smithi152.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.62037.log.gz 2024-04-20T11:39:07.799 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.52915.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37895.log 2024-04-20T11:39:07.799 INFO:teuthology.orchestra.run.smithi152.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.52915.log.gz 2024-04-20T11:39:07.800 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.53964.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62518.log 2024-04-20T11:39:07.800 INFO:teuthology.orchestra.run.smithi152.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.53964.log.gz 2024-04-20T11:39:07.801 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.37895.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59579.log 2024-04-20T11:39:07.801 INFO:teuthology.orchestra.run.smithi152.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.37895.log.gz 2024-04-20T11:39:07.803 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.62518.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50652.log 2024-04-20T11:39:07.803 INFO:teuthology.orchestra.run.smithi152.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.62518.log.gz 2024-04-20T11:39:07.803 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.59579.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57605.log 2024-04-20T11:39:07.804 INFO:teuthology.orchestra.run.smithi152.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.59579.log.gz 2024-04-20T11:39:07.805 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.50652.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43623.log 2024-04-20T11:39:07.805 INFO:teuthology.orchestra.run.smithi152.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.50652.log.gz 2024-04-20T11:39:07.806 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.57605.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44148.log 2024-04-20T11:39:07.806 INFO:teuthology.orchestra.run.smithi152.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.57605.log.gz 2024-04-20T11:39:07.807 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.43623.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43102.log 2024-04-20T11:39:07.807 INFO:teuthology.orchestra.run.smithi152.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43623.log.gz 2024-04-20T11:39:07.809 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.44148.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34392.log 2024-04-20T11:39:07.809 INFO:teuthology.orchestra.run.smithi152.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44148.log.gz 2024-04-20T11:39:07.810 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.43102.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65570.log 2024-04-20T11:39:07.810 INFO:teuthology.orchestra.run.smithi152.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43102.log.gz 2024-04-20T11:39:07.811 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.34392.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55560.log 2024-04-20T11:39:07.811 INFO:teuthology.orchestra.run.smithi152.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.34392.log.gz 2024-04-20T11:39:07.812 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.65570.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65813.log 2024-04-20T11:39:07.812 INFO:teuthology.orchestra.run.smithi152.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.65570.log.gz 2024-04-20T11:39:07.813 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.55560.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62781.log 2024-04-20T11:39:07.813 INFO:teuthology.orchestra.run.smithi152.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.55560.log.gz 2024-04-20T11:39:07.815 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.65813.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46417.log 2024-04-20T11:39:07.815 INFO:teuthology.orchestra.run.smithi152.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.65813.log.gz 2024-04-20T11:39:07.816 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.62781.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37610.log 2024-04-20T11:39:07.816 INFO:teuthology.orchestra.run.smithi152.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.62781.log.gz 2024-04-20T11:39:07.817 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.46417.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29232.log 2024-04-20T11:39:07.817 INFO:teuthology.orchestra.run.smithi152.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.46417.log.gz 2024-04-20T11:39:07.818 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.37610.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73737.log 2024-04-20T11:39:07.818 INFO:teuthology.orchestra.run.smithi152.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37610.log.gz 2024-04-20T11:39:07.819 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.29232.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76830.log 2024-04-20T11:39:07.819 INFO:teuthology.orchestra.run.smithi152.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.29232.log.gz 2024-04-20T11:39:07.820 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.73737.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42533.log 2024-04-20T11:39:07.821 INFO:teuthology.orchestra.run.smithi152.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.73737.log.gz 2024-04-20T11:39:07.821 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.76830.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77208.log 2024-04-20T11:39:07.822 INFO:teuthology.orchestra.run.smithi152.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.76830.log.gz 2024-04-20T11:39:07.823 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.42533.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69585.log 2024-04-20T11:39:07.823 INFO:teuthology.orchestra.run.smithi152.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42533.log.gz 2024-04-20T11:39:07.824 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.77208.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75051.log 2024-04-20T11:39:07.824 INFO:teuthology.orchestra.run.smithi152.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.77208.log.gz 2024-04-20T11:39:07.825 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.69585.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37853.log 2024-04-20T11:39:07.825 INFO:teuthology.orchestra.run.smithi152.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.69585.log.gz 2024-04-20T11:39:07.827 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.75051.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63111.log 2024-04-20T11:39:07.827 INFO:teuthology.orchestra.run.smithi152.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.75051.log.gz 2024-04-20T11:39:07.828 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.37853.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34896.log 2024-04-20T11:39:07.828 INFO:teuthology.orchestra.run.smithi152.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.37853.log.gz 2024-04-20T11:39:07.829 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.63111.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45185.log 2024-04-20T11:39:07.829 INFO:teuthology.orchestra.run.smithi152.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.63111.log.gz 2024-04-20T11:39:07.830 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.34896.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71053.log 2024-04-20T11:39:07.830 INFO:teuthology.orchestra.run.smithi152.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34896.log.gz 2024-04-20T11:39:07.832 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.45185.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53750.log 2024-04-20T11:39:07.832 INFO:teuthology.orchestra.run.smithi152.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45185.log.gz 2024-04-20T11:39:07.832 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.71053.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33282.log 2024-04-20T11:39:07.833 INFO:teuthology.orchestra.run.smithi152.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.71053.log.gz 2024-04-20T11:39:07.834 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.53750.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77352.log 2024-04-20T11:39:07.834 INFO:teuthology.orchestra.run.smithi152.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.53750.log.gz 2024-04-20T11:39:07.835 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.33282.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72019.log 2024-04-20T11:39:07.835 INFO:teuthology.orchestra.run.smithi152.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.33282.log.gz 2024-04-20T11:39:07.836 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.77352.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69459.log 2024-04-20T11:39:07.836 INFO:teuthology.orchestra.run.smithi152.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.77352.log.gz 2024-04-20T11:39:07.838 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.72019.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70089.log 2024-04-20T11:39:07.838 INFO:teuthology.orchestra.run.smithi152.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.72019.log.gz 2024-04-20T11:39:07.839 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.69459.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43325.log 2024-04-20T11:39:07.839 INFO:teuthology.orchestra.run.smithi152.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.69459.log.gz 2024-04-20T11:39:07.840 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.70089.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43731.log 2024-04-20T11:39:07.840 INFO:teuthology.orchestra.run.smithi152.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.70089.log.gz 2024-04-20T11:39:07.841 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.43325.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32505.log 2024-04-20T11:39:07.841 INFO:teuthology.orchestra.run.smithi152.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.43325.log.gz 2024-04-20T11:39:07.842 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.43731.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49622.log 2024-04-20T11:39:07.843 INFO:teuthology.orchestra.run.smithi152.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43731.log.gz 2024-04-20T11:39:07.844 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.32505.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49412.log 2024-04-20T11:39:07.844 INFO:teuthology.orchestra.run.smithi152.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.32505.log.gz 2024-04-20T11:39:07.845 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.49622.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44232.log 2024-04-20T11:39:07.845 INFO:teuthology.orchestra.run.smithi152.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49622.log.gz 2024-04-20T11:39:07.846 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.49412.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70268.log 2024-04-20T11:39:07.846 INFO:teuthology.orchestra.run.smithi152.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.49412.log.gz 2024-04-20T11:39:07.847 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.44232.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32043.log 2024-04-20T11:39:07.847 INFO:teuthology.orchestra.run.smithi152.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44232.log.gz 2024-04-20T11:39:07.849 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.70268.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38105.log 2024-04-20T11:39:07.849 INFO:teuthology.orchestra.run.smithi152.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.70268.log.gz 2024-04-20T11:39:07.850 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.32043.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42852.log 2024-04-20T11:39:07.850 INFO:teuthology.orchestra.run.smithi152.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.32043.log.gz 2024-04-20T11:39:07.851 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.38105.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54699.log 2024-04-20T11:39:07.851 INFO:teuthology.orchestra.run.smithi152.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.38105.log.gz 2024-04-20T11:39:07.852 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.42852.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62861.log 2024-04-20T11:39:07.852 INFO:teuthology.orchestra.run.smithi152.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42852.log.gz 2024-04-20T11:39:07.853 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.54699.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63250.log 2024-04-20T11:39:07.853 INFO:teuthology.orchestra.run.smithi152.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.54699.log.gz 2024-04-20T11:39:07.855 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.62861.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38147.log 2024-04-20T11:39:07.855 INFO:teuthology.orchestra.run.smithi152.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.62861.log.gz 2024-04-20T11:39:07.856 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.63250.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58514.log 2024-04-20T11:39:07.856 INFO:teuthology.orchestra.run.smithi152.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.63250.log.gz 2024-04-20T11:39:07.857 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.38147.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66209.log 2024-04-20T11:39:07.857 INFO:teuthology.orchestra.run.smithi152.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.38147.log.gz 2024-04-20T11:39:07.858 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.58514.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51613.log 2024-04-20T11:39:07.858 INFO:teuthology.orchestra.run.smithi152.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.58514.log.gz 2024-04-20T11:39:07.859 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.66209.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53656.log 2024-04-20T11:39:07.860 INFO:teuthology.orchestra.run.smithi152.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.66209.log.gz 2024-04-20T11:39:07.861 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.51613.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68418.log 2024-04-20T11:39:07.861 INFO:teuthology.orchestra.run.smithi152.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51613.log.gz 2024-04-20T11:39:07.862 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.53656.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69921.log 2024-04-20T11:39:07.862 INFO:teuthology.orchestra.run.smithi152.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53656.log.gz 2024-04-20T11:39:07.863 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.68418.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67161.log 2024-04-20T11:39:07.863 INFO:teuthology.orchestra.run.smithi152.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.68418.log.gz 2024-04-20T11:39:07.865 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.69921.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50694.log 2024-04-20T11:39:07.865 INFO:teuthology.orchestra.run.smithi152.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.69921.log.gz 2024-04-20T11:39:07.866 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.67161.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65204.log 2024-04-20T11:39:07.866 INFO:teuthology.orchestra.run.smithi152.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.67161.log.gz 2024-04-20T11:39:07.867 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.50694.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38357.log 2024-04-20T11:39:07.867 INFO:teuthology.orchestra.run.smithi152.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50694.log.gz 2024-04-20T11:39:07.868 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.65204.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58019.log 2024-04-20T11:39:07.868 INFO:teuthology.orchestra.run.smithi152.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.65204.log.gz 2024-04-20T11:39:07.869 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.38357.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53526.log 2024-04-20T11:39:07.870 INFO:teuthology.orchestra.run.smithi152.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.38357.log.gz 2024-04-20T11:39:07.871 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.58019.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70616.log 2024-04-20T11:39:07.871 INFO:teuthology.orchestra.run.smithi152.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.58019.log.gz 2024-04-20T11:39:07.872 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.53526.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34518.log 2024-04-20T11:39:07.872 INFO:teuthology.orchestra.run.smithi152.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.53526.log.gz 2024-04-20T11:39:07.873 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.70616.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59709.log 2024-04-20T11:39:07.873 INFO:teuthology.orchestra.run.smithi152.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.70616.log.gz 2024-04-20T11:39:07.874 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.34518.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.51248.log 2024-04-20T11:39:07.874 INFO:teuthology.orchestra.run.smithi152.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.34518.log.gz 2024-04-20T11:39:07.875 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.59709.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52831.log 2024-04-20T11:39:07.876 INFO:teuthology.orchestra.run.smithi152.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.59709.log.gz 2024-04-20T11:39:07.877 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.0.51248.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61506.log 2024-04-20T11:39:07.877 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.52831.log: 90.5% -- replaced with /var/log/ceph/ceph-client.0.51248.log.gz 2024-04-20T11:39:07.878 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32952.log 2024-04-20T11:39:07.878 INFO:teuthology.orchestra.run.smithi152.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52831.log.gz 2024-04-20T11:39:07.879 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.61506.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64712.log 2024-04-20T11:39:07.879 INFO:teuthology.orchestra.run.smithi152.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.61506.log.gz 2024-04-20T11:39:07.880 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.32952.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33036.log 2024-04-20T11:39:07.880 INFO:teuthology.orchestra.run.smithi152.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.32952.log.gz 2024-04-20T11:39:07.881 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.64712.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63608.log 2024-04-20T11:39:07.882 INFO:teuthology.orchestra.run.smithi152.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.64712.log.gz 2024-04-20T11:39:07.883 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.33036.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65040.log 2024-04-20T11:39:07.883 INFO:teuthology.orchestra.run.smithi152.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.33036.log.gz 2024-04-20T11:39:07.884 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.63608.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29892.log 2024-04-20T11:39:07.884 INFO:teuthology.orchestra.run.smithi152.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.63608.log.gz 2024-04-20T11:39:07.885 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.65040.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.65040.log.gz 2024-04-20T11:39:07.885 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59793.log 2024-04-20T11:39:07.886 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.29892.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29766.log 2024-04-20T11:39:07.886 INFO:teuthology.orchestra.run.smithi152.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.29892.log.gz 2024-04-20T11:39:07.887 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.59793.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66904.log 2024-04-20T11:39:07.888 INFO:teuthology.orchestra.run.smithi152.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.59793.log.gz 2024-04-20T11:39:07.889 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.29766.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70490.log 2024-04-20T11:39:07.889 INFO:teuthology.orchestra.run.smithi152.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.29766.log.gz 2024-04-20T11:39:07.890 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.66904.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69753.log 2024-04-20T11:39:07.890 INFO:teuthology.orchestra.run.smithi152.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.66904.log.gz 2024-04-20T11:39:07.891 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.70490.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.70490.log.gz 2024-04-20T11:39:07.892 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.69753.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55044.log 2024-04-20T11:39:07.893 INFO:teuthology.orchestra.run.smithi152.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.69753.log.gz 2024-04-20T11:39:07.893 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-04-20T11:39:07.894 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.55044.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48851.log 2024-04-20T11:39:07.895 INFO:teuthology.orchestra.run.smithi152.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.55044.log.gz 2024-04-20T11:39:07.896 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42407.log 2024-04-20T11:39:07.897 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.48851.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40185.log 2024-04-20T11:39:07.898 INFO:teuthology.orchestra.run.smithi152.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.48851.log.gz 2024-04-20T11:39:07.898 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.42407.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50257.log 2024-04-20T11:39:07.898 INFO:teuthology.orchestra.run.smithi152.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.42407.log.gz 2024-04-20T11:39:07.900 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.40185.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32001.log 2024-04-20T11:39:07.900 INFO:teuthology.orchestra.run.smithi152.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.40185.log.gz 2024-04-20T11:39:07.902 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.50257.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71704.log 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50257.log.gz 2024-04-20T11:39:07.902 INFO:teuthology.orchestra.run.smithi152.stderr: 2024-04-20T11:39:07.902 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.32001.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.32001.log.gz 2024-04-20T11:39:07.902 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31791.log 2024-04-20T11:39:07.904 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.71704.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50778.log 2024-04-20T11:39:07.904 INFO:teuthology.orchestra.run.smithi152.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.71704.log.gz 2024-04-20T11:39:07.905 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.31791.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52065.log 2024-04-20T11:39:07.905 INFO:teuthology.orchestra.run.smithi152.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.31791.log.gz 2024-04-20T11:39:07.906 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.50778.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.50778.log.gz 2024-04-20T11:39:07.907 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33198.log 2024-04-20T11:39:07.908 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.52065.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52065.log.gz 2024-04-20T11:39:07.908 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40843.log 2024-04-20T11:39:07.909 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.33198.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61422.log 2024-04-20T11:39:07.910 INFO:teuthology.orchestra.run.smithi152.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.33198.log.gz 2024-04-20T11:39:07.911 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.40843.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.40843.log.gz 2024-04-20T11:39:07.911 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64753.log 2024-04-20T11:39:07.912 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.61422.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73860.log 2024-04-20T11:39:07.913 INFO:teuthology.orchestra.run.smithi152.stderr: 89.1%/var/log/ceph/ceph-client.admin.64753.log: -- replaced with /var/log/ceph/ceph-client.admin.61422.log.gz 2024-04-20T11:39:07.914 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- 87.9% -- replaced with /var/log/ceph/ceph-client.admin.64753.log.gz 2024-04-20T11:39:07.914 INFO:teuthology.orchestra.run.smithi152.stderr: /var/log/ceph/ceph-client.admin.43190.log 2024-04-20T11:39:07.915 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.73860.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.73860.log.gz 2024-04-20T11:39:07.915 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32253.log 2024-04-20T11:39:07.917 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.43190.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37238.log 2024-04-20T11:39:07.917 INFO:teuthology.orchestra.run.smithi152.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43190.log.gz 2024-04-20T11:39:07.919 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.32253.log: gzip 89.3% -- replaced with /var/log/ceph/ceph-client.admin.32253.log.gz 2024-04-20T11:39:07.919 INFO:teuthology.orchestra.run.smithi152.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.48419.log 2024-04-20T11:39:07.919 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.37238.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.37238.log.gz 2024-04-20T11:39:07.920 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48335.log 2024-04-20T11:39:07.921 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.48419.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37769.log 2024-04-20T11:39:07.921 INFO:teuthology.orchestra.run.smithi152.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48419.log.gz 2024-04-20T11:39:07.922 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.48335.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75909.log 2024-04-20T11:39:07.922 INFO:teuthology.orchestra.run.smithi152.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48335.log.gz 2024-04-20T11:39:07.923 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.37769.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.37769.log.gz 2024-04-20T11:39:07.924 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74031.log 2024-04-20T11:39:07.925 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.75909.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64671.log 2024-04-20T11:39:07.925 INFO:teuthology.orchestra.run.smithi152.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.75909.log.gz 2024-04-20T11:39:07.926 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.74031.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58719.log 2024-04-20T11:39:07.926 INFO:teuthology.orchestra.run.smithi152.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.74031.log.gz 2024-04-20T11:39:07.928 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.64671.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.64671.log.gz 2024-04-20T11:39:07.929 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5/var/log/ceph/ceph-client.admin.58719.log: --verbose -- /var/log/ceph/ceph-client.admin.48671.log 2024-04-20T11:39:07.929 INFO:teuthology.orchestra.run.smithi152.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.58719.log.gz 2024-04-20T11:39:07.929 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62205.log 2024-04-20T11:39:07.931 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.48671.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33450.log 2024-04-20T11:39:07.931 INFO:teuthology.orchestra.run.smithi152.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48671.log.gz 2024-04-20T11:39:07.933 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.62205.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.62205.log.gz 2024-04-20T11:39:07.933 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74256.log 2024-04-20T11:39:07.934 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.33450.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.33450.log.gz 2024-04-20T11:39:07.935 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41546.log 2024-04-20T11:39:07.936 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.74256.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.74256.log.gz 2024-04-20T11:39:07.936 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53300.log 2024-04-20T11:39:07.938 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.41546.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29850.log 2024-04-20T11:39:07.938 INFO:teuthology.orchestra.run.smithi152.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41546.log.gz 2024-04-20T11:39:07.939 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.53300.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34728.log 2024-04-20T11:39:07.939 INFO:teuthology.orchestra.run.smithi152.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.53300.log.gz 2024-04-20T11:39:07.940 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.29850.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50820.log 2024-04-20T11:39:07.940 INFO:teuthology.orchestra.run.smithi152.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.29850.log.gz 2024-04-20T11:39:07.941 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.34728.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40927.log 2024-04-20T11:39:07.942 INFO:teuthology.orchestra.run.smithi152.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.34728.log.gz 2024-04-20T11:39:07.943 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.50820.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58094.log 2024-04-20T11:39:07.943 INFO:teuthology.orchestra.run.smithi152.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50820.log.gz 2024-04-20T11:39:07.944 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.40927.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65366.log 2024-04-20T11:39:07.944 INFO:teuthology.orchestra.run.smithi152.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.40927.log.gz 2024-04-20T11:39:07.945 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.58094.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72099.log 2024-04-20T11:39:07.945 INFO:teuthology.orchestra.run.smithi152.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.58094.log.gz 2024-04-20T11:39:07.947 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.65366.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67857.log 2024-04-20T11:39:07.947 INFO:teuthology.orchestra.run.smithi152.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.65366.log.gz 2024-04-20T11:39:07.948 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.72099.log: 87.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55002.log 2024-04-20T11:39:07.948 INFO:teuthology.orchestra.run.smithi152.stderr: -- replaced with /var/log/ceph/ceph-client.admin.72099.log.gz 2024-04-20T11:39:07.949 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.67857.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46630.log 2024-04-20T11:39:07.949 INFO:teuthology.orchestra.run.smithi152.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.67857.log.gz 2024-04-20T11:39:07.951 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.55002.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.55002.log.gz 2024-04-20T11:39:07.951 INFO:teuthology.orchestra.run.smithi152.stderr:gzip/var/log/ceph/ceph-client.admin.46630.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.33078.log 2024-04-20T11:39:07.952 INFO:teuthology.orchestra.run.smithi152.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46630.log.gz 2024-04-20T11:39:07.952 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60530.log 2024-04-20T11:39:07.954 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.33078.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57977.log 2024-04-20T11:39:07.954 INFO:teuthology.orchestra.run.smithi152.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.33078.log.gz 2024-04-20T11:39:07.955 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.60530.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29579.log 2024-04-20T11:39:07.955 INFO:teuthology.orchestra.run.smithi152.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.60530.log.gz 2024-04-20T11:39:07.956 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.57977.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32463.log 2024-04-20T11:39:07.956 INFO:teuthology.orchestra.run.smithi152.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.57977.log.gz 2024-04-20T11:39:07.958 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.29579.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.29579.log.gz 2024-04-20T11:39:07.958 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30354.log 2024-04-20T11:39:07.959 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.32463.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.32463.log.gz 2024-04-20T11:39:07.959 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62560.log 2024-04-20T11:39:07.961 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.30354.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45596.log 2024-04-20T11:39:07.961 INFO:teuthology.orchestra.run.smithi152.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.30354.log.gz 2024-04-20T11:39:07.962 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.62560.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.62560.log.gz 2024-04-20T11:39:07.963 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41462.log/var/log/ceph/ceph-client.admin.45596.log: 2024-04-20T11:39:07.963 INFO:teuthology.orchestra.run.smithi152.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45596.log.gz 2024-04-20T11:39:07.964 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50564.log 2024-04-20T11:39:07.965 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.41462.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34140.log 2024-04-20T11:39:07.965 INFO:teuthology.orchestra.run.smithi152.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.41462.log.gz 2024-04-20T11:39:07.967 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.50564.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.50564.log.gz 2024-04-20T11:39:07.967 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32910.log 2024-04-20T11:39:07.968 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.34140.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34140.log.gz 2024-04-20T11:39:07.968 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45428.log 2024-04-20T11:39:07.970 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.32910.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68970.log 2024-04-20T11:39:07.970 INFO:teuthology.orchestra.run.smithi152.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.32910.log.gz 2024-04-20T11:39:07.971 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.45428.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49832.log 2024-04-20T11:39:07.971 INFO:teuthology.orchestra.run.smithi152.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45428.log.gz 2024-04-20T11:39:07.972 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.68970.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.68970.log.gz 2024-04-20T11:39:07.973 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.49832.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49832.log.gz 2024-04-20T11:39:07.974 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38501.log 2024-04-20T11:39:07.975 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57279.log 2024-04-20T11:39:07.977 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.38501.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66480.log 2024-04-20T11:39:07.977 INFO:teuthology.orchestra.run.smithi152.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.38501.log.gz 2024-04-20T11:39:07.978 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.57279.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.57279.log.gz 2024-04-20T11:39:07.978 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30396.log 2024-04-20T11:39:07.980 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.66480.log: 87.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71746.log 2024-04-20T11:39:07.980 INFO:teuthology.orchestra.run.smithi152.stderr: -- replaced with /var/log/ceph/ceph-client.admin.66480.log.gz 2024-04-20T11:39:07.981 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.30396.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38638.log 2024-04-20T11:39:07.981 INFO:teuthology.orchestra.run.smithi152.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.30396.log.gz 2024-04-20T11:39:07.982 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.71746.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.71746.log.gz 2024-04-20T11:39:07.983 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76446.log 2024-04-20T11:39:07.984 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.38638.log: 89.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43371.log 2024-04-20T11:39:07.984 INFO:teuthology.orchestra.run.smithi152.stderr: -- replaced with /var/log/ceph/ceph-client.admin.38638.log.gz 2024-04-20T11:39:07.985 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.76446.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74508.log 2024-04-20T11:39:07.985 INFO:teuthology.orchestra.run.smithi152.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.76446.log.gz 2024-04-20T11:39:07.986 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.43371.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43371.log.gz 2024-04-20T11:39:07.987 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52107.log 2024-04-20T11:39:07.987 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.74508.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.74508.log.gz 2024-04-20T11:39:07.988 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54657.log 2024-04-20T11:39:07.989 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.52107.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45143.log 2024-04-20T11:39:07.989 INFO:teuthology.orchestra.run.smithi152.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.52107.log.gz 2024-04-20T11:39:07.991 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.54657.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48629.log 2024-04-20T11:39:07.991 INFO:teuthology.orchestra.run.smithi152.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.54657.log.gz 2024-04-20T11:39:07.992 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.45143.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67117.log 2024-04-20T11:39:07.992 INFO:teuthology.orchestra.run.smithi152.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45143.log.gz 2024-04-20T11:39:07.993 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.48629.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77646.log 2024-04-20T11:39:07.993 INFO:teuthology.orchestra.run.smithi152.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48629.log.gz 2024-04-20T11:39:07.995 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.67117.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.67117.log.gz 2024-04-20T11:39:07.995 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.77646.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30144.log 2024-04-20T11:39:07.996 INFO:teuthology.orchestra.run.smithi152.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.77646.log.gz 2024-04-20T11:39:07.996 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52999.log 2024-04-20T11:39:07.998 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.30144.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32810.log 2024-04-20T11:39:07.998 INFO:teuthology.orchestra.run.smithi152.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.30144.log.gz 2024-04-20T11:39:07.999 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.52999.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.52999.log.gz 2024-04-20T11:39:07.999 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33492.log 2024-04-20T11:39:08.001 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.32810.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33156.log 2024-04-20T11:39:08.001 INFO:teuthology.orchestra.run.smithi152.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.32810.log.gz 2024-04-20T11:39:08.002 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.33492.log: gzip -5 --verbose -- 88.5% -- replaced with /var/log/ceph/ceph-client.admin.33492.log.gz 2024-04-20T11:39:08.002 INFO:teuthology.orchestra.run.smithi152.stderr: /var/log/ceph/ceph-client.admin.41987.log 2024-04-20T11:39:08.003 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.33156.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.33156.log.gz 2024-04-20T11:39:08.004 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34644.log 2024-04-20T11:39:08.005 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.41987.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70047.log 2024-04-20T11:39:08.005 INFO:teuthology.orchestra.run.smithi152.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.41987.log.gz 2024-04-20T11:39:08.006 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.34644.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53792.log 2024-04-20T11:39:08.006 INFO:teuthology.orchestra.run.smithi152.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34644.log.gz 2024-04-20T11:39:08.007 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.70047.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46674.log 2024-04-20T11:39:08.008 INFO:teuthology.orchestra.run.smithi152.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.70047.log.gz 2024-04-20T11:39:08.009 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.53792.log: gzip -5 --verbose 87.9% -- replaced with /var/log/ceph/ceph-client.admin.53792.log.gz 2024-04-20T11:39:08.009 INFO:teuthology.orchestra.run.smithi152.stderr: -- /var/log/ceph/ceph-client.admin.42677.log 2024-04-20T11:39:08.010 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.46674.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46674.log.gz 2024-04-20T11:39:08.011 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55663.log 2024-04-20T11:39:08.012 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.42677.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58887.log 2024-04-20T11:39:08.012 INFO:teuthology.orchestra.run.smithi152.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42677.log.gz 2024-04-20T11:39:08.013 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.55663.log: gzip -5 --verbose 88.2% -- replaced with /var/log/ceph/ceph-client.admin.55663.log.gz 2024-04-20T11:39:08.014 INFO:teuthology.orchestra.run.smithi152.stderr: -- /var/log/ceph/ceph-client.admin.62476.log 2024-04-20T11:39:08.014 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.58887.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.58887.log.gz 2024-04-20T11:39:08.015 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34020.log 2024-04-20T11:39:08.016 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.62476.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31875.log 2024-04-20T11:39:08.016 INFO:teuthology.orchestra.run.smithi152.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.62476.log.gz 2024-04-20T11:39:08.018 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.34020.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.34020.log.gz 2024-04-20T11:39:08.018 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.31875.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54090.log 2024-04-20T11:39:08.019 INFO:teuthology.orchestra.run.smithi152.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.31875.log.gz 2024-04-20T11:39:08.019 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37322.log 2024-04-20T11:39:08.021 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.54090.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28881.log 88.2% -- replaced with /var/log/ceph/ceph-client.admin.54090.log.gz 2024-04-20T11:39:08.021 INFO:teuthology.orchestra.run.smithi152.stderr: 2024-04-20T11:39:08.022 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.37322.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37322.log.gz 2024-04-20T11:39:08.022 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54132.log 2024-04-20T11:39:08.024 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.28881.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69012.log 2024-04-20T11:39:08.024 INFO:teuthology.orchestra.run.smithi152.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.28881.log.gz 2024-04-20T11:39:08.025 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.54132.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61869.log 2024-04-20T11:39:08.026 INFO:teuthology.orchestra.run.smithi152.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.54132.log.gz 2024-04-20T11:39:08.026 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.69012.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63650.log 2024-04-20T11:39:08.027 INFO:teuthology.orchestra.run.smithi152.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.69012.log.gz 2024-04-20T11:39:08.028 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.61869.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77040.log 2024-04-20T11:39:08.028 INFO:teuthology.orchestra.run.smithi152.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.61869.log.gz 2024-04-20T11:39:08.029 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.63650.log: gzip -5 --verbose -- 88.2% /var/log/ceph/ceph-client.admin.77520.log 2024-04-20T11:39:08.030 INFO:teuthology.orchestra.run.smithi152.stderr: -- replaced with /var/log/ceph/ceph-client.admin.63650.log.gz 2024-04-20T11:39:08.030 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.77040.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.77040.log.gz 2024-04-20T11:39:08.031 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28069.log 2024-04-20T11:39:08.032 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.77520.log: 87.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49328.log 2024-04-20T11:39:08.032 INFO:teuthology.orchestra.run.smithi152.stderr: -- replaced with /var/log/ceph/ceph-client.admin.77520.log.gz 2024-04-20T11:39:08.032 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.28069.log: 93.8% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-04-20T11:39:08.033 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30060.log 2024-04-20T11:39:08.033 INFO:teuthology.orchestra.run.smithi152.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.28069.log.gz 2024-04-20T11:39:08.035 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.49328.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75177.log 2024-04-20T11:39:08.035 INFO:teuthology.orchestra.run.smithi152.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.49328.log.gz 2024-04-20T11:39:08.036 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.30060.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70704.log 2024-04-20T11:39:08.036 INFO:teuthology.orchestra.run.smithi152.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.30060.log.gz 2024-04-20T11:39:08.037 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.75177.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64917.log 2024-04-20T11:39:08.037 INFO:teuthology.orchestra.run.smithi152.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.75177.log.gz 2024-04-20T11:39:08.038 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.70704.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72787.log 2024-04-20T11:39:08.038 INFO:teuthology.orchestra.run.smithi152.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.70704.log.gz 2024-04-20T11:39:08.039 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.64917.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41504.log 2024-04-20T11:39:08.040 INFO:teuthology.orchestra.run.smithi152.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.64917.log.gz 2024-04-20T11:39:08.040 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.72787.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57363.log 2024-04-20T11:39:08.041 INFO:teuthology.orchestra.run.smithi152.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.72787.log.gz 2024-04-20T11:39:08.042 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.41504.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32169.log 2024-04-20T11:39:08.042 INFO:teuthology.orchestra.run.smithi152.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41504.log.gz 2024-04-20T11:39:08.043 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.57363.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28506.log 2024-04-20T11:39:08.043 INFO:teuthology.orchestra.run.smithi152.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.57363.log.gz 2024-04-20T11:39:08.044 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.32169.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66522.log 2024-04-20T11:39:08.044 INFO:teuthology.orchestra.run.smithi152.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.32169.log.gz 2024-04-20T11:39:08.045 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.28506.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67413.log 2024-04-20T11:39:08.046 INFO:teuthology.orchestra.run.smithi152.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.28506.log.gz 2024-04-20T11:39:08.047 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.66522.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32421.log 2024-04-20T11:39:08.047 INFO:teuthology.orchestra.run.smithi152.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.66522.log.gz 2024-04-20T11:39:08.048 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.67413.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44487.log 2024-04-20T11:39:08.048 INFO:teuthology.orchestra.run.smithi152.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.67413.log.gz 2024-04-20T11:39:08.049 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.32421.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48089.log 2024-04-20T11:39:08.049 INFO:teuthology.orchestra.run.smithi152.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.32421.log.gz 2024-04-20T11:39:08.050 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.44487.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46035.log 2024-04-20T11:39:08.051 INFO:teuthology.orchestra.run.smithi152.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44487.log.gz 2024-04-20T11:39:08.051 INFO:teuthology.orchestra.run.smithi152.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-04-20T11:39:08.052 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.48089.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73562.log 2024-04-20T11:39:08.052 INFO:teuthology.orchestra.run.smithi152.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48089.log.gz 2024-04-20T11:39:08.053 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.46035.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60404.log 2024-04-20T11:39:08.053 INFO:teuthology.orchestra.run.smithi152.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.46035.log.gz 2024-04-20T11:39:08.054 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.73562.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51740.log 2024-04-20T11:39:08.054 INFO:teuthology.orchestra.run.smithi152.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.73562.log.gz 2024-04-20T11:39:08.055 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.60404.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.60404.log.gz 2024-04-20T11:39:08.055 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48809.log 2024-04-20T11:39:08.056 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.51740.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37568.log 2024-04-20T11:39:08.057 INFO:teuthology.orchestra.run.smithi152.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.51740.log.gz 2024-04-20T11:39:08.057 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.48809.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48809.log.gz 2024-04-20T11:39:08.058 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.37568.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.37568.log.gz 2024-04-20T11:39:08.201 INFO:teuthology.orchestra.run.smithi152.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-04-20T11:39:08.320 INFO:teuthology.orchestra.run.smithi152.stderr: 93.6% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-04-20T11:39:08.362 INFO:teuthology.orchestra.run.smithi152.stderr: 90.4% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-04-20T11:39:08.366 INFO:teuthology.orchestra.run.smithi152.stderr: 2024-04-20T11:39:08.366 INFO:teuthology.orchestra.run.smithi152.stderr:real 0m1.706s 2024-04-20T11:39:08.366 INFO:teuthology.orchestra.run.smithi152.stderr:user 0m6.147s 2024-04-20T11:39:08.366 INFO:teuthology.orchestra.run.smithi152.stderr:sys 0m3.003s 2024-04-20T11:39:08.367 INFO:tasks.ceph:Archiving logs... 2024-04-20T11:39:08.367 DEBUG:teuthology.misc:Transferring archived files from smithi152:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7664528/remote/smithi152/log 2024-04-20T11:39:08.368 DEBUG:teuthology.orchestra.run.smithi152:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-20T11:39:09.890 DEBUG:teuthology.misc:Transferring archived files from smithi175:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7664528/remote/smithi175/log 2024-04-20T11:39:09.891 DEBUG:teuthology.orchestra.run.smithi175:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-20T11:39:10.574 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-20T11:39:10.583 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-20T11:39:10.583 DEBUG:teuthology.orchestra.run.smithi152:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-20T11:39:10.591 DEBUG:teuthology.orchestra.run.smithi175:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-20T11:39:10.728 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-20T11:39:10.738 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-20T11:39:10.738 DEBUG:teuthology.orchestra.run.smithi152:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-20T11:39:10.746 DEBUG:teuthology.orchestra.run.smithi175:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-20T11:39:10.792 INFO:teuthology.orchestra.run.smithi152.stderr:bash: line 1: ntpq: command not found 2024-04-20T11:39:10.799 INFO:teuthology.orchestra.run.smithi175.stderr:bash: line 1: ntpq: command not found 2024-04-20T11:39:10.801 INFO:teuthology.orchestra.run.smithi152.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-20T11:39:10.802 INFO:teuthology.orchestra.run.smithi152.stdout:=============================================================================== 2024-04-20T11:39:10.802 INFO:teuthology.orchestra.run.smithi152.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 52 +699us[ +696us] +/- 72ms 2024-04-20T11:39:10.802 INFO:teuthology.orchestra.run.smithi152.stdout:^* hv02.front.sepia.ceph.com 3 6 377 51 -146us[ -149us] +/- 66ms 2024-04-20T11:39:10.802 INFO:teuthology.orchestra.run.smithi152.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 51 -575us[ -575us] +/- 90ms 2024-04-20T11:39:10.802 INFO:teuthology.orchestra.run.smithi152.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T11:39:10.809 INFO:teuthology.orchestra.run.smithi175.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-20T11:39:10.809 INFO:teuthology.orchestra.run.smithi175.stdout:=============================================================================== 2024-04-20T11:39:10.810 INFO:teuthology.orchestra.run.smithi175.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 50 +712us[ +712us] +/- 72ms 2024-04-20T11:39:10.810 INFO:teuthology.orchestra.run.smithi175.stdout:^* hv02.front.sepia.ceph.com 3 6 377 50 -158us[ -168us] +/- 66ms 2024-04-20T11:39:10.810 INFO:teuthology.orchestra.run.smithi175.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 49 -579us[ -579us] +/- 90ms 2024-04-20T11:39:10.810 INFO:teuthology.orchestra.run.smithi175.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T11:39:10.811 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-20T11:39:10.823 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-20T11:39:10.824 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-20T11:39:10.834 DEBUG:teuthology.orchestra.run.smithi152:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-20T11:39:11.048 DEBUG:teuthology.orchestra.run.smithi175:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-20T11:39:11.222 DEBUG:teuthology.orchestra.run.smithi152:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-20T11:39:11.315 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T11:39:11.316 DEBUG:teuthology.orchestra.run.smithi175:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-20T11:39:11.407 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T11:39:11.407 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-20T11:39:11.418 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-20T11:39:11.428 INFO:teuthology.task.internal:Duration was 1107.426987 seconds 2024-04-20T11:39:11.428 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-20T11:39:11.436 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-20T11:39:11.436 DEBUG:teuthology.orchestra.run.smithi152:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-20T11:39:11.445 DEBUG:teuthology.orchestra.run.smithi175:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-20T11:39:11.573 INFO:teuthology.orchestra.run.smithi175.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-20T11:39:11.582 INFO:teuthology.orchestra.run.smithi152.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-20T11:39:11.984 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-20T11:39:11.984 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi152.front.sepia.ceph.com 2024-04-20T11:39:11.984 DEBUG:teuthology.orchestra.run.smithi152:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-20T11:39:12.044 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi175.front.sepia.ceph.com 2024-04-20T11:39:12.044 DEBUG:teuthology.orchestra.run.smithi175:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-20T11:39:12.119 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-20T11:39:12.119 DEBUG:teuthology.orchestra.run.smithi152:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-20T11:39:12.126 DEBUG:teuthology.orchestra.run.smithi175:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-20T11:39:12.207 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-20T11:39:12.208 DEBUG:teuthology.orchestra.run.smithi152:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-20T11:39:12.213 DEBUG:teuthology.orchestra.run.smithi175:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-20T11:39:12.380 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-20T11:39:12.390 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-20T11:39:12.390 DEBUG:teuthology.orchestra.run.smithi152:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-20T11:39:12.471 DEBUG:teuthology.orchestra.run.smithi175:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-20T11:39:12.558 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-20T11:39:12.569 DEBUG:teuthology.orchestra.run.smithi152:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-20T11:39:12.577 DEBUG:teuthology.orchestra.run.smithi175:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-20T11:39:12.673 INFO:teuthology.orchestra.run.smithi152.stdout:kernel.core_pattern = core 2024-04-20T11:39:12.677 INFO:teuthology.orchestra.run.smithi175.stdout:kernel.core_pattern = core 2024-04-20T11:39:12.719 DEBUG:teuthology.orchestra.run.smithi152:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-20T11:39:12.766 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T11:39:12.767 DEBUG:teuthology.orchestra.run.smithi175:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-20T11:39:12.821 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T11:39:12.821 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-20T11:39:12.832 INFO:teuthology.task.internal:Transferring archived files... 2024-04-20T11:39:12.832 DEBUG:teuthology.misc:Transferring archived files from smithi152:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7664528/remote/smithi152 2024-04-20T11:39:12.833 DEBUG:teuthology.orchestra.run.smithi152:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-20T11:39:12.932 DEBUG:teuthology.misc:Transferring archived files from smithi175:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7664528/remote/smithi175 2024-04-20T11:39:12.932 DEBUG:teuthology.orchestra.run.smithi175:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-20T11:39:13.025 INFO:teuthology.task.internal:Removing archive directory... 2024-04-20T11:39:13.025 DEBUG:teuthology.orchestra.run.smithi152:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-20T11:39:13.033 DEBUG:teuthology.orchestra.run.smithi175:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-20T11:39:13.088 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-20T11:39:13.099 INFO:teuthology.task.internal:Not uploading archives. 2024-04-20T11:39:13.099 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-20T11:39:13.110 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-20T11:39:13.110 DEBUG:teuthology.orchestra.run.smithi152:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-20T11:39:13.115 DEBUG:teuthology.orchestra.run.smithi175:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-20T11:39:13.162 INFO:teuthology.orchestra.run.smithi152.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 20 11:39 /home/ubuntu/cephtest 2024-04-20T11:39:13.168 INFO:teuthology.orchestra.run.smithi175.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 20 11:39 /home/ubuntu/cephtest 2024-04-20T11:39:13.171 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-20T11:39:13.183 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-20T11:39:13.208 INFO:teuthology.run:Summary data: description: fs/volumes/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/misc}} duration: 1107.4269869327545 flavor: default mon.a-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 mon.b-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 owner: scheduled_rishabh@teuthology success: true 2024-04-20T11:39:13.209 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-20T11:39:13.320 INFO:teuthology.run:pass